From 4f113d0a5fc8a11be38b006699c1eabb7a4ececa Mon Sep 17 00:00:00 2001 From: Param Saini Date: Mon, 25 Oct 2021 09:58:20 -0700 Subject: [PATCH 001/628] Create README.md --- README.md | 168 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 168 insertions(+) create mode 100644 README.md diff --git a/README.md b/README.md new file mode 100644 index 00000000..80aa3b2e --- /dev/null +++ b/README.md @@ -0,0 +1,168 @@ +# Oracle Database Operator for Kubernetes + +## Make Oracle Database Kubernetes-Native + +As part of Oracle's resolution to make Oracle Database Kubernetes-native (that is, observable and operable by Kubernetes), Oracle is announcing _Oracle Database Operator for Kubernetes_ (`OraOperator`). + +Since Oracle Database 19c, Oracle Database images have been supported in containers (Docker, Podman) for production use and Kubernetes deployment with Helm Charts. This release includes Oracle Database Operator, which is a new open source product that extends the Kubernetes API with custom resources and controllers for automating Oracle Database lifecycle management. + +In this release, `OraOperator` supports the following Oracle Database configurations: + +* Oracle Autonomous Database on shared Oracle Cloud Infrastructure (OCI), also known as ADB-S +* Containerized Single Instance databases (SIDB) deployed in the Oracle Kubernetes Engine (OKE) +* Containerized Sharded databases (SHARDED) deployed in OKE + +Oracle will continue to expand Oracle Database Operator support for additional Oracle Database configurations. + +## Features Summary + +This release of Oracle Database Operator for Kubernetes (the operator) supports the following lifecycle operations: + +* ADB-S: provision, bind, start, stop, terminate (soft/hard), scale (down/up) +* SIDB: provision, clone, patch (in-place/out-of-place), update database initialization parameters, update database configuration (Flashback, archiving), Oracle Enterprise Manager (EM) Express (a basic observability console) +* SHARDED: provision/deploy sharded databases and the shard topology, add a new shard, delete an existing shard + +Upcoming releases will support new configurations, operations and capabilities. + +## Release Status + +**CAUTION:** The current release of `OraOperator` (v0.1.0) is for development and test only. DO NOT USE IN PRODUCTION. + +This release can be deployed on the following platforms: + +* [Oracle Container Engine for Kubernetes (OKE)](https://www.oracle.com/cloud-native/container-engine-kubernetes/) with Kubernetes 1.17 or later +* In an on-premises [Oracle Linux Cloud Native Environment(OLCNE)](https://docs.oracle.com/en/operating-systems/olcne/) 1.3 or later + +In upcoming releases, the operator will be certified against third-party Kubernetes clusters. + +## Prerequisites + +Oracle strongly recommends that you ensure your system meets the following [Prerequisites](./PREREQUISITES.md). + +* ### Install cert-manager + + The operator uses webhooks for validating user input before persisting it in Etcd. Webhooks require TLS certificates that are generated and managed by a certificate manager. + + Install the certificate manager with the following command: + + ```sh + kubectl apply -f https://github.com/jetstack/cert-manager/releases/latest/download/cert-manager.yaml + ``` + +* ### Create Operator Image Pull Secrets + + Sign into [https://container-registry.oracle.com/](https://container-registry.oracle.com/) and accept the license agreement for the Operator image. + + Create an image pull secret for Oracle Container Registry: + + ```sh + kubectl create namespace oracle-database-operator-system + kubectl create secret docker-registry container-registry-secret -n oracle-database-operator-system --docker-server=container-registry.oracle.com --docker-username='' --docker-password='' --docker-email='' + ``` + +## Quick Install of the Operator + + To install the operator in the cluster quickly, you can use a single [oracle-database-operator.yaml](https://github.com/oracle/oracle-database-operator/blob/main/oracle-database-operator.yaml) file. Operator pod replicas are set to a default of 3 for High Availability, which can be scaled up and down. + + Run the following command + + ```sh + kubectl apply -f oracle-database-operator.yaml + ``` + + Ensure that operator pods are up and running + + ```sh + $ kubectl get pods -n oracle-database-operator-system + + NAME READY STATUS RESTARTS AGE + pod/oracle-database-operator-controller-manager-78666fdddb-s4xcm 1/1 Running 0 11d + pod/oracle-database-operator-controller-manager-78666fdddb-5k6n4 1/1 Running 0 11d + pod/oracle-database-operator-controller-manager-78666fdddb-t6bzb 1/1 Running 0 11d + + ``` + +* Check the resources + +You should see that the operator is up and running, along with the shipped controllers. + +For more details, see [Oracle Database Operator Installation Instrunctions](./doc/installation/OPERATOR_INSTALLATION_README.md). + +## Getting Started with the Operator (Quickstart) + +The quickstarts are designed for specific database configurations, including: + +* [Oracle Autonomous Database](./doc/adb/ORACLE_ADB_CONTROLLER_README.md) +* [Oracle Database Single Instance configuration](./doc/sidb/ORACLE_SIDB_CONTROLLER_README.md) +* [Oracle Database configured with Oracle Sharding](./doc/sharding/ORACLE_SHARDING_CONTROLLER_README.md) + +YAML file templates are available under [`/config/samples`](./config/samples/). You can copy and edit these template files to configure them for your use cases. + +## Uninstall the Operator + + To uninstall the operator, the final step consists of deciding whether or not you want to delete the CRDs and APIServices that were introduced to the cluster by the operator. Choose one of the following options: + +* ### Deleting the CRDs and APIServices + + To delete all the CRD instances deployed to cluster by the operator, run the following commands, where is the namespace of the cluster object: + + ```sh + kubectl delete singleinstancedatabase.database.oracle.com --all -n + kubectl delete shardingdatabase.database.oracle.com --all -n + kubectl delete autonomousdatabase.database.oracle.com --all -n + ``` + + After all CRD instances are deleted, it is safe to remove the CRDs, APISerivces and operator deployment. + + ```sh + kubectl delete -f oracle-database-operator.yaml --ignore-not-found=true + ``` + + Note: If the CRD instances are not deleted, and the operator is deleted by using the preceding command, then operator deployment and instance objects (pods,services,PVCs, and so on) are deleted. However, the CRD deletion stops responding, because the CRD instances have finalizers that can only be removed by the operator pod, which is deleted when the APIServices are deleted. + +* ### Retain the CRDs and APIservices + + To delete the operator deployment and retain the CRDs, run the following commands: + + ```sh + kubectl delete deployment.apps/oracle-database-operator-controller-manager -n oracle-database-operator-system + ``` + +## Documentation + +* [Oracle Autonomous Database](https://docs.oracle.com/en-us/iaas/Content/Database/Concepts/adboverview.htm) +* [Oracle Database Single Instance](https://docs.oracle.com/en/database/oracle/oracle-database/) +* [Oracle Database Sharding](https://docs.oracle.com/en/database/oracle/oracle-database/21/shard/index.html) + +## Contributing + +See [Contributing to this Repository](./CONTRIBUTING.md) + +## Support + +You can submit a GitHub issue, or you can also file an [Oracle Support service](https://support.oracle.com/portal/) request, using the product id: 14430. + +## Security + +Secure platforms are an important basis for general system security. Ensure that your deployment is in compliance with common security practices. + +### Managing Sensitive Data +Kubernetes secrets are the usual means for storing credentials or passwords input for access. The operator reads the Secrets programmatically, which limits exposure of sensitive data. However, to protect your sensitive data, Oracle strongly recommends that you set and get sensitive data from Oracle Cloud Infrastructure Vault, or from third-party Vaults. + +The following is an example of a YAML file fragment for specifying Oracle Cloud Infrastructure Vault as the repository for the admin password. + ``` + adminPassword: + ociSecretOCID: ocid1.vaultsecret.oc1... +``` +Examples in this repository where passwords are entered on the command line are for demonstration purposes only. + +### Reporting a Security Issue + +See [Reporting security vulnerabilities](./SECURITY.md) + + + +## License + +Copyright (c) 2021 Oracle and/or its affiliates. +Released under the Universal Permissive License v1.0 as shown at [https://oss.oracle.com/licenses/upl/](https://oss.oracle.com/licenses/upl/) From 1c78279322c9976ee567bc670865e17a7103a8c5 Mon Sep 17 00:00:00 2001 From: psaini Date: Mon, 25 Oct 2021 10:02:38 -0700 Subject: [PATCH 002/628] Added Oracle-Database-Operator files from the release-0.1.0 branch and initialized the repo with code and README.md --- CONTRIBUTING.md | 50 + Dockerfile | 34 + LICENSE.txt | 35 + Makefile | 181 + PREREQUISITES.md | 33 + PROJECT | 41 + SECURITY.md | 34 + THIRD_PARTY_LICENSES.txt | 405 ++ THIRD_PARTY_LICENSES_DOCKER.txt | 5970 +++++++++++++++++ .../v1alpha1/autonomousdatabase_types.go | 224 + apis/database/v1alpha1/groupversion_info.go | 58 + .../v1alpha1/shardingdatabase_types.go | 306 + .../v1alpha1/singleinstancedatabase_types.go | 184 + .../singleinstancedatabase_webhook.go | 195 + .../v1alpha1/zz_generated.deepcopy.go | 848 +++ bundle.Dockerfile | 25 + commons/annotations/annotations.go | 83 + commons/autonomousdatabase/reconciler_util.go | 127 + commons/database/constants.go | 177 + commons/database/utils.go | 619 ++ commons/finalizer/finalizer.go | 122 + commons/oci/database.go | 524 ++ commons/oci/provider.go | 134 + commons/oci/vault.go | 67 + commons/oci/wallet.go | 181 + commons/sharding/catalog.go | 462 ++ commons/sharding/exec.go | 107 + commons/sharding/gsm.go | 489 ++ commons/sharding/provstatus.go | 421 ++ commons/sharding/scommon.go | 1252 ++++ commons/sharding/shard.go | 472 ++ config/certmanager/certificate.yaml | 31 + config/certmanager/kustomization.yaml | 9 + config/certmanager/kustomizeconfig.yaml | 21 + ...tabase.oracle.com_autonomousdatabases.yaml | 185 + ...database.oracle.com_shardingdatabases.yaml | 446 ++ ...se.oracle.com_singleinstancedatabases.yaml | 337 + config/crd/kustomization.yaml | 33 + config/crd/kustomizeconfig.yaml | 21 + .../cainjection_in_autonomousdatabases.yaml | 12 + .../cainjection_in_shardingdatabases.yaml | 12 + ...ainjection_in_singleinstancedatabases.yaml | 12 + .../webhook_in_autonomousdatabases.yaml | 21 + .../patches/webhook_in_shardingdatabases.yaml | 21 + .../webhook_in_singleinstancedatabases.yaml | 21 + config/default/kustomization.yaml | 75 + config/default/manager_auth_proxy_patch.yaml | 30 + config/default/manager_webhook_patch.yaml | 27 + config/default/webhookcainjection_patch.yaml | 20 + config/manager/kustomization.yaml | 12 + config/manager/manager.yaml | 46 + ...tabase-operator.clusterserviceversion.yaml | 48 + config/manifests/kustomization.yaml | 8 + .../rbac/auth_proxy_client_clusterrole.yaml | 11 + config/rbac/auth_proxy_role.yaml | 17 + config/rbac/auth_proxy_role_binding.yaml | 16 + config/rbac/auth_proxy_service.yaml | 18 + .../rbac/autonomousdatabase_editor_role.yaml | 29 + .../rbac/autonomousdatabase_viewer_role.yaml | 25 + config/rbac/kustomization.yaml | 16 + config/rbac/leader_election_role.yaml | 50 + config/rbac/leader_election_role_binding.yaml | 16 + config/rbac/provshard_editor_role.yaml | 29 + config/rbac/provshard_viewer_role.yaml | 25 + config/rbac/role.yaml | 174 + config/rbac/role_binding.yaml | 16 + config/rbac/shardingdatabase_editor_role.yaml | 29 + config/rbac/shardingdatabase_viewer_role.yaml | 25 + .../singleinstancedatabase_editor_role.yaml | 29 + .../singleinstancedatabase_viewer_role.yaml | 25 + config/samples/autonomousdatabase.yaml | 11 + config/samples/autonomousdatabase_bind.yaml | 15 + ...onomousdatabase_change_admin_password.yaml | 20 + config/samples/autonomousdatabase_create.yaml | 26 + .../autonomousdatabase_delete_resource.yaml | 17 + config/samples/autonomousdatabase_rename.yaml | 19 + config/samples/autonomousdatabase_scale.yaml | 21 + ...tonomousdatabase_stop_start_terminate.yaml | 17 + config/samples/autonomousdatabase_wallet.yaml | 23 + config/samples/kustomization.yaml | 12 + .../samples/sharding_v1alpha1_provshard.yaml | 28 + ...sharding_v1alpha1_provshard_clonespec.yaml | 73 + ...harding_v1alpha1_provshard_clonespec1.yaml | 65 + .../sharding_v1alpha1_provshard_orig.yaml | 58 + config/samples/shardingdatabase.yaml | 11 + config/samples/singleinstancedatabase.yaml | 79 + .../samples/singleinstancedatabase_clone.yaml | 41 + .../samples/singleinstancedatabase_patch.yaml | 36 + .../samples/singleinstancedatabase_prov.yaml | 35 + config/scorecard/bases/config.yaml | 25 + config/scorecard/kustomization.yaml | 20 + config/scorecard/patches/basic.config.yaml | 14 + config/scorecard/patches/olm.config.yaml | 54 + config/webhook/kustomization.yaml | 10 + config/webhook/kustomizeconfig.yaml | 30 + config/webhook/manifests.yaml | 59 + config/webhook/service.yaml | 16 + .../database/autonomousdatabase_controller.go | 549 ++ .../database/shardingdatabase_controller.go | 1782 +++++ .../singleinstancedatabase_controller.go | 1762 +++++ controllers/database/suite_test.go | 103 + doc/adb/ADB_PREREQUISITES.md | 100 + doc/adb/ORACLE_ADB_CONTROLLER_README.md | 376 ++ .../OPERATOR_INSTALLATION_README.md | 92 + .../ORACLE_SHARDING_CONTROLLER_README.md | 111 + .../create_kubernetes_secret_for_db_user.md | 25 + .../provisioning/database_connection.md | 44 + doc/sharding/provisioning/debugging.md | 43 + doc/sharding/provisioning/oraclesi.yaml | 99 + .../provisioning/oraclesi_pvc_commented.yaml | 100 + ...y_cloning_db_from_gold_image_across_ads.md | 47 + ...ing_by_cloning_db_gold_image_in_same_ad.md | 43 + ..._persistent_volume_having_db_gold_image.md | 40 + .../provisioning_with_control_on_resources.md | 39 + ...ith_notification_using_oci_notification.md | 79 + .../provisioning_without_db_gold_image.md | 34 + .../scale_in_delete_an_existing_shard.md | 44 + .../provisioning/scale_out_add_shards.md | 31 + doc/sharding/provisioning/shard_prov.yaml | 46 + .../provisioning/shard_prov_clone.yaml | 64 + .../shard_prov_clone_across_ads.yaml | 72 + .../provisioning/shard_prov_delshard.yaml | 52 + .../provisioning/shard_prov_extshard.yaml | 51 + .../provisioning/shard_prov_memory_cpu.yaml | 67 + .../shard_prov_send_notification.yaml | 75 + doc/sidb/ORACLE_SIDB_CONTROLLER_README.md | 264 + doc/sidb/SIDB_PREREQUISITES.md | 14 + go.mod | 16 + go.sum | 741 ++ hack/boilerplate.go.txt | 37 + images/adb/adb-id-1.png | Bin 0 -> 54775 bytes images/adb/adb-id-2.png | Bin 0 -> 31989 bytes images/adb/compartment-1.png | Bin 0 -> 79686 bytes images/adb/compartment-2.png | Bin 0 -> 34377 bytes images/adb/instance-principal-1.png | Bin 0 -> 76459 bytes images/adb/instance-principal-2.png | Bin 0 -> 62643 bytes images/adb/instance-principal-3.png | Bin 0 -> 76392 bytes main.go | 143 + oracle-database-operator.yaml | 1308 ++++ set_ocicredentials.sh | 206 + ...autonomousdatabase_controller_bind_test.go | 298 + ...tonomousdatabase_controller_create_test.go | 232 + test/e2e/behavior/shared_behaviors.go | 482 ++ test/e2e/resource/test_config.yaml | 16 + test/e2e/suite_test.go | 236 + test/e2e/util/oci_config_util.go | 249 + test/e2e/util/oci_db_request.go | 166 + test/e2e/util/oci_vault_request.go | 255 + test/e2e/util/oci_work_request.go | 97 + test/e2e/util/util.go | 138 + 150 files changed, 27561 insertions(+) create mode 100644 CONTRIBUTING.md create mode 100644 Dockerfile create mode 100644 LICENSE.txt create mode 100644 Makefile create mode 100644 PREREQUISITES.md create mode 100644 PROJECT create mode 100644 SECURITY.md create mode 100644 THIRD_PARTY_LICENSES.txt create mode 100644 THIRD_PARTY_LICENSES_DOCKER.txt create mode 100644 apis/database/v1alpha1/autonomousdatabase_types.go create mode 100644 apis/database/v1alpha1/groupversion_info.go create mode 100644 apis/database/v1alpha1/shardingdatabase_types.go create mode 100644 apis/database/v1alpha1/singleinstancedatabase_types.go create mode 100644 apis/database/v1alpha1/singleinstancedatabase_webhook.go create mode 100644 apis/database/v1alpha1/zz_generated.deepcopy.go create mode 100644 bundle.Dockerfile create mode 100644 commons/annotations/annotations.go create mode 100644 commons/autonomousdatabase/reconciler_util.go create mode 100644 commons/database/constants.go create mode 100644 commons/database/utils.go create mode 100644 commons/finalizer/finalizer.go create mode 100644 commons/oci/database.go create mode 100644 commons/oci/provider.go create mode 100644 commons/oci/vault.go create mode 100644 commons/oci/wallet.go create mode 100644 commons/sharding/catalog.go create mode 100644 commons/sharding/exec.go create mode 100644 commons/sharding/gsm.go create mode 100644 commons/sharding/provstatus.go create mode 100644 commons/sharding/scommon.go create mode 100644 commons/sharding/shard.go create mode 100644 config/certmanager/certificate.yaml create mode 100644 config/certmanager/kustomization.yaml create mode 100644 config/certmanager/kustomizeconfig.yaml create mode 100644 config/crd/bases/database.oracle.com_autonomousdatabases.yaml create mode 100644 config/crd/bases/database.oracle.com_shardingdatabases.yaml create mode 100644 config/crd/bases/database.oracle.com_singleinstancedatabases.yaml create mode 100644 config/crd/kustomization.yaml create mode 100644 config/crd/kustomizeconfig.yaml create mode 100644 config/crd/patches/cainjection_in_autonomousdatabases.yaml create mode 100644 config/crd/patches/cainjection_in_shardingdatabases.yaml create mode 100644 config/crd/patches/cainjection_in_singleinstancedatabases.yaml create mode 100644 config/crd/patches/webhook_in_autonomousdatabases.yaml create mode 100644 config/crd/patches/webhook_in_shardingdatabases.yaml create mode 100644 config/crd/patches/webhook_in_singleinstancedatabases.yaml create mode 100644 config/default/kustomization.yaml create mode 100644 config/default/manager_auth_proxy_patch.yaml create mode 100644 config/default/manager_webhook_patch.yaml create mode 100644 config/default/webhookcainjection_patch.yaml create mode 100644 config/manager/kustomization.yaml create mode 100644 config/manager/manager.yaml create mode 100644 config/manifests/bases/oracle-database-operator.clusterserviceversion.yaml create mode 100644 config/manifests/kustomization.yaml create mode 100644 config/rbac/auth_proxy_client_clusterrole.yaml create mode 100644 config/rbac/auth_proxy_role.yaml create mode 100644 config/rbac/auth_proxy_role_binding.yaml create mode 100644 config/rbac/auth_proxy_service.yaml create mode 100644 config/rbac/autonomousdatabase_editor_role.yaml create mode 100644 config/rbac/autonomousdatabase_viewer_role.yaml create mode 100644 config/rbac/kustomization.yaml create mode 100644 config/rbac/leader_election_role.yaml create mode 100644 config/rbac/leader_election_role_binding.yaml create mode 100644 config/rbac/provshard_editor_role.yaml create mode 100644 config/rbac/provshard_viewer_role.yaml create mode 100644 config/rbac/role.yaml create mode 100644 config/rbac/role_binding.yaml create mode 100644 config/rbac/shardingdatabase_editor_role.yaml create mode 100644 config/rbac/shardingdatabase_viewer_role.yaml create mode 100644 config/rbac/singleinstancedatabase_editor_role.yaml create mode 100644 config/rbac/singleinstancedatabase_viewer_role.yaml create mode 100644 config/samples/autonomousdatabase.yaml create mode 100644 config/samples/autonomousdatabase_bind.yaml create mode 100644 config/samples/autonomousdatabase_change_admin_password.yaml create mode 100644 config/samples/autonomousdatabase_create.yaml create mode 100644 config/samples/autonomousdatabase_delete_resource.yaml create mode 100644 config/samples/autonomousdatabase_rename.yaml create mode 100644 config/samples/autonomousdatabase_scale.yaml create mode 100644 config/samples/autonomousdatabase_stop_start_terminate.yaml create mode 100644 config/samples/autonomousdatabase_wallet.yaml create mode 100644 config/samples/kustomization.yaml create mode 100644 config/samples/sharding_v1alpha1_provshard.yaml create mode 100644 config/samples/sharding_v1alpha1_provshard_clonespec.yaml create mode 100644 config/samples/sharding_v1alpha1_provshard_clonespec1.yaml create mode 100644 config/samples/sharding_v1alpha1_provshard_orig.yaml create mode 100644 config/samples/shardingdatabase.yaml create mode 100644 config/samples/singleinstancedatabase.yaml create mode 100644 config/samples/singleinstancedatabase_clone.yaml create mode 100644 config/samples/singleinstancedatabase_patch.yaml create mode 100644 config/samples/singleinstancedatabase_prov.yaml create mode 100644 config/scorecard/bases/config.yaml create mode 100644 config/scorecard/kustomization.yaml create mode 100644 config/scorecard/patches/basic.config.yaml create mode 100644 config/scorecard/patches/olm.config.yaml create mode 100644 config/webhook/kustomization.yaml create mode 100644 config/webhook/kustomizeconfig.yaml create mode 100644 config/webhook/manifests.yaml create mode 100644 config/webhook/service.yaml create mode 100644 controllers/database/autonomousdatabase_controller.go create mode 100644 controllers/database/shardingdatabase_controller.go create mode 100644 controllers/database/singleinstancedatabase_controller.go create mode 100644 controllers/database/suite_test.go create mode 100644 doc/adb/ADB_PREREQUISITES.md create mode 100644 doc/adb/ORACLE_ADB_CONTROLLER_README.md create mode 100644 doc/installation/OPERATOR_INSTALLATION_README.md create mode 100644 doc/sharding/ORACLE_SHARDING_CONTROLLER_README.md create mode 100644 doc/sharding/provisioning/create_kubernetes_secret_for_db_user.md create mode 100644 doc/sharding/provisioning/database_connection.md create mode 100644 doc/sharding/provisioning/debugging.md create mode 100644 doc/sharding/provisioning/oraclesi.yaml create mode 100644 doc/sharding/provisioning/oraclesi_pvc_commented.yaml create mode 100644 doc/sharding/provisioning/provisioning_by_cloning_db_from_gold_image_across_ads.md create mode 100644 doc/sharding/provisioning/provisioning_by_cloning_db_gold_image_in_same_ad.md create mode 100644 doc/sharding/provisioning/provisioning_persistent_volume_having_db_gold_image.md create mode 100644 doc/sharding/provisioning/provisioning_with_control_on_resources.md create mode 100644 doc/sharding/provisioning/provisioning_with_notification_using_oci_notification.md create mode 100644 doc/sharding/provisioning/provisioning_without_db_gold_image.md create mode 100644 doc/sharding/provisioning/scale_in_delete_an_existing_shard.md create mode 100644 doc/sharding/provisioning/scale_out_add_shards.md create mode 100644 doc/sharding/provisioning/shard_prov.yaml create mode 100644 doc/sharding/provisioning/shard_prov_clone.yaml create mode 100644 doc/sharding/provisioning/shard_prov_clone_across_ads.yaml create mode 100644 doc/sharding/provisioning/shard_prov_delshard.yaml create mode 100644 doc/sharding/provisioning/shard_prov_extshard.yaml create mode 100644 doc/sharding/provisioning/shard_prov_memory_cpu.yaml create mode 100644 doc/sharding/provisioning/shard_prov_send_notification.yaml create mode 100644 doc/sidb/ORACLE_SIDB_CONTROLLER_README.md create mode 100644 doc/sidb/SIDB_PREREQUISITES.md create mode 100644 go.mod create mode 100644 go.sum create mode 100644 hack/boilerplate.go.txt create mode 100644 images/adb/adb-id-1.png create mode 100644 images/adb/adb-id-2.png create mode 100644 images/adb/compartment-1.png create mode 100644 images/adb/compartment-2.png create mode 100644 images/adb/instance-principal-1.png create mode 100644 images/adb/instance-principal-2.png create mode 100644 images/adb/instance-principal-3.png create mode 100644 main.go create mode 100644 oracle-database-operator.yaml create mode 100755 set_ocicredentials.sh create mode 100644 test/e2e/autonomousdatabase_controller_bind_test.go create mode 100644 test/e2e/autonomousdatabase_controller_create_test.go create mode 100644 test/e2e/behavior/shared_behaviors.go create mode 100644 test/e2e/resource/test_config.yaml create mode 100644 test/e2e/suite_test.go create mode 100644 test/e2e/util/oci_config_util.go create mode 100644 test/e2e/util/oci_db_request.go create mode 100644 test/e2e/util/oci_vault_request.go create mode 100644 test/e2e/util/oci_work_request.go create mode 100644 test/e2e/util/util.go diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 00000000..722965b9 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,50 @@ +# Contributing to This Repository + +We welcome your contributions! There are multiple ways to contribute. + +## Opening issues + +For bugs or enhancement requests, please file a GitHub issue unless the problem is security-related. When filing a bug, remember that the more specific the bug is, the more likely it is to be fixed. If you think you've found a security +vulnerability, then do not raise a GitHub issue. Instead, follow the instructions in our +[security policy](./SECURITY.md). + +## Contributing code + +We welcome your code contributions. Before submitting code by using a pull request, +you must sign the [Oracle Contributor Agreement][OCA] (OCA), and your commits must include the following line, using the name and e-mail address you used to sign the OCA: + +```text +Signed-off-by: Your Name +``` + +You can add this line automatically to pull requests by committing with `--sign-off` +or `-s`. For example: + +```text +git commit --signoff +``` + +Only pull requests from committers that can be verified as having signed the OCA +can be accepted. + +## Pull request process + +1. Ensure there is an issue created to track and discuss the fix or enhancement that you intend to submit. +1. Fork this repository. +1. Create a branch in your fork to implement the changes. Oracle recommends using + the issue number as part of your branch name. For example: `1234-fixes` +1. Ensure that any documentation is updated with the changes that are required + by your change. +1. Ensure that any samples are updated, if the base image has been changed. +1. Submit the pull request. *Do not leave the pull request blank*. Explain exactly + what your changes are meant to do, and provide simple steps to indicate how to validate + your changes. Ensure that you reference the issue that you created as well. +1. Before the changes are merged, Oracle will assign the pull request to 2 or 3 people for review. + +## Code of conduct + +Follow the [Golden Rule](https://en.wikipedia.org/wiki/Golden_Rule). If you'd +like more specific guidelines, see the [Contributor Covenant Code of Conduct][COC]. + +[OCA]: https://oca.opensource.oracle.com +[COC]: https://www.contributor-covenant.org/version/1/4/code-of-conduct/ diff --git a/Dockerfile b/Dockerfile new file mode 100644 index 00000000..0c181a48 --- /dev/null +++ b/Dockerfile @@ -0,0 +1,34 @@ +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl. +# + +# Build the manager binary +FROM golang:1.16 as builder + +WORKDIR /workspace +# Copy the Go Modules manifests +COPY go.mod go.mod +COPY go.sum go.sum +# cache deps before building and copying source so that we don't need to re-download as much +# and so that source changes don't invalidate our downloaded layer +RUN go mod download + +# Copy the go source +COPY main.go main.go +COPY apis/ apis/ +COPY controllers/ controllers/ +COPY commons/ commons/ +COPY LICENSE.txt LICENSE.txt +COPY THIRD_PARTY_LICENSES_DOCKER.txt THIRD_PARTY_LICENSES_DOCKER.txt + +# Build +RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 GO111MODULE=on go build -a -o manager main.go + +# Use oraclelinux:8-slim as base image to package the manager binary +FROM oraclelinux:8-slim +WORKDIR / +COPY --from=builder /workspace/manager . +RUN useradd -u 1002 nonroot +USER nonroot + +ENTRYPOINT ["/manager"] diff --git a/LICENSE.txt b/LICENSE.txt new file mode 100644 index 00000000..4ac08f59 --- /dev/null +++ b/LICENSE.txt @@ -0,0 +1,35 @@ +Copyright (c) 2021 Oracle and/or its affiliates. + +The Universal Permissive License (UPL), Version 1.0 + +Subject to the condition set forth below, permission is hereby granted to any +person obtaining a copy of this software, associated documentation and/or data +(collectively the "Software"), free of charge and under any and all copyright +rights in the Software, and any and all patent rights owned or freely +licensable by each licensor hereunder covering either (i) the unmodified +Software as contributed to or provided by such licensor, or (ii) the Larger +Works (as defined below), to deal in both + +(a) the Software, and +(b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +one is included with the Software (each a "Larger Work" to which the Software +is contributed by such licensors), + +without restriction, including without limitation the rights to copy, create +derivative works of, display, perform, and distribute the Software and make, +use, sell, offer for sale, import, export, have made, and have sold the +Software and the Larger Work(s), and to sublicense the foregoing rights on +either these or other terms. + +This license is subject to the following condition: +The above copyright notice and either this complete permission notice or at +a minimum a reference to the UPL must be included in all copies or +substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/Makefile b/Makefile new file mode 100644 index 00000000..a0bd6870 --- /dev/null +++ b/Makefile @@ -0,0 +1,181 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +# Current Operator version +VERSION ?= 0.0.1 +# Default bundle image tag +BUNDLE_IMG ?= controller-bundle:$(VERSION) +# Options for 'bundle-build' +ifneq ($(origin CHANNELS), undefined) +BUNDLE_CHANNELS := --channels=$(CHANNELS) +endif +ifneq ($(origin DEFAULT_CHANNEL), undefined) +BUNDLE_DEFAULT_CHANNEL := --default-channel=$(DEFAULT_CHANNEL) +endif +BUNDLE_METADATA_OPTS ?= $(BUNDLE_CHANNELS) $(BUNDLE_DEFAULT_CHANNEL) + +# Image URL to use all building/pushing image targets +IMG ?= controller:latest +# Produce CRDs that work back to Kubernetes 1.11 (no version conversion) +# API version has to be v1 to use defaulting (https://github.com/kubernetes-sigs/controller-tools/issues/478) +CRD_OPTIONS ?= "crd:trivialVersions=true,preserveUnknownFields=false" +# ENVTEST_K8S_VERSION refers to the version of kubebuilder assets to be downloaded by envtest binary. +ENVTEST_K8S_VERSION = 1.21 + +# Get the currently used golang install path (in GOPATH/bin, unless GOBIN is set) +ifeq (,$(shell go env GOBIN)) +GOBIN=$(shell go env GOPATH)/bin +else +GOBIN=$(shell go env GOBIN) +endif + +# Setting SHELL to bash allows bash commands to be executed by recipes. +# This is a requirement for 'setup-envtest.sh' in the test target. +# Options are set to exit when a recipe line exits non-zero or a piped command fails. +SHELL = /usr/bin/env bash -o pipefail +.SHELLFLAGS = -ec + +all: build + +##@ Development + +manifests: controller-gen ## Generate WebhookConfiguration, ClusterRole and CustomResourceDefinition objects. + $(CONTROLLER_GEN) $(CRD_OPTIONS) rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases + +generate: controller-gen ## Generate code containing DeepCopy, DeepCopyInto, and DeepCopyObject method implementations. + $(CONTROLLER_GEN) object:headerFile="hack/boilerplate.go.txt" paths="./..." + +fmt: ## Run go fmt against code. + go fmt ./... + +vet: ## Run go vet against code. + go vet ./... + +TEST ?= ./apis/... ./commons/... ./controllers/... +test: manifests generate fmt vet envtest ## Run unit tests. + KUBEBUILDER_ASSETS="$(shell $(ENVTEST) use $(ENVTEST_K8S_VERSION) -p path)" go test $(TEST) -coverprofile cover.out + +E2ETEST ?= ./test/e2e/ +e2e: manifests generate fmt vet envtest ## Run e2e tests. + KUBEBUILDER_ASSETS="$(shell $(ENVTEST) use $(ENVTEST_K8S_VERSION) -p path)" go test $(E2ETEST) -v -timeout 40m -ginkgo.v -ginkgo.failFast + +##@ Build + +build: generate fmt vet ## Build manager binary. + go build -o bin/manager main.go + +run: manifests generate fmt vet ## Run a controller from your host. + go run ./main.go + +docker-build: test ## Build docker image with the manager. + docker build --no-cache=true --build-arg http_proxy=${HTTP_PROXY} --build-arg https_proxy=${HTTPS_PROXY} . -t ${IMG} + +#docker-build-proxy: test +# docker build --build-arg http_proxy=${http_proxy} --build-arg https_proxy=${https_proxy} build . -t ${IMG} + +docker-push: ## Push docker image with the manager. + docker push ${IMG} + +##@ Deployment + +install: manifests kustomize ## Install CRDs into the K8s cluster specified in ~/.kube/config. + $(KUSTOMIZE) build config/crd | kubectl apply -f - + +uninstall: manifests kustomize ## Uninstall CRDs from the K8s cluster specified in ~/.kube/config. + $(KUSTOMIZE) build config/crd | kubectl delete -f - + +deploy: manifests kustomize ## Deploy controller to the K8s cluster specified in ~/.kube/config. + cd config/manager && $(KUSTOMIZE) edit set image controller=${IMG} + $(KUSTOMIZE) build config/default | kubectl apply -f - + +operator-yaml: manifests kustomize + cd config/manager && $(KUSTOMIZE) edit set image controller=${IMG} + $(KUSTOMIZE) build config/default > $$(basename $$(pwd)).yaml + +undeploy: ## Undeploy controller from the K8s cluster specified in ~/.kube/config. + $(KUSTOMIZE) build config/default | kubectl delete -f - + + +CONTROLLER_GEN = $(shell pwd)/bin/controller-gen +controller-gen: ## Download controller-gen locally if necessary. + $(call go-get-tool,$(CONTROLLER_GEN),sigs.k8s.io/controller-tools/cmd/controller-gen@v0.6.1) + +KUSTOMIZE = $(shell pwd)/bin/kustomize +kustomize: ## Download kustomize locally if necessary. + $(call go-get-tool,$(KUSTOMIZE),sigs.k8s.io/kustomize/kustomize/v3@v3.8.7) + +ENVTEST = $(shell pwd)/bin/setup-envtest +envtest: ## Download envtest-setup locally if necessary. + $(call go-get-tool,$(ENVTEST),sigs.k8s.io/controller-runtime/tools/setup-envtest@latest) + +# go-get-tool will 'go get' any package $2 and install it to $1. +PROJECT_DIR := $(shell dirname $(abspath $(lastword $(MAKEFILE_LIST)))) +define go-get-tool +@[ -f $(1) ] || { \ +set -e ;\ +TMP_DIR=$$(mktemp -d) ;\ +cd $$TMP_DIR ;\ +go mod init tmp ;\ +echo "Downloading $(2)" ;\ +GOBIN=$(PROJECT_DIR)/bin go get $(2) ;\ +rm -rf $$TMP_DIR ;\ +} +endef + +.PHONY: bundle +bundle: manifests kustomize ## Generate bundle manifests and metadata, then validate generated files. + operator-sdk generate kustomize manifests -q + cd config/manager && $(KUSTOMIZE) edit set image controller=$(IMG) + $(KUSTOMIZE) build config/manifests | operator-sdk generate bundle -q --overwrite --version $(VERSION) $(BUNDLE_METADATA_OPTS) + operator-sdk bundle validate ./bundle + +.PHONY: bundle-build +bundle-build: ## Build the bundle image. + docker build -f bundle.Dockerfile -t $(BUNDLE_IMG) . + +.PHONY: bundle-push +bundle-push: ## Push the bundle image. + $(MAKE) docker-push IMG=$(BUNDLE_IMG) + +.PHONY: opm +OPM = ./bin/opm +opm: ## Download opm locally if necessary. +ifeq (,$(wildcard $(OPM))) +ifeq (,$(shell which opm 2>/dev/null)) + @{ \ + set -e ;\ + mkdir -p $(dir $(OPM)) ;\ + OS=$(shell go env GOOS) && ARCH=$(shell go env GOARCH) && \ + curl -sSLo $(OPM) https://github.com/operator-framework/operator-registry/releases/download/v1.15.1/$${OS}-$${ARCH}-opm ;\ + chmod +x $(OPM) ;\ + } +else +OPM = $(shell which opm) +endif +endif + +# A comma-separated list of bundle images (e.g. make catalog-build BUNDLE_IMGS=example.com/operator-bundle:v0.1.0,example.com/operator-bundle:v0.2.0). +# These images MUST exist in a registry and be pull-able. +BUNDLE_IMGS ?= $(BUNDLE_IMG) + +# The image tag given to the resulting catalog image (e.g. make catalog-build CATALOG_IMG=example.com/operator-catalog:v0.2.0). +CATALOG_IMG ?= $(IMAGE_TAG_BASE)-catalog:v$(VERSION) + +# Set CATALOG_BASE_IMG to an existing catalog image tag to add $BUNDLE_IMGS to that image. +ifneq ($(origin CATALOG_BASE_IMG), undefined) +FROM_INDEX_OPT := --from-index $(CATALOG_BASE_IMG) +endif + +# Build a catalog image by adding bundle images to an empty catalog using the operator package manager tool, 'opm'. +# This recipe invokes 'opm' in 'semver' bundle add mode. For more information on add modes, see: +# https://github.com/operator-framework/community-operators/blob/7f1438c/docs/packaging-operator.md#updating-your-existing-operator +.PHONY: catalog-build +catalog-build: opm ## Build a catalog image. + $(OPM) index add --container-tool docker --mode semver --tag $(CATALOG_IMG) --bundles $(BUNDLE_IMGS) $(FROM_INDEX_OPT) + +# Push the catalog image. +.PHONY: catalog-push +catalog-push: ## Push a catalog image. + $(MAKE) docker-push IMG=$(CATALOG_IMG) diff --git a/PREREQUISITES.md b/PREREQUISITES.md new file mode 100644 index 00000000..af82a341 --- /dev/null +++ b/PREREQUISITES.md @@ -0,0 +1,33 @@ +# + +## Prerequisites for Using Oracle Database Operator for Kubernetes + +Oracle Database operator for Kubernetes (OraOperator) manages all Cloud deployments of Oracle Database, including: + +* Oracle Autonomous Database (ADB) +* Containerized Oracle Database Single Instance (SIDB) +* Containerized Sharded Oracle Database (SHARDING) + +### Setting Up a Kubernetes Cluster and Volumes + +#### Setting Up an OKE Cluster on Oracle Cloud Infrastructure (OCI) + +To set up a Kubernetes cluster on Oracle Cloud Infrastructure: + +1. Log in to OCI +1. Create an OKE Cluster +1. Provision persistent storage for data files (NFS or Block) + +Note: You must provision persistent storage if you intend to deploy containerized databases over the OKE cluster. + +### Prerequites for Oracle Autonomous Database (ADB) + +If you intent to use `OraOperator` to handle Oracle Autonomous Database lifecycles, then read [Oracle Autonomous Database prerequisites](./doc/adb/ADB_PREREQUISITES.md) + +### Prerequites for Single Instance Databases (SIDB) + +If you intent to use `OraOperator` to handle Oracle Database Single Instance lifecycles, then read [Single Instance Database Prerequisites](./doc/sidb/SIDB_PREREQUISITES.md) + +### Prerequites for Sharded Databases (SHARDING) + + If you intent to use OraOperator to handle the lifecycle of Oracle Database deployed with Oracle Sharding, then read [Sharded Database Prerequisites](./doc/sharding/ORACLE_SHARDING_CONTROLLER_README.md#prerequsites-for-running-oracle-sharding-database-controller) diff --git a/PROJECT b/PROJECT new file mode 100644 index 00000000..70e34778 --- /dev/null +++ b/PROJECT @@ -0,0 +1,41 @@ +domain: oracle.com +layout: +- go.kubebuilder.io/v2 +multigroup: true +plugins: + go.sdk.operatorframework.io/v2-alpha: {} +projectName: oracle-database-operator +repo: github.com/oracle/oracle-database-operator +resources: +- api: + crdVersion: v1 + namespaced: true + controller: true + domain: oracle.com + group: database + kind: AutonomousDatabase + path: github.com/oracle/oracle-database-operator/apis/database/v1alpha1 + version: v1alpha1 +- api: + crdVersion: v1 + namespaced: true + controller: true + domain: oracle.com + group: database + kind: SingleInstanceDatabase + path: github.com/oracle/oracle-database-operator/apis/database/v1alpha1 + version: v1alpha1 + webhooks: + defaulting: true + validation: true + webhookVersion: v1 +- api: + crdVersion: v1 + namespaced: true + controller: true + domain: oracle.com + group: database + kind: ShardingDatabase + path: github.com/oracle/oracle-database-operator/apis/database/v1alpha1 + version: v1alpha1 +version: "3" diff --git a/SECURITY.md b/SECURITY.md new file mode 100644 index 00000000..5acefa28 --- /dev/null +++ b/SECURITY.md @@ -0,0 +1,34 @@ +# Reporting Security Vulnerabilities + +Oracle values the independent security research community, and believes that +responsible disclosure of security vulnerabilities helps us to ensure the security +and privacy of all of our users. + +Please do NOT raise a GitHub Issue to report a security vulnerability. If you +believe you have found a security vulnerability, then please submit a report to +[secalert_us@oracle.com][1] preferably with a proof of concept. Please review +some additional information on [how to report security vulnerabilities to Oracle][2]. +Oracle encourages anyone who contacts Oracle Security to use email encryption, using +[our encryption key][3]. + +Please do not use other channels, or contact the project maintainers +directly. + +For non-vulnerability related security issues, including ideas for new or improved +security features, you are welcome to post these as GitHub Issues. + +## Security Updates, Alerts and Bulletins + +Oracle issues security updates on a regular cadence. Many of our projects typically include release security fixes in conjunction with the [Oracle Critical Patch Update][3] program. Security updates are released on the +Tuesday closest to the 17th day of January, April, July and October. A pre-release announcement will be published on the Thursday preceding each release. Additional information, including past advisories, is available on our [security alerts][4] +page. + +## Security-Related Information + +Oracle will provide security-related information in our documentation. The information can be a threat model, best practices for secure use, or any known security issues. Please note +that labs and example code are intended to demonstrate a concept. These examples should not be used for production use without ensuring that the code is hardened, and in compliance with common security practices. + +[1]: mailto:secalert_us@oracle.com +[2]: https://www.oracle.com/corporate/security-practices/assurance/vulnerability/reporting.html +[3]: https://www.oracle.com/security-alerts/encryptionkey.html +[4]: https://www.oracle.com/security-alerts/ diff --git a/THIRD_PARTY_LICENSES.txt b/THIRD_PARTY_LICENSES.txt new file mode 100644 index 00000000..792cde69 --- /dev/null +++ b/THIRD_PARTY_LICENSES.txt @@ -0,0 +1,405 @@ +Operator SDK +https://github.com/operator-framework/operator-sdk + +Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + + ------------------------------ + GO lang + https://github.com/golang + + Copyright (c) 2009 The Go Authors. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are +met: + + * Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above +copyright notice, this list of conditions and the following disclaimer +in the documentation and/or other materials provided with the +distribution. + * Neither the name of Google Inc. nor the names of its +contributors may be used to endorse or promote products derived from +this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +-------------------------- +Logr +https://pkg.go.dev/github.com/go-logr/logr +Apache 2.0 License + +------------------------- +OCI Go SDK +github.com/oracle/oci-go-sdk/v43 + +Dual-License: UPL + Apache 2.0 + +Copyright (c) 2016, 2018, 2020, Oracle and/or its affiliates. +This software is dual-licensed to you under the Universal Permissive License (UPL) 1.0 +as shown at https://oss.oracle.com/licenses/upl +or Apache License 2.0 as shown at http://www.apache.org/licenses/LICENSE-2.0. +You may choose either license. + +Copyright (c) 2019, 2020 Oracle and/or its affiliates. + +The Universal Permissive License (UPL), Version 1.0 + + Subject to the condition set forth below, permission is hereby granted to any + person obtaining a copy of this software, associated documentation and/or data + (collectively the "Software"), free of charge and under any and all copyright + rights in the Software, and any and all patent rights owned or freely + licensable by each licensor hereunder covering either (i) the unmodified + Software as contributed to or provided by such licensor, or (ii) the Larger + Works (as defined below), to deal in both + + (a) the Software, and + (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if + one is included with the Software (each a "Larger Work" to which the Software + is contributed by such licensors), + + without restriction, including without limitation the rights to copy, create + derivative works of, display, perform, and distribute the Software and make, + use, sell, offer for sale, import, export, have made, and have sold the + Software and the Larger Work(s), and to sublicense the foregoing rights on + either these or other terms. + + This license is subject to the following condition: + The above copyright notice and either this complete permission notice or at + a minimum a reference to the UPL must be included in all copies or + substantial portions of the Software. + + THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + SOFTWARE. +------------------------- +Kubernetes api +https://pkg.go.dev/k8s.io/api +Apache 2.0 + +---------------------------------- +Kubernetes apimachinery +https://pkg.go.dev/k8s.io/apimachinery +Apache 2.0 + +----------------------------------- +Kubernetes client-go +https://pkg.go.dev/k8s.io/client-go +Apache 2.0 + +------------------------------------- +Kubernetes controller-runtime project +https://pkg.go.dev/sigs.k8s.io/controller-runtime +Apache 2.0 + +------------------------------------ +YAML support for the Go language +https://pkg.go.dev/gopkg.in/yaml.v2 +Apache 2.0 + +------------------------------------ +YAML marshaling and unmarshaling support for Go +https://pkg.go.dev/sigs.k8s.io/yaml +DUal license: BSD-3-Clause, MIT + +The MIT License (MIT) + +Copyright (c) 2014 Sam Ghods + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. + + +Copyright (c) 2012 The Go Authors. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are +met: + + * Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above +copyright notice, this list of conditions and the following disclaimer +in the documentation and/or other materials provided with the +distribution. + * Neither the name of Google Inc. nor the names of its +contributors may be used to endorse or promote products derived from +this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +------------------------ +Ginkgo +github.com/onsi/ginkgo +MIT License +Copyright (c) 2013-2014 Onsi Fakhouri + +Permission is hereby granted, free of charge, to any person obtaining +a copy of this software and associated documentation files (the +"Software"), to deal in the Software without restriction, including +without limitation the rights to use, copy, modify, merge, publish, +distribute, sublicense, and/or sell copies of the Software, and to +permit persons to whom the Software is furnished to do so, subject to +the following conditions: + +The above copyright notice and this permission notice shall be +included in all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, +EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF +MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE +LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION +WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +--------------------------- +Gomega +github.com/onsi/gomega +MIT License +Copyright (c) 2013-2014 Onsi Fakhouri + +------------------------------------ +Operator Lifecycle Manager (OLM) +github.com/operator-framework/operator-lifecycle-manager +Apache 2.0 + diff --git a/THIRD_PARTY_LICENSES_DOCKER.txt b/THIRD_PARTY_LICENSES_DOCKER.txt new file mode 100644 index 00000000..55a05d95 --- /dev/null +++ b/THIRD_PARTY_LICENSES_DOCKER.txt @@ -0,0 +1,5970 @@ +Operator SDK +https://github.com/operator-framework/operator-sdk + +Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + + 4th party dependencies + --------------------------------------------------------------------------------------- + +github.com/blang/semver (MIT) + +The MIT License + +Copyright (c) 2014 Benedikt Lang + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. + +======== +github.com/fatih/structtag (Fatih Arslan) + +Copyright (c) 2017, Fatih Arslan + + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + +* Redistributions of source code must retain the above copyright notice, this + list of conditions and the following disclaimer. + +* Redistributions in binary form must reproduce the above copyright notice, + this list of conditions and the following disclaimer in the documentation + and/or other materials provided with the distribution. + +* Neither the name of structtag nor the names of its + contributors may be used to endorse or promote products derived from + this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR +SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER +CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +This software includes some portions from Go. Go is used under the terms of the +BSD like license. + +Copyright (c) 2012 The Go Authors. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are +met: + + * Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above +copyright notice, this list of conditions and the following disclaimer +in the documentation and/or other materials provided with the +distribution. + * Neither the name of Google Inc. nor the names of its +contributors may be used to endorse or promote products derived from +this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +The Go gopher was designed by Renee French. http://reneefrench.blogspot.com/ The design is licensed under the Creative Commons 3.0 Attributions license. Read this article for more details: https://blog.golang.org/gopher + +===== + +github.com/go-logr/logr (Apache 2.0) + +====== +github.com/iancoleman/strcase (MIT) + +The MIT License (MIT) + +Copyright (c) 2015 Ian Coleman +Copyright (c) 2018 Ma_124, + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, Subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or Substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. +===== +github.com/kr/text (MIT) + +Copyright 2012 Keith Rarick + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in +all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +THE SOFTWARE. + +======= +github.com/markbates/inflect (MIT) + +Copyright (c) 2011 Chris Farmiloe + +Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +======= +github.com/maxbrunsfeld/counterfeiter (MIT) + +The MIT License (MIT) + +Copyright (c) 2014 maxbrunsfeld + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. +===== +github.com/onsi/ginkgo (MIT) + +Copyright (c) 2013-2014 Onsi Fakhouri + +Permission is hereby granted, free of charge, to any person obtaining +a copy of this software and associated documentation files (the +"Software"), to deal in the Software without restriction, including +without limitation the rights to use, copy, modify, merge, publish, +distribute, sublicense, and/or sell copies of the Software, and to +permit persons to whom the Software is furnished to do so, subject to +the following conditions: + +The above copyright notice and this permission notice shall be +included in all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, +EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF +MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE +LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION +WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +====== +github.com/onsi/gomega (MIT) +Same License and Copyright as github.com/onsi/ginkgo (above) + +====== +github.com/operator-framework/api (Apache 2.0) +======= +github.com/operator-framework/java-operator-plugins (Apache 2.0) +====== +github.com/operator-framework/operator-lib (Apache 2.0) + +======= +github.com/operator-framework/operator-registry (Apache 2.0) + +======= +github.com/prometheus/client_golang (Apache 2.0) + +====== +github.com/prometheus/client_model (Apache 2.0) + +====== +github.com/sergi/go-diff (The go-diff Authors) +Copyright (c) 2012-2016 The go-diff Authors. + +Permission is hereby granted, free of charge, to any person obtaining a +copy of this software and associated documentation files (the "Software"), +to deal in the Software without restriction, including without limitation +the rights to use, copy, modify, merge, publish, distribute, sublicense, +and/or sell copies of the Software, and to permit persons to whom the +Software is furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included +in all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS +OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER +DEALINGS IN THE SOFTWARE. + +====== +github.com/sirupsen/logrus (MIT) + +Copyright (c) 2014 Simon Eskildsen + +Permission is hereby granted, free of charge, to any person obtaining +a copy of this software and associated documentation files (the +"Software"), to deal in the Software without restriction, including +without limitation the rights to use, copy, modify, merge, publish, +distribute, sublicense, and/or sell copies of the Software, and to +permit persons to whom the Software is furnished to do so, subject to +the following conditions: + +The above copyright notice and this permission notice shall be +included in all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, +EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF +MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE +LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION +WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +======= +github.com/spf13/afero (Apache 2.0) + +====== +github.com/spf13/cobra (Apache 2.0) + +====== +github.com/spf13/pflag (BSD-3-Clause) +Copyright (c) , + +Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: + +Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. +Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. +All advertising materials mentioning features or use of this software must display the following acknowledgement: This product includes software developed by the . +Neither the name of the nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. +THIS SOFTWARE IS PROVIDED BY AS IS AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +===== +github.com/spf13/viper (MIT) + +The MIT License (MIT) + +Copyright (c) 2014 Steve Francia + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. + +==== +github.com/stretchr/testify (MIT) + +MIT License + +Copyright (c) 2012-2020 Mat Ryer, Tyler Bunnell and contributors. + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. +====== +github.com/thoas/go-funk (MIT) + +MIT License + +Copyright (c) 2016 Florent Messa + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. + +======= +golang.org/x/lint (BSD-3-Clause) + +======= +golang.org/x/mod (BSD-3-Clause) +======= +golang.org/x/sys (BSD-3-Clause) +======== +golang.org/x/tools (BSD-3-Clause) + +======== +gomodules.xyz/jsonpatch/v3 (Apache 2.0) + +========= +helm.sh/helm/v3 (Apache 2.0) + +======== +k8s.io/api (Apache 2.0) + +========= +k8s.io/apiextensions-apiserver (Apache 2.0) + +======== +k8s.io/apimachinery (Apache 2.0) + +========== +k8s.io/cli-runtime (Apache 2.0) + +========== +k8s.io/client-go (Apache 2.0) + +=========== +k8s.io/kubectl (Apache 2.0) + +========== +rsc.io/letsencrypt (Apache 2.0) + +========== +sigs.k8s.io/controller-runtime (Apache 2.0) + +========== +sigs.k8s.io/controller-tools (Apache 2.0) + +=========== +sigs.k8s.io/kubebuilder/v3 (Apache 2.0) + +============ +sigs.k8s.io/yaml (MIT) + +The MIT License (MIT) + +Copyright (c) 2014 Sam Ghods + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. + + +Copyright (c) 2012 The Go Authors. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are +met: + + * Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above +copyright notice, this list of conditions and the following disclaimer +in the documentation and/or other materials provided with the +distribution. + * Neither the name of Google Inc. nor the names of its +contributors may be used to endorse or promote products derived from +this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +==================================================================================================== + GO lang + https://github.com/golang + + Copyright (c) 2009 The Go Authors. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are +met: + + * Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above +copyright notice, this list of conditions and the following disclaimer +in the documentation and/or other materials provided with the +distribution. + * Neither the name of Google Inc. nor the names of its +contributors may be used to endorse or promote products derived from +this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + + +4th party dependencies +--------------------------------------------------------------------------------------- +golang.org/x/crypto +-------- Copyrights + +Copyright (c) 2009 The Go Authors. +// Copyright 2015 The Go Authors. +// Copyright 2016 The Go Authors. +// Copyright 2017 The Go Authors. +// Copyright 2018 The Go Authors. +// Copyright 2019 The Go Authors. +// Copyright 2011 The Go Authors. +// Copyright 2010 The Go Authors. +// Copyright 2012 The Go Authors. +// Copyright 2013 The Go Authors. +// Copyright 2014 The Go Authors. +// Copyright 2020 The Go Authors. +// Copyright 2009 The Go Authors. +-------- Patents +Additional IP Rights Grant (Patents) + +"This implementation" means the copyrightable works distributed by +Google as part of the Go project. + +Google hereby grants to You a perpetual, worldwide, non-exclusive, +no-charge, royalty-free, irrevocable (except as stated in this section) +patent license to make, have made, use, offer to sell, sell, import, +transfer and otherwise run, modify and propagate the contents of this +implementation of Go, where such license applies only to those patent +claims, both currently owned or controlled by Google and acquired in +the future, licensable by Google that are necessarily infringed by this +implementation of Go. This grant does not include claims that would be +infringed only as a consequence of further modification of this +implementation. If you or your agent or exclusive licensee institute or +order or agree to the institution of patent litigation against any +entity (including a cross-claim or counterclaim in a lawsuit) alleging +that this implementation of Go or any code incorporated within this +implementation of Go constitutes direct or contributory patent +infringement, or inducement of patent infringement, then any patent +rights granted to you under this License for this implementation of Go +shall terminate as of the date such litigation is filed. + +-------- Dependency +golang.org/x/net +-------- Copyrights +Copyright (c) 2009 The Go Authors. +// Copyright 2016 The Go Authors. +// Copyright 2017 The Go Authors. +// Copyright 2014 The Go Authors. +// Copyright 2015 The Go Authors. +// Copyright 2010 The Go Authors. +// Copyright 2009 The Go Authors. +// Copyright 2012 The Go Authors. +// Copyright 2013 The Go Authors. +// Copyright 2011 The Go Authors. +Copyright (C) 2009 Apple Inc. +// Copyright 2018 The Go Authors. +// Copyright 2019 The Go Authors. + +-------- Patents +Additional IP Rights Grant (Patents) +"This implementation" means the copyrightable works distributed by +Google as part of the Go project. + +Google hereby grants to You a perpetual, worldwide, non-exclusive, +no-charge, royalty-free, irrevocable (except as stated in this section) +patent license to make, have made, use, offer to sell, sell, import, +transfer and otherwise run, modify and propagate the contents of this +implementation of Go, where such license applies only to those patent +claims, both currently owned or controlled by Google and acquired in +the future, licensable by Google that are necessarily infringed by this +implementation of Go. This grant does not include claims that would be +infringed only as a consequence of further modification of this +implementation. If you or your agent or exclusive licensee institute or +order or agree to the institution of patent litigation against any +entity (including a cross-claim or counterclaim in a lawsuit) alleging +that this implementation of Go or any code incorporated within this +implementation of Go constitutes direct or contributory patent +infringement, or inducement of patent infringement, then any patent +rights granted to you under this License for this implementation of Go +shall terminate as of the date such litigation is filed. + +-------- Dependency +golang.org/x/sys +-------- Copyrights +Copyright (c) 2009 The Go Authors. +// Copyright 2019 The Go Authors. +// Copyright 2018 The Go Authors. +// Copyright 2020 The Go Authors. +// Copyright 2012 The Go Authors. +// Copyright 2011 The Go Authors. +// Copyright 2015 The Go Authors. +// Copyright 2009 The Go Authors. +// Copyright 2013 The Go Authors. +// Copyright 2016 The Go Authors. +// Copyright 2017 The Go Authors. +// Copyright 2010 The Go Authors. +// Copyright 2014 The Go Authors. +// Copyright 2009,2010 The Go Authors. +// Copyright 2017 The Go Authors. All right reserved. + +-------- Patents + +Additional IP Rights Grant (Patents) + +"This implementation" means the copyrightable works distributed by +Google as part of the Go project. + +Google hereby grants to You a perpetual, worldwide, non-exclusive, +no-charge, royalty-free, irrevocable (except as stated in this section) +patent license to make, have made, use, offer to sell, sell, import, +transfer and otherwise run, modify and propagate the contents of this +implementation of Go, where such license applies only to those patent +claims, both currently owned or controlled by Google and acquired in +the future, licensable by Google that are necessarily infringed by this +implementation of Go. This grant does not include claims that would be +infringed only as a consequence of further modification of this +implementation. If you or your agent or exclusive licensee institute or +order or agree to the institution of patent litigation against any +entity (including a cross-claim or counterclaim in a lawsuit) alleging +that this implementation of Go or any code incorporated within this +implementation of Go constitutes direct or contributory patent +infringement, or inducement of patent infringement, then any patent +rights granted to you under this License for this implementation of Go +shall terminate as of the date such litigation is filed. + +-------- Dependency +golang.org/x/text +-------- Copyrights +Copyright (c) 2009 The Go Authors. +// Copyright 2014 The Go Authors. +// Copyright 2016 The Go Authors. +// Copyright 2015 The Go Authors. +// Copyright 2017 The Go Authors. +// Copyright 2012 The Go Authors. +// Copyright 2013 The Go Authors. +# Copyright 2012 The Go Authors. +// Copyright 2018 The Go Authors. +// Copyright 2009 The Go Authors. +// Copyright 2019 The Go Authors. +// Copyright 2011 The Go Authors. + +-------- Patents +Additional IP Rights Grant (Patents) + +"This implementation" means the copyrightable works distributed by +Google as part of the Go project. + +Google hereby grants to You a perpetual, worldwide, non-exclusive, +no-charge, royalty-free, irrevocable (except as stated in this section) +patent license to make, have made, use, offer to sell, sell, import, +transfer and otherwise run, modify and propagate the contents of this +implementation of Go, where such license applies only to those patent +claims, both currently owned or controlled by Google and acquired in +the future, licensable by Google that are necessarily infringed by this +implementation of Go. This grant does not include claims that would be +infringed only as a consequence of further modification of this +implementation. If you or your agent or exclusive licensee institute or +order or agree to the institution of patent litigation against any +entity (including a cross-claim or counterclaim in a lawsuit) alleging +that this implementation of Go or any code incorporated within this +implementation of Go constitutes direct or contributory patent +infringement, or inducement of patent infringement, then any patent +rights granted to you under this License for this implementation of Go +shall terminate as of the date such litigation is filed. + +-------- Dependency +golang.org/x/tools +-------- Copyrights +Copyright (c) 2009 The Go Authors. +// Copyright 2014 The Go Authors. +// Copyright 2009 The Go Authors. +// Copyright 2013 The Go Authors. +// Copyright 2018 The Go Authors. +// Copyright 2015 The Go Authors. +// Copyright 2013 The Go Authors. +Copyright (c) 2017 The Go Authors. +// Copyright 2017 The Go Authors. +// Copyright 2011 The Go Authors. +// Copyright 2016 The Go Authors. +// Copyright 2010 The Go Authors. +// Copyright 2009 The Go Authors. +// Copyright 2012 The Go Authors. + Copyright © 1994-1999 Lucent Technologies Inc. + Portions Copyright © 1995-1997 C H Forsyth (forsyth@terzarima.net) + Portions Copyright © 1997-1999 Vita Nuova Limited + Portions Copyright © 2000-2007 Vita Nuova Holdings Limited (www.vitanuova.com) + Portions Copyright © 2004,2006 Bruce Ellis + Portions Copyright © 2005-2007 C H Forsyth (forsyth@terzarima.net) +Portions Copyright © 2009 The Go Authors. +// Copyright 2012 The Go Authors. +# Copyright 2017 The Go Authors. +// Copyright 2014 The Go Authors. +Copyright (c) 2013-2016 Guy Bedford, Luke Hoban, Addy Osmani +Copyright (c) 2015 The Polymer Authors. + +-------- Patents +Additional IP Rights Grant (Patents) +"This implementation" means the copyrightable works distributed by +Google as part of the Go project. +Google hereby grants to You a perpetual, worldwide, non-exclusive, +no-charge, royalty-free, irrevocable (except as stated in this section) +patent license to make, have made, use, offer to sell, sell, import, +transfer and otherwise run, modify and propagate the contents of this +implementation of Go, where such license applies only to those patent +claims, both currently owned or controlled by Google and acquired in +the future, licensable by Google that are necessarily infringed by this +implementation of Go. This grant does not include claims that would be +infringed only as a consequence of further modification of this +implementation. If you or your agent or exclusive licensee institute or +order or agree to the institution of patent litigation against any +entity (including a cross-claim or counterclaim in a lawsuit) alleging +that this implementation of Go or any code incorporated within this +implementation of Go constitutes direct or contributory patent +infringement, or inducement of patent infringement, then any patent +rights granted to you under this License for this implementation of Go +shall terminate as of the date such litigation is filed. + +===== Dependencies Summary + +golang.org/x/crypto +golang.org/x/net +golang.org/x/sys +golang.org/x/text +golang.org/x/tools + +======== License used by Dependencies +SPDX:BSD-3-Clause--modified-by-Google +Redistribution and use in source and binary forms, with +or without modification, are permitted provided that the following conditions +are met: + + * Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above +copyright notice, this list of conditions and the following disclaimer +in the documentation and/or other materials provided with the +distribution. + * Neither the name of Google Inc. nor the names of its +contributors may be used to endorse or promote products derived from +this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +==================================================================================================== +Logr +https://pkg.go.dev/github.com/go-logr/logr +Apache 2.0 License + +4th party dependencies +--------------------------------------------------------------------------------------- +cloud.google.com/go +Apache 2.0 +=================================================================================================== +OCI Go SDK +github.com/oracle/oci-go-sdk/v43 + +Dual-License: UPL + Apache 2.0 + +Copyright (c) 2016, 2018, 2020, Oracle and/or its affiliates. +This software is dual-licensed to you under the Universal Permissive License (UPL) 1.0 +as shown at https://oss.oracle.com/licenses/upl +or Apache License 2.0 as shown at http://www.apache.org/licenses/LICENSE-2.0. +You may choose either license. + +Copyright (c) 2019, 2020 Oracle and/or its affiliates. + +The Universal Permissive License (UPL), Version 1.0 + + Subject to the condition set forth below, permission is hereby granted to any + person obtaining a copy of this software, associated documentation and/or data + (collectively the "Software"), free of charge and under any and all copyright + rights in the Software, and any and all patent rights owned or freely + licensable by each licensor hereunder covering either (i) the unmodified + Software as contributed to or provided by such licensor, or (ii) the Larger + Works (as defined below), to deal in both + + (a) the Software, and + (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if + one is included with the Software (each a "Larger Work" to which the Software + is contributed by such licensors), + + without restriction, including without limitation the rights to copy, create + derivative works of, display, perform, and distribute the Software and make, + use, sell, offer for sale, import, export, have made, and have sold the + Software and the Larger Work(s), and to sublicense the foregoing rights on + either these or other terms. + + This license is subject to the following condition: + The above copyright notice and either this complete permission notice or at + a minimum a reference to the UPL must be included in all copies or + substantial portions of the Software. + + THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + SOFTWARE. + +4th party dependencies +--------------------------------------------------------------------------------------- +testify +* Copyright © 2012-2020 Mat Ryer, Tyler Bunnell and contributors. +* License: MIT License +* Source code: https://github.com/stretchr/testify +* Project home: https://github.com/stretchr/testify + +------------------------------ MIT License ----------------------------- + +Copyright (c) 2012-2020 Mat Ryer, Tyler Bunnell and contributors. + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. + +===== Fith-party dependencies === + +github.com/davecgh/go-spew v1.1.0 + +ISC License + +Copyright (c) 2012-2016 Dave Collins + +Permission to use, copy, modify, and/or distribute this software for any +purpose with or without fee is hereby granted, provided that the above +copyright notice and this permission notice appear in all copies. + +THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES +WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF +MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR +ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES +WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN +ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF +OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. +========================================= +github.com/pmezard/go-difflib v1.0.0 + +Copyright (c) 2013, Patrick Mezard + + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are +met: + + Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. + Redistributions in binary form must reproduce the above copyright +notice, this list of conditions and the following disclaimer in the +documentation and/or other materials provided with the distribution. + The names of its contributors may not be used to endorse or promote +products derived from this software without specific prior written +permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS +IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED +TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A +PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED +TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR +PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF +LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING +NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS +SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +=================================== +github.com/stretchr/objx v0.1.0 + +The MIT License + +Copyright (c) 2014 Stretchr, Inc. +Copyright (c) 2017-2018 objx contributors + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + +============ +gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c + +This project is covered by two different licenses: MIT and Apache. + +#### MIT License #### + +The following files were ported to Go from C files of libyaml, and thus +are still covered by their original MIT license, with the additional +copyright staring in 2011 when the project was ported over: + + apic.go emitterc.go parserc.go readerc.go scannerc.go + writerc.go yamlh.go yamlprivateh.go + +Copyright (c) 2006-2010 Kirill Simonov +Copyright (c) 2006-2011 Kirill Simonov + +Permission is hereby granted, free of charge, to any person obtaining a copy of +this software and associated documentation files (the "Software"), to deal in +the Software without restriction, including without limitation the rights to +use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies +of the Software, and to permit persons to whom the Software is furnished to do +so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. + +### Apache License ### + +All the remaining project files are covered by the Apache license: + +Copyright (c) 2011-2019 Canonical Ltd + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +Copyright notice +------------------- +Copyright 2011-2016 Canonical Ltd. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +========= +NOTICE.txt. + +Copyright (c) 2016, 2018, 20 + +=================================================================================================== +Kubernetes api +https://pkg.go.dev/k8s.io/api +Apache 2.0 + +4th party dependencies +--------------------------------------------------------------------------------------- +----------------------- Dependencies Grouped by License ------------ +-------- Dependency +k8s.io/apimachinery +-------- Copyrights +Copyright 2017 The Kubernetes Authors. +Copyright 2019 The Kubernetes Authors. +Copyright 2014 The Kubernetes Authors. +Copyright 2020 The Kubernetes Authors. +Copyright 2015 The Kubernetes Authors. +Copyright 2016 The Kubernetes Authors. +Copyright 2018 The Kubernetes Authors. +Copyright 2013 The Go Authors. +Copyright 2009 The Go Authors. + +-------- Dependency +github.com/go-logr/logr +-------- Copyrights +Copyright 2019 The logr Authors. + +-------- Dependency +github.com/google/gofuzz +-------- Copyrights +Copyright 2014 Google Inc. + +-------- Dependency +github.com/modern-go/concurrent +-------- Copyrights + +-------- Dependency +github.com/modern-go/reflect2 +-------- Copyrights + +-------- Dependency +gopkg.in/yaml.v2 +-------- Copyrights +Copyright (c) 2006 Kirill Simonov +Copyright 2011-2016 Canonical Ltd. +-------- Notices +Copyright 2011-2016 Canonical Ltd. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + + +-------- Dependency +k8s.io/klog/v2 +-------- Copyrights +Copyright 2013 Google Inc. + +-------- Dependency +sigs.k8s.io/structured-merge-diff/v4 +-------- Copyrights +Copyright 2018 The Kubernetes Authors. +Copyright 2019 The Kubernetes Authors. +Copyright 2020 The Kubernetes Authors. + +-------- Dependencies Summary +k8s.io/apimachinery +github.com/go-logr/logr +github.com/google/gofuzz +github.com/modern-go/concurrent +github.com/modern-go/reflect2 +gopkg.in/yaml.v2 +k8s.io/klog/v2 +sigs.k8s.io/structured-merge-diff/v4 + +-------- License used by Dependencies +SPDX:Apache-2.0 + + +---- Dependencies Grouped by License ----- +-------- Dependency +github.com/gogo/protobuf +-------- Copyrights +Copyright (c) 2013, The GoGo Authors. +Copyright 2010 The Go Authors. +Copyright 2010 The Go Authors. +Copyright (c) 2015, The GoGo Authors. +Copyright 2016 The Go Authors. +Copyright 2015 The Go Authors. +Copyright 2011 The Go Authors. +Copyright (c) 2018, The GoGo Authors. +Copyright 2018 The Go Authors. +Copyright 2017 The Go Authors. +Copyright (c) 2016, The GoGo Authors. +Copyright 2014 The Go Authors. +Copyright 2012 The Go Authors. +Copyright 2013 The Go Authors. +Copyright (c) 2019, The GoGo Authors. +Copyright (c) 2017, The GoGo Authors. +Copyright (c) 2015, The GoGo Authors. rights reserved. + +-------- Dependency +golang.org/x/net +-------- Copyrights +Copyright (c) 2009 The Go Authors. +Copyright 2016 The Go Authors. +Copyright 2017 The Go Authors. +Copyright 2014 The Go Authors. +Copyright 2015 The Go Authors. +Copyright 2010 The Go Authors. +Copyright 2009 The Go Authors. +Copyright 2012 The Go Authors. +Copyright 2013 The Go Authors. +Copyright 2011 The Go Authors. +Copyright (C) 2009 Apple Inc. +Copyright 2018 The Go Authors. +Copyright 2019 The Go Authors. +-------- Patents +Additional IP Rights Grant (Patents) + +"This implementation" means the copyrightable works distributed by +Google as part of the Go project. + +Google hereby grants to You a perpetual, worldwide, non-exclusive, +no-charge, royalty-free, irrevocable (except as stated in this section) +patent license to make, have made, use, offer to sell, sell, import, +transfer and otherwise run, modify and propagate the contents of this +implementation of Go, where such license applies only to those patent +claims, both currently owned or controlled by Google and acquired in +the future, licensable by Google that are necessarily infringed by this +implementation of Go. This grant does not include claims that would be +infringed only as a consequence of further modification of this +implementation. If you or your agent or exclusive licensee institute or +order or agree to the institution of patent litigation against any +entity (including a cross-claim or counterclaim in a lawsuit) alleging +that this implementation of Go or any code incorporated within this +implementation of Go constitutes direct or contributory patent +infringement, or inducement of patent infringement, then any patent +rights granted to you under this License for this implementation of Go +shall terminate as of the date such litigation is filed. + + +-------- Dependency +golang.org/x/text +-------- Copyrights +Copyright (c) 2009 The Go Authors. +Copyright 2014 The Go Authors. +Copyright 2016 The Go Authors. +Copyright 2015 The Go Authors. +Copyright 2017 The Go Authors. +Copyright 2012 The Go Authors. +Copyright 2013 The Go Authors. +Copyright 2018 The Go Authors. +Copyright 2009 The Go Authors. +Copyright 2019 The Go Authors. +Copyright 2011 The Go Authors. +-------- Patents +Additional IP Rights Grant (Patents) + +"This implementation" means the copyrightable works distributed by +Google as part of the Go project. + +Google hereby grants to You a perpetual, worldwide, non-exclusive, +no-charge, royalty-free, irrevocable (except as stated in this section) +patent license to make, have made, use, offer to sell, sell, import, +transfer and otherwise run, modify and propagate the contents of this +implementation of Go, where such license applies only to those patent +claims, both currently owned or controlled by Google and acquired in +the future, licensable by Google that are necessarily infringed by this +implementation of Go. This grant does not include claims that would be +infringed only as a consequence of further modification of this +implementation. If you or your agent or exclusive licensee institute or +order or agree to the institution of patent litigation against any +entity (including a cross-claim or counterclaim in a lawsuit) alleging +that this implementation of Go or any code incorporated within this +implementation of Go constitutes direct or contributory patent +infringement, or inducement of patent infringement, then any patent +rights granted to you under this License for this implementation of Go +shall terminate as of the date such litigation is filed. + + +-------- Dependency +gopkg.in/inf.v0 +-------- Copyrights +Copyright (c) 2012 Péter Surányi. Portions Copyright (c) 2009 The Go + +-------- Dependencies Summary +github.com/gogo/protobuf +golang.org/x/net +golang.org/x/text +gopkg.in/inf.v0 + +-------- License used by Dependencies +SPDX:BSD-3-Clause--modified-by-Google +Redistribution and use in source and binary forms, with +or without modification, are permitted provided that the following conditions +are met: + + * Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above +copyright notice, this list of conditions and the following disclaimer +in the documentation and/or other materials provided with the +distribution. + * Neither the name of Google Inc. nor the names of its +contributors may be used to endorse or promote products derived from +this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +---------- Dependencies Grouped by License ------------ +-------- Dependency +github.com/json-iterator/go +-------- Copyrights +Copyright (c) 2016 json-iterator + +-------- Dependencies Summary +github.com/json-iterator/go + +-------- License used by Dependencies +SPDX:MIT +Permission is hereby granted, free of charge, to any person +obtaining a copy of this software and associated documentation files +(the "Software"), to deal in the Software without restriction, including without +limitation the rights to use, copy, modify, merge, publish, distribute, +sublicense, and/or sell copies of the Software, and to permit persons to whom +the Software is furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS +FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR +COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER +IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN +CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +=================================================================================================== +Kubernetes apimachinery +https://pkg.go.dev/k8s.io/apimachinery +Apache 2.0 + +4th party dependencies +--------------------------------------------------------------------------------------- +-------- Dependency +github.com/elazarl/goproxy +-------- Copyrights +Copyright (c) 2012 Elazar Leibovich. +// // Copyright 2012 The Go Authors. +// Copyright 2011 The Go Authors. + +======== Dependencies Summary +github.com/elazarl/goproxy + +======== License used by Dependencies +Copyright (c) 2012 Elazar Leibovich. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are +met: + + * Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above +copyright notice, this list of conditions and the following disclaimer +in the documentation and/or other materials provided with the +distribution. + * Neither the name of Elazar Leibovich. nor the names of its +contributors may be used to endorse or promote products derived from +this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + + +====== Dependencies Grouped by License ============ +-------- Dependency +github.com/PuerkitoBio/purell +-------- Copyrights +Copyright (c) 2012, Martin Angers + +======== Dependencies Summary +github.com/PuerkitoBio/purell + +======== License used by Dependencies +Copyright (c) 2012, Martin Angers + + +Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: +* Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. +* Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. +* Neither the name of the author nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + + +==== Dependencies Grouped by License == +-------- Dependency +github.com/pmezard/go-difflib +-------- Copyrights +Copyright (c) 2013, Patrick Mezard + +==== Dependencies Summary +github.com/pmezard/go-difflib + +===== License used by Dependencies +Copyright (c) 2013, Patrick Mezard + + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are +met: + + Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. + Redistributions in binary form must reproduce the above copyright +notice, this list of conditions and the following disclaimer in the +documentation and/or other materials provided with the distribution. + The names of its contributors may not be used to endorse or promote +products derived from this software without specific prior written +permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS +IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED +TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A +PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED +TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR +PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF +LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING +NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS +SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + + +==== Dependencies Grouped by License ===== +-------- Dependency +github.com/mxk/go-flowrate +-------- Copyrights +Copyright (c) 2014 The Go-FlowRate Authors. + +===== Dependencies Summary +github.com/mxk/go-flowrate + +===== License used by Dependencies +Copyright (c) 2014 The Go-FlowRate Authors. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are +met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the + distribution. + + * Neither the name of the go-flowrate project nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + + +===== Dependencies Grouped by License === +-------- Dependency +github.com/evanphx/json-patch +-------- Copyrights +Copyright (c) 2014, Evan Phoenix + +======== Dependencies Summary +github.com/evanphx/json-patch + +======== License used by Dependencies +Copyright (c) 2014, Evan Phoenix + + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + +* Redistributions of source code must retain the above copyright notice, this + list of conditions and the following disclaimer. +* Redistributions in binary form must reproduce the above copyright notice + this list of conditions and the following disclaimer in the documentation + and/or other materials provided with the distribution. +* Neither the name of the Evan Phoenix nor the names of its contributors + may be used to endorse or promote products derived from this software + without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR +SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER +CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + + +=== Dependencies Grouped by License ==== +-------- Dependency +github.com/NYTimes/gziphandler +-------- Copyrights +Copyright (c) 2015 The New York Times Company + +======== Dependencies Summary +github.com/NYTimes/gziphandler + +======== License used by Dependencies +Copyright (c) 2015 The New York Times Company + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this library except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + +========= Dependencies Grouped by License ==== +-------- Dependency +github.com/hashicorp/golang-lru +-------- Copyrights + +======== Dependencies Summary +github.com/hashicorp/golang-lru + +======== License used by Dependencies +Mozilla Public License, version 2.0 + +1. Definitions + +1.1. "Contributor" + + means each individual or legal entity that creates, contributes to the + creation of, or owns Covered Software. + +1.2. "Contributor Version" + + means the combination of the Contributions of others (if any) used by a + Contributor and that particular Contributor's Contribution. + +1.3. "Contribution" + + means Covered Software of a particular Contributor. + +1.4. "Covered Software" + + means Source Code Form to which the initial Contributor has attached the + notice in Exhibit A, the Executable Form of such Source Code Form, and + Modifications of such Source Code Form, in each case including portions + thereof. + +1.5. "Incompatible With Secondary Licenses" + means + + a. that the initial Contributor has attached the notice described in + Exhibit B to the Covered Software; or + + b. that the Covered Software was made available under the terms of + version 1.1 or earlier of the License, but not also under the terms of + a Secondary License. + +1.6. "Executable Form" + + means any form of the work other than Source Code Form. + +1.7. "Larger Work" + + means a work that combines Covered Software with other material, in a + separate file or files, that is not Covered Software. + +1.8. "License" + + means this document. + +1.9. "Licensable" + + means having the right to grant, to the maximum extent possible, whether + at the time of the initial grant or subsequently, any and all of the + rights conveyed by this License. + +1.10. "Modifications" + + means any of the following: + + a. any file in Source Code Form that results from an addition to, + deletion from, or modification of the contents of Covered Software; or + + b. any new file in Source Code Form that contains any Covered Software. + +1.11. "Patent Claims" of a Contributor + + means any patent claim(s), including without limitation, method, + process, and apparatus claims, in any patent Licensable by such + Contributor that would be infringed, but for the grant of the License, + by the making, using, selling, offering for sale, having made, import, + or transfer of either its Contributions or its Contributor Version. + +1.12. "Secondary License" + + means either the GNU General Public License, Version 2.0, the GNU Lesser + General Public License, Version 2.1, the GNU Affero General Public + License, Version 3.0, or any later versions of those licenses. + +1.13. "Source Code Form" + + means the form of the work preferred for making modifications. + +1.14. "You" (or "Your") + + means an individual or a legal entity exercising rights under this + License. For legal entities, "You" includes any entity that controls, is + controlled by, or is under common control with You. For purposes of this + definition, "control" means (a) the power, direct or indirect, to cause + the direction or management of such entity, whether by contract or + otherwise, or (b) ownership of more than fifty percent (50%) of the + outstanding shares or beneficial ownership of such entity. + + +2. License Grants and Conditions + +2.1. Grants + + Each Contributor hereby grants You a world-wide, royalty-free, + non-exclusive license: + + a. under intellectual property rights (other than patent or trademark) + Licensable by such Contributor to use, reproduce, make available, + modify, display, perform, distribute, and otherwise exploit its + Contributions, either on an unmodified basis, with Modifications, or + as part of a Larger Work; and + + b. under Patent Claims of such Contributor to make, use, sell, offer for + sale, have made, import, and otherwise transfer either its + Contributions or its Contributor Version. + +2.2. Effective Date + + The licenses granted in Section 2.1 with respect to any Contribution + become effective for each Contribution on the date the Contributor first + distributes such Contribution. + +2.3. Limitations on Grant Scope + + The licenses granted in this Section 2 are the only rights granted under + this License. No additional rights or licenses will be implied from the + distribution or licensing of Covered Software under this License. + Notwithstanding Section 2.1(b) above, no patent license is granted by a + Contributor: + + a. for any code that a Contributor has removed from Covered Software; or + + b. for infringements caused by: (i) Your and any other third party's + modifications of Covered Software, or (ii) the combination of its + Contributions with other software (except as part of its Contributor + Version); or + + c. under Patent Claims infringed by Covered Software in the absence of + its Contributions. + + This License does not grant any rights in the trademarks, service marks, + or logos of any Contributor (except as may be necessary to comply with + the notice requirements in Section 3.4). + +2.4. Subsequent Licenses + + No Contributor makes additional grants as a result of Your choice to + distribute the Covered Software under a subsequent version of this + License (see Section 10.2) or under the terms of a Secondary License (if + permitted under the terms of Section 3.3). + +2.5. Representation + + Each Contributor represents that the Contributor believes its + Contributions are its original creation(s) or it has sufficient rights to + grant the rights to its Contributions conveyed by this License. + +2.6. Fair Use + + This License is not intended to limit any rights You have under + applicable copyright doctrines of fair use, fair dealing, or other + equivalents. + +2.7. Conditions + + Sections 3.1, 3.2, 3.3, and 3.4 are conditions of the licenses granted in + Section 2.1. + + +3. Responsibilities + +3.1. Distribution of Source Form + + All distribution of Covered Software in Source Code Form, including any + Modifications that You create or to which You contribute, must be under + the terms of this License. You must inform recipients that the Source + Code Form of the Covered Software is governed by the terms of this + License, and how they can obtain a copy of this License. You may not + attempt to alter or restrict the recipients' rights in the Source Code + Form. + +3.2. Distribution of Executable Form + + If You distribute Covered Software in Executable Form then: + + a. such Covered Software must also be made available in Source Code Form, + as described in Section 3.1, and You must inform recipients of the + Executable Form how they can obtain a copy of such Source Code Form by + reasonable means in a timely manner, at a charge no more than the cost + of distribution to the recipient; and + + b. You may distribute such Executable Form under the terms of this + License, or sublicense it under different terms, provided that the + license for the Executable Form does not attempt to limit or alter the + recipients' rights in the Source Code Form under this License. + +3.3. Distribution of a Larger Work + + You may create and distribute a Larger Work under terms of Your choice, + provided that You also comply with the requirements of this License for + the Covered Software. If the Larger Work is a combination of Covered + Software with a work governed by one or more Secondary Licenses, and the + Covered Software is not Incompatible With Secondary Licenses, this + License permits You to additionally distribute such Covered Software + under the terms of such Secondary License(s), so that the recipient of + the Larger Work may, at their option, further distribute the Covered + Software under the terms of either this License or such Secondary + License(s). + +3.4. Notices + + You may not remove or alter the substance of any license notices + (including copyright notices, patent notices, disclaimers of warranty, or + limitations of liability) contained within the Source Code Form of the + Covered Software, except that You may alter any license notices to the + extent required to remedy known factual inaccuracies. + +3.5. Application of Additional Terms + + You may choose to offer, and to charge a fee for, warranty, support, + indemnity or liability obligations to one or more recipients of Covered + Software. However, You may do so only on Your own behalf, and not on + behalf of any Contributor. You must make it absolutely clear that any + such warranty, support, indemnity, or liability obligation is offered by + You alone, and You hereby agree to indemnify every Contributor for any + liability incurred by such Contributor as a result of warranty, support, + indemnity or liability terms You offer. You may include additional + disclaimers of warranty and limitations of liability specific to any + jurisdiction. + +4. Inability to Comply Due to Statute or Regulation + + If it is impossible for You to comply with any of the terms of this License + with respect to some or all of the Covered Software due to statute, + judicial order, or regulation then You must: (a) comply with the terms of + this License to the maximum extent possible; and (b) describe the + limitations and the code they affect. Such description must be placed in a + text file included with all distributions of the Covered Software under + this License. Except to the extent prohibited by statute or regulation, + such description must be sufficiently detailed for a recipient of ordinary + skill to be able to understand it. + +5. Termination + +5.1. The rights granted under this License will terminate automatically if You + fail to comply with any of its terms. However, if You become compliant, + then the rights granted under this License from a particular Contributor + are reinstated (a) provisionally, unless and until such Contributor + explicitly and finally terminates Your grants, and (b) on an ongoing + basis, if such Contributor fails to notify You of the non-compliance by + some reasonable means prior to 60 days after You have come back into + compliance. Moreover, Your grants from a particular Contributor are + reinstated on an ongoing basis if such Contributor notifies You of the + non-compliance by some reasonable means, this is the first time You have + received notice of non-compliance with this License from such + Contributor, and You become compliant prior to 30 days after Your receipt + of the notice. + +5.2. If You initiate litigation against any entity by asserting a patent + infringement claim (excluding declaratory judgment actions, + counter-claims, and cross-claims) alleging that a Contributor Version + directly or indirectly infringes any patent, then the rights granted to + You by any and all Contributors for the Covered Software under Section + 2.1 of this License shall terminate. + +5.3. In the event of termination under Sections 5.1 or 5.2 above, all end user + license agreements (excluding distributors and resellers) which have been + validly granted by You or Your distributors under this License prior to + termination shall survive termination. + +6. Disclaimer of Warranty + + Covered Software is provided under this License on an "as is" basis, + without warranty of any kind, either expressed, implied, or statutory, + including, without limitation, warranties that the Covered Software is free + of defects, merchantable, fit for a particular purpose or non-infringing. + The entire risk as to the quality and performance of the Covered Software + is with You. Should any Covered Software prove defective in any respect, + You (not any Contributor) assume the cost of any necessary servicing, + repair, or correction. This disclaimer of warranty constitutes an essential + part of this License. No use of any Covered Software is authorized under + this License except under this disclaimer. + +7. Limitation of Liability + + Under no circumstances and under no legal theory, whether tort (including + negligence), contract, or otherwise, shall any Contributor, or anyone who + distributes Covered Software as permitted above, be liable to You for any + direct, indirect, special, incidental, or consequential damages of any + character including, without limitation, damages for lost profits, loss of + goodwill, work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses, even if such party shall have been + informed of the possibility of such damages. This limitation of liability + shall not apply to liability for death or personal injury resulting from + such party's negligence to the extent applicable law prohibits such + limitation. Some jurisdictions do not allow the exclusion or limitation of + incidental or consequential damages, so this exclusion and limitation may + not apply to You. + +8. Litigation + + Any litigation relating to this License may be brought only in the courts + of a jurisdiction where the defendant maintains its principal place of + business and such litigation shall be governed by laws of that + jurisdiction, without reference to its conflict-of-law provisions. Nothing + in this Section shall prevent a party's ability to bring cross-claims or + counter-claims. + +9. Miscellaneous + + This License represents the complete agreement concerning the subject + matter hereof. If any provision of this License is held to be + unenforceable, such provision shall be reformed only to the extent + necessary to make it enforceable. Any law or regulation which provides that + the language of a contract shall be construed against the drafter shall not + be used to construe this License against a Contributor. + + +10. Versions of the License + +10.1. New Versions + + Mozilla Foundation is the license steward. Except as provided in Section + 10.3, no one other than the license steward has the right to modify or + publish new versions of this License. Each version will be given a + distinguishing version number. + +10.2. Effect of New Versions + + You may distribute the Covered Software under the terms of the version + of the License under which You originally received the Covered Software, + or under the terms of any subsequent version published by the license + steward. + +10.3. Modified Versions + + If you create software not governed by this License, and you want to + create a new license for such software, you may create and use a + modified version of this License if you rename the license and remove + any references to the name of the license steward (except to note that + such modified license differs from this License). + +10.4. Distributing Source Code Form that is Incompatible With Secondary + Licenses If You choose to distribute Source Code Form that is + Incompatible With Secondary Licenses under the terms of this version of + the License, the notice described in Exhibit B of this License must be + attached. + +Exhibit A - Source Code Form License Notice + + This Source Code Form is subject to the + terms of the Mozilla Public License, v. + 2.0. If a copy of the MPL was not + distributed with this file, You can + obtain one at + http://mozilla.org/MPL/2.0/. + +If it is not possible or desirable to put the notice in a particular file, +then You may include the notice in a location (such as a LICENSE file in a +relevant directory) where a recipient would be likely to look for such a +notice. + +You may add additional accurate notices of copyright ownership. + +Exhibit B - "Incompatible With Secondary Licenses" Notice + + This Source Code Form is "Incompatible + With Secondary Licenses", as defined by + the Mozilla Public License, v. 2.0. + + +======= Dependencies Grouped by License ===== +-------- Dependency +github.com/docker/spdystream +-------- Copyrights + Copyright 2014-2015 Docker, Inc. +// Copyright 2013 The Go Authors. +// Copyright 2011 The Go Authors. + +-------- Dependency +github.com/go-logr/logr +-------- Copyrights + +-------- Dependency +github.com/go-openapi/jsonpointer +-------- Copyrights +// Copyright 2013 sigu-399 ( https://github.com/sigu-399 ) + +-------- Dependency +github.com/go-openapi/jsonreference +-------- Copyrights +// Copyright 2013 sigu-399 ( https://github.com/sigu-399 ) + +-------- Dependency +github.com/go-openapi/spec +-------- Copyrights +// Copyright 2015 go-swagger maintainers + +-------- Dependency +github.com/go-openapi/swag +-------- Copyrights +// Copyright 2015 go-swagger maintainers + +-------- Dependency +github.com/golang/groupcache +-------- Copyrights +Copyright 2012 Google Inc. +Copyright 2013 Google Inc. + +-------- Dependency +github.com/google/gofuzz +-------- Copyrights +Copyright 2014 Google Inc. + +-------- Dependency +github.com/googleapis/gnostic +-------- Copyrights +// Copyright 2017 Google Inc. + "// Copyright 2017 Google Inc. \n" + + Copyright 2017 Google Inc. + +-------- Dependency +github.com/modern-go/concurrent +-------- Copyrights + +-------- Dependency +github.com/modern-go/reflect2 +-------- Copyrights + +-------- Dependency +gopkg.in/yaml.v2 +-------- Copyrights +Copyright (c) 2006 Kirill Simonov +-------- Notices +Copyright 2011-2016 Canonical Ltd. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + + +-------- Dependency +k8s.io/gengo +-------- Copyrights + Copyright 2014 The Kubernetes Authors. +Copyright 2015 The Kubernetes Authors. +Copyright 2016 The Kubernetes Authors. +Copyright 2018 The Kubernetes Authors. +Copyright 2017 The Kubernetes Authors. +Copyright 2014 The Kubernetes Authors. + +-------- Dependency +k8s.io/klog +-------- Copyrights +// Copyright 2013 Google Inc. + +-------- Dependency +k8s.io/kube-openapi +-------- Copyrights +Copyright 2018 The Kubernetes Authors. +Copyright 2017 The Kubernetes Authors. +Copyright 2019 The Kubernetes Authors. +Copyright 2016 The Kubernetes Authors. + +-------- Dependency +sigs.k8s.io/structured-merge-diff/v3 +-------- Copyrights +Copyright 2018 The Kubernetes Authors. +Copyright 2019 The Kubernetes Authors. +Copyright 2020 The Kubernetes Authors. + +======== Dependencies Summary +github.com/docker/spdystream +github.com/go-logr/logr +github.com/go-openapi/jsonpointer +github.com/go-openapi/jsonreference +github.com/go-openapi/spec +github.com/go-openapi/swag +github.com/golang/groupcache +github.com/google/gofuzz +github.com/googleapis/gnostic +github.com/modern-go/concurrent +github.com/modern-go/reflect2 +gopkg.in/yaml.v2 +k8s.io/gengo +k8s.io/klog +k8s.io/kube-openapi +sigs.k8s.io/structured-merge-diff/v3 + +======== License used by Dependencies +SPDX:Apache-2.0 + +======== Dependencies Grouped by License ==== +-------- Dependency +github.com/PuerkitoBio/urlesc +-------- Copyrights +Copyright (c) 2012 The Go Authors. +// Copyright 2009 The Go Authors. + +-------- Dependency +github.com/gogo/protobuf +-------- Copyrights +Copyright (c) 2013, The GoGo Authors. +Copyright 2010 The Go Authors. +# Copyright (c) 2013, The GoGo Authors. +// Copyright (c) 2015, The GoGo Authors. +# Copyright 2016 The Go Authors. +// Copyright 2016 The Go Authors. +// Copyright (c) 2013, The GoGo Authors. +// Copyright 2015 The Go Authors. +# Copyright 2015 The Go Authors. +# Copyright 2010 The Go Authors. +// Copyright 2010 The Go Authors. +// Copyright 2011 The Go Authors. +// Copyright (c) 2018, The GoGo Authors. +// Copyright 2018 The Go Authors. +// Copyright 2017 The Go Authors. +// Copyright (c) 2016, The GoGo Authors. +// Copyright 2014 The Go Authors. +// Copyright 2012 The Go Authors. +// Copyright 2013 The Go Authors. +# Copyright (c) 2018, The GoGo Authors. +# Copyright (c) 2015, The GoGo Authors. +# Copyright (c) 2016, The GoGo Authors. +# Copyright (c) 2019, The GoGo Authors. +// Copyright (c) 2019, The GoGo Authors. +# Copyright (c) 2017, The GoGo Authors. +// Copyright (c) 2017, The GoGo Authors. +// Copyright (c) 2015, The GoGo Authors. rights reserved. + +-------- Dependency +github.com/golang/protobuf +-------- Copyrights +Copyright 2010 The Go Authors. +# Copyright 2010 The Go Authors. +// Copyright 2016 The Go Authors. +// Copyright 2015 The Go Authors. +// Copyright 2010 The Go Authors. +// Copyright 2011 The Go Authors. +// Copyright 2018 The Go Authors. +// Copyright 2017 The Go Authors. +// Copyright 2014 The Go Authors. +// Copyright 2012 The Go Authors. +// Copyright 2013 The Go Authors. + +-------- Dependency +github.com/google/go-cmp +-------- Copyrights +Copyright (c) 2017 The Go Authors. +// Copyright 2017, The Go Authors. +// Copyright 2018, The Go Authors. +// Copyright 2019, The Go Authors. + +-------- Dependency +github.com/google/uuid +-------- Copyrights +Copyright (c) 2009,2014 Google Inc. +// Copyright 2016 Google Inc. +// Copyright 2017 Google Inc. +// Copyright 2018 Google Inc. + +-------- Dependency +github.com/spf13/pflag +-------- Copyrights +Copyright (c) 2012 Alex Ogier. +Copyright (c) 2012 The Go Authors. +// Copyright 2009 The Go Authors. +// Copyright 2012 The Go Authors. +// Copyright 2010 The Go Authors. + +-------- Dependency +golang.org/x/crypto +-------- Copyrights +Copyright (c) 2009 The Go Authors. +// Copyright 2015 The Go Authors. +// Copyright 2016 The Go Authors. +// Copyright 2017 The Go Authors. +// Copyright 2018 The Go Authors. +// Copyright 2011 The Go Authors. +// Copyright 2010 The Go Authors. +// Copyright 2012 The Go Authors. +// Copyright 2013 The Go Authors. +// Copyright 2014 The Go Authors. +// Copyright 2009 The Go Authors. +-------- Patents +Additional IP Rights Grant (Patents) + +"This implementation" means the copyrightable works distributed by +Google as part of the Go project. + +Google hereby grants to You a perpetual, worldwide, non-exclusive, +no-charge, royalty-free, irrevocable (except as stated in this section) +patent license to make, have made, use, offer to sell, sell, import, +transfer and otherwise run, modify and propagate the contents of this +implementation of Go, where such license applies only to those patent +claims, both currently owned or controlled by Google and acquired in +the future, licensable by Google that are necessarily infringed by this +implementation of Go. This grant does not include claims that would be +infringed only as a consequence of further modification of this +implementation. If you or your agent or exclusive licensee institute or +order or agree to the institution of patent litigation against any +entity (including a cross-claim or counterclaim in a lawsuit) alleging +that this implementation of Go or any code incorporated within this +implementation of Go constitutes direct or contributory patent +infringement, or inducement of patent infringement, then any patent +rights granted to you under this License for this implementation of Go +shall terminate as of the date such litigation is filed. + + +-------- Dependency +golang.org/x/net +-------- Copyrights +Copyright (c) 2009 The Go Authors. +// Copyright 2016 The Go Authors. +// Copyright 2017 The Go Authors. +// Copyright 2014 The Go Authors. +// Copyright 2015 The Go Authors. +// Copyright 2010 The Go Authors. +// Copyright 2009 The Go Authors. +// Copyright 2012 The Go Authors. +// Copyright 2013 The Go Authors. +// Copyright 2011 The Go Authors. +// Copyright 2018 The Go Authors. +// Copyright 2019 The Go Authors. +-------- Patents +Additional IP Rights Grant (Patents) + +"This implementation" means the copyrightable works distributed by +Google as part of the Go project. + +Google hereby grants to You a perpetual, worldwide, non-exclusive, +no-charge, royalty-free, irrevocable (except as stated in this section) +patent license to make, have made, use, offer to sell, sell, import, +transfer and otherwise run, modify and propagate the contents of this +implementation of Go, where such license applies only to those patent +claims, both currently owned or controlled by Google and acquired in +the future, licensable by Google that are necessarily infringed by this +implementation of Go. This grant does not include claims that would be +infringed only as a consequence of further modification of this +implementation. If you or your agent or exclusive licensee institute or +order or agree to the institution of patent litigation against any +entity (including a cross-claim or counterclaim in a lawsuit) alleging +that this implementation of Go or any code incorporated within this +implementation of Go constitutes direct or contributory patent +infringement, or inducement of patent infringement, then any patent +rights granted to you under this License for this implementation of Go +shall terminate as of the date such litigation is filed. + + +-------- Dependency +golang.org/x/sync +-------- Copyrights +Copyright (c) 2009 The Go Authors. +// Copyright 2016 The Go Authors. +// Copyright 2017 The Go Authors. +// Copyright 2013 The Go Authors. +-------- Patents +Additional IP Rights Grant (Patents) + +"This implementation" means the copyrightable works distributed by +Google as part of the Go project. + +Google hereby grants to You a perpetual, worldwide, non-exclusive, +no-charge, royalty-free, irrevocable (except as stated in this section) +patent license to make, have made, use, offer to sell, sell, import, +transfer and otherwise run, modify and propagate the contents of this +implementation of Go, where such license applies only to those patent +claims, both currently owned or controlled by Google and acquired in +the future, licensable by Google that are necessarily infringed by this +implementation of Go. This grant does not include claims that would be +infringed only as a consequence of further modification of this +implementation. If you or your agent or exclusive licensee institute or +order or agree to the institution of patent litigation against any +entity (including a cross-claim or counterclaim in a lawsuit) alleging +that this implementation of Go or any code incorporated within this +implementation of Go constitutes direct or contributory patent +infringement, or inducement of patent infringement, then any patent +rights granted to you under this License for this implementation of Go +shall terminate as of the date such litigation is filed. + + +-------- Dependency +golang.org/x/sys +-------- Copyrights +Copyright (c) 2009 The Go Authors. +// Copyright 2019 The Go Authors. +// Copyright 2018 The Go Authors. +// Copyright 2012 The Go Authors. +// Copyright 2011 The Go Authors. +// Copyright 2015 The Go Authors. +// Copyright 2009 The Go Authors. +// Copyright 2013 The Go Authors. +// Copyright 2016 The Go Authors. +// Copyright 2017 The Go Authors. +// Copyright 2010 The Go Authors. +// Copyright 2014 The Go Authors. +// Copyright 2009,2010 The Go Authors. +// Copyright 2017 The Go Authors. All right reserved. +-------- Patents +Additional IP Rights Grant (Patents) + +"This implementation" means the copyrightable works distributed by +Google as part of the Go project. + +Google hereby grants to You a perpetual, worldwide, non-exclusive, +no-charge, royalty-free, irrevocable (except as stated in this section) +patent license to make, have made, use, offer to sell, sell, import, +transfer and otherwise run, modify and propagate the contents of this +implementation of Go, where such license applies only to those patent +claims, both currently owned or controlled by Google and acquired in +the future, licensable by Google that are necessarily infringed by this +implementation of Go. This grant does not include claims that would be +infringed only as a consequence of further modification of this +implementation. If you or your agent or exclusive licensee institute or +order or agree to the institution of patent litigation against any +entity (including a cross-claim or counterclaim in a lawsuit) alleging +that this implementation of Go or any code incorporated within this +implementation of Go constitutes direct or contributory patent +infringement, or inducement of patent infringement, then any patent +rights granted to you under this License for this implementation of Go +shall terminate as of the date such litigation is filed. + + +-------- Dependency +golang.org/x/text +-------- Copyrights +Copyright (c) 2009 The Go Authors. +// Copyright 2014 The Go Authors. +// Copyright 2016 The Go Authors. +// Copyright 2015 The Go Authors. +// Copyright 2017 The Go Authors. +// Copyright 2012 The Go Authors. +// Copyright 2013 The Go Authors. +# Copyright 2012 The Go Authors. +// Copyright 2018 The Go Authors. +// Copyright 2009 The Go Authors. +// Copyright 2019 The Go Authors. +// Copyright 2011 The Go Authors. +-------- Patents +Additional IP Rights Grant (Patents) + +"This implementation" means the copyrightable works distributed by +Google as part of the Go project. + +Google hereby grants to You a perpetual, worldwide, non-exclusive, +no-charge, royalty-free, irrevocable (except as stated in this section) +patent license to make, have made, use, offer to sell, sell, import, +transfer and otherwise run, modify and propagate the contents of this +implementation of Go, where such license applies only to those patent +claims, both currently owned or controlled by Google and acquired in +the future, licensable by Google that are necessarily infringed by this +implementation of Go. This grant does not include claims that would be +infringed only as a consequence of further modification of this +implementation. If you or your agent or exclusive licensee institute or +order or agree to the institution of patent litigation against any +entity (including a cross-claim or counterclaim in a lawsuit) alleging +that this implementation of Go or any code incorporated within this +implementation of Go constitutes direct or contributory patent +infringement, or inducement of patent infringement, then any patent +rights granted to you under this License for this implementation of Go +shall terminate as of the date such litigation is filed. + + +-------- Dependency +golang.org/x/tools +-------- Copyrights +Copyright (c) 2009 The Go Authors. +// Copyright 2014 The Go Authors. +// Copyright 2009 The Go Authors. +// Copyright 2013 The Go Authors. +// Copyright 2018 The Go Authors. +// Copyright 2019 The Go Authors. +// Copyright 2015 The Go Authors. +// Copyright 2015 The Go Authors. +// Copyright 2013 The Go Authors. +Copyright (c) 2017 The Go Authors. +// Copyright 2017 The Go Authors. +// Copyright 2010 The Go Authors. +// Copyright 2009 The Go Authors. +// Copyright 2012 The Go Authors. +// Copyright 2016 The Go Authors. +// Copyright 2011 The Go Authors. + Copyright © 1994-1999 Lucent Technologies Inc. + Portions Copyright © 1995-1997 C H Forsyth (forsyth@terzarima.net) + Portions Copyright © 1997-1999 Vita Nuova Limited + Portions Copyright © 2000-2007 Vita Nuova Holdings Limited (www.vitanuova.com) + Portions Copyright © 2004,2006 Bruce Ellis + Portions Copyright © 2005-2007 C H Forsyth (forsyth@terzarima.net) + Revisions Copyright © 2000-2007 Lucent Technologies Inc. and others + Portions Copyright © 2009 The Go Authors. +// Copyright 2012 The Go Authors. +-------- Patents +Additional IP Rights Grant (Patents) + +"This implementation" means the copyrightable works distributed by +Google as part of the Go project. + +Google hereby grants to You a perpetual, worldwide, non-exclusive, +no-charge, royalty-free, irrevocable (except as stated in this section) +patent license to make, have made, use, offer to sell, sell, import, +transfer and otherwise run, modify and propagate the contents of this +implementation of Go, where such license applies only to those patent +claims, both currently owned or controlled by Google and acquired in +the future, licensable by Google that are necessarily infringed by this +implementation of Go. This grant does not include claims that would be +infringed only as a consequence of further modification of this +implementation. If you or your agent or exclusive licensee institute or +order or agree to the institution of patent litigation against any +entity (including a cross-claim or counterclaim in a lawsuit) alleging +that this implementation of Go or any code incorporated within this +implementation of Go constitutes direct or contributory patent +infringement, or inducement of patent infringement, then any patent +rights granted to you under this License for this implementation of Go +shall terminate as of the date such litigation is filed. + + +-------- Dependency +golang.org/x/xerrors +-------- Copyrights +Copyright (c) 2019 The Go Authors. +// Copyright 2018 The Go Authors. +// Copyright 2019 The Go Authors. +// Copyright 2011 The Go Authors. +// Copyright 2012 The Go Authors. +-------- Patents +Additional IP Rights Grant (Patents) + +"This implementation" means the copyrightable works distributed by +Google as part of the Go project. + +Google hereby grants to You a perpetual, worldwide, non-exclusive, +no-charge, royalty-free, irrevocable (except as stated in this section) +patent license to make, have made, use, offer to sell, sell, import, +transfer and otherwise run, modify and propagate the contents of this +implementation of Go, where such license applies only to those patent +claims, both currently owned or controlled by Google and acquired in +the future, licensable by Google that are necessarily infringed by this +implementation of Go. This grant does not include claims that would be +infringed only as a consequence of further modification of this +implementation. If you or your agent or exclusive licensee institute or +order or agree to the institution of patent litigation against any +entity (including a cross-claim or counterclaim in a lawsuit) alleging +that this implementation of Go or any code incorporated within this +implementation of Go constitutes direct or contributory patent +infringement, or inducement of patent infringement, then any patent +rights granted to you under this License for this implementation of Go +shall terminate as of the date such litigation is filed. + + +-------- Dependency +gopkg.in/fsnotify.v1 +-------- Copyrights +Copyright (c) 2012 The Go Authors. +Copyright (c) 2012 fsnotify Authors. +// Copyright 2012 The Go Authors. +// Copyright 2010 The Go Authors. +// Copyright 2016 The Go Authors. +// Copyright 2015 The Go Authors. +// Copyright 2013 The Go Authors. +// Copyright 2011 The Go Authors. + +======== Dependencies Summary +github.com/PuerkitoBio/urlesc +github.com/gogo/protobuf +github.com/golang/protobuf +github.com/google/go-cmp +github.com/google/uuid +github.com/spf13/pflag +golang.org/x/crypto +golang.org/x/net +golang.org/x/sync +golang.org/x/sys +golang.org/x/text +golang.org/x/tools +golang.org/x/xerrors +gopkg.in/fsnotify.v1 + +======== License used by Dependencies +SPDX:BSD-3-Clause--modified-by-Google +Redistribution and use in source and binary forms, with +or without modification, are permitted provided that the following conditions +are met: + + * Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above +copyright notice, this list of conditions and the following disclaimer +in the documentation and/or other materials provided with the +distribution. + * Neither the name of Google Inc. nor the names of its +contributors may be used to endorse or promote products derived from +this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +====== Dependencies Grouped by License ===== +-------- Dependency +github.com/davecgh/go-spew +-------- Copyrights +Copyright (c) 2012-2016 Dave Collins +// Copyright (c) 2015-2016 Dave Collins + * Copyright (c) 2013-2016 Dave Collins +// Copyright (c) 2013-2016 Dave Collins +// Copyright (c) 2013 Dave Collins + +======== Dependencies Summary +github.com/davecgh/go-spew + +======== License used by Dependencies +SPDX:ISC +Permission to use, copy, modify, and/or distribute this +software for any purpose with or without fee is hereby granted, provided that +the above copyright notice and this permission notice appear in all copies. + +THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH +REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY +AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, +INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM +LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR +OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR +PERFORMANCE OF THIS SOFTWARE. + +=== Dependencies Grouped by License ==== +-------- Dependency +github.com/emicklei/go-restful +-------- Copyrights +Copyright (c) 2012,2013 Ernest Micklei +// Copyright 2013 Ernest Micklei. +// Copyright 2015 Ernest Micklei. +// Copyright 2014 Ernest Micklei. + +-------- Dependency +github.com/hpcloud/tail +-------- Copyrights +# © Copyright 2015 Hewlett Packard Enterprise Development LP +Copyright (c) 2014 ActiveState +// Copyright (c) 2015 HPE Software Inc. +// Copyright (c) 2013 ActiveState Software Inc. +Copyright (C) 2013 99designs + +-------- Dependency +github.com/json-iterator/go +-------- Copyrights +Copyright (c) 2016 json-iterator + +-------- Dependency +github.com/kisielk/errcheck +-------- Copyrights +Copyright (c) 2013 Kamil Kisiel + +-------- Dependency +github.com/kisielk/gotool +-------- Copyrights +Copyright (c) 2013 Kamil Kisiel +// Copyright 2012 The Go Authors. +// Copyright 2017 The Go Authors. +// Copyright 2011 The Go Authors. +// Copyright (c) 2009 The Go Authors. + +-------- Dependency +github.com/kr/pretty +-------- Copyrights +Copyright 2012 Keith Rarick + +-------- Dependency +github.com/kr/pty +-------- Copyrights +Copyright (c) 2011 Keith Rarick + +-------- Dependency +github.com/kr/text +-------- Copyrights +Copyright 2012 Keith Rarick + +-------- Dependency +github.com/mailru/easyjson +-------- Copyrights +Copyright (c) 2016 Mail.Ru Group + +-------- Dependency +github.com/onsi/ginkgo +-------- Copyrights +Copyright (c) 2013-2014 Onsi Fakhouri +Copyright (c) 2016 Yasuhiro Matsumoto + +-------- Dependency +github.com/onsi/gomega +-------- Copyrights +Copyright (c) 2013-2014 Onsi Fakhouri +Copyright (c) 2014 Amit Kumar Gupta + +-------- Dependency +github.com/stretchr/objx +-------- Copyrights +Copyright (c) 2014 Stretchr, Inc. +Copyright (c) 2017-2018 objx contributors + +-------- Dependency +github.com/stretchr/testify +-------- Copyrights +Copyright (c) 2012-2018 Mat Ryer and Tyler Bunnell + +======== Dependencies Summary +github.com/emicklei/go-restful +github.com/hpcloud/tail +github.com/json-iterator/go +github.com/kisielk/errcheck +github.com/kisielk/gotool +github.com/kr/pretty +github.com/kr/pty +github.com/kr/text +github.com/mailru/easyjson +github.com/onsi/ginkgo +github.com/onsi/gomega +github.com/stretchr/objx +github.com/stretchr/testify + +======== License used by Dependencies +SPDX:MIT +Permission is hereby granted, free of charge, to any person +obtaining a copy of this software and associated documentation files +(the "Software"), to deal in the Software without restriction, including without +limitation the rights to use, copy, modify, merge, publish, distribute, +sublicense, and/or sell copies of the Software, and to permit persons to whom +the Software is furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS +FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR +COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER +IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN +CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +====== Dependencies Grouped by License ===== +-------- Dependency +gopkg.in/tomb.v1 +-------- Copyrights +Copyright (c) 2010-2011 - Gustavo Niemeyer +// Copyright (c) 2011 - Gustavo Niemeyer + +======== Dependencies Summary +gopkg.in/tomb.v1 + +======== License used by Dependencies +tomb - support for clean goroutine termination in Go. + +Copyright (c) 2010-2011 - Gustavo Niemeyer + + + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + + * Redistributions of source code must retain the above copyright notice, + this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright notice, + this list of conditions and the following disclaimer in the documentation + and/or other materials provided with the distribution. + * Neither the name of the copyright holder nor the names of its + contributors may be used to endorse or promote products derived from + this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR +CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, +EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR +PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF +LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING +NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS +SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +=================================================================================================== +Kubernetes client-go +https://pkg.go.dev/k8s.io/client-go +Apache 2.0 + +4th party dependencies +--------------------------------------------------------------------------------------- +----------------------- Dependencies Grouped by License ------------ +-------- Dependency +github.com/evanphx/json-patch +-------- Copyrights +Copyright (c) 2014, Evan Phoenix + +-------- Dependencies Summary +github.com/evanphx/json-patch + +-------- License used by Dependencies +Copyright (c) 2014, Evan Phoenix + + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + +* Redistributions of source code must retain the above copyright notice, this + list of conditions and the following disclaimer. +* Redistributions in binary form must reproduce the above copyright notice, + this list of conditions and the following disclaimer in the documentation + and/or other materials provided with the distribution. +* Neither the name of the Evan Phoenix nor the names of its contributors + may be used to endorse or promote products derived from this software + without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR +SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER +CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + + +----------------------- Dependencies Grouped by License ------------ +-------- Dependency +github.com/pkg/errors +-------- Copyrights +Copyright (c) 2015, Dave Cheney + +-------- Dependencies Summary +github.com/pkg/errors + +-------- License used by Dependencies +Copyright (c) 2015, Dave Cheney + + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + +* Redistributions of source code must retain the above copyright notice, this + list of conditions and the following disclaimer. + +* Redistributions in binary form must reproduce the above copyright notice, + this list of conditions and the following disclaimer in the documentation + and/or other materials provided with the distribution. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR +SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER +CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + + +----------------------- Dependencies Grouped by License ------------ +-------- Dependency +github.com/gregjones/httpcache +-------- Copyrights +Copyright © 2012 Greg Jones (greg.jones@gmail.com) + +-------- Dependencies Summary +github.com/gregjones/httpcache + +-------- License used by Dependencies +Copyright © 2012 Greg Jones (greg.jones@gmail.com) + +Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Softwareâ€), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED “AS ISâ€, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +----------------------- Dependencies Grouped by License ------------ +-------- Dependency +cloud.google.com/go +-------- Copyrights +Copyright 2020 Google LLC +Copyright 2019 Google LLC +Copyright 2016 Google LLC +Copyright (c) 1996-1998 John D. Polstra. +Copyright (c) 2001 David E. O'Brien +Portions Copyright 2018 Google LLC. +Copyright 2018 Google LLC +Copyright 2014 Google LLC +Copyright 2017 Google LLC +Copyright 2018 Google Inc. +Copyright 2020, Google LLC +Copyright 2017, Google Inc. +Copyright 2017, Google LLC + +-------- Dependency +github.com/Azure/go-autorest/autorest +-------- Copyrights +Copyright 2015 Microsoft Corporation +Copyright 2017 Microsoft Corporation + +-------- Dependency +github.com/Azure/go-autorest/autorest/adal +-------- Copyrights +Copyright 2015 Microsoft Corporation +Copyright 2017 Microsoft Corporation + +-------- Dependency +github.com/Azure/go-autorest/autorest/date +-------- Copyrights +Copyright 2015 Microsoft Corporation +Copyright 2017 Microsoft Corporation + +-------- Dependency +github.com/Azure/go-autorest/logger +-------- Copyrights +Copyright 2015 Microsoft Corporation +Copyright 2017 Microsoft Corporation + +-------- Dependency +github.com/Azure/go-autorest/tracing +-------- Copyrights +Copyright 2015 Microsoft Corporation +Copyright 2017 Microsoft Corporation +Copyright 2018 Microsoft Corporation + +-------- Dependency +github.com/go-logr/logr +-------- Copyrights +Copyright 2020 The logr Authors. +Copyright 2019 The logr Authors. +Copyright 2021 The logr Authors. + +-------- Dependency +github.com/golang/groupcache +-------- Copyrights +Copyright 2012 Google Inc. +Copyright 2013 Google Inc. + +-------- Dependency +github.com/google/btree +-------- Copyrights +Copyright 2014 Google Inc. + +-------- Dependency +github.com/google/gofuzz +-------- Copyrights +Copyright 2014 Google Inc. + +-------- Dependency +github.com/googleapis/gnostic +-------- Copyrights +Copyright 2017-2020, Google LLC. +Copyright 2019 Google LLC. +Copyright 2017 Google LLC. +Copyright 2020 Google LLC. +Copyright 2018 Google LLC. +Copyright 2020 Google LLC. \n" + + +-------- Dependency +github.com/moby/spdystream +-------- Copyrights +Copyright 2014-2021 Docker Inc. +Copyright 2013-2021 Docker, inc. Released under the [Apache 2.0 license](LICENSE). +Copyright 2013 The Go Authors. +Copyright 2011 The Go Authors. +-------- Notices +SpdyStream +Copyright 2014-2021 Docker Inc. + +This product includes software developed at +Docker Inc. (https://www.docker.com/). + + +-------- Dependency +github.com/modern-go/concurrent +-------- Copyrights + +-------- Dependency +github.com/modern-go/reflect2 +-------- Copyrights + +-------- Dependency +gopkg.in/yaml.v2 +-------- Copyrights +Copyright (c) 2006 Kirill Simonov +Copyright 2011-2016 Canonical Ltd. +-------- Notices +Copyright 2011-2016 Canonical Ltd. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + + +-------- Dependency +k8s.io/api +-------- Copyrights +Copyright 2019 The Kubernetes Authors. +Copyright The Kubernetes Authors. +Copyright 2017 The Kubernetes Authors. +Copyright 2016 The Kubernetes Authors. +Copyright 2015 The Kubernetes Authors. +Copyright 2018 The Kubernetes Authors. +Copyright 2020 The Kubernetes Authors. +Copyright 2021 The Kubernetes Authors. + +-------- Dependency +k8s.io/apimachinery +-------- Copyrights +Copyright 2017 The Kubernetes Authors. +Copyright 2019 The Kubernetes Authors. +Copyright 2021 The Kubernetes Authors. +Copyright 2014 The Kubernetes Authors. +Copyright 2020 The Kubernetes Authors. +Copyright 2015 The Kubernetes Authors. +Copyright 2016 The Kubernetes Authors. +Copyright 2018 The Kubernetes Authors. +Copyright The Kubernetes Authors. +Copyright 2013 The Go Authors. +Copyright 2009 The Go Authors. + +-------- Dependency +k8s.io/klog/v2 +-------- Copyrights +Copyright 2013 Google Inc. +Copyright 2020 The Kubernetes Authors. + +-------- Dependency +k8s.io/kube-openapi +-------- Copyrights +Copyright The Kubernetes Authors. +Copyright 2018 The Kubernetes Authors. +Copyright 2017 The Kubernetes Authors. +Copyright 2019 The Kubernetes Authors. +Copyright 2016 The Kubernetes Authors. +Copyright 2020 The Kubernetes Authors. +Copyright 2015 go-swagger maintainers +Copyright 2017 go-swagger maintainers +Copyright (C) MongoDB, Inc. 2017-present. + +-------- Dependency +k8s.io/utils +-------- Copyrights +Copyright 2018 The Kubernetes Authors. +Copyright 2017 The Kubernetes Authors. +Copyright 2014 The Kubernetes Authors. +Copyright 2015 The Kubernetes Authors. +Copyright 2016 The Kubernetes Authors. +Copyright 2020 The Kubernetes Authors. +Copyright 2019 The Kubernetes Authors. +Copyright (c) 2009 The Go Authors. +Copyright 2010 The Go Authors. +Copyright (c) 2012 The Go Authors. +Copyright 2013 Google Inc. +Copyright 2009 The Go Authors. +Copyright 2021 The Kubernetes Authors. + +-------- Dependency +sigs.k8s.io/structured-merge-diff/v4 +-------- Copyrights +Copyright 2018 The Kubernetes Authors. +Copyright 2019 The Kubernetes Authors. +Copyright 2020 The Kubernetes Authors. + +-------- Dependencies Summary +cloud.google.com/go +github.com/Azure/go-autorest/autorest +github.com/Azure/go-autorest/autorest/adal +github.com/Azure/go-autorest/autorest/date +github.com/Azure/go-autorest/logger +github.com/Azure/go-autorest/tracing +github.com/go-logr/logr +github.com/golang/groupcache +github.com/google/btree +github.com/google/gofuzz +github.com/googleapis/gnostic +github.com/moby/spdystream +github.com/modern-go/concurrent +github.com/modern-go/reflect2 +gopkg.in/yaml.v2 +k8s.io/api +k8s.io/apimachinery +k8s.io/klog/v2 +k8s.io/kube-openapi +k8s.io/utils +sigs.k8s.io/structured-merge-diff/v4 + +-------- License used by Dependencies +SPDX:Apache-2.0 + +-------- Dependencies Grouped by License ----- +-------- Dependency +github.com/gogo/protobuf +-------- Copyrights +Copyright (c) 2013, The GoGo Authors. +Copyright 2010 The Go Authors. +Copyright 2010 The Go Authors. +Copyright (c) 2015, The GoGo Authors. +Copyright 2016 The Go Authors. +Copyright 2015 The Go Authors. +Copyright 2011 The Go Authors. +Copyright (c) 2018, The GoGo Authors. +Copyright 2018 The Go Authors. +Copyright 2017 The Go Authors. +Copyright (c) 2016, The GoGo Authors. +Copyright 2014 The Go Authors. +Copyright 2012 The Go Authors. +Copyright 2013 The Go Authors. +Copyright (c) 2019, The GoGo Authors. +Copyright (c) 2017, The GoGo Authors. +Copyright (c) 2015, The GoGo Authors. rights reserved. + +-------- Dependency +github.com/golang/protobuf +-------- Copyrights +Copyright 2010 The Go Authors. +Copyright 2016 The Go Authors. +Copyright 2020 The Go Authors. +Copyright 2019 The Go Authors. +Copyright 2018 The Go Authors. +Copyright 2015 The Go Authors. +Copyright 2017 The Go Authors. +Copyright 2010 The Go Authors. +Copyright 2014 The Go Authors. +Copyright 2011 The Go Authors. +Copyright 2015 The Go Authors. + +-------- Dependency +github.com/google/go-cmp +-------- Copyrights +Copyright (c) 2017 The Go Authors. +Copyright 2017, The Go Authors. +Copyright 2021, The Go Authors. +Copyright 2020, The Go Authors. +Copyright 2018, The Go Authors. +Copyright 2019, The Go Authors. + +-------- Dependency +github.com/google/uuid +-------- Copyrights +Copyright (c) 2009,2014 Google Inc. +Copyright 2016 Google Inc. +Copyright 2017 Google Inc. +Copyright 2018 Google Inc. + +-------- Dependency +github.com/imdario/mergo +-------- Copyrights +Copyright (c) 2013 Dario Castañé. +Copyright (c) 2012 The Go Authors. +Copyright 2013 Dario Castañé. +Copyright 2009 The Go Authors. +Copyright 2014 Dario Castañé. + +-------- Dependency +github.com/spf13/pflag +-------- Copyrights +Copyright (c) 2012 Alex Ogier. +Copyright (c) 2012 The Go Authors. +Copyright 2009 The Go Authors. +Copyright 2012 The Go Authors. +Copyright 2010 The Go Authors. + +-------- Dependency +golang.org/x/crypto +-------- Copyrights +Copyright (c) 2009 The Go Authors. +Copyright 2015 The Go Authors. +Copyright 2016 The Go Authors. +Copyright 2017 The Go Authors. +Copyright 2018 The Go Authors. +Copyright 2019 The Go Authors. +Copyright 2011 The Go Authors. +Copyright 2010 The Go Authors. +Copyright 2012 The Go Authors. +Copyright 2013 The Go Authors. +Copyright 2014 The Go Authors. +Copyright 2020 The Go Authors. +Copyright 2009 The Go Authors. +-------- Patents +Additional IP Rights Grant (Patents) + +"This implementation" means the copyrightable works distributed by +Google as part of the Go project. + +Google hereby grants to You a perpetual, worldwide, non-exclusive, +no-charge, royalty-free, irrevocable (except as stated in this section) +patent license to make, have made, use, offer to sell, sell, import, +transfer and otherwise run, modify and propagate the contents of this +implementation of Go, where such license applies only to those patent +claims, both currently owned or controlled by Google and acquired in +the future, licensable by Google that are necessarily infringed by this +implementation of Go. This grant does not include claims that would be +infringed only as a consequence of further modification of this +implementation. If you or your agent or exclusive licensee institute or +order or agree to the institution of patent litigation against any +entity (including a cross-claim or counterclaim in a lawsuit) alleging +that this implementation of Go or any code incorporated within this +implementation of Go constitutes direct or contributory patent +infringement, or inducement of patent infringement, then any patent +rights granted to you under this License for this implementation of Go +shall terminate as of the date such litigation is filed. + + +-------- Dependency +golang.org/x/net +-------- Copyrights +Copyright (c) 2009 The Go Authors. +Copyright 2016 The Go Authors. +Copyright 2017 The Go Authors. +Copyright 2014 The Go Authors. +Copyright 2015 The Go Authors. +Copyright 2010 The Go Authors. +Copyright 2009 The Go Authors. +Copyright 2012 The Go Authors. +Copyright 2013 The Go Authors. +Copyright 2011 The Go Authors. +Copyright (C) 2009 Apple Inc. +Copyright 2018 The Go Authors. +Copyright 2021 The Go Authors. +Copyright 2020 The Go Authors. +Copyright 2019 The Go Authors. +-------- Patents +Additional IP Rights Grant (Patents) + +"This implementation" means the copyrightable works distributed by +Google as part of the Go project. + +Google hereby grants to You a perpetual, worldwide, non-exclusive, +no-charge, royalty-free, irrevocable (except as stated in this section) +patent license to make, have made, use, offer to sell, sell, import, +transfer and otherwise run, modify and propagate the contents of this +implementation of Go, where such license applies only to those patent +claims, both currently owned or controlled by Google and acquired in +the future, licensable by Google that are necessarily infringed by this +implementation of Go. This grant does not include claims that would be +infringed only as a consequence of further modification of this +implementation. If you or your agent or exclusive licensee institute or +order or agree to the institution of patent litigation against any +entity (including a cross-claim or counterclaim in a lawsuit) alleging +that this implementation of Go or any code incorporated within this +implementation of Go constitutes direct or contributory patent +infringement, or inducement of patent infringement, then any patent +rights granted to you under this License for this implementation of Go +shall terminate as of the date such litigation is filed. + + +-------- Dependency +golang.org/x/oauth2 +-------- Copyrights +Copyright (c) 2009 The Go Authors. +Copyright 2017 The oauth2 Authors. +Copyright 2015 The oauth2 Authors. +Copyright 2018 The Go Authors. +Copyright 2014 The Go Authors. +Copyright 2019 The Go Authors. +Copyright 2015 The Go Authors. +Copyright 2016 The Go Authors. +Copyright 2017 The Go Authors. +Copyright 2018 The oauth2 Authors. + +-------- Dependency +golang.org/x/sys +-------- Copyrights +Copyright (c) 2009 The Go Authors. +Copyright 2019 The Go Authors. +Copyright 2018 The Go Authors. +Copyright 2020 The Go Authors. +Copyright 2012 The Go Authors. +Copyright 2011 The Go Authors. +Copyright 2015 The Go Authors. +Copyright 2009 The Go Authors. +Copyright 2013 The Go Authors. +Copyright 2016 The Go Authors. +Copyright 2017 The Go Authors. +Copyright 2010 The Go Authors. +Copyright 2014 The Go Authors. +Copyright 2021 The Go Authors. +Copyright 2009,2010 The Go Authors. +Copyright 2017 The Go Authors. All right reserved. +-------- Patents +Additional IP Rights Grant (Patents) + +"This implementation" means the copyrightable works distributed by +Google as part of the Go project. + +Google hereby grants to You a perpetual, worldwide, non-exclusive, +no-charge, royalty-free, irrevocable (except as stated in this section) +patent license to make, have made, use, offer to sell, sell, import, +transfer and otherwise run, modify and propagate the contents of this +implementation of Go, where such license applies only to those patent +claims, both currently owned or controlled by Google and acquired in +the future, licensable by Google that are necessarily infringed by this +implementation of Go. This grant does not include claims that would be +infringed only as a consequence of further modification of this +implementation. If you or your agent or exclusive licensee institute or +order or agree to the institution of patent litigation against any +entity (including a cross-claim or counterclaim in a lawsuit) alleging +that this implementation of Go or any code incorporated within this +implementation of Go constitutes direct or contributory patent +infringement, or inducement of patent infringement, then any patent +rights granted to you under this License for this implementation of Go +shall terminate as of the date such litigation is filed. + + +-------- Dependency +golang.org/x/term +-------- Copyrights +Copyright (c) 2009 The Go Authors. +Copyright 2019 The Go Authors. +Copyright 2013 The Go Authors. +Copyright 2020 The Go Authors. +Copyright 2011 The Go Authors. +-------- Patents +Additional IP Rights Grant (Patents) + +"This implementation" means the copyrightable works distributed by +Google as part of the Go project. + +Google hereby grants to You a perpetual, worldwide, non-exclusive, +no-charge, royalty-free, irrevocable (except as stated in this section) +patent license to make, have made, use, offer to sell, sell, import, +transfer and otherwise run, modify and propagate the contents of this +implementation of Go, where such license applies only to those patent +claims, both currently owned or controlled by Google and acquired in +the future, licensable by Google that are necessarily infringed by this +implementation of Go. This grant does not include claims that would be +infringed only as a consequence of further modification of this +implementation. If you or your agent or exclusive licensee institute or +order or agree to the institution of patent litigation against any +entity (including a cross-claim or counterclaim in a lawsuit) alleging +that this implementation of Go or any code incorporated within this +implementation of Go constitutes direct or contributory patent +infringement, or inducement of patent infringement, then any patent +rights granted to you under this License for this implementation of Go +shall terminate as of the date such litigation is filed. + + +-------- Dependency +golang.org/x/text +-------- Copyrights +Copyright (c) 2009 The Go Authors. +Copyright 2014 The Go Authors. +Copyright 2016 The Go Authors. +Copyright 2015 The Go Authors. +Copyright 2017 The Go Authors. +Copyright 2012 The Go Authors. +Copyright 2013 The Go Authors. +Copyright 2019 The Go Authors. +Copyright 2018 The Go Authors. +Copyright 2009 The Go Authors. +Copyright 2011 The Go Authors. +-------- Patents +Additional IP Rights Grant (Patents) + +"This implementation" means the copyrightable works distributed by +Google as part of the Go project. + +Google hereby grants to You a perpetual, worldwide, non-exclusive, +no-charge, royalty-free, irrevocable (except as stated in this section) +patent license to make, have made, use, offer to sell, sell, import, +transfer and otherwise run, modify and propagate the contents of this +implementation of Go, where such license applies only to those patent +claims, both currently owned or controlled by Google and acquired in +the future, licensable by Google that are necessarily infringed by this +implementation of Go. This grant does not include claims that would be +infringed only as a consequence of further modification of this +implementation. If you or your agent or exclusive licensee institute or +order or agree to the institution of patent litigation against any +entity (including a cross-claim or counterclaim in a lawsuit) alleging +that this implementation of Go or any code incorporated within this +implementation of Go constitutes direct or contributory patent +infringement, or inducement of patent infringement, then any patent +rights granted to you under this License for this implementation of Go +shall terminate as of the date such litigation is filed. + + +-------- Dependency +golang.org/x/time +-------- Copyrights +Copyright (c) 2009 The Go Authors. +Copyright 2015 The Go Authors. +-------- Patents +Additional IP Rights Grant (Patents) + +"This implementation" means the copyrightable works distributed by +Google as part of the Go project. + +Google hereby grants to You a perpetual, worldwide, non-exclusive, +no-charge, royalty-free, irrevocable (except as stated in this section) +patent license to make, have made, use, offer to sell, sell, import, +transfer and otherwise run, modify and propagate the contents of this +implementation of Go, where such license applies only to those patent +claims, both currently owned or controlled by Google and acquired in +the future, licensable by Google that are necessarily infringed by this +implementation of Go. This grant does not include claims that would be +infringed only as a consequence of further modification of this +implementation. If you or your agent or exclusive licensee institute or +order or agree to the institution of patent litigation against any +entity (including a cross-claim or counterclaim in a lawsuit) alleging +that this implementation of Go or any code incorporated within this +implementation of Go constitutes direct or contributory patent +infringement, or inducement of patent infringement, then any patent +rights granted to you under this License for this implementation of Go +shall terminate as of the date such litigation is filed. + + +-------- Dependency +google.golang.org/protobuf +-------- Copyrights +Copyright (c) 2018 The Go Authors. +Copyright 2018 The Go Authors. +Copyright 2019 The Go Authors. +Copyright 2020 The Go Authors. +Copyright 2019 The Go Authors. ", +Copyright 2018 The Go Authors. ", +Copyright 2008 Google Inc. +Copyright 2021 The Go Authors. +-------- Patents +Additional IP Rights Grant (Patents) + +"This implementation" means the copyrightable works distributed by +Google as part of the Go project. + +Google hereby grants to You a perpetual, worldwide, non-exclusive, +no-charge, royalty-free, irrevocable (except as stated in this section) +patent license to make, have made, use, offer to sell, sell, import, +transfer and otherwise run, modify and propagate the contents of this +implementation of Go, where such license applies only to those patent +claims, both currently owned or controlled by Google and acquired in +the future, licensable by Google that are necessarily infringed by this +implementation of Go. This grant does not include claims that would be +infringed only as a consequence of further modification of this +implementation. If you or your agent or exclusive licensee institute or +order or agree to the institution of patent litigation against any +entity (including a cross-claim or counterclaim in a lawsuit) alleging +that this implementation of Go or any code incorporated within this +implementation of Go constitutes direct or contributory patent +infringement, or inducement of patent infringement, then any patent +rights granted to you under this License for this implementation of Go +shall terminate as of the date such litigation is filed. + + +-------- Dependency +gopkg.in/inf.v0 +-------- Copyrights +Copyright (c) 2012 Péter Surányi. Portions Copyright (c) 2009 The Go + +-------- Dependencies Summary +github.com/gogo/protobuf +github.com/golang/protobuf +github.com/google/go-cmp +github.com/google/uuid +github.com/imdario/mergo +github.com/spf13/pflag +golang.org/x/crypto +golang.org/x/net +golang.org/x/oauth2 +golang.org/x/sys +golang.org/x/term +golang.org/x/text +golang.org/x/time +google.golang.org/protobuf +gopkg.in/inf.v0 + +-------- License used by Dependencies +SPDX:BSD-3-Clause--modified-by-Google +Redistribution and use in source and binary forms, with +or without modification, are permitted provided that the following conditions +are met: + + * Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above +copyright notice, this list of conditions and the following disclaimer +in the documentation and/or other materials provided with the +distribution. + * Neither the name of Google Inc. nor the names of its +contributors may be used to endorse or promote products derived from +this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +----- Dependencies Grouped by License ------ +-------- Dependency +github.com/davecgh/go-spew +-------- Copyrights +Copyright (c) 2012-2016 Dave Collins +Copyright (c) 2015-2016 Dave Collins +Copyright (c) 2013-2016 Dave Collins +Copyright (c) 2013 Dave Collins + +-------- Dependencies Summary +github.com/davecgh/go-spew + +-------- License used by Dependencies +SPDX:ISC +Permission to use, copy, modify, and/or distribute this +software for any purpose with or without fee is hereby granted, provided that +the above copyright notice and this permission notice appear in all copies. + +THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH +REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY +AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, +INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM +LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR +OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR +PERFORMANCE OF THIS SOFTWARE. + +---- Dependencies Grouped by License ------- +-------- Dependency +github.com/form3tech-oss/jwt-go +-------- Copyrights +Copyright (c) 2012 Dave Grijalva + +-------- Dependency +github.com/json-iterator/go +-------- Copyrights +Copyright (c) 2016 json-iterator + +-------- Dependency +github.com/peterbourgon/diskv +-------- Copyrights +Copyright (c) 2011-2012 Peter Bourgon + +-------- Dependency +gopkg.in/yaml.v3 +-------- Copyrights +copyright staring in 2011 when the project was ported over: +Copyright (c) 2006-2010 Kirill Simonov +Copyright (c) 2006-2011 Kirill Simonov +Copyright (c) 2011-2019 Canonical Ltd +Copyright 2011-2016 Canonical Ltd. +-------- Notices +Copyright 2011-2016 Canonical Ltd. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + + +-------- Dependencies Summary +github.com/form3tech-oss/jwt-go +github.com/json-iterator/go +github.com/peterbourgon/diskv +gopkg.in/yaml.v3 + +-------- License used by Dependencies +SPDX:MIT +Permission is hereby granted, free of charge, to any person +obtaining a copy of this software and associated documentation files +(the "Software"), to deal in the Software without restriction, including without +limitation the rights to use, copy, modify, merge, publish, distribute, +sublicense, and/or sell copies of the Software, and to permit persons to whom +the Software is furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS +FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR +COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER +IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN +CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +------ Dependencies Grouped by License ------------ +-------- Dependency +sigs.k8s.io/yaml +-------- Copyrights +Copyright (c) 2014 Sam Ghods +Copyright (c) 2012 The Go Authors. +Copyright 2013 The Go Authors. + +-------- Dependencies Summary +sigs.k8s.io/yaml + +-------- License used by Dependencies +The MIT License (MIT) + +Copyright (c) 2014 Sam Ghods + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. + + +Copyright (c) 2012 The Go Authors. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are +met: + + * Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above +copyright notice, this list of conditions and the following disclaimer +in the documentation and/or other materials provided with the +distribution. + * Neither the name of Google Inc. nor the names of its +contributors may be used to endorse or promote products derived from +this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +=================================================================================================== +Kubernetes controller-runtime project +https://pkg.go.dev/sigs.k8s.io/controller-runtime +Apache 2.0 + +4th party dependencies +--------------------------------------------------------------------------------------- +-------- Dependency +github.com/evanphx/json-patch +-------- Copyrights +Copyright (c) 2014, Evan Phoenix + +-------- Dependencies Summary +github.com/evanphx/json-patch + +-------- License used by Dependencies +Copyright (c) 2014, Evan Phoenix + + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + +* Redistributions of source code must retain the above copyright notice, this + list of conditions and the following disclaimer. +* Redistributions in binary form must reproduce the above copyright notice + this list of conditions and the following disclaimer in the documentation + and/or other materials provided with the distribution. +* Neither the name of the Evan Phoenix nor the names of its contributors + may be used to endorse or promote products derived from this software + without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR +SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER +CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + + +----------------------- Dependencies Grouped by License ------------ +-------- Dependency +github.com/pkg/errors +-------- Copyrights +Copyright (c) 2015, Dave Cheney + +-------- Dependencies Summary +github.com/pkg/errors + +-------- License used by Dependencies +Copyright (c) 2015, Dave Cheney + + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + +* Redistributions of source code must retain the above copyright notice, this + list of conditions and the following disclaimer. + +* Redistributions in binary form must reproduce the above copyright notice, + this list of conditions and the following disclaimer in the documentation + and/or other materials provided with the distribution. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR +SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER +CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + + +----------------------- Dependencies Grouped by License ------------ +-------- Dependency +github.com/hashicorp/golang-lru +-------- Copyrights + +-------- Dependencies Summary +github.com/hashicorp/golang-lru + +-------- License used by Dependencies +Mozilla Public License, version 2.0 + +1. Definitions + +1.1. "Contributor" + + means each individual or legal entity that creates, contributes to the + creation of, or owns Covered Software. + +1.2. "Contributor Version" + + means the combination of the Contributions of others (if any) used by a + Contributor and that particular Contributor's Contribution. + +1.3. "Contribution" + + means Covered Software of a particular Contributor. + +1.4. "Covered Software" + + means Source Code Form to which the initial Contributor has attached the + notice in Exhibit A, the Executable Form of such Source Code Form, and + Modifications of such Source Code Form, in each case including portions + thereof. + +1.5. "Incompatible With Secondary Licenses" + means + + a. that the initial Contributor has attached the notice described in + Exhibit B to the Covered Software; or + + b. that the Covered Software was made available under the terms of + version 1.1 or earlier of the License, but not also under the terms of + a Secondary License. + +1.6. "Executable Form" + + means any form of the work other than Source Code Form. + +1.7. "Larger Work" + + means a work that combines Covered Software with other material, in a + separate file or files, that is not Covered Software. + +1.8. "License" + + means this document. + +1.9. "Licensable" + + means having the right to grant, to the maximum extent possible, whether + at the time of the initial grant or subsequently, any and all of the + rights conveyed by this License. + +1.10. "Modifications" + + means any of the following: + + a. any file in Source Code Form that results from an addition to, + deletion from, or modification of the contents of Covered Software; or + + b. any new file in Source Code Form that contains any Covered Software. + +1.11. "Patent Claims" of a Contributor + + means any patent claim(s), including without limitation, method, + process, and apparatus claims, in any patent Licensable by such + Contributor that would be infringed, but for the grant of the License, + by the making, using, selling, offering for sale, having made, import, + or transfer of either its Contributions or its Contributor Version. + +1.12. "Secondary License" + + means either the GNU General Public License, Version 2.0, the GNU Lesser + General Public License, Version 2.1, the GNU Affero General Public + License, Version 3.0, or any later versions of those licenses. + +1.13. "Source Code Form" + + means the form of the work preferred for making modifications. + +1.14. "You" (or "Your") + + means an individual or a legal entity exercising rights under this + License. For legal entities, "You" includes any entity that controls, is + controlled by, or is under common control with You. For purposes of this + definition, "control" means (a) the power, direct or indirect, to cause + the direction or management of such entity, whether by contract or + otherwise, or (b) ownership of more than fifty percent (50%) of the + outstanding shares or beneficial ownership of such entity. + + +2. License Grants and Conditions + +2.1. Grants + + Each Contributor hereby grants You a world-wide, royalty-free, + non-exclusive license: + + a. under intellectual property rights (other than patent or trademark) + Licensable by such Contributor to use, reproduce, make available, + modify, display, perform, distribute, and otherwise exploit its + Contributions, either on an unmodified basis, with Modifications, or + as part of a Larger Work; and + + b. under Patent Claims of such Contributor to make, use, sell, offer for + sale, have made, import, and otherwise transfer either its + Contributions or its Contributor Version. + +2.2. Effective Date + + The licenses granted in Section 2.1 with respect to any Contribution + become effective for each Contribution on the date the Contributor first + distributes such Contribution. + +2.3. Limitations on Grant Scope + + The licenses granted in this Section 2 are the only rights granted under + this License. No additional rights or licenses will be implied from the + distribution or licensing of Covered Software under this License. + Notwithstanding Section 2.1(b) above, no patent license is granted by a + Contributor: + + a. for any code that a Contributor has removed from Covered Software; or + + b. for infringements caused by: (i) Your and any other third party's + modifications of Covered Software, or (ii) the combination of its + Contributions with other software (except as part of its Contributor + Version); or + + c. under Patent Claims infringed by Covered Software in the absence of + its Contributions. + + This License does not grant any rights in the trademarks, service marks, + or logos of any Contributor (except as may be necessary to comply with + the notice requirements in Section 3.4). + +2.4. Subsequent Licenses + + No Contributor makes additional grants as a result of Your choice to + distribute the Covered Software under a subsequent version of this + License (see Section 10.2) or under the terms of a Secondary License (if + permitted under the terms of Section 3.3). + +2.5. Representation + + Each Contributor represents that the Contributor believes its + Contributions are its original creation(s) or it has sufficient rights to + grant the rights to its Contributions conveyed by this License. + +2.6. Fair Use + + This License is not intended to limit any rights You have under + applicable copyright doctrines of fair use, fair dealing, or other + equivalents. + +2.7. Conditions + + Sections 3.1, 3.2, 3.3, and 3.4 are conditions of the licenses granted in + Section 2.1. + + +3. Responsibilities + +3.1. Distribution of Source Form + + All distribution of Covered Software in Source Code Form, including any + Modifications that You create or to which You contribute, must be under + the terms of this License. You must inform recipients that the Source + Code Form of the Covered Software is governed by the terms of this + License, and how they can obtain a copy of this License. You may not + attempt to alter or restrict the recipients' rights in the Source Code + Form. + +3.2. Distribution of Executable Form + + If You distribute Covered Software in Executable Form then: + + a. such Covered Software must also be made available in Source Code Form, + as described in Section 3.1, and You must inform recipients of the + Executable Form how they can obtain a copy of such Source Code Form by + reasonable means in a timely manner, at a charge no more than the cost + of distribution to the recipient; and + + b. You may distribute such Executable Form under the terms of this + License, or sublicense it under different terms, provided that the + license for the Executable Form does not attempt to limit or alter the + recipients' rights in the Source Code Form under this License. + +3.3. Distribution of a Larger Work + + You may create and distribute a Larger Work under terms of Your choice, + provided that You also comply with the requirements of this License for + the Covered Software. If the Larger Work is a combination of Covered + Software with a work governed by one or more Secondary Licenses, and the + Covered Software is not Incompatible With Secondary Licenses, this + License permits You to additionally distribute such Covered Software + under the terms of such Secondary License(s), so that the recipient of + the Larger Work may, at their option, further distribute the Covered + Software under the terms of either this License or such Secondary + License(s). + +3.4. Notices + + You may not remove or alter the substance of any license notices + (including copyright notices, patent notices, disclaimers of warranty, or + limitations of liability) contained within the Source Code Form of the + Covered Software, except that You may alter any license notices to the + extent required to remedy known factual inaccuracies. + +3.5. Application of Additional Terms + + You may choose to offer, and to charge a fee for, warranty, support, + indemnity or liability obligations to one or more recipients of Covered + Software. However, You may do so only on Your own behalf, and not on + behalf of any Contributor. You must make it absolutely clear that any + such warranty, support, indemnity, or liability obligation is offered by + You alone, and You hereby agree to indemnify every Contributor for any + liability incurred by such Contributor as a result of warranty, support, + indemnity or liability terms You offer. You may include additional + disclaimers of warranty and limitations of liability specific to any + jurisdiction. + +4. Inability to Comply Due to Statute or Regulation + + If it is impossible for You to comply with any of the terms of this License + with respect to some or all of the Covered Software due to statute, + judicial order, or regulation then You must: (a) comply with the terms of + this License to the maximum extent possible; and (b) describe the + limitations and the code they affect. Such description must be placed in a + text file included with all distributions of the Covered Software under + this License. Except to the extent prohibited by statute or regulation, + such description must be sufficiently detailed for a recipient of ordinary + skill to be able to understand it. + +5. Termination + +5.1. The rights granted under this License will terminate automatically if You + fail to comply with any of its terms. However, if You become compliant, + then the rights granted under this License from a particular Contributor + are reinstated (a) provisionally, unless and until such Contributor + explicitly and finally terminates Your grants, and (b) on an ongoing + basis, if such Contributor fails to notify You of the non-compliance by + some reasonable means prior to 60 days after You have come back into + compliance. Moreover, Your grants from a particular Contributor are + reinstated on an ongoing basis if such Contributor notifies You of the + non-compliance by some reasonable means, this is the first time You have + received notice of non-compliance with this License from such + Contributor, and You become compliant prior to 30 days after Your receipt + of the notice. + +5.2. If You initiate litigation against any entity by asserting a patent + infringement claim (excluding declaratory judgment actions, + counter-claims, and cross-claims) alleging that a Contributor Version + directly or indirectly infringes any patent, then the rights granted to + You by any and all Contributors for the Covered Software under Section + 2.1 of this License shall terminate. + +5.3. In the event of termination under Sections 5.1 or 5.2 above, all end user + license agreements (excluding distributors and resellers) which have been + validly granted by You or Your distributors under this License prior to + termination shall survive termination. + +6. Disclaimer of Warranty + + Covered Software is provided under this License on an "as is" basis, + without warranty of any kind, either expressed, implied, or statutory, + including, without limitation, warranties that the Covered Software is free + of defects, merchantable, fit for a particular purpose or non-infringing. + The entire risk as to the quality and performance of the Covered Software + is with You. Should any Covered Software prove defective in any respect, + You (not any Contributor) assume the cost of any necessary servicing, + repair, or correction. This disclaimer of warranty constitutes an essential + part of this License. No use of any Covered Software is authorized under + this License except under this disclaimer. + +7. Limitation of Liability + + Under no circumstances and under no legal theory, whether tort (including + negligence), contract, or otherwise, shall any Contributor, or anyone who + distributes Covered Software as permitted above, be liable to You for any + direct, indirect, special, incidental, or consequential damages of any + character including, without limitation, damages for lost profits, loss of + goodwill, work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses, even if such party shall have been + informed of the possibility of such damages. This limitation of liability + shall not apply to liability for death or personal injury resulting from + such party's negligence to the extent applicable law prohibits such + limitation. Some jurisdictions do not allow the exclusion or limitation of + incidental or consequential damages, so this exclusion and limitation may + not apply to You. + +8. Litigation + + Any litigation relating to this License may be brought only in the courts + of a jurisdiction where the defendant maintains its principal place of + business and such litigation shall be governed by laws of that + jurisdiction, without reference to its conflict-of-law provisions. Nothing + in this Section shall prevent a party's ability to bring cross-claims or + counter-claims. + +9. Miscellaneous + + This License represents the complete agreement concerning the subject + matter hereof. If any provision of this License is held to be + unenforceable, such provision shall be reformed only to the extent + necessary to make it enforceable. Any law or regulation which provides that + the language of a contract shall be construed against the drafter shall not + be used to construe this License against a Contributor. + + +10. Versions of the License + +10.1. New Versions + + Mozilla Foundation is the license steward. Except as provided in Section + 10.3, no one other than the license steward has the right to modify or + publish new versions of this License. Each version will be given a + distinguishing version number. + +10.2. Effect of New Versions + + You may distribute the Covered Software under the terms of the version + of the License under which You originally received the Covered Software, + or under the terms of any subsequent version published by the license + steward. + +10.3. Modified Versions + + If you create software not governed by this License, and you want to + create a new license for such software, you may create and use a + modified version of this License if you rename the license and remove + any references to the name of the license steward (except to note that + such modified license differs from this License). + +10.4. Distributing Source Code Form that is Incompatible With Secondary + Licenses If You choose to distribute Source Code Form that is + Incompatible With Secondary Licenses under the terms of this version of + the License, the notice described in Exhibit B of this License must be + attached. + +Exhibit A - Source Code Form License Notice + + This Source Code Form is subject to the + terms of the Mozilla Public License, v. + 2.0. If a copy of the MPL was not + distributed with this file, You can + obtain one at + http://mozilla.org/MPL/2.0/. + +If it is not possible or desirable to put the notice in a particular file, +then You may include the notice in a location (such as a LICENSE file in a +relevant directory) where a recipient would be likely to look for such a +notice. + +You may add additional accurate notices of copyright ownership. + +Exhibit B - "Incompatible With Secondary Licenses" Notice + + This Source Code Form is "Incompatible + With Secondary Licenses", as defined by + the Mozilla Public License, v. 2.0. + + +----------------------- Dependencies Grouped by License ------------ +-------- Dependency +cloud.google.com/go +-------- Copyrights +Copyright 2019 Google LLC +Copyright 2018 Google LLC +Copyright 2017 Google LLC +Copyright 2015 Google LLC +Copyright 2016 Google LLC +Copyright 2014 Google LLC +Copyright 2018 Google Inc. +Copyright (c) 1996-1998 John D. Polstra. +Copyright (c) 2001 David E. O'Brien +Portions Copyright 2018 Google LLC. +Copyright 2017, Google LLC +Copyright 2017, Google Inc. + +-------- Dependency +github.com/go-logr/logr +-------- Copyrights + +-------- Dependency +github.com/go-logr/zapr +-------- Copyrights +Copyright 2018 Solly Ross + +-------- Dependency +github.com/golang/groupcache +-------- Copyrights +Copyright 2012 Google Inc. +Copyright 2013 Google Inc. + +-------- Dependency +github.com/google/gofuzz +-------- Copyrights +Copyright 2014 Google Inc. + +-------- Dependency +github.com/googleapis/gnostic +-------- Copyrights +Copyright 2017 Google Inc. +Copyright 2017, Google Inc. +Copyright 2018 Google Inc. +Copyright 2017 Google Inc. \n" + + +-------- Dependency +github.com/matttproud/golang_protobuf_extensions +-------- Copyrights +Copyright 2012 Matt T. Proud (matt.proud@gmail.com) +Copyright 2013 Matt T. Proud +Copyright 2016 Matt T. Proud +-------- Notices +Copyright 2012 Matt T. Proud (matt.proud@gmail.com) + + +-------- Dependency +github.com/modern-go/concurrent +-------- Copyrights + +-------- Dependency +github.com/modern-go/reflect2 +-------- Copyrights + +-------- Dependency +github.com/prometheus/client_golang +-------- Copyrights +Copyright 2018 The Prometheus Authors +Copyright 2012-2015 The Prometheus Authors +Copyright 2013-2015 Blake Mizerany, Björn Rabenstein +Copyright 2010 The Go Authors +Copyright 2013 Matt T. Proud +Copyright 2015 The Prometheus Authors +Copyright 2017 The Prometheus Authors +Copyright 2019 The Prometheus Authors +Copyright 2014 The Prometheus Authors +Copyright 2016 The Prometheus Authors +Copyright (c) 2013, The Prometheus Authors +-------- Notices +Prometheus instrumentation library for Go applications +Copyright 2012-2015 The Prometheus Authors + +This product includes software developed at +SoundCloud Ltd. (http://soundcloud.com/). + + +The following components are included in this product: + +perks - a fork of https://github.com/bmizerany/perks +https://github.com/beorn7/perks +Copyright 2013-2015 Blake Mizerany, Björn Rabenstein +See https://github.com/beorn7/perks/blob/master/README.md for license details. + +Go support for Protocol Buffers - Google's data interchange format +http://github.com/golang/protobuf/ +Copyright 2010 The Go Authors +See source code for license details. + +Support for streaming Protocol Buffer messages for the Go language (golang). +https://github.com/matttproud/golang_protobuf_extensions +Copyright 2013 Matt T. Proud +Licensed under the Apache License, Version 2.0 + + +-------- Dependency +github.com/prometheus/client_model +-------- Copyrights +Copyright 2013 Prometheus Team +Copyright 2012-2015 The Prometheus Authors +-------- Notices +Data model artifacts for Prometheus. +Copyright 2012-2015 The Prometheus Authors + +This product includes software developed at +SoundCloud Ltd. (http://soundcloud.com/). + + +-------- Dependency +github.com/prometheus/common +-------- Copyrights +Copyright 2018 The Prometheus Authors +Copyright 2015 The Prometheus Authors +Copyright 2016 The Prometheus Authors +Copyright 2014 The Prometheus Authors +Copyright (c) 2011, Open Knowledge Foundation Ltd. +Copyright 2013 The Prometheus Authors +Copyright 2019 The Prometheus Authors +Copyright 2017 The Prometheus Authors +-------- Notices +Common libraries shared by Prometheus Go components. +Copyright 2015 The Prometheus Authors + +This product includes software developed at +SoundCloud Ltd. (http://soundcloud.com/). + + +-------- Dependency +github.com/prometheus/procfs +-------- Copyrights +Copyright 2018 The Prometheus Authors +Copyright 2014-2015 The Prometheus Authors +Copyright 2019 The Prometheus Authors +Copyright 2017 The Prometheus Authors +Copyright 2014 Prometheus Team +Copyright 2020 The Prometheus Authors +Copyright 2017 Prometheus Team +-------- Notices +procfs provides functions to retrieve system, kernel and process +metrics from the pseudo-filesystem proc. + +Copyright 2014-2015 The Prometheus Authors + +This product includes software developed at +SoundCloud Ltd. (http://soundcloud.com/). + + +-------- Dependency +gomodules.xyz/jsonpatch/v2 +-------- Copyrights + +-------- Dependency +gopkg.in/yaml.v2 +-------- Copyrights +Copyright (c) 2006 Kirill Simonov +Copyright 2011-2016 Canonical Ltd. +-------- Notices +Copyright 2011-2016 Canonical Ltd. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + + +-------- Dependency +k8s.io/api +-------- Copyrights +Copyright 2019 The Kubernetes Authors. +Copyright 2017 The Kubernetes Authors. +Copyright 2016 The Kubernetes Authors. +Copyright 2018 The Kubernetes Authors. +Copyright 2015 The Kubernetes Authors. +Copyright 2020 The Kubernetes Authors. + +-------- Dependency +k8s.io/apiextensions-apiserver +-------- Copyrights +Copyright 2019 The Kubernetes Authors. +Copyright 2017 The Kubernetes Authors. +Copyright 2018 The Kubernetes Authors. +Copyright 2016 The Kubernetes Authors. + +-------- Dependency +k8s.io/apimachinery +-------- Copyrights +Copyright 2017 The Kubernetes Authors. +Copyright 2019 The Kubernetes Authors. +Copyright 2014 The Kubernetes Authors. +Copyright 2015 The Kubernetes Authors. +Copyright 2016 The Kubernetes Authors. +Copyright 2018 The Kubernetes Authors. +Copyright 2013 The Go Authors. +Copyright 2009 The Go Authors. + +-------- Dependency +k8s.io/client-go +-------- Copyrights +Copyright 2017 The Kubernetes Authors. +Copyright 2016 The Kubernetes Authors. +Copyright 2014 The Kubernetes Authors. +Copyright 2018 The Kubernetes Authors. +Copyright 2019 The Kubernetes Authors. +Copyright 2015 The Kubernetes Authors. +Copyright 2020 The Kubernetes Authors. + +-------- Dependency +k8s.io/klog +-------- Copyrights +Copyright 2013 Google Inc. + +-------- Dependency +k8s.io/klog/v2 +-------- Copyrights +Copyright 2013 Google Inc. + +-------- Dependency +k8s.io/kube-openapi +-------- Copyrights +Copyright 2018 The Kubernetes Authors. +Copyright 2017 The Kubernetes Authors. +Copyright 2019 The Kubernetes Authors. +Copyright 2016 The Kubernetes Authors. + +-------- Dependency +k8s.io/utils +-------- Copyrights +Copyright 2018 The Kubernetes Authors. +Copyright 2017 The Kubernetes Authors. +Copyright 2014 The Kubernetes Authors. +Copyright 2015 The Kubernetes Authors. +Copyright 2016 The Kubernetes Authors. +Copyright 2019 The Kubernetes Authors. +Copyright (c) 2009 The Go Authors. +Copyright 2020 The Kubernetes Authors. +Copyright 2010 The Go Authors. +Copyright (c) 2012 The Go Authors. +Copyright 2009 The Go Authors. + +-------- Dependency +sigs.k8s.io/structured-merge-diff/v3 +-------- Copyrights +Copyright 2018 The Kubernetes Authors. +Copyright 2019 The Kubernetes Authors. +Copyright 2020 The Kubernetes Authors. + +-------- Dependencies Summary +cloud.google.com/go +github.com/go-logr/logr +github.com/go-logr/zapr +github.com/golang/groupcache +github.com/google/gofuzz +github.com/googleapis/gnostic +github.com/matttproud/golang_protobuf_extensions +github.com/modern-go/concurrent +github.com/modern-go/reflect2 +github.com/prometheus/client_golang +github.com/prometheus/client_model +github.com/prometheus/common +github.com/prometheus/procfs +gomodules.xyz/jsonpatch/v2 +gopkg.in/yaml.v2 +k8s.io/api +k8s.io/apiextensions-apiserver +k8s.io/apimachinery +k8s.io/client-go +k8s.io/klog +k8s.io/klog/v2 +k8s.io/kube-openapi +k8s.io/utils +sigs.k8s.io/structured-merge-diff/v3 + +-------- License used by Dependencies +SPDX:Apache-2.0 + +-------- Dependencies Grouped by License ------------ +-------- Dependency +github.com/fsnotify/fsnotify +-------- Copyrights +Copyright (c) 2012 The Go Authors. +Copyright (c) 2012-2019 fsnotify Authors. +Copyright 2010 The Go Authors. +Copyright 2012 The Go Authors. +Copyright 2016 The Go Authors. +Copyright 2015 The Go Authors. +Copyright 2013 The Go Authors. +Copyright 2011 The Go Authors. + +-------- Dependency +github.com/gogo/protobuf +-------- Copyrights +Copyright (c) 2013, The GoGo Authors. +Copyright 2010 The Go Authors. +Copyright 2010 The Go Authors. +Copyright (c) 2015, The GoGo Authors. +Copyright 2016 The Go Authors. +Copyright 2015 The Go Authors. +Copyright 2011 The Go Authors. +Copyright (c) 2018, The GoGo Authors. +Copyright 2018 The Go Authors. +Copyright 2017 The Go Authors. +Copyright (c) 2016, The GoGo Authors. +Copyright 2014 The Go Authors. +Copyright 2012 The Go Authors. +Copyright 2013 The Go Authors. +Copyright (c) 2019, The GoGo Authors. +Copyright (c) 2017, The GoGo Authors. +Copyright (c) 2015, The GoGo Authors. rights reserved. + +-------- Dependency +github.com/golang/protobuf +-------- Copyrights +Copyright 2010 The Go Authors. +Copyright 2016 The Go Authors. +Copyright 2020 The Go Authors. +Copyright 2019 The Go Authors. +Copyright 2018 The Go Authors. +Copyright 2015 The Go Authors. +Copyright 2017 The Go Authors. +Copyright 2010 The Go Authors. +Copyright 2014 The Go Authors. +Copyright 2011 The Go Authors. +Copyright 2015 The Go Authors. + +-------- Dependency +github.com/google/go-cmp +-------- Copyrights +Copyright (c) 2017 The Go Authors. +Copyright 2017, The Go Authors. +Copyright 2018, The Go Authors. +Copyright 2019, The Go Authors. + +-------- Dependency +github.com/google/uuid +-------- Copyrights +Copyright (c) 2009,2014 Google Inc. +Copyright 2016 Google Inc. +Copyright 2017 Google Inc. +Copyright 2018 Google Inc. + +-------- Dependency +github.com/imdario/mergo +-------- Copyrights +Copyright (c) 2013 Dario Castañé. +Copyright (c) 2012 The Go Authors. +Copyright 2013 Dario Castañé. +Copyright 2009 The Go Authors. +Copyright 2014 Dario Castañé. + +-------- Dependency +github.com/spf13/pflag +-------- Copyrights +Copyright (c) 2012 Alex Ogier. +Copyright (c) 2012 The Go Authors. +Copyright 2009 The Go Authors. +Copyright 2012 The Go Authors. +Copyright 2010 The Go Authors. + +-------- Dependency +golang.org/x/crypto +-------- Copyrights +Copyright (c) 2009 The Go Authors. +Copyright 2015 The Go Authors. +Copyright 2016 The Go Authors. +Copyright 2017 The Go Authors. +Copyright 2018 The Go Authors. +Copyright 2019 The Go Authors. +Copyright 2011 The Go Authors. +Copyright 2010 The Go Authors. +Copyright 2012 The Go Authors. +Copyright 2013 The Go Authors. +Copyright 2014 The Go Authors. +Copyright 2009 The Go Authors. +-------- Patents +Additional IP Rights Grant (Patents) + +"This implementation" means the copyrightable works distributed by +Google as part of the Go project. + +Google hereby grants to You a perpetual, worldwide, non-exclusive, +no-charge, royalty-free, irrevocable (except as stated in this section) +patent license to make, have made, use, offer to sell, sell, import, +transfer and otherwise run, modify and propagate the contents of this +implementation of Go, where such license applies only to those patent +claims, both currently owned or controlled by Google and acquired in +the future, licensable by Google that are necessarily infringed by this +implementation of Go. This grant does not include claims that would be +infringed only as a consequence of further modification of this +implementation. If you or your agent or exclusive licensee institute or +order or agree to the institution of patent litigation against any +entity (including a cross-claim or counterclaim in a lawsuit) alleging +that this implementation of Go or any code incorporated within this +implementation of Go constitutes direct or contributory patent +infringement, or inducement of patent infringement, then any patent +rights granted to you under this License for this implementation of Go +shall terminate as of the date such litigation is filed. + + +-------- Dependency +golang.org/x/net +-------- Copyrights +Copyright (c) 2009 The Go Authors. +Copyright 2016 The Go Authors. +Copyright 2017 The Go Authors. +Copyright 2014 The Go Authors. +Copyright 2015 The Go Authors. +Copyright 2010 The Go Authors. +Copyright 2009 The Go Authors. +Copyright 2012 The Go Authors. +Copyright 2013 The Go Authors. +Copyright 2011 The Go Authors. +Copyright (C) 2009 Apple Inc. +Copyright 2018 The Go Authors. +Copyright 2019 The Go Authors. +-------- Patents +Additional IP Rights Grant (Patents) + +"This implementation" means the copyrightable works distributed by +Google as part of the Go project. + +Google hereby grants to You a perpetual, worldwide, non-exclusive, +no-charge, royalty-free, irrevocable (except as stated in this section) +patent license to make, have made, use, offer to sell, sell, import, +transfer and otherwise run, modify and propagate the contents of this +implementation of Go, where such license applies only to those patent +claims, both currently owned or controlled by Google and acquired in +the future, licensable by Google that are necessarily infringed by this +implementation of Go. This grant does not include claims that would be +infringed only as a consequence of further modification of this +implementation. If you or your agent or exclusive licensee institute or +order or agree to the institution of patent litigation against any +entity (including a cross-claim or counterclaim in a lawsuit) alleging +that this implementation of Go or any code incorporated within this +implementation of Go constitutes direct or contributory patent +infringement, or inducement of patent infringement, then any patent +rights granted to you under this License for this implementation of Go +shall terminate as of the date such litigation is filed. + + +-------- Dependency +golang.org/x/oauth2 +-------- Copyrights +Copyright (c) 2009 The Go Authors. +Copyright 2017 The oauth2 Authors. +Copyright 2015 The oauth2 Authors. +Copyright 2018 The Go Authors. +Copyright 2014 The Go Authors. +Copyright 2015 The Go Authors. +Copyright 2016 The Go Authors. +Copyright 2017 The Go Authors. +Copyright 2018 The oauth2 Authors. + +-------- Dependency +golang.org/x/sys +-------- Copyrights +Copyright (c) 2009 The Go Authors. +Copyright 2019 The Go Authors. +Copyright 2018 The Go Authors. +Copyright 2020 The Go Authors. +Copyright 2012 The Go Authors. +Copyright 2011 The Go Authors. +Copyright 2015 The Go Authors. +Copyright 2009 The Go Authors. +Copyright 2013 The Go Authors. +Copyright 2016 The Go Authors. +Copyright 2017 The Go Authors. +Copyright 2010 The Go Authors. +Copyright 2014 The Go Authors. +Copyright 2009,2010 The Go Authors. +Copyright 2017 The Go Authors. All right reserved. +-------- Patents +Additional IP Rights Grant (Patents) + +"This implementation" means the copyrightable works distributed by +Google as part of the Go project. + +Google hereby grants to You a perpetual, worldwide, non-exclusive, +no-charge, royalty-free, irrevocable (except as stated in this section) +patent license to make, have made, use, offer to sell, sell, import, +transfer and otherwise run, modify and propagate the contents of this +implementation of Go, where such license applies only to those patent +claims, both currently owned or controlled by Google and acquired in +the future, licensable by Google that are necessarily infringed by this +implementation of Go. This grant does not include claims that would be +infringed only as a consequence of further modification of this +implementation. If you or your agent or exclusive licensee institute or +order or agree to the institution of patent litigation against any +entity (including a cross-claim or counterclaim in a lawsuit) alleging +that this implementation of Go or any code incorporated within this +implementation of Go constitutes direct or contributory patent +infringement, or inducement of patent infringement, then any patent +rights granted to you under this License for this implementation of Go +shall terminate as of the date such litigation is filed. + + +-------- Dependency +golang.org/x/text +-------- Copyrights +Copyright (c) 2009 The Go Authors. +Copyright 2014 The Go Authors. +Copyright 2016 The Go Authors. +Copyright 2015 The Go Authors. +Copyright 2017 The Go Authors. +Copyright 2012 The Go Authors. +Copyright 2013 The Go Authors. +Copyright 2018 The Go Authors. +Copyright 2009 The Go Authors. +Copyright 2019 The Go Authors. +Copyright 2011 The Go Authors. +-------- Patents +Additional IP Rights Grant (Patents) + +"This implementation" means the copyrightable works distributed by +Google as part of the Go project. + +Google hereby grants to You a perpetual, worldwide, non-exclusive, +no-charge, royalty-free, irrevocable (except as stated in this section) +patent license to make, have made, use, offer to sell, sell, import, +transfer and otherwise run, modify and propagate the contents of this +implementation of Go, where such license applies only to those patent +claims, both currently owned or controlled by Google and acquired in +the future, licensable by Google that are necessarily infringed by this +implementation of Go. This grant does not include claims that would be +infringed only as a consequence of further modification of this +implementation. If you or your agent or exclusive licensee institute or +order or agree to the institution of patent litigation against any +entity (including a cross-claim or counterclaim in a lawsuit) alleging +that this implementation of Go or any code incorporated within this +implementation of Go constitutes direct or contributory patent +infringement, or inducement of patent infringement, then any patent +rights granted to you under this License for this implementation of Go +shall terminate as of the date such litigation is filed. + + +-------- Dependency +golang.org/x/time +-------- Copyrights +Copyright (c) 2009 The Go Authors. +Copyright 2015 The Go Authors. +-------- Patents +Additional IP Rights Grant (Patents) + +"This implementation" means the copyrightable works distributed by +Google as part of the Go project. + +Google hereby grants to You a perpetual, worldwide, non-exclusive, +no-charge, royalty-free, irrevocable (except as stated in this section) +patent license to make, have made, use, offer to sell, sell, import, +transfer and otherwise run, modify and propagate the contents of this +implementation of Go, where such license applies only to those patent +claims, both currently owned or controlled by Google and acquired in +the future, licensable by Google that are necessarily infringed by this +implementation of Go. This grant does not include claims that would be +infringed only as a consequence of further modification of this +implementation. If you or your agent or exclusive licensee institute or +order or agree to the institution of patent litigation against any +entity (including a cross-claim or counterclaim in a lawsuit) alleging +that this implementation of Go or any code incorporated within this +implementation of Go constitutes direct or contributory patent +infringement, or inducement of patent infringement, then any patent +rights granted to you under this License for this implementation of Go +shall terminate as of the date such litigation is filed. + + +-------- Dependency +golang.org/x/xerrors +-------- Copyrights +Copyright (c) 2019 The Go Authors. +Copyright 2018 The Go Authors. +Copyright 2019 The Go Authors. +Copyright 2011 The Go Authors. +Copyright 2012 The Go Authors. +-------- Patents +Additional IP Rights Grant (Patents) + +"This implementation" means the copyrightable works distributed by +Google as part of the Go project. + +Google hereby grants to You a perpetual, worldwide, non-exclusive, +no-charge, royalty-free, irrevocable (except as stated in this section) +patent license to make, have made, use, offer to sell, sell, import, +transfer and otherwise run, modify and propagate the contents of this +implementation of Go, where such license applies only to those patent +claims, both currently owned or controlled by Google and acquired in +the future, licensable by Google that are necessarily infringed by this +implementation of Go. This grant does not include claims that would be +infringed only as a consequence of further modification of this +implementation. If you or your agent or exclusive licensee institute or +order or agree to the institution of patent litigation against any +entity (including a cross-claim or counterclaim in a lawsuit) alleging +that this implementation of Go or any code incorporated within this +implementation of Go constitutes direct or contributory patent +infringement, or inducement of patent infringement, then any patent +rights granted to you under this License for this implementation of Go +shall terminate as of the date such litigation is filed. + + +-------- Dependency +google.golang.org/protobuf +-------- Copyrights +Copyright (c) 2018 The Go Authors. +Copyright 2018 The Go Authors. +Copyright 2019 The Go Authors. +Copyright 2020 The Go Authors. +Copyright 2019 The Go Authors. ", +Copyright 2018 The Go Authors. ", +Copyright 2008 Google Inc. +-------- Patents +Additional IP Rights Grant (Patents) + +"This implementation" means the copyrightable works distributed by +Google as part of the Go project. + +Google hereby grants to You a perpetual, worldwide, non-exclusive, +no-charge, royalty-free, irrevocable (except as stated in this section) +patent license to make, have made, use, offer to sell, sell, import, +transfer and otherwise run, modify and propagate the contents of this +implementation of Go, where such license applies only to those patent +claims, both currently owned or controlled by Google and acquired in +the future, licensable by Google that are necessarily infringed by this +implementation of Go. This grant does not include claims that would be +infringed only as a consequence of further modification of this +implementation. If you or your agent or exclusive licensee institute or +order or agree to the institution of patent litigation against any +entity (including a cross-claim or counterclaim in a lawsuit) alleging +that this implementation of Go or any code incorporated within this +implementation of Go constitutes direct or contributory patent +infringement, or inducement of patent infringement, then any patent +rights granted to you under this License for this implementation of Go +shall terminate as of the date such litigation is filed. + + +-------- Dependency +gopkg.in/inf.v0 +-------- Copyrights +Copyright (c) 2012 Péter Surányi. Portions Copyright (c) 2009 The Go + +-------- Dependencies Summary +github.com/fsnotify/fsnotify +github.com/gogo/protobuf +github.com/golang/protobuf +github.com/google/go-cmp +github.com/google/uuid +github.com/imdario/mergo +github.com/spf13/pflag +golang.org/x/crypto +golang.org/x/net +golang.org/x/oauth2 +golang.org/x/sys +golang.org/x/text +golang.org/x/time +golang.org/x/xerrors +google.golang.org/protobuf +gopkg.in/inf.v0 + +-------- License used by Dependencies +SPDX:BSD-3-Clause--modified-by-Google +Redistribution and use in source and binary forms, with +or without modification, are permitted provided that the following conditions +are met: + + * Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above +copyright notice, this list of conditions and the following disclaimer +in the documentation and/or other materials provided with the +distribution. + * Neither the name of Google Inc. nor the names of its +contributors may be used to endorse or promote products derived from +this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +---- Dependencies Grouped by License ------------ +-------- Dependency +github.com/davecgh/go-spew +-------- Copyrights +Copyright (c) 2012-2016 Dave Collins +Copyright (c) 2015-2016 Dave Collins +Copyright (c) 2013-2016 Dave Collins +Copyright (c) 2013 Dave Collins + +-------- Dependencies Summary +github.com/davecgh/go-spew + +-------- License used by Dependencies +SPDX:ISC +Permission to use, copy, modify, and/or distribute this +software for any purpose with or without fee is hereby granted, provided that +the above copyright notice and this permission notice appear in all copies. + +THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH +REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY +AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, +INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM +LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR +OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR +PERFORMANCE OF THIS SOFTWARE. + +----------------------- Dependencies Grouped by License ------------ +-------- Dependency +github.com/beorn7/perks +-------- Copyrights +Copyright (C) 2013 Blake Mizerany + +-------- Dependency +github.com/json-iterator/go +-------- Copyrights +Copyright (c) 2016 json-iterator + +-------- Dependency +github.com/nxadm/tail +-------- Copyrights +© Copyright 2015 Hewlett Packard Enterprise Development LP +Copyright (c) 2014 ActiveState +Copyright (c) 2015 HPE Software Inc. +Copyright (c) 2013 ActiveState Software Inc. +Copyright (C) 2013 99designs + +-------- Dependency +github.com/onsi/ginkgo +-------- Copyrights +Copyright (c) 2013-2014 Onsi Fakhouri +Copyright (c) 2016 Yasuhiro Matsumoto + +-------- Dependency +github.com/onsi/gomega +-------- Copyrights +Copyright (c) 2013-2014 Onsi Fakhouri +Copyright (c) 2014 Amit Kumar Gupta + +-------- Dependency +go.uber.org/atomic +-------- Copyrights +Copyright (c) 2016 Uber Technologies, Inc. + +-------- Dependency +go.uber.org/multierr +-------- Copyrights +Copyright (c) 2017 Uber Technologies, Inc. + +-------- Dependency +go.uber.org/zap +-------- Copyrights +Copyright (c) 2016-2017 Uber Technologies, Inc. +Copyright (c) 2016 Uber Technologies, Inc. +Copyright (c) 2017 Uber Technologies, Inc. +Copyright (c) 2019 Uber Technologies, Inc. +Copyright (c) 2016, 2017 Uber Technologies, Inc. +Copyright (c) 2018 Uber Technologies, Inc. + +-------- Dependencies Summary +github.com/beorn7/perks +github.com/json-iterator/go +github.com/nxadm/tail +github.com/onsi/ginkgo +github.com/onsi/gomega +go.uber.org/atomic +go.uber.org/multierr +go.uber.org/zap + +-------- License used by Dependencies +SPDX:MIT +Permission is hereby granted, free of charge, to any person +obtaining a copy of this software and associated documentation files +(the "Software"), to deal in the Software without restriction, including without +limitation the rights to use, copy, modify, merge, publish, distribute, +sublicense, and/or sell copies of the Software, and to permit persons to whom +the Software is furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS +FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR +COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER +IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN +CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +----- Dependencies Grouped by License ------------ +-------- Dependency +sigs.k8s.io/yaml +-------- Copyrights +Copyright (c) 2014 Sam Ghods +Copyright (c) 2012 The Go Authors. +Copyright 2013 The Go Authors. + +-------- Dependencies Summary +sigs.k8s.io/yaml + +-------- License used by Dependencies +The MIT License (MIT) + +Copyright (c) 2014 Sam Ghods + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. + + +Copyright (c) 2012 The Go Authors. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are +met: + + * Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above +copyright notice, this list of conditions and the following disclaimer +in the documentation and/or other materials provided with the +distribution. + * Neither the name of Google Inc. nor the names of its +contributors may be used to endorse or promote products derived from +this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + + +---- Dependencies Grouped by License ------------ +-------- Dependency +gopkg.in/tomb.v1 +-------- Copyrights +Copyright (c) 2010-2011 - Gustavo Niemeyer +Copyright (c) 2011 - Gustavo Niemeyer + +-------- Dependencies Summary +gopkg.in/tomb.v1 + +-------- License used by Dependencies +tomb - support for clean goroutine termination in Go. + +Copyright (c) 2010-2011 - Gustavo Niemeyer + + + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + + * Redistributions of source code must retain the above copyright notice, + this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright notice, + this list of conditions and the following disclaimer in the documentation + and/or other materials provided with the distribution. + * Neither the name of the copyright holder nor the names of its + contributors may be used to endorse or promote products derived from + this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR +CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, +EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR +PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF +LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING +NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS +SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +=================================================================================================== +YAML support for the Go language +https://pkg.go.dev/gopkg.in/yaml.v2 +Apache 2.0 + +4th party dependencies +--------------------------------------------------------------------------------------- +NOTICE FILE: + +The following files were ported to Go from C files of libyaml, and thus +are still covered by their original copyright and license: + + apic.go + emitterc.go + parserc.go + readerc.go + scannerc.go + writerc.go + yamlh.go + yamlprivateh.go + +Copyright (c) 2006 Kirill Simonov + +Permission is hereby granted, free of charge, to any person obtaining a copy of +this software and associated documentation files (the "Software"), to deal in +the Software without restriction, including without limitation the rights to +use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies +of the Software, and to permit persons to whom the Software is furnished to do +so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. + +=================================================================================================== +YAML marshaling and unmarshaling support for Go +https://pkg.go.dev/sigs.k8s.io/yaml +DUal license: BSD-3-Clause, MIT + +The MIT License (MIT) + +Copyright (c) 2014 Sam Ghods + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. + + +Copyright (c) 2012 The Go Authors. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are +met: + + * Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above +copyright notice, this list of conditions and the following disclaimer +in the documentation and/or other materials provided with the +distribution. + * Neither the name of Google Inc. nor the names of its +contributors may be used to endorse or promote products derived from +this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +4th party dependencies +--------------------------------------------------------------------------------------- +-------- Dependency +gopkg.in/yaml.v2 +-------- Copyrights +Copyright (c) 2006 Kirill Simonov +-------- Notices +Copyright 2011-2016 Canonical Ltd. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + + +======== Dependencies Summary +gopkg.in/yaml.v2 + +======== License used by Dependencies +SPDX:Apache-2.0 + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + +The following files were ported to Go from C files of libyaml, and thus +are still covered by their original copyright and license: + + apic.go + emitterc.go + parserc.go + readerc.go + scannerc.go + writerc.go + yamlh.go + yamlprivateh.go + +Copyright (c) 2006 Kirill Simonov + +Permission is hereby granted, free of charge, to any person obtaining a copy of +this software and associated documentation files (the "Software"), to deal in +the Software without restriction, including without limitation the rights to +use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies +of the Software, and to permit persons to whom the Software is furnished to do +so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. + +ATTRIBUTION-HELPER-GENERATED, with additional LICENSE.libyaml text for dep yaml.v2 added manually: +License file based on go.mod with md5 sum: 07d719758b249492cf0e58171fd77fd3 +=================================================================================================== +Ginkgo +github.com/onsi/ginkgo +MIT License +Copyright (c) 2013-2014 Onsi Fakhouri + +Permission is hereby granted, free of charge, to any person obtaining +a copy of this software and associated documentation files (the +"Software"), to deal in the Software without restriction, including +without limitation the rights to use, copy, modify, merge, publish, +distribute, sublicense, and/or sell copies of the Software, and to +permit persons to whom the Software is furnished to do so, subject to +the following conditions: + +The above copyright notice and this permission notice shall be +included in all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, +EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF +MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE +LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION +WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +4th party dependencies +--------------------------------------------------------------------------------------- +-------- Dependency +gopkg.in/yaml.v2 +-------- Copyrights +Copyright (c) 2006 Kirill Simonov +-------- Notices +Copyright 2011-2016 Canonical Ltd. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + + +======== Dependencies Summary +gopkg.in/yaml.v2 + +======== License used by Dependencies +SPDX:Apache-2.0 + +===== Dependencies Grouped by License ======== +-------- Dependency +github.com/golang/protobuf +-------- Copyrights +Copyright 2010 The Go Authors. +# Copyright 2010 The Go Authors. +# Copyright 2016 The Go Authors. +// Copyright 2016 The Go Authors. +// Copyright 2015 The Go Authors. +// Copyright 2010 The Go Authors. +// Copyright 2011 The Go Authors. +// Copyright 2017 The Go Authors. +// Copyright 2014 The Go Authors. +// Copyright 2012 The Go Authors. +// Copyright 2013 The Go Authors. + +-------- Dependency +golang.org/x/net +-------- Copyrights +Copyright (c) 2009 The Go Authors. +// Copyright 2016 The Go Authors. +// Copyright 2017 The Go Authors. +// Copyright 2014 The Go Authors. +// Copyright 2015 The Go Authors. +// Copyright 2010 The Go Authors. +// Copyright 2009 The Go Authors. +// Copyright 2012 The Go Authors. +// Copyright 2013 The Go Authors. +// Copyright 2011 The Go Authors. +// Copyright 2018 The Go Authors. +# Copyright 2018 The Go Authors. + Copyright 2009 The Go Authors. +-------- Patents +Additional IP Rights Grant (Patents) + +"This implementation" means the copyrightable works distributed by +Google as part of the Go project. + +Google hereby grants to You a perpetual, worldwide, non-exclusive, +no-charge, royalty-free, irrevocable (except as stated in this section) +patent license to make, have made, use, offer to sell, sell, import, +transfer and otherwise run, modify and propagate the contents of this +implementation of Go, where such license applies only to those patent +claims, both currently owned or controlled by Google and acquired in +the future, licensable by Google that are necessarily infringed by this +implementation of Go. This grant does not include claims that would be +infringed only as a consequence of further modification of this +implementation. If you or your agent or exclusive licensee institute or +order or agree to the institution of patent litigation against any +entity (including a cross-claim or counterclaim in a lawsuit) alleging +that this implementation of Go or any code incorporated within this +implementation of Go constitutes direct or contributory patent +infringement, or inducement of patent infringement, then any patent +rights granted to you under this License for this implementation of Go +shall terminate as of the date such litigation is filed. + + +-------- Dependency +golang.org/x/sys +-------- Copyrights +Copyright (c) 2009 The Go Authors. +// Copyright 2019 The Go Authors. +// Copyright 2018 The Go Authors. +// Copyright 2012 The Go Authors. +// Copyright 2011 The Go Authors. +// Copyright 2015 The Go Authors. +// Copyright 2009 The Go Authors. +// Copyright 2013 The Go Authors. +// Copyright 2016 The Go Authors. +// Copyright 2017 The Go Authors. +// Copyright 2010 The Go Authors. +// Copyright 2014 The Go Authors. +// Copyright 2009,2010 The Go Authors. +// Copyright 2017 The Go Authors. All right reserved. +-------- Patents +Additional IP Rights Grant (Patents) + +"This implementation" means the copyrightable works distributed by +Google as part of the Go project. + +Google hereby grants to You a perpetual, worldwide, non-exclusive, +no-charge, royalty-free, irrevocable (except as stated in this section) +patent license to make, have made, use, offer to sell, sell, import, +transfer and otherwise run, modify and propagate the contents of this +implementation of Go, where such license applies only to those patent +claims, both currently owned or controlled by Google and acquired in +the future, licensable by Google that are necessarily infringed by this +implementation of Go. This grant does not include claims that would be +infringed only as a consequence of further modification of this +implementation. If you or your agent or exclusive licensee institute or +order or agree to the institution of patent litigation against any +entity (including a cross-claim or counterclaim in a lawsuit) alleging +that this implementation of Go or any code incorporated within this +implementation of Go constitutes direct or contributory patent +infringement, or inducement of patent infringement, then any patent +rights granted to you under this License for this implementation of Go +shall terminate as of the date such litigation is filed. + + +-------- Dependency +golang.org/x/text +-------- Copyrights +Copyright (c) 2009 The Go Authors. +// Copyright 2014 The Go Authors. +// Copyright 2016 The Go Authors. +// Copyright 2015 The Go Authors. +// Copyright 2017 The Go Authors. +// Copyright 2012 The Go Authors. +// Copyright 2013 The Go Authors. +# Copyright 2012 The Go Authors. +# Copyright 2013 The Go Authors. +// Copyright 2011 The Go Authors. +-------- Patents +Additional IP Rights Grant (Patents) + +"This implementation" means the copyrightable works distributed by +Google as part of the Go project. + +Google hereby grants to You a perpetual, worldwide, non-exclusive, +no-charge, royalty-free, irrevocable (except as stated in this section) +patent license to make, have made, use, offer to sell, sell, import, +transfer and otherwise run, modify and propagate the contents of this +implementation of Go, where such license applies only to those patent +claims, both currently owned or controlled by Google and acquired in +the future, licensable by Google that are necessarily infringed by this +implementation of Go. This grant does not include claims that would be +infringed only as a consequence of further modification of this +implementation. If you or your agent or exclusive licensee institute or +order or agree to the institution of patent litigation against any +entity (including a cross-claim or counterclaim in a lawsuit) alleging +that this implementation of Go or any code incorporated within this +implementation of Go constitutes direct or contributory patent +infringement, or inducement of patent infringement, then any patent +rights granted to you under this License for this implementation of Go +shall terminate as of the date such litigation is filed. + + +-------- Dependency +gopkg.in/fsnotify.v1 +-------- Copyrights +Copyright (c) 2012 The Go Authors. +Copyright (c) 2012 fsnotify Authors. +// Copyright 2012 The Go Authors. +// Copyright 2010 The Go Authors. +// Copyright 2016 The Go Authors. +// Copyright 2015 The Go Authors. +// Copyright 2013 The Go Authors. +// Copyright 2011 The Go Authors. + +======== Dependencies Summary +github.com/golang/protobuf +golang.org/x/net +golang.org/x/sys +golang.org/x/text +gopkg.in/fsnotify.v1 + +======== License used by Dependencies +SPDX:BSD-3-Clause--modified-by-Google +Redistribution and use in source and binary forms, with +or without modification, are permitted provided that the following conditions +are met: + + * Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above +copyright notice, this list of conditions and the following disclaimer +in the documentation and/or other materials provided with the +distribution. + * Neither the name of Google Inc. nor the names of its +contributors may be used to endorse or promote products derived from +this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +==== Dependencies Grouped by License ===== +-------- Dependency +github.com/hpcloud/tail +-------- Copyrights +# © Copyright 2015 Hewlett Packard Enterprise Development LP +Copyright (c) 2014 ActiveState +// Copyright (c) 2015 HPE Software Inc. +// Copyright (c) 2013 ActiveState Software Inc. +Copyright (C) 2013 99designs + +-------- Dependency +github.com/onsi/gomega +-------- Copyrights +Copyright (c) 2013-2014 Onsi Fakhouri +Copyright (c) 2014 Amit Kumar Gupta + +======== Dependencies Summary +github.com/hpcloud/tail +github.com/onsi/gomega + +======== License used by Dependencies +SPDX:MIT +Permission is hereby granted, free of charge, to any person +obtaining a copy of this software and associated documentation files +(the "Software"), to deal in the Software without restriction, including without +limitation the rights to use, copy, modify, merge, publish, distribute, +sublicense, and/or sell copies of the Software, and to permit persons to whom +the Software is furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS +FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR +COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER +IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN +CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +==== Dependencies Grouped by License ============ +-------- Dependency +gopkg.in/tomb.v1 +-------- Copyrights +Copyright (c) 2010-2011 - Gustavo Niemeyer +// Copyright (c) 2011 - Gustavo Niemeyer + +======== Dependencies Summary +gopkg.in/tomb.v1 + +======== License used by Dependencies +tomb - support for clean goroutine termination in Go. + +Copyright (c) 2010-2011 - Gustavo Niemeyer + + + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + + * Redistributions of source code must retain the above copyright notice, + this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright notice, + this list of conditions and the following disclaimer in the documentation + and/or other materials provided with the distribution. + * Neither the name of the copyright holder nor the names of its + contributors may be used to endorse or promote products derived from + this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR +CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, +EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR +PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF +LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING +NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS +SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +=================================================================================================== +Gomega +github.com/onsi/gomega +MIT License +Copyright (c) 2013-2014 Onsi Fakhouri + +4th party dependencies +--------------------------------------------------------------------------------------- +------- Dependency +gopkg.in/yaml.v2 +-------- Copyrights +Copyright (c) 2006 Kirill Simonov +-------- Notices +Copyright 2011-2016 Canonical Ltd. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. + + +======== Dependencies Summary +gopkg.in/yaml.v2 + +======== License used by Dependencies +SPDX:Apache-2.0 + + +===== Dependencies Grouped by License ======= +-------- Dependency +github.com/golang/protobuf +-------- Copyrights +Copyright 2010 The Go Authors. +# Copyright 2010 The Go Authors. +# Copyright 2016 The Go Authors. +// Copyright 2016 The Go Authors. +// Copyright 2015 The Go Authors. +// Copyright 2010 The Go Authors. +// Copyright 2011 The Go Authors. +// Copyright 2017 The Go Authors. +// Copyright 2014 The Go Authors. +// Copyright 2012 The Go Authors. +// Copyright 2013 The Go Authors. + +-------- Dependency +golang.org/x/net +-------- Copyrights +Copyright (c) 2009 The Go Authors. +// Copyright 2016 The Go Authors. +// Copyright 2017 The Go Authors. +// Copyright 2014 The Go Authors. +// Copyright 2015 The Go Authors. +// Copyright 2010 The Go Authors. +// Copyright 2009 The Go Authors. +// Copyright 2012 The Go Authors. +// Copyright 2013 The Go Authors. +// Copyright 2011 The Go Authors. +// Copyright 2018 The Go Authors. +# Copyright 2018 The Go Authors. + Copyright 2009 The Go Authors. +-------- Patents +Additional IP Rights Grant (Patents) + +"This implementation" means the copyrightable works distributed by +Google as part of the Go project. + +Google hereby grants to You a perpetual, worldwide, non-exclusive, +no-charge, royalty-free, irrevocable (except as stated in this section) +patent license to make, have made, use, offer to sell, sell, import, +transfer and otherwise run, modify and propagate the contents of this +implementation of Go, where such license applies only to those patent +claims, both currently owned or controlled by Google and acquired in +the future, licensable by Google that are necessarily infringed by this +implementation of Go. This grant does not include claims that would be +infringed only as a consequence of further modification of this +implementation. If you or your agent or exclusive licensee institute or +order or agree to the institution of patent litigation against any +entity (including a cross-claim or counterclaim in a lawsuit) alleging +that this implementation of Go or any code incorporated within this +implementation of Go constitutes direct or contributory patent +infringement, or inducement of patent infringement, then any patent +rights granted to you under this License for this implementation of Go +shall terminate as of the date such litigation is filed. + + +-------- Dependency +golang.org/x/sys +-------- Copyrights +Copyright (c) 2009 The Go Authors. +// Copyright 2018 The Go Authors. +// Copyright 2012 The Go Authors. +// Copyright 2011 The Go Authors. +// Copyright 2015 The Go Authors. +// Copyright 2009 The Go Authors. +// Copyright 2013 The Go Authors. +// Copyright 2016 The Go Authors. +// Copyright 2017 The Go Authors. +// Copyright 2010 The Go Authors. +// Copyright 2014 The Go Authors. +// Copyright 2009,2010 The Go Authors. +// Copyright 2017 The Go Authors. All right reserved. +-------- Patents +Additional IP Rights Grant (Patents) + +"This implementation" means the copyrightable works distributed by +Google as part of the Go project. + +Google hereby grants to You a perpetual, worldwide, non-exclusive, +no-charge, royalty-free, irrevocable (except as stated in this section) +patent license to make, have made, use, offer to sell, sell, import, +transfer and otherwise run, modify and propagate the contents of this +implementation of Go, where such license applies only to those patent +claims, both currently owned or controlled by Google and acquired in +the future, licensable by Google that are necessarily infringed by this +implementation of Go. This grant does not include claims that would be +infringed only as a consequence of further modification of this +implementation. If you or your agent or exclusive licensee institute or +order or agree to the institution of patent litigation against any +entity (including a cross-claim or counterclaim in a lawsuit) alleging +that this implementation of Go or any code incorporated within this +implementation of Go constitutes direct or contributory patent +infringement, or inducement of patent infringement, then any patent +rights granted to you under this License for this implementation of Go +shall terminate as of the date such litigation is filed. + + +-------- Dependency +golang.org/x/text +-------- Copyrights +Copyright (c) 2009 The Go Authors. +// Copyright 2014 The Go Authors. +// Copyright 2016 The Go Authors. +// Copyright 2015 The Go Authors. +// Copyright 2017 The Go Authors. +// Copyright 2012 The Go Authors. +// Copyright 2013 The Go Authors. +# Copyright 2012 The Go Authors. +# Copyright 2013 The Go Authors. +// Copyright 2011 The Go Authors. +-------- Patents +Additional IP Rights Grant (Patents) + +"This implementation" means the copyrightable works distributed by +Google as part of the Go project. + +Google hereby grants to You a perpetual, worldwide, non-exclusive, +no-charge, royalty-free, irrevocable (except as stated in this section) +patent license to make, have made, use, offer to sell, sell, import, +transfer and otherwise run, modify and propagate the contents of this +implementation of Go, where such license applies only to those patent +claims, both currently owned or controlled by Google and acquired in +the future, licensable by Google that are necessarily infringed by this +implementation of Go. This grant does not include claims that would be +infringed only as a consequence of further modification of this +implementation. If you or your agent or exclusive licensee institute or +order or agree to the institution of patent litigation against any +entity (including a cross-claim or counterclaim in a lawsuit) alleging +that this implementation of Go or any code incorporated within this +implementation of Go constitutes direct or contributory patent +infringement, or inducement of patent infringement, then any patent +rights granted to you under this License for this implementation of Go +shall terminate as of the date such litigation is filed. + + +-------- Dependency +golang.org/x/xerrors +-------- Copyrights +Copyright (c) 2019 The Go Authors. +// Copyright 2018 The Go Authors. +// Copyright 2019 The Go Authors. +// Copyright 2011 The Go Authors. +// Copyright 2012 The Go Authors. +-------- Patents +Additional IP Rights Grant (Patents) + +"This implementation" means the copyrightable works distributed by +Google as part of the Go project. + +Google hereby grants to You a perpetual, worldwide, non-exclusive, +no-charge, royalty-free, irrevocable (except as stated in this section) +patent license to make, have made, use, offer to sell, sell, import, +transfer and otherwise run, modify and propagate the contents of this +implementation of Go, where such license applies only to those patent +claims, both currently owned or controlled by Google and acquired in +the future, licensable by Google that are necessarily infringed by this +implementation of Go. This grant does not include claims that would be +infringed only as a consequence of further modification of this +implementation. If you or your agent or exclusive licensee institute or +order or agree to the institution of patent litigation against any +entity (including a cross-claim or counterclaim in a lawsuit) alleging +that this implementation of Go or any code incorporated within this +implementation of Go constitutes direct or contributory patent +infringement, or inducement of patent infringement, then any patent +rights granted to you under this License for this implementation of Go +shall terminate as of the date such litigation is filed. + + +-------- Dependency +gopkg.in/fsnotify.v1 +-------- Copyrights +Copyright (c) 2012 The Go Authors. +Copyright (c) 2012 fsnotify Authors. +// Copyright 2012 The Go Authors. +// Copyright 2010 The Go Authors. +// Copyright 2016 The Go Authors. +// Copyright 2015 The Go Authors. +// Copyright 2013 The Go Authors. +// Copyright 2011 The Go Authors. + +======== Dependencies Summary +github.com/golang/protobuf +golang.org/x/net +golang.org/x/sys +golang.org/x/text +golang.org/x/xerrors +gopkg.in/fsnotify.v1 + +======== License used by Dependencies +SPDX:BSD-3-Clause--modified-by-Google +Redistribution and use in source and binary forms, with +or without modification, are permitted provided that the following conditions +are met: + + * Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above +copyright notice, this list of conditions and the following disclaimer +in the documentation and/or other materials provided with the +distribution. + * Neither the name of Google Inc. nor the names of its +contributors may be used to endorse or promote products derived from +this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +===== Dependencies Grouped by License ============ +-------- Dependency +github.com/hpcloud/tail +-------- Copyrights +# © Copyright 2015 Hewlett Packard Enterprise Development LP +Copyright (c) 2014 ActiveState +// Copyright (c) 2015 HPE Software Inc. +// Copyright (c) 2013 ActiveState Software Inc. +Copyright (C) 2013 99designs + +-------- Dependency +github.com/onsi/ginkgo +-------- Copyrights +Copyright (c) 2013-2014 Onsi Fakhouri +Copyright (c) 2016 Yasuhiro Matsumoto + +======== Dependencies Summary +github.com/hpcloud/tail +github.com/onsi/ginkgo + +======== License used by Dependencies +SPDX:MIT +Permission is hereby granted, free of charge, to any person +obtaining a copy of this software and associated documentation files +(the "Software"), to deal in the Software without restriction, including without +limitation the rights to use, copy, modify, merge, publish, distribute, +sublicense, and/or sell copies of the Software, and to permit persons to whom +the Software is furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS +FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR +COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER +IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN +CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + +==== Dependencies Grouped by License ============ +-------- Dependency +gopkg.in/tomb.v1 +-------- Copyrights +Copyright (c) 2010-2011 - Gustavo Niemeyer +// Copyright (c) 2011 - Gustavo Niemeyer + +======== Dependencies Summary +gopkg.in/tomb.v1 + +======== License used by Dependencies +tomb - support for clean goroutine termination in Go. + +Copyright (c) 2010-2011 - Gustavo Niemeyer + + + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + + * Redistributions of source code must retain the above copyright notice, + this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright notice, + this list of conditions and the following disclaimer in the documentation + and/or other materials provided with the distribution. + * Neither the name of the copyright holder nor the names of its + contributors may be used to endorse or promote products derived from + this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR +CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, +EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, +PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR +PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF +LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING +NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS +SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +=================================================================================================== +Operator Lifecycle Manager (OLM) +github.com/operator-framework/operator-lifecycle-manager +Apache 2.0 + +4th party dependencies +--------------------------------------------------------------------------------------- diff --git a/apis/database/v1alpha1/autonomousdatabase_types.go b/apis/database/v1alpha1/autonomousdatabase_types.go new file mode 100644 index 00000000..4916ad46 --- /dev/null +++ b/apis/database/v1alpha1/autonomousdatabase_types.go @@ -0,0 +1,224 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v1alpha1 + +import ( + "encoding/json" + "strconv" + + "github.com/oracle/oci-go-sdk/v45/database" + metaV1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "sigs.k8s.io/controller-runtime/pkg/client" + + "github.com/oracle/oracle-database-operator/commons/annotations" +) + +// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! +// NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized. + +// AutonomousDatabaseSpec defines the desired state of AutonomousDatabase +// Important: Run "make" to regenerate code after modifying this file +type AutonomousDatabaseSpec struct { + Details AutonomousDatabaseDetails `json:"details"` + OCIConfig OCIConfigSpec `json:"ociConfig,omitempty"` + // +kubebuilder:default:=false + HardLink *bool `json:"hardLink,omitempty"` +} + +type OCIConfigSpec struct { + ConfigMapName *string `json:"configMapName,omitempty"` + SecretName *string `json:"secretName,omitempty"` +} + +// AutonomousDatabaseDetails defines the detail information of AutonomousDatabase, corresponding to oci-go-sdk/database/AutonomousDatabase +type AutonomousDatabaseDetails struct { + AutonomousDatabaseOCID *string `json:"autonomousDatabaseOCID,omitempty"` + CompartmentOCID *string `json:"compartmentOCID,omitempty"` + DisplayName *string `json:"displayName,omitempty"` + DbName *string `json:"dbName,omitempty"` + // +kubebuilder:validation:Enum:=OLTP;DW;AJD;APEX + DbWorkload database.AutonomousDatabaseDbWorkloadEnum `json:"dbWorkload,omitempty"` + IsDedicated *bool `json:"isDedicated,omitempty"` + DbVersion *string `json:"dbVersion,omitempty"` + DataStorageSizeInTBs *int `json:"dataStorageSizeInTBs,omitempty"` + CPUCoreCount *int `json:"cpuCoreCount,omitempty"` + AdminPassword PasswordSpec `json:"adminPassword,omitempty"` + IsAutoScalingEnabled *bool `json:"isAutoScalingEnabled,omitempty"` + LifecycleState database.AutonomousDatabaseLifecycleStateEnum `json:"lifecycleState,omitempty"` + SubnetOCID *string `json:"subnetOCID,omitempty"` + NsgOCIDs []string `json:"nsgOCIDs,omitempty"` + PrivateEndpoint *string `json:"privateEndpoint,omitempty"` + PrivateEndpointLabel *string `json:"privateEndpointLabel,omitempty"` + PrivateEndpointIP *string `json:"privateEndpointIP,omitempty"` + FreeformTags map[string]string `json:"freeformTags,omitempty"` + Wallet WalletSpec `json:"wallet,omitempty"` +} + +type WalletSpec struct { + Name *string `json:"name,omitempty"` + Password PasswordSpec `json:"password,omitempty"` +} + +type PasswordSpec struct { + K8sSecretName *string `json:"k8sSecretName,omitempty"` + OCISecretOCID *string `json:"ociSecretOCID,omitempty"` +} + +// AutonomousDatabaseStatus defines the observed state of AutonomousDatabase +type AutonomousDatabaseStatus struct { + // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster + // Important: Run "make" to regenerate code after modifying this file + DisplayName string `json:"displayName,omitempty"` + LifecycleState database.AutonomousDatabaseLifecycleStateEnum `json:"lifecycleState,omitempty"` + IsDedicated string `json:"isDedicated,omitempty"` + CPUCoreCount int `json:"cpuCoreCount,omitempty"` + DataStorageSizeInTBs int `json:"dataStorageSizeInTBs,omitempty"` + DbWorkload database.AutonomousDatabaseDbWorkloadEnum `json:"dbWorkload,omitempty"` + TimeCreated string `json:"timeCreated,omitempty"` +} + +// AutonomousDatabase is the Schema for the autonomousdatabases API +// +kubebuilder:object:root=true +// +kubebuilder:resource:shortName="adb";"adbs" +// +kubebuilder:subresource:status +// +kubebuilder:printcolumn:JSONPath=".status.displayName",name="Display Name",type=string +// +kubebuilder:printcolumn:JSONPath=".status.lifecycleState",name="State",type=string +// +kubebuilder:printcolumn:JSONPath=".status.isDedicated",name="Dedicated",type=string +// +kubebuilder:printcolumn:JSONPath=".status.cpuCoreCount",name="OCPUs",type=integer +// +kubebuilder:printcolumn:JSONPath=".status.dataStorageSizeInTBs",name="Storage (TB)",type=integer +// +kubebuilder:printcolumn:JSONPath=".status.dbWorkload",name="Workload Type",type=string +// +kubebuilder:printcolumn:JSONPath=".status.timeCreated",name="Created",type=string +type AutonomousDatabase struct { + metaV1.TypeMeta `json:",inline"` + metaV1.ObjectMeta `json:"metadata,omitempty"` + + Spec AutonomousDatabaseSpec `json:"spec,omitempty"` + Status AutonomousDatabaseStatus `json:"status,omitempty"` +} + +// LastSuccessfulSpec is an annotation key which maps to the value of last successful spec +const LastSuccessfulSpec string = "lastSuccessfulSpec" + +// GetLastSuccessfulSpec returns spec from the lass successful reconciliation. +// Returns nil, nil if there is no lastSuccessfulSpec. +func (adb *AutonomousDatabase) GetLastSuccessfulSpec() (*AutonomousDatabaseSpec, error) { + val, ok := adb.GetAnnotations()[LastSuccessfulSpec] + if !ok { + return nil, nil + } + + specBytes := []byte(val) + sucSpec := AutonomousDatabaseSpec{} + + err := json.Unmarshal(specBytes, &sucSpec) + if err != nil { + return nil, err + } + + return &sucSpec, nil +} + +// UpdateLastSuccessfulSpec updates lastSuccessfulSpec with the current spec. +func (adb *AutonomousDatabase) UpdateLastSuccessfulSpec(kubeClient client.Client) error { + specBytes, err := json.Marshal(adb.Spec) + if err != nil { + return err + } + + anns := map[string]string{ + LastSuccessfulSpec: string(specBytes), + } + + return annotations.SetAnnotations(kubeClient, adb, anns) +} + +// UpdateAttrFromOCIAutonomousDatabase updates the attributes from database.AutonomousDatabase object and returns the resource +func (adb *AutonomousDatabase) UpdateAttrFromOCIAutonomousDatabase(ociObj database.AutonomousDatabase) *AutonomousDatabase { + adb.Spec.Details.AutonomousDatabaseOCID = ociObj.Id + adb.Spec.Details.CompartmentOCID = ociObj.CompartmentId + adb.Spec.Details.DisplayName = ociObj.DisplayName + adb.Spec.Details.DbName = ociObj.DbName + adb.Spec.Details.DbWorkload = ociObj.DbWorkload + adb.Spec.Details.IsDedicated = ociObj.IsDedicated + adb.Spec.Details.DbVersion = ociObj.DbVersion + adb.Spec.Details.DataStorageSizeInTBs = ociObj.DataStorageSizeInTBs + adb.Spec.Details.CPUCoreCount = ociObj.CpuCoreCount + adb.Spec.Details.IsAutoScalingEnabled = ociObj.IsAutoScalingEnabled + adb.Spec.Details.LifecycleState = ociObj.LifecycleState + adb.Spec.Details.FreeformTags = ociObj.FreeformTags + + adb.Spec.Details.SubnetOCID = ociObj.SubnetId + adb.Spec.Details.NsgOCIDs = ociObj.NsgIds + adb.Spec.Details.PrivateEndpoint = ociObj.PrivateEndpoint + adb.Spec.Details.PrivateEndpointLabel = ociObj.PrivateEndpointLabel + adb.Spec.Details.PrivateEndpointIP = ociObj.PrivateEndpointIp + + // update the subresource as well + adb.Status.DisplayName = *ociObj.DisplayName + adb.Status.LifecycleState = ociObj.LifecycleState + adb.Status.IsDedicated = strconv.FormatBool(*ociObj.IsDedicated) + adb.Status.CPUCoreCount = *ociObj.CpuCoreCount + adb.Status.DataStorageSizeInTBs = *ociObj.DataStorageSizeInTBs + adb.Status.DbWorkload = ociObj.DbWorkload + adb.Status.TimeCreated = ociObj.TimeCreated.String() + + return adb +} + +// +kubebuilder:object:root=true + +// AutonomousDatabaseList contains a list of AutonomousDatabase +type AutonomousDatabaseList struct { + metaV1.TypeMeta `json:",inline"` + metaV1.ListMeta `json:"metadata,omitempty"` + Items []AutonomousDatabase `json:"items"` +} + +func init() { + SchemeBuilder.Register(&AutonomousDatabase{}, &AutonomousDatabaseList{}) +} + +// A helper function which is useful for debugging. The function prints out a structural JSON format. +func (adb *AutonomousDatabase) String() (string, error) { + out, err := json.MarshalIndent(adb, "", " ") + if err != nil { + return "", err + } + return string(out), nil +} diff --git a/apis/database/v1alpha1/groupversion_info.go b/apis/database/v1alpha1/groupversion_info.go new file mode 100644 index 00000000..60029108 --- /dev/null +++ b/apis/database/v1alpha1/groupversion_info.go @@ -0,0 +1,58 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +// Package v1alpha1 contains API Schema definitions for the database v1alpha1 API group +//+kubebuilder:object:generate=true +//+groupName=database.oracle.com +package v1alpha1 + +import ( + "k8s.io/apimachinery/pkg/runtime/schema" + "sigs.k8s.io/controller-runtime/pkg/scheme" +) + +var ( + // GroupVersion is group version used to register these objects + GroupVersion = schema.GroupVersion{Group: "database.oracle.com", Version: "v1alpha1"} + + // SchemeBuilder is used to add go types to the GroupVersionKind scheme + SchemeBuilder = &scheme.Builder{GroupVersion: GroupVersion} + + // AddToScheme adds the types in this group-version to the given scheme. + AddToScheme = SchemeBuilder.AddToScheme +) diff --git a/apis/database/v1alpha1/shardingdatabase_types.go b/apis/database/v1alpha1/shardingdatabase_types.go new file mode 100644 index 00000000..c372b991 --- /dev/null +++ b/apis/database/v1alpha1/shardingdatabase_types.go @@ -0,0 +1,306 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v1alpha1 + +import ( + "sync" + + "encoding/json" + + "sigs.k8s.io/controller-runtime/pkg/client" + + annsv1 "github.com/oracle/oracle-database-operator/commons/annotations" + + corev1 "k8s.io/api/core/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! +// NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized. + +// ShardingDatabaseSpec defines the desired state of ShardingDatabase +type ShardingDatabaseSpec struct { + // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster + // Important: Run "make" to regenerate code after modifying this file + Shard []ShardSpec `json:"shard"` + Catalog []CatalogSpec `json:"catalog"` // The catalogSpes accept all the catalog parameters + Gsm []GsmSpec `json:"gsm"` // The GsmSpec will accept all the Gsm parameter + StorageClass string `json:"storageClass,omitempty"` // Optional Accept storage class name + DbImage string `json:"dbImage"` // Accept DB Image name + DbImagePullSecret string `json:"dbImagePullSecret,omitempty"` // Optional The name of an image pull secret in case of a private docker repository. + GsmImage string `json:"gsmImage"` // Acccept the GSM image name + GsmImagePullSecret string `json:"gsmImagePullSecret,omitempty"` // Optional The name of an image pull secret in case of a private docker repository. + Secret string `json:"secret"` // Secret Name to be used with Shard + StagePvcName string `json:"stagePvcName,omitempty"` // the Stagepvc for the backup of cluster + PortMappings []PortMapping `json:"portMappings,omitempty"` // Port mappings for the service that is created. The service is created if there is at least + Namespace string `json:"namespace,omitempty"` // Target namespace of the application. + IsDebug bool `json:"isDebug,omitempty"` // Optional parameter to enable logining + IsExternalSvc bool `json:"isExternalSvc,omitempty"` + IsClone bool `json:"isClone,omitempty"` + IsDataGuard bool `json:"isDataGuard,omitempty"` + ScriptsLocation string `json:"scriptsLocation,omitempty"` + NsConfigMap string `json:"nsConfigMap,omitempty"` + NsSecret string `json:"nsSecret,omitempty"` + IsDeleteOraPvc bool `json:"isDeleteOraPvc,omitempty"` +} + +// To understand Metav1.Condition, please refer the link https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1 +// ShardingDatabaseStatus defines the observed state of ShardingDatabase +type ShardingDatabaseStatus struct { + // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster + // Important: Run "make" to regenerate code after modifying this file + + Shard map[string]string `json:"shards,omitempty"` + Catalog map[string]string `json:"catalogs,omitempty"` + Gsm GsmStatus `json:"gsm,omitempty"` + + // +patchMergeKey=type + // +patchStrategy=merge + // +listType=map + // +listMapKey=type + CrdStatus []metav1.Condition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type"` +} + +type GsmStatus struct { + InternalconnectStr string `json:"internalConnectStr,omitempty"` + ExternalConnectStr string `json:"externalConnectStr,omitempty"` + State string `json:"state,omitempty"` + Shards map[string]string `json:"shards,omitempty"` + Details map[string]string `json:"details,omitempty"` + Services string `json:"services,omitempty"` +} + +type GsmStatusDetails struct { + Name string `json:"name,omitempty"` + K8sInternalSvc string `json:"k8sInternalSvc,omitempty"` + K8sExternalSvc string `json:"k8sExternalSvc,omitempty"` + K8sInternalSvcIP string `json:"k8sInternalIP,omitempty"` + K8sExternalSvcIP string `json:"k8sExternalIP,omitempty"` + Role string `json:"role,omitempty"` + DbPasswordSecret string `json:"dbPasswordSecret"` +} + +//+kubebuilder:object:root=true +//+kubebuilder:subresource:status + +// ShardingDatabase is the Schema for the shardingdatabases API +type ShardingDatabase struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + Spec ShardingDatabaseSpec `json:"spec,omitempty"` + Status ShardingDatabaseStatus `json:"status,omitempty"` +} + +//+kubebuilder:object:root=true + +// ShardingDatabaseList contains a list of ShardingDatabase +type ShardingDatabaseList struct { + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata,omitempty"` + Items []ShardingDatabase `json:"items"` +} + +// ShardSpec is a specification of Shards for an application deployment. +// +k8s:openapi-gen=true +type ShardSpec struct { + Name string `json:"name"` // Shard name that will be used deploy StatefulSet + StorageSizeInGb int32 `json:"storageSizeInGb,omitempty"` // Optional Shard Storage Size + EnvVars []EnvironmentVariable `json:"envVars,omitempty"` //Optional Env variables for Shards + Resources *corev1.ResourceRequirements `json:"resources,omitempty" protobuf:"bytes,1,opt,name=resources"` //Optional resource requirement for the container. + PvcName string `json:"pvcName,omitempty"` + Label string `json:"label,omitempty"` + IsDelete bool `json:"isDelete,omitempty"` + NodeSelector map[string]string `json:"nodeSelector,omitempty"` + PvAnnotations map[string]string `json:"pvAnnotations,omitempty"` + PvMatchLabels map[string]string `json:"pvMatchLabels,omitempty"` + ImagePulllPolicy *corev1.PullPolicy `json:"imagePullPolicy,omitempty"` +} + +// CatalogSpec defines the desired state of CatalogSpec +// +k8s:openapi-gen=true +type CatalogSpec struct { + Name string `json:"name"` // Catalog name that will be used deploy StatefulSet + StorageSizeInGb int32 `json:"storageSizeInGb,omitempty"` // Optional Catalog Storage Size and This parameter will not be used if you use PvcName + EnvVars []EnvironmentVariable `json:"envVars,omitempty"` //Optional Env variables for Catalog + Resources *corev1.ResourceRequirements `json:"resources,omitempty" protobuf:"bytes,1,opt,name=resources"` // Optional resource requirement for the container. + PvcName string `json:"pvcName,omitempty"` + Label string `json:"label,omitempty"` + IsDelete bool `json:"isDelete,omitempty"` + NodeSelector map[string]string `json:"nodeSelector,omitempty"` + PvAnnotations map[string]string `json:"pvAnnotations,omitempty"` + PvMatchLabels map[string]string `json:"pvMatchLabels,omitempty"` + ImagePulllPolicy *corev1.PullPolicy `json:"imagePullPolicy,omitempty"` +} + +// GsmSpec defines the desired state of GsmSpec +// +k8s:openapi-gen=true +type GsmSpec struct { + Name string `json:"name"` // Gsm name that will be used deploy StatefulSet + + Replicas int32 `json:"replicas,omitempty"` // Gsm Replicas. If you set OraGsmPvcName then it is set default to 1. + EnvVars []EnvironmentVariable `json:"envVars,omitempty"` //Optional Env variables for GSM + StorageSizeInGb int32 `json:"storageSizeInGb,omitempty"` // This parameter will not be used if you use OraGsmPvcName + Resources *corev1.ResourceRequirements `json:"resources,omitempty" protobuf:"bytes,1,opt,name=resources"` // Optional resource requirement for the container. + PvcName string `json:"pvcName,omitempty"` + Label string `json:"label,omitempty"` // Optional GSM Label + IsDelete bool `json:"isDelete,omitempty"` + NodeSelector map[string]string `json:"nodeSelector,omitempty"` + PvMatchLabels map[string]string `json:"pvMatchLabels,omitempty"` + ImagePulllPolicy *corev1.PullPolicy `json:"imagePullPolicy,omitempty"` +} + +// EnvironmentVariable represents a named variable accessible for containers. +// +k8s:openapi-gen=true +type EnvironmentVariable struct { + Name string `json:"name"` // Name of the variable. Must be a C_IDENTIFIER. + Value string `json:"value"` // Value of the variable, as defined in Kubernetes core API. +} + +// PortMapping is a specification of port mapping for an application deployment. +// +k8s:openapi-gen=true +type PortMapping struct { + Port int32 `json:"port"` // Port that will be exposed on the service. + TargetPort int32 `json:"targetPort"` // Docker image port for the application. + Protocol corev1.Protocol `json:"protocol"` // IP protocol for the mapping, e.g., "TCP" or "UDP". +} + +type SfsetLabel string + +const ( + ShardingDelLabelKey SfsetLabel = "sharding.oracle.com/delflag" + ShardingDelLabelTrueValue SfsetLabel = "true" + ShardingDelLabelFalseValue SfsetLabel = "false" +) + +type ShardStatusMapKeys string + +const ( + Name ShardStatusMapKeys = "Name" + K8sInternalSvc ShardStatusMapKeys = "K8sInternalSvc" + K8sExternalSvc ShardStatusMapKeys = "K8sExternalSvc" + K8sInternalSvcIP ShardStatusMapKeys = "K8sInternalSvcIP" + K8sExternalSvcIP ShardStatusMapKeys = "K8sExternalSvcIP" + OracleSid ShardStatusMapKeys = "OracleSid" + OraclePdb ShardStatusMapKeys = "OraclePdb" + Role ShardStatusMapKeys = "Role" + DbPasswordSecret ShardStatusMapKeys = "DbPasswordSecret" + State ShardStatusMapKeys = "State" + OpenMode ShardStatusMapKeys = "OpenMode" +) + +type ShardLifecycleState string + +const ( + AvailableState ShardLifecycleState = "AVAILABLE" + FailedState ShardLifecycleState = "FAILED" + UpdateState ShardLifecycleState = "UPDATING" + ProvisionState ShardLifecycleState = "PROVISIONING" + PodNotReadyState ShardLifecycleState = "PODNOTREADY" + PodFailureState ShardLifecycleState = "PODFAILURE" + PodNotFound ShardLifecycleState = "PODNOTFOUND" + StatefulSetFailure ShardLifecycleState = "STATEFULSETFAILURE" + StatefulSetNotFound ShardLifecycleState = "STATEFULSETNOTFOUND" + DeletingState ShardLifecycleState = "DELETING" + DeleteErrorState ShardLifecycleState = "DELETE_ERROR" + ChunkMoveError ShardLifecycleState = "CHUNK_MOVE_ERROR_IN_GSM" + Terminated ShardLifecycleState = "TERMINATED" + LabelPatchingError ShardLifecycleState = "LABELPATCHINGERROR" + DeletePVCError ShardLifecycleState = "DELETEPVCERROR" + AddingShardState ShardLifecycleState = "SHARD_ADDITION" + AddingShardErrorState ShardLifecycleState = "SHARD_ADDITION_ERROR_IN_GSM" + ShardOnlineErrorState ShardLifecycleState = "SHARD_ONLINE_ERROR_IN_GSM" + ShardOnlineState ShardLifecycleState = "ONLINE_SHARD" + ShardRemoveError ShardLifecycleState = "SHARD_DELETE_ERROR_FROM_GSM" +) + +type CrdReconcileState string + +const ( + CrdReconcileErrorState CrdReconcileState = "ReconcileError" + CrdReconcileErrorReason CrdReconcileState = "LastReconcileCycleFailed" + CrdReconcileQueuedState CrdReconcileState = "ReconcileQueued" + CrdReconcileQueuedReason CrdReconcileState = "LastReconcileCycleQueued" + CrdReconcileCompeleteState CrdReconcileState = "ReconcileComplete" + CrdReconcileCompleteReason CrdReconcileState = "LastReconcileCycleCompleted" + CrdReconcileWaitingState CrdReconcileState = "ReconcileWaiting" + CrdReconcileWaitingReason CrdReconcileState = "LastReconcileCycleWaiting" +) + +// var +var KubeConfigOnce sync.Once + +const lastSuccessfulSpec = "lastSuccessfulSpec" + +// GetLastSuccessfulSpec returns spec from the lass successful reconciliation. +// Returns nil, nil if there is no lastSuccessfulSpec. +func (shardingv1 *ShardingDatabase) GetLastSuccessfulSpec() (*ShardingDatabaseSpec, error) { + val, ok := shardingv1.GetAnnotations()[lastSuccessfulSpec] + if !ok { + return nil, nil + } + + specBytes := []byte(val) + sucSpec := ShardingDatabaseSpec{} + + err := json.Unmarshal(specBytes, &sucSpec) + if err != nil { + return nil, err + } + + return &sucSpec, nil +} + +// UpdateLastSuccessfulSpec updates lastSuccessfulSpec with the current spec. +func (shardingv1 *ShardingDatabase) UpdateLastSuccessfulSpec(kubeClient client.Client) error { + specBytes, err := json.Marshal(shardingv1.Spec) + if err != nil { + return err + } + + anns := map[string]string{ + lastSuccessfulSpec: string(specBytes), + } + + return annsv1.SetAnnotations(kubeClient, shardingv1, anns) +} + +func init() { + SchemeBuilder.Register(&ShardingDatabase{}, &ShardingDatabaseList{}) +} diff --git a/apis/database/v1alpha1/singleinstancedatabase_types.go b/apis/database/v1alpha1/singleinstancedatabase_types.go new file mode 100644 index 00000000..7afb7ae6 --- /dev/null +++ b/apis/database/v1alpha1/singleinstancedatabase_types.go @@ -0,0 +1,184 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v1alpha1 + +import ( + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! +// NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized. + +// SingleInstanceDatabaseSpec defines the desired state of SingleInstanceDatabase +type SingleInstanceDatabaseSpec struct { + // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster + // Important: Run "make" to regenerate code after modifying this file + + // +kubebuilder:validation:Enum=standard;enterprise + Edition string `json:"edition,omitempty"` + + // SID can only have a-z , A-Z, 0-9 . It cant have any special characters + // +k8s:openapi-gen=true + // +kubebuilder:validation:Pattern=`^[a-zA-Z0-9]+$` + Sid string `json:"sid,omitempty"` + InstallApex bool `json:"installApex,omitempty"` + Charset string `json:"charset,omitempty"` + Pdbname string `json:"pdbName,omitempty"` + LoadBalancer bool `json:"loadBalancer,omitempty"` + FlashBack bool `json:"flashBack,omitempty"` + ArchiveLog bool `json:"archiveLog,omitempty"` + ForceLogging bool `json:"forceLog,omitempty"` + + CloneFrom string `json:"cloneFrom,omitempty"` + ReadinessCheckPeriod int `json:"readinessCheckPeriod,omitempty"` + + // +k8s:openapi-gen=true + // +kubebuilder:validation:Minimum=1 + Replicas int `json:"replicas"` + + NodeSelector map[string]string `json:"nodeSelector,omitempty"` + AdminPassword SingleInstanceDatabaseAdminPassword `json:"adminPassword"` + Image SingleInstanceDatabaseImage `json:"image"` + Persistence SingleInstanceDatabasePersistence `json:"persistence"` + InitParams SingleInstanceDatabaseInitParams `json:"initParams,omitempty"` +} + +// SingleInstanceDatabasePersistence defines the storage size and class for PVC +type SingleInstanceDatabasePersistence struct { + Size string `json:"size"` + StorageClass string `json:"storageClass"` + + // +kubebuilder:validation:Enum=ReadWriteOnce;ReadWriteMany + AccessMode string `json:"accessMode"` +} + +// SingleInstanceDatabaseInitParams defines the Init Parameters +type SingleInstanceDatabaseInitParams struct { + SgaTarget int `json:"sgaTarget,omitempty"` + PgaAggregateTarget int `json:"pgaAggregateTarget,omitempty"` + CpuCount int `json:"cpuCount,omitempty"` + Processes int `json:"processes,omitempty"` +} + +// SingleInstanceDatabaseImage defines the Image source and pullSecrets for POD +type SingleInstanceDatabaseImage struct { + Version string `json:"version,omitempty"` + PullFrom string `json:"pullFrom"` + PullSecrets string `json:"pullSecrets,omitempty"` +} + +// SingleInsatnceAdminPassword defines the secret containing Admin Password mapped to secretKey for Database +type SingleInstanceDatabaseAdminPassword struct { + SecretName string `json:"secretName"` + SecretKey string `json:"secretKey"` + KeepSecret bool `json:"keepSecret,omitempty"` +} + +// SingleInstanceDatabaseStatus defines the observed state of SingleInstanceDatabase +type SingleInstanceDatabaseStatus struct { + // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster + // Important: Run "make" to regenerate code after modifying this file + + Nodes []string `json:"nodes,omitempty"` + Role string `json:"role,omitempty"` + Status string `json:"status,omitempty"` + Replicas int `json:"replicas"` + ReleaseUpdate string `json:"releaseUpdate,omitempty"` + DatafilesPatched string `json:"datafilesPatched,omitempty"` + ConnectString string `json:"connectString,omitempty"` + ClusterConnectString string `json:"clusterConnectString,omitempty"` + StandbyDatabases map[string]string `json:"standbyDatabases,omitempty"` + DatafilesCreated string `json:"datafilesCreated,omitempty"` + Sid string `json:"sid,omitempty"` + Edition string `json:"edition,omitempty"` + Charset string `json:"charset,omitempty"` + Pdbname string `json:"pdbName,omitempty"` + InitSgaSize int `json:"initSgaSize,omitempty"` + InitPgaSize int `json:"initPgaSize,omitempty"` + CloneFrom string `json:"cloneFrom,omitempty"` + FlashBack string `json:"flashBack,omitempty"` + ArchiveLog string `json:"archiveLog,omitempty"` + ForceLogging string `json:"forceLog,omitempty"` + OemExpressUrl string `json:"oemExpressUrl,omitempty"` + OrdsReference string `json:"ordsReference,omitempty"` + PdbConnectString string `json:"pdbConnectString,omitempty"` + ApexInstalled bool `json:"apexInstalled,omitempty"` + + // +patchMergeKey=type + // +patchStrategy=merge + // +listType=map + // +listMapKey=type + Conditions []metav1.Condition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type"` + + InitParams SingleInstanceDatabaseInitParams `json:"initParams,omitempty"` + Persistence SingleInstanceDatabasePersistence `json:"persistence"` +} + +//+kubebuilder:object:root=true +//+kubebuilder:subresource:status +// +kubebuilder:subresource:scale:specpath=.spec.replicas,statuspath=.status.replicas +// +kubebuilder:printcolumn:JSONPath=".status.edition",name="Edition",type="string" +// +kubebuilder:printcolumn:JSONPath=".status.status",name="Status",type="string" +// +kubebuilder:printcolumn:JSONPath=".status.role",name="Role",type="string",priority=1 +// +kubebuilder:printcolumn:JSONPath=".status.releaseUpdate",name="Version",type="string" +// +kubebuilder:printcolumn:JSONPath=".status.connectString",name="Connect Str",type="string" +// +kubebuilder:printcolumn:JSONPath=".status.pdbConnectString",name="Pdb Connect Str",type="string",priority=1 +// +kubebuilder:printcolumn:JSONPath=".status.oemExpressUrl",name="Oem Express Url",type="string" + +// SingleInstanceDatabase is the Schema for the singleinstancedatabases API +type SingleInstanceDatabase struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + Spec SingleInstanceDatabaseSpec `json:"spec,omitempty"` + Status SingleInstanceDatabaseStatus `json:"status,omitempty"` +} + +//+kubebuilder:object:root=true + +// SingleInstanceDatabaseList contains a list of SingleInstanceDatabase +type SingleInstanceDatabaseList struct { + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata,omitempty"` + Items []SingleInstanceDatabase `json:"items"` +} + +func init() { + SchemeBuilder.Register(&SingleInstanceDatabase{}, &SingleInstanceDatabaseList{}) +} diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go new file mode 100644 index 00000000..1f7b44bb --- /dev/null +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -0,0 +1,195 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v1alpha1 + +import ( + "strings" + + dbcommons "github.com/oracle/oracle-database-operator/commons/database" + + apierrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/runtime/schema" + "k8s.io/apimachinery/pkg/util/validation/field" + ctrl "sigs.k8s.io/controller-runtime" + logf "sigs.k8s.io/controller-runtime/pkg/log" + "sigs.k8s.io/controller-runtime/pkg/webhook" +) + +// log is for logging in this package. +var singleinstancedatabaselog = logf.Log.WithName("singleinstancedatabase-resource") + +func (r *SingleInstanceDatabase) SetupWebhookWithManager(mgr ctrl.Manager) error { + return ctrl.NewWebhookManagedBy(mgr). + For(r). + Complete() +} + +// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! + +//+kubebuilder:webhook:path=/mutate-database-oracle-com-v1alpha1-singleinstancedatabase,mutating=true,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=singleinstancedatabases,verbs=create;update,versions=v1alpha1,name=msingleinstancedatabase.kb.io,admissionReviewVersions={v1,v1beta1} + +var _ webhook.Defaulter = &SingleInstanceDatabase{} + +// Default implements webhook.Defaulter so a webhook will be registered for the type +func (r *SingleInstanceDatabase) Default() { + singleinstancedatabaselog.Info("default", "name", r.Name) + + if r.Spec.Edition == "express" { + r.Spec.Replicas = 1 + } + // TODO(user): fill in your defaulting logic. +} + +// TODO(user): change verbs to "verbs=create;update;delete" if you want to enable deletion validation. +//+kubebuilder:webhook:verbs=create;update;delete,path=/validate-database-oracle-com-v1alpha1-singleinstancedatabase,mutating=false,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=singleinstancedatabases,versions=v1alpha1,name=vsingleinstancedatabase.kb.io,admissionReviewVersions={v1,v1beta1} + +var _ webhook.Validator = &SingleInstanceDatabase{} + +// ValidateCreate implements webhook.Validator so a webhook will be registered for the type +func (r *SingleInstanceDatabase) ValidateCreate() error { + singleinstancedatabaselog.Info("validate create", "name", r.Name) + var allErrs field.ErrorList + + if r.Spec.Persistence.AccessMode == "ReadWriteOnce" && r.Spec.Replicas != 1 { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("replicas"), r.Spec.Replicas, + "should be 1 for accessMode \"ReadWriteOnce\"")) + } + if r.Spec.Edition == "express" && r.Spec.CloneFrom != "" { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("cloneFrom"), r.Spec.CloneFrom, + "Cloning not supported for Express edition")) + } + if r.Spec.Edition == "express" && strings.ToUpper(r.Spec.Sid) != "XE" { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("sid"), r.Spec.Sid, + "Express edition SID must be XE")) + } + if r.Spec.Edition == "express" && strings.ToUpper(r.Spec.Pdbname) != "XEPDB1" { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("pdbName"), r.Spec.Pdbname, + "Express edition PDB must be XEPDB1")) + } + + if len(allErrs) == 0 { + return nil + } + return apierrors.NewInvalid( + schema.GroupKind{Group: "database.oracle.com", Kind: "SingleInstanceDatabase"}, + r.Name, allErrs) +} + +// ValidateUpdate implements webhook.Validator so a webhook will be registered for the type +func (r *SingleInstanceDatabase) ValidateUpdate(oldRuntimeObject runtime.Object) error { + singleinstancedatabaselog.Info("validate update", "name", r.Name) + var allErrs field.ErrorList + + // check creation validations first + err := r.ValidateCreate() + if err != nil { + return err + } + + // Validate Deletion + if r.GetDeletionTimestamp() != nil { + err := r.ValidateDelete() + if err != nil { + return err + } + } + // Now check for updation errors + old, ok := oldRuntimeObject.(*SingleInstanceDatabase) + if !ok { + return nil + } + edition := r.Spec.Edition + if r.Spec.Edition == "" { + edition = "Enterprise" + } + if r.Spec.CloneFrom == "" && old.Status.Edition != "" && !strings.EqualFold(old.Status.Edition, edition) { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("edition"), "cannot be changed")) + } + if old.Status.Charset != "" && !strings.EqualFold(old.Status.Charset, r.Spec.Charset) { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("charset"), "cannot be changed")) + } + if old.Status.Sid != "" && !strings.EqualFold(r.Spec.Sid, old.Status.Sid) { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("sid"), "cannot be changed")) + } + if old.Status.Pdbname != "" && !strings.EqualFold(old.Status.Pdbname, r.Spec.Pdbname) { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("pdbname"), "cannot be changed")) + } + if old.Status.CloneFrom != "" && + (old.Status.CloneFrom == dbcommons.NoCloneRef && r.Spec.CloneFrom != "" || + old.Status.CloneFrom != dbcommons.NoCloneRef && old.Status.CloneFrom != r.Spec.CloneFrom) { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("cloneFrom"), "cannot be changed")) + } + if old.Status.OrdsReference != "" && r.Status.Persistence != r.Spec.Persistence { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("persistence"), "uninstall ORDS to change Persistence")) + } + if len(allErrs) == 0 { + return nil + } + return apierrors.NewInvalid( + schema.GroupKind{Group: "database.oracle.com", Kind: "SingleInstanceDatabase"}, + r.Name, allErrs) + +} + +// ValidateDelete implements webhook.Validator so a webhook will be registered for the type +func (r *SingleInstanceDatabase) ValidateDelete() error { + singleinstancedatabaselog.Info("validate delete", "name", r.Name) + var allErrs field.ErrorList + if r.Status.OrdsReference != "" { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("status").Child("ordsInstalled"), "uninstall ORDS to cleanup this SIDB")) + } + if len(allErrs) == 0 { + return nil + } + return apierrors.NewInvalid( + schema.GroupKind{Group: "database.oracle.com", Kind: "SingleInstanceDatabase"}, + r.Name, allErrs) +} diff --git a/apis/database/v1alpha1/zz_generated.deepcopy.go b/apis/database/v1alpha1/zz_generated.deepcopy.go new file mode 100644 index 00000000..2766d559 --- /dev/null +++ b/apis/database/v1alpha1/zz_generated.deepcopy.go @@ -0,0 +1,848 @@ +// +build !ignore_autogenerated + +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +// Code generated by controller-gen. DO NOT EDIT. + +package v1alpha1 + +import ( + corev1 "k8s.io/api/core/v1" + "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/runtime" +) + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousDatabase) DeepCopyInto(out *AutonomousDatabase) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + in.Spec.DeepCopyInto(&out.Spec) + out.Status = in.Status +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabase. +func (in *AutonomousDatabase) DeepCopy() *AutonomousDatabase { + if in == nil { + return nil + } + out := new(AutonomousDatabase) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *AutonomousDatabase) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousDatabaseDetails) DeepCopyInto(out *AutonomousDatabaseDetails) { + *out = *in + if in.AutonomousDatabaseOCID != nil { + in, out := &in.AutonomousDatabaseOCID, &out.AutonomousDatabaseOCID + *out = new(string) + **out = **in + } + if in.CompartmentOCID != nil { + in, out := &in.CompartmentOCID, &out.CompartmentOCID + *out = new(string) + **out = **in + } + if in.DisplayName != nil { + in, out := &in.DisplayName, &out.DisplayName + *out = new(string) + **out = **in + } + if in.DbName != nil { + in, out := &in.DbName, &out.DbName + *out = new(string) + **out = **in + } + if in.IsDedicated != nil { + in, out := &in.IsDedicated, &out.IsDedicated + *out = new(bool) + **out = **in + } + if in.DbVersion != nil { + in, out := &in.DbVersion, &out.DbVersion + *out = new(string) + **out = **in + } + if in.DataStorageSizeInTBs != nil { + in, out := &in.DataStorageSizeInTBs, &out.DataStorageSizeInTBs + *out = new(int) + **out = **in + } + if in.CPUCoreCount != nil { + in, out := &in.CPUCoreCount, &out.CPUCoreCount + *out = new(int) + **out = **in + } + in.AdminPassword.DeepCopyInto(&out.AdminPassword) + if in.IsAutoScalingEnabled != nil { + in, out := &in.IsAutoScalingEnabled, &out.IsAutoScalingEnabled + *out = new(bool) + **out = **in + } + if in.SubnetOCID != nil { + in, out := &in.SubnetOCID, &out.SubnetOCID + *out = new(string) + **out = **in + } + if in.NsgOCIDs != nil { + in, out := &in.NsgOCIDs, &out.NsgOCIDs + *out = make([]string, len(*in)) + copy(*out, *in) + } + if in.PrivateEndpoint != nil { + in, out := &in.PrivateEndpoint, &out.PrivateEndpoint + *out = new(string) + **out = **in + } + if in.PrivateEndpointLabel != nil { + in, out := &in.PrivateEndpointLabel, &out.PrivateEndpointLabel + *out = new(string) + **out = **in + } + if in.PrivateEndpointIP != nil { + in, out := &in.PrivateEndpointIP, &out.PrivateEndpointIP + *out = new(string) + **out = **in + } + if in.FreeformTags != nil { + in, out := &in.FreeformTags, &out.FreeformTags + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + in.Wallet.DeepCopyInto(&out.Wallet) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseDetails. +func (in *AutonomousDatabaseDetails) DeepCopy() *AutonomousDatabaseDetails { + if in == nil { + return nil + } + out := new(AutonomousDatabaseDetails) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousDatabaseList) DeepCopyInto(out *AutonomousDatabaseList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]AutonomousDatabase, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseList. +func (in *AutonomousDatabaseList) DeepCopy() *AutonomousDatabaseList { + if in == nil { + return nil + } + out := new(AutonomousDatabaseList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *AutonomousDatabaseList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousDatabaseSpec) DeepCopyInto(out *AutonomousDatabaseSpec) { + *out = *in + in.Details.DeepCopyInto(&out.Details) + in.OCIConfig.DeepCopyInto(&out.OCIConfig) + if in.HardLink != nil { + in, out := &in.HardLink, &out.HardLink + *out = new(bool) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseSpec. +func (in *AutonomousDatabaseSpec) DeepCopy() *AutonomousDatabaseSpec { + if in == nil { + return nil + } + out := new(AutonomousDatabaseSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousDatabaseStatus) DeepCopyInto(out *AutonomousDatabaseStatus) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseStatus. +func (in *AutonomousDatabaseStatus) DeepCopy() *AutonomousDatabaseStatus { + if in == nil { + return nil + } + out := new(AutonomousDatabaseStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CatalogSpec) DeepCopyInto(out *CatalogSpec) { + *out = *in + if in.EnvVars != nil { + in, out := &in.EnvVars, &out.EnvVars + *out = make([]EnvironmentVariable, len(*in)) + copy(*out, *in) + } + if in.Resources != nil { + in, out := &in.Resources, &out.Resources + *out = new(corev1.ResourceRequirements) + (*in).DeepCopyInto(*out) + } + if in.NodeSelector != nil { + in, out := &in.NodeSelector, &out.NodeSelector + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + if in.PvAnnotations != nil { + in, out := &in.PvAnnotations, &out.PvAnnotations + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + if in.PvMatchLabels != nil { + in, out := &in.PvMatchLabels, &out.PvMatchLabels + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + if in.ImagePulllPolicy != nil { + in, out := &in.ImagePulllPolicy, &out.ImagePulllPolicy + *out = new(corev1.PullPolicy) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CatalogSpec. +func (in *CatalogSpec) DeepCopy() *CatalogSpec { + if in == nil { + return nil + } + out := new(CatalogSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *EnvironmentVariable) DeepCopyInto(out *EnvironmentVariable) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new EnvironmentVariable. +func (in *EnvironmentVariable) DeepCopy() *EnvironmentVariable { + if in == nil { + return nil + } + out := new(EnvironmentVariable) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *GsmSpec) DeepCopyInto(out *GsmSpec) { + *out = *in + if in.EnvVars != nil { + in, out := &in.EnvVars, &out.EnvVars + *out = make([]EnvironmentVariable, len(*in)) + copy(*out, *in) + } + if in.Resources != nil { + in, out := &in.Resources, &out.Resources + *out = new(corev1.ResourceRequirements) + (*in).DeepCopyInto(*out) + } + if in.NodeSelector != nil { + in, out := &in.NodeSelector, &out.NodeSelector + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + if in.PvMatchLabels != nil { + in, out := &in.PvMatchLabels, &out.PvMatchLabels + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + if in.ImagePulllPolicy != nil { + in, out := &in.ImagePulllPolicy, &out.ImagePulllPolicy + *out = new(corev1.PullPolicy) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new GsmSpec. +func (in *GsmSpec) DeepCopy() *GsmSpec { + if in == nil { + return nil + } + out := new(GsmSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *GsmStatus) DeepCopyInto(out *GsmStatus) { + *out = *in + if in.Shards != nil { + in, out := &in.Shards, &out.Shards + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + if in.Details != nil { + in, out := &in.Details, &out.Details + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new GsmStatus. +func (in *GsmStatus) DeepCopy() *GsmStatus { + if in == nil { + return nil + } + out := new(GsmStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *GsmStatusDetails) DeepCopyInto(out *GsmStatusDetails) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new GsmStatusDetails. +func (in *GsmStatusDetails) DeepCopy() *GsmStatusDetails { + if in == nil { + return nil + } + out := new(GsmStatusDetails) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *OCIConfigSpec) DeepCopyInto(out *OCIConfigSpec) { + *out = *in + if in.ConfigMapName != nil { + in, out := &in.ConfigMapName, &out.ConfigMapName + *out = new(string) + **out = **in + } + if in.SecretName != nil { + in, out := &in.SecretName, &out.SecretName + *out = new(string) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new OCIConfigSpec. +func (in *OCIConfigSpec) DeepCopy() *OCIConfigSpec { + if in == nil { + return nil + } + out := new(OCIConfigSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PasswordSpec) DeepCopyInto(out *PasswordSpec) { + *out = *in + if in.K8sSecretName != nil { + in, out := &in.K8sSecretName, &out.K8sSecretName + *out = new(string) + **out = **in + } + if in.OCISecretOCID != nil { + in, out := &in.OCISecretOCID, &out.OCISecretOCID + *out = new(string) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PasswordSpec. +func (in *PasswordSpec) DeepCopy() *PasswordSpec { + if in == nil { + return nil + } + out := new(PasswordSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PortMapping) DeepCopyInto(out *PortMapping) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PortMapping. +func (in *PortMapping) DeepCopy() *PortMapping { + if in == nil { + return nil + } + out := new(PortMapping) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *ShardSpec) DeepCopyInto(out *ShardSpec) { + *out = *in + if in.EnvVars != nil { + in, out := &in.EnvVars, &out.EnvVars + *out = make([]EnvironmentVariable, len(*in)) + copy(*out, *in) + } + if in.Resources != nil { + in, out := &in.Resources, &out.Resources + *out = new(corev1.ResourceRequirements) + (*in).DeepCopyInto(*out) + } + if in.NodeSelector != nil { + in, out := &in.NodeSelector, &out.NodeSelector + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + if in.PvAnnotations != nil { + in, out := &in.PvAnnotations, &out.PvAnnotations + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + if in.PvMatchLabels != nil { + in, out := &in.PvMatchLabels, &out.PvMatchLabels + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + if in.ImagePulllPolicy != nil { + in, out := &in.ImagePulllPolicy, &out.ImagePulllPolicy + *out = new(corev1.PullPolicy) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ShardSpec. +func (in *ShardSpec) DeepCopy() *ShardSpec { + if in == nil { + return nil + } + out := new(ShardSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *ShardingDatabase) DeepCopyInto(out *ShardingDatabase) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + in.Spec.DeepCopyInto(&out.Spec) + in.Status.DeepCopyInto(&out.Status) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ShardingDatabase. +func (in *ShardingDatabase) DeepCopy() *ShardingDatabase { + if in == nil { + return nil + } + out := new(ShardingDatabase) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *ShardingDatabase) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *ShardingDatabaseList) DeepCopyInto(out *ShardingDatabaseList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]ShardingDatabase, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ShardingDatabaseList. +func (in *ShardingDatabaseList) DeepCopy() *ShardingDatabaseList { + if in == nil { + return nil + } + out := new(ShardingDatabaseList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *ShardingDatabaseList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *ShardingDatabaseSpec) DeepCopyInto(out *ShardingDatabaseSpec) { + *out = *in + if in.Shard != nil { + in, out := &in.Shard, &out.Shard + *out = make([]ShardSpec, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } + if in.Catalog != nil { + in, out := &in.Catalog, &out.Catalog + *out = make([]CatalogSpec, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } + if in.Gsm != nil { + in, out := &in.Gsm, &out.Gsm + *out = make([]GsmSpec, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } + if in.PortMappings != nil { + in, out := &in.PortMappings, &out.PortMappings + *out = make([]PortMapping, len(*in)) + copy(*out, *in) + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ShardingDatabaseSpec. +func (in *ShardingDatabaseSpec) DeepCopy() *ShardingDatabaseSpec { + if in == nil { + return nil + } + out := new(ShardingDatabaseSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *ShardingDatabaseStatus) DeepCopyInto(out *ShardingDatabaseStatus) { + *out = *in + if in.Shard != nil { + in, out := &in.Shard, &out.Shard + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + if in.Catalog != nil { + in, out := &in.Catalog, &out.Catalog + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + in.Gsm.DeepCopyInto(&out.Gsm) + if in.CrdStatus != nil { + in, out := &in.CrdStatus, &out.CrdStatus + *out = make([]v1.Condition, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ShardingDatabaseStatus. +func (in *ShardingDatabaseStatus) DeepCopy() *ShardingDatabaseStatus { + if in == nil { + return nil + } + out := new(ShardingDatabaseStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *SingleInstanceDatabase) DeepCopyInto(out *SingleInstanceDatabase) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + in.Spec.DeepCopyInto(&out.Spec) + in.Status.DeepCopyInto(&out.Status) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SingleInstanceDatabase. +func (in *SingleInstanceDatabase) DeepCopy() *SingleInstanceDatabase { + if in == nil { + return nil + } + out := new(SingleInstanceDatabase) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *SingleInstanceDatabase) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *SingleInstanceDatabaseAdminPassword) DeepCopyInto(out *SingleInstanceDatabaseAdminPassword) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SingleInstanceDatabaseAdminPassword. +func (in *SingleInstanceDatabaseAdminPassword) DeepCopy() *SingleInstanceDatabaseAdminPassword { + if in == nil { + return nil + } + out := new(SingleInstanceDatabaseAdminPassword) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *SingleInstanceDatabaseImage) DeepCopyInto(out *SingleInstanceDatabaseImage) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SingleInstanceDatabaseImage. +func (in *SingleInstanceDatabaseImage) DeepCopy() *SingleInstanceDatabaseImage { + if in == nil { + return nil + } + out := new(SingleInstanceDatabaseImage) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *SingleInstanceDatabaseInitParams) DeepCopyInto(out *SingleInstanceDatabaseInitParams) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SingleInstanceDatabaseInitParams. +func (in *SingleInstanceDatabaseInitParams) DeepCopy() *SingleInstanceDatabaseInitParams { + if in == nil { + return nil + } + out := new(SingleInstanceDatabaseInitParams) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *SingleInstanceDatabaseList) DeepCopyInto(out *SingleInstanceDatabaseList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]SingleInstanceDatabase, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SingleInstanceDatabaseList. +func (in *SingleInstanceDatabaseList) DeepCopy() *SingleInstanceDatabaseList { + if in == nil { + return nil + } + out := new(SingleInstanceDatabaseList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *SingleInstanceDatabaseList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *SingleInstanceDatabasePersistence) DeepCopyInto(out *SingleInstanceDatabasePersistence) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SingleInstanceDatabasePersistence. +func (in *SingleInstanceDatabasePersistence) DeepCopy() *SingleInstanceDatabasePersistence { + if in == nil { + return nil + } + out := new(SingleInstanceDatabasePersistence) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *SingleInstanceDatabaseSpec) DeepCopyInto(out *SingleInstanceDatabaseSpec) { + *out = *in + if in.NodeSelector != nil { + in, out := &in.NodeSelector, &out.NodeSelector + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + out.AdminPassword = in.AdminPassword + out.Image = in.Image + out.Persistence = in.Persistence + out.InitParams = in.InitParams +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SingleInstanceDatabaseSpec. +func (in *SingleInstanceDatabaseSpec) DeepCopy() *SingleInstanceDatabaseSpec { + if in == nil { + return nil + } + out := new(SingleInstanceDatabaseSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *SingleInstanceDatabaseStatus) DeepCopyInto(out *SingleInstanceDatabaseStatus) { + *out = *in + if in.Nodes != nil { + in, out := &in.Nodes, &out.Nodes + *out = make([]string, len(*in)) + copy(*out, *in) + } + if in.StandbyDatabases != nil { + in, out := &in.StandbyDatabases, &out.StandbyDatabases + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + if in.Conditions != nil { + in, out := &in.Conditions, &out.Conditions + *out = make([]v1.Condition, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } + out.InitParams = in.InitParams + out.Persistence = in.Persistence +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SingleInstanceDatabaseStatus. +func (in *SingleInstanceDatabaseStatus) DeepCopy() *SingleInstanceDatabaseStatus { + if in == nil { + return nil + } + out := new(SingleInstanceDatabaseStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *WalletSpec) DeepCopyInto(out *WalletSpec) { + *out = *in + if in.Name != nil { + in, out := &in.Name, &out.Name + *out = new(string) + **out = **in + } + in.Password.DeepCopyInto(&out.Password) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new WalletSpec. +func (in *WalletSpec) DeepCopy() *WalletSpec { + if in == nil { + return nil + } + out := new(WalletSpec) + in.DeepCopyInto(out) + return out +} diff --git a/bundle.Dockerfile b/bundle.Dockerfile new file mode 100644 index 00000000..4ca3c52f --- /dev/null +++ b/bundle.Dockerfile @@ -0,0 +1,25 @@ +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl. +# + +FROM scratch + +# Core bundle labels. +LABEL operators.operatorframework.io.bundle.mediatype.v1=registry+v1 +LABEL operators.operatorframework.io.bundle.manifests.v1=manifests/ +LABEL operators.operatorframework.io.bundle.metadata.v1=metadata/ +LABEL operators.operatorframework.io.bundle.package.v1=oracle-database-operator +LABEL operators.operatorframework.io.bundle.channels.v1=alpha +LABEL operators.operatorframework.io.bundle.channel.default.v1=alpha +LABEL operators.operatorframework.io.metrics.mediatype.v1=metrics+v1 +LABEL operators.operatorframework.io.metrics.builder=operator-sdk-v1.6.1+git +LABEL operators.operatorframework.io.metrics.project_layout=go.kubebuilder.io/v2 + +# Labels for testing. +LABEL operators.operatorframework.io.test.mediatype.v1=scorecard+v1 +LABEL operators.operatorframework.io.test.config.v1=tests/scorecard/ + +# Copy files to locations specified by labels. +COPY bundle/manifests /manifests/ +COPY bundle/metadata /metadata/ +COPY bundle/tests/scorecard /tests/scorecard/ diff --git a/commons/annotations/annotations.go b/commons/annotations/annotations.go new file mode 100644 index 00000000..aa4c4e15 --- /dev/null +++ b/commons/annotations/annotations.go @@ -0,0 +1,83 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package annotations + +import ( + "context" + "encoding/json" + + "k8s.io/apimachinery/pkg/types" + "sigs.k8s.io/controller-runtime/pkg/client" +) + +// PatchValue defines a general structure for patching an object +type PatchValue struct { + Op string `json:"op"` + Path string `json:"path"` + Value interface{} `json:"value"` +} + +// SetAnnotations attaches the given metadata to the target object +func SetAnnotations(kubeClient client.Client, obj client.Object, anns map[string]string) error { + payload := []PatchValue{} + + if obj.GetAnnotations() == nil { + payload = append(payload, PatchValue{ + Op: "replace", + Path: "/metadata/annotations", + Value: map[string]string{}, + }) + } + + for key, val := range anns { + payload = append(payload, PatchValue{ + Op: "replace", + Path: "/metadata/annotations/" + key, + Value: val, + }) + } + + payloadBytes, err := json.Marshal(payload) + if err != nil { + return err + } + + patch := client.RawPatch(types.JSONPatchType, payloadBytes) + return kubeClient.Patch(context.TODO(), obj, patch) +} diff --git a/commons/autonomousdatabase/reconciler_util.go b/commons/autonomousdatabase/reconciler_util.go new file mode 100644 index 00000000..2ceac5e6 --- /dev/null +++ b/commons/autonomousdatabase/reconciler_util.go @@ -0,0 +1,127 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package autonomousdatabase + +import ( + "context" + "fmt" + + corev1 "k8s.io/api/core/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/types" + "k8s.io/client-go/util/retry" + "sigs.k8s.io/controller-runtime/pkg/client" + + "github.com/go-logr/logr" + "github.com/oracle/oci-go-sdk/v45/common" + "github.com/oracle/oci-go-sdk/v45/database" + "github.com/oracle/oci-go-sdk/v45/secrets" + + dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + "github.com/oracle/oracle-database-operator/commons/oci" +) + +// SetStatus sets the status subresource. +func SetStatus(kubeClient client.Client, adb *dbv1alpha1.AutonomousDatabase) error { + return retry.RetryOnConflict(retry.DefaultRetry, func() error { + curADB := &dbv1alpha1.AutonomousDatabase{} + + namespacedName := types.NamespacedName{ + Namespace: adb.GetNamespace(), + Name: adb.GetName(), + } + + if err := kubeClient.Get(context.TODO(), namespacedName, curADB); err != nil { + return err + } + + curADB.Status = adb.Status + return kubeClient.Status().Update(context.TODO(), curADB) + }) +} + +func createWalletSecret(kubeClient client.Client, namespacedName types.NamespacedName, data map[string][]byte) error { + // Create the secret with the wallet data + stringData := map[string]string{} + for key, val := range data { + stringData[key] = string(val) + } + + walletSecret := &corev1.Secret{ + ObjectMeta: metav1.ObjectMeta{ + Namespace: namespacedName.Namespace, + Name: namespacedName.Name, + }, + StringData: stringData, + } + + if err := kubeClient.Create(context.TODO(), walletSecret); err != nil { + return err + } + return nil +} + +func CreateWalletSecret(logger logr.Logger, kubeClient client.Client, dbClient database.DatabaseClient, secretClient secrets.SecretsClient, adb *dbv1alpha1.AutonomousDatabase) error { + // Kube Secret which contains Instance Wallet + walletName := adb.Spec.Details.Wallet.Name + if walletName == nil { + walletName = common.String(adb.GetName() + "-instance-wallet") + } + + // No-op if Wallet is already downloaded + walletNamespacedName := types.NamespacedName{ + Namespace: adb.GetNamespace(), + Name: *walletName, + } + walletSecret := &corev1.Secret{} + if err := kubeClient.Get(context.TODO(), walletNamespacedName, walletSecret); err == nil { + return nil + } + + data, err := oci.GetWallet(logger, kubeClient, dbClient, secretClient, adb) + if err != nil { + return err + } + + if err := createWalletSecret(kubeClient, walletNamespacedName, data); err != nil { + return err + } + logger.Info(fmt.Sprintf("Wallet is stored in the Secret %s", *walletName)) + return nil +} diff --git a/commons/database/constants.go b/commons/database/constants.go new file mode 100644 index 00000000..fac4719f --- /dev/null +++ b/commons/database/constants.go @@ -0,0 +1,177 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package commons + +const ORACLE_UID int64 = 54321 + +const ORACLE_GUID int64 = 54321 + +const DBA_GUID int64 = 54322 + +const NoCloneRef string = "Unavailable" + +const GetVersionSQL string = "SELECT VERSION_FULL FROM V\\$INSTANCE;" + +const CheckModesSQL string = "SELECT 'log_mode:' || log_mode AS log_mode ,'flashback_on:' || flashback_on AS flashback_on ,'force_logging:' || force_logging AS force_logging FROM v\\$database;" + +const ListPdbSQL string = "SELECT NAME FROM V\\$PDBS;" + +const CreateChkFileCMD string = "touch \"${ORACLE_BASE}/oradata/.${ORACLE_SID}.nochk\" && sync" + +const RemoveChkFileCMD string = "rm -f \"${ORACLE_BASE}/oradata/.${ORACLE_SID}.nochk\"" + +const CreateDBRecoveryDestCMD string = "mkdir -p ${ORACLE_BASE}/oradata/fast_recovery_area" + +const ConfigureOEMSQL string = "exec DBMS_XDB_CONFIG.SETHTTPSPORT(5500);" + + "\nalter system register;" + +const SetDBRecoveryDestSQL string = "SHOW PARAMETER db_recovery_file_dest;" + + "\nALTER SYSTEM SET db_recovery_file_dest_size=50G scope=both sid='*';" + + "\nALTER SYSTEM SET db_recovery_file_dest='${ORACLE_BASE}/oradata/fast_recovery_area' scope=both sid='*';" + + "\nSHOW PARAMETER db_recovery_file_dest;" + +const ForceLoggingTrueSQL string = "SELECT force_logging FROM v\\$database;" + + "\nALTER DATABASE FORCE LOGGING;" + + "\nALTER SYSTEM SWITCH LOGFILE;" + + "\nSELECT force_logging FROM v\\$database;" + +const ForceLoggingFalseSQL string = "SELECT force_logging FROM v\\$database;" + + "\nALTER DATABASE NO FORCE LOGGING;" + + "\nSELECT force_logging FROM v\\$database;" + +const FlashBackTrueSQL string = "SELECT flashback_on FROM v\\$database;" + + "\nALTER DATABASE FLASHBACK ON;" + + "\nSELECT flashback_on FROM v\\$database;" + +const FlashBackFalseSQL string = "SELECT flashback_on FROM v\\$database;" + + "\nALTER DATABASE FLASHBACK OFF;" + + "\nSELECT flashback_on FROM v\\$database;" + +const ArchiveLogTrueCMD string = CreateChkFileCMD + " && " + + "echo -e \"SHUTDOWN IMMEDIATE; \n STARTUP MOUNT; \n ALTER DATABASE ARCHIVELOG; \n SELECT log_mode FROM v\\$database; \n ALTER DATABASE OPEN;" + + " \n ALTER PLUGGABLE DATABASE ALL OPEN; \n ALTER SYSTEM REGISTER;\" | %s && " + RemoveChkFileCMD + +const ArchiveLogFalseCMD string = CreateChkFileCMD + " && " + + "echo -e \"SHUTDOWN IMMEDIATE; \n STARTUP MOUNT; \n ALTER DATABASE NOARCHIVELOG; \n SELECT log_mode FROM v\\$database; \n ALTER DATABASE OPEN;" + + " \n ALTER PLUGGABLE DATABASE ALL OPEN; \n ALTER SYSTEM REGISTER;\" | %s && " + RemoveChkFileCMD + +const GetDatabaseRoleCMD string = "SELECT DATABASE_ROLE FROM V\\$DATABASE; " + +const DataguardBrokerGetDatabaseCMD string = "SELECT DATABASE || ':' || DATAGUARD_ROLE AS DATABASE FROM V\\$DG_BROKER_CONFIG;" + +const RunDatapatchCMD string = " ( while true; do sleep 60; echo \"Installing patches...\" ; done ) & if ! $ORACLE_HOME/OPatch/datapatch -skip_upgrade_check;" + + " then echo \"Datapatch execution has failed.\" ; else echo \"DONE: Datapatch execution.\" ; fi ; kill -9 $!;" + +const GetSqlpatchDescriptionSQL string = "select TARGET_VERSION || ' (' || PATCH_ID || ')' as patchinfo from dba_registry_sqlpatch order by action_time desc;" + +const GetSqlpatchStatusSQL string = "select status from dba_registry_sqlpatch order by action_time desc;" + +const GetSqlpatchVersionSQL string = "select SOURCE_VERSION || ':' || TARGET_VERSION as versions from dba_registry_sqlpatch order by action_time desc;" + +const GetCheckpointFileCMD string = "find ${ORACLE_BASE}/oradata -name .${ORACLE_SID}${CHECKPOINT_FILE_EXTN} " + +const GetEnterpriseEditionFileCMD string = "if [ -f ${ORACLE_BASE}/oradata/dbconfig/$ORACLE_SID/.docker_enterprise ]; then ls ${ORACLE_BASE}/oradata/dbconfig/$ORACLE_SID/.docker_enterprise; fi " + +const GetStandardEditionFileCMD string = "if [ -f ${ORACLE_BASE}/oradata/dbconfig/$ORACLE_SID/.docker_standard ]; then ls ${ORACLE_BASE}/oradata/dbconfig/$ORACLE_SID/.docker_standard; fi " + +const ReconcileError string = "ReconcileError" + +const ReconcileErrorReason string = "LastReconcileCycleFailed" + +const ReconcileQueued string = "ReconcileQueued" + +const ReconcileQueuedReason string = "LastReconcileCycleQueued" + +const ReconcileCompelete string = "ReconcileComplete" + +const ReconcileCompleteReason string = "LastReconcileCycleCompleted" + +const ReconcileBlocked string = "ReconcileBlocked" + +const ReconcileBlockedReason string = "LastReconcileCycleBlocked" + +const StatusPending string = "Pending" + +const StatusCreating string = "Creating" + +const StatusNotReady string = "Unhealthy" + +const StatusPatching string = "Patching" + +const StatusUpdating string = "Updating" + +const StatusReady string = "Healthy" + +const StatusError string = "Error" + +const ValueUnavailable string = "Unknown" + +const NoExternalIp string = "Node ExternalIP unavailable" + +const WalletPwdCMD string = "export WALLET_PWD=\"`openssl rand -base64 8`1\"" + +const WalletCreateCMD string = "if [[ ! -f ${WALLET_DIR}/ewallet.p12 ]]; then mkdir -p ${WALLET_DIR}/.wallet && (umask 177\ncat > wallet.passwd < /dev/null; do sleep 0.5; done; fi " + +const AlterSgaPgaCpuCMD string = "echo -e \"alter system set sga_target=%dM scope=both; \n alter system set pga_aggregate_target=%dM scope=both; \n alter system set cpu_count=%d; \" | %s " + +const AlterProcessesCMD string = "echo -e \"alter system set processes=%d scope=spfile; \" | %s && " + CreateChkFileCMD + " && " + + "echo -e \"SHUTDOWN IMMEDIATE; \n STARTUP MOUNT; \n ALTER DATABASE OPEN; \n ALTER PLUGGABLE DATABASE ALL OPEN; \n ALTER SYSTEM REGISTER;\" | %s && " + + RemoveChkFileCMD + +const GetInitParamsSQL string = "echo -e \"select name,display_value from v\\$parameter where name in ('sga_target','pga_aggregate_target','cpu_count','processes') order by name asc;\" | %s" + +const UnzipApex string = "if [ -f /opt/oracle/oradata/apex-latest.zip ]; then unzip -o /opt/oracle/oradata/apex-latest.zip -d /opt/oracle/oradata/${ORACLE_SID^^}; else echo \"apex-latest.zip not found\"; fi;" + +const ChownApex string = " chown oracle:oinstall /opt/oracle/oradata/${ORACLE_SID^^}/apex;" + +const InstallApex string = "if [ -f /opt/oracle/oradata/${ORACLE_SID^^}/apex/apexins.sql ]; then ( while true; do sleep 60; echo \"Installing Apex...\" ; done ) & " + + " cd /opt/oracle/oradata/${ORACLE_SID^^}/apex && echo -e \"@apexins.sql SYSAUX SYSAUX TEMP /i/\" | %[1]s && kill -9 $!; else echo \"Apex Folder doesn't exist\" ; fi ;" + +const IsApexInstalled string = "select 'APEXVERSION:'||version as version FROM DBA_REGISTRY WHERE COMP_ID='APEX';" + +const UninstallApex string = "if [ -f /opt/oracle/oradata/${ORACLE_SID^^}/apex/apxremov.sql ]; then ( while true; do sleep 60; echo \"Uninstalling Apex...\" ; done ) & " + + " cd /opt/oracle/oradata/${ORACLE_SID^^}/apex && echo -e \"@apxremov.sql\" | %[1]s && kill -9 $!; else echo \"Apex Folder doesn't exist\" ; fi ;" diff --git a/commons/database/utils.go b/commons/database/utils.go new file mode 100644 index 00000000..4e1a143d --- /dev/null +++ b/commons/database/utils.go @@ -0,0 +1,619 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package commons + +import ( + "bufio" + "bytes" + "context" + "errors" + "fmt" + "math/rand" + "strings" + "time" + + corev1 "k8s.io/api/core/v1" + + apierrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/types" + "k8s.io/apimachinery/pkg/util/wait" + "k8s.io/client-go/kubernetes" + "k8s.io/client-go/kubernetes/scheme" + "k8s.io/client-go/rest" + "k8s.io/client-go/tools/remotecommand" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/event" + ctrllog "sigs.k8s.io/controller-runtime/pkg/log" + "sigs.k8s.io/controller-runtime/pkg/predicate" +) + +// To requeue after 15 secs allowing graceful state changes +var requeueY ctrl.Result = ctrl.Result{Requeue: true, RequeueAfter: 15 * time.Second} +var requeueN ctrl.Result = ctrl.Result{} + +// Filter events that trigger reconcilation +func ResourceEventHandler() predicate.Predicate { + return predicate.Funcs{ + DeleteFunc: func(e event.DeleteEvent) bool { + // Evaluates to false if the object has been confirmed deleted. + return !e.DeleteStateUnknown + }, + UpdateFunc: func(e event.UpdateEvent) bool { + oldPodObject, oldOk := e.ObjectOld.(*corev1.Pod) + newPodObject, newOk := e.ObjectNew.(*corev1.Pod) + + // Handling the Pod Ready Status Changes . + if oldOk && newOk { + oldStatus, newStatus := "", "" + for _, condition := range oldPodObject.Status.Conditions { + if condition.Type == "Ready" { + oldStatus = string(condition.Status) + break + } + } + for _, condition := range newPodObject.Status.Conditions { + if condition.Type == "Ready" { + newStatus = string(condition.Status) + break + } + } + // If Pod Ready Status Changed , reconcile + if oldStatus != newStatus { + return true + } + } + // Ignore updates to CR status in which case metadata.Generation does not change + // Reconcile if object Deletion Timestamp Set + return e.ObjectOld.GetGeneration() != e.ObjectNew.GetGeneration() || + e.ObjectOld.GetDeletionTimestamp() != nil || e.ObjectNew.GetDeletionTimestamp() != nil + }, + } +} + +// getLabelsForController returns the labels for selecting the resources +func GetLabelsForController(version string, name string) map[string]string { + if version != "" { + return map[string]string{"app": name, "version": version} + } else { + return map[string]string{"app": name} + } +} + +// getPodNames returns the pod names of the array of pods passed in +func GetPodNames(pods []corev1.Pod) []string { + var podNames []string + for _, pod := range pods { + podNames = append(podNames, pod.Name) + } + return podNames +} + +// Poll up to timeout seconds for Object to change state +// Returns an error if the object never changes the state +func WaitForStatusChange(r client.Reader, objName string, namespace string, + ctx context.Context, req ctrl.Request, timeout time.Duration, object string, statusChange string) error { + return wait.PollImmediate(time.Second, timeout, IsStatusChanged(r, objName, namespace, ctx, req, object, statusChange)) +} + +// returns a func() that returns true if an object is confirmed to be created or deleted . else false +func IsStatusChanged(r client.Reader, objName string, namespace string, + ctx context.Context, req ctrl.Request, object string, statusChange string) wait.ConditionFunc { + return func() (bool, error) { + log := ctrllog.FromContext(ctx).WithValues("IsStatusChanged", req.NamespacedName) + log.Info("", " Waiting for ", object, " ", statusChange, " Name : ", objName) + var obj client.Object + if object == "pod" { + obj = &corev1.Pod{} + } + if object == "pvc" { + obj = &corev1.PersistentVolumeClaim{} + } + if object == "svc" { + obj = &corev1.Service{} + } + err := r.Get(ctx, types.NamespacedName{Name: objName, Namespace: namespace}, obj) + + if object == "pod" { + if statusChange == "deletion" { + if err != nil && apierrors.IsNotFound(err) { + log.Error(err, "Pod Already Deleted", "SingleInstanceDatabase.Namespace", namespace, "Pod.Name", objName) + // No need to wait if Pod already Deleted + return true, nil + + } else if err != nil { + log.Error(err, "Failed to get the Pod Details") + // return the false,err that reconciler failed to get pods + return false, err + + } + log.Info("Found the Pod ", "Name :", objName) + if deletionTimeStamp := obj.GetDeletionTimestamp(); deletionTimeStamp != nil { + // Pod Found and Status changed . Return true,nil as No wait required + return true, nil + } else { + // Pod Found and Status not changed . Return false,nil as wait required till the status changes + return false, nil + } + } + if statusChange == "creation" { + if err != nil && apierrors.IsNotFound(err) { + log.Info("Creating new POD", "SingleInstanceDatabase.Namespace", namespace, "Obj.Name", objName) + // wait as Pod is being created + return false, nil + + } else if err != nil { + log.Error(err, "Failed to get the Pod Details") + // return the false,err that reconciler failed to get pod + return false, err + + } + log.Info("POD Created ", "Name :", objName) + return true, nil + } + } + if object == "pvc" { + if err != nil && apierrors.IsNotFound(err) { + log.Info("Creating new PVC", "SingleInstanceDatabase.Namespace", namespace, "Obj.Name", objName) + // wait as Pvc is being created + return false, nil + + } else if err != nil { + log.Error(err, "Failed to get the pvc Details") + // return the false,err that reconciler failed to get pvc + return false, err + + } + log.Info("PVC Created ", "Name :", objName) + return true, nil + } + if object == "svc" { + if err != nil && apierrors.IsNotFound(err) { + log.Info("Creating new Service", "SingleInstanceDatabase.Namespace", namespace, "Obj.Name", objName) + // wait as Service is being created + return false, nil + + } else if err != nil { + log.Error(err, "Failed to get the Service Details") + // return the false,err that reconciler failed to get Service + return false, err + + } + log.Info("Service Created ", "Name :", objName) + return true, nil + } + return false, nil + + } + +} + +// Execs into podName and executes command +func ExecCommand(r client.Reader, config *rest.Config, podName string, namespace string, containerName string, + ctx context.Context, req ctrl.Request, nologCommand bool, command ...string) (string, error) { + + log := ctrllog.FromContext(ctx).WithValues("ExecCommand", req.NamespacedName) + if !nologCommand { + log.Info("Executing Command :") + log.Info(strings.Join(command, " ")) + } + if config == nil { + log.Info("r.Config nil") + return "Error", nil + } + var ( + execOut bytes.Buffer + execErr bytes.Buffer + ) + pod := &corev1.Pod{} + err := r.Get(ctx, types.NamespacedName{Name: podName, Namespace: namespace}, pod) + if err != nil { + return "", fmt.Errorf("could not get pod info: %v", err) + } else { + log.Info("Pod Found", "Name : ", podName) + } + client, err := kubernetes.NewForConfig(config) + if err != nil { + log.Error(err, "config error") + } + rc := client.CoreV1().RESTClient() + if rc == nil { + return "RESTClient Error", nil + } + rcreq := rc.Post().Resource("pods").Name(podName).Namespace(namespace).SubResource("exec") + rcreq.VersionedParams(&corev1.PodExecOptions{ + Command: command, + Stdout: true, + Stderr: true, + Container: containerName, + }, scheme.ParameterCodec) + exec, err := remotecommand.NewSPDYExecutor(config, "POST", rcreq.URL()) + if err != nil { + return "", fmt.Errorf("failed to init executor: %v", err) + } + err = exec.Stream(remotecommand.StreamOptions{ + Stdout: &execOut, + Stderr: &execErr, + Tty: false, + }) + if err != nil { + return "", fmt.Errorf("could not execute: %v", err) + } + if execErr.Len() > 0 { + return "", fmt.Errorf("stderr: %v", execErr.String()) + } + return execOut.String(), nil +} + +// returns a randomString +func GenerateRandomString(n int) string { + var letters = []rune("abcdefghijklmnopqrstuvwxyz0123456789") + + s := make([]rune, n) + for i := range s { + s[i] = letters[rand.Intn(len(letters))] + } + return string(s) +} + +// retuns Ready Pod,No of replicas ( Only running and Pending Pods) ,available pods , Total No of Pods of a particular CRD +func FindPods(r client.Reader, version string, image string, name string, namespace string, ctx context.Context, + req ctrl.Request) (corev1.Pod, int, []corev1.Pod, int, error) { + + log := ctrllog.FromContext(ctx).WithValues("FindPods", req.NamespacedName) + + // "available" stores list of pods which can be deleted while scaling down i.e the pods other than one of Ready Pods + // There are multiple ready pods possible in OracleRestDataService , while others have atmost one readyPod + var available []corev1.Pod + var readyPod corev1.Pod // To Store the Ready Pod ( Pod that Passed Readiness Probe . Will be shown as 1/1 Running ) + + podList := &corev1.PodList{} + listOpts := []client.ListOption{client.InNamespace(namespace), client.MatchingLabels(GetLabelsForController(version, name))} + + // List retrieves list of objects for a given namespace and list options. + if err := r.List(ctx, podList, listOpts...); err != nil { + log.Error(err, "Failed to list pods of "+name, "Namespace", namespace, "Name", name) + return readyPod, 0, available, 0, err + } + + // r.List() lists all the pods in running, pending,terminating stage matching listOpts . so filter them + // Fetch the Running and Pending Pods + + podsMarkedToBeDeleted := 0 + for _, pod := range podList.Items { + // Return pods having Image = image (or) if image = ""(Needed in case when called findpods with "" image) + if pod.Spec.Containers[0].Image == image || image == "" { + if pod.ObjectMeta.DeletionTimestamp != nil { + podsMarkedToBeDeleted += 1 + continue + } + if pod.Status.Phase == corev1.PodRunning || pod.Status.Phase == corev1.PodPending { + if len(pod.Status.ContainerStatuses) > 0 && pod.Status.ContainerStatuses[0].Ready && readyPod.Name == "" { + readyPod = pod + } else { + available = append(available, pod) + } + } + } + } + podNames := GetPodNames(available) + replicasFound := len(podNames) + + if readyPod.Name != "" { + replicasFound = replicasFound + 1 // if one of the pods is ready , its not there in "available" , So do a "+1" + log.Info("Ready Pod ", "Name :", readyPod.Name) + } else { + log.Info("No " + name + " Pod is Ready ") + } + + log.Info(name+" Pods Available ( Other Than Ready Pod )", " Names :", podNames) + log.Info("Total No Of "+name+" PODS", "Count", replicasFound) + + return readyPod, replicasFound, available, podsMarkedToBeDeleted, nil +} + +// returns flashBackStatus,archiveLogStatus,forceLoggingStatus of Primary Pod +func CheckDBConfig(readyPod corev1.Pod, r client.Reader, config *rest.Config, + ctx context.Context, req ctrl.Request, edition string) (bool, bool, bool, ctrl.Result) { + + log := ctrllog.FromContext(ctx).WithValues("CheckDBParams", req.NamespacedName) + + var forceLoggingStatus bool + var flashBackStatus bool + var archiveLogStatus bool + if readyPod.Name == "" { + log.Info("No Pod is Ready") + // As No pod is ready now , turn on mode when pod is ready . so requeue the request + return false, false, false, requeueY + + } else { + out, err := ExecCommand(r, config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | %s", CheckModesSQL, GetSqlClient(edition))) + if err != nil { + log.Error(err, "Error in ExecCommand()") + return false, false, false, requeueY + } else { + log.Info("CheckModes Output") + log.Info(out) + + if strings.Contains(out, "log_mode:NOARCHIVELOG") { + archiveLogStatus = false + } + if strings.Contains(out, "log_mode:ARCHIVELOG") { + archiveLogStatus = true + } + if strings.Contains(out, "flashback_on:NO") { + flashBackStatus = false + } + if strings.Contains(out, "flashback_on:YES") { + flashBackStatus = true + } + if strings.Contains(out, "force_logging:NO") { + forceLoggingStatus = false + } + if strings.Contains(out, "force_logging:YES") { + forceLoggingStatus = true + } + } + log.Info("FlashBackStatus ", "Status :", flashBackStatus) + log.Info("ArchiveLogStatus ", "Status :", archiveLogStatus) + log.Info("ForceLoggingStatus ", "Status :", forceLoggingStatus) + } + + return flashBackStatus, archiveLogStatus, forceLoggingStatus, requeueN +} + +// CHECKS IF SID IN DATABASES SLICE , AND ITS DGROLE +func IsDatabaseFound(sid string, databases []string, dgrole string) (bool, bool) { + found := false + isdgrole := false + for i := 0; i < len(databases); i++ { + splitstr := strings.Split(databases[i], ":") + if strings.EqualFold(sid, splitstr[0]) { + found = true + if strings.EqualFold(dgrole, splitstr[1]) { + isdgrole = true + } + break + } + } + return found, isdgrole +} + +// Returns a Primary Database in "databases" slice +func GetPrimaryDatabase(databases []string) string { + primary := "" + for i := 0; i < len(databases); i++ { + splitstr := strings.Split(databases[i], ":") + if strings.ToUpper(splitstr[1]) == "PRIMARY" { + primary = splitstr[0] + break + } + } + return primary +} + +// Returns the databases in DG config . +func GetDatabasesInDgConfig(readyPod corev1.Pod, r client.Reader, + config *rest.Config, ctx context.Context, req ctrl.Request) ([]string, string, error) { + log := ctrllog.FromContext(ctx).WithValues("GetDatabasesInDgConfig", req.NamespacedName) + + // ## FIND DATABASES PRESENT IN DG CONFIGURATION + out, err := ExecCommand(r, config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | sqlplus -s / as sysdba ", DataguardBrokerGetDatabaseCMD)) + if err != nil { + return []string{}, "", err + } + log.Info("GetDatabasesInDgConfig Output") + log.Info(out) + + if !strings.Contains(out, "no rows selected") && !strings.Contains(out, "ORA-") { + out1 := strings.Replace(out, " ", "_", -1) + // filtering output and storing databses in dg configuration in "databases" slice + databases := strings.Fields(out1) + + // first 2 values in the slice will be column name(DATABASES) and a seperator(--------------) . so take the slice from position [2:] + databases = databases[2:] + return databases, out, nil + } + return []string{}, out, errors.New("databases in DG config is nil") +} + +// Returns Database version +func GetDatabaseVersion(readyPod corev1.Pod, r client.Reader, + config *rest.Config, ctx context.Context, req ctrl.Request, edition string) (string, string, error) { + log := ctrllog.FromContext(ctx).WithValues("GetDatabaseVersion", req.NamespacedName) + + // ## FIND DATABASES PRESENT IN DG CONFIGURATION + out, err := ExecCommand(r, config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | %s", GetVersionSQL, GetSqlClient(edition))) + if err != nil { + return "", "", err + } + log.Info("GetDatabaseVersion Output") + log.Info(out) + + if !strings.Contains(out, "no rows selected") && !strings.Contains(out, "ORA-") { + out1 := strings.Replace(out, " ", "_", -1) + // filtering output and storing databses in dg configuration in "databases" slice + out2 := strings.Fields(out1) + + // first 2 values in the slice will be column name(VERSION) and a seperator(--------------) . so the version would be out2[2] + version := out2[2] + return version, out, nil + } + return "", out, errors.New("database version is nil") + +} + +// Fetch role by quering the DB +func GetDatabaseRole(readyPod corev1.Pod, r client.Reader, + config *rest.Config, ctx context.Context, req ctrl.Request, edition string) (string, error) { + log := ctrllog.FromContext(ctx).WithValues("GetDatabaseRole", req.NamespacedName) + + out, err := ExecCommand(r, config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | %s", GetDatabaseRoleCMD, GetSqlClient(edition))) + if err != nil { + return "", err + } + log.Info(out) + if !strings.Contains(out, "no rows selected") && !strings.Contains(out, "ORA-") { + out = strings.Replace(out, " ", "_", -1) + // filtering output and storing databse_role in "database_role" + databaseRole := strings.Fields(out)[2] + + // first 2 values in the slice will be column name(DATABASE_ROLE) and a seperator(--------------) . + return databaseRole, nil + } + return "", errors.New("database role is nil") +} + +// Returns true if any of the pod in 'pods' is with pod.Status.Phase == phase +func IsAnyPodWithStatus(pods []corev1.Pod, phase corev1.PodPhase) (bool, corev1.Pod) { + anyPodWithPhase := false + var podWithPhase corev1.Pod + for _, pod := range pods { + if pod.Status.Phase == phase { + anyPodWithPhase = true + podWithPhase = pod + break + } + } + return anyPodWithPhase, podWithPhase +} + +// Convert "sqlplus -s " output to array of lines +func StringToLines(s string) (lines []string, err error) { + scanner := bufio.NewScanner(strings.NewReader(s)) + i := 0 + for scanner.Scan() { + // store from line 3 as line 0 would be blank, line 1 - column_name , line 2 - seperator (----) + if i > 2 { + lines = append(lines, scanner.Text()) + } + i++ + } + err = scanner.Err() + return +} + +// Get Node Ip to display in ConnectionString +// Returns Node External Ip if exists ; else InternalIP +func GetNodeIp(r client.Reader, ctx context.Context, req ctrl.Request) string { + + log := ctrllog.FromContext(ctx).WithValues("GetNodeIp", req.NamespacedName) + + readyPod, _, available, _, err := FindPods(r, "", "", req.Name, req.Namespace, ctx, req) + if err != nil { + log.Error(err, err.Error()) + return "" + } + if readyPod.Name != "" { + available = append(available, readyPod) + } + nodeip := "" + for _, pod := range available { + if nodeip == "" { + nodeip = pod.Status.HostIP + } + if pod.Status.HostIP < nodeip { + nodeip = pod.Status.HostIP + } + } + + node := &corev1.Node{} + err = r.Get(ctx, types.NamespacedName{Name: nodeip, Namespace: req.Namespace}, node) + + if err == nil { + for _, address := range node.Status.Addresses { + if address.Type == "ExternalIP" { + nodeip = address.Address + } + } + } + + return nodeip +} + +// Get Datapatch Status +func GetSqlpatchStatus(r client.Reader, config *rest.Config, readyPod corev1.Pod, ctx context.Context, req ctrl.Request) (string, string, string, error) { + log := ctrllog.FromContext(ctx).WithValues("getSqlpatchStatus", req.NamespacedName) + + // GET SQLPATCH STATUS ( INITIALIZE ==> END ==> SUCCESS (or) WITH ERRORS ) + out, err := ExecCommand(r, config, readyPod.Name, readyPod.Namespace, "", + ctx, req, false, "bash", "-c", fmt.Sprintf("echo -e \"%s\" | sqlplus -s / as sysdba", GetSqlpatchStatusSQL)) + if err != nil { + log.Error(err, err.Error()) + return "", "", "", err + } + log.Info("GetSqlpatchStatusSQL Output") + log.Info(out) + sqlpatchStatuses, err := StringToLines(out) + if err != nil { + log.Error(err, err.Error()) + return "", "", "", err + } + if len(sqlpatchStatuses) == 0 { + return "", "", "", nil + } + //GET SQLPATCH VERSIONS (SOURCE & TARGET) + out, err = ExecCommand(r, config, readyPod.Name, readyPod.Namespace, "", + ctx, req, false, "bash", "-c", fmt.Sprintf("echo -e \"%s\" | sqlplus -s / as sysdba", GetSqlpatchVersionSQL)) + if err != nil { + log.Error(err, err.Error()) + return "", "", "", err + } + log.Info("GetSqlpatchVersionSQL Output") + log.Info(out) + + sqlpatchVersions, err := StringToLines(out) + if err != nil { + log.Error(err, err.Error()) + return "", "", "", err + } + splitstr := strings.Split(sqlpatchVersions[0], ":") + return sqlpatchStatuses[0], splitstr[0], splitstr[1], nil +} + +func GetSqlClient(edition string) string { + if edition == "express" { + return "su -p oracle -c \"sqlplus -s / as sysdba\"" + } + return "sqlplus -s / as sysdba" +} diff --git a/commons/finalizer/finalizer.go b/commons/finalizer/finalizer.go new file mode 100644 index 00000000..bafe110e --- /dev/null +++ b/commons/finalizer/finalizer.go @@ -0,0 +1,122 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package finalizer + +import ( + "context" + "encoding/json" + + "k8s.io/apimachinery/pkg/types" + "sigs.k8s.io/controller-runtime/pkg/client" +) + +// name of our custom finalizer +var finalizerName = "database.oracle.com/dbcsfinalizer" + +// HasFinalizer returns true if the finalizer exists in the object metadata +func HasFinalizer(obj client.Object) bool { + finalizer := obj.GetFinalizers() + return containsString(finalizer, finalizerName) +} + +// Register adds the finalizer and patch the object +func Register(kubeClient client.Client, obj client.Object) error { + finalizer := obj.GetFinalizers() + finalizer = append(finalizer, finalizerName) + return setFinalizer(kubeClient, obj, finalizer) +} + +// Unregister removes the finalizer and patch the object +func Unregister(kubeClient client.Client, obj client.Object) error { + finalizer := obj.GetFinalizers() + finalizer = removeString(finalizer, finalizerName) + return setFinalizer(kubeClient, obj, finalizer) +} + +// Helper functions to check and remove string from a slice of strings. +func containsString(slice []string, s string) bool { + for _, item := range slice { + if item == s { + return true + } + } + return false +} + +func removeString(slice []string, s string) (result []string) { + for _, item := range slice { + if item == s { + continue + } + result = append(result, item) + } + return +} + +type patchValue struct { + Op string `json:"op"` + Path string `json:"path"` + Value interface{} `json:"value"` +} + +func setFinalizer(kubeClient client.Client, dbcs client.Object, finalizer []string) error { + payload := []patchValue{} + + if dbcs.GetFinalizers() == nil { + payload = append(payload, patchValue{ + Op: "replace", + Path: "/metadata/finalizers", + Value: []string{}, + }) + } + + payload = append(payload, patchValue{ + Op: "replace", + Path: "/metadata/finalizers", + Value: finalizer, + }) + + payloadBytes, err := json.Marshal(payload) + if err != nil { + return err + } + + patch := client.RawPatch(types.JSONPatchType, payloadBytes) + return kubeClient.Patch(context.TODO(), dbcs, patch) +} diff --git a/commons/oci/database.go b/commons/oci/database.go new file mode 100644 index 00000000..985d299b --- /dev/null +++ b/commons/oci/database.go @@ -0,0 +1,524 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package oci + +import ( + "context" + "errors" + "fmt" + "math" + "time" + + "github.com/go-logr/logr" + "github.com/oracle/oci-go-sdk/v45/common" + "github.com/oracle/oci-go-sdk/v45/database" + "github.com/oracle/oci-go-sdk/v45/secrets" + "github.com/oracle/oci-go-sdk/v45/workrequests" + "sigs.k8s.io/controller-runtime/pkg/client" + + corev1 "k8s.io/api/core/v1" + "k8s.io/apimachinery/pkg/types" + + dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" +) + +// CreateAutonomousDatabase sends a request to OCI to provision a database and returns the AutonomousDatabase OCID. +func CreateAutonomousDatabase(logger logr.Logger, kubeClient client.Client, dbClient database.DatabaseClient, secretClient secrets.SecretsClient, adb *dbv1alpha1.AutonomousDatabase) (*database.CreateAutonomousDatabaseResponse, error) { + adminPassword, err := getAdminPassword(logger, kubeClient, secretClient, adb) + if err != nil { + return nil, err + } + + createAutonomousDatabaseDetails := database.CreateAutonomousDatabaseDetails{ + CompartmentId: adb.Spec.Details.CompartmentOCID, + DbName: adb.Spec.Details.DbName, + CpuCoreCount: adb.Spec.Details.CPUCoreCount, + DataStorageSizeInTBs: adb.Spec.Details.DataStorageSizeInTBs, + AdminPassword: common.String(adminPassword), + DisplayName: adb.Spec.Details.DisplayName, + IsAutoScalingEnabled: adb.Spec.Details.IsAutoScalingEnabled, + IsDedicated: adb.Spec.Details.IsDedicated, + DbVersion: adb.Spec.Details.DbVersion, + DbWorkload: database.CreateAutonomousDatabaseBaseDbWorkloadEnum( + adb.Spec.Details.DbWorkload), + SubnetId: adb.Spec.Details.SubnetOCID, + NsgIds: adb.Spec.Details.NsgOCIDs, + } + + createAutonomousDatabaseRequest := database.CreateAutonomousDatabaseRequest{ + CreateAutonomousDatabaseDetails: createAutonomousDatabaseDetails, + } + + resp, err := dbClient.CreateAutonomousDatabase(context.TODO(), createAutonomousDatabaseRequest) + if err != nil { + return nil, err + } + + return &resp, nil +} + +// Get the desired admin password from either Kubernetes Secret or OCI Vault Secret. +func getAdminPassword(logger logr.Logger, kubeClient client.Client, secretClient secrets.SecretsClient, adb *dbv1alpha1.AutonomousDatabase) (string, error) { + if adb.Spec.Details.AdminPassword.K8sSecretName != nil { + logger.Info(fmt.Sprintf("Getting admin password from Secret %s", *adb.Spec.Details.AdminPassword.K8sSecretName)) + + namespacedName := types.NamespacedName{ + Namespace: adb.GetNamespace(), + Name: *adb.Spec.Details.AdminPassword.K8sSecretName, + } + + key := *adb.Spec.Details.AdminPassword.K8sSecretName + adminPassword, err := getValueFromKubeSecret(kubeClient, namespacedName, key) + if err != nil { + return "", err + } + return adminPassword, nil + + } else if adb.Spec.Details.AdminPassword.OCISecretOCID != nil { + logger.Info(fmt.Sprintf("Getting admin password from OCI Vault Secret OCID %s", *adb.Spec.Details.AdminPassword.OCISecretOCID)) + + adminPassword, err := getValueFromVaultSecret(secretClient, *adb.Spec.Details.AdminPassword.OCISecretOCID) + if err != nil { + return "", err + } + return adminPassword, nil + } + return "", errors.New("should provide either AdminPasswordSecret or AdminPasswordOCID") +} + +func getValueFromKubeSecret(kubeClient client.Client, namespacedName types.NamespacedName, key string) (string, error) { + secret := &corev1.Secret{} + if err := kubeClient.Get(context.TODO(), namespacedName, secret); err != nil { + return "", err + } + + val, ok := secret.Data[key] + if !ok { + return "", errors.New("Secret key not found: " + key) + } + return string(val), nil +} + +// GetAutonomousDatabaseResource gets Autonomous Database information from a remote instance +// and return an AutonomousDatabase object +func GetAutonomousDatabaseResource(logger logr.Logger, dbClient database.DatabaseClient, adb *dbv1alpha1.AutonomousDatabase) (*dbv1alpha1.AutonomousDatabase, error) { + getAutonomousDatabaseRequest := database.GetAutonomousDatabaseRequest{ + AutonomousDatabaseId: adb.Spec.Details.AutonomousDatabaseOCID, + } + + response, err := dbClient.GetAutonomousDatabase(context.TODO(), getAutonomousDatabaseRequest) + if err != nil { + return nil, err + } + + returnedADB := adb.UpdateAttrFromOCIAutonomousDatabase(response.AutonomousDatabase) + + logger.Info("Get information from remote AutonomousDatabase successfully") + return returnedADB, nil +} + +// isAttrChanged checks if the values of last successful object and current object are different. +// The function returns false if the types are mismatch or unknown. +// The function returns false if the current object has zero value (not applicable for boolean type). +func isAttrChanged(lastSucObj interface{}, curObj interface{}) bool { + switch curObj.(type) { + case string: // Enum + // type check + lastSucString, ok := lastSucObj.(string) + if !ok { + return false + } + curString := curObj.(string) + + if curString != "" && (lastSucString != curString) { + return true + } + case *int: + // type check + lastSucIntPtr, ok := lastSucObj.(*int) + if !ok { + return false + } + curIntPtr, ok := curObj.(*int) + + if lastSucIntPtr != nil && curIntPtr != nil && *curIntPtr != 0 && *lastSucIntPtr != *curIntPtr { + return true + } + case *string: + // type check + lastSucStringPtr, ok := lastSucObj.(*string) + if !ok { + return false + } + curStringPtr := curObj.(*string) + + if lastSucStringPtr != nil && curStringPtr != nil && *curStringPtr != "" && *lastSucStringPtr != *curStringPtr { + return true + } + case *bool: + // type check + lastSucBoolPtr, ok := lastSucObj.(*bool) + if !ok { + return false + } + curBoolPtr := curObj.(*bool) + + // For boolean type, we don't have to check zero value + if lastSucBoolPtr != nil && curBoolPtr != nil && *lastSucBoolPtr != *curBoolPtr { + return true + } + case []string: + // type check + lastSucSlice, ok := lastSucObj.([]string) + if !ok { + return false + } + + curSlice := curObj.([]string) + if curSlice == nil { + return false + } else if len(lastSucSlice) != len(curSlice) { + return true + } + + for i, v := range lastSucSlice { + if v != curSlice[i] { + return true + } + } + case map[string]string: + // type check + lastSucMap, ok := lastSucObj.(map[string]string) + if !ok { + return false + } + + curMap := curObj.(map[string]string) + if curMap == nil { + return false + } else if len(lastSucMap) != len(curMap) { + return true + } + + for k, v := range lastSucMap { + if w, ok := curMap[k]; !ok || v != w { + return true + } + } + } + return false +} + +// UpdateGeneralAndPasswordAttributes updates the general and password attributes of the Autonomous Database. +// Based on the responses from OCI calls, we can split the attributes into the following five categories. +// AutonomousDatabaseOCID, CompartmentOCID, IsDedicated, and LifecycleState are excluded since they not applicable in updateAutonomousDatabaseRequest. +// Except for category 1, category 2 and 3 cannot be updated at the same time, i.e.,we can at most update category 1 plus another category 2,or 3. +// 1. General attribute: including DisplayName, DbName, DbWorkload, DbVersion, freeformTags, subnetOCID, nsgOCIDs, and whitelistedIPs. The general attributes can be updated together with one of the other categories in the same request. +// 2. Scale attribute: includining IsAutoScalingEnabled, CpuCoreCount and DataStorageSizeInTBs. +// 3. Password attribute: including AdminPasswordSecret and AdminPasswordOCID +// From the above rules, we group general and password attributes and send the update together in the same request, and then send the scale update in another request. +func UpdateGeneralAndPasswordAttributes(logger logr.Logger, kubeClient client.Client, dbClient database.DatabaseClient, + secretClient secrets.SecretsClient, curADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { + var shouldSendRequest = false + + lastSucSpec, err := curADB.GetLastSuccessfulSpec() + if err != nil { + return resp, err + } + + // Prepare the update request + updateAutonomousDatabaseDetails := database.UpdateAutonomousDatabaseDetails{} + + if isAttrChanged(lastSucSpec.Details.DisplayName, curADB.Spec.Details.DisplayName) { + updateAutonomousDatabaseDetails.DisplayName = curADB.Spec.Details.DisplayName + shouldSendRequest = true + } + if isAttrChanged(lastSucSpec.Details.DbName, curADB.Spec.Details.DbName) { + updateAutonomousDatabaseDetails.DbName = curADB.Spec.Details.DbName + shouldSendRequest = true + } + if isAttrChanged(lastSucSpec.Details.DbWorkload, curADB.Spec.Details.DbWorkload) { + updateAutonomousDatabaseDetails.DbWorkload = database.UpdateAutonomousDatabaseDetailsDbWorkloadEnum(curADB.Spec.Details.DbWorkload) + shouldSendRequest = true + } + if isAttrChanged(lastSucSpec.Details.DbVersion, curADB.Spec.Details.DbVersion) { + updateAutonomousDatabaseDetails.DbVersion = curADB.Spec.Details.DbVersion + shouldSendRequest = true + } + + if isAttrChanged(lastSucSpec.Details.FreeformTags, curADB.Spec.Details.FreeformTags) { + updateAutonomousDatabaseDetails.FreeformTags = curADB.Spec.Details.FreeformTags + shouldSendRequest = true + } + + if isAttrChanged(lastSucSpec.Details.FreeformTags, curADB.Spec.Details.FreeformTags) { + updateAutonomousDatabaseDetails.FreeformTags = curADB.Spec.Details.FreeformTags + shouldSendRequest = true + } + if isAttrChanged(lastSucSpec.Details.SubnetOCID, curADB.Spec.Details.SubnetOCID) { + updateAutonomousDatabaseDetails.SubnetId = curADB.Spec.Details.SubnetOCID + shouldSendRequest = true + } + if isAttrChanged(lastSucSpec.Details.NsgOCIDs, curADB.Spec.Details.NsgOCIDs) { + updateAutonomousDatabaseDetails.NsgIds = curADB.Spec.Details.NsgOCIDs + shouldSendRequest = true + } + + if isAttrChanged(lastSucSpec.Details.AdminPassword.K8sSecretName, curADB.Spec.Details.AdminPassword.K8sSecretName) || + isAttrChanged(lastSucSpec.Details.AdminPassword.OCISecretOCID, curADB.Spec.Details.AdminPassword.OCISecretOCID) { + // Get the adminPassword + var adminPassword string + + adminPassword, err = getAdminPassword(logger, kubeClient, secretClient, curADB) + if err != nil { + return + } + updateAutonomousDatabaseDetails.AdminPassword = common.String(adminPassword) + + shouldSendRequest = true + } + + // Send the request only when something changes + if shouldSendRequest { + + logger.Info("Sending general attributes and ADMIN password update request") + + updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ + // AutonomousDatabaseId: common.String(curADB.Spec.Details.AutonomousDatabaseOCID), + AutonomousDatabaseId: curADB.Spec.Details.AutonomousDatabaseOCID, + UpdateAutonomousDatabaseDetails: updateAutonomousDatabaseDetails, + } + + resp, err = dbClient.UpdateAutonomousDatabase(context.TODO(), updateAutonomousDatabaseRequest) + } + + return +} + +// UpdateScaleAttributes updates the scale attributes of the Autonomous Database +// Refer to UpdateGeneralAndPasswordAttributes for more details about how and why we separate the attributes in different calls. +func UpdateScaleAttributes(logger logr.Logger, kubeClient client.Client, dbClient database.DatabaseClient, + curADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { + var shouldSendRequest = false + + lastSucSpec, err := curADB.GetLastSuccessfulSpec() + if err != nil { + return resp, err + } + + // Prepare the update request + updateAutonomousDatabaseDetails := database.UpdateAutonomousDatabaseDetails{} + + if isAttrChanged(lastSucSpec.Details.DataStorageSizeInTBs, curADB.Spec.Details.DataStorageSizeInTBs) { + updateAutonomousDatabaseDetails.DataStorageSizeInTBs = curADB.Spec.Details.DataStorageSizeInTBs + shouldSendRequest = true + } + if isAttrChanged(lastSucSpec.Details.CPUCoreCount, curADB.Spec.Details.CPUCoreCount) { + updateAutonomousDatabaseDetails.CpuCoreCount = curADB.Spec.Details.CPUCoreCount + shouldSendRequest = true + } + if isAttrChanged(lastSucSpec.Details.IsAutoScalingEnabled, curADB.Spec.Details.IsAutoScalingEnabled) { + updateAutonomousDatabaseDetails.IsAutoScalingEnabled = curADB.Spec.Details.IsAutoScalingEnabled + shouldSendRequest = true + } + + // Don't send the request if nothing is changed + if shouldSendRequest { + + logger.Info("Sending scale attributes update request") + + updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ + // AutonomousDatabaseId: common.String(curADB.Spec.Details.AutonomousDatabaseOCID), + AutonomousDatabaseId: curADB.Spec.Details.AutonomousDatabaseOCID, + UpdateAutonomousDatabaseDetails: updateAutonomousDatabaseDetails, + } + + resp, err = dbClient.UpdateAutonomousDatabase(context.TODO(), updateAutonomousDatabaseRequest) + } + + return +} + +// SetAutonomousDatabaseLifecycleState starts or stops AutonomousDatabase in OCI based on the LifeCycleState attribute +func SetAutonomousDatabaseLifecycleState(logger logr.Logger, dbClient database.DatabaseClient, adb *dbv1alpha1.AutonomousDatabase) (resp interface{}, err error) { + lastSucSpec, err := adb.GetLastSuccessfulSpec() + if err != nil { + return resp, err + } + + // Return if the desired lifecycle state is the same as the current lifecycle state + if adb.Spec.Details.LifecycleState == lastSucSpec.Details.LifecycleState { + return nil, nil + } + + switch string(adb.Spec.Details.LifecycleState) { + case string(database.AutonomousDatabaseLifecycleStateAvailable): + logger.Info("Sending start request to the Autonomous Database " + *adb.Spec.Details.DbName) + + resp, err = startAutonomousDatabase(dbClient, *adb.Spec.Details.AutonomousDatabaseOCID) + if err != nil { + return + } + + case string(database.AutonomousDatabaseLifecycleStateStopped): + logger.Info("Sending stop request to the Autonomous Database " + *adb.Spec.Details.DbName) + + resp, err = stopAutonomousDatabase(dbClient, *adb.Spec.Details.AutonomousDatabaseOCID) + if err != nil { + return + } + + case string(database.AutonomousDatabaseLifecycleStateTerminated): + // Special case. + if adb.Spec.Details.LifecycleState == database.AutonomousDatabaseLifecycleStateTerminating { + break + } + logger.Info("Sending teminate request to the Autonomous Database " + *adb.Spec.Details.DbName) + + resp, err = DeleteAutonomousDatabase(dbClient, *adb.Spec.Details.AutonomousDatabaseOCID) + if err != nil { + return + } + + default: + err = fmt.Errorf("invalid lifecycleState value: currently the operator only accept %s, %s and %s as the value of the lifecycleState parameter", + database.AutonomousDatabaseLifecycleStateAvailable, + database.AutonomousDatabaseLifecycleStateStopped, + database.AutonomousDatabaseLifecycleStateTerminated) + } + + return +} + +// startAutonomousDatabase starts an Autonomous Database in OCI +func startAutonomousDatabase(dbClient database.DatabaseClient, adbOCID string) (resp database.StartAutonomousDatabaseResponse, err error) { + startRequest := database.StartAutonomousDatabaseRequest{ + AutonomousDatabaseId: common.String(adbOCID), + } + + resp, err = dbClient.StartAutonomousDatabase(context.Background(), startRequest) + return +} + +// stopAutonomousDatabase stops an Autonomous Database in OCI +func stopAutonomousDatabase(dbClient database.DatabaseClient, adbOCID string) (resp database.StopAutonomousDatabaseResponse, err error) { + stopRequest := database.StopAutonomousDatabaseRequest{ + AutonomousDatabaseId: common.String(adbOCID), + } + + resp, err = dbClient.StopAutonomousDatabase(context.Background(), stopRequest) + return +} + +// DeleteAutonomousDatabase terminates an Autonomous Database in OCI +func DeleteAutonomousDatabase(dbClient database.DatabaseClient, adbOCID string) (resp database.DeleteAutonomousDatabaseResponse, err error) { + + deleteRequest := database.DeleteAutonomousDatabaseRequest{ + AutonomousDatabaseId: common.String(adbOCID), + } + + resp, err = dbClient.DeleteAutonomousDatabase(context.Background(), deleteRequest) + return +} + +func WaitUntilWorkCompleted(logger logr.Logger, workClient workrequests.WorkRequestClient, opcWorkRequestID *string) error { + if opcWorkRequestID == nil { + return nil + } + + logger.Info("Waiting for the work request to finish. opcWorkRequestID = " + *opcWorkRequestID) + + retryPolicy := getCompleteWorkRetryPolicy() + // Apply wait until work complete retryPolicy + workRequest := workrequests.GetWorkRequestRequest{ + WorkRequestId: opcWorkRequestID, + RequestMetadata: common.RequestMetadata{ + RetryPolicy: &retryPolicy, + }, + } + + // GetWorkRequest retries until the work status is SUCCEEDED + if _, err := workClient.GetWorkRequest(context.TODO(), workRequest); err != nil { + return err + } + + return nil +} + +func getCompleteWorkRetryPolicy() common.RetryPolicy { + shouldRetry := func(r common.OCIOperationResponse) bool { + if _, isServiceError := common.IsServiceError(r.Error); isServiceError { + // Don't retry if it's service error. Sometimes it could be network error or other errors which prevents + // request send to server; we do the retry in these cases. + return false + } + + if converted, ok := r.Response.(workrequests.GetWorkRequestResponse); ok { + // do the retry until WorkReqeut Status is Succeeded - ignore case (BMI-2652) + return converted.Status != workrequests.WorkRequestStatusSucceeded + } + + return true + } + + return getRetryPolicy(shouldRetry) +} + +func getConflictRetryPolicy() common.RetryPolicy { + // retry for 409 conflict status code + shouldRetry := func(r common.OCIOperationResponse) bool { + return r.Error != nil && r.Response.HTTPResponse().StatusCode == 409 + } + + return getRetryPolicy(shouldRetry) +} + +func getRetryPolicy(retryOperation func(common.OCIOperationResponse) bool) common.RetryPolicy { + // maximum times of retry + attempts := uint(10) + + nextDuration := func(r common.OCIOperationResponse) time.Duration { + // you might want wait longer for next retry when your previous one failed + // this function will return the duration as: + // 1s, 2s, 4s, 8s, 16s, 32s, 64s etc... + return time.Duration(math.Pow(float64(2), float64(r.AttemptNumber-1))) * time.Second + } + + return common.NewRetryPolicy(attempts, retryOperation, nextDuration) +} diff --git a/commons/oci/provider.go b/commons/oci/provider.go new file mode 100644 index 00000000..f5bd6cda --- /dev/null +++ b/commons/oci/provider.go @@ -0,0 +1,134 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package oci + +import ( + "context" + "errors" + + "github.com/oracle/oci-go-sdk/v45/common" + "github.com/oracle/oci-go-sdk/v45/common/auth" + + "sigs.k8s.io/controller-runtime/pkg/client" + + corev1 "k8s.io/api/core/v1" + "k8s.io/apimachinery/pkg/types" +) + +const ( + regionKey = "region" + fingerprintKey = "fingerprint" + userKey = "user" + tenancyKey = "tenancy" + passphraseKey = "passphrase" + privatekeyKey = "privatekey" +) + +type APIKeyAuth struct { + ConfigMapName *string + SecretName *string + Namespace string +} + +func GetOCIProvider(kubeClient client.Client, authData APIKeyAuth) (common.ConfigurationProvider, error) { + if authData.ConfigMapName != nil && authData.SecretName != nil { + provider, err := getProviderWithAPIKey(kubeClient, authData) + if err != nil { + return nil, err + } + + return provider, nil + } else if authData.ConfigMapName == nil && authData.SecretName == nil { + return auth.InstancePrincipalConfigurationProvider() + } else { + return nil, errors.New("You have to provide both the OCI ConfigMap and the privateKey to authorize with API signing key, " + + "or leave them both empty to authorize with Instance Principal. Check if the spec configuration is correct.") + } +} + +func getProviderWithAPIKey(kubeClient client.Client, authData APIKeyAuth) (common.ConfigurationProvider, error) { + var region, fingerprint, user, tenancy, passphrase, privatekeyValue string + + // Read ConfigMap + configMapNamespacedName := types.NamespacedName{ + Namespace: authData.Namespace, + Name: *authData.ConfigMapName, + } + ociConfigMap := &corev1.ConfigMap{} + if err := kubeClient.Get(context.TODO(), configMapNamespacedName, ociConfigMap); err != nil { + return nil, err + } + + for key, val := range ociConfigMap.Data { + if key == regionKey { + region = val + } else if key == fingerprintKey { + fingerprint = val + } else if key == userKey { + user = val + } else if key == tenancyKey { + tenancy = val + } else if key == passphraseKey { + passphrase = val + } else { + return nil, errors.New("Unable to identify the key: " + key) + } + } + + // Read Secret + secretNamespacedName := types.NamespacedName{ + Namespace: authData.Namespace, + Name: *authData.SecretName, + } + + privatekeySecret := &corev1.Secret{} + if err := kubeClient.Get(context.TODO(), secretNamespacedName, privatekeySecret); err != nil { + return nil, err + } + + for key, val := range privatekeySecret.Data { + if key == privatekeyKey { + privatekeyValue = string(val) + } else { + return nil, errors.New("Unable to identify the key: " + key) + } + } + + return common.NewRawConfigurationProvider(tenancy, user, region, fingerprint, privatekeyValue, &passphrase), nil +} diff --git a/commons/oci/vault.go b/commons/oci/vault.go new file mode 100644 index 00000000..6bf790aa --- /dev/null +++ b/commons/oci/vault.go @@ -0,0 +1,67 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package oci + +import ( + "context" + "encoding/base64" + + "github.com/oracle/oci-go-sdk/v45/common" + "github.com/oracle/oci-go-sdk/v45/secrets" +) + +func getValueFromVaultSecret(secretClient secrets.SecretsClient, vaultSecretOCID string) (string, error) { + request := secrets.GetSecretBundleRequest{ + SecretId: common.String(vaultSecretOCID), + } + + response, err := secretClient.GetSecretBundle(context.TODO(), request) + if err != nil { + return "", err + } + + base64content := response.SecretBundle.SecretBundleContent.(secrets.Base64SecretBundleContentDetails) + base64String := *base64content.Content + decoded, err := base64.StdEncoding.DecodeString(base64String) + if err != nil { + return "", err + } + + return string(decoded), nil +} diff --git a/commons/oci/wallet.go b/commons/oci/wallet.go new file mode 100644 index 00000000..3faa4f16 --- /dev/null +++ b/commons/oci/wallet.go @@ -0,0 +1,181 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package oci + +import ( + "archive/zip" + "context" + "errors" + "fmt" + "io" + "io/ioutil" + "math" + "time" + + "github.com/go-logr/logr" + "github.com/oracle/oci-go-sdk/v45/common" + "github.com/oracle/oci-go-sdk/v45/database" + "github.com/oracle/oci-go-sdk/v45/secrets" + "sigs.k8s.io/controller-runtime/pkg/client" + + "k8s.io/apimachinery/pkg/types" + + dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" +) + +// GetWallet downloads the wallet using the given information in the AutonomousDatabase object. +// The function then unzips the wallet and returns a map object which holds the byte values of the unzipped files. +func GetWallet(logger logr.Logger, kubeClient client.Client, dbClient database.DatabaseClient, secretClient secrets.SecretsClient, adb *dbv1alpha1.AutonomousDatabase) (map[string][]byte, error) { + // Get the wallet password from Secret then Vault Secret + walletPassword, err := getWalletPassword(logger, kubeClient, secretClient, adb) + if err != nil { + return nil, err + } + + // Request to download a wallet with the given password + resp, err := generateAutonomousDatabaseWallet(dbClient, *adb.Spec.Details.AutonomousDatabaseOCID, walletPassword) + if err != nil { + return nil, err + } + + // Unzip the file + outZip, err := ioutil.TempFile("", "wallet*.zip") + if err != nil { + return nil, err + } + defer outZip.Close() + + if _, err := io.Copy(outZip, resp.Content); err != nil { + return nil, err + } + + data, err := unzipWallet(outZip.Name()) + if err != nil { + return nil, err + } + return data, nil +} + +func getWalletPassword(logger logr.Logger, kubeClient client.Client, secretClient secrets.SecretsClient, adb *dbv1alpha1.AutonomousDatabase) (string, error) { + if adb.Spec.Details.Wallet.Password.K8sSecretName != nil { + logger.Info(fmt.Sprintf("Getting wallet password from Secret %s", *adb.Spec.Details.Wallet.Password.K8sSecretName)) + + namespacedName := types.NamespacedName{ + Namespace: adb.GetNamespace(), + Name: *adb.Spec.Details.Wallet.Password.K8sSecretName, + } + + key := *adb.Spec.Details.Wallet.Password.K8sSecretName + walletPassword, err := getValueFromKubeSecret(kubeClient, namespacedName, key) + if err != nil { + return "", err + } + return walletPassword, nil + + } else if adb.Spec.Details.Wallet.Password.OCISecretOCID != nil { + logger.Info(fmt.Sprintf("Getting wallet password from OCI Vault Secret OCID %s", *adb.Spec.Details.Wallet.Password.OCISecretOCID)) + + walletPassword, err := getValueFromVaultSecret(secretClient, *adb.Spec.Details.Wallet.Password.OCISecretOCID) + if err != nil { + return "", err + } + return walletPassword, nil + } + return "", errors.New("should provide either InstancewalletPasswordSecret or a InstancewalletPasswordId") +} + +func generateAutonomousDatabaseWallet(dbClient database.DatabaseClient, adbOCID string, walletPassword string) (database.GenerateAutonomousDatabaseWalletResponse, error) { + + // maximum times of retry + attempts := uint(10) + + // retry for all non-200 status code + retryOnAllNon200ResponseCodes := func(r common.OCIOperationResponse) bool { + return !(r.Error == nil && 199 < r.Response.HTTPResponse().StatusCode && r.Response.HTTPResponse().StatusCode < 300) + } + + nextDuration := func(r common.OCIOperationResponse) time.Duration { + // Wait longer for next retry when your previous one failed + // this function will return the duration as: + // 1s, 2s, 4s, 8s, 16s, 32s, 64s etc... + return time.Duration(math.Pow(float64(2), float64(r.AttemptNumber-1))) * time.Second + } + + walletRetryPolicy := common.NewRetryPolicy(attempts, retryOnAllNon200ResponseCodes, nextDuration) + + // Download a Wallet + req := database.GenerateAutonomousDatabaseWalletRequest{ + AutonomousDatabaseId: common.String(adbOCID), + GenerateAutonomousDatabaseWalletDetails: database.GenerateAutonomousDatabaseWalletDetails{ + Password: common.String(walletPassword), + }, + RequestMetadata: common.RequestMetadata{ + RetryPolicy: &walletRetryPolicy, + }, + } + + // Send the request using the service client + return dbClient.GenerateAutonomousDatabaseWallet(context.TODO(), req) +} + +func unzipWallet(filename string) (map[string][]byte, error) { + data := map[string][]byte{} + + reader, err := zip.OpenReader(filename) + if err != nil { + return data, err + } + + defer reader.Close() + for _, file := range reader.File { + reader, err := file.Open() + if err != nil { + return data, err + } + + content, err := ioutil.ReadAll(reader) + if err != nil { + return data, err + } + + data[file.Name] = content + } + + return data, nil +} diff --git a/commons/sharding/catalog.go b/commons/sharding/catalog.go new file mode 100644 index 00000000..a24abeda --- /dev/null +++ b/commons/sharding/catalog.go @@ -0,0 +1,462 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package commons + +import ( + "context" + "reflect" + "strconv" + + databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + + "github.com/go-logr/logr" + appsv1 "k8s.io/api/apps/v1" + corev1 "k8s.io/api/core/v1" + "k8s.io/apimachinery/pkg/api/resource" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" +) + +func buildLabelsForCatalog(instance *databasev1alpha1.ShardingDatabase, label string) map[string]string { + return map[string]string{ + "app": "OracleSharding", + "type": "Catalog", + "oralabel": getLabelForCatalog(instance), + } +} + +func getLabelForCatalog(instance *databasev1alpha1.ShardingDatabase) string { + + // if len(OraCatalogSpex.Label) !=0 { + // return OraCatalogSpex.Label + // } + + return instance.Name +} + +func BuildStatefulSetForCatalog(instance *databasev1alpha1.ShardingDatabase, OraCatalogSpex databasev1alpha1.CatalogSpec) *appsv1.StatefulSet { + sfset := &appsv1.StatefulSet{ + TypeMeta: buildTypeMetaForCatalog(), + ObjectMeta: builObjectMetaForCatalog(instance, OraCatalogSpex), + Spec: *buildStatefulSpecForCatalog(instance, OraCatalogSpex), + } + + return sfset +} + +// Function to build TypeMeta +func buildTypeMetaForCatalog() metav1.TypeMeta { + // building TypeMeta + typeMeta := metav1.TypeMeta{ + Kind: "StatefulSet", + APIVersion: "apps/v1", + } + return typeMeta +} + +// Function to build ObjectMeta +func builObjectMetaForCatalog(instance *databasev1alpha1.ShardingDatabase, OraCatalogSpex databasev1alpha1.CatalogSpec) metav1.ObjectMeta { + // building objectMeta + objmeta := metav1.ObjectMeta{ + Name: OraCatalogSpex.Name, + Namespace: instance.Spec.Namespace, + OwnerReferences: getOwnerRef(instance), + Labels: buildLabelsForCatalog(instance, "sharding"), + } + return objmeta +} + +// Function to build Stateful Specs +func buildStatefulSpecForCatalog(instance *databasev1alpha1.ShardingDatabase, OraCatalogSpex databasev1alpha1.CatalogSpec) *appsv1.StatefulSetSpec { + // building Stateful set Specs + + sfsetspec := &appsv1.StatefulSetSpec{ + ServiceName: OraCatalogSpex.Name, + Selector: &metav1.LabelSelector{ + MatchLabels: buildLabelsForCatalog(instance, "sharding"), + }, + Template: corev1.PodTemplateSpec{ + ObjectMeta: metav1.ObjectMeta{ + Labels: buildLabelsForCatalog(instance, "sharding"), + }, + Spec: *buildPodSpecForCatalog(instance, OraCatalogSpex), + }, + VolumeClaimTemplates: volumeClaimTemplatesForCatalog(instance, OraCatalogSpex), + } + // if OraCatalogSpex.OraCatalogSize == 0 { + // OraCatalogSpex.OraCatalogSize = 1 + // sfsetspec.Replicas = &OraCatalogSpex.OraCatalogSize + // } else { + // sfsetspec.Replicas = &OraCatalogSpex.OraCatalogSize + // } + + return sfsetspec +} + +// Function to build PodSpec + +func buildPodSpecForCatalog(instance *databasev1alpha1.ShardingDatabase, OraCatalogSpex databasev1alpha1.CatalogSpec) *corev1.PodSpec { + + user := oraRunAsUser + group := oraFsGroup + spec := &corev1.PodSpec{ + SecurityContext: &corev1.PodSecurityContext{ + RunAsUser: &user, + FSGroup: &group, + }, + InitContainers: buildInitContainerSpecForCatalog(instance, OraCatalogSpex), + Containers: buildContainerSpecForCatalog(instance, OraCatalogSpex), + Volumes: buildVolumeSpecForCatalog(instance, OraCatalogSpex), + } + if len(instance.Spec.DbImagePullSecret) > 0 { + spec.ImagePullSecrets = []corev1.LocalObjectReference{ + { + Name: instance.Spec.DbImagePullSecret, + }, + } + } + + if len(OraCatalogSpex.NodeSelector) > 0 { + spec.NodeSelector = make(map[string]string) + for key, value := range OraCatalogSpex.NodeSelector { + spec.NodeSelector[key] = value + } + } + return spec +} + +// Function to build Volume Spec +func buildVolumeSpecForCatalog(instance *databasev1alpha1.ShardingDatabase, OraCatalogSpex databasev1alpha1.CatalogSpec) []corev1.Volume { + var result []corev1.Volume + result = []corev1.Volume{ + { + Name: OraCatalogSpex.Name + "secretmap-vol3", + VolumeSource: corev1.VolumeSource{ + Secret: &corev1.SecretVolumeSource{ + SecretName: instance.Spec.Secret, + }, + }, + }, + { + Name: OraCatalogSpex.Name + "orascript-vol5", + VolumeSource: corev1.VolumeSource{ + EmptyDir: &corev1.EmptyDirVolumeSource{}, + }, + }, + { + Name: OraCatalogSpex.Name + "oradshm-vol6", + VolumeSource: corev1.VolumeSource{ + EmptyDir: &corev1.EmptyDirVolumeSource{}, + }, + }, + } + + if len(OraCatalogSpex.PvcName) != 0 { + result = append(result, corev1.Volume{Name: OraCatalogSpex.Name + "oradata-vol4", VolumeSource: corev1.VolumeSource{PersistentVolumeClaim: &corev1.PersistentVolumeClaimVolumeSource{ClaimName: OraCatalogSpex.PvcName}}}) + } + + if len(instance.Spec.StagePvcName) != 0 { + result = append(result, corev1.Volume{Name: OraCatalogSpex.Name + "orastage-vol7", VolumeSource: corev1.VolumeSource{PersistentVolumeClaim: &corev1.PersistentVolumeClaimVolumeSource{ClaimName: instance.Spec.StagePvcName}}}) + } + + return result +} + +// Function to build the container Specification +func buildContainerSpecForCatalog(instance *databasev1alpha1.ShardingDatabase, OraCatalogSpex databasev1alpha1.CatalogSpec) []corev1.Container { + // building Continer spec + var result []corev1.Container + containerSpec := corev1.Container{ + Name: OraCatalogSpex.Name, + Image: instance.Spec.DbImage, + SecurityContext: &corev1.SecurityContext{ + Capabilities: &corev1.Capabilities{ + Add: []corev1.Capability{"NET_RAW"}, + }, + }, + Resources: corev1.ResourceRequirements{ + Requests: make(map[corev1.ResourceName]resource.Quantity), + }, + VolumeMounts: buildVolumeMountSpecForCatalog(instance, OraCatalogSpex), + LivenessProbe: &corev1.Probe{ + // TODO: Investigate if it's ok to call status every 10 seconds + FailureThreshold: int32(30), + PeriodSeconds: int32(240), + InitialDelaySeconds: int32(300), + TimeoutSeconds: int32(60), + Handler: corev1.Handler{ + Exec: &corev1.ExecAction{ + Command: getLivenessCmd("CATALOG"), + }, + }, + }, + /** + // Disabling this because the pod is not reachable till the time startup probe completes and without network pod configuration cannot be completed. + StartupProbe: &corev1.Probe{ + // Initial delay should be big, because shard setup takes time + FailureThreshold: int32(30), + PeriodSeconds: int32(120), + Handler: corev1.Handler{ + Exec: &corev1.ExecAction{ + Command: getLivenessCmd("CATALOG"), + }, + }, + }, + **/ + Env: buildEnvVarsSpec(instance, OraCatalogSpex.EnvVars, OraCatalogSpex.Name, "CATALOG", false, ""), + } + if instance.Spec.IsClone { + containerSpec.Command = []string{orainitCmd3} + } + + if OraCatalogSpex.Resources != nil { + containerSpec.Resources = *OraCatalogSpex.Resources + } + // building Complete Container Spec + result = []corev1.Container{ + containerSpec, + } + return result +} + +//Function to build the init Container Spec +func buildInitContainerSpecForCatalog(instance *databasev1alpha1.ShardingDatabase, OraCatalogSpex databasev1alpha1.CatalogSpec) []corev1.Container { + var result []corev1.Container + // building the init Container Spec + privFlag := true + var uid int64 = 0 + var scriptLoc string + if len(instance.Spec.ScriptsLocation) != 0 { + scriptLoc = instance.Spec.ScriptsLocation + } else { + scriptLoc = "WEB" + } + init1spec := corev1.Container{ + Name: OraCatalogSpex.Name + "-init1", + Image: instance.Spec.DbImage, + SecurityContext: &corev1.SecurityContext{ + Privileged: &privFlag, + RunAsUser: &uid, + }, + Command: []string{ + "/bin/bash", + "-c", + getInitContainerCmd(scriptLoc, instance.Name), + }, + VolumeMounts: buildVolumeMountSpecForCatalog(instance, OraCatalogSpex), + } + + // building Complete Init Container Spec + if OraCatalogSpex.ImagePulllPolicy != nil { + init1spec.ImagePullPolicy = *OraCatalogSpex.ImagePulllPolicy + } + result = []corev1.Container{ + init1spec, + } + return result +} + +func buildVolumeMountSpecForCatalog(instance *databasev1alpha1.ShardingDatabase, OraCatalogSpex databasev1alpha1.CatalogSpec) []corev1.VolumeMount { + var result []corev1.VolumeMount + result = append(result, corev1.VolumeMount{Name: OraCatalogSpex.Name + "secretmap-vol3", MountPath: oraSecretMount, ReadOnly: true}) + result = append(result, corev1.VolumeMount{Name: OraCatalogSpex.Name + "-oradata-vol4", MountPath: oraDataMount}) + result = append(result, corev1.VolumeMount{Name: OraCatalogSpex.Name + "orascript-vol5", MountPath: oraScriptMount}) + result = append(result, corev1.VolumeMount{Name: OraCatalogSpex.Name + "oradshm-vol6", MountPath: oraShm}) + + if len(instance.Spec.StagePvcName) != 0 { + result = append(result, corev1.VolumeMount{Name: OraCatalogSpex.Name + "orastage-vol7", MountPath: oraStage}) + } + + return result +} + +func volumeClaimTemplatesForCatalog(instance *databasev1alpha1.ShardingDatabase, OraCatalogSpex databasev1alpha1.CatalogSpec) []corev1.PersistentVolumeClaim { + + var claims []corev1.PersistentVolumeClaim + + if len(OraCatalogSpex.PvcName) != 0 { + return claims + } + + claims = []corev1.PersistentVolumeClaim{ + { + ObjectMeta: metav1.ObjectMeta{ + Name: OraCatalogSpex.Name + "-oradata-vol4", + Namespace: instance.Spec.Namespace, + OwnerReferences: getOwnerRef(instance), + Labels: buildLabelsForCatalog(instance, "sharding"), + }, + Spec: corev1.PersistentVolumeClaimSpec{ + AccessModes: []corev1.PersistentVolumeAccessMode{ + corev1.ReadWriteOnce, + }, + StorageClassName: &instance.Spec.StorageClass, + Resources: corev1.ResourceRequirements{ + Requests: corev1.ResourceList{ + corev1.ResourceStorage: resource.MustParse(strconv.FormatInt(int64(OraCatalogSpex.StorageSizeInGb), 10) + "Gi"), + }, + }, + }, + }, + } + + if len(OraCatalogSpex.PvAnnotations) > 0 { + claims[0].ObjectMeta.Annotations = make(map[string]string) + for key, value := range OraCatalogSpex.PvAnnotations { + claims[0].ObjectMeta.Annotations[key] = value + } + } + + if len(OraCatalogSpex.PvMatchLabels) > 0 { + claims[0].Spec.Selector = &metav1.LabelSelector{MatchLabels: OraCatalogSpex.PvMatchLabels} + } + + return claims +} + +func BuildServiceDefForCatalog(instance *databasev1alpha1.ShardingDatabase, replicaCount int32, OraCatalogSpex databasev1alpha1.CatalogSpec, svctype string) *corev1.Service { + //service := &corev1.Service{} + service := &corev1.Service{ + ObjectMeta: buildSvcObjectMetaForCatalog(instance, replicaCount, OraCatalogSpex, svctype), + Spec: corev1.ServiceSpec{}, + } + + // Check if user want External Svc on each replica pod + if svctype == "external" { + service.Spec.Type = corev1.ServiceTypeLoadBalancer + service.Spec.Selector = getSvcLabelsForCatalog(replicaCount, OraCatalogSpex) + } + + if svctype == "local" { + service.Spec.ClusterIP = corev1.ClusterIPNone + service.Spec.Selector = buildLabelsForCatalog(instance, "sharding") + } + + // build Service Ports Specs to be exposed. If the PortMappings is not set then default ports will be exposed. + service.Spec.Ports = buildSvcPortsDef(instance, "CATALOG") + return service +} + +// Function to build Service ObjectMeta +func buildSvcObjectMetaForCatalog(instance *databasev1alpha1.ShardingDatabase, replicaCount int32, OraCatalogSpex databasev1alpha1.CatalogSpec, svctype string) metav1.ObjectMeta { + // building objectMeta + var svcName string + if svctype == "local" { + svcName = OraCatalogSpex.Name + } + + if svctype == "external" { + svcName = OraCatalogSpex.Name + strconv.FormatInt(int64(replicaCount), 10) + "-svc" + } + + objmeta := metav1.ObjectMeta{ + Name: svcName, + Namespace: instance.Spec.Namespace, + OwnerReferences: getOwnerRef(instance), + Labels: buildLabelsForCatalog(instance, "sharding"), + } + return objmeta +} + +func getSvcLabelsForCatalog(replicaCount int32, OraCatalogSpex databasev1alpha1.CatalogSpec) map[string]string { + + var labelStr map[string]string = make(map[string]string) + if replicaCount == -1 { + labelStr["statefulset.kubernetes.io/pod-name"] = OraCatalogSpex.Name + "-0" + } else { + labelStr["statefulset.kubernetes.io/pod-name"] = OraCatalogSpex.Name + "-" + strconv.FormatInt(int64(replicaCount), 10) + } + + // fmt.Println("Service Selector String Specification", labelStr) + return labelStr +} + +// ======================== update Section ======================== +func UpdateProvForCatalog(instance *databasev1alpha1.ShardingDatabase, + OraCatalogSpex databasev1alpha1.CatalogSpec, kClient client.Client, sfSet *appsv1.StatefulSet, catalogPod *corev1.Pod, logger logr.Logger, +) (ctrl.Result, error) { + + var isUpdate bool = false + var err error + var i int + var msg string + + //msg = "Inside the updateProvForCatalog" + //reqLogger := r.Log.WithValues("Instance.Namespace", instance.Spec.Namespace, "Instance.Name", instance.Name) + LogMessages("DEBUG", msg, nil, instance, logger) + + // Memory Check + //resources := corev1.Pod.Spec.Containers + for i = 0; i < len(catalogPod.Spec.Containers); i++ { + if catalogPod.Spec.Containers[i].Name == sfSet.Name { + shardContaineRes := catalogPod.Spec.Containers[i].Resources + oraSpexRes := OraCatalogSpex.Resources + + if !reflect.DeepEqual(shardContaineRes, oraSpexRes) { + isUpdate = true + } + } + } + + /** + + for i = 0; i < len(sfSet.Spec.VolumeClaimTemplates); i++ { + if sfSet.Spec.VolumeClaimTemplates[i].Name == OraCatalogSpex.Name+"-oradata-vol4" { + volumeSize := sfSet.Spec.VolumeClaimTemplates[i].Size() + if volumeSize != int(OraCatalogSpex.StorageSizeInGb) { + isUpdate = true + } + + } + } + **/ + + if isUpdate { + err = kClient.Update(context.Background(), BuildStatefulSetForCatalog(instance, OraCatalogSpex)) + if err != nil { + msg = "Failed to update Catalog StatefulSet " + "StatefulSet.Name : " + sfSet.Name + LogMessages("Error", msg, nil, instance, logger) + return ctrl.Result{}, err + } + + } + + return ctrl.Result{}, nil +} diff --git a/commons/sharding/exec.go b/commons/sharding/exec.go new file mode 100644 index 00000000..e368bf74 --- /dev/null +++ b/commons/sharding/exec.go @@ -0,0 +1,107 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package commons + +import ( + "bytes" + databasealphav1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + "net/http" + + "github.com/go-logr/logr" + corev1 "k8s.io/api/core/v1" + "k8s.io/client-go/kubernetes" + "k8s.io/client-go/kubernetes/scheme" + "k8s.io/client-go/tools/clientcmd" + "k8s.io/client-go/tools/remotecommand" +) + +// ExecCMDInContainer execute command in first container of a pod +func ExecCommand(podName string, cmd []string, kubeClient kubernetes.Interface, kubeConfig clientcmd.ClientConfig, instance *databasealphav1.ShardingDatabase, logger logr.Logger) (error, string, string) { + + var msg string + var ( + execOut bytes.Buffer + execErr bytes.Buffer + ) + + req := kubeClient.CoreV1().RESTClient(). + Post(). + Namespace(instance.Spec.Namespace). + Resource("pods"). + Name(podName). + SubResource("exec"). + VersionedParams(&corev1.PodExecOptions{ + Command: cmd, + Stdout: true, + Stderr: true, + TTY: true, + }, scheme.ParameterCodec) + + config, err := kubeConfig.ClientConfig() + if err != nil { + return err, "Error Occurred", "Error Occurred" + } + + // Connect to url (constructed from req) using SPDY (HTTP/2) protocol which allows bidirectional streams. + exec, err := remotecommand.NewSPDYExecutor(config, http.MethodPost, req.URL()) + if err != nil { + msg = "Error after executing remotecommand.NewSPDYExecutor" + LogMessages("Error", msg, err, instance, logger) + return err, "Error Occurred", "Error Occurred" + } + + err = exec.Stream(remotecommand.StreamOptions{ + Stdout: &execOut, + Stderr: &execErr, + Tty: true, + }) + if err != nil { + msg = "Command execution failed inside the container!" + LogMessages("DEBUG", msg, err, instance, logger) + if len(execOut.String()) > 0 { + LogMessages("INFO", execOut.String(), nil, instance, logger) + } + if len(execErr.String()) > 0 { + LogMessages("INFO", execErr.String(), nil, instance, logger) + } + return err, execOut.String(), execErr.String() + } + + return nil, execOut.String(), execErr.String() +} diff --git a/commons/sharding/gsm.go b/commons/sharding/gsm.go new file mode 100644 index 00000000..720c00df --- /dev/null +++ b/commons/sharding/gsm.go @@ -0,0 +1,489 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package commons + +import ( + "context" + "fmt" + databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + "reflect" + "strconv" + + "github.com/go-logr/logr" + appsv1 "k8s.io/api/apps/v1" + corev1 "k8s.io/api/core/v1" + "k8s.io/apimachinery/pkg/api/resource" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" +) + +// Constants for hello-stateful StatefulSet & Volumes +func buildLabelsForGsm(instance *databasev1alpha1.ShardingDatabase, label string) map[string]string { + return map[string]string{ + "app": "OracleGsming", + "shard_name": "Gsm", + "oralabel": getLabelForGsm(instance), + } +} + +func getLabelForGsm(instance *databasev1alpha1.ShardingDatabase) string { + + // if len(OraGsmSpex.Label) !=0 { + // return OraGsmSpex.Label + // } + + return instance.Name +} + +func BuildStatefulSetForGsm(instance *databasev1alpha1.ShardingDatabase, OraGsmSpex databasev1alpha1.GsmSpec) *appsv1.StatefulSet { + sfset := &appsv1.StatefulSet{ + TypeMeta: buildTypeMetaForGsm(), + ObjectMeta: builObjectMetaForGsm(instance, OraGsmSpex), + Spec: *buildStatefulSpecForGsm(instance, OraGsmSpex), + } + return sfset +} + +// Function to build TypeMeta +func buildTypeMetaForGsm() metav1.TypeMeta { + // building TypeMeta + typeMeta := metav1.TypeMeta{ + Kind: "StatefulSet", + APIVersion: "apps/v1", + } + return typeMeta +} + +// Function to build ObjectMeta +func builObjectMetaForGsm(instance *databasev1alpha1.ShardingDatabase, OraGsmSpex databasev1alpha1.GsmSpec) metav1.ObjectMeta { + // building objectMeta + objmeta := metav1.ObjectMeta{ + Name: OraGsmSpex.Name, + Namespace: instance.Spec.Namespace, + Labels: buildLabelsForGsm(instance, "sharding"), + OwnerReferences: getOwnerRef(instance), + } + return objmeta +} + +// Function to build Stateful Specs +func buildStatefulSpecForGsm(instance *databasev1alpha1.ShardingDatabase, OraGsmSpex databasev1alpha1.GsmSpec) *appsv1.StatefulSetSpec { + // building Stateful set Specs + + sfsetspec := &appsv1.StatefulSetSpec{ + ServiceName: OraGsmSpex.Name, + Selector: &metav1.LabelSelector{ + MatchLabels: buildLabelsForGsm(instance, "sharding"), + }, + Template: corev1.PodTemplateSpec{ + ObjectMeta: metav1.ObjectMeta{ + Labels: buildLabelsForGsm(instance, "sharding"), + }, + Spec: *buildPodSpecForGsm(instance, OraGsmSpex), + }, + VolumeClaimTemplates: volumeClaimTemplatesForGsm(instance, OraGsmSpex), + } + if OraGsmSpex.Replicas == 0 { + OraGsmSpex.Replicas = 1 + sfsetspec.Replicas = &OraGsmSpex.Replicas + } else { + OraGsmSpex.Replicas = 1 + sfsetspec.Replicas = &OraGsmSpex.Replicas + } + + return sfsetspec +} + +// Function to build PodSpec + +func buildPodSpecForGsm(instance *databasev1alpha1.ShardingDatabase, OraGsmSpex databasev1alpha1.GsmSpec) *corev1.PodSpec { + + user := oraRunAsUser + group := oraFsGroup + spec := &corev1.PodSpec{ + SecurityContext: &corev1.PodSecurityContext{ + RunAsUser: &user, + FSGroup: &group, + }, + InitContainers: buildInitContainerSpecForGsm(instance, OraGsmSpex), + Containers: buildContainerSpecForGsm(instance, OraGsmSpex), + Volumes: buildVolumeSpecForGsm(instance, OraGsmSpex), + } + if len(instance.Spec.GsmImagePullSecret) > 0 { + spec.ImagePullSecrets = []corev1.LocalObjectReference{ + { + Name: instance.Spec.GsmImagePullSecret, + }, + } + } + if len(OraGsmSpex.NodeSelector) > 0 { + spec.NodeSelector = make(map[string]string) + for key, value := range OraGsmSpex.NodeSelector { + spec.NodeSelector[key] = value + } + } + return spec +} + +// Function to build Volume Spec +func buildVolumeSpecForGsm(instance *databasev1alpha1.ShardingDatabase, OraGsmSpex databasev1alpha1.GsmSpec) []corev1.Volume { + var result []corev1.Volume + result = []corev1.Volume{ + { + Name: OraGsmSpex.Name + "secretmap-vol3", + VolumeSource: corev1.VolumeSource{ + Secret: &corev1.SecretVolumeSource{ + SecretName: instance.Spec.Secret, + }, + }, + }, + { + Name: OraGsmSpex.Name + "orascript-vol5", + VolumeSource: corev1.VolumeSource{ + EmptyDir: &corev1.EmptyDirVolumeSource{}, + }, + }, + { + Name: OraGsmSpex.Name + "oradshm-vol6", + VolumeSource: corev1.VolumeSource{ + EmptyDir: &corev1.EmptyDirVolumeSource{}, + }, + }, + } + + if len(OraGsmSpex.PvcName) != 0 { + result = append(result, corev1.Volume{Name: OraGsmSpex.Name + "oradata-vol4", VolumeSource: corev1.VolumeSource{PersistentVolumeClaim: &corev1.PersistentVolumeClaimVolumeSource{ClaimName: OraGsmSpex.PvcName}}}) + } + + if len(instance.Spec.StagePvcName) != 0 { + result = append(result, corev1.Volume{Name: OraGsmSpex.Name + "orastage-vol7", VolumeSource: corev1.VolumeSource{PersistentVolumeClaim: &corev1.PersistentVolumeClaimVolumeSource{ClaimName: instance.Spec.StagePvcName}}}) + } + + return result +} + +// Function to build the container Specification +func buildContainerSpecForGsm(instance *databasev1alpha1.ShardingDatabase, OraGsmSpex databasev1alpha1.GsmSpec) []corev1.Container { + // building Continer spec + var result []corev1.Container + var masterGsmFlag = false + var idx int + // Get the Idx + if instance.Spec.Gsm[0].Name == OraGsmSpex.Name { + masterGsmFlag = true + idx = 0 + } else { + masterGsmFlag = false + idx = 1 + } + directorParams := buildDirectorParams(instance, OraGsmSpex, idx) + + containerSpec := corev1.Container{ + Name: OraGsmSpex.Name, + Image: instance.Spec.GsmImage, + SecurityContext: &corev1.SecurityContext{ + Capabilities: &corev1.Capabilities{ + Add: []corev1.Capability{"NET_RAW"}, + }, + }, + Resources: corev1.ResourceRequirements{ + Requests: make(map[corev1.ResourceName]resource.Quantity), + }, + VolumeMounts: buildVolumeMountSpecForGsm(instance, OraGsmSpex), + LivenessProbe: &corev1.Probe{ + // TODO: Investigate if it's ok to call status every 10 seconds + FailureThreshold: int32(30), + PeriodSeconds: int32(240), + InitialDelaySeconds: int32(300), + TimeoutSeconds: int32(60), + Handler: corev1.Handler{ + Exec: &corev1.ExecAction{ + Command: getLivenessCmd("GSM"), + }, + }, + }, + /** + StartupProbe: &corev1.Probe{ + FailureThreshold: int32(30), + PeriodSeconds: int32(120), + Handler: corev1.Handler{ + Exec: &corev1.ExecAction{ + Command: getLivenessCmd("GSM"), + }, + }, + }, + **/ + Env: buildEnvVarsSpec(instance, OraGsmSpex.EnvVars, OraGsmSpex.Name, "GSM", masterGsmFlag, directorParams), + } + if OraGsmSpex.Resources != nil { + containerSpec.Resources = *OraGsmSpex.Resources + } + // building Complete Container Spec + result = []corev1.Container{ + containerSpec, + } + return result +} + +//Function to build the init Container Spec +func buildInitContainerSpecForGsm(instance *databasev1alpha1.ShardingDatabase, OraGsmSpex databasev1alpha1.GsmSpec) []corev1.Container { + var result []corev1.Container + // building the init Container Spec + privFlag := true + var uid int64 = 0 + var scriptLoc string + if len(instance.Spec.ScriptsLocation) != 0 { + scriptLoc = instance.Spec.ScriptsLocation + } else { + scriptLoc = "WEB" + } + + init1spec := corev1.Container{ + Name: OraGsmSpex.Name + "-init1", + Image: instance.Spec.GsmImage, + SecurityContext: &corev1.SecurityContext{ + Privileged: &privFlag, + RunAsUser: &uid, + }, + Command: []string{ + "/bin/bash", + "-c", + getGsmInitContainerCmd(scriptLoc, instance.Name), + }, + VolumeMounts: buildVolumeMountSpecForGsm(instance, OraGsmSpex), + } + + // building Complete Init Container Spec + if OraGsmSpex.ImagePulllPolicy != nil { + init1spec.ImagePullPolicy = *OraGsmSpex.ImagePulllPolicy + } + result = []corev1.Container{ + init1spec, + } + return result +} + +func buildVolumeMountSpecForGsm(instance *databasev1alpha1.ShardingDatabase, OraGsmSpex databasev1alpha1.GsmSpec) []corev1.VolumeMount { + var result []corev1.VolumeMount + result = append(result, corev1.VolumeMount{Name: OraGsmSpex.Name + "secretmap-vol3", MountPath: oraSecretMount, ReadOnly: true}) + result = append(result, corev1.VolumeMount{Name: OraGsmSpex.Name + "-oradata-vol4", MountPath: oraGsmDataMount}) + result = append(result, corev1.VolumeMount{Name: OraGsmSpex.Name + "orascript-vol5", MountPath: oraScriptMount}) + result = append(result, corev1.VolumeMount{Name: OraGsmSpex.Name + "oradshm-vol6", MountPath: oraShm}) + + if len(instance.Spec.StagePvcName) != 0 { + result = append(result, corev1.VolumeMount{Name: OraGsmSpex.Name + "orastage-vol7", MountPath: oraStage}) + } + + return result +} + +func volumeClaimTemplatesForGsm(instance *databasev1alpha1.ShardingDatabase, OraGsmSpex databasev1alpha1.GsmSpec) []corev1.PersistentVolumeClaim { + + var claims []corev1.PersistentVolumeClaim + + if len(OraGsmSpex.PvcName) != 0 { + return claims + } + + claims = []corev1.PersistentVolumeClaim{ + { + ObjectMeta: metav1.ObjectMeta{ + Name: OraGsmSpex.Name + "-oradata-vol4", + Namespace: instance.Spec.Namespace, + Labels: buildLabelsForGsm(instance, "sharding"), + OwnerReferences: getOwnerRef(instance), + }, + Spec: corev1.PersistentVolumeClaimSpec{ + AccessModes: []corev1.PersistentVolumeAccessMode{ + corev1.ReadWriteOnce, + }, + StorageClassName: &instance.Spec.StorageClass, + Resources: corev1.ResourceRequirements{ + Requests: corev1.ResourceList{ + corev1.ResourceStorage: resource.MustParse(strconv.FormatInt(int64(OraGsmSpex.StorageSizeInGb), 10) + "Gi"), + }, + }, + }, + }, + } + if len(OraGsmSpex.PvMatchLabels) > 0 { + claims[0].Spec.Selector = &metav1.LabelSelector{MatchLabels: OraGsmSpex.PvMatchLabels} + } + + return claims +} + +func BuildServiceDefForGsm(instance *databasev1alpha1.ShardingDatabase, replicaCount int32, OraGsmSpex databasev1alpha1.GsmSpec, svctype string) *corev1.Service { + //service := &corev1.Service{} + service := &corev1.Service{ + ObjectMeta: buildSvcObjectMetaForGsm(instance, replicaCount, OraGsmSpex, svctype), + Spec: corev1.ServiceSpec{}, + } + + // Check if user want External Svc on each replica pod + if svctype == "external" { + service.Spec.Type = corev1.ServiceTypeLoadBalancer + service.Spec.Selector = getSvcLabelsForGsm(replicaCount, OraGsmSpex) + } + + if svctype == "local" { + service.Spec.ClusterIP = corev1.ClusterIPNone + service.Spec.Selector = buildLabelsForGsm(instance, "sharding") + } + + // build Service Ports Specs to be exposed. If the PortMappings is not set then default ports will be exposed. + service.Spec.Ports = buildSvcPortsDef(instance, "GSM") + return service +} + +// Function to build Service ObjectMeta +func buildSvcObjectMetaForGsm(instance *databasev1alpha1.ShardingDatabase, replicaCount int32, OraGsmSpex databasev1alpha1.GsmSpec, svctype string) metav1.ObjectMeta { + // building objectMeta + var svcName string + if svctype == "local" { + svcName = OraGsmSpex.Name + } + + if svctype == "external" { + svcName = OraGsmSpex.Name + strconv.FormatInt(int64(replicaCount), 10) + "-svc" + } + + objmeta := metav1.ObjectMeta{ + Name: svcName, + Namespace: instance.Spec.Namespace, + Labels: buildLabelsForGsm(instance, "sharding"), + OwnerReferences: getOwnerRef(instance), + } + return objmeta +} + +func getSvcLabelsForGsm(replicaCount int32, OraGsmSpex databasev1alpha1.GsmSpec) map[string]string { + + var labelStr map[string]string = make(map[string]string) + if replicaCount == -1 { + labelStr["statefulset.kubernetes.io/pod-name"] = OraGsmSpex.Name + "-0" + } else { + labelStr["statefulset.kubernetes.io/pod-name"] = OraGsmSpex.Name + "-" + strconv.FormatInt(int64(replicaCount), 10) + } + + // fmt.Println("Service Selector String Specification", labelStr) + return labelStr +} + +// This function cleanup the shard from GSM +func OraCleanupForGsm(instance *databasev1alpha1.ShardingDatabase, + OraGsmSpex databasev1alpha1.GsmSpec, + oldReplicaSize int32, + newReplicaSize int32, +) string { + var err1 string + if oldReplicaSize > newReplicaSize { + for replicaCount := (oldReplicaSize - 1); replicaCount > (newReplicaSize - 1); replicaCount-- { + fmt.Println("Deleting the Gsm from GSM" + OraGsmSpex.Name + "-" + strconv.FormatInt(int64(replicaCount), 10)) + } + } + + err1 = "Test" + return err1 +} + +func UpdateProvForGsm(instance *databasev1alpha1.ShardingDatabase, + OraGsmSpex databasev1alpha1.GsmSpec, kClient client.Client, sfSet *appsv1.StatefulSet, gsmPod *corev1.Pod, logger logr.Logger, +) (ctrl.Result, error) { + + var msg string + var size int32 + size = 1 + var isUpdate bool = false + var err error + var i int + + msg = "Inside the updateProvForGsm" + LogMessages("DEBUG", msg, nil, instance, logger) + + // Ensure deployment replicas match the desired state + + // Ensure deployment replicas match the desired state + if sfSet.Spec.Replicas != nil { + if *sfSet.Spec.Replicas != size { + msg = "Current StatefulSet replicas do not match configured Shard Replicas. Gsm is configured with only 1 but current replicas is set with " + strconv.FormatInt(int64(*sfSet.Spec.Replicas), 10) + LogMessages("DEBUG", msg, nil, instance, logger) + isUpdate = true + } + } + // Memory Check + //resources := corev1.Pod.Spec.Containers + for i = 0; i < len(gsmPod.Spec.Containers); i++ { + if gsmPod.Spec.Containers[i].Name == sfSet.Name { + shardContaineRes := gsmPod.Spec.Containers[i].Resources + oraSpexRes := OraGsmSpex.Resources + + if !reflect.DeepEqual(shardContaineRes, oraSpexRes) { + isUpdate = true + } + } + } + + /** + + for i = 0; i < len(sfSet.Spec.VolumeClaimTemplates); i++ { + if sfSet.Spec.VolumeClaimTemplates[i].Name == OraGsmSpex.Name+"-oradata-vol4" { + volumeSize := sfSet.Spec.VolumeClaimTemplates[i].Size() + if volumeSize != int(OraGsmSpex.StorageSizeInGb) { + isUpdate = true + } + + } + } + + **/ + + if isUpdate { + err = kClient.Update(context.Background(), BuildStatefulSetForGsm(instance, OraGsmSpex)) + if err != nil { + msg = "Failed to update Shard StatefulSet " + "StatefulSet.Name : " + sfSet.Name + LogMessages("Error", msg, err, instance, logger) + return ctrl.Result{}, err + } + + } + + return ctrl.Result{}, nil +} diff --git a/commons/sharding/provstatus.go b/commons/sharding/provstatus.go new file mode 100644 index 00000000..ac7e8c02 --- /dev/null +++ b/commons/sharding/provstatus.go @@ -0,0 +1,421 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package commons + +import ( + "fmt" + "strconv" + + databasealphav1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + + "github.com/go-logr/logr" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/client-go/kubernetes" + "k8s.io/client-go/tools/clientcmd" + ctrl "sigs.k8s.io/controller-runtime" +) + +// CHeck if record exist in a struct +func CheckGsmStatusInst(instSpex []databasealphav1.GsmStatusDetails, name string, +) (int, bool) { + + var status bool = false + var idx int + + for i := 0; i < len(instSpex); i++ { + if instSpex[i].Name == name { + status = true + idx = i + break + } + } + + return idx, status +} + +func UpdateGsmStatusData(instance *databasealphav1.ShardingDatabase, Specidx int, state string, kubeClient kubernetes.Interface, kubeConfig clientcmd.ClientConfig, logger logr.Logger, +) { + if state == string(databasealphav1.AvailableState) { + // Evaluate following values only if state is set to available + svcName := instance.Spec.Gsm[Specidx].Name + "-0." + instance.Spec.Gsm[Specidx].Name + k8sExternalSvcName := svcName + strconv.FormatInt(int64(0), 10) + "-svc." + getInstanceNs(instance) + ".svc.cluster.local" + K8sInternalSvcName := svcName + "." + getInstanceNs(instance) + ".svc.cluster.local" + _, K8sInternalSvcIP, _ := GetSvcIp(instance.Spec.Gsm[Specidx].Name+"-0", K8sInternalSvcName, instance, kubeClient, kubeConfig, logger) + _, K8sExternalSvcIP, _ := GetSvcIp(instance.Spec.Gsm[Specidx].Name+"-0", k8sExternalSvcName, instance, kubeClient, kubeConfig, logger) + DbPasswordSecret := instance.Spec.Secret + instance.Status.Gsm.Services = GetGsmServices(instance.Spec.Gsm[Specidx].Name+"-0", instance, kubeClient, kubeConfig, logger) + + // externIp := strings.Replace(K8sInternalSvcIP, "/r/n", "", -1) + // internIp := strings.Replace(K8sExternalSvcIP, "/r/n", "", -1) + + // Populate the Maps + insertOrUpdateGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasealphav1.Name), instance.Spec.Gsm[Specidx].Name) + insertOrUpdateGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasealphav1.DbPasswordSecret), DbPasswordSecret) + if instance.Spec.IsExternalSvc == true { + insertOrUpdateGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasealphav1.K8sExternalSvc), k8sExternalSvcName) + insertOrUpdateGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasealphav1.K8sExternalSvcIP), K8sExternalSvcIP) + } + insertOrUpdateGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasealphav1.K8sInternalSvc), K8sInternalSvcName) + insertOrUpdateGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasealphav1.K8sInternalSvcIP), K8sInternalSvcIP) + insertOrUpdateGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasealphav1.State), state) + } else if state == string(databasealphav1.Terminated) { + removeGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasealphav1.Name)) + removeGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasealphav1.K8sInternalSvc)) + removeGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasealphav1.K8sExternalSvc)) + removeGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasealphav1.K8sExternalSvcIP)) + removeGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasealphav1.K8sInternalSvcIP)) + removeGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasealphav1.Role)) + instance.Status.Gsm.Services = "" + + } else { + insertOrUpdateGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasealphav1.Name), instance.Spec.Gsm[Specidx].Name) + removeGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasealphav1.K8sInternalSvc)) + removeGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasealphav1.K8sExternalSvc)) + removeGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasealphav1.K8sExternalSvcIP)) + removeGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasealphav1.K8sInternalSvcIP)) + removeGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasealphav1.Role)) + instance.Status.Gsm.Services = "" + } + +} + +func UpdateCatalogStatusData(instance *databasealphav1.ShardingDatabase, Specidx int, state string, kubeClient kubernetes.Interface, kubeConfig clientcmd.ClientConfig, logger logr.Logger, +) { + mode := GetDbOpenMode(instance.Spec.Catalog[Specidx].Name+"-0", instance, kubeClient, kubeConfig, logger) + if state == string(databasealphav1.AvailableState) { + // Evaluate following values only if state is set to available + svcName := instance.Spec.Catalog[Specidx].Name + "-0." + instance.Spec.Catalog[Specidx].Name + k8sExternalSvcName := svcName + strconv.FormatInt(int64(0), 10) + "-svc." + getInstanceNs(instance) + ".svc.cluster.local" + K8sInternalSvcName := svcName + "." + getInstanceNs(instance) + ".svc.cluster.local" + _, K8sInternalSvcIP, _ := GetSvcIp(instance.Spec.Catalog[Specidx].Name+"-0", K8sInternalSvcName, instance, kubeClient, kubeConfig, logger) + _, K8sExternalSvcIP, _ := GetSvcIp(instance.Spec.Catalog[Specidx].Name+"-0", k8sExternalSvcName, instance, kubeClient, kubeConfig, logger) + DbPasswordSecret := instance.Spec.Secret + oracleSid := GetSidName(instance.Spec.Catalog[Specidx].EnvVars, instance.Spec.Catalog[Specidx].Name) + oraclePdb := GetPdbName(instance.Spec.Catalog[Specidx].EnvVars, instance.Spec.Catalog[Specidx].Name) + role := GetDbRole(instance.Spec.Catalog[Specidx].Name+"-0", instance, kubeClient, kubeConfig, logger) + // externIp := strings.Replace(K8sInternalSvcIP, "/r/n", "", -1) + // internIp := strings.Replace(K8sExternalSvcIP, "/r/n", "", -1) + + // Populate the Maps + insertOrUpdateCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.Name), instance.Spec.Catalog[Specidx].Name) + insertOrUpdateCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.DbPasswordSecret), DbPasswordSecret) + if instance.Spec.IsExternalSvc == true { + insertOrUpdateCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.K8sExternalSvc), k8sExternalSvcName) + insertOrUpdateCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.K8sExternalSvcIP), K8sExternalSvcIP) + } + insertOrUpdateCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.K8sInternalSvc), K8sInternalSvcName) + insertOrUpdateCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.K8sInternalSvcIP), K8sInternalSvcIP) + insertOrUpdateCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.State), state) + insertOrUpdateCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.OracleSid), oracleSid) + insertOrUpdateCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.OraclePdb), oraclePdb) + insertOrUpdateCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.Role), role) + insertOrUpdateCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.OpenMode), mode) + } else if state == string(databasealphav1.Terminated) { + removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.State)) + removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.Name)) + removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.K8sInternalSvc)) + removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.K8sExternalSvc)) + removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.K8sExternalSvcIP)) + removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.K8sInternalSvcIP)) + removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.Role)) + removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.OraclePdb)) + removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.OracleSid)) + removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.Role)) + removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.OpenMode)) + + } else { + insertOrUpdateCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.State), state) + insertOrUpdateCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.Name), instance.Spec.Catalog[Specidx].Name) + insertOrUpdateCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.OpenMode), mode) + removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.K8sInternalSvc)) + removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.K8sExternalSvc)) + removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.K8sExternalSvcIP)) + removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.K8sInternalSvcIP)) + removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.Role)) + removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.OraclePdb)) + removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.OracleSid)) + removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.Role)) + } + +} + +func UpdateShardStatusData(instance *databasealphav1.ShardingDatabase, Specidx int, state string, kubeClient kubernetes.Interface, kubeConfig clientcmd.ClientConfig, logger logr.Logger, +) { + mode := GetDbOpenMode(instance.Spec.Shard[Specidx].Name+"-0", instance, kubeClient, kubeConfig, logger) + if state == string(databasealphav1.AvailableState) { + // Evaluate following values only if state is set to available + svcName := instance.Spec.Shard[Specidx].Name + "-0." + instance.Spec.Shard[Specidx].Name + k8sExternalSvcName := svcName + strconv.FormatInt(int64(0), 10) + "-svc." + getInstanceNs(instance) + ".svc.cluster.local" + K8sInternalSvcName := svcName + "." + getInstanceNs(instance) + ".svc.cluster.local" + _, K8sInternalSvcIP, _ := GetSvcIp(instance.Spec.Shard[Specidx].Name+"-0", K8sInternalSvcName, instance, kubeClient, kubeConfig, logger) + _, K8sExternalSvcIP, _ := GetSvcIp(instance.Spec.Shard[Specidx].Name+"-0", k8sExternalSvcName, instance, kubeClient, kubeConfig, logger) + DbPasswordSecret := instance.Spec.Secret + oracleSid := GetSidName(instance.Spec.Shard[Specidx].EnvVars, instance.Spec.Shard[Specidx].Name) + oraclePdb := GetPdbName(instance.Spec.Shard[Specidx].EnvVars, instance.Spec.Shard[Specidx].Name) + role := GetDbRole(instance.Spec.Shard[Specidx].Name+"-0", instance, kubeClient, kubeConfig, logger) + + // Populate the Maps + insertOrUpdateShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.Name), instance.Spec.Shard[Specidx].Name) + insertOrUpdateShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.DbPasswordSecret), DbPasswordSecret) + if instance.Spec.IsExternalSvc == true { + insertOrUpdateShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.K8sExternalSvc), k8sExternalSvcName) + insertOrUpdateShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.K8sExternalSvcIP), K8sExternalSvcIP) + } + insertOrUpdateShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.K8sInternalSvc), K8sInternalSvcName) + insertOrUpdateShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.K8sInternalSvcIP), K8sInternalSvcIP) + insertOrUpdateShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.State), state) + insertOrUpdateShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.OracleSid), oracleSid) + insertOrUpdateShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.OraclePdb), oraclePdb) + insertOrUpdateShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.Role), role) + insertOrUpdateShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.OpenMode), mode) + } else if state == string(databasealphav1.Terminated) { + removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.State)) + removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.Name)) + removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.K8sInternalSvc)) + removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.K8sExternalSvc)) + removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.K8sExternalSvcIP)) + removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.K8sInternalSvcIP)) + removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.Role)) + removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.OraclePdb)) + removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.OracleSid)) + removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.Role)) + removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.OpenMode)) + + } else { + insertOrUpdateShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.State), state) + insertOrUpdateShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.Name), instance.Spec.Shard[Specidx].Name) + insertOrUpdateShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.OpenMode), mode) + removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.K8sInternalSvc)) + removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.K8sExternalSvc)) + removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.K8sExternalSvcIP)) + removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.K8sInternalSvcIP)) + removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.Role)) + removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.OraclePdb)) + removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.OracleSid)) + removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.Role)) + } + +} + +func insertOrUpdateShardKeys(instance *databasealphav1.ShardingDatabase, name string, key string, value string) { + newKey := name + "_" + key + if len(instance.Status.Shard) > 0 { + if _, ok := instance.Status.Shard[newKey]; ok { + instance.Status.Shard[newKey] = value + } else { + instance.Status.Shard[newKey] = value + } + } else { + instance.Status.Shard = make(map[string]string) + instance.Status.Shard[newKey] = value + } + +} + +func removeShardKeys(instance *databasealphav1.ShardingDatabase, name string, key string) { + newKey := name + "_" + key + if len(instance.Status.Shard) > 0 { + if _, ok := instance.Status.Shard[newKey]; ok { + delete(instance.Status.Shard, newKey) + } + + } +} + +func insertOrUpdateCatalogKeys(instance *databasealphav1.ShardingDatabase, name string, key string, value string) { + newKey := name + "_" + key + if len(instance.Status.Catalog) > 0 { + if _, ok := instance.Status.Catalog[newKey]; ok { + instance.Status.Catalog[newKey] = value + } else { + instance.Status.Catalog[newKey] = value + } + } else { + instance.Status.Catalog = make(map[string]string) + instance.Status.Catalog[newKey] = value + } + +} + +func removeCatalogKeys(instance *databasealphav1.ShardingDatabase, name string, key string) { + newKey := name + "_" + key + if len(instance.Status.Catalog) > 0 { + if _, ok := instance.Status.Catalog[newKey]; ok { + delete(instance.Status.Catalog, newKey) + } + + } +} + +func insertOrUpdateGsmKeys(instance *databasealphav1.ShardingDatabase, name string, key string, value string) { + newKey := name + "_" + key + if len(instance.Status.Gsm.Details) > 0 { + if _, ok := instance.Status.Gsm.Details[newKey]; ok { + instance.Status.Gsm.Details[newKey] = value + } else { + instance.Status.Gsm.Details[newKey] = value + } + } else { + instance.Status.Gsm.Details = make(map[string]string) + instance.Status.Gsm.Details[newKey] = value + } + +} + +func removeGsmKeys(instance *databasealphav1.ShardingDatabase, name string, key string) { + newKey := name + "_" + key + if len(instance.Status.Gsm.Details) > 0 { + if _, ok := instance.Status.Gsm.Details[newKey]; ok { + delete(instance.Status.Gsm.Details, newKey) + } + + } +} + +func getInstanceNs(instance *databasealphav1.ShardingDatabase) string { + var namespace string + if instance.Spec.Namespace == "" { + namespace = "default" + } else { + namespace = instance.Spec.Namespace + } + return namespace +} + +// File the meta condition and return the meta view +func GetMetaCondition(instance *databasealphav1.ShardingDatabase, result *ctrl.Result, err *error, stateType string, stateMsg string) metav1.Condition { + + return metav1.Condition{ + Type: stateType, + LastTransitionTime: metav1.Now(), + ObservedGeneration: instance.GetGeneration(), + Reason: stateMsg, + Message: fmt.Sprint(*err), + Status: metav1.ConditionTrue, + } +} + +// ======================= CHeck GSM Director Status ============== +func CheckGsmStatus(gname string, instance *databasealphav1.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, +) error { + var err error + var msg string + + msg = "Inside the checkGsmStatus. Checking GSM director in " + GetFmtStr(gname) + " pod." + LogMessages("DEBUG", msg, nil, instance, logger) + + err, _, _ = ExecCommand(gname, getGsmvalidateCmd(), kubeClient, kubeconfig, instance, logger) + if err != nil { + return err + } + + return nil +} + +//============ Functiont o check the status of the Shard and catalog ========= +// ================================ Validate shard =========================== +func ValidateDbSetup(podName string, instance *databasealphav1.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, +) error { + + err, _, _ := ExecCommand(podName, shardValidationCmd(), kubeClient, kubeconfig, instance, logger) + if err != nil { + return fmt.Errorf("Error ocurred while validating the DB Setup") + } + return nil +} + +func UpdateGsmShardStatus(instance *databasealphav1.ShardingDatabase, name string, state string) { + //smap := make(map[string]string) + if _, ok := instance.Status.Gsm.Shards[name]; ok { + instance.Status.Gsm.Shards[name] = state + + } else { + if len(instance.Status.Gsm.Shards) > 0 { + instance.Status.Gsm.Shards[name] = state + } else { + instance.Status.Gsm.Shards = make(map[string]string) + instance.Status.Gsm.Shards[name] = state + + } + } + + if state == "TERMINATED" { + + if _, ok := instance.Status.Gsm.Shards[name]; ok { + delete(instance.Status.Gsm.Shards, name) + } + } + +} + +func GetGsmShardStatus(instance *databasealphav1.ShardingDatabase, name string) string { + if _, ok := instance.Status.Gsm.Shards[name]; ok { + return instance.Status.Gsm.Shards[name] + + } + return "NOSTATE" + +} + +func GetGsmShardStatusKey(instance *databasealphav1.ShardingDatabase, key string) string { + if _, ok := instance.Status.Shard[key]; ok { + return instance.Status.Shard[key] + + } + return "NOSTATE" + +} + +func GetGsmCatalogStatusKey(instance *databasealphav1.ShardingDatabase, key string) string { + if _, ok := instance.Status.Catalog[key]; ok { + return instance.Status.Catalog[key] + + } + return "NOSTATE" + +} + +func GetGsmDetailsSttausKey(instance *databasealphav1.ShardingDatabase, key string) string { + if _, ok := instance.Status.Gsm.Details[key]; ok { + return instance.Status.Gsm.Details[key] + + } + return "NOSTATE" + +} diff --git a/commons/sharding/scommon.go b/commons/sharding/scommon.go new file mode 100644 index 00000000..55fdd5dc --- /dev/null +++ b/commons/sharding/scommon.go @@ -0,0 +1,1252 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package commons + +import ( + "context" + "fmt" + databasealphav1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + + "regexp" + "strconv" + "strings" + + "github.com/go-logr/logr" + "github.com/oracle/oci-go-sdk/v45/common" + "github.com/oracle/oci-go-sdk/v45/ons" + appsv1 "k8s.io/api/apps/v1" + corev1 "k8s.io/api/core/v1" + "k8s.io/apimachinery/pkg/api/errors" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/labels" + "k8s.io/apimachinery/pkg/types" + "k8s.io/apimachinery/pkg/util/intstr" + "k8s.io/apimachinery/pkg/util/rand" + "k8s.io/client-go/kubernetes" + "k8s.io/client-go/tools/clientcmd" + "sigs.k8s.io/controller-runtime/pkg/client" +) + +// Constants for hello-stateful StatefulSet & Volumes +const ( + oraImagePullPolicy = corev1.PullAlways + orainitCmd1 = "set -ex;" + "touch /tmp/test_cmd1.txt" + orainitCmd2 = "set -ex; curl https://codeload.github.com/oracle/db-sharding/tar.gz/master | tar -xz --strip=4 db-sharding-master/docker-based-sharding-deployment/dockerfiles/19.3.0/scripts; cp -i -r scripts/* /opt/oracle/scripts/setup;chmod 777 /opt/oracle/scripts/setup/*" + orainitCmd3 = "/opt/oracle/runOracle.sh.sharding" + orainitCmd4 = "set -ex;" + "touch /tmp/test_cmd4.txt" + orainitCmd5 = "set -ex;" + "[[ `hostname` =~ -([0-9]+)$ ]] || exit 1 ;" + "ordinal=${BASH_REMATCH[1]};" + "cp /mnt/config-map/envfile /mnt/conf.d/; cat /mnt/conf.d/envfile | awk -v env_var=$ordinal -F '=' '{print \"export \" $1\"=\"$2 env_var }' > /tmp/test.env; mv /tmp/test.env /mnt/conf.d/envfile" + oraShardAddCmd = "/bin/python /opt/oracle/scripts/sharding/main.py" + oraRunAsNonRoot = true + oraRunAsUser = int64(54321) + oraFsGroup = int64(54321) + oraScriptMount = "/opt/oracle/scripts/sharding/scripts" + oraDataMount = "/opt/oracle/oradata" + oraGsmDataMount = "/opt/oracle/gsmdata" + oraConfigMapMount = "/mnt/config-map" + oraEnvFileMount = "/mnt/conf.d" + oraEnvFile = "/mnt/conf.d/envfile" + oraSecretMount = "/mnt/secrets" + oraShm = "/dev/shm" + oraStage = "/mnt/stage" + oraDBPort = 1521 + oraGSMPort = 1522 + oraRemoteOnsPort = 6234 + oraLocalOnsPort = 6123 + oraAgentPort = 8080 + ShardingDatabaseFinalizer = "Shardingdb.oracle.com" +) + +// Function to build the env var specification +func buildEnvVarsSpec(instance *databasealphav1.ShardingDatabase, variables []databasealphav1.EnvironmentVariable, name string, restype string, masterFlag bool, directorParams string) []corev1.EnvVar { + var result []corev1.EnvVar + var varinfo string + var sidFlag bool = false + var secretFlag bool = false + var pwdFileFLag bool = false + var pwdKeyFlag bool = false + var pdbFlag bool = false + var sDirectParam bool = false + var sGroup1Params bool = false + var sGroup2Params bool = false + var catalogParams bool = false + var oldPdbFlag bool = false + var oldSidFlag bool = false + var archiveLogFlag bool = false + var shardSetupFlag bool = false + + for _, variable := range variables { + if variable.Name == "ORACLE_SID" { + sidFlag = true + } + if variable.Name == "SECRET_VOLUME" { + secretFlag = true + } + if variable.Name == "COMMON_OS_PWD_FILE" { + pwdFileFLag = true + } + if variable.Name == "PWD_KEY" { + pwdKeyFlag = true + } + if variable.Name == "ORACLE_PDB" { + pdbFlag = true + } + if variable.Name == "SHARD_DIRECTOR_PARAMS" { + sDirectParam = true + } + if variable.Name == "SHARD1_GROUP_PARAMS" { + sGroup1Params = true + } + if variable.Name == "SHARD2_GROUP_PARAMS" { + sGroup2Params = true + } + if variable.Name == "CATALOG_PARAMS" { + catalogParams = true + } + if variable.Name == "OLD_ORACLE_SID" { + oldSidFlag = true + } + if variable.Name == "OLD_ORACLE_PDB" { + oldPdbFlag = true + } + if variable.Name == "SHARD_SETUP" { + shardSetupFlag = true + } + if variable.Name == "OLD_ORACLE_PDB" { + archiveLogFlag = true + } + result = append(result, corev1.EnvVar{Name: variable.Name, Value: variable.Value}) + } + if !shardSetupFlag { + if restype == "SHARD" { + result = append(result, corev1.EnvVar{Name: "SHARD_SETUP", Value: "true"}) + } + if restype == "CATALOG" { + result = append(result, corev1.EnvVar{Name: "SHARD_SETUP", Value: "true"}) + } + if restype == "GSM" { + result = append(result, corev1.EnvVar{Name: "SHARD_SETUP", Value: "true"}) + } + } + if !archiveLogFlag { + if restype == "SHARD" { + result = append(result, corev1.EnvVar{Name: "ENABLE_ARCHIVELOG", Value: "true"}) + } + if restype == "CATALOG" { + result = append(result, corev1.EnvVar{Name: "ENABLE_ARCHIVELOG", Value: "true"}) + } + } + + if !sidFlag { + if restype == "SHARD" { + result = append(result, corev1.EnvVar{Name: "ORACLE_SID", Value: strings.ToUpper(name)}) + } + if restype == "CATALOG" { + result = append(result, corev1.EnvVar{Name: "ORACLE_SID", Value: strings.ToUpper(name)}) + } + } + if !pdbFlag { + if restype == "SHARD" { + result = append(result, corev1.EnvVar{Name: "ORACLE_PDB", Value: strings.ToUpper(name) + "PDB"}) + } + if restype == "CATALOG" { + result = append(result, corev1.EnvVar{Name: "ORACLE_PDB", Value: strings.ToUpper(name) + "PDB"}) + } + } + if !secretFlag { + result = append(result, corev1.EnvVar{Name: "SECRET_VOLUME", Value: "/mnt/secrets"}) + } + if !pwdFileFLag { + result = append(result, corev1.EnvVar{Name: "COMMON_OS_PWD_FILE", Value: "common_os_pwdfile.enc"}) + } + if !pwdKeyFlag { + result = append(result, corev1.EnvVar{Name: "PWD_KEY", Value: "pwd.key"}) + } + if restype == "GSM" { + if !sDirectParam { + // varinfo = "director_name=sharddirector" + sDirectorCounter + ";director_region=primary;director_port=1521" + varinfo = directorParams + result = append(result, corev1.EnvVar{Name: "SHARD_DIRECTOR_PARAMS", Value: varinfo}) + } + if !sGroup1Params { + varinfo = "group_name=shardgroup1;deploy_as=primary;group_region=primary" + result = append(result, corev1.EnvVar{Name: "SHARD1_GROUP_PARAMS", Value: varinfo}) + } + if instance.Spec.IsDataGuard { + if !sGroup2Params { + varinfo = "group_name=shardgroup2;deploy_as=standby;group_region=standby" + result = append(result, corev1.EnvVar{Name: "SHARD2_GROUP_PARAMS", Value: varinfo}) + } + } + if !catalogParams { + varinfo = buildCatalogParams(instance) + result = append(result, corev1.EnvVar{Name: "CATALOG_PARAMS", Value: varinfo}) + } + + if masterFlag == true { + result = append(result, corev1.EnvVar{Name: "MASTER_GSM", Value: "true"}) + } + result = append(result, corev1.EnvVar{Name: "CATALOG_SETUP", Value: "true"}) + result = append(result, corev1.EnvVar{Name: "OP_TYPE", Value: "gsm"}) + result = append(result, corev1.EnvVar{Name: "KUBE_SVC", Value: name}) + } + + if restype == "SHARD" { + result = append(result, corev1.EnvVar{Name: "OP_TYPE", Value: "primaryshard"}) + result = append(result, corev1.EnvVar{Name: "KUBE_SVC", Value: name}) + } + + if restype == "CATALOG" { + result = append(result, corev1.EnvVar{Name: "OP_TYPE", Value: "catalog"}) + result = append(result, corev1.EnvVar{Name: "KUBE_SVC", Value: name}) + } + + if instance.Spec.IsClone { + result = append(result, corev1.EnvVar{Name: "CLONE_DB", Value: "true"}) + if restype == "SHARD" { + if !oldSidFlag { + result = append(result, corev1.EnvVar{Name: "OLD_ORACLE_SID", Value: "GOLDCDB"}) + } + if !oldPdbFlag { + result = append(result, corev1.EnvVar{Name: "OLD_ORACLE_PDB", Value: "GOLDPDB"}) + } + } + if restype == "CATALOG" { + if !oldSidFlag { + result = append(result, corev1.EnvVar{Name: "OLD_ORACLE_SID", Value: "GOLDCDB"}) + } + if !oldPdbFlag { + result = append(result, corev1.EnvVar{Name: "OLD_ORACLE_PDB", Value: "GOLDPDB"}) + } + } + } + + return result +} + +// FUnction to build the svc definition for catalog/shard and GSM +func buildSvcPortsDef(instance *databasealphav1.ShardingDatabase, resType string) []corev1.ServicePort { + var result []corev1.ServicePort + if len(instance.Spec.PortMappings) > 0 { + for _, portMapping := range instance.Spec.PortMappings { + servicePort := + corev1.ServicePort{ + Protocol: portMapping.Protocol, + Port: portMapping.Port, + Name: generatePortMapping(portMapping), + TargetPort: intstr.IntOrString{ + Type: intstr.Int, + IntVal: portMapping.TargetPort, + }, + } + result = append(result, servicePort) + } + } else { + if resType == "GSM" { + result = append(result, corev1.ServicePort{Protocol: corev1.ProtocolTCP, Port: oraGSMPort, Name: generateName(fmt.Sprintf("%s-%d-%d-", "tcp", oraGSMPort, oraGSMPort)), TargetPort: intstr.IntOrString{Type: intstr.Int, IntVal: oraGSMPort}}) + } else { + result = append(result, corev1.ServicePort{Protocol: corev1.ProtocolTCP, Port: oraDBPort, Name: generateName(fmt.Sprintf("%s-%d-%d-", "tcp", oraDBPort, oraDBPort)), TargetPort: intstr.IntOrString{Type: intstr.Int, IntVal: oraDBPort}}) + } + result = append(result, corev1.ServicePort{Protocol: corev1.ProtocolTCP, Port: oraRemoteOnsPort, Name: generateName(fmt.Sprintf("%s-%d-%d-", "tcp", oraRemoteOnsPort, oraRemoteOnsPort)), TargetPort: intstr.IntOrString{Type: intstr.Int, IntVal: oraRemoteOnsPort}}) + result = append(result, corev1.ServicePort{Protocol: corev1.ProtocolTCP, Port: oraLocalOnsPort, Name: generateName(fmt.Sprintf("%s-%d-%d-", "tcp", oraLocalOnsPort, oraLocalOnsPort)), TargetPort: intstr.IntOrString{Type: intstr.Int, IntVal: oraLocalOnsPort}}) + result = append(result, corev1.ServicePort{Protocol: corev1.ProtocolTCP, Port: oraAgentPort, Name: generateName(fmt.Sprintf("%s-%d-%d-", "tcp", oraAgentPort, oraAgentPort)), TargetPort: intstr.IntOrString{Type: intstr.Int, IntVal: oraAgentPort}}) + } + + return result +} + +// Function to generate the Name +func generateName(base string) string { + maxNameLength := 50 + randomLength := 5 + maxGeneratedLength := maxNameLength - randomLength + if len(base) > maxGeneratedLength { + base = base[:maxGeneratedLength] + } + return fmt.Sprintf("%s%s", base, rand.String(randomLength)) +} + +// Function to generate the port mapping +func generatePortMapping(portMapping databasealphav1.PortMapping) string { + return generateName(fmt.Sprintf("%s-%d-%d-", "tcp", + portMapping.Port, portMapping.TargetPort)) +} + +func LogMessages(msgtype string, msg string, err error, instance *databasealphav1.ShardingDatabase, logger logr.Logger) { + // setting logrus formatter + //logrus.SetFormatter(&logrus.JSONFormatter{}) + //logrus.SetOutput(os.Stdout) + + if msgtype == "DEBUG" && instance.Spec.IsDebug == true { + if err != nil { + logger.Error(err, msg) + } else { + logger.Info(msg) + } + } else if msgtype == "INFO" { + logger.Info(msg) + } +} + +func GetGsmPodName(gsmName string) string { + podName := gsmName + return podName +} + +func GetSidName(variables []databasealphav1.EnvironmentVariable, name string) string { + var result string + + for _, variable := range variables { + if variable.Name == "ORACLE_SID" { + result = variable.Value + } + } + if result == "" { + result = strings.ToUpper(name) + } + return result +} + +func GetPdbName(variables []databasealphav1.EnvironmentVariable, name string) string { + var result string + + for _, variable := range variables { + if variable.Name == "ORACLE_SID" { + result = variable.Value + } + } + if result == "" { + result = strings.ToUpper(name) + "PDB" + } + return result +} + +func getlabelsForGsm(instance *databasealphav1.ShardingDatabase) map[string]string { + return buildLabelsForGsm(instance, "sharding") +} + +func getlabelsForShard(instance *databasealphav1.ShardingDatabase) map[string]string { + return buildLabelsForShard(instance, "sharding") +} + +func getlabelsForCatalog(instance *databasealphav1.ShardingDatabase) map[string]string { + return buildLabelsForCatalog(instance, "sharding") +} + +func LabelsForProvShardKind(instance *databasealphav1.ShardingDatabase, sftype string, +) map[string]string { + + if sftype == "shard" { + return buildLabelsForShard(instance, "sharding") + } + + return nil + +} + +func CheckSfset(sfsetName string, instance *databasealphav1.ShardingDatabase, kClient client.Client) (*appsv1.StatefulSet, error) { + sfSetFound := &appsv1.StatefulSet{} + err := kClient.Get(context.TODO(), types.NamespacedName{ + Name: sfsetName, + Namespace: instance.Spec.Namespace, + }, sfSetFound) + if err != nil { + return sfSetFound, err + } + return sfSetFound, nil +} + +func checkPvc(pvcName string, instance *databasealphav1.ShardingDatabase, kClient client.Client) (*corev1.PersistentVolumeClaim, error) { + pvcFound := &corev1.PersistentVolumeClaim{} + err := kClient.Get(context.TODO(), types.NamespacedName{ + Name: pvcName, + Namespace: instance.Spec.Namespace, + }, pvcFound) + if err != nil { + return pvcFound, err + } + return pvcFound, nil +} + +func DelPvc(pvcName string, instance *databasealphav1.ShardingDatabase, kClient client.Client, logger logr.Logger) error { + + LogMessages("DEBUG", "Inside the delPvc and received param: "+GetFmtStr(pvcName), nil, instance, logger) + pvcFound, err := checkPvc(pvcName, instance, kClient) + if err != nil { + LogMessages("DEBUG", "Error occurred in finding the pvc claim!", nil, instance, logger) + return err + } + err = kClient.Delete(context.Background(), pvcFound) + if err != nil { + LogMessages("DEBUG", "Error occurred in deleting the pvc claim!", nil, instance, logger) + return err + } + return nil +} + +func DelSvc(pvcName string, instance *databasealphav1.ShardingDatabase, kClient client.Client, logger logr.Logger) error { + + LogMessages("DEBUG", "Inside the delPvc and received param: "+GetFmtStr(pvcName), nil, instance, logger) + pvcFound, err := checkPvc(pvcName, instance, kClient) + if err != nil { + LogMessages("DEBUG", "Error occurred in finding the pvc claim!", nil, instance, logger) + return err + } + err = kClient.Delete(context.Background(), pvcFound) + if err != nil { + LogMessages("DEBUG", "Error occurred in deleting the pvc claim!", nil, instance, logger) + return err + } + return nil +} + +func CheckSvc(svcName string, instance *databasealphav1.ShardingDatabase, kClient client.Client) (*corev1.Service, error) { + svcFound := &corev1.Service{} + err := kClient.Get(context.TODO(), types.NamespacedName{ + Name: svcName, + Namespace: instance.Spec.Namespace, + }, svcFound) + if err != nil { + return svcFound, err + } + return svcFound, nil +} + +func PodListValidation(podList *corev1.PodList, sfName string, instance *databasealphav1.ShardingDatabase, kClient client.Client, +) (bool, *corev1.Pod) { + + var isPodExist bool = false + podInfo := &corev1.Pod{} + var podNameStr string + var err error + if sfName != "" { + podNameStr = sfName + "-" + } else { + podNameStr = "-" + } + + for _, pod := range podList.Items { + if strings.Contains(pod.Name, podNameStr) { + err = checkPod(instance, &pod, kClient) + if err != nil { + isPodExist = false + } + err = checkPodStatus(&pod, kClient) + if err != nil { + isPodExist = false + } + err = checkContainerStatus(&pod, kClient) + if err != nil { + isPodExist = false + } else { + isPodExist = true + podInfo = &pod + break + } + } + } + return isPodExist, podInfo +} + +func GetPodList(sfsetName string, resType string, instance *databasealphav1.ShardingDatabase, kClient client.Client, +) (*corev1.PodList, error) { + podList := &corev1.PodList{} + labelSelector := labels.SelectorFromSet(getlabelsForGsm(instance)) + if resType == "GSM" { + labelSelector = labels.SelectorFromSet(getlabelsForGsm(instance)) + } else if resType == "SHARD" { + labelSelector = labels.SelectorFromSet(getlabelsForShard(instance)) + } else if resType == "CATALOG" { + labelSelector = labels.SelectorFromSet(getlabelsForCatalog(instance)) + } else { + err1 := fmt.Errorf("Wrong resources type passed. Supported values are SHARD,GSM and CATALOG") + return nil, err1 + } + + listOps := &client.ListOptions{Namespace: instance.Spec.Namespace, LabelSelector: labelSelector} + + err := kClient.List(context.TODO(), podList, listOps) + if err != nil { + return nil, err + } + return podList, nil +} + +func checkPod(instance *databasealphav1.ShardingDatabase, pod *corev1.Pod, kClient client.Client, +) error { + err := kClient.Get(context.TODO(), types.NamespacedName{ + Name: pod.Name, + Namespace: instance.Spec.Namespace, + }, pod) + + if err != nil { + // Pod Doesn't exist + return err + } + + return nil +} + +func checkPodStatus(pod *corev1.Pod, kClient client.Client, +) error { + var msg string + for _, condition := range pod.Status.Conditions { + if pod.Status.Phase == corev1.PodRunning { + if condition.Type == corev1.PodReady { + // msg = "Pod Status is running" + // LogMessages("DEBUG", msg) + return nil + } + } else { + msg = "Pod is not scheduled or ready " + pod.Name + ".Describe the pod to check the detailed message" + return fmt.Errorf(msg) + } + } + return nil +} + +func checkContainerStatus(pod *corev1.Pod, kClient client.Client, +) error { + + var statuses []corev1.ContainerStatus + var msg string + // msg = "Inside the function checkContainerStatus" + // LogMessages("DEBUG", msg) + statuses = pod.Status.ContainerStatuses + var isRunning bool = false + for _, status := range statuses { + if status.State.Running == nil { + isRunning = false + } else { + isRunning = true + break + } + } + msg = "Container is not in running state" + pod.Name + ".Describe the pod to check the detailed message" + if isRunning { + return nil + } else { + return fmt.Errorf(msg) + } +} + +// Namespace related function + +func AddNamespace(instance *databasealphav1.ShardingDatabase, kClient client.Client, logger logr.Logger, +) error { + var msg string + ns := &corev1.Namespace{} + err := kClient.Get(context.TODO(), types.NamespacedName{Name: instance.Spec.Namespace}, ns) + if err != nil { + //msg = "Namespace " + instance.Spec.Namespace + " doesn't exist! creating namespace" + if errors.IsNotFound(err) { + err = kClient.Create(context.TODO(), NewNamespace(instance.Spec.Namespace)) + if err != nil { + msg = "Error in creating namespace!" + LogMessages("Error", msg, nil, instance, logger) + return err + } + } else { + msg = "Error in finding namespace!" + LogMessages("Error", msg, nil, instance, logger) + return err + } + } + return nil +} + +// NewNamespace creates a corev1.Namespace object using the provided name. +func NewNamespace(name string) *corev1.Namespace { + return &corev1.Namespace{ + TypeMeta: metav1.TypeMeta{ + Kind: "Namespace", + APIVersion: corev1.SchemeGroupVersion.String(), + }, + ObjectMeta: metav1.ObjectMeta{ + Name: name, + }, + } +} + +func getOwnerRef(instance *databasealphav1.ShardingDatabase, +) []metav1.OwnerReference { + + var ownerRef []metav1.OwnerReference + ownerRef = append(ownerRef, metav1.OwnerReference{Kind: instance.GroupVersionKind().Kind, APIVersion: instance.APIVersion, Name: instance.Name, UID: types.UID(instance.UID)}) + return ownerRef +} + +func buildCatalogParams(instance *databasealphav1.ShardingDatabase) string { + var variables []databasealphav1.EnvironmentVariable + variables = instance.Spec.Catalog[0].EnvVars + var result string + var varinfo string + var sidFlag bool = false + var pdbFlag bool = false + var portFlag bool = false + var regionFlag bool = false + var cnameFlag bool = false + var chunksFlag bool = false + var sidName string + var pdbName string + var cport string + var cregion string + var cname string + var catchunks string + + result = "catalog_host=" + instance.Spec.Catalog[0].Name + "-0" + "." + instance.Spec.Catalog[0].Name + ";" + for _, variable := range variables { + if variable.Name == "ORACLE_SID" { + sidFlag = true + sidName = variable.Value + } + if variable.Name == "ORACLE_PDB" { + pdbFlag = true + pdbName = variable.Value + } + if variable.Name == "CATALOG_PORT" { + portFlag = true + cport = variable.Value + } + if variable.Name == "CATALOG_REGION" { + regionFlag = true + cregion = variable.Value + } + if variable.Name == "CATALOG_NAME" { + cnameFlag = true + cname = variable.Value + } + if variable.Name == "CATALOG_CHUNKS" { + chunksFlag = true + catchunks = variable.Value + } + } + + if !sidFlag { + varinfo = "catalog_db=" + strings.ToUpper(instance.Spec.Catalog[0].Name) + ";" + result = result + varinfo + } else { + varinfo = "catalog_db=" + strings.ToUpper(sidName) + ";" + result = result + varinfo + } + + if !pdbFlag { + varinfo = "catalog_pdb=" + strings.ToUpper(instance.Spec.Catalog[0].Name) + "PDB" + ";" + result = result + varinfo + } else { + varinfo = "catalog_pdb=" + strings.ToUpper(pdbName) + ";" + result = result + varinfo + } + + if !portFlag { + varinfo = "catalog_port=" + "1521" + ";" + result = result + varinfo + } else { + varinfo = "catalog_port=" + cport + ";" + result = result + varinfo + } + + if !cnameFlag { + varinfo = "catalog_name=" + strings.ToUpper(instance.Spec.Catalog[0].Name) + ";" + result = result + varinfo + } else { + varinfo = "catalog_name=" + strings.ToUpper(cname) + ";" + result = result + varinfo + } + if chunksFlag { + result = result + "catalog_chunks=" + catchunks + ";" + } + + if !regionFlag { + varinfo = "catalog_region=primary,standby" + result = result + varinfo + } else { + varinfo = "catalog_region=" + cregion + result = result + varinfo + } + + return result +} + +func buildDirectorParams(instance *databasealphav1.ShardingDatabase, oraGsmSpex databasealphav1.GsmSpec, idx int) string { + var variables []databasealphav1.EnvironmentVariable + var result string + var varinfo string + var dnameFlag bool = false + var dportFlag bool = false + + // Get the GSM Spec and build director params. idx feild is very important to build the unique director name and regiod. Idx is GSM array index. + variables = oraGsmSpex.EnvVars + for _, variable := range variables { + if variable.Name == "DIRECTOR_NAME" { + dnameFlag = true + } + if variable.Name == "DIRECTOR_PORT" { + dportFlag = true + } + } + if !dnameFlag { + varinfo = "director_name=sharddirector" + strconv.Itoa(idx) + ";" + result = result + varinfo + } + + if idx == 0 { + varinfo = "director_region=primary;" + result = result + varinfo + } else if idx == 1 { + varinfo = "director_region=standby;" + result = result + varinfo + } else { + // Do nothing + } + + if !dportFlag { + varinfo = "director_port=1522" + result = result + varinfo + } + + return result +} + +func BuildShardParams(sfSet *appsv1.StatefulSet) string { + var variables []corev1.EnvVar + variables = sfSet.Spec.Template.Spec.Containers[0].Env + var result string + var varinfo string + var isShardPort bool = false + var isShardGrp bool = false + + result = "shard_host=" + sfSet.Name + "-0" + "." + sfSet.Name + ";" + for _, variable := range variables { + if variable.Name == "ORACLE_SID" { + varinfo = "shard_db=" + variable.Value + ";" + result = result + varinfo + } + if variable.Name == "ORACLE_PDB" { + varinfo = "shard_pdb=" + variable.Value + ";" + result = result + varinfo + } + if variable.Name == "SHARD_PORT" { + varinfo = "shard_port=" + variable.Value + ";" + result = result + varinfo + isShardPort = true + } + if variable.Name == "SHARD_GROUP" { + varinfo = "shard_group=" + variable.Value + ";" + result = result + varinfo + isShardGrp = true + } + } + + if !isShardPort { + varinfo = "shard_port=" + "1521" + ";" + result = result + varinfo + } + + if !isShardGrp { + varinfo = "shard_group=" + "shardgroup1" + result = result + varinfo + } + + return result +} + +func labelsForShardingDatabaseKind(instance *databasealphav1.ShardingDatabase, sftype string, +) map[string]string { + + if sftype == "shard" { + return buildLabelsForShard(instance, "sharding") + } + + return nil + +} + +func removeAlpha(numStr string, +) string { + + reg, _ := regexp.Compile("[^0-9]+") + processedString := reg.ReplaceAllString(numStr, "") + numDigit := processedString + "Gi" + return numDigit +} + +func GetIpCmd(svcName string) []string { + grepStr := " | grep PING | sed -e 's/).*//' | sed -e 's/.*(//'" + var oragetIpCmd = []string{"/bin/bash", "-c", " ping -q -c 1 -t 1 " + svcName + grepStr} + return oragetIpCmd +} + +func getGsmSvcCmd() []string { + var oragetGsmSvcCmd = []string{"/bin/bash", "-c", " $ORACLE_HOME/bin/gdsctl services | grep Service | awk -F ' ' '{ print $2 }' | tr '\n' ' ' "} + return oragetGsmSvcCmd +} + +func getDbRoleCmd() []string { + sqlCmd := "echo -e 'set feedback off; \n set heading off; \n select database_role from v$database;' | sqlplus -S '/as sysdba' | tr '\n' ' '" + var oraSqlCmd = []string{"/bin/bash", "-c", sqlCmd} + return oraSqlCmd +} + +func getDbModeCmd() []string { + sqlCmd := "echo -e 'set feedback off; \n set heading off; \n select open_mode from v$database;' | sqlplus -S '/as sysdba' | tr '\n' ' '" + var oraSqlCmd = []string{"/bin/bash", "-c", sqlCmd} + return oraSqlCmd +} + +func GetShardInviteNodeCmd(shardName string) []string { + shard_host := shardName + "." + strings.Split(shardName, "-0")[0] + var oraShardInviteCmd = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", "--invitednode=" + strconv.Quote(shard_host), "--optype=gsm"} + return oraShardInviteCmd +} + +func getCancelChunksCmd(sparamStr string) []string { + var cancelChunkCmd []string + cancelChunkCmd = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", "--cancelchunks=" + strconv.Quote(sparamStr), "--optype=gsm"} + return cancelChunkCmd +} + +func getMoveChunksCmd(sparamStr string) []string { + var moveChunkCmd []string + moveChunkCmd = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", "--movechunks=" + strconv.Quote(sparamStr), "--optype=gsm"} + return moveChunkCmd +} + +func getNoChunksCmd(sparamStr string) []string { + var noChunkCmd []string + noChunkCmd = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", "--validatenochunks=" + strconv.Quote(sparamStr), "--optype=gsm"} + return noChunkCmd +} + +func shardValidationCmd() []string { + + var oraShardValidateCmd = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", "--checkliveness=true ", "--optype=primaryshard"} + return oraShardValidateCmd +} + +func getShardCheckCmd(sparamStr string) []string { + var checkShardCmd []string + checkShardCmd = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", "--checkgsmshard=" + strconv.Quote(sparamStr), "--optype=gsm"} + return checkShardCmd +} + +func getShardAddCmd(sparams string) []string { + + sparamStr := "--addshard=" + strconv.Quote(sparams) + var addShardCmd = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", sparamStr, "--optype=gsm"} + return addShardCmd + +} + +func getShardDelCmd(sparams string) []string { + sparamStr := "--deleteshard=" + strconv.Quote(sparams) + var delShardCmd = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", sparamStr} + return delShardCmd +} + +func getLivenessCmd(resType string) []string { + var livenessCmd []string + if resType == "SHARD" { + livenessCmd = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", "--checkliveness=true", "--optype=primaryshard"} + } + if resType == "CATALOG" { + livenessCmd = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", "--checkliveness=true", "--optype=catalog"} + } + if resType == "GSM" { + livenessCmd = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", "--checkliveness=true", "--optype=gsm"} + } + if resType == "STANDBY" { + livenessCmd = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", "--checkliveness=true", "--optype=standbyshard"} + } + return livenessCmd +} + +func getGsmShardValidateCmd(shardName string) []string { + var validateCmd []string + validateCmd = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", "--validateshard=" + strconv.Quote(shardName), "--optype=gsm"} + return validateCmd +} + +func getOnlineShardCmd(sparamStr string) []string { + var onlineCmd []string + onlineCmd = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", "--checkonlineshard=" + strconv.Quote(sparamStr), "--optype=gsm"} + return onlineCmd +} + +func getGsmAddShardGroupCmd(sparamStr string) []string { + var addSgroupCmd []string + addSgroupCmd = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", sparamStr, "--optype=gsm"} + return addSgroupCmd +} + +func getdeployShardCmd() []string { + var depCmd []string + depCmd = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", "--deployshard=true", "--optype=gsm"} + return depCmd +} + +func getGsmvalidateCmd() []string { + var depCmd []string + depCmd = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", "--checkliveness=true", "--optype=gsm"} + return depCmd +} + +func getInitContainerCmd(resType string, name string, +) string { + var initCmd string + if resType == "WEB" { + initCmd = "chown -R 54321:54321 " + oraScriptMount + ";chmod 755 " + oraScriptMount + "/*;chown -R 54321:54321 /opt/oracle/oradata;chmod 750 /opt/oracle/oradata" + } else { + initCmd = resType + ";chown -R 54321:54321 " + oraScriptMount + ";chmod 755 " + oraScriptMount + "/*;chown -R 54321:54321 /opt/oracle/oradata;chmod 750 /opt/oracle/oradata" + } + return initCmd +} + +func getGsmInitContainerCmd(resType string, name string, +) string { + var initCmd string + if resType == "WEB" { + initCmd = "chown -R 54321:54321 " + oraScriptMount + ";chmod 755 " + oraScriptMount + "/*;chown -R 54321:54321 /opt/oracle/gsmdata;chmod 750 /opt/oracle/gsmdata" + } else { + initCmd = resType + ";chown -R 54321:54321 " + oraScriptMount + ";chmod 755 " + oraScriptMount + "/*;chown -R 54321:54321 /opt/oracle/gsmdata;chmod 750 /opt/oracle/gsmdata" + } + return initCmd +} + +func GetFmtStr(pstr string, +) string { + return "[" + pstr + "]" +} + +func ReadConfigMap(cmName string, instance *databasealphav1.ShardingDatabase, kClient client.Client, logger logr.Logger, +) (string, string, string, string, string, string) { + + var region, fingerprint, user, tenancy, passphrase, str1, topicid, k, value string + cm := &corev1.ConfigMap{} + var err error + + // Reding a config map + err = kClient.Get(context.TODO(), types.NamespacedName{ + Name: cmName, + Namespace: instance.Spec.Namespace, + }, cm) + + if err != nil { + return "NONE", "NONE", "NONE", "NONE", "NONE", "None" + } + + // ConfigMap evaluation + cmMap1 := cm.Data + for k, value = range cmMap1 { + LogMessages("DEBUG", "Key : "+GetFmtStr(k)+" Value : "+GetFmtStr(value), nil, instance, logger) + str1 = value + } + + for _, line := range strings.Split(strings.TrimSuffix(str1, "\n"), "\n") { + s := strings.Index(line, "=") + if s == -1 { + continue + } + k = line[:s] + value = line[s+1:] + + LogMessages("DEBUG", "Key : "+GetFmtStr(k)+" Value : "+GetFmtStr(value), nil, instance, logger) + if k == "region" { + region = value + } else if k == "fingerprint" { + fingerprint = value + } else if k == "user" { + user = value + } else if k == "tenancy" { + tenancy = value + } else if k == "passpharase" { + passphrase = value + } else if k == "topicid" { + topicid = value + } else { + LogMessages("DEBUG", GetFmtStr(k)+" is not matching with any required value for ONS.", nil, instance, logger) + } + } + return region, user, tenancy, passphrase, fingerprint, topicid +} + +func ReadSecret(secName string, instance *databasealphav1.ShardingDatabase, kClient client.Client, logger logr.Logger, +) string { + + var value string + sc := &corev1.Secret{} + var err error + + // Reading a Secret + err = kClient.Get(context.TODO(), types.NamespacedName{ + Name: secName, + Namespace: instance.Spec.Namespace, + }, sc) + + if err != nil { + return "NONE" + } + + // Secret Evaluation + for k, val := range sc.Data { + if k == "privatekey" { + LogMessages("DEBUG", "Key : "+GetFmtStr(k)+" Value : "+GetFmtStr(value)+" Val: "+GetFmtStr(string(val)), nil, instance, logger) + } + } + + return string(sc.Data["privatekey"]) +} + +func GetK8sClientConfig(kClient client.Client) (clientcmd.ClientConfig, kubernetes.Interface, error) { + var err1 error + var kubeConfig clientcmd.ClientConfig + var kubeClient kubernetes.Interface + + databasealphav1.KubeConfigOnce.Do(func() { + loadingRules := clientcmd.NewDefaultClientConfigLoadingRules() + configOverrides := &clientcmd.ConfigOverrides{} + kubeConfig = clientcmd.NewNonInteractiveDeferredLoadingClientConfig(loadingRules, configOverrides) + config, err := kubeConfig.ClientConfig() + if err != nil { + err1 = err + } + kubeClient, err = kubernetes.NewForConfig(config) + if err != nil { + err1 = err + } + + }) + return kubeConfig, kubeClient, err1 +} + +func Contains(list []string, s string) bool { + for _, v := range list { + if v == s { + return true + } + } + return false +} + +// Function to check shadrd in GSM +func CheckShardInGsm(gsmPodName string, sparams string, instance *databasealphav1.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, +) error { + + err, _, _ := ExecCommand(gsmPodName, getShardCheckCmd(sparams), kubeClient, kubeconfig, instance, logger) + if err != nil { + msg := "Did not find the shard " + GetFmtStr(sparams) + " in GSM." + LogMessages("INFO", msg, nil, instance, logger) + return err + } + return nil +} + +// Function to check the online Shard +func CheckOnlineShardInGsm(gsmPodName string, sparams string, instance *databasealphav1.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, +) error { + + err, _, _ := ExecCommand(gsmPodName, getOnlineShardCmd(sparams), kubeClient, kubeconfig, instance, logger) + if err != nil { + msg := "Shard: " + GetFmtStr(sparams) + " is not onine in GSM." + LogMessages("INFO", msg, nil, instance, logger) + return err + } + return nil +} + +// Function to move the chunks +func MoveChunks(gsmPodName string, sparams string, instance *databasealphav1.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, +) error { + + err, _, _ := ExecCommand(gsmPodName, getMoveChunksCmd(sparams), kubeClient, kubeconfig, instance, logger) + if err != nil { + msg := "Error occurred in during Chunk movement command submission for shard: " + GetFmtStr(sparams) + " in GSM." + LogMessages("INFO", msg, nil, instance, logger) + return err + } + return nil +} + +// Function to verify the chunks +func VerifyChunks(gsmPodName string, sparams string, instance *databasealphav1.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, +) error { + err, _, _ := ExecCommand(gsmPodName, getNoChunksCmd(sparams), kubeClient, kubeconfig, instance, logger) + if err != nil { + msg := "Chunks are not moved completely from the shard: " + GetFmtStr(sparams) + " in GSM." + LogMessages("INFO", msg, nil, instance, logger) + return err + } + return nil +} + +// Function to verify the chunks +func AddShardInGsm(gsmPodName string, sparams string, instance *databasealphav1.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, +) error { + err, _, _ := ExecCommand(gsmPodName, getShardAddCmd(sparams), kubeClient, kubeconfig, instance, logger) + if err != nil { + msg := "Error occurred while adding a shard " + GetFmtStr(sparams) + " in GSM." + LogMessages("INFO", msg, nil, instance, logger) + return err + } + return nil +} + +// Function to deploy the Shards +func DeployShardInGsm(gsmPodName string, sparams string, instance *databasealphav1.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, +) error { + err, _, _ := ExecCommand(gsmPodName, getdeployShardCmd(), kubeClient, kubeconfig, instance, logger) + if err != nil { + msg := "Error occurred while deploying the shard in GSM." + LogMessages("INFO", msg, nil, instance, logger) + return err + } + return nil +} + +// Function to verify the chunks +func CancelChunksInGsm(gsmPodName string, sparams string, instance *databasealphav1.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, +) error { + err, _, _ := ExecCommand(gsmPodName, getCancelChunksCmd(sparams), kubeClient, kubeconfig, instance, logger) + if err != nil { + msg := "Error occurred while cancelling the chunks: " + GetFmtStr(sparams) + " in GSM." + LogMessages("INFO", msg, nil, instance, logger) + return err + } + return nil +} + +// Function to delete the shard +func RemoveShardFromGsm(gsmPodName string, sparams string, instance *databasealphav1.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, +) error { + err, _, _ := ExecCommand(gsmPodName, getShardDelCmd(sparams), kubeClient, kubeconfig, instance, logger) + if err != nil { + msg := "Error occurred while cancelling the chunks: " + GetFmtStr(sparams) + " in GSM." + LogMessages("INFO", msg, nil, instance, logger) + return err + } + return nil +} + +func GetSvcIp(PodName string, sparams string, instance *databasealphav1.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, +) (error, string, string) { + err, stdoutput, stderror := ExecCommand(PodName, GetIpCmd(sparams), kubeClient, kubeconfig, instance, logger) + if err != nil { + msg := "Error occurred while getting the IP for k8s service " + GetFmtStr(sparams) + LogMessages("INFO", msg, nil, instance, logger) + return err, strings.Replace(stdoutput, "\r\n", "", -1), strings.Replace(stderror, "/r/n", "", -1) + } + return nil, strings.Replace(stdoutput, "\r\n", "", -1), strings.Replace(stderror, "/r/n", "", -1) +} + +func GetGsmServices(PodName string, instance *databasealphav1.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, +) string { + err, stdoutput, _ := ExecCommand(PodName, getGsmSvcCmd(), kubeClient, kubeconfig, instance, logger) + if err != nil { + msg := "Error occurred while getting the services from the GSM " + LogMessages("DEBUG", msg, err, instance, logger) + return msg + } + return stdoutput +} + +func GetDbRole(PodName string, instance *databasealphav1.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, +) string { + err, stdoutput, _ := ExecCommand(PodName, getDbRoleCmd(), kubeClient, kubeconfig, instance, logger) + if err != nil { + msg := "Error occurred while getting the DB role from the database" + LogMessages("DEBUG", msg, err, instance, logger) + return msg + } + return strings.TrimSpace(stdoutput) +} + +func GetDbOpenMode(PodName string, instance *databasealphav1.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, +) string { + err, stdoutput, _ := ExecCommand(PodName, getDbModeCmd(), kubeClient, kubeconfig, instance, logger) + if err != nil { + msg := "Error occurred while getting the DB mode from the database" + LogMessages("DEBUG", msg, err, instance, logger) + return msg + } + return strings.TrimSpace(stdoutput) +} + +func SfsetLabelPatch(sfSetFound *appsv1.StatefulSet, sfSetPod *corev1.Pod, instance *databasealphav1.ShardingDatabase, kClient client.Client, +) error { + + //var msg string + //status = false + var err error + + sfsetCopy := sfSetFound.DeepCopy() + sfsetCopy.Labels[string(databasealphav1.ShardingDelLabelKey)] = string(databasealphav1.ShardingDelLabelTrueValue) + patch := client.MergeFrom(sfSetFound) + err = kClient.Patch(context.Background(), sfsetCopy, patch) + if err != nil { + return err + } + + podCopy := sfSetPod.DeepCopy() + podCopy.Labels[string(databasealphav1.ShardingDelLabelKey)] = string(databasealphav1.ShardingDelLabelTrueValue) + podPatch := client.MergeFrom(sfSetPod.DeepCopy()) + err = kClient.Patch(context.Background(), podCopy, podPatch) + if err != nil { + return err + } + + return nil +} + +// Send Notification + +func SendNotification(title string, body string, instance *databasealphav1.ShardingDatabase, topicId string, rclient ons.NotificationDataPlaneClient, logger logr.Logger, +) { + var msg string + req := ons.PublishMessageRequest{TopicId: common.String(topicId), + MessageDetails: ons.MessageDetails{ + Title: common.String(title), + Body: common.String(body)}} + + // Send the request using the service client + _, err := rclient.PublishMessage(context.Background(), req) + if err != nil { + msg = "Error occurred in sending the message. Title: " + GetFmtStr(title) + logger.Error(err, "Error occurred while sending a notification") + LogMessages("DEBUG", msg, nil, instance, logger) + } +} diff --git a/commons/sharding/shard.go b/commons/sharding/shard.go new file mode 100644 index 00000000..08ba490f --- /dev/null +++ b/commons/sharding/shard.go @@ -0,0 +1,472 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package commons + +import ( + "context" + databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + "reflect" + "strconv" + + "github.com/go-logr/logr" + appsv1 "k8s.io/api/apps/v1" + corev1 "k8s.io/api/core/v1" + "k8s.io/apimachinery/pkg/api/resource" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" +) + +func buildLabelsForShard(instance *databasev1alpha1.ShardingDatabase, label string) map[string]string { + return map[string]string{ + "app": "OracleSharding", + "type": "Shard", + "oralabel": getLabelForShard(instance), + } +} + +func getLabelForShard(instance *databasev1alpha1.ShardingDatabase) string { + + // if len(OraShardSpex.Label) !=0 { + // return OraShardSpex.Label + // } + + return instance.Name +} + +func BuildStatefulSetForShard(instance *databasev1alpha1.ShardingDatabase, OraShardSpex databasev1alpha1.ShardSpec) *appsv1.StatefulSet { + sfset := &appsv1.StatefulSet{ + TypeMeta: buildTypeMetaForShard(), + ObjectMeta: builObjectMetaForShard(instance, OraShardSpex), + Spec: *buildStatefulSpecForShard(instance, OraShardSpex), + } + + return sfset +} + +// Function to build TypeMeta +func buildTypeMetaForShard() metav1.TypeMeta { + // building TypeMeta + typeMeta := metav1.TypeMeta{ + Kind: "StatefulSet", + APIVersion: "apps/v1", + } + return typeMeta +} + +// Function to build ObjectMeta +func builObjectMetaForShard(instance *databasev1alpha1.ShardingDatabase, OraShardSpex databasev1alpha1.ShardSpec) metav1.ObjectMeta { + // building objectMeta + objmeta := metav1.ObjectMeta{ + Name: OraShardSpex.Name, + Namespace: instance.Spec.Namespace, + OwnerReferences: getOwnerRef(instance), + Labels: buildLabelsForShard(instance, "sharding"), + } + return objmeta +} + +// Function to build Stateful Specs +func buildStatefulSpecForShard(instance *databasev1alpha1.ShardingDatabase, OraShardSpex databasev1alpha1.ShardSpec) *appsv1.StatefulSetSpec { + // building Stateful set Specs + var size int32 + size = 1 + sfsetspec := &appsv1.StatefulSetSpec{ + ServiceName: OraShardSpex.Name, + Selector: &metav1.LabelSelector{ + MatchLabels: buildLabelsForShard(instance, "sharding"), + }, + Template: corev1.PodTemplateSpec{ + ObjectMeta: metav1.ObjectMeta{ + Labels: buildLabelsForShard(instance, "sharding"), + }, + Spec: *buildPodSpecForShard(instance, OraShardSpex), + }, + VolumeClaimTemplates: volumeClaimTemplatesForShard(instance, OraShardSpex), + } + // if OraShardSpex.OraShardSize == 0 { + // OraShardSpex.OraShardSize = 1 + sfsetspec.Replicas = &size + // } else { + // sfsetspec.Replicas = &OraShardSpex.OraShardSize + // } + + return sfsetspec +} + +// Function to build PodSpec + +func buildPodSpecForShard(instance *databasev1alpha1.ShardingDatabase, OraShardSpex databasev1alpha1.ShardSpec) *corev1.PodSpec { + + user := oraRunAsUser + group := oraFsGroup + spec := &corev1.PodSpec{ + SecurityContext: &corev1.PodSecurityContext{ + RunAsUser: &user, + FSGroup: &group, + }, + InitContainers: buildInitContainerSpecForShard(instance, OraShardSpex), + Containers: buildContainerSpecForShard(instance, OraShardSpex), + Volumes: buildVolumeSpecForShard(instance, OraShardSpex), + } + if len(instance.Spec.DbImagePullSecret) > 0 { + spec.ImagePullSecrets = []corev1.LocalObjectReference{ + { + Name: instance.Spec.DbImagePullSecret, + }, + } + } + + if len(OraShardSpex.NodeSelector) > 0 { + spec.NodeSelector = make(map[string]string) + for key, value := range OraShardSpex.NodeSelector { + spec.NodeSelector[key] = value + } + } + + return spec +} + +// Function to build Volume Spec +func buildVolumeSpecForShard(instance *databasev1alpha1.ShardingDatabase, OraShardSpex databasev1alpha1.ShardSpec) []corev1.Volume { + var result []corev1.Volume + result = []corev1.Volume{ + { + Name: OraShardSpex.Name + "secretmap-vol3", + VolumeSource: corev1.VolumeSource{ + Secret: &corev1.SecretVolumeSource{ + SecretName: instance.Spec.Secret, + }, + }, + }, + { + Name: OraShardSpex.Name + "orascript-vol5", + VolumeSource: corev1.VolumeSource{ + EmptyDir: &corev1.EmptyDirVolumeSource{}, + }, + }, + { + Name: OraShardSpex.Name + "oradshm-vol6", + VolumeSource: corev1.VolumeSource{ + EmptyDir: &corev1.EmptyDirVolumeSource{}, + }, + }, + } + + if len(OraShardSpex.PvcName) != 0 { + result = append(result, corev1.Volume{Name: OraShardSpex.Name + "oradata-vol4", VolumeSource: corev1.VolumeSource{PersistentVolumeClaim: &corev1.PersistentVolumeClaimVolumeSource{ClaimName: OraShardSpex.PvcName}}}) + } + + if len(instance.Spec.StagePvcName) != 0 { + result = append(result, corev1.Volume{Name: OraShardSpex.Name + "orastage-vol7", VolumeSource: corev1.VolumeSource{PersistentVolumeClaim: &corev1.PersistentVolumeClaimVolumeSource{ClaimName: instance.Spec.StagePvcName}}}) + } + + return result +} + +// Function to build the container Specification +func buildContainerSpecForShard(instance *databasev1alpha1.ShardingDatabase, OraShardSpex databasev1alpha1.ShardSpec) []corev1.Container { + // building Continer spec + var result []corev1.Container + containerSpec := corev1.Container{ + Name: OraShardSpex.Name, + Image: instance.Spec.DbImage, + SecurityContext: &corev1.SecurityContext{ + Capabilities: &corev1.Capabilities{ + Add: []corev1.Capability{"NET_RAW"}, + }, + }, + Resources: corev1.ResourceRequirements{ + Requests: make(map[corev1.ResourceName]resource.Quantity), + }, + VolumeMounts: buildVolumeMountSpecForShard(instance, OraShardSpex), + LivenessProbe: &corev1.Probe{ + // TODO: Investigate if it's ok to call status every 10 seconds + FailureThreshold: int32(30), + PeriodSeconds: int32(240), + InitialDelaySeconds: int32(300), + TimeoutSeconds: int32(120), + Handler: corev1.Handler{ + Exec: &corev1.ExecAction{ + Command: getLivenessCmd("SHARD"), + }, + }, + }, + /** + // Disabling this because ping stop working and sharding topologu never gets configured. + StartupProbe: &corev1.Probe{ + FailureThreshold: int32(30), + PeriodSeconds: int32(180), + Handler: corev1.Handler{ + Exec: &corev1.ExecAction{ + Command: getLivenessCmd("SHARD"), + }, + }, + }, + **/ + Env: buildEnvVarsSpec(instance, OraShardSpex.EnvVars, OraShardSpex.Name, "SHARD", false, "NONE"), + } + + if instance.Spec.IsClone { + containerSpec.Command = []string{orainitCmd3} + } + + if OraShardSpex.Resources != nil { + containerSpec.Resources = *OraShardSpex.Resources + } + // building Complete Container Spec + result = []corev1.Container{ + containerSpec, + } + return result +} + +//Function to build the init Container Spec +func buildInitContainerSpecForShard(instance *databasev1alpha1.ShardingDatabase, OraShardSpex databasev1alpha1.ShardSpec) []corev1.Container { + var result []corev1.Container + privFlag := true + var uid int64 = 0 + + // building the init Container Spec + var scriptLoc string + if len(instance.Spec.ScriptsLocation) != 0 { + scriptLoc = instance.Spec.ScriptsLocation + } else { + scriptLoc = "WEB" + } + + init1spec := corev1.Container{ + Name: OraShardSpex.Name + "-init1", + Image: instance.Spec.DbImage, + SecurityContext: &corev1.SecurityContext{ + Privileged: &privFlag, + RunAsUser: &uid, + }, + Command: []string{ + "/bin/bash", + "-c", + getInitContainerCmd(scriptLoc, instance.Name), + }, + VolumeMounts: buildVolumeMountSpecForShard(instance, OraShardSpex), + } + + // building Complete Init Container Spec + if OraShardSpex.ImagePulllPolicy != nil { + init1spec.ImagePullPolicy = *OraShardSpex.ImagePulllPolicy + } + result = []corev1.Container{ + init1spec, + } + return result +} + +func buildVolumeMountSpecForShard(instance *databasev1alpha1.ShardingDatabase, OraShardSpex databasev1alpha1.ShardSpec) []corev1.VolumeMount { + var result []corev1.VolumeMount + result = append(result, corev1.VolumeMount{Name: OraShardSpex.Name + "secretmap-vol3", MountPath: oraSecretMount, ReadOnly: true}) + result = append(result, corev1.VolumeMount{Name: OraShardSpex.Name + "-oradata-vol4", MountPath: oraDataMount}) + result = append(result, corev1.VolumeMount{Name: OraShardSpex.Name + "orascript-vol5", MountPath: oraScriptMount}) + result = append(result, corev1.VolumeMount{Name: OraShardSpex.Name + "oradshm-vol6", MountPath: oraShm}) + + if len(instance.Spec.StagePvcName) != 0 { + result = append(result, corev1.VolumeMount{Name: OraShardSpex.Name + "orastage-vol7", MountPath: oraStage}) + } + + return result +} + +func volumeClaimTemplatesForShard(instance *databasev1alpha1.ShardingDatabase, OraShardSpex databasev1alpha1.ShardSpec) []corev1.PersistentVolumeClaim { + + var claims []corev1.PersistentVolumeClaim + + if len(OraShardSpex.PvcName) != 0 { + return claims + } + + claims = []corev1.PersistentVolumeClaim{ + { + ObjectMeta: metav1.ObjectMeta{ + Name: OraShardSpex.Name + "-oradata-vol4", + Namespace: instance.Spec.Namespace, + OwnerReferences: getOwnerRef(instance), + Labels: buildLabelsForShard(instance, "sharding"), + }, + Spec: corev1.PersistentVolumeClaimSpec{ + AccessModes: []corev1.PersistentVolumeAccessMode{ + corev1.ReadWriteOnce, + }, + StorageClassName: &instance.Spec.StorageClass, + Resources: corev1.ResourceRequirements{ + Requests: corev1.ResourceList{ + corev1.ResourceStorage: resource.MustParse(strconv.FormatInt(int64(OraShardSpex.StorageSizeInGb), 10) + "Gi"), + }, + }, + }, + }, + } + + if len(OraShardSpex.PvAnnotations) > 0 { + claims[0].ObjectMeta.Annotations = make(map[string]string) + for key, value := range OraShardSpex.PvAnnotations { + claims[0].ObjectMeta.Annotations[key] = value + } + } + + if len(OraShardSpex.PvMatchLabels) > 0 { + claims[0].Spec.Selector = &metav1.LabelSelector{MatchLabels: OraShardSpex.PvMatchLabels} + } + + return claims +} + +func BuildServiceDefForShard(instance *databasev1alpha1.ShardingDatabase, replicaCount int32, OraShardSpex databasev1alpha1.ShardSpec, svctype string) *corev1.Service { + service := &corev1.Service{} + service = &corev1.Service{ + ObjectMeta: buildSvcObjectMetaForShard(instance, replicaCount, OraShardSpex, svctype), + Spec: corev1.ServiceSpec{}, + } + + // Check if user want External Svc on each replica pod + if svctype == "external" { + service.Spec.Type = corev1.ServiceTypeLoadBalancer + service.Spec.Selector = getSvcLabelsForShard(replicaCount, OraShardSpex) + } + + if svctype == "local" { + service.Spec.ClusterIP = corev1.ClusterIPNone + service.Spec.Selector = buildLabelsForShard(instance, "sharding") + } + + // build Service Ports Specs to be exposed. If the PortMappings is not set then default ports will be exposed. + service.Spec.Ports = buildSvcPortsDef(instance, "SHARD") + return service +} + +// Function to build Service ObjectMeta +func buildSvcObjectMetaForShard(instance *databasev1alpha1.ShardingDatabase, replicaCount int32, OraShardSpex databasev1alpha1.ShardSpec, svctype string) metav1.ObjectMeta { + // building objectMeta + + var svcName string + + if svctype == "local" { + svcName = OraShardSpex.Name + } + + if svctype == "external" { + svcName = OraShardSpex.Name + strconv.FormatInt(int64(replicaCount), 10) + "-svc" + } + + objmeta := metav1.ObjectMeta{ + Name: svcName, + Namespace: instance.Spec.Namespace, + Labels: buildLabelsForShard(instance, "sharding"), + OwnerReferences: getOwnerRef(instance), + } + return objmeta +} + +func getSvcLabelsForShard(replicaCount int32, OraShardSpex databasev1alpha1.ShardSpec) map[string]string { + + var labelStr map[string]string + labelStr = make(map[string]string) + if replicaCount == -1 { + labelStr["statefulset.kubernetes.io/pod-name"] = OraShardSpex.Name + "-0" + } else { + labelStr["statefulset.kubernetes.io/pod-name"] = OraShardSpex.Name + "-" + strconv.FormatInt(int64(replicaCount), 10) + } + + // fmt.Println("Service Selector String Specification", labelStr) + return labelStr +} + +// ======================== Update Section ======================== +func UpdateProvForShard(instance *databasev1alpha1.ShardingDatabase, OraShardSpex databasev1alpha1.ShardSpec, kClient client.Client, sfSet *appsv1.StatefulSet, shardPod *corev1.Pod, logger logr.Logger, +) (ctrl.Result, error) { + var msg string + var size int32 + size = 1 + var isUpdate bool = false + var err error + var i int + + // Ensure deployment replicas match the desired state + if sfSet.Spec.Replicas != nil { + if *sfSet.Spec.Replicas != size { + msg = "Current StatefulSet replicas do not match configured Shard Replicas. Shard is configured with only 1 but current replicas is set with " + strconv.FormatInt(int64(*sfSet.Spec.Replicas), 10) + LogMessages("DEBUG", msg, nil, instance, logger) + isUpdate = true + } + } + // Memory Check + //resources := corev1.Pod.Spec.Containers + for i = 0; i < len(shardPod.Spec.Containers); i++ { + if shardPod.Spec.Containers[i].Name == sfSet.Name { + shardContaineRes := shardPod.Spec.Containers[i].Resources + oraSpexRes := OraShardSpex.Resources + + if !reflect.DeepEqual(shardContaineRes, oraSpexRes) { + isUpdate = true + } + } + } + + /** + for i = 0; i < len(sfSet.Spec.VolumeClaimTemplates); i++ { + if sfSet.Spec.VolumeClaimTemplates[i].Name == OraShardSpex.Name+"-oradata-vol4" { + volResource := sfSet.Spec.VolumeClaimTemplates[i].Spec.Resources + volumeSize := volResource.Requests.Storage() + sSize := volumeSize. + if sSize != int(OraShardSpex.StorageSizeInGb) { + isUpdate = true + } + + } + } + **/ + + if isUpdate { + err = kClient.Update(context.Background(), BuildStatefulSetForShard(instance, OraShardSpex)) + if err != nil { + msg = "Failed to update Shard StatefulSet " + "StatefulSet.Name : " + sfSet.Name + LogMessages("Error", msg, nil, instance, logger) + return ctrl.Result{}, err + } + + } + return ctrl.Result{}, nil +} diff --git a/config/certmanager/certificate.yaml b/config/certmanager/certificate.yaml new file mode 100644 index 00000000..3aadc474 --- /dev/null +++ b/config/certmanager/certificate.yaml @@ -0,0 +1,31 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +# The following manifests contain a self-signed issuer CR and a certificate CR. +# More document can be found at https://docs.cert-manager.io +# WARNING: Targets CertManager 0.11 check https://docs.cert-manager.io/en/latest/tasks/upgrading/index.html for +# breaking changes +apiVersion: cert-manager.io/v1 +kind: Issuer +metadata: + name: selfsigned-issuer + namespace: system +spec: + selfSigned: {} +--- +apiVersion: cert-manager.io/v1 +kind: Certificate +metadata: + name: serving-cert # this name should match the one appeared in kustomizeconfig.yaml + namespace: system +spec: + # $(SERVICE_NAME) and $(SERVICE_NAMESPACE) will be substituted by kustomize + dnsNames: + - $(SERVICE_NAME).$(SERVICE_NAMESPACE).svc + - $(SERVICE_NAME).$(SERVICE_NAMESPACE).svc.cluster.local + issuerRef: + kind: Issuer + name: selfsigned-issuer + secretName: webhook-server-cert # this secret will not be prefixed, since it's not managed by kustomize diff --git a/config/certmanager/kustomization.yaml b/config/certmanager/kustomization.yaml new file mode 100644 index 00000000..8dd6746c --- /dev/null +++ b/config/certmanager/kustomization.yaml @@ -0,0 +1,9 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +resources: +- certificate.yaml + +configurations: +- kustomizeconfig.yaml diff --git a/config/certmanager/kustomizeconfig.yaml b/config/certmanager/kustomizeconfig.yaml new file mode 100644 index 00000000..f8e232fa --- /dev/null +++ b/config/certmanager/kustomizeconfig.yaml @@ -0,0 +1,21 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +# This configuration is for teaching kustomize how to update name ref and var substitution +nameReference: +- kind: Issuer + group: cert-manager.io + fieldSpecs: + - kind: Certificate + group: cert-manager.io + path: spec/issuerRef/name + +varReference: +- kind: Certificate + group: cert-manager.io + path: spec/commonName +- kind: Certificate + group: cert-manager.io + path: spec/dnsNames diff --git a/config/crd/bases/database.oracle.com_autonomousdatabases.yaml b/config/crd/bases/database.oracle.com_autonomousdatabases.yaml new file mode 100644 index 00000000..51b0b6ab --- /dev/null +++ b/config/crd/bases/database.oracle.com_autonomousdatabases.yaml @@ -0,0 +1,185 @@ + +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.6.1 + creationTimestamp: null + name: autonomousdatabases.database.oracle.com +spec: + group: database.oracle.com + names: + kind: AutonomousDatabase + listKind: AutonomousDatabaseList + plural: autonomousdatabases + shortNames: + - adb + - adbs + singular: autonomousdatabase + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .status.displayName + name: Display Name + type: string + - jsonPath: .status.lifecycleState + name: State + type: string + - jsonPath: .status.isDedicated + name: Dedicated + type: string + - jsonPath: .status.cpuCoreCount + name: OCPUs + type: integer + - jsonPath: .status.dataStorageSizeInTBs + name: Storage (TB) + type: integer + - jsonPath: .status.dbWorkload + name: Workload Type + type: string + - jsonPath: .status.timeCreated + name: Created + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + description: AutonomousDatabase is the Schema for the autonomousdatabases + API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: 'AutonomousDatabaseSpec defines the desired state of AutonomousDatabase + Important: Run "make" to regenerate code after modifying this file' + properties: + details: + description: AutonomousDatabaseDetails defines the detail information + of AutonomousDatabase, corresponding to oci-go-sdk/database/AutonomousDatabase + properties: + adminPassword: + properties: + k8sSecretName: + type: string + ociSecretOCID: + type: string + type: object + autonomousDatabaseOCID: + type: string + compartmentOCID: + type: string + cpuCoreCount: + type: integer + dataStorageSizeInTBs: + type: integer + dbName: + type: string + dbVersion: + type: string + dbWorkload: + description: 'AutonomousDatabaseDbWorkloadEnum Enum with underlying + type: string' + enum: + - OLTP + - DW + - AJD + - APEX + type: string + displayName: + type: string + freeformTags: + additionalProperties: + type: string + type: object + isAutoScalingEnabled: + type: boolean + isDedicated: + type: boolean + lifecycleState: + description: 'AutonomousDatabaseLifecycleStateEnum Enum with underlying + type: string' + type: string + nsgOCIDs: + items: + type: string + type: array + privateEndpoint: + type: string + privateEndpointIP: + type: string + privateEndpointLabel: + type: string + subnetOCID: + type: string + wallet: + properties: + name: + type: string + password: + properties: + k8sSecretName: + type: string + ociSecretOCID: + type: string + type: object + type: object + type: object + hardLink: + default: false + type: boolean + ociConfig: + properties: + configMapName: + type: string + secretName: + type: string + type: object + required: + - details + type: object + status: + description: AutonomousDatabaseStatus defines the observed state of AutonomousDatabase + properties: + cpuCoreCount: + type: integer + dataStorageSizeInTBs: + type: integer + dbWorkload: + description: 'AutonomousDatabaseDbWorkloadEnum Enum with underlying + type: string' + type: string + displayName: + description: 'INSERT ADDITIONAL STATUS FIELD - define observed state + of cluster Important: Run "make" to regenerate code after modifying + this file' + type: string + isDedicated: + type: string + lifecycleState: + description: 'AutonomousDatabaseLifecycleStateEnum Enum with underlying + type: string' + type: string + timeCreated: + type: string + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] diff --git a/config/crd/bases/database.oracle.com_shardingdatabases.yaml b/config/crd/bases/database.oracle.com_shardingdatabases.yaml new file mode 100644 index 00000000..17dd4e23 --- /dev/null +++ b/config/crd/bases/database.oracle.com_shardingdatabases.yaml @@ -0,0 +1,446 @@ + +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.6.1 + creationTimestamp: null + name: shardingdatabases.database.oracle.com +spec: + group: database.oracle.com + names: + kind: ShardingDatabase + listKind: ShardingDatabaseList + plural: shardingdatabases + singular: shardingdatabase + scope: Namespaced + versions: + - name: v1alpha1 + schema: + openAPIV3Schema: + description: ShardingDatabase is the Schema for the shardingdatabases API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: ShardingDatabaseSpec defines the desired state of ShardingDatabase + properties: + catalog: + items: + description: CatalogSpec defines the desired state of CatalogSpec + properties: + envVars: + items: + description: EnvironmentVariable represents a named variable + accessible for containers. + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + imagePullPolicy: + description: PullPolicy describes a policy for if/when to pull + a container image + type: string + isDelete: + type: boolean + label: + type: string + name: + type: string + nodeSelector: + additionalProperties: + type: string + type: object + pvAnnotations: + additionalProperties: + type: string + type: object + pvMatchLabels: + additionalProperties: + type: string + type: object + pvcName: + type: string + resources: + description: ResourceRequirements describes the compute resource + requirements. + properties: + limits: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + description: 'Limits describes the maximum amount of compute + resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + type: object + requests: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + description: 'Requests describes the minimum amount of compute + resources required. If Requests is omitted for a container, + it defaults to Limits if that is explicitly specified, + otherwise to an implementation-defined value. More info: + https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + type: object + type: object + storageSizeInGb: + format: int32 + type: integer + required: + - name + type: object + type: array + dbImage: + type: string + dbImagePullSecret: + type: string + gsm: + items: + description: GsmSpec defines the desired state of GsmSpec + properties: + envVars: + items: + description: EnvironmentVariable represents a named variable + accessible for containers. + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + imagePullPolicy: + description: PullPolicy describes a policy for if/when to pull + a container image + type: string + isDelete: + type: boolean + label: + type: string + name: + type: string + nodeSelector: + additionalProperties: + type: string + type: object + pvMatchLabels: + additionalProperties: + type: string + type: object + pvcName: + type: string + replicas: + format: int32 + type: integer + resources: + description: ResourceRequirements describes the compute resource + requirements. + properties: + limits: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + description: 'Limits describes the maximum amount of compute + resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + type: object + requests: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + description: 'Requests describes the minimum amount of compute + resources required. If Requests is omitted for a container, + it defaults to Limits if that is explicitly specified, + otherwise to an implementation-defined value. More info: + https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + type: object + type: object + storageSizeInGb: + format: int32 + type: integer + required: + - name + type: object + type: array + gsmImage: + type: string + gsmImagePullSecret: + type: string + isClone: + type: boolean + isDataGuard: + type: boolean + isDebug: + type: boolean + isDeleteOraPvc: + type: boolean + isExternalSvc: + type: boolean + namespace: + type: string + nsConfigMap: + type: string + nsSecret: + type: string + portMappings: + items: + description: PortMapping is a specification of port mapping for + an application deployment. + properties: + port: + format: int32 + type: integer + protocol: + default: TCP + type: string + targetPort: + format: int32 + type: integer + required: + - port + - protocol + - targetPort + type: object + type: array + scriptsLocation: + type: string + secret: + type: string + shard: + description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster + Important: Run "make" to regenerate code after modifying this file' + items: + description: ShardSpec is a specification of Shards for an application + deployment. + properties: + envVars: + items: + description: EnvironmentVariable represents a named variable + accessible for containers. + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + imagePullPolicy: + description: PullPolicy describes a policy for if/when to pull + a container image + type: string + isDelete: + type: boolean + label: + type: string + name: + type: string + nodeSelector: + additionalProperties: + type: string + type: object + pvAnnotations: + additionalProperties: + type: string + type: object + pvMatchLabels: + additionalProperties: + type: string + type: object + pvcName: + type: string + resources: + description: ResourceRequirements describes the compute resource + requirements. + properties: + limits: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + description: 'Limits describes the maximum amount of compute + resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + type: object + requests: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + description: 'Requests describes the minimum amount of compute + resources required. If Requests is omitted for a container, + it defaults to Limits if that is explicitly specified, + otherwise to an implementation-defined value. More info: + https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + type: object + type: object + storageSizeInGb: + format: int32 + type: integer + required: + - name + type: object + type: array + stagePvcName: + type: string + storageClass: + type: string + required: + - catalog + - dbImage + - gsm + - gsmImage + - secret + - shard + type: object + status: + description: To understand Metav1.Condition, please refer the link https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1 + ShardingDatabaseStatus defines the observed state of ShardingDatabase + properties: + catalogs: + additionalProperties: + type: string + type: object + conditions: + items: + description: "Condition contains details for one aspect of the current + state of this API Resource. --- This struct is intended for direct + use as an array at the field path .status.conditions. For example, + type FooStatus struct{ // Represents the observations of a + foo's current state. // Known .status.conditions.type are: + \"Available\", \"Progressing\", and \"Degraded\" // +patchMergeKey=type + \ // +patchStrategy=merge // +listType=map // +listMapKey=type + \ Conditions []metav1.Condition `json:\"conditions,omitempty\" + patchStrategy:\"merge\" patchMergeKey:\"type\" protobuf:\"bytes,1,rep,name=conditions\"` + \n // other fields }" + properties: + lastTransitionTime: + description: lastTransitionTime is the last time the condition + transitioned from one status to another. This should be when + the underlying condition changed. If that is not known, then + using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: message is a human readable message indicating + details about the transition. This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: observedGeneration represents the .metadata.generation + that the condition was set based upon. For instance, if .metadata.generation + is currently 12, but the .status.conditions[x].observedGeneration + is 9, the condition is out of date with respect to the current + state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: reason contains a programmatic identifier indicating + the reason for the condition's last transition. Producers + of specific condition types may define expected values and + meanings for this field, and whether the values are considered + a guaranteed API. The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + --- Many .condition.type values are consistent across resources + like Available, but because arbitrary conditions can be useful + (see .node.status.conditions), the ability to deconflict is + important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + gsm: + properties: + details: + additionalProperties: + type: string + type: object + externalConnectStr: + type: string + internalConnectStr: + type: string + services: + type: string + shards: + additionalProperties: + type: string + type: object + state: + type: string + type: object + shards: + additionalProperties: + type: string + type: object + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] diff --git a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml new file mode 100644 index 00000000..d7abe5e0 --- /dev/null +++ b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml @@ -0,0 +1,337 @@ + +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.6.1 + creationTimestamp: null + name: singleinstancedatabases.database.oracle.com +spec: + group: database.oracle.com + names: + kind: SingleInstanceDatabase + listKind: SingleInstanceDatabaseList + plural: singleinstancedatabases + singular: singleinstancedatabase + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .status.edition + name: Edition + type: string + - jsonPath: .status.status + name: Status + type: string + - jsonPath: .status.role + name: Role + priority: 1 + type: string + - jsonPath: .status.releaseUpdate + name: Version + type: string + - jsonPath: .status.connectString + name: Connect Str + type: string + - jsonPath: .status.pdbConnectString + name: Pdb Connect Str + priority: 1 + type: string + - jsonPath: .status.oemExpressUrl + name: Oem Express Url + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + description: SingleInstanceDatabase is the Schema for the singleinstancedatabases + API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: SingleInstanceDatabaseSpec defines the desired state of SingleInstanceDatabase + properties: + adminPassword: + description: SingleInsatnceAdminPassword defines the secret containing + Admin Password mapped to secretKey for Database + properties: + keepSecret: + type: boolean + secretKey: + type: string + secretName: + type: string + required: + - secretKey + - secretName + type: object + archiveLog: + type: boolean + charset: + type: string + cloneFrom: + type: string + edition: + enum: + - standard + - enterprise + type: string + flashBack: + type: boolean + forceLog: + type: boolean + image: + description: SingleInstanceDatabaseImage defines the Image source + and pullSecrets for POD + properties: + pullFrom: + type: string + pullSecrets: + type: string + version: + type: string + required: + - pullFrom + type: object + initParams: + description: SingleInstanceDatabaseInitParams defines the Init Parameters + properties: + cpuCount: + type: integer + pgaAggregateTarget: + type: integer + processes: + type: integer + sgaTarget: + type: integer + type: object + installApex: + type: boolean + loadBalancer: + type: boolean + nodeSelector: + additionalProperties: + type: string + type: object + pdbName: + type: string + persistence: + description: SingleInstanceDatabasePersistence defines the storage + size and class for PVC + properties: + accessMode: + enum: + - ReadWriteOnce + - ReadWriteMany + type: string + size: + type: string + storageClass: + type: string + required: + - accessMode + - size + - storageClass + type: object + readinessCheckPeriod: + type: integer + replicas: + minimum: 1 + type: integer + sid: + description: SID can only have a-z , A-Z, 0-9 . It cant have any special + characters + pattern: ^[a-zA-Z0-9]+$ + type: string + required: + - adminPassword + - image + - persistence + - replicas + type: object + status: + description: SingleInstanceDatabaseStatus defines the observed state of + SingleInstanceDatabase + properties: + apexInstalled: + type: boolean + archiveLog: + type: string + charset: + type: string + cloneFrom: + type: string + clusterConnectString: + type: string + conditions: + items: + description: "Condition contains details for one aspect of the current + state of this API Resource. --- This struct is intended for direct + use as an array at the field path .status.conditions. For example, + type FooStatus struct{ // Represents the observations of a + foo's current state. // Known .status.conditions.type are: + \"Available\", \"Progressing\", and \"Degraded\" // +patchMergeKey=type + \ // +patchStrategy=merge // +listType=map // +listMapKey=type + \ Conditions []metav1.Condition `json:\"conditions,omitempty\" + patchStrategy:\"merge\" patchMergeKey:\"type\" protobuf:\"bytes,1,rep,name=conditions\"` + \n // other fields }" + properties: + lastTransitionTime: + description: lastTransitionTime is the last time the condition + transitioned from one status to another. This should be when + the underlying condition changed. If that is not known, then + using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: message is a human readable message indicating + details about the transition. This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: observedGeneration represents the .metadata.generation + that the condition was set based upon. For instance, if .metadata.generation + is currently 12, but the .status.conditions[x].observedGeneration + is 9, the condition is out of date with respect to the current + state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: reason contains a programmatic identifier indicating + the reason for the condition's last transition. Producers + of specific condition types may define expected values and + meanings for this field, and whether the values are considered + a guaranteed API. The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + --- Many .condition.type values are consistent across resources + like Available, but because arbitrary conditions can be useful + (see .node.status.conditions), the ability to deconflict is + important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + connectString: + type: string + datafilesCreated: + type: string + datafilesPatched: + type: string + edition: + type: string + flashBack: + type: string + forceLog: + type: string + initParams: + description: SingleInstanceDatabaseInitParams defines the Init Parameters + properties: + cpuCount: + type: integer + pgaAggregateTarget: + type: integer + processes: + type: integer + sgaTarget: + type: integer + type: object + initPgaSize: + type: integer + initSgaSize: + type: integer + nodes: + items: + type: string + type: array + oemExpressUrl: + type: string + ordsReference: + type: string + pdbConnectString: + type: string + pdbName: + type: string + persistence: + description: SingleInstanceDatabasePersistence defines the storage + size and class for PVC + properties: + accessMode: + enum: + - ReadWriteOnce + - ReadWriteMany + type: string + size: + type: string + storageClass: + type: string + required: + - accessMode + - size + - storageClass + type: object + releaseUpdate: + type: string + replicas: + type: integer + role: + type: string + sid: + type: string + standbyDatabases: + additionalProperties: + type: string + type: object + status: + type: string + required: + - persistence + - replicas + type: object + type: object + served: true + storage: true + subresources: + scale: + specReplicasPath: .spec.replicas + statusReplicasPath: .status.replicas + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] diff --git a/config/crd/kustomization.yaml b/config/crd/kustomization.yaml new file mode 100644 index 00000000..6b3d488e --- /dev/null +++ b/config/crd/kustomization.yaml @@ -0,0 +1,33 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +# This kustomization.yaml is not intended to be run by itself, +# since it depends on service name and namespace that are out of this kustomize package. +# It should be run by config/default +resources: +- bases/database.oracle.com_autonomousdatabases.yaml +- bases/database.oracle.com_singleinstancedatabases.yaml +- bases/database.oracle.com_shardingdatabases.yaml +# +kubebuilder:scaffold:crdkustomizeresource + +patchesStrategicMerge: +# [WEBHOOK] To enable webhook, uncomment all the sections with [WEBHOOK] prefix. +# patches here are for enabling the conversion webhook for each CRD +#- patches/webhook_in_provshards.yaml +#- patches/webhook_in_autonomousdatabases.yaml +#- patches/webhook_in_singleinstancedatabases.yaml +#- patches/webhook_in_shardingdatabases.yaml +# +kubebuilder:scaffold:crdkustomizewebhookpatch + +# [CERTMANAGER] To enable webhook, uncomment all the sections with [CERTMANAGER] prefix. +# patches here are for enabling the CA injection for each CRD +#- patches/cainjection_in_provshards.yaml +#- patches/cainjection_in_autonomousdatabases.yaml +- patches/cainjection_in_singleinstancedatabases.yaml +#- patches/cainjection_in_shardingdatabases.yaml +# +kubebuilder:scaffold:crdkustomizecainjectionpatch + +# the following config is for teaching kustomize how to do kustomization for CRDs. +configurations: +- kustomizeconfig.yaml diff --git a/config/crd/kustomizeconfig.yaml b/config/crd/kustomizeconfig.yaml new file mode 100644 index 00000000..fb9995dc --- /dev/null +++ b/config/crd/kustomizeconfig.yaml @@ -0,0 +1,21 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +# This file is for teaching kustomize how to substitute name and namespace reference in CRD +nameReference: +- kind: Service + version: v1 + fieldSpecs: + - kind: CustomResourceDefinition + group: apiextensions.k8s.io + path: spec/conversion/webhookClientConfig/service/name + +namespace: +- kind: CustomResourceDefinition + group: apiextensions.k8s.io + path: spec/conversion/webhookClientConfig/service/namespace + create: false + +varReference: +- path: metadata/annotations diff --git a/config/crd/patches/cainjection_in_autonomousdatabases.yaml b/config/crd/patches/cainjection_in_autonomousdatabases.yaml new file mode 100644 index 00000000..072e3f9e --- /dev/null +++ b/config/crd/patches/cainjection_in_autonomousdatabases.yaml @@ -0,0 +1,12 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +# The following patch adds a directive for certmanager to inject CA into the CRD +# CRD conversion requires k8s 1.13 or later. +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + cert-manager.io/inject-ca-from: $(CERTIFICATE_NAMESPACE)/$(CERTIFICATE_NAME) + name: autonomousdatabases.database.oracle.com diff --git a/config/crd/patches/cainjection_in_shardingdatabases.yaml b/config/crd/patches/cainjection_in_shardingdatabases.yaml new file mode 100644 index 00000000..6ef22218 --- /dev/null +++ b/config/crd/patches/cainjection_in_shardingdatabases.yaml @@ -0,0 +1,12 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +# The following patch adds a directive for certmanager to inject CA into the CRD +# CRD conversion requires k8s 1.13 or later. +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + cert-manager.io/inject-ca-from: $(CERTIFICATE_NAMESPACE)/$(CERTIFICATE_NAME) + name: shardingdatabases.database.oracle.com diff --git a/config/crd/patches/cainjection_in_singleinstancedatabases.yaml b/config/crd/patches/cainjection_in_singleinstancedatabases.yaml new file mode 100644 index 00000000..4e454c1a --- /dev/null +++ b/config/crd/patches/cainjection_in_singleinstancedatabases.yaml @@ -0,0 +1,12 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +# The following patch adds a directive for certmanager to inject CA into the CRD +# CRD conversion requires k8s 1.13 or later. +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + cert-manager.io/inject-ca-from: $(CERTIFICATE_NAMESPACE)/$(CERTIFICATE_NAME) + name: singleinstancedatabases.database.oracle.com diff --git a/config/crd/patches/webhook_in_autonomousdatabases.yaml b/config/crd/patches/webhook_in_autonomousdatabases.yaml new file mode 100644 index 00000000..55540503 --- /dev/null +++ b/config/crd/patches/webhook_in_autonomousdatabases.yaml @@ -0,0 +1,21 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +# The following patch enables conversion webhook for CRD +# CRD conversion requires k8s 1.13 or later. +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + name: autonomousdatabases.database.oracle.com +spec: + conversion: + strategy: Webhook + webhookClientConfig: + # this is "\n" used as a placeholder, otherwise it will be rejected by the apiserver for being blank, + # but we're going to set it later using the cert-manager (or potentially a patch if not using cert-manager) + caBundle: Cg== + service: + namespace: system + name: webhook-service + path: /convert diff --git a/config/crd/patches/webhook_in_shardingdatabases.yaml b/config/crd/patches/webhook_in_shardingdatabases.yaml new file mode 100644 index 00000000..fccda7d0 --- /dev/null +++ b/config/crd/patches/webhook_in_shardingdatabases.yaml @@ -0,0 +1,21 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +# The following patch enables conversion webhook for CRD +# CRD conversion requires k8s 1.13 or later. +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + name: shardingdatabases.database.oracle.com +spec: + conversion: + strategy: Webhook + webhookClientConfig: + # this is "\n" used as a placeholder, otherwise it will be rejected by the apiserver for being blank, + # but we're going to set it later using the cert-manager (or potentially a patch if not using cert-manager) + caBundle: Cg== + service: + namespace: system + name: webhook-service + path: /convert diff --git a/config/crd/patches/webhook_in_singleinstancedatabases.yaml b/config/crd/patches/webhook_in_singleinstancedatabases.yaml new file mode 100644 index 00000000..66687d8e --- /dev/null +++ b/config/crd/patches/webhook_in_singleinstancedatabases.yaml @@ -0,0 +1,21 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +# The following patch enables conversion webhook for CRD +# CRD conversion requires k8s 1.13 or later. +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + name: singleinstancedatabases.database.oracle.com +spec: + conversion: + strategy: Webhook + webhookClientConfig: + # this is "\n" used as a placeholder, otherwise it will be rejected by the apiserver for being blank, + # but we're going to set it later using the cert-manager (or potentially a patch if not using cert-manager) + caBundle: Cg== + service: + namespace: system + name: webhook-service + path: /convert diff --git a/config/default/kustomization.yaml b/config/default/kustomization.yaml new file mode 100644 index 00000000..04f37b2c --- /dev/null +++ b/config/default/kustomization.yaml @@ -0,0 +1,75 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +# Adds namespace to all resources. +namespace: oracle-database-operator-system + +# Value of this field is prepended to the +# names of all resources, e.g. a deployment named +# "wordpress" becomes "alices-wordpress". +# Note that it should also match with the prefix (text before '-') of the namespace +# field above. +namePrefix: oracle-database-operator- + +# Labels to add to all resources and selectors. +#commonLabels: +# someName: someValue + +bases: +- ../crd +- ../rbac +- ../manager +# [WEBHOOK] To enable webhook, uncomment all the sections with [WEBHOOK] prefix including the one in +# crd/kustomization.yaml +- ../webhook +# [CERTMANAGER] To enable cert-manager, uncomment all sections with 'CERTMANAGER'. 'WEBHOOK' components are required. +- ../certmanager +# [PROMETHEUS] To enable prometheus monitor, uncomment all sections with 'PROMETHEUS'. +#- ../prometheus + +patchesStrategicMerge: + # Protect the /metrics endpoint by putting it behind auth. + # If you want your controller-manager to expose the /metrics + # endpoint w/o any authn/z, please comment the following line. +#- manager_auth_proxy_patch.yaml + +# [WEBHOOK] To enable webhook, uncomment all the sections with [WEBHOOK] prefix including the one in +# crd/kustomization.yaml +- manager_webhook_patch.yaml + +# [CERTMANAGER] To enable cert-manager, uncomment all sections with 'CERTMANAGER'. +# Uncomment 'CERTMANAGER' sections in crd/kustomization.yaml to enable the CA injection in the admission webhooks. +# 'CERTMANAGER' needs to be enabled to use ca injection +- webhookcainjection_patch.yaml + +# the following config is for teaching kustomize how to do var substitution +vars: +# [CERTMANAGER] To enable cert-manager, uncomment all sections with 'CERTMANAGER' prefix. +- name: CERTIFICATE_NAMESPACE # namespace of the certificate CR + objref: + kind: Certificate + group: cert-manager.io + version: v1 + name: serving-cert # this name should match the one in certificate.yaml + fieldref: + fieldpath: metadata.namespace +- name: CERTIFICATE_NAME + objref: + kind: Certificate + group: cert-manager.io + version: v1 + name: serving-cert # this name should match the one in certificate.yaml +- name: SERVICE_NAMESPACE # namespace of the service + objref: + kind: Service + version: v1 + name: webhook-service + fieldref: + fieldpath: metadata.namespace +- name: SERVICE_NAME + objref: + kind: Service + version: v1 + name: webhook-service diff --git a/config/default/manager_auth_proxy_patch.yaml b/config/default/manager_auth_proxy_patch.yaml new file mode 100644 index 00000000..c88bb7a7 --- /dev/null +++ b/config/default/manager_auth_proxy_patch.yaml @@ -0,0 +1,30 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +# This patch inject a sidecar container which is a HTTP proxy for the +# controller manager, it performs RBAC authorization against the Kubernetes API using SubjectAccessReviews. +apiVersion: apps/v1 +kind: Deployment +metadata: + name: controller-manager + namespace: system +spec: + template: + spec: + containers: + - name: kube-rbac-proxy + image: gcr.io/kubebuilder/kube-rbac-proxy:v0.5.0 + args: + - "--secure-listen-address=0.0.0.0:8443" + - "--upstream=http://127.0.0.1:8080/" + - "--logtostderr=true" + - "--v=10" + ports: + - containerPort: 8443 + name: https + - name: manager + args: + - "--metrics-addr=127.0.0.1:8080" + - "--enable-leader-election" diff --git a/config/default/manager_webhook_patch.yaml b/config/default/manager_webhook_patch.yaml new file mode 100644 index 00000000..cda36c4c --- /dev/null +++ b/config/default/manager_webhook_patch.yaml @@ -0,0 +1,27 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: apps/v1 +kind: Deployment +metadata: + name: controller-manager + namespace: system +spec: + template: + spec: + containers: + - name: manager + ports: + - containerPort: 9443 + name: webhook-server + protocol: TCP + volumeMounts: + - mountPath: /tmp/k8s-webhook-server/serving-certs + name: cert + readOnly: true + volumes: + - name: cert + secret: + defaultMode: 420 + secretName: webhook-server-cert diff --git a/config/default/webhookcainjection_patch.yaml b/config/default/webhookcainjection_patch.yaml new file mode 100644 index 00000000..51b1069f --- /dev/null +++ b/config/default/webhookcainjection_patch.yaml @@ -0,0 +1,20 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +# This patch add annotation to admission webhook config and +# the variables $(CERTIFICATE_NAMESPACE) and $(CERTIFICATE_NAME) will be substituted by kustomize. +apiVersion: admissionregistration.k8s.io/v1 +kind: MutatingWebhookConfiguration +metadata: + name: mutating-webhook-configuration + annotations: + cert-manager.io/inject-ca-from: $(CERTIFICATE_NAMESPACE)/$(CERTIFICATE_NAME) +--- +apiVersion: admissionregistration.k8s.io/v1 +kind: ValidatingWebhookConfiguration +metadata: + name: validating-webhook-configuration + annotations: + cert-manager.io/inject-ca-from: $(CERTIFICATE_NAMESPACE)/$(CERTIFICATE_NAME) diff --git a/config/manager/kustomization.yaml b/config/manager/kustomization.yaml new file mode 100644 index 00000000..30ed1f75 --- /dev/null +++ b/config/manager/kustomization.yaml @@ -0,0 +1,12 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +resources: +- manager.yaml +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +images: +- name: controller + newName: container-registry.oracle.com/database/operator + newTag: 0.1.0 diff --git a/config/manager/manager.yaml b/config/manager/manager.yaml new file mode 100644 index 00000000..362aac61 --- /dev/null +++ b/config/manager/manager.yaml @@ -0,0 +1,46 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: v1 +kind: Namespace +metadata: + labels: + control-plane: controller-manager + name: system +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: controller-manager + namespace: system + labels: + control-plane: controller-manager +spec: + selector: + matchLabels: + control-plane: controller-manager + replicas: 3 + template: + metadata: + labels: + control-plane: controller-manager + spec: + imagePullSecrets: + - name: container-registry-secret + containers: + - command: + - /manager + args: + - --enable-leader-election + image: controller:latest + imagePullPolicy: Always + name: manager + resources: + limits: + cpu: 400m + memory: 400Mi + requests: + cpu: 400m + memory: 400Mi + terminationGracePeriodSeconds: 10 diff --git a/config/manifests/bases/oracle-database-operator.clusterserviceversion.yaml b/config/manifests/bases/oracle-database-operator.clusterserviceversion.yaml new file mode 100644 index 00000000..6ec37dd1 --- /dev/null +++ b/config/manifests/bases/oracle-database-operator.clusterserviceversion.yaml @@ -0,0 +1,48 @@ +apiVersion: operators.coreos.com/v1alpha1 +kind: ClusterServiceVersion +metadata: + annotations: + alm-examples: '[]' + capabilities: Basic Install + operators.operatorframework.io/builder: operator-sdk-v1.2.0 + operators.operatorframework.io/project_layout: go.kubebuilder.io/v2 + name: oracle-database-operator.v0.0.0 + namespace: placeholder +spec: + apiservicedefinitions: {} + customresourcedefinitions: + owned: + - description: AutonomousDatabase is the Schema for the autonomousdatabases API + displayName: Autonomous Database + kind: AutonomousDatabase + name: autonomousdatabases.database.oracle.com + version: v1alpha1 + description: Operator to manage Oracle sharding + displayName: Oracle Sharding DB Operator + icon: + - base64data: "" + mediatype: "" + install: + spec: + deployments: null + strategy: "" + installModes: + - supported: false + type: OwnNamespace + - supported: false + type: SingleNamespace + - supported: false + type: MultiNamespace + - supported: true + type: AllNamespaces + keywords: + - Oracle + - sharding + - db + links: + - name: Oracle Database Operator + url: https://oracle-database-operator.domain + maturity: alpha + provider: + name: ShardingDatabase + version: 0.0.0 diff --git a/config/manifests/kustomization.yaml b/config/manifests/kustomization.yaml new file mode 100644 index 00000000..2a0f628b --- /dev/null +++ b/config/manifests/kustomization.yaml @@ -0,0 +1,8 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +resources: +- ../default +- ../samples +- ../scorecard diff --git a/config/rbac/auth_proxy_client_clusterrole.yaml b/config/rbac/auth_proxy_client_clusterrole.yaml new file mode 100644 index 00000000..b3e55375 --- /dev/null +++ b/config/rbac/auth_proxy_client_clusterrole.yaml @@ -0,0 +1,11 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: metrics-reader +rules: +- nonResourceURLs: ["/metrics"] + verbs: ["get"] diff --git a/config/rbac/auth_proxy_role.yaml b/config/rbac/auth_proxy_role.yaml new file mode 100644 index 00000000..fd7483d6 --- /dev/null +++ b/config/rbac/auth_proxy_role.yaml @@ -0,0 +1,17 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: oracle-database-operator-proxy-role +rules: +- apiGroups: ["authentication.k8s.io"] + resources: + - tokenreviews + verbs: ["create"] +- apiGroups: ["authorization.k8s.io"] + resources: + - subjectaccessreviews + verbs: ["create"] diff --git a/config/rbac/auth_proxy_role_binding.yaml b/config/rbac/auth_proxy_role_binding.yaml new file mode 100644 index 00000000..5632b87b --- /dev/null +++ b/config/rbac/auth_proxy_role_binding.yaml @@ -0,0 +1,16 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: oracle-database-operator-proxy-rolebinding +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: proxy-role +subjects: +- kind: ServiceAccount + name: default + namespace: system diff --git a/config/rbac/auth_proxy_service.yaml b/config/rbac/auth_proxy_service.yaml new file mode 100644 index 00000000..72b1ecb8 --- /dev/null +++ b/config/rbac/auth_proxy_service.yaml @@ -0,0 +1,18 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: v1 +kind: Service +metadata: + labels: + control-plane: controller-manager + name: controller-manager-metrics-service + namespace: system +spec: + ports: + - name: https + port: 8443 + targetPort: https + selector: + control-plane: controller-manager diff --git a/config/rbac/autonomousdatabase_editor_role.yaml b/config/rbac/autonomousdatabase_editor_role.yaml new file mode 100644 index 00000000..bd6172fd --- /dev/null +++ b/config/rbac/autonomousdatabase_editor_role.yaml @@ -0,0 +1,29 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +# permissions for end users to edit autonomousdatabases. +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: autonomousdatabase-editor-role +rules: +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabases + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabases/status + verbs: + - get diff --git a/config/rbac/autonomousdatabase_viewer_role.yaml b/config/rbac/autonomousdatabase_viewer_role.yaml new file mode 100644 index 00000000..af24143f --- /dev/null +++ b/config/rbac/autonomousdatabase_viewer_role.yaml @@ -0,0 +1,25 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +# permissions for end users to view autonomousdatabases. +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: autonomousdatabase-viewer-role +rules: +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabases + verbs: + - get + - list + - watch +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabases/status + verbs: + - get diff --git a/config/rbac/kustomization.yaml b/config/rbac/kustomization.yaml new file mode 100644 index 00000000..6325cf8c --- /dev/null +++ b/config/rbac/kustomization.yaml @@ -0,0 +1,16 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +resources: +- role.yaml +- role_binding.yaml +- leader_election_role.yaml +- leader_election_role_binding.yaml +# Comment the following 4 lines if you want to disable +# the auth proxy (https://github.com/brancz/kube-rbac-proxy) +# which protects your /metrics endpoint. +- auth_proxy_service.yaml +- auth_proxy_role.yaml +- auth_proxy_role_binding.yaml +- auth_proxy_client_clusterrole.yaml diff --git a/config/rbac/leader_election_role.yaml b/config/rbac/leader_election_role.yaml new file mode 100644 index 00000000..083cd887 --- /dev/null +++ b/config/rbac/leader_election_role.yaml @@ -0,0 +1,50 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +# permissions to do leader election. +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: leader-election-role +rules: +- apiGroups: + - "" + resources: + - configmaps + verbs: + - get + - list + - watch + - create + - update + - patch + - delete +- apiGroups: + - coordination.k8s.io + resources: + - leases + verbs: + - get + - list + - watch + - create + - update + - patch + - delete +- apiGroups: + - "" + resources: + - configmaps/status + verbs: + - get + - update + - patch +- apiGroups: + - "" + resources: + - events + verbs: + - create + - patch diff --git a/config/rbac/leader_election_role_binding.yaml b/config/rbac/leader_election_role_binding.yaml new file mode 100644 index 00000000..01f5d630 --- /dev/null +++ b/config/rbac/leader_election_role_binding.yaml @@ -0,0 +1,16 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: oracle-database-operator-leader-election-rolebinding +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: leader-election-role +subjects: +- kind: ServiceAccount + name: default + namespace: oracle-db diff --git a/config/rbac/provshard_editor_role.yaml b/config/rbac/provshard_editor_role.yaml new file mode 100644 index 00000000..473a41c0 --- /dev/null +++ b/config/rbac/provshard_editor_role.yaml @@ -0,0 +1,29 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +# permissions for end users to edit provshards. +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: provshard-editor-role +rules: +- apiGroups: + - sharding.oracle.com + resources: + - provshards + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - sharding.oracle.com + resources: + - provshards/status + verbs: + - get diff --git a/config/rbac/provshard_viewer_role.yaml b/config/rbac/provshard_viewer_role.yaml new file mode 100644 index 00000000..36b5bb4b --- /dev/null +++ b/config/rbac/provshard_viewer_role.yaml @@ -0,0 +1,25 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +# permissions for end users to view provshards. +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: provshard-viewer-role +rules: +- apiGroups: + - sharding.oracle.com + resources: + - provshards + verbs: + - get + - list + - watch +- apiGroups: + - sharding.oracle.com + resources: + - provshards/status + verbs: + - get diff --git a/config/rbac/role.yaml b/config/rbac/role.yaml new file mode 100644 index 00000000..6d763212 --- /dev/null +++ b/config/rbac/role.yaml @@ -0,0 +1,174 @@ + +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + creationTimestamp: null + name: manager-role +rules: +- apiGroups: + - "" + resources: + - configmaps + - secrets + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - "" + resources: + - events + - nodes + - persistentvolumeclaims + - pods + - pods/exec + - pods/log + - services + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - '''''' + resources: + - statefulsets/finalizers + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - apps + resources: + - statefulsets + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - coordination.k8s.io + resources: + - leases + verbs: + - create + - get + - list + - update +- apiGroups: + - "" + resources: + - configmaps + - events + - namespaces + - nodes + - persistentvolumeclaims + - pods + - pods/exec + - pods/log + - secrets + - services + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - "" + resources: + - pods/exec + verbs: + - create +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabases + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabases/status + verbs: + - patch + - update +- apiGroups: + - database.oracle.com + resources: + - shardingdatabases + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - database.oracle.com + resources: + - shardingdatabases/finalizers + verbs: + - create + - delete + - get + - patch + - update +- apiGroups: + - database.oracle.com + resources: + - shardingdatabases/status + verbs: + - get + - patch + - update +- apiGroups: + - database.oracle.com + resources: + - singleinstancedatabases + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - database.oracle.com + resources: + - singleinstancedatabases/finalizers + verbs: + - update +- apiGroups: + - database.oracle.com + resources: + - singleinstancedatabases/status + verbs: + - get + - patch + - update diff --git a/config/rbac/role_binding.yaml b/config/rbac/role_binding.yaml new file mode 100644 index 00000000..1ec61dc5 --- /dev/null +++ b/config/rbac/role_binding.yaml @@ -0,0 +1,16 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: oracle-database-operator-manager-rolebinding +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: manager-role +subjects: +- kind: ServiceAccount + name: default + namespace: oracle-db diff --git a/config/rbac/shardingdatabase_editor_role.yaml b/config/rbac/shardingdatabase_editor_role.yaml new file mode 100644 index 00000000..f9660cb7 --- /dev/null +++ b/config/rbac/shardingdatabase_editor_role.yaml @@ -0,0 +1,29 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +# permissions for end users to edit shardingdatabases. +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: shardingdatabase-editor-role +rules: +- apiGroups: + - database.oracle.com + resources: + - shardingdatabases + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - database.oracle.com + resources: + - shardingdatabases/status + verbs: + - get diff --git a/config/rbac/shardingdatabase_viewer_role.yaml b/config/rbac/shardingdatabase_viewer_role.yaml new file mode 100644 index 00000000..717c61a4 --- /dev/null +++ b/config/rbac/shardingdatabase_viewer_role.yaml @@ -0,0 +1,25 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +# permissions for end users to view shardingdatabases. +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: shardingdatabase-viewer-role +rules: +- apiGroups: + - database.oracle.com + resources: + - shardingdatabases + verbs: + - get + - list + - watch +- apiGroups: + - database.oracle.com + resources: + - shardingdatabases/status + verbs: + - get diff --git a/config/rbac/singleinstancedatabase_editor_role.yaml b/config/rbac/singleinstancedatabase_editor_role.yaml new file mode 100644 index 00000000..90a19c43 --- /dev/null +++ b/config/rbac/singleinstancedatabase_editor_role.yaml @@ -0,0 +1,29 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +# permissions for end users to edit singleinstancedatabases. +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: singleinstancedatabase-editor-role +rules: +- apiGroups: + - database.oracle.com + resources: + - singleinstancedatabases + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - database.oracle.com + resources: + - singleinstancedatabases/status + verbs: + - get diff --git a/config/rbac/singleinstancedatabase_viewer_role.yaml b/config/rbac/singleinstancedatabase_viewer_role.yaml new file mode 100644 index 00000000..84bea03d --- /dev/null +++ b/config/rbac/singleinstancedatabase_viewer_role.yaml @@ -0,0 +1,25 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +# permissions for end users to view singleinstancedatabases. +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: singleinstancedatabase-viewer-role +rules: +- apiGroups: + - database.oracle.com + resources: + - singleinstancedatabases + verbs: + - get + - list + - watch +- apiGroups: + - database.oracle.com + resources: + - singleinstancedatabases/status + verbs: + - get diff --git a/config/samples/autonomousdatabase.yaml b/config/samples/autonomousdatabase.yaml new file mode 100644 index 00000000..c1dfc078 --- /dev/null +++ b/config/samples/autonomousdatabase.yaml @@ -0,0 +1,11 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: AutonomousDatabase +metadata: + name: autonomousdatabase-sample +spec: + # Add fields here + foo: bar diff --git a/config/samples/autonomousdatabase_bind.yaml b/config/samples/autonomousdatabase_bind.yaml new file mode 100644 index 00000000..d52b9c18 --- /dev/null +++ b/config/samples/autonomousdatabase_bind.yaml @@ -0,0 +1,15 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: AutonomousDatabase +metadata: + name: autonomousdatabase-sample +spec: + details: + autonomousDatabaseOCID: ocid1.autonomousdatabase... + # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey \ No newline at end of file diff --git a/config/samples/autonomousdatabase_change_admin_password.yaml b/config/samples/autonomousdatabase_change_admin_password.yaml new file mode 100644 index 00000000..47ad1d65 --- /dev/null +++ b/config/samples/autonomousdatabase_change_admin_password.yaml @@ -0,0 +1,20 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: AutonomousDatabase +metadata: + name: autonomousdatabase-sample +spec: + details: + autonomousDatabaseOCID: ocid1.autonomousdatabase... + adminPassword: + # The Name of the secret where you want to hold the password of the ADMIN account. Comment out k8sSecretName and uncomment ociSecretOCID if you pass the admin password using OCI Secret. + k8sSecretName: new-admin-password + # The OCID of the OCI Secret that holds the password of the ADMIN account. It should start with ocid1.vaultsecret... . + # ociSecretOCID: ocid1.vaultsecret... + # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey \ No newline at end of file diff --git a/config/samples/autonomousdatabase_create.yaml b/config/samples/autonomousdatabase_create.yaml new file mode 100644 index 00000000..fb58b656 --- /dev/null +++ b/config/samples/autonomousdatabase_create.yaml @@ -0,0 +1,26 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: AutonomousDatabase +metadata: + name: autonomousdatabase-sample +spec: + details: + # Update compartmentOCID with your compartment OCID. + compartmentOCID: ocid1.compartment... + # The dbName must begin with an alphabetic character and can contain a maximum of 14 alphanumeric characters. Special characters are not permitted. The database name must be unique in the tenancy. + dbName: NewADB + displayName: NewADB + cpuCoreCount: 1 + adminPassword: + # The Name of the K8s secret where you want to hold the password of the ADMIN account. Comment out k8sSecretName and uncomment ociSecretOCID if you pass the admin password using OCI Secret. + k8sSecretName: admin-password + # The OCID of the OCI Secret that holds the password of the ADMIN account. It should start with ocid1.vaultsecret... . + # ociSecretOCID: ocid1.vaultsecret... + dataStorageSizeInTBs: 1 + # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey \ No newline at end of file diff --git a/config/samples/autonomousdatabase_delete_resource.yaml b/config/samples/autonomousdatabase_delete_resource.yaml new file mode 100644 index 00000000..e075787e --- /dev/null +++ b/config/samples/autonomousdatabase_delete_resource.yaml @@ -0,0 +1,17 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: AutonomousDatabase +metadata: + name: autonomousdatabase-sample +spec: + details: + autonomousDatabaseOCID: ocid1.autonomousdatabase... + # Delete this resource to terminate database after the changes applied + hardLink: true + # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey \ No newline at end of file diff --git a/config/samples/autonomousdatabase_rename.yaml b/config/samples/autonomousdatabase_rename.yaml new file mode 100644 index 00000000..8aa2ae8b --- /dev/null +++ b/config/samples/autonomousdatabase_rename.yaml @@ -0,0 +1,19 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: AutonomousDatabase +metadata: + name: autonomousdatabase-sample +spec: + details: + autonomousDatabaseOCID: ocid1.autonomousdatabase... + # The database name + dbName: RenamedADB + # The user-friendly name for the Autonomous Database + displayName: RenamedADB + # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey \ No newline at end of file diff --git a/config/samples/autonomousdatabase_scale.yaml b/config/samples/autonomousdatabase_scale.yaml new file mode 100644 index 00000000..4a7c85e9 --- /dev/null +++ b/config/samples/autonomousdatabase_scale.yaml @@ -0,0 +1,21 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: AutonomousDatabase +metadata: + name: autonomousdatabase-sample +spec: + details: + autonomousDatabaseOCID: ocid1.autonomousdatabase... + # Your database's OPCU core count + cpuCoreCount: 2 + # Your database's storage size in TB + dataStorageSizeInTBs: 2 + # Enable/Disable auto scaling for your database + isAutoScalingEnabled: true + # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey \ No newline at end of file diff --git a/config/samples/autonomousdatabase_stop_start_terminate.yaml b/config/samples/autonomousdatabase_stop_start_terminate.yaml new file mode 100644 index 00000000..83d831cd --- /dev/null +++ b/config/samples/autonomousdatabase_stop_start_terminate.yaml @@ -0,0 +1,17 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: AutonomousDatabase +metadata: + name: autonomousdatabase-sample +spec: + details: + autonomousDatabaseOCID: ocid1.autonomousdatabase... + # Change the lifecycleState to "AVAILABLE" to start the database + lifecycleState: STOPPED + # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey \ No newline at end of file diff --git a/config/samples/autonomousdatabase_wallet.yaml b/config/samples/autonomousdatabase_wallet.yaml new file mode 100644 index 00000000..34953403 --- /dev/null +++ b/config/samples/autonomousdatabase_wallet.yaml @@ -0,0 +1,23 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: AutonomousDatabase +metadata: + name: autonomousdatabase-sample +spec: + details: + autonomousDatabaseOCID: ocid1.autonomousdatabase... + wallet: + # Insert a name of the secret where you want the wallet to be stored. The default name is -instance-wallet. + name: instance-wallet + password: + # The Name of the secret where you want to hold the wallet password. Comment out k8sSecretName and uncomment ociSecretOCID if you pass the wallet password using OCI Secret. + k8sSecretName: instance-wallet-password + # The OCID of the OCI Secret that holds the password of the ADMIN account. It should start with ocid1.vaultsecret... . + # ociSecretOCID: ocid1.vaultsecret... + # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey diff --git a/config/samples/kustomization.yaml b/config/samples/kustomization.yaml new file mode 100644 index 00000000..4b419923 --- /dev/null +++ b/config/samples/kustomization.yaml @@ -0,0 +1,12 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +## Append samples you want in your CSV to this file as resources ## +resources: + - sharding_v1alpha1_provshard.yaml + - autonomousdatabase.yaml + - singleinstancedatabase.yaml + - shardingdatabase.yaml + # +kubebuilder:scaffold:manifestskustomizesamples diff --git a/config/samples/sharding_v1alpha1_provshard.yaml b/config/samples/sharding_v1alpha1_provshard.yaml new file mode 100644 index 00000000..60f5f007 --- /dev/null +++ b/config/samples/sharding_v1alpha1_provshard.yaml @@ -0,0 +1,28 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: sharding.oracle.com/v1alpha1 +kind: ProvShard +metadata: + name: provshard-sample +spec: + shard: + - name: prod + storageSizeInGb: 50 + catalog: + - name: catalog + storageSizeInGb: 50 + gsm: + - name: gsm1 + storageSizeInGb: 50 + replicas: 1 + - name: gsm2 + storageSizeInGb: 50 + replicas: 1 + storageClass: oci + dbImage: phx.ocir.io/intsanjaysingh/db-repo/oracle/database:21.3.0-ee + gsmImage: phx.ocir.io/intsanjaysingh/db-repo/oracle/database-gsm:21.3.0 + secret: db-user-pass + isExternalSvc: false + namespace: sample-shard diff --git a/config/samples/sharding_v1alpha1_provshard_clonespec.yaml b/config/samples/sharding_v1alpha1_provshard_clonespec.yaml new file mode 100644 index 00000000..414544bd --- /dev/null +++ b/config/samples/sharding_v1alpha1_provshard_clonespec.yaml @@ -0,0 +1,73 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: sharding.oracle.com/v1alpha1 +kind: ProvShard +metadata: + name: provshard-sample + namespace: sample-shard +spec: + shard: + - name: shard1 + storageSizeInGb: 50 + nodeSelector: + failure-domain.beta.kubernetes.io/zone: "PHX-AD-1" + - name: shard2 + storageSizeInGb: 50 + nodeSelector: + failure-domain.beta.kubernetes.io/zone: "PHX-AD-1" + - name: shard3 + storageSizeInGb: 50 + nodeSelector: + failure-domain.beta.kubernetes.io/zone: "PHX-AD-1" + - name: shard4 + storageSizeInGb: 50 + nodeSelector: + failure-domain.beta.kubernetes.io/zone: "PHX-AD-1" + catalog: + - name: catalog + storageSizeInGb: 50 + nodeSelector: + failure-domain.beta.kubernetes.io/zone: "PHX-AD-1" + gsm: + - name: gsm1 + storageSizeInGb: 50 + replicas: 1 + envVars: + - name: "BASE_DIR" + value: "/opt/oracle/scripts/setup" + - name: "SCRIPT_NAME" + value: "main.py" + - name: "EXECUTOR" + value: "/bin/python" + nodeSelector: + failure-domain.beta.kubernetes.io/zone: "PHX-AD-1" + - name: gsm2 + storageSizeInGb: 50 + replicas: 1 + envVars: + - name: "BASE_DIR" + value: "/opt/oracle/scripts/setup" + - name: "SCRIPT_NAME" + value: "main.py" + - name: "EXECUTOR" + value: "/bin/python" + nodeSelector: + failure-domain.beta.kubernetes.io/zone: "PHX-AD-1" + storageClass: oci + dbImage: phx.ocir.io/intsanjaysingh/db-repo/oracle/database:21.3.0-ee + gsmImage: phx.ocir.io/intsanjaysingh/db-repo/oracle/database-gsm:21.3.0 + scriptsLocation: /mnt/stage/sharding/scripts + secret: db-user-pass + isExternalSvc: false + isClone: True + isDeleteOraPvc: True + namespace: sample-shard + stagePvcName: pv-stage-vol-claim + nsConfigMap: onsconfigmap + nsSecret: my-secret + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volume.oc1.phx.abyhqljrrmuodpfn3jevjr4kksaaubyjnusykgthfsahi4eazyppiuhbtwwa + pvMatchLabels: + failure-domain.beta.kubernetes.io/zone: "PHX-AD-1" diff --git a/config/samples/sharding_v1alpha1_provshard_clonespec1.yaml b/config/samples/sharding_v1alpha1_provshard_clonespec1.yaml new file mode 100644 index 00000000..67988b2c --- /dev/null +++ b/config/samples/sharding_v1alpha1_provshard_clonespec1.yaml @@ -0,0 +1,65 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: sharding.oracle.com/v1alpha1 +kind: ProvShard +metadata: + name: myshard-sample + namespace: sample-shard +spec: + shard: + - name: shard11 + storageSizeInGb: 50 + nodeSelector: + failure-domain.beta.kubernetes.io/zone: "PHX-AD-1" + - name: shard12 + storageSizeInGb: 50 + nodeSelector: + failure-domain.beta.kubernetes.io/zone: "PHX-AD-1" + catalog: + - name: catdb + storageSizeInGb: 50 + nodeSelector: + failure-domain.beta.kubernetes.io/zone: "PHX-AD-1" + gsm: + - name: mygsm1 + storageSizeInGb: 50 + replicas: 1 + envVars: + - name: "BASE_DIR" + value: "/opt/oracle/scripts/setup" + - name: "SCRIPT_NAME" + value: "main.py" + - name: "EXECUTOR" + value: "/bin/python" + nodeSelector: + failure-domain.beta.kubernetes.io/zone: "PHX-AD-1" + - name: mygsm2 + storageSizeInGb: 50 + replicas: 1 + envVars: + - name: "BASE_DIR" + value: "/opt/oracle/scripts/setup" + - name: "SCRIPT_NAME" + value: "main.py" + - name: "EXECUTOR" + value: "/bin/python" + nodeSelector: + failure-domain.beta.kubernetes.io/zone: "PHX-AD-1" + storageClass: oci + dbImage: phx.ocir.io/intsanjaysingh/db-repo/oracle/database:21.3.0-ee + gsmImage: phx.ocir.io/intsanjaysingh/db-repo/oracle/database-gsm:21.3.0 + scriptsLocation: /mnt/stage/sharding/scripts + secret: db-user-pass + isExternalSvc: false + isClone: True + isDeleteOraPvc: True + namespace: sample-shard + stagePvcName: pv-stage-vol-claim + nsConfigMap: onsconfigmap + nsSecret: my-secret + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volume.oc1.phx.abyhqljrrmuodpfn3jevjr4kksaaubyjnusykgthfsahi4eazyppiuhbtwwa + pvMatchLabels: + failure-domain.beta.kubernetes.io/zone: "PHX-AD-1" diff --git a/config/samples/sharding_v1alpha1_provshard_orig.yaml b/config/samples/sharding_v1alpha1_provshard_orig.yaml new file mode 100644 index 00000000..fc7ba66d --- /dev/null +++ b/config/samples/sharding_v1alpha1_provshard_orig.yaml @@ -0,0 +1,58 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: sharding.oracle.com/v1alpha1 +kind: ProvShard +metadata: + name: provshard-sample +spec: + shard: + - name: prod + storageSizeInGb: 50 + nodeSelector: + failure-domain.beta.kubernetes.io/zone: "PHX-AD-1" + - name: test + storageSizeInGb: 50 + nodeSelector: + failure-domain.beta.kubernetes.io/zone: "PHX-AD-1" + catalog: + - name: catalog + storageSizeInGb: 50 + nodeSelector: + failure-domain.beta.kubernetes.io/zone: "PHX-AD-1" + gsm: + - name: gsm1 + storageSizeInGb: 50 + replicas: 1 + envVars: + - name: "BASE_DIR" + value: "/opt/oracle/scripts/setup" + - name: "SCRIPT_NAME" + value: "main.py" + - name: "EXECUTOR" + value: "/bin/python" + nodeSelector: + failure-domain.beta.kubernetes.io/zone: "PHX-AD-1" + - name: gsm2 + storageSizeInGb: 50 + replicas: 1 + envVars: + - name: "BASE_DIR" + value: "/opt/oracle/scripts/setup" + - name: "SCRIPT_NAME" + value: "main.py" + - name: "EXECUTOR" + value: "/bin/python" + nodeSelector: + failure-domain.beta.kubernetes.io/zone: "PHX-AD-1" + storageClass: oci + dbImage: phx.ocir.io/intsanjaysingh/db-repo/oracle/database:21.3.0-ee + gsmImage: phx.ocir.io/intsanjaysingh/db-repo/oracle/database-gsm:21.3.0 + scriptsLocation: /mnt/stage/sharding/scripts + secret: db-user-pass + isExternalSvc: false + namespace: sample-shard + stagePvcName: pv-stage-vol-claim + nsConfigMap: onsconfigmap + nsSecret: my-secret diff --git a/config/samples/shardingdatabase.yaml b/config/samples/shardingdatabase.yaml new file mode 100644 index 00000000..40453b49 --- /dev/null +++ b/config/samples/shardingdatabase.yaml @@ -0,0 +1,11 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: ShardingDatabase +metadata: + name: shardingdatabase-sample +spec: + # Add fields here + foo: bar diff --git a/config/samples/singleinstancedatabase.yaml b/config/samples/singleinstancedatabase.yaml new file mode 100644 index 00000000..ac140361 --- /dev/null +++ b/config/samples/singleinstancedatabase.yaml @@ -0,0 +1,79 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: SingleInstanceDatabase +metadata: + name: singleinstancedatabase-sample + namespace: default +spec: + + ## Use only alphanumeric characters for sid + sid: ORCL1 + + ## A source database ref to clone from, leave empty to create a fresh database + cloneFrom: "" + + ## NA if cloning from a SourceDB (cloneFrom is set) + edition: enterprise + + ## Should refer to SourceDB secret if cloning from a SourceDB (cloneFrom is set) + ## Secret containing SIDB password mapped to secretKey + ## This secret will be deleted after creation of the database unless keepSecret is set to true + adminPassword: + secretName: + secretKey: + keepSecret: false + + ## NA if cloning from a SourceDB (cloneFrom is set) + charset: AL32UTF8 + + ## NA if cloning from a SourceDB (cloneFrom is set) + pdbName: orclpdb1 + + ## Enable/Disable Flashback + flashBack: false + + ## Enable/Disable ArchiveLog + archiveLog: false + + ## Enable/Disable ForceLogging + forceLog: false + + ## NA if cloning from a SourceDB (cloneFrom is set) + ## Specify both sgaSize and pgaSize (in MB) or dont specify both + ## Specify Non-Zero value to use + initParams: + cpuCount: 0 + processes: 0 + sgaTarget: 0 + pgaAggregateTarget: 0 + + ## Database image details + ## Database can be patched by updating the RU version/image + ## Major version changes are not supported + image: + pullFrom: + pullSecrets: + + ## size : Minimum size of pvc | class : PVC storage Class + ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany + persistence: + size: 100Gi + storageClass: "" + accessMode: "ReadWriteMany" + + ## Type of service . Applicable on cloud enviroments only + ## if loadBalService : false, service type = "NodePort". else "LoadBalancer" + loadBalancer: false + + ## Deploy only on nodes having required labels. Format label_name : label_value + ## Leave empty if there is no such requirement. + ## Uncomment to use + # nodeSelector: + # failure-domain.beta.kubernetes.io/zone: bVCG:PHX-AD-1 + # pool: sidb + + ## Count of Database Pods. Applicable only for "ReadWriteMany" AccessMode + replicas: 1 diff --git a/config/samples/singleinstancedatabase_clone.yaml b/config/samples/singleinstancedatabase_clone.yaml new file mode 100644 index 00000000..c324c1c3 --- /dev/null +++ b/config/samples/singleinstancedatabase_clone.yaml @@ -0,0 +1,41 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: SingleInstanceDatabase +metadata: + name: singleinstancedatabase-sample + namespace: default +spec: + + ## Use only alphanumeric characters for sid + sid: ORCL1 + + ## A source database ref to clone from, leave empty to create a fresh database + cloneFrom: "" + + ## Should refer to SourceDB secret + ## Secret containing SIDB password mapped to secretKey + ## This secret will be deleted after creation of the database unless keepSecret is set to true + adminPassword: + secretName: + secretKey: + keepSecret: false + + ## Database image details + ## Database can be patched out of place by updating the RU version/image + ## Major version changes are not supported + image: + pullFrom: + pullSecrets: + + ## size : Minimum size of pvc | class : PVC storage Class + ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany + persistence: + size: 100Gi + storageClass: "" + accessMode: "ReadWriteMany" + + ## Count of Database Pods. Applicable only for "ReadWriteMany" AccessMode + replicas: 1 diff --git a/config/samples/singleinstancedatabase_patch.yaml b/config/samples/singleinstancedatabase_patch.yaml new file mode 100644 index 00000000..4d2d03b0 --- /dev/null +++ b/config/samples/singleinstancedatabase_patch.yaml @@ -0,0 +1,36 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: SingleInstanceDatabase +metadata: + name: singleinstancedatabase-sample + namespace: default +spec: + + ## Use only alphanumeric characters for sid + sid: ORCL1 + + ## Secret containing SIDB password mapped to secretKey + ## This secret will be deleted after creation of the database unless keepSecret is set to true + adminPassword: + secretName: + secretKey: + keepSecret: false + + ## Patch the database by updating the RU version/image + ## Major version changes are not supported + image: + pullFrom: + pullSecrets: + + ## size : Minimum size of pvc | class : PVC storage Class + ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany + persistence: + size: 100Gi + storageClass: "" + accessMode: "ReadWriteMany" + + ## Count of Database Pods. Applicable only for "ReadWriteMany" AccessMode + replicas: 1 diff --git a/config/samples/singleinstancedatabase_prov.yaml b/config/samples/singleinstancedatabase_prov.yaml new file mode 100644 index 00000000..7123b117 --- /dev/null +++ b/config/samples/singleinstancedatabase_prov.yaml @@ -0,0 +1,35 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: SingleInstanceDatabase +metadata: + name: singleinstancedatabase-sample + namespace: default +spec: + + ## Use only alphanumeric characters for sid + sid: ORCL1 + + ## Secret containing SIDB password mapped to secretKey + adminPassword: + secret: + secretName: + key: + + ## Database image details + image: + version: + pullFrom: + pullSecrets: + + ## size : Minimum size of pvc | class : PVC storage Class . + ## AccessMode can only accept one of ReadWriteOnce , ReadWriteMany + persistence: + size: 100Gi + storageClass: "" + accessMode: "ReadWriteMany" + + ## Count of Database Pods. Applicable only for "ReadWriteMany" AccessMode + replicas: 1 diff --git a/config/scorecard/bases/config.yaml b/config/scorecard/bases/config.yaml new file mode 100644 index 00000000..0650fef2 --- /dev/null +++ b/config/scorecard/bases/config.yaml @@ -0,0 +1,25 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +kind: Configuration +apiversion: scorecard.operatorframework.io/v1alpha3 +metadata: + name: config +stages: +- parallel: true + tests: + - image: quay.io/operator-framework/scorecard-test:latest + entrypoint: + - scorecard-test + - basic-check-spec + labels: + suite: basic + test: basic-check-spec-test + - image: quay.io/operator-framework/scorecard-test:latest + entrypoint: + - scorecard-test + - olm-bundle-validation + labels: + suite: olm + test: olm-bundle-validation-test diff --git a/config/scorecard/kustomization.yaml b/config/scorecard/kustomization.yaml new file mode 100644 index 00000000..d9c19e9b --- /dev/null +++ b/config/scorecard/kustomization.yaml @@ -0,0 +1,20 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +resources: +- bases/config.yaml +patchesJson6902: +- path: patches/basic.config.yaml + target: + group: scorecard.operatorframework.io + version: v1alpha3 + kind: Configuration + name: config +- path: patches/olm.config.yaml + target: + group: scorecard.operatorframework.io + version: v1alpha3 + kind: Configuration + name: config +# +kubebuilder:scaffold:patchesJson6902 diff --git a/config/scorecard/patches/basic.config.yaml b/config/scorecard/patches/basic.config.yaml new file mode 100644 index 00000000..67fa78c9 --- /dev/null +++ b/config/scorecard/patches/basic.config.yaml @@ -0,0 +1,14 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +- op: add + path: /stages/0/tests/- + value: + entrypoint: + - scorecard-test + - basic-check-spec + image: quay.io/operator-framework/scorecard-test:v1.2.0 + labels: + suite: basic + test: basic-check-spec-test diff --git a/config/scorecard/patches/olm.config.yaml b/config/scorecard/patches/olm.config.yaml new file mode 100644 index 00000000..521bc8e6 --- /dev/null +++ b/config/scorecard/patches/olm.config.yaml @@ -0,0 +1,54 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +- op: add + path: /stages/0/tests/- + value: + entrypoint: + - scorecard-test + - olm-bundle-validation + image: quay.io/operator-framework/scorecard-test:v1.2.0 + labels: + suite: olm + test: olm-bundle-validation-test +- op: add + path: /stages/0/tests/- + value: + entrypoint: + - scorecard-test + - olm-crds-have-validation + image: quay.io/operator-framework/scorecard-test:v1.2.0 + labels: + suite: olm + test: olm-crds-have-validation-test +- op: add + path: /stages/0/tests/- + value: + entrypoint: + - scorecard-test + - olm-crds-have-resources + image: quay.io/operator-framework/scorecard-test:v1.2.0 + labels: + suite: olm + test: olm-crds-have-resources-test +- op: add + path: /stages/0/tests/- + value: + entrypoint: + - scorecard-test + - olm-spec-descriptors + image: quay.io/operator-framework/scorecard-test:v1.2.0 + labels: + suite: olm + test: olm-spec-descriptors-test +- op: add + path: /stages/0/tests/- + value: + entrypoint: + - scorecard-test + - olm-status-descriptors + image: quay.io/operator-framework/scorecard-test:v1.2.0 + labels: + suite: olm + test: olm-status-descriptors-test diff --git a/config/webhook/kustomization.yaml b/config/webhook/kustomization.yaml new file mode 100644 index 00000000..ef3ca64c --- /dev/null +++ b/config/webhook/kustomization.yaml @@ -0,0 +1,10 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +resources: +- manifests.yaml +- service.yaml + +configurations: +- kustomizeconfig.yaml diff --git a/config/webhook/kustomizeconfig.yaml b/config/webhook/kustomizeconfig.yaml new file mode 100644 index 00000000..afc69aba --- /dev/null +++ b/config/webhook/kustomizeconfig.yaml @@ -0,0 +1,30 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +# the following config is for teaching kustomize where to look at when substituting vars. +# It requires kustomize v2.1.0 or newer to work properly. +nameReference: +- kind: Service + version: v1 + fieldSpecs: + - kind: MutatingWebhookConfiguration + group: admissionregistration.k8s.io + path: webhooks/clientConfig/service/name + - kind: ValidatingWebhookConfiguration + group: admissionregistration.k8s.io + path: webhooks/clientConfig/service/name + +namespace: +- kind: MutatingWebhookConfiguration + group: admissionregistration.k8s.io + path: webhooks/clientConfig/service/namespace + create: true +- kind: ValidatingWebhookConfiguration + group: admissionregistration.k8s.io + path: webhooks/clientConfig/service/namespace + create: true + +varReference: +- path: metadata/annotations diff --git a/config/webhook/manifests.yaml b/config/webhook/manifests.yaml new file mode 100644 index 00000000..5cb37b30 --- /dev/null +++ b/config/webhook/manifests.yaml @@ -0,0 +1,59 @@ + +--- +apiVersion: admissionregistration.k8s.io/v1 +kind: MutatingWebhookConfiguration +metadata: + creationTimestamp: null + name: mutating-webhook-configuration +webhooks: +- admissionReviewVersions: + - v1 + - v1beta1 + clientConfig: + service: + name: webhook-service + namespace: system + path: /mutate-database-oracle-com-v1alpha1-singleinstancedatabase + failurePolicy: Fail + name: msingleinstancedatabase.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + resources: + - singleinstancedatabases + sideEffects: None + +--- +apiVersion: admissionregistration.k8s.io/v1 +kind: ValidatingWebhookConfiguration +metadata: + creationTimestamp: null + name: validating-webhook-configuration +webhooks: +- admissionReviewVersions: + - v1 + - v1beta1 + clientConfig: + service: + name: webhook-service + namespace: system + path: /validate-database-oracle-com-v1alpha1-singleinstancedatabase + failurePolicy: Fail + name: vsingleinstancedatabase.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + - DELETE + resources: + - singleinstancedatabases + sideEffects: None diff --git a/config/webhook/service.yaml b/config/webhook/service.yaml new file mode 100644 index 00000000..08333c36 --- /dev/null +++ b/config/webhook/service.yaml @@ -0,0 +1,16 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: v1 +kind: Service +metadata: + name: webhook-service + namespace: system +spec: + ports: + - port: 443 + protocol: TCP + targetPort: 9443 + selector: + control-plane: controller-manager diff --git a/controllers/database/autonomousdatabase_controller.go b/controllers/database/autonomousdatabase_controller.go new file mode 100644 index 00000000..2f250489 --- /dev/null +++ b/controllers/database/autonomousdatabase_controller.go @@ -0,0 +1,549 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package controllers + +import ( + "context" + "fmt" + "reflect" + + "github.com/go-logr/logr" + "github.com/oracle/oci-go-sdk/v45/database" + "github.com/oracle/oci-go-sdk/v45/secrets" + "github.com/oracle/oci-go-sdk/v45/workrequests" + + apiErrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/types" + "k8s.io/client-go/util/retry" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/controller" + "sigs.k8s.io/controller-runtime/pkg/event" + "sigs.k8s.io/controller-runtime/pkg/predicate" + + dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + adbutil "github.com/oracle/oracle-database-operator/commons/autonomousdatabase" + "github.com/oracle/oracle-database-operator/commons/finalizer" + "github.com/oracle/oracle-database-operator/commons/oci" +) + +// AutonomousDatabaseReconciler reconciles a AutonomousDatabase object +type AutonomousDatabaseReconciler struct { + KubeClient client.Client + Log logr.Logger + Scheme *runtime.Scheme + + currentLogger logr.Logger +} + +// SetupWithManager function +func (r *AutonomousDatabaseReconciler) SetupWithManager(mgr ctrl.Manager) error { + return ctrl.NewControllerManagedBy(mgr). + For(&dbv1alpha1.AutonomousDatabase{}). + WithEventFilter(r.eventFilterPredicate()). + WithOptions(controller.Options{MaxConcurrentReconciles: 50}). // ReconcileHandler is never invoked concurrently with the same object. + Complete(r) +} + +func (r *AutonomousDatabaseReconciler) eventFilterPredicate() predicate.Predicate { + pred := predicate.Funcs{ + UpdateFunc: func(e event.UpdateEvent) bool { + oldADB := e.ObjectOld.DeepCopyObject().(*dbv1alpha1.AutonomousDatabase) + newADB := e.ObjectNew.DeepCopyObject().(*dbv1alpha1.AutonomousDatabase) + + // Reconciliation should NOT happen if the lastSuccessfulSpec annotation or status.state changes. + oldSucSpec := oldADB.GetAnnotations()[dbv1alpha1.LastSuccessfulSpec] + newSucSpec := newADB.GetAnnotations()[dbv1alpha1.LastSuccessfulSpec] + + lastSucSpecChanged := oldSucSpec != newSucSpec + stateChanged := oldADB.Status.LifecycleState != newADB.Status.LifecycleState + if lastSucSpecChanged || stateChanged { + // Don't enqueue request + return false + } + // Enqueue request + return true + }, + DeleteFunc: func(e event.DeleteEvent) bool { + // Do not trigger reconciliation when the real object is deleted from the cluster. + return false + }, + } + + return pred +} + +// +kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabases,verbs=get;list;watch;create;update;patch;delete +// +kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabases/status,verbs=update;patch +// +kubebuilder:rbac:groups=coordination.k8s.io,resources=leases,verbs=create;get;list;update +// +kubebuilder:rbac:groups="",resources=configmaps;secrets,verbs=get;list;watch;create;update;patch;delete + +// Reconcile is the funtion that the operator calls every time when the reconciliation loop is triggered. +// It go to the beggining of the reconcile if an error is returned. We won't return a error if it is related +// to OCI, because the issues cannot be solved by re-run the reconcile. +func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + r.currentLogger = r.Log.WithValues("Namespaced/Name", req.NamespacedName) + + // Get the autonomousdatabase instance from the cluster + adb := &dbv1alpha1.AutonomousDatabase{} + if err := r.KubeClient.Get(context.TODO(), req.NamespacedName, adb); err != nil { + // Ignore not-found errors, since they can't be fixed by an immediate requeue. + // No need to change the since we don't know if we obtain the object. + if !apiErrors.IsNotFound(err) { + return ctrl.Result{}, err + } + } + + /****************************************************************** + * Get OCI database client and work request client + ******************************************************************/ + authData := oci.APIKeyAuth{ + ConfigMapName: adb.Spec.OCIConfig.ConfigMapName, + SecretName: adb.Spec.OCIConfig.SecretName, + Namespace: adb.GetNamespace(), + } + provider, err := oci.GetOCIProvider(r.KubeClient, authData) + if err != nil { + r.currentLogger.Error(err, "Fail to get OCI provider") + + // Change the status to UNAVAILABLE + adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, nil + } + + dbClient, err := database.NewDatabaseClientWithConfigurationProvider(provider) + if err != nil { + r.currentLogger.Error(err, "Fail to get OCI database client") + + // Change the status to UNAVAILABLE + adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, nil + } + + secretClient, err := secrets.NewSecretsClientWithConfigurationProvider(provider) + if err != nil { + r.currentLogger.Error(err, "Fail to get OCI secret client") + + // Change the status to UNAVAILABLE + adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, nil + } + + workClient, err := workrequests.NewWorkRequestClientWithConfigurationProvider(provider) + if err != nil { + r.currentLogger.Error(err, "Fail to get OCI work request client") + + // Change the status to UNAVAILABLE + adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, nil + } + + r.currentLogger.Info("OCI provider configured succesfully") + + /****************************************************************** + * Register/unregister finalizer + * Deletion timestamp will be added to a object before it is deleted. + * Kubernetes server calls the clean up function if a finalizer exitsts, and won't delete the real object until + * all the finalizers are removed from the object metadata. + * Refer to this page for more details of using finalizers: https://kubernetes.io/blog/2021/05/14/using-finalizers-to-control-deletion/ + ******************************************************************/ + if adb.ObjectMeta.DeletionTimestamp.IsZero() { + // The object is not being deleted + if *adb.Spec.HardLink && !finalizer.HasFinalizer(adb) { + finalizer.Register(r.KubeClient, adb) + r.currentLogger.Info("Finalizer registered successfully.") + + } else if !*adb.Spec.HardLink && finalizer.HasFinalizer(adb) { + finalizer.Unregister(r.KubeClient, adb) + r.currentLogger.Info("Finalizer unregistered successfully.") + } + } else { + // The object is being deleted + if adb.Spec.Details.AutonomousDatabaseOCID == nil { + r.currentLogger.Info("Autonomous Database OCID is missing. Remove the resource only.") + } else if adb.Status.LifecycleState != database.AutonomousDatabaseLifecycleStateTerminating && + adb.Status.LifecycleState != database.AutonomousDatabaseLifecycleStateTerminated { + // Don't send terminate request if the database is terminating or already terminated + r.currentLogger.Info("Terminate Autonomous Database: " + *adb.Spec.Details.DbName) + if _, err := oci.DeleteAutonomousDatabase(dbClient, *adb.Spec.Details.AutonomousDatabaseOCID); err != nil { + r.currentLogger.Error(err, "Fail to terminate Autonomous Database") + + // Change the status to UNAVAILABLE + adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + return ctrl.Result{}, statusErr + } + } + } + + finalizer.Unregister(r.KubeClient, adb) + r.currentLogger.Info("Finalizer unregistered successfully.") + // Stop reconciliation as the item is being deleted + return ctrl.Result{}, nil + } + + /****************************************************************** + * Determine which Database operations need to be executed by checking the changes to spec.details. + * There are three scenario: + * 1. provision operation. The AutonomousDatabaseOCID is missing, and the LastSucSpec annotation is missing. + * 2. bind operation. The AutonomousDatabaseOCID is provided, but the LastSucSpec annotation is missing. + * 3. update operation. Every changes other than the above two cases goes here. + * Afterwards, update the resource from the remote database in OCI. This step will be executed right after + * the above three cases during every reconcile. + /******************************************************************/ + lastSucSpec, err := adb.GetLastSuccessfulSpec() + if err != nil { + return ctrl.Result{}, err + } + + if lastSucSpec == nil || !reflect.DeepEqual(lastSucSpec.Details, adb.Spec.Details) { + // spec.details changes + if adb.Spec.Details.AutonomousDatabaseOCID == nil && lastSucSpec == nil { + // If no AutonomousDatabaseOCID specified, create a database + // Update from yaml file might not have an AutonomousDatabaseOCID. Don't create a database if it already has last successful spec. + r.currentLogger.Info("AutonomousDatabase provisioning") + + resp, err := oci.CreateAutonomousDatabase(r.currentLogger, r.KubeClient, dbClient, secretClient, adb) + + if err != nil { + r.currentLogger.Error(err, "Fail to provision and get Autonomous Database OCID") + + // Change the status to UNAVAILABLE + adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + return ctrl.Result{}, statusErr + } + // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue + return ctrl.Result{}, nil + } + + adb.Spec.Details.AutonomousDatabaseOCID = resp.AutonomousDatabase.Id + + // Update status.state + adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + return ctrl.Result{}, statusErr + } + + if err := oci.WaitUntilWorkCompleted(r.currentLogger, workClient, resp.OpcWorkRequestId); err != nil { + r.currentLogger.Error(err, "Fail to watch the status of provision request. opcWorkRequestID = "+*resp.OpcWorkRequestId) + + // Change the status to UNAVAILABLE + adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + return ctrl.Result{}, statusErr + } + } + + r.currentLogger.Info("AutonomousDatabase " + *adb.Spec.Details.DbName + " provisioned succesfully") + + } else if adb.Spec.Details.AutonomousDatabaseOCID != nil && lastSucSpec == nil { + // Binding operation. We have the database ID but hasn't gotten complete infromation from OCI. + // The next step is to get AutonomousDatabse details from a remote instance. + + adb, err = oci.GetAutonomousDatabaseResource(r.currentLogger, dbClient, adb) + if err != nil { + r.currentLogger.Error(err, "Fail to get Autonomous Database") + + // Change the status to UNAVAILABLE + adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, nil + } + } else { + // The object has successfully synced with the remote database at least once. + // Update the Autonomous Database in OCI. + // Change to the lifecycle state has the highest priority. + + // Get the Autonomous Database OCID from the last successful spec if not presented. + // This happens when a database reference is already created in the cluster. User updates the target CR (specifying metadata.name) but doesn't provide database OCID. + if adb.Spec.Details.AutonomousDatabaseOCID == nil { + adb.Spec.Details.AutonomousDatabaseOCID = lastSucSpec.Details.AutonomousDatabaseOCID + } + + // Start/Stop/Terminate + setStateResp, err := oci.SetAutonomousDatabaseLifecycleState(r.currentLogger, dbClient, adb) + if err != nil { + r.currentLogger.Error(err, "Fail to set the Autonomous Database lifecycle state") + + // Change the status to UNAVAILABLE + adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, nil + } + + if setStateResp != nil { + var lifecycleState database.AutonomousDatabaseLifecycleStateEnum + var opcWorkRequestID *string + + if startResp, isStartResponse := setStateResp.(database.StartAutonomousDatabaseResponse); isStartResponse { + lifecycleState = startResp.AutonomousDatabase.LifecycleState + opcWorkRequestID = startResp.OpcWorkRequestId + + } else if stopResp, isStopResponse := setStateResp.(database.StopAutonomousDatabaseResponse); isStopResponse { + lifecycleState = stopResp.AutonomousDatabase.LifecycleState + opcWorkRequestID = stopResp.OpcWorkRequestId + + } else if deleteResp, isDeleteResponse := setStateResp.(database.DeleteAutonomousDatabaseResponse); isDeleteResponse { + // Special case. Delete response doen't contain lifecycle State + lifecycleState = database.AutonomousDatabaseLifecycleStateTerminating + opcWorkRequestID = deleteResp.OpcWorkRequestId + + } else { + r.currentLogger.Error(err, "Unknown response type") + + // Change the status to UNAVAILABLE + adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, nil + } + + // Update status.state + adb.Status.LifecycleState = lifecycleState + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + return ctrl.Result{}, statusErr + } + + if err := oci.WaitUntilWorkCompleted(r.currentLogger, workClient, opcWorkRequestID); err != nil { + r.currentLogger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*opcWorkRequestID) + + // Change the status to UNAVAILABLE + adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + return ctrl.Result{}, statusErr + } + } + r.currentLogger.Info(fmt.Sprintf("Set AutonomousDatabase %s lifecycle state to %s successfully\n", + *adb.Spec.Details.DbName, + adb.Spec.Details.LifecycleState)) + } + + // Update the database in OCI from the local resource. + // The local resource will be synchronized again later. + updateGenPassResp, err := oci.UpdateGeneralAndPasswordAttributes(r.currentLogger, r.KubeClient, dbClient, secretClient, adb) + if err != nil { + r.currentLogger.Error(err, "Fail to update Autonomous Database") + + // Change the status to UNAVAILABLE + adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + return ctrl.Result{}, statusErr + } + // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue + return ctrl.Result{}, nil + } + + if updateGenPassResp.OpcWorkRequestId != nil { + // Update status.state + adb.Status.LifecycleState = updateGenPassResp.AutonomousDatabase.LifecycleState + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + return ctrl.Result{}, statusErr + } + + if err := oci.WaitUntilWorkCompleted(r.currentLogger, workClient, updateGenPassResp.OpcWorkRequestId); err != nil { + r.currentLogger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*updateGenPassResp.OpcWorkRequestId) + + // Change the status to UNAVAILABLE + adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + return ctrl.Result{}, statusErr + } + } + r.currentLogger.Info("Update AutonomousDatabase " + *adb.Spec.Details.DbName + " succesfully") + } + + scaleResp, err := oci.UpdateScaleAttributes(r.currentLogger, r.KubeClient, dbClient, adb) + if err != nil { + r.currentLogger.Error(err, "Fail to update Autonomous Database") + + // Change the status to UNAVAILABLE + adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + return ctrl.Result{}, statusErr + } + // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue + return ctrl.Result{}, nil + } + + if scaleResp.OpcWorkRequestId != nil { + // Update status.state + adb.Status.LifecycleState = scaleResp.AutonomousDatabase.LifecycleState + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + return ctrl.Result{}, statusErr + } + + if err := oci.WaitUntilWorkCompleted(r.currentLogger, workClient, scaleResp.OpcWorkRequestId); err != nil { + r.currentLogger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*updateGenPassResp.OpcWorkRequestId) + + // Change the status to UNAVAILABLE + adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + return ctrl.Result{}, statusErr + } + } + r.currentLogger.Info("Scale AutonomousDatabase " + *adb.Spec.Details.DbName + " succesfully") + } + } + } + + // Get the information from OCI + updatedADB, err := oci.GetAutonomousDatabaseResource(r.currentLogger, dbClient, adb) + if err != nil { + r.currentLogger.Error(err, "Fail to get Autonomous Database") + + // Change the status to UNAVAILABLE + adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, nil + } + + adb = updatedADB + + // Update local object and the status + if err := r.updateAutonomousDatabaseDetails(adb); err != nil { + // Change the status to UNAVAILABLE + adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, err + } + + /***************************************************** + * Instance Wallet + *****************************************************/ + passwordSecretUpdate := (lastSucSpec == nil && adb.Spec.Details.Wallet.Password.K8sSecretName != nil) || + (lastSucSpec != nil && lastSucSpec.Details.Wallet.Password.K8sSecretName != adb.Spec.Details.Wallet.Password.K8sSecretName) + passwordOCIDUpdate := (lastSucSpec == nil && adb.Spec.Details.Wallet.Password.OCISecretOCID != nil) || + (lastSucSpec != nil && lastSucSpec.Details.Wallet.Password.OCISecretOCID != adb.Spec.Details.Wallet.Password.OCISecretOCID) + + if (passwordSecretUpdate || passwordOCIDUpdate) && adb.Status.LifecycleState == database.AutonomousDatabaseLifecycleStateAvailable { + if err := adbutil.CreateWalletSecret(r.currentLogger, r.KubeClient, dbClient, secretClient, adb); err != nil { + r.currentLogger.Error(err, "Fail to download Instance Wallet") + // Change the status to UNAVAILABLE + adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, nil + } + } + + /***************************************************** + * Update last succesful spec + *****************************************************/ + if err := adb.UpdateLastSuccessfulSpec(r.KubeClient); err != nil { + return ctrl.Result{}, err + } + + r.currentLogger.Info("AutonomousDatabase resoursce reconcile successfully") + + return ctrl.Result{}, nil +} + +// type patchValue struct { +// Op string `json:"op"` +// Path string `json:"path"` +// Value interface{} `json:"value"` +// } + +func (r *AutonomousDatabaseReconciler) updateAutonomousDatabaseDetails(adb *dbv1alpha1.AutonomousDatabase) error { + // Patch the AutonomousDatabase Kubernetes resource to avoid resource version conflicts. + // payload := []patchValue{{ + // Op: "replace", + // Path: "/spec/details", + // Value: adb.Spec.Details, + // }} + // payloadBytes, err := json.Marshal(payload) + // if err != nil { + // return err + // } + + // patch := client.RawPatch(types.JSONPatchType, payloadBytes) + // if err := r.KubeClient.Patch(context.TODO(), adb, patch); err != nil { + // return err + // } + + if err := retry.RetryOnConflict(retry.DefaultRetry, func() error { + curADB := &dbv1alpha1.AutonomousDatabase{} + + namespacedName := types.NamespacedName{ + Namespace: adb.GetNamespace(), + Name: adb.GetName(), + } + + if err := r.KubeClient.Get(context.TODO(), namespacedName, curADB); err != nil { + return err + } + + curADB.Spec.Details = adb.Spec.Details + return r.KubeClient.Update(context.TODO(), curADB) + }); err != nil { + return err + } + + // Update status + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + return statusErr + } + r.currentLogger.Info("Update local resource AutonomousDatabase successfully") + + return nil +} diff --git a/controllers/database/shardingdatabase_controller.go b/controllers/database/shardingdatabase_controller.go new file mode 100644 index 00000000..73575df7 --- /dev/null +++ b/controllers/database/shardingdatabase_controller.go @@ -0,0 +1,1782 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package controllers + +import ( + "context" + "encoding/json" + "fmt" + "reflect" + "strconv" + "time" + + "github.com/go-logr/logr" + "github.com/oracle/oci-go-sdk/v45/common" + "github.com/oracle/oci-go-sdk/v45/ons" + appsv1 "k8s.io/api/apps/v1" + corev1 "k8s.io/api/core/v1" + "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/api/meta" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/types" + "k8s.io/client-go/kubernetes" + "k8s.io/client-go/tools/clientcmd" + "k8s.io/client-go/tools/record" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/controller" + "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" + "sigs.k8s.io/controller-runtime/pkg/event" + "sigs.k8s.io/controller-runtime/pkg/predicate" + + databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + shardingv1 "github.com/oracle/oracle-database-operator/commons/sharding" +) + +//Sharding Topology +type ShardingTopology struct { + topicid string + Instance *databasev1alpha1.ShardingDatabase + deltopology bool + onsProvider common.ConfigurationProvider + onsProviderFlag bool + rclient ons.NotificationDataPlaneClient +} + +// ShardingDatabaseReconciler reconciles a ShardingDatabase object +type ShardingDatabaseReconciler struct { + client.Client + Log logr.Logger + Scheme *runtime.Scheme + kubeClient kubernetes.Interface + kubeConfig clientcmd.ClientConfig + Recorder record.EventRecorder + osh []*ShardingTopology +} + +// +kubebuilder:rbac:groups=database.oracle.com,resources=shardingdatabases,verbs=get;list;watch;create;update;patch;delete +// +kubebuilder:rbac:groups=database.oracle.com,resources=shardingdatabases/status,verbs=get;update;patch +// +kubebuilder:rbac:groups=database.oracle.com,resources=shardingdatabases/finalizers,verbs=get;create;update;patch;delete +// +kubebuilder:rbac:groups=core,resources=pods;pods/log;pods/exec;secrets;services;events;nodes;configmaps;persistentvolumeclaims;namespaces,verbs=get;list;watch;create;update;patch;delete +// +kubebuilder:rbac:groups=core,resources=pods/exec,verbs=create +// +kubebuilder:rbac:groups=apps,resources=statefulsets,verbs=get;list;watch;create;update;patch;delete +// +kubebuilder:rbac:groups='',resources=statefulsets/finalizers,verbs=get;list;watch;create;update;patch;delete + +// Reconcile is part of the main kubernetes reconciliation loop which aims to +// move the current state of the cluster closer to the desired state. +// TODO(user): Modify the Reconcile function to compare the state specified by +// the ShardingDatabase object against the actual cluster state, and then +// perform operations to make the cluster state reflect the state specified by +// the user. +// +// For more details, check Reconcile and its Result here: +// - https://pkg.go.dev/sigs.k8s.io/controller-runtime@v0.6.4/pkg/reconcile +func (r *ShardingDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + //ctx := context.Background() + //_ = r.Log.WithValues("shardingdatabase", req.NamespacedName) + + // your logic here + var i int32 + //var ShardImageLatest []databasev1alpha1.ShardSpec + var OraCatalogSpex databasev1alpha1.CatalogSpec + var OraShardSpex databasev1alpha1.ShardSpec + var OraGsmSpex databasev1alpha1.GsmSpec + var result ctrl.Result + var isShardTopologyDeleteTrue bool = false + //var msg string + var err error + var stateType string + resultNq := ctrl.Result{Requeue: false} + resultQ := ctrl.Result{Requeue: true, RequeueAfter: 30 * time.Second} + var nilErr error = nil + + // On every reconcile, we will call setCrdLifeCycleState + // To understand this, please refer https://sdk.operatorframework.io/docs/building-operators/golang/advanced-topics/ + // https://github.com/kubernetes/apimachinery/blob/master/pkg/api/meta/conditions.go + + // Kube Client Config Setup + if r.kubeConfig == nil && r.kubeClient == nil { + r.kubeConfig, r.kubeClient, err = shardingv1.GetK8sClientConfig(r.Client) + if err != nil { + return ctrl.Result{}, err + } + } + // Fetch the ProvShard instance + instance := &databasev1alpha1.ShardingDatabase{} + err = r.Client.Get(context.TODO(), req.NamespacedName, instance) + if err != nil { + if errors.IsNotFound(err) { + // Request object not found, could have been deleted after reconcile req. + // Owned objects are automatically garbage collected. For additional cleanup logic use finalizers. + // Return and don't requeue + return ctrl.Result{}, nil + } + // Error reading the object - requeue the req. + return ctrl.Result{}, err + } + + idx, instFlag := r.checkProvInstance(instance) + // assinging osh instance + if !instFlag { + // Sharding Topolgy Struct Assignment + // ====================================== + osh := &ShardingTopology{} + osh.Instance = instance + r.osh = append(r.osh, osh) + } + defer r.setCrdLifeCycleState(instance, &result, &err, &stateType) + // =============================== Check Deletion TimeStamp======== + // Check if the ProvOShard instance is marked to be deleted, which is + // // indicated by the deletion timestamp being set. + + err, isShardTopologyDeleteTrue = r.finalizerShardingDatabaseInstance(instance) + if err != nil { + //r.setCrdLifeCycleState(instance, &result, &err, stateType) + result = resultNq + if isShardTopologyDeleteTrue == true { + err = nilErr + return result, err + } else { + return result, err + } + } + + // ======== Setting the flag and Index to be used later in this function ======== + idx, instFlag = r.checkProvInstance(instance) + if !instFlag { + //r.setCrdLifeCycleState(instance, &result, &err, stateType) + result = resultNq + return result, fmt.Errorf("DId not fid the instance in checkProvInstance") + } + + // ================================ OCI Notification Provider =========== + r.getOnsConfigProvider(instance, idx) + + // =============================== Checking Namespace ============== + if instance.Spec.Namespace != "" { + err = shardingv1.AddNamespace(instance, r.Client, r.Log) + if err != nil { + //r.setCrdLifeCycleState(instance, &result, &err, stateType) + result = resultNq + return result, err + } + } else { + instance.Spec.Namespace = "default" + } + + // ======================== Validate Specs ============== + err = r.validateSpex(instance, idx) + if err != nil { + //r.setCrdLifeCycleState(instance, &result, &err, stateType) + result = resultNq + return result, err + } + + // ========================= Service Setup For Catalog=================== + // Following check and loop will make sure to create the service + for i = 0; i < int32(len(instance.Spec.Catalog)); i++ { + OraCatalogSpex = instance.Spec.Catalog[i] + result, err = r.createService(instance, shardingv1.BuildServiceDefForCatalog(instance, 0, OraCatalogSpex, "local")) + if err != nil { + result = resultNq + return result, err + } + if instance.Spec.IsExternalSvc { + result, err = r.createService(instance, shardingv1.BuildServiceDefForCatalog(instance, 0, OraCatalogSpex, "external")) + if err != nil { + result = resultNq + return result, err + } + } + } + + // ================================ Catalog Setup =================== + if len(instance.Spec.Catalog) > 0 { + for i = 0; i < int32(len(instance.Spec.Catalog)); i++ { + OraCatalogSpex = instance.Spec.Catalog[i] + // See if StatefulSets already exists and create if it doesn't + result, err = r.deployStatefulSet(instance, shardingv1.BuildStatefulSetForCatalog(instance, OraCatalogSpex), "CATALOG") + if err != nil { + result = resultNq + return result, err + } + } + } + + // ========================= Service Setup For Gsm=================== + // Following check and loop will make sure if we need service per replica pod or on a single pod + // if user set replicasize greater than 1 but also set instance.Spec.OraDbPvcName then only one service will be created and one pod + for i = 0; i < int32(len(instance.Spec.Gsm)); i++ { + OraGsmSpex = instance.Spec.Gsm[i] + result, err = r.createService(instance, shardingv1.BuildServiceDefForGsm(instance, 0, OraGsmSpex, "local")) + if err != nil { + result = resultNq + return result, err + } + if instance.Spec.IsExternalSvc { + result, err = r.createService(instance, shardingv1.BuildServiceDefForGsm(instance, 0, OraGsmSpex, "external")) + if err != nil { + result = resultNq + return result, err + } + } + } + + // ========================= Service Setup For Gsm=================== + + // Following check and loop will make sure if we need service per replica pod or on a single pod + // if user set replicasize greater than 1 but also set instance.Spec.OraDbPvcName then only one service will be created and one pod + // ================================ Gsm Setup =================== + if len(instance.Spec.Gsm) > 0 { + // for _, OraGsmSpex := range instance.Spec.Gsm + for i = 0; i < int32(len(instance.Spec.Gsm)); i++ { + OraGsmSpex = instance.Spec.Gsm[i] + result, err = r.deployStatefulSet(instance, shardingv1.BuildStatefulSetForGsm(instance, OraGsmSpex), "GSM") + if err != nil { + result = resultNq + return result, err + } + } + } + + // ========================= Service Setup For Shard=================== + + // Following check and loop will make sure if we need service per replica pod or on a single pod + // if user set replicasize greater than 1 but also set instance.Spec.OraDbPvcName then only one service will be created and one pod + for i = 0; i < int32(len(instance.Spec.Shard)); i++ { + OraShardSpex = instance.Spec.Shard[i] + if OraShardSpex.IsDelete != true { + result, err = r.createService(instance, shardingv1.BuildServiceDefForShard(instance, 0, OraShardSpex, "local")) + if err != nil { + result = resultNq + return result, err + } + if instance.Spec.IsExternalSvc { + result, err = r.createService(instance, shardingv1.BuildServiceDefForShard(instance, 0, OraShardSpex, "external")) + if err != nil { + result = resultNq + return result, err + } + } + } + } + + // ================================ Shard Setup =================== + if len(instance.Spec.Shard) > 0 { + for i = 0; i < int32(len(instance.Spec.Shard)); i++ { + OraShardSpex = instance.Spec.Shard[i] + if OraShardSpex.IsDelete != true { + result, err = r.deployStatefulSet(instance, shardingv1.BuildStatefulSetForShard(instance, OraShardSpex), "SHARD") + if err != nil { + result = resultNq + return result, err + } + } + } + } + //================ Validate the GSM and Catalog before procedding for Shard Setup ============== + // If the GSM and Catalog is not configured then Requeue the loop unless it returns nil + // Until GSM and Catalog is configured, the topology state remain provisioning + err = r.validateGsmnCatalog(instance) + if err != nil { + // r.setCrdLifeCycleState(instance, &result, &err, stateType) + // time.Sleep(30 * time.Second) + err = nilErr + result = resultQ + return result, err + } + + //set the Waiting state for Reconcile loop + // Loop will be requeued only if Shard Statefulset is not ready or not configured. + // Till that time Reconcilation loop will remain in blocked state + // if the err is return because of Shard is not ready then blocked state is rmeoved and reconcilation state is set + err = r.addPrimaryShards(instance, idx) + if err != nil { + // time.Sleep(30 * time.Second) + err = nilErr + result = resultQ + return result, err + } + + // Loop will be requeued only if Standby Shard Statefulset is not ready or not configured. + // Till that time Reconcilation loop will remain in blocked state + // if the err is return because of Shard is not ready then blocked state is rmeoved and reconcilation state is + err = r.addStandbyShards(instance, idx) + if err != nil { + // time.Sleep(30 * time.Second) + err = nilErr + result = resultQ + return result, err + } + + // we don't need to run the requeue loop but still putting this condition to address any unkown situation + // delShard function set the state to blocked and we do not allow any other operationn while delete is going on + err = r.delGsmShard(instance, idx) + if err != nil { + // time.Sleep(30 * time.Second) + err = nilErr + result = resultQ + return result, err + } + + // ====================== Update Setup for Catalog ============================== + for i = 0; i < int32(len(instance.Spec.Catalog)); i++ { + OraCatalogSpex = instance.Spec.Catalog[i] + sfSet, catalogPod, err := r.validateInvidualCatalog(instance, OraCatalogSpex, int(i)) + if err != nil { + shardingv1.LogMessages("INFO", "Catalog "+sfSet.Name+" is not in available state.", nil, instance, r.Log) + result = resultNq + return result, err + } + result, err = shardingv1.UpdateProvForCatalog(instance, OraCatalogSpex, r.Client, sfSet, catalogPod, r.Log) + if err != nil { + shardingv1.LogMessages("INFO", "Error Occurred during catalog update operation.", nil, instance, r.Log) + result = resultNq + return result, err + } + } + + // ====================== Update Setup for Shard ============================== + for i = 0; i < int32(len(instance.Spec.Shard)); i++ { + OraShardSpex = instance.Spec.Shard[i] + if OraShardSpex.IsDelete != true { + sfSet, shardPod, err := r.validateShard(instance, OraShardSpex, int(i)) + if err != nil { + shardingv1.LogMessages("INFO", "Shard "+sfSet.Name+" is not in available state.", nil, instance, r.Log) + result = resultNq + return result, err + } + result, err = shardingv1.UpdateProvForShard(instance, OraShardSpex, r.Client, sfSet, shardPod, r.Log) + if err != nil { + shardingv1.LogMessages("INFO", "Error Occurred during shard update operation..", nil, instance, r.Log) + result = resultNq + return result, err + } + } + } + + // ====================== Update Setup for Gsm ============================== + for i = 0; i < int32(len(instance.Spec.Gsm)); i++ { + OraGsmSpex = instance.Spec.Gsm[i] + sfSet, gsmPod, err := r.validateInvidualGsm(instance, OraGsmSpex, int(i)) + if err != nil { + shardingv1.LogMessages("INFO", "Gsm "+sfSet.Name+" is not in available state.", nil, instance, r.Log) + result = resultNq + return result, err + } + result, err = shardingv1.UpdateProvForGsm(instance, OraGsmSpex, r.Client, sfSet, gsmPod, r.Log) + if err != nil { + shardingv1.LogMessages("INFO", "Error Occurred during GSM update operation.", nil, instance, r.Log) + result = resultNq + return result, err + } + } + + // Calling updateShardTopology to update the entire sharding topology + // This is required because we just executed updateShard,updateCatalog and UpdateGsm + // If some state has changed it will update the topology + + err = r.updateShardTopologyStatus(instance) + if err != nil { + // time.Sleep(30 * time.Second) + result = resultQ + err = nilErr + return result, err + } + + stateType = string(databasev1alpha1.CrdReconcileCompeleteState) + // r.setCrdLifeCycleState(instance, &result, &err, stateType) + // Set error to ni to avoid reconcilation state reconcilation error as we are passing err to setCrdLifeCycleState + + shardingv1.LogMessages("INFO", "Completed the Sharding topology setup reconcilation loop.", nil, instance, r.Log) + result = resultNq + err = nilErr + return result, err +} + +// SetupWithManager sets up the controller with the Manager. +// The default concurrent reconcilation loop is 1 +// Check https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/controller#Options to under MaxConcurrentReconciles +func (r *ShardingDatabaseReconciler) SetupWithManager(mgr ctrl.Manager) error { + return ctrl.NewControllerManagedBy(mgr). + For(&databasev1alpha1.ShardingDatabase{}). + Owns(&appsv1.StatefulSet{}). + Owns(&corev1.Service{}). + Owns(&corev1.Pod{}). + WithEventFilter(r.eventFilterPredicate()). + WithOptions(controller.Options{MaxConcurrentReconciles: 50}). //MaxConcurrentReconciles is the maximum number of concurrent Reconciles which can be run. Defaults to 1 + Complete(r) +} + +// ###################### Event Filter Predicate ###################### +func (r *ShardingDatabaseReconciler) eventFilterPredicate() predicate.Predicate { + return predicate.Funcs{ + + CreateFunc: func(e event.CreateEvent) bool { + return true + }, + UpdateFunc: func(e event.UpdateEvent) bool { + return true + }, + DeleteFunc: func(e event.DeleteEvent) bool { + _, podOk := e.Object.GetLabels()["statefulset.kubernetes.io/pod-name"] + for i := 0; i < len(r.osh); i++ { + if r.osh[i] != nil { + oshInst := r.osh[i] + if oshInst.deltopology == true { + break + + } + if e.Object.GetLabels()[string(databasev1alpha1.ShardingDelLabelKey)] == string(databasev1alpha1.ShardingDelLabelTrueValue) { + break + } + + if podOk { + delObj := e.Object.(*corev1.Pod) + if e.Object.GetLabels()["type"] == "Shard" && e.Object.GetLabels()["app"] == "OracleSharding" && e.Object.GetLabels()["oralabel"] == oshInst.Instance.Name { + + if delObj.DeletionTimestamp != nil { + go r.gsmInvitedNodeOp(oshInst.Instance, delObj.Name) + } + } + + if e.Object.GetLabels()["type"] == "Catalog" && e.Object.GetLabels()["app"] == "OracleSharding" && e.Object.GetLabels()["oralabel"] == oshInst.Instance.Name { + + if delObj.DeletionTimestamp != nil { + go r.gsmInvitedNodeOp(oshInst.Instance, delObj.Name) + } + } + + } + + } + } + + return true + }, + } +} + +// ================== Function to get the Notification controller ============== +func (r *ShardingDatabaseReconciler) getOnsConfigProvider(instance *databasev1alpha1.ShardingDatabase, idx int, +) { + var err error + if instance.Spec.NsConfigMap != "" && instance.Spec.NsSecret != "" && r.osh[idx].onsProviderFlag != true { + cmName := instance.Spec.NsConfigMap + secName := instance.Spec.NsSecret + shardingv1.LogMessages("DEBUG", "Received parameters are "+shardingv1.GetFmtStr(cmName)+","+shardingv1.GetFmtStr(secName), nil, instance, r.Log) + region, user, tenancy, passphrase, fingerprint, topicid := shardingv1.ReadConfigMap(cmName, instance, r.Client, r.Log) + privatekey := shardingv1.ReadSecret(secName, instance, r.Client, r.Log) + r.osh[idx].topicid = topicid + r.osh[idx].onsProvider = common.NewRawConfigurationProvider(tenancy, user, region, fingerprint, privatekey, &passphrase) + r.osh[idx].rclient, err = ons.NewNotificationDataPlaneClientWithConfigurationProvider(r.osh[idx].onsProvider) + if err != nil { + msg := "Error occurred in getting the OCI notification service based client." + r.osh[idx].onsProviderFlag = false + r.Log.Error(err, msg) + shardingv1.LogMessages("Error", msg, nil, instance, r.Log) + } else { + r.osh[idx].onsProviderFlag = true + } + + } +} + +// ================== Function the Message ============== +func (r *ShardingDatabaseReconciler) sendMessage(instance *databasev1alpha1.ShardingDatabase, title string, body string) { + idx, instFlag := r.checkProvInstance(instance) + if instFlag { + if r.osh[idx].onsProviderFlag { + shardingv1.SendNotification(title, body, instance, r.osh[idx].topicid, r.osh[idx].rclient, r.Log) + } + } +} + +func (r *ShardingDatabaseReconciler) publishEvents(instance *databasev1alpha1.ShardingDatabase, eventMsg string, state string) { + + if state == string(databasev1alpha1.AvailableState) || state == string(databasev1alpha1.AddingShardState) || state == string(databasev1alpha1.ShardOnlineState) || state == string(databasev1alpha1.ProvisionState) || state == string(databasev1alpha1.DeletingState) || state == string(databasev1alpha1.Terminated) { + r.Recorder.Eventf(instance, corev1.EventTypeNormal, "State Change", eventMsg) + } else { + r.Recorder.Eventf(instance, corev1.EventTypeWarning, "State Change", eventMsg) + + } + +} + +// ================== Function to check insytance deletion timestamp and activate the finalizer code ======== +func (r *ShardingDatabaseReconciler) finalizerShardingDatabaseInstance(instance *databasev1alpha1.ShardingDatabase, +) (error, bool) { + + isProvOShardToBeDeleted := instance.GetDeletionTimestamp() != nil + if isProvOShardToBeDeleted { + if controllerutil.ContainsFinalizer(instance, shardingv1.ShardingDatabaseFinalizer) { + // Run finalization logic for finalizer. If the + // finalization logic fails, don't remove the finalizer so + // that we can retry during the next reconciliation. + if err := r.finalizeShardingDatabase(instance); err != nil { + return err, false + } + + // Remove finalizer. Once all finalizers have been + // removed, the object will be deleted. + controllerutil.RemoveFinalizer(instance, shardingv1.ShardingDatabaseFinalizer) + err := r.Client.Update(context.TODO(), instance) + if err != nil { + return err, false + } + } + // Send true because delete is in progress and it is a custom delete message + // We don't need to print custom err stack as we are deleting the topology + return fmt.Errorf("Delete of the sharding topology is in progress"), true + } + + // Add finalizer for this CR + if instance.DeletionTimestamp == nil { + if !controllerutil.ContainsFinalizer(instance, shardingv1.ShardingDatabaseFinalizer) { + if err := r.addFinalizer(instance); err != nil { + return err, false + } + } + } + + return nil, false +} + +// ========================== FInalizer Section =================== +func (r *ShardingDatabaseReconciler) addFinalizer(instance *databasev1alpha1.ShardingDatabase) error { + reqLogger := r.Log.WithValues("instance.Spec.Namespace", instance.Spec.Namespace, "instance.Name", instance.Name) + controllerutil.AddFinalizer(instance, shardingv1.ShardingDatabaseFinalizer) + + // Update CR + err := r.Client.Update(context.TODO(), instance) + if err != nil { + reqLogger.Error(err, "Failed to update Sharding Database with finalizer") + return err + } + return nil +} + +func (r *ShardingDatabaseReconciler) finalizeShardingDatabase(instance *databasev1alpha1.ShardingDatabase) error { + // TODO(user): Add the cleanup steps that the operator needs to do before the CR + // can be deleted. Examples of finalizers include performing backups and deleting + // resources that are not owned by this CR, like a PVC. + + var i int32 + var err error + var pvcName string + + idx, _ := r.checkProvInstance(instance) + sfSetFound := &appsv1.StatefulSet{} + svcFound := &corev1.Service{} + r.osh[idx].deltopology = true + if len(instance.Spec.Shard) > 0 { + for i = 0; i < int32(len(instance.Spec.Shard)); i++ { + OraShardSpex := instance.Spec.Shard[i] + sfSetFound, err = shardingv1.CheckSfset(OraShardSpex.Name, instance, r.Client) + if err == nil { + // See if StatefulSets already exists and create if it doesn't + err = r.Client.Delete(context.Background(), sfSetFound) + if err != nil { + return err + } + if instance.Spec.IsDeleteOraPvc && len(instance.Spec.StorageClass) > 0 { + pvcName = OraShardSpex.Name + "-oradata-vol4-" + OraShardSpex.Name + "-0" + err = shardingv1.DelPvc(pvcName, instance, r.Client, r.Log) + if err != nil { + return err + } + } + } + } + } + + if len(instance.Spec.Gsm) > 0 { + for i = 0; i < int32(len(instance.Spec.Gsm)); i++ { + OraGsmSpex := instance.Spec.Gsm[i] + sfSetFound, err = shardingv1.CheckSfset(OraGsmSpex.Name, instance, r.Client) + if err == nil { + // See if StatefulSets already exists and create if it doesn't + err = r.Client.Delete(context.Background(), sfSetFound) + if err != nil { + return err + } + if instance.Spec.IsDeleteOraPvc && len(instance.Spec.StorageClass) > 0 { + pvcName = OraGsmSpex.Name + "-oradata-vol4-" + OraGsmSpex.Name + "-0" + err = shardingv1.DelPvc(pvcName, instance, r.Client, r.Log) + if err != nil { + return err + } + } + } + } + } + + if len(instance.Spec.Catalog) > 0 { + for i = 0; i < int32(len(instance.Spec.Catalog)); i++ { + OraCatalogSpex := instance.Spec.Catalog[i] + // See if StatefulSets already exists and create if it doesn't + sfSetFound, err = shardingv1.CheckSfset(OraCatalogSpex.Name, instance, r.Client) + if err == nil { + // See if StatefulSets already exists and create if it doesn't + err = r.Client.Delete(context.Background(), sfSetFound) + if err != nil { + return err + } + if instance.Spec.IsDeleteOraPvc && len(instance.Spec.StorageClass) > 0 { + pvcName = OraCatalogSpex.Name + "-oradata-vol4-" + OraCatalogSpex.Name + "-0" + err = shardingv1.DelPvc(pvcName, instance, r.Client, r.Log) + if err != nil { + return err + } + } + } + } + } + + if len(instance.Spec.Shard) > 0 { + for i = 0; i < int32(len(instance.Spec.Shard)); i++ { + if instance.Spec.IsExternalSvc { + svcFound, err = shardingv1.CheckSvc(instance.Spec.Shard[i].Name+strconv.FormatInt(int64(0), 10)+"-svc", instance, r.Client) + if err == nil { + // See if StatefulSets already exists and create if it doesn't + err = r.Client.Delete(context.Background(), svcFound) + if err != nil { + return err + } + } + } + svcFound, err = shardingv1.CheckSvc(instance.Spec.Shard[i].Name, instance, r.Client) + if err == nil { + // See if StatefulSets already exists and create if it doesn't + err = r.Client.Delete(context.Background(), svcFound) + if err != nil { + return err + } + } + } + } + + if len(instance.Spec.Catalog) > 0 { + for i = 0; i < int32(len(instance.Spec.Catalog)); i++ { + if instance.Spec.IsExternalSvc { + svcFound, err = shardingv1.CheckSvc(instance.Spec.Catalog[i].Name+strconv.FormatInt(int64(0), 10)+"-svc", instance, r.Client) + if err == nil { + // See if StatefulSets already exists and create if it doesn't + err = r.Client.Delete(context.Background(), svcFound) + if err != nil { + return err + } + } + } + svcFound, err = shardingv1.CheckSvc(instance.Spec.Catalog[i].Name, instance, r.Client) + if err == nil { + // See if StatefulSets already exists and create if it doesn't + err = r.Client.Delete(context.Background(), svcFound) + if err != nil { + return err + } + } + } + } + + if len(instance.Spec.Gsm) > 0 { + for i = 0; i < int32(len(instance.Spec.Gsm)); i++ { + // See if StatefulSets already exists and create if it doesn't + if len(instance.Spec.Gsm[i].PvcName) == 0 { + if instance.Spec.IsExternalSvc { + svcFound, err = shardingv1.CheckSvc(instance.Spec.Gsm[i].Name+strconv.FormatInt(int64(i), 10)+"-svc", instance, r.Client) + if err == nil { + // See if StatefulSets already exists and delete if it doesn't + err = r.Client.Delete(context.Background(), svcFound) + if err != nil { + return err + } + } + } + svcFound, err = shardingv1.CheckSvc(instance.Spec.Gsm[i].Name, instance, r.Client) + if err == nil { + // See if StatefulSets already exists and delete if it doesn't + err = r.Client.Delete(context.Background(), svcFound) + if err != nil { + return err + } + } + if instance.Spec.IsExternalSvc { + svcFound, err = shardingv1.CheckSvc(instance.Spec.Gsm[i].Name+strconv.FormatInt(int64(0), 10)+"-svc", instance, r.Client) + if err == nil { + // See if StatefulSets already exists and create if it doesn't + err = r.Client.Delete(context.Background(), svcFound) + if err != nil { + return err + } + } + } + } else { + if instance.Spec.IsExternalSvc { + svcFound, err = shardingv1.CheckSvc(instance.Spec.Gsm[i].Name+strconv.FormatInt(int64(0), 10)+"-svc", instance, r.Client) + if err == nil { + // See if StatefulSets already exists and create if it doesn't + err = r.Client.Delete(context.Background(), svcFound) + if err != nil { + return err + } + } + } + svcFound, err = shardingv1.CheckSvc(instance.Spec.Gsm[i].Name, instance, r.Client) + if err == nil { + // See if StatefulSets already exists and create if it doesn't + err = r.Client.Delete(context.Background(), svcFound) + if err != nil { + return err + } + } + } + } + } + + // List the stateful for this instance's statefulset and delete the all the stateful set which belong to this instance as a left over + sfList := &appsv1.StatefulSetList{} + listOpts := []client.ListOption{ + client.InNamespace(instance.Namespace), + client.MatchingLabels(shardingv1.LabelsForProvShardKind(instance, "shard")), + } + + err = r.Client.List(context.TODO(), sfList, listOpts...) + if err == nil { + for _, sset := range sfList.Items { + sfSetFound, err = shardingv1.CheckSfset(sset.Name, instance, r.Client) + if err == nil { + // See if StatefulSets already exists and create if it doesn't + err = r.Client.Delete(context.Background(), sfSetFound) + if err != nil { + return err + } + } + } + } + + r.osh[idx].deltopology = false + //r.osh[idx].addSem.Release(1) + //r.osh[idx].delSem.Release(1) + //instance1 := &shardingv1alpha1.ProvShard{} + r.osh[idx].Instance = &databasev1alpha1.ShardingDatabase{} + + //r.osh[idx] = nil + + return nil +} + +//============== + +// Get the current instance +func (r *ShardingDatabaseReconciler) checkProvInstance(instance *databasev1alpha1.ShardingDatabase, +) (int, bool) { + + var status bool = false + var idx int + for i := 0; i < len(r.osh); i++ { + idx = i + if r.osh[i] != nil { + if !r.osh[i].deltopology { + if r.osh[i].Instance.Name == instance.Name { + status = true + break + } + } + } + } + return idx, status +} + +// =========== validate Specs ============ +func (r *ShardingDatabaseReconciler) validateSpex(instance *databasev1alpha1.ShardingDatabase, idx int) error { + + var eventMsg string + var eventErr string = "Spec Error" + + lastSuccSpec, err := instance.GetLastSuccessfulSpec() + if err != nil { + return nil + } + + // Check if last Successful update nil or not + if lastSuccSpec == nil { + // Logic to check if inital Spec is good or not + + // Once the initial Spec is been validated then update the last Sucessful Spec + err = instance.UpdateLastSuccessfulSpec(r.Client) + if err != nil { + return err + } + } else { + // if the last sucessful spec is not nil + // check the parameters which cannot be changed + if lastSuccSpec.Namespace != instance.Spec.Namespace { + eventMsg = "ShardingDatabase CRD resource " + shardingv1.GetFmtStr(instance.Name) + " namespace changed from " + shardingv1.GetFmtStr(lastSuccSpec.Namespace) + " to " + shardingv1.GetFmtStr(instance.Spec.Namespace) + ". This change is not allowed." + r.Recorder.Eventf(instance, corev1.EventTypeWarning, eventErr, eventMsg) + return fmt.Errorf("instance spec has changed and namespace change is not supported") + } + + if lastSuccSpec.DbImage != instance.Spec.DbImage { + eventMsg = "ShardingDatabase CRD resource " + shardingv1.GetFmtStr(instance.Name) + " DBImage changed from " + shardingv1.GetFmtStr(lastSuccSpec.DbImage) + " to " + shardingv1.GetFmtStr(instance.Spec.DbImage) + ". This change is not allowed." + r.Recorder.Eventf(instance, corev1.EventTypeWarning, eventErr, eventMsg) + return fmt.Errorf("instance spec has changed and DbImage change is not supported") + } + + if lastSuccSpec.GsmImage != instance.Spec.GsmImage { + eventMsg = "ShardingDatabase CRD resource " + shardingv1.GetFmtStr(instance.Name) + " GsmImage changed from " + shardingv1.GetFmtStr(lastSuccSpec.GsmImage) + " to " + shardingv1.GetFmtStr(instance.Spec.GsmImage) + ". This change is not allowed." + r.Recorder.Eventf(instance, corev1.EventTypeWarning, eventErr, eventMsg) + return fmt.Errorf("instance spec has changed and GsmImage change is not supported") + } + + if lastSuccSpec.StorageClass != instance.Spec.StorageClass { + eventMsg = "ShardingDatabase CRD resource " + shardingv1.GetFmtStr(instance.Name) + " StorageClass changed from " + shardingv1.GetFmtStr(lastSuccSpec.StorageClass) + " to " + shardingv1.GetFmtStr(instance.Spec.StorageClass) + ". This change is not allowed." + r.Recorder.Eventf(instance, corev1.EventTypeWarning, eventErr, eventMsg) + return fmt.Errorf("instance spec has changed and StorageClass change is not supported") + } + + // Compare Env variables for shard begins here + if !r.comapreShardEnvVariables(instance, lastSuccSpec) { + return fmt.Errorf("Change of Shard env variables are not") + } + // Compare Env variables for catalog begins here + if !r.comapreCatalogEnvVariables(instance, lastSuccSpec) { + return fmt.Errorf("Change of Catalog env variables are not") + } + // Compare env variable for Catalog ends here + if !r.comapreGsmEnvVariables(instance, lastSuccSpec) { + return fmt.Errorf("Change of GSM env variables are not") + } + + } + return nil +} + +// Compare GSM Env Variables + +func (r *ShardingDatabaseReconciler) comapreGsmEnvVariables(instance *databasev1alpha1.ShardingDatabase, lastSuccSpec *databasev1alpha1.ShardingDatabaseSpec) bool { + var eventMsg string + var eventErr string = "Spec Error" + var i, j int32 + + if len(instance.Spec.Gsm) > 0 { + for i = 0; i < int32(len(instance.Spec.Gsm)); i++ { + OraGsmSpex := instance.Spec.Gsm[i] + for j = 0; j < int32(len(lastSuccSpec.Gsm)); j++ { + if OraGsmSpex.Name == lastSuccSpec.Gsm[j].Name { + if !reflect.DeepEqual(OraGsmSpex.EnvVars, lastSuccSpec.Gsm[j].EnvVars) { + eventMsg = "ShardingDatabase CRD resource " + shardingv1.GetFmtStr(instance.Name) + " env vairable changes are not supported." + r.Recorder.Eventf(instance, corev1.EventTypeWarning, eventErr, eventMsg) + return false + } + } + // child for loop ens here + } + //Main For loop ends here + } + } + + return true +} + +func (r *ShardingDatabaseReconciler) comapreCatalogEnvVariables(instance *databasev1alpha1.ShardingDatabase, lastSuccSpec *databasev1alpha1.ShardingDatabaseSpec) bool { + var eventMsg string + var eventErr string = "Spec Error" + var i, j int32 + + if len(instance.Spec.Catalog) > 0 { + for i = 0; i < int32(len(instance.Spec.Catalog)); i++ { + OraCatalogSpex := instance.Spec.Catalog[i] + for j = 0; j < int32(len(lastSuccSpec.Catalog)); j++ { + if OraCatalogSpex.Name == lastSuccSpec.Catalog[j].Name { + if !reflect.DeepEqual(OraCatalogSpex.EnvVars, lastSuccSpec.Catalog[j].EnvVars) { + eventMsg = "ShardingDatabase CRD resource " + shardingv1.GetFmtStr(instance.Name) + " env vairable changes are not supported." + r.Recorder.Eventf(instance, corev1.EventTypeWarning, eventErr, eventMsg) + return false + } + } + // child for loop ens here + } + //Main For loop ends here + } + } + + return true +} + +func (r *ShardingDatabaseReconciler) comapreShardEnvVariables(instance *databasev1alpha1.ShardingDatabase, lastSuccSpec *databasev1alpha1.ShardingDatabaseSpec) bool { + var eventMsg string + var eventErr string = "Spec Error" + var i, j int32 + + if len(instance.Spec.Shard) > 0 { + for i = 0; i < int32(len(instance.Spec.Shard)); i++ { + OraShardSpex := instance.Spec.Shard[i] + for j = 0; j < int32(len(lastSuccSpec.Shard)); j++ { + if OraShardSpex.Name == lastSuccSpec.Shard[j].Name { + if !reflect.DeepEqual(OraShardSpex.EnvVars, lastSuccSpec.Shard[j].EnvVars) { + eventMsg = "ShardingDatabase CRD resource " + shardingv1.GetFmtStr(instance.Name) + " env vairable changes are not supported." + r.Recorder.Eventf(instance, corev1.EventTypeWarning, eventErr, eventMsg) + return false + } + } + // child for loop ens here + } + //Main For loop ends here + } + } + + return true +} + +//===== Set the CRD resource life cycle state ======== + +func (r *ShardingDatabaseReconciler) setCrdLifeCycleState(instance *databasev1alpha1.ShardingDatabase, result *ctrl.Result, err *error, stateType *string) { + + var metaCondition metav1.Condition + var updateFlag = false + if *stateType == "ReconcileWaiting" { + metaCondition = shardingv1.GetMetaCondition(instance, result, err, *stateType, string(databasev1alpha1.CrdReconcileWaitingReason)) + updateFlag = true + } else if *stateType == "ReconcileComplete" { + metaCondition = shardingv1.GetMetaCondition(instance, result, err, *stateType, string(databasev1alpha1.CrdReconcileCompleteReason)) + updateFlag = true + } else if result.Requeue { + metaCondition = shardingv1.GetMetaCondition(instance, result, err, string(databasev1alpha1.CrdReconcileQueuedState), string(databasev1alpha1.CrdReconcileQueuedReason)) + updateFlag = true + } else if *err != nil { + metaCondition = shardingv1.GetMetaCondition(instance, result, err, string(databasev1alpha1.CrdReconcileErrorState), string(databasev1alpha1.CrdReconcileErrorReason)) + updateFlag = true + } else { + + } + if updateFlag == true { + if len(instance.Status.CrdStatus) > 0 { + //meta.SetStatusCondition() + meta.RemoveStatusCondition(&instance.Status.CrdStatus, metaCondition.Type) + } + meta.SetStatusCondition(&instance.Status.CrdStatus, metaCondition) + // Always refresh status before a reconcile + r.Client.Status().Update(context.TODO(), instance) + } + +} + +func (r *ShardingDatabaseReconciler) validateGsmnCatalog(instance *databasev1alpha1.ShardingDatabase) error { + var err error + _, _, err = r.validateCatalog(instance) + if err != nil { + return err + } + _, _, err = r.validateGsm(instance) + if err != nil { + return err + } + return nil +} + +func (r *ShardingDatabaseReconciler) validateGsm(instance *databasev1alpha1.ShardingDatabase, +) (*appsv1.StatefulSet, *corev1.Pod, error) { + //var err error + var i int32 + //var err error + availableFlag := false + + gsmSfSet := &appsv1.StatefulSet{} + gsmPod := &corev1.Pod{} + + for i = 0; i < int32(len(instance.Spec.Gsm)); i++ { + gsmSfSet1, gsmPod1, err := r.validateInvidualGsm(instance, instance.Spec.Gsm[i], int(i)) + if err == nil { + if availableFlag != true { + gsmSfSet = gsmSfSet1 + gsmPod = gsmPod1 + availableFlag = true + } + } + } + + if availableFlag == true { + return gsmSfSet, gsmPod, nil + } + return gsmSfSet, gsmPod, fmt.Errorf("GSM is not ready") +} + +func (r *ShardingDatabaseReconciler) validateInvidualGsm(instance *databasev1alpha1.ShardingDatabase, OraGsmSpex databasev1alpha1.GsmSpec, specId int, +) (*appsv1.StatefulSet, *corev1.Pod, error) { + //var err error + var i int32 + var msg string + gsmSfSet := &appsv1.StatefulSet{} + gsmPod := &corev1.Pod{} + + var err error + podList := &corev1.PodList{} + var isPodExist bool + + gsmSfSet, err = shardingv1.CheckSfset(OraGsmSpex.Name, instance, r.Client) + if err != nil { + msg = "Unable to find GSM statefulset " + shardingv1.GetFmtStr(OraGsmSpex.Name) + "." + shardingv1.LogMessages("INFO", msg, nil, instance, r.Log) + r.updateGsmStatus(instance, int(i), string(databasev1alpha1.StatefulSetNotFound)) + return gsmSfSet, gsmPod, err + } + + podList, err = shardingv1.GetPodList(gsmSfSet.Name, "GSM", instance, r.Client) + if err != nil { + msg = "Unable to find any pod in statefulset " + shardingv1.GetFmtStr(gsmSfSet.Name) + "." + shardingv1.LogMessages("INFO", msg, nil, instance, r.Log) + r.updateGsmStatus(instance, int(i), string(databasev1alpha1.PodNotFound)) + return gsmSfSet, gsmPod, err + } + + isPodExist, gsmPod = shardingv1.PodListValidation(podList, gsmSfSet.Name, instance, r.Client) + if !isPodExist { + msg = "Unable to validate GSM " + shardingv1.GetFmtStr(gsmPod.Name) + " pod. GSM pod doesn't seems to be ready to accept the commands." + shardingv1.LogMessages("INFO", msg, nil, instance, r.Log) + r.updateGsmStatus(instance, int(i), string(databasev1alpha1.PodNotReadyState)) + return gsmSfSet, gsmPod, fmt.Errorf("Pod doesn't exist") + } + err = shardingv1.CheckGsmStatus(gsmPod.Name, instance, r.kubeClient, r.kubeConfig, r.Log) + if err != nil { + msg = "Unable to validate GSM director. GSM director doesn't seems to be ready to accept the commands." + shardingv1.LogMessages("INFO", msg, nil, instance, r.Log) + r.updateGsmStatus(instance, int(i), string(databasev1alpha1.ProvisionState)) + return gsmSfSet, gsmPod, err + } + + r.updateGsmStatus(instance, specId, string(databasev1alpha1.AvailableState)) + return gsmSfSet, gsmPod, nil +} + +func (r *ShardingDatabaseReconciler) validateCatalog(instance *databasev1alpha1.ShardingDatabase, +) (*appsv1.StatefulSet, *corev1.Pod, error) { + + catalogSfSet := &appsv1.StatefulSet{} + catalogPod := &corev1.Pod{} + //var err error + var i int32 + availlableFlag := false + + for i = 0; i < int32(len(instance.Spec.Catalog)); i++ { + catalogSfSet1, catalogPod1, err := r.validateInvidualCatalog(instance, instance.Spec.Catalog[i], int(i)) + if err == nil { + if availlableFlag != true { + catalogSfSet = catalogSfSet1 + catalogPod = catalogPod1 + availlableFlag = true + } + } + } + + if availlableFlag == true { + return catalogSfSet, catalogPod, nil + } + + return catalogSfSet, catalogPod, fmt.Errorf("Catalog is not available") +} + +// === Validate Individual Catalog +func (r *ShardingDatabaseReconciler) validateInvidualCatalog(instance *databasev1alpha1.ShardingDatabase, OraCatalogSpex databasev1alpha1.CatalogSpec, specId int, +) (*appsv1.StatefulSet, *corev1.Pod, error) { + + var err error + catalogSfSet := &appsv1.StatefulSet{} + catalogPod := &corev1.Pod{} + podList := &corev1.PodList{} + var isPodExist bool + + catalogSfSet, err = shardingv1.CheckSfset(OraCatalogSpex.Name, instance, r.Client) + if err != nil { + msg := "Unable to find Catalog statefulset " + shardingv1.GetFmtStr(OraCatalogSpex.Name) + "." + shardingv1.LogMessages("INFO", msg, nil, instance, r.Log) + r.updateCatalogStatus(instance, specId, string(databasev1alpha1.StatefulSetNotFound)) + return catalogSfSet, catalogPod, err + } + + podList, err = shardingv1.GetPodList(catalogSfSet.Name, "CATALOG", instance, r.Client) + if err != nil { + msg := "Unable to find any pod in statefulset " + shardingv1.GetFmtStr(catalogSfSet.Name) + "." + shardingv1.LogMessages("INFO", msg, nil, instance, r.Log) + r.updateCatalogStatus(instance, specId, string(databasev1alpha1.PodNotFound)) + return catalogSfSet, catalogPod, err + } + isPodExist, catalogPod = shardingv1.PodListValidation(podList, catalogSfSet.Name, instance, r.Client) + if !isPodExist { + msg := "Unable to validate Catalog " + shardingv1.GetFmtStr(catalogSfSet.Name) + " pod. Catalog pod doesn't seems to be ready to accept the commands." + shardingv1.LogMessages("INFO", msg, nil, instance, r.Log) + r.updateCatalogStatus(instance, specId, string(databasev1alpha1.PodNotReadyState)) + return catalogSfSet, catalogPod, fmt.Errorf("Pod doesn't exist") + } + err = shardingv1.ValidateDbSetup(catalogPod.Name, instance, r.kubeClient, r.kubeConfig, r.Log) + if err != nil { + msg := "Unable to validate Catalog. Catalog doesn't seems to be ready to accept the commands." + shardingv1.LogMessages("INFO", msg, nil, instance, r.Log) + r.updateCatalogStatus(instance, specId, string(databasev1alpha1.ProvisionState)) + return catalogSfSet, catalogPod, err + } + + r.updateCatalogStatus(instance, specId, string(databasev1alpha1.AvailableState)) + return catalogSfSet, catalogPod, nil + +} + +// ======= Function to validate Shard +func (r *ShardingDatabaseReconciler) validateShard(instance *databasev1alpha1.ShardingDatabase, OraShardSpex databasev1alpha1.ShardSpec, specId int, +) (*appsv1.StatefulSet, *corev1.Pod, error) { + + var err error + shardSfSet := &appsv1.StatefulSet{} + shardPod := &corev1.Pod{} + + shardSfSet, err = shardingv1.CheckSfset(OraShardSpex.Name, instance, r.Client) + if err != nil { + msg := "Unable to find Shard statefulset " + shardingv1.GetFmtStr(OraShardSpex.Name) + "." + shardingv1.LogMessages("INFO", msg, nil, instance, r.Log) + r.updateShardStatus(instance, specId, string(databasev1alpha1.StatefulSetNotFound)) + return shardSfSet, shardPod, err + } + + podList, err := shardingv1.GetPodList(shardSfSet.Name, "SHARD", instance, r.Client) + if err != nil { + msg := "Unable to find any pod in statefulset " + shardingv1.GetFmtStr(shardSfSet.Name) + "." + shardingv1.LogMessages("INFO", msg, nil, instance, r.Log) + r.updateShardStatus(instance, specId, string(databasev1alpha1.PodNotFound)) + return shardSfSet, shardPod, err + } + isPodExist, shardPod := shardingv1.PodListValidation(podList, shardSfSet.Name, instance, r.Client) + if !isPodExist { + msg := "Unable to validate Shard " + shardingv1.GetFmtStr(shardPod.Name) + " pod. Shard pod doesn't seems to be ready to accept the commands." + shardingv1.LogMessages("INFO", msg, nil, instance, r.Log) + r.updateShardStatus(instance, specId, string(databasev1alpha1.PodNotReadyState)) + return shardSfSet, shardPod, err + } + err = shardingv1.ValidateDbSetup(shardPod.Name, instance, r.kubeClient, r.kubeConfig, r.Log) + if err != nil { + msg := "Unable to validate shard. Shard doesn't seems to be ready to accept the commands." + shardingv1.LogMessages("INFO", msg, nil, instance, r.Log) + r.updateShardStatus(instance, specId, string(databasev1alpha1.ProvisionState)) + return shardSfSet, shardPod, err + } + + r.updateShardStatus(instance, specId, string(databasev1alpha1.AvailableState)) + return shardSfSet, shardPod, nil +} + +// This function updates the shard topology over all +// +func (r *ShardingDatabaseReconciler) updateShardTopologyStatus(instance *databasev1alpha1.ShardingDatabase) error { + //shardPod := &corev1.Pod{} + //gsmSfSet := &appsv1.StatefulSet{} + gsmPod := &corev1.Pod{} + var err error + _, _, err = r.validateCatalog(instance) + if err != nil { + return err + } + _, gsmPod, err = r.validateGsm(instance) + if err != nil { + return err + } + r.updateShardTopologyShardsInGsm(instance, gsmPod) + + return nil + +} + +func (r *ShardingDatabaseReconciler) updateShardTopologyShardsInGsm(instance *databasev1alpha1.ShardingDatabase, gsmPod *corev1.Pod) { + shardSfSet := &appsv1.StatefulSet{} + //shardPod := &corev1.Pod{} + //gsmSfSet := &appsv1.StatefulSet{} + var err error + var i int32 + if len(instance.Spec.Shard) > 0 { + for i = 0; i < int32(len(instance.Spec.Shard)); i++ { + OraShardSpex := instance.Spec.Shard[i] + // stateStr := shardingv1.GetGsmShardStatus(instance, OraShardSpex.Name) + if OraShardSpex.IsDelete != true { + shardSfSet, _, err = r.validateShard(instance, OraShardSpex, int(i)) + if err != nil { + continue + } else { + _ = r.verifyShards(instance, gsmPod, shardSfSet) + } + } + + } + } +} + +func (r *ShardingDatabaseReconciler) updateGsmStatus(instance *databasev1alpha1.ShardingDatabase, specIdx int, state string) { + + var currState string + var eventMsg string + var eventMsgFlag = true + + // Populating GSM Details + name := instance.Spec.Gsm[specIdx].Name + //ServiceNames := shardingv1.GetGsmSvcName(instance.Spec.Gsm[specIdx].EnvVars) + + if len(instance.Status.Gsm.State) > 0 { + currState = instance.Status.Gsm.State + if currState == state { + eventMsgFlag = false + } + eventMsg = "The gsm " + shardingv1.GetFmtStr(name) + " state changed from " + currState + " to " + state + } else { + eventMsg = "The gsm " + shardingv1.GetFmtStr(name) + " state set to " + state + } + + // if currState != state { + instance.Status.Gsm.State = state + shardingv1.UpdateGsmStatusData(instance, specIdx, state, r.kubeClient, r.kubeConfig, r.Log) + r.Status().Update(context.Background(), instance) + // } + if eventMsgFlag == true { + r.publishEvents(instance, eventMsg, state) + } +} + +func (r *ShardingDatabaseReconciler) updateCatalogStatus(instance *databasev1alpha1.ShardingDatabase, specIdx int, state string) { + var eventMsg string + var currState string + var eventMsgFlag = true + + name := instance.Spec.Catalog[specIdx].Name + + if len(instance.Status.Catalog) > 0 { + currState = shardingv1.GetGsmCatalogStatusKey(instance, name+"_"+string(databasev1alpha1.State)) + if currState == state { + eventMsgFlag = false + } + eventMsg = "The catalog " + shardingv1.GetFmtStr(name) + " state changed from " + currState + " to " + state + + } else { + eventMsg = "The catalog " + shardingv1.GetFmtStr(name) + " state set to " + state + } + + //if currState != state { + shardingv1.UpdateCatalogStatusData(instance, specIdx, state, r.kubeClient, r.kubeConfig, r.Log) + r.Status().Update(context.Background(), instance) + //} + if eventMsgFlag == true { + r.publishEvents(instance, eventMsg, state) + } +} + +func (r *ShardingDatabaseReconciler) updateShardStatus(instance *databasev1alpha1.ShardingDatabase, specIdx int, state string) { + var eventMsg string + var currState string + var eventMsgFlag = true + + name := instance.Spec.Shard[specIdx].Name + if len(instance.Status.Shard) > 0 { + currState = shardingv1.GetGsmShardStatusKey(instance, name+"_"+string(databasev1alpha1.State)) + if currState == state { + eventMsgFlag = false + } + eventMsg = "The shard " + shardingv1.GetFmtStr(name) + " state changed from " + currState + " to " + state + + } else { + eventMsg = "The shard " + shardingv1.GetFmtStr(name) + " state set to " + state + } + + //if currState != state { + shardingv1.UpdateShardStatusData(instance, specIdx, state, r.kubeClient, r.kubeConfig, r.Log) + r.Status().Update(context.Background(), instance) + //} + if eventMsgFlag == true { + r.publishEvents(instance, eventMsg, state) + } +} + +func (r *ShardingDatabaseReconciler) updateGsmShardStatus(instance *databasev1alpha1.ShardingDatabase, name string, state string) { + var eventMsg string + var currState string + var eventMsgFlag = true + + if len(instance.Status.Gsm.Shards) > 0 { + currState = shardingv1.GetGsmShardStatus(instance, name) + if currState == state { + eventMsgFlag = false + } + if currState != "NOSTATE" { + eventMsg = "The shard " + shardingv1.GetFmtStr(name) + " state changed from " + currState + " to " + state + " in Gsm" + } else { + eventMsg = "The shard " + shardingv1.GetFmtStr(name) + " state set to " + state + " in Gsm." + } + + } else { + eventMsg = "The shard " + shardingv1.GetFmtStr(name) + " state set to " + state + " in Gsm." + } + + if currState != state { + shardingv1.UpdateGsmShardStatus(instance, name, state) + r.Status().Update(context.Background(), instance) + } + + if eventMsgFlag == true { + r.publishEvents(instance, eventMsg, state) + } +} + +// This function add the Primary Shards in GSM +func (r *ShardingDatabaseReconciler) addPrimaryShards(instance *databasev1alpha1.ShardingDatabase, idx int) error { + //var result ctrl.Result + var result ctrl.Result + var i int32 + var err error + shardSfSet := &appsv1.StatefulSet{} + //shardPod := &corev1.Pod{} + //gsmSfSet := &appsv1.StatefulSet{} + gsmPod := &corev1.Pod{} + var sparams1 string + var deployFlag = false + var errStr = false + //var msg string + + var setLifeCycleFlag = false + var title string + var message string + + shardingv1.LogMessages("DEBUG", "Starting the shard adding operaiton.", nil, instance, r.Log) + // ================================ Add Shard Logic =================== + if len(instance.Spec.Shard) > 0 { + for i = 0; i < int32(len(instance.Spec.Shard)); i++ { + OraShardSpex := instance.Spec.Shard[i] + // stateStr := shardingv1.GetGsmShardStatus(instance, OraShardSpex.Name) + // !strings.Contains(stateStr, "DELETE") + if OraShardSpex.IsDelete != true { + if setLifeCycleFlag != true { + setLifeCycleFlag = true + stateType := string(databasev1alpha1.CrdReconcileWaitingState) + r.setCrdLifeCycleState(instance, &result, &err, &stateType) + } + // 1st Step is to check if Shard is in good state if not then just continue + // validateShard will change the shard state in Shard Status + shardSfSet, _, err = r.validateShard(instance, OraShardSpex, int(i)) + if err != nil { + errStr = true + continue + } + // 2nd Step is to check if GSM is in good state if not then just return because you can't do anything + _, gsmPod, err = r.validateGsm(instance) + if err != nil { + return err + } + // 3rd step to check if shard is in GSM if not then continue + sparams := shardingv1.BuildShardParams(shardSfSet) + sparams1 = sparams + err = shardingv1.CheckShardInGsm(gsmPod.Name, sparams, instance, r.kubeClient, r.kubeConfig, r.Log) + if err == nil { + // if you are in this block then it means that shard already exist in the GSM and we do not need to anything + continue + } + // If the shard doesn't exist in GSM then just add the shard statefulset and update GSM shard status + // ADD Shard in GSM + r.updateGsmShardStatus(instance, OraShardSpex.Name, string(databasev1alpha1.AddingShardState)) + err := shardingv1.AddShardInGsm(gsmPod.Name, sparams, instance, r.kubeClient, r.kubeConfig, r.Log) + if err != nil { + r.updateGsmShardStatus(instance, OraShardSpex.Name, string(databasev1alpha1.AddingShardErrorState)) + title = "Shard Addition Failure" + message = "Error occurred during shard " + shardingv1.GetFmtStr(OraShardSpex.Name) + " addition." + r.sendMessage(instance, title, message) + } else { + deployFlag = true + } + } + } + + // ======= Deploy Shard Logic ========= + if deployFlag == true { + _ = shardingv1.DeployShardInGsm(gsmPod.Name, sparams1, instance, r.kubeClient, r.kubeConfig, r.Log) + r.updateShardTopologyShardsInGsm(instance, gsmPod) + } + } + if errStr == true { + shardingv1.LogMessages("INFO", "Some shards are still pending for addition. Requeue the reconcile loop.", nil, instance, r.Log) + return fmt.Errorf("Shard Addition is pending.") + } + shardingv1.LogMessages("INFO", "Completed the shard addition operation. For details, check the CRD resource status for GSM and Shards.", nil, instance, r.Log) + return nil +} + +// This function Check the online shard +func (r *ShardingDatabaseReconciler) verifyShards(instance *databasev1alpha1.ShardingDatabase, gsmPod *corev1.Pod, shardSfSet *appsv1.StatefulSet) error { + //var result ctrl.Result + //var i int32 + var err error + var title string + var message string + // ================================ Check Shards ================== + //veryify shard make shard state online and it must be executed to check shard state after every CRUD operation + sparams := shardingv1.BuildShardParams(shardSfSet) + err = shardingv1.CheckOnlineShardInGsm(gsmPod.Name, sparams, instance, r.kubeClient, r.kubeConfig, r.Log) + if err != nil { + // If the shard doesn't exist in GSM then just delete the shard statefulset and update GSM shard status + /// Terminate state means we will remove teh shard entry from GSM shard status + r.updateGsmShardStatus(instance, shardSfSet.Name, string(databasev1alpha1.ShardOnlineErrorState)) + shardingv1.CancelChunksInGsm(gsmPod.Name, sparams, instance, r.kubeClient, r.kubeConfig, r.Log) + return err + } + oldStateStr := shardingv1.GetGsmShardStatus(instance, shardSfSet.Name) + r.updateGsmShardStatus(instance, shardSfSet.Name, string(databasev1alpha1.ShardOnlineState)) + // Following logic will sent a email only once + if oldStateStr != string(databasev1alpha1.ShardOnlineState) { + title = "Shard Addition Completed" + message = "Shard addition completed for shard " + shardingv1.GetFmtStr(shardSfSet.Name) + " in GSM." + r.sendMessage(instance, title, message) + } + return nil +} + +func (r *ShardingDatabaseReconciler) addStandbyShards(instance *databasev1alpha1.ShardingDatabase, idx int) error { + //var result ctrl.Result + + return nil +} + +// ========== Delete Shard Section==================== +func (r *ShardingDatabaseReconciler) delGsmShard(instance *databasev1alpha1.ShardingDatabase, idx int) error { + var result ctrl.Result + var i int32 + var err error + shardSfSet := &appsv1.StatefulSet{} + shardPod := &corev1.Pod{} + //gsmSfSet := &appsv1.StatefulSet{} + gsmPod := &corev1.Pod{} + var msg string + var title string + var message string + var setLifeCycleFlag = false + + shardingv1.LogMessages("DEBUG", "Starting shard deletion operation.", nil, instance, r.Log) + // ================================ Shard Delete Logic =================== + if len(instance.Spec.Shard) > 0 { + for i = 0; i < int32(len(instance.Spec.Shard)); i++ { + OraShardSpex := instance.Spec.Shard[i] + if OraShardSpex.IsDelete == true { + if setLifeCycleFlag != true { + setLifeCycleFlag = true + stateType := string(databasev1alpha1.CrdReconcileWaitingState) + r.setCrdLifeCycleState(instance, &result, &err, &stateType) + } + // Step 1st to check if GSM is in good state if not then just return because you can't do anything + _, gsmPod, err = r.validateGsm(instance) + if err != nil { + return err + } + // 2nd Step is to check if Shard is in good state if not then just continue + // 1St check if the instance.Status.Gsm.Shards contains the shard. If not then shard is already deleted + // If the shard is found then check if shard exist + // validateShard will change the shard state in Shard Status + chkState := shardingv1.GetGsmShardStatus(instance, OraShardSpex.Name) + if chkState != "NOSTATE" { + shardSfSet, shardPod, err = r.validateShard(instance, OraShardSpex, int(i)) + if err != nil { + continue + } + } else { + continue + } + // 3rd step to check if shard is in GSM if not then continue + sparams := shardingv1.BuildShardParams(shardSfSet) + err = shardingv1.CheckShardInGsm(gsmPod.Name, sparams, instance, r.kubeClient, r.kubeConfig, r.Log) + if err != nil { + // If the shard doesn't exist in GSM then just delete the shard statefulset and update GSM shard status + /// Terminate state means we will remove teh shard entry from GSM shard status + r.delShard(instance, shardSfSet.Name, shardSfSet, shardPod, int(i)) + r.updateGsmShardStatus(instance, OraShardSpex.Name, string(databasev1alpha1.Terminated)) + r.updateShardStatus(instance, int(i), string(databasev1alpha1.Terminated)) + continue + } + // 4th step to check if shard is in GSM and shard is online if not then continue + // CHeck before deletion if GSM is not ready set the Shard State to Delete Error + r.updateGsmShardStatus(instance, OraShardSpex.Name, string(databasev1alpha1.DeletingState)) + err = shardingv1.CheckOnlineShardInGsm(gsmPod.Name, sparams, instance, r.kubeClient, r.kubeConfig, r.Log) + if err != nil { + // If the shard doesn't exist in GSM then just delete the shard statefulset and update GSM shard status + /// Terminate state means we will remove teh shard entry from GSM shard status + r.updateGsmShardStatus(instance, OraShardSpex.Name, string(databasev1alpha1.DeleteErrorState)) + continue + } + // 5th Step + // Move the chunks before performing any Delete + // If you are in this block then it means that shard is ONline and can be deleted + err = shardingv1.MoveChunks(gsmPod.Name, sparams, instance, r.kubeClient, r.kubeConfig, r.Log) + if err != nil { + r.updateGsmShardStatus(instance, OraShardSpex.Name, string(databasev1alpha1.ChunkMoveError)) + title = "Chunk Movement Failure" + message = "Error occurred during chunk movement in shard " + shardingv1.GetFmtStr(OraShardSpex.Name) + " deletion." + r.sendMessage(instance, title, message) + continue + } + // 6th Step + // Check if Chunks has moved before performing actual delete + // This is a loop and will check unless there is a error or chunks has moved + // Validate if the chunks has moved before performing shard deletion + for { + err = shardingv1.VerifyChunks(gsmPod.Name, sparams, instance, r.kubeClient, r.kubeConfig, r.Log) + if err == nil { + break + } else { + msg = "Sleeping for 120 seconds and will check status again of chunks movement in gsm for shard: " + shardingv1.GetFmtStr(OraShardSpex.Name) + shardingv1.LogMessages("INFO", msg, nil, instance, r.Log) + time.Sleep(120 * time.Second) + } + } + // 7th Step remove the shards from the GSM + // This steps will delete the shard entry from the GSM + // It will delete CDB from catalog + // 6th Step has already moved the chunks so it is safe to delete + err = shardingv1.RemoveShardFromGsm(gsmPod.Name, sparams, instance, r.kubeClient, r.kubeConfig, r.Log) + if err != nil { + msg = "Error occurred during shard" + shardingv1.GetFmtStr(OraShardSpex.Name) + "removal from Gsm" + shardingv1.LogMessages("Error", msg, nil, instance, r.Log) + r.updateShardStatus(instance, int(i), string(databasev1alpha1.ShardRemoveError)) + continue + } + // 8th Step + // Delete the Statefulset as all the chunks has moved and Shard can be phyiscally deleted + r.delShard(instance, shardSfSet.Name, shardSfSet, shardPod, int(i)) + r.updateGsmShardStatus(instance, OraShardSpex.Name, string(databasev1alpha1.Terminated)) + r.updateShardStatus(instance, int(i), string(databasev1alpha1.Terminated)) + title = "Shard Deletion Completed" + message = "Shard deletion completed for shard " + shardingv1.GetFmtStr(OraShardSpex.Name) + " in GSM." + r.sendMessage(instance, title, message) + } + } + } + shardingv1.LogMessages("DEBUG", "Completed the shard deletion operation. For details, check the CRD resource status for GSM and Shards.", nil, instance, r.Log) + return nil +} + +// This function delete the physical shard +func (r *ShardingDatabaseReconciler) delShard(instance *databasev1alpha1.ShardingDatabase, sfSetName string, sfSetFound *appsv1.StatefulSet, sfsetPod *corev1.Pod, specIdx int) { + + //var status bool + var err error + var msg string + svcFound := &corev1.Service{} + + err = shardingv1.SfsetLabelPatch(sfSetFound, sfsetPod, instance, r.Client) + if err != nil { + msg := "Failed to patch the Shard StatefulSet: " + sfSetFound.Name + shardingv1.LogMessages("DEBUG", msg, err, instance, r.Log) + r.updateShardStatus(instance, specIdx, string(databasev1alpha1.LabelPatchingError)) + return + } + + err = r.Client.Delete(context.Background(), sfSetFound) + if err != nil { + msg = "Failed to delete Shard StatefulSet: " + shardingv1.GetFmtStr(sfSetFound.Name) + shardingv1.LogMessages("DEBUG", msg, err, instance, r.Log) + r.updateShardStatus(instance, specIdx, string(databasev1alpha1.DeleteErrorState)) + return + } + /// Delete External Service + if instance.Spec.IsExternalSvc { + svcFound, err = shardingv1.CheckSvc(sfSetName+strconv.FormatInt(int64(0), 10)+"-svc", instance, r.Client) + if err == nil { + // See if StatefulSets already exists and create if it doesn't + err = r.Client.Delete(context.Background(), svcFound) + if err != nil { + return + } + } + } + + // Delete Internal Service + svcFound, err = shardingv1.CheckSvc(sfSetName, instance, r.Client) + if err == nil { + // See if StatefulSets already exists and create if it doesn't + err = r.Client.Delete(context.Background(), svcFound) + if err != nil { + return + } + } + + if instance.Spec.IsDeleteOraPvc && len(instance.Spec.StorageClass) > 0 { + pvcName := sfSetFound.Name + "-oradata-vol4-" + sfSetFound.Name + "-0" + err = shardingv1.DelPvc(pvcName, instance, r.Client, r.Log) + if err != nil { + msg = "Failed to delete Shard pvc claim " + shardingv1.GetFmtStr(pvcName) + shardingv1.LogMessages("DEBUG", msg, err, instance, r.Log) + r.updateShardStatus(instance, specIdx, string(databasev1alpha1.DeletePVCError)) + } + } +} + +//======== GSM Invited Node ========== +// Remove and add GSM invited node +func (r *ShardingDatabaseReconciler) gsmInvitedNodeOp(instance *databasev1alpha1.ShardingDatabase, objName string, +) { + + var msg string + //var err error + count := 0 + msg = "Inside the gsmInvitedNodeOp for adding and deleting the invited node " + shardingv1.LogMessages("DEBUG", msg, nil, instance, r.Log) + //status = + for count < 10 { + _, gsmPodName, err := r.validateGsm(instance) + if err != nil { + msg = "Unable to validate gsm sfSet. " + shardingv1.GetFmtStr(gsmPodName.Name) + " Sleeping for 30 seconds" + shardingv1.LogMessages("DEBUG", msg, err, instance, r.Log) + time.Sleep(20 * time.Second) + count = count + 1 + continue + } + err = shardingv1.ValidateDbSetup(objName, instance, r.kubeClient, r.kubeConfig, r.Log) + if err != nil { + msg = "Unable to validate sfSet. " + shardingv1.GetFmtStr(objName) + " Sleeping for 30 seconds" + shardingv1.LogMessages("DEBUG", msg, err, instance, r.Log) + time.Sleep(20 * time.Second) + count = count + 1 + continue + } + err, _, _ = shardingv1.ExecCommand(gsmPodName.Name, shardingv1.GetShardInviteNodeCmd(objName), r.kubeClient, r.kubeConfig, instance, r.Log) + if err != nil { + msg = "Invite delete and add node failed " + shardingv1.GetFmtStr(objName) + " details in GSM." + shardingv1.LogMessages("DEBUG", msg, err, instance, r.Log) + } else { + + msg = "Invited node operation completed sucessfully in GSM after pod " + shardingv1.GetFmtStr(objName) + " restart." + shardingv1.LogMessages("INFO", msg, nil, instance, r.Log) + break + } + count = count + 1 + + } +} + +// ================================== CREATE FUNCTIONS ============================= +// This function create a service based isExtern parameter set in the yaml file +func (r *ShardingDatabaseReconciler) createService(instance *databasev1alpha1.ShardingDatabase, + dep *corev1.Service, +) (ctrl.Result, error) { + reqLogger := r.Log.WithValues("Instance.Namespace", instance.Spec.Namespace, "Instance.Name", instance.Name) + // See if Service already exists and create if it doesn't + // We are getting error on nil pointer segment when r.scheme is null + // Error : invalid memory address or nil pointer dereference" (runtime error: invalid memory address or nil pointer dereference) + // This happens during unit test cases + for i := 0; i < 5; i++ { + if r.Scheme == nil { + time.Sleep(time.Second * 40) + } else { + break + } + } + controllerutil.SetControllerReference(instance, dep, r.Scheme) + found := &corev1.Service{} + + err := r.Client.Get(context.TODO(), types.NamespacedName{ + Name: dep.Name, + Namespace: instance.Spec.Namespace, + }, found) + + jsn, _ := json.Marshal(dep) + shardingv1.LogMessages("DEBUG", string(jsn), nil, instance, r.Log) + if err != nil && errors.IsNotFound(err) { + // Create the Service + reqLogger.Info("Creating a service") + err = r.Client.Create(context.TODO(), dep) + if err != nil { + // Service creation failed + reqLogger.Error(err, "Failed to create Service", "Service.space", dep.Namespace, "Service.Name", dep.Name) + return ctrl.Result{}, err + } else { + // Service creation was successful + return ctrl.Result{Requeue: true}, nil + } + } else if err != nil { + // Error that isn't due to the Service not existing + reqLogger.Error(err, "Failed to find the Service details") + return ctrl.Result{}, err + } + + return ctrl.Result{}, nil +} + +// This function deploy the statefulset +func (r *ShardingDatabaseReconciler) deployStatefulSet(instance *databasev1alpha1.ShardingDatabase, + dep *appsv1.StatefulSet, + resType string, +) (ctrl.Result, error) { + + reqLogger := r.Log.WithValues("Instance.Namespace", instance.Spec.Namespace, "Instance.Name", instance.Name) + message := "Inside the deployStatefulSet function" + shardingv1.LogMessages("DEBUG", message, nil, instance, r.Log) + // See if StatefulSets already exists and create if it doesn't + controllerutil.SetControllerReference(instance, dep, r.Scheme) + found := &appsv1.StatefulSet{} + err := r.Client.Get(context.TODO(), types.NamespacedName{ + Name: dep.Name, + Namespace: instance.Spec.Namespace, + }, found) + jsn, _ := json.Marshal(dep) + shardingv1.LogMessages("DEBUG", string(jsn), nil, instance, r.Log) + + if err != nil && errors.IsNotFound(err) { + + // Create the StatefulSet + reqLogger.Info("Creating Stateful Shard") + err = r.Client.Create(context.TODO(), dep) + + message := "Inside the create Stateful set block to create statefulset " + shardingv1.GetFmtStr(dep.Name) + shardingv1.LogMessages("DEBUG", message, nil, instance, r.Log) + + if err != nil { + // StatefulSet failed + reqLogger.Error(err, "Failed to create StatefulSet", "StatefulSet.space", dep.Namespace, "StatefulSet.Name", dep.Name) + //instance.Status.ShardStatus[dep.Name] = "Deployment Failed" + return ctrl.Result{}, err + } + } else if err != nil { + // Error that isn't due to the StaefulSet not existing + reqLogger.Error(err, "Failed to get StatefulSet") + return ctrl.Result{}, err + } + + message = "Statefulset Exist " + shardingv1.GetFmtStr(dep.Name) + " already exist" + shardingv1.LogMessages("DEBUG", message, nil, instance, r.Log) + + return ctrl.Result{}, nil +} diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go new file mode 100644 index 00000000..c69ba5b3 --- /dev/null +++ b/controllers/database/singleinstancedatabase_controller.go @@ -0,0 +1,1762 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package controllers + +import ( + "context" + "errors" + "fmt" + "strconv" + "strings" + "time" + + dbapi "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + dbcommons "github.com/oracle/oracle-database-operator/commons/database" + + "github.com/go-logr/logr" + corev1 "k8s.io/api/core/v1" + apierrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/api/meta" + "k8s.io/apimachinery/pkg/api/resource" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/types" + "k8s.io/client-go/rest" + "k8s.io/client-go/tools/record" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/controller" + "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" +) + +// SingleInstanceDatabaseReconciler reconciles a SingleInstanceDatabase object +type SingleInstanceDatabaseReconciler struct { + client.Client + Log logr.Logger + Scheme *runtime.Scheme + Config *rest.Config + Recorder record.EventRecorder +} + +// To requeue after 15 secs allowing graceful state changes +var requeueY ctrl.Result = ctrl.Result{Requeue: true, RequeueAfter: 15 * time.Second} +var requeueN ctrl.Result = ctrl.Result{} + +const singleInstanceDatabaseFinalizer = "database.oracle.com/singleinstancedatabasefinalizer" + +//+kubebuilder:rbac:groups=database.oracle.com,resources=singleinstancedatabases,verbs=get;list;watch;create;update;patch;delete +//+kubebuilder:rbac:groups=database.oracle.com,resources=singleinstancedatabases/status,verbs=get;update;patch +//+kubebuilder:rbac:groups=database.oracle.com,resources=singleinstancedatabases/finalizers,verbs=update +//+kubebuilder:rbac:groups="",resources=pods;pods/log;pods/exec;persistentvolumeclaims;services;nodes;events,verbs=create;delete;get;list;patch;update;watch + +// Reconcile is part of the main kubernetes reconciliation loop which aims to +// move the current state of the cluster closer to the desired state. +// TODO(user): Modify the Reconcile function to compare the state specified by +// the SingleInstanceDatabase object against the actual cluster state, and then +// perform operations to make the cluster state reflect the state specified by +// the user. +// +// For more details, check Reconcile and its Result here: +// - https://pkg.go.dev/sigs.k8s.io/controller-runtime@v0.8.3/pkg/reconcile +func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + + r.Log.Info("Reconcile requested") + var result ctrl.Result + var err error + completed := false + blocked := false + + singleInstanceDatabase := &dbapi.SingleInstanceDatabase{} + cloneFromDatabase := &dbapi.SingleInstanceDatabase{} + + // Execute for every reconcile + defer r.updateReconcileStatus(singleInstanceDatabase, ctx, &result, &err, &blocked, &completed) + + err = r.Get(ctx, types.NamespacedName{Namespace: req.Namespace, Name: req.Name}, singleInstanceDatabase) + if err != nil { + if apierrors.IsNotFound(err) { + r.Log.Info("Resource not found") + return requeueN, nil + } + return requeueY, err + } + + // Manage SingleInstanceDatabase Deletion + result, err = r.manageSingleInstanceDatabaseDeletion(req, ctx, singleInstanceDatabase) + if result.Requeue { + r.Log.Info("Reconcile queued") + return result, nil + } + + // First validate + result, err = r.validate(singleInstanceDatabase, cloneFromDatabase, ctx, req) + if result.Requeue { + r.Log.Info("Spec validation failed, Reconcile queued") + return result, nil + } + if err != nil { + r.Log.Info("Spec validation failed") + return result, nil + } + + // Service creation + result, err = r.createOrReplaceSVC(ctx, req, singleInstanceDatabase) + if result.Requeue { + r.Log.Info("Reconcile queued") + return result, nil + } + + // PVC Creation + result, err = r.createOrReplacePVC(ctx, req, singleInstanceDatabase) + if result.Requeue { + r.Log.Info("Reconcile queued") + return result, nil + } + + // POD creation + result, err = r.createOrReplacePods(singleInstanceDatabase, cloneFromDatabase, ctx, req) + if result.Requeue { + r.Log.Info("Reconcile queued") + return result, nil + } + + if singleInstanceDatabase.Status.DatafilesCreated != "true" { + // Creation of Oracle Wallet for Single Instance Database credentials + result, err = r.createWallet(singleInstanceDatabase, ctx, req) + if result.Requeue { + r.Log.Info("Reconcile queued") + return result, nil + } + if err != nil { + r.Log.Info("Spec validation failed") + return result, nil + } + } + + // Validate readiness + var readyPod corev1.Pod + result, readyPod, err = r.validateDBReadiness(singleInstanceDatabase, ctx, req) + if result.Requeue { + r.Log.Info("Reconcile queued") + return result, nil + } + + // Post DB ready operations + + // Deleting the oracle wallet + if singleInstanceDatabase.Status.DatafilesCreated == "true" { + result, err = r.deleteWallet(singleInstanceDatabase, ctx, req) + if result.Requeue { + r.Log.Info("Reconcile queued") + return result, nil + } + } + + // Update DB config + result, err = r.updateDBConfig(singleInstanceDatabase, readyPod, ctx, req) + if result.Requeue { + r.Log.Info("Reconcile queued") + return result, nil + } + + // Update Init Parameters + result, err = r.updateInitParameters(singleInstanceDatabase, readyPod, ctx, req) + if result.Requeue { + r.Log.Info("Reconcile queued") + return result, nil + } + + // Run Datapatch + if singleInstanceDatabase.Status.DatafilesPatched != "true" { + // add a blocking reconcile condition + err = errors.New("processing datapatch execution") + blocked = true + r.updateReconcileStatus(singleInstanceDatabase, ctx, &result, &err, &blocked, &completed) + result, err = r.runDatapatch(singleInstanceDatabase, readyPod, ctx, req) + if result.Requeue { + r.Log.Info("Reconcile queued") + return result, nil + } + } + + // If LoadBalancer = true , ensure Connect String is updated + if singleInstanceDatabase.Status.ConnectString == dbcommons.ValueUnavailable { + return requeueY, nil + } + + // update status to Ready after all operations succeed + singleInstanceDatabase.Status.Status = dbcommons.StatusReady + + completed = true + r.Log.Info("Reconcile completed") + return requeueN, nil +} + +//############################################################################# +// Update each reconcile condtion/status +//############################################################################# +func (r *SingleInstanceDatabaseReconciler) updateReconcileStatus(m *dbapi.SingleInstanceDatabase, ctx context.Context, + result *ctrl.Result, err *error, blocked *bool, completed *bool) { + + errMsg := func() string { + if *err != nil { + return (*err).Error() + } + return "no reconcile errors" + }() + var condition metav1.Condition + if *completed { + condition = metav1.Condition{ + Type: dbcommons.ReconcileCompelete, + LastTransitionTime: metav1.Now(), + ObservedGeneration: m.GetGeneration(), + Reason: dbcommons.ReconcileCompleteReason, + Message: errMsg, + Status: metav1.ConditionTrue, + } + } else if *blocked { + condition = metav1.Condition{ + Type: dbcommons.ReconcileBlocked, + LastTransitionTime: metav1.Now(), + ObservedGeneration: m.GetGeneration(), + Reason: dbcommons.ReconcileBlockedReason, + Message: errMsg, + Status: metav1.ConditionTrue, + } + } else if result.Requeue { + condition = metav1.Condition{ + Type: dbcommons.ReconcileQueued, + LastTransitionTime: metav1.Now(), + ObservedGeneration: m.GetGeneration(), + Reason: dbcommons.ReconcileQueuedReason, + Message: errMsg, + Status: metav1.ConditionTrue, + } + } else if *err != nil { + condition = metav1.Condition{ + Type: dbcommons.ReconcileError, + LastTransitionTime: metav1.Now(), + ObservedGeneration: m.GetGeneration(), + Reason: dbcommons.ReconcileErrorReason, + Message: errMsg, + Status: metav1.ConditionTrue, + } + } else { + return + } + if len(m.Status.Conditions) > 0 { + meta.RemoveStatusCondition(&m.Status.Conditions, condition.Type) + } + meta.SetStatusCondition(&m.Status.Conditions, condition) + // Always refresh status before a reconcile + r.Status().Update(ctx, m) +} + +//############################################################################# +// Validate the CRD specs +// m = SingleInstanceDatabase +// n = CloneFromDatabase +//############################################################################# +func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatabase, + n *dbapi.SingleInstanceDatabase, ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + var err error + eventReason := "Spec Error" + var eventMsgs []string + + // If Express Edition , Ensure Replicas=1 + if m.Spec.Edition == "express" && m.Spec.Replicas != 1 { + eventMsgs = append(eventMsgs, "XE supports only one replica") + } + // If Block Volume , Ensure Replicas=1 + if m.Spec.Persistence.AccessMode == "ReadWriteOnce" && m.Spec.Replicas != 1 { + eventMsgs = append(eventMsgs, "accessMode ReadWriteOnce supports only one replica") + } + if m.Status.Sid != "" && !strings.EqualFold(m.Spec.Sid, m.Status.Sid) { + eventMsgs = append(eventMsgs, "sid cannot be updated") + } + edition := m.Spec.Edition + if m.Spec.Edition == "" { + edition = "Enterprise" + } + if m.Spec.CloneFrom == "" && m.Status.Edition != "" && !strings.EqualFold(m.Status.Edition, edition) { + eventMsgs = append(eventMsgs, "edition cannot be updated") + } + if m.Status.Charset != "" && !strings.EqualFold(m.Status.Charset, m.Spec.Charset) { + eventMsgs = append(eventMsgs, "charset cannot be updated") + } + if m.Status.Pdbname != "" && !strings.EqualFold(m.Status.Pdbname, m.Spec.Pdbname) { + eventMsgs = append(eventMsgs, "pdbName cannot be updated") + } + if m.Status.CloneFrom != "" && + (m.Status.CloneFrom == dbcommons.NoCloneRef && m.Spec.CloneFrom != "" || + m.Status.CloneFrom != dbcommons.NoCloneRef && m.Status.CloneFrom != m.Spec.CloneFrom) { + eventMsgs = append(eventMsgs, "cloneFrom cannot be updated") + } + if m.Spec.Edition == "express" && m.Spec.CloneFrom != "" { + eventMsgs = append(eventMsgs, "cloning not supported for express edition") + } + if m.Status.OrdsReference != "" && m.Status.Persistence.AccessMode != "" && m.Status.Persistence != m.Spec.Persistence { + eventMsgs = append(eventMsgs, "uninstall ORDS to change Peristence") + } + if len(eventMsgs) > 0 { + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, strings.Join(eventMsgs, ",")) + r.Log.Info(strings.Join(eventMsgs, "\n")) + err = errors.New(strings.Join(eventMsgs, ",")) + return requeueN, err + } + + // Validating the secret + if m.Status.DatafilesCreated != "true" { + secret := &corev1.Secret{} + err = r.Get(ctx, types.NamespacedName{Name: m.Spec.AdminPassword.SecretName, Namespace: m.Namespace}, secret) + if err != nil { + if apierrors.IsNotFound(err) { + // Secret not found + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, err.Error()) + r.Log.Info(err.Error()) + m.Status.Status = dbcommons.StatusError + r.Status().Update(ctx, m) + return requeueY, err + } + r.Log.Error(err, "Unable to get the secret. Requeueing..") + return requeueY, err + } + } + + // update status fields + m.Status.Sid = m.Spec.Sid + m.Status.Edition = strings.Title(edition) + m.Status.Charset = m.Spec.Charset + m.Status.Pdbname = m.Spec.Pdbname + m.Status.Persistence = m.Spec.Persistence + if m.Spec.CloneFrom == "" { + m.Status.CloneFrom = dbcommons.NoCloneRef + } else { + m.Status.CloneFrom = m.Spec.CloneFrom + } + if m.Spec.CloneFrom != "" { + // Once a clone database has created , it has no link with its reference + if m.Status.DatafilesCreated == "true" { + return requeueN, nil + } + // Fetch the Clone database reference + err = r.Get(ctx, types.NamespacedName{Namespace: req.Namespace, Name: m.Spec.CloneFrom}, n) + if err != nil { + if apierrors.IsNotFound(err) { + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, err.Error()) + r.Log.Info(err.Error()) + return requeueN, err + } + return requeueY, err + } + + if n.Status.Status != dbcommons.StatusReady { + m.Status.Status = dbcommons.StatusPending + eventReason := "Source Database Pending" + eventMsg := "waiting for source database " + m.Spec.CloneFrom + " to be Ready" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + err = errors.New(eventMsg) + return requeueY, err + } + + if !n.Spec.ArchiveLog { + m.Status.Status = dbcommons.StatusPending + eventReason := "Source Database Pending" + eventMsg := "waiting for ArchiveLog to turn ON " + n.Name + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + r.Log.Info(eventMsg) + err = errors.New(eventMsg) + return requeueY, err + } + + m.Status.Edition = n.Status.Edition + + } + return requeueN, nil +} + +//############################################################################# +// Instantiate POD spec from SingleInstanceDatabase spec +//############################################################################# +func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleInstanceDatabase, n *dbapi.SingleInstanceDatabase) *corev1.Pod { + + pod := &corev1.Pod{ + TypeMeta: metav1.TypeMeta{ + Kind: "Pod", + }, + ObjectMeta: metav1.ObjectMeta{ + Name: m.Name + "-" + dbcommons.GenerateRandomString(5), + Namespace: m.Namespace, + Labels: map[string]string{ + "app": m.Name, + "version": m.Spec.Image.Version, + }, + }, + Spec: corev1.PodSpec{ + Volumes: []corev1.Volume{{ + Name: "datamount", + VolumeSource: corev1.VolumeSource{ + PersistentVolumeClaim: &corev1.PersistentVolumeClaimVolumeSource{ + ClaimName: m.Name, + ReadOnly: false, + }, + }, + }}, + InitContainers: func() []corev1.Container { + if m.Spec.Edition != "express" { + return []corev1.Container{{ + Name: "init-permissions", + Image: m.Spec.Image.PullFrom, + Command: []string{"/bin/sh", "-c", fmt.Sprintf("chown %d:%d /opt/oracle/oradata", int(dbcommons.ORACLE_UID), int(dbcommons.ORACLE_GUID))}, + SecurityContext: &corev1.SecurityContext{ + // User ID 0 means, root user + RunAsUser: func() *int64 { i := int64(0); return &i }(), + }, + VolumeMounts: []corev1.VolumeMount{{ + MountPath: "/opt/oracle/oradata", + Name: "datamount", + }}, + }, { + Name: "init-wallet", + Image: m.Spec.Image.PullFrom, + Env: []corev1.EnvVar{ + { + Name: "ORACLE_SID", + Value: strings.ToUpper(m.Spec.Sid), + }, + { + Name: "WALLET_CLI", + Value: "mkstore", + }, + { + Name: "WALLET_DIR", + Value: "/opt/oracle/oradata/dbconfig/$(ORACLE_SID)/.wallet", + }, + }, + Command: []string{"/bin/sh"}, + Args: func() []string { + edition := "" + if m.Spec.CloneFrom == "" { + edition = m.Spec.Edition + if m.Spec.Edition == "" { + edition = "enterprise" + } + } else { + edition = n.Spec.Edition + if n.Spec.Edition == "" { + edition = "enterprise" + } + } + return []string{"-c", fmt.Sprintf(dbcommons.InitWalletCMD, edition)} + }(), + SecurityContext: &corev1.SecurityContext{ + RunAsUser: func() *int64 { i := int64(dbcommons.ORACLE_UID); return &i }(), + RunAsGroup: func() *int64 { i := int64(dbcommons.ORACLE_GUID); return &i }(), + }, + VolumeMounts: []corev1.VolumeMount{{ + MountPath: "/opt/oracle/oradata", + Name: "datamount", + }}, + }} + } + return []corev1.Container{{ + Name: "init-permissions", + Image: m.Spec.Image.PullFrom, + Command: []string{"/bin/sh", "-c", fmt.Sprintf("chown %d:%d /opt/oracle/oradata", int(dbcommons.ORACLE_UID), int(dbcommons.ORACLE_GUID))}, + SecurityContext: &corev1.SecurityContext{ + // User ID 0 means, root user + RunAsUser: func() *int64 { i := int64(0); return &i }(), + }, + VolumeMounts: []corev1.VolumeMount{{ + MountPath: "/opt/oracle/oradata", + Name: "datamount", + }}, + }} + }(), + Containers: []corev1.Container{{ + Name: m.Name, + Image: m.Spec.Image.PullFrom, + Lifecycle: &corev1.Lifecycle{ + PreStop: &corev1.Handler{ + Exec: &corev1.ExecAction{ + Command: []string{"/bin/sh", "-c", "/bin/echo -en 'shutdown abort;\n' | env ORACLE_SID=${ORACLE_SID^^} sqlplus -S / as sysdba"}, + }, + }, + }, + ImagePullPolicy: corev1.PullAlways, + Ports: []corev1.ContainerPort{{ContainerPort: 1521}, {ContainerPort: 5500}}, + + ReadinessProbe: &corev1.Probe{ + Handler: corev1.Handler{ + Exec: &corev1.ExecAction{ + Command: []string{"/bin/sh", "-c", "if [ -f $ORACLE_BASE/checkDBLockStatus.sh ]; then $ORACLE_BASE/checkDBLockStatus.sh ; else $ORACLE_BASE/checkDBStatus.sh; fi "}, + }, + }, + InitialDelaySeconds: 20, + TimeoutSeconds: 20, + PeriodSeconds: func() int32 { + if m.Spec.ReadinessCheckPeriod > 0 { + return int32(m.Spec.ReadinessCheckPeriod) + } + return 30 + }(), + }, + + VolumeMounts: []corev1.VolumeMount{{ + MountPath: "/opt/oracle/oradata", + Name: "datamount", + }}, + Env: func() []corev1.EnvVar { + if m.Spec.CloneFrom == "" { + return []corev1.EnvVar{ + { + Name: "SVC_HOST", + Value: m.Name, + }, + { + Name: "SVC_PORT", + Value: "1521", + }, + { + Name: "CREATE_PDB", + Value: func() string { + if m.Spec.Pdbname != "" { + return "true" + } + return "false" + }(), + }, + { + Name: "ORACLE_SID", + Value: strings.ToUpper(m.Spec.Sid), + }, + { + Name: "WALLET_DIR", + Value: "/opt/oracle/oradata/dbconfig/$(ORACLE_SID)/.wallet", + }, + { + Name: "ORACLE_PDB", + Value: m.Spec.Pdbname, + }, + { + Name: "ORACLE_CHARACTERSET", + Value: m.Spec.Charset, + }, + { + Name: "ORACLE_EDITION", + Value: m.Spec.Edition, + }, + { + Name: "INIT_SGA_SIZE", + Value: func() string { + if m.Spec.InitParams.SgaTarget > 0 && m.Spec.InitParams.PgaAggregateTarget > 0 { + return strconv.Itoa(m.Spec.InitParams.SgaTarget) + } + return "" + }(), + }, + { + Name: "INIT_PGA_SIZE", + Value: func() string { + if m.Spec.InitParams.SgaTarget > 0 && m.Spec.InitParams.PgaAggregateTarget > 0 { + return strconv.Itoa(m.Spec.InitParams.SgaTarget) + } + return "" + }(), + }, + { + Name: "SKIP_DATAPATCH", + Value: "true", + }, + } + } + return []corev1.EnvVar{ + { + Name: "SVC_HOST", + Value: m.Name, + }, + { + Name: "SVC_PORT", + Value: "1521", + }, + { + Name: "ORACLE_SID", + Value: strings.ToUpper(m.Spec.Sid), + }, + { + Name: "WALLET_DIR", + Value: "/opt/oracle/oradata/dbconfig/$(ORACLE_SID)/.wallet", + }, + { + Name: "PRIMARY_DB_CONN_STR", + Value: n.Name + ":1521/" + n.Spec.Sid, + }, + { + Name: "PRIMARY_SID", + Value: strings.ToUpper(n.Spec.Sid), + }, + { + Name: "PRIMARY_NAME", + Value: n.Name, + }, + { + Name: "ORACLE_HOSTNAME", + ValueFrom: &corev1.EnvVarSource{ + FieldRef: &corev1.ObjectFieldSelector{ + FieldPath: "status.podIP", + }, + }, + }, + { + Name: "CLONE_DB", + Value: "true", + }, + { + Name: "SKIP_DATAPATCH", + Value: "true", + }, + } + }(), + }}, + + TerminationGracePeriodSeconds: func() *int64 { i := int64(30); return &i }(), + + NodeSelector: func() map[string]string { + ns := make(map[string]string) + if len(m.Spec.NodeSelector) != 0 { + for key, value := range m.Spec.NodeSelector { + ns[key] = value + } + } + return ns + }(), + + SecurityContext: &corev1.PodSecurityContext{ + RunAsUser: func() *int64 { + i := int64(0) + if m.Spec.Edition != "express" { + i = int64(dbcommons.ORACLE_UID) + } + return &i + }(), + RunAsGroup: func() *int64 { + i := int64(0) + if m.Spec.Edition != "express" { + i = int64(dbcommons.ORACLE_GUID) + } + return &i + }(), + }, + ImagePullSecrets: []corev1.LocalObjectReference{ + { + Name: m.Spec.Image.PullSecrets, + }, + }, + }, + } + + // Set SingleInstanceDatabase instance as the owner and controller + ctrl.SetControllerReference(m, pod, r.Scheme) + return pod +} + +//############################################################################# +// Instantiate Service spec from SingleInstanceDatabase spec +//############################################################################# +func (r *SingleInstanceDatabaseReconciler) instantiateSVCSpec(m *dbapi.SingleInstanceDatabase) *corev1.Service { + svc := &corev1.Service{ + TypeMeta: metav1.TypeMeta{ + Kind: "Service", + }, + ObjectMeta: metav1.ObjectMeta{ + Name: m.Name, + Namespace: m.Namespace, + Labels: map[string]string{ + "app": m.Name, + }, + }, + Spec: corev1.ServiceSpec{ + Ports: []corev1.ServicePort{ + { + Name: "listener", + Port: 1521, + Protocol: corev1.ProtocolTCP, + }, + { + Name: "xmldb", + Port: 5500, + Protocol: corev1.ProtocolTCP, + }, + }, + Selector: map[string]string{ + "app": m.Name, + }, + Type: corev1.ServiceType(func() string { + if m.Spec.LoadBalancer { + return "LoadBalancer" + } + return "NodePort" + }()), + }, + } + // Set SingleInstanceDatabase instance as the owner and controller + ctrl.SetControllerReference(m, svc, r.Scheme) + return svc +} + +//############################################################################# +// Instantiate Persistent Volume Claim spec from SingleInstanceDatabase spec +//############################################################################# +func (r *SingleInstanceDatabaseReconciler) instantiatePVCSpec(m *dbapi.SingleInstanceDatabase) *corev1.PersistentVolumeClaim { + + pvc := &corev1.PersistentVolumeClaim{ + TypeMeta: metav1.TypeMeta{ + Kind: "PersistentVolumeClaim", + }, + ObjectMeta: metav1.ObjectMeta{ + Name: m.Name, + Namespace: m.Namespace, + Labels: map[string]string{ + "app": m.Name, + }, + }, + Spec: corev1.PersistentVolumeClaimSpec{ + AccessModes: func() []corev1.PersistentVolumeAccessMode { + var accessMode []corev1.PersistentVolumeAccessMode + accessMode = append(accessMode, corev1.PersistentVolumeAccessMode(m.Spec.Persistence.AccessMode)) + return accessMode + }(), + Resources: corev1.ResourceRequirements{ + Requests: map[corev1.ResourceName]resource.Quantity{ + // Requests describes the minimum amount of compute resources required + "storage": resource.MustParse(m.Spec.Persistence.Size), + }, + }, + StorageClassName: &m.Spec.Persistence.StorageClass, + }, + } + // Set SingleInstanceDatabase instance as the owner and controller + ctrl.SetControllerReference(m, pvc, r.Scheme) + return pvc +} + +//############################################################################# +// Stake a claim for Persistent Volume +//############################################################################# +func (r *SingleInstanceDatabaseReconciler) createOrReplacePVC(ctx context.Context, req ctrl.Request, + m *dbapi.SingleInstanceDatabase) (ctrl.Result, error) { + + log := r.Log.WithValues("createPVC", req.NamespacedName) + + pvcDeleted := false + // Check if the PVC already exists using r.Get, if not create a new one using r.Create + pvc := &corev1.PersistentVolumeClaim{} + // Get retrieves an obj ( a struct pointer ) for the given object key from the Kubernetes Cluster. + err := r.Get(ctx, types.NamespacedName{Name: m.Name, Namespace: m.Namespace}, pvc) + if err == nil { + if *pvc.Spec.StorageClassName != m.Spec.Persistence.StorageClass || + pvc.Spec.Resources.Requests["storage"] != resource.MustParse(m.Spec.Persistence.Size) || + pvc.Spec.AccessModes[0] != corev1.PersistentVolumeAccessMode(m.Spec.Persistence.AccessMode) { + // call deletePods() with zero pods in avaiable and nil readyPod to delete all pods + result, err := r.deletePods(ctx, req, m, []corev1.Pod{}, corev1.Pod{}, 0, 0) + if result.Requeue { + return result, err + } + + log.Info("Deleting PVC", " name ", pvc.Name) + err = r.Delete(ctx, pvc) + if err != nil { + r.Log.Error(err, "Failed to delete Pvc", "Pvc.Name", pvc.Name) + return requeueN, err + } + pvcDeleted = true + } else { + log.Info("Found Existing PVC", "Name", pvc.Name) + return requeueN, nil + } + } + if pvcDeleted || err != nil && apierrors.IsNotFound(err) { + // Define a new PVC + pvc = r.instantiatePVCSpec(m) + log.Info("Creating a new PVC", "PVC.Namespace", pvc.Namespace, "PVC.Name", pvc.Name) + err = r.Create(ctx, pvc) + if err != nil { + log.Error(err, "Failed to create new PVC", "PVC.Namespace", pvc.Namespace, "PVC.Name", pvc.Name) + return requeueY, err + } + return requeueN, nil + } else if err != nil { + log.Error(err, "Failed to get PVC") + return requeueY, err + } + + return requeueN, nil +} + +//############################################################################# +// Create a Service for SingleInstanceDatabase +//############################################################################# +func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Context, req ctrl.Request, + m *dbapi.SingleInstanceDatabase) (ctrl.Result, error) { + + log := r.Log.WithValues("createOrReplaceSVC", req.NamespacedName) + + svcDeleted := false + // Check if the Service already exists, if not create a new one + svc := &corev1.Service{} + // Get retrieves an obj ( a struct pointer ) for the given object key from the Kubernetes Cluster. + err := r.Get(ctx, types.NamespacedName{Name: m.Name, Namespace: m.Namespace}, svc) + if err == nil { + svcType := corev1.ServiceType("NodePort") + if m.Spec.LoadBalancer { + svcType = corev1.ServiceType("LoadBalancer") + } + + if svc.Spec.Type != svcType { + log.Info("Deleting SVC", " name ", svc.Name) + err = r.Delete(ctx, svc) + if err != nil { + r.Log.Error(err, "Failed to delete svc", " Name", svc.Name) + return requeueN, err + } + svcDeleted = true + } + } + if svcDeleted || err != nil && apierrors.IsNotFound(err) { + // Define a new Service + svc = r.instantiateSVCSpec(m) + log.Info("Creating a new Service", "Service.Namespace", svc.Namespace, "Service.Name", svc.Name) + err = r.Create(ctx, svc) + if err != nil { + log.Error(err, "Failed to create new Service", "Service.Namespace", svc.Namespace, "Service.Name", svc.Name) + return requeueY, err + } + } else if err != nil { + log.Error(err, "Failed to get Service") + return requeueY, err + } + log.Info("Found Existing Service ", "Service Name ", svc.Name) + + m.Status.ConnectString = dbcommons.ValueUnavailable + m.Status.PdbConnectString = dbcommons.ValueUnavailable + m.Status.OemExpressUrl = dbcommons.ValueUnavailable + pdbName := "ORCLPDB1" + if m.Spec.Pdbname != "" { + pdbName = strings.ToUpper(m.Spec.Pdbname) + } + if m.Spec.LoadBalancer { + m.Status.ClusterConnectString = svc.Name + "." + svc.Namespace + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/" + strings.ToUpper(m.Spec.Sid) + if len(svc.Status.LoadBalancer.Ingress) > 0 { + m.Status.ConnectString = svc.Status.LoadBalancer.Ingress[0].IP + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/" + strings.ToUpper(m.Spec.Sid) + m.Status.PdbConnectString = svc.Status.LoadBalancer.Ingress[0].IP + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/" + strings.ToUpper(pdbName) + m.Status.OemExpressUrl = "https://" + svc.Status.LoadBalancer.Ingress[0].IP + ":" + fmt.Sprint(svc.Spec.Ports[1].Port) + "/em" + } + return requeueN, nil + } + + m.Status.ClusterConnectString = svc.Name + "." + svc.Namespace + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/" + strings.ToUpper(m.Spec.Sid) + nodeip := dbcommons.GetNodeIp(r, ctx, req) + if nodeip != "" { + m.Status.ConnectString = nodeip + ":" + fmt.Sprint(svc.Spec.Ports[0].NodePort) + "/" + strings.ToUpper(m.Spec.Sid) + m.Status.PdbConnectString = nodeip + ":" + fmt.Sprint(svc.Spec.Ports[0].NodePort) + "/" + strings.ToUpper(pdbName) + m.Status.OemExpressUrl = "https://" + nodeip + ":" + fmt.Sprint(svc.Spec.Ports[1].NodePort) + "/em" + } + + return requeueN, nil +} + +//############################################################################# +// Create new Pods or delete old/extra pods +// m = SingleInstanceDatabase +// n = CloneFromDatabase +//############################################################################# +func (r *SingleInstanceDatabaseReconciler) createOrReplacePods(m *dbapi.SingleInstanceDatabase, n *dbapi.SingleInstanceDatabase, + ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + log := r.Log.WithValues("createOrReplacePods", req.NamespacedName) + + oldVersion := "" + oldImage := "" + + // call FindPods() to fetch pods all version/images of the same SIDB kind + readyPod, replicasFound, available, podsMarkedToBeDeleted, err := dbcommons.FindPods(r, "", "", m.Name, m.Namespace, ctx, req) + if err != nil { + log.Error(err, err.Error()) + return requeueY, err + } + if m.Spec.Edition == "express" && podsMarkedToBeDeleted > 0 { + // Recreate new pods only after earlier pods are terminated completely + return requeueY, err + } + if readyPod.Name != "" { + available = append(available, readyPod) + } + + for _, pod := range available { + if pod.Labels["version"] != m.Spec.Image.Version { + oldVersion = pod.Labels["version"] + } + if pod.Spec.Containers[0].Image != m.Spec.Image.PullFrom { + oldImage = pod.Spec.Containers[0].Image + } + + } + + // podVersion, podImage if old version PODs are found + imageChanged := oldVersion != "" || oldImage != "" + + if !imageChanged { + eventReason := "" + eventMsg := "" + if replicasFound == m.Spec.Replicas { + return requeueN, nil + } + if replicasFound < m.Spec.Replicas { + if replicasFound != 0 { + eventReason = "Scaling Out" + eventMsg = "from " + strconv.Itoa(replicasFound) + " pods to " + strconv.Itoa(m.Spec.Replicas) + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + } + // If version is same , call createPods() with the same version , and no of Replicas required + return r.createPods(m, n, ctx, req, replicasFound) + } + eventReason = "Scaling In" + eventMsg = "from " + strconv.Itoa(replicasFound) + " pods to " + strconv.Itoa(m.Spec.Replicas) + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + // Delete extra PODs + return r.deletePods(ctx, req, m, available, readyPod, replicasFound, m.Spec.Replicas) + } + + // Version/Image changed + // PATCHING START (Only Software Patch) + // call FindPods() to find pods of newer version . if running , delete the older version replicas. + readyPod, replicasFound, available, _, err = dbcommons.FindPods(r, m.Spec.Image.Version, + m.Spec.Image.PullFrom, m.Name, m.Namespace, ctx, req) + if err != nil { + log.Error(err, err.Error()) + return requeueY, nil + } + + // create new Pods with the new Version and no.of Replicas required + result, err := r.createPods(m, n, ctx, req, replicasFound) + if result.Requeue { + return result, err + } + + // Findpods() only returns non ready pods + if readyPod.Name != "" { + log.Info("New ready pod found", "name", readyPod.Name) + available = append(available, readyPod) + } + if ok, _ := dbcommons.IsAnyPodWithStatus(available, corev1.PodRunning); !ok { + eventReason := "Database Pending" + eventMsg := "waiting for newer version/image DB pods get to running state" + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) + log.Info(eventMsg) + return requeueY, errors.New(eventMsg) + } + + // call FindPods() to find pods of older version . delete all the Pods + readyPod, replicasFound, available, _, err = dbcommons.FindPods(r, oldVersion, + oldImage, m.Name, m.Namespace, ctx, req) + if err != nil { + log.Error(err, err.Error()) + return requeueY, err + } + if readyPod.Name != "" { + log.Info("Ready pod marked for deletion", "name", readyPod.Name) + available = append(available, readyPod) + } + return r.deletePods(ctx, req, m, available, corev1.Pod{}, replicasFound, 0) + // PATCHING END +} + +//############################################################################# +// Function for creating Oracle Wallet +//############################################################################# +func (r *SingleInstanceDatabaseReconciler) createWallet(m *dbapi.SingleInstanceDatabase, ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + + // Wallet not supported for XE Database + if m.Spec.Edition == "express" { + return requeueN, nil + } + + // Listing all the pods + readyPod, _, availableFinal, _, err := dbcommons.FindPods(r, m.Spec.Image.Version, + m.Spec.Image.PullFrom, m.Name, m.Namespace, ctx, req) + if err != nil { + r.Log.Error(err, err.Error()) + return requeueY, nil + } + if readyPod.Name != "" { + return requeueN, nil + } + + // Wallet is created in persistent volume, hence it only needs to be executed once for all number of pods + if len(availableFinal) == 0 { + r.Log.Info("Pods are being created, currently no pods available") + return requeueY, nil + } + + // Iterate through the avaialableFinal (list of pods) to find out the pod whose status is updated about the init containers + // If no required pod found then requeue the reconcile request + var pod corev1.Pod + var podFound bool + for _, pod = range availableFinal { + // Check if pod status contianer is updated about init containers + if len(pod.Status.InitContainerStatuses) > 0 { + podFound = true + break + } + } + if !podFound { + r.Log.Info("No pod has its status updated about init containers. Requeueing...") + return requeueY, nil + } + + lastInitContIndex := len(pod.Status.InitContainerStatuses) - 1 + + // If InitContainerStatuses[].Ready is true, it means that the init container is successful + if pod.Status.InitContainerStatuses[lastInitContIndex].Ready { + // Init container named "init-wallet" has completed it's execution, hence return and don't requeue + return requeueN, nil + } + + if pod.Status.InitContainerStatuses[lastInitContIndex].State.Running == nil { + // Init container named "init-wallet" is not running, so waiting for it to come in running state requeueing the reconcile request + r.Log.Info("Waiting for init-wallet to come in running state...") + return requeueY, nil + } + + if m.Spec.CloneFrom == "" && m.Spec.Edition != "express" { + //Check if Edition of m.Spec.Sid is same as m.Spec.Edition + getEditionFile := dbcommons.GetEnterpriseEditionFileCMD + eventReason := m.Spec.Sid + " is a enterprise edition" + if m.Spec.Edition == "enterprise" || m.Spec.Edition == "" { + getEditionFile = dbcommons.GetStandardEditionFileCMD + eventReason = m.Spec.Sid + " is a standard edition" + } + out, err := dbcommons.ExecCommand(r, r.Config, pod.Name, pod.Namespace, "init-wallet", + ctx, req, false, "bash", "-c", getEditionFile) + r.Log.Info("getEditionFile Output : \n" + out) + + if err == nil && out != "" { + m.Status.Status = dbcommons.StatusError + eventMsg := "wrong edition" + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) + return requeueN, errors.New("wrong Edition") + } + } + + r.Log.Info("Creating Wallet...") + + // Querying the secret + r.Log.Info("Querying the database secret ...") + secret := &corev1.Secret{} + err = r.Get(ctx, types.NamespacedName{Name: m.Spec.AdminPassword.SecretName, Namespace: m.Namespace}, secret) + if err != nil { + if apierrors.IsNotFound(err) { + r.Log.Info("Secret not found") + m.Status.Status = dbcommons.StatusError + r.Status().Update(ctx, m) + return requeueY, nil + } + r.Log.Error(err, "Unable to get the secret. Requeueing..") + return requeueY, nil + } + + // Execing into the pods and creating the wallet + adminPassword := string(secret.Data[m.Spec.AdminPassword.SecretKey]) + + out, err := dbcommons.ExecCommand(r, r.Config, pod.Name, pod.Namespace, "init-wallet", + ctx, req, true, "bash", "-c", fmt.Sprintf("%s && %s && %s", + dbcommons.WalletPwdCMD, + dbcommons.WalletCreateCMD, + fmt.Sprintf(dbcommons.WalletEntriesCMD, adminPassword))) + if err != nil { + r.Log.Error(err, err.Error()) + return requeueY, nil + } + r.Log.Info("Creating wallet entry Output : \n" + out) + + return requeueN, nil +} + +//############################################################################# +// Create the requested POD replicas +// m = SingleInstanceDatabase +// n = CloneFromDatabase +//############################################################################# +func (r *SingleInstanceDatabaseReconciler) createPods(m *dbapi.SingleInstanceDatabase, n *dbapi.SingleInstanceDatabase, + ctx context.Context, req ctrl.Request, replicasFound int) (ctrl.Result, error) { + + log := r.Log.WithValues("createPods", req.NamespacedName) + + replicasReq := m.Spec.Replicas + log.Info("Replica Info", "Found", replicasFound, "Required", replicasReq) + if replicasFound == replicasReq { + log.Info("No of " + m.Name + " replicas found are same as required") + return requeueN, nil + } + if replicasFound == 0 { + m.Status.Status = dbcommons.StatusPending + m.Status.DatafilesCreated = "false" + m.Status.DatafilesPatched = "false" + m.Status.Role = dbcommons.ValueUnavailable + m.Status.ConnectString = dbcommons.ValueUnavailable + m.Status.PdbConnectString = dbcommons.ValueUnavailable + m.Status.OemExpressUrl = dbcommons.ValueUnavailable + m.Status.ReleaseUpdate = dbcommons.ValueUnavailable + } + // if Found < Required , Create New Pods , Name of Pods are generated Randomly + for i := replicasFound; i < replicasReq; i++ { + pod := r.instantiatePodSpec(m, n) + log.Info("Creating a new "+m.Name+" POD", "POD.Namespace", pod.Namespace, "POD.Name", pod.Name) + err := r.Create(ctx, pod) + if err != nil { + log.Error(err, "Failed to create new "+m.Name+" POD", "pod.Namespace", pod.Namespace, "POD.Name", pod.Name) + return requeueY, err + } + } + + readyPod, _, availableFinal, _, err := dbcommons.FindPods(r, m.Spec.Image.Version, + m.Spec.Image.PullFrom, m.Name, m.Namespace, ctx, req) + if err != nil { + log.Error(err, err.Error()) + return requeueY, err + } + if readyPod.Name != "" { + availableFinal = append(availableFinal, readyPod) + } + + m.Status.Replicas = m.Spec.Replicas + + podNamesFinal := dbcommons.GetPodNames(availableFinal) + log.Info("Final "+m.Name+" Pods After Deleting (or) Adding Extra Pods ( Including The Ready Pod ) ", "Pod Names", podNamesFinal) + log.Info(m.Name+" Replicas Available", "Count", len(podNamesFinal)) + log.Info(m.Name+" Replicas Required", "Count", replicasReq) + + return requeueN, nil +} + +//############################################################################# +// Create the requested POD replicas +// m = SingleInstanceDatabase +// n = CloneFromDatabase +//############################################################################# +func (r *SingleInstanceDatabaseReconciler) deletePods(ctx context.Context, req ctrl.Request, m *dbapi.SingleInstanceDatabase, available []corev1.Pod, + readyPod corev1.Pod, replicasFound int, replicasRequired int) (ctrl.Result, error) { + log := r.Log.WithValues("deletePods", req.NamespacedName) + + var err error + if len(available) == 0 { + // As list of pods not avaiable . fetch them ( len(available) == 0 ; Usecase where deletion of all pods required ) + var readyPodToBeDeleted corev1.Pod + readyPodToBeDeleted, replicasFound, available, _, err = dbcommons.FindPods(r, "", + "", m.Name, m.Namespace, ctx, req) + if err != nil { + log.Error(err, err.Error()) + return requeueY, err + } + // Append readyPod to avaiable for deleting all pods + if readyPodToBeDeleted.Name != "" { + available = append(available, readyPodToBeDeleted) + } + } + + // For deleting all pods , call with readyPod as nil ( corev1.Pod{} ) and append readyPod to avaiable while calling deletePods() + // if Found > Required , Delete Extra Pods + if replicasFound > len(available) { + // if available does not contain readyPOD, add it + available = append(available, readyPod) + } + + noDeleted := 0 + for _, availablePod := range available { + if readyPod.Name == availablePod.Name { + continue + } + if replicasRequired == (len(available) - noDeleted) { + break + } + r.Log.Info("Deleting Pod : ", "POD.NAME", availablePod.Name) + err := r.Delete(ctx, &availablePod, &client.DeleteOptions{}) + noDeleted += 1 + if err != nil { + r.Log.Error(err, "Failed to delete existing POD", "POD.Name", availablePod.Name) + // Don't requeue + } + } + + m.Status.Replicas = m.Spec.Replicas + + return requeueN, nil +} + +//############################################################################# +// ValidateDBReadiness and return the ready POD +//############################################################################# +func (r *SingleInstanceDatabaseReconciler) validateDBReadiness(m *dbapi.SingleInstanceDatabase, + ctx context.Context, req ctrl.Request) (ctrl.Result, corev1.Pod, error) { + + readyPod, _, available, _, err := dbcommons.FindPods(r, m.Spec.Image.Version, + m.Spec.Image.PullFrom, m.Name, m.Namespace, ctx, req) + if err != nil { + r.Log.Error(err, err.Error()) + return requeueY, readyPod, err + } + if readyPod.Name == "" { + eventReason := "Database Pending" + eventMsg := "waiting for database pod to be ready" + m.Status.Status = dbcommons.StatusPending + if ok, _ := dbcommons.IsAnyPodWithStatus(available, corev1.PodFailed); ok { + eventReason = "Database Failed" + eventMsg = "pod creation failed" + } else if ok, runningPod := dbcommons.IsAnyPodWithStatus(available, corev1.PodRunning); ok { + eventReason = "Database Creating" + eventMsg = "waiting for database to be ready" + m.Status.Status = dbcommons.StatusCreating + if m.Spec.Edition == "express" { + eventReason = "Database Unhealthy" + m.Status.Status = dbcommons.StatusNotReady + } + out, err := dbcommons.ExecCommand(r, r.Config, runningPod.Name, runningPod.Namespace, "", + ctx, req, false, "bash", "-c", dbcommons.GetCheckpointFileCMD) + if err != nil { + r.Log.Error(err, err.Error()) + return requeueY, readyPod, err + } + r.Log.Info("GetCheckpointFileCMD Output : \n" + out) + + if out != "" { + eventReason = "Database Unhealthy" + eventMsg = "datafiles exists" + m.Status.DatafilesCreated = "true" + m.Status.Status = dbcommons.StatusNotReady + } + + } + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + r.Log.Info(eventMsg) + // As No pod is ready now , turn on mode when pod is ready . so requeue the request + return requeueY, readyPod, errors.New(eventMsg) + } + if m.Status.DatafilesPatched != "true" { + eventReason := "Datapatch Pending" + eventMsg := "datapatch execution pending" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + } + available = append(available, readyPod) + podNamesFinal := dbcommons.GetPodNames(available) + r.Log.Info("Final "+m.Name+" Pods After Deleting (or) Adding Extra Pods ( Including The Ready Pod ) ", "Pod Names", podNamesFinal) + r.Log.Info(m.Name+" Replicas Available", "Count", len(podNamesFinal)) + r.Log.Info(m.Name+" Replicas Required", "Count", m.Spec.Replicas) + + eventReason := "Database Ready" + eventMsg := "database open on pod " + readyPod.Name + " scheduled on node " + readyPod.Status.HostIP + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + + m.Status.DatafilesCreated = "true" + + // DB is ready, fetch and update other info + out, err := dbcommons.GetDatabaseRole(readyPod, r, r.Config, ctx, req, m.Spec.Edition) + if err == nil { + m.Status.Role = strings.ToUpper(out) + } + version, out, err := dbcommons.GetDatabaseVersion(readyPod, r, r.Config, ctx, req, m.Spec.Edition) + if err == nil { + if !strings.Contains(out, "ORA-") && m.Status.DatafilesPatched != "true" { + m.Status.ReleaseUpdate = version + } + } + + if m.Spec.Edition == "express" { + //Configure OEM Express Listener + out, err = dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, + "bash", "-c", fmt.Sprintf("echo -e \"%s\" | su -p oracle -c \"sqlplus -s / as sysdba\" ", dbcommons.ConfigureOEMSQL)) + if err != nil { + r.Log.Error(err, err.Error()) + return requeueY, readyPod, err + } + r.Log.Info("ConfigureOEMSQL output") + r.Log.Info(out) + } + + return requeueN, readyPod, nil + +} + +//############################################################################# +// Function for deleting the Oracle Wallet +//############################################################################# +func (r *SingleInstanceDatabaseReconciler) deleteWallet(m *dbapi.SingleInstanceDatabase, ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + + // Wallet not supported for XE Database + if m.Spec.Edition == "express" { + return requeueN, nil + } + + // Deleting the secret and then deleting the wallet + // If the secret is not found it means that the secret and wallet both are deleted, hence no need to requeue + if !m.Spec.AdminPassword.KeepSecret { + r.Log.Info("Querying the database secret ...") + secret := &corev1.Secret{} + err := r.Get(ctx, types.NamespacedName{Name: m.Spec.AdminPassword.SecretName, Namespace: m.Namespace}, secret) + if err == nil { + err := r.Delete(ctx, secret) + if err == nil { + r.Log.Info("Deleted the secret : " + m.Spec.AdminPassword.SecretName) + } + } + } + + // Getting the ready pod for the database + readyPod, _, _, _, err := dbcommons.FindPods(r, m.Spec.Image.Version, + m.Spec.Image.PullFrom, m.Name, m.Namespace, ctx, req) + if err != nil { + r.Log.Error(err, err.Error()) + return requeueY, err + } + + // Deleting the wallet + _, err = dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", + ctx, req, false, "bash", "-c", dbcommons.WalletDeleteCMD) + if err != nil { + r.Log.Error(err, err.Error()) + return requeueY, nil + } + r.Log.Info("Wallet Deleted !!") + return requeueN, nil +} + +//############################################################################# +// Execute Datapatch +//############################################################################# +func (r *SingleInstanceDatabaseReconciler) runDatapatch(m *dbapi.SingleInstanceDatabase, + readyPod corev1.Pod, ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + + // Datapatch not supported for XE Database + if m.Spec.Edition == "express" { + return requeueN, nil + } + + m.Status.Status = dbcommons.StatusPatching + r.Status().Update(ctx, m) + eventReason := "Datapatch Executing" + eventMsg := "datapatch begin execution" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + + //RUN DATAPATCH + out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", + ctx, req, false, "bash", "-c", dbcommons.RunDatapatchCMD) + if err != nil { + r.Log.Error(err, err.Error()) + return requeueY, err + } + r.Log.Info("Datapatch output") + r.Log.Info(out) + + // Get Sqlpatch Description + out, err = dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | sqlplus -s / as sysdba ", dbcommons.GetSqlpatchDescriptionSQL)) + if err == nil { + r.Log.Info("GetSqlpatchDescriptionSQL Output") + r.Log.Info(out) + SqlpatchDescriptions, _ := dbcommons.StringToLines(out) + if len(SqlpatchDescriptions) > 0 { + m.Status.ReleaseUpdate = SqlpatchDescriptions[0] + } + } + + eventReason = "Datapatch Done" + if strings.Contains(out, "Datapatch execution has failed.") { + eventMsg = "datapatch execution failed" + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) + return requeueN, errors.New(eventMsg) + } + + m.Status.DatafilesPatched = "true" + status, versionFrom, versionTo, _ := dbcommons.GetSqlpatchStatus(r, r.Config, readyPod, ctx, req) + eventMsg = "data files patched from " + versionFrom + " to " + versionTo + " : " + status + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + + return requeueN, nil +} + +//############################################################################# +// Update Init Parameters +//############################################################################# +func (r *SingleInstanceDatabaseReconciler) updateInitParameters(m *dbapi.SingleInstanceDatabase, + readyPod corev1.Pod, ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + log := r.Log.WithValues("updateInitParameters", req.NamespacedName) + + if m.Status.InitParams == m.Spec.InitParams { + return requeueN, nil + } + + out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", + ctx, req, false, "bash", "-c", fmt.Sprintf(dbcommons.AlterSgaPgaCpuCMD, m.Spec.InitParams.SgaTarget, + m.Spec.InitParams.PgaAggregateTarget, m.Spec.InitParams.CpuCount, dbcommons.GetSqlClient(m.Spec.Edition))) + if err != nil { + log.Error(err, err.Error()) + return requeueY, err + } + log.Info("AlterSgaPgaCpuCMD Output:" + out) + + if m.Status.InitParams.Processes != m.Spec.InitParams.Processes { + // Altering 'Processes' needs database to be restarted + out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", + ctx, req, false, "bash", "-c", fmt.Sprintf(dbcommons.AlterProcessesCMD, m.Spec.InitParams.Processes, dbcommons.GetSqlClient(m.Spec.Edition), + dbcommons.GetSqlClient(m.Spec.Edition))) + if err != nil { + log.Error(err, err.Error()) + return requeueY, err + } + log.Info("AlterProcessesCMD Output:" + out) + } + + out, err = dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", + ctx, req, false, "bash", "-c", fmt.Sprintf(dbcommons.GetInitParamsSQL, dbcommons.GetSqlClient(m.Spec.Edition))) + if err != nil { + log.Error(err, err.Error()) + return requeueY, err + } + log.Info("GetInitParamsSQL Output:" + out) + + m.Status.InitParams = m.Spec.InitParams + return requeueN, nil +} + +//############################################################################# +// Update DB config params like FLASHBACK , FORCELOGGING , ARCHIVELOG +//############################################################################# +func (r *SingleInstanceDatabaseReconciler) updateDBConfig(m *dbapi.SingleInstanceDatabase, + readyPod corev1.Pod, ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + + log := r.Log.WithValues("updateDBConfig", req.NamespacedName) + + m.Status.Status = dbcommons.StatusUpdating + r.Status().Update(ctx, m) + var forceLoggingStatus bool + var flashBackStatus bool + var archiveLogStatus bool + var changeArchiveLog bool // True if switching ArchiveLog mode change is needed + + //################################################################################################# + // CHECK FLASHBACK , ARCHIVELOG , FORCELOGGING + //################################################################################################# + + flashBackStatus, archiveLogStatus, forceLoggingStatus, result := dbcommons.CheckDBConfig(readyPod, r, r.Config, ctx, req, m.Spec.Edition) + if result.Requeue { + return result, nil + } + m.Status.ArchiveLog = strconv.FormatBool(archiveLogStatus) + m.Status.ForceLogging = strconv.FormatBool(forceLoggingStatus) + m.Status.FlashBack = strconv.FormatBool(flashBackStatus) + + log.Info("Flashback", "Status :", flashBackStatus) + log.Info("ArchiveLog", "Status :", archiveLogStatus) + log.Info("ForceLog", "Status :", forceLoggingStatus) + + //################################################################################################# + // TURNING FLASHBACK , ARCHIVELOG , FORCELOGGING TO TRUE + //################################################################################################# + + if m.Spec.ArchiveLog && !archiveLogStatus { + + out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", + ctx, req, false, "bash", "-c", dbcommons.CreateDBRecoveryDestCMD) + if err != nil { + log.Error(err, err.Error()) + return requeueY, err + } + log.Info("CreateDbRecoveryDest Output") + log.Info(out) + + out, err = dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | %s", dbcommons.SetDBRecoveryDestSQL, dbcommons.GetSqlClient(m.Spec.Edition))) + if err != nil { + log.Error(err, err.Error()) + return requeueY, err + } + log.Info("SetDbRecoveryDest Output") + log.Info(out) + + out, err = dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf(dbcommons.ArchiveLogTrueCMD, dbcommons.GetSqlClient(m.Spec.Edition))) + if err != nil { + log.Error(err, err.Error()) + return requeueY, err + } + log.Info("ArchiveLogTrue Output") + log.Info(out) + + } + + if m.Spec.ForceLogging && !forceLoggingStatus { + out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | %s", dbcommons.ForceLoggingTrueSQL, dbcommons.GetSqlClient(m.Spec.Edition))) + if err != nil { + log.Error(err, err.Error()) + return requeueY, err + } + log.Info("ForceLoggingTrue Output") + log.Info(out) + + } + if m.Spec.FlashBack && !flashBackStatus { + _, archiveLogStatus, _, result := dbcommons.CheckDBConfig(readyPod, r, r.Config, ctx, req, m.Spec.Edition) + if result.Requeue { + return result, nil + } + if archiveLogStatus { + out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | %s", dbcommons.FlashBackTrueSQL, dbcommons.GetSqlClient(m.Spec.Edition))) + if err != nil { + log.Error(err, err.Error()) + return requeueY, err + } + log.Info("FlashBackTrue Output") + log.Info(out) + + } else { + // Occurs when flashback is attermpted to be turned on without turning on archiving first + eventReason := "Waiting" + eventMsg := "enable ArchiveLog to turn ON Flashback" + log.Info(eventMsg) + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) + + changeArchiveLog = true + } + } + + //################################################################################################# + // TURNING FLASHBACK , ARCHIVELOG , FORCELOGGING TO FALSE + //################################################################################################# + + if !m.Spec.FlashBack && flashBackStatus { + out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | %s", dbcommons.FlashBackFalseSQL, dbcommons.GetSqlClient(m.Spec.Edition))) + if err != nil { + log.Error(err, err.Error()) + return requeueY, err + } + log.Info("FlashBackFalse Output") + log.Info(out) + } + if !m.Spec.ArchiveLog && archiveLogStatus { + flashBackStatus, _, _, result := dbcommons.CheckDBConfig(readyPod, r, r.Config, ctx, req, m.Spec.Edition) + if result.Requeue { + return result, nil + } + if !flashBackStatus { + + out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf(dbcommons.ArchiveLogFalseCMD, dbcommons.GetSqlClient(m.Spec.Edition))) + if err != nil { + log.Error(err, err.Error()) + return requeueY, err + } + log.Info("ArchiveLogFalse Output") + log.Info(out) + + } else { + // Occurs when archiving is attermpted to be turned off without turning off flashback first + eventReason := "Waiting" + eventMsg := "turn OFF Flashback to disable ArchiveLog" + log.Info(eventMsg) + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) + + changeArchiveLog = true + } + } + if !m.Spec.ForceLogging && forceLoggingStatus { + out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | %s", dbcommons.ForceLoggingFalseSQL, dbcommons.GetSqlClient(m.Spec.Edition))) + if err != nil { + log.Error(err, err.Error()) + return requeueY, err + } + log.Info("ForceLoggingFalse Output") + log.Info(out) + } + + //################################################################################################# + // CHECK FLASHBACK , ARCHIVELOG , FORCELOGGING + //################################################################################################# + + flashBackStatus, archiveLogStatus, forceLoggingStatus, result = dbcommons.CheckDBConfig(readyPod, r, r.Config, ctx, req, m.Spec.Edition) + if result.Requeue { + return result, nil + } + + log.Info("Flashback", "Status :", flashBackStatus) + log.Info("ArchiveLog", "Status :", archiveLogStatus) + log.Info("ForceLog", "Status :", forceLoggingStatus) + + m.Status.ArchiveLog = strconv.FormatBool(archiveLogStatus) + m.Status.ForceLogging = strconv.FormatBool(forceLoggingStatus) + + // If Flashback has turned from OFF to ON in this reconcile , + // Needs to restart the Non Ready Pods ( Delete old ones and create new ones ) + if m.Status.FlashBack == strconv.FormatBool(false) && flashBackStatus { + + // call FindPods() to fetch pods all version/images of the same SIDB kind + readyPod, replicasFound, available, _, err := dbcommons.FindPods(r, "", "", m.Name, m.Namespace, ctx, req) + if err != nil { + log.Error(err, err.Error()) + return requeueY, err + } + // delete non ready Pods as flashback needs restart of pods + _, err = r.deletePods(ctx, req, m, available, readyPod, replicasFound, 1) + return requeueY, err + } + + m.Status.FlashBack = strconv.FormatBool(flashBackStatus) + + if !changeArchiveLog && (flashBackStatus != m.Spec.FlashBack || + archiveLogStatus != m.Spec.ArchiveLog || forceLoggingStatus != m.Spec.ForceLogging) { + return requeueY, nil + } + return requeueN, nil +} + +//############################################################################# +// Manage Finalizer to cleanup before deletion of SingleInstanceDatabase +//############################################################################# +func (r *SingleInstanceDatabaseReconciler) manageSingleInstanceDatabaseDeletion(req ctrl.Request, ctx context.Context, + m *dbapi.SingleInstanceDatabase) (ctrl.Result, error) { + log := r.Log.WithValues("manageSingleInstanceDatabaseDeletion", req.NamespacedName) + + // Check if the SingleInstanceDatabase instance is marked to be deleted, which is + // indicated by the deletion timestamp being set. + isSingleInstanceDatabaseMarkedToBeDeleted := m.GetDeletionTimestamp() != nil + if isSingleInstanceDatabaseMarkedToBeDeleted { + if controllerutil.ContainsFinalizer(m, singleInstanceDatabaseFinalizer) { + // Run finalization logic for singleInstanceDatabaseFinalizer. If the + // finalization logic fails, don't remove the finalizer so + // that we can retry during the next reconciliation. + result, err := r.cleanupSingleInstanceDatabase(req, ctx, m) + if result.Requeue { + return result, err + } + + // Remove SingleInstanceDatabaseFinalizer. Once all finalizers have been + // removed, the object will be deleted. + controllerutil.RemoveFinalizer(m, singleInstanceDatabaseFinalizer) + err = r.Update(ctx, m) + if err != nil { + log.Error(err, err.Error()) + return requeueY, err + } + log.Info("Successfully Removed SingleInstanceDatabase Finalizer") + } + return requeueY, errors.New("deletion pending") + } + + // Add finalizer for this CR + if !controllerutil.ContainsFinalizer(m, singleInstanceDatabaseFinalizer) { + controllerutil.AddFinalizer(m, singleInstanceDatabaseFinalizer) + err := r.Update(ctx, m) + if err != nil { + log.Error(err, err.Error()) + return requeueY, err + } + } + return requeueN, nil +} + +//############################################################################# +// Finalization logic for singleInstanceDatabaseFinalizer +//############################################################################# +func (r *SingleInstanceDatabaseReconciler) cleanupSingleInstanceDatabase(req ctrl.Request, ctx context.Context, + m *dbapi.SingleInstanceDatabase) (ctrl.Result, error) { + log := r.Log.WithValues("cleanupSingleInstanceDatabase", req.NamespacedName) + // Cleanup steps that the operator needs to do before the CR can be deleted. + + if m.Status.OrdsReference != "" { + eventReason := "Cannot cleanup" + eventMsg := "uninstall ORDS to clean this SIDB" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + m.Status.Status = dbcommons.StatusError + return requeueY, nil + } + + // call deletePods() with zero pods in avaiable and nil readyPod to delete all pods + result, err := r.deletePods(ctx, req, m, []corev1.Pod{}, corev1.Pod{}, 0, 0) + if result.Requeue { + return result, err + } + + for { + podList := &corev1.PodList{} + listOpts := []client.ListOption{client.InNamespace(req.Namespace), client.MatchingLabels(dbcommons.GetLabelsForController("", req.Name))} + + if err := r.List(ctx, podList, listOpts...); err != nil { + log.Error(err, "Failed to list pods of "+req.Name, "Namespace", req.Namespace) + return requeueY, err + } + if len(podList.Items) == 0 { + break + } + var podNames = "" + for _, pod := range podList.Items { + podNames += pod.Name + " " + } + eventReason := "Waiting" + eventMsg := "waiting for " + req.Name + " database pods ( " + podNames + " ) to terminate" + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) + r.Log.Info(eventMsg) + time.Sleep(15 * time.Second) + } + + log.Info("Successfully cleaned up SingleInstanceDatabase") + return requeueN, nil +} + +//############################################################################# +// SetupWithManager sets up the controller with the Manager +//############################################################################# +func (r *SingleInstanceDatabaseReconciler) SetupWithManager(mgr ctrl.Manager) error { + return ctrl.NewControllerManagedBy(mgr). + For(&dbapi.SingleInstanceDatabase{}). + Owns(&corev1.Pod{}). //Watch for deleted pods of SingleInstanceDatabase Owner + WithEventFilter(dbcommons.ResourceEventHandler()). + WithOptions(controller.Options{MaxConcurrentReconciles: 100}). //ReconcileHandler is never invoked concurrently with the same object. + Complete(r) +} diff --git a/controllers/database/suite_test.go b/controllers/database/suite_test.go new file mode 100644 index 00000000..9b62324a --- /dev/null +++ b/controllers/database/suite_test.go @@ -0,0 +1,103 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package controllers + +import ( + "path/filepath" + "testing" + + . "github.com/onsi/ginkgo" + . "github.com/onsi/gomega" + "k8s.io/client-go/kubernetes/scheme" + "k8s.io/client-go/rest" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/envtest" + "sigs.k8s.io/controller-runtime/pkg/envtest/printer" + logf "sigs.k8s.io/controller-runtime/pkg/log" + "sigs.k8s.io/controller-runtime/pkg/log/zap" + + databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + // +kubebuilder:scaffold:imports +) + +// These tests use Ginkgo (BDD-style Go testing framework). Refer to +// http://onsi.github.io/ginkgo/ to learn more about Ginkgo. + +var cfg *rest.Config +var k8sClient client.Client +var testEnv *envtest.Environment + +func TestAPIs(t *testing.T) { + RegisterFailHandler(Fail) + + RunSpecsWithDefaultAndCustomReporters(t, + "Controller Suite", + []Reporter{printer.NewlineReporter{}}) +} + +var _ = BeforeSuite(func(done Done) { + logf.SetLogger(zap.New(zap.WriteTo(GinkgoWriter), zap.UseDevMode(true))) + + By("bootstrapping test environment") + testEnv = &envtest.Environment{ + CRDDirectoryPaths: []string{filepath.Join("../..", "config", "crd", "bases")}, + } + + var err error + cfg, err = testEnv.Start() + Expect(err).ToNot(HaveOccurred()) + Expect(cfg).ToNot(BeNil()) + + err = databasev1alpha1.AddToScheme(scheme.Scheme) + Expect(err).NotTo(HaveOccurred()) + + // +kubebuilder:scaffold:scheme + + k8sClient, err = client.New(cfg, client.Options{Scheme: scheme.Scheme}) + Expect(err).ToNot(HaveOccurred()) + Expect(k8sClient).ToNot(BeNil()) + + close(done) +}, 60) + +var _ = AfterSuite(func() { + By("tearing down the test environment") + err := testEnv.Stop() + Expect(err).ToNot(HaveOccurred()) +}) diff --git a/doc/adb/ADB_PREREQUISITES.md b/doc/adb/ADB_PREREQUISITES.md new file mode 100644 index 00000000..ec51729a --- /dev/null +++ b/doc/adb/ADB_PREREQUISITES.md @@ -0,0 +1,100 @@ +# + +## Oracle Autonomous Database (ADB) Prerequisites + +Oracle Database Operator for Kubernetes must have access to OCI services. + +To provide access, choose **one of the following approaches**: + +* The provider uses [API Key authentication](#authorized-with-api-key-authentication) + +* The Kubernetes cluster nodes are [granted with Instance Principal](#authorized-with-instance-principal) + +### Authorized with API Key Authentication + +By default, all pods in the Oracle Container Engine for Kubernetes (OKE) are able to access the instance principal certificates, so that the operator calls OCI REST endpoints without any extra step. If you're using OKE, then please proceed to the [Oracle Autonomous Database document](./doc/adb/ORACLE_ADB_CONTROLLER_README.md). +If the operator is deployed in a third-party Kubernetes cluster, then the credentials of the Oracle Cloud Infrastructure (OCI) user are needed. The operator reads these credentials from a ConfigMap and a Secret. + +Oracle recommends using the helper script `set_ocicredentials.sh` in the root directory of the repository; This script will generate a ConfigMap and a Secret with the OCI credentials. By default, the script parses the **DEFAULT** profile in `~/.oci/config`. The default names of the ConfigMap and the Secret are, respectively: `oci-cred` and `oci-privatekey`. + +```sh +./set_ocicredentials.sh run +``` + +You can change the default values as follows: + +```sh +./set_ocicredentials.sh run -path -profile -configmap -secret +``` + +Alternatively, you can create these values manually. The ConfigMap should contain the following items: `tenancy`, `user`, `fingerprint`, `region`, `passphrase`. The Secret should contain an entry named `privatekey`. + +```sh +kubectl create configmap oci-cred \ +--from-literal=tenancy= \ +--from-literal=user= \ +--from-literal=fingerprint= \ +--from-literal=region= \ +--from-literal=passphrase=(*) + +kubectl create secret generic oci-privatekey \ +--from-file=privatekey= +``` + +> Note: passphrase is deprecated. You can ignore that line. + +After creating the ConfigMap and the Secret, use their names as the values of `ociConfigMap` and `ociSecret` attributes in the yaml files for provisioning, binding, and other operations. + +### Authorized with Instance Principal + +Instance principal authorization enables the operator to make API calls from an instance (that is, a node) without requiring the `ociConfigMap`, and `ociSecret` attributes in the `.yaml` file. + +> Note: Instance principal authorization applies only to instances that are running in the Oracle Cloud Infrastructure (OCI). + +To set up Instance Principle authorization: + +1. Get the `compartment OCID`: + + Log in to the cloud console, and click **Compartment**. + + ![compartment-1](/images/adb/compartment-1.png) + + Choose the compartment where the cluster creates instances, and **copy** the OCID in the details page. + + ![compartment-2](/images/adb/compartment-2.png) + +2. Create a dynamic group and matching rules: + + Go to the **Dynamic Groups** page, and click **Create Dynamic Group**. + + ![instance-principal-1](/images/adb/instance-principal-1.png) + + In the **Matching Rules** section, write the following rule. Change `compartment-OCID` to the OCID of your compartment. This rule enables all the resources, including **nodes** in the compartment, to be members of the dynamic group. + + ```sh + All {instance.compartment.id = 'compartment-OCID'} + ``` + + ![instance-principal-2](/images/adb/instance-principal-2.png) + + To apply the rules, click **Create**. + +3. Set up policies for dynamic groups: + + Go to **Policies**, and click **Create Policy**. + + ![instance-principal-3](/images/adb/instance-principal-3.png) + + This example enables the dynamic group to manage all the resources in your tenancy: + + ```sh + Allow dynamic-group to manage all-resources in tenancy + ``` + + You can also specify a particular resouce access for the dynamic group. This example enables the dynamic group to manage Oracle Autonomous Database in a given compartment: + + ```sh + Allow dynamic-group to manage autonomous-database-family in compartment + ``` + +At this stage, the operator has been granted sufficient permissions to call OCI services. You can now proceed to the the installation. diff --git a/doc/adb/ORACLE_ADB_CONTROLLER_README.md b/doc/adb/ORACLE_ADB_CONTROLLER_README.md new file mode 100644 index 00000000..38fd6ff5 --- /dev/null +++ b/doc/adb/ORACLE_ADB_CONTROLLER_README.md @@ -0,0 +1,376 @@ +# Managing Oracle Autonomous Databases with Oracle Database Operator for Kubernetes + +Before you use the Oracle Database Operator for Kubernetes (the operator), ensure your system meets all of the Oracle Autonomous Database (ADB) Prerequisites [ADB_PREREQUISITES](./ADB_PREREQUISITES.md). + +## Supported Features + +After the operator is deployed, choose either one of the following operations to create an `AutonomousDatabase` custom resource for Oracle Autonomous Database in your cluster. + +* [Provision](#provision-an-autonomous-database) an Autonomous Database +* [Bind](#bind-to-an-existing-autonomous-database) to an existing Autonomous Database + +After you create the resource, you can use the operator to perform the following tasks: + +* [Scale the OCPU core count or storage](#scale-the-ocpu-core-count-or-storage) an Autonomous Database +* [Enable/Disable auto scaling](#enabledisable-auto-scaling) of an Autonomous Database +* [Rename](#rename) an Autonomous Database +* [Manage ADMIN database user password](#manage-admin-passsword) of an Autonomous Database +* [Download instance credentials (wallets)](#download-wallets) of an Autonomous Database +* [Stop/Start/Terminate](#stopstartterminate) an Autonomous Database +* [Delete the resource](#delete-the-resource) from the cluster + +## Provision an Autonomous Database + +Follow the steps to provision an Autonomous Database that will bind objects in your cluster. + +1. Get the `Compartment OCID`. + + Login cloud console and click `Compartment`. + + ![compartment-1](/images/adb/compartment-1.png) + + Click on the compartment name where you want to create your database, and **copy** the `OCID` of the compartment. + + ![compartment-2](/images/adb/compartment-2.png) + +2. Create a Kubernetes Secret to hold the password of the ADMIN user. + + You can create this secret with the following command (as an example): + + ```sh + kubectl create secret generic admin-password --from-literal=admin-password='password_here' + ``` + +3. Add the following fields to the AutonomousDatabase resource definition. An example `.yaml` file is available here: [`config/samples/autonomousdatabase_create.yaml`](./../../config/samples/autonomousdatabase_create.yaml) + | Attribute | Type | Description | Required? | + |----|----|----|----| + | `spec.details.compartmentOCID` | string | The [OCID](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm) of the compartment of the Autonomous Database. | Yes | + | `spec.details.dbName` | string | The database name. The name must begin with an alphabetic character and can contain a maximum of 14 alphanumeric characters. Special characters are not permitted. The database name must be unique in the tenancy. | Yes | + | `spec.details.displayName` | string | The user-friendly name for the Autonomous Database. The name does not have to be unique. | Yes | + | `spec.details.cpuCoreCount` | int | The number of OCPU cores to be made available to the database. | Yes | + | `spec.details.adminPassword` | dictionary | The password for the ADMIN user. The password must be between 12 and 30 characters long, and must contain at least 1 uppercase, 1 lowercase, and 1 numeric character. It cannot contain the double quote symbol (") or the username "admin", regardless of casing.

Either `k8sSecretName` or `ociSecretOCID` must be provided. If both `k8sSecretName` and `ociSecretOCID` appear, the Operator reads the password from the K8s secret that `k8sSecretName` refers to. | Yes | + | `spec.details.adminPassword.k8sSecretName` | string | The **name** of the K8s Secret where you want to hold the password for the ADMIN user. | Conditional | + |`spec.details.adminPassword.ociSecretOCID` | string | The **[OCID](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm)** of the [OCI Secret](https://docs.oracle.com/en-us/iaas/Content/KeyManagement/Tasks/managingsecrets.htm) where you want to hold the password for the ADMIN user. | Conditional | + | `spec.details.dataStorageSizeInTBs` | int | The size, in terabytes, of the data volume that will be created and attached to the database. This storage can later be scaled up if needed. | Yes | + | `spec.details.isAutoScalingEnabled` | boolean | Indicates if auto scaling is enabled for the Autonomous Database OCPU core count. The default value is `FALSE` | No | + | `spec.details.isDedicated` | boolean | True if the database is on dedicated [Exadata infrastructure](https://docs.cloud.oracle.com/Content/Database/Concepts/adbddoverview.htm) | No | + | `spec.details.freeformTags` | dictionary | Free-form tags for this resource. Each tag is a simple key-value pair with no predefined name, type, or namespace. For more information, see [Resource Tag](https://docs.cloud.oracle.com/Content/General/Concepts/resourcetags.htm).

Example:
`freeformTags:`
    `key1: value1`
    `key2: value2`| No | + | `spec.details.dbWorkload` | string | The Oracle Autonomous Database workload type. The following values are valid:
- OLTP - indicates an Autonomous Transaction Processing database
- DW - indicates an Autonomous Data Warehouse database
- AJD - indicates an Autonomous JSON Database
- APEX - indicates an Autonomous Database with the Oracle APEX Application Development workload type. | No | + | `spec.details.dbVersion` | string | A valid Oracle Database release for Oracle Autonomous Database. | No | + | `spec.ociConfig` | dictionary | Not required when the Operator is authorized with [Instance Principal](./ADB_PREREQUISITES.md#authorized-with-instance-principal). Otherwise, you will need the values from the [Authorized with API Key Authentication](./ADB_PREREQUISITES.md#authorized-with-api-key-authentication) section. | Conditional | + | `spec.ociConfig.configMapName` | string | Name of the ConfigMap that holds the local OCI configuration | Conditional | + | `spec.ociConfig.secretName`| string | Name of the K8s Secret that holds the private key value | Conditional | + + ```yaml + --- + apiVersion: database.oracle.com/v1alpha1 + kind: AutonomousDatabase + metadata: + name: autonomousdatabase-sample + spec: + details: + compartmentOCID: ocid1.compartment... + dbName: NewADB + displayName: NewADB + cpuCoreCount: 1 + adminPassword: + k8sSecretName: admin-password # use the name of the secret from step 2 + dataStorageSizeInTBs: 1 + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey + ``` + +4. Apply the yaml: + + ```sh + kubectl apply -f config/samples/autonomousdatabase_create.yaml + autonomousdatabase.database.oracle.com/autonomousdatabase-sample created + ``` + +## Bind to an existing Autonomous Database + +Other than provisioning a database, you can bind to an existing database in your cluster. + +1. Clean up the resource you created in the earlier provision operation: + + ```sh + kubectl delete adb/autonomousdatabase-sample + autonomousdatabase.database.oracle.com/autonomousdatabase-sample deleted + ``` + +2. Copy the `Autonomous Database OCID` from Cloud Console. + + ![adb-id-1](/images/adb/adb-id-1.png) + + ![adb-id-2](/images/adb/adb-id-2.png) + +3. Add the following fields to the AutonomousDatabase resource definition. An example `.yaml` file is available here: [`config/samples/autonomousdatabase_bind.yaml`](./../../config/samples/autonomousdatabase_bind.yaml) + | Attribute | Type | Description | Required? | + |----|----|----|----| + | `spec.details.autonomousDatabaseOCID` | string | The [OCID](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm) of the Autonomous Database you want to bind (create a reference) in your cluster. | Yes | + | `spec.ociConfig` | dictionary | Not required when the Operator is authorized with [Instance Principal](./ADB_PREREQUISITES.md#authorized-with-instance-principal). Otherwise, you will need the values from the [Authorized with API Key Authentication](./ADB_PREREQUISITES.md#authorized-with-api-key-authentication) section. | Conditional | + | `spec.ociConfig.configMapName` | string | Name of the ConfigMap that holds the local OCI configuration | Conditional | + | `spec.ociConfig.secretName`| string | Name of the K8s Secret that holds the private key value | Conditional | + + ```yaml + --- + apiVersion: database.oracle.com/v1alpha1 + kind: AutonomousDatabase + metadata: + name: autonomousdatabase-sample + spec: + details: + autonomousDatabaseOCID: ocid1.autonomousdatabase... + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey + ``` + +4. Apply the yaml. + + ```sh + kubectl apply -f config/samples/autonomousdatabase_bind.yaml + autonomousdatabase.database.oracle.com/autonomousdatabase-sample created + ``` + +## Scale the OCPU core count or storage + +> Note: this operation requires an `AutonomousDatabase` object to be in your cluster. This example assumes either the provision operation or the bind operation has been done by the users and the operator is authorized with API Key Authentication. + +Users can scale up or scale down the Oracle Autonomous Database OCPU core count or storage by updating the `cpuCoreCount` and `dataStorageSizeInTBs` parameters. The `isAutoScalingEnabled` indicates whether auto scaling is enabled. Here is an example of scaling the CPU count and storage size (TB) up to 2 and turning off the auto-scaling by updating the `autonomousdatabase-sample` custom resource. + +1. An example YAML file is available here: [config/samples/autonomousdatabase_scale.yaml](./../../config/samples/autonomousdatabase_scale.yaml) + + ```yaml + --- + apiVersion: database.oracle.com/v1alpha1 + kind: AutonomousDatabase + metadata: + name: autonomousdatabase-sample + spec: + details: + autonomousDatabaseOCID: ocid1.autonomousdatabase... + cpuCoreCount: 2 + dataStorageSizeInTBs: 2 + isAutoScalingEnabled: false + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey + ``` + +2. Apply the change using `kubectl`. + + ```sh + kubectl apply -f config/samples/autonomousdatabase_scale.yaml + autonomousdatabase.database.oracle.com/autonomousdatabase-sample configured + ``` + +## Rename + +> Note: this operation requires an `AutonomousDatabase` object to be in your cluster. This example assumes the provision operation or the bind operation has been completed, and the operator is authorized with API Key Authentication. + +You can rename the database by changing the values of the `dbName` and `displayName`, as follows: + +1. An example YAML file is available here: [config/samples/autonomousdatabase_rename.yaml](./../../config/samples/autonomousdatabase_rename.yaml) + + ```yaml + --- + apiVersion: database.oracle.com/v1alpha1 + kind: AutonomousDatabase + metadata: + name: autonomousdatabase-sample + spec: + details: + autonomousDatabaseOCID: ocid1.autonomousdatabase... + dbName: RenamedADB + displayName: RenamedADB + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey + ``` + + * `dbName`: The database name. It must begin with an alphabetic character. It can contain a maximum of 14 alphanumeric characters. Special characters are not permitted. The database name must be unique in the tenancy. + * `displayNameName`: User-friendly name of the database. The name does not have to be unique. + +2. Apply the change using `kubectl`. + + ```sh + kubectl apply -f config/samples/autonomousdatabase_rename.yaml + autonomousdatabase.database.oracle.com/autonomousdatabase-sample configured + ``` + +## Manage Admin Passsword + +> Note: this operation requires an `AutonomousDatabase` object to be in your cluster. This example assumes the provision operation or the bind operation has been completed, and the operator is authorized with API Key Authentication. + +1. Create a Kubernetes Secret to hold the new password of the ADMIN user. + + As an example, you can create this secret with the following command: * + + ```sh + kubectl create secret generic new-adb-admin-password --from-literal=new-adb-admin-password='password_here' + ``` + + \* The password must be between 12 and 30 characters long, and must contain at least 1 uppercase, 1 lowercase, and 1 numeric character. It cannot contain the double quote symbol (") or the username "admin", regardless of casing. + +2. Update the example [config/samples/autonomousdatabase_change_admin_password.yaml](./../../config/samples/autonomousdatabase_change_admin_password.yaml) + + ```yaml + --- + apiVersion: database.oracle.com/v1alpha1 + kind: AutonomousDatabase + metadata: + name: autonomousdatabase-sample + spec: + details: + autonomousDatabaseOCID: ocid1.autonomousdatabase... + adminPassword: + k8sSecretName: new-admin-password + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey + ``` + + * `adminPassword.k8sSecretName`: the **name** of the secret that you created in **step1**. + +3. Apply the YAML. + + ```sh + kubectl apply -f config/samples/autonomousdatabase_change_admin_password.yaml + autonomousdatabase.database.oracle.com/autonomousdatabase-sample configured + ``` + +## Download Wallets + +> Note: this operation requires an `AutonomousDatabase` object to be in your cluster. This example assumes the provision operation or the bind operation has been done by the users and the operator is authorized with API Key Authentication. + +A client Wallet is required to connect to a shared Oracle Autonomous Database. User has to provide a wallet password to download the Wallet. In the following example, the Operator will read the password from a Kubernetes Secret to download the Wallet. After that, the downloaded Wallet will be unzipped and stored as byte values in a new Kubernetes Secret `instance-wallet`. + +1. Create a Kubernetes Secret to hold the wallet password. + + As an example, you can create this secret with the following command: * + + ```sh + kubectl create secret generic instance-wallet-password --from-literal=instance-wallet-password='password_here' + ``` + + \* The password must be at least 8 characters long and must include at least 1 letter and either 1 numeric character or 1 special character. + +2. Update the example [config/samples/autonomousdatabase_wallet.yaml](./../../config/samples/autonomousdatabase_wallet.yaml) + + ```yaml + --- + apiVersion: database.oracle.com/v1alpha1 + kind: AutonomousDatabase + metadata: + name: autonomousdatabase-sample + spec: + details: + autonomousDatabaseOCID: ocid1.autonomousdatabase... + wallet: + name: instance-wallet + password: + k8sSecretName: instance-wallet-password + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey + ``` + + * `wallet.name`: the name of the new Secret where you want the downloaded Wallet to be stored. + * `wallet.password.k8sSecretName`: the **name** of the secret you created in **step1**. + +3. Apply the YAML + + ```sh + kubectl apply -f config/samples/autonomousdatabase_wallet.yaml + autonomousdatabase.database.oracle.com/autonomousdatabase-sample configured + ``` + +You should see a new Secret `instance-wallet` in your cluster: + +```sh +$ kubectl get secrets +NAME TYPE DATA AGE +oci-privatekey Opaque 1 2d12h +instance-wallet-password Opaque 1 2d12h +instance-wallet Opaque 8 2d12h +``` + +To use the secret in a deployment, refer to [Using Secrets](https://kubernetes.io/docs/concepts/configuration/secret/#using-secrets) for the examples. + +## Stop/Start/Terminate + +> Note: this operation requires an `AutonomousDatabase` object to be in your cluster. This example assumes the provision operation or the bind operation has been done by the users and the operator is authorized with API Key Authentication. + +Users can start/stop/terminate a database using the `lifecycleState` attribute. +Here's a list of the values you can set for `lifecycleState`: + +* `AVAILABLE`: to start the database +* `STOPPED`: to stop the database +* `TERMINATED`: to terminate the database + +1. A sample .yaml file is available here: [config/samples/autonomousdatabase_stop_start_terminate.yaml](./../../config/samples/autonomousdatabase_stop_start_terminate.yaml) + + ```yaml + --- + apiVersion: database.oracle.com/v1alpha1 + kind: AutonomousDatabase + metadata: + name: autonomousdatabase-sample + spec: + details: + autonomousDatabaseOCID: ocid1.autonomousdatabase... + lifecycleState: STOPPED + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey + ``` + +2. Apply the change to stop the database. + + ```sh + kubectl apply -f config/samples/autonomousdatabase_stop_start_terminate.yaml + autonomousdatabase.database.oracle.com/autonomousdatabase-sample configured + ``` + +## Delete the resource + +> Note: this operation requires an `AutonomousDatabase` object to be in your cluster. This example assumes the provision operation or the bind operation has been done by the users and the operator is authorized with API Key Authentication. + +The `hardLink` defines the behavior when the resource is deleted from the cluster. If the `hardLink` is set to true, the Operator terminates the Autonomous Database in OCI when the resource is removed; otherwise, the database remains unchanged. By default the value is `false` if it is not explicitly specified. + +Follow the steps to delete the resource and terminate the Autonomous Database. + +1. Use the example [autonomousdatabase_delete_resource.yaml](./../../config/samples/autonomousdatabase_delete_resource.yaml) which sets the attribute `hardLink` to true. + + ```yaml + --- + apiVersion: database.oracle.com/v1alpha1 + kind: AutonomousDatabase + metadata: + name: autonomousdatabase-sample + spec: + details: + autonomousDatabaseOCID: ocid1.autonomousdatabase... + hardLink: true + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey + ``` + +2. Apply the yaml + + ```sh + kubectl apply -f config/samples/autonomousdatabase_delete_resource.yaml + autonomousdatabase.database.oracle.com/autonomousdatabase-sample configured + ``` + +3. Delete the resource in your cluster + + ```sh + kubectl delete adb/autonomousdatabase-sample + autonomousdatabase.database.oracle.com/autonomousdatabase-sample deleted + ``` + +Now, you can verify that the database is in TERMINATING state on the Cloud Console. diff --git a/doc/installation/OPERATOR_INSTALLATION_README.md b/doc/installation/OPERATOR_INSTALLATION_README.md new file mode 100644 index 00000000..d01c468f --- /dev/null +++ b/doc/installation/OPERATOR_INSTALLATION_README.md @@ -0,0 +1,92 @@ +# Oracle Database Operator for Kubernetes Development Environment Instructions + +Review these instructions to set up your development environment with Oracle Database Operator for Kubernetes. + +To set up your development environment to use the operator, complete the following steps + +## Check Operating System Requirements + +Your system must be using Oracle Linux 7 (7.x), with the Unbreakable Enterprise Kernel Release 5 (UEK R5). + +Validate using `uname`. For example: + + ```sh + uname -r + 4.14.35-1902.0.18.el7uek.x86_64 + ``` + +## Download and Set Up the Go Programming Language (GoLang) +Oracle strongly recommends that you use a higher version of GoLang, at least 1.16 or a later release. + +1. Download the Linux distribution of GoLang from [Go’s official download page](https://golang.org/dl/) and extract it into /usr/local directory + +1. Run the following commands: + + ```sh + tar -C /usr/local -xzf go$VERSION.$OS-$ARCH.tar.gz + export PATH=$PATH:/usr/local/go/bin + go version + ``` +1. Add the `/usr/local/go/bin` directory to your `PATH` environment variable. You can do this by adding the a line to your `~/.bash_profile` file. +## Set the GOPATH Environment + +Set the `GOPATH` environment variable. This variable specifies the location of your workspace. By default, the location specified by `GOPATH` is assumed to be `$HOME/go`. For example: + ```sh + export GODIR=/scratch/go/projects/ + export GOPATH=$GODIR/go + ``` +The Go workspace is divided into following directoriess: +* `src`: contains Go source files. +* `bin`: contains the binary executables. +* `pkg`: contains Go package archives (`.a`). + +## Set Up the OPERATOR Development Environment (Dev Env) + +Install the latest version of the Operator software development kit (SDK). For example: + ```sh + wget https://github.com/operator-framework/operator-sdk/releases/download/v1.2.0/operator-sdk-v1.2.0-x86_64-linux-gnu + mv operator-sdk-v1.2.0-x86_64-linux-gnu /usr/local/bin/operator-sdk + chmod +x /usr/local/bin/operator-sdk + ``` +If you encounter an issue with these steps, then refer to the more detailed steps given in the following documents: +* [Operator-framework/operator-sdk] () +* [Install the Operator SDK CLI] () +* [Quickstart for Go-based Operators] () + +## Clone Oracle Database Operator +Clone the [oracle-database-operator](https://github.com/oracle/oracle-database-operator) on your local machine + +## Review and modify the Cloned Repository +Make changes as needed to the repository. + +If you want to run the operator locally outside the cluster, then run the following steps: + + ```sh + cd oracle-database-operator + make generate + make manifests + make install run + ``` +If you want to run the operator inside the cluster, then run the following steps: + + ```sh + cd oracle-database-operator + make generate + make manifests + make install + make docker-build IMG=.ocir.io///: + docker push .ocir.io///: + kubectl create namespace oracle-database-operator-system + kubectl create secret docker-registry container-registry-secret -n oracle-database-operator-system --docker-server=.ocir.io --docker-username='/' --docker-password='' --docker-email='' + make operator-yaml IMG=.ocir.io///: + ``` + +## Check the YAML File +You should see file `oracle-database-operator.yaml`. This file will perform the following operations: + * Create CRDs + * Create Roles and Bindings + * Operator Deployment + +## Run the Quick Install + +Follow the steps mentioned in the `Quick Install of the Operator` section of [README.md](../../README.md#quick-install-the-operator) to install operator with your changes. diff --git a/doc/sharding/ORACLE_SHARDING_CONTROLLER_README.md b/doc/sharding/ORACLE_SHARDING_CONTROLLER_README.md new file mode 100644 index 00000000..1c85dc45 --- /dev/null +++ b/doc/sharding/ORACLE_SHARDING_CONTROLLER_README.md @@ -0,0 +1,111 @@ +# Oracle Sharding Database Controller + +Oracle Sharding distributes segments of a data set across many databases (shards) on different computers, either on-premises or in cloud. Sharding enables globally distributed, linearly scalable, multimodel databases. It requires no specialized hardware or software. Oracle Sharding does all this while rendering the strong consistency, full power of SQL, support for structured and unstructured data, and the Oracle Database ecosystem. It meets data sovereignty requirements, and supports applications that require low latency and high availability. + +All of the shards together make up a single logical database, which is referred to as a sharded database (SDB). + +Kubernetes provides infrastructure building blocks, such as compute, storage, and networks. Kubernetes makes the infrastructure available as code. It enables rapid provisioning of multi-node topolgies. Additionally, Kubernetes also provides statefulsets, which are the workload API objects that are used to manage stateful applications. This provides us lifecycle management elasticity for databases as a stateful application for various database topologies, such as sharded databases, Oracle Real Application Clusters (Oracle RAC), single instance Oracle Database, and other Oracle features and configurations. + +The Sharding Database controller in Oracle Database Operator deploys Oracle Sharding topology as a statefulset in the Kubernetes clusters, using Oracle Database and Global Data Services Docker images. The Oracle Sharding database controller manages the typical lifecycle of Oracle Sharding topology in the Kubernetes cluster, as shown below: + +* Create primary statefulsets shards +* Create master and standby Global Data Services statefulsets +* Create persistent storage, along with statefulset +* Create services +* Create load balancer service +* Provision sharding topology by creating and configuring the following: + * Catalog database + * Shard Databases + * GSMs + * Shard scale up and scale down +* Shard topology cleanup + +The Oracle Sharding database controller provides end-to-end automation of Oracle Database sharding topology deployment in Kubernetes clusters. + +## Using Oracle Sharding Database Operator + +To create a Sharding Topology, complete the steps in the following sections below: + +1. [Prerequsites for running Oracle Sharding Database Controller](#prerequsites-for-running-oracle-sharding-database-controller) +2. [Provisioning Sharding Topology in a Cloud based Kubernetes Cluster (OKE in this case)](#provisioning-sharding-topology-in-a-cloud-based-kubernetes-cluster-oke-in-this-case) +3. [Provisioning Sharding Topology in an On-Premise Kubernetes Cluster (OLCNE in this case)](#provisioning-sharding-topology-in-an-on-premise-kubernetes-cluster-olcne-in-this-case) +4. [Connecting to Shard Databases](#connecting-to-shard-databases) +5. [Debugging and Troubleshooting](#debugging-and-troubleshooting) + +**Note** Before proceeding to the next section, you must complete the instructions given in each section, based on your enviornment, before proceeding to next section. + +## Prerequsites for Running Oracle Sharding Database Controller + +**IMPORTANT :** You must make the changes specified in this section before you proceed to the next section. + +### 1. Kubernetes Cluster: To deploy Oracle Sharding database controller with Oracle Database Operator, you need a Kubernetes Cluster which can be one of the following: + +* A Cloud-based Kubernetes cluster, such as [OCI on Container Engine for Kubernetes (OKE)](https://www.oracle.com/cloud-native/container-engine-kubernetes/) or +* An On-Premises Kubernetes Cluster, such as [Oracle Linux Cloud Native Environment (OLCNE)](https://docs.oracle.com/en/operating-systems/olcne/) cluster. + +To use Oracle Sharding Database Controller, ensure that your system is provisioned with a supported Kubernetes release. Refer to the [Release Status Section](../../../README.md#requirements). + +### 2. Deploy Oracle Database Operator + +To deploy Oracle Database Operator in a Kubernetes cluster, go to the section [Quick Install of the Operator](../../README.md#oracle-database-kubernetes-operator-deployment) in the README, and complete the operator deployment before you proceed further. If you have already deployed the operator, then proceed to the next section. + +### 3. Oracle Database and Global Data Services Docker Images +Choose one of the following deployment options: + + **Use Oracle-Supplied Docker Images:** + The Oracle Sharding Database controller uses Oracle Global Data Services and Oracle Database images to provision the sharding topology. + + You can also download the pre-built Oracle Global Data Services `container-registry.oracle.com/database/gsm:latest` and Oracle Database images `container-registry.oracle.com/database/enterprise:latest` from [Oracle Container Registry](https://container-registry.oracle.com/ords/f?p=113:10::::::). These images are functionally tested and evaluated with various use cases of sharding topology by deploying on OKE and OLCNE. + + + **OR** + + **Build your own Oracle Database and Global Data Services Docker Images:** + You can build these images using instructions provided on Oracle official GitHub Repositories: + [Oracle Global Data Services Image](https://github.com/oracle/db-sharding/tree/master/docker-based-sharding-deployment/dockerfiles) + [Oracle Database Image](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance) + +After the images are ready, push them to your Docker Images Repository, so that you can pull them during Oracle Database Sharding topology provisioning. + +You can either download the images and push them to your Docker Images Repository, or, if your Kubernetes cluster can reach OCR, you can download these images directly from OCR. + +**Note**: In the sharding example yaml files, we are using GDS and database images available on [Oracle Container Registry](https://container-registry.oracle.com/ords/f?p=113:10::::::). + +### 4. Create a namespace for the Oracle DB Sharding Setup + + Create a Kubernetes namespace named `shns`. All the resources belonging to the Oracle Database Sharding Setup will be provisioned in this namespace named `shns`. For example: + + ```sh + #### Create the namespace + kubectl create ns shns + + #### Check the created namespace + kubectl get ns + ``` + +### 5. Create a Kubernetes secret for the database installation owner for the database Sharding Deployment + +Create a Kubernetes secret named `db-user-pass` using these steps: [Create Kubernetes Secret](./doc/sharding/provisioning/create_kubernetes_secret_for_db_user.md) + +After you have the above prerequsites completed, you can proceed to the next section for your environment to provision the Oracle Database Sharding Topology. + +## Provisioning Sharding Topology in a Cloud-Based Kubernetes Cluster (OKE in this case) + +Deploy Oracle Database sharding topology on your Cloud based Kubernetes cluster. In this example, the deployment uses the YAML file based on `OCI OKE` cluster. There are multiple use case possible for deploying the Oracle Database sharding topology. + +[1. Provisioning Oracle Database sharding topology without Database Gold Image](./doc/sharding/provisioning/provisioning_without_db_gold_image.md) +[2. Provisioning Oracle Database sharding topology with additional control on resources like Memory and CPU allocated to Pods](./doc/sharding/provisioning/provisioning_with_control_on_resources.md) +[3. Provisioning a Persistent Volume having an Oracle Database Gold Image](./doc/sharding/provisioning/provisioning_persistent_volume_having_db_gold_image.md) +[4. Provisioning Oracle Database sharding topology by cloning database from your own Database Gold Image in the same Availability Domain(AD)](./doc/sharding/provisioning/provisioning_by_cloning_db_gold_image_in_same_ad.md) +[5. Provisioning Oracle Database sharding topology by cloning database from your own Database Gold Image across Availability Domains(ADs)](./doc/sharding/provisioning/provisioning_by_cloning_db_from_gold_image_across_ads.md) +[6. Provisioning Oracle Database sharding topology and send Notification using OCI Notification Service](./doc/sharding/provisioning/provisioning_with_notification_using_oci_notification.md) +[7. Scale Out - Add Shards to an existing Oracle Database Sharding Topology](./doc/sharding/provisioning/scale_out_add_shards.md) +[8. Scale In - Delete an existing Shard from a working Oracle Database sharding topology](./doc/sharding/provisioning/scale_in_delete_an_existing_shard.md) + +## Connecting to Shard Databases + +After the Oracle Database Sharding Topology has been provisioned using the Sharding Controller in Oracle Database Kubernetes Operator, you can follow the steps in this document to connect to the Sharded Database or to the individual Shards: [Database Connectivity](./doc/sharding/provisioning/database_connection.md) + +## Debugging and Troubleshooting + +To debug the Oracle Database Sharding Topology provisioned using the Sharding Controller of Oracle Database Kubernetes Operator, follow this document: [Debugging and troubleshooting](./doc/sharding/provisioning/debugging.md) diff --git a/doc/sharding/provisioning/create_kubernetes_secret_for_db_user.md b/doc/sharding/provisioning/create_kubernetes_secret_for_db_user.md new file mode 100644 index 00000000..0d66f49b --- /dev/null +++ b/doc/sharding/provisioning/create_kubernetes_secret_for_db_user.md @@ -0,0 +1,25 @@ +# Create kubernetes secret for db user + +Create a Kubernetes secret named "db-user-pass" using a password in a text file and then encrypt it using an `openssl` key. The text file will be removed after secret is created. + +```sh +mkdir /tmp/.secrets/ + +# Generate a random openssl key +openssl rand -hex 64 -out /tmp/.secrets/pwd.key + +# Use a password you want and add it to a text file +echo ORacle_21c > /tmp/.secrets/common_os_pwdfile + +# Encrypt the file with the password with the random openssl key generated above +openssl enc -aes-256-cbc -md md5 -salt -in /tmp/.secrets/common_os_pwdfile -out /tmp/.secrets/common_os_pwdfile.enc -pass file:/tmp/.secrets/pwd.key + +# Remove the password text file +rm -f /tmp/.secrets/common_os_pwdfile + +# Create the Kubernetes secret in namespace "shns" +kubectl create secret generic db-user-pass --from-file=/tmp/.secrets/common_os_pwdfile.enc --from-file=/tmp/.secrets/pwd.key -n shns + +# Check the secret details +kubectl get secret -n shns +``` diff --git a/doc/sharding/provisioning/database_connection.md b/doc/sharding/provisioning/database_connection.md new file mode 100644 index 00000000..7f64bbd5 --- /dev/null +++ b/doc/sharding/provisioning/database_connection.md @@ -0,0 +1,44 @@ +# Database Connectivity + +The Oracle Database Sharding Topology deployed by Sharding Controller in Oracle Database Operator has an external IP available for each of the container. + +## Below is an example setup with connection details + +Check the details of the Sharding Topology provisioned using Sharding Controller: + +```sh +$ kubectl get all -n shns +NAME READY STATUS RESTARTS AGE +pod/catalog-0 1/1 Running 0 10d +pod/gsm1-0 1/1 Running 0 10d +pod/gsm2-0 1/1 Running 0 10d +pod/shard1-0 1/1 Running 0 10d +pod/shard2-0 1/1 Running 0 10d + +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +service/catalog ClusterIP None 1521/TCP,6234/TCP,6123/TCP,8080/TCP 10d +service/catalog0-svc LoadBalancer xx.xx.xx.12 xx.xx.xx.116 1521:30079/TCP,6234:30498/TCP,6123:31764/TCP,8080:31729/TCP 10d +service/gsm1 ClusterIP None 1522/TCP,6234/TCP,6123/TCP,8080/TCP 10d +service/gsm10-svc LoadBalancer xx.xx.xx.146 xx.xx.xx.38 1522:31401/TCP,6234:31860/TCP,6123:31383/TCP,8080:31892/TCP 10d +service/gsm2 ClusterIP None 1522/TCP,6234/TCP,6123/TCP,8080/TCP 10d +service/gsm20-svc LoadBalancer xx.xx.xx.135 xx.xx.xx.66 1522:30036/TCP,6234:31856/TCP,6123:32095/TCP,8080:32162/TCP 10d +service/shard1 ClusterIP None 1521/TCP,6234/TCP,6123/TCP,8080/TCP 10d +service/shard10-svc LoadBalancer xx.xx.xx.44 xx.xx.xx.187 1521:30716/TCP,6234:30246/TCP,6123:32538/TCP,8080:31174/TCP 10d +service/shard2 ClusterIP None 1521/TCP,6234/TCP,6123/TCP,8080/TCP 10d +service/shard20-svc LoadBalancer xx.xx.xx.83 xx.xx.xx.197 1521:31399/TCP,6234:32088/TCP,6123:30609/TCP,8080:31978/TCP 10d + +NAME READY AGE +statefulset.apps/catalog 1/1 10d +statefulset.apps/gsm1 1/1 10d +statefulset.apps/gsm2 1/1 10d +statefulset.apps/shard1 1/1 10d +statefulset.apps/shard2 1/1 10d +``` + +After you have the external IP address, you can use the services shown below to make the database connection using the above example: + +1. **Direct connection to the CATALOG Database**: Connect to the service `catalogpdb` on catalog container external IP `xx.xx.xx.116` on port `1521` +2. **Direct connection to the shard Database SHARD1**: Connect to the service `shard1pdb` on catalog container external IP `xx.xx.xx.187` on port `1521` +3. **Direct connection to the shard Database SHARD2**: Connect to the service `shard2pdb` on catalog container external IP `xx.xx.xx.197` on port `1521` +4. **Connection to SHARDED Database for DML activity (INSERT/UPDATE/DELETE)**: Connect to the service `oltp_rw_svc.catalog.oradbcloud` either on primary gsm GSM1 container external IP `xx.xx.xx.38` on port `1522` **or** on standby gsm GSM2 container external IP `xx.xx.xx.66` on port `1522` +5. **Connection to the catalog database for DDL activity**: Connect to the service `GDS$CATALOG.oradbcloud` on catalog container external IP `xx.xx.xx.116` on port `1521` diff --git a/doc/sharding/provisioning/debugging.md b/doc/sharding/provisioning/debugging.md new file mode 100644 index 00000000..545bf034 --- /dev/null +++ b/doc/sharding/provisioning/debugging.md @@ -0,0 +1,43 @@ +# Debugging and Troubleshooting + +When the Oracle Database Sharding Topology is provisioned using the Oracle Database Kubernetes Operator, the debugging of an issue with the deployment depends on at which stage the issue has been seen. + +Below are the possible cases and the steps to debug such an issue: + +## Failure during the provisioning of Kubernetes Pods + +In case the failure occurs during the provisioning, we need to check the status of the Kubernetes Pod which has failed to deployed. + +Use the below command to check the logs of the Pod which has a failure. For example, for failure in case of Pod `pod/catalog-0`, use below command: + +```sh +kubectl logs -f pod/catalog-0 -n shns +``` + +In case the Pod has failed to provision due to an issue with the Docker Image, you will see the error `Error: ErrImagePull` in above logs. + +If the Pod has not yet got initialized, use the below command to find the reason for it: + +```sh +kubectl describe pod/catalog-0 -n shns +``` + +In case the failure is related to the Cloud Infrastructure, you will need to troubleshooting that using the documentation from the cloud provider. + +## Failure in the provisioning of the Sharded Database + +In case the failure occures after the Kubernetes Pods are created but during the execution of the scripts to create the shard databases, catalog database or the GSM, you will need to trobleshoot that at the individual Pod level. + +Initially, check the logs of the Kubernetes Pod using the command like below (change the name of the Pod with the actual Pod) + +```sh +kubectl logs -f pod/catalog-0 -n shns +``` + +To check the logs at the GSM or at the Database level or at the host level, switch to the corresponding Kubernetes container using the command like below: + +```sh +kubectl exec -it catalog-0 -n shns /bin/bash +``` + +Now, you can troubleshooting the corresponding component using the alert log or the trace files etc just like a normal Sharding Database Deployment. Please refer to [Oracle Database Sharding Documentation](https://docs.oracle.com/en/database/oracle/oracle-database/19/shard/sharding-troubleshooting.html#GUID-629262E5-7910-4690-A726-A565C59BA73E) for this purpose. diff --git a/doc/sharding/provisioning/oraclesi.yaml b/doc/sharding/provisioning/oraclesi.yaml new file mode 100644 index 00000000..ffc5734b --- /dev/null +++ b/doc/sharding/provisioning/oraclesi.yaml @@ -0,0 +1,99 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: + name: oshard-gold-image-pvc19c + namespace: shns + labels: + app: oshard19cdb-dep +spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 50Gi + storageClassName: oci + selector: + matchLabels: + failure-domain.beta.kubernetes.io/zone: "EU-FRANKFURT-1-AD-1" +--- +apiVersion: apps/v1 +kind: StatefulSet +metadata: + name: oshard19cdb + namespace: shns + labels: + app: oshard19cdb-dep +spec: + selector: + matchLabels: + app: oshard19cdb-dep + serviceName: gold-shard + template: + metadata: + labels: + app: oshard19cdb-dep + spec: + containers: + - image: container-registry.oracle.com/database/enterprise:latest + name: oshard19cdb + ports: + - containerPort: 1521 + name: db1-dbport + - containerPort: 6123 + name: db1-onslport + - containerPort: 6234 + name: db1-onsrport + - containerPort: 8080 + name: db1-agentport + volumeMounts: + - name: data + mountPath: /opt/oracle/oradata + - name: dshm + mountPath: /dev/shm + env: + - name: ORACLE_SID + value: GOLDCDB + - name: ORACLE_PDB + value: GOLDPDB + imagePullSecrets: + - name: ocr-reg-cred + securityContext: + runAsNonRoot: true + runAsUser: 54321 + fsGroup: 54321 + volumes: + - name: data + persistentVolumeClaim: + claimName: oshard-gold-image-pvc19c + - name: dshm + emptyDir: + medium: Memory + nodeSelector: + failure-domain.beta.kubernetes.io/zone: "EU-FRANKFURT-1-AD-1" +--- +apiVersion: v1 +kind: Service +metadata: + name: oshard19cdb + namespace: shns +spec: + ports: + - name: db1-dbport + protocol: TCP + port: 1521 + targetPort: db1-dbport + - name: db1-onslport + protocol: TCP + port: 6123 + targetPort: db1-onslport + - name: db1-onsrport + protocol: TCP + port: 6234 + targetPort: db1-onsrport + selector: + app: oshard19cdb-dep + diff --git a/doc/sharding/provisioning/oraclesi_pvc_commented.yaml b/doc/sharding/provisioning/oraclesi_pvc_commented.yaml new file mode 100644 index 00000000..a6facda0 --- /dev/null +++ b/doc/sharding/provisioning/oraclesi_pvc_commented.yaml @@ -0,0 +1,100 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +# apiVersion: v1 +# kind: PersistentVolumeClaim +# metadata: +# name: oshard-gold-image-pvc19c +# namespace: shns +# labels: +# app: oshard19cdb-dep +# spec: +# accessModes: +# - ReadWriteOnce +# resources: +# requests: +# storage: 50Gi +# storageClassName: oci +# selector: +# matchLabels: +# failure-domain.beta.kubernetes.io/zone: "EU-FRANKFURT-1-AD-1" +# --- +apiVersion: apps/v1 +kind: StatefulSet +metadata: + name: oshard19cdb + namespace: shns + labels: + app: oshard19cdb-dep +spec: + selector: + matchLabels: + app: oshard19cdb-dep + serviceName: gold-shard + template: + metadata: + labels: + app: oshard19cdb-dep + spec: + containers: + - image: container-registry.oracle.com/database/enterprise:latest + name: oshard19cdb + ports: + - containerPort: 1521 + name: db1-dbport + - containerPort: 6123 + name: db1-onslport + - containerPort: 6234 + name: db1-onsrport + - containerPort: 8080 + name: db1-agentport + volumeMounts: + - name: data + mountPath: /opt/oracle/oradata + - name: dshm + mountPath: /dev/shm + env: + - name: ORACLE_SID + value: GOLDCDB + - name: ORACLE_PDB + value: GOLDPDB + imagePullSecrets: + - name: ocr-reg-cred + securityContext: + runAsNonRoot: true + runAsUser: 54321 + fsGroup: 54321 + volumes: + - name: data + persistentVolumeClaim: + claimName: oshard-gold-image-pvc19c + - name: dshm + emptyDir: + medium: Memory + nodeSelector: + failure-domain.beta.kubernetes.io/zone: "EU-FRANKFURT-1-AD-1" +--- +apiVersion: v1 +kind: Service +metadata: + name: oshard19cdb + namespace: shns +spec: + ports: + - name: db1-dbport + protocol: TCP + port: 1521 + targetPort: db1-dbport + - name: db1-onslport + protocol: TCP + port: 6123 + targetPort: db1-onslport + - name: db1-onsrport + protocol: TCP + port: 6234 + targetPort: db1-onsrport + selector: + app: oshard19cdb-dep + diff --git a/doc/sharding/provisioning/provisioning_by_cloning_db_from_gold_image_across_ads.md b/doc/sharding/provisioning/provisioning_by_cloning_db_from_gold_image_across_ads.md new file mode 100644 index 00000000..ae1d7f50 --- /dev/null +++ b/doc/sharding/provisioning/provisioning_by_cloning_db_from_gold_image_across_ads.md @@ -0,0 +1,47 @@ +# Provisioning Oracle Database Sharding Topology by Cloning the Database from Your Own Database Gold Image Across Availability Domains (ADs) + +In this test case, you provision the Oracle Database sharding topology while provisioning the Catalog and Shard Databases by cloning from an existing Oracle Database Gold Image created earlier. + +This use case applies when you want to provision the database Pods on a Kubernetes Node in any availability domain (AD), which can also be different from the availability domain (AD) of the Block Volume that has the Oracle Database Gold Image provisioned earlier. + +Choosing this option takes substantially less time during the Oracle Database Sharding Topology setup across ADs. + +NOTE: + +* Cloning from Block Volume Backup in OCI enables the new Persistent Volumes to be created in other ADs. +* To specify the AD where you want to provision the database Pod, use the tag `nodeSelector` and the POD will be provisioned in a node running in that AD. +* To specify GSM containers, you can also use the tag `nodeSelector` to specify the AD. +* Before you can provision with the Gold Image, you need the OCID of the Persistent Volume that has the Oracle Database Gold Image. + +1. Check the OCID of the Persistent Volume provisioned for the Oracle Database Gold Image: + ```sh + kubectl get pv -n shns + ``` +2. Create a Block Volume Backup for this Block Volume, and use the OCID of the Block Volume Backup in the next step. This example uses `shard_prov_clone_across_ads.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: + +* Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` +* Two sharding Pods: `shard1` and `shard2` +* One Catalog Pod: `catalog` +* Namespace: `shns` +* Database Cloning from the `BLOCK VOLUME FULL BACKUP` of the Persistent Volume which had the Gold Image. +* OCID of the Block Volume Backup: `ocid1.volumebackup.oc1.eu-frankfurt-1.abtheljtjlc7oce3sgq55vnskb4sjdip5sdaighm54hpmlcg7avgc76pjbea` + +In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) + * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. + * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `shard_prov_clone_across_ads.yaml`. + * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../ORACLE_SHARDING_CONTROLLER_README.md#3-oracle-database-and-global-data-services-docker-images) + +Use the file: [shard_prov_clone_across_ads.yaml](./doc/sharding/provisioning/shard_prov_clone_across_ads.yaml) for this use case as below: + +1. Deploy the `shard_prov_clone_across_ads.yaml` file: + ```sh + kubectl apply -f shard_prov_clone_across_ads.yaml + ``` +2. Check the status of the deployment: + ```sh + # Check the status of the Kubernetes Pods: + kubectl get all -n shns + + # Check the logs of a particular pod. For example, to check status of pod "shard1-0": + kubectl logs -f pod/shard1-0 -n shns + diff --git a/doc/sharding/provisioning/provisioning_by_cloning_db_gold_image_in_same_ad.md b/doc/sharding/provisioning/provisioning_by_cloning_db_gold_image_in_same_ad.md new file mode 100644 index 00000000..d633989f --- /dev/null +++ b/doc/sharding/provisioning/provisioning_by_cloning_db_gold_image_in_same_ad.md @@ -0,0 +1,43 @@ +# Provisioning Oracle Database Sharding Topology by Cloning the Database from Your Own Database Gold Image in the same Availability Domain (AD) + +In this case, the database is created automatically by cloning from an existing Oracle Database Gold Image during the provisioning of the shard databases and the catalog database when the Oracle Sharding topology is deployed using Oracle Sharding controller. + +This use case applies when you are cloning from a Block Volume, and you can clone _only_ in the same availability domain (AD). The result is that the cloned shard database PODs can be created _only_ in the same AD where the Gold Image Block Volume is present. + +Choosing this option takes substantially less time during the Oracle Database Sharding Topology setup. + +**NOTE** For this step, the Persistent Volume that has the Oracle Database Gold Image is identified using its OCID. + +1. Check the OCID of the Persistent Volume provisioned by above step using below command: + + ```sh + kubectl get pv -n shns + ``` + +2. This example uses `shard_prov_clone.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: + +* Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` +* Two sharding Pods: `shard1` and `shard2` +* One Catalog Pod: `catalog` +* Namespace: `shns` +* Database Cloning from the Database Gold Image present in Persistent Volume having OCID: `ocid1.volume.oc1.eu-frankfurt-1.abtheljtmwcwf7liuhaibzgdcoxqcwwfpsqiqlsumrjlzkin7y4zx3x2idua` + +In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) + * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. + * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `shard_prov.yaml`. + * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../ORACLE_SHARDING_CONTROLLER_README.md#3-oracle-database-and-global-data-services-docker-images) + +Use the file: [shard_prov_clone.yaml](./doc/sharding/provisioning/shard_prov_clone.yaml) for this use case as below: + +1. Deploy the `shard_prov_clone.yaml` file: + ```sh + kubectl apply -f shard_prov_clone.yaml + ``` +2. Check the status of the deployment: + ```sh + # Check the status of the Kubernetes Pods: + kubectl get all -n shns + + # Check the logs of a particular pod. For example, to check status of pod "shard1-0": + kubectl logs -f pod/shard1-0 -n shns + ``` diff --git a/doc/sharding/provisioning/provisioning_persistent_volume_having_db_gold_image.md b/doc/sharding/provisioning/provisioning_persistent_volume_having_db_gold_image.md new file mode 100644 index 00000000..106fbb85 --- /dev/null +++ b/doc/sharding/provisioning/provisioning_persistent_volume_having_db_gold_image.md @@ -0,0 +1,40 @@ +# Provisioning a Persistent Volume Containing a Database Gold Image + + In this use case, a Persistent Volume with a Oracle Database Gold Image is created. + + This is required when you do not already have a Persistent Volume with a Database Gold Image from which you can clone database to save time while deploying Oracle Sharding topology using Oracle Sharding controller. + +This example uses file `oraclesi.yaml` to provision a single instance Oracle Database: + +* A Persistent Volume Claim +* Repository location for Database Docker Image: `image: container-registry.oracle.com/database/enterprise:latest` +* Namespace: `shns` +* Tag `nodeSelector` to deploy the Single Oracle Database in AD `EU-FRANKFURT-1-AD-1` + +In this example, we are using pre-built Oracle Database image available on [Oracle Container Registry](https://container-registry.oracle.com/) + * To pull the above image from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. + * If you plan to use image built by you, you need to change `image` tag with the image you have built in your enviornment in file `oraclesi.yaml`. + * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../ORACLE_SHARDING_CONTROLLER_README.md#3-oracle-database-and-global-data-services-docker-images) + +1. Use this YAML file: [oraclesi.yaml](./doc/sharding/provisioning/oraclesi.yaml) for this use case. + + Use the following command to deploy the `oraclesi.yaml` file: + + ```sh + kubectl apply -f oraclesi.yaml + ``` + +2. After the Database Deployment is completed, switch to the Single Oracle Database Pod and confirm that the Database Instance is UP and running in RW Mode. +3. Shut down the database instance cleanly, using `shutdown immediate` from the SQLPLUS Prompt. +4. For this use case, use the modified file [oraclesi_pvc_commented.yaml](./doc/sharding/provisioning/oraclesi_pvc_commented.yaml). This file is a copy of the file `oraclesi.yaml` but to keep the Persistent Volume claim and delete the database deployment, it has the lines of the Persistent Volume Claim commended out. +5. Delete all the components provisioned for Single Oracle Database Deployment EXCEPT the Persistent Volume Claim by applying this file: + + ```sh + kubectl delete -f oraclesi_pvc_commented.yaml + ``` + +6. Check the OCID of the Persistent Volume provisioned by the preceding step using this command: + + ```sh + kubectl get pv -n shns + ``` diff --git a/doc/sharding/provisioning/provisioning_with_control_on_resources.md b/doc/sharding/provisioning/provisioning_with_control_on_resources.md new file mode 100644 index 00000000..e30ce506 --- /dev/null +++ b/doc/sharding/provisioning/provisioning_with_control_on_resources.md @@ -0,0 +1,39 @@ +# Provisioning Oracle Database Sharding Topology with Additional Control on Resources Allocated to Pods + +In this use case, there are additional tags used to control resources such as CPU and Memory used by the different Pods when the Oracle Sharding topology is deployed using Oracle Sharding controller. + +This example uses `shard_prov_memory_cpu.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: + +* Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` +* Two sharding Pods: `shard1` and `shard2` +* One Catalog Pod: `catalog` +* Namespace: `shns` +* Tags `memory` and `cpu` to control the Memory and CPU of the PODs +* Additional tags `INIT_SGA_SIZE` and `INIT_PGA_SIZE` to control the SGA and PGA allocation at the database level + +In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) + * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. + * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `shard_prov_memory_cpu.yaml`. + * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../ORACLE_SHARDING_CONTROLLER_README.md#3-oracle-database-and-global-data-services-docker-images) + +Use the YAML file [shard_prov_memory_cpu.yaml](./doc/sharding/provisioning/shard_prov_memory_cpu.yaml). + +1. Deploy the `shard_prov_memory_cpu.yaml` file: + + ```sh + kubectl apply -f shard_prov_memory_cpu.yaml + ``` + +1. Check the details of a POD. For example: To check the details of Pod `shard1-0`: + + ```sh + kubectl describe pod/shard1-0 -n shns + ``` +3. Check the status of the deployment: + ```sh + # Check the status of the Kubernetes Pods: + kubectl get all -n shns + + # Check the logs of a particular pod. For example, to check status of pod "shard1-0": + kubectl logs -f pod/shard1-0 -n shns + ``` diff --git a/doc/sharding/provisioning/provisioning_with_notification_using_oci_notification.md b/doc/sharding/provisioning/provisioning_with_notification_using_oci_notification.md new file mode 100644 index 00000000..a551b401 --- /dev/null +++ b/doc/sharding/provisioning/provisioning_with_notification_using_oci_notification.md @@ -0,0 +1,79 @@ +# Provisioning Oracle Database Sharding Topology and Send Notification Using OCI Notification Service + +This use case demonstrates how to use a notification service like OCI Notification service to send an email notification when a particular operation is completed on an Oracle Database sharding topology provisioned using the Oracle Database sharding controller. + +This example uses `shard_prov_send_notification.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: + +* Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` +* Two sharding Pods: `shard1` and `shard2` +* One Catalog Pod: `catalog` +* Namespace: `shns` +* Database Cloning from the `BLOCK VOLUME FULL BACKUP` of the Persistent Volume that has the Database Gold Image created earlier. +* OCID of the Block Volume Backup: `ocid1.volumebackup.oc1.eu-frankfurt-1.abtheljtjlc7oce3sgq55vnskb4sjdip5sdaighm54hpmlcg7avgc76pjbea` +* Configmap to send notification email when a particular operation is completed. For example: When a shard is added. + +**NOTE:** + +* The notification will be sent using a configmap created with the credentials of the OCI user account in this use case. + +We will create a topic in Notification Service of the OCI Console and use its OCID. + +To do this: + +1. Create a `configmap_data.txt` file, such as the following, which has the OCI User details that will be used to send notfication: + + ```sh + user=ocid1.user.oc1........fx7omxfq + fingerprint=fa:18:98:...............:8a + tenancy=ocid1.tenancy.oc1..aaaa.......orpn7inq + region=eu-frankfurt-1 + topicid=ocid1.onstopic.oc1.eu-frankfurt-1.aaa............6xrq + ``` +2. Create a configmap using the below command using the file created above: + ```sh + kubectl create configmap onsconfigmap --from-file=./configmap_data.txt -n shns + ``` + +3. Create a key file `priavatekey` having the PEM key of the OCI user being used to send notification: + ```sh + -----BEGIN PRIVATE KEY-G---- + MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQCXYxA0DJvEwtVR + +o4OxrunL3L2NZJRADTFR+TDHqrNF1JwbaFBizSdL+EXbxQW1faZs5lXZ/sVmQF9 + . + . + . + zn/xWC0FzXGRzfvYHhq8XT3omf6L47KqIzqo3jDKdgvVq4u+lb+fXJlhj6Rwi99y + QEp36HnZiUxAQnR331DacN+YSTE+vpzSwZ38OP49khAB1xQsbiv1adG7CbNpkxpI + nS7CkDLg4Hcs4b9bGLHYJVY= + -----END PRIVATE KEY----- + ``` +4. Use the key file `privatekey` to create a Kubernetes secret in namespace `shns`: + + ```sh + kubectl create secret generic my-secret --from-file=./privatekey -n shns + ``` + +5. Use this command to check details of the secret that you created: + + ```sh + kubectl describe secret my-secret -n shns + ``` + +In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) + * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. + * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `shard_prov.yaml`. + * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../ORACLE_SHARDING_CONTROLLER_README.md#3-oracle-database-and-global-data-services-docker-images) + +Use the file: [shard_prov_send_notification.yaml](./doc/sharding/provisioning/shard_prov_send_notification.yaml) for this use case as below: + +1. Deploy the `shard_prov_send_notification.yaml` file: + ```sh + kubectl apply -f shard_prov_send_notification.yaml + ``` +2. Check the status of the deployment: + ```sh + # Check the status of the Kubernetes Pods: + kubectl get all -n shns + + # Check the logs of a particular pod. For example, to check status of pod "shard1-0": + kubectl logs -f pod/shard1-0 -n shns diff --git a/doc/sharding/provisioning/provisioning_without_db_gold_image.md b/doc/sharding/provisioning/provisioning_without_db_gold_image.md new file mode 100644 index 00000000..d806b97c --- /dev/null +++ b/doc/sharding/provisioning/provisioning_without_db_gold_image.md @@ -0,0 +1,34 @@ +# Provisioning Oracle Database Sharding Topology Without Database Gold Image + +In this use case, the database is created automatically using DBCA during the provisioning of the shard databases and the catalog database when the Oracle Sharding topology is deployed using Oracle Sharding controller. + +**NOTE** In this use case, because DBCA creates the database automatically during the deployment, the time required to create the database is greater than the time it takes when the database is created by cloning from a Database Gold Image. + +This example uses `shard_prov.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: + +* Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` +* Two sharding Pods: `shard1` and `shard2` +* One Catalog Pod: `catalog` +* Namespace: `shns` + + +In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) + * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. + * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `shard_prov.yaml`. + * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../ORACLE_SHARDING_CONTROLLER_README.md#3-oracle-database-and-global-data-services-docker-images) + + +Use the file: [shard_prov.yaml](./doc/sharding/provisioning/shard_prov.yaml) for this use case as below: + +1. Deploy the `shard_prov.yaml` file: + ```sh + kubectl apply -f shard_prov.yaml + ``` +1. Check the status of the deployment: + ```sh + # Check the status of the Kubernetes Pods: + kubectl get all -n shns + + # Check the logs of a particular pod. For example, to check status of pod "shard1-0": + kubectl logs -f pod/shard1-0 -n shns + ``` diff --git a/doc/sharding/provisioning/scale_in_delete_an_existing_shard.md b/doc/sharding/provisioning/scale_in_delete_an_existing_shard.md new file mode 100644 index 00000000..8f793c96 --- /dev/null +++ b/doc/sharding/provisioning/scale_in_delete_an_existing_shard.md @@ -0,0 +1,44 @@ +# Scale In - Delete an existing Shard From a Working Oracle Database Sharding Topology + +This use case demonstrates how to delete an existing Shard from an existing Oracle Database sharding topology provisioned using Oracle Database Sharding controller. + +**NOTE** The deletion of a shard is done after verifying the Chunks have been moved out of that shard. + +In this use case, the existing database Sharding is having: + +* Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` +* Five sharding Pods: `shard1`,`shard2`,`shard3`,`shard4` and `shard5` +* One Catalog Pod: `catalog` +* Namespace: `shns` + +In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) + * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. + * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `shard_prov.yaml`. + * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../ORACLE_SHARDING_CONTROLLER_README.md#3-oracle-database-and-global-data-services-docker-images) + +NOTE: Use tag `isDelete: true` to delete the shard you want. + +This use case deletes the shard `shard2` from the above Sharding Topology. + +Use the file: [shard_prov_delshard.yaml](./doc/sharding/provisioning/shard_prov_delshard.yaml) for this use case as below: + +1. Deploy the `shard_prov_delshard.yaml` file: + ```sh + kubectl apply -f shard_prov_delshard.yaml + ``` +2. Check the status of the deployment: + ```sh + # Check the status of the Kubernetes Pods: + kubectl get all -n shns + +**NOTE:** After you apply `shard_prov_delshard.yaml`, the change may not be visible immediately. When the shard is removed, first the chunks will be moved out of that shard that is going to be deleted. + +To monitor the chunk movement, use the following command: + +```sh +# Switch to the primary GSM Container: +kubectl exec -i -t gsm1-0 -n shns /bin/bash + +# Check the status of the chunks and repeat to observe the chunk movement: +gdsctl config chunks +``` diff --git a/doc/sharding/provisioning/scale_out_add_shards.md b/doc/sharding/provisioning/scale_out_add_shards.md new file mode 100644 index 00000000..a8bb5b11 --- /dev/null +++ b/doc/sharding/provisioning/scale_out_add_shards.md @@ -0,0 +1,31 @@ +# Scale Out - Add Shards to an existing Oracle Database Sharding Topology + +This use case demonstrates adding a new shard to an existing Oracle Database sharding topology provisioned earlier using Oracle Database Sharding controller. + +In this use case, the existing Oracle Database sharding topology is having: + +* Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` +* Two sharding Pods: `shard1` and `shard2` +* One Catalog Pod: `catalog` +* Namespace: `shns` + +In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) + * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. + * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `shard_prov.yaml`. + * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../ORACLE_SHARDING_CONTROLLER_README.md#3-oracle-database-and-global-data-services-docker-images) + +This use case adds three new shards `shard3`,`shard4`,`shard4` to above Sharding Topology. + +Use the file: [shard_prov_extshard.yaml](./doc/sharding/provisioning/shard_prov_extshard.yaml) for this use case as below: + +1. Deploy the `shard_prov_extshard.yaml` file: + ```sh + kubectl apply -f shard_prov_extshard.yaml + ``` +2. Check the status of the deployment: + ```sh + # Check the status of the Kubernetes Pods: + kubectl get all -n shns + + # Check the logs of a particular pod. For example, to check status of pod "shard3-0": + kubectl logs -f pod/shard3-0 -n shns diff --git a/doc/sharding/provisioning/shard_prov.yaml b/doc/sharding/provisioning/shard_prov.yaml new file mode 100644 index 00000000..508192d7 --- /dev/null +++ b/doc/sharding/provisioning/shard_prov.yaml @@ -0,0 +1,46 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: ShardingDatabase +metadata: + name: shardingdatabase-sample + namespace: shns +spec: + shard: + - name: shard1 + storageSizeInGb: 50 + - name: shard2 + storageSizeInGb: 50 + catalog: + - name: catalog + storageSizeInGb: 50 + gsm: + - name: gsm1 + storageSizeInGb: 50 + replicas: 1 + envVars: + - name: "SERVICE1_PARAMS" + value: "service_name=oltp_rw_svc;service_role=primary" + - name: "SERVICE2_PARAMS" + value: "service_name=oltp_ro_svc;service_role=primary" + - name: gsm2 + storageSizeInGb: 50 + replicas: 1 + envVars: + - name: "SERVICE1_PARAMS" + value: "service_name=oltp_rw_svc;service_role=primary" + - name: "SERVICE2_PARAMS" + value: "service_name=oltp_ro_svc;service_role=primary" + storageClass: oci + dbImage: container-registry.oracle.com/database/enterprise:latest + dbImagePullSecret: ocr-reg-cred + gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImagePullSecret: ocr-reg-cred + scriptsLocation: "set -ex;curl https://codeload.github.com/oracle/db-sharding/tar.gz/master | tar -xz --strip=4 db-sharding-master/docker-based-sharding-deployment/dockerfiles/21.3.0/scripts; cp -i -r scripts/* /opt/oracle/scripts/sharding/scripts/;cp -i -r scripts/*py /opt/oracle/scripts/sharding" + secret: db-user-pass + isExternalSvc: false + isDeleteOraPvc: True + namespace: shns + diff --git a/doc/sharding/provisioning/shard_prov_clone.yaml b/doc/sharding/provisioning/shard_prov_clone.yaml new file mode 100644 index 00000000..0acc0ec5 --- /dev/null +++ b/doc/sharding/provisioning/shard_prov_clone.yaml @@ -0,0 +1,64 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: ShardingDatabase +metadata: + name: shardingdatabase-sample + namespace: shns +spec: + shard: + - name: shard1 + storageSizeInGb: 50 + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-1" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-1" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volume.oc1.eu-frankfurt-1.abtheljtmwcwf7liuhaibzgdcoxqcwwfpsqiqlsumrjlzkin7y4zx3x2idua + - name: shard2 + storageSizeInGb: 50 + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-1" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-1" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volume.oc1.eu-frankfurt-1.abtheljtmwcwf7liuhaibzgdcoxqcwwfpsqiqlsumrjlzkin7y4zx3x2idua + catalog: + - name: catalog + storageSizeInGb: 50 + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-1" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-1" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volume.oc1.eu-frankfurt-1.abtheljtmwcwf7liuhaibzgdcoxqcwwfpsqiqlsumrjlzkin7y4zx3x2idua + gsm: + - name: gsm1 + storageSizeInGb: 50 + replicas: 1 + envVars: + - name: "SERVICE1_PARAMS" + value: "service_name=oltp_rw_svc;service_role=primary" + - name: "SERVICE2_PARAMS" + value: "service_name=oltp_ro_svc;service_role=primary" + - name: gsm2 + storageSizeInGb: 50 + replicas: 1 + envVars: + - name: "SERVICE1_PARAMS" + value: "service_name=oltp_rw_svc;service_role=primary" + - name: "SERVICE2_PARAMS" + value: "service_name=oltp_ro_svc;service_role=primary" + storageClass: oci + dbImage: container-registry.oracle.com/database/enterprise:latest + dbImagePullSecret: ocr-reg-cred + gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImagePullSecret: ocr-reg-cred + scriptsLocation: "set -ex;curl https://codeload.github.com/oracle/db-sharding/tar.gz/master | tar -xz --strip=4 db-sharding-master/docker-based-sharding-deployment/dockerfiles/21.3.0/scripts; cp -i -r scripts/* /opt/oracle/scripts/sharding/scripts/;cp -i -r scripts/*py /opt/oracle/scripts/sharding" + secret: db-user-pass + isExternalSvc: false + isClone: True + isDeleteOraPvc: True + namespace: shns diff --git a/doc/sharding/provisioning/shard_prov_clone_across_ads.yaml b/doc/sharding/provisioning/shard_prov_clone_across_ads.yaml new file mode 100644 index 00000000..98d9ee56 --- /dev/null +++ b/doc/sharding/provisioning/shard_prov_clone_across_ads.yaml @@ -0,0 +1,72 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: ShardingDatabase +metadata: + name: shardingdatabase-sample + namespace: shns +spec: + shard: + - name: shard1 + storageSizeInGb: 50 + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-2" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-2" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volumebackup.oc1.eu-frankfurt-1.abtheljtjlc7oce3sgq55vnskb4sjdip5sdaighm54hpmlcg7avgc76pjbea + - name: shard2 + storageSizeInGb: 50 + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-1" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-1" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volumebackup.oc1.eu-frankfurt-1.abtheljtjlc7oce3sgq55vnskb4sjdip5sdaighm54hpmlcg7avgc76pjbea + catalog: + - name: catalog + storageSizeInGb: 50 + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-2" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-2" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volumebackup.oc1.eu-frankfurt-1.abtheljtjlc7oce3sgq55vnskb4sjdip5sdaighm54hpmlcg7avgc76pjbea + gsm: + - name: gsm1 + storageSizeInGb: 50 + replicas: 1 + envVars: + - name: "SERVICE1_PARAMS" + value: "service_name=oltp_rw_svc;service_role=primary" + - name: "SERVICE2_PARAMS" + value: "service_name=oltp_ro_svc;service_role=primary" + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-2" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-2" + - name: gsm2 + storageSizeInGb: 50 + replicas: 1 + envVars: + - name: "SERVICE1_PARAMS" + value: "service_name=oltp_rw_svc;service_role=primary" + - name: "SERVICE2_PARAMS" + value: "service_name=oltp_ro_svc;service_role=primary" + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-3" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-3" + storageClass: oci + dbImage: container-registry.oracle.com/database/enterprise:latest + dbImagePullSecret: ocr-reg-cred + gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImagePullSecret: ocr-reg-cred + scriptsLocation: "set -ex;curl https://codeload.github.com/oracle/db-sharding/tar.gz/master | tar -xz --strip=4 db-sharding-master/docker-based-sharding-deployment/dockerfiles/21.3.0/scripts; cp -i -r scripts/* /opt/oracle/scripts/sharding/scripts/;cp -i -r scripts/*py /opt/oracle/scripts/sharding" + secret: db-user-pass + isExternalSvc: false + isClone: True + isDeleteOraPvc: True + namespace: shns diff --git a/doc/sharding/provisioning/shard_prov_delshard.yaml b/doc/sharding/provisioning/shard_prov_delshard.yaml new file mode 100644 index 00000000..91dfdd28 --- /dev/null +++ b/doc/sharding/provisioning/shard_prov_delshard.yaml @@ -0,0 +1,52 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: ShardingDatabase +metadata: + name: shardingdatabase-sample + namespace: shns +spec: + shard: + - name: shard1 + storageSizeInGb: 50 + - name: shard2 + storageSizeInGb: 50 + isDelete: true + - name: shard3 + storageSizeInGb: 50 + - name: shard4 + storageSizeInGb: 50 + - name: shard5 + storageSizeInGb: 50 + catalog: + - name: catalog + storageSizeInGb: 50 + gsm: + - name: gsm1 + storageSizeInGb: 50 + replicas: 1 + envVars: + - name: "SERVICE1_PARAMS" + value: "service_name=oltp_rw_svc;service_role=primary" + - name: "SERVICE2_PARAMS" + value: "service_name=oltp_ro_svc;service_role=primary" + - name: gsm2 + storageSizeInGb: 50 + replicas: 1 + envVars: + - name: "SERVICE1_PARAMS" + value: "service_name=oltp_rw_svc;service_role=primary" + - name: "SERVICE2_PARAMS" + value: "service_name=oltp_ro_svc;service_role=primary" + storageClass: oci + dbImage: container-registry.oracle.com/database/enterprise:latest + dbImagePullSecret: ocr-reg-cred + gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImagePullSecret: ocr-reg-cred + scriptsLocation: "set -ex;curl https://codeload.github.com/oracle/db-sharding/tar.gz/master | tar -xz --strip=4 db-sharding-master/docker-based-sharding-deployment/dockerfiles/21.3.0/scripts; cp -i -r scripts/* /opt/oracle/scripts/sharding/scripts/;cp -i -r scripts/*py /opt/oracle/scripts/sharding" + secret: db-user-pass + isExternalSvc: false + isDeleteOraPvc: True + namespace: shns diff --git a/doc/sharding/provisioning/shard_prov_extshard.yaml b/doc/sharding/provisioning/shard_prov_extshard.yaml new file mode 100644 index 00000000..953fe23f --- /dev/null +++ b/doc/sharding/provisioning/shard_prov_extshard.yaml @@ -0,0 +1,51 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: ShardingDatabase +metadata: + name: shardingdatabase-sample + namespace: shns +spec: + shard: + - name: shard1 + storageSizeInGb: 50 + - name: shard2 + storageSizeInGb: 50 + - name: shard3 + storageSizeInGb: 50 + - name: shard4 + storageSizeInGb: 50 + - name: shard5 + storageSizeInGb: 50 + catalog: + - name: catalog + storageSizeInGb: 50 + gsm: + - name: gsm1 + storageSizeInGb: 50 + replicas: 1 + envVars: + - name: "SERVICE1_PARAMS" + value: "service_name=oltp_rw_svc;service_role=primary" + - name: "SERVICE2_PARAMS" + value: "service_name=oltp_ro_svc;service_role=primary" + - name: gsm2 + storageSizeInGb: 50 + replicas: 1 + envVars: + - name: "SERVICE1_PARAMS" + value: "service_name=oltp_rw_svc;service_role=primary" + - name: "SERVICE2_PARAMS" + value: "service_name=oltp_ro_svc;service_role=primary" + storageClass: oci + dbImage: container-registry.oracle.com/database/enterprise:latest + dbImagePullSecret: ocr-reg-cred + gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImagePullSecret: ocr-reg-cred + scriptsLocation: "set -ex;curl https://codeload.github.com/oracle/db-sharding/tar.gz/master | tar -xz --strip=4 db-sharding-master/docker-based-sharding-deployment/dockerfiles/21.3.0/scripts; cp -i -r scripts/* /opt/oracle/scripts/sharding/scripts/;cp -i -r scripts/*py /opt/oracle/scripts/sharding" + secret: db-user-pass + isExternalSvc: false + isDeleteOraPvc: True + namespace: shns diff --git a/doc/sharding/provisioning/shard_prov_memory_cpu.yaml b/doc/sharding/provisioning/shard_prov_memory_cpu.yaml new file mode 100644 index 00000000..a542ebab --- /dev/null +++ b/doc/sharding/provisioning/shard_prov_memory_cpu.yaml @@ -0,0 +1,67 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: ShardingDatabase +metadata: + name: shardingdatabase-sample + namespace: shns +spec: + shard: + - name: shard1 + storageSizeInGb: 50 + resources: + requests: + memory: "1000Mi" + cpu: "1000m" + envVars: + - name: "INIT_SGA_SIZE" + value: "600" + - name: "INIT_PGA_SIZE" + value: "400" + - name: shard2 + storageSizeInGb: 50 + resources: + requests: + memory: "1000Mi" + cpu: "1000m" + envVars: + - name: "INIT_SGA_SIZE" + value: "600" + - name: "INIT_PGA_SIZE" + value: "400" + catalog: + - name: catalog + storageSizeInGb: 50 + resources: + requests: + memory: "1000Mi" + cpu: "1000m" + gsm: + - name: gsm1 + storageSizeInGb: 50 + replicas: 1 + envVars: + - name: "SERVICE1_PARAMS" + value: "service_name=oltp_rw_svc;service_role=primary" + - name: "SERVICE2_PARAMS" + value: "service_name=oltp_ro_svc;service_role=primary" + - name: gsm2 + storageSizeInGb: 50 + replicas: 1 + envVars: + - name: "SERVICE1_PARAMS" + value: "service_name=oltp_rw_svc;service_role=primary" + - name: "SERVICE2_PARAMS" + value: "service_name=oltp_ro_svc;service_role=primary" + storageClass: oci + dbImage: container-registry.oracle.com/database/enterprise:latest + dbImagePullSecret: ocr-reg-cred + gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImagePullSecret: ocr-reg-cred + scriptsLocation: "set -ex;curl https://codeload.github.com/oracle/db-sharding/tar.gz/master | tar -xz --strip=4 db-sharding-master/docker-based-sharding-deployment/dockerfiles/21.3.0/scripts; cp -i -r scripts/* /opt/oracle/scripts/sharding/scripts/;cp -i -r scripts/*py /opt/oracle/scripts/sharding" + secret: db-user-pass + isExternalSvc: false + isDeleteOraPvc: True + namespace: shns diff --git a/doc/sharding/provisioning/shard_prov_send_notification.yaml b/doc/sharding/provisioning/shard_prov_send_notification.yaml new file mode 100644 index 00000000..af723c8e --- /dev/null +++ b/doc/sharding/provisioning/shard_prov_send_notification.yaml @@ -0,0 +1,75 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: ShardingDatabase +metadata: + name: shardingdatabase-sample + namespace: shns +spec: + shard: + - name: shard1 + storageSizeInGb: 50 + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-2" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-2" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volumebackup.oc1.eu-frankfurt-1.abtheljtjlc7oce3sgq55vnskb4sjdip5sdaighm54hpmlcg7avgc76pjbea + - name: shard2 + storageSizeInGb: 50 + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-1" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-1" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volumebackup.oc1.eu-frankfurt-1.abtheljtjlc7oce3sgq55vnskb4sjdip5sdaighm54hpmlcg7avgc76pjbea + catalog: + - name: catalog + storageSizeInGb: 50 + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-2" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-2" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volumebackup.oc1.eu-frankfurt-1.abtheljtjlc7oce3sgq55vnskb4sjdip5sdaighm54hpmlcg7avgc76pjbea + gsm: + - name: gsm1 + storageSizeInGb: 50 + replicas: 1 + envVars: + - name: "SERVICE1_PARAMS" + value: "service_name=oltp_rw_svc;service_role=primary" + - name: "SERVICE2_PARAMS" + value: "service_name=oltp_ro_svc;service_role=primary" + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-2" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-2" + - name: gsm2 + storageSizeInGb: 50 + replicas: 1 + envVars: + - name: "SERVICE1_PARAMS" + value: "service_name=oltp_rw_svc;service_role=primary" + - name: "SERVICE2_PARAMS" + value: "service_name=oltp_ro_svc;service_role=primary" + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-3" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-3" + storageClass: oci + dbImage: container-registry.oracle.com/database/enterprise:latest + dbImagePullSecret: ocr-reg-cred + gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImagePullSecret: ocr-reg-cred + scriptsLocation: "set -ex;curl https://codeload.github.com/oracle/db-sharding/tar.gz/master | tar -xz --strip=4 db-sharding-master/docker-based-sharding-deployment/dockerfiles/21.3.0/scripts; cp -i -r scripts/* /opt/oracle/scripts/sharding/scripts/;cp -i -r scripts/*py /opt/oracle/scripts/sharding" + secret: db-user-pass + isExternalSvc: false + isClone: True + isDeleteOraPvc: True + namespace: shns + nsConfigMap: onsconfigmap + nsSecret: my-secret + diff --git a/doc/sidb/ORACLE_SIDB_CONTROLLER_README.md b/doc/sidb/ORACLE_SIDB_CONTROLLER_README.md new file mode 100644 index 00000000..b640aa2f --- /dev/null +++ b/doc/sidb/ORACLE_SIDB_CONTROLLER_README.md @@ -0,0 +1,264 @@ +# Oracle Database Operator for Kubernetes + +Oracle Database Operator for Kubernetes (the operator) includes the Single Instance Database Controller that enables provisioning, cloning, and patching of Oracle Single Instance Databases on Kubernetes. The following sections explain the setup and functionality of the operator + +* [Prerequisites](ORACLE_SIDB_CONTROLLER_README.md#prerequisites-and-setup) +* [Kind SingleInstanceDatabase](ORACLE_SIDB_CONTROLLER_README.md#kind-singleinstancedatabase) +* [Provision New Database](ORACLE_SIDB_CONTROLLER_README.md#provision-new-database) +* [Clone Existing Database](ORACLE_SIDB_CONTROLLER_README.md#clone-existing-database) +* [Patch/Rollback Database](ORACLE_SIDB_CONTROLLER_README.md#patchrollback-database) + + +## Prerequisites + +Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISITES.md). + +## Kind SingleInstanceDatabase Resource + + The Oracle Database Operator creates the SingleInstanceDatabase kind as a custom resource that enables Oracle Database to be managed as a native Kubernetes object + +* ### SingleInstanceDatabase Sample YAML + + For the use cases detailed below a sample .yaml file is available at + * Enterprise, Standard Editions + [config/samples/singleinstancedatabase.yaml](config/samples/singleinstancedatabase.yaml) + + **Note:** The `adminPassword` field of the above `singleinstancedatabase.yaml` yaml contains a secret for Single Instance Database creation (Provisioning a new database or cloning an existing database). This secret gets deleted after the database pod becomes ready for security reasons. + +* ### List Databases + + ```sh + $ kubectl get singleinstancedatabases -o name + + singleinstancedatabase.database.oracle.com/sidb-sample + singleinstancedatabase.database.oracle.com/sidb-sample-clone + + ``` + +* ### Quick Status + + ```sh + $ kubectl get singleinstancedatabase sidb-sample + + NAME EDITION STATUS ROLE VERSION CLUSTER CONNECT STR CONNECT STR OEM EXPRESS URL + sidb-sample Enterprise Healthy PRIMARY 19.3.0.0.0 (29517242) sidb-sample.default:1521/ORCL1 144.25.10.119:1521/ORCL https://144.25.10.119:5500/em + ``` + +* ### Detailed Status + + ```sh + $ kubectl describe singleinstancedatabase sidb-sample-clone + + Name: sidb-sample-clone + Namespace: default + Labels: + Annotations: + API Version: database.oracle.com/v1alpha1 + Kind: SingleInstanceDatabase + Metadata: .... + Spec: .... + Status: + Cluster Connect String: sidb-sample-clone.default:1521/ORCL1C + Conditions: + Last Transition Time: 2021-06-29T15:45:33Z + Message: Waiting for database to be ready + Observed Generation: 2 + Reason: LastReconcileCycleQueued + Status: True + Type: ReconcileQueued + Last Transition Time: 2021-06-30T11:07:56Z + Message: processing datapatch execution + Observed Generation: 3 + Reason: LastReconcileCycleBlocked + Status: True + Type: ReconcileBlocked + Last Transition Time: 2021-06-30T11:16:58Z + Message: no reconcile errors + Observed Generation: 3 + Reason: LastReconcileCycleCompleted + Status: True + Type: ReconcileComplete + Connect String: 144.25.10.119:1521/ORCL1C + Datafiles Created: true + Datafiles Patched: true + Edition: Enterprise + Flash Back: true + Force Log: false + Oem Express URL: https://144.25.10.119:5500/em + Pdb Name: orclpdb1 + Release Update: 19.11.0.0.0 (32545013) + Replicas: 2 + Role: PRIMARY + Sid: ORCL1C + Status: Healthy + Events: + Type Reason Age From Message + ---- ------ ---- ---- ------- + Normal Database Pending 35m (x2 over 35m) SingleInstanceDatabase Waiting for database pod to be ready + Normal Database Creating 27m (x24 over 34m) SingleInstanceDatabase Waiting for database to be ready + Normal Database Ready 22m SingleInstanceDatabase database open on pod sidb-sample-clone-133ol scheduled on node 10.0.10.6 + Normal Datapatch Pending 21m SingleInstanceDatabase datapatch execution pending + Normal Datapatch Executing 20m SingleInstanceDatabase datapatch begin execution + Normal Datapatch Done 8s SingleInstanceDatabase Datapatch from 19.3.0.0.0 to 19.11.0.0.0 : SUCCESS + + ``` + +## Provision New Database + + Provision a new database instance by specifying appropriate values for the attributes in the the example `.yaml` file, and running the following command: + + ```sh + $ kubectl create -f singleinstancedatabase.yaml + + singleinstancedatabase.database.oracle.com/sidb-sample created + ``` + +* ### Creation Status + + Creating a new database instance takes a while. When the 'status' status returns the response "Healthy", the Database is open for connections. + + ```sh +$ kubectl get singleinstancedatabase/sidb-sample --template={{.status.status}} + + Healthy +``` + + + +* ### Connection Information + + External and internal (running in Kubernetes pods) clients can connect to the database using .status.connectString and .status.clusterConnectString + respectively in the following command + + ```sh + $ kubectl get singleinstancedatabase/sidb-sample --template={{.status.connectString}} + + 144.25.10.119:1521/ORCL + ``` + + The Oracle Database inside the container also has Oracle Enterprise Manager Express configured. To access OEM Express, start the browser and follow the URL: + + ```sh + $ kubectl get singleinstancedatabase/sidb-sample --template={{.status.oemExpressUrl}} + + https://144.25.10.119:5500/em + ``` + +* ### Update Database Config + + The following database parameters can be updated post database creation: flashBack, archiveLog, forceLog. Change their attribute values and apply using + kubectl apply or edit/patch commands . Enable archiveLog before turning ON flashBack . Turn OFF flashBack before disabling the archiveLog + + ```sh + $ kubectl --type merge -p '{"spec":{"forceLog": true}}' patch singleinstancedatabase/sidb-sample + + singleinstancedatabase.database.oracle.com/sidb-sample patched + ``` + +* #### Database Config Status + + Check the Database Config Status using the following command + + ```sh + $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath=[{.status.archiveLog}, {.status.flashBack}, {.status.forceLog}]" + + [true, true, true] + ``` + +* ### Update Initialization Parameters + + The following database initialization parameters can be updated post database creation: `sgaTarget, pgaAggregateTarget, cpuCount, processes`. Change their attribute values and apply using kubectl apply or edit/patch commands. + + **NOTE** + * `sgaTarget` should be in range [sga_min_size, sga_max_size], else initialization parameter `sga_target` would not be updated to specified `sgaTarget`. + +* ### Multiple Replicas + + Multiple database pod replicas can be provisioned when the persistent volume access mode is ReadWriteMany. Database is open and mounted by one of the replicas. Other replicas will have instance started but not mounted and serve to provide quick cold fail-over in case the active pod dies. Update the replica attribute in the .yaml and apply using the kubectl apply command or edit/patch commands + + Note: This functionality requires the [K8s extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/k8s) + Pre-built images from container-registry.oracle.com include the K8s extension + +* ### Patch Attributes + + The following attributes cannot be patched post SingleInstanceDatabase instance Creation : sid, edition, charset, pdbName, cloneFrom. + + ```sh + $ kubectl --type=merge -p '{"spec":{"sid":"ORCL1"}}' patch singleinstancedatabase sidb-sample + + The SingleInstanceDatabase "sidb-sample" is invalid: spec.sid: Forbidden: cannot be changed + ``` + +* #### Patch Persistence Volume Claim + + Persistence Volume Claim (PVC) can be patched post SingleInstanceDatabase instance Creation . This will **delete all the database pods, PVC** and new database pods are created using the new PVC . + + ```sh + $ kubectl --type=merge -p '{"spec":{"persistence":{"accessMode":"ReadWriteMany","size":"110Gi","storageClass":""}}}' patch singleinstancedatabase sidb-sample + + singleinstancedatabase.database.oracle.com/sidb-sample patched + ``` + +* #### Patch Service + + Service can be patched post SingleInstanceDatabase instance Creation . This will **replace the Service with a new type** . + * NodePort - '{"spec":{"loadBalancer": false}}' + * LoadBalancer - '{"spec":{"loadBalancer": true }}' + + ```sh + $ kubectl --type=merge -p '{"spec":{"loadBalancer": false}}' patch singleinstancedatabase sidb-sample + + singleinstancedatabase.database.oracle.com/sidb-sample patched + ``` + +## Clone Existing Database + + Quickly create copies of your existing database using this cloning functionality. A cloned database is an exact, block-for-block copy of the source database. + This is much faster than creating a fresh new database and copying over the data. + + To clone, specify the source database reference as value for the cloneFrom attribute in the sample .yaml. + The source database must have archiveLog mode set to true. + + ```sh + $ grep 'cloneFrom:' singleinstancedatabase.yaml + + cloneFrom: "sidb-sample" + + $ kubectl create -f singleinstancedatabase.yaml + + singleinstancedatabase.database.oracle.com/sidb-sample-clone created + ``` + + Note: The clone database can specify a database image different from the source database. In such cases, cloning is supported only between databases of the same major release. + +## Patch/Rollback Database + + Databases running in your cluster and managed by this operator can be patched or rolled back between release updates of the same major release. To patch databases, specify an image of the higher release update, and to roll back, specify an image of the lower release update. + + Patched Oracle Docker images can be built using this [patching extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/patching) + +* ### In Place Resources or Objects + + Edit the `.yaml` file of the database resourece/object and specify a new release update for release or image attributes. The database pods will be restarted + with the new release update image. For minimum downtime, ensure that you have mutiple replicas of the database pods running. + +* ### Out of Place + + Clone your source database using the method of [cloning existing database](ORACLE_SIDB_OPERATOR_README.md#clone-existing-database) and specify a new release version/image for the + cloned database. Use this method to enusure there are no patching related issues impacting your database performance/functionality + +* ### Datapatch status + + Patching/Rollback operations are complete when the datapatch tool completes patching or rollback of the data files. Check the data files patching status + and current release update version using the following commands + + ```sh + $ kubectl get singleinstancedatabase/sidb-sample --template={{.status.datafilesPatched}} + + true + + $ kubectl get singleinstancedatabase/sidb-sample --template={{.status.releaseUpdate} + + 19.3.0.0.0 (29517242) + ``` + diff --git a/doc/sidb/SIDB_PREREQUISITES.md b/doc/sidb/SIDB_PREREQUISITES.md new file mode 100644 index 00000000..43da0127 --- /dev/null +++ b/doc/sidb/SIDB_PREREQUISITES.md @@ -0,0 +1,14 @@ +## Prerequisites for Oracle Docker Image Deployment +To deploy Oracle Database Operator for Kubernetes on Oracle Docker images, complete these steps. + +* ### Prepare Oracle Docker Images + + Build SingleInstanceDatabase Docker Images from source, following the instructions at [https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance), or + use the pre-built images available at [https://container-registry.oracle.com](https://container-registry.oracle.com) + + Oracle Database Releases Supported: Oracle Database 19c Enterprise Edition or Standard Edition, and later releases. + +* ### Set Up Kubernetes and Volumes + + Set up an on-premises Kubernetes cluster, or subscribe to a managed Kubernetes service, such as Oracle Cloud Infrastructure Container Engine for Kubernetes, configured with persistent volumes. The persistent volumes are required for storage of the database files. + diff --git a/go.mod b/go.mod new file mode 100644 index 00000000..fada426a --- /dev/null +++ b/go.mod @@ -0,0 +1,16 @@ +module github.com/oracle/oracle-database-operator + +go 1.16 + +require ( + github.com/go-logr/logr v0.4.0 + github.com/onsi/ginkgo v1.16.4 + github.com/onsi/gomega v1.13.0 + github.com/oracle/oci-go-sdk/v45 v45.2.0 + gopkg.in/yaml.v2 v2.4.0 + k8s.io/api v0.21.2 + k8s.io/apimachinery v0.21.2 + k8s.io/client-go v0.21.2 + sigs.k8s.io/controller-runtime v0.9.2 + sigs.k8s.io/yaml v1.2.0 +) diff --git a/go.sum b/go.sum new file mode 100644 index 00000000..67d9f3df --- /dev/null +++ b/go.sum @@ -0,0 +1,741 @@ +cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw= +cloud.google.com/go v0.34.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw= +cloud.google.com/go v0.38.0/go.mod h1:990N+gfupTy94rShfmMCWGDn0LpTmnzTp2qbd1dvSRU= +cloud.google.com/go v0.44.1/go.mod h1:iSa0KzasP4Uvy3f1mN/7PiObzGgflwredwwASm/v6AU= +cloud.google.com/go v0.44.2/go.mod h1:60680Gw3Yr4ikxnPRS/oxxkBccT6SA1yMk63TGekxKY= +cloud.google.com/go v0.45.1/go.mod h1:RpBamKRgapWJb87xiFSdk4g1CME7QZg3uwTez+TSTjc= +cloud.google.com/go v0.46.3/go.mod h1:a6bKKbmY7er1mI7TEI4lsAkts/mkhTSZK8w33B4RAg0= +cloud.google.com/go v0.50.0/go.mod h1:r9sluTvynVuxRIOHXQEHMFffphuXHOMZMycpNR5e6To= +cloud.google.com/go v0.52.0/go.mod h1:pXajvRH/6o3+F9jDHZWQ5PbGhn+o8w9qiu/CffaVdO4= +cloud.google.com/go v0.53.0/go.mod h1:fp/UouUEsRkN6ryDKNW/Upv/JBKnv6WDthjR6+vze6M= +cloud.google.com/go v0.54.0 h1:3ithwDMr7/3vpAMXiH+ZQnYbuIsh+OPhUPMFC9enmn0= +cloud.google.com/go v0.54.0/go.mod h1:1rq2OEkV3YMf6n/9ZvGWI3GWw0VoqH/1x2nd8Is/bPc= +cloud.google.com/go/bigquery v1.0.1/go.mod h1:i/xbL2UlR5RvWAURpBYZTtm/cXjCha9lbfbpx4poX+o= +cloud.google.com/go/bigquery v1.3.0/go.mod h1:PjpwJnslEMmckchkHFfq+HTD2DmtT67aNFKH1/VBDHE= +cloud.google.com/go/bigquery v1.4.0/go.mod h1:S8dzgnTigyfTmLBfrtrhyYhwRxG72rYxvftPBK2Dvzc= +cloud.google.com/go/datastore v1.0.0/go.mod h1:LXYbyblFSglQ5pkeyhO+Qmw7ukd3C+pD7TKLgZqpHYE= +cloud.google.com/go/datastore v1.1.0/go.mod h1:umbIZjpQpHh4hmRpGhH4tLFup+FVzqBi1b3c64qFpCk= +cloud.google.com/go/firestore v1.1.0/go.mod h1:ulACoGHTpvq5r8rxGJ4ddJZBZqakUQqClKRT5SZwBmk= +cloud.google.com/go/pubsub v1.0.1/go.mod h1:R0Gpsv3s54REJCy4fxDixWD93lHJMoZTyQ2kNxGRt3I= +cloud.google.com/go/pubsub v1.1.0/go.mod h1:EwwdRX2sKPjnvnqCa270oGRyludottCI76h+R3AArQw= +cloud.google.com/go/pubsub v1.2.0/go.mod h1:jhfEVHT8odbXTkndysNHCcx0awwzvfOlguIAii9o8iA= +cloud.google.com/go/storage v1.0.0/go.mod h1:IhtSnM/ZTZV8YYJWCY8RULGVqBDmpoyjwiyrjsg+URw= +cloud.google.com/go/storage v1.5.0/go.mod h1:tpKbwo567HUNpVclU5sGELwQWBDZ8gh0ZeosJ0Rtdos= +cloud.google.com/go/storage v1.6.0/go.mod h1:N7U0C8pVQ/+NIKOBQyamJIeKQKkZ+mxpohlUTyfDhBk= +dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU= +github.com/Azure/go-ansiterm v0.0.0-20170929234023-d6e3b3328b78/go.mod h1:LmzpDX56iTiv29bbRTIsUNlaFfuhWRQBWjQdVyAevI8= +github.com/Azure/go-autorest v14.2.0+incompatible/go.mod h1:r+4oMnoxhatjLLJ6zxSWATqVooLgysK6ZNox3g/xq24= +github.com/Azure/go-autorest/autorest v0.11.12/go.mod h1:eipySxLmqSyC5s5k1CLupqet0PSENBEDP93LQ9a8QYw= +github.com/Azure/go-autorest/autorest/adal v0.9.5/go.mod h1:B7KF7jKIeC9Mct5spmyCB/A8CG/sEz1vwIRGv/bbw7A= +github.com/Azure/go-autorest/autorest/date v0.3.0/go.mod h1:BI0uouVdmngYNUzGWeSYnokU+TrmwEsOqdt8Y6sso74= +github.com/Azure/go-autorest/autorest/mocks v0.4.1/go.mod h1:LTp+uSrOhSkaKrUy935gNZuuIPPVsHlr9DSOxSayd+k= +github.com/Azure/go-autorest/logger v0.2.0/go.mod h1:T9E3cAhj2VqvPOtCYAvby9aBXkZmbF5NWuPV8+WeEW8= +github.com/Azure/go-autorest/tracing v0.6.0/go.mod h1:+vhtPC754Xsa23ID7GlGsrdKBpUA79WCAKPPZVC2DeU= +github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU= +github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo= +github.com/NYTimes/gziphandler v0.0.0-20170623195520-56545f4a5d46/go.mod h1:3wb06e3pkSAbeQ52E9H9iFoQsEEwGN64994WTCIhntQ= +github.com/NYTimes/gziphandler v1.1.1/go.mod h1:n/CVRwUEOgIxrgPvAQhUUr9oeUtvrhMomdKFjzJNB0c= +github.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAElWljhcU= +github.com/PuerkitoBio/purell v1.1.1/go.mod h1:c11w/QuzBsJSee3cPx9rAFu61PvFxuPbtSwDGJws/X0= +github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578/go.mod h1:uGdkoq3SwY9Y+13GIhn11/XLaGBb4BfwItxLd5jeuXE= +github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc= +github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc= +github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0= +github.com/alecthomas/units v0.0.0-20190717042225-c3de453c63f4/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0= +github.com/alecthomas/units v0.0.0-20190924025748-f65c72e2690d/go.mod h1:rBZYJk541a8SKzHPHnH3zbiI+7dagKZ0cgpgrD7Fyho= +github.com/armon/circbuf v0.0.0-20150827004946-bbbad097214e/go.mod h1:3U/XgcO3hCbHZ8TKRvWD2dDTCfh9M9ya+I9JpbB7O8o= +github.com/armon/go-metrics v0.0.0-20180917152333-f0300d1749da/go.mod h1:Q73ZrmVTwzkszR9V5SSuryQ31EELlFMUz1kKyl939pY= +github.com/armon/go-radix v0.0.0-20180808171621-7fddfc383310/go.mod h1:ufUuZ+zHj4x4TnLV4JWEpy2hxWSpsRywHrMgIH9cCH8= +github.com/asaskevich/govalidator v0.0.0-20190424111038-f61b66f89f4a/go.mod h1:lB+ZfQJz7igIIfQNfa7Ml4HSf2uFQQRzpGGRXenZAgY= +github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q= +github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8= +github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= +github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= +github.com/bgentry/speakeasy v0.1.0/go.mod h1:+zsyZBPWlz7T6j88CTgSN5bM796AkVf0kBD4zp0CCIs= +github.com/bketelsen/crypt v0.0.3-0.20200106085610-5cbc8cc4026c/go.mod h1:MKsuJmJgSg28kpZDP6UIiPt0e0Oz0kqKNGyRaWEPv84= +github.com/blang/semver v3.5.1+incompatible/go.mod h1:kRBLl5iJ+tD4TcOOxsy/0fnwebNt5EWlYSAyrTnjyyk= +github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU= +github.com/cespare/xxhash v1.1.0 h1:a6HrQnmkObjyL+Gs60czilIUGqrzKutQD6XZog3p+ko= +github.com/cespare/xxhash v1.1.0/go.mod h1:XrSqR1VqqWfGrhpAt58auRo0WTKS1nRRg3ghfAqPWnc= +github.com/cespare/xxhash/v2 v2.1.1 h1:6MnRN8NT7+YBpUIWxHtefFZOKTAPgGjpQSxqLNn0+qY= +github.com/cespare/xxhash/v2 v2.1.1/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= +github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI= +github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI= +github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU= +github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw= +github.com/cockroachdb/datadriven v0.0.0-20190809214429-80d97fb3cbaa/go.mod h1:zn76sxSg3SzpJ0PPJaLDCu+Bu0Lg3sKTORVIj19EIF8= +github.com/coreos/bbolt v1.3.2/go.mod h1:iRUV2dpdMOn7Bo10OQBFzIJO9kkE559Wcmn+qkEiiKk= +github.com/coreos/etcd v3.3.13+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE= +github.com/coreos/go-oidc v2.1.0+incompatible/go.mod h1:CgnwVTmzoESiwO9qyAFEMiHoZ1nMCKZlZ9V6mm3/LKc= +github.com/coreos/go-semver v0.2.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk= +github.com/coreos/go-semver v0.3.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk= +github.com/coreos/go-systemd v0.0.0-20180511133405-39ca1b05acc7/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4= +github.com/coreos/go-systemd v0.0.0-20190321100706-95778dfbb74e/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4= +github.com/coreos/pkg v0.0.0-20160727233714-3ac0863d7acf/go.mod h1:E3G3o1h8I7cfcXa63jLwjI0eiQQMgzzUDFVpN/nH/eA= +github.com/coreos/pkg v0.0.0-20180928190104-399ea9e2e55f/go.mod h1:E3G3o1h8I7cfcXa63jLwjI0eiQQMgzzUDFVpN/nH/eA= +github.com/cpuguy83/go-md2man/v2 v2.0.0/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU= +github.com/creack/pty v1.1.7/go.mod h1:lj5s0c3V2DBrqTV7llrYr5NG6My20zk30Fl46Y7DoTY= +github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= +github.com/creack/pty v1.1.11/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= +github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= +github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/dgrijalva/jwt-go v3.2.0+incompatible/go.mod h1:E3ru+11k8xSBh+hMPgOLZmtrrCbhqsmaPHjLKYnJCaQ= +github.com/dgryski/go-sip13 v0.0.0-20181026042036-e10d5fee7954/go.mod h1:vAd38F8PWV+bWy6jNmig1y/TA+kYO4g3RSRF0IAv0no= +github.com/docopt/docopt-go v0.0.0-20180111231733-ee0de3bc6815/go.mod h1:WwZ+bS3ebgob9U8Nd0kOddGdZWjyMGR8Wziv+TBNwSE= +github.com/dustin/go-humanize v0.0.0-20171111073723-bb3d318650d4/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk= +github.com/dustin/go-humanize v1.0.0/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk= +github.com/elazarl/goproxy v0.0.0-20180725130230-947c36da3153 h1:yUdfgN0XgIJw7foRItutHYUIhlcKzcSf5vDpdhQAKTc= +github.com/elazarl/goproxy v0.0.0-20180725130230-947c36da3153/go.mod h1:/Zj4wYkgs4iZTTu3o/KG3Itv/qCCa8VVMlb3i9OVuzc= +github.com/emicklei/go-restful v0.0.0-20170410110728-ff4f55a20633/go.mod h1:otzb+WCGbkyDHkqmQmT5YD2WR4BBwUdeQoFo8l/7tVs= +github.com/emicklei/go-restful v2.9.5+incompatible/go.mod h1:otzb+WCGbkyDHkqmQmT5YD2WR4BBwUdeQoFo8l/7tVs= +github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4= +github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c= +github.com/evanphx/json-patch v0.5.2/go.mod h1:ZWS5hhDbVDyob71nXKNL0+PWn6ToqBHMikGIFbs31qQ= +github.com/evanphx/json-patch v4.9.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk= +github.com/evanphx/json-patch v4.11.0+incompatible h1:glyUF9yIYtMHzn8xaKw5rMhdWcwsYV8dZHIq5567/xs= +github.com/evanphx/json-patch v4.11.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk= +github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4= +github.com/form3tech-oss/jwt-go v3.2.2+incompatible/go.mod h1:pbq4aXjuKjdthFRnoDwaVPLA+WlJuPGy+QneDUgJi2k= +github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo= +github.com/fsnotify/fsnotify v1.4.9 h1:hsms1Qyu0jgnwNXIxa+/V/PDsU6CfLf6CNO8H7IWoS4= +github.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ= +github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04= +github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU= +github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8= +github.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8= +github.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as= +github.com/go-kit/kit v0.9.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as= +github.com/go-kit/log v0.1.0/go.mod h1:zbhenjAZHb184qTLMA9ZjW7ThYL0H2mk7Q6pNt4vbaY= +github.com/go-logfmt/logfmt v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9GBnD5lWE= +github.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V4qmtdjCk= +github.com/go-logfmt/logfmt v0.5.0/go.mod h1:wCYkCAKZfumFQihp8CzCvQ3paCTfi41vtzG1KdI/P7A= +github.com/go-logr/logr v0.1.0/go.mod h1:ixOQHD9gLJUVQQ2ZOR7zLEifBX6tGkNJF4QyIY7sIas= +github.com/go-logr/logr v0.2.0/go.mod h1:z6/tIYblkpsD+a4lm/fGIIU9mZ+XfAiaFtq7xTgseGU= +github.com/go-logr/logr v0.4.0 h1:K7/B1jt6fIBQVd4Owv2MqGQClcgf0R266+7C/QjRcLc= +github.com/go-logr/logr v0.4.0/go.mod h1:z6/tIYblkpsD+a4lm/fGIIU9mZ+XfAiaFtq7xTgseGU= +github.com/go-logr/zapr v0.4.0 h1:uc1uML3hRYL9/ZZPdgHS/n8Nzo+eaYL/Efxkkamf7OM= +github.com/go-logr/zapr v0.4.0/go.mod h1:tabnROwaDl0UNxkVeFRbY8bwB37GwRv0P8lg6aAiEnk= +github.com/go-openapi/jsonpointer v0.19.2/go.mod h1:3akKfEdA7DF1sugOqz1dVQHBcuDBPKZGEoHC/NkiQRg= +github.com/go-openapi/jsonpointer v0.19.3/go.mod h1:Pl9vOtqEWErmShwVjC8pYs9cog34VGT37dQOVbmoatg= +github.com/go-openapi/jsonreference v0.19.2/go.mod h1:jMjeRr2HHw6nAVajTXJ4eiUwohSTlpa0o73RUL1owJc= +github.com/go-openapi/jsonreference v0.19.3/go.mod h1:rjx6GuL8TTa9VaixXglHmQmIL98+wF9xc8zWvFonSJ8= +github.com/go-openapi/spec v0.19.3/go.mod h1:FpwSN1ksY1eteniUU7X0N/BgJ7a4WvBFVA8Lj9mJglo= +github.com/go-openapi/spec v0.19.5/go.mod h1:Hm2Jr4jv8G1ciIAo+frC/Ft+rR2kQDh8JHKHb3gWUSk= +github.com/go-openapi/swag v0.19.2/go.mod h1:POnQmlKehdgb5mhVOsnJFsivZCEZ/vjK9gh66Z9tfKk= +github.com/go-openapi/swag v0.19.5/go.mod h1:POnQmlKehdgb5mhVOsnJFsivZCEZ/vjK9gh66Z9tfKk= +github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY= +github.com/go-task/slim-sprig v0.0.0-20210107165309-348f09dbbbc0/go.mod h1:fyg7847qk6SyHyPtNmDHnmrv/HOrqktSC+C9fM+CJOE= +github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ= +github.com/gogo/protobuf v1.2.1/go.mod h1:hp+jE20tsWTFYpLwKvXlhS1hjn+gTNwPg2I6zVXpSg4= +github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q= +github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q= +github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q= +github.com/golang/groupcache v0.0.0-20160516000752-02826c3e7903/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= +github.com/golang/groupcache v0.0.0-20190129154638-5b532d6fd5ef/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= +github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= +github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= +github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e h1:1r7pUrabqp18hOBcwBwiTsbnFeTZHV9eER/QT5JVZxY= +github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= +github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A= +github.com/golang/mock v1.2.0/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A= +github.com/golang/mock v1.3.1/go.mod h1:sBzyDLLjw3U8JLTeZvSv8jJB+tU5PVekmnlKIyFUx0Y= +github.com/golang/mock v1.4.0/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw= +github.com/golang/mock v1.4.1/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw= +github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= +github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= +github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= +github.com/golang/protobuf v1.3.3/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw= +github.com/golang/protobuf v1.3.4/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw= +github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8= +github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA= +github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs= +github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w= +github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0= +github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8= +github.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI= +github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI= +github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk= +github.com/golang/protobuf v1.5.2 h1:ROPKBNFfQgOUMifHyP+KYbvpjbdoFNs+aK7DXlji0Tw= +github.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY= +github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= +github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= +github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M= +github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= +github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= +github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= +github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= +github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= +github.com/google/go-cmp v0.5.5 h1:Khx7svrCpmxxtHBq5j2mp/xVjsi8hQMfNLvJFAlrGgU= +github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= +github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= +github.com/google/gofuzz v1.1.0 h1:Hsa8mG0dQ46ij8Sl2AYJDUv1oA9/d6Vk+3LG99Oe02g= +github.com/google/gofuzz v1.1.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= +github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs= +github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc= +github.com/google/pprof v0.0.0-20190515194954-54271f7e092f/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc= +github.com/google/pprof v0.0.0-20191218002539-d4f498aebedc/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= +github.com/google/pprof v0.0.0-20200212024743-f11f1df84d12/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= +github.com/google/pprof v0.0.0-20200229191704-1ebb73c60ed3/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= +github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI= +github.com/google/uuid v1.0.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= +github.com/google/uuid v1.1.1/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= +github.com/google/uuid v1.1.2 h1:EVhdT+1Kseyi1/pUmXKaFxYsDNy9RQYkMWRH68J/W7Y= +github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= +github.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg= +github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk= +github.com/googleapis/gnostic v0.4.1/go.mod h1:LRhVm6pbyptWbWbuZ38d1eyptfvIytN3ir6b65WBswg= +github.com/googleapis/gnostic v0.5.5 h1:9fHAtK0uDfpveeqqo1hkEZJcFvYXAiCN3UutL8F9xHw= +github.com/googleapis/gnostic v0.5.5/go.mod h1:7+EbHbldMins07ALC74bsA81Ovc97DwqyJO1AENw9kA= +github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY= +github.com/gorilla/websocket v0.0.0-20170926233335-4201258b820c/go.mod h1:E7qHFY5m1UJ88s3WnNqhKjPHQ0heANvMoAMk2YaljkQ= +github.com/gorilla/websocket v1.4.2/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE= +github.com/gregjones/httpcache v0.0.0-20180305231024-9cad4c3443a7/go.mod h1:FecbI9+v66THATjSRHfNgh1IVFe/9kFxbXtjV0ctIMA= +github.com/grpc-ecosystem/go-grpc-middleware v1.0.0/go.mod h1:FiyG127CGDf3tlThmgyCl78X/SZQqEOJBCDaAfeWzPs= +github.com/grpc-ecosystem/go-grpc-middleware v1.0.1-0.20190118093823-f849b5445de4/go.mod h1:FiyG127CGDf3tlThmgyCl78X/SZQqEOJBCDaAfeWzPs= +github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0/go.mod h1:8NvIoxWQoOIhqOTXgfV/d3M/q6VIi02HzZEHgUlZvzk= +github.com/grpc-ecosystem/grpc-gateway v1.9.0/go.mod h1:vNeuVxBJEsws4ogUvrchl83t/GYV9WGTSLVdBhOQFDY= +github.com/grpc-ecosystem/grpc-gateway v1.9.5/go.mod h1:vNeuVxBJEsws4ogUvrchl83t/GYV9WGTSLVdBhOQFDY= +github.com/hashicorp/consul/api v1.1.0/go.mod h1:VmuI/Lkw1nC05EYQWNKwWGbkg+FbDBtguAZLlVdkD9Q= +github.com/hashicorp/consul/sdk v0.1.1/go.mod h1:VKf9jXwCTEY1QZP2MOLRhb5i/I/ssyNV1vwHyQBF0x8= +github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4= +github.com/hashicorp/go-cleanhttp v0.5.1/go.mod h1:JpRdi6/HCYpAwUzNwuwqhbovhLtngrth3wmdIIUrZ80= +github.com/hashicorp/go-immutable-radix v1.0.0/go.mod h1:0y9vanUI8NX6FsYoO3zeMjhV/C5i9g4Q3DwcSNZ4P60= +github.com/hashicorp/go-msgpack v0.5.3/go.mod h1:ahLV/dePpqEmjfWmKiqvPkv/twdG7iPBM1vqhUKIvfM= +github.com/hashicorp/go-multierror v1.0.0/go.mod h1:dHtQlpGsu+cZNNAkkCN/P3hoUDHhCYQXV3UM06sGGrk= +github.com/hashicorp/go-rootcerts v1.0.0/go.mod h1:K6zTfqpRlCUIjkwsN4Z+hiSfzSTQa6eBIzfwKfwNnHU= +github.com/hashicorp/go-sockaddr v1.0.0/go.mod h1:7Xibr9yA9JjQq1JpNB2Vw7kxv8xerXegt+ozgdvDeDU= +github.com/hashicorp/go-syslog v1.0.0/go.mod h1:qPfqrKkXGihmCqbJM2mZgkZGvKG1dFdvsLplgctolz4= +github.com/hashicorp/go-uuid v1.0.0/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro= +github.com/hashicorp/go-uuid v1.0.1/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro= +github.com/hashicorp/go.net v0.0.1/go.mod h1:hjKkEWcCURg++eb33jQU7oqQcI9XDCnUzHA0oac0k90= +github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8= +github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8= +github.com/hashicorp/golang-lru v0.5.4 h1:YDjusn29QI/Das2iO9M0BHnIbxPeyuCHsjMW+lJfyTc= +github.com/hashicorp/golang-lru v0.5.4/go.mod h1:iADmTwqILo4mZ8BN3D2Q6+9jd8WM5uGBxy+E8yxSoD4= +github.com/hashicorp/hcl v1.0.0/go.mod h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ= +github.com/hashicorp/logutils v1.0.0/go.mod h1:QIAnNjmIWmVIIkWDTG1z5v++HQmx9WQRO+LraFDTW64= +github.com/hashicorp/mdns v1.0.0/go.mod h1:tL+uN++7HEJ6SQLQ2/p+z2pH24WQKWjBPkE0mNTz8vQ= +github.com/hashicorp/memberlist v0.1.3/go.mod h1:ajVTdAv/9Im8oMAAj5G31PhhMCZJV2pPBoIllUwCN7I= +github.com/hashicorp/serf v0.8.2/go.mod h1:6hOLApaqBFA1NXqRQAsxw9QxuDEvNxSQRwA/JwenrHc= +github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU= +github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc= +github.com/imdario/mergo v0.3.5/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA= +github.com/imdario/mergo v0.3.12 h1:b6R2BslTbIEToALKP7LxUvijTsNI9TAe80pLWN2g/HU= +github.com/imdario/mergo v0.3.12/go.mod h1:jmQim1M+e3UYxmgPu/WyfjB3N3VflVyUjjjwH0dnCYA= +github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8= +github.com/jessevdk/go-flags v1.4.0/go.mod h1:4FA24M0QyGHXBuZZK/XkWh8h0e1EYbRYJSGM75WSRxI= +github.com/jonboulle/clockwork v0.1.0/go.mod h1:Ii8DK3G1RaLaWxj9trq07+26W01tbo22gdxWY5EU2bo= +github.com/jpillora/backoff v1.0.0/go.mod h1:J/6gKK9jxlEcS3zixgDgUAsiuZ7yrSoa/FX5e0EB2j4= +github.com/json-iterator/go v1.1.6/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU= +github.com/json-iterator/go v1.1.7/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4= +github.com/json-iterator/go v1.1.10/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4= +github.com/json-iterator/go v1.1.11 h1:uVUAXhF2To8cbw/3xN3pxj6kk7TYKs98NIrTqPlMWAQ= +github.com/json-iterator/go v1.1.11/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4= +github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU= +github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk= +github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU= +github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w= +github.com/julienschmidt/httprouter v1.3.0/go.mod h1:JR6WtHb+2LUe8TCKY3cZOxFyyO8IZAc4RVcycCCAKdM= +github.com/kisielk/errcheck v1.1.0/go.mod h1:EZBBE59ingxPouuu3KfxchcWSUPOHkagtvWXihfKN4Q= +github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8= +github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck= +github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= +github.com/konsorten/go-windows-terminal-sequences v1.0.3/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= +github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc= +github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo= +github.com/kr/pretty v0.2.0/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI= +github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ= +github.com/kr/pty v1.1.5/go.mod h1:9r2w37qlBe7rQ6e1fg1S/9xpWHSnaqNdHD3WcMdbPDA= +github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI= +github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= +github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= +github.com/magiconair/properties v1.8.1/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ= +github.com/mailru/easyjson v0.0.0-20190614124828-94de47d64c63/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc= +github.com/mailru/easyjson v0.0.0-20190626092158-b2ccc519800e/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc= +github.com/mailru/easyjson v0.7.0/go.mod h1:KAzv3t3aY1NaHWoQz1+4F1ccyAH66Jk7yos7ldAVICs= +github.com/mattn/go-colorable v0.0.9/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaOChaDxuIBZU= +github.com/mattn/go-isatty v0.0.3/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4= +github.com/mattn/go-isatty v0.0.4/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4= +github.com/mattn/go-runewidth v0.0.2/go.mod h1:LwmH8dsx7+W8Uxz3IHJYH5QSwggIsqBzpuz5H//U1FU= +github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0= +github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 h1:I0XW9+e1XWDxdcEniV4rQAIOPUGDq67JSCiRCgGCZLI= +github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369/go.mod h1:BSXmuO+STAnVfrANrmjBb36TMTDstsz7MSK+HVaYKv4= +github.com/miekg/dns v1.0.14/go.mod h1:W1PPwlIAgtquWBMBEV9nkV9Cazfe8ScdGz/Lj7v3Nrg= +github.com/mitchellh/cli v1.0.0/go.mod h1:hNIlj7HEI86fIcpObd7a0FcrxTWetlwJDGcceTlRvqc= +github.com/mitchellh/go-homedir v1.0.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0= +github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0= +github.com/mitchellh/go-testing-interface v1.0.0/go.mod h1:kRemZodwjscx+RGhAo8eIhFbs2+BFgRtFPeD/KE+zxI= +github.com/mitchellh/gox v0.4.0/go.mod h1:Sd9lOJ0+aimLBi73mGofS1ycjY8lL3uZM3JPS42BGNg= +github.com/mitchellh/iochan v1.0.0/go.mod h1:JwYml1nuB7xOzsp52dPpHFffvOCDupsG0QubkSMEySY= +github.com/mitchellh/mapstructure v0.0.0-20160808181253-ca63d7c062ee/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y= +github.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y= +github.com/moby/spdystream v0.2.0 h1:cjW1zVyyoiM0T7b6UoySUFqzXMoqRckQtXwGPiBhOM8= +github.com/moby/spdystream v0.2.0/go.mod h1:f7i0iNDQJ059oMTcWxx8MA/zKFIuD/lY+0GqbN2Wy8c= +github.com/moby/term v0.0.0-20201216013528-df9cb8a40635/go.mod h1:FBS0z0QWA44HXygs7VXDUOGoN/1TV3RuWkLO04am3wc= +github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= +github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg= +github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= +github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0= +github.com/modern-go/reflect2 v1.0.1 h1:9f412s+6RmYXLWZSEzVVgPGK7C2PphHj5RJrvfx9AWI= +github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0= +github.com/munnerz/goautoneg v0.0.0-20120707110453-a547fc61f48d/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ= +github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ= +github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U= +github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U= +github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f/go.mod h1:ZdcZmHo+o7JKHSa8/e818NopupXU1YMK5fe1lsApnBw= +github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e h1:fD57ERR4JtEqsWbfPhv4DMiApHyliiK5xCTNVSPiaAs= +github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e/go.mod h1:zD1mROLANZcx1PVRCS0qkT7pwLkGfwJo4zjcN/Tysno= +github.com/nxadm/tail v1.4.4/go.mod h1:kenIhsEOeOJmVchQTgglprH7qJGnHDVpk1VPCcaMI8A= +github.com/nxadm/tail v1.4.8 h1:nPr65rt6Y5JFSKQO7qToXr7pePgD6Gwiw05lkbyAQTE= +github.com/nxadm/tail v1.4.8/go.mod h1:+ncqLTQzXmGhMZNUePPaPqPvBxHAIsmXswZKocGu+AU= +github.com/oklog/ulid v1.3.1/go.mod h1:CirwcVhetQ6Lv90oh/F+FBtV6XMibvdAFo93nm5qn4U= +github.com/olekukonko/tablewriter v0.0.0-20170122224234-a0225b3f23b5/go.mod h1:vsDQFd/mU46D+Z4whnwzcISnGGzXWMclvtLoiIKAKIo= +github.com/onsi/ginkgo v0.0.0-20170829012221-11459a886d9c/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE= +github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE= +github.com/onsi/ginkgo v1.11.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE= +github.com/onsi/ginkgo v1.12.1/go.mod h1:zj2OWP4+oCPe1qIXoGWkgMRwljMUYCdkwsT2108oapk= +github.com/onsi/ginkgo v1.16.2/go.mod h1:CObGmKUOKaSC0RjmoAK7tKyn4Azo5P2IWuoMnvwxz1E= +github.com/onsi/ginkgo v1.16.4 h1:29JGrr5oVBm5ulCWet69zQkzWipVXIol6ygQUe/EzNc= +github.com/onsi/ginkgo v1.16.4/go.mod h1:dX+/inL/fNMqNlz0e9LfyB9TswhZpCVdJM/Z6Vvnwo0= +github.com/onsi/gomega v0.0.0-20170829124025-dcabb60a477c/go.mod h1:C1qb7wdrVGGVU+Z6iS04AVkA3Q65CEZX59MT0QO5uiA= +github.com/onsi/gomega v1.7.0/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY= +github.com/onsi/gomega v1.7.1/go.mod h1:XdKZgCCFLUoM/7CFJVPcG8C1xQ1AJ0vpAezJrB7JYyY= +github.com/onsi/gomega v1.10.1/go.mod h1:iN09h71vgCQne3DLsj+A5owkum+a2tYe+TOCB1ybHNo= +github.com/onsi/gomega v1.13.0 h1:7lLHu94wT9Ij0o6EWWclhu0aOh32VxhkwEJvzuWPeak= +github.com/onsi/gomega v1.13.0/go.mod h1:lRk9szgn8TxENtWd0Tp4c3wjlRfMTMH27I+3Je41yGY= +github.com/oracle/oci-go-sdk/v45 v45.2.0 h1:vCPoQlE+DOrM2heJn66rvPU6fbsc/0Cxtzs2jnFut6U= +github.com/oracle/oci-go-sdk/v45 v45.2.0/go.mod h1:ZM6LGiRO5TPQJxTlrXbcHMbClE775wnGD5U/EerCsRw= +github.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc= +github.com/pelletier/go-toml v1.2.0/go.mod h1:5z9KED0ma1S8pY6P1sdut58dfprrGBbd/94hg7ilaic= +github.com/peterbourgon/diskv v2.0.1+incompatible/go.mod h1:uqqh8zWWbv1HBMNONnaR/tNboyR3/BZd58JJSHlUSCU= +github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= +github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= +github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4= +github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= +github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= +github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/posener/complete v1.1.1/go.mod h1:em0nMJCgc9GFtwrmVmEMR/ZL6WyhyjMBndrE9hABlRI= +github.com/pquerna/cachecontrol v0.0.0-20171018203845-0dec1b30a021/go.mod h1:prYjPmNq4d1NPVmpShWobRqXY3q7Vp+80DqgxxUrUIA= +github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw= +github.com/prometheus/client_golang v0.9.3/go.mod h1:/TN21ttK/J9q6uSwhBd54HahCDft0ttaMvbicHlPoso= +github.com/prometheus/client_golang v1.0.0/go.mod h1:db9x61etRT2tGnBNRi70OPL5FsnadC4Ky3P0J6CfImo= +github.com/prometheus/client_golang v1.7.1/go.mod h1:PY5Wy2awLA44sXw4AOSfFBetzPP4j5+D6mVACh+pe2M= +github.com/prometheus/client_golang v1.11.0 h1:HNkLOAEQMIDv/K+04rukrLx6ch7msSRwf3/SASFAGtQ= +github.com/prometheus/client_golang v1.11.0/go.mod h1:Z6t4BnS23TR94PD6BsDNk8yVqroYurpAkEiz0P2BEV0= +github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo= +github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= +github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= +github.com/prometheus/client_model v0.2.0 h1:uq5h0d+GuxiXLJLNABMgp2qUWDPiLvgCzz2dUR+/W/M= +github.com/prometheus/client_model v0.2.0/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= +github.com/prometheus/common v0.0.0-20181113130724-41aa239b4cce/go.mod h1:daVV7qP5qjZbuso7PdcryaAu0sAZbrN9i7WWcTMWvro= +github.com/prometheus/common v0.4.0/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4= +github.com/prometheus/common v0.4.1/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4= +github.com/prometheus/common v0.10.0/go.mod h1:Tlit/dnDKsSWFlCLTWaA1cyBgKHSMdTB80sz/V91rCo= +github.com/prometheus/common v0.26.0 h1:iMAkS2TDoNWnKM+Kopnx/8tnEStIfpYA0ur0xQzzhMQ= +github.com/prometheus/common v0.26.0/go.mod h1:M7rCNAaPfAosfx8veZJCuw84e35h3Cfd9VFqTh1DIvc= +github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk= +github.com/prometheus/procfs v0.0.0-20190507164030-5867b95ac084/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA= +github.com/prometheus/procfs v0.0.2/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA= +github.com/prometheus/procfs v0.1.3/go.mod h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4OA4YeYWdaU= +github.com/prometheus/procfs v0.2.0/go.mod h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4OA4YeYWdaU= +github.com/prometheus/procfs v0.6.0 h1:mxy4L2jP6qMonqmq+aTtOx1ifVWUgG/TAmntgbh3xv4= +github.com/prometheus/procfs v0.6.0/go.mod h1:cz+aTbrPOrUb4q7XlbU9ygM+/jj0fzG6c1xBZuNvfVA= +github.com/prometheus/tsdb v0.7.1/go.mod h1:qhTCs0VvXwvX/y3TZrWD7rabWM+ijKTux40TwIPHuXU= +github.com/rogpeppe/fastuuid v0.0.0-20150106093220-6724a57986af/go.mod h1:XWv6SoW27p1b0cqNHllgS5HIMJraePCO15w5zCzIWYg= +github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4= +github.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM= +github.com/ryanuber/columnize v0.0.0-20160712163229-9b3edd62028f/go.mod h1:sm1tb6uqfes/u+d4ooFouqFdy9/2g9QGwK3SQygK0Ts= +github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529/go.mod h1:DxrIzT+xaE7yg65j358z/aeFdxmN0P9QXhEzd20vsDc= +github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc= +github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo= +github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE= +github.com/sirupsen/logrus v1.6.0/go.mod h1:7uNnSEd1DgxDLC74fIahvMZmmYsHGZGEOFrfsX/uA88= +github.com/sirupsen/logrus v1.7.0/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0= +github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1:OnSkiWE9lh6wB0YB77sQom3nweQdgAjqCqsofrRNTgc= +github.com/smartystreets/goconvey v1.6.4/go.mod h1:syvi0/a8iFYH4r/RixwvyeAJjdLS9QV7WQ/tjFTllLA= +github.com/soheilhy/cmux v0.1.4/go.mod h1:IM3LyeVVIOuxMH7sFAkER9+bJ4dT7Ms6E4xg4kGIyLM= +github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA= +github.com/spf13/afero v1.1.2/go.mod h1:j4pytiNVoe2o6bmDsKpLACNPDBIoEAkihy7loJ1B0CQ= +github.com/spf13/afero v1.2.2/go.mod h1:9ZxEEn6pIJ8Rxe320qSDBk6AsU0r9pR7Q4OcevTdifk= +github.com/spf13/cast v1.3.0/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE= +github.com/spf13/cobra v0.0.3/go.mod h1:1l0Ry5zgKvJasoi3XT1TypsSe7PqH0Sj9dhYf7v3XqQ= +github.com/spf13/cobra v1.1.1/go.mod h1:WnodtKOvamDL/PwE2M4iKs8aMDBZ5Q5klgD3qfVJQMI= +github.com/spf13/jwalterweatherman v1.0.0/go.mod h1:cQK4TGJAtQXfYWX+Ddv3mKDzgVb68N+wFjFa4jdeBTo= +github.com/spf13/pflag v0.0.0-20170130214245-9ff6c6923cff/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4= +github.com/spf13/pflag v1.0.1/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4= +github.com/spf13/pflag v1.0.3/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4= +github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA= +github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= +github.com/spf13/viper v1.7.0/go.mod h1:8WkrPz2fc9jxqZNCJI/76HCieCp4Q8HaLFoCha5qpdg= +github.com/stoewer/go-strcase v1.2.0/go.mod h1:IBiWB2sKIp3wVVQ3Y035++gc+knqhUQag1KpM8ahLw8= +github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= +github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= +github.com/stretchr/objx v0.2.0 h1:Hbg2NidpLE8veEBkEZTL3CvlkUIVzuU9jDplZO54c48= +github.com/stretchr/objx v0.2.0/go.mod h1:qt09Ya8vawLte6SNmTgCsAVtYtaKzEcn8ATUoHMkEqE= +github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs= +github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= +github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4= +github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA= +github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= +github.com/stretchr/testify v1.7.0 h1:nwc3DEeHmmLAfoZucVR881uASk0Mfjw8xYJ99tb5CcY= +github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= +github.com/subosito/gotenv v1.2.0/go.mod h1:N0PQaV/YGNqwC0u51sEeR/aUtSLEXKX9iv69rRypqCw= +github.com/tmc/grpc-websocket-proxy v0.0.0-20170815181823-89b8d40f7ca8/go.mod h1:ncp9v5uamzpCO7NfCPTXjqaC+bZgJeR0sMTm6dMHP7U= +github.com/tmc/grpc-websocket-proxy v0.0.0-20190109142713-0ad062ec5ee5/go.mod h1:ncp9v5uamzpCO7NfCPTXjqaC+bZgJeR0sMTm6dMHP7U= +github.com/urfave/cli v1.20.0/go.mod h1:70zkFmudgCuE/ngEzBv17Jvp/497gISqfk5gWijbERA= +github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2/go.mod h1:UETIi67q53MR2AWcXfiuqkDkRtnGDLqkBTpCHuJHxtU= +github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= +github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= +go.etcd.io/bbolt v1.3.2/go.mod h1:IbVyRI1SCnLcuJnV2u8VeU0CEYM7e686BmAb1XKL+uU= +go.etcd.io/bbolt v1.3.3/go.mod h1:IbVyRI1SCnLcuJnV2u8VeU0CEYM7e686BmAb1XKL+uU= +go.etcd.io/bbolt v1.3.5/go.mod h1:G5EMThwa9y8QZGBClrRx5EY+Yw9kAhnjy3bSjsnlVTQ= +go.etcd.io/etcd v0.5.0-alpha.5.0.20200910180754-dd1b699fc489/go.mod h1:yVHk9ub3CSBatqGNg7GRmsnfLWtoW60w4eDYfh7vHDg= +go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU= +go.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8= +go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= +go.opencensus.io v0.22.3/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= +go.uber.org/atomic v1.3.2/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE= +go.uber.org/atomic v1.4.0/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE= +go.uber.org/atomic v1.7.0 h1:ADUqmZGgLDDfbSL9ZmPxKTybcoEYHgpYfELNoN+7hsw= +go.uber.org/atomic v1.7.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc= +go.uber.org/goleak v1.1.10 h1:z+mqJhf6ss6BSfSM671tgKyZBFPTTJM+HLxnhPC3wu0= +go.uber.org/goleak v1.1.10/go.mod h1:8a7PlsEVH3e/a/GLqe5IIrQx6GzcnRmZEufDUTk4A7A= +go.uber.org/multierr v1.1.0/go.mod h1:wR5kodmAFQ0UK8QlbwjlSNy0Z68gJhDJUG5sjR94q/0= +go.uber.org/multierr v1.6.0 h1:y6IPFStTAIT5Ytl7/XYmHvzXQ7S3g/IeZW9hyZ5thw4= +go.uber.org/multierr v1.6.0/go.mod h1:cdWPpRnG4AhwMwsgIHip0KRBQjJy5kYEpYjJxpXp9iU= +go.uber.org/zap v1.10.0/go.mod h1:vwi/ZaCAaUcBkycHslxD9B2zi4UTXhF60s6SWpuDF0Q= +go.uber.org/zap v1.17.0 h1:MTjgFu6ZLKvY6Pvaqk97GlxNBuMpV4Hy/3P6tRGlI2U= +go.uber.org/zap v1.17.0/go.mod h1:MXVU+bhUf/A7Xi2HNOnopQOrmycQ5Ih87HtOu4q5SSo= +golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= +golang.org/x/crypto v0.0.0-20181029021203-45a5f77698d3/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= +golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= +golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= +golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= +golang.org/x/crypto v0.0.0-20190611184440-5c40567a22f8/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= +golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= +golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= +golang.org/x/crypto v0.0.0-20201002170205-7f63de1d35b0/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= +golang.org/x/crypto v0.0.0-20210220033148-5ea612d1eb83/go.mod h1:jdWPYTVW3xRLrWPugEBEK3UY2ZEsg3UU495nc5E+M+I= +golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= +golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= +golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8= +golang.org/x/exp v0.0.0-20190829153037-c13cbed26979/go.mod h1:86+5VVa7VpoJ4kLfm080zCjGlMRFzhUhsZKEZO7MGek= +golang.org/x/exp v0.0.0-20191030013958-a1ab85dbe136/go.mod h1:JXzH8nQsPlswgeRAPE3MuO9GYsAcnJvJ4vnMwN/5qkY= +golang.org/x/exp v0.0.0-20191129062945-2f5052295587/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4= +golang.org/x/exp v0.0.0-20191227195350-da58074b4299/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4= +golang.org/x/exp v0.0.0-20200119233911-0405dc783f0a/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4= +golang.org/x/exp v0.0.0-20200207192155-f17229e696bd/go.mod h1:J/WKrq2StrnmMY6+EHIKF9dgMWnmCNThgcyBT1FY9mM= +golang.org/x/exp v0.0.0-20200224162631-6cc2880d07d6/go.mod h1:3jZMyOhIsHpP37uCMkUooju7aAi5cS1Q23tOzKc+0MU= +golang.org/x/image v0.0.0-20190227222117-0694c2d4d067/go.mod h1:kZ7UVZpmo3dzQBMxlp+ypCbDeSB+sBbTgSJuh5dn5js= +golang.org/x/image v0.0.0-20190802002840-cff245a6509b/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0= +golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE= +golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU= +golang.org/x/lint v0.0.0-20190301231843-5614ed5bae6f/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE= +golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= +golang.org/x/lint v0.0.0-20190409202823-959b441ac422/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= +golang.org/x/lint v0.0.0-20190909230951-414d861bb4ac/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= +golang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= +golang.org/x/lint v0.0.0-20191125180803-fdd1cda4f05f/go.mod h1:5qLYkcX4OjUUV8bRuDixDT3tpyyb+LUpUlRWLxfhWrs= +golang.org/x/lint v0.0.0-20200130185559-910be7a94367/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY= +golang.org/x/lint v0.0.0-20200302205851-738671d3881b h1:Wh+f8QHJXR411sJR8/vRBTZ7YapZaRvUcLFFJhusH0k= +golang.org/x/lint v0.0.0-20200302205851-738671d3881b/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY= +golang.org/x/mobile v0.0.0-20190312151609-d3739f865fa6/go.mod h1:z+o9i4GpDbdi3rU15maQ/Ox0txvL9dWGYEHz965HBQE= +golang.org/x/mobile v0.0.0-20190719004257-d2bd2a29d028/go.mod h1:E/iHnbuqvinMTCcRqshq8CkpyQDoeVncDDYHnLhea+o= +golang.org/x/mod v0.0.0-20190513183733-4bf6d317e70e/go.mod h1:mXi4GBBbnImb6dmsKGUJ2LatrhH/nqhxcFungHvyanc= +golang.org/x/mod v0.1.0/go.mod h1:0QHyrYULN0/3qlju5TqG8bIK38QM8yzMo5ekMj3DlcY= +golang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg= +golang.org/x/mod v0.1.1-0.20191107180719-034126e5016b/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg= +golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= +golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= +golang.org/x/mod v0.3.1-0.20200828183125-ce943fd02449/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= +golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= +golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= +golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= +golang.org/x/net v0.0.0-20181023162649-9b4f9f5ad519/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= +golang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= +golang.org/x/net v0.0.0-20181201002055-351d144fa1fc/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= +golang.org/x/net v0.0.0-20181220203305-927f97764cc3/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= +golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= +golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= +golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= +golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= +golang.org/x/net v0.0.0-20190501004415-9ce7a6920f09/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= +golang.org/x/net v0.0.0-20190503192946-f4e77d36d62c/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= +golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks= +golang.org/x/net v0.0.0-20190613194153-d28f0bde5980/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20190724013045-ca1201d0de80/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20190813141303-74dc4d7220e7/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20190827160401-ba9fcec4b297/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20200114155413-6afb5195e5aa/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20200202094626-16171245cfb2/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20200222125558-5a598a2470a0/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20200301022130-244492dfa37a/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20200324143707-d3edc9973b7e/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= +golang.org/x/net v0.0.0-20200520004742-59133d7f0dd7/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= +golang.org/x/net v0.0.0-20200625001655-4c5254603344/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= +golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= +golang.org/x/net v0.0.0-20210224082022-3d97a244fca7/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= +golang.org/x/net v0.0.0-20210428140749-89ef3d95e781 h1:DzZ89McO9/gWPsQXS/FVKAlG02ZjaQ6AlZRBimEYOd0= +golang.org/x/net v0.0.0-20210428140749-89ef3d95e781/go.mod h1:OJAsFXCWl8Ukc7SiCT/9KSuxbyM7479/AVlXFRxuMCk= +golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U= +golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= +golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= +golang.org/x/oauth2 v0.0.0-20191202225959-858c2ad4c8b6/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= +golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d h1:TzXSXBo42m9gQenoE3b9BGiEpg5IG2JkU5FkPIawgtw= +golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= +golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20181026203630-95b1ffbd15a5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20181107165924-66b7b1311ac8/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20190312061237-fead79001313/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20190422165155-953cdadca894/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20190502145724-3ef323f4f1fd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20190606165138-5da285871e9c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20190616124812-15dcb6c0061f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20190624142023-c5567b49c5d0/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20190826190057-c7b8b68b1456/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20190904154756-749cb33beabd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20191001151750-bb3f8db39f24/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20191005200804-aed5e4c7ecf9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20191120155948-bd437916bb0e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20191228213918-04cbcbbfeed8/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200106162015-b016eb3dc98e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200113162924-86b910548bc1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200122134326-e047566fdf82/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200212091648-12a6c2dcc1e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200302150141-5c8b2ff67527/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200615200032-f1bc736245b1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200625212154-ddb9806d33ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200831180312-196b9ba8737a/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20210112080510-489259a85091/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20210119212857-b64e53b001e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20210426230700-d19ff857e887/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20210603081109-ebe580a85c40 h1:JWgyZ1qgdTaF3N3oxC+MdTV7qvEEgHo3otj+HB5CM7Q= +golang.org/x/sys v0.0.0-20210603081109-ebe580a85c40/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw= +golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= +golang.org/x/term v0.0.0-20210220032956-6a3ed077a48d h1:SZxvLBoTP5yHO3Frd4z4vrF+DBX9vMVanchswa69toE= +golang.org/x/term v0.0.0-20210220032956-6a3ed077a48d/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= +golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= +golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= +golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= +golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk= +golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= +golang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= +golang.org/x/text v0.3.6 h1:aRYxNxv6iGQlyVaZmk6ZgYEDa+Jg18DxebPSrd6bg1M= +golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= +golang.org/x/time v0.0.0-20180412165947-fbb02b2291d2/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= +golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= +golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= +golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= +golang.org/x/time v0.0.0-20210220033141-f8bda1e9f3ba/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= +golang.org/x/time v0.0.0-20210611083556-38a9dc6acbc6 h1:Vv0JUPWTyeqUq42B2WJ1FeIDjjvGKoA2Ss+Ts0lAVbs= +golang.org/x/time v0.0.0-20210611083556-38a9dc6acbc6/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= +golang.org/x/tools v0.0.0-20180221164845-07fd8470d635/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= +golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= +golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= +golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY= +golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= +golang.org/x/tools v0.0.0-20190312151545-0bb0c0a6e846/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= +golang.org/x/tools v0.0.0-20190312170243-e65039ee4138/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= +golang.org/x/tools v0.0.0-20190328211700-ab21143f2384/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= +golang.org/x/tools v0.0.0-20190425150028-36563e24a262/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= +golang.org/x/tools v0.0.0-20190506145303-2d16b83fe98c/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= +golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= +golang.org/x/tools v0.0.0-20190606124116-d0a3d012864b/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= +golang.org/x/tools v0.0.0-20190614205625-5aca471b1d59/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= +golang.org/x/tools v0.0.0-20190621195816-6e04913cbbac/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= +golang.org/x/tools v0.0.0-20190624222133-a101b041ded4/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= +golang.org/x/tools v0.0.0-20190628153133-6cdbf07be9d0/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= +golang.org/x/tools v0.0.0-20190816200558-6889da9d5479/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= +golang.org/x/tools v0.0.0-20190911174233-4f2ddba30aff/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= +golang.org/x/tools v0.0.0-20191012152004-8de300cfc20a/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= +golang.org/x/tools v0.0.0-20191108193012-7d206e10da11/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= +golang.org/x/tools v0.0.0-20191112195655-aa38f8e97acc/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= +golang.org/x/tools v0.0.0-20191113191852-77e3bb0ad9e7/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= +golang.org/x/tools v0.0.0-20191115202509-3a792d9c32b2/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= +golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= +golang.org/x/tools v0.0.0-20191125144606-a911d9008d1f/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= +golang.org/x/tools v0.0.0-20191130070609-6e064ea0cf2d/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= +golang.org/x/tools v0.0.0-20191216173652-a0e659d51361/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= +golang.org/x/tools v0.0.0-20191227053925-7b8e75db28f4/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= +golang.org/x/tools v0.0.0-20200117161641-43d50277825c/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= +golang.org/x/tools v0.0.0-20200122220014-bf1340f18c4a/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= +golang.org/x/tools v0.0.0-20200130002326-2f3ba24bd6e7/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= +golang.org/x/tools v0.0.0-20200204074204-1cc6d1ef6c74/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= +golang.org/x/tools v0.0.0-20200207183749-b753a1ba74fa/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= +golang.org/x/tools v0.0.0-20200212150539-ea181f53ac56/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= +golang.org/x/tools v0.0.0-20200224181240-023911ca70b2/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= +golang.org/x/tools v0.0.0-20200304193943-95d2e580d8eb/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw= +golang.org/x/tools v0.0.0-20200505023115-26f46d2f7ef8/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= +golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= +golang.org/x/tools v0.0.0-20201224043029-2b0845dc783e/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= +golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= +golang.org/x/tools v0.1.0 h1:po9/4sTYwZU9lPhi1tOrb4hCv3qrhiQ77LZfGa2OjwY= +golang.org/x/tools v0.1.0/go.mod h1:xkSsbof2nBLbhDlRMhhhyNLN/zl3eTqcnHD5viDpcZ0= +golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= +golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= +golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= +golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 h1:go1bK/D/BFZV2I8cIQd1NKEZ+0owSTG1fDTci4IqFcE= +golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= +gomodules.xyz/jsonpatch/v2 v2.2.0 h1:4pT439QV83L+G9FkcCriY6EkpcK6r6bK+A5FBUMI7qY= +gomodules.xyz/jsonpatch/v2 v2.2.0/go.mod h1:WXp+iVDkoLQqPudfQ9GBlwB2eZ5DKOnjQZCYdOS8GPY= +google.golang.org/api v0.4.0/go.mod h1:8k5glujaEP+g9n7WNsDg8QP6cUVNI86fCNMcbazEtwE= +google.golang.org/api v0.7.0/go.mod h1:WtwebWUNSVBH/HAw79HIFXZNqEvBhG+Ra+ax0hx3E3M= +google.golang.org/api v0.8.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg= +google.golang.org/api v0.9.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg= +google.golang.org/api v0.13.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI= +google.golang.org/api v0.14.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI= +google.golang.org/api v0.15.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI= +google.golang.org/api v0.17.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= +google.golang.org/api v0.18.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= +google.golang.org/api v0.20.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= +google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM= +google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4= +google.golang.org/appengine v1.5.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4= +google.golang.org/appengine v1.6.1/go.mod h1:i06prIuMbXzDqacNJfV5OdTW448YApPu5ww/cMBSeb0= +google.golang.org/appengine v1.6.5/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc= +google.golang.org/appengine v1.6.7 h1:FZR1q0exgwxzPzp/aF+VccGrSfxfPpkBqjIIEq3ru6c= +google.golang.org/appengine v1.6.7/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc= +google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc= +google.golang.org/genproto v0.0.0-20190307195333-5fe7a883aa19/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE= +google.golang.org/genproto v0.0.0-20190418145605-e7d98fc518a7/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE= +google.golang.org/genproto v0.0.0-20190425155659-357c62f0e4bb/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE= +google.golang.org/genproto v0.0.0-20190502173448-54afdca5d873/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE= +google.golang.org/genproto v0.0.0-20190801165951-fa694d86fc64/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc= +google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc= +google.golang.org/genproto v0.0.0-20190911173649-1774047e7e51/go.mod h1:IbNlFCBrqXvoKpeg0TB2l7cyZUmoaFKYIwrEpbDKLA8= +google.golang.org/genproto v0.0.0-20191108220845-16a3f7862a1a/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= +google.golang.org/genproto v0.0.0-20191115194625-c23dd37a84c9/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= +google.golang.org/genproto v0.0.0-20191216164720-4f79533eabd1/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= +google.golang.org/genproto v0.0.0-20191230161307-f3c370f40bfb/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= +google.golang.org/genproto v0.0.0-20200115191322-ca5a22157cba/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= +google.golang.org/genproto v0.0.0-20200122232147-0452cf42e150/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= +google.golang.org/genproto v0.0.0-20200204135345-fa8e72b47b90/go.mod h1:GmwEX6Z4W5gMy59cAlVYjN9JhxgbQH6Gn+gFDQe2lzA= +google.golang.org/genproto v0.0.0-20200212174721-66ed5ce911ce/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= +google.golang.org/genproto v0.0.0-20200224152610-e50cd9704f63/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= +google.golang.org/genproto v0.0.0-20200305110556-506484158171/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= +google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo= +google.golang.org/genproto v0.0.0-20201019141844-1ed22bb0c154/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= +google.golang.org/genproto v0.0.0-20201110150050-8816d57aaa9a/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= +google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c= +google.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38= +google.golang.org/grpc v1.21.1/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM= +google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg= +google.golang.org/grpc v1.26.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk= +google.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk= +google.golang.org/grpc v1.27.1/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk= +google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8= +google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0= +google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM= +google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE= +google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo= +google.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= +google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= +google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= +google.golang.org/protobuf v1.24.0/go.mod h1:r/3tXBNzIEhYS9I1OUVjXDlt8tc493IdKGjtUeSXeh4= +google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c= +google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw= +google.golang.org/protobuf v1.26.0 h1:bxAC2xTBsZGibn2RTntX0oH50xLsqy1OxA9tTL3p/lk= +google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc= +gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f h1:BLraFXnmrev5lT+xlilqcH8XK9/i0At2xKjWk4p6zsU= +gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/cheggaaa/pb.v1 v1.0.25/go.mod h1:V/YB90LKu/1FcN3WVnfiiE5oMCibMjukxqG/qStrOgw= +gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI= +gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys= +gopkg.in/inf.v0 v0.9.1 h1:73M5CoZyi3ZLMOyDlQh031Cx6N9NDJ2Vvfl76EDAgDc= +gopkg.in/inf.v0 v0.9.1/go.mod h1:cWUDdTG/fYaXco+Dcufb5Vnc6Gp2YChqWtbxRZE0mXw= +gopkg.in/ini.v1 v1.51.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k= +gopkg.in/natefinch/lumberjack.v2 v2.0.0/go.mod h1:l0ndWWf7gzL7RNwBG7wST/UCcT4T24xpD6X8LsfU/+k= +gopkg.in/resty.v1 v1.12.0/go.mod h1:mDo4pnntr5jdWRML875a/NmxYqAlA73dVijT2AXvQQo= +gopkg.in/square/go-jose.v2 v2.2.2/go.mod h1:M9dMgbHiYLoDGQrXy7OpJDJWiKiU//h+vD76mk0e1AI= +gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 h1:uRGJdciOHaEIrze2W8Q3AKkepLTh2hOroT7a+7czfdQ= +gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWDmTeBkI65Dw0HsyUHuEVlX15mw= +gopkg.in/yaml.v2 v2.0.0-20170812160011-eb3733d160e7/go.mod h1:JAlM8MvJe8wmxCU4Bli9HhUf9+ttbYbLASfIpnQbh74= +gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= +gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= +gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= +gopkg.in/yaml.v2 v2.2.5/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= +gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= +gopkg.in/yaml.v2 v2.3.0/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= +gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY= +gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ= +gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= +gopkg.in/yaml.v3 v3.0.0-20200615113413-eeeca48fe776/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= +gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b h1:h8qDotaEPuJATrMmW04NCwg7v22aHH28wwpauUhK9Oo= +gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= +gotest.tools/v3 v3.0.2/go.mod h1:3SzNCllyD9/Y+b5r9JIKQ474KzkZyqLqEfYqMsX94Bk= +gotest.tools/v3 v3.0.3/go.mod h1:Z7Lb0S5l+klDB31fvDQX8ss/FlKDxtlFlw3Oa8Ymbl8= +honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= +honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= +honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= +honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= +honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg= +honnef.co/go/tools v0.0.1-2020.1.3/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k= +k8s.io/api v0.21.2 h1:vz7DqmRsXTCSa6pNxXwQ1IYeAZgdIsua+DZU+o+SX3Y= +k8s.io/api v0.21.2/go.mod h1:Lv6UGJZ1rlMI1qusN8ruAp9PUBFyBwpEHAdG24vIsiU= +k8s.io/apiextensions-apiserver v0.21.2 h1:+exKMRep4pDrphEafRvpEi79wTnCFMqKf8LBtlA3yrE= +k8s.io/apiextensions-apiserver v0.21.2/go.mod h1:+Axoz5/l3AYpGLlhJDfcVQzCerVYq3K3CvDMvw6X1RA= +k8s.io/apimachinery v0.21.2 h1:vezUc/BHqWlQDnZ+XkrpXSmnANSLbpnlpwo0Lhk0gpc= +k8s.io/apimachinery v0.21.2/go.mod h1:CdTY8fU/BlvAbJ2z/8kBwimGki5Zp8/fbVuLY8gJumM= +k8s.io/apiserver v0.21.2/go.mod h1:lN4yBoGyiNT7SC1dmNk0ue6a5Wi6O3SWOIw91TsucQw= +k8s.io/client-go v0.21.2 h1:Q1j4L/iMN4pTw6Y4DWppBoUxgKO8LbffEMVEV00MUp0= +k8s.io/client-go v0.21.2/go.mod h1:HdJ9iknWpbl3vMGtib6T2PyI/VYxiZfq936WNVHBRrA= +k8s.io/code-generator v0.21.2/go.mod h1:8mXJDCB7HcRo1xiEQstcguZkbxZaqeUOrO9SsicWs3U= +k8s.io/component-base v0.21.2 h1:EsnmFFoJ86cEywC0DoIkAUiEV6fjgauNugiw1lmIjs4= +k8s.io/component-base v0.21.2/go.mod h1:9lvmIThzdlrJj5Hp8Z/TOgIkdfsNARQ1pT+3PByuiuc= +k8s.io/gengo v0.0.0-20200413195148-3a45101e95ac/go.mod h1:ezvh/TsK7cY6rbqRK0oQQ8IAqLxYwwyPxAX1Pzy0ii0= +k8s.io/gengo v0.0.0-20201214224949-b6c5ce23f027/go.mod h1:FiNAH4ZV3gBg2Kwh89tzAEV2be7d5xI0vBa/VySYy3E= +k8s.io/klog/v2 v2.0.0/go.mod h1:PBfzABfn139FHAV07az/IF9Wp1bkk3vpT2XSJ76fSDE= +k8s.io/klog/v2 v2.2.0/go.mod h1:Od+F08eJP+W3HUb4pSrPpgp9DGU4GzlpG/TmITuYh/Y= +k8s.io/klog/v2 v2.8.0 h1:Q3gmuM9hKEjefWFFYF0Mat+YyFJvsUyYuwyNNJ5C9Ts= +k8s.io/klog/v2 v2.8.0/go.mod h1:hy9LJ/NvuK+iVyP4Ehqva4HxZG/oXyIS3n3Jmire4Ec= +k8s.io/kube-openapi v0.0.0-20210305001622-591a79e4bda7 h1:vEx13qjvaZ4yfObSSXW7BrMc/KQBBT/Jyee8XtLf4x0= +k8s.io/kube-openapi v0.0.0-20210305001622-591a79e4bda7/go.mod h1:wXW5VT87nVfh/iLV8FpR2uDvrFyomxbtb1KivDbvPTE= +k8s.io/utils v0.0.0-20201110183641-67b214c5f920/go.mod h1:jPW/WVKK9YHAvNhRxK0md/EJ228hCsBRufyofKtW8HA= +k8s.io/utils v0.0.0-20210527160623-6fdb442a123b h1:MSqsVQ3pZvPGTqCjptfimO2WjG7A9un2zcpiHkA6M/s= +k8s.io/utils v0.0.0-20210527160623-6fdb442a123b/go.mod h1:jPW/WVKK9YHAvNhRxK0md/EJ228hCsBRufyofKtW8HA= +rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8= +rsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0= +rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA= +sigs.k8s.io/apiserver-network-proxy/konnectivity-client v0.0.19/go.mod h1:LEScyzhFmoF5pso/YSeBstl57mOzx9xlU9n85RGrDQg= +sigs.k8s.io/controller-runtime v0.9.2 h1:MnCAsopQno6+hI9SgJHKddzXpmv2wtouZz6931Eax+Q= +sigs.k8s.io/controller-runtime v0.9.2/go.mod h1:TxzMCHyEUpaeuOiZx/bIdc2T81vfs/aKdvJt9wuu0zk= +sigs.k8s.io/structured-merge-diff/v4 v4.0.2/go.mod h1:bJZC9H9iH24zzfZ/41RGcq60oK1F7G282QMXDPYydCw= +sigs.k8s.io/structured-merge-diff/v4 v4.1.0 h1:C4r9BgJ98vrKnnVCjwCSXcWjWe0NKcUQkmzDXZXGwH8= +sigs.k8s.io/structured-merge-diff/v4 v4.1.0/go.mod h1:bJZC9H9iH24zzfZ/41RGcq60oK1F7G282QMXDPYydCw= +sigs.k8s.io/yaml v1.1.0/go.mod h1:UJmg0vDUVViEyp3mgSv9WPwZCDxu4rQW1olrI1uml+o= +sigs.k8s.io/yaml v1.2.0 h1:kr/MCeFWJWTwyaHoR9c8EjH9OumOmoF9YGiZd7lFm/Q= +sigs.k8s.io/yaml v1.2.0/go.mod h1:yfXDCHCao9+ENCvLSE62v9VSji2MKu5jeNfTrofGhJc= diff --git a/hack/boilerplate.go.txt b/hack/boilerplate.go.txt new file mode 100644 index 00000000..dd467e1f --- /dev/null +++ b/hack/boilerplate.go.txt @@ -0,0 +1,37 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ \ No newline at end of file diff --git a/images/adb/adb-id-1.png b/images/adb/adb-id-1.png new file mode 100644 index 0000000000000000000000000000000000000000..4cbf8c5a730b4cad2496efccce19b24f1daaccc2 GIT binary patch literal 54775 zcmZU(1yo!=^FNF)6o(dfr?|Tnic67F+}+(BTHM{;THM`jk;UB|ihE&^cl*fq`JMN_ z=j7bn$w)G@ndBy)@Gpu|sP72gK|w*G%1BG7KtVwtKnf)y0)%qdR4@X05w{c<{~{wU zPWHvg-pta*6bgzi-q^qZONNnd#L&>dU}S=U{+*M%N?2Hwih=)V&tT6`Pj4QOEHh0< zXA2Kw3s$WUs!+MN*%~E^?5U1id8tV$mjP%qeBjdm;=0;=T(zdyj3&$SQePf(76C09qhElj3`g;iz##7G_xXgCar`-W-slOF{| za2OvCZ2*S~hq{lr=TCGo@5B;f-@yi`r-%RS?QL=!1?4mY1;xMm?d{DQ{PyOb%ErpN z3G@2}0V>r}PdC)}!U^J4byE!)GX({xkB~Yd6m+O16da@m4JibW0tE$|@B<0}d166| zL>|n4TcHo~VE?I8{$&(X6_=5LJXMXIOik^aE$m%7m{)fosunEOG+Z@4i;Ob`wxXHPp9Lk}i9XUcyo`L7-cQ)go*O9vNAdpokfdJT>2U0sAIDE=Dy=lAb^ zntE9N&yt<}(XGtZ);e_FYQC*eaV?KbJTBU%Pmf#_VcGk2(wUidZGdWxFd?qT z`!>P5`y%Q=@X+Uw@4|b-SvdA1+;koy9ZW<3#otFI4D_Tr`S*AE|05wlQ=6S3{--@c zl;M+Tc%8su9{v-T8P(367%bCFn<-=!)u76XF7KH!lU;_nQhi-E5w;i(OE}_Bs zBLXD|8$r0}Fzcpd$(+ew16r6PK;F|ji=S-mD9Ms)|Ls=>DRe5=!Aik<&m3XJYOLo_ zcYBXH99e$`7~A)OqQ1%UgtmMiv@tEb;0P4P-QASp#JhwQSf*v1ZZ9w63Oa0n6Ee6; zK8tyX4X$uT3=$ha25joXrj3M;-4s)th@25=YmsFR0Le2pl5cLSW;mf>J1k1VJHK6kd9HbNE}EYoz!O z5|whebSA)!j<3X3R2ocj*NWvb6Bghnc?wqsm4xHH{1&?MM|kMI`7Px6*|Z)u1of5XVA88k(kLKNkLHP@2Ji2dd2p%FxUI z@b%x3;k89<*?unOZkSz=HMi{DUqVyU( z*~qJ+F6nR2LsE>Bs)7s&Z(vi1=xj8NWWJqIWg9&1oX!eXM5~N14MOS+=>`54PIC}B zje)ZM@|~%hN_y6BRiONo&-ti&`)OiII9yfI_VzZtQ2XG>h%e#mf(W&Inrrg(jn9c>H^9|uHv0@57 zJ=*Q!EVo~fDLVcv%YWfZoqYCwkdBjM$zYa>Eg8er_hm}gTm~sy`Iu)1!N5?_wEaRk zT9gV34i;^?Q%aGYvKv#cNKkps=CzB0qc|&d?oMhj*0k#+8$iymGC4%V7p%z*`Qc_$ z17JO$+5ctou<>%Ya~7Fe_$7yAeaCdTN~q4{_+2p54GF(V-mTmzwX<+0t)(S?$t!tp z?+jl@BDuZMR)xYMR9U!xO-0<6X!vK+pB0>{nKaB|^{SAF)nf+Ikelg?kb0g~KJ0%_ z-Lzpx>Ff}srlTv{=mo32+X+}f3x4!W912cq$H%4z0x_(0E8f`^njKtRm`h(|^}_C- z433#hcEPHksT_dwDf{|G7xq;GUrNm4s%S~A{B+wS3|M1ROvk>BWn6I2@s%Z>wg;Uu zgL_eul*y;3O#AKKi>-ATKj*@i zkrIO>W10;&3LIHdK7`7p^E~WV+WTUa@dt{-%mdNV{d|KVkv1wBc~V4a!_Pe?DJf-# zL}A$m{s%TZ2aDd}haF|Gtx3o57u@(8y|a^!A@oGgPiN-g@|g)u3J3>M;G?et+?`e zRPmLoO>$17nDk85qQ>*O6o8GXm^0JM0Kb`9V*uYUHX%R4LO$I^FYde_xIzO`%+j z=4?$jn6m|hkBRec;>ic;Y)b6<2H9jc#nx%hB7;1kZI7(M(vtT4#zQJ)SJ7ZN2QJ|> zb!io2FU7W<`f&=AKHzg+mExqeX8LkMcILOjt|s{f$|~z3l)-_+N_UY2@i-BvQr+dP zvt!c=YzFn}#4cYinaT5Ylba?g@Ae^mk`c^>u55|VZftj>qoCwqsTi1VaUb@`hQy#> z&5Q;C^L&>iv_zjjQLQSc8sWfKtD@g^t=Gej=6@4?;6|sn>E%|h8H|ek-mK5= zK$&|6yk+@LXPy|~xh77zJ7~By_BX9G2?C zUJ5LQ30W<1;EzC|@ky%HQc<6LsUoG>>XCqMwPT-6u5W#B9SPHdQA}?4iumXqi zNwQD0qfiQ^Kg6KmRK!DCRyHr01fA%pM4tECN+9ob-uH3s8ob2dV6xgDoYWPX##nv) z^ZLfxeGhL59bT?VaRYinr>spn?VLxl6Kf~vm zm?Sb#u_{eQlYS(Z^^O7TewynQ%Hv+-V?uf&C3Q^*O30&6-;%P$rW!rVuQr3&q45Qedi4iOE`XS1FH8Ud3L3=YQzEl#j}qbndtRlLA$;=VRfeqq>( ztOLZE<-<<0yV?>piB1b2)sGYyJ$3>L9Fq#2Y64p9$|TZ$9j;Vj9x^9kjP*Zr!n`qg zlpn66OUyR7%Bb(?nEaxT-*{p?w&m(~YOhT)=E$7td6VHR%s4yR?}DQr8JQfu_-JV0 z-D0%v&`dI(%YJDLTial&x%|hwCjc5N$T8$0`^0#6F(tD*J>~&F?Z-9SWgAn!PT3); zw(!2OFphnb(5&`j_KmSlit%hstUzoy=rddAM6hiiQhxFxAL+ti6aMTXb!xN{J`+w- z|G_ZlGpHMfP`jyKo)&zLz_utfwF2{zj836S;e*9}QTX#8j(N4^cM6T-28mA|G=Pl+Cn>RI+vPO5(lrH~N|xFoIZ4n*m})QJzndGey)Hr~*S z9gd8GI97-NZ0G~2yHm-9%DaW_vi=tNr+Yy9P}7N$N-Xq=YN<~h(+ezV{F)mXH{oyv zyDDR+SS0UMy=YZq@50)Nv0=UFc&LJt!{Jub0*v;v@?rV-M-?ruw43Yz^r?7t&64|1 z*ZY(@tMcP5Iyhl5F$s+)+*{xJ&*F08Ek}2PcZj4%v87@;Po^$>|ImkFC(=k<-Q6Wx zS9M5~7(DLGdg{ae`Y7+iKU!;?zIkFSK}F=@W$g4h;XK$aC&ESFr%@EGn22j|n$LFT z58F()zEL3!H;A2Q-W?5xC=Ux8JNmxIPa(P$BgTn{@ejFUsgYB!-{P-tER5a*X}z|G zplr$6gC2>>Li^1wjkc@Q4-D#QEt*rDyatI|=p=%4-uI_$!5R>@V*PHvl6R-oQih*3 zx_nycBb~kwHrWn9TP;@U_SG#Nrn_Hkbbj}=S#O)yaG=9&JPHYcQ%xn7eK=pY zNOZ{hAiRdMHqODpp~`KuWKflOIvj;>Y-0B4>FFuC0m)5$ixf*E&1Mq>st6NlN$HPl zMaDs&QFE|6(Q0_3@!0xCva+&l&hIWZH`8P^N%o0E{5pE$>#*d0l&h9zAOW~|4#X=Y zV~8wAf@!=0oEEM4GId?bhWgh_e|P9TE7g7$Sa|yklqXJ^l{Vtu#vhay~n@bUs}?b)ld78=J&Af zUK0C7_28uqMkeuYwcEejGScmA1wWl>$OikKJZ0G1#`|Eh1`>zgb(9noQCuVU(Dz79 zY-75OR>|?vrnR}(>AM`C#w{ufo|7JGhe>|P4e(dDij8vtL=V?w^8=qRhnBC$z!yFH zPrH1jmX`u`f9(3zTV3wa6{PG~t1@|nIt0hPSfSV5x8kmLDkG@~MpLYtfm2@IeQdut zq_4jyP?h(Mu(w*td0=I+y=amLK2R-Re& zh=}?K#dxWuo>i6iXYHCF~ ze#f8ZVI6#dt<0G@e)3(qjR+aOVyJw4Vu;8{F!)}EGYP7@db*E}%r~6@g8T+sguz_D zA4HzyXE^Fvx1B5~;lIOQzh+H}fl_7NE6VLRx(I=>O0vI4#y~$hr&3_Sw6ZfX*^;|W ziwwU;x6NzR>^iS95QtB^XpK4awt6t20Ky4S00S&NDS6@3o(bqMs=0b|rNVVc@>HGO z+UgaH(|XtK9A_kmb|gPru1cGa3Vg}x zjwKc_3kwM5mPvGS(EJ{Z2atU=BS3TTQEax~w7WJ*2Ti^MsG@lgneYM0EE%;LMo@Z^ zgs7=MbK9+1ASUb$uxrpDg=)xWZ~{?AvdCEN7jo|&4>IjduJ?c?HW}`=nTfm_H2$h+ z2^uCQMKWVg>`F!ot$bx1o3+Nn1o&ZWrc{o~Gznz;&uhs5W*$VPHU8?Qh>SaIqiGJeq=hASt zi^@@vGwsKpn(U=eO&W!hIVXBpqy#(`?WXbDuSTPS7L~Oo)@;Tv;q)RlwD+paH%r0; z+>dU{%~m~Ye!+hiKOojL?skjFs0`H7S8(xB(AoGj_jK#J` zHLHkLX+sM>fh-8Q$)udCsd~QJuP=QJcT!F_F|?do>LpbOt{lGzrY+6(J^Rz~p2xOK z*V9B*j7rNHrpoD2ioDRq*G?@O_Ydv80>MU5Md@7du-WyLYDChMweBp5cEFkBgh-}O zYwdFHG&{|*J?k-uCnS3-6zaXq{Br&nSI)V~DkfYOGEZkb6F3_9u6&2*tAqo+7$Az!HW z0|Q$((cH#!ALCuhWLQPy#s?CMdaq?nlA`_bQ&+NeNvT?$3`_(^C-?`4hchplh$87v zB@zY>6;Y{00XOVl>64Y{pIeY zV7T5o4HmT5@z6S_I&q3T!7L(-T*Lk!>EW=AbXeRc^7v^fi7=yMoLM?oTDpOC7)nEi z%M}2BZ70-sXzaEVU@GJUwnm*MY?*XCY}g%Hqr&@PL3#i!JC62&)b+lQ;+Kl3tfXac zn_qAkXg6jr(&twelncRl*TP8YS0}&FtLue^q6y7MckC4jr0*FsJ)Mnu!=Zt3>^CIg zu@BOD0~wxE5ntMgQWQ`$^qW{thPi1USG|b@mQy{mu|M8x-uo^zTDvS-jW2bF;g03m z+`->3TF#c>XW_@jqDm_5;AOUJaU(Fo+Dqnj!+JlsH?DK7jb zo#)@^VY>gJ4oOj0l_n=rjO@!0`CBV8j$u=peOCI?GOw!X;}QP+amFrkJI!LsV;(7A z#yi@k{-YYy)TG!@2FqdQy!#i?0>!bRacobglUeS$_3h}FmlnkeZ2;_>vp?)P;b!%>9&Iqa;W_EOrsDvn zEdUmR57Z(BYPY$x@XHL9e-)sloNB}jj-(8CGlt$RHCQH=1IC=I04GwY8J6iHOo;OR zJ?!Z$)z0kw=(J`{m0?K0tiUVjkn!BBbO12=40Q2~L7IREyITXY0q{pp^Y@~I;kkyN zV8#p|=hWztVNZJM@HEhOMmq2pGo~w(L%R9c@=ph<-D>LUSyo^Bz3QX!mhPWH4OuO< zo3+u>JryFrwC?m2S3~dk8p3yw-TBx-{Xhr3z{!@P1(uOpY0oTiD)`3(sdUP#FSv(M z5YamveQruaiH5Y(7d$ZBBf#?>93M$V?Cp0ZYB!r;vJfoFQG>>Byka8gux*j{jyHlB zKK?XSEzXfR{*eBn_8`7P1I>aTj!ZLzclrH8$_SI8mh8oYeRw} z=%XVd5~j(|HM*i@=}Mw-`F-En))Q($!!kj?-mcSR9~bYetLv06ep}1Wx5SOq-lh}= zI6OEMW-z0o@4!18&5DkFVh?-6GQbYo>?c;_1xrdG#mhFeGGa3pGbJc5N>WCHaP~x! z_z}Z>_FSUIiSvL3@ZQRZ%J7~tfd&>SBhYpzFhOEWl;}@(QEy=U$_JzfEfd{=o?9ZD z(|{r9j9{upJOPW0sO(q6j2|#J#+uJd&*`Mqs|~I|6#=Co^2bY-MO|Azq888WNz6uV ztaoe$>DdId`%M`byO+YVD?mhm>Fa^PSnulSz_n*~-~(RnBZjKch>IJ>p|2K`^gDT_ zH4Wx!Y-hAT4Q3ebXnPI3HLoa9Cq!v6I50zp5)mnw3&BMr=v;0khI|$6$?=|-0@d|8 z8FtGDRDF=GEoHMaUsCd!&1&eC1`YWA4Vh&vOD$uJP*UcjYJW6jr}+{@W4R-iZnF%! z+vaMInG}{oDWAyk5LPm6ExP3I#5Iag8S{VG$4MFCgMI47v-N z)c#J0m*Onxljk{{ZSC{0@$gdj_Vxyr*a}cb)^l+(!gBTGAUQ4}uT643JwPhmE0W&_dyEHX2N5SFA~%6km}E*4UHXd*-!-v576>kc*DY4X-q@#{MX?KH z-@H6^%_>`e$ZF^iZO|=jGtj?!e@wK}I~NKElo2~aXPRe>&258O4E`3pm9DSk0m8mn z2?W_qXZIH~C_ZV$ifdUMGzX%6zD+2l)M35PTFhbHX5%5aVn zwH&{SLy`t$5dJ1`>#%ELd}U&(4l(x)xhbA~%Jz*45ii*1kT%-t8SV|vAOi?z=q$T0 zy|s3RoAhz>U<;t|Q+>wb+nqxj2%+kM&L+T|*y2o`J=O8PEPrWt)4S&}El`(eO=scq zIv5Mw%9Z_1N-Q5^U_p80++9Y|j8BD81Cgn1HkhI?Lc)4Ylp*xi=C}PIR#Y=$UtM zHqKIkd}Q85^?Swfws7`1(J{!xxRw)Z*jp7ReIJm(&xQrVrs9WTuBPH)kd8{=F9Vh& zJ0|@GytDGyx(6e1Gp%hzMy-L-z9u3mzFA=S{jJc)8s<~)@x@jU$my!x)_k>uF_Blx zI^5Z3$Z&lrm+yGHE+{$hacZUKX_hCzcCwo>v#vg-fx8xU>3$&FlAd;0w~5Ko@Qp&x z75=bo!L>Wzdegqq%gDK=lEp#yrYX~GOk4yU4^7~ar6Q^gY(v%F+xcb36%3>|RhKfd z^VDoM#O{ayWU6NR-S!i1XZuRmiKS#=*MuyE$w-wJ%0;CK5temo~i_zti=pzf&CX@VaN zxJx8)kp2}db|$EitIG?2wmaFKgj}K!?@7kiFWhq%(pmFY||)6F7SiJ4nxcCc>@R*#4x z5uVe~vJpty^;7d$a>Np0?y-FBPZ#E5O(0t2!RL*e5=e8pA<1@bQ6Cnar5B+h*DC~Y za*?8~H03;7ch`58EyvwzB2*{F?dj%O~m-cLMtKVQp%CVz5iq$r_gN({+BpG@AlEF_!A z_#`wLRtvU~0i3YJ&P4I?s=r<64Y^E+WrmEnDkwreqX7<8ld1=hw8`*Zpg0K(G`0%NX<3N0uvk3 zNoV7vHI@9TiPK_8u_#TqiUJe!71P472s&1nLZLBpHb&6RUr+{ND z4zIO(pC;JZ>He%o_P2cT(2Tle^d(^`8>3-<4m8sy_DgSFS62A;%UA%rQ@ErzHCo>| z)u4$coC8Z=7wCN{Lh;b}vbHKbdJ<;9_s5l;+Pd$p>AYn-7KI9pb45OvTX9sD5yeBN zk^Ra?3NJ6`TVyBBofr%y3rtH%C#LE+Nw8d$=QI}G)){8iRT;~jrsEH(*EtY^?r>XOsS zTvFL6LGkt})k%#tX>no3tkW4>Hx!!i!&MVKSsti^Do|HM0#+>@_Xq7xaARosXP@@! z|hUNp2H(U$Tqjmr< zZTDBxEmwrmz8?O|9B04rU?)r|gcKl{8=Zmnf{k>;d>PboNpO>&Vsm0X)`WM(GiKoT zEbM-+v-|!td!Y{Z5`R4kL-54dLU6ZjSCiti_3m2r<-ew4FdY;`;fjyrI~nZ>d_taMI}+n?O)F7c~HXyOnga*jX|8Tud7WH> zxpO)#BYhez%564l3G2`BEIyHw9c0uk`S8vv^wR6LqxXTic%f)hY*W#uvUHGioOQ#H z!m3k^r}VDYCB^5~^E%yo$6LEZQCjW)u*z3t5pi)^k^%3Jmvx_vL>}^9muTfs*hC%l zA84Hq`|I0lye{~fzwvSUqe$aP-&2A@N&N%E7r)rZ^>d>L3JTiVFvoDLM{phI?b6Pj zeeS5nVGK;p6A|1#p3p@YlZo)DcS2n#?{~<+M-bUdg{+0>`JJ&L3qDs-H(P0HQP5*@ zIKI0}EbYGLxY*TQUd?;10<6oL9mG_0u(`CaW&)S(#H^>T0vP;bCs`YM&BuQ_csJc& zK`o8`tG7whqd@B>AG8D_i)?XUPu^1J82cM15{&HK>o(`*j>G46H_vBVfX$&3BLZmA zM!3;2Up__j_h_!?7v>F01UHq>Of7CchfmAb6)2$Y?Adv0hS|9;yBjx9tmi_N(pdxO zk=MVOomF2^I*W;|2}Qev!FQ6+KC{YAVAUu4ZN&x0_|W^#aEOZ+=CjI*F-eIedyVQBKe_)R4>AD5q2MDSDclC!rqNer;4v8UBW>jSUy zc3WHak4u7L`lgsF>8>6l!G03(;aU;CTdH0h~*_GQ3 z7UQ-bY=-G|ORJ8Z4TLqzVm%1KZK4T@H<~j|K5~EV-NpSM*PM|SvysGfAC+-fFf>)+ zAuC-%J_d~_;qRX;mG2svkni20eGxEz|H{oC-@|>M;n6g{a%NdulAal8dmP+vg=Yb> z2ltnjN4z>`V3J|Zu6dQjv2PcB*pmOLBLZZj2GP=Cumt`iup0UY1 zXGFl~E%~V=Tirho^1ou8*E4q88=qU_nt#JF2NWs)1MN`#)Prgy?7fJfh-9cxfyns* z#{B9@M^&usk?t=%f+wt%NlnUB4GyFOcVwad!#3Q!9(E#qwM~oT^A;RT+mH_o@=FO~ zgBIM;)bO_?$sNW^x3~oFA!4{nIr++_X*Jf*dGUyJY&=-J3J^9b7^;K!r|o(%XS%yX zQ3K)5#Y;-%MQaJRd8YI*Oy42M8aeXHeF$JA04i3)*B(@=5SM877wN+k-UGReK5=rC z9cLCV#{3Wnfuz_8SrVV+snEepd=KFF=fbVasQx#M1A?(Zgj<_RQ>rkB-wJEA$fES& zhauMc!{6_wp}!mcl>h?Ch!M3}w~w|ynOAv)U`8N)Y=In}YhJbrfeAGC%4v6(r#KJi zlaA(Y=84`1&IShf{+c_c(;oW^=U51ZfJAguJ**}O{_94F220E-u7i?6Rc8o(;NL(a z2-bv$^Wfw09G?b7fw$4y-m&)kU4nYgv)x)sFPIs6u1*@eMnoe2`CL$jwk>kGL4;^M z#TGjo7~qQWhBdrmS1;?5FwSS@3H~GFAPpFp`+AVrQ{22t2IEqX(&wFfgUtWK<-)WbzTJo8{-?U;DasE?E; zPO2|Ue@vp7ib8CWAE(;;Xrp{>0q>o36hHCCNe5$20h3I25OICIdr>p*=SS02yC8#U zfe6i$QtNWCff48R7uEEyVOqR{TO&Itp;Ns1FB%Hsp&u9y!D;hF3;z$J8J8&!{8mQj zKXJlhsb zIy|;|G5`;m5)ky5DCJ*2WcDPXI(yi-d^^ruk{cakxSfxVwqRA5;m5qZzfLHWnl+W1 zaqA%YfUR@~w8%y+R-uQng@R1H$GPloibDP|U6D4Kr-x(uS3kS=>j(jAn3HyJqobpy zo4xNnuNogrUVcmrZx*lNu$3psDrRKUfIv9RU9s!@L0%r7{htEnS{BvR}O_1d*oE zI)smfvZJzc3SckrAF^20YG+D#zLUg8B7q$scv-hu&M4PzhK=f|0-y)Y+mCVikf?`L z>jYN#2Bhp3VLZmix2+2LN`{Kgq7+WZAz>tiiqjouaD{rEswlK;0AB|4<|VBt`}2Lt zA!2G#!lGS*_%9wAF|QI2{A;JuQ3E4%mh|gjxuS2~VpxlX%^ra$D)$%h+1lK|r>~nM z)4_moE_Wx}pDi208&2n*xH8}vVw3GdQdwogZF*B(Wj$tWbDbY;Xq-!5%@Oca&}qkB z*H=w9W2opy#`uc)zo{o>J`;*p_`L=?O(*lnx~Z2`=sXvvbABBw*VlRPJ-iiU5L^^zy0O(bm-@99?2L}0=(8HcaIQJv6{izx0TeVDV z09x{^xYTGALv>>6+pE-ai&uK2C$}vpDX9oPm*>X@Hzm!stw&{dxcnNS1WG`T8}XtvLXM}pUUBYSYqAq)?znHDx8mC!z?$5q;(&TpBQH*r2 zJ>TEmq>6*xfcY=&`$ze(PW$K=2`B!~9Lr69m3#dIaub~>10XJa{w4fMY{KLW&l5*w8YOiOAgn zqE}LF9jA=qW>pz6V_$||g;uFgG%`GlC{q3QYEIW~vl(B0W6K<^f2#stO`7htAk|sj zQfby&RB2k%79`CFlKw{cRI0A6qGoZuMZmTHk%_*4uS9KBxmiWtX;e0HaFx(Dtf|2X z47JzXjg@*vRGi@ zsn*F{30gqPnRx5($Zt#xD|~b<*_Pf$icy-2_2eyz6P?QaH1kw+B2x;4oJnkrKB3u#6IwhJAq6VsvIzu%Lwa;h8meTyb2*-j{+ z7$9VNF>E@6i=G$Z)o|jdH@S$hB^yz)wv19|mxjMawp_$K+4&~zVNwLwC+om?S4PyU z^=IZWHzT|UNW4%}jOI$UKvRW1c`CDFV-kIYMPF~n0_|_%?1Z}LjBPY? z$fz@Cn2(>k1!3a7r+}QNEhViBbINPn$M3<`Spj{3@*)w!&IR3B7zg7|2as!Bkx0RM zvgs8_(j|o%YDKnAa(Rj%bGhhM%-8HcD>1XkyR;uj3&y=2U^>B%^XvWD3bXGMdZ4Bc`}n}iUUlsAoP~Ds zR2!|1L-=^dHuo_WF+({$ptzw-1BA~ZCaIW4A%q1UjZ_j+X99+^=-^&3J_tXVhszw1 zYo;G3K-)HN&P9gExv;H!k9xBRq4vS4M{{=07}eM zSYLD~73>&MGYE;h*>Aq%1B{GZsTP}eLth_;p2ypDf`LWI=IOvd4|U*?%xu&Ucs}b* z28?Ft^r+yTVf{LF>u$FJHN)JS@c>D8jz0w-_6qA9=bKA@&P9Mx@S^zLQb&$F4ALny z9ZXI|TzG%;>0$v;6U|A?SM%2|)19u8uY-S8rN$WMlkEy6SnUzB6w)XR!jZ`zbj0kV zVt6RY7x>+7*%upIgxIV$Ake(1V4HL9xQceLYw*Qy+1ZiM2ze{4 z!u43q5;}E0u5IpVti*HZIlHB&xoRJTOR06!|J{7AVN9-rmWX2QYcw z*`iB*%l^7O>4(Z>+u?#33DsLfaB(45)&80PvqaF;d9v+hCAXt-`(2XuxwrI?Lc4U= zi&xGmpjH521#W)XEF^o?c-#r*+27^S3ljVwdGgY|DFQFo^SRD^SY7k_k;uY@8T}XN z^HzH(Td8^Q?XQpC(zq7CT($_jH{fUZ>V7@npOYH()uKN9H1K;!Jvo0Ck!NR4ssY9K z7}l=wS=Z@!omCF?T}BF2B|NV&#yQgywGEB(%9C;T#GgNk2wBNrQEdfM4vnH;_-mAm zry&{X9mK&WRtNNurL~Lil}nwcGM%VnStU)wJNm;9*M+A+jVD!$c%rX(lBRpSq8H(f z!HaYmBaDu5+rIxmh`#1et;*6lgHd0t4@CntOI73x@H{ndhFztyIHJf;^H_U15dpe1 z9P+U4o0pW14%XYJ_yteKQ#*^qKj?Wb%xxkcTv?yDpx?EjkbntIducD;RU(=@IkW=i zFW{y-E*)6uLisAw>YZ*e{9U9;)4HN?&2MljTNz-?Kft&=(@(I(5$iL?7fs?6p5qDg z1%28rN6X45%`H~l-G8e26$mQ~ILu=70<-V_2r8XoNJBG>A|P2F6dNBN^E2`zMk0L=Y14cu(1CnUU4w z3pJ{CmEf3A{yhIm#K4$%mS7DvQ<1KB#f#Ud0E^)6R2eyOD)s%R8ydYUja4@^KMeu@ z<8oS)*^K@9)L5E})Cc{?=C#MIi@li>`{4eKxA7qRHe{sGu8JllB<`+{;`W?bEN*vANh?~#c0 zVtl@ZjoU<9j%_qEuQp3C+{~xkno@%)Z7`8%=Kn+}MH~a*H@4O79@--&ntA1ATWvpo zqfTd63z}lj-+u$6cc&QfdN~^Hl46{fWmb#G-@Z(EHKPBvnp@$I{FxAK-IXlm=hHcz zC*bxDSEO6C-cBCZX(eB4LSf5ETt^MKRHSH&pplYiO0g!>?|Jf9jL z50zU_G$Wf%)WNeFr^-HTzzz_CbdSj#>~IU|*AYe=UHb`Ie;;UD8`xjuTKcb(9VAXT zWTFvR0YlMACZ@2; z-@}Q{)(2XaUpZcIFPef;iOo2by>oZ56n57TWD|swzd+ycD$FvT&=KWWd^!}qU`s3& zhb5-dZ5uK=!$dS;OiUChqz4t zCpiyh^dARpdN`Bdr8Em?XZGzWuCFD!Udi+UC0+-+OZIw+E_i`Bd{sbowV*WXa0zF| zSQ=d<=ZBby6YB~}j!h>!)xKWx+nFi_i^e*uE!u_ce8<`LbnH~h(EEOQQU;OdZvEhj zA7l3unWoD@llwh6x(Vn(M7%yx&j$C=sC%JTSTsgP4(qWO!y$BEx*o4pHU9uZr(|RC zV!;e{kC;RgRvC2@DMc6k#-rrtC?70>H7eLPZ%%00G{X`;$l>%QT$Nnq(0J;*?4z=u zKjYIdzir<)^W;26`n()_lU-Wy2BRkRQi77VHQq?QH(V+n-sa?CEWgXFG>9$qT}>`) z=X$15Dojh_mrMP*i9)l(a28^kAGm}w;=cW}tCXhm_IyZ-N{+k0k5&!a!SDYD-UL=? zrjF^qJtaLK@Wc|Zh`WC#2l6axaow0%R6Va({u=rQr}#tE(=oim?O1olcm>ayu~a0h z8%6+*kJAVxZEnX&bq2)}*K{8U@*FdH=u2k5j6gQIY{boB^ghxjsq;QD1BN`@R1jXo zeb4oUnzhjt$H>aeqeq9j3UNfeIqt?|TEG$g)D*Z`u!H4b0mSfpN%zN_sCfTMr!!ns z;sII$lF<&=kKA3(i37H_zRi511v&`$+~ zP40#c)P+GTFG zt6^u_?Iy5U6Gx&vXdvU;5>nX18m7(4z9Qi29GFnPs)b6~7NsX7k2c1x*qMa52Xy;o zQPpwfw>*0x;TyJ+AbI){R&PJtZwl_=@g{ja&wbPIAesGM04Zzc)=7l6z9}9L0DSgb z)3g`rin$v8JMseLx6e<1cYP zL#1>mJ2ONMNUb8iPQBMfnhQu_dHyt{)ICx$yz$W9Jz5_t5(T{}mc@;3_l};Rs>Tux7VS?tma>PEaas=>wx>>4Df9TBwjYSG%`K?byp-cb;Dl#_D;}t9 z2l>6)Bzh!rQFO*w4UZOnu{6!!Nvx7ewDbKQYrL}dcj-=V@VPyZo4?K}*IzsO3_>@* z>FLEbOJUKsTS$c6;*86k6bf_&2;*TMQBg{w@L19)<=7A*{*KYT`H18Rk|IriE{{gX zV-;m?JEFL$iB~%dHVWU+GiJUQYg$}-#G8EN=`J-8U2Zdt9M2J2wRqvyQ;@k$rI>n$ z(X*u)F2jnBc7=+Yt1!!5j*Xte-naM$_tLXE zlj4$!76l|`Dxyl;mArvb!{OI1Rg)WHd}Ms*80o&?&Nu<6gT=u(zUZEL>YuHW*j$Sq z$k*2c$h&Y4XOadCYTe|NCOfxs8p zQ)8NGn%g5v>#bYKzQV6$bW8dLM%XBZPO*37yrBDjXlg;9kkDw40#uK;(M*3vF&(*3 znl-+UXdsQ4`dWy^eJdh|1pPD??Zql6zYi^>OyAa?DJ#CZCmp_CmX(hg+Q<>|n0{o% zw+u@^*(J{VuJ5;-jkQ_HBR{!6gKYik5~Yofnocqg1iKNMHyCMfJm)2&6AS3|3Ahh6 zHGUmZ6MD$zsTqD&&q8yjeFDmvA>*)-h@ig0`g-*{p2Et1ar!;@Am6$TIdRxr&UlA{ zb(6tmUF>e(Ro&`0jG?WEzw^aduiY&>%m4L}8T)3#7d%BCym#Jy!TtY{bCIeWBb&suAL)^~C? zm^yFKvs4hG_B#OC+#DQgq>gopIs{p0N-zrzp4r-sUJvKCQrx&y2zy&wMuPD0@O9>s zMBHVjQsypokB^U5Ki8vNN$l>9{}hB@kv5yypXB*t1cfv>VITrQ4)Co!xZ&CEN z-&+zei(V6Q+a$ei4J_0)Myyzj3ab!0>QH3W2AhzhDH-wW>mP3y6 z*SJ&370`){v5m`q_5hQ{Bp4`7_H8#Hs*3K(pt=_fcF})70E~Uvyw3C^7xH{muFwYr z?^T6u(*3IiK>rEW%7txxpjzPNTYVTVoFti$9Jdsw&QNjc1Kr1C;;O^Lr0KOd+qfZN zcs_}w=*-Q~@>$)J{_u76LC|NzKG&2<1O$u2CqjgwO4xU>8GZ>pzMuOC2DW(X3);ei zpi~>&_%&JK_8M))(s_R)1@qQK&nLk!!)klpdzGOyoo{&^>yY8Yx47BJXh81vMY zF?f$0CWBG&NUQ6SPJ>ti_}1+&n)BHGQnPl%BPT_=`RUS`;P6`2=5E=Jp3p^9yVfGm zrVlE>^?f^dVe4vbfV&-uP=F}$hd|hAKN%lz*;ICX#Df3sbeF0E_xtjx5Yg1G!aIAu zmvQk#f(v(uLw3G0_&~>NZb_lPB0uw89HacG^T9;2dG1^DSSDAqNOU@DgEeU8TaHK3 zuKkQNo##d+bs_yBFzo;YS7iY~&SlXjUI_E<)sz86sQJeGvjH22SMH{D^i*kgSKZEnesu=iq-0B{WS35fSyXIufVB!XUe5m1E{ ztJ|rD*kW{_B1WSZ)Zw@{=HxKpgHeE7D3voAWOmf{j*-d1L`Jv{J8OF)735Wj3vy~c zmp)~%`D9$es?P{d67AB+uR&)nE?g>xz>e1U&v1%1ME#45v^vY;5>iqoR7>4P!4;1O z_#RIKM5)8+5RUSxT9g9OJlr#%GG1(FR8hoaM_H=<3n@=e!G3!LU;Tv5$nb;{iNmaU zrq%JA_h+@yR$%k7d?IM~%-$f%SQ2G(?3;C+{nc*cY?7E+&~N8P?|VX%6o_WS$gkhH zryW-`1@Ge|-A^d8H%SyiBQkgtrDc~}V9>cKWg6(61eW91hb(5&9cEV=T_NwO0V#qW zxUiFZ`-TjXTf^9)WNH`=qM90CLYC$S;lCc|i&-RIjA^bOn!p(I{<;cbNg<56-}2%k z>}o;m6_Z+~r2(ueoa%hCL%rD}-ZvhX89R+L`@oaq8{iJev2H*LWp?>`F2CCD9;@x0 z@nk?*Aa=Um3A<9=gv;=74P@dn8Rt0H>+~A&@^DXhG@r<#k&GqcDD1G$70b*Kc(Hmq zt)JXmbsoaAefm|BlYq-+mUeNJ_GcP#^7UJl21)a()2HF*Je$QDk-=y_9F+YC+s{aI z@)wN)P|Nfz7*?oY4`lA&Ba+hMgUQW)3&Z!&X?a4KYs7wDExy6=p{j{4VmnYN{D4g(ts=wpOB;{BHJ*YiV`5+g5wGPFgC{d-AjX?&(2K=6w z*p1{`WA2I^gikDaFMw;HhWLCrT60F@J0v)==qk+Fb!X{%H7GL(3mXj9=E~7pBjkMP zrc)7fK8RDXVHD65NTtT}(X^oW>e>Oe{rduAgP;(`7Tt5v;HJ+1%=O*W2gO4*(d3C` zAQhJ7!K6NM_v@b6_47%!7#R|Hgpl3* z&sOAZ8RAwHJpM%wRaSpLO1U(Ptj;hbe5*Q-z2t9uo4qEWO4-yZ-F7^Nv}@)mnkdf@ zF}D3$-_>aj*W<6w7k3LA0shj-ETOQy&UF|%uRIZ(;M6+Clt=#uQQ(R8M04!#yT3bO zC!L^ba@XH&eE2EcpRH)yUK9Tj7Z)GZ6>7!*L$~*gZ^wSp^zk%#Ggv79lV6j#Het9m zM||2ON6x(ZhO2v}BTG6LWbxxP0Qn0uLTG5-TtNwj<%-sT?;z5bl!k9eC_T!n`;4TA z(=KsB-xkFxSp&_&8IFAh&lzYr_9d%um}6c4?bxsYt)#HffT2|8=T?^Jq(lM${$lQ% ze*DY%3+{Yobt>1o$68zi1@Kub=Akk%;tkj`v5-B(J5;;CV7ap4IzWQP#=Ma@Vsi7R z`yV32oC~6A$v;q8wvvU93pOGeU{&JMWklSoDao{6V16(3{+?8!S3gLh;x?h*P4_Rp zQi=4JRH0Z%JAwEm7@fGV(odHO8#RklQuTV9@h`kGg@@*__Ec`luw-XkRCvm1lJQr= z2t=&pVUcd~vh68(D$P!NpXraL-FNT3x{L2#Ev$badk%~LTD0JCqYQG$0lkuXySc>cOKfBPfA z8txKz3-Nx|ymr+Tl@KGmLP3CO!DK<>@1M2)qB@zP2KF6pm8GShnXuhEdVkSe&+>!N zsQ@s=e7b>8M32Qki#JsiJGf0RCUh(Ft!V$AMFTU42D=^^KL}pAFCvIGFv# zsB-JD0}ussn|t!OPquH0Ff+te(}pI;%>HOKazOc9LPXx-H;=Ad_=PM)U~`3Xv{#Dc zUDJY~9SjoCpShqTu*J$BHwz~B-gt7k;vX}jNvFI}nG?Ug7lh@_&rs7wkWRs%{XJ2e zhTa}im>N#FlphuU3QnPG!_-H<&gs|F^@y z|8B>z|NnKT$+3@Zi_0MQU)%Y+TT>%-EkU}MaCNPJ>?!o`*_2=QN&K~dfAp-TI{uC- zUNL4joFV&F|DGKx_`>99l}fZ`dlYEHmRMmmwa~BXMz1{|L>^S$vRxB(W$5;`!NUxi z#V>G{2;zB^AlRzDi?5Z^;aL{6H-#8sOUHPZh?wLFUE_hFg>Nb-6u_N(BI=H zt=kdb9Nm7LTIn%?%0ezjLgzk%l%hobf><2;>eEKFVWd+MsMJgh$+Gq>x)M>j^t~i! z1E0QVepMn~;jD;X$a#{jx2jcBOP;P0qiQ{FYA@2R{?dBU78$5@qX4xmRbQG0BCJv) zH2GZdE#$G3(JUz7l2R~7DC+H#KY`{yy58TNb~nLkplF8f4rz8u^yvltz_0sq{UH~y z6nDBr&r*!##64Sat)#YnV0}!*DdaYpCHkQztUEL{mCsv3tdo;l{IeVcI@40A&ebw} zencXeKFp7+q^0T;mLwFP?O0Z5t~pjBJoWFv9A%pdi~cGtWj6C*vgZnurJMyl#^~Uw z-WgqDt?_WytRTWcEa3NNG+Z*VAqCWr9^t3wdYd5V|+uH*mwcHrqQf#A7tL{ zlp{#)0>lE(NJ(>%2fzkkReaE@jZ%Gh$J_Qyo#&YoF1JyIj`xb9=^odoykFr`Uwwis z12^5yb`z}al9e$^;Hi+yQy7wXs`;5fX}?OjHizPl+gal}*Eg-FwQjxS?F3QJg8~1* zB;a0I=0!qW4B=ipKo=IH11PqyIe)CRRtz%5YMTi?j{nH5Ij}-;tA|Lq#OH0gfvQvJ z039Ge3JN;XUpT<0*(buJkdPx}q5gvS>iRYT1TRRvAm_81+?R zwSBfjw!-C41uyVCe1`I>wcYlR1Y&~7X*u;6yPt*gy>WP8Km3^IEk`mIuC( zDp1wO5Jq_T!XPp)*tdS5iS>-UxXbG4RmpYVTD4xC>l4SiL|ov~382Hu(R zg%goE&CbML9;DE|^D;GmHPi9Tb_s*!Soq%k=2fNR^@f7vh)lqu1rtj^x3_1SkQ{%m z0g!l~{oVa~DPN&`(U0dA0HRtbS+3UhQoI42)-#sjD6*{A2g@AIR<01gKp7_~sdi+i zD-JG0!dAJue45lGY%(UHCFF2+bjPncTx17usOz3`V0k6@32pTjk=pk~Z=uys>C{}-d)lw&Iemh-2 z|D@`|{r#a1|2%H$azaI2hjeAWIzH>vxQ@%R{6#N8A_Z@e*TLXE=@! zB~%n0DmFs{VMn%EHy9;tbpw;u4FUOTx{tyb@8&R_#^)b@y5=rSyD8#15@k8fAgGQ_ zuSvO*;Fh83aDz|)npFQGyn}YhjCcMTyV1LBYoGS3eUwbG6D%4d7%s)x>y2LP<>*O~u0<(@x{>b+zHxL(h3<1>ziRtcfQ5{B4+E4A<^I0gJ1MyPF(#!@ z>yE?`=CTMffQOXN{eq2c4rs1NeGr33E8zWmgz$l%{QZ3uJurK2C|9n^*b!!Yv%)g- zrDg0TYK%RmWe5p+sCvmGCu%4b24n%;6i}rjb~^PsIj;kqi?TkcA$^9C8FnIiIX;)m zH+8-e68LVd`3ol-3AfW>fJreRS&BQ3P*zcVHTb(4g`&{?r`C5W zp>VhLnJBuKt*y7~ja+7oP%+s1Vf6~>nbyNPBekP{Y5QbDD7=`WHo#UqSy9#Rf*w}&|fW^F|p_z3;G;Z?z$a`uO%*8N}xvygbGrc9wrV} znyV?3y}perExUK5XSVe;rGcpK4__b__2f)H&}OF2)Y&Hg8@Jh+*;)owoa1-~E!RQ` zu7{0-dPkgbx&|)=T#yHx%itd;0!V%6(G4Y%$>2p2?&YXvkp^MqrV^6JlB{1Cn&+hl!dXh1uVNvqC%T?@Wm7y=~d32Y;w zvx}hsc%|&J1XSzlo%gD8KO|Co+O^QV-BbDOTh&SPEnl9!WE~vxDs<{aa$YX9(F4Mx z3ttcX-zDcOR^S-;G$=)01ns9=S#N$?)*1F=w?3zMlUp7TiB0cJ5-z=#TEqY7(F_k@ z;pM8Emlt?uclC>|NTF1PI{xdD6{r8AfNV5KU=&*7@hrArsvtwbKjx<2?6&V>y1`*g ze(`_uc4new)d6XG7G+63lJMWI2mxUMlN=il=X~EU^haNEx&6Y!Lq|8keaAZmJH1D9 z0WU?jCo?mzL{rT;SyNUMS>s5cx6waTz6Cgx)v_}leT}Jk`egTYvCrcbe}zEuh<5d> zY;hF8wy=IeH#Br$kxY}s^9g#Nm+QkjlWU%vHajBvS`OX!CTDvq@x1w{-4O4Nz- ztAzlgvZ;xQtb&5zd?{otO!kEXwzEmW2PhQyuN%zpBLIl!EGxD5Y9(!70siFi(qs5r zvLlAiZkrpr=wxIp&x^SdPfb>~`rWx^!Qa^BUrz&(Sr!hNjobyHcZhux>gGz{ z9OHTyB2z{Roh3(FYROEt%|Gd96`0;DM75wYbvy(yC*`&4+tQ5s? zthmuI*FJ8Yu95^l%Yx})G<%-7Nly+lm-r&#O)2*E_eV9~V(Y}hp`^b1dS{-g;m#(Z z6c48q5}K{3c7L%c=xh1~9hpL!3x><$^L+`S(eZ{8e&F3=eR6Cp<$}{j^!an5JDEqhXAlEl2)t4Zotp_uifrj-HlU^KEFB>E8&c$5NyxqESJ;i)}5NzH0s3 zzojIYfrG?<^=~P?<^Ok$sUgn)v#$DE6Q8KH&i_I{xxdl5y6S!UGJJ#HKOI-X{*CuI z_UC78K!N6eg0Y@-kRATkIeYRcCe6Q7`FBrR;Go#*CL*y#D{z)J=-;@_A?$Aiyy9Fu zsYKAA_3zONmkWcQSi7SI!6pQkuWVV4p(&p#NV0O9;-T_u|ncR&)ot zNT92u6tPgIe${#CG1OL)ip=9`tU-abx5r_3<=UK+qsLLjF%bLXn)RklhsidwIG|qh z+Tn>t`JmdYSD#G{YQhGpjJC*H$35a~UJOJX4R`6TC~YKqC#aL?iu0@|`lZCR`U}V1 z67myZ;CyCre0HQb_GjPAD?D8>-n_oFSZq1!DPQgQ)Fn$r7o$l-`)bsrtEcSD>A9T0`)6x8SJ9iT6s!_I(8(h3 z;p4p;efd{C`NamqJSZ-{l_DT-UVo^m&|%dVeA%E=(bPPW5XXOU zm{<7r(u_|!i`)4A${Q96oXmZ!-`&H88`2^Ql#AwhyF~Lmx`+jy+u~ar?q_{Vz9nQ; zIrrJ*Vm-LS**j5O$6^8isNfZTS5}n9@c~{LuI1y+KOX6LYOEz8)uS+uvRL~Uo~st3 zin{S)L3eHSk#tp7dYqrLtz5_Zu4uYj5#D5$6w(n43q#dbhp3@-#$n%u)Nmj-W0n6##4#Fv3VSvgS@u`z;GKFI;N z2tF!0u~Fb?q3Qv(+@Jsj8=1UO+g;LOh~2d3w}#dnug5&4a>O%C5~3+uy3x@+jb*e# zro^62x9KWhNgCR*)POvLtLy88pjZA8!4x~fylKidgft3YG1SjXvD?*H5P_Z2af^kCkS2cBMJ%yAf5APQ#kJKs^ zov#+#2^#74nwq~%ic4gOO{_Lp$&8^2%925y(P^EIyRc+@vtelQ5-m8he#h}SZvT(cA>0G1C*f;E$uK@BFVPl z4w0;o6}yQRDYfy-ltaW1x?sO8!)rr)&mI>A`e4yzp6TQiX}5b8i^W#-@78C0A-FoF z^m_+pna>y|Ow=ldTe=RD-$vsp*~qYLwSMrGb*3pC?hAn^qZ>#a9UW2}xlC%s_j8=- zE?u}Xl~CB9GBEciWZ!hW0(D$x1QHs2OGx@y*}LYprRN0?vtDhhod-w4BmAH8)D4P< z!y~e1ak9fvjBsI~75N+7IL{+Ix&)CyBB_hNzP!Hr#8+>U&;@(`lIcJ@C<8Jjy!bnn z^Ir}F_Rbyz2O$Xq6g2E$aErG#ws-8{HowF+{w?py^^qc)JA=y{91zr+!7hFzzQ-n=vR*oGVAv zVw$4kyVA*uXSNu3FOraxLJ}$XrX=Y9fQ8Fz_EX1sWv$MC68UJ!ow&kQu+I0!%kwVR zKOqUy+hjYme)lH`PA^TxJQW;U0oxvcdsEgIOw1_b1i6{oisE`B8~r3Xi+C zj`j>G@h>_a8+J|1Ksgh4&r1}=0);okBUh++J>Q|u=}L2eSl;i;0xo@Qe`+R>B6jw! z+1~K1Y!&MsmF=s7*_ubV%SrP0h^W+F?{iptAeDV*$aPtOA#Cf>IV>~J@sAP7uCb3@ zU(b(Zf0B!LR($RUxl<;h^_;X-fo%e@ae~5Za z9{>GDYHw^xa@}1GuduO?LxJ>{-b}MTHC>UnZC22z@9eCKXl?)!<{5^3ShbV#>`x*0 zJXQSYvzgK+dV12GW~wcDz54SZwxiEf%kqcy)0Z`@?7X}@;d$FtT793VMwcT361HOb zlpbOKk#S(EU}sHLs6w6J4mR$Tkg`#nKS-$A^JFcyP_h|{$9`Lb2ToO)pZ&HcPk5=t zvV2Up28s|=aMbEl{57EC19rPeGy5sb9U*yc!#0VIv2_Z}XC!onoQzGjOYgN(E|XtD z0trAWDAjKM;OF@gtf*rL6zo!ccr_6DNmL@=9y-<_#OO6Tu?;>PqJvOV{vUnz&AF7N-Hj+V0d?aEUD;d5Ziwo z0H|X#@=Gc-*!5!wp8fBuAeeucL|~@~M?t{{Zk~e{hnDtsfP#a2_@*1^7ihrqLb>#UPa01Fc9Lfwi+}isSI&#wrvra-sGBs_H8=cuA(w?rA*?Z-%^X}DUU1&cBs=H zrk7NUq(YBT^JIWWJFpZAKMT>hq|tGO@~93Sm9<2er4o1{p-O-W=nE6;y_$pfCkP}| zKTJkdmHH%q(EiyW7Y3+-N9HC(poTf&Fod?wC9@5o0I!0*E6dsZ-$cNL*!BY^lI1Ho zlwY|=U%5Yse8Yi6%pz8~X9CZ%eUG(!gX4VYfE&Qt^8>FMtFp0!Ge7f*f)5r0eNyLy|3pQDRQ zoGssjRJN5Vv1eBT1#qGCi&oQDk_bY9vft~UL6xWZg+KN~cd6@B-7nxPgBp|PFM5>7m=xor**0BmoNF{d9M@46i9tBIEl_b_;W z(j$TE9uT~I+!9-x`jlGvb9u4L7C(=HD=riDCs&0A+_(#&!E%jCp^mG?tfuKo8u+Q4tp8ogh|eUtdLVX6)tq@oW+FaS4;YwQg?ZHoc1 z_QrVnryoDcGc$kFDt)z-h4H=`gKzcVBr@0~b~-w>Y;}AJLOhz5sl%pc+*<7u12P~F z^sk5GGNZC@@zvQ~385l4$)1KtQ&=9~qH>~-Ix$;aluRZmU{aTEsnYB;dOl6#TYE7) zeUuS1bdMKb>2<_?D^<3ULJIs+L$5i&9}93l3SgsiVs@^tn=a|tf_ z^syoML}}n^=0(!h8m5+(4tcTIiqb@YKbICuBt%ia&loSS?JhGfXPHcs%j>J)B<7vN z>z&`i9iC(a?JVmemGUJvSOiGKXIb2Ev?{rQfn7bbYp>a5)hRmWxc$3Ec)YEzIiSQ! zs#&oAc5!;(lN9#?Gxo^BP%s821cdgR8KKE>1@!3ciZ@~srEQy3|8g>x%mtfA!>rA! z!nv3dwd&8fmJp|2!phjP9BVy9=yybfj)5Bk$ZE1%+61^JSAtgSWd(nZb zSB*54xE>J-xEuyPZz_IgY=OXiDumtTN1{o?&7R&QmM*A*Pa%@We}Z40D_~)bNq1mT zQa&9hlp)Sf3#pw zylx=Ki$oxC;b5a|AhkXkkPGo%9G$w)B07S}7-Wck`iI4Y+V|YjU7*fBJnHn1o&tmx z$^{WF(eWSpVUc~EW7jI{*IV%i#0vQRR&rX@j^vXCMrm7a+dVpE?E78BGG#I3DMxOu zs`zMs{l0U(=F0G0*pQsh%T){^WUMl|(bm5Am?z}@1H4OF;K?}Ds+tI?)s!@>X37~g z-}G`L%@s#lz=`UdZ?qiY@zqxiKPDeip?{VDP%|M>QTT{q6-h#}g?x2LT;F;-9y1}{ z4aVNjSM;f-zymYR=sH)vh=w9qUJVY8=Q8Wr{8BDzn61!~F!~bv^GyLkR+|mKG04Nwc#8z3O0Ls!t&G(xFB}2kRY2UR&!hTj zo?R?ZC=BBEH5$5viPyU~0#ppB1qzV~kfH=JV$Ef6v2m1kAvLw!6FrD0Z zB51m7K?#26n6?BNFfTwY$IW#bQEft;cM}op-B;}M&Cr7mxYzWhgyCWM&sJfWajy%k zn+5V=C#-=Z!Ap#%Y8*V)yxKOEbkPC{uYrG)HdobVJ7&OIuC#s64hD?Nx85J#tcGtt zTr)*1JZZ;iF$1%GT1n9ix4uexbFb7wdKY@%EDPgPj?_!Wnr_@j(;gni?hoi=Ua}>P zN8-jst8DzL$Jx|;B2aRMvz*QgLnLBR&aTvAucA0orAmcuStJ{jo3{*caRQt1hCJkJ z@GJgp<+{C%6Ow;aIw0{Y?oFG*u9FQM`r=2qsIWIVX72M1TI^`DfJiDH=Aj6pxVv z*@I^_LqH5oQQ4f@3azio1Odn}_Z3sW&`fVFK|}gJble;v*?po?^iaH5@6bvxA4V+Y z430|gMxK1oW^VAVb9jbgK0PpC>?Qx4CDO{}5MEQ8A2&XU9fl}rH&?a5l+9$h<&nZH zZiUO~9s2GEaJW2$Crj3xUZR!GXqfiyBX4!SLCcb|HKc1wmW(|Y6MIO=7+;Z{_Yv=5 zFBp;MmemzGT=W(kS-_Or(g{bdetf>!t>Wj?4@Wp$4Wpp?k{|@#!#AbOIxdfKeo|t6{%^6ud!1QM(;O zDxgQtSL3|<=Enkj8jinm4({4kbakwu0j^?Co>yxa6zeXK(UeMt)S0-`)T@&%#PO_ zxlxG2_VzaY3o=1`{Q=W2u^&i8DU^@V=*AYWX14QPvc2GE&(WDqJM*>?=p%&d*|B?q zeI;39ywty4ABQ6zGDGilZWfd_-All7_t=JzM|SfeYeWEln8{fMt>R8nCK8>gVrg~+ z5Qiwej4=>8BU-e)B_(}}JM*N4g3Wjwm6+xZOP_Q*Jru%R?Mo)Un$U(6$6joZHCO4x zt2V)IE^#T4(s?_&307;>1mbb=T%TTXLsQ@{6bM$oH9m4x z>-UwvgvjBu!D($_L_>BV#-9J_gw;ANG&nSVaZ^TduxcDDceva3)W> z*d9Yi;ymDpEFQ(;j!vT1SF8gGhx)cPCi`Aa>(v=`y4b8KW%sf-z;_CSR~AX*qiEK8 z1>g2lD%DxMKNxFl{8mVjXEl`~+X>e!oCmq3qb7@4u*4J0q#a$AmCEIebr3}B#kdHM zsSvX;1ribDC7bdu<(#al^8N5In;2H}7gt${f0(1hyi@_4 zgvTYqCfjVE55p&vbucnX-F2c|qji`BVZjp%40;~sukN6HMK-29Dt!;X9ZnbtH4*!_ z`MnBzbt!I)_-%xGIN4>9b&ge@eZ&qB5EEXmcVnRxX3*3MgN~O{K!rK$WYTnCw#9=^gv!Kio2roeGS>UA8x9bAM=e~t*l*GWr z)L%2J35|^9gM4)NMu1F1LOj6RTac-@_N{tN$=;r~A>E96Qy<|KIcllfnsW#r;q@Hz z62eNnkoFLlKN2+JKkYpl!q05Xudn^Xmi^=XJow1(e%Alxt)A-7o6z^ub$=zVTL4M; zRt2;|*@yqV!$-{4#*PqQ>C?1Q&wtvcVo^XYE}Jb*w`{@N&SI_V@U5$yKy$k87`qtw zKP@mf5aNAP3VSFl*Km5n2-$_XmN(re@?8!P2Jvs(2!a5B=87AeHzbYU7m6|<|2;rN zXg^fw*Riz9+IAs}95apo>s2V|@1d(R?F$9bn#KRyQ5b~K_rL2m7<7(d(fSarsSW;V zmuyNslTqodrSfc;%>Qd}8VHZ+EH6Zd_%#^po%jf!|7ow%AjiO%)06GJaHX$e?%&gu zR0j32#O?o9Dr+*x@lP+0iPV=vl%Jmx8S6C5^Lt`{f|aAThJGp;u(w-lyHOs$ot%Wn zidCvT#6lQ(MMf^ExFH%H5Vvk~?xc^qK`f#M2>2jY6F#3k=Cnk(80h({&pbC{6Ex=P zD)n8RuXzvOd_e5s1^)y?kx~2^vM(@?*`s0D7t+fgG>Vi|_BP$p6+IBA+3Zkt4Rr@g z-w5H8+ivfhcJU?==-<8VcQQm^`kyOs2@kFDljXY{5t{~O%x7Yj0?GO$f!Ezq6hJS( zXt>D8g46bQ-u`Z08Oi+L0zMwNv+a>%8SFB}(SRS_WTqY;L!V{6hob~W9m>1#Ssk-OSyY<;wH zW?%XJp^HzbYpc;ftvjLsMrvJ0vvD~GpvV#K3l066&*CqBv5yJ5$$>&C5cCmR)c%YS zI9w^LOE#LU!@d2Izb6>eSU;sGHj1}?ou{T`Gdwd~s4*%U=;+f~M6t$FwKnC| z`)ci{FEbBXoCs)F92x1cWaa0u4`7hY(wT%{4}6pdCSF>bF9}$u4m=bG#iLodyNo-H zIDJB$gJl{@<4ul6UF-a2I32&VF4y^ECI1og0_tN~jZ}5QHXwIQ!1KInn)817j;eK4 zC4ckm^fE!nI~L28=d<8G+9+^=fV1AuxbG#*mn+tOcZn^lnFf>z&5tMsfecOCFg= zC*-V5hL{%&Yura&W6sjhJ0jNqj25rL;C3AxWl5>Bsnw%dPmvVv_j21rBpsryV)-2N zK@x}18eB4kQmifZTr_Dyv9}k0L{73Fg3NFzoXAHKXe!4tc|`7QG-(RL?-D<={5&4^ z*{Ie1q}jX;SZZ-1IWEUhg^@2_DlpbwYO;g(%Pf(L8)Gw%p=Y%o)!1tB=B!4Ijj=6M ze#@wHJu}jb-9gv~7Tx>+aK*^sa>>P}b9hFPgR_xg;7zrcvzO}14{MNyxF?dl@sq0* z!lCo2%cTn$2N5iGNu*5VQbwQ;ehA~9ryJ;)F#q-H#Bmkdgzx^D{fIS=v-SIms5OUi_2Z$(5K5zavz ztMRs!<=5t7`W!wiJa?U=nabc(n96eGH6Z4i&k?$~MI+olZ(fa5D3W zI6YDcI4YhxGq7EJO}7g^YAj$%+#4 z@7aX^U_w6*NxMmQ%tASD$t)G5yAH&f_0w-MiryVhS93B2(uX4hRKxtPcQxU0=Ed9G zEa34SsndCP0FF)#sl0P^GK1#f+3Lm&TvH_pcD^-AbVvrlp^6;CWhgbti!`9TOY=;cbr^4JycMD$&{tI&!krs z<+q!sm%#1lGxt+4ng81Y~chvg|*XXwl zCz;#nAWbQkL*eOUnmEF?FBLEPG=R2jd>z#a&eNS@TFyC~F)%}s;0@(koWX2b%x=G+ z#%e0Vl$dX+LN=DnkSLeV^9_&VLt8Z1kUQ47&gK@M6WAXfTPQN)w!QbJ6BEOdD9K<+l1^+4$Qsf~snlbl!l_(24CsDZBlcBZ=dNic4$tfHlA)$^Ju!`qNa4nBwcXu zd!BP<_N8u`bjfwwW?5}lmEcj@4#JpOZ0~(67b~(`dX~mVdX>x6US-uQ$%60x2uqN{ zDdu|8yaMJq5I67UsC&J?gnl30`%sbS^u7>jMKGjLt0PxwG@?0LHtz8rUX`@(KGXJR z-oNN4@Ch%d}*kwwo_1<3;}z#-r$|U zg8Z3RYcz$Hl7|G9JjhJO`_iaqZneaR(`vb)sN3^RU8grnC18snxCb4iTS&yMs%+5e zcY|$H-HRy>2Ln_3rN0~!H55}CL-yCU`0j1%&Gz9TIVh{EGI(zo)$DY4jHH%lxv=59 zStB|LRIh|2U(?IqY0CBv$_M2rqK?P`?03Piv9tw6zL{qOk*TOZ6%73HDVR?muLA5Y zd8f_%uBNfB!XRAE^S~?KfKKSOOF-y*c}?F%dij zp&O8DniXM!W5H%59u#a1^pR;_SOCkl(z!qOWR7bwb@aNuj449z@2EOJ-l+C4ehqp4 zTM(*pk<$Q$-|1(MLz~^@@s&ESHFaDbzaPBTNz}@4TvMNVIK6kliC8=NAl$Ty-T%7A z{z?JJat%n7@(D_`b5`_J3OVYB?7quM0mbb!kYtI`0HG#wnItegYl)uv80>I*n_ZHc zBgr4TjLki>{bc+lN`b3RkHd~M>eRU%P&XcznsN(;JES2y=sAkm#utbJg~hOC%^XYpptW$8t1DB@nB} z;_k0O$xd0Lu~Y>c!@TIke2xlTakn!e@;8{6xMl8Y*3Ss(bR0(3-q2x~Sohbv22~0% z$N^FHGAccNM7}V+keyMymUGt2>Y))H0S^p#^jT$7D1jgX>l^AupAdq-kqX_sk7vvV#v0tcH?z6oMzr_0>baGkGNZ+3d(UZa4 zW126f;^s)~k8os50`#UVPyhmq!C|p#u@_}nGS<}CJRf{|pLqLd5N`{KogdZ(XyTYk zcw3YghY`5na-U>pN0mE@zsz?M}DMX6WxClJm2f_Ad2S^0ci~bHCfNSsfckG?ZttNQ0g6YnI zzJii8oWDk-6fkMzV-XvhS34aBJc!${U8|Lx5S6t>D4}BoBHLz+Zu2Ax+O&C?DP{{) z%efKTB*`zkq_loY9cAX$TI+IF%+eM8Az3YAFg=u46iObco!C#3&De+N&6r!rtC`P#MP-T{2oCbSKxA~xYKQCo=~*ihL4+iu1sA- zXLFo?!Wm|BHL~Vx0*kkhA>%CKNsAo93;iOTA#JPxqQPt;B7?ZI&6kt4y7M=5KMx(FaeWJN5$9^)_l9+4su0xyvtKN z#pgVn)9aWiza6QB5Jv_?0;pEsm#SN+B*B_*<|FAG7$GTnbw=Ybp^e!h823Pl*@R(A z%f$v)oW+NiVNveHtCyXUiBg{Jk;D>&f>``@MY~?-EWL*VmR!R22ib*t1_S?`3ODPC zL`t|A4&Oc{%~pdjFMABXk{*+yR^(UMj-HL6&sW{ex2-bA&;{k*!tA1pPtN9vdfD3G zWsY&9t{MH2j?L_4-{do9D@kusDiWq96cQ+mwU89KUH3a?ZI{87M%j*HJ`qqUAcjWF zS8b36OL~@l<+5Ehuz|vH-029V%u$e0D?>S=5BwTLNQOz|WG`iAeMNkC%&)hH@vjzu zkc#eyY|m1@(-V3?K0O{!Hs!r&+3lf#M$!DTnr!*4l!p2CEKi+Vzj|mrL=aK&q);k^ z$!Fz)FA1t83a>xEctnm1LTzpmTm7iBV+r9nBOy@T@j^aaLepa9FhbbEj7x6`GGry` z6>fb>%uhT*XN@6X`N>Mcgj7Ug?5a{akI$)Mw~yJDYvf!4CVa@{Vu9oNY4H)eFS@i7 zz0OQ1b`xyC^YU{TKMVQys1Rgb#T@>btHTbgb&f2ZPDfp_M7XQ#&ytJsWNqy<1qDs= z#r<860=TEX7T2=`5wDu~`PzK#n-`%db2663gS>FI&gaO;$Oz5O`uXUnq#>1kSA|&l zZwJC~OGXlrw5ip4a~!Nt$Lnjb240F+z;Jnn<06U-C5?61`yh@){Ct#ruRUobI2kTz_5&yKx3vRp8DH>Ic2g zrcnCJ1S`-0ej{#2zwLx3UrG$el8i8@qNoaBU15DNByb!+>IU3GHgQWgDaX-uws!=L z0?G-SrlYD2=m|K8zpwJ|z{wi=@IBuXK&4iy`aE6k|BtD6aI5r<`?a%OlWp6!YqH(S zw%uggwr$sByCzPajLFz*e$U?T^B(IDSa(PFx^RB*oskNC?zRzv*l3@}JJ>Il@&-p? zv|cRB-jZ<7p(%blSGYrS*$t(VnOvK1NCzmAI*;EOYaUR&wo1t&d2Ke-#GIE`oHQAUif$08~q3m)}934?p}84 zPu`xmcSrZ0Z2JWdT5E=%G{2aSJv4B4`QC1#Qv-?mUec~J)XlN>Vu37nipNu2MMoB! zsjbCUP0rmxE25(-1C7I&lHd-%;G$(Cx_V}m!?aY z!libhpjFnUeAXZ^`csldxscH)KiOOQB15^5#iPr{oCFMl|Wj$|^>>@5%@UGmbOC8Px-hUXP>6bO4RND>?8WqyiSW-a&(Dv~=#b~< zg?4{G{5yrr^toH#z9*iYcHiq?*zY^-yM0H`{Thb;Z7k3aO_py8QO|>_7L3ATxito{ zGN61G5(xOcZ?8Tr$t^tCEpbyY+_~BoTVx%7EH;%MFLjT9*0zNbs)9{6-e&p)+KI&ED7wWz3ptO74%<*Fm#c@V zNK~xDaVc#-e*vqy;Ij4qgL@7lpaGWCpmZzDJ;Xj~4}md0GE3eo`z)T~o9?GH(P{N7 zZi_BT0xdVP45{-^z$5?P_J#(k1;$GRycT#d%6WU@hrrmnx7GO{n6Wmp6Gbnm{%*Rm znnA}t&moA{v)kd`zG*R)6cSM*N#F(vwk zzQq$70d?~K_xU9I$CIGxQxy0v4VX>6kt9VUhH_|zTWMt>042tz!u$j{>!*P%?v~%r zwrnDJ-6i4<8SgFPl2IByBt;zs3Ht1b%@-|-mmmqS*7fwoWNt~lC~(oB8=P#R#!hB3 zWhTGWwcw5?%p)U599&^|K<4B)%^9q9`f>3&^eEVaPaTG)5@LSgx)(xc{;7$#hx7R2ZY-+I_HTWy-DSHixM4iJmCq9N45ey0Nk@Y}JCntj{17Y@#$bM| zjfl2i>O=;+_%4DjX`l-OX1g(OY4X+pU>O8Ib0$VyzSor&0zmb`3i5x->6|&x@O65!9efN)1%SxV9u$Li!Fp&Y2Y z|9|K@upm$f+hss>BcCx2w=O`*gz$PzlJd=}JD9(@6}NRs%X(Cl&2NikU^9OI`AC)R zbE`A|(M<4vIkGPU> zaL1=_J= zIbYbeSrSN3W5MW6_M${#=xfKV9b;miotwpVnb8Y=w%$)keOdR1dBBHn!BE2mk)q2- zjp&SWR)md7r-YC9e_{7KZP%n6Sys9M}Dj~Q&c7oe~@}6(3C^)6(f2DPw z`$fHHF^WHLM)bjD=h6aP2?yrB2erf4A<(OiA}Jp)bf{DDIQ>asO`DthU=Sr2v(ewt z{-RHzs>s2Ze?b=%iVu8AGXr2v<1>j#DWve+W6H6mMAK`xfP}~>RdIIE>vm%PjSQb^ zwOOO!b!K1}sT{u;7nX*4Mhl)y`gRU@u7K5%s8;=DF0GC)VZH1S2as)FAu6N(qi!*g zprhk)%J`&JO}N|)k19H)Wj>LiXhXK7=Ya>dD0H7bnRHa$%6rv;PO}s_9OZ(L%8co_2$0XWEYk0`=BBbY|BN0-wKZrk?~@4yynlVnK1l zFrwW8dvS)lUv*iU&EqJ>=XJ(-ISQdkHli+Jz($)QVuc39QQ}@}SZq{4=Qg%adP63X zHhPz$6sb*^&6p+{=r^Et77Kekp3!BP7*}VGM9IdQgkU8C`!ES6kpi9b`q9}ys{%x@0o*KJ(gn8gtN}aB?;ABUb5Om zlJwiE!U%^eb>(hLEv118N8f8}^~7$j&mZp!obF(MK{6>YXjpR81jJhiYg`}2+}am> zXNxKFUS-{^hb(rxvc(@9m6e(IOs(fW^QYNdH55yzj9sQ{_GUx$7BUv=SRESlya4o7 z$*t|-bBDCW_!66WkNw)9764@;$YR<9m7TQ9~)6a~3$t zA9!=5kUX9)O6mmDNW|hRTd!4wviIyC?G+ZY0+u~HJ#V98@R#BLCOOEMV|0?-<8#=v zvQMt)P<80OttP?lCU1Pt;*>jP(4pjMvPg<5l><@iXQD}_O60)fw7EPqnP-JXMCxo1zJp zv@=Co7ZFj8&l^h0lT$h;Snj2n3YN zrgN%cQ-MbTVwNJQCA}};qpaZ0V7Sw!)6txB+9m=s0krZ8LBX$;t@|h|L31Pux1$Wd zQnasjx(!<|MPHo9{{erZJ5IHFHN~MYM1~Va1GIPp@KXJS;7MsY;o^S(^EP_jMoGw! z(DBh{2+_%h%zFUI=}+8sis)}CKrv+amb1zAh!-cMNaVYcL#O==txAowzmvFNqyRPP za=^GH^X|#~9^M&;=fiWOd}7%!X{H{O$vpJ<;6B-yT0>La(f8Lk zuYH#|2tbfO;!aySG8(azV#S&R0Nv!YYv)aU}=I`K$%#|aNcyg`f4_2zOTOEz+ zF3ygQ3BE>>)2CznPs>elgnSe@OQmTfA6+us%rT{b`r1Zi^-}-PbyLarY+2luN?K?n zd=;o)o%(EtLmo*BVz2*DwAt6-a0R`G9Au#U*^&%7^WzpzvviG?vU1zs3N@Oq^cG7! z%|`u==xSt+dk;sFY*Er26hC#l)b$l*6J>ImX9<|havSwNM$5jzGezVQ;l z>aN&tQw3Fv8FVbmnh&2Ato+DmK`C16P@ru<3EOL~>>I1sJ2UfE@M5J;Kql`!kxUkO zpLH6U)OQsHEe@$uiM%Cqx?+=+ZsMSj@l#mUqmc^K58-ucLrB+Rv!)5zqe>j6|N;I%o0PF3{oXqM4VmmLJxQ5zUMt~`!puS}8whhdk1 zJ5JNS$DB}S-c!8HlI5RwINTFd!9IoiQJrA4Xd3pQAu^)VahiPe{z&vOo1uR7dULPRa;gs}5+yPLA!U~({>|=9 zZes?O8kH?|rzCqDug`rp^`H&SJLq#0Y^4&v6c>H=9NkQ4b%>c3|4OC|l~ze#`wy2W z-y@;nrk)oG;~_n4*h&R~I0e?Mtv@!1Cs%Pn!P9c}V_ zp^YS>N^||}&b-*@k&nWdqh~}qYza;sfn7fP4=t&GO{(j_{r25ly*1Pj1%vbEZF;~|p7(BXP z@asI+y8$4vt4OBMX-g;cJlBTkpFhyqe!#RNeMeM3f%oCQ+_}i- zT%QfRl zd*Ld92*ltS&aAfc>VMc%52zF?lQH+t2mQKa!E7fHRwAL-*+qgok{DFIot#~l@m~P` z^t-ap&#RZeCNb_$W=N#s^HPRhTFzglmR37RbrdXXjbgdE&gx-8jdG8j{* zvznD!*$L(A0p}VH3~RE8jMUg;m24*|8$ib_D1rkJ`=)8j&FAs>h~$OHG+)>RKi`te zmAN|h!Id_+f9R+es1t^U38-Bjw!9=I0+XY0R?1{jNomv(E}!odP9mlJZNTS{e!x~P z^$N*?;kZizZ^&&P<-cqN#6+jnCf$`rM@x;Z(#mf^5Rghn5Z7j{hG@RwY&7|`r_ibE zDH~*gzKU>5=ks=MR&d$x)R{i}7Mf<+tTRh<;TvWe3_;wMx=su&_RgX7O3I*(rCYmY zgEn>$fAI5yln%6lTo7OnNo#Q@qYg3YR#nL7vS^NaDkkfsP7T}iaMxvqz==~BOYO8; ztDEDOIF7)#0SL6Fq zR71rFT_zFf;Z`Y>cUoCzP-^=d^`f_O{jtM*y!-Z=wSoiZ>jCF@ppZbEH+pEl+lqy7 zQfV!&JE(LTx{QEeMcL!`6_$Prgs?j{6NfF!CKDJ?4}ZvKHr-QSV1DB8G?Z_F#himZ zH$!Jk0A)xJf$&h7l_q1h9ZrlEv(ZydyCro=#Is0$D$#00D5)B@z;p{4d*Lkrcj)@d z+{A>4*3alXwg+rV^xY1#@X#LzD&ecKrAW53g`zyeV%}*v-Hsw&ZqLZiCWMXZW0j$PUYv7N(xdz^H|BGXuS8_|wm8MrTwVn&FfPu#zxz+}!oL zfmLf{2144Re>&l8jsVk9)oS>5Yx;C1zpOnd9HnXLlExCJe#%*R1ALt*RO@6qinDaJ z+mv~9^UzjCM!Y0R8AxdMh4vzy=q-Z#J_*dm9fW=oc~rf8Bh~8YLB$HId)5x{&Ckaq z#dn?FY_RH9V>FZxePmBf_3P-T*f+2NpqS9MKfv7>*+1o zN>@e;Sa^jC2FVdhprUrwUwlrdJgF0ALb2M1&L@(aL$!TO=wG4RK*_k!4tnYKbhmsxBi6eQxI6Ngs-A9o$ zH_2rpaoSWQ8(^I3gUwyK$9`AIrWl#jI3K&`ON;RX6ZsLFdPA)=5{Ifl$nQ;VF{`I! zK}(8Ml1bKL%qp`QO*JjQ{)A)F4g2xSv^VrIKLl!Z)YOZS!vRjpR@Oy$Y_Q;`jaYwZ zl~>*qe5jC!YCCudA)AgG4|4p)Ku~amsx-9PUN;G+(INsYS(nJ%UrXg8V!_wG#U;p_ zp7eAs%hOV6srC87APfjqK8`a5Y$1AI2x+NDIq|^$<}JqL0`a)hI>J-U`rUb($pbIM zArc(K-q!2EL*e?@-O88pi(=#H{-sndn->P4j?HHNr3dBA+z<2SFEA@)8Zgt?H#6LR zU2A&d(Kc86SFZmyC*VAL?0pSW$rM7Z|I4j+_sjQ-**rjaBfq<_@9Xz;#J{VxP#zeT zV&2znGBX=+%lkonL#c!f`@Z^H`^#bJ5qPU(w8a_vW{eIO@vvA@4D}07n8~%|23Uf` zKu;V{E-6-V`gMGdL|MdCywXTcd}92{iSe9>;Rz|75qJjwd-lKgJUgAy*4?lC^+L#bK(A!~`?_)US^BDPUz??dN?}ku4UVuVZ zVo(`@$%>5F$CaOU$VDEe`aKd=m>1>hT*xp63=ixAFFuoGd&I5iank>Noo^nP)n2(0 z`0Z~bqgj7DpiM5nJ@(lxBS0t)*D`~-uw-gifgB$n3JIT=bmdRWGWyh~x_H#_91T!2 zNMIBQ4|M8x0o*JwWI9_iUvI8V%GOvWmdb8x;PRauBskoq&0^q(Xj|)e{0YHuwc$8U z1QW-O*)4XFYy1(YT;)9M5qa1@(NKmC1UYeF%PA8n?RcZ_OZs?hh56B7fay=Ge`$=f z)vqy?G-7snrk~6SLu{l+2AzvPQR~=Zr2ekmirx1kZx(5lg7V+G4Pz#W*t3 zzC%Lb&ymAk#P6i(wq2uAd8(o%ojlp*keR}6oR+@Oh|lfTek>gupg+7#>h)m>zkx+UwN;TS`#?b!?YZmDgwm|=n4Wz(r1 zH~w0kMK9Yk?dsAXcq2wQC`Aw^{B;u8Wy4`bb(=U*W_lh5X3I~Yv#L4T?f#(= z8K>s#K2MxcX>plkHF;65*UOj*bp0R+c#eqUgyDA}bf3mqv*Ik4&d>kLuE^A2F92#z z5DvfPm0Q8Pq1>BL-@q#e?u%itGcfhfkIbkcLiT-wEC^LbZ4QI=S7-S2Rtn>Pbg@+g{NPuE4$w&!tv(qEW{T^^Pse&I0->)p?drk2OKOA)Xw z&>;YKl2KtvN0*g;`P6irjX)Tm@G%b{B~A=}X_hPJj6EVzdA{5Z&`Mr>I$am#u+|;f zVae!&x%*^|yLql`zMiT&n$A*OBt1~X20UbPUlL`8A&52ehg-jswBC%58;q#A+K=bC z;5Y@Ll7*@s3;BGaS#30gu^!z^8$Thi{NTZUM+O}kT=*2ypRX_P;8{}I>sJnB8DN}c zltw|-JJ6~XeGE8X$+7#-55r26_%9w9n9u+Er*|v(_SI?||L(1|F|c8yDut;qRsUI# z+GuBpT77gh@NMRohZPvOe_W*aslaP|Ae3J;QK@_?6sz@b1iySvbrnFbO!;-E*P%pe z<_N6m`AH|dhO|E5rO=hl?bw$_x3+LvzXv!_s8&t3Ct)D)a_A@Be28jkT&c@H#Srw4@SKl;*r6VtC#~hEI~R>?9Hi};M~u?XW_$e2ntegV zg4XP1SVWkQjS0A#fkGCLMKq&lG}$?FoGXo&SdG84u|FqYfl7D`(q3K?jF3POB=jrt z{|z5~cLVpm%xm=6HO^#qi&Lw$avg&y-fgJ}Vj}bnb+}t*0j#ssJGChZ z2BjCaUPUGQe*}Ucp{#%2Wy7(j4;fve5G53DI0l!NlbWa89x!(yk3r{;cg+RgCbz%% zBx3Txp)cw6dhO&D0a$##fD>AoGy~JMpBOuRL(m->}sYX zV5WicJ4-xN3M zFGky8df?Vv!Go@&BA`mj!|j|HL8R)8Mq!dlnKws@j9~Mwa&YUKQ{pgMPFc*9`^5GsnG$E2OHq@= zH`FbK*J7j$gE>>6^i_=eq_-(v>xvZXs9NN|!+fsYe}BEDbw&q}rUD4IocfBhXrB%R zQ~Co-h_6Au_C0rWYkrrucKKmwz98#Etyg|hn<%5LlwS38rnmc$Ssfl|sz zF_iT`fb~rG*RT&q(f3roWU|cF91qDdz2}9N*FJNI7C*x-2^qgFyxk zh$uaMlV@eNC+Ex?!!}I|>5eyYw1)C6&_at-x3vxV)w7nTo%zFQ5zv&gq^jmSxOd$O zTNlD_xlQbE{fp^3U5n5GoAbZ;D-OJhZG_a4Hce)wk z^^#0I;KN5dbfuWfdz(!ERvQ|FDWcN)EN4p~I(}f|Vdi|>f5eiPsB^W?J_w`0d4)>q z-%@z*b$EZm;~gltr01zJC9d+ZIE{=cfe@Q4DUl!?tDgzy2@d{algdFojq%Cx7|m#f4V& zvQ+cX)N)j9CD)5mGs`DVjnb~Jp{I)$9!?`!#&x;EoEVV~DU$ro=C~3;h@UV250LyB zUg-SY+uk{+m&_q3JhPLl4@v0|om6s#DDuFi-2u!#!?Let=|>_#lZ+;niJo_Mi~*NH z3pVP|r8OplNI!0iTwE8pp3&pz@~dW5Ggr~a>9fQ#&MvqaAni;6UW&js;Hu_u(7%){ z&EWK}Png(8Ccl+}WiMAV{z|Ag*)I;)X?}tXHHABuCkL3~A%MA}d`p{X(GJm0e%$MI zGrS2-n9&)x6(4yPfIP#>pqo*`XMzd!lK<*8gqFjfrNwGwAAgE1E5 z?wPHJG5uD;c*p_Qlp02|ZldmqHo5K-*4sZSa3 zY9gc)saPJ?gn<`1jVXWPRK>VBVQM+)b8|N!DMZTB$UOh8*CDx>y%sie!u#MY0(!9J z)q{8z)!)NO?=dmzcH@1seo~N&_-MUD_drrKB<@WM7Wu)+d} zwJwWLll%lXcaWjB?1_Avci>H)ElxvoY~%C=1&yt_O-zb1~sDcN_nKWbD@3% zE&FRlwm9P2wT3$6xp$*H0irr*a){XBbRP#aU1IzVpPPyaz^u9lbgD?0QQ>n`t5KKY zIo(sbm1utLM|^lJ31+zLJx8+L%w&+suOBQ_e#>EVkf{Qm*0mM#TZLEcx1wTy>1L5g z3I92i8NrLm5Tv17vlpHXR8y%@B0LoGtms5(yIe1x_qNaI9d9U~$)T!VwOj3hUZG7@ z9{Ei>n#t{kxd=`qmQ)yqJIDgGnv+9E5vNY3B>)R6jLF&)N8PHehloj|G%MX7%$(fcgSpJ`2C(sZ9MZD8Fo9s4dvWEm*pC_rmsz=QHbl*IHah;00t zStm93PwJHPF@n{?n4C6J`3eCFChdWibz5&&TjKF0zY(iEX?Pr=W|&&5FO5tddm=wX zRJ-E@t#*qm1uF|CmWKydWbiw})zy9~BYOEXgmt9fIr^s~ig{10ZdOKn2 zJZn)2Xhc0TSec~fW@}O1ebiMiiIp=q#9iGy8Tt{2gKgwe)4=0CuNj=LST4?C`>Y_>&d)%c*(3) zHUTd5Byvc*|3HnMY1fo0H4T&ulShyS*?k)Jos4F(JApBGX-WTX);VU-3!1x4+-BtE z9870EARipXQsC0s{HV}YrKVW&i*%VYIgpQ>>2&EfM?-@320BbhtUzJgZIliD*iZT`De=4m8 z2+#G@6f8MHQJH$38jDOK(1&@pN_aq-5}Vy#T`vtxN+d*3m?HJS}1t&>HJ9ZFZnM*GDfRmlL&X2qLms~jm=FmF$% z1(gx$^*U6_q+N3NlC5ek#qn~ydfn6|1r{ zsSRL+n!@=w3o#Oci(Qn>4$P@6Bn$#7S=Yo+X*Fsrt_Cn4Uz~|~uErGa zMVRFGh6FmqNNXmGIn;}8fJPNny@A7KArEfMzy%RgTt(b!w_N8MjdI3-{F8a@)5Rkj zR+}r635YT-tOcabW(#F=-3~R5m4{V_O-9vG1T`pBglX5=_fl$WHW)k$*|UQM4Z3+V z$ot%Q2od0gOTD}qh|j6fcc2gxgB`=`{8l+4Y8ilOpRS3M*-RX~_G}C~ zO*uwVvfWpOUn4GR0g(`w3AlxG0`qr-pLBs5ur5i)-$NfI%RX>zzS`;ZIEVewVMHWieS(XyN6Bq7E@jxZ<1Zx4$N{ zp^x1jCZI)WU7WMgsEnF*dg|fZnDeul_>(TnAl#Ss%Qn&{4Ms*Ioh|^8*>Kgs4ntYZ zXL9Dm@2#pZAOBCd$Ge zR{~zj$gr3Um9Yivst&KqC%MoA2Vw%t^m7Jh<;d?y=z z^>{nYxxO8oi+LF?ei0}$8~vB;d?vvdIDUM-Q{ZWLO&?)MmI&TpbBcp3qg3_39g#zd zH7ry00q5J+6;2-bq_bV19LHI_A!sbq+M7VfidhDbP$Pmfbu{qjfP&J3P$VRE>< z6SU-o`wQA6oLpmA9R)|bgd%Zwddv1wttB#%%~!n&Vzk>*7#6rYLohHyzln};U$!S2 z$|z_%Z7b5ieEzAJkNa8N85FDeiPoZxCA=fg(+ZXr6? z{fMumc0`SRX?hq$ZNS~ndw2DtO<^#UCOPvs<72+h2hYE8kZ#4TxHQ(>>*;9mis$KV4Q)vOj%tIAzH6l+l309{%$Pp2>CZA zMK>2P58f7q6slI2#wNSl$Hy_Ov{{4OrN*8lI5~gX&W;<5a{wz{42Ki=d0QZIbk$lH zfUyhd@~aO}@L$%OhHxlYpx@ZAzRsFY$k|n3p8hOKc8TDXf~TMlb?O<@8l6((L1JxA zT8)(+;M9ZPi^zi|!$&vu1PM(lJ_n_Fvl>2B4`82zU7SfR7e)_Z{s4!=Mfh2?L^Oj# zwQrgLrNcyG$LqGh=}&OOE3i5R=6?L){CeS4saag)9iLptxBhE5Pf!-emqGY<50;j6 z#C#VOt3(>3G7RUr9-J<^YFo+;QaP_4QIds=%o{KK?U=4F7pAJ^n!?lK`nJeMAswa9&VhkFWj7> zuqA_qhv6YQUN#W9y@MkCDib#n_6391&h0UN7_XyWx^zDgs`;3>9TpgZnQQmEQ{Zs* zAY*4zL?XY;LJG%ibyS)?h2F1OGglD?N;quy70a7(6K|k9<^YgF+EXxZgam@lMn;@` z9tbG)jl&7J5I%+;XZ|?3fO;*FsF;DDOX(z+b& zYe^vbUJV^NqsDmcFRCwA$=KmUzo?i3$k>E@Zb{iYqf{X9xW#%qBzbdRY_UbT{QRGA zkzAMitf>6&SBoMS>J(=>crOObfL(O@$T$c%qI&~rcAHTc%mHAnfBgmc0tGYT zfhU1hX0rW2Vz~=9>t*x@LkEd~Cysd;aj{HW07q~hcyV52%Wv>X8AoM7_H+A}69 zavu*221~y?^Nj2W)Lh`PW9=(5CY9;)G&&(L>Tdc>ez0jC4o$${K_HYJTl~)WmIryD z%AJ`&?_tJnO%b<#TG_~~_7gIjx;LSo*g{}9U8^kBv6+KpqBuPMn}*pqGC{m$2qpMI z+2pe><<^0Y3e?dH z@@}kD*(EMz%vO&d-a!zH@Dy3{3D0r?lVNXJ0^f-g$qzhl;JI-nW5m`!el=cZJ^%KgO|1pSEm*I+IrSSt6X(>!D~Dn<(u z@w}_=;i-RwEeO=xgD@*}*D-Qhm_e;i;a9E$8h_q!#Ml@v=3G8g)$YqrZfGNWMkRJ{wK80Bd{3*ggDxn<>@3_#%Y-`X>`$9 zGaw%y&r3exO&_6CF4xX__>ho0T6=>45Btfa(D@>eFpgSaEYESVCtADbSb_$=pO5<= zPZtMrrrQ4ZEZgy`chxgb!@Sd3GST1bccEXO-)BzVm= zq6Jx^^t_bMY416dO;c2P2`aKIO`~)K{cm?1_k(Xh`@#yhE0bx8YN~9DvsnP zjn)y{Kvu_NJll!`eMgBYnbhxJ(ZE0t2^-S&;?d=jNCh1z@v$($>Lz#ke(;A}7S6Th zqLl%RVUd3+2*q?m&nrN!*SmvsWrXhC)?m}YvLef&VV^40_9uRso}Hv zRp!j-OpX&o4i@=m4Ml3?!Run30c%enqsUl68$`>J5skue#sTHeg6Wb*qu78S0OAXo z*HymFq11}c{~nY%nr zk4OGUje)Wr+|f7W5w<)C`4n81P`Z6)R`i6TIS%OAJnf=kd+v*8nvKA0GWdf0C79nt zkqxl`n3;yTWPN_-a=C>f**ho>03$!a$v&@HIKvHvD4RE_`7-Qn8tN2AGUzyF3wVR+ zs75Ta168KuT?#-rfD?Ndys^du^*E`*-7GVvW1Qmq-gA)61G9|i8FW?`S#71wZSW%A zP605GA{YK=+||PrA0k=250e6BP~hI#90`5ZYtv{V zz4*Ff#!1`;dGm0X)eCgS7oW`**t_0xN{LtAT%Ujg5-?8vErRR!_DEEJ%||V z-MkXujUqyfFHWYo0sO<=bac@V;{t-NF~Qq8OiaI*2hG6qdKXGv1O`bp$ z5W7)}S_%1$=Yoc5Enoi%9{YVC%qj~XThBk$MTOnj`C)9`K;paGnfe#?^|#^f&t0ax z$7`WQ$o$yRzg8S@lcBPmW^*av&V=@_)j0$JWX{TQ$MpPD08^o{r%>3Bnvq5-QJ!&% zlImi8i?HYrdMQtF9ZzV*5Qg91;=19alEfO~ZJpgyhK($Z4LZj|6Ohng@g@woCFqBC z<(_>J54MJnS+K@xhi_$^b^y@i%+N0V5ufgRyNuHZlne>_t;Y_9(Pe_5l^V4|e1kPyA z8|XZM8WnuG391nxdb=t*X0=sEH?n~X@0W7ia^`Nx8RD%UXuE|7o}i!h`02tV6B3|G zfPx;GeMlP``tfWB@6Y!*J7^5{8XeVoHI#*AqM+GueBRK>yt3>2-Uiy1EUwpFOkk_X z=?r?l{kW+6XcSkDLmJ4Th;T`LyT9B(c%OlYx9(T-f_b5f2eax6DvD*CUwO*(QjL`a`|*A3@59iJAEJ%m=V@ZiFh9~BH`f+C`n-o{ zHnB6zy?&JG?0$PS%nxI&ar~Ip&GXEX0p9Z8yv_QfZ@tgv~-F4;6WsxojTB1W*V&Fiy8~B~Y265aVH|Dk3f!#|u>|~vuk&K4vtq4rVZHmlCPK$fJpYmagS8M1!I5usI??7JtXc z^I`G8AxrLCRyiVvL(O!yBM!L-?#)Zy$>xw#C>Y2Ae#m~WgI|LXr{UXxZ-!gM{lPPT zoS&2JZ2XJ0P_Ba#cL4vGlkK3>mYlS7&i$Al@!Gj&&4$DM&nybv_E;68&HzF)+ z*Vd-1d@s9(mZs)Ivld#-R{tD1eX4=-ncrdaj|LnVZuk3$fc3ot*lg9Txu7*t)IRR5 z!K@|LiW}+uxez}vP5ybq2To`PLg6sjp&;7RogD>VGI8!tG>!WHd020N%~xjApMgyb z^DPeG{x3dXmA8ghQ}5Q)H$EzX-(mON!#omrzNRlNmVXDBduMjy!0GWe{+biNVM99F zwIe;{TXP-?i#I-Dn*pY#_NFjyykEd3XsagwF8QFsM{V{0=eT)K{8uB@2EqE0E19JX z8LoQM?w8&@UuXOnDxkFm_<8j_XE` zUHhs7MaYK=85{~B%W@6VWblFO4-1-ZV5y;1X+}Cse353I10vGa9l}0m{n-M7@c>~Z zgwU9yObCi}x@MWUvF8uyG^Zu=XaT|OM<3Nap3sFq-zG^toT#)D8waj?wVHYVWw8ut z_K-&}3}Y|B!6hIP^Kn}>xzw05x$Nlwv{AmE;sa@veg!?+fzTV}bY`cEcoS(K}J-u@XAE7rfq2S@a1tFF#5t8ml1iUJ)X1aMh$=~_F#H5 z!o*!|J<`EXG^PcNCuDWxbC?nd`clW_oAtC=0lU0l8^87rJacS-&5rb5A~+3=YR^(i_g@_P4<97;>&7kx>xluvOA5QmLltZq zUO8ev(<}T;WsajNa)lgW%mVTa4#0@XwxSK6_eDuc^$vEcG?U!XU1^E>0q<`ph7>!p zMUqwPEvQs|`lF4Grun$I#cEx?N{vq@fiQJT{#QicXu~pE7hsUL8JGp9v^$zEL{EbL z(Vae*jSzOu2((MB;+BQHxu$jmPHbd6^ed$AB#~;H@-=YQ6`e3>97JawU>NQvtA(`qJPVI-!H)QzSdg8;q! z2bXeX-8d9M{<}miqdli&^9X%ztVURLm9%=jVzZHq-}Kr^p+?+s)Ho+K@?PL~^!fdc zGMRUvhiQ!JrG5U2f&7c=3H>| zS^g9$F;8V3LbF@W+Bjh-ImZ`tkKzYqorm(uNT!USI!bIyl=r$C`liyU=)zznXm{`60G>ZQ z1>8ycnJKgtPd^bq0b9?%tp&MAIM8Yu&|)iMMAwY`YWAzv?Q*&iFO*Ql5?OgJIe}g^ zxIa+fxi_8b)wD{cZMGL`vc$^e=8dj1lDF7Ol`47hC!7rSVGCvGIgF1!uNG|B zzVs_W+4|Rqp^qjQftlM6COK&^V^s8?w9;A#(y^kTsN}GTey9?(n^6+V-UV07;1yC) z%c3?%5xNXxrBUh_t)!&X6{sXA#3+a)Bnc3rq{N{KW-xMT{=2#I_$I_`_{?I*26LgA zUf2ECJJ%nxAG_}TEM(ymYc{JBhm)wBA-i^_l2x-2ar=`86IMoOh4J|KbaZ_&=+ht8 zUVnwGu`UFmiooH@!%z6{gE2@PTx?KiwqRM|;MS1RNlu@jU%1!ilKZ|VexuW%VLBz` z$4E%=0~_b1&dEBCc1GB0jyGEHFfhrwZC*D&z6D})d`DYauv`X#a`;_Tcu zbLM$hs}e@6lurEJh96r+(BQFii`^u`pD3y{5OvsoV0cpEHBT7=#MP_ggvDuaSxX+_JQA#Hf|7w^<*i0CJ{d-7Dn*QR z>PoIDnY7unQ|Q6@pW!Jj4+#FXC70w*UkBhzM-FV|6S2e4XIVb!jJd7?-q-#Oej=j# z2dTNVDeS!@tA|s}eu`+2$z2kG7D$i#plm+h<5CQ^%IsqsR}n$Q3T9Dmey^B~SJFs+pcG5s;r6UVIjc+2YU#e3m9$Bpbgae&q^ zX+39&XQEte*E?lcFIed?VO@edR@XVX7W6@c|JL1fpo4l&P;7edBbkJs^OCZ;%sF7b_Dy+7Db=Dd>71V`d@Q|BM?YdAO^K~V;{ z)po-a%ab#bpOx~s8AeC!f@&wWrj)<6y;kh|!XeLa#0iiCR(KUI@SB*(Xx}5SZu-RF z`HOKbMSkng!ESSLXi_nyWan_|Lf#3_SX{itfEWv4Ehdn+`w52~7-*nK{d_=X=SWU5 zj@+nzqPu({C^?wd|9noBY8C1v3sNSlX0GD;kl4|}PPQJa6gM(>1KQehJB~0d2dLrd ze7!ZTZajD~)uRRp?6xTwF=VSnq*@E_-3P`iVG2(+kASMh2DngLjS;(I47EvuZ=3k-kUJeVkWtHLAS~nP+@0qS9tfkyne0Dn3x4{WOF;aJy>gPt z;FYg~K|bd4Ik}KB=_9=j3D9y|joEifHiud+x&0KQ3bTd+Exl!G_+d+n*jXsfCFPwc zeY`H!wLR9vXc`wwAUi@*Kjcv2Zk2Vbu~IMHII5-ZM!b6+P(V=&yvIF$+K3d19S6Tt zUkNTi)?RfbLXGX1pG}uTl#koe&pqS;k_~_gh%w6@pPD0Om2X;bG ztBfhNbpKro!fRMWML{JF_7X9t&Y=BBQ8W@lq~h+Q-c0r#h4296{-XGBn*sV%w;O4m z`0K&h^L+kwSlNC|tao0xwnoqL5Z?lh-8UM4&RKLI8b1D(0$+aDLY;CStwq3Lmuf0r zE#O0+J6}$rY@s8g^vwutk3O(9^Mr{2p$X~c$@Ym5o-Y}U!lxwmK7uk~Bg5CvtD;o4|e7t}klY&dVls{3j>-mnkl3;cln-fD8P<=~Cmg3eJiZ z6B$sS$(^Mb`C0}g_-QN0k)A7B%v5E)8UOR4?V5o*`^>D0Yv>5eN&u4~AenTfxlM6z z5Qk>EiQb~`8OFel(Ec&;Qxo5ju_*#JX6!a{eyE&xiYCEHEy8=dwtXPxJr7P!&%djuMe?Oph?jSy7YseiCbZPBgdb7n{BnMsux-)b16l ziCx)T$@kCC{oHc%7m+|O&TPIs7);2#+G-)&*K6ot$qEEC{Qd;6cFuP9Ym$Yzl~`lx zd*Z(pfEz$G=t*|p*1wFUE&qKfe4Eopu|i?(On{&)3D!F4OCc1yl?(V*#Ggv75FEW~ z`hBM+E5lIGus*}>+T}S(+2OR|NotRx`*`NH4ITYz$)dz?31m5OPFQJ9+?+}f*g)(ol2*m+`0MDwKBM_l*Dj%?WD zZYNoSmg_?FQ-@GF5A${C59PBCf|h5BeWC}Z2liPo`7{%myjihy^uaBt8fE^c$}Z>0 zF`oDaWtByArN9p`X$#AcRqCY2E+=hlsiGZaMXwqR)BW*Ksrvg#6b?J*QACfNsH0INTL>yx6n^u^)bhO`mX?+;B zVfd>>tsG3b?TSugxFYhgQ}a1MNT6ui}FA0QI{PKnvGkkW@dG#c9M zE&mEvlG8aTrtL20p?C&ruogzw*;W&)3UHo{b&%W@L^E28bY?8ia~%W9&5*ej&u!iL zUPH)7i`qCQGh6zkLJR@o#8ElFqjO4NN{M!>Z!qIG98jq%#ssR$Cd{b{jVN=I$bQeuL1Khs9@ z3ik(QTl{@_!S!hRYth#2#&Q_#%#74S+3}RqLLGr=@e-^VqG#%n@Wf~3^aOiNnEp^P z+-U3NI)zFr_p2&`@~Kj1nZzFzIn--dQar2N3Y2v7y8s{N;jFcZd7@nEo&EdG4;jWg zenyeK?k0JxlFCCfd>UePc}r_Fwp<31;$na2i*0Rmi7>)Li-Y=Bu3xP zivzteO&=CxJO5APBLaTU)@9PGm+L!Z{ESBvymILL@xY!Wsa4xaZASo6X?SEtgGB6! zA0|C4`#j{a6IPI2ls~IDnIy9U5;(jtB#rRz>FcMJKWK4h*39IcPO`apO7M7DFa5i~ z%8U1pGw*?D(R@F-zP{d7KLd@`FO}$Z*@(~`8sWz;)yo_jc5V7haExf2U+&q9 zt@%YJKJPAW$dgo%mltH3nOF@AdhB5Ks!sx9>Nb(K2u$Rt+?cxD-BEoHw6wwx{6U~KAdkb>F z1~C>wkVVQ96WKf&*DlCG7t!I_c6adMAuEjiTKXeut-dto=)7IYup5=@t0n zg|0W-g3PS^*!idU9FZh3<5cW7q39CrdeO~@wFzE;sfV|1awcEXj zKPqG$0tR2$W9_}y&ULSQCi1PK6b330DjXafhK#hh3LG4QBka8m@ErD^!1Sj#92~ru zrI^@T88I>7TPJ&SOB*vdIQm2rLql8{Fg?`B$j}fv24X;Ua#x9nh*2^0AMWb!8tCf& zGz82{*VSErg}sib)&uuhxx3LCJqCDJ!wfB(<@$C8y^`G?Wsc(`N~0JJDJFa7P>_#L zhj<_)oRy*bR;{P2D;>cUl$Q1~V1ZP)K^kXLS@@a+n~$_-70#GWWI?LSN;D`d@R(S* zLD)%>auR)jK)68>H_3w+m)MF0;BemrY6`vkN$=5w?Z8C%iMl=yNt|zwKn>rl&?1?h z8$pagz55M|mXcIAv)^ydp5^Dpf^c529ZJoaQQif4iQn6SrkkW5NI@roS=zg zT*8Zv&Ob=_3bPNN8lR?zwCiVV0r%Jfpl5e=ysL}%otVKhv#rh)H2^ z$gmI>_K1H%_$L*?@e|@d?{ICuGm5B+$;iM$RTC#OGdt&Z_AcH}rI#>KbCzll7l?v9 zuZg`avyrL2u^F?6t;6pkaQq&;uvc3%7bBpDt&N>CuZIBTpB%if_utVhl)yh(T&x8s zAqsDSV)jmEKn`Yh=GT;hs6Zf)-^tXRS4CX%@8Yme0+jDuTpV~=Slr#+ncd$o+dEmX zu=4Qmu)JntVPj*0UJ?O`I$pTrBPFfWPG$8QZ(M2vAb~ zuIPV%f3?%h!}8yn?41AZ7OaCTzf)LPnP0Q~Pd2P5|L<7dx0W7eHV|=3TbO!abqI2@ zaq$1i|Nl$*x5mFpsymxGiP_u23c3jXtKYv1|2OgfE%>KOt$(Xz<>vlxmH$ckyCgr$ z@1Flh6Mw1sPb^H&f~fo~{~I$wR5f6}4jh~?oQ$}Lng{%G_Ok%>*&l<*sgRado&2xD z&m_>nFVW?XuPxjS(HM=R17G>KqUngg6~&1@J=Jy7N)3JTG3W8{w96f-85!?dx;bs9 z+x407yzx03&H90AtByk;)rI4LL<0X;5a<&syx~(&4mtvGBg6$zt^XAm0=Of81pfWu zParHP-X)_ec+9T3_@CnNbdG}0|B>}uBs3gWawI%^myz#3;?}9h2mdGgx42vEK+1yy ze!tQ5OT~MhKTWwnN65Zj;hNarjQZ1(Dn4xKw7y*Fd&O+Ga6$c|KTXr3d~NjWC-|!e z!k;qNE=bGSEk!p~jT8Si>RK8ptM4RnJhx~6?VAIVcn!59@q4bJEKjsqR63P*S(?Uy+b@nt zB!Me&UB5y&TJ?}M|7bUGC04kQ8WXg9cZE{JU{+gawJN^3$sc0p?w&q>22%j#4~sE} zSAV3_e+QTj4XZTTu!6E)G$9CSkih*lHh=Uf1~8S`oOHD&Q&~`lRrL3;{SL5)@FSf+ z{Wrz|=>i`?aq~jMzlrnTv0Mk3qQS;#F=*%o&ws`wp%L2wH?8rmxBkCkfjWTgSCK#Z zu@-xl{%>|FfN6`;^7u>_mXj}+Z#Md?YTeyR*~JfKALnqOUjg3(aElPzTu(fjw#Te}&dsMd|J7jnSHFecyM6O&UN668bHAcpJTz|(V)a5*0j2aQXeUvlX%s@8>E<0N zH52YumkxhVLSZrq;Gpz!rhi!?kL-FC^!$Y$b6gi$ctG5da_#)cPYD^t&FiM|tLJ|@ zmGFcOtxj)Jk1Vb+tIb@$Si^RRt&N78pw&rg91c?u9IuW3Q1o|m7F+?_=``bI_I80w zt^zwWdxS_n>3bh47MP-9$lsVZ&K#xi!s_o9{q(it-h6SLgvwECZX4x~rWeGp->nsY zC7Z^z4};CfZ3!gW81fvZdZ*FsjPh^%iT^C1zt&6~aGjc2%J5S3zoP`zLvrJbT*_nJ zY}5U3yaQ5Ae3xA`EM?U82tV;~Cu(1^B-C_6dj$wt+j2_K5XD@qJzYz+4Z+FUZeSQNh#I7YUHQuSz(L$s}#+YD}OVMO{=ooP3g~CwGB1 z{b!L=R_RESAbnD5jk*xe`KH3DFOx;4q@PN?*TJLYmz|StlG;31FDt#c8EQ5&8j?+` zjvcOl!U*-Y;7I5-NM|s3s@I5=*>yqeS!j_&Db~wP_qp36iZsc55>Qj&v`vg|Rez5h zU(7%gjmKHT?>OituJthpw+Q2T!eu#KHMC`Uh_R;Fln62=keSdRWey^fw`XiPSIZ z6)x4>GV(GhD*aK?D#Q4TZD!HQ4Eo|x^jmUMk-jUmD`|?0aH?L#d5*XSYFdFwWyxbY zrmjvU$@9P3wbc`}@uICrWs{Enjb0Y zmo_Srw4=*F>pI>1vbnU97^Xl34o9tW>u55u#W*`HjTR1ODg#A*>m6h9O|NrXjeJuY zp`$u|+nVH=0h?JLiRhONZnm2w%pR94Z- z&sgs(i=u#TlP*2l=R`!hR3ur!qM7O+Fti^fedFib`e#v7n|RcFuKf&am9aP(xWkI) zVr^!8r{*jT@<|sN?p4*$o;+8d^Zcl94m!n?!%q#)NKVQ1JsLVO0H?ChmZ#*EI4s&e zJRib1fm=h4E2g2V_(siu_TIF;fk|JXupl{KgYWeT7)&{d)#wJl?w5!FT-va z?adn6ODdv2@TTMPV^O{hn2uqseLhLB93XA7`81&>`;H7s=Rs>Na%t;U7M{BJ^uLMc z2^Xka-*>>zs4h#82+!-;=*X)>kqN>;W8;WF@%av*pOTz|b>^wU<24Yq$eTB?4|=fCJ>qfyq`9#u@q zmo28e_^1tEdG8R|!YAR?0A|r7gGgAZ(=&tS=ti7PG>6fI%9G=?PkChX&we?{d{ZhY zrajAA5k9?js+QO4kU8ZmDI}R`sypDav&+*h)0h^1-P`o+4fj+AMzJJO8m-N2PN6pQ zx$N_arJ|%5WwdvL&uMm6sjOwx`|XeBL@!QrN_P$tN48r|Ir0lXdLJZvvHiR+U{kwl z`QA0`H1wE~@)IvrLAGh!*S;bqE6H$nmc>9Z1Bedb7I9 zLZ&79w0157{EEe2m3n9JFZ|vAIS`%D;L*97X|X@z&u)#Oaw@MBI;KaeRRsv|3Tme9 zyrdD6*xJnhkW?$kG&~|;z`_BV8xtrQS|?AHOA}GusYj7TQY@p|NvXPgr$1iyS|H~G zMfP*Wq-m>S9tFBHEPn&4N=BJ`pF=M~e&c4Xt5uEpeEBH8(6jG`*+ja}Fpu>izs+WfB)% zS&{((^-y~bOM^#ccw8wH2)7Z6qco9FYmiBmeUzHC6klP~S5}?Wh=1tUuJ@Jf@zSN( zpt>omu&*Rb`_zu7EW>8*(2Ji(Z`{wf0=F*8>Dn(PhFxTp2qdW4-Y7ioYhAHC)1E<+ zH`i0EpG{TN$*beFzlT{bC!3i;ll*!|_3bdxh9BuNc?-K3PpgpdcDW6!9lpb|KDKl*Dp-E$MbqcnpIn+4{ z26TI_U#MwVwHOChGEx)WA0Z%QAx^#3^d{}+j6;34?P;41-5n`AmzulMOUUeN$*xXT z_ZKQ<9u%IJKY1_08roML{D;)ZnG#OFoo1Lt$yPStg`M1JNR+A`r~5jxC|DKHg)Xj$tn4MuXxdVp<8$Sa9XT4~ljNV&>!&Jf zn#lD{Op4MR)~GMe9ccU>!dWJUuI{MZGWkN&0C@-COZ-6UhI8wfmkXNrO}`<;(s*H!DgwS7n9MT zxRRq&(23hEbjc+^$0CzTXf+kBi92g@R>S6Y{oM1%o_ftY->9~mB_dCQA=@0clo;_% zgt%dIqP(Dj(h#;e6As#{C~aI8FXxJ1^$wq#L7xx4K9mWsQ%cH$h&y&4o?TSnE-;rJrto0H2daP+Kvz zU+(qZi#3+_>W5Yic;Xn((?ZVD!we z+idF(8WAQZfsC{i&u>cV?6#n>A0t5+HSu$c;zO)hM9lx(goXWtftoADqk}(YWL4#d zAnNs}V9mzdL@o)d_lgzsrAv-?)21Ifz>CI>w9LyuBu@3W(K@!-4oX&fc6>&65-3yJ z7!l$0y8hDYGAg1C5W1g8u3ALllfw=L--gx72};MjXfTRO@o1$>*jO(q^fA&ZXx~`s zmN#9#EmiQWKP0Wd|CD4!c=YAHPpkI1xf$B+iQW&(*lQfQY$BYm-|}74RF9^eTsd%y zLQ|rymGnB4w>s(`tZ=l8`jS0_(&-Qi0fmg&@+AIy%J%tzULd7-QX$EHtRZTGaGnQD!qD3k9US=)T0-7z0g*Ze7_Hvh>jDydF1qoFKGSUDQU z)o@6=L>!|=s^y)aK;4hF%(oSXwUR@MTc{FD0(;$c6-io~1EMXTjx=cGXHzaM+d6S~ z^Vk%JrZoM_-?JU^`qgQY9{7v9@;o+cS%PS$otD<>+zPHxG|HlAqM(je2Slta-J1mV zyiFxpWfkLQRt&*yT*bIKb9Q42eEZ&a``k2YU;xj??S~r0tJBi0t*++>tGu|C<-pVfECIhc0Lum%B9(LFjIO^YF&ZG-Tn>{f0 zn@&#EkLh)4lfN1m(*d3Rs>#S3lUQlDw2X~Z^*MO?KLZ;`4#dtSYt`2&l+v|r5+dhr zDa@);l8{k<{>-uI?B}-z#;f~-7E);!vf1FoL$+hJ#t^mXA&&~}B&+>xpNXmZnc;dZ zD~65qPEM^|CPlU&7tzcM4dhY7+6+ISvcwgKEWyh%)#&NsY*2jK9@OWF$&odxB=L zZ4d5P;+IK(ixD*5xRca`;)3RzjbwV83tEyVCevH7*d>`{A|2lkpR4GsG!|P;{aT}# zEy!c$HJUx`mJK?cdz4XY;Bi*W+srh^uU%*X23+cS(s-$-u-AX(ZW~lAV;U=YnWjWr zkr+ip2etb8p~ZI;!IjQkV}3UIfu1}lMZt1Nx@`k4Z&9P-7QtQoMn+T&XS7sObq) z1oivowtUcO4W`Z~n&wd<=p!krPR^`{|1neU?GjznvLZvHRdF)r%}sfM^t~8~j#R8>ma`P7mTg?WUb#a#E6&%haP+Mc zyK4NP)$(ed_qTM8D{mU597XNQxvwQ`ijH^UOukes_9gfe1sBKZW=_>fhucNGmL%)6 zw-6K32@NjGN{tELLD^a_dURgyM^4-1$}6 z?ab2qd#_j_p92?CgA(TaYdum>!iekPk!_i{^M!&paj^--aEi9R%{`^UReYRq9F?KQ zm93IL|8f{)nxv8*3txJvc4;+RCR1IN0d2@`MJpvuvvQBx+_OHmLX!v~)aA8F)`G}D zJR#KypDP=|N|G7Kba8lgx++DqQwnjvwzZonBr?O=3jO7MQaNx{mQGtb zMqg$A!)ZaG21$KyIs|t5_6>@39c#j~v!0umVJfJX>f{adcN-2{XDt)ev!DAGkf4U5 ztE_3b4yH!UKTKWn>*)6#5+5>@n6l2qA|wna+nRkCQ9uKgSKA&Ht3Ih#o=6Q)9MoO( zv`l|JoH5b$vYxG}=TJXdE*!z1C2h0-Cf0@4{4_&QS9P8BIaJY$ekK!L=c7G~-!3g^ z{5XkYZEiF>|E=^^v{~1p8UeqTkqF$NyHah4^KDeJ#SE|XO4h2u?nf1o#%17cae##K zkye4DZ*^ynp~okdN+5lgKTU*otV~JL(y*1SPVrn#xmsm9_F3u|+c#+Rw=zmPYMRT{ zrOD#K!3UB;ivg2P_0vw@=GOxuWD1Sy<*wimND02YaKyY{u9q?TkdEHqD~+a*Sfb1? zSI!t^6!ir!xyF|&oov2-sCkn#b{OTWp~;ieIVPK-%q!Gsjk8WRYq{p@Zc9U$Om-Si zs5^dniBDQ8BO_Dea-ed%lV)JLJNiMT*~!du@>8J8>9Uq=GJUQ$@9tf#Ya#F6rNrx> zKPH3dO0ltHRR@tL%#@jx2M+Vxibv_sddu3)Vi@yDNASnW^! zL-5EXh7gO^0R9YYaAMVd+(cF;uWQLE``eiTyc5?Ue}MceAwiv+rWLnFKwqe#CpBj0 zGAAnnHibi0!=)Fz(|{Yb2#}f>sKA&&Vp^U=w1Z-qw}TXDQgzQnLczP=>$)l?U`g}n z8@lzbU8nx_ik}@H1u8@E>BhpL8tG(l6>DW3C=2T@tS(SjIW6V$Nu3XgjrDT{z->_6 zlmTl*s`U;;zn##k4Wj0Mn4g8zAD3yik)GI9&r#wmZ9z`y-(-Bp`1k>330kJ+UOt=4 zY)(&34A!}a5)UbP+RTtR2sfS_G^p2*#@h~B&ZCX6#ePORQ8i)KwevP$4h^xot17_I z`g&+syG1%-51Lye2_%t%qhxj3mTvwr&KKL3!mRf#LS9xjs?4BMzeK&1Hcj8F`10VZ zlxzy4*?Y%0P<1BSwqTXxmL$(+1XaG}*TRclEFse`I5+$8f_Kt(oeyq2S2LoXbnFL1D^z zir*(rf!KQ`o-{B^7|t4+U=a>omwJx2eR5nok?MBxO{U%J%5*G)gH1oDrSs<|TJu&M zpXttUiqjBHn)LZ*%MM22FM+pJJ?F!+RkVqi;Ky=dw~u~zmY}_o(uyQng$(HeS)!t> z2^v;bWxM7b{K)wDw7nev)d-=jXL-vq0+#Bf8XQ55MRB=4tG+yik9Ks9Jaf1#(BE!` zK>i3gB)gd}%4`l@!pL8?c#7!%6&pZaA&D&JUrnv;ptWIy;X`Pd67( zxaT;+QW!FxAK|^{-fP0o6nrypR_0F0X)9?K-tz9P^b1?vORvjcoewq|zP1+hT(z7| z7&q({YG8i?J(t9d7yV4h1haI>qyhwoD4xBnmJEm+M7aHI+8R)k$V(;gz=n?v?|8V^ z?dX+pe-xo(@cIQR@8oJYx!=Nc{tXaY$4yOWSW;9djMwCcX}-I*-BV`fZ%n=xc|hPt zE-Tl4@6qrP06REs-qqy+=7}*fCVg!fpiuL4zxEW{nDAV&ss*2N9%jVka=I;fWh8+5r; zWfVN{rU;MPL80g#a^Ii$*SXM#h3YUu7|*>1@SHjxYE>Bq#*p&OIRvTUW)d6%AaCc}Jb#$>grL;e zEV6xlkxh?0LT!8haI2&7`R-5Va-ulLMLrc~+SUB5giY?D_3q$u!OJd2=j}u(ZJt2x*TwHo zGps9EH$?sU%|(q{w~Mo?{SESGHegwA~$+?xIT4L1$p=h1q@0 zx3M`mQb3FzIgQn>GZ>SXzPCYV)8>~57j%YaFN+G>Bw`P?4D7d~IfGEn4Vx4kbqxG& zcO=t+w3q=!W%#Bv)t9f zCQKpm;EdGb&9ZZR+tr*7XBd5^Q}^)zv3>FjCH|>PI1^*8ND-gY<~@0G<0EOT0qh~? z2?0&!*5ev~h*#}Y=tt^s9BB2yKK%p6^hRgp>Gy-fxjx+t1d414| z$3j02lo=r*T!A5Tdp#Jd0WAOJ0A;T_g2ZDzkWV-4-LmVvo+L>e-zhiG*5#Yaz~%&; za7}$riE*JPpUg6??hiE#{a__*aG7A^s#huY<+$uZ(MK=YRnI+A=Ot0aFYz92wd1@a zWncmWpK21X{ans&F?5Dt6yl`dXiNd1=UK;ng%&CW%E}BAdSEbQ#i18N62RQFj$jRX zA&3OqbSc6fS!#6{dx3+`}4~F&q614boLr(|$QE7xq=~{+KG9)Ga-c2R}9e ze`MRTDBGMGfbgu?s2fRxzf3Lx7p%GF|J2S8JKmkHW;KaV4dV>2dNv1KrC%boz}C42 z^L*9Mjf(z5=@=s4TPRih!X^fQ^#KQ5UDjk4h=xSo^E^;=UupROLP+tcuVG5cklW{{ z!@+bx^c9!yfJYD{=nWP!IHnslV$vL+9hks2o#npjd9mWYw%Cj0+izts@QhdVS8;fC z$m9gJqcMARlj!5Svv{G$cfK+oKMHtfMDu<=w|RD(qX)aXCN{laQm<0by+3-it<^ZJ zQ6P{q#1avvim{nKF6#ucLb$B$Dm(EvHvy&ufjrK;>3E3I@@nb2jzXf+c|jb23nY{P zeuWJ7A;2s{QI@f3NS%C^{P(8bs!5Tu9;DHbSUJ3&Fz?v91d@p7sqA3$1n$Fdq;=Zh zZDDHZk=uvsr3q;a(~32%hkU#C7B{k+)uJ3fM%aiK=z45~x?#GLA)#&t^kazP(7Mt3 zl?9^_FtguKG52L*gM$;$0TZx;$QGaB8){;xbaki<7ft#;0!5%pRm#W*i;tEId5Gy7 zb_L~-BJ9h79rR+phs&?AF*ubG^{BZ*5GJ`}%6u;AcPy#pFx_xxaa!Y>A}2-TmD{g% z6oPnJbXw%^))v>k!+T%mv+R2VxbR0O$rBU@x?$_b%oHb@F_jl=IwRu;Wp5I8I_L%Z z4AcxnvqNyJvDwBQo{4!>>kHm)zi(j6a14JPH%WZM(v*i*p$Tq;>;peF=>)5D&Fq~u zN5e*ddq%KbOu~D|?nN8tU$g*bggye+DOMcGP#N6m;LKPQ4uB=`euv{(`^`r!Ift)_ z1S7bqpVmvL1$0~uIquGj$j8(pLgQ-EZ74X5g8C1|{ia3xu#NL1C8xM-bd_kan{ z%bciYLhr8m>@&Snt+6@5p^&PCw}Sad);Qb9D6H@L+(gK5pc0BGW>aLk(IUvw&(FOe z0N4g=%ROc1f+DIt9w>l>ZH7W9?qqhR|zw0e-t+IF2}PDxsjdm1xtNa?oi=O`M#X!CY7 z9+daCnpt!`=(Mfz-Biy2P6Z@_37;>-(d}Z0ZfODSz!H?7#u^zqZgxYJ@R{1~hHH@H z52FKUPZ6*b6`5uD1PP?adBUePq_qLl%GY6I(E0}re|Y;LqPju?#Q-QNhtI+)M{vLZ z5{M?iCqtqP5Z-jiWo^DWTk{Y9NqaaT8{rB_Bi^Bu-^Auzjx#c*V+uTltW?=-2oAVH zqZKT)&;z|+CmTI`7!<0jR@sp?r$Ad!rgohiAC*>^L^KxH2H4l359iu8F*}lP-l6w+ zUZ0Oh0~17yNBWHMUsEaU$A60Nhn{?0gtD6iNPrCie(QiC%n;U@#1;1eGFP!Kg_ zbPE~>iI`0`3mtB7q_G@nB%?!#uyLWrigMMIvM#P1dWUnD{^u06cki%}QeFw@=lV|Z z*CDdK0!Qjn=_P-W`1*rKw#uZkpKNeWrZeX42%Vh*fR_8kV7LyRHQ{-vGVUuKee=Ow z3<#OY5sg7tfFpK@3-}@eZ~O=97X5V#`5rTjwc$iyUpWT5UZTE2SiwA{#S*woD1Ypm zhTkw|8wkS7PNq=kt2sB`F~#0ojo>orrwDnS5zbZ>B|-*-T)%M5U~P}&O(s|^4jv+| zaeDH-kvm}XXW<39=;ygpZy5ZTL7-LWL5u|ki6M@pA^N56r z`~5_p^?8sf9+Sq*a!ao6cswzeVw}*U2ihIhUA`vC9DkWDIY@Ak#uaSSO*DGAav8#8 zQng((*-hh5QMHv#?SF9k4aNcqpi0=o3DB)BKtw{RaULMFSkSk^CZ>G{z_kNkbDFe9 zVuK`lp(9a}WkO;GBvz%&pF^T%3*XZzan2L(1=BF|eKJ%IhTAlzTF-vf^a(59G2ISL zG={j;aEj`Uh)`K*Q*f}b;i4+$!z9z(G~V<{oI4b2$x4`QQ8v_fWj{WENhfd;AK#}+ zNXJfkic(_iBD@~l$E3oN+GFw2b3pgF`qgxJegWyFo}{QZ#3))W6|X$Eysg$31K*f4 z4ik-=jd5^dPXNY}%DB1f~3qOJ)s!FLO^R~k>o z0>NEI3})wOcawWIX+4GAF%0-sZRZ2T=Bxz*V88BivJ*O&UbbNk_T%DkosZ=v5%ng$ zhRXweuT|{S*sN5!Ct62NrMhI6PtOqNf77(C9gxz$16gc#hmnRStXMroKUF0NA1?I9 z2n;!F8GSHGlB`O_wiuuu!)mV8#<9k5)ZI4HiH30os6%EyS63?}tMYd=>GhsOetMBg z3TDMdYDfdxp>96oG3r-FGpOW#b+)#qu_LNR zA(_A@&A_|qHYO{Y{p*_qwu9+?^1`#{b;#hz+a5d=JHVOImCcOyvnivmx-KZuC|!7> zy{KwsMF86(^x^ym*dAs$8Sf-21R1dPpQLLKC55mIA$|~bFsj`_-=naDAm7T93wKn- zT=bkoo$fnMg%W(A!A#@8g3tA>BiI~dZAl_yi)BvW*<_}0XtG|cO827h&n;h%v+JAg zNA;v9=ht-;LI|x3eiA7fJc=!7&zK~HlS?g6C;(~5w5E0xubrMT|IwtdVUV^QFN4ei zDeh_fQ=#2yakrUN3`r`!v={^&paX7F{XF;b5H&v}GPS+Gr?AW@DXwtijag_5_pWpL zwn=4Z4bKD%mSM9<9&ZSHmQ?tc5I0A%Mw^8fSSwBV!y?Fn;Wfrr5vejQ0kG{Mo5KE5 z7fR5Me~8{vkoM7;0;|PFewE%#w&-D61gew~e_X^ZavsfB{=slC zJ+Q4HTw?Oss2I-_r-MM&-WwBuXqfomTXS0)+q#%f$a{KD_<_f(TQzlkcynV^1kBou zAk8D(bN|WqEcI`4&PX+(0bzbN%jw|$DEt+pq+tzxrQ}AO$&QsO3K`jy(O;UZ!+0Z0 zV^C>6+L71iMz?@_hfrtvnGI(f!VUg>q$E5(k@tf6u#QWFUWyo3hWf})LWXM?`I29( zinUH+Y~p0*{Adfp%`Rm$Uil_MuUrn3mqs`ZP zGc}awuD*SKwn3u7p})bci&mi$m$-jb_r}t{mRHHZkVBqD&-`SlZ2aBBBhL*sG^x*a ziaYAg)k=0`SHVdMR5^qLMF>3ae54r5NVv16ez!~gobce|dsoxOvX}u$$-P?G?v$4HjmYKV#y`9(F5ZeT zxtiw!DNO`d=mfCMh)~QMX{&J)9{mqH?(JI~D1Id9%eqG^>3jfd*JF{k|(?r@CA(eX@(GI@P8sJDPb(U%Ilb;oqwGFzv+dv zzw1IG>D>Nrq`m}<r!f_FRQ6EE!KR0Gy)VP9+PCH}(>{7r}r_-%BEhN8;k z2>RytuQHj!ZnX{bFmR4#Ue7&y^rOJiB&`C>3_khhANVwH;GbeXmNA5g#ScurH8ATA zdgipB8!r2`P{C|CnPDmVe*bfPzVc_ui)lF)8O4Y1->-Wf#40GN{2sh`VAgDbL^QtB zdLXjwQiHA8pwhH!!(Mw z)O2#IO^%<-lprj0Sq?$TOgb9)%)0cw;aF)fLvoyU3}*XF-(4Nip`f5Fm4=eouC^`4 zZEbFb5a7~EOB2m|vZ&-ov^z5~FvxJ)elxYK9#B?Sn(iI=H9B=4>E%v{M-%T7i2zJbc0AM~{J+l`pzi~Y! z!P|Vpb`O~BXGu3>Tx;pG!IyHW;Kb6(4sBkSeIN`COnt8;sAWE$ZfUyTgT|8lRY^$i zwDZZY)elM$4%Ia9#Vc9d@cq?vKHX?Nv-2{u4rVR;26Fp7T+OGFiMsXS zNI{`|^T4?MnT2)6Kp}x3Q;feuU1a$JS(e z<({(gK{D^MWf-EvW(vi&oGnS=znm1Cvq*RiZk-T%YP*_fzgY!Al4N>c5%Icw8HIs= z(_ozj`GHtsi&TrcY#2Q_ZR1-p)0b1GLQ{@V~OK?mDdG68K``E zegfjeJWM3xP_vWjLGm}63$~4G=13PXBe^D$sPPnr=k34%&#JqQ+dJY?@}5_qgo(nj zjNwEnG6p4qtK`HX7}9r^a0~-fUe~z74&>ojuGM#DtOrz(cQ8Phx;G5teK3|F#9>3v z`=maI`8Tdnrq`-r*R=6m4z0LpH^Z7*P;m8Iht(`2&;4;NQ!LGQA)Y8WJHPKjhhIL& z9Ctq69jVlsD;z9Tf5vgr-HoAHF2k;dEvdx;nuqJ3Tk&!*++rt^?qgN;e6cDWgH}zx zTDkVDC}L2l+v#%iT-we^TBD!&a?Ag?lC8LbB(6n;MenBaBnlwa#=8}LyxpclXez1c z>F?}HW=l0Q3l1+vSh(B?=7CmC4OJ1x_4E2FMT(@gr@q&V7Sqkn7Nw|~1ZEq|R9F=? zFvwzoN1Dpf8P@vv-=L;$Qx3+HJkT8S%Q*C|>gB4}QTYW7$s7^5SvApU6@DdyMY#X1 zzBG}Y){@o+D)J7QSNL!;bI2EleLJnx&&%xj@{asQ0Pu;YO$hUu0VY77%C?IrpSz(V zr$fu4fs)<9Cmi^)<^*N{$N2{;sW^5y5Am0JhSui((r^k0ZH{X^ z$73_nT*ll+WP8o~On~Y;Z+@s$Y*bk|8Y7Cw*ZcvaHW3z zE?)oxz9l~e;rG4T@aqc_8C^yu_c1h$b;f4Nm;b<~(rBl5zg%M`R~uk>jym5g%mb-` zK@u-AUyJ!@0i&A)tMi2O{2p&Doa5Ci^;d@F&y-Ra)sfgDtucBeJVmPRL`EH=ojx|4 zlSPo_?cr@4CgY};dcL`xb9dQT6k&RG-c4B7zo6smYMWh>zdY|(gjN1v1ixY}uFz>5zYsy9;FB-emHpZz`~cs4{%xR zj4spZ9N#PlIx+J^!|6l`4_8MC{1|=Hz>CMlIJfxW&DTwq1G+oRXjW0)GS?U>-HRqo zPblC)j!5tgvj?{b48s}ja1ZKH(O9ptuuU%bIFVju8;tLEywmQ1GX_O>! z%RycS!}Pw2IZ}h)Nh1%3Z>XWClb{*WRe~ciEx%+UE1-iTk}(eSJe`|X6rV=84{$-- zfj%-nB}H!QifoKX(d#vUPu*9(Q3zHqNnZI9fwyHI&~LTg6?m}~FO+eQMk%4I+LY+{ zwF?!Q(3H(A8artCb5DrE=!?Mp{V=pQpj`0gS$atG!6&p+W&C&Ras4S_t@ut+n0!?G zU2e49D=qKNDZ5j=ZdTlv`a0j3d44Oyc{iOOc}1bYfj?y!*m}~iG#ZL}ZXA@WL@1`) zn>P^LJIAC`o`XifXoKHeHO*Zioux3`^fkx#T8nr-1`AvV*?}~D?ezCYBVSmoPIR^A zm+%vVL%4zAuft|(dgeD0r?LFfY;#p}k|>IjwkG_Kuss0(P1a(19y32+CNbOTP086> z)*fG#$GNdLpmQ?{bex;!)TH_J^J~2MGYcGu-O-i(w@puyz+WL<oz0t&qIjg?l)ri?V-NC3pb&R~hXb5%0S3-*{BaLbuyJiW8(n34M zdNzn{ly%Bbx_T})=b`ygL{%)xoRDtGvMDe{+1l_OrmNAFSS9fOw6(IoPo$wKOy}me zb76$sQME48xlcA1`Gi&vSWKd-56tq}EtwHFa?X#|@%d_goq^*)0m&M`ozzJqzD`!G86Y@F{b znTW$`As7{l@>qgPo}j_ds2^c8-PsWdKwwnBUwYo0BLaQJm%ubm5rYR%yu=&F85|O$ zncH|>n0L@)F&tU(h_&a5)#DNWanD)9EQ#wgf3EKEC)>J|GXe^5?kZ31E`;QOBvw<> zcm(xenlERy*+B{8r}$R~rf-s^lQ7b|_iKdEWS9I~TB=x*N+tP|Mg`H|29y~{Q*H@= zjFQ3Wnre>SxIcn!3k?8o3<9ir8uz0H)rX+)P;Hn;&|tY+U`M>*!e9QF9IO+Aa^DG< z6TYN;``C$kX9RB?DBhI#IM0SA`^fsjFRh(uHtk~P(QUBg!z(adeoYZITJ8{rBpZfpP@ zaaQ{((w90)8=>6@P(;3~PMAA=x=KXsV5R5=(+J~o5MhOpQ7Eca^@mC2gLgLF7fpJ; znL|n3IUCIb1ahd$0$7Yg9aYqU5!t30HSPVj?jaaWlUM_^ zrt#sMJXR0J*3=sEZzR2MPHpi8pv31NA)ozU-3BM}L2-2vfD zqz4gM=Io@2tB3hbagL&9WaDIA-yuQ03TA>c(0xitPZt_#NTd(k2wvfdT6WA9KF8B~UllmN~HX|=PQA>rj z#Ml2&&4u+v;WPzuUv`(oN8`$o%+VOHJG?jc3{2SA-TN4kf@WkdGrr8{7|)VmZKD79BrL`UDZR*_WZUFRf=r6u8E%_{B()U(on3S=69e+zUXp zcvsX>a=ZlA5s7IMDK->qqfcAft4id{8^-;!O~A1XU|R;akq-XY_Rn{);E<}=pVjnK zLSa{Hi;0g7L_|jfb3D={`iRS6*r{KHL$J$qn_k{Bt&6piTT#b^aw-R|uX_g63F8-M zLU{cpBG@woX2TTf9JioEK>Z-OlZQKn5r#LP8Ea;&TA#SnFkgb;2uI z-M}VX$8h5UN(~Hzpv;#w2#LXPMy$b2ILAs^#GaAf){|9%OkRNVzU{y->lb~Z;b_TQ zFh9KxF8~u%8|i3rgbVgdU^m_c+Lm?g(!py1;BkV{MVo+*O|K~e-WSO?q&K{ie+Pb! zEmjuLBvwp^U1>n~XeG1gbeM1&Zu3MaQDH${ZJKKGO>CJIjblMF;um-n&H^0M6wu8L zVG&#WF#BTqXDd}g4z`1wuHPZk{4C6#x*$bJV3U3Y+Q#YOkGO|5U5+3>q&#UMpG1@P zR0&CrbRNiHV-a$Xqe;O$HLvfjfXTfM7tLUjLQkFG!J?lasQa;rm4dbHMQ|8#(yu1& z;%KHU2QA~+VE&pM+GzBJr8kaZc}y7r5h4-5j4~?;cW@ZCprF*VP05D@q02D-8a=F1 zwO(T+X;5l&rd^xNbG287wxJ%mLK^HBpXrinc=|OW#HddoOQMVGk*#UmJQ_bge!Ed) z&@TJZ9HONr60;tGL*=%xIfObX^Z{ogfwr=v6_2&@dh~P_q-4Wd<4y7X+Yc&)C-dmU zUkl?wUw(Y7>j$KggO#TsU}>~dVPhEVf#DRS9%&8cH|*IZ0%;-gP{v^*JUiEJGR*DI z7de%dBEk<8arlXzxJ9=IbOe7#jcx(TKrUp{7@5>dG+5iEqnB{3iMO#tC42}$Mq+{$ zM)g!ece#iylFza&k@uR(Y>CCuAKr282*0JUg}kJ&ol)`{Mka_G63gs{6L^O<-z-TK zhiHer_D;~@Q^kG{_azaVX#`0D^YJhGj(%ys<9h!}XSc>zj&jSLgA2}oxe*u;Ao)Oj zEfiD4Qe7{E=I%qhdY}FZRE*m^SCQ|=&qVur;4h=_A_#OM49lb2$2s&rpincEm`KYl z<(Wp|6({t)Ma~BUbeJi0gLRWQ0fSRV3G8JFZ9ySEQ_dSiXV_x|VVLy--+}LcX}AHq zb&D{vipsmKkQpQq7((}-^V4JHjn^+sZ0GXpZmpA*hfnOIPKAxCNOcACL+mK>>V&*v z+4M*P>0ww~-n*X`e~=#-K8~OT{qjLeZ-RbpJZmDBcls-GP#+qId>cgoL54~|9G!C5 zQ&5VA#6&>6$C2Ik(peg@IsLvKc>pO&n1ZD0km`fgwS4Z3aLP z6Qn;kkM0YF1Rw-Qaj{QOy@hPO-Q29<@cH4RqbmYyIg}JBt8E8!Twm^#B4Gy%HElLM zaFSU~2kTtrL{n9@;uy+T=+F!A94^U z3@;8GN#}lTxl$ATa+?Mtw6TIdK&&_O_+PlwlQ7UGk=r`p5k2B`?^C6*XR`oL>Czq2 zltIM!B3nEAKj`0YSBVo`;PHEIfbL*}$`^w*k}sTR1l6t!N0gmMoX<<=w|&j@#Qx1r z?*KSPiUgS>ReVa|mh71;wFfyM_7MN4>RUvZxTarIWdC1h?-kWlyS;BK1PLM}fHY|d z9qFhbz4wmNi*!W+L3(eYcj;9?5D-MFgd)9zB1rGOgLJ9-fAYS2@9{hMj=nLX2N_|l zm6etC%z4l2zB4|Ik%$6Bf^LZG>nZpJ2de(8wN;Gdh>47RaQ&E2+@JRQ?#`f8@Zo!_ z;I6;ZeA0iS=6zrnUQ|Gy+A)elemF+8sAPRY%4t3uO{1kb=ZGe3G{^8 zth-X_00y!;*#!z2F&$m=wSjBjH&#_Gt+y(vJgr=ATO*3n`9IC`$3GdeU?{1T70YYY z<;M)paINDU;|&eKeo+r08D<~TL9OL%T$34}KUGu9lAq6c@PNjckNjdF9e zBxN8|@CjGbnK^?C(7n#Y!xU0S+8D|0MgVxrE&k{1eIsCWhRci_c2f|MpgVoX$jsb_ zkp>^V961Xu;1EhH)O9Tq(7R;KJfwABxLL&o_`4AWRvmN_yH4K|bx|fDgu@08!mMVDp)QKxDkl~>yH_uh4E)0BSjhG7ps4A-;+ z{|fmNqfjl9l(q;Po|Y-7`|)K@eX(QTovs~x;8V>$mHyL8D7@cB=Bvm*7u^@6e+>ct z%tVDn^jWFN*C}TEFd`CuJ^;-!2m{crdiitnpQ$jYO}*>xeT-6SJA>A2y4Ye*6ZdO! z-uNMD0w6HKi;edMPm8s*@?`L?Z90}JU3U#U0OlO6U<`0I*K91Lej_6Bo%dn5{$A>U zI?!tSyK8tYx_SWp$pT26L?M^09-!^LIEkavIt7&K)T7eZdL!kuZ(#v*Ll_3I^W zAcZQpOvAL%_TdbEXgQkARiwUcQ>g;q=HK7Ve|rHFW>({HXYvlQA?*o6#kwGQ*#Tv) zdimL({dU-!VMP{w+AcVXcGzz@THpgy)RpSz-Q1Zh8@C>klIJZ9r{0K!YSF0TiHAjL znsvE&i=L0USbzD&rvCl0gyU(^`&rAdd7ldhKR`7@KyfSiDVxbdliRN|!fumc%}vcn z_$$5wgN|e%j`h*`U74-i-iAW3`zh8P#HE%}s|iK+B|O0Ox(Z;$x9PxlNyZ4J8B5>( z+&mWsC;0;ec+>ns{VHoYRs?CH6WCDtDh?kh6LwtVVtZ)6vD>T_qzIgYC{!&i9Cucm7?oiZbvm9^jVeknlrpDOiR zIKqqqOu!S(n_<2HD6QcuhEY#d+gNZ9`vEo9+NFNK4L>?e%2fB0ft4jGR^;m8Uc;~C zU(J8#dU!g17rl=mhSH-gL?OyXbH|EcKA}p^Q4(h|^=4FCm+}CD4;==r2`d%9Z5%_) zV=1n$$6DL3j_(0bcQdx1sMa!I8I>7l~e>fwbk{);Fc^z86ZGZ}Qc1Z7k!V&07wGY6S$l zUK|2PuMs#9n7~c#0sHNB6Q2|9G*PcFM51$!g3;$TQJLO{z1Bn)OP*lLvkK+{al)7L zY@yU|dR*ZM?I`Uo4vTdlAM*E;jg^7v$jp}i#6nwOxdcZlM_Ot{%VLa$5$|RKZC=<3 zEbtzC#roqA3Uk#=K97ijEa7qHQ_Qdp3?^(Qks{UP=1MBY2WLxbI@04X@v9$UH|?J{ zG3br_pxZVIxPNz2d^*&YtOwUK&x)YN98lVJpq6$%ZH~f78XWTTUFP4ctwX zf`m>O3*?!?34y?jsk@XbqS5r$`NAcxYl_jE@EMQwtW%$mxF_QI_yWJcg=(9RMK`pl$Tx!nG3wyT7DIak_tZ_i z?r#KXS72Tf^&@hQUve8tqZ0Z3t*QnAmoTyEB58ZxoH?ZKQH&SNLQ=spY?o`pFxcF zeW)|!dr6vbkBcRH!=tT}iIkK%VSz9#Tm9FAj#uBN9=QgsgmnHCYyCxux5qrAq$iQO z57gdn$egN$uLE7(3MZso;wxY=`4<dk2R_eB~r`1&Eh3esgZY7 zE}}S@&6&qTqVL(WXu&0!#hKUn?WR*lgZYS<9m^iL4q*>VH&P_Wvt+xs8P%_6I>po> z((=QV163p$mu`1c7Uz_bs6B4&YY#-7uqT&Kb~^u&phY}W*^A>o&Ji*4)^j`ij+y3I zha-g(gkkLPJ5Y8m2@d;eVZGwi9mh_-y7%(dj9edCEVqF!%ug?8Y&xVqTc!8DoK^~o zXbdu&d|_;< zk|RUpQqcIIq6f^B@8TZMOv8%-)jrn6#u(Kkg=PL^j)$^5Z8X@bNQNC zJhu3uV{;xMtv#aTm_1M}mzz=mMUH}FA4~a_I-wYOn9NigUd{g1K5K_* zZ;mc&dm>QUT8w339ps!%UstyiNLE5^-mi=$%6L={BG~jBS59A5(4oV5*oeK=yom%@ zIS54X4lU_85Hn9JN*B47inCdK?87>?7R+sC>uXdk^>ESQPS~*4Igv?oj~s6fT1dwe zE`diqR068m&W;rh(HRt0Ik-0PK6fA^+AXbH50LQL7`>sp$;@;CO=0%wX6%B51QQVN z_pyIYO@b14yNA6ctS0D&)AhB{$UY10jd>VF7e-1Vc<>ypkX9X}A?MEP+pd>$r;8?% z-hqg(SSn8s2?QM^=jV}cHV1m%ghIiGRi1^P;>b&7;kaRVFiH4WjxuYI#W5B>2OstF zON&FoRR@)OP}<#mReiio$4T=}nHSiRigK)GowCa(H>b%?_M{`V5*2Dg} zQA>|hC1XRN)F70`H?=e8U^bX_F(!CI!f2Zv8S0?HWhL1XXZkI6Nl|nK%!TV`xqI4w z-&3a4dAP@tAL@Rn_Lr9y%ceFEQ$);Gc2pcH?_oE~WCc zX49supcY304x}bDSvrgg?LZJHHyQPWBN$y-r>;e|9O2s$5$f``sU^Mlyhwxad~+QE zH&sCV=I?{L=Ly5eR@jQ(o@{Uz^|&|urhsIh?b$Fm$R@RPYEhaml|NQ7c`Q4Zwr>(* zXRWm;Hz?yUX_e9oEG4f5Ncr~f_TO(s=;@*-`Rd*NOun2O4nTKt6+6&6uV=VKYx`WGJFQ@53zzn7Kg+*oRv z2u7O)N2=9VCpii||4vOQ?%OEZD8Z^<6K(=$Qoy44!NtLiW^vqo%hG_Ads7+3Zx(>c z*Pxvjn5p;Hjel|35=34v@u9qwC{lJ7F2vJ0tK>}sGw``Qt?6W)c>sq3{pHCmn6sXI zd)>Z$_z7%&U1F+TP~kY4vv?Zsm2LKf#o-WH4{e#kOpA!bFw9il>&K(hxd7i>j^oe) zw&;1Yod5!}n{fsgL7`7*Z5xVNZuC>~`og*PJoLawkM~P?7ZF{r4<0*1iKQc@)8H~K z`IfAkn0)L|+Oof@Lc1`&XP%9qZd`|=k>u7c*2LJT;w#IQV@<4pewDW)ofp>w@s!Mq zHVJq|+mgn+9|pfgx0pi4?;eP$3`4o@WQ%y*?_6qu^s5q%;|M5bGq>lPQXazukTWn9 zVhkfA`pAL}L8HC~{9H*pw|ZX6mcBwKl%sji89F1#6U^?7x2q;>t8A%os7yg_HtdK& zQWsK-QlmELn8i5=jsodKB(|WJd1A053HnQ`Z zz^tuBQe|DpYG8PRTVG(&L5}0pW?@w{DG8%!bOwrChJL-#cS_s!pOd#_EU-Z_acA<5Kl z{(jK)kjrrDyXf0cl`zp|JP|?pi&9j5#nM~XnF+@a_!OFcwi%W8U#Zt;pAURtsTrBp zLJ;XyzeKi@mFIIzJyYPfS9&{&S5hFgd0RMtl{u6Os(edewfN@;gF6JRgWWW4Mg|;j z4e$^yGnVD``%8)`re@!zcgp7u7<;2B!{^+Kf->kxIL4q$j!@IJTTu>A3I;B=((&3> zlrA)Att>h%u#jV@!@6)Mi_TBd)UnEY3=;FiuOYdGu}D5>0u}0(GGzkCHLgt{TH z9oU2xcekl*9>;1E|K=fcC>1jbTF@XQvp@bdQuou9Ln>{N28*iHY~yAHL>@v@6zcvE ziqmZb_>=KFueb2q&Lxy8Xo>V-`povmP~_*A*}2#gfJ=y5+lh->^9hCl>5vVb*|UFV}J6u)KUL3fL546;mdN70A^K3GO}QAtxWZV{fk_ho(&R?xkaX zTcif(vd`2*mcQZAe}H#Xeb2sn=y>r#0?B7YVs!S*M4)UyZ&?oLKg1fAv;LSZnM+Aa z`AOv?WL1iCVQDIkOvBHk@Jtr(k?4g5-%fA`l5k(=n?Q#Z)q&>4BD*Pn*n!}tU`vyJ~esgir0*cn7+ znSO*;8X}c4Y}9is)@{xvl4vUANME>AFJ-VpsgO(u%MvLwk_k6uY+$uYcALEvTo-E< zZ2An00M-{$d#yz5Btp4#C(Bw^8L&R2_-@cc$!e*FA88^TD`u@LPInoGCC9=?qnB=` znXBXpq)MR6;S6z4yO`?V;tqG}1vpeHC0-&PFHb$686u-KT`%gVG>>SS`3vR6vv=rQ zG7I`6D}E7l_=2QNm*=(}y-{%WNUPq;pPmVN+OiCTy5g#w2=J*+9pU89y0DLebUh`- z;k9U`AwIH#*ILESOl-RJyB|nZdsU+!Gc7$vPu^;EX%o|(aX=3pt$b6xD$16hR82Z# zWENxQf&7MW-M9BuQP5nZP8PFQ0sT#728a4*^gVR06k{EdSiSul>0P?O{?)8xjo-=MP# zGX5d1_bX812Vzp;g)aJ5z<@%YJL+L#CpOz>y+sq^>9x39S}EB{?~e4S-v`E2^YV*c z#YiUbIY+85O`EiaC)mh$iFP?}!v1TK1V3i+Gr`wR~-JB{8y?=Pkw0 z4Te$)X*5{UPBW4(r4{>`DL#-D-W-)R%&2UK2D{*7SXX+-OtxX@{loYbcE;B`TlD#TtnrcZswEIHb#V|~kDEgBB)A1W5CoH+MMk+lde}!E~#u&?h z?0kgyl}u^Tn51$3gNo{VbP|;JRp!_-TwZ6j-jE2t$YH|Qj=pMFkEMjX0rmycU* zkJg5lnlJznKg{SL_M9j{_7z$054vY>7n}XKN=RzdU2>%6?rGbxvbYIps@cvP{*H|t zj8$ci6LJD9N5E!-blbI~+sLlVKf$Je*5`8Q|1Qb}z*b!v4cQBqqBDVjKHSIBD<6}3 zd7+1rk@#g1kNU?GUwIiZ`3S#btkM~%csKR&X?^9x;$emR+rnO*b^n_+oZ|&hTkzEM zk&>DKgR+V24Q0dkebM=51tB)~KNjeW|6SE~G3Ng7cmVMP@+PXo${_NYGvpACRlzQO z6!)?$;=j|7I6KG}*+?tluPo00C2;=%aeJ^ayf${$!u|j47mPs4tDu(iS?T4HdKLN_ z`G}gDoa5r!pO}#XhTWb>fcJ=nTMWj|vlQ4hRIni9G!31_xfI7z{zsx?NNf){X0~yy zNO!6N$e{mVzyEIF8fqEYas6lgh0S!{HIX#{i$L{_{y+}Qtn%d?fcC;kY_MEvwL(X=J zr!}76H83oT;=M;G0TuHD>j9qG_v>sd3Oo5E$7`NHLI)+~#c*lztnzSoGl0@A0ub~! z6K0``<7LLRwZiG|_})nC03kHL0i;yaY0h$$U>kXarHa|rJI@tXi9X%=*taoF#2}|o zW&JH5ypW&kud?yR_un6YO8+)BmG#L7m3Z*9V((ifJ6jF@`3gXyYQG#SWh1@+v^?~E zo@`{FwQ<~eKX~sB7 zs+dXE=-YlP?ZN9I*Op4-`hPFNoEr=`!ULeQn4pKM=jwusC=y7?7{Kv4F|@0yims7k za8`z(#Rru$mq1k!fF#5va~e*Gx&fq|&#Vnn44mm4Mre(HhEHIyp{fJ8>?}qW7Ugb2 zngY#bIX;_V#x!BKGUuHMO}Q9qn*^8j?^!tgbd2G6r(j@_2!;aFjtR`F`O=|;m;nWb zjsGvzdJdGhe51NIe9jvh%F|YVxrQ4(ob)T5!?Wh|4S7_vgtRd+Awakci@Ntn$$BuG z^IJN90!FIBiLU>knqh6l3;FS0ElY?$4e+n)nBh*pxb*Cf&w2(gr(UYfGO- z`nWPF6W7WL=NTj^4cB}8@g?r%LS%o(e6_@|mR*(xx3L1+1NVU)U%}hE@y@-6J*BuC zvrRr;lZ8iO#LwvG6bQxwF?j-HY{-Bsj7`BF@Pt)zlIXIgt(P`arNyR{ZCW@7ZA<%r zR=W+sEg1#sE-Mh>o8l1)h)slS9CxX|UV@4HA`T@O>u*Ym>OHsrdZL4wxEc9fpJ>&* z)GpSm$Rm0Pr5&%35c%77^adDn`x|~Wv6r}D1usY*|5yZSdGhO~bGc+LuXk1wjCPA?F1pvjS+}Hltv%lCLjW*Li2Tl;2 z$lF2|b&Y^NcP*zMxVh1t=)B&gWLg(R( zIQpK);ys~@)c31Dq+yTVV#T-hovtLP-x~_dlz3V09`Ab|v6{RGMp4Oovkm=c2g}hE zq97S&xWVmp6>9N5!!?*VM(0aTAVf)6)Dkhe^wTbFY(X#JOjynK+T%Th+?4LYJ!rp& zOXLOQv2Slels7@5!h!Sfq8<5moZ$cXBpTtqN=6fz?s5MZsZWKDvkkuj(@7xevx>IY}35S5SAUupbF0MD~2a0^9>5FQ&%;{gE7(^NvUz%2VjCw0bX%V%%~3> z1k74Ah!K?0;fADdLbr(n9y{8B6b!z|dp5{`jox#NRBe=bxm)`Vz%8YcMzAuSnfjX1 z&XcR7K|>A_kKzp1AUv7Vs0ZK6kK=@AI^rG~l_jd`Zv+*SIWm6L4 zJd)h?cwGxt!W}ILg1*C9QU~z}&F`o!Wm+DCnL?7`hjd0(54G4F#t|x$W3}F9*LUQm zKRB>qm&5Na;Ph;<3fCapQ$vZsa|!OLcPI^lLGZKkM6-jyD59Tahw<5!NQbzBqLQ~3 zeFQc?R8w>&cvtR2J?Zs!;D?gm-CzIw5UqdS7V%_`E!1s`V70sHiElMw9~0dQxpLvZ zXFplE27$d|_fPKb!aU-JV4wWpxKP4Ka#mb&PVS{4gh=K*u$z2)GY(8J4DtaFt#rilqmq0liM9$< z;SY>@21Smd>cT7P7sX_oOCebZ8z`A5nX$k6L+&xvEf5fXn-9xAoW1AkFU%1I7)RW& zT&e8ms?vU6%GkLJtF()Xq^f?cjoO@xxy0n?!7LD`l_NsTh{8Po`j<0aOL zVvmO`#H3{$gWvx7YjQ0fz_G+^g~P7r{6v@6#vKRGS%lK-5_uynL^-Z?tAxa*c4d==b^vXYrekMfskNdFJ$zv;20WyFApop7%kMp z)<5WbslC;y)GcgT4Aw@u${$oE6EoXD!*wzT(yJkYa`tuS@3w1^3${=oT`Hi{y(KP?2!!3CkrR+8Rza<;Vf$PL_Z zi{MfK;}jy}a^`>T%q-vDBwwOaR@ZNuHwk)ie#oW&MJoA|5vQHu!|9(qho_)ykr0#h z_j=0Wg1<+I_q#i*n5vCvsk2-n`0R_R@v^~)w!&l_mHl8`8UoG))r23)e_pP%`dgo6`ZMg;yC4&t%T31^KBbL7g0hN<_vu|XhJUCP}ccNvaLwFeEB^g=rcUCa}Llt~B1>UjJsm?B&JuYMx9pYXTc)ys=&Lkt$h{ zQ5(-wN-B`(ay8vI$d{RuQB>`qq@0t^GcE>LLpMUb(sqqi{Fv$_Srj}pM{Dx4rS))B z(oS#WoK!SZ+WTzzH&%UB4HScFBnMh|WO`?o6pR^I*3=KSRyD4%=j+*z{<%fhm-DVT z!(s|n0b|4xWht4?_+)1rbPGf6zJsgD+2Sl0=8sYA5o zP@hoyoxeItbYVYUC4FGXPWVF7PWj0%Xe;cpx%!m*g7fD-j+eV8^^dRl+4AmZX1p3q zJ})G=yWqR*W!JE-#95hk$`{yPIC&)QytgR+irD`6#a`_#=0uZ8L6_KxUJzN`Zwcbs zap@LBRRm-olj_M$x1QA1_d2-lIqs?9aQ+(~>Wah)IM%=+QRJXBGmj7@9|_M6&6J5% zB#B95okXOn1|wM$DNTkdv4$eH?sBrgt2XN5#7{*>RI@?I*vlu?1i57zYOLa?k27O^ zSd1C6UJz};JT=@%YlIGIkCzN~?bx4T-*OqfkiVB7LQEm%#JR!%>z1VJlaZ=-$bP7Y z6GWK5cyGPp!%m!9zl_)zX!aAHiF$Q&gE`QK8w`9Mgqj*DskyN zdvyFTekrmNrL&fT6qA<+rJ5qWx{tJCYMvW5{{+8b#PU7(RX!~BFs(u|<39W5YAbU- z0}Pl`m%u+(k9+^zW8OdmHll?>BdX0)sV5%nO<2S}!k2S?6FpW{vK2cPpYg6eUm=Uq zXU#y)-GLx{O;z}4P-L?J$(b7RYEksm`m29AFkCHe!reIn}6rD=OdjeHqmnJE>1EX_M7{Fc0DY+V2GQ0!EA*lFs$5?O<(_z z$pp5$sJJs*Qrr9^{WljMm(RuVfuDT174dfXZDnk>GD%+NB(y={?PyLjd@PKo^*K1Nsqw7JmL~1)JAuk$>NT!Zy9m) zJWIZ*(}Nm@$Az&IJW3+^ck=;n;R^z<-NH?n4ZnHmb6X{za%CHkpRKNg^P{{s1(?2^#Uf{ll@+Ia7u0ITV6jPJ87Ri@Qq`+im|w=kHPE%p(a7k zr6%2tId%;~2;5Tp6)4= z?K|_&-IAU8T$P<>86)PH2wUQtL`6K1{g2-=1%IlEom)Yn{Fec7vG9Wpn$EFrqJmVwV#+l0mkhN_7Pt+$8u* zt4{_sm6&D^Lo{G2R}ibD9_*4ldUJPmef_e_@e7mfvkLQIP9yY>1?pezV=v78uCAIl zsCF&J-!duNf(XMQ18h$IGwv=V)$~Xv+Lrp!BnSAoNFn=iK7c=S1*uunpx(9-nD?$4 zEz)cUNaAX@`%GCKK%ZGz%aPI{6QISloo#Td1*qQ?X!fqIfZ@3Ygy!gve$*DdOV#*j zr$oENTAtP;p9Z9W0W6X45@e}0UV&uD9W1z{|6}mA7q!*Zkx%9sy0wb&41e|l<-aRf z$0smmFHryNPxZ9R(FIZ{-Jx`fjrPtjhmE(_xEs$f1iuOY-R3j+v;)@)E{AENjykv? zvzT-v5E@q2g11{;=d@lgP`2MSj~muu z5hK_IQrq~NF)Y)zucuqX=7)?@`B>Uh;Mkr32W|nP#0qIZ8e1?udqsBM_f2fqZ>hZP z7#}zJ4U8q)i>$$Ad#%k8xbOEgW!y17>pxdV>dmnfk`FcH!72O%q>V2iFSW$5wro6< z{7v)qsncuvU53X8^lf<_M{7C_u6E?v;37EkdQzxX^QdH8FRR|A6twxi;$dA+Pcbf~ zy-(DDSbLr!@H!jArR*KtSt2uB`=0h{TcETS7?;l;BVCK=a-&s&(J*5dv@!N1SX8g| z!I?3>%VkjA)4v1SvhPbk<*5aXqReX&whI^fa!EupphL$1gWUi!?nK2;4vo)2)4Btk zgl-uEz@T5Da6i`n{ zNMP&mw*`01MrYFNlA9mR!`Yz>GEmsx;SazcKN>>;a0t?e+nW`@Sjp zgNXoBOf=={YE3yz?5t(|c!)XIve5{Wl8G)I~eK?nN_{F;mRQuHNAJ^d=$HV6b9+1%&EzEBwypLw*5c)8hOelNfM z)nhuUi%5YJz|!2o{1WbiFCIZ3h_kI}29xUYQsUYX?C6-+7{M9VJh-40@o?d}84!NyVeGD|DE?QZ zzv<5fmGnT!FI~O{XU+zwLWkA3d#?ewM{>}&V5f#YFdOWUale}z8aB~rRVD7AOme7F zN=_srb?10{e6ao9KqtGB3S%;CJyD(qiQXq1#)aV1=7AgL>@G9M6@b+Y?Cr}252QJG z*MEVQ{P-mZldy7|bxAqdo1OlsTSAK${%#j^5KfM#;Epl@99PTqu-xzo=^u+jRKl*M zvj9fs;xBRnOEv#W6LPFx_0=V4bA9#@KJC*NFxF9%rGR2UDmxgO7*dzsnOYCdNN%_XkhXxXmp3wEYaBpqJ@0zm1Yk;WCao z_kuD(?&Mmubz)z@%B2YfJHD*EKJZi%zjmG{Ih9se6uk#Hz0hE=Qfyya#?VROTUQ@| z%V;9K=G9Io`Lkoru=y6gq0=)#S9<@8<4vGz*}WGN$w_?fz3)FP&u=T=SY}i)9IOp= zKv9^5N6SrI^f;mB)6XWNbkZHHNV`)gPuC-EVh>>pDk-rvT@L7zEyHSu_)tfp+Awda zor&H+@+p5L7U3SX+s;I5`|!%TaDaS*#(nby-rMH9-xazimBISz)vcBcsuD|{o3zOU zRQ%4|c(9j^Y9db$94aQvKtP0;dh|X$%=(msS*0AX&hBD;4E%bUlhukp^I^|um!JW0 zP+|HdIH|}8n{p3ftfnG~RG^^6gfD

)|qJQ+8UqsV?@=6oL$W#eQ$Woi$o~emJ?$ z#p%I^x)}H>t&0;OKCx8H7xZ^ITGVYX)Jib9$%|Og(^Ri-Y9a?Rzcb^xwevVQ4kl#eUMIw!+N1$@W&)#+<2$(s^W_^1Y5f4=rwMaHMk1 z6ooJ@RX#7ZEC$sV88Lb>gvw~(D+qSrDuW9HH`S-X;QGD#?W>#Lq|@PyFlU)tPI=Jy zMNoQvmH(Z8G`0U!6oBM=X04I3snI{y77TASIJA?vj_E%?m!@_kRu8LMz>&&_Ul4>_ z@b9#dgz(_uag&Z8Kvc6uCb{ZXh~JUOg^`+dNqepd&7 zTI4d}P0|}Y&mu$lVfz>gSvPQrm;N}oHaPs6TmCZ*5=XLgON}_23T+*x7Gz!H3QYn?>d63RqcxR}i;o=3I@a-?x%cJ;9_+{lmEc9Ez9O7mc zo|3=^Ws<6Lmoo0Cv3vf$JL>d+>pR!p6I@!+wpCQ&CijkhuCRB9!(aQj(D8Ra{2+X% zB<9^gp)Xd%@(5t|*||CiVXe|IMy_SUM+`m=PAFqY{u~=5_H!8 zr>f3j#R|h((Iz(; zxaILnc8*PKx5<~Hr$YSMCf4%8Ds8J{H|P)U-<2Wx+#+FnxJ5ii8j5a9=#je7L7a#C znw}m))G8u4Wx0&g1%0&E6`z9IM|OmEN1l6e_`8Gdb~Yo-JrLW>bV|OF$OJ{TTO}Wc zG>TLXv(Rz#1spq<2$3-=6z>!L(ti3Q+v-^-S>ut(A~S|xbhJZ&z;94>pV^1d=O_%s zwR2RJ^^v2bvc$+oJ%x&D8GvL%IjQvy!k3izx`h+O)4W;E^sdg{UrbFb)z=? zsI&6$;=*b{Z29!ed1R^Vk@Z-PN!HG8F$eiSmH!#lD3hmMr(b;AeD`p*Y7aZALn~>$ zl?tN$(YxSlC_%O4QZB8O$D$)#%4soxZZGsS5SQIZ{?08&vctPIFso($j(%J!=peo6 zKK$*H>l?MEr}TK!Vw9)h&GtC0Ajpo(?K9E0{%NLe+57wsu|FI)*v&aJ+gTF}{0xt3 zFQ^|)!K^BR>J{BZY_M+*&_ma{%q-MKyM|Ep20A>)e%7<~o+dlX+Kej&1Y7OW5eP<2 z78j?lFOxQ82es)n(T@!33|uY;Yc+3u=3Igesk}%zJ*r(Xtl1jf$xl#wGq`2=`s*sK zabsS?%Fa@*$FJ>i3y#9|&LL^vEYcP3BmR?kK?t+#+-TN0Rmw0)Xk&>s`}pZxRq?lM zfq7>23>#tArh&~W4$uV_F+#R<=EeT9OxO2Ir+q>vDrMf)b$z~PAr0qTiKSya@a7@@jixi?Uzm3&wLJ9 zweMiD$r^2#f35e@Tc6s!u09hN&rLl2ry~@De2_T`O&nIwVRh6TIT_YdFm```A^R(U z;g}L)r#LM^eeiY>?am%8TEH1J+j?s0a%FhA5U?KOz;(nxBHfSD(-#~NmEr%Ko~vT? zkhw~EyfU&yE&Or)U87%h&2zq|BfCs*TnwdHmvkS zt34KC46l{QGvBQdsxgl)^kn!?!%MHB2#PoDw*fi=|7n*cAos~Y10C@wdn@|CeY*q` zrju8^WiWg9zukS29>moXlrLLf{RK}c{)V1&&sz|Q-%Gj#*% zT@&L6<`b5cyoCeShV_8-zlJ4&{h=cpGC ztOTS)7(9GFQ}~Z}A27VsXk$F+I+1+BDn18(ykOVTvgBv8|qSeQTKca=9;gYH0qxaj#$L89{$45tWVBiy`(gi2L zf#1Nj072vN6IFFHO=)v^c>v9485#f*ZUq4UEPbL5`11J$0KgK%08pPt)X%S2KIp%# zpdk5R|CT|T|0)Wrh)PR;9#u@7%*^bZE$v-MO+qU_T`gLvYPx93%kh}l+cFrL+8diO zc-T7pH39H>@O&0+&0LI#JZx?3oOwL>N&ch3^I85YW+Wl{kBW;mKZ&Ni5|OCAlNk{^ z0~-Sqi2y7S5fPu0sX33bn8g2>f7bX(EL~h2co-So-Q5}7SsCn|EEt)&xw#pcSQuGY z=sz{+ojvVbj6CS=oJs$2@~dwft|y ze=OCU&74H-Z9ffN1pXJ_|1ti*jsIW6|Af@~-;ivqT>m%Z|7rOjOFqWGng1Uo{(<>F z(odWPVEGvT+h+o>!{I}pb5#f+Ehen$0dndKWvDk;cmKnErJh_XaUqKS;kP#sLq!z2 zi##MgUPNl-Ls@@JR-foiSW1cr8CB+}6dSA=*!5u7wpL$de)s$RVy%KyPj9orTspNx zEB>PE@N{a*ufxK^&D-NIxmRlEzbC+sY^e^D4|?fh zy7U!veS@EjwoB?K%{Q_2^>)a}u~3p32oxppd^oqr`eJAoX7FG5GlM*0D_{T3mj@0+ z4U`AYs!H--433PU*-j9Tj9+PZ|M~|)|JrgTd7Ur*Onrm!Q@P!r&>9XS5R+61Y~|u) zmz2268)gCE+oKs(FdrS+s}}K@kA_(b^?anzFwxfG34JgmLNakV*x&~KZ;>Ww7x)<& zHIyR~=pU#v;yfZNO5oLOpy_N@Ne*7UW2Ua&!qk)+3+`v#Y0cC{y!+>Ly_hO z2J*MwbU4p@F3X#_#;7j+lA`DAVHMst1Mf@=5BK2dbT34GN?*c#SjRtKfp!1gqqY^? z`QoAPsn1ihpq=zN0CZu%SBt?NVk;|uBwkL}>P}L-7%u|jt`n~?}Fa)g{EM6msKx5wr3clZky`DY!N7eKg zc*onON?4ht+#(<*w=QEJV-Zjg5KMYYQ?*7$MUkI9!f?AXsDS!({?UlvS6g5)HLNEa z8WgFse4^Epl^NR`nJseSAk}$AF4#~l1l$Sd9;;AD$$h-ndtdl*dp!R5YMP0fxR_8& z+~p;0>{A&Me*ZbhT|TbvCPPC*>s^QKq+=seiwC$!Ra?&1N!vW1?#}EY4TinRQ&LbN z36vAjh>E>&y<$C2{gc8A3yoLmtmBf^R9h>PmLYjr}TjLAHwfnQmehQzaVpX*;J&rOdCOnKTz3abr=G=)+dK z!(+9?<&e%um8)5O-ybZSmV?#2jI1(bqo6cq3%Y{$Ggnv#hKF?oUOh@oEeym6HNKq0 z;(GjsFpB_ndrpo}2C-he?|LEY{dIx3kCd96>Y8HGuzJS1#SDx1a{En6y{BmnZzfD) zK2g+K(;9W0si|4l`p--!ejH$!@S|Nt(22m#QZ`cvd9or)UcsJqRAl(g*kv zc}a<~uClU!Ie$-j*aCm?neddy~4}*69=BmM`HD+XF%wY0M zMo&Q>PcWg3{#Z@8P)`3YS&SGjTuvQkfe3(&IaZ+@J6_sM7#sTnoj6qT1K<&S-^tBt zYUoth3SvOxTw-ZIHK@CWQWq1IZeY`YwOdB^<`d>w4i7<1PhJ|QP0d^mcm%1znJ>jQ z3x|b%sty!MN^zbo5f5_R-^Umkl3$BS+z*j=j^L8R;~c7dO!IqWMi0R2ckYTKbPAGL zS+wmSgNEJQgE852SeTp4XK@U0yP2|jEqmjf<>0pL0KDmxpU>7+3}TJir}|3 zZvjuD{u7M@m8o$}ISg@C8v1=Ywx-5Din98CucNb5cW!>3oPlBLzLQ-v`%Tj9<#eSX zZ1cwtsvCHA6*+r%VerRJQ!_JZYSXVhes2@&D-bI-7_vU4v3T4hZ$=yYtP83;n&xZH z#LhuMK@)`IEgVhqFoyJo0dl%k5kW3w>WwCnF-(SN6VpqToD_Fsn2$!>fEP*ekqN6L za`e)@fe_8M?^Bu`Mh#jm+?GAqtJ*n2`8L3?5XJ0w^=givkOdj z9jNsfRC&z4MtIWdwZ};VX;q7S*d|~wKUP#9Ab%EsN7Dst@A%|@(+=$rSWu$ zMx|ktAs}`{%$xDJe8iS9XQGfUV#5lbmz-d0tck!6Zcq_EVTFFY9*d8J=fy;Qj1UcocfO4* z4y{srUTUl1SXpJzpp{F^mZQt=M6ikwavS)sBjHBK`Y_0Ny5jchod|dM$yqf9HkGJJ zl(C`lK^X#^HMvlJFqcS76{fW&a*u?5og!0uBWxA(G_7JhJYVyNdL;z`>d@a>(bHu& zKQyOISnLoEsvj%E3Q_SfBD(2PZm?L8leK(D$JV-SC_u#=GI znf=e<_sUa}$`tzhl<9YeqT|bxhDRkG*t2<@r!J@3dpzdvYMD-jvVpM@cc?4z0>t%~ znOY9zK16My;z}YmjaU;o)MlEaP7}z%E3Bm>$OE*d=BCYX6o~TQ((z2S)arO}@Eq9wUnE%vF3gDbV=+F?Bgn^`0biO{Z+ zo34NFC!^;y&G-fVeuQV1Z1vG<3-I~3YFFj(A65-lZC9-`Q$O#{$`|#ggCo6MR3)f( zHLVW6ResGg8LWXH^mn3U9^mhmtH0}<8Nr7`+wM8Pl%N(*`->XoVK`r#eeF)t^YRg?7ne zc1=X7g*%u^T>7mGOJLKNBkWo?o3Mi*$gQE^eNPy#a(pP?*aMb^f1e~Hgs!gdYs9`C z8%d9xjlKmMU#4 zTGcS8vdBe|X@T!&kr8O{Rpxt)@Q#eSRZ8W!oOXEYj4MNA(|qum8%@?}TIEcPlUbtS zi0o*1dUlOuOoA^&ps=@Fz~C6}o6GI~p@H#;DD$b15O8UE`4p8h1)Ra)-86NmIb&{$ z1Ah1O>`H^4-> zW<2g!m=bOa9Y8+UmtyNIr`mi}SWrLmzJqN^7(k55-3=8M@MBFMc^S7fv~3cY)f9yG$(yzFc|Fj7y~ zl;w|8`-8)V{wonqz_Dh;n`u+RrWHOmzgHtL!BaEO%REYM&A23f`IXo{YpN{cfcthH1s8F@CzNJI5Q;0LtG0-orxu=f}lH4Y;r!C z5Y?j`XL!VQ&z8oqDK{P21X7WFVp9Bw$pc-k5SM6XdMW*e^f~lP_fq9%v^3=s_}H1C z19)l-2dCOHc|%RCAE+eH9f98=~=t$z;_+uq$JcpHHbg! zjO35El_$`UzSWO5P3vysF)Vhs4qBmH$ac;)(uEnDV6ZqFa4t2cD{<1OXU6$tXf?j| zPEG4iC0TdH@%wu+W<<=)v(Qiq zUfKC=nY_NwjctPO}ZO>(>Zk1RCaIZkuJMdCY?{BJ(a63nl78A`4!sTI!?S-CYE|_X9CSzZJwe zlN`C%Nx^HNYN%(hWp(lxzvoY7;x0Pa$Z^U2eh^at9kHUBIHZb z7{;31^Ilp6pP9M&moJ^ejzf;Za?)8(%LK9H&EX=sF5C($SOXDnf}Yo=^9sh(0_HDT zv9>MfB;n?ZA7tk`Eq3gD4xPyMsYRgX)Eug{KfvqxZQxE2fX!y_2?jJ4dvR}4CBQ#x z&oY;xwj}1KjIm#7P%5)_zwdvC`=Bd{u`Rw9)BpbPt6XK$S0F96TwVsCkxZ*zl~BCR zig(;JZV$Qe73{n=}u=tuv2e&&6FTSLWMZ5dps?;74> z6!d(q1di4bUEPFOhhT<{jYZ1t5aZo&J|nE&N4U<4qt6Q}D`fMKV%2s8tU*_ag+}tC zV@viv!ghPslteUoo6yo&HPAPbXu$%kG#jUdq(Z|{@@SVNt5*kL!qMO<$zVCEkGM)n zNNg)S6>hUFpZJudZa81oGV!P%QBhGROl8f2l4@P^Jh@hwziKKgIU5%R?1>5Zy?U_L zIU4_Nbu2=uFP`Qr9zQ_`--|(`J=Ts`R^d|l?(90PMx>F3{%z0bU?L+-V{T_G6Eu)5 zLz5E&mWJ#ebg6LCx_DSR&(&e>@K0QYxxza-bY*aBtzPG_H9G8Gf{)T{e!ziDYG0OZ z2~(kQo%5l>Ji&oA2Z+G6NNA)_0>KrLy=vlPPAeJ`EHtc4wSZ?_kmK2ElLS>m+$bg5 zOB8l3{lF4|hNR!JzYW~SvVXCt-!h@<&UgM7&%O{C31Xt(dl8-RI4lG4MVBPMZ5Y>j z1!LJ^1&u|zzr$&?VKVOJjNv-wvRcG`MV88Xlhsd;V_ObC#iGH*B9Aw$C$S)A+L>dk7xCcUK%ns_MAz~mwn!IMyxehV=D9CuLi7e*;+a*hERL@zly z(ieB(+IS_L{Ne@(X9UJDuZAtUarQ4oW3&Fz2v9SM5Nb-?9g*$90ha7u zF6t{@4R_rbmDU@ay@63Rl;IxNJm3CJxe3v2Tlt@zLn#Os^WTL0xMrP{P4SBdBXah+ z>JllE?yNp?-bNt82XU>G0ksG|m>E)eY|@*#VQmrq9x9-Nmz~9->{kwhj1|QD;O3jqtVn1Hoem zYqHxwYu!6#p+t^Q)TcdHSk&r2f^WAfq`h<}5v=fX!Bu$dr3gC z2+*<9^Ts@w8-kwMWRo*Rlkp{T5gyj{9_54tQhmq6f?!dyz~)Nc&ojnXj%OhDtbQn3 z?f0{({ylCA3>@v8*$0c3$$)ZnM!2mZjzHF{34d()q)Y&5LB%m=SHsh?Q)*OxN9~LN z1%Qr~bUfW0zr~S#i9uu)E(aR|jgObPCSS@aN`?^v7iUwyS)|lxZij=zpNep?e*mZj z!>Z2>iTrk_({cgaYH&@((92E6M#}RFDK}x97|}<*=8GTE&)8B;%Zagd)XZq>Sjy~= z^Ed|}p|pF{0Ti*CN%}w&0Z{`_l{RJXmj|nBi!bZi=}4X2qYfJV^@{4Fzrn(zi9X+z z$dkqD#&9T%#g zTz-|(1Luq62BW-`sIpi@ldzh!%ZF1(ChzQ+AQle{#D1X+ntk7~otzF~vnyhXtRyER zgQr1;IUp#L*smN|cECp$a45+Xl3#b-Pqb>Mr!N*==NeIuzd9@{&UBjZH%(jk+K-o! zcBRf4a=$w|wV;DtTWpJjhN+`gi?n$a)r*k_&~jBMoXr26e{Ts56`vn7@x5+Oq0_Bo z>6~(u_39jYL9ax|>y+3Y_bVYP3CQ5E%|ww={YlkI{cE&?pTB~^FA-9sp^>lFo*GXp zjP{LMr8K>ALbPD~s`mo3=bo*~x>FH2`l=c~!plsa*Lg^%W#$<-jgzE?v}Q^z-Kn6z z;s&ekr;K8#Q(pX4tEkTu#@6C4A&=d95u0yW|60aukhx|s>29<=<8e)(=FMFvHq5`u zd$G)Q_`QXcf0!@!?#Az?p+tP}3Ju?I(<-)w@sJa2g$u=9b!UtZu8#{XhzT5MO6uSN z54Nw6%V+yqPhXzaABT56(8aube{gneR(Q_Zz;e7UeyA@)J~FMd*so{+5-8~TiY|I8%I(+r`5@?ZP&d)siHhC|g?g8p~Z_L>n4=m@xu+Ns)s`bXSjL8=P z4!mcCi8>x>b9wEMfjg8x3e(~<+r(${_zq>44r_u@R%Twuek`k**wrzxvUhl3pN*@u z@kEh&x8)?9N3P752{|*eCA{7?w2*xzE(ExxO6Bn)gdxd?8e>IbD1o#(@3j_?VYKKA z1M9D6+9fI+6U*{xs2SLkVj5O0GhifjAt|vd5MQ9`3NQ((POZ?r5uc;`>6Ea>!x56xMaG8Vod%Qjnh|+E{GA1-VH~zkhk!8JJAL(%43qIVIF)r^2IK zy*JBznYu=&8i>S%c)ohd0gdLWLu?KeNv%*d7U^Nq7_p&+K)<<;qm???cQ=;COAq%f z2TurN4K#@nKPVxQFbnQB$$Zs{4O&6HwxN_iNwp4Y)XMeI9>aObcv`ujlYRBv)W>Ob zFw$9397w8Anrqs@YgHx}oDkL3WJDVdTOf#vy_42oJ}edondETvan#kCM!#VptZAsK z42Q+8Guy0>xx%(j?Jcv)HESa>%0t$mJqQ4X+2C%-H#*Jc2y5@fEky+Tk{}F?AP}%D z4`?C_!im24EV@_NS2~J#4XpgpX^++=kCjZNtZ_++ha84{30#VRSsQugt{!EixASS{ zqVSx?dE=l4xdplZgAuY|4^F{CaChQW@47%_vBXWCsI1^tZE9-103pde?&-PIS~wwQ z`UVu0B!e}-&(9ZX^P6SA<__xo+(t$5ucgHABFiyk4#H&SU`0F4Q^ci(v}!g)T97^A zKit^r&I?)hKHu~6RN4kw313E?-d0q~ldD7DHALtv)lutx$%%d-w(;9Cw^ zeSIV=P!L35!TG4dPQIdE!osz-&h=xOd8Rpiss4N-?t>QArmW31^23OrnRPY&p5WGt z_CZ?{eSn|LQ1_Z^I2X2pl51|&hB7VQk&)8z{4ESf}_czHpt zB25GeJC}p{_E&ku@{k1cD2mi^M*-w>-_yaOS+)s?E$$=1Fksr@C}Ie0-88bL1uZBr zU%Z|So!6**AMi*iJZh?A=oHcX=qSQjmGC@gP!@!9p|*p&*^%1x^tG8tjtq*8zkeJ- zj`Yfql90%GnY^O+c^S2V)2+gvn)--p$lBmSpi!;`Lm_6S5KpG5hFK~%Q)nqj=%P51 znAM_FQY#FhN8XxU6KLqb2knH2gA97hRrVWdE_r<3(&($AW9tt2WG_=$e(pKt7`%-P z75j4)@Y}prJ0(|D$+%WjdbRuaTzQO)BqAy6qz+96wa4rnqr z8I5faad-y!cq`+A!eB8%@q>qdV9)Rv%I^#$Pvcl+54owq^1>ok*RI4qn{rw(3qnwR zXi~leitO?wUj>$f#TW~d2lWm@>%Gp49q58W#j80lb>_PSkjj$h@L)W%!Tu+jA|!@#P%h}m>) zDNyK9(Ac^FHDO~1^L%mi8y%A=G|##r!Hc>{ybc@OV@LQp!ACmn zF@%@+r4@?nv3F-VTV>FnKP%*19*_uMcqo*pKn&Y`6LCj=jg8kZreJ-%wLg-qb`+tK z_Qsh&OgW#lC6?9dviT!IIuIy4(lUXjtxe&fQ95P*5@qi76hhKb0O=KC&aIl(gSkmg1nbC}|)b0AMNiCJeI4&*so%Qg<+13AO9T~Gd91wWuHX=WkkL4I(K%Rb_G{O^Pxg2 z?X)LwNGvoo42kR~&G~l-wzvpJI@*ag72Oe#4e*pu*X>|vq#DshlJNZ@GqQG_ZXk51 zSSSkyWTCb2t;$r?Bcr>BVkd`&bH%d50vRu6`v+`7b`4oPR-HEM^&5Cws_KClyq8;W zaURNsEUS5m+20$8)v-Y?;v#xwY^(+T7;TY1oT&)!dgHW$1&FL-!HS_ilfhaLFeT<8 zkx(iQITja_jSkuJPG~`N4*9_ji{c=dWtfC4!2)+-s6>_~GQ-lq$wza$pe8{?Fm3N+ zS~PN4jF=Zf1-Mb>H72AZlXxMZUs8i|Xz8lemldk7sav`0t~n_Ue;>M6)0G1pc1*eF zOi1t*)X`BFX|j`W$DPQBo~dp<;({VX!q6B(tI5$nH?v$csmr6h{+Ic4CL z1VqrIHg*DL3NEZdtfTQH^KQB()JwSU=#^ZwCMC0R{o}6t;;5hT3!U~Nmk}Qb+A#Yt z*9B`&%41Grm7~b1bkySw!AoLZr(&*09v>Pl&F@Z$Qyhms>FY|h1r+O5j&QcghgOxz zd9gO@vt;TK5h<$@l4PJ;f9QkG&n*p~zn1?L+SuB4Le*H!{-LtXNyr1OO@I!1^A3Qv zf(6rv2#Ty&R2GZC5)LkcFogPD;($V{N9ZC4Q>(?~-99nE+#L<@-#tUx6H(_-_lbEb zF+9dVCyF|l1TJ<{xzr;;T2}i+s_}-(kU4zx-9=3V`ykS;?<4b+b4lE6P{&yqLGy@w zvFa=&&`ZqkBN$wv3V*6{6;woxHk{>#KR7$y=%zFR2?040K&E3eh}ddi8A|k^e{{jz zA3k`y!xHs$wSSaKXx{+{n6uy2uB%O}BN&NN4^a_J9g6x55?_lmo6Ned zJ}gN7Yx50bt0w(EbRPp~ikH^D^)ZFddGs%R5?;nT@VBrwT7evU^#L}bvzS-~gc;#2 zHladX!SkDiM%ouxi&)`*dIvV(F@h$7ji>UaljmnFsZD^ycTnZSk5{7qP*dmVoi)WfKxnbt`5Y~{OE|;~2d-{EZjPZmt-n`p=`E!nV`)+63 zsa2&Ak^Ia8&{mRhMuUk~ZhWfhYU6)m)=<(M77eEsSfDD#!3oC3$1gKCgB2Vb8Hdvt z@Im4lkBUW%wsT)WH18Vl`N|n_WfsH7HAVs((U6ClIc5@5LQmGUfUNK%ved~DXIudh z2Py+culZ(rojhA4}aalZMK2pIzqcOzP1|%4`C4%Wiyn)n4Kf)=I)$ z*=sP7*Tlq+@*VDsU0R7lUsN~dzwhzfz(Dxj@aMpt_Uelj*W7{cNp8y0N|_06`f&H# zt2~doQ>VqY>4y8jSo@S>1)Zm>dJQS&y%`-(wTI)tf#1DHz=x=aQ33g%=L4 zYb(X2I)%CB!|9z2GrZ%=+=gp)1D>6+*G2Z1`^$EI#aG*_-nM|vo{KfgimD0g z*_LzO>r>6RH~Wf8@##XN539VqC3=A>zSswRtE=wM_XOm<2~rmCc8anA{F{6->98Og z-t=mb z$B#aS*t4TX> zh08p1^vFCJDPL8}w>Y?2<8^w$CZAG2*0uQWm+0hhf`PXAM;LM>DTykYrj>%X3u~0Q zLsEX<>lV^fPanS-=b{OH;X@0khQ2-xzE&5AQownRck%vuEV80|ie4Zdwi|r$rIyL` zeW>v*fl!9r!(_*_?`junf1(aCsQ}``rI9=zg3dD4E)p=z;k%5k&Bn=#Mz>LN?1O1_ zm#-609eEy~>vKISF=?4Zo{ti0cO482s%Gn2GTU1SBx_k`|iV7|lQ3rKVpaLU%me+zkhb$&{eFNEG-)SzQ$UaQMX9_2_5 zymh4R^(B^ec51JP^^fsOWx(M1&r`0vNggo0Dp8apaBwOcMvciI1~I&j$6Oo*O?OQk zJtn@-Iw-BE&`LyZ8U;t+E!;I+7~K6_24wyJ%NhKn#RdJP#Ysz2XMK{X{?Z6P$%3T$ zGM_&`Z_xuQut7W&*{DB-NFqWh0R+ERR#nCGslll}%E6OUnoh;px|=)@B(ruL%8w6c+e0$5@z+7eLv6DniX|B69TZtp zK_%uV#W1h}3SE}O8yL{+I+D}GhBW<>f+`ojwRxf1BMf^*I&hBPfa?Z}S zn&KaYvOVTawA&?}w{o9kM_Z2;-)S)+ovKeC^uqZo8Za#(-w*!!uZ|OvGB~tXI(^Qj z4gbt}M*lw~OEH-ZkeIC*!v8`&E&PdU?)=iK@;}T^5b=Bz(8(ciwy~mEs?+(K6#ba4Q79WbjWFGp~}J+8zu426#6$Ni#Dui zgvIkaNRjoIdcW{3Nk?2c9nJWamw#pdM5E&`8nA(~$pG`}L7bf?kBsl0C7LQFH8WkA z+A%)eA~V)a8vqc2O*Jlp^b3eBclo~wIHNeApygKt=kk&@?*GsE^Fx4O$p|T0bvLX0 zi`h{dbgL5RBckJ9qyG%2Pum<6hXU{)y!pKD?alK4Y7&7mBm*G+%kESHZ|D6hafNbu zh+Vq>?EZi3|06n@`7iPHKUx9jzin*u{`H^#H{>t;d{NPVeIK3wBbwRZ)n6<7w}-jZ zK-brQq5i-1K84?`7f}EInqu@IJZ%4E!=A?msE`Y}eg-~U75KNWdIuu{{~*p36#N^( z#UDId2lN;E%$v8q&X0ejGrK64E@46)_J~6HM)EO`xIF#ZHPoP!#9QaF*g0^rlwzSj^Qb*_m{=;^R0WWbzKoI@^cW=WniShh)joQ7H$|jkDua7{0mVpB zOUy~4-FzPcyIvD~S5wI5+90I^Z z{$#xxl{RjP>}iDi?PmpJu*_OyX1DOhD;dF}!P*aEDg@iQS@f&h>rbMYlMB!L1&L-$ zlOEmD!ca^_^JnMin%QsMMMlazjq$hOLJ*PA4WOUHB7Hs4E@;CDqzw)z3-7p%gJw+; zyx!u!uDbY;Uq?U-STBdsW{VWE{Xe#R}o`QmC zYgAI&F4I?7XtpiiB3l#ReLVy>Jra65k1Qg;I~vjHuN$=f7Bg%%-HdR3FPMj6Pq^7! zJM)vJa%pkNP8q?-UM1vPbbjY<#U=`)y}jrDilB_NBk@stL|Ph_S(&Pudh^fRMN{*f zV9D4d<|GTL3xNVBg2%JvR&B(K6&RMvuKg&<=IP;SMZ4DB@kBOU*l<&WU7#Qd4D1 zxqr`+aiK&W(AnnQE^8?@V^&{2h!L0@A;hkpTj1+0`}}lj2Nw{YR^|cqEZZ0D$nqKm zSEBLZP)+F~ zk!kM?%8>F7h&u|Jo&?gzWty_G5*4GsT~F(rlaUQ{;})Si5>ngEN!;kIb#;m|6#<44 zhjJKv+VM?wO5CQO{BNHt94e~c^!a49rK++d;2)fRx3nx(vY|18?=|{O7qN%xtP50yd;^tl%DYElC z-aE)j<_cuHfl$i%AKwGATCrR=EE9kS9Bppi=)(6}La| z$5uHpYm)}ksB_3v*Ov8|llr47>sIntMrok`JaCe3UVHnb6JCD1yW0;{kVviXS$ynI zuTW8~!<}wKij%$_huoxt;;%M})vQXqj z!mhh|ypzAA$Nt`380>U!1sMge=^}NuhCIb$-OgwL0a!- z8!=p?Ao8s%`eak$sMgTgI|;_p&J3(ADse8ak;uCQ96i=bZf zl~j430{VU|QF&A_i<)xrEdEVaW8_IMTq(x(D!%FByN&miEso2XoOC+3{_~LJX!M1z z))2o(RA=j!Fi*-_1bJ3Z9vtRcWV1RAElp;Dt-;VZ70d$ioE-Wb!f0w-P%kPZdPrrs zpKoTbt#fr1{4UWEXufnt-h>Lm{@~i0WIbbcQqRW$mNHnNma?uXIGo1fKm#l@*@R?T zMLEsW9OZMDkciDA39L;PlNcVa z6MY#F&5lg=3To7j&hv|@1E>Nv1>aAZi!?n%AJ>$tZ1msC@%UR6-#6`Z6TTl|XE5D- zkzVT=RshxxM%2dS*5WBy)vpu9IM&o)dS1d46Ec^|j~H#~FNOWmUsRI!5}yg|Ii+mYDLMFD^E&!R%1_Upx*u z=gfF*;l4#}4tcIVk6XpnvHY{Lx?qF4Cgrx|)!^wgi7&prLyYn|E37kl&hZ`u@TK9fbUA zzQrI!>^?4OJ&N0D`&4>#aw@qmh|dJEtf$N0C-D*JyA4CD1b&^;50U83tfQG5G>tDK zVdR$^@({-^U_QS31uZW+#e(@dEt2q0l~CKnjYY4UN%1%9ZKtD!;Y&px-EEa^hfJkL zyi%J#?BwF1J8<1nF_G~8FON^SD&3+O3esPjL&PP%5PKdxGLsVFW<3)x&mQa)`gndc zrR^0A`;&kKc15#@1`TY$jUn&y1bIIsPz_isUGexIpzlCklT+UK=yf5SnPtrQeceHl zmF^kAhCw_I_Tvd(4tB2nX_~=-EO@U1{oDWwjCkr@E!AY35k5o3$TBr%pUUl{A+U{z zmS(EXxUo>kWJX6`$9+3s#4y`WOmYPd*eMz17orp3AauU~Dztqr80)0LRwJ zO!BckcqpzHDe2~-TlzYqN^9gSxXf*KFT%>IV!#{I|FgHT6%LR^fPL#kvmk$2roMs+ zT3M6c8RLTPh?a4_+1q#3|rwuTNfQy(<7J7ZeZ7b0&)xoVxSws0{w08oA2~u%=Mz+2LOkg<%YFa|-@YNZy}^ zh=T&{g2Tt+XVLbSfn6*<>!t%T(WzjTOQS^&5Nv3sR@)lt+TYMC0MPsT#05^wv|ucg zt8KzjAz3KB;SXo=DHlBmdC%(5`rrbf?)LTuI5E8gC-yyx#;8ehd?56o@5r5$^RXjQ zih|UVNT*FdMY1-?jJBA--2@=hMv=>fDy1>hcLRYSiD^lU zkM|R^q`iAEREo!Uw0@|?;s@b9-k8kf#7>9n(nCAfZw7*)j;J9RAGRzPLyTjZtt=!a zMf*id21%3F>SGrlPwt-QNdDnEM0y18K!4GrG*Rh^G~VuKdT3Wgt+y){n79C0n${OE-*FZAfi4xZa_J&7@S{x->B&l36vSXeRv4S^0B>{fs87L>rV ziMGP@K-ltlWZ9g<5v1Z#+!+$OY8cbtyyJ*)4=6`qi^RBSfAv|@8R+N*L*GG}E;ex* z-?@L~sMJL?ID@$w%%IRMkI7)u3yaG$RgWTr3$BS6$ODJisCZ zKH^I(H8ht0)nqTw{v+*mlhH2#!)ldQkUm zqfZTGre|g2$j}}+>n|e@M4bZhS?BMt>c5Wb?~}}idL~m`oFm?RJT4fhXv~&T;>kFF z-(e(w2*I}Yg_0My)&k%~fc@zsQi7z@0#}1qRnAmQHlMAOWRc08U_n)w&z0#1% z=@qmJkGe3Krfnowof`ea_ySDA=C3_9QUf#SqXI4>^@zkW1h4>&`ZQj}y7OcpDUqhR zl$lm2o$nLj^4MJ-kHA3Xy}kd@nuKueL=D|MQt5}aU90#QN zw+CmYo8ghJO&`2K)3^N%|BsEwG~e1HVjh7|b<1EZjiH*iw>Tz7jS(&?R1gs1>lzC` z6OXIV2)_j=dtmmv3J#A~r_bvY?N-1yB!^IAj4UoC$T)0zUZbWFKL5oMREh44A9Ph( z?*z#yxW(?4t?OPN{fQtziKyXrj0;?qcJWR1zNa3xm^>aYsYQ{9^lrn5iLP^r3_c5i z)s{8rTb1E?!-&xo-O34P7DnvQUa)fzZQ`S6Y(WzSH*4_maQoLaL-GBr_`|z>sr?l#x*Zo-nu-$bFQAn3$BD9DJjZ!EUhAaew>Vg!S9srv-GNYS8esqk)8c zOuesaz_>Ad&?cs(R4feH9^fd;n>SJ~Dl((D{+kkFXN&S{|8qKo@YhoL?;JYPXgHQX z%W!)MMa*7Sr_CmZW$V0dDi{MV`8?kUr+ZPE#J!BxzF4?ao4$w-8r2gJ(WZ98(e|zl z)ttaMH??|>W}rzLX@78n?i-uk@8H&s>1hiVBwgw-zC(z#|L{{&5bpNdmvoxvNBROM zs>EoM(Vu-7#ea*QPM`@hcPCUZwNSG=G0}A&JyK?9V$5C5M?HBD6zP2osr^nxgGFIiPo_lwqj04Q5HwY zTye`q#o!`*IzaRMITIw;7=j4bng{wib!}8$;nW{Q&Gtgdkt+kaHi>67MX#Oe_XVf< z%_`wj0sBAmK)#QPhT$xMVT3l9exZ`0M7&^^fyCfxjvFiVwZR__e<~6QJ~tv5@=>BT z*&?dq-aab(oR1iGMP~;*Pev~kQ`zCzNm1C8b8~YOfAYMg-jKz_{HG=|Soc@0YuIIi2t?OK4B4|tWdL z1k4~8A;askiFR&B^?1BSM?cOP!w@E0t^>02yhUlYNaHfN=Ad`HT7-1gz>QSE3I+l6 zg3!wgn)@t~xgu@ESINC!8WerECtGkq0*uD)QfhB`HWG0%z1>H;^IH7KSgDaXXH=NLNV!f}J4$v{;ucWd= zMav^9CzTxM1~)FFHQ#&l5EhijrT6qh6LO>5;UXK0bIy%b<=&MaI=^=3#8;4xeixndbp0x|5@@PBXP`-sCT27gZl z0aL!@6!J1sJPw7!+WY*E)%Be0M`Dxp(wIpcf0#)GX-uL6&ce5^pL>a;|A(!&jH+{K zx&V=b6WraM;1-wh%uda_SeJ{AU%8S#l2u;8xu=nlTTio$-(Q2(T;b zA9nuoW3T`pfGRx`9u5nWI{LMiu4Hj>eP#bove(-m>!iq5LUdR1)SaDd!?omB=nmr| zaWkgoM^4xTzy9B`3HC(1xLd{-7c6xai*5Z5UAesbWms{KT@KG**8^|K#Ra~e^h~GZ zBrx=XYmJTv+rG82o(D7Gu5*qEUYx?%tHp)D`8s*&_*R~3LD`xKb^qH7ASI=xM^N{( z#3nrVAJ-6;*AU=e6jbyWWdO{GXzHDENK9-WHyg?Iin<~AXZp&xI7Jj352b3bSr&yp ze_@xk4jn4BdSWd)*#V?;F25k>#UsH~L40dciAFZ;bbQ6}b5;Gn6Q^-ihIB{2oSuS) z_gfMMHmP_R0wiJr@b&!|Jg#-ka@y?0?B{!sXoQ26N-#d3=Oa*VObh>X|JzW(t_=n% z@JU>M-;6=S#H4(`)qPhGkHIsRFWLHPSpgWlKASGOn(q{#7krwE*;Vj)DhNZgR{xX^ zinRk;As`~bX~Yt5Kv13%^VfTgu#|(tjQry5@bgv-I-21C^1wZ^(X-6Cw3t%ZrQeJ- zN&ut(&4lTR+*h_tg<(b;;sZ5`^!yzfE0h77mAC<<(VWI!C5B-4W?Qvp z{j&06D1m(}wslW&4a&M2%(IQEQ?=g%BcYed#;YagG6qK^j`8$Rssczv`Mcff1}}fX zdTH(G4WTDs+zzYbQ?iU1uq^Et#fLSxk!iCJ5c_7Vdm-h8DT`kN0sq@m0sxeM}FLD@sp)6i&00L3c2m{iIg4T zP0|d$tBlf*abu1+?^C)8krwSrJb~5HRBlJ7N^7wNp>*c~61@?=dK54|AsQ~~nUnY8 zw8wVU9Ioe{V;Lb|6~8rR(enFG5kPlHKD)uj(pDAE*%XT{nPqGvEvaeI@l}NNr$Z*x z0ppLu$Z*@I1#pg^__Pxbn9n6o7aKe0}FD5=Mq1aR8&2{ekWo zeBMZ!8JkTvi@5IBf}HrzH~hIm*c&GR_ml}SKV?WtWpXoZ0foyI z&E-vBnVUKF&ftJ$5qtF{4cH154yG8Sg!q8i{hBSf&-wln9k+q!(l z0q$k(vC_i&y9L|oT8xb@(0N(kB5%@|-P_~|%8G}84#DQnDqxgwsSUv z7>y^7(xv+rSnXeL=Q9{F2BL{=>3+?4sDL$JZ&Ws|C+}x_$Y=jNqE+woZl8kY6+vZA zj*diJY<2!s)_jC%|XphjoJeRSukv)N?@0I0V9yq9qoF>A^f5ps%DuMuwr zGVco$+4TPP(Ha&?`~4t3IzuT=!gdf#jW*sT6YX|WSrU@6crOSp0deibSGAKx*$;1} z*)|M&z6pFO_8{u+;@@dm(34*&lN8;e0BRCH1P79;F(cR`^0^)IRX}*uYV0ZXt(2ko z1|-u^Eb8{AowMkX5Mm-!mEt1UVRhW;ot!jpTgG%7plj(O8}b&9Yg2?lLXRw>55%6Q zwhR!$TuPNWYqD4(GVyKu*(wxPuQih}tOC^`Gg|h7>r~1I(JN(kR_ieA2`b&*`Qw3etRkSIQ%5!QsIl}{>^7K*lablmB+Q7g-Ne72I@T@uTy=^2whB|UFnNu+Kmk4Akj&12PMll1v zfbW81(hLKZFYG2%H+*B?-R*2Iz9=-;l`Io<{?tW1h=#qO6glKY2EM=N$$Ia|zpRq} z^3f*4eYO_xsRk2Fdx_$ugraUp0$SIcK5}h}o1Mw}>`P8T0o@=9^fu_5p0oHo6&tHG zjmZvDR(^5?c9Wnp?8ormv!f#m=VGZ^M?x}}Eaf4R4+*M2eIeWHWLn{f1rv7Ix&TkG z1*-u2DiPBc;Y~LR=gU5@EIT5{R6$b_CDgAq`0lpCbRT>lthk@YmtZp1bLeinw1T!! zHa^XhnLcH_QVS5!>dU#R^CEa`vXAPhyrtC!(y}k=Bi4o6t6Dsn2ZEPC= zyDPcvQa}EexYHDZuB9$J zS#2o&x^g`_r(rME-qfZ>cSO;Y_Gyb#N*0%4VDoCB?g_nMl-X1uI!>U*F~gpEYPZh% z)uMp;H^;$b9Is_E+Y}t~#J+jt@LMK_8LQQsMAe$fW^yZil*CgJBIP0rz9^1o%cbOf4H;CKw6-Dn zU2dPQj@n#~0zAT#zVprm-5S-ccl(L2;FZy_p^*s@+P;>0Vg9uGusBfX@)&ZZ+vzTC zQA6(QNS@@H?J$73kA3ceLqGh;3NPX>!h};TCz$4;%WdV82e<+vdt9#4)W-ej_PQ&=hYzIs2`#%9;-;-gnps_oi?gjOu$FR~% z-S5D)iTIjB9T%hJj%y4z&fVU-%nJ}bAKPLw$i3XVH;FPe+{_6%XPyLeexjA&*$BmM zXfHeqA}wEwXNaL)Zl%GVWI(nxZZZUOMfvORNos8;i`Wbm#1aXD%4-P?@$rR2iATQV zkovf9l{Bi}`y)N_hH(e!`n>Bqwr&rGB^aT&I5Jm}gV~!d;$qt^Ow}(4UxuY+@KF5# z=hE;lCb#h$mULH#Tf&1K5lMbopCC#(f4f4C^1SwhLyhrnY=TtSUUPb37kv$CFs3g4DcmTh}}s&9o&J2#&?WfDjsbUtE;*Y#eyBy=tcuY?H+k3oitM-$LWNl8K5RXyo?UW%o$8i(H;F01~iQqw;}#$z8$ z+_V(v=@v1b$mT~QBKiym2Y1y4L!8xK#=PGilTetvp#|yc zw8v}PN$$8e>Q^@&9s+@t*7AJ=8>tl6^EtyFpG z0|OO;L)i)D(R!#*d{LBZIEhK^*dgKDEr=6xm&xml_N!kDw%wq@lYCUvn)h(R#T)Dz zsuoeUEi!jsgp7APlSOQwJb#&WeTHc)1-(|_?A=AS;q#qHA%Bn}(H>GpG7uZ%5kGsB zQ~bsQDG+OHjgK(K-Mn>=8XXF;nvH3H2I53|Bd*_yShd#^ZmLJ{h4?ZUVQ1avu^qJ{ z1>WrIQM^wmGWlOfy}fgUYfqZ!TH|N4?EN`$8~u7_#Qq4cK_66u9J4Lc*KxFKbir1n z2O>ZRdsqVx(q|FU0-hWeup+gxnRtuJj!K7>R-8Z{cGfRM93+i^IhYQE7O*O0$1B|e z>9WjQ4vEi>%OuD~TjP)&^}Z$Ob)T@ju&#Ta@!jNoDCC1F#1V5D7(WLZBwI&c6m1Q5 z=;e2BL~FylD=y)fPR3ddBL-kZ)IjC~d>TY*Z4df44x>-lT2-%5ZU$<5dPzoy9I;0D zx#QE%B6WSvH;cD9@#w5Bn`S~9Ifsu$`kFdpGO#<?7)5tFevqikLi^ zhfzx^yPBJ=UFg}X<`oNAeKj=hc72gX00tD;SOZU5kvN$B-=U#a(%8{4f=WxxsHHus zWc^7(g|Py=jb!%e-*S7!{!c{tid)0Wjm51_Hio!Famx<^?%gl(VW)Ao5iA)(?StD+ zX|niMx9fg%M;$ zLslZowFF=usgWP}iP{TWl%LDFOk>{I(HG{e$gRfZ-lSkf{mweAFh3=JVUPt{BALAJ z!Aj4?O@wAOB-@w=<2A%wnfqfIV8_wCjeljltyxG^F!mW9{mX1`C1j$ChO?WZAI?PJ z>;Sdo@nyz*5!e;F_6>{%K}fr zBRTBm`G2m4f5p4pl%g(RY|W}1df@Z9gw$>BoZ#EHNMDRqEqlAz-F9$KqQ)Fk* z9ri?;%ieJ49W#ef-9VQB^bWaHE~5sj*M3j-MDsuJRy3x&t~zV2GU~HxD2r$8leU9l zwv3}A?!|%-C-E@I14G_ThPUa);|KfjS+?7j+I5#AR&EybGevxD8}r6uz#*h-P)1ZI zTpuzm_@~vY(W-rLp7X>H?EU2@o_X5nQQKc_elB0VQW|jCwH%ie$G5ug4%NCFwOKgL zb7h5V3@|Vn$$I0hx-?ejmPWnl=OyOi6=X5BGe!(Mbp`WcL4$B6!Cc!TwQk)6|T!_|gJZ$*M| zl2~~iL^fJU#5LI*3JA!4eyzU1VYVrv?JM?L+E9UOc0x9MR++=Vv@o63vK=^j>kdL} zIq!JhZCm%6eR!r-EAzYZM)efEU%8e;U!1r{sL;4gTk-MX16s5kefz|Qf=q!2=?bii z5s2sD_D}VANJvnopfm>iaYkBaw+W#Jt>agxNR`J8aTB5Yr>_@*dGP$+=OLiQ#MY0* zM^**n0GdXG8D#|9K=7ngiCoY6fO_A~%1oOxA;q}DVbA}q;DfU##7n9eoZE=J?l#m* zWrsjO{ty*WfDVx1<;atW&Fgd9@8>Wv6*3k}gR|PfMas$pL(wbo^Hbhm_b+`IM?h>i zH&34^L-Bljv>?E{K}RnV^nT$6Yy}NnZVwpuKwCMh*9g7c`}j~k-<_p+oVTgI?fbmJ zNcXf?egjl|p$#;*W{%6W1LvknzjF_b^{N{cWl*-&sGIOTw+dm*?c39JYOWOg=yU0t zH+u>RsfieDq5`6vj)>%nff%?B&K7Wgc^pkfKp}bAjg_OnziKJZGZ*220bV$93Pk8= zXght_I*Dy(jfD7~#29{aR2nE|n0847m|%MaV*W#0e_!k#LhdU(k(zEw{$*HBQli)$ zAqjE!V499=SSwFD9uoy|KnVu@a*m;Tp`6#IfA7|WI&XW|skil+E|j{xuXZC({9iLu z5TzA?9HjaKh%Z9?Mi@_WmCa?aT5U$PAeuGx-egWcYVobnO~k*S%7x`70khpMI+SjO z{JdNINPn|@JuCPfEPX)Q@t>ZUAEZ(&2>B$iUIdN%>U4$gU{&c=-^4-L%8xX6D=g?I z-ybK9$7ExQod41{K{EXQU`SSRx(Zq1S3dslpFux^EiYyT*xj~3W;T7Tru&z48tzLE z(4qmasylZ6XVibjIrt3Ay-v2Etx#;g=*)Q_~9CVmsxgMmzdj+l0 zF)rI0D+Nbi{KW9cNQ5U@m+H;@N+p0sln|)dV|FU=N0knBT%7*a*&b<^@S9tTbI6ZH z3$O-IitamVX|HA3^;HA~b*JLm^M&%Y$*{{dCSGeBuWsPqH>_ZnZ6&x57Ip~vh6q{H zMDNDoA?-1j%~$f3-Lqwzcn5-W&6&0FF{BVl5W}Ym5#kJGlG51tYWVnksGI}8H|iV% z6!rS&IFW(WZ`OnknJ1&7q9TI2KvyzRoRJhMPTpjAPFrRZ1-70!guHC<^xl$cL>}ko}~@qVjIIGz-BOz?h)@aeti(V?Lg%1 zwj30C9At$jVvhgl!{I`z!a`Y*+C|if$p5M`H{l;X@?uBAv+ytF{~FE`eyn%Nbspyb zyA6b>L34+jq}h=_gQ=ANk6rBap~9^3kRSh}_E7BK%QpgbEOAB$sm;b&r^C4IB zp;CU<8mdO#zk7Ug@ndGQNRx}Jxwrp`nicawF;=)ceujf(?=&~n=#zqFAz z{6{>_t^XLe6#oBbfZQLtH#3j?c^v=E0wft|+;z3Nq4wOr9}49q^EH;0`#n21*IYQ{ zF*>7ksSxaqe%odwYx~bUuA5-zj^2F#o%0eLI1>{w94vf4{pd(hVKYwmni6hl5-?Y1 zYxk5~Ea>sucXfAlj`_JCGIE$lK+BoOB~w>q1vuCf>mx?m&+&LP_yM`my2Kx$?zyt6pizf*&bez+*V zLbkk|&iJ6zo^^le5e*kVgw>|sS@F9%Ddw*6plWcEM}V{My7_nJoWd@?2RW!*f}3w4 zG0C>CJ8)^PJt=zH7nrtcIQa(XMi0$W7Zt5iZYm9)yS2n4T;30zHD6Q~4rZt7g_Y?4 ze*YdU=v~g+*aIDk=Hl<`=<1sg`BUm+fBF3a_dBwH-JjMd^95D~S=Qr;2$Fr#`3C(7 zTfvrTXri5_AL$dD*B!FM6r z(TTotL9Lenh$0dug=nG$@-9*vR*YYsonSt*sPqLTqlw!gcO!=d=5}HjQs*7|th?!A zTgYE@*0NdkpZI7$e+HhDtT4sC@-~LY{Cp$g ziZcMbke;AI6h*)nB=B*ABsF)5KC0oddp?Q+DtQoP3=DRO;nEku*Tzpn4bJoB#i&cp zbvUu*nv#x6V_2Gg_vVtk4nzepK3+;{cFpMhD!ISAmx?I>_10Hm<$nGbTNV-6@EAcd zDMP`@qJH-FWO!BG`9f6X2QCsJ`nE!l)_x#PgY~xtLS;zR=R$e0-a3H-8INuedwj@G zd%1UlNyw(-`Byx*U8rY=XBmO-Ri3oaKommJ3J!S5nQwG>s;8oQ{MZW{K&NmA0d>)_d1d@JdC`acNr_!m3j&c9uBQVrvT*y)jO{YFGYasY3Ktxl|BOW zLGIW^AX3Rq#2J(M^ZnrP{W|++SXX3M_rrZm;f9 zBuMPQ0e<6fi%r&STVQ`EuSHrb9v}D(r>TVl@qi7T%6A643Nm1jD1y%$jEn8g?M8(; zL0cPeyGFIWl%LqE{AIP4o4m;V5u9PVzH`o2S-AWqwEOOw!&(iJwF+fJ-W2+9>K(c4 zbK2n{EFuHPuY3O&KJr3bz$r0~nPgyK@K8nHdXo~W!qQB z107Tfd~csBo{({_z483t4|4X^Au)6N9Kmod73TdY7d|niW_Su`?1S5Vwse|^R> za_-R*EK^8^#ESzQI6nxKFiCG{=;@md<1zS({%uN%hMX0<`+#Drt6WkMOYqm1n_cBn z$D;Mvi&Y!BZ(-JfQe|Iv{LbG{x&nVG5+??>ibDB2YB_)Z&G)vWyEC?*lU!dCGqTCg zOJP39Tn}Q#*$GDzuON>Ir=-R+&fV6-kmw1PA7{vXYMnu2H?y_MO5?H4=X;DG9-QxR z2ob%H3FOpb*0` zdG8S+Kb7)AMTP`6ZwrdAr1ST%ZJH_Mz;h@-o|E=ywVtzpKC9(NmQ&)u=LM5-VEB zzKhiW?8YCi-``*zgSY^$wP8Dg;Hq8ZP1>dykz4r?J7sa}uLryfVjHGcDZuJZ{XPeD z`q&s8h*l);`)vtpaGyaie#TLCGeFWoBzZ3gHVm~(kjA4HK%I`FKqK)C!eWxmo+ zDWGg?LOb24ly)PzU>S;{)aTE1$!)fs>d6e5^sfbE8=N$;jLY}K#5wM3YH)kw>2&ui zAYS|M5KC2n5_zzciFk5_Ry}ySPcB0?7sB7a!&lPN430nomCkOf_ySCcT zXXLuc+td@%qz1^|mkAQaLf=IA+m(7?q}1F>UoK2LS;8e>KfUIEOcNXeV=me;_0`t1 zM96j)ZJA@asNXd**R^n-M!>hIf_0Aim)V1jLV*8!0hf;7Z7qZ*tm`&|-)g_bWGLZl z9;_1@#5l-_Epssa3)q$%r`;aV>l4td(qxv{+Rm5b6yUb6eKnrqsE>rGabEtM=CPV} zRsH_QlBf|^xqn2I;8}`4vkZ}{N$~Yn>~XRB-RN%r>A`KpV?=f69#rbX<8cPD0<$*F z^rxC3Q*ed`jQ5@(ma@rPJ=<&ZyYk(Wm?I@aLv>W@+piB>2EyS$LI@CFPA9qjjmc4z z84?3OG1R48bO=;#DGF?%1nT%%ljhwG|4zg|n9{*PY{-H_7SfoW%EuweMh(-`m$A zt-7aISGB_n+0n=-S4kL)0zrMU)kvV>-$Y*xN#S8>NI^fZGSpacy-c*!*E7`o%C8yO zYG>wUV>)X;AN9CDK5r9B0CjfYt;E`}-7rJ0g;fJEKw$axh6z=q`vx{+xS#N*TX^sa zuBU|+zIZ*y7|dW%Ees|gSZlBgZ$JIs+)Y{`@Oe0}uTX2j3PNz8Cm?rYgBQIf2lO%U2wyQE*+n2J3JkiP zAgXubeC4R`{2wnAG_>#t#6<+7%qq#+KbE$N-%@gbnCHhA7j`|*h^y!^ik&XbJz#FS zZGHhE(FUVC=KVdKz9tY^j*myq33Nhff8=xc75s?|o>XnXiTj~5ZhB7+_Mtk{{{RTf z)CYUD0C{_K;+@E(CU`p#$^q^df#);d4mN|Y4Y z8dwdaI$#S9yr_08ffqZPkk`P|bjw_gH}Yeta7gU#m}Sfd%+|EFy|R7nQd-Yx(|^5L?( z#=oP3Mo~ZojZ(Ac>jPiPKLHv!m)t*A0WJiLE$Al;kJ?j%c`CbXk`SSW93_%W+zRAj zGk=;TtLUT=igZ8F2=NmF7moxVf1*>Q31k|2vqF+&5^ryu>}X#hA@1Uk%!WZz3GJ-sQ6xbT2S*v&XZNumX>r!mVTmd8b%>!HeNbtK#s(&bl3;SD6 zUGw37%v>eGl3$!5)%NV7<3gh~7galpzeYfDKfB$`JTduzeE*{3{^c-mO)~2XM>K(8 z(7Cr(=yXx|X8$K*3Yp|Yydd5zWy!X_^erjo8i+%r_bl4|wbC!gwT@e^zF4rbd z&6Ra?OG_<$*~x9gdIIjlRR(b!x#kbVqjhuv27d3EdE{eGEAhqnzER2;&!SEqd8olz zX)MM-`UaCf9Yf!pAyvSnky8|Ok&Rw5aVb1bh!~qe5PI%)yu~G59TLxQ-Xd6*Y6Ghg2z8 zSXj}h@+9V5S1!bpWkPRnZw76mXslBm5wk$)k3}p$(I_Fb zz~EpdexGhFu^4&lI5IH@tqvzB=j_U0_+9C3A6>7I*bO}=i+OnSZza7qxVjsN-=8-l zVI{x*eY?NL%GZDWg2R^!v&X3BPp6`|;0e!Ks1VY~#QAlOA?;$Q zg2_I*IACK#F*>Ci?OoIh`SqTPm>ykk4E((vze}f9Y(q*&=4WE zflxBZ+siEo7a|GJ+#bh#a^ORDPdDnGx2G|wZ+C8<4`&(0$UaXPp5i857U21$O~h#6 z5K@DSYx_`YMLP6u9=dE6^`ds4T*5&Y9uShmF5~jt-C@7G)=+}xuRz+JW#4^UOA8qS z|3ebn#z_l8zq!I@fW}pWZU^MzAvVfTjj$NB%GLp4;;P*#$SqK3p`m6{ChwZB!YaVEbhUgKyt*06>MAb(03qSOb~-wBOU84$~U>Q1Z2Hn3^TVT1^S z2hZT-R(jQPsr&ERc5{R6Er*jFJOC?Oh-X>;fmXbwl5q5`ob#7~hnryQ>Ck@{zX-_> zDv~_dDl4bQt3B}aEVHp{;J|9o_x=T4f1V-u#;aa+ z#AY&l9&Z=#_Ad9tH5!e@>VfG9L|E~@S&%A}%UJ5&9*$5&61Wr<}6 z|3wa8tC6;J)(&&opLobU*iZmxY{P+_!ri4p{|PQcTLT?jLHJBbcHp0UI`BEImb zjHU#Z*qbbXjJIZ(^|C8@rs%`;E`2Jy4Yvk+nXT1%+dYv;Ri^r-Y#(bXj~6Rm;I-?v%OOaTuf z)plFvJ~N`5!zsT*p$R;Z96!vy*_FY>LCUU(bVwFT^O?dxCiK9EN0`)0Pn)N$-ZIBh z-CuZ1oePP5V&C*9-g+cutc7v0u+s#L8J1k|2U-49uKNnlzWwW!2a1xo`qeSwhuhKzZp80dTD=26d@LL6rO=IO2&-K%oi<{mNCF(CBk@r+vb3e~TI zVOPCiSUGg#(t-|_)nG+a{AtdW65=s^`n5(W!9Eoa@^_*RZK<`LkX@AFpQ|T|n*+fW z$>XKn9zx=5Z^O^268?sgW;qr`Lnh~f*-wqAb%j| zN6mkxF3{)9I&2VvhgM>VqyoArW$v zUpzp?R=4ytSWA*?N15NcWd`brbULY+EqI1qTgFc1juLIyxK3rvo^$&Ycm(``+Ruz%0n7Pjd7k|<9cZVo7M<0LzJhaBB2mK% zGb!QTr^VK#%JmjEzq!L4Z=m->)I5oXYogsWSqW6De-?ZT zVhuj9U2j2Ft*Q%#$bj|CNS+{%51eick`~ePwJb z%De;|p>J7qdAP6-+4Fy0H2DuFfOhw08kkOd;APKEUCzxK;?3b2mb=5Q=m<8R__o8r zAdBbaZ45WCS6J8UneFT4be>Zf* zN4**p^f)8?QN1dv*^clV3cqD+f*UPdQn$?o(vZqauh7<*7!e7ndwzXlV2{v4jnj#~ z=QTZJU4K2VK&{o_QxB~2X6uC939lOt3{Dxz%-{npasJiJ?vQ7pY1$-x1(pls;Y?AF zm^l4Ps~5txdFzEf61b`fQX-rleF3h^84vKSiA9AutwZ#$iAQC<{F_*#L zhGLE-<1nT>FclJ;740)i1uS0AXxFt(jL#2xF8Q!ciZ=&&r*T@7&b#^#Fkge(JcgJ% zG?i-7<;~(XO4FMd^Qm)3+m7v?v)jSUIcLI$suH+b!yfkwDjWW*% z@+F4mRB}70K>$ilWet0CY)oN_C=r~9;ILj95@r!_C}eZk-^nzD?q+H_W<%uIA$eNmjrPMy5jCS_)c@+MWBb zBOqWiaTN3LT$_x=quNH}`VlPFZSoXjGifyE+S672_>I=_aKt+YeW@R)$cDiX8}s|? z3kXG;c;1d7Q%Re5A8alN>2Ib`jvyvXoje9Ndv8Ot`4fo_a8^bp)J&(2s2*%=l0UfZ zr8Il)!qX+v2OwE%RUR^>o%E@33cup*v<%|Gs`+`CXKp%;pnB=*If{h=(Xc#`Y z)F$1@_rpU6s*u6yT^}51p*ImJ+OFzC<;yovTKC$1Ufz1DUdE%)kcE{JS-E%W05u>s z<+ljPRltL~-tRi@fo4r8!`^$jU`ywyvdxV3(P{2Y#kQ;&J`@v2((drYyiKLj;t=Pw zS@n~A0(UlB&CKgnPxEu))A6`9-Z7o7p=1cr$@4UiG|1U!((1L1B>E=I_OZ{SK0F_v z5GYBrhtr3Tv)dyg^d$3$AT`}7O8bIH9yMJo6|u(8Tj1E1OjRzq)IodhW%L3q`e4dx z5M&1sPoAR?@N51_4Oq8~nBB z4dpK?x;+=9uT$-gnt!gl5PMYI6psYu)(e7W$!05*FunU1AN93=pMucHIZ9{}X40oo z&i{iUsVK1*`GbT!5XDkS>H7U?sg-Q7mh;xV?CH>0YycjPF{sB>tI>c_|C9G?EQYSK z8B;Y6iiDTg+Tf#~x`Kk_o7L;hV-yrKmvnkmay_ezN4)EeHp@}lPM?2BU2A*XrFC4h zt`CKtn5O`3Dgk5y})r@ zGN&mZ6Oe`Fi>czZYKYsEl%uNeKOA zvNxttY7+Q5)$C49^(7;sJLt+#Z=DEHsj;kw_+b|)4KEjq|+yj z4RjZSs4*LYpTl>L=6n^Vi4|mOv|9Gtk`=`*>6rEg1SXyRR0VS}AVayDuE8zft>{$NsMXoOrbj z;h-AO4CSYj`3PMx7cS_XOQO}Kyv6H@DWwYp2?v&J`;zRTo>rUX7_p5Uauj-W6w~on zAAxB8nST%yp94Up_<69_@L_1H`32Fd>`e0x7g9j?P$m>6uMAz0+hkiMj|Em;oZ{RXJFtVqw zjcd^sWPt&G+)m#(B`;VWy-wIcFv!m*3Rs{%VFaOQ)FoR?Cg6RQI7IYNU1Gso1KZ06 zQTq(wOE;y;W*S2>{m1JfB7|UtE44VNVA@N?qihryEa+LfdpYbDTq~9<45}O%3!IieIyMH~;cUr)f9>kXFRYxD*25s7Q+iAo^!CC$H zIC4vquN&1`xBWk{E+0D2_qR_Luic8jL1BE3MK+z#?Tyy!P$D*Ls?B#2sdPp{tTwJwsWzW8d%nIn zTdDrHc5ZV-Dw@eb0NgF;pp~CKu;K8XcT$BfZb{dp8il%lSuf5Yges0`)ED{I^NcDq#0+-W832^}$@SKU|!5k(giyzhgK@r|;MNtJjNu5H2 z>+8h3Bgw+M(wK3)s+?6RsKsVe-IOD0tIUC>1Hazxm`p5Psj2=sCg;J@A~tjL^T^j& zpTE8?aUkm3OcfI0P4n5kO=A-9dc(gL^a>j7i2k|lw^O0f^e^^pe=r#5OQin}ajMDx z5n)P)O*ZUFCdA=sMjlS2lO|gY+LvYS6qByr+4-Yg8yxtPILv9cWF*7BX7&DmQO9Q~e{iRg(DLtqsitas;R(a!hp4Z0arr)`bMr4-9 zogq*rOQ}uVm~9>sCZ-&kHfQLehnZ#7&ZP>=Pa*_pO-`8dSsjC1_b>)QOVFrQCTlnX zug9)JyAuxNgrZX`WVype`Zg3H-$lwCs~75d|G9ipv|4D!ocHvU1Z@$-3`JRQqDDrqQhbLWDxm(7(1(|I<_@f2X{|`yF&;p+}&M6Ah^2( zcXxMphXjJV6Wrb1-Cb_8_c`a@$NRc~9*o(myZ^5Gs;2K>*m)x~XPbYPZ457)Iw(j? zXQ%YWWTLV{;cd>d4*sY7Cv668tE<6eV*E!W?1(92dMt`{l9@BwIg53n34E)Xt*6Ha z{lV$6^({LLHL4SolQjsa%biV#xDg_fo28bAc0C$$hXA6_Uz5j_-zGQ5D{m0`lPHro zWH*8VuZ6JnMc{{L_(7{51+ANSWNGym$6c|Pft5fVy^k%6Ml9kL)KbJ{I0;*AIl0 z7mOByPj4(X2|E?!T*Vsm(~XW{D9OF+Jp89YF=Xy|1aa7BQ`zyZIA2~c)e23vz9l~3 z#?3ECWV2MN4kg~dBz-(rg8xj`WU0<82YwXj z-e_0pB{%%y0%Dvv{(h2D6wd$0enHax@$XxXjt}?#cpDZ3K)05VcuQOBhX9nf;7tYEqE~X{E z*Xgkw&iyupjDd?=>)vM55^7{D7syG+=8 zcgohL(Awm{Y!pXgz}vE4;o!YEepl!PZ(VLq9*X%l+mk0cXbwmh>K|Or+^?yw#nMkM zKI^2$_%|>eC*P5x1|>;9za-~32Tqk=Ds%9P%SZ!xO^i8t2Pbb?cTn0Cm}q)y!wfEO z+x2$4$N?9G^6EM8YW9<8)SvYhE-qJt$3(bq^K3;qqgVLhE&@+I z8XAp1BOr8!7n0%`%5}zk zy6sI`1S2_9IC}rldC}p)$Z@31VE+s|i$&pei>4w}8;%IFxc*Fqq|KG^YYI%>E1>vu z5=KwNGv^OjwK+U|bGa+8s3AXN1OxV+ru}*zk2($Wi_Ru!{T_)_Bk}qZ$q_R1t}|n##VgA3 z#oeE5cr)3ZPPskO4!3GmwF*c0U<#aq9M`|~(B(~B=gD>4W;CNo3Je5*i~oejofD7O zKAuJ(?Ufbe-2G;%+|Ig+$Clm(K`!k-^`x}Q-&aQmjPTt_6)DZ-BIY^~Gzr7*`Q*|3 zq!^}uFAZZTG4E7<`=4ek;%J95O@Hr)F`E@TOXg_3=@sYJ%=x93W$5c{C{JHQ0r3&1 zwvA%C!m{sW50|lpllg1xx^doZuK5$@NaaHE9RG&TBs?TE zbf=#g`@381(+A~4X8+jw_lBs0;RAxFX5be!#tH86XipvliFk8d!kmS-JJQG7eG}ye zd^bdf#lIQxgMCq+2|XZ6U}WUS=vKY=e5I#Ss9^{(GBpD+Z4575ol9;=*lm4r7by8Y z+|@JD4LZ3}yFXlNoaX%2eeFb%461$YN0HPW%xLh$E84AcNjnww`(FReK~yk-X$yhF zlvJrmB48MsX%c0R%dOEjq)>z52Z z7`uIL9;2?0`4Sn>ttN^lT?|L!sZ<7o4O4i6S;J6_A(99eQ(fdh`yC3tY{Idf&Y9J9 zf?|*IHw+_>O!b*H(?BYPU$C}~CW61tu8sM~fnj1ClHEe@!j@REatL%DH}-}Smh`II zF!>AdWT-ou&vv?KG`%}nf^eOT3C3!*sk`{>Z{ufr#c_YMw#Ub`T`&dToT`E_?B89R zhFtpCcd=IT2wqryG(o3RmD8k3Q)lEl_F{5u%lmL(z8i~ z+@A9J%PcvjY(Zdu#p*!qO49c5V{N_aT_!3dl{t)0mdf&bXn+39Q^?Wz&t(Zb9LUs$ zlrcx0(57pT*z&$__+AqahEt5q(8eUpW1YDSo0kZpo@3ad;Si(SkjXr9n_APj3Zz@wu-l?2pAZuV_kRS@((;92 zM~_V+i+;I(d`ba%rMF{si^?zvvtXZGO}dBL#*ool7Cl47tEljPu$Fs)%v(#=>!pt7 zt3Qy_PW+;2x*MHH;~CNT0}ZVuQYwj_>0*(!T%{x?z*8Dc;T5?Mf}lu0L#FUs3i7u8 zr~dxXn;uCzuT>4_lZbi~8raSrQ1|)GFO>1;hrQt@oGjg)QJ2NLAU}Wx`@k{RW4;Ap zdvK|ONXP_*Q7A12_+sk^8xk{5E_s(xL~}HhN!q4f>Z8#D^t{7AW#RJGtwx&Yie(Z! z1Fua>HsypQUg9?%cM!4Ae?l3SGBdI@#6q~Zx^5O`5v~&Xoi7}p^}iGqDtmqMAv@k2 zHDoMS%R?7MaZ8EyqzHT->E$iG7ehI+UT+u(`hgPG^&ze1v=i3%W50BBmLl#D0!R8S z*NuJ}@KzW-4!g{Hsu9yO}g&+R`f)XRA$p zt9j>e(?e`dCmzQ4P@WNPc zG}Mb&Zh=1A6OM?(3r#7PC9?Ew#DwM`^42nSYtFE}dt`>#KE>}OOBgso(jEAj46o%v zJu1S}eQfFei?&LA+p-Q~p>8un&ikS=bg_FPvBd0N7geNW{oFB;8LrMZ48m-y(}zkP z>Fv%TecG&R8BZepbC%12Y1`g69f-$Me~&R?=p8nP>DxgX6^{uR*;;J<4{I!P#>TfR z!0U!MD&)2M;6c5FAE)vXrl8NWi$aE@l_^%i=4Y_DtntsB#I%QE#Ln%{bzK_;gt#NS zzQrv3sRKWhCuo04Pzc?$yS)kMqX!dJwp9F`OtOos%BJc3P8WDDlfn=>A<34E`nz>P zc`!%e711pA6_ZXa5NFu!96kdbVSC7r@0&HhD?HB%SACvARw(kYsy?C&CQrFOB^I4QMGpy;43JlpO__d8~ns<$z zsE53w{EwD9W)^C#$vEGwTkbj$x8hJ`izFWu7ZRP(08L~)BPLt*S}x?!FVpGlf>Eab zqEIPIbv$JGw>#7|zeF?OrPJC5EBa5{f|go6dX|m(-qn+fFz;Y{d7EtF#S?WlCfXGE zn1cm;#!IB$J!=rF6Hk46g}Rh+llu2w4dSX=DF_^4AHj{Epfob0o1AR}>qz`axk6c1 zmA62-wy4xA@vt6EwU7=!Lthb*U_^mkMzn;2Op1(HWLo%}$MsjOT(&lg zYXHE~h3uT`Wm=1q4r z;r)3+CJ6bEO3^Fpj)@_L)NMS4Ul{Bc6s0$NY#l-9V~2dpV+#nFYzq03PA-0hF>;oyle^)2i@=iP|x5F`ph_M&r9p+C)E?ht3 zEVU_2)3QmhcV9J4;AU0c(s8CTu;ZVFUnRXo=P68ykK~wzi7Qd@=4E!0<$Ve!i3ZumKp)&|pN6fp^2 z{@rtr`z+ZGzs_?+b&J}eDgDhrKRU|)8RFw!ML(EFQn${`U_$us_Oa|3MC|20aY{BL z$yW4;j6W)GDNGYJ9p<-fo;*!sPsSGY+>x;9JS`{XrhiM}f?qzj9JNrshFLs)4x)TD zuEV^CBq0joEaH4w9l}7nGT6*=qKxPayWZ3u8}_)gcQWD22p1f#{%E%UouYs`zQ@y# z<|jEsY@$4xfZ3=%p7AONW!)Jl>XU8aAL2c}9gc_ULVd$LKG$de7%}^dSebM>qxMKk zV>zY_Er5s{wTX_pA#!_zw_N!Y}r_B*e?WW+I zuFow=oFZTa^LBdv2U7jV0vO;2Fqsa>IbQ5JgIH zecqqKwGafdb9Mg$U8rxc!VD+BL)ROc8d7UI2E9w&pGsYeB;87%?~JK!WeT{JQuZd2 zMN}&3!xLZtu9%~OFR7b(?aw)b61%u~DSGa11$sANZ4p4f|G%u9>$=|NPwxw|i3z5% zNUP8SYxb=yeCCKv_bF2I%iEDImN(9FBA>O@bKNtFhT+0$0~VOv*fuO8yt{#aolTBl zv1%D-+mkJC8O*8y|Lx0gU>$%3k~p2Fn=EI44gL7lTRc&W*WnS*^Z5Z&LmzIfA?&G# ze%G<4CjwX+{>?TjF1L3praidBQVG1?eqhttL7wm_~_bAyce-j$TVZ zcWS*)6z2v+M78Z{Wo3=YdxyMX_|3PQQ)BJS67rgjJU~IELdnsG6c}L(6%`OQ3#{T8 z3=|ZpSKD%bItM8@1o%ehRza*&5}Z$|v&3HgcMpa~H}PT+OG8uRB;*EzP%)%+%_46~ z7`E7O0NE?-ZGi{1B3TeZUwjD(8Cg$@xu>gXS|qJj4Fk^f?+_>6`*6c3%+#wQX}o|Y zmNF9URqpe~1M_5D72%doE<;q%8XB?KBSF0g8_UZp%j;ddqz^pdc5=NBWHPBe$x`W8 z7dB38&LxDJWeoO)p9E6yr%ND;Y&Lt2Hq%#}e;-o%&f$I*<<1?ZOKryOtERIh1}vUv z)Gr-$tL|tmF>#mB(!EWy68G7~rfy{Qlp7IcL4i*DTLCuH5atA^qa4_TUCATESz+M> z>^#{~E6@6}vz~RZ+!C-hzfIPfIBt*myw|85$__XZDwP~>ku87ZYL*#^XYu0S5^{#e z{{b@%O2K%?wO zWOwkF?qlZg!a1fT%EVAO(BN@7;8?fcfspZeL#;9vN!Z5VC=|-2m^ub$j*w!KT(x`b zx5twdYQzPGA6|{0>+}jrHYwZv0h-Fzi!r^~2CPUnlkw`+Y*fQwFqvSW7%bW`Pq}1s ze<}|{GFlhNp!Hp<9G!23j}4G*|DBKV8{uZNT!`duw98?O`cZF-&{Cl(HYIwqG3LSs z5YME2ZGRv9^Q@yrnYlg+6V<<+U+E1$SP}5SWD6Z7nq28I8AxY+ZoVVz3CeQ!*`k!o zRiBZ>;#?;;7)kuZW_7=wcIMRuFtYH5Z3NX0sZ3RLj*zEwK?e=auP+8^wDLssO?B1* zqBrIj6k|HU?d&o`@i@&V52{6fX9;vatA8mn0>mQvoA`rR|`xqmVNZ-`)Mqks*t4uG)o3|<5^mJi^J z`7*}&8aG$~B0Gw|sB;`le13D+x9EVY@RCvl+VdjRn``rn?_C)NQHnd$Pgsl>3s@?b zJu}yW6PR2G3QbpBeF?>+4*voe1cpn3?3;3oxd?Nylz1N_ZueCahHKyZb=0BA8xA-IF>I5Ag!AvxOwbO8shT0Y4UJOK*faQ4 zp;w@45MuTRB}B{S+oJX!*}MZF#zD`5NsKk#OsjZ~!}-96{BbyG?*41PfzFRjfp>DYO$Dq;;qRw1i7KnvhOdrsq}rEDXQBaC7A7rd zH{}$~2|~a7#tC1iB7Wn_mEMuqL&r+eauAv=oN4qQlyXFcZ2O%dV#gk8)z$n^9xB0h z;909>FP5+^e$Cj0izum_>9O%wnw)#coRt4F5a}2e7JeM=n#(d8vsLuRH!P?ICx^I3 zV6Y|!U|?Dj?ZWe!eT22Or|~kLr1ze;(u1y=J?epnww@UO*q}pc@t%U@Z?!!5NBPA> zd_F36ERHH;k;8auiTx?v#igyYLn5S%v+L(DHB=uvBwK*^f~Qf!a>;~T0W2M0U74i4 zcBQe4Kl4R|#cwjr{EnD8V#OvYzN(kcZIejz0DHvjr=Wj&&ecvz<0a=V{xqC5*HV;M zLx76(%vbD{^+m2k^hcrK1siE zx(p?1zI%h0+rt~*Df3*Jt&L&M=^&caOam?BAMvC?5@)`N718B*nYiPD!GC*JhNd?1Wg!g%LBosZv* zY4LfNq1{_>Y%rG#PPnQ}Tr`!|$J=9niW_qhsoCM}P5O;ALd;2udjGomS*+=zvK05( zVeIpz=QpN0J`~Qf=jS2DVeJ+~9=BSNYKj|>*94>cZM*e&gDsuC#Z)+o(}(v68SG;; z7>|5n9VELu0#dj%vc^ueIR4BL2$pSnw4&_<3fG{xmJ|_t#-(=Y5M2qM-~Bi~o29c(CJ*5xxP(8&zxVQnk%o z`0L2{9FlrYE5B%CGlSY#2~@?OfhbBi1=5i|6Zs-$8XKA=i+M!w<`{z4Uxb<*l=a0b z&ns3S^uRs*Vg@~!L>CfeYpB;h`V5M z(OveBdK@h7i$jzfRYkj9YsX7G%-z}UjJKwXDrYiD^(USm9-U3sZ7g;j(%Bt)@3ECj zzY{#z?fBo_;EHMuiC>wnr!X0{F_uXdU+l7k#)!-Q4stJD22y*e@OPT&}izsL`*0F?s6vocZTv&M(oM#R}z- z1fqIP!0Z>&s5Tr652^9`M0QtR_zFaJA{Me*B1T&S6_=tv2AJq8yM;n(pv)D}61e{X zrr=XNwTX5o%o@{)d7Edzhd0CW-V{N=k<~;gAoC*$xXh#4^2`_+qNVI@e+Gq&p2;CK=kLJ}?P z?wi^s6cib~Fz-VUKe?0c6g}#l!BtmikcHPiIZ+{><9;Wjhz1An^c|N8=6?>NM0%;^ z1jkaU_ZsRNvG!U#U{E#+PESnS2t%yBG`wk(S`WrmQGgIZckbFZhD%+P!|7jvZ)q$M z;iE}ZTEAdN@L2x%Mh3aLl*793A*TjmzTg~ULH{kev6t-+Fi$O~XJ%f8WoJyAE>)N@ z;e1yg+U0S%qT3KJnW9UwWy%WsGf0!rzKcd-^_cVcV28JzBb+mtD>?fMl^p`RgYdn= zvR@xXO>oO)@kITuDI01m(hQ?$!NwPs*QH6w`Wi)(K$Gl2IQk5al^VNHMNs7d z(;j({|4@@!5t{_(>^R?tY>d2nT2qWiBz(UIsYG+yr*nSO#T|9txAccL+oP}3R+9u%M((C&GxFtq&848I%>H!fhw)asW|JH-^)T(ES6RzW zvaNo$*BjM~gKO$y%`)iSSkS}I(YtGx?@_jKYfN@?EH#D(U4-UrI{kkt1Btv|PND#m zoD3>F=Ni91;7X&~AatCTfdO_S-4C65BV2LlAZ3qqW=<t#|ME~_aK|eP zOO{jPk~OalgttoL%CT?8NVC2KCFRK$-O@3{n_aUU)$1$_)?$f=s8=?nW)f7bY=5f< zsKY(n*&J0XQP+jZ?$7n>2_ykIWj|f1mFp<93J1&Iq%ww)v~yfB&}TQlv%X1b0Pc3_ z(^W6V?808;%9G4|*f7eC_I@}o510IjD&|qi=;7NS`_*`qYb~KGP)|wl4ef-Zy09zi zwq^ojmF>3oS-HKH$CI5#g$fsGCi5i-v^#P)z;VV}7^Z5XH7=-Fp=Las!jZs>@X%QN ztJ7O`twU)?N`G`uDWXuP-yGwjGrqF-!I-AohWGheDv&aPEfv=GMt?PZ5XP~DbdboE z8GuO6bzsHeRr=?Fq#ELc+n;K}x4cgn^hW%=oTD`@RTo?-&ZgTng;YJVmFKbNhDS-C z6>fiKlqIF&UH9}3jfuVTa;lm~q|*+@Q4u68E{ z!tk4uhMZ(q)xG1jQeo7DRFu3pVspvF9?)h?&%+)${a`G1CoRT54TgFhZhR+{laDqz z)RP$Vhxo4qQRl?@^nGn_O=SU_p9^x7VAXLheUkxZ*+II+b3&*wBGub^m#!C4$UaHP6THL^tc~X|ZN_#(017p&~Xc24TvXB$KGR{0U}?jku*F zmlyo5sQQ{4e)90kdVyPhvO^aEemYxx5O3e#MaV9d zjJ-s5-QY~4B^=x;8cT)5T�GpebD{mMv;pi()$kXxfO3(dCHc3S7lHc9-png+DV3 z5w=}FnspJje$tCs|B#DPUx1@m_>tX3y0d?-uj zU3AxwMCwJk!q4GY;i_Xal_cj`4lw#SwtlS{JfI=|IL>~lXSLeE3mpS@I91Kl-i$lC z!jbvUkW4@U?u7N=wO01%n*73HL`FeTt`4Ih_B=36XuxYrh->#dlp~p>nBXrlJ`R~FY-XGm5fMipYPad$R*=}YsTtQDj1>&cCG%K7j55~EENqw1TdQ&W#p86H z6bG|we@6ZBZKMcA;9DN~{pQ;)HnQ4YLsM*6)Z~Z#Te;m$?~CyXjQiIj5q7)NmK5vy zNIC^&S%LWTj1Vk5?zHR$oP=9@l>=Q`k8O?H8xu*JZ3~9f17fKI1A1W$^d& z;z7N_i?)T5?slQXIu(YsM(d$v{s{z3Bm>WW-#bflXBeh9_oTo2O#WSRL>MGWg2(QT zn1qd-)qS$u zovgLH#Orj5PTVN7l*8BRk3yWh;pd4s1J|+{Vcq{6Pp=AY91_9|4c{D7!oRolU-0v@ zPf!3qUZf8l5}1QzGm7~i_9x;wVwM(kvwk%ZIKT7(uNoP%p88fOp@xy z2G?d`Gbu|o5smjeT~QQNbfCK0Gs(3-aN}bT{8gq!m#}$U7vNJ4_{z2n#y?LoO-wBMhK3-0IPA2r!xMM;fUrchI&jrm( zgm-Hb*5L*3|v(G z`-hpwQgy2T^3VSs1~qsF#88em@tN_TKR<6E%`SDN*~X}*#!jZiw|Xn>%ILCJsasa4 z5SoD`O$O&~9GJ(h=G!x#fmV>v`m_+kQ~~Y^EE*Zb`{CnrXa5s)m`AlCuB2BgWl!Z+ z9WyvluBsS6XTamMhXmqyCWGG1pdZE-GDljSz7G?Z4}sFEucVi|10hufQhLCQmWG&h zmzjs&0kpdd120f16IVo2NoSzpj@G_~Gk_|=23(8}KhO?sUk8qt-#L?cG@E>Qdi7!= z0Mre-l5WKpkkFdpPvCUFs>^t%GHlm7a+)P!xSkY26)nFgC-bDZW^vT~zRBR`iAK;k zdp#CaGJ;9IWw5D!y*`_Pn&N!`|50~*k1OpB`RiU*Xtzc1$qKE3(1$(Q`w6Ay$3IKg zu@&h}lBO)GaZtDRsssJ4VyWYLiCQhTnTfN;s#j1SK`qCg&r0DiDtm!{vVAzKbSEd3 zh!MBAF1SwSVI9Mr`AF(YQ%zZ@uiMEYl17_5 zlvor&cN0?D2-X6lG3yQgVvR-UPT20|dL+yDx^u4SikDwPyiryDp>Tjbfh^j_*rAMg zqtp2ZPCrop>Cuq=j|U5H7U&bO=bG@Ny(#E=N2|D>8NhT;N77gg;I7VWyRv!!6tzEIi0a{{W3^oJN$04-#qdoW+E%_w`t?20fmh>?#a|qt1kbnhZ5wzH zkY~D-f-4ru{;k>sD~!MIsV@W)jGt_M)ZtU!Z$@Nf;J{atm6M3r3!SU(-HowEjc(OC z335TGw0yF0E0Ae+t}fQUUgp}@csiN^)U$LCdTefIiNbMQnWHi|cq`k|WiFbcu) zu|%n$$A?r5Z2*1CM?m2IVo!wPk-tZy(C?QT6DdA+R6KPn!$SlbNKOog@dKd7i>zg8 zGI(B>6+%9SOd0>N#N69e{;}*3_@ajv%=dn|Z(2KZ#r}KWYJe5@ZFiZ%Y^T!>NVvXC z%FF)vEMZCj3lE>eOyG;Rz|`Hnf>d8j9W9(iqXMt7;zl>i?zICBi-aTsfHXEwvzB&R zM~U>=8Y!{!UBM8h1m!MIL}GQQc--zEfppp5khmiGmhZaE=F=aRoKF5;Asyv-)O+?; z{xRKheXpp|I$eRVY`$h44MxEPTS^mIKYQ=E(NR(MA{2DXe_uCtVjX{TM({`#X|Ft9 zatX=Z^+LE$?uI(_;rJZ>6A)<=g(^yDl6Z_w56sQi;KMhQawj^CDqw!(gYGJ zC`V~2Ft%LscU{O%OBu-vmJt++6gU#GZ4)s}4 z9Syt}zP$04DbGCqwz)ZEVH|hjdD(Z&z873)Jz4xu;@&y%xRjGwF+dbOik5lG=kQlR8b{dksFWC(wLiPz7rf(WGMVTh85 z;f5-37SsB$aNX_Fq6kfyr=){bjnULVmY1jV6Y?Ggi-HVW2|kS*EM}uWlLeOrPT-t; zYh4t2vtkl!jnAnWh+v+fvb#AiY2pLr9l-(g4M+K6w*fuOaIMaN3uwpcQwe6*qfr+( z8@}_v`}ToWQt*UB_~NZK26zd!A00CDeUkS~I)N`L8^T~~thmPfz-+OXqTXvl-ya-`eJSL{8Vc@Zr6iL8eRt)DR-IBDOnsS)b) z^-8`vk(#!XbH6Kz7hE8;x{vX%Ts4+~s?gUh=I5b1l6esLPQ9|3fmxVzq?HBVI*QjG zqA%LXOnCYw5p&}+jZ~FUs;0;dapb2Ogi6PLaIZ3Fmd$U#mY!vn%P0D`5HK>io4gdB zh&Cn8{ZVR4+Wu{y>x6N+B#KV2T}P&6qA@qI$K%Du#BI8{M964#pWg#VT4~-kYP2ge zlXHtiIEfH(INOnHSaxwaJ|VubV^~87FA<7Ny2*XbL4G$=@dH1BCce|tl{AanAs#w5t;`YQ&%THq2pwzpq=_MG*qta@TRx+|2?>0p#X#cg*< z^`7nex&r5#0flX>CVhAa*JMXlBo`*VrV*^1qtdMsvEX12dC2qf141F;!og32u|Rv% zDXd=pJ*_7Dzo_oUv*oB(aZ{17TCd!V(Lk6_KNGfa@MXC1N@22%>LTMa+D1S5*S+kK zCCAP=`VWBEC|23^fEaT+dm$q2@}^{UXAD1)jwm8&s+>^Y|HovN5}Q7qPn#fKk3HsR zi1A2z4E8AeI%9RH%f&npMV@pKCzfT5@ZTI>K==}hZh9QOm>6n10E8%K#kA}iF1{h_ zvd3X%UYkw<&VsZ})SyE7RfSfo)#GhGE3wxr-nnE<7dwl{xc3^#-P6h=;v34!HGt!)Z#k zK0UA0l23p`XqY;tXOxxfo#mw#SPz807>o~PFL`4*BNdlG*)%O^_@^?)j}9U}%2X-j=ls@ug%6h%P!a;hKc6d4|S zfWDNC8HG5I%QD0ESlW_B#j`TU@Uso$qZ00TrKMfVy!uNm=pw^Dz_=${>+R#P9T2a~>rgycb| zkx`Dq(P=%x0A2ZZuHT-LE0^lc^l1K=zI6(Tfvg6Dn1}F~5tFa*fEo7Rr%edv`y|_3 zup;K>QXj!2Q+}*A%ph4;Y1wC$PpnVmaN7nERC38{mHa_svXjg{c6`d#3psEro!5>) z47-d=&&JyBHTC9)&HX;y_2DeC3XKXw-IcZ1GgMySjgWxkG7xP8?yRfMp>@x!bZJ;i zty~#T(m4lb`dw>|4A2X{`qqEfh|4j>%Fkhy^R@pl83IBQqe?5Aef@X;bnA zp|13LqE`{M7yGZmvSq?3j9!~ONAfuIlX0g@I{hj76v}#;0>J>?0UQ80!ejr!SL#U8 zeNn4}$DS-=k&foeDE$0dhaRw>rSQvyxP|Y4h1v27odwGWRKDz41KShXk+_|CA0y!m z40Fq2{Iu=-+#|kHb^p0pnj?#+eu100M>b_4Vu18)dli>ZX|PxV^ARY#F3bilm#O(* z89UrwIoq6fMSr97PemzP0lEiNboW)G>DV$j>-ddY$wk0l5z(8iEYySh$lPMS#_ytS z4CuNsjBxsccQP$l4DD5^CX~O;&KP*M;M89{Z?0Nl;f}o6I{&duTxTWS6MFa8(_6zh zliFxN)fDZIl*z!anw6>2GGuk9^I@9D^(xh&ByWhW(x7 zTMwDOSGxK&uKwn_&6WoX8X3-bNkKGGh|!#M499w6P02@0X7ONToUF*vIw_s6-9sf? zFfQWIp^KtU=8H-x9)^_6INaT200EmFZafD_T?ae%RmK(B`VA;=CsTU`(5}u^>hJNU zLUNDXGs=kTR~o+&mmxL08Q1?f72rqg;AZT&nELpmd)HnsBun6g>J1g_MI8hnZp___ zi45<}3q5BBB&1d`=P=0vOvaKyz{X;!jM=3R8DZ2sBJ0z_JqfKt&XAX9luYfp#?KP@ zdDEw7gyOy_;pi4~#gA+8$E4liFkZXEkym5d4Z}V7Vi8##^>&y#Q;8>Ji7?4T`M;Xj zUO(UrvG>KmB4;lMyh2H`&@c(8)0x=PF>3fohgS+B+9F`5WNAF~&VUc}--o5>#>Uhs zSKb}Y_TE#QWR-(lLbtzg3kLDB2E4dTN-&Z!`_Fy!u9}9GcF|An2E{z>>|@%&PlAgx z@(s7_UV>0DqB-=T8QDCcX>V>W+_uoLcbbVd%kfaPJ-MQA;2$x@xfPOeAOpm#az%6b zSGjwbei!Ot1zl92FhcSX5hV`?A#4p*>sKUyu=5AvZqD4O=Q{nrLht&s`s(@!!NwSZ z=o9P;OOh)(>-+@(6pZMVfDEm`yMy0ip%F8IHbLxQQ^X|7$_JWvKDLB<3vR0pNhqI~j@e4&G~&PPHTh7DRufv86P(Vv^KZ}+Y<*(iJN^MxzNB$r8kAss5S*>MR|_6b5=W*hIZM(3^UX^*k=%+7B;@Nj|?m(pVdbX+*BjHFHX zQKxbRw(*8U20~pZ&av8W%+;*f_P z}ih8n8C&-+R6j;UpGHZW3e9}F;f3huZBw~#wLmo zZfj|zDsUr_lNJ3mF3YPwi^m9d#~`ktEwb+BWEqtmts^y z(am-7E#pA&Q(-ZMk?3#xfm|%kKWvPQ*50WDY8_4>2D5?wVl2@f+tkX7r^&WNT2!WfUFm~~8U=h&&5sO`zM<@}If;TYvhj~RgX2Dv zVe0lC~iN&r4jyB8~{=ueh$o~AvXqDv4q{G{%G)^G-ep}mRakC5O_l=V%(IFSDvM<6BHvHi8d_KkAV zhA+MN_OdM&%oIO^<3$BzpyYao-LwA6o$jt@>@7M0iAmz@aIZv=4Au2XLkPalcXJr! zk~#QQ`#RE{;QAvaGVkeWmwiwL(CVg^n}07A{()WW;3c>6#}Ee{A-N?AXVq`Y&P2L2 z+7CmniHvoYXzV)}!TI*|FipT#?x~8ITFOCE1({r`j*>`U!(I5S@ixC|xA}rA7wiqj zAk7`vWq*>*9Ygn~Z7_^0o-TMp{dy5}N z&O*BPl~Hh$W-$J1WO|R%1j1!KyX?&ywVB4VX{G{MV7Yb*ZpC;Ie_>MV6H2Q4;f13T z;QGUXyUr4*oOmY>_-YNrZp3gwOJV>CF-NndO$E0HanFReZ$(IrnXR7t->*efb2@Z? zJQD#N0F)EbM;MIpSm}-%XG8GeU*f|FuwtUd63v^Z1%;pKH;3mbZ`$zA8-|1-*|9%; z+K(R8^yi2^wCHX>$D%`5Y&GAAGh0qD0yjlfMif?q7MWJD5e^HERPnUNFLMbtk<_F& z?rE+<>GhVE@FHO^YMv+ejPbhvq8s|(E&u_>2t$h8^KMus6pjy}U<_@;>h;dQdSf`H z6;NcayaUH?hBl~u=gK(P`^H;Xw2LSN3sFGahjScqB_f8q^7FaFB!k537L!KBDCC@q zhx$A0m|G-!0ypI6*J|uX?&O4%)|<^RcfU`C0vZlen(7adj~A-a zjCWQ&ZlAWo-ns76`o<)<;2FGiGK~vf8Q$?&`x)haF&SYFBc>8diH=iCPFZ2)H!|Hm z3sIH+HW>vHbS64q*|e5OCMYqzbRvq*1bSPX!>d(}9`MuYg+m&&`P(|CarL z@mlDu-D8{(;ow(4I5$_kN@pre6Rw9P*ku{rJXyO{BCqBoFpP;jXUdnC6}wH&ujMDBE^kXOqn__o&cq}{Nc`1>5kS1hAR&zef5dQ$qD{#6D@+xi zmuJMTcS)5wW4jp8&O9}pIGcZ>HL|TOIsHj`Pg4cKJw26n2&cKxm60RaT3KaMjE(E} z8O0pzn#$w-8RNGoE77_=f8lk{nAGYJnUlomxElTUT+SOEWPWGqk7`cGlf|&Ft_aCCdWRdom1dy~C1 zr2g~*s=M)-_)>k<$G(9I+Z!TZ*p@Psq672Y7trt5Gi-~ml^;wja~@x-U0%~FPa31> z!7I(cUwtP$-79)Whu<9!aMT9NF29}jxUk4d7-RwgpdIMkE6d9u{zFBCOI%^~Jh{um zzJXq{E#)rGLZcgBp>&4NkZP+!jgBWG{xrhx6h}9Nsn|0vo8WNoP%DmMD8;ZL4XQO# z5YcFKt%)}As7Rr}o|7uYXe_HhU#et=u%;(F=xG8a=LM&GYAWMZMz^;@beF}_$xK6r z@OmkRsU|mEaQm(Mp$E9#mJY5BkReR5Gv7(1sxmTxm^b40twk< z5R{E9jy_!CkaUqo9DrbyYL6OhQlGbR3tSqe;&BG;gdvtSR%S9R<=Ee4ztOZSxOPXk z_b%`beP8eCXIbAbSsAK9*J4PeSK0)e@N?yB-nq_;PDfIr>q4Y?Exf0{pNz^Kq~Lzz z%v6|8`n)x)tM1mai*y}@0(7U&>X_7cqb369Qd5d#O2&5^X}Cs4hVES`Ba94|ryp^8 zZv0tR;mm`^kMyH9J3ze{Q0JEb1mCL$e&>)FXT6-g{m~!uygpbM*uF*8RV5pnp`AOQ z7^bgucp`XY@XzLBO6FC)Q&T#bN;(*CProAdd~o1^zB%ix+4xTpzr#ki`jy%@BJMX`ZH{)Nmz z9YvK=E~-WU1^Hl&0~_cFKW~yi(cO08O`}hLEk2CR_ZHnW>gEuw1Pjb zX2?2`vawK(yFh{y#)rV_+TI+Kz3TjcvPW?1qie*tXHwW@9IfZL>jR z+qS;xx%WHw?2!r>#Y!v?F(X;OV&rii&n|BNo*Ng zLbG`44j4u3D(k!+=0Peummox$R#geTQn2@moMwp@-#;5k~g!69C2EPa&1O7SYI_ba^8gOB~f?YfZ=jr)H2f zcujyiF!3arc40S8>6Qhk4*F~3z029TkG{pLsp+KdPTR%g^zebkyYt;BnHYvXf!%I1~@DtyPP!jN-_(k;z z9}RYiOWy2NuKf&D>`FBn*&RFz;(|-wTae8skcs$d-5n1Sxndi=29EDqZ@Y>n;%aLg zYBHZ|rd}*)uK=A5SYR(J^2k8yM<64F!mfn9qhlN{O1Oqm@Vp{TO)6F!Y7HF(j;rwL z`^>~K0$4RX*lM`P=_70!w@K^m%Of}guAuiHe%@MDM@D2&{qZo{`By>rWvdBd2izs@ ze9tdRE1dzmYU`41S)6hf^A1M`#SM96zhYg?W@S&}r?GwTq`Ff>gF5fYsXy;u5W{@W zlJ$x`j7X9Nzf6-F4A`Wj>MOD&wg34!oeP}EgCn1>SKAk9q$yQ9>Klr=f3xF|`hq>c zTc!!ODqzSnSy{30+!-Wf$T$ak_`K_}In(`dT-RlPlFVxL=PkEw_jEue9gLqFq6HmA ztlLsKPn!B=p<0K~cb=^455DC49%wntf;Z84CLhS*i>Wn+G)d!?RjJ3wzGhUCxECIm zhl*Br{ATFw$%dCuxL%-nEdgkosGmJ_|F!KJ{S?XNGKpV@q@A>U77KQfDfUx=E0^3H z2XyC&r+$0R{Rzzum0W0-&o3=Vn~kmhGe^pE3&W>#Z|eCso6uzHQXSpF4-Q0xdCeQE z>Go{&?`~N1M&eWKF#Y*z=Q}%eX{<%_^cg9n!X-sSVqU3fiB5+pUBM%T@@b+nL2PYi zZJcuR6(0sUQ;M}tRNA0tg_t<>H~z7(FHbK9G{~OD)}_<6oh5K?iF;q%%dsNHQmRWY zujZS&hDSM0RC7x;^W*y;ETJxFQ-eHgo>oeL^O@^^aGIS|C;-`I`lBpGiL^Ya5 zWlX^OWpkqjLx7AHHGqOOdnObl%r4sPYX&un*cyR^;VM*9k$x=t=fr%Rw2C;jx$&)*%1zSK7M~#<2WAs|!(+aGt)wW|Vf< z?@RTQQZdbsH}h)+wbAavG+?Qz;`|u=0uhOVg5RZDM+uAq9O&1^&xDch9c|@L-h-LG zb7Y%4p6VQKMKeB$Y2r=%aBu)gJptJRIayF(W1-~n7ph1wJSxuPOOCJ6fnW7>LpOett0Gp zxr0!F53h<8PKOU&7Fls=H(2B&N`{Fbwxm6aM9oMI^4wV9V738JmI0d%ghSX%s9dC* z)zoIpN;5nfXlB=@1Y*pCCe|^)&T9t;{k3}!G$o8_6$`Y+4B^?K*(>rah9QGvdUK=@ zMwfuaBcMQ-zHhI5VE{V-b)k0+yPziv`d~{d_fiFykD1eYou&>2dNag2sWP2Ql;AUe zOBnYLRHMOiW-P)#`);SgBud19lUXMIJlyi>D;D9utY@6JDrZ&@lQ6mAvKkJZi2-agC1k-criT5%BH z#5V#dk(<>!4}$Sfn0mb2c`5lWdzLQZU!m0dOks5@5pwDuzdW~?W))fsou+tb1+t2b z&8Qcb<4Xvg{xcaSdSVSG?@(fbPbnbC0kJ`YTmAV#Q=Y(9wX(^gKygvu8|C@E!3Sol z538Hx%jA5SUhmUy-d=}|*OY@a(zquOv|MgnaGJnCYv->h93Wfdk~ogtl_3=(q6R^= z@Ulsh&ou4H-_UuueB^u4_EBukyTuLnL^ox#UD+DIv$=k<6l3=-gR)SWdEh}HfI<12 zegQ$4979jyKyBv)U%9#`<}Z0Ukob5n{<43u)I{9lU9t`v^1vPyxlTAxT_U9yfHhUX zi={$E$SR=z6^3GmQmjz5+rqd)DWhm{0F zn((vW^mS#e=G~mYs7Ap^(M^PdHu9HtyDFuFA4{pe6Cs9HfPy*8v9laSKFXxYYeX8q_WJtdwwD z=tj+iN^6C@UttX#n$mvjA)%A^nH-(ZQ%isDl&btNuf$Z)`er5gQL=f*50SbUxe%fK z4frV=Yl+j{?M1K#u9?J!r2A`2eHk0!WqDcH(5AqwVKm4N<4;bX(4r1Jh*LPwFA#1V zdLg*;Bq$juW~V0~*U5pqg15gj?l**Ulc)N+fa=8Nm5Y)H-qc(jhM7x+{OKNf0S-{4LYvG@yQ3Vo+Wmj8Qa{tBY#R@UeQF8c&3nR12y2P|6;X zR>oQ+Jau;A+}F|THRwLP9A^`IVy77aYsud^qmL3M4%6;I3Zl>Y;o@MnW3H5+!2pp~ zol2Cz;G0dL{7Vuv>Q$%F(EGEN?CxdAOZRsszcG;SUR(u_#rnR z7hnj`a@nkZ=BUz&XR>A7eZ3de-F!;q&2xCP0e`N4 zqu$3_3ZOF&5@>@*9Z@SgJk4NEz)r%4{~T2xbTa>BzuoWfa&L``=%KFY6)4+|6BN5l z{Tv6jFe`0GxKORtrj*R6M@OT?TQ5`G^hu&ocGzm4UT|2X(``U&>6dd#5&T4q`=7yf zR#WMi`kx{Wmz$_h@JX#^NBU2bSWzc==kB!je%m=oBwisM7R8-RLFk&}7 z|IXnLZ2uvhH()x(xW0Bd}{w*yF9U>^!f!XiUJ0 zI<{azwp=WG7%ef$g~8#Gado&KpMMiPHsI^?R)m*L9T5E42&jpNgQVElH#lQv2~jyr z-6l^p3*BFA;`vO~p6rp z2aplC8g~y7OO6;ql20v&&i~pS>DXH#mKa;exl8JOaxCQwTBuDD)${!T!BATI=yxh@ zq#G~)XIlP!KeHo2G!~2|ssi7BCq~EU+<_s-PVx49ogyQw0CU;C#bsV;!(kz^@G}*m znQW&8i6nYx!l?dAY$7&Q?%ttYYIwaAn`3-D4q0}l*Wn^owqOfh`PK*r0%$uDzn`^@ z4S|OqpWf_^G6I~cNISYPwWH&u51BM}>#9zsuOv;0Oy5LDa}j9PFkF=&3gl;zX7Vic zOb#(PrLmY`p!6h#Y?o>!TJN4@_nCA_iO#E1^YJax)#H^1;77LdsS2Enby>}ywF|JF zl~2AP&7CtUQ&+=*SaMk<4O+L3{@y8evH^VU!v%A%APsFL@gWSs!3^^9(A=0%Y+9jx zU&q1tII#`K)um*}Ez$YALg?vHZKNhCaL$EW;Ta%zo}!GXaUqB#y0fD;fZbSh%_Vn3 ziai7x=-b5~M2EX)23+=CPxG1FXk+oW^WQ>VH0IsIOPO1NT~1GB+HLMKLeM*d0xqW* zsZCw${d-E}?Q~$sY~LmG4>Y$EYGKqecx>ZhahbH+l}D3lX>n(Z=a<*X2W7j#_lVfv z4X+Ijkm3a#DNoSe9O*6|C8=Dc1XXS1XV@FuS6kb*g zUvCBUs&v{)(Rfb6>%Mbmun`eN9nUA8Uo@#dfCa~~!EhVzXrxqYC*alr+WzL8uhrV} z%7wC32RL1uKX~_vO2P)QggT)uvWw>)18%UZ4l|Fc+{*7>W?H;uO0`QPjfMHahOB=V zWJBmzl1nI{ESO10&??;+{#|?my1h-#0T3+_%GuJ)j@cDoDQ@#-do-!st%&(B`HiS$ zZc;)*LgQ??0mXN9LnfvAON<#&ic3|4YZ#Uv=R}4S)#{B&`8>AiY54J@i;QM;mV&Q> zSY|9L*lExA=nj2|pHHeon@H{==;Sx_kubjJSS8AN^>aZ|S=KxIgz=!utl*b!Voj%u zVw|>1H1MWCqLJXYU8{;T6>!OP_PF$08fq)yje8xzWQ(NHvEd6SG3pwRWyVUa+Q{+D zCJjLSlYckveoGL5QLf4uQyFx%w|nHMT&~17l0g?bo_<@-dOBx^e*1o@qk%m$YO41U zbQa0Ad-F~#jnvG5_URHMC2S4yo9k)PI(X+G5|sMU@4}8x@`*Sclhvc9-fa=;uaDPI zt5^Uh|W@5raSJ!lv}HA9eQ< zbaZUl6#sO4IwL}GnL?g1v136>9M&tj!!}jmuvgmNG40APtkzi z%H1u|K-|yZo3Y*_Bh|o|3zOMRZ1U$E8xYRPlOcn^0>$^$=d_z^J4dnjeB;IBRP|>3 zEr6)>oJJT;0_89014mtm!o_TRd{Gu0d*9PHwA(vS9?zt}n5gKS44dC6cG{=e;59Ae z2HJ+Dk+I7DcS$}%-MooF$WQ|j@)fd2v{xdlhl%ZJvU*N-I)7{&gdpT|R~CGAs{xGW z(5A3mpPp9~a=j8E>s;ca7PrL8KjAVpVWb4hTt)bI%&WQ`kGtdE_f`M0`;GQg;%PdP z%)`d+Ni`DTwFP#ye40n!KF;M-CF3l_f1=`>m1+V z^3Xo=r4C^4NQ8JTtHW*Libd>=^T3<*A6_X^w-;;-)qL-LPNh(xROJPzw~S|ezMoPI z5IH7DP|zWU>rP?8TzDqN7)daoD5pLPTV}hYlijVCrBWC1ebIq`vGV90c$M?f-oiLI8K%?wNbWym0 zw)>V%!LB{TA%#YT#%eAo4CTY4eZG2?^t($TdBqNuIZsLM6Q)B1GnKLfa){IWd=vTZ zYw%dSpf&9@Q_(HY0if#}T@)%FExA%bN!cvkNw}9{_JktQZSxU>^14E7Bwvq{#hvXD zRZ=_Xa@pa0a-VbCft$9SZwHoFB2ZdHSo<6B}^b-hGI zJW{JtY|?LQWZy*@pO!FG;%vVgBQ_sZV?1E-gy=scj@n%QB$*p-I+`d#8U3WWgW!Y5 zfwd$HzI)Q+%{-X#OmD7O9g-+Q|Jy=?Xt~jXY$Ag_R$NAXbb5eVMZ0dB*d0sClu9ap6UWYKW8r3l z*0wRRKnjg6zqngu@Eqobm{&9`m72VJE{$pvz@48_?7JJqC!N=Y=!_==_TrJiAQA|b zm@oY|MJoI7#GqN3fZ|*@615qbj_e{Q+(gF6RbI*4WwpDbA@spvD`0eSK}=F8_HC*0LM*9#~~L=B88?b-pQ4AObKn}NrI(Mgxg^R zQ3K3_T@lk%9E_#uh=>gG5CC_ILn{Mq=T-qQ2~~4N_vFc~U53V^Df9 z;m%curaxW%WFRzjIBXJfh+lECRtO?&dpP1*zo9|B9n&ZvB5p5VsD)-ry5`wJ;_A)z zhK|!dO`jpj`on-H%v zK(dj*4gZZbQ&1?NAqP zRGO|Tvc>4LmOt}f3Qj6-ev4oe>P&t}EetwE73BcsaV`u{h6u;Bfl>!KMr@v~2$ty( z^SFpwa9J=c!6(KIR`QKL{!lStk$v#DliD9o7EcoZf0V|gtEk6Glt|fN(0EL5^j7$_ z&%`i)(hy7hB~s~z-&U^(A35lS1(rU5aJjMDc}uz4oNq*_CbD9s1`-bkK3Twxd%D9+ zAm+%S`A3Vpdkgs^?y`VkQoN_9Ok+jbZX@!+Y3SE|7|x*JKK8)wI`Zy~jh!yiwq%f# zp{LWpG&-Pr0Q4`%B)Y>*!$<9S;Nc)a{Xu`3k8$>IBqn!;VJwidBfef^ zqE>FyJ)W;(6oYzD8QbEs&L}_ea~DnS8~;#hlILDf2lyReTw#FcXob$-vm;+D({0TEWKs?(44a<{A79IVlS-pc z`y}=%DmHHY={ix-%wBxjVD1ZE>)=PR_UDf7b}pFDf!%#w6=Cw#{e&nJ5EBOfY?@Kn zAW+^AY&`FtolnPke$=lCBJUv@Fkd)*+S_9E+mAW{URZf!0)g_lCnxM%Vs;4Gk4{Q_ z!y?6GY=!+Sh#y_&-Gpvx9=C+_0M4OK<4RvVYc{kVdlchXNLg^}FtsnzM^t<-C#ErZ>y zE;*>1m)T%c7WZS}D`KJ`DCid6c2sX~?>wSC*17n;e66)uBrb1}y0%1jCm#tB&b!gQ zKkGbxe(FS2F;~lwbL7M`BG?Bgghm6aQOK>sE8yqh@`5b;R9ZA_-T3Ml4Brh4>r3R* zz19SIAevCbw6QUzXB1~871Zl&nGN4~##&D7hw! z5mzx}@BF-6gV(vBgE*(Ks}*Ud_d(-6f_~74+~ZTr&5wg6NiKvcC?X?6=2k*cQ8A|s z)--`8ZY|ip)Xv3`5zRIHdS2$mSgdc+s7LCGWkAt{A^)^#nGC$-1kZ&*v@1NTb z4ocXm*%d)Z5L<~?YXo8^7A~sb5WJ>>%;FdVkW^V#QmN#E_Ax zjj6w5XgFp8OasIpx@#DE=;xEk^2zfi+0Ep?`+6w<55d1(rjNJaG|YdRRsU_-kot9N zo>(XP0*wzygquJAo=}VFPfb)dOo2yAf(Cap_`l=0|7{T5M?%%&;?!~eGl~DUvHArd zexibH6QO(`_+LNB5(bBqq{@5ed&mEd36tQFZrXSGi;VyKsZjol!G1cV-MSD4r&^7g zQUC7R5x{^^NMMF`+r(*z#6wI+HG^@*9_^eSqYqu zf{zL#(a;|otN-=h@v{i2;ISdrNx!tET9vOFd3`vT024gT(V})$=9D*sSj9k~qTHn> z9v#lI+ILUHFFy<h__)^lMaP%oj zCvdS)JRK})NMt!ZUCcJx%{?H&BYGBD8nI#>^oN3c81i?01kS%*zVPZzF0ua|_s!1c z=2l671zKPbxI9zjPCk;tM|EEL;mZ^&wKtj#WEE#q0D20QVd)flS|Er4O6=`pK@hRC zFBCEPD&H4_b-{RDs*qtng;zY>)MFhII&^sRop|iFYw!@G9nDQT<2xdS(Q|z+wm@L? zkyq*4!}F%jli}u2Jc|);i`{$t?o^hThwo#j7iUcqU^c!?A^s)jlak@n-b?lF&UqGv z$m`?w1u;p!4<4-$(wp`e57p4}Mwz+{BxKv?@t%|bVfK5n49%m$agtFIV;`x6PjL=~ z5z@zBON}r5#eR7&=)(3O3-tP66+Wo6L)JNkjFjqkt}53n3NF<<7oVPO;KP1yXDV*$yb)~9$0CzNkyM78J~qrlqFrS=fP%avDw0Q{DbJKv4*Gq(!N3UBC&eGQ}Fg|~MBK`0&u3(O&z%o)#;6F52ud14Kv!PU5?iBP<{%J_q z;naRaI>t?Umk&)wrm0sY`*J_<>()wAnlMYgd2qfu`>w zNJ7%#v5ML5HR2CFNA`y9kK|tIHf5U44(hfvf!&mgqci3F+gSXo>pfcQ$lo1L-ur>} znt(NuK`XWWd8W)jaP*XZ!i>`j5b$ ztmVyRQWS~f&LIWw z(Fjksrm0r|O_LE8Yt}>9OlGdSL zTcOpY1Q?7`FV%fV73ylcR*GS~ejp~#H7f`b5`+Qsao6PZ4QIDdloNYt6x^ehO?3>9hvffE;ySu|k z`;qVHV%)|^_5|HsYj7Q9SaEUa_~VHyjdF<}#&Cv8GE*vLsY+2S4f+?Uy>OGB>1~>` zEd1mnj`l{?xOpUJ{5PS8P2aV9JYXC8gpL&qa9O2_5RQz;j(xx4qzc;C zTPzx9?u{iT-pL#c2cx$ce>AGjz~i;&=LgP}Y7#mURQpoh+p47l<}*Q_WbZ~(;RS9k z@ujWJ+@-np*f>5PJ7!BA4}Oyjm=!qrzOPU?RtY;VTqDBMG^#&KyjV(N68mrJoa>s5 zhc`P@hqN`ceo5ZZ?io+k+bg$-YHqS8L>P}VZH%_`t?c#tPq02F#E|R; zh2ukn{+Y-aUau?%+}5mrS^TD7YPG5TTCOaj4A%8q$UA@N>8)b01@L7&X*c|>@Epwe zcOeuN^~w_ylFd}}`y_>4j@iwdz-Q{>m4~akyvG79St}p=PH@HarX(Nx_$X02W8pO= z_=o3S<%nN&*+gjFDv@j{=AzPQ8JjDN?$e!z>D$JVKCV8e8{QiYw5R9HrG z6mrKZO0Ja;FHtRX?fV`JP)5|^gR%+=N&ps8lw7v!$s;P?M+W6q3S~2Nu&}cK~AQ1CvFzR%YZMK)H;*K2mcUm^7aW~;450eyU3T4O> zhN2bA_nH7f&cruYshG(N{dESXHF46&PnXgpT^~G}jc1D#S<6lF?4>G8^r)yp=z z{yKjutpi2+o#3X<$!bNK6J++B7!kxJ`DG3FZ~sI%xS!eiVG?<|?y3lI^cb8i zz9wxCgbRlql7||cy_(Nycrz9SgB)z1(qHH^G?b~v+xyC?j4Kxf>Hp~p<(BA|>_69g zxzWthaTRWtgu3>E`T>h+aZ;*GnL>*xZSI<$=XR6CsQH=GYMl|V2}-c-JZ74wONs+x z)1;yaG%3FABRDKD3HuOn=u*mORU*{L-8|0}nUG5*@ux}G1NOm*LX`3;d*xPKTt8B1 zbtMPio}+drlH{I!-#h4aI<*@7kgU@s;cRVT+laA7w1rL<#H5p{$*xGS_xygm9pB+M zOO=>^N~ngqj|bFJ;Z!!ev!;_{VJzq-e^v+LNyW6>iMd3x@RBG5eOCP+x2pYVuE%VB zlTMn66yifkT09@Bd|X2NEz`Q@G9MMIp@~KHe^3YW?2X2WBWr1Jm@A^&M0=w_=;Z^5 zYjYUL1*F!sMVFGee5N2>O0{mFC4D*Ei#YrqkZP)S{Psf4Y3u@oV+2k!f5Uw-y!{q% zAX*n<8$hF%gmtzK9_XCh?dLi3Q_6eZsl3O#$$Sh+6B^RrKX9qeI2lisc9bXeaZMkJ zYKKt3=I3gs_c!pnnF2XFAX7P_-fUHPTg;94X~k^>+#rAvrd0$UYnTqmx{H0!K}-U& z(?@}fl>(79JcpqLiN=dXa*627Sise-Ogu1oya2_egncepO~G~hGG|B_aMfc9PD3kPI1w39q;W*?Xycq<^<_IX@EmYGzUGIim#Pgm%91~VB%Mwt!F*WAbBulR2JHR3%R#aoG zmx7LYl7`3k3T>S!1s7g_3-Oy(Z8mlRf6-iF)(NJ&bF~FM6_hV+q3JHvSU)Ngv=rjH39+5kN|~N0jYsfbu%b$Jraq>o#l$8!YoxhZf7= z=HdgO@S$8@pMh`+$x}+!;F92G_fL4<{2`>#E$~ZZR8wT_opUA-THe*4rs-ZinX4f4 ze+Woi8VLP3Pl#z7=9laruuFJ2Thg>G1N;cf@go=FTV5X5#eMs+a-Uva$_pgVS}?{{ zF<1{bMsui&0Z(GhZ85f*dJkj;SN^Z@;cWFGR?R=Ev|A{Dpi?Dw4I&I_&khTW_gz#8 z6@1*HzzOyq0RHqDTs|rk@9-&Al4PGj7J}+M_Lk+Ijux&gr0XF-N5N&(s#F{)0^J{7 zPwojdJ8TjxVCy+cn3zWwhTVbziq7{h;7fx`JiKBR<81WQkBx+h$j+}gv}uTZW1(E*c%x#=({89;Wz#5 zcToR$<~+7Ur1fuCC~TY2gwQr+h1m45q3VIrf%lHRb-keJhriLOl5klZgs2xA+x*m| z8nm}N4>UNPLHr|AxpLTmpsXMid(e7;L?z#cS+f>s(0q-&fNO&V*IYZdVO9P>95ab8 zLkHkV6r7U_zd#%iv|!)zK4tMcf2qvKHe6=;P+bob#tXDy7cjHy+XJ zd9jHeu~5gRw^0zD3)p>aB-@Owq?SFt{bmwyKgsF%ZfL&RShw0A6Itmv z%7v#5Bd*_=4 zuXmXp*^Vt6rbgZ;WOtaV{UvuAXf^c&{YVW93&nY19;R}4u%!tikyzINEGrVk$NmId zZ47`@5ZMG>|Ayw*3QYAy1gk{>Z3Qb)Y6@w2r^hjGih-8y!y+s&7<1KU%%Geyql*7@eNi!^=Qzo=F2izDqVk?=B&YE31dA|x|Fji>dVzy)xB(A6P zW4r0=9sbV`p@6$Q(DALILg_3@<$5nflYOtMrEE%0$`h!%);q7Qx@bJEiPNX6ip}Li z(cE9%2WY{Fa01EG`Uzitj-#47eNR2;?j$A?+y?Jgr9fp${ZPo}Wk9EzLQ!Bgm??ZQ zypCLoFE$gW(JW2&0+Sg;Xcm^w5-2`bzPEQEYvg>ML}}G#c?gT)e&4j78zN^Z#=Pqt zpc92;ag9Fh9>-MMJavPjRH2By!t{I~R@P2$bXTZ^0Qz8kZZ3D<2*=v_BZb)|1zwHZ z9^Sulx|V zqMME@v8Nl;{ON zg`?sb13Vfm!eXYZrNGYO)9;eXxe8#xz?L3^VA`=uM!^Cdh>NcVA;ou_Z?4OKd;MSCo4pUB zPlenxc*!O=WgzCPSKH;vNHk)nsI|UCU@?{?M?GIQupLJ85t$urKEKx7MPtnn>F|!G z!up=zcufKccUrAoAJNaJ@cUDN2Wo0Gh+KwUJz{w#8wmhDH`}Sry&x{sdbhd*ON$BA z14?y*wfKC`e3~6DY>AnlXMlVo9)!AAfT&#}Ja;&;ybbCfUdiZ!{>oyIZgEndo5-Xh z8VK`|&MM|*5T)sL!dtsPk7u(^rqJ?YDHXk@U8p_69? zYBw>hwOXqq(@b+cEYeRjjpnp7XpJ^6xW6|7^A}|;gYm;(AVi~RI*;4B{=x`T=hf5R zax0Lc*e>bw8OYCoJXoyS-YUu+oDFkyV1WSIL%=JxiR0FmTy8!4adX%o+XdjjE&Q zXCr>81ojEKat8?Sj?RvXi1vTl2k`2-EYG1<-LDFipJrN&w_VG-GizN9>ib`Dd(E8jWq zC7|xUNgJ?yIOY2^)HA-cFQU1X+)1m51qje9dj?Z#3T-d^180!6*wJq;SAiACWVPS? zm&pne44nK-yS;$9Mm1*z2$k{g_@veZTA}O&8TE?f`Xgl-@$o<(4;uqP*Q=ivhd?fB ziVX?`5dT{7avO^Df!PxdUx3kw>?l)jT4?XbU`Y0dB5RCms{thd|CFf>Qf<@!NLebyn#oO0$riOJ!m=!zs_Ql(R$ z_B)HHL9kPYT!ZdNVXF`DE*>9C z1_A;^8DUUOuTXvkfgpvO90dmH0+8cqg`LDSG>w}u;qO-v*{EaCkClwDq}-f%r6a83$n*ygW`-#s#m zUHHDQLOh*Z;N}umQ9964j`O=QaZ579Iqoyf*e2|#2aow#9%!kNYuG~K)MBp^%gXmN z#GKZl4PgcP&z5cZKL)PpH{WHnSE?yY@>t|M7km_p&26Xinn=TTAB@?7+UZ>FQ2Fh9 ztE`{=`L-Ym8Bq&84lP`c(C|@PBd>_EEgvTJ$j{iT)x+UfMa1m`W)Ba95D)ST%!fN% zZJU(NmNZN@7rHkZX zyHqpbA=0>Ady#iJC0;PG*A%6uL+3YxWxXc*^7IzL=Unp8=`T@+TR-<`lV+$EieK8U zU2a0Qqo{Qo3+WchTa<)0ga1T!nbOC~+j6NI1cAZg>wvQd?@c9$xsu$9z6K)ie`Y9F zd~c^ksohuvuuI1AnEH_Ejv>GY$MJhoi1S8l%LlzFwzpqug+E-d`wws!MHG@gBl4R@ zyd3P?k8e>3@t$uZ!`)287RHY;oDo>nJ2~@WOAJk9r}*OS-kSI{Q0E76X^F4R)LUQr<9FzQt;b-M)*K$wIO*+GIChub zuk*b}8^}E|h6CwQ&pW&8U$t-mYmp-53!lf-=1@+h-bY$h$I7u9p+)FhoE~XPev4IO z8u=Tnf{zI1`L(;Ngl~%m-ML~=$B2)H+-Mr@g*G6p2GYOiScpCfAryqj&=D!RaiF*O z_&m>lz5=A6pU!VRZ|2~B^YB8qaYmGqC_aZRHE*%s5Ses4i#jKO&A)l5uvB3R> zpo$_7!h$uQhABW<%oAN1kipIiviARWCtt9sU$-xK(DSi2_raz<6j*0or;cH&Z8fJVx8`v8+gHNrgh@6rM(Ep~rrpW+6mqeZBhBDrJV>LC!Tu@wRy#(d?zx#}Qd!9hyW zqE{y;J2Y1XE*LNa>4+i@>6&_iX$Xm}U^NgM+pSLDeb{O1ApLjk6ox_*3=(FlXmLDP zQfBSokGN#G$t23IlGICDE!xz*$f32|fobEB!3MUNrXtP((<&}@!*|&bLvYcy^c3!B z5OhbA&jIN>Lovs;9MOFS$sfz)An@RcEP_+R;D_s5H}U*E^~DmKt8r(@I>rlVm#ju$ z-1w~WOr{yIF*yt^K_9*ucKOqwB^BirQ@C|*Oj|24o zkF~wU?=}>#HEl@DI^hWsDV06ve)$&9LO$E|SnIX*7HtdSt|+%wMbvzRc9T6UA(Ums zN0z44j>G@rlnN1meo7btcVRaA9`CL=Cfgot)x6uU#}o}jH~%iR^j~Ck(;PMI3yk8$ z%k0dv!X4G@fo~D+Ux~n92$Q6WOrMNy`LCr8- zqtm61|9>E@`QO74L#^o`AZ*f7pGCgf#|D3mn0;^N}-v`M3{Q^p6W}A1M@v{H_Uz@*`ityD_b8x^v z=p5^v;7xJ>gjr_mPbh4=)rk)}uCTSdyo106!p&TF{I3&Ef zniwMBQvK4IpMUr5($3!A9@X~Z({{M;g_z_uB$2Dvxuly$Gr#&5Lmkik#@$({(3JvWfGNm_UMXBJo;zY|4I}C8q!PLF<|@=+fpiU% z>jE?a0$S)8f)wB6s$`%uEU8aY;~}r+KRwM%lPXZoZhKqw2n{j3_+u6z0m^J|9^&lK zWJLe>kza0z0;sTG$U(|NW@hFmg{o8KS}&=#Kq9D}7pKFiU#1%(0>p4dgh&i&MV77b z1U)4LSp=#P>WH{QnL3$}gm(0@FT8%5ygYoc)rfr!+@%Y=JiJmk^&f!w1JDCiO?ryS z;uibxb|Q#?q7sSl=fFCnoLk=E=|`s;A27`1C&mi7nlJNncR)&|f=r9uL>b@g5RJhu_;oQ7|lVEgm`KW!-fJ8krhaHg++S8X&?08PXMpoJ;( z&d*aQ`-3bV03qFrtyP}D0=jvai&jx^SPNvx&zZNNAEAe@5{mc6Tj1mABsirKcnep- z#elRlI@uK>pb4vXC7=G8QMZ#eG=?Anu#1%jymERXv-*h_&ewb;8FkwWWZ!fZW3|Y- zGgQYIV})XXh%PvUCAk>ARtLHJ3n6*{K8@%31~n4P8AU|JCI=vqw@3uMv;c!#bx?-Q z2MGpyT?)Og0hM;4YAuMwQFy3Ku?j=>ZPEeOd$vfv5Ul&yIm78ZGHSDdVZAIr)oKOE zt%=I2u(X+N_cxwQ)dg^TGCr3JRadEZuND#V!m5#}4L!gNjatyvi$Y*Og~5{Y7FO{~ z4zCH^{Pjph^+q!dpXPsBo&Ms%B(ET6#PLiiK4wVpIug6 z9b0GPi*kjeh*U_6a(AoyO(OxHk93u8SKiyr{^Td~*&;MkeL^&}pBukNR1Q9If6}=6 zJ&=a|-`@{6lYg{Mzn-Wa(|H{`t=2kaW(wu#0$+Sf@H_^7QS1&uT=h7?Nme8CN|$a2 zJH^jA0UnHudfsfJWooqKI(&^vB0pVr?vE8dj%Q;ibkY-9-JkjI%8?IeRpEr2^~sIW zSzK!oUJUBBU~Wlt15g7M%Z!zdW;+rvXcDJ*>0i1xeQ^auIIWf`<$7{g0!TNmuHu4V z(aS>`JZg_l z?I=nV!Ky9qpd_%!U%g0Dt;g*OT z0`Rq3S(jt(RFgG4W=f|8zOu`70f`o*LZeM{P%fRc$>v2eV$fC^F?GRQHTSzro$Kpk z_Oj9EABG$0WQsrMMW#Diqe*s-_R3h6BOY_s9Kspx5z@ zwn}q)M-VQ!8~RJ6|65Z06#^Q;x(N_X-!iGRA;H0^_2e2~^~RHm)*%c`f zELt#~R&*vW07<9{*5_4Vc4D9vc1KuK;sZx(zru#cY+3Y2@+g`I|O$pxI4V&+;i{u`bUq^YwW$Y^cq#Qs%Fi3=UHg8 zSSz)H8JU#xCsi@Ow~Hh)LHVv;0qS^^OYwcI2Rqy`t69?5`N1JgbaC!Sp&@iT%V z03-6Dokk&pK~fnL0%89N|EHp>!v=L%DXv@38Tsr|ONS~tn?Kl{F2x%mx)?K-=yU~3 zt}~hUE3$jhdksXAjm^M{kjk@0jcIU>sb2%uh!dnj_jlaFftt7i@Ci~g)V0^t7JEz; zD^e5(#l9gyh~~;lIM8sN+(x=4WsA^L`PSe-{hci3!^M7OI zDs=xj6-0`yTwvPL#(qqdVN3H5=eq#~H0C_~v76h-`+25D*`Y+H@^t#ZC&FL67G3Ii z(1Z}l_hkQXdK-betWdnV_=L~nKK0hxaS$rv0SJph*L(t#RDsYYdYwaA5|g&h=mi9( z7S4~0AqpmEzrcMEaIejOt83TW?FHS8fDdO$uQx75fU9S5TRfZ{gL?Hw3i5Oi;8w(Z zF2#I2A^^TyX{N$wVhejlE43x645rWX!A zM#Au!0U{b6r|n>GIu;KAM9T5E@;1{DiYufbT(UCjHpm7;ozjF|3GX>S%~+wS=BG*l zf}sJf+SPb0sTd~Yu^wMbP5LE2BNC%G+a~NWD5FV5cE*{|`&*c0t_3Tla8twsl6*mB z8|MiYm_ifk7rSGuU;XdldmS=@#J3R-{r|gjIQxs~CK3e26oi%T_{H+{>Twz_7MY00 zd)h|Bf-MoxdRA%Yq5!isS#l_e3B9A0;K%b%`Gygo=};sb(O2JxD}n18=Ub?y9cw<# zgaVAmyArPb?`n!v^x{9Hk!Z+M)xmcqa!r}UrC9Q@B^{Yk z26ApJMilc&0 zMT*anueP`vav=zsq4oPr8$UhJr)+-vaadD=Ta3wM%vjj0Z@s z8t_5x%ZX_KZP!`3EQ{y;-)m4qSE!*pHB0qe2lzVNBRyagQtH-pZYJi0s#VMLt()xd zK?;zx@8$Pe?LdG=)==1S!f;|Z5^>Ohz)Yw;F-OPwgM38RQaI;1E$bm@rS69sf&e8S zkwk`og-e(YJvt*}kkR{I{AyAmN8Lv0DP(N&S!lV`n*;oN$8COv@Oc0-KI29=NaziM zaBr87tQ*+=0I8E_7SO=rzSNASKF27h^L|98sQe+g2|tv`T<$9@2k^ruv74JgjTN>` z|F22rgx&N9=+&6UB`|2T#p#DsElrJ4e`w9&GgS@)y{G_WO_YR9h(rB7m5rUzJQtfR zR+1z8B=RoX5@^&SI;{|q+FehF%-Hdf0r@hA63D45?jkTpcIzIcS==r3QdgctI@kke zRYqb$wFTOOT)y_S5cO3ULgRBB`U=^qLc*7;j+7LHT_23S~1sUbe*wng>`zx-6GS+%^!|YqVr_ zyrap*O1nzgyu`WQq6`SFj>E~oO~@{G6y^k>WM@e#2zC1Jdz66pe&T#5@4egf*NCWNehTOl}`IoWzy>Crp6Zv(xO>6Dnx+ZG=z z*RqX0^}SQ?=WLf*-!Fk@^#LPj;)1hof*6tg$5E&aTRk6Lxj;4#&y3!Vw%Wed-fFKq zx&nns86mq~ZLG#0VVi2|>5vsN=;fK-M_b^@)Xtht`s|Zfo&P}8Ju(?hc%DBpu6VXh zWnbqcblxK2ULQ{cH8V&iy=P)_~A4n8IgvWIkbn;C7yv;;jWw`%!-&!U6-kUzDh&+H9|Y z&hvu5H@SgY;_y*sgyisFD>f}k$B-ttuc+<`dB8#V}}zwCH5 z2TJ~5vJ3f*iD`ipv&aX=yym|-CONE7bJxzFxsIl{?G`N)`eqP!->MHA>m>MPIF(vs zCT6z{vc53oFsrSR*!5ty1{&>?t^S)nBl>5?YqBC=9P{)4&4}qh5_z+)+pm%y{c!0I zY2wP{pST|S6C%%2tRd-?%tU{Vyqjc2uQEg?hrl>dTjP`EwqKp0oGR|bXG`J3&@Y6? zb8P=3-f-80XGgR9t`;iTpCke=8MoM_5eR(HUi(E^0_hj6n)+=<*|&i0@Bz8b3R?I6 zXiD@N@mMd-rtZU!UlZNVYL5p0INKB!4Fv5YZs>3~K1Fql#Z0lz7nW?6)v00gGaXN-y0VTe z5pWmfHg9GkF7|JE4vFothB`yC6eO3ch>O`2Qul3VEG z%lywXvcxiBvpJn=G{;IQX73z4#2dRi~_I&eTvL~!B!_T*k6(*pMEWJwK53KNw9 z?s5|%7Ha6|1ll_(-1bifK^7^D- zlA~-o>@_`v1EJqHrO)7yCwgZ2Tm{{&yJpp#z${iWgk+-D&E*a4=;8(`#yBfJZWe^F z*C-MHu5{w5q<5vVvVcG4+h#ykWNCI#TP7=rS}Q_uNDn+U7K+!$6DN%0eZjX)X$It_bDq5y6ac46pM?f>3mT z~66o>RCRX7WVNxm*H$fB`e8cQ9a#kDW5ODe9vEGpQgn{b|;5e)0HsD4C zsjKxft_bCiy0IX zW@U$i1iT-m7K5IoM-F_?K`aGQg}%x=;dtIdOgCP5Tf?zOa)l801m0nC3fzje)ejw=2}e1g@mFz9Nzi5~elU(lY|mfc*_#fFc(j>8^u^Qbmrw6=&u zQbG-J=a@nC>ks(u&Vw!dV(Wi-G8?17=kkk2TEi&sbitpZai`>I58(Q|i0hUD!4J5d z-|HF!Zo;+HiRwc9Zulp%5Eoe_T5#H90GMC+^jjGMDfAQH_)3nJSu7+2gX0Ly~xW2G+J(KI@|tqxz-H{R4Zgk1&CP%*{*Db5RFE6)JlrPn-COl;=d zarV;?D`S$0-HXFl=4dJ^NBc&*4c1Y2L@0a;kc>g&d z;B}qi;MM4(!gs*})5i^Fmu))E?w)(mEf^S(Zf>1yE2@&_#9u=aw z4ep0=0{(c)H{bU35ff7LaclkzxWOmJB`fst@Z1~WfK0oNM}bL5(<6FFf8Hi+dVT8K z7J!sW`9^>>hxS(Y@wM-Bb#{q&xW9EV;OiXmf_PgkSQ3$C2<;15JfgopOU>5r|0HoB z{6uG`{*I1N2Su@fUzt%$JDZfyYiN%Zq(|Efzd}}EmcQ);gxbq`i;c@RFLw_xdh!d2 z5xkS-w_K$jJ$u9+r_B1*?`MZeHkSL~x2wDvMqN!oR`r9US%hJvk2#BguF9CyT$vbp z*@(;jLHMW`DcRg=nn>qA&fJ9q@VZKh>4$&@^iWPu2n@{s@Ld*pdzghU7H1uzHRpt& zhr}YWE~?`!tSQ18A#z@#PLtyOh%p2~#|J6@{Y12f7h2(JB+*4B586iE&6rx!yD(xC zG66Oki8W(1P=C7+Mt3uQ+??j+ybO}(*T|Jek~S%dzm#8xkQjpNqxl!Rf~~jl2Lqa% z6$NNK&M&UIeVoY`!bH}VVhOrP_12h$v%sw5Q`wSgPNy{S7@lX0y*4W~O3<;=2V{-5 zn;Ad9e@8YMmt4}n{DOHVlNDj;`>0Y&K^brIcv5AKZLazDSJgx69(b3Gns=7TQt+fW zSbihp3%@^=QgaN=1$qq2(umz}U)t|1zQ9cQWhM!_#rEDh$ZY0-)yq8K4!9gbmD2=1 zNXA-4-vhE59LqH8oy@jvO$&rPo)RJOv+(VKWSREvyQW&RBz>88O0qNu2g?Vx=Ps@Z zuFXDAnnI+Ku`Lv6*f(Cw&))7R3wy$Gmz`o1s)b_)6g}T!5?kgmUiMovNk|I%TPh%3 zNVYVnjau@4&<~qJ#J<%HeF*pY z6fRsWmFe+bN~u7#>-BDxTv_uK4rW&LPWkf2`;}r4B3{oe*Vht0pu2}?-DyjsE)bbm z_lQ9r*CH>OD57U>^`cEG%zYSYNMRKy3oD|F?SeX@&@Y3XTC5!dVxLg|9if;2|CL6R z`+?LGL*9#q`!6#5Y1p{Vq6rJ_?DKVzHx0_v$paO~B!qrT^KdNMo%0ct6uv`?iQ%YA z=t4G|&cVCa-6wFj#RV5FugosT+pKFt2*=bQ=zC0@!;eFNB2{_a{=siq7Tg@zyFH;c z0;lkOJ-|x6Dw$jEM%u?vnT1qC*#&!dCy8qP8}8VSHE*)y&KD*26BbDS&07%qS*f#w z001%zxfc!K69Jl?v8QJ&q|W}?jF0-#L75rbH;b}+r%^N}zl(ZMkCC=7!rlno zD)+$ZW7Qg!;pu+-Btbnm;SMt*+bXx20cUJmXz9NjAv z``P3c-S9-VtU}Y~Z?lJ>xcR9h%~}cK5E?2xRi#`U^=)%WWL@MpC{hCB^j>K-4x&-X z@Z)g-{KbBer0%BBfOeztACCzl#vP@k*cXwO{ihIg@?7OMrAt}$ye~4q2+f~!s^4?R zLr#YqNIkjp^cnM4GkLB3kv~UcPC@jpm{wNp5h(klQtlS$+g!+0FmF_-pKYpMvK5%} zaZmo#2PaAmttMIX4zWiPhlQa5FCUPxK*^(lA?Va77NJ4ZvHP`vgxYW6n;UdthT6Bd6_0nnKJY$eyl zK}!77z)x~m+c~_Eqzcv5Whe4Nk1s=MalTnBNXP>-&?X|`(CqF`JF)r#wL|ZFX}A4o zJHJ3gN>VZh9t~k-d@ZxcIXswy@Z(N0m;8zY7t3Uh`YuNE9 zIaQ>D<8GC7VMZgWe*AKGczPkQ4E!uq|2qR)HjB;R7t~)|ngp$C=2rhraolhpvg?o_ zKnNAwh2{G|h$dDT4s7yUL1ZCbhNAUX@mJHe7pZu14&*ddgdFCefM$D5_*vd9HLDrN zUp`yOZh5hzcr!2*bmyA_r8%AFu9VS~OmuNLoc0DlXrF2_7DM*;$W%AxhBc*CHe}|Y zHzRv3(Rm~7YbvVtekmp+{Jh`6Y0$!wl3l|b5LjtgR{F-OMwTM@esOTG#|K|5HG9D_ z{UlgXenBnim?V53w@g(|l$29{d0D@}DOg>-3j9@ieyOkH>tXcOvrF-aZnaKT!5`3( z;dDM5S5a;a!~2@>uV$kUqhB1?Ysj8E5)HDw00gkmFq2c;%&aS-FPdxgXBYACIoWV{ zcRKvLo3yyT3w4_fU}? zIvbKx(7&w*{N`*InaoPt;3|9?Pmt`_Y8jcajRZ;RJy3s~vV0pmT?$uokYethG@;)v zJC)55pCZG9pLgF~mQgQJ(TLJm-@_+u9@JwI`s;-wiAYKJ2`{17X=BrDA(TQM1?PtE zm`5Bc!0=`y?X2Z}{@Ri?J2e;FKLH8#S+yV$2T*D2h0poYsrLRlI~jfjz};dIJ~1N~ zmC5|dtVb)?nOd56#KZz!4obn$6;1p2MS68S>q3a?kFAp9H}_`5a#-%3pSEj1KYa6k zjhHFD{>ELdhjR|kI=|BPQKQ%Nm`Kjg~I#w1uz6!2*KM!zbA*w6Qygy zWS&Fp(khwC-)0m5Z`Lt?YQpC+vXO`cV3H7-WUQ{N$xvigh+{sgnUGyx&c(A!K&OfL zaxkMjx{wO@@4_7Rx2!zoa;~}@Of;uyGH9sB_jq^SH%J0kWK?MPy%?3| z%2OVAO2egkwMRfYhM_nnJo8U}T4MVSsW-VPO0S`32h%JCx^E<%mdopc@nvo`3F@Pk!3Ip2 zsSyq)JA5qDSH9ZWsM>G_5L_VOhfQsUD~&Py4>+n2WM@4#rNEHJcYspI)HHE zV_UxcvFp=GaH5DHuC?egXw^(k<=~IYjQ?7s;Dyh=YUZc`@ zn{ZuA2+&%W8PDhVq~rcHDH9*V_P14-AcNe?C~s$bH~sMZw75dZbUVm%r>p;k24O3P z;sw<4MC<{Lnr!N6X+X1$u{gYzB1V~A@tNhc%4cxq*Pf`k_Oh@GyE)J!oe-gY&jy$2 zM~{Y~$vWk!^XOvKuIl%ME#N0?ytZdTp?Vk$DL!N`vV5aN|NP_ZnBeA zpKfb#LHVAv&@j&N#x&0%-gHyXQ|;a~C+k=vH0!e?mTMujZV_FU^ zu^Ov3BpgXOCoLk+S(q&vC*Q#sm3#3ldL117XCI&9$ah*q9$WFAC+^9l@FnpJ{{ZU< zTReZ3>z+r3+2>Gq-Tam-iz19&=wwr=srvj{f@F+}j?P&UP~?PT*1!98;@!T5I=-0B zYm?0#^H4^2{j2nR-A)1zr`eQS{z204O_ggGFtm$#K;a(d$J6J2jKT7Ok_((E`R*4dJzDfZ%t?C$IhWt1|T#z?eY!K#Pic0)g#Ra$U)F4W1_ z;b3cjJQ-SRvGw3K(?A+PnY7U03ul7%8u4Cln{B+!X6#)br!1EG zZF}S*t9{icHY-)ZqpA8bX5WX~ELse~&n^tzr+N&JSXzdWlfGDj`Y-Gi=Yu=1u{`9q zv7>5}7c*SrCbASu%=FvayTgSnhJ&>oZHV4}$e%9e$svMOY&7(2Yx`%uvJ zL6R3H3HLcxs~V4s&1Rcz5rLkU?_seP)kRYEaD6f%|MQ&Bov&)9U$Qrm$5*qQuj~`( zq_HB1w>sQjPi^s0%V2AoWA~O-G4#`zYm%Q^l^Ykm7oVIMHi0EBcw63{@&^LFhZuQj z9{@O!`HXSH62gH;9-xPGfW5NZI~hhzm8D0M20updPHMAgm@{i&v*B2nFk{fbdjxqX z=cu+sjjhy5yO2)aFNHr7*yQApuzZnvIa;lRykjK=VR@;2U4k?Z^YTC@6E~{^usV-7 zip&aEeCaSpb@a-aj0X!%bi|#9=G9$-2)+B{9!RtEALJ=B0E)PIMLcsqK6lCgcII8| zy6T!B6QO~1U#5>$i_J)9eQ{uhu3jB4KvP6{+#q|PP-XAP=`25mm(UMWVp}#0XS4FTs8tQlsVm_*m(x5{E>gxtUqD76%2Vk$tq%es$wE}dAK`Dg z(4+^yew}Vm&EOLXJkj+G)@*CKlU@wZel2sf5%#uqhaY0!`pITqG)T7~3ep)S&c|nf z84{1+T3&mJf{oK%aU_S%xm zHUa6<^hU?Ape0)3FSYn4{kfttM`<-dtc{91>Ty4jTW0vTQ}1b(4!j7QL#>!fyOf`? z6u(ut^}9YGJ5-4K-0Ew3I;JwtOd-VI4T-a;bS~gbPgZSk04#j#cjXvlwYmrlyis}_ z3)zgd0RXRS0jN=Xtzm;)#0=pMMFFdYqKUIISN~}uvU8}pE>jIl`JN9m)JYI`Fv>Y$ zqayx?vHU0BsNYYm*m0~nXZIW}!av=z4D>8dP(<$cwL?rzKuL{AcPny~5^GzC<#Sl& zdOM5J8ljGsZN2rim;taS2$MAMW1((?v{)OAABDs}mtDA6h02WvQ_l~G%oJ+@OL&R} z6l)kM zPtGfdFeZ3Kj}(ZHjv4CENAgmdyra zWT@cYwBRb(K+7@W()3Ub6x*|HDz14mVAlmw;Rz_guyK}?{W}8 z=f|Uiuf*T8qqrJ~vr!(MH-}Fis@Cd(_v2(P3xuRyjM0C8zo^F$W!GSe zCAu?FuZ<+*E-?V#MC^?xZ}6dWO*H}JN4e%1zL*%@ z_TC6%+@Q@FQ1_bCcEcLSkTKo@*m!tGqe#(@mX_x+*8w*bWHyZ}c1No6r>x#?SuRp) zIXbx0=O8Jj`LN7@#MY4vlC&_-RBp0)hDjwk^T7ZCrwHQjjKML?x(ww`zxKq-gCr3^ z2R)Ys#Nn7)Cgbs({8C7Rj86TjYS=&y8Jz%rE`oWi4bJnM8dar8R}cnI2CU@A4KSiS8$U*TOgk6&H|(Dg%ns@1LMsSmXZRQXx5#_{1$CLk9AA2voG^ zYQn?sY-|RQ*t|pMGb2|5D?7%z`!u<0PDzA_{-D!Kl{4VYyQ3dTL8Rmz1|=OhZHkX1 z;Cd+bg$R#^*$fceG%UgxhjvKyv&Gn+o z3Nnh(cfj5lfNlXd>L6EtIvJAuRD_-rxb=M4>=kLv(hP?s!ZbjO$ozEC57=u$;o(#N z&qytMKSHNLJ$zq@QF*s3Z#O^PpdoS0XHr(0R`*ns2lwMMrj zpf#&%BR&_(WM&6r1fnAVPqh#n!f;oQoTDaP7 zFl7AJ?^ll#5SRV<=0~1i@^e-@6T~DenXGeYu77RNO@<+wyB7i+cT_E6(tdRP%PPX< zr2M;mVPMAsZ~ObD*&;;}{q zd+AwC{cg_f{X2$aL=u7^#65}m5B1_~HZFw*^8kD>e&Q>$O-eA@XCIU$Etn;^BV-if zpIGhHCB?95QyW-hAWcv&)hlKRJw}u-xj;+^2pp z41ew=Cz|0*FIko$jR;Jj|8t&cw;f6g=DeK$G3R`qpCcn|1S+ph+P7nI5Jss#Y2WRn z+J`^~Oq6$6Ed;n`+0(uD<$=`DuSFC>tlqKivxq;l22~OpwAkOpA^O9G)&1=8!C8r2}Bj z+|Un|$9B{Xw*;&<6THGNtr>2s*(jd0!*?Qz`^enp?~TB(fPm23j$b9!HTbzRoSN4l zbDj$%g}`qLbAF^~^lCGs3~KQQr#x@>RmNkv@j_rCs5g$FAYz*o;+6?e%;I|FcILG-(d3PxzS&;bkTg^E#F;S63#a( zoP&C&joL~Y?oa5*2d3*RAuOJOtc1>{9zQ1dETM5l7A^R(Z~p1w{-P$(zh*1nLkuir z10eq9wRs^sCcg&nVGq}T=7IPWKTdJ@bZ)x!QVFA>QpnVGPVg#}CqcBOzIPj;S{$`W zOJ%zd<0gK|p*pPW$BMvBz7t;l=8WF%qYfSZizBpdh~t-F$gh6~xyj#5zeP6HycgZO z+W&7H?rr=%pn~xKu04?sEdGr&k?J@5G%>^b&pp2FRTv4+4q&UL%l|hb4zyfCy>o$f zAR$?~#6}cZq1)U;cf(w^NU~)}85cj@KGibN}zs#W*?H|9JzDL)L}O#54Ua kueuIx-~Hz?7+_vu$~HjuamfahP>_#;jEZ!nBq-qj0R>IyA^-pY literal 0 HcmV?d00001 diff --git a/images/adb/compartment-2.png b/images/adb/compartment-2.png new file mode 100644 index 0000000000000000000000000000000000000000..3b00d0a44d4598cc584be3e666f9b547d97b71bf GIT binary patch literal 34377 zcmZU51yo!+*Ea4lK(S(jyF+nzFFwHF?(XhT+#On4TAab%9f}qyPH}Cq;*@XNd+&Si z`hV6soRDNECp-J(?Cd-bv9DF-FwtJ3!N9;^D#%M~z`(%SLDPFENYLMwVr4&JU|^+e zrKDafNJ#--yE$9gI#|NMFr=89nBXd~FpQX*nwX4CFf*dLd27VP#A}!YkM#`p4E6NB z9R=oPgTb3bfK7O%%c1pQAUpFQu2X- z9{x~FJUVI*b{;#8qn_05?IJ*26ff4o*geDy=JxqZ>OO|TwJWs8m#WVP9CN{tWNIKe>L(?JJOc!=5Dqw9=6U-z~AkfnmK!Vh)_}e z?&yC%e?6zAkL|xbIl2EmEa(8)f0wXxvT?BguWhKP@b6r~*S0>E4j^e;N2q$BeTcr| zi2Kq{}%qgf`7Wy`?pIjZjS$U`Ja-%C5734 zkNiKH_)E=ya-n(_MH6QK-<*k}=~Z6{!@!8cC`e0a`M{nOAcpA3EDmFj&#tY}niRh^ z3yFQh%2ZrKR&-xpvlbb-W*D-z28)Cg6ADMQwg!Rh@O6htUe!33!c|KQq25H{hx_}+7%i?pX1jwX@K45{d;D_Im8dn1?`v1-4SHHCXyPZF`&=)d2vHEWciFC*R zYW=T5db=RZ|2-H~KDii$UtMViZ`sPN;d~XEO&_21%ebat4;{&VAgQ?bu0JsicITLhQ z;3w|MS>#t<95!?2_ys&HXjv0hAYl>fB+k%!e>?UU}xSfE!*PE!MEJ>Ng~pYgd%h z*)LYBRHp3zcletq3~-UMbBY;Ew6?ak7ZZFNBO*7e)aJv9+UPGoU|i@hJubQ)C%PS$ zWR0l`ozj55-d?8jA;iOEX^mSb{4idhtjqe}WWADoJ8d?nQ)9$BC43p5A9%fVvEtl4 zI+nql5z}b<;5ErR&H+1@7>QqzPI0^OTeve zZYQ<6HN6Pa1I6t5IXr8LYy^!DrX-NnLMdTW8_Iabs-2 zFtWv((VVex$~Dc@x5?S_tG@q?`OL=Byn@!02UpH{TMOddd$6CMU*_Pbyi-TlX`U$E z;`PLY&BJ+skkd+waR1|a8%W?N&*@P%k(Sx~p($a!e!I&P0gVK< z{S~@F^|(928)mp)k83>EY>dT2o3Z0~_RDYZnLk)h6s=uFIA_uU2Z7D?JWf_8$(P&_4BeJ zubZ6=d#S=bquMfzT54(R#^g-Fk=Nlp|5&-K+ z+=M7yP3V;85IUI5faT|-ikh8u{~sA~>|LPcCP(JxHLpBTWx~(Kz>Es5%)N;`d;b>G zqJRqp=v(IP0f{D(m>!8{ae}@Waq!gd!QprXPf*J!?9zXItvH zEXQmFkZO`d@8ymB4rv8`&T0MxuX=9;a+&ra3}O}|IYpDY#XRndJ*Qmkj#pIY^KB9L zW?u8lL6uo)q_nA2MCt!@|MSZwm-RKNUZ+)~cyUVe)Afq|#+Jp({Wk--w-CUl;F|xB z&Y$G0EIDk$I1Gs>CfUS-9yYF8ouH%q7wYea0cbI3G!{_3>RhDY;igAEg1A!Qdnt!7 znMoGND}_M^K)8*dhpM3_Q_K5}=U?>n1s)m%di`Pj7+Oh?YLSKZlzqC`Eb3n9T#}db1*J(u@%S8lw*0#oW|nA-MfW=^LRc?@pJ2w ziiJcBLwH64NqCn87)T3XR07=(B~jGSAXoZ)F%^ehVefv_ct8r3MUwU@3cObDQxcKi zefZH`Be>=|OlEfq4rNLH@{QO!KSdik?j;RjUlenG zDCTMpKG#$NCRXzB`$R#-2XW)%aJB`pdu6dh>isZ~Y^4L>!AR}~En5rCBH&Gvls`bj zL&o{+Y6s;7A{^LwRrC5Y)0o=VHb8EA&)pz2#7w zLeB-ga=xD!_D7lob-v$Q_nB(n874}7g=9($$pcI)qf^qPc}=4~Zuo3t(yl$6HW%S4 zjtB()n6JpXE^x~?zL%vCz1@WOq?BryA>(Trxqoe4g?nwDC?Gra67{9lIq{TXd`UW@Vvs;#v^gZ*{a1bJMiBE_z0R z^Yi*x8ZsgtN7L`Z#Rpu5&EAEfXk_^`q)^M1V?BJYuNiWmA#Z!DIh;r~&m`SJxbp?q zGnTNwQJx~^Qi^>wz9DV}5Z~J*d531lO4G&mhSvf)j_$kZnWXM7pk<3KWeSL@)+I-E z$9t`@qD?wSMKE|hKsz>QQxuUp29N*c#{)Rda1OWi7(VGH+DVzqP1PZ^C>cP9P4p!k zv6Ncn+lFBQY4YGlB;>I#1O~X7l8NDz*}w|roVr|W-=PUKJ#a&d#bgvOIPmx{*9~Gx zb%wpdX!2Z}C_w~#eW!rYbuAadU#pDqnEV21n*r7IESl%)@QWO)Xn43iM=sIh&8quU zB|Am3T;PtB;xtYlF0J81KPw**hM)#_QOfPtmj z#`}C~mg0d^(|2)2{Vl>9P*3(RpZpC8oI<-OF(lT#?q?^E5(5&8dxz10aZOW6GdK7q z93QbRjlpr=iJ7rLM_;GPsDvAkqbFZ1QpvSMdjZ5yhf6bT<@YVdD(d7KS8YiAiQNoz z60da4Ob)k<&&dacyI@MO#gv4_r(z#|W;5mdpoU0EdJyrGbshWquKtlhotvyO(Gm3| zIzZyBbIZ3V7JH)x;eq^1Sv`~{+8vUGsI=WGgZ9uCSK(aiQ+1omqm$KPL6x{9P?7H7(Te83n%0DeD#|6afUv{VTAH78DFXU*L}rM1CB5@ z)>TS~ZziGX5dFDra*K_mXtl~}^XP|@s~w`?DA*l(p*pKm4FZQlLWKB@r3^W8n!K(m z;~Q+weg4d`K1CCG46+5utpm_x_$cAi!%dbPMWEK&w-zneyDg)M!)0cc0&wC#-YBFD zAfH=WD-0JdR8M&~NnZpz#)@;5NRn(4j7va-6kw$xTgB}v&|o@1?c^H{eBsqs9x*;G zuL>5eml=)DPXz)7W-%#5!B`?!uTLp-xXdPi82}!$uMC80$s~DTz*OA#EJfzWASW8J zO_Zk1`0rU@x7apjN}m}7UuF`s!G&sci-w?18=ZH*UTpDv*OXp`Mw4 z0uD`K;$_bV)c@@O#T6uhjKx;fj9dLc+s_>l0pHj@PuBImjjNcBL91s|NanGM-%>S4 z#ma0PnS2w@`v#U+DYJF)&5?!4H#1W`a2;r{70*r_JPpQBZaML z+*KYMbxY1cM?G25@on<2-a6#gfmzgE3rShwZ`V5k(-bWBGAS)a*^EjgEERCB5z=+* zB((uxdobT;Buh!;@o)eT}X%ts+pQ`o4oUsD6?l^tf`uvk#RXq_eOv@_HeZCitz; zz?&(L*^g zZN<`;TZU)s#%e>|j4lVH{b`EmF%`az58-0iG-h8MMp1Vl9!LZNR?2mLBFFYPm1=z| zV}3RW``cC)mcd9T#pKzP?>;j`ynuvt4Vfcqgl0s0PPhcls%=dj;2|W$!nN#V^N#k_ z(X6&kiQMF;zP=jL7<|Q!ZH0p22kIa-=74dwvNQY1^ajCnC=!#?>Pax>YmRkcS*FIw zt0#I{45h_F|C1d0EJr85FxE_+5tc>Fz$lK3SX}vK=$^7FQ-olQchx1*oy;}C)zBwi z--*FLsGPC~7)hILjhbebT{2n!u7#$Z@KnZL^H}9Upb@icGeF88R_~@Cs#mhT`p`_aB9G8ETUIyOpc0Tf%MN(d&XuibphcIK&52O zv#827sADFGp2;s%pFKM)g(X6BO!vMjf7x_a!s`x9KOl&^f-Eu2G%nc`fQ4Z;1lKme zyQRAKvKJ0L5yV4jTZw1A-+O;-AP3CA*(Bc<=xvJ?rXULOr4)QT>;I|0H%Q&?Pn&Tc z(RCHN&o?DmsF25W{@Em-NZ1GBXSKy%CG726xY^hn(-ZMw6$OhuXqY0n!%xHqb_eWG z?H=+gVjZC9-UEw#u(g>aa-DsKDt<4+Ke;zs1(|g9JM>7{9eUgQGYKb*-;*v%uQ(~+ zUg=zP`0{Hr*NpJdiffv7ep95IH7v|P0M4NE z%WJ)Kg0N`+^thBZ;c3uwTnQ1#f?!;!fz1LdOqwMB#AhpvuG@=Mr{UR~YCZPR=QBxQ zpWUFh^j#3`PTH?&(H%U=s0`F%6KCU*t#MQUi70zyUS=HP8YL;zBh1IeLLo3F2?+4CxZJ%vO2=9VzNKnnVsEy3Z;+rMBVDrZ0~e7mbWc4vt)&lFq%f zhjxcn##N1%DS5&@U&NyB5!?K%G#opNZ?snFARD73c^Y{*>&Koe?V*Q>4W(sIM79)A zeKQ_+To&0Uc7AE&u(OKBH+6^5YlD+?P=&B_nVIk>;(}?}kBY#%|eum1kYp)n@c&3iBA8jaF=we(WJlwWZw(2Uooa3-Ui~4sGCVN>8H8h;21gae+U&d6acoCCT zTU&8lZuKvRfW}$RiTS#`!epXl-ZRGW<|G5Yix7m!B7fgjz@s0H*^oPE3yYw3nK~ck z_t)D)XNbT$J2k%==9;LF_C(nTxKtrKfJD^NQQiO?&pY-kbJ|voLGCu#=&z0~FW?X* z`k8w+t6U@{1mz;~D%{g5l$Lm$?4R&ja4PZ+|OVG^= zUsS%07uYa*6X%O+`PmEu-eC1hC9&UE)9AxX($pR$Zdv61@H4V2IBJ&&8=S6wS4lyT zQH;CeI#r=B026}Xd`7dEc_~#o-LsEWK1m-eWYcgr+mBT4;==3RW}hAC(tWK(T@kHBl}`kh=DVB#h)AH zQSjN};&Z^7@t?pOEU%`%_OuR%^!P-Gf(;J(&;d7qTPeG97)oq&0>@^y<3j5kduL`Q?c+EOrQFcwlqs2K~hMds7Sw%$CwYLw*|*&n zmD365h+w@`6jEZ%LXzA3fTxMYqdm(RPhp{aYI*8?QhU}S>z#W4C6OY`8m+HxlRa>f z%F2RIPbXid(n%%fdI(bA805L;xJTZgv!ieKB~wbxjiDdz8-*N%b4yq8e>~BXpu8GD z(sjc4^>xY_?kpea>~T!#W5-YWn)}y@LDss3=1XL1vfahic^kh7Gep(F=x!?b8Afc2 zI3wtA7~&vD2(GlDunr+ys7yR-1VUYeCq)=cYbb-BFx{e9IJWV36IO|>aN^ZkQBWlZ z{3|YbP!Nzhf>nI_mb&Y5vi-Z3J!v>eCfTYZ0L{#O>60jzEwPTVUB zX6j)O2iY-MT@M~J4S~q=1f!`dO+y%neYxl2;u|srlWE)G$2^`WqGJraL5P)+?{0=& z=WrhiK{C}DFEh4}8Elw~lxnS!>1Y@~PV~fuOoV^3+*d3gaY(JStI;%~Z>2}bXI&#- znQbRM4Dt$cnuuTD)^}fqkcP9PUcj0Tdrtc91Q>eVS^7=9W&Z$3?Qd>LiTiaG>o)kU zb<}sSZI9H2efmdEy%iJc$LH>+`@EqL^V7ju3{JYqC1HWNZ*7Bp;{v<6yN@rTyS{Y$ zFmV+VIG6IDx~I6LxX&uAvK0Q=voc^HxcL>Ho%EJVnZ5132*r;O1i;7>&>+Q;`VQ=x zDD*A&y`6w@bS(zyp>%E@Q)L3VTS9Xxy z^jUs5LB1{-*W+EnD((OpABPS;1h!^SiJ*cPNiW%YnI?$?Tl5ZD2UI@|Pso<3#NxE6 zY&=8w#E%U^5C}Gju+I`qcCwJdm2Xi~xT0ug36w!1;EUl0t=!2`;6Bwy1koI6HbbNM zzLn2G}?`4BqTXu(&Ab zAHY~1%~ijj;*Y_-$nM0}+vba)&MkF5cg4t##igNRb(9kAzI(5ddKex6ev?zdH6JEJ zjB5}>{sa+~f>TK5Jj%fk$xaGRMt;ZJdeu9kr-gAbQH*RfgFp%IPi^`0X#Pp}bJovD zzx3^?dx{K1g;Q^Tv4TVSyoDb&Invz%#wHP>GXv6WtF=Tg<-MNHgo3JEShVr>bMRq& z1X?Lu{11^`{lk&4H8NUw$;V5CDAry>tHlSnA*dy36fm9{?avIfMwbD%|h(2S4(9bk?~wB6sbcZXUERnM+5;bt>&mhjpI)tk;HuT zi*Q6?yUy=~Q7h=&TE5qE5e)ATG&)SaBy3oVS+aPb{Bt6iDjCAhv3~114%i;(iBn4G z77e^-Al^JJ@?ims5qPM^`z331hBoQR<)glZJ;b+Lp zpT~=8!FZ*Zj2v}VjxT!$WMbGrn_D#fV^+2u|&`{b*cl(V~i;m z_n29xb=KK4iwy%UKyGc{7qqQ=aE+sny?R1^*&$kxAQ=&TUiCFz6$gRX5 z&AdT^MY77A2uX)*vP#Fp=JoFEa0sPk^WfGhAxv-IQ}*NspkVZVr&-ACn>7jNa$8DD zML`lJ*__ZA2=RVB6`qb@vL;0iPsvTHcN7M6Vlj-SaBUf~-~>r(CEK5CHcY)8#t_ad z3juE#y~*lDj8whe8Yd{=PkeaumHK`PmjJ+xLj5EoxBOrhb-w?({E&nB<8NZO#1*t)LxVx;-{R;8G9czSS>^C21C zY#k|@xB}pIp9p3VOLiwXvP?zfbOl(AFsG@N#s!%~`{$cHXsU^*Senb2lc}L#Icbl{ z)*uGGS7V|(Gc6-p&?pH$!Fxbfh?;g2;gARkyjK+^;{1|*Il`H#&Pm8!*Anb=w|U=u zU-tE;<3Q`+xOBHfoqZb`haDsd&0lzAOcYs3KQ5IyHwBmy&}72NBZpVvATlH0_4FZf z_NC&gV4TfdES+i=dV%&r<~MQ@Y{tELu)k|lyQPO$0(Pco5TFLeo^_O)Es^Hk)!yNrf?zoKh;fG+OMKCMODe+oE2tG;=?*q4O$;rAtkKBb(blk;h@~ zHL6a7<3mB3@I~=%C1z90Hss?H0y0N7pLZdT-KzZgM~u^OkziLn6PaX@2%HKMgrU?% z^9P4|A%cmiV`mSh&z^G^%hehE^QiA4-%9oMBr_Jvfr^|21x`WVRsVo?Qj=~x!Mx0&*iD56Fvs#PbbT&G|cUFOJw8d zNK5nEpgKqTRQutynF$M;wSAETA z(w+Mq0S*nL-bAR{{OUU~CvBosOSjhrqik15u{icmc=&f6+$1O}12_0hXZXM9lGk7; zFhvJEm4AHtFMNZR9G2dW_SXx4U(20l;R^`uv!G&)K zpaL>+HA?@BuPc(m1*mU7-K=MM9`hOTv<`TB;CYrxfLi#~-HJ0&h{gVc7I~HonaUw^ z|IGO;_#GK7vFGfa6rPS(aVFs9)Ss^Zf`FU=Ofb3C@{mHIhs;G(daYcVTz$yCPahm} z@%F0z1%C8ohKq;BzNB$HtlQFC&9fyTvER82GD#$(5u1HVjNZyThU z4{z_XZY1tEIlG}%fWfL@D3BqjiDDy{REV72`OZtYIWP^6-Fo<+LzUMaC@->0Ik(NP zjPXib-}}wBNOTr|fxe1RqaUN>IEkjvP^H%$`7?{DG2$A>rL6jGa|-$j9ZVhni#0>h zJh&({GIP?OeN%A*U1|SC7C{YC9fd|=j>WD-+H|Yn@6)e8dV`*W%BPl=NK=4O{}|QZ zLW(qS3b?%9fte!}eL}nc&N^eLc&aoj?fs_WzuF9;eogh;K&aFzy#HfMig0j-uMLV>igQ$ud`n>YKnHId)VtKsBV-rr7K*bGDc-N|IgC+;bc_j zxOA$r#@F-Mrhakg>Qw91zBJNqfRp@zWaZVpveUxmB!A1-*5ft1CC315l;l$U!X0yw+Pgm(p3mW;2*jFoP=zm zyl_UyFU9|HxxWmxlD`3G;;6Df%h;F4j#Zf4GMe+evKYemzW)O#LFe{u>F?>rcVgiG z;9zJXZr|@`uRUHes^dA1rsPmdVXVwcEDqFb&`kbtA1UL1ti!@uWtWlS8LC)~$!&nH z*&TL}VMx|rA7VJ6YLPgc4n}4X{uekV{tcMCBWM@-K0?8iF{vO|D|I-@o@wP(DG(ax z0?~LUfX{oCeQm!AkAm_ZStPP;>%}3=*{gZlJHc zdj|j5@mqe%WEyp5;bT;tdidsQx*O7E`RaR}k9o}DyR`TV8+kx(3PX{PEz9T+a zj)DswX&26v1yA&gR=2)hYl{Jy@9l#x*XfOnhR9A08h<)A*B7ZA{C7rs%Z(Z(;5OWf z4AWrO9y}q&{Cc~dM&%v=n9#C1Wjb98?cwm9Ay77P4-q_LyE2rqb1n69)c;!re1+7v z&u7sr|5<`wbNn`9M0*}1^ep0?-8OiVk%DmR@s&`mbSC@}MdB28S@w+n!b0HuSXSn8 z#_VqUb7R_b2fxDoPX~r6Ok;+PL}Rwan^JnRa6~6PAl?(wL`HM4A%iDVMhbKPdAN`k z`UiTIOk@p@+xl&b2ic51>$oGOsyBzTKOj+Vm&5fmA`OAwz!=A49h4p0gbsq3A-s)A zR2sUo;vOC#gG*VSKTrB%xDz{e4toi6hK~KJH(sVv^TV)W_{(?VK=YMXl?=My>m`cj zLxnq^hRs@!GEg)hV~`%A*)r))%mzT6UXNZA@pV!jL|z?seZ?s<9e349p$X z-Oe1LJgrbldwEy~G%xV5(^vmqu_8b8c{1G|p1fOmJb5iVqbl&};$k};h@i&92XMgYVM}A5vzth)-iK|Uf@kK?9BFnm% z7TO;cdr)UT*y*+ILu55RxwQ%VIpDSKe$nIdyH;e>p0nz>9o%{NcO{VojuJKYNx+S3 z_Lp0J@B*4|s7=0x0zs%Uurl9qn)Xl#y86Z-rim{oj_>-I$QMZ!TL24p=+{0(OMhildQ-bpJ zD;pnWs(yzKlXb2GU@Qc8(lty9PgWFjfrAE(gshCrayaCRkd7U`c3M$$yWOYM=2Y(EGy(1 z3PX=W{{C#E#fU2ccs+*khpi(aMo&^nbMhs~F5DN!elDFN=l#OI?gzr6ZMCkKHv|D>r~H=5)Zf z!eaDIBbU`AlO6@jn}Em1C5ruVwKOh^p<-Sx3$v&WA-~wi7gwaji7FUe$4GIY_u@#D z8O(j&`t69bZ9LfodGikzZFIMd5oY$@3hMq+VHKg0N`SX zyV@yhrOo*~{IA=5ksz+vB6`lwhNe@x8%orGB5m8ZnxXjh0JGK-*-4ig6g)O+-#Z^m zf7*_!M9Vs0qm439CIOW5=cy^J)lH{(2}Ohzzu8q9)$Sc^6$C_ySydb-qvs-YS_ z(J~+>;*k>*?RKyPGKRiDOsaU~wYG#1d_1mvy_#tc_l`R0UOtEL%B(0zf=R(P)I9-{ zC>@P}Yvg!&I@;EhOMF;xHb{-%_9emsiHUFSF+G{RU}s+=sfYMArZ+eY?5o{#T} z_W&=59Zfm|hG&^tl-*n#f*)1z^P3{%htsRdOmY@$+u2S_TH3^eJ5^v@iDGEjNT2vG ze(M93N7A~!DL^{`m(*<6mu&ifKG;1t@@S#yKs$^p_we0=2S(HqHXc0g|ABzU#r6r=Q@yhxtx2I z8mljC`(Ywxzn(7kBhG1I(DXwLxDC8OPxIswKi=%>ct}RlqtbzXK~yCAWR;>HCrawZ ztaEHxnfs)lv?PPaCh`f$1kq>0Pbx$$T^-3z?q&zR&?aJdrVI&sU4B9#|U6#vl^ zR_H;CA&kO|X&21=d+jiVz1W)lv319avD4^R7~BH=l|#T^+2YFAn6-M&k4^H{uk}5M z@=g4JFO>Qa!aBpp-Ar9hc}sQlk@Ba(ACXfdgq-Hv8tYt^m~0K&15mmXT1L^icZ)lZ#yaBXDWh0GTi4a}ypn;JZnH}KU+R0;mTR2^TDx`e z%5bjCpBn1h=I4Ft?E97HWqLgY%xypBzc8C+LI_)`Q`Kv&mV@`*tY{TPcZZk3=rd8H zLAaDd8;;-KG+Gl?rn>Dm5ilT$SzItxNG9){xC_i~5sxQY%kFe&)Q5Q{q$5bjuIH{t zF$P#tQNI5Cs}d)3Vnom;ry)m$%71c;m&<;}PQc^gsCSDH+~>$VZuu~`xKZllr|-!| zbyS7cqJuRykJh43SQY=jy#uTq@EPSpgbZe*Ubw^U+-}_lb?_)&XWnSAb9qH(eI(`A z=dk8a<4Jf~3I!%*xr8GCR5LDCKppfG!J2e!$Flfc*Vu|xkD9QXmvv8ROpWXZS#$#-mKPWW1UF;oJ?W7aZkDpWS*>A>9SpITcL0sI_ z_jMTo-}z{P%rQR_DSUzRJ^XbB^4)8(;yNF43@33j%>Dpa{iv3R?pXO+cVGL*RC;uh z6O-nHQ$=;p1Dt}t*+@P|?>jIs<2WpmY5Eh80GHo+DVK#SCelg4C1*>~;x5j+?_~CU zQw`-4oUJ8sLYL{QgXZn5v2>1=i)l#wXnfa71gP!j#q1g4EE8ZRu(^s!DGWVbuOu@Q z$@_x+-4Ex2RShrHub;6&X0$xTgR_(aH+uRnz`4m)ccBoY$8{cGy8LJ%EGGuM{hCbl zjhvU6t0>h6$=z4Zy3AznI!h$5EmToC?OU?DZ=DSUPIpHIm#F6#GV8nU7yemL8L&{& z(YV>3iOcF|e7<6f$$y$Gc+SuiR91QPnQnILF=~HiX7@Aki>a~;7<5i0hr7E>jk)V9 z0M(l1vgvU$xA36WnR(IXR$HFdatO_hL}jq+QmSx0Ct|*5SzyB@D47pE5KkOKW?mi> zSB@FgC*0||o(;2%Z(vJKYH)jp`BmOUBNNNjXsHDA6fq9G%XLIhKml$VNF}BWjjqD3 zV?uo54ijZ~!xED{hP9If;+l)QMr=KXk1b9jjQrZhW|}JggM+(X0}{pADj^wQn3)e4 zmwP}DZ&V_-KagAk00~aT+QNGTR|nor+3tSn`!Rhp-DKK|r9pTFPkp9`4<0NDtG*h% zGA+7)xj1zGmb!ou5Ap%l3}s1|w12Ac;H8E>$V+-{s4eSB%pQ(2-+S@8O-u2b_R*%C zs>l7^C0qN4a=*I;<{9@%g!7JvUk^V@4BNdls`PnWaCpvY-D%LCIkW4kmXCy1w4b2yLQ#V1i?mo5NtBrFlMs z7s_|3*fnoeWTfJ%W5%n3^BuJxQ_*p{i(T&B;vI+dQF6D}-$ zioh4{huWZW8N=!?w&6}u z9)kDxRn?-KPg((myS`<>`z8~gV;3)9_YMMpn`Bc7DEXw!kJD82;UJ;>6N~za{5Y#k zbX+w4Q!8lVz8TSTqx}da-3NhawRzc&8Oknc`z4VHw1JWgYQP>dG8z>+tYnYx#Z4xP z3vuACC&mVyEkI%7E}uu_C}0`cQ}$tnGS7>u+|Ej&}2$0hG@o45h&QX1`>> z!}jg^Lm}H^wZr5Ur!zrbr$b43XQkw%Qqexa{7!-wLPB~6lI=S(U!WWZMko>^A>Z$) zY?*yLu1RU*y&&~?>bJqJ!(_vU9cp@NYN^7&YrQhP`j5X!2{Yz&)J)!c@x)r^4Tx0J zZ%z9p4;%3sB|~dnB3~eVVFxF?k=MuR7QjzgU|YbEe+gS@z2Z=x0QO$zDZMKh~2 z8$@gUxnQt6uWPLR)C46ooMmKsUVPn7QrPGUERl-Fnl00$*K6}A3%D4iUuF{ow%g5@ zmtSu7hG)dj{wh^}Z;=Iss(QE^A?lJ-YZFS2bIle65x2?m==Jr11F-VplALo~*s5{R|RC#(v2{*8H6z z_}AU1&nO@4p{%fsNlGUY0eECAt>PxPT}6S@hA%tslSESmJf44eyOB-6S;;2we` zXwb@Wn=kid8sjGLb_ds)A$HAYY5vljmIU_m)qJ0IKPcd98a1SZ62>;HlJg+VMM{zY z$O*{%joOS=%zNdylfbv0Y#eka^M|T-H6u4qhDuDtZ55x2%3^LDa2`2VWuSY1^MgI` z{z^RzF(rh*<6?}NE9lWD^h#Ws-2a4=mKJod2c|M9T0r!WWF1FCw6;B0CbKaR~=R;X=;ulbw&?{OU`l5ik$DfPFibvDOC0RzZ zuY#YSyj$<~@=9E{hnDGMjw|)*=k$j@BtdCC>anzr@7eDq#D4>m=A#4UcUAb-Xp+S^ z0`_vOuku-pIzT_oC<2{Y&XH&vT0-yq?{>2?-iFd5I6_%hlEIL2L)SrkMMdZ=dodck zVo99r*&oG`#|;fW*;PPO)avxhe&?%r`--+6MImU3PSS3i-)X<7M(Aq2HcG2uir9g8 z?Z!wD?hQh^SDJN;LrIdvVt{RsC#cqU8yAf<`KW0SnSsNFK3k}?U;vTXD#PVw zICeR2xl9~ETQF1~+AsZsV*+lF;RAEghtSRttqJi~=Q4fNao1ZQTo*KzXuE za%!(+9$csv5Q8&s9bwN%-;17G&Gkm2=+!qt;&J4no#x&<@VH|1Nl~#g@hxL@D>OyE zn89v3EaAgYI1u$b(mGtK|8&%c5G8W`RcftAU*Hj5JhUcFA3SxMl`oFWX_ z>nilrYsK-C*#R=1n0zv4>S49eH;S}4{|wQ0cYyeCs^Pu9; z6RAG=WHLI8O;JlKI34NL45ciV`5?eS64n(W@he*xVp+7AoSf2=_tZqQ!ToOnJhqY| zR7a?bvg>}6IjbT!V?~>+NY6vkV~-Lsu6skWdA80}#kM^P0>><{S%QBP9JdtOq2so| zVYpC}r>%Iq_?vw`C)FcHnXxKBN&beP(~3qCgGC|{2n!&TM`u&Eik8H>#iR_2mjD1s z(xc@~=D*eWLZVFB@kT%X^S=Oon#oc>03;Kk9GF}3zUZHrZt@+(^;EH^b><(D1!wk7 z^y)3|;}4e!b`a~V25_mO@Z-H87_)>iwD(ilqEbvc<)sUfD8$tUR^?k7k-!=3D=H++ zEG9hk19oZV`y95N`T*pgrPz1nx;2H@B*YEqq5!HR+&I~y*y#5)4HM26(_bpHL7%5l zCDDxlRuFNzbE|>TP|Cm^egehS%7aAG>oO&h6Kd?OaSOyrX2F-LAn7~Vi-@!TEA1R7sHg9Qk7fMCJhHMnbV3+@&mSa5>7ySqyuxLXMB?jGFT3I68f-tXS~6|muGt=F(_u6Z%rvKDhi0DX5h{`7q^dE-NtL0y~sIe%lMzqn+*5ks3&YH*N;tq`SZR@HvJ;j3Yt@SEeGM~-D)CMlj zo=gsqvQkkug1hQKt^QrYRT#eMH`hb^hDSywrc!wa#sxd8SwZPPS@oNVl-eAW*h^s- zY4)_(+>cEhsMJW~&9F`GNH^2YhYGX~2a9sJec-KpP*$7DmAG_1j4lcuC>A-*XJI(3 z(yWPT*>`eOYgjlue89MRAJ_kAd>`syqA<_8*ZR_I zp72lnu3a3gPo)z#+(E19C=@);Y#d6MuXU)T@f?@3(X|d@`o5NlFds`VjWVtYN{JGhbgO!eYsqi=U#*2Pz#Roelh7he`-^M}f>i3J$Plkm?Uv z4*%ud&!qJhi}BO2;W~Ssl-Mk5zCZMUCO%?%-EaT()knO%v~8t8$4gU~YGuB7YQkVh zGVATsYbmK&Sd{B(!D2}9q~i65*~Dm6XTMmeNa_^Y1d4>t-IwZ5LEMKT3HxEHu)Y`@ z7+s!JB>kV*o<4JZI@|BeIC)3vfl|!qR#!|m)XC9EETzg&&f%yE!xD$u`Ia`wX$s1W z#Ek1+LdziQRQn|Z1N-AVN)M6xi6@q_f&g_ZnK^%=!uA zqcGZ%lNVby(UIz~>^&eApZo6Vhw5__9WvqC^JDsKXJK-ntPX*MCREezVbkQTUy4*P zdoPjc9qz4#Q45hP3c)Y+6aM^GZTwHxOO-9GO}5Hg>7vHg+H-d7iPqv`n6FEBR$JsMLlTK)qQr!NpCRKaSS<`sSnkD>xt#rIjACqe55X044Y(`lNqPa z-p%&oYkM)#Z0+)WDw*PtY|HLHCEf*6g;@C1YiB>KQfIUYd#4io<!cJ)7Wn?g7n6dL0QbjwN|kZ*S_pp05nAjJ6HEXT+BYEkf<*_MB8kk_l(OiQ46L zRsPkFi(hFsTlaX9##CbZP|wy>pH-B;y<0z0(jvNj)VLor=P@;Ou+VBH8+lAgV|v$Q zwRth~`ESv-UcYIuHGY8jMP9*s;RaQVgYvo!N~ zX?APgGuJLLYl*$$qM9D-#I}|12L-RUOq6re^5U&K0B%xc!dVi@id8&(GY>LVSFBS} zwi6Pd&JCo+tWLzhP9C~rMe%4+e{}Y|-Mp|AVCaj)a!6Bi2^s8(+WjgJv+{>6aP6$D zyI<>YXa5LowvrzcFB!Gt?>!C;x!HI5BTLY-^9C(p?ATemLUbEFA&hYB)YEfGTA9CL z)AlFuALZJ;p$RNsS0K$MN~=a6^S(H3qYYeb{{4gph@;;sl8#pxJA1p)EWpWRvWsX! z=c?9QCLbJ`g$e81st=j>gvyW~Ohk@laq%@Dw*6h$4}}BxD4MAITZT%A5VKU8ed(X& zweTR*A0CHW7B^m9AoD;r5h>98q=GuPH{(TFIGh_E6`^LV=0XK(^~@17asL+m$wZ0V zbEUAjX&ctHa^Q!o2E(^;pe?NiF@Kgpl7Oq-`X#y}|HIMdKZ5kx7Ka$4uYWav6OEGg zxX$SJhzdxsP5S1~UoJWdj_24jVUq2YGXIW!*5E9C;Gm^bV{6|Mn};tK5SeF&!a%Zb zwyIMdN^$=~Gw>#@Bqv@IwUXvuT2cEKm8Mb!+*@c>3x^W^DM;b@(8B|?AFas+27Nt^ zX&_Gohzm%NfZ0m3jkDY&bypS`{hz-8N!YRYpsZ=FkZI$WsL((hYhRaQ*+q4N23873}cN^ zBEz<)o3tY!jhU=65?^pVEZYY1_atVcUMk6l^&rAe;GF!h{D&O*E#TUqS#L=VBt=Rt zVCx5M*ZqQOhy6*q6jY4RElTdCEVfr%zWzXx_#p(YtqgHV)___xs3#0-2xwA%3MT*{wNR9RJ+lP!vcM-ZSrIAO)Sw z6Gz!!tScE#c`u&n_3Xm66)p0q-5X-E#l_y`<>&2oy8W19tP^mJ(k3R~T7GYYY3=|? zvqE)^`3w!Y;1wPTG{hgfg!=ud!Xd!vfl|-oI^pVcRj(d^lgO>&zXPb3Pe*7hp8pw1 zA2Zw*I5SlGCKH42G%qyDcdPwf=SO{PUdwBe_k;kVWCuVf zd#wXTVZ(dUhJew-xb}00;T-@-429#okIyHze|fr9OXqxj0;Elm$#pD?-}XD$U3rVO zuD&u9KhmaMf?Wy*yF<{sDDMYp6c1<1(gdp)D8(K!GLRDqN5VMb!y|1qTHE%)6U53j z;AhW<&#Q9m6~trWkG!=`^!PrxV%~0soxkDjfTv6)=>`BaHSYi!I^XuV2ucEFrn7L_ zr#Wr62<0+Ol@+fS4|3zT%Yb$;!65?3d;t(IUS-ZR9^uf@z>x=VC%GO6<6J-8ZZU(` z6V!C6)pQ&ari)diZ=1v~(<+~30H>ASNc@uS0*{BA$!sC06rQi;g1ul2q3&^_qQeii zgzXG~K;TxxbaI|Ldk|bivw9{{6ode$qKdHyftN=mold~AW#XPx zJTm_7a##9N(C|A@i&d-8jXV*iOUm+kZW?5+hKun(GwjJ1sEr`E2NM3~RlrCEjBE+V z#l#SUbS0Qvy)!i0QrWGF$2dzJ`hDU*A{$D0+{aR#Ym1%$3R3zEp09~DqukPA!g#B$ zhxFeS^8y_XrX{yUb^z-X%KiQQ$$Uxlbv!MeT4@5Ut3dCuOg^piD*kLnoJFD+p7|1wd)q)AjR}N-MXeEVKV#M*N zJt9wQVj>rg1^0+dqowr+@*gs3|NP|6w2EO{}qU00)#a z796dd6~Y4>+Gx~%j=4dfMEWk}&&EQ$#EaE|ginRA&$sT6euZaOCU~&w_QH3}aV*jr z$#Oy4L(Ty~i;1P1$2oH0YXWU$eS)7kGTL+^Hq5Jg-8&z97? zik~~0KW&k)z9_E{xYS6*qv`C03_hHX1ieDxT0sP+z6!Z(;#Xb#RIYd=p2 zXF;j=RF)g~_~oCud+$XP;qIA**sXbMoyE!X$AJBKPJZ6%-T_uJgYfKHXyEu@5>0=W}*z=#~PESb8s5{j$$A}03E$q(Rx$t0BA@Ye~)=R=h<3DJ6|L+ z>XkRvxkTw#nGFr0s5B1h(P0rG=C0= zmHx&(iRJKa1bxT#Ua+=#TI~h6+Px7+hl)G^pcaTmCH?R@90H30B?E9$kY@Zu$Dtjr z{-u%F>+xLNqoipoR@xzLu`u_yMIV%Z1D7^EuY!w&4Zn+9uiYTm`p783EHt;h)w1K9 zj%zvR*X}*Rn_%M(DEc(;^AUpePXM`&Xt~?bGOt${GN z#-%SmtV9oTuQ%FkH3()7smOyHF#+V-)I-Bx&=>YeAM$k~M?a|%#xc{0enPI-(WOjB zdFDuI~G=AJ%29}~8$EjXFvIaI$(BRC-bVqundgVr`yU}ze zaPjWC0^TStnPz$Ty4`trARF$B$VZ;@-YD~;N=)Y7j18wOH;<*E3DvaVqY{E-8|xZ1 z7y+v=HvCJ~F4;~koN`w8la^oUu8ZFRyPCGWt}txa;qho@?ULz$SJ863=b|{A`q?W~ z)0=@?zPHM?aJA+$QiN39(`ResRPP#k=^K`FtuN`A^+STN$s+8uQFuQ&8*O*ITu3t> ziX_7#_@U1sZ#~SddaAHQx%sdfbc#l-`as)dHt9N3T?%>tI4m^DDS#4_LpPo@EPdg@ z*C$5gy=@#NRvKRMZ(Kv(;_h_}p7C#FJb}^`ybqFu(0=Kc5QT2kiqd;@=&~qMXF#c z`+AO|>SIfCYQd+9pSK?mnPpc%bf)T&@EB|SK8;U6jN~eBAp3qUI|gs-e0Zw>LmFYb z>SFD_i|us`ksSPm$(ZipJ1Abp?*5G^EUm~GxJYq^PvH<*8u=ik!3h-KYZoVxVLAi& zZa!>ESoM5o+szcYPR&5=Eg{y=VWN5zFbJfs!tORIhB}1=w_Id%aDchQI-&0UT^`bN zv`)Q@##2VoJL6PXVR#d=D?vi-)W;nhSfMVhSL>aSgTMVUqsb0-(?xwwy;%M zZGbL55}AM)t|)wHU43YcahCc;C@+!~1bu^ZaXHtm&Xik#$#k94w$0U8Wh-Ks`4b38 zYrcsX(JuhSj;jsINbev76sQ$_Iz=pd#*nb&u*DT^=ZCG4AXqFc3an$Wf!euEOG0#C z&uEWqW?`<7H!M%hEZnQ-i!jFRU$-kNYLr5q$BXrGP{_Q3V-P6Jqg*H+uZw{KAs{S$ zmBfxh=tgDRN;T9pcG6+$_i7H_x)$2>eGL*8?Ke-l5{i<~u16p7eI7~M9m9Am*4lmE zjY)}BIlh@%%K#6BClLrI%EAT`|%j%Kujl;bn0`uZe^`+1ulo*zZ-mIuo3kTnY8oV zvDo)C#|wh*Um750$I!S6NTNn#TkH26s}8-c~7 zd*MtiqN|&^z5LR$W@M{ZYZIefqAQEZs7bd;8J^_lMo?QdTu!o?7%^oV2LBqQlW7LxxKCHg#Wy*gWGpS-?2U{YRG~2);qBp;d^vj; zgF@WJ*>Ptq9B3d_1iytjbeJzl7E&m}&N!kEy}y~h`%PZ1*>z<{9L$<;L$+DBfk2hM z=_ifa6C1&W!g>Zmo+4Y>Zs@bu{kRCs789TR%710oZ+DVaS+}{^F!6-qM$)6u(Gn)? zw;`ZFP@tZztwDI45yKv?Zbk_(d|h!VM8s!Tb|qL=!rkPxS*a>sq|+y$=&rG$lN)rl zH(|;l*qyp?aU(y^-o`VE=t<=U3^MW8oZByW_?gVe0czfSI2@CWq1a~BZf+B(ss0$} z15PQsdOTMz;h5my*<7`Xw9by-7X?gOEhk$G3iOS>+(uPhFMqFSAi6|HLS1i?_FZN} z@ejLk4q-pUxFB{{k8)1D<*#qh^lTIek=};E5$-^g$lUG^bT|_ZBv8M1COE4eZH*CH z3=9Z*Vv1WT1byqSb`htr8VPQS#e z!~}Xt!;AxLs_OB#_tEdy`M2<@j^I5gV&=-+I2?a{404!FF{MV`;<(#OraGtdJ2H^n zCI_7Hnr-=*RAa<_KMC&^4)?h7>2D;t_VZyL zv*JbdyRJlzNM!_KbEubcv zOw`+0fXSr|D0_1MN=2AenjmiPdQ}~W5!b`uGJh7`WgKGRec8oD?zD5e>ZR5IIAh3b zql0J(1Sc*d>$_*pUTw%?*qNZXtgvzwYQsWn*k`6Vw|8Mc24i5%`J&h5^6WA+t-;M_ z!^2mv+L}eDd=5<HE`kFd%s6;)CbJSymu)3%#*+x4HX_n*w)F-C}{9Rw^;+k{tSA_8NJ!cUBTeXh7s9qwItj;X1J?>_Am#O5t`XNfSEl(r(e z_9JZOu{{o>-jNHL4a_F4ER}{E1;|kJP*;=2(lx$RO$s<`wvMsUWBfaIbhIra>%!#f z@nLOZ!M1D1T}l%q?TaGn)9P`-*KwQ`AlgWD(ylIlSk@XY+dLbjvz6v(|C)+A<~U}JgnBIWJ!t6`!Zs|_GhFoxcMy4)GZ zz;BGA6Z7hWS7aRcnv z^zLNUmm)I=WRw}=Op1srA~JfGifv*RgQdsaba*k zu0fZUlDbN-c|LFHsvfEd?HHTd5%azj3Pt}pQlfHGmL!EzK2dBC64mfJ)ZsAIBgdU! zdm+>qbpwO2lS_`DnwVeDgR&r!Q}NTlM~yZUCw{Hl4Kbhgv5x+jy_9B2I`iLS=XK)f zSg|l2i?)zzO?yWqMz89-=`f`c4`d7)pmD->jO2QScg?0q+naW@v7en7Cj3{UEuQIw za;sd$hBOFvaS+T{QQe2K3V?#j9D>(`MgQO(m{%u-3<(b}F|cnK$+wP^1P9fSx(%mO zH7oQ|tJ#qt@3^AwsMD~H5wvu57q3|U(E}(Ff#JnB34kr--`>^= zz96{sg#zCn%zd1Bvhz9)@Isqum15R3IA96Lce@30D4)51kibxo&&>Ld{98VXf=)dt zGKocQ!ZxK1AT9|Z3(+LdJR|wvnc5S>UrC4fw|mG(3JQS$T^VS)Rbv9d z3lc7p%7NDFlswGL{r_pP{z%f!7y((-69O7w;P$UXP0;*L?;p3hK0;y6b^Z?^1q&WW z!2FgG(+JCFCH1R!mg-5vk}eaZ$MpaF7?%{d@Fn%+(|?6dV71j-0!aSDnzKr}iCC$i z(>|BQ3()xK>rgGFu#I7(@2~aHvY-bK3%!oCBHe}iJxNI?CNhn=18z3MuGr}uPVdC3!k(G^}ux+qA=yAiUt{msO zzyC$`L%XX%T(7pzHBeZ7Lkd9Pe%Uj{icfmK!Y=E7KRD}NB501il~NX%IL|FsTi*I` zzOD`YKRun{PyVnd@yq?RtT80hc^fJ!e%yR$Y5%9+ZfT{|fY6=K-n>aaSNx9LK7=(g1IYk*c#@`jS0}mlF43zTe!3-O~?R&@@0xG^p$RrREhXjTH zcXm3!rK<)~+JUWsxNU%AmK4U&EYbRWZ*d1$0viK{+nSk=)y6|o0dFv-X>I|pT4{j( zRci_OI~W6yirthfhZ+7N7!UXaP+C;N@T|rOwle`^V{$R|nabITULezkNg{{0s{@dl zAq~@v0;^v(T}UjS$BAaztXEqn>#eltIoyBSH0`8X)UE_z=~6`tJOd0zfEscU3{FYu zouWm9$y^Bmb*oB$1c*F4UL~0l`28=WrT_d>}GDjin^uYCYEiFe)WB zdm|E-^&`wD^Fx%2l_f*oiJk(;0obf~2w-y21ID{4n!}^Kmoi|}*mwycCvYB3?$#d= zkF+Ns_$RqDZ8Y?{-=~~`8Cn2t_sj;2qG$s=I>l!#ut!n-3_=Fc(fK?aw`FD1KDplh`f9w<9byOI=0gCq zzP+Wd;YViZ(lsM}4?L3u8^e%pG!Bn@hq98C=O|KzPOy+Nfr+(+=g-1r2?4AB&{R@m(uB#tHX>ea066TzJ}brrKB<2QZG~y_q-qSgy_G zGmZLi(}Sf~q!q)-zOkOlVa?F6>iHvoac}zywCXV0vppoG=0Km8Tt2Sp@rS zKV40!Fg%4&NDwo~(Yai~qd5;!7fJwSj5=i1E_wbtQDAi{wEFnci)*dC>gAw)XZTLPL;esE_$GpTTF-)nnzy4L7Ab_CLil%_WfQA+h2)h8L?|AZi`%5u+ zq))v!)L|-b4E@AH1t4!Ervuh(5J`fyYLyD*QiH;tHt1=qtm68xXuOp5Y*sqaWU){A>gdV#$;sDnXO-&cS3*y$>C&KqyCqZ7J0JLH#x=d_BTH#5Y(9C(7(Vj z+kw&EXCbnx_6!^0Ft7Eu)_u3Y69^W}0o6Gcm}B*|Vx^9n9ddNXzD*Ux+5%pASyJxc);oG&shh4Qr3(Oa@pl%l;Xr6~Mv0aDLcwsOt0rky@ueD>s0?&E z>g~ZsZUCgjYJ%86o_{t-TQSZ4-DtavUu(Itk-qKWga^#w9fj_J84Mo>e4FJP8Z`iN znz&!!AATI`Vu;Un!STs|qHi*`DLlVpYW@wwJ&=Vjh8ct=E#`1?o6SLvZN3uwb z!5v&n^~V#kOm1>naoMH5+vMy9sS%^UoiA)9Kd2tFEwwokfje#7M6Oe2vzV1>b~>g3 ze88(8Z`)pG^u3l#yYm;0hIvjKmA*7xW0L0#YX)Q!qv!-zkv`K`m;p3e6?fphT5)b0 z-2*P({K}2yb0AfXTqu16bk|ZekOiH%GXm_VmabfM-4G6McJ`b%I?n!%hq3C4REn{E z(dv^vQd%>`^bIA|K3zZQq2Vw~eKN3-Zxk!b&(J-UXMj7?-opzPlavg}w4W8o{5tIJ zAFY2(^Vzo&n`>iYsFj>lt3Ik=L_5_9u|%mwn-#^^#6PmSKI(9Mahub)pEL)mnJm`g zqKwY*E-!OHkb*h}aZCJqva`x-j}aZeavI-tFNcP3V~~XqB%b)n01e{3UW%wU22%Cw zg>B9+6WW0_>S3*iaQVoP0zq}C=zL0XwMAkkr~VDX5{rq#)0{sK8JpQ1N2@J1fbmBi z^{Ts9LRNd>w{37jOwnfzTG;at^f;>XyvSpb?A+=iQ#%MwPq}Uc&|T6*4)&yzR#ZQiYFymfE-2ErfrT+I&pNCe)1Oi@cu$ z@(Z7Trh8JxrtCS(Rrlv*qO#IjJlK9T?<3B>SvvSViwbr;Zi>{5)SZeAOt}49vZw!^ zC_4C&oVyiA`HBMHheQNGF;qXfHtGLc%C>>7cLw4}tMeqm3CQS}6k=($z=f*6Ot${= zO<0AqyufBlFNmRryHlN~eTxF^nSPe%&U&wu^1~sr6rqYTVMr~JPwSYNZ}AjM)po%* z(T19!aH#HYvt0>A=+N{c*)(o8u^*W~G$XpgN#*ZE^pBhCSkC%(x0_D&qI+o7^DnYA z^hNqm;;j2nVoH!3wTo5yT|)fMH+zSAlC=pg_hbqgXJ54x--8F?vA@ljL8d7 zs7!Wr80daDyNSYlFj! zyif0XZH=RQT~Zsnu{T@AjNV!@F=Q*|eL%88h=h^_TkH7xDs`)7H(2Pc(`ZJsuzU%Ps%8s?|WZfQt}K^Mq^ zPRuWDpUTM?Tc3gIS|`v+VS9UfXJ8wEk;~o(NK}WG;y^AdjTP~EV`B^;$U6hy@=5ZA?h{bkL9g2fYc`EeH* zBEy)lXS!|cZP>a}PJoLpH2;xYWKg7o?bX;&GFsn>p1<#`LeTGCc=V^wySGbey>h;fo3i9{*a<4|kAs$i8 zq8^!TvFK5uuxfp{WDO`c47!6+cg`=ccu6zQ9AwkjM_P1RR>!)4ZB{xvw~Xt7qHAnM zrxq6MiuL^T^joK2j%r#M;DY1pKpyzg*Tw;RK&|C%E8q|ACQdAp+pPkEKm~t%Rs{8j zbjvu@$#CCDXWNdC+e`{k1|q#IMjs_KTpL2t%qAj!JrS^LJ1bFq`KiJO5L1>a9riz> zn*o$oo!vPT;l$Z%q1(Ulc9Yl3%t z07+!nEs)8`05bO!ka<)ajDLi<7yJ9EI1xd>*WtspRbY&;d^{%rSXp$5t~LM2rk&(HFJXfFyA+R;k86fFc z5{3}PazV%LBs!%)n{LWi61_eJ-h{H+lm3 zSjl(?FaX(6!?97L{6bL*u`6b;<)I+gqpb0Q66gUeP)h4+0v52vt_Xpv{Nv5jkl(gM z3N7ZP$tmNjFhRy&#xttQ#?jcRq`R0r(;__0^>rzCA6$vz_HPi&?P4Ga{GeVBfWRi zF0yH4doW~(Zt#>VKvS`pnx{lk|17&B$M_)JW<3VWKVg*&0SZliF*t{`D;mO&hs{r5 zuvX#{`VrM_Evjnylh@=ACekmDJbl|26iBop7^2yl(2cuiZ-5H&aeW&h3LGwnC1OO^ zyDlJlBZ%ZV;dIMxC*Xd!fj*gmF~=z${KC(9gB;q1(K7Cw1q=&IbjkEf-Vofc4z~QI zML=90)CJV?tusH5%?)iO{|3C`0%!VmBkeNU#(;fNimx~%T;O^E0Ar-St0U0P2V;)z zH#kKuL#87jsR%)*34cmXypDw82V<3ZT!f&2zQ z15^ITXX{;77R1lDHW4a2;1zv=&ulXPjBsG-ON%TPgF3cqg@)BeIx6JGQ50OvtZ zq|^A=j1}BPLf<_KaC*9c729e`J~cK7zFw}FtiCD>HVO%sJ^9zXa9SLSPxWsB#LM2^ z-dOy3@P#dT47!{UZ*=-T_{4xz_uor>Ty5jOsyMZ7GlI<{4v(}sZMi=vaXLNY$zxjf zd-RGr)u{Ly(`x^wE-Mq6MumIKB6;>Xk6!y>BlT>qts$@~0%<*P6&YpoW}eql<$H^j zOXi~0U3w$E{po`XQqd~;quuYuRz1ZT1?Xl;@6n*hih=_2Qr(~2l^gR7xlNmqJeGTz za^a&1hb{caK+=fhoCa{t+Rp$?M6*zOHzHf41q5t~g{SLzXMAY0pTE$J6VqYC|A6cX z2&^dJzR6{nFHKd3zKg=Slgs37B14}UYQlS@4Zt_QYGzF)K@b!R3mNi+WrlELee~yUHUBc^p3^i?3K${g4tF~}#dX8mvK0uk3%=rK)r;e<>VNp)7PS8&pEXtdD2Mw?-ya2{W*P@kNa^|DzQ+O0Li8mBC15Ni9hRc;H~Laf6UIz=Fe^bB>gTs)_&=H zIl`fp3Pf46?6)6^RNhR(E6Q_Q+AVQK(L48ogr=juQrgQ`2_6Ddon8K|Rgl_SVs}8c zsKp%2hNUWl<8tu5@TheYiD@|qj%|g&4{p=bkbpTCAqPc3b-JP9D;cCS)+L$|dTA5o zcU(lnm=8J2g(ohT&2}=lQSaCW7h~NUZ(VEz=VM`=?6MREM9J?!jk~~BRt!I|)ag{W z&f@YJ?C3xJ0LLkB{a>x%b;EV(Lv!QuJ9F(R?>mF2oLjJcbJ6knTx(rs_qO(0Ud|o_ z=4p0Z2AtR}URq~cG+HzxBI9R<;o!JAdCj+a#@~r#2g8cut>Cv%A%;w*OoUtv*8h5a zYO)fErR$tiD~pr~ccdRPHg@~lMPm@r(ST7HHbz)H>K0lH^{aExUJGiHL(wm!N**p%yirY=D`YK=1m7blU=FogHB#Rkh!=)7r3elosQlqse)&Mot@*<5Cz9p2hN zCzELE3*W?RILDT*bXvQ;8GJEd>)12R+wlZp+ zjj?A%5!Yq2Ma0}=U{|X_FF*8{!+BXFDurTbk6&xajpVmxPY24N#H7VTbG+m2!j ze;DCFZ>%6$Z5i-a@vlnCpqGkZRzWBKWyM`c}FUu-JLAv)u|IQ;+-KSuWF{sgHZ zN_IK4GeGe+!?PX;t}sj*_(u1TzQGm6biDRH%g&Gx*Xo7r-#kOY+9){wur|R@@EiX} zf6`ZCbWq#Y*f^UZfjsUhUn@`)kkuyLLFGkEGUB*}HC_+c?mAsA2{xQu^9eidrs(GB z`Y}CoD%L>=K;5|wWMS`55p-)fL0N)#TRnIjkhvfOADf(- zC@7LJLCs0xw^g+GQF0g6Lku-J^4wPPeucrcEnZ221Yd$O_I6MviP}Llh!F3^;oWfF zQ7~x_=e{x>;yMVdrRO8wom@#EDN|K;+`ERmT(sC76AuzQg{6 z&t5j28D(tB2v?*8&i+C?{(Ri059u7yEl|QmaNQ^Oqj$loJprZ1(e6_kd!NVvUkC?Q zMFH$!kDKgT%hKUo*wxQ%Jrk25QnjP|PphQ@&(!wFVzLU-j`ZfZ17A@0tbo0n5Ur~8 z2;&y^p$o}Gq@6pl*@e5;2Mm*`kY!$*3bS?@SwhRgcAt@>7p7o|no1UqEz<@TPWlGR zp+})~BIj&EM%+m`=m(2Is{C?`hN{wC%+*ZCUG~6ikus`41mqB;-0=P zp-mWJJGxj`0>0dYXcj}D=naby=zIw`-{;Z|T4`y?V7)W2U(_9~`R$Wq0*!nLD7;*F zUFqJOdwb_ReWnfcKUW=1=bBn+6n=MM+K4UhUYWm;FqOo6J6C4X3=-YtWUA~>t7q5M(6DW&?LJ2Adb7V`3AlDF3-hDxu#t|VvS5V{+z*T7iR4ao`gK4%sRnMK8hqr;po}X*HE}kff->mQuE(`*VIEH5 z>noq|sYZgFGaX4c&JZBr8`M`vWtM~;7!)162htrYY}5`3&;7NGBCj}WQwX7fPY$Bg z__c+XtW3EpHu@4t2^FRBVuEMoy4;?YYZ%0Pw+n8cNJfF}VpN3aB`zVa}^8 z_p6!iIO3#5;^6R)^bg;(q@Fx=e}E*ATSl!DUV!iWXZ?wjht6Kk;_ z#?M1WBqu3{hci7v+BtN=amtA!41OyrfB-We>Can%)w`{tMJ}fPw&a53u)K$~-_@SHPSoFsR(zd5RtYaw|Yatm5-TdH_ z%r6&ggCar`9hf6iY)sXa6*@toz09OxW0_Swo$a}$^4+)Q6LBDP>^YCQr2|SgFc^?M zhuU7!p-q$d8x;Z(VPay_Rw$dPFAVt?4;#0V>Jc-U80DUDXwwJF`S3M^y*?gYZdre9 zL*LO%0|{z8!JgzvL1(gA}d&+8ZYdykyr zgy%7rU%$P?3&TEV*L?gP^2&1W!^Y@zg&jSnf#)!YT%^?b4Qht1CilZUsrbtCxqFCT z^m>bYwwW4y7}@x3SaUoVigoq03SG(I3p-9|*3c z&UGEvBMhL7UY^@=>Wwl)5qhCV9V;b&4W=^YWE%}?)JYfuy|?{b$QssIx^nZdA$p~D zpOre5H@j3!^Jnl0LKjOC{Zi~RF08m??u0n^qZ}S{X?Ey4+TFC?SbLI0fE<0_ls>M) zpW(PsB^wO8bInYbeDeli{BzV#SheF0_H8;Q1Uk1~Oid{yVGL-q1maW`exy<}9lgVA z_}#oKt2-|$n16TbL~nty-P#syn@wh$zZ#*EP1GFsIA(@Qtwi(Qydh zJ}cgPwXoAnxM&AY-G-~Lrsv#05=SY=n76m6Y2_Drovg=doBTD~GVTLhF0m}-AL1L} zf(04Ln2L5ERBk#K`}57w_d@22>LYAj_)tIF-dml!SyvipO^djEqKuXcG{1o)ETSm384&Hn*>;F+zzgkHo;Tcu?@kT6Rx z6AEM@Ks@#5!$%aYRCJv!N{;#$2KMLgJxl<7`eN{r><=vVr>Te^XgYYW7V^(8Nyg~` z$nIBXzQn&x3LZeyjA=QV*dJ{1&!VCQZTF^bu{PA7)>pELIh z5)~>;@tfnQ(-!}Y3<4MVi3I4RF^zQ0zlRlsW&uSj484+8{I@MT6`E|iSW?S4 z{4iMXHP9g(ed_-lOgRoX@3&*F;stdY>A!(3(d>G-t+(cNFw%7x|MbSk9Ic;vx|UCz z4OjZ#Lr!D*qzjkT>wmBR&t=PkFFU`i8hpWjeTbld`-NEfR^@*U;PEXeT$Wm^lK7ts z1K)i!w@AcNjQ@O*1r<0iOF}{azh@wV!~u73F_RR^LhL_XlmXvd8Dr||e~l+s4$t5=Z5lA=P& zx)0w7yMEgmbrxn-yt!QUmlDa{KlD6ma^n`r;yx<$O5V$$i z7<0RK0GdbVqMn)yFE95u%Qx;EUvAtV=Pxgf+A^vRts{7!|~tK)Yd&=b8|0BvsnGmEFA_0tP`gX%5$@2(2{}`6gM|_ zC$N?Mr!4cNvI}7Jm?sHF1-$DASzJFoEEPJT7ViMVpiVHmyqtkfvzF?prRFiDZhWOU z4-Oc~A{QWa>Mb<$1eO^W@Xg(mhF)A+>dKdl6SlCh&?`@zyeA|x%{#C^8qg}#ZVpTeb{_p75Bq7RCl`bo54*lP3dK(IiqMTIe z-hW!A(ST8|X>K+1&*;ro0OsJE8EqET{|)@?L^ubFnca;51dEW60~#QY^Mk(fDu0cu zqk`!J2`Z9K%y@7!c%xq~^VcH#2Z>)K&dwQIG_#md{^v0I-~-Rf(?2bBBEY0`TGV9wXVOq%1AX>oOCJ8Gr3fGL<@H-bA5MLh&Hh)w OA4xGe(Q;vZ-~R)7B(VYj literal 0 HcmV?d00001 diff --git a/images/adb/instance-principal-1.png b/images/adb/instance-principal-1.png new file mode 100644 index 0000000000000000000000000000000000000000..10ca10b4bd9f04a3b45b84375cf47e34b28d055e GIT binary patch literal 76459 zcmZU*1yoy2w?B-P;7)OOclSb(qQ%`qptwWP;85IM3WW+zad%0O0>#}a?hgO-dGGVy zyS^`Loy^RhEwlH?IWzg4L~E!iV4{(t!NI{{Dk;io!ok5m!me5rWEce=&I=p%Eo&z$ ztDz(-OReGNY-8sDgo9&FvNAWvQ(|Wxvam2W9~xz2L38ugjEaiYG!Gi-8l&#->h0>L zPRcYkUL(d?L)7kpD}LYIXpa#~eP7EtR9*)Z?4|3|IR$CO96Ub~ku#ALkkwM+~(`5G;;3Mtekp77^92_Yug@Z#(j(`Ke zQasotSBUU$E&O94;(znCe;H-8WR;X)sg{)+5a{G?>+Er;q!bS8uo*jTeGh$A6;Uf^ zM@|cCXGg|7mtXD z2p2am7cVadjDy47*U7`ehr`L8?mtTY)guRVw{o*{@vw7tqW-Jb!qVB(LxPs}uc7~b z{?ksNkKO-Ta&rG4w_px({jK5R;pFD}uWp#A_+P51hMf=4L0`_!5f&bp4M_oRVex_AYwS!YL>poirD#`k}O|8M30SMVQ`2LEf4pO63ln*2XC|05~R_1E+N z6N&$X`5!7Q&XQ>2T>tGeNi^cRK-kQbhEtN0(e{BqGy(Xh{k{vV!$+V&h37?~4{Nt} zPAuAJIG+DtZ~t4@-u~g_=gflc50ixj!ymAWV+B6>S8Bf#LO~iD`mn?Hi=cdW_nz&Y z6=l5S#IT!F^qZ9vu}z6_kyYQ7i~O71@iHtKerAN2VA?_ySR$Pv-_@u*tHxILxAY$( zGCYI-5z4g;VCCIk?Ss;VAZ>?19JwzrL9mxhA@ze#vDM3IBvDp`MxE)A*zXJR ztob8fV2$5J0W;!C4nOA&#lO8J={I?(F%0 z?YRH-O$5&5llEV~{@Pw3gYjR>8$-1IrHsKNs2cW40o?48U@>vQ!ui_W`t!T}hcTI9 z!xC8Ye+nSHyLF7K-l;SKYY84k^VuwwtK?G^XnHLM(xejRgx$(JSohNLVKFR5B{?s z)GOj>CwP(m+%1of6Irj;vh%0?4Y$Kz1zrzdJz}I=X$s|5oiyxaV!A|Rq)}gTyDGz3 z{~ZTMpMvoMA2uq!u>4EeMS}NS5M!#gFLI27lNfI%IO5h4d;KNg)az}B*{@}z*S>49 zWqytvKir>nokqSdH1YA?o4|rP_8Jz{y0;or-oi+V={x~`kP=rS#g}JSJUjw+0bxZ( zu1;yMBeisi;fF6xZN`r{3362Pd@4KRSsVv~PN}i6lY~DQ zn7CGl=GNhK!GAs0qMfkp;o@S;;9Btt6@T7zMjsf}T^!i_^wg=N^V@C8WHpsS>;|O~ zyfHScyydSqv<~!UQ%S~I~zZ)4@8Z*zU zqb?2bQy4xi3U&Q%==|-~FCTQtxze=sbWXNer$lu5xWpv72qh5|_u=!j%bQj0m9t5=aCkz~?|i&XxO4cH7- zMKXkzc2nmFt@x`yU)>v3u&B$Lg2f+;DTeavx}#2Ank~D=JxFoW?^MbA6@i%$3Vd7@^cY~bV1cdB@bhR(#e`v;p+_KP!OI|S5FTDpyJ|b z^t;^Sbgc>aQc|Lg@Q9eL%{A-K(kr0HE-lFuPiGvRKt$q$f-x+H&9}nJtouyKx_hu` z|3|>{+{d8BKdpSJTD9fXD`KoNj0a6k;BKXL^I@`Yt@(3!EeSJx)#tK=lid{buz_G&HeItm=K5(o~m$PHF{ zD7oB%_3E|Qi=e9JfTsR1^~9&gC#V?+S)7;lO^45cr|`S!)kMB%cA`Gb7?Sd`qS!th zIADpoHWw-;X8*9YjN7t+-VlH)fK@EzMxe_AkU^d}5?$_)kT zUk=w=ZyPq&fyRy_d+>N{rn#*Q%y+p$+fta~QbILE-Q1)toe-k07k+Re{*qS)0ts#Q z2{Vg!$+Q|CsAcD_i@bHlqjm%C0aW>UEz8q4MUG3%VFL)NIIuefHn5nec2dwy-k9RE-nTjlM;Ap>a&>m6pND4 z%vVO@E}(im2BrY=sw_Wo|KJ=n&kGdC?`zb>DkV<|raoEjq6&K*jA$^%s9{j3vk~)( z2;L}2*5v`4GQIvlP>D2^rpkwo0|NS}4n}<5LM2kPaIJAzYEyl~Q|C>DAhV|3ab0wn z-}NgcEyQ{pn1yr;*EiVz4jab^D>ZyG%buz8_NU?Hq4%0=`yPYX~~8 zIG_|9DdGWFDWn41Xf(uboA9dJ)%L}g?f-c0bi0Y{osq0{DSRS@xNt_E<@TO!4gw+R z9Mx_8Sw8IJT3IT)NBIW<1NGZ}%CMY${-fi8)zti|VC9qJ$%SZdRwZA289%6H@F72> zQ9Ntlt|RT&Rg3#drE6yXQ=FqZLl*sT$67AeRBP!)DpQqMnWx%r?!dD5u1A~mREue& z-BG}EgX^u*tNXbctI1Yx=Y+6vpGPaHT|S?!A>86adGnr+rT2^Hcl!Y)-tqf8VqmGM z`>Gsk)t0Sdj}oUsGi@x)i_oj4N9nX|2-cKZz%htEMO>USj+_IW1DEwVo#(mhnroy0b>e zMSS{Y%Ct_TVXX6q_u(dqjjLV%`RwFH5f?=Y&U*YyzVgm!dPt9@2wYGcD~9n?q4uev zS9`4wqxizl8gXm2kkw}Qva{7pUSN#e+qV{WWqpZz`4WB@XLv?f(r@>S9=G7$vJrEL zV&vxWU;^x~5e-p%5>`bL-+$xQC0N?Uh#$e()B*!nWMpI*F)u8QkxT1BDGA~9UI3+# z@H7q%j!|fod>koo%aYVE^A>bbS`N*)v3)9fRrE78COh+5$n4Kcv5JcF9Z8u{qf=>R z+rTO` zPY0%&s>2xRsY&QOk}#4gEhdZv1J9qc5H2j*Ir1F<4S*`XYL!JUeWH#2>z}iZix)zj zb*8VyN|^-9(LwcxLD107Xggb<8@a;oT7#X53?!Uow4kAgF__@H(yL&%y75fv7Y!E#Dlr{ zy{+Zb*7i37lwZ3;kp(@p@5SU~i?61K^R_KhCX(u^n^3l*y1VXdp1iXMdOw=Kju)MJ z3@pV#;FFLQSU^%Q*SZv~Q}NvIcZc>qUiTzOPh%wn8nRr23#-~ZBp;auhGNnV3J-Ni zdMbZ>p(ttTJSjN5V3{;L1fk7vcPr!I^iM9OYNALs8Npo)5nwX*H4=lwGLTBre-3IxT!n{K;Yuv4 z*hPNZ=(uRJ;X-ZQ*|Zjd4)tUuIcy}q6}F8h=ILQ3gXX;Ee0GKD6Z6>9xb4E=eS%s? zyfDv#5dkXVJFU2_Jj9W;CoAoX7`L&Iwyb<*B&hS#{Z)jpfzXWa@lvxq@&GV@e}6wY zH`jpxK6zlRn;g+HR#PsEIMa!qz+`MiQE{=+Sx8P<TE`NLm$5D{D|#I1F#XGIu68=xLxkOx5)2{^U*>c z>3I|`wl=E9G$1lm@<}+a<#a&bP$G#&hg>xv_4qglH4si+2qA8ttaf(pbESqqh1|?} z!CK6hS;NIcA;rRkILMkV<#+9Hk!NR{S5(di8sm4I`=Q5WWiT=}CU|7dQ`W|w;n;oq zaDBx08a@zYzBt9(wR%XD8hvF$k~(~?gFinS6g$Xk7g2@)@%M2&oU1{$y8hJecSEZ3 zElS7Yj{h~9!~;jP6dj%HS85|57`D`Jah`tE)lH99_m4(Tdu?H!4)&>Pb+fLvtv!rr=j9A5?fl#=Wb(BNiJcO(9_Sv)zOXg zc(jfGEN7|`Kx^ug$y6pyo!wFL6>;C}QvVW6oMi649?9!FVogW2&466~qICJAS7QYq z4LjOoKTD7zR|tf7jY!>c}pVjoDwxVtovi# zaugzA%r~tv(zeX_zPFaOCHpamHI}WpeR5WhNbxeus(#g!?4eH~cME~t+k{3^*RN(0 zjvN;AtrbH+K}Gc;o#;H2@x&rMLtPz;20nRBwO9^JtnsNCBUlG&{Skp=>=}27u?BjE zWyBf6iI2VouJbFYp?wZDo)v7sFORx+TLG-`k2yll`O#x%xo2}+nOuum#2Pi`R91x{ zZcLsg@pTFw2pmR`OA%-FV=VJs;)n3T& z&Sk2nw<p<(zZga4=)=H3+Lh2~QzT4*- zdf7Mnqd5aDJ3J+37=R>O|1_Ym?uy>T|M^W1otGc15ez&%#obigA4bqJ*ydK9 z?5<=fifLLOZ$qB33=OmS26FlAN~xS>Wq@E;U|4BQG&+F$=ZmW^TgjG%}|;2tv*mVQa?Bqk#RUXlo8;BbxJRdM-HIF z`hHkUpBom2B)ddj?vsGX_#mc~gUEfsVCQCTMT*ryPdb)+r%=ntLh?ADK!XjTSK_9a zN~Cva9-~gGt{Bb%COKpZ0WDWBpat*X0sbTX&mTZ*ZR3x5h$r6YM>fcY$?O+&XBzUB zUAOdD>HXkrU*77Da<<{DryQR|oR3{)*3e8In+co1*uh7ADFf_#v@l7)XV0wFM& zIyw)Hgok>Eo^7O;_g2ZRv@JEI4+I3=^`-H?SKHWR!9g#j!>nF*n5U;Mbw46X4Lwpg zh}NM$j4iMuQG}*O%c)NCkE^%W2j4^kP?PE$OMylP&C#7;Hk^ z5fu@Ay`ApW)Op`>OM@}+*}NAtAt0Q%*14vFIpv6`FMk~VqBu5IeG)`E)@49>!sD>q z^1C@I;zUJO+#tXH9gg}Bq&fd@`uhFRS6>JrCbkQEdyEWh{oVE75X*De$L<)igC4G< zLlAYsF6BaX3fFTfwz65N8)L4zp>gm>j@Lb6&MemX;To)1WhF)L?}w_pmE^P5V{`ss zDn;gz$V*zqtTy`(PLb%zd#fpzPlULM;e$JXp_W3aWD$fr>NZPiZ!<7dk<#DcjUT zEXXZ;J?MEi6kmn)z0Hl{0{Etn-=B)2=AMlc=wrFAQBA$)g!Q?KIxz0eeLfg${Q)%+ zXcQiH1o=3@+kM-Y1Hnq2!hhX1_Q>Lgc8_>4)?oRhxR?74A8|HNh9oQf>@LrZml6W> z)h6(X3sZj;?#i5l`8al|3tn>`JFMuBy3r5%ByxdvbWF^b6){3dNX?4e7vm_G)&9y- zzBx1gDK8-YszINZqtnPBuLjszsM0nls%3hOII_*j%~@l<`txY$@piDwi_z3UY+&1a zYdAPAnyjvYvCFg`f=eQ|W98Ipz#@U969=Weq&K zbY=6Y#NHI!`A-$a`~5xbf`AT$yiTNtU5XuYdC_7RQKL*8^Dm+}!)bdVx!&=%1Uj`Y->j<^>x`%aX-sK0Nq@M z?)^r$(Y1ZBZJJ7UiJ*w3Ansey`WpUra0%c~@1~BiTpZ|9bC)U8Jv+|KaGm{R?b;Ct z>ggSDT2%alqFizjB0dQtR1vC#V|KV=_kyW-Y8lnU(cumOKwSsVADW-+ID0G{5{t@jp`iA-) zNfai~+$Do*+ht%{uKZ=(LfxffvhRc-j5?zb*XRm6Z1{1HU@jfJRYSpa=UR~~9UlpL z+;T1Ej+%YPLPY@gE|T>o=(lFXLpYF-x7sY%7~|xFmzPxLYNOL4e9bbp_$4`4kK{`| znIL3DfQ-9$ffGp!LoMdY8+MKvL+P7nGzdg zm?5lgqivIHYGtRgWpW9!zexINEb%}D~LSe}bU23mT&M4l3qNnI9XjS|1! zEeCu}p=iA!PPv0)Hgk~j2G3*?GpiX(&Ihe=+gP5a0@ts=z^Os7E)^)HwS8RlLI!&v z)Mx-es1(gd^+p>i&sU(0;`6yE;FdVyNS+O89K}V4d2dc}u$S6=qX<1OJ;?9boMLWJ zgl+3y0whdNpG}}d-!|SIS-Od)FD(q&4u*)j%iDo`Jj^#dlW9kD$YsS}r4MGsQCx~J zr$}pZhi(NA@_c0g*Sd8v;U=frx|*+~rdYuPPIr)Zlnm?yL8chFb~W*-1+~d4Om`ACz>BLj`S~Kk|u3gY_>G5JfXIo74`m5NE)ibV(YxbOxHp>7(NF z^0B=Kso_Th5A=4(4OR7XE(4I-BaHMT+r_D7hn>OtJ=GjRl1Ezx>iG1p4xNRE4QAC` z?~!*-NTZm}ti*7IcE-hhmXR0Zp$SKd{b8Qe5q*)sDOTy&d2Et|r?~SA-!JSQQ0un7 zKDjBT`Kt1be(~rG_pAnoonD@q>|Y#SZ6~Cw&pS+nz>QH)uLg$2+!4fWccSthh4S0n znyT}Jv%A6FsHaFz8#AlCs0{Z-;oZdz5=j|DmtW+~2iIhYzQr|T&9OECWrg$&1yW}i z*PLZP_eBFKMF&TIA8bv2+Q|->xv)_e+RA>Vn!smA`DqbF_)-YZk9cO$XF@s1$Ltyb zTrKMqj&E|_5o$x0U-r(O2`-^?n-j^aGHZ%4%b(FQp{<}mDz~{I>psPYc&|Q)CVu?t zrc8mi4di{8jDUecNVvW8)rb>YSakXu0KC74pUCe5XY&x~vX=89KEtHezkbQTSHLGK z#I^5G-?kwDit2T@N6)V%^qq~*4J~rrz!@=v8X03RG7>L7N|ifW!*v~AP8{C3`%#1# zL%+&SL$WZ|+I*Qbb7E;*33)@tZ716ASPgGW*kx@ExA@lO(?$=rRKKWirBZF# z7mKC2qzDbIcuGFBPl$F#eBe&$7-+u8E8nGDT!jHRPhBhu$r{SGx*^1QT>)kX7*duHWK$wL)j&O&1e8Yp2;0OU z*qYjq@MmZx6}}C#Q+@hDLAk+R;F_xxYOOT+MV2f&0kw=MKe0kF=+&1VIEw#-!1`Qo293-A8cnv6JYQ8W=wd_4`*D}`=W@!V$=D(l=VARhK&otAL|sRb}I8zWoq1LZ(q&C?l*a% zFxrZqC;2yJJQ1s093#xue44NuOrSv29JqhkLOmZ-P~OVBkxKT}U)=RQT4EG}GhO$; z^(hVR5G5Iz{VcS_nCgv zR^KEPKshIJueoU)Qmywg_2mcjJd)`tWF|X7%7|0X>gAEBvSnId6DzdsA_h_@e5dR{ zr7t&NN-d58nvbcMhc9xF@4Ba*Z}1-;Yq`pC{jjPXx=vYsp{HYY9cLhNMlRwOH&%<+%s_V{{pc4$LpWjX8fSOu z?O=IqUoo-`&w-kofjBbIDXkhtUwkNsYBPDCFf%VFXcC&S<9e+SkFV0}H&v?2*SCrG zdh@+O5eB(`mJq;wq_;Z{d_`?{MtPAf>iC@oFxz&UCPD#r2Xuk|+Gi1@~xDann2uZrw)^Pd_jn8hGJg?;`I zK{JdNq6oKtf@ke4r~?}kPANB4v-yXolU%>5R8vG_ZUdY9!X4EN_C1C^)^^tB_eB@P z+AkIm^H4z6nFfIappOj{kQ$g;+HOgw3v1GyZojdokj^Y7NKsFB$Dl`Q!Z^+aqr5gk zRE@nkMP`Ac^8m;In&H^h(H+>&tFN#z=r)JF{C7%#_{1sLDJZ8MaeeeLOPoKIt3&D# zg3q>v>;`#j;-!r7Tzt?IpaP5V?R(q=aeay4l-Y*kIiOl4TW_y4LrAC~cvlVaV?y0j z6Y{|3{X^2URqfv9cv#(}F#l3mO))g^{zmF?PM@L_{1qMvDkWlbJ>K?>oBPz(P?Yo| z+K=^pDUVZXaU9$Mg>~YgaD^AnJ}-$5VGs&R0R+JqB;VkKcp%enK^NZuT&wN!XcG&z z+11F1TyaEh8fyCFa>xyk!FvJAvh0by^qOA(N{*_?Tl&89&3{Mf09h%y)GNC^IHMX@}Mo9;qpANd7iqW>_>mSMb$f!MV>s@JLoPtiC z4)*I=5B@jOG|2c)kz3^nAi=}#*NVu74)N4g9E)DE@T4FY&Mu62BW^zP=3D>m5Bg{k zS?AFVfN%i~#?lR9Ys!VI zuQuSU5dq}Mr~vI1tLjyeiL-I!m0gMFd6}5iJVc1)nrA~X$0x)u?PGraV_&J~9(HV= zg4hQ|B)Rywuxh;u>#xjB%QIlhe!8p3+WEMFsi}7b-6kK_iED&okl1Yatm2zTcPv5Y zQU=L##7;g9CnLR2jdA1ZpCau8+Y~10-}9>H6sRgUAC{-j9tI>!6i^vS88x^+k2&rc zN~u4EoaHXgiAas?8ww3H3Tq;dfPl4jER9ceB+1+)8FojW1KXa@XW17L$xIJ$7uyoM zU8@Zhx680QLi+j=mrudbS`7NZ!BTo^lIz_r`E8C{O~uly6;LyD+;YgL@^0>@WRvSh z)F!P1eBepWI8_MJYIloOBr0(tpXQA~=7(`zPCjk;h_x>2x`SZ2i>wc^&~M2;1#!yb zE|leOXzNO?1}rI;WlH!f+!J>Xr|vpNqk$=1bKt|ZVZTF{SgH{G^EtWS`eK6i1A-R# zlHbTDXp-axT7)6pYC{HaCpEo3-+xpvt@3nrF(8ce9pq0o#SDC$uvT}RMmA)A^XIJL zeBQ6JAhn7#MpYzTUvU707EI3PJowEZNw~FBZ0pM_8f<0STnRoZPO|yyYR_6e{2VDB zw4>K(;UyV%WTlYcWM%Si<@1+)A~@TwYf%H)5(g=HInE=j9FeHF>2#t9qT_f7h(4sc z*y2N{U*Cd#6WNIDh$CZYGxT#ejk2=OeU08$i0k>I*_TG|ZR@AIWUO4L{HSUoh>0v7 zZ~0UG?Fz}C02{15b2o8Lgi z%qA}(S;{rCR)VIc2EPu`+}9dIJ5^PH!HZSIOUsRC`hsFws3uUZE{q?j>X&2NIO2$)d#Yxq2LYC+4lhm$?sQprIedcT=o6OA5k znJ)r>g`KT!mddgNPD{VXeUA1La8=EVjODz z#(b@M!huNKl`Qw!0sdm!QQ4@L6d^c11P=AoAH}3Te^=ok4_iX};OoC|Ganf2Xcb-f zqw;^CWiU7yD-0~r%JZ4g*S}*@EXiu6BhVdek z`Aqabs5Sy{Wn{2XnZ#;~0|~ENZf@7>yv*m`(bg6*?ceJ^Gm=`=^R7j;i@$RJj{*u3 z_{K12q{@o!=7r1aqhK;O*ClVvtsi3QE2B$EVu>R&cQ{`wz7a}Eb&fGLy(@!P8nL@&jRko+nCDFi=0ox+04e!Gr|7&>|XrLc|t{O|dufd%EQ>tc(mv z9IvAwsPOg<(4;iU!+Ce_N6>DC@>P==fdU8K3@1=SBk}h34@M`VU_wj@%1;&qkjui9 z9_qK zY0`hP<@jTR?O(7R=0C(xEa`hb|6R4@gj88sDQVv=fcUnMjc)A( z`}56Jn%?tO>EU8fHJhH!!QTcPg~8;UM^OVioAiD6z*R~wO}5c{^Ck$7PnT;eULVXk zN-t=n5%mSwEt`E`ov%H44?7q5%EXtHD4Ezxrcw~5FBpF7rwXm2(j^ZNuM!n3L-X{k z*NpTJD3rV7ryE}&RZ-)1yJb zcUu#O`(a{LWY9yu&M>=53wQU>p5l+D755*MfB{0}gs@e^#0#2-nJ2R(_@~bdE#B6> zy-S97t-6XqYzQXs@T~nj0mV@2+lh9w^pnRuA`Jv`#6Z2TW}BkZ(LpYW7zW&0@~b}I zrnR+4u4knZdm(~~s%fDfWMy^>gof`pGsTr{Ox(UWm~|RT8eNtmAtB|_xaJ{wA250^ zJycIi8~AKvtN$$S0cL0&F)LYKW{`FIcY2^7alZI%b7^WCqulwf3LKMs|9!aViAPL8 z3&Uxp@uZZl`z4Z#B(mq|a-<#NXN~HG1p_^dLyQBbh!_IbsP`!-9M73qSd*uF0b8Lj z9jpr1OpSCiqsV>AA)@=Zlq_bxdkV?5&IxUIBL$dV@kkC#i}|Ltwwp$6xkeF4$YqP8AgeCEbEZ zCDAaE095bjWFaH6+!8qq{Zy6M^c;P@-y^iW*ByXdyu$NjecKcbLyXY|!eg!5SMz^7 z!4Bkn`|9p|!7OOi|NIII{#X>{a17ANwOIyDJ?GspW5Tx?OS;orrsjG&&PZ-^Ou4nV z+lRuVQEcV&UsyIDfKOPU=SmPCv%hu+9dtqhy9n9@;CA23$$XZciNALPeABZ z#=FbRZr*j}*`X$*s#Mv6stgn7Almr{E$I$&~sJC3@A-1qRAb zs2FJJeHL`qr`5k|1N)OBfGDg%Y1_aE=86p&40*Lo_HLTxt>vJ0lsB9*SgM14UvA~4 z>~x5E&k?ddt{W>!_$W#SoTv_uO0u|fcFHr3WyT#4Da(I|A$Yy@hDP*aQu$;r@I1}= z1@4MDgX2JVxV5UCodBu;U)1eh!zix4*m72F$j3?5bbCVUbYNOqQbn69d;B30K0Ue|kafBl|!J}v^ z9lK`Z$GJx9zTfU&J~zpoPpK*Xxj(8sGCNUr)vdg}4;9xqDgM!@=<}iveXdGxa;3AL zX#2+dnB82vGam#g%~AHSO#3{dgV5~{hp4liqE?D{QNN`VTfY&HXqt~ZkYLalIRJM z0$Aqp-&#P}PoBwz`H0wH=-hUt{d@k~3 zTKL855-~V_t0OtZ(>Tz7^+?mykXGiakq+>SodW-t8si$(qN6uErS&OvW+Pi_oNQ)O zQWAK92iB_|GW?~mMLw4bB{2~}Hnbl*^raN^%+k90pv@Zd&iwl|k(55a!y2_oW`ZX*EEonQ;a+3pzn0Dbgb*C@rWfU7{oArAmyxT2m^g9hR$9$->^) z;M$+55Szr{K3Evk#8Ps74?oldhlU9HB1wHMje0-=zxwF|j`TN!poG0;qYZrI&>=Ji zx}$1>o!#s!A6BxWWk1D}4j2-C2{U(%ZN2@Y^6K2PCi3~$uEYG+9RC{e_vk)lPs!svQ}cSInJ$_ z$^c7Z!0%PjFo(_|At5W-7{=7}RZ}2H->$dcD~_XF!tZ%mtm96Sx#`KGnm@Qj7Zbas zW^}vBhIg~*HxoxHlN}y!YQO? zhyA!UOMT@(U+a}+aC1yTVyCh|k18c>!)bCE-*&b^h1tL4aU*0g*74d}JLFfR+VS17 zI>XP%PVtA^ap2-T;d$O*5R*h<=Hulv*GMC!pCKL%0F>N!KHS-krOBLu$(Av+1gT66 zX!U|D5f>C4~1T{#S_t{nx*_<9j9@lp}6YG_w!seHn0|iSn=l8+OVq55q0dM8w z3z*Leuw_i#tPiIR@Pw#M14J2|PIPRukT7%<0nDBh`- z7zdDTnEB=*g`~3KBm^Rq)Yn5KQ(oFGnMAT&G2~WMJFfE-=q-l0eV@N602f=*`shaewOor*e!4P*YvxiyD7T$#t6anX8rc!zYmdcw-E2=dQA6ImNAC|sq6{`z<;E|_M%D7^Kgnfy`&ofA{pB6i= zgSX7nl>tAWTD>9nzyi_~F%FWIP;8WnH`h+9tnOSdAu5?XgE$ZQ9AqT~SFX@<0^$ZfMpK|A!@OB~=k-F`-Jn*O?tPPu7=x-`ohJQuz)W&7OX|FYPJ* z2+3Q0l*;0_uYBKgeY`;1%%q0GLxu7Zbv6Z3s#*!yihgNbWro{M1Iini7q~PfiOvov zeE&XDWZp%eaOLCihS{A^VNmf4m?7v&y1>+&g?qSI`H6MF~L7YWFF16y^fh= zfn?Nqj6NNQpXTJ~EcTb#B?Mty`j^N<1layAJQE_gF6HVe)9Ry=XBs(Isxwh_ll0~D z<9FS&C70IuhYz-2e2@1n#&N>-OKVK$mzbnHZr)pXPe~XedpA9vZx_HV19j* zd^p#%0KK*ZVDNAC&2!@<&OV=u;k^GsGoFY;K5=&XpBRjS43Y1zVY#Ox4Z8N&P3cRI}`| z3xwUl7~xqja`M@a2jiV-M=rN~5UCEA!!XP1Z9oTC3P>z>L$JGt=i)P@s8A&>HS}Sd z%C9!(3F(;oYr3-ZZO&|Y>iEhF z_Me8C%s+NdJhq6fy${~Qd_G*pTv1T#*-M(cPugT)RKFv573QuTs8x&EcT%nQ_l}qM@S$VZSQWTJ$0h zLm+&lPB{Fc|BRJ8oVC~v_C}5G=5Mg$VuE{#pw${%vemC}iz!*PBcE>hwt8M3zNhXI zWzPr9tjOZMrov?E)T~=k=e4yf)FKB(rhv@CNY>z?A*;#cnaQ%%^5iI5AGl}ir2=jQ z<|9e6=6gW&c$pTxqbfo-*6EK#jO$Ce?6Ggb!iwx&P!Fk4qK5|0AHc_-S$LluJ-z%a z;0Ab-y(w)5a~%Q5$T@Dh?54ITqf|a&7J0J;n-n(Z#_FY3q>XD?t5+!$yZ6Z9!pC4e z6?5_DUuf0#kdM&pPjMJZvI-(&JaDDudK+sNb8IV!a`oiaYP(YE1aJ!>lLC|_!Y|(s zFyJ3T%zfoVhxfV5u?EPo@kN;|uqM5SHm-xVbP((ZrJipI2y%kDXq>+p3>Ju^3@Td!qG})q$>6qoRRQ zX;gPfm$Jf9%F(n3j8K=@UPAidO#|n=IiPxee^AxCrNcFQ3lKTo4RF%89|w|X$$f4!){7;yJ-crM*R<&lr7MT>QPsF74H z;6ciOVc8Ys{Yg=@W^NWoIEg{jX7txFcJx79(SfVzV=L&nodOnVO11kttVttXjueaH zK~Zh=l4y0}y=q=_{*xI#70#%~9WJJ+Hj$~Wy6BOaK?h`mHUk^7n=xExe9ZOV< zq8g&uHgk^~d^8x#;qBd}qExH&#u_x>X!jFOXd52i&~WBkXTdDSI7-Zy(4x7;AmTCm zZv$3&5cp$+#V9n~3QTQ{CWZtHQIwAhOUhf^SgZ3bUd<{wJXE^LKKe>4eDN-(ggnzi zC)TAr?Fzw}4@`_o1i1u$S8+-D!cvUg7cVh6PHE^|tqZq;Z5QL6ByG*{qZ%mw18R9k zNHowRm*C0s3t`+;SGA@5y zLGwjx&Rj8d4m;udo~(;?3wf~oS!UobATLHb@jbJ3Z%g;JvK$3%&6afk(P_uB2CI>? zn;%*4|yqh5-8tS_chnE-2L?tM{fjX3Jhg&Ax$CiQTw{PXeXK}KXDt90rBp* zaO5Ywz9r+RGm)Eq13RHxW5-#0UAaUzSb~xNhldCwMdBnNLS|7Nc`TnAkt^5MtujRK zHfFVrJu=;y8!(C#;gzz7tCU<2h_HCeE1g_sBR5JIK7s)1UGtEoy-eM~m=H+}o6^E2 zo2p=_S>bWDaTKAgZS+4VQ_quMQBjU=@ji*p62-y~2_?QL6rkJPLRFYivv%gBmSLD_ zXcm|0`H%eCnRT5zni)kIL92Y+kjF4 zUNHxW4JGU1d~CZxICCHD^$|u=%i^Qz3j(BLBm=n~G0HVPAKF(NR9#u^;fP|D%!8%M z=}$vW$cUWD$*|2k#h7B!)zuhq*0jP%o++nS*I7>d$B(>)Oycg2duc?+#1NNCuUwM$ zd8ZP3MEFV2-kFten_;bnXQ&+d>X|t|k_uWqShh><(`4+JUB9_?`Y`|an3ES`nxZcC z^poZR!r7{Q-0|Rq8o6U*)+Fm#fskKQ)p4)xdoe2X1u5_0TUj9HZYJkPQiD3&X(ZU& z-SWX(`OWs7JO|jBf$(Yz2hm5~OxDAjK}pS0_bmnx<9pa&LP+{o{&q_}i7sv~j=&Cc ztJaR;Rcr7m^rUwG;J~+P9G#gYO>ev^Ntlw+^wE?lYjDc^L-?!*ox*of;cgvX*C!}+ z-4^)x|ET)Qu(-CSYZ!%w;7)K4?iSpg#x1zJdvGUMaCb?t5Zv7Y!JXiQ#wEDZSbv@U z+&QF^PT7zje3!*)qbrbZ8y%CEi zTjhM%+iwdHHwRoNCh7tMqS>Dt(4;SHMwEtme^Mto(X*gPaM|4jU_oRWf~W5g<_yAE z{m+o{j*o6^_=9ecEEBjs6JC2c3o#>#P39VWu>~&(&QmZgeT#QpeP_vedh#`?-sUaA zTYj6hlk>dxc`h#dXUlBgQ9QM(SLe6tuWXTAU3b^26IUM_x$jbH)U_x7)dJ{iA2R;h zJk44&UK`qa@^sFg#vXKu(SAK86u%S8oZUEk-f^rFGoa{8?z>yAhJio+48Hy{o8GFc zw!cE25YjTxBB#fSAce~WaEgJM_=%^RJH1>_VEqVB>lxlzLNVsr#zoVQQ2|xn5?c6l)Kg6&u?yk|j0)zC?G81Gyq-2Zf!^COg+eh5UnP zcU}$stQI|{&0I=s_}zEjRmk3A%213QX|NaB-ye5$MC!u^U?qIhF3&*`*(fxz+(aEFt742JBu{s?_GjcgVhw+sv(LENjxKZ*kQR zme|{GeMWHyV^&^dbd0F{Th2o@w>s-MECrB2Z_j)LuqRL7w6C}$9n2&aa7|7;#>#)c z%^5^FaV`4Tmp$s>Dz5?Gbv3NqiS;N?h5Q%>n==KXY3L%~_ejZTE zGYoJsZbZwJ0QJR*zMU9ADBIIR`m!u(W2kr%AE~O!BqIf1Yx3=3&Mw(T^r9Q!`K<)- zv-|$Z+htfo&5L?GbqOukb$(oOx#BY)RiSpwk5jtf=bb{!Kff5=2hjPlqK5J-3efwb zT=yrk-FM>oM%aVFiWZBlmaC*@ZKUEd#W-b5f=E~125%N4FrshdY-6gNXfyXKk(&4b z@6XR3LFLxUK_{%4iiw9C`%NuP@j|pt^C|%c9?_gg+|A3t;y7HB{j>0W>7IGvCxdtV zNF}5h{Z%_eln3=(Fl(FNFO&*UK3Y5ZY)`afL@{rO6W1VnXp_e9?D1ov+*+FzIs4A(7wcRp4 z{L{Q$!VL7Niti}-6JhoC!joP$lSZVR+kQ?UEEZz8Oq8Qol-2wFN*peaF~$8Atlcjq zN?$oL(bvwJcz~F7C&F4V0Aw$Nz{HKZIx@3yE$>?jWJs6w2NyNH5_9rJt%a1(Pi4h+ zUxCuFDJOO+7_C`_gEqyh4lt`*&GUXl4nL-x*i7_lZR3z8q_2w(2cBcA2dZyW@LCkg zlL`aQc?cQ!!Qd*Xox8y)$4R>Kzwzu+OwqMh}#ix_Pdin5uiE> zGWL-u@IMm(Dl0qT&K>9zjs}e{R+km9CNIc%zEmK2>cbyq7iQ+LA!bx@%NdP11)Y>^eE8@y!^FmgA_wBB4>?Gn11s zme*dh)&1~$Z#yn8bwj*}#T*LM;e9AEkOdGpg*pq-;NE=;+Eu_dhZrsB;`u8MLa@^u9huHoumQ0MT!MdxO%MYuY6J zc1FenpYF0j_Z$ATdO}-LQtb~|b6>^Z3>tfMs~p3_)pD~@5F77EO`kvSjE5hp4LsW_ zn^Lc}DJSdfK=-qeVQZ#cd@hw^$OYuhhtQH^$zRF>uI6n@5soL&)TMCtwudPO7rR}7 zLBT|6;mv9o`m%R-t`wq)&pW<128%U)F?NmOF!LPW#LS|@8iyWWfn2T+uE>4dV{oRV zgYi_v)91lk7emD+z)zTBuW>6bs@z<_j}LEl;;l@N5YJ7$xrpp5gwB(RF1C%I??_yu$2oT*(<3S-1-YWLYA3oO!@Vit9B}7DYsb7{ zyGQL)fv2NuEhAmbH4(UGe0sM>f)gQ$dDv13s&bg8naa#Meau0T>uww68lF!tpnl^=i~Uc+Wnvh^yr|V4;ytPKo8J@gV&l6pBQNyPjz7X$8GpTjG{@ce{ zA1jwy(ZEg3`ejG6orJJ-UGpN3ciy<`FZmQCh}{;9h!dd-hR$1d5zJynpIEc~ z)kJq07l{`O;!{bSG5*r`b}Rwi!8K?=*n;!RS4~vKa1-(Cw4d{w-jH$Bt-=b~Gj7MD z4_=lv{nx3DFGKl5fm6S-vdZ2XcTeW?qy`*O^clgtKfWJKPIPl~+wW(SLfq>KK7*XU zi4W=zQ=I62hxWEor#79{fL9S-a9}~lgm#u67!w39tm6mj>E`l2GX%=ZC&|R(1NHTD z(8z_C9|+ZmihyxoHp4a~A*{!ofTOgLe(CUcu0d2VR)md{KNFuN*!;Ks=Ys#}w!gY0 zjz9Lk>iC&YzZ{a_%j-J2RGqq{HwXJx2gNz{w|$bxIi6XcGf21%wwA{9%PBMQd9)!s zIp;=f9=uiMxbTQ2?H(`zzu;c|HW8s-;Mbds#C2WvzX|QE_kQNQHqU+qhQ|?Q)xu2o zzdkRS_-ONPma>y(`Zx<0`tICBaDG=0G4#CJ@#6=%OqZgZ5iZvUq066YX_R2^HO??sE)5@b;yB((aJV4Ze5nF$HggmddK0oDiHLdXZq z_%cWGF<-@lu8R@lSALUG<8*>VublY58PTG;+D>vX$ivm+CAx&~E>C7o&ICXf#a^u! z!KjnSQ5^mj?5`9-cOu*tg@KG4TI%7#%??y=3$yt*H&U?cxmiS_ddo1fWJ^|nikl(x|5>%KF5_F zl!`r%>5{Vb>=PrN*$#bnCqE|HEpXX{z=+Id{i#S&FM#7Wb9=d>dEcAHxE!=|dfS$VUS3i%q8fA+z3#Q<?onSR}x z853}Du{T`+`Rt$zxbH?Q6^a@~qLVSbZ0{(OlkhpeWy5v7>vdY=9Ub;v_Sm1`q=USx z0@qcM3At*STbCL7pEh6Jt-0)&F{f76bj_jBL6s!9;U>};H7D*QdK}IBtujYD&%2ld zi<0Lcn{W*9aH$9=nH;Ng-bEEYwp|HPA!0}tYX#x%5Gk=CMO~F@_7*LTJFs+Kc#pS> zyJDm5izf%F936TR5#~CT8?LOp6|jPHEj7)%r&*VG;7elEO&fJ`8Ojc(`>x)uuwoB! zqirq$Jt}D$n2QJ}kTC81InR+>XurNO;0vmpd`@FNoXDte5o&A7B`vF7R)u?+s$>h| z8HdY|g)7~k`778_x9Gi+}2H^!8VNiEHf5NuD3_d*p16?i}&E9>TLw@bsm;U}O09rTe;z z6lUqI2&ebAY;JK0v+<*RhPPSweB3GUB5xkQbK5=5gWKp9SL(xEZ&dL;%o<4rkBRri zgAn^3n(7{b>{XN3KSjXHC?i%03Z$iq!cG*XjIvuHlJdR}KXO@4=td@|-VaU&$(P0l zhR{Q1JqmI8hR^5e?I)rT;jsN%aL4NcQ4PBep-OiP zZrM0e0B)di?g3j%Z3}syrRYGwszhPevA`<r*jY*QPto&@c4YJWMauXpN$1?nE zYwy|=A!SXjVkx!LbxIa=B^<=$@L`BE@fAsH@SAHJW}Hd9!NQP9`0!N$IYf)X)X#jX z*l`|%BF#b$BY&}=>p%^QaNccMHNCBteP<4F5S!Gh&RE;)OYOEDY{KU{G!xI;_@VE0 zzXlcu=2P;1VnS&!%f%Ng3U-nj0GZRnn$EKn0+Rsj#*IrZr&X;)nyEIL1^NOXjkC^k zQEsWSdw&l}f#?<#y5`V}!(GtsXzFhF{YK_{{S0d+KV7@v+rWktce zxM{i0)(^?a*zN#!tGdvfN_31UDSPk3cMpkWB6s}uKk}OEd%WU7WwWX>n_V(6JG{fB z1x@$oxxF7S;SeY05%*h$hJ}vavvs#$k{rye>X34u!B?3jk_sV9yNf7+mq9%)nQn^? zp9MhG(uAGD>XmB`%3aGE&pEj@0++eCtuYIZr06I_CgXYho-``bUHV_$u1rK(q;Cv1 z-6aho7VlB`QGxA|!Q-raijIyf!ut>!sP@8pg%F1f$E1@JN2$9!GLSc@dnoUWczVr+ z35{N2k-h6ehx^)E^yce<840$JKmN3Nnbhwjs&Ll}Rn@GVLq{El2?#Vh2CbDJk{YNL zR>p1GOCC8wt^0V4Z+n!!?+9CSkoq9>c2nE*MGIc=E#Or3hV1*J4Heq3EYy%Di*HnBK~H4M77)~|W4Zn>1D7bWDJL7<{(G}>!PZJA@PEjvVof|I z7i10hXVdz8E_G`Uh@T1z{V~=|J972zX){?kraa%*6ZDPW9+d|>SU;}6%9R(!vUMO~ zc%xeV{ERpBtm@*AFm%9DaK%BsKi6%tf54f@JZ$uj1U&v;iyIvBv-1 zKvb3pN&Wowhj7yHd0Zoj@}+Qh0&d=2OU8SdjcXnZ%mn?YCDiL*vNiYgE+F{c(5A)> zcRxB|5y8{Rwq02P--egGySz8FZgUO=!w0Y11ic(`#~45T_>{@=Zs*e=zt2S-6MhH2 zf_ujbUvKC=9;QAn895-eR+LP00o!Qo{Ej_h3qJ@xgy;eeqz3u(F0YBtFI%(gcZ4!= zTMYWU#Sv;fV zX!c~(_*7FhaGeKnD#b1%_@WbhLa3yRJfa~3CK3B4@}IUtw+3Cwsb^@d9}e7}>=ypxS<+E$ zMqI1~Qv{{bEXw~wT=<6vIpg@3zu9;_2mk*Zlfb@^e-P|FFJdh-|4*)vRlE;625VhR zU*6!@t^vho^6qWUYe zZNI2DlP?Ue_XK+QV>jGLok-%0SJOU=6UY|f(91uzUR~w}AP&wJGa3={ROdj?ZgT=0 z;6NGs3og35B4#>GjK{tA-|om=g$0CEx?mWy_X4) zPaN9Op^qG*ak6tj&EEBpLvt=-YdpMO!Cd+S%02B9kSjVxrua~js^)$b)DAHa6Fk7v zhJ714^$7-q&zN`R@DI6llLgzB;0GU<`OMYNoRYsI`ajOAZ7{qrc~`iv?~`IN?SCuj zvv@8gbTSv>mg%A$%Kn|VJ^^tk=e7{eu1Ws?xdAS8dV6wi*5KVHwf?;;KO&S2`|Lp( zkN+PvmjU1u2Gys`?fVWT7ys{$WT;RgcftZWQ~bXRYliMGi}rTT1$U4~^WTSTl2StF zHT`T_!ol{hrsqFOq~au0|4g=%uMo7q#s;4R zcMXo4pIaC{Tx-GSM1~A6?K~a|wdF<@4QW6(z%W9@^B*1`N3%xfOAX||(VQ={yzy;k zb6)`3Cc3u6xVfuXj)aN`4N5$-Hd{;je)D3$=n3kCM*8jNS59OC12YpG`ffwWhWU-{G1wVEYl*(Ct zO2a2GBNGvaX0mA)D36Hwmp)=1GhWwrNU(LE;WGe+OS`(15HQFX#eyEEuTr0S8MeB= z(m+$LL>xXy@||i-uBM)Y6mFdRX3`k4-#{(34;Vt9KqE($^Guun3w3Nm^XU5_X8SH-oz-x(}bb8&--lh{WCAJ-LW z=#M`=A4DANf=wVwP>zbcu5?=*shA8PJS^U~x2yxd@w{*hS~@*vmcrK(Xae5Sap84S z=?j3Y$@Q?h^gik!hh>X#%*bW12bmZ@{${7;ejyA%^&6qS4^%gPd|@y7R)?h^-aL^P z+%7G9n=U$Auc4bn?~oD1^7{Jz32Nh3F<{xbpIexh*CxGvu1^Md51RV9BD}1oxsRD><80xajx?M(w2wV!d z2`fXHK#q%-mGz>kg(0R{8}VSW&u@Y?1(1-?q|-@{FM=ptAVrkLlyxjjfUZ__l;M*% zI%B)p9)r^$Ri-g!Q`V6*DE&ea$&0`i+CCiEX=wXc7Nj;jGb$=7>|Ar|XYJkMgF7Zn zv7wmNi?U1|>rvxC9S zvMbAV<}vVxC$CDA754_Puu$xxP$B$&B z57zRvqMRx2SYz;AV7;zE8E5zT zkD23|i4dqp$+J1~PpL7)vabyaJy62T-htR_i=ppPfUsG|$5Nshdo3tDktjmgk*Cbt zhl+smepC^17uODctQWvKSoA;3JQsd7AQJQ=4SFK+5Vg5d#P4xZx`nds{jZ*ayOC?b z24-{YNzYx5+`n9p$;d8G3cAFzPNiHYG2xB$)I)h7Z;xX3_QPW}pOAPOtC`mU8S->mw$-z;CpQm$s+4c8OTQ~5%bB?#gVYmt5}bJ{i%rFew{TT&{(D*?L+wKTPmcddj?6$gRI%I zSVk)MK1!}eOtmpgQtiVBr!LnDBh746Qoj@Tb0?lm1lRZ+KgzqI$8A9=ytk;0c_D3i zWbmee@%pXb2@uF%RIv|dkS?-WnHe(@L4IT#_gfs}6dZI9e8vIVTyvVG^um8Czkhdq zG*0*=Cp(I4I3CfsF{X+SGz%ifm;!Ran|wMqSY*xA@Yy?GZ(8)ZKgbg{8^TL?g9`F_ zv!g`Wa!!_p(R83!-&T1JjAST$Hwxda-G$wI{ zmQHyPyFZckdby5(%K!$;G+1~yoUC>ee3od7BjeUEf4oZu*#5y@mYpw_`&Lp?GI}|I z%jLeThMb6$D#^dkV$`GW?(QCs16BXyS$2EPHSc-1(rv?yYbs)X`VSRw#4>t6rL zu9LlmrNHuw2)Fx+0 z=#*@!dR&jAYC;iyoEV?ut0}A}rf~tid-8JTvC2lf5tAv3$G(t!a58MjJW0jD-7Ryd zP5ZiyljKi3@#J+%{-n=Y37Kv&5RwiMK<1Tg<@Jf*=-A*c_8(+7|C;FV<A#QpV}%sFRPb`;&A)G1Fp{D^5JjnKLd0jC zM&&>CfQv9_5$seaEz4g87>J|;x|B?gPCDowyVOR`y8oSsW2!#f7s57X;Aqcrvx{7< zdKArbyg|$OHG4OmfUmD^Z*5Wp#20xmHn2&&(ZAcb7bn4Y^a;cC=Y)dS|uc;?f zR^_uhqI?)WfKYj`b;<-m4l?7yL0Er1Zp1tl=JLIqhTbe+UMQb-^{@ z(mR%v#5|cHjL5TOKFtInoT!2dx;&MeHPrT*5tsySBBuDU3{-W`gW2GZ&&~rcvY+={ zASvSf{h#0k@}6zWY%tan2C(9)d00Z1z;bWGI$_R&AJkWZ8QwVDZ*iF)QX-=sQ@M0q zf9bk?jl$6?%p4W}|LeObhl9}aOeuhytzBKkoWx+DX>Bja0pf{OFP66x;8vs(FTg*n zFyv7tFj_uuj5qN_V^97Wca~e}>(_U*Jgi1zb^B8~#yS2p9=b^Oe=s-A4E0eQ{m9ct z33Td|n271ng>Ay?4@7y(YDtAXe`zIxp>J;o4j^D1cPp<0nyAJgPU)hqXTlVeHmhxy@`>=;^tR>=b%%++RG7M#AiclpptF z7j5B~^610Xc z6nfe@@xr`56oT}D!^O!qdexma#1Hfj>>X#T7o1vZ>JHDZ(V$rNx~jjq@YST6`#!bWhjV3mKAr{nlcSHw^euS^Cd^eMgy}8C_ z@%3rsyf8qirsKrigQc|^)1>z0sxX+^r0r@hlFx3{|FHroPZh~CE8KgEex|twYMExn zkoxwb`o{7yOdzK_t4-LP)tVuz#(!A$`3Kcs)y zFHy}xnz&rk!e^>vnqGyTMN6*2=b@xjTf8mQ> zN3I2@1${QUzmWf_OaA;b57M9v{8y?6H@9|cWdzvp5Q)t_Bw^X#XsdsKX(o$VZmG?j zOgb#=cw-qUg9$Os#0Qa+Ye_0=QX1{;3h4@arKYc4IB_{-h#2I_`JHzPw@i4n6|u38 zh`W=i2Cuq}##g&jj54SMZ~f!Ew|>xqJ%*jz-FC_<+#p<=llA=JA<#$fS8||oA`!xG zFaV;3$6$nBACkkHQ1;2#UF;&pa`YN4(?;j56|aZTcL%&!)w7PvO5BwhS&0|Y$zKo@ zuglO&G_p0!ZETo(dR}oh&)JQHf+>wRM7IgB7*01F-s!0on(i)B^tp3%jAluJ4)1Ld zAk#>MVUWTv-1NloXaG0t&;zN@dO}hpziviT=pw4cnjIsgdt67^8|=nSzJF!cDJy9T z?9DzA^HkD)tv6`fyC6%)|%G`9(B5uHfeL*l1qfSX`XOW~_xlF2qJfMaAW` zZ`d9T>3Kkem!Gu9D|HLk*Z6M9d3kx6EVk&Z0fLz_%aNe0d50CB*#qpt)#-iT$o6-d z_26-qX|UMYDUc)PjTV^jXx{Sr2(DeKH|o=#l47}KzDUz4y#mQ@1)J=s zz*qZoOR444=_-Zy$(Wr1jGDnY%pL@+sYru&Pm=Mf=F=L#*?@UeD=w*HAN5DNBF0TY zs_)METo@uy_NCs6mnh^E z(ipUL>Ea>UudBDMmI{R`tF6v8Xnmk3OojS)Z>(l?L@bB$WE#9du=39Z8l~<60=;8W zT>U+uE;q9Fzysds4%BELT)pRUh2}9iVL!AF-SNvbP00vIPdiy#m;<{%O2@Z-j}vw$ zl0V=`rCxLiY~f|PbP&Ot!-v8@g!X8qv71zu`uKw)ATjFdKS%h?=s1xEVF2rAdRwOZ zXF{UrlakcHUh9_Govtw@h;SDJZDtxZTFAbirkyUp z&!xSiZOCE@*>gYl)?6Md;vt+TuQpRgY4@waOt`~n$&wfA?0OXikyo=pkhQOco8K~l z$j=rWaP!uanU|^MV!#uNvuw_%$=ns=$UdI?aCXv!luaF&{(CNX$eX?iSmbJau&XDG-yoJCUIb_!8Oc2}b#Kh5e&WILJA68B8+;9OSV=Y4<-bfbXMIE${r4 zv~h%(L>1_F+3&V`I$-+FbiEtB!(Dm(t+^%Z&r>Mkn zof=uuDrrbIoR3s>KkK(Em(lzKF|KoTfE#`&L;fAhWr>^DZ)>xEj1Z+hd;$I4tQRi@ zG5iKB{MY0Qi&2aSBLzg#vZ|86cr-yN%Wny>^OJtw{W;3qP}_k9fLz9o2LPOpi_#3}b_X#x7=nKS=_Q4Pgxn-^RKBCWyufYm67Q^HPFD^8 z(+@&;b7*6bgOn<7jLgq%x{L-)HdXzPNCW2r)NuotpEq2FS zkQRQy(u8Qku#Djztv(Wg^q*WwgH#grNdx!u?m>E$Ya7_NMMQ8xd2=O#{kmRem zEfHlwNrY2U>Z-yCCaEAB{ZyNxtj~mYg{}hEm!aEem!7m~|3J8RPr7rZ%B%?^pMJJ2 zyK41bgb{iIi3R? zMhhmasc*8xAYTur^bS0;qr%|h;{yU}O|18>qyQLGxm+?_KDVn(a2kN@c2ctZLAK;@ z%c&HgNcd56F7ZldUUEtJrvepGQNnWxHLS%vj>r(6`TtE)h(uqQ_(eRa7yGE)Q^+N!5_WXBu_xu*s~1SbXID01R0pWwLmlIL>4A~(PRFZU zRE*M|ydf3Bx@sP`8NF~58*4|+w8DB!V^o(GqxRr**l-8Gu@eptmm2&oGJ`ppD)HBp zar|{<900elO-+%roOf@?M{nGNO}ddke0%;NSSA>F^T{r!=So724fgXiw8? z8d-UrStDOz9-&tMy?&^l&8S{s$0Aj{VG0QlfKkfaXgNj+?x(qMdsqgJCb3oH9KUvp zrs6R#eb<_1(X)j2Iu&6c;x0-5@#bpmps*?PobIKBmZzOV?!yAAy~)~Fq^MYX?5^{T zRPHkZ8k2&&?;D_w9Ty;yCd`LtU-yHMjP1OKchcV+kIcF1RCU@94%jP`yw}cR+BeaI zd#!z-{^I|7#sE-{+|xT*=lq0&Bzml+`7*Wt<{dFHaZPQ7ad5QZ;zY6&=0O_{p+Rs!+~-Zz9rT;O1qlIrsLOdRXP?iKL))Z#u%mHtL}MNU zm%AwXrp_X{(tO>mQUeh&n71&{(Iw*l+?V(y1JuIl`!1 zZ`SSU_hGg5MTG>x_xJbnJ3E5$HzykZHwj&KewQRn% zsypy0p|z(B-+ZBiwZR`m_2DQ}dMm->fW1PK0@W?c`k$8oI_RNUl9a?@6q<@5yKyu^ z1+6aQg+g}pk5N6pPN?~4myNpGAQ(06&sMDnv3^+yNY zF%#p9Hl}h@ZWrD1oV!-6AgqAEsPc05D#TZ?B4amjVAT}gk;ePX@jL59o4e7lw{o_I zkT!#+WMuGf1BJJ&MR!lZQ!QOzcf~w6GaDc?>in>gwuM?8Vo>j|>~1~H#Hc}&ZqVAT zrYfXH{iZ++)W!F&JL${)+jy6B2hguy(+T4<&9}KNUoK=MkLD_>6&};>s^h&LK}(J) z;xyYt!dHkyGv%>qta@4$Cmy?pz+W}5GEPsP-z*wi8+8~#_$pqGz&x__{rRJWJ5Sz9 z-jG`E5=C^mZm)m}rJwH0`}Y*W->oB)_HSwNuHfIh7(Qj^ETPq}#fsjL6sh(_Ipy?c zU$me-dVP(9kCkHqo zRnvSli?h2#F;%}|vli3**u`f98&j(bCBza91--0KCI5P^%xm7~!GX_tG(NX=_fc6D zn8%Q8iBO!xCF6cO(~BOHL(9!*#~u0luIn+C)+5))N(46$kWu)FuRMNgjgor zp6m*{=5VmcFMs_2L z)Dv-phKq|S8?Wm{d2#Ptjb+hIeVnvUVH5S@U-B|N^lHPA{eC2o$|vp!DnX-Ke86S? zW>kN6CR$LqU-N=*2C)>T^dwX=8+*S!6{-KDPlva~u@J{KF}yh-G!y&TR_k$=unV+n zFTqe{D~>84c{p}`xQiFMkc}|`*rpxh)LV;CEwwON zwo1becl2a235`xayowLq4FT7u-VJd6BG&isXv1|oGJD8%CmAl0pfQl>q5es$j^1G9 zP0YT+54Y4qKHDbum$V6s^yBmt&FRn;_4&m@u?N%slto|6U}0U|Uye0D2gbC);=L+Eo(rR=TIaBA zS>elJ?LZevBDu^ryQFpj{tU9`aK|sRv3+hq1tYL83~faO)0~5a3_17)NYcQI>OV)z z`npO}h})mP&k$_7@Gk*vh#it}qd;Zats8|8A5 zlaiFL6UWI&Il}uZ7d^-Hs2OaT9nIer8)a{cjFcDez`LXk3JXf{4rh zYfZCJ<-SIzHQv#NO%mGSnv?<_`@5ivrO`|=b|pley-td<(IM8gB+9Djo3q#KG&XfK zr9tYIcALVRXM3bPf7U@rvr!w-2;~JkR&>hduxE2uaQ*%gXaVbSDY1fRmJ%x*N$qQy z<#dTQD)y|D6PbZK|Di^OsOXH=7r=KVR2N@^p{^I*8g&`fp?Y|^4 zZv|39^mLm}bOeI7(E?CvehiA=4ZIH*KcU2;?GL__BNvM0Zu|@05khq&`Vgd_>GGV9 zC=D8HvD-h6D^BHgQc`&Ssp0jdTVOJ2dEa0w7)Ip7a3KC|x1DzKR}l( zWb9q)-KyG@VE!V#gjB#y0|jrS9K(w=KcXQ_tvGy87(OBBW_LP;vzSJ)Kxx2h z6n+MH{y3S7g8~Y%92w}pugXnE{MpV>y7rDVt3>D{6W{L5NEkyA{C6`~DQy*#w_^&q zg3S{KZIWXjTv&|!@1Ueorfz>v^>Wdge(2aaQeI$oKK?j~-0g*l5%?RY4g%pBdhg7+(FSJI2tx~d+ zyEt!gK0D@o+(R0(1@-JiNE?y=nShcN2)0enDL>k4r~4=E0&lWys8wrDxL%@W$8>+m z9-43v#>A47YqVPP*y+}mO=S1J1-31l?4oFV_Zb^pg}a`Qofe%Xd%q#x;A3)2VJtU3SJGl!R}^f-YhE4mY(8Cxi} z3Po_vbf1oFC?ADEm|wgX-9}_MNDk0{4T*P<@t^2rMN7M`H}YB|S_M1!P>S3^ar7WB zK>hoTg}<3~(Eq*iy_Dj1&!DEmcc}$r%%R_(Y$lCL^h2fZVze59R}9l0{$7IzkJg{a z4Em%O^*6p=r0m-j7*_j}gFN;PyUreUBL|fOHIN=VI&yRV$};EQ5Ec#A;WxU}iAiLF z06xLrKZk!pCA5j4Z?p?HKxfJ*C^yq(ksKAtax|X@-#WffQ`XLu>nia1`N!mkzi0hj z8J#guSPaQ$D!6tIy5Il)&vJn#w#X#=zTJSnu&)_@m!M>$s-s9h4svt*tf#jXAc>K(6(@xn3viSO_v;rOGNZc;pJCO>zYNo_c~Z=70B*PcwexjG2`B zl{b|t)Y)XKp=zi}lS#toGJa*EGd{ahP6L1!!z_7Kp=b&FY*L~$YM#O@$NRR_wf@^n zHL8{I$w(qYf|zp3Wl3#%g^!=}+XI=~_5_>!CBGW5!+#+?f24R`(gpCnk!D%iFK0%) z0x|Ei^a^f*=-S%a3CNNEXaP@SP>cH|np>m|yFvLKE3N9($y_%Gt3tOXEdYH|e1C#S z4g9D6M9ftyC{Mryfz%u-4 zp1b|ojS;)8ELdoYS;eN~r^R2y#n~!>vx#`R7f7&2-%V?fp>$=IyFd3;T|~h3CadoC z28vo0aI^~a@5f2;n%ckQtEUQ5Be@nB3RrGU#605F$I&`HEmWqZdLC&2GmSy=SC*qR ztV%FW`n*9M5})xV=m!u3D-PB>59rOlIpqszA{;%NkqCI4;XHN%Yir29NcgZ&eWB85 z(c*Mm{IptlkDfz_aC7!mpy^x;DWj>vP~tgPVXlO2t`WpxVI4h8GGwT~F1?AZuwJH; zt{{tkTB|RKn(^T^krMJiQM_44vcIlW?u`y9~Y`P_kR;iS<1F zrK00K^b*$eP*fSP;a}$?wLOI@OUd^hX7^LI)7~HDhi4LBB(}AkF`mo8$s*vI>fSgk zb+aNAx|aCN0-(+WFR<5XNwi}AL<)R1ydp@CDM-_h8975(S93Qdf>`3)scN;en7`V~ zmWQrcru3RtKNG$`3N@lWJ4i{`HrTn_KT#C_ys!8vcG0uEyb4?x!?S!W;p94fnzaW6 zpaI6NS|IOZ3$YT9#Lxys+0ShR?pLN+4#fPY#4KEuUR#@$kzi@b7UhXz?-%6*jdgK# zqJDH9E|%NqfZd*3Q^qDgEV^v4Jzn6|rcJ=F6>y%N^po*c2esH9-Gu-Cc)+ z_5c@wvKQ*=albs9tLBegBiFj3+|w?ofLZwmP9Nll*-}qs*xVO8u=}PwqcOi2k_*-C z@-$4k(83?6!dlv@vXHP*8X0dwV`{2pADy|7=SdSOzmHaf%{Y@kCMm3rp;UvUy(%oC zc+0%a^ytW?=H7;g2?uUM$K`h|2I^k6tjI}APFgjly4J=(dwpnSOk194pvqi{3N5)# zt)j}eb0V2O-<7d+)-U>qLf@VS1_t`&rR95s1+4Bj!lNncY~h=VeYyVTzJviq2537* ztwEL0Qd48v$K7SfIKeY~wTX<~YFz;|`hls)Tfe=8m1xG6TSql464`w%6$pXqj-S}P z24cM}xkt6fOrcdoVYR@4R_ozILd5^>Z@|N^XQnY~92&Rw>vmyYwWNZCp; zGsznL&&Y}gL5W4?hWqOeneH-Gp^r();k_xC`prr8AXPlRrC(Pt2ZCn8_AqA~x5-)+ z>y&vk8ey-SuW(xrU5Rwz_;b@@Fp?jlYo;f8N{um*#p<`g!if{ISc^f^ORB@J?w%TG zJ|@M}>4k;8#>cr3jY+gx+`s!k5y$@dsWLp;wzgUCznf?x1TUbIOsGDhlQU}u<_5eU*3cs=dBk3A1+sq3<(BjMh(M>V^k@5>%@;qHrFMEGx1>0d*XCapdU$j_9B!`i zf8)wjPTaE5-Aig3p^2pTkkZhji@~Et!JzsdZ^R8#KM(IQ&y`v50SkGXC&I-v4%i=K zo+1>vte0UpsPNM2kpQV2I|nis207&uFg!4r^(|fP9>B7=qrIK!r;$CVSgsn?*YO}d zM)39ikZ=!&NlB7XuFm(P)EzQGKgqI*Eb7!@^vtFsjU8f&3e33~bzB;ePvyzsPrzE( z0jsqHqqg4C46%&Kl7_+g!(FOYY8+{62)u3OlF$qIvioSt! zGOE!qltIq@=HhkoaAK-c_YKsBH~Y`6g-7^bTKPgBJApo2yA2kbhkLKrTJGM&8n^d~ zK*xlgRt)M!u0y>)uAz)R z{9s&P$1Hb{0F2iBd+y7!Fv!Hrlo8nK0d|lw-o#~H+td>>L*If6gip)j|I)I)K-KjM z92D)xI$X4RV($f1IR4u%8`W!u&<>WHIt;u|Ob~Q9s*pigTIBoKMAb$LLA2Qa6NRX$ zriH7}Y7%dCJEXxJPJ$tIO0i5D8TpOhw1dF@<4d`ka>ACIpxnSKWwqCn11~Kq?Xy6^ zK6>BJ!BKbForQ1}z!gC(Rr*~AoTn{{ZY+l{b*EA^jwQ*0YvB4U=C$F+=Wr!w3fAg+ zMiRXF`BT_qW)7|wl98d}y6rHZ+`}bx#c3VehfheM({vu<{f0F2-WdC0(}$@&@#p?* zh|tasQ{HiJwCcV-hy&hqdX6y6?CKLX3pKmxOu5?EJtl3G{*AO)dr$Xgsfmuj?kCHC=R?~qd?S>l84LJpK-Z&=pXCe zSq@*6bhy1vf*3TfB7g~A?mTtT-j3hvfy*ly*y8RIKhi4@Q^N`Pt~iT5LY6h(T|`$a zHWmm}Xl$Zl&O~y{s$uSflj)Nf*p}+G=atPbyBvi_X%ijYjb45Jc2jB7n@s3<6!~YS zT2Bv^+fbz>1WfRX9PYkcv8Z`cx5n~NL2>hsB{9$Hzme>78B(e;q`Z9w64TEK}G+YzVB>C;ySKPpm)^oQtqJ+-FtsXWqtHa`EJ&;);qK; z_{9CR`O?u&Cl(IVX%t z#7!l=VGxUs&OhJ|x)D05%?d@^EYr&|v5Pp)iwT5Iw>QSszSZu9V5e zXbvu(MgQ9wVrtS=@QFv@@RHK@ta0*iv=m>Ql`C-jA?KoPi9&>E{Tt_RkKSWc_7zsQ zp|Vw9zL*8DQPs=?TU*&6Vctqm4C^inr7=x?o-{4Y)c zyVx(8oW^9Mj*BL=VF!nw%|R$~d&H22#7?j(Klk+a_)AwrySBC78&+631AtEsUP-T4NoB<&VA1EJ~1mR(sf%btiN}(t>?vAlIeIaKr7cb zl`YZ!QVjONEec^KF%1>@{-JtR4XGc^m%?}E%{X38d1t05%&^2iHp5W)AD8Imv&kix z-v}%<8`-pYytX2U?2aTRqT7dc_`HT#k{?tXno9f-AOt^Oxt#OV!r(%B3OHEzzuVqM zv5DRC$}hVRv|LA^d*iV5Srw(0nqI?$%A$umWos}A+I=KD=^0B%G zS?A-#YTfrfx)x*?td?(%-30dnsxAa0A0sab7Ft(_d7I}qNmvWT;?_@$i3&E$cZNOQ zUCpRC91^G7EA?lwQ!&{*dS&-9;GWAe&fH}j&iT4h8F^fLk$6v0a#3QCu9kiKz|3T~+RABdY(D)x2 zTppP1;3#**PX66_`E+(Wq1v^y$-&?y9S&86}VSw^L@t2V50PX+bHcRE}OtuD{r_)W7yr<=116r*e3r#QP3{802D=?>3q0l zL}C9lq5t+v@_=^*RTEvpJpmec$V*MSt4kv;m^9V%{ijxsSDz|$E&%Ep5WfHoA z1h^8tJx)!CBo1xsCU zk)M-+QVYcLlRFD0k`q1ZpFhu!h)2wdM7Y8hi1)%$hKo~_vyg(OSD&QrT!_SO+svLD z(7gi0sQk-ZcyRYaQ$KqqGn7T~dA{XAt)ePpzHR0Y6yk}X+@A_E0N$XegpA*0H^otY zEE>w_&6Hr&_dtx;JIHg`Exc=KZZ5K^NEukcCu~KGSYAojR91_I$fxmm#>DGZa-@sN z&3qPAsO_ctttEu=mg#jt(D+ob+HyAyT+(QkPxfAV$s=Y;DDP3$WWIl8`nBUuI=`-Z~U*(Iam3gWITCr zZy8cQOP>>EBRJW=u)EcI!NSjhk2^$06+dOge;IR1uI=gKVo_sOeV!yzP8i_wj2?If z$0uC>;hTdSZ>%Uy7=p z@K}2ovBVDHxO6ob5_0h#DNLU9-Pii#+8Af1@T$V~{O|K}0W#mlu^_%FaP)dq5&c!! zad01XJm2DB7}g0n)k@#ht-_QWPWU3`r(|S#?agAxf){>TYp?gNTb|@@Xu!r2_XTaP zc5J;l20%sDd%F8yWm)Li|1)Oa1tAh;V!F_(c475EDa1KiJ8y)W9vJt3HHZ_^f_&%m zj^0lkJQhC(wRo-z;~5F{Qy$%&v?$$w&i)_Lk@?H%K%W(h)yJrc1o6%ck=Tk`g%#G& zoM_Kj_WXX|;Fbp|3Td~BqR!aI>(G6EATA+&(L(y)V1xv5i3lDcDF4!0H1F_blK7YB z+c6lMd<c=zna{Fpj?SDnWE)pTANSXB)ln>iNQ#i-8D^N3a zUJkDbjiob!^ZpfZTcTv%$NX(2KAbx-8S-32;-ZgzI%}X%BAxek7ush7ztq0}hXwh6 zU_lQjeNZoEDF5%uNFN&RS4I6%FK{y~KqU?G8;W>i@?PH4-o}Gal6_E^V(b46oqqfO zdtVplXkp4d@AsmqpZ}9qg{;X8Y1w;USmBjasT@7S0+``7=bohfQwOtOub0w1cg7Al zwQ8s!psmpBV#nSSs8ZyoPa@FuTlw&B%%#r`rq|w_DEE?tH%#I9{?KO~UYhybOvo9n z{dY-BZuONRzb?>(Ugg)swP{{T>Y1?j7-;cv=~$kvvf|$d7rDF z-|jVKZTE+>2zMQshhJdA<$?d6WB~0ae1wCpUVSUV5$xJum;_i_Ngw`Ym;z4e$A@=-2QTT;qM92U z`wGA?RNkJheMby*9%bFDE4so`34`P}#;Z@Uv4-#>HqRRyhLyKOp9mW+_dU|hu-4a+* z$a(UD!g%StPrkm95N}Ej(y2GrXdE)*b6rUeMOh-^XGHx5b0O%tN8_094U=r`KFOM? z@@eI(cqZkzG=8b?Tf)epYWlc`y6}aB!XHK95fM}RZ>@%sZG4A@)sC~}#TdbCwuj;= zlcNH|95rXSTDw zyuY6R&F6EYDIzLTXnTqWc-->b&O6N~ug`u@goeR56XLV)>5aB2(47^)u}cdSv1ox$ zVY2AvpmWK1Qg5D8z5Fth;a}jEZ-(5iTLsojb$V=OQp~rt$$%qyKa{B_)*nIfy{Y!Eina~S!7q`AkL)l1Q!6ai*;GAMOvxo_^H;6Krp?lq@70Pd zDs%Drt1=l#P@epZO{?CxW-FADA}ZM!@DbGG@m4+4P*-J;Y zULt1f*7WHT85#Q=VqRWGgn!w^Mq{GmQCUnE&eeYR!1-)cHk$PZ3jGb&Vyz&H-A_G$ zDj~h;e6UTN-el&XwCwVSWkMKxwL9)TC`_U!CatF9Y=i{4sTVaqz7C%Qo;cfQLA&GC z_UB57xy_SvX^w{s`C^?}om>=#NDg|0Upqpl@2+O)&wTE#A==uRDs^zWVzxSqV>veOcj{GU;Dt{%5+Gn6=VtP#Vj>d#Zr;u4~%h$nG-)_6Yl1d%eB@5clU7 z2t`JiJK!V|F{s}1ja#jSS2#(*s26wK-XGk5AwSoW85IsG@s#&|7(3g!tGys&Explt zopM(;1%&=wz{FH%ehxmAO&w}^R&rFV{amf*E86a({ToVO;QmH{1v}`Hg&ldQOkLS5 zSAMLlH3YPRD4)fHjOlY#?3s3tw+7qZ)>e4mwaEgxBw~}TuPusWdGDig+3KnB6^+1< zk{$O=vA5o4xmIIo<2urUlx9;qdyV`@j5{2xSi8cK6j+r5y&Pd^bao=JogaoSd5saG z$Hl=RX(FvX2cPbJ#`}%YHW7u#A#~Cu_HPsv^h~i2U~w8!cgTfcqo3M3|9N$ByAywA zv?T{2cgJ)GZ@Y=y1)$gd{$iu41AI)zV-6cg{|17lml7#Io807M3g}!9*_~J^`78$w z`PKcbZ0E;jLazCEo>2gh>fd94Usn5eDXKYwI_-nSGv*A;=LGHTa#`F8Rk|&->nLaB zL%ArtjyH)6nosF7CCb8UcK?Lpja05BJKzbPj*xY_6BkO;n0amS{?sngC(}|ky23jI zUd%ia(Lr8m9?h0g0XAOzv%PBz3?_k%83%lBS+AyxrCQP)BfkG~r(bOPr+7ry9u}^4 z2CWX6x0MdbfQ_(esC^D=MT~3eIQn_JVaAf{sbYnQC>g#t)jYetAs{-014TyKg%7_Z z3d->pl?W6{`P;sQ=soc)*_D9ctztGl+nxx6*KC%=ivp-%MD@^ zD6HM{@8EUL%a573S23400;#SQFEv`yc2;U58j+%@ziI|wTPD#i{JhXG>cSL3$3NHjgkEw82YxLwCXDb4jK*uydIOR; zkOhtivb^y=nA#-TZJ(A?J1@ZhP;C8@M7qX!pC8-vF^4h%w|(t<1tImy@wmK|CPU zGS%m-a`k;E51P_^Zq&YEkrWVeCIR4J1pv9FBua`2a;nJxW%D@stJ3J{9?e$;7>Dot zO?9d}xwAa<8x!C1+MF^iRc@(}%Vc>eZ5X?xPh`w(6y&)`q*W993cbtP`8mU|*&_Mf z)xRaE9$;w3=vM@N z2kJxF;}eoyVzGx0MZOA)zxVy_3Tv*nZ@B3WRb;g>wBhZ(WAZdu(Yl?jU@Cmq_4NKH zjm4x#yZpX{oMGEDKv&Ji*1VgBH2`qu954HI5E3lDI^7`z4ru!(`ad zY0y+QUI_$B$P82&_*->W3$Cb=6lFUT=LnkER#k^YCdjYC4BJ%y@$flB4=!YYt5kE* zuToXwk0FME4KAB`g>F{!>3BX5oVp5}Fzv2ecS&e;uZ~-)BI@ozx&(__JKA8w(H7lJUFD4u7OvcOtG}c{PMLp4l6h zJi`72{Wi5i3jKVQ^t+wJP;dkh8$gs_Z3o$HUZjZm1j8Gx_E*-~y&Jhey|`R5M8RqL zdoI{4+zgqdVl|W~}jk2&i$Bz2JKeyh_(VMt7RO3goFDwSP%x*b<=R24`w zPO;27`7hKUZ5!wRw87@!qujQ>;HgBap4+;p*bOnI*KW@F;jYv6X1CVCSe#?VtPAE! z3p$L_aqS7;&na7?>t*P&$67kAAh@dTw2NgGlnX8GGB|Z1h}otJ{iHDa3X`GWsZj)l z`|uC(6r$~$M&)WGvMr=!&eSkPGZ(M4JjlE8tM41M^hQb5Yp?4V8i%pcVmnLhf50FO z=^w~`0pB6IGUx)CY(|3nfDVJ0SnxneT4VY3^e;8%cK&0Tq_#IHn?A+d|6t`E)sEPw zy+^MX+=e4t*dUeJ2et|0@B%vwP&Fh4JRiUwpJ9ia&x?gUxirpEZ)&Kgg)ko{uCYe= zq;}Y07nT=A*#xOt5K?@Pnnxx(b%W^-R%RG#;z)D0SGIOL1C+GLaiwWa0Wlj|V2X?82)uk-&Z@c-u3+ zc06K@2jdJJM^~k1AoU`qH@<1&1_I5fB7YoIaG??O^gNLLt_|4Q*Zy3L7OPOq1J(`Z z8`tY6V9n>ZjY>COstAt9|I5rQB0$EuG}z$3yobuM8J4W8Zuv%YI)*&@xjrR@DVXkbjVu9qyoF;?kT|I5+KO1n_Ew`NkiEynK5%gx zI4I@b)Hu}`7^A_O!gjMR_^?I;yF-5~wSU#&B#y(MKt=R)?G*533Wi72dfk_cu?q{o={>yVRzBJV7H6|#AyffMP7XnOAi)>D(NfgUUD^5Psp`WvKwL0dc zh>k?Au8lc4pvS%C@TrC|>N<;Md9L`CL;&8HwOaE~qTLAFM=Es}JhAKONG`zUOSxfp*U5vz3)hunFpJsh)yok3Xm%h+Ni8m?Jc6|riB@N8`cjov zvWlImvSb?WdhgX@`Ynx4VThj*j;DJz$1GR(Z{%0l8LAbhXYHPK#Nix7T@H4AGO}h& zODFvDXYop+9#&0JI&M9pw;^SZ4EsjE6|m$)#v}$h`$l{j^@h9kdM@-K|R!Z z48>cEPg&NOHz1FU-0=E+W>`r=D9oHnq{DSl?WXSq@gvh1V~GDN1HQmQ<*6R(z)$(& zTpMKPxzhh_T8AJt&)_vSEY&xr8>Icb*W0lJlO7kbwD90QU~4d>7}U>lmw9>3X}c2N zNB`z$ehBW+!wn=tQYvV&ge6-*-yx)w@4JbzusD1|MhW;FMT{Q~9q{~PI{_^tfIS;d zJCNj7z$HX0N9i>UuEqGa0xr^1tf;FuTfa?TU%)aSH6WjGesB*~1>&=jo;VrGQe8-7 z8pU-{ObVRWc(b0_&DL-GQy-jORM*Zw_aJ%5Bq9|xj~?9&%~c|mMBbn0+3dgB0#FG9s0l^LmZg$*@1(|PjtPXRFv}%Qo3Tpk zBOznFGX}!2Yzi>`zTsf^O6nB44sL?a%oFUZ0Nd){1n+Nk25X>c%MR5E4JVMn_U-3J zXHA0!mtQ)zouOR;339(_heb$F-zOO7ixB_-jl>rKTWhgkhr~&R^&qxeX_W0Ji7ISt ze#va>nKx_wO9TO6w}2>M7APb~cEJ`vTqq+*g?*R353yOlH1U3oqm&s|o3tQ#`-sma z-@kj4ig=|}Ev&emlCV?K(>K`>VXtc9fQoNYTZgsy19Cc@^#2gJBt#!W5$D|2!o!*c zRgOZ>#C3{R1GXVw$=iJ0ABnn{sOrC?7XR_2ThI{yKrbn2ciVm91Gf=|jf?{SM`-`h z0gpvU(rVh~zXap{-*oPWa=swZ1$tR{gBgAK-`h6~{DVOTj~)B;|6a2BcpaUxELJURT~bTdAUFmfvRMr=H=-DweAC`8aRS>*vmp z>DdeAppV5+o<#=|5N4JDK%)1p7(OP0Fs<|tS?!FhtKye@wER?=yKzC2>*Q2TIB9ON zY>GBdgS&$IO9xge9d&L;e?(9!E>l9DtJijj3k5+cqgHh5axwAH6lUi!7xNlCF*#;a zV1sXGi`nLj#odbr$-@j-8`spI=crWr7~~JzVy_q%-yk$~8gLpO6yW@_l*`O9$6WB6 zh90D+fBT5v;S@odNtJ%)#u5rZT` zV`;{cUewf0*j6feBtE8fL*`dST_cKvZ1a0Ls|AZS37Q2rLAQ_14Ud#PjK`hu^(PZ!!Ewh z^R&3$#hxtHX{UWDt#*B@$`pvzUlqi}ny1q)$f6QkWsNdNLbne8?;xQ)iXo0-HC8;m8X-JiP!iXD88 z5T?aSks?^*Z%R{sUGI&Sl@#JbhvS6H!^S0_H7XZNFzGEO6cXr?b|x}ah;nCc0bwBF z&ArpuFzyJ#<-C7 z1zzBIMYzslH#AHJxy|YWhhp$g?3LVXqFkmz32h6|r z`c6WS7UOkVY(H)7W0Q<6IZyoqcGwiez6%l~8Y-jaxaYdu^|`I4+Nu0*;U{aJV1>=Z zGFf~r1>aJYC}6HPPj)CwV4{h5Y0X!dUp~jUn}WiGKmLLt_mSOp-EyYDecFZv z^_^4DdFR)t-a|x9L&F5Hm|9H;1dUV1^L#1y#|a&p1+qM<6B82(;|4c=-km6X51F&m zf>&e*|FSv0S^@-h+Y_wf?!+4;OiVz60s1TnSFcaDfDImhr6{n_P)?#(6PIVMkve=! zMMiydJ#PX_nAGlgI3d3bocGPH4wcO62eNd3aUhhI0E14jzXS4f+{f*h2C5PNSDHUC zF_?y&ez`#5OU!liH&DQT<@!=hB!f|hO6~cN@b7GDH8o}si+f07^_rLmDh^$A?A^2L`M0e*gXw#lkViK-;BSbZ^wtSEb@SmNe!~Wr z*@54Lo`8k--H|wdyiQ>nIsANfMSF5pMXM~u_IHibMp3qCLYL$tiA4r-=0JZ6s(D`) z(W&ux#li){e>*wc&W#pi{E9c@Unw932)?91rwWyk;&V2?FVSdF%+29j>LMU|{@9@$tC-r5lnu@}G9 zG&Gt7hU>+u#F1R?{fE;TL+pPdtM>)Wz7xTF$^;#*2Cim1+_>7$8T%p}MTZm4a*{wy zbwEy?Sn>(nQGbC!?Iow6VAm9dAl@Rv0D(#dDlgw_!4guh2!%J&p1ZND*zw@s3%$A! zA=}O%Hy0wE!;H#gsed$14fPYJ?~D`LMWmgrgSItv32fXB*$U=Z2n+;$ceYWyP05Dw zgUYQC+Kn`ejky$rHoR9?I24gP5b8xN0wW>q_>ky-R@I`4{-kkVxox3FwNEe>K?PRH zb2f+->^5_;`Cs&ctVCGv-XJT!^YVPPj$W+KUkQkU4(RvTNAq5K_fOK~| z3?b1_`j@zGv1W0`#8!4us_067?2a`j-ieRxosXa}(?K^l-YXOWT zAqi1Rb$9X|wTS**XB=i62JuqoAD#^c?aR;;!5=s8l&!jOg(o+M(^8#8**$A1is4_r zppzQnROxnt+ifN|;uvuu9ZNrCd}X}~jujrvVg!imMBy{?@PW;IYBJxKQL8oG=C#%@ z$ZZ<=$ zjB?YoRkm#Y7fga!+}CN%fZj%MR8VkLiK(vkpl zu0(gtv*J1RCtgopE&{Ep6elH2X|DxUPq2)IP%5M{4r1f4bG6~5#BJ4RlHdBA+=&Ku z-2a0ZJ~njA;r}&%FolUx;?6t(!58Ik3Q12Bh`veF_IbJ9m9SeHpG~FcMPYA*2FEO< z13H4c2_AZ~`jeTy;&D41MfNu*(`w0O#G$5(&7v;Wye+j8TiOv*W&q#B2hPZ;7t=(2^5QfqMxsUFDFyblNADtMBku_TQR7Rtg?)zsleyhy?plaG=8*N7n*sR`^ zFE2Z&jZk|jq3hIcgO~WYPv%TzF_UIWRiv4NT35t%84GQR3_9I;Db@>=JLGDMWs8FN zVBxun#`J^OlBEj`suc>+r}X2)4ga`JDEK@Y2g;>!q(F3X@F1G?X-lBK$_yvcSaXN% ztC&xWggK~m*=KlooIGMbqY@@ij%Wz;C_weNs}=bQimVu}Ged~@ZtpST@}AHSMOEm| zNl1{wTSrIX&xmmWWP1k8JcoxAF?aAqE^}<}AIrX}iBHQ5;4pi|+!2iNg5xwAt-of< zw7Yoyr$Ky!#kc*jGs__%k~ zIFtn+Ojp(edGNc#m3g}#!)^)YfoO{`QA6b=9NA1mZ69SVA43I~pHN1#)AaHRwAd=8 z1$wZ_vVkWhDB1&s#%U-qQOETJX0u7E!98|+34Wl|jEc3;x%JyFPm+lA$%F{yoBKIw zo#7&#n(utM8uTlMtf$n{?=p2&Y4?+YY?J*g$apeww!b@#$!#rluwRRqtdR8f;gLnp%P$AWY?XIFwmz#&{qhqRZ^ zBL}**)KL58yR$%x`^6}2+M4eTQ`sDC=JcKXskpPQPJ))ntrP(B{qj)&Z`BwCr_P5#LReLZe zNtN=M`bM!w*ji8cZ>8nS=Y6Kq%r^wS_f*|LsLAIueh>@D>&4iW5gWwQcw?bL&c_m5 z)_nfs+0VXLX|n8jo@hcKu4Wpx3+C>sKsqz&(GWHHm_fgRgzvaqL^M`zU+zI{OnxtT z+`iGi*|?VtBx+5KJ6#wz$B5TW%MzT%0D-nQaQEvGK8r=r%lCHaYY-Sr*^nR6n{{*; zE0a1!Cwqm8C~o91=I|h6(3d(^VR&!sIg%rPj(5NnlquZ`R{PV%mivNIVfofzv66#H ziwfHzDaCqJsg23TXTp)CDda47ZSsExcn4zt61AsKmBuAXjK1I&%&QXyQ@Ewdwy?ME za}2lzKAA^X?)#{!5WAr|ji<_aee-Cjn|(wG_=4!)W`YsY&~F+uW0TAO@%o_s@l%qb z;moBfp7X0zepE(>qHQF|*y$kV73(18*SzZWkxvo`=T_U5tLnak9X3j?UepbVd?7;d zB&vHp7fo~HEd!-9q)w3!xUy2ttViMG7_VHmA{u}jr~%`6x=eDg^g7|qMv&#A@lWos(%-Y% zVGHH!V+vI6t9J;h-XUJlzwc8|8~$(LTJK{tpsx8j$ZF58>ihYWa`r^kYMSF6I_xFX zMw{l+h>WQ5P>>nRjv_5r!tZO~@r814f7aiYgh6D&Nm;_(g#nZS+PEnR`B_&?(95ch zPq5xPT6}#(hJdGV`?-aDPNIUe&+A?%h2Su7jG~g#)&58!yscD4v41$+1?r3sfsPJE ze+hrj#hR720j29cH9Y@a*PFsX7to(>aUUOhBZ+iIUYHd`v}Ek9EBFQ3I@OT=-KrO1 z@wSOl7QOH`OOICuMs+6-FOwi`6~hMX8JjjyV~=ctv-g>;Ak+iO=6X(Va5q!q7L0bY z;^C3{v=E2yjFHiw!ogJ?R$AtciAf%?2PMggA0BbtP~jn*5+S1_3S~aU;ja6ZBZ>2{ zKvck=iN4gL0DIT}M5K+=k1hgYEgJ%i6cckR>gA1$dHMtYn{!iIk#3$TYf{DVm}es@ zxP1?AyC_%%Bv`g79lM~_O3fC1OI4=xqbQGEu@^=zt+J5m0@Y4q{WfWGq2nGmqB!@x3d+PS ze^zru?36o;q95TLE>)X=e|>G)LodiD1#-5tLT+*=dv64gpH1%COpksX1Xz0kqg@4# zBZ!UL+PE{Dw7xhpWK1wd;K#{E?)DX@qj$A#JJCN>*C`ZjzqXe8Yx2ux$FEN)M1k%| zwHiYIxxqlBb_5hgmEwcAGj*T)K#-pz*3+ljxL)r*C3X3wq7Yv$b# z54vgsPi5lx8?HP@5yoB`TeqB$c@I$VLZ9FLMvyX+BT6RdBvJ)dw#_emfLpGR;jA|a zwKEte-hN+gaNocMcj2YW`AiXTv<5^xt46LHn;v>@4BI81K3(uQ_O`@2VTc~{I4m?Y zeFA^;*|uVOZMvan9pQj$Lxa*!mKBrKH9jRQ1!ee^s414FIj z-M@JhvHnj1fj?&qSh)G^FlGTDXKeV!Ll$xC0HQ~L+^Ypsot!p-(fSLWzBh~+!Wc5f zcFQ&PthE`de8z!KZhxni2#18#UccYEXZ~=t) zzuvS?Qcy@5HgSV3t)N0S;C7gchn?wjd%aMQsUi#pXeAUlLd*IyjORTv2!TD`6K{qH z@FW}0b-s)~J*hptIYB31SL}Ck!Fqcg1Rc}ht24#R6E2nY*8G0Ib`p3Nm|}>Vd)^|5 zt3GGF@(+@MR3%%zSQZ&d`pnytrX+PTbgT7__qA7Wp?^RF=ndVAh>0<1YM}}g@)F-* zMOR;x9RLc?h)FwacwI&Lf7W}{e#M)M!%gUyDNp`t(rfmqP((E{-O92c+;wsV3*0{T zHdv*~_?zU|u(YU9o>(NvqhcVn0gz2;f4|0^M2P>vSxQ3^uRItXb~uod9ng;!7mf3O_%AG2i+7nqeIy?979`@SM_kCab6i zx7%j?+@hgF)sB>=MCkgnl;56rigaUMBJ}n#>Op366%G6wufy#g*@qbV!yl|yMTfn# znzU&!8mi8ac+ z`p5Z`j@ST;3=OS_iALpe!EX*W^U<8`%JpW<-%K5GW>H6^*%E?LWOXw)HF0CUZS@zgk^e=Rpz zk~Rs=IGK$E1Zi^akcoI*?~IZj#y)HnW#Oa0kVd;nYbzdXqIBX_1TZ%wvw;ok3JGIzh>}GgMD7 z<%PwJM)|Af5OzDv;IHuM*pSv(ue(cFZMr1@(;oNK6|ODIt#cV*+UY7m_Pp0SL?P=0 zV1&|8=UIxk#D_&DzuJe20Sy3I5E(;Q%CmqhC}a9bfYWDkd6jh4mtv4#aU)PrQdO{9 zrPa#o+rgLCW;r+r&vrSWMK|VjlH^+<+NWS1i40?OY%t|J_~`28k&1j$)kkpf``=5H zPEm3p3cYFtwC@jr>S-7|IxKYCvFW{ieb8!bBfUd$M?TX44i69scIzcAM1ftrKZQiN zx@uWFgOy*XX+Qs(&o{Z;l98hATN3^9_IcQjS?iK_ScTV$_7D(UNjgK3ugt^-0WZMx zeC^evO(N+ho8?Q4BT@&L36*8e7M0qbYRfa(dT8M}I5)%)7j~JEy3p|1wE2qZbYo`O z+61@(L~@x~e!;}`V21PQKLdcfg;Vozd~CIIW9)24bmFklRru&mR55#DKs(TBmw|2x z%OJhRViIjfQg{T7hdGN?guBGC<{)HG3C`XLlXfd3rSf4gNcrJW7_pgmvuS~SBL@4z zfBZvNGTe5Lrc~pPqFTY^|Lp?6lfz%`9{e#v^qsRTuxlPnwC;QZ?a}F(iQD(GF1k2^ zL*Wtu2&tR-~7&J{I z>K4$vq8Ku0V|D2@?yTu3r!OBb{$Z4KqUC4`s+My5tN*=ecEm*QB=>@5*ezO28S3~k zZ)&E5ZoRbUo*{;IBAU2&4TpICfuC!j`O7PRTTwJ z;HVJJ&)9!^6RgfmU-grPIBXV~G-AaQmg4Tyz9^msWB&@wh|K!wlk3; z3lq9*%90;h=C3u5GCu$IVngB=3S&Skxsq1pZM36QqfZKKYAg4ctX`{u7q6Ha^0X&8 z0dnUh8|x`MG=_b1w+Sd0QwntQ7adI;GRKJILp-;7PPO(LXddT zyZLKSn~+^LgWpZbY)p?K8^93?BQVv!3Zwtnsn+Mzo=<}Ep2_p4wq!f-8~h%~tDnp^ zza8SsnC*OCycA$$a33#mYgJN8Lg4gv^{hS1p*jEkt)CbwPp(p@L;YlDDn@WJjUFhz z=7*w5?Vs0uqUh@G7FKSMG8#&wOqd(Z`T71Lj>_$_q*tZeUgY}l2~LzgSVa9$$)Ank zgcJwT?L)AWX#?6A;(%wZrP|;}Zf;ZL(X_Y7=IC#~2WDRcX58!Kr?&ZqT<60V|HxDp zwQzg5Ujf_2@Fs5g>3M$bbMASSRqsykLr~FcL@zGw?)SI=nh+PTTSYHQ$~`SGdQh||*w$2g zK+aIu`xqRR$$N{VNnn3Ad{s0FV|?G^WA9k^W$N7>8A#7kulHho?b$-Y7WBgmYRxpn z4q7zAR9Gbnl`#d69hVZ(JkGz)A(Kz#bSnF6WoBB#F7HnZQrxJ%2^;v0e7m*o?k)3u zm)yxN+=hFL;uux>YVALz4|+|BCyn~DollTcp|0aJ9%ju>LXlwAK&IYP$LpfC)B#p^ zyLz{pwBf?Tij}z?#w#T<)oNQhCLqd`QLrB|4TUD4kGljqU=B=|IN&t{wi1(Jqwku5 z-P$?&B&HUYFXjJZ>YD@OX!o{bv$1X4Zfx6X?8bJ}*lwIOww=aiqsD4%C*S0G-gD0P z=g!XT+`Dt{jq7(|iP!Xq5F9g@gm{M3XmLyTY-05VhoC=prqH3Vz4=iivlb8HtsN!~ zC#apqg&a;J%uYO{*CQpM19}vtlG5kQ#nIzXxA|9-U7?-8*_SfXqe$WA)9H{AIV;|h zsrAvqM}3qF2~d%td;+2A9^qqb(szf>$+q(?7-1;6<(1(qvRqj%sMkw`KlL!|DQq+- zT6b)V0jBjimOS`9cH zRpcdnrB6@zN{~`*x89|DM$DQUByz*R`Z~N(AjEg(Z3uCSB|>(_#;n-|ICdI8>_cBz z+Xo(fDDXToQGOKrA`6}Y{uH=#_p7cu<60D$VGh$DK?Way;vwp09ajOmcx?3Yb1~^f zDcgZOK@m>sKa7EYLC2}k0ON-#$lTW7|J1@hE^~ket7Y~*9w`A3si@ zmd``=A0hMbFPc^2zQ1d7FhE&T@aK{51sAxgBt7AXGOMh9)uC4z!ot>9B>K`m?UVnP z3DhADGthToxK2+<{1^&9E3Z+mr*O455=o9qa+n26-KRwI7jCirY>9Vf+~T%ykfTIl zGG;qs`_p}#V{jy`iRnw33T5h@l>d?YYnWUHFPasnsgB8YVa|lp|8CI=Y<3WGmuo(e z%MmyJsU`YDVmS<;QtwiBS<9vDLo629Bc?7;!p1eYN0{0MvlBxm0C>C{ZxHIg>gabuDaG6O!n? zN?rHem1sPj&Tr{$36#?Gj2(ocVAb8>b!mcKel3h@Yl&1?nB_)A3^ufih8s42zZt2- z?4FIV6Hq-HAIS=JAM1q8p=#G9w6$B=+sOw;vaCr|*$5?-d`IE*i2t9`)I;H;V@^pw z@-S}HT6QBme+|;>@X`LMPRq0-fe6O-6@1Qt5+0K&6NzBjBZ#$D2>e551E7{ujv3n>{`@rv2h*3AQ^#xcKfe z53+;`8UgUkBX#fvdB~C;gK(#^&V? zVae{QldAb~)4EE97U?B5Qt-;ra@`^yi97yV)>Wp-JU(XW#-Oo~yf2u>cgbN4a z&R}i;={LeZUbuZ@Ub>9RT*cS_GM?(n{L#~5#hCS|^^=xrS&Q$QY%!%qDAcquXLcD4oUJrNBc> zEQLhKOIfBujux9(qWp7?tWW2Ho6;iogHpi_hCBd!9;#oUv^b4oY)dcQh%H(uMB~!2x4C=How+{#MobMpH{)>S`f){j+V1NH+U5sfR zbe?mO_1TOMJ&Z;jh8)~KRj`=^*^hGd56NitT(kJ-z+LT46krZpb-#r@%M~x1I|8CKv4gI)&^3M^;qVb8_e}wx$pjleMGAjTui7&7{BfE(2j(Ff+}r{ z!~0sQJjHu!>!ZmGk{Z;13gR!F%E*BZoiy{SJ>)qkIQWLQ%fMFIEA<-`xG#;C(gm^@ zk&i)blcW_3O!_y-u?Z+BNSP{p?Ly7wNk^K=h2xZp0C+$(Om{l7E=&@vA;+mpk7yC? z-DE@!>VxIthYua>osH69TLA_a`{*G7VPf_$C3WzqFci!4DL2fYuleS+1Kw|qbJVd? z=b~RhqpmclE1>N5#RQ z~lDmK(%=sC5%7%=4t$`@8C$Tcm&o^7+?+% z{UxFbltX4bwlp%WQoj zopm$of!bv(#K!%hQG2=Anfl4`SQg^w8x;6++7B}?$qprhVMP-S~)N_xgA8R_On0(6TE}IC|XuL;H3JBGM zQMPDtM8X#%mb;k(+ROzc)d!Y`BxST_os%Cn0ln0E31Lz?*xbkGkeb~QmY~3@n~+@xhqvIt3b!z>r}2q> zgs9(BUyz6z(88;^kOsd`=G>}?1Ond;lRi=38;Mr`=`Ee-qT?P?{-)k`&r4rD4)PKv z@&~G(q*6Rr5Lm2dR$=FjSsNLp2ONjdp{X}mu;Jj8H?i1zWE_0a=X+B@`_QXh|QTT%$d``ddYw);X z3xfUNXV#lcrXsqEKvOC?^^>oseANAt(TN>et^IF+5}|a=`q_6`3?nw)l=7Q^9>}Z9 zS4p`ZY#!gItTu?W1vc&3!r44|cjFpQiC@7YgBT?r$=YS{|kTO4sb+Ve!Q$N%hd^TsrrHHz@n!uH1e zl+Obu(v#nH^=F<-Fr}}=u3^_-@VO-nfk~SqMLzI)JeS+KTnXGoiNQ{{RfVzE&Dv+= z<#O*pWVBfA5IBd+HV^5Oi-1{6%iG3yDi=K}g{(m7w~8w2AKlEZIr40l3qJy z`>~(OVIof%A*Bu0G>kN11GU#TlHHyPqRRvB4d{`9vySSR4EiJj?O#LOE&|a zp2MdS8?A-$#Z_bC=@*Z^W;UMyL23@I5%=R)9C&7G0KJt`-umHTXXH-bd$X7uiKI_6 zZF&r8EN0x>D4EF*bt(*+Djch)pnJPN zsPGwOHhD4mC&nYsyOj_l}z_D(balMWqmJd`R0uR_@_{fz1huZj0jntkPg{+@z=EwOU zB4_>oG0OlQ9%@AVmaawtUfe&})4vW;9#|;A(Qf!eHkbdN2)q*{OpmaEeLE5RugCw# zt|+047^J$N&GnnkdW& zz&VNk|2dF>TG(B%D^ZYKF2n!BqA-B_zBF4CTGagWX~bZG#7RF8VPK~7xls9XFZO!D zG5d@u`0&uTVB;j8PQ1Cy|XA4?WkU`xi0*pmt z4@HX=BFTrn(BKVDT;oOIKaR% z3Z=V`fefZ*lJtg%fPfCz!#@6yz&M)Ct2D7e)ycoJYcjMwiK%+rBE9M7P{gs9|pN^BAU#`24*LouUd^y4d>0&Nxd(kUX6G(iLU* zcRx*Jvr;6W$Rd$Knsi;zipfrgBR)kQ@AG^@tkoxa-%M^}Qk?7Vz}efKuiLx3)R0%Z zDf`jKQ9#z5T&lp_+vX3W$GRaZR-2X2Fc_EQeqfqMFE-^=8GObKNFl4?F#la3cDZ+2AC-Pn z{|;kXe0w9=r}WNavyT8s0qmu*g#at2mudixx^4BxHVd$iuqw~Wd z>~hVep_AixppNjUL`Gv!7+@xqDxr`oKn1v*E7N2kD~H`!6{*$sJvnRN+|$Aizz?wl zEQ=VB+|@xuYuQE$nce>ZPL)iW5nRqzqfH%)J!o}Au39@I;QDFzBgW@eJh zrhM)ms*k3+yF>aEa&6_++}rKpf%SWbm(#lTB#M<2hnvIpjR?$Zbii>_@U3)xWrqg| z4hc+EfjFJDSQQ>@ibk_C(8I8!l-!J{0Ru>#to=oVV*&G&_i5mAD0tBhSwb4L z4^hWuAK~aGz1#CTW%jzwQ5m24jG0-(`Zqqfq3YK(p(CeUapInbK1jspi!N>^n;rh+ zB~L>nqU*RY240w{GKlh_`O6KN$ox?Ai8gtoKO`&2M_-8WnYvt%j%JJd+Q%{OL1Yl< z8IR@)F`LOF_x2|;S--xZJN@!L15^|(gx_HQ)VEOkRB)oyoh-Eda9MhGF=VL)?HcF- z?SnbM!i5V}x~!XNMP-{aktD_pF+gC8>U~DA?j;+H^hfx# zw}YO)(NClvX#%-doL=_ojVA*3`Z9M{=kbX9xHV)*?w^y@ z(p3M(o!NLyX(`Zo0p{WN{b6=Em#IV!n*(Bh2>mH&Q+B6WD0LWc3b{1-_J((5eRR|^d&PwDYn|sRW2?u}6+D_!zU1!{4ww!o8$KUli}x-fzz&?eQXb3S&Jd`ym!gT_jK_U^f_HdrdOW9Ho>X|1&!JK7$r%aF{=|Y6DusI&z`rJ>v%o1_+#lC1Nw_4GggM z=)U|aTT7Jv1cT30Z=TGSb5N*Wa^}e7!vOes#eX)MUFD)r_L^83n};nh+n&!gAK54` z5>3C~#cM%fHJ1Y{1~%eG<}Amm>kq^n6`2Iu-4d)Z#tIB2fohxTitAaR`wW%JUFSCK zXIuSlwT#BI{F|;^cJ-8M9j4^b)RZ^Fuc_a0W}{-Ob6>daipQT$k*zlF#Aq~&L&#-N z44wCetNJF>oP+ z+o4t?j@7%pOpf1>Ps(hzAKR?lJoWe4dC$}jd1o})FyEDxPca{YDq3w0V%KJ zp7Jw!bmT|1xAb2%n!qKubk-fNN6$sQj%Q;2Qjd!#|0suT7!#JGZZE{#aVGDkaTR7m zLi0O_3XN!)X;%VuFg1|gIGP1zQZ=gt720O9W~m{MnA;ZdPso-(_`{dPyD&E@RhTM4 zlkQYYoF(7yJRGOSzBS^RpAJ^Sm;??e8W1UvtvC&QrJvQJSy(F5CU_->R=tZir&}ry zdGMdm%h>cG0saZO_u{w_C+uENUcol> zarRK;T%E%P33Ds4Ii63IUzQ+R#)J9Mg|aM-P74UO63gS;YVWq^SqJMrJZK=bEF3fYwJtlm!$%-Ve4eN(c+FW5{R4a<;JJIE>!)dS+$)-by#)5CcT6G_0k z0z3)Me@jJc*{{{fSB5~l+r~+6#3nfx$&MnUm5g%3V1&XSq{fSNX&Am1(sW1UZn&kt z^h@33N;RFXl%Fr!@!f+gnbraEnQPE^fOAV4zM!+7N{MT#Yg%|zRAM1i7hsr^Ae&k> z>;q$RG*`UvNTHA=Rs7D4*(WWX{l+_ z@PrVl05-SeJZV?GWQ44L!*h2x2)r*qEE5<{g+kc%aYa}epEl2a*Bf>b@;}9U1pJv^ z980F9)vSJ!9zzmPJATjCwWQH4ZV^08x>}}9rlT+oyH!o(A+pVQ_;OYd_G%tGse%S3 z{Wpnbcd@d)?vv0-Z+|zIBo$Sec4d(3@l`xv*Q18RRkU($O!K?R^xRa3@734`lt-jS zi6n$zHNa+Q`hj_WI0sS-W14NgA{6@PA82*6=Sg2ocQJ&ejCDZmi2-VzjrhZpjrn9l z-<$YkFj3WVl4edOmoYozf+(9$uGb?~uia1e&7f7|?_vdsWzWF7R)d^ipxZdAtB%`A znB|6QlfwoX{Juh>?M1w1@Sqob#!1BnCUJt0(uwfSj)Gkwx!n4R3kG^{Zw*8izb}+V zdy-L4x>--nj8TKRy3xbkGZBTL+=^|CQHPsgXJIwyeHJX$K`nOBZH+2@Hh2vivG2fQ5{|>+X~^x8JokL0L-d1yfRHJ%_bs#=8&n|} z9eQ4%l@Ws>5^<#Pxt(}t2Zo%cJ>>g60FaLu8c9qb)mD+tAq}1P>&AT$hQblF z&Kl2pF0nq7PNYmW3B5YT#<@8U3K%u&2rqemYH#~2Fy&bugvJ7Uy~pwa7+34jh3NbR zDQiaYxld|o?m>{Z;>Sp%nK&NPs8F@YM4Y8-Ts?W%Xlp1?defV&azo<*O;jA>oG`L4 z&lA!j+OUb+N81viV+fDQQL|da#qh#IruSDmGFBMPqzZp{S`VP936sg!B!(c5aEL73 zLr$RNZNsf2ZW8u=+WZAX5z$O+c^S2bhm%PseEu6I^*2Qd>g^mFhohmIustR`MIwy! zA@A#Z3&WtJ7~sr?*Ly>A^j`1T&TgY68yXDg<$ea#m_6#;Ze53>bHIeZ4dQ$JhTEMe z`Xsmx|J|F~u!&qQ4EAPgZgK(;%LxU6+ho-4t0T1KN9Z>x|rsb>>FG>lb1z z2)#U`8Yb)cX2&jpG`e6%eZ9jSDBoV9$-T$LK|j^rHuvK@eddliT9B-89Nz(Vw42w( zLSy}-nZ{u2*)Bo9C&8U-#ZRN`mqI%b716I$gU^Y@Zc{D>(=P-U_U)7`OVji~;i znB&&09V-|miYUy4cK5R_0zQeZ09kXTjT~!4!62>RxsoYs{D7kS2*&YLDVPgQY+r8= z#eAW+pnDZlN>zH0{fQ>yYb0&>kvI!FI8d4k)4lTy5kmYGr?fx3Zs)N-x=y|PJ~7|v zF>$L=t%lvB*~Cr{+Vb`C%1xz6Lsq-VI#|$*C(m;ZVOWC3 z^Q~|1YNDogbQi$P!=R97*k2yah!s>GK-#}rSB`1ikosgnfxlyAc!f7j6k_`r%^y}q z_j*O5yr`o1u>sup3*9E`PQ6g+=t{4>I$U%7$GkcN8=!L5Q4SMV%L(oxmrWb`J(piU zqw$C|TIhrtm`T(IwaX>U8xgM3-NX=(8}HFa5~MTc&qhOzzzWRHK%z*4yFLB{L;DDX z303^IDBw8FYHs!*qb!z6$4zMe;#s!Q(fKsge7;gC+Sc}Pb220}yx9uZvohSlWY7i& zO4_wMk*u2T9%&M~H>Is=HW23*Ve1g_I{@Beh&{RE7oDinrIN&#&Kwb+hA&!%wg?tv z01*ti-in!I9YhUZ=leqG=<5%9BH~aXiytUY#b>Un{bkmpcI6? zA2LL^{*rwf{W2T$crB&zW!)BFF)5dE8nn3n&bQLXVX>$qB%52Uvun2f(7c@vA4go8G94m#u@J5c)z zmYjYlidvsUbPh80`#~%#`b=vBBll)e&)v>^Ixah_(INIoxQFSs%q;odp9w_mBT3xe z|8<*fF|aFxL~B_Bm@^If9kV|}{_R$!J90Z>v_&T4Hb+f(( z=|nuvfG%x+J+B9lOk(?5ipi;CAetFGPhmb5irG9ujooQr(2zr3?Ik3@Z0oigM=|%r zjH(qpHU7do?-jdPos-eVK}hd|4`u6Ga>D>mGd;{gKCKH>8T)LeiBWB ztTD5XOwasw=6rQovB%TJvNcJ-uQs`8q2Y4orR$+yVQkJS*`hEZ3oXO6#a6#XWz1fB zzdz6Ugt1S(=Cs#XZ#oEZtfa~U0G1lriPxrYWWRm!I_ z`PwsQ0n6AKqXj9aDHs?T3kf=yB~-|9rKh?)vnor(|j z?=k6JncBUG%q{d95ywPf#(yQ^R9=Ff8u|WuG*<)_gIrzvpTXy%(+LCY9oWv(QIV-` zva$<8cTeaR$_?S1wK@zbU=n?4ldI@Y7Mj;#A~hNxno`uLe3rCQ6ApRic<5=i2fCLt zHgUbO;cL!oc#my0O)%lzDfgEOq=>CDgy{F6@A%IJwjqs|46 z+vFhyXt;-x@OuCg)c6;|5IhUkKvWQhCML=p5UNoM;&k<$Dx@A7g_Lphz!z%A;DMO) z2dV}8KAiT1Rn{0kEY!pjuy7{|v@2^s<8#A{b-Dcf;l5pzHXx}~w&vp6j@X|-7ymme#sfdTb zK@YKC_!p>wY^8A$*ZJ*b++weSPq0dQxiTn~X}{dVX*amGo^=C0C$wCyOpbE!SYj-_ zWO7&}LrK{27Y!yVGg=L{p?^eYs|`xhGw#z(jTSNx>zo&O5w}e%RAInyM;`pacd|Jw zyPz;##xptX5*N6aAg>F(iJjYhZom1=zS8|7#__qnsow|)o9IfbH=+x=SOf=>vbv$8 z;F3vo#IjN3mLQjA^hL~njf`k_5pv`2AgbG1bsAj|R#v}4A!<^@a-&qg`|@cV?8WqS z0_s9i6z}G7q9(P(n-@;ge84$Ge4`(sPOr@eqeiL%4Dvd^<{Sn~bT=~X9Q6ZfXP``p zU^86AdXYNRWfB$=NaHQ)bxQOPL~kSNzV!IZWN9Wy^bM&Jmu_xXY1m_g>ruGAwS~Jp z#0M84H9pJm_N37N`>vVk4VCYoD(^WwY|I0w z1HD%XJ3kr7zn@1?cp1cXX!huBfT33G-NsS@kznc^FhUgtMNP_Z$8T5@kJ21&gd5%$ z)THiH;xE)ZCEciGad>C&PjCL2f+-~=0S7T~koZG4mt=*W&u7kCeH7p0VHs4)b8`+D z3~D7PLFnN4d|J}hg8GCzR~F1cKf!DJzqpE}0Uaw9b<;Zlfvio!PRQ>CA|6RV>D|$X z*~Vt))@=4As%l0Bg~-CshV0tRR5W|IfvjYkLf(>@&jyHw7qW?xn_6KC{!4m(@Qo-L z`@*V7eqUaQ>zx{1OLzD9p~`g5rPsBLyT(drONM!9=>Ch(bNKaaVxN#yuCGeXW_z?u zN+Ys?O*Zm63FdLRc3tvX-SEw$z?GkYbr9Vf4?Ya z-?#mU!~;|XAn&Q=24ln`vV}q~M@DaBsBxn8L`#En8*lx)=|5T6+HrFNGW)Tswate}<5tpfW9cU$x zpd}+~za{phd3spGUG)EYa0!?h9xx8kJ%{MV;Y>!!DjJQ-K5gp8NG2)OQN} zHc2Y2HXSnaj_MF)BbXbT@IE>5ZY+5L+~GW~d5M?v$8yhHN~(+%9y6I5vkJXFk@5hU z_Li74P#uKQ<&D74Qm|R_6edz_H`PKQWOB)rMf9(}8O57s{@y@-`eL@1l?20EkCmWh z67{o5H?Hmv?JVMCQR+<|^iSB5Js=r2%f!H5`W>N{`C8Y02Gz1tXBTvHcns)XJD1tz1>TD`@So zzauXZ(2da+%g-OvJ<`u^$_BI9Irv1HGCzbKLR!MfdBKkZ$`rdEn|-sw;X{C$j+Msv zp3m9!Ba1A3tAMbiY?LFRYy$O-oF(^D)V57t2XJ?c?^+}XRycPx|ndQ05LxpWAa`dvwp2rKPG*KYk5 z_}u2lbD_J?aQB<@@H1kD=LvrOhXUr`vZT}J z&2_T}MNkRq@oUK%@{s6WYA2IB#hF;1|K-p)fPuYhhbdlE_PRLqMKhcJx1iZG_IYzA zG&S}HJxcl|{(AhQxc8qAcHsT^Cvy27TKN38SP8JS7(U9X1sg&PA^ZIQqnAH^5GE?i z%r{V_i2O=8@qp-q!=jt9bqHw zCKpV=<=cPT^6$xmh3_eAB5ZR1y(vHBM*~Y*RJ-UGaY2V?O9c%EvX+$5# zu1Qef`C%OKhgPfcDVGmx1$>bvJKOKc1HIOT4Yoq&J_!D^q z6GO&)GVR`ms)VVjDZHJK+}ZkdDtv17Xhkc$`B0UwQ6H1YH58Q_VDAyT+L&~r%{Fol zpE9z0RPVNdhUybZ)zV6llK1K@{l?Ej?vx*5bcVe9G@)DPaxaoDXp`fS#m~Sb0|OW6 zR^-^rKifGb`3VJ^THqNW@!9|@;9j<=qT}-mg4O>$9FGW~X8pd|5@UXGTjqm#}^anU|Du46+mhJH1{Dh8P z@k~H~??yDSMM6TteHv(ej=9o{Ne5OuY?GaZpC>((2-pb83?f+N9rcZZ9D_})_WrQs zC>aXIS)Z8^2}Nonl3HE4!Jn;f>~8@w>GnHZ`Hk~<6%5WdH;KRAef<=2nK-f{60p^H zKa*M?Gw-5!{ApKdnVmoBu6l1QL&fnqKf2DTky zUAfHiI1S0QiOPvH=2HpUD~l$*S<&!Cd%GZd>MvpoCDY$*r4hSqE_ ztzPEbum9I|N3->n*(}Cb@+_X?_PqAdd}nnFH_sRqfl$&upvgeeB25ruZuQyw$qqzE z{1jo9!WQb%yS+`cntqEV6d*ZIrUkMa{D}oMUk8My4P`Ls=&`CFBEYe)dix1k-%4^R z6sz!OD~uVO+!aZHA_H~3B|hY8HlKSD#zlqJnSgXWY1vx)F_Ui7xyEsLd3iaxT$-@p z4%La$|G zY}5BrH!!7A;mhG|p#69Vojv4BgP2HC{+c#to9MDAZ0avK~XowOS44Gi1NP_5RUR9nF6WGnw- zs8etG>0nKI8zvzf%}T8_l$niu<49n%vp$+GF7ptH;DI|8LdNE3bbPvL_ghmCwBHa- zppYIPYs8?*LSD6JwZE!nbz5`HcG!K4zuUE{FEp=*n*C`d&{(Clj|D-lW3h*u0H|e|ThFshtXW){(0eiQ>>07Kdbf_YRhB ze!ih=^?j~N0^2FnXnovR=LKWE&qsl%3uvvc)N9rjga9AS;VAGRBd)ecyt{gh?WXQ$ zgqCktfgD8Yxj{bV1lS}r@FIcZZ-3Pdk7g`OZ;0*$urCmAOX}YtfN!{Nw7Jb$b0YAy2kX^Qo%TQ9;~kwV!}N|ja#E=Rhq^}QLZ?+Z)?Ld`?s zQR);lXrQ1y>&w88Q~_PePWxLSUGv*Ed0Z*2Vl(3}w|f-G`H$5P4}Ia4CBwon;GiOb ztY7;O(piplr9(j4tubVedm%3M_YxH<2avv}eote7mU|dUXgzX=8oY(7?*NMmSxA}$ zzlmkT?RI?RoozD*FM|@HayY|B39?=IHdAJYb~(^!Z;zQVKY+8enM0aB%b-EsU7^>m z-0fnJ?R}GII;7ago~wt$S8!>wP+Dr?ozq~qLM6yc&-?=buwPBPA`XrRwDdstYDs)m z#P4Z161%^>PV^0rkuiVB#+$Fo!)<}~*#u%Zlr2UCznCBeaS}xww?Z&r_u3gx7ZO=* zfjm4zYdwM1Q&sYVfKXOTe-=?gyMY!ncQFYdhJY^5!Oaf}DuoOf+95$<76F5=-)$K= zsorQo4D-?jhd76X`22kjoAh)!oZ@gwM!BaYaV*QqRGlrC_1YZ=d4eAh4{a==Pi1}M}Ou~(3CMqpe7ZoK1% zLc`G7qI>CWru={mj&=fFlgt$aN4(@Pfds$57K9)Xj@5wB)S~a`U>aK!!W^5xYc8bJ zgF_hD_2BcJN5F;}R%_uq%wYM~e;6CWfB>vE?Z%L33@{WfQtMy2UYx)iC90>gx#1_# zZgTF-;`KI|0Ycl7-o%DvUiY)jZo}kZ3frj>?D<^o@a1+TZ}7`w{dZCbMk}z$ndvNAR0=g^tFia?5m23iYr4#JVl;j8r~2=0fu&&Pu{= z$zcr`Q|Dwv(%#z}94K(M#*q6)Aq}P^pcJus}@=S{^}~&7)M~il??|13W^C@!trvcC(g1tW`A{E zsLbj&PTT^bKpkD)lf)Q{dD=d{2pBtW3^ zqLB`EW#0eK?#Sdbu>|4LfME9pX5nYTVB_J;F5V{O-yDc;9`;=NW9--_%?*n(k;f;5$ zHi&qlDDMmpUdkAwTuw9aB|C%}KQGlQ3-j(A z(d-#{9i+=0*1K<@LI@DXv6rnC324w>#75gE=N3=J15Ze1N^zxZ%xvUbbwN}F)MV%k z93|ferCdo#r9b;>EVkgrfLvSwv?*9JMmOc*DCK_(j8I^KA;epGZI1X&F6GUV_2ift z9Gdhg>pK;~_z~;)gB~xJ^GR%*gaF8aZwU2cuWy^I7c=ilD$b0M<(mn_iCuWvvbI9* zZPMw5v?jf-N({b0q*1Nz_U>UJLXyq8&{It9l#`h)e~jZbD)~%?XJmZ9_h!rn1#e>0>`WPNsn=h??E6}vqf(KHwwrmJ0~*!S zs(%KHB-%N%NE`!1vdS;GpCf;k&co<++GJ9n%_p*y8TIetyTK&H* z=EWzJ8cp6c@JRcno=@gNkely}`lYgO_Z3veV_l`{0)< z+KuG9kilLncIF58q+kY;`Verx!&=5fN(9_k%~6|n2kg1rq}#9yU<=?dB<>GLIugIn z+XeECra@6cnQRlVdTM@Nl3e>@^cg=bE@Fqgp6cu3btFqFo(?2Sn*_G%CS)@#8vK!m z6VfFK<9Nbw2%liY?}HX8I`jq}cc@#6h+Gh`9>i$%2)nA2rH0ffkX&RaTQ3iM|5gH= zzaLzNSow({x8@5q74-wv3H;4buPcMDaH8zk8!_gCrm8S-ZXQ*5w>bM_~kz+O<;)_ zu|(LEocsUI1UTjYaP$4SiY+bvvyuaz7C_Ah`0)i@kCFfEZazSzO??sKlpmcfnKuGT zXA2)olTAeN&7Lhus1z2-V#%h74{YMXo)jf2?A>1B_xR|4(WQ6s54gvP?P!Ptbr2#_qLlrd|rgG)4Zg{k7ML!L+vt%*jAxp#D zrxzvaI4aOYj_JSbfiaw$Vd`+P7C~3un-ghZS3olGO*>ODsD-bFWn7~+Oa@}I_QY!_ zO`h;p)x)AmRZ+U_iPo9bVeho%uymSL$?B9gh2vi~Q7h~iGzL}luDyvzE+=89%8umM zU95R7BkL^V&8Pg(TQ+~gQH=Z%YV8{}W%&Pky2_|1qpnLgk`e;aA~AG#!_c6B4Bg!z zAl*oZ3@we)h(o7@ba%%P(j`5>H~7Bq_xya;jVJE9Yu|Iu-W#FPBVm%|Rk%)_Ql7R7 z$=B=ThfWdhN}m3Ax#d8yrt~i9gIDKy?gpaHfN0!aR*hJJHn`EKDWQvaE3GQ=YcLW0(JC+bF059JYfB z7(OH3Cpi^20`A4T`h4cF!*?-=l(es#9Q+2EUF&H~JQftHiFq>m{1)OCKINu)Se<7^ zV|!Z!jBB?Wq%FcR^6yo6O^gITI$DZpy-LyHhRqUwVGefZ=zk6S`g5<*ZGf6=4mj=K z|M}kgJU(x7;wj$O>*CsFc_Cz zIEiFL0MYt3=}IMq8>OwpnlO2Xo=OE|ElmZ_p+2l^$vlMiA4OU?B?-X!J@DmBC7EYE zr}u35*Xm=-eS8yrby@VnP~Nb1d%c}jfo9GV+g;AyFI zQ5>w0+gd!*C){*jlpX?vS_~v=s!`7~RQTz(+F067mShY+S%y4U9qrBe2C3XyCVSW| zsg%t*1MI<-NmJiG+GVV~Ux<#TPFILJkpX4EAC(7Pz&>NyOpXNz)luC_$VmBT4eMF` z$nEnR<+jlfyL2M2uEMBUUhV60zB@3bu+g9`Kvmj*EG&Vw&_XFoXY}fe<~3!3Jmv-0jzRsAvzmMPzalP*`7UH=#rjJ5&}6uwu8Hew~4s=jow_E0eX-7j>t*5#{9IN= zEsJ$h7Hb_=J;((t-`M!YpT(h#eAKFzW76Yx=F(ztPJ+G7$fiB<-3+Hw!K?CRxjIX> z1oIB0#9W`{xWRgC2Xa~C%?e{DAbHrlJcE?9Y^Bjw36AyO3VZFLVlM$WGDBVRRHWEzVSgIGhLXJL z^k{X>m%ERq;=YrMJrJ@X5V%j1jJY1XLXC@ouyqiOlFyrGUBhfo%(sN(m9abcX2rZS z*XE`s&9r+omT86r3!f@~r;;dR^7!$epNO+VoPOQ8e>!=CQMzYoSkk41se@+EgR>SL zcDn|*7^1<4%Q!JAcHg$GJUUz9=iyw-|7yjtSGVV#zpLXO%u=nMz zycbLj1d~tzz(v*sl=Xl5E-4|LyEQI6WtQ>k>MfHCa(jKm5t#*tE3 z^t9><7H-hRK8-t&!`XCCm?&lwr?x5qS37nGr-!5&tk21pwCErS9Rlyeg}0>mf@FAH zy#JuXlh|`xXh~_L+8G*9T(qCGim~Q(GOvS~;xf!J zX7acZ4c!sp4J|$poYO1{#`sQ(Xj*-mCoCr1L!R?|&<+m&$+HA}Ghh^t_8> zJ{KWCf2$nT;IOmD;V{~6$`VpzDjn14^}~7i_?Ixw%uL>+nj#j*%=cya1~N$5L3t)M zJ@7i>MErUBnrtkOoJ0r%SMo9(B2Q7JOH|XEs*r)hU2$G~V6l0MwbHQn*zezl|d9`2biIJI(y{fk!?4l_MQzGdvBT>}kQO-|CPG;}1+)mMowuFHuMPf{}6g zI}Rova(e#i>Ilybe?a>X`u!oA44U zwm`LoEV#8>A%1||BFLAdWXyAF0Z!D3cg#}G_iNhb4~t?Pn(H@K){W+<-CU3(7f7Sn z8O+qn6sTgIVM%>?P0YN-zY{nOEgHV_PjXnjtG0Wrj^0Bz|7`XdZf~TYmO$e$KM+aP zqKw3(P-;{W#V)brwVt@#=!v};Xb{Le2IJu(6-7s)?dNT>I&(10?DM?&{}pF(ql0^? zM~mrmdZ-*=^ZTQ#EBHF`_hs9r+}31qd9BGAmd$(vBLb+A<_uQ`nev^GB+d6cnasN( z%BH|!nGosx@Pk)gF__CQN|j3ewe(9OV%b;?=HohC>=7Y8r0BjyxGxgPi4>%=)M$|- z;*(A~*Wb~pI*3~s!o#^rn<63uyM{|L2;Tv4c?6oiYxR_Mlu@N2@2xif)D7dIGNJcjn`RDY|!%)meBu=CnmC^bPkEjwm zciK^)WEZr8B7!y`6tjDTkO-;@3StIn@9bIhH;j@D4fcLav9xVPk=onhVYS5lqJ0jZ zypHdbQx`{Ea@h}V`N>+!LkqDOg!$jFNaw26C?$Tebw)r^dvSi5$d$|hG-K}E{q+yv zOGN%#a*8P7OcOrfgHrnBJ# zKGm|C7&z>1q_+MIN6Voe)qNg;6LhE>uNw<-0DNREfb{LNly4SOC61G|wlcs_%B^}6 z9b(>5I_uj3o?VwC*Cx;1`f16I;w0AGN^Y`C&-9x;r<&0E7KdKDv)+#ztxZW-{0)Lq z7Z0AJeqw3VWZ+}gD09l)4fDPkEzNufW?&|Y&roHT!p%z$iY-)gjs&O!>#Ref6n#MA zFgZ{Ev}<=Su2b+%e@?sIwlEOP-t;~MArYzRGGlm`EGz{cck4H7#0o(|E@nU}fsGFn zF{!cr>b&UL`&sL@B9V+trc?Dn#}MU!RxqqfNsB~i%e@op2UU8wNej;TRIbeEO&BJT zl5o-JWix_`T8^fByNK2-zr45?KWTb6#cVmV&-s?@j<|?WOob!d4Lud2n517|Lkrj4 ziG#su0+h(JTumHb5^udNF&I2Nl9NIepbeyS<&(vMmJgZn|5D=U!K9<7RM0lE0+i>%y?b(vjDskgyC5IH=3b*>W!XH@Zd+0Z!T>z)y^50%)Rswo_0rT z3=hN|!m8_Eq`GUwZM}azt6VgCuU&+K)#oQ*TK0tv^dY&q35d$s4Z+8F>&+{DmRkE3 z)_C4A{U3^GW%5+b?7yF*c5}B0aeA?8+3JMoZi1* zh?>2KOf(q@0}E@jlzwA2yD)BH6-ghvlK2+4DGboWnOtCvHGjF5mzj zS<=u)knZ&!SIl84h8)f#!|`Rr2CG^16O=#f7h?CFk`{4gZ7`?pvsb|BnhXsXLj=3q zTSxKGu>eoemqhrN$s}AUD0xh{LE9=2&Yil2%tL6IONc;K# ziBj`tntI?DVW}cfK4|di!jq}8|wr(l^~vi(28PPw(~>X7w>)C z8sg3)#rFVytX|s>LE&5|JiCD8tG5{LtM?%vp8~eelV50$*&w=vFbB>wkIb6N!PH}9 z;BF6|B9_rxy83+3W&_l4` za_-vf#_e>T_Sz}@BNRac#z`D#IB9xhq~XaNE&goqiY%HI#cj zaBWXW3PTcZkD49(THvL$hUft1Mj2ZLo1VsP;_%mAE4cJv4_4@h03)RVzgBJ(lb00B z4Urxv%Wvq|cTz7Mg`YvrDh7&~W*>BC7rWJ&t3_h$P&!1WNRan<3ka|jX}NmM&~kMc z{&X+QGOvoVhrWsN+W@&()pnLCh1T37@@hnf#+aSc?50wNP35@d^OPl==u+!5gH-t_ zkNcQVJY>0ev$2WrblHxqV0pWLY1h1asjC5k(_wN3A?JBpN24qEmbW1t#df`JCbPE$ z?{$qt)pB%H(|6S~`D31J>50@4D!tZq#asgP4xFg0nbViBx5q+R!sd#oH__sQ(NtH@ zI>}B7h+TBwZ|k?3;ii9*fqP!zZ(=WBR#R4Bhhy&~Wx3GTGoUQ;kbvb|98H?KbiF={@Xp#ipVUkq2hbki*Ij?yWmJXrXC+Wj<$UQq zXKgmCTV@sFJYz&xh@?PonJ0y|_Z~VnPWW3iipk8bdaG_7A-8Fn&t)nAM_M9YlPW8+&}N`}aSXwi?U6oW7`_9(AFH$bB}>9AFwtM|Hn* zBzw1q+!e3Y((MPgfR~n|MD<7WF`exi3y4zTvxHZ2xuO*1qk6ob z^k1rYSQjOjr;oKGM{ZAU4@G5|Yzq%b>7%SKAQRb}qjoHPZ)nGHMUB(EZVNbigm{sup3rz}mAvc_vFaRdv9I0N|Em64gib*BTlf}T3p6MiEX z)?@dqT?SR?r&1+@zgdEq)O`nG9pLD`&+nQOsNtqC(uU{dnW9lkDx)dFbuAFbRmDXU zTeF|?b(YlWSNnAo+bpBz=7}qDSpt9W#chsf_Upei1B3^ES1y@4t#L%wn9-7f3F_h8 zHYJ3e3uZH%06D;|ap0X1?8nH?{(c1)>S+9EP1x3~f0D%U>O}*FIKE2(KdU$_*TWB1 zHKf8uZlR3S(IH|$>I)YtBQG@Mol_{bt~aJaYCwir_SX7foG%m5_O~~$sy_RBb!G+q zJ6Cjv1`Df^Cf9*N=d*t>INc6jy1iD(a=hu}u()q7=KgcA+N?T~9Sa*PX7%tf5aVpb z9ePUq(S5Hx(S@4lvnn^;)*wH9fm9B@JB;^vmNj{?X%6VtcJ^Z*DSffRp{fEl^}2 zat8HrJZd3*-g;rMu0|v5k9UOMin1ECx7B`%dmuqEu)cE1YiPKMWM@#aS?^t7VkuNX z(H&)FL7GOd#S~iyIeXguKH%|lw%FfloUk$ecM%@7=DS88Xj}i1(LXZz3#iO;&liz_iV1TIn*20gFLCu0eu znpwEwC5lcVnV*@_ZbJ$H;rH!cI}2;G=7N} zcWbPeV0$H=HsQMFz;JD9KAG!Q99{4|O0L|pzD*e;SsxKMYlPrn_F{XWYU0sez#+L@ zyuCRP_io27q(83Wf*-a{?Z2!{4=ufQGZ3h8h8F-dE76F7E90oRB;9Dsat@$PXisc} z_mCctF9<0;C5Qxt+upbF)egXbrR^R!y!0UvZPeA)M6I>)r}aS5!KFg%gXXAtt3TSK zx=zJv9>EpZ{%Fe?gD_O#F(?&9AoaybxgFF6@rD)=p8}Q69w8q)4A9Km7-{9Ji$V$- z{gL$3TGUdYf~hD8X^wO_m-OsK`?WH>{>mGL%8s$SIQ= z{g@Km#Hepz;?S4DI#E_f&4XopPc}PE-+ylUWz)Q6PzS3_Wqj>*6O@OgWK3?2%^*_B zSlkHd{w6xj9L|_D-CGtjWbOop*>F=8k^Xi?XJ`U$AF~I$xNF?x6^HO4%nLG+&fRpf z#Z~1qf&W5(3ek|z!WGz(3577#a?ZcxtxNKqfKt#k6W5m)Nc-e|il(i*vIlPn5h`}0 zK^dw_@%1??0y!%1`xy;zYwT9!_mvJq=g8uH)sX>olxOx9tqzG7awp`l~Zwhp}o1EHzAGn=V5t>xZ zw`NR@gtIK_HS+gU4>#qa8cg}M`$ggS@Y=xj&JXFM;UlNhB{C%b$t@n)s zZRRJq?w^och;7Jdgj@$X_Kox&#;g_)(`2@YyPr@gSHB8%DG}MYer^*p=cng;4jnPT zi;H1>p0nQ$W+l|bVc!ZLnn!E7FV{AdpB#S?^0`iVIITQ2|GsR#Z^Ix}PI!EJ=Zwor z&TZXc>-+{OR(U|LfDjx;6Ddun91tVmUS#$Dd6OQJhG(0obx|d%s)Ew^evtyxR8}F8 z7#i!YA?4i=0cynNXl@!-`cQ30rxpJTbO-t8L{+N`ikXjoU)**}4-aS+yF)o5PikF= z03!(RDSa0a<-0Hi^`pC)Ejv8OSotr|V>PW1wX_r+PTnB3z9NTEt_rSxL{G(eCsE72 zwUSelkpai<_%DM$%&PUOcH?3EWJvk(v4`(+_9NbaTe?GpRiJ#kyGWGg@w}A=WrOXf zE&eoo*Q|qX=1(`~xQ}h8o;O9Lx*!9yM>4?pcKe?`Pl?~IZI}@0f^O`aUdwn*+aCqg zTs|zIMq}{3&98%}qa0PIDGlQAT#YcU>Wq;yWxu2>&EgmN=Rwh-JzR0i21 z-~irt0<%sb4hcA;XuTf@*uE%Avdi^4*(F_RUp!gPv9mqiidW7SwLbJ+w@R@le^~U9vtJUX$-3ik$faw+!5rgtYaO}&+ zkN7P7O+d;Mi9pQUX^EAw7g-~p{_3CnZI7hSSeu^lu;hMVrdT;}VY$fO<#unarb=y|7X)0Q~LaN_8xTv-aa z!>kMzPY|#veG{&urTj+PspRKma!f81WNUlUp0)qC$Gl|I+-Rt=V!P2{?4Z#>aaz$< zn-ny6(Krh+QYt50P^gT6yod4I#gVjXxvtpxWC|+yvNs~<`D4Fwd&(+9mkQ*Jicqv3 z)x0oo9j{HAY|xI+s&jaGSQR){He*tm50UTIgj0{?lr5b#vrAJ9lvnU_%ylahE(JP4 z1|Jz=mViD_w}jc!;Y<>c>MhaxE#2`MgB<&r!t2l#(j0TpfrS{m?LyBVc~`xmHcHWx zcc4$ElMmE~;p@i{Tnd-D>AN>j{=V-s3Hq{GmGXaeyjhVMUtmbCpJG^$of!PyEN4^| zXA$U-X;~Td!nKw`K*EzpfaajfjCJDAlLz0&dQA?U(M41dbLh!_u#rSS!Vf|~d66M` zHgoH`p^o|D_AyZeg$4l;LoyE^iG0SVCGG03nwEF5lM^L=kmN1KJnckW>f%xLHg^H_)BlJdXShHh|>;Nw0|+4&m7 zf*1zK-8iAOiTH0LNgC?l*pRV~zZdyi4*zK;>4RxO?5A_H-R3BZ_Mfi(3N(f5KfA^v zlYi1aKP3XvHsh(f{mOZSipBq?+hR=o(KW`}d;7l|Us4fJJ1V>)8vnf*0um}=-a07; t5%Yg%(Zkd@9r|P)h5hei&}h&EcWBN4lc^1)ClTNmP)<#@O4=mk{{ZIVFFpVO literal 0 HcmV?d00001 diff --git a/images/adb/instance-principal-2.png b/images/adb/instance-principal-2.png new file mode 100644 index 0000000000000000000000000000000000000000..31afe00f0508f6fbd0ade522c2e03ff1b2ffa56f GIT binary patch literal 62643 zcmeFZcQjmU|2`~8v?!wlF^n37sKMyL=t7VL(TNsBFB2{LXrs62L~x>$(R(i;qGa@5 zB6{zBJJ0ido^y`#UB9*7cfEhU)>!uJa^L$d`*UBP>$bR^S43*Du-~u zJ^Clf(H#;|yjBU!mt>?%cVEw9l;4iwC-pVwX~ZaZiHT|-jz&N8ka|DarQh;sq)Wb~ zE%3c{4+cvC1q=Oza8qVE8^3rCtOZ1+RDhoj?|CPJFLy^S=fHr5%{ZV z>S$*6*2&V&xxqgpA1G?f>bbVF_EQxRQ#)HOkJWK`}b~cM*4r9;%qI>sQvUAy^Nis8NC44V=f*>34D5bdNIe><{}y} zx&OHw_$JP1>FjJT!p-gG=Emj5&t>Om!ObfyEX>Wr$IZw02sq=B6a1~SvHPR9PE7wU zF#r}2vKaTuaP+35Fs}>bd)^jWnE8iqXR^;?jc|Q4NZZlQj5j507T*LOSn9gs{k&DuxTZxsZ?0 zFXa$?5{FcxfJHsfe-}G*Py1x>cuwB}Vn)$u;_n}FD5fkyY6qV-HtfmYKh1aw->evKoQ`;Mi-lfF>j{pkdD!0zLcGULTe-Ol zm;4zfdx)#uOueu}Y~w$!z}+yz>dB=OC@p#%gWvC?I%ar%xmQq zAHo02jVXLv?lX>7|Hl#WTX?k_)a=1GZ+J-=lhpbDZ_59ZQHf%b(#nS6ap;u44k99d z^lGGvf?b2T&D9I^k15wpgE@|0yU-Cf`TiEx#{QsW1^o<(6Tv;sQtrkzTkEmH9vbD* z(3@j|cyj0WWhNb=<7-2+<53L$o4-neKgusBaOs=K1bbWm8Y@yRA!!Ll6SPLsdJ@ep zo=eXhv^II;PRb;=U!HN@oMMcnX-`moNE_X%w}(@g=g0g3U`9LL%11U9tR@kb)BVZ( zHlu&M_}R)OATX+-DKvqyowS*$hoihN-!ZLM8Mocht+veaIqLN(GTmSKs@_R;XoXTb|{%nc%)nC zv^wCr*OR9mNe6+~Xyhu5Pt`aTz522~Ru6Zgk@QZLKYsW;-^guhmy$!jwlad|*x&1D zJdf)ve{n=ptwJ?Tgl3cD=ELzG_cilQ%EP{D@%c$XLqYh}Ddj|=4!;-P19cHnWhSj+ zJ*(IUPb&$y>%XOY?#+&U`Be}NrLp*|RdU2eb3Q1#?Rh>qyHybvXCmmbuIAeJs0!V1 zJhKXapqfe|dzJ92$=j<)CfIZ{g!Rf+OW!5paBI5n)6MLh6sT#|1^1nk^`o;Q(c8%k*ZEFC8 z36zxc#3T!%a_dImW7ETSoeqIst&^nE8@lc>$(6^ch1Bi~VP!?sc;NvG`+-_QuX4ac5-CAyc-ROGPuV2D)p zQH^udncB@(n_#;teN!)1iht;OFw})EH|xu`oVa61Bmeli?!A-bXm@r9|!vsz1|EOt#?fbJJn|6O z%b*=C5<>?bn?=P07Nt0a*?Rc?14I0tVGa=d;_fZ=y1nH-+3e!fS<^0wbb}r~6^UW; z)~JgM=UXAQ@Me6Mia4?s%Jq4=+Svr_PeMh+|EOgRH?x3L6N#;F!r(=Dqs_NK-MW}P z*!su>J(CCO0xUO|ITGIs8G2nQt;eK}zl1)WVLcO+mz1S9PV~8>mT+T@A*aJ6Tpu0J zs?fm~X`eh>qpM1;poAS)I}GXuO>Vm8tI*tB%%5QIA)h4p6b!%F{W$x z1%N4hTEC`~dxMA}Z{;Cg&Ih_)vaX+!_HN#8^II~7@y+`t6gJbfqreLAhDOk)6bP0& z5%&cOEr&KgL?6??ds+0#EJer;^~a0PXQ+4%t^1QoacsswP_F%gafj?Pp z7Pi`9zOGy6S~NLu1+)X692ZBfTWJaExDef>wQs(4)Qm-ZB6uZJ2|PRfzc{ptQ;%7_ zEMkbmD7jq?;vc@)<6G`aJicOjS#~>}g5#yL?wZ@1H-E&lYjz&1)4F~a>K&@$xg%gb z_E@_G?;a3`RaXftZp><5KQ(%-+tZW{O{G5!e7@c$$=(R%C@!rSln_lL^+Q-r`x8r8 z9AYy2n7R^UsBaC_x;; z`%8C8td3`Wni7?RNzQrf=boHz`&|EmPoivpxf=brI^VWiO?5&hdYx}pZT51M9Jrnx z{#v!A-zly=dnkJPF?RFvc-HL`+V7jpfxhQ%hupNd$5CaLOmOY9RB;bf7}_*aeC+C^ zOHw2!iT#Z2?-TJE2Hw&{Oka)D>V)LgDKX_Ui1$fWSdGu+FTim>uYhQY zb?2`AuJrTu6uZVUU?qID+#7$%kGVIhW%#A(a?|dp1qT9aeOp6FBiNtgIw)~om?*x} zj3rJSEfjG7$YeH*yV3H%Y9wEsi}d^w}Xm|a-VV~X)eN7R97WhvaA=+()h{(!is@%U}_Ay~q_UPN&;qWN%h@&^Ll!pUiEW=JyV*(tHLExK7btOwgaJs4JV zo5xf8#}oOK{s%VgP@#y+`s+;fs<--)1`K@FRpD*{q{?8Bl5!&Fp;-jK#enm!lri?b zW1nbASb8-i@_fG9C)|rc9kOtDUK$(eF+h?~8OHPGdu7PiRDt0f=Xv(HrH&v*+0b|7 z8`1J~yCNYgUo3}Oa~wz1Ql#>C(^yhCU6|D=ZM(=OOOx~U6eu1(FU>)U^d`c-C(^GW z=t+IsW}@=Bz3pghnN|?x44&|O6@>$a(&d>N_t43}r&HWisFnZ^a-E=p8{zud4&Ks- z%+LCDC~nPgrui9u#+SGR2@(>Q8~jHzqT8>Ix2E;iM(j$(&pKis*7!u6Z)r0k(2KG< zq#(9-BF3=XC>)4qX&89`p|ZG4KqwonG8yOvY)PRqjZ-~gIgN9)9zen2I7voY?rV6< z3ucS6i<2_bhzA?Qeca<$!)lW7$1yBvQ&snRhlnSPL(CMWCWGw>7C3D3W^om_1Yo#~ z2W#6Kmh2qI(+uP?4RKOzK_cDq!e{rt``sqsLoAf)-$X07BcBvc%XF112DH)G)z6cY zD060POnmt)-$z2tq7izICE=dE1jNQg#t4@8upY#R-EX9Hq~*511S#clAqEVNfeId* zUw-JdgNha=)kkZT3rdlW(C%1G-Q}OS>AajnaTR^I7Pdfdm zyX^@A%us(a;AM>vPEcZuLzql^pKp%ksX+{9x;0sJRL6_;MeFtlL^r=HbAJhnxc`8q zPuz?(%{j(_TeoZNN+{vYo-mhRwOV*^E`5SRy{SyM|06^uJ_QGQIZ^+uKMt|r*3-JH zvtO6D*bon~8IFQP@`IP75qzYWu_D?*J%R}WG1c6gOR;K49<%Q2EYYo>i)(w2So z``TV0#r6~;8_Sl+Bw+IkeY_y&gG;33tg$@UAs}oXfZ#Q>8;eP0-tC!WGjZ#XU{e{On5JtC4I`GOWaTCMw#dLe8ni8J@5^eAI3*`pj-W) zA3YbX(cxCsfaCUuE^~y>X0=^SksELpO_dtU1m)(`ST(KRNCW_CXV0@|d1kZt)lInU z>lOT@Se)E!Rw_IFpBNDUk6JvPiZA~Gx`+UZ#q~+{Ke(S~Jh%$qD8e0L;%_vAu@uXf z^Y--1Npp(6_%FGA36o>tl3Gu!^}TcDYl_!(Z#;(=yLGuB!MY&yk$&ySc~Y!IbAWJr{RG8O z)Q1nXnw!h*pQifB&bsvaihX3m?z!As`l=Qp0p#4RFG~#4Y$1EG)8(>%b_hNhFnY8| zHpZo*s+n>CoPQ6c;8;OUAN(BNoV06lQTGFK($|VUWsGK;d-Uq{Vpl|!_vP6*KzoEcqDj_O z6&1?>SUgE-=xOXpgwkNi@Q;8t@=yyqeoQ1p1$aLb+GIE>D|9uS|e3}$SbAdWC&-Qg&l6eC{iG0tt>^7_ieByLm?!x9<#Osx|Iy zf29ua@qEGJ$>n^!&3yL+vBI(r@=T_2w|QlT49{Zl#pm1~!t{7ZsCA=eRAOzJ7R> z%x6~LXd=rbFJe}ga{<8V+GY;T+He<%#tPAPU^zi`M$k+G%-iMp`F4|?MXEzk#TkHE z>%~u+-=Ry5f0H3VLYKb(@w$LUb2HPQxcu^Hl6zW*$ca4fxNf`A=`D6vrsA5N4ro6w)OH9iYj*8z_1ZW#EmEBpaqVUZ*1q<>Eb0rm*+q8GCYfSR8N_Z(AHHP z*Y9^+>421MAW~#8MEH)7jjX-En~`oJku~~~;L{^uwA`S)9qw=%%C1N!{Gu_(F@pbRaB>`^@(< ziIDh0f1v$&Y`3rHv`n_$V%1n)eY>v?#FIs z8D>eafC<8s-}62iw`eSk8|0MCwL=(yEdy{Cmg=_UWIb2WQ4BJAANx| zOC}|w5UfZhBGZlQvtK3pBg_;1ZYQQ)AbGm^g#gqFcha^A2o=t*tW)N3R5(6{vemWA z*p*LWU?ah;9IbP++r}|7|C(+_i`<-ZrOyNz>@?3b``Y6EVZ`bsbHV%o&5)4XhpV-M z$35}M4$%hCs>1wxMmT~$KI;zNrteb%AxE6hS;6pGDvPtPo^t*4uZ-lfOAcrs=yA5l z*&y?HX8NPCh{e9iN~k--xG81L`%{g-ycyfu%>>&r{XtR}4l)VJEiP4DY6Tm4+ZzX& zhHRJ8iNA9W66Sp~qClyIg+p{8BJ4`ql@>-nB72>*i0-}}rzUwh5^0DesAe+n=~}&W zF0fNGC zuF)?9EJjv@>GvgE0wmCOv@-3rtUH6^mjYuC&sm>^lgl;Wg`hoxuScCdtLcv|)e@2~0d$ipdZY zE{iQ=j)6_0VW$8zsBNC(KTh6#XxLaE571$^PY8 zhludol}5mx6G#o#vznyne5B^>bl+&YzLNNao1V;X{tAdEUUVaN&t_K*=jb}qw~bYI z=vFZ`*XPl%RgyGe7gUC4UC_?eb{&hf+_Ui7w1SXc03jITeJ)yVgHIu|k zv~)%(h$ab#B{WfCTYjxA;gJT$Di>af%{a>Vq?#1B~dE1G?;41F#SB?e%XMiH000hc4nY0O8a zl$4Mp58S?f(#<`_UAk3i@eyU(4T43o6**5;9<81*Xd4l*IW_CP7_lbZcjQ4?kXV~P z963wc6d`Oj-y?x7;_@li)}?N)d-tp3D{OsNJ-%NRs6Pen=0+6&hZ9+Z1?&&ZvQvRG}C+KEl>Q-f zKk)8sA#TN%zdw&G%2J*AsrgH=XnLiTnC2(t@ear#_Zp@|Z8yOhbGyo2$tJ}s95lp3 z2OeH%zL%b6%gLV0ec*2iY6ah-CUt4anfE5VOv``vY9hgTm)wpcS38t|h0(V~b;ZQo z)kJ8MpEbmU(XnQ&`8E#Ki=tO7<(pb^O#@gqpPn)*c^-L8W|!qW?d1{N`M~QHbR5i* z$(-QesN#@8--TuS-J(;h-}J0U&qB28P8X>%f5@K0MxduJ;dZ~ko~b+)KdP(dqpV4i z$S0Nb1$fdgFodF4OF@J+R8GAzDkegdQocHac(U5!2+$IpqQ92EO33D8MPM@(%Sfwy z7h0h=tv3Jqriz!J>ws0qN5<>jpINN0SK5;@oNV%KW+YqzssznJ;G`;p6~SEDczWWj z4}CRz8=k-#s+9&wYwSB{E$*$xA?T1=Z9x%?yBedk+Nj`jCspBPa<>hVP-s8Y8Cp8k zCM(Lh+!`Wz5G6C6T&({Ybr_Ohrp^k&Oz@m1sqhSAaW#g6haQ71ur=-VK%CH}kr0xP zNd7^w;WVXGi?hCRSvsW_i?aZ>mR;5l?_&wD0$9&wq%DW;^ih@tE}P?a45W$HWwwQ+ zi@1Cs0G*U^g4Eyh%%@AjNNwp&bk5`+lb~RaNp2lPAjnMKZ$o=Li58N#ITt@R;S#nJ zgG+tOp;seM>5j+q1Hk;8P zKF3sJtU~u)Fo+EDy|H12fM6?Ps~8h9u=2{2_bWKn)8JuT%j`||q1(5qo)L{XaHTpW zH5rdvt(E*`aWFDudGC{y6bd6U3Hkz`a*Io{Ct93zfGHvfu_;@LS(68$@ZA7hdPvA1 z)C=l3&z>+MOAS^f9bn~NmnjT5^JnT4-1|;lytS`-Ins7KS6?1+BrR@mT5_$PuEi!# zm%3Ccj&3_P!n*A0X%A`7>gtIg5ZU`KDEN(Q8$BR;dZI$NzQPYb1#b|xJ-LF~^+ur~ zXDS~FcSD%KHn^!h@@bV)mkzr5!Dj{h2h_f#oe6vgi6WS4sFn~B_%5~R2{95)%9pJ=MdS&JZ7k@X zL0q0uX0TI2-1dSbq9V6jCx~Nha>aj~gsx#hHx5>&e2sWz^ z`SkFwHVN{#m>&AyZePszdYl+Mn{hZfM&SmI4&V;?wF;sYAG5)R{M;ud_s#68nz$p~ zsNDW?>ww;>MxUHH=>Rf6I^1w20+0A5?fED9>yn7vjy84`tO!SvZ!gZIb9>ZXLc=yr zteXBJr9PQw+?h08$Jc5a)%_FY{@bj;wgV7u_!lI9VBO%=0^$fVM z*$numICf<6iEglZnTLyclATC3@%4kQMc zrZ6P~)^7^P#=pI5+VTdM_x?`Q{eRjB_Hh?aKpuSg$8&ZLjgJcNUZQNr%R9aUKUJGO z!~kynve2Q72G}qgK;Y34eDP6XsQcxW@BrQTXE5*jVBCRS;TwZnGeOEk(fk6Cl@(ASJJ`KkHPy{InVEcDiTC zX*>DyvwBhOBb{r-L5?>j8;b*<0zwf&(AMD zC%h=s9`a7-?=tY%d_nWAsRXbHS*fApvj*Z!ItrbCmPfmDCK!7xB$Bcyp(zay%D3zq zL9#1K?pJ(fO1er)Wh$~pCba)FINc6>f&z(%l)L;A{B4xcw{Hn-?2pM!Xz(~jN%;qB z_r0_&blse2#CAH}TShtm{LW@a0kAmlm_KTKuNusd6e2YP_0dx^5CQLv$K;H|b9 z*U+tf9r7Yyy9Zz)Oj(o@%7HGIRpty6Z9Cmx9rr#Ls5C_icLLduB_MfM0^Rl7hhwA4 z%&DBfNge;0W>kNI$??|@J4Ax^ThwiWd)fda^qYw(!>4%Yu5UB@cF}~xuu+=%nY9Nz z!GR0llOD<)mgrPW0ExIM;r+*G*$AEUqfOfl zG_Rm(=iSIDLfW$;LOL-hsoFB3x(h(wpcPruY<>%o5?+l6m|>bicu{F@{AY9rrJ)Jn zktqXuLC(b!;#K_S^UI>p1be{>Wv-h3nRlOcna_IVTHm-Z*Y{3Ne6h~SjrH1eYpNUQ zj5u#eG#o3?WFwMe0g8)Dzf|94bd)R3 z8!S1n-Ins$Duxgu(f_u3{(<#ScHq+!hkC-W#u9)DW`YA`SXmfX?FbQc4kJb6kkyB+ z_b50uu5@JNx%Kd;B9}G$!mj{h(J;R0LL>ezPT%{K;%MEPTOQPu``a2e0T6f%db?{a zY_Y6rp#gJMu7{&dPIcxg8OGDI?}j@gA&}A0i5v&m`c}>AOnyVbSkWX6;7{bD_^%MNM2T6)yp}z=E~X*R9PTk<-k(HwXp!>t)T;bxIv=}uOcnH%2)q@K72WLX zzY(uxfZAK`)eXbag{jx}9(@%u^r$R`@2{xG+2#mLUSn9J^UJt%nenyrRK-P4=KRGc zI%7Fx7EU>)AZ~y!%YCU(?(6-bnEp$f`Bqu31I~B2D|WjT82<;(3GBgGiCjSGwwh<342F#SzW zjB>hIjpr#orRJ1Smn$|Eoa8xdwb~FF{3+FCB^f15>NA}8v_i`NPOY}|Yaqyt`#wux zNtBdR7fBG0sosuYN$(-`;}+A8MWG1#%F|tRnK{c`Jn*;8bFcug5>6QzG5_dTEzG9C#`01UuS)Mrk)~^#rEbvCR^2x2_>-3n4xM}Eb$GKc4$4{m zV0eW6Wr^zLngr_9yXk)RFMw$Nq9%Dh@Ew}X9%UO-4PoU{+UD+@+(1FR_mbvvzLdQ{$r zqgq!+vBmmoyLoDF!@T9Wa;i;!E*&k$+N@UXK6`UKRZPH_G5Sd_ zy?%AZBVLEWS7Ih|pak(f{E%)MWnMTRi(gmxGR%OXFm37g1wECF5$En;QyFBO;ja~XB*jG4{l$@;EK z^YJO2&Bw?QP{QkSXXn0z%C&Plue)BO|8S`;3EU#KVR`%^2UlO!TMW$59)=SdM}1n0&!_BCwZ3Yi2`Dsd@c;+Z#vjVUyvk>SGA+%G9-2+Ceq3 zL6@(L&0X6khyOzU{Tmi_H({6SJNC#ZC6wsdDkm19={hD-MA%+|Up>y%X6DLaeZ7a> z-(aPDMXRav;P0-VQeskQwe4rxckyNc$nL2k|i3|TPg+D3As|8kg zs*+sg#7|AFD1ca+09<>q-)AbVVvF=@_?262OrQ49QMXgorG-`MZO+da+syQ1q_R`5g! zYr3UHnoC03yY+%2;IWugm#2-d44d^nFPe59eyVuPG6@CjP<1B@VX>RN&bou?;+ufL zbbJZ@4*1w=FM-IDyfPb43pjxOxLoG;8IRAddAR~S-%jdj))e=4Lho_uSx(ow^r0wm z^MAYPjsWS(JjKn8ul{r;<)};YYUp!wF<|u)wsN^XpS`vLTz8Gz8x?~nz>N0bYhJ`8 z-qPp^#hsZ{2n0}k#D1+WvBws99B;wL((nJzH1PF`exmf&<;`!I>!SjQM%1-X{B$YS zb<(z~rZndYaOdd&Sn1;Ox1kJR?Z)$Z)@}x@oG`=X- z8$W&i$MYVYZ}&>@$R4;i{C0l)_FUcL)HWI!0+bDomxW1}SI13mH2EY_JBLvvw&UD= zsk=bL`?bk@B`M0GjZbm3{N~|LKRoV%(*cU<7+|FktP=+)%`Xh!??1LQ z0vxE7t6ARb232p<-c&d%=INo}Fs z@a`~GXg_J!B9%&L+PnQ+Se1j>5F-=c7VX5JZi9%{y^$OG_sk0#qvchZJ@j4w*VYEW;? z$VaEj#|H^EetpaGAKG?bs4sOM|9&9cre4S=~fqXnK+VYrS-s4UO7uN^#1vRmr`el#}jSO zq4UQ_h;lz~r&&iR1x=>)hblCK-)pql@U%4lKC4myNJeK*j)?@jl@uBZleE8E|^{Cr<{ zaFW*lAPo=Vry1lQ%+tbv%ZP`A&3cO!c?qEJUCFbhmYN5z9Ij4R>zMi^z)!%VSC{7= zBiu9OCfga;7kT;s>0)p+BoA>}^#m-8M}RK;>PNBu^boO`{rCJ}?|FJU&z+Vdz>`&z zDFY*<6&y*cj8&5ummRvkoW0g*^1hr38kD@A>Z89SbDd9lP6uBCaRIqxo+RKf4PW(TQCAC~9pExO0RrI}m4@WN&a~U?rYUUl`s_!O zXT8Hg&`ZV&GLLP3vL=SPF^L?4CL{UZd-~gwK+yU{V*jmsZ1fwhHaerFV)nN&js=fWJ$F}&dk%HKL3%dDyJ|uVJN%kOd>eM&24OvSw zA<3V^fX=gOipRRbtIEG9=6&JrR;UJ$zmzn%blHC}@X~<^(BYXm#0T+Z=EB@W0!KU& z&}Eq5Tpu_9r!}J6-LqIjs0)9JBn)xaW*soAz7r1MTYh)EZfjS68bqLF^PU*R$Vqh$ zAoWvOyqPA*c|)501>Rf8{s+@!csPa#U}3KRI@91W$x%|j;767v>+`l|`Cf+5&Z_nk z9wETLD{-+Svn{&s47hy<-i!Q%L_clER6nFw1=TV*i1B&%+qExOT*JrU_A^;_sgWcb zyX~h7N0W9{vzJ?Ljmf;(k(E7S##fA`$;#pIHaSAVA(iJ84qa4HNhM$$#1cnwb?doW zQ_b3wCh6?!LKwZ>nn(N}FT@4`yM#3`i|Q@Rkn~2$VW}kpqT6B_Kl)`?FjpTyIStCu zt0JqQc50_fVBWZ?cR4(10^XAO$??E4GM~m}m=(#tg4=b^M)GWpDTB;PcQ}>CG}6d+ z>SI|(PARbzF~NW&Zkb{%E8Ji}2iJUsW0iD;fzlbuT}F5()sA}hHUZ^H*`QdVejReq zs<#bWy|?2LUM-L5!GGS<5@wCE2GxFJ9x!wdw5@l_P5s;lQzBvKY(m(abZbF-bgM6ew4mF7Ufu2|ng(t?{Vql6 zY7HDg6Ul)G0}vvCzO6Z5jJB+nf-d8R&B!YF!RCI%2^+)WV$kQRV!wqNC^Ms+=52LL;O%oNRv3ed{e8&0 zHHHt)Qn=PIS|^ga`Cu7*yKL2dU!RLb$X55Nvp4BeKPIQUtY8L&=GXeZhalh9muLf* z(H92k@LPF+Id(XZU7FS+86IFOH0xY3NR6a~;V$?KL(~w$E4a~4XFV77KRZ*t_f0zu zigSh6RJX`dtza>RxG|0+n9J6&OHPWPnx0`Xvwg|O}+ z;HuR=y@FVR`{=;6ML!I<4rX7CU*px~w$E`*uPn*R9Z;D&{$kgZY!r|Ou+Wt)otV!s z$t~;?&UZS6)wDXCX5MaVuN%_kW$ z_)+dP?6eYy9;CsiQBQR#b$RB++IG9aPr4nwGY!E){qi=;?qX$qA+A@qSL=+J9u7MA z6CPYHgmIycaK0R$c2YU02h>DXOE(Jwqt5-lIX<2!{E_X9)3Nwiy{yyq0C)kql-ea6 z2oHG#nArCnsSNBZ9!f%kJOun@AVrR zK|D`i>u3HCn}8s?F>xgr{OAOhf~Itz}gs69X7!G+jGZv>-UXI*H)a-?ZOR zA)6gnU~6`fPO$id^#S-USTTT(hzdnA&EV)C8@%YpFZUV)Cnz6N6?Z^f8KXvIfZ{mH z{;@-Ob?ehYr=vD3AA)$*i$O4YVJ=S|&V29qA zif?Xb2&mCG%g}vn)l#e@v@|(}JTzP}KjTc{z~8Z60WI6WNgXDNM)#L1IpjkIX!POONkO1hGY7d*?#7M%$fQWXuSVh!L&iuH8_MN)04nvgi5uOFvu?j)WpZabUG=_888EL1{~T_@ttmgLU|tiaRwh8 zEV_PwB|)j}*p>7EUu50%^nnuhDcG5)3hqGg2`8nQ;L zic#Xe@CQ%dsy(B_g0@mzJb!qOuYarKlWcWpB23;f#$Ij#PnX0An%+|LaUSOZ&iwn>P=9}!K(hP2&ny(6{`HoL%i56=XWZN}*xb-JON2)Ojy20 zFtxmo|M7qR7l7Gb$9ZTH6E6-)8$)#PZrkZc+5jWVNVlVXy4g;P_Pax9_1Or=c?9&S zxu^?{Xsb6BNmBXW{%m2v~mt7~9q!F|rVp&dru83St% zu9cylVGl;oi(CQAYPIE@yEKd}S;nTrHj@rH^cpi*BDj5({?mA;UPu`43U6VXSo=Mj zWm*t{5SJXiL=Am&2Mvt&IK?bsjRti`HLIj&&uN=W-6FdQ-4b{UDZBB>hk^H)L~z%X zZ$sHrzK~ft`Oph+`FRg98U!5~(M=Gn-aTtw{Z{2pEioKyf3cUg7<)yZo)ilkMSvYu z30#~u+6M_Mqm!s2-Ic_yjfkCmoftC;{9UWrFytG%A`SNfB$%zaXA&g-%U%Ey*P`M2 z#{TvYc&l+OE4r9{}dN=LzBiB$|Z7gyFr4G?L zbO(an@9^K?AcS>zx86)H%s|nuk=`6BdXp?N@YGsLsMnLuIHN zO|fjhxr#ZAW^_NR&WY2CDx{vLh;cjX;p?x4=IJHTWa%wj5KQuWSX?9%q%t9-`GPoynD-KOAPc~|*B-m& zXPo~b$w+SdIMQJHq@H3m+5WQAdN+O#EHPnfc!Ff2lpwQhYhilWe5nK#;LKH4VUQeP zIwqg@EPEW|>K9AR?N4u-%&f{b5H(4;Oo%t+OFOmgJKWh|4NV{U%9q;?TZ&q?6>G+v z@42+$bRihvP=zR;S93q$`bavN`xF(=tu@2MD$_)}e#o$qAI7?S+ryu5X_bPMGe5Z9 zB9(uuqvh26C)@FzGL*D*g9l6)d8D?)P zX*NC<^HdqN`rkub(W|;!NStak<_~ga$uyUb@b|0Dv?XEJwMsNPIWW|nhAa|GuGks* z(l_r0KjJ3c3>a*j|8dRMcVR5dWU3%xz_3e=>UTo(byl|VX>5sx$tOrtPh>%u6UJB) z7H32P?g~0(ko95At2~Zm@rJI%8RM~ck#ylKq@1>CJnO`~rJVf3z~k%Y(@xUep0ZYV zdQX5YTFwSX2kppqJAj*ZJ82p(Vh9M(tqq>3WGDUFtYLnE60N~vSxpDR8;ee2>ONfx z&~nZMTyqi#tgl9%j1Juk4>3y)Cvb|Zkue+}2u!h`a-!?uG#hn;v3w?56IJtqAwZAJ zxH?vGy+V&8wv8pd=H2IK1lH%ltzQ0=hK^lbBF1!+) zKcA-hR1R`5OYlR08Ae%}AxygEt0I>8j7vfeW+N;}HETbB>+s|9D<76|VAYCze z%j!rG?N{XB!e>DqBP3*E81LEcY(v$dC(K)e4EKHc`}g=$q1n7~i?{F?4D*TQ!@6lf zgsBqktpY#vOyfCTpvG)pc?g18&RW{Q3J1}-{?G;JATxMgWN3H>`O`jpf9*$8#$?Z^ zEvcmNdzq&Jj3dg)?_7S3U*T+Q!{?N+11%|t!1W0C0FM1BN&tCPXP+x7eVMr}W?K~at zc6-S+?WeaS95)MsR%ZPF{L&CP@UNVNLn67{rBNl7w-IHKte0R4kds z##CjEJG|PEn75rz0_sRi(HaPE4#YlduAV<}jC`;kk#GNUpDYD2wbM+^n1o>1+BcOT zd66?Kh8-Y`CEBhcHWX|k8#a!JW~v7>`iU9qfS0u!cSmV{ttU^{TF4|_H8SIl(dw@(5N7(bc3{{ zgdklaT|f(X*x-Hk|hcMUbvyST6Gyq^2=Jm0_Id4BPu4$K^L%sP%)Yu~r; zw%F)kp#h;obnBc0-zM5FA};#cBgHXDoL3&r_2KRHi}JAzgkW^ux8q>!_9F(D9`V^8 zmF*@8^9V>+UD;ueCCgL`hE?}-$08?NheV`#u#kJC24^~w3etqVTcfi3e7fk5vbryf zJ(jBYf2Kq$fkKg8rspr-;^Q|bqUAcR%!Hlmd6!$x^*xW7_YhQ`h@ak1i0s4Z*@d3k zyDvWHKWknLEZt#Z9w|5weYB8X+7^>{gaRqpIUdDW%s{O*;8oiXnpn73>c&<%S^(p& zw3wKML(8#hHJI34?P?qPrn=>)ah|+46k=gpN4H(#q890hwm+s?wQpE?siQrS&++e0 zIpCfSrtx@AZL!XfSdm2v)CEs|DU;JKZN@prLj541U-phVe6ePve8EiWkNm3vY z&74nyrY!W^VD%W(a;!AZePYZuaWZB4#OSx2$JBr_Uv)(D(V}-M*MP`lPG)%2=tb11 zo3mc&)W1y|Uhc6%4M9V@44-hs86EXE71Uc|(=I!X#Nud6`aaWsn!#&lE1QvKto1>~ z%!xu-p+{g;Gpzogjh4Yb=1FGa`Z<3K|6j0%*OffQyx7K(;E&{YUHG1R=+B;_;&htH z)9z)Up1s=5<#=0&vV-PzF#WMsiK8)g*d}gXT|9MIe*O6K=xFF8R{8C_gFoc``ADFU$DJ$!A}-AF zXXB}f2|FTv8tKDxs=ww3e?dYQUVuOHAVQk_>p%W;o@#}Jye$dSa|8c;ofTwosxR~& z8~XhD$PR)R>8-h!M}Q^a#*e@K-i|QBBzn!pS=+dkY&k$#CVpF(x(+>7dGY5lQAH4Y zU2l!g>Z!$fhe7ak8uhY%zN>`>c>hDz5csS>SC8d|!!cc+LPH%sL-zFU>7GKa!++CJ zz$$VF2X)rlt@gwfFSVEUe&9(e^QP+ITqB{?8yMI58KWg@ClAp+vpo z&lCI!h3h^l&l|))U-fDF9rv}-)NTIfBN6GNr2oFW{u(0KUI@~F;rfJ`-1i50g$Zn^7(M@Y}@}n3{mciQ}%Z)Oa+aI@KXCHSpZ;<}tJO z%;(3?qEvprpC>%0Q4JxRDCK>Yb3LEJ@mBS(;>*8$2)bt|E4k@1Z{FO=N=7`c4kgT{ z%5ni*g_QWY<}RHHLhHMi)0I|MpvBZ8cVZ7fvNz^du7M$ zIwD1X*EVB+Aqc{JcacObn|6FvpgEE~UOf%dT?QbH7OZ1iqHnA~YZ{a9K*$WV2`VyQ zYR3)!x(2|3dijsBofu{LZu=12-pxNc`a*NKUCP3hHp}9dpuefb2;lCq*Pzcj7Io@0 zlrDS<*of+c8c?nMd~hG3Pc~{Ma4kh0Q%L1jW<)NPD`lU zBj03F(P{utGz9{n7I^e>_BN#9UBy;#*rsZ|i$g6iJM7HZ4VD8|ZH1^A1mkufrR+niD#Vo+ZNsJ7b?`{rBxir~t1EPFL~XYQVfV z?!{gHk&vjc@z7<2;7&i0#G>b&;sM`G6*SHMuncmz<{dEN4yA#%HYJ(8(cX{Mt6O7v z7Qhk~^qSobE7SN6@&nk=qsw#e2;S8g<$Sk`BVI=LM~UlwiFg9W@URzwLn%@yTyV7< zIWk~*NgXTHnb!dUBUWEy(8IMDUV04(wh&p1>BLhJ?Va}xZqDalVD6xh2)e0d=Et-z z@;2bXxTzx6y)kh_5H|qd&AN9hQbn5$94qs$pPYR=v<_>76?$Ix0z=7WFQxpR zOG&xK^l!lY2pHs2qI!W|FTGVjU0}*$Ye0?~y9?|Cv+IyBva>+udn|!-UHHwx{o=L} z(LGTEDca!%_5rR|a4vEp+O^#dU*AP1-+)~2G9Ur(d&)c)D=pO`ueUV zk|tdNx_-~j#`PE;1)@+T+#aXhI&n}r-UXblSx%8JN7HT$?|6irFy6g`E;=J@-#@>e zs)3AYKLg#Enx`tf0Hj&-9rx!>LEnJ=TA&)x5Ivyxxt2+pct8aDQ7V+ z7@6u0M3#kL9H?+PXEnR52|h?p1-4nyWAM#V>{_x%DzeUFOb`fx>FGEsF) zxB~j%(6t9!`^x@-jGR^7VTU~{;nrm<wBMS7sz5jPE$t5~o4QJ3w& zj)x3$Oz%!g163K$mxD*i*!45(6QIZINx6j`&e#1oqo-BBs<9B#1g##JwE;()y%dq!H|A_i9 zIvNq-t8Qy+EaLF9_0-z7Hg_mc#%+F&mwcN^bi|PvAZ0mSa~%{FbfOT(f&J~!_dJ79 zKN7@U4Rei30`bpgS819f=8rFXb#9itBP*N_V|u-O9!`PLW6XF`2MD`s&XbZKcu7&T z=4;KYjGczn&wIH{sc8hL2XK1JUblroA+9i12J%o!DgwojD!#4nSt6|e1>l2^a0S7u zo^Hc?_f8nXe|U~Z{&Pv9=AC8Gahh!M;ZXFnWo^GY6jYx&N$oSY5)Yh6*$T4z8o!) zn~@05qeL==5`%otp_#6X#56rI33D#nFFgBh197PNY~!jtO&U7=G)rs)Y4#WZJOI3C z0gS+IIanqbl$gtq`1DS&DA8F}7z&kaSN>ePfib`AP;Aeh7)U4Iukq?9z`()Xmo_}L zqzw18uJlnf3YoXf-b{7J(K`XWPy?wKSZ@{s#&eTa%NC`+9#)+Eue|r&>PmMUWye7{ z;D9Tm7nKRo`rNtA9hk48PyN3|FtV@QFvc9Yc`RqDV9%=y3qTEwm+?e)um}&b5IBK`P~Z}} z)xZ}k`GL>omV%?b-O5c+{)+Fok3IB*&eEXl!7oMfkD_lQza_dEqgqI!2*<@9jiP>& zwg`v${qHo{zpe^?gzoW&d)g_yDqmKEOw!ydKq}3ZTye?dAsZe63dNq;a!?T8Sd?pp zKPK5$TD3`N&f6n<2@m$L%Ux`oK0 zM{X}*MO$2qT@cAj(x`62e_<^*y>cGv>i;b*=ydCkNn+CgPS+?3*%AU}Q>#Qvt2nk5 zV=3S1gu%5$gL9XJWMI8N1_@lAv?w$n(Ylb-f#TPB3VYA7=*rb^Qbjtkhq-{Gy(z-> zC70n=`>DM;{SGbgli9EDk+=Ex&m1YQ0+qCEH)&G?pw3Zr4m9FdflqNM8C7ax)@ji? z9QW9%56O~(KDOiGy!NQX5PpKlK=lLA@(t!OM(Ly3Z%U_c?P~#J;^5(_VH_#Fg~6du zpB7C^Wxyi(o|2&We(_kbfyC_vqKD4uEZgz&0QvO8ZF7+=`AQ8rFq^s^X|fSd(A{s{#>M}kWB300ETB)wzb!X2yHXX~zY5%Py5`;$WxF|aIj{RT_MFg}mzOT+WaE#I zMaUKbNqmTwvr*TIAMhi5BBxiLX=khHFypK9!y6KNlsuqKsit^7Q6F>i$NbOr^C!gF zRH`H&7iazT6vsK!Bo3>Rj?gEUC6eT4gGw@S;?RvDO62mVf0!PH?(#d8@e-*ycD9Yg zf-dA9Tq&2ossi5pjovMx0`&s?6uzm>qn-o9H{>n2jm*s9vI^{f90^i{&`~EfK~vac z!)P4qZgK^sh)bI3KVa@tJ}h@TMYoBP8YF` z^~5k)031(dz9A0+mSjd?1Ck&9JcyAuAyx?Yv9nAM@p`2irU0=JD9Yb3m2PQW(N zdbN_Q2#iFU#rl4L!R_4|FWjGBCME3-vF16Z~W9Cx0r!r0fb*BpRL$FVvrxgwTv z9`LVsL7fin3_vY*mptyWNaC<40c(yH3oEczc7br+dfZ;0{RR?V5-#iXaOR!mpk3hc zyQ6PAIoHpRfqLl z@O8o4&Gp9vf*1T2zQKMYP>)>|^&BlZ`9S*u`XQvd~sO_{wu9IE^O{DSDfs1%_bee(I_Y=~xz#nvK z^JXS>bRVS!Ovs-0?o`j-_H6}QVWrK3i4>SGB}|`z8qVb16hPEdbgqxb3vcj(tFWE) z)S7&e3h~*nTv(^3Wp(=*C0yWQ1@A7>;yKuV3;LLr6|9a`={$BFvaVaXIndnh6#g8V zQlt1M1KA2C(9Z-fwg~IQ!^!JKP1hb0qJtXHG1;k6xt?0Dk&21i0R1+y5&LwiN`2yP z?d{E_mQ%5yh0K)Yb;`z}4c2m2*WWUTq1h$;oD>1d2|2;*jRcuyR~e* zhTbh47Sxujp0ti>W$J+Nnr+qMM^-gh^_FU#(;-aI7mrT12xQsX2DsNrTdUDg>Hh~) zzYC`;mw@qB#L&v?8F6RAM* zqJ=WE8m`Sopav-W@O!jyf$7(7NiS0=@u%4WIl z(e804erICzOZsnMmNi(ogKM-sniGCU^v_|c^^0a&PVEDv^LiO_YaI}68z|(v0Ti+^ z1L17H`XQE?WL&l($li&9@H^^%x-V7)SIyc+Oc$Z_u4!5s;C_2C=;=1EqNZh}oMYj* zMJ!nG1-BY?6O=ykSX1>NRPeh6F+;#sYqNikYwT}CiTD8#fJCy_={E+tP-M=$a&SLP&`CCv6Sy#R1($mY z@(EidQ6u~YbUkn6K%mRr%;iq_Qms|*SvVn*< zb00p(lwz1u5w3X$gOm36mLk9pQDwpxvup8HpSQa5sybXy~$`f&vWmhg)*P`mRGjygu;UmG$UuVK>hWv2{}eNeB?dFdlzz7g`zGPszGo$}8btM=Chs z(T=Z1wB!d}_O#ebrEBB6kmWsKKY@ zm!x>_Db|wl)*dqB0= zjl zit6y@c4rp(lh*4oJQXX}dm#Mwobl=N*VyP2_+qag zn^T}n2q{oz!sbLMU<4u}E)ADStBjnS`xYE}rHtLA?#}yjysLelaw&$ z$IHvT-+oj!PB7%jkTDzO}~*?P}*Kn^2hHUQsLN&t7fRoh?#}5)_2uXjY4S1}E{0YX0m(Vd%i3O$UixWFdNYKX5tc(RTqv zX9O>x^nOaBVPH~M0q*#2h(EHVzY;IfO)BcmaBE8}aLIRQM)b4Dq$4UJI5o!`Ua_ZL z$nt=oqV0iR7#IITErc=vj^sQz6{O%59n^g*{{sn{>}R4%!HuO}H3#Y;qtEm5qkQP? zZX&}yXi6EdD{TiOQ84?0SMG^NsoNFo1GjkXDw-$EfE-J3=OEdzNv839xIjoZRz%ncAcexJe z2G>7%jndqP=~|SJsg{yn&YI?=)TCOJ_Hz*9x$xVrbnyw)(6PKer(9kUiOR-mt~->H z)r@RwLip@#<2F&(3`|L!eO&NR%c>iH#zGdKZ@yi_Lq2ov3aRY(*{7@;uS*(6Px?8g z8BtTo&U{{k9DX22jOdXj@8;*O1ML!95?t4ySI!+e%24X6KEFKAzTBpFnGETa@!n3$ z!<;0xN6+Zjv&wr)tmF`ijJ%npxLoaSH!kh4@pelEt^My$f6(r(pb=A`UWl#h3k#9Z zB=CdI*mT+t6b>Y6#U`dk6GIUo9kkYjO;H$pk!nsOCN8v#*{xxVRu!AZ&PJzpVZw}VabD;gXh=hr}JPQqo$$dJiFX|{z$vs8=vE7Anne41bg6LO2 z#mh;sMWVmG-2q;s3&{m>5{sA$FneO;-Fd|;M%bO&PrZzt9Kah<)OYu{{Y^~o5Txm7_P25Oj5REj2sRZs*bg43%{`U zB``LBol+;Dd;hUuZKK{Y+O`^LQ^x^+l(CoVg#1mgpuS*eY(p{>ZaV3Cl(F*3LtXY2{ThTR3bCR_N*`l& z?rM#5)fSeJ@|qx_8~fJElJ+gG($Q;zeg}3+$Nu=D`l({w=7wQq_uSfc0!_o(A1@5> z7>I%|V(gm&Eqj&lx4Zk3mKCLVtaMk4q6qyB#LVszZZz}1XY)9! zbuV%#4%mcrt#yvL3^ORZw=z`JgfSjSni3-F4rrjJDtMe7ZeZeOS_q`-0 zk=iwS53S09{yCDs0bUtu;G$Lz{QVYIMKbk+{z=pi2W>$b_!U$r%)*4Ei`3__`Cmx* zs`mL28@X%fY?RhCT2{_32$R!jI-G1?MB=+y8?LQ)QHRNz{j67&n?WLoLbfA+Y2DNm z;+U_oge>Qcv5zISfOO+@-HD&V(Ef&xoBQaQdW&GEboNi%CxV@L5|&(2nE0t(>PQy| zLSB)+K5|GlsR%gdE?`e9Tr57*PTqvA&cpTZ>(lDob^~bw7T`T{M~+}kQk*5TaDR`c zkw40C$9$(?bB72xb>EODqV`A z0&gyVc&@t==q^}w{&Ke;8&g}=Rp3w1B#Kz@pCjSSM`E-`4Ts|=OLoQ5&YvIbR#PzA z^BJnKh)5uD?+8aUMKNif3lkVpb=mynuj%Aa$4)0%3*#HoYtG0T^$B=1U^IrRL>NhZ z#AvN7Ptrk!ijM-7uzya#LbDH#6XTpeG#wW|SV;OwF>Rg6+u;WKRw{NJw99X_H>!}H zb&pPM=claW2!LHT9Dch>mE0p<^dTGKXv~aK z5xr+M(DjBhh&OY+6dhGLGulY0{rfa68rMopBnHZf0MUQhT3M33!LDni-o=UuXq=#iOxi zo8RGf=gdCfUPJI)iz(DsJGX7Q6K=7&#(vKxwP!~ zC%Gr%i`toz7ZyPm-+TGeypWF=22kg9Q+I5s8ocWXU5l%s)032lylh?}iX`S=xMXef z8u?FWsNGM_2gDLEjDvCJXsZM$Yj^H($IMig;1wws%Iy(1sz1m+@ZrCB{uVm{g1s^7 zEBA?(s|Zgt*F-igvY_hdbN+bkR{nadzHy8??D|jk?r6 z@=6y@$7|cPJ2*G6cjFQl@7N-IV5Bm!MK|IvpTd=mG8dypXyp`YK=cTWp6cA{x5>_~ z28&YiiKE=D9J~oxFv3T~UqV;lbhCKUS%PXqvoV<_ps#uQWD$|(VKUjrcB=z&e@$M} zs0B%i9-KUS$tFA>OzQ3j0o0N{8NR(-w06R(r(8=sdx!Zq{lgNOhD_YP307gB*^7L- ziqIn#KOLBA6PbjGAkaS3-EsdyBYD05>b9Su{@h`u8-FHK zc_nbaA~+(w&odX|rM&V9Nsa*Hiy=7;Vsk)o!aAH4pRb(iYX&v9(kMih7NyT z+gu)Y3@QbZaK|HdRN|2|YF=~IuGAVQtRqrm81;E{SBJi0%<-GkEvUpjO-P=A2vZV# zfjBYv^%s}glle7mTO1l)Hsq`yLRGxDP1>qRzrAJIMAo}goWp&qYT+j)2g4>CJ|;WY zV3Q)_>a1y8LpSNZiH(Sq2nbgsF*>?EiWRgXf?I#vhlY5snNU3P(?}YQ3Y+f^LP`Bv z3h8rZqOYP(I7ONp)IL$jC6k}M1y2r@(;D?!`8o6y4|8qG?N!i@lgC=3iSc#ZN-6srD%Y?i z=_MMsH9_&WQ{PZ zK(kt-UVEDu7$4&R@m%|@pY1zWS5&|BV$i!wCqe+mkV`O))Q(VXwrpiYp9y~1;7U~)7 zVk$nzovdb6fAqz-LlGm$Z-tRm<`-epub}(m;aCDRw0o>B@TFv;dFb19R??`FExQqf zSD4Zyqy}8OALNCe5)sC1jDIfA`VeNs;!md1qU%>?>3g!=>ZEaPFD7nn)-I*+PLSj6 zIx`&hQ(zstjfI4{6cX&RW(u26k{!e#on>uc`+4!ckwN6Nm)gwq!tS}?LQTMx?} zfo4~o_2;Mg_^BN~Jy*Akwj0Mn&D;X{ zGx$FJaw2M-;`(*kUS#kP$^07M6zr!tpCj4eFN_Q8LhZqXvAk)F)g)Nbu}Aa+Qlm+X z>mla*aPQloh2E0p28Uxw;4@f7l;MKkaJ zC~h}jo^#y=SFhG1qpaKSYPXolGow1C@@JubVav=rSv1*MM>tcKaPfoyQs{VPg-^&& zZ33Nw!%;kv-1oDVRj0`Eq-GKhX?X4(j`j1<(`aTtui`P(G&$lnDLnLuwZ;LNm`ap3 zD-E%d?@j(wilS4+EWrp?5&_-uW<=wJtg)p+LdNZAD}AV$T*j8NTMtcJi6Z75c+>Mu zynI-t5A-o7O@aj6Kd3Onzk5P7cL%sF79Iz^CPBZqB@R&}2tVQ2KFTpClW-e%zm_f0 zeDQm{u)MJ0G=;ACsp%Qv8i83>{>>McZw2e{uqxEwOXScBUfv8#g%H0v^_M2}7f@({ zg0QZKP&)h8UTdU+tT;uN6$5t|2HM`w4&RLBUr2Bh4+pplEDe&*c?J%j8-A)vH&9!v z{l-a0`Aqxt;270sKX7l*^YpmRV8bcFfa|2`8QkV=(6Hb_(*xU~j|Gdw?h@QVm0}Y! z(F~ZK1v$$QJJH>P7{l@}0Z{h@iO#3ALL0v|E&IKaRkVct{qEJZJk+*H{yG=ndSmTx z#WlfSRBBOPDF@YgK^g)1YSH;q-z0a=xIT40b<8Jj?HFFp{j0HQ^1ai(3R_O-25*l{ z5v1zYtBz_f)wi!sOb3BXZc0CFZ0c2BN?|%(%Gc4Z26G@TZBGpbt5-oa7qa z1&&Hdr!89*v4>nPD@0UonvsU1RK8?1VgdT}LqyxPSKfFDMU)*B+@bBHaI=8XFanPl zBiUZywoiNuY0b`I%!VI4tg zAgjxo{LtBnE(UMZ#Kx_8HKU}SQ}ypz^a$sjX-<5-=;USc0bSVd=&9dO$(($uxR@!Y zQM`@f4#uChthA_bUBP@!;xL^IfgM1WzhbQn3HFu7iRLl{bn`majLEnZ^DPH51 zNpFN%OhMC}F`UgnR2a@-;LT#s+VH08$;2nRcP}rl)&2yH!33o(B1|w3g()$Ic+zv93OFbjooRPd|N)(Gne>N7~_ehfOhUC{* zhW$E}dp`NmgkHpdJHIXA9NNi87V9YXm{^eUMD07pf$i(`(OwwNnR2#Vx7PA9n(` zxWeE9s;;n6#hrSFRPdHk@%gxblJAP-tBY4(R%4EeMILbfX7Q3XN7|p*nb* z5E*pLGKJm58XbhHrRV5`4P&N%8tLIQ=)#uyns1M$5UUAVO}5r;-}jVg%T#4{Yt)Sh z`H(%PD$IiOziJ5oVxBp)I&0%|A@qI_@#}+f%d?4h0Xe(S$%t=fN@&oSxo>%K$=3gF^d_vS5wgj_8Ve;E!%&*!KB*WSBJ&V)BEb|0U zX;=>I&3*YMyRkPG`W2;#7AfuflN04#sD<&>xk@wy39^_=GGCyU&^$>)Ag(P&Va9p9 zMpEvHm|z%S@G|Y>@R_(oi{9mKU{poEK6(Nonz!wJvy_lS9|&Amu4nRr0i}c&XCii& zhU9lhJL^q^#cr+-eKU@GrBdHq1agUCr4md2d+ckLO5xP%9ILa?2OK+UTF`!ZyH&MD zd#QU;4f#jt6q8>;-{FBoO^A@~&}FpAkmF;jklW*%c#%jk%&?{iK!%!K7Bx zuuFY+j<}JdyTHI5LDHgA{bZX{c41eGZJ&SAv^fANhs<$`6?tq8@$^73;AUczA|R{_ zjWpDYNn_|^)DgkVuT{JQ9JY;VqU&#lN4-p_2mOVg=30_%N?GT^%bHfgWr@`k?B2gg z?q-L)hSXziVu^hq88rV^{c)9~l}`HJj$t${(OCEU2=?6(bfRDD%!YH_k*O1s$03C1 z`3LQlZKLNe=*Fght97BLbz*YWNMcT~?eo35#&P#^T(Dv^yq8FV{@VP0BFj8fx7-xd zM;B`BAno;jF#t34$JIBahgVDJ#hg)g`($S5#YwO{Iapt2j26prPxi1Kz2CVcl+~$) zv+huKob1}pH_qM1Pk?{fb%W{l=Zn*s1V)Wvd4be7h2zSNQXMW}_X-nd&T;dHBuh_A zP9^G0#N;L>L>~)0(Vhz!r{j;ckK*MjdI5G0GpS%Ur+3EiT;}iKfq1mqU#!LrvQe3L~cXOvT%?~lx^vXSDj(eGQ12=`4@;PW_uEQ&= zKX`Sr)4Ok|ky@zaC(1@9&6#6yoz9BA9`d(l*@DLG>k=^ z)o|0=v=OX`-N#nrjUb#>0!35RF*xaKM3fL}3fyE5(|t-`40}{a`q*H=RFdgr-6Q%y zca=$%HdL5*SkT9So-#SAI;zb&)w=G`pX6BPm{dSwUmb~DH%M@o-P^%Cz9xi}jlg{) zez=pl^J~au7#C5R>p4sKT9>;;bGQ37wVYIDwiSb$H|laTo-7w#V`REgI=y z*a@n>Q#>2F8!7edPUTc~`*YAVOB>B?AKRX`F8uy_$OgtMz6jQZ9LhVx5GkVlT zg_vJ%6?VdFzNu$AHpb)lsys*QDp5vE4n{UU^@)CF!>8hevvdb)NS^u2*P%5&nhhH@Vh=$LDcdEC8$tHtPsXA%`-He|aW@?y21 zX&%Ge2@z&+1Jf;^&<4QQ>sM zyEcPiy#@iSiKw%BNe3Ti_=_W_jH3@jAa*FD37$^Pn9bc>@6m_GnipQEOj0ceL)a9IM^d1T7!PaR+rG7H=o#Vl+f4OiOv5Pogi&Qeu@C zn^!--<8?uQ@!@v3n^VVWDTX12z#2IeK1_E(dc2UZL*ItMGDJU(MY0NmKt;3!UZ=fD zGHO=3K5w-!44FAdb4cmyi}yrBV8%m)zs%T{ktG^OQe^F|x>V|cHx$k#%+Gzus+73V zcUl_S?XsRbR2y?iYMlEZ&tlj)$0}6Y3dKR>n1kRwF3;j_7Eh-tD&}puIWZxbh_*TT zD$6^w2J_H4eU(!edvuKHHJ5reXK6JT>ULvVos*#XQr~V@gZGfyh7nYnbg&Z~l0FC! zzw^X2T%&C>nmNXujc?ff_W?vYvGTn$Dt|&E5$&2(Irni;E^-+|Ds47)x3{5YIg4*w6Vb(W&b79_;q~(Bq^3OU*BU3_@94j{Lka%Pt2>;{^{X0 zdPHdvPQiX-VwwN^%731rNQhDaw+88BO|LHce?RNpk6fU7MS&MCefLi<%&348<7?^l zWshy*e;v%fpIO&K1l03Ed4m;iWU+U5a#CJeGTT%4&6;7nktV-~*+4qg*CMMkm%!od!m}32 zki85&g8#B9{~#38}49$j9(_i5&Xjt@D@s=03$@etJ?k_2cHWB z43QIo>~HP=KX3WbUbMDLM)W4T(wD`wO%((7A674Qg``!ad2|;(y}fQ!_@_fsPset8 zM`M3fg%PjGMOHU^PQ{I_Lr~y1^@Dhe4z)`?CHeUs9!G>GBqD+{(bDj zwtWZ0XP&eYNJ}(A{mKT{WO8KoiQ)%{F2vvm&RNHO7Nu}~)A*A(SvArMZTRkqd=B2t zk>rz;Un_MsYS|V4^f~nHqrj$pEnUiAR1SX*{+32oC0=v3DP?-8Y$fj~@RAjS=q1IA z@~(6~=J7c}O7~kx_%{bN4Vsjyv*(*LXZeYk@b(S+-c~S-Z6B7-_v|%!Js5>;4h#_d z!>HXtd32!h?lt?;t5SF?LF45rwz5htkT zLU+QZ?N+bemBV*aO?uutO;^2_(6Z=52Tn&I^mTp~9ka069+r6TwI-!3b4CAmhy`fZ zFJGv_@CMgmn%8q{Hge7p$xB)I?dQZWrhbYCk$pP?XXd(|!fSU!)MFcrB?p-{ zt-g9!0pHd3P{q)G=kRMx+zKxp0Do|M;8@KDU7@dlKH%~M8`P!45(nMr0F(Z;vUW$& z6>$lWVDezMKJjq+29;LC4ZzO5nLqE~9KrHDZ+)rP;;Y)ewv}4p4ydat*zCRsC;BGH z7#-UKylN1C5P*YnVB2PS7f}3NOH|zH76Lny55WAG-6%nx(tpJXTBrr!turUS~^5#1yZg+ zQV)WZH?e8$GqpF{I=5dyV(m*3!PEC3R5UxqzU%~Gq0#upt5tgdlsa}QO`A+r0l-Vg zwst(O*#kxa+XQIGb~GO4+8QuK_S=e)Z;6F@T=eQ>gJjs*q}s2vTuz|BgpCz?I1p$m zF>}_=8!BE_&bgRr*3N2?7+wNwrTa*3aLZTXtnDEd*g(+ZYu%;i<@9{XPOS&`C`dIV z-EBK4K-^LnsMF;-kprl<#n;_RX`hC&@oJ7v3O)G&;&*GNtiNYvb_u#*vUfX>uS-i9 zj~|mc&J1&qy8&3Uc1ifSZ9duUsCyIW`+F*ynUXJiQov5+la*Hhz8lyzt)z#K?c>(C z{jwcEhn}yxUo2O;KJEFQwI{AZGPU(4lLcWO#7g<= zw^RzHt(NC(Slg~7?Xi5d{N+)4D#0mW5S0VfQBEjRJB;fnOWbqgz3P4eR(c?DnM!0xH< z^XDwDeyf&2&fK z?HcWMkXr1O8V-KL^Q@c(rM!ij$LcHuXWF*+Mb-r45gm*BQ8&FEL`~b-h=YgVDWb?v zNkKsDSXd&Tp0IXWooMhY$Fmz00T^@^x|ccsq40)ma4K~C*u}YVTCp^l!$*Vv_$oWe zvL~1xSaHSF6m$v+{ha}eA=es592YKA){;3H&Nh|Ni8XU8bVFvO;m&!XOb0*IYQ%i> z#Q{d>@MZAJ_>|dPu-V|o3V`)~;q!Hto~w#`o|v62)sR6BNO|l(XxR0?8Wll*5px*A zbF%N^zMEOKPBPYeNp@TTfQhf**X-k}l>+1>=Y|2%HlM6cZr8`U33guIe8ND^{h?cx zSIHP?flaeQIeOg3ioYM@pKPC9iEpV3xh#M#=Y+;&CDb64BQr&H*Y)>agNeh(xL}W1 zQ-NTY{>z^Qu_&C25GwjE&KzJd&9 z;p)=f$Go3$s@kTj@8vF=^IQnzFJ{##4&3BWt**AgjVU3I<6=CGO?cSTEqd^d8*;F> zH{@4lN;O5dBDm-XBTMR}aOMNI`BB2g^LF#ztl2e3JROUNuP+a0_Zrsw539r#E5WDm z#UaL2O@goav|=l74tniqnU93E3s)cnakBUiy^g#M{APYRN4R_F%ylgED1ztBST*_& z9cr?Sb*3a()n8;(<)$7B^O$5z3GqC7|WSONX1!MAuJR08k(FN-#3 zvySrMh7UQikn$Qsh3vlSw(~9nwodv1la;PWt-K&r8G*~m!R%5`8>q`FgHGZcP;{=R zw^U(ruSE0(9DXo zoLb@&0GMdb&x)z%cewE;0BieeA<13)9%r?%;&69HEtk!2lT8!5uDfPBNgVTT$FM~# zPp;#fQ0j0#wrfDrOYJ)kx{d(Bc0AY%EG7{ePs9A!WRU+%790|IS?alb*XHzb^k z@2j%zCkF^oJH5SqRp|VNDW?x?XjkZki(FfYGE+Mc_&(sl)1r$a7oSNz=yKvMZ_ z6=VqVRXvZQ(_{;s+8NkDn3md^8$ha~5&}n&1+i?$rj{E47eQo$SRMyH-TFxUEE4m8 zfnpb&qL*LOAqz%a4B@B712rZ!a+3`WI|}RNdz(yr-H;paF22Ncvyj4BGv7)Or>70e z5a3jluU!keYTIMvH$vAHZlE1Lhb`6}{7AEM-`2dCYZ9JKGCuioWNOp9RTupL;hmfM zg$aE!qk_cciZX&72zZ+aP;Xo>BYE~!gkUD>K&lqhsIvIG`*Q?Na{1Qv(x@g6`OLAk zig>F5VrEpWYX`GhY}#U|R_y2u`#OtGbm9K{orw`QRqKi-ac3EKeGne~8ni|AX{+kg z>(U9j$?Sc2OE^uiTnCy7UMWg?XfR)HRtnhZm7+Jc#!!FZ)4&DAB_PXfe}|*c=+B9I zF4bK8w5a*81GVms_SoqtA>0wH7COHjiEoJ6C9n6!((ZN*8BUnwtlhM78BH1ix7IIn zCk2kHS`V+kFaLT-Mx%LW35f3uz%nRjmtXI*+UaPWV#09ud7<^TLFzoZuHkZae<1iZ z1U?i{QtvPyK$fXitBRfiRh=E>fQT^Nk@?q2(G&zY-M3(xli--J@UdYMwL9{#@i5e? zuY&5FMFKu&UfMrGGI!w5bg}ZFi|aD8foz$fu!?P))zhHa3&Tdq_o@RG?f4#b!jNfQ z)GOXfmZSc{0MBLmvY!LU0ta3@IhngU*IR1N8RH4thLq>2uKV543HOGVuwV?7+1&BO z(RsrmTle;$fo=0p5YnZoIi#>=+Mgp6Q{FPGgiFo;fIO0XN|Wh4)@aZIuB7F z95H)fS01=`P;f+tlmss!aJ;~MTXnE)=sg=;)wjet>7B#%!W=ENoo~evOmhYu4%#8F z+^J`AM}I97qE|Rv{d$E(U>Ty7-)>yBKZ+)Azy?XxWYGZf+y_0Dje2ba#dZhW0LUJ7 z&)V<3Dk1}0T$pzR&qUC|e}qHfohD8A{!{F7ymD7n+}LKt2CyW|&w^dP7ba$9p&Kpf zUMY=-fr2%-heft)$(T$1ud0_cCpg%epc_6ncb<1G-rluenE}w{Hs6HtnPHrpR`z9j zys1+?`OTmk4LyS0LrB~(`}ONJ4sZ$>1Y>dR^g?@vc)JS zO58+saLG-L*~wyK|AF+?BCk=K+ZJ~I&ILA~q3IFSyvg92pDf7mw(;0AsX)-bRpp=Y%d@rL&D z{gfQHRodywGonv!($w6 z&$kCGx6#QFON=4~X`>_r*!-l5qXmrvFxHet6nzeSSOi%%^9GgZDWzgbhrcb8$h_#? zZ`v?(XQ&DD4 z(2YZS@WaO`9)z0PS{AYA$3Xka7JaLXha^A4B*|+S-$r|({@h`l|2VY6t!GeRH{9FK zI_iSJ%_m_RU3dCJs7{$&e!$RASouduRAqT3^e;`anYY9F>OxSFP6WZQx;`Xb)T)~t8d%=&I2=&v1)aDnH{9Rv{KJFk}`1{PH$~uPh0A!#Um^fV6 zIgT_IL*w-qI&jld0x{91cHb+~f^av=#|`T!uHzV+XR}5GCoc%RPXqwdDV6$E-+~qkqz!4=Y0QnsJnBg58NyC;nnqzeQd*s)tj!@GO zo)fmPW&Z&n*s7xe7SKMk38RoodBCoK0djtfp2G#MCU)O3$_hGSEFm?9`bAo zes6V6xu~yjn9yca6a^BDa1eyRm{L`%L~wmSyPI(r&KTcBIijAYZ?=IfLF65l4Y~^n zGq00(G925+uQ@kD2TfFFoU5dK7{tTqPNc33WVue|;5YPal-SvJ_f*S%)EWM6ytUh8 zo~CyOC>bPN-c;IlJtp6T5@o(BJ!N`QajO}5y$UP3I~pS(~ey7uwSgt%SXA$Xud|UZBu+P7LPBWCSh%< zv5=T7OQH%)XG3Zq`$%shmLu3gL{hAp{(MYOjCw~T{VD)UAUPpj2@QE8q}C>q2*C-# zZYVq#J2|;}QXT9~i@g5jZQG>e&~wF*Dz-Lx!;6|}jwTqkb0=@F0FbiCgc-W-&lCGv z`Uc#7w;v~-5HR=S#YOb#K6*Zj*#&cPEU7mNK8fVLnv{@}s2dG=EO}>ZBi(jHX=8VD zBkm_c=HOpSD`GfXA;T}|OQO}!xH$n9y$ze^;ig9@vqv_Hn~d$<4rROZOSnHhRGSDP z85D*qauMVdh?JJ`I z1B&2=>c}_O2k1}`GOtJpC+PBk-_w`@|8pOorwHevewS|6TwD!R@iGhdQ!bFOG#54b z0s6f7V;_6OtlRZqc_ncaCr!8UpGti}U<1&tNbiqAOoPB_C(TmyjdwT6jsSh--G*ed zu^;&ecoFP|+~F0iS*QBS+IEJG^R~e3fuq$UN!?6t=hXuz86<{L81_cii%|&6EDwB6 zcy`#lNSM%r0-qHMr6L}rZO9+T0r}{BkDSrDNi!2qynWvS%=vn~)bS|E#)38Wk@^C9N-%a)plA5Cx%~&G_XM3_nUbR<|2<`AC-VmUx|= zoNL+9DHF`L)Xz0mY*V*LtLxSGTl1=hz=<$fCPcqfAUT=Xsc=}$ZEP_FcR*%9>3Lq7 zFSVrMXGcsb*5aETz_dqq3N768fRR6vf33Gf#m&Sw6-MB_CT18&na^gomqNuF-?&oz zjrE{`aD+d0Jle(Q6Pv9Pv4Wv>B@!PyA5KBg;`b=1R+4&MB+1#`Q=Sk!qRCFxw!OJ- z_cJNGAPdX9`TDoBnO+HYve)WiDXY{&7xMUJ?MJYZyEV)+G`h-&?HiLG%GChvm4&dL z&Qja9yzBR1F59)58XzTcJN2bY?dV!UL8C4Edu3WlX<@MrQQ~A6z7H8;ji<3#}JYYBjRKwWa zN_1gVq2J{q(cBc=xIZ-$WOy-U-l;~d>uX?dn(7mgevKb1BHpOS!%>B>PS?_Y@- zR~qsbRmk>s%8Mwnu{R-f4^3w+Ch#k#p0Ex1%(k2!gg2M{l2b#-$8`G#x%NT?i`iCc ztb;!4SBz)K;w*uF9rZF>VXDl3IZBrd8c zwht8%7gq#%EXiumLM#Cq?ihiOu9c^)bSog_!bG=m^BVN#%!ojCgyIWs$O}FGo~SBT zwatU)U^OC3=u(&_wNZa^?n;!CYZR`_uZV~dti!M4uQ1SCK<>R^$sNN0`~j=QVI;&@ z8;I{ryD|vZFHBdQ_{Lj(H-&$@cE8rDDuS7LVuK)2RgjKFWXVKWvxnqsobxUV5#d~B z7&L!EU~ZYVzTpOnBZvy@EwSt-HH(W27-SsW!xr=z zUsLL8k<}5%#xhUQ`qz3*GM3faPX*}mz<#eO=l5jaB3CImOJ5*M7!M~07?!vNu=;jG z=RX^y#XJjLKuJ+Csj-XX<(=w%sLa9sBGa<_rK8_F8FXaK4xKLKo8M$(VMBp9Q73OP zTZf&+Di8OiTV18e5hCds-NR1NwA^BX&Sw&S_gu}}WHH^WBhP+~F@cAb*7M=ORe5;J zgp4s@)Z^CVy2j`J3%DKh>qZR5LCjPH_>L!mD6r>T?W=+=d+|m0EBs5zSi@cyvhCV2 zS34pPt)~1V?YtIo0!ceS5R4@2{_}YoD}~1Raz)ZnrJYxW_*X0~gIfjgrY5zR|0j@C z-8)3X>zN--7!6!}Y(nUa9Msn+1oyKUEMR=PK2A2wma0n}I>H$B>u7hwp(POYN)Cci~F~MUngZSWcJ-DN}4FD#sfZLDmA++AbAhx1RCzS9;VRVl8Ci+IkO>jL3QiPq2)i z3-KM*7)@b$*Nh8Y{^l3i>LUpz>%t-;i0GQ`OtO}FerUMgZH00X-5RTUcDXlh($ug} z^+$s1R(n}wE=r?U;C`LvqFdlli~l>_QStEhc}(fsxns;~dCo=Z`tkQL}Rs_j72#5Yt||sX`F#`Tr0BV=qOz{%y$y=Pc{K<;pDT_>yk6x zs={w&?eSv_7nJ4l)^8Q$s_e)khs0Skaj589U#BZ=xsj4qA~Kr4WOE;*vJ3npC%lphv6B{cpE@NRh6! zlzOwy1m$0N33Lbh@B)U!Z`fQRf*>!ow7wTFNFBk5$2XDG{IRz?R3{Nu%O^dqy(z3{ zv^>Z+tp1}|nS{27lDf-V+=wF6{TJ&EL(&a{fxw*q7ggtRwe%k+B?@`%{ShcoyP2RrCp6 zbh;TgTKNY2UC#~pUG_qKCV+^!)8~5ZK{f|ck5^vJ$MvMl^qrADqr`Q~Wv{|`x77WI z$Y9ZE;r6{ZNVwCNSKnOEHilgNS1GYMxDN8OsT2}_4ktB|dt3eiB06QT<1V#f?EBNb zRGl=V4PIkjoOD|9!F`=dSdaIiq(ARf%1{QLqj5o?Z(Im!*V;}ATy!{>^xEHbz3jSR zfMGhJCC;EOurp=jehF*St++p9USn}oP5=1A2SW~ekYwQO7MkUNGx|M^4T&=t6OtX%YXPllc z1eb)cexm2+dk?1Pv6Q-G-Qg>%l^?oAx@p8OmsFN4V8oEo$}o-Xye#FVY~S> za@9I`A1GE+O_lSeCjQ&Avc{>@M&BQ3Et5LMB#~40ZitpHr|;h3>;#Jz=Q5&wJ=Bw` zB0)Z_jFn6oe(b(&Y^y2_>Bf_G6B7HRzgesR`|+%<^-!S(cfo^{`;L2>@g_lI_Q(1! zlD-ZhWXz!ElDWmxfiRN$Qr^kC!hADYL+KT2?oaBTOI+N!9#N_EQ=|#=VnO9X|bnBKhS?N{Rh!@+aKYrtfQy5PueBDr3Syn=R=4L{$ zhgq{YvnH6yFgLTv&e>jilZNF@;beCD70H>r90&2-gzKyIT#oQYe==qC0N4C|t~UY$ z=?NxMOAHKxt3%BE@RFz2-t_a*+DE@!Kxo+OxdrC(;F1haoaxLJGiF1ALF?K~u#L+iqy>BXWwt@wM35frIQ-tXL zia<+IZ(c!^U*FtsLHYgs-JdIq;b3`=cKmHoNk^*c+a40AC-a_W&AdsH?pcw*qeaf> zS_qG_+KABrl+2iw(O-PI=fLcl-Gir&^6_$<{MgRL%l zwqZLBGe(Myu&v83Tz2c-T*>t5{1G|)upx!q=*8Vuil;TZ+{wNhO$wj~V}v~Fa%bMQ zYtt-vxliP7z_+@jG)u(hxOkcINQCcoj=Y9&)Dk5b)-+y~V9R2~WXttkUQPO5del~H=e0(Ad5e2Gg{Q6;b1bn~G^;7S ze0f8|!d?J*=UrXd-#N;c;}tYh6k4$84B>~e%bHDoJ0Y#)#`F2jACmM6zDjKhL|Bj^ z=Etp9Ls9~VuTse!ClwwY#RP6p$~PkAQiaIRpZhu*Fy$+zWg6k52dvd-wB!V{fFU@DVBAEdMFIg{p2{D|4e?`Z^Qhakx@Xjw;VBAmY(2HW}l z2E|A1+T7p(LOP)J%plEdTTyRjuCV?LQ0Cxv6NY`RWp~}vR+m4XanyiK`euF%_}ZeM zjq(|7nQ|C>r(0m5Vw(3*S4dk3GkP~SqGlH-Lonhw%I?qUt#Mdq(*FfLd#WVT`xA$h zCN~YH-gN}e zgn~%ti}h?-l_~zc6d*!4(zjXaQebse13 z0y{GpNI$Gs{w{ChIX$%CqWV=Uy0uelqKusLEq3mN+ z5{D-srKv@f+0mva=(FFAnP(&n8{z8yW5CKW)ahwTqugE@8`K_Q_Pd$=#?3iiloX1C zDNi=&0{V6n|HXqmy3^>3;nC^HyY>vdKP`L1_tCOW+dscFPGNa{L&{&W`9kAPlZ7M5 zO&qt*?N8_gNSdPN+WL5Jzkv{ zi9YLn=&KdK=$A-cf9~IJ$i!8XtfF`JMYd4rG`n4QqM#TL_2wt2o^40XXo%EBQ)%Os{LH-)z7j*-c=rj6&3f$eau_jbTTI z@$Zr|jWWxvNuo7}nPJUbx zjpsqzknkmzad=NWv(JE!4F&OJE0PK0Tc?J%XjIMm9J_~^Na*n^>kcgDW1s0}Cs%xA z{(wrMvX!Waw^52xW5{HYGC^&isJS3!99!qd5t8=ZD27 zJsdIL&ViQB3n(m`e=Nz+_U}U)$7gg9ANzv{(=|beR!icToJ6SUhwl zYaG&?@>TVkT{~nq^%(`BcH!ZZE%D&fgS$v?_N%pb$2MI*GvF_EEzJ`%I`B*{-M=Gc zG>zXfakiS!4Ww269<}3^&t&6?T5Vt^O^Uobb4YPL?`iLf8Y-!Wd))FyB88K$WMcs< z&V3{8yb7%=905mAB*IVx8Ou3PNb(*e)FxXSlIPvfk>FL^HQ?_~GHFYIHI}2G?O`$X z!Ka^p)Wy47hp#N)p7SVkimcay906$yv_pD=MHR0bq@Zq&p{x-K)E zQfq1DugrY>D3WSCAdJ^)x`;o$r}rqP=cs=oeng<%E@mA1?cF6N%@daFHm#=dmK!1B z&Q0oQvgeDUmld${f?@@f$zMjd)kp|m8f<=3GAd~`*CmkNOo(yL)FgGUn9m{aN1$7$ zb&Uvh+*3ABcC$?iU-OilIrTpY;4(5gATp%q>6d;Rv23?lgJ+~7)eK{onu~HDt8+94 z3m9FDMxMFV#=hXwNS=myt0xi|VH;Fsu%QyFGiVqzr5y5&yxtymq^$TDm7v&EBV7B!3x;;mH7?PdKWsbMk zez;&aL76>^nVV|j(c9n?gM+u$fo2s!A3P35dmN)_xv>|zIS+!aPQPQpzIzXQ@A1A2 z?!fc4S5J)SerY!+Jjbryo1B{0as@I*q?iH4(6~&h3=ysb?+r(=I-_tJv)@ov3)=gC ztNbg9hf8WB;>eu5+%JoFt2cereP%B}A$YMn_%>g~KQl)bYjC(08LKLnp9hU$FbICJ zF31M$AunEXD#L$2*ea+rUq2RRqH(Q{2no^gT=F^7KyhXorUX4rG0IMl>#tr} z_eMg#5f61@-3A)sMTSeX|ZUnT%e2Z3h{F2r2k}PvIr_9VPu-5ZJBCS8j$FhfT&}=7|bAWwOY3PGo z&3a~=A-o%Exm+F1czucNlpKS*TsCOc`^~*7mkQevEa2C6w}tu$X`i6U&0J6M0xkJ{ zn4KzfAT-*^Z!*}gweq2a8-P!UG@e8Eir3d&2P|$Rr|rA5Sod=yiEc+}Jzpk8+ZbUq zS_c(W5@wkhGr>OC5?Y}!xwIQH+)!+1uA@BnbKePLV;L{IN(h!xd5q8`?rn+l8^JT; zRP!)%=%lQy+KXmB4{y)c>yp&op!NC+S)f|J3BvU{F>aCely0|YC7Ao(Bgh&B{jOW! z3Vg=qe#29+XE8MzdCos)pjM9z`&>_k^&9~8lLHpG>XNe>%7xO16goA&ys;9ow^VMI z+hKXv2RmxB%FCG66gv`!Xtf??>0p=1e)o;GNZy zI_^%(gcHp1m%Y!nrm$i?weu;ac}yH!Xn$(}v222*;MSe4|DI0@^uy0k7VkRNsTR7|56Ve!#1Zd-f^;Lfv=|Me z(eVJNf*A|6Q#)zbGYvyu?R!yvy;t(Ru1dgrkY0PcH-4jv+Q})njg-AdTFEzgypdR< z@IyH^1@Y2lwV29=v0mE0=m`e>7+9@dva|RlKpt*u+;r$0~$}A0!FHE~lLbfAWZ> zZCQZq$eEI;{50ErkT~&=I&#||HbRB}aRn0RN67p%@SJP%+NtSpD!8dxc0$LkgqqIs z+9_T&GcVr8!Mm(h*MCTSk^g@E@Q&5Z0(cOBTxs=}u} z>wYLyGa);J{jASHal|4V}W3q*7r#0SBg@V;K(S2T~4{kJYyOp;n{|H?58zj`~bCD2_M|F0K%zJ$9LS1iWA{`xOJ^%n+eM$;X(fB)lOOWVZ&?&HDs z;|sBW)cS-}m_0!wX50zN|602IbYDJPG(kE4ieIpUU<^6LI+q;!s|Xg*h)hPNHM=zqR_p7xcVcl>z=R zQtd2BuN)NJ{$S|19N(JawXKz+JIxSJ1D6c*$U-BJe zXS92`c^ouwcnpYH9I&sSb$n&2HP-S@NEjOvOGM1?-y z9oB=Ns0UMbA0S=4Kn#&{?k4gbDg1c} zrnNZ_lRxXRyXIHPml-cmR1~^j#TpM5yvVr#q@;&0+`0nMcH>nPWB^+YIS+tsF!Pvd zA>@NtKc#@D)|)lak_!Pjzhi*06#w`GPoqA2q#$gk#0WqwQl4-STbJ+6b=yBM#t6KY zoX6!CE2CF@p+shR*Dc!uyXx(*&oCf2D&+}Su-(77c*A#5YUfQ~DeIETVWAA$WcEz~ zOkTQRhqG`W07@7SrOTY5Y8IG>e*7;OI1r3_VIg{b!mcvr>Fe#=w9-^CHXfJ%qi3F%^57>H? zw&J%;&FhW$$X3|3u4i&oD6bvX&+(VmbX8fzNZ=0vtNi)Fkmr%?coFEk4B*%0$_t<~ zV^5h+aR+1D%fRs1g%aDPA0iF{lG>@b$)@v{R!#~E&UylJSd2su@hiiyc|v|+zSMKm=iTGF zE)d`(Nu2`wY~^(v0c^x!BA`i~egSlCbG7K@F>tt?sit8NWW34O+y8J_znGcnb{0AT zWDz|j&y)N@-^&Ex6KXUqz%ngv(zDNhLzuE^hi~t8@E!c zOZsKM*Wk%S`KW=tFF621k%i?kS>}wRhV{sSDIn=RF#t#y+%IqUQGZ6$@RyK}R~Z4H zf!fHVHt^7$9)ZVioXX*#to0O{z;a_xoK(}c!OtwIK~E&mGS3;PZ)UL+HfOXn%i

&Qv*fZdIIkI`?_Vz>O>alEzWuK zR_94yJ84CM2RSWB%8+|hm$6NnOXyJomk(UY_2EN7`kaDJgPq zjwjhg-DGz{n4I2hZ|#&8{eHkAyVn@lrmEPdJKkHzFqEbWlcDWqtr2`b^1*vVlk*qJ zjig%xAR=B(%7O)*2pcGN?yG)UIgeLV3QF{u*hi{&SJZ_=ajOHo0p+ zfS(f5!T*=hf42(*cBmJcqGql~>eH%H#qWb7shwnBk|j^+7{qY6??rIzoyzPsS|%|G zmG-BfBn6KVp*X*tlkkt5ZrS?aZlY;V*ReW*Y>{T$WZ|(9Bju=hO|prTjH(X@1fI8a zc|-y;`1{76^7b!66}>3Q>1wL!q~qgTVUy(wPlC!$(N2Iu^M;X_KihDLIiNv%rF(zl z0PsFrZ6Hj4JcO8aL&Pv`VZFP#>8-`UCngWjG~btbMc*)5wz4M+hAFGh z`!_y;8oR(mX(dSZ)HB<<=zIqLl1885fI7VWU;H;g7Oxw;0GwhFA(j)w2fpoBcVw|{ zK{Rhc`?Zk7f{fO6kjy>z{w*#ynm82#`&7C2p%qaY;D$h1fA}{3>;yXkkQ7$feIls@ z5#M+6Jtb-v>s=P*@!!+bM!KdTd3^*SheMAE)1ED0%ibj8iflXqI`gQ4>+=nVYsp~0 z^cD52$dHe*?LhlYGeIT(ZhX~vHAG=g(Uu$;y(YO-Z=1t~toC5E=411lBKF4RzSh{7zMO3@c zZc)p0-NLaq4C3TD`-Xc*0T}MCr8=MA+u2wh@iSeEc`*M zji6l`G8v#RqCn$>qW}x3f2v6ntdEte;q2K`|mL6|*qXFcVDrn?&`&@3je2 zll#>5M#bBc>#}s}9bmM`axKo>mW2cY;#hDevY-U}(6H+d(XLG`>onX1_B^o{05m9w z;-;3?nq`8lBYO5kf6}q}zqCQ!ihd?iH7>UUvI{1MxbZ$E&pA@sk8GWoeBIt>ALeJT zKL@VK-p)=f<8kb{oCeav;q8o?bP5&C0I~Xik_o3!>7su2ZAJ3SJL%UOAkCCSIjtWA z7c-hT9Z&WGy$CtaWIFQX5z+(875rt;)}IKL_ajRJLOAAcTGBm_8Z7`lU0J}?&aJH; z*f1H(`2f9E@I(sk0^;sNov+&{`)XtxHWRhCNI!?>Q4Ev3;P|HND=qxC;u0Puj++%} z_Zgct5{i{JezN z)lXlp%m}qkgag@~rme3kwn0r=z1hPX3m;m;k)xjGA0EFq>svDmV=AGojd3H|+|dUb zGnK{02qKPVInQErk0(Zd^Uj*YzTQzpyzm8>nBA}YcG79>hUXrYiS3NB(jGpr@hnq; z3VXly5lvo$9`sSVCRr3}MrT=n@nISpsNv`eZm=(9tM_vKHt1_ctTAKlgtG1f-*XtS z(GZBz1|N)SFP$YW2x=n{9LgG#{K?NHfbVvdu;c4d=ipmT1l8K1l?S@-jO4g8WL6hC z-}h2Q$vrVmTiTS8!T5UStLX!)-5SSj`$x_vApz`Ix)Fkp-G*t#^LKHuJ76Tl9r9?K zlcik+Wq6IJElvEWu86VLW)OpMf$?`%EQWZC0*6GS>Zns#<^f4uvsohV%pYf;|FdyP z5RVw*Eq1_ zn6&04$Ep$wKEd_Nv1;F+b~q06)rq+!ICzmfQQdD_uh>P48ZiC@_>pK75`YvD)H4V= z(}d+e83%VNB-LHov_xvp`AtM+H_F5osvn*7d(DUOTE}!~3y4tqfEJ~dk&~pi7I2ey z+hw+}Bj+S;Dv5s{&R}ET1nHpO9H4edXrgsL;Y(#k)?3Nxgav^^M`6ZYMhwH z$OnA$qI`$5_@l>XN~BqR&Sg$}&9-rG*z{S~pD1mGGKQ9!A0M!`#pYw@6{oB_9biPlRc~5k8>)y{LjmYSI!{hM>RBL1u4&!C; zX#)CJPRsa?i{P-GG2JFYSWesWg)mz%gPe?NZMTm`vg0q(WBd+%S)nH`(q%Sxa#}+m zE7_c^nS;}&lGz!JhgE;5hMvfzIPK~;xi%Xa%=~)FUokp}zqxL2R0b{Gs<~A`e8z!; z)(HnfxtDnSpAX{!=BJ%Sy?O0Er}NGVM;Yu6Pfv=7#aN&B3=1p&_^{dXFLsC}?|u>8 z*cU*@bkd@%X740O%aGx1G$C60K^tlr4V^#buhTo(a6}%djDwFds(Z}$Qu;JMujSH2 zAg(3&^thY+xakw+xh;X>vv-VO5xYD;JKJ3mXhb7#YblMwjK`g9 z#3=#^5G>*fetcz0Kq3;U6o%eV*CJ{md=G?vY^7H91dK|&A3D(#MR54@$Pc-d-%+qb zXI6`oqC8GZ^;+(}3r)Atctmo%$gW)%;62_)YuZD|hkzyrS4O;TWWMIMSlOVxN=S-6 z297vy8=t9WxBOa@-LT?EWujRTL9{dqb;o@*hnG;}b74d-Pmt~_GR~cm*BqRbJ5QrN1zp7t-Zg)F+j5h@9le{YL1A#!M3V6YtrRkz+-fulJ0xb>5Z3S zcsR>lG*5dzgB~_`ouy_yyr)HA6?}&?xsctN2@8S8T)s%|>B1#C%~D*wKJ3MdY1-Y? zGf3}LNvyIA9+A%q9A#{hs$7B2`!LUXd=FhdT%E6H@HZx3+}pW2dNUH1x@T2OP5UmS zisk3H?P74sM@3egH(9;>T3A-8U z`_-T!ED{!$;WQQEtvS&_!1gjj(PAcJBEKoeODQ&Xma*CBNuj$|w1XHmI>JafT&}29tX!UxtTr6MXm>=XkGMoSdpG z+G@y)PhwftEuQ<`;2y{}=tq`}@im!AET_-pydu1le|`XQdqh$67w;@wFFB=KDsNZ| zYuV5C-DqWOCU%>#4|sW=-sA(2!?P}V90WSg9&=>>VCb?5-0vJbPCOt@-Z^y_Rpt1W zuEaY3YBsehuZAQE#tAq{F@ZK?$-FM|G+WKqMO5FG_^yuvnVvpK zHIXFc;ewC3gmR&mYg9Bh#_8Ed#xr$TUrFg3 zI^$XBG9OQ_`Y63Zb7;^*(i&JU(HI#SnRq3k%g=K)Fs0oa9ysdj*!g`DU)wh3Fe7&= z()OrnzHAoS`KNUT5MS?q3VHuO%_Bh62#9QI`}B40ez6Hw|FfH6H}=dyxfbtw=jPpR zD4L2&Lbpk*)JsA(D$?_&6!j*)EC`&wEeA0Y1P3qx@mi+oP_NWJB*J!?@R=FD>? z^;vwLaB`1cvV+FyOwF&<0kO%|L%j6N_!EDo=ELp=J@N4u!gXyPHL)BCUT>0~UZD8( z(mzZV=*k{DeN5CD_hi!&3M(P)k)Hfu%lVrUO@KBFhej%r(W^26$Moj+>A;RqTR{0% z{WiX8OVsuvhlhoA+h7o&koCLznZFb){IP;~G!w#rG5UtLAOduL08Y=it(Bb}Y6D@h zg!)__a(s1<+)(P;Hs5O6FG<5yhm0$<u^G)o8Z@1^M|fE3 zx)g_B44ItLb-Mg$YWZUErSxG%U4R<+-dtT_T>kbjzgd&4(8@Z7x@Ny%@xe5}g>S-3 zt=+b#nwoCOqBzM8e)LZ0(&KQirWss5zju9b@l&tBr=`)So$pSWdAiJ}Rqj^zKYK%D z1V2IfooDY0`Cx0oV_;i%S^>S*PuFsn)66EWk;Y7X&Yxf#W%T*ivwdk%eXNQ-k@rgY zz9rdD(u`qWR_-F3k7S2cdzRERd>Z#Obhdqw?7MEZ$EYS92685Si}YJ-)s8=2=#{NA zcoF>X$oNh<>V>-~f$;sYAKLEPe9sSU^0Z|9zc^TI8qX*#vg8vL=wEuTW>mc8YxllW znTP+C_INLP&$YBw&zgA3pn)3U2@&nRZ>eF!y9eO;u@DGwC zviZ-1^0XqgiLKDc?!atAeiZy?6Z5~Mx>$TEN6dU*o4^ChBZ~fa6u#rJ-j#OcGrl^Z zAR1a4dQ}*MKf>$q?5fbW{cB9U-`|>lu9>X+2!yb>O4KsLv}`{A#JAdyY&O^Z1QNj) zk?44dY=Zr2;{6>CV&7p)oZO9!EyWvqC!;p^%#EeH$Le%N{@wri9}hU7>K2|9R{W!Y6alTN%BVc?Z|z6D1XcaL*ZY4Im`DlxmmSO$15NoKU*Z4!e~SmyuDq&Z@Q(sMz?{^i zM~^Z5%m2M&;liidUu3=bN5L$1bT&QTmF3^umxVzeC4kyhZ&Wq@uVbn(s6P_ul-Wx4 zKYueorTp|BsQpJ*LDxSDAV8<|N49F=|KE1J@>71Q9ZwMN9|cuZ(TTnJ^|p(a#9X!) z@=!xoSs9s1z#CH=u)$>f;IcjEc)m3#sF?8n^G9no6N>`nA0KI$ivQlXl+h(1ocB9` zaFQk9fnGgXsy!XfF0NB;I#>c2^i7q6+cVgj?v9#Em2xGYyel)d!d=e+MTe?3?>nvS zp^Vm$h$N}$Cp+r@{g>T>E&VfFyWOXCvQVW5@HO6&(1k$i0EwwVK)k`ARJ*-}s+KGH5_Zg~SM$SFRaFOoXJFMEv9B6hTB%^RKDNQh zV)eWI5KhqI7X+k|&&6tOUN;GGaYm@Uz@2KfdR%syw1)j{d~_&&?HJ-JJ6yG;gwBWs zyWIgYTo3zz{pvxI2xNEcH_#kocGH21fdRSYGr&vr0FY6s1{}c;DpVZ--5$N4FG;vZ z08KRgiAuxYVWcGzJD11H%YePLC7>L}3d)TAzr*nh>U}&aHHTj)h;F|`#jLw~?p$C~Y0bCIa4$oeHR4CMG zaxSdQIs$@T=bNl)Kfy|#yEr3^AKv_ZAeo2)k0X){$3-mpv>z)t$KGS}{`TB^B2vgk zSo~X{y>_*!JUokIc!vQ-Ru_JpU&a3qY4y0qLlu1JWk=CT~Q(qLWTNUEyiiVip`M2 zmQT5)a}+r0tsXA>fH`J695~yVS!i^6V3SiQw4SdkRk{KqQdi%jHdBBTzM9kZ9%Ag) zaOO6*0^c)zz=jfr<3KYL3+8w94wl1mDS^A`YO%;8xDutA$s_{5- z_bu&lpHsGn05QUs%s(reQECB>8i>o^wCjg}P&j$9YT+2>;f|OAqISJ)EifSgGH(wrmKV^~ z&C$2pv}!GbEWv=y@gd;xr&g}lDtZmP{>EV6hgwqLTsP&gS93eMzq3E-M7d+|TGnI&%xI0~>jobN|0oJpQcUdZ!|#smfZ4C_kt{CBmlzPH zdX>g%^p)A^{hInYL9|YK=?3)z`Vayf-T!X>d|<`zd;_RgS%AGxB`W_=i$PJyy++#Z z&;A6auo?bB6}!hzxUr4=>q9?qq{`_D-1p%vL1sO0)ZPsTgbBBpy^XB_Z&eFlKqc=0 zkk_jPqyzsPwLkhaU(Hz^0>{K>pM`EW;;w)yW`U*l_)i9)%m!aZ*n`b4!%YE+uPk!r zU;LpC&A{|~?@5O7i2%zVa@`qKF5Uj$<(9$^6w4`EZ}JBp*a)QS0y=Y7PjeLzojvzr z5ZAG9xOwv)m)R2VkCuEih65W?)+N`Sk2UPwziu>L@PVN2b30=XIS84ggQD?qi&UF!OvdiZLg zAu=w)oVJVAZi%0q^v!pX8cHgvZuHDopWnnHkbl?OkiuZa7Z%6b`T$lHzgEx>@;B$%A6lsiU;NxQhEXqH5 zQ#Rw+k;kV4RhkCeg0y!Q6 ztWgC{ybj*TGX{R?*q5(&@ZsmTJa0cBb4;9dyR+M%SvK&-`AtAujnlHsZ+{0ERTYJW ztl&K{^h#k#gPo#zmJ>M@9iU3&W6HapG6lPu(9-|t+uNZ@F%V@v(uISvBWK@N9VgXK z4{qfU7ieYXnQn`rDAoe5$a^;_fC`r-pwCpK`^-Duaj?4Staocfuu84_^5Tzow+j;` zbXb=FR-&@tR^B|sgZk*O(I*hamV2wq``8tg!q@AUbkDV#0jtiI7ea~3zZDllvcnufbtGM~e^snb0 zlCOY4z!4oSyaH_N%D}^(>ELGH|~#*fSWhX+f7wW8nt~=y<-B+ySIn=a;^) z(T)OnL{=E?ULXhKF|RK8UU&Sb}%Y#>Co|m^XKQ=6xNkn&o55 z&Tse%y>;d4XEc_E9?2O53A~P_BXl!g+38%v^Ibf0nKnCd#RpI_e1>|u*ta_CzfoxB zC8aU8%UeCpC^mSMSs>%H<=maza7)-q&9eTGoTsv#n}DYoYx$n!7B#uokN)U0&cVAA z-EQGf@B)Dy5m?&*sE*;}Z)6Z;zegRkEu(y}W#uG_V~4DgCjkrh4L1s)7Z^6`GS6)~ z;}7^BD#wRz<#8hsj*g>jD%Z4rVA}cSY)?_O?^&JN=J8wmDeQ-^Ho(Un$HGniz)wCs;OJkt3mi^BmKYkO;h@#xnu1h7b#asbZP!d>?^oc zR>S;KU6L;rY;Tcy+j$VfQ|UB57O9!BGfigZ=UYQdKc|0C!xO!51uh-vF@Qr#E1}6tYUk8rTbtO@m!&{>=?jN=yEX@Y3RA~pc`|=! zjwDL-tL5FR(lm&WUtl57qN%H(K86vYCiEg$HYP+Id>E=9ye~J5gxs&Ro#eLdsDPgq zmm5K{12bg&M<5-~!uqWhART6d`uMzGdn~{5VsEvc73aSBO8K>6kXlL^)HOUdfYT9X zJIHw1^Ot#d1+W-Go%z?)E}{KgviXX@iuCHVh`8(7qH^mPI;7C@i&FMtJcO1mn=0e@ zEezMihF^Tp4PXQ5x5@EseQ&Nd12fO2j$1?So@Y#rO*Z47LOx(mg$UF8VugLL^!0!` zU|Zk>NmV{+A@Z5sBRl%;keUEFwb(M1C@J+yDYiiuyj0r^qhm*TRBXP`ZX0HYf9)!A zz|&b3eVR(At`FyDu%la9bhpHGOnDAreFHJeIEZrbIYu~NOp!vK?Bpia|&HP)=W)1fK)ENyOm3rxs!5R`%i2JoV;lQ-;j+k0cG&hT4#kJ zVIUh`tw?K!x@$9StJpY!$!ZN2l&9g;K&AFA()FqHuteuOF6CY^s*Fqm(b6c10-KTy zQ%^ABGePK8yQ|D_6F*ie_cU`CEQs#47COLNE+2mTNRW5m^(ItLZcMIaFmOwtHrSSt zK#gXAhWn~EF*$q@N7F95uZ#@wx(3F+@t13o+a3(T%*eZ!C0vy2mcx%|-JHNe$<$V8 z#%*vd@-hZWVB2Jtf}KP*Ytr7A%RjMO8o9I=3zA|Zz9rG$U?xG%7F;OMq>Wb>0UHgX zv!KsHU?9e`bj29f;uoc$;!Dulr)9r3Wb>dCtnDN3lGKllE#ugpED^h0Q z_fahemp3UJlA#HL5>{Gu@Jwq4!EJL5gXASlS2mghbS^)Pr1|Ju=muwf^_RCn41}-c zCU@vQYyM*hw+SJqh6eQ+MZ+W%o8k~6#r_ePcO8c zj!9WZBGYqH?7as@(_&Ugq+=*K$jcGq1Ac1Ap+Eax_X4biq6Z}6DTQPrf7vY zz7;yEV3>}-vz_`$Xp|4(Sq-o9EtLyo+4>k8m{+OFs~{HLnNv3GR{}A7JDPpr$ot5V zYw;(my5;!hFi`$e&4c_A;4B{|UlDBuLVfA`v<%UX8Xnq*mp~*4Xi!h`IK<%Bv#sj} zXE|x?=UA;+Ta}{(w+k3gTyYM$__{bKGiM-&Ye79fc(xv|(@IAbot4AQJZCtT$Pob&_|c0VlUm^!N!y&C zi4$^+$%XGE3-gUtgd~(plDW9e`wQa14KN}3ikDuI=XUmhvw66~#LaSl3~%B*e;@dX zTnYat00FxLcyLZ))#|HxAAj^*?x{?dAQ$F)p<(8q5f3lG^fCdQdUnhlUu?B@D-J^Lpph2 z**08)hgjT=t$3>pt_t-}$B!F*BG*{ZG8;ornr%;71NI(9q|-Sq|;fPXZSAsa7Gf8ZH%-=)Wltt zu)5wCMR#matun=yJx(%licQvn#R>w&lLPZb_GcAZf{)G7Fa&DPNWba3_G8@bef4@! zz4KJlaS#uN)?<12CC;xAM$KqZRk2gu1{!C{j^kG&in=r!M(gxNTW9i%bfua5i+Zz$ z!>b;T4y*L=$>Uo>Vli9%SWGnAhk?X;#gPC6=n>$6DG1U@LI@N9$2Mh^frNiR{1gU( z^eot7i))_q;)8&J?ZGY?*S@PHfiT-MC?P_OCkg>!OT(8UBYoba5l8l)yf#g%caC|n zS`ZyL_ zcsyg1e2wTeAQm5#euKI^xrL6GDS~#@J>FKWE<51hQ zXXj96xht_?J-igUUt2hR2Ihw(CgT*-_zA56>n8?ZSY?p+)!SX;;okdaC75L-=PJ8F zd%T_#REJ7g{&0?migFjEO{eNKu#))qo?cJ>3OcCL#)qrs-Zzdo$I7Kuyn4ykZH}#Q z1*_LYf4vf4Jl43ndhSQ1*h5a^*2v-SBahsnC?dqNfO|R2;G?OD6Ag8wwLelm$adOj zkdt-o7ATr0niJRS?CKSinV2z`UI(6}$2EYBF``^*n+lVIlRo8?D8<3xiI$<3$p_s% zvZ4*EOi~0zG?mvY8OOz>Fo>YQr02Ax8|ZaF4?ndQ_Jo!q95F4uTi|EBk)!g4O7E*7 z*t&_M)$0pHNKK73vMSINAP6LU9mQkFmyFq2<={j(hq9dd zNk@X*;=(jtcwVZ}vlel6DDhLL#LpblM5A)9=z~@DX47FcgU^?UUr+AloToeNQQicP zLbLn7YxvpHG(|e|NSX)OAUU=|a?^cUmC?c_QJgr}ZD~jpw@i;#YsPj{bzD*njSBjN zH=HBYIX_%>Y`@vP6;zj_QaV0FITrZsOZfQgac+N8ce*LqcVr+e0EmI(ZyuL$pn zo*$jhN##|;l*b1fPpKR7?l&x)=zYZ-Zy`FmXS9Osu_!K48qKJ2;!a}WdhxEfD`(W> z?K@6;{m|4rl{sutL@JF|oqJ*30ncP2S4Gyx_$4iF*>R&fUj1553Ju8V5l7cS;>s{p zak>|($GX_mI&YI`tu*4=X3$onx#3Hs7k(gGbFlz*{kGd4)6$co@2+aq;Y;1+x}5L7 zz7~6Lu{@&UKOs7wgiDpcii3hMmq}}WO-u`zNh~ptsFRV5K;QN zZkszZ!=WZu;du`;eznk0qfBo99nTDWBQ%j=Ns3KV=Veq8zXra)y6jOVcJH3T%!Qv} zg}i25brVe?Wjs0TiIoeF=RLCd=CHk=sHnEhx2{ti zE8`N0qx0}gL7R>lL-$?xvNUa{-R_}zm9oIPXi@4 z99jzUuw6jPh>XXKZy9kq`uXhj)Oha_kCNrtkq^XZZc7A^CN{S8knA;KL$#0uhm zxq5o#JdwZ2icY+NGPTH!8OzJB@WV%6_bH;vgv1jd*{c7mbsSxL?$_Jt!S))w)*iDtz+V=x+So zV-jy$x)RSM*7u(@Y+f!T7ewrSI}=@?pxeq_a4#fBY*eVEcNB-x@xXD6iZnSshD`Fj z`?eXBKKrh#{z6CNnKzP$_iBy%wDY|*$o~j%TcZF)$JfDd@5ftuB0N-OB0PB}b837c z*uBDY+bs0C^=${2GJ7|T(;FI;9BgXdNhjGh#Nu|(OR=ji-2U^N1B6emE(>h@YM1-_ z2me8Ro__@}j|7J&T7Q1^Pf{-as6ZbS6UFj7pODRPwdYKyrpMoCeXbBXRw$p+VcABU4iRD`{!s&%2>TFPB~?X|L@1-G{^N zvUiPU?H*M*Pv6;{s=JQsh`x7CTU6ig-)*LXcowmQ7KmRDCm>9w+ws`kL#qjs? zBo$+8j@u&lUTA)&sENMQ|GlvyY=LO{S_Ae7h1{p7?Z0{Qrpo)je?L~QOQ?ys+}(Ho zsG7kXT77b3JI`!E#~(AOLn06juYosuKB)PfaTEZGxFX`~N0Y4oj_3Rl*tzKk%(j=~ zWiMubD>;(}Ox)v{#}*s@eK*x@K_Dr>uML&>8)0wY3~=mce5mtxf4kFclIrVQ;y9SIa72Dcxh{(ki}}| GLH_|(h*d`b literal 0 HcmV?d00001 diff --git a/images/adb/instance-principal-3.png b/images/adb/instance-principal-3.png new file mode 100644 index 0000000000000000000000000000000000000000..602d209a804a8df994162a4b3236bd3ace6257ca GIT binary patch literal 76392 zcmZU)1yodB+doVWDcuY`ba$5^jihv^$k5&0-5{-iph$;EcMSuGlt{Pa(9JjA@AKU6 zyZ+y-wb$8a@2k(QbDiJBX}(r`fk}ahfPnBqSxH_S0RizIUh2^S@D^Jong#fcoP(U4 zrm~zIt){z+odehw0f8mW+QNcBnS*7-($d0WWP+U))7?iqHa1b)B4n%wN;}xo-_uK* zmTO|NMuxYBtlNk1QLDGr5j&Cgu7PW$y3tmsoBj(a@30APrS*0`;s3}N6Hq2X10Xfjg6Zs@{3-W32(9HUPPNso|V3)^NDp#e3W zpCHVRrwgIMD?N2!CT&48KsIz~*tD1P=dfaLe^jXBC<0>@F(d6S(e9E47JkW(^1b+E zDpi<8exv?kmhmW6Y%)_yt?%K{M?=x z7IsUbc_NJ9DrxRj`a=H(9#uVC17$mPbp$qe9UTEN#sL8ZUPFWz3V1<4Ku(WFK!aZi z;6=U!>A$^*_a(^x*6aTg%Ie4|E5olk*6y~p&K_@FJoC1xkl~`{9CQsl4b;`dtX-VA zENxt@Y`J`$T>r8lNcf7ut4_9_mbAW3U}q08UrG9ZD8%6Pzs=nAwEvKJI!e+TsB6;7 zxwzZX3UUc>@z6_Q($dmOxZBu?Y0E48haG+=N&nW<(^ZU{+sDU;%ZH!K#oeBpS5#D# zn}?5^kB<{h!Rg`W>}l!C>FmMqKSKU1N8Z-M+TFp`)4|1=_ODz^D;F?wQ#(z{A{;x^_e*XWf^8fVwhf{+4ujl_K z68{tCe_G*jmco?a{`Z+lVfJ&fCLiQxcnxdH$&fW$u;-MqHBv4SI1=*ZP zPH1kn9523WZhrTrx%tt@$kFM`ELU^O+~?+&H^#>LR)Wd7o!9i@C@;wn39h1_&kM%s za>hGR(D7(bMpJ$Vbe|+%OY8>j`b`F`cu3V6p-h*cvmnKX(*M0IlOUcr8}on7{paqV zMlzJl!)Av6r2rzD@7~S&z7l)>c>*Vx8pJ=Xnb_YD)PxZuzAE9wdnlTc}8uC6BL(@{20$pF@6;b}IUNI6vrYM~GzzLJ4c{-&QQ zBM-4witShy|LxT0;a=aO@Os-WzBOlFl+_ID4T=~)Qfv172xRO%l;gu#1zgoGdKlAa zk^J*(`Ca<^#1CM`dyk)nO$*SJ(Da~P7ltfNd=XOgO7druk{VmK|9VRm3-3h>|N4sc zzb(5+h;YuZ@7Y!Vjfp2Ml6uXePYKW{6K-*V9Li@{eB$Rne>_;iNWQ{}j^f}l0zQ24 zV@EPY2|G3G1bsF)ud1rT58~G&br2xd!GDX0q;kcHp!$&wX&FDvthAKR@_{TaA&G>B zruf+4)z=Jq*A<2mn~)VBMAWswS0#Xo`;s06a%tzxfyqLVEw5xoyPRM@ zfB)O^Iwn+X&+9J{6oBZT>#@IR`QDr3WxzUTT%g8>_EyF-9&(Lqs6ujfMYFq(-fWL? ze9@3TTqi#r+&QMmi{KpDPc+K)?1G8T@QC3zQ z?nc@cgMZXmc$&yy17N=B*+LTbo0^o-ZYThu}W` zTi(T-tn5)zmrgV4=jm1yYrA}>S=i9i%GelN8w~0)l3W-NX&f#_e{*}QZ1sRZF2G{e z=NfOoIZ`?rs-mx_H@2Qhv{>(0tGgMi)ovQs*f>=rxqbJAJLBvuo~f~_8v{F!ChqM? znA{FZ<6VGrS8CbIa3tF+NEM&|7W!hBM{BiNyH>oa>N3=%u#vXn@lJm1v{cU3$cUy2 zaa4e~qmG!JlhahMW|`JnF)1~TAzGPyNk^vx_C*MR+>iBrh(qZOeD6%TGT)g0spcJ%bPB0w%wy`r$2&DVk$2VBc$N+wRubv|Vi6Hn{epHlUO3Zf(!DYUqeV-?`4n)z{2A!dyOcy->SSx0v35 zEb$^PB~DJZVIH<~bTjg4+Er{e&ZT?qeoi*32|!th#s&``y~pQ>bQkNR#+S0@_1iK8MnoZujEWqesdd$}@aFzrjNWMxHQN%$jtMwa2V}Co3yU5Y-?0o`26xWkI8VbGlv236 z^6Ee?Vc4E)fKnqG$y}eTa+iv%h)PMVB!<}+AGD2!1^La_nAWl3rWe(r3locE7xd*8 zW@Rxx-d*iV6_84PGCMir!NWY`;3pO3H6hAyJ#@>QZ-1Y_{ma z1M-fMUq-rpv}CHuy7jQ9oN?8AT~j5s+^d8wetnoh{6#3WYzQuDjX;(tGD%^VB>V9AnXZnO9+RvE!s>=-{9iZAa3+26Ht_AP3FJ z_XCs;hEg*Dn83qNYCv|>SFak}&V?|B5?4%5(fo8^zhW8!5EV7%FbE`btTyU*XhVX%Z#d+x3TM&}Y#~NLicMYb2%!#;?~Ws#QhXjtC}_3CZJ7y&fb`#c=jpXEEa5 zPilFsMn)S{U+^*`$WE^%EpF`1xGu~5k6$Ok_nNY@Vc?aH#Z<|qG?cL}amzjwxgzg1 zH%9Y-!XJL$=YJnSBDURoHu-yGQ?doJ*G+IXMPoLHQ^vD7jV;^~1Yrh6pZM}>ssd`R z$>+0Ib-z~}x9yCswnzyTK00mwTD_90lt znQ82 z3ecmSgL2138<`i{Gp@(dZP{hnq<-+%ttU?Dtbp91U7}x#!~?|8#})0=@;RRUPa^I> z)nVLc7g!gBP>y`eMcU)7AT%3#Gljkkh`+TT-3xwvN*4ZY%IM~+QsuacwALdvK$}w* zs)s%Km!7Gz-Udp)!@**Y-495-&W9{I^VrpvA*a)y(qk6=-@2AebeT$jdH2~CGLoi1 zXZcAI-LaQ`L}s{+mf)P6tVYz-a;-;XvGutmvqTbE(n5VO?inr+62E`c=5{e|@k$B+S4}mS?pj7g;jO%orH!WWJLS%av{$wBBpGe{5k3I^sB2WFE7u_o2u0Mm}sIV zH%{;jUdhb5J){0A)dbAU)Kn&%dD>B34#|vjV3ppy*qiSEnKgft?{`LhFmeZ*Y~zAn z=Q19cqS($-Iaf2lJ`qRVx*nDX4#bkp7|9wM3T?N7dc5llDA_@$*8bVDh9Xz6)=v9_ zVb7)&o;tM^X&nox>ej)Gqn@iZFF>}#J?g>Bj)kJD?@j?vJ3e#cR>Kzq(O&D|?8g+p zQ(-r|ycUxU6~DZqu*(t?NR9&w;kmtqzm=*4E4Qtc|IVVWMIj6Bxt&RctuarX`Dw~+ zoym}G*T6KO!WIZ()Sroa-xFr;BAs%1<GSD1aq8Tzzkbymoh; zA{aSJ4nEf_U$)tDkMm3W+2Xt!r@#Uh?z7iPCYSu-i4fjO-T0P(RkA6cQVH0ri9Kh* zB#MLrq*|U7DMUp+1VjinfTRktglD4+e7G=p>eksmvr)zoBpE_(ct6UptfYHO$l(CR zI8X=68=UT)d(9onW)c~V@*jD zv=r_%@_kC| zr6?=+L0=1PQD)k=y5AMql4L1ILoHVRPj)2_$N+4f$EXJQxTIdH33+}3@Lw@jY%*`# ztQicR?LzXbyZB#x<;(P%t2V575paF%#GwLD1XNIQ>}2L_H*k`Ltab$r5%ip3g9D6E zZt@?)`RZ3VBKQ`&-qE+;roarC)K7HH@^EXDD21UGkl=cs6SD}`tU48rydjx9<}1)+ zVj^~_JD3>tO1LGhz2{(Yagj+1#>k~>;9F>$e&Mif5v=F8_Bj1fv3(qB;>f$G-ybN7 zzH%UJ=Ft|BDv5qjFOMs1fC#-x?#cT@)S@z7y=Yb{3|T=r(hUxyla~AOm4yRuAkF{q z8(qN8JZWJa3e_y&rJPlHj`Zi-_Jzs9&Ad8*Y7LVYEq*^Ni zHVCNa#yww40UIkc9C&Dj8L`AqyjfWCTfqEfE;63VQ(yEou!#hr~s0gxoGi3mX{t6#zK|nwNaZz;#DdS`U7v6Slt*P~GxD4&CO)OGp zVXx4LxRz)UaJ&PNq!Xjr6O0YS5x%IZ14Er-aA}+YTIu))Lx%XamSQXRMspeS6sRT} z2oPKBF~{?=j)Ho(sZTFu5|wKFPQ~;1yyDYR&Ipj_o>A^*Em@9X7%%`O&Kij@8%p&T z&SLAbtVy$T*cItXe#aVlHo+kA{HvK;;Y3CIK3O^a(%LQoeg? zCTB~HS9x{jDqB&9MtNGu`R=6fv}h4O^*!pjtvDev8}Xqvwr))d}FwQjy ziE*55^1Pv!oJppT`YFI;pKBs+XEyuRqN@r#o=rttWk8Ncn$SRCPgqqa-#X$gE2s{E(;LW&}>1 z;yA%p=D~^BCkr(tmg$KxMP#1dbY$dc$_bd5h(Pchp~NF)5G6b1ckuQ~&(fpMF2pJS zT2|K7)h1!%^_t5gl#v%^NGIxsw{4;PVJCD{Ha|jqG-YcjEEbvVo@JB?e!Nc=LStLm zv>1JQ-M6zg9;loP{0Toia4Sdi+dZXKDk}j(uB=D;`IPb0h6Od>E-^uWb{_@H_~Jw=KyZ zK)gC;W6D1o8yW*QRPtDuVKAat)R{EEf^!Qv9e1Yf3Tv$Q>f`JH8@_kNWD_>Xc1%8< z$>Adsx7Y#keYGbnK<)Ju{@aj3Vro6#GYbf^sHJi z+}8-TXGgWXk4?Rr`>TG@%X0;#Pd}^i(B^h3wSZWlm7k*T9hFgctiYbAyLSvM)BO!E*%x zVO&df^v2K35XQnkcXIWABG8IeG7ch^Kly;XZ3+a&#cEK`Ac%7xKNLg=1B8Vv2*WU= zUJQhep>hh<*z(~}U8|g0c$~g%Xb|f6I31Oedr4dGD#O@YZlR!<4%+vyuwk)Lyu-Fb1J?EDEB&v&&Ek>h@*zZh>BLto0dEO`|U zixbYFVEM$%;ahmCb47H4Hdp90c$Wqdm173fz&Bey&dqc2Z5xT~@v}-z?OTRLe(9l7 zu+XXajPZ6@x9G>#Z2A@JN6sCHw8#Lj9p4LLNj7O+O!#sc%O^}6OP)nqVI5*$Y~9J3 ztu}t5r)Ljd@mc~xmb|F zCSt+UZC|8lxd=A0!n=1e-4Js%&_R90-)h^5Q~s9q6TW)~btwaXSP0n=mC z*w2M6ILIDKMo}2JLxOq~39bjAk_P~i{1kG$j-bSsR<(;7B)dl*OcLP3@4^`+q+++O$57rb5>*`aX1Wk1U`tm4tGmKjPZLK( z*h0F6L+3g;3zMRrZlB*}$a)#+!lgFaYulp6BHBK0EA=frazU8zETBIJx56TR? zt*?QG?N_1WGjq}Lh8SsAJ_r%#k}l=(Ep+vsWX|i<8Q_JCP0D~-kR#e3SYr4qm_E#e z;24L`MDYieD(<@KkAxFWnWacDc?_uVzuR_64A{(e`gHK2L3CO^Ou2w?&2o}(9kRNYb_3*sq~HpQ|Ew#G zuywUUqV7;*5guMc*yE0<2l7~-$pb}@3EvqGlK=VB({qUE#24hf)EzQJd$m{@k!$hU}<$)NJ))fKfof#Nn0H0VGBO57&v6MjONk4Mfo5rJM-V zCSOyIy6Ux-XrcUyD}mheCa;8$p%5E6G%kkgC+&0cL$6p}iign1r9Tv`q#(eMF7I26 zj5{&7T^H!;J>Hw`wWIa}dbn3TZ~&5jFR+}J{grflGjRy#2AHZ$&6+FfUbjj1^Sz9r z=t2gHvQm=(kIUF-eQ1P_D)M<*wLlg>Hvn2^pg5*ss0=c(vj7s1D@WNt{L}H`$R`>> zY&D{kGr@)Xh5K=W&my@Vw~^wdzb$*j>YPFNY?dIEv7UxJ=A~FS-@=BvW#3|yiJq(G zYznBSvXb}Cn0M3BzKPKJ=CAV#^?dB$CLBCPRqYmYP46Y{g&Y>-bDtLj88`KXv6Scb z)E&+dPKrCj-euZ-Vt8^J{S9rpvWbhhl~uU?HA zbB>9~n$1L)Fj$l>;8)j<&3BzW zS^;(8&+Po}m@(_JF334l0Gz`VG=kSL6FgCx4(sbLB^I66!`sNv4Qlr_B}!s#ED9-d zrv|o_0UNSbJIemYuhF;2T-Vm{KVG;}ZhUwS3ipCZl%AIjrps$!_20NLyvVBnO4?-- zqy1>HLwXJ+%HwxrJELHAtW%A^NyYX|$Cg>b4b7`1NKS47mDHElRZfY_xv5XL5m&lm z)4bPxK5cyH*W9Trd7%=U;}mc;?{Z#7KSLNH!%=W9LMD>rP;m%7r9rcWLiQfyUana{ zWowDDQ?))S=JlpTXgq&=vg=td8FLFqZh36P(FW1s*aQL&E%qd=ErZ~B*RSbCdnOS! zw~76WxtvQZhF)ttG-KB19)pRti)Spj7&*uc81a+b=NsR-F;XLZh<`2Um_hO*IM1yu zByhfiUch%~n@6SJFc?h1n*i576q3IUqn;S4W{yj2RK|ILzs4X4ur^|+bH#;a<5dV= z*1G$KHwF&5F<}!hV&p&_K^QH_S-R;jwo68r*@#F8Yv}mvo(7^+unq?A2cow5xQ*=h zu!np}Xo9tS>!9w{gv7|uz}rGS#49W|M~?X`tbDX_S7+6+UI|u0SXl>0vDcQ@-yK;7 zlE>ygZ&A?cNUJ4hyvYyk?+5inqz zuQs&410IZAU}sKRy$fkH*XGI|_onO0%yElD+!SvuRv!)?lj^(CRTGiodokX4J1rb~ zX)Wnn>^{Y}$=Z~A*ftxt`%U_J!*L{|zxqX#7CEj(ro8gplfGq>bFRNY$FaR zbdgq`;O1??teKJ6RPHd<9I8H>j8j_jL36=!o}8w|3xjUA1aN4Mkp1^09f8bPI|rNx ziGdhgo)j0`2#v59_&H@#zdD-KH;UJMGFNDtg>c(|p(5iKU2frilj6p@*`+nla66Vs zE0=ssA&)aQI{Cq)4%3)8O)%3*$8#xU*hp&9d{^&V*y+p;BrwO?XWFc>K1#_SCBhmya`H3;KQkjMJr^Ls{Z!7r&lu(yG^aYFoS)>b&c0EWH zOdG`N#Rk!qlK9D;n>P{0!ZIRA(M-gRH)WK_p8Z= zYK{(48XA`kzQVz*ZotOL_iJuHIW;(S8uc48lc@^Ru6-Jzauw+LFj5_MYhViO&h zyux3x?ikHr&KKjuA;xH7s4wl{$oW<`!OBjBZK!djKbIi_%W!kLZ=~U)8eeA~wjHwf zZ=o8m54$WS=#_Flx1reS4S+s|cjqyWmp1fv4SQy$OyZq0Dd-}^);n5Ie~3=}tp-4}2rqewC&?Rh0G*{Ole8OPul#7!Nz~i~ydT%IwUeVCgvuU@lyT@7`t@u1e7Xjmm zq%4lKrX``?ZAUjgyCEn5rAp=?eXB_XF9YTxRQeTw`r|$2q2#`#26FUPJ*7k!h0hkXcW;$@uGF0s&vm8F%ictwD4&a|FAqRB{+B(tM`!L;7~84>J|B=_tH{&k_do7@HM%?ZiOkHEi2N28$>Sx8k9 z$vvZvc!bI(hX|cYoxmn>G#`$E1|;yaE0vQW5On0Vvgx7V-grGR@%`B#*(1=*&J1XA zMC*9a+hM@1;m6Cl>0Y;8+}P1^Y@|434_w)V{Ce`P!KzIQz1anVSJYGk4bXz^OMbW1 zkhoA57Y{^?#d@rHJ*Dv|r$c6T>}RyO=9Y(=uaz$XgTasBMd+<~YpLU-#3j~{hqLwW z1Is7JWwO-`C32SHtkH_*hVu-0r3M?5isoywocY^fN9eUu!67mv=ehtnWT|DtCjnR5 zFxsPgEvO}eovXUm!&qBkXuH^B&1Gz`#hui*c$-f;fo`*HXR}}qOZY&w3ifV6HW&dZ z5bYqbkja7vss@*J)a^<&lQ9x0?yG9 z<9-xC5sMmM@7@b3W~-lB?bahRpVT+<2IIZ%vYyek41!kUFGWt=lv&B_%B7PXY;d57 z(>8~!Slh#N^^G;U_(Q|Skuu}vCClB2lP*j0ii|2>G^wG?t~+fvUYkg>niFTf=i~U2 z8aqjZAZqtJd8Urul_Ugxt9*M6lr1abXeVXD)^KEbU}Fyw3b7*k%|JG$wf?5=5;-}Ov zqD#y}Yd3>Y+LD$(ivMBihd`dH+6 z5Bwss1a^8pJY;b{-1#nIzf#SdMa1lXc(^e?e*IHNV>ZO+>4)#1{lM1z5Md7%Z#zie zINO-9i80Qd_gZUQ6}oEzaYXyQ5cRm7A{(Gh9Ic{sy|RM~cg2IVU$m?JB+)z)$1$Q2 z2r@GQyr%>&f|Q}|#uQ-x{AA46sxa?~{+;bRW&RMxIsS93GrBL!_bLTc)E7_@p08gs z9G?dE%0?sZKTsXnWQpy5dw_lucr+xGSSnUQJK47OiS`K09<_asSsijVAf4WMd2HYD z(m=*!lRpXYYR!bMhc`Y0`TZ>UclPN<=Rj|V#Wk9^Zgv96gLT$|d7Be#6=R$R&e3rl z6OH*oTjxIkI{W$G=YL7$tLz?|Ioi7B9v-B&$_{XekLL1{-aK6Awg_ zF2#9Z2LwwUN1rBiq>00z;RvHAf#}O^MZ1bvMYx^12bKQ?TKUSzjIJQ__#46jVE=%Q z_PG9{l5(>=%-l) zzXdhp8A=4b(21tlnGE2x~7K{S+MdKWLqAawVgr>)%vM=HU@3IiQ{T znc({UHK(pN+=RDWNVlTH%L=n!eVi?)6aGQ>{Nt`HT=9KV`P^U3-9LeJO$bfz{C^;K zZ`Vq8P4gL)8HQC!TGr)p$(&c3m#aQwqID+z4Ej?tUq$gM>OXkAy+d>X+5iTyx*&A4 zLf?h?w`;;?`xiR%Bfm4}ZRm=hZdXl|jJj9S&CQJ`XXzWGzqS#gP^1OP3%s5_JBl(4 z^2#Tgs>lo7vfhYpcF2C8NAD^{tEpqm?)GTCby|wNE9cdm47YzRT!x|H8g@xaV=kTZ zws7}Ts|!P#c#11q9Q~9s{wZJ)4Fx4OrxkiA?m$>&5HHhCzt$_OVeYe7Q|Y)9FN4NV zqTz{#TrNXv*_@zK846l6zb~#4m!`5D8EyKm@oQHB73To z{*9r&Y|o3u14!2wfuga0k$?hdh{86t{P#NupZ*maR`b_p{m&Yv>I#T8D-mEbSV z$G?Z=qZJNm{}P5KCHbDf6X*)KKbBOWyStq zT%t3+tK8LhyAu-D&i}Pt>#yykU*qEc;t>C1yE9y#4_1O19PIZ$Sjz9Xe<6dPIGdUN z>s_cZ3Pna|j@Vw?D;4g4O)(`xTvc$NZ}}Qw27i{idT-)IzbIbDD!3k?zA8aP@=7T# zF0S>xzY+LWF8z<_x14ZyR=#lBUugmf2mvGd$KNML^xwRhH4+spq<1x=T|2-^zrM=W ze}-irE`^+SR5t2q!@c}P6Q*FZnoW5BRL^@hg>0UMSg%E}!Y=<17<9q2a^<4KS(-pE z)2~m=Vd=+wzTJC&lDR5z{{bkP&eZI5tLd!=e)p~n5Q4oTWng;ouA^^(V%pKsv5e%_ zUpDOQootcdY;LZ_#^zDb@mF?1!C4#=Xcz}T`bzr2ja~_esOI<{$>E(k<^0^wc-$qj zQi{jIbJZB)FU_~_tD&%v=1SG(-b(%pavuu;`umpfeB1@$lWtYORZ4oL6#gZyyYco( z4WGX{{@x#SO}gY`QU^d!OxKm|?GWvHz_Y&=F7EYg6@i9c-fsS03EPG5P1&jiu%d8a&WP=GPg# z8J{)-LV=>G9Qk^jSq0UQYM-wCqo}kKlh)aH(S{Ff>@ZiS+G`f*0}Resuq(LRaH9gcAnN8`L>Jl zZBAm?ZLULdlngE7kYh-UL^?V;LEoPd$ORof+t^d*7(wU901%f_EO+ycBWC6Pu~^yi ziPi6MvfEJ&HmqgBF&6AK)EUNeJ^SiRoO(x0Jzv#tRZ{-6YgKQyQ zIZ8is>G)u(`r|joHK}sNwK=9PAsnLjIx-zV)%Vgm^jhL0zo3!%3HTeW0?HnZ-f_cL z56DQ2mJ8_QeBh>PbC}`d4`kx*_y%pcbiI9A00Z;uw4M^`vtwy9Byv)Qm>ug4?ZA)4 zO-!I9Ao~ngE^`$c?kMv~C!?Pzv(2%s{&-oQhwCH#sFHeG_=5zCoV}UzKH;qS@!j0D zRtn!@gL78L{oQ!F+eIC#u)j=)iQ6U>pW6_yZST(kBmZS(AkL3kBt!4=9&a|zXaXHA zJEDR2zusXa>@j$dKy$XJjJGV#aCBohm%WdEY6nuJJ+={5l=nHpq0me8Q?l|iX*X}O zGe=*g1nDT42x^Qa#Bb{wWtY{gaxFc#j=FELWUQz<1jevw}DB-J=upF;z8O)ACNo%4xh|mPh&AvR+p@X;o z5;e}lsnwDqUF0izv*xi9g$WX4X~6xBva?=v$IZoLYL}i0rbC|(3wKlaPm5@m4F<_= zdnp_=M}5P@&`XC&qT%^TbAp%@6jA*#=){37tlNy(+*~@FBO)d#^9dFK0yX7q6+`fA zzc>3BhaWI!VZvNS>9RH}GU}irOHn<(dOp~_M{bXwBIw;6DY;>E-6b z(ljoDSbBh}^Of7}bWp3Zp>jP+iE|b)aRt?M=8)B(P*eq0UjF>+)@ zl~az_ILh<_e8ZUP#HS^{MMI0n1w5%wKA3uUU&m#VlJrX)u}jt2Ec76Lr_BOYRU; zzNTAb81m@@KJbUm-9qNv^F@4>Ak#yAQVOiTPA^{mc6 z1($d7!?}DSgjX>Hx^au=tf_VE%9?(A7VPD1Yj-H19|$9lw;Ru6r@WiC zr?I>|KKJvhpMJx?Gpo41t&Bv@>Ed-u&nf~i?tQD0b8y4|cGX14m=#Lhli@@({i)dD z6A}xK;qZ^LZ>O0=Bn9CMw5Dqip$%<${L@Mn?pJxx!VCaz4D2fS;EV$s=eLNx81tIAoCob0P}(b43ZC* zxK&qfbF38%tnJqzzg0J4uye$zeaRY^w6yii?ChKc%0B@Ym`^&cNEhoJUW~LsL`o!2 zH_gMQG)A+V$01@qwXx{dLrk(aJs2MQ8$MeTIYs<|ew)jyp&0iRcv`#)t43|R;?ikr zMO|lyo6;Fh2l^weG|yizBDK#l?z>r50}maPL*XzNf^p{OGXQ$2no#wF+K#Dz+u>5P zanaqAuYp{_T>BT+}js zAgCzv`_|Yuhek$kivqC0W!~*ku3_=(-7Cty$Z@=C52ac&yZtaJwBK%)_B&1K1wGgy z=YMR|{r7qtHc+P(<*%~Xo!bb#+5t`*oQ#8mOeyYo+lnsTbTm)9G}`gD*w76sO!Y_; zuY(-h72^4LY4kkU9;u9xar>dp?fiI1Dk1r>%!cIFm(^hTjg06~EDcGvrMa&HD(}A{ zXQk(NpOu&dEp4CmsQZi*IZk4>-jtBYPF1RkX2DM9^H$G>O zu9$(oNSV@|bU(#E9Nu<+pnj8H?4wc>{vf>Kg**0a^3wk*GNn)6@47f7>-y=oVch){ zCs8r2-_34!&PdneR;??JY5O6=h!@i*ZFaAjV)(h>Yg24R!fZQyxp0+YSW6-&rrM)G z8wR_sK`;o!EhDP~)+fZY-vS#2_1`|drVuqde!5EG)k#R@betpGt@`{&WuH_WAyP=S z>+BSFqWa+HXvZD>-7xb=KdtGrOh}m_qV}@0-C2apgY_E14Wmf$<7FPRtU<9HMrGEA z4}s*FnV^X1GsG0**Cm=5?B5V=7y4>_d*VBCciO9deuoFrM5p z^6+=PI$ttXmRx|3GD0SNrJw@2NJFl?d#hj57m0?@q0i_`nO<-m6OTdF9h3YdJABI2 z-gnAmbKMEsmC7mvb5UN%?#aEmg>+pI#r(tdE4OT8rayz9G($?)kt56*)PbYFhK zRpPR1^%Pp_4rr}5zl!5+oC%-yJVnaO8ecgwwq3C`PmSZ>rO;k&w2jCtkxC}_YAXd2 z+y>ApTan_)6H0S{a(NpI71VR-s}ym@ph*Xb+vVnJA>@({?@cfWwjp;%^G{%+s=5&= z_hI2D4W`K=d!o9|_7){Gy==U5+0&R@9_yBi;~D*;o9_D;w>fVzDPDyI^{w;$F`?wH zs^*ArN9327Tdm5<%E^+WV^2@$a{b8BvBRXSAxVFb!=3d}^XeOiZ2#*wZ&0fxqBM6_ zGL#NQ$fWF$c$N^qXuSt7)r}dX`d`PT=V5VWF+K;G##7ud`KpR!=KEeyTD4f88?h*V z;xvQ)Xx?e3Y;aK*I?9ONk$B>r>Q8&mF<<%5VQ5~yr=D8_%_nk_ zngunQTyNJ}Y94D9^QwQDUx5jxYKDRhJ71dJ_v;KbsUqzyvST-Pw{MT-ccY{mA}CfD zURo0=enu8=9ZBd8#;(}al_RsjfpE-h@6<|y7=|=@A?LykH%a>C`toJJy;oaHViS9- zPiznL4m~sWOvLb=|i{|tx1S3m2*^eiZZIsEb3`6Rv z#5A1=KOD+0`HI&2>)S2XN^(HA)pqMI9Zwtl z=?x*ZyswrXB)9ty1RPHmz9|UR7y-hI*v#RuW7)v{5l5`dyk)x_l=(p4$;m9YPFN-qf{hB`n6fc-i>Zf3Vn%e74o{Bnz~r2QMpF{m;Pil)Eg}O z0iu3qWz%5fl2HeEE+eca2EqQU+z(Y7qtVaRMx>zK!as_DCekUI45nFXbl6(VO8T&> zUv{V)mN#pEA_*#W({kNo=RLRwh0g)RdffCxY@t!Pw^Tic*WKw@=Wcj&k&l*AF9z-o znNPZs4ey19g(Ml$dM+sJ?7sOiH*Bx+An$J5S#z8X(^&@FMc=b_%(i|t<|M3`OeX^u zHX5Tp@+7By)3DNm7`AkWl(5O@*i3YFuPe zH>pJiQHC~Y?4R3Ps)xaC=zIgJ8&g`z9+;u52e0`4kk1i#oZ`bdFD*Ee$$Ot7MI|ZSl=&N)IOV4Tgyw$*vfMZq`!H|gaW$i|1X{RSf z_f_0r+SPzF!`ojzG=i8jKSfFNyj*vb5BmBXT@cgs#?ThNEAWd45d^Ye-HmY>@O>-i zWU7(opjYMzQ0%R8Z9+jzxPHEkPs_MzG>xd^biSVRYB;klSVchH1SgkP@V&<<4x4{E zW%uw#X8s5!?7u)kIOPptOE4M#)Ms&n?bIXHBX}ceEL^f$ke*Q6g>e-0U3x~}k|G;` z+2iGy9sKr3`(RTR=`!SyZdnxb=bvzd-$P0HTS>wzp?P0)$1I&cC`=w;3$OTQK=SyP zOnf|BJV4bXU*L%j&X=LcjPq;kGQT^~hE<2II)5-!)T%w5d8OBLF6Lg^OM1rW_Bgl* z0&lKDI#gb|_GlKDVrXaPF`$w!#%{)yK{+so@x^*Clm0lW!5nyA%wQh^MPytIo7JL| zsCPBlQ<=DeQ~SS%rgbRRYLZj(V<`hSL1-Sy#-OPYsdd+uh+@Qolf5au)EwsF8Erh` zv;u(Fm{u$!QaV!MET+cv`NJWSas>ZXb1`=>1cq7NtTA}#*ElNRcJbDlLdyS?2RN&p zWG|M3@Ef%SKa0+Yyt2S`iK-tv_58w*blehH-RO1Xo+ssjSk2&8m)v1ru<~BFPjfYc znJ=kL5?CfdOQ%J@p1RO(-ZaoN&5KX{l(XuaVB)&}W+-R9u={y%ZNkeDAjY4ZFEott zGIwG9P+>cE>IxexBP=3@obXW#@@HQ_*}0^72tdF&KXN}bpmI0OVCNIoIiIfKazhR7 znH!N1aW66(a=nP+-|dfxe@OblyV<8|OUA)-p*#}yCgLb3R7~C?FHB`=J*d4@Q|K_? z9@@Q6_G$X9m|6R(SFVeEk>8G?VZ)a7{(Nl;Hl>i-Uqo@gCmfKzh_;)@T;%&Qo7hzrx{MwZKV(;U3dzF%WF)`jfQ$u??lwn2a=H!y!h6hWGB$re+K=yyutRl|g!~ zAqj>wXZOHg0^T=ooyJO?O%zMahJMS-CWwl>=)YNgW6K>naHCWU(ZI(O&0j!c`>$xGAXb?2Yk z6SX$I*!gk>Pc~DKJ0y2s+!=5GF+S!SgZ%U(9eYsa?i0IsH~Ptaa|?f!$Ax3jqJRnd z+b}idtTlqg;jzR%z$I16M@TL7eD!A=mdstNbsnwZRbOonn{Mm!J?9e9r$&HlST4fd zZJ!vD%XM(Q+fW<~;i}Oi{q60RClaDmRPN%5>?0L540`=L*k@)omzz@>Dvt2G;?gy3 zpKm6m4_|MeKW{BGfUhjC{7N}7iB zV-ww-DtD*+ao4>wM#PYvd669FS0Hs=ZReGb-@(_`GBT?Bs!zeFin-4IV+ILRGb8ab zyNaO$MQx7VyxX`1yd=zfmwhDd9Jhxs>Cq0F;E8HeuE*^JyJE?CJJG)DKDVW`el%HL z1GXq~W+TZFu_;l>mOPgl5JZUA+LlC;mNcMwb(~!j}8ZSz*WZ zGvEKm)K^Bu*(}{+L4reqyGw9)cXxLS?iOHhcXtaOEVx^6C%C%?2<|X*hxeTKeD~k1 zSu;=9(^9>wde?5Cg#GYg93A_WP&zH^oEjzWltFM(m|2$()MAR33>5zqy$OmCOc@{V zGtabFZUcE@YzVQ4wzy-)i`EE67290B>~0}#F5m+ja|iX)y=czfu*+mtCm)V|w?O~5 zafSzei2P79OUv^E3E!P?3z8s^s3mVR8Bw4epNR~7!l)}&z%5hCn@@ALOXjrIa@tW` zXB$8f6uKuBFMKFn6vEr-4dzQjJ0)_N_NP)wGAVb5YDWuk93zLonYNLB_J61ryl; z@dmai)zq}^@vg@dnY%a;-ZvAmr5`z%4R>x1{Zb!Bm+|_>#p1(aX%8@1N;zI!t;eKj z7@wMFvEMs(Swm8n92N=Bbe3SH2kvDyn&9+zgcmEd+@}BD=SQXxydzdRQ+h$e815&9 zeT3P8D&Q}ArgVX|hu*+T18uB&LUYHDo}FJKsP11o33x(W52TcG9pqm_Ddu?@Oh0(A z&d_a7>@ILa&=PDu?3Eco7!+f{z3$_=f&xPN!njvZ?k)J@zU>MZpcgrR+~zne-I+4jOkDuAcrt5N3-p@qwRqDO0OOCl7=(>CH zbR51`imMA@&y=BeXxD+@GdR)QU;06@u6n)h>YrdeS2}jLWaLPM<9X=K2}Rti(_lG@ z%%#uN^Ek*9L%W4xE%)&V10pr%V@BCA(#Y2NXKid=d{sGz*s62N^utPcH+CrYlfTc? z)RxGA^~rRZ>2krtPtA?7e3bTcioQp#5iIa*jTxgwG5uivP4HoH7c9|hH9En*3T(9T z{S^}04L5E?wL@x7+hVwG0~WvaB7eB+=UQ^)r2roMxL!P$FC!s7Ej*R`#61tuf0?v; z@j+49-R3C6g>xph&@#iBUi?RUBm^~}d=x|O*Ao3%l#6~OAxu4}vT@3#^)LKhR%_23 z+y}_xBqUt%kUF_gxG%j=32@6!_?gI6Lb+?rJE5YfIBc;^MGpK9BtZ(w4>|}+kLzcm zY&VCCe0iwGt1Tk@;bMlj0f9Kl!A=?Sg?*e{YjsGhX26oWrHD43R#;ZWEFM&!ogPwh z;k02jwTV^F^#T#?9%5aqF78<5_&=1tr}dp82Zs1^Xxb#=URL|bUkW?t?1uysnayD~ z)tKR>G`Y|kHH7e5s6C_$h)&b|?4GwMoy2qCR_Kh|47>Nc+H+B=>3)ogJLm$>8q=uc zTwRkXvJ(?c7PE)qiNPhvw4;Q-*NQ28zi}CYYs|wQ+tRko;hGT)1RwrxL|X)u?K3fT z(@YD2I*Ut7n-Opo3!wfA7KB<1#As;AmfCjlSuM`Q zHjZ9q;R)5rG`l~u{Aw@ik;K`8eH}`dbCUE)iMaBV%7n0gHa>v5&e;bSdN>_36L%T( zhOqc2TxPcXiaD9s{gE_7%#A<(fCSsVVy>Z-v0j+zzy3 z^l5v?qHXwNZ7gnus;OB`G6orjPESXIaG&Zgt$(Ow5i+9{y5rK%y!!A#mCKOKaB0D4 zBftt3ue8?(rrQBF$G8r4}a+}^Rv!A z+HfzGm28>Fvr^yH390P#ng)#qH9St&Y1Bi-h4E)lV?|9>tn0fT22&r6rlpBDYNIfmfv6zZWmW;9geVBOQzLQx7 z;E($$xJemFSL*hDn@BMJb?@m~Z?UFV=4%7Xqtb8yFoYCg3TqFRYOG+Ve-L$=2G!F! zfiujCzaJFliXZPvi6Hp;$Yv=dpfF4^zhg#?6Wu$Dy4hFtt%oOdlC6YScXmIva8U&o z)u|#uG-E()``rJW@L@#W%a+OeMS;8LAj`Rb?AN^I1`fr<(&i?p$Y@L?1K_#G;D)a? zv-5k&4o-fvTjDTSKUjFZQcx${{k-35AYpIrx7drt+}P(!Kt65D)`R^SW`4ZDW#%j~ zR;<^VN?EG$pj7n4Km;7xh6kvHlKv*6n2^_a-)(*hh${E-Q)0Z6mpq32ahs8H* z6W*qVa(f(ZgAa4Fzz~vrZyy)*B&if1mwm#dZF;sv4l=zE2h-LLdb{U3h|o0w%~RRQ z55R8NvDV>e^r-xWWo^FI)tEBvGsbphuK~2D|BKWR?uGUkq^mYYy0+KHoiQaWv$X?5 z^6kSW`h#}9zo{zAH^Ay)fLTNuN?QMLUzNkBcGa_*az4kt#?PNx*|_Ws4%*BD!Y@CK zf8bV*(4@%g`)NA*GSr8jMW2!mwNht-JY|)EdS(-oqy~)`9B(xKNh10_Q*a~UEz`!e z^oyQHer|^C-`D*w`Ghq*fms-HDZM@|B@GrBOmC3K=Spe-CW1X2e0*D?KdNque@~Bq zz)kR`#MrsKPCr0W_w7!1rQV_;q=;PKeIXSi6f^0VtJn4W!}%Kh&EaJ7%&iv!x%RM55I9JZ#=zt$JW1|28&9@U}ZA zG$`z!&l3VZ$mC|4D&nOwIR;HW+G}biD zK;+~Ny+yg5xMa&|&dD1OyW-uOt)6aUcm}{FS|_z27Ioj}9rKQOn3AmBdS@rp(@b_7 zf=#iWCA8C%YSUPXA3eQ(+qL{J#rp2MVeG#X=YK@5!NV6f0{U)(%sT^udH{&uQ5KDgZ z2cJAP)EF5nGKN&(`<(lO<}P*##r}+L zFS(^sud443c1`$!NnT90;N0cMPH6M1Z2#b*9b0$zyI(GfKVGlWh0>UWZv6<_*$7c$ z_ekR7>n=NAUyShN`X#iC&U*LQqrJ7y&aU!Y~c@ z&v*{2Uhn7yMf5J6Xog{&52%SVFh9%qNTwi`80NK~MZ(hEn+!=uyv z&~<@}3f!3);?Z+-Fizd_y;0zC*^ijWU=a&LCZv1Y+m$E6=W_VOi0ri3W53496M9p9 zG?i;R@fylJ+)zcx=k#h~S*q_k$#H$Z=C~(9pGsfb`U{BytTe$EJU(BcHi;$D&Apo> z$CT>1>8@yK?}IY-hWZB2q|Vcox2!9( za)jwr^R(GDT0Qywe9nXuGm%@_QqZYmK#_p-^C0Drx{~N-E-!O>-+(*90dF#EU|e;+z%4{f~R)}G|*>L(Fzj0z_P9RtclbTGSBlt(M4v-3tu zup7U`boWH)Xc;)r$(Sck_mupQb^465Hg}9dh)k`Ni3~2x@o_Lun08J4%JK07;doyORpP1NPd5 z71_FhSk=UgcsmksJ9AV#dDXE)5&LvGLu@+jVT#)vUl+Svx{j`bl6ejj@3@=YbV=9t zwi*5tAM%3zk+g-MvnQ6Orz4@=v#4X--KOBC8g~A|x5%%1ZXh=!K7 zom!&Z+-%(FNFuqO`0Ye`tkawV7cSIRn`?UK{X>r&#uPbK-_hYAu0?RLsUW|oYh7GY zct49|ES^OQMAqxgwW8>z+8tTk@Hji!PpUyRU>qS|7~uya_HMX|c#8uvzl%+D*00J? z@ICPq$O^1avpGs2=(Wc?+&|b7+rGjO4cndM*7c8TO0u7*KHT_UawPav_{N=kQr{E9 zLgj!iYwrU0RWvkI)B8?3xx<{cj{-Sg$NR9Dt?FH5jHFeEMd{fuAf*1_R2 zu8wxPBM!)o7_3GywP!ttLY!hIg~IVwY15SkF6M-Lq)c^vDc$+xW(PiwRhx;vV}_|t zBtVVq%P(w0_ux9JFK@kX_rG9ermm&PIlk8&ZR?~th3&&c>i*94+@xjdy2Y^Uz%+lp zwI)Kyah54p;=d|h_uTTcw6wbq$}1!|TC*TokuU0Cb_OR(=G`n4V1Q(m; z1AK}aXl)gik;Z{>^=tHfy&9(B0;dMa$$n8HGHL$|#H&O`ITE#TKPGyN*Mz_4wCT;M zuIu(t*ed<4B(dEo0b#~DOpsUbFrD440|yN^9v1B4vp2`S!_L@Ma)p01v#3SDcJ5WB z6HmZ}OW{H|@vz*}?U>HP*}_BR+k#{YI|jC-JUk!D6?*<+2G7hW9HlB z@@poM`{a@?@>e9F%oul-3CZ9t1;5hV4DqmT&UFz(6k>5Y^>!Tcil0)v|wR zWOS58d|Kzq?_tAM{si5K2PWcq5F2D?xjqrTw(FH3$-Nmg*pPa{pPs^w6aIV~ zwt7W!=b9&1w-l7d?4*F9@HbaYk z?3Hwp!rUlI6Bq02{{k)~N4FZ=vP=84#P9}l)KbgN_NHlGA+rhz$><3_U)*rHcpzen z3$$zL0qra~Jk?Z3JF+*Fxr#`^nK4543EoVwiN-&!Idl;E9Q&hSzR_?yavK&&2W@nC z*783eE5+*Mk~kB-{;8~G-DQnGgc*wNf97AB2R**tEdRkUwOqqC>YU-8`dyV5-@+vD zcI!BSl&|-CyzROZiv6q2A+buk?pMa^iNEDS++kFA`=S?+iG0RaqmASp&LZN!ds76j&}P9ef| z-F&Cyt54E~<2+p`@br3YY0+sVw_VH~``$o2t-js4+9?IuO>d8zby2XyNY6f5`Ej%s zMiHfD3y}S^*5fb4zp90!kjm6u$VL>ILg6zl@DLzPe^4g$RzW&*TH7;~VCyt7)6DyE`DG*(`%fM`df* zVchb$z}x?k3!$MLM!E%jnemFl+38w#Lwq8aA;MZR~`rQ+i%;Ep@n*R_Xofg3y%9n;IHMal$?mf4x7y@Q$ zE_wS=+Yo-1`1b){p<(cNj+>@!QCEdkCq=BztF!T= zqJ0fN3X>9`rsP4X;6+87r+X()V!o3)8H_^qVjB(!r6!vDM-J2x-hRNlalz0zu@*bx z6IYCS@)8JS&2ALTKED&d*wDU$DK^$awbeh;&T^75z>0`#t_Rtq`I0IyQ+zOw--~lF z+f)O|zpwEg@2J_wmLHbEyCJ{ZqL&*xtuv5s_!Q26dshpA;CMoaP6zagR5;>}X|ZLG zll2CW21vnDi)hLzI^^x|V~@J}12leInNt3TPMpI49xk9=LzP^7BE9#P$kWCR)9BL0 zQ&O>2E;Cd0mCfGLQ&VDNRxFv*6nc4eHM$VZfT5t_^{!m4-1TPgVd~hCKJ(}I-f!oJ zI+wZ+2tyGW9ep_fIs$)=LzjG3hT+A4?7masURe4MtpuNBrpDZHzdzt0mP7hKHvDIR z9MHbHm7^T)9Z6y^X#6`EU~cG#cZzLkzlVq}o5sJBVha7;Y66+>ga0F|g1NsW+hJ!5 zF387M^UaXR>z4K;pIJLdH0al$T-}O=ww;em+A*t5yfyMjT(NV7UQuJ|`wrE83}~~&*rnT>r`#5ga4iH=8zn2aJ#GtN z?%)$KhvzIFN|99!R7j~==Sm#qV{Vn?Q(8m3uI?VhZ1($H64S<@f*IBlspY_vwvp+b z)a(JgXV7#7YnOsq<8Ns2$ngoag0Gmk8QkJgh={_!a8(Uetv^SOdlcDx`%_``W35E@oS#yRlyo*Fee zK0Y2y(y+|ryugs*`Ke{-bv?XGj57}~{yG%}@$Ok{hL!;D;=2Y|kn%|~r9w~Vam2@42TR2y(V@9<5S)mL`iz2)meb{Gx^QKCooVUmQ$@t2#^UpUMX!%QCRyzZ z=}01GaFL%3lnJ)9gbwb+)sD*wa*K?jX8SD?BA4y`X2xB3NQi8Hxi$hkk~l8;zPKcB z#1`U1<;0l)j2tY-BN3Yu$>KdU1uL)SwX4!dCw*heqd3KgE6>g~@Dyoq*yGA0mECLDKj`KIdt&Zup4U^HkH z2oB8UHUa0;7!Y&aJw54PjIO&oU)PMJEx*zEB77SofACW=1a3^1{4D6DfoQ)@IqK%abvAG6oVK()P-h8a8 zZFldK{J`SiCx-dh&RYj1d7?1b)O|ljKZt{#xr4L%-mjk0Q#shcMFK+^m}X8((#U#89`shT->5=F;AljK}*OY*GkR*MlI>(}iIBs*S|f7NkX;gI~j z{BxJ>YEI`ZGfVBF1^wMW78!-~yVt^JtncnU%W+S9-Sm}3FNYWqmy*#3X;;~+@Gbsl z-C`P5cU_Ct$2nybxJ#|b^#4Iz31?pDN)Up~reJH!8;KMAATLEnn>RD_Y<&V%- z6}c+-ZPnOkxCbwz6PP6ix96sFv6yFi`$|tM;|5j({LFS^`t!giDv+N8fBewpKAlY2 z^edtSE2<)Roj1rk8quH*Qr0^!sA{9*;cD4)K3Bju(!SfQ;3_0$NpVZmC$K+c%amM3 z1nSoJTamzsflEP^^Oh*_Jw5dtp9b7AwCyH+@M?GS8N=P4UyJJXjuhq?Ligt$xjRPx z`X}FGKcEV@Pwu)r$?&kwv;}zOVs*RqdcEhM@}1fi(z|VDnclja@jUPZc_D6Iwe-g1 zGnFYGW)z04ogTYcJ#0Mns}qOV3X&W}UAt6Wl}T?Me{Flps~r#3GIEs8!SWH}hR)+n zWRTE~ul-Ob^f(f4K6ds&()sb#{{oLP1n!w&GyhAOiy{fgl@}&I{q4ZFx)Hx=b_IT+ z`{%qdqI`S-NhUuaMxH=Ow_b67WS3G7x$fq8yse>3#!H(fq&(HXr)vaW43n_QtVlpC z`XThFnn|f5h*eof$D-4|LPIUlnBd!?i=+MLbZDnoymq1c?r(d%Vw5Q#8$tqFb3Q+s z2*j>7|GHcAoVcM+q5E{B`WUtuikzlV$Rn0t+(ls(0xl zL`S52(H{>s+Z>C>mg}oX=Ke(eJHdyv+fk=QUv}HC$8}D3Z!jNxY#&2UfQZ%gL^p)% zZ}*#zX7KV7v?S-BP>gOC^DDt)zkua8_CY>toOju^%YJ{$7PBUF91m^y{oc=96?fYg z6z=r5i(J?qK8_JO{T5;HZtO`i`I`VXiD9^ZL|Ijru4MV}mk{sIlpQq+^)cZ~q|YRW zh(}XNkk0FWcEQ8+c`OdBUv2wfNb|u2EPN~-`FBa{y09?ZtWthDuP-SniE5aw!F>N` zAqEx^-3O;%V720=eQ<{2gYjw#0_zuA^rsKf4vKkpE6s8!1d4%J#Ky3YYCnNxU@xBg z36d$nyUq9VqDLgNmIG$Y-Gg_F0dVtq&u&;oLO~(5O^Gu9@HuH$w5q6wNl;tqC}Q~K zHp*%>7k}wjp+kw-lazj#gyZXiuM0l$(xs=JHs#tPBiE9bIimu^>9GzfB1GBCx6YRAuZN#J&$3#EAmaKYt*ynt*8OV1`jY6{^h8~G<<+8`2C2gX@ z643=k<=3GZ?^l^I>3_0#x)1DdOfCob$)l)$T6ElG?D76eF}v~BrA5Bw}&@16<#KQbcQ_Oxn8!irRboz#TA00>WNYlzUh>9kyMiT z#D+Xfx%5JPqbRMS(p^ma&TI5D{Q~K>4yHZ3eR_{w_9h-{QVF0%iCFA@vgBS7Ese_e z_G4mWGYI)U3L*?Nr#0a>j|}J6rge0C1`&s*?6DdfutdbgtN|~qh$BrP=%78 z8p`_nm~@}-VpD{=UX?Og+`q>Va>&pi4EK9_x;EIEmfa z=)B{q3<{mF;B!tO1?S!xXjW-YdJMmzq(Kose#l8OJ!?C% zGYTe~NrZR%K{Sl(=Rh=Z>h73=$N#keyuo-oU@YFr+8UO9p2Mv(A~71n>P> zX#^F)6_;}`Xu)EswOK;^dJp%>9`z8NstG zr<{CKn0t0IlV`Fjda}njtj!2%xVuyB$*H-5ZM5*CrIFo+<6?x5*}0)kO$DKX5|{M} z(&IX!e{x&;=9iwAz44(;M?L}|D{Fm>MxXdB=zrk@W6V1q7_KtWnb5|I%m_4};PA{^ znXIHab+~+v(!k&LGa@lrj(W&80>Yzs>m;#rZx*k0)9W;!Et7NU`-DE(KMZkdB&QTv zCnaT3T|w{e%eCl}V>4wiuw3px3>&V`Cb<;uqT}o@x15fsgZ=@g|3qO_@8L3`7J8C% z=!(Zt!2#Zx%mHz5W^A^jmp}W&?E&}5EIakDw=Pt{RH3&6;$776 z^a6-*F=BELctUq3Z#45&4g%EB}2&md5CYP57GmQMCDy>G7vLUtLiSi42W&hzoEo=lQ?9E*b5f zGTG7*ra>FBTPu55sHC3pIkgJr0seO*1-LJgaGwXDgF3?_(_}8SdDtF}9qtPFfq zz^t@fFbN?YnE{mQ>;_9atl0-SSkRF-6E-aV#~vX&vpndBznd3%7dbZq4|x~o50pha z#uo)|)vj4Ruz{^n!t4WrHH-^Y<0Qst%<$q8+bn;KXE>FD2n@`q6c*ddm1|;B7&P=T zcqnIoWZ81rNk-W*$?WVfejZOx^!0X;VYYEP+wfF=SAEr>OG~DxsriGNKyQWeWOjfz zn%S@R@$(2Z3JzHL8u9vQkprRyHeHCQPd76e{FDW8u}=g(HjZGCO~%6i%0tYPw$Jk|o_sd`EXp z9UZB=yMY+n7fiXlz9gHTz5`4o%DO#E%B+Q(y$R@hy$;M#qZz_mkDgYrzGm<^rl6q$ zc4%A}B&fQ`-e{Bls?b8@A5X5?}d&|z4kgMA(Rg^W=m2QeD(nhl${&#;kRxXb?E3|8h z(;B+rWZsqzDk(KSqRJ!EFKLHD|CxGs%OPEMgwmA)qwBqJpyW%YixYx4(nKf=LC zH;o+>i){Q+?bJ2_>n$Y;X;y0)K}?xsiI6aoqs4M{%lyDh?_qA>3=R<4#)U$#!-!~U zi4XbJSED*ukMS{^kMQ)dT(hmy0|~yhwxq7*e3?N(*N?1-T3)TJvzZ}V_OJfR-CYQ6 zL&@6_m~J1pVKLjd!s-JwjTAoXW>SBkxc$;{p0)e|W;#{ zd;Ri0Ufua-<$8iXSU5+7B+A^8d!GAdBO??C+!vsrfY{h-{T^pAMx!=52}JCKt0LHO zqcu<(lJ%T)HkLxlBdE(N{}W?|(zD-!M(I=*Q-{YzXtBQ-ohA)~Rl1K1?D9=EdpT{3 z!xof?=Lbkf8bp9jspeyZRD5Ax3p&0Yens>QxOfD}EI(z?O2g%_i8XDPNr}!5&yLVt?g}~Vd#bj3sd$G>+h0EANc4=3{1%cO z#gH(+Z0TAk1>txAldY!4ee4BOS^D9m0ba-0^hSF9cVBBk_CR*-0}JI{AWls0KCY|z zdw5nU~$_tSyqo8aZ+4ztTvcjPIy>)>cb z%@~w{5c22k*Pxv8pcK5=#bXO^y1-hhVtIMfr0*hX-f-?0Q@XVRemTMP(B;R!Wk z+Z=L%mo5@iZ8St?=Fdx~HqU9L`)8t%{kK>6*Bp;KpYHDGh*w(7vrFfz*@Kbr)NLV8 z2*+}UR)o7Zg|@d{slzT25=C;fL6K<;CcLsGJ1i@(55j>ljOvXJ0hSO>!odcD(4c-x z-`AOY{z55fK%P^c&66C;F=1Bk8$7Y>YA~KGeP2wlkl;y8!bpY|x;%_EYWh{Kt$whC zj7W#gA|;^rr5-<$QL}0#9 z3~YdaH<^Z>(4L+xaa{lO2~Tvgtuv=?=d=J22xIYHtFgR7_pHZmu^jf+&5Tw#3aNQ;@!*}rL<%YKV z-r!u8j`FY;E-x<=1}^_vG&2ZMENyo>j3)o`SyIf5|31=su)qK19)Xb3tQ6@Fr2NHt z%E;J35DWpuwcGJkpBHLS&`--X9H>QkvwsE_JW_#H@Zw@;`F@YhMl5HPo{UPgMx_~w z5>0+fhFhZ=pNdH^nH~0pN3;1lxG5XU$1>0VymW6A8wakh3NbS)1GPJFacd849w(Lr zcxBsE2lt1n`65CU;DQ{Ps1SUN3vOr*3uuuOUzN=3_MO5 z7>({^>J0O{yr#tDa?8a9?i|lviFx-p6XNm!+9&clKAo`NjJ9<3K(KhGmy9n-uN}tT z5%{28X~0+`cP93dph3eA z(AAa6t_yg(@F)E#0d|`T{E&I7RG0t!{PTA*2jxLLQkv?52Qk9(`nv0^|l7@;DE5Ouv6O~^TcFa9Il3Zz34SG@O%h6v{0 zCk9qyhPmz!*ZV1Fc{)xG8Qn(BHo*YE<)yQq|M?KZ)AcE7Jq}D>v&YoQ*;#RZX#bl3 zLA%fanEf;UINETheP5NCavUyzJFQEdJ%KQ7l?|K|)p=c`^P7Yr$ajDHD8pYvQ`i&F zK_mds_4vy}0{}qI@Gi>+`nC-Wt~^Cv1muCWtN-I6z~e#=Kk&m`DF{H!`4{eIrB6YM z`P7+?`NJl(Y@-zi?QBu3$;v#DEdvXGb0SABF z&1Yp9rWi>kj)x^9bD2in%KM4qSdukGA0fU5kCL*u*c?sfify$U3se_4_nM411>ib` zPD8xG1_cK@uP^^)=*_eO`?&Xkh(Lh(PI^zF!ZMff{rWofbT?FrYX98fcVjok8g#;l zPR?#}^W@-$K&ED%ZjW^(;~0vn3#asE*|dIwexALz&Xvd@ zlyxrpRv-;xEbl&?@fDYbCQ!Lke7J@H2UmM-Sqn%-YoUU-m<6c4;SBiw&X;Mo`23Qv zfw&dAb&Z1Dhpbf~5_!btM0P}ehphh@Z3-1gqj52&?ihv9BFK^j6Bg;7lZz<>=9^F| z{76Yc@IUDSoDbPwi5~obkRB+QGKN1}KxcB7MUD0BaXBy5?*ORP81|4xVo;^T7-OuH zNX%h^uqdq4x{5<4ah`52HljG zJq-9n)+`9dtVs-VRHG;exmV9ibwoZPH;E?PUY9f&XZ%5B{wNBi{bcwp4q#=bFGi`y zxOqoWIB#<4wd3yMsOxR6!yy}k15B$E=nPQodV@z zUR|r+CX9lGeZhD{<6;-4PHWql&6LQOx1Unnlwfs&c()Jvg`JbouY8QUj&cVT8F4|O zdbS-fG~wR3)GrthmkCd`Tlh&S8M6OLdn0NRVe@g=vv$tAoJISmS0?X)it3S<+eh<7 zfyEX}R-&o#V+f?$$sZisES1jrH!h5)?#xe$I2fQ~XZ>(%Fe(xy>Bf9v=Kh=1R;$+$ z#yh5}b{u+~aZtI~N8yQN(j2Gs;9%zmXK4st6C{_cro^#$AY!C-N7oXPBu!Lr2YuOV z74B>4;f%Y#Z82`oI_7ZvPWL3Mi5y4fcxQQN1{gpE|MQS%A?cdUR#J}yh!J@0NN%Q7 znEbpO*%|UlLR{F>tRlW4=B|-FiN9us_+U`v2pb5`yl#;oZ|elAEr=Zdg3V|k5~7qa zU#_b1L#u*h3k4|BY+5^fW>~C_CigJQ1zm|tnMVr7P{E~x0H0KePT73_)+DEOl5Y?A zrR#zv7yg~-ueGj&AHIMfp>GmJNCZjHLYx+B@;wvuc^c&6ySGM%7}hk%wU~_zGr3%Y ze6LmLby`f4x%r#-Zwi{HzMnU!#S%7o%uiLU5?k_0o`?MUodyNh^v%xvaU~weGf{bU ztP+w1o#_*7w$e{Lf|AtW=qG~ zd@hlj2F$i-Y0(-M7d-X}QG4d;Gh^$g5f3S(x#M$@KWT}a{61i#RcU0#iIA$dKXN#^ zUx47g)`bV*cX_d8h-`D9fsve9`=1Tls=G-~4sqb5nT;en`~V45N-^zkHj@Q@@xRJ) z!)ZuUt?I=OFMwj%{m6vlALezxq}NlCce-0FrUSQ1>?Lu{fu& zTdx?_la#?W*s0o-E$6eAbbO*sKs9NKq%d1$Pa*_DzNm!QZU&Xu(BTNU1!ST8S*Rc_ zL6_v;JCexCFVZP$cY+lpSM$$;JBMJ02SZO&gg}}B!L&t_QjY%s{8$K$Yodr&lMdgz z9~k#!p@<{b5)FBAgKjHbFUU|k&52-PCAp62b~#RmPDMGU$28b zW8i1*h&ka}eyAsUHwhJN67Y%G>x^My!tNh52}Fj(N4q?A`dPweS10=6;84Ml$NMnH zccwdfbvu)}QoaIXB<=)8a=S1GGEva|@QaPMm(6vhoK}rB3aU@YQLaRqoDFQf`2@^M z$HGcFI*)WFvQH=HjIZR$dbca~iePOd`IuhA0H>>6u|l%!Vhu=bwp5XlVbhl4e6@LI zBF_kwAWAJ3?`o@C?&fJtf<%@-V@GI48?TJcVeeAf~wZxazEf+`dIc4j4;ZfBC+7}~5vekU-8FcZ9)33DJ4jWnM`f7hx|D^1BiVo06Uz>_X> z@ayFdk&Uk*TnLP}Wu7EK(}NdJJX>JkXx214{O_O!LQwuqE}I%ta>bHMk0TzpQ{jSoboAgmUQ>JPDC69-%Aj)bpy5XqHy#KJL*erZ;BqdWmNK`bDvLTH?oc0VH zIV6UR%Mm_2-#&^G5E7JFHp#l6V(=pf-hADk$kQl|!7r7UKn{8M5FE*UaX{JhLnG~5 zQ{`wCPjrHD^qT}D=^N*bUxp=GgaXWWG@ocOSv$tj3H$#_oGJ)c^r1#tk(|V{`8R# z+gglOupP>L^hD*}(wIlet4y3-f?t!~Vgp}SQ{Fv$=U5czx$!t7=qp_pbFPtnQ zK=DAT))$i_#!QtQlKGEiyNa+~%i?`tyCRO@!I=D8L62@(5a$D!NBtw{nyT|}MHqFV zLWJAu>$vteDf43rqT79|NLwrQ4s%XLfMTjZ(}%_jxS}LWyyY$H_#^LIMCo+K*jJc% zKO@k~Z24gLk14B22VRgfBh{k`OO-h@vhqglV6FVvrdM~z?&3k0By3k}lT!oUKPd3h z5nkCq(R4h#N65yypsls_*oeri!Loy*(8|n&u5=O*;*TO-YIa_hAGm$vD-^P%GfWEC z_a%Et`4poYW2{WXHvbVn{C1kePgN5})d(mE6Z|yUh-v8l@2T3)=6XUWV zFDR)Hpdp7=hHVdy=^@^sly26)xG?Y-?dRfe<`6+%w~4yesDR%bgA0yB|II+YKgJOr zyw*2I?i}HD*kb+%Zg%0JR3hH|f1oK6JYh_*{twe%a=r)1TZiVBcK=1l@8+8l1%8d_ zQrRDVT{5<0{}+(x>G)=G2wFJ~wE`UHNaM1Acqt zcbLt>v=UwbFHu9VQv7YRvftm+u1XcnicXF&0)0OSQ)c*W?$C{|nlC zNiJ>R2Y45dM1Uujv8U2oM7aa8PsJk3xIJT19gCC2qj33hG#P=J$zgJ<>a^y6zYCd< zI?~6?gKV~HPb~fomLVgDEQwvq`Nb!Xb9m}y=fau>1-x0?Jl@j2Jh_Jw@Ve-e37@%& zt~!EDT+8nyWU=VKtTb4N1R`if|JZ7mKWvM8^%}h@LtxIkZDReTH0`&;e-1+36MSg9 z)4y5z->W*K1!||B$uNWAGet>ww&=}KvjMC1v;NiHT`JGqknG|F?p=cgVrf)#a!G;s z3+E(7nU&_rtWQ7+y@7srpd^XekXyMiI8-|mpb%2bFY<>2Y$*I$s-e?s1~GsgT~cRT zz1+sbnw;RP{Zcc}#D@xTJB$&9PfglCZ&5QjZ414O39C09%ke&74RJdiXDoj=)fjTC zD49!tQ#pNZZ3OfrHd=2A*ISHIv|MZ^Gx(n;iIDrp1RGWBB#CVI1VSLkb2c}--FdI3 zD20Pi<(7vmi%OG99CU=GK7=f3#~}Yznm#xjE)n0(GW5(@%9wSF9e&S3m^rvo^v18e zOc|skbdu|^S|p(AaVD1D*^Y0Dg@@i1Y1O=_VH5?;|1q+xfS8&*3(A^35%G1koTj=d zjsIO6RzhPvUaMNySS*t)%Ig8rOrlhvt0=s8T&e1tn^fU8a(L?AO8++1@Fh8$%aalJ)}9HM*}O4_-%s+wthPQzDxO#-c(sYdmpx39jje`MG_Eh+dOu^- zA$k5-P1Wr^MCV0!GjfO_2hsJ%Inu=3x)2GGdEl4L= zuhUg6yG`h%{4pB5ZG5)~GFf-CUMwwgv)R(s^1Qy~v3?eze(IF+Z-bWM!_EjGNO&EE z)LX0ne0ufNa*ej$v{XXx0?ic^RxOyy^n&u6vAG$cSh}!NwNh8YugCkvp}h8> zlb0Hmm&z^S+svn*jQ7pCJ|!}zN){WX_NzTu1c)2!Ndgvhw*}tsIlN6?DDz)B38m6I zP)V#BvR9~`Oq?F~suiL-nXG_Tc_Kb}ggrAfB?cFen*>%53DKNCPK4k03oD+7r|B<8 zLjF??EZkI}wUrf7Zkb>X)tcQd4%K`b+udDde(a5J9;XySad=;7`PVHb!x-v)Kqm0S z66WXEPf3jSHXgq1izBDs=RvPojnr`_EQwhMT?ZNW@r{fIk8|DAL(%-xbMceWp0I&m zDPC~&`2Yb9e@9JKu3OLCrzT6@*^J-! zJS6ds84werso__fU3{!2ePB|6P*3MG7P`3zE+FYLQnpQCx5oI2jTv2884W(F7<+qp zpd{JuZlzSJ@!5biyAtemCE5|(4)H;13hWc<+YR=a8lAH$Igm0MS+HH|b93q}F891Z zQYfyL#CiGvBRvxg*G>iX8AYg6l`=|pubtyyKGSu!hH(s)}7Gum|O~?zqALCQ|?1vDX$lscZr{9`AIS;)OUf~Kw z1tlEs&Y2nJ3Zt^DThI?S&h-4-ze1u^irGaumsPYv(vZumY!>qI#3XiFKb(Fp|NK>$ zhA8Gml^!}QHUFEaTb7D zhyWkN6A=5yclBa!yOy34BU|EgInzO2eC^q)C(g}^94s^8&N9tIU0tinKYOkz9HeHE zT*wO~0(PYC7G%Hq#>EZ#xti9h|2#$rFb@ov5hJd}tkql6eAn;&7_?3^87g^!NhDOx z?x||ZEiHd%kj1&BB>|w=`6^8W%PwYTH+dTHJq>9)MbzUVD!~-=^6~6XB|&YM$B}Bh zfE_4r5$)uNr9rvbdIKw&Y~GSrBv28G(QVq?5#z|^#d8oOZKb>f8cs-s&}4Z2?DQDC z^0j|N)4^Dx0c)`mJ+c3$8lz!*mFP!wgV%A^(PaAY^$(akD3)d~VHP%j_q|?F2~)vy zel#R;OYRRlt#fSr{L$C#ncqoRL30Vfu411pk$Ue_kM%9pG2&>e(Vr~IDPACU??(L| zZ#mW0j$r4yWJJUC;)~uLw3I2?2PWQ;aAu<*>Y10ziG1wQfs=;5OPFmx_NblZr$3Wb zwE^$L`+4e{n8DWu@Fa+j#Vr-@8n?1fjizYHrAZtly%M11YZ{vqywl$M{_elCX01u` zot$%?y+3;&Bk3E!Wcai;US$o3myxoTWD-~oaJ9M`G3owbBEukVv;C0it(S!A=XWe+ zQzuAzf0C0vicv+GhQa3&#~9hfRbKR0h{i$UKLvnzo2y=59O{rM*R9jWsoXZK^)RIQ z_t3}WP+_0fWLdcNHGzM2X0!+dujiH=M136{RJDGON~gO$jfGqq7&<>w3PW-NXds@T zvc9O)V6L+q2FCMHYt&Q!n%7j=vFt0qhcyHA4h7b?=bQp`KP1t7qq><1p6GWM{N=AI z8S#%`X$9 zPO3^7uSI%_6M1^Aq49<(gz>FOq9{`ovyg^#Sa(t$}KuuvtNfVEZ8_zc}Q%_&>+^NT6(UlH|t4#_^rv|5ylJbsz zk8@Ef+)=Alhu_WgA}KPvNny|`5#)fj@2k^h@P9#0y}5Feg-IHMlgb)R~YWm0pG5Y~g|;zoy`v*HH7NxYrBpwHzu zf$E3FPe$<9YU*C7jHy4;>Vm6gYGXkq-b0%M*F&T;lUh#7z-3yV zNsVFDOoCf_!ZVPF39jhg&bOp;I74y)UEV$BvEA;X*Lt~LWY`hepAabuZG~FvP4=9I_Y`*A-}5uj~{?jOE9I%5HccH!-TxsR_XVh8g2)hR{WMJxP~W z%B69oY}`B~OwNJ07>|zY;lQ?lN6VILG;rHXYsaF^tl@U;gb=tjk3_k69~--^X74QZ z@QEx{Z_=}ijs1<$0cRu>Xf-7SkS%zEAj-3-<6&U2s_kpU+4I!tI5L20xFn<0J`*}&d)L%W9J>W}!>AT30 z{%}G1SK8N!jrqz&vO0gn^r~^TD=jw4x4!X588%fdFg?tTLEr#nwbwp)^L?ij+js+a z9`_v}T5yTYO19;$Ec;V`1Dv?uQP(<-OWm-Aklu7R#^e3k=!oQKDQqv`uo{cpbIm9o?g}<`o=h`W@VY`VDye z8BiiU;HOHAGWh=fc=%R&uK~ICIusk9dRgw1BL=^8gkxg(1TpVHI|vs?C>Uken8e3f zH4sNzlHK>G(@ z%l?XuOv2mgJup5)iC(v(002(VW~egv>Hs;nPx8%x&_Vy&QyQx4&4yZYOhdvDRWN|U zbzeYn=dR;=t@`ablkholTeE^5bf3`q=~87ORKQRkzxn(8p(`RiqwCN?0Plr=Ie;hC z26RRgVI&XopY?k9q8O{%X8DOOS-5Szk*Gfi7-Vw^+;nbiF`X&ocQ{-Qc%me1}?BL|&p1vXiW8qcd&BU$}_(B1pz=*!Vno!+?8Cs6-^O^?DzLz=V+kE(pehi4K)9HaEzGg)+&0j?n(DXpvW{hgZ#XP8R)^0Hr>P@)uhF`2a;~ zLA4R2`J&UZK|+EBOO|*N(hj~s^w@?=^)B;<2DSQXh|tZ#*@+ThR?p3<_S4$ zC=l#pY&d=(A0{bgwEO~fw7nA)=wc<~0L;AU^h z|J8`Pxgg-xJw6&2+RGLS+`qD|P99%%8DFzgS)M70o$jDzWB{~N{fJ&S$jB1MgM#L9 zXkD9bKFkZHu`0N%nz{o61W^)Zs0Lc@eDC7ysM)Bc6hoF zVOubX-&Rk%GOr-O9FF6Y_ZPDoJUgay;Cc){Xt&o|t=L)+uBJ`NnIngJ*xx>Gijd-u z_hwYqTvvH<*Y;P>gH)y`EvUUV^N-myB?v%RIYH$$Bm^V}f40Q;55Vm^V%|IsnT@Je z3}rs{P_rjV@Jjqy-sjyupq69pUo-Yei9$p&lFTMp+Cy&83GS^a4|3?kaSK87e7LFT zNW396RtO1UH+8*Ro%% zI8J|`Y$%-e_Yqd^XeNZ=FOH}x-BjRmYQl8bn9ogCbu}FlTd{^oKds1>{YFwHY&a(0 znlg<1Bw%eGBzT$M#$eubG_#%`6+Xkg2BLWdSpAW90>FNJ=oIt*NAmj;*Kd|wq(}>W z<7@k=9kWbPm@1laeekB5+sV6V3+Tawg}I5_+WJ+OX+~A#NMZYf*lz9YOsA>v;M}|8 zV@K3oZqBS)8xW1)$=hg!iR#JCWVL^ZLt3Vox8&YeI=>-T@R_6=0^(Qw4^8IBPcvKw z|M^oelA{GDRyjJ3n-6;$O@@)c$c!Kz=hcpYTUeCEkTBodYDsixC-=vcHay7|sNUW9=tc6>y$H&V{`s*y?e3c$e^%Qx% z^8NYlE(&?3HgN&T=}cr8lpLDv0xT@7UCl%O4(WIrO`6~4KB9}lUS)~lvkuhBLQ})L z-@&6fK2L-M=ZCVeFW**DrAsw9$#;m)CbR^=9ntvIWQB}*@xR<`OL&2K%}m_8rXYmI zWAYzg_xCR#3XyWt2Q`^l5IDJX%3Wet7DuJG%s8k0&6YHT!T@sAbid`8Z$O{bZ{4+q zY<;^iSJ?n_Wuhf%+$8XH-8*Nw^i*6pVC1x(RU}SjR!BZhXSRTPu-OwfcvWlP%;;wH z2~Jo5KVDecQI&sn!@e52cpt4buD*d-Qeb}C;-mQCZsE{m6SX1u_{uu6 zg+@fdVFC28g4=?EFm572p;u<|rDEa%aF_ zr@^*|Qucs0Wik!J*BV^t@+!D`tzJi<$uf0X&vVSX?uTl(HJJdyWGU0{;nO_}I*2Bb zU_Wp3pC=x;EES72IvN`VuAr%!f#73Sp5d!)~c3B>;A(wSLhavI__=iH9atgg>kc%=6cVTHw0*&iq*v9CD z804QOu!oA0d3gDw9?$u2fZtnO-;Sosc(QO@2tY=JKN2%8LA-Fe{TjQ>2^QVKk%Lo* z-(jZfLR`Hl;?XQq6edmT?T-89)6?tzpgV<$n}h&T8fbTE@IJ2}__$8~*p9>%(3rqV zgwEoF+PE(m8qGy__ZxXxOXgSwkuT^Vl+h~haX+9LTc>f#M1ZHplhp9#rWpq z31X2@PIJw;5s)Yg{e9e1ZBr(KzZs=2RtOZ!Uz0bQJqn?n9t?I~c)MApZTg~0b4$;U z58xt^e3sGWc4C~yUxHSPSEg3Xp>ojU3=I?^UTeC{k0#tuVh{e7+vg?2UaC|T)(66Z zsRFn3=bxLtCPG2+QBukuvFMamA%{?IR!U$V11+pNzV4U4HrNtfz+I{JPIyvM*e2AwTr);TJQy_0z>L z9+z-6L221SZi=d9igZSPF9p%LJdmf(DY*v1Dl`(NE(cF8Hg5r#*HfuFGkJZK-nR$w z5ypu^&xLK~6Um0XAEjmNVBp$H`w=aa(9IOEC?wDezvMV!xSjJHx7uqDrVJEff}B@7z) zn1#j=E=tLMVnO<}`r9DR!x7}pTp4`{n(@LBafd}As*!)<7VHuUcXWxM;GIqv5`@z? zTMOLz7M;aokowxG>UwyU{3SLP6t4@AJ_=yBeRWUK>|r)avelq3A8D|!*yG_3Ru4*+ zAMeyZ#&CttOJg)tcNIGqGCjjOzVc?lyBUh(b9!6n6d~NCM1}&FeQ^*zr>9ZMQ2{T# zEhC|$5HsEAuoCWoBkMnsNe1K0gcVip`}170(FI`05pty1mkOXMibRcHHb$d;Ua2(v zr0OX##VOi*2Fz*xD8x~}7M(BIA1!wc0aDz*CS($UI^RmY1Z)y1+3f~cRLf)t==3VX zum%KgZdrgPy#aih($?w4ib<-Yg&ihOe`MI*9>k;{Xh71%x+=~(90JRV(`ITOfF-4% zh`OS5zk-N-zWc~x;|u8I@qYN_Ucm_BxJ@u1IyqjjUi9hdZU~$GDKg3xk+|7rp*X0M zF2#I2dw`u&5iS^?5#oXF^&v&Aj#XYuT7J&^ZL1Z7e|H1tDEPLANq`sqe?+7!IFuk|eAD1`g&1yhUs%`z3UmShN~A&b z?QfAoNym&8$%>!gZ}OWE;o~4(uTV4yxGt($vq`JLkQSHB@E{tM*JA|5MLwK zF<)uv8S#=E5`jp#4FL(!ASJ#eM2v<@tJiv-$RcF(!&k+&`E7*fN1*0MWhAciWdzhk9u2Yj(O9u5AVm7KI z;*fWCoO{$0@hIv9if8PfUYfi#g;omHGzXC`zPw59`9IcXlJN991y%QYr?pfrw+Pc-yVA;&YPhVe*uOllz0O5UNg>^LECYd{LG1QPVC@;C^|g>pT89y^1yQvbSw%9uFDCP|Hj~>PBx=aVu-wOn^ocq*uW>W zV>^Ewmtv}C`!S%IBK?Zi$?kZgBnF%D;E)-bq(lqquuqjstDW3=7o@^dH-<$eO6&L$ zkW~3;assGuDywR5{ocP4`q`-AD<&?6ug1B>Ph>M#E}xFXpFd}|u(=}%__oC@gox$N zGgC@L&re)QVwtq^zZS(zjbNYf@*FQ#8_MW%RG~6|5DwZV%6(mh9@^PybRKtocAHd$ z4_QzU&(^DRAwm35lw4bNw4};I=nx~}ViK<9Z**RB>LPg|tZ+oJPuYV~iOm)2 zJ3E}d{f-~V@+4I*2Eodkc|87_K6h`16kx9AlOcA~C``i^J=;u19Wa@QsK<-%;>U{x zOt^6^+G#Ays74G6dmJG40JsX6e{ugnLZR@@lkUTQeJaYn-gl-mD6DssCT7y4&GWLH zfh)_b0ZA?@Zv{z3O!3@h#gVcT==6LTuQCe&*)l$6+yliQLUJlR9L0^((XdeY+<;RpQbP=9!* zG>eT5b0`Q`&OMGRw?u|Th1vi5aw6tX>i)mFVoh%s1&tJ^LT(R)DtzCWJg&pr{|ZuMV*%kGW+~K2Q!f-_B-sNxyG3 zs`kD#V&cY7xI(u2R^Et*h?|V|L6edG@cVFJ$q)>T+UR!C^G9+W%D61+_+hafEUwyP z3!TAkQZApthGPk(^hB$k=@Y|6P^u^3S5NYJFkRttcy+ zIZyBTf=i*_0}L-uf@wImwM+t^$WsOIP57K?k^k-6PBTWVghm`J@ql z(1OR4`k^W{Oq>3~7C4mYlH~`v5NNN*G4iTa^}ipNn*IrG%f;ndrm?H7R$vd`67zV* z{bk+b1);J6_)??*4(L_tyjvn+<6hQ+l#~U|7)>%=cKHNE6+#zdeXG_mFzoaq6O1E) zfeo*0euHjT59r;$^q)Tqowc~w+x7@;^{eY$uOuEwN{tD7N0kN|A>-DnLdKOqkma!! z_F?$sZg-4=mG?jlw5Yu~))Ck0;}CG~9pK$s{BmM2aY9tF3~Q~A>-UPE1Bzk(Dm2ha zXQ!_4?oZ`r&9-|;8ngmo?Xwg?J?Idl-0NFIV!Jq@F=DqXtvOvwCUMlQU%ci9Y5*&t z1tBxcpQ+#TZC`}j9JR)u$-B=WAucKG>x$Gvnw5^9W@~=C=jL=h-oQ}$=Eu?ul$Hrs z06?BDi1KnP5t@Q-DvK8N^MlNQJl}ay(7X4GHPMNdrk#r1_(;w?F8>X*JAe%0e)BYX zRJIJm{8ioV25L<^>LNi40nlMeAtx9eDW%Oi8?CqQJ7uu>23PAhiRQqJhfNJ;o8$g@ z+wSoUyiWz(gFF;ywzr{O8uc#<@L%WDmGw*@$Fmft%;BgN-(>tT{}gk-jV7p)Dly}6 znta#qbd`AV<5!=_c}xs83ZxT`B}M}D4&N{kl4#ZartEqoR4(f?!kJQ@{U~XV+zdg^ zLrVR(Q&0j|DHyU$7`l9zFazpS3MB1nRe8vuY9iHhq<$5@^hyCxY!_q`hsmMUsBpeb zPbdp=RW7xfHK~!$KwQov9R3aA&AB!;exi?8>b}(O2G(tNQjHeqvc%(jrA7PWLC`i} zVwG39Lf?-x@iOr9x z=iJc8(;E$f`VNPkJ2f#pC@mbTYrnmYzW3QaQgFigmcZ3ccri0ZTfS56qdE`}b%g|8 zS>+U3F{@TxSb18}YK1u3lfS|pTI}>uFp<2lVo7!OOkiSWg-h^e*1l~cb{8zJ|1&ej zgXC7=A&1AKRX+S~>e@@{3|v9<(y3RgOc3_z^$7?&G1ib&z?oh17zunNp&b%IK2Q>7 ziO(a8T7cRyz66VUtv6>;KB`D!9s)Mv&{nyr74=J61}VBCroJLjTa*;bCF!n=`)Y%o zop4yNFKjbL^IHw5$|TW$Dok^%^3SoeJOzsswA?AeOMK+iV-?gFju`Pq^f;${UzB|q zZn=B^%1Q{&7W;T!*6LXaBxrfk&VbW|8!nI)AW^CE)>LRk)Rb z>BFF`SdXngBkZGp^$$n%Nx@wS#&kdks$G$6R;z;$Id?7z_QPv{|Fu2>26p1hnLNf< zTb$=_8+ZEN0frc^cKdwRT5+{HD_+?19_}%2H!dKTik8I9&=%#%*f*eOcDX2zju{tV zyXZ0|a5Vt>M;y7nB!|jGUDy(YDpc9;6B#hQ$~-HW=yW}gVASsvqw+XiG#8#3tewHe z!Xn>BA*P7Y4In?|b_!h>^WgPFk{};`<%7hyRA1BT&SHg*59QzPd~RK|H$ZVsiA zqtdhEAD#}^x71?M5ct-t{}-9AL%aoosxa|k#OTK>6h|J?P>ZL$hR@GM^1q`X0Qs&Vxus@c=PjOSx+-}+j5zzvMOqdv|w#;9Vp zXyNzqu|W-)B_Q|C(;!E#{Pi;$nUB&FV$RAx7~!AZw&Q^&4K{e6{iI#$^SQ$f*6@r? z;+{?LTu0tbZtxULY_dZKmmJH+xID!rj_W^7uXChHVVuc3IPWhJ1w>Y9*C z5ca`IK|QpgLdZ(d3S%0U^uk_Ve-}Y}?YATE_fN(fS69pvy3e%g&AE_l40LM;4V=>^ zcqEdYu0BT1Y3LH5Tdr{(R@&Ur)Z?H&iQ3F6^ebc0@RtGNf|V1?6GcG1A$V(btJSXv zQS~WkdP?S&0-$BUr&>1)!>4zb?+?^BNTDm~68vq|l8JdW;ilZXgUcUgN%2y|!0-!| zWlN&G8Na3@CQ-U6#;ZXEDzixcs%Xd-m{Tq%Dcrf4GLul2Seq>Zs69vs&@wmDtRRk%0CsVpZyeqqY}{taW{I`w4@s6%>ye84VZ@nrigeJK%k;_7yI_kdtFq zUod^z^5&sxRE9M(uYvw-$R5UPkLG2CPFg?$$i2%k2()M1Kzm&kdE3nWjBn~xJ=AcW zqDmo35|($wsHp&*foF@*?9OcXKfyit;}d91dvl|H{qWFt+xiFsQ^4^95#RsS>IE(D$d5Y<47wClkgOR=9At;w+k|+~U zbz4^EYq81Sv@i16e`8~H@Od4SQ+IxkrTxsCMo?+Y<)Ez%PXnlOir?|+!Yc}1ucJxM_>EgCN#akzGhBrazQEbiplaL**TjMWxf7|3&<&`yn z43dAjqrj}`O1yD!Xh0Z2Q#8J?#Hot8e4b)}5r-Q7N*z^bM2w^>y)Qf}V&uc7YFlp0 z=hgHKb}|gz2GXU%l1a7^>Kfz_!kj|RGZoZOi8ke9v9o@p4XK#NTBG}w>b>e;*KjQ? z+Q6KBTaxIX4LX>iw`|j2X>E;J4?9wr}R7>Zqn_-t1$7vCX|w+_`z3aZ7RZ^uuz zk0vbc)>=GDKrwC6QAUGq+Su8o&ylTWGkq5G>16$Ll#ISK0n*p*2`ZGnt_}YIzk@vy zz&BdcT{9D#b82GLX|;5305fz_Dt!1qsm!;gNT@!213FD=$zUYMT+G#`2t?q!Jx_S( zYX5ius?(`T=Mv`Lz#Uc!KbN=MJ>R&k-Y89cU&<;r>9J-1^LuOW^pquo-Mqg8p0dk| zD}&v!T^6`tHJ#L0@_q7-L`s_C3Ds}8pvHVUTV+Hkldi+Pk+#}?0 zqV6O6d-;+KKOfIfShK6!tndpvX6{SPwi5zOLAc-bn$#Zb<$Lb7pHt4@D<|AA4}^;S zgJcZJ{&sN2++Ra7klZ*DLNzirJ2}OUd#Azh9W+KqmH9z*M7~_Q$4(|vg;u@@=FiM# zP-Hy~wvyoD!ZCf^wu$n+Zg%?XHCjy~jc>raJff&}nzOBSAK}nJcLh8RMb@vtPR3+y zxeC)tcix@Q=-j3ypCIc^3|BOn18M~msWoiz1WhWFE9RAb?ucj=<8ORDj;;4@KJvDk zc;5o0{#I%%K7H_~AIXE9P|_pxd`d?|Cr=^iI|^=}@w^^q6;NE6DQ^(vGFPC3Qo`Q9 zsB_6;;;zH>Wy)Iy&U8kS+zHXQkEj`nVc9;pw|;Fln;n7v33I;QQDOBc)chm3X#S;V z1OrCnqoT=bK>A5~mN`$xaP<%odyM886z&#;MH#ZnFLpn^OPG4{F5o+p=%?9v{W+N} z0RX8L!ReBu_@*Qaa;&%i`Wk{iK-n#xZ9%vI(#Y6cFt(mMeQd$lF*H75>8l)Hl=wPd z&(w)eld{7@aL+a~rWu@|GLd_e`jWhqs+C%XU)w#8`SYOrrl(UVqcH70)|VJ2dqbtn zcjqlYHJ|xYL*mh(!U>D=qey8G+$r6&zVs7#0Wz)sSW7>E?%5@w&kNX)%p81}ZFQRk z$7Ay;WXJO92c20=af6|DzUR61v8eTAjAYEiQm87pe#cnNoWVmcILu{yJd@fB}C{ya$j52h& zle1zLI_I5X%7}JTdbxW$?cHgC7R{NtYiF6X#sOdAU)Z1y5zJ}5&(Ak@2@}kl_ha4L z!HZPFq*ep42jl|NcReXj9N~U;!SV|$fIm1-nFVg-=4X1U8U?{gKgUYvR9!!;<}G3g zAjQSU&lw{FeZTq2i-_bCw~ajZU2sZXY@gTWxrf>euCpJG+56kP<3k-x9Q*$&M2j*1 zAUuzIC~k9MB_v3gVR2M$a%uWiFhW9b@NmwLy|i(Pwx^{}>fm8BU$Ur7U?d!Pu!7=m z0Wpyk+U4uoqoZABHl`@D&Jim>Va}sorLf4KN+eCI}Vj7TvjTaZJAX=#db<9fi{HdT)W0t1*GR4!5AJ4hTO?p z9_rc8)BnXHwhaDdtvuN%X=MsJLHU_p44=whftin0*+D}{=QalDd6{Gews@;;41>0c z(x!JzU?crF51(5(DpbdR?bC2Zw17ggdm*pOl;r_quJW{jHAX2zgMh7;u>G`uD?*%tx% z#nQWE3MU$Aj2Y}>uuPf#dMk2VA1!Ksp>7XA1U+j)CX%0@zXts*>g=X#$@gR2ihhA+ zzrKWOoevrWAIij=l7w^$8`XQ1Qk<)TqESx8bQ59w3bm4;Fe5#T0ZrVHo7}9{v*fOC zOqN0yjO|+>{2D?Jhm{LndEhjdkU#3K;EG-`l`(3u(okmqo6K0ojoWqs#onQZ#Ch&o zykD_$u)tpMJrs2KSjnUomR!|nBdE~)zenYr95K*>>y!{{wntBUts0{slj3kc(q}%B zEwSeY$PE)jN;?=6TSBb7F|Sukqc@aREtMzRL&g#g*NBa007ukl{}xkhMT-q-K5*!_&1#xn#@ z7Z%1SDI;PUL^o~W*2Zf`)ppk-kpIw)qLbf2ySi%8YwdLm;^_@^%V)LplV9&g&r>1D z-OF0!#me~k+!Ig}JC0j`$zxJ|{}l2tNS+`ilDXXh4Du5;g>Yt!jxUU@C(}JkAPl-5 z_wOU9SdYxj;g>!{D9N4izh6gwHW`#>CCe(H^Zw*SE^ORnh6^7YukitMx;yDkmw9N=hj&%mLvuNIb zb&#$s%ZCBk7P%N5sF~c!puK7$JG@VZKzenFAw`YOZ}`bZwbGglRD;7~>Zmssq3}B% ze_?O);v!XDD9BtvtdC&Wm&yqoFakQ0H3Os@(z@cNTU0UW4&C<$4Nf{j5!?}#=9+%X!21-2z!2+{>^TB#<6(x zH3eT|;`l!1-6eHnBzr$1kU>Jl*-Tjdao1VjTndT7q^2${#Z5elPXlwF2_JK$$KNXrvRX7_)RdKEJBhDojE}jsTC+) zhv$YP=dU05%uaOx(rL_ZTXXt_1#T>|RzR~xh=}>@4L7^70Lk#qr`f)pHh1p^*|Xz; z$|%-cuVl5HnFGw`7Bw?sY}u3+i3Q_O$mWzLw)=yX?YDfWMa~99xIF``oH4ND-fxx9 zYmwt@7~(NGENL{`m6iUxrpfjCeG(@tl@K296xYGL{n-2L2Q0i84$A(ssr2Dpwixqb z8Z(kMtDAy&7<_IP&O6(sLwc7cpnB^tw!{_Brc#q^IZR_P zfI1cnOi9=;lZ_&WE-`-J*vp`5dMIlR%|xy|T4Nv37tMpL1OS+IBb*D3#~;584Z7x; z<2L5U!|KWIBry1L@kVP~R`rDi{N47P;K0TEx6tA(aYOs%{^IdUu=V}@uoz%j9I7e| z4O_4kd}y3SI;(~D738KL#TR!a3@gchuK1+Sw8U%VeL3M>Liv^w@ED?GpOvf&!b0ad zLxc1*9%*LvmAhW4yq-S0z(j3>JOPnzk+bVKohE(JgH`ElJP2v2w%?w9BF(#4)=Q6e zaezGOMVs5UV1EG_3>*uA)le$#A0oD;N5L38fM)+fU7CHdQEq;ol?D^MZ|JRQcQ&dr zKT7X+!P51ov4ZD}>)8T557bA}>hvS;NXjiCd~TX}$s*dF2Lx2>EHzq=wP^KJ9m(Aj zVsv3&=#|Yx2?D8jsIgg8!S;ogRr6j^sw>|>1WP_L9i&x{lr#Z?157UsJHI(cuC}1N z{H^(a)rCtK2$h)n_J&dQqR>^|;zj}ghY-cJLK#6Dnw;x&|tnPyNSWb#DYJ+(@urTgG&s(O@u z*@o<#@4mrnWZ$P*@Y(0PNX@LP%B8owRtjSK6pbB+ly1x$dY~R4ZZu!j&mVx zI1e!sFN4vww>~0WM*MaN$~hP}3-s@Jm3M^YcwVg2a1sPkl6XGL&g{I*4xk2q^=C)= z|1swNu1FEAaiL%hj@V$kf{83!>&j)Dwo;~dM6@SRP*!7bh?3u4EkKvlGyY2i26Yz9 z+-^xkgkY2zO)$Y~TNmHYhB(1kHH=MYy4;hq@1zg>T<`QV(xnMVl{c7WGZkq)M9U(_ zxfrjR$bmA%$>r55E!hX-IR;59%5Oq=NMC{i^B?wC*U8x=9MEhZ4U72bE4l<^mK)qI z$vhwkVE_VL>px-dFX?{fh2iiS`|Asr__rd}Uw*WFt+4<2o%r@m)u&DIoRWy%PQ^p9 zcIrI}JLnC>T2VD6v}KX`9nXs~g|wP1d_6dtDTzSk=AjOF2XQD!vp#Ziq)`U?i+G6` zJq#RZ+_l^DaIgWOoKG6mKy)Iq;`g6Fu#+pyU>QCqz{rEy`2f)ZtA{}7ah6EGxxX(a zZ6+J25HA)~QbMAm@p>}!dMS8*erEBR@VXOLFkWbr37Se9Hj@d;_9|h&G}OBG*W_M7Q3j$n65{uFZU=v3yo0d#zjf zY%i`Q%VHwBgHNahs>H;04B0XawJOwr{t5DmPpkHKb(HI$bg8~j&`%~qbg=gjwBBu! zW6O6$G2El}^8$hTXXSEPlRlUFbtAh7X6bZ-o%2oB^R1}|ySqec&+;XwG7>Kw9$_oG zS=q!?)DTA`_$_Yz`R~9l?~)f677WXz69b02;xDr)Ll}ZP9>?GBPJ{_>amDJ4LHg=H zaSw$YO~LWC^osAXV3n{aO}y~uU5k7BWGZKoPxJQe{lWn&(hoDURz6!u+J;C9j<}dA z_iz|eCOBT;wg$gIu?Jz)?2_C8TAo?-N{<8Ebrs@#i(99LQtVKM;GD zFez<)Mx>29&*Dj7*x!;QZT0J@3!~iSee+2G2io>$qi6$*$|>6q4=RC>wixX~c2fd% zKT0^((#g~9CcqZJh+tU^J1ga3fNuKCp)--MQ+OR<zU$PZ9o-5#P_=so_ic>n1bxRlt80#eQCu*t%f3e*!Y zMy}&}TLpKz5gEdTI5_QaBu?yXvJfqnyZg6U=IW?7)HEnMcF776vD!Cel&jPEw^POo zuOUt{<^#Vdv-UrTYb>b)~|FY)==W&-zmDihD7z3ZSq!R*_YaRxW5=rjZirk}Muokvw**ZbR6A;|u&jHA_o z&l&gkRd@cau9M!&w?#9edBl7heBYADy!cda?Ky7zRy`;LHoM;FZW3fT_OE`dZ7pDB)xY}FB(~=>~OSR=(S)xLob8iD|87- zmcNT{giXXGAq1EA<;Nrnr+VL|4}L=EBN-`thtP1>Kjd-GEWuLzbtFK`NlM zq$FSqDDC(az>H}vk?^s%A9p0|t?;)$0|VOa7wDr66@GJ=z&vI%P_HHlNhOMV zcC&Pf9+0I0VnECGYjoOV_#R)dTbOJqj3k|6uuTooBuDI@Zdvd+Oa%&q{Ys?LW{a1A zF)8rI5wpm8t>acXzH+mUL~KfqOe6+ifq#PmC#6)l3CZQDGKCwZ-fA82Ar~1`n281k zNw;wO)5jUeG99Y65);cron!BikqM;L$w;p+a)K-nD!jfAB+0q{ed5gtWsJF>Ws;KF zex((ek?XwtmhJG_hZ%Gxuu-2ICh1L+tXZefvvK*y3WY%VOf%jsJuEm*wBvsI;P3eK z?(r)k2B`H-9`>)2R!c52o;goHq>*8Q2o)m{v0;CGJ)@=NL_03(Is9?q zZ}F5qF3prh&b|+LR_*Y^!{e*3Bc>=QrQZEP{ANzPtWx)t)0*gl!erHw6GuJ>b;zQa z_~R+`DD&2gH9a|DtY`P+%IAMjMoILvQEOQZ%ID_^E0SbhasJHJ<3aOgg%w{B!Z6~)NrOfvhg)fA^{{Jq3JW2@vDs@H% zKmz6g!25^F$6kaV+^9Gok>*fmcENhtdakOl#=R)nfGRE?ALf)oP$szvJK`q>K*$`d z2&Yk%E>*{O_TQjf zOYK#Ik{RXh@-zStCta8;8MODRTyh9i-m}YMn8qwV?-}AGBK8*TpE35s)4&pms-cuA zcQk_%=_0_G%Q*{XU(9jZyX5r{o;k?|()rNx z7k;CJEqjywHtSlxluLr~pyC%>$6bCf$-+(~_*qNa!9Gbj$-5FnnTK;vK~E+hXHzjd zsB`|n0vkYc2_2(PDwX9(Ap{@AQ4pg(lz$9X4S8Z z$F;+d0S4QgR5n{FoA=dcy|QdJ7ZtY6IHPF?$L85koDM-xuxGd86RwaY#&?C1E!DAHo_LvX*a7f|9qSY7kmoJRUdqkwFu6!GlVYq{Vb63LlA&fW303_0ixB6nEJkqcQNQ zO5N#l>Xduq(^wKfi{%Y`_D`|JtG43^+{b}XnLkXeAm;D5-p8pvvIX@)A%e>mUvqK` zML9V;$79iJi?CZw@_3)6gI~SP&tIlb>PKEMB@l3u1?SMZSITqU&#r=zNCaizo#Ajp zhDPHiE7Yu-WzEsij4N;7@?}4SA8H|cM^2IZ%xN^?8lrk0x3aKX9;ekrIPli` z2!cb;gsmQ`=k7Llw%{TxhUKPNz-p0WmD&PXAnN4OsxsdC9y3o8?4Sx``pc%ykkQn4 z8LZ3U^EXx9sFRw-&>7j4xN1YKUi4q;Dr(e1x%{O}7%4N^o?8?KeC~e!UeA0z5xB(T zaDw^czi=#95~EeK`Q`O8W5SUDFdz3Ga#xV@SvNQ1)E*tP-%aCVuH)E^8y^~@c2|q) z|1tH=0eQWR-`U)Ut zg0$wsE1x-q7nVQeOPb}nbc|@Ud8J@KVtOOBSsxGe2xKJ~w-ClM52_Z8MpWcLn%Qq< zM> z5?df%AlUmsZ=Lp8wWqsq!Z1jSW=5lX1JlkT?|ftkay?OpQV5Vmr%?)LYG`}Dq#1ShzufI)PL9H>-x*D-TZa$O) zvK;h>CK6%fXBw|1`7FVyy))kt@tN%?;Vy{!n1$j_GpPS;gbvQmJVPx^*Z=eTgF4AY z*~fi779#*IU=@>aW>9M|;A1E2Fbv)m|9J5=|H4zKXA?cgRmBq?A}?2|**E>MN)aBJtuing%iBBl ze=9ZqyHa8s7^c{T9&{S=a#cnucgKBUh%g1G@tqI)B{xp^$C~mbyc45Jw?+LNg?A_} z2B8I*@^N1RrK#V-4~u2ebe6~p3*XzHS&jR33dX%wS<|_0EAB1-bF6+6BsOFhifID% zIuu_L6)e2fq=!=&ZO}_`9kKjzCsKrgwpFZgdSzO}AYQzeqFho%gel3Bi_`quBClr>Ey*MXGVZ{>}BT z@)guC&2cM##0wXUgy{7S&wb^Srhkgvl^a!!)2cVhScbFki=J8dCZ9f@|;s|?@IB^3>meZbHQWQpZ zWRWsa1hcZNIUwj|MvQQr+iP2g!zij*^E#pB;%bzk3#Ei_`Da$^8n zo{lDD^LxNE#iM->(&Df8FbsxtJrCDY66mX4m0}O~_1N&vu_V=F&ipWOpT{a~*vT=Q z*d%~I9_&Kj?n6b=!^49 zfV=Q)DUBfKdEe1n=hVlGVN(;Q#X&3mC&GA_y-Zu+voTHMH8td+`>JHe;iL?UM{;nc zfGrKJ5VY{~1odi%1%BMebZ;fTb$xAV+#4c5 zCuo!k+lxE0*g(Z)d}%$_VD^ZbtorPav+0iDouM@G71pAJZ_g;Xb3gN~#C>(wxC)&v zd(b|A2$y9wwTG(?>#NnfyixM^@tQhW@)ma39mP{@y@G>+R!ba)FBki8qg^6wBOOoe zpY8wx)nxK|>nau6p}WS6P&XYC=~IM7gsya?8j(VIw)e)F+#= z3$EiG73r>Qh}oQ3(|E~`)wX$lKZV0zVGxbRm56vV^u*+$(^F4W>{NNrqAEv*LWUN+ z*!dV2*1mY*59j$dy3;!#3`}nG1NKU4VWFV%=5D-)wvu&WgnOIn*^(l50*AVIVLd&~ zo{B;Tx~FulJ;CH&(7i6?@Mn|9C!3?OyV>B-B=S+SC~siziHDOpLZ> z-KGk?9aVoVl^FHCCBL+*#ftflIM_@>|>sPgLvk6O`39a>zZqe2Cd zTp=Xy3->-eBJoJ@0To&eR!(YeDF>4Y`u1PWA^7%jytaQlnonO|RQmW^ai~F)9z!O< zyPXw^6Ic+?e&+_`G20-LY5|DE1`grwe4#aPf9laULSQ68(;PSE#{RO{Qs7(7*j&3! zV`91T@X_~K%%YcT0;|l+aYCh_qt+;TpdL5f6aV2iZkTCgXyEBk0@&wEDpkdo7wc%) z0d7ukh?`bt8$@J*7$DXv`PVMMzruZJ%g5pMy?z14vFP#}UD?Cr^95`BN`(fvP*?5< zkcin`S6xLvotS+)>{7)@D7KIRzZZq<9#T9GM2|Dnk+a#V%5Wk*ebejf?li>&oZ}`R z&@|G1gc{xDiv!*!ImkHCh1_U19U(m24-}>UG-YxuPkln3{GX=YN1)YP7Jy5N)zC=s$1BFaM(4(p)XW7O&;M6Ox^#TAa`frUgqS?VYoB~1acJ`hc zWZInO)IgYv64L9#&`g->lBI*tC*d@XlNKdvXth_shpdeC(xRif!q(1#srj0V9>xhl zoV2KnP8DeQiJ%cfgK|Al-QX zPwSf~^LvxI7L%w&vGC>b?0Qlontj0A|NSl^0f)KUDy;FOfH3Y(ziD;aY^e&(RKXvUqW? z%`@eL3cssN59h+Um#83l>QGlVUn{B75%&UHw{_xrW48|>;CA{9h)a==P4s!(kNBbB zsD|Kpe~OO2JK|O#Q0h}^bsYqxu)uW^Z&_D(%$k#Xfi5Qg)eMnli<2!(F_CbE7(`P_RSbzgRRsj$3FmR2y&G~gHaEG zzbB#2Nc{%!!Tt7E1h9fAr~{UQMop z?QR4faKdtXR_XidWwDue9f3&P+8vj!3u5~dQW~`azbHZ;sdm{2EJhN!ZaXfUMMl}= zYJW?X+Ng)zYNudKyO#o7gkQY_@mi@-6l4ZcvneByN8bV+ zoFtLDacoxbR+)N}@$5e01`AGiQ(0kRsE^+kVkxW`V(cN#u%yGf{> za&>;0H78%SSMNj~%M>!}H{E%y3%)2Bo`N(AL(r1GB~kX!%oWjb&ug*-JNPVweE8VzA{F1QeW* zejTGFJ;&TREOf@38!S{Fv*%hEajQ5NNi@=&28!4oQOlH4m(aBYFqKUR|<@yf=-pXjrd-S#YGdldubm2M0W z7-z4si}619rklq*L(B;j926Vr0=BjEPnLyOV=0Wf7#+_f8mA6Y-7hVF>c>pEHI+|y+LUXUUk{GqcsPLn}*Kdj&n(gZ#jB=C1+)#W?MLXxJkO-LW0uw>NnAlXx`Owug zv%H7I>|7+o0YM3+yBoD20fSMDPmHsj8-6Cde)llwwC>b_r@WJp_xLRLGhbSQxCs4W z;uG4q*pNojEs_$?jH)tNr^_lLogR_FqqaNL=AJ0hSMbLVK8_u`F6TnjX*C~;m*)w0 z5Feg=33xOpWpj_|Q9Ai>xh6~#`SY(^dP0+fsWi0I ztDIPpRN2xTm8ADhkVbycjJX_BBG1;gNq7x-l-Uc!yWq%rrd}^M+Aw<8E`_oPU}!WO z_-%77jo9{My>?1$oT7!55p}LUKAHVz%07jbQ!%=0kD}pj_<1t}HH3$^qT*4;$mHEed{0dKt_oswPd~j3PeTI=P+qaqS?6@X5%*r&N%SJ$bI?1To%+5S}%SjCKQgwt}t z1Wx9Ij$w?U8~w99oTFgiL#k2+c-!HI>wTxId<){2;|p_mX2Ox$IKhEFu6 z`}e08C&sF*@PGdhCX8WXt7To)Jq4V_|3irUBsP?R6s!7o70ROh-2cKA0N?}v7j447 zx*hxXOMm(Q`%dz}TLxQWLj2!7zyiVn|Mv;+pT#mRg$fZX_WzC@{(|=|i|F<*HQ1r~6ggK_*RPBEuuK)b&JAgYeKl`5Q+;jn0l>a+7L^m}VJ`PR^ z!op|l2y7;e*$~Xbt=yciB~W=igub2CCX-wT(EfrW+Z~qmpPA@kZt}iE1_$~X3WFg+ z38=b#NI77?h7jR@Yw1Z|*QK_SzV|ke9c~pz{d>f)2jZx_-ig2&LrXz}gGY}0s@E7l z)e9qYG+WC2lZ)+yS*5|z!9kQpo33wzR-K+s?D<||Q9JaWy#YJEi$(;Bq2n9u-7y6a z$1d`?Wm_1+e=^O7))rEAZsLFs>a5~YQEFT#!#qGpp`&xq%AFhRN;3D6#S0@zNg_h0lL)vWuY8W zH+7_g1MjLxFr3)_RUzTGb^4=5=;8vfExM8i?Z`O zE^RiDHq;GTNyg)N7`3sfvw4J|f$jDiT|wWk(&Saw;;^Qk=;Gqcocq>t5&bvmq&IDUBmC1L(U^e^EeuzL-k*0QQme^LhzY2ax-+`xDTdCCM05 z=tD^IIG*JJ77}@@Q{DHxO|O!047yA`)&g0wlxp1s0BNB;OhZ)&xiK#^ni4(gt1~#2 zfbSWR-~w*fE-%3u3-E_#L{i_rT0!uXILPno_z&A@h7#(wMuKN+3N|@n9-~r zFC|k`dwc1!j)A5KCQi=A4^i9ZHUqtkYHrvI8~mIW)hCfx9BEt}mT$xe9ufZf1CB3> zpurwI7s0g`)v7>b^5?chdL6tk z2?McHsE6}%K$Q zm`E^OuijRUzL90g?dQ(T6Le^Bd- zwh<%9gd0HjO?B(`TzD-mXblCVC6F^^c_Y9_rc}n~=KF*^E)r>-?N9&m#I~2-qnG^t zBhmh2QFzKc@*~lb9qs&1JVuYi_X^FOhHk zsp^ZpbSASuQ0XHF&nqPNAcsY-X=Bj3nZl&|;BXu zMF{W_+7L!uaU_A!;_mnA?5IcgI({HT8IX17mqg+7Xhduisx~;d7lNG6r*&CdjAI@5 znPhSSs;@sYc)CJbz&_9*9w#|Tho>%L|Mqyga38G2?J~YVK0_h|`KVlnP#!WX;<=D6 z9!xC&kXFh5k=Zpm@LP-LRh8LrmIsEK)SVOBL9WD5uoQS+E6>b~_cq;8?{`Iosb6EG zn6&4c={sm4qtIn;aWqj5Kq&rv8XT3i1XpMS z;I-Mj5MI7OzKjIGnNlBfS*HlSYn7h80(CO942f#3^b7l$P;W+UTvpbV6xzvOKG=>)W&uX3qD)*Q zXSkQ$J`(z7a#>nTP)CQ$s$_Dd2IAfi-6yA0av<6^ZUD}~^X8xy4?p;UV6k4Rj$E3; zdTF-q;(o|Yx6%k9^@@Xu828~DSE*uq{A~Gi%WU;)T7^cvNRb$r6e=Cj*!!}}sW_N^ ztU(n%v${oZ!G0!pxY2l~;@(6$6pSS@rI37;*afaYMW;&W<0^_;2yNG3_nPjTMvcap z`@_aLNNIp?t^$4JhvyyL@kVLjvhgsJVF3_hc6Gdi%L)!5o-6ujG+XjrmO3BZYLlhKU>@YIi}<=(pz2KzSVX?vzk90}BybTW1|k6v12K>^zTf*N>sA?| z18s3=5zVsG3m}g%TOU@P~0qy z@?dYMyb+|$k$%1pIv>9jN4aXO&9{z3jG+XYB02NB6mwrdJSCZLc6{S|xoe1;D+*6? z!cD||XNgDr`(%+7vi#);E~_`R^lV>srz>}x61&%O{7(i4L^dtNEa$P0UN(S|S8_mZ*2^W48;wMHE9E0C&pb){5kkm>c(n!xf_^1AfapVPoz ztT1Y`E7$c<=_02eiJ8@0V=*ct&Ev6m9L-9|bhV8Ma*5L5z##e!?9$cD6+&G*CjcE@ z{zJ*CJv}jA@_JM+dkuJ7*EwGN1P$G;NFa^9{kZ?rZ zcEn!qz3(bDq?q-3MWgZgXy0vCFI6M7(vAF&|Oc-(SRHFbfK@WVWyjf%cRF6biw*%S3$VcvD+B96~36pw2zH0EIubnj-(g@bHjmX0hjrqC% zVVaT~EwfVk?0m&K5KW*aRXJ->zv(-3h1|`$`P&&u=xc)rt-uSJFPJ zelazzui4iMN)_m@k8r4TwnaD0q`QDJ`{m`l_eEWdWo#J@>;omU`1qB{m)&#-pIVSq z67Cm4uZvAZv1b{!X8eV$t!}Aw_F4x4a>uXqVsH25QG`=^r@umBldEcbpsz&?tjozk zF|z7pID!-}guhUwW?&i;K?=ntsuTnjT211QCy%TV1a@NY1}^+&kJmdY@dDovq8zF9 z8lgsm;Qs^`^Uf0k^9eWzuPXCsI_13PpdLK|-+rjw4~P(sJ8ZLgnviLT_01NWOk4+oN)T!A21L2YmC?{A z9LB>Ep^ae^uh4eo*FD&M-@fP871;48M36AcHi8nld|3si>A0T40PEc*>skgzNfy6P zgzJg?72yiC*wa1KdQDt?7(=tyW7*ZSwz$H)9z7|jb$jf-t&M0;+whc+;-@Gm?6JFf zu`v+b#{aVvr7XA~7p<)wjFX+27c=iY4*BvOfx-Y)pIGci6MBnR9qcCnaW1r}A9sLK zqSE$&ic&~rE@~i1hFH^3BYA>mH(-?)#No1i6m=jZ%lFAD8`nmz_h{eyVLXBYHQJp* z{Nd^gPKA0SJ3Sb;D@9N&ZG+aJR_ZiS&*uS_$(sQzFjCM^oinQuRDst@2><}5v)^Bs z{y=4b3~zO@s7x*@Nw~1bWejW&{jR?#`_T;Iaf8^ig+0{YJskZ&nMBWqLl@?{aLmPbiaN~V$wJdgHsAq5;sE$S>c@VJXtozTWd)9hzq%kad zqa$wUc#z>YLYP&XfzML=E)0F}J`P%JuIGGEkv%AIMBx-rb5gz!!+bu^KX#~9)MQkv z$k=@K%&uOyzkO}Sv6kn`vv`neGDsxhwNgL(TL0$G;N@qFHn){RqpD+mq0^|*_E`}# zb)d_5ymF#TU#Y+fIqhYcc(_nA*9-UdD;HT|rJFp?ymS=YcLrX(mAC#<{55 z2Isl^))DO&575-Ts46-ZUGym!KKzYz%+Q1(?&Ty9QVR$B+e!p)6Z_k^d3NTiHNhCe zoX$@=m?W%b5AsJa1&=30pA6f*Y)HE(XbRS_j+zlKpow^q>T$i4C4z(k(LJ3{k@YlK zj6LrCbK-c!v?g5>9y?zv#_>^ejVC&Kal)QTg9s7ucV#gZ@jrA!bPuW*+ck-`YyC0m z-byM8ye*d45DEZ;-opDsOH+B@q9?7sB*f*#u{joWzCW$F4e0pGLQgd z3h3%>Lex%dVCV2&N4q1*iAQe4lXckR>zDzQYVyWpUy`des5r`0em`AH*X?(q6X>DL zwF~ngpH3r0&5~26Hq)dLHmFZ~e0ab}w^f3Ab4fI>-=}M3k38>+$j@4LidDQQw_W%t zHyqI2H{ec0q<$&r6et?20Z3>xLy%99{Ymg4c3y(OgXME)n|p>QA#dI^9#ZlK?5!4C zw9Qw@LT^+78CdoHD2n4dQ0OB)2pdV#y$OdKhTRhPs2i-hfdg@+)lG1Yu)BM3$#%d; zXHZgBlCulcQ4P)$YLS)HP_1A=1F656N)X_lE8`$uS|#M#4*|6(#hb;bfh4N9qp1Oi z&}<#^e=MMxU1-hRM9TFik$Z%H|M-Ra>EbfcYxz-DI+CRAC*0gfjIeYSZ%)@_mGegp zJ4^U&6oARgfolEbdaj)*zw`Jy@2ed2YIe#)M8bM13c5@5DsAWLhm^kN^S-lcK&vpI z7vCOpgQfIX!$I=U-)>b#fD?*7L3QMvA6ObfNbRZiTvvD|^R=THj^DNXW5PP}q^B}~ z_(}M#$GeNlAC(?|O!6d#%jCDAKmMPE2^Rad*}dzX;i50cZBvr3?9hT`WJksufrWtt z%ah;&Xi-W8Jz+G6m?<+jJ!4hvp#^HajT$cdxlg;W1HBZCI&+5Xf|*XX%om~Cc@zcT zM*4?WyA4_OJ)fLP{kKOPqa1JF0k{nlyo2jZcp;nPUcRp{s?gF41&dKP7Y8M9+J`_$ z&MSA5)f`3rK|#A69DHSdU60ddHP=kV@5&Ba^mz?3R9^`N(&#eL_6_L30R42ubR>G* zhHA^OLa$fWta5)IOuba@&OsT3j-dl3YM=)u397jbxOFwSjlag{)tX1Re z%Avp@c+7jXp~vo12y_w`y`21^RgYax9pHYr-G6;dc?o{-CovnoweYUO2;k6Ay?Br& zQG?q_jLvaAnZ+&DsA6|O1gUmiDqGO3HyrCuH%x#yQYJ3e3g{dr!?Kw>+ApToRmx!@ z(2c&323;cDw9KsPua$&3z1?~CEdal2w1mr?32pAzgL>thB-Hkd^vQIa7O7D!9wYl` zb!N8&oKM(raK$wS8?>YE8A%elxZQpQ9KM&V4gBBJpB^KL(4rV=Qz#h{)X`9HN4pSN zuzp91VydH|%~)x&(QS5bM^Hi4V{b+82Dd*U_cmOJ(+i7LAW%b5*wO@{Kyeuow7#-I zD6)G}JiT7?oLk#c6P0+mI|yfRrd_xS8{P*Ir^DmbZHuNbD#8g8VAH5{6)Q)|-Y4V7zZ!T0no*c>~OXaw`zW;i2z0pmEnK`ijrXkj8JfKSf!bwJE z{hB^%&b#lS^RzJW<32VtCYzervuWIVCq|quC$}jfFE)p%8|oZBd)c=0dm^Od*Wq`c z^q_;4_^?5aO4WwqLhvu?(cbYW`12!J3@T`EcYI31U$`%*tBAc}a=WnN1%vb0V;$%c z8LGxFo4w>v490w!SD^30tO|ucf7E7sct7toy{}7~c-d8Zl_R?-CF%OjK-hU!m_@@F z&9pSFpeuMSttaWcmSzCLzp(y?W^oF(Lj= zz5A7 z3vI}Tp1E}caYS+#DrRa`-C)o`rB zs#c*{%}9Ev{+?Hg@YM*#KRTmnBqUnuWHm^%{%bpRQXs+I_z2wGMuO z*C5LYKD9~P)n;UVwNo9ZxeHsq^U2$6?j({2{_fL;4StPN|9x>yfdXkB<}8uF<2O4Z z21*r`lT$*#w5f=5&KAs2Nr*I?|HpY_QcF^q_U~(d?Jm?{SPL~(BHU_lfI-P%lo{nSLNq%n2(02gNXS^_=g33 z+$?PJv)iyI+wW3;u8v}0*IT_M%~9SosAL)P$>(mXH7+uyCtKd3lJ8?h@VUtMe`2wK zx73-2Ad=M6XL4kP(yR+n4cL#Sb8DqHoHGgvauWl%4BuvKW@qL@5ejO2 z(Gd~Ju{k}eLy``R@;UIB1a2)FvaWluwBSMs1|gAvAd_FZ)V_8Q%oh2VUaV$snifCN z$l*|zc70gCa#D`46kBKMrmz!s9Ypo-|L~PyD>l6u0pqM9n=h|2J>ep4i2 zPJeXwny)Q2?6xaquPOEENz(fsb3O&dSp+h2_FIHKtd~c?^lDPGwK-VA1^6 zi^+5n>BSILn-Qn3H`bIu2@T09q@lHpmpyPNX=Xk z;8*n<(M`cY{CKqtR z&iu8JJNnfN&I1{z)XNtG#67Ofo!!llNleVlDM(^TlB;B~ub~sGKa5=tu@a{cz$K^- zUsHW@G3DvLs^A2u0H|P=PK4Za(ki85a2v4R+p(mX+Y%q6{KJ33O`gaLrSLws>j}sy zkX_h*-YCRpEXwB+UmQpr9=XbA=&ox&N7(Awd4SQk;82O-(fjfL->LG9a`ec7ecv&kk`=hP4qap;3a>0fm? z-~%&*LBW~ssxleUntDmTVeZFzI5YbBnm1HtDCtYm2)h}wf)e9+zN{ZGoj{8rg;%;V zQI2OO*Lj-J^MfyvNl;RI=TR2#t?x7}9+x@(+OC>|Vr!E=iiT!&l(oOS^ZFm?$nJcTgIZoOU%ucrSyFeXDmm=*<3BP-UZ-B=Yd+Ub# zc7q*)X=XP>ggD>__MUIYwR0p9gzlLnJ<-DyyC(-8D#ELw0{iL3e_4ds$&5;SI<8QU6ejFb8nWwzM@niR$$p`<(tmmWw_q|sSYBm4gjT(IXyUC18 z!;dy{+W)SDmQMEn ztrN|*&#~tR8dk1e9vwOGzDK(Kjb}($uF>z#u{JP8j8f8=zCui_*-o90m{nCYRyK?*8V5*(?kMY1~)Hv*nD(` z!}T6v-_3f!Z%?5cXdM1D&qo>KJ`Q)*4brXX+8+QqOC;b!>*B&`vHX0H8YL+1fNeh# z`;|z@{gZmNyR?*))Cr>Ix#2~6$Uj%+GopXY@JY6iFDd*-%2+_9A(eDhh>;MEJ%OO* zlgjUnM;og0_!dUgPqm~P>20p|>o{1g=e!yiv<~{FMaIXa zjeh5>#WT6ONXK)d<2)m_+;8LpHS01aW7_m*L@B%8@WQz@Wf-)v+O4jSN)Uko1*V;G z4yz-uZUc4V6v;I+M*rMNB(%WQdJXj)N(fi1|N4V9xWGvcMfu8+`2}R}mpqnCC#g1+ z!It6l(3q_gkSy=aNlmTMlNGx8D@@gkuC}%z?%{G<+RiR)$;Q;OxN!=K2oEo!SC<+v z=kWbD66nHkuWD~6WIuR_=aiEfN+o4r*U`WRspd%YLERvI^zcd@KRvNZ>JmwTG{=`C z6!TDXx%T|t4fO%>P>a?#_MozDa-849Q0k1*hKYwTF6 z+fuRQdv)U;HG`#Ll*I*Q?V%G8&;xj4?vy7TeHeoPRpvWeRJIz7O-=vHi61t8lRc4sm&`ZV z(LSvCZiIQI&5r~Ytr8ZXBKu3;!=is4Ct_IOv0MB7L2wc@iTi+o6F9`w?mgl2Xg&Ao z^0t$%GGi{OkpWMJ77k+Fnpig^;nu#mx5!?QX@zA>EZg=khfWUfF5WEQb zrUCq~ByA!SLxZKthkbJCg7-<{?!g3^rfu#x32}8ys znbJ!QHWe#~PBd;h9hCL^jOe!68A{B)y5>tY$>Ev-xO=1N;t&uJM(?@XtnN?5TRS^& zMYc?YOPonceF!3uO-ZsT0><52ok_SI+bIkMR)=|jL{M7eBdh+XX2C?4;+FfWw)ut8 zB<4tx3iifUVfl2er?34_*Vt;l#zYgqb#YQ7OdhJ>aC}E-KAqF35~vVMEdtT9z0bs! zdz;PYswk60PaA#Y&Di1kLP0_;EQ7^B&Smo)!H{}c%67(j*|bjKI({?C4Q$IB=TLdzT$2=;LRu>0pmi;6z_)7Y&BF$*L$2fkGpEXk#S_x(-ZVJEWe(NX#QT zaE%7l@}zu*;{M2apb{}GGOt-+j!5B-_M+nnXn$&Ejgol5tp@m1!o5V1J^6QWc=+P( z-}{VjN;a#_4E~e-okx>e9X9I_+sx+6{GCqWJ<}z(T+HvL30i;XX#Inh%~hMvyv6=1 z49sVW;#H5ideh@%cz>FI(nSM}iyTiNQ_7uY2)(`4k2bA*i&P|L4uo6)L{*8tO6t zdfACK>SwQng@y@#rjReZrE-;~1%!BUSH=-^Z1%*=2QKSgOff_Paq*z^SxNvW%=4f$ zWP%_iVZ-5D4ypWAXq?bGna38v6>x6BX~Wd{l;$a4A{Ktqc>BClj>lMrpTJl6JX=MPJJxy=HDL_kD)8`wJA@nyWpW3Y3)5(_8msgSE?yH2m*tFiLrbM_Hn2bBr5oHxe$P&ZVfZz8sccH& zpP&EJMputkothySaW9Fl<88Wdp5)PH(rg!D<1XaZLb~6nR8@#CXNc>3Qp{!R9Tt^X z`pQVUcDRT9r3CUO^q*If7c*3z*`57i@E1LfMKR{Q-cp|WDchCjWt8F2Om2J81_tq~ zox1l8d*;Sa=5o_`UckTRRH|I&T|RHCn$F%ZIa|fZYPn@r0ml3oa~L{-xalWo&H7B9 zlY)%Ly<9}-3FHZUqcyAf5-vKqHzI5E`E>p6HgH0V@RJw`925-f(1Ucn@m(S@1xvso z!1`CZP=47U*K@U?m;<#M^;j+HH@e!}tF7Zg&BI1ranhrQJZAD8pbCYTJd5sj#?CPu z$#TH)xH}k!f*BQ7VER_;#U}X3fNd1OcJOk4@t|iUJ1f=z}k0t_u+0djiZ7~ z2Q0XGODRZNZyO!!w7Ek>YABqo{9do;rRS;}5;S_PsUOnSDi-M_Nx7McME~pLdbs#I z$|GJLlwsp+&}OZFxp-EZ>9oJTlXc`sWd*w1bb~;p3(B?7KqmUi#Jb-1Y(c7|g-Ydj zFfd9h>CYl6o2FIo;>d;Wzk2syA1)31sNosLhPx-X0__A-(t=518IbU9=okq6{QQKK zg+PAbpdza~B$%K{&_Qvoj1*eoqoc1E)GNgoGlN6|F5K7l=a{Z9#HFCFVt;>hL!Ov0 zk2`&Y3pQ$mR2K(LX2-Va!QzQmWlB*pz>cT-CZ(3BRaFSvU!MMj|Ux51jD z1-)P}w=93lK-=s)!$~j`RlVJ1T5tJ)SR6@2K(dM!wk=>V9R=f%_91F2_I6QI=2GT=gk?>e_k2sFW zGaAnW!e2s|V^4?wSTaXus}A4@z5+%&T8Ta!utvJ9`9hOkm+6{6>S!c3`sV8lKzDe_ z^iAm_Y(OJZ5KQBTJ_H$*)X7M)K}OfR3XA0=OTmclQsM_Qo$8_m5}9_MmusupsF)w@ z(cSN7-nhIT0kh_FipfCZl~IpmOtxY=V@zSJNC!23?&`cO)4FoeSd7 zVYmt6r~TsUe*?3DXpA$YlI=?$Q}J7|77|))Hg^5oWvGF3P>(no4KYG_8T<8zJl zI9b*vYXGe3)u)r%2;0h3Ur>&8v-#@J4lM?jW?QaD?_rI!hZzMPz!8*Z51TqYHr3|s zy6AZjmifyK-aou?`Z@#U3fBOS*K1ktsNn&@Kd9sA!76RlfPyskQ2-sxIZ zWg4)~*WuuqsZg4~Ol3@$lkFpor(Cp$fDgw z%%C@1YhZ9_=f0yV)IvUGhC%OjzLta-hB_Ed?ZJs4sm?D@NHBWJA3x;rolqzeY#5E6skdR%v-(6OUg?K*G$1R;dj|EN;}EuY4s z_cw=`qbyk++>+fpp;ppHXI~KXAb*7uAd(beob(*PpTRxV`L=ulO{hTJq5FztBk@MspFUXhX-PR0aC)b{hEvYSEX z?+aooQEGQe%;7}8`{_!J z2qpKV$<|WpL(~mj<9e+~FQ5ayqTl07065@c%tCVM)Bq|lnvQGwAf)c>?;t+cCx0FPhr@?nv7EzW`@Yz<(UPSI<{bC zx9kk*9dw%DEwF1ixDZob=H#I5o}13L#+8H$dP~$OgOSV_L-8d126V*(j@bH3AB%M`o8zPOLDR&Pe@FveMKQd8L&He? z_8-~e{>tiMU`ik#)lx8t)^mWOkK~UH_|x$U`Xdd)Q0?{a#3*Mj#f=09#I+`bGaOC9 z^m?G+Nq?`Z*TTQEtKXGtpM2`t5A03@a3O;xGP!TS)c&Ca&aEXwtw@~cGsf6kV7V?x z%4nGRlO&OL;`%IqWe&%5+r*$J6@8!~+YTnHOHpU^%HYXh*n?y?H#EG(lSHH!3ft-> zF^4WE+0qLSjxk!^`S~Xgfy?^O5io&MVQD-9MR(5R34PqSIHrPMSqj|WeNE?@4QLNj$u1;+9G^BF?10a?`y*y2=%K~ zp0p3B>6XwM&D#$WdV0mx6HyOn=`#5$>HA+C05!0y+}7>SKHbS?Z`78xC-@BO=!~r= zZJLRb_6~yM9H^#vTqjnKazZWcRR@vyF;(4c0F(*ygKG}T4b>aRTl|su=R1vLM=K0> zzf%6L-@DVyKL|ri%wNyF-emPJdZV^1DP2c+bN$jjyrui&@AJ$5f;DK^Xf<6Y+dVZ~ zXE-<*144jk?jN`WknfX3$Y~$nz9-8T5B$%|e*m;OFZzy(a!=vc3CjEnX{nmalv^z?+sQulhKX+8u5L-18qoGTa zy3a9Sv<5`Ok$%u)Z!`1zsdaPFg9kyyDX^FA8&lJ-&1Hq zhiA)dXIj@fu7@{|>9FH(`edXOzE{*eloh+>nRtAcaPC8V+*5aVAAxK_+6rmC9*E5} zDL?;KE5gx&p*lQ5i4naRBja3Jrj8Z*A=ntVZ0Sc`1eGSf7rMwXt=TRVgDv*Q%4eHH zkz+QV?>jZx!XtP;b)*qh*Y5I7yr3#AbAm^03NSOtGVtdAe1@km-R9le{(ntw0?G1gtA7B1uuF08m zotd23d+oK?+Ms6UsPZD4QY;ZH3Muy-ymbhX;Hm;M4{rawHUmL>OM06r6ba*Db5+9_ z6Hu+rMwFcqdc`Ua1r%1z6@AYY^oB`By>|#OV#51cCj9@jekkODHB*eQ69~GNvSx|4 zvd0&itCr>mOOj zVd>=#%NM;`^&U{&AVP!KY&S4PXz)We|GsD5Ce6TV`0}Rutu#mVCMzL^9-`h8gF+G+ z(VYr4SAIcAc?e?#I0+)B*mk|EA^-750WAYq^QoUA7CEf)(l$T~M;A#iHO?Y+?vxmF zuTrdzy7R)C5TYSx)NtFF3$QBYEKF`wK%HQs_uCXa)w3+qR?LARvW}eM?*UH)S56I&EJ)V=y?BMNDS^4U)x{=Wwh?ay!!Ct=uj!12`W3-4>3QzT8B62Q zJO~u7Iv4=(%n{04#7NIdvTTbnOgK<0;!Jq$qQz5{RY$maCiW(bqB+K04dcXd*$)lE zC8yek(HD_JGDuTap`CCj_p=RMI>>-2HBpK|?82R&_3EC2h*UsU8p}g^LLplrdLQl} zW~bTU%&hl8EkC{?fvlm;d)gDWdiMV4oKYVOCY=(Pmym6h-hyF_M%=@)Sdz-(e zmfmjC+F<)zjmht~L8Ie5TaI>2L4kpue0p)*vSiF1a81z$=`_I%0TQ+00ax%XYg0 zSe7`aJh$H4YO8J9k;`jX39u9Eu3e>gnW;kvg=YJSK6pt{H=Yq&U4hF#d|l7 zDLIqL7Vi{&6L9Gz&#ZBW?F^fC0Pi(g{ZW|w4mQ;lbdQL-h4JmK+2$!_3d?uc%o*5B zJF;eKITGW>2HCJ#*O4TCxITh%bu$0igfci-PeEt6ykIwF_3F(gbk-Af`7D`yVBaiu z!@En;w;C48an%8TEMF}_`-#Za*0=J^#v>G51R{o6{ro(SKq|~-a5s$lGogA_m4Xqw zZf{tzZzNis_Y*f6pLNN_;jAkC=nc5l&5n+TyeaW=cy1^J>z)oBx?X|`$$aw7bDiO4 zn}8zd&51=4VPI`BzjMu;NhD%GeK=2Am@Kt5zP)TZoosg@FI#B{s(iGK&ic@#?JT~o zm|EX_b8|a{6{J%+RkBu#6{syo^cAWKIST|JnruU` z(0i(u@Uvk7=cKstJwLYn=L{N+kL@R!I4e%VvYW5JCWJydQ(#yrL$w4;#cEGN35eHUh9?7V3#o_grlU`(ZAyrZp1z)T=e<-_z$rU*XPQ z!Vl38Jbq;q!X1W~h0E|gCVHvK+_ZpJeda#2U;q~m#`{b7{RVi$jdiMkF{l>L4&dXF z^vBTgh~e)bz2O$L9sZfFpK^XI zZcZq9K1*bTMbMhv@4`NpP1#7|(0qlTGL$5d^TFj#eg4GsU4%c6C9rxV%!RUS*NKqz zdPnJrtxolH+e@9|Q+6r;WCaNOo@dBj3ucV}reDEuj)&@o9tySCCQWvKixBV2XlYu8_LkT}M z?@PS`(WO#}kJI^;v95EObc!q>FX>glGaO(?y7-KP`orztD>yb*OrlrXj8R>DqdU3Z zdGH@H_NsyUrwoB}1Eeb5qEGeh!IWDIp5+V8-w(X*8A!V>owuZxl`Yd~ zqNBkvH(i|0-5|WodRYsre^s)q_yi~ErucMP#RMJvx3`0)v`C&8Hyj0-#xlx@F+3^C z#mb2a*7WZ2%k=F0CT zu{uwIYoADcQ6CwivsTBzz`!%EeD6m=it?=l78EZ_85O=;BzKg^WUf$KU*F>;ph-zR zggNK8B*EpCm4Be?icBy)oxzhn)$tk35$dhd6^EQS(UaZZQu9-0=X&64Q~ zL&J>eXNg(6WwuV0Ot_lqR(#Z;ulx3aXG`FSgJaK03!9++EX!oAJr`pdU#d_bLhZLl z3*8Qzpc|^CY}X7`He$nZnOW@tJpUOWV=>6C=cLoCw0 z;fFZWx>&lfCriU$LQaof)ibU%FNGrC(MUU+o#prPNa$X>Ge3mW{OorpZEFnpIW#2j z1sU{kCR(lk%QNqqZVFFq_d(I@WrMRjCabkc%8c*1PR~1Wl~-h#l9?HO-9du|u!`Q- z+)@3TjB-)r9AcxfB4E{`JlCMS?n;*zOnUoG>rhFyS1*nh+7&qm2tb7MM~`>zBYJ44 ziLz*_R){{hW^JgSc*r_O5pcz^77zr1D0qq48H)|pgo1-4q!_NJ_PUCY!{^-;Okz2T zQZH&sgu=v9kjP+ZbjPQO2xaO&8LOsPUA={(R)%I(FlcP-lq7;h4nIY|Wn6*(d#@Tf z;fZ$ww>AFIGcj=!KfALq%1its{6eJf#9a*4dZO2xel>Iw6r`*eDA65!lSwvhLo??! ztKNI1XK@UFfPFvesr88uJ9^G#`1(MIVK4*^HDyMRM2wO23cJ(76m{I~S?tEdBeA z5K3aN?+mRn&dj#~H%GX4Yn@SVmfU%Pkb-TEyb~ z%Q6hE<*SDY5}{L65j7>D5< z?rimu+{=ET59r0Lj~$x)S<3F!WW5OV-2GX;d=tEnhv`K+iuu~=6zSG{qxlolaZ|0& zTu#tr`x<-N1ec`7Q8o853j55IbC=tGOm)NVvLf-wO&`B>$4zAfp(I|~X`+SPf7Btg z=`Q$X2>q$VzYs!VLK%wGiKPwFX88mnjy%@d7|G%4sC)Bz8Tom`$Tu^i>B6R1j2?E2 z9q27kcocG=3uanHAiQdEI_I52E7u9oR$2!Fqp&UEelJOwFopl#rTrZr)c1SZAZmf_ zGnY-3q~J;3Db&N;9#nG)POe($X!@1m8`?A6_6iC;31~NYlb@bKox~vX&|_uU^1R-m zbZgCTK+Lf0au*d5X|+@tz-_N1+dp_R8m{yOtACuiMnsN}ZRe4XRjP8sIgB->`}>j5auHuJ2u3i~|TKKo*FC{>M`; zWkh!)I{oC2;H>aVamfi25$n$?y#-$4M|Cd0RAQuP9z_ej+s51=NUo8R6|35o_Fip4MGin>`l?)bG@!EEb@y(670euWgdy zp65fUYYK0T{JB+KjORoolS&-12%aC&W6chN$;#6(Ap|ZY)p1u0N6srtvX@Vuu(Jf5 znPI5dc~jV}@8;}Ee+~6hX)%>z3HBb}3i6PNh#`XD)AugzA`@Tv@W;@K;2%grc>&di zo!~dd?8Xjcd~KbdJEpJd3rR;=(?<5E%h~KR`G6n*_S!2-x^uDjhKrhKKG)T>emTpN z&s5Brypema(Y=*BMHJZIC7P&)2X7;`o8h(95wiMYG8|t1O*LND zr>}L?8C~47feG$$C+hu>g#ENOzL#c#LXLynLy8_1?u66(Iw{1Jj*E#j;dlL)C=yIa z+o27%w1HnQ4pL3%E6{`_5n)7CAFCT$L75GF@j(P|$u^&nY{>7+1=$=AE4lBJ4T+Li zpY|6mzf*{OQ+mLgZ!{Y-v-6~KGlBA`XgM60Z=*|@cm1wqIcI^y}D%ENgQJmD7Ry@ouH%EMdXl)_*HAlh)w z&saAk3+E3w&l4X_3jG&0x+5Ca18rla6>XWYdnzHnE$d?YMM@Imn4_MsQ- zK)~vf{dAw04_FgmUZH-t4Q-z$K&twu0_F!_X-8O^?bUI)c-$?mSo>F-XuF;F%``fy zKq9{**i1fzB3jReH|eW#Hm3m6M9$EgX>(kulQ%p%%$2s+=s|5bC5Jn%iq;20??h(* za>OCPEe@vs{T{v?k zqCmZQJV(#Hyq?uCOpzqa@u@*j6hn(nGw@4KUaS813WwO15~Uoa_Ce3Cr=S5~@3j(+ zgj{}du+MMw@%}w0dgWCG3_6>_g#pizotq&p=krCNK$Z6=G>-BM`akvtJP>Ag zn$g~-bQ#UruUjjQAG+^eR+j0TCB-|X>J~rqoQc5O8cZX5=ofj?X)u_CF;QiEVG}yC z_zY*uPuXp z%|nfK&!RI0w%1fjwln%bEk|CAuRHH-W3^Uk$h7t9N4F#4w_&l_aZ@_!K%_{3es%_! z4tRcx#)0aWNBjWle58&dF@ftzsNp@~xZ%|JZ>cYhh~@~rV#g2$I#BZ=gPv3cJD#r1 z*iumI=%llPY$LXuv5_!Rne_DeMLC5Z?|Z0D7}&*Tt97IN$Aa)A4#M^guZvzdT|_UW zur%C5`(D&KFpbG5;_xcG(VTv+yJkV=)vXGSogu3v`o+i2viQXRo>7fIq;}1qx-4}# zltqDf6xY6er1^eybEg@C*hN)CzB{NbDvF-w{z`t5-Y57QFzdop8A4@lXVRx1T zrC&_R)k+3G9{GfI$puCAnEf_pE~I__Q;rC_nJ1trf0Vshb%#7F*8QrsZ^$ujN^H}S z1|le@Maz55+RY;kOMQPmeuCHz<1~VXxkKW;!~a??d98ql*Z%cbL_9?G~KYW z4n_FDY-O(x#P-;33lnVgtFDIa?fP&Q`QbVR;XbYBo7RYUl=)nAtKNDR+37dws24kf zm%`!U3g~HR(obiW#7vJ>T=7?>{ALmQwiAw4Ds%WN_QfM)Ss3kSmIjP9G1{01TxR86 z{E$zqP>NfQa2n+4o|l4=2+E6`N`7BokxNN-v{58URdPm*B`2j!x1Z>ZpYtx1hXqv; z0CGc2D^b>KeSlewG$&X^c>f}!C>M#sJ&Ge=g1C>4;hM^u> z-!HlpO{20Kl)Us8jM{Yo%;2;JU5np8J@*?c!z@NE!d!PfT`V;gE*tIO1RQVWfWXY( zeA=(l`07SwPrUqY!H#Li$IQ{Go2{|-nT;C;m1uIY#nzj$xG7oTMmu-zRkfWFMZ&B# zDLKRlRTOzJ|ENVYi^slUZN=OG8WST?NZ?lU!+Uh2MYu3W+2sv9d`sRZb?l^iPQ1)) z#G=_%-q9P@1K{=aKSU5A;}X6x(k>3^IT$Ae@M22J)a22iu(onEhm?GKhk~CG^WQt| zY7Z!$je&0Z_qZG64`!@)W&+q}$-!UZ%JM`x%650{Tu)%OAxS7-S;N$yw!Gy-MwkLdKjx zi)@-6@b+zur3v5QE8Ni!rgvWzrmg}&3%#LA7WNq@_FZulc)(PAdNcY>3~WqI3)j1) zT}w2)zMJ&oah#~c456t`&a|k$DO*LA#_aj*%$5Wd??HfT~xRlnrX^rK03JTYrESOJjE$$7>=b23QsTbEo^vnMpe~D;!G~Kutc>22 zKflynMDS4BmSc6Ys177n5#z(fd z35PU0g?3Lo2OoR?sRKhRm0enDg&m^KdSW-wmYIan9#uWR>BPqg+_*+&0@!irWKR`z zsFApQKWjWiGz+K;Gr{x1x4bXZ;I**Ns)xrKHfATS?Na`~VuY8;Tl;)a{!&8yFUGNz zqOVWnmq9g@`EVQw2=DyxFdfMHI6%wiTREamFavmy#RCM7Keg@1qZA5SZTv^wSmf)W9Cl_$7ny5&)S+SAtZZ^NDZ$9X+D7Vr8cSO$+%xX(jF zNld5+RpS9+|Cx~vjRM@Ca4fh#_;Mk{bXq%XIkFLKI95HE%;Wj7C z-5gVyh}Xvo0A}VuNrYD_&WL1(Piw0ShedeOA3fOGCbT}YJS7F7pD^AL`&`{2s8I*P zWNAq3Cx48+4Wyt;L&$h^lBQ<6x*MbY(34f)m`lW)8J(27dbCPtOlY@>@pW^ky};=> zLIZWvTSILaLJx_SB`|N#E&W0kw#2VHwrSxb(*b$29;x@n`c3n4ZC7Q()CO!5Zvgs$>eq5x*T*(tR`-Po_ZETN`eqjJungN)b zB$V_zu=1{L%4n+`whqFDUwMEGqJhdW+K>SS@vBja%g)&p&&Vy{sDK*=%psdItwk{Z z0@)dJ)^Upy*UqBi(fFLg^-5$_MkPuf)q=B|odwg$y<*e6QLXek0$&D-|42~Y0u#mP zHFqp++FB16!UsQpACq|=45LB_Xa~O4w0u8*s}6wkjcxM<({oRo^w2ke{s<^<5fA|f zX$Ygf16{X*05Jq4pnq2thFQ)+r(@7+GJV}?AOeaAJ))(thuq}lWp%9XTd+6+l0IEm z_M_G*H4fsxpUjH}DAljonXW#usv-Ux57y z+6(#LrKPCitgjxTK<1~X82`@V0GMS6yI*mnP^SKy9xz0#WWmMq|39I!7uH`Bt@yvm zicuh?f4Jx0hNr;)4up&OTYx?#p921U&BvHHkbEz?P4zN`w@8Q?KEdFAntwA4rau8S XSqWN3&&@_4z(4ZRDpFMvCV~G45VaSG literal 0 HcmV?d00001 diff --git a/main.go b/main.go new file mode 100644 index 00000000..9e68368f --- /dev/null +++ b/main.go @@ -0,0 +1,143 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package main + +import ( + "flag" + "os" + "strconv" + + "k8s.io/apimachinery/pkg/runtime" + utilruntime "k8s.io/apimachinery/pkg/util/runtime" + clientgoscheme "k8s.io/client-go/kubernetes/scheme" + _ "k8s.io/client-go/plugin/pkg/client/auth/gcp" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/log/zap" + + databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + databasecontroller "github.com/oracle/oracle-database-operator/controllers/database" + // +kubebuilder:scaffold:imports +) + +var ( + scheme = runtime.NewScheme() + setupLog = ctrl.Log.WithName("setup") +) + +func init() { + utilruntime.Must(clientgoscheme.AddToScheme(scheme)) + + utilruntime.Must(databasev1alpha1.AddToScheme(scheme)) + // +kubebuilder:scaffold:scheme +} + +func main() { + var metricsAddr string + var enableLeaderElection bool + flag.StringVar(&metricsAddr, "metrics-addr", ":8080", "The address the metric endpoint binds to.") + flag.BoolVar(&enableLeaderElection, "enable-leader-election", false, + "Enable leader election for controller manager. "+ + "Enabling this will ensure there is only one active controller manager.") + flag.Parse() + + ctrl.SetLogger(zap.New(zap.UseDevMode(true))) + + mgr, err := ctrl.NewManager(ctrl.GetConfigOrDie(), ctrl.Options{ + Scheme: scheme, + MetricsBindAddress: metricsAddr, + Port: 9443, + LeaderElection: enableLeaderElection, + LeaderElectionID: "a9d608ea.oracle.com", + }) + if err != nil { + setupLog.Error(err, "unable to start manager") + os.Exit(1) + } + + if err = (&databasecontroller.AutonomousDatabaseReconciler{ + KubeClient: mgr.GetClient(), + Log: ctrl.Log.WithName("controllers").WithName("database").WithName("AutonomousDatabase"), + Scheme: mgr.GetScheme(), + }).SetupWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create controller", "controller", "AutonomousDatabase") + os.Exit(1) + } + if err = (&databasecontroller.SingleInstanceDatabaseReconciler{ + Client: mgr.GetClient(), + Log: ctrl.Log.WithName("controllers").WithName("database").WithName("SingleInstanceDatabase"), + Scheme: mgr.GetScheme(), + Config: mgr.GetConfig(), + Recorder: mgr.GetEventRecorderFor("SingleInstanceDatabase"), + }).SetupWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create controller", "controller", "SingleInstanceDatabase") + os.Exit(1) + } + if err = (&databasecontroller.ShardingDatabaseReconciler{ + Client: mgr.GetClient(), + Log: ctrl.Log.WithName("controllers").WithName("database").WithName("ShardingDatabase"), + Scheme: mgr.GetScheme(), + Recorder: mgr.GetEventRecorderFor("ShardingDatabase"), + }).SetupWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create controller", "controller", "ShardingDatabase") + os.Exit(1) + } + // Set RECONCILE_INTERVAL environment variable if you want to change the default value from 15 secs + interval := os.Getenv("RECONCILE_INTERVAL") + i, err := strconv.ParseInt(interval, 10, 64) + if err != nil { + i = 15 + setupLog.Info("Setting default reconcile period for database-controller", "Secs", i) + } + + // Set ENABLE_WEBHOOKS=false when we run locally to skip webhook part when testing just the controller. Not to be used in production. + if os.Getenv("ENABLE_WEBHOOKS") != "false" { + if err = (&databasev1alpha1.SingleInstanceDatabase{}).SetupWebhookWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create webhook", "webhook", "SingleInstanceDatabase") + os.Exit(1) + } + } + + // +kubebuilder:scaffold:builder + + setupLog.Info("starting manager") + if err := mgr.Start(ctrl.SetupSignalHandler()); err != nil { + setupLog.Error(err, "problem running manager") + os.Exit(1) + } +} diff --git a/oracle-database-operator.yaml b/oracle-database-operator.yaml new file mode 100644 index 00000000..ae464a57 --- /dev/null +++ b/oracle-database-operator.yaml @@ -0,0 +1,1308 @@ +apiVersion: v1 +kind: Namespace +metadata: + labels: + control-plane: controller-manager + name: oracle-database-operator-system +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.6.1 + creationTimestamp: null + name: autonomousdatabases.database.oracle.com +spec: + group: database.oracle.com + names: + kind: AutonomousDatabase + listKind: AutonomousDatabaseList + plural: autonomousdatabases + shortNames: + - adb + - adbs + singular: autonomousdatabase + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .status.displayName + name: Display Name + type: string + - jsonPath: .status.lifecycleState + name: State + type: string + - jsonPath: .status.isDedicated + name: Dedicated + type: string + - jsonPath: .status.cpuCoreCount + name: OCPUs + type: integer + - jsonPath: .status.dataStorageSizeInTBs + name: Storage (TB) + type: integer + - jsonPath: .status.dbWorkload + name: Workload Type + type: string + - jsonPath: .status.timeCreated + name: Created + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + description: AutonomousDatabase is the Schema for the autonomousdatabases API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: 'AutonomousDatabaseSpec defines the desired state of AutonomousDatabase Important: Run "make" to regenerate code after modifying this file' + properties: + details: + description: AutonomousDatabaseDetails defines the detail information of AutonomousDatabase, corresponding to oci-go-sdk/database/AutonomousDatabase + properties: + adminPassword: + properties: + k8sSecretName: + type: string + ociSecretOCID: + type: string + type: object + autonomousDatabaseOCID: + type: string + compartmentOCID: + type: string + cpuCoreCount: + type: integer + dataStorageSizeInTBs: + type: integer + dbName: + type: string + dbVersion: + type: string + dbWorkload: + description: 'AutonomousDatabaseDbWorkloadEnum Enum with underlying type: string' + enum: + - OLTP + - DW + - AJD + - APEX + type: string + displayName: + type: string + freeformTags: + additionalProperties: + type: string + type: object + isAutoScalingEnabled: + type: boolean + isDedicated: + type: boolean + lifecycleState: + description: 'AutonomousDatabaseLifecycleStateEnum Enum with underlying type: string' + type: string + nsgOCIDs: + items: + type: string + type: array + privateEndpoint: + type: string + privateEndpointIP: + type: string + privateEndpointLabel: + type: string + subnetOCID: + type: string + wallet: + properties: + name: + type: string + password: + properties: + k8sSecretName: + type: string + ociSecretOCID: + type: string + type: object + type: object + type: object + hardLink: + default: false + type: boolean + ociConfig: + properties: + configMapName: + type: string + secretName: + type: string + type: object + required: + - details + type: object + status: + description: AutonomousDatabaseStatus defines the observed state of AutonomousDatabase + properties: + cpuCoreCount: + type: integer + dataStorageSizeInTBs: + type: integer + dbWorkload: + description: 'AutonomousDatabaseDbWorkloadEnum Enum with underlying type: string' + type: string + displayName: + description: 'INSERT ADDITIONAL STATUS FIELD - define observed state of cluster Important: Run "make" to regenerate code after modifying this file' + type: string + isDedicated: + type: string + lifecycleState: + description: 'AutonomousDatabaseLifecycleStateEnum Enum with underlying type: string' + type: string + timeCreated: + type: string + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.6.1 + creationTimestamp: null + name: shardingdatabases.database.oracle.com +spec: + group: database.oracle.com + names: + kind: ShardingDatabase + listKind: ShardingDatabaseList + plural: shardingdatabases + singular: shardingdatabase + scope: Namespaced + versions: + - name: v1alpha1 + schema: + openAPIV3Schema: + description: ShardingDatabase is the Schema for the shardingdatabases API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: ShardingDatabaseSpec defines the desired state of ShardingDatabase + properties: + catalog: + items: + description: CatalogSpec defines the desired state of CatalogSpec + properties: + envVars: + items: + description: EnvironmentVariable represents a named variable accessible for containers. + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + imagePullPolicy: + description: PullPolicy describes a policy for if/when to pull a container image + type: string + isDelete: + type: boolean + label: + type: string + name: + type: string + nodeSelector: + additionalProperties: + type: string + type: object + pvAnnotations: + additionalProperties: + type: string + type: object + pvMatchLabels: + additionalProperties: + type: string + type: object + pvcName: + type: string + resources: + description: ResourceRequirements describes the compute resource requirements. + properties: + limits: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + type: object + requests: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + type: object + type: object + storageSizeInGb: + format: int32 + type: integer + required: + - name + type: object + type: array + dbImage: + type: string + dbImagePullSecret: + type: string + gsm: + items: + description: GsmSpec defines the desired state of GsmSpec + properties: + envVars: + items: + description: EnvironmentVariable represents a named variable accessible for containers. + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + imagePullPolicy: + description: PullPolicy describes a policy for if/when to pull a container image + type: string + isDelete: + type: boolean + label: + type: string + name: + type: string + nodeSelector: + additionalProperties: + type: string + type: object + pvMatchLabels: + additionalProperties: + type: string + type: object + pvcName: + type: string + replicas: + format: int32 + type: integer + resources: + description: ResourceRequirements describes the compute resource requirements. + properties: + limits: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + type: object + requests: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + type: object + type: object + storageSizeInGb: + format: int32 + type: integer + required: + - name + type: object + type: array + gsmImage: + type: string + gsmImagePullSecret: + type: string + isClone: + type: boolean + isDataGuard: + type: boolean + isDebug: + type: boolean + isDeleteOraPvc: + type: boolean + isExternalSvc: + type: boolean + namespace: + type: string + nsConfigMap: + type: string + nsSecret: + type: string + portMappings: + items: + description: PortMapping is a specification of port mapping for an application deployment. + properties: + port: + format: int32 + type: integer + protocol: + default: TCP + type: string + targetPort: + format: int32 + type: integer + required: + - port + - protocol + - targetPort + type: object + type: array + scriptsLocation: + type: string + secret: + type: string + shard: + description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster Important: Run "make" to regenerate code after modifying this file' + items: + description: ShardSpec is a specification of Shards for an application deployment. + properties: + envVars: + items: + description: EnvironmentVariable represents a named variable accessible for containers. + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + imagePullPolicy: + description: PullPolicy describes a policy for if/when to pull a container image + type: string + isDelete: + type: boolean + label: + type: string + name: + type: string + nodeSelector: + additionalProperties: + type: string + type: object + pvAnnotations: + additionalProperties: + type: string + type: object + pvMatchLabels: + additionalProperties: + type: string + type: object + pvcName: + type: string + resources: + description: ResourceRequirements describes the compute resource requirements. + properties: + limits: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + type: object + requests: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + type: object + type: object + storageSizeInGb: + format: int32 + type: integer + required: + - name + type: object + type: array + stagePvcName: + type: string + storageClass: + type: string + required: + - catalog + - dbImage + - gsm + - gsmImage + - secret + - shard + type: object + status: + description: To understand Metav1.Condition, please refer the link https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1 ShardingDatabaseStatus defines the observed state of ShardingDatabase + properties: + catalogs: + additionalProperties: + type: string + type: object + conditions: + items: + description: "Condition contains details for one aspect of the current state of this API Resource. --- This struct is intended for direct use as an array at the field path .status.conditions. For example, type FooStatus struct{ // Represents the observations of a foo's current state. // Known .status.conditions.type are: \"Available\", \"Progressing\", and \"Degraded\" // +patchMergeKey=type // +patchStrategy=merge // +listType=map // +listMapKey=type Conditions []metav1.Condition `json:\"conditions,omitempty\" patchStrategy:\"merge\" patchMergeKey:\"type\" protobuf:\"bytes,1,rep,name=conditions\"` \n // other fields }" + properties: + lastTransitionTime: + description: lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: message is a human readable message indicating details about the transition. This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. --- Many .condition.type values are consistent across resources like Available, but because arbitrary conditions can be useful (see .node.status.conditions), the ability to deconflict is important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + gsm: + properties: + details: + additionalProperties: + type: string + type: object + externalConnectStr: + type: string + internalConnectStr: + type: string + services: + type: string + shards: + additionalProperties: + type: string + type: object + state: + type: string + type: object + shards: + additionalProperties: + type: string + type: object + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + cert-manager.io/inject-ca-from: oracle-database-operator-system/oracle-database-operator-serving-cert + controller-gen.kubebuilder.io/version: v0.6.1 + name: singleinstancedatabases.database.oracle.com +spec: + group: database.oracle.com + names: + kind: SingleInstanceDatabase + listKind: SingleInstanceDatabaseList + plural: singleinstancedatabases + singular: singleinstancedatabase + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .status.edition + name: Edition + type: string + - jsonPath: .status.status + name: Status + type: string + - jsonPath: .status.role + name: Role + priority: 1 + type: string + - jsonPath: .status.releaseUpdate + name: Version + type: string + - jsonPath: .status.connectString + name: Connect Str + type: string + - jsonPath: .status.pdbConnectString + name: Pdb Connect Str + priority: 1 + type: string + - jsonPath: .status.oemExpressUrl + name: Oem Express Url + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + description: SingleInstanceDatabase is the Schema for the singleinstancedatabases API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: SingleInstanceDatabaseSpec defines the desired state of SingleInstanceDatabase + properties: + adminPassword: + description: SingleInsatnceAdminPassword defines the secret containing Admin Password mapped to secretKey for Database + properties: + keepSecret: + type: boolean + secretKey: + type: string + secretName: + type: string + required: + - secretKey + - secretName + type: object + archiveLog: + type: boolean + charset: + type: string + cloneFrom: + type: string + edition: + enum: + - standard + - enterprise + type: string + flashBack: + type: boolean + forceLog: + type: boolean + image: + description: SingleInstanceDatabaseImage defines the Image source and pullSecrets for POD + properties: + pullFrom: + type: string + pullSecrets: + type: string + version: + type: string + required: + - pullFrom + type: object + initParams: + description: SingleInstanceDatabaseInitParams defines the Init Parameters + properties: + cpuCount: + type: integer + pgaAggregateTarget: + type: integer + processes: + type: integer + sgaTarget: + type: integer + type: object + installApex: + type: boolean + loadBalancer: + type: boolean + nodeSelector: + additionalProperties: + type: string + type: object + pdbName: + type: string + persistence: + description: SingleInstanceDatabasePersistence defines the storage size and class for PVC + properties: + accessMode: + enum: + - ReadWriteOnce + - ReadWriteMany + type: string + size: + type: string + storageClass: + type: string + required: + - accessMode + - size + - storageClass + type: object + readinessCheckPeriod: + type: integer + replicas: + minimum: 1 + type: integer + sid: + description: SID can only have a-z , A-Z, 0-9 . It cant have any special characters + pattern: ^[a-zA-Z0-9]+$ + type: string + required: + - adminPassword + - image + - persistence + - replicas + type: object + status: + description: SingleInstanceDatabaseStatus defines the observed state of SingleInstanceDatabase + properties: + apexInstalled: + type: boolean + archiveLog: + type: string + charset: + type: string + cloneFrom: + type: string + clusterConnectString: + type: string + conditions: + items: + description: "Condition contains details for one aspect of the current state of this API Resource. --- This struct is intended for direct use as an array at the field path .status.conditions. For example, type FooStatus struct{ // Represents the observations of a foo's current state. // Known .status.conditions.type are: \"Available\", \"Progressing\", and \"Degraded\" // +patchMergeKey=type // +patchStrategy=merge // +listType=map // +listMapKey=type Conditions []metav1.Condition `json:\"conditions,omitempty\" patchStrategy:\"merge\" patchMergeKey:\"type\" protobuf:\"bytes,1,rep,name=conditions\"` \n // other fields }" + properties: + lastTransitionTime: + description: lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: message is a human readable message indicating details about the transition. This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. --- Many .condition.type values are consistent across resources like Available, but because arbitrary conditions can be useful (see .node.status.conditions), the ability to deconflict is important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + connectString: + type: string + datafilesCreated: + type: string + datafilesPatched: + type: string + edition: + type: string + flashBack: + type: string + forceLog: + type: string + initParams: + description: SingleInstanceDatabaseInitParams defines the Init Parameters + properties: + cpuCount: + type: integer + pgaAggregateTarget: + type: integer + processes: + type: integer + sgaTarget: + type: integer + type: object + initPgaSize: + type: integer + initSgaSize: + type: integer + nodes: + items: + type: string + type: array + oemExpressUrl: + type: string + ordsReference: + type: string + pdbConnectString: + type: string + pdbName: + type: string + persistence: + description: SingleInstanceDatabasePersistence defines the storage size and class for PVC + properties: + accessMode: + enum: + - ReadWriteOnce + - ReadWriteMany + type: string + size: + type: string + storageClass: + type: string + required: + - accessMode + - size + - storageClass + type: object + releaseUpdate: + type: string + replicas: + type: integer + role: + type: string + sid: + type: string + standbyDatabases: + additionalProperties: + type: string + type: object + status: + type: string + required: + - persistence + - replicas + type: object + type: object + served: true + storage: true + subresources: + scale: + specReplicasPath: .spec.replicas + statusReplicasPath: .status.replicas + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: oracle-database-operator-leader-election-role + namespace: oracle-database-operator-system +rules: +- apiGroups: + - "" + resources: + - configmaps + verbs: + - get + - list + - watch + - create + - update + - patch + - delete +- apiGroups: + - coordination.k8s.io + resources: + - leases + verbs: + - get + - list + - watch + - create + - update + - patch + - delete +- apiGroups: + - "" + resources: + - configmaps/status + verbs: + - get + - update + - patch +- apiGroups: + - "" + resources: + - events + verbs: + - create + - patch +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + creationTimestamp: null + name: oracle-database-operator-manager-role +rules: +- apiGroups: + - "" + resources: + - configmaps + - secrets + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - "" + resources: + - events + - nodes + - persistentvolumeclaims + - pods + - pods/exec + - pods/log + - services + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - '''''' + resources: + - statefulsets/finalizers + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - apps + resources: + - statefulsets + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - coordination.k8s.io + resources: + - leases + verbs: + - create + - get + - list + - update +- apiGroups: + - "" + resources: + - configmaps + - events + - namespaces + - nodes + - persistentvolumeclaims + - pods + - pods/exec + - pods/log + - secrets + - services + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - "" + resources: + - pods/exec + verbs: + - create +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabases + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabases/status + verbs: + - patch + - update +- apiGroups: + - database.oracle.com + resources: + - shardingdatabases + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - database.oracle.com + resources: + - shardingdatabases/finalizers + verbs: + - create + - delete + - get + - patch + - update +- apiGroups: + - database.oracle.com + resources: + - shardingdatabases/status + verbs: + - get + - patch + - update +- apiGroups: + - database.oracle.com + resources: + - singleinstancedatabases + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - database.oracle.com + resources: + - singleinstancedatabases/finalizers + verbs: + - update +- apiGroups: + - database.oracle.com + resources: + - singleinstancedatabases/status + verbs: + - get + - patch + - update +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: oracle-database-operator-metrics-reader +rules: +- nonResourceURLs: + - /metrics + verbs: + - get +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: oracle-database-operator-oracle-database-operator-proxy-role +rules: +- apiGroups: + - authentication.k8s.io + resources: + - tokenreviews + verbs: + - create +- apiGroups: + - authorization.k8s.io + resources: + - subjectaccessreviews + verbs: + - create +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: oracle-database-operator-oracle-database-operator-leader-election-rolebinding + namespace: oracle-database-operator-system +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: oracle-database-operator-leader-election-role +subjects: +- kind: ServiceAccount + name: default + namespace: oracle-database-operator-system +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: oracle-database-operator-oracle-database-operator-manager-rolebinding +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: oracle-database-operator-manager-role +subjects: +- kind: ServiceAccount + name: default + namespace: oracle-database-operator-system +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: oracle-database-operator-oracle-database-operator-proxy-rolebinding +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: proxy-role +subjects: +- kind: ServiceAccount + name: default + namespace: oracle-database-operator-system +--- +apiVersion: v1 +kind: Service +metadata: + labels: + control-plane: controller-manager + name: oracle-database-operator-controller-manager-metrics-service + namespace: oracle-database-operator-system +spec: + ports: + - name: https + port: 8443 + targetPort: https + selector: + control-plane: controller-manager +--- +apiVersion: v1 +kind: Service +metadata: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system +spec: + ports: + - port: 443 + protocol: TCP + targetPort: 9443 + selector: + control-plane: controller-manager +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + labels: + control-plane: controller-manager + name: oracle-database-operator-controller-manager + namespace: oracle-database-operator-system +spec: + replicas: 3 + selector: + matchLabels: + control-plane: controller-manager + template: + metadata: + labels: + control-plane: controller-manager + spec: + containers: + - args: + - --enable-leader-election + command: + - /manager + image: container-registry.oracle.com/database/operator:0.1.0 + imagePullPolicy: Always + name: manager + ports: + - containerPort: 9443 + name: webhook-server + protocol: TCP + resources: + limits: + cpu: 400m + memory: 400Mi + requests: + cpu: 400m + memory: 400Mi + volumeMounts: + - mountPath: /tmp/k8s-webhook-server/serving-certs + name: cert + readOnly: true + imagePullSecrets: + - name: container-registry-secret + terminationGracePeriodSeconds: 10 + volumes: + - name: cert + secret: + defaultMode: 420 + secretName: webhook-server-cert +--- +apiVersion: cert-manager.io/v1 +kind: Certificate +metadata: + name: oracle-database-operator-serving-cert + namespace: oracle-database-operator-system +spec: + dnsNames: + - oracle-database-operator-webhook-service.oracle-database-operator-system.svc + - oracle-database-operator-webhook-service.oracle-database-operator-system.svc.cluster.local + issuerRef: + kind: Issuer + name: oracle-database-operator-selfsigned-issuer + secretName: webhook-server-cert +--- +apiVersion: cert-manager.io/v1 +kind: Issuer +metadata: + name: oracle-database-operator-selfsigned-issuer + namespace: oracle-database-operator-system +spec: + selfSigned: {} +--- +apiVersion: admissionregistration.k8s.io/v1 +kind: MutatingWebhookConfiguration +metadata: + annotations: + cert-manager.io/inject-ca-from: oracle-database-operator-system/oracle-database-operator-serving-cert + name: oracle-database-operator-mutating-webhook-configuration +webhooks: +- admissionReviewVersions: + - v1 + - v1beta1 + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /mutate-database-oracle-com-v1alpha1-singleinstancedatabase + failurePolicy: Fail + name: msingleinstancedatabase.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + resources: + - singleinstancedatabases + sideEffects: None +--- +apiVersion: admissionregistration.k8s.io/v1 +kind: ValidatingWebhookConfiguration +metadata: + annotations: + cert-manager.io/inject-ca-from: oracle-database-operator-system/oracle-database-operator-serving-cert + name: oracle-database-operator-validating-webhook-configuration +webhooks: +- admissionReviewVersions: + - v1 + - v1beta1 + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /validate-database-oracle-com-v1alpha1-singleinstancedatabase + failurePolicy: Fail + name: vsingleinstancedatabase.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + - DELETE + resources: + - singleinstancedatabases + sideEffects: None diff --git a/set_ocicredentials.sh b/set_ocicredentials.sh new file mode 100755 index 00000000..fe2e5a5c --- /dev/null +++ b/set_ocicredentials.sh @@ -0,0 +1,206 @@ +#!/bin/bash +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. + +# Parse command line arguments +POSITIONAL=() +while [[ $# -gt 0 ]]; do + option_key="$1" + + case $option_key in + run) + COMMAND="$1" + shift # past argument + ;; + -configmap) + CONFIGMAP_NAME="$2" + shift # past argument + shift # past value + ;; + -secret) + SECRET_NAME="$2" + shift # past argument + shift # past value + ;; + -n|-namespace) + NAMESPACE="$2" + shift # past argument + shift # past value + ;; + -path) + OCI_PATH="$2" + shift # past argument + shift # past value + ;; + -profile) + PROFILE="$2" + shift # past argument + shift # past value + ;; + -h|-help) + cat <", 0)) + } +} + +func CleanupDB(k8sClient *client.Client, dbClient *database.DatabaseClient, namespace string) func() { + return func() { + + Expect(k8sClient).NotTo(BeNil()) + Expect(dbClient).NotTo(BeNil()) + + derefK8sClient := *k8sClient + derefDBClient := *dbClient + + adbList := &dbv1alpha1.AutonomousDatabaseList{} + options := &client.ListOptions{ + Namespace: namespace, + } + derefK8sClient.List(context.TODO(), adbList, options) + + for _, adb := range adbList.Items { + if adb.Spec.Details.AutonomousDatabaseOCID != nil { + By("Terminating database " + *adb.Spec.Details.DbName) + Expect(e2eutil.DeleteAutonomousDatabase(derefDBClient, adb.Spec.Details.AutonomousDatabaseOCID)).Should(Succeed()) + } + } + } +} + +func compartInt(obj1 *int, obj2 *int) bool { + if obj1 == nil && obj2 == nil { + return true + } + if (obj1 != nil && obj2 == nil) || (obj1 == nil && obj2 != nil) { + return false + } + return *obj1 == *obj2 +} + +func compartBool(obj1 *bool, obj2 *bool) bool { + if obj1 == nil && obj2 == nil { + return true + } + if (obj1 != nil && obj2 == nil) || (obj1 == nil && obj2 != nil) { + return false + } + return *obj1 == *obj2 +} + +func compartString(obj1 *string, obj2 *string) bool { + if obj1 == nil && obj2 == nil { + return true + } + if (obj1 != nil && obj2 == nil) || (obj1 == nil && obj2 != nil) { + return false + } + return *obj1 == *obj2 +} + +func compartStringMap(obj1 map[string]string, obj2 map[string]string) bool { + if len(obj1) != len(obj2) { + return false + } + + for k, v := range obj1 { + w, ok := obj2[k] + if !ok || v != w { + return false + } + } + + return true +} + +// AssertUpdate changes the displayName from "foo" to "foo_new", and scale the cpuCoreCount to 2 +func AssertUpdate(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName) func() { + return func() { + // Considering that there are at most two update requests will be sent during the update + // From the observation per request takes ~3mins to finish + updateTimeout := time.Minute * 7 + updateInterval := time.Second * 20 + + Expect(k8sClient).NotTo(BeNil()) + Expect(dbClient).NotTo(BeNil()) + Expect(adbLookupKey).NotTo(BeNil()) + + derefK8sClient := *k8sClient + derefDBClient := *dbClient + + expectedADB := &dbv1alpha1.AutonomousDatabase{} + Expect(derefK8sClient.Get(context.TODO(), *adbLookupKey, expectedADB)).To(Succeed()) + + By("Checking lifecycleState of the ADB is in AVAILABLE state before we do the update") + // Send the List request here. Sometimes even if Get request shows that the DB is in AVAILABLE state + // , the List request returns PROVISIONING state. In this case the update request will fail with + // conflict state error. + Eventually(func() (database.AutonomousDatabaseLifecycleStateEnum, error) { + listResp, err := e2eutil.ListAutonomousDatabases(derefDBClient, expectedADB.Spec.Details.CompartmentOCID, expectedADB.Spec.Details.DisplayName) + if err != nil { + return "", err + } + + if len(listResp.Items) < 1 { + return "", errors.New("should be at least 1 item in ListAutonomousDatabases response") + } + + return database.AutonomousDatabaseLifecycleStateEnum(listResp.Items[0].LifecycleState), nil + }, updateTimeout, updateInterval).Should(Equal(database.AutonomousDatabaseLifecycleStateAvailable)) + + // Update + var newDisplayName = *expectedADB.Spec.Details.DisplayName + "_new" + var newCPUCoreCount = 2 + + By(fmt.Sprintf("Updating the ADB with newDisplayName = %s and newCPUCoreCount = %d\n", newDisplayName, newCPUCoreCount)) + + expectedADB.Spec.Details.DisplayName = common.String(newDisplayName) + expectedADB.Spec.Details.CPUCoreCount = common.Int(newCPUCoreCount) + + Expect(derefK8sClient.Update(context.TODO(), expectedADB)).To(Succeed()) + + Eventually(func() (bool, error) { + // Get the current ADB resource and retry it's not in AVAILABLE state + currentADB := &dbv1alpha1.AutonomousDatabase{} + derefK8sClient.Get(context.TODO(), *adbLookupKey, currentADB) + if currentADB.Spec.Details.LifecycleState != database.AutonomousDatabaseLifecycleStateAvailable { + return false, nil + } + + // Fetch the ADB from OCI when it's in AVAILABLE state, and retry if its attributes doesn't match the new ADB's attributes + retryPolicy := e2eutil.NewLifecycleStateRetryPolicy(database.AutonomousDatabaseLifecycleStateAvailable) + resp, err := e2eutil.GetAutonomousDatabase(derefDBClient, currentADB.Spec.Details.AutonomousDatabaseOCID, &retryPolicy) + if err != nil { + return false, err + } + + adbDetails := currentADB.Spec.Details + + ociADB := currentADB + ociADB = currentADB.UpdateAttrFromOCIAutonomousDatabase(resp.AutonomousDatabase) + ociADBDetails := ociADB.Spec.Details + + // Compare + same := compartString(adbDetails.AutonomousDatabaseOCID, ociADBDetails.AutonomousDatabaseOCID) && + compartString(adbDetails.CompartmentOCID, ociADBDetails.CompartmentOCID) && + compartString(adbDetails.DisplayName, ociADBDetails.DisplayName) && + compartString(adbDetails.DbName, ociADBDetails.DbName) && + adbDetails.DbWorkload == ociADBDetails.DbWorkload && + compartBool(adbDetails.IsDedicated, ociADBDetails.IsDedicated) && + compartString(adbDetails.DbVersion, ociADBDetails.DbVersion) && + compartInt(adbDetails.DataStorageSizeInTBs, ociADBDetails.DataStorageSizeInTBs) && + compartInt(adbDetails.CPUCoreCount, ociADBDetails.CPUCoreCount) && + compartBool(adbDetails.IsAutoScalingEnabled, ociADBDetails.IsAutoScalingEnabled) && + adbDetails.LifecycleState == ociADBDetails.LifecycleState && + compartStringMap(adbDetails.FreeformTags, ociADBDetails.FreeformTags) && + compartString(adbDetails.SubnetOCID, ociADBDetails.SubnetOCID) && + reflect.DeepEqual(adbDetails.NsgOCIDs, ociADBDetails.NsgOCIDs) && + compartString(adbDetails.PrivateEndpoint, ociADBDetails.PrivateEndpoint) && + compartString(adbDetails.PrivateEndpointLabel, ociADBDetails.PrivateEndpointLabel) && + compartString(adbDetails.PrivateEndpointIP, ociADBDetails.PrivateEndpointIP) + + return same, nil + }, updateTimeout, updateInterval).Should(BeTrue()) + } +} + +// Updates adb state and then asserts if change is propagated to OCI +func UpdateAndAssertState(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName, state database.AutonomousDatabaseLifecycleStateEnum) func() { + return func() { + UpdateState(k8sClient, adbLookupKey, state)() + AssertState(k8sClient, dbClient, adbLookupKey, state)() + } +} + +// Assert local and remote state +func AssertState(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName, state database.AutonomousDatabaseLifecycleStateEnum) func() { + return func() { + // Waits longer for the local resource to reach the desired state + AssertLocalState(k8sClient, adbLookupKey, state)() + + // Double-check the state of the DB in OCI so the timeout can be shorter + AssertRemoteState(k8sClient, dbClient, adbLookupKey, state)() + } +} + +func AssertHardLinkDelete(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName) func() { + return func() { + changeStateTimeout := time.Second * 300 + + Expect(k8sClient).NotTo(BeNil()) + Expect(dbClient).NotTo(BeNil()) + Expect(adbLookupKey).NotTo(BeNil()) + + derefK8sClient := *k8sClient + derefDBClient := *dbClient + + adb := &dbv1alpha1.AutonomousDatabase{} + Expect(derefK8sClient.Get(context.TODO(), *adbLookupKey, adb)).To(Succeed()) + Expect(derefK8sClient.Delete(context.TODO(), adb)).To(Succeed()) + + AssertSoftLinkDelete(k8sClient, adbLookupKey)() + + By("Checking if the ADB in OCI is in TERMINATING state") + // Check every 10 secs for total 60 secs + Eventually(func() (database.AutonomousDatabaseLifecycleStateEnum, error) { + retryPolicy := e2eutil.NewLifecycleStateRetryPolicy(database.AutonomousDatabaseLifecycleStateTerminating) + return returnRemoteState(derefK8sClient, derefDBClient, adb.Spec.Details.AutonomousDatabaseOCID, &retryPolicy) + }, changeStateTimeout).Should(Equal(database.AutonomousDatabaseLifecycleStateTerminating)) + } +} + +func AssertSoftLinkDelete(k8sClient *client.Client, adbLookupKey *types.NamespacedName) func() { + return func() { + changeStateTimeout := time.Second * 300 + changeStateInterval := time.Second * 10 + + Expect(k8sClient).NotTo(BeNil()) + Expect(adbLookupKey).NotTo(BeNil()) + + derefK8sClient := *k8sClient + + existingAdb := &dbv1alpha1.AutonomousDatabase{} + Expect(derefK8sClient.Get(context.TODO(), *adbLookupKey, existingAdb)).To(Succeed()) + Expect(derefK8sClient.Delete(context.TODO(), existingAdb)).To(Succeed()) + + By("Checking if the AutonomousDatabase resource is deleted") + Eventually(func() (isDeleted bool) { + adb := &dbv1alpha1.AutonomousDatabase{} + isDeleted = false + err := derefK8sClient.Get(context.TODO(), *adbLookupKey, adb) + if err != nil && k8sErrors.IsNotFound(err) { + isDeleted = true + return + } + return + }, changeStateTimeout, changeStateInterval).Should(Equal(true)) + } +} + +func AssertLocalState(k8sClient *client.Client, adbLookupKey *types.NamespacedName, state database.AutonomousDatabaseLifecycleStateEnum) func() { + return func() { + changeLocalStateTimeout := time.Second * 600 + + Expect(k8sClient).NotTo(BeNil()) + Expect(adbLookupKey).NotTo(BeNil()) + + derefK8sClient := *k8sClient + + By("Checking if the lifecycleState of local resource is " + string(state)) + Eventually(func() (database.AutonomousDatabaseLifecycleStateEnum, error) { + return returnLocalState(derefK8sClient, *adbLookupKey) + }, changeLocalStateTimeout).Should(Equal(state)) + } +} + +func AssertRemoteState(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName, state database.AutonomousDatabaseLifecycleStateEnum) func() { + return func() { + + Expect(k8sClient).NotTo(BeNil()) + Expect(dbClient).NotTo(BeNil()) + Expect(adbLookupKey).NotTo(BeNil()) + + derefK8sClient := *k8sClient + + adb := &dbv1alpha1.AutonomousDatabase{} + Expect(derefK8sClient.Get(context.TODO(), *adbLookupKey, adb)).To(Succeed()) + + AssertRemoteStateOCID(k8sClient, dbClient, adb.Spec.Details.AutonomousDatabaseOCID, state)() + } +} + +// Assert remote state using adb OCID +func AssertRemoteStateOCID(k8sClient *client.Client, dbClient *database.DatabaseClient, adbID *string, state database.AutonomousDatabaseLifecycleStateEnum) func() { + return func() { + changeRemoteStateTimeout := time.Second * 300 + changeRemoteStateInterval := time.Second * 10 + + Expect(k8sClient).NotTo(BeNil()) + Expect(dbClient).NotTo(BeNil()) + Expect(adbID).NotTo(BeNil()) + + fmt.Fprintf(GinkgoWriter, "ADB ID is %s", *adbID) + + derefK8sClient := *k8sClient + derefDBClient := *dbClient + + By("Checking if the lifecycleState of the ADB in OCI is " + string(state)) + Eventually(func() (database.AutonomousDatabaseLifecycleStateEnum, error) { + return returnRemoteState(derefK8sClient, derefDBClient, adbID, nil) + }, changeRemoteStateTimeout, changeRemoteStateInterval).Should(Equal(state)) + } +} + +// Updates state from local resource and OCI +func UpdateState(k8sClient *client.Client, adbLookupKey *types.NamespacedName, state database.AutonomousDatabaseLifecycleStateEnum) func() { + return func() { + Expect(k8sClient).NotTo(BeNil()) + Expect(adbLookupKey).NotTo(BeNil()) + + derefK8sClient := *k8sClient + + adb := &dbv1alpha1.AutonomousDatabase{} + Expect(derefK8sClient.Get(context.TODO(), *adbLookupKey, adb)).To(Succeed()) + + adb.Spec.Details.LifecycleState = state + By("Updating adb state to " + string(state)) + Expect(derefK8sClient.Update(context.TODO(), adb)).To(Succeed()) + } +} + +func returnLocalState(k8sClient client.Client, adbLookupKey types.NamespacedName) (database.AutonomousDatabaseLifecycleStateEnum, error) { + adb := &dbv1alpha1.AutonomousDatabase{} + err := k8sClient.Get(context.TODO(), adbLookupKey, adb) + if err != nil { + return "", err + } + return adb.Status.LifecycleState, nil +} + +func returnRemoteState(k8sClient client.Client, dbClient database.DatabaseClient, adbID *string, retryPolicy *common.RetryPolicy) (database.AutonomousDatabaseLifecycleStateEnum, error) { + resp, err := e2eutil.GetAutonomousDatabase(dbClient, adbID, retryPolicy) + if err != nil { + return "", err + } + return resp.LifecycleState, nil +} diff --git a/test/e2e/resource/test_config.yaml b/test/e2e/resource/test_config.yaml new file mode 100644 index 00000000..4a5f8027 --- /dev/null +++ b/test/e2e/resource/test_config.yaml @@ -0,0 +1,16 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +# The path of your OCI configuration file, which usually is ~/.oci/config +ociConfigFile: ~/.oci/config +# The profile you want to use in the test (Optional) +profile: DEFAULT + +# Compartment OCID where the database creates +compartmentOCID: ocid1.compartment.. +# The OCID of the OCI Vault Secret that holds the password of the ADMIN account (should start with ocid1.vaultsecret...) +adminPasswordOCID: ocid1.vaultsecret... +# The OCID of the OCI Vault Secret that holds the password of the wallet (should start with ocid1.vaultsecret...) +instanceWalletPasswordOCID: ocid1.vaultsecret... \ No newline at end of file diff --git a/test/e2e/suite_test.go b/test/e2e/suite_test.go new file mode 100644 index 00000000..3fd5cd2e --- /dev/null +++ b/test/e2e/suite_test.go @@ -0,0 +1,236 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package e2etest + +import ( + "context" + "path/filepath" + "testing" + "time" + + . "github.com/onsi/ginkgo" + . "github.com/onsi/gomega" + "github.com/onsi/gomega/gexec" + "github.com/oracle/oci-go-sdk/v45/common" + "github.com/oracle/oci-go-sdk/v45/database" + corev1 "k8s.io/api/core/v1" + "k8s.io/client-go/kubernetes/scheme" + "k8s.io/client-go/rest" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/envtest" + "sigs.k8s.io/controller-runtime/pkg/envtest/printer" + logf "sigs.k8s.io/controller-runtime/pkg/log" + "sigs.k8s.io/controller-runtime/pkg/log/zap" + + databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + controllers "github.com/oracle/oracle-database-operator/controllers/database" + "github.com/oracle/oracle-database-operator/test/e2e/behavior" + "github.com/oracle/oracle-database-operator/test/e2e/util" + // +kubebuilder:scaffold:imports +) + +// These tests use Ginkgo (BDD-style Go testing framework). Refer to +// http://onsi.github.io/ginkgo/ to learn more about Ginkgo. + +/** +This test suite runs the integration test which checks the following scenario +1. Init the test by creating utililty variables and objects +2. Test ADB provisioned using adminPasswordOCID with hardLink=true +3. Test ADB provisioned using adminPasswordSecret with hardLink=true +5. Test ADB binding with hardLink=true +**/ + +var cfg *rest.Config +var k8sClient client.Client +var configProvider common.ConfigurationProvider +var dbClient database.DatabaseClient +var testEnv *envtest.Environment + +const configFileName = "test_config.yaml" +const ADBNamespace string = "default" + +var SharedOCIConfigMapName = "oci-cred" +var SharedOCISecretName = "oci-privatekey" +var SharedPlainTextAdminPassword = "Welcome_1234" +var SharedPlainTextWalletPassword = "Welcome_1234" +var SharedCompartmentOCID string + +var SharedKeyOCID string +var SharedAdminPasswordOCID string +var SharedInstanceWalletPasswordOCID string + +const SharedAdminPassSecretName string = "adb-admin-password" +const SharedWalletPassSecretName = "adb-wallet-password" + +func TestAPIs(t *testing.T) { + RegisterFailHandler(Fail) + + RunSpecsWithDefaultAndCustomReporters(t, + "Controller Suite", + []Reporter{printer.NewlineReporter{}}) +} + +var _ = BeforeSuite(func(done Done) { + logf.SetLogger(zap.New(zap.WriteTo(GinkgoWriter), zap.UseDevMode(true))) + + By("bootstrapping test environment") + testEnv = &envtest.Environment{ + CRDDirectoryPaths: []string{filepath.Join("../..", "config", "crd", "bases")}, + } + + var err error + cfg, err = testEnv.Start() + Expect(err).ToNot(HaveOccurred()) + Expect(cfg).ToNot(BeNil()) + + err = databasev1alpha1.AddToScheme(scheme.Scheme) + Expect(err).NotTo(HaveOccurred()) + + err = corev1.AddToScheme(scheme.Scheme) + Expect(err).NotTo(HaveOccurred()) + + // +kubebuilder:scaffold:scheme + + k8sClient, err = client.New(cfg, client.Options{Scheme: scheme.Scheme}) + Expect(err).ToNot(HaveOccurred()) + Expect(k8sClient).ToNot(BeNil()) + + k8sManager, err := ctrl.NewManager(cfg, ctrl.Options{ + Scheme: scheme.Scheme, + }) + Expect(err).ToNot(HaveOccurred()) + + err = (&controllers.AutonomousDatabaseReconciler{ + KubeClient: k8sManager.GetClient(), + Log: ctrl.Log.WithName("controllers").WithName("AutonomousDatabase_test"), + Scheme: k8sManager.GetScheme(), + }).SetupWithManager(k8sManager) + Expect(err).ToNot(HaveOccurred()) + + go func() { + defer GinkgoRecover() + err = k8sManager.Start(ctrl.SetupSignalHandler()) + Expect(err).ToNot(HaveOccurred(), "failed to run manager") + gexec.KillAndWait(4 * time.Second) + + // Teardown the test environment once controller is fnished. + // Otherwise from Kubernetes 1.21+, teardon timeouts waiting on + // kube-apiserver to return + err := testEnv.Stop() + Expect(err).ToNot(HaveOccurred()) + }() + + /************************************************** + * Custom codes for autonomousdatabase controller + **************************************************/ + By("init the test by creating utililty variables and objects") + testConfig, err := e2eutil.GetTestConfig(configFileName) + Expect(err).ToNot(HaveOccurred()) + Expect(testConfig).ToNot(BeNil()) + + SharedCompartmentOCID = testConfig.CompartmentOCID + SharedAdminPasswordOCID = testConfig.AdminPasswordOCID + SharedInstanceWalletPasswordOCID = testConfig.InstanceWalletPasswordOCID + + By("checking if the required parameters exist") + Expect(testConfig.OCIConfigFile).ToNot(Equal("")) + Expect(testConfig.CompartmentOCID).ToNot(Equal("")) + Expect(testConfig.AdminPasswordOCID).ToNot(Equal("")) + Expect(testConfig.InstanceWalletPasswordOCID).ToNot(Equal("")) + + By("getting OCI provider") + ociConfigUtil, err := e2eutil.GetOCIConfigUtil(testConfig.OCIConfigFile, testConfig.Profile) + Expect(err).ToNot(HaveOccurred()) + configProvider, err = ociConfigUtil.GetConfigProvider() + Expect(err).ToNot(HaveOccurred()) + + By("creating a OCI DB client") + dbClient, err = database.NewDatabaseClientWithConfigurationProvider(configProvider) + Expect(err).ToNot(HaveOccurred()) + + By("creating a configMap for calling OCI") + ociConfigMap, err := ociConfigUtil.CreateOCIConfigMap(ADBNamespace, SharedOCIConfigMapName) + Expect(err).ToNot(HaveOccurred()) + Expect(k8sClient.Create(context.TODO(), ociConfigMap)).To(Succeed()) + + By("creating a secret for calling OCI") + ociSecret, err := ociConfigUtil.CreateOCISecret(ADBNamespace, SharedOCISecretName) + Expect(err).ToNot(HaveOccurred()) + Expect(k8sClient.Create(context.TODO(), ociSecret)).To(Succeed()) + + By("Creating a k8s secret to hold admin password", func() { + data := map[string]string{ + SharedAdminPassSecretName: SharedPlainTextAdminPassword, + } + adminSecret, err := e2eutil.CreateKubeSecret(ADBNamespace, SharedAdminPassSecretName, data) + Expect(err).ToNot(HaveOccurred()) + Expect(k8sClient.Create(context.TODO(), adminSecret)).To(Succeed()) + }) + + By("Creating a k8s secret to hold wallet password", func() { + data := map[string]string{ + SharedWalletPassSecretName: SharedPlainTextWalletPassword, + } + walletSecret, err := e2eutil.CreateKubeSecret(ADBNamespace, SharedWalletPassSecretName, data) + Expect(err).ToNot(HaveOccurred()) + Expect(k8sClient.Create(context.TODO(), walletSecret)).To(Succeed()) + }) + + close(done) +}, 60) + +var _ = AfterSuite(func() { + /* + From Kubernetes 1.21+, when it tries to cleanup the test environment, there is + a clash if a custom controller is created during testing. It would seem that + the controller is still running and kube-apiserver will not respond to shutdown. + This is the reason why teardown happens in BeforeSuite() after controller has stopped. + The error shown is as documented in: + https://github.com/kubernetes-sigs/controller-runtime/issues/1571 + /* + /* + By("tearing down the test environment") + err := testEnv.Stop() + Expect(err).ToNot(HaveOccurred()) + */ + + By("Deleting the resources that are created during the tests") + e2ebehavior.CleanupDB(&k8sClient, &dbClient, ADBNamespace) +}) diff --git a/test/e2e/util/oci_config_util.go b/test/e2e/util/oci_config_util.go new file mode 100644 index 00000000..70ee1cab --- /dev/null +++ b/test/e2e/util/oci_config_util.go @@ -0,0 +1,249 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package e2eutil + +import ( + "errors" + "fmt" + "io/ioutil" + "os" + "os/user" + "path" + "regexp" + "strings" + + "github.com/oracle/oci-go-sdk/v45/common" + + corev1 "k8s.io/api/core/v1" +) + +type configUtil struct { + //The path to the test configuration file + OCIConfigPath string + + //The profile for the configuration + Profile string + + //ConfigFileInfo + FileInfo *configFileInfo + + provider common.ConfigurationProvider +} + +func (p configUtil) readAndParseConfigFile() (*configFileInfo, error) { + if p.FileInfo != nil { + return p.FileInfo, nil + } + + if p.OCIConfigPath == "" { + return nil, fmt.Errorf("configuration path can not be empty") + } + + data, err := openConfigFile(p.OCIConfigPath) + if err != nil { + return nil, err + } + + p.FileInfo, err = parseConfigFile(data, p.Profile) + return p.FileInfo, err +} + +func (p configUtil) CreateOCIConfigMap(configMapNamespace string, configMapName string) (*corev1.ConfigMap, error) { + info, err := p.readAndParseConfigFile() + if err != nil { + return nil, err + } + + data := map[string]string{ + "region": info.Region, + "fingerprint": info.Fingerprint, + "user": info.UserOCID, + "tenancy": info.TenancyOCID, + } + + return createKubeConfigMap(configMapNamespace, configMapName, data) +} + +func (p configUtil) CreateOCISecret(secretNamespace string, secretName string) (*corev1.Secret, error) { + info, err := p.readAndParseConfigFile() + if err != nil { + return nil, err + } + + expandedKeyFilePath, err := expandPath(info.KeyFilePath) + if err != nil { + return nil, err + } + + pemFileContent, err := ioutil.ReadFile(expandedKeyFilePath) + if err != nil { + err = fmt.Errorf("can not read PrivateKey from configuration file due to: %s", err.Error()) + return nil, err + } + + privatekeyData := map[string]string{ + "privatekey": string(pemFileContent), + } + + return CreateKubeSecret(secretNamespace, secretName, privatekeyData) +} + +func (p configUtil) GetConfigProvider() (common.ConfigurationProvider, error) { + if p.provider != nil { + return p.provider, nil + } + + newProvider, err := common.ConfigurationProviderFromFileWithProfile(p.OCIConfigPath, p.Profile, "") + if err != nil { + return nil, nil + } + + p.provider = newProvider + return newProvider, nil +} + +func openConfigFile(configFilePath string) (data []byte, err error) { + expandedPath, err := expandPath(configFilePath) + if err != nil { + return + } + data, err = ioutil.ReadFile(expandedPath) + if err != nil { + err = fmt.Errorf("can not read config file: %s due to: %s", configFilePath, err.Error()) + } + + return +} + +func getHomeFolder() (string, error) { + current, e := user.Current() + if e != nil { + //Give up and try to return something sensible + home := os.Getenv("HOME") + if home == "" { + return "", errors.New("do not inlcude a tilde(~) in the path") + } + return home, nil + } + return current.HomeDir, nil +} + +// cleans and expands the path if it contains a tilde, returns the expanded path or the input path as is if not expansion +// was performed +func expandPath(filepath string) (expandedPath string, err error) { + cleanedPath := path.Clean(filepath) + expandedPath = cleanedPath + if strings.HasPrefix(cleanedPath, "~") { + rest := cleanedPath[2:] + home, err := getHomeFolder() + if err != nil { + return "", err + } + expandedPath = path.Join(home, rest) + } + return expandedPath, nil +} + +var profileRegex = regexp.MustCompile(`^\[(.*)\]`) + +func parseConfigFile(data []byte, profile string) (info *configFileInfo, err error) { + + if len(data) == 0 { + return nil, fmt.Errorf("configuration file content is empty") + } + + content := string(data) + splitContent := strings.Split(content, "\n") + + //Look for profile + for i, line := range splitContent { + if match := profileRegex.FindStringSubmatch(line); match != nil && len(match) > 1 && match[1] == profile { + start := i + 1 + return parseConfigAtLine(start, splitContent) + } + } + + return nil, fmt.Errorf("configuration file did not contain profile: %s", profile) +} + +type configFileInfo struct { + UserOCID, Fingerprint, KeyFilePath, TenancyOCID, Region, Passphrase string +} + +func parseConfigAtLine(start int, content []string) (info *configFileInfo, err error) { + info = &configFileInfo{} + for i := start; i < len(content); i++ { + line := content[i] + if profileRegex.MatchString(line) { + break + } + + if !strings.Contains(line, "=") { + continue + } + + splits := strings.Split(line, "=") + switch key, value := strings.TrimSpace(splits[0]), strings.TrimSpace(splits[1]); strings.ToLower(key) { + case "passphrase", "pass_phrase": + info.Passphrase = value + case "user": + info.UserOCID = value + case "fingerprint": + info.Fingerprint = value + case "key_file": + info.KeyFilePath = value + case "tenancy": + info.TenancyOCID = value + case "region": + info.Region = value + } + } + return +} + +func GetOCIConfigUtil(configPath string, profile string) (*configUtil, error) { + if profile == "" { + profile = "DEFAULT" + } + + return &configUtil{ + OCIConfigPath: configPath, + Profile: profile, + }, nil +} diff --git a/test/e2e/util/oci_db_request.go b/test/e2e/util/oci_db_request.go new file mode 100644 index 00000000..5fcd3aa1 --- /dev/null +++ b/test/e2e/util/oci_db_request.go @@ -0,0 +1,166 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package e2eutil + +import ( + "context" + + "github.com/oracle/oci-go-sdk/v45/common" + "github.com/oracle/oci-go-sdk/v45/database" + + "time" +) + +func CreateAutonomousDatabase(dbClient database.DatabaseClient, compartmentID *string, dbName *string, adminPassword *string) (response database.CreateAutonomousDatabaseResponse, err error) { + createAutonomousDatabaseDetails := database.CreateAutonomousDatabaseDetails{ + CompartmentId: compartmentID, + DbName: dbName, + DisplayName: dbName, + CpuCoreCount: common.Int(1), + DataStorageSizeInTBs: common.Int(1), + AdminPassword: adminPassword, + IsAutoScalingEnabled: common.Bool(true), + DbWorkload: database.CreateAutonomousDatabaseBaseDbWorkloadEnum("OLTP"), + } + + createAutonomousDatabaseRequest := database.CreateAutonomousDatabaseRequest{ + CreateAutonomousDatabaseDetails: createAutonomousDatabaseDetails, + } + + return dbClient.CreateAutonomousDatabase(context.TODO(), createAutonomousDatabaseRequest) +} + +func GetAutonomousDatabase(dbClient database.DatabaseClient, databaseOCID *string, retryPolicy *common.RetryPolicy) (database.GetAutonomousDatabaseResponse, error) { + getRequest := database.GetAutonomousDatabaseRequest{ + AutonomousDatabaseId: databaseOCID, + } + + if retryPolicy != nil { + getRequest.RequestMetadata = common.RequestMetadata{ + RetryPolicy: retryPolicy, + } + } + + return dbClient.GetAutonomousDatabase(context.TODO(), getRequest) +} + +func ListAutonomousDatabases(dbClient database.DatabaseClient, compartmentOCID *string, displayName *string) (database.ListAutonomousDatabasesResponse, error) { + listRequest := database.ListAutonomousDatabasesRequest{ + CompartmentId: compartmentOCID, + DisplayName: displayName, + } + return dbClient.ListAutonomousDatabases(context.TODO(), listRequest) +} + +func deleteAutonomousDatabase(dbClient database.DatabaseClient, databaseOCID *string) error { + if databaseOCID == nil { + return nil + } + + req := database.DeleteAutonomousDatabaseRequest{ + AutonomousDatabaseId: databaseOCID, + } + + if _, err := dbClient.DeleteAutonomousDatabase(context.TODO(), req); err != nil { + return err + } + + return nil +} + +// DeleteAutonomousDatabase terminates the database if it exists and is not in TERMINATED state +func DeleteAutonomousDatabase(dbClient database.DatabaseClient, databaseOCID *string) error { + if databaseOCID == nil { + return nil + } + + resp, err := GetAutonomousDatabase(dbClient, databaseOCID, nil) + if err != nil { + return nil + } + if resp.AutonomousDatabase.LifecycleState != database.AutonomousDatabaseLifecycleStateTerminated { + deleteAutonomousDatabase(dbClient, databaseOCID) + } + + return nil +} + +func generateRetryPolicy(retryFunc func(r common.OCIOperationResponse) bool) common.RetryPolicy { + // Retry up to 4 times every 10 seconds. + attempts := uint(6) + nextDuration := func(r common.OCIOperationResponse) time.Duration { + return 10 * time.Second + } + return common.NewRetryPolicy(attempts, retryFunc, nextDuration) +} + +func NewDisplayNameRetryPolicy(name string) common.RetryPolicy { + shouldRetry := func(r common.OCIOperationResponse) bool { + if databaseResponse, ok := r.Response.(database.GetAutonomousDatabaseResponse); ok { + // do the retry until lifecycle state reaches the passed terminal state + return databaseResponse.LifecycleState != database.AutonomousDatabaseLifecycleStateAvailable || + *databaseResponse.DisplayName != name + } + return true + } + return generateRetryPolicy(shouldRetry) +} + +func NewCPUCoreCountRetryPolicy(count int) common.RetryPolicy { + shouldRetry := func(r common.OCIOperationResponse) bool { + if databaseResponse, ok := r.Response.(database.GetAutonomousDatabaseResponse); ok { + // do the retry until lifecycle state reaches the passed terminal state + return databaseResponse.LifecycleState != database.AutonomousDatabaseLifecycleStateAvailable || + *databaseResponse.CpuCoreCount != count + } + return true + } + return generateRetryPolicy(shouldRetry) +} + +func NewLifecycleStateRetryPolicy(lifecycleState database.AutonomousDatabaseLifecycleStateEnum) common.RetryPolicy { + shouldRetry := func(r common.OCIOperationResponse) bool { + if databaseResponse, ok := r.Response.(database.GetAutonomousDatabaseResponse); ok { + // do the retry until lifecycle state reaches the passed terminal state + return databaseResponse.LifecycleState != lifecycleState + } + return true + } + return generateRetryPolicy(shouldRetry) +} diff --git a/test/e2e/util/oci_vault_request.go b/test/e2e/util/oci_vault_request.go new file mode 100644 index 00000000..ef7a793c --- /dev/null +++ b/test/e2e/util/oci_vault_request.go @@ -0,0 +1,255 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package e2eutil + +import ( + "context" + "encoding/base64" + "math" + "time" + + "github.com/oracle/oci-go-sdk/v45/common" + "github.com/oracle/oci-go-sdk/v45/keymanagement" + "github.com/oracle/oci-go-sdk/v45/vault" +) + +func waitForVaultStatePolicy(state keymanagement.VaultLifecycleStateEnum) common.RetryPolicy { + shouldRetry := func(r common.OCIOperationResponse) bool { + if _, isServiceError := common.IsServiceError(r.Error); isServiceError { + // not service error, could be network error or other errors which prevents + // request send to server, will do retry here + return true + } + + if vaultResponse, ok := r.Response.(keymanagement.GetVaultResponse); ok { + // do the retry until lifecycle state reaches the passed terminal state + return vaultResponse.Vault.LifecycleState != state + } + + return true + } + + return newRetryPolicy(shouldRetry) +} + +func CreateOCIVault(kmsVaultClient keymanagement.KmsVaultClient, compartmentID *string, vaultName *string) (*keymanagement.Vault, error) { + vaultDetails := keymanagement.CreateVaultDetails{ + CompartmentId: compartmentID, + DisplayName: vaultName, + VaultType: keymanagement.CreateVaultDetailsVaultTypeDefault, + } + + request := keymanagement.CreateVaultRequest{ + CreateVaultDetails: vaultDetails, + } + response, err := kmsVaultClient.CreateVault(context.TODO(), request) + if err != nil { + return nil, err + } + + return &response.Vault, nil +} + +func CreateOCIKey(vaultManagementClient keymanagement.KmsManagementClient, compartmentID *string, keyName *string) (*keymanagement.Key, error) { + keyLength := 32 + + keyShape := keymanagement.KeyShape{ + Algorithm: keymanagement.KeyShapeAlgorithmAes, + Length: &keyLength, + } + + createKeyDetails := keymanagement.CreateKeyDetails{ + CompartmentId: compartmentID, + KeyShape: &keyShape, + DisplayName: keyName, + } + + request := keymanagement.CreateKeyRequest{ + CreateKeyDetails: createKeyDetails, + } + response, err := vaultManagementClient.CreateKey(context.TODO(), request) + if err != nil { + return nil, err + } + + return &response.Key, nil +} + +func CreateOCISecret(vaultClient vault.VaultsClient, compartmentID *string, secretName *string, vaultID *string, keyID *string, content *string) (*string, error) { + encoded := base64.StdEncoding.EncodeToString([]byte(*content)) + + base64Content := vault.Base64SecretContentDetails{ + Name: secretName, + Content: common.String(encoded), + } + + details := vault.CreateSecretDetails{ + CompartmentId: compartmentID, + SecretContent: base64Content, + SecretName: secretName, + VaultId: vaultID, + KeyId: keyID, + } + + request := vault.CreateSecretRequest{ + CreateSecretDetails: details, + } + + // Send the request using the service client + response, err := vaultClient.CreateSecret(context.TODO(), request) + if err != nil { + return nil, err + } + + return response.Secret.Id, nil +} + +func getVault(ctx context.Context, client keymanagement.KmsVaultClient, retryPolicy *common.RetryPolicy, vaultID *string) error { + request := keymanagement.GetVaultRequest{ + VaultId: vaultID, + RequestMetadata: common.RequestMetadata{ + RetryPolicy: retryPolicy, + }, + } + if _, err := client.GetVault(ctx, request); err != nil { + return err + } + return nil +} + +func getKey(client keymanagement.KmsManagementClient, retryPolicy *common.RetryPolicy, keyID *string) error { + request := keymanagement.GetKeyRequest{ + KeyId: keyID, + RequestMetadata: common.RequestMetadata{ + RetryPolicy: retryPolicy, + }, + } + if _, err := client.GetKey(context.TODO(), request); err != nil { + return err + } + return nil +} + +func getSecret(client vault.VaultsClient, retryPolicy *common.RetryPolicy, secretID *string) error { + request := vault.GetSecretRequest{ + SecretId: secretID, + RequestMetadata: common.RequestMetadata{ + RetryPolicy: retryPolicy, + }, + } + if _, err := client.GetSecret(context.TODO(), request); err != nil { + return err + } + return nil +} + +func newRetryPolicy(retryOperation func(common.OCIOperationResponse) bool) common.RetryPolicy { + // maximum times of retry + attempts := uint(10) + + nextDuration := func(r common.OCIOperationResponse) time.Duration { + // you might want wait longer for next retry when your previous one failed + // this function will return the duration as: + // 1s, 2s, 4s, 8s, 16s, 32s, 64s etc... + return time.Duration(math.Pow(float64(2), float64(r.AttemptNumber-1))) * time.Second + } + + return common.NewRetryPolicy(attempts, retryOperation, nextDuration) +} + +func WaitForVaultState(client keymanagement.KmsVaultClient, vaultID *string, state keymanagement.VaultLifecycleStateEnum) error { + shouldRetry := func(r common.OCIOperationResponse) bool { + if vaultResponse, ok := r.Response.(keymanagement.GetVaultResponse); ok { + // do the retry until lifecycle state reaches the passed terminal state + return vaultResponse.Vault.LifecycleState != state + } + + return true + } + + lifecycleStateCheckRetryPolicy := newRetryPolicy(shouldRetry) + + return getVault(context.TODO(), client, &lifecycleStateCheckRetryPolicy, vaultID) +} + +func WaitForKeyState(client keymanagement.KmsManagementClient, keyID *string, state keymanagement.KeyLifecycleStateEnum) error { + shouldRetry := func(r common.OCIOperationResponse) bool { + if keyResponse, ok := r.Response.(keymanagement.GetKeyResponse); ok { + // do the retry until lifecycle state reaches the passed terminal state + return keyResponse.Key.LifecycleState != state + } + + return true + } + + lifecycleStateCheckRetryPolicy := newRetryPolicy(shouldRetry) + + return getKey(client, &lifecycleStateCheckRetryPolicy, keyID) +} + +func WaitForSecretState(client vault.VaultsClient, secretID *string, state vault.SecretLifecycleStateEnum) error { + shouldRetry := func(r common.OCIOperationResponse) bool { + if secretResponse, ok := r.Response.(vault.GetSecretResponse); ok { + // do the retry until lifecycle state reaches the passed terminal state + return secretResponse.Secret.LifecycleState != state + } + + return true + } + + lifecycleStateCheckRetryPolicy := newRetryPolicy(shouldRetry) + + return getSecret(client, &lifecycleStateCheckRetryPolicy, secretID) +} + +// CleanupVault deletes the vault +// Anything encrypted by the keys contained within this vault will be unusable or irretrievable after the vault has been deleted +func CleanupVault(kmsVaultClient keymanagement.KmsVaultClient, vaultID *string) error { + if vaultID == nil { + return nil + } + + request := keymanagement.ScheduleVaultDeletionRequest{ + VaultId: vaultID, + } + if _, err := kmsVaultClient.ScheduleVaultDeletion(context.TODO(), request); err != nil { + return err + } + return nil +} diff --git a/test/e2e/util/oci_work_request.go b/test/e2e/util/oci_work_request.go new file mode 100644 index 00000000..4721fa0a --- /dev/null +++ b/test/e2e/util/oci_work_request.go @@ -0,0 +1,97 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package e2eutil + +import ( + "context" + + "github.com/oracle/oci-go-sdk/v45/common" + "github.com/oracle/oci-go-sdk/v45/workrequests" + + "time" +) + +func WaitUntilWorkCompleted(workClient workrequests.WorkRequestClient, opcWorkRequestID *string) error { + retryPolicy := getCompleteWorkRetryPolicy() + + // Apply wait until work complete retryPolicy + workRequest := workrequests.GetWorkRequestRequest{ + WorkRequestId: opcWorkRequestID, + RequestMetadata: common.RequestMetadata{ + RetryPolicy: &retryPolicy, + }, + } + + // GetWorkRequest retries until the work status is SUCCEEDED + if _, err := workClient.GetWorkRequest(context.TODO(), workRequest); err != nil { + return err + } + + return nil +} + +func getCompleteWorkRetryPolicy() common.RetryPolicy { + // maximum times of retry + attempts := uint(30) + + shouldRetry := func(r common.OCIOperationResponse) bool { + if _, isServiceError := common.IsServiceError(r.Error); isServiceError { + // Don't retry if it's service error. Sometimes it could be network error or other errors which prevents + // request send to server; we do the retry in these cases. + return false + } + + if converted, ok := r.Response.(workrequests.GetWorkRequestResponse); ok { + // do the retry until WorkReqeut Status is Succeeded - ignore case (BMI-2652) + return converted.Status != workrequests.WorkRequestStatusSucceeded + } + + return true + } + + nextDuration := func(r common.OCIOperationResponse) time.Duration { + // // you might want wait longer for next retry when your previous one failed + // // this function will return the duration as: + // // 1s, 2s, 4s, 8s, 16s, 32s, 64s etc... + // return time.Duration(math.Pow(float64(2), float64(r.AttemptNumber-1))) * time.Second + return time.Second * 20 + } + + return common.NewRetryPolicy(attempts, shouldRetry, nextDuration) +} diff --git a/test/e2e/util/util.go b/test/e2e/util/util.go new file mode 100644 index 00000000..885985f1 --- /dev/null +++ b/test/e2e/util/util.go @@ -0,0 +1,138 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package e2eutil + +import ( + "encoding/json" + "io/ioutil" + "strings" + "time" + + goyaml "gopkg.in/yaml.v2" + + dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + + corev1 "k8s.io/api/core/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "sigs.k8s.io/yaml" +) + +// GenerateDBName returns a string DB concatenate 14 digits of the date time +// E.g., DB060102150405 if the curret date-time is 2006.01.02 15:04:05 +func GenerateDBName() string { + timeString := time.Now().Format("2006.01.02 15:04:05") + trimmed := strings.ReplaceAll(timeString, ":", "") // remove colons + trimmed = strings.ReplaceAll(trimmed, ".", "") // remove dots + trimmed = strings.ReplaceAll(trimmed, " ", "") // remove spaces + trimmed = trimmed[2:] // remove the first two digits of year (2006 -> 06) + return "DB" + trimmed +} + +func unmarshalFromYamlBytes(bytes []byte, obj interface{}) error { + jsonBytes, err := yaml.YAMLToJSON(bytes) + if err != nil { + return err + } + + return json.Unmarshal(jsonBytes, obj) +} + +// LoadTestFixture create an AutonomousDatabase resoursce from a test fixture +func LoadTestFixture(adb *dbv1alpha1.AutonomousDatabase, filename string) (*dbv1alpha1.AutonomousDatabase, error) { + filePath := "./resource/" + filename + yamlBytes, err := ioutil.ReadFile(filePath) + if err != nil { + return nil, err + } + + if err := unmarshalFromYamlBytes(yamlBytes, adb); err != nil { + return nil, err + } + + return adb, nil +} + +func createKubeConfigMap(configMapNamespace string, configMapName string, data map[string]string) (*corev1.ConfigMap, error) { + configmap := &corev1.ConfigMap{ + ObjectMeta: metav1.ObjectMeta{ + Namespace: configMapNamespace, + Name: configMapName, + }, + Data: data, + } + + return configmap, nil +} + +func CreateKubeSecret(secretNamespace string, secretName string, data map[string]string) (*corev1.Secret, error) { + secret := &corev1.Secret{ + ObjectMeta: metav1.ObjectMeta{ + Namespace: secretNamespace, + Name: secretName, + }, + Type: "Opaque", + StringData: data, + } + + return secret, nil +} + +type testConfiguration struct { + OCIConfigFile string `yaml:"ociConfigFile"` + Profile string `yaml:"profile"` + CompartmentOCID string `yaml:"compartmentOCID"` + AdminPasswordOCID string `yaml:"adminPasswordOCID"` + InstanceWalletPasswordOCID string `yaml:"instanceWalletPasswordOCID"` +} + +func GetTestConfig(filename string) (*testConfiguration, error) { + config := &testConfiguration{} + + filePath := "./resource/" + filename + yamlBytes, err := ioutil.ReadFile(filePath) + if err != nil { + return nil, err + } + + if err := goyaml.Unmarshal(yamlBytes, config); err != nil { + return nil, err + } + + return config, nil +} From 4926350977ff900e605d773a351b3fdbe93ada2d Mon Sep 17 00:00:00 2001 From: Ting-Lan Wang <61528376+harayuanwang@users.noreply.github.com> Date: Mon, 25 Oct 2021 10:52:16 -0700 Subject: [PATCH 003/628] Update ADB_PREREQUISITES.md Fixed minor typos in ADB_PREREQUISITES.md --- doc/adb/ADB_PREREQUISITES.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/adb/ADB_PREREQUISITES.md b/doc/adb/ADB_PREREQUISITES.md index ec51729a..057a8687 100644 --- a/doc/adb/ADB_PREREQUISITES.md +++ b/doc/adb/ADB_PREREQUISITES.md @@ -12,7 +12,7 @@ To provide access, choose **one of the following approaches**: ### Authorized with API Key Authentication -By default, all pods in the Oracle Container Engine for Kubernetes (OKE) are able to access the instance principal certificates, so that the operator calls OCI REST endpoints without any extra step. If you're using OKE, then please proceed to the [Oracle Autonomous Database document](./doc/adb/ORACLE_ADB_CONTROLLER_README.md). +By default, all pods in the Oracle Container Engine for Kubernetes (OKE) are able to access the instance principal certificates, so that the operator calls OCI REST endpoints without any extra step. If you're using OKE, then please proceed to the installation. If the operator is deployed in a third-party Kubernetes cluster, then the credentials of the Oracle Cloud Infrastructure (OCI) user are needed. The operator reads these credentials from a ConfigMap and a Secret. Oracle recommends using the helper script `set_ocicredentials.sh` in the root directory of the repository; This script will generate a ConfigMap and a Secret with the OCI credentials. By default, the script parses the **DEFAULT** profile in `~/.oci/config`. The default names of the ConfigMap and the Secret are, respectively: `oci-cred` and `oci-privatekey`. @@ -97,4 +97,4 @@ To set up Instance Principle authorization: Allow dynamic-group to manage autonomous-database-family in compartment ``` -At this stage, the operator has been granted sufficient permissions to call OCI services. You can now proceed to the the installation. +At this stage, the operator has been granted sufficient permissions to call OCI services. You can now proceed to the installation. From 0916b01b8d433ad7f9890f2a603b6b3f0d4f3b09 Mon Sep 17 00:00:00 2001 From: Ting-Lan Wang <61528376+harayuanwang@users.noreply.github.com> Date: Mon, 25 Oct 2021 10:53:05 -0700 Subject: [PATCH 004/628] Update ORACLE_ADB_CONTROLLER_README.md Removed unused bullet point in ORACLE_ADB_CONTROLLER_README.md --- doc/adb/ORACLE_ADB_CONTROLLER_README.md | 1 - 1 file changed, 1 deletion(-) diff --git a/doc/adb/ORACLE_ADB_CONTROLLER_README.md b/doc/adb/ORACLE_ADB_CONTROLLER_README.md index 38fd6ff5..3055c1bc 100644 --- a/doc/adb/ORACLE_ADB_CONTROLLER_README.md +++ b/doc/adb/ORACLE_ADB_CONTROLLER_README.md @@ -12,7 +12,6 @@ After the operator is deployed, choose either one of the following operations to After you create the resource, you can use the operator to perform the following tasks: * [Scale the OCPU core count or storage](#scale-the-ocpu-core-count-or-storage) an Autonomous Database -* [Enable/Disable auto scaling](#enabledisable-auto-scaling) of an Autonomous Database * [Rename](#rename) an Autonomous Database * [Manage ADMIN database user password](#manage-admin-passsword) of an Autonomous Database * [Download instance credentials (wallets)](#download-wallets) of an Autonomous Database From dc5c53ca4d173c2ef14e562b59ffe940319aee2d Mon Sep 17 00:00:00 2001 From: Ting-Lan Wang <61528376+harayuanwang@users.noreply.github.com> Date: Mon, 25 Oct 2021 11:28:34 -0700 Subject: [PATCH 005/628] Renamed to README.md and fixed broken links --- ...RACLE_SIDB_CONTROLLER_README.md => README.md} | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) rename doc/sidb/{ORACLE_SIDB_CONTROLLER_README.md => README.md} (94%) diff --git a/doc/sidb/ORACLE_SIDB_CONTROLLER_README.md b/doc/sidb/README.md similarity index 94% rename from doc/sidb/ORACLE_SIDB_CONTROLLER_README.md rename to doc/sidb/README.md index b640aa2f..0ed5d7d4 100644 --- a/doc/sidb/ORACLE_SIDB_CONTROLLER_README.md +++ b/doc/sidb/README.md @@ -1,12 +1,12 @@ -# Oracle Database Operator for Kubernetes +# Managing Oracle Single Instance Databases with Oracle Database Operator for Kubernetes Oracle Database Operator for Kubernetes (the operator) includes the Single Instance Database Controller that enables provisioning, cloning, and patching of Oracle Single Instance Databases on Kubernetes. The following sections explain the setup and functionality of the operator -* [Prerequisites](ORACLE_SIDB_CONTROLLER_README.md#prerequisites-and-setup) -* [Kind SingleInstanceDatabase](ORACLE_SIDB_CONTROLLER_README.md#kind-singleinstancedatabase) -* [Provision New Database](ORACLE_SIDB_CONTROLLER_README.md#provision-new-database) -* [Clone Existing Database](ORACLE_SIDB_CONTROLLER_README.md#clone-existing-database) -* [Patch/Rollback Database](ORACLE_SIDB_CONTROLLER_README.md#patchrollback-database) +* [Prerequisites](README.md#prerequisites) +* [Kind SingleInstanceDatabase Resource](README.md#kind-singleinstancedatabase-resource) +* [Provision New Database](README.md#provision-new-database) +* [Clone Existing Database](README.md#clone-existing-database) +* [Patch/Rollback Database](README.md#patchrollback-database) ## Prerequisites @@ -21,7 +21,7 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI For the use cases detailed below a sample .yaml file is available at * Enterprise, Standard Editions - [config/samples/singleinstancedatabase.yaml](config/samples/singleinstancedatabase.yaml) + [config/samples/singleinstancedatabase.yaml](./../../config/samples/singleinstancedatabase.yaml) **Note:** The `adminPassword` field of the above `singleinstancedatabase.yaml` yaml contains a secret for Single Instance Database creation (Provisioning a new database or cloning an existing database). This secret gets deleted after the database pod becomes ready for security reasons. @@ -244,7 +244,7 @@ $ kubectl get singleinstancedatabase/sidb-sample --template={{.status.status}} * ### Out of Place - Clone your source database using the method of [cloning existing database](ORACLE_SIDB_OPERATOR_README.md#clone-existing-database) and specify a new release version/image for the + Clone your source database using the method of [cloning existing database](README.md#clone-existing-database) and specify a new release version/image for the cloned database. Use this method to enusure there are no patching related issues impacting your database performance/functionality * ### Datapatch status From 45e93f98e6dbf4b658649828e4a15ffd3ce98f24 Mon Sep 17 00:00:00 2001 From: Ting-Lan Wang <61528376+harayuanwang@users.noreply.github.com> Date: Mon, 25 Oct 2021 11:30:50 -0700 Subject: [PATCH 006/628] Rename ORACLE_ADB_CONTROLLER_README.md to README.md --- doc/adb/{ORACLE_ADB_CONTROLLER_README.md => README.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename doc/adb/{ORACLE_ADB_CONTROLLER_README.md => README.md} (100%) diff --git a/doc/adb/ORACLE_ADB_CONTROLLER_README.md b/doc/adb/README.md similarity index 100% rename from doc/adb/ORACLE_ADB_CONTROLLER_README.md rename to doc/adb/README.md From 3e658f9311b28a7f805f5548bfc3d14d5b858e97 Mon Sep 17 00:00:00 2001 From: Ting-Lan Wang <61528376+harayuanwang@users.noreply.github.com> Date: Mon, 25 Oct 2021 11:36:53 -0700 Subject: [PATCH 007/628] Update and rename ORACLE_SHARDING_CONTROLLER_README.md to README.md --- .../{ORACLE_SHARDING_CONTROLLER_README.md => README.md} | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) rename doc/sharding/{ORACLE_SHARDING_CONTROLLER_README.md => README.md} (99%) diff --git a/doc/sharding/ORACLE_SHARDING_CONTROLLER_README.md b/doc/sharding/README.md similarity index 99% rename from doc/sharding/ORACLE_SHARDING_CONTROLLER_README.md rename to doc/sharding/README.md index 1c85dc45..710df0df 100644 --- a/doc/sharding/ORACLE_SHARDING_CONTROLLER_README.md +++ b/doc/sharding/README.md @@ -1,4 +1,4 @@ -# Oracle Sharding Database Controller +# Using Oracle Sharding with Oracle Database Operator for Kubernetes Oracle Sharding distributes segments of a data set across many databases (shards) on different computers, either on-premises or in cloud. Sharding enables globally distributed, linearly scalable, multimodel databases. It requires no specialized hardware or software. Oracle Sharding does all this while rendering the strong consistency, full power of SQL, support for structured and unstructured data, and the Oracle Database ecosystem. It meets data sovereignty requirements, and supports applications that require low latency and high availability. From bb7684bc7d4d656ac2c8387884f842d509ac5a5c Mon Sep 17 00:00:00 2001 From: Ting-Lan Wang <61528376+harayuanwang@users.noreply.github.com> Date: Mon, 25 Oct 2021 11:54:10 -0700 Subject: [PATCH 008/628] Fixed broken links --- doc/sharding/README.md | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/doc/sharding/README.md b/doc/sharding/README.md index 710df0df..018043aa 100644 --- a/doc/sharding/README.md +++ b/doc/sharding/README.md @@ -43,7 +43,7 @@ To create a Sharding Topology, complete the steps in the following sections belo * A Cloud-based Kubernetes cluster, such as [OCI on Container Engine for Kubernetes (OKE)](https://www.oracle.com/cloud-native/container-engine-kubernetes/) or * An On-Premises Kubernetes Cluster, such as [Oracle Linux Cloud Native Environment (OLCNE)](https://docs.oracle.com/en/operating-systems/olcne/) cluster. -To use Oracle Sharding Database Controller, ensure that your system is provisioned with a supported Kubernetes release. Refer to the [Release Status Section](../../../README.md#requirements). +To use Oracle Sharding Database Controller, ensure that your system is provisioned with a supported Kubernetes release. Refer to the [Release Status Section](../../README.md#release-status). ### 2. Deploy Oracle Database Operator @@ -85,7 +85,7 @@ You can either download the images and push them to your Docker Images Repositor ### 5. Create a Kubernetes secret for the database installation owner for the database Sharding Deployment -Create a Kubernetes secret named `db-user-pass` using these steps: [Create Kubernetes Secret](./doc/sharding/provisioning/create_kubernetes_secret_for_db_user.md) +Create a Kubernetes secret named `db-user-pass` using these steps: [Create Kubernetes Secret](./provisioning/create_kubernetes_secret_for_db_user.md) After you have the above prerequsites completed, you can proceed to the next section for your environment to provision the Oracle Database Sharding Topology. @@ -93,19 +93,19 @@ After you have the above prerequsites completed, you can proceed to the next sec Deploy Oracle Database sharding topology on your Cloud based Kubernetes cluster. In this example, the deployment uses the YAML file based on `OCI OKE` cluster. There are multiple use case possible for deploying the Oracle Database sharding topology. -[1. Provisioning Oracle Database sharding topology without Database Gold Image](./doc/sharding/provisioning/provisioning_without_db_gold_image.md) -[2. Provisioning Oracle Database sharding topology with additional control on resources like Memory and CPU allocated to Pods](./doc/sharding/provisioning/provisioning_with_control_on_resources.md) -[3. Provisioning a Persistent Volume having an Oracle Database Gold Image](./doc/sharding/provisioning/provisioning_persistent_volume_having_db_gold_image.md) -[4. Provisioning Oracle Database sharding topology by cloning database from your own Database Gold Image in the same Availability Domain(AD)](./doc/sharding/provisioning/provisioning_by_cloning_db_gold_image_in_same_ad.md) -[5. Provisioning Oracle Database sharding topology by cloning database from your own Database Gold Image across Availability Domains(ADs)](./doc/sharding/provisioning/provisioning_by_cloning_db_from_gold_image_across_ads.md) -[6. Provisioning Oracle Database sharding topology and send Notification using OCI Notification Service](./doc/sharding/provisioning/provisioning_with_notification_using_oci_notification.md) -[7. Scale Out - Add Shards to an existing Oracle Database Sharding Topology](./doc/sharding/provisioning/scale_out_add_shards.md) -[8. Scale In - Delete an existing Shard from a working Oracle Database sharding topology](./doc/sharding/provisioning/scale_in_delete_an_existing_shard.md) +[1. Provisioning Oracle Database sharding topology without Database Gold Image](./provisioning/provisioning_without_db_gold_image.md) +[2. Provisioning Oracle Database sharding topology with additional control on resources like Memory and CPU allocated to Pods](./provisioning/provisioning_with_control_on_resources.md) +[3. Provisioning a Persistent Volume having an Oracle Database Gold Image](./provisioning/provisioning_persistent_volume_having_db_gold_image.md) +[4. Provisioning Oracle Database sharding topology by cloning database from your own Database Gold Image in the same Availability Domain(AD)](./provisioning/provisioning_by_cloning_db_gold_image_in_same_ad.md) +[5. Provisioning Oracle Database sharding topology by cloning database from your own Database Gold Image across Availability Domains(ADs)](./provisioning/provisioning_by_cloning_db_from_gold_image_across_ads.md) +[6. Provisioning Oracle Database sharding topology and send Notification using OCI Notification Service](./provisioning/provisioning_with_notification_using_oci_notification.md) +[7. Scale Out - Add Shards to an existing Oracle Database Sharding Topology](./provisioning/scale_out_add_shards.md) +[8. Scale In - Delete an existing Shard from a working Oracle Database sharding topology](./provisioning/scale_in_delete_an_existing_shard.md) ## Connecting to Shard Databases -After the Oracle Database Sharding Topology has been provisioned using the Sharding Controller in Oracle Database Kubernetes Operator, you can follow the steps in this document to connect to the Sharded Database or to the individual Shards: [Database Connectivity](./doc/sharding/provisioning/database_connection.md) +After the Oracle Database Sharding Topology has been provisioned using the Sharding Controller in Oracle Database Kubernetes Operator, you can follow the steps in this document to connect to the Sharded Database or to the individual Shards: [Database Connectivity](./provisioning/database_connection.md) ## Debugging and Troubleshooting -To debug the Oracle Database Sharding Topology provisioned using the Sharding Controller of Oracle Database Kubernetes Operator, follow this document: [Debugging and troubleshooting](./doc/sharding/provisioning/debugging.md) +To debug the Oracle Database Sharding Topology provisioned using the Sharding Controller of Oracle Database Kubernetes Operator, follow this document: [Debugging and troubleshooting](./provisioning/debugging.md) From 6b98955a45850f6a55d2b32fb24dd13a8e28b681 Mon Sep 17 00:00:00 2001 From: Ting-Lan Wang <61528376+harayuanwang@users.noreply.github.com> Date: Mon, 25 Oct 2021 11:57:34 -0700 Subject: [PATCH 009/628] Fixed broken links --- .../provisioning_by_cloning_db_from_gold_image_across_ads.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/sharding/provisioning/provisioning_by_cloning_db_from_gold_image_across_ads.md b/doc/sharding/provisioning/provisioning_by_cloning_db_from_gold_image_across_ads.md index ae1d7f50..75a6f743 100644 --- a/doc/sharding/provisioning/provisioning_by_cloning_db_from_gold_image_across_ads.md +++ b/doc/sharding/provisioning/provisioning_by_cloning_db_from_gold_image_across_ads.md @@ -29,9 +29,9 @@ NOTE: In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `shard_prov_clone_across_ads.yaml`. - * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../ORACLE_SHARDING_CONTROLLER_README.md#3-oracle-database-and-global-data-services-docker-images) + * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../README.md#3-oracle-database-and-global-data-services-docker-images) -Use the file: [shard_prov_clone_across_ads.yaml](./doc/sharding/provisioning/shard_prov_clone_across_ads.yaml) for this use case as below: +Use the file: [shard_prov_clone_across_ads.yaml](./shard_prov_clone_across_ads.yaml) for this use case as below: 1. Deploy the `shard_prov_clone_across_ads.yaml` file: ```sh From ee3ddfaef3350083ac141638e981b9fa99f4861d Mon Sep 17 00:00:00 2001 From: Ting-Lan Wang <61528376+harayuanwang@users.noreply.github.com> Date: Mon, 25 Oct 2021 11:58:25 -0700 Subject: [PATCH 010/628] Fixed broken links --- .../provisioning_by_cloning_db_gold_image_in_same_ad.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/sharding/provisioning/provisioning_by_cloning_db_gold_image_in_same_ad.md b/doc/sharding/provisioning/provisioning_by_cloning_db_gold_image_in_same_ad.md index d633989f..531c3839 100644 --- a/doc/sharding/provisioning/provisioning_by_cloning_db_gold_image_in_same_ad.md +++ b/doc/sharding/provisioning/provisioning_by_cloning_db_gold_image_in_same_ad.md @@ -25,9 +25,9 @@ Choosing this option takes substantially less time during the Oracle Database Sh In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `shard_prov.yaml`. - * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../ORACLE_SHARDING_CONTROLLER_README.md#3-oracle-database-and-global-data-services-docker-images) + * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../README.md#3-oracle-database-and-global-data-services-docker-images) -Use the file: [shard_prov_clone.yaml](./doc/sharding/provisioning/shard_prov_clone.yaml) for this use case as below: +Use the file: [shard_prov_clone.yaml](./shard_prov_clone.yaml) for this use case as below: 1. Deploy the `shard_prov_clone.yaml` file: ```sh From cacb7894a85bf76d2803cff7d73b419198a258c9 Mon Sep 17 00:00:00 2001 From: Ting-Lan Wang <61528376+harayuanwang@users.noreply.github.com> Date: Mon, 25 Oct 2021 11:59:45 -0700 Subject: [PATCH 011/628] Fixed broken links --- .../provisioning_persistent_volume_having_db_gold_image.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/doc/sharding/provisioning/provisioning_persistent_volume_having_db_gold_image.md b/doc/sharding/provisioning/provisioning_persistent_volume_having_db_gold_image.md index 106fbb85..581fc62c 100644 --- a/doc/sharding/provisioning/provisioning_persistent_volume_having_db_gold_image.md +++ b/doc/sharding/provisioning/provisioning_persistent_volume_having_db_gold_image.md @@ -14,9 +14,9 @@ This example uses file `oraclesi.yaml` to provision a single instance Oracle Dat In this example, we are using pre-built Oracle Database image available on [Oracle Container Registry](https://container-registry.oracle.com/) * To pull the above image from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. * If you plan to use image built by you, you need to change `image` tag with the image you have built in your enviornment in file `oraclesi.yaml`. - * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../ORACLE_SHARDING_CONTROLLER_README.md#3-oracle-database-and-global-data-services-docker-images) + * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../README.md#3-oracle-database-and-global-data-services-docker-images) -1. Use this YAML file: [oraclesi.yaml](./doc/sharding/provisioning/oraclesi.yaml) for this use case. +1. Use this YAML file: [oraclesi.yaml](./oraclesi.yaml) for this use case. Use the following command to deploy the `oraclesi.yaml` file: @@ -26,7 +26,7 @@ In this example, we are using pre-built Oracle Database image available on [Orac 2. After the Database Deployment is completed, switch to the Single Oracle Database Pod and confirm that the Database Instance is UP and running in RW Mode. 3. Shut down the database instance cleanly, using `shutdown immediate` from the SQLPLUS Prompt. -4. For this use case, use the modified file [oraclesi_pvc_commented.yaml](./doc/sharding/provisioning/oraclesi_pvc_commented.yaml). This file is a copy of the file `oraclesi.yaml` but to keep the Persistent Volume claim and delete the database deployment, it has the lines of the Persistent Volume Claim commended out. +4. For this use case, use the modified file [oraclesi_pvc_commented.yaml](./oraclesi_pvc_commented.yaml). This file is a copy of the file `oraclesi.yaml` but to keep the Persistent Volume claim and delete the database deployment, it has the lines of the Persistent Volume Claim commended out. 5. Delete all the components provisioned for Single Oracle Database Deployment EXCEPT the Persistent Volume Claim by applying this file: ```sh From 19e126b0228e503b5e368bb094e5b18cd374c2b2 Mon Sep 17 00:00:00 2001 From: Ting-Lan Wang <61528376+harayuanwang@users.noreply.github.com> Date: Mon, 25 Oct 2021 12:00:47 -0700 Subject: [PATCH 012/628] Fixed broken links --- .../provisioning/provisioning_with_control_on_resources.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/sharding/provisioning/provisioning_with_control_on_resources.md b/doc/sharding/provisioning/provisioning_with_control_on_resources.md index e30ce506..734cad89 100644 --- a/doc/sharding/provisioning/provisioning_with_control_on_resources.md +++ b/doc/sharding/provisioning/provisioning_with_control_on_resources.md @@ -14,9 +14,9 @@ This example uses `shard_prov_memory_cpu.yaml` to provision an Oracle Database s In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `shard_prov_memory_cpu.yaml`. - * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../ORACLE_SHARDING_CONTROLLER_README.md#3-oracle-database-and-global-data-services-docker-images) + * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../README.md#3-oracle-database-and-global-data-services-docker-images) -Use the YAML file [shard_prov_memory_cpu.yaml](./doc/sharding/provisioning/shard_prov_memory_cpu.yaml). +Use the YAML file [shard_prov_memory_cpu.yaml](./shard_prov_memory_cpu.yaml). 1. Deploy the `shard_prov_memory_cpu.yaml` file: From 3ec37769e819e127cdcc7292fc95812363f4cb44 Mon Sep 17 00:00:00 2001 From: Ting-Lan Wang <61528376+harayuanwang@users.noreply.github.com> Date: Mon, 25 Oct 2021 12:01:38 -0700 Subject: [PATCH 013/628] Fixed broken links --- .../provisioning_with_notification_using_oci_notification.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/sharding/provisioning/provisioning_with_notification_using_oci_notification.md b/doc/sharding/provisioning/provisioning_with_notification_using_oci_notification.md index a551b401..7df7bc05 100644 --- a/doc/sharding/provisioning/provisioning_with_notification_using_oci_notification.md +++ b/doc/sharding/provisioning/provisioning_with_notification_using_oci_notification.md @@ -62,9 +62,9 @@ To do this: In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `shard_prov.yaml`. - * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../ORACLE_SHARDING_CONTROLLER_README.md#3-oracle-database-and-global-data-services-docker-images) + * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../README.md#3-oracle-database-and-global-data-services-docker-images) -Use the file: [shard_prov_send_notification.yaml](./doc/sharding/provisioning/shard_prov_send_notification.yaml) for this use case as below: +Use the file: [shard_prov_send_notification.yaml](./shard_prov_send_notification.yaml) for this use case as below: 1. Deploy the `shard_prov_send_notification.yaml` file: ```sh From 8bead00e0586f4be8148955c67d1a843ee3aca56 Mon Sep 17 00:00:00 2001 From: Ting-Lan Wang <61528376+harayuanwang@users.noreply.github.com> Date: Mon, 25 Oct 2021 12:02:08 -0700 Subject: [PATCH 014/628] Fixed broken links --- .../provisioning/provisioning_without_db_gold_image.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/sharding/provisioning/provisioning_without_db_gold_image.md b/doc/sharding/provisioning/provisioning_without_db_gold_image.md index d806b97c..0a908b52 100644 --- a/doc/sharding/provisioning/provisioning_without_db_gold_image.md +++ b/doc/sharding/provisioning/provisioning_without_db_gold_image.md @@ -15,10 +15,10 @@ This example uses `shard_prov.yaml` to provision an Oracle Database sharding top In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `shard_prov.yaml`. - * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../ORACLE_SHARDING_CONTROLLER_README.md#3-oracle-database-and-global-data-services-docker-images) + * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../README.md#3-oracle-database-and-global-data-services-docker-images) -Use the file: [shard_prov.yaml](./doc/sharding/provisioning/shard_prov.yaml) for this use case as below: +Use the file: [shard_prov.yaml](./shard_prov.yaml) for this use case as below: 1. Deploy the `shard_prov.yaml` file: ```sh From 77d528a14422e3df3c150f6056c6341f860242c6 Mon Sep 17 00:00:00 2001 From: Ting-Lan Wang <61528376+harayuanwang@users.noreply.github.com> Date: Mon, 25 Oct 2021 12:02:42 -0700 Subject: [PATCH 015/628] Fixed broken links --- .../provisioning/scale_in_delete_an_existing_shard.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/sharding/provisioning/scale_in_delete_an_existing_shard.md b/doc/sharding/provisioning/scale_in_delete_an_existing_shard.md index 8f793c96..9334741b 100644 --- a/doc/sharding/provisioning/scale_in_delete_an_existing_shard.md +++ b/doc/sharding/provisioning/scale_in_delete_an_existing_shard.md @@ -14,13 +14,13 @@ In this use case, the existing database Sharding is having: In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `shard_prov.yaml`. - * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../ORACLE_SHARDING_CONTROLLER_README.md#3-oracle-database-and-global-data-services-docker-images) + * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../README.md#3-oracle-database-and-global-data-services-docker-images) NOTE: Use tag `isDelete: true` to delete the shard you want. This use case deletes the shard `shard2` from the above Sharding Topology. -Use the file: [shard_prov_delshard.yaml](./doc/sharding/provisioning/shard_prov_delshard.yaml) for this use case as below: +Use the file: [shard_prov_delshard.yaml](./shard_prov_delshard.yaml) for this use case as below: 1. Deploy the `shard_prov_delshard.yaml` file: ```sh From 784e460b95a854bf873b509bdc1b74ada7b350e5 Mon Sep 17 00:00:00 2001 From: Ting-Lan Wang <61528376+harayuanwang@users.noreply.github.com> Date: Mon, 25 Oct 2021 12:03:12 -0700 Subject: [PATCH 016/628] Fixed broken links --- doc/sharding/provisioning/scale_out_add_shards.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/sharding/provisioning/scale_out_add_shards.md b/doc/sharding/provisioning/scale_out_add_shards.md index a8bb5b11..174d8c6c 100644 --- a/doc/sharding/provisioning/scale_out_add_shards.md +++ b/doc/sharding/provisioning/scale_out_add_shards.md @@ -12,11 +12,11 @@ In this use case, the existing Oracle Database sharding topology is having: In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `shard_prov.yaml`. - * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../ORACLE_SHARDING_CONTROLLER_README.md#3-oracle-database-and-global-data-services-docker-images) + * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../README.md#3-oracle-database-and-global-data-services-docker-images) This use case adds three new shards `shard3`,`shard4`,`shard4` to above Sharding Topology. -Use the file: [shard_prov_extshard.yaml](./doc/sharding/provisioning/shard_prov_extshard.yaml) for this use case as below: +Use the file: [shard_prov_extshard.yaml](./shard_prov_extshard.yaml) for this use case as below: 1. Deploy the `shard_prov_extshard.yaml` file: ```sh From dd405d43b74ba401a71f7bbd260dcd506a6b4d97 Mon Sep 17 00:00:00 2001 From: Ting-Lan Wang <61528376+harayuanwang@users.noreply.github.com> Date: Mon, 25 Oct 2021 12:05:08 -0700 Subject: [PATCH 017/628] Update anchors --- doc/sidb/README.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/doc/sidb/README.md b/doc/sidb/README.md index 0ed5d7d4..92dea684 100644 --- a/doc/sidb/README.md +++ b/doc/sidb/README.md @@ -2,11 +2,11 @@ Oracle Database Operator for Kubernetes (the operator) includes the Single Instance Database Controller that enables provisioning, cloning, and patching of Oracle Single Instance Databases on Kubernetes. The following sections explain the setup and functionality of the operator -* [Prerequisites](README.md#prerequisites) -* [Kind SingleInstanceDatabase Resource](README.md#kind-singleinstancedatabase-resource) -* [Provision New Database](README.md#provision-new-database) -* [Clone Existing Database](README.md#clone-existing-database) -* [Patch/Rollback Database](README.md#patchrollback-database) +* [Prerequisites](#prerequisites) +* [Kind SingleInstanceDatabase Resource](#kind-singleinstancedatabase-resource) +* [Provision New Database](#provision-new-database) +* [Clone Existing Database](#clone-existing-database) +* [Patch/Rollback Database](#patchrollback-database) ## Prerequisites From 4bc985b17dce407ee697111dfb8d1a48db6bb568 Mon Sep 17 00:00:00 2001 From: Ting-Lan Wang <61528376+harayuanwang@users.noreply.github.com> Date: Mon, 25 Oct 2021 12:08:37 -0700 Subject: [PATCH 018/628] Fixed broken links --- PREREQUISITES.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/PREREQUISITES.md b/PREREQUISITES.md index af82a341..0dd557ae 100644 --- a/PREREQUISITES.md +++ b/PREREQUISITES.md @@ -30,4 +30,4 @@ If you intent to use `OraOperator` to handle Oracle Database Single Instance lif ### Prerequites for Sharded Databases (SHARDING) - If you intent to use OraOperator to handle the lifecycle of Oracle Database deployed with Oracle Sharding, then read [Sharded Database Prerequisites](./doc/sharding/ORACLE_SHARDING_CONTROLLER_README.md#prerequsites-for-running-oracle-sharding-database-controller) + If you intent to use OraOperator to handle the lifecycle of Oracle Database deployed with Oracle Sharding, then read [Sharded Database Prerequisites](./doc/sharding/README.md#prerequsites-for-running-oracle-sharding-database-controller) From 755c32d50974481cc1207193c863ba94a61ca3de Mon Sep 17 00:00:00 2001 From: Ting-Lan Wang <61528376+harayuanwang@users.noreply.github.com> Date: Mon, 25 Oct 2021 12:12:42 -0700 Subject: [PATCH 019/628] Fixed broken links --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 80aa3b2e..30c9528e 100644 --- a/README.md +++ b/README.md @@ -92,9 +92,9 @@ For more details, see [Oracle Database Operator Installation Instrunctions](./do The quickstarts are designed for specific database configurations, including: -* [Oracle Autonomous Database](./doc/adb/ORACLE_ADB_CONTROLLER_README.md) -* [Oracle Database Single Instance configuration](./doc/sidb/ORACLE_SIDB_CONTROLLER_README.md) -* [Oracle Database configured with Oracle Sharding](./doc/sharding/ORACLE_SHARDING_CONTROLLER_README.md) +* [Oracle Autonomous Database](./doc/adb/README.md) +* [Oracle Database Single Instance configuration](./doc/sidb/README.md) +* [Oracle Database configured with Oracle Sharding](./doc/sharding/README.md) YAML file templates are available under [`/config/samples`](./config/samples/). You can copy and edit these template files to configure them for your use cases. From 011970b1901efcde9cf966456fa2ac55a1eb4349 Mon Sep 17 00:00:00 2001 From: Param Saini Date: Mon, 25 Oct 2021 12:29:40 -0700 Subject: [PATCH 020/628] Update README.md --- doc/sharding/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/sharding/README.md b/doc/sharding/README.md index 018043aa..6da5adad 100644 --- a/doc/sharding/README.md +++ b/doc/sharding/README.md @@ -62,8 +62,8 @@ Choose one of the following deployment options: **Build your own Oracle Database and Global Data Services Docker Images:** You can build these images using instructions provided on Oracle official GitHub Repositories: - [Oracle Global Data Services Image](https://github.com/oracle/db-sharding/tree/master/docker-based-sharding-deployment/dockerfiles) - [Oracle Database Image](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance) + * [Oracle Global Data Services Image](https://github.com/oracle/db-sharding/tree/master/docker-based-sharding-deployment/dockerfiles) + * [Oracle Database Image](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance) After the images are ready, push them to your Docker Images Repository, so that you can pull them during Oracle Database Sharding topology provisioning. From 76429c65951d83fb511b52f5f1309ee61fecf667 Mon Sep 17 00:00:00 2001 From: Param Saini Date: Mon, 25 Oct 2021 12:31:14 -0700 Subject: [PATCH 021/628] Update README.md --- doc/sharding/README.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/doc/sharding/README.md b/doc/sharding/README.md index 6da5adad..beb155d6 100644 --- a/doc/sharding/README.md +++ b/doc/sharding/README.md @@ -28,9 +28,8 @@ To create a Sharding Topology, complete the steps in the following sections belo 1. [Prerequsites for running Oracle Sharding Database Controller](#prerequsites-for-running-oracle-sharding-database-controller) 2. [Provisioning Sharding Topology in a Cloud based Kubernetes Cluster (OKE in this case)](#provisioning-sharding-topology-in-a-cloud-based-kubernetes-cluster-oke-in-this-case) -3. [Provisioning Sharding Topology in an On-Premise Kubernetes Cluster (OLCNE in this case)](#provisioning-sharding-topology-in-an-on-premise-kubernetes-cluster-olcne-in-this-case) -4. [Connecting to Shard Databases](#connecting-to-shard-databases) -5. [Debugging and Troubleshooting](#debugging-and-troubleshooting) +3. [Connecting to Shard Databases](#connecting-to-shard-databases) +4. [Debugging and Troubleshooting](#debugging-and-troubleshooting) **Note** Before proceeding to the next section, you must complete the instructions given in each section, based on your enviornment, before proceeding to next section. From b15fd11d6014341a1d97c6b14c29d2f1919c643f Mon Sep 17 00:00:00 2001 From: mmahipal Date: Tue, 26 Oct 2021 13:03:51 +0530 Subject: [PATCH 022/628] Update README.md to remove Operator Image Pull Secrets --- README.md | 13 +------------ doc/installation/OPERATOR_INSTALLATION_README.md | 12 ++---------- 2 files changed, 3 insertions(+), 22 deletions(-) diff --git a/README.md b/README.md index 30c9528e..76d97495 100644 --- a/README.md +++ b/README.md @@ -49,17 +49,6 @@ Oracle strongly recommends that you ensure your system meets the following [Prer kubectl apply -f https://github.com/jetstack/cert-manager/releases/latest/download/cert-manager.yaml ``` -* ### Create Operator Image Pull Secrets - - Sign into [https://container-registry.oracle.com/](https://container-registry.oracle.com/) and accept the license agreement for the Operator image. - - Create an image pull secret for Oracle Container Registry: - - ```sh - kubectl create namespace oracle-database-operator-system - kubectl create secret docker-registry container-registry-secret -n oracle-database-operator-system --docker-server=container-registry.oracle.com --docker-username='' --docker-password='' --docker-email='' - ``` - ## Quick Install of the Operator To install the operator in the cluster quickly, you can use a single [oracle-database-operator.yaml](https://github.com/oracle/oracle-database-operator/blob/main/oracle-database-operator.yaml) file. Operator pod replicas are set to a default of 3 for High Availability, which can be scaled up and down. @@ -120,7 +109,7 @@ YAML file templates are available under [`/config/samples`](./config/samples/). Note: If the CRD instances are not deleted, and the operator is deleted by using the preceding command, then operator deployment and instance objects (pods,services,PVCs, and so on) are deleted. However, the CRD deletion stops responding, because the CRD instances have finalizers that can only be removed by the operator pod, which is deleted when the APIServices are deleted. -* ### Retain the CRDs and APIservices +* ### Retaining the CRDs and APIservices To delete the operator deployment and retain the CRDs, run the following commands: diff --git a/doc/installation/OPERATOR_INSTALLATION_README.md b/doc/installation/OPERATOR_INSTALLATION_README.md index d01c468f..d9c8b893 100644 --- a/doc/installation/OPERATOR_INSTALLATION_README.md +++ b/doc/installation/OPERATOR_INSTALLATION_README.md @@ -57,17 +57,9 @@ If you encounter an issue with these steps, then refer to the more detailed step Clone the [oracle-database-operator](https://github.com/oracle/oracle-database-operator) on your local machine ## Review and modify the Cloned Repository -Make changes as needed to the repository. +Make changes as needed to the repository. -If you want to run the operator locally outside the cluster, then run the following steps: - - ```sh - cd oracle-database-operator - make generate - make manifests - make install run - ``` -If you want to run the operator inside the cluster, then run the following steps: +## Build and push the Operator image ```sh cd oracle-database-operator From 4259e58252e7e4ea6f4facd41eaea6dbbc1a24b8 Mon Sep 17 00:00:00 2001 From: mmahipal Date: Tue, 26 Oct 2021 13:23:11 +0530 Subject: [PATCH 023/628] Revernt changes to Operator Installation --- doc/installation/OPERATOR_INSTALLATION_README.md | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/doc/installation/OPERATOR_INSTALLATION_README.md b/doc/installation/OPERATOR_INSTALLATION_README.md index d9c8b893..d01c468f 100644 --- a/doc/installation/OPERATOR_INSTALLATION_README.md +++ b/doc/installation/OPERATOR_INSTALLATION_README.md @@ -57,9 +57,17 @@ If you encounter an issue with these steps, then refer to the more detailed step Clone the [oracle-database-operator](https://github.com/oracle/oracle-database-operator) on your local machine ## Review and modify the Cloned Repository -Make changes as needed to the repository. +Make changes as needed to the repository. -## Build and push the Operator image +If you want to run the operator locally outside the cluster, then run the following steps: + + ```sh + cd oracle-database-operator + make generate + make manifests + make install run + ``` +If you want to run the operator inside the cluster, then run the following steps: ```sh cd oracle-database-operator From e06137838ad962c9ead246ed9a23c14e25baad33 Mon Sep 17 00:00:00 2001 From: mmahipal Date: Tue, 26 Oct 2021 23:13:54 +0530 Subject: [PATCH 024/628] Updated sidb readme, seperated yamls into seperate directories --- .../samples/{ => adb}/autonomousdatabase.yaml | 0 .../{ => adb}/autonomousdatabase_bind.yaml | 0 ...onomousdatabase_change_admin_password.yaml | 0 .../{ => adb}/autonomousdatabase_create.yaml | 0 .../autonomousdatabase_delete_resource.yaml | 0 .../{ => adb}/autonomousdatabase_rename.yaml | 0 .../{ => adb}/autonomousdatabase_scale.yaml | 0 ...tonomousdatabase_stop_start_terminate.yaml | 0 .../{ => adb}/autonomousdatabase_wallet.yaml | 0 .../sharding_v1alpha1_provshard.yaml | 0 ...sharding_v1alpha1_provshard_clonespec.yaml | 0 ...harding_v1alpha1_provshard_clonespec1.yaml | 0 .../sharding_v1alpha1_provshard_orig.yaml | 0 .../{ => sharding}/shardingdatabase.yaml | 0 .../{ => sidb}/singleinstancedatabase.yaml | 0 .../singleinstancedatabase_clone.yaml | 0 .../singleinstancedatabase_patch.yaml | 0 .../singleinstancedatabase_prov.yaml | 0 doc/adb/README.md | 32 +++++++++---------- doc/sidb/README.md | 14 ++++---- 20 files changed, 23 insertions(+), 23 deletions(-) rename config/samples/{ => adb}/autonomousdatabase.yaml (100%) rename config/samples/{ => adb}/autonomousdatabase_bind.yaml (100%) rename config/samples/{ => adb}/autonomousdatabase_change_admin_password.yaml (100%) rename config/samples/{ => adb}/autonomousdatabase_create.yaml (100%) rename config/samples/{ => adb}/autonomousdatabase_delete_resource.yaml (100%) rename config/samples/{ => adb}/autonomousdatabase_rename.yaml (100%) rename config/samples/{ => adb}/autonomousdatabase_scale.yaml (100%) rename config/samples/{ => adb}/autonomousdatabase_stop_start_terminate.yaml (100%) rename config/samples/{ => adb}/autonomousdatabase_wallet.yaml (100%) rename config/samples/{ => sharding}/sharding_v1alpha1_provshard.yaml (100%) rename config/samples/{ => sharding}/sharding_v1alpha1_provshard_clonespec.yaml (100%) rename config/samples/{ => sharding}/sharding_v1alpha1_provshard_clonespec1.yaml (100%) rename config/samples/{ => sharding}/sharding_v1alpha1_provshard_orig.yaml (100%) rename config/samples/{ => sharding}/shardingdatabase.yaml (100%) rename config/samples/{ => sidb}/singleinstancedatabase.yaml (100%) rename config/samples/{ => sidb}/singleinstancedatabase_clone.yaml (100%) rename config/samples/{ => sidb}/singleinstancedatabase_patch.yaml (100%) rename config/samples/{ => sidb}/singleinstancedatabase_prov.yaml (100%) diff --git a/config/samples/autonomousdatabase.yaml b/config/samples/adb/autonomousdatabase.yaml similarity index 100% rename from config/samples/autonomousdatabase.yaml rename to config/samples/adb/autonomousdatabase.yaml diff --git a/config/samples/autonomousdatabase_bind.yaml b/config/samples/adb/autonomousdatabase_bind.yaml similarity index 100% rename from config/samples/autonomousdatabase_bind.yaml rename to config/samples/adb/autonomousdatabase_bind.yaml diff --git a/config/samples/autonomousdatabase_change_admin_password.yaml b/config/samples/adb/autonomousdatabase_change_admin_password.yaml similarity index 100% rename from config/samples/autonomousdatabase_change_admin_password.yaml rename to config/samples/adb/autonomousdatabase_change_admin_password.yaml diff --git a/config/samples/autonomousdatabase_create.yaml b/config/samples/adb/autonomousdatabase_create.yaml similarity index 100% rename from config/samples/autonomousdatabase_create.yaml rename to config/samples/adb/autonomousdatabase_create.yaml diff --git a/config/samples/autonomousdatabase_delete_resource.yaml b/config/samples/adb/autonomousdatabase_delete_resource.yaml similarity index 100% rename from config/samples/autonomousdatabase_delete_resource.yaml rename to config/samples/adb/autonomousdatabase_delete_resource.yaml diff --git a/config/samples/autonomousdatabase_rename.yaml b/config/samples/adb/autonomousdatabase_rename.yaml similarity index 100% rename from config/samples/autonomousdatabase_rename.yaml rename to config/samples/adb/autonomousdatabase_rename.yaml diff --git a/config/samples/autonomousdatabase_scale.yaml b/config/samples/adb/autonomousdatabase_scale.yaml similarity index 100% rename from config/samples/autonomousdatabase_scale.yaml rename to config/samples/adb/autonomousdatabase_scale.yaml diff --git a/config/samples/autonomousdatabase_stop_start_terminate.yaml b/config/samples/adb/autonomousdatabase_stop_start_terminate.yaml similarity index 100% rename from config/samples/autonomousdatabase_stop_start_terminate.yaml rename to config/samples/adb/autonomousdatabase_stop_start_terminate.yaml diff --git a/config/samples/autonomousdatabase_wallet.yaml b/config/samples/adb/autonomousdatabase_wallet.yaml similarity index 100% rename from config/samples/autonomousdatabase_wallet.yaml rename to config/samples/adb/autonomousdatabase_wallet.yaml diff --git a/config/samples/sharding_v1alpha1_provshard.yaml b/config/samples/sharding/sharding_v1alpha1_provshard.yaml similarity index 100% rename from config/samples/sharding_v1alpha1_provshard.yaml rename to config/samples/sharding/sharding_v1alpha1_provshard.yaml diff --git a/config/samples/sharding_v1alpha1_provshard_clonespec.yaml b/config/samples/sharding/sharding_v1alpha1_provshard_clonespec.yaml similarity index 100% rename from config/samples/sharding_v1alpha1_provshard_clonespec.yaml rename to config/samples/sharding/sharding_v1alpha1_provshard_clonespec.yaml diff --git a/config/samples/sharding_v1alpha1_provshard_clonespec1.yaml b/config/samples/sharding/sharding_v1alpha1_provshard_clonespec1.yaml similarity index 100% rename from config/samples/sharding_v1alpha1_provshard_clonespec1.yaml rename to config/samples/sharding/sharding_v1alpha1_provshard_clonespec1.yaml diff --git a/config/samples/sharding_v1alpha1_provshard_orig.yaml b/config/samples/sharding/sharding_v1alpha1_provshard_orig.yaml similarity index 100% rename from config/samples/sharding_v1alpha1_provshard_orig.yaml rename to config/samples/sharding/sharding_v1alpha1_provshard_orig.yaml diff --git a/config/samples/shardingdatabase.yaml b/config/samples/sharding/shardingdatabase.yaml similarity index 100% rename from config/samples/shardingdatabase.yaml rename to config/samples/sharding/shardingdatabase.yaml diff --git a/config/samples/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml similarity index 100% rename from config/samples/singleinstancedatabase.yaml rename to config/samples/sidb/singleinstancedatabase.yaml diff --git a/config/samples/singleinstancedatabase_clone.yaml b/config/samples/sidb/singleinstancedatabase_clone.yaml similarity index 100% rename from config/samples/singleinstancedatabase_clone.yaml rename to config/samples/sidb/singleinstancedatabase_clone.yaml diff --git a/config/samples/singleinstancedatabase_patch.yaml b/config/samples/sidb/singleinstancedatabase_patch.yaml similarity index 100% rename from config/samples/singleinstancedatabase_patch.yaml rename to config/samples/sidb/singleinstancedatabase_patch.yaml diff --git a/config/samples/singleinstancedatabase_prov.yaml b/config/samples/sidb/singleinstancedatabase_prov.yaml similarity index 100% rename from config/samples/singleinstancedatabase_prov.yaml rename to config/samples/sidb/singleinstancedatabase_prov.yaml diff --git a/doc/adb/README.md b/doc/adb/README.md index 3055c1bc..58e4730c 100644 --- a/doc/adb/README.md +++ b/doc/adb/README.md @@ -40,7 +40,7 @@ Follow the steps to provision an Autonomous Database that will bind objects in y kubectl create secret generic admin-password --from-literal=admin-password='password_here' ``` -3. Add the following fields to the AutonomousDatabase resource definition. An example `.yaml` file is available here: [`config/samples/autonomousdatabase_create.yaml`](./../../config/samples/autonomousdatabase_create.yaml) +3. Add the following fields to the AutonomousDatabase resource definition. An example `.yaml` file is available here: [`config/samples/adb/autonomousdatabase_create.yaml`](./../../config/samples/adb/autonomousdatabase_create.yaml) | Attribute | Type | Description | Required? | |----|----|----|----| | `spec.details.compartmentOCID` | string | The [OCID](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm) of the compartment of the Autonomous Database. | Yes | @@ -83,7 +83,7 @@ Follow the steps to provision an Autonomous Database that will bind objects in y 4. Apply the yaml: ```sh - kubectl apply -f config/samples/autonomousdatabase_create.yaml + kubectl apply -f config/samples/adb/autonomousdatabase_create.yaml autonomousdatabase.database.oracle.com/autonomousdatabase-sample created ``` @@ -104,7 +104,7 @@ Other than provisioning a database, you can bind to an existing database in your ![adb-id-2](/images/adb/adb-id-2.png) -3. Add the following fields to the AutonomousDatabase resource definition. An example `.yaml` file is available here: [`config/samples/autonomousdatabase_bind.yaml`](./../../config/samples/autonomousdatabase_bind.yaml) +3. Add the following fields to the AutonomousDatabase resource definition. An example `.yaml` file is available here: [`config/samples/adb/autonomousdatabase_bind.yaml`](./../../config/samples/adb/autonomousdatabase_bind.yaml) | Attribute | Type | Description | Required? | |----|----|----|----| | `spec.details.autonomousDatabaseOCID` | string | The [OCID](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm) of the Autonomous Database you want to bind (create a reference) in your cluster. | Yes | @@ -129,7 +129,7 @@ Other than provisioning a database, you can bind to an existing database in your 4. Apply the yaml. ```sh - kubectl apply -f config/samples/autonomousdatabase_bind.yaml + kubectl apply -f config/samples/adb/autonomousdatabase_bind.yaml autonomousdatabase.database.oracle.com/autonomousdatabase-sample created ``` @@ -139,7 +139,7 @@ Other than provisioning a database, you can bind to an existing database in your Users can scale up or scale down the Oracle Autonomous Database OCPU core count or storage by updating the `cpuCoreCount` and `dataStorageSizeInTBs` parameters. The `isAutoScalingEnabled` indicates whether auto scaling is enabled. Here is an example of scaling the CPU count and storage size (TB) up to 2 and turning off the auto-scaling by updating the `autonomousdatabase-sample` custom resource. -1. An example YAML file is available here: [config/samples/autonomousdatabase_scale.yaml](./../../config/samples/autonomousdatabase_scale.yaml) +1. An example YAML file is available here: [config/samples/adb/autonomousdatabase_scale.yaml](./../../config/samples/adb/autonomousdatabase_scale.yaml) ```yaml --- @@ -161,7 +161,7 @@ Users can scale up or scale down the Oracle Autonomous Database OCPU core count 2. Apply the change using `kubectl`. ```sh - kubectl apply -f config/samples/autonomousdatabase_scale.yaml + kubectl apply -f config/samples/adb/autonomousdatabase_scale.yaml autonomousdatabase.database.oracle.com/autonomousdatabase-sample configured ``` @@ -171,7 +171,7 @@ Users can scale up or scale down the Oracle Autonomous Database OCPU core count You can rename the database by changing the values of the `dbName` and `displayName`, as follows: -1. An example YAML file is available here: [config/samples/autonomousdatabase_rename.yaml](./../../config/samples/autonomousdatabase_rename.yaml) +1. An example YAML file is available here: [config/samples/adb/autonomousdatabase_rename.yaml](./../../config/samples/adb/autonomousdatabase_rename.yaml) ```yaml --- @@ -195,7 +195,7 @@ You can rename the database by changing the values of the `dbName` and `displayN 2. Apply the change using `kubectl`. ```sh - kubectl apply -f config/samples/autonomousdatabase_rename.yaml + kubectl apply -f config/samples/adb/autonomousdatabase_rename.yaml autonomousdatabase.database.oracle.com/autonomousdatabase-sample configured ``` @@ -213,7 +213,7 @@ You can rename the database by changing the values of the `dbName` and `displayN \* The password must be between 12 and 30 characters long, and must contain at least 1 uppercase, 1 lowercase, and 1 numeric character. It cannot contain the double quote symbol (") or the username "admin", regardless of casing. -2. Update the example [config/samples/autonomousdatabase_change_admin_password.yaml](./../../config/samples/autonomousdatabase_change_admin_password.yaml) +2. Update the example [config/samples/adb/autonomousdatabase_change_admin_password.yaml](./../../config/samples/adb/autonomousdatabase_change_admin_password.yaml) ```yaml --- @@ -236,7 +236,7 @@ You can rename the database by changing the values of the `dbName` and `displayN 3. Apply the YAML. ```sh - kubectl apply -f config/samples/autonomousdatabase_change_admin_password.yaml + kubectl apply -f config/samples/adb/autonomousdatabase_change_admin_password.yaml autonomousdatabase.database.oracle.com/autonomousdatabase-sample configured ``` @@ -256,7 +256,7 @@ A client Wallet is required to connect to a shared Oracle Autonomous Database. U \* The password must be at least 8 characters long and must include at least 1 letter and either 1 numeric character or 1 special character. -2. Update the example [config/samples/autonomousdatabase_wallet.yaml](./../../config/samples/autonomousdatabase_wallet.yaml) +2. Update the example [config/samples/adb/autonomousdatabase_wallet.yaml](./../../config/samples/adb/autonomousdatabase_wallet.yaml) ```yaml --- @@ -282,7 +282,7 @@ A client Wallet is required to connect to a shared Oracle Autonomous Database. U 3. Apply the YAML ```sh - kubectl apply -f config/samples/autonomousdatabase_wallet.yaml + kubectl apply -f config/samples/adb/autonomousdatabase_wallet.yaml autonomousdatabase.database.oracle.com/autonomousdatabase-sample configured ``` @@ -309,7 +309,7 @@ Here's a list of the values you can set for `lifecycleState`: * `STOPPED`: to stop the database * `TERMINATED`: to terminate the database -1. A sample .yaml file is available here: [config/samples/autonomousdatabase_stop_start_terminate.yaml](./../../config/samples/autonomousdatabase_stop_start_terminate.yaml) +1. A sample .yaml file is available here: [config/samples/adb/autonomousdatabase_stop_start_terminate.yaml](./../../config/samples/adb/autonomousdatabase_stop_start_terminate.yaml) ```yaml --- @@ -329,7 +329,7 @@ Here's a list of the values you can set for `lifecycleState`: 2. Apply the change to stop the database. ```sh - kubectl apply -f config/samples/autonomousdatabase_stop_start_terminate.yaml + kubectl apply -f config/samples/adb/autonomousdatabase_stop_start_terminate.yaml autonomousdatabase.database.oracle.com/autonomousdatabase-sample configured ``` @@ -341,7 +341,7 @@ The `hardLink` defines the behavior when the resource is deleted from the cluste Follow the steps to delete the resource and terminate the Autonomous Database. -1. Use the example [autonomousdatabase_delete_resource.yaml](./../../config/samples/autonomousdatabase_delete_resource.yaml) which sets the attribute `hardLink` to true. +1. Use the example [autonomousdatabase_delete_resource.yaml](./../../config/samples/adb/autonomousdatabase_delete_resource.yaml) which sets the attribute `hardLink` to true. ```yaml --- @@ -361,7 +361,7 @@ Follow the steps to delete the resource and terminate the Autonomous Database. 2. Apply the yaml ```sh - kubectl apply -f config/samples/autonomousdatabase_delete_resource.yaml + kubectl apply -f config/samples/adb/autonomousdatabase_delete_resource.yaml autonomousdatabase.database.oracle.com/autonomousdatabase-sample configured ``` diff --git a/doc/sidb/README.md b/doc/sidb/README.md index 92dea684..9c30a5ed 100644 --- a/doc/sidb/README.md +++ b/doc/sidb/README.md @@ -21,7 +21,7 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI For the use cases detailed below a sample .yaml file is available at * Enterprise, Standard Editions - [config/samples/singleinstancedatabase.yaml](./../../config/samples/singleinstancedatabase.yaml) + [config/samples/sidb/singleinstancedatabase.yaml](./../../config/samples/sidb/singleinstancedatabase.yaml) **Note:** The `adminPassword` field of the above `singleinstancedatabase.yaml` yaml contains a secret for Single Instance Database creation (Provisioning a new database or cloning an existing database). This secret gets deleted after the database pod becomes ready for security reasons. @@ -118,7 +118,7 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI Creating a new database instance takes a while. When the 'status' status returns the response "Healthy", the Database is open for connections. ```sh -$ kubectl get singleinstancedatabase/sidb-sample --template={{.status.status}} +$ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.status}" Healthy ``` @@ -131,7 +131,7 @@ $ kubectl get singleinstancedatabase/sidb-sample --template={{.status.status}} respectively in the following command ```sh - $ kubectl get singleinstancedatabase/sidb-sample --template={{.status.connectString}} + $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.connectString}" 144.25.10.119:1521/ORCL ``` @@ -139,7 +139,7 @@ $ kubectl get singleinstancedatabase/sidb-sample --template={{.status.status}} The Oracle Database inside the container also has Oracle Enterprise Manager Express configured. To access OEM Express, start the browser and follow the URL: ```sh - $ kubectl get singleinstancedatabase/sidb-sample --template={{.status.oemExpressUrl}} + $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.oemExpressUrl}" https://144.25.10.119:5500/em ``` @@ -150,7 +150,7 @@ $ kubectl get singleinstancedatabase/sidb-sample --template={{.status.status}} kubectl apply or edit/patch commands . Enable archiveLog before turning ON flashBack . Turn OFF flashBack before disabling the archiveLog ```sh - $ kubectl --type merge -p '{"spec":{"forceLog": true}}' patch singleinstancedatabase/sidb-sample + $ kubectl patch singleinstancedatabase sidb-sample --type merge -p '{"spec":{"forceLog": true}}' singleinstancedatabase.database.oracle.com/sidb-sample patched ``` @@ -253,11 +253,11 @@ $ kubectl get singleinstancedatabase/sidb-sample --template={{.status.status}} and current release update version using the following commands ```sh - $ kubectl get singleinstancedatabase/sidb-sample --template={{.status.datafilesPatched}} + $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.datafilesPatched}" true - $ kubectl get singleinstancedatabase/sidb-sample --template={{.status.releaseUpdate} + $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.releaseUpdate}" 19.3.0.0.0 (29517242) ``` From c9cebaecd3cacc64c93a998046504dfca3ef701f Mon Sep 17 00:00:00 2001 From: mmahipal Date: Tue, 26 Oct 2021 23:13:54 +0530 Subject: [PATCH 025/628] Updated sidb readme, seperated yamls into seperate directories --- .../samples/{ => adb}/autonomousdatabase.yaml | 0 .../{ => adb}/autonomousdatabase_bind.yaml | 0 ...onomousdatabase_change_admin_password.yaml | 0 .../{ => adb}/autonomousdatabase_create.yaml | 0 .../autonomousdatabase_delete_resource.yaml | 0 .../{ => adb}/autonomousdatabase_rename.yaml | 0 .../{ => adb}/autonomousdatabase_scale.yaml | 0 ...tonomousdatabase_stop_start_terminate.yaml | 0 .../{ => adb}/autonomousdatabase_wallet.yaml | 0 .../sharding_v1alpha1_provshard.yaml | 0 ...sharding_v1alpha1_provshard_clonespec.yaml | 0 ...harding_v1alpha1_provshard_clonespec1.yaml | 0 .../sharding_v1alpha1_provshard_orig.yaml | 0 .../{ => sharding}/shardingdatabase.yaml | 0 .../{ => sidb}/singleinstancedatabase.yaml | 0 .../singleinstancedatabase_clone.yaml | 0 .../singleinstancedatabase_patch.yaml | 0 .../singleinstancedatabase_prov.yaml | 0 doc/adb/README.md | 32 +++++++++---------- doc/sidb/README.md | 14 ++++---- 20 files changed, 23 insertions(+), 23 deletions(-) rename config/samples/{ => adb}/autonomousdatabase.yaml (100%) rename config/samples/{ => adb}/autonomousdatabase_bind.yaml (100%) rename config/samples/{ => adb}/autonomousdatabase_change_admin_password.yaml (100%) rename config/samples/{ => adb}/autonomousdatabase_create.yaml (100%) rename config/samples/{ => adb}/autonomousdatabase_delete_resource.yaml (100%) rename config/samples/{ => adb}/autonomousdatabase_rename.yaml (100%) rename config/samples/{ => adb}/autonomousdatabase_scale.yaml (100%) rename config/samples/{ => adb}/autonomousdatabase_stop_start_terminate.yaml (100%) rename config/samples/{ => adb}/autonomousdatabase_wallet.yaml (100%) rename config/samples/{ => sharding}/sharding_v1alpha1_provshard.yaml (100%) rename config/samples/{ => sharding}/sharding_v1alpha1_provshard_clonespec.yaml (100%) rename config/samples/{ => sharding}/sharding_v1alpha1_provshard_clonespec1.yaml (100%) rename config/samples/{ => sharding}/sharding_v1alpha1_provshard_orig.yaml (100%) rename config/samples/{ => sharding}/shardingdatabase.yaml (100%) rename config/samples/{ => sidb}/singleinstancedatabase.yaml (100%) rename config/samples/{ => sidb}/singleinstancedatabase_clone.yaml (100%) rename config/samples/{ => sidb}/singleinstancedatabase_patch.yaml (100%) rename config/samples/{ => sidb}/singleinstancedatabase_prov.yaml (100%) diff --git a/config/samples/autonomousdatabase.yaml b/config/samples/adb/autonomousdatabase.yaml similarity index 100% rename from config/samples/autonomousdatabase.yaml rename to config/samples/adb/autonomousdatabase.yaml diff --git a/config/samples/autonomousdatabase_bind.yaml b/config/samples/adb/autonomousdatabase_bind.yaml similarity index 100% rename from config/samples/autonomousdatabase_bind.yaml rename to config/samples/adb/autonomousdatabase_bind.yaml diff --git a/config/samples/autonomousdatabase_change_admin_password.yaml b/config/samples/adb/autonomousdatabase_change_admin_password.yaml similarity index 100% rename from config/samples/autonomousdatabase_change_admin_password.yaml rename to config/samples/adb/autonomousdatabase_change_admin_password.yaml diff --git a/config/samples/autonomousdatabase_create.yaml b/config/samples/adb/autonomousdatabase_create.yaml similarity index 100% rename from config/samples/autonomousdatabase_create.yaml rename to config/samples/adb/autonomousdatabase_create.yaml diff --git a/config/samples/autonomousdatabase_delete_resource.yaml b/config/samples/adb/autonomousdatabase_delete_resource.yaml similarity index 100% rename from config/samples/autonomousdatabase_delete_resource.yaml rename to config/samples/adb/autonomousdatabase_delete_resource.yaml diff --git a/config/samples/autonomousdatabase_rename.yaml b/config/samples/adb/autonomousdatabase_rename.yaml similarity index 100% rename from config/samples/autonomousdatabase_rename.yaml rename to config/samples/adb/autonomousdatabase_rename.yaml diff --git a/config/samples/autonomousdatabase_scale.yaml b/config/samples/adb/autonomousdatabase_scale.yaml similarity index 100% rename from config/samples/autonomousdatabase_scale.yaml rename to config/samples/adb/autonomousdatabase_scale.yaml diff --git a/config/samples/autonomousdatabase_stop_start_terminate.yaml b/config/samples/adb/autonomousdatabase_stop_start_terminate.yaml similarity index 100% rename from config/samples/autonomousdatabase_stop_start_terminate.yaml rename to config/samples/adb/autonomousdatabase_stop_start_terminate.yaml diff --git a/config/samples/autonomousdatabase_wallet.yaml b/config/samples/adb/autonomousdatabase_wallet.yaml similarity index 100% rename from config/samples/autonomousdatabase_wallet.yaml rename to config/samples/adb/autonomousdatabase_wallet.yaml diff --git a/config/samples/sharding_v1alpha1_provshard.yaml b/config/samples/sharding/sharding_v1alpha1_provshard.yaml similarity index 100% rename from config/samples/sharding_v1alpha1_provshard.yaml rename to config/samples/sharding/sharding_v1alpha1_provshard.yaml diff --git a/config/samples/sharding_v1alpha1_provshard_clonespec.yaml b/config/samples/sharding/sharding_v1alpha1_provshard_clonespec.yaml similarity index 100% rename from config/samples/sharding_v1alpha1_provshard_clonespec.yaml rename to config/samples/sharding/sharding_v1alpha1_provshard_clonespec.yaml diff --git a/config/samples/sharding_v1alpha1_provshard_clonespec1.yaml b/config/samples/sharding/sharding_v1alpha1_provshard_clonespec1.yaml similarity index 100% rename from config/samples/sharding_v1alpha1_provshard_clonespec1.yaml rename to config/samples/sharding/sharding_v1alpha1_provshard_clonespec1.yaml diff --git a/config/samples/sharding_v1alpha1_provshard_orig.yaml b/config/samples/sharding/sharding_v1alpha1_provshard_orig.yaml similarity index 100% rename from config/samples/sharding_v1alpha1_provshard_orig.yaml rename to config/samples/sharding/sharding_v1alpha1_provshard_orig.yaml diff --git a/config/samples/shardingdatabase.yaml b/config/samples/sharding/shardingdatabase.yaml similarity index 100% rename from config/samples/shardingdatabase.yaml rename to config/samples/sharding/shardingdatabase.yaml diff --git a/config/samples/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml similarity index 100% rename from config/samples/singleinstancedatabase.yaml rename to config/samples/sidb/singleinstancedatabase.yaml diff --git a/config/samples/singleinstancedatabase_clone.yaml b/config/samples/sidb/singleinstancedatabase_clone.yaml similarity index 100% rename from config/samples/singleinstancedatabase_clone.yaml rename to config/samples/sidb/singleinstancedatabase_clone.yaml diff --git a/config/samples/singleinstancedatabase_patch.yaml b/config/samples/sidb/singleinstancedatabase_patch.yaml similarity index 100% rename from config/samples/singleinstancedatabase_patch.yaml rename to config/samples/sidb/singleinstancedatabase_patch.yaml diff --git a/config/samples/singleinstancedatabase_prov.yaml b/config/samples/sidb/singleinstancedatabase_prov.yaml similarity index 100% rename from config/samples/singleinstancedatabase_prov.yaml rename to config/samples/sidb/singleinstancedatabase_prov.yaml diff --git a/doc/adb/README.md b/doc/adb/README.md index 3055c1bc..58e4730c 100644 --- a/doc/adb/README.md +++ b/doc/adb/README.md @@ -40,7 +40,7 @@ Follow the steps to provision an Autonomous Database that will bind objects in y kubectl create secret generic admin-password --from-literal=admin-password='password_here' ``` -3. Add the following fields to the AutonomousDatabase resource definition. An example `.yaml` file is available here: [`config/samples/autonomousdatabase_create.yaml`](./../../config/samples/autonomousdatabase_create.yaml) +3. Add the following fields to the AutonomousDatabase resource definition. An example `.yaml` file is available here: [`config/samples/adb/autonomousdatabase_create.yaml`](./../../config/samples/adb/autonomousdatabase_create.yaml) | Attribute | Type | Description | Required? | |----|----|----|----| | `spec.details.compartmentOCID` | string | The [OCID](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm) of the compartment of the Autonomous Database. | Yes | @@ -83,7 +83,7 @@ Follow the steps to provision an Autonomous Database that will bind objects in y 4. Apply the yaml: ```sh - kubectl apply -f config/samples/autonomousdatabase_create.yaml + kubectl apply -f config/samples/adb/autonomousdatabase_create.yaml autonomousdatabase.database.oracle.com/autonomousdatabase-sample created ``` @@ -104,7 +104,7 @@ Other than provisioning a database, you can bind to an existing database in your ![adb-id-2](/images/adb/adb-id-2.png) -3. Add the following fields to the AutonomousDatabase resource definition. An example `.yaml` file is available here: [`config/samples/autonomousdatabase_bind.yaml`](./../../config/samples/autonomousdatabase_bind.yaml) +3. Add the following fields to the AutonomousDatabase resource definition. An example `.yaml` file is available here: [`config/samples/adb/autonomousdatabase_bind.yaml`](./../../config/samples/adb/autonomousdatabase_bind.yaml) | Attribute | Type | Description | Required? | |----|----|----|----| | `spec.details.autonomousDatabaseOCID` | string | The [OCID](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm) of the Autonomous Database you want to bind (create a reference) in your cluster. | Yes | @@ -129,7 +129,7 @@ Other than provisioning a database, you can bind to an existing database in your 4. Apply the yaml. ```sh - kubectl apply -f config/samples/autonomousdatabase_bind.yaml + kubectl apply -f config/samples/adb/autonomousdatabase_bind.yaml autonomousdatabase.database.oracle.com/autonomousdatabase-sample created ``` @@ -139,7 +139,7 @@ Other than provisioning a database, you can bind to an existing database in your Users can scale up or scale down the Oracle Autonomous Database OCPU core count or storage by updating the `cpuCoreCount` and `dataStorageSizeInTBs` parameters. The `isAutoScalingEnabled` indicates whether auto scaling is enabled. Here is an example of scaling the CPU count and storage size (TB) up to 2 and turning off the auto-scaling by updating the `autonomousdatabase-sample` custom resource. -1. An example YAML file is available here: [config/samples/autonomousdatabase_scale.yaml](./../../config/samples/autonomousdatabase_scale.yaml) +1. An example YAML file is available here: [config/samples/adb/autonomousdatabase_scale.yaml](./../../config/samples/adb/autonomousdatabase_scale.yaml) ```yaml --- @@ -161,7 +161,7 @@ Users can scale up or scale down the Oracle Autonomous Database OCPU core count 2. Apply the change using `kubectl`. ```sh - kubectl apply -f config/samples/autonomousdatabase_scale.yaml + kubectl apply -f config/samples/adb/autonomousdatabase_scale.yaml autonomousdatabase.database.oracle.com/autonomousdatabase-sample configured ``` @@ -171,7 +171,7 @@ Users can scale up or scale down the Oracle Autonomous Database OCPU core count You can rename the database by changing the values of the `dbName` and `displayName`, as follows: -1. An example YAML file is available here: [config/samples/autonomousdatabase_rename.yaml](./../../config/samples/autonomousdatabase_rename.yaml) +1. An example YAML file is available here: [config/samples/adb/autonomousdatabase_rename.yaml](./../../config/samples/adb/autonomousdatabase_rename.yaml) ```yaml --- @@ -195,7 +195,7 @@ You can rename the database by changing the values of the `dbName` and `displayN 2. Apply the change using `kubectl`. ```sh - kubectl apply -f config/samples/autonomousdatabase_rename.yaml + kubectl apply -f config/samples/adb/autonomousdatabase_rename.yaml autonomousdatabase.database.oracle.com/autonomousdatabase-sample configured ``` @@ -213,7 +213,7 @@ You can rename the database by changing the values of the `dbName` and `displayN \* The password must be between 12 and 30 characters long, and must contain at least 1 uppercase, 1 lowercase, and 1 numeric character. It cannot contain the double quote symbol (") or the username "admin", regardless of casing. -2. Update the example [config/samples/autonomousdatabase_change_admin_password.yaml](./../../config/samples/autonomousdatabase_change_admin_password.yaml) +2. Update the example [config/samples/adb/autonomousdatabase_change_admin_password.yaml](./../../config/samples/adb/autonomousdatabase_change_admin_password.yaml) ```yaml --- @@ -236,7 +236,7 @@ You can rename the database by changing the values of the `dbName` and `displayN 3. Apply the YAML. ```sh - kubectl apply -f config/samples/autonomousdatabase_change_admin_password.yaml + kubectl apply -f config/samples/adb/autonomousdatabase_change_admin_password.yaml autonomousdatabase.database.oracle.com/autonomousdatabase-sample configured ``` @@ -256,7 +256,7 @@ A client Wallet is required to connect to a shared Oracle Autonomous Database. U \* The password must be at least 8 characters long and must include at least 1 letter and either 1 numeric character or 1 special character. -2. Update the example [config/samples/autonomousdatabase_wallet.yaml](./../../config/samples/autonomousdatabase_wallet.yaml) +2. Update the example [config/samples/adb/autonomousdatabase_wallet.yaml](./../../config/samples/adb/autonomousdatabase_wallet.yaml) ```yaml --- @@ -282,7 +282,7 @@ A client Wallet is required to connect to a shared Oracle Autonomous Database. U 3. Apply the YAML ```sh - kubectl apply -f config/samples/autonomousdatabase_wallet.yaml + kubectl apply -f config/samples/adb/autonomousdatabase_wallet.yaml autonomousdatabase.database.oracle.com/autonomousdatabase-sample configured ``` @@ -309,7 +309,7 @@ Here's a list of the values you can set for `lifecycleState`: * `STOPPED`: to stop the database * `TERMINATED`: to terminate the database -1. A sample .yaml file is available here: [config/samples/autonomousdatabase_stop_start_terminate.yaml](./../../config/samples/autonomousdatabase_stop_start_terminate.yaml) +1. A sample .yaml file is available here: [config/samples/adb/autonomousdatabase_stop_start_terminate.yaml](./../../config/samples/adb/autonomousdatabase_stop_start_terminate.yaml) ```yaml --- @@ -329,7 +329,7 @@ Here's a list of the values you can set for `lifecycleState`: 2. Apply the change to stop the database. ```sh - kubectl apply -f config/samples/autonomousdatabase_stop_start_terminate.yaml + kubectl apply -f config/samples/adb/autonomousdatabase_stop_start_terminate.yaml autonomousdatabase.database.oracle.com/autonomousdatabase-sample configured ``` @@ -341,7 +341,7 @@ The `hardLink` defines the behavior when the resource is deleted from the cluste Follow the steps to delete the resource and terminate the Autonomous Database. -1. Use the example [autonomousdatabase_delete_resource.yaml](./../../config/samples/autonomousdatabase_delete_resource.yaml) which sets the attribute `hardLink` to true. +1. Use the example [autonomousdatabase_delete_resource.yaml](./../../config/samples/adb/autonomousdatabase_delete_resource.yaml) which sets the attribute `hardLink` to true. ```yaml --- @@ -361,7 +361,7 @@ Follow the steps to delete the resource and terminate the Autonomous Database. 2. Apply the yaml ```sh - kubectl apply -f config/samples/autonomousdatabase_delete_resource.yaml + kubectl apply -f config/samples/adb/autonomousdatabase_delete_resource.yaml autonomousdatabase.database.oracle.com/autonomousdatabase-sample configured ``` diff --git a/doc/sidb/README.md b/doc/sidb/README.md index 92dea684..9c30a5ed 100644 --- a/doc/sidb/README.md +++ b/doc/sidb/README.md @@ -21,7 +21,7 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI For the use cases detailed below a sample .yaml file is available at * Enterprise, Standard Editions - [config/samples/singleinstancedatabase.yaml](./../../config/samples/singleinstancedatabase.yaml) + [config/samples/sidb/singleinstancedatabase.yaml](./../../config/samples/sidb/singleinstancedatabase.yaml) **Note:** The `adminPassword` field of the above `singleinstancedatabase.yaml` yaml contains a secret for Single Instance Database creation (Provisioning a new database or cloning an existing database). This secret gets deleted after the database pod becomes ready for security reasons. @@ -118,7 +118,7 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI Creating a new database instance takes a while. When the 'status' status returns the response "Healthy", the Database is open for connections. ```sh -$ kubectl get singleinstancedatabase/sidb-sample --template={{.status.status}} +$ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.status}" Healthy ``` @@ -131,7 +131,7 @@ $ kubectl get singleinstancedatabase/sidb-sample --template={{.status.status}} respectively in the following command ```sh - $ kubectl get singleinstancedatabase/sidb-sample --template={{.status.connectString}} + $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.connectString}" 144.25.10.119:1521/ORCL ``` @@ -139,7 +139,7 @@ $ kubectl get singleinstancedatabase/sidb-sample --template={{.status.status}} The Oracle Database inside the container also has Oracle Enterprise Manager Express configured. To access OEM Express, start the browser and follow the URL: ```sh - $ kubectl get singleinstancedatabase/sidb-sample --template={{.status.oemExpressUrl}} + $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.oemExpressUrl}" https://144.25.10.119:5500/em ``` @@ -150,7 +150,7 @@ $ kubectl get singleinstancedatabase/sidb-sample --template={{.status.status}} kubectl apply or edit/patch commands . Enable archiveLog before turning ON flashBack . Turn OFF flashBack before disabling the archiveLog ```sh - $ kubectl --type merge -p '{"spec":{"forceLog": true}}' patch singleinstancedatabase/sidb-sample + $ kubectl patch singleinstancedatabase sidb-sample --type merge -p '{"spec":{"forceLog": true}}' singleinstancedatabase.database.oracle.com/sidb-sample patched ``` @@ -253,11 +253,11 @@ $ kubectl get singleinstancedatabase/sidb-sample --template={{.status.status}} and current release update version using the following commands ```sh - $ kubectl get singleinstancedatabase/sidb-sample --template={{.status.datafilesPatched}} + $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.datafilesPatched}" true - $ kubectl get singleinstancedatabase/sidb-sample --template={{.status.releaseUpdate} + $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.releaseUpdate}" 19.3.0.0.0 (29517242) ``` From 64251d3b44d6ad464189a8bcc9e649626fe91798 Mon Sep 17 00:00:00 2001 From: mmahipal Date: Wed, 27 Oct 2021 12:49:42 +0530 Subject: [PATCH 026/628] Changed doc to docs --- {doc => docs}/adb/ADB_PREREQUISITES.md | 0 {doc => docs}/adb/README.md | 0 {doc => docs}/installation/OPERATOR_INSTALLATION_README.md | 0 {doc => docs}/sharding/README.md | 0 .../provisioning/create_kubernetes_secret_for_db_user.md | 0 {doc => docs}/sharding/provisioning/database_connection.md | 0 {doc => docs}/sharding/provisioning/debugging.md | 0 {doc => docs}/sharding/provisioning/oraclesi.yaml | 0 .../sharding/provisioning/oraclesi_pvc_commented.yaml | 0 ...rovisioning_by_cloning_db_from_gold_image_across_ads.md | 0 .../provisioning_by_cloning_db_gold_image_in_same_ad.md | 0 .../provisioning_persistent_volume_having_db_gold_image.md | 0 .../provisioning/provisioning_with_control_on_resources.md | 0 ...rovisioning_with_notification_using_oci_notification.md | 0 .../provisioning/provisioning_without_db_gold_image.md | 0 .../provisioning/scale_in_delete_an_existing_shard.md | 0 .../sharding/provisioning/scale_out_add_shards.md | 0 {doc => docs}/sharding/provisioning/shard_prov.yaml | 0 {doc => docs}/sharding/provisioning/shard_prov_clone.yaml | 0 .../sharding/provisioning/shard_prov_clone_across_ads.yaml | 0 .../sharding/provisioning/shard_prov_delshard.yaml | 0 .../sharding/provisioning/shard_prov_extshard.yaml | 0 .../sharding/provisioning/shard_prov_memory_cpu.yaml | 0 .../provisioning/shard_prov_send_notification.yaml | 0 {doc => docs}/sidb/README.md | 7 +++---- {doc => docs}/sidb/SIDB_PREREQUISITES.md | 0 26 files changed, 3 insertions(+), 4 deletions(-) rename {doc => docs}/adb/ADB_PREREQUISITES.md (100%) rename {doc => docs}/adb/README.md (100%) rename {doc => docs}/installation/OPERATOR_INSTALLATION_README.md (100%) rename {doc => docs}/sharding/README.md (100%) rename {doc => docs}/sharding/provisioning/create_kubernetes_secret_for_db_user.md (100%) rename {doc => docs}/sharding/provisioning/database_connection.md (100%) rename {doc => docs}/sharding/provisioning/debugging.md (100%) rename {doc => docs}/sharding/provisioning/oraclesi.yaml (100%) rename {doc => docs}/sharding/provisioning/oraclesi_pvc_commented.yaml (100%) rename {doc => docs}/sharding/provisioning/provisioning_by_cloning_db_from_gold_image_across_ads.md (100%) rename {doc => docs}/sharding/provisioning/provisioning_by_cloning_db_gold_image_in_same_ad.md (100%) rename {doc => docs}/sharding/provisioning/provisioning_persistent_volume_having_db_gold_image.md (100%) rename {doc => docs}/sharding/provisioning/provisioning_with_control_on_resources.md (100%) rename {doc => docs}/sharding/provisioning/provisioning_with_notification_using_oci_notification.md (100%) rename {doc => docs}/sharding/provisioning/provisioning_without_db_gold_image.md (100%) rename {doc => docs}/sharding/provisioning/scale_in_delete_an_existing_shard.md (100%) rename {doc => docs}/sharding/provisioning/scale_out_add_shards.md (100%) rename {doc => docs}/sharding/provisioning/shard_prov.yaml (100%) rename {doc => docs}/sharding/provisioning/shard_prov_clone.yaml (100%) rename {doc => docs}/sharding/provisioning/shard_prov_clone_across_ads.yaml (100%) rename {doc => docs}/sharding/provisioning/shard_prov_delshard.yaml (100%) rename {doc => docs}/sharding/provisioning/shard_prov_extshard.yaml (100%) rename {doc => docs}/sharding/provisioning/shard_prov_memory_cpu.yaml (100%) rename {doc => docs}/sharding/provisioning/shard_prov_send_notification.yaml (100%) rename {doc => docs}/sidb/README.md (96%) rename {doc => docs}/sidb/SIDB_PREREQUISITES.md (100%) diff --git a/doc/adb/ADB_PREREQUISITES.md b/docs/adb/ADB_PREREQUISITES.md similarity index 100% rename from doc/adb/ADB_PREREQUISITES.md rename to docs/adb/ADB_PREREQUISITES.md diff --git a/doc/adb/README.md b/docs/adb/README.md similarity index 100% rename from doc/adb/README.md rename to docs/adb/README.md diff --git a/doc/installation/OPERATOR_INSTALLATION_README.md b/docs/installation/OPERATOR_INSTALLATION_README.md similarity index 100% rename from doc/installation/OPERATOR_INSTALLATION_README.md rename to docs/installation/OPERATOR_INSTALLATION_README.md diff --git a/doc/sharding/README.md b/docs/sharding/README.md similarity index 100% rename from doc/sharding/README.md rename to docs/sharding/README.md diff --git a/doc/sharding/provisioning/create_kubernetes_secret_for_db_user.md b/docs/sharding/provisioning/create_kubernetes_secret_for_db_user.md similarity index 100% rename from doc/sharding/provisioning/create_kubernetes_secret_for_db_user.md rename to docs/sharding/provisioning/create_kubernetes_secret_for_db_user.md diff --git a/doc/sharding/provisioning/database_connection.md b/docs/sharding/provisioning/database_connection.md similarity index 100% rename from doc/sharding/provisioning/database_connection.md rename to docs/sharding/provisioning/database_connection.md diff --git a/doc/sharding/provisioning/debugging.md b/docs/sharding/provisioning/debugging.md similarity index 100% rename from doc/sharding/provisioning/debugging.md rename to docs/sharding/provisioning/debugging.md diff --git a/doc/sharding/provisioning/oraclesi.yaml b/docs/sharding/provisioning/oraclesi.yaml similarity index 100% rename from doc/sharding/provisioning/oraclesi.yaml rename to docs/sharding/provisioning/oraclesi.yaml diff --git a/doc/sharding/provisioning/oraclesi_pvc_commented.yaml b/docs/sharding/provisioning/oraclesi_pvc_commented.yaml similarity index 100% rename from doc/sharding/provisioning/oraclesi_pvc_commented.yaml rename to docs/sharding/provisioning/oraclesi_pvc_commented.yaml diff --git a/doc/sharding/provisioning/provisioning_by_cloning_db_from_gold_image_across_ads.md b/docs/sharding/provisioning/provisioning_by_cloning_db_from_gold_image_across_ads.md similarity index 100% rename from doc/sharding/provisioning/provisioning_by_cloning_db_from_gold_image_across_ads.md rename to docs/sharding/provisioning/provisioning_by_cloning_db_from_gold_image_across_ads.md diff --git a/doc/sharding/provisioning/provisioning_by_cloning_db_gold_image_in_same_ad.md b/docs/sharding/provisioning/provisioning_by_cloning_db_gold_image_in_same_ad.md similarity index 100% rename from doc/sharding/provisioning/provisioning_by_cloning_db_gold_image_in_same_ad.md rename to docs/sharding/provisioning/provisioning_by_cloning_db_gold_image_in_same_ad.md diff --git a/doc/sharding/provisioning/provisioning_persistent_volume_having_db_gold_image.md b/docs/sharding/provisioning/provisioning_persistent_volume_having_db_gold_image.md similarity index 100% rename from doc/sharding/provisioning/provisioning_persistent_volume_having_db_gold_image.md rename to docs/sharding/provisioning/provisioning_persistent_volume_having_db_gold_image.md diff --git a/doc/sharding/provisioning/provisioning_with_control_on_resources.md b/docs/sharding/provisioning/provisioning_with_control_on_resources.md similarity index 100% rename from doc/sharding/provisioning/provisioning_with_control_on_resources.md rename to docs/sharding/provisioning/provisioning_with_control_on_resources.md diff --git a/doc/sharding/provisioning/provisioning_with_notification_using_oci_notification.md b/docs/sharding/provisioning/provisioning_with_notification_using_oci_notification.md similarity index 100% rename from doc/sharding/provisioning/provisioning_with_notification_using_oci_notification.md rename to docs/sharding/provisioning/provisioning_with_notification_using_oci_notification.md diff --git a/doc/sharding/provisioning/provisioning_without_db_gold_image.md b/docs/sharding/provisioning/provisioning_without_db_gold_image.md similarity index 100% rename from doc/sharding/provisioning/provisioning_without_db_gold_image.md rename to docs/sharding/provisioning/provisioning_without_db_gold_image.md diff --git a/doc/sharding/provisioning/scale_in_delete_an_existing_shard.md b/docs/sharding/provisioning/scale_in_delete_an_existing_shard.md similarity index 100% rename from doc/sharding/provisioning/scale_in_delete_an_existing_shard.md rename to docs/sharding/provisioning/scale_in_delete_an_existing_shard.md diff --git a/doc/sharding/provisioning/scale_out_add_shards.md b/docs/sharding/provisioning/scale_out_add_shards.md similarity index 100% rename from doc/sharding/provisioning/scale_out_add_shards.md rename to docs/sharding/provisioning/scale_out_add_shards.md diff --git a/doc/sharding/provisioning/shard_prov.yaml b/docs/sharding/provisioning/shard_prov.yaml similarity index 100% rename from doc/sharding/provisioning/shard_prov.yaml rename to docs/sharding/provisioning/shard_prov.yaml diff --git a/doc/sharding/provisioning/shard_prov_clone.yaml b/docs/sharding/provisioning/shard_prov_clone.yaml similarity index 100% rename from doc/sharding/provisioning/shard_prov_clone.yaml rename to docs/sharding/provisioning/shard_prov_clone.yaml diff --git a/doc/sharding/provisioning/shard_prov_clone_across_ads.yaml b/docs/sharding/provisioning/shard_prov_clone_across_ads.yaml similarity index 100% rename from doc/sharding/provisioning/shard_prov_clone_across_ads.yaml rename to docs/sharding/provisioning/shard_prov_clone_across_ads.yaml diff --git a/doc/sharding/provisioning/shard_prov_delshard.yaml b/docs/sharding/provisioning/shard_prov_delshard.yaml similarity index 100% rename from doc/sharding/provisioning/shard_prov_delshard.yaml rename to docs/sharding/provisioning/shard_prov_delshard.yaml diff --git a/doc/sharding/provisioning/shard_prov_extshard.yaml b/docs/sharding/provisioning/shard_prov_extshard.yaml similarity index 100% rename from doc/sharding/provisioning/shard_prov_extshard.yaml rename to docs/sharding/provisioning/shard_prov_extshard.yaml diff --git a/doc/sharding/provisioning/shard_prov_memory_cpu.yaml b/docs/sharding/provisioning/shard_prov_memory_cpu.yaml similarity index 100% rename from doc/sharding/provisioning/shard_prov_memory_cpu.yaml rename to docs/sharding/provisioning/shard_prov_memory_cpu.yaml diff --git a/doc/sharding/provisioning/shard_prov_send_notification.yaml b/docs/sharding/provisioning/shard_prov_send_notification.yaml similarity index 100% rename from doc/sharding/provisioning/shard_prov_send_notification.yaml rename to docs/sharding/provisioning/shard_prov_send_notification.yaml diff --git a/doc/sidb/README.md b/docs/sidb/README.md similarity index 96% rename from doc/sidb/README.md rename to docs/sidb/README.md index 9c30a5ed..426e4483 100644 --- a/doc/sidb/README.md +++ b/docs/sidb/README.md @@ -237,12 +237,11 @@ $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.status}" Patched Oracle Docker images can be built using this [patching extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/patching) -* ### In Place Resources or Objects +* ### Patching - Edit the `.yaml` file of the database resourece/object and specify a new release update for release or image attributes. The database pods will be restarted - with the new release update image. For minimum downtime, ensure that you have mutiple replicas of the database pods running. + Edit the `singleinstancedatabase.yaml` file of the database resource/object and specify a new release update for release or image attributes and apply the `singleinstancedatabase.yaml` file. The database pods will be restarted with the new release update image. For minimum downtime, ensure that you have mutiple replicas of the database pods running. -* ### Out of Place +* ### Clone and Patch Clone your source database using the method of [cloning existing database](README.md#clone-existing-database) and specify a new release version/image for the cloned database. Use this method to enusure there are no patching related issues impacting your database performance/functionality diff --git a/doc/sidb/SIDB_PREREQUISITES.md b/docs/sidb/SIDB_PREREQUISITES.md similarity index 100% rename from doc/sidb/SIDB_PREREQUISITES.md rename to docs/sidb/SIDB_PREREQUISITES.md From 043fccc4e91d37ee197e4384cf835166d81c918b Mon Sep 17 00:00:00 2001 From: mmahipal Date: Wed, 27 Oct 2021 12:49:42 +0530 Subject: [PATCH 027/628] Changed doc to docs --- {doc => docs}/adb/ADB_PREREQUISITES.md | 0 {doc => docs}/adb/README.md | 0 {doc => docs}/installation/OPERATOR_INSTALLATION_README.md | 0 {doc => docs}/sharding/README.md | 0 .../provisioning/create_kubernetes_secret_for_db_user.md | 0 {doc => docs}/sharding/provisioning/database_connection.md | 0 {doc => docs}/sharding/provisioning/debugging.md | 0 {doc => docs}/sharding/provisioning/oraclesi.yaml | 0 .../sharding/provisioning/oraclesi_pvc_commented.yaml | 0 ...rovisioning_by_cloning_db_from_gold_image_across_ads.md | 0 .../provisioning_by_cloning_db_gold_image_in_same_ad.md | 0 .../provisioning_persistent_volume_having_db_gold_image.md | 0 .../provisioning/provisioning_with_control_on_resources.md | 0 ...rovisioning_with_notification_using_oci_notification.md | 0 .../provisioning/provisioning_without_db_gold_image.md | 0 .../provisioning/scale_in_delete_an_existing_shard.md | 0 .../sharding/provisioning/scale_out_add_shards.md | 0 {doc => docs}/sharding/provisioning/shard_prov.yaml | 0 {doc => docs}/sharding/provisioning/shard_prov_clone.yaml | 0 .../sharding/provisioning/shard_prov_clone_across_ads.yaml | 0 .../sharding/provisioning/shard_prov_delshard.yaml | 0 .../sharding/provisioning/shard_prov_extshard.yaml | 0 .../sharding/provisioning/shard_prov_memory_cpu.yaml | 0 .../provisioning/shard_prov_send_notification.yaml | 0 {doc => docs}/sidb/README.md | 7 +++---- {doc => docs}/sidb/SIDB_PREREQUISITES.md | 0 26 files changed, 3 insertions(+), 4 deletions(-) rename {doc => docs}/adb/ADB_PREREQUISITES.md (100%) rename {doc => docs}/adb/README.md (100%) rename {doc => docs}/installation/OPERATOR_INSTALLATION_README.md (100%) rename {doc => docs}/sharding/README.md (100%) rename {doc => docs}/sharding/provisioning/create_kubernetes_secret_for_db_user.md (100%) rename {doc => docs}/sharding/provisioning/database_connection.md (100%) rename {doc => docs}/sharding/provisioning/debugging.md (100%) rename {doc => docs}/sharding/provisioning/oraclesi.yaml (100%) rename {doc => docs}/sharding/provisioning/oraclesi_pvc_commented.yaml (100%) rename {doc => docs}/sharding/provisioning/provisioning_by_cloning_db_from_gold_image_across_ads.md (100%) rename {doc => docs}/sharding/provisioning/provisioning_by_cloning_db_gold_image_in_same_ad.md (100%) rename {doc => docs}/sharding/provisioning/provisioning_persistent_volume_having_db_gold_image.md (100%) rename {doc => docs}/sharding/provisioning/provisioning_with_control_on_resources.md (100%) rename {doc => docs}/sharding/provisioning/provisioning_with_notification_using_oci_notification.md (100%) rename {doc => docs}/sharding/provisioning/provisioning_without_db_gold_image.md (100%) rename {doc => docs}/sharding/provisioning/scale_in_delete_an_existing_shard.md (100%) rename {doc => docs}/sharding/provisioning/scale_out_add_shards.md (100%) rename {doc => docs}/sharding/provisioning/shard_prov.yaml (100%) rename {doc => docs}/sharding/provisioning/shard_prov_clone.yaml (100%) rename {doc => docs}/sharding/provisioning/shard_prov_clone_across_ads.yaml (100%) rename {doc => docs}/sharding/provisioning/shard_prov_delshard.yaml (100%) rename {doc => docs}/sharding/provisioning/shard_prov_extshard.yaml (100%) rename {doc => docs}/sharding/provisioning/shard_prov_memory_cpu.yaml (100%) rename {doc => docs}/sharding/provisioning/shard_prov_send_notification.yaml (100%) rename {doc => docs}/sidb/README.md (96%) rename {doc => docs}/sidb/SIDB_PREREQUISITES.md (100%) diff --git a/doc/adb/ADB_PREREQUISITES.md b/docs/adb/ADB_PREREQUISITES.md similarity index 100% rename from doc/adb/ADB_PREREQUISITES.md rename to docs/adb/ADB_PREREQUISITES.md diff --git a/doc/adb/README.md b/docs/adb/README.md similarity index 100% rename from doc/adb/README.md rename to docs/adb/README.md diff --git a/doc/installation/OPERATOR_INSTALLATION_README.md b/docs/installation/OPERATOR_INSTALLATION_README.md similarity index 100% rename from doc/installation/OPERATOR_INSTALLATION_README.md rename to docs/installation/OPERATOR_INSTALLATION_README.md diff --git a/doc/sharding/README.md b/docs/sharding/README.md similarity index 100% rename from doc/sharding/README.md rename to docs/sharding/README.md diff --git a/doc/sharding/provisioning/create_kubernetes_secret_for_db_user.md b/docs/sharding/provisioning/create_kubernetes_secret_for_db_user.md similarity index 100% rename from doc/sharding/provisioning/create_kubernetes_secret_for_db_user.md rename to docs/sharding/provisioning/create_kubernetes_secret_for_db_user.md diff --git a/doc/sharding/provisioning/database_connection.md b/docs/sharding/provisioning/database_connection.md similarity index 100% rename from doc/sharding/provisioning/database_connection.md rename to docs/sharding/provisioning/database_connection.md diff --git a/doc/sharding/provisioning/debugging.md b/docs/sharding/provisioning/debugging.md similarity index 100% rename from doc/sharding/provisioning/debugging.md rename to docs/sharding/provisioning/debugging.md diff --git a/doc/sharding/provisioning/oraclesi.yaml b/docs/sharding/provisioning/oraclesi.yaml similarity index 100% rename from doc/sharding/provisioning/oraclesi.yaml rename to docs/sharding/provisioning/oraclesi.yaml diff --git a/doc/sharding/provisioning/oraclesi_pvc_commented.yaml b/docs/sharding/provisioning/oraclesi_pvc_commented.yaml similarity index 100% rename from doc/sharding/provisioning/oraclesi_pvc_commented.yaml rename to docs/sharding/provisioning/oraclesi_pvc_commented.yaml diff --git a/doc/sharding/provisioning/provisioning_by_cloning_db_from_gold_image_across_ads.md b/docs/sharding/provisioning/provisioning_by_cloning_db_from_gold_image_across_ads.md similarity index 100% rename from doc/sharding/provisioning/provisioning_by_cloning_db_from_gold_image_across_ads.md rename to docs/sharding/provisioning/provisioning_by_cloning_db_from_gold_image_across_ads.md diff --git a/doc/sharding/provisioning/provisioning_by_cloning_db_gold_image_in_same_ad.md b/docs/sharding/provisioning/provisioning_by_cloning_db_gold_image_in_same_ad.md similarity index 100% rename from doc/sharding/provisioning/provisioning_by_cloning_db_gold_image_in_same_ad.md rename to docs/sharding/provisioning/provisioning_by_cloning_db_gold_image_in_same_ad.md diff --git a/doc/sharding/provisioning/provisioning_persistent_volume_having_db_gold_image.md b/docs/sharding/provisioning/provisioning_persistent_volume_having_db_gold_image.md similarity index 100% rename from doc/sharding/provisioning/provisioning_persistent_volume_having_db_gold_image.md rename to docs/sharding/provisioning/provisioning_persistent_volume_having_db_gold_image.md diff --git a/doc/sharding/provisioning/provisioning_with_control_on_resources.md b/docs/sharding/provisioning/provisioning_with_control_on_resources.md similarity index 100% rename from doc/sharding/provisioning/provisioning_with_control_on_resources.md rename to docs/sharding/provisioning/provisioning_with_control_on_resources.md diff --git a/doc/sharding/provisioning/provisioning_with_notification_using_oci_notification.md b/docs/sharding/provisioning/provisioning_with_notification_using_oci_notification.md similarity index 100% rename from doc/sharding/provisioning/provisioning_with_notification_using_oci_notification.md rename to docs/sharding/provisioning/provisioning_with_notification_using_oci_notification.md diff --git a/doc/sharding/provisioning/provisioning_without_db_gold_image.md b/docs/sharding/provisioning/provisioning_without_db_gold_image.md similarity index 100% rename from doc/sharding/provisioning/provisioning_without_db_gold_image.md rename to docs/sharding/provisioning/provisioning_without_db_gold_image.md diff --git a/doc/sharding/provisioning/scale_in_delete_an_existing_shard.md b/docs/sharding/provisioning/scale_in_delete_an_existing_shard.md similarity index 100% rename from doc/sharding/provisioning/scale_in_delete_an_existing_shard.md rename to docs/sharding/provisioning/scale_in_delete_an_existing_shard.md diff --git a/doc/sharding/provisioning/scale_out_add_shards.md b/docs/sharding/provisioning/scale_out_add_shards.md similarity index 100% rename from doc/sharding/provisioning/scale_out_add_shards.md rename to docs/sharding/provisioning/scale_out_add_shards.md diff --git a/doc/sharding/provisioning/shard_prov.yaml b/docs/sharding/provisioning/shard_prov.yaml similarity index 100% rename from doc/sharding/provisioning/shard_prov.yaml rename to docs/sharding/provisioning/shard_prov.yaml diff --git a/doc/sharding/provisioning/shard_prov_clone.yaml b/docs/sharding/provisioning/shard_prov_clone.yaml similarity index 100% rename from doc/sharding/provisioning/shard_prov_clone.yaml rename to docs/sharding/provisioning/shard_prov_clone.yaml diff --git a/doc/sharding/provisioning/shard_prov_clone_across_ads.yaml b/docs/sharding/provisioning/shard_prov_clone_across_ads.yaml similarity index 100% rename from doc/sharding/provisioning/shard_prov_clone_across_ads.yaml rename to docs/sharding/provisioning/shard_prov_clone_across_ads.yaml diff --git a/doc/sharding/provisioning/shard_prov_delshard.yaml b/docs/sharding/provisioning/shard_prov_delshard.yaml similarity index 100% rename from doc/sharding/provisioning/shard_prov_delshard.yaml rename to docs/sharding/provisioning/shard_prov_delshard.yaml diff --git a/doc/sharding/provisioning/shard_prov_extshard.yaml b/docs/sharding/provisioning/shard_prov_extshard.yaml similarity index 100% rename from doc/sharding/provisioning/shard_prov_extshard.yaml rename to docs/sharding/provisioning/shard_prov_extshard.yaml diff --git a/doc/sharding/provisioning/shard_prov_memory_cpu.yaml b/docs/sharding/provisioning/shard_prov_memory_cpu.yaml similarity index 100% rename from doc/sharding/provisioning/shard_prov_memory_cpu.yaml rename to docs/sharding/provisioning/shard_prov_memory_cpu.yaml diff --git a/doc/sharding/provisioning/shard_prov_send_notification.yaml b/docs/sharding/provisioning/shard_prov_send_notification.yaml similarity index 100% rename from doc/sharding/provisioning/shard_prov_send_notification.yaml rename to docs/sharding/provisioning/shard_prov_send_notification.yaml diff --git a/doc/sidb/README.md b/docs/sidb/README.md similarity index 96% rename from doc/sidb/README.md rename to docs/sidb/README.md index 9c30a5ed..426e4483 100644 --- a/doc/sidb/README.md +++ b/docs/sidb/README.md @@ -237,12 +237,11 @@ $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.status}" Patched Oracle Docker images can be built using this [patching extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/patching) -* ### In Place Resources or Objects +* ### Patching - Edit the `.yaml` file of the database resourece/object and specify a new release update for release or image attributes. The database pods will be restarted - with the new release update image. For minimum downtime, ensure that you have mutiple replicas of the database pods running. + Edit the `singleinstancedatabase.yaml` file of the database resource/object and specify a new release update for release or image attributes and apply the `singleinstancedatabase.yaml` file. The database pods will be restarted with the new release update image. For minimum downtime, ensure that you have mutiple replicas of the database pods running. -* ### Out of Place +* ### Clone and Patch Clone your source database using the method of [cloning existing database](README.md#clone-existing-database) and specify a new release version/image for the cloned database. Use this method to enusure there are no patching related issues impacting your database performance/functionality diff --git a/doc/sidb/SIDB_PREREQUISITES.md b/docs/sidb/SIDB_PREREQUISITES.md similarity index 100% rename from doc/sidb/SIDB_PREREQUISITES.md rename to docs/sidb/SIDB_PREREQUISITES.md From 0002118d73cc725ead45ac59239b66426ff54728 Mon Sep 17 00:00:00 2001 From: mmahipal Date: Wed, 27 Oct 2021 12:57:59 +0530 Subject: [PATCH 028/628] Changed SIDB readme --- docs/sidb/README.md | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 426e4483..c64c2a32 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -8,7 +8,6 @@ Oracle Database Operator for Kubernetes (the operator) includes the Single Insta * [Clone Existing Database](#clone-existing-database) * [Patch/Rollback Database](#patchrollback-database) - ## Prerequisites Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISITES.md). @@ -233,18 +232,17 @@ $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.status}" ## Patch/Rollback Database - Databases running in your cluster and managed by this operator can be patched or rolled back between release updates of the same major release. To patch databases, specify an image of the higher release update, and to roll back, specify an image of the lower release update. + Databases running in your cluster and managed by this operator can be patched or rolled back between release updates of the same major release. To patch databases, specify an image of the higher release update, and to roll back, specify an image of the lower release update. Patched Oracle Docker images can be built using this [patching extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/patching) -* ### Patching +* ### Patch the Database - Edit the `singleinstancedatabase.yaml` file of the database resource/object and specify a new release update for release or image attributes and apply the `singleinstancedatabase.yaml` file. The database pods will be restarted with the new release update image. For minimum downtime, ensure that you have mutiple replicas of the database pods running. + Edit and apply the `singleinstancedatabase.yaml` file of the database resource/object by specifying a new release update for image attributes. The database pods will be restarted with the new release update image. For minimum downtime, ensure that you have mutiple replicas of the database pods running. -* ### Clone and Patch +* ### Clone and Patch the Database - Clone your source database using the method of [cloning existing database](README.md#clone-existing-database) and specify a new release version/image for the - cloned database. Use this method to enusure there are no patching related issues impacting your database performance/functionality + Clone your source database using the method of [cloning existing database](README.md#clone-existing-database) and specify a new release image for the cloned database. Use this method to enusure there are no patching related issues impacting your database performance/functionality * ### Datapatch status From ea3dd930212d0fe25bc9b386ec0df19146b9d44e Mon Sep 17 00:00:00 2001 From: mmahipal Date: Wed, 27 Oct 2021 12:57:59 +0530 Subject: [PATCH 029/628] Changed SIDB readme --- docs/sidb/README.md | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 426e4483..c64c2a32 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -8,7 +8,6 @@ Oracle Database Operator for Kubernetes (the operator) includes the Single Insta * [Clone Existing Database](#clone-existing-database) * [Patch/Rollback Database](#patchrollback-database) - ## Prerequisites Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISITES.md). @@ -233,18 +232,17 @@ $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.status}" ## Patch/Rollback Database - Databases running in your cluster and managed by this operator can be patched or rolled back between release updates of the same major release. To patch databases, specify an image of the higher release update, and to roll back, specify an image of the lower release update. + Databases running in your cluster and managed by this operator can be patched or rolled back between release updates of the same major release. To patch databases, specify an image of the higher release update, and to roll back, specify an image of the lower release update. Patched Oracle Docker images can be built using this [patching extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/patching) -* ### Patching +* ### Patch the Database - Edit the `singleinstancedatabase.yaml` file of the database resource/object and specify a new release update for release or image attributes and apply the `singleinstancedatabase.yaml` file. The database pods will be restarted with the new release update image. For minimum downtime, ensure that you have mutiple replicas of the database pods running. + Edit and apply the `singleinstancedatabase.yaml` file of the database resource/object by specifying a new release update for image attributes. The database pods will be restarted with the new release update image. For minimum downtime, ensure that you have mutiple replicas of the database pods running. -* ### Clone and Patch +* ### Clone and Patch the Database - Clone your source database using the method of [cloning existing database](README.md#clone-existing-database) and specify a new release version/image for the - cloned database. Use this method to enusure there are no patching related issues impacting your database performance/functionality + Clone your source database using the method of [cloning existing database](README.md#clone-existing-database) and specify a new release image for the cloned database. Use this method to enusure there are no patching related issues impacting your database performance/functionality * ### Datapatch status From 2c5ca3934a463f0d1470588e701119360cc69955 Mon Sep 17 00:00:00 2001 From: mmahipal Date: Wed, 27 Oct 2021 13:12:24 +0530 Subject: [PATCH 030/628] Changed SIDB readme --- docs/sidb/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index c64c2a32..4046f98a 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -236,11 +236,11 @@ $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.status}" Patched Oracle Docker images can be built using this [patching extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/patching) -* ### Patch the Database +* ### Patch existing Database Edit and apply the `singleinstancedatabase.yaml` file of the database resource/object by specifying a new release update for image attributes. The database pods will be restarted with the new release update image. For minimum downtime, ensure that you have mutiple replicas of the database pods running. -* ### Clone and Patch the Database +* ### Clone and Patch Database Clone your source database using the method of [cloning existing database](README.md#clone-existing-database) and specify a new release image for the cloned database. Use this method to enusure there are no patching related issues impacting your database performance/functionality From b20ed5091401084fa41b6cf92aa67066bf2db790 Mon Sep 17 00:00:00 2001 From: mmahipal Date: Wed, 27 Oct 2021 13:12:24 +0530 Subject: [PATCH 031/628] Changed SIDB readme --- docs/sidb/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index c64c2a32..4046f98a 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -236,11 +236,11 @@ $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.status}" Patched Oracle Docker images can be built using this [patching extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/patching) -* ### Patch the Database +* ### Patch existing Database Edit and apply the `singleinstancedatabase.yaml` file of the database resource/object by specifying a new release update for image attributes. The database pods will be restarted with the new release update image. For minimum downtime, ensure that you have mutiple replicas of the database pods running. -* ### Clone and Patch the Database +* ### Clone and Patch Database Clone your source database using the method of [cloning existing database](README.md#clone-existing-database) and specify a new release image for the cloned database. Use this method to enusure there are no patching related issues impacting your database performance/functionality From 5d4d632b715d228688c77fc377fc9f83473a95dc Mon Sep 17 00:00:00 2001 From: mmahipal Date: Wed, 27 Oct 2021 13:18:28 +0530 Subject: [PATCH 032/628] Changed SIDB readme --- PREREQUISITES.md | 6 +++--- README.md | 8 ++++---- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/PREREQUISITES.md b/PREREQUISITES.md index 0dd557ae..7fc3cf21 100644 --- a/PREREQUISITES.md +++ b/PREREQUISITES.md @@ -22,12 +22,12 @@ Note: You must provision persistent storage if you intend to deploy containerize ### Prerequites for Oracle Autonomous Database (ADB) -If you intent to use `OraOperator` to handle Oracle Autonomous Database lifecycles, then read [Oracle Autonomous Database prerequisites](./doc/adb/ADB_PREREQUISITES.md) +If you intent to use `OraOperator` to handle Oracle Autonomous Database lifecycles, then read [Oracle Autonomous Database prerequisites](./docs/adb/ADB_PREREQUISITES.md) ### Prerequites for Single Instance Databases (SIDB) -If you intent to use `OraOperator` to handle Oracle Database Single Instance lifecycles, then read [Single Instance Database Prerequisites](./doc/sidb/SIDB_PREREQUISITES.md) +If you intent to use `OraOperator` to handle Oracle Database Single Instance lifecycles, then read [Single Instance Database Prerequisites](./docs/sidb/SIDB_PREREQUISITES.md) ### Prerequites for Sharded Databases (SHARDING) - If you intent to use OraOperator to handle the lifecycle of Oracle Database deployed with Oracle Sharding, then read [Sharded Database Prerequisites](./doc/sharding/README.md#prerequsites-for-running-oracle-sharding-database-controller) + If you intent to use OraOperator to handle the lifecycle of Oracle Database deployed with Oracle Sharding, then read [Sharded Database Prerequisites](./docs/sharding/README.md#prerequsites-for-running-oracle-sharding-database-controller) diff --git a/README.md b/README.md index 76d97495..cd72a209 100644 --- a/README.md +++ b/README.md @@ -75,15 +75,15 @@ Oracle strongly recommends that you ensure your system meets the following [Prer You should see that the operator is up and running, along with the shipped controllers. -For more details, see [Oracle Database Operator Installation Instrunctions](./doc/installation/OPERATOR_INSTALLATION_README.md). +For more details, see [Oracle Database Operator Installation Instrunctions](./docs/installation/OPERATOR_INSTALLATION_README.md). ## Getting Started with the Operator (Quickstart) The quickstarts are designed for specific database configurations, including: -* [Oracle Autonomous Database](./doc/adb/README.md) -* [Oracle Database Single Instance configuration](./doc/sidb/README.md) -* [Oracle Database configured with Oracle Sharding](./doc/sharding/README.md) +* [Oracle Autonomous Database](./docs/adb/README.md) +* [Oracle Database Single Instance configuration](./docs/sidb/README.md) +* [Oracle Database configured with Oracle Sharding](./docs/sharding/README.md) YAML file templates are available under [`/config/samples`](./config/samples/). You can copy and edit these template files to configure them for your use cases. From 19fa5127ca05c1196642f2f583d8a0343c265ce3 Mon Sep 17 00:00:00 2001 From: mmahipal Date: Wed, 27 Oct 2021 13:18:28 +0530 Subject: [PATCH 033/628] Changed SIDB readme --- PREREQUISITES.md | 6 +++--- README.md | 8 ++++---- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/PREREQUISITES.md b/PREREQUISITES.md index 0dd557ae..7fc3cf21 100644 --- a/PREREQUISITES.md +++ b/PREREQUISITES.md @@ -22,12 +22,12 @@ Note: You must provision persistent storage if you intend to deploy containerize ### Prerequites for Oracle Autonomous Database (ADB) -If you intent to use `OraOperator` to handle Oracle Autonomous Database lifecycles, then read [Oracle Autonomous Database prerequisites](./doc/adb/ADB_PREREQUISITES.md) +If you intent to use `OraOperator` to handle Oracle Autonomous Database lifecycles, then read [Oracle Autonomous Database prerequisites](./docs/adb/ADB_PREREQUISITES.md) ### Prerequites for Single Instance Databases (SIDB) -If you intent to use `OraOperator` to handle Oracle Database Single Instance lifecycles, then read [Single Instance Database Prerequisites](./doc/sidb/SIDB_PREREQUISITES.md) +If you intent to use `OraOperator` to handle Oracle Database Single Instance lifecycles, then read [Single Instance Database Prerequisites](./docs/sidb/SIDB_PREREQUISITES.md) ### Prerequites for Sharded Databases (SHARDING) - If you intent to use OraOperator to handle the lifecycle of Oracle Database deployed with Oracle Sharding, then read [Sharded Database Prerequisites](./doc/sharding/README.md#prerequsites-for-running-oracle-sharding-database-controller) + If you intent to use OraOperator to handle the lifecycle of Oracle Database deployed with Oracle Sharding, then read [Sharded Database Prerequisites](./docs/sharding/README.md#prerequsites-for-running-oracle-sharding-database-controller) diff --git a/README.md b/README.md index 76d97495..cd72a209 100644 --- a/README.md +++ b/README.md @@ -75,15 +75,15 @@ Oracle strongly recommends that you ensure your system meets the following [Prer You should see that the operator is up and running, along with the shipped controllers. -For more details, see [Oracle Database Operator Installation Instrunctions](./doc/installation/OPERATOR_INSTALLATION_README.md). +For more details, see [Oracle Database Operator Installation Instrunctions](./docs/installation/OPERATOR_INSTALLATION_README.md). ## Getting Started with the Operator (Quickstart) The quickstarts are designed for specific database configurations, including: -* [Oracle Autonomous Database](./doc/adb/README.md) -* [Oracle Database Single Instance configuration](./doc/sidb/README.md) -* [Oracle Database configured with Oracle Sharding](./doc/sharding/README.md) +* [Oracle Autonomous Database](./docs/adb/README.md) +* [Oracle Database Single Instance configuration](./docs/sidb/README.md) +* [Oracle Database configured with Oracle Sharding](./docs/sharding/README.md) YAML file templates are available under [`/config/samples`](./config/samples/). You can copy and edit these template files to configure them for your use cases. From 36d3cbfa30879d4c63d0c712458f7f268e7881ae Mon Sep 17 00:00:00 2001 From: mmahipal Date: Thu, 28 Oct 2021 20:11:57 +0530 Subject: [PATCH 034/628] Added links to create pv, secret --- docs/sidb/README.md | 2 ++ docs/sidb/SIDB_PREREQUISITES.md | 1 + 2 files changed, 3 insertions(+) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 4046f98a..168a2395 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -24,6 +24,8 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI **Note:** The `adminPassword` field of the above `singleinstancedatabase.yaml` yaml contains a secret for Single Instance Database creation (Provisioning a new database or cloning an existing database). This secret gets deleted after the database pod becomes ready for security reasons. + More info on creating Kubernetes Secret available at [https://kubernetes.io/docs/tasks/configmap-secret/managing-secret-using-kubectl/](https://kubernetes.io/docs/tasks/configmap-secret/managing-secret-using-kubectl/) + * ### List Databases ```sh diff --git a/docs/sidb/SIDB_PREREQUISITES.md b/docs/sidb/SIDB_PREREQUISITES.md index 43da0127..2c13b8e2 100644 --- a/docs/sidb/SIDB_PREREQUISITES.md +++ b/docs/sidb/SIDB_PREREQUISITES.md @@ -12,3 +12,4 @@ To deploy Oracle Database Operator for Kubernetes on Oracle Docker images, compl Set up an on-premises Kubernetes cluster, or subscribe to a managed Kubernetes service, such as Oracle Cloud Infrastructure Container Engine for Kubernetes, configured with persistent volumes. The persistent volumes are required for storage of the database files. + More info on creating persistent volumes available at [https://kubernetes.io/docs/concepts/storage/persistent-volumes/](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) From 0897c076cd4e6101a5270b27caf3d4eefae2971d Mon Sep 17 00:00:00 2001 From: mmahipal Date: Thu, 28 Oct 2021 20:11:57 +0530 Subject: [PATCH 035/628] Added links to create pv, secret --- docs/sidb/README.md | 2 ++ docs/sidb/SIDB_PREREQUISITES.md | 1 + 2 files changed, 3 insertions(+) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 4046f98a..168a2395 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -24,6 +24,8 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI **Note:** The `adminPassword` field of the above `singleinstancedatabase.yaml` yaml contains a secret for Single Instance Database creation (Provisioning a new database or cloning an existing database). This secret gets deleted after the database pod becomes ready for security reasons. + More info on creating Kubernetes Secret available at [https://kubernetes.io/docs/tasks/configmap-secret/managing-secret-using-kubectl/](https://kubernetes.io/docs/tasks/configmap-secret/managing-secret-using-kubectl/) + * ### List Databases ```sh diff --git a/docs/sidb/SIDB_PREREQUISITES.md b/docs/sidb/SIDB_PREREQUISITES.md index 43da0127..2c13b8e2 100644 --- a/docs/sidb/SIDB_PREREQUISITES.md +++ b/docs/sidb/SIDB_PREREQUISITES.md @@ -12,3 +12,4 @@ To deploy Oracle Database Operator for Kubernetes on Oracle Docker images, compl Set up an on-premises Kubernetes cluster, or subscribe to a managed Kubernetes service, such as Oracle Cloud Infrastructure Container Engine for Kubernetes, configured with persistent volumes. The persistent volumes are required for storage of the database files. + More info on creating persistent volumes available at [https://kubernetes.io/docs/concepts/storage/persistent-volumes/](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) From 630935d2328eb90129ef0e347541105b9b321799 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Sat, 6 Nov 2021 16:10:14 -0400 Subject: [PATCH 036/628] Add debugging and required policies --- docs/adb/README.md | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/docs/adb/README.md b/docs/adb/README.md index 58e4730c..6faaf2a0 100644 --- a/docs/adb/README.md +++ b/docs/adb/README.md @@ -2,6 +2,12 @@ Before you use the Oracle Database Operator for Kubernetes (the operator), ensure your system meets all of the Oracle Autonomous Database (ADB) Prerequisites [ADB_PREREQUISITES](./ADB_PREREQUISITES.md). +## Required Permissions + +The opeartor must be given the required type of access in a policy written by an administrator to manage the Autonomous Databases. See [Let database and fleet admins manage Autonomous Databases](https://docs.oracle.com/en-us/iaas/Content/Identity/Concepts/commonpolicies.htm#db-admins-manage-adb) for sample Autonomous Database policies. + +The permission to view the workrequests is also required, so that the operator will update the resources when the work is done. See [Viewing Work Requests](https://docs.oracle.com/en-us/iaas/Content/ContEng/Tasks/contengviewingworkrequests.htm#contengviewingworkrequests) for sample work request policies. + ## Supported Features After the operator is deployed, choose either one of the following operations to create an `AutonomousDatabase` custom resource for Oracle Autonomous Database in your cluster. @@ -18,6 +24,8 @@ After you create the resource, you can use the operator to perform the following * [Stop/Start/Terminate](#stopstartterminate) an Autonomous Database * [Delete the resource](#delete-the-resource) from the cluster +To debug the Oracle Autonomous Databases with Oracle Database Operator, see [Debugging and troubleshooting](#debugging-and-troubleshooting) + ## Provision an Autonomous Database Follow the steps to provision an Autonomous Database that will bind objects in your cluster. @@ -373,3 +381,30 @@ Follow the steps to delete the resource and terminate the Autonomous Database. ``` Now, you can verify that the database is in TERMINATING state on the Cloud Console. + + +## Debugging and troubleshooting + +### Show the details of the resource + +If you edit and re-apply the `.yaml` file, the Autonomous Database controller will only update the parameters that the file contains. The parameters which are not in the file will not be impacted. To get the verbose output of the current spec, use below command: + +```sh +kubectl describe adb/autonomousdatabase-sample +``` + +### The resource is in UNAVAILABLE state + +If an error occurs during the operation, the `lifecycleState` of the resoursce changes to UNAVAILABLE. Follow the steps to check the logs. + +1. List the pod replicas + + ```sh + kubectl get pods -n oracle-database-operator-system + ``` + +2. Use the below command to check the logs of the Pod which has a failure + + ```sh + kubectl logs -f pod/oracle-database-operator-controller-manager-78666fdddb-s4xcm -n oracle-database-operator-system + ``` From 2ea8b5c14cf5f6468147d4bc70be0535c8ae102f Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Sat, 6 Nov 2021 16:10:14 -0400 Subject: [PATCH 037/628] Add debugging and required policies --- docs/adb/README.md | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/docs/adb/README.md b/docs/adb/README.md index 58e4730c..6faaf2a0 100644 --- a/docs/adb/README.md +++ b/docs/adb/README.md @@ -2,6 +2,12 @@ Before you use the Oracle Database Operator for Kubernetes (the operator), ensure your system meets all of the Oracle Autonomous Database (ADB) Prerequisites [ADB_PREREQUISITES](./ADB_PREREQUISITES.md). +## Required Permissions + +The opeartor must be given the required type of access in a policy written by an administrator to manage the Autonomous Databases. See [Let database and fleet admins manage Autonomous Databases](https://docs.oracle.com/en-us/iaas/Content/Identity/Concepts/commonpolicies.htm#db-admins-manage-adb) for sample Autonomous Database policies. + +The permission to view the workrequests is also required, so that the operator will update the resources when the work is done. See [Viewing Work Requests](https://docs.oracle.com/en-us/iaas/Content/ContEng/Tasks/contengviewingworkrequests.htm#contengviewingworkrequests) for sample work request policies. + ## Supported Features After the operator is deployed, choose either one of the following operations to create an `AutonomousDatabase` custom resource for Oracle Autonomous Database in your cluster. @@ -18,6 +24,8 @@ After you create the resource, you can use the operator to perform the following * [Stop/Start/Terminate](#stopstartterminate) an Autonomous Database * [Delete the resource](#delete-the-resource) from the cluster +To debug the Oracle Autonomous Databases with Oracle Database Operator, see [Debugging and troubleshooting](#debugging-and-troubleshooting) + ## Provision an Autonomous Database Follow the steps to provision an Autonomous Database that will bind objects in your cluster. @@ -373,3 +381,30 @@ Follow the steps to delete the resource and terminate the Autonomous Database. ``` Now, you can verify that the database is in TERMINATING state on the Cloud Console. + + +## Debugging and troubleshooting + +### Show the details of the resource + +If you edit and re-apply the `.yaml` file, the Autonomous Database controller will only update the parameters that the file contains. The parameters which are not in the file will not be impacted. To get the verbose output of the current spec, use below command: + +```sh +kubectl describe adb/autonomousdatabase-sample +``` + +### The resource is in UNAVAILABLE state + +If an error occurs during the operation, the `lifecycleState` of the resoursce changes to UNAVAILABLE. Follow the steps to check the logs. + +1. List the pod replicas + + ```sh + kubectl get pods -n oracle-database-operator-system + ``` + +2. Use the below command to check the logs of the Pod which has a failure + + ```sh + kubectl logs -f pod/oracle-database-operator-controller-manager-78666fdddb-s4xcm -n oracle-database-operator-system + ``` From 9fa24120d2595de5e3261ff31973803384e063fc Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Sat, 6 Nov 2021 16:18:26 -0400 Subject: [PATCH 038/628] Add AutonomousDatabaseBackup --- PROJECT | 12 +- .../autonomousdatabasebackup_types.go | 122 ++++++++++ .../v1alpha1/zz_generated.deepcopy.go | 90 +++++++ commons/autonomousdatabase/reconciler_util.go | 123 +++++++++- .../reconciler_util.go | 97 ++++++++ commons/oci/database.go | 36 +++ ....oracle.com_autonomousdatabasebackups.yaml | 124 ++++++++++ config/crd/kustomization.yaml | 3 + ...njection_in_autonomousdatabasebackups.yaml | 8 + .../webhook_in_autonomousdatabasebackups.yaml | 17 ++ .../autonomousdatabasebackup_editor_role.yaml | 24 ++ .../autonomousdatabasebackup_viewer_role.yaml | 20 ++ config/rbac/role.yaml | 30 +++ .../adb_backup/autonomousdatabasebackup.yaml | 7 + config/samples/kustomization.yaml | 1 + .../database/autonomousdatabase_controller.go | 16 ++ .../autonomousdatabasebackup_controller.go | 222 ++++++++++++++++++ main.go | 8 + 18 files changed, 958 insertions(+), 2 deletions(-) create mode 100644 apis/database/v1alpha1/autonomousdatabasebackup_types.go create mode 100644 commons/autonomousdatabase_backup/reconciler_util.go create mode 100644 config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml create mode 100644 config/crd/patches/cainjection_in_autonomousdatabasebackups.yaml create mode 100644 config/crd/patches/webhook_in_autonomousdatabasebackups.yaml create mode 100644 config/rbac/autonomousdatabasebackup_editor_role.yaml create mode 100644 config/rbac/autonomousdatabasebackup_viewer_role.yaml create mode 100644 config/samples/adb_backup/autonomousdatabasebackup.yaml create mode 100644 controllers/database/autonomousdatabasebackup_controller.go diff --git a/PROJECT b/PROJECT index 70e34778..7ad79eeb 100644 --- a/PROJECT +++ b/PROJECT @@ -3,7 +3,8 @@ layout: - go.kubebuilder.io/v2 multigroup: true plugins: - go.sdk.operatorframework.io/v2-alpha: {} + manifests.sdk.operatorframework.io/v2: {} + scorecard.sdk.operatorframework.io/v2: {} projectName: oracle-database-operator repo: github.com/oracle/oracle-database-operator resources: @@ -38,4 +39,13 @@ resources: kind: ShardingDatabase path: github.com/oracle/oracle-database-operator/apis/database/v1alpha1 version: v1alpha1 +- api: + crdVersion: v1 + namespaced: true + controller: true + domain: oracle.com + group: database + kind: AutonomousDatabaseBackup + path: github.com/oracle/oracle-database-operator/apis/database/v1alpha1 + version: v1alpha1 version: "3" diff --git a/apis/database/v1alpha1/autonomousdatabasebackup_types.go b/apis/database/v1alpha1/autonomousdatabasebackup_types.go new file mode 100644 index 00000000..227aeff0 --- /dev/null +++ b/apis/database/v1alpha1/autonomousdatabasebackup_types.go @@ -0,0 +1,122 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v1alpha1 + +import ( + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + + "github.com/oracle/oci-go-sdk/v45/database" +) + +// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! +// NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized. + +// AutonomousDatabaseBackupSpec defines the desired state of AutonomousDatabaseBackup +type AutonomousDatabaseBackupSpec struct { + // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster + // Important: Run "make" to regenerate code after modifying this file + DisplayName string `json:"displayName,omitempty"` + AutonomousDatabaseOCID string `json:"autonomousDatabaseOCID,omitempty"` + AutonomousDatabaseBackupOCID string `json:"autonomousDatabaseBackupOCID,omitempty"` + + OCIConfig OCIConfigSpec `json:"ociConfig,omitempty"` +} + +// AutonomousDatabaseBackupStatus defines the observed state of AutonomousDatabaseBackup +type AutonomousDatabaseBackupStatus struct { + // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster + // Important: Run "make" to regenerate code after modifying this file + AutonomousDatabaseBackupOCID string `json:"autonomousDatabaseBackupOCID"` + CompartmentOCID string `json:"compartmentOCID"` + AutonomousDatabaseOCID string `json:"autonomousDatabaseOCID"` + DisplayName string `json:"displayName"` + Type database.AutonomousDatabaseBackupTypeEnum `json:"type"` + IsAutomatic bool `json:"isAutomatic"` + LifecycleState database.AutonomousDatabaseBackupLifecycleStateEnum `json:"lifecycleState"` + TimeStarted string `json:"timeStarted,omitempty"` + TimeEnded string `json:"timeEnded,omitempty"` +} + +//+kubebuilder:object:root=true +//+kubebuilder:subresource:status +// +kubebuilder:resource:shortName="adbbu";"adbbus" +// +kubebuilder:printcolumn:JSONPath=".spec.displayName",name="Display Name",type=string +// +kubebuilder:printcolumn:JSONPath=".status.lifecycleState",name="State",type=string +// +kubebuilder:printcolumn:JSONPath=".status.type",name="Type",type=string +// +kubebuilder:printcolumn:JSONPath=".status.timeStarted",name="Started",type=string +// +kubebuilder:printcolumn:JSONPath=".status.timeEnded",name="Ended",type=string + +// AutonomousDatabaseBackup is the Schema for the autonomousdatabasebackups API +type AutonomousDatabaseBackup struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + Spec AutonomousDatabaseBackupSpec `json:"spec,omitempty"` + Status AutonomousDatabaseBackupStatus `json:"status,omitempty"` +} + +func (backup *AutonomousDatabaseBackup) UpdateStatusFromAutonomousDatabaseBackupResponse(resp database.GetAutonomousDatabaseBackupResponse) { + backup.Status.AutonomousDatabaseBackupOCID = *resp.Id + backup.Status.CompartmentOCID = *resp.CompartmentId + backup.Status.AutonomousDatabaseOCID = *resp.AutonomousDatabaseId + backup.Status.DisplayName = *resp.DisplayName + backup.Status.Type = resp.Type + backup.Status.IsAutomatic = *resp.IsAutomatic + backup.Status.LifecycleState = resp.LifecycleState + + if resp.TimeStarted != nil { + backup.Status.TimeStarted = resp.TimeStarted.String() + } + if resp.TimeEnded != nil { + backup.Status.TimeEnded = resp.TimeEnded.String() + } +} + +//+kubebuilder:object:root=true + +// AutonomousDatabaseBackupList contains a list of AutonomousDatabaseBackup +type AutonomousDatabaseBackupList struct { + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata,omitempty"` + Items []AutonomousDatabaseBackup `json:"items"` +} + +func init() { + SchemeBuilder.Register(&AutonomousDatabaseBackup{}, &AutonomousDatabaseBackupList{}) +} diff --git a/apis/database/v1alpha1/zz_generated.deepcopy.go b/apis/database/v1alpha1/zz_generated.deepcopy.go index 2766d559..78806d2f 100644 --- a/apis/database/v1alpha1/zz_generated.deepcopy.go +++ b/apis/database/v1alpha1/zz_generated.deepcopy.go @@ -75,6 +75,96 @@ func (in *AutonomousDatabase) DeepCopyObject() runtime.Object { return nil } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousDatabaseBackup) DeepCopyInto(out *AutonomousDatabaseBackup) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + in.Spec.DeepCopyInto(&out.Spec) + out.Status = in.Status +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseBackup. +func (in *AutonomousDatabaseBackup) DeepCopy() *AutonomousDatabaseBackup { + if in == nil { + return nil + } + out := new(AutonomousDatabaseBackup) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *AutonomousDatabaseBackup) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousDatabaseBackupList) DeepCopyInto(out *AutonomousDatabaseBackupList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]AutonomousDatabaseBackup, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseBackupList. +func (in *AutonomousDatabaseBackupList) DeepCopy() *AutonomousDatabaseBackupList { + if in == nil { + return nil + } + out := new(AutonomousDatabaseBackupList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *AutonomousDatabaseBackupList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousDatabaseBackupSpec) DeepCopyInto(out *AutonomousDatabaseBackupSpec) { + *out = *in + in.OCIConfig.DeepCopyInto(&out.OCIConfig) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseBackupSpec. +func (in *AutonomousDatabaseBackupSpec) DeepCopy() *AutonomousDatabaseBackupSpec { + if in == nil { + return nil + } + out := new(AutonomousDatabaseBackupSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousDatabaseBackupStatus) DeepCopyInto(out *AutonomousDatabaseBackupStatus) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseBackupStatus. +func (in *AutonomousDatabaseBackupStatus) DeepCopy() *AutonomousDatabaseBackupStatus { + if in == nil { + return nil + } + out := new(AutonomousDatabaseBackupStatus) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *AutonomousDatabaseDetails) DeepCopyInto(out *AutonomousDatabaseDetails) { *out = *in diff --git a/commons/autonomousdatabase/reconciler_util.go b/commons/autonomousdatabase/reconciler_util.go index 2ceac5e6..094641d3 100644 --- a/commons/autonomousdatabase/reconciler_util.go +++ b/commons/autonomousdatabase/reconciler_util.go @@ -41,8 +41,11 @@ package autonomousdatabase import ( "context" "fmt" + "regexp" + "strings" corev1 "k8s.io/api/core/v1" + apiErrors "k8s.io/apimachinery/pkg/api/errors" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/types" "k8s.io/client-go/util/retry" @@ -57,7 +60,7 @@ import ( "github.com/oracle/oracle-database-operator/commons/oci" ) -// SetStatus sets the status subresource. +// SetStatus sets the status subresource of AutonomousDatabase func SetStatus(kubeClient client.Client, adb *dbv1alpha1.AutonomousDatabase) error { return retry.RetryOnConflict(retry.DefaultRetry, func() error { curADB := &dbv1alpha1.AutonomousDatabase{} @@ -125,3 +128,121 @@ func CreateWalletSecret(logger logr.Logger, kubeClient client.Client, dbClient d logger.Info(fmt.Sprintf("Wallet is stored in the Secret %s", *walletName)) return nil } + +func getValidName(name string, usedNames map[string]bool) string { + returnedName := name + var i = 1 + + _, ok := usedNames[returnedName] + for ok { + returnedName = fmt.Sprintf("%s-%d", name, i) + _, ok = usedNames[returnedName] + i++ + } + + return returnedName +} + +func getOwnerReference(adb *dbv1alpha1.AutonomousDatabase) []metav1.OwnerReference { + ownerRef := []metav1.OwnerReference{ + { + Kind: adb.GroupVersionKind().Kind, + APIVersion: adb.APIVersion, + Name: adb.Name, + UID: types.UID(adb.UID), + }, + } + return ownerRef +} + +// CreateBackupResources creates the all the AutonomousDatabasBackups that appears in the ListAutonomousDatabaseBackups request +// The backup object will not be created if it already exists. +func CreateBackupResources(logger logr.Logger, kubeClient client.Client, dbClient database.DatabaseClient, adb *dbv1alpha1.AutonomousDatabase) error { + // Get the list of AutonomousDatabaseBackupOCID in the same namespace + backupList := &dbv1alpha1.AutonomousDatabaseBackupList{} + + if err := kubeClient.List(context.TODO(), backupList, &client.ListOptions{Namespace: adb.Namespace}); err != nil { + // Ignore not-found errors, since they can't be fixed by an immediate requeue. + // No need to change the since we don't know if we obtain the object. + if !apiErrors.IsNotFound(err) { + return err + } + } + + usedNames := make(map[string]bool) + usedBackupOCIDs := make(map[string]bool) + + for _, backup := range backupList.Items { + usedNames[backup.Name] = true + + // Add both Spec.AutonomousDatabaseBackupOCID and Status.AutonomousDatabaseBackupOCID. + // If it's a backup created from the operator, it won't have the OCID under the spec. + // if the backup isn't ready yet, it won't have the OCID under the status. + if backup.Spec.AutonomousDatabaseBackupOCID != "" { + usedBackupOCIDs[backup.Spec.AutonomousDatabaseBackupOCID] = true + } + if backup.Status.AutonomousDatabaseBackupOCID != "" { + usedBackupOCIDs[backup.Status.AutonomousDatabaseBackupOCID] = true + } + } + + resp, err := oci.ListAutonomousDatabaseBackups(dbClient, adb) + if err != nil { + return err + } + + var ownerRef []metav1.OwnerReference + ownerRef = append(ownerRef, metav1.OwnerReference{Kind: adb.GroupVersionKind().Kind, APIVersion: adb.APIVersion, Name: adb.Name, UID: types.UID(adb.UID)}) + + for _, backupSummary := range resp.Items { + // Create the resource if the AutonomousDatabaseBackupOCID doesn't exist + _, ok := usedBackupOCIDs[*backupSummary.Id] + if !ok { + // Convert the string to lowercase, and replace spaces, commas, and colons with hyphens + backupName := *backupSummary.DisplayName + backupName = strings.ToLower(backupName) + + re, err := regexp.Compile(`[^-a-zA-Z0-9]`) + if err != nil { + return err + } + backupName = re.ReplaceAllString(backupName, "-") + backupName = getValidName(backupName, usedNames) + + backup := &dbv1alpha1.AutonomousDatabaseBackup{ + ObjectMeta: metav1.ObjectMeta{ + Namespace: adb.GetNamespace(), + Name: backupName, + OwnerReferences: getOwnerReference(adb), + }, + Spec: dbv1alpha1.AutonomousDatabaseBackupSpec{ + AutonomousDatabaseBackupOCID: *backupSummary.Id, + OCIConfig: dbv1alpha1.OCIConfigSpec{ + ConfigMapName: adb.Spec.OCIConfig.ConfigMapName, + SecretName: adb.Spec.OCIConfig.SecretName, + }, + }, + } + + // fields with mandatory:"false" could be nil + if backupSummary.TimeStarted != nil { + backup.Status.TimeStarted = backupSummary.TimeStarted.String() + } + if backupSummary.TimeEnded != nil { + backup.Status.TimeEnded = backupSummary.TimeEnded.String() + } + + if err := kubeClient.Create(context.TODO(), backup); err != nil { + return err + } + + // Add the used names and ocids + usedNames[backupName] = true + usedBackupOCIDs[*backupSummary.AutonomousDatabaseId] = true + + logger.Info("Create AutonomousDatabaseBackup " + backupName) + } + } + + return nil +} diff --git a/commons/autonomousdatabase_backup/reconciler_util.go b/commons/autonomousdatabase_backup/reconciler_util.go new file mode 100644 index 00000000..c95a2608 --- /dev/null +++ b/commons/autonomousdatabase_backup/reconciler_util.go @@ -0,0 +1,97 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package autonomousdatabase_backup + +import ( + "context" + + "k8s.io/apimachinery/pkg/types" + "k8s.io/client-go/util/retry" + "sigs.k8s.io/controller-runtime/pkg/client" + + "github.com/go-logr/logr" + dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" +) + +func UpdateAutonomousDatabaseBackupResource(logger logr.Logger, kubeClient client.Client, backup *dbv1alpha1.AutonomousDatabaseBackup) error { + if err := retry.RetryOnConflict(retry.DefaultRetry, func() error { + curBackup := &dbv1alpha1.AutonomousDatabaseBackup{} + + namespacedName := types.NamespacedName{ + Namespace: backup.GetNamespace(), + Name: backup.GetName(), + } + + if err := kubeClient.Get(context.TODO(), namespacedName, curBackup); err != nil { + return err + } + + curBackup.Spec = backup.Spec + return kubeClient.Update(context.TODO(), curBackup) + }); err != nil { + return err + } + + // Update status + if statusErr := UpdateStatus(kubeClient, backup); statusErr != nil { + return statusErr + } + logger.Info("Update local resource AutonomousDatabase successfully") + + return nil +} + +// UpdateStatus sets the status subresource of AutonomousDatabaseBackup +func UpdateStatus(kubeClient client.Client, adbBackup *dbv1alpha1.AutonomousDatabaseBackup) error { + return retry.RetryOnConflict(retry.DefaultRetry, func() error { + curBackup := &dbv1alpha1.AutonomousDatabaseBackup{} + + namespacedName := types.NamespacedName{ + Namespace: adbBackup.GetNamespace(), + Name: adbBackup.GetName(), + } + + if err := kubeClient.Get(context.TODO(), namespacedName, curBackup); err != nil { + return err + } + + curBackup.Status = adbBackup.Status + return kubeClient.Status().Update(context.TODO(), curBackup) + }) +} diff --git a/commons/oci/database.go b/commons/oci/database.go index 985d299b..e29afc78 100644 --- a/commons/oci/database.go +++ b/commons/oci/database.go @@ -457,6 +457,42 @@ func DeleteAutonomousDatabase(dbClient database.DatabaseClient, adbOCID string) return } +// ListAutonomousDatabaseBackups returns a list of Autonomous Database backups +func ListAutonomousDatabaseBackups(dbClient database.DatabaseClient, adb *dbv1alpha1.AutonomousDatabase) (resp database.ListAutonomousDatabaseBackupsResponse, err error) { + if adb.Spec.Details.AutonomousDatabaseOCID == nil { + return resp, nil + } + + listBackupRequest := database.ListAutonomousDatabaseBackupsRequest{ + AutonomousDatabaseId: adb.Spec.Details.AutonomousDatabaseOCID, + } + + return dbClient.ListAutonomousDatabaseBackups(context.TODO(), listBackupRequest) +} + +// CreateAutonomousDatabaseBackup creates an backup of Autonomous Database +func CreateAutonomousDatabaseBackup(logger logr.Logger, dbClient database.DatabaseClient, adbBackup *dbv1alpha1.AutonomousDatabaseBackup) (resp database.CreateAutonomousDatabaseBackupResponse, err error) { + logger.Info("Creating Autonomous Database backup " + adbBackup.Spec.DisplayName) + + createBackupRequest := database.CreateAutonomousDatabaseBackupRequest{ + CreateAutonomousDatabaseBackupDetails: database.CreateAutonomousDatabaseBackupDetails{ + DisplayName: &adbBackup.Spec.DisplayName, + AutonomousDatabaseId: &adbBackup.Spec.AutonomousDatabaseOCID, + }, + } + + return dbClient.CreateAutonomousDatabaseBackup(context.TODO(), createBackupRequest) +} + +// GetAutonomousDatabaseBackup returns the response of GetAutonomousDatabaseBackupRequest +func GetAutonomousDatabaseBackup(dbClient database.DatabaseClient, backupOCID *string) (resp database.GetAutonomousDatabaseBackupResponse, err error) { + getBackupRequest := database.GetAutonomousDatabaseBackupRequest{ + AutonomousDatabaseBackupId: backupOCID, + } + + return dbClient.GetAutonomousDatabaseBackup(context.TODO(), getBackupRequest) +} + func WaitUntilWorkCompleted(logger logr.Logger, workClient workrequests.WorkRequestClient, opcWorkRequestID *string) error { if opcWorkRequestID == nil { return nil diff --git a/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml b/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml new file mode 100644 index 00000000..d47fd202 --- /dev/null +++ b/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml @@ -0,0 +1,124 @@ + +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.6.1 + creationTimestamp: null + name: autonomousdatabasebackups.database.oracle.com +spec: + group: database.oracle.com + names: + kind: AutonomousDatabaseBackup + listKind: AutonomousDatabaseBackupList + plural: autonomousdatabasebackups + shortNames: + - adbbu + - adbbus + singular: autonomousdatabasebackup + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .spec.displayName + name: Display Name + type: string + - jsonPath: .status.lifecycleState + name: State + type: string + - jsonPath: .status.type + name: Type + type: string + - jsonPath: .status.timeStarted + name: Started + type: string + - jsonPath: .status.timeEnded + name: Ended + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + description: AutonomousDatabaseBackup is the Schema for the autonomousdatabasebackups + API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: AutonomousDatabaseBackupSpec defines the desired state of + AutonomousDatabaseBackup + properties: + autonomousDatabaseBackupOCID: + type: string + autonomousDatabaseOCID: + type: string + displayName: + description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster + Important: Run "make" to regenerate code after modifying this file' + type: string + ociConfig: + properties: + configMapName: + type: string + secretName: + type: string + type: object + type: object + status: + description: AutonomousDatabaseBackupStatus defines the observed state + of AutonomousDatabaseBackup + properties: + autonomousDatabaseBackupOCID: + description: 'INSERT ADDITIONAL STATUS FIELD - define observed state + of cluster Important: Run "make" to regenerate code after modifying + this file' + type: string + autonomousDatabaseOCID: + type: string + compartmentOCID: + type: string + displayName: + type: string + isAutomatic: + type: boolean + lifecycleState: + description: 'AutonomousDatabaseBackupLifecycleStateEnum Enum with + underlying type: string' + type: string + timeEnded: + type: string + timeStarted: + type: string + type: + description: 'AutonomousDatabaseBackupTypeEnum Enum with underlying + type: string' + type: string + required: + - autonomousDatabaseBackupOCID + - autonomousDatabaseOCID + - compartmentOCID + - displayName + - isAutomatic + - lifecycleState + - type + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] diff --git a/config/crd/kustomization.yaml b/config/crd/kustomization.yaml index 6b3d488e..c6d82e86 100644 --- a/config/crd/kustomization.yaml +++ b/config/crd/kustomization.yaml @@ -9,6 +9,7 @@ resources: - bases/database.oracle.com_autonomousdatabases.yaml - bases/database.oracle.com_singleinstancedatabases.yaml - bases/database.oracle.com_shardingdatabases.yaml +- bases/database.oracle.com_autonomousdatabasebackups.yaml # +kubebuilder:scaffold:crdkustomizeresource patchesStrategicMerge: @@ -18,6 +19,7 @@ patchesStrategicMerge: #- patches/webhook_in_autonomousdatabases.yaml #- patches/webhook_in_singleinstancedatabases.yaml #- patches/webhook_in_shardingdatabases.yaml +#- patches/webhook_in_autonomousdatabasebackups.yaml # +kubebuilder:scaffold:crdkustomizewebhookpatch # [CERTMANAGER] To enable webhook, uncomment all the sections with [CERTMANAGER] prefix. @@ -26,6 +28,7 @@ patchesStrategicMerge: #- patches/cainjection_in_autonomousdatabases.yaml - patches/cainjection_in_singleinstancedatabases.yaml #- patches/cainjection_in_shardingdatabases.yaml +#- patches/cainjection_in_autonomousdatabasebackups.yaml # +kubebuilder:scaffold:crdkustomizecainjectionpatch # the following config is for teaching kustomize how to do kustomization for CRDs. diff --git a/config/crd/patches/cainjection_in_autonomousdatabasebackups.yaml b/config/crd/patches/cainjection_in_autonomousdatabasebackups.yaml new file mode 100644 index 00000000..78280137 --- /dev/null +++ b/config/crd/patches/cainjection_in_autonomousdatabasebackups.yaml @@ -0,0 +1,8 @@ +# The following patch adds a directive for certmanager to inject CA into the CRD +# CRD conversion requires k8s 1.13 or later. +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + annotations: + cert-manager.io/inject-ca-from: $(CERTIFICATE_NAMESPACE)/$(CERTIFICATE_NAME) + name: autonomousdatabasebackups.database.oracle.com diff --git a/config/crd/patches/webhook_in_autonomousdatabasebackups.yaml b/config/crd/patches/webhook_in_autonomousdatabasebackups.yaml new file mode 100644 index 00000000..1a4eacb6 --- /dev/null +++ b/config/crd/patches/webhook_in_autonomousdatabasebackups.yaml @@ -0,0 +1,17 @@ +# The following patch enables conversion webhook for CRD +# CRD conversion requires k8s 1.13 or later. +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: autonomousdatabasebackups.database.oracle.com +spec: + conversion: + strategy: Webhook + webhookClientConfig: + # this is "\n" used as a placeholder, otherwise it will be rejected by the apiserver for being blank, + # but we're going to set it later using the cert-manager (or potentially a patch if not using cert-manager) + caBundle: Cg== + service: + namespace: system + name: webhook-service + path: /convert diff --git a/config/rbac/autonomousdatabasebackup_editor_role.yaml b/config/rbac/autonomousdatabasebackup_editor_role.yaml new file mode 100644 index 00000000..1d210196 --- /dev/null +++ b/config/rbac/autonomousdatabasebackup_editor_role.yaml @@ -0,0 +1,24 @@ +# permissions for end users to edit autonomousdatabasebackups. +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: autonomousdatabasebackup-editor-role +rules: +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabasebackups + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabasebackups/status + verbs: + - get diff --git a/config/rbac/autonomousdatabasebackup_viewer_role.yaml b/config/rbac/autonomousdatabasebackup_viewer_role.yaml new file mode 100644 index 00000000..3be0c5cb --- /dev/null +++ b/config/rbac/autonomousdatabasebackup_viewer_role.yaml @@ -0,0 +1,20 @@ +# permissions for end users to view autonomousdatabasebackups. +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: autonomousdatabasebackup-viewer-role +rules: +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabasebackups + verbs: + - get + - list + - watch +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabasebackups/status + verbs: + - get diff --git a/config/rbac/role.yaml b/config/rbac/role.yaml index 6d763212..bef8ac3b 100644 --- a/config/rbac/role.yaml +++ b/config/rbac/role.yaml @@ -97,6 +97,36 @@ rules: - pods/exec verbs: - create +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabaseBackups + verbs: + - create + - delete + - get + - list + - update +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabasebackups + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabasebackups/status + verbs: + - get + - patch + - update - apiGroups: - database.oracle.com resources: diff --git a/config/samples/adb_backup/autonomousdatabasebackup.yaml b/config/samples/adb_backup/autonomousdatabasebackup.yaml new file mode 100644 index 00000000..93da9f77 --- /dev/null +++ b/config/samples/adb_backup/autonomousdatabasebackup.yaml @@ -0,0 +1,7 @@ +apiVersion: database.oracle.com/v1alpha1 +kind: AutonomousDatabaseBackup +metadata: + name: autonomousdatabasebackup-sample +spec: + # Add fields here + foo: bar diff --git a/config/samples/kustomization.yaml b/config/samples/kustomization.yaml index 4b419923..1a6e104a 100644 --- a/config/samples/kustomization.yaml +++ b/config/samples/kustomization.yaml @@ -9,4 +9,5 @@ resources: - autonomousdatabase.yaml - singleinstancedatabase.yaml - shardingdatabase.yaml + - autonomousdatabasebackup.yaml # +kubebuilder:scaffold:manifestskustomizesamples diff --git a/controllers/database/autonomousdatabase_controller.go b/controllers/database/autonomousdatabase_controller.go index 2f250489..00d281a4 100644 --- a/controllers/database/autonomousdatabase_controller.go +++ b/controllers/database/autonomousdatabase_controller.go @@ -112,6 +112,7 @@ func (r *AutonomousDatabaseReconciler) eventFilterPredicate() predicate.Predicat // +kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabases,verbs=get;list;watch;create;update;patch;delete // +kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabases/status,verbs=update;patch +// +kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabaseBackups,verbs=get;list;create;update;delete // +kubebuilder:rbac:groups=coordination.k8s.io,resources=leases,verbs=create;get;list;update // +kubebuilder:rbac:groups="",resources=configmaps;secrets,verbs=get;list;watch;create;update;patch;delete @@ -486,6 +487,21 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R } } + /***************************************************** + * AutonomousDatabase Backups + *****************************************************/ + if err := adbutil.CreateBackupResources(r.currentLogger, r.KubeClient, dbClient, adb); err != nil { + r.currentLogger.Error(err, "Fail to update Autonomous Database") + + // Change the status to UNAVAILABLE + adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + return ctrl.Result{}, statusErr + } + // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue + return ctrl.Result{}, nil + } + /***************************************************** * Update last succesful spec *****************************************************/ diff --git a/controllers/database/autonomousdatabasebackup_controller.go b/controllers/database/autonomousdatabasebackup_controller.go new file mode 100644 index 00000000..0cf425c6 --- /dev/null +++ b/controllers/database/autonomousdatabasebackup_controller.go @@ -0,0 +1,222 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package controllers + +import ( + "context" + "reflect" + + "github.com/go-logr/logr" + apiErrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/runtime" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/controller" + "sigs.k8s.io/controller-runtime/pkg/event" + "sigs.k8s.io/controller-runtime/pkg/predicate" + + "github.com/oracle/oci-go-sdk/v45/database" + "github.com/oracle/oci-go-sdk/v45/workrequests" + databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + backupUtil "github.com/oracle/oracle-database-operator/commons/autonomousdatabase_backup" + "github.com/oracle/oracle-database-operator/commons/oci" +) + +// AutonomousDatabaseBackupReconciler reconciles a AutonomousDatabaseBackup object +type AutonomousDatabaseBackupReconciler struct { + KubeClient client.Client + Log logr.Logger + Scheme *runtime.Scheme + + currentLogger logr.Logger +} + +// SetupWithManager sets up the controller with the Manager. +func (r *AutonomousDatabaseBackupReconciler) SetupWithManager(mgr ctrl.Manager) error { + return ctrl.NewControllerManagedBy(mgr). + For(&databasev1alpha1.AutonomousDatabaseBackup{}). + WithEventFilter(r.eventFilterPredicate()). + WithOptions(controller.Options{MaxConcurrentReconciles: 50}). // ReconcileHandler is never invoked concurrently with the same object. + Complete(r) +} + +func (r *AutonomousDatabaseBackupReconciler) eventFilterPredicate() predicate.Predicate { + pred := predicate.Funcs{ + UpdateFunc: func(e event.UpdateEvent) bool { + oldBackup := e.ObjectOld.DeepCopyObject().(*dbv1alpha1.AutonomousDatabaseBackup) + newBackup := e.ObjectNew.DeepCopyObject().(*dbv1alpha1.AutonomousDatabaseBackup) + + specChanged := reflect.DeepEqual(oldBackup.Spec, newBackup.Spec) + statusChanged := reflect.DeepEqual(oldBackup.Status, newBackup.Status) + if specChanged || statusChanged { + // Don't enqueue request + return false + } + // Enqueue request + return true + }, + DeleteFunc: func(e event.DeleteEvent) bool { + // Do not trigger reconciliation when the real object is deleted from the cluster. + return false + }, + } + + return pred +} + +//+kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabasebackups,verbs=get;list;watch;create;update;patch;delete +//+kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabasebackups/status,verbs=get;update;patch + +func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + r.currentLogger = r.Log.WithValues("autonomousdatabase_backup", req.NamespacedName) + + adbBackup := &dbv1alpha1.AutonomousDatabaseBackup{} + if err := r.KubeClient.Get(context.TODO(), req.NamespacedName, adbBackup); err != nil { + // Ignore not-found errors, since they can't be fixed by an immediate requeue. + // No need to change the since we don't know if we obtain the object. + if !apiErrors.IsNotFound(err) { + return ctrl.Result{}, err + } + } + + /****************************************************************** + * Get OCI database client and work request client + ******************************************************************/ + authData := oci.APIKeyAuth{ + ConfigMapName: adbBackup.Spec.OCIConfig.ConfigMapName, + SecretName: adbBackup.Spec.OCIConfig.SecretName, + Namespace: adbBackup.GetNamespace(), + } + provider, err := oci.GetOCIProvider(r.KubeClient, authData) + if err != nil { + r.currentLogger.Error(err, "Fail to get OCI provider") + + // Change the status to UNAVAILABLE + adbBackup.Status.LifecycleState = database.AutonomousDatabaseBackupLifecycleStateFailed + if statusErr := backupUtil.UpdateStatus(r.KubeClient, adbBackup); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, nil + } + + dbClient, err := database.NewDatabaseClientWithConfigurationProvider(provider) + if err != nil { + r.currentLogger.Error(err, "Fail to get OCI database client") + + // Change the status to UNAVAILABLE + adbBackup.Status.LifecycleState = database.AutonomousDatabaseBackupLifecycleStateFailed + if statusErr := backupUtil.UpdateStatus(r.KubeClient, adbBackup); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, nil + } + + workClient, err := workrequests.NewWorkRequestClientWithConfigurationProvider(provider) + if err != nil { + r.currentLogger.Error(err, "Fail to get OCI work request client") + + // Change the status to UNAVAILABLE + adbBackup.Status.LifecycleState = database.AutonomousDatabaseBackupLifecycleStateFailed + if statusErr := backupUtil.UpdateStatus(r.KubeClient, adbBackup); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, nil + } + + /****************************************************************** + * Create a backup if the AutonomousDatabaseBackupOCID is empty + ******************************************************************/ + if adbBackup.Spec.AutonomousDatabaseBackupOCID == "" { + resp, err := oci.CreateAutonomousDatabaseBackup(r.currentLogger, dbClient, adbBackup) + if err != nil { + r.currentLogger.Error(err, "Fail to create AutonomousDatabase Backup") + + // Change the status to UNAVAILABLE + adbBackup.Status.LifecycleState = database.AutonomousDatabaseBackupLifecycleStateFailed + if statusErr := backupUtil.UpdateStatus(r.KubeClient, adbBackup); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, nil + } + + // update the status + adbBackup.Status.AutonomousDatabaseBackupOCID = *resp.Id + adbBackup.Status.LifecycleState = resp.LifecycleState + backupUtil.UpdateStatus(r.KubeClient, adbBackup) + + // Wait until the work is done + if err := oci.WaitUntilWorkCompleted(r.currentLogger, workClient, resp.OpcWorkRequestId); err != nil { + r.currentLogger.Error(err, "Fail to watch the status of provision request. opcWorkRequestID = "+*resp.OpcWorkRequestId) + + // Change the status to UNAVAILABLE + adbBackup.Status.LifecycleState = database.AutonomousDatabaseBackupLifecycleStateFailed + if statusErr := backupUtil.UpdateStatus(r.KubeClient, adbBackup); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, nil + } + } else { + // Binding + // Add the Status.AutonomousDatabaseBackupOCID and update the status + adbBackup.Status.AutonomousDatabaseBackupOCID = adbBackup.Spec.AutonomousDatabaseBackupOCID + backupUtil.UpdateStatus(r.KubeClient, adbBackup) + } + + /****************************************************************** + * Update the status of the resource + ******************************************************************/ + resp, err := oci.GetAutonomousDatabaseBackup(dbClient, &adbBackup.Status.AutonomousDatabaseBackupOCID) + if err != nil { + // Change the status to UNAVAILABLE + adbBackup.Status.LifecycleState = database.AutonomousDatabaseBackupLifecycleStateFailed + if statusErr := backupUtil.UpdateStatus(r.KubeClient, adbBackup); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, err + } + + adbBackup.UpdateStatusFromAutonomousDatabaseBackupResponse(resp) + + backupUtil.UpdateStatus(r.KubeClient, adbBackup) + + r.currentLogger.Info("AutonomousDatabaseBackup reconcile successfully") + + return ctrl.Result{}, nil +} diff --git a/main.go b/main.go index 9e68368f..8ebfa939 100644 --- a/main.go +++ b/main.go @@ -98,6 +98,14 @@ func main() { setupLog.Error(err, "unable to create controller", "controller", "AutonomousDatabase") os.Exit(1) } + if err = (&databasecontroller.AutonomousDatabaseBackupReconciler{ + KubeClient: mgr.GetClient(), + Log: ctrl.Log.WithName("controllers").WithName("AutonomousDatabaseBackup"), + Scheme: mgr.GetScheme(), + }).SetupWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create controller", "controller", "AutonomousDatabaseBackup") + os.Exit(1) + } if err = (&databasecontroller.SingleInstanceDatabaseReconciler{ Client: mgr.GetClient(), Log: ctrl.Log.WithName("controllers").WithName("database").WithName("SingleInstanceDatabase"), From 92f7b5e4704a69a1e710cda1e25dfc7505ac4576 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Mon, 8 Nov 2021 09:58:22 -0500 Subject: [PATCH 039/628] Set AutonomousDatabase as the owner --- .../adb_backup_reconciler_util.go} | 48 +++++++- ...onciler_util.go => adb_reconciler_util.go} | 26 ++--- commons/autonomousdatabase/common_util.go | 103 ++++++++++++++++++ .../database/autonomousdatabase_controller.go | 48 ++++---- .../autonomousdatabasebackup_controller.go | 46 +++++--- 5 files changed, 205 insertions(+), 66 deletions(-) rename commons/{autonomousdatabase_backup/reconciler_util.go => autonomousdatabase/adb_backup_reconciler_util.go} (64%) rename commons/autonomousdatabase/{reconciler_util.go => adb_reconciler_util.go} (91%) create mode 100644 commons/autonomousdatabase/common_util.go diff --git a/commons/autonomousdatabase_backup/reconciler_util.go b/commons/autonomousdatabase/adb_backup_reconciler_util.go similarity index 64% rename from commons/autonomousdatabase_backup/reconciler_util.go rename to commons/autonomousdatabase/adb_backup_reconciler_util.go index c95a2608..d819ce71 100644 --- a/commons/autonomousdatabase_backup/reconciler_util.go +++ b/commons/autonomousdatabase/adb_backup_reconciler_util.go @@ -36,10 +36,11 @@ ** SOFTWARE. */ -package autonomousdatabase_backup +package autonomousdatabase import ( "context" + "fmt" "k8s.io/apimachinery/pkg/types" "k8s.io/client-go/util/retry" @@ -49,7 +50,41 @@ import ( dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" ) -func UpdateAutonomousDatabaseBackupResource(logger logr.Logger, kubeClient client.Client, backup *dbv1alpha1.AutonomousDatabaseBackup) error { +// Returns the first AutonomousDatabase resource that matches the AutonomousDatabaseOCID of the backup +// If the AutonomousDatabase doesn't exist, returns a nil +func getOwnerAutonomousDatabase(kubeClient client.Client, namespace string, adbOCID string) (*dbv1alpha1.AutonomousDatabase, error) { + adbList, err := fetchAutonomousDatabases(kubeClient, namespace) + if err != nil { + return nil, err + } + + for _, adb := range adbList.Items { + if adb.Spec.Details.AutonomousDatabaseOCID != nil && *adb.Spec.Details.AutonomousDatabaseOCID == adbOCID { + return &adb, nil + } + } + + return nil, nil +} + +// SetOwnerAutonomousDatabase sets the owner of the AutonomousDatabaseBackup if the AutonomousDatabase resource with the same database OCID is found +func SetOwnerAutonomousDatabase(logger logr.Logger, kubeClient client.Client, backup *dbv1alpha1.AutonomousDatabaseBackup) error { + adb, err := getOwnerAutonomousDatabase(kubeClient, backup.Namespace, backup.Status.AutonomousDatabaseOCID) + if err != nil { + return err + } + + if adb != nil { + backup.SetOwnerReferences(newOwnerReference(adb)) + updateAutonomousDatabaseBackupResource(logger, kubeClient, backup) + logger.Info(fmt.Sprintf("Set the owner of %s to %s", backup.Name, adb.Name)) + } + + return nil +} + +// update the spec and the objectMeta +func updateAutonomousDatabaseBackupResource(logger logr.Logger, kubeClient client.Client, backup *dbv1alpha1.AutonomousDatabaseBackup) error { if err := retry.RetryOnConflict(retry.DefaultRetry, func() error { curBackup := &dbv1alpha1.AutonomousDatabaseBackup{} @@ -62,14 +97,15 @@ func UpdateAutonomousDatabaseBackupResource(logger logr.Logger, kubeClient clien return err } - curBackup.Spec = backup.Spec + curBackup.Spec = *backup.Spec.DeepCopy() + curBackup.ObjectMeta = *backup.ObjectMeta.DeepCopy() return kubeClient.Update(context.TODO(), curBackup) }); err != nil { return err } // Update status - if statusErr := UpdateStatus(kubeClient, backup); statusErr != nil { + if statusErr := UpdateAutonomousDatabaseBackupStatus(kubeClient, backup); statusErr != nil { return statusErr } logger.Info("Update local resource AutonomousDatabase successfully") @@ -77,8 +113,8 @@ func UpdateAutonomousDatabaseBackupResource(logger logr.Logger, kubeClient clien return nil } -// UpdateStatus sets the status subresource of AutonomousDatabaseBackup -func UpdateStatus(kubeClient client.Client, adbBackup *dbv1alpha1.AutonomousDatabaseBackup) error { +// UpdateAutonomousDatabaseBackupStatus updates the status subresource of AutonomousDatabaseBackup +func UpdateAutonomousDatabaseBackupStatus(kubeClient client.Client, adbBackup *dbv1alpha1.AutonomousDatabaseBackup) error { return retry.RetryOnConflict(retry.DefaultRetry, func() error { curBackup := &dbv1alpha1.AutonomousDatabaseBackup{} diff --git a/commons/autonomousdatabase/reconciler_util.go b/commons/autonomousdatabase/adb_reconciler_util.go similarity index 91% rename from commons/autonomousdatabase/reconciler_util.go rename to commons/autonomousdatabase/adb_reconciler_util.go index 094641d3..11ce331b 100644 --- a/commons/autonomousdatabase/reconciler_util.go +++ b/commons/autonomousdatabase/adb_reconciler_util.go @@ -60,8 +60,8 @@ import ( "github.com/oracle/oracle-database-operator/commons/oci" ) -// SetStatus sets the status subresource of AutonomousDatabase -func SetStatus(kubeClient client.Client, adb *dbv1alpha1.AutonomousDatabase) error { +// UpdateAutonomousDatabaseStatus sets the status subresource of AutonomousDatabase +func UpdateAutonomousDatabaseStatus(kubeClient client.Client, adb *dbv1alpha1.AutonomousDatabase) error { return retry.RetryOnConflict(retry.DefaultRetry, func() error { curADB := &dbv1alpha1.AutonomousDatabase{} @@ -143,23 +143,14 @@ func getValidName(name string, usedNames map[string]bool) string { return returnedName } -func getOwnerReference(adb *dbv1alpha1.AutonomousDatabase) []metav1.OwnerReference { - ownerRef := []metav1.OwnerReference{ - { - Kind: adb.GroupVersionKind().Kind, - APIVersion: adb.APIVersion, - Name: adb.Name, - UID: types.UID(adb.UID), - }, - } - return ownerRef -} - // CreateBackupResources creates the all the AutonomousDatabasBackups that appears in the ListAutonomousDatabaseBackups request // The backup object will not be created if it already exists. func CreateBackupResources(logger logr.Logger, kubeClient client.Client, dbClient database.DatabaseClient, adb *dbv1alpha1.AutonomousDatabase) error { // Get the list of AutonomousDatabaseBackupOCID in the same namespace - backupList := &dbv1alpha1.AutonomousDatabaseBackupList{} + backupList, err := fetchAutonomousDatabaseBackups(kubeClient, adb.Namespace) + if err != nil { + return err + } if err := kubeClient.List(context.TODO(), backupList, &client.ListOptions{Namespace: adb.Namespace}); err != nil { // Ignore not-found errors, since they can't be fixed by an immediate requeue. @@ -191,9 +182,6 @@ func CreateBackupResources(logger logr.Logger, kubeClient client.Client, dbClien return err } - var ownerRef []metav1.OwnerReference - ownerRef = append(ownerRef, metav1.OwnerReference{Kind: adb.GroupVersionKind().Kind, APIVersion: adb.APIVersion, Name: adb.Name, UID: types.UID(adb.UID)}) - for _, backupSummary := range resp.Items { // Create the resource if the AutonomousDatabaseBackupOCID doesn't exist _, ok := usedBackupOCIDs[*backupSummary.Id] @@ -213,7 +201,7 @@ func CreateBackupResources(logger logr.Logger, kubeClient client.Client, dbClien ObjectMeta: metav1.ObjectMeta{ Namespace: adb.GetNamespace(), Name: backupName, - OwnerReferences: getOwnerReference(adb), + OwnerReferences: newOwnerReference(adb), }, Spec: dbv1alpha1.AutonomousDatabaseBackupSpec{ AutonomousDatabaseBackupOCID: *backupSummary.Id, diff --git a/commons/autonomousdatabase/common_util.go b/commons/autonomousdatabase/common_util.go new file mode 100644 index 00000000..a344d3f6 --- /dev/null +++ b/commons/autonomousdatabase/common_util.go @@ -0,0 +1,103 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package autonomousdatabase + +import ( + "context" + + apiErrors "k8s.io/apimachinery/pkg/api/errors" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "sigs.k8s.io/controller-runtime/pkg/client" + + dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" +) + +func newOwnerReference(owner client.Object) []metav1.OwnerReference { + ownerRef := []metav1.OwnerReference{ + { + Kind: owner.GetObjectKind().GroupVersionKind().Kind, + APIVersion: owner.GetObjectKind().GroupVersionKind().GroupVersion().String(), + Name: owner.GetName(), + UID: owner.GetUID(), + }, + } + return ownerRef +} + +// func newOwnerReference(adb *dbv1alpha1.AutonomousDatabase) []metav1.OwnerReference { +// ownerRef := []metav1.OwnerReference{ +// { +// Kind: adb.GroupVersionKind().Kind, +// APIVersion: adb.APIVersion, +// Name: adb.Name, +// UID: types.UID(adb.UID), +// }, +// } +// return ownerRef +// } + +func fetchAutonomousDatabases(kubeClient client.Client, namespace string) (*dbv1alpha1.AutonomousDatabaseList, error) { + // Get the list of AutonomousDatabaseBackupOCID in the same namespace + adbList := &dbv1alpha1.AutonomousDatabaseList{} + + if err := kubeClient.List(context.TODO(), adbList, &client.ListOptions{Namespace: namespace}); err != nil { + // Ignore not-found errors, since they can't be fixed by an immediate requeue. + // No need to change the since we don't know if we obtain the object. + if !apiErrors.IsNotFound(err) { + return adbList, err + } + } + + return adbList, nil +} + +func fetchAutonomousDatabaseBackups(kubeClient client.Client, namespace string) (*dbv1alpha1.AutonomousDatabaseBackupList, error) { + // Get the list of AutonomousDatabaseBackupOCID in the same namespace + backupList := &dbv1alpha1.AutonomousDatabaseBackupList{} + + if err := kubeClient.List(context.TODO(), backupList, &client.ListOptions{Namespace: namespace}); err != nil { + // Ignore not-found errors, since they can't be fixed by an immediate requeue. + // No need to change the since we don't know if we obtain the object. + if !apiErrors.IsNotFound(err) { + return backupList, err + } + } + + return backupList, nil +} diff --git a/controllers/database/autonomousdatabase_controller.go b/controllers/database/autonomousdatabase_controller.go index 00d281a4..0a2bfbe8 100644 --- a/controllers/database/autonomousdatabase_controller.go +++ b/controllers/database/autonomousdatabase_controller.go @@ -146,7 +146,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } return ctrl.Result{}, nil @@ -158,7 +158,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } return ctrl.Result{}, nil @@ -170,7 +170,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } return ctrl.Result{}, nil @@ -182,7 +182,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } return ctrl.Result{}, nil @@ -220,7 +220,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } } @@ -260,7 +260,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue @@ -271,7 +271,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Update status.state adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } @@ -280,7 +280,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } } @@ -297,7 +297,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } return ctrl.Result{}, nil @@ -320,7 +320,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } return ctrl.Result{}, nil @@ -348,7 +348,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } return ctrl.Result{}, nil @@ -356,7 +356,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Update status.state adb.Status.LifecycleState = lifecycleState - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } @@ -365,7 +365,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } } @@ -382,7 +382,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue @@ -392,7 +392,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R if updateGenPassResp.OpcWorkRequestId != nil { // Update status.state adb.Status.LifecycleState = updateGenPassResp.AutonomousDatabase.LifecycleState - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } @@ -401,7 +401,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } } @@ -414,7 +414,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue @@ -424,7 +424,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R if scaleResp.OpcWorkRequestId != nil { // Update status.state adb.Status.LifecycleState = scaleResp.AutonomousDatabase.LifecycleState - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } @@ -433,7 +433,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } } @@ -449,7 +449,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } return ctrl.Result{}, nil @@ -461,7 +461,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R if err := r.updateAutonomousDatabaseDetails(adb); err != nil { // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } return ctrl.Result{}, err @@ -480,7 +480,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R r.currentLogger.Error(err, "Fail to download Instance Wallet") // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } return ctrl.Result{}, nil @@ -495,7 +495,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue @@ -556,7 +556,7 @@ func (r *AutonomousDatabaseReconciler) updateAutonomousDatabaseDetails(adb *dbv1 } // Update status - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return statusErr } r.currentLogger.Info("Update local resource AutonomousDatabase successfully") diff --git a/controllers/database/autonomousdatabasebackup_controller.go b/controllers/database/autonomousdatabasebackup_controller.go index 0cf425c6..966182f1 100644 --- a/controllers/database/autonomousdatabasebackup_controller.go +++ b/controllers/database/autonomousdatabasebackup_controller.go @@ -55,7 +55,7 @@ import ( "github.com/oracle/oci-go-sdk/v45/workrequests" databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" - backupUtil "github.com/oracle/oracle-database-operator/commons/autonomousdatabase_backup" + backupUtil "github.com/oracle/oracle-database-operator/commons/autonomousdatabase" "github.com/oracle/oracle-database-operator/commons/oci" ) @@ -84,13 +84,12 @@ func (r *AutonomousDatabaseBackupReconciler) eventFilterPredicate() predicate.Pr newBackup := e.ObjectNew.DeepCopyObject().(*dbv1alpha1.AutonomousDatabaseBackup) specChanged := reflect.DeepEqual(oldBackup.Spec, newBackup.Spec) - statusChanged := reflect.DeepEqual(oldBackup.Status, newBackup.Status) - if specChanged || statusChanged { - // Don't enqueue request - return false + if specChanged { + // Enqueue request + return true } - // Enqueue request - return true + // Don't enqueue request + return false }, DeleteFunc: func(e event.DeleteEvent) bool { // Do not trigger reconciliation when the real object is deleted from the cluster. @@ -130,7 +129,7 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req // Change the status to UNAVAILABLE adbBackup.Status.LifecycleState = database.AutonomousDatabaseBackupLifecycleStateFailed - if statusErr := backupUtil.UpdateStatus(r.KubeClient, adbBackup); statusErr != nil { + if statusErr := backupUtil.UpdateAutonomousDatabaseBackupStatus(r.KubeClient, adbBackup); statusErr != nil { return ctrl.Result{}, statusErr } return ctrl.Result{}, nil @@ -142,7 +141,7 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req // Change the status to UNAVAILABLE adbBackup.Status.LifecycleState = database.AutonomousDatabaseBackupLifecycleStateFailed - if statusErr := backupUtil.UpdateStatus(r.KubeClient, adbBackup); statusErr != nil { + if statusErr := backupUtil.UpdateAutonomousDatabaseBackupStatus(r.KubeClient, adbBackup); statusErr != nil { return ctrl.Result{}, statusErr } return ctrl.Result{}, nil @@ -154,14 +153,15 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req // Change the status to UNAVAILABLE adbBackup.Status.LifecycleState = database.AutonomousDatabaseBackupLifecycleStateFailed - if statusErr := backupUtil.UpdateStatus(r.KubeClient, adbBackup); statusErr != nil { + if statusErr := backupUtil.UpdateAutonomousDatabaseBackupStatus(r.KubeClient, adbBackup); statusErr != nil { return ctrl.Result{}, statusErr } return ctrl.Result{}, nil } /****************************************************************** - * Create a backup if the AutonomousDatabaseBackupOCID is empty + * Create a backup if the AutonomousDatabaseBackupOCID is empty, + * otherwise bind to an exisiting backup ******************************************************************/ if adbBackup.Spec.AutonomousDatabaseBackupOCID == "" { resp, err := oci.CreateAutonomousDatabaseBackup(r.currentLogger, dbClient, adbBackup) @@ -170,7 +170,7 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req // Change the status to UNAVAILABLE adbBackup.Status.LifecycleState = database.AutonomousDatabaseBackupLifecycleStateFailed - if statusErr := backupUtil.UpdateStatus(r.KubeClient, adbBackup); statusErr != nil { + if statusErr := backupUtil.UpdateAutonomousDatabaseBackupStatus(r.KubeClient, adbBackup); statusErr != nil { return ctrl.Result{}, statusErr } return ctrl.Result{}, nil @@ -179,7 +179,7 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req // update the status adbBackup.Status.AutonomousDatabaseBackupOCID = *resp.Id adbBackup.Status.LifecycleState = resp.LifecycleState - backupUtil.UpdateStatus(r.KubeClient, adbBackup) + backupUtil.UpdateAutonomousDatabaseBackupStatus(r.KubeClient, adbBackup) // Wait until the work is done if err := oci.WaitUntilWorkCompleted(r.currentLogger, workClient, resp.OpcWorkRequestId); err != nil { @@ -187,7 +187,7 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req // Change the status to UNAVAILABLE adbBackup.Status.LifecycleState = database.AutonomousDatabaseBackupLifecycleStateFailed - if statusErr := backupUtil.UpdateStatus(r.KubeClient, adbBackup); statusErr != nil { + if statusErr := backupUtil.UpdateAutonomousDatabaseBackupStatus(r.KubeClient, adbBackup); statusErr != nil { return ctrl.Result{}, statusErr } return ctrl.Result{}, nil @@ -196,7 +196,7 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req // Binding // Add the Status.AutonomousDatabaseBackupOCID and update the status adbBackup.Status.AutonomousDatabaseBackupOCID = adbBackup.Spec.AutonomousDatabaseBackupOCID - backupUtil.UpdateStatus(r.KubeClient, adbBackup) + backupUtil.UpdateAutonomousDatabaseBackupStatus(r.KubeClient, adbBackup) } /****************************************************************** @@ -206,7 +206,7 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req if err != nil { // Change the status to UNAVAILABLE adbBackup.Status.LifecycleState = database.AutonomousDatabaseBackupLifecycleStateFailed - if statusErr := backupUtil.UpdateStatus(r.KubeClient, adbBackup); statusErr != nil { + if statusErr := backupUtil.UpdateAutonomousDatabaseBackupStatus(r.KubeClient, adbBackup); statusErr != nil { return ctrl.Result{}, statusErr } return ctrl.Result{}, err @@ -214,7 +214,19 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req adbBackup.UpdateStatusFromAutonomousDatabaseBackupResponse(resp) - backupUtil.UpdateStatus(r.KubeClient, adbBackup) + backupUtil.UpdateAutonomousDatabaseBackupStatus(r.KubeClient, adbBackup) + + /****************************************************************** + * Update the ownerReference if the AutonomousDatabase is found + ******************************************************************/ + if err := backupUtil.SetOwnerAutonomousDatabase(r.currentLogger, r.KubeClient, adbBackup); err != nil { + // Change the status to UNAVAILABLE + adbBackup.Status.LifecycleState = database.AutonomousDatabaseBackupLifecycleStateFailed + if statusErr := backupUtil.UpdateAutonomousDatabaseBackupStatus(r.KubeClient, adbBackup); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, err + } r.currentLogger.Info("AutonomousDatabaseBackup reconcile successfully") From 28017e435585985d84b14e52b2e49d66c8521caf Mon Sep 17 00:00:00 2001 From: mmahipal Date: Thu, 11 Nov 2021 14:32:50 +0530 Subject: [PATCH 040/628] Added ORDS --- PREREQUISITES.md | 6 +- PROJECT | 9 + README.md | 8 +- .../v1alpha1/oraclerestdataservice_types.go | 147 ++ .../v1alpha1/oraclerestdataservice_webhook.go | 141 ++ .../v1alpha1/zz_generated.deepcopy.go | 166 +++ commons/database/constants.go | 248 +++- ...ase.oracle.com_oraclerestdataservices.yaml | 202 +++ config/crd/kustomization.yaml | 3 + ...cainjection_in_oraclerestdataservices.yaml | 8 + .../webhook_in_oraclerestdataservices.yaml | 17 + config/manager/kustomization.yaml | 2 +- .../oraclerestdataservice_editor_role.yaml | 24 + .../oraclerestdataservice_viewer_role.yaml | 20 + config/rbac/role.yaml | 26 + .../samples/sidb/oraclerestdataservice.yaml | 53 + .../samples/sidb/singleinstancedatabase.yaml | 6 + config/webhook/manifests.yaml | 42 + .../oraclerestdataservice_controller.go | 1298 +++++++++++++++++ .../singleinstancedatabase_controller.go | 174 +++ controllers/database/suite_test.go | 3 + docs/sidb/README.md | 342 +++++ .../sidb/application-express-admin-home.png | Bin 0 -> 143392 bytes images/sidb/database-actions-home.png | Bin 0 -> 231151 bytes main.go | 15 + oracle-database-operator.yaml | 257 +++- 26 files changed, 3207 insertions(+), 10 deletions(-) create mode 100644 apis/database/v1alpha1/oraclerestdataservice_types.go create mode 100644 apis/database/v1alpha1/oraclerestdataservice_webhook.go create mode 100644 config/crd/bases/database.oracle.com_oraclerestdataservices.yaml create mode 100644 config/crd/patches/cainjection_in_oraclerestdataservices.yaml create mode 100644 config/crd/patches/webhook_in_oraclerestdataservices.yaml create mode 100644 config/rbac/oraclerestdataservice_editor_role.yaml create mode 100644 config/rbac/oraclerestdataservice_viewer_role.yaml create mode 100644 config/samples/sidb/oraclerestdataservice.yaml create mode 100644 controllers/database/oraclerestdataservice_controller.go create mode 100644 images/sidb/application-express-admin-home.png create mode 100644 images/sidb/database-actions-home.png diff --git a/PREREQUISITES.md b/PREREQUISITES.md index 7fc3cf21..0dd557ae 100644 --- a/PREREQUISITES.md +++ b/PREREQUISITES.md @@ -22,12 +22,12 @@ Note: You must provision persistent storage if you intend to deploy containerize ### Prerequites for Oracle Autonomous Database (ADB) -If you intent to use `OraOperator` to handle Oracle Autonomous Database lifecycles, then read [Oracle Autonomous Database prerequisites](./docs/adb/ADB_PREREQUISITES.md) +If you intent to use `OraOperator` to handle Oracle Autonomous Database lifecycles, then read [Oracle Autonomous Database prerequisites](./doc/adb/ADB_PREREQUISITES.md) ### Prerequites for Single Instance Databases (SIDB) -If you intent to use `OraOperator` to handle Oracle Database Single Instance lifecycles, then read [Single Instance Database Prerequisites](./docs/sidb/SIDB_PREREQUISITES.md) +If you intent to use `OraOperator` to handle Oracle Database Single Instance lifecycles, then read [Single Instance Database Prerequisites](./doc/sidb/SIDB_PREREQUISITES.md) ### Prerequites for Sharded Databases (SHARDING) - If you intent to use OraOperator to handle the lifecycle of Oracle Database deployed with Oracle Sharding, then read [Sharded Database Prerequisites](./docs/sharding/README.md#prerequsites-for-running-oracle-sharding-database-controller) + If you intent to use OraOperator to handle the lifecycle of Oracle Database deployed with Oracle Sharding, then read [Sharded Database Prerequisites](./doc/sharding/README.md#prerequsites-for-running-oracle-sharding-database-controller) diff --git a/PROJECT b/PROJECT index 70e34778..296f26d0 100644 --- a/PROJECT +++ b/PROJECT @@ -38,4 +38,13 @@ resources: kind: ShardingDatabase path: github.com/oracle/oracle-database-operator/apis/database/v1alpha1 version: v1alpha1 +- api: + crdVersion: v1beta1 + namespaced: true + controller: true + domain: oracle.com + group: database + kind: OracleRestDataService + path: github.com/oracle/oracle-database-operator/apis/database/v1alpha1 + version: v1alpha1 version: "3" diff --git a/README.md b/README.md index cd72a209..76d97495 100644 --- a/README.md +++ b/README.md @@ -75,15 +75,15 @@ Oracle strongly recommends that you ensure your system meets the following [Prer You should see that the operator is up and running, along with the shipped controllers. -For more details, see [Oracle Database Operator Installation Instrunctions](./docs/installation/OPERATOR_INSTALLATION_README.md). +For more details, see [Oracle Database Operator Installation Instrunctions](./doc/installation/OPERATOR_INSTALLATION_README.md). ## Getting Started with the Operator (Quickstart) The quickstarts are designed for specific database configurations, including: -* [Oracle Autonomous Database](./docs/adb/README.md) -* [Oracle Database Single Instance configuration](./docs/sidb/README.md) -* [Oracle Database configured with Oracle Sharding](./docs/sharding/README.md) +* [Oracle Autonomous Database](./doc/adb/README.md) +* [Oracle Database Single Instance configuration](./doc/sidb/README.md) +* [Oracle Database configured with Oracle Sharding](./doc/sharding/README.md) YAML file templates are available under [`/config/samples`](./config/samples/). You can copy and edit these template files to configure them for your use cases. diff --git a/apis/database/v1alpha1/oraclerestdataservice_types.go b/apis/database/v1alpha1/oraclerestdataservice_types.go new file mode 100644 index 00000000..334005a7 --- /dev/null +++ b/apis/database/v1alpha1/oraclerestdataservice_types.go @@ -0,0 +1,147 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v1alpha1 + +import ( + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! +// NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized. + +// OracleRestDataServiceSpec defines the desired state of OracleRestDataService +type OracleRestDataServiceSpec struct { + // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster + // Important: Run "make" to regenerate code after modifying this file + + DatabaseRef string `json:"databaseRef"` + LoadBalancer bool `json:"loadBalancer,omitempty"` + NodeSelector map[string]string `json:"nodeSelector,omitempty"` + Image OracleRestDataServiceImage `json:"image,omitempty"` + OrdsPassword OracleRestDataServicePassword `json:"ordsPassword"` + ApexPassword OracleRestDataServicePassword `json:"apexPassword,omitempty"` + AdminPassword OracleRestDataServicePassword `json:"adminPassword"` + OrdsUser string `json:"ordsUser,omitempty"` + RestEnableSchemas []OracleRestDataServiceRestEnableSchemas `json:"restEnableSchemas,omitempty"` + OracleService string `json:"oracleService,omitempty"` + + // +k8s:openapi-gen=true + // +kubebuilder:validation:Minimum=1 + Replicas int `json:"replicas,omitempty"` +} + +// OracleRestDataServicePersistence defines the storage releated params +type OracleRestDataServicePersistence struct { + Size string `json:"size"` + StorageClass string `json:"storageClass"` + + // +kubebuilder:validation:Enum=ReadWriteOnce;ReadWriteMany + AccessMode string `json:"accessMode"` +} + +// OracleRestDataServiceImage defines the Image source and pullSecrets for POD +type OracleRestDataServiceImage struct { + Version string `json:"version,omitempty"` + PullFrom string `json:"pullFrom"` + PullSecrets string `json:"pullSecrets,omitempty"` +} + +// OracleRestDataServicePassword defines the secret containing Password mapped to secretKey +type OracleRestDataServicePassword struct { + SecretName string `json:"secretName,omitempty"` + SecretKey string `json:"secretKey"` + KeepSecret bool `json:"keepSecret"` +} + +// OracleRestDataServicePDBSchemas defines the PDB Schemas to be ORDS Enabled +type OracleRestDataServiceRestEnableSchemas struct { + Pdb string `json:"pdb"` + Schema string `json:"schema"` + UrlMapping string `json:"urlMapping,omitempty"` + Enable bool `json:"enable"` +} + +// OracleRestDataServiceStatus defines the observed state of OracleRestDataService +type OracleRestDataServiceStatus struct { + // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster + // Important: Run "make" to regenerate code after modifying this file + Status string `json:"status,omitempty"` + DatabaseApiUrl string `json:"databaseApiUrl,omitempty"` + ClusterDbApiUrl string `json:"clusterDbApiUrl,omitempty"` + LoadBalancer string `json:"loadBalancer,omitempty"` + DatabaseRef string `json:"databaseRef,omitempty"` + ServiceIP string `json:"serviceIP,omitempty,omitempty"` + DatabaseActionsUrl string `json:"databaseActionsUrl,omitempty"` + OrdsInstalled bool `json:"ordsInstalled,omitempty"` + ApexConfigured bool `json:"apexConfigured,omitempty"` + CommonUsersCreated bool `json:"commonUsersCreated,omitempty"` + OrdsSetupCompleted bool `json:"ordsSetupCompleted,omitempty"` + Replicas int `json:"replicas,omitempty"` + + Image OracleRestDataServiceImage `json:"image,omitempty"` +} + +//+kubebuilder:object:root=true +//+kubebuilder:subresource:status +// +kubebuilder:printcolumn:JSONPath=".status.status",name="Status",type="string" +// +kubebuilder:printcolumn:JSONPath=".spec.databaseRef",name="Database",type="string" +// +kubebuilder:printcolumn:JSONPath=".status.databaseApiUrl",name="Database Api & Apex Url ",type="string" +// +kubebuilder:printcolumn:JSONPath=".status.databaseActionsUrl",name="Database Actions Url",type="string" + +// OracleRestDataService is the Schema for the oraclerestdataservices API +type OracleRestDataService struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + Spec OracleRestDataServiceSpec `json:"spec,omitempty"` + Status OracleRestDataServiceStatus `json:"status,omitempty"` +} + +//+kubebuilder:object:root=true + +// OracleRestDataServiceList contains a list of OracleRestDataService +type OracleRestDataServiceList struct { + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata,omitempty"` + Items []OracleRestDataService `json:"items"` +} + +func init() { + SchemeBuilder.Register(&OracleRestDataService{}, &OracleRestDataServiceList{}) +} diff --git a/apis/database/v1alpha1/oraclerestdataservice_webhook.go b/apis/database/v1alpha1/oraclerestdataservice_webhook.go new file mode 100644 index 00000000..3a546d46 --- /dev/null +++ b/apis/database/v1alpha1/oraclerestdataservice_webhook.go @@ -0,0 +1,141 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v1alpha1 + +import ( + "strconv" + + apierrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/runtime/schema" + "k8s.io/apimachinery/pkg/util/validation/field" + ctrl "sigs.k8s.io/controller-runtime" + logf "sigs.k8s.io/controller-runtime/pkg/log" + "sigs.k8s.io/controller-runtime/pkg/webhook" +) + +// log is for logging in this package. +var oraclerestdataservicelog = logf.Log.WithName("oraclerestdataservice-resource") + +func (r *OracleRestDataService) SetupWebhookWithManager(mgr ctrl.Manager) error { + return ctrl.NewWebhookManagedBy(mgr). + For(r). + Complete() +} + +// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! + +//+kubebuilder:webhook:path=/mutate-database-oracle-com-v1alpha1-oraclerestdataservice,mutating=true,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=oraclerestdataservices,verbs=create;update,versions=v1alpha1,name=moraclerestdataservice.kb.io,admissionReviewVersions={v1,v1beta1} + +var _ webhook.Defaulter = &OracleRestDataService{} + +// Default implements webhook.Defaulter so a webhook will be registered for the type +func (r *OracleRestDataService) Default() { + oraclerestdataservicelog.Info("default", "name", r.Name) + // OracleRestDataService Currently supports single replica + r.Spec.Replicas = 1 + // TODO(user): fill in your defaulting logic. +} + +// TODO(user): change verbs to "verbs=create;update;delete" if you want to enable deletion validation. +//+kubebuilder:webhook:verbs=create;update,path=/validate-database-oracle-com-v1alpha1-oraclerestdataservice,mutating=false,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=oraclerestdataservices,versions=v1alpha1,name=voraclerestdataservice.kb.io,admissionReviewVersions={v1,v1beta1} + +var _ webhook.Validator = &OracleRestDataService{} + +// ValidateCreate implements webhook.Validator so a webhook will be registered for the type +func (r *OracleRestDataService) ValidateCreate() error { + oraclerestdataservicelog.Info("validate create", "name", r.Name) + + var allErrs field.ErrorList + + if len(allErrs) == 0 { + return nil + } + return apierrors.NewInvalid( + schema.GroupKind{Group: "database.oracle.com", Kind: "OracleRestDataService"}, + r.Name, allErrs) +} + +// ValidateUpdate implements webhook.Validator so a webhook will be registered for the type +func (r *OracleRestDataService) ValidateUpdate(oldRuntimeObject runtime.Object) error { + oraclerestdataservicelog.Info("validate update", "name", r.Name) + + var allErrs field.ErrorList + + // check creation validations first + err := r.ValidateCreate() + if err != nil { + return err + } + + // Now check for updation errors + old, ok := oldRuntimeObject.(*OracleRestDataService) + if !ok { + return nil + } + + if old.Status.DatabaseRef != "" && old.Status.DatabaseRef != r.Spec.DatabaseRef { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("databaseRef"), "cannot be changed")) + } + if old.Status.LoadBalancer != "" && old.Status.LoadBalancer != strconv.FormatBool(r.Spec.LoadBalancer) { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("loadBalancer"), "cannot be changed")) + } + if old.Status.Image.PullFrom != "" && old.Status.Image != r.Spec.Image { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("image"), "cannot be changed")) + } + + if len(allErrs) == 0 { + return nil + } + return apierrors.NewInvalid( + schema.GroupKind{Group: "database.oracle.com", Kind: "OracleRestDataService"}, + r.Name, allErrs) + +} + +// ValidateDelete implements webhook.Validator so a webhook will be registered for the type +func (r *OracleRestDataService) ValidateDelete() error { + oraclerestdataservicelog.Info("validate delete", "name", r.Name) + + // TODO(user): fill in your validation logic upon object deletion. + return nil +} diff --git a/apis/database/v1alpha1/zz_generated.deepcopy.go b/apis/database/v1alpha1/zz_generated.deepcopy.go index 2766d559..e158d8de 100644 --- a/apis/database/v1alpha1/zz_generated.deepcopy.go +++ b/apis/database/v1alpha1/zz_generated.deepcopy.go @@ -417,6 +417,172 @@ func (in *OCIConfigSpec) DeepCopy() *OCIConfigSpec { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *OracleRestDataService) DeepCopyInto(out *OracleRestDataService) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + in.Spec.DeepCopyInto(&out.Spec) + out.Status = in.Status +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new OracleRestDataService. +func (in *OracleRestDataService) DeepCopy() *OracleRestDataService { + if in == nil { + return nil + } + out := new(OracleRestDataService) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *OracleRestDataService) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *OracleRestDataServiceImage) DeepCopyInto(out *OracleRestDataServiceImage) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new OracleRestDataServiceImage. +func (in *OracleRestDataServiceImage) DeepCopy() *OracleRestDataServiceImage { + if in == nil { + return nil + } + out := new(OracleRestDataServiceImage) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *OracleRestDataServiceList) DeepCopyInto(out *OracleRestDataServiceList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]OracleRestDataService, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new OracleRestDataServiceList. +func (in *OracleRestDataServiceList) DeepCopy() *OracleRestDataServiceList { + if in == nil { + return nil + } + out := new(OracleRestDataServiceList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *OracleRestDataServiceList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *OracleRestDataServicePassword) DeepCopyInto(out *OracleRestDataServicePassword) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new OracleRestDataServicePassword. +func (in *OracleRestDataServicePassword) DeepCopy() *OracleRestDataServicePassword { + if in == nil { + return nil + } + out := new(OracleRestDataServicePassword) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *OracleRestDataServicePersistence) DeepCopyInto(out *OracleRestDataServicePersistence) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new OracleRestDataServicePersistence. +func (in *OracleRestDataServicePersistence) DeepCopy() *OracleRestDataServicePersistence { + if in == nil { + return nil + } + out := new(OracleRestDataServicePersistence) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *OracleRestDataServiceRestEnableSchemas) DeepCopyInto(out *OracleRestDataServiceRestEnableSchemas) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new OracleRestDataServiceRestEnableSchemas. +func (in *OracleRestDataServiceRestEnableSchemas) DeepCopy() *OracleRestDataServiceRestEnableSchemas { + if in == nil { + return nil + } + out := new(OracleRestDataServiceRestEnableSchemas) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *OracleRestDataServiceSpec) DeepCopyInto(out *OracleRestDataServiceSpec) { + *out = *in + if in.NodeSelector != nil { + in, out := &in.NodeSelector, &out.NodeSelector + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + out.Image = in.Image + out.OrdsPassword = in.OrdsPassword + out.ApexPassword = in.ApexPassword + out.AdminPassword = in.AdminPassword + if in.RestEnableSchemas != nil { + in, out := &in.RestEnableSchemas, &out.RestEnableSchemas + *out = make([]OracleRestDataServiceRestEnableSchemas, len(*in)) + copy(*out, *in) + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new OracleRestDataServiceSpec. +func (in *OracleRestDataServiceSpec) DeepCopy() *OracleRestDataServiceSpec { + if in == nil { + return nil + } + out := new(OracleRestDataServiceSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *OracleRestDataServiceStatus) DeepCopyInto(out *OracleRestDataServiceStatus) { + *out = *in + out.Image = in.Image +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new OracleRestDataServiceStatus. +func (in *OracleRestDataServiceStatus) DeepCopy() *OracleRestDataServiceStatus { + if in == nil { + return nil + } + out := new(OracleRestDataServiceStatus) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *PasswordSpec) DeepCopyInto(out *PasswordSpec) { *out = *in diff --git a/commons/database/constants.go b/commons/database/constants.go index fac4719f..81b5e549 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -91,10 +91,115 @@ const ArchiveLogFalseCMD string = CreateChkFileCMD + " && " + "echo -e \"SHUTDOWN IMMEDIATE; \n STARTUP MOUNT; \n ALTER DATABASE NOARCHIVELOG; \n SELECT log_mode FROM v\\$database; \n ALTER DATABASE OPEN;" + " \n ALTER PLUGGABLE DATABASE ALL OPEN; \n ALTER SYSTEM REGISTER;\" | %s && " + RemoveChkFileCMD -const GetDatabaseRoleCMD string = "SELECT DATABASE_ROLE FROM V\\$DATABASE; " +const StandbyDatabasePrerequisitesSQL string = "ALTER SYSTEM SET db_create_file_dest='/opt/oracle/oradata/';" + + "\nALTER SYSTEM SET db_create_online_log_dest_1='/opt/oracle/oradata/';" + + "\nALTER SYSTEM SWITCH LOGFILE;" + + "\nALTER DATABASE ADD STANDBY LOGFILE THREAD 1 GROUP 10 SIZE 200M;" + + "\nALTER DATABASE ADD STANDBY LOGFILE THREAD 1 GROUP 11 SIZE 200M;" + + "\nALTER DATABASE ADD STANDBY LOGFILE THREAD 1 GROUP 12 SIZE 200M;" + + "\nALTER DATABASE ADD STANDBY LOGFILE THREAD 1 GROUP 13 SIZE 200M;" + + "\nALTER SYSTEM SET STANDBY_FILE_MANAGEMENT=AUTO;" + + "\nALTER SYSTEM SET dg_broker_start=TRUE;" + +const StandbyTnsnamesEntry string = ` +##STANDBYDATABASE_SID## = +(DESCRIPTION = +(ADDRESS = (PROTOCOL = TCP)(HOST = ##STANDBYDATABASE_SERVICE_EXPOSED## )(PORT = 1521)) +(CONNECT_DATA = +(SERVER = DEDICATED) +(SERVICE_NAME = ##STANDBYDATABASE_SID##) +) +) +` +const PDBTnsnamesEntry string = ` +##PDB_NAME## = +(DESCRIPTION = +(ADDRESS = (PROTOCOL = TCP)(HOST = 0.0.0.0 )(PORT = 1521)) +(CONNECT_DATA = +(SERVER = DEDICATED) +(SERVICE_NAME = ##PDB_NAME##) +) +) +` + +const PrimaryTnsnamesEntry string = ` +${PRIMARY_SID} = +(DESCRIPTION = +(ADDRESS = (PROTOCOL = TCP)(HOST = ${PRIMARY_IP})(PORT = 1521 )) +(CONNECT_DATA = + (SERVER = DEDICATED) + (SERVICE_NAME = ${PRIMARY_SID}) +) +) +` + +const ListenerEntry string = `LISTENER = +(DESCRIPTION_LIST = +(DESCRIPTION = +(ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC1)) +(ADDRESS = (PROTOCOL = TCP)(HOST = 0.0.0.0)(PORT = 1521)) +) +) +SID_LIST_LISTENER = +(SID_LIST = +(SID_DESC = + (GLOBAL_DBNAME = ${ORACLE_SID^^}) + (SID_NAME = ${ORACLE_SID^^}) + (ORACLE_HOME = ${ORACLE_HOME}) +) +(SID_DESC = + (GLOBAL_DBNAME = ${ORACLE_SID^^}_DGMGRL) + (SID_NAME = ${ORACLE_SID^^}) + (ORACLE_HOME = ${ORACLE_HOME}) + (ENVS="TNS_ADMIN=/opt/oracle/oradata/dbconfig/${ORACLE_SID^^}") +) +) + +DEDICATED_THROUGH_BROKER_LISTENER=ON +` + +const DataguardBrokerMaxPerformanceCMD string = "CREATE CONFIGURATION dg_config AS PRIMARY DATABASE IS ${PRIMARY_SID} CONNECT IDENTIFIER IS ${PRIMARY_DB_CONN_STR};" + + "\nADD DATABASE ${ORACLE_SID} AS CONNECT IDENTIFIER IS ${SVC_HOST}:1521/${ORACLE_SID} MAINTAINED AS PHYSICAL;" + + "\nEDIT DATABASE ${PRIMARY_SID} SET PROPERTY LogXptMode='ASYNC';" + + "\nEDIT DATABASE ${ORACLE_SID} SET PROPERTY LogXptMode='ASYNC';" + + "\nEDIT DATABASE ${PRIMARY_SID} SET PROPERTY STATICCONNECTIDENTIFIER='(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=${PRIMARY_IP})(PORT=1521))" + + "(CONNECT_DATA=(SERVICE_NAME=${PRIMARY_SID}_DGMGRL)(INSTANCE_NAME=${PRIMARY_SID})(SERVER=DEDICATED)))';" + + "\nEDIT DATABASE ${ORACLE_SID} SET PROPERTY STATICCONNECTIDENTIFIER='(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=${SVC_HOST})(PORT=1521))" + + "(CONNECT_DATA=(SERVICE_NAME=${ORACLE_SID}_DGMGRL)(INSTANCE_NAME=${ORACLE_SID})(SERVER=DEDICATED)))';" + + "\nEDIT CONFIGURATION SET PROTECTION MODE AS MAXPERFORMANCE;" + + "\nENABLE CONFIGURATION;" + +const DataguardBrokerMaxAvailabilityCMD string = "CREATE CONFIGURATION dg_config AS PRIMARY DATABASE IS ${PRIMARY_SID} CONNECT IDENTIFIER IS ${PRIMARY_DB_CONN_STR};" + + "\nADD DATABASE ${ORACLE_SID} AS CONNECT IDENTIFIER IS ${SVC_HOST}:1521/${ORACLE_SID} MAINTAINED AS PHYSICAL;" + + "\nEDIT DATABASE ${PRIMARY_SID} SET PROPERTY LogXptMode='SYNC';" + + "\nEDIT DATABASE ${ORACLE_SID} SET PROPERTY LogXptMode='SYNC';" + + "\nEDIT DATABASE ${PRIMARY_SID} SET PROPERTY STATICCONNECTIDENTIFIER='(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=${PRIMARY_IP})(PORT=1521))" + + "(CONNECT_DATA=(SERVICE_NAME=${PRIMARY_SID}_DGMGRL)(INSTANCE_NAME=${PRIMARY_SID})(SERVER=DEDICATED)))';" + + "\nEDIT DATABASE ${ORACLE_SID} SET PROPERTY STATICCONNECTIDENTIFIER='(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=${SVC_HOST})(PORT=1521))" + + "(CONNECT_DATA=(SERVICE_NAME=${ORACLE_SID}_DGMGRL)(INSTANCE_NAME=${ORACLE_SID})(SERVER=DEDICATED)))';" + + "\nEDIT CONFIGURATION SET PROTECTION MODE AS MAXAVAILABILITY;" + + "\nENABLE CONFIGURATION;" + +const DataguardBrokerAddDBMaxPerformanceCMD string = "ADD DATABASE ${ORACLE_SID} AS CONNECT IDENTIFIER IS ${SVC_HOST}:1521/${ORACLE_SID} MAINTAINED AS PHYSICAL;" + + "\nEDIT DATABASE ${ORACLE_SID} SET PROPERTY LogXptMode='ASYNC';" + + "\nEDIT DATABASE ${ORACLE_SID} SET PROPERTY STATICCONNECTIDENTIFIER='(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=${SVC_HOST})(PORT=1521))" + + "(CONNECT_DATA=(SERVICE_NAME=${ORACLE_SID}_DGMGRL)(INSTANCE_NAME=${ORACLE_SID})(SERVER=DEDICATED)))';" + + "\nENABLE CONFIGURATION;" + +const DataguardBrokerAddDBMaxAvailabilityCMD string = "ADD DATABASE ${ORACLE_SID} AS CONNECT IDENTIFIER IS ${SVC_HOST}:1521/${ORACLE_SID} MAINTAINED AS PHYSICAL;" + + "\nEDIT DATABASE ${ORACLE_SID} SET PROPERTY LogXptMode='SYNC';" + + "\nEDIT DATABASE ${ORACLE_SID} SET PROPERTY STATICCONNECTIDENTIFIER='(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=${SVC_HOST})(PORT=1521))" + + "(CONNECT_DATA=(SERVICE_NAME=${ORACLE_SID}_DGMGRL)(INSTANCE_NAME=${ORACLE_SID})(SERVER=DEDICATED)))';" + + "\nENABLE CONFIGURATION;" + +const DBShowConfigCMD string = "SHOW CONFIGURATION;" const DataguardBrokerGetDatabaseCMD string = "SELECT DATABASE || ':' || DATAGUARD_ROLE AS DATABASE FROM V\\$DG_BROKER_CONFIG;" +const EnableFSFOCMD string = "ENABLE FAST_START FAILOVER;" + +const GetDatabaseRoleCMD string = "SELECT DATABASE_ROLE FROM V\\$DATABASE; " + const RunDatapatchCMD string = " ( while true; do sleep 60; echo \"Installing patches...\" ; done ) & if ! $ORACLE_HOME/OPatch/datapatch -skip_upgrade_check;" + " then echo \"Datapatch execution has failed.\" ; else echo \"DONE: Datapatch execution.\" ; fi ; kill -9 $!;" @@ -110,6 +215,96 @@ const GetEnterpriseEditionFileCMD string = "if [ -f ${ORACLE_BASE}/oradata/dbcon const GetStandardEditionFileCMD string = "if [ -f ${ORACLE_BASE}/oradata/dbconfig/$ORACLE_SID/.docker_standard ]; then ls ${ORACLE_BASE}/oradata/dbconfig/$ORACLE_SID/.docker_standard; fi " +const GetPdbsSQL string = "select name from v\\$pdbs where name not like 'PDB\\$SEED' and open_mode like 'READ WRITE';" + +const SetAdminUsersSQL string = "CREATE USER C##DBAPI_CDB_ADMIN IDENTIFIED BY \\\"%[1]s\\\" ACCOUNT UNLOCK CONTAINER=ALL;" + + "\nalter user C##DBAPI_CDB_ADMIN identified by \\\"%[1]s\\\" account unlock;" + + "\nGRANT DBA TO C##DBAPI_CDB_ADMIN CONTAINER = ALL;" + + "\nGRANT SYSOPER TO C##DBAPI_CDB_ADMIN CONTAINER = ALL;" + + "\nGRANT PDB_DBA TO C##DBAPI_CDB_ADMIN CONTAINER = ALL;" + + "\nCREATE USER C##_DBAPI_PDB_ADMIN IDENTIFIED BY \\\"%[1]s\\\" CONTAINER=ALL ACCOUNT UNLOCK;" + + "\nalter user C##_DBAPI_PDB_ADMIN identified by \\\"%[1]s\\\" account unlock;" + + "\nGRANT DBA TO C##_DBAPI_PDB_ADMIN CONTAINER = ALL;" + +const GetUserOrdsSchemaStatusSQL string = "alter session set container=%[2]s;" + + "\nselect 'STATUS:'||status as status from ords_metadata.ords_schemas where upper(parsing_schema) = upper('%[1]s');" + +const EnableORDSSchemaSQL string = "\nALTER SESSION SET CONTAINER=%[5]s;" + + "\nCREATE USER %[1]s IDENTIFIED BY \\\"%[2]s\\\";" + + "\nGRANT CONNECT, RESOURCE, DBA, PDB_DBA TO %[1]s;" + + "\nCONN %[1]s/\\\"%[2]s\\\"@localhost:1521/%[5]s;" + + "\nBEGIN" + + "\nORDS.enable_schema(p_enabled => %[3]s ,p_schema => '%[1]s',p_url_mapping_type => 'BASE_PATH',p_url_mapping_pattern => '%[4]s',p_auto_rest_auth => FALSE);" + + "\nCOMMIT;" + + "\nEND;" + + "\n/" + + // SetupORDSCMD is run only for the FIRST TIME, ORDS is installed. Once ORDS is installed, we delete the pod that ran SetupORDSCMD and create new ones. + // Newly created pod doesn't run this SetupORDSCMD. +const SetupORDSCMD string = "$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property database.api.enabled true" + + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property jdbc.auth.enabled true" + + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property database.api.management.services.disabled false" + + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property database.api.admin.enabled true" + + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property dbc.auth.enabled true" + + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property restEnabledSql.active true" + + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property db.serviceNameSuffix \"\" " + // Mandatory when ORDS Installing at CDB Level -> Maps PDB's + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property jdbc.InitialLimit 5" + + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property jdbc.MaxLimit 20" + + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property jdbc.InactivityTimeout 300" + + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property feature.sdw true" + + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property security.verifySSL false" + + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property jdbc.maxRows 1000" + + "\numask 177" + + "\necho db.cdb.adminUser=C##DBAPI_CDB_ADMIN AS SYSOPER > cdbAdmin.properties" + + "\necho db.cdb.adminUser.password=\"%[4]s\" >> cdbAdmin.properties" + + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-properties --conf apex_pu cdbAdmin.properties" + + "\nrm -f cdbAdmin.properties" + + "\necho db.username=APEX_LISTENER > apexlistener" + + "\necho db.password=\"%[2]s\" >> apexlistener" + + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-properties --conf apex_al apexlistener" + + "\nrm -f apexlistener" + + "\necho db.username=APEX_REST_PUBLIC_USER > apexRestPublicUser" + + "\necho db.password=\"%[2]s\" >> apexRestPublicUser" + + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-properties --conf apex_rt apexRestPublicUser" + + "\nrm -f apexRestPublicUser" + + "\necho db.username=APEX_PUBLIC_USER > apexPublicUser" + + "\necho db.password=\"%[2]s\" >> apexPublicUser" + + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-properties --conf apex apexPublicUser" + + "\nrm -f apexPublicUser" + + "\necho db.adminUser=C##_DBAPI_PDB_ADMIN > pdbAdmin.properties" + + "\necho db.adminUser.password=\"%[4]s\">> pdbAdmin.properties" + + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-properties --conf apex_pu pdbAdmin.properties" + + "\nrm -f pdbAdmin.properties" + + "\necho -e \"%[1]s\n%[1]s\" > sqladmin.passwd" + + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war user ${ORDS_USER} \"SQL Administrator , System Administrator , SQL Developer , oracle.dbtools.autorest.any.schema \" < sqladmin.passwd" + + "\nrm -f sqladmin.passwd" + + "\numask 022" + + "\nsed -i 's,jetty.port=8888,jetty.secure.port=8443\\nssl.cert=\\nssl.cert.key=\\nssl.host=%[3]s,g' /opt/oracle/ords/config/ords/standalone/standalone.properties " + + "\nsed -i 's,standalone.static.path=/opt/oracle/ords/doc_root/i,standalone.static.path=/opt/oracle/ords/config/ords/apex/images,g' /opt/oracle/ords/config/ords/standalone/standalone.properties" + +const GetSessionInfoSQL string = "select s.sid || ',' || s.serial# as Info FROM v\\$session s, v\\$process p WHERE s.username = 'ORDS_PUBLIC_USER' AND p.addr(+) = s.paddr;" + +const KillSessionSQL string = "alter system kill session '%[1]s';" + +const DropAdminUsersSQL string = "drop user C##DBAPI_CDB_ADMIN;" + + "\ndrop user C##_DBAPI_PDB_ADMIN;" + +const UninstallORDSCMD string = "\numask 177" + + "\necho -e \"1\n${ORACLE_HOST}\n${ORACLE_PORT}\n1\n${ORACLE_SERVICE}\nsys\n%[1]s\n%[1]s\n1\" > ords.cred" + + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war uninstall advanced < ords.cred" + + "\nrm -f ords.cred" + + "\numask 022" + + "\nrm -f /opt/oracle/ords/config/ords/defaults.xml" + + "\nrm -f /opt/oracle/ords/config/ords/credentials" + + "\nrm -rf /opt/oracle/ords/config/ords/conf" + + "\nrm -rf /opt/oracle/ords/config/ords/standalone" + + "\nrm -rf /opt/oracle/ords/config/ords/apex" + +// To handle timing issue, checking if either of https://localhost:8443 or http://localhost:8888 is used for ORDS Installation +const GetORDSStatus string = " curl -sSkv -k -X GET https://localhost:8443/ords/_/db-api/stable/metadata-catalog/ || curl -sSkv -X GET http://localhost:8888/ords/_/db-api/stable/metadata-catalog/ " + +const ValidateAdminPassword string = "conn sys/%s@${ORACLE_SID} as sysdba\nshow user" + const ReconcileError string = "ReconcileError" const ReconcileErrorReason string = "LastReconcileCycleFailed" @@ -175,3 +370,54 @@ const IsApexInstalled string = "select 'APEXVERSION:'||version as version FROM D const UninstallApex string = "if [ -f /opt/oracle/oradata/${ORACLE_SID^^}/apex/apxremov.sql ]; then ( while true; do sleep 60; echo \"Uninstalling Apex...\" ; done ) & " + " cd /opt/oracle/oradata/${ORACLE_SID^^}/apex && echo -e \"@apxremov.sql\" | %[1]s && kill -9 $!; else echo \"Apex Folder doesn't exist\" ; fi ;" + +const ConfigureApexRest string = "if [ -f /opt/oracle/oradata/${ORACLE_SID^^}/apex/apex_rest_config.sql ]; then cd /opt/oracle/oradata/${ORACLE_SID^^}/apex && " + + "echo -e \"%[1]s\n%[1]s\" | %[2]s ; else echo \"Apex Folder doesn't exist\" ; fi ;" + +const AlterApexUsers string = "echo -e \" ALTER USER APEX_PUBLIC_USER IDENTIFIED BY \\\"%[1]s\\\" ACCOUNT UNLOCK; \n ALTER USER APEX_REST_PUBLIC_USER IDENTIFIED BY \\\"%[1]s\\\" ACCOUNT UNLOCK;" + + " \n ALTER USER APEX_LISTENER IDENTIFIED BY \\\"%[1]s\\\" ACCOUNT UNLOCK;\" | %[2]s" + +const CopyApexImages string = " ( while true; do sleep 60; echo \"Copying Apex Images...\" ; done ) & mkdir -p /opt/oracle/oradata/${ORACLE_SID^^}_ORDS/apex/images && " + + " cp -R /opt/oracle/oradata/${ORACLE_SID^^}/apex/images/* /opt/oracle/oradata/${ORACLE_SID^^}_ORDS/apex/images; chown -R oracle:oinstall /opt/oracle/oradata/${ORACLE_SID^^}_ORDS/apex; kill -9 $!;" + +const ApexAdmin string = "BEGIN" + + "\napex_util.set_security_group_id(p_security_group_id => 10); APEX_UTIL.REMOVE_USER(p_user_name => 'ADMIN');" + + "\nCOMMIT;" + + "\nEND;" + + "\n/" + + "\nBEGIN" + + "\nAPEX_UTIL.create_user(p_user_name => 'ADMIN',p_email_address => 'admin@oracle.com',p_web_password => '%[1]s',p_developer_privs => 'ADMIN',p_failed_access_attempts => '5' ," + + " p_allow_app_building_yn => 'Y' ,p_allow_sql_workshop_yn => 'Y' ,p_allow_websheet_dev_yn => 'Y' , p_allow_team_development_yn => 'Y' , p_change_password_on_first_use => 'N' );" + + "apex_util.unlock_account(p_user_name => 'ADMIN'); APEX_UTIL.set_security_group_id( null );" + + "\nCOMMIT;" + + "\nEND;" + + "\n/" + + "\nALTER SESSION SET CONTAINER=%[2]s;" + + "\nBEGIN" + + "\napex_util.set_security_group_id(p_security_group_id => 10); APEX_UTIL.REMOVE_USER(p_user_name => 'ADMIN');" + + "\nCOMMIT;" + + "\nEND;" + + "\n/" + + "\nBEGIN" + + "\nAPEX_UTIL.create_user(p_user_name => 'ADMIN',p_email_address => 'admin@oracle.com',p_web_password => '%[1]s',p_developer_privs => 'ADMIN',p_failed_access_attempts => '5' ," + + " p_allow_app_building_yn => 'Y' ,p_allow_sql_workshop_yn => 'Y' ,p_allow_websheet_dev_yn => 'Y' , p_allow_team_development_yn => 'Y' , p_change_password_on_first_use => 'N' );" + + "apex_util.unlock_account(p_user_name => 'ADMIN'); APEX_UTIL.set_security_group_id( null );" + + "\nCOMMIT;" + + "\nEND;" + + "\n/" + +// SetApexUsers is used to set Apex Users, pod that runs SetApexUsers is deleted and new ones is created. +const SetApexUsers string = "\numask 177" + + "\necho db.username=APEX_LISTENER > apexlistener" + + "\necho db.password=\"%[1]s\" >> apexlistener" + + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-properties --conf apex_al apexlistener" + + "\nrm -f apexlistener" + + "\necho db.username=APEX_REST_PUBLIC_USER > apexRestPublicUser" + + "\necho db.password=\"%[1]s\" >> apexRestPublicUser" + + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-properties --conf apex_rt apexRestPublicUser" + + "\nrm -f apexRestPublicUser" + + "\necho db.username=APEX_PUBLIC_USER > apexPublicUser" + + "\necho db.password=\"%[1]s\" >> apexPublicUser" + + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-properties --conf apex apexPublicUser" + + "\nrm -f apexPublicUser" + + "\numask 022" diff --git a/config/crd/bases/database.oracle.com_oraclerestdataservices.yaml b/config/crd/bases/database.oracle.com_oraclerestdataservices.yaml new file mode 100644 index 00000000..418b9370 --- /dev/null +++ b/config/crd/bases/database.oracle.com_oraclerestdataservices.yaml @@ -0,0 +1,202 @@ + +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.6.1 + creationTimestamp: null + name: oraclerestdataservices.database.oracle.com +spec: + group: database.oracle.com + names: + kind: OracleRestDataService + listKind: OracleRestDataServiceList + plural: oraclerestdataservices + singular: oraclerestdataservice + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .status.status + name: Status + type: string + - jsonPath: .spec.databaseRef + name: Database + type: string + - jsonPath: .status.databaseApiUrl + name: 'Database Api & Apex Url ' + type: string + - jsonPath: .status.databaseActionsUrl + name: Database Actions Url + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + description: OracleRestDataService is the Schema for the oraclerestdataservices + API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: OracleRestDataServiceSpec defines the desired state of OracleRestDataService + properties: + adminPassword: + description: OracleRestDataServicePassword defines the secret containing + Password mapped to secretKey + properties: + keepSecret: + type: boolean + secretKey: + type: string + secretName: + type: string + required: + - keepSecret + - secretKey + type: object + apexPassword: + description: OracleRestDataServicePassword defines the secret containing + Password mapped to secretKey + properties: + keepSecret: + type: boolean + secretKey: + type: string + secretName: + type: string + required: + - keepSecret + - secretKey + type: object + databaseRef: + type: string + image: + description: OracleRestDataServiceImage defines the Image source and + pullSecrets for POD + properties: + pullFrom: + type: string + pullSecrets: + type: string + version: + type: string + required: + - pullFrom + type: object + loadBalancer: + type: boolean + nodeSelector: + additionalProperties: + type: string + type: object + oracleService: + type: string + ordsPassword: + description: OracleRestDataServicePassword defines the secret containing + Password mapped to secretKey + properties: + keepSecret: + type: boolean + secretKey: + type: string + secretName: + type: string + required: + - keepSecret + - secretKey + type: object + ordsUser: + type: string + replicas: + minimum: 1 + type: integer + restEnableSchemas: + items: + description: OracleRestDataServicePDBSchemas defines the PDB Schemas + to be ORDS Enabled + properties: + enable: + type: boolean + pdb: + type: string + schema: + type: string + urlMapping: + type: string + required: + - enable + - pdb + - schema + type: object + type: array + required: + - adminPassword + - databaseRef + - ordsPassword + type: object + status: + description: OracleRestDataServiceStatus defines the observed state of + OracleRestDataService + properties: + apexConfigured: + type: boolean + clusterDbApiUrl: + type: string + commonUsersCreated: + type: boolean + databaseActionsUrl: + type: string + databaseApiUrl: + type: string + databaseRef: + type: string + image: + description: OracleRestDataServiceImage defines the Image source and + pullSecrets for POD + properties: + pullFrom: + type: string + pullSecrets: + type: string + version: + type: string + required: + - pullFrom + type: object + loadBalancer: + type: string + ordsInstalled: + type: boolean + ordsSetupCompleted: + type: boolean + replicas: + type: integer + serviceIP: + type: string + status: + description: 'INSERT ADDITIONAL STATUS FIELD - define observed state + of cluster Important: Run "make" to regenerate code after modifying + this file' + type: string + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] diff --git a/config/crd/kustomization.yaml b/config/crd/kustomization.yaml index 6b3d488e..c290b81d 100644 --- a/config/crd/kustomization.yaml +++ b/config/crd/kustomization.yaml @@ -9,6 +9,7 @@ resources: - bases/database.oracle.com_autonomousdatabases.yaml - bases/database.oracle.com_singleinstancedatabases.yaml - bases/database.oracle.com_shardingdatabases.yaml +- bases/database.oracle.com_oraclerestdataservices.yaml # +kubebuilder:scaffold:crdkustomizeresource patchesStrategicMerge: @@ -18,6 +19,7 @@ patchesStrategicMerge: #- patches/webhook_in_autonomousdatabases.yaml #- patches/webhook_in_singleinstancedatabases.yaml #- patches/webhook_in_shardingdatabases.yaml +#- patches/webhook_in_oraclerestdataservices.yaml # +kubebuilder:scaffold:crdkustomizewebhookpatch # [CERTMANAGER] To enable webhook, uncomment all the sections with [CERTMANAGER] prefix. @@ -26,6 +28,7 @@ patchesStrategicMerge: #- patches/cainjection_in_autonomousdatabases.yaml - patches/cainjection_in_singleinstancedatabases.yaml #- patches/cainjection_in_shardingdatabases.yaml +#- patches/cainjection_in_oraclerestdataservices.yaml # +kubebuilder:scaffold:crdkustomizecainjectionpatch # the following config is for teaching kustomize how to do kustomization for CRDs. diff --git a/config/crd/patches/cainjection_in_oraclerestdataservices.yaml b/config/crd/patches/cainjection_in_oraclerestdataservices.yaml new file mode 100644 index 00000000..d2b5d4ee --- /dev/null +++ b/config/crd/patches/cainjection_in_oraclerestdataservices.yaml @@ -0,0 +1,8 @@ +# The following patch adds a directive for certmanager to inject CA into the CRD +# CRD conversion requires k8s 1.13 or later. +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + annotations: + cert-manager.io/inject-ca-from: $(CERTIFICATE_NAMESPACE)/$(CERTIFICATE_NAME) + name: oraclerestdataservices.database.oracle.com diff --git a/config/crd/patches/webhook_in_oraclerestdataservices.yaml b/config/crd/patches/webhook_in_oraclerestdataservices.yaml new file mode 100644 index 00000000..c9398c23 --- /dev/null +++ b/config/crd/patches/webhook_in_oraclerestdataservices.yaml @@ -0,0 +1,17 @@ +# The following patch enables conversion webhook for CRD +# CRD conversion requires k8s 1.13 or later. +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: oraclerestdataservices.database.oracle.com +spec: + conversion: + strategy: Webhook + webhookClientConfig: + # this is "\n" used as a placeholder, otherwise it will be rejected by the apiserver for being blank, + # but we're going to set it later using the cert-manager (or potentially a patch if not using cert-manager) + caBundle: Cg== + service: + namespace: system + name: webhook-service + path: /convert diff --git a/config/manager/kustomization.yaml b/config/manager/kustomization.yaml index 30ed1f75..7258b286 100644 --- a/config/manager/kustomization.yaml +++ b/config/manager/kustomization.yaml @@ -9,4 +9,4 @@ kind: Kustomization images: - name: controller newName: container-registry.oracle.com/database/operator - newTag: 0.1.0 + newTag: 0.1.1 diff --git a/config/rbac/oraclerestdataservice_editor_role.yaml b/config/rbac/oraclerestdataservice_editor_role.yaml new file mode 100644 index 00000000..bf2b4d02 --- /dev/null +++ b/config/rbac/oraclerestdataservice_editor_role.yaml @@ -0,0 +1,24 @@ +# permissions for end users to edit oraclerestdataservices. +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: oraclerestdataservice-editor-role +rules: +- apiGroups: + - database.oracle.com + resources: + - oraclerestdataservices + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - database.oracle.com + resources: + - oraclerestdataservices/status + verbs: + - get diff --git a/config/rbac/oraclerestdataservice_viewer_role.yaml b/config/rbac/oraclerestdataservice_viewer_role.yaml new file mode 100644 index 00000000..a0a39cfd --- /dev/null +++ b/config/rbac/oraclerestdataservice_viewer_role.yaml @@ -0,0 +1,20 @@ +# permissions for end users to view oraclerestdataservices. +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: oraclerestdataservice-viewer-role +rules: +- apiGroups: + - database.oracle.com + resources: + - oraclerestdataservices + verbs: + - get + - list + - watch +- apiGroups: + - database.oracle.com + resources: + - oraclerestdataservices/status + verbs: + - get diff --git a/config/rbac/role.yaml b/config/rbac/role.yaml index 6d763212..e75a0f84 100644 --- a/config/rbac/role.yaml +++ b/config/rbac/role.yaml @@ -116,6 +116,32 @@ rules: verbs: - patch - update +- apiGroups: + - database.oracle.com + resources: + - oraclerestdataservices + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - database.oracle.com + resources: + - oraclerestdataservices/finalizers + verbs: + - update +- apiGroups: + - database.oracle.com + resources: + - oraclerestdataservices/status + verbs: + - get + - patch + - update - apiGroups: - database.oracle.com resources: diff --git a/config/samples/sidb/oraclerestdataservice.yaml b/config/samples/sidb/oraclerestdataservice.yaml new file mode 100644 index 00000000..f5d58a25 --- /dev/null +++ b/config/samples/sidb/oraclerestdataservice.yaml @@ -0,0 +1,53 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: OracleRestDataService +metadata: + name: oraclerestdataservice-sample +spec: + + ## Database ref. This can be of kind SingleInstanceDatabase. + databaseRef: "singleinstancedatabase-sample" + + ## Secret containing databaseRef password mapped to secretKey. + ## This secret will be deleted after ORDS Installation unless keepSecret set to true. + adminPassword: + secretName: + secretKey: + keepSecret: false + + ## Secret containing ORDS_PUBLIC_USER password mapped to secretKey. + ## This secret will be deleted after ORDS Installation unless keepSecret set to true. + ordsPassword: + secretName: + secretKey: + keepSecret: false + + ## Secret containing a common password for APEX_PUBLIC_USER, APEX_REST_PUBLIC_USER, APEX_LISTENER and Apex administrator (username: ADMIN) mapped to secretKey + ## Mention a non-null string for apexPassword.secretName to configure APEX with ORDS. + ## This secret will be deleted after ORDS Installation unless keepSecret set to true. + apexPassword: + secretName: + secretKey: + keepSecret: false + + ## ORDS image details + ## Add 'database.api.enabled=true' entry to below file while building ORDS image. + ## https://github.com/oracle/docker-images/blob/main/OracleRestDataServices/dockerfiles/ords_params.properties.tmpl + image: + pullFrom: + pullSecrets: + + ## Type of service Applicable on cloud enviroments only. + ## if loadBalService: false, service type = "NodePort". else "LoadBalancer" + loadBalancer: false + + ## PDB Schemas to be ORDS Enabled. + ## Schema will be created (if not exists) with username as schema and password as .spec.ordsPassword. + restEnableSchemas: + - schema: + enable: true + urlMapping: + pdb: diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index ac140361..3e8b0ed7 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -41,6 +41,12 @@ spec: ## Enable/Disable ForceLogging forceLog: false + ## apex-latest.zip file should be present in the location '/opt/oracle/oradata' + ## Download using `wget https://download.oracle.com/otn_software/apex/apex-latest.zip` + ## use `kubectl cp :/opt/oracle/oradata + ## Deploy OracleRestDataService CR to front APEX + installApex: false + ## NA if cloning from a SourceDB (cloneFrom is set) ## Specify both sgaSize and pgaSize (in MB) or dont specify both ## Specify Non-Zero value to use diff --git a/config/webhook/manifests.yaml b/config/webhook/manifests.yaml index 5cb37b30..d9deb4d3 100644 --- a/config/webhook/manifests.yaml +++ b/config/webhook/manifests.yaml @@ -6,6 +6,27 @@ metadata: creationTimestamp: null name: mutating-webhook-configuration webhooks: +- admissionReviewVersions: + - v1 + - v1beta1 + clientConfig: + service: + name: webhook-service + namespace: system + path: /mutate-database-oracle-com-v1alpha1-oraclerestdataservice + failurePolicy: Fail + name: moraclerestdataservice.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + resources: + - oraclerestdataservices + sideEffects: None - admissionReviewVersions: - v1 - v1beta1 @@ -35,6 +56,27 @@ metadata: creationTimestamp: null name: validating-webhook-configuration webhooks: +- admissionReviewVersions: + - v1 + - v1beta1 + clientConfig: + service: + name: webhook-service + namespace: system + path: /validate-database-oracle-com-v1alpha1-oraclerestdataservice + failurePolicy: Fail + name: voraclerestdataservice.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + resources: + - oraclerestdataservices + sideEffects: None - admissionReviewVersions: - v1 - v1beta1 diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go new file mode 100644 index 00000000..0759f13e --- /dev/null +++ b/controllers/database/oraclerestdataservice_controller.go @@ -0,0 +1,1298 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package controllers + +import ( + "context" + "errors" + "fmt" + "strconv" + "strings" + "time" + + corev1 "k8s.io/api/core/v1" + apierrors "k8s.io/apimachinery/pkg/api/errors" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/types" + "k8s.io/client-go/rest" + "k8s.io/client-go/tools/record" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/controller" + "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" + "sigs.k8s.io/controller-runtime/pkg/log" + + dbapi "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + dbcommons "github.com/oracle/oracle-database-operator/commons/database" + + "github.com/go-logr/logr" +) + +const oracleRestDataServiceFinalizer = "database.oracle.com/oraclerestdataservicefinalizer" + +// OracleRestDataServiceReconciler reconciles a OracleRestDataService object +type OracleRestDataServiceReconciler struct { + client.Client + Log logr.Logger + Scheme *runtime.Scheme + Config *rest.Config + Recorder record.EventRecorder +} + +//+kubebuilder:rbac:groups=database.oracle.com,resources=oraclerestdataservices,verbs=get;list;watch;create;update;patch;delete +//+kubebuilder:rbac:groups=database.oracle.com,resources=oraclerestdataservices/status,verbs=get;update;patch +//+kubebuilder:rbac:groups=database.oracle.com,resources=oraclerestdataservices/finalizers,verbs=update +//+kubebuilder:rbac:groups="",resources=pods;pods/log;pods/exec;persistentvolumeclaims;services;nodes;events,verbs=create;delete;get;list;patch;update;watch + +// Reconcile is part of the main kubernetes reconciliation loop which aims to +// move the current state of the cluster closer to the desired state. +// TODO(user): Modify the Reconcile function to compare the state specified by +// the OracleRestDataService object against the actual cluster state, and then +// perform operations to make the cluster state reflect the state specified by +// the user. +// +// For more details, check Reconcile and its Result here: +// - https://pkg.go.dev/sigs.k8s.io/controller-runtime@v0.8.3/pkg/reconcile +func (r *OracleRestDataServiceReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + _ = log.FromContext(ctx) + + oracleRestDataService := &dbapi.OracleRestDataService{} + err := r.Get(ctx, types.NamespacedName{Namespace: req.Namespace, Name: req.Name}, oracleRestDataService) + if err != nil { + if apierrors.IsNotFound(err) { + r.Log.Info("Resource deleted") + return requeueN, nil + } + return requeueN, err + } + + // Fetch Primary Database Reference + singleInstanceDatabase := &dbapi.SingleInstanceDatabase{} + err = r.Get(ctx, types.NamespacedName{Namespace: req.Namespace, Name: oracleRestDataService.Spec.DatabaseRef}, singleInstanceDatabase) + if err != nil { + if apierrors.IsNotFound(err) { + r.Log.Info("Resource deleted") + return requeueN, nil + } + return requeueN, err + } + + // Manage SingleInstanceDatabase Deletion + result := r.manageOracleRestDataServiceDeletion(req, ctx, oracleRestDataService, singleInstanceDatabase) + if result.Requeue { + r.Log.Info("Reconcile queued") + return result, nil + } + + // First validate + result, err = r.validate(oracleRestDataService, singleInstanceDatabase) + if result.Requeue { + r.Log.Info("Spec validation failed, Reconcile queued") + return result, nil + } + if err != nil { + r.Log.Info("Spec validation failed") + return result, nil + } + + // Always refresh status before a reconcile + defer r.Status().Update(ctx, oracleRestDataService) + + // Create Service + result = r.createSVC(ctx, req, oracleRestDataService) + if result.Requeue { + r.Log.Info("Reconcile queued") + return result, nil + } + + // Validate if Primary Database Reference is ready + result, sidbReadyPod := r.validateSidbReadiness(oracleRestDataService, singleInstanceDatabase, ctx, req) + if result.Requeue { + r.Log.Info("Reconcile queued") + return result, nil + } + + // Configure Apex + result = r.configureApex(oracleRestDataService, singleInstanceDatabase, sidbReadyPod, ctx, req) + if result.Requeue { + r.Log.Info("Reconcile queued") + return result, nil + } + + // Create ORDS Pods + result = r.createPods(oracleRestDataService, singleInstanceDatabase, sidbReadyPod, ctx, req) + if result.Requeue { + r.Log.Info("Reconcile queued") + return result, nil + } + + result = r.setupORDS(oracleRestDataService, singleInstanceDatabase, sidbReadyPod, ctx, req) + if result.Requeue { + r.Log.Info("Reconcile queued") + return result, nil + } + + result = r.checkHealthStatus(oracleRestDataService, ctx, req) + if result.Requeue { + r.Log.Info("Reconcile queued") + return result, nil + } + + // Delete Secrets + r.deleteSecrets(oracleRestDataService, ctx, req) + + if oracleRestDataService.Status.ServiceIP == "" { + return requeueY, nil + } + + return ctrl.Result{}, nil +} + +//############################################################################# +// Validate the CRD specs +//############################################################################# +func (r *OracleRestDataServiceReconciler) validate(m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase) (ctrl.Result, error) { + + var err error + eventReason := "Spec Error" + var eventMsgs []string + // If Block Volume , Ensure Replicas=1 + if n.Spec.Persistence.AccessMode == "ReadWriteOnce" { + eventMsgs = append(eventMsgs, "ords can be installed only on ReadWriteMany Access Mode of : "+m.Spec.DatabaseRef) + } + if m.Status.DatabaseRef != "" && m.Status.DatabaseRef != m.Spec.DatabaseRef { + eventMsgs = append(eventMsgs, "databaseRef cannot be updated") + } + if m.Status.LoadBalancer != "" && m.Status.LoadBalancer != strconv.FormatBool(m.Spec.LoadBalancer) { + eventMsgs = append(eventMsgs, "service patching is not avaiable currently") + } + if !n.Status.ApexInstalled && m.Spec.ApexPassword.SecretName != "" { + eventMsgs = append(eventMsgs, "apex is not installed yet") + } + if m.Status.Image.PullFrom != "" && m.Status.Image != m.Spec.Image { + eventMsgs = append(eventMsgs, "image patching is not avaiable currently") + } + + m.Status.DatabaseRef = m.Spec.DatabaseRef + m.Status.LoadBalancer = strconv.FormatBool(m.Spec.LoadBalancer) + m.Status.Image = m.Spec.Image + + if len(eventMsgs) > 0 { + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, strings.Join(eventMsgs, ",")) + r.Log.Info(strings.Join(eventMsgs, "\n")) + err = errors.New(strings.Join(eventMsgs, ",")) + return requeueN, err + } + + return requeueN, err +} + +//##################################################################################################### +// Validate Readiness of the primary DB specified +//##################################################################################################### +func (r *OracleRestDataServiceReconciler) validateSidbReadiness(m *dbapi.OracleRestDataService, + n *dbapi.SingleInstanceDatabase, ctx context.Context, req ctrl.Request) (ctrl.Result, corev1.Pod) { + + log := r.Log.WithValues("validateSidbReadiness", req.NamespacedName) + + // ## FETCH THE SIDB REPLICAS . + sidbReadyPod, _, _, _, err := dbcommons.FindPods(r, n.Spec.Image.Version, + n.Spec.Image.PullFrom, n.Name, n.Namespace, ctx, req) + if err != nil { + log.Error(err, err.Error()) + return requeueY, sidbReadyPod + } + + if sidbReadyPod.Name == "" || n.Status.Status != dbcommons.StatusReady { + eventReason := "Waiting" + eventMsg := "waiting for " + n.Name + " to be Ready" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + return requeueY, sidbReadyPod + } + if m.Status.OrdsInstalled || m.Status.CommonUsersCreated { + return requeueN, sidbReadyPod + } + + // Validate databaseRef Admin Password + adminPasswordSecret := &corev1.Secret{} + err = r.Get(ctx, types.NamespacedName{Name: m.Spec.AdminPassword.SecretName, Namespace: m.Namespace}, adminPasswordSecret) + if err != nil { + if apierrors.IsNotFound(err) { + m.Status.Status = dbcommons.StatusError + eventReason := "Waiting" + eventMsg := "waiting for secret : " + m.Spec.AdminPassword.SecretName + " to get created" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + r.Log.Info("Secret " + m.Spec.AdminPassword.SecretName + " Not Found") + return requeueY, sidbReadyPod + } + log.Error(err, err.Error()) + return requeueY, sidbReadyPod + } + adminPassword := string(adminPasswordSecret.Data[m.Spec.AdminPassword.SecretKey]) + + out, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | %s", fmt.Sprintf(dbcommons.ValidateAdminPassword, adminPassword), dbcommons.GetSqlClient(n.Spec.Edition))) + if err != nil { + log.Error(err, err.Error()) + return requeueY, sidbReadyPod + } + if strings.Contains(out, "USER is \"SYS\"") { + log.Info("validated Admin password successfully") + } else if strings.Contains(out, "ORA-01017") { + m.Status.Status = dbcommons.StatusError + eventReason := "Logon denied" + eventMsg := "invalid databaseRef admin password. secret: " + m.Spec.AdminPassword.SecretName + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) + return requeueY, sidbReadyPod + } else { + return requeueY, sidbReadyPod + } + + return requeueN, sidbReadyPod +} + +//##################################################################################################### +// Check ORDS Health Status +//##################################################################################################### +func (r *OracleRestDataServiceReconciler) checkHealthStatus(m *dbapi.OracleRestDataService, + ctx context.Context, req ctrl.Request) ctrl.Result { + log := r.Log.WithValues("checkHealthStatus", req.NamespacedName) + + readyPod, _, _, _, err := dbcommons.FindPods(r, m.Spec.Image.Version, + m.Spec.Image.PullFrom, m.Name, m.Namespace, ctx, req) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + if readyPod.Name == "" { + return requeueY + } + + // Get ORDS Status + out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", + dbcommons.GetORDSStatus) + log.Info("GetORDSStatus Output") + log.Info(out) + if strings.Contains(strings.ToUpper(out), "ERROR") { + return requeueY + } + if err != nil { + log.Info(err.Error()) + if strings.Contains(strings.ToUpper(err.Error()), "ERROR") { + return requeueY + } + } + + if strings.Contains(out, "HTTP/1.1 200 OK") || strings.Contains(strings.ToUpper(err.Error()), "HTTP/1.1 200 OK") { + m.Status.Status = dbcommons.StatusReady + } else { + m.Status.Status = dbcommons.StatusNotReady + return requeueY + } + return requeueN +} + +//############################################################################# +// Instantiate Service spec from OracleRestDataService spec +//############################################################################# +func (r *OracleRestDataServiceReconciler) instantiateSVCSpec(m *dbapi.OracleRestDataService) *corev1.Service { + svc := &corev1.Service{ + TypeMeta: metav1.TypeMeta{ + Kind: "Service", + }, + ObjectMeta: metav1.ObjectMeta{ + Name: m.Name, + Namespace: m.Namespace, + Labels: map[string]string{ + "app": m.Name, + }, + }, + Spec: corev1.ServiceSpec{ + Ports: []corev1.ServicePort{ + { + Name: "client", + Port: 8443, + Protocol: corev1.ProtocolTCP, + }, + }, + Selector: map[string]string{ + "app": m.Name, + }, + Type: corev1.ServiceType(func() string { + if m.Spec.LoadBalancer { + return "LoadBalancer" + } + return "NodePort" + }()), + }, + } + // Set StandbyDatabase instance as the owner and controller + ctrl.SetControllerReference(m, svc, r.Scheme) + return svc +} + +//############################################################################# +// Instantiate POD spec from OracleRestDataService spec +//############################################################################# +func (r *OracleRestDataServiceReconciler) instantiatePodSpec(m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase, ordsInstalled bool) *corev1.Pod { + + pod := &corev1.Pod{ + TypeMeta: metav1.TypeMeta{ + Kind: "Pod", + }, + ObjectMeta: metav1.ObjectMeta{ + Name: m.Name + "-" + dbcommons.GenerateRandomString(5), + Namespace: m.Namespace, + Labels: map[string]string{ + "app": m.Name, + "version": m.Spec.Image.Version, + }, + }, + Spec: corev1.PodSpec{ + Volumes: []corev1.Volume{{ + Name: "datamount", + VolumeSource: corev1.VolumeSource{ + PersistentVolumeClaim: &corev1.PersistentVolumeClaimVolumeSource{ + ClaimName: n.Name, + ReadOnly: false, + }, + }, + }}, + InitContainers: []corev1.Container{{ + Name: "init-permissions", + Image: m.Spec.Image.PullFrom, + Command: []string{"/bin/sh", "-c", fmt.Sprintf("chown %d:%d /opt/oracle/ords/config/ords", int(dbcommons.ORACLE_UID), int(dbcommons.DBA_GUID))}, + SecurityContext: &corev1.SecurityContext{ + // User ID 0 means, root user + RunAsUser: func() *int64 { i := int64(0); return &i }(), + }, + VolumeMounts: []corev1.VolumeMount{{ + MountPath: "/opt/oracle/ords/config/ords", + Name: "datamount", + SubPath: strings.ToUpper(n.Spec.Sid) + "_ORDS", + }}, + }}, + Containers: []corev1.Container{{ + Name: m.Name, + Image: m.Spec.Image.PullFrom, + ImagePullPolicy: corev1.PullAlways, + Ports: []corev1.ContainerPort{{ContainerPort: 8443}}, + VolumeMounts: []corev1.VolumeMount{{ + MountPath: "/opt/oracle/ords/config/ords/", + Name: "datamount", + SubPath: strings.ToUpper(n.Spec.Sid) + "_ORDS", + }}, + Env: func() []corev1.EnvVar { + // ORDS_PWD, ORACLE_PWD is required only during the First ORDS Installation. + if !ordsInstalled { + return []corev1.EnvVar{ + { + Name: "ORACLE_HOST", + Value: n.Name, + }, + { + Name: "ORACLE_PORT", + Value: "1521", + }, + { + Name: "ORACLE_SERVICE", + Value: func() string { + if m.Spec.OracleService != "" { + return m.Spec.OracleService + } + return n.Spec.Sid + }(), + }, + { + Name: "ORDS_USER", + Value: func() string { + if m.Spec.OrdsUser != "" { + return m.Spec.OrdsUser + } + return "ORDS_PUBLIC_USER" + }(), + }, + { + Name: "ORDS_PWD", + ValueFrom: &corev1.EnvVarSource{ + SecretKeyRef: &corev1.SecretKeySelector{ + LocalObjectReference: corev1.LocalObjectReference{ + Name: m.Spec.OrdsPassword.SecretName, + }, + Key: m.Spec.OrdsPassword.SecretKey, + }, + }, + }, + { + Name: "ORACLE_PWD", + ValueFrom: &corev1.EnvVarSource{ + SecretKeyRef: &corev1.SecretKeySelector{ + LocalObjectReference: corev1.LocalObjectReference{ + Name: m.Spec.AdminPassword.SecretName, + }, + Key: m.Spec.AdminPassword.SecretKey, + }, + }, + }, + } + } + // After ORDS is Installed, we DELETE THE OLD ORDS Pod and create new ones ONLY USING BELOW ENV VARIABLES. + return []corev1.EnvVar{ + { + Name: "ORACLE_HOST", + Value: n.Name, + }, + { + Name: "ORACLE_PORT", + Value: "1521", + }, + { + Name: "ORACLE_SERVICE", + Value: func() string { + if m.Spec.OracleService != "" { + return m.Spec.OracleService + } + return n.Spec.Sid + }(), + }, + { + Name: "ORDS_USER", + Value: func() string { + if m.Spec.OrdsUser != "" { + return m.Spec.OrdsUser + } + return "ORDS_PUBLIC_USER" + }(), + }, + } + }(), + }}, + + TerminationGracePeriodSeconds: func() *int64 { i := int64(30); return &i }(), + + NodeSelector: func() map[string]string { + ns := make(map[string]string) + if len(m.Spec.NodeSelector) != 0 { + for key, value := range m.Spec.NodeSelector { + ns[key] = value + } + } + return ns + }(), + + SecurityContext: &corev1.PodSecurityContext{ + RunAsUser: func() *int64 { i := int64(dbcommons.ORACLE_UID); return &i }(), + RunAsGroup: func() *int64 { i := int64(dbcommons.DBA_GUID); return &i }(), + }, + + ImagePullSecrets: []corev1.LocalObjectReference{ + { + Name: m.Spec.Image.PullSecrets, + }, + }, + }, + } + + // Set oracleRestDataService instance as the owner and controller + ctrl.SetControllerReference(m, pod, r.Scheme) + return pod +} + +//############################################################################# +// Create a Service for OracleRestDataService +//############################################################################# +func (r *OracleRestDataServiceReconciler) createSVC(ctx context.Context, req ctrl.Request, + m *dbapi.OracleRestDataService) ctrl.Result { + + log := r.Log.WithValues("createSVC", req.NamespacedName) + // Check if the Service already exists, if not create a new one + svc := &corev1.Service{} + // Get retrieves an obj for the given object key from the Kubernetes Cluster. + // obj must be a struct pointer so that obj can be updated with the response returned by the Server. + // Here foundsvc is the struct pointer to corev1.Service{} + err := r.Get(ctx, types.NamespacedName{Name: m.Name, Namespace: m.Namespace}, svc) + if err != nil && apierrors.IsNotFound(err) { + // Define a new Service + svc = r.instantiateSVCSpec(m) + log.Info("Creating a new Service", "Service.Namespace", svc.Namespace, "Service.Name", svc.Name) + err = r.Create(ctx, svc) + if err != nil { + log.Error(err, "Failed to create new Service", "Service.Namespace", svc.Namespace, "Service.Name", svc.Name) + return requeueY + } else { + log.Info("Succesfully Created New Service ", "Service.Name : ", svc.Name) + } + + } else if err != nil { + log.Error(err, "Failed to get Service") + return requeueY + } else if err == nil { + log.Info("Found Existing Service ", "Service.Name", svc.Name) + } + + m.Status.ServiceIP = "" + if m.Spec.LoadBalancer { + if len(svc.Status.LoadBalancer.Ingress) > 0 { + m.Status.DatabaseApiUrl = "https://" + svc.Status.LoadBalancer.Ingress[0].IP + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/ords/" + m.Status.ServiceIP = svc.Status.LoadBalancer.Ingress[0].IP + m.Status.DatabaseActionsUrl = "https://" + svc.Status.LoadBalancer.Ingress[0].IP + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/ords/sql-developer" + } + m.Status.ClusterDbApiUrl = "https://" + svc.Name + "." + svc.Namespace + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/ords/" + return requeueN + } + m.Status.ClusterDbApiUrl = "https://" + svc.Name + "." + svc.Namespace + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/ords/" + nodeip := dbcommons.GetNodeIp(r, ctx, req) + if nodeip != "" { + m.Status.ServiceIP = nodeip + m.Status.DatabaseApiUrl = "https://" + nodeip + ":" + fmt.Sprint(svc.Spec.Ports[0].NodePort) + "/ords/" + m.Status.DatabaseActionsUrl = "https://" + nodeip + ":" + fmt.Sprint(svc.Spec.Ports[0].NodePort) + "/ords/sql-developer" + } + return requeueN +} + +//############################################################################# +// Create the requested POD replicas +//############################################################################# +func (r *OracleRestDataServiceReconciler) createPods(m *dbapi.OracleRestDataService, + n *dbapi.SingleInstanceDatabase, sidbReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) ctrl.Result { + + log := r.Log.WithValues("createPods", req.NamespacedName) + + readyPod, replicasFound, available, _, err := dbcommons.FindPods(r, m.Spec.Image.Version, + m.Spec.Image.PullFrom, m.Name, m.Namespace, ctx, req) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + + log.Info(m.Name, " pods other than one of Ready Pods : ", dbcommons.GetPodNames(available)) + log.Info(m.Name, " Ready Pod : ", readyPod.Name) + + replicasReq := m.Spec.Replicas + if replicasFound == 0 { + m.Status.Status = dbcommons.StatusNotReady + m.Status.DatabaseApiUrl = dbcommons.ValueUnavailable + m.Status.DatabaseActionsUrl = dbcommons.ValueUnavailable + } + + if replicasFound == replicasReq { + log.Info("No of " + m.Name + " replicas Found are same as Required") + } else if replicasFound < replicasReq { + // Create New Pods , Name of Pods are generated Randomly + for i := replicasFound; i < replicasReq; i++ { + if i > 0 && !m.Status.OrdsInstalled { + // Wait till the first created pod to installs ORDS . + break + } + ordsInstalled := false + if m.Status.OrdsInstalled { + ordsInstalled = true + } + pod := r.instantiatePodSpec(m, n, ordsInstalled) + log.Info("Creating a new "+m.Name+" POD", "POD.Namespace", pod.Namespace, "POD.Name", pod.Name) + err := r.Create(ctx, pod) + if err != nil { + log.Error(err, "Failed to create new "+m.Name+" POD", "pod.Namespace", pod.Namespace, "POD.Name", pod.Name) + return requeueY + } + log.Info("Succesfully Created new "+m.Name+" POD", "POD.NAME : ", pod.Name) + } + } else { + // Delete extra pods + noDeleted := 0 + if readyPod.Name != "" { + available = append(available, readyPod) + } + for _, pod := range available { + if readyPod.Name == pod.Name { + continue + } + if replicasReq == (len(available) - noDeleted) { + break + } + r.Log.Info("Deleting Pod : ", "POD.NAME", pod.Name) + err := r.Delete(ctx, &pod, &client.DeleteOptions{}) + noDeleted += 1 + if err != nil { + r.Log.Error(err, "Failed to delete existing POD", "POD.Name", pod.Name) + // Don't requeue + } + } + } + n.Status.OrdsReference = m.Name + r.Status().Update(ctx, n) + m.Status.Replicas = m.Spec.Replicas + if !m.Status.OrdsInstalled { + m.Status.Replicas = 1 + } + return requeueN +} + +//############################################################################# +// Manage Finalizer to cleanup before deletion of OracleRestDataService +//############################################################################# +func (r *OracleRestDataServiceReconciler) manageOracleRestDataServiceDeletion(req ctrl.Request, ctx context.Context, + m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase) ctrl.Result { + log := r.Log.WithValues("manageOracleRestDataServiceDeletion", req.NamespacedName) + + // Check if the OracleRestDataService instance is marked to be deleted, which is + // indicated by the deletion timestamp being set. + isOracleRestDataServiceMarkedToBeDeleted := m.GetDeletionTimestamp() != nil + if isOracleRestDataServiceMarkedToBeDeleted { + if controllerutil.ContainsFinalizer(m, oracleRestDataServiceFinalizer) { + // Run finalization logic for oracleRestDataServiceFinalizer. If the + // finalization logic fails, don't remove the finalizer so + // that we can retry during the next reconciliation. + if err := r.cleanupOracleRestDataService(req, ctx, m, n); err != nil { + log.Error(err, err.Error()) + return requeueY + } + + n.Status.OrdsReference = "" + // Make sure n.Status.OrdsInstalled is set to false or else it blocks .spec.databaseRef deletion + for i := 0; i < 10; i++ { + err := r.Status().Update(ctx, n) + if err != nil { + log.Info(err.Error() + "\n updating n.Status.OrdsInstalled = false") + time.Sleep(5 * time.Second) + continue + } + break + } + + // Remove oracleRestDataServiceFinalizer. Once all finalizers have been + // removed, the object will be deleted. + controllerutil.RemoveFinalizer(m, oracleRestDataServiceFinalizer) + err := r.Update(ctx, m) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + } + return requeueY + } + + // Add finalizer for this CR + if !controllerutil.ContainsFinalizer(m, oracleRestDataServiceFinalizer) { + controllerutil.AddFinalizer(m, oracleRestDataServiceFinalizer) + err := r.Update(ctx, m) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + } + return requeueN +} + +//############################################################################# +// Finalization logic for OracleRestDataServiceFinalizer +//############################################################################# +func (r *OracleRestDataServiceReconciler) cleanupOracleRestDataService(req ctrl.Request, ctx context.Context, + m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase) error { + log := r.Log.WithValues("cleanupOracleRestDataService", req.NamespacedName) + + readyPod, _, _, _, err := dbcommons.FindPods(r, m.Spec.Image.Version, + m.Spec.Image.PullFrom, m.Name, m.Namespace, ctx, req) + if err != nil { + log.Error(err, err.Error()) + return err + } + + if m.Status.OrdsInstalled && readyPod.Name != "" { + // ## FETCH THE SIDB REPLICAS . + sidbReadyPod, _, _, _, err := dbcommons.FindPods(r, n.Spec.Image.Version, + n.Spec.Image.PullFrom, n.Name, n.Namespace, ctx, req) + if err != nil { + log.Error(err, err.Error()) + return err + } + + if sidbReadyPod.Name == "" { + eventReason := "No Ready Pod" + eventMsg := "ommitting ORDS uninstallation as No Ready Pod of " + n.Name + " available" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + return nil + } + + // Get Session id , serial# for ORDS_PUBLIC_USER to kill the sessions + out, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | %s ", dbcommons.GetSessionInfoSQL, dbcommons.GetSqlClient(n.Spec.Edition))) + if err != nil { + log.Error(err, err.Error()) + return err + } + log.Info("GetSessionInfoSQL Output : " + out) + + sessionInfos, _ := dbcommons.StringToLines(out) + killSessions := "" + for _, sessionInfo := range sessionInfos { + if !strings.Contains(sessionInfo, ",") { + // May be a column name or (-----) + continue + } + killSessions += "\n" + fmt.Sprintf(dbcommons.KillSessionSQL, sessionInfo) + } + + //kill all the sessions with given sid,serial# + out, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | %s ", killSessions, dbcommons.GetSqlClient(n.Spec.Edition))) + + if err != nil { + log.Error(err, err.Error()) + return err + } + log.Info("KillSession Output : " + out) + + if readyPod.Name == "" { + eventReason := "Waiting" + eventMsg := "waiting for " + m.Name + " to be Ready" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + err = errors.New(eventMsg) + return err + } + + // Fetch admin Password of database to uninstall ORDS + adminPasswordSecret := &corev1.Secret{} + adminPasswordSecretFound := false + for i := 0; i < 5; i++ { + err := r.Get(ctx, types.NamespacedName{Name: m.Spec.AdminPassword.SecretName, Namespace: n.Namespace}, adminPasswordSecret) + if err != nil { + log.Error(err, err.Error()) + if apierrors.IsNotFound(err) { + eventReason := "Waiting" + eventMsg := "waiting for admin password secret : " + m.Spec.AdminPassword.SecretName + " to get created" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + r.Log.Info(eventMsg) + if i < 4 { + time.Sleep(15 * time.Second) + continue + } + } + } else { + adminPasswordSecretFound = true + break + } + } + if !adminPasswordSecretFound { + log.Info("AdminPassword Secret not found . Omitting OracleRestDataService uninstallation") + return nil + } + + adminPassword := string(adminPasswordSecret.Data[m.Spec.AdminPassword.SecretKey]) + uninstallORDS := fmt.Sprintf(dbcommons.UninstallORDSCMD, adminPassword) + + out, err = dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, true, "bash", "-c", + fmt.Sprintf(uninstallORDS)) + log.Info("UninstallORDSCMD Output : " + out) + if strings.Contains(strings.ToUpper(out), "ERROR") { + return errors.New(out) + } + if err != nil { + log.Info(err.Error()) + if strings.Contains(strings.ToUpper(err.Error()), "ERROR") { + return err + } + } + + log.Info("UninstallORDSCMD Output : " + out) + + //Delete Database Admin Password Secret + if !m.Spec.AdminPassword.KeepSecret { + err = r.Delete(ctx, adminPasswordSecret, &client.DeleteOptions{}) + if err == nil { + r.Log.Info("Deleted Admin Password Secret :" + adminPasswordSecret.Name) + } + } + + // Drop Admin Users + out, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | %s ", dbcommons.DropAdminUsersSQL, dbcommons.GetSqlClient(n.Spec.Edition))) + if err != nil { + log.Error(err, err.Error()) + return err + } + log.Info("DropAdminUsersSQL Output : " + out) + + } + + // Cleanup steps that the operator needs to do before the CR can be deleted. + log.Info("Successfully cleaned up OracleRestDataService ") + return nil +} + +//############################################################################# +// Configure APEX +//############################################################################# +func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase, + sidbReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) ctrl.Result { + log := r.Log.WithValues("configureApex", req.NamespacedName) + + if m.Spec.ApexPassword.SecretName == "" { + m.Status.ApexConfigured = false + return requeueN + } + if m.Status.ApexConfigured { + return requeueN + } + if !n.Status.ApexInstalled { + m.Status.Status = dbcommons.StatusError + eventReason := "Failed" + eventMsg := "apex is not installed yet on " + m.Spec.DatabaseRef + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) + return requeueY + } + + apexPasswordSecret := &corev1.Secret{} + err := r.Get(ctx, types.NamespacedName{Name: m.Spec.ApexPassword.SecretName, Namespace: m.Namespace}, apexPasswordSecret) + if err != nil { + if apierrors.IsNotFound(err) { + m.Status.Status = dbcommons.StatusError + eventReason := "Waiting" + eventMsg := "waiting for secret : " + m.Spec.ApexPassword.SecretName + " to get created" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + r.Log.Info("Secret " + m.Spec.ApexPassword.SecretName + " Not Found") + return requeueY + } + log.Error(err, err.Error()) + return requeueY + } + // APEX_LISTENER , APEX_REST_PUBLIC_USER , APEX_PUBLIC_USER passwords + apexPassword := string(apexPasswordSecret.Data[m.Spec.ApexPassword.SecretKey]) + + m.Status.Status = dbcommons.StatusUpdating + r.Status().Update(ctx, m) + + configureApexRestSqlClient := "sqlplus -s / as sysdba @apex_rest_config.sql" + if n.Spec.Edition == "express" { + configureApexRestSqlClient = "su -p oracle -c \"sqlplus -s / as sysdba @apex_rest_config.sql;\"" + } + + // Configure APEX + out, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", + fmt.Sprintf(dbcommons.ConfigureApexRest, apexPassword, configureApexRestSqlClient)) + if err != nil { + log.Info(err.Error()) + if !strings.Contains(err.Error(), "catcon.pl: completed") { + return requeueY + } + } + log.Info(" ConfigureApexRest Output : \n" + out) + + if strings.Contains(out, "Apex Folder doesn't exist") { + eventReason := "Waiting" + eventMsg := "apex Folder doesn't exist in the location /opt/oracle/oradata/" + strings.ToUpper(n.Spec.Sid) + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) + return requeueY + } + + // Alter APEX_LISTENER,APEX_PUBLIC_USER,APEX_REST_PUBLIC_USER + out, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, + "bash", "-c", fmt.Sprintf(dbcommons.AlterApexUsers, apexPassword, dbcommons.GetSqlClient(n.Spec.Edition))) + if err != nil { + log.Info(err.Error()) + return requeueY + } + log.Info(" AlterApexUsers Output : \n" + out) + + pdbName := "orclpdb1" + if n.Spec.Pdbname != "" { + pdbName = n.Spec.Pdbname + } + + // Change APEX Admin Password + out, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, + "bash", "-c", fmt.Sprintf("echo -e \"%s\" | %s", fmt.Sprintf(dbcommons.ApexAdmin, apexPassword, pdbName), dbcommons.GetSqlClient(n.Spec.Edition))) + if err != nil { + log.Info(err.Error()) + return requeueY + } + log.Info("Change ApexAdmin Password Output : \n" + out) + + m.Status.ApexConfigured = true + r.Status().Update(ctx, m) + + // Copy APEX Images + out, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf(dbcommons.CopyApexImages)) + if err != nil { + log.Info(err.Error()) + return requeueY + } + log.Info(" CopyApexImages Output : \n" + out) + + if m.Status.OrdsInstalled { + + readyPod, _, available, _, err := dbcommons.FindPods(r, m.Spec.Image.Version, + m.Spec.Image.PullFrom, m.Name, m.Namespace, ctx, req) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + if readyPod.Name != "" { + log.Info("Ready Pod marked for deletion") + available = append(available, readyPod) + } + + // Set Apex users in apex_rt,apex_al,apex files + out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, true, "bash", "-c", + fmt.Sprintf(dbcommons.SetApexUsers, apexPassword)) + log.Info("SetApexUsers Output: \n" + out) + if strings.Contains(strings.ToUpper(out), "ERROR") { + return requeueY + } + if err != nil { + log.Info(err.Error()) + if strings.Contains(strings.ToUpper(err.Error()), "ERROR") { + return requeueY + } + } + + for _, pod := range available { + // ORDS Needs to be restarted to configure APEX + err = r.Delete(ctx, &pod, &client.DeleteOptions{}) + if err != nil { + r.Log.Error(err, "Failed to delete existing POD", "POD.Name", pod.Name) + return requeueY + } + r.Log.Info("ORDS Pod Deleted : " + pod.Name) + } + + return requeueY + } + + return requeueN +} + +//############################################################################# +// Delete Secrets +//############################################################################# +func (r *OracleRestDataServiceReconciler) deleteSecrets(m *dbapi.OracleRestDataService, ctx context.Context, req ctrl.Request) { + log := r.Log.WithValues("deleteSecrets", req.NamespacedName) + + if !m.Spec.AdminPassword.KeepSecret { + // Fetch adminPassword Secret + adminPasswordSecret := &corev1.Secret{} + err := r.Get(ctx, types.NamespacedName{Name: m.Spec.AdminPassword.SecretName, Namespace: m.Namespace}, adminPasswordSecret) + if err == nil { + //Delete Database Admin Password Secret . + err := r.Delete(ctx, adminPasswordSecret, &client.DeleteOptions{}) + if err == nil { + log.Info("Database Admin Password secret Deleted : " + adminPasswordSecret.Name) + } + } + } + + if !m.Spec.OrdsPassword.KeepSecret { + // Fetch ordsPassword Secret + ordsPasswordSecret := &corev1.Secret{} + err := r.Get(ctx, types.NamespacedName{Name: m.Spec.OrdsPassword.SecretName, Namespace: m.Namespace}, ordsPasswordSecret) + if err == nil { + //Delete ORDS Password Secret . + err := r.Delete(ctx, ordsPasswordSecret, &client.DeleteOptions{}) + if err == nil { + log.Info("ORDS Password secret Deleted : " + ordsPasswordSecret.Name) + } + } + } + + if !m.Spec.ApexPassword.KeepSecret { + // Fetch apexPassword Secret + apexPasswordSecret := &corev1.Secret{} + err := r.Get(ctx, types.NamespacedName{Name: m.Spec.ApexPassword.SecretName, Namespace: m.Namespace}, apexPasswordSecret) + if err == nil { + //Delete APEX Password Secret . + err := r.Delete(ctx, apexPasswordSecret, &client.DeleteOptions{}) + if err == nil { + log.Info("APEX Password secret Deleted : " + apexPasswordSecret.Name) + } + } + } + +} + +//############################################################################# +// Setup ORDS in CDB , enable ORDS for PDBs Specified +//############################################################################# +func (r *OracleRestDataServiceReconciler) setupORDS(m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase, + sidbReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) ctrl.Result { + + log := r.Log.WithValues("setupORDS", req.NamespacedName) + + readyPod, replicasFound, _, _, err := dbcommons.FindPods(r, m.Spec.Image.Version, + m.Spec.Image.PullFrom, m.Name, m.Namespace, ctx, req) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + if readyPod.Name == "" { + eventReason := "Waiting" + eventMsg := "waiting for " + m.Name + " to be Ready" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + return requeueY + } + OrdsPasswordSecret := &corev1.Secret{} + if !m.Status.OrdsInstalled { + err := r.Get(ctx, types.NamespacedName{Name: m.Spec.OrdsPassword.SecretName, Namespace: m.Namespace}, OrdsPasswordSecret) + if err != nil { + if apierrors.IsNotFound(err) { + m.Status.Status = dbcommons.StatusError + eventReason := "Waiting" + eventMsg := "waiting for secret : " + m.Spec.OrdsPassword.SecretName + " to get created" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + r.Log.Info("Secret " + m.Spec.OrdsPassword.SecretName + " Not Found") + return requeueY + } + log.Error(err, err.Error()) + return requeueY + } + ordsPassword := string(OrdsPasswordSecret.Data[m.Spec.OrdsPassword.SecretKey]) + + // APEX_LISTENER , APEX_REST_PUBLIC_USER , APEX_PUBLIC_USER , APEX ADMIN passwords + apexPassword := ordsPassword + if m.Spec.ApexPassword.SecretName != "" { + apexPasswordSecret := &corev1.Secret{} + err = r.Get(ctx, types.NamespacedName{Name: m.Spec.ApexPassword.SecretName, Namespace: m.Namespace}, apexPasswordSecret) + if err != nil { + if apierrors.IsNotFound(err) { + m.Status.Status = dbcommons.StatusError + eventReason := "Waiting" + eventMsg := "waiting for secret : " + m.Spec.ApexPassword.SecretName + " to get created" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + r.Log.Info("Secret " + m.Spec.ApexPassword.SecretName + " Not Found") + return requeueY + } + log.Error(err, err.Error()) + return requeueY + } + apexPassword = string(apexPasswordSecret.Data[m.Spec.ApexPassword.SecretKey]) + } + + // Get ORDS Status + out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", + dbcommons.GetORDSStatus) + log.Info("GetORDSStatus Output") + log.Info(out) + if strings.Contains(strings.ToUpper(out), "ERROR") { + return requeueY + } + if err != nil { + log.Info(err.Error()) + if strings.Contains(strings.ToUpper(err.Error()), "ERROR") { + return requeueY + } + } + + if strings.Contains(out, "HTTP/1.1 200 OK") || strings.Contains(strings.ToUpper(err.Error()), "HTTP/1.1 200 OK") { + + if !m.Status.OrdsSetupCompleted { + cdbAdminPassword := dbcommons.GenerateRandomString(8) + // Create PDB , CDB Admin users and grant permissions . ORDS installation on CDB level + out, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | %s", fmt.Sprintf(dbcommons.SetAdminUsersSQL, cdbAdminPassword), dbcommons.GetSqlClient(n.Spec.Edition))) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + log.Info("SetAdminUsers Output :\n" + out) + + if !strings.Contains(out, "ERROR") || !strings.Contains(out, "ORA-") || + strings.Contains(out, "ERROR") && strings.Contains(out, "ORA-01920") { + m.Status.CommonUsersCreated = true + } + + // Setup ORDS + out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, true, "bash", "-c", + fmt.Sprintf(dbcommons.SetupORDSCMD, ordsPassword, apexPassword, m.Name, cdbAdminPassword)) + log.Info("SetupORDSCMD Output") + log.Info(out) + if strings.Contains(strings.ToUpper(out), "ERROR") { + return requeueY + } + if err != nil { + log.Info(err.Error()) + if strings.Contains(strings.ToUpper(err.Error()), "ERROR") { + return requeueY + } + } + + if !strings.Contains(strings.ToUpper(out), "ERROR") && !strings.Contains(strings.ToUpper(err.Error()), "ERROR") && + !strings.Contains(strings.ToUpper(out), "could not execute") && !strings.Contains(strings.ToUpper(err.Error()), "could not execute") { + m.Status.OrdsSetupCompleted = true + } + } + + // ORDS Needs to be restarted to ensure ORDS Installation works fine + err := r.Delete(ctx, &readyPod, &client.DeleteOptions{}) + if err != nil { + r.Log.Error(err, "Failed to delete existing POD", "POD.Name", readyPod.Name) + return requeueY + } + r.Log.Info("ORDS Installation Pod Deleted : " + readyPod.Name) + + eventReason := "ORDS Installed" + eventMsg := "" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + m.Status.OrdsInstalled = true + r.Status().Update(ctx, m) + + // Wait for 15 sec to update OrdsInstalled to true + time.Sleep(15 * time.Second) + + } + return requeueY + } + + result := r.restEnableSchemas(m, n, sidbReadyPod, ctx, req) + if result.Requeue { + r.Log.Info("Reconcile queued") + return result + } + + if replicasFound < m.Spec.Replicas { + // Requeue if all replicas not created + return requeueY + } + return requeueN +} + +//############################################################################# +// Rest Enable/Disable Schemas +//############################################################################# +func (r *OracleRestDataServiceReconciler) restEnableSchemas(m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase, + sidbReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) ctrl.Result { + + log := r.Log.WithValues("restEnableSchemas", req.NamespacedName) + + // Get Pdbs Available + availablePDBS, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", + ctx, req, false, "bash", "-c", fmt.Sprintf("echo -e \"%s\" | %s", dbcommons.GetPdbsSQL, dbcommons.GetSqlClient(n.Spec.Edition))) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } else { + log.Info("GetPdbsSQL Output") + log.Info(availablePDBS) + } + + for i := 0; i < len(m.Spec.RestEnableSchemas); i++ { + + // If the PDB mentioned in yaml doesnt contain in the database , continue + if !strings.Contains(strings.ToUpper(availablePDBS), strings.ToUpper(m.Spec.RestEnableSchemas[i].Pdb)) { + eventReason := "Warning" + eventMsg := "enabling ORDS schema for PDB : " + m.Spec.RestEnableSchemas[i].Pdb + " failed ; as pdb not found" + log.Info(eventMsg) + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + continue + } + + getOrdsSchemaStatus := fmt.Sprintf(dbcommons.GetUserOrdsSchemaStatusSQL, m.Spec.RestEnableSchemas[i].Schema, m.Spec.RestEnableSchemas[i].Pdb) + + // Get ORDS Schema status for PDB + out, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | %s", getOrdsSchemaStatus, dbcommons.GetSqlClient(n.Spec.Edition))) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } else { + log.Info("getOrdsSchemaStatus Output") + log.Info(out) + } + + // if ORDS already enabled for given PDB + if strings.Contains(out, "STATUS:ENABLED") && m.Spec.RestEnableSchemas[i].Enable { + continue + } + + // if ORDS already disabled for given PDB + if !strings.Contains(out, "STATUS:ENABLED") && !m.Spec.RestEnableSchemas[i].Enable { + continue + } + + OrdsPasswordSecret := &corev1.Secret{} + // Fetch the secret to get password for database user . Secret has to be created in the same namespace of OracleRestDataService + err = r.Get(ctx, types.NamespacedName{Name: m.Spec.OrdsPassword.SecretName, Namespace: m.Namespace}, OrdsPasswordSecret) + if err != nil { + if apierrors.IsNotFound(err) { + eventReason := "No Secret" + eventMsg := "secret " + m.Spec.OrdsPassword.SecretName + " Not Found" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + r.Log.Info(eventMsg) + return requeueY + } + log.Error(err, err.Error()) + return requeueY + } + + password := string(OrdsPasswordSecret.Data[m.Spec.OrdsPassword.SecretKey]) + urlMappingPattern := "" + if m.Spec.RestEnableSchemas[i].UrlMapping == "" { + urlMappingPattern = strings.ToLower(m.Spec.RestEnableSchemas[i].Schema) + } else { + urlMappingPattern = strings.ToLower(m.Spec.RestEnableSchemas[i].UrlMapping) + } + enableORDSSchema := fmt.Sprintf(dbcommons.EnableORDSSchemaSQL, strings.ToUpper(m.Spec.RestEnableSchemas[i].Schema), password, + strconv.FormatBool(m.Spec.RestEnableSchemas[i].Enable), urlMappingPattern, m.Spec.RestEnableSchemas[i].Pdb) + + // Create users,schemas and grant enableORDS for PDB + out, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | %s", enableORDSSchema, dbcommons.GetSqlClient(n.Spec.Edition))) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + log.Info("getOrdsSchemaStatus Output : " + out) + + } + + return requeueN + +} + +//############################################################################# +// SetupWithManager sets up the controller with the Manager. +//############################################################################# +func (r *OracleRestDataServiceReconciler) SetupWithManager(mgr ctrl.Manager) error { + return ctrl.NewControllerManagedBy(mgr). + For(&dbapi.OracleRestDataService{}). + Owns(&corev1.Pod{}). //Watch for deleted pods of OracleRestDataService Owner + WithEventFilter(dbcommons.ResourceEventHandler()). + WithOptions(controller.Options{MaxConcurrentReconciles: 100}). //ReconcileHandler is never invoked concurrently with the same object. + Complete(r) +} diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index c69ba5b3..c5d8366a 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -215,6 +215,20 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct } } + // Install Apex + result = r.installApex(singleInstanceDatabase, readyPod, ctx, req) + if result.Requeue { + r.Log.Info("Reconcile queued") + return result, nil + } + + // Uninstall Apex + result = r.uninstallApex(singleInstanceDatabase, readyPod, ctx, req) + if result.Requeue { + r.Log.Info("Reconcile queued") + return result, nil + } + // If LoadBalancer = true , ensure Connect String is updated if singleInstanceDatabase.Status.ConnectString == dbcommons.ValueUnavailable { return requeueY, nil @@ -222,6 +236,7 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct // update status to Ready after all operations succeed singleInstanceDatabase.Status.Status = dbcommons.StatusReady + r.updateORDSStatus(singleInstanceDatabase, false, ctx, req) completed = true r.Log.Info("Reconcile completed") @@ -1254,6 +1269,7 @@ func (r *SingleInstanceDatabaseReconciler) validateDBReadiness(m *dbapi.SingleIn if m.Spec.Edition == "express" { eventReason = "Database Unhealthy" m.Status.Status = dbcommons.StatusNotReady + r.updateORDSStatus(m, false, ctx, req) } out, err := dbcommons.ExecCommand(r, r.Config, runningPod.Name, runningPod.Namespace, "", ctx, req, false, "bash", "-c", dbcommons.GetCheckpointFileCMD) @@ -1268,6 +1284,7 @@ func (r *SingleInstanceDatabaseReconciler) validateDBReadiness(m *dbapi.SingleIn eventMsg = "datafiles exists" m.Status.DatafilesCreated = "true" m.Status.Status = dbcommons.StatusNotReady + r.updateORDSStatus(m, false, ctx, req) } } @@ -1656,6 +1673,163 @@ func (r *SingleInstanceDatabaseReconciler) updateDBConfig(m *dbapi.SingleInstanc return requeueN, nil } +//############################################################################# +// Install APEX to CDB +//############################################################################# +func (r *SingleInstanceDatabaseReconciler) installApex(m *dbapi.SingleInstanceDatabase, + readyPod corev1.Pod, ctx context.Context, req ctrl.Request) ctrl.Result { + log := r.Log.WithValues("installApex", req.NamespacedName) + + if !m.Spec.InstallApex || m.Status.ApexInstalled { + return requeueN + } + + m.Status.Status = dbcommons.StatusUpdating + r.Status().Update(ctx, m) + eventReason := "Installing Apex" + eventMsg := "Waiting for Apex Installation to complete" + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) + + unzipApex := dbcommons.UnzipApex + if m.Spec.Edition == "express" { + unzipApex += dbcommons.ChownApex + } + // Unzip /opt/oracle/oradata/apex-latest.zip to /opt/oracle/oradata/${ORACLE_SID^^} + out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", + unzipApex) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + log.Info(" UnzipApex Output : \n" + out) + + if strings.Contains(out, "apex-latest.zip not found") { + eventReason := "Waiting" + eventMsg := "apex-latest.zip doesn't exist in the location /opt/oracle/oradata/" + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) + return requeueY + } + + // Install APEX + out, err = dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf(dbcommons.InstallApex, dbcommons.GetSqlClient(m.Spec.Edition))) + if err != nil { + log.Info(err.Error()) + } + log.Info(" InstallApex Output : \n" + out) + + if strings.Contains(out, "Apex Folder doesn't exist") { + eventReason := "Waiting" + eventMsg := "apex Folder doesn't exist in the location /opt/oracle/oradata/" + strings.ToUpper(m.Spec.Sid) + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) + return requeueY + } + + out, err = dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | %s", dbcommons.IsApexInstalled, dbcommons.GetSqlClient(m.Spec.Edition))) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + log.Info("IsApexInstalled Output: \n" + out) + + apexInstalled := "APEXVERSION:" + if !strings.Contains(out, apexInstalled) { + return requeueY + } + + m.Status.ApexInstalled = true + // Make sure m.Status.ApexInstalled is set to true . + for i := 0; i < 10; i++ { + err = r.Status().Update(ctx, m) + if err != nil { + log.Info(err.Error() + "\n updating m.Status.ApexInstalled = true") + time.Sleep(5 * time.Second) + continue + } + break + } + + return requeueN +} + +//############################################################################# +// Uninstall APEX from CDB +//############################################################################# +func (r *SingleInstanceDatabaseReconciler) uninstallApex(m *dbapi.SingleInstanceDatabase, + readyPod corev1.Pod, ctx context.Context, req ctrl.Request) ctrl.Result { + log := r.Log.WithValues("uninstallApex", req.NamespacedName) + + if m.Spec.InstallApex || !m.Status.ApexInstalled { + return requeueN + } + + m.Status.Status = dbcommons.StatusUpdating + r.Status().Update(ctx, m) + eventReason := "Uninstalling Apex" + eventMsg := "Waiting for Apex Uninstallation to complete" + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) + + // Uninstall APEX + out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf(dbcommons.UninstallApex, dbcommons.GetSqlClient(m.Spec.Edition))) + if err != nil { + log.Info(err.Error()) + if !strings.Contains(err.Error(), "catcon.pl: completed") { + return requeueY + } + } + log.Info(" UninstallApex Output : \n" + out) + + if strings.Contains(out, "Apex Folder doesn't exist") { + eventReason := "Waiting" + eventMsg := "apex Folder doesn't exist in the location /opt/oracle/oradata/" + strings.ToUpper(m.Spec.Sid) + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) + return requeueY + } + + m.Status.ApexInstalled = false + // Make sure m.Status.ApexInstalled is set to false. + for i := 0; i < 10; i++ { + err = r.Status().Update(ctx, m) + if err != nil { + log.Info(err.Error() + "\n updating m.Status.ApexInstalled = false") + time.Sleep(5 * time.Second) + continue + } + break + } + r.updateORDSStatus(m, true, ctx, req) + return requeueN +} + +//############################################################################# +// Update ORDS Status +//############################################################################# +func (r *SingleInstanceDatabaseReconciler) updateORDSStatus(m *dbapi.SingleInstanceDatabase, updateApexStatus bool, ctx context.Context, req ctrl.Request) { + + if m.Status.OrdsReference == "" { + return + } + n := &dbapi.OracleRestDataService{} + err := r.Get(ctx, types.NamespacedName{Namespace: req.Namespace, Name: m.Status.OrdsReference}, n) + if err != nil { + return + } + if updateApexStatus && n.Status.ApexConfigured && !m.Status.ApexInstalled { + n.Status.ApexConfigured = false + r.Status().Update(ctx, n) + n.Spec.ApexPassword.SecretName = "" + r.Update(ctx, n) + } + if !updateApexStatus && n.Status.OrdsInstalled { + // Update Status to Healthy/Unhealthy when SIDB turns Healthy/Unhealthy after ORDS is Installed + n.Status.Status = m.Status.Status + r.Status().Update(ctx, n) + return + } +} + //############################################################################# // Manage Finalizer to cleanup before deletion of SingleInstanceDatabase //############################################################################# diff --git a/controllers/database/suite_test.go b/controllers/database/suite_test.go index 9b62324a..fdebedce 100644 --- a/controllers/database/suite_test.go +++ b/controllers/database/suite_test.go @@ -87,6 +87,9 @@ var _ = BeforeSuite(func(done Done) { err = databasev1alpha1.AddToScheme(scheme.Scheme) Expect(err).NotTo(HaveOccurred()) + err = databasev1alpha1.AddToScheme(scheme.Scheme) + Expect(err).NotTo(HaveOccurred()) + // +kubebuilder:scaffold:scheme k8sClient, err = client.New(cfg, client.Options{Scheme: scheme.Scheme}) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 168a2395..535d37d1 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -261,3 +261,345 @@ $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.status}" 19.3.0.0.0 (29517242) ``` +## Kind OracleRestDataService + +The Oracle Database Operator creates the OracleRestDataService (ORDS) kind as a custom resource that enables RESTful API access to the Oracle Database in K8s + +* ### OracleRestDataService Sample YAML + + For the use cases detailed below a sample .yaml file is available at + [config/samples/sidb/oraclerestdataservice.yaml](config/samples/sidb/oraclerestdataservice.yaml) + + **Note:** The `adminPassword` , `ordsPassword` fields of the above `oraclerestdataservice.yaml` yaml contains secrets for authenticating Single Instance Database and for ORDS user with roles `SQL Administrator , System Administrator , SQL Developer , oracle.dbtools.autorest.any.schema` respectively . These secrets gets delete after the first deployed ORDS pod REST enables the database successfully for security reasons. + +* ### List OracleRestDataServices + + ```sh + $ kubectl get oraclerestdataservice -o name + + oraclerestdataservice.database.oracle.com/ords-sample + + ``` + +* ### Quick Status + + ```sh + $ kubectl get oraclerestdataservice ords-sample + + NAME STATUS DATABASE DATABASE API URL DATABASE ACTIONS URL + ords-sample Healthy sidb-sample https://144.25.121.118:8443/ords/ https://144.25.121.118:8443/ords/sql-developer + ``` + +* ### Detailed Status + + ```sh + $ kubectl describe oraclerestdataservice ords-sample + + Name: ords-sample + Namespace: default + Labels: + Annotations: + API Version: database.oracle.com/v1alpha1 + Kind: OracleRestDataService + Metadata: ... + Spec: ... + Status: + Cluster Db API URL: https://ords21c-1.default:8443/ords/ + Database Actions URL: https://144.25.121.118:8443/ords/sql-developer + Database API URL: https://144.25.121.118:8443/ords/ + Database Ref: sidb21c-1 + Image: + Pull From: ... + Pull Secrets: ... + Load Balancer: true + Ords Installed: true + Persistence: + Access Mode: ReadWriteMany + Size: 100Gi + Storage Class: + Service IP: 144.25.121.118 + Status: Healthy + + ``` + +## REST Enable Database + + Provision a new ORDS instance by specifying appropriate values for the attributes in the the sample .yaml file and executing the following command . ORDS is installed in the root container(CDB) of SingleInstanceDatabase to enable PDB Lifecycle Management . + + ```sh + $ kubectl create -f oraclerestdataservice.yaml + + oraclerestdataservice.database.oracle.com/ords-sample created + ``` + +* ### Creation Status + + Creating a new ORDS instance takes a while. ORDS is open for connections when the 'status' status returns a "Healthy" + + ```sh + $ kubectl get oraclerestdataservice/ords-sample --template={{.status.status}} + + Healthy + ``` + +* ### REST Endpoints + + External and internal (running in Kubernetes pods) clients can access the REST Endpoints using .status.databaseApiUrl and .status.clusterDbApiUrl respectively in the following command . + + ```sh + $ kubectl get oraclerestdataservice/ords-sample --template={{.status.databaseApiUrl}} + + https://144.25.121.118:8443/ords/ + ``` + + All the REST Endpoints can be found at + + There are two basic approaches for authentication to the REST Endpoints. Certain APIs are specific about which authentication method they will accept. + +* #### Default Administrator + + ORDS User with role "SQL Administrator" , `.spec.ordsUser` (defaults to ORDS_PUBLIC_USER if not mentioned in yaml) credentials are required to call certain REST Endpoints . + + This user has also given the additional roles `System Administrator , SQL Developer , oracle.dbtools.autorest.any.schema` . + + This user can now be used to authenticate + * PDB Lifecycle Management APIs + * Any Protected AutoRest Enabled Object APIs + * Database Actions of any REST Enabled Schema + +* #### ORDS Enabled Schema + + Alternatively one can use an ORDS enabled schema. Access to the certain APIs will use the credentials of the ORDS enabled schema , which are defined in the `.spec.restEnableSchemas` atrribute in sample yaml . + + This schema authentication can be used to authorise database actions of this schema + + Note : Browser may not prompt for credentials while accessing certain REST Endpoints and in such case one can use clients like curl and pass credentials while calling REST Endpoints . + +* #### Some Usecases + +* ##### PDB Lifecycle Management + + The Oracle REST Data Services (ORDS) database API allows us to manage the lifecycle of PDBs via REST web service calls. + Few APIs : + * List PDB's + + ```sh + `$ curl -s -k -X GET -u 'ORDS_PUBLIC_USER:<.spec.ordsPassword>' https://144.25.121.118:8443/ords/_/db-api/stable/database/pdbs/ | python -m json.tool` + ``` + + * Create PDB + + Create a file called "pdbsample.json" with the following contents. + + ```sh + { "method": "CREATE", + "adminName": "pdbsample_admin", + "adminPwd": "", + "pdb_name": "pdbsample", + "fileNameConversions": "('/opt/oracle/oradata/<.spec.databaseRef.sid>/pdbseed/', '/opt/oracle/oradata/<.spec.databaseRef.sid>/pdbsample/')", + "reuseTempFile": true, + "totalSize": "10G", + "tempSize": "100M", + "getScript": false + } + ``` + + Execute the follwing API to run the above json script to create pdb. + + ```sh + $ curl -k --request POST --url https://144.25.121.118:8443/ords/_/db-api/latest/database/pdbs/ \ + --user 'ORDS_PUBLIC_USER:<.spec.ordsPassword>' \ + --header 'content-type: application/json' \ + --data @pdbsample.json + ``` + + * Open/Close PDB + + ```sh + $ curl -k --request POST --url https://144.25.121.118:8443/ords/_/db-api/latest/database/pdbs/pdbsample/status \ + --user 'ORDS_PUBLIC_USER:<.spec.ordsPassword>'\ + --header 'content-type: application/json' \ + --data ' { + "state": "OPEN/CLOSE", + "modifyOption": "NORMAL" + } ' + ``` + + * Clone PDB + + Create a file called "pdbclone.json" with the following contents. + + ```sh + { + "method": "CLONE", + "clonePDBName": "pdbclone", + "fileNameConversions": "('/opt/oracle/oradata/<.spec.databaseRef.sid>/pdbsample/', '/opt/oracle/oradata/<.spec.databaseRef.sid>/pdbclone/')", + "unlimitedStorage": true, + "reuseTempFile": true, + "totalSize": "UNLIMITED", + "tempSize": "UNLIMITED" + } + ``` + + Execute the follwing API to run the above json script to clone pdb from pdbsample. + + ```sh + $ curl -k --request POST --url https://144.25.121.118:8443/ords/_/db-api/latest/database/pdbs/pdbsample/ \ + --user 'ORDS_PUBLIC_USER:<.spec.ordsPassword>' \ + --header 'content-type: application/json' \ + --data @pdbclone.json + ``` + + More REST APIs for PDB Lifecycle Management can be found at [https://docs.oracle.com/en/database/oracle/oracle-database/21/dbrst/api-pluggable-database-lifecycle-management.html](https://docs.oracle.com/en/database/oracle/oracle-database/21/dbrst/api-pluggable-database-lifecycle-management.html) + +* #### REST Enabled SQL + + The REST Enabled SQL functionality allows REST calls to send DML, DDL and scripts to any REST enabled schema by exposing the same SQL engine used in SQL Developer and SQLcl. + + * Run a Script + + Create a file called "/tmp/table.sql" with the following contents. + + ```sh + CREATE TABLE DEPT ( + DEPTNO NUMBER(2) CONSTRAINT PK_DEPT PRIMARY KEY, + DNAME VARCHAR2(14), + LOC VARCHAR2(13) + ) ; + + INSERT INTO DEPT VALUES (10,'ACCOUNTING','NEW YORK'); + INSERT INTO DEPT VALUES (20,'RESEARCH','DALLAS'); + INSERT INTO DEPT VALUES (30,'SALES','CHICAGO'); + INSERT INTO DEPT VALUES (40,'OPERATIONS','BOSTON'); + COMMIT; + ``` + + Execute the follwing API to run the above script. + + ```sh + curl -s -k -X "POST" "https://144.25.121.118:8443/ords/<.spec.restEnableSchemas[].pdb>/<.spec.restEnableSchemas[].urlMapping>/_/sql" \ + -H "Content-Type: application/sql" \ + -u '<.spec.restEnableSchemas[].schema>:<.spec.ordsPassword>' \ + -d @/tmp/table.sql + ``` + + * Basic Call + + Fetch all entries from 'DEPT' table by calling the following API + + ```sh + curl -s -k -X "POST" "https://144.25.121.118:8443/ords/<.spec.restEnableSchemas[].pdb>/<.spec.restEnableSchemas[].urlMapping>/_/sql" \ + -H "Content-Type: application/sql" \ + -u '<.spec.restEnableSchemas[].schema>:<.spec.ordsPassword>' \ + -d $'select * from dept;' | python -m json.tool + ``` + + **NOTE:** `.spec.restEnableSchema[].urlMapping` is optional and is defaulted to `.spec.restEnableSchema[].schema` + +* #### Data Pump + + The Oracle REST Data Services (ORDS) database API allows user to create Data Pump export and import jobs via REST web service calls. + + REST APIs for Data Pump Jobs can be found at [https://docs.oracle.com/en/database/oracle/oracle-database/21/dbrst/op-database-datapump-jobs-post.html](https://docs.oracle.com/en/database/oracle/oracle-database/21/dbrst/op-database-datapump-jobs-post.html) + +* ### Database Actions + + Database Actions is a web-based interface that uses Oracle REST Data Services to provide development, data tools, administration and monitoring features for Oracle Database . + + * To use Database Actions, one must sign in as a database user whose schema has been REST-enabled. + * This can be done by specifying appropriate values for the `.spec.restEnableSchemas` attributes details in the sample yaml [config/samples/sidb/oraclerestdataservice.yaml](config/samples/sidb/oraclerestdataservice.yaml) which are needed for authorising Database Actions. + * Schema will be created (if not exists) with username as `.spec.restEnableSchema[].schema` and password as `.spec.ordsPassword.`. + * UrlMapping `.spec.restEnableSchema[].urlMapping` is optional and is defaulted to `.spec.restEnableSchema[].schema`. + + Database Actions can be accessed via browser using `.status.databaseActionsUrl` in the following command + + ```sh + $ kubectl get oraclerestdataservice/ords-sample --template={{.status.databaseActionsUrl}} + + https://144.25.121.118:8443/ords/sql-developer + ``` + + Sign in to Database Actions using + * First Page: \ + PDB Name: `.spec.restEnableSchema[].pdb` \ + Username: `.spec.restEnableSchema[].urlMapping` + + * Second Page: \ + Username: `.spec.restEnableSchema[].schema` \ + Password: `.spec.ordsPassword` + + ![database-actions-home](/images/sidb/database-actions-home.png) + + More info on Database Actions can be found at + +* ### Application Express + + Oracle Application Express (APEX) is a low-code development platform that enables developers to build scalable, secure enterprise apps, with world-class features, that can be deployed anywhere. + + Using APEX, developers can quickly develop and deploy compelling apps that solve real problems and provide immediate value. Developers won't need to be an expert in a vast array of technologies to deliver sophisticated solutions. Focus on solving the problem and let APEX take care of the rest. + + **Download APEX:** + * Download latest version of apex using + * Copy apex-latest.zip file to the location '/opt/oracle/oradata' of SingleInstanceDatabase pod . + * Use `kubectl cp :/opt/oracle/oradata` to copy apex-latest.zip + + **Install APEX:** + * Set `.spec.installApex` to true in [config/samples/sidb/singleinstancedatabase.yaml](config/samples/sidb/singleinstancedatabase.yaml) + * Status of SIDB turns to 'Updating' during apex installation and turns 'Healthy' after successful installation. You can also check status using below cmd + + ```sh + $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath=[{.status.apexInstalled}]" + + [true] + ``` + + * To access APEX , You need to configure APEX with ORDS + + **Configure APEX with ORDS:** + * Set `.spec.apexPassword.secretName` to a non-null string in [config/samples/sidb/oraclerestdataservice.yaml](config/samples/sidb/oraclerestdataservice.yaml) + * This is used as a common password for APEX_PUBLIC_USER, APEX_REST_PUBLIC_USER, APEX_LISTENER and Apex administrator (username: ADMIN) mapped to secretKey + * Status of ORDS turns to 'Updating' during apex configuration and turns 'Healthy' after successful configuration. You can also check status using below cmd + + ```sh + $ kubectl get oraclerestdataservice ords-sample -o "jsonpath=[{.status.apexConfigured}]" + + [true] + ``` + + * If you configure APEX after ORDS is installed, ORDS pods will be deleted and recreated + + Application Express can be accessed via browser using `.status.databaseApiUrl` in the following command .\ + + ```sh + $ kubectl get oraclerestdataservice/ords-sample --template={{.status.databaseApiUrl}} + + https://144.25.121.118:8888/ords/ + ``` + + Sign in to Administration servies using \ + workspace: `INTERNAL` \ + username: `ADMIN` \ + password: `.spec.apexPassword` + + ![application-express-admin-home](/images/sidb/application-express-admin-home.png) + + **NOTE** + * Apex Administrator for pdbs other than `.spec.databaseRef.pdbName` has to be created Manually + + More Info on creating Apex Administrator can be found at [APEX_UTIL.CREATE_USER] + + **Uninstall APEX:** + * Set `.spec.installApex` to false in [config/samples/sidb/singleinstancedatabase.yaml](config/samples/sidb/singleinstancedatabase.yaml) + * If you install APEX again, re-configure APEX with ORDS again. + + More info on Application Express can be found at + +* ### Multiple Replicas + + Currently only single replica mode is supported + +* ### Patch Attributes + + The following attributes cannot be patched post SingleInstanceDatabase instance Creation : databaseRef, loadBalancer, image, ordsPassword, adminPassword, apexPassword. + + * A schema can be rest enabled or disabled by setting the `.spec.restEnableSchemas[].enable` to true or false respectively in ords sample .yaml file and apply using the kubectl apply command or edit/patch commands. This requires `.spec.ordsPassword` secret. diff --git a/images/sidb/application-express-admin-home.png b/images/sidb/application-express-admin-home.png new file mode 100644 index 0000000000000000000000000000000000000000..6c08bbbcc936e77b417a7520bd944105f627f4a1 GIT binary patch literal 143392 zcma&O1yG#Jwl++F1P_wnE(y+{!6CT2yF<{y85|NEf=lpV!QGwUKKS77?k?Z#ea^l2 z)c@D1Z0enw=IvfBt7WZzp6&@%l$St5CP0RPfkBg!6jg?SLGXcrfhR?L1AX)OBDn(t z^GeK8L_|?aM1)My5om5{V+I356Jug%h$Y29({E&CXxKkOPmAp6rW_Iyp={_o*fG%2 z-SOM7wMiDTEdVfj}El`;b4yj4<8SomD#V8DEs~b&0do=%z><1c^)4xqSo)LE@EW~H7cs- zFE(bx4B?PM3%FLZz7Bh`DzZu%%ww2dAVZI3uQOHovpn6p|)08rolY^myh7n<21zW*m&gl!#8A-@-0-DvV zrK+a0rkpH~3DB0w$P{R7#sspp|4Rgh55xlv+L}2Vk%4S&?3{Q&{1ktu;DLt!dJUi; z`#XuVH9v)>oFbVB(9w*HgNdDqnL+@WjEs!W(bSwrSycSr;?V#2DZV&6+w%YbZfffS#fWO-QAG-LLqW^vi)wBRIAK-s`O#nGtN&N-}<}-|xsE{h?)nPgUNL36s zNWlX!D_}WQQev-Fy4wGzAK6!9k@1zFFwF{>udOJOm4>f`gfQUY3#Ct5Lncx$V>2Fi zQcV*_s-#@IK4Lt+1s}J%9S<)yo}@3G9&+vwwG;cE$jM4dz9&G6hWTgD=WFulA3tc; zxVuaz;z@flvw_#{_C&P*Oc(}96V}Ja2Me2+QjK06%U7qL@t+Dtf4-)`z{REhARYhn zG_{q4P(lWl>hnK(XpsW8`=!=6KVXr>{PR}&*@Kagkly=LgDE#8rTpL}|H-pz4C}K_ z?T?FF-AxFal|>#)%_sZy+czpvgXd@2)I&7EnGdfFusW;~cYkPxjzbD3cc|z-_9iV= z1F*&a>b&^r51Vd~EwaSDItDg?`?qVCqOWpF%8n#z^rQ_G zyt~+n8H-*;tucLvMdn}f(GP6pb2Xa(XPtLUOOvs%_;;4qqjR!ricwur+Q8gJ33%CN?01#@d{~410i11rJhmN5W?^;BlHpCo|WZKPVX5 z1E<{n0q`D=9%vj~H2nvkY@h?o{tgKq;iB5}E)!PbkK-+x8~%}snUeT{^g-im5>t$y zi*+{;B0ni++<1=V>n#^*Ii`qpj(b|9(_TP{1+uTaCDtN;MO~j-cjitnX`I!BrBM?~ z6@+tM)c$Q;!t{E)r)swwdCu89*iwfBQaiz(T~@<5D;gFAf@EIwd=sG|;l5ZEKr+RJ z)vbxq%Xr@>^`{xJ%J|CHSMY4%8j>LjBzR-DLPxIhQD!_{r(2LgmWrLK&|S5LDoqf8 zFr<;V8<|d3Cj23)35}~!3f4_Fy0fzQ13RB8l|)qq0Y1HwMb4(ZB4cRtwR|X8XFBJ3 zCT)%($2BUcu}YuH;_VTDyWGMwqQ6nn@&jbxTVAu&h^`vnKDbsn(D8#_I7k+jDq0TF zW~G)hH6p%?>QB8(_6m#y4}a@P{n$Cl+4n<9*=vO5h)Hl=- z%l+lsp2ssZ5vYq=naBQ{BnfuRJKZ-?sNMOaL&pvKSSnAI&iuY>d%R!zs98Cki?8z= zK`ftk&Ws}RN2QAkq|eW5qs;m*AUbPqZ3Ys&01O60YuNvYOc6t_>-jfy(1=OY#&YAnye(|_fSnNZSdF$tM50!J8fAN zqx|zo(SUW@8I7UNc+8Yd83fz6=mQ?R6ds0UaB?tGy+YY1C`YeVxPRa|d z3C(|BCp6n7ZCH#BqK799oVK;*ed&IEhMRqI+IBNA%t5$02k0HDPPaP;iLSgbD|uWt zL6iiQPU>BnQ@7&QD2dE;>TTni&{F&7sx64k%6L;bZ4>KP+*fLEt4M+KEm}UQY@8w+ zFyS|XkF%!@S9J?{LUG&Ymrh`7cE^)WyL`8^{(i|l!BsKteC{$fV}tJxy?mJ}`SNDD zcVf0_eHY+ke%8}kA;;6Ptnhtz?bUnKXWc4Q@rcL8$!!hq)+J9@6n*wTuZi{_VT9Zc zXIWs6K5}$%PiYn159Ot3WMu%Zj9<2{#ncv;d{(1nb?UUmM=}6iq(`MnB~BsO-hHE5 zG=fi+x^#&QA0HyqPw~`_QaDVZAx8l%=fz5={4^eK?LIBtKyN$&vCm5tAS^0UB|-^kc%8o?8fQ{85N zjL7!00Jw09vl*LSmwxD3mnu&>ao9e%47^(R_Q>q8AO`;!oElW`yh{Z@^}5Rb2xxl@ zTXfXP4u(N}b&T)66&Lqoyn6a(#X$pNB}|^Zu8NXmie*n6@av!i+Fa*~Wu~w8nNB zu;&TeRmSd33EBIaTCjLz>@_hki4fQ-x9olbCnYKAIGs^%n8o)p@%iFw@YzeoKDG5s zSsc@n#}A(%R8JL3d+J7GPu9rHj|wrEnNhJ~;W(+*>0B{XA^EP}yaF~3>40_uYB&P| zG8ZdIte+nYtwvo*3x{(^iB7wI9S!L$ocH~pMCM!0n~UQI(v+&09PuGrH)ocN&aP#T zK1>RN_4}dn7W3oRJ1hcj+XJn0uDotfj0kT@;$N=3HO&iP4sA%i?~@G~K=L5{DE_+| zA|9_<)mPSt*+lxBP;o{|G&p-d+fRMSb~$7t|&ej>ZuQVrrMNX_8*KE|^%rAAEbRm+26NQpIeK|&pSD-oG2E1SEyYSkTzB^gm`kuaK8*`O6EJC)8uyU zyohf1p|jz%no^wkhyof)ERRl|uZy8epdTvYupT)oo$@MiQALeZJu6?Se##TRvo@VN zRmhXf5J!p5n;ZG^<}O+NmG#mGJJ;mzHjNz-0Ah2O)_j z0~C*TXdn0;FEtCxQBaPX>yx^BdWHaOJ{(C=lYk%mp4ZayS2pG9rA6Fs>)lek>``@a z&dW8nOM~Rk+T*xpToQjaYyR{x9sAStDyRo8)5&d5*AMRj@!SgpNuNe{CKk{UQKLI9 z>9{kapf2WHyLp3swgk-d54a2JNHEF|(oaq0A(05UXF zy5MXk6we2G@m@bU0YvdVMxVXm-;8~6IV1HYU}+3S&Z^+l_f-qrqKaB~zrdBu3?_;G z^_?))$(!_;(xm-Sn`FApuqO;Vp}E^fV3ed*8x38rT3-5pahU>VzPICD4e%fjB*Q04~ zh;E0={#0W76g{qu!%=x}L@b>eos8U;R*@;vK%9;sf_x&5$&RPUyG^o;!p?Ix1wpf3 zt$I7+WPZ=c%a#Wwi6}IJDYsKE_s_n2Q+cUrGdoBkpxq<{K7f#rkg`^w58@O^-o-_r zfi2=_YLD$a``Tn)wa8S|1++F(tkQSnU9TcEe}&I=)9TvwrqJQZ1s1G3JHlD3QhARO z9PBWfW)x|WRFJXO7pZqjk%%}wN95ct=yk_ga;n!13;*Hld= z)N_JnRqvM?#U8s0SGmTPY@j4BZB(IWO|-Z;M+ZE~IhSw6Erx@0djVJC`9N%h0x%ah_gOylyE_ zCdDFKfCgg3)xmUv#+1y>{r>cK&M%Q;igTsQMHv1%b&V`NP~T24dL2(cZEG;zHC{|q zRB~v0WcbX0_a#=Jp9wL}tcIk;kleVF3`-kjMseoZNAV<%!L3R%Wrj<)M`>ZkZn8nTc*gXC8oEMEZn$M6lap~T7_ zf5G(t)N@jIN5;Rtji5^WA zPMV`Q%xl}*S5Vs2F%I>A8sFo#$mL)7W3NVkR)6PL!0Yn}+M8h<@W*;6TR+WO%8uVEQiSMORad@5F_jSgHF_aP?RB;75Jm*t5Z$4RH10D6QtUe;J%u|Y92a|eJG(SHttbOoY z9eX|Vsl_piB2dn5`x_4q;&CE>z(!}Fiouv-35ew>EzhP}Bha`;>Zt9=JFkxMCkkI= zX2WWawG-0LAa)sfE2WC{wpybT$b!8lqy1DAY~{4`+nV1SGsFKMZ05*df(0~;+~#XC z^U8jK1B_OtOWysoZlvx*yz&yNJkJ$#Bt~pjs;^0M0-(lM=d^zU9OeuYjzjdZI_+%c z7C!uXRt&Yd$#FL#HCqm8s}n?QzKL(p6W1W6lOd2}g(g093~j?(=&(!Q4DJ~+ov%`x zX!EhHcCX>fSI;iz2vgP#rvr2wO z5asiFak&|Ixs~poByrq?z1@20@&dqX&@t)oQ%-c5?;~mXh~B~86?u>V$QQVQ^!-89 zM`v@x`}z(yw=(EMnPz47MKsYvu-cbE3o9nMI};^svc3yllm5E~E>JY^4_@qwIHFh02eXEpD&IWeD#L``^lB%k@5d)WxeSVWw)(LXz9aDz|EU~m91C!3kkxDuZQ=w{>7`C-oGe#5%8Qu# z-rEE3Jf4m3+y|uJRo#>>FXintckv%^HR6aaTTWU?rf;5h(byFjGk5ENVmqTxu17fE z3=wuM$KMFv_Jg_w7EWm8AZe@$wxuZ?TLaY3)aB?tT>(Ksvs})n2fv~ajt>|SIlLqw zxn94%v-_T?CYK}^QY`DX=Lh8qlx1ZE zx3CN^LKDo_grfSm{9KDL(Kv9>;B>EAahfPGv!>0?sDS8cEGfOtHu1%dWWmv_FVo9) z=n4{-txw=oSrcF0cBg)S?LPMY+evg5(9&+vMEug*X%|D{ij`p0 zBQLDXggxIbW3h27!5@&!yfT`VPn_U_0KR*2TaJeWqfYa+7^LjH+|vV6-D76Fc?G{2 zK>Jb0Y~0Jj+v1N?J2opH#u<;BA~(LX^~MPZMCw&jsF)!pJU zwI;HRQNV&Wp`@0NhR+c4c?(FE8=Yo$~mnqvq>-FB;UVL#N!s=s&>FWypU6HkBrQ zAZNgf=OL2i&$lu7wX3USCr*#%m&PwwuG2&D@n|oH30lQaSJE3nd6moSyylWvKs{(_ zLGEn^*N1Yi8KSiLi3$!F?$>8i4>km%GiM29 ztabZ|4@2{&gQ<^fi6aC{+ZK@;>TInRrsi{9h#?lMi=2GTlRPK7H&zP(y`sx$SD6;l zKSCv>7+9^SW>iOC3VoO{?A~mhaox_WB#0fTe?d;%gS}X1kT9jOE zmt0+(b7DP8vaIsBlAde#Z5L8(0XYcC8w9wG;%8jMR>Hq|qkManm|fp_l0CApH6o?b zisMUue|@qjl%ig^dW4~s2`@>84Fxq-UUeD&sm1;_a%5y3nNZ+fVN7QYvhe)3^aY~g z?Fe7pVe%jAdyaW}bMx+H$nEaX&5ULX3P$-K%J^lj9X=d#IsUQmDWuy61jSo9VsMXz zu*g>clPMBX^yfNq46I?WeVw}|5zU{-CBGebtar4J9`Z6&0iETX4Vp>tWn8ejEe<{W zSZ_oARQ(PKc)aFWk!UER1?742gKM>*3&N^2RW!AR{BMioe_+uyuwNwz(mZQMDyi5Qf)5PR33 z$yp!33OT`kis0h?XL)%*dxVNzfULgoGTs*ytqvMY2K$jU*Pz>$t)8U+6F!=nr5gipZyP z16V36v54Q%L)o^urx`AvKh@L!h3u8w8NCm|2|g_IKUjkQP27>CVx5A2ZERQkQ!dXB zJ|4!)oA%L>?$6%#zi7qJpFaArCqXI0Gq}yMe_9L-NxOfy{S`*Y_o$?X2CmR3g)xm5 zhrKqB<48B*KO_8eNick@aImnKTL6E5f2`aGcC1tESHl27HQ#sM)8r40eDK*O##B5X z<*~5V0bm%A+Mo3M7rb7M2%TDzlJ-B_zP4KR4N^rDyh3g-j2U>d<$#2Qg^g|7;#K|I zo?G;v#9evZSGtTomAF#l}!THGTv=!Shi^uK8 zr2GlE{IMCGC|b1rb0{KuT-^xakRNVlQfDB_ohyvitjPA4(v@&9>#KVMq3-% zfWV(Qg`;+p@Q!|KZx3uQjwpG=R{eJU768a@VzUyF-^%{M=M zhT@qLXrByd)3wj)jC;bW?7H4)HTw{suMD24SmfCiK2B86xx}`ESjIO8_dgr}RB+1T zt_V4IPn!|>ETBWNEMDi#ib&v{4>*G#0B1mI%JjUfBhpF`P)-NdY0!Lpe9U$D=7AM2 z#xmxxqLL4ePAFC%SZ;k7DUD~)(k)UgqgrZmjMadv+&-#SxJXGCK0>i7 zkMRyaHU|=PiV_&6Z~C3;MRjH*XWrs+|wMkKAzi(7VX(lv`P-;dYlbf>V3We))d%Pan6A8E1tk)V2lkV(Q4fnMgv76Jt z(d$;4#U|U-wDFU|CYJ+`#X*;65?&u&^#kL$uYt%?-w(>&F8^Zk$jQkgH->7Rjvw7e zV(jISu^B*8BD3|r?b|9e3_|`HPlaLOQIOY6o@lrab8eh|m-k|V23upJ$u+P1;xEiUo$1bOUZ z$SGm2d)KERX{wt0UP(VgKmy{q&hlTamo5|RIEzCYt`9Dgl({B=yK7N~wpmJ8h4~#pJXzb57+d!CCqqj3}4c;5)U*^we3l z4>KE=mzv1ycIuP}lmrBTqCEX}A7*$@2G2~Wb1lHlxmt0(zZhG55zz6?ugPIfmf96E zJbz_ste?~rPUbFsKT^iNkWc6NK(A32|BcY9;`UJ0_c{BZRb?9LC9v;g zIrh_k<4`pnMdS^7A~H7JU<0ttH6EPFX`t_OXZBmyZ)+epn^a)afRf8q3di|T1CS`=#Gejv*=@yJxmH^(6CxCRydAqCjK8Q==UhlM z>)S!U262BDh9|k4KL{$+>t~k0*-VN4q)7dD0UPZOi84;f$BLD<^F) z`sxficEav{AGp3+SOsxEC;Ma)@i_OZ{s!?bIW)6ddl=;{R+$Z%KHhA|WflbZkRDz{ zt2SI~v*9j#UC7lhaZU^Hxa~3A%0@lxTWdADmgeY}&ev`pXYdn;Q_5#>(dP6;B(~lY zzl@2XCxhX)wLH8{Y*xG<(sK-&CtE$;EXSDls-L%T4F(CG4jbnWzQt~n5AGa%{h68n zc-YYAb$6cJsFW*-IXOIOQRC1hezz7(O0)uJc-0|-&a;gHLick2d7zkUK9b@HJn@vf zD{FThC{W57-g))xF^Zn5pt+zgELI!!ej?4s*o*yS9UF6l$W z*sK-bXnuc~rmeE>OWnFGd>2cvpifKWrfy6LE-(OQGZ@rrIqr|yVAl5|*OlS-y0RT(Q zOM94bp57s9C+#n7Q>7Z<_geZG-!eVUX|G!Nv-O`pP?4VrXS;(A1)gs=CJr(@H!J%{ zrkz61%fglw_?`oXb(jU!PBt(p#U)46xbrfZL=nLWGS80}3dxsUq8AMGmZ(0=FVe3I z?&-u`+H$G|n&I!Bn@`N3Ct6q!M~uwN?usH?eW8XaV`#86y}e;O44Xaz?|#s_oM0SX z7$3)E(Rpa3_rYzlSB}+XUzIl3DL*mHsO?k8u-YoT!u`&;$U-ok6E+Dr6Xs6yD7% z30oDsH6_}G*(LGgB*wqIjiRyBY2b3Ny+7(5wbY#><+v(;xr~*-PU!%JtG2Bk0BXU0VbV2WD>+!LeSIA{`Eb3L zl#6U)(if?uH9&C0dm}t$BJ5g zt(|X2Oevh@&5ss{T8T)3hiVJ;hi4{t-kDxoLkeyAFu@0;G$Ar=MJ7BD&Yed1av8*A zjHT7ku*M~=KuN+d9AQ1X5jcCzF2$~mE0QY#Q9SdIilOOm?=$g3V$0|QCsH=>f-fEe zqmi<3L$?e-K7Q<$^P**w)}L3cW-GGrU45R2Hd6Xtz#_gdpD zGGDWr4M~n0i^&l~>IHFMjJ7{ZJjn}z^hJ^U5Fh(UeM7K)+zK>a@X-+L0&PK^Gs*xD z!s{lRxvB~A9nx;e0MuT$RO+Yfw=HMZ2ie$~3$C_i2|-i(HD6*xZuN>f?OF7=zMgJr zPATSo!MuvFEm(fIlT9U5T@38w6_rJ7Kdm|`LSZ7{v@#t<*z+oP@J5>}{pwn>z1wkN zZ^q)ZElXLVb8;U|l;xNjw)Mk_Ii%t=wwJ>Yk^H4U$D1RfD+=KzvsW_pdo8F_qexbVxWKsZN!T*R8rE&ANeo(Oa zCDHH&CX`K{jU21lxA=Jmue?1~LRtxpGo9C>;60tX@K=JxRIQ=9V zIPFv%w`%uSvh5N`P1X=$qdoVRoZiv%wYKrK8_&mc2Dv-7+x)K;AFk z&{Z!>KF)l@Cn-6F5p=qaaNZsJ?W`ozBRQNJOGh9al5%%Y{%g@gTUfM9R~SzdG3spp z>|ruWQ0f)-Rf$`6`TlBWnbTf)rOEeQ50PTRh~1z-6jJZgtTXe#u|UOU3^VeomHB7& zD>{G?4ksBr+qOxg^&oyFA$UHhJ<$Hg3{f9f+71ARTU0`9$iVl>Oh82}9Gi_KLj6n6 zVkei;bh!-H^HIZU?R=YvC*`QH;ZRm)XsTVy_*9^V$Jf*oyh{UNL4%pq9xd4+g1yey z8f!O(js+VNW<#_41O$|-j)fY#St7x5$yYOrCGOxeZy0jCjH~pkhu=Yg2&9{Cb)BRH ziFA6>IJ=#20kJ_4>P`}o2zZT6{~!#!oE6eh^LDnEX)$aJpUDH;ZqQ0>&MDsVS+|^)oJruim#@V{fkUy`(ib)a zH87+q3t(>ERDUfh4fg!4IgWsum2QMVJ<00kohgf>yw=#F)2IH^@YRL4jiB2u`rbR{ zsAZo6+v0Ugy&&7u8q2WjT#_zA+Oa{Wqbu%gF8%Jp&Cb{VL z&z>8{yr@rO{}rX_NP<5+NozY~E53IeolBkP@J<-wtbIO9waMmLg_n?!8vWX{sXN8} z-B?>X(QbvpfjUykzX5N~{XQaXa)m2uhk^12Vq@;p*8_~38$ac>3F7lHn5n4!M9x;% zO6tY6^l-D)m6~t2cY1w3Wiz^hBS+{tPc`Cky^J|Amrg#+rr*X6RL3Qgs>OCc!jmor zjlazKY@cpK1*7nBmkN1tx$@cm?wG07LG9Ps~ffAbHPa2 z?eojXp%i*PT2#GUpOVv(W}0`J7b8pO!D@FJ&~`tu=KbZsqdM@%^C26-vTCLvA5Hb3 zzW0?H8tJp&WkYdrBUPuB@7We^`k3Ali^Y4Tnz~(QrxI!Gf0vX~HPP)W&^tvXlix#RlQcOpBp2gZKrO z$}>f3(x4a>Ql)oh5LpmM!OCZSuQRdgC{EC~e2zY)KKEy1|1NcGxtpRNQQ-J$BRgX| zA-?7p{|A)-Rf)(+Y7Tb0hM%7Hwr}ZaxP22wuPz$5xy+X~Z9L(ioReqpja#XC z!S57ZXxJN+u*m-f?IFC^)+D2$@#Z!g-f3Q z5)bDyW*i`Ix%-P#MqWK#! zxi*x3(@I;_?K}R4o=vdYT=Zo;yKdHI$2!t!lQGCBgA7>8n_A@VSgDw&R25udov!&L zPgJ;8y>_-x)Mxunrmv++QL!pSUgmWE;8`dGX5 zy`G_ytG$4{Wv$YnQQB<79bKqd?O>)i&i(QB%$z|p1ny!J-{`Fo!)@wRg^~zm4dvxE zxgO~Ja#A7HCo+@E4hPj{sY#9COS-S?Q|Vf5K1u@_pK}kg6iPVR*ai@#jnZfQwMav7 zO$KGlK>AMnLFK^;q1%d)xu4g6x1*&0vZHgJo=diAr_noZJ%iK7MO=f?FSy*kI_y9? z#uPn!b0^hP$G!!kc74}O9~W%z>1NSvlhMKPcfnLNx|6=W;zN@gzHxdBFR`A*n=!&` z?jJNl+K2VpRDAe7Bz+IcX~pQ&N1V@YnvvcSXJvRF-KCPRaeAlVLKIt)rkSp~9JCNJ z{liZd8)bYi;FCY!#*uPUz6*-_e&Q$X)aB zW<=H}m!7orM4iv*;BHN;G5eo3Tbz(kW^^>W4Q|p_33mmmm+H%<6!zs{!;+UQu2YoR zXtoO#S@K(-RxPx7H!r3}<^%SlAk1yS9FXH5fZo|{TwDG^Cnd`KuP5x(jPOXC*ERYn zTzU$xe-i@R#G01orl*d}vOIa0XQV#9aB+8a0*<{q{BSo(uZl8GyWoSV(IyOM(@dLdk?=yZLE!4Z|lEQMdAXR!G8R^H+yUbOYeblfT zCF15_cVOhUU-i>6gu!gKU8oa}PASv}krqLYPynr8T6nzG5a66LWH7Xi)TSTC2wI5A z6@SZ2Ah+YCUa1CRpeU6>Pq5X6p7{k9Ao(M`A^%j$eLV8mBm6qVJnH$R@_i<)Zhmgi zhq`P)EkcPq84bQ2?$W9C3v%iMW!HYMUb|;LAh+)NvB0Bu>GW;)Rpx#+Ovoa} zU33oay5TaPyIk$%ea|}lkJd!)j>={9`ZZr2&bOZ6$&V}uK%-0jYw9d>xoPTlC$;!*w#C)aeb0XSe? zy!lhZ_(SDmWgLEr;+I}qVco`4?FI&Lx5mq2O7G{=XSXyim$yX24C-^UQ&H5tOY zCuX{9;~MR*;NkIwvH(gNp|~^mmdD}!P45-&@b%Dtzwf9k1wTk7!NriH^myz|W=k!f zdf^twLzzk5AQ`z5p+BU@yBmAkD$om%bi|>ddo6;sm6w?`5Sy@&=Md?nOvNN= z{JccHAm7)&=qo?a&JudQJQX-`+?zTAOiSLo3V-|;MAiBGptF2#8!Mv~jo`LQv)ll5 z-d5JPwq}&UaV=DuNK(T7d3HcBk%^N;0Fh%p*T|a`$_IpFd4Nyvm#%IEq40Au`-7O~ zVU!>tQ{I(`BP}}myJ3~HCehH?kA3AA9r8rHO8YA4`;!oMK`A$1pC|rDj4oIMuJb5ahPeCJe;jO9BPsZBip0t z^j$%36R0!2&a=l#y&nW(v+?1}n)dfv+=sT`Z!HtG+>AD}O7Ndz)HOwTNSP;)2e9!G z2ogUhi~J}m*I&M3=L7^~1y7-Xr|SwnBLJTBGxv5dy2)A52zg|oxtj#6h@s%(zdEM9 zdaTaYoTgKo&bhhH12eg-?L~W3XEv-=Vhw@I7xyN>F(4Y4ZaR zjjhC(Tv)l8iGn*ZjkC!w!^r{D{ytVQq(Lsy_8J)qLZ@qoFWSc99!L`vho=RqQcZ{ts z51Meb*k~sO@LrO{fFa9C%^t^t$_IpZbLnVz)iE-&Os;;0$M?fT;Uw$|ES<9(zz*+y zv&c`p&RJ;NYfxkt@RaE}$|A3jfjPg%7I=bg-EV}phKr&}_9*OJcUNLdESlUQ{Du&@ zCE24$V;o~0dQS>R&{vRBSA-d$S`cm9`QIm%DGF>dt$;of%=U0PbsRgg@#dXmCVJn# zl~LAjWjYE~dC|V1Y}=F>R*fHT=UgdWG^St!K@ zrZ4pL_DyEwdq4hM;vr6vM(r53F7c#+&}Qt2+$A$HO%~FD zp*%G|?_#Y6PsgafTeVfD6S8k)#QnW*^M_K-$X-K(W_3EdLv{xtMn?DEw3OU!AnpqE ztX^JL*ojiqL2gC+sLAd`UU5zPmHNBE*+dZ#lU`E<|D_eLyvU8u(-q|WcQEOXifMh0 zw(da-4mT%5!)TF9F*l)uIMe)-qT zz=Z@uDzErliP0VTL^f+t=v{Yzbl?~hq3TZ_msAF=9~UjxI3S&UEZ<-62`8Q=gYUvv zuFu=%XG&uzD%EcsBwwa%XU_px>$S3mdp!sReU+s5+f?hT+IzMaCd_gfd-M zP>_t4YE{oy)NDv3SCg->w#m%zXQKxY2Yr9oCS>D0FPKHX$TdK0vF6xp8Yv9lfg<*w z-o2S~Uf)L3Z=hti;iaItbURnARU?lF&Q|=!93BV{U~6X4SJ@*9CD z9J!;14`NzL3Y)gtwDCW`1@I{V0=<#Ko$&+({6~r!)<4MnSaUy*7~H+NpXab1rhp$> zjehp_!ui2vZ~prw zzGuhn!ed~LR@u@d&O6f&90X!|r(5pW<-VP_k?6`ZwT26nF0yL0te}v}*@Q3~NFOkbk^eyr(H_T6qJUa`hbjI0&gSjeN~#0S*x4$= z3+Ia0+^#J~oX3d2~)j4^hpZ#s)ehWj>tXg_XNqatL0W2;<>9F+nDw z+IiY?*PdTcr~Yu@;-+vXFT!Rr%&f34o6aLA9!W%Q(ig#DJbBFTxjEhzC7sM_mM4ES z|M8`=6~oa=c(_Sjq{!!fFJHMxm9|i|F;@1;y{JUHR{9M(zHyQ8m&WG4C0Klk>zcB-`9>VZl%ZTwpXjWv<*C!r8@%5R zQ~h8${W)`ww{;qsTNasKbZ%CX)*wFhaj(XT$`qX%32^fVg=4-d=RK@nrCS95jmlt2b zr#A%iM41mf1KiQ{yL_1isC|+6l=W_HA|m>kGm9%LcuMt>=j%~Wev4r32Fh3~0*)6< zA7*Tu$V~H*ux-j;rjraTHLtsRn{muonk3=Avew;32!-etLMho1E9fo#383c{fkfn9 zp-R4$po{6RXA4Bwn-KzGyf!tdY2a7WE$myiud&T zveH)YL3;%9zF&i4gDu!`xD82uuZ~!u>6dEKzyy^+&(|Qw_2D#5pJhf)(?xbFuH5l1 z)u!{y7Xw4it1U)ZjS9}5F7kcKNv6`8^io_nMoKNt>Nnb0$S*1lSD`2|`;N`7)27~w z$O5sTtFE1;Z}I1`-s)gTDVpWU0GpMY39nY+4Ugk3B6bY#AZQqyPJf>yp9z0HshUB4mU)4|K2V_ zPM9ZIR{lGN)A#q1;eB_-S!H>l9@RusEbk1y@|?(N%21VN>;pnC2+Cf3C#VFd$oa|y zriaT#7xcp?x1Rx#Ze-#w?sZ%vp3Md9Hk7#WLnV_MYvq-jgknS|KQPh8Ob zl-;HDJKt*P#j~5GVhPS_tXIXDp9-1BpSKB#jWR2xjh4TPRe{bj%>^#qf7U-2WKgN6 zd_`>$GISjAOjhUY$85HE@!@U|9MCDM+!l4dhX3mt(@G27kL)r4nh{TV^D~cDn{)Ev z^fXmJ1koBwmtnFCf?nY6PIIWxMM4lU{3WP8=O~EiJAVk5_{06Cpiq zvQ1!sr|_lYVVu5@kbnnXZQw3+nc%ll?)EJ8=(`sIH|e}iZBa-A*;t)?8)j(P zG1xXX_;sVUdVO{Wb3CrrLvi!@s1B$O9v$I;L4=c5>J{II(n4ef{gByOs~Pq_48i}$ z*qg_**>&sVtyZhF)d{VksSdQpPN=!nK}${5mPzG#sS^^_4g*L;7!E>0VeH1X#;gr4Xl&YKs;3Y--A z#0d5|DWu0b{rj@-c$1tbPgU=t_Rt=KtXhpT?<+kK0AGCFAaK-6n|c^t?TXJ@E$xbt zez8Y$L5o0he|0^^=jRA*K}pw`-E&DS0Z{!W@f4d@4l~NK4I)osKKm~YJjwkDSrCa4 zo8g}^j(MCbEVlXu^ebD&C@r;nz;E`KN|VlaXZi5cp=vs<0^42(PHXHxS=tp{Umb{8 z77%{?Qo8j~obL90cKV@`M~}00!X$&^R%%BpTw;cMYW?0u$)j~KLA$c%1|+0r4~cJ} zh@eV>q^)l4#e2Pr7)!jv>bz@qyQO$$lZ~H!&crXyz5}(}#Y+9(0@e1z;zK-t^L6k~ zdbb)&8Qpy{ytm|If!g&rh-a48nP0+;m~zJ+;r*<2-2*fT|7J@Xe0;od7aOb5S8rNq zSL@qyRGedVx33z)%g3HpP>OKs>Dv;2OX~6q#P|3c*Pg_1u}`-SUJ;v=_&*U%>!~}! zW9a#WQu7XJYQq|JZ^ff&4^Tk%XC%XSQ*Q^l)*IZshm9%2--qc&lc}T4+Xjqz*f~|X zJC6wO6)t`==~{pkz3I7hq^v3Ae(2}jMdOR{mR|4hO4%0rV@9cS0&7c#cu>$Ld*1d# zXy$-Gulg%3beB484C?8op?G9ElBjGjr%9cjNd}+ZYA`VydXM}^Vqg?F z2JEp#(gDMPVnLq*05(wsEe}QI{QOseZ4x%R{e$ z!fx{Gbq>3RPhFkjmi%0>X-#x}JgaZK)*Drg)ogYZ%xX(@F7q25Xtfit7-d2t6Iab5lZMP!LnC(9BmVAO;?4yx7=&r zO}Dw~c{*NaHK8=^99i9Ow(VAI^zmRF1L|B*`tZQSlWaTU-E6kzWj$@A^)-90c8y6F zbx`5V3^HY?Q12{6uO#&@W?zW+O1xyUty1_nQ!lr0!y|lF8nlNT(M>4}o6nlk+N9&&v3+pP<*rjqM_j zA3YTM)Q#`;?&1wey`XZ3q%wuln#ypB27Ri5F1#r*P?{n~_@F0%L3b=o^}kGHlnK$s z-qtGrJ|9p9p%H2ot}w4?1Q5#|>s{JEK2;2mEDX>+eE4vrrAPEeLfZ0UvYo+S=t^dc z;(AbrKsJA1_Fgg)qn`DCBRy@$6J%oAvKOA?5yLM@6@5C$eTq}1Ar<|qqXg!=x<@l7 zpkE&Q-gOL|1zYp`eP(9&+D%__hnHuOoo|RcW{W^O7XgmmX@NHPW(RfM7 z%4s}>z@sIs(bv}FRbGSJs;9b~kB`P4+WUa6h4~#*@dEMTIi~iRx_7@NJuqsf?m*{Z zIRe5GCt2O-$!kOSH=zEYdpo4Fny*~2L8RPlLlcY!o&8Uf>K~(*9B_J{nbZ%LY>nF) zSl4q3(#DZjBXj3EO!&tWYVNA%=jNmj6kn*6jlFUbcgUEL4?Ly}O4W>?C|UJp{8%4qO}hDQ#I8;?R2~T*Yk{bEChD z!!LdKQ)dm2YV8za-*v`;gM#e~9S&>67%k|X0%a=&KTF8_QVjEFuCxBRLP2b>UzFkd z&BG6HcU0yBc)-}N*KeVpI5pWWN@!)e4<%gBsLRp6eH)~r4y&9vGoJA0mW=Ta0;dX< zn>73SSRXD{lpo1yr2#J<_|FE4L?h4B9)E`^@0*Cpw=v!8dRKq@-2EC2D0u%|nAZ}u zw)4Aue$3!{(A1U$Ap6uxorx<|e!9&^@{eNJ_WuvoyhDd_lwlDmkOXSqU0$X1{3iQ@ z4CnNp*^vID)ghwe`nT=Nc8Mu}gJ|nNdn1+)@4UrVu)KICbt(F9NbEoO@s#%Ne0ijB z8LIx1p7zK0`p*l-&+sj;9~iFkFO08qR{OjEb)C=J=+woF*EhxcOFyL&#Aeq1Py6&Q z%zdFZbFN45@mi)NJ}yQ5?N<}t8N|NaN}G1VBmTYp_;1tl+(cmAUHMpeSmcMy|Ni#h z8lQx3)6xk=O1NX;%EC~Gq`y15P`z&^_Td*c*}KnaO8;%0(0|@Un|8*_C;s~({r`XE zy$?^3_uUEjJKZ|}VMS72?eN=lI7zMlW9Qg^J!Jje^K6-o{8l91r{c_?+jRaPZ{p7o zzBeE4PN*%G)BjAx_FsB;lkbh*1o73iA(4OQ4gROx9;@$+Whg{ghXWn{)YJN(+BgoE z>HdZXQTOlvpcRtx^vxSz9qAjBDw0nV~?%K09Z#d^HMGb*dQ-m(lJh!`OLt% z91GvJ#is}UJ&Mau#^2sHyq8a-`;|0wW#tmWE~ z(@LQcF9`C3f|$B91zAHf29a+w_SpbzkE@34+xO5+{d#rd`CIzi(sV<8({170@zTG! za@OBoH|S9t|J>bm$af-FBEM<5_GOTI9@4K4kT=?R$X|N4Wp0BW5~^x_{m#g7AoS_U zlizQ^nbC9dzsAObj=tKv`|iOtxr2QvNUxqrU6-csfGhnYWLmxKly@KehC~E%5E)vt3As*9|k*YC(Zcxkmhe z`i=QP-`@fqrY8JCA|L7x2cJ8C{_IT(*8TBdP~SIl-So%RSh;nJXJKih0UzxEkxy@n z%*ii2m8S$E<=DDEWZIP!_T~JUO5i`1nBC#sF3@nSxL=u?`te&XKNHlxm?ahzP%2yu%^53gg=c=P9hscWx1iV@ zK`yM}dF+?JYyEmr9Qz~3j-Tq_afFFw(Ueh({qNPHvF;>Y_Fcfe?z(3Ea3va+j@BBS zJ=`XCSXw{+WD<|rs@qQUN7QBC9EHob*RfZAnur(cdt02?Jrf%K=JE|J>~HX=@{Hs% zNZPdcfy;c@_O9L{r!zyfeuYbHAtdgsmSxSu;s?OG;!`I?M9%3NXPno)zwW?`Mk;p4 z;zH(aUYz&T3Ij53&prA5J4y_RIQORZ=fvW|i07h=L4A9~iX5(GBEKEtd(2JW(o_fx z*3a!TrViJh(@`?!+N~C5XouO5&=@nea8S={WM`U6;*(C>yCqp5Oh{5c!g8!Q|84{y z|Cd{Dj@+9imL`f5LCS%{J=U!W=NHvVQaphVy}NUAhr8jZUui1lagyq%e*j+#@7o8Q zu%u~Uyvx?zj+ESDfrjdX6N7c|Ou+cj39)NoN}H-ZroX7S#+*Z%uRKg0L7K?r)ds(p zuc>L@P(ysp7<0Kip2h)Q07+&?>I(lYmp%?ZGhVp*WVXxxcTa0}f;FxyVU$M#)@84$ zT!%Lmk%G95*ib29*;gS$dJf_juOF1sd1j^!61xVS<56#AHI#i;zBRg=aM*b*Dd&;W zOZ%VC-5~8Bwsd+#F*cnU#O5!dxB~A7-i{3ZD@KzeG12mtr5}|7pM7XCO1q1K6VBd= zTK+1jzB_Wcl%ko+L>>&iC#sMNc}WTu68&^}wmL!?dn!g$@nhqQu>?^?NWzfZ>OpZv z+OW?}`QWhbL+-?~%wHMmH+jfvkC&iXbZZBH?GAMf&gCGxJgPl1!s)q{8m07wmTy3r8VG5{ihvS(hKhsoj=ofAgjFlkXv| zzZD%uAYOhu6UJ!=XHpYVK$4EPe)fCI%V`)qxxal|RMF~`asz`$`YxOHAR@#x$o1zLP2w(E9Q3Kq(|`+I|4;W!bhuO9H$U2Cn(zApc~pU%3|WYAxTn`jcVJtduAN$m9A~vcpXvP<4Z%#*?O#x$ZYSIt->s7A#o+q7Sdp5oTYJLER=)& zc)@%iW7F`a%NaFDY?_M8eDApjlSCa$t;rAABir-&8M|qQt zM`0{Q_3+7;>Ia1mV2aAr-*1YlD*jpm4}QH-w^n1@)YNnq))-l_lD{*~<51XBEL~0^ zUQ&J7EW9>g)8FMzwfD&5MRX~T$HJq!d9HsxK)uX+QRvyZwj?j6!_x7?@mt*!&8w}G zKaLztS6QJ_P_@|9=V3aQ&35u;^)FAEd^6ECN-moF83Rc^XKj!ZK+*68Z>(N+?~Okq z23u**<#h@gWRg%Wb!D$L^EXbGJwyNz$sd*M@5q>N+b(_YPB^r}5>BAkoo}-_x)uN6 z-~roJa>kg~dwp_!@aj)(*04Mea-C=oqpcPlkl#ptzD2$M`qNgO=`VusY_I9n>QOm_ zMd=4Hxw>`Bxv) zt>2yL-kg%MLq*biPlgnkSb&%n_IMq@W_df+Pf<^swNkrp-#tOVd$XdSBHcx1y?`I@ zn2iCa&lx6vJe{VSd#_JOzBQwEIA`O;a=q^ytf1_%kZFY2SoNXp5`P@xmDc+B3z^?< z6fs=(;<@;@HKXD>FRb`y0f)xH<^!z5J2@jgaG-6q_GmXe=@a7iTTy@LXu6)lL4lrO z<$;hnJ2{K0RBc9^sL;`}z6~oLNlPQ2kV)qn2Jc@|yf$T{e1&=eZYL79-gb1o)CxjQgkzx)BZ2F_5zTASnuV3F3xt{IIIcGZfpxZuN#2 zqQeT%QLjBkZp|LTFEbT$_)s99uV-6i)PMiRu^i6q$u@bXHjxX?c73uXr>O#483+LX zUQW1CaLp8kc&2}a6%yN1LE>km5Mf%hB#w-L% zDmS|ST$#>q*nO$_h+Tbx@-Q~bJeQn0ict}m`WlqNT?&SvOYf77j8QQ zabz}XHGu6e?Sq(QgbZAl1MYaoqsQS5yDKNsrMp$#N9{a{#UKrD4j-i12ioKNzYh(J z+Y>CDU*=_=b+2K598$8c_zsSwY*z*Xox@I7&Ew-Rb-yc4uGA&OT|z z^|^B{=J!+~6}9OjUaCVMu&fay-!Uj_)Fq`V>g-vGQ!gEko;v3I)=1_1al3CK666{& z_a|3FwsEt+u4stIKfaf|@}3cPz4xPER|Z|Z_ionM=C+B|T}tLymL^e3E4V_FDEG>B zNNvPbhZPKC_5O%CN}N=8F*zk`q8W^Vk0u&xs7pS`-q4sefz+iqgtXC+GFrHa4p4x; zWbhX#uke4=fB#e~U>w0bf-(m*b}3Bj9KHBXLahty3d`s}0I!tm z5|;}IT&W!-r1SVyQ^`=qsOoI%+uob=!&DwKK>ng8bRzfFNd>ST1U=NJk-!MBO?Qsh z?ivDhd3%~CYIboQdRab}cQ*V$p&(6L-G<>Zje@R|*_=byckJB#4<8YuN7lbjTh?G` zoAwyuWfU0O!^JBYvT0+Sy{vw76ZR|43@EI_n-dA^yvSWQ(OG%PVS_@OjkKS_(=Sa8 zt=nS*)YEX)<1fbEA(^oiw{8aoWBixavRC6#EpjQWsx7aoWV|XC*|{sD0@+hBty#%@ zuRomN*Y(uD)x^IUx%I`CWyR8v@POiVG)FT1XWSdQoqn_@MHDQMs? zB%AvVv>htcE37qls&L$d78&NxdcR-a9cM#0Qg}4p+ET*-D z$l0(^3CltG5;W~BkT#SosCTq&{o`^_b)`w#a=jzu(V%C)LwI=?+VA&a{W|wPP2K~IYWvni;IzZf}Y9Z6$txO!I527EjDu!D>$J$F)!*-X7Ei2cpKO#C(u$4m@ z>1x|jto}rk4OxmGUKFAMWX~#EN}K3?x!oIW@vz?k zGqaxS%ZHg)g5H>$7M*&swxB{qlVMXYns{URRgF16NHI)N@;4y^_AgRgQSwv1wyfIO ztZMYUe6?+86>9q0#_z=85mpsSX4E@3es0f}*Zl0vSs~*!L)~}o14?IIX{zQ{;?X22 z1f?7fO0Td6h~hudW!3tZ%BpoAKG9;8z?8Z*JS~vu*__sq1r@~b&%Ei6v z^IZp*@%s`FX8i`}@x~Tl%|COtvP*}&MTBkDF}htdf(y38^2Z{a z_E%H+0M%oqxlK@BL`IF^(5iJUi0giS30uY^U!T4}X=KmgvQrf7A5lC{2E@k3GO?jp zT?h`wFbgc-3h%91p8^ERQ-mipVJuq?_E`=mbt}T&b33&gAMeexh0v{xiyAa1vJKUN za{>nY!7ME%0B^aXo z3ZXG`m>wq5*xFK&1HgbS?G<1pt6}rhsgtm#ec>X&0gv!eONTdbT^=9CM&Gqy(f$%+^ZBxtHr&92?S zOK0WFQA3RHf_oupxM6WK1~t(FwymzUzT#S^xfDf`T^c#-%o=L7>{|%$bPxT43N2~t zU$2$5v~~2YGRhhmyf}#DPLNQ>=a|*j%cYk&7nqSx-5NuA&XycgnYU^861PQ>@_vC0 zLMB*Z7frs>)!QOp@eE?EK81hjKDcFXqf@p98~)A6Pk5mwnRL{}m27q`B#rVS>pY;m zHK>(IJ1%s4L1Dbdt*Z9(;?nx$O)|mx;SemvKD-tc4erlpe-Y;tfwzqIZW8WZ%u_Obsvf2ra^y30>bc~b|LfmI*d#e z&t&8rz*)Dla0q|fl6RPdF0D(QqfoefPUo3@^&;C4P*4~J zh#Mlh;=FmzlN~UbW_ZnYXH__)6E$`NN~^5qM`bT}%-4I8_YTp%MlmtY*ezov0K4J% zW89g);Lxmg-uVct<|gmB+};X6bwV(}b7v_ne=j94Eu6OKn!8!6+<@$4-r2u(sVrc| zDZ>#`R1VXw8G`(xGE!%-^tdqqKf!5E-I13%obV_Fae8Sz#=SPE?Osf) zq>ZWRvT1=Y9Km%SPozxm5`(~`eV?oO}i^s=hFfR|jjmtEE1m;-QZ z-1eqt+4y>Ef!-+iE3s@3DZ$6K=}LE8L*kS+C&Oo?mljf)-~O@mm{mjbRftcbzp z87naFO;)d6ef*HnxaO6Im&jr*Ee9mR+&;?zusJ2cGHZ|Mfb4g2zRj0@fiB`Z(SZr? zy3dY?Mx(7%z@c0auwaXbgi#Q|fUgFQu@_#Mtbl;n%0Zd6lb*`EAyw_F2Y`caZsFfl zM1$No@RK3UdSXCY>{qcCVYJvP@HS`o-E1h3WXR;Jv8isEZDdGX2!F7?dcYI1oP^x0 zg{@d6l__g&>lKpCToS``sMuKtP$NQAu%m>pOx_s)Ew>Zt4q1Wn7{5f1W9QL>5_=A! z4CsnK0u$XF^nr)*85^IWtqvI$sAPwC1Lj#}i$;95isL;x?<8vrZC!w!c&oWwx#qI{ zYZVTyk`Ri0)zmPDbxbz%3hY=&J4aG){x?yMDXT1a#O-egM% zN$OgJ69DV9D_gzhQIk!N&J+8xk=Q%5>0O5WXYf`~C|LO+b|LH=}p|acOqVwn+nU*KXy!g`^mwL1@&_}jG4Ew>1vou z&aE|_s$$Xb&1Q**Rl^FNBHK&vTt^VhE7HV6VYdK|wIg}R?E1+To#1&Z^EqZgCh-PyZ&nulj_q9CK$g4; zFACQ!%vm>s-h3~Y40FAKZn@MdIfYwDY(hgR2kt)Ig-X;JAg#((5$W&lbn#V-3pGf* zx=KpFHR@mSE*>-zf}kB=-pk3XX^gbyj}(^*!!v(FFQ-(_Tx2pE+0l12W!=>{YeSTo znb$Ov{EoEGHy|O)6SUk{?Auq7^xI+%{|#&_mn1 z95Uq?7^*)flhY)GC)46TiSEEN2#fHBgHzRkn*-P<#^s6dQ%JaWgy+a&1N_a$GI6%EsmEoyH14r-&Ulx7#XS zyF2aMv6-JzcHy84l962z3L6XfVq?RQc=0;n%fk^+F`D$HE)g0IR3a zL$65J69Ju$HE+!OFlV%re^v3 z(RfYNb>NRGhieuReXFX?yoIwM{YvstQc-8a$Y)Ya%h>PSQ)ho6i#t33Lylk*l~5?b zsX6p~TD4fc3ZqQ)0LoCONp7v|Ye1aQz!T^&cWTpD@Gj6Faq2fcd%GT_8rmz=FERFp z@3&8SL&IZbaU5#C?=B$)P!!4uK>74bw|g7Kb(Pa*X!B{QYe75bKwj#ko?0`B>4iVb`M=gg9LOOubvF z;P>;(grm+CdSgD4ThmZQ$@qpATUq&ceQsWoUK2J%yr&7QAzBzf1wfoGNv>EfC3fM$ zTNw3LT>8=E`F7$VZWdUiffsYsnQM-IiE7X|n7Iqwb;)*{UK0PQ4R6Ge$%p+9T05O%Fw)mYM@i4|EeNOAcLq)9wo!0C; zHSY(Axk!k!lxlIn#I^SEMveGb zB4B5_FQ{Ul^Nrq=0W5GIPhjPU&D|2&S4R~aF5OBIQ?iD))2Tg0sK!`$by7<-n=G)9T^VpBr2tA0Y>Ecr$ zVUc65n1j)r*>sN!6V7t)lfLgyvq60w-Oa2=rPJEGxyBXQ=9Fx1fvF)FRZUx%GsNj* zLhdWP`o~=cZ<)3owF2M1CEi}a&@j-S>Vq*|nNK*vBV%E+@QY%mZ3EUY@pb?+wLvn z?|C%}F}&?zBP^C{E_(D161I?UDX%Jd_ohyd`~cRcpWKeVRA5)T#2u}-;aW{|dNW%! z=&P=g!vj@Yd7DVU<~L<@z{snH3i%TAd4%4W6v~UsDAZlF z-11oyXW_Pqys>Xh>255u;PS|)+pIAzs`p9YA{hS8+|9RXMRr}xi{NoZr~kpYsZiKh z#q*nVsz>O2v~sK1{BlL)W0ZdbZImCOzzjnJc*%wNUrQVh!HKbf^_GQD{4a17qcj*VRzeib{%ljI>iD&|$UINb-A@9K%v9wu0sjnI2axLR%w$EZ| zKUt$u_b*qdDTuFphw1YBV-aS=54Lku>Iha&ueNkuuH zooR#y4-km(z*1`ORa)&Gef(i=*Q6|V=(aS{W&&~gYW>Ph`hv&FfCEv@6T@)6Yx_}G z2g<>u`U}GkR}k<30*U*!1Nn^CfK|EGQ_~qJU9&a-Vl&s$oGnFoEs5blKi2eZTKDpg zW%4{M1Ns2M*ZWEj+8JM#%t2z=Y1$66t;fcC4sjhE@mn57rL{AO*Bnp-yi8H*LRrN? zr#PNOOIR16B(Y+*>}b_62DaX5WKX*IDfs1hWMy+l59iph3}>j@FMc$qz#t%YIm~9B zmbXfQi&PE}X+R+ERc zqOhK+3u%Bd-oMh^ALf5+s_FFp{oaqi<|I zs&K&YYMr)={qO^ehSG1N-#UWG$t0l`p7 zs6h7N`AZ)?e%D!Lh@H&S4JCLwYj7P-RMIuhfv4_5SAv9LaFRD!B!zK2q-6d4Sju{s zW?hM+=mrg$s9Wrz2T9}RqNXYFMi%#wxKq*b z$Ek{43TTodsH}HQCOzF6?IEOiv$I+vdZvJ2N(is?*ZgUlpUZvSGqh08K{3$OKaclFSWny?cn ztlaa~U`kmia2-n24wO}85H8Kd*N7<=^|Rmw1wW_rjE_&O_kXr8jZxu$F%zTfv$-2^ zY!P}La(=F>Tb)on^`0qtX&Bx!R#ls+qEx^9NEgQ91>c_dqs}G%LC>gLnVF#)y?h%o zoUj758=I$@xoTPO$^$oc&%bOQLob2Ax-YslcdXv=wY%c7Jzi_`5w-SI+L)MZyF;tuCZk^;% z3Vw^?&$O|?0{IbxCeDNWVzs+qYo2OD1)`ciNtOt?Yph0t>IJ6e=3x1tHbP+XwiXk>fy~}>+R@@4T3#{F zg+Q)v4oHCpgn9)7RH&O23K?atGm|CLsnpD4+ztMNji{yEP09BY=vDyq z7d%>~CPtZE!&!-k^-lyaA~B@GlkQHR;QS)*&e!jAW-dZ%+EfE`n}iQw=gOEFA%!sx zr^1&T_YU6{11)t`3ZgD%RR%i%DZzF8dhieZ-QoIvEAX&cJ-}DbhE8mWoCgXu{L$$6 zYR~4|EieN?LmdpK&AH0Fs$w#`l2>AnoymPgfl4AiTP%4XL+@HyV~s-cTVe-zmL{f+U+ zj;o{PYRUJj7P-Mn0iz@@l4{A@UqW6DgUX>3^zdNdXk7NiMr94ZnHXv);v%rStae2B z+W3w!=^I3t9~q(%8dr2|T+zO6F!p%ED~|b6d|j>ISb$(L@rMmgclq0{2Aiu@pIEJ~ zHzqmEgPPcg{LtNdN6Jjs4yQwbcw3JZt)EP5@T5vr2GbVrBx|(@i@n60`J@~+Z^*5P zMDa4J{%p-1l7L14ya#<$kb zrPK&1)qFd8ULgEucmZ8e9TUhgn^#rL7wr`iRao9WR5RHwKY|Zun(M}|@#<-{o@KQy zkh_E>))*`sH{28OtWd*wKPnJx0sj)dHk~qJ<=Ge3-=oi-%|etlw7`VtId^BxNA#ZQ zbYycjU$p4VLv~|#miEj*2hvZ<#qu_r)1#q&Xjm68ebzu1l!X2P{-25R|F_fprmyIO z*AA>rhipVC8}}>*C$$i9+yhYXLtVII}mpp?(hAo^M!Ku5A>B~=+ zqVm{btu-G-#Lnm22M|Y?`QyF(o@QQA@!A7+-%s^n24q@ZB9BV&caBWT4a1MOh4TXW@WTf<-fQ9wpCQRGE==l zVVp&;Mr?hIjT^u-^O@6sd1ubk*=+)3rIq*5Ds7z;z*TbW(YHTNdD7jC@Z zc8zVKgw^qy+kBq;#DS3T4MUKd@Ph!#x0Sth1`!Y|h26Lna5TB;@(Q>}aO0$qDu>jq zDYAzg(~v8U(BvTD0F_;)o;?>Iig4ylU@h9NB09<<@=(Tc&^DVm^7%)Mq4F)yE}f_* zUUSZTkMRAV`Yv?da`0TYG>-<37u78j?&xyc;+x^&;oU`PNbt#)-pqg-^jC0$z!#k= z;xifp<@--u9<=0#4Aq@$sKI3vx2<~^DbfIAd=jV$1tgc`{bQ%kP0+T3P@$XtED$(p z)M*sHlBxa#?D(})KBKZ*%}aUdMFT$C>fH1l&gfe_QJe~J0|8Ow$BNrie!S@Nx=68L z{^S(NvkWCSd3ZpNt!to`?LRQo;6gW2w`TkuRxqE(Av&6eQ18_4b2&ZQ-ESSJUWs)HT}E9D;zWE+v#AtXg|x zV=LXVkK8Tnr6NFEa7Zj%&I-F;@)m@Gh5wQZ7)!IQCAnGx9_Nn-TI$M>$54Ftwj9VP z{mubh!3f@NpDYzH=VVL?wzP@)kUMY$_E&Nw{UMjG3uBrsH7NzZ3tGk)o_rpAx;uu? z<4qXa=w_~J53evv4}d(F`|;|L`Tt@kjD?CSNqE>(M7lP|pVbTq#XVZLoxgEV`?o0z zxy__eXR^89x2R>B@`}Bl)m!xi1K>)zky2*vxW$Cm3lXs|Z_Pe8fku$XIvD4r*-oE+ z7t;tN)l%dsHmWsEt){pJJfbs*Ml4(1j4$M92!?NyLE(MMh3zDB6Yb8od-SaiY4uaU z=2GyyMe_jz+e|lP1$>HhzZ9RH70kYnZn5Nop~!XQv}}CNsh+8y&I;PR1VMsz=!`xp z<4TuLUv_YdM)%1{QPRi2w6~=_Z>0$nYjI>qrFU}&7><8R^<69}d?bFrzZCOVag(BX zbTh7MTumk5*XS7CmJ($GS0j7L&UX8>ye0Rqt%#P3EqyGhyz;Pi;AyMo#h;?;P3W~Z zX*|*hBaQ*6wDfs!R>Q92y9}-?-WaoA&yw&0mh;Y;ZrS#c(+c)(@$+s#eH*sV>#-3o zvZqFl)Cgl12RVyVGHcLMT!`?OsSwskAVoOULalv90RS!~^TG?7q-XQ$AmHSp8Sv?Y zr9DG{+cznhep_WKK@Xk}#XE?Yl9|yhdjgRt=va;WO{8~v?7!p^ckp2qyLp8l<<5|{ zuy&2lS?(jdK;^|dRivKi-^|1-;g)<^CfhVz4OC$n%iB^0RJkEX!rzEcfwroHHu-S- z>6Z!hSmy5`9dvR6C^}b>P!f`V_15z8^JC^~m+kOV@>EX-+Y&Y&~?B_agOLI)o9V zp8huTUTsVSi}$%uRnqtP5AU2BzPrdS3Qi%|U-7*C&crsgzN^6WFIIsq)rX-4VLitR zf}ycBn7pn!=>iOwwEErLKpgf~XYuq?E3l;n~}c zYiYpngTKBk#WVBr)L$l6#V;AM_<8(e16z5Hv1bK;Rc-jM^<=m3LG{yhjtemOBg7m+ z8|PpAFGh;3rzV7z{KK}L+j2Nied-|3rtA*Bp7K1t#yu5 zAFvN6B$YKZ^lVLy!HC-QUwwcfX}HC+gdaHT!eqQz6w*fH)q>p!$aMlK!>+<8Zh}22 zC7D&2Fpg`~cP0+dDHx{M4^%AjZXaZm;_eKnKKQEX(cIM2Nzw7FYhFJ;$n>tA*4;Yx zdXZS>X9?nIBS%{Z8n44XuEvQS;yffoGO58RLmHcs}p8Z5%iwZ3C3Np z2SNH4N1nD)xa^ zFOGD$vm7(Asr z5fW+gL8-@v-F?rPjtf5sk~oi9Ne};<+aP;lU1zecs%-^g(Z;r8z-AedTl?}R*==N$ z@V;X~!YFX?D7U8Iv>(K#&PoU$87y2M{Dq#&(3M!IbB5WJ>zoWAs3C(YBJ z7F~%GI{Cur^dIKoaYW!~aBJ8$XM0IE`rwHLrld+cm|RCFwL!IC7#xWT}K4`@((c4+Eg6Vw;fq{2^O@rRdrp%T6geMWVUl%M5wA(j&Kjq?Wl0| z?Nq@isB(Hgn(O@^L6~U#^EDhxu1g4(P5?PmwMwOhzbI0aa8utR$!IJ*{X4F`SV#rD4Lr0bkzkt==}C@QBVT=uJz!Hr=JcQLs#;X4Y6ZU@P;bB;)umOSRm)&}6g)9p#rdb+Yt9&5C9 zYl4Mifpo*$@{IxFSDt8*MK_ZnMVlc{JO1)~nV#fL<`}mWmFgWJF#uVnZ;H~G5rH+s znMj>u*&)Mbl=#0G_^kgl{_8pS$6JfVtB3fkQIEV!j5o$DW7g5XLwbd=J-=EPUvV*ABbu!sUl)eu-Wz(U&ntT_zB0Q0RI8W`A@*k zzkbb$7u;0M_OmSXw9BO6n#N)W27(i!8d1rUa@|0ncY7o$O|X?{hL~CPaEkhYLLUIK zqfM=Iktd-!2co8S<91O>{ObR(9+n#0~AnX@7S9qwZcmb%GD*5t#YT-bwlI-1&6(g3Z2gC||3_AKv8D z^Xm_G!@cy2){WYS9CiPA?}(EEgU4{ZLHjt1{83>~sq-aoK3GZ5=xfaSXQK;8JemUWvPcl&$b%;d8#9KyRBv9BsP})c{a2UpIWE95nufCmHUpIS~u!$bF}>9 za3it=4s9}T#rpsG%H8`8ZOY6&{wn|Hmtyh5B^x0ug4M2nna_WI&fSFT@{N$lu05E@ zKbaE&#@**{$4>Ij;rs`L;lHYlz^Meuosna|GlZDC|2TlT_d6OP@qNEmaBmKTE6aa8ZT5^tw^k#@|w1o*0;~W0U@QBLB&?UEdM$nYe->CZf9g~<`EkkrHkbeS-C4t&cHpH}NQLn}=pP+`z%$F7r15IZV}?`18>!MP%)OS$KVMtHGrNKC|Btcv4yW?}y6!I{>- z#QK0*FtVlnl<>bAWTgCe=3Z1>An^*v-uvr6PYEb4B-TYWi2RRy`M{$N{szQwj@G|K z2G`#lxX7O$FKGdGDB*AMpG0+8vB@4+rg!G=fS3XR z;!-=K*Q-D8cmVcXH2j8f5X0;5{`${LntsC)9uu$R{)19o@3hu9we1!kO8<9h{bJPV z7dg3oWxlFV_3vW9{T5hLQ^Vi?xdi+(%KdTQ#lHq9X|1&17fEJ<9Dk2GBClzUAwG^u z#ODN8{|7z@cB(IsK!UL<2X*uh5k?9CAa^N+eeLh=ZZh}~09^h}pZ?B+Tl7E>^L@GD z+23u}h%F88%$crt><`ion&txpaQ`Sz^FGnPb2gL~Re;Z`EV~|hkb=%U#hR=p%;ZIT zJ`_qsTiX=MERx!~xr#s_3=tJXD=8Kuv|4YOp?rmcc9bIu`%FB}RSa7&gyNZFZk_U} zP1b42n!N}W=o=l|@$|{y_3oBldo5xHMQYI4{sHZediAWvpaR9CPx}Ei9A(B^Ub<$~h{}-6sYR3~$mAY?+|!GlJhi zlgJM^R$YqV`F8URMZMpnr%refUY?CGHg1Du+X%j)w@z^irEh3oQ)%Q_rG9D+QiieJ zCHdUxFFs4kAriSNRrl_iA27!Aoxgntb*|jBW1rpl4+^HW4Cd1pBPKz5NIy_II#~Hx z5SqzS%=)~xbs|E z>^1OBLCP9d3Js=IUICw_R-tGtf(&LxCZa0znr{ebj~*V|C^o!*Kws0)43khYT`hdf zrYLov+9?LmB_t#Yf5N2o5~X3ep13AuU6!DVCm4HR&i`5x$dCKJdcDJ|4N(F% zm>=|w$x5tY*5GkWZ91qlmIWtO*Flw8hot^5Up;)-j|&|F*HK7!NUo@PWPq^e@!6C{)6Az< zd^{9KOx2oDsV6}Q7qk)S>)lccER{YOQTxmPhU-*mqnYdbVunI5 zdbU!8GFSLJOLe7>S9m}YqR7kuXGJ@uu*S+kJ`x(T5Y~yfx(uc%$Al6WAJzERl zWtuo;Xr9um%bjBIW+KL|!-_fnhmVL!BDELco3|a- z>Ueg)F{g?CPAY;*KFc5A7OQ2El;d(!iy>^>h6YxIDFNXH@8Y!z5$Ovc6%$yCopu>l zZ~>CoOSa&56z9fR6<}HQXo>=i>5EWqQZ}0S1_UiQrbu~A%PPN6+;(|&8hazBFksRJ z@r4l9U`79xp5%_oIIquVaG%ue=30sH%~+6_74ik8=f5>QvX4JNPyoMoev8us+7DSf zy`O$#9D~3YP!LL4JFHW=ABsKSdrMydas3s3jP<+Vd9X3}ct2>E_*@AYJWhWcU(SCX ze26k%Vk`J{)%h^V)Y^;5gIzqtmg3o?ES<2f%U^^D#29IW!OzU7JZFYupiG`(qJ_cX zD(UD3quHq1wgyupf?I<386#N%U(wV*U=Ry9+2W29Ui(KLFDak55Kd8<?>fBso)_(&7}Nbwk!ME^h*~b6KCXL?*LCw=g7;=xBFlWC=x^h z7!u{})+ezKp3n2s*3g%L+JjF9fcVl3AQLYcY|P3{ZeFehdvk-@1#|IQ!-&-xUvhvx z?%I1{5DVn}fOSOH4e`<_0#8lCXs;*?C&X}EI2E*sjA7rfRwzxGTj%4L7VN{D@vU&+ z(mi}eN6>=4jjns=*RH|*Z~tKsH5Y(TQ@5Rsx<4$MxkAI|XOM>2YY9oE^adER(C2WO z_&|-o^o6v&HzW{Lq#A=3}_i*hsjV&LQp#JOJYI?soo9c%7m`}hJN-Bth~NvNd8 z%*B10SjbK9&1?>u6OAU^c+2fgF_8$h1?Dw);o=spX0kl<$^CiH4m3FbJVV}(qg5u>X_bcX4EjnzGz zbG@gQcL+N3fy{#*l>-DboeeY`Gfh!d&A#3dk!h=#+~nsL2kbTS5-X^VL&Mm%nJsOr5My(dJvaF6v%##HJ9HJR zf!%Y^nj8I(cz#TtODXT(`Bi^83j+zIu`5#|Jpe&8%$c9BN>)G{kA7P_BR_-j11q6| z5q_Xqq}l3+Vkz%hNvDr8*a8Bo=+H(w3uNFn1iJ}zH#3GMOT7uJH*jdO-lDcu3e-#X z53KvfStubD!6o(6>l^vW8tXo3D;m2q-mxeF`;Z9TQb1c`$w1@RPuVIW&RWCM6>_E` z^F6jke70sUOJ(p9GhFOSxUGu{_ftCpiL$-AmYT2Lx65q%3GXTO>YKGFN)b#`c)~w4 z0)#Tl|0w~Aa`S>PY?GVA-A1j2C9%WhLjl7VkC+*LA;iP*)dr7M(p`YziDsfE3vVE} zM}DIjI|jRb)|c;37(^(ls02Rx@%dn#!}us|dIdQ2+H2vys#>l^(&#dEx`uUYcR09F z{dc@nw(zQJ4wm|HSTv*)q^tau8;fjaTt|AAZTp}ytf65=UUO45GUlSa#!7ZK`-L$H z^1cq$q*moYfl@o*O0e0mW<6Q_iS{-Jq#!;Nkq?XoZ^Q_MN0O-}Dsht#B4P$AOMu~s2#-d?+9!nr9A|U#jhwA5lerz|J`^DM zM+p-4j-lc{!&gr5j^d2^oTDMVeUrg8>jjXc)=FO^OglzTGFd&x`!y=4$DjhQOjf@6 z1WcU_kh61v5*4s(O8E9ad{yN2&dyGt5cBKw=bHWUP-E2s8n5d8tnbXNZ^y%c<=Dr!KI#eJFyS)9t0>d z_+Uht?y+xKziNRRortLw1B)RSbWqXkw6&GbOCZyYS=yfB^HA~1!fJ2Jn-K3{2J=hZnT%nH72 zA46Y_SOA4if|=jr-x{cpv{>`eBiK^<>JcZG?|Fq*2T~!R=>EXp6R3+WRTb_ZSW^5% z$F?o_5mbiM(gl@SC*}+G2hdlhFX{5o$xyrnkPBevt#H|n@fDQT6l5TJue-uCiaLA(U7;*8v3^x&DHx3R zlw~2`BChr8cR5gtG7g zC>O~@RMw~u7{<1c1=vJ3TF9}_A?B#4qG4V3hERpR=^-Aas#cW=?)PL5}SE$l{;2KOSTa~yddxzJXW%AN4&z!?@G=f!{ZJP z(I|G)_mB{?vps5L$l893Q#8gg3LIiFvFl?b`}7O?I&T_YTL}|1&v>CXV$~vVuw<)N zgk9WDTAd4V|9&E|e-8FYnhEaO{K+Q9%PgYnH_inPv@4TsiU1~N+0?S#6WiQ{E`x&XD+$XIukyAK9IeE5Z8Um`f5yEa@mCyrMJ;{a_j*Se zrW{=RST!9@l;M14_{#uQ?R>`i9*U!V%mg#HUT=p19nSf5MbF?df6;xbq36v0Vx_fz zh&WOdsNyAVM}0oGKWnapo@eq#dcWr~kbt4RWz3jWU40o0M$_$eT)9{E3A0R)MyDTS zJyY9HKjzTrgu+3ZSY>Fd)9RVzpqhjqxNJcXlZT2TYTC4^8zC)n!Vfh5-l_Fc2GC)jD3X zn{q?fF832lec|{af?*v1>V*im@0!)v7MxCa$0mdcUa2*O?8G6Ou=<09*nXS`7S4o9 zAWi!T1#ZDh-DFF6 z@@xgxLj&VBB`5UQQ+~f!X%1537KCGxl!v1S3(S;-ERDc2=E4%D?b7Qzq>0T&(V&{m zc9KklU7J);4Lxi=sd(pUAy06FW!nU)Ye5O!HcN^4)?i8Bz1XQc;%ceQt2`Z!ZTjPY zlS@iz_#xDzCi_$EbW|WX_!loP87jsUHd#54X7f%w5w>IjVf&TupbHwbTixISGe|$- zUv1{jS+T85uh|)>QZBXy!#QFT_NLVk-@aG8?Nc&EMJ+px{(*xlv5s>JbMW5&hmBdY zg5Oy{m1E1xU@H7A5kxuPGgofx@AwS=UEJk55zPudz2(VqFwI+`p?Oq%cY$>DF(&W*CLB#1|`&FA5ob~98%1}b?PKklADhPw1ZO1XcU>2r_DVvdsZipwgRw!Kz zE~BT;Ql83u{2{>vhYb!#JO9XjS7u}7dB~9aV!rE$|euXz3e))232Y13jao~yLvErKP8yv zn|vAwqkkO?qa*3q7N!+;`1sO4>Ef<^X0eThkJ%R`JAa(lO-F<6i^jIADmcorL-3U& z6TD*x3e5WqDY+9@@TJAa)(`++uL1CS)KQMja-T-|+@BJ31OQ|U8=!3y$dj=@ZceJC z7p^^O45{IGh+`uuQn@z|L^ytm%JeG)3n0o#;c+`)t&xhkE30iEg!w@a5{k>=V#+~i z5ttN^SH9=6>oo(aF^BAm+*;*VFbrj$50bg9((PkFTh^6EjPTVntMSpc87O+^A>#d1 z?>>h_QOVieK|x>mdv4?!2!y`BfoNqaVt)*Sifwz>VhwYR2}M?O#?T!Igc3J$`rmf& z8-mq1fCk)BzX^F?y6jKKD4?|(Q;XOIjoI*9Pw}|jwB<+8YAa0G%TrW_Z}=7{8wa7X zI&nqfOmOa79Gex#Rml90+UEGed8Q{;SRziJfMXQ6abpALu*u62q#qnst_l^tAD}(D zRO?apV6rcA9K?!JG~jp0Jw$NSf%x$f6?FBI6ZcU8Al!o_MeClgC&?NE&x%z3Wx~Hn ze{<5Z{r)mJ*sX)fq0u+p8jQzQD3G7erZCgF}tFOB#bk77`f9UPdUT3Yi;ReGH1e@ao_gJXzib=T_)3=J9o;kLwah18V9f% z@VLymMBSLe-Nx}6&}(D_A{s;k34H--7Bt2?Vd4#ZEA6z0xlmDStvv=0sDql%UeLhU zirv_@!0@srvs9x=16%o_iSc}Qg);cFxMSirZRO4pYA2@ zKJa=nFhv{4Ay1vg|P!cW@!lFrmq_ zU$z0&GWp>}^p|H5OnSmGAS18t=r?mMx5HOemx}ls2d>_qn;N(2{{`lRs8MCEue3de z4l|Wkq1nJ*<66L8Gf)(yTz=xQ^oxkuJJ_)qM{UAdzcTs8@iRsfM|yJx^Krdf=MCzM zfwfmaM4Vtb8k|}pA^nxvqPd+dO z#^W??KQ)S%DiRTIQ$1G?$6?fmms-d#hmpXs$Y<t;iL6l#5h$vxYG@-p^`X`6WdSTtB{Vvy>)p$){|{`S;P+!uS% zxQ@yuJ{wmJU3=%8hiCPPk2ixjkctJ!upx@9jmk`7L9mw&H4dsov zuY=(+T9Y_P=pldk-&!s25ZQNczgD|53%mxEk@^=vb>JPyEYgZuB(Zht(E2bVwX!SY zzFuLXEp96|+k8FsMD1jXk8KLN6qE*Z()5fTlh(W^6Jao36lqnjW6ugjO?BIZ=sxOv zB*T+XA`#U!(D)>Df>i`YWkQR2GuYs3i0cl6(BVJ)#2XHvPI=XHuTN?pX@~qjGYNQu zv=CCx@IDxO32@*NI&$!**2l{A8fpc}ylSjKl1Mw8?)un&9)5esC89lnB5pf44s<@M z@K_rLLrewXkv>CWNvE#x$HvZ4f*x$#P-H&4w&)8{m2-e>=&V&%`Lj2~0_CDvRoq~& zjmm)lGsa)|dUdHp(;zK96yneP5t;_fRBT>-Be!2yBl*+LV+GCl(HQmP8Cv`^pfmXn zV0J8UyeZaLatvz&b8~Mq7%rytmQ74tFZSzGeh$%X75ogMj=dC`aH+`FnSxwf!QgF- zCWKrqwylz3<+HrMaV==jdL5KLALcH)fV0}$k*#eyuTRZ`GhU=?t0wgkkCxK0yd z3#Zj-SZC|J*`8_vs?mqxXMhCDY3OK;LG9CfXzXbNp^#1g!z!|uz&DWF`ZOhku2ICj zWx!6TO)4AINe$#c1fSl_;c;t_AScot0wD&F3*m6kjeJ%E6>4sad1&sA>j46QVulM) zILb`rKaj@uwb3#uiLoUKDhjg&Y6n%JIHtxS8HIOh5}B;oGsVqHvg&l#Lk{V(*DgBf zI&W_=J!?D31FPmCbX8S(%2K#(CGMf(K~nk6TA`D`$Th;3S$^B7grD+desJ=fw@Ep8 zT?OA_1-q6E*5ZLJ&)}>yHg1r@wMm^?@ADP==N5ba4F9@+_u+hk`xN#u*I0sCfDUwq z)Xmk_O6r;u=s^{mA%zuz@+nZEN8isK2i0-ekMPvW*|XQzB)iPe=+NYANydfFX}?4^ zcR9&_$51h0xvnmysm>g>4)fPZKN|-YHarKSg>%F^MzC(SDYG|qX1V9I8BoRslp6;C zkM%)?-|p(8g~XMT&%68Uhn4zi>cYtx7f9VH0uMmL)==+Ov%z$ABA6LmQu9>Ypftm{ z_h=EvDWSeFS~6A<44dBNN~GwH*f79dz4+~8Dib)%HqLlHga+C4;yGLd}7Qd>*<9-1=h3q zX?ihrTvLKDep}Ls`g|;JgF#S7OGYEyhn||LSf*1ByL}P{r|&e`SQWD<4BMz`+Lg?2 zkxcEC=1?RR-99MCNoLw@W~^w^<+YRh3ITu}$Zv;%I=B3K0-k_Ifi8Ywg1)ZhwX!78 zp^)@39VH|y7=($Tg$i>kj}$)Z%Pzp>Cg_?Jx82O|nr?iSsM=Rxod|>iK3b$K!XHy93lpM%z25DH5 zZtFrc7Pt)$c81>D(Ck2xhb8(aURUoQ%R0rMmS6zdu`7w~hzX|E3BrNa($P!?kbI*O zWwLsssQn1ao?6$n^0zm|j=k+4;);clR2*e=X zc1tVQ&~dWf1_<^^%qq)tl&!RPSU%y?bE85B)39&sg}#Tcx=-}o-Uk+k{4X))I)P^+ zmkAg2>rutp^wmt7uKNDA^^rt#W+(OvFc<&%$64K3W^8>3Wj*Iv!2!HD0{O#-#6N@K z#xYwpu3@UMIkR{99f}4^Ks#GoFG#qTi59bHY)*P?2UB9BR$qQ#UV5AJ`H=%O0SBse z*4i-oQ0v}2=iEPyl*w?cIJTpII>|mgA8rj|wP7z#zwE3^%_{`W#{b}$6MV&jO8voc z1!lS90$S13q5sNdQe)(MuQx`7^R@wf)uOf0;24@jTQL(N_c&A-%_?&u7zB z3X!B9qP~gEw)o&V<>4D0XuBU;6O*hzrz;vp*4XR1s%k1kpLHQQI{D8K;9R>m4N3Ay zY*~T$XirPRpCuVKt-$NRxKq>Z<^xT6MpiAYgaWn7q@dcjPw-%MCA-`{+jevKS45ro zxrJ#b4>9!dtNFT9ou!XuJ3*&-0*rw-X;O3VUig1!0nC!xY;h#BpjIZ&Q58~`^flO2 z{(`52O7bA@mP&KnuUiKd7OwJJ9HX*Dar?#boPOm*>%%Lf>!K4vUd!qZMli<&Lwa;g zBw>zOr5-65pdw=)REO2mAwYoAgoF-@woy$0ZSp5@tJ2Q#jWpClW3m ztj!*n;XCB-BM1!OXlh;$3Ho1;^6&2stNIPQ4=2IG+4eb?gOTj-*_k|Gc1A(2h=*gJ z0Dwn0NcgeeuYUMD7f3zb4{?9qVKN8ku zQh{Fk+K8b7GNl|Sd_IDBf zBtX%ZJ?$s1Hbp;tf7S4RaR11=L-q51Qg|MLKiGYMRV%sx_jf(br9}S5{WCWJ?jIi4 zaZdY_wEa@;J(8b9{&uMDVT(CPmGu;Am5Hvj*?0>G@$7O|rTn)rY4z@G+q zfkDRjvDndr6CnF({cTfE{GW?E-&5WvIs`)wR)GH? z;qY%Kc6joIbM9Yxs{o8{4=8j9ss7zq)qh`f+Qqr>uNU0|UTPcIYrm%YcS;}rzUa&= z=d%BB(cf>2jtrT) zZvNF~Zv!^#QRazYYVp97V#~l87_+8(%cG~vv3!9_txG>v+ zK_j)-%r{S+zwbx)`UC60@o)tmhk+4)mB6c#Cr{DR(0)!9aCCQf=il2}ovTWYw^()~ zizK$8=Sn}zX?3=r!Qt&!ul%=7vky!G7x&1Nx25?%uBI*9 zV$fRjT3wY&{rJB(ARVOc{G*c(uIg-O=N%tw{AYL8|63{hKff`A(}`#UH*uBz&o@Dx z{Q8k_?sbO>?Y}o&zTxl8YGrwvXKhydNg;esiGYSo#I@6%Vz__asKO=kn)4MY-#PEX z>sA-4&D#vR3rxLgE%Ym1ym^>%r(ZOgv5WuxRn2o}C11XWnuVM=7d?`Cd|ktW_K$sX zb$AuvB~4@wdNcE%yT89chX5f(I^_4NMxU(@!f`VSh2#&heWwNRHM~;l-4+iwH2c>> zs)21~4!sk2lGGDEBPozU(J?Y}Cy3FP{>BG}c zI%=vVGShmp9!t@{m$LPG@Yyzr%gMW;`6%L3w7-u$pR5;7nQ=qN7J3r+sHMg*q zX#aIazl7IHcAqx+Tb_EbVZhO5pwIa)-qnFL&U=N=JWcoZXkNyZ7oY^IYaBECHIGa- zPaivb`(tJ(r-)=U=bE=<0jLtI^OQc(=y8gymBOQ*#kph|T0x)FptH~zIuts?U* zUYgTAzz!Ra+ZOHDFTdrK@NI*XMpqtTiKWh+y`TYNOyYP!i51rUY}kU@ zi>ND)W^%$u6EN1T@O6drD!~J=-8tW3{yf8`ygm%4Bi$$_OE6C3+41j5EJ{Czo6&eWy0$I1lT;c^tnoSm_?Ow>?o?F9LlsRlB#AI5Su&ipQQ! zJbLTsR1UCoNCA(PF>J)z^k3J$qRALP+nFBdG}CvpnM~B-LGutke57m$8Dv^^T9AMx zk)!dzPR|P#9my;SshkG^YuK9KPk7vA)f=-So*MNJ$US{uw=~w8WlN%XyT5DW7K*=R z7RfyQrC)fXZ$37=Euk{Xd&11f@nsxZZ?5u7VClW{ADPE$sn?)qrF+nfT5CSLn+#|7 z2KD-DOhd}gosLo|uh&lll53>X>PKmO0Ac zsgbW^jXhbuA8VVqz5U}=AxE;70|BKAwYJ*)2_Ws-EXk}t)wd<`vc^NXm$Am(8^kY> z{3=JB&#Jq&I^A@R{oeh_Ov8$loora;@a&3a{G}3$j#%c29pQozY#BUc&2x`fY(%Ph zxp*O7$~ikpfpY*K_N6JOdIH>03w+sFG>TID=F$$67 zJ>YOp=vp(M?|029SO;A!j_j8BJ!L)QMMrF#dtOR~uqZUW%4VRjD-N7x4f+-*YIplH z;L(149l2No>pRv}<_PR)J6!y->jJdGRSWF{&qIub#tDerW8#c?7k{f|(X`oYsH%L4 zYG2zLe>(@2&+mKEx9sT{u)Ve?_nFFb*T$v~fgvx}hvaTet&`2*rv~YBdIn}B3Tqv? zZRD>p*!4?~c_VX-gPfnj3vh zmyk5qSzg>`{!L%0H!r-`{nWdDK(Z|ZQqC@h!pxJ`>Jg%SqT^;k>g_B{Un^m~bMMtZ ze=e}lb56+12?m_VgRW0aSz05?PSZWt{57UoxW&KfxN5+sy0iS+?)BRZR>%~@2x@=1 zN#?w+XH2+cQ}y-J=HRWph)sT}%gxfEj_K5c9YFK#K{{3jVWZU+_Kef4v-3V&L#hA3 z+=KI=lOM(03l#M&zKx?Gnba@|gPieo(ua(d!>;VA39iLTqmo1UEzruUopzb-d=F;m zDK_RupiL`Ifs?_<;e#JKe0ZBfB6wmekFrFvPVDlY&} z@mP~8&x7gK!q#2^p~%;2l_d3th9Qv?J0}#FEEjEJnpMZ@iH`*|Y2Q?%cBrc{F+$*6 zvndONU%^R%%6D91PoMcL-g8EoHis__EbpDvldBku_Srrs>yVcJl-}-!|Anze&0vi; zKbr#0ZnhA-=FeWZs+q))(CkCwD!3=y`;WHOGRDRE9?8fZkiV+*k27SnH@yN)(7(H*KRK{do+$q zZ%sgbdpEL!9;E<4*`gs@B@iLESmshw6WYNpk}{HTWc5)>WQG24OmDEd2e0PYQ3)DJ zJ7bynJ64ghixJJrf;H)1J%4`Ni+XrAaQ>uxcI<{Q$p879>b3`v>_nlhA;%Ec{`JJ3 z@+suk(fXCMw=d%aTZKI+zI#!;GVWhHd&z~7-|Z=2VIC&8=6iBQ3gnMn8NcppPlu77Dy&Qu8rU#cRUeJyo8I(2J8VXPN=wZh(X z91Y?WNzrayzRxoHJWXNaYh(9%yN{1qAvs|YMG1U25sDxz?i#~^|E5$SFiJqw8l0}{4Q2L-NjYP{`ujJX@^~d^% zmO+aYug7*(d9XUSt7c(SE<3ABKD>#2{97=LLy@XO$=_{?E#(J!uYgC9ZnRV1pimru6nz(n9HBO%h@p=FE$ z7v(;=X6ME5aDC)OFE?nl|8au-!1?MsM=Fgv=W-)YX=#}e2DbUH|7AITbT)JZU=&E^G>A3UAxWjn z*u9bIvWWZ<$kPHtsnfncq?_%{5|M^UXeg&e%q!Ce7;C&4@tr<@OwHGjkBt{qBR+^o z4EX(cO%G1Ejpq_g>gHtUbeEIpKIbuW(|J$PuE23HlGYltKDbV1mOmE~)ox_>o>5_s zM`tQ@soHb!0uC`YbqF+FlWP}z_!|7f3BKmqjV+k1*Uh9YH8jNTL+X`Xj;B+h%35`g z6YPl z+b2DzL6OBNt}`yxY@*=n0?bI3_>vb-=!*n1Kr|MNU&Qm#!a=Dq=cd<1b;&_3i98e= zU|8bU^6$mWMuF?Km@kTNsO}8ncYtoccF}9`>Vm|N1GiRIdwJEN-=hE|bFjEQ(Zz$F zTy~o!wSIk)k(DIq8yMyfV6*7t_}O>W4(d?Z7xCR@JRRZkuZpDs(FL(K^Cm{jIIUO) zG#?S#`Fe8i4u#8C&F^0dbAIY~qMPXQI^X}_-?I8kIh#vGnzWhg9%{IU*fE_CpkFqt zu8_dc^%6^MO_RCezHBayEhmo}$5x19TfMBGn7e1ei~>J37wFuVp7!0E@>R=z7@u#G z;u`{A_|TZp@2u3uKuIUr*ETNBfIZx~KZLtAJkQOpcz*n{)?N0e67eIy7O(DJTOZt0 zEVCJ}>DZg7_1OeAabJqqTUM)G%b2wb;$l)6ITYISi>>Ssz@*+l!4BF%Zo61MpOr4f zxPa{Xd%2L|00OcOkA<9MwKVkuK|MfB?o-`O(qWSloqEdjxh24AWFOZZ>gJw$(9+mW zcSg+nr_MjDMe~uf#rNcn*M$iUc#9g8cKWDEs}|B^8pan-+ffZvHx;jb#O;BbuH>@^ zGDfeSP*{3U*c$Hg-M;lv!Km{4s0jJ(2ke6vcM6_2hCu)&D5n&+2Zq&B`8*#O#ZO?0 z?`?av3RnN?D>C&N>&|hP*L6ab=vAtCuoX(oe9K8GPR*WNq@ZiAc&*Jj50p4&e4*8C zj;(X`iOO!v*IuT!C)#uMHiduQh_a06)h!KssqiW?vw*wMXRfE*ITu1#9bCRNxJCEBl6?4QNF3aB%9TQ5*zYhxO3r@ z(Jww%Py1|leIVdD){E@mzO9AV#NDgiSt?phMnc}Q_O2uZuV$H4SA0SJf>}#h=su`p zntg()3A?*>Vp5cTX~LaNDfwdU&iz5zp-ZlKxS5JAC3$+CLMP z2%H=UDEs>1s|n=$vuI~~a|RHn9v=u+v`R@PtULhDmW`@Vh4+?yZrV`s9qv!v?ET>b z8uFx|!b_dTTJu(oWK_8xb07oQMj2L349R_)o*54Z(&n@-euz#;1ShhtSB!{-0g{#R znb9L}#zDR3WWLZCc3+ft%A}1Wbja?M)`9N)lx@6MVN}>Z_Cd88B}!eJjd36H(Jp|I z43125l~|S3dae5+6DJ?g*)@z{C8966%Bc)GT-=_BY+6e7-6Q2g&j^{zS!!wexS*!A zUQ5^#8tzODRnD)qP5rQFmrN+PUKJ#ZeP1`K{7EPhZW&`V<9!_ImE5$`r3&mwf3I&b z^b(#+{=)fAsBG~uqE66lHz0_AkJX@F*nJgP-P;E2kM8}R5YEDgo-{vOS=UxH$d21d zph`-85i*RKA1cwsehCqO{iWijqUp;@v!1W^ZIxVM1ep^Z`(dkyu3Js|+sD3! z>#F;*v2+F-a#5oAU!+}ds3`fi3eR*9QEaC(ABjn74Lk`b*X_3!AB+`lCpQMsd&^DXX!-0G~-1dULB)eQ&iFDz-~O zD~+x_zkN>>v;rsJ%i-){6)gn({bqnw@Im>7*jw&6Eqpop@!M>f?veBzWiERf6MqGx7L&d1;v&@oY_*uiFzB@6jSI%E1=nGU( z*C`XQCSa+f84B24GdZg1Z6%ot&hV_;_#XbypYEw|11e76DbHTc;3#;Au#z`ze0&URO>;KtVfI9wq8(i?!5vtGkvsDz^604v#u7p zV`uZ-XK5)VinPcFmA~3X2Y=q~gl9iYnbF!JQr!5UKPI_UPq96Lc zcU)!)F;C6ZW!f`by9?8!u1&*%Vi}9x&bz7Gzx-NhkxntzG|NZSzekEG-3+3{cFSrF z8C%)iyLtV5BV+VKbk(*~9W~u|utnA#E`ybtFexy+8Gik;dW>6A;0MSQ-ed0`Mn5_a9P)$3v$sAbFXm>j36lv)9VfAQz&3+n@4FcGq0Cz zE;mbe&s_be;+BQpA%^a4M(p(PR8FKpgBecIu0Fa%IcvxMa0+H0Ose+idj8lfNzVRG zc$8PNoa0jFv2!y6map7Ib#U``$a%$PM#n%QUBrw=Gfn>Y6z}Oh z9xH-cdGT1j$}%L^n7`Sh^sykw70bwB6v`>gWU%x7?Cngi^;e5=qV_3i9)p~2xsVAi zG3$-U^t%hvZu!bVc;$W(t;|=aY>vO&Ui5v{^1J|q1RD*;Dj=A`PMK>5d0o7l%j5(U zhsw4|_NRF#)TGq%%B?U(?+w1hUH@>y;=TM_B^?YZM1FnpqRuA1L0XJJhHtHW_}x>< zqL+~@aK8P~dv#2=TN{O+eX2CkRv+j*t5#$BuGbiL;`*Lx?V85=`dqf5?J~0(!{sw_UD1e^5YMGJBfMLW>WIABehmB z&E{m52@Atj&Okk*jDfVuBQ3wIegu9|OjX!B5ii+AN{U!}nJjLLumKLOo?rZgvHPBU zTa{R9MakuJ47<85`($mNeARhntM4ZH8v-Q6>)}vr+2vhZ`nrCZr6DIiN196g8&ovut~^S(RBh+@<+Lp^Kd;ob zcj$@F_WG#zglP@$l-4%Zvj3_4<}cHbrNi+i@#bpyLvP7|qim|+S(AX)R3Sbx&ifQu zi}_MIS-5*cP~;Y^2g1p-#n3_NkjTI-6Jo)Ju+F8@tbxmkmD*m_a~tu%%4<{ak8T3y zHXM1c>0z9|mRqD$+mqHlJ(0my2Hwxg0W#3&GSekxrG#rhO+7aBwSWINkVn^rdvqe8 zYIS*C@5zg*J2A6s6=0#BYu4&n zM7r_Ywj|@5hKS4`k8r)7v|}36p25;ZBg54ujoXHOM=PM}Svy9Q>nSHhre-5WOpj2l zK81U^Py}DwCh7TK-ukh(p6R$aQ*d}bION>Ry5ywESS^R#(p0aw5w-@=~Vaf;~J&U)}BTgirfKi1PUPnbFJbjibJ~v`BWH+C!1O2~B@eaMisOJqvQG^l9nwxetdH*-cGP zv%%8sus%D|wXJ4Cce5SZrkeMX4+^XRYysRad^4x>c)C+07ZB&Y;Xg$z<4SZL88U$%%2lw3;e4 z%hOT5Q{z*}^EgThiPtN9Lqsr|yR+!iy~C z&nZYu&J}-1Qk8xdsQ*~id9Z9Mn44~HNrf@LbiM*s_;7nxBBXUa2Tw2l)8puK3w(%J zwh`5BiIn-Rrjv~zw*Bnx_Vi0BU;dFk^-X+e-I2|` zxt69?l&O;Q_&9T8e~Gz4oOkW=-bUj{N)dFArUoXdnC#~MCQ-u1$*}obN;UQWaQ4=5 zRdwCku#y4-N;fFof^-YgNOy-@q{~gG2q@j1N^iQm*_%dk(;d>%4d3#7@AKSxpL3q? ze!t((Kh|2hU2D!U=A2`Wab4HE8AYH?<@eAH&yZ2(LhUolv6)`80|#8B85RYoz!=S>k(+mf#ueB8I4rY( zZm0Qto;?E6zuSerg4D>G?_#|eHMNaIFz?L260Bm=O0eai3a`w|aGM~-R)IUT?AK{@dpK5}2>-Y9TK$6ze(+4f5aRD%(h3myh1M(W3WmcwasClC6s}mQ4d1g`|}i|8d;mSzB*Zr!*1=+tXQi z^bB|>SZn=WH2oq^W8l0|dDWO+)BS9FouNT?z8#BfX^v*O=gATh!4Bbs9QQX}Y(3Q$ zamk$*y{*4Y`2CfC5KVnEE0Fr)v#q?%T}SL%l4LpPlEVrMu}uIDm6%nE`H(t(T4Z>O z5`OJ-I+1X6W#wmZF2G55+#A<2Bt<%Wi!Jiw5%E&V{h^NtkmGw3<8|F+6YrQ%n;~u0 zL1$r}NBtgM z`w}9&%vHij-fMzR=~uc}FP>id4Dj|bv|=2hz04c7*Pm$sg{+d%*Pd?eCB%<^dXsX) z8gkHl?mawVJO6{ls6pN0mFKdUYQ<}`*1AtA`s;{s@AtDa>vvl!CyIEG9CL?`2Bf}(PJHHI$?{k zn&OO!RiMtYwwdAWXnyKx*amvq;v84$Qxt4k5uh2EQB|)iorJYF8{|a+*|+~?r%C{b zFj1)G!i6e#)hUe?bR&^!)M%{W(s1YQq52x;KhE@>b!V~!Oz$Sp+Tvs$Bu){)L;LE> zZi|KL}|Fsn|Ohd;lm z9Bs7OoOgrCbUfM3rF-!E%e5FFgUh=syK!;3nQ<&1LbH6LlQ&vn0;Jfwv+VaLEAxyR zgiQu@^DwOHXm5vo?yQDa_`-XW)QnFzBvmbK7rdn@Mb9DZH=Ax2V-=vDR*`1AQ;$$@ zM3k$XY20nz_YCt5;9B~m)!;ZL*-8RdAWUPKHA@s^+m^)&o4xJ8U~h;SyztsB_qoU! zKKxVyHea)0(F9 zD-$G<%l~@&P(wK|CYGdhM3=XWj<;pf$Dm3yZ`S|w zt?r<1wM2L)f92*(t)|FGgzgV4hZq+c%>_fqyW*mdKZffHTB?vk z2ZOwsBcDP1$`SUe&!fIA7$p<~&(fA3yd9gVSSxh7Cb}M@6m%V(rG9&rV)v`k%XxV& zi%^@(!EcuR`irV+o+7!$OjA?cgX7s)T zPhbSV(n!YY-bZz45*#RHf?_Hk z+oI=j+{u<#BfF0+c=d88oAKySnzFACpE+CX4q>#~gV$o}Cy5aSnHVE4m+!~14gnW6hYY<6sct>9}Y$g-VLN?Y1XA=d*5C;Rt{kZO03ypj~bI+D+(fEdxY0p!2Dnan{w4?RS48gfg-Aja9!>yg7>!g;8 zZRYw9%U=cgmoA0RA3wDk8?U_gUTE?u3Qfe7;6g%JbMrc=wjG;kLM+~KIor#lctn?Y zXq|(6^`_vEBnRpE-ZICt##|n6qM6zrOgCyBZ46S|%+|zUr3vs;I2|k? zCBQRwv{87qVZ-UdYBg30XXFixXyr}iPYceA@6shTXPXi~94vGW`#)GnVsHJ?I^8+) zDE-f|gYye_g5=5~_gfsY6xMcCy>o;2;35&oMdJEvJ|L{SKR;=`!DHNk7h`Ip$c^Oq zutm*)Q%izKd0RAp60#EW+7`_@-QPyD^GvFMR;9v_ef(&gatB)0q$dh3Z?F}um>EXX z@8B<6g!U23#OueYfb+8pKeQ}M^+pM|tCFx8HG+5E-F=;aT97!6l6w%G`#neVskWRP zb*;G_+*@$9$^Wo7Ge1ICbpV_#gf*S2W#p~HreF7jQz$XS<>Z53BY#s>uocQ+L&@E(_G?AUIvbrNo-P?L-|D zrCPH4^=K|Nwj0V9jq?GhRtIactveT68+D%W%*OTg1VhK{dZj{5NQohGDzK+>GbX<4 zg=I{|e53%76f*9Mldv1){*x-J%2#EcQQ7n$;gpwF@gIInjkpL74F4)w-CW`bQ+Ns< z=~+xR1yJ*D*f+Y`p#w;==u+;$GOhYixX!f`$_y8%)mC(PX#y$~mrezUq5!b|hP6aO-`;&)4yT#2c|9-7X3GwUvH zXS+Jy7MdNm+kGy*E8UY86}MPF;96K@LW%iTwKnri5kfbaR4wI%en*ewI=rrNiXxk0 zB=QW1Su_J~`5tx!!KzW^B89>kNH=Y|_w)%*e zu=GAJFxv>-8f}ooXy=nDR8YZiVvz1YJ5osD;eK0V{voD2Lq6JWpJsEeX>;NFCY=?B zcG>*lN{aKoqrXvF6pir$7Rd{$B7{NtG@?@Wf&gPKs6ys7t+TMlhLO^BeG9hwed9qB zYY-MC)`#8%9=oAA-@^qj;^#tjqwwiRl<+9>$$UEqhIixcOtqefY6P0IPMxveK=PI+ zZ>L0et3;h!Z34D4+2CRFIJk3X1A9k@fL)h?vgy7BOfqs%FqXHz?;5iG8J+XxAHDrL z2>`;Xp!9A3UQ4?{Q&2;W)62~dFyZr~P1jqh;ZFoIuW~3S2lUv&uB~K(?p)eDF<88< zl^V~;<)T7E%@s2aA*gZzXEw7v4ArWQndNN$Xd=F)29;9I2M$bS%PX}xO!pfFR&TYd zQEylZR`(TD_vjuvck%_VAA)- zA6w>{%}a&Oeo!ARA!2{3;%u>BsDJi=luEqU01-7`{sUh zf$9JNQswxxdP^mdZ|g)7%4Kiy@J<-cq>?$|+W2uf>YSJ63tcA+Rf@}7VB=c?7D=DYy)tN)@wcD^7d@4;eAeB zgQCXuqO&9eHvN-D8K?b^PC{v-o1*DBjp^;WcUM^CACB9_JzjdkVBzF3o9SLs^9DZ8 zt>R?)w!CJGk0#(of%j54wX1FcU=?S5L=f@4W>Hjj^WfQpdq&hhJ%rW=mVIxkQ?`mg;jD8DU{U8Zq4cl?d zhAr-t8A^)3jorw2v}`rPuytnj7v&$Hn&Srt+Z3(=$IL?7nr*6(4%MwDN*F&M2W#gG zy6kh7^TCS^HhWjtBcR#EYPI2bZ2GUA4;FGX`+bt9sz)r19N~xiQS*uNiqGbmN`ifx%?Yq0<0$hkN{_5<{NIV7Be6%8>;cVNasBNh?^hT z2BKSCI1bS=G820@S~p6n&RC=Ugp|LY)@rh7+qo`zeB3LNX0!{B`lV2Bts^Oc`jcf;knS?sJ0D~B`- zR42N`(s;mysaWO2&-DdOO&w((f_=y#Y^T_e>Yu>)5^CJX(mV(DoG2b8YYIv>tK5t` zQr=<0M!5<)xZj$}poXIEYx&j-6|y`#Qn~gN+@_8jv79;b@>`>zZ0$o+gn(1c`PWr$ zWA9$_EQ3Fa|7I#NgaSV?KLq4Y!HeVHajg;mm??g(&0>W#Pvj0sE&L@IN1qpmT*^|_ zquDpn@tXW?wjQB<(9LcjESlbKl>tzbY(MybW~>EBc>)G5Xjq zLWGEp+j8nyDEBJ3I1LRU{>KJ9eJYdp=IaXY=_!WZ>pv{@AHMGCQ{Z@@=lTP`69Im+ z@qhp2|DU$_a(D}WGnM+Mjh9LLRj!9#yh$_Y)chj>^LsMpZ`ZzB2nj%k1qWaLuZH`( z$8-NGYm84ZTjTz>q4~G}{9m`P#e7TBseQYP{(Hat&rc!+e$`{QcL{5P|EX6B0bgYv zk(n6wPk*4w{IJ1Yu~@j%`5*U{G(ti^NxJY?Q=b1Dn^;ti<_+bRYy8hm{hz$X5a7)1 z#Ni$P(+?A4qOsSRecf%xv`fVE<)3yy6OJ!rGt}_qOUmEu=o=$8X+52aCiJ%{Q-;g0 z(tL=ffdwP_UseU?V&%hrj^7&ag{}=-?TZxb<9}$X3Z0nN@ZV#-|KxjAIlyaDM!VcZ z|7oXx=6uNH{q1q=#6RvoIb2{k=X^Q#{tqAi$6q}G0qJQeVg|@x-BjAnyZtXMfInv? zs=wX(#nJ(*j2!Of?SE?b@57F&@7F5BsjT@Y4(E?g`ppi&bH+CSRV|io==vY`fjU|? zF3@f0v2p+SCvv!T3P3}s|1TOcgY^E8)aGkZ#6R^?)kI(sV0@ff`&Uo@tvmmA>-YYb zF=t-u{O;Dj`RpIt`~P~L{Q~e8H6lK{f84txz<6W+w!)kI`VXqx|HoVWKDdkidXd2Y z#fyGv@GA4jy=Dx81zX&d5fpzAaYz+9Hn!RmNj4CbLEOfEePtZ>GpRSlH zK$mgi4P+eDGfejU!EIIL@FU}|kpE^yDz6Yu=D5`6r#`33vv)uhDdOdBF;xwsDL!s@~-373hi_Ss8S(aNK!GK&!iRasLtE}YP3Rw zImUc75VHc<9YkKbaXj$*@Kw=iw0FJ#b8@uI7b)@*JqeR*xRFo2Dz)ffBI7qM%Q@|{ z-34k=9_!Gij}K@8hzMJ~<0gUU`-K)SXm*hRi&S$8Rqjl1ry8m)%+(ZsA;wo|gZqn;UV_?hkiYoj=BO5&lKTy6yTXl{>{YPH zg@|(UK-|C2HU)ma!oNU6W76OH2JfMUYmZR^2w*e*q6_IXTUM73F&JfioO8I|%aXe? zfT%UrsAzltyhG)@yFT=WCp}3Q3pyiz+tYmhX_)A_G}m^SRtB4afZN50bhg|C)7hH3 z)6HsxXl`=NCrg>CYufQK2n+liV^znJ+nmnonpU-T)1NuI zqH+WS{Eh66;r9-vvFFC|^$}=rZz1tWAO3fkX%E8vejhC8u@Tz!wvTaZ40gMz4-F;G zv8pXmZE{8HjpXomSgZv=r%5N*2dY&vDqB;u+enao2k@^cgP6*e zo#eE0*+c5^v>)u*;2+-&`(hcbj@nMoi02=0Se`I|_1t>6__dAF1vf)MI+;}(D4Dr}vn5$kGOrXR^|J-u-eb4-Kc#&Xaer`hDHt(h(tm5>EJvc-J*7(2)1TcX( zbrz8)1uWM)?Phmo(C5m1c;AKY&eZqoi|bl1XcG&&SMf`epU?ZWlBp+@EqWhg-pqSY z@Y*WoiqDF=qRzW*l1FQ-Du+>?Hr)WMC^K@bD$cek8;;>E6+uRap$RpjI0ydTeDwvx_2M*{Nr6d7b3 za=H_AxSF%*`%vIra{h|T+cr-7W3PkT^Q7ud`$_%Fqi(sp9l`#N0F1YL$1M0iPabv~YP@4!wg+YlDK;}zanHMhe~{g?8I%58M5I;czonnlBR|j{wuBe5 z)=v}*BU$yI6pXzEz!9d!!poCB?5pMR?lcyYMM*kFW$|JvB6l9-*syTfx_XdLWGcUt z+`S*7!f0s0*sl`V#x1#}!gT-Yl)T3tNG<{EW(Ythe`&EM=6oBQE^NocmVda2>}uq7 zf6JE^WA}#Ea3esLl<0X`%6a_pCY$_4gUhBIMT+Qhc{I_V0*H z9~o(h=P22#grkp~fm$*3b4b4njJ#H!@%G{ZzTP?j^Y9!m$w42DFMK`t6xS@)y)8x& z|49%wc|3Ixc1py#KMHodX?KW)Fg=ok5BnDCHlTaZ$6C9v{q2iKje3Apnmag5{A^%r zOtnf|rCtR;Zo0z!C74{)eZ0$IZ$7VfiTax!)~@J#ksp)!DxD4mO-vDolsTU@^z}q< zYT($1%1GAp4H}~F4>*G-7p~d`eC`l(^+QE4Vq?O!CQgAL+tsafA z3YRb_&qBRb-rI?5u`kGkGfx=e4xb6eFyZ{{$i!M>~ zigZaRS|(A?uvHDq-*Z`w=x_5{XU2DIy;-E>_w4HOIGO+6`4CF5a=tKg&C{DLmpqul zLuyMDa^XrL(^OTguvHPQa!tH1058^~CuBF|vMJVRj4!CLY23+94$t{OGD~zW^i}R7 zivV3&{R5mr_BSZAqFGlw`hwOIi4p^Ex<&#Kz(ux1z8zof7QLgj;&{sjL;+%|V;{|) zAoxWvzl&R{w7U`n+x>i>TIYiZg=RM|-n$=*jRzE_(2j>1Qzq0BS>niG#MkDFHbCTJ z!J{fxA9(4rJEg){gx=Y>PhpK0*z=h~^OZq=tqN#~b`RAW+J+KMp?h{Fxc}l93mwD| z57B_UyHg(Oul%+jjL+0}+F(Hy1LdsUF3_*8XxnqhZ)q21LF2(b`Y{FYS!a-}-OUZ2CL%!i?j zJrc$33`CaU5>I1koe0LHdqPexpG>}wlYv$+X5NcgxGTWdPOac6uJgd+6Pey@Tf-*k zT|Q~@rdUPT7V!%k41rc(HtZpKw;Mk!#K;&mhqryE)s_5dt zx#5iB+(EW2O~;%_$wA%Rj8D8Vc^67Ta2#{;RQ7DGE$_iIhMG*8fKR426HfibwfP;5 z$vOvFM55kn`FN~`AxCzfyELLudT1 z&B(;JCyVI=&NSs^b8$N37cKRc~US4%KclYqg{f z6+{cLNA%QJZVr$H530t0(mVG*%}O}Ua>)=;Rppg4?p#7K*erKLUl#u(c(RD`^{)Qf zZ8x*4D}w*8y_i-HzX`GrW0RLZBv6VN#BkB9Jz>sQEI-yVgqE2N`fk|ai4UZ0rl1H6kZW|wSS7bC}{n!3-<`R2PYuAzYr> z0bF(yqc`5U@diPwaTMU42{o&hNz21=l?>jJc{;UE_gv}$WQkrUBSFv=oNiLXG3DYA zy?k-reMHwZ0q~pj+C}&N1oJW`=rMkeU2NUbSB~TXqx3 z0nV8MJk0+0&4+>C1hnxJ)R1Eg=5EX0FF*gb;2=We9&$Ooy5iiEoV2m`31C?oi8YM$_t97*?Wy#r{R8_9Qq;Ra&t(kZc zpA3>u;F!R=KT#`ZQQJF5n@A|zs7iR1fwYrgzgM2yC!t6k(d z&XqbPks{VLxPCuTsNrZcM)%G;$je!S=wP;Sm^aV3d%oG+MdSSh=fKC)FbnM)(f5RS z^ZRb!%cbeRiKPg7gb$<%!q=dQO?ogK7WPd@#RzK{2`^{xu>>kspLvGYyY#^{R3|d@ zpHJP%Z+AhMSlloZ zkV%;Xy7zZ`l*5kK_tJbI68V@g zky&}vY;5r7GWG5>Iqd!76o=VF2k-v;L-`S_s>PYA=AnTC!+urz^{*s~51v85V=B#8r`hQ(Z~~Uk9)?csXV_4|m1>EHjTWnPUZ}Az zwGJshs#3`5F}yCHMt9AZ7-i|OYn zr7dT|ly|ew^Ge#4d(4jG;o!yjAg}Ecw5vtdM6+q|%O{g~+EuOhch@baMOL-82q3P2 z-x5f`uKfG`ERZRg+0hhv`qzM`eh@&*QcI;%@lKO4)h#Sw+}96r{UsO~9rC>AX*Eqd z%v3&8!MB1wF_ukjMqjtuvYH)ZADOoqbZX|51n@6{jkGa3(x^)$903V~M=Tu09v$;^ z*~U%Wk^EooH<#KGq2%U%ACbdMpHLQA!As0aI@XzH(2#jWF7)J#^!2lBbFwZ5NlDFn z@r?|+L^m~bLOj|6vT4goK4!itBM68_7&flg>#kPIQzC|#t@)i=@U=^Oyb#YgQO2mE zm5(QaRNJrHlcSFTk`RTE#o%a_dGQV$5=ytdm3`D}?rVp-JHI#_ylCY)9XX@e#WDx~ z4^>)ghpt0&{35{VZv^`vvxrzq`f|S&W|qm<)#$h#j*z50l8a}u(Dt=bXp%f20Q+Mm zEG8m`B(TKe_iJ64YDq)bo2xqe)O6!O%3NYw7?KU^iVZT$&*?B&Y>VBZUtJSNOBYM} z%jTP7YrX>l-WGW)co9dip7nOw2JR%HcJeFY$BYIIp_c*IMkh(M5UmoBq6`})+%$w6vpdZdQntzx_m?G>=zPmn zxe?RBXcySfyfGyg%38q4B=fHg)a9NiG=$4)pT{%nDbnrS%f+hFCD$16R<-T1ckR@b zJ8>ef-q%}wr+P_Xu8>k<8;ZM9u4}UeFU=U*qpf&1Xt^jw1`*=nx0%MtJCP)r%au4x z5j7HFmhU;mv!?PGpD*Ut@%S-a)cpvS&Pm33!1+I_Q-dS^p)4Qku*4`a`bBV=~_(4tI0Z~0Z4XY!&ZitA3iP8 zWSg^}Dr&INj{mcgc1fz(rua5FWCY05+P7K(22_5Wh0gN=suFQ-{qbzvl;v4HpTGqA z&m@X2iZtCAAepI;bJ}W-0dKIZGD#T!FziBW6hxI&Bh>QZ4v@HT4T{{nc|E*D)^9WlWwBnT|FM`k`a84 zRbI625DFZ0eQhH{1IP-Tec2yuYk1HP%|^dMK1Q>x*+bB(v3JuG6Q-@hZ;kK2n5ef-XAV++S<$Y4+GO)0;nA@63zAAl?9vr$&wm(yya#;0U(d@PZ z`6rWx*d36Q(IjYH{I;D-9K1EqkcEsB4mJ!kdOQt?Y+ZuyS3zNp2@VP&^Ch7V0 zO^-b%R$HF2DkX#&E?(|qqR>y|aCf<5yBeJDdHn!+mGKOkNKpR1QY>v#J#a#0@-7W; z*--m;B`cc)0fS(OH+WI?D#{zw%^4asI`0isV%@K)WdyuiZ1JLP zg>EQ>e%hgAV0c!q{xF>^+6dqBNv$fT#qvzN1b+C0rTRZGTwUtAz>G6G$_~lzpC!F{NfM)T<%HKAzt;e5v zR=Dw8dwTFdN28~nHrk`1U& zC#$EDx%vpY!7)LO@g(nB!E?aSnyhTH$Ns#=bTLrZANM7!_)L;BUhsYD2Wyff4g*#T z!Y-Xr9=qv|E_UoQvi^zZGj&v&>+G?zW6qczF)|=E%sKujg$V@KRh6@XaG>#x3 zSTXb*f%*qZ%X`#7EF#eoBUU=t9rIkVt$B*wIxuZb&bSlms&#kbLwb`Dl>Qd6Xr;JF zdo48H+C|UXI5N1i_Nm@wmbM@1>p4nWet*X1Ncz;%RE^CcazSJtgGT2#NBlnd?qK{k zp)fHp)evnF1#4}S{c(|h7@16RXn6s;EFaXEq@N2iD*!)*hM6p9tT2Bsz`Sa=wAZ6h( zEfc|tQMS^6GL<7YQO8l{V`RJ|=CPm*ixTn3ZT#oAi2DKf@=GvF zh>@Oys-Zzt5DtSumO>TJ!yS5YQQS^*vflyIM+`FhUQ zbZ@3vZt1FuQ4O@1-aJ}xe5z_-CZiW@WPS5WRCkAf#q*gfrFcjALa`M_HjpWAnHr{h zc9&sq6)S9)xvDSFvLrw{S#Go=2xB7#cdEBOyv-5sMn8KC?k?0o zM*>rwEe|idWbtQr?SX}MTrK;)uUuqPfV3)$Gwb;{L*UMP+`Ike1Cdp4l9D1;ApYlV zIn#|{opSAZnMt`EPY%{iX)J1J9$;m*t6Nky>tjkuyfll$JQ>2Sf@)tD{~uRxKHRSi z!DqIdAG(QWH(GEQ!u_r6=~pEQl(efRGCWEC_0*3`hN=?k5kO1rya&=mHu2VwS@wS+ z5V^)~ zGxRv7#E6EnZSma(V@{hZu2r1+Zw*`}g~4AjNhjrP!>G{$cHxWO5Xz=iyz2d^;y75?^iiFWZb zj31~MfU4_Utl&5Fl|+6195`$3qgZOlN^$X^dZHa zKf@&a8Uk$L2r7TbH;TPlEYz?9%erezg%X*xuJPFdl^Z-&MP;a##(B1}m%iC`$7rE_ z6R90Utggmn&Y;0n){HeF*6Jk5v8zkQK-*##X_ZayjI2U({*#%hSUL7IrCriq4!g}n zQm5Fj;>MZ43(8jY_1>j+|7;LM6RNQf)CI!gJdzwiNCa~b%~DRb(0=Cg_5S#N$FNNc zLiS1QCi98UT4bZ=;OTmH+~!Kv9d8a zo2<>*2G?f|9M2P=PQvV`8jRRE%gGa#?Ig8}?J5(1>|;*6uCVED=kpKOXEudG;S+B9 zL$z3Z=s$Y*=Nmgq+6^}@fU0q5$cG|!Wp#g(Y%{lszPM3sgOKy4%Xh0fSq-+KU&3nJ zTxw^%TlCC)J8rpS<0hdA1`Sbix{|D%oGtvgW`4bfZ;1Opx^Y(R6{nryj5;d z-X1p${C0&Mx3eT;>%#~!*ZW8vfX@ac-AF(4N#?S#RMfI;j_1X=KJ9E^+Zd}y3E(c0 zoUbtLM)!;3K|1Vz=DMgIyr@AEG>gN2@P^qg-^iM)&4*ckvA%K+NDjN*H=C%|tAY$# zE&By*s7>+Ba37eOnZ-mKydt%{J?Er=mm09g6{>T3T?Ij3K^yd1#F3{TYK@WH^K;yL znfYk4oAvb_WDp}qf5K8@kaAn_>!f}B&DpFSNol!1w4EiZoZaby(R(BvNWlrt6{d^0$!!#1 z*y0AMGNY@*@Vw4Q0fvJR?mn`7I)C9wIo_kmTbsHfn(1uxIUt3fl-Rpep^7hRc(q9n zK5(mK>JEO-V%RnEdREJ7+O&I><;LFMTBnj&ezxd}RPF^u`{Q7zM?6@b*EVxR6;S^P zxZw=S`Pa|AbAbA^8s!{K2Arto^0>PmB2;Py)Q~yfjUjNh;q$SiKqDyajy$&HV@A%D zGQH8@Sm%{#d=C0_`PY+d+Z2T8IW7zCvvm@!Dx!6rHZCC_SqI76S5ld{5M=9s*ic{7 zN~p(3Ktp`L<&{Du-Ycu2bVh!?|qV;!MVB_5nTh7P& zg6Aq;NFJeU@Q5E`bjKP*bK1_`e6KO}Kj0x%0;DnC@|?6)zr3URoPFpO= zsrPvYUr-+6x7u@Hfyn2=?7o+yw|jn674cAT)`4WI?(Vdsc!B3fpB={rv!3LrD3JpzBg5`+`kb)BXqTBxt_IhD!C%6CB)1vphwqR~M`2s~~`7w!a;U zY%ZeFXQ)i;NM*%{=X<^6Sx7dWw_n&PY$jXn*-{T5gbL^4drUNJAz}-^-*&&P&eBw! zyFFg#jZcyeFT_yS;A}wN&o^OxA3UGT(UVeQ~oX@>S9U)}7g1ahFu9xZRr)S1sXN<2Y6NhA9YpL_N z;ka=aNDg>;?*1tEEsr{Ea*J~i(0D*D;QHMFJXc>MYLwXWf$;G-d)CY{8|5yU6H7(| zxs<{dF1^x4r89mgRMMN> zxOY=#5lN>GQ)MO}@7iBd@VQbMXSjpH>Ya`6{d^Bv?5)%t%fspnLnB@s41V4TWq$&r z3ZfGFbn8XhdvyR3GTj#+@!Y-OXgw!N#<8^xaBopIV^aR*HVMZ^Kmi8ZIrRs4vGd9| z@q3wA&58x872??Mpz0E91my84KY~^*XUdAxzY;}DR1b2V6`4}wYvaE9l1#)A%%brw z0M~QerjYH02|jvl)Wf)z{MT_2%OVQm%a%HVL5&Ce)#)GcwB#o%*82*td&s3slq=u8 zFIu!mgCyrIyJPP zC9zm^D<-`JLI`6p>Ncs%g7UH0^Hf9V5vTX9{UZI8+kee3u+)=T^%WC?idwXyXtk{n z4_d$|m3p?bd&6rYSa4B8ognvwPkP_xY_4X9gqwJAXL*z!cNEFC+VyyMvbk}!<$Zr*lw$2<_+zch-ttxHU&8Qra`}){49edX zfPX`ns#5;~Lx(kY@ce}W6eFQgu9Gog1a`VCmrkHvTd4bX<0c6JRVMdM!t(hZxdB!{(IpNVM%vpgPq^Oo1y{v`rgZh}fQY%BBHE0Bh&cBT{k8uWhKmH? zKpkbEv2{-mUj1g+$SX81y9Is43=vjl%_4={us5a{DO?uM%P)_5fek~9w%Rs(Iy+B> z{#brgwtT|SVH1E&i3GI%E-5mBos!HUPB&4ak_weS?O9hWCX5r4Z=nJ~|#JpdF^OPC2p6s&!>skVG<$3J>@`4fm`F%5bUx zuKiqt-A|WLd>7Ipr*onT(Fiy{XafI)H%&(1c5%Hw!Apn2_2;W!g3_K>BQVF4DGnn6 z?Emh_C2R;6KEc2l1YqExm3XAPjOO$3(!##GJHlp~PwMWn2wAjQ=Y|qmViv*Z+G4GX zD(0!1_nQZc#{-@j4(~odKbpPDDew(O-pb^P!^%3x5%;C}^yy45!|R|WHFa(dhwY7M zKVfZZ<*>+g)9}JI^my8e^)E-eNP9X}?i93E-}Ekp+m6l0b5BV*($o3J_gDPRC;6%T zUNg0A>%IPPK=(yYeR7JEvoJI z`A@waf0tGFdMt=n&Z3l@D0u~n!2RC#6?NCBfNa^ky#9t;} zH2BJ0{O^qx86Om^0BuiNJK!BLS#>K$E71^5VHud1+__qiB@$9YomM-$C8$#(XygznzclWYR=$6Fk1E&QPJ#nS(pi@j{ z=%%H0yl8v2ReWiCjDfzkTJPjfu_@?x?q9rlF=CcHTT?oouOktRUfKiX+%V6jVPP>1 z<={7@=ezUNnm_zD_b<>^L7fe!6>i5iBhG92D$$b-aZR_SKJqFfHtb@bT)eGUR!6HW z)SNc7481gtxOoBXV~n>OPQ%GI5UrwUquN`hp3Av%HR(&otprZzUF|R71!nsbUvOck zB5a)JQ(oj36roA2W++%>4CE*J6IG&_!~0V;%?ccI5SCNtA{|KXbVBL*X~;04mEv|K zr_;vcFTo|$_ct(6xoV7aAx6etYiaErLi-V{7ot%)?M~gp+1E0=2KOjz@YfLqt+<>=^WODJ*2^;e^T z6j3F|;mw8=E9mm)DC)sx*c3wl7`@`wZJJ%*^Amf(YX;|h9l&S1{5p5@F7Ix#e7AxG zP|nTG``i5dg*H!Kvh}5E^LAO(9ZXl6%d4loA5-GUC1le#hCc3Vz1e0KI8#b&F^jO9 ztqFS@MwDfvU6R!i5x?bMlX@&1+p@wEamVL<HtTEimOC4f6IY zxV8MXulUz;CZvr-w0+s$KxvKn*IaS&2rs`3`QeLtw~A?$s9J{@x=~r=H&#h|v!vw% zLIYYR+W|V2RSIU*eB@KDMz;Axvuw?23P}PJ6wiTUBI5o0QDyVk(!v;&v)#PrC-WVs z-Are=sGHby8nQ!wgd8#UGDYUsefrt<3xKx+FO>&`1+pbzSh)?B$$9wV3u8>iyh;{q z8cbR$Pk%H#9(9uVke|DP{;3!TUvXrEUOl@qev)(W5jH9|S!8DrR+2N8)+=URWZhbF zQSZ6bJe$Uv76W1Tv*;_#ZF$&G+5pdI{kmK$ceXs?!jH-tgJEup-QjX^%(5xYHb|yS zk5`6IwqDY(Ju^htuZ@+?)4{)93li!8`2Ezo^*bo6ac zSr$`~KH+DsC*zbEd@AYgR~BEw!_TTYgoO=$VxweHhMJiQnMM{@hQ|l})T*}RZtQK6 zs4Gxu4t#1~{_t6*)2UHQu;fx`!&kGfZ>AZF5idRRL@qxWx`6pV{EV#E8q|37(f7wN zljGhk$_ejnu47K$N0F4guNDa<267q^9zh-&5NFo0jebJ*9GaO5&^>ze#X-4e2#cXVYi#&)beR!8(e6SW z<*E9!Bu?p{?CiB5Rjw<9gCmm9>QmgtB+^VNPxcLTfkX?}nH8oJCm}<69ttGCtFydr z-fjK$(#N6Id62p1K_i-_zu-V&hIdM%*hn{VI2Gj%T?g4nQ{slaH0`j9J>P>E>)DWMmvjULrJig#YluX(ZVGcEgR(}{hq%7RHl4a*SSf$$B!}ZDv9q7PFA!x^}h>$aKKu5^T^LSrASb$ zn(+F0QIidUGOL8LjU9ea`yL`XutLz`m|m}v2{{7HC z-)TY5V8UKdL?rDIr2VSrTxA(A0{q3u5h+AA`2g3qSG*9DT#uT==j{42jhVr%{c4aA zznt>qpds)(Rp>Ve`d($`F`?E>8`V#5l&}w~7-b1vVV)Nyw&uY;^_cJz-(J~Cq`N@w zyb|<4#?gocpZD?&2D4KxK<>+VrkL%_l)iNMjJP@@HJLsr_(A*TgYgOPUr=E2#8MO^NY#n2S4|DE+TWqkLW>Xecf|uEexQqk-hgH&Ht6 z=hMYz@71jRNgi}C>5BEw9kd;irp={OC{{j{)+~Zpt03@{Df z7(ABjsx{2xxgKrdLjWRK55gu*s=`&ll_gc}VPxtnaxI@+CbSXD4suMXe&Q)%u3?3X zpt-W0z#Tmi+8cpd1uU5(ubb8Cyt_QodBSm_yu2{!=A;6BH-%QoRHFUbuI%xE=W6z3 zl(91>RI{vfd&FK*hnVLF4E3k?yf9ROG$^{VPAn2sc9Msn=<;ZPKih)03Jp)3w^~+O zxUAIAijfMyEJF0H5M8)!jE7n$V)nHs6`#}itBb4!U(_#&Gn=9{Ltvic-DQhly<}2&lLrocEsrvv&Ewjv+9QGdH&}WEtP$bn{cmY9 zf1Uo_qA?b-4lbz|pVa-a8KXqX=RfVsl&Q z#I>BZz{RDxddWk1rl(S_8DehhPmk5r7^cnZwP+$4PGZr*R7s&**q@16`NM`6CaBvx zOIhCNV1Vlo6Ed`=aQkjfP(&bgL`$FT3ipClZ~@Lxt2~!pDdGaxkCe;16AfmHXslxq z`fM{J0Kw>5$aN(CRcVz!@X>RMQ9JM_oTVW^FT|{0D}rrFR_udz0E?U=XoePhZ*Oeb zBvgCk4d2Ty-rG9`3S=83u|Wr#Ht(ZwZ^j`)H@nquDSz18Q^uon2s8>OvX1=|F?MK&jsXX4p812p++)mG#rrK78h+Af3l!zS(_FwgWH&|xbyzo}& zGWJ(;)=9zbzZZdiYL5{Z*)sH?Ovq#?kS~Ak=o(rozc+d}qVfJIVQ0l;XsFbw!7&z- z)jJI5D+wl@DrJdqa+*#5v*7V7#)PF&$NiG9^#9y@_F3;XxW?&jc>2tDRsY|kD^LSB zVB8+j(ptd-B!n#b^HWw>LD5Z#(DAYPX)N3z+yOQ>!~I{?5YOJQk-qQgddu^wP;|G} zQk~}B^wc}wYy5}wD8)QOJZvtqrjT5GlAG?T)kK{rR21bv7 zSj1`hiDweWg7)Mq;;?6j#O|*FVR@fPa~tstDIHW1CSQ-M7ClBl%UaiAMIy>B@VTp! zagac9E@`@0Mo1Y}sdhhPl~8b0!>1q}1Z^=0)iju*bmb0Qc~2-{aNzB%)c{E=KUXz) zGvV9QVZ|Uq1KR6!N2r7`?&W$PX64J-TskLva6D1KIhHQmLx$+qyQCHh$9bCtiaUO8 ziz`@ua!72Q>gTELSyDc_orDfa=REKT(^G&iho;QWJgT%^@eB0x?0Lq=(&V~h+cImP zs^@i{%WMioaK7Rx@!-6%A9a4?QBrkARwU^Cq9-mt*yx<`TZR7}kiOmqEcV1oW^}T54BWJ4w3y zhl2VA*N~<**)%e#Guo~G20)-88@xjoH~v|p>574X{muXN z%qd;E-$F)n)|pHK4esam)b7!N32V!(^~%U>wKkt7Rh`8a8tSs~XE#RB9MA^r*n+OL zYH)a^4VqoyPgKPVQM^FR9bBeHaFlDcdgPn-_8oXaC?%}COn;t5?vWEWGWi;pLS;~^MJM^ESA)DGy;wtN8G+J< z?m7%iqVL;znB6D+eeUrdE5?I}2X`ufGs2}NX3IUzV%1X5=JAK?u@cFw>+X)L+fdC~ z0q9Dde$vK9Q0iphV?tiwINoNI*+Dd-ay6p`N|YHG7;s5|qvB4S)swkwJm*uW@CgWp z)%f$5rG}=I&0<2lZ5QVzBMJHh5l_(i_(&T?Zm6b=E}%+cuY)Z!PY7MT07D%%L=HAM zyke;m+JAqvN=0S44^?Z5(9t_Ox!)|9a;P6w30HHv4myJtD9|+zoKkM5gtn=r<#REx zQ*Lh;hV#yBEr!Y8G_-YgE@OT6jBxP$;dOO|82cTaM=c^aYQI9?8}+Zg$>07Br{C^~ z2`;H!3H(9iqS4(^AU32Iz=|RlGz=BH3M_aKgE*@+YGarw)hdRTrb<7VTA`EZ*FgHo zSf!eZLWv6Of;=NK=VE)pOzMIH#Un>41wHx!Jywb$v}svp9su*Jg&FuX`F0w`Di@Rd zc?9WnblvRh$mM;{pNgV$n^S*GDn@plE6tXURcSw)I-QE4AHe&s?l5s-ly1An}m-zwph?>^Yx~?sRMF;VF26uf@ zoc`BCY_d4M5?aU;n!>=hsX%b1^{=LBx0<@i=tWsAASv?^wveLOsNs_LNT!yWcsH;v z0}FCOZ?I_PDbLj=^p*>oe<#d0x>vHQw~IZ?M;)LX5_wFjjIpU$-Xu*ns`nWHGr^B< zxyG|2$~BVkuZ42b-*ot0KPoE5a>&VYtyWNQucXb_-G^uuyTp;tj8BkYZ9%oCapM*r zM!5`Q_d5!hFw9kT1-$NkGt{P=x8(YX6`L@khy3zn|6BO+W9FShs0&0T29E7)kGYS1 z6yUnIdRQe{VAIU@z+eYB=9u&zKZ{Lm9n;3@9rb6?*M8gW4JEDfj?h`qyOB zt5C}!EwL3Ii+SrDC)x@&S$%++H~zM{fAa3CTlCSa9772&vwC|qQXMMktk841F z$k8@6Nh854R}b#?)qu4FeyZ)B?_S~zC6K*q0m9T?N*;}9ahtTJ2R^~oVYJ$}W+P)W z&-kKyuy%noAsP-~r?e*91)p4_stguAl(;*Q{=^rT&LdGzu($eaiusG^%LVVt-9XHsqVpFz#{D@p%%@s&Lc=m~+ zUr$T?0gVrD7!Aq04?#Q_c;#&~Q5E&qgBSd24UovX?&34>Nr^}G!(}bWz+rPFTX22O zh}9na>bEV{4E%~b#Ici(gPXfvd4Bm&V6QMAcyhc*s^()68pW6(Zfz_XQdaSGT!Z#n zm>aI!jH=+kVR0P&9rx@8;|9yyo!Axz#m*hoH;E=7b~lmw^y%IVwgjYnalCT< z(F8?_uFs{|9^noUW@*c=s7NNOa`B-S*9VXl7f0^<<-C*>=lR9bO zfc0ptWUV{VXO0pMmh@5by-C7EaGjQe^hl_1Dfx9Tm(LdvCa^8GZ3Bh>{63ncs1>rK z-4_`4_ubX`|EQ~e2)?{CHdGBwX?MMxIv6jFE;PrJ@BfMNb=XqfOd%A&5RgkfT9?Pp z4-KUx9_5uMN^`&Nym?rRo~0nN(yy9vx>~{wt6Y4#E4W}K$uGK9jX`1Eg^q6Jzl7?z zNKN-2K39h0n7-?o!b~AMo!Q98cJXeeVj43qaP@hJbCR)l?5ZDaqmFBb9G<4meNGE| zQd`bLMqruVVd$5Eg~`B1nPSsn=Hpk+OfZn*#fXT!jtBD-J13%OZ0TpfRtS?| zH;l0VZ1OIk-)p*71B7v*&#>43d9e7;CgbfD>R3QZ=Z)p~A9h7o5*XCn)?E*g4;pQ4 zpbJa=x#GKTabNPQI38@6V{C{}rr`0@Fg7uwC2@kvXu_rLdDJ=xED#=-#QkyG|3gWn z_Cv$nPV5As`Tl!|)_;6we{{hcAZW45XC?(OuuLlKzwWYQ5M&&X8D#vgW{Cgmr{Cwi zi`jmf#549^`#a&i6j|Z!6w#pgF zm;d9F@00l!6wnuXERBv5{HZqtyuTv~Umy7Tg66dXFlTBU+336zzn(G_^8f+6*%x01 zuJuQdaf=#0E5Jqjp4RFy0GJ!RIK4U-5%=HUnQ2WvreYJ3O%-^Z?s@fb{K!>N|0wSD z&m0Q^jb_sb>$!$4*`nAFl`Yv40_*6f@Fl&n%}!t7%?quoDb}6GY*JD1JH>{dJagEc zwSP@7pn38fZvns1o=2q5A@w`SZ43guz%yzT9E>1)%=Kr`Owl z<8wA#5B$a2&q+9n8SDM%7~i z^suED_i&7E8WPsnp6>3SFlkghkL&vM?RBH-YNFN&6xEnJ^#y*EV4~;6?0~3?-7+Qo zGI_bv6_X@PHzDNX{8Z<4Eb!$GvR=&lY%bGy`6+#T_?Q$3Ex&GPU-hnkXWaugo_Dc9 z?YAe8rTbj-y01@2!hgSZ|F;Mw{4*WAA8T(suCB{$d0FHC-uage=-4ce%?q`2tFA7F zSgWosf9EC%dv;?CSf{_v?hTI@$0CZKI|ZM|ro~)~=rX+5yJ{w`0~~)l+WBkIzx__t zC60=E>>FI^voK^&g8BPHP5#-oO2lymb=;)jlnHIeAodSEs>rFUEg`j>HoeiYgxX@r zS&=_%->3bkio^qM`_Y`5uppqL;C}tq5YLu`!%z)P`;)-08J?_A(;*!wX~v149dK zzHGXKuHyLEGPGV#uTZVL&4OxwGHnsOo(zf~z^c}b)~vM}0<6l62MnXs^lCgn!h8mP zQ?7B64XOf<#|8#s-NYKv3AL&*$UfMYpAEz6eX`^T#g0h zc>N6$M6lY@()OFb(%Wr2(d@ZuPf+^}%305R>h?g-ZZ_HvBznjw1PlhDFU~M)886N` zD*`2d;=!!Z&y08XC%I$7PDD<;HBOKaKeinJ8*Ey`%ocVj* zAHQ-2oN-PH7ps-C+#b-4Mt+5k9XJe+jC=!PNyo?ycWjC>K}uSAN%nt}7C}^)zKG>_ z{<$traa+P9{OaUY5?x;Nc4fZ3&$mBFS-sZil04hDy8b-`WS@j_wr^|fo@hM33@%qh zMQ0jB_C=7HZP2ryxb96=-n5o~7(dx+;nXDVt+AY*Tc8_OdcilQFW}oDd~+D}c8!hZ zjY6IvrwsIF%;!0yRYN1Xy1M`24bYgdNEce;Z*Rw;lL+U~{rY`v+zfD4;Ah`Za2a2I z_p?n5{rmuhv+gHhR>9@CA)QK9)DN^Nb@Bx3!Vwe@LG3Z6Y*TFZQ%XUHsbPh>;p{;s z-WR{tX?%8;K#3!DV`#~z6JwH5q|6g}uSNGV< z`+9o@#Urlq%u60qzI{Je4+0fY@;U@O2AcCX9&6aZvr}&TI@I5K$zjaV1zaIyR{qGK z1@A3|7k#dHXv*ntXW`pmG?_d;{8m?u~_~!yxB7^i(s}Gw3g?X!c(FS5Nym22tge1Jm z6SQ!Uiq~#`*%wLv)pFYE?7cDu2`)}?Ascvm{WV=4hwV@GG$2)(T(@_;7pAhwvHIDA zjsmif3gIBPa6lL<5v>{z5iyvClV(DN-Z&sFV}4y9Ps)8ntQ$|a+71p7ypp-(E3kVj zFD{{;g4yQ0(@W}m*UvH&v$s@J?10obV1{(gG(?aspkovn%A8qqGhi{9lZa`h78Ys@ zm`F;{TPDVvZe#xL!SqL}kO%;0%(ntXt1gvz=XL^UJa*;95dA#`LW2&BQ)}1>#YUT` zw0NUTGFOse=7O*m6_n0O(TP2y6=ic_I&FLd@%g zMXxLua=tiiPyJGRyw`Ic{yo=k3ai@GBC$6Jes_pTgrVYH8Wt z(His_$1$o4AiI(-1&n&7fdR5Y?WXVfSTvy#pM?N^&QaQ(RX-^$vCL~dt`XFbpWPQ8 zBN0KGD3(I-)!PQxR+7WrL2RkCDskOzoA0!-fO^68P04=@vIe2n2E@ss(o#q>$f z32Je)x=5t0ra!?K^(WhLD0%2r7*uY1T!~9~an%dyiHG6W*Utwhj3j?|o@;-ZycW2R zrRFr;I5NVZo?g}u4{SeYk2@dGm8<+k;)H_NN)1@*vHHf-u;wCZb;kX`RD+&YJ1c+61i+zAME{<8k!;k` zkMM>+((Hh8`MKARCyWPc*d}BTCg_3cSg;`0^Y?Nh640cobOn734r1Sj45@CvZUn)F z5EM8QKyo7kqmQ~l@lmT7ZkO2aTZ#p!)7sQ-e7plYaP}c$&>&Yu`U7IGW7at< z3|T-3p=zYx*io+G#vVks7-R4*v1oalfTJQvY;JZT#{s0It~AzmxY_ zqq^ZJVVa+sQg*HU%yfxEYCd=)*{V=VY-FaR%%)CE{KFA`A8G4EO1YV?G`4y%NK9)| zCbSS-Yctu~0sx95Zbw;y2OEnO9g~tYkt5#rbshnyZ_b%yR&ezeyoS7HWsw4)n3eHxADBN2Qm^4#4T7QsccZD{jk5BA;1gUl-)5UF))E7!i{q&XSm^T)Q_a zCpfYipiTxnk#j+6yw@834eV0qCZ?vO`khd7TP9Y_rQdB_e0sMO%-SDbB(? zd^E8*W;*w%>SS9$SBJ0CgDn6k)BUiAI9DtjVE+ZyyL4nhmGW2!fG6#hH^m`{)=t=RMjnr@KfW zf;@)_0ThT~8dfp(*UxJc9tghf=D~3G#+-=~eYVpuBSMsI)C^i`GqrzAw3y=tvtxSv z8E~lzYd;gcz}^HAO_xlYKWv_=7Sd(}rD4s$Qed~|v=>^iPsd>deX{*#>CA7No1L+| z+MH+l#{Ttm{Hw!t{Yd%e^tW#)_^Rq3%7^cz%yJMUxwa~ZmSrp&j^l#r#n!_Mpsvkg zqEH#FvH!oZAu=*iD7u)=Wq9Ij&}Nru|?55^u=^BjLBArds=>k zEuhIcOwTm<^Z{iQ*DzZPi-G&Lz1dhyY?2`wx!X^TE@W=LnfdJEqqi16pQ8}9v3T>l z-lt&tJy0Gk)$Z{ZrQEqNwx5}#Xz#Td%hv3%pG8<<7~_7GhmQ1l%PA@>YF1pD4Ltx$ zZkjPJtO;PiY{c%P^Ala#Q!VRY#SDbD&9;fb*_l|NYhIRZJi6W}_u})1<@O}+;~E0c z2tvsy%RiDnA+M9^Qq<~NeHefq9If99*=v)Brg{zOTz~eUUcE@>L@t~HhnqE<5Stl- zXPriAkZK{I)>5vtg4pV*(1TaI?GY5HDN=#LcI^w3#7j*j;t%Q*#D+8iu+kMF>TX2% zGj;)(>#?iWLoC{o%`h8`~T1yQ~)XTX1b z%MAfc4}x+xHw)D@0|w?6E)_62otQ74p}lk?5B@3ip5yc4Px)Unwr;R@K8QGG|I=(f zZ<_)`PvH_HDpZZh=Rm4uD{(z?e|oeE^@vZnufxI+;e~tr0;goi4GSJ)CdEgl9UIuq zpEvXTiMP=N-@9TMW3&pH4y&`HchM1Tm|BbL?G6e0Ky&CtitgIg-cBQ=1MeRF#4K4y zCYifhe9#MLCf{Dap`G|zmT2_qX$e)}+Zj}CMq)(vOo9%t9vqxH?DHE zq?|T=C>iF5Y?L1*9Q7poo~@DWOM?)4VGX(5kcW zid5NV?PyRNt-y=NnQk=CpSrb{4NJhKJX2nAyy^6z?JJzm%FbFy_B4=oC4)~D90>c%1`sw6by_?9E+H8B=LLIGF|eZ*{4ps;xubDOHn*j z*-2c^ui{|`>vxTqiM>Y{Z=T^?rQU0;E(3g#2kb^Dl4pu)nY2zK9^DTEheUU?g$y*L z&+Lp*hDA?}C?!e7dSC@vjkf(3BJ!i?eAZWNjyZuTItbVelwi?lN_|2YBO;BgFL%FJ zotN2o9Xyn^nGnS%_v01~yfpgKL%^0owvbx4Sv|(#ibPv`m!8|`9c+UARfpfD2p!Hn zYM3iT7xl{Ft{OzXmu)?1#5~yL!D6y| zW~hfq&UL5ov1Cs9%joJB(2e)eg5%f#0B+~VI#IAc>F``ks{NhXYh6!cW|Mi3FwFqN zovKJ@VjOy)&*`2dCP@E*H~e)tcC*OK4~L4){v%|Ur&MgtblomaPH`z}I7@Ns#_W_h z6awl+ESC>rFil6&_7YudZ!%efJz6ocBMP24>#WS#EV}izKqOl=R;Eb-Q^4B;#~HsF zJwhplTjyPcw{lG7zAb?RL)VZn=L-!93v>IBilO+$lar`!b(i!>R4Y({sHZzZ*5F_s zL}~k4+$;#`Mrw~ zvX|)D?MDeyWOzXr@hrNEdHuf}&4#S2aXM&-n!7|2Uhgj2omasGByp&|rF&mYfGSH$ znCMJL++jNDeOt#{lP@O7?#(Z~%d8RBY~?q&c9^;-u5Ph8XzCPpL(vM@6?tHpMv$|B zPxjx4rd;xbY&580uw(mT8M1XcmuWT*>Kyx?d21lbJGVVr#cZ;v*+et^?yQX2MfBIt zy%+55;Hmb?dmS{sT7i$fDg?TuXv`+d(d2lm|avOZoX1&JA zEkTU`qt3lG9(=l&y@v?!hPExPnOI!feSHZ`Vx_gVDN)}rFfB3SU$~r|--%kRO$sAC zk4g7*NNOou8Lm{n^N+J}ctyFroRj?Al9yjrhkd?`h z+7kDYXBTW5l@HNHP>@ng2KrTJsQM$`n)>-v}&8r&w>xxBu8Lu6*5vQxIDDJ^#W z`zbzMMpF@~O7(oS3t;nRfu%emj9e%^?W8`Kzcz%kIGusCgtZoj(!OCHQLSPmMSF2PpCLm$*}{HW6L2 zztyihj|6tnqJ406gZlY5i3}y9p4kK_tdn5(l@-_#7I$i9OGK|{Q|I|g(p?=Z-C%U9fb>+c2u_K`@B=kxb%fjQ8y zh(bnU6}HE@C?K1UtB=EWp}%GR)+c-}%Ahr!$6NFObai(4GQk=ke2!^aUp+1<#W0A9 zNk3fwLQthFcAuD75BbQYZW!h5TFpV($dH%GDzaH?kd3!ZvI;WD?34Pb*UW9RU>vYS zDcCGUut$dby@W(9!|!bisKWvPIyUU4)FcJ(Pn@!>>#*q#%-%hm)ePUSr|I0QlV6I_ zB3V%+A8Iw22XPXjGcZ-B*ILUCPd~ylgqMDlp^gR;c+E zv?S387+=n>ZU{1uu*1?2)folK?@;*vxc$7Ic;c9NTaN#z#^_!;ysmpdx^q34P>;P* zGjGuzY@S|n0}LeKJC$Ux>?D#Yoz|H-KfkbY-(vz@V#7t#=M?IR@4dHdV}_?J{JVD2 zE`0AX^6Ih~)?)Z~5$GEn`lFuo+IeP|SjM*f;h1o~yKvO^ll-DHqU@E}73cFk?xi4S z87tNDc*)+VuLDYX{SqSx@ZqYsMNy}MGa)f<8_9~nB>T_bfy8D*zPl*Wtn-}ZcdoG; z*H;Iq)1efry*cNfnMRUZIcXmNZ8ibW8Hy^1Szi}2U&Tv4z)eB0>TnruT4TRx(rPuv zQaLs+OEXtf89u)#E=SAsS$W1U-L*or*~XIE95(<@PDn887SP6e-wo)I4{UBKjrt;R zpHV3Fh15z$QYy6B42FdEGp3}Z(9YL`zW!cfWp!S}70Z_aE1sB|NkPxCV%3E|mUB;s zsOBq2J8Io8uct>l`ZY+nO7uR#lkmuU?(8e8UcqGp_I%sMo@4R zi9x`xni&hfYsCz|*WK~*HbB6s-}PdRhSJw4S6~NvIpsRq=xF*_#oxcF3U-hbeETcGefEdK?EFmr_Wd@?C^V2tS_+9kvW!&WlY!zsk4nN_u z>fExRjv#a1)30)m$+Qx@qKt1uQn&WzFuOY&W_h+|vYQ+0i-Cc(e5@HE#t)*q$%nhu z8zgQ~#DqkHwmMFg2B@GA@NgE~Ad>NQTOQVZ(!zWGH7vR6lzvR$6hUnuT?DR z%xO^!A&I2n<|n!vJ{8^O?938YM3D>I)n4(23%I6mw+M9_tx*Z&;5G4bpSF}})(!?_ zxyJXwbdspr)Y64aS328T!7s!Dtph+OW~{?W%*oD?4J8Nh&z9CC@1BPkGg_F1s>jDhh#M^;N3F;bvWCfKNE#Qy}sQdUJj7OUt2aB_& zTq-VB!&v{yUk2=)2bc7Gbo?R+haF)~ZNAQslx2+477rxpm-jx*(+#+7`YV`anQhmC zZ(Mfis*o_|i)4z%bFP4w6&U~8=5L!H8;|tQ8(y7Hfvy$Zeg7zR?eISX#`-tNy!DL~ zZc)}wfmJ;ULW_z$Jy6>g7N=#oe0p(wl@>I*#cdgLNEK%VduN6W=bf8?sQKqJM;$~! zCp8ybt%EpIKL*F4V1rtmes23>F`ee|Z!;`m_7#m7*QZG2<}anFaznse(YGUg1a)TA zm|*~-Dd?ah!s%F9v=;34R&QItkf-2tiWXuJS^x$)W{OExYk*b@>63zN$?`N)HTu1q zv$aNvE1OJ8GeF4UmHlo4 z$(|);_oKh!iM|xQ1Dc2~#D$j?dD$gSj2k%Cj-LK6sTYM8$S0k*@J zMvdC=yF~YH=Xg)^0m|&9_O2zioQ)vfAhITGj1L{nEYKRzL=?WJKY`d5UraBv3@xRplmFmJjcQYYL^(+S4Pz&XO45tq{ zjN>$H*n;#oMu=n0Phm}x2IvG__IxJ+fT^uu;W9nY1@>Tx>Rl3=d3#|XRGbN5x?{5E z@^{_+BR;M(>XsT(J$C>0rkAE|CNh+WU+;TrWWom|PJq$FgyUi9$H_-5XrA84^f+J*-O>={Uzsa7UxQp>F;yFU&zY`HfmQ zxuD|QPD;RarHErW?aw>AH6FGgNFe@qb~dwj$AQ7Zb>GbPPj*!^k71?4$&Ied8SggI zw0{QjHoGkBr~jT&xrp^VozUF@!mBNw2)eK4tc2U`a&y#hv~1@KBI5>p_R7GM8zZLP zJ)KlF=!FmBSq^jVG_0U>D*Kk(`~)vJv&ev0VClBYGAE2JU|?G3{nU8eRt}2Dan%fq z_}h^H3yg>!{-d3p5Y<|-R}ZHdkt-NfCtXKb{j z_#=vNM?H7SqW*CV1Z`D&SZO_!1%3c-YSoTsxX_-AuH~w?s*vcgLgKKt zTX*ksJNxW@;Mze6+QO z2De5&b6dBz1~NkHrJB%Sy3??1Tr&;LHA(4ys~7S`(ZLj%(V;DY_Fs2@boBjwkh$X6MAZZcrx5 zi_0Zlh&-aFO()fP0+UE@cebsI>EdwD&hzt!FG7CNJ+{f`yV3+rsGDn_o4)ldIq6Tz ztdJDNwFh`O)GTzbF7QVvqR64Rk4nw=%h+d6MZB6i+ljqpNv*w)96uz=`1X1vcDgon zEUDY8e(_e(fpLUv&d^Ti+32jatkqY)zsz(DUno82N^z{{e~Iy_0ufK);sWlq2`Q>b?K)F-P!kEBYPu zoNpjZm5%mlnwKX{eoFon$S4Vz`wV8Qryu*b+goy!mEPdsFj5`&dPLU3>|R`%wz!8p~Z zdqjG1M!1crFKzp)>Ys53Yl9UNnr&gg!t=$o?bhS)%PpgR<-B_A3qww-sM=2&}p#Cbet4JJ);TDANy|mUT^(qp$+9? z{yX@hbM_29=+66z`wHXy6vS|X(VuKgN9NgPX zPqrtuA}NJ{o`33*&06p{26gyhZES3`z<^lsVZ9b6O~~YYUNY`51|kx7I9R#+n3(IT zUW%rL(W<~80T_bJEvjlxasBv&q5SGTKQaE!_FJ;49)J|KiBkqxX7nZ zMaIT?x0u-Pjge%$;snCLcNwuCthGx3{A>tdgVur8yaD-eR=e4~%~Yk1WygxE;fB)_xjPa2ibFd&f~HJmASwnq0X00ivvc=pXWGcj}Mi% z&`uYreRW;(&|=YTdV0k}_>pnT2iVatb-4oYaeGqyT*b@j0+h~~-+Bz2-&(mS7v&Id z)b2pah5&m06=0-YmApR)PUU-Z)VWJM?B#>HLG6{L(5dI1G|@Q=`=`QfciYn;xPxb< znx#d@yVFv#>3qz9vKS4VL_YutZ9q&HXB7~e;ot3reOe7tE>{q1a@l)n?kX!AxjPT` z2}}n}mWBXq&A2Zz)iwL_N@zPA7=O6@ta`#wtXybFg#Js=X1?w#i%v5d`KuW6At20( z3pO;=N(~3f85@c2rAhX;&+}|`s!6tIE7u&%jYT_#or6wl9})ZqjKk#VVkS7+9}S-f~*u%hvOVPx1KLc z#nZ*vU1CjE3E+1=id#@-&A-4V4Ny?(360lgJY1-5U0JLSO&z&W7!$rp5jfh+zIGLQ zu0>U6#pAS@(Fix28d^%aB;7Y#>TiyIHD6~_EwgqB`YiGswzqUA5>rPMpyx*5-^#$4 zL$ti{?8?!BJukWela$ShIXu}Fz^5pV!J8Xc7UY244dv_Ue!KQfuWrA5V7e{-yxyvB zQRkS@R!Jl6Z-5Q>(Jj|Yex_`1Zv_uB{$E?LGi?0LTuQGBtj0$C^l0jlBu+xN!39SD z%`l&pxvKKECp*y_7YAgN?J*U%{TU0bMnytBXTsJedJhadHw#yo45+*hd_{JfAgZ3o zBBt-{%ev_^u0j}a=c(eIt8;YWtBUf8G8~*WPHSk5#=%%pyH&%IS65vGydC>qOiH~E z+567+(Btq>=FQM4b=#?e7{F?$>y%pp4sHE?32Mi>5oNIJUOdw|fxPNMfphL9C~ik< zaf1dD^jMf{BlP}l+gRQwDCm|SbESGa_Qy8mDtjjr50BREkXWxj+${YY)a#<)K#NbA z`lw$viDx3jhQ}x^w0l3~78;3Ljm78nKK{P(V9$#3n21m}24GkQRR}SuD7a;}duIp* zDi*3;qh`@_S%*E|Sr$SL|6;~gU&Ln6trPF+!r z9-#Re8#+Cv3+YHoe$&B~2iDC_dkj)YN|SBmtNiCuF+r7Hx75_zTFdF!M^VBrk+Iu} zR?JO`#-r7y)0Wl3%%!=;g%IZvP0o?itD%J7w@yaalhqqw_ft54=p~{~UI2{iLvwSx zkjU4}ifE^ZzhR~B`5B`ng)zW@=tZeBW1((oQln(PTuST%tc}Zy)nK>}wOG*OZ$S}c zgFqU~@AWTVfBG!n8HxEdK<m6_xQ&Ilhuk-D7^H?N;i@Fm%;^GF(aXO?Ky_XeP_ zov=Ih6K(3fK#O^`k1083kI7Qdjh@lWL(F+h$wCF<4#O_~@?_$Gn{l!RQ_R=RPksQ> zs{XQeg5O#@SwYIrTX9M?>+xKdet3idJqn9#-EIBw%EYgZvKlP0S+r4o-UVyaxN3Sv zL+5Nxv+^CD^>!Wi9B~vL&#$hnng9+=8}w{faN(NRI4UN`@ThtV_X(c1cw_d%EQO(64pfk!OTom`N zU2c(KJ)iU3YZ+WBd?~Qr!WzdcT-l-OroHa~9Lehfm~8n8AU5C>C?kBIrICNQYz4X3 zGW{tl+kqY};0JlM)pxu9@%Y0?zH{VEtG3YLkFIRjkRhrY1CVsx0&I2PR(#nhXL`GP zlY%>gaVX=>k#*7^KGld1^9N+s)Nnk@5*>`#fjq$BJ(P#r6Nab%!%0ashEndLwrKQ7ZA__8;UNZ@8orrR8P=*Xt=iNBO~lYX!~XA=7Ed6JM<(4$g~0vr{` z(NuQBdu4<;ZU2Y8w~mTx{{lw^L=YqlN>U7zR=N>XR8)|b6j4xW=%GU?NxdRnDh(3S z3^0U*bl1QP0|Jsm&(QC1@AY1|xDMa9)?M%Y-dp!S*5Nbz?9cAKKYM?U{&uEYoZras zK$%gnlC8|w8!>lwBKytpzR{tKV!Hopb~sc0NIj_g*X3 zL4fZZG+VysP!)8civ3=XXH4co>(AFF1YCI4SauAsNeicK$)BUHBR*AKs;vZO0~0+~ zD<_OvudyC!@DwG-x4bFgaRN&++uAhGIVc@h$$IKLJCISm5|S!C&7)WTwA^RF&TYvz zwPNGt)%mY^FKno$Eq>kwV0Z$!XY4DdcSMFYNsD^CKMU|}?F(T`Jk#PoPIg()R?Qpe z-$@tztU@W0wKQ;*&zoiZ;`4fKjp&xB-G0rh$4PZaY#$hRpY_K(7O`c2ysPfKh@fSr z24k;+#8SvwnXtoD$ZABjA(Hl*P0I2QOQ#2w`D9&|U8(X$=L7UlzV5_tMtm4NZKQ(> z*j9auVq>8HfvxCaY+zu@VDdE?K3cizOR_=9+Aen$%F?x$Zh$pY^EcLza6y0jy zPkr9SLh3srWTYBx6D^a2L?`ogw;ZqCrg5(rG@yHel%7?$wIRQyCvH6+P|HA_ss}H> zg252N$#9I-Rbl?3g(D@|I09P{u8?x3L^s)CsKL}bg&)Fc=Cvork>i>)H*DR;w{>cZ z9Aw;}iVCl}bt`REly9jqQ1Hiv_OcDR1;;Zo+Uiz12crZx4P!+eIyM6r61a_luZlGZ ztUAt%=^!QMq4z5bbb8jl&!c6>&&mZdJ@XT4gL{ciYrsA(5iI5-tceUJA<@S+Vn|k< zB>7Fk6O0U}P%qy%OMI27IxAqZz8y>DGrKz=`D-pyEn5N@WxCzEak1&WCxe)W9&+*KfX@ z(tMck{U;gUm5~|(Tk-GS!@`)(4rD{hX$sw)_L`(@|EfGpxq0+@Fp`P8FuDhQQW7SU#o89i7H{? zAN&5@zU9*sTtA=D{U3&=?rcp4rKHe@E4OVCS(oA zN(a$#j_Y|5SzNo@kl9I-L$-C=|J?A}37#G((t#+-noX`JRVFWsy{T;L9|w+V8iB8qXW#jl zuh7yX`TF@OdLeUu#QaN3FRvs1a1Cw4mWt?E2$Fmv7Wd%$mb;zT#)hXX!4}hFZhmSR z6q#D-*BfvPt@yQN zg22t;EQOEc6R7hzq+6SG={L77r=6GTrkD&jbuy-F8n*(OuB0>$02Ae7fo?cBQ9-~4 z<5IK9&UEWs^I{Kt+iV@aDO{f zW}sEeP4MH{(ilDGg^Q<;of$hTIB&l);=yk}tViXdal(JayG&4TxyEH{gWvQT6KCP7 zpRQw<2kl@(;(>*N^XUSmvdu5^Gm&iezZuOJ>(Ih9)27lV}QI?WAXFpx-7 zh?7*!{NZn9FI;ZmBRA}0bgHj{l<{S3S!@-D*JaJLU}+orG$!XCvcz#@{^}>o#7QS& zMW)^R0=$TcMSw2FCJPG6V+_;{I4_^j<2N%&`G@;W%yvAsRt>yR@5(6Fu7<{Z(12o= ziLVpSPUR`qUN6!%+Y;(pV%FC=$x5yJo^imb^bOhr$=c1A#c!7B@<`&{+X?Z(0$AP2 z7A<8=AeD>M_K)*Gi=oSNQW^2q@T=AKH`F_S5Sq(sKrjYb`lq? zT+Y8cb%`YB7*Qkyh=8kYtK|aTte@Ltvg>SX%MO%sIO>*QTp`12&uxGg(J3=IeHqRKm(|6iwRT3M$+;N7!MWk|O zm~>C^7MP=pU)Ecouu<+8{=4>y+d3)DeO-LetMHre$@4Bp=HF1n=GCLUEwa?&B`n%T zJ-%~jv=GrBql3-9nwb_dm#X(RI(b5W8$W_dcn6Kv73r~f=nw)r+Rr#NkxM*G#=4FCZIGkFTmK$UtFCGH%1<3;=H4ySa{VjQ zy;0ikI^8C8Ov+r{vZS+OmYbe1_9aIBQhgXBYhnMeOHOR%DBC>?bHO;1bY8n;ECrQU zBYho9>$w?sZv*nDA`@k?{hF6`mo8S%voNkj4H?#vmM#x}_92?5e0{EkFgJyFF5)w( zfzYS2I(aHA@ko+#Ue4@se%H}5m#bKr%+G>PFqW^GYBlQ7c`h5+Zx-#8)3)L+b#tVG z6kn7s4)^(8j30zK7Mdk&R;t4d^b8Qa`?6(HUdSQbSTSz;_e zwp1TOEjlUvQiee{#eVJ1VxeYjd@G6SmqR?tmjl9f8&(tB<<4C-(D&RxvP^7@2S(ky zZU5b`M#|G-d&Ixh=50@e(#kOD_*47*(L&R+MZNsqGA?^l?iS!zj{%-4EZpCJ47k4C(RG^G;8%K9|p{zdn&1>eZ z&CtIEn#F@C7&u`d!o+6Z{JYjae7-Hw+GvT6BH^>ecSI`l`tlryo@;Rzn;O4TaEGan zP<-ypt9iO}feRG6aN^wPwAji==Y1N1;g<&S3i2-G$*HTy(N9;>RKN5`$ehuh8Jjoe2SQbE|Ncdv9>rCNt&0xVCbnnMMb5`QvA6+JG!1?GB2GQ zuc4SC4*kK$XRe-Q`F)_MG>?v|VUflJ4z#AD9KwxjUmYQxl0#!{ca` zik#faognjpmXFJLs%EJTAtHw%#PVdYk4OjkvU7Pca@TJ+H8t^m#JnCQmXK`qb*-Nk zb{V%ZYQGj&U1s!pw@$G=KI2KG|Q8d_SBu15R|uhO96EGg*Rzd>kDG; zede_(a0+|j`@0vFx3`!)l@wm~`qPuTz8Pg)5*4>_>sBo5=@bjuplVt1(xTRn1>*B` zQgY1*)Quaa_RH;-i&?->&OKGT_LJ07xD9%Ow>R;R!{^F%$B!b(UYCR)F_b;+B(&iI zb6xUQX%`>g<_uZuLRrq4)#s3=PY2rQV+|~i-OwXOW2@#1DR1a4pAYz^Tjm%+x#iKq zNYbe2nvA~{D|cOcGwndA0pCiB*)hQbEr(fo@lR%)G|Snl3KA z%VM8&U)3`slFVRARgk6eZ{r&FbhkPvV^8f^#0agR?SLM!TSp-sMX9DrJfAJhrhB*| zkNZ2xHPoy=nLB&o}{1E%rpSzwK4(r`4%`oW*3zv8S|VU2y5u^}lKGe)R% z(?qU&(JYe~qkMLR4wwm{*OOGz^BQw@A11BU;PseZWz!Z^8_)jZI=d4X;p8=#w*}^r zlFY}GTa;-7jvYekJZjU={Fc0~xt8f>COh_xFjbvz@vcFZyZDG)P%@JXpI2!v z_7}1J=)Gk(aZc_j;dgq8blrEmWAFh1_u}TYR0-*cZ>`4mF*Jsijwl623IxnUWE-Be z%VqR(Fw&g-J({3?4jWOe_*&}EFvpt*5856dKD*nuijZ-syxQVR`nH7aL#$K?>XqUY zi5dlMAd^@RuV&HX^5`Dk@_Gq!%$;N}Yxs;MPQtyWw(Gq=(u>1R?~izq;6?A8LKMAy=h^5Ei^ZPv z19R4Jy>>$NoRYSW4?o?)57tQbx)%4b6s90LU12mjo5@s+3v_QYnJ`R%R=99;E_3Et zGShs+6fs|7^(q)0f}+|j7WopPD9JBSl;c9Of49dc$9S$OiK{b?nMS~ zNOxQRbb*Zxrci_V+q43wr?hw%JU`5DxDTBX^!fYThSNjRWg{|J3_3?S9rFZ>-g@cy z@#2Mxk-Q&)^SpPv1%HObHNrKejN(V=ROR$lv-W0(nJruZQATHy#1*I9+>4k^!aEsp z2_iPi%>A29j3XuqBSs>Z=On|0t_j&dO>}4?Q!G_<}-bAPp4b8mL01nT!61m=xi!JUYvUn zA*yOOX)jGYw7hHs%)DQlxFYXz>-O|=fQ zBv`g?F?#D`R_z1MsJEsYe_UqxMo!oEy8#mT^0fHMa;Krkv7?uL9}5<)0~g4-gyCg< ze*{eN#tmcymA_Afz+DYdnk%KA=R!kn>%x){L@rIrtHWK0%tNU(+4~>?*c`??&91u@CZm-Jo0nWgwXn|kRP6eHFk8JOY0 zFspId15Q9$-p93`l#p=tB5R_rUdXA zmhIRCQfes{1tFkBn8^C(9DeQj(YgpYfuM2t3s3J`_1G!)HddV**=uKPH>TgsSwBH* z_yG6F6dT381IDH9bhH1_EljJ#H<+rbl(hq#rG=)BmnL0wr&;+lS{p%l8s)O|Il>LT zz0N^)e8=}*EPrtb)Q%&q*-&pP&aJ1KiIqwS5pWxR#ws)ju zxROm8jje>snV$QZ7*t?Fq$5epvzw!YHZEqbOvJ5Jx`cXL%@|g(je7hDziQx&4q9k~ zt`(vhfxbszMsY&h;h0Bqz#1aSiv_=Gux{`agwZufwtpwCY{NY_m;LlH*SUl@o)_L+ z``{URNq3n|?Oex4xy-r@>Gq>$91#9#v>}cmA}d9U9)*C-s4v;yonveL_@8z-hF#|&Ern-#HOArAGQn&42JJa#N4U3Ze1cjE1n=yp66zq zmuA$c77Jcu$?ok?I?oF9s_Z;3 zaUSbS`pVkhbw#-fhYLn*%)lT;Qq=yi0_6uay&UBmLl)zYj8tS{F5+OX9`msr$L#2SSrub2DAJkd&CpNW*hUtn0B(vyWU7XUD(xC%m{r=qtzM zn80qzOpifC);g*4M;Bp{&C?|a+9$BN{ILwt>&cMRm8oL7k%ITuP)AMN@Udx5c4|&qkf6<)&Oz8V>CV|Sj2lC*+Hh?q1@gdWzF5o^ zr=LxJ@~|Y?;8T4`fyq_IWc5rAquZH3(Af4@3$m1xXQO*kDJib#T9!|DCc5gPs=0+*JSzkR9+}X0&6pK2A+(Pda;UpD95Z*78+#jRJuw( zS}LN;mmFiK*(bEkbXUFsYnGGUi?dL0{G4o3n&l?g>GIgkb+k3zDM+aHO|7_y>*Wa} zsjU}hA6UQU$+DfC!`eZtI#w#mip$mdjmCx@mnN=5g31a;6Qp>y!~MJ#2>4U;m)vn1 z7n{8q1*}xMSAlsgD0$q@%*05Xn8XgZ&PDI{3g&@W6~<1bk$}I5&_RXH&dF$V- z-iOwXHpr?__|x$CF%QsoMsvj#+_{OH8(3mmYORK_gTV8TD zc!bDD_Hl0&|K^HGqCh2Fba^x7$053XPp1>be0Jl3>vH>ilpcy&YS8w%0D)jxY)h13 zRnjvvxI(Akkj}fwl@|Gu6`2P8b$9tuBcY+FjUKH_0!B;l5C)YA-z#^SDR0kKz_@u@ zP~0txZh|um5Zi?bTp!N8_dJEvDJhLb71a!d!p!HK5|0E6CfBV?3gB7^(;t)J!CbF< z*JDrDsk-^xnYjHqt5YMZe0{3+io_`vY4;;|_(zFNtMKz*!xVnh>@=8Fouoax`-~F! zX9YX{q;813!Sw|%8k&s7^`ylM`67&Ti0Gu%f=T8Mn3mFXjf(7?^UgN`Ga>$#_!;v8 zC4-IS;H($k&p+9B+m}s`8d^s2k1iAoTidNIs%q)l4cY0$Lg5_zXZ4+|ha*vl>ja}npY_#=wEp?SnRx!@isk9K&Butvl zwj@xK%`GBt;3kRWad=c3w-_0U+}yn6!>{GEzAU_9%(xB7gwd7+ou}eOjLxj_D;Hhu zESMH7b41Gb>N2K>(XaIP!J>H_f@!Z_L%wsWG{#l6RI$gq4_$aE?c<~L1BiL!0d)eOlIO-EouleyYl{>K#ZH28BH_9d%9*w>hES&(+*@pRdM^QmX!)VX|%Fc(vm@g+titbFRrU2cw#LuhugC& zVmruL=S*82OmluLNy;R0aPpPLi=G9w9O5l2*RTYscb&ia_3DHnHzjB1B4-?=6v+DMY@*4l{k5?tw26Fu3M1LkH6Np8!!TJ1uNlFe@- zn0DTlW`)_nYmtMwS7YGQTPHRgIRi%%pP2W)G#@Qn)}V?rz$<|z3aw3-=s~x6YQ$V9 zhg);F7DTyncn3zMH0!XPF7CYr8CSe*zded1gB3h8>$M(`5-ANa$=_ubTmGw<@-O!t zd$B!}4m>)&li?=)MXj~SG4XFs_nCkDQThVnnkfp*AFK~#E|hQg->V}X+DU;5l``o~ z&tX^B8}2{}CY$4khGu(L@?(KR`CeQ0kmS&{7v6^K--Cm(UPG?W)O~<&$$z#gie1ca z@aBk1*i6m}gJn!R^sYFYzEQBz!G3DAqE*`Yc2yr0{wmPHQKs}Igau>JUrEZ zlUR8a*%Fl(Sb6fN>SeGh`p21eO*v4VQutlxe$sqbGt)^2otK)?KMaD8*D*Mmu5|aY zQhjr!(;CB$zoHLx5&uhp0UpI;2&b;6Pn4_M-uP2{|A2H^`mqlqG~lyaSVq48b-Xh* zp5dCqbEhz^B`ZA$6|<0urU>OB=3*YhDf0&3vf>_}+8#8*v`qCj_U!{Kabc+qUMSB{ zOY9#8;fTj?gi7f~UZ~%)j^v#Fxrbd-u@~*#wQY;|v$F;=T8(n&j_kwdz}hohit0En z(VdrU2ci7^Bz}x=>MA~;tCnOS{s9ns5r}{4p_y`JGt?Q+H|C#6{SD6jh(`}i3eo3S zEh&FiaCpx>Gy`EFe1~#4Nh#>DTyCxj5bg~KY>3UC4RnAYB?D5{?E6xad=#m z5A$S;vuC~MYHyWQfSJ9OugO)qA+bZb+Gib%oO>enD?SKX;_NP_O!}M!u?e zB8H}*-!Q5z%MG5!EAFx$V9^;U1K9AwqX{VCJ%^#rqQ(6 z);J*^?XA%`PddIN^;VZ(a%Sl000JS4D4Bg?@)t?SF3wNuZCzV7nWhm$IX_z)sLl;lPp6gggJXEienv_PYTm86-ft93W>A~wlRKJoR(=3lm3g}A zTXOWNV_#qH-dZ91`7^a0${L$p9VDg}l^F3A;EW}-|5Ql& z+mF>a8@L~RNnoF`lYAEW)@j46&y`jv&m}*%h3m#I4x`E zXM(U#K0w#wl6^4F8;I^D%h6gNzE(-TnL<=~_m(e(!`QiQ&e2DFain3|R(En!26L>5 z>zJX=`Hs8Ug~5G=2jU_@Zdqk%5%q4kYUZ<1j=$KR^kc@OO*kDQ&ZwVuSCG)7E429G z^8HbV`R$;Q**O^IAsJ+Qsa3tEmz1ZJcP1B8u3u4sSH@%)S-dgO8+)onWFk8IS@i3u z(ABF3YW+JKK%TO+mRFaoRx>nPfGT%1+n5z)+ObD5t8=U;k$FSyf{GP+$cWExG< z+I-ROwQi2{&F&6_-rMX;a__eW|`5&KxZN=tJKM>Dh zUq_0*P%2sq5=%EDms<3-XuD2i3)_m+)h>FjztRJAF;!K0F;2%6>YO!NC;x$x+MM5z z@?9AVav~K%#))CqYSZL?D&t=J)$>p~eZux=uE?Jc<5iE2H^rVdFJ7ZTd%x#%VS6wL zJL_mrJ3r6mEg(xL;i!`D3iLS&>LH$O;>H~z>Of2VBeRm7Z6X_3qyN&lkLG7;Z*qF3 zSjVYU)$;MyxZYVw-Cv5*EWY7p=1@KLPgaGVvLB5?o-u3*KN;&V9+AD4{Xu_6cv^@{ zIr)A%lhIdfhgDl8k&k}0x0e=V>zO?Zk`4Dh%TZNW42i(L^+NV zT}jvTals;ZX51>a^@rU((BIhNh|kLfo*1pgFqB(QCZa>{6ngTti1S4Q7vJf(N?-%{ z-mv*_jg)yp5oR()n|>TJT-(t0n>4CeqU{;Ct=^L8fr4Lfm4rHl&zy)ghrl zX8&^Npo393uYO_nN{^P0Swa6LPQ3zgZKq}dbcnJ78E^s^jNR7h>=4F|*h7ZXUF}tc z$Ci~x*VxqbCd))}jnrWeVy+1!*mcM;xZdcq91QAKcQoklc&5sb@46+UC3XKr&-WSI zU&z1^X_u_o$lU3kO&Y+glZN&2oOY((=~AJg-6RvU%`ZL;7Uv<`}U;#0DM|_KxPKMchk5 zjr;LsWjbi602R;P0Omj_B^~c|R0*xlTtF?jA}1aNhHq1C=r|8Kz>6DsGR*b|f923ven%Amfl#goI~Y zwRm56-vigw7JyU#8Zd8ZiOkEJg8cjP>4Wc&CnPksY}(SY7~F@Q$45f707mN2*{hc_ zXn(j(`ju0z3*!ndHp=Yy7We zY&;Wy9&={Uoxcgbcfp_9;6Zjt(cj1z@z}`v5CkbTcmScD_UgV4ZiTY-k4`P{lfda~ z2lk7=0X(vGcPYq|L4JtSCF5vwtB!5y-%EYz$I7Q@1#Ikv%||NHZ91#R{X@CQ;89k3 zMeB2S^c^GwUJopXF6lWq)8?#YTkLcySAVA1(hnpU)L&LZhivIGD$$~%m)~VSUHc+n z4oebh5&K>=-|G`c9jiAlBfKHMQ<~IV`d~lGg2w?Xe4mx<*!4ZnJ;-?G^M&9%4CM&C z^k`hj7CGtG`Uk`DKbnU(*-;JZ(Fx)3}1tT=;R<2FWqQy>Jl4mjLxQ|Z30C$rX zuF8>q%+iVZj2&IB&TYOxy#_0caN7yZMBYSwvkaYi5$DuuT#YiBDYEFV zVJ7X}MECF-;!@`QESs1V?Gn&#>%Pv zg@3d28-&=XYy`*qH1~S<)#Mg?QRgulb?+T01|ky~;Z*s7O+71^_L}%6`cs(C&Srli z8diwv8@(zV2ycSG;Cj>q*{EpzhT(CD0Xo4=vP@zTEdQ4ILU0aoQhdur)BU?msqt{usB!g zY&~BZ4(Z&Nsz2j3nEH<5zF{C{nNVFiev4e*l${ET2c{pKpPCYttyNslsSp(}UF&A6!=H;vqbr!) zE+!aQ_mYJo*__u`Fqw8zhFP*zZOzAg3KY}JJ7$n>QqwWrP7_SdGPHddo;axobqd&N z9mG|!;OS7GIb%r)$6x}3w`<{3YagGsH?q0XuR0HR6{^mD?A*A5giFpWTeMtf zG?@FwZnm>hSSl55S@BT+cGJy8hCaw+PDzYP%tQ&)?c7ZMEUrPy3^q3W0c|}2FJvc9 zp7^U+_`4^$tN?`jrP2O-VpfkT!U3N^Qxa|4-sa{NfzO=5R7iSeoLMw5={?_K(E3>r zHoZVp{nbr`&3UUQXk((>v3D!I7j8d_t-2ZJDn7HZ-Rtv7(&6EXCc8Zom47pl>kl$? zLEYSE-6(p*9n3;`Zgm)Byo5lbwz(o`oNBdx4qPu&Y&CT)7O1NPK z6Rh`ox8wS%KN04{j#YzJ+QgNhrSkRi`PwYE>=G<+0=m6q!eeTV&?w$`?E?Vab{&1O zVm}6xd_)Bg6GwR z*r{?#!*<%eaF~9ZmCoe!|BB|?UYC9MjJ;!vb#t~41>IT9?8cKN z2Pj8e&}=x*+KNJYMA8dEi00~$exXILT>0r^%hNbSPzH6f_)cd&BtYUM0%yKJ=m`^wSbh$+}Rr;W^{Y_J^ zt@>OAHH6fBlL4AaY?jK1AUGu!`}F<59ih-xl+Deh=uSg%t>(MtrO+`kDb>&9Gg9%6 z484*C3!(d~M3?VX1fZ%;acl1lJ?_PB+n+|i)NP_HTUrR(7#4*3apb>O2<6s$*j#Zx z*P@fNyS03~On;;=X4h|3=VLdm-m^ZR*n(+c=E8@}u!t*e8M;i8&u^;+xH|Xnbvu|l zWsI(f+W=kOci7j_Ty7z zcWrZrm-dR=McfNm#K}>3jc+HpxH(i-kOHt{&Z=kb&j#u( zr9!>+7s`v+=9bblE^ar@bQhC~28o%rjW2BHnd!fXK8fhO8pZY_@3#RVJ)5s|&Fbb3 z^FhX24X7>gQoCo<6U$XYolL+t@En1LF}D$}st1W_4^aZoDa_U~*E6 zeP+9wD(<*m=`eG;6Fqz#>w4t?>J`r9A9^hfN()RP?~3yLL|vV!K?r_e^p0zS**)enFOX;FGbJVIh~ zW)xlY-Ykiy{5z^zr&E+DVKdcE7{ri85@IXM58r2Ds#I=R5XYRWT7Gb8Q{#6tmQJ8SnX) z8Absc|2KZ=nDte*DipJutLoz`SM zTGdIsRKP8xy#0eEVSYMw{00fn7kvuztpT4 zUD3<7CG0#br}xJP-U@DIp>5W>F z=<+C}__)L+ms<5Z&cm~i2t)se-I?$hN4J$C=P?NvNYrMXMeitunpcBO`AJ!CpV**B zvxCirDi#mqjUmx32#vW7mqeh=pm_Vs8^4;E zFh>R#gGU8cO@W5ZZW}QQf^}PoLYvB*^O9N-iTz)%1gR+Q4CQT93$f`83Ex-=C&>-` z2w`V(TTc#KX_)%FvjkCXoUwUxjzB2AC5NTJCbrPxdeM*Hg1w3@$1OBF)(}4Cbh^H| zex0UINcWn1;*cwfQAHb`wL8_JS6Gbq%4S}8#m9Gc)14=i)xJ8Jig7nNTAJHC*XF8P z<_fX(Ii#D(mO$n@bK|t0Ej1rora%Ner|2BX>B&pzFF^z`FebIeC{x{|M|_?P55(%t zjgo&VOe0_TZr!Oj_aSXdIYTU9poIMbGzcr348Ir+oj=|^?X)xPpr5?@xq60XT=gcR zg>5cSwV@UG;c@FHcxw?NxuO`|{I|$d7sPx$pF+k+QNyii+|#yJrbl%Wru*{=dad31 zY5GK9%+ZOx1f6}3S<$6@A%nZKzA!Cn9-SV8cA?pKngUV+CgaAitjX|nGu$b? zAL%b4qsv!%2w2}TAxkjSPn#r}2n4-^I;zoYpvt=1alTyd=E zI$@q8GJf#E-ni34@+2*zmPFyox3>(Q`@tP3D+>BabH%Enp`Y5suI%IgftBkIfPf3b z$@q^9=%4H8Z%PBuvJZuJE1rqs(>y{bG-$s&*_%l1R@poPSrthIyikFxn1p}+Z$H96 z_&iFgAaRdGDfd7ByH@u{-X!>Px72_)bY&kS_W88k7qx`}RZ(nWlEGfmA6O;*_|xv8 z`oq4MLyDL#7vsCF_3nt0AA;O(yT~0!zSxW0yw%ZxP);Wi7$U6VETOiRNy5UhyRr?cz`$0iSq>14yk#$OHUu~8hr!Q_IV(;4ynl{NgJv>3Ug}GqWl5AtHS%Fa!&*K_>SRvabHD+2mo{Mp*( z_lo=e^2RPN@FNVEBV)>U;Xinw5Sf!zeXeRtAG9_&~(JV zpG)!8f1*4{@jXv2YsP2JYuRiQDjXE%zn@4qk^%=Ni&xHob374PrM*tY;tI;lYm{wM zBCjrxfQk>$3zp}|Ewb@suk$_db{0~gE}MJ%6U&|*9pwMzN4w=#ZjqfsfDr)ASRc4v z340#Gr;k9j-2VZ57*rOY+j=;j06bxL?(@yRxbmw%&Ts(+zkQMhTuX|H!xy6DFV*ZV z-W?DC=^VhU)g=AZ{&ir%?*#nL8bALZDCrOQ`SJ5w&!@chB%}uf%w78TV_R*dE99Uu z-9W`VS~xi9AAJzQ-%mX50SDB4cM^EPrBxk}ntutDay(PD8jgz(`w{dAljSrZ`LbxC zcTY)P{f+KH>G7}vB>zBY`nQ?*|u%^8hasxjxx@9ug_B?pm>Xuy{}_R(~^|iQplw z`C`<_LA$!=N&IbK3x`9y50tzHudy?MQr7RZB94oD3?H0kLO^WxA1IR3yDd^xO8cvb zf2qaKKdertaBPl;DlPnc@JmTkj0GaGu`%thpsM!t65u`_$#}{BQ}6ePgS)f?;EDbv zP>=9eXfvxIk3?v=4w|GrPa=+YJrO5?M_g1D2Hiji8({Z!TW*489MXos1|lx190KYh z&?NM1Oe+ucg-W2C<9&qS!r0P z)P_*vguj~R_I|LXVN!OEm zW^}a|FMyf~;xiWkUaFK3f*$irn*$Q`rV3*Ihzh(5;U+*??L(nu0^OD4k&i4v^L@nt zAYr+hN)6u5P)u_9KTs@+zS3NmH>rO!)(02Xbf85FpMM$L;i|+_k1FAs`HQQ-Z3P3c0V^vmCD09A zqf!Jw69Qf3wdVDzQ94pO^D1>RtJlemk@)xSQM1$>`sK#*ms z%)twa|J{>qQ^g_o{NI6cJOBu?T7-*bel;=i7yj`_fe1yu{tGQQTABb_8J!%$;Am0q zPWQ;y=NAm0BER1=}`b+pjZ8B%bP4iZk0z>AYLoIzdg zcb>^OgFlD>YFUP{kI|sw^#%@jqJIfgp|pf|@_ZCD=%(CXiuH>>q{s2| zY2&IO<9De4oc#JJoO@$LK=Rgr4UESboCLS~#lQx}j~@ccVYe)XT%%$Hr5NEKWiitF z{pY0z(5v4qi$&SG5`yn@6$rA7;+_B>r#JxN?j^VaY>I9j>V61%`$4*`Y;@X|mF&ywUVn5n?Sf3z!%F}nLs z77ob98<@1jv6T5Jr_dU3%J%G1_AqFQen7&K_*66cpla-S@{|KeSj>zG!D)5{i1+^k z^^{!^7|Mu9Qq$N&z=KI+24%35Q2QgjlXnOxW#H_0 zsVO}LoMN}*sg36rv=qy|K$~-=LwCF9IhWWkgYMIBH^>TCAq72{^w8c7vIKHRz@t(Y zAjl%UTc8L|u-(BphFtbb;AjB_w&D~U(tjLu?-xc7hTjL%8I}$KMsg~pMgu|Or+&!P zPrVYr%f>ts&<1V~rK3|<6}?bOU$`ECZsE%rphkEow2s|`g==&KJYm@#zM4^DX90cT z-QCjJf1vn0G=aQ_GqLI%h+CG9=ie>2ytqQcbHIb{)#SC~z{!i=gPY*x7Ja~dcwN*k z0yUf8fy&dr1?r+=Fd7J+-Yd87*NA`YMKN5Ot2TXJOQXXJyhh>*Y+!9nm$;vT%Ji30K<(1-y+S4UKBIvkt0qyq0u=x6 zz_^FCIo%;;AVQ#$rTw_W3%o%10Ekc~Lhk2)qm=-lbu&q)4IHftiWIxR6b=Cv5AbqO z?eU-tXldMCS`FakqeNw1(3gf>QVho9*bEJ|J)}nX9@xU+(Aa^5B{hOZ=pXX8Ujh4p zIX=U8cRj#qmIlQ8|AC4i2AV7&mr~E{YvDN%7$lMaI&D%%yL|v{|9IoITf6iZCjsB*lilM& z!4-V)rUx1z$g=*HAbda;_R{UL1`uSueD`y9{y--7hhkdo_G0*O`hoXieB4D#bPN0v z9TuY9<}wi<@k4;U1bDex@N$$!6x384{Xz-wk~Q$<--+~oTk`v6<5d6&(nc3{&G<)w zN)RV7&DL@#G#emck%!}dJ**i9Xd!YYA^ZhCf0J2yayOyW!y5tRC2;c)FnKBIb;z|t zHh{!-WF1)ml`-H}Boa7RoD3egbu9s`>!i9n4%A*ecL3fPs_@|mxQqdnzrtk>Y48S@ zyrSLmet|v7Aw5?eyOps-J^Vu`wnEs1GuvYD``IeSNmT z{axa(ybs%A{^d5cQv5@%A^YFj7cXl8?K(hRW#cR*cyk{YHxR~xbwhgWppY?j{i>?3 z@saOeH|gJIxu<_S{NcDmtPz{^C9~^!Ugi_#n;Tn0W(b9d^?W~H;`$5mEb&b*0~%O7 zj$Ft0kUo_p%_1#Rmj7VvLLs~A`65>RR$!F#ROd*HYG_53V1%K~Elbjk*pv=dXl!P%1ie-EX0e8!t2 zh3#yL6QIVKrR~bXG%Ki9xl4EMn#py#E;mq}8PJs;DV*iOI|52hejQ-dq>~x8+cvRaV(n2tqc~Okzi=US zA}Q|r)*cw}m(OyyrFL0Yx5m#M&Q0Ew zSt@YpkpAB+g|ri?O)Xk}vj8hF`~zGli`Jehk4}O2 zXCJ-bszU%y*XKomELJNVLNaOTkwaU?{|`jz7OBmd*Qi@@(lLj~Y4X=B|NX}OfKR>; zoKpX5hQ|glPlu{c{4C)GSH$+IBcQS%{f@gcS1pJKe4B@+@?`&mONS$|T9YXK#hbo`mbSRKND@Z}%o z_<50Z@~)Y91sELR(y56=Pu+3S8t%7DhdDfc!?@fjE5fB`Z>P~$|*uNkilgw_)Z;Gs1T!zRWaZvHz7LmbTy z4X%lIEf0A1PWe0!h))6~T|_}icEm#|#Ld&Zp|lZ0IdIvp9zhVOt^nZ^5Of}%y4?n9a5_YX+3Wuw2T=cBOqVAu_ZINn$GgBi zH?hs#%@X1R!Ph<(wC$Eg|24z2Yn~2OpZ_s8eH70p3OLd4d?}cH82>YX%PW(XBi=PU z&kiv>NxNbG)`=BhOU@j^gy$X)c0+3^ z@X#8FVFB}WxcT4BIuFh2Srh@wbMg?&)4m&@tS?W3&oCan3#6v1|K!Vl(14Y&VF%0v z-ys#A-?@3h*ohFlazXjOrCM!B{1{8zDfp}lnXn_MBRR_r1f7ScZvPd<{*R6A zyIer+Dd3SyjveClEP%p|cbxPPxDN$TGC33_+C|-UDf7XiFWo&_?*o%eRU~E?*qC@I z26v<#R=U+ujpk?Jvk70G);t9L0NP7(Mpy$*2kz9wU@7zD074&Qc+;DbJh z;4?CJnCmxc>{O5%!Wrnc6WosDD(L%?Q9K^G0C_?uW~c42@?O0mVcYI$s?3dAT6=P9~SUrdU2$lasBk!$u6W0RbsfWkh7_SKr;H7qc^cfR##5CcLg?Ep!F!v zLf^q~yveG$TRp6$1y3gkQ^*ku*1SDzz{aaz<#}ybsa1~I_GvNj5wiy1lcGAhuOlMY zc%)kEfe4?c`){RQdoN5OPp-j(o}%9Myv7CDo6 zg7|WqY-8yI$%4)YVyb-F6U|YDO;}VTK@=p%A!4CR`PufS5VEmTa?uR3Gs@iP{Z5FJ z-ZWePk;9nfGu-^iZcfxpJpuyPq2{#bJ2jdWN~0I8WrZ*Z5*a^_8Fg}~yh4BfO0_pR zhjQ`*7aj$zxq|D1Sz{8w#-_!3r=XY-Y`u+iY7Y?80eZ8;G z%edyA6ho&iMH`2LL|i_ACfd+4oP7K>gQ4a6@s0Z5xQ@l3ERWE};-%KkW%|dPV(*YS z_90c||HhA>6B@QRp6)I!ozL9iAvriU`&{1}!l$nd+UVfX%asO@1Vr6(6QBcDGD$N7 zMh1mlj}^q3#8YTI_q2C-mO$X`lZIW!B1uWID&Egzl-#|APt@aqfcIjiM~IVhTA*iGD}PRpCX6?^0O*-r19p6lV8A`k1omM@ zV4qvloV~i^e>oA{a33_cpulD~ZN)3TBLDD&_dnaN;@n!wI_Gi$&YI&EZVWr@&f4^f zFTfCBkdd-JHr;^pp~`id*MCh$3(xNhD|dT4Ayb~FvoRh)QeW6YO87`{@alIH6Wt-_c@wAHf~Bfa3e~PKj^^!mI86ke78<8TwAsy{?qTa-PjYZnm8*Ke_ePc1z~? z#nM~T-kj#3^7riL>8ce`U?h0CG1RNTfK?sKI?zKCZ(wg`(7aP$0;lsr>jOX!7gZ!L-&eSD%pAobnkjbLgB-is1}#`OSl zJlzXZW3@AZR&AUlS&y<7!D0oyo*G)Q`&p{8X;c2=FM9#K4b{UZ$x=%lFmF)Y=ygDTi5wtLiN@ZD!bHN1a7^n28ohS?Q`bvi?3uHc{%v+ss7ODn)O8 z2h_SMpPflk)QpqDH|{2XCP9`X0KoP0Dx~);F4@OL&Ae7H_&t1b&94nAA6?w$jQnL=XbUb_1xC9>aUjTot({aI#KB)Pf)owC>iM`6dBs2?Kc zdXr;k{3n!!@sM=Jee6WE&b6fd=l4xQD#l~jIHzR|;WvF5Qs>}eE=lDu;Z5s_&hSvP zy7wX6ZhJUbb`-xqCAqOt%KadFdnhO0Y7$|IiE5BNQTrQCMM}`TZBX6@H4NQz8~UkP zbTc~Ue&x%SI_F554oHgh%&H}sF|QTuDk%EMxk1`P(fdTYlxnVG=;2P~*$KUwIW}fA zNHZ%Zq0lhqG$iNo=D*|Frgq4=M)W1qyNc4v4>^*3px{dp`ZQS`&xVy9kmBX$D;$`< z^P{W&KGmORyHDJBz#XCsm{l5%D&Y6Qt$ew36L589b;oh~e5^U-`*nFZg4F9dh$u3L zM!NYZ7u zeacj1Bn8AMAYcy0&YdLMw=?~@g_GNtJx?=8P;mrk{_`H2Ga2}mzjd;18&FQ~{!(?< zfSOPX>}17c;Z2Qk35$-%)IjUb8(R_4y>XB-&v5!vC%mlr^a`-wBI#~72`UnhD`vVr zVbKPsw=XIFAy}#Kh! zZHCQ!9g=cAX7qm2&Cx2%mt-1d#J$2((#)JTF>^EHP)J$wj9wI6*ln>bM!Y~QeSvw? zf|)|S2qBIe)2++}4sI(Wf0`+{qy@!DcF4ZkWI1*j{-{+V(KCI( zz{UqtS;?!{X#+q4ytWYO6ior-z^$5~O_T4f>C4T_i{g6(7q>6`si(5%rF1d%eKa@M zP$_+?yk;PC!~>nMQHw;4fC?mI%;L7DMoNw4x2p7mJ^`a2^+L0=Z6G?CEqk-JiLdEy zH`AwDwfDWoy!+;yx^6Z<^|WYwt-oe6j+yga^_ffJ99qf)TI)EWl0cb1Jbk0RE+Ea> z@W5POJnA@KJmTSH5j@#)OirY%A8m_h=xD16AMc+>fmz=iXBxaKLSgv{Yu)rP-Cv*; z$1tWwp-r>N!agVb`I0sR(BZjnj*?Y;KS{2Fk4w6XPg0g?n& zzttF`pdW)4v%(18@ynQAF$By<6Xtb&dvff>z+rsqIoig(tOdJH*&_O`Jb4~Y_w{cc z-cyL5TNgg$HQ_wr-dFeow>-0p5^8sbP2Wl=-TKbMrvd)h&@j%3bItmVNaFo?bOm8Flq*tRfYZyzAe?h31Jkvl@W zCmK|EJ1PzGj!Iwqbg8j73h(U&wLhx!aO^!Cb+^mAhZc#-E;#Y=)= zE5A>Tab~6)mg(#jf(gv2^@)AEHQnNKw=Io92SG+B?Z}{~WEEeCrY2 zgH+L44jwiuZd2D*KI+$qRJ>i-ErgZDOajP(S)h*Z=Av%-ypO)?yQ0Nj~rv zzuO*ir**SkCu?=)?A^4!*uWdsUz@l`?F+ZEGYPLOo)Ui3b)EWjO{V-oeFBk;Uroud z-mnhWl!%@d*Ve|oU`AnG=NM}ZRdcPI(|Wpr#3RH+1qC6LN0XrPK>~$=l2u&Cm!wYE zYRWW20}EEoAoyGU?)GD&dw=M5g{0Ku`c0k?3P87%SE_faTx;0o%fqEr&%2sq7s|c0 zUhJGHfC?OsssOwf`0|Bm7L5*x=N~-tFNA#9T737iX<)een#@YyFLxCVqiYt+0iW~s zmFL^Z%(4)|6Y_+j`VIwXa=8sxD5<>R;+>*aLNwvPYFzHlj6LeP!5u13 z^9ZZG9Yy>~h-cwqxTqbf2t)R;OOHfMl4#i5Lf`FoH`x{asM7F??x7^aep6n4 zM9)gP>Vv;{77iblYR;`jfx8#_8s*3+?6~``*v4AhlhT?oRSu7fvzvrb3__g~j%n30 z6&|tcINEAf1pnzZDvfW){io%LPGtku49S};6u`P_DWd-S>a_;0fk)@i3bbHPT9c$K_vMKa98 za_N2P;}RYpprDHK(< z3XanUj$z_A-2Aga)$o80{j`oJ{rjiaOatpfRPX0mlNITC-Cs1%i<)ni`swK+Guih! zIQ|ATL_T?j(8H7R7u7!z*LoWb0_&yaBcQR`0`q-Zf@p9J_}Zj0=L_B)5hZMYB5@ke{ACznoy^`a-4 zX^NKuyF*9PW_m3Fr1$KM19H$BW@t0~{lN6 zBTT%!^M2i7M|&O>rf)nkHuo31XU4foIndOQ{|S|@DZ-m?vVkKC{_x?mmEHv$#fB!E zwkRkpcX@rtUMFV#IZN=u(cMSi|7ZVQB`TKF`Z?@wOJIG1$Wv!iy1ClDFK|LD^vy$p zT9)2|km|a9VH*BEh}+jLAz19F_3z{!VXVG6woN>j7nY?W*@P05V#d9-{?1x;@lQdFHZmpGM%); zuoE#JUtU#PbyWJd%{D7OZ=qJL;*M=0K<~RVOWIBjAn&ePX%A$s-VCbnO-j%_T+cT% z_x+%_nP;<)3l}mK{OejJq@7V`UP4tstr10)t5e55A*3OUseyI^*V3VgE@@=8cVE-- zU50`5oF{_VxSr}xB@Zn^rkn!Qe)2IeDuGcTqc@vqDMoeZ8o}G6HvLy z=?b??`z$3W=D9byo5^Qmypv}`JE~1*R>rkCU=aHawm$ImC@t3yJk{UQvuXr_6&-nSd^d2gORD55Q* z@VTOC;hrra6RMKjg@P}gdAhE+7a6j#ua@xc*e*#V%%}^B*y*j~w~qwH2`xzA@KC4U zHIW=o7|;hQI?$sM#xz}hO>LoVSX8c3IhY7knW96w78Am-{ccX0a) zV!GOvc`Dqbr2N9VU5g6x=#E(Zs?ke+sdM3gEjaTC{W6}zz&jj>7L8!>UfwTbwf!Fw zVS_y)OjVCvbGz-V27D)0*P?+fMU!>V+8jv==WG z<{u?PQz20O8zM$j^|@4Ce+a`Yha5-sm-&-e)9<#d$;%-}gjGK#{!XIIAd}S}7;Bj9 zc@*?RpdGV2IsL)2XM(@aq--dJtfH>TSW~WtqS(0+xh2?#beRC|G&~l&6Cp%;-r!o1P z0*L$MnR&|P-3^-TC(^-NME$8aH9K(}6RomsmEp@O_B|!U+j;)Vf!ivuDait+Z<=X+ z<%zwWCqg%`NkPZW6-g1_!iXy@<{8V~Wl@BwhD%pTq&-&W%L-BYH*(2ZPpK`x<_*@= zc}%Qv3c;jL#?&hZN9Sbc^N)2S&9kDIh@I>Zs4?)AP3;7jY+C(^hh~;Q&Z%(xI@K#8 z@re4tj{tvp#?8zo+vth(fub8Rc@~99g$$FlD-u)1;?|N**+=zfkQ_lQ_}>O%^U*G# zUjUqoqWNN5<5G=?S23Xq=>0Dd$)+A>Zs-?w^H_~bx6jM~MPmc$xy_8_czCebeJ94$ zqqU^(^jdhQvWuONH`Mdj&DY-4jL{_)E`bEDgA~|sf{#1}MqHTR$~E39D7_K6h!@*n zx@UjxM$H}dP zcPjay)Dk91BqsbA8p~$REKzDJdj2ro)0mpJHlfq`h}ksEIAe2A<tizQ4t1;+RyXzrPNbrhS3DjU@3xRMhO|uOW!*X45}>-rO+-f_ngoJtAUv;4|8J5dnj=oycT8;x3|njc>+r@;A5+^ zS2x@w`5SWk{1xWo*|-t`pKxc!4Po;inayrHqOz~pMJ9D`CQUfWE5G%JF<;OB%SN#A ziV?NH^vkzez}AKLR+z%Lywvu_`odL*utvPV7zgg9yL-$OzyO!!bhhPDWlzfyE#6_r z69^g5D0-#h`+QJ4OeKz>GAFJuYE{F!cK!u!k2dx&CL2Sv%9YtzHpB@-qqpDYU%tX> zam>DlU!S(=VLDhdMRhI@8p#gn8bKkAy&nMqqW^vy-R~SZfE32t@IFdI#Y^25BKG?C}gXRPTCOiJlyoR(7MyA zHlyHNEgK5>w~^l`r3HI4&Ng*^9!1xA-vmfzk8ChwqVvb68m%Pw(dB|@U~o!tL-8BZC@Qi@@+NW>Q#Bhbr+-ZcmmLFzN1Q^ zd+P=Uoh^SVe5~o&<0?`!U(Vxhaz}-GMze8|J~E;QpYM)3oZVSq{Y|sP$tAf*o)DFu zYASA6dt5p3nn@bbN|Ssl^}G1np_+&};`t;+;LV-woQi=l3eR&=jgZPUjHtts37pLE zuKP(U2d$a=DlElj!X?Lv{nizQT0Z87SEY|L#z`)|ZJPdAxj~YsF4GP4cWX8BpY=mj zNqfqYZ!}T{0n|5p++9J#e_@`_5y}c8`es$lV;%z5fNjvc;1+9KbD&RS)rgkigr_N7 z_dB6l(-EEw(eC*jjsGkF#O|-hu}QpaYo?w1T8r4Md+>e`8WZib7;?9b)ktEZo6UBQ znK`7wFrU{vN5m>OOekct2;(x%y@k(=|7ak)8twj>aoHOyS>0HKv|(F0EImNc`QbQU z5c)fX-g}izYUlmX$1YyKSE3Z7X>XOPMSg;!CZ8>EZQsl79anmgCq0w4dKsi&xTSL=V3CETX+J+}byjpGU>jw?1fAw9Q|QtGbjh`aLPT+wsF)ewldsgMG)zF{$=YtFBW}9qFT$ zQ-!FT6gWVHpF{J4>nVOOHS8yRoM>Zy)R1ySM`Q4vs`f2%l6WjcSm<;5Ha?k_zg)na zO-*wcg(d@i8MFil_I=p$1zDq#XN;zLg`NJO>9Zam+!zMZ-`a%&#!lFr5dpPOaRzLa z2fH6@Z*8$l3gn8iXjLQkdssOEhy<6+93f9?u%F0Kc@w+PKEQWgXm2Vs;02AaGk6A%Zj`$_Fxo96AZ zj&{T5D{pq#?5?AD=j&r<-;J3fX&h!8k|O1aMZ3o`UiXD*DZ6KaA1fd0s<_+r`l-k%ac>2qm*Lz=>E|wORQ(E`=K3jN|d*F5Q zXavc)gtV=R+saHsXeu48g-3i03(7lUu?9zu*8HPBG!_{-Xzc{cCE&Ky4`CF33Olq7RBvnPPdK(F zl&m6$87K*p#T`_i`CD~phDs|lAhj*Ciz?k@{y=$vLzmMmdakf*Y%U2e=VXKE!&S4P zyBCI#x!!$aePq(Wy#Lm`)$kCf5!`;Bd&tbVMp&j`pACKSJ^}ta#LPSX5n*Xnz-5YO zJ7L8fJxU3W`5Q5K$3b>su4t_G_+4B`mh`C$1)k37;xsfWI>AaaFV-Gj~lh{NTvYME{J*Ui)qrxEI&UkiO%d=L2h zcV*p`i!&Abi1;_8ai5_4)u$2xTCF)=dpghRgV`bGin4qyaE}upH|+WNAN7;df2-Mh zZdpH5bOSKsOSB4Z<~~K!;({zda}>nzr&+hv<7c7VifYB7=;7L$JA=9}BO-OKFC}!c zusW0wR=-Oywq4G{x%zR|pXKnAIAiq(^iLFNgb+Q`IsMPVn72c2a5Y@}hH)lRG=ooH%*gdzorT6_t&4Q0v! z)aCdkfu*-Ybo8|HEHN92zka{LytZdq{QHo65dkOyFL1OgLzJn@8$N;DyzW2GYE z11CWYXH_14)7}TRlYKyUEiNFrR`Wz*@yc4xE$;|7P2Et#P zhucVW_%mqe<{`;^l6mDuK{8RgPJkqkR6%)1Lj^&mzE2COer^>BDrQ&U&jL&f3#njN zA9giEIo-&|_i8PSX*R2SAkBe}wt6SP)zfj_RopyA4THbPiW0ywPswy2WVuZ?k?Zq~ z)X>#|9UId@Wn}Pr?1ixYhltt{y+^doIq4XufT8kqfvEx8Kp)HC?Ouy&^>(KBF#Uta zw?I+ecR7n(I}2%;R!R6L%`qXvogZyA6*qiuq?Hl-;?D{E$J?6s#)O;(y^ zSq}=pVmw5|9Ikndr43~N+!uTAB#IHw69b{9G?ESl+LA^x+G1gw;5fIn%{^Pz&Kd4e z*-{i0QRF7k^%6DV+M+@Gzq%ZsMAeY5PJf8Y(b&=zT#W7=Ro5|i1*VIhnE7voo}jGM zl1wV7=Mo_?$1Uzx>8~eLvLk`8CX`hInY5teK*@R^2G&&Jb{1}1dJzXcVokgF`f!jX z{ZnE-AYlSS&R!TQv06Z-z)wRsySt>07Nz1@h{wiQ5IV3^wz zSv_yU5*2MJ)F(Y71B0f>3H{_;)npzej1~yWOKc$%501D!U@~c&H}*d%P*JUODf2@R z3vrPKfh2Mzep%tfw=}X!#FK~4ll*v$w3g4VfW~v<c@@|t%ddgpkleY7m z@#~6%{)2{Bs`HJCu}lr*hz}Dq1h`c>+%mY5-GkC9uJJ%hAl^k>K9@=7*0$tU*q4kb zwRVz(WBwCpX+P5T>KPu#u(Ov0HI6$%%0cw~-fD+Amq+)V5UuqmuP zh1AsfP{Et93C`*4=NRoiWH{XH`8{o*#Iwimjh2DVQx_Hdm}G&qH=fn#(x`Q72c8U9 z&4?KfF9DKFc`f7;g9pOZ!yP#&?Btn|WO?mk-Nu)!ZC)K<~nhm=VL{leu-=i$14Xy49YZa?szzC?oM%ypU}q2j+?2kAVz zy~6{+On*%VX)R+&Wi}l04b8LI5aE`B9D2olhwix%U4I1Qt4}^%-36@in+u3@b|_F_ zZZ|)gX{>K1YN)3;IB3{t>gYdNpNabv?pgEzH)CdF%9!V*<20vDeogay$$$;nt8{%x z8TT7}Z#`@~yf=Mk;RVR=7#U|f9uba^v?S`E%_xX?aA1;A2(DkpK8aA>HfdEdbe8!R z-8k2_iS>(GrCwrxLf z=$J|}&Emi!y6XQNdwKQARoTSULxiD?*xsBUiphVyivC{{rS7EpfReVbYs8$;|NDFI zDI|V=x+mwLs@1J6ZPl57JtY7A;G*TduKu@x`7T{Rzto|1d=N14uRQSoZPWRmi%fJs z8G-x9p}?6;DB158u(_i5sQ*pPvLQ=6pf})V%Z#wH_CcCHCYxr? zZ+=*2lao!udqF{4A~!+AAlK|9S{4cwiMTv0^XywD4?vbMk=Y}iaPhZDTI=M96g&yt z5^=@86qm5XD{PSkFI4@0{Uz4&& zT*0QuLvorgoMETo&AC4LRkUMV7cQx5(8@!_O%U(5z&`pvABgla1ocIl)~{b)<&}u7 z*LeQdC1RMO>4*JFy5%i$`rs0CJq+x)arqYXhF&^WRC8?YC+W-jx}9a!@Av-vf1@sj AUjP6A literal 0 HcmV?d00001 diff --git a/images/sidb/database-actions-home.png b/images/sidb/database-actions-home.png new file mode 100644 index 0000000000000000000000000000000000000000..6c272f4683e4963f0068b4d7813fe4c20a7897f6 GIT binary patch literal 231151 zcma&O1ymf(@;ECogCA z>@YpmJyYFP-Ss@x8~ROF3;_-s4gdfkNQeu|0|3xI001N*%m>gFXk~$V002zbOi1XP zgpd&7H#-|+GfN`?fHKxVR~J>{3+0fWp04iDI1Lq?owIyMNQAtu_efVaVNcgE;h1h* zil*i!Ch{hPVh4|5k&OdSy_O?-F6lc;9~!cHb}aFC zu``O!S?WiE2E+A!W(V?YM|nn zP<@TRb^vM|<6`<|Vi&);3-}E7&kc$Zbq%df{r2~x2ADMB9?raHmt!v_P}96}7iFDI}Ny;(=_@P5CL z^^ceMZNPe;`3(j1zZ0*5MQo^s%|t{JTArW5!cU=OX-rqYL-{0Sz zu^=FBXmi$W034D!TG($+p&+#?8>vbd%g6wzLFX_4uwXL)B(SjneHnwFB~BuA-A2OF{iw+=zoiY{^KDwb#Sodq^EawcBXS?qO-9x zp=aRW;GqA?NYBVf3%Y~W-qqSc&xO|7p5(tO`JZ}(jqDBV%xoRZY^(|Y)T^g&G)tPTYoApWX*Sfd5l#Ho{#c%x7T%5n+)daN^%k!fqGsB$Tv)vlng7 zsYbP@)~r(CupI}ktX%d7^BZI5`>D*Zw&Fi)xkl_Vza8 z#ux_)Dk0cEJfa~51qJg(!uR87He^sSQ86jQG3W~(PEatAVgRuJ=-}Tc1Bh+(hbM=~ zwEICI5%7Mwy1GhrZpgs;XY!zCd{n?;VPWlVhxA}q8*Q>l$;sWy$NWk8!T;frnhb#4 z^nAXT()IAj0BP1_J=Mz>aY;#398d3= z1K}l0R#*gt#I7zvSU9*iAD^zfM68(Fhu~CDO05*!kM-LsLpq6HD*fd0udfy~`Ab-aKcxxX{ zfB)SoHbu#NNK={lU_g3#oDP!#NmQ`;R#i=%@IXa@l=QcegC*(%UX^VM0TDVO=U^!c zzL3bTt2^b0m+c?_*S_HwMQtikErRj~FAxzASw%(tvS;P=-N;1n=L;4F{m1R?T^D`% zw0&*DU*ffP0`ir@K8j-my35vlsb9SD#FIo?QTO_qnH?!YV~M8|{YV=uefrRAJ~j{y z#*tb&*BGuOQ3{yM%xBwMrJt2daX9x|p%VI9h!awVfM2rojpzPVITZ|Gol0Za)IRIU zw3;pGLo~ii%oyhdRV9fA^pb;zrl}s=brI1~u?VoRytKt`TYw`iHu09(QF6T{mNq+= zDl-*Ya&1n{ki=qdkx}PTRHL0~!|bPUn$MI6*ie$vBua@SJ@uW)Gjy!vvic{3L;Xev zqz5&mup9}ju}{~VPQFjqyp5oztO57+0k~b51m5&<#QjaH4`PHxwp+xNq9P&%#Vp+3 z1kpp}Jd&T1d{&!91Qo>)zX^Pzzx#j~5)wnbm5YjtM}wmT=!iMhstY`@*$`ztnq!u? ze6$PcYMno(UnFn9)srvRje+bN945cpjBGjdY`J{lSs@nJSKmh)F6HV*j#s+%z0A9b zKi!J^wn#1k%Ya$}ZmUcE+KfZXttn!|NBi|Fzq|l($Ly883oD85JGbxRV#+%=1o&S) zYtt2saFOguGW6FT-rgG(yCPK@>T}MM)TQ0-@-MU$OJoYb{-xi~3r>aH;SaI;SoxzCf5QGD&d z1IMp?h~!5easGn&mVw9eO)jv*nG}`Nt#a8P=fcpPC=mRpj4N z4BIvIXwk!~u}!~TJ~S3Gk6i>&1QWq|Cn;PCIOU#7T=`eAfi|*t<&<+hJaJV}P!z4Z zEYv2=^UzS7{+@MG$1Aktqg?^%;%Cdc^V;c9yC=am(i0Ck0=_S3yzl&o7wQK<=8*(i z+F(TdIQH?ujP=hy@i*Gtfk~lg-Z9}AueRB;7qHE@DxKO|u!{6|=rqbLxWk;SD93v5 zn-IWJ44tUagos+iVGFjI?nwkD|MvF@jRos(QNn+76)|KmvlSY0Zg(9jo4F^J$bMJm zYlY<=?`w@U#&bFo!LQ5rG+4&28ZkTv^pMJ<5epRxV5}_kRmB!_SC`rhonh6Ld{8p@*8-dI7 zLlP11U8CWghdLdIo9G`bH?(CyQNTZqdOf5dq?oTfVfHm3690McpvL-4hjH&fqUN{~ zoe{X8L#o4jH!mwTn#B_g@A5=7oXMq<=9c;XTv?;{YJQ{MX`MLsY}LU1*k`~Cyt-ep z5wF%4Q7GSNyvVzLFnEeYCs=NE;NHVxSK2cE`;xSt4F0$VVU-%yLTd%Bb{2~ zhw%^AE*Y@=FP`~woiu$q2b{=3-+$T7nNRe4Sv`{ho>@?Fu)x6t1nkZQGcwwdX<*}I zoU%fa&%}1fm`OFQN6@I6Nk~Z5?HLK)58BKZ%+P%J9$oThE3~AvJMi@8gf-aO|lR8J0OF;Z~Fz)dy$qrlOT_?WEU&448xImes5>|7~CT-Q6Q?16OBjWLeNv+*H(Bp0{xzKX92<4czZPf))?!~$-z;w}C zwL-JmaCtTxa=nF}_w+HhaP>~k|HGZpmY-`Oew#3SC-5W2ZeM7;=nLL`&nTnO z2=mcWZE)Tw@9|yBX|zaGM;HdZag^Q*zzA49aaSuRmdYsS_yc*HMp$SJ&%X3~UW=JO zU)FNs7d1_YV6$vu_fs zWSjQa#l$-cxD3ldrNyF&4g5C@tBL`_%EG0$*|r*q>FH(JL`5lSN6!U_Kb~?)2Re>S znMXhJ@?-AjTE80_?ZpnoAkL!nw7|1@&hHHO&Bb4j4+oDij8ZD+85|W(zT!nwkdsU3 z>gKq;1dS{eK}yVKbcV>{;Uts3iw+KNdH>u!Or;Esl$$d8)uC)*K3B3NvVhe&RueY% z{Pqe>23i>&&zBqA*P3bXtJj;41e1u4a9y;~Xtg@TeQ2CbUeNJYSogTWE7oj|<9?a) zFFX%`)ftt6f*Y#RqaT)w|Am=*->{-yc*rLfh0AEZ3xm!D97dDlrG0;UuJm~9Km6p_ zb|#&|J`K}qG@29(4s|Wn>E*e5FsxuSm17dQaDC}~`lz_Y0ROxjKe=m_+1S`pq*9^D zu~h3G4exr-5gHyHKPl$@a}}bk!Tn-wD=sC4(s^x8gC`MVq;?XK(a$VT6k#uP@M$DJ z+l%#Pq;mUZbcTK z&2f&kd4s9SPw7`0bvG8J$r6w)q0eC zm=3dTyjJkhZvRr^uEMdt_Sj6bMp#1ZYC$JD`J&@_{q*7+irvWUk^W5EgzNRb(#0y8 ze7!k3Nyldzb;S?xxTGI3=#rM~y%Nt>8gMSO#Kc;rn|_egMs*7XNyddOO%~a5*$*hj zw|*8hGHmpRJnJFAk`{FST*4yHILM|4KuMmA?Gxnxp z(8(xRzx3l#p<2-OReU$*YNwOM9`kk#V7A^?tOuif*b3wUj)JBFqpLVFyNh<9c!TBE zb_?%;=3wLc@(@fna2@D#b*kOw5?c0RpIX=P%Bt4nQk@D0XHZ+wCKYklr-LYH{|qfpZ@Wwh0UG|6q}*b&?40Wt}qU(V6YZK-ceZf4W3_q2m@q z)DL$4*NA%c5>hhZOR9b(kmoi8eqpF$!l^TfLqj5&bz;zC>0S!JTcKUHlQ-9U-}U`S znXnSFM5UM?KFDm*#N|zpP!w-gW?VJ{?a>-NV6pahfp%Mf5z-81$Nkqn7zBh#9hkm9 zZb)rPEjDBH>p4}ZSw699z`!t~3Ik@LcEjaM;6pfPkvlDdUIa;Pu%w5kUj`Qe?GP+( zb_`)RwcQ}aGU;*K{s+V*D%kFKNnOFJE|5egktH6hsqKzm z!~km&41_CO3PQk%ynOCYnLf_-s69f1p2%|a?%5hYDL;=T5yP9|@OlYAi6*kVSgs#6 zXkF0u?hQzJJ8!+Bo+??mJTNdkZK^C#EZ&xHvL#Y$J7g4ch-w=_w#z)*62oU~)$zJ& z&U@&1K4$^{swz^<)-XpdAXKdJvnH=59idcfmCVKs=IoNVLb3R#5FqG)9^N9*XyFn- z(`cD!HdC1XbvD}E^o+IziGC|@Fe;T(?G*_h<{yOg{K~|DAB~RbT(h}04XLaRS|Zz} zTG2Y-Uq7tn{M>W@{=H`3wr%yNp6O^QG!;CDH_AO$R%b^@XJPEyk_)q%>=)@6P-Kv= z9co&f!qIH42Z8FWxdzg;<}Gn7j@%tIj|2+U4gtT`a9Kwb^$-+dAJGbXext}u2e z$jKxH^=o!V(TEP{mm7ryA;`GJfjfvposVLpkC*$=Pxtvx5 zj?RDI*;VL^9nj&(q%*S;GxaGirZSo27Y3OG1>a?@wUTM5rl7PCZTFOv{5oIvd?v+4 zQY=zCVd6@D+tkd-tkshlV#r8qzdn8+PPb{NEmyCPQ_*pi(DuADLV&hH`jx4^GUl|( zamuuB>lA!>^I^l6C2qfw)HTnE!JZ zahML)3Sw8hoH8=V3WKNIvDrO1hh3_ddBbrb^BNQvrM6WMEpR>6M(;HRIcm2!6t^17 zi{C5_3=WPS&HUPn;#wA;CKi$Afr%SBc;3gUl!_$|gz}V0f2u`6&E!(Q9wZph0M@JM zFy=-u-0RRJht>j4H2G}SR|i@#UG4au9TJ$oci-+r^RXzE{lbaD=P|!ta5`OFZ!U(L zg#BIUXhBj~sLZC*;qkljBF`8<+w-X!rMS^@Y5T#a8OLDaH1#^EWmPC5i@`n!iwhK* z?vFofNhZ^govm!>UEf7vvj~1$m^-%U&~bqxxn&MeX?x|Ty{J$`kkMxs5x;|S1QLi zZg8;0dRsD&Yiuy4Aa&p3FKGB%b>~Vq@(u5I&+8O7OcIRqZ{tgFlSVcMpx|QRx-G2c zY_*)jH$2}EoG?yB>l=pP3ozZHY(8R_iSI>#pY;IRc?S;$Pl$Eo_dPe>GqVL&5PUDXn zg_cJ&`@VjpdtAWA4d#~{2yzOh5xOEV*Y{u0`b?usR*j8zA)WKPo;rYpkbyV^|u zG2Tos4p%q+*T#!ri6SM*$~`W6)gw>5IMOLmk6W z>O0LhL<23v){JEr7!u89c`^aIjg-a3*^=~~$D0GDpXKOji_PB+4P`?^LzRiTDO5N~s=dXqCR-D+MVcxtQWP59cgQDos&vf10-S{_D?p7ZD_%A>mAQGkKxE&Vw zllfsa?Oy=>)h0;ug>s7M+oOXSa4|E|rSf#PN5uuLf|th|sc&07BHEF^H9P9+P+*XN z^a<8Jq*!{aI$L(A8BE!Rqge{X#I+Jj9%P2jw}@)kRWcL(??kPJt%lJsv0YTG5-(hL z$Ws%wDhF~L#o*+EaL?wKiXvRjPFLF^cbqajNDNQnNa54S+gSs>xS7$erQyweZpdBj z^}Pq(cpnq``xfo*w9h!pGde3?n<^&n*!MG)!=vSpGlZ6nz^ki|?D6S=l=8)r=y!m3 zv84Ac-qjX+E$hFIj2CAZTrM}^C;~om^f_N9cJ{uro~=i&ry02l$rq796ciLeB;Cdk z1^U|sh{a?yUG*HRD)~2{iXYqwTu*Q7cb%GjbF&j3)ekDjTi@4bs6Pt{4j$aaO)zP& z#YviTbI45q0$E1JZugq<=Gn}VXCji8?bm17%s+nVD z_3M1`-?mBOOwIW6m$C6*3tvNG@J~Lk>{!0Oef!X?tVF)jLO$7B()QHsD1U9(ft;Ug z1}R5CK)|a`AB1W8jtPQf$Lz@2k^U-sOU@6ri1brHfR#_Z8WuK&-(lUc0}_mk9!vZFKFFQ)v$*{%$G%jp%H~0_t@3 z0rw%nf3)$B4nhkQ7ha@oL|24od0k`;n3o zciPvG-Uyp)F^v@e)zKJ2frAa)F!Eap2%vs4jNjbUtv)(YSN*2z6NN~|Dj_SWXhkc58eh}k5_V&KW42e8KnA1PhDqZ>hCR(DC7l{i`d5nB7bnPP zjfh@t8!he@oxU2wKrsO!Zfff5%}7Y#vZa8iZ4mD#9QSEFNFOzB^3dKha_Z0T@gz zXtG5mz@X4uswq}Y=Y31{J-+&gWFQ>GxBaygGA8~-jO4% zK`X!wQ7QL&wfP1##I3N!gbm~8t(L_z-5f!&&xm}PDT9L7;U&>hjZx{BjLvUK^v^;f zvhJ^Skq;ZiR-xTnTT7%MNXqVepk6@G1oa1`O`p%S8Wh=_j?|@NsW%S9fGalYV?PC0 zwtu1-MlRowkW;VlE@W?^*&EoSp`c6e>bdpcz zC-P}x!Yg5$ul?Q#*^gtROeNvXbn6i@u*G2ebgELq(;FyH$$+fR)azSY`M=_xq?@t) zJ_ASNStKTDy#ges_9^?tufPp&yzPs_Nxo!irBWB%(KTP%^hWW{4o@oZR{tIO4=C); z=c{Rxk3aX7l!43A)}Wv-p1;qx(1-_o=uzs5?Ie@UVgR`A9$i1|{8zL8@-N2f@_I1m zivL8S6cpU{@RCR~aC>{F@^FoB(+>pbMl>Ay?`Hy3{W%s*&})eh9o*D~g_}GsyMqfy zJ3OBYOcr>7b4_Mwuv3Arj|XtC0Mdumb8~9_wMbrcTZauq2d1UYCra?H}x%I5t*4hF~sK;sFlA52VLH2{BtFV<4-D; z+B+D~raV&~f<~FrZ5`IC%Aes%P@$$tH`nAsji;;xSE=Vq}a#A95%=rQw7}Ew73C4d>rHyz)_$x>vJlNymfPd z?|@vR(F3!3gC!B)3%kVA{mBlpHv=X6>ddpXJ-j2~o73s+R>^@sy?<#nD;}pr`h8|a zVfe{1E3rEwm10u}B$Uxhb9=?daZ-PkGS%-TO65*6#%LgWpDR^KmQ18BIB_{_)NXf^ zSFOa+p{1l0hxt#ckXm)jT5_jcfJ zZfP04jZB45?Fs@(fGu-;j%;rnFHGbE`_C*vPm(ivV z*b&D9^Ig4Nzg_W0PA<7oLwFa~5iGnO5EQO3-lQ{l7D{I^s@-33>E25vQ4_7TI5GXK zRHBgi_(K6p&WV)#>D^EuGTnkId9~2Q(nhs$ew?xcorf1hT0=sPq(t+@Jv^|B<)vR^ z#JCf)C?gWo)!O3TR(;QvN54rJeQS5;o<*ioZp(K#_WBiOMHI#94m^$1DU?kL z2(Z{WI>UV6w1mg!7SD3KhK-+X7adnu83Xu@mvp`{dw_zvz_!PI4X?Rs?D~#Zr^nlw zXgnf>?Ym|7IypYIDqY`V@E<0RH3iC!gU>CFCj|-zbWxrQ2y!Pb+o6MdOGN&6X*GP@ zsFe$kXwiIBoo`R278p^lT3n4MXl%Jro|c-km@j)HDh>&+7q7s4`vp2wk0U88SCkfq zGa~nPO$(Vc=Gc)$`tT_R1_nmYC-`1`JcBVrJWg>Ckc78`%jczOJe~%MyIY;T+~%r3 zbY8nWM@tlP?2;MMy zeY&F>shpd*OUb4Y^PW#zUnt|V*Twmdu4sr79OU?wO;XdPfs5;nHkCR;0yn<*22x~g zwlSRAKP%9m9(|z+9Jto!X*G;C);(2lg|MQOD)fQsP8B{KUT;Sf+1Fqjjw*6$?Z%O( z>wAw`rS=;O2G48G8{frh!`uiu?OPC$?%ARDHS>k&HbFis<#Ee%+j-;Hisl)9*U?la z*VO`?D+u)Y)%+0z3Bq{npK=U~%juT6cXjz1dsTMcdz07IlD+*zUV#C&na>?FS9U8v zNNHX>tZHyT76-|sah2$a38@$Fu@C*>ev{YCv&r%f! zaRUnwpCUnd8w>`brJMBvgscmt8~kr?&j}GY&w;%+*jb5H>%dhC5T5ARmgRCyrEvZ< zxWc(v&2hU##Q0+y>6k-DM=yyIYfN_5u4dq-PgQ-P6S%QPm95hkBg7I-sG0kCB#;&Rdv__C^w!wl8`_!V|hN81;ID#~W#XSFGVh zvC6QFd1P@n8;vATF}mFzh~fq{+WLP$_}z}k*~e+WKbA|pPbc=iV2~#qx}CPkD*-Zn zC!V0m7zR+1961v2gU{9deS%86VzGRUNwoKSo7P!A)AI{=Ieo2WZPxzae_ppXse>cn z%qBxXBc@?AR1SDdd-?f+M?!oWFq>&8UzwU@XLUSQY-?tb8$>p$`kpsnz0&l#<7(jK zce{ql!nSlh(l5dT%9_qMPAPns%{PZsl8h-^y5GIL?;a6H18#EBywkyJ5D(Puqn6v1 z^|(@y6XJ54vYA|^l<``dk6hjzYc#*HH{8S)4?wo9+N$!Jm2%km%Oc*GmX!O&j?vCC z&vJd0!)x+nwYgTlMVg~nwYkJ{qh-l}&uAnmeqdZf;+)%6u`eQSoZ6I5r)6>W;&~Q; zj$2!*nk|)u&8%2h!hUefk-@ehBl@sf^TR)b$90ivK`MyEX71c(=7C_w#l}K z4Q|)n-7V-Y6R#&Ow+hGk-e2C_wIio&Bn*k_a z(sJH{Wh9m)nB0Spfpsx=*mbLvPr&GQ_GVNDb*_C7hWwE}!=c{it^N#gO zgA4(4wR_voHxLm{n$BvK1cy$Y!i~H$5INH%4E(rUi=%S4FNEzH8s{{8iM&u;o%uBs9aDp37zT z^SB2RKDX)d?s&RNi^CBy-_gvECfVMeo5=ht2KB= zoWbC(L8*AUJ@OvA`!J2==#iOi@Y|YIK0TQ&3OU!Nv%TH}xaREE-k-7&yPmcVKw`66 z)P46lE9(q?D0_K(_F!@=mZz#zV^h#a`HvyEAr*Ka9QyBP?|aR5TT=7YrfLrkr!uu( z5+uGa(1`eqK(6n|I5_F#V%R5dxom`B)aL4fKdgQ+yHr?3;d4;Wl{CwpgcxMOKl-6i zKZ#a#`v>Q%esGA??++`aym5v;AY9FllD=*HD8}XTTKMara;|zE6$9t(!6H@ZrG7qs zn%K71czo;|%=78QZ<>Qj`wiG*_4DXGC5E4AZu~vGE(ZuVZ)5iKX*Km0%EcyUR0>J+ ze1p;Wu@KnvIWo_P)XVsaV4JJxTD$b$#5tW>6(+JcNTfcGsl?Vdt^SmY>l^GJElC_T zx_a3t2n_r9AgEc8wesGSI(lUwHin^5tK29hi{3t8@g&usL&@Q`TV>Gpk`|7>lkxOy zqjveMxg)XMq6JNYDfDgNYGTd3#MW6kiB6r6Y_=5XrmM+9r98p*&TYI!LGDF09_$ruuSj~w6fuSygXe-_OT zUJ*n;$6pk!uc94iYx+z)@x0oQ9PVa7YqE1tjyM0>QQjE zjy`VLCo8arkEgR)3~lbq|B|)Z`_UERgbe%suv7CjRon6FV9gVte>%^pQe$`6YG&UF zoof>3N2~2tkEm{cvufi6J|~ui7~Pux6VhN3ZMqUA5@W{HLR!5AnV@%ub$dOE6Q0w} zIE&I(I-7vqq9RlD6dMM3uZJx$F}wg%?dom0+U#X+OV;#7O9A;#FP^0at5&#?!SRD% z?!byC2gR7Oxn@zr@R%@k8qAAhET#$%Td)(;*`i#~BCDqiorr7G6CwBRa5`_m0^?u{ z6xWJU27Ob@6LlQ)C`SmXTdFl-6Ew&7gn=TaaiLYOQ+=+l94t9`LX>l!?e|AcL7IoU zP#!IF)R*}h(&^G=SRHntm#dc+-qE2~*xdN3UPO}g1ng)MpK{?RP^#7BZ++|V;2;6)`1*npOK|ko z()@2&EE2CW|6|->p(ZPv|4syw6DXPj7FprhYB9N;K$ z4t6G1I!yU|)8f#8)Ot$PI#>fs#}{akWk=EI3MR#M%R{jQSbFFvUw&*L19|x#bOS`6 z>w=krE}!leb|8V)49@J9JlEm`dB9EHlK|MAaUQKzowY`r&Q<$K&jAkDn=YPnIo`*` zRrdgUt9brR2KUSsAQP{6BFqF7u8v{(UU z8Kg@yYq|h{r>?&g%hh{VckWml*br5rCqxQa(Y>v(mu7AJ8}T6_FxO2cQ!~wVK*8x0 z+7$hqTZf`)nIM0K)Vf>8Y-5iA2)Rig)HnB3w{S*$K3)B=m3<|tb8`{FV&U34XvVBG zxj@DDM4;i(ZPO{abd4?@vR~Qxa2odG*i2Q^gRlb+7>J^d5fI{#3Jn=5$WYrjM?Z|RruemDncz!}38Ph0W zFvWL&%U`Te*>OcV@T1T8UAY%CZ~Zut1`?d1jdDi6^PO{rVbUqH)eVn{_Vr1_IpWP{ zGc+Cy0^0SoCz%r;=fu~`^BJ3d422+)+44JvcF%)+xmIb?u&2h!etLmQW$PHmk&o}B z%gMwJg9{bk!zRQ&dz%jzm5$s8IVII#Sv;dCxJlQWJ(iv;ZcoADH(G>Sv97FM;CbtX zeDsL7(Q@4p_2PW&^F`|dOwJ@cw&gOtcH5#0fTWHAK5ORz#SdWqnzKXXyEm$mvhz^u z4#Y>Nqe*+4qJS4%dwYEzXMM|8!Mla3v~k(#u~wc44V5_l@l?xkaoH6B1Wm9kc;H`; zx3a@I?DusQsEdZ+a_z2nsjx>@&g&yIp;*jkzYbneWVCJaQs=Y^dY6x=lt=#Bf@Jm-80Q@vo zHf+cdvUltOWbS*p1;mY0hq0#`^vQorZ{as4^PvnShFxBd z&wR9eC3H$kgC4JUY!UI3CzYK)V+Bl3JU5n63uCb;WHWMFrqx`xLw`x{O4ilO$MhV* zzBRxWeHtv?66 zMWkP+=ffit2~SR-#_q_GPb~SJYtObb-g-Q_C!eEBgihj@Kez_uZ1Gt*mwEh{_UgwO zjW!sK)jePGRi)G8w|Ow)xihnbm9$tt{4#$?Xc~*f&}e1b@l|kNGUs~XLWvR$7V9a; z4#!hBGy!Lc^|xrPfWh{6U+Bo9KnP?VE(g;}{W@>(%cjZ><>UXj{6vG&&w#9}~y{A9mk57PK zCedn07nucUi4Iwmt#M@7E!FHv!x#quf(lrX6eQkp=P1FF1U~fv13@7(?#}k!aEASb zz7>oNLbSFjFZ!!oHgBdM6n8W2_w{2my#(4T45Lj@QkcXJ`l)7=!9NS&g-%3&cL_|H*b#Y z^}LJ+Psu?}+}_s*5Y{&0oXo9TIp@}i^&W`@H;wwE6CK#JhDFS+Tr5u}!YSl@_gnsh zNd*|hX&T^s;Vpl{qe(1mHajF^saZ;8$&;#QKL{ax)}1Z*$CqO5_OuA>yHJnkGe{=f z^HSv>AXp7S6NjH`f*GP|_y{q7wFQ=--4ARoQK6#+QagysZ4 zkKXA3ilPs1&2C#+T2xGU8qTuz>0qwUC;^m@7L1Gc-lV-i5F5D(1-IVT((@P-#GKU7 z!Y`~BEC!?Wg0yEQmXEO2W-t>U(sWqmCo*^_XjHz)m4{9ag4Skc+}Tpc3LmrDe99+~ z>D(`4xEy>T1boU-6RVG zEFuWex#QyGgDPYdqjfO@Xmy};OFMh1ySD53`v=7ChNIOTPhcT+L)wu;CNdruO3WMb z6afLv>A`v11qch1{wKU8WESi4qoAm?B>DSA?;SL>jwbpn#PeC>x*fcII$W#?^Db+a z@l`=oUhe1zFPM&(ZTnY!Zt67_kA~VvTnF z(GyW=eD^!iVv>_w7tD{rTF-s}BiDZQL2{X=VVJIm*m^II*pqp+FPS_CDp;2e-XxpE z&XuJ^Wsr7IlbP<1uyimQ&a2b&bsVnO`Ec5SG82nR$=B!joRIXB$U9__1V}C;g<^Oa z8UZu|(G6sNLj1zbwE~KPP*70ECS6{ABQqBCjb`bi3$!C;toO_MUN6Y!SbF6s$5{C5 zFzuGE8$CPAOT8TRuBxw`a_Cku9=x>1u*=FgE_?==UqgOXoOJzAv8S!=u(!=>!wQhi zkaJPs(ipi2Oh95G%4Zh7IG!s#Q7|)#_{C6x#{I?T>Z?vbZX^5(1|7Tm^AqCG%u88E z=0^MYC`J_W_SRk0>HrLMWH7{~A;H^`pSX1(O_O1DQAy=%Ym82Ih=H+PFaMZzNhtK}Arib6cV6JSc6>9inShI7aC#ff|70gt>7ZBX-3D+%DxaO>9PVN0hiF9S!yiHY%0epLLd*^0vIL+bN&EPgVmW zL$fk0v$?J>1g8W|zmF0Yy48S8qO{4QG~)GC!OFmU2!P)|7>%bSx4m76HSmW!7<)Ae z%1aM9y*F5@>N7PanTRIGHz>-Dbb(Kl8wS43c~G{5_q0B5H2X zGB0=Wy1OeZF>8~?b=LjqTrEdxuD*&L4z zhszg~Yxik!$G2R=39{?LI*}_i>P)Q#*)%fC)!UaXlfJ0^rk1>lf|kWUSx_-w*VG{3 zuJ*1VRPCL&QLS7rncfd1U^XdLt}l7q>Y5Qghesk))h~5?*%oPqwgZuiur$8cG$)Id zMj-|;JhAwW+uL!{hj0f33{|v>MLEsZw6Pe?YDDHO6T{Mi@4Dqf3t`+%NrNcy^P-bVBL5~@FM1iZ?#YSi16!qL+`qKpBtVU z*BRtqW(=|H#szbNx6Bw%cLtaBeLFM}9ti~n{Su?sKH~CS7G1nVhg#sACex_Q@0L|n zcuH9Y1()7MN7Mr8mNyB~NyMU3Uq&CqazMcNzzjC{*A%^ukDtE?D;W#{)8)T4MCHGo zbMo2Hw*9d!nC}S2jq7fuu7qGnN?q~TKfhMs9RxFgQtyoHY~0h4qW>ek1)7JDLA2;k zTEQ1uN2;ge4o`O(tf)tWDKGh2(m9n)L%-)PjRxy>LBN~&s;XMcDUul5R0T0h4;lP+ zwIH9sF1e^zvjs7vZ23SPWUr-KFS9@WUn85KMp1`_q|BYF-T-c%0Owt`<|ez zaPo12jc7FaKr@MM`%j{(Sx`Vr^nIGJ(cYB;6x;7~|Es;}k zRWgR^yMl;b%UVgI0eAcTl zsxn$3D2#XKbw3B%;_<@b@+g~mn}?#B`K;RM6uJKjA5Z6e&kSzquaRy#q5C>zey%er zo6SiDGp#AwJJu8bdk(b5Q{fhvdF~F*7asooC;ex~D&lf381a;8j_r`$<+{*d;e0tc zXt!KT?1}fW4khZlUM5`F5a04yg*JJFgTJUuzEZkVAw4c12r_wOG+(KX|3vd$lGEYH zSP*_Qg+j_oTJG$3X#JbhwW@T% zYK?c@Xp4!tmr2MDQ7!)Gw*Xe_jo!u#-rpbwAU<=1cO&vuZAQQGV6{o%uyxFpfhuX( z&mfUzhu4ej{;9>q75)JM8Fguwqk6f@+^27uivpBjt zk8YlrR3Iq+*M!$el(N%OCFsA?4XK&!VhVondH+1^j+s8`Kjx0`E0g-TmxW-vI9F;j z-Y#|T2U;blna%!gq0_0$Hk{M_a*)T8{8NE5T0xxzl~bd+X3wbm<&n|z*a5A0>QS{@ z@G;YQ{Ql=-GM-8W`=MC(NTPSzVFoD2v%HyNt;R^okw2}})i0X`7z#Y`Jn@CdP7*CK z+SEQuVaS`y<5cGHWE%xVoRR(u(R{MZ%e@*l?Zsv!8jlPg-Q#btsT#~?$oJRz&K=!w zxatLL5pC%hYXzR`q4Im-X;B_}QAlu!FY6JZ&)Om^h~D(M%bt7RJ&PVe*~XRF*fSm= z$|lNo3bgP}nzvF!F)Ms@9mV_(tG{#6sf}+ku0z8j z_XF!FUoYsF^wV3IN8kI4y`-^h`j<0KCuK08wb(}RU2&}1%&)XpPj{!$xWJo!)RgZ{ z-W&L8-0#lzxPn(NuAP!Qg|xe}GEWFAttQD0YPt=LR!gLF=JN?~=rrjIP_Kdy8ojUT zS%LHF@tK0QzgP>8*zw-G{Fl?@IGrqdsQATcer^-M=TgT!e`e)JTj5HVArO49J5_ZB zwgV$#=G)!b5Rlf!6$L1FL7aBl+0+zreOzO7Dt z>h*Vt3IYcjH<4dvHn{_hz%cI)#?wuNPxi;t)AxrG;RP`Hic&UgI$tX?G)Ap{V`iAHP%nGRHK<%*eByL=56oHZ@h)>?M#|yi-OmKDUj= z1>$;VGKpm!Z}5rGZL}Yc`8}6|Af>NsxC-t2na-UIE0-S{;2pm$EPZRROy%LOwRbgN zX+kOv`sce zTLzc&IW`kF#1z6VHR49zgc5Z>iXc$xRBed|GrVA(o;m_$M4YkZ7?sWsd%tJiYi6DoB2MarbX^_`%Kl-p zn@f|nj3Sl#Cku!~L7BNnSoce7yIVYuziH?t5fLhnCpO98b1}iC+(3EDCJ~f7<&`}m zVIx{V-p%r`K?A2zMywN{=+-``ZOL%b74e-Vt2Ci&#wx^u}tkZk4Pu-xMaGP@h@%{$;(LErww-hw@j zh0{dh$%*Erfm1Eqfkva%TvbQ1efrOKo<>u@N-Zq89?)h>ETk0>Bdj?+v4*?l2?KTs zBBJDXA#AN92()iMPELQsnBXE{TuFL3@c`IQrn5B)i-~-MRCG4R$hS}Z+MQLgQd$gR zCERa>7~StJoqKV7QP+pe7(&uu+Rx9I#rSk1>bc(iuZKvGMz8m1aM#x=c5)}{CVJmU zQV)^mw4VFC_z=N->>*9LqP)nT;SDr&TfqKaBIqb)_PqNVoU ztM*J#U0SvGZfmcU*eg`kh)u){wRaE^MCAS5&;7j5c;CBspMUcCeaJY!b6vl4u5*p= zwe7|JglYLI>4m>&2`wpFp;Eo~Mbk*JzW$F$w&iO6CC1VXId@9Ni1}5~*N1bVR9c(U zbZbvnq(R$?Xba!!V3Iuepz}lqgJ{&5UvWKOe7nkNw5a3PSy-L#Df#uwZV`66G>%QfJF7=XrWRT^smpvujPa;Ln zs}^72XF-2kwP6zak!O&u4lN(oI%4RIpm=dq648{y8>-9!R*d#VbxWpYf(aA)nX>*1 zeKLWasH1uM)?LF?7+H{DNBeWYT{4K-JhwI|gE6Gv9QG^2hGj>{PHdg8lb8hu3Embp zhM_Dou7&*~5;6-{C8Kcj@+y%=IJ=)Cqj0Zwws^$pW7Mi~{n~Q0&LC4*NJ>eh+q*ud z3o#n<9HNi-^$R!Dbfdhhi{_us$bI_wQK7QsqIKc#)REWjVf zL(I9J^VVkIU7>H9fc~8`15ZQVO{K6WYO}W-a+2lepMDW=I}v>FX!hHo=fx}rjFHV~ zw-=CIZdb1x%f}m^nzUM#ES4!B=DUjYdwnFMrKqn!hu#ZP=|a`Dz}PNO%ldC0O=vwr zB1*7`^C;&>B6jd!CwHMKU%Ir$%Dt7_`~(Ga5iLhv-(p<6bYuHAP{s8cykA9M%>Dp~ z-c3pjhr^Sz7O9Mz0>pvLgO_=@G%l4Uzoy&9T*Hs7qn>ElVziy1sN?hwaf%lKfn+lD zEt?Ts(b?Z970s7rvzR4W0(pL$FiU<>MTkO0gJuVn$9Fhn85&A{F;7$Gk5cI~RGU!V z7q+-#jdLHIo-tW^ZB5mQ?}-ix@R$`_5ZO;tym(SLWMkIEo@oN|Rqz(+5RjM0 zf*e|O{|BVb%@-*Oii$pG=;-QGza2nz@CXOub?`Fd92jYPob+BAnqSrOiJ6oT-POEFp zdcEo{`{fSyuS96hQ#37DCja1v4;s%^71FX@x^(LkzvhF1kKyIZnAegozC8MvYk*kN zX|&f?hM4;i*kyyBU+Fcw1+3n=+i5IQY8^n)YQhK=sK{ z*1O^i@^AM$9x03RZX;h>2mUf6>tC3LPEH`Dlr;~X;y_E-aFC`s%?4D3pIOUzZx46p ze>zl_N!AE~{gmaKJh;7Z=u~>lLrd;5e45tYAf2rF76b3eT>EqEV|DeOIf1#o-)G~_ z{@`S|bSdXkN?VVBoc^nzN?-*D@4a#}CMISrH?@Gk^wB;gIy(FM2UsM^F~CFB$xNlr z5jGx?9LX?o&+N<_o6$=nF}Y)Jp8u?54GY07hVsg7D?a!fKs`@!S@p`Bcg-SwBKr4# z>DT(_BNFX@hVwg2B_!aGx-iGDt2)R@HR=v#hNg~~6UKQDMyjCV9w$R0Q|BDtn# zy10;|$hanDYGCUP8zjG1ob^p$IZ7xt;^c0f$J4}L26O%c=3^{G5x#aIz2HWk$|#dC zl&Ou?!q^@j>P6OkL6aNxI}Fo52fb~^OSUPw4W|oT=lvfZvOfeI%KBP?b3S(6ly#7_%bDcFVTw)6cKi}cT-g!ocKLr&q8 zxca`W@Z!$TEu?;JpNNW zp(UCvuR{9PKFl8f{;B%EQ>sp5FHMGh@b?n__kTl6w1D5x{9(rOze+h;A(4k{c}C&W zxfhA@;pwvf>*qqqjDt`0n(hAkw|^(2&RxopQ}+lu8N){h*)~tdLbsjAbDqcCSUT4y zvf@2ObM9Q+ zoVemcIjE!n^htALdN@ud9t{YjY}d3vx*pVsLdo4Z&$ zjTjFbHNuE>(hoMb@v>qqZ(=Fs1#E{~^Y*PHX*^&V=ZLvY4K#9gA91CA1{Fu`^QTv1 zMiKt+E=~H*Alu%-Z+-WB8+k1u9CRH2IkGA)(L1|$_~9D^4PtWWitCIwVB65%ns2O)O3N?0CS_%p)RZa;{W z98eJD1BK(t(%40-S8Q>5rqy#oZx5V4Qrii}&mYX?20w4t$GIxWylJM`shOOnm$}#a za~F1@%_i>b!uFd-)1QP$SwWl!O*4&}1N`YP8%Ni%;_NXAi^nDd|>dp(JrS)vx+ zlPP27s8@VDlswdxP?!070OC@w17C_BeCcY)yLkH~l0|Mbh^7&zo%7*0`Rm^Hdb)nna-Z4t;?jI`R4nEg1lzkh|FtU)HV4zaZ{ zaX&KS9v}Tml7=DWtSt*Y+&q+mj{IOe$utSJyrGp~y>;2s=eK+Mss`sR0K3@W6F*T? z?cp{3l#PvK2s4GG%J#LPLNdc&VyXFFIe>8aG^+HQ) zo9o*LQjz&J$Rcfx&tI&r%?TZ4iQAQV4HV>d5If&Sbe05{nsNy>?pGflPd}g6zQ)MP z_3_!v1IXgOiyPMA48td*8fTtAUGGO?Fb#G^6jqx)2NbsQ-0_;3nLuR;C+* z=YPgck2knAS~ddCWS>=Bw&}*(9{FyQO|+@6X?Y}&kmH{oCW$dZHBWECkLI3;GQEl^ zo2;z5@U^KFbvEInG)*9FpF@?XU0-MRvT5w*T4dA{D&bf6OiRra2sNeRP$f@0mA-3I z3n?d2hywgfA`R_kqd!u|0URiB-_N8Li#-mchimJamiQUF8_1^Uo7! zS*k)|EVnfZ4Zg_Ay1wE>{Ly>&I4lMzGD=5AcQf5ll#X{hR~kc<~XK`smF*Cbk{zDYYeT=D5a*{%&SN*fC^s!9KaSkE zK+!*$iO0nlel+YFb&;{&kH_46+NQ;H`?&%-0Oy~w-DX{ASQEZFFF_klX2IOC!_Z=c ziHg^Gw$_<#0WEra0H7cr+i*SZOFhnT9%vjyu1m&sWJ2_|R*@#`!+zLI=Ycq)^(Yjc zCKNgyxZma3FS}T9_P%H9MivlC>{ki9;%guLFt>2Y4f1)WZTJ0;PiFaM6M_3%K!InH zCt9`+z!#6ueCper@P#XP$KSEAT>OI7;r^h8X{dM1Ts)t!9JSkd_S_J@rQqSj?l9<96y%RpsVYjyi7Ct9*<{PrE~YaCB{h=v zo$}<_ulVhSqF-Jmdb>86NI2elJIFFbzC*P5xQ6mZP<6EUE(E%<>?f>?C2IW$;>I-Y ze;$DvoqpRsj0?BwmmfAe^oj34Dm4|o3CffrNQ%<`mi?j-z)+x5_qjw@?)x|pm=^Lg z7_Cqe{c`3u$ur>@fcv1^gjTtXS8Jpe?325+jDL;#LtkZHY-h_#t@tA~A9wYr&VPJ3 zzRVHK*{QtY>{bY4zIGwm>l^nE-e{!RPS{(pwg&vf&OwYNQ7d?&gw_*RaO~$-8vnA{{_`RV*d|_)pr}e z${XJ)mOejJL7C#y${8>xmo<7$28c~I3%pqZZ|7s5a*mhVI+6KqM#~cWUwd9N?ISr- zcnFx?jg530uYc&;w0Ol}XW21o9d^{eg7>gN_$`noG3SC!WbVIo*^U#5EXbdz>jJ4&s*8!^}kXCXS0* zDBd;s7g}0}v-2W%gIjNG6um3SS`DbmO@5P{|I0*-KV38}Rh-+3aOGU9hRIwb%v)~$ zZm{8?lL-B?$lRSDU2*-gLNkz^aA~(DJ|o|a=x&yBTn5TaIF%vN?$v$XmUA%%1o@yW zuhjv)QRs8?^xDr~Hqe9f&BXiSq5gbtK*6h5&rVK_8XJl=J);Pt(1@StZ$5s6hG0At z$1l->r$Xh$X2+Qp!+cHkFZk8U;(DHhN7cLydvib-6%Z+NfQ$+}oBm^~WabKEq9oeN@&9-PvqdgAZ-=fegT01v; zFrLfmpjh9K!>Y=6|0efM%A6c3a%~+3hSlh;gJBh4mJk(R$Ymu z=^0{>t=&>P7z4KRwcXF~?Q^k!y%bFm(ALb5T3*j9LOR$ZaQo}yCOSM?u0LM%nV;V9 zxpLA$`N4M!y(wFThi^`BNmW?KT5$C~W5LlO>@--$KTzwW#-3K~Nx1Y=!{5A#U(u=0 z&9ti1dUh-B8BaFe8*}oRyO=e#N5>@~NED*+v-z_mQr=M;|+}T*B97Qujhl3A($>y*B!CK?k)pcgm9e zwKeoAgFBX8UMC$Wo*IH2p0oFD zU=wE99)VBWD?9UA?XgoRjO*M(!!-lO(|BU_`1jb0b^Va74i-w+;li}@V)th)cvLHu z(lmGbzbM=K^_N`)xn6S^D$1aIWjC-u$WLs1%TjxQ2^h8XzMkTb-r)k(O zi%#0{hiki|w=*ZIrVj-;vD?wmx$ZfnO@$r8k)nZia}Jd3(RW5eI#5E_X{v4&>z!6T zaaf;z2ddF=dQ`7oQ6!LbH#i->Z3gxbsO33jPS_*$+JQ`U9e-YtF+EdsFg<^2(mz|@-}s{3$iAaz1!(mBA`&Hrs?1p zGuHWUtE@B$I0CbT^VNp#=>~K!W;(D{z;nv?RR;}c2pPp+y?ioNC@#vAu=&XKC`g(n zRo;XpMPfa;nL60p-$V-)%%)*=+eW=dEo_)iG^L_DuiI`sw%uieTbKazY2!(wNj%8- zj@BDq`1@KjPVagg%xGS31ncYyn(%tYGDUR=T;0urG}RfjG9cX z-0p9L?Cmrn6vgl3@oJYP4TeFFsdUd8oo>`j;>r@>s#cJPI_qT-l_lp}m%3v#&L))D ziyC@E>MC-Z1h1gu-;Eya80w8XtPbbJTy`3r6!8g`%&i9rlY{edD^qCsAgG^q;U2`n z0oe>IzO+5$FPfv9w&Za;JZJbdF`EKJ>b$QT%Qa*x9DC``qWt)z)y(LyB*t&)<9&pYU2l5ptm1o8-LLG zXv~uTrHcJS?Swe9Q@+4XYE&jFo%f&eS=!kv&eWq8z2!_u&#H~3JUjJGCe*I#vQ>eT zpbH1?Q0qo39`tCl8vQB`$;9(>i$hlgFLT)bR2{50oZNqSF3I}vC3D`;O8#yKz?4Pb z1%FfV-=5_#aHIG>0U56|N=>ck2TWtwL$D)v{|_Fl^uBlo38G#opH=10WS^v3S| zcw&mQRbr*fxZ)eS4FQhP5{&$M9fuP`EqEE4V+P%-mUf4YZYo*P-A9V;uNR&oKaJ13 zuh*N2jO{*Wr;(WVk@a0<|GNHg$;<;PEMnx$mt50LM;{T|6~`wd&^&Ar(#1L~uRem> z5s_BWE}!N)$sG4wjc5I8Td&4PPf}{)`%zqw3|%h^#9*^Hljw}9hYK+ERkS`OO__Ru zE~d4VABY}2?o|Rkrj%%_o`$ayTo0H%pFNaMer$|r8)X`gjtF58?St=d8 z;S9>-gO_NW`UbchD)|oFEj!$85eF9@8Y67MqtH)pKW&fhyXN~le2 zDtZtWDXCt#MX%0wKykP`F<#vWW&UyiWtH5x*H);*<4cvsl~w_7>*btIx?|>D?=cNz zZm2^}ypw8Xp?xFSd5Mmr72n#_>t4`KKqkI6_f=5O>~FcgOj_T>8r5tigAPd4tTl|i z^jYhxdm?S8(>#XpiU+kzMND{K7s0W^PmJzMC|5h9kA}aCH61$Fr|YF7ai%tE4s)IlsrOrU($6bl+RXENOW0)X@8+;u}mF-l%rZ{O+z(GB$C{L!#-bXK(MH zoGugbgtOgKJgEU1gOFUb8hXOP@W;zOg1P!6j14x?#YdEdiJ0efTI!hK<5p8&%SFYbyOMLQRixq2bVb(-+6bV&j(?YSebLbm ziktM^dTgz$d9;uM)A`vp;ent$$jBpKZ34M{1v{7oW|dk^>g}%ba;O#c)Gda+U$lNQ z1MRKeniPrX&y*8N=uP07$g(}vpLtiW(#t!sv3+5-PcW7%_za^$^O}nsZN&P$ZseK- z0M@7P9J??xxiK$In^q?wXHkh_CGct(cF&T7k7eaiXPaf8)bN2K1xl&8)kfVH2@=2(4+R^?Jf0}YsW+zFVZ#xw!A3b7#RjIq1o8yb` zi`|)iS>_5l+CH`Fc;ZvBbms<3e3m5mWsX{g&PlVo+P9S5p0GDjF&w<=kUYfd!v>qK z-DlOW!V--KMB?;Q9Lcg$B~5z2?ql#UEmCX6K%?1BQ7hFM5jg7w4S!57LSHTB+uu6Wq*~l1(6jSk)wvC_LLZw9x3xVKlPa@H zyE0U(By&i_%Rzis5a}&jas7AJAjZ0uDSln}Hs!-~+z1xP?n(dwv{W9gjGYl#IFEhf zEseDikB#r{pi}v&3rVe%PHt(t{k+pf!AD3tl{9uCSL2<|#F~{jgnnk9_4&xt#VJkR z{|MBn)jbv`cwY-ni_j7bmG${{$?ZNmgfgT`3*S;2xjdO;Bf~7`m3>Kc(OjIh&br-^ z{7zw`_vT&J+EIx7Nly5R`%`{1Jx6`Qc1p)jN}vx3P;#04bSFfDRzk+=$Z_^4PFg&7 zj*xpyn_X;Bb|d#xkTLl*lZhtj`Il5qbn~gx#`=!;*4H^ady+*q@AVkwaaz2jlrSM} z4_fUeF+))`SEkP&Wqh~Rb21;{6DmQx77d+NXTCc&iwzAk&3ywd%i`00ROdgRSKL!6 zYKG?uV{SMo0K8v;`1CFzFK7pN&^l>GIOWHd242)~#QyW*N3c^px^%K;q4A8*>FPu8 zAP9~zne3|>_}*~VB6P8JI8T)eh{Of1;agVQ;DIJ)YElQdiKFT=>JZTs&WilQC+aCT zULR(RlSbCpXE6Euh43uCSnfE`J_d_D)luUjO8eL65C#R#y|c1fU!PgsJax1e-NDo4 z(mH3e6m)c7%5%R&O7bMqx;we=sSJ1~6SHOhSkSE2PXwsnfoW~A7V1@1fTaVsg796< z@^n2F%|0!PW>jZ9#Hu8rq(^fma zaq5~=(oh<_7RzO1tD452d+lzQlZ-FP;!WWcU-4)OR$S&DX9up4a4EXkUIFfbS_ z_xj>M>%yX?MplHcCG#lk&2LA|$wh=Rf8IW&FS!wgbS(!ZA?o+TFjrHEBkK{O__cNM z&pBl{rmCc@RIYAR$}(>D5I^9cRsQ`@URj17?99FI09vMGC0}{Ze9!CeCb1@E_B}Nkb{?&1uf^+dBzS*k;;3fAY5F!e8si ze@$9J$nqCufS&$uN&M=HhMXePqU{YW`Pamraow#V+@N;&&qe zR?gaUvLf8sZLiKSPSJ-Em47v3|DM!s&Qr-|6u%eocN)I{zhq(IUoP7J{wdDnbAB}B zv*srkD+s@v;E)im*~5b!t7Q@VZ{JEr(fYh)xYE3lup#|>RSQXKVZ44SdFFpmUqu>W z>KE_)n5AfbI}Zv}1*9I#??uLNgA&l28>WxwelN8r z0~ohDwJ>+~!uHRk*ctbUFf=qSQTBi+592saj-oo zX=}%&NF#*Dfc&iXJH}(Se|}TToRZ2hZ=Tv+8RRmkaVXxHBEFS9TIR5edE(9??)bDT zRVs`)4Hy^B1@~;}cEkOq;K8Kuleyu4ZAE{n7C}s%(Z#hGQPZIjLu8RkzC`cMfXV_b zm!CWHycL$c)DDAXA6Dj3$m&vqTRZdZ2>}|D-FbqOW++#`$ueq4{_#QBLSGDW(rdY^ z;@NtRO@okGaT-hrI@cV4q|Y&n(ySn&rgdNtn0~ox^-d~a#(WgaqzpF07Pl<@?d~ZV zdY>4eg4tqo-p21w2Y$YfxG6dKlV$tuJb!BB`Z^j6%%T;X)BxhflLjGQ@smKfV^y*VUx5jbAC1ukTT)=-7bb`?+QF>Ks1p_C!lrI1!n7M8%t zjcv*Ryn(+_nODN0Z*Y#Pi@Uj-+JZ+8Rh3%tFb#tZLZEkqI1F-RhuS|HVb3nWhS`(mZ~QVa()Su zXFr?drI~YYzdAo_jteM<{moLvU`Ykkck`G`RDJEJ!M9vD3xi904llXrj8K{ZkFI4{ zOC7rR#8CZ?N&WE;%HYe{Ug0-JKcPZ6GS8l@Ma=kYC0cypovcE8R74!X{ZEu6W&;?FbyL_$z;497A zg!aBa*ARwSD$Oi;!F3cg38)@MKtMD-<>-2EHQT;)J`v0B(aCx722aF@C!Zf;6vHoa z4@@_8)`z0U&Xq72?_;UxVxIknvPA>E4x7AQ^JE3xKEy7?1FTNf@Llo6!FgqfD|WPW zGWI&yP&GxiFh?uxv2BmRtcwBLxf=D=xm~5nrJ=8`8sS#mTv>9#gZrDepXWS*95(uM zb;WRhd3BCS^CCR1(;1Xx&uIv=5A<2T^?G+PHd*AeTKbrqYF@-Dci?DGh5uxK^*KQs z;~KqZTe& zG1Y=BY7L1RH?_9XJv!py^4>=gf}rUlgHuvIyOjWB2bVwU$+O~9dDA-y3QuUTHJO<} zpE}L0RFUJcvf^&gVDaI=?kj25LZ4~yi?U?L;Dc1Kp%x2s|Aqlf<@>t}3j;A$Wj1^F zfqpHE2)fe6z9s_Te^;4I%zB)(FUVaY<;&K#_cN)6P+gfdg;|Oo%@VIW~ zKs8un^I+@4L|)}EG76|dHh?T1$_JWr);tBevoYU$kuCRgefa_O#UlW`V-1k6J=R7F z^=qF=0D1%{u)@j}05=u`^dU4ctZ@|t+V|03`T>9wZn{iV-nAd#beO1g$R5SHSTv~! zx=z)`iaY5AYA3rP4#YQ=MsdacNg{H!8_S4$K1YiG+WKisD40Di2d|!PbX|2;O%y9X zu5@e)8qLFs*5uER+4dy^=WSAKn`@{Ft)u4EoebJHM;q2VyW&GdV0{^X1&pgs)eu3R zDF(<}K;XH8| zicjJ&!Nf>!%pA8@_hA9SPEH6O0EKl|u}o}%3ZCduvk0&Sj@!w+SSjF>oK|3$53;v& za9Cx}043qld{ts7>P(=m6CaHh0k6~H>ZY?=*aB-}f84E@8%I;|#$wE<*NCUyE(inj zgIW2{R;&bPM>&Th+0{LN-SLxk*Fpn$O23jHI|Y^3ZD*~fi!c8;x=>1n3K~^+hMqdT z;nM#8GC)tI2Pm2Gy<>$>KE14aziInICWoO89>=Sd2rLAQWb`r6JE#8;Xk!XRYHM+; znM(i($k%HeWkZ(C=#?6(w!JasrKNz03RSoQ0LPa%LpD8w!35{v7~(Lw3X57AcFM*c`YbdkEd&K zc7-z~84E%?p?hp%-}ICpn@75v&#5ljs*m%+xUQN}(m;Sld zNJmwqM1;Y>r6#?&7%KBY07VJBHI>j zT~Of6IdfAju_s|MP0T@u?`c7_-1iGyk5mZzn*bVvzrud-M))N@9Tk~9b8lJ0EuG1A z`9t=SfiZ(WOp8#aiw85L6FU@UDEC7tONAqnpU->`s0V8o5F1WJ`#iu@H33ZUmC4;SdU!hNtSml=zWSLvq^%UAYQU<6KM_) zYo;YcYrSDw)*}a99)v!&I`C2Cl<4C7stozSzC>dmiL%8cs0m?MzR72^zQ(`rZu7L$ z?$7S3v>esq$g**(A6g9w(aBmMvv)M3g%{m-B?4>O2T77dGHllkdFjiB=Qzbt?#B9?s&Bfv@n z@;)kwFL<~YEtSKlKm9NJ~OVns5Z-qts~B4}bES|;A0Z$!HO<&SNBzx-sZ z(xyT1C(#|(UFIN5BS&F@L2j@evOZU=9PVDzdYfO{rTo8>hsYF0{V{u2^x14 z^F9C&`FzFt(8RA6ZI1y^EqCDl#zh|Ws|`Isb+lp!)CpjaI;FyPfl7Qqp!PG4g6izp z#YkzQ0a0rG$OoExvBq_ppHPbQqx=)S*t%nm#y0(5_|9D84LqJ0+{~zVtZIab;XockWW^ti5O5or8XJ^%LJo`3T4{Ixqk0*zpCZ)LN{UGl#z(A-@Y0~NyW^_#@ zk)KT}x?$1d-|y#cHR&0$)oNy!L7P_H9rJ~kTVriOhtti~0OtBdRGRIjpfl<;J%>v94bvGyey{25*%N7*FU$xQMWXk^mX0zP^NJ*FihG2SxC^Hg z%r^erg&QhBczUC>wCjzW@6XsudlWLh4!!RDV}{T!Fxt83gtBf=rPUIYv^g?f9V&aR z7Z1u0h_Z{oXl7_ z*j0wGH+~Kcyj^-aU_Z$t_kEgm(bc}2TMxrS!b4w)^O0;XIsW0k0sI|eG$rKbaw+SS zA-Gq0nFk|r(E$&1jp%hHjsW!eWx?yEDc=His{z`^XKqGs13bX-lua{e^eRL#eGJ)K zZf+)6(GDaE=xd4Q}kIrcuekjwE* z3yDG+SI)c;EM0}151`P`Uk{%HOkf1SEO4GNyrYx9s%nkP{nKT)eV0-*L)r$!0d$IU z1^E`~mGT4JolC@sz;!{)yo$pU9U)SI`WZIf5wie# zV6tCgI#Lu^8Zk^q8P^Rf?TNZ{1{~`vpGa+}m)k&gD6q01SXS__Tf+=N9}O$jy7}iNn@CN29K36-XB0j~B%&WNS|p?&fH? zFb$VOz)q>5L4++ivySgL7ypbrpGdPuaVEy~irRceMaenpiQjD*`?KWtQ|8aefJxP! zH7govC-8V~PwH@1f?znCKw@2!&D|vNA4>_=)=WqEj=_1}=`8A%a%v2YRG1S!Q!WYD z>vrA0Z*-q=Wwe?Q1>Q&3a0Kpih*@1&-f_7y{Gguh%xVJJWA({`g zL6BhV0#nhpthqbEStXpm5{?b9{njG|+WU%?nk1m%O0Ie9%g-I~z#0@&QVV+=bykhb zq%C1czKWyw1R1fkGv6_3GDOyxd44pn!)I?r`z2g2jN>0COCdD{Sa21x=owceYRd|8Rd zNY2{8;eH!n?d_P((_ob;26mu*QxF1o75Q-4;msa4sM>dZw3j-#Y6f8KPhfZqr4(T? zNK<%JZ4JP2ehlDWHhr`*WgP_SRw@F>(*pj}he9JrA)N=UesI`$S7tgY5K*1QZIgJ6 zb{#*LSpLD@2IQjosK7QT^4ioSD$TadQ7wxnp(&UdlpZ_OjN3dfIN$Sui8m2kJD-%% zmy^HMZ&rISQW>H-<~tjGo<-X5v0lM-Dz)HScIwQIE=>(pH3kjjtND*}@ivR!m@5Ji z9x1Yp(dRd&d#z6-l6bO40*X>|hRw|5gmiNsTNDd8Ua2*MX4sM(J%7CKOXX_Qj-bf_B@S|!*tkqIPO0^I;lShMPaN`=AuQp+qxzlI# z=RY_7B<}21?NaqmZQ}N}B@%)bB?-J@VMroO6=o(r>^=Bd$%olQPxKW|R5$D+3ny-T zPZW75tC=cR=(RG?dM8Q3#gH}L`)J^{antr%l3Al!AqEBlvyb%;0rt@Z=0d=ZArPl@ zNeaN#ZPl>D`{-5HC-b4}gqJ5m1qOXbP0Fy>;(-Qf0YN+MOcGA|Lj1!ResY%pi)Vnx1PE*8{k`Z>C;LL2&oP<^NpnqWQm) zwzXw^Tq2!QD7@a;X!x7CNJo}dBlH_jglf&Z>FL?z1~swk9ib0Rtu~LuNzXJ;=H;1C zFu=?77`p~{6MQ3-(%&qIFL7Ab*v1q2y%+JN0-JScF&>(&2&oNJeXjccSL&du)q5?{ z)#Zs%%^o`;>Bqf|6VEZYKS42^WzxF&)>Zk`SIupdOQQXuii^C~RT%^IErjDEn1r2Ky1#UF zX5er+wzBSYIIfQI{Ah%wizbL%WM~m*?YVS21^20V)qPdfhb=7S&rn%Q@s)uO#a_lXB+&6n!JcVAAV(}vxnp8DkJrCy zHQZFreGTIIg9hE;3pIHp`pUbRaSs^|%y2ocueg(vk@*o!pq4lgVIpX^VuXn0{*bW| zZ!^kUFXTQa3&tf<8&yg1KHp@-as}+I^gRs37%u=-UJa5>pZl7q_LDDiKVE3AUoUqB zea+P{7Qp`LtqEmfc+m~&PZnvTQ_FrPw^wc6N-NfwH5+qx9eUYJlh}q;!le3 zF1q!H@K3;yzBRF|O@ENu+aS<)QcB7dHcqAG?RZ%!gVL|57 z@m`kiqaHn}_e=FcUHb56;eWIOvL}EKwm%g+2^ws|9HNcWY$gAn4FaKG76U(Nwon7i z2MN8;k3Oy~e_|3ceDePC?qeEeNryKkTO+HY5@BL`Jy&DJR4vU=YYsHIKIt1npb)(P2i!1YU_p@id+mk{;sy3%30{z-19r==U~ zELBC%%AqD%P?^a?)|!p4c)*sAeQqmjnWYgWu2E!XrZVd(mw7svTq~<;G>mlJof0su zC#WK?Nm;`DG9s?!E6xE()jt^I&1e=ed-L%>RJ)!-fezD#KZ&yqsHj-`K_YbQlD{wj+GV0h)KQmz#OkA8?khD$Jn|%(DsRTci z7M>N%JmA)Q^^9-fLSs;jB)%^O?a2Ltdlj%hR>~h&#Kc+Rz9~JO3uq zuyvjzB4rwX&UUkh=|IF{&6z+l1U!N^BlJXZ&p~7wb=_pO^G3kdOc+SRe<_Y+`@rqF zgtA9d8nLil)UL;Q3bYAhevM>3Ui zyRl=CX=2HfFsHz9yT{UQlRIf9uB6bg>f2R`*`~r+WjCJ-TuSj&qCc9RgiN@;;%L=s zF;-#I(~4lAq=P`&0)Y@m^EvyjZvfVMB1Nxde@z zVZF4e6v=n1TI)}9k*4uo2z9V{wv6QgO`-APYH?0SB%45~QBAegzGGEXi2-clC;J|W zlNj9DB-M7KRx41KeFh#p6G@&fZ1EuQe84qn35*tqXx+g(S{?~R5Lj2H*(98kal_85 z08y=OR6h&oKRxbjj`~g}`=S|P|19po>r8q79E0}0yW}g8O8VNyJQ#Pc{=0jwHV)v+|f5Nje@{N3G{&XD@*AJ{X=>3_$D*%JUd&c-%!+Z;9fwx#V+q* zJd~%pIsxRPX?D)p2y8Cxy$r1YO8F8Ku-rS7XG*7Itaxv#j|s=#HMmF!tbkzY_zZ=$ z%zdJ4!hq;44RcE?QOTNgL7{~^dmNNnSZqZGM1!8ClYNck%P+O-&(um_7>Ub~-pZgh z$jEEellvqMG;0YY%!biB`+O7yeFa=kt4)zoCAaaR8m9&|`Rwf=4L+NVHstGF zW`cwQLBOrmXDQ@Gy3w&P^CshB9Fvggo}0+hKM&6jC=AgCpofSAp=Bpf-}6e{JoR!c zKsay%MZpi3Sltj=@?gVe!j_ENFgl0VuuDAJrQ|bq>?2(@LZ=|1Egjsw$E|Z^VpR3$GTrnA6^XK?Ehb}LDHGqM2CdsfKHJFe@xTQ={j z9Y90+b&DgDNY=;QSNd>r5l>qRfgjy`;!V64m5-Jv5CQ{{hsBfJw)57(5Jip3$AD&U z6y+04vZSEG(q9z|?)!N$6e3O<$7gqKXo*`a1}c6!^#=CPb_Rzu)!Dh>7fsg1HXm7ecSUHDH1MIWw+zf=6)1^Hd$3T)%6*y#ukEwpU5c7ob@kR0rdgKoC{ zd`7r4y+I9yaPL#Hm-!6xjbh#yrdy*4lZQ7#&IVeXpX4+vH(X3$RR-d=wjvGYfQ7hKkf zM8#b;h?d_pY9QTcS7I(iKYX+UB%}9^fQk|upov-|_5lNTuLb?n8^8+aJ@rh|*$E1Yev{!ad zy1G0wA{%W28ZG59RB_=+DfL>rO>_o6q;p*TsR;h(K13&EwYBiSJp<>NH2^*^4Aw+< zch)2L_Z!X!Zw&U}cR>?q<|5G@iUnJsK7U&YG~41(N&=Oa?*7>vh_sUbQfVnG?TgT2;iAmPIR*$L@)%rNAHN5e^2ywsaLf0WVI)!K%#8b(6uszR|ScoPc0qm8sQ z0_kMw6Zxw}2B9-vsay_7ktF+;r&@-|0=(}*$&reXZVj(k5XvTE=MmV67<;DSc1yp~ z%1_9{v{cw{rk5%3-5}-GK$^aw0<&#jmB2;YRa)0QsO@3Pc zywh_(sBVgy#nAKiD==aqxf1NKtn57rrm>lcavDSF22$2L7Ae>B8s!?$5v3f~FBtZQ z9Zu8rh*|?YaF8G>6*Bo-B;HSLiKLH_a}_!*9#>?mGwU{#`{>=67|MxQ7gZS?<#kO0 zc@=F|j7m2k%6$CU_ZsL9Ir%uil}KoAxAsw;2t$6aw&8`?ph{mP;b?Mprl4m{Rw0Ac z??fR$zHuX{zi9ClB3f$mboQ>%WHA2~+x56EYi-PG#LXwU`t(XKWqPv~QKyJL#C|nh zb)LWWQb0jYeHRTv#A!a8a6s$=E@%1&h)s;{E1|R1WVOw$4Ob~RwkMKsfr=ccpVFjb zloNVeU6~G3&J_7>OPVmM7JPKw@G`9t+_{MyU z78miSkqEs{@Lpj0I@vnGeZR4?|13xc)rx-B>j{P*g2rz; zCZe=6`OP>o-nV+)5@@$LRV=o;8@@_nqT7%oY(!%biAsMN9QwsCKPEU!d|3fIjF2bV z1NaIoTrJ~PVV3dy!S7mIr(CB6(hI^rJ>!q3VSD%@qXZac`vN>ov#HoS)jFf>V(vV` zL%(VF@A7Gf?;q@zJW2(?fe@76Se_4PcV3(;Xn%o(p$zC`!XIOI6tTK)rd5d7diwkq zjW%at3J@SM?(x_qA*-ZfxAgs;8paKKZ?Hp;e(LB49b435Lwv^J31Zm67k?>G3Rp0ZABL$|=e(i#)Gpe=~ zKz0ZopcpiEj<5Ayk@zpMAumWR(>>jWUJhFNhYo31R$9-rb>UzGw=zh zLIU_ew7bV#uWtW0(7H;zQ(RC;P|%<+d5->n%`*UX!AAEG_B^2tYb_A?mv{M(^BxGM z5`BEMr)-h&KNb);o^senK4`uZbpLw{oi@N%f!cx1EOW*mfSLccrLj@1AB+NJ=Etr% zr~l>n{&{dm1CO#%>Ao8B@ss@T-!}Lh5UPxPZPqlw@K?h=LPkDb?TrJN)v`?=SK7{S z1>Bf-mhx+V#!!qdxMcsvmKh+^Kb(E@=3f@_-*zMfDKN%HBS~mMqd(d;(rok72PY5k zyEKtosc>Sh(xuj);gDni&mntaDih@5fc|`lfhC>m*@W=UGhi=;0^6P*3BkYo*UP-D zznsv*Rk)RMus$^hS8bFb=~pWcIt8#Crt&#DGtAjLzAKb|8Wm2SR&z}T*cYA`saNF+8@n>@uuFf9{757&s6c4-Q{Q9~glD;F7hFe35YY!%I2-}cPH>3++y8hV|F)$9$LL}iP58Y#0kr9h zuMcyqksA%qh>cRto~=<)Br?&((MZLIV%^6jleG>WLN5k3m^2R%6=j6!&!9t)YnJyr zCcoK3Haa~Qax%D_9Y^0B6ZY#o9M@(kkBk$}1bol$_bV({ke$vEkHGwbfI;F>>r|V( zvikEe`}fPGN@;n*tjfr4akd({4lkW?|IL=;BCn%7(q%5=qjovsnKC+n;~VQLXCc?t zR4vIFoK-Z~^u#}F=kS79P#7rqzbVz#cPrPWet2^l`!F>6*LZ&465$#Q-0e%}Efd+eLV0F-hPpxmrnfeXau zo=R%-S5ZKS1WvPq4P)m%unv*ajH7s`kO;UrvKvneO&4|_sTL^TAyo{%1$NdoieQ%5 zPeUl{ceB430x(zTDXelU2v6E$0hq^+?TOra()oI^iDHE{yA}tfbnbE*;O38$A-t94 zm3+6fnPFntj<#mgQ>X5D}1m>E^d zzYo6@_S*$H2ItH%q62o{2HS)BLqC!q#-T0lzgfIL*~25MkITy+tR^=|cIFDRTr8)5 za$R)h=Brj1aDtuh$(ePVHH5F4S`uICbV7haEC5T$N_;V%KR1miy%Tgkd}}q{R9R<# z9$ui7Ae&xy_|lnxIjJ-WlCm1oJ|Mj3=QHUZ>z`pk4r~{<2Uy3iT^Djv&*$Ak$bR1| z*>oQdyC2CUQTNmmcvoIcmuaRfG&1^|wrA>Wk#lt6*#J!tO~UjR~`$aM*U`of}FHP`^C(k4(c zlAs@Gs=hvy#u*O`i{$hcfRd~Oy>U4F5FXyp618F@->Ch6@l4x1kpV<4Ub~P(tNTT^ z-kZkG1|UrF)ZGB^{Z5;fAd-{S-u7uVgSC;`9c^UYFj_D zS6ryU5Y89@oor$N>wpt5wcU@dFJNq2qd6%+gBod|yX-p9kN1>(c)?| z*XCJ^&#d+Olg&quf3rW>qyXj+4X-z;mx9zS)%W*(4_uQSn*3tRkall5fGCvh&BJQY z!i+;B5eo#7m-BsDt?mkZyRCqO$85O8i39k53}4eg#E3JdL8yg8i>+vFh;JpyVm7qd zS@UqQmBG=t{rRF>)~)1n5~`*0#d?1Nhv_eD%}24!+BJgDPb>6W8+>EX6do)Ct%?Tc z;DiSkNzz65=Y!U4HUa=hF=dcGfm)^eG7k#>H*)1C+Q9EX^)uc;G5QfO@XrC}$dBor z$Etj1znit5Ub%f2^tc>j#RX5TP|R-G$RMJ@9$^6aH`(C!$zTY&fr-=4Fmi?6=}Ia< z@V|aAhLlINnbrLkgW+lPh44*{OI=DUF@E ziUdo_*;|jyq|A%QTL9k7i3W79F;S>+kltpyp+f{;qt$+G6Q6b7w%`bTf^kT%kgnzB za%co{-U)Hj2kkJly4|nL-_LUiC%!xLsB&z_25!uMv4g0a$mF;hRo{wGTweQ=&H18W zdz2sxKNKGYD$*D8*}hL?rva9|OfbI_*mSXeZ&CajlhpWO%a;6Dqe7p3{N55NM-!9> zcNj&OfLWhaB#P7~h?~8@G8fVPNPz9ktO%}5y}aN79fk885Ng7zS^zn>Um_FtQuD zQ8?7&Gy4@_fIE%SNyw`E0qS6!dSHbl4Rug0lxMy5c1;1KD}Gf9`Tsgn5lKJx8`Y$^`JQ4+8o~U4|;leVMwkH-BgZho-B8X)L0F26InN;ad>h#h%s6S z9pZRAM=n;(!o@heo>>g;AVxU@%%xp5l7ja%jO^O)7|?(^aXiD?0OU5z_J5X4Sae>f zels#ayKvy%|NKSwprZ=g0{WYp9*LauC0rvjaEin-VH-Fr^WBx zTFi&6&a)uhpD}DWjkC7A;vg$DeH&~EbGr9y#5Fz}#F~a3LZ}WkxW9=OcEE{|lPDIj z3!kg(fSztoD6Xc-qFno_)$Tq~EtjTB`%n{=RYVlsLZMUm72xgLe!m1m-7=8Q(JXKh z_(-SgVJ)_oNB4Gp^|3Ke6hR~GydM*u0ulQgRLjPr6K8<&Aw+~Lv(!{d*N?$Pz5ZmB3*M_gJb z{n)GPi|wlmN|^V?dPNgjHYgaWGVP7RV;SIR=<_RfyVGjMO+3)!{1_?uddnR>=~lTr z1dF2SxgmUlZE~H9VqPr+i-fyvokfLew*8$F*)L~>BW$G_{~WpKB!In&k4`I%eaZHU zczf(5Sl|1UOWRFkKt;`sJ(UtYE!F&0>+^7|3S?iFWp* z%s2*SWhkQWYLrWh-CP`=w%Tr3GkzE-NoF;W*Qi?9mOsDnJ62+%)18r5t&~Sg2=q2! z)jNHK2U`FD-uMYD3vOY_ywgP{2>bghO~x|=l^Z>OQem&D)0C^7i;P9Sjnkdl$_*dk zF^&qIN>6B@c%`=Y@ERHDz-g(jrK=XGr{5pw7=WiQ#G&&T&MQODIfX4s6B&InhTotV zgo^@n8ngSm+YWoV;yr}-=7k?nl29#GXEaRl&JM3yZC`7;@~bvm80FaWk2LL!#VH!{ zRvdqYG;a@SGh^+TWdlfico_Svw^sFguQNp*4wTXXE(ClSx|0(xsEsRQ$7=654{zh) z-Q&LyYtDiy(4nxnRuPsYdQ^SSeEpu7I3og`Nu#P3mY`E=$QG9SrzFc!5h|@*f{KnW zJouChB(38sByp{DCG%>^-k6ot-?OVXEG~c6Xl^i!$n!HA6YPCCqm0^(!3V~dOU~;| ztS|!R53TY^!jbq@T0+vu+a(<4DekHTt_RlTyl92%Pq^r=b-VWb!@s_9{Lq$*)h!i9 z>6-3Ob=^vKJ&#v~JC+OUq~#g*M{bA(#mk1!UqUjGPB{qR#Q{|a0tBvLlm1xg_aG+ z#X$e+M%g-9!cq$Fv4GRjFBSAv@Xhc7+Unc|34_OeMz;BImXLoJm(BjC65QU3^r7wC zhqT%gw3SyKLSv32t^T)At<7eIj zZ_XV=8i_u?g*u;_j+WMw&^9zHE8J&up)OY2C}jwhJy5SO<}h1)t(NE87DXn~198b* zPF5h9jnb9$$i8@-2wS&zE{3BFudle(9N%}_isOL?$sb2D$2*Z9@)Dgwl&$)+D$+t;p^|HmU&=;k!pJ~aFn?U*j=i`r*U?a!hLkQf$AHz%Ya;o-RyRNb8bKtPt zmcDWuQ~vFX0on0c)AL(Gbw_T;yHzvYmiKZ$CxAWG^XScsY6j8+hA0$?zb^Q>j#S2ZRKRy%VzYd=E;#uxl|u z}7I(mfy)GKLqAk9ESf#gu2AWYUxjV$m_x*DB( z1TR%el5Mr?mHiAZ5^OZ>5og8nBoL*68B59)>eTzCe2D}CLyY*ee8A27T(N$gB*#%? zXd%#UoK+NmwS+hUpOpb59qPFbuj;ZKQhH8q|L~LeH}CoRocF&Ek;1d|gxz%}O#L*4 z?T8bO3yS`s4QU>YrO8?MuWE}TIVvoW&Y5+9s9$6}ks9c9e@JF^ZRRgy#}Po0wl?yg4Big4Ve|h zH?};LM>6_w!Ay?)W;&%+CQ8fk05)Dg)n1^p#*NFVC^*+(9bT@>p?kR0Y}WXK(@n6} zaF|y7T+nH!n<&l9Z_CNt@jwUMPCV zT&F1WYJI=oM77k3MZ4Xy$X1I(3c?MdF)Q;m_QP`iRYLQ`^lDMY3DHTuQ14$AIcj%v zCg>G4>z1QdFh~s~L+Q|qzr-_bwU^poH-e&LMnRmQ2DrHbj|oyJy9xjJ-%=@e00p2T z0|T4Odg_;=OZHwnw?&IDZYPD#stX`K=4Lfq-k)=!Zw1K4VSL=~AE_%*`!%xUJ=8zr z?F98n8^^1(;jxY5qKC7^^-~rAkcTnU-?WNkAIx+J0w(#8WKbTP?)|GO++IM-Cqb|o znpB>~Ij_4CbY;4>S-~wZTb2E0;o_hXFa4{fTdQJ^liB(_5L1NpN1v4?v1(75w^?;+ z%e_qDCh9vJzP;`By-q*Npp}W2I8100f}9UJG)&nQom#aiU=1*!$t%k|RMqFNtLZAB zV7TY;b5gbs?Z8Hj3h+C$8FRlpT2XCvv-7SR&4NATfAbRnk0DPMf(ARHFQT7qc0VI~ zcyZaU)=&UX1-on*aJ*ykPleFeb`-Nyjy$!g!NevP(Ex0B`sDN|bzn|@;wym;Zj;7wZFyi3$7AgK0hzdG1# zztM2MI;CvN;ltK2*?ll6cc2c&fHeirU7LC6OvJq#F8Db4`g`R%o3;vp5($%tA6UV- z%LXcM+48)aYzK+Mj6lRz$+$>?lxgf*34QsS7%9u?b+u|E?N5E);B4P;y1Hj zy&HUb_62i9-iTHZK4@yxI>wboadr&gzQ1>2pg8X`CE z=i-;I*bW%?xORboh-NigN-mX}#B=vC*7Y^1Bqg2njr@Ua12VJ2Pajn47NznYc-sNE z#kl>e2-$ggz9IJWq$ORjoH~4-8+M+RDdadydSg-J55}~b-_Q*wgW0u41E*G>5=msY zeuougtH<&KWM#p*3xs?YY?fp1Zx;;v`fg#&NEzNH1ZaEk(mfvY(OI0|?i?4|{(SwQ zZon_kLsJZcHbA4xo1iDrJB!z~NWZG6zqi$<@~JLj@~MoaY`wL)5gp!CTB}is%T;Mi zo%M_~Ak#}NbmQp@pnN8oj>?(#c!RfC5skM;o8r8zCNqz&&x{_S_za@VSh3Bs>NnGm zD{$YB<1t9oIwBZF;YStvQG18|~>G<+b12UC6uXYdgpFHIRX11AUrv=Ii{V z7HV8W>+Gi0WizUMvl`~2j$5Zt-0PX^927V`;2Xnq4@098sY6aO$|0x_f-Fg?Y@-{mvfxqi(1SfQt%2Qte?f#PW_DVZI}i%kkXB zh8E&FZ~4U#9wTbrRrk!?R_aP#{sOZaQql-a!fua;tNl#RwOZXOXcf|@9{Xis@Hjy9 zT)co9OaZ{Z;V|k%N)z&KNCX7jgfEvL-pm~`lf>SDk0Fk0n|95X12j+$lMn}vxpvK} zTSjdT^Vf9i;MvY^-ZJY^L=&=oY#{bQNE}FhW`8VhjWz;pd2=v9O)A@1m9;5-dHTj zcdKqpD2!jX;_N3p8_d8R!P-Fd=hs-yzhKlWszDsX>961ek~?=+@0)KN%(qrc@(aca z%*>xH%sQ=Ib2YmhCLU~c@2oYT)SVj^Sql*@)`yhsA5T$S%NC2RnRTEM2p_AAZ;fRe z+I>Abpk+J|?g@J?1(C_tIlpVWD)|wX^0Hcw-+SKF`>+xIW|@di=d?H(Zr5EK{iD&< zCEfhQdd}Pqhgve3KI`Tet(u~4ZCU*Rq(RJYsc8ab7PhceCLhq`)NX!$JdJL#F*2NS z(ujuO_njrWzf&#)x-^76mWerpw=}r~p`~7sH+B+d3VU_3dT#!t(r&a&_5~ErE0N1f z%OhEnRJ(fL;_we^04Om9o0a$hr3- zuP0cDr){siooH({$+|mam0E-{UXYG480Ka0vzFnG+V?H!T!LKJ-=4}k^4nC8FSOc= zD$8V8Vs5=;vh$1jSf9H?5ND5|Vp4BNhxR?=(VD?`+}1u&LX?~3kGv8OC9P`N%)ZTL z8t=Hc#74qsE5S7!ljuj}!S!p!F7?+Q((AD7GdamDB}7|X+~U{P=S(#C(4vvZam8&q z()k7g!WbKRuBxvx_~Pc3x&6%@dOV!xmJ_zr` zlA;7)V{ab}4Fc3nDY={d?(b4t=c(-Oxyh)TF}Xm@z_hFM_S2ZeH8x+O-j9YIgos^+ zd@4R`tY+&lD^*7QPTrC^vgby(_Pkv)HBKz5)@LN9Ku{WUEAA&tLG)}OC}mm%w&yOdkC}G9dLS|k$uf!ZlptoK7VjNQVl4$>_gf~G~Ucb z=!=$B08~7kiAF#uAc(ZX?I4{p5C#QdDaJ%Ff+u3y+->Q>CkT@@Z=}{pcQ$=eg?X3H z0{Z^eOrjT$No;ILd0n1OGo^%Xo$0>dbxGt#$NNdPV6`X$SKRPOe75TyTvG+85N``u z^4{$WDbIyOyqPQX%=a=DXqjKRFL$?x>YVoXfw9_KTtz9Y8(oXpV7m3&cjlJV(G4a} zlGK#Iyh@|cA_EH+AR$V96pW3EhW747Tw{4q{ZC}_E9+YJIpS+7)M*l%#JbE_Ue8y& z33;hsmhk)Nf|`wXWTj;avVDU-8H2u5tSBqXd@CozDSRZGEam59GejWZMyjQ&{e!(Q z;UgJ~o8R4!0(i*Ulh`sNM=`|4fZ?<3Ek)|>TU}S~YQK$B6?OU^TWC1CWLm_ zEl{oeMdajo!rXUtD&06am==^p$ob-XEd>tbS=;TQjebbxN zqc5dB&ebRR;?b~AoeMqLO&BLA9=k-D$Y9pgNf$PH3Po5fw1U+(9kE0`i8kuG337h z0fmBd07V=FXv=g@PT~RR#m^}zDU`l|D$T1!38_~`#-Y`FA(1{+-l5lixlk@~yunn7 zCAeoY{}#pNMb0)H#Cmk*PrJJRYQSHS(^W9D@S$-w; zyLq?Mo6euy_xQT6e>~h@qFla$OCP-QTvz?KP0WdH^qz&uU9CVyj$OGb^`N_0$*@Li zVK$zh6NN43iKY7qKlld>Nce5M+X`HBJXmzRgn+(uGWfkzsZQpux{Ii<{}Q%vz3WLC zQWY^qUzN6%D~@W7XSuOB?n_O7Odhe49=gbwBlulBt6bk-(iFC=corU2`Am~KiqG#z z!^y-q*0`IvVRn9rNScf|X>E)E(bmoWS&g@_$4ND)v_kKo_hZ#u?5}NcR;>p% z;o3Nicj1b?eMfA(j?J&rXGeKo74^nl_+Fo?$aJ5fGaNer>$MFkf*eCRA?u%wF9Rfyd?3%^tF zrH7Gh!ar;T1~Tbn%#vU&a;ky~N^4V7m|c%6a~~oDPym><*rDjyYKd(XQ%j}{e0bSs zF8$o1v|Rr8e|<*N2?j)uD&M{!soc4kFYACJ45a8|-ccXfp=R>CK4sD<$A0Rn#>LKu z0Mu{C zFFgDFaD)q8qFJKAjmwm@GTKU>K8!~A^V-Zttx;oJ1mhXw!tREzst&#EYS>f(shRr} zyw~^mGJm9RZ{PZA9m2Ajk}%a>u|3Nx%+6#%rncuN;wrx(VXgC}%tkIo93%wU^xCW2 zjR?QxM|)Hm&p@Cc%3|1xMi51j=em*!Dxk-#j1NtzX3U{Nr)?ZwSwGa}%JTB7c*X`Iv! z5?K>oXhsn8sYSCv)=uHYlsq%qPFvytY`HE0MJsTGyfBKW)!$7q=aH1V{aAnZ+Ry|% zPi?I8bAp_C8>$DhYLmes9#bc%2=ur#rn=R*%mIbHA*IITZ!^6tK+|bjX)M2M5AD$3 zyGwiCg+Rx`BGgWmSUvp~50brgV*G8%v`{Y7>k|A$ZQ8f+unlzu%q z`JB@v*puY1^zc7jJpWw4UhLTpuLsQ*il07bT~lh8z6q23#GC+Px#L0Q8Ogz~%*04K zqj?Wo0JEvoYBS8v_DK3T9WUO&;w)$M@iZ8znNfCXOpAHj&%-QouG(f6IMBB z*B?3&yri0s`s$Etfe8}}#%zzJRKe9+n9uXcDtf{1O}481ja2+R96x_6dS5Jo)py17 z4kfOQ;+Ywqujxi%eZ}Uz{{G{ugN&4%wO%i(qy%(5^b~>}`<}(Pz^TEAGyzPyoV)m> zCOdDeA4PPW$C;~CoXs!&Bv5Nk(x;o#oLj<8oC|oHp6_u_maMYJEo_=ZubAEZM0m5#z za$`|{i=E~pJa=%xa9ymWzNo6}>r-%*UxZc7vU zW}!rxDE*Foxij)tkeu-&r(^(rMha)}f%@`7Z8e6^m! zgwUgK=~YKLcWic8K|(Ld-lhlTQ8mS5d4r&Fxdtk`qViO(yciW+Jy6j5+TN}TODlP; z>Xi9e1=B*ZhN|r190gm7g<2vd3Ly#TG5YK55m(e6_w$SMiOnyv`txreYoxv~8H4S`6E*BGj{kA zf!YkI#JxZ;1u6;f##vxR(_r(%=2M>e#&NN~srN>IzxM6f^nDt#|bFA-e8Z+@c0u2G;9M*@vq zHZs$!W3DK-#qOJz-DB0#w<3)wvr(Wj5k_g(gw0fD2w>XvzAQhg?)#F1)CGfXf6JeK z|C%7Gg5BgjdmjpSksr5);+z8^n{ba4CYQ%yqTs*;x4odmr;voL0cEq45sp``JNSsU zuB}e?LOkTKS&d$FCaWik+7X}(O`K|Pj3g7fDwdMg;9v-mEO!eVYf0K%d%duQ)o$DC z55g@LOoCV*rga8%fyb}kWamv7YD^pQc|Ws|IVMYBX*S`|k{pv95?(5l`NKp1r=k4+ zFTTI~@4x@`=QEVI;omTP)(JO8KJu9moZeMb(^ebCI=nhbaku_ux>iu!|5SmN!_g|L z&Gcj*JFr6C;*dAdM@9Rj`boRDT}>F{NeViLBY9D1PXE(0TnwB#vA!U)r=92GJZ(`Ux@@s%x{wM0fTE{+5|x=5xw65@ z5?>#$O!WWYt@%->y9I;Rva#4|SuV0%r|6A0x|#NQwRw?a`#$$;Oc(C@Sa1lN`C6{^ zhh_R>*8cO;e#bLkSj60o*|R@D*l02TG$$OJ_`eA1 zf3MH~J!x4}Y8fx|KIz!s|7`N*4ci|({_l_dm&oi-zuO6m5lvpailb_^i4q#4-msyx z7re3DGi+hzixt*Av@ByQ(pcANdJ|8tbSvIsVP^JFSZRiv8`|DZy-`Y4Z889ZMB9o| zJWxfFL)N0e|IhaLKfe&=CYADc^+)S0s`Q(#`_3`x?gvjof68=}`nw&kq8oE)lxxkO zTt&3xi-w=fA3$9Tt$%1pCr}My@d=XX>K=I-x8+ZgstM3&2CmM(GU=;++noL-j@23S zK)Cs24f}gM>sNo$*&4dP#iD!^qAEB9o2=%P2NNC{->c{U{kEB<$$u1=Mg( zOdn;!VmQQjqx1(HffjefPEAU4Z(EchV}JVsMc6m!eI~CXrOYx#aHH+NMZ$mabas?G zQelh*jS=zi2(GwRBecuK){v#WvtP3#sNP>5ol1&bs%BCF4kDrMH1RY&j+(WH-~3 zQgv39lCt@>_1DrHndHD<_iPq&9$)ux#u?;WNeN>41l6V0Q2DdF%ac!G++iJB51u^T z@;v`Zff)uFsBH0o&-RAgkykHV4eGres z!R?G@nZ>R2Zk=Y@x%y-}F8ca* zY!6doG%Enf-!Lw??-%3?leGSV@;95CH)RE{O0><&2f%>B9FRq zM#;8hGnwgh5PUDJCM>#}e|V!*MzQJ6%g>eEz58{jfjx>$ZUUaCYjQjPCOxSCjn-*{ z?Wu?Zn|k>q%1vYC=PZ4_#FD$vkM}|zy{~Nbg$YxX$lijf624~)crCRtf5h~;MlH#l z$TixDI67+yd8j+HT)Fi~h3%eEC-<@J`$}*#iSOK;8RRX3zSXb}Oz%gPVIHp`Sh+-|12> z#O&rY3rkvfY5kPi=Y<9DEL2Ha)Ynu}1a!hYU}K>QzPCbtSL7)$8{y>bt;pzuADR?_ z3!G-V-T~k>9?&hEu?6z}4qYpan1@5*&o;9Y79cv2s3`I2JTwMuN6pmwBL#Ii`nHc! z=0ttsI?R`@BP54UHh>ikWZqlTA$bAAwX*hkV6 za3?*ui4qmO`9v*b;>Pm^#x06nqxR%NRLkQ9|0%^6UX2;*tR)q&^$ZTo5cIP=&KG68 ztl|>>bgM+YifASIDQDu7F>imp&t~-@`mk=Me-<#!y z{)`@L|8qd`-8P4GFvivzd-u+k66w#V5A9Qu^?UMn9C0mn8^}WxJg8{6bkEnMGXgcN z8@HnXOY&j-XYQ|5POL?nH~9UeBu)A`D{mer zU46Hmx1{dIiWHRE&J?2Wc7vn_Hu)-u@6s#8J>_Xo*fj$69U1JE|@=3z5={tdtRu;--^ct{>$K=$P+X0|Zf$KA~Y*mP1= z<;7~dC^N(xUN+db9`uCuCU-(aH?DeERsGHaU6oru-~-br74z!N>n@QKV)FW3QgWTw z9ci?Xj}vm<=_3$~PdDEWJn-;Kmo9`fX2?LH`PnhdrPtF7%BT9@9Pv)`uPs#i@)*@% zR}8P+4m@Xnh9h!XEv?^b;0>c9MgX+!w<$0*&$n^R`HmG5BtigUxI}rUZ9$MHt6fGgIPhk=orS|&5 z8S0%pUDf_)U#7FL!VycgekIDu*IdFlFjmvyu}!<9=6X|$r_ z*cvDs9z={sPoqoDPjgCrG=>p0;SddON|dTts1~zQ*CY=b3V1&g&}5lPo;F-*c{O7f zy&>_!0Q3`Htd5xLMLLK{xIwbIqaZbB9w6#y&nb+?BK&CpaF^ z=IEY;&s3hJbp>O?m8R7mS3&rL!2(mE3~@_$!WZAa^ahwjOz}&+Ec>bigVN7WRHfdi zHv7pj=LkOrk5UlsE=VSPUKRc@^0UskoEhZ1!F;GE5Cde`QO&rEeG!*9Yu9ynubefO z$(NeNuv_`7eM-Ing$>C$FdK5n>UB>ok}BsDHOLLB-^DwtcNSMiU3R4NEuTNOvCtD} zsR@2uTNdOmnOGM*_WKa`N#Yv_s}Bn6BxMUNqx=zyYFNf%qfX!`Qqw0cjG^$ND&qyY3F1Uit2N~J+Nj32JCg`uyS8Il_-YtWCU@yifMXqn- zlp;L_(?XR?0yy_=%j<4B+~_(g>Lo9-p~vJ$^{HvE7@&;uxipji;^HA%P#3AXRqgig z_`VDhzEzWv%b;RfP2gzIfR7BYt_;la2|5|rJ)VA=;IePc%gWGR6grHcQ zlsTc(1k0b)R7b9leSh7(94Y%o@D0j-&D>VBIW^oe z*US=m1}gDCCn6^XH5|Uy!tN@{=Iw;JHO!0YGnrlcEATOIv6=N437PfL7$=)zS9SB< z8me#KLWDiHD@dDhrcX-_*`u-{8-_yjoV1K&g7yk7x&FMxv_Pz?)a9yYP@RIZdgk zy_y?@H(6m5o-(VcZ(EfrwW0%_b6)*3Fz9@Zd`p;vn_${o$4bxf9sL*ay?x&ifuu6E z)T{5*=KXgWN2i{c?%+2sChD+j_!tO0(~OAQ_Me^WpGx`&*i5meVPB z)`{RUyN-!F;{Q|1(BGw@Jn!mDyiHMVjXr~g(OBWr_g&~hZbB*s@ zUQD*U+~2vJ)AHs0a4WeI)Tl>paaz9iIY>L+<|eoNyY75*YvmdxP9c#hz37FGy``!S ze03n=-~&rUm&uPunXM!NKthYEm%2j#x!_hoKsRk8Iz>g?+>Au9QutdI^ey7vn zLl^{sN3V4zg-U0?W-TQg2@v~&9>4c|n{+U;kwTdx1&_XtEgBzft&0?Mu}_9f_QPyx zEwA?t<5P1*#lCb+d5-PYBiiPs>9(=MBH@w4L)Nfmb{jUM>6L}m%=f=i2!b;R@a-gb z>#U!YRlk7Urug6-@xSP)DegUGEsXowrEz*(dC$Zj9i!#H&kC8GwbP?#de?Xz^u(m? zv{PZx_mr3`vOsF39K=9193e+u#`Uz>a$GKnSJ>Q@C6fO~MsXHX0=a! zzI?IhjhU{{q!0{d(to(Nkc7SV-4>((b-{pN?஛=k-S7wPUk03VbtK<&E#-Fw} zk~iKVyW#B&Uu5x1oMYM;ydY8Z$Q?zj9pzSM)=oaHD@jEEvYl=xXwMTJUB*z#>$!&} z$ZdcNaGiGI^E{hvE#C`gNUyOwSPK%_fEU-X-5v@!PD(mvt&b+^c~`Bxo2zeY@$Ws_ zY}-FMg+90Vo^sf@LwdlT_nuvT!RwJ4e=_a`wsoA#6w=B>ccZ#q~%d(Es3rKx_{qP6Jj&AOl###jM5gqrQE2HS{cXNHPsJB4qc69DW3=Zlq?R>X-M4A-{IJITvz-mpz|iDL2(Q}` zh5im%-gRt6-fq0V9g<;j+4W9uvay->I4PrJtRPEyu{X6yV@AqoH0yikaHq@>6TiKk zo7t6}n1xB=+`TB8)X8>cw~6-M@mlv6gTG4>Lr&gy*R{|I$u2&NA+0^jgi>q z!z(FoPL>NU0AQBcrjrJ-Y4AF3EjEqo1Lbz&9U zgV)m4=`=8bNM~;u%|U}tD^1dID4AXYqH2v$EvIGu{&M~)NNenZuP;4{SDq&mXW0o> zH;oCej+R6pd*oWOSyAFfzd=sh7j?7)#SyIVyS2WUi8JXG8|k=Fy+57WhdT$GrbQve zv;nlNcw{(ee{RLjFwazkdwQlEZtOm;bWFK~P*Vq!Exa&=glIOD$?KD)hj7*Fl9ICd z6OH13QWJF{jT6T>^me!56MRdxlGcT*y4UtbFS6(FpXN`fc9C#N|E6SqHf}9qWTB@7 zr3Z`gwfMQ`>H*{h0}Me01LA7DVn`~HC}3A&Jye_iN2+7+HDfpZAHFf^`p+}y^%T`bNfseJf-22ZNB zgnM5oVxc($Cr7zx?o9I=jdlXDP4rW{0CY*Gjd~$R_fSiG%eO`CnG=?2!o4H00Z|k#dHh_iP5Q>Y9Gqr6t`}Qa|Ae)m{BHh);&l2Zb$GJZFORgSoT;-SPzYpHXs8yyRpJ*{l6*EtH_{*DWgL3Tb+w9+Y92fUzSa zr5u@T4~{O9fA#Lcs@rB+o^@xLtDNQ#mZsFgk6H?p`k@H!nGLc~r-9>^n^|S#>}>HZ zyfAaCjV#H8{0oQS9Tb^E6)hR4ts$Ho!_;?@BV(&leulaP%E0vOMUeIZ5qC2Dl3Kpc z3GXRMj0(uXLWZ!VlHU*>O~ocrr*m9H>iU(Ji-vdsxtO$4vqPT^w>|vLSePOAfUEmH zHBTh!_W1;AWfCa%gZlji6~+hT6^w_Q`NizW!|v?KOZ*(c@1`R=LbuXuC)+A{sUo7~ zYt7q@D(ce=xG+9+Csfj!H`I_?m-yIn8v(!Q`Z`E*E7&RZ8{b;6cV-9Lf;kN-E6!w{ z}lgLcdPM;BLw9aGM*di*wk~S?Y^* zN#wjG_W>Ift+cHy*|R7t^!$68Jm1eO*nvxd)E@))`?Wv(qM|nzd0>0{Fb?eayhD-R zReEdru?`7miJ$p{*_mBlbkyE`hMx2P$J_+j?Cvn>tY%Y|vni|O&wP_@#$_0buTByQ ze;lSp>^u`T>g&YkmcCRDgkiw#T65$)RP|Q->6|)h`eqH<;?l|R5I^$&v`UJ=+_?)c zBB4@R!>j9}db{EKK|qr}V_D}wu=Rr>C8T`Lmg9RHR$P3vWMnA;4`cM{vP4ZB8`KwD zV>2}!7Sn~HEs-#5H4sb8@L9-=*`!RYI?H7ZfBvf#%W`OxD06?3t~TwFF~7WIT#;HV zO{N{jU98Q2{W0A!~9Vy z9_Osd9Qi$bRf`d)vaq?0qL2tB5v;!Nao_@%;@D#82C&(rt9-0qaisVT?=56g@*xV* zfTijm7qyji()G&6#IjLle;^;K45rAHA(CMR1)*ScInOObf|kZ0SvJ2^{5h6dmYDG| zZ@()D*5a%FACUPO4C3zkXr<{NE>)qNX#;dlLvQUm)LPG(d2Zz0`i~eYHigSd>VpA2 z`D>XPSL&kRiDa4LnEhBOdd?q08Dj@eDLnF*sUf`xBD`D_;fzNY**m%|?<1d;ha#ec zC`Qgo2t%mVA_Ul@R{&Tep6WmE{)PrZS**XW)`ZP=mH!1wAZCpQpkQQ{dz;mouL-CY zH&s3__1!gavNTjG(6UzEO4ch;6k&`jw<&=Mo*R>WiA(B4x@A3ZC99? z+>$OHV@Kx35ReLYj#olN45RO;55aH}ELM zx7kvg3vT|djP7XwXp(SwxFR3V01tZuuT)k#Kz9cW>!R>4H@{rTA#_IiCB_5e}ybv7=?G9$Rap;HD5^B6#=Jn$K$W75tv+)OsG} zj+(9se12#4b#4}vz7Be7v-tS({mZJ2Ih`G0l-epqLujj%^5uq%Fd1ddN9GtSg%|w5 zYZ~1;V&=!X<)n$Bc)0@9pZ5;+CCqH9SxRP(1F=0wzG;X{iUF8YY~d!T$+gdW$f}N8 z!$71ZlcC!YDT`g7aSUYli1gQ&^klgS+z$QAb~Bo_R=xwG%04aCXvy1GwZl9e9|0Jj zIj>; zb=Xrc)XwnYjo&iu%Ah{CgyvRW=FAK5KHElCtJ(+!WlROX_rN9(tTE;SC32zbcS)00 z=*8T0w{-7fuk+I*M1dhq#<1x``*%d7j%#R3fK)gpu2& z3`OPF&3w)FJa{fgvil&(^enA0o~g&fPp;9n;~IXMJv12yw9$fYc+sJ8T&c7{5ZQif zJZ;5f*IC8(eJ`O{ql!jK{rz{yW}C%)y#S%Ff3X_b^=BB9F9uww|Is6U{#x_G9;@Uz z{=RPB`2A$reWm1?n#Y;8;jV&i*1z1n)q(ND*U%p~Hy$~nm-Nkuw>Lh>aHJ1me|U5$ z?=g-2D?95vl;nr1=*H>`T&=xk&e^78@TVe5I(TtYZ0H5aizr+Jb(U(8ia2w^n7?_a z>gd$b8aAAW8L6*dc+pHRRg^AkT)Jkgis20Pi=^;5y6X5*ib$#EqQuI~9^l_6h296| zx0xl?F>U}+?FxUZnjp8Xs7B~&JK>*vzsED2Xrg66=92dzJ&MqL{@$$U^e04@~Jel-qKv1k;f2y;qsNKj9h8)2Twy78b4^} zND15egz*fqrft>p=eY$8E(=N;n3!jI5LR=Iu~L?@(@x5es#+K%WeWSoxW0v@OJ#Fn zrm~Uih(v2z7gMXt40#xhkDSkO9X^>XY@35jLQudAKTy9)*qV=5qNQkt4#L)B6&=@s zZhrg!*?R5opIK<|N~o+GhMg`=BUe8zVKU1YXQ=7EhcR})cF_-^Oe~fx@CFyEZOkU8 zmPU_UnqXxX=Ollo0kfh?mL-2)!}sU67{6W6>-5&U&BEZV30moRQ`ouu$w!&cuzNK? za#6O8Ml{j4R%pAH=@#M?ZX#i3Q)$2vOe#)lFna}7tiqUjy*z^WXhD+80 zhz8u7IMafN0Q*^QuLyy{3g?Bl8w`#d*!E8L^i^G_nY)*EY5zB7`QLn5$f#Crx@7ai zVnewP47}w1Kx-cHE#YKMf^ZYJ5)!S*b{%1`SUJ{z^1%uE4RMK1ruax1=ySr1nDih11Y>PS8~5|%6^*^4JG-R zHLaRmY_g2D#r=vz=QH*|NHczbRa#9m;PNkbc()6&lI{klZ|eBDQh6E&LJ_hJNAkqt zIL`vZA5tK7SYlZisYpW>#Ojc@h)7dP#4nRRgX^%kroj96nZ79y@e@y}fjPx&G&T-B zVJy?=vbLtoOf88k4xQ{PC`z>)12k+5wFi!*>Orcheb-LhTUJ^i6e1SkIxFg^Nu7$P z`G&6{I`Dn>NB*W_!~#yS%3nal=sF_BEi*>VQLPSKYpq$*R5>ZPgq_^ILixxmIOxwB z4$D?kgD*t^O^wMyl&EK=D3;@A?cur5cyWN_w`V!3Xr!?{R+IA7I*PgQ&Y`Tol5xG8W(6s`*;|0Qa_fLAlKKFK0_Ot3?^H2@E4Ul zd>AJORP7cq1#T16eRfoyHy&pu$XTrSjUT3izthRj=avtsHj1gvcIqmOpm_xt<^^Xc zO;oj+xZui%87Orrv>oo-3k5)>M=Jv5#v=C#_q`|0rhR^pVQIb5(fkQe0jy9nG<;LO zV0Z0gQW4{mo{l|^JvP881$kA#(gHOBnbB<@99}Lygt1zWyW1=uL~7#Gj=^AqwA*Vg zZ|YBL2cUTH*Sz93zaAsu*9Pti&{*FyiyqI9?=^Zvp zbyye-cuE?%c>V5?+r)j2|8f=+e+JycgxWhBU_R9qJ*;~3LfOa@>J|8oDyDU`Z?1lA zBL&>c6hC@4%^%)H{+~GmKVttb<@aFRM|0Mk)i0ER<;(M<8QJRr`;Qpg1En)4-RCaw z`~xcEm?)fo1}B5LP2(felH~IAN325d1wQ(3bfTH2I2(JRUEt~-W2)DN?VgtHGc_3l zAy+7ZSv*hlZ3C3pz)maeX=<~D$5l+-BX*IRB_qs!Slj9u_UA#SJ*P|{8T3{T`jkpJ z>|>dWQrn3qip5q#8eHh8(DH_a8SyfX$jV@J1U_>W0q(>H_=3g$at*bWu@Uow^mMdC zV*mBBxWC7cRDJe&3R!hsc&~}NrUCVkQO8^+4 z_-_4CP15%PmF~&$vhSnq6U4WyKkio}9fZB9wIr+Y-oss-6!`@SQrW8YOT4f0li{ol zcfcdEazw-1R2?FccMY<&U4K&7PpAH*LQw{R6BVwC3N8^AF1W;SL@jiF({ z7G?rmQl})MZo?{rWc6X6?v|S3Nw$FZ7r`7uZYwhi*>-Tlqp^KPZh7wOHfih{b8TuW zR2bo2h9k2rTIzB7J4lv+5q0lq zr$wa3)2R%z*%0wquKvQkhAbjAafsCCv=l7Tj>a>U@|3T}R*G-ZxIA;GH&8(n89-|z zx8gN)G1AsTP<5Sbv{0KwEtacA;o@gh_!g!1L;JgIm@%x1GsJyaSDZldWmjd9}?Z|D@^s8&+x|9F8q&Lch>_p;mzD?qt6WT+< zIpKdj6Y8>8XX0&duXqIn4pIshDZ&Ou&Kaz*N@EK*wBNZZGSe}0$=GioDAuh{W zNIHDX5Z^@-eU-~GFtR9_H9vWsQ~28a-{P$QgvK`aNwekOQja3}w+20yBF`ZiRT-?j zbBioUMh8P^#6%1uRK`LKv?vO|eT_&_C`pyzP>~}DCVI4{&T0ns59$`AH#<`bB4vCo z@KzZw;ZLcUMN6PCuAubIQ(})F+^=G`yS9$ld9Mw>sLvnt*bSK&Qb1l&}cQAF4$683u~S4S;uDj5no(PCBi z2gw`-tr!+gAkKlY`EsJPgCBfk4>;(74=s~SEeoTDK+|+FbQ_-base@xw&M}}J>~v7 zct~)>S=XYvbqZpq2j)i02b46-QiZKiy(^ONaZhlP8_lsiaytysTr#T+B?YVa5m23v z=c(tu4W(6C%!J3{t5RZ9#cFH6B@geur+a;o?LSiu!`H112lWTCWJ4r)|1Y1Z{GFC^ zcm^juX|`~~Tn5?|f^-P3nd=@EWeg}s2_!R08h(JMAA`=FrhF@nyEE}QWlTyb)X1$7 z9YOeEiq;e1oE`G|usQg*1AmxvMx=~g*~a>9A7R=ASxCK8$n6#BFt}xT$e%`rw*kkD zhF!UFF__^&M|;3t^-L@THwqjqe>%uUh&Rd3d>C37yf)73Ze0zS|2?L)LXe z@kv#prA*rP_M#YC_XGy>qC2_>Z8Y#e?ORdEK&h|HUVq*1j7SIQi8%gbrh~f8068pu zFQj9YE05mk`C?P87c>ObHm_ZB{(2<9Ph1`|;*Q!Oo*Qu~g-KFn91?j2QpOWB zodjn(MdOu=FIkrIQ$!a?D=$=)aB~uI3sCi!w~}OH$h^6x`rt+)7UIN3$Rs3K3Via@ z-&(J^{oCk1e?Vq*)`6CA9DU3nVRwk4ce!9M*4hpR^Ofgw-oNl}J;QYp&rz9H70myk z%=)6^9V@XSLVDLw;`l-4>_V7)ka4bAQI6%!;D#eZ{9FWd0d z+CAU%oOjqRzMbMJU_B4&G7rS6%+_M?UUDxnAAp*OEOuM#bu zv_`qnuB6&Z(+rGot&PAh{FT(UJUaWyV6{@RO5<7vJ9-J?iZno>$gD$*jC&9 z#F{z>9FKqYZ)mZ`8$D1G%g4XM?LH!koJUQ5HrM={<&B!M%z$Bf9jko`QD(7j+38Zy z_6PX@QF0?Oe~heyg2drf*=QZ+<|9<9;UtENZ?Ys!&pLA&5*&wvb)%M=6Q4FBXH(u$ z5k_GSFZM()f{a*?llFZW4>U93w|o6Lcbx{Aq0HSFX-hMfDt>UwaY4492%GQAUQ0^+ zTvhnzV!-Ap#LCU-9?bC9*-y@tZfzzLh&i{}U4`6P(+f*((eMEdPMwO)`I{hnk!6R1 zcmCc(@b!CKv>0oMz0J0&&81-`5tk zk;q_7{Th>Vq-~u9s)UoyF~pt4`i+SK#Iizml+qwy^>k1d+x}mM?iICBznmZZi(j@( zt$0yLVZHL0TMZX1I;aIo4F#K&%GWe%K$H-{ZhrP+x2pL#*6FI*bnc)_%WhECF5Baz z&U%}u{;KOZ#tfUW_3x?7Wh1MrZYUZO;-BXu8!e@PfxV~B?*<>xm{i5u(e3KA)dmH3 z$de92zIn%AnpEnJn-}vAyD_YuyE}g@;^?SUs zYD)!EZ-JUOkLEOU{S&mn8MivUG>Nr_x@ZHvV5QP>{j-lp$A0JP ztSyOx^26@|l%AKYEmScbFFw8Og|f`4zuYne-W;MS>G+XUYiV79GPmY)0_eY=(Y`bfs%Bp?CtlaIq(q0$RR%?@ zr8<65fQ;1f`t8vPvjln32%13SgyZA;ZxV?=)gQ~UqMs+s%6*?6ZyPsY0!fKVVzJbl zo(_*r7tRM^>ngU4O$a$Ge>TT z2e_e}#L-|^scXFJF0k?0<4YQ=u={oUHcayZqja2VycZ4aibw27I7LO|c9-qf(g_{n zhywf;pUB<0jfsISzyMMt*=t+p6}$e9T#1tC^S76f``^j#=bHdy`H8em)l(VIA||(u z>YFEH7sic+_+uHam@%po;RJt%os^(B)m~IJMq+;|7Bd1VLsvh zO9#j9NfSXFcY;to)ZyPT+6j5#2@3{{nRV5sPW!RE zi^e4K=iuBpwr_GkxPZ-d#AZ|mjYy{|^<|fPw7$S@fB-PeU?;1o0x$pS*|fnvA0(h~ z6>PklSMI}g+tB=0c8#YMjhJJ3V*Gt!Bp|{Cb5Q@aBN-3>X+LSm6WDr3g6g}#A@kp8w~pyLgYiZ{_Op;oyA@jqXEE*k#%;gpqMr>X33 zR+wG*4)#^;)2!7q)P2`x+Z8evCIL$tnmH)ViWAb8VQEd~pdB1RF!wqJ{ZOl4&dc7B za*?>{+T^XZ<)eN3hURkA=>08xU!T_iedj{4WGR3rZEynlHYT;zUc>XXL{iP=_2{;$ z-M9ESWVj`#qe=jP@=k~JbE3t=kdOyCkh4j89j07?(#U`m!N) z^V$2-e47q?zLWFA(%NK$K#8yHRXROyxLMm7!saWSy!Yz>QhwXIvU7Ucy@S5t zQ*IM})54+;9e(25V3E>&UX^2F8%o5>-PezChXhTSSUo$lW$UC)DW zaM~V`{!bUc5OY22@*PnSz_+nim=Q_yJf5G`GdkmUQMC8g`zFH)2D#9Ej`G$y5U1Dc z2k4oj%FtFg>?{`RveR&e*-6CSeXZ$5_~}{S8m!(Fz@lY-fW_E)dk8CHgP`7`l)C&n&1m2@b!FS1zbO^U1S4q^t@aSLdG`Xcu)Zb||3mBx- z%iwrc+_!OED>Vb$?Db0ONCb1qA2tTRI>@Aal5Kg3bhF z)a-v45Sau|hg|-Rc2|Vfc}DE#=IPZyso3ug)NSXhKNCJx^j$J$F=L}`8Con}GLi#i zLTQVlQ4=vN^qb@a7I#^8_$>`^+JLu8`JkHG{*9e|<3sS)#eH+|>>-O%dqoU|8$a*W zWlxN5n!_VGP2vz@LP^AprlaB5DuPkkyGkK{b-1nC=6B%r^_`F?cNK)i=#M!){_EjE z!DWbg^$izNO;Z1yazi+s!!IN(+{fQ&WD0V8yqVg5aR0r+>0-bWi{Yo!?DKgP_s#{z z0g~YCma@Td>;=B|NCLbeC5F9y9fYE3ue{iXH3JH^@7q)oA)R6q{fq0iXhMii$~#Pk z=;F@!+u!;D5T9(Y{#H4iw;I~CbUu%HNqKg{H~GSDzCwWV<`ngiAO#P@v}{iN`~FRn z$0Hp(V@Bu#V`eCK3PIIcF70YqLVxQ*m82vP)r*4;>*(my>AU}ZJ3gnEMN-3Sd>4HKp38)41Qm|MW$Lh&E) zBNgZK?OSr~+3f?;ot~$~U!HGaUHsqlM&{SSGQrOj@vPKSyXQf3MNy#WLwB@Drn%N_=_RJ3k)CC+KEc;GsM z>HU2$aJJ{df65SEQI!asXG2JXWxd3z8BADdRmfG@L?V-~;iJu~AvZ^K(+iC0VLX{Q zFqdrhqd(g`a=y89#ay3kv>}F5B8&E?du;0E5)L1@HTnzTshf#r%$LR+4JTShHv%f_4)Ro&664+RPZkj zRpJF;Y)Kb>kBtQh^*U<6^;xQw`M zT-rqDR@xH`%04oEM)K^P^Z(9&h~PCmIFZ!0ShdA~>fL_knxy(r?_!Y4e1TIssRe9Y zVblLEZ@w~I`a2~;K)koc-M=3S&(q^UK8^8pB;fl>nmXgfH;HRsY>0&`x#cA({Z>lb zxhGCKE6r)%!t06}NmMPP2@yZJU37<=V8H@1oFuvBc=6E!I z^e@kpdPOFy?G%2WGPugDkm_xaI6U!}gl<7U%j+XYaIG6lq%I*_%xGASC6`Cg*5D*V^M?efL<1Bz&w$z{MZ&^QxAF0~uxCbg<7l8snB zj7={0A6o6r^nnb>c|+b(+r_H;f7oP=$Rl$AzZpGY4R!#j%;tTk;O1682jT|837p2a zJ&m?0;5cT)?$tgiNflu)gZ6IywbMX>jV>vwE<7Ed13r?JSwKdlsp7J1{`OA&viECJ?NA|jbKmS|8s!UZKYCPEsc3eHqzr}mOsAGDqj zE|R7z;^k@i<(n|BP5HKg=K{<2n$gpp^+Cr$Ktsga@FUYViqH#8uf40--qr&47l^U|KgRQaOaja=Ko)f~PM30jkY7{|MtK2}3 zdpj(2xoq=3vbed`2$9VX@V;9_5wzckCB5Pz5@=%_cKTw_@DElvUjNw$c>c+2maLwt zj4H*wT%(Z-t;#nJ{VG?&+{hLDGM0}$9ubTwz1XfMJc_q2uI&>ufEuAwEehTh7|KY$ zoSneoce4)x%5;F-B(i1I)igp!UX2p$NMhRLiTD*_i%&;HeooBUu?n`^_adgRcgmg^ zyV!86!N*-al9VJZ8G)iAkwiS5UUc{qAt_J8jlzUw&dTP`*uu*7ASI4ASm;1q3Crb1au#3JT{@kT4oI#v9eWJ%<99$cd>}aDb5xwUpiOuSh2* z6!*qGKG)MZ1qaP`=Gc%(S0xzAz2!J z3^*-mR3Vm({&qlwtH*aB6;_Ml4&qianN6|h5B>SdpARE{7?146#~6kHqq7v?a~f*0 z@FKvXv`gsyl7g}tIst&72p}D7jebL-M(SD*E5hI4WT7LATgRJD+#i3Yq zKYP*a-BICUm9EDms$Y>?UBpg7^CQvSvOnrhHM?6c!9dK@dSsswt(_Th*&&p&mF{_N z7KCvKPO$v)nOxW>%vhpw?E)^_F=)d_l^w)gqoq+(u+&uuu3+p?DbOD02y zxLqGGa< zUzK!5yJmIX^N|;{Xq2Q-TRN7qGC-(1tC=Mb<3NV#!ieDcdQbl{_IlQk?s#8~1@W^L zyxDq(;}iAwxUeQhnxqNhmFVyRl+E3`qI8oQzN<$iVjQU?8d@4Yj-&Yd766tdtI5>a zXZITg)a16`s&!i;be%p##j?JFmBAPk(v@p!RXW@r62-Z;hl|1j-l$<~OL@%f}G2vv+wPP zA}?tjhX_+7_OD!BFGbk8v*voFG7{;QkRICc)&DL6QdTmP=qZ1VMfP4)GfhQ)2s15j za1y`Eah`kf|6oSJROV=}%U3m?TSw#}VbT;A|51kM%_?NJ6)b5mdn&OlW={m)4C! zoic)ev5X@g7H~P~^@d;qkP4_MIyVk360noE4!EH~x4Q29j4TX#Ib^nFQ-OR~?JN7%#aa>P-Xr|)q|E+i8YHxddBpN*&DwWLUSMYL z8XHp35BdO`@ldiHV!S)FcRHTPF{*Y{JR(1aIbD~lRo|;mJM;QoXV?5fV@koZ9lfbI z{XrOd2dwX$-Uo}0u3CQV365J;+#UhpCTGXpWj;dmGwq4^sqy3G71xi)27l9=h+7#S zx*$P%%i_Z@yg(h1mYN2m8JWtYbD>GEbpS!r73vU1zvykxSEgcsVCrr`$@28hoB-qQ zbpfc_EHH*!(%Y6;J=S{AQk!9i8Thd48h^d{e!_y_H(I(sna68dWwKK`JcD2_Y6r&`qY2n;)dlo&GNdNmsG2G>)6ible5IJ5iUfhQSIk}(qEMt%m@YD6S-&eeL*-b-WY z{l)$fbcl7~;+3D$vmf!>o#_CrBzXiqUwj5;{S61UXEUNqgSoGLmj5Od==|AXDU z@rUMRxd?Rg@K2{WjwAZRj_HCHHuyb$WvKGRsKvN{9^2kS%fJ2KQO@O0@89@Hx&aKQ$^^m z(Zv+}{VvIl65x&h1mrP9c0nbP3K8+e^}$xFu{O9GSaYG9glnm*S`igggBuk^?T653 zG+!vE?}sD$2Sr5y?J+!o$3noLO<7d-ygkDmvkbS_)9pXY`8>rD@AmiS~E)be%WvG{J?94n+wm!Iek-QBuxFY zm;SBTdhzXt&a7|Pcz0=UBXesjPs5uw^gVH>*uqzF5v_-T=%&*mF6=j4J+B9375W4J zvO2sxr}|Y3j>iuS?tB;?99f>f_&-PG;Q%?!l{#3^o&uljVY0l0_fJrcpm$t-6+eF= z-JN>hBLL}IX={y&Zn`zO%B;EF#Heq~fa~+3qgE5)HKMxK#H*%;jfu75M<_D`6Arni z12XHqCphluW_^7f05TRgTw{Fbk4fnw&%@2tS(zS!-?kizLXGV$a+jtqZ>R2~p$<6T zaoY}4`s1`xBi6!U0JeRszMc!U8vn8)Hvx5d?e6nV8TfFjy`y%fGJE^p_Upk0i~Zpe z#|OVy(a?#Kt+DD9+)?SpZoClG3JCg&zTcXKv=U*jvZ@;cMh@tmeYBmf~k}L{aV_fq)3Ue)mWcN z6c#UF*C)&=jcC`3jTGzLG~+7K?jhzXxke?ePD*p^S=irSf+ABZjiSaF65=n|QPAGJ zmXdH_s?6YGFk;-N*TUP6I$M}9a$HI3?=x0XJB|kG_sB4JAm5ei`vcln5bu{x*i%o1 zFT8SM)vqpb<+qcSmVPwNc1GoTMc~`q;Jfzkb<_N(JOKhSZK6bBxasvyuVX3xisvhD z%lJ2@uK2ud8}h3Fl8Cm@V8NIsucJ!x&tkG0P^CN873^Ez<#f7!+mBMsCoY*}!2J zxso}n>%mmv0^t4OF#f>1O5(P>YvIil6~T%Drjat3~^wr5*2vt16hDVFMWpcIQLrPe& z1N7S-kVzR%y&N-+z1|ZQEc7@#h$Fdgu^EbY6jwFv^ljPFT!fMYF-ind?XyovA0W+n zn=PI5;{?lrR!A$5TscAhQ@x;{5>`yy_JAIx+su?{(gcCv85F4G>`ExIbk^(kgt@2s zmqX|%E|2nwp&DBjfAOP0P-irb5?6Cqy@MrS^xyWq!KGr2%{cMm`KF8z%h51R*MW?r zqb*m6a!OWu#q0J(y(F6ODM}1-mBFUhIksGE+D<36nS&!~Hpd}oVmG-f;*nGBS9O+FW9WdB=e^YB&Y*e>Os_7xnR8&)%N_#&cJSHl{dZW#yf zHf9bIpG5)tb3>Bfx7LFM+gfB;Xk(x6*ZOqd)o-S5q0eHTI5cjz>QwC3G_4?DbWg;o z`dGtUhm_qcQ78U0BR@=8!O=gF_w1d|_3;G*Bb>)sIMzyLIKzohnP4p*)oJ%_W0z@| zhUH#k$0UI$FV8w{ak$er>-(&4Au7Emd)hT+>wnLMn+|4npZ${8Rqx4IeZd?i1mX@q zWjW%XyrA*ohy5Krrn9xa;b>qqGyK<%FK;tvb=gpyglu{L0LX&Rh4Y)n#1^uBp<8w9 zUO1`yGg7=kOy?nIuH0^4jmH7^Y9oPGrqQnw8h{f)dz0DC?md^ZkB8Ay6iR>`{va`t z&uGfIf#qpe`*mc(a_NvzsHO@yuX$zHox_YNowWM#QOB#qG+;dPJ}Ka+8grc25!eWJaHBXN<6JumWshB4XFDM;y{fkmU3`}=ZRQHF~X?_|EU z{+Hx77&Z3!iw+=Ro!HG(3yvO}7w_^5Tl}&PMMu@CW+NuY`^36tm$-|w;)}?)Z|)9m zMG}B}>_$?M_Nvwv^0Hwm?B$KQDa^xhgj*xabEghvLa^ElK!ViF z^sV8m3s~t~kE4$~l!n<7D1!t$$^x9oB>DBf%|rUT=GnS5VRXYJnL@ZQ#yHC>-eH2T()*-EP%WeFd& z|5|uJ?pqL&r5NWpTT~zLW!G^%*Xo3t$oll=YCEc}rHMH*!i#jg`bB*ri-obr7OT@^ zTpwDYM$c*1eudC2;q#Gi{Ky%iOIpe@eF{Hbg=dt=JIx3Q3iN0elS+HlIzsxlb`)7? zO;ZK}fe^7%Pos^!xo`av4M$_*RhE-c|(NUA7e%Fx~r?`|IubsM_@ z^1FB5^pyCzrS(5de;r`t`?gQ^5uL>ELZoY9L0+?^tuD8|o*m9q)|i~w^y7B{HA=O4 z&OOOh?vdzv>$lr3Mlz1P%398`6m}vb8tE0)@0pnu;3EENkg6fNY5R_ync%XHW>HwO zb*3KeN7msL+}^$2nAte#Sn@q{zq@df#15m3cwtRwN$;&^dA(E)j$M$b;ks-1MV_OZ z4yQG#DLpus&aGYMHWu*L5osr%&VRj_I8ivYOlck5#8pC+Sk^T3IpNk;d{@sS)X|q- z;BSy?XV22S&#qRx55BL6Pj-VG@{zbbvB1a!c@;Pnx+Ka`~5>^6t%;&wbqzzo{v4$^WtdGyQY~w zRf?l8nAQn{I7*HRmA~#w-CO4+%N?HIUAy*Hr(Hqiml|yLVGY?xW6R|ONls|-D0ccq zB4aRG9}+q1<|Qi|L+VLcT4*_D7&8SH<~Im~%YV?rkBp64duA z-38g}7~AU}JJXsDifV0)`Y`-Vwh>O!nvZwy6`37owfq8goO4&1L@UQ`B94H-tfzNx z50nuNF)eQ~9;zsxqhQBp5+9w19_uY<@}E%T7a#-^ z8Eg!kthULT`#!>Ka^G8&o7#4D1xoSh`}sIUggR0qwIM3Cit}=LkZE?ACtl^;!zpg| zD$~0bW!~BI>WX+H2l1eOb4((+9bR~0Xt2zXQJzkvNg?HS?fm-s6`XmBx( z-tK@2Kv#oFTF8dilklFey+$2$76+gp1X^Dje>o{y#lja>q^LX}AxPclsDIG6WWWG~FGISuo%kDUOKnbLVxjlrvG_qv%N-kLiCvfNkf>2S*;VBSN!^)a6_dasv=eAzKa593> zu*x+0Ad1nQKukpiy4Ps`yW`}5OKaad_UQVy2w&HT;pFFo+@7nojKJq|jDb5bh`m^| zL;2`5esxhGe9>MSodOf?7XZYgZsxV#8GfcPAXwSY5BlKO|hO z_@0YEFzm2&nKLAidgB+t^tCDtp9|P{-eo_$4mUa&dnj6wMn8-I2T9vAFc;Bq5#{J zt5h{4B%&*Mj`{o^05&`|4{--XHjl{>%lUH@+@!3l)d{=<6O_D=P;j2MY zSSUm~r4+)oyvjoUYwo zOTRR4T~38_f<6RQT;0-o;3r?_hf|?W87&Vz%`u~1@J1tzIw2hJ_PMOsQ+c@e)w{%BZ z=yIB=lR1ubGuSowQD#!J%Wr2!0!pbqlrID5!&&0i07e6E*{wJc%U#;Dg%(B{O2Et* zatsEo9#9z*`{gu&7sUilWz#*N-U;hpWIdL*^*~cOZl1*wXQ!$znxWtZK%V(PHkd!8 zxhM84C(9!5#QM(hyChOj1oe?ZZSqe~mNt=;cWRYFYKS;`5IIkSsQM9SPok4#2$6Wl zN*K@3l=8-zw6O-9P@GW3qbT-_Mwgj7M6JtX$#p-%lX*kpd(^|)3pI!P<4JldwVqI* zy5oQd)Gg5zFT|{3)H-(F7)|WbVmz>Dn~vT-T*Mw${d_Z)y)ge%buZLR>C+L}amR82 z{S`|LyH6Rl{-hK(wb{{9gjS~H4Vo+N!U4FBG*VepMGQ12$sT)L2Q@j%$sKnJ&eVUa zIG7<{zm;(+_L&IG)%nz{CzW;+&VrmC)A18^cj+^q>7{lFL4!Cd(>udjx(|06@W<7% zhSkr{AE$O|93C>2Y3SO1J@n~b@#M|k;$ddvkD*X6oU5VgX$)uQTcSv3A8P{`oDN({ zj}!>!uNm`Q#A7};J&z#mS4-pbA-YCrbJL>oWWX3b z5w{)X#lmXdV1r7M?yE|dR#hk$%A0}h0#I?;)SKqR*d+kA%`IyYSsYflK!VQC+NhCk+;YCq z8SYpa2M=FcG78q$3X`b>y;H7HbPt#~bH_CgMt{_{I(Qw}-Gw?TSNd7>Hloqvp5JZ# zI+b2S1h!ty+gd)Uk+0%_KVh8Qg;}_YNAYIeSrQ)kif_zzE4gUgp=6YIiL_J*+SZ9c zk)C=DeF8g$uWiF4wr94bs72s_4lI`WT2j4$UR+(n&kY~`X;gg5L-_kNdfS08cC`>k zx=P~HV53{|Qm%aF)$K>WZ*~68dK&Wnda!q3t#sBWFN1ZWHBPQp`e$)XS*X&Pph#1? zXKaHyPt|zN1AU2Gmc@aHmd;U&XX1)W1`Me%hSG?y@Ee4qXv?FlI9S@4{*LYUP4ge8 z!W@P;Q*l?|=Qo4_8qP0;H2!4@rG;M+4m-;RK`cxWQsz8J5C@g_Sq9=Wq-lCQz-Q{>F6w7HtHgCIYL zueDF}ws-C_`1JezAM~odtXDbA9i0J~*q&Lf;aL_KU{_U<|3ec03n@-nP%02Uc|Y-f z{k~gn^Ps^5ANdMClD~)eR9Fq6z9bq^sG20yIb}0kbd3pRt@Q$H-FCM;x?@fI-I{<< zOSx@#ma08BnpwTmGIy5oAlV;4<57g%7uUe9S{**t)>xtgME|A7IYahc^2J)gHKli`8 z{DZyUx#-IvtjT8S6902?{KN0~SfzH1O*>gnZJv=AR9MfEa=TvN$3zUj{>vu+$A)i{ zg1UBe6n|?-^oQ5_=KwCgLzi^snA3x6o>iVw_jsx`LBQOI8S~4FHrV->LF&2azs`LTj;i2-I+WCFUnZWS58ZF3m>zL`inl0u5lFMgteR7Buf9J8k)j7qJaMnMweCquC zE-87um)?LeMr2qp^FMqkLH$mNF7oj}kFRH$DCxOKkJel2;osx!jZA8T+)t&ylsKVMlo*K7B$v;4nvU4dGm zLV4Ia@AohE8}dMjzWC5FUF)7CEhWwsy$9Wt`R63yf1eyfaP5Ennm>&Y=i?{~%JusP z_?SiXep}K{(>gXopq>AnSymB?2P&7o^w$?>uj= zX@y==`A?U>eO5@MA781z@}l6n{>|q6=?aX5)_Q2`47@6^)-kup75k`pVl#Bo8>Oap ziEZ*NWn&Dmh6547zCd!x-gq0X#x)tL<=quXU=N+iANS}Kde5fzv>)y&D*G4jx+f^UC(0z|jSN$hw zw)mnqOn<)1KgI+?9cMYN)&KAhf3i`6U!cn4GAV65$`nZYJ6=4Z=bK;_>8Ik`&|(2Q zxIa9V7LyYqx%H7<-P=Jc|%(CXaKH`>~KF$l(c#HEiAM%d zgX*c|nEX+4N2lH(xAX?vnjZeXvsH<-Xh1BW9p<)kV21NY7Kq{IKR0&7=8{qCk| z<9T3!jM8y70gqeRV#(9kij`@#^Nqo$qXxw}v2`glRMnm>_8}?9ycS9Zk4-Fd9veV? z`ya9P>e&~>T*l5YgO%|AVC=}(25#F4K1A^RWepIK`2>}PWUUWK1umajwRWOYU0Xa5 zFbW)cRFi~+S;kcnhSM@IysF&Y zr%E)!N*Ar%d*iW#?E(q!ZUq{xR&r>9NS{yr`vPPY)u2N@gK?(>Bjyb{G~6ll+_)<{?><6rGnjBZ=bx<}%BYqo5NZYVJ4iYXSQz&_0~+WRhx zl+uvkkAYwX;*Dt(hz@O!u#w_<`A33hqBvUbK~>4;%+mfrKZZ3*}8lhNkGta zL+q+H@(`Py!q9)7yvdMHMr@BQBTuU~Ewlbj#$l_FKz9PJjcla9Flr)I6GM7S}`2$r@EfAn@MEU_x2WLVjLr)+Wu3xVE z($+3bpzVcM!j_C;B_OVQhO1O-XIr7}y-#6w>VAfOJV~^KD|{~;vSDr;Vo^Q0)D_wH z4oH8{Mhlg3uo`X^LgXazPp@!V1gAfu^r(P{_KUsXdHDwg-AS9>1GNkvo8_|bCM%xJ z-W1yMcvBd|{05rYM5P-ai9|AVh@mt;nEj-yl-vCgngyq(C$41fOQsXX3{ZJ;_j;vr zJd)wo4dh;`&iy87s-U;^0ZFz9R8IIVk%zNhAcvEW&>;TtB#croPbv;&&-N4R?9<|! zk^M^!VxM$Uz#8a4Ko`l|CP&h_HclUX4U%gR?Gk<6<$FYsV~??A(BO#GW6Dr5uIK; zXDOai4tf;(DRiv$?PeFi=k7ettVz0O6=qmDt7MkDi80iR)qFW%1Z&E~W^wZ#ak<$& z3FTfj-&rT-6)u+}O+!=D_k*!kom9eK+h{}a!ZB}aEA+Qq-{Wz7@Ekg?_ee3k&9g+* zXxs3Cw8jYF7R3ZergEYVtCV%Oy4`fo$Bc@t>A}qpcJH3>Ej%#@RMGzz@lD51Was-w zEa_BEznp$)-N#>8iOdeHxVW*HsXK1d-ZI=gN>A`13G2G-jw6^2&S&qg54i#p=vyma zItUd@=00`zO9x=W@Veg)%$#)P4ZDtX+(GqAWU#W(;Vu`I>6XfoJZTOERp zDWmB!*bpy_TB=uAXeX$u?e9!A<+C(Q6r}Rso1Fd%BA3q-%?F_sD#g8I#R9c2>pW(~ z4Jxe;#5Fh?F+?28L}G*;q@GCijEo}0a(?czr1MpwGLc7s+-i(3uSvAiQ#siTADJyi zX)G(7udoYyPpUEKlA11$pJAmBmvVPvpMuTIh|4rNaqRNLd&94W?Vmb$moLE_cKS@o z{9xW;b?liUc*@mUPk8x)50~bYNIPRGtRHqhoOEoVx>dSOzsnVm>zZM+NLHM0ux$gP zN}N;@zfIi7jjg??IqSZfpK^LSZyd-KbRa`<&ftG_$X8%>aYL>T|6fc5zcGBSM1ndS zkK1#$5C*c;sJ8IdT}VO{t(ovh8Y_AdlfgtUhT_6VBZ3rqk1znxKRhIL8E7+YKdVFf zP2xE&0!f!rheLR#5M5KZxnJ;5m4Rl+JPq9FhNPuMFIOe{S~HE-%n;3Q4*sa=deV(T z!?i|ZfvOLR**zyv`5DF_j!9hEGJU=ze&eJyM)R`ll%5&k@O=$G5i7(d1&G z(nkO@1Px@IH=C!tJ$j_x*pyD;CpB?}Hgx^>qmWwT=}*S9S+M!kI$zqWoHc8$Iknoa zJEOfo**6`?r&m`jC%R1Z@s%4*s=9N>Jl3i^7Od~})(J_;c823LZ|;jsVOoKB&DR@u zcJzu0@oh_S6v;~PAaa?EX$#I8Q{OppXFBD&w~vC*>0e+Vr%ApJ_gz&OEJ;rv8kDek zuPeK*mji(Gg&dG>RqFQ6SXS)Nsuj^j zECKu}HKY7RWcs?}DrO7%4(b8a?0q44XX6wW=#k z%Edy3S4D>N&S^kAy|}0@UZeakmVQS9*+({MbG(E(xOJD!YdpoG<#}B!Cnd=RN+t~T z*fd@@tqg5B^cJDvXaB>6s>~m@>l%&jCNo8HYVe^8vQ6?9A-vk>f;74z=p%k4L8v*- zSuKl0%G-R_x9sl^Y-A%KDcF!v)yr1`Tfc2 zo7!R(!AEBs2K7dV_!>nu@&p?7R}Q91(>oC7GbduS#ksH(MgU=@OUAb%ex0niedoNn zxMABOz1W|n_Bh_aUdRSN_6!drRPWG&=!3GFUC-Xvn1DW(aoNv}YPGoEt)w;jqWfhhygh?@pCl=pj0 z)pbAgD<3BbVWK-NK)wwylBC>d^dUUybU8=EAc0Z~Gu;%gU|iGtaF!E`$hMJ@l<>Ha zG$_yZfmMpZ^^y6}?N?%5E`#vB4eKwM{%>AF5ehXhpk<*l|q&2t6eH_YTz{dG3u zcs;Oj`E2BCtGg>amrqNoK8!~4mI1Anb9uE|J;NjtfS3aQch0i#u^aKO+dXo(S&B9^ zi5#!T#(b2^vhd6wj^6}+8pLus;p-SqSZPY>xrs|`ly5tkK%SOm+NS@}z#GrraCg4l zbyAkBd-Ef2cQGrhD6G`zXiAOZK|l%5e_HXmW1<@RP>} zp1MPwqdtqKJ44dg9XViCv(r7E%Y}Qypx#g2?y3cf%xxsPFqLl}BZ-q&w_f9F`qg`y z26!}3&eUk#4KAB+3l-O4Y0|i`&^#%!U{<%N=P2i2w-#poPkersGMkdlhB9<$s*Ea-F@B=&SsQ+eT`RCz8OBay6q`pRkTTIjm z>4jpmsUbyDeC58}KX2~+9->nEwg31@w$_r#0EQcV`Nf$$me=h#AR`uFa-a{F_qw7C z^$xbhuJZBb5T;PBk^6jWFiTs8C*$h|D(vK}E0sr^V51vQsSK*_XwgWn-Db&>*+La( zSoy6wjKmh-5So3Ty~$*uw8KaWr*J)=%{?xs{ej-T+u0BctvDp#q5`QKV#;D{Qp`5~ z9*^ZiS6NpCsnli=FBzCjGAS#i?n#@iCyfEQeWomnHZIR|B|47GY(JCz$!)g~eV-W8 zXSo-E0dnoAzXx;xRAx82TxPSec5|UU#B^=IRUS@(OC~NXPi0Jf?~enDybwWGNCbRe z)A}qN+Jt^unl5`K?f2L=>W%wA81Bg{$SU84KGbS+H+cZETJ>`OewhAmpz4poU-x4v zyF0}jpdk}6{*}K+du21jkEWl*=kzQ8@tGhziu-x*=|d;E$k1!C&l!|jJW&Wf0JV$HS+GR%)GEiH%8YDxyB&d71t6McG~&Ug%1 zXh2^j zx_X90si6(Ag&AL!ib}%3B3{kt7|r80{6H%tWnuMf!T;)yU-X{_$YI_+A#rmB(kRp_ zTH1`u=n|XX$Q^+jMxjuGfVh>)U|i&cJX$(e@y7nfAI37qod=!v9V^shm+o(XFLl;E2;x# zcg~j9mM`1mb@_~gBr+-RUhmUOL3;(m9#8HzTO;u18mt%?()d#bq$M+7A&q_+fHH0# z`0;!uj5~rc(oYpeo8s)8oB?@e4OS_%UN*6%iU<(Zk@xt4igTIPPbZ7MWSD8mxhDmy zE)IE;8M9fe0;HoKtY%wpRaUfxUU83?Sl$Ethig8a7wnN8RK9Q zg36(XC@d#G5#j)1 z7B=CZyoeV`w>0Yt(Gl z>iP{1RqJv4uPtXSmag&rXkSM9_lO@PJ-jTJVN}M_x}HZ%Bn|2qb$i_v)EG`yPLbZl z78fSBpn6->CNN?oPz0hAqXRX$^-MSN{8uNJ0o^N;lZw|_Ng4qXdM_XcYX>`lv2UcY z^SpcmSuL&Ul+++8ibnwh{h_4Q{8y<|daQ~j6DT%uW;;y%NIDni9H!$KD>&7)C>076 z2zq=5FtpjvzkD>UZ?;+VM*+}6UZp<-AtjhL26I&F_1{`*wiZ4Z4Qj$A*==`^nkC3J z4UNyY*!rM%jKU)A-9tNBY?6xS3oZH_VJH3=iKLCNmD3dCv;N+afG&w0zTaNg8v}du z#YGlF0Wdh=Rw>_#p7=EP%Am}%NAc$N887tYf$uV-4p&T}U=FnX^@v=gGkQR~iaWZ< zk&RBf!*{UT8gh%%s%o|v!jrbMkt7D{&KjIQ^QQaPj>AgwT`xjUX_SFYb%MNZho6;4 zu*=JaMwbL{TU-M>R|ui*q#{l+LUBm`vpxI=0KVOZOTBkO+8~t-7Nw(eJ=yr7>OT?? zBmBKDP)jt1Bc-atJnH~AR+P%pxi{ck*?^XsDD~2;R0D}?-kJ-S*3i7>-i{rAwLPTW z^(p+#5-980YB`e5jE*P0Kd)Myf+bVL)C9tabhOvvJSx4|)n z_kwY?Xz$m67SCQDovY!$79pmqnOZ-O#9n2D@Zn^k_;lNO-zqvev( zC)SthS8o>CSrJE0@**ad3836T<>^&-V!Fk8 zNojbBBbM=nht*|Jq{tRp8hQuv`H-!`fAY+KypJ5v(OM4?`jJcf9Ee9gWx2iyRYb4U z)uc8VOZ3@g*5!<+HWLxkjcYVn;7(<)5XczET!4MbrxB@Q#<|XR&$iX%}~1@?CuhS0YNz32&$FI<<(PE*_WC3%m(0VswXPtKVJJ zWy`6e7)@%2&Zi2|zqoX3uRQhIw4!pgzIpqW5?N`gbkgIjTL$se?B|Mj&qZg@6IX>4 zy#I#sy3}`g+G@591x-`Jxx75uDKafc>YNY8&4N~>%7ZKwT?#)4fS+K_`jNHk@=jrC~Cs)aiEPG<4_T$<8LW~GS*2X(4waQm+Y|l9dTTgcyJaC^u zgAixA%eW~6f0eFTZJi7$efR87h@;ZVA~u9>{Kf2V*xiXn+~=f)@N9@E+2k0^B=9p#plS4KXqCl;5($p!3_Z(xj6}vc~`7k*B|o&4LX{9DXr+ z{0^1{{w$k5GE4Y07jkO{{@3aMyp$c( z@R85PwxQ@cI+Ykky_sAhWyLqjI5{PLbkd@<%v*^2M(*TA{7U!0_3ol?houMcx}TT( zJN9YGz`PICsftZjgXmv40RVJrTB`M)_*%?#-M$)xWI1(!DAQ7d{)7cz4MDNwRYkgI zftgo*dMwb85BG9l)l->oX;K-#sb#e>H#W0iDCUKwa&f{mDX`uTOm=dhxO>nBaPhTw z)crtS_c_8j(@ghW9JPwqfT^@S{&YdOGAx4!5*x91?;8AL^ zd1<^wfgKMI?ajL2_JaZq_$BcvNBueUfyC-pw!f%-SMMM5z$yeYXLc5~^>lXfhpVcU z2^@J~(bvzQx*p-tW^ zQKdIAi~`CvyR4tcUzWoThO(c0Z2dGV>`8MP(c_<1ptW!mbUtD=_G!rwtPkcqXa18kqrJ@(uf%U>}azcCy;j_I|Hrfm$W;VRuF#92UHOZam zsKhch#YVW)Xc9xqrIj4(#|}%p>8rhNw|DdoOR>e;kH@oTSbBM0WrJOdojz+S0w}0<%Z=p?n z05B>L4a6qS9}KWNgcy1OuKI>Hbhh!{*k%)(M)P#Wk^?kqNAP#GPjmphC1qN9h93oc zKoo^}ZWj&i5{-m{>W|OUhIjZ#$KVMh?!@|5pz{jkID{K(=@&Gdn}DJe-IZBR*PEp1Yq2;SB2GJG)te{Qv|{J~w}@y3+dqFfV;ODNbs zT4_J?rl*%vy1cR(f`fjCLN#v7R9Ap5mBy~ChD*=)%c!tUgqom}sOI^I$jWF6wkEXL z@6D?trWY%;NX|T7Kg6$6HrK|a`aWfG9&6NpLXlvTIjcfHBK=WvIQ>Za?J5mtE|SNK zVXLnmhNxM15|k$IW*$~DAJ-UFB1 z9KQa>2uH*Qa&*9^ka$RWVR96tM%lgKlQ07L&Qdv#nTH+NSh>m~J&I1Y=n#aeZCc;A z?N=SqdwMlnuCjW!lp#BiPcAe&daHuQ)@0jim=BATJHA3|CR^7>FIu!AKK!77IoJ)i zA9l7930qeta<|xC42vl*up+Tnz14Smmon=SdD~it-2C}f&8zTvmBi-1$f*9E(7X+X ziv_A3sz%tRx*J~5WVp9)e9v{Yq8Kr?0QYv%%W-&~n3=cyIEK5;m@HzRfC^$c7eDm$mg zyU&b8sT0bqQHgI_Qlaky#UZR5peFkoGw2d3T(_vQK9%1j6D+mkSv-Vy`igb0lpqwX zmD=9WWZOpATwJ!1Xq~<%{V|09E#|`!xKNLT?P#8W2tVYO$HQ1A1sE%UJr25p-&=fp zd#AHK(6-+?gT<#pNuX9Uhl!z`Dcb zHrFU2z(Gtc!i#V_gigNZB#x7;S6 z|5f~rN-6vQoQ53m`=qs>p2U;S2MNPX-rt1b9(~_AIw~uj_On@e3d>w$g$P5s_Om!74#5M|z+kY!m%ONg~=2{9B7$CjR`O z04pMbHa8~*-@KCF8;G#m78Ac#pAl@LG8skOTzuQ9m4Cm+sB7~e-iuuqCr&BHsq}+J z{ME63$$fQfqfi8V0)?e>p$1!sDwbAE+Ibx{dg+SRIInY1XC=~TCZT4=S|0}6Ae~(H zdfr^~&-dS$3d&gNg8fFWNMbF!!WaZcJn}T;aVitTY%g*`@se+w9ck512&1wa1Zlm0LB9-skV;qBdWx{!+ym7gV^=3%cZox+{gR*l^gpyn8C&5xg z=|`5ncs)_ElN}s1uQvpKMf)@R34!zMAgecPYf{4Rh2a)#&CPG_+8+;~n zL^kiQ4n(7fV2Txsd|O;DghxhgRsM(u{?hh5chrmX_pBOEX770+ z*!>#tktNb4IYffdb4#QLzBVxymfsqoS+^9+Hf|5$;|8YBzqb28?{&)^z^_D_G5}xX=3TDsLz<( zSkFVoqS56U^uSaOpRnQWjjlm~rc(P_SJ5P11c*5<93OqT%xHtU=tEMlOYQXDqn1(v z9jLPE(bNGoz8BZWMS1Mk!b0!v+YCY zyY4$d^n58QG$|cVf*(c09oapHp&8nez zGLNf0iF7Jv1OZ=N;^jaTaUzQux_X^?5iK+=m4J=>VSjPHGSM2DL_BnB)1wb0z5Bgu zCP}sCl0czVUqe%~(r^#AMG@4u{P4bOQjfg$lv;TJlA09QeCzPV^--}@wHB0}VZ&&2 z4^4}Xv%?+zdq-$AhwK8{-0wak=SroZpGVg?94K2LnjgLXJTR73dV1-ef3gF8wz+V4 zcAm$k)#eIwqsm|bs*e0qDDbDn@Qo;vNYn5ay&Ufhl(pz)K))+8<3ZiAZx7{^u6xD@ zXXl%EEyIo2y@{G(es&04@<>rmisyw8iq0DBdYo*G{Wn%32gNW~CM|-46LyE(H7|{y z+_hYAq`q+npDyIP>MW1-(qc0jj{DWMis-2B7&bVr+Xp`7bygQdNF798bXJ2lPLNxz zkLSN~AUa@!@{(2xRFBD|(vaKjZ&9Vk#^&$G-RFx|b8u>oe)tc{Qyac0j98jZayM2z zTTWWQW%mUJK}u{NGq%@i_J>z=>S22+x?T13(#C`%Zooq4D zd0er#4ql4CYnu5jDI5jDcbnIv|D$4FW%P-qR51Oj(zA-mp!?bF`p~%ytUb^eSVja2 zzU9&%_-MaBBR1znQU2PaiBPYJxxwXvRX8*y^n>P0Pqtdj)wEiXLS9FtCy$iC(h_Xb zj&_Yb>6msysEe13H=M}BGfyax=YiNOUBmcZLUZQjj(oYm8=8BF6>%v z?K4|K!!OVn2FnX}VE58gCN;pda0oI2-{Z`yBs&b>_Hv|aYO`^~5F>Tf=6qS@VkMk9 zVe_qX2&0&$%4%0Scw{B|gh1JyK(tj%XiK7!%HDu#hVfiRD`c<^l*`W6ZQUL?_weLa z0_8$#G4wo-p7f}Nj0T*OcxL|^P`DHcL@VFf{Fx6awwu~tD)$uITV|ljr{Fi9dUoyl zqq~_k>l7(2iqhy=7NH}Dl;`bckM3qj>-A?;PH$UYd-*l+8N35;qYbH0oHqF2KolrQKH8^);k ze*OjA?x1P)Qs3sKbMPi*~wiLP+>CZysJC@qQL^wbiBuAoLl$#dETLt@81%* z5ZBFrODYQll@G{evi~NS)p8!P`1gWEOTo-IlvY8u1M&56=u%iglzz>fJf1|fVDYEt zGP6p7#*0?FO{)C&`#&y*EXs3>!xleRp#+d$9R!kkMF>sW@_p@LCl-oW2MX;d*%f^6 z8(0xdr^~T@4el)bQ0MjI&?T-xg8)Lq^%tgFtQD7jAD%TWeF#=vuJE9zg^D=1fd$c~ z`V=Fo6NR8;^O2E?M`1fzxv2)W_c

6Lj|FRfkQ#{Gxd8uJv;z_|}egO6Um^;=L6W zk0p|d9D%G6coA6g9E! zr>VB@$*q2(s^5Crx^niR!EpL|(c((~sx2Iko{(zR!m=1?uk8zq{x{RqSRAjkMq~@U+lOV%)rtAx5Xl2T zzrAbg+;>P(EY)GV++XbcwAAc)HPqm|N3~6_^=pN_kNwDDQTpFhgPMt@e4D zgfBtIkqMG72)eV_8ZQk&(Q7NC2d52~ySgsd+HXFRnr0d&el`&gl-M6b9Ht(Y*9Nya z!c*@^E2CE_S)Z@bP2YDVi^bBQ9Xo5(?#ACeKn>+p`V}3{5Tc>pl)-{mAHo7&aJ&?j z{hHan6nYR(-xUl&%2^1FeanmF4@%pShVFaytf=?X0N3Uwl~v)xFFN(N2k7GynyAHQ zI-?_)3M=#xu?^G~$HhNke^(T#B%%Z4AQ!Wk=d6T?RJd*Ou{SxD1k5%}@-g@KBOPZ$ zSl(05y}Xq2WG3VQL!eMiV&&Q4AhBkh(}n3(+kDZ5*|v<(Qk&lA<5E(fwdjqIG3d0c z)+YD%p-j{EI00I;1O0X7H6DYmKhQN#*P7L2N-(5-xyeBQ(ygb`<-(Bml~<|K(OQ%q zvXI*ARRy3{2KSXSFbz4qKhGNT`mb0aax6!L|y;_0+ukbS9Rsl!gH4Okj&+pPCcsyq^UNuIG|wiRw5 zLa~Q11&_n&9AfNxSPL!%I|>0N^Zi@~?~gQU8$@T6@7QX$t97%ap z#`h}35OcZ9KDJ?KBbi$ziZL*Qz0sB&PxHnp{Fg5F&e)YxcALh7=vWjosjKAH!Z>Bm zLl-j~lNJ2@Bvj%F%x{nMh5R#8?|W=!xFUrDP_OQ58lnnZ(zn+A;do~ZMAl6;CXrt} zH6E*7dCrS4+t7yNS*0=vJ=8-9?xbI_Diww{c*G|J_Em4wf^_A5xL1jrQNj};c-fci zGkOa=qT1VnL3}?Zj1%dWqS_{Ik>r9or)A~ z1lRx?oyRjbd;#QlPF*5T;{y^`(D_mFq%uF$`IG`0e6_fZg5^et|JLz96Rs70>k$*B zd|`{%0<2OTs1e5%YT)$7nXM2v*ROXjrwkR(A&JE{;Ao(zG(&>7%^n}kA;eRMo?x8S z?8b=khpY;HX%PbtO3H?3nYbm}$FUo zj-K_-F^_`zYUb^@b`{jB!B}9Z_~-cBw)R@{(B&aU>z8&BidV2OVOu2Mh52<|ho>X5 zr0&JUW_J`w?{2aUYsK#!_KIPjvH4*9L>>dWU@nN%rbrrWF2>vxT)-`P0lF#)PBLG` zbNUcX^@gw0Fd1!evzr=Q=OSSKmQK*7_kne?G=0sk6|LXkaFx9h*w7)BND?;|VzwF` z=hat}<@HT>SsW#y8#LGn$<%+Mmt)Kwjee7c9m~|~hlZeGM9PXoSyx5j7=qrd|C-9u zC8zkJnXR0V`KcH0N^6TqRV&GMTLtPcgA-DG8cX8+g62WITEmC5BPpv)lI^eFfa(5`5MDRl1_y1G_>Q=PdT&)S9Lr{duV=An=K5QV z_X_W(*??N!{Q>+cU{DIWA>EL?#tJ<|QBqY%rw-)Aht&0_A1CKjw(oZ2sj!}ciy^IN zZgb!EU-=nWx&2UqNs$=SWl*A*QbncJY@KY0CeXU`E#+Gi{Wq?$`q{A$*926$?tsh& z0E8?FlzqiZwu>xVyUrch?ZyIqBZty?1~6+vl7+Mt)@EM>6uh>s@O;bIxZz zinvAUV&Be-;sgWbOE$#UNcr%3<+BV9n9GaomK&@*m(^_ZXfg$zNCoaOdO2eb?;<8n zT2M|;Cya~-QdXvu3E*FzgF1Id7M`Px0tu`cW7_W2JyE6-@H;8cIToA5YsM)WnS|Vb zN#+&*ba6a^de%NrGPN7aEH6XMf1?m5>r=&Jc!d!)D1&!`9f=~IG{pjH=S-x zd5PYqYdc-Zymuw*q(X5|WmAehinstCkA-2Z7d*>u4RP>+SP_DT1$RzE3JWqCj6qg) zH!pV;Ng1;9*wKwPnkL}=Gh;p60 zdcge;2PEwJ7Lu0Uq8#`hZ-^di@#^E zQ$ToQoTFxbXJ`8yuGL@=K2Km)ZqleagdfK#^{_r&`uR9Kv@0}w!S0-Y4s-|A2a!iI+%cs~Y(q785ohrAV6ae;)d8!QHO_z2IYCf@@2bV%Mr&>MW`^u8Ff!~6?W*@ads%pE43;8c~T&dWpKg(#so z+D?GRX}M+_L2|i{c#OjFTHceH*;BmV#W=xak?`4;bfs|*q4T2?!a}Wir1?a?!NjDW z{nj4pTcXpV5t?7J>4a(wiG6?kzEU}C1ad7_{72?w?U0&PKg_w!c`r(+Z%iGLpuzyg zm?7ZVUY*M0hd#+1Jp?fi4Z%G5eMg}6jgt`WhKXPgxynB**3{rPp}s2Nt^n=Ge?4(S z>FQW{v8za;oRks)pzUZ{eb@cvdasqFF~;g=alP}-`$A02cSvL_Ex6XtObomES=i1Y z!+H@w!PW&!=d^X~`92h!dpFg~Wj14omcVmZKvRWvuZB?rnfFqgNmQ*?`BDze(k5LO z>E+_d@*-)<95~?*wV(l;ZIuGF4{-8Njl+Cm^sakgUK3j8b~qk{TXn*+i>>J990NH> zyAlS&-mH_6Q??34GpxYp_EgXGm}*AGXx*+1{74gTTdh1WE{SB=hy zYU+)iyxdvbQ+hJ9I4al)YoI}`3CX{)Ou#Pi^j*noquzc&;Yeaf*g}wBzGkxUdtNUR zJ&!h3#3wAcwTXHw+D<#rSwGM-$F4Xxv#zdg^7~91P_VOsdwVI=N6+ySrwIfeFBJzO6@m)mceGDmuM|?M))jef` z6E^k@;y)r?=1*W%W@Cn3!RYS4PAC9w=^Pe@mHBWK)Y;Yb5y+2A&`Auapyh}yhpv-%PNZGf|ISdUcH=x?H9X7ck4&ObMzQCzw?I|m0OLA3l@{dg^u)%7!bX{(GfYE)Onu|F ze2oS+dNCAulX*w)gLg*-33WZf4@f*C`)bo*^c|FC2GMSwbeZAIH3x#!q-OFl4) zP4Q_#ht2u`M0>H8#wC1IvCoLvDhPV@_7r$5dXaFPm@gWF!G-)4;4c|X9QJsBoo|`= z`=()Z3=sRNP7Lvx#DDA~f?tWQv<`G{FnpHR6+G+*eR_F`H9$lyZLN&}=j|F?2}p$W zmRM?ZMnw_ze9-+#j->p}3(PaSDmhk3vJ8YWdz`oI1o zIU<(%<*tm`yF#DB7+Fk3Q!|AhZSZ=TLFQ{ofU}Go z@oZ6xN6kkvooBdJ%%J@uv9K0y|Hb2-{6k>$7rIBn1&irk{;4tt6S%%>u6HrBjYFdt zb>>32IoE2V?|j^VmfQ(SW^vd)3LE~MHxD0gFYYJAhiDRy~z-jm2qTj4OT zqr>eXbzSLZ-yI{wDOQQbd3v-@RWA{SHbxu6Y9%MM)aa>PcRMark{HWenScq`i)WK~ zRa~3Te8bWYvf{UhKVl!$97IlqxOUC5s$JMoZx-^8mB5o>eN(^`**y z!cSC2vT_^;?r-`08#Z4BwM@~TICww1zMmD+%QN2NaV1@N)5?=ai-jDykaD-^%=Tm zue*ZgAGK#i^dtt=n;A)0nvAf&N1cUwI{&&MQf6Vg{>LWyYzAdS$ZxD=Zq5t*{2HSJ z^+dse?&M3xMo;c58;jg=kvH$4Ru1qW<=le$Ccbw+klvh7i@mmk;+UochAHJp%|1P+ zHeBC4o(vyT9Uis(pO)sD4b7S(!*Y(9S)h`|2(Fe!o zd_G#JKpWas;L6Gj8d8X3=AKgb%i<`He2r*%k7n z(d3@wvMjj4sa@jBj)d~caPur^uGX@|0%S+v}(YIOQ%_T&fM*K*zuZ%@K71W+gv+!2K0p(?qP zxU6}K>ipjJ&(hgj=1&%OzhtrjO%QOqrGcM%bL^@mj%&2SpMb#sAH9dvr>5B4%~o&X z3#0-xDvjDw-$a_K9X#GjuV#zuq(?pJYOg<(Y?@Q#rF-&#Ecy>b3g`_-=L{&bd}HKz z&huQ)qYD+zyJ}A$DAHekb^gp+6m-?W1gPPp8x3jmk0G6g;;&{j z_CyDGiw}{@jy|M39U=g8lPug0?wD7$nh`Le%kJfLMP$9ltTH{0 zHwhtE!okEIO0^QAhc|Y|{$ksOD5u(9*Y)=QS)u>`WPAT-M1Kawv*!jexoqG}AcTS6IN;YtL{fU0SiA?GnV$73*pn4)DjmsA`a90dQ|XHUQM z=vE{rtJrdeqf1GPp|VFkMb-;u`ZR zh*fFMc^u7_@W|kJFQ;o@VXH$Qd^%FZoi4_tFkaYcirgMo5o{t zynOHG>MAS)+gNG5Qry%n&jqm?j|3UHiX`gegAJa&uBm~S=v@jS)5PtT@GYt)_^UpT z5c5Ip0cHAIiPXlr+YwC)O>FkI5&X`Ye>Xq=3wZc1P|@BbMedqWFLCh&DiLAxWCf@# z^Ge%P9U@z5GAfC_Iv2k`M5>a+>=4%{JJg}*81uBT+%BjR+_3S&UX98yI{S07+WKPG zJ37@jEWY4-dT0Q$Y?7q^hKYPCT0Ar5DGdNTuDbt8`dQ^GOr6LaL(8XUnpyv}CcJHd zHp&gJ8x|U)HPYD}S;Od?M=eDXFOAui>X~;k-~6iKO)@K@ee{F4c>fCO^F5oH4Z*2{ zc`@qL((QfjT#W-R+`kn` zr+*=UTb>;(%dFO*Q2!^{A&=1bSz2~j>4X%9MR@m`T>`y&g>Qxye!d5Gb&_WWRx)ek z@mq4)#4I645P^3>t?FI(e+-Cch>1vgb<4iarn+}9zUDGcyd-*|!S8Ms<3WhVZ|LW> zOK7@mhz}p8MpuFlM_DFaA{sXivdfE?9}#(w*Xj=Vu?~kA(#?_CABvS{HXkM|`09tQyit^i9ERU%l6T zp$)r}^`UMVaxF$Cec^880OJx|)VCngeT7#JKP ztu5oD2dO?gHx%a!txqi80`7UJw`D)<#HQ?*)6RKv1KYpattP-n3`VCOj@X!Qi3R#l%CZUlZ*j$18Ep=tMHJ>@8f#50 z;@YCCLE{xwQ_rTFUPNJx$g0D6+#p}oR+m7OWG)3avtd@SVQor}KkSFQ2}F9f=XFm< zhj8HnyMZApdZsW^JBbAK*BB*j*^NZ(gAsyMiyO1coaK6J-}(={MFM;-N8h(nFU7*0 z31}{y1?31WYYA!s?gzi= z5a`cnLTLU{kZ?aPDZ;l3V0&&2gfcT2p03adxRGRUWz;HMVUYEkey=-^d4!${tpClo zd>BHK@n*xsaOu;~hx)Z(2^87(g~=**l{D|g-;FHhh|8gRw-H2n2mVf^N<9R=N-?;hn)HGcVOh#see_ALN zJJUGdxUz0|_Vz`_2?bf1TO`w2piSus8TCZl*yQV#*SdHTFA9{`2sS=Jf|ZHgKggX8 z6JxlF)gEDU-S)xW&`8>(y#M`}k*Q-351!)w9FxPUXIQ%{!7IT^_lns@1hM|(5?cBWF^`RK)+lq=m74S z)<{3N3mMgkD)o4i{;7q`VWT%;srpJFri|p67>;efoS_y;1k^6eVd$77?Y3aMYPB|q zW1g(S+n<)HiyU34j1RBtJQHfMuXt4*b;&B5?8#s^GguS!{o4E+F-Ob(gi%_X#eqA~ z!_((H5wH7B90W#m;%dBrgJc1u?aaF$OlW?NqfO0l7J{mLNBnvB#s*B1D+NkXxl$r_oDWxEl znheM-)1}?L2612j0>WnmX)$5d-KZvi4Zj=21H7TrT79%x#s6`_d<*&9gPA6xPuMY| zezujx-;BvF@s>3!x^{%=*m~###(q?bHY6oaj^`{7I@eG;v*7_B6pA;@YBF(ℑHG zaJ76dq%yUcuQm2Mt&e)?{6ro!ZnY%`FFn$f##DSytZaa8OG+HalQ^qUcO>QI<@Mwq zL48-x3kM@S64B{?lPYHe%y=*i?rNhqaI1;M4n(INg!dE-(Z68x2*5m92xJ0vo{b7t z)zc!UUAvrBK$V)rX}zZ}y>9O0hvVVVi4pe8ch2Jt*A=Fm1?O)^PXnuXVKl2 zt_iybM1FW_Qzd~ZQU1fAv(ytbN-Us+bOg;UXQUUj(`v;+akY+gl=Dh-vs-QtZC!?g zTOX>tEUd9Pp3UI=p52yXPCu%lq_IrKbnK-Gmv*{yA2$croum5M6?cE@=J)Vz zOwy&E)q6K>@IJh>DqhH;Q6OC~V~TLEush^!!0cf)U&!8@Dgcl!HQ-bZ@Hw9#gna*w z0DJ4_W~kLB>K2kL0w?VK+r@>1j4!UDmF_l*#G6KYVn!keU(C5y!-itMU1Wz;T+d1r zx-V7^y>GN9<05GdY+jKbFEF*8rpp+e_F*%46K<*)U31U!E$em8%D`r>=B@RfDrZs2 z*pvlcp|9pdAT5UP#ruPx``yv0dJibY;I3jotNf~HXMN@AikVa8*8PE0`gGUDR9%v& zdj3~syXlWt7YooOOpyn?gj0K0A}R!JLPZS+)n0*spqAh3hXdN{%#>2bh!vg~W3ilS z{ns4)gFXOb4r1q8j`g6GUBLIm-8C+nap!Tg(#QRmT_Usi^J#q#E=cK^y4{gr`pzpo zVgWatL6?{@wXPvb zuc85_sJ=bYp~CdlY$gMq2zG~|jxgSc#kLyOmysy-Jzz?gonpRaTbYq<1w|X9Zp^i~ zeZ=nVcwz^hZkA@I(5Am)bUi&n_XhejHINa$(gE+syZ>OB_kHzJ8(;Eo1hM?>M)w)*tImk{}JIiz%I88L1%xF##ALt(8(H@$UXQ9XIb@(l@2Ki zrJR+>M#7aofJ1s?P^g@A2zv7A;Hm!Y>X1`KK0iCtSa>z5?Hp^dZT)U8T6@6maZlQd znyxa)WZ#+G^y+cto4RcRUn~?w-7Z9r4}~%h&LK|qqju|I5*Ft7=az?l*(*u1(x9XeWPdo^r*d=sVGac6R1)vld|dbp7uvS zJ%CU*lXxQMS-(&>x;m`FeUZQ_`;byeVfnyFOwP*S9SZ&DT|5Wg50FypudvqTx40i> z7!;Yx(}&nM?Bsg?RP&Hi7Nc4ARC(lvVY*DJYJbn6fG}}i=Bp_Y?1Eo=37(@4s%~C;M4pnE3;@Q zI64XVe4$&RF2;g+^nL}wB|-@I!sh-Ie$LWm&%8-cC-X72M;9)4spV4OqEX#zm& zRm8oXQMg^^G??;BUX8S+9+!U?<)=kK#DiG2Om%3rziQo2KLtK5a;o^!9Bp-R`?827 z@5lTUA4n{9b)cf*6B67w5AV9@uADeg8k%hcEiYp{c#@Oyx$$6%=X2n4RCD~6_L`@Y~{N~$f#MPE$>DM z4!)hy)ofzq4Je5I9q9U$5@60Z0OguCRoTG5M9aB6<&Ku0QturVuj& zX>S;-_qwkYv}_}{ROjL;@!k|rSSn~+HuZA5Xj==McU6v~F5lvdS)Es_o7|uK8WJ#vJHJzVbS))A{wJjxQz@?1=cisk*NgG)pR3h0W_jYGW~`w=*%0d zUWW(``(-;1=f1(z2;TZ1Pt{+r0z*=L8zuKJ(*t6~PrU(McKBmb;X+A+kc8@PKt`{M zrL4|@s7zIyxMgqJ^pWLxUHih3r;MndEd5$W`@8ka)N|Q?d(_CQ;%OtTL47mBWG>UX z__S?*mVUukBdl(R%ozBALVrWkzZPBVw*1krxdF{SmPfQ1Twk#fa|R+r{X=6|(?q1- zv3x&4)e02H;_=5^9s2><2ud5CBB_Zel-xaNU~}74@n@G@2;7^gj|Fhi<4XkylO?vh zrwd&jzD>h%+WRA#awRVQZusZB)MkEI8%BZsn9zIe7=7`8FR5a4JQljc@g!!a{_d}7 zqjyc2ssxmT#r0*~vc&HJhpe*A;fHg9NF=v?!c>gK_1`>furN()py_xLnb-sqQqBE3 zM3L+wkgH8~>kz+_P0hn+u1;{UfrTnBdtsR%p2Bz~$*JDw3P5&W**#HWkJT&$F+kM* zVnN{NQ08E$6g^f+O8PK+tqnWs;Q=D1n|e21C2d#{k&Z60_&1)&j!r!Rf0Yl>0i)PY zHMQ=FW3_<%`#MHbG1|+e8cy7N)%W_PA9kDSe|PWoixIS?Mj|AbbA@}+nfkR>Rq!73I*wI z07=B$F+68Y**BUVR9bq?9%Ss>w>gDvUHx|l67ju4k3sHdKFeICtq&)%M9|aB4VUuG zgR0zt_QX)y-1xe;B{6`^)xJp1J~k}~@dR1~u;~e6eGG)Tv-}pZ_k$;ya-^CsNH*f{G;dVoL!B4UKeZwsXRT@0%p$-t2BJpdiP}N z4{*$1+{!hy1D9xT-~Yq`nK;FGQY0tx@49Vk);ZV8h@BM)9+)S2-$yU6yYqb1hw`LhSxy6$K9dp8vX?;LmQy zg3yjs%HBNKG~SQ(APSVsnv*WpESrU0lja)emPDMw_vANsG5g@hO4!tF1#gn3CIG^R zPaoHwdBPNC0(J*nDq8>I%z75m<@^<#S!Qfo(rWqf7(DvxZr$UkVICC9lv_*F9!yApyev#8pizQcK< z%+4a?!>KO@8>UO9#r++&my2klT3E_yT2j5YLyFiA22$sQ-aopn#cI@M9yC)KLnMu5{+uhd7VloC`=y$qmF4`#pKL_J)vZ1=Iw_Q^UL#E;YO4tw zw}f<*(6M9LRA{B(OXu@GuT{orgNn^=+$8`Di~_q?M_e}3PopV*L!faaP@F3&t9d37 z9X)ec#QOUKdE;beWCt z_Of)D(<*Dkk3UJXA)^cp?p@u>T^mg4>^_u>%n&0pNtAcV{3DV+b`+Y-*R-z8*Bv$sMkv# zKX3r!sHz`t#s)h#yTaakJeNp%7_ZhKkyfr#kBKhdi2TRqSoa#FX(MMyf41H+^|Sqj zY-?Y81DCDf61gZT}u0-2d2#z8G@->mRK>-j;)vQ2+egMoTYXq+XJN<#+zw*yQRd~7T&!DwC7nzf4*OTdyLLD*UNz`v^kl+R zrsv#XfGi7~>QFr{ifF%3jfDtGcRO)Ah2O-v0JrJoia&$EUxrjdKRcX~p|+=`tCZb$ zGwrCX{@6-04K>RvLumFe7wArk9UP_f1|_=~L%`yIY5bTyp}?S>iJ4^28F3nSer#lo zAQScZC1We*(da)(MF19JRfC9bM#_Ypxn{Tt1xeS(>mDxgj`Igx_$uoYNn zNRQc{Ya^A%abFmO!2h)wwiupVQsyC@Ww!jCO{eqVK%k8&uT<;1yVLLeLgXmKZDnQRUTf_0#O*}qwqR-R5DVNR(VQ8IcO&UP;JqbVrO$0PL4SV5X zOrZ5Fqu;o*_fP`$F{{t`WuQq$zhM3XIY5()DaKYsn$m-G`44I`i38)M0)(r!uuTAW zL>pZ@ibM!lQ5Ft^xeU}~NuPk89-P?~3tj_I!XqSI&H+~P#TMvB0(nmR_D0&>8!2K^ z{b_gmrikd*LvOUQy&5bT0H8m|8HE9gGDe%dH@u`)cJ261J7U-zDmMl8IZS)1c#{IfR zxutp7y9!lv9h(uo_Cb~ZZD3|o7P@sJ;J&R;&)QIXsZ27aa}V^CJ=uJjPpGX=c;);f zuy4kG3{RHIN_U8uptPLN_xH+}WMiuK`f=k2_u(WIKNl3c*DeJh%6F`6d$i?Gb znDkS|z5r)P5Fb}P_eQn~Ys z0|^Yy<>5_wA|>k)wLqtuSgfH}@kNGBHhf-v0H6oA5W&zSL{mg;o_+c4FGKLQp0*%L zHNELspjCEptJPJnF~PsrV}FsOHR2ucM>n?KTu;xX38sx(b9HJJl@H`};_We4q=5mP z-i0?m`M=qCiWGUdr7c$$uy>mvTg~I&viZ`@QfLQyh{Ro(t^ZJ04qZ zHD@!%^=4k_D5lN&Ruk@SG&z{B;l;}6caq5WL`UmE;iv1^A8vc_m23dnbJ9Jo+T$up3@*b<1ElZvt+&5xw78;t}H;@!Txwu zba;j4HasE-e`+!`R80_skGHTj9cB35YGxkf1_19<6LaHyz+3jf3MAU-N#+p2_-2KY z{t~yh^(TKiy`xq1Y~uZBx)P`+!gH}Y{I0@3)B;}e!zPc-leJOANB#|*%jRt8%K6#O`q~ilql{Z6_dq=JP=`nNZQ~*W+|O>xb~-tuKCzGS2BC3q7B5RAw?D zG}=!^U2}@7K#K6Hn(P;K5;zq)NV^^azbUh}Mpab+lIK!-EjZt=orEq~YcZ%%Fchy& z`i+*fN*g}6B@$=1h;`5C)ZckOJ51;~>3Z5E&0e#+z6bA^_*kt6ZqYguvyC47Rg8F{kmJL0MjrB1#U2HbPRW;9myyqQ}`qpF?pqIhOOtf7^t`xq83fS+1p zVgr0hWAeZof6tL8K!f#lN?7EM3m1A6E`ZljvZCS5i$Y9z)#TQN^~ z)|3%AuU3v1aj;S{yU9P`_mCfWI&-h3b%tM8iOMqkyrM?z=HbjW5Zi3fG85Pq<*0BvE zaeS)LK@@1}&1;L3_EyKj#>i_8u!uZ^Rv_A^56PLh*DQ^DmA8lxCz)jepO(V%X=a;{jHulM`F!L~jwt)$?8_+k3z}V(hvoLq{kPQ>tvvK?R#`(hYe)JCu z{lU6OgBQE1pEpzsvuK{Ar zIx!)e>+&1S!54p>ZfnPs6&_M8@0seewsznUX|Ok9(=t$NF}j3ib`#FyIqdpkEs1AT zj$D=)#4#0;y0bT_b9e}-DNUS*x~hwO0KK;63$`J`jUXF?e$zIN7AlV;y8A*j{-@X@ z6^GBxBB8hEw{MM@_3K__fNXPuSMZRU#p3qS&!pO-F=Ei(Zx%1eSAOQp0#mz@S|4j@ zkck|6GKqd5J8ti>CLiDVl`W-6edxMdkJH@O_L(PjQ(t4d6nhhTxssb}gND@xf%4g7!%w&5;MT0h^npg^2jGA*NEaKxY4br=P%K#<+%9UquaG-iBxUe zWBS$Gw5CP(kgv(;F9e9D}w$`EYtW6u!&an<@?6&tG*}R-N4Ovl=eM z9%%qkWU6gd!#}7p5QfEKir?!)((G>*G`nh9p8kA7Gr?H=WiZc(?Z5_K`QnoEOAcp~ znW1VB^bdqQpXF!OsjVEV0J~FZ4P|4P%ftCZoSt&?^hAEP>HJMh^v06AmImoA7jB;W zy)RYoxFWN6yq}H}93eUf3w2YWmp5W}R=RbKw`@L_Z0tog-1aJ11L-F0{oM0!r>}Nv z`fnzjo!ZU^dJ}F}zjCf3qlBQuU0^Uk_@@UcMR!wgDuspV)c@z=ZC_L*H_OGQQ>kH= zxVT;NWcxT}CNjmm`c3NE^|jB%Yg|F|@Q0ou%5q6>;*~lQ*}^FQ=f?^^2B7&Ox|vV3 z;cjDF=_B7YX)x}2QP*z2;Pg#}fUQmwV^*`Ka*l@P!xHEXa~5j0Ba?pRb(GUUBtU4u zo5D0>Er;A%zv-=X^Otl^gXv|-Kvq`A3mkxX- z+jcorX?UVazM3DzEX?yEcg<%5VJO39?>%BD@VqC2d zhI=b@<_-oEEz~$209^j;OY3p6KRcaz{bQnR^;67tm2o1bQlpk`eNTMUh9k+)j$H3! zQ{6=FF1ayw?o-@Sn@9me|8)t2?cimhRQd}zMHhcK5(BU9D%Qb4b8RkV1kA}x;D0sw zv7|WZH$tClkM3c=0kvO(l*de~SwJL6F^W*Rv z^~S*);rdzTBNEY}f;$uXsV%Js z6HVKDCYv*M^K-cd32LD&JT-|hjM~W3TQDa}S$MBjIZba^qUQ7Vzu9o&d5Z$9|s zcMpl-2c_n30f;Zz+&b+K-zNoFd#s9rY*#lyrfPD^tFSi+4xe;BAVRmDiX5)G0o4x= zd>_$z(!{G`$t@hI^u{tz*{?!TD$#zEcwunq)ee~|is_667Ek?QcP-Tp-fmMA&cxGQ zJK8YZfm>|$o)Lz26-7I{2gVpC}h>|1b{Rl~^5~9)BJt<$JfQnEAQVQ7x{> z+#-X&8dD|s+CKus;Q7dlbC^8Rr`+I*D@;^Bu zT{6&V<-U@M=;t+CV!gfhe`yJXA_+B&YUZP8?$!{CaXv8tmJ@xh_=DhaCe{F->&=!= ziEzsR4XnI8y04R=@lt!I(%W=M9Z6p3iE#7PMf=NJW>ve3IGw(inul1_S2973WsPg2 zBSdqjyWTqIO$ct7n9gLpfF@vRZ@D}}oXi;&(0mo$=-wx)4f6y$xBk3 zVPjBSpNia%-8r5|sKSQ|o@EyK8(rJrTV0I(<^c`dG4X>`NHOvL8sF7c!~a6lF%2E9 zyu9+%CBibP;Ov|eODc^TkdfJ)`51RZnEOxf^n(-P2z9Bo77W2xS;cM^Wy7N}Pv0iI zjRozT>`I7%LTxGncoD-CK8G9pv|e#^l>`{KA9l1$1*kRK8FifQOB&-DsfcFSP8DKlhR?OCLv3tUPPkhQ_xsFCPNUK0y<26Hy_(<4jSk2JRl96`wRU+~c_xeXhlU=7%_zx^|6yVdyl;h(mW>aj*IdjpK*Zwo2OpVyDvccVJbV z{cfkb;nfhp@|`?wdfl&&Dw~5rPNxDM(?5T+lJ-$$@w77k0Jkl@#QhHUR&tr#!fmb# z1rZwDE`dC4_IH5|F(y`=9EcSke?sTF4O(C~VWt|5S^}EqsHG{u5mo(Gdpx-7Sz^V| zL~JcP43TCKRZ<@wquJ?hf0zQ#-qjLFwE$;-XYW~^^~0%0gg1=TWqj%dHJRU)^yzKP;SW$=Ziz9*U~Hy|6Wg)sbP zV?F^Mv}6>Cye8Xf@Z*~?-N|YL(|SQyV0Wm^(x&J8+%bKAQzqBj2hE|}+h(_WJa}Cu zeB~Xq4q$olDQpe29z64fULZ<*;uwWQCk zCAl+6np#Yn^_Cc+bgE_Zy1s|1+kInd^6r?Wf3h?nG$51|iJG;%QoVpE&6S9D=6x%S zhh~mVJg#@TW&cEf_3nF_l>VmwOM&nDR6Yv-9Mb&vVvwH~z;(}dBE@f;03n9}A#pOORz*8jcB+rKXS^S93k7{_ywaNj9q_Ev>1uT!4pbIlMw zfYQS^p+F%b!b~CUlO(NK!q6p{M`if7>pO~6JbwWX1X!j>N8yp(H!4NCdTqt=IsRP< zwXwYhso^n{8qs;*(#7H6CRZv%HtAE@K-r@1S=(>7KWUyvjp5uNSK&_X*}uL@2p-#) zF6knvgcp5RP*xj8y93Tro;cuR4SB|#mEr@Z(VGO-l~j8f{x3{PTq<1r^m?s8wA@b!SFy~nPF^)~rm+IkzVkRk>>q)fxMq;!3!Ct6t>28d1)In4R zvy%csC+e!d! zR#9mQS(@(>m7$7LtE}T&bvWV}xIe@mLZ_i_1DCDcN;Uhd#(*eu^ZmA-I zH_eR9zk2-&2BH~wSyTw%atQa|xiY4JsQsVG{jv3jyOP)qaxE^-hKyx`4dkKef|IfSq&rX?MlqT?GLpsSVe)qmdT{*fh<2Lldt0~7Lb zo&=lm50gg4vBB>OZKR~IdM};{M}+pno)YjPSW#iFD?h|lBm~)$f}KdGpbQ-MH`;I0 zYKJ3>D0hB*q{^}YA;Y7E#XkXtn=VMB=de$1;h^%y>cn`HqU_SEt~@AM=E-5ndsDha z?+lr2xk(okxxWA}Cn;widCo#Z1uu77Q(%XR8)}P0@D`HpORMBckpngN8ncNgJ8+DR zrt@K)4XDp5X&9SJfYto&Qy>y@`a>HI%Jm7Dx5aarY=}rz3krq#_++kl*gV_TEHm(&THjEcJUKt z!J1^ugxe1DL&G`82?|oap?`HeRW;qlpr-}(X)V?%kI!A=LvMs@W&~L}zIa1(2^M98 zoPus+fRO90;rP<>O^YP%&ryP_nAl?OnZsv%!sG$s{1H+o;Qjj-?vUBUJi>Sv>eFnh zoHp1`J-z3K!yV**>qKehL;8bHe-GA?E1f&Jad;4U0AJH8Db9r!u7`2bR*2t3X@DQe zkV8I;kKnUKk*7arO*8QhdD6IcN>{4phyQ3Ny@OhFtZ2iHM zF$nZm^7EgSTuc9`UAYMzN||qo2guz&)J}!gt#bM(J_Wq`%c9Ec$b~V99&8TJG?_mS zX&Jv$Y?mnbl*Zq&+-uW&@WKV|_Xu>FJ;|SF$CcP3S!t!_BD)lQGyo~Fs$heP)@hBn za=adY?5*a}eT~r2aawacN~k>lCHrn)hiS-6e5_>g?XQ7OXmW%;Z$ZYj>M(ne^vsm*S7UC#29 zc7B;I+4M0R?REN?G68_G8ninv{znz{w~*47{a=)Q1yo#Fwsx=p!6mqRa6*8>2~O}} zAy{zNLaK0g34!1eAi>?;-3d-`cMWd;P4~=uGd=x!`e(6N+*-G)&ONgCx4&%^vWKD! zDow1zcOEmvR|WXM{_xv)o+v@*DLExn?(qD!_*QTTjzqH|c!P0YlJ>#kwzqRH*l-6r z+VRA#%XuJHUUiy$;hp4irGCMdGE`7XTikI?%8L9%p#EISs)Wi*Rn$xTMB`St+|BSEAscgrxJEu4AbTb5kFi4ExtjERB!1gWP~V8*9LvL(3k!7`4ax07mG zB~+r7db2MC9eSB^LSX|0rxTV0xO-}{@R$TkmA*Fun*&`CFO`^s@CDZDi8!D;mt0ER zif?ie&%Ov_7~J4L51%e~CEep1#J2r#PLvIRgBg0IwWgmlE<4<0@J|hQhAXy(P4RE~ zyQ7*~UWsyC7fl~lOgN@2RWRmnDFnb2h3en8*4ssGgddsgcao)B{)!o6e$2pwHO9}E zgfE?`Uh_<-daw`Mv6<0#SL_76@#&F~;twtUejh0-de8mWHknQ*quuFX?`uk;Q86pFC!C)`+P7Ml zj7I4C08MqL$RGC^x6nLFUc|G?p0fDpc2k(6bb04qJN3|_U$-9ni#VSg3Np42Rq%$+({}FJY{cB9X$A3kZ=$RHpuc`?eGpw)Gjr8~PJH6>rsgz`3RSCg#I+F7&3M0@ z>WQXR`ugEeBCCBRc%c#BtP!U{bp*i!?aBi~l10$0U9ccnEhpC%UU*12U*CE`M~8X* z7&s~vus()cwuwVHJ271y=?(5JB`T4Z(@i5ea)nmBubIK~yMOw%1{g!<1*7v%% zE0?y6D;fTUibTTPRy=Oo_Eg9cihty3|52C1D@NqvPQrezpH{nW`_@)rpbl741My`J zk}iXZ*oa^Ia9M}u3uQ9#^EXlG)Kj7OdSChw<}By9QyR}(wQVd;I>(iYdn6d%{&3!Z z+5G_RCE3%;?>q!SHqaIHnl1a^(w?`Dfq4esQD#JO*mo%x;)XcY{nQotn3w#&4#aAevjkvZ z$S@1rV_P+(O7?1EO;MW8NcXhUHGyU}Z~1USO%-h#{j7Zg`i~Uk!IqEpr%0Yh$_Pu` zjKNQ7Ir}Rp8WA5}&oZpqt_cBG-186rWh)cCQniT~#B}7oe`^IE*l>^j-f#%Z zUcM0?z-%tEz#7&!vek`}Q!5Yj1^+OjI=n&5u-r)bWRKJPnH$S>sy|E{`zc@IgcE5C za{5F+HO2_L?FY2IIc4|+)Kt^VrIZD;_jH_Z!;lIM>n6yC5&SoGKt(G{+doX0=;imX#`a*ZwtXW%%1rFquv9%o_MP zkPloywI7dDHg@8lm3GlNx+Ti>DXF$|*A#rh+4&T((I@15h(~l^ zIS={CRQiC1qJMFNlW=SO+^l&?hrW1KY_>FG4CZj;4vFb9CB7GC5B0}Pu>uhH%?{{h z9Mhc2P9R6(czU?vSYZVj4W&dGK7T{*MKY+LG7)kh#ic=XLi1p-sy?{jh@E!IkNynm z-#iqW39Vx`VI{o{T5?`Jwf3S$^l3qrm`W?61j)vCa{*e)a z2*J*xeD-KYfBbTuBYYS7jk709yep|!Ef?P1Bhv7CKJSU&3>Q z(9c~XP4&yGi7~x8UtjgB=7j-S_^ay~9#@u&xGFZ9z8V&`s@5~l0DHZ4asa=~bwzG} z3Gbo?O&;g{ieEOvyK-3V1q0b@SLQ?xyo@MOITwIsnz$<^eHP0QB$bizPQdzA=)lc| zm8PF{L0qO?V!w1cFa2gXu|XpGXW~k6J0;91=A;my{~wj9PiC#%1uo;1x$mT`iD3sh zIGW1(DbVomSt58N=%~-|oscb~0y)1~H_#tS;$Rc#v-!?^z6FfvR#Vv3e?1@ zCD&1n806nRNn5Be!sX}=Am$OP^-{-4G}70Ul+HYU;#~0;4QCWeqCyRtfzK}U&bo0F zjbv?3?LG@o`tbZJXvFig+fMTjnv8?)*cPePqlZmIT8V=Du1Z9F>7GHUk>rhfu^)_% z*6OC3@T;9uwA0`(_0TRVz)ePqKXOA+<$~wwLC}POOz@fHeRcZE1W<&W{aT|# zQeq3J!O?rwfw}zHiqx0)-MDjgshbF<*75wNm$ChH=0Q(ObNA}CkI%bT*0|3;7l+@F zd-&o|$=PZ*kFKM6(yX5Hh|wpW0aO%~Oggj!lXxHpgy@pe`Qo?4nk*^%fYH*_81ykNE`;0qeHT{QrctnplT z^aIWgO0tk_*&T~>yN5yKJuaQ!T^)E)OE0@ELD2q65EQ~z99C04_p#(hJIAM%vV^DZ zfc{33YSq-a<+7#u(?|Q`lb&OPC}B&A%&)e9Dh--ptk-GPWP7jY!ZpKtO06?okF!sP ztwMjQbg62w^sdSzGiW}m?Q)Jk9@*&n0b=f~%F4pgPSN>D1AT)GS2Q+iu(IEmHMK%M z-vrkJUYY(-6u!QEil7I0DdfKgPdkuAUFFiIHRFY)Ga$bW^4HmlQS1Oc)8Au%fi-1pyHUr_`qzwERy}Y2D24vF?#S;5<(G}1p|GrMXIIPXyD>j| zZth>!gLE{pwMv9S+4dUi23~YvG9^{#grU=uQ3)SkgBo?I*&eON2B`NzJbeeWoMVT7 zdet%8)in_$xlN9Z?HRE`$3wp|ZYhZF;XL$!aj3a9QRiZ<9nz#VoxCV@Vbyg>O0+z< zB&C$^V138#Ec9}-*L3#^C~Si%2(zUmGb8`!Q0jk%``bT|d-%7ey{OMcSmOA^)U`q2 zeutaEoTI!JpvFMV`hmwTZy{OrR=U5Vkjo9Ie4UY?Sg*g5aAkR&Cap2&nBf(C&*hPS zpni1%t?RzW$%Wd;3eX%c$!-EjoU#v{AY_)0P)ebZu-L03qd9qFeu@*sRLMJbJd!tV z+TCq@X8nc>VOk|eVo0yiU{Uoff1welUxvZrJn)x2DX!S6%zUG6_4=Pj>Yrfw57Ot? z<#9i9Y{4v2q5~c#hOS0T507p1+nE**kFRHk?=JCyTG|mnKy0!Qf~=Z&q-fXI5y|l_ zZ-1q!yZSDs$w1k{chW4^KEn4a_%yPKk^+8WrH1#1CG5Z@lgSzF&715Mm&SIpstnoq}3ZqQ~O|u;iopmgGAzyNULi!1H{8 zCj+wB@96#k*8V-}?B8B&B^Lc~@{_}C^a2d|rVx~jO$6QSO)p%mxrnRP5~a6%n0MAY z_cLv$Ip+7mi+Ot44fs>#&h|V3IxmOp>dR!0+57{FK~fKKo3=VC)l#Lk$T=c zk9$EeL6<;LaR^YJ>atGK1svrU^ngKiu(nC2A<8YxuAfTH{Z>(H=$e=S6SJ&8rNHHA zO)3Kk89Ci{=KD78xyL)02+0I4S{PLS}=S^!?j-qZ{0kJjrcQGZ6a{6Y5rcBzX1 zgBa-T`TQL(-JcO~f424SZsBPVOJ`by*-Za#VMBn2=mX-2XWH6UXfgU%e%XtE@Att2 zb`^$%i0I~MWPtGhEZOoOpQZZ}28bEyqjmuO4aM>Mn)ytNiU@e8rljZ&Or*sA-SJ5; zZ(!TG|J)uMAdSxYQy1O8iHm^G|1pCi-oPWl@csQq(so8!tI)yO z5!cgrKboCzUKMI}9J<#>at76C>d{|VclLKxj21g_HXKkhF>7Wr)H~iWlv~6B)@`FB zz*J`J{LmdHWW!Go2u_c_w5R?TYB)Dsd6KyQ%nx% zX5lv&_7zrYl!j^4sQt3Dxw8{8t0J{FlXh;hUsQlkZJjicZL|2(K3Dz3qqe?YL@=0T z&wA>s5O>LYqp7_6=A&x~BiRo?U{)&_2NUDNv(^#$2-5#9a+dXIm1;kDqF$e9f%rfM z`|F8-Ck&6AJ2!$%&(mURtjQ980>=k3^3XxRtIziec0ahy1B)rHATRy#2WWjT4#BHP zH);w0KBS(Dse?tmaH~1cr$Z`9$e5fqo6zNR0M||FoiX#fT zC?a2~)jYTTD`X?Nbi9`PW~8MHQnOu+k{|vd)K7&kKiSFO3N{dK!(Wg#|0AUusbCX1vwJj8;;MpM_=XsDRgOJ=%u9AV66D1L_U1sb+WhbQuJuZTtS?)7ZY%N z6taI7U!o&c{xwA2!>Wee{(L3oB97x%I1owq6{lNup=9YoJveHo!e;hJI<+08d5di| z{0IRwz&HL4Z2-o~R*Cv;6xlPKfedP|uv`zAJQcc{=~!c6avwi>R~)uOn2Ud>C;rQJ z@!^SSHShbDENDpw3T-7KBBGm*V}38~u=8|nIFt7jfu(18?QM`cX}0V4#ss|f>0D}*?xXzS#&M{3=*OYyZTKL=1PLH{7*x@{#|V~ zlEA)b`jpibgw}ypt$wty=sZ7`ld|~?-@RE)RL9v z%S`!!Y(RuOZ89v4w1YeK2f*li*x@)GsLd2v2@R zn)uQ+ODC+G;P&hZ&VtkQqTlwFrb|T{o9jdHNz0u-mfvmC*|&k|FOnM$Qw*g}NtIpg ziECq7TBa&3kITxC97VsLjA*G4U>lYd(}F-V>l9?%4zSsS35MMef@4fWy8FP3qNERV z3HsexF5}fY>LZ5MuiO?59L;}_q&r&$SgiuJSfp_L7_{Ite15eeD*ERc9|jE0I@-aw(KsEFfx4Tv=7y+@ghz;74WBgVol z15?KnSUte0FGgy5n7V;SUQIr?yt4RpY6qx4zLKFDg}d4~S(pBE4DqkP0x)W|>m0bg z^WL%Qg;Xjq=>{z4M+OpM^D%}ofR^f=`13al0rh^>15dJ?%j*pUPIz4m!xrWjqI9H7 zBUQtOW-=J)<1;$D@H{1+u4rF^V_$v`v3tFLJ;U%6Y~Q;a4%9oAsDJ2XWMO7e{wm6nkeFOx zqI|&hI+KQD_gbR5hFk?-R#fDX;P_?1-x@k9vLgYAOl69+ZJjHK9eSXTn$rA?E7`?2 zQG%o+aOtGjV8Nu+We~H7(CY11(W%CO%*hKao_>u=REOsbWJUUCxG((tWeT0FQ>8(y znpLa@U7_EYZsYLN2WX9n=v8fIUVf)_9|ngwemPP~Yyi3#vS{CgNhugu2LaL8)UQ`R zn&M9C#?UE-yl3I_I&p9DlDzx62hC z_ce|VM6fXk#@b@73uh$VY?1?-SN5UbmdCvWjx45z)e;BxZ|qVLIoj5_A?g}_pt}lk zJ!&G-87S`fcH0jZeMc!n0c_M8N1SHkVsCfrstu}Ew5Cp$#WEUF!HicRr5ss_EG;eV z7^qE-FYiH(^`BRp1;C5(dF6Xhy23w^6O_{eA5M&=}AXZ*}op)*@a0k;r?e!*%uij6Pq_FVf?9`@x!IzW(X+y@!w} z@}@G|H3v0~ucGp}1*$0a%!Ou;L8aCk?(hgtGjF#w`ztE1m*9?9XK}*!#_@*hP}-|h zjRy(GqXhh{N9~tE%qbvTr}y^};#1Y2W;c`zTg`x|#iPlYkWX`&t})Jb!=K3&8_5hd z2FuJwoAEazcgPhg!-G+RSXdvWRE)lv;;tVB;UGGH*xC(vN<H93AnO$+^-c9RU^x+6>ArS0PK{Fui~ zm2-5P_Kn}^yRH}T;QmU5T90di>JrT_AaF7x+7HZ{9_fQvWT>{Rf#_poJ>NaYRsDxV zMh2at=A^BeG?seP80K$q?;Xi~vAlRDo%&KZg`dS21kPsNQ0caJZu7YlNq%A&F%>A} z#l{ypGk zu|!!``g$?wqf6E)J zIq%%y23I9C)Z?20M#tx3i+Y}Z(=Gc^8ZDVu0|r|o`NK)4eggX!oWfOFk#DXnD`K!5 zB?lc1JTv9)tZ@p28FN}66Vf`l6YC?bE%L=DfI?JESWZJJA$Bj6$)GPfUlC8&>~o>o z7p^jLxNaqNy}6T5q)W|LiTLSBu+~0F($9|`vH|qS=}fv-QEL27-dlvv&8tsk3ApHp zlH-TFeNX(N>9RO?t9G(^9JY!WH5V^+7QSY^$^Q++3n#XoWoCuFp^aUYz<*)K3LBGi6nr1VS`C zDD~Fca>u;M(jymBpTW7~^#!}8`%XFtrMR^Egp(AX;U0=B- z;GUBCK)BU1z=Rz`(EI)`6t;*Q<;U-B$i^10C`x)%`u%t`GCn~zuaWlpIAbH4a5Zz8 zG>cemboq+j_-L)uX3cEF%;$W}d9IpZI~mX86#IoBXMUX*RKl z?1Kp|x8!E`huB6R59ySKoc^~;tXk+^^9`EDRk|aHE~Vx>%(yIiaPf+y=sbpL?ca_+ zb}pq_;^pyOyit(4hJx$ydKS?JN$jUM2GP~m)K2Mn3`>vK5AaoV}_iB#f2#(E@?W)`kCA5DRU?_{}lCSL&=%h zoY#8TQd}73Uc$nTWUqww*>@KS@Qau2CV|dL)FtarHfB9;mRY|J#FCoIK#VJ>GQ4Jg zD*tpBoDg&~$N17zlE_0SbhMQJ@<8f^>!Oy%@Fdiejh>Zy%Qky{XlKQFkLXiO&w(}e zn_2MZgQdly57_+TBvaj^&9drwVGEY=yN#8z9dkZS9x|_{8VioJn?%}wG8cuN*xoFh ze(Iyf!dynYx60e>{p~RK{(pck(5?Z@C&l*HNzZMoHo+&e00yxnIkBI~15845s@272QKD z-zn!1a=y!@&X$;E>2`5P>v+U3?P6l6o}|aOh&=J@57cu~-n<{L<+l6WrErXs~D0tDevP>EpJE zluWbdayvOn7DEAn$mb4?Dql_ zfyYzAN5+z6pgZvDT;19YV8oEH*0~qaZF4!34U;VTfu$~1Cv!^2>Lr!6x5GNfXnf1f zhT=Yn+PKZPWZ`(#?Y9*%-JNT8uVx4yO!LO0tkO$||I`HMV=m5794$6ag|ujQ@12}H zh^$t?9WK-sa;Ypa%iUbx;?#VtKYb#s_nbQ!VL7XTCwlr+>G{j+_P!2~m!l#GOLK2~ zIW}4#WZJxD_x7+#R{9{}wZ`TS9@X*QvvBQWBF($W{k9>d+@<@UD-GM?Xx_z7<$aK%hv<#Xmr}+r1F{;$`s=kX64od&iFoTU^~+Skp&r3T8q#9DZGo3+ksaoAo`Xrxc#1$a+-_c-3W>_m9Kq!iV|x%4lUh^r>8gN>kuY@BH8-VxAB#9T@#MzrLBrK@bI*rfsU%9Wv8^w#eb2IKor=+jmOI0?jx*Lc_ge3wbY|}eZdj(^Zk`!QMz8j zR`&+Xdfng%cVVJtXWQ*G8*1%=c8$No*`l(1Nt$5bYwNu)B7JXYj5MU=`}G9EL?KI* z_MC+$oJdIE?7Mr9wv8l&Gq#j0cHM&J6Tu*@O;gheDqfeHgQ3BzLE^NXC)h1n<-uvD zs;03`+1IGx+A}#^2lt&yEXJCbTa^#ezIO*qTc4$%s%MWguFqgt$XbOjuzL3aP<;o1 zUE-XtsOEo=Y6p7NkJa?(^?s&KW*lBkK^AXkj--H4=XiJqN2m#%4 zKd+E*>!9lET|-b`mZ}!UG$C-nQ#yoms7f})eoUPf4Zq?@?y;HAV^bd_lDf9nQcj!G zVeBj`&R9eU$`h)#%(^k^TMISu!svwQS01E{pOs^nK8a7rxfMKxb$)yMU6g~LcPrR| z!GpDrfris9=ECB=%$Cfc?ifjza>dtm^C8}dr5Jj4;w`sG#Y+-A{=&MYCS*1Y9+HK+ z*BZ~`ID#@4>r66kRBgYLk#Nr?JlU5twBO!tST`;pbTc_iydjZIWsPYAo0upzk>ww+ zG?ROnmEhI0_Y2s3SG@MhO6u@y59S9MbDPC6>g2kVzm|AqlL?mO#dpdkx&GKYrw%o4 z@wz9fdffM|DbZc!gTnW?qXdRPp7h-H&@kZw7W5Xq-O=k_;bQI6K$@s&XVu1Fra<)V zZwu`QE7T;y9_p9+Fg%6UKdKPFy>N7_`$Q`(AcR}}-en~7Y?7EKI<0dxx3JXGbMmYH zne)49SHUzvuaHXfw^9YRs)n*ne`yad1-8=mXC-sE>DI>Picpmxyj5oSaN#UiX4}vP zk2ZI%YCsiaTf9__I7}}51;>BkXwlm;v7g31s{ zn~p3pX_*32j6ix3ENd52?k4RETyB)ipqdi3I)7rPZCOj?=kiq?P@#)bfG zK^lw81}<|=JI@!SMkE0sKFPUa+OxHMeg-t6vB{e4yZ@n_XM+MVvQu|I#bXI$zSg$) zX}9o-q@~^5*k;yz_g*>IPHMtx>UB{}&j&Uey_P5j-8^)_QXNZ_iJ~$tkhFvhlOCkx zJUIn$?{K5xCdqPRm!8EKZx#=08q6cmVpL}T{KOO3XL{{bjL#_w_$W5xuMdk`=8`wkck7GnD! zB7ltlVgejlIGm5sGBJKwqPtwmigDOba7@(na7eHMc^~lk8!g^QqHiDPrv61^*KWsk z2!E9|)<#kl$GX;EF-pIHQ9u`O)@_E#S)|$by!8I_2HWvB*mKo8bj}Si^8N4wl!gVSZnt#V)aDa6ANk zzZFFsyet1>ZKT+5S#hQC5(w>4336uSsD<4SBbzJ=*aiDHx}n4uMZV-B`p^T|*K7L= zmDp~V7x_{{zz#Qc~W7Tl5%*`&ww z18^wxsBOzTK>Vvs_`s$y;LKZM=vv?W44!F$a%~tZWDxras*I2e=e~R26;X1!I84kh zqEX^yf3u9<-FS^T7|dAc@wHIxCYx!j*^T&-Z@Mo5Ez}rF^metbmzbP1Yidc?i#7c@Ty@sj z8Q16`=B45F`BX9ftM-qyLzjq1;_+vOP&YRz(pvuPqW4qO@cyl~GRx8N_$ef$E!T@c z)7Wqpt?FF+D$9dp6yT3mi=0zvHwKRj#w*Kl!h-nW0H;r#l5Uk|m-wL{>21!s&z;T} zg@kfdD=0TdQi5OQsp#LNNW80Z_Uoq{j9}Gkio}*c$OaeL6E=(geNX;9xUT z***)33E#MnMzo#1oi{0k#K?vfWv(XM zWh#O!TSCG{E9?5dM1j&dYr=Zp5k#4lV4a5)-hoHV zj10TJ@@WxUCgch;bC|1@1q}9$?lYxJT#go^?pUGG z)LMAwFW%Qupp)`SK2|@wmLl4EVggfVJRXQkS5JV)ruXd@QN;+3T)-8!;WEJgCp!<1 zSkdlhrObEwDMI*3mz3oXZ|VP!1^A8LY4ws_&bkberZ<9Nr88FqIEY!e)Ey4`FuX5M; z<`GZ5M=T3zgchhVnkwO6Y;r~LLvKhrtI9>&?x4r52b2owyi%4-1l4)-f61<6iXqpk zNN?UhtO&9hZaa8|8kb266}xCX6I?)gzWa){#N0MT*!c-!P4UBvDYmdXLF5F7W@#LU z4E+BwYQKmv^~aptAp}MeVLW)TC0oMa-%U2FRnjfi8)A@+fVdl$?Wd%I%uEOx*1fULsiJV&%x2KQb1#;KcVOCz|L9tUx_X~=pJh-h#C$IGo z+&*i4WAp1&lW6?_Z+qBJa5EtM7tdk;l;$>g>XQ%$2QpELRs+5ea~x@AjwZn0JJMWH z;x`HC^=4TXe!fJ9LP|$^?cBB<4N~86 zFje*Fev+NN*{jt^GE6py?A3BaJjVyPK|o(!x&zj3spoU5QUzZckAw~6M70mL!1}8!(=TM=rKh z+Ok$EoX2O7llAD=y4y(n(h>x)WgjvFDA81?YyNZ+{!>`Cyzl|3pB#_uI{{I>{Bnkm z3DSX^H09ifqr?E==y{p*@nr-KBaao~ioj0A=IfEMis*?c^lTa>KCJjpYI{`bh41ne zyXuEN93@cx8nXU#VakVnJ|tVc7E~U&lCrZ==-;ZsL8Q??#OdJs z$L(Q5)Bc@Irg2sa?jwKK12^-UmJ=6txmg?*5a^00kU|Up&)5Htk3LG@3i|^GgNHd? z=zYsoP1CSiDJb$_F&rSdnVzy;Kvz4Ptsjx2%DISv2Ci_++MPFX0&4F&BjTR`Z2xEv zUWuN-WJx5)G%GD5h0Dq(g82;A#U{^hQ);<#YS_i}t6pB}K`kxhBric(Yu+a_m5z8O zQHO|LOHrj?o7AjFiFBmodO9DS+5v9g(amBa10(QpA-pk!`hsSP>*0XaAONX#72)oO zAau6$FH!&R>x|ifRLWych<)pT&lmzZ!CC3Q#+j)!3shL&JsmN!(iFn~)_XR=p~9^V zvd47eX6*s9VN~H1c`Z^Ie<17m)J{m=noY}zg-h^d7f1Qs0)QjlF6~1gM|kK*^T8-R z7XewVi%9_k8h*4;7>tB^68E0ix^MR%~Al0)IX z*wS;oc3TQQ7fn5*EP?*$M_^FNg86PT7%H2Cp2v~TV}pqQ?C9=vR;4hFlgy)4VJcWiS~RV-_#30)aDwN1PQRmlgwPD}Z*!&8(HA!XDl?^K zM92vvYRTI|h&F?6!y1LCo8dRgaND|JZOv+@TQ;GhugO~S;7gtwv z1k>op_6@pfnPRoQ=BWL-U_Y(t)-eC^S?b%B-C0A%6t=KLHR9^GzK7jioS00`R+ZZ) z7@tN7&U(;0=}se^Ug(lN->n{v=&~%HZG^k{?hZg|A8;}3w!D6rUJTFHS0A0@?KYOm zuN;|%$|o}My!3z<${*kt^-sUFn3Y_PdaTR61Hg6Q7G@jCoa^Rj+B;6KXUJV%A5ATG z7;C5CzD%nc1?0cF)wH|_WXxpfG%vzU(uTh6oZ+|Uv z$}7z`e2Cn!x0l6j+9`@PA|a=h*-aKL2yaxsPc|&n6~pTnc?aCN|9sneP=K$BB^+~O z-LQswV7mLUIG>rZdhVUxsmIAWrR(9fa)AW?yYYD|lGLloiXJ2yZY%U1POPA(7?1!XaG*o z>ft#GP*007PL9YY^SR}Vj5vf2qZg!h8-w=G8LxKh)(C*KwA#{kId8E8z$t;-EF?q& zyVMa>L#gi!jVLgQ&^Qx@t_R}gk*Y<0!1Fp@a|#o!rb<5CkfD;q9`mvD=8Lu|e}KHyJKTbxXz z5d{-iTVgvLkdCi`)8G`bh4rZ*UXYTsfQPTlFTvM_QHT%`(=J9EZ5@S$3Yur8^lYYh z+q0mQnjcNckdsao9(cFNbP6_YA^I`>g1svQe1V>Dtfpw*8kDDsdI6p|_*q6(H|$v? zBUa4)XR`2$=@IA1Tq*0U+&MxBd#LrZ~p=T z{VzEu?ENGNH`34?`Lbc!sp9qq5)-s2VwO`ZEVO&ex;TT>_*^xDw=X4D4TK8hhDQIJ`gL#ZZ7HaRv6&@Uv}P`GZjQ+ z>i63_ThXbdt&e4p^1Vg2duLfa;CS%1l4NSB1r6ib(fz{$@iVPdwI+s-G40A=h_Q#8 zTxPxrF!5FQpKLwR0UuZQ^HUl{G0L2?Pt#5Sk@Qi}mQrqe39GqTh!KPRXfmNIBHep9 zLjS?grbqO_hsARA>%e0yPhDAai)G@Af*Vv_xxY2|cG6>@Tr)PyMfg-DVUrK{&D@b8zufK; zGJs_!0;5i~^xhhbaZ><&A}NZZB#iNVxWP0uEZ#jC6V(SYB_c;gT z7M2bwC2i^%43K~3CUExCyT^~eR5n!-S`tJmB$4E}jE@pd=myzxV2ND){K&3;pB&fa zi;U80@HgS1aAa#mPZTC5fwxf}iDTIEe9OhU@$I(fgl_%_nHvQ=!xJ9Xl=cj3OQ-aX zb*uvD>}hLMubAV)T_1tmO8P9AhmY6J3FR4hMJAIp3|uBJ=70;XvJO?vpNv6jzYZj% znyVRGS(7x#5p(U(E(1yfs!{s#-Z=$Y)o~Qt@>}U}AWFH|wDO;Rx+mLISjlM;n0FlU zPDd0*912U`3O@v2iCBiqrj@-1nI0xb2$=GwfP3CcIHoI9h2^=|H%f62Yoz?Bg8y#Z zC!@Q;gF5VO@#T`5&+0rWsKIeC#ZS~{YvMC~wM<|zmh?&$id(T&^$rOJ=blFb zKJ;&ZH9*+6dMQ!_cP_=OS80~WyUNY5g zoc7usRe*;VPowAITidG%K~Fb~)vGl~#ryF3(`9;%nWiI|08H#Uo=<0d)7NpOcSd+B zoSSy94}H7NbFNIL=`W8&Hg3GAzWK!@g@KPFb(*E14W5T!z%-F9THvm0vhlc699wEgod;aQmuPm?Q-7H9s`e8myVN+B#ox z6S>@^uXeLNFU_6L-%^^dcgHg|Z^0tma$Bsir%g!_lQ6RZsBfXKC@Oek1TF9D@^rmD?eyUE2)Jhjyo?iq~s zVAwh~mw7gU&)|jJot;+MI9_Wbq|~o`t(lF@EzNP`87{{R3d^3j><;HFfto*HFq0B4kbX>(KjX3;nQ7jVLaLjr9vU8fiM3aaw5Svm$Rm7*9P$*6F0V#9c}tLM2DK#*6? zR;*()lg5UzaLaqFDK^+lY`-5_kF9h7?(0>)!m{I`I&j2ffc0(FWlG@Z>6QDH)Uk}r zTaxnMjSzO|2mGux_ks4{nd)6$RD4C!r(1ZDy{mN6xt-_14sXutDiz|e0k25l zEtqwyv$`5`c)ndQyxk(}qTjhRbiA5pDlYp#J78^HNQ-#Pc5P&=0!UzY69tfW zT)6t_Y>|6{;cra0k}+7}p8eZi{@05;OxQLbA3f%WQ(s;ZNAXp*>5417&r-BCw6sJ^!i^q&!2pZYyY;Clff{fFk_KUTZSI_?nZDA;w%mZ;H-xkdMh(%d%X|znV z1%gRBF3dk>&Sw38?VI0@{na(nAi)%zF2>P)X#hD8B?th5d;K%7zD~;s|GtM$_MXi` zZR{I&E&IN#zsInVB2PHT2RVb$JJznd(pX_o$|gK*-1BFSh6U}E_)Fv5ca!L_gRYemTw2;#for1>WmUthwb&Ydn=q=Yz&dZGAM(DGL%G%dvG|PU1z#u4D_OIbm@; zxHf2Ajp>b>BC1QZQt~*)^W$qaSMMeiwAk>SF2Kk?I0PS-%3|Ylq44IN+L7iix6&$x z>v*B(*a*EQqyJ%d0O~wvJ`G%yGAJ=Th2K_Xp(nzXRn%?oVNi7pp5BeDut^xpqY3>; zF>F11_VfGwR3BK2L$kt&bSv5C#5DB8Aq1SWv>q1gBwElrquYDWL1s&v18Q#AO;uws;s?KlHwwiFb()+{|i4()6 z*-dDrgJh|R#!FN(;c^FYVrM&$h-B9oD^V$-413N^*aR0>sBk|sqW%*#2SeJM8VgQS(tX@9|LA$e-MiWhS8u$0@x9c-5B2v1 z_65;UF?D-W^x8v{71g{Obc$rb%HAempfy4^qEyoWffH?fz87Vs7pD4v_CCM zT$P9161oI2R;9~$Ly3)-(740#zl?_>iWBT`J1eYuh05t-r)@aL8<~6%plF=L*9$da z7IKP8Gd>S()Nx(jMZ`(ID`kW)O?qdyqOVT8#9MAkL z#klA-ndwOdZ^{Iy`nY3ELhe%G;}Fa%6Sw-k?*r?fXVUWs)W#2rvPME{)XsuE;=Btj zg&v(RL8xT~sTQlziafG<-nb4dp~TOh{8+a5ciDFO^W(F0##ZiDiQ#=rW2oAF6l&}< za$5#BctgT^Ds#OvywC`~Ed)TvZ}Z%NNfUo&LB}7q`0Oo~`lDRzeU+SQq^Qf*B!Np! z`;^d}oHk)}=BdShZ4CJFQ*u=C+NtWK%U`yny56|i44rd6sR)zusSNq-Q~IB)BLEto zBl#q{H-nRJa|F2yNi@@rWC~D7rJ8hF+$qJjrOrF|lvZBzeq1|((?L>mLMgOZ-`Kq2 zD^Ti3_rM{_CK@_UoNl0Wj3|IOpQA2dcW=UAetdwc*jpDT$|V{ggj1%j5EM?1ax!lk zxR>d6=y{6b1X5YwnTx0USSs4DiIEI3P8}@%^x*ywIWy$@C5%i%L7+*{8C6iGdM9$u zEMi${B%}4JBPb$}tgS`c)BCzlL}~2%F&@TewHqPU#=Ix2{p7yAwj1a>G(Sbm{mu?9 zh&v{MQZw8MH5o%p`HJhPYuZxG1RU{|iez?4|r1a;< zu2pVI07k?B01qhwwHbO#_qiOI0M$)zS{$+VsVO3uw<T$389Nq35TWfp4zD4aQ_n<} zRjx3AX7%Cm&2mPopWS}Wplp1LCrvf;TTr#jMT&&;qp^!|@xB&G(nl)gj5f2)&X5%J zotdr3$-}JSjPA{n`2>M4ZqHG;%=po3nO;iP-qG@VBl2l<==5*r^dJIE4Lo(QP zSFtJ{uu|A?S-BEYN!ZGmq&FEKXhTZq(x5ne-{U~ctlUcYW-!mW2@ZaJX zSnukb!ZN7SQvU7?x~P@0Ne zKBb8k83Qv2k>%|S`GhO6i*y?qdU_a)1&-&?*4q++*7>5uq!W@Gq=&^HFOAHRSuzj6 zZ=RE3Othig=!0t17E9^D+izWnWgfzUo~MUoNmjB*+R@Wsj*`xuxk7duCad&PlrX}`M9{AD5*J5{ttU`x18foNjg_6(`IGj`!pIp z9b{>v!uFqD^?!OIFMX`uwbN^hA!XFT12`lH-u&<|kp~T< z?<6C34ND7xm{iWRSW&>M3SO+1GQT5!rO_55`=MUbp`_t>5s0;HC?5tW#*g=ZZ5*_6 z?_f+sKA(J^vRHm39Q(|Z%6n8{69fu2z&>m0t_MVQ)U#^?!&=wU6t@p))oLtr#2&d* zkA(@pUT^La-8CKcQ*S4r0hJT5)M9k%!);%rBqq|L*6KJXRC5060g)VN)hTm1@>zO} zW>pzMS0;(rtq*;AC35IAv4<@vlH_|5Ixk*`ym%ua`pK5$kZ>5kBXu-1Aw*^ZT+Q#4_p%&S2wXMD)~nNhgJY|!@vz77|t zu?`7-Qzd(?lj#ncS}3_=j-E!LPzl1S_+Xa6!_1?3c_dvnU~o%u9I3i5o|O`YU*|wU z=KgRO>U?n~s6E+iF4PLqn;gV2^zM6!(X4=W2bm`kRe_mdvm@>qN2{1y^-vtbo~aN{ z7N@K@s5+by%Pc-nOxeu~Ntipxzx1tkp9Y;hXHXT$`9WwOYvcvaca|7k&G2qD4+aTH zix|55c9s_seXr+fxE8)O>C|5%#Vq1-7O=#R0A#K0&M{uZR1`i$whgo;z%x)9+xhuu zZ2b;rFq0Mj>drJAXnhndQ7{zdKF_XiIlJN{RbELseH`Y0_6$S`_?NRuDmn+ zV*=llF|P`_U|v(HWI1#>uzs}OOvW{e=H4l0rm{CX%S9W9-%xK)7lSp+U^I=^w_Ma; zlhQ?J4R`6+a8%E2P&2u~M!8JMy3{9Xym+~%NG(Q{P;E_K!AHL6x`52=RpQ%s?i8Cp z7cUT)rYlR*IZ-p`>IJ@;Sm4RGP-W*1drwv=-pN#T@|dff(CJ*p?<(!E18$T|o4S-v zR>^?CP4-F-y0FWZ+RH#@{N5%E^#)?Qx$khf*lK$_Ygl-P@g#|H$^kpiywU%22$KSR zkYK<%mpgCA($opq)`De;yRre;#D@F1qH>SjPkA@2$K9{~0(5Uh^#Ks{_4aK}lL-sf zVa9cDaZqD?={J}SSPEL!DfspM229!y-yvOQ37#xRKx`C{evqf(60=r@xka4q6U&4) zU!EPQp3dq4h_L%-AqAaMTAO-)E)}m$B<;iM73Xp_*c2J_c^2Im6|7*m6Mhn>xs z60^5(5+FPAny12v|H3Qb%eV-b+RaCQ(?kL2!(1fxluO7mjnVqPQBsi zG+2jU3D#3)i%@aSo?6_Sy@BCV09$Q zuAJ>UiNXrtko;mL6+kqyG)>TFsg$bvyY-5Lx`Q90pn->hyk$OP!zc8hn5XQm6+A90 zsJm}E!L~Ii`vRV~ORS`|{%_3t?_JrPS+t~wU#+sfFKJnweiNU{qJ$tp_w&aeK)}qa3jOsX2FVNsYaD)S73wC=K4$&Rf#L*uB zVad;k%^g*TwCXwde#*ic#2EtpDruYO9#Xe-4 zy5$V47H^NJvHG)TOz@RFDI>u~<-6YUjo1%&hCkY^z{$Sc&A5jc+z$ZwM#|8(Iqwnp z5w@DPHa&`YZQ+n|u)AOYqVTHe{lU zj_7nwIUvs1g6xdTu;W~oF;pBwlS1BP)S}pG=U70J3!}Pi#+fCk%=yO&h{Xhj3CYe; zT3^g=chTj{dFy1#sKS7Z8GeDAgy4zd)-}<%Xhr2Yxr^tVX9pbkdi$=5JEOp8LJ(0j zBN5C84hK$cMDqjUbHcA^f7;>ByNpVGE5=kMNUi9_ za&2@caeFM>QvXx1eU%V%uF@Rm;qhUj+(757@{u%t49^c&UJ%*H<@pL5$aa(W{fzE+ zM!xl9ZFe*b)=mpgQRH15yLBq$|BcyD>TZ&{kA}zY|xf)x23Eg1n%Is{Em56&|vTggn6MVs@|lQgEGZUP8U;qa7zO zZftPe_83KL^RfEyHbU;ih-~IUvDJVx=#4q0Y*cb$AkRHzuE5XBMaMpTef%hH$6Qx0 za_+P^Z1z6=iS8@_UZp>r^yj%bNn~t2$Ajw+brow1cfQ+1U$`bMQXT+}9fllVo(LI5 z9OjP_t4x`L-oUT?2aik_{dSi9?8Vb17ZpDS}a&pwKMq)b#N^rxI5Jp#{5 zk9+QWQr+0zYL(T=$YlkpaSTg7)@q)$XBBDFe>-WFO;4?L3Bx zNcU=w%%!;uLU9^7QkMq>-NgpgK)ZPCYltF54U4U-B6JO}pTlA3HVyNAgU#-I7B5uy z;lOKpd>w%1sB+MFu?o{$%v(e_3gQw1j{2G9Q>`ywP1qcdn(|m0g^+p(kS5c-{8dq=!cpS4`7g!dLwdiKm=y^)_r0wjmNh{PJ)PX$tDz5DMrmV)+ zN*JPMzcjsZQ;6XCIESb-!t2j_UEaG{J4<+x?t}%1+J3s^rN!deY20e=7NlSX$kW4pjEM%5j=W0Zhbntd7aBXl>`lUTznjLd0kW`XP; zgMm%-de^JO{Vos1{2^#J!3gv0v`d6*bIsCpswxCCqL0m=YZ_I>Sny7Wu1j@tE|Dct z>$BvPSb$7s@0pruSJH;%j%_M-U2&2jc$sfTwJaHv;BpUl8R%BHjBBfDEVCW?j|R4G zpq)LxrbuSR3tNOBEra3Ke4_S>y;T_4K>zt~^PzFmzhNz{EIxbzY2!8^^zwx4tRCO3 z{F4@_{ie*L&Bt;p0r@zptcM!AC;HrE+#I6H#~)h~6%b@rbJ*mf_408!{zNMQ^FnseQn;5ouHvN8Ztug+rwG!?CNwA|#eCrTz{1vHk47W{? zDIA;YCH|I=25*yKUMB3o%beX zMCJNYi)bPmkj*=wi({vH(sILrCm88zg@~JI^AwsQY=E-aA5l08njOh+elOh8oI6-e zs<-m`XYbVhts8YfttF?P@asCY?^g-}9=KB*_mb#3P~Wr=eD1dx zJ|eh76&<~7&eOoZ(6UCt4s26)2~LT(kqUaNrnD2TRJn#102-Y$4QRAl-}e_j%i|>C ziEcuPt6U@7dc*wFqhvwR6k%SVK*5u!^{_DA>a(#=kcI46J$lHq?DcZ@Oo!4`!=RL2 zChrtikj1ek6Wp56vev3%;S2r+&;WU}72@any+0JP2;QB5Au) zzQ1X5S7s&ri}Nm6ww+hS^Xu_;#HUS0!_2VBejkm% zez#D=u2p}FmfQNmcAAgJu)E{v04d%&K_Es^v0i^f6)k)ah*ZASZX`?puD*yXPimGh zC#~In_+A6xNPOGB%4Y_?OyE1%bGR6|v113>Pr@j1L_wW|J4_!5T!dVBQX*%I@B?;h z$*g9t%aF5?ZSw)TJ9@?$rB@;Ga!ZJSQ_#o64aR!wGuBbkvSIA=jqzXH!R%gJhcdb0*a zkxM4|#zOG}f-s_A>pC?eT zNQ=ed)(RV0X9(aneplitSnchB%+5PU5dfTxw9jL2b?F;EI2nJS3Vy88>?gFwT*R8F zNt61XV$?g6J`2Bpee5p^+933`1&{Zlth{KhLiK*1uL|REjqH2%$r8_DGK9*ZEbhW3vR=nIK(ZnR z;D!@tj<@ET934c3h`nN=9$VW@Olp9E?B`cs`6t7VLbd7J$^AwU>Pfv5L+4$)_lsut zZI~3SE7iE8Vtlv}1zC%AamkL74lL<HO(;b!_&<(cGY@%B3x zm^0Wjp~&hf6*ZX$kMmKd)$@ypxP(cbQSZ*n%mi`{yWv#gqr0#U!?R5X8q{55ER1H> zE${GQ9tn;hq&5lPTLXC~S08eh&m&&zdu5a;-7`7T3H})ZU`K9-;=p=7k=m*$_t~EUvM2Xo0uEU z2Z~~Ke2j|>=_Kg;zCwzRm~U3RvJ<=9=7KFLqq?D4ao3xk50zi$%e$5GK81}T`H9op z6BYaJo)HJ$N~=MHEJxgS;){oTG;3;`vl@bm7SSwAIxJ%E1RFO(lQ}pMlxh0qY};LC zeWi{=TLmKpi>KTWc|Qub+{m)7tc%P;>PXqcxIbRyD59sYJ@dsj|;#PV}@g`mcVubtFhr7j2 zCsI}BhL=*sv=b!b-An7O*CS^r)mM5|yC1QH5+%o7Et&w07_SPCY4@CWX>th$H*=tp z0@ZvNBKDnEU@b&WubmFe*lQR!*bGXxbirCo%D3P5WoJ!-JJ^+jFh^@7D|ZRhZuC$l zQ`2lI*`-H&hMO-0Q=1~v--Fheu+4Dz)J(h{m+I$=M@$b>T8x3%va(>%4_tU4RuKU} zB6wy$27P574AI-PQ;zoq)Ox}q2mssyYHQx1%}Kn*rp*nibn`YXc&3}A4q#LgtU346 zLk>({A4UmYFgC^{loA)O(|j_&T!f9jt`iR&{u-v#>|X$?55own;R3P_iz26HuzXW= z-o4Cms0vP{7FUi-;59(O-DFnPGD1Qbq(3~ON(zkdJFeo@eMu{xkzgYdwuQH()nmmlFpBg zyl~y^aCX>YZB81ky7*mkR`d(;KNDw8_SIybR;R0azbo{4;_-wve>Y_zv27Z3f@AlZ+ zc~oRv@ny8Wyi3AmOs{n(j5rvmyq+yJiQ+5=s{6$h)IB>6?^hlpl5R+K%L4lcuyS&D zL7ib_2vr9$t6V0#U{2_@J0qpE$ia63WDn8W<^T$UQIlI+SCR+)u6EO*H+|5KrP>l> z+c~92wQ=VYd1EpFD}tDmuA1k(Upq6g6clEQuETHVJ5&F%h>rX6v_)ge#bVY9OJC7* zJ4I!*klow>cfYXs-~myy`|Rop=kglYIkFOl?1Uf9d2_h|O7jJH4 zZvHXeE#dv2@Q>IsyzN?}*}S53$~(zp`R6+5D1a1X5<{yi47o%~x^&@FNW`FCMnE8_ z{2O7!g0s|IuWB2R9XkHx^}Oy?@&&Jmi`Ia}z$0oY^NhiAZip|@_GX`7j)1|S%rxWW z*>xp1^G^BY8n1Nty8FHl89yiXm6efxCF=2H(6eLvfz7BFiMX@FpY4mDJCy1&K?tc$UuDhC*1JqjUh0)Z?pby7w;L*&8G;296;&a}=j(&ZLxcnliR)4>{>)9wzL z9vyAWHt1E-j?fpHW1tNkGxU(5h59J?sU_;eP=p5o3OCP%df|H~Mp9Bz*C^3cZ?7vj zx&vIuHYh;+^@%vW^6=D&`W3K0oKGgS{NWe}-$HvTp{@!@4i=#Gfxw`L@t{aRW(=sW z(y;+5A#DtQ*0&uke0VxKc-H@6GSYECrbTJjbO8z6&<`4M}fK&f(^20xW5$5;N4L$&mJs}xB{&7S9=N2H(?3LS-zRZ`D z^`@g9UMErOQY|hsez#2q<>RIi>}8*S%hE~L4XM;k?wm~f4ZIM0)r%es6Vome&E8jk zIq3V}@9_t1pM+u*sMg^h60^J4EyoqB2y|Mx}OuPLncSGZ1gq*G?Q z-0;Tpt|GJphgqOK$sjAlpmk3np90bg%_aZV*a7&Q>g8va4@f<90=EB_7W0^_yd+xi&*=A=fsQM+~YO$;)5VAqL@eXTk2lB!h`e z50+u#Nm7<{XM%5OlQoo;TXexaF+qj4S6?iFc+Jz%G7ezD1(j_|DTwlTMtGfjx?ck7j`wV znd;&ScpPR)Ecm|>6QiP$kHh(e)cTZ|v1aK=3wWHpJi3pV7A^P$mfvLlgux(Fpe501 znN_Kzm(lx1#R|O-14&gh)dNWO*yxWL)ev7j7|O2z z3zF_i@0RoJAscLUDRPE9Nn=XFut-$EB5i`@$7Z*41gfIxo(nJ6Re*u@r0sR-q|ofE{d13+GwN^Spl-HHVCl~ zbscb3$Q}zOjxUFRI23 zKs*dW6Q#ahU%hgG)o6A*!uY5N=#I2S2$ffV4Fbe7$#_GmPBADyZEfSk%X~3|w_`^X zsP}jucN&3ewbCBiJ=YINRC8<9z`>774$XeJrX&8@vGGLwO9cwmQfs0H*{wtDxaOQ@ zHIfZ6J!;4HEVgg}@s`v&*r?VXziJKO!Ra;OFW~|ea%XA>DxBBRv>L|l>txK+kmlcF zOJSsJZq}s2+AwbH@87PUcEL5?=hV`*2K!mQXh4G$rh~xr%ohLi^Z)*UR0^~VxWBYdNwIJ3CP(Oh467Ns`7r}wOmsXy4tR* zWdo=KIiPK!aj|huOrJ^?s|H@f#@X9#tmE@c5D@<~2E?UyPPe!0ZtiatkF_;;3`a!M z2X;CgCzfes_q>4^zbvEHwBP*4Fb3m8FCByYTMS&6S{y~|GN-V<-mbNz?6;cqEC6y5*tI38 zMaP&?#>GZs%>uDkF^q9gJoX$ZNlCmPbKd52-WFS-Y@`BubMdDb-YwSiKocEV9%(5d|2v8Q-EMz}o`ii$taAne4jUx_4TWXi1x@S-T zcEWeLgHoMlUMGw+(#Rm2qIB`7kh9xu24#}1FWe#a&A_>x{1xkq(p-kUGqja@YXaV1k zyVf#s%oG4j@B*M(KO2LY+lznH@_0Yz?~&;5TIS>uMnz(X=7@`Z$oT4;8T%9Cd$qmO zTiLJnPEIJq>`;nUC4%V(X?kDDjBmi#BTIH;^Trt-o;#y%V1wlv#{vUu8kq0d-r3c= zJXWJgkXV^7U}!}k5I%VS(Z_C8AFUK+gG*milXERnq_i3oB_Z>#@P3U$1a-zEfa|c1ZS1BWom`zppN*yGf#)&d(7E;WiI&^gr|g#X`$XrzI4FFa zd4BW&Q7?{3m-zY4PI?|h*mCacB|5ubZJ-T!rxWkypc_4PrqR|k<00W};Q8dnp{qWx9I-Xs+Qg%4VQ2G@ z+%QFPBQQ3;wls#`g4+Jx0io6GBer$~iNC`>;!2rpN^dwrZUO0XouPpAHu)N_b;DHu z%Sw5BbeZGyZ)Pu~6M}kNKK|iw{60u0jtKtP4@zOdzGbZu9_fVdWm)R>7D#dBNIK#< zy~st^DXB4?o@>LFFy?2ax^w1ZpF+mOdP=+lT#X%9txcF9(W1oH;e{<g0t{sXsHO z-``%uXB?UY)D{{LUc?0cU7SJUI)x5>Z)#Z92d{(L6=`%;7t zuNDwxU|D5Fr0?_l3IB5b{eIUg$Y_XnaR1}W!vhx|Zf;^>9{XSaw?Y3wd#fe5))&{; zUI*we|EKF;-66`rn;5;2e(vk%|AXiKx9z_8?fP@S|LyVAFMsP!@ptu@AgI4soqq2f z5uZBkb~aYllM?Ay|I={!B))1#T1`B`azXu{%txX<3gLgckQR*{n&|QI(V#c&g!ylb zK?Dic+QCM$!U>T=u@`9#WcqM(dTJRZW^H2wUq^}xOiMQT_KEp<-tA>p2<^{Sa|K?Hp?hBN;Gf&7~Mp}ZcwJq(^WgRusdH8!)w&C!vI5zwhRr~v1?`TW|YbT z1JZ|3xy@TBWH$yrf%stbYrUVss(@+y2VDdW&(q#v&cg>?vdx=gg^I--iM#BI+W8aP zY&D&cP6}nu4;`o#pznsddUr{9ljEs%vG>F@n%)*XDtN!Px@JJDR?O;Z|3n%;n(iHu zT+;{gfV1eHuQcl3vCvqrW5YDRCnEAKHCyaJ2BuXq(3qO6$aw^BvLLry0;E-;w_D|B ztG%@I+CNy?e=}U4I;~euqleCC-MH-AjI3CwBmyEH@Vhg&ne?ftsl%KO3QV72>6OBwTVeBo?sz*PI1MBuZw_!FPYJ|hOz zJ2Z1A5pUsS_dBX+YB~QbC|4N%UT&;Rkqd0=DU5ORKQMrQ&C%c9#H!RgA%R=H`kwTy z(k)tBL<;0sO5KNIYtM%&qpX3a7RRH&B=<8fs7^94<(p`UmrT_57jy9Ml4@RX7AcGE zee4lW@4W`vvj^lMFvgX&Zg#nxMic(X@cyISW(&v!7M+e&9qBC90901@3(ETX`b^Q~ z6T>R^^S+i7dxVf@9>R?+N-IwNf7duM}D z-&~ofPGS!?d#;$YI_o4Ci|JF1Vr?E`M570vLiU{va0&m1jmNhII{C22!p^~?!cRoo zBWXM~2$=a}l0&_hU58yzzs3d4+S@N8LS9!yM8rOkX-DQS?e7JQcBn`xIkxt`12Hi% zoAH?m7JcNb=FR}7&bM?{VsS500}zc&C)=e|=gYd3zrqH>ZCJUU4DZrrAqDBw{$Z~VAWE9W~j5JP4_ zQVM6DbP@+S@kZpimAJXNIq``wEuCQJ?O(oXI|HusJcRB+V6;r(_BZ^W2G{8xGo`v^ zHils!_gmYIud!}IUTga023J<=J)uEMyj2EctxzYImo>uO7QmE*MNE9b%IWnjQz*E``}TnzL1843&L}d zZN~qfTn4Km?~O&*jb^L=>1sCFSCLei1~X7w%fG+#Uv2AFRloUmi|)_OPJi?KlV14x z-md_Jqqt$AI>>HQiPf*u-0T87I$BDrya!cZ-`E;!ypF@q^;YHaBsc82Ip(TA5=QMD z%N3;O?1=|?H~*EGMjKDsCvhE&2mEZmKl@vEpuGsQKaz>Nbytw+k(#jsx5cxky^!HV zG`n`ZeRt?f#d}$0Z2f22*yZh1zqAgB7 zgZppx2tvvag)=mBl}2oiGB4)7T!TTq7wC@)mmBfcHlYE)St#2ef-I5Kh;_f_4$?`W z%)tB^Jo1$Meu2zt4Wr&|hZ?Ab=4)Y=px%|KTlkh%p)e4hT1UVGva0vVmoS0ll?JHO zWQmQoe#zp7gJ`Z5Ka@IxH8-DO*5X@dbVcD-!L`M*H`QeCtM-*KuDnMnM!F)6A{+iq z=8uq`@+1a3y_~ANPT!ee(E}cyukSQ&E z<7u>l!+DP%;#Oys{FGdyStv(=MX!VoPgQi?53)$S?khXFat)@xO#ZGJ7V&Oi<(R@~ z@S$&gD$3)g5iE1@oJW;@_r`GF=@DtV$&cQjyJZbxjx_y0>{wYm`xyUGN}`|d|tAn z3D5j-IBU#r(ftCzh`x(1S`DXOSuN6L)aZQH8MnKZ$xyqijU0O~nkwvq#jx|f7dVy= zpUgPthZBj!MI^UhpBk8}H=t?EGIC~&$wBoc4BQ|5iuE&%Fw?(e4{Ghz`=nw z^Kg((-OshYV^tw`WtNuC?fFKjXs$CtYF|ORSjNSZu2gKrdf1fktfm| zp5q&P4W8}ziNA5$*G?MsO5tPmV{034t;Mpfd8}Kj>nUUJB{*;c&Z+B7WvZfcPE|<> z(Zo`7D zfWPbATfL9Gs7u#*lgWoN_^HT@ze{v`;cPME-7ClM^~_0p2n16l^yd>eQErFMr%|+O zC$>ZI7}Qb^&>_RO75aUWLbu;?MkDbJ;Ox69pqfricZ3S&>r0?N2KD2j!UA;FR$Mxj z>5B-O9mktfTunYVvEyrY_*yIJrc$U#jt1%S)5?Bh!=I6$GsD1KlksiB+h!x*x`+f9 z*a(uPuF9khHZ7sCh4emozq(sPPrF{^9-gGO(j=`q7isp=?3k+Ccl)W*mq*`}&hx)3 z1dVPdUNVq-73o~K#UZVFdwcJ4Ta70EjWgH%3SwmQT`D85K$+&ZlTwZbtI@fvdY}Tu z-T|4}?w7(1<`J{R!x4rZIlY043TjqY>pr8UtLDICi~XcxRYASI>Iqswmxj*<)IE>< zb=Hq(F6@A8ukpLQ{P1Vb79+enacNCHyCtF&u4VvQPx20k_8{B^TV1KOaWH)H^ITS+ zj$)j?H@Z&Kn2n;EbSg~G{c(Js$4w%!fTWy!sEG^ zz0de9j`4GdY!tPkvSuXGr?j29CoP+Yb~!uu_3^5LPeo))`h8q#opn~A$ZsF4;2|^E z`n1H0g$a`yv;^FU4ibjBc8U}R=sBav%$QRJJzvsMvEOWF3D&zDRq{T`CdjYtVqv9n z@ps-Tl}ZT~r*n^d?*CHPfT8+6gDK$%DZ)M}Qf<)J$xLnN>af|%&40`dx$A2~OlDt~ zT^%ykXdrCVjy#^_>#v+ujZ_>-Lyh3cLvsIlM>l^iEc15?@zh@ASjE+zkXlhya3P6o8UtuvEuwM&!-q%`om%cB0 zpI4!+v7?o*HlBdRF)2{u-s@?iMhIYHyL!+vIOoO~?X~I>Ps(k(*ZVJacDYUe-VPXj z`aa&NRB3RUvA;g*jYF)pG<|IpXWOe?Fi~D(p|ieO7^(dFYmI540TY#GjyZPDyBG){ z>cDZYwXB~q8`&1!-vG>rDD#=Ow4W+{d4UZP8h4=4A%RhriP&Q0=jZ;ffDJABwz(dz z7aD9!*G(D^Fyl8P+E32N$>!NLzMD4(cP)&IoMP(eOnr%-nvs-!2gnyqJh~aB8_N?_ z5}!vSO;Ta3hMR{DFEM8zB9I&;60|qlJ|RuXK(%YkFp3_G3Ilb;mgZ=-s;({72#3j< zU2;ZZi4PJ+Y8)o0G~`JZf+U_{3l(K`p9fuR;%sM>6;*JJRnCdXZR51UWN%V*VTwi<6aX`)%EO=M}#!w7|=1VS@k6&+9h95Q{Z+|M&rVM#;2g7HCo zy^?d51YaZ=k>NT2>X(zHQ4u>3Ek~6!mpom}R0G|2$*b0(v+T6Jl38nM7TPadpkn_( zOsgXn?1Shk-6!3_tN4d!{olR8lPd%htX*4Q|KjWGM1^J&<}!3nZxlhOuc0&j zBw$P@TjDcOo;*4>cKxQjtF<-aeXU7(T2}LfLj)j8Y z1#hfg=MpxMMF|Z(=8S@bR37H!d@Z~?U)l=ohbr6aVDZ3VBvf5&|2erTG*m5OwzVt+ zx~XKFajZjr=Ag#v>Ek^22TzWdL{UwTF3%SHPjz$}J`j$iIl9v6p4rL-A8Ld!?-~_f zTVFp^b2`wq#!_vpZJ8R(PBvR(Lu{ zJYDTb)ma!nqnX}K9KrJF{EcIbh6&9Ptg8O@GgiO$7@)$YcuYe*wj!u2&MsKB ze2F(XbcQcit294?vQ!^bTS+KXMm3o;jd`*PI;~MHIq(e6%6>%9kX2i30{BRZuT#Lub{B{cB<~lJ zZtrDp^(_Vje>au3=hBq)_uq%(QoNP%BSinaop-uyVd-RVw9j>*?Xh`biYUTY%LVK zclIpQhWTF~s)7q4hm67!&W1zAGneaR;^~p?J(Q83xE@y`cq$J_6$g2%-DYc)Ntc@Z zk^$Z=r?Po>9L89N=PX>9F!)R3y=<+;ojGsT0a9N`0$NQ)h_i~TuM63OR*V=~SX^qCDcy+Ii= z#q?d_n%4|tSKkpISyC+i%qwRm_{15D35D0jewD&Cnl+_xMSbFkpM=Ji>_KU;;o_GO z8YvZ=oZ0Na=62~#Rk+Ra zc1DVu>|r_vY2A@OafzC{{fs#=woY537RG8DnL7h~T_lwi1m`cJUcQeC>#SP~vV>zBA zprXY&F4RZcj%*+yS}*if)8}X^O_kcCwpZv;Z7A`AI_RHA<5svVFXmoC1LNEmv(ET8 z#i;A5HS5H>a$_#n{RMq{ogtaop-Hmpw5Tj9njcPgKjIuhxOfb{;fHTGgvDc4MC;EEgK*qb(dVcVx^%iby zi~FeV$>d4N=~K}P5w~EOUYvX2r7YTl`xYYd#+_FL^;r@Tr59$zrj^;?NvjFjV+l@F z+f@;@j^Z~SdtwW1i(PnP3yWg#X=L+# zB4y#{8Ewv_QawQ5xGp^(zd5U9fRM)0{pQA*;#IeL!OSZl!L>5{Xm)D4SXqnVvZAhT znrTU8gk$=&B)W%;zb{%OKVmjJvWddI%r2A(f4C9W!sLCGALj?+8G*}siAf8ewejBI z{Uv)wK^3Ult7DRnwl`Izo*k$eGw?^YwH~5&PKXX3hD9+(diVElX3X%$m<60@jB#UZ z2(g_G?#Og5)oz@H5oXypd9T2ic81URAwM#1HM`%x083wYrG^@{3Zod8uH8Pr_Na-~ zwhGtaE}fHq4g<*@rz4FbJ+)4Z1;=Fn2M+PyA6J>MoXLIE8HO&^EOo{Zivq9(YVJ}J zqhe=jtVBCDGcS#gu7heyk`mTx3Sh|`EZ0VK{AToEyc~qCM32h1ai<&hoU_tW$Q@jX zKvkT@L-SW~Bg;cqLeb$ierEe3)#@%|DC3{RPJZ{N0CY`4*hfE3NaT>`bhx`}i^=~T?|I93%E>>_udh>y zAu+ikPwQJ0)7*{rmDk9;JMQ3k$)B4fWKQRTFxpijuZVQck(^zA+4&fy_2C>Fqjoc% zN||=plnYQJnd=d})PPy|q;O<^FvGOFq%G<8aKq*Z1|e%LrHB35H*s!)Nw(khy$OlI z^Cvf?&lah$<%$%`WuVZD{#<*lFyF6Lj`vo%6unSUQMSBZDtjhszbLJ8li8aisZzS} z`C!)sOn7!TO~8?GYmC^C0;W4jh9OdgyyFyx&`t+NN6N?Ld^=08D{yzY$th>le5N>Q zug&3LM&>f&OMZmV-AfomoNnM>*?bZ?V+XTkq;C_L!>}2YCGLnEbN<_6|KYOgx3&Z5 z6`R+(^B?WD;4kEar%KZ8L#t&3J;Y)fOwE za|PNhe9W5lx{#wR`U2si{{hqfpO4M>kSHi91)6ni2(sIyX2o-Jbq-MR?+E=~cnPzM zj%=4KU$C1fABPnsuo_IY>tdG8WMaF!HNNOEt^J?DA8c05n6*ezG;&?7Dr6{eG|skW7}wG^8!EQzS3_rtT?oOG^dfemqws1wJIcW3ocX`b#__r31$ zaj9j(i^1`_4;9>}XRanaVSSucA-vq&nOcQR3VGkCHCx@`3khXAqxgl^d7|Bx1S#4%AWcM(xIXEa5175s;hO}Vsmm=;CMVwX<8qy4E2 z#Rr-K=k&iz3Ki!TH79E$v>Xl7lcuJOpC@bmNchP7=|9nx_G5~c*C7cMFPf-P`AWPv z`CP4W293gdFIQm52ZWI1l-m{Y&g&Ju$$@~I;vE&-n-D*i>HYZzuy^(C41cEZ>I*@y z5d2D7;mT0n!XxiH^gYkJ8(O)tJ9=f3G?OQ!i^)(9no=c5q z_UfQHYU6D@>B8>tP3{aifnE5zQnFx`k~%8MF_j@H+U-aQ!`QcRbY2RJb+#|cH>leP zn*()^*41`l4j7tz9+<-|huF{1B*7S?%l2h3hr3MeDxxwwz%2MXV4lq)gqqL z)+8{k7QOKJo8V+zE<-kvQ6SLS#$9~qe0KG*C9 z)qA+Raz5Wg8%SV^_$40F(cRq*QHiixsKGi~sO{aG%pK-39!U{~6se782qShykgq_T zI=^F$fj#ffdb#X(pL1Bu`%6TS3C~cFaobYaZVd$3Mq`;*Cv#fqEj2jIRvP1+Zx4r@ zZw+)(RtGxFHieULVk4u!6q@)gRc^f|vdh!yad+8%w$VFWPizl7SyCU-H!|7Kc6+f? zZo4I&!s|4m`P7Lc*tIoy&cHG^Q)i=3B^lYaKcMngHPk9Ts)=ns>?%oj7l)KO0QDzK1JV^9 z&Tk_@A|Bd*N4Mly^!->|ccdvenhku^8I&4ifjyIP8rVv8ds=LdU&hOlhx`F#+ewp) zEw|Z!!Rp@=4Qb(!t)?M)I+V<##}x#H*bXaQ0Spw!7Grk z*=kdEcXC0FB?x`W8=?wfc>ASHyE{7wl199_Z3J7ioCb=QOdCwi{b&t#TnV?Qq|+|zxN=R8#A%Z)5eDru(wR`uFki`+Lilj<*&uzG zlk|5d8}rDa&CBzqh@+J^`bNU{VnJuC$E0eU+=51u%9gm$m*<%G)PpXC26L_?e8vfjKX;Snofk8ApIa=j*GcTaQ0{tViwH)dmx{k2g;H<#24C{II-Ny-?7vD zUAKEtss>ikgMQ#}w4SJI6ddS)4V6%!_#!O?UiBAVeTQ()hW=%>8qMU3@(hAjvnzXn za`D|19~)OuF|gvwBYQifYGGl5T}%oMD&u3WPfxv3Y$tz9>5ojWp5h4Jdvl#La5kr1 zH#?oAabFKIwf+R>$|Q_5rL%+M)nZf-+lZ&8&2^3n1sg#1wuTyTv*?-!L9WAVoqmwb zT}Uy>Wv`jG=Pk9p#ag}HP7>(&_?QN+2|UB-elg6M_azgd!g$oucQz_D7kjGGSRV8B zt8c`gdM__8{g>rK2pGod?Tk12;mWRF-uwlL>!U&$N#*jPE0|PbKu|>_Ga7$LoRm6d zLcBs~iQ12)8fm>GMNl8((JYX)7rSMwA7Isou0l<`dmWJ>G^ib>|A?HS%y3JYS8QeZ}~g46DFBL zk9@97Z^)vom_1C9W8;^qM~>M_1+Rgn1r8}V3F>t73-eM(=(r7uD?e&<+;cCZFy%q3 zNE2vSan$7eOwR9r&c2`QL2KBg{#8#SIlf(>!s^-a@_B60Dgqac&pcJX;`)8Qehz)z zNNewpx!D>^?rV#M9~imeNf*&Mr>M}%mlIm8@5*2&t&ycB@A2Rua8>c6*RIYuF2S*p zqf4e5rz%X=3MD7&hh_o^Z+cnqP`75PLv}>w20={XLM@1^eXA!jnOhkzGumVb#vH8K zZ-~_sjkUG4OgT4s{q*anCc?Ut&Q+4jm-vy?;fi z`Xtb<0@xzt+qBLKfJ6?BGK?1f8XY`dIpi8tAKx1LO;az< z1Xq&`%=hY3eN05DC zd)UK+N9N27rXlD=C9K-rA>1i=y{jF>^+Bzjd-;7Rrkbs^kY{Os3uN_vtmKHZc#0xLH4#REXzgSJ0Uc!%p6IRJr1NEcN^0Om~jk;}0G`Z-qy;%nv%0_VEFwyUG zKBv#^6Koe6y^yFe{bZUXt7dc8C$R7y*yX7!3+6QJHjFe~ZOmD!`uuR^MKSmM=jgB> z$G!UpCbeDZe*i_IisbmwTI^&MQo&z2;1%V92eclXeILa@=+EN->1V|!B;)n2d<88@ z?9KX}(hJlRMq?}kv=%3#mL@JIvU3+G3&o>nzYc33EtF>jM$SIJppuT)91z#;5Y0DJ zE1gcL^V;DEAK2+Mn}0ejX&oN@l}|Yi;NUKvbJkbBu}|1Ll?JI+PX%iIf7ptx9QFn0 ziLvGFviRHU$WQ~=D9ARVYJ2v|tFn&V%_Y$xsXC>)rt523(vbr?R~s{n7h9=aa1#em zMf#K>saKmBVh3PentfE7*_+O=?5lBuT2{)iT8h>HPbGrX_2ydUN|y>@SFN1vjjDb_ zqZ3u@9qoS46w39)HBP5_R)&YM*v+Qe<|>UdsSApg%XEantQ|8yw(}Fm)zj2LMXQzU zCg+>|3N_|x2MpSOg|PXk&;xp_BNF9I&*V~oAH}WfYb8?tJI=}8&WZJ0!1V@>yyj*R zNjF9+9t@hR@%^4D2Mvy08rG=uH~9USkbi4=YX*R?CPYnMS0{3OgM)EYl#_>v|GkD!vM1J5gW!^f? zATDrHI%VQkY+E67@o;y75W@H+MnuMKpBNBdLGB$u3xHF@AdcOZU3HJ~0M?>Xd!~4S z4wm@l?oi}3*c!nXAVaM0!JxKBt2WA5YQy!hUt_JI#o9iX3&p#-RG9&`CqB!}_b$aAxHL8i4%#5u)*k{g z^MVGGq06ZxDr+&LJTkXCM>GeTFiiocuMpFgpoU>T0jHoP#r2Jv)H40a4>3nY9*ijh z-*Mx>l5IQzYNM(?U{0YB&%pzf@6ppH4Acx6~?lbB6)Q(_9Nm<7GV|IoTNwa;uB=c7d*DX^SpH zp0I=7FcNo<;4kv`PR3#vd>%Y`i$Tl!CJgdv83%!mp|HQ6gPA@$9qOR1E(u3O*?ly) zBmSem(7$tJZvVE@EAVRO$<1IWiqJ2IzPA>!S8w{>^z`f+-A<1renfc0_atfP%h7M2 zRT|tPEIMv$0FIn}D+r*G-D8R1s6rv;qMy9u94+yIWJ^1|1_Mzzh=$k?wDjz~1hRKa zkX}zpU=Vn^J*hEe`#%^K2~xjHC2kNn>3YL&5Vod#<8aO6E@+SI_|QCpL{jY~c9DLq z;T2n`fYWJI+-WR`-F}x{c~vD@op-a}R@sC3=@w^Szi+5@EKihhh~!D540At{aE9IP z>D4S&9JPi^Udm+4fl!eK3(3L^Q>_EPg8>sRwH)JI3$FzX`(*9|^n@mHK=^6I&Hf4d zFxvYeg3?IGn=b@=>6!jS*)ol4A+5N970IlnenJsq3Nx5-mk_=vXViUjbmgtXclupU z8#K=whot(RqH(_>#-E&PD!(OD$vk=(gle^O2bzm2NH=;8URXQY+s9goeMG%?y>U?j z3OpC&`C6+8B32`v>%+Mms;UySqPb$ZBsMe$6D(rzbLC21kO6;4q+F#Hu)NlpOZQ#8 zy%12VFvwX*<#iG{@|1XtNaaY~7&K8gQ(pi&5>ZmE86u;^9pa|Q@Q&|lR z4xP1)5O`Y7V%HM85NvQFX<#c1pS8hjgN=J^R9b+(=v-Zg74Ck?{mmZh!tbd=+fZ9< zyg(RL3WSf4JhWQs)?D8=6Ihn(Zv;iimxUez7oxQCk=Isc@kPY(Lb!moWx~!YLIEw| zemMxU$21eXf6BoCcT9$$@?_|axZ_}Lflkj|9hUfq2dYV)uK7vzoCf2@2w9_s2m?AWhfN^Qfw{i@Q&ovYGD)cj#q%LJ zPyCT46y$fO6G-UzAN)H4Fp{r4*`0ZHJCN8+#^BI!sZg(y=Cv21C?(Q(JTJE86d`l_ z7vBPR7BhW7ATOttmM6$)8TrI~I;8|QY>CNSqx58L4ZaZ5vCqfp`2J+U;7!CC|6k$` z`$%lBh*@sHUJiH$F<07@3q`91My6&$T)%SKyT^Q6uXp&*&aesYs|5*cd4Ivmw~f8h z;`y=FmU&n%X6%~CmfiL>Vda$bp32&!^64slx1ZcobNyOhq~e{|p=gMrcpbsS3`#vn zsnov7eD#`WVO3`MiH=1U2mx9!&kr}?75Saa3thJ*1Wm*n2LKyP2e3hSge2r5@WFmz z4bVIXaGRCubq|%Fv_35xz%Mx6-yM_=FGN%Od0`B-j8>Y1Z;j;TM|E;z-zgw?J>Jqn zieV=U^7Fznice>~f34k5-?=p zwn}OXB=`LBry9=4LHW`-m@%Bh4(y~(8eH0PhhA+qEz<0ICEKQi_=`@tsLg4`Bajdu zj!~)LC0iMf^{UXfyf5tr9{`#9tidXW*%WHAdbNlT0ClUZbYs*s*eA3yq)U=3E-pY}N@0djQc_|v{??>8061YNBwgeE%(HGD zSsvdpiF{$ulPsVHq|v{b$d=h%Zsv8pJ~TZjU2&b*-rO`;fYOrywPzRaNt1rhRCzV^ z=ag4635?{;L~~QijJS`D%JhAR9%a$8-{W=eQ8(xZwQ(%ncMoQ* zbY<^Od11EP-8x++SG`1*QLZAL3Hu{I?cZ6FZDsPj!+M~73JT}{{uUIU(th{QvTEf% z_RhIl;V3d!^8FpiDT$G8^ ze7cYNJ(%W84fs1FsiIp0aSA}GK2@qsp7dh)FAaH3Wbh0ijYkd38B7O;d>UycTb7=H!`z3y4)D9?g>=fYU@@QK$Hrb}@A zC-2znr5aso<>6HGbsZN&?LRcbejog|+dMR2q3Owq!zgzc{<;v!(_3(*LE#sxAW-8) zTQhj8TaB~HZF;*JXGb&F5L=>7m_KR!r|jA!s`8D9AcXWkm`&;%7JB_qwY0I;>7}6u zJh3<@E8@XZd*fey6CPU3j7VsWADt8mq!~g<#-8x=C3a^M^l-+9%Xg!nq~fuxy?$XEuT zx`4{FYSSK=nLh%onIf+1RGz}k<)l2(y@!@+nmq7=zq51y);IX1XRIkA73@!`n2V}G5E?D28H(wE&5`Oz1J>PA}onNPwriWi@ucL#XUE^=< z5CkV=)?e02Z_!BaTMn$MZ|s4iH{Y+(w2N4r;n6a;Z_w??>)Wr47_AUCe}5yf?E7k| zv=Yg4=a=U}X(XQmj~)GgmHZaDENsI(t!*8dd#n9z{W!Gs4;t`a@n$5eIaMofIBx$$ zS!RoLSMnX&xO3_}^J^S_Z=2frW`d4^m3dT~l^@57GFZK8+ z+Y^q*1Yh8_IRDx%o_=aGe-foJRO66qyMFic@K=k6BY7^byI0@s4|EXXEgCPern}@2 z>ZPJ{KkDB~ttx_w$+6sNWK%GWBgi>=>84Eq&i8M!>aHx>ieQS!R)r~|zD?%1(pIDK|#O+V8 z3h2nlfy8Sv^I5!S(ACoshCIS<+|_mzw=FA#wLI#Kxqll_|J^VHS`@!;=EYwMeGJ}R zV|?IuvlslML{YWbn4?iz^Fsml=jEv>2AQ$EPLL%HpxYVEh%~S6>%bgiUe9J&T1pm1C=Cy z8C&x1;=%6PIVRI>tAl9hHT(1aufIP5NR>sxI7O9FVOkXI)^3};4^+pcsi`R!e|nV6 zNrgD2;ag(7`fw^RCB&+Sb2J|es25gG(~5V;TUz>KtFc~<^tjhsT{Qwb-(nx#lkjqu zd@|~%s!0kdOzt01N^YfwK{qp-D@uUG4b{q$QV?H6K#V6Rw85nt;c%`A!gG~Bv+>N4 zNCi*}4>MzR)-8gsj8An z6kU}N3YBHjsT&Oz7|WKEW+uN*s8*6x?;S|>YgTLD*;!Ey(k8=iChTlsKxoJ$gVuWm z1fs=n$7)VS!VYM>3wbAu`+aCzTJol;@{j6_(p)8pHZN#>k2m%MDr@wKsT^a3^fdd! zo9TQuBN@KK53Of->tk)A`LTbYVizvJ=)R2%m-y(z+o3wm5+ z#)aUi>m`M%phuT1DzJ`8Gzh4<}I<$NsiY@~Q85ffTbQj$XE-=_B*wD#_0MZMl#Mr z0YulKh8cHpnM5Xn&G@gAfSKeYby`f$X=J(HM#o;({4^UwqC21>RLr^W$bY)l=oNKq zc0AKGX{2mE8K_sJaZ_y-id5NF%U2eCnSgmjW^s^ce15tNdM}?!F?BZe?a4?)fcL?+ z;~ScJ?OhG$o%2gVo{413ax!77ZhZosG5D5)}@`=}H(mQjX{E7oK0o znIIxkVsA1gGZXkVR(0yXUn->j+4#VN-%w*^Fb^O~pJGNErM>5|W~d4;AXkO$pD|7` zJUw7S^=Vo0*TeF@3v29HUw4?aAoea25Rqk~Pd#NQjg7U1#t_z#Bwj(+(4;W*iX5w- zaxC`6IdcmhkE;IX`n7 z`!y6s>S{C7N(cv0rU_JKNdht>I-p4-?13Wen#tqgMEz%xk`Qc2=uYU|h0bD|rK5zO zm6^4IjO~HEWj}%CqY{o{^lf=WZT~%_Vj!u#h#}wLKwWOo75v3+Q+`FY@wdC3;)0_y zNrR^~otmKP%x$yBMypO;ZV~CxA8G^UxS9k~+dNHMGJU^B$|Nk}+?O<^W{clO^`4<* z_+s7=7@gCPc7^G5phnm3L#domeI}-QS{y}n4JRr`9QEAU*P&gd2GMav)TK^?^E317 zT{5Vg{oXA{F2hYs?+XzdI-pAYf%$tfmy1YD=}>>sPTP#t|EgM76#{8Px>) zns58QxS#minV)@GfpWuklLzGk+DO*$G-8eIJaFZ5qsL@uEQ=h3YvQ_aVa!3Nk9K#UwnUnb^z>RkbFb+M8iX%=LV8+0iDz>YUu)@=R<5a_?b%to+ zeLOQLp!XK8jA7STPZn5t20ft=+IE*gsEnYltkt}_+xB!Qo}W0Gt_s^=N~|JY{2WUk z;If^>ox+vac;xI%;*jJKZikDs#036$aKS9NUSybEYo%%^pq&INPLGpzAkLiGt;c=t z!9!6xQ`S0@kg&ro{ek-I=jf;+=$8yJLA!QT)kHzx8tKtD=q_q!VJj?{#c4PXRvi-l1ffnp7xEKYw>=NQ+q0T@QcKjz? zh$?ny5)gdoc4QpaIi3|o)xTHk0wfaRO9bgznMj-N1`H4f8RNmEO7QU6aCClhp$@Y> z8QU__W&AKP6Sz8!qxaW2c&8^WS~vxP)D1iCN(pa$&Cl&8Pm_EZ=HO}1H0Cg6V@>8k z+54ZomhWmFt1($WVf``siG)yqY4*0`FsRE&U#>E$7QBtlm9`i}swRi!82n<4J0Cx4 z#PW_QGIrC1hyaBUd@;kBo{`|w@s#9@562L(7HHQLu!eMzQRS z>Og28@yM+~>RNj52xnpqi9y&VrCUva3&VkTa?zPajXLO}=64aid3D$zq0UHMJD?~p z)2vK(hin6a@`C>ml$#s=Dt^2!g*2BWadvxhZcBeZqs5gx;xT7Q0fDTKZ*)WNZZ$N7 z+?V~e$1`DHE{ST}kiHv-6Z$RLwYHgP7wHyhl&a=*Gj4HA2nR1X*ndmIOX{2D*n zZ3?lOLm85uN*fJ0VWO5nJ!d-(>87 zaivt5UE8+e4#dBndv@9Nv;ibXho!Buq-&WEYCBNN zGV6S%LWeHkpZzJvC-uQO`L)J}u99)@z+iM~(LXTg&RpZAT841 zAX$JghLl5zVk|>}bb*P<&E_u@-!Yt*n}TR7R{2SQ+nW2|!AO#8| zv{NfSBYyd1EMyUM;O=kK6-D5|I%zdYs6oFTMDEP;&+rlxUR8?s4)3^hM*jojQR+ww z!DK$vd^-MHRoJwLKzu(O{eCi8KyD-gz(s$x0m(KuoMhs>X`g&{DcV}kc28ruB+$h` z-d-CDwHt{qe_$irasF%0)pHM|*Qtg!#(xQm{wKWQ|MCYOr$96f)SL1BT@i!EvIyYt z*Z!GHEg9QY#6V*7j0~uXeo;I}{p6L^)}#2pTpn#91iNGU?`*f2g4^1JkOsc0?To(e z&F_Sj84lw*Yt_Hn8^r$rQX5$a0X*-Y?ouL2FYxI?p!;kh06ev}arGR?H|Wg4zbMzc zrnP_O8#D2wJ&BV)B}M1fZ8hzvB86gre}m+Pe}sn*+5RziheF z{^uG1xTc@iwE6Z^p3eLtvbMGr0Z^sLlck9p%T@AMoEdbVUoVng2RTaYbHFh10`VmKV$qr40Bh?D@}&jcz8 zqPB*|QqLIw|Gmx}em<6Qa5cD7Zq)7)(>IfG#Q7&n!N^vf=k_5Y>5csO1nQ}WuLX&+ z-t=Ps4+gaczz7wmW5H5&Q^0HKqSYIDvXE$p9$(;(8)ru05&yc*`3}h_>;ajRQQLXn zh~2czvJx8r#{^RZDn=deA5%4qR@@wt+W=1(-`Uxx2l>|adPRvc{Rec7R_vufo$330 z8pL{aKjqW2vyO&t*5NZ*?(<(}4+MV_DFEF-&=HFjEN7zqRN%yG3v<^^2KRjQ|fkR$}3wq+g_5ELZ+b^qF_*_+X7=V7& zPvdab=(~f4yQMTs#O*EbQ1T7$fYK8x@#1=T53Oh(Rn~U)KgZ}^=H#P_g9V&l#gR=%OV?S9f%YZ4TT}*08P8${uph^D8 z8uXQPo*i93{@_!ee|C{tG-9_+hZvuMp`$kIV`P96uKFQAkBT}sB-RC|vKO=c_Z$8w zc6rY!%lYNYAW|lHAgcota6pNylxj#mhrLSjm|HTCen#SN1q_D_+%fxqv=#q((fxD& z`TLCO%!*v{)c6R*$`aWCw#A`zE=}Ji99g5ot&3a(tflF`6iRYOD=kWR2_5jgK<&5I=9{$4OYhuT4^&k zuj^TOz9z4kYXKIL0BNNzKu=e0J#mu50ytx79OQ%a@C%bC*fS!WJ*7rNmO0 zsQa4M?AjUtBv&^N^$_}9i*)+YZNm_9H0DF+J~s6xL}^9L67$OS&~s5!XCEa{^Sd~w zGr*+)Fo50C-#`FO`Q)rDJWT_#V<6K7=;G__uc=%^e}-8c#Oni~ex3Dx9y{M3*^o>d z{199X{a*<<|G5$_i}`#rWtF-XE2~M=#7KK^* zSzGU{)KSb;r_C^C0+x0WesLK#Fh>ars1(Fwd`JKDCfQ}$!R48}HUm-jj3~c~(|&y> zJ0T(G&7A=(b)Qlm6*>V^2$S0r z%>n93TOB;eLf_gC;Uy}rHAGMhW~K4`ueiT|*2=5EWYo6%EKGo%sFrwCDvF1{VvFI& z$Z$b&TiytZ#r7c(4u{5}JDgUno@jPe=DGJ8U>24SrGYlWVK-l8n4pDzuFJ5%(*y#m zK8a+I7k-cP09R{(rE4Lu93h)AyT7r!NuQl0it|Z38guzl%~l;W44xkXZ0qdr{-evB zMHqD*69~_SdO6RmY#WbHMeXuCwit*Q$3&0Qns59$AZu@Z)s-#^?7!l_{*nCCR*3(D zQ1A_~t_ht_?aB6UI|^coH2DX8v=H-Rec|OLX!3@+hxjrbVl!>)_m0M!;?-79rACW1`3690hw*F4nEY)l?z!+2B&*o@jCtBj$J$~r5 zRBpEXa-m*5np=DPD-B$+{*1}2I%EUOLwwnfmqX^-lDSEvh1kAHmU~4w0Ec*X`{-mR z@L$wtleu40Kkc%+mc(W`a?CurEQ(M)ayqAe3~DU0+kXqjU*XVy$)JsYGGk>)5<&9P zIT<+x5dmR&8@AK@KRe%rd^kU?@}o8ENAy^Y01>IQI7i)8E3kGM$o8}&(6`g4GAm2- z;CROv+>!}aNmRJ)bnwRo9+7cK)>x$ykSf%f8@jLDBlh2?{-{}j_5!i0RNxs5RtYSz z7Cl3Y2T{vi*N%;6_ugh$8ovO#0X4Rsug;4Qh<52{psqAu;w)9C@|n}yMz_C@Z; z94kV#kXamw?8S?cVesk35290#``Or_G%BC{Ke;&ygHp3CXi06y~gshI7THO;h%>ubwazGh1eiUTW z^3s_D{xM82nb1v`40czM9!HTngnwuDT5uh9Skp&AQf7^85QV|GC~r?-5ZMslaLb z<&m!wcjdKEMbPWlq;fQsfF@g>e4`>)KGC{&fi`lH|NT$+`)huut~8DF6Sv{c|e#&xNY%6@2e2JKClf|CJHD5p*+#*P0&pUwb2D?c{`^wYUTd)`(ia zcea7P?OuBSlJ2Dw@P~ojE*0rz)T#B^vV^+p_Sz+y^5BHK(d~F-gdWu!?!UR+Pa(BL z<6S>JlpKgLo@U-SX9xepjr(XXbJ_Gn3 z6jU!zk@3df?AiQc{J(w(t5506p8o!z4CH&(R-`;$RB!L7Ho#b80?^|2NAv6ro(~P@ z(s3W~vp=UM0j&9_AUs+QlwNLBZy{8g)#KyOBO@bk>b~a@|Hr99vOr)VCH~#b`Kb%UYP9kI&@k@uA9elE(G%o_qKwbLUzeDk|`*DpWp?yI9w&eQ`kC zlQmne|E#zplMOg)*e`Y`b8*DOh~qw}l*j@Kn(V$P%0g$W<)%zPia!SEb#_nJI@!#o z-UUvGQvBNi00(o!jar6s3Zy4`hBy3*|MMg01bCx*J;Qt@bh6x>#jM}+#TNs>zqB;H(>bL+y}dA=Z*(f1xW=!H*4y^D z+#QeosM`To7)#&P2wi>C9Yh-Q)*pvl=T`cY2^c<@BoX^Hl#87(u3ZW4-py_0mK@dPZ5}a?knX?({`>I9_tc-MPkW?(J>fFI3Ebzw(}A z0>4(7Y{?+fyVrll?|mqe!CTYY^DXJ!N`rn}>8Q5>;+PdDv*})s`D|Xi4*Oz&4K16` z<@`qS%eNv;2gUgbXpVeB5(@N4nAzFZbie4f%6!X<#)~`X74zF-?fM@{)km_NmWwR6 z-QoB#^VOE0#nGM3L|;u#`=CG%gpn_{axfA~##UeGOVLzxiH8se5YLo~PJc*HoY}q& zR%3scjMlUHp*(Y(EhN9CweNGdQNOil+ zYT-3qJxh&y5Id_yb^2K(}3NeIvDIw?#efl8z9M?$kiMW51BJBOp zcLrXQp2OQJ*FATuQPDK14DF@lZMB-E z#m!a<@`He+$3WBF^bOOEg6{q0syPgkz2IrHOh5OKts81OD4N$;D0hg-)jU>?b5D%8 zUC!5ENM{Jm#Yeo({wSCaF_03~U2*GX|AR{Bz1AwnTLdPHadp^mBnNx(LS6S%?~gC) zcL5XB!Cw=es+E-lz1DE64<>l45_Wx}ERtgk8la)J6v2e5YG4c?rKxu!jokKvB|Rq1Fy5p)mX*@LK5=bR^G7bp3Ck}WX9oA%;c#@?a!3x1`{x*RPJe} zOKKG@a@%i+uB~ z!ayp-4Q>!xr_H7IgUk8mRYCvCB11CAZ>TwUuFNProBNP+5{sq(@Y&AjSxj+Fcj5i8 zl|@rRcW_U+?P1qmmB$Vl@nY+3qdrs{7%1hUEA4 zm_#g*U03-b0|?Y=?dh}$;55>-A|=~{bCE@HvYwYp`Jej-mFd8*8% z=@)Yy{Spr|52iiq)?91|(~v@w84S58#mGYnXgE{ugFSmvBY)nO+^yr%s0L+AYB0MCU? zK}}!1?q0pQLRt%~cI%)eIr@ZFWx1rD=o;VbA&yrj9zw9}-})8(#bb#^?dJzm;W8^3 z+7tJccFO)v-|RSGeSQc3bvraIL>ia+M`=|c#`b6yo7t5c8IA0#Y~A0bhNYbuV~|Z) zIb4FnBqWuF@9nEb4;Z7;JBINE!oc@s15dVJ8`? zTyee80rfYZgV4Naq*eR;Twz^wAL(O7Y@{lbwFyDCi7*0&TyT@mEJB}1zC%8@~;QGry&-KlS3@0aUG3SHRu+H_|&(S?Q z!YRXfwtTr-LHGMSq!PV-RW^bykS@Ff2SLvMrGw7+B3?1C%rU;aC%?;GC~d7`Qj<)j zWBO0S_^R%H&yK+G*NQaiay~bkZJAjW1wYvF2cv)HFE{8%j3>)|-N{U$zd7dZQVi+X zb@5|=-L*Ax3+iK>5y#ldee>`bskmji(rmR(nL<_>*%uvS{Valv^MRE68k9Wt)maAG z)wJWuVb`cT=(yi*#BYe(BGjT8@4K$`kph_A#E5}0j?bVz^;?*7z6GeTzj@gn&y>i8 ztn~Q}OMXt}9@Ut5yNj>#R11-2ee9pyiGAzxm3{7jp=>TJV*>IaoD*K7RJtV=-#Ac7LC0X@9ofqo9A#J-9-cb~ghJ zVAf9Y``0l8hl=C`G$^66O+<3ID9{g?_kYU!RXl{_CLD`8>t$ML@m)H+zIAd=OHC^& z>DOl$`3s$H-pfsP2|zyoZ`r0edabTlf4XGzV@_-TJswVZIIQDI5a_Aa5|5PIGgzzH zQxu1+uAo^&oK>k2C@%IDsZZnhQOWt7C5bTI7Wd)MHhPt4l&giHcUQ?%4IAW_dy}*c zJv$ZhWy{*C>%Uj~%lw;mc6)|-XZ^v*>W!v!;LkL_sibf#-yfXBC>(HDsfcJ1sRR=; z$R2QO;xEh3?z%dfN>me9Wwa(d$h1}j@gH^F7jtDnDz90|3~{FY=uIFcN*6Ra52085 z+{epqZ8BV0=~+7!RuzYxH1qK%vwX!rvk7#`ZnGfeS%#uq>3C&RBh@vfziE7RS{+}g zqEfubble{f?^rfRQAop?xj5Sib`o#{^2L zIh++YRCNXqtAShV?H+!~O31))CmSBz9JV|L)5qE=d*`Y=c&4@cAN%GXB(?J=+n5b- zBUjik2tmpx43WJamWK7Z zEtG6onXge2Fsv~v5ts!ShpmkqX(;5c2ax7tcX$hxz0t8mP;X&ha$fuGUPvpSmU3@~ zqr-f~XS^m@72n#ZQmi(r|3!T|=WDKPiK-g7w`TrV*2Y6KtmH$)F8vPv~8hl95CgS$chj6+f*72H&HRh z$O6q|&AV6ExMLmN+r_+ni2=P)l;XeV&BNr~m&2TF zX&1LELZTDpa36d_DFdXB{*)e$MG%jvyAZ`;l<7g3B7Fe%0V_ z3SJfDXI<}EO*dzQzSZeyq4yeUKX~0(m zRraVC1i%JjAZ@9e-K1qs33-GaNjyEPJA z8v+TzA;H}ncXti$?hrgk?y$H z!E^gIWQ;w=m^SDqmm$+qYlA@+c@e$TNtUNZcX2TPK9GD|tWuSzWZKvH&BoJZdD%GX z=_Gq@mQ1-;?M8IDTTb8Z4QdgfT7ibD&rhOi_B$r(+>xs5xGC^zYnE8;ujbcj5#y<8=9`l!mr~f8%o(E(H9hRmYNz^GdYF0GtqIBs-G|Tjic7W~E2Gg_Wd) zff=+bYgI&|`>uj*(J?=*HJeY7@q7_RCeM&o5}>KaWb#_RGC!a!s;`&Zm1E$Td@A3Np(sj_bvu4`V{vU zhf&|5ex*d*qt1r66;p(F+3p0NPA%!f{j&Ba(&F^E z0J(V5sVrEW1%OH>e*SeDC7g|!@QD*(|4ss(-BK1WkbfdLQ%bq>p@X#`!u7uj#qS@x z)D;;mW|99yoM#AR*nb*@IBawn+G4}oDFs@H4W(j>y!?=gf+f9m4}y02-h|Uxz-XvL z$1+eUER~I>5;+!3xG?ma&G@+B+RP``7VjhUFz%CC-OGcYLmXGWaWD3|6Zl3G$A}p& zhs^e-e@lFzFkUy{w0h8pZ66)I!C&p#%R<|kkXIy{<%T8DX-4jJP%V_|OZ)WuX4=tU zJ8&&hUd5+e+Tf!FN`6oqwL>Rl8;?j)O8GtSw^KyZ;m#jTdRf)K>1O4%_aK>&q%bb4 zi{?U72OaD|@_rE;gWBPF{xx0x+to6$^xIF#pX;)O1KiXK%-!>o9+)Ev)&s={X0_f; zOU2ZD>y&S);}7jKR{;QQy>r5jq0k!qb`cc?m-fs*cMmY551b;x0K0}vX1zGisXyP z%*;$losB`lFEO#y6Un&#o$0y6_RV~`e!)En@2Ag)v&DIJ{=2jb!p=Vy%b;zNPqgo= z{6gef^;(@dsD3%9)VY($gBOkT-Q3!eoHqG8wKO+w;n^7lCS#T$FtO!^ba300X!v*w zMm)>X$>P`Z%JdHs__wbhlt5N?)trDyq!^04_`ebyPo!8%S+U?(xN6HCaOYt@+z~op zx5Yk0i7l5)$mmR2Va$*DjLT-ZKR*9>x6wJl6C8}!$(7Bt;t_j{X3!q){=P2Rh?JvD z`||D|F1)X;rR|lG(1{mw!5(A_QEnlL5m&UxK9EB z>#MUrdTrq%g&heFz2boKZYR)^3B~YdAg}PmE1k8PHo|W zv4*1-y&P`B;X&?yiMsgyJB_smy&A0uiLj4YaG6d6Wn-HBtnV*Y_0dS?gCa{;w7o2E zH5~&L2LMP82NaN4nd`E;=fI~A=3!wO=Jnel7*cKZ^LClKLrbC~2IvuG7~drH*Ey34 zz8n8q(f-k?xaE}Q6x`S$N;UXok$%!0(N;3=nClq47mO{`>Tzbq zZWz*si4??u051~ZfIPehmbm94R)o&^cZaf3MbVKS^p_v>g$GqDfLlLwUB+tx@0oT%v5=TWxarKpa0y~;uhK!HdUBwFi9>__Or@ZKr3FDi*e_$V*%Ar6LJt%XB%MoP$=P zq+_zGw03B?do#(Ka7lLGm?AxtMy>QxZu}MxK?lK`gq4)5?W74)Jz2qfE;i1c`T%U@ zvYgX8#q%V>#A-~YXvX14^pIEO$ahwMKW=N1c^XO9V2G@LfebGKf&fx}6ph^H>V9-; zFLgs`dM(%4);VuW zr8PyBO_yI8t8=J{9gdlGbAy}K$V{t#)WTnxint=_h0yX z*!%mnKR2RM%_-Gigup*QgMVi5|NA3RR+_HFtPG0{dEWp0=YM0h{(9N|zdjE%Q6k5| z1 zS&Tb=yT1FsT^Iaru_4fSbT{47^8Ygn{XcCP{PWK=ft5&aOxn=?XRZG~7jG$0O)!4F zJtt6o3E$2qssF-EVH!8FJzDTa&X!N-cbgI{mF6p>KZo=62TRqWW zj9^7e%T!{VCybU0I^^Z|D0URHRv-(vs$fLE?T!`w1lWd`0BBw&^MDC&ZGK;9bNSZY z@WfOLl-^ka5dj5tkr;zSrbqi@MQg{a$!cYM_Nt4yyA=UF=p!%il5j2He%{Ef_3;AC zVSOaoy7`uT=Pha(H>nG%Pi5FqM5EhbzxpZZquT&WXTaIorZUFMp}&`dNT>*l0qAc> z2a*0To><8Z=Yk2Rjq_&bbsIlQm5Xk(-LDQaTKhh|hwN2RNgtfl^2nX(+y5e34%p0@ zD~Ziv;rnM4}8lBGbjq2kY)kf{F?q2NJ^z zVL|%D3l1}FN_$`i)}#7#{O`v?PhD-Hc$N6}oeJ!_N1?S^p3++s3WHQCRY$MvIJ}eQ zUU_6T$J8|Fn$2yfVqm@?8t5%=BRH7PTZoo=%K1c7AyUws#&XLa<=RHR{Y&y_yQ%(b z0H-nF;C>DW*dPdw2Ujyszt)_lVF#}nB`jc#syr|+u{@W=+wL&sMhZA^f6V ztvc_7?qj*qb3q_`j1=m&HZ zCRN(-ijJ-|;pT11us)dz2QCDOtHSalk6viy2SQw-yiAw=vjZ>5gF4@e65rsLAtc&V z>hak?&q@#yH)8yrkP7{Ux;|5U_*+y6G?6D)_&u3(0Oz&48xU!E|Dq#N19_)<)H4j| zerE}37Xu*21^LVyzOufRA~}#db$aX|cWH-Qp*%aP^37R-a1FF zIc}x{t%7s2lcTQ8nh^J7389z@?OxgoPj z{6|j+sN_y<>hlY4TKnHBJLh){Z}j})=gFG8trA64hKysU$*<)JIAZ10>nL7u4J(^# zlNRP_lc~4-oU#==%8h4|WZY#TYRuD4&nCt&Rt+nYEmE(yk7sZ&*!BE1Elvcjv^Hd? zijaIGpI=B#HNzmX-RS~cn7q2~qqTe}a)X`|A9kL-brAGV!|Dr{_(HtT@j~-1QqMV^ zyL!o@Se^^72E#5nMC+teY0V0pw_*?yS^ij1V9BnUWT!i%bAx11c!;c~-bl2O)%SXS z+NcUfVF>&sm2g>*EqJiyJEUwJnle1jaiiwzPui}dzkeOaqmion#xXf5b|4}*XlYnp zq0O-AGM%)F`Z86Vrx;qVxwIt5FUReRL$42FsW zfD_i1nf-$W$h2ZjtZtA;d7tisvef`Zp(i>@s~bj&M>B3 zTn;(j`=4Sxm?P0Mv4%-Q+cWs|eK7IP`(_1j@6^4V@TZS4qC;0NZB81{jQU;Zc6&0k zWgbXfH_9{0t0#;^N(Xwa0vrkrwaJQ|0{zj@;pl8V9mUPf1u2t-=~YUaatV-|$Fo_VNUW=v%7Wm6?4jrl;`b)RO6 zeLV{9VvmSEB<@#@Vb6v^z(H@xIgf_(;|Z_JE_|D{L_t;)8(QthzyN{>h7FAr^`BbX zdF|FE+q#F+N4Su8#*JAm(!GnzVOI;yGktkO4&iFkygxK&vz+bcMiDOvu$G(7YUaM> zgQKDCtZ(#OzD67POEgda+&#TM)}YWg3tqm>xY=t*Nf`eTDfN&TLveZ2@Hk&&%$RX# zzSFAP!9?B2G`vx|*(OA<--%)0K66pd=x}&jKXB-&5~oI3vM(4l=ztG#xLM>W;8ISZB`XFQ=! z^C{!^YR9_INYP8u(;ie9^TxWpg%rjoAXkjUtmQ8VzW?CJgBCg==#n=kxKy%6G*3O2 zzN*ezdJ4R{m8cJKQcKbg(azI-V))oPv7Xm~hF;<;v`_L&DD-slH6B4&XEH?vmwn!l zMTx@MRc2$B&%zMs>#QyIK=ykw3+&TE<1j z;W%*`>e(Deu7Dhoz2iok+`H>M?VXeJl>9P?E&(Zb88(<$U0v}rx~`wZoccxFd(dOi zSQ!&Z^mWW*LV;_FcD0HOwU301aH;vrVJsFRFp=&Ty#&uq&Q6u~ANsU*NY=RiYAa@S zHc+b6mTq-@dFoh+#pSpokDbcpct%H!L!phhh#hM;wGMH{$&(T6^KE_Hoag5LpwP`; zm9-yf8qMHc$RSvo=<4)k=46yu$iQ5#m8F3O-*(N5s~V$Sn~w|A^ci}+Jh5@=v=GLf zlw&^6EFV@L=RMn5+V5&>X-#{aRFNdX9xm;d&6GR*P6a7x<1;1xxEjj<48uZ8-^%43 zngbD^JEfO*!+Qz^U*C-BhOV0)I&$4nKK6TB&itZ3G%V)-)b6$9=c)Pm?E_2IN>f`J zKxqTi`{4h$(uf#kqxKn}aCJ+P3`O@@N}7^#g?viWQxTin7j`bEQW9NXg~a8g>eE=y znW-Cd~@8amM2TX^r^1b(0F2iL#1ZB^`d1WC1#tKWaGX&bB}4bVOk&c_INPS&y` z7vQ;l?jeI)PkwlFE`VzSE3b|p*jOTwpSB~_J{1WNkMIm2i-E*dL7-jh0x$;n{HsAU zcBfGX3Emy2vlo=Nk_dQ|@tj=gr-g|N_YfCQ7qQ?spP{~Pq zVuH7j3n$Wjut4i~DcVA!<>Kf@bKH*w+ z<4((shty93e!F4QNU1#xS2L8M1$8bGsPCB6c&W%qYB$TU(swMFo}H6zl&~t>`Ai1N zZ;UB~sjqL)5ohx$U=WA_d8%mQ? zu==7UdQ3ShZL%8H)o)8itOZG19+HjP?quS0dv%iB5z+Lc9O(ggZ!F7AWNWkK5&O;u zcSCU2`Cj*hVg-uTYSr3UhSQDpmf`O72Nh2V5;AW}s_Rkf;QP&Jq*iYVhT%CN<9Sg> z5U0R<>kch{N#yzrdTkfmU}@S%p&3-fS?rEUr4r||G9tf)FNYg)e<9Iq^IF>aeM7u@ zy3KO<_0Jzp6LQlD_ahv-lOWD0%gbCJ40Us&3#Cx`IM&o@52fyd>(aEeikkjL8{Yy& z0z=!kvy92###g;cdi1sn1@DaC%zwvIM4J*k{7E!Bwa9sAG%xdg$a%Y!A~hd-8K=OE zG283${=<@wr~C4I2*IL*fole&6aFmfDum;5y;I!w5V}r#A^SjpS?mc;$H`hLVSI;a zeJV6*dzc5h2EneTXijt5FAg|&+v9EmnDzO;Uz4A%jYjv;1-aUgd`(So&#@F5rWZNK zcPJ2FrTMxUjMEa;7e<-Tb0UuL8U)>Ql108tpfm3tbq#@h2mM?8Rm&5une9g1! z{k00;OA})E1iu+Gudh82%Iv1jdD=nNX15wjo^KGNrgDQ>FL)x1R#D&0Ef$q~5zN#< z9*KJSriG*K25(tbbR_niwU-TajAX1zg$OR$GgnGC?2Q&VqX$fu*v0y332SRFn@Tc- z0`ws+J5cd`^Ml`m9QAWC(R=&!Xt19xa@Gw5mn`~Rm^U0T<%zs-mHWRA^5zXOi`8Ad ziQQbtAde5#qn7b-1cSxJtT52UFVcS$;n%S2oiYrCmz78nt0@!U{|wCm((yj>4bKCJPY|5=x%C^14A zi0aRSQ7#tTTCRc35$lM_jlgh6n2{Vd3ZM8Dbt5S=%^`~pnoEovdmV9L38pk~oL}b~ zI1jfz=M+f{9(!k(#OR2FRj$d#lEMS@%M=Y)Ak=?Ugy>`&ej5QPs>&G-pHbFN;r<%R zm5TGAecyE6AuQs1ltN$10eA*T`Y?7gWLQ0!%4VxzX?78cg%@DLk*rKh1%TzWlE+$sCp+IAcQX&Y~6$B`yHR1)BQrUx6mvx8bIdl6gf7PtGs2N>a zQ^kFxun-AYP?uj-BkK31c!=AN3#w14mmiuB!!4N`3UcLjnIRw?F^=#~BE?ck8FLz(Q{Z!8#u_r}*xp94DG;b&XCo!(k~ zyU5;HEhykdDPO(ukj3ANgo$?A7L1yT>|Bjk7O%-O}0v?4n>$~KdHhvkCam z;t^Lv-{Raxk65~U*?FeI-&)hX$@^rCTyrsj>AyFju(0rD?DdGR>DZF5XL%nQ-Ocmb zC^T_z={o2rM|N&`=2tEx^V<1HkMc#c++-WYE0vfhyZF~A8E$@|hL*o+X>evcK? zd!uFGJt<6&H!78U{HN7fOtt7@tZV*>>FB+HDfB?U(2{{gSH{)f2WVNKO+6#P!Y{M! z{dC2hz`o)~nm8Qh`!Pvl z(?Wq$zVeKR>zY0L_D=rj@d3T3oZh8j=^h;b21$vAYK$d+U z0~^m5Zeg3CDm?g<1O<2Y3kOp_`!@gO9sldNi3N?I#WKaU8QGz zt9mK#GMDQ?H8dbF;{&hXIRebFAdJ%1LJ2c2@wg?wZt@B#FF8WEi>Q~XhDNPmen)Tg zExRC z4Z{UI@zFXvsu}LEpcut!+}$DZIlj(BKO=2nm;G?E!^33k-u)y@d~H3NE=XOz959Eb zEDOzO4Phr_vgGrIg|R=CZIdT*%R_R{@v7mY{D10IX9(;FK6$({3K z2IB=A*4m6ZJ@@^o80T10wP0NPk4^pG-^`{e+AsT%#-bRUk&~Y~x~`vkzDT2p{)zaj z0L0-_UGcvDY;4iG^!k2n4!nmGlUHLI=uvnJ`^Q4z<7n;2cSl#{b0bkVm-P^sY1w{ECN!T6 zwlpwLW(l6Ty~spbfMZ3@00)^`-f{EvWTJ(WoqC!>Vx@llyYccw7Le@-(#y-lQ^}ob z@U)@w8&i^r`KkHS=lZ90k*8$0#D>E(0qu7C*~8E0r>5ICI(M01dy|&V@qy@0{&kv; zA2=?(gxlv~E$v>{nPKg>XDEL|YyZIWUB1ttPA_Cg{zTy*jKCLE>?)StTt~}YJtR?A@RbG)<=etRLz~+g^D=On#2P~?6=u2RWem$k zOR11?)WMK9DYxxyB67X>BVWn&58(A-9{s@KmxZ}UJ;-Ig#sY;64#LzRF3oI5L&J5? zoHyf!XGVBUv%0MMs?a*N4t2~{6iVBcJQp8Y_` zYR(n}p^ejdN~=yfj9R5wr`+3ZnOK+BCT+`C7Ibb;557E#URU#byNh9s>Djeo9 zyK~kw8X7P%nxE!apBOn9tc`V(B-YE6EtPF8QmVjdL#gvrbTBuN9tWS-CVWfI2gHkT z)u%S83pvqk6yeeoFRVV7OLjb6t55KdIY%qwb__FSn*WyI3FSVOXE^zT%+#IxfvSJ6 z+;p`8b%z(GeBr>Kg7~_x_mh70ftM}D^fxFoZxP-H8Qvu_P0qw?V>eAS_7f4H{Fg^z zdxapEYS~qi!qZaU=vl*SjQ9>oJg?){Uvj`CvOJGQW^eK! zEmOa=>D(#J`eBVMsXS#-ZbpYynt-5}c!FfCmtdf9e-GADe(QgyV88qPpZFSF)i?hP zeaz}gIN;GyYS#1Rd%VA3wgANXR|N$guGgyK>kJE}Xp)#6DI|$PK{Sm_av25H-+u&W zQ7C|a`l2^%%PNDzYg6{Yg{Nxcb?39?1V4IJQzh7FfmkEZk??1hmZul99>b1@jayCO z>KRPcOR1~rVd$h)CI~4cLFlEy1;@F< zj7WIhypEMaKV`m>UpK#hE&9sRveRCG(lHz0A(72;z+N!6i~`_sW|7s_NF)? z&SzmDyJ7KzT!FIab0T=hMSHcCSXWbQ zuQ{~*T%<9$qirb&8&{~-EBv=KC>;the@d9kQVWY@FDW2u3Yc5 z5bolcat|tCilU)!B9=}B7D?F*$DLG0&IRT!@e(O?;`XT!nWfha#eazH*qYQkmoqti zU2gHCu2;!HWtsFkJ-H*{j#r!l1uXJAUPU@JPU$m~wn%mCF*G|KCnI)@FNrw+u z<%vjBWtgW=V#1KS(JRLjw+zFNO7vuwZ9tRQlJv-5!G~b0*;X3;8`J!cj%3_M@F#;J%?|?DYe9Ze;!@ zqkK+kFd_7%lSkVs^=S2HZCm#6?6Qf?AcK9H(KJ6PS zehZt1=an<{5Cu1{e=^zRRj6NmZca#)IePxMN<=4VK5fh;s1?>#N@A;UVLCJxm90_l zjXn&ASw*`htKcw~EdZUUfEbxNmL)?vxWbMg6xC=I~`wizLm8}S1Wf`4Y zvRR>JQ6|NYu^YtmN^dtCsc0T^mR;a`9y(C*EhY@5$E4E+8r>aYvdWvbXimdzo&mZWy}h2iJjX82#vt{? zJYs&tJo^+qM&6{=xjs$|9Y74mdX`H+lJ*8TF3VYGE4Ac1w|d}aCiT)fm6PimrXuyr zJjDrqHBZ}tAH4#;@)JfzYD!BSi)sP6+j>5v{TExSo=8D@#5eE+#mLp$TF@)T*m~W` zNTB&9oNGH*vO(DDplVMVhOP%-=s)8xQ(o8hTpwB2(fW5})`LkfunL}Pmo%Dw=N;>J zDD+~Zn1;hI@$=bC0U06fK1=tHleQ1Pr+8pCDkShZA;HR9c~JsVBYMQ)+D_agccl$_ zZ4~pqW%UlW12rsFAm`C)EU#~|H>>~3NbHrdGdeHF0#Fv^)srTOo`ap>(@;qprmJ>o zEICdW9D?yhI(?|GFJ9UQ*lvfx=Q|vmkq7*tS5e&%nVZ8=d^n%pscaq?LM z%FD+;as!j7&kE!B^|h8#I!I9+xZL3?eN7=%-pxP`SqH$h>_sj0Lqc2q_W>PrK#v@W zaM_#9N;VRbb*;yS3wGs^<$5Qi^Z9A^PN$eGZ?8)Lq{45;hx;ZD0KZdAyV$Fz$VfE0 zZl(@F3yr6NpzGZgYtvG{JzG7jheOuuX|^m;kNlJQqX6d9 zG}UbSN0*64+ZEq{s5>ZSS+unge5*eWu!-oEZh(AQik>_5vewultY5sx0$a?S)Hw9+ zz1P&hx_*Gq3qG_wf{GWO=Sgo|tCgz(zoRp|9zEEUE=*uHmN|CEuu zqv@i~Hho-jHb8y(m-_3i+sn{z@Ye{}A20U56e(}dknHf~%13)DguOCItLctkw?X=@ zH|J>8*(aD-kMB_Y!hXQjk4%U)UOkDETs8@KAh_b_@46Y(Wh60Gf~A<<(#59#xS9h^ z6hx^<5AhX=BLSat|LxQeJ)`~uPsfS?;FgPeuO`PMn)&JVoWx%`rN2oX5ko&? zf9*t&AX@rgDbN3hUgeKuMFjg3)*F+5t9&YvuG2k1wT)T6?FSgzj!vt#-Pa>^*Mm$V z$}q4rMrti2H>g!um?gc5Jx#pH)Ief;t{AZlCw#wgFaa&qrWzq%BizZIg* zw#em-^xA)^bl$!HwIf6R_0J9lW<_4C9g%fBUDOM}*sMFn57Ny|@4F)~NM8AF-J7Sj-UW%9 zV#59B5nbW1h2r(Wv7M96rFF#8cklT&cA*thnXZXYQ_KfE;-+uc#-GZJtNVu*HwY%m zg9l*0#Jf8C&rRi{r`N2Vf-q3yagfAl5$ew0-kYd0{o`goLlF7!>k4is7mv$?W{lHA+aFVniS4XV81piHB@{(`L_3Rx7bAdd7c`SeyPal z2dB|A4!FC5p%Jnodrz(1^sb;*yGH`mA0SXfbB%QKaanSvvew4N{rs~$ z4S^EyQY-iTA^HT);R;_iY8anL=QH(1Vr35R%PGN*_D#W$DwGAm03nR8#sS~L zEo3FAYH2{FH^$RwCn9iliIj7mWHvl;{Bgze{7G0r*%+9ux5^R;*KUB`r^4o!BjZ)4 zFK!=#wT|26oJ71-)^ZPQyyE-{P-y27@>i!ZWLgQPX<2sVh_&ya`G)qEm&e=MS-E-k zk9xe<6>$-d$*oz}ktH7!b>`^Om#vyB&&85yH#2OfmuE^C;bN8ioTy8crYcjxtT#6k z;6gB{KBS^|{g(efy#V4|lP9l4{2d(_EY$pW?<;Z+z!Am%^S7_FIA+9Bu%>?FA=4Vw zg$~mPae2_cuMG%kjydX{$M$$#Ddkfb?@1=|qfHucubuz1S_rCKYt!@^u5uHK7TI<# zJx_L^G*7K7^hl^x0j&<bc?SHBGy2HD)FNo9fx=W^M|LUmXMK$TVj==wE6M^Y5@3jMwL9mjM zuh_`7fofofKUIl{9fxET!M|vEPgl^T zXLEYRFVxKJZu+MC9=jpV21;ZIU$f1}M)r7kqKVt+3bB_vQ+Js3p--h^%ScuUX_u?3A7V*$Cw}gj zcxkMHsIso7)ZLJU-oNz#nvJp!5mUL~B%8x;uEQI*%1Ao(()Ri6j06J0Iow<<+hR;K zC!EODny12o&1*SwS|zC>AbRultMj6#+z*V|KrytIM6FQ~|NDHn8L<0Hl7a0|#fY3v zo0TY4%jTgJT!*8z-f$87tP_2u=yoG{CG@^VrG5KK$Sq+Teq37K{mB=!$(mH-s`ZO8 zS&F_lXeI{NLYfRmeBW10JbZBr-DvJ?R&*p%3zlfQgnCWfK<8FsIDckk2j4RV6|HF-pfjlASuD_`XQk z-BrDn52^sIT=%~UA33!hlDa?7C_JUTo{kZy)rFNjUnq66Pw0;SvAKh9B>F+@347x< zY{A255u4fG1~uSKdeF43Xv{WA0eBCYQdUEtNSl#Fle9TZY0Xcnp@Zz;d|6?3Y{tV~Fi2YJt z=l=B~`9yX0%lw26pf^5$}^mz@nfBI_G+n04|QF0F7xSRIW!j4W0~1(FQ0h| zzamaobBQp~9Nk_PrMffvTYRn8D?&B=AZ09Z!v)6g;ltTp_(Lh@J{Q;&rXZa&k6%2rV3vR&nJmkz>OxDs4rWbb1!n z3Bfkz4e3?l)`gsTF`o32@IoeaK4S%h)aTUlxxb1#zq~EBn*0GVZI`)Q%#96Gf08ON zH9H22tt{pcK#!^<_BwDwN9PXkaiZbFDcr`UY(ob~@!Xdr%F|(3aM$_}PWBCtKsH~i(LZyUPPF~s24~ieS@`=;GayMh$cRe9Emdr-&h@t@2hVMqFNVohjwp`|| zIAsN^OtwS2mrDIQ6+ic^`DCDH?4|zrO!_#Lrt$k_7#CVW4hQ4cGGk7rB>+d5nWUOt zdu*c3t&>&v^l~qct^gaiqG3Sd9g6Btcg{%Mtz06R3AH*By&^uQWruhTfAR| zd7Yj{1rTYcVpsRLCs`3;t6tp};pWWg7l}|f*>83EF!1~CD8^&y!0k%4u7TJ^2p%Io z@~QP{HEP$+eQSZ^oaDSO-XjLGMMqG*^(O1%+S~S?FH1t!&`rZ7*WY;UN!nRp_h3kF zp7l_^i>yR;3rh$lXt;k*c=PLzk?=G3W390FEb+g|HGfA$)jTQCDB;XZ9*y$Bay_w! zX})4w>ekP2*^ubW?1?f9%y@%!LB4~rv_`j+wnBGNfaOR$+jGe&?j$s_hDc_W1G6=E=vxaWCy})g&`5PH>Zbk!Z)N+er@xV|29E<6PENc0#Sd| zooSzh5iS^nnNK$BPw0yC=V3bcOaz+zK>ou71jQ zJlBmcI@`p|zcD8#M92ruds5nOXyTSyzO_4j!`o$7pA+Gk;wFXu#QHcEMCYxlP6-lX zv?zy&*CfA0NAlPDQx+Y?Nw@#IEc&%itrq$Xe!Z`H za}27L%nk>W)#@6JvH(E$@p_V=NOhN>!*!2G?=sqCtFQEZxxbx@>m7zkZ^EzwC#v~z zSa5we#WytZ7~?jMnA{3Wmi4xb#2+^0xO+h1?X!+dPaW}bOd37WaYkBpe(U+Scmo$Y zyJEEkmCEU;@sU!bO(Q(bF=fs;a6OgWgM*04`S;7E^0rsuI~_L0Up=oaC>FLNEg`n; z0ty@QtxNfbmBBMA@ce{M{X)kr61ivj044&deb0%d+{mKQh zB$(uGvN%X1b-6g6L`E=4TQB1eeddTya@bk9+mz->}(Jz!X3#Y1{&RELJ<;z*`%*v0oEI`#fZb%qWlUBQsIx~0hHe%Y5pW+%iri8 z8_ad9WZG8_&$XZ4+}ne!>0wFjr`rDzf@icje)b#3^|Lq;1Z&114; zWy!>yYTJoBDD_NZ>|$muhJtNs7O&dLwgIzk zV@ph2hfW#JpsVw2Gi5~1)?jA;d9H8jIUWPbQ_ai$HrwqLuoL6%hly&(DYq*!v_sOW zerGU+EfaEYJGm%1YyUa+Y8H+)0B1R)?eQxVn5_$vL&=`gPdS z9T_;pP4^R0;CHEK4ubhYsc7gqDP4WJdfG zL3LZ8S>V3?`jaQVGLFC&{k~%Hb@l2`D_cn~=efBkpV6g>Ae4cd3>=Oj0i_L3N`c`d zVbZj^M;SN>rG-dP41KaX2Kjtw^on57lQ-wiv{YZrv=neXB7`+>HPNNF`_P1u1V>+V z3{`IUR?Sz}PP;7!OliIx{SRR%C!h7;bWiHj2mLZ^975+?$ARycub$*gl9s}<)_+|g&VzL@iq5y_=) zOeeI&agHT1+Gm_kH0iWm0~j&ex!A>3M)J@n4Hlc$_5EFN$U#}&>1mkLtY7}*@H9bu>5ODw$A9syaW`gYMP)W=Ps z5YV{hWquV7(ha=zS{K29^_dyV9qIQ+iw$J$!`26axnICQ2O!5_6EmKeMFgtz^up>M z8v{3s-dTh{Evn!v90a6?;AhKCohFjG|5G`4f9;A$%3S#n_Lnn{m& zqIugelVUzVv@Uywo#PlEN?MZ96bJ{B>iK#2jc@UFx4pq|@Uj+MZnvASiRK^*&}2j7 zN^bh{xbB>?=SFY(f%3c~ZM?2co6k=%_xcFSz9C`muFu7OqjJ zdY1%Kkmg~LV4}9yHYLVUgwLI!s-eoAVu-2z_xTdOx8mx&dt|OS?(Iis^wbZ9-M6;WcaUic7gEF%_W` zEWO=DKHX|nq{Hf??~YIHZfuE_yjX3s{Z>BuUblU!jU5Tm=Uvuh3#z;MKE59K} z3tBU>KLiPfgj?b5E_3xgPo@st(`*;kpI@cijMVQ?NJlHcJL7>$cI6pP^WYs9^KD(d zO5craGJ!UeZX|S!YVEv-k7J{w(|N_=EkFu6_RCmI7Q73Uv%cv)&VPP1Yz@0U5s;w@ z$1Q+&e6x_nnm(w-w;j5eb9>yfBrzaSuKMozFxy;bek06s1L>RDZ@A%@-~5Dx@I?Oh z5C*+z!{?gN`eS$gEc*7}M4-Zf69^}+cdKs6)LhCe_v#G*g-G#dDs%)djlG*esm``9 z%qrC>eTCRFd^ojcBlc1iD_4_pt%bYzP{v})FCO-d3EU1KHLkldle%-HU-<1*xqi~7 zUBZE8@l%%2U2p!7ZguT$dm2mH*INh(3C#U}7`qCfxVCHy1VWGi!Cis{clQv2G!on` zSa1vO5FiN}+#7d?27)ybAP^+DHxQ)p#vA8%X8!ws{``6Kri!9&QGGAB?>%Rqz1Lo6 ztp-I?$JRwXRRoul6c>6ASY3#rxS!}>uX8#6ZVH-5b+as@BWEk$gzG}o=1GKExu}n- z65$#b1p-0^hl^*?$CS2w{YP6+(N56yqCE*br!!CNHWdgzm{DycgE~m6-MpTiYHp|U z=$vpSUA!JvfhsCPRc#Hm(*2=1Ki(R`Q*vV{sM~S(&>@S)A5LD|RAKjXZ$T#>Hg;N) z5c5Qy$c>2)-Vfq>a;Zx(D~GY|YP6udaReE#^IO5@?T^H@@fkjk3RF1ldWMRUfe2^J zu82e`@HF03`dPeS&I_5);31r=-)$D@c*fO)iWYEcm15Y!6j)w9;m(qx>6V6Su_Gs6 zK1vB4hoZLUpku~ernwXL zEoynTMo)TYvUsLhSsoV`msPt+a=CNegQ7WOnspo)=9k{HL*NC;KY%}+=lZa>1R}I- zt%e;l7q95W6oy%#>>43C5)B;uj@>U8OpF&M@+?Y9of2n60SK{UrUOW0!gIMN&aEYC z2>CbV@EuRtl`uT+UZC zBdT1zGT@VZhiT1S^O%B;_P6p+oz*Ydd=O9}J~ihCUMh^*f5KorxubCGc+W7*-Jzaz+F$FB>dTXzr($B>D&NYeAVS@!SDdpIP>f*^5px%jjmot!p_c z!X)6cp~omvFOeK?<+tfL|0kTdFwz<{pJ#w<r@T65dp z&sT&3c3PU0^Ru2)xjDPn-*()mstu(WKbW+iM%(G28~X*|;T#^8}~EM)bF z_;~rx-t7ZdEerfn0)fo;UJ3XEpN6*VqcrC!6$7+8{K@d&lz-syCz=J*FaHWy*8CRt zT9zCNda-&6yZs{h$MZvsm_L5rp^$JH=OMHRH7knGZO`H%jqon>OoRF_&tDhj-L;l& zSBakIBP3aN3b(iG63;ev_!ud3Bm;#v**93;aGEwT5OHMzB?I4=BqZ%+u-)aiu)49#tovW-x+==7}IxWkvk*JpRT)AuRB*=BR2%v2wJ`X>c9LAte`gz;A zN6%_pR6R}srNuKt&6-2gv;K}y82O{Ui;^Lon6RVIHs!T7_o%f}MgokjXz??R7|m9D z1(Ux{o6Zy~s3<2rp|9{)yR+S!_u8jLxh5dD`2ACy|7lzQ9lHJ7?+_KUjX6bztw@CG zhYDazwMjnUryVdMy@Ft3;`g0 zP&I{tC(~ASzi-U@i$ueLVy>FA(gn>QaMl{ z4xtS=&+p7}Y|{Sz6koz)H%V!mRj4GEVz+PX_PMH&8gkQB!20=tA3P@2WE3}`LNnnN zM*Gq%2}^y$;~@89EBXrQ&;UXIHi=!>1y*luiD>)&Xjd}ducg>rpQ4j=!O z|2if=Hs!uA%EolgjwMG8CgHbL#=~x1b%R4@!PolrXij4lk=7ZsRvq?5?XkQFdN>U0 z<2IdVXcC^6CB;ESN$q-=Z^OMLuMxso1@C8*K6_sG7S>@8TPV0FH;_x6VC7Is&bVtX zx+o_udSM6akatMH!>79#qJDs8walt~cr|<^_Mn#Fw*qg6X2EYFf~0b8#G&RG?~>7e z`#}7Y4beaO6di40zQ1bW!f`TWok!`?+pS6%A<>`kHd3%R?(Q?A|1D#PYy)YQln6!z@;_rSnA zaSNNF)$4CR*iyKv#y!#(R5e9u`} zhnwX!V#Q5pz41uwOYBYRLp6sqPQw{?(tAg#7Au9gWxvE`uH$X)z%YrvS1lJ(HPzgqho3;Z=X)1j`sI_^W90%d?8?h)S$SN#;??{2MgbVx zao1-9m5LVQ753=wofc6B)PQoY2k}NT`eX;`>B^dSZG)R8W6l0#@yn3V?RWfn>Vl9y($dm*A} z6*uw&#w8{d!;pxo*42-2#q9YCly1ZmnociEZB*-h2rJ5zUOoX)3#*i;VQmD@+eQbw{g-1j7}+KaxZzL3$mVTcTml(l*?Yr7y= zh7d%_t(blD5H>1$Sy7*ZMe>~u*%5GQiuvjR7Wi%EpbejjS(FtMms8?=2e3+0mZjo`Ucck$JM=x`WRE^hNk-0aAw{7vKGFgEj`rW~-YvyAQl#6=nzbk0OD7nA(WV^!@6CDcJXtLSpFN zbSTzlcTf_PL7D%^j;GxTdH#`}Y53LMcC`YObo_afcNRAz9;Hs(6zcJ^IT_R1kms z^l9_HqlP^DwBr>r3-ZV+3~;vj^DpDY?+=)y2a^qt3=f;kzt8&kSH^*VH!L#A2^JA* z4D@NCO#e?#=&B$zdwf(ewPp05-B5n;6@W5R_AeZ_{(F3Mu@Q2|#l{APPr9=HMKJhp zgC}o&l&zGM6fS#*SioNmvcI2FqfAR|ZEkK}$HT)@p%h$g^(mny&<6M{e!pCxi=LW#Q*mkI|V+|RS1DTt8J=(`B--r zN)|ecFxDjWW<$kHs=n7*LU%?uH+r|BE%CPWXe0*<^6~k#!IHe>5%Z7;WtS$#-N&bc z2;P&4cx%)oN5`jOk!?5Y%-$dV7wIvh~FseI|Nzucl>m~N<-|GGes_w|cMU4zJ$z-FT^c21ZLlGe8p4i82w|NV6Q`j$)iT%wAaNM45SOOK2r;D3R zB;-+T&=0#;@^W-cqL}oO_74d<1LiF8*Qo4|aVUlP&p9&L5VH{?7vcwp(Hk!`!hThq z=CB;8K3Qt_z#3Img?;NdVL})Z{_`MG2tXky8S&YP;?x9HqrT)?zumEjpVAR;@j?$Q zX+PJ=@i2J*tx5Y~TcET`!d?g#jN2Lz(#j#)%K`A*@uK|lY_0zklmU_lh?s&sbS+1> z!rcdO=vjGD$Rz65Ce6WS0}EFDH-n~?>=~ZBF|uMpJ&eYW&nN`YNEj)a2ZRcK)*9o! zQj9;JIj`70*@VAaE|5AQVvCuShtnI?dulg$xx74$la$TOxl^y#Q2C~?Hje~uxXv_Y^Uyp;?Du3v^Gqv+jT-?;55IuPZ{D z_ji<6Q7PS5?#zwyIXy)`@AjVgj^gv`8q7xfahTfw=;TWzGs zz?40`y{?-RJ{`q9R;-HsF7|VSt!-xy)SLGF@d5*+vz21U6lfY{!C@U#!yFp~4F5s8 zS=@DeC-#NMU1x4?eg|#F3G$4{PN3(rREc^294wy6(HEl{Cp9;&NYb>yR!pmUk&x{^ zBV?xl9Oj;JhFITK4Gau>N9dytZ7!`XY~tQKrJosM0RSyFgCPh5J&W#*Iz5E#gqTED&u;f*W^Ii2ePfCX?E5_PQ6-{58FIpFT9;qkj zg-@*ZKr^IO5v7rAzjGR8b069;fU9)mLr@$;{-I6CnKp5FHEKDs;i-?VR!Yk)+qcGy&OYN1*01^!e7dyegfQ~F?; zOG6)P#ERgja$TBT0X034uID-rTI0xJU^KoMvL>f=3H3?`t9`u&!r3qsJU1Bip~9-r zb^`D)=yZVy`w)VXM+ZN@Av$F$n8?z86M|48b=vkQBak7vdynyMd2qw<`17>Y*oHKF z_GzYMdHomTiLd#->pHIq*z74VT;gOZ9TLkgs5x2pShQM0U7s@b9y6k8WCs^r%aX8J zq5iewI7A#-3|-#%@%8=kVnW#-Cs~WnKqQCXf0vH7Tl=9FT2nzhIwfP@ zOa8WihLOI%TqP7Z(SAbGU0e1*8&t>J;sqxj1xg$zLE?b*Z|oE9VpR~j%?#{)19&7+ zpFZ(B_5j;_&rp0_cSGC=DTS=r7F#Gq|0D6sHr|g zGo!J^>n24Kc)xZJz8>KVY}H=@~BHl-0s8pid-3jkVHVgmPljd&|)w06{R6|No-# z58p+>7$sJfQvsGj=0vUUfw7UZ5*tzgARo>YR$5VrMh${H>4(ioT<-*4;HbV^s3W0P z*nvg`mUwKzs?XVeN7xo6MGK-V$E7^xjDR_G^@K7%+4_&Y^gVAtfdFIcd08c4-SXuZ zeG05|zOYdae!CTdnI=z{c(;+|PV$foAhz;FnhY{Ye6xb$oLK8N_VV_37t0@sG))?h z4XErV^5omQ4zej`5tuTkotrXw zPoQs3YW7!er)-^T-}By|W4C;~&ruTP{n2Xt<+u&D$OYWPsVxTgR}v&<){HZsQ4ny< zKA`J*;UfH1!y9*&xwqn`F9T0jjv&nh&+rKa8%s-{RO<6nRJ8_$!~z5Dg&l6mD7~Le z%HzJQVshj$zkls7?!qs~VfwKnQ>%{7$~Jr9&pBO_wbPvj&xim?|Vfo`$S^ zNG&^^nVETl`&7kS$!1|eM3{c#K6_xa=qz{$DK_?VLd|!=r{?CiWi>)ZmGzmug_fWp zU)9Q;JAcMUm$o-liwk13$`@+Z1zl~=S5vx)QeZoi$|G> zZ?*5V-BzQp@5_1U8T4+%RnX#848Mo@6+6@TRS{4E-5z#P9tY7_tcNt41T*HYT+0tr z6#iwcgGj%qklnEqA8ZQ#gU=#Ny{EX$M(=_&A7nSE{%yke#|%UoHlrpWkj41t9PS^c z<3AiCD&l(d+D5G)iuQl?ZvMFFKb#SdZN|sPC;FH7(oTnhh^pVWzdY3uY*I7+Qq@5} z(PHgy2b^-7n3y2nv=jW03_7Q1+cAa-57Wyyl;~#Fx8y$@@%lK!je9#7)0lAlxaU{S%H%r13?=*-e=ElpJLhf^NO+<#u zLYD*p=hcTuI-^b;r;j- zG`k4p2TA5wGwAhZi|5tEBxe8}w9?VxXC53WOStyYmuJtoEQd*WK7I{nBnR!qY9!mq zY-%mciO>pfCj>I3`j54LWHGrR{dz(olYGtLVzwxHck52X9$Whp5A{1tS}9`(maubf zfqvE%-oUJkdwdRf#vQf^AWIGVT^A`>p^Tr<$ccF!enSA;czAfh2**r%c6Q~J@Tw{f zBV*&Wot;mZM69`Q`##s!)+$(eec$|j-akB@|2=eW?Lwysh#s-@$w z7k88ASc9ArwA>RgmhH*RX_z6~pUh_7dOs@=_SIDs7AXT_xg8>mdSEt&T3Xf|s z-i*o)S) zHx#3s??C$jRr%88K}fDt>|kk$I&&+rDC}6|JAz`+XJplhEYKvUlxj!+0abnd{Z6dau-8+<bRj)ZeO_J(1mDF+xEs{^ldUM2^HZtHww;H8H}KQslePOsAX{h$&LIx@MB2 zqG!uV4)xu0Kc3xe74)|RK~p44cnjn97d{m2O!U(Gaatg?D12otHK-90ciZUsY7trI zd+dOkBUMtvGwdOg1L#P}22Hnrl2Xi-#2*qgj!+hEwP7Rv-k{TcV{>3yY-hajUGoW6 zN$m7#nX`u#b+Td=%5}uCzp8J+C^{7$%L8$BrDsiC1J|2;an`AK!8;a@^Psuo-#N&- z#!tinx@F-jov^6SAo5+kU*%|g(d)XE(@*qI3A=7T6+{mt-1Mj_r3xKh@?d@p+P=FH zZucXY;mjy#h?^z_K7AnCOS%0-9%Ddfc!;vS(urf>d4o^luljanYr5ao7neE?^l(d; zz8kseRfAq_+!4)wqO7D{9nO|IYBBS{{5!M$!BYbd8SJ;9zL^JNikhHI(=@)z8$l@( z2x0)n8-*a)CA~vKN)H3kU(>&KD&L_;T=Vju2kq~|l&4c;{$#=t=5|F93pctG>^5zG zw8OuwcYi8gBYg&3{f%{6H z?;SzVcHA_UO@V%iiKuY;+_jw$>?0<=rO`2zf0*=!YNAjjuG;izl9h~0Qo7P- zTkv@8aD>8j4CHwWHPEU5z?CKGl*TzPrS0iMGN_{~1JhrRFuS4w%c9lpDaOzIoUl2O zMxX~S(lcf+%yXCm9^bWDY!?I;yZ=DVjJ&a|Sc!xi27LT+-q;ggxFAS+^Y? z#k}8h1PkVSh5%c+R#|9)Jk3K~g-!bY(6rgyP@3&Ti}(+#R%Zo;7cUF>LH4aJYPS-n zNQy=-QVgiLhf>n7o3a0fWBthjyefy~pnu#vnj2tRuaoI@|7wVnrYF|p;a9C?RyisW zywc=}j|+{L^NRBOUD_ZiZ|fh)+h3EY+pk?7R+Mu@T*jeE2A3jsqF40v`~;1L{lT4;b=L8-#ZA+I zVGbtktkVoKIPOxZUJQ4~y@6q_k}u0j9E3Bir7fQi4HOsLD zsWJfr{l#r|hvem|iwh2eCSJ_JESc2XU&g)?FOcZqhikE@flr}R?x3co{xvu&F%w zq14NiB_x0jB7oTY(^CpV?-%PA`q%jeq8u1P)4H-GLWMArWt3w(7q!M5tAow;c%-5@ zD{faIRC5dSdmSN9t@2fDG`nXF@!$D<85!+Xw>g7={S0|Im(OpDSf^hpvW!$_eEo`6 z$zByNQ!l7(cCZ>7gRfUv`lu^hNe6+GwK2R4zzrX2kNjAR$Nz3}x~095*CZv&CkFPE z>(^FYuiQ$Dbd4xAHF;%fNoa=WK?z4tQq@^R%|42k5L+?%c|Z@BL`lwYgZ(Jg~jyDG>2nNI|( zx6vD=*Fe$sF}+x^I-Y8Dv|qz@kNP#~GY zxq;%EZPxg6+GCtX31wPGvp=2COp}#zXCj)NkJSc|bb$HXw>=$wUPFez3ydQJw6_}s z?|S89WuuCpju!yXKMXw_`(KT`iKc{0DQ9qF0=C8`_7=a|&;(}lQ(DxbZO zndj=f#<5aL{r1p*izJVZe%ofp5qK3qZe*0Dj`04zgL_wJiEv*G(0%;)vF9SY?Ram4 z?0Y(Wn`F)7w!l;3(W^@t?Z6`uSe%ha{f>y%o2!QVm3(C~vIl0C4!Ze1wpoX+F|ex4 zmNJ=o>nol+8ugJ(e(5%yk#m9FOaqGFS>L}bNRED?J@0wx`fHW*u4qVR)?wncO})t% z+MLgOEv8FSu6lg$_0?f+8WfiW(YM>sY~O4FKe7W^S)Ujd4!CBz>(5xt?^h? zx&mi^rHPtD@>**~Og>cr0y>sjR6j!LUWTf}}cVSKlx2sKH2pOtBM zoO0K@kYP6M@zHZeZrg?Ye0nn$WJ!UX3^lYI73d<^kqQwEoZ1-#nK)_f#p74|tOiR3 zQIiO=KEriJh#0&5CW*<&s%OoFQ>Kg}0waK93o9V(ll=vP(ZNsqFBeO zU|OhIi-o;lzOt&4M zk^=(gf1$Di&yfjY5gAF>A}hDcsAXUfO)^x-}1J54c%blsxZO zQ$@WiVnMd!uR^~HaHPz#p@;K+&iE9 z`p>W(CVH}MfT$lb<=2H$_SX;u;BGaPK31g4jIa*lc8{H~+du6zw&Nii`pF|$z+xTE z)#hWS0<}X7+J5^6L*R$-$Ot8U$Z4qoIGht&fnp{BrV8`Zz&AkKo-*km2~Y#apO?yL z)q>31oHRaP=i+=fN$hYeu*i40{@&?b^krHU`kDXcsK99DbzkCHGMx2 zqG4bVQ!^xG^Ch8gQRj}JMy)5;ygd;EVwA!=gHvw}s&VPzoi@IS3x<(7JD7Ch-D?Em ze|-~9#%}xlCQ24z@%`}gP>H%_ALNQ9~ ze7#k7v#h+*)*<$#f{0bJZ3Rk?BB()9@`X-gEzhO>wy6oPCzlB}cRNDBuo^Y1#u!Y8 zY|6EMti30gYYiGI)$AY_bbwfk2VU=geKi0ZFOo12aCX>-2C?Az;Zh3*qM7=gDT-Jy zcm8j!`cEs021BmWba_JGfiJG~?@kPM>ig<Dlt|H9L{}9Z)BL~ zErcFzsgcLv6WJ@2ZLjHpT!^bBmKt;sdz{L4mU#zm>Jz0;6Eg}DqYn4Bz7Qeyc#U^k z$*1zC3ArY9zCzo-aMF)TS-$BJQ4!UfpUW*yGP(~uc~13HGh2%ZdS7Kf-ZVQa9B5Wj zDW+^%p@hED=8lC(Uq67Vs3;lI<_;>Jy?Yp3UmWgJIP!v}2q2nOK$EzRFoF!v{cv{tFu0g~5+fA|}a3_w4!ij%Rba>}FJ2eup=su9|%7rra%U5QdC%giC2UhcZMyQ8Y; z-{yWQN<4N;Bl?{jE4lnd&deKo+ZU&$ckZ~bg?#zBHXniA=G{;SAIxpZBG%d0G9p2i zV+BExDsRi7qEs)sJ-p)fUuZOaZh8aXPp)zKJb;}L5O+$N@QXX?Cni>Wc-EIf!}&`{PL`=jt%_PDe6ovA9~_W?=RO zIg&katf>F);Ci8%Tu^tFPn`}+5u5c53Fx)nN^Oj#q6bLrd@?eb2YwE`Iw9H^yLo4d zDGMD=Upfoyp*ty)n)VMETVEdor#vJ^hsL$*+@R`~A=vY6gm{H;c$INMxuxKu`P8MU z%h{zOA%_-{jBa&51){>Sc;+plV4BM1#p!>gzeG(Y=6TR}C~635dJgT!n`gjIxegr~CA`Ya!xAN^ z0if0kdL6VrlHF#;gPmFb0#aucbQ^mP*yA4t?jm6~jVZW9-x!~hBC9@?8Tr`}iI7n*;TEZb=qz`*Ag-pSx z#dm_EbB;eQj7dedg7ny|WU(E)R-e_cR6p&cnHqCZcrXGv(FZ0w{F(+l^c31Q)R|A} z@cV-WInmJo5ip(MY{6&x;>#W@Dqk z*Yyfl=jF~Y$NIS11)I0KH5TKW>tpI=FZKQB@>JRdj-H~K6b8ouX~dV-18qz7 z%bN4x`jy7IX8!_kSGit>ZI-M;nQex<@~xjIQRADPnwcz9U7p&ahyiBR(J6jr)KrUqU=J?-&C>+J5@2Xhyhj8T%%Q0N6I& z+Cxf&q2h8qM9TclER$&NLp!{b_j`dA5Xvmk{H?#au3=VYy~GNO*>kz={V+#Z#clc* zsodQ}%D=U=As!*AU(st>uiD+=k_g_DM(S|sn`*+BFGVfBrD0sdR!ai#n4O)t*!4Tf z^UTd_8$be*#W99W^!7Tq>o*7@;Vx3ubzB-fWPsb^5ZIr4cpa0e-lWX3@grLkt{Vb% z6!Tom3|J0CYL8#7F$l}6 z^l_UlxGuYkoln63hz5{mw)%F8$j!v)b1#r2T~?7&I1D2Yu;ugHwH$KR8$kpzfbTF; z4o8yoJ=^BX6jqquoIfSr-dphhsTKM)Pp{6GV`-W1mu$odSYN*&3tIWW>m|1L< z<^6*D{nEqgHkp*)fkE}TJh-aF5TnyX|Oqz{S9HJ4(B&&mWGjwVyv`oUIR&iyil}%94>8^ z(7}O5Nzx}e-IF}@k&^vRoQx{UhQ5O{RLmg|n0)MHQQEW!qG0uhl%OjSbJ;CS zU$$CLE2W8^;woneVv$Xh$Tm6XMdVieMBj%#)fuX-p#LMU0tpm_p8%N>5WbQ<@J zny;H}I{5YPU`t;s04m_h<3!LCPhT6f7W5;rn{A)<|F>3!Xwe<9!jV*oXpI;UrQB=G zZHZv*g{C1C>%WzDoLEVm#^fu3HyjW@n8e0#I)RguQ*9#tK=~lL6&?Z zP}Thf5Eh$m^*6+ZKr7%9w?qhQkT9rxpdseC?Y_rEM ztO1@zf`3+R{=EiuP7Mxgz%qbu$N9?v)<>)MTom^{Hw@ssy@wUJJ{&r(j}Wt&J{8Dp z4Dm~${1}hq+Tu^O)QQYSqAb7P>nLR}=or}^4tV>kwzGdP=OP~-kMZJ7CH7lJKW;&$ z$M=CYtM0#^cvhRn+NW|!988e7jV|;*rt-z@`t(Q)GL&2Fp7H+Db<3?Fd4cndKE*=9 zJlK5l1TCUj?Jj+EnhLaW1r%?j5UMtrp+R|}lP7xNWMoUpC%eV|JxzoymS*m1s?Pi5 zfUs8i66?Yf!UX&Luhye~Koi^NQE2?n$lfvp->5(%K7RfjJoMS|&Q+@MKmbWo{^`%C z8od&X-^nZsJvd~nj9zrLqCULsf+kiuXP(=A$qYe1E82dYI-LG7=Er?#x|Z_$9~+;Z zTd}QGj8F0$l!S=lW#`NId?9Y}yJ5QK_Zurco?OyS69{~vN;7XGp0Y%OK;gJsti`)%CZzbkvV&c{UvTC}FT7 z6kJ{g#43pnFK{=wGhsXz*!f9t8c6ZUeE@gYVs~+5l~$SZp7FJ^>0`AT@gpg}32#nD zE-m8YY3ZO!(_gp(|J&+0@6>6yc%fZJT2qU<_UpL&Pc_ncxU8eCx$UQ#fzb~`*n+5^D7-F7g{qCmw*rE) zKB${|%3Lg=jd!XoMLv3b$BPT`Yo@Rk<8} zeF@7f)VZ3~**u^HY-@V^a-MdBL&LyJ(}M8JITSx^emzq0>ps!D={o?+$bprpa=pn) zJ266y)_u#tb=LZLC~8>0<4x@=pY`*;AG$K}4?cPd5lyVH%Fqo2{IQc;wabs7+c{i0 z^ISL>gjb#LZBO_pQwThFB6&c`u^=N?B zW!K-^NZ>D`V_a7XQP3Xt0-Vtv&Nzo`~o&P_)mG%8MpL!u;07^Igm2s#hKkml+2KtAB;Z?o$*Z+Dgyh1Ub5Ve*+G z_Nu7YH(iX`Y~t=4{l@`fXA9c;zmT&j6w-ew7oH?`ejNER2Y* z!k?KAN`8wx6Vc|>w-Nr~ZAVoe4(LBXL1e^!#HJ?_^nOew@}O&)?PH}*-AAbmmu;TM zM$2%eF0_qdQ`FyG2dnCHnRJ`}%kKVw#B-iR=RU@p%Z1Iq^X|x3y>wPI&-<%3OyGMK zIx~WzZTeO!Y>D|+Yz1$2F!BA*`N&vyQg^R%65M_cDU1}i#bS*Eh-{Zw(iG-L($L=Y73gddd4DL{- zO-qcR@)yrU4*dDT`1iOch*5en$oixmz^mv^!XB&SL9t#C*jxKHkl0w~s=Gt17Su57 z_lRQXRVD;n)$zz_xy9Od`>W}qi12RL>RnrN-UYeNX==F^0VAVu`5(wTT_SquBd%Y+ zp?9RJ(y)(z0=*aiALxB@zAD0EEsyR=y9*Arh#T3XF3ADhz15n$xRtW?Qbw|&a;s!) z19TRusv#x{gN!q>c*b4zh_hs*2j<5?=cFPhT|zQVpqgRzDj7g;{1Z%J8s})lkojW5 zt}&rXBN2Str(|)n%XrcdZK)qOiEKuspi9;bRizxK!BZHVQ& zEK(_@Y{5pjKwdEWlfPl;Fb~gozblC<1=oBr-SyqQVukr3OB;3o+=j*oxo7nwRvE4?(VD)%U9%$5jLxKKF6VWTi?u^z7E^HZZ<3Ntz} zM*_vw5829T!7{`+MWkrXkgbFeiMeUI(u8I^V&j~k5<*n%Ty|zkWmEDuUmozBf#wDaR=AF+{)x&CO27! z*3a6!eGEUHi+JtsUVB{>JSu|lN&JsmjLA<`s(SG*&%_R5_;+}}n>2}dh^*Tdpzk=P z-UXa@_OrH4|J2Rq?wH|*fY?GlevW*LKB~@F(9+qHE!-Kmjwx0Od2g zFC2fNUKf${nL1V*%VsbKl=@!kT*a!p*y{%+?1w7Kh?*UOOxaqKDX)`Mts#$|N@YA@ z2KlCbjSm9vLpbcm>-|iHOB9*Y%LPkW7*oN(_9qc$1q*Rr4>O>5i+lEzQni{hRK+#g z-BW&EPI>A#IkiooFhqje8X$mjAVqdL=k^v6550bnHy)0w)eJzoF;v7ZdctnJT*qPR z!D4=$FH2cB)3p05S^pn&`KEgvJH5edyAbV#CJ172DzOvXyswhgtiX~~rlYoEpJH^y zKntNl@O}a(7{>x`r}8@9&9z3M&aYpPUWf${B1kJ?nQbMUA=z`NUJMl?Mp;-XzPQ%- zs5&ai>pGkD#zmAMOkZZ`Jfoms-0xKi3kyqeG7gxNBkCOfx1B~uTUZXqozW9Md|lm4 z)miA}G+%!`0j5wo+qpfLm(=?w?G5e4;d7Hzdh@F~Z)G`>>DS|3C%PB+S$5 z9;^mkvhOFGOJ{dw=hd1s4{fY$pW0(w92z~;)B%0J{5dvABy<8733sm_I7E<(p1-=F z7HpK$9LNg*%8>HzIHymNBi~D=ovb5Knem^A(B2@;zFT@-?_ooRqiAi9$`p5rNhNtj zTmY>O4d17#M5zQ=Bh;NNuHTL)9lu+=vB$pk!C|kea#wILu&*May7Pj=6%oMI~kdcZU-{sRRagKB_f^-@lmlUSMA^+o1X|qYj^{nT9pyx0CqcukaWn(#xX9ARQ5&r6fL#&Th(9I^ zf4FAoV!Mo5Rk2=O_}$~38ivN|Q>f;fHvBr~zMu0a-ZeU^!dpK!E~zoQP3WBqyyeBT z0>t46oaDKW_+R^jcf`~MOL+Z}H9@LBn~$%IT)TkG_x$qDscJ>Q7aPq6lkZ)%@z!n}4~vLf_&lgl$ve z#a*q~(c&RRd8gHMG7fNYuaga{H@U1EKF3~FG!RbNcBR*kv zMJ;vfC5~qk=or&w59jeX$8bn6tJh_+@t=dSbSCLE8P zI$S)*HU?gO=onkM-tzENi|f}Xx`QEF)0Dd*Mu6PMq6-x*YS z&6(cRtsTjsu3*T8-=-fM*03r-EgvP1_;DUJ-O5{G6FW;qqMLugFCB~elKq2XKpn?j zNC_kBhu6AH#O^IHrHhk-vi>f1p(D#K_4ie$cJ+Szh>F+SzK;(FzMD9*)*9hK>KH`bEm)d;}B8-!wPN76N=J;1mP-%fPpIay09d!UkSZbgfUQkNMh z?!A}Q#xSG4E)uwIei2e*b2ufVIK8ubfQM+sJn{foP0B1+dtk0jt~8SmoP&LJFE&2* zONBw?ACXyXL=x3*mTZY$wmHCk&VY4~fBthp%-?m<;4wh;$z%IRyyN4;-!Rp&2TErPN#BkvY&X~2({Y5acX zSqBUWE!YaJ*7`t^RJ@vLer9jtc{Bb$@ zs|L2?AYo(Cv85)%1jdYM)uAA04P?>;mO6?9TUlIPzYahD8rB*2AP{#)UZM$3ZHg&8{b3UUC(lycKlI>ZUVEF z?~Gx$)2^>)J1@zqQ>c2TOP}q|pV>lut*Vx{-S1rq`}5by$An~f*;KPU_CqmAJw8rv zDk1Vy<=Xe(F1dNCjzNtjDougcY53|=ZNih9k5?qE%5B}A%@4eGE3JCJe(1t`!ra4- zbMVr#kcj)zlGUH)Ai9}=)gx?9Lx~fmJayjGi}H8sEIW;-;3{54A;i7ar^*XNZq=QB z)4LZsoL^>5_%QxfdVBAe|?G9qmlpP4Eohul(hZ$bI^DaG5K%iLu(eh_K zwX&CkYmA?chr;%M5;h({^of}_#f-YA?7f};+ayCAEqklhtoKNDC0csdIr#Qw8U$H; zlwOTnk^5M8;ozB(^$&=%$g7_06L==iH52{AE8I}~25c8si@i^VcQeQF&RM%pdd%3l z&ZlFFo`Y6bW%w%p2fut{2PuTrX3_sfP?z=s%kPx1nV&*!-4na6T-5)?SD$h3$gu z)cWLO@3&u~YFE>=&(er@ZQ{~Z)r2%Dcs}q{X@HIn2~=p>@b-?R?LHaa+6T2zR@7%D z@eI!Z#T&_^H7mBhzi$}4G4N)q?=cg%iOE!4DJ0)oiY*wu!A481_0iaEqknebBd>E| z26LXZ{4rY~_-kMdgC5E_Mn{fps^2AHV(z~>Gr6L@QpqY5YI zZe~8pj$beg#f1nRzj3q7iTj#N0Wu82->GO2C+IE4PPmFQz{C?g=sQvMJ7Z1PcNKvJ zcDsWP=u1ltYtZbw&w>xPJFGt|xYJ^&-QH7ZF+i|SRQBr88LF_i)}^f6oAV)Z{7^=F_e?QzTWB~%VNu!xNRupujtv$cK(ymYmJ3_wPt0r z@)7E`Gbh>yoJD}C`xZ*9Qw=&^T-a3J`<$l-xF#haAm(V)Z7%J7r2$c3FR*ZM*s$u( zy~xjkQN9F=YB5H@ubW>S8_fR>(0-=~VDm~pA|SdZ3)DX22Q0s}o+usF9|LwbYU;ti z*oAdb3UU2to})98H^v_Iu1MF=Bkhx$D{Fb?Rcc>MS#}H1(oxKxUC_xzCvQ`)Syb<7 zbAbalV8iLoiOIXe^?9q6n>=U8$FPf=yQ7nvLf@`9owNzI(Ptj{IFQ;SA+bykUK|P> zp7I`JOQwMxzr@4yD;Jl5EkJE*?xLC!u4TpSI;{VKEnRsqN^l$2P3W^dm)N@@rgnVLl(jhI2`V$yv%i1TYq11N?kh{Q`1)$NmVTYYJPt|?} ztzGSn&2MHDgOuj4m)$ls=!q(}oqsCb$_nAww75E?1@TCLTRd&A>Z=($8@R?cT79et zNoF|0NIUL@CAGm~S}%6s&5q$iMmAb@Ze{UK&f+=P%60`8@MOOC#T!#eK!y*Jm|B~A zIO1*0`ocSPZyzvaXR%6ii3;&Oj!+>hDuV4LzD|vEx2R~@EkXqQi~toG=lj18Gq#@k zs4d(W8vtr@#{xaR{2Edekh4CzEs3;3|+?t>py*TEh?)tI)ja>Ieb|$_7 zuV;!vds!nkhb{`SJr)=osF!NY30FQ;HDhUe=W@mPeeV^)!PQLZC%%^RD!V5~gLcX6 zb9ws3j18Y2IH>r^XX9j_cWP#vqwaflbD*X3X^>K$1}mjxB7~*H`wtU`Ti%Ifoke0^ zjHEmz=!99UfBwlgrf`|TD;~lz?j%Vb@)3(j{7p7~N(oO9|NCy~SC@Q;^NFvO$MSTv z<`WzI*;kWybNf3A#T<{4Ysj1Jvf~hvP%8D-50LN`DH`418#*ZtI6Iboi^BPFZd`zv z`zW=^`Mqqp*R1NAw{wT(1|`sMe~Z%5q4c1a$}J48OhH-7MApbx5jUk^3XdwqBNo~| zGb6H)%i9~w!M*D5PMte<&R`@zC9!;~^G7{Y)$+yxcV7l~lkUCJ^=MBaA9PD_>a%uQ zCcj%A1UJQ&wH3n$)0<1=y@(fB%y;YjnrhTJosS9@gMTWR~f+LQx5l`ZWX zaiA!0yp19Nc`upgr$z}6Q!;FgG~@-7epk8#e{^HnEkU}h;mCi^A-2Vz36?nJ{`>dD zZRzu$LEz_IQ|enN*8&=1y35c=a)S)@kUPV$gG1~Y=Tsf{rr&GJVwB&tj{B#U!n_s; zJ@IFk9IF#s6#|zQMlAz+3(gl zQT)KTj1lRMVS%0Jc^)>HEjRJS^LPHP)mrr-^N01*t*_B}*gUm%LASN6M*UA__{6d3 z$vAlrw-^PMZf)zlcg&?JspgmPySzBe!^ho^N^ZMU|b0Jxxj9QOBt6B zeTWKln{ugc6z@kC&s}Lb`MQL3r_FK?i#TPpuYx>Hga!; zCD(^aaKJSG$+-sWHKi}dB8?rj1@grew&j;jOvq6qX8AQm(IUnjL_Swp#ZVB6gr%yrvd7ZK`h$S4d< zPfs%rECzb-^TBR)mxxbzW5lkAD?gUrO_{WGe3-@Me%mOl?5wPm|7%Tak28;`aDMs% zp{3jZR*vyWkwIm&}M%2;dyX<3ijPV1djgfaXIBqdO;U?|SdQ$Bs%K)SG z9(*;Hx9G?zOr$tp_jExc0Ho0t#jSIG}8Y-3&;8y(yH^sfj}QTWN{cu%$jKdzqX!wOKza3m13I zznOnC+ggT~lF3y&BoxBe~wwgE1v{{a7;b4_R3F3Hjt;n(doaZ#!*!{$+y{9_x9x=7y zi2~TXki_5|B3G5y-T2;M@;$xgn4m?Sx@fvRKD`Yg3y|}05|j0vHs~= z52~^|;|GQ0+FyE=mG_0tsiBABZgqbd7NiI5!%b*!YwOt$J?)a+Dh0G3jtIJC<^IRk z=vVD{qk#K;t592f>{#Cz76l*E_yA&EEzdZMFgY;bV5~ zP?APSh1vLZokh~khpb%lQ~?dpn2PrFInJ(zC!S$qvn##aHehr8BDq{Iz4uz@UDhMD zUp!$yHKd1bDGXOy0( z8l#+#UWI>L%BkMZ)+mTE0}Z_xC*6FilpdjU%|EMknO*iKTzy)wM_b(gwRD z!hbkNSCL)sh6&uR2F=omG2NDl1a|e&`$wA*hz|!D4pZ+;S_iM;JXto#bTKN@e!*IW z5sl7!1kW8;Qx~#@8@j!j0(<}y6S)!n2z z($UI!*~m~t1Cq1yF%PI%+@U6zZLG0gVDCRdYs;bzILfxO$c%{%d)P6-ZR1JvLYTsR zU{)Z?>k2}cEv%QARu-w5G+RqU;bSCW^r#RV@y5mVt1c&uI~^7U;Tg%!1)mX_?@OPi z1wIHkJ!cQey}qESGeYKWf7i&*6DrU`kLw$CT$4=21i>m?__BY9Xc(|TF(xGQ1tmYQ zV1y-Z>pX9>!s(L5KU1fzMQoy;Y_@7{8@Gn$;?D22)d-7&!t<}O*2%ZUI~fFWkXX`h z4fE=lyt%%fpd=bxb?xRv*s|fEH?mzsK79@8Dd@HtTo{X#5_9w#o79NWO2|wxWC<^{gI4uhYS65W~~aeAymIk@QopyUScm*Jyt7DJ|Q?cFN%=6 zToyJM^d?W~X{nyA`w8Dh2mGX9cV^MeLYMY}O-Hw+z7Jz-9K5l=P21IgmG(4In;I0D zsup?iyJ!Us+83YZ{PFkzL__3}j_yADo1k2oRo#o^+@YrjGG1Zw_2M}%@3y*^bAl)8 z^o0kNv7zc+m((ljAD`$IxP2P>VVrDA2ja}gN^;nGtmgP*Y5^Kzj#ctE@-Ji@q<~i) zA~xc~b}%us>^bmT z4W@(`j5oR(JkyB@ zLMUdLV@cVfIZeGqY{{2b*HD!criWakUB6}QpHD)aMbEy}%0~IY)~|-_2z_`^9Wzh& zpE=&SfRgzr>ob7s3@uj5OzBTO_q)&KaHe!Mc z>#4*3i77>;fr*hy)AR*%Ey}!X=l+-g^O|uh^hZwsWMt>mB1&I6d9xf{Lj|6n2tTXoF=`x?BNe^GlkTOP|ukm9JDGlMlIj0HIZ{a?B6@;g-o3KuCsk6uR7v5V4`> zYIXY+A!~*PD;?-6FRpjCF4Q*V?zM2HOth&8p~R(aFn;Cs_mlUIr+C+oxF^k!kl?<` zkRBNnd?CZ|SZ+r5%N7*eama$}l!e@?L;DnbIxe9bRfXd!t}t>YkK4krIyVCfk5`&X zuRo`Lf!Yt#`lO#%eMzPYCPawaBRwrJyhYvx^^5*0^>@l;CJ2PFmp)jq>=zVxq#}28 z;|i|$jgmEg>7jpVxM5-Q!s6yJ!vz-J6o%#>JMam!i4@gT`v$PVbOg}0;A z08DijoL+4*H)}@JiB#NTg;$wed;>0Oka4@cb5kMAR79d-{I1UNy587#8zxo_gx`_M zq$H$kyBrFTu<$+~mc14okL)gHN(tae5p`GHE`zAVFvGL!tM}O_c zo&y_lhpHX?!f44Sp6~y@?fPr%VX&pa6%$o)n*Hxg+mAeXmM5kS@EtSw*Hl}t(wIYg zon-R=`0d0$pMEPfmNU#zhd*9wVtJ?i?aRNPsERT#@H8*v0D12IWWHu*XI8iVb31n( zkl%h84hQdugk<;Ii}pUg|Ft>)_c!c3%ow?wx8Wn(zu$bt{2Q^Vrc8=_F(P}_Pk;5| z-|phTk)q~1k}rOf^^5(%_Wlp$X*{8otQC7F9D1@;x8?Vd3RM)d3F}f?@s2m!PI)7W zLF4E-P3g%!54K?Pc7{l*Yuet<3BQm1nV4W1o@l$vD03gh%RCbt^*nC~7%+p}=;u$gY_|28x(xDGF}DQRnwhtN&1k6A6|CDvLR?_0>UXIXcc7UaLjxEB{#)Bm~);BU=Q<=SGRLf#&=1K*Q^zrXEBCBqY)VN#se zP+i@2_kM;RLRIT_xhV+mxx8n4jvT4|69aXBaQK#@X(H!ier7uCMM8wOXK=`$=4Hu8vFHTnPn6Dbi8;O9TGEGQ$My>`3iuB^ zY*mRm`Mo{;hgJ$Xv)|!r7W;`E|I<9^8A z37($UNB;TnTeJRgZ{+EpaqsvRPjs#VUjKx$X4Wej8X7wV5Bvm=I2?Fx&t%?W|NBq8 z(KY)ghd&nW++Mf!PB5HXRr-?~EZbX)&|6#1m$SL0k>&EX*i}`D*+dly5mUSP_a=U; zP|Z(SZJpU?mVFehnZF5Izm}b}{h?XEb2WP=U%R+c zJ=leQ!Ed+oA5WUgn(xS`h@tQPB*bDlY-l+U-I~(yVtc<8(wzAx>}&nUcYMp29lv1T zzxOZn7wr3i4f+N9zR_;gZD8NugMUH2zj^Vs4f+N3zT(F(sP{u8{|oAUB`JTJXw_d( z?^~w)f_guIt-qk&FQ~_|=h?3e-@hW%|AP$Q-|*%a)cczkU)$ziQ12^#{DOMFpx$4q z@++(NFOwYdE35Z|9L%q*-q-oeU&Y)XXZe3Yy??uyUr_Ham-Dq2vHU9Fe(iyN6>$F@ zEB~cRs=o@j-*wHu3bc6gxc`(CMqBf*zP;~bxX}N<`u091Q`d^C z%+uhZMN=M6w(W`veYH2TZK}`AqgSnb)l`_ywcxe=H3pp8Hg6Xckm|-vyaX^9W7>AdRn|=`^J+S@moD( zDOzv-sU6?IwX=GmvmPc_jQ4!{zRdiW>d@B1GbXCq0&%U;Z|8r~A)MA_TQ2GFa@S9~ z{Pr+fnBAK=o7sanHGk4$dr+NyCi4tSfK4<6p;VRZHDZr(1d~nV$n)iOm&R7JWJT-n zjs7*{F}#I~Ps=fj7h&KJfiU+DAGN~{UD@|O zs2mIZ5|qR)USHzOs)U-p+NDfiK+_8A@qu^|Vs)ERhp4M>r+b9y=xc?ZEX|~j#3l-6 zB{n7?V;+vQz%X(6sMZLZ-VC18g$eU-H1F<^f|0uNJ@}{~pW#jmgGGNdIRQ)^1dGBb zUj#eD1yhaoD=szD?3=b{Y)M{oUwk};+7?V{>k~-qEQe9&-KjWmaQS_~(ibo#;%GVV z-FF8uT~dfMyLsK3^`QZphbarL!Tz#HU2P@!D9(a7;#8y(p#Y;(u$^=oo}QpEHmhZB zye`|g9948tzD*16v6#(s9->*cX@aaZAYs5X46Jp4ciS$aP}YOheM396Fs{(1+#K^4 zl$5G=$gluvTKd>=m5nzX9mGarLzGM7qL&~nzkXuNrq@d8-XhU5#K2VE;skkw6 zmdZFqnGirPAEA+vPOIKUvlT$Z^QMMgq-(^2mI`&Xtd~%oIh9FnHf-7)6m?#Q&;xoF zvQ?8}N)@hD_iJjQ&+E$bbZ8@fskV>p~r;)JU zDCI9FID|;UA2zVTH0(;+ZlxT`x+~o)aX{-fSwOa?-Bvu1GR2PuIzw|B*WtzHv9PhaN8SwK>7 zW_Gk((Q(?RHsvAUGI^i(r(0o*ezV1Oyo7t~Vl%MCAbKm9qE<6l=rY()6)iXY9Et=K z)s14rfXJ)DEnw=Lq3im~(VCILB7>FGnv+vxG(E_dkc1vByrdLNelibXSfqri`b_^~ zeDzecDY>uH36?O2tbQ-u(e2_@-l?YO!fe)Mr+ArEO}m0g75XG5t}jH*Qetd;gdLwnkm`RGJiHks2EKyLsV+KRL9x;dMD zm5Vj56T-&}bB*vZ5_ARfwYPHdwZ`lU$d456KUT)8aprh=IX62Iyf5{wIOXjM z_8Rm(Y2|H4Y+Kw%a}p`e7kHu0q)Zv<88i*U#UxR&ejSv4fk4cML=hRNs)bxJO{xrr z?I6o(t*0CzJ8xPQ+a%)@P9aAr>Suhxf;3XBHQh(iBB;r3KZ&<{Ed|l@{JKCK22%5~ zpF(r@A1`czSrtUy!r(@K*ajNZ$eI9E2bEcxM(J76HjC+-D~n|Y3ozO&ETLt;LV?DH z0H{|m*wr6}5_3&tu^Ty7EUQCexTbM$xtchDAzz&$+XTRtMn%&bGRh<&TEHb58IJ}I zMf>CL<_m};Uz&1CX#4fW)n@4B;ang|sAq`4MTqJ39fG8CgfjWrrTualyc?@KJEOeo z?j|Z~NaODFQzpcio;&o8(s2wS5@UKX)CBr0VBBDFZ?T@xBLcFvQ8@}?u`8Uai0m>&WNO2y@ATPRl* zp42wmT2eyGHDL@rQD>Sya}^tDol{=0*Tyw| zcsez`5eTJksaq#?(YtAJBU}(+_oXy|bSo(1vvfPPUQkk*VUE2A+iPGwJ4v@MD_gB( zLIW-Ukp13`gp{Hd)z4Z$vS+dXm?V9;A0J%kt3)Ru@SJnh&57K41h+}hGIwxkhP$G~ zI@_3sR5b$>HD=vab|kb$B7)Y-$8HhSVgtOsBsWey_kK2o4i1oQGphqF3Z6*1wr*sb z@tJ%##@~!+WTJqL*lCyIX;9_mYtn-4lsHN{)1AnZN=U|f5DjT@h^FM20DH_X+3E|4 z=MeJqRBAk`?uV__XCA4#AIvAVG%ypi)oj5xQFwv&+Je#~z>-5d+9?tF1&)gXD7tv0 z9}{(jS{pNDZ)yx5y2{stlqD<|Blso}T?1>A>VwLmT!m%>d#ZY7p?3p4?KrOwqpu+M zFYfn}ql1{ww@(LeW(Uu4KFfWqrEhg*It5^Gg%biwX23J5e&P_t=DC5!q9_zvxfQD4G z3T;D7P7eG~zKMFpMG!GnO1P*#oLG-ls_U2pG@#dt zygkEPNY-mtqEoI-0@QU6rak~ebA0icWt+&L1!V2qTypulTUeCZ(mc zHqZkAM{Xd4smLN>MVOQ?N0zh4mGva?6V^?}8Vbl(2dhHgZm^*UIPpk-qml$xYLNj5 z!3p^zhEiG>A5LnxQap|%`6f(-GPx+-)_|{e2?8Au>;VMo2BxBimBEw`(uwe=u;oZQ z5A50S8Nh=oFMvd6U3>JtEgrAc?-X5Fm$J;IBg#L z0&uH<45Tm&AVbK$(glr1ZXo~yNk;nl(r*RJ#d@jy~R*Z8pv}Y1s{NQi2xWMLcXv|9oEkguulriRDuk>)?5qB(}zT5 zFrPm~xW?u1SrbZMuWAY;ZtAtKpJG_Tly|O9i1SmCMUo0!M7bbN>`^&`ybn@P)rF~U z?wx&N0r1kHdIOY94Dc?)_lR3rTaem~I?sdVk~__?N5ci3C;FvNAdR8*_S6(I$OPL7 zJX&M0(i*78HuSl`~(fs`Be4@L5qPdqb9+|H0z9B(#piuv%t#eVIi^_Fr2-dK z3$Q@ToCi(MyQULNT+ya}VxTcVzobse%gg3q-(&++HY~`GXlAm!*gzcb#HHEp8wFgE zLawE1s1z*_kj!}!$H`=E5gybgQU0IjV1nm15PyME>e#KAr2lXbvrJ@ z3i`!PJjUe-^v?+ss+N6_6`T&07Tu% zB&_>!yYA0HT?mmXGIbS!_3p9HOX#O5g|7sNS=ZHbl;bHG4t2rW-AxQuF;LH_A7KuC zb-1x9#b}7f@u*!Y$c0F_ez8isJ_|$Or}pxDg3uKyS~h;Suj!nZU<7{)*^urcnN9KI zUupnZRp;EJfFN3YaX=;+=$LYxM|4}EYanhh$YdX} zeWf^X%@G)0)TkpB-lhDaY8)a*ziS~*)0;_kd&@Q^0VzxlrjhamocnFh3)tt5(la*`Wc5ny&g4v+=7h4DuNm?n#)r28oH>hfi)eQGZ=%ADAyI%0O7j8RE~v!3%4*Icy-|^j;9FMO=m@;^t|l$ z+e@eVW-=?e}-LN#_+FLz++msH#kVp<06DTCevw8@tP zU7t}$gJMvL3Zs+zw-OR_(}^`U4dV=Ckl_H9LphJq%FA?Cmt%qh5kE0JLi^lse2 zSsZZqOAS1--&O%fTZeR-xu@7z_y#JYRutF>dcjz)A+ALgSeceblYxgySeFtE4jRXf zJT@Ocbu5>;5p4P@6*1muBG2|E;L0?lp56yq|IzhBOw17{o+e-KH#pEUeTWTpd(IxW znX4U`r!nnpj|RD9nMk!q02E~;bwU&8J)w%GQa$px0{3?ZuX-%lVGq5^RlWXjek)4t z_DY1`DqnPjJsX{G*I&#Sk%nCBvrRuW7y^1Ocq{pfM&1&%9>iSEAI2Z#gEUYWOa&bY zggdjPJoq#Ea4lQJ13u{I6Kr>HKd@Gcg+w5F2W`!#v^T} z;~$Va+K>uY=w{O}L>Ou985uhyZ$S^B`xh>(Y&ej+aqcNuW2BDKmv<{c?lSGfc&s>9 zMn!}ePyMpfp9XQnl)49}qt=QwZ6Lzy<@-H?kH1@5`9vhICjBVERPDez67W_h@%mUo zzj~(4NP$3U(zWVAPJcNI&N$Eqf#FgU&uJ4#ZGC&aUL~~$PwBCNp0>w~7z+0AOU1|9 zOQqyZ!G#ZEtC|$KAdC=hI0-PUo4WGM#U=N#626{Lo8cTJc=vcGAQEb%9Wfpy5G>UM zr(m&-LkksU7R7x&7@ST`vOzloIjA%-8^=*+(qk4IH=u|al6dlJd5>LBj{<%az4pTe z7##@BDojW+KMK+0GKOXwYfBbF`iH8)9G?AInqz+H_DZ!#3x7|>g&uXE&w$$Rb0}}P z7dpT6lqi(crS7`+P5dc)%Gz`FHRz%dSlqm<0AP_s|INcKe~17 zz{W6p#*`goxs@Byr$y@E!KV7V$SqX#*RH>}g9UndKd(^G5}7KH@@+XbzSJSV`gW1{ zOjk?jDxa@4a<2^6$W|h{{blwKVbg!A3PXpg`-DS~3OU&S*80Br@a$vKogF)veL$Bl z=>1t=ute?6z^%-5xHQx|XFIuWQ*+pQRg%zr+HsBNXQMWlg+sXMyidLhCCEQd`k zQJU;O1fD5Tb#H{x=8xKVd=?V)6XP3ew$kcjjj~x0v=Cl zs#>vl(S*l~2M3eLooF!W`g48}y7xOYm_{$A@hlOmnwqdGBJL Date: Thu, 11 Nov 2021 14:55:45 +0530 Subject: [PATCH 041/628] Readme chanegs --- PREREQUISITES.md | 6 +++--- README.md | 8 ++++---- docs/sidb/README.md | 6 ++---- 3 files changed, 9 insertions(+), 11 deletions(-) diff --git a/PREREQUISITES.md b/PREREQUISITES.md index 0dd557ae..7fc3cf21 100644 --- a/PREREQUISITES.md +++ b/PREREQUISITES.md @@ -22,12 +22,12 @@ Note: You must provision persistent storage if you intend to deploy containerize ### Prerequites for Oracle Autonomous Database (ADB) -If you intent to use `OraOperator` to handle Oracle Autonomous Database lifecycles, then read [Oracle Autonomous Database prerequisites](./doc/adb/ADB_PREREQUISITES.md) +If you intent to use `OraOperator` to handle Oracle Autonomous Database lifecycles, then read [Oracle Autonomous Database prerequisites](./docs/adb/ADB_PREREQUISITES.md) ### Prerequites for Single Instance Databases (SIDB) -If you intent to use `OraOperator` to handle Oracle Database Single Instance lifecycles, then read [Single Instance Database Prerequisites](./doc/sidb/SIDB_PREREQUISITES.md) +If you intent to use `OraOperator` to handle Oracle Database Single Instance lifecycles, then read [Single Instance Database Prerequisites](./docs/sidb/SIDB_PREREQUISITES.md) ### Prerequites for Sharded Databases (SHARDING) - If you intent to use OraOperator to handle the lifecycle of Oracle Database deployed with Oracle Sharding, then read [Sharded Database Prerequisites](./doc/sharding/README.md#prerequsites-for-running-oracle-sharding-database-controller) + If you intent to use OraOperator to handle the lifecycle of Oracle Database deployed with Oracle Sharding, then read [Sharded Database Prerequisites](./docs/sharding/README.md#prerequsites-for-running-oracle-sharding-database-controller) diff --git a/README.md b/README.md index 76d97495..cd72a209 100644 --- a/README.md +++ b/README.md @@ -75,15 +75,15 @@ Oracle strongly recommends that you ensure your system meets the following [Prer You should see that the operator is up and running, along with the shipped controllers. -For more details, see [Oracle Database Operator Installation Instrunctions](./doc/installation/OPERATOR_INSTALLATION_README.md). +For more details, see [Oracle Database Operator Installation Instrunctions](./docs/installation/OPERATOR_INSTALLATION_README.md). ## Getting Started with the Operator (Quickstart) The quickstarts are designed for specific database configurations, including: -* [Oracle Autonomous Database](./doc/adb/README.md) -* [Oracle Database Single Instance configuration](./doc/sidb/README.md) -* [Oracle Database configured with Oracle Sharding](./doc/sharding/README.md) +* [Oracle Autonomous Database](./docs/adb/README.md) +* [Oracle Database Single Instance configuration](./docs/sidb/README.md) +* [Oracle Database configured with Oracle Sharding](./docs/sharding/README.md) YAML file templates are available under [`/config/samples`](./config/samples/). You can copy and edit these template files to configure them for your use cases. diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 535d37d1..6a26630c 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -116,16 +116,14 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI * ### Creation Status - Creating a new database instance takes a while. When the 'status' status returns the response "Healthy", the Database is open for connections. - + Creating a new database instance takes a while. When the 'status' status returns the response "Healthy", the Database is open for connections. + ```sh $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.status}" Healthy ``` - - * ### Connection Information External and internal (running in Kubernetes pods) clients can connect to the database using .status.connectString and .status.clusterConnectString From 898bbbd7d28b2f7953d99b45b9d08c7e1a8b6009 Mon Sep 17 00:00:00 2001 From: mmahipal Date: Thu, 11 Nov 2021 16:52:08 +0530 Subject: [PATCH 042/628] Changed Image Name --- README.md | 2 +- commons/database/constants.go | 3 +-- config/manager/kustomization.yaml | 4 ++-- docs/sidb/README.md | 10 +++++++++- oracle-database-operator.yaml | 2 +- 5 files changed, 14 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index cd72a209..bfd5d368 100644 --- a/README.md +++ b/README.md @@ -19,7 +19,7 @@ Oracle will continue to expand Oracle Database Operator support for additional O This release of Oracle Database Operator for Kubernetes (the operator) supports the following lifecycle operations: * ADB-S: provision, bind, start, stop, terminate (soft/hard), scale (down/up) -* SIDB: provision, clone, patch (in-place/out-of-place), update database initialization parameters, update database configuration (Flashback, archiving), Oracle Enterprise Manager (EM) Express (a basic observability console) +* SIDB: provision, clone, patch (in-place/out-of-place), update database initialization parameters, update database configuration (Flashback, archiving), Oracle Enterprise Manager (EM) Express (a basic observability console), Oracle REST Data Service (ORDS) to support REST based SQL, PDB management, SQL Developer Web, and Application Express (Apex) * SHARDED: provision/deploy sharded databases and the shard topology, add a new shard, delete an existing shard Upcoming releases will support new configurations, operations and capabilities. diff --git a/commons/database/constants.go b/commons/database/constants.go index 81b5e549..8b376051 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -220,7 +220,6 @@ const GetPdbsSQL string = "select name from v\\$pdbs where name not like 'PDB\\$ const SetAdminUsersSQL string = "CREATE USER C##DBAPI_CDB_ADMIN IDENTIFIED BY \\\"%[1]s\\\" ACCOUNT UNLOCK CONTAINER=ALL;" + "\nalter user C##DBAPI_CDB_ADMIN identified by \\\"%[1]s\\\" account unlock;" + "\nGRANT DBA TO C##DBAPI_CDB_ADMIN CONTAINER = ALL;" + - "\nGRANT SYSOPER TO C##DBAPI_CDB_ADMIN CONTAINER = ALL;" + "\nGRANT PDB_DBA TO C##DBAPI_CDB_ADMIN CONTAINER = ALL;" + "\nCREATE USER C##_DBAPI_PDB_ADMIN IDENTIFIED BY \\\"%[1]s\\\" CONTAINER=ALL ACCOUNT UNLOCK;" + "\nalter user C##_DBAPI_PDB_ADMIN identified by \\\"%[1]s\\\" account unlock;" + @@ -255,7 +254,7 @@ const SetupORDSCMD string = "$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-pr "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property security.verifySSL false" + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property jdbc.maxRows 1000" + "\numask 177" + - "\necho db.cdb.adminUser=C##DBAPI_CDB_ADMIN AS SYSOPER > cdbAdmin.properties" + + "\necho db.cdb.adminUser=C##DBAPI_CDB_ADMIN AS SYSDBA > cdbAdmin.properties" + "\necho db.cdb.adminUser.password=\"%[4]s\" >> cdbAdmin.properties" + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-properties --conf apex_pu cdbAdmin.properties" + "\nrm -f cdbAdmin.properties" + diff --git a/config/manager/kustomization.yaml b/config/manager/kustomization.yaml index 7258b286..24e25662 100644 --- a/config/manager/kustomization.yaml +++ b/config/manager/kustomization.yaml @@ -8,5 +8,5 @@ apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization images: - name: controller - newName: container-registry.oracle.com/database/operator - newTag: 0.1.1 + newName: phx.ocir.io/oracassandra/OraOperator + newTag: test-0.1.1 diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 6a26630c..52cd8d69 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -334,7 +334,7 @@ The Oracle Database Operator creates the OracleRestDataService (ORDS) kind as a Creating a new ORDS instance takes a while. ORDS is open for connections when the 'status' status returns a "Healthy" - ```sh + ```sh $ kubectl get oraclerestdataservice/ords-sample --template={{.status.status}} Healthy @@ -377,6 +377,14 @@ The Oracle Database Operator creates the OracleRestDataService (ORDS) kind as a * ##### PDB Lifecycle Management + * To Enable PDB Lifecycle Management, Grant SYSDBA to CDB Administrator + + ```sh + $ echo "GRANT SYSDBA TO C##DBAPI_CDB_ADMIN CONTAINER = ALL;" | sqlplus -s sys/@ as sysdba + + Grant succeeded. + ``` + The Oracle REST Data Services (ORDS) database API allows us to manage the lifecycle of PDBs via REST web service calls. Few APIs : * List PDB's diff --git a/oracle-database-operator.yaml b/oracle-database-operator.yaml index 4d4e2ecf..2c344eeb 100644 --- a/oracle-database-operator.yaml +++ b/oracle-database-operator.yaml @@ -1412,7 +1412,7 @@ spec: - --enable-leader-election command: - /manager - image: container-registry.oracle.com/database/operator:0.1.1 + image: phx.ocir.io/oracassandra/OraOperator:test-0.1.1 imagePullPolicy: Always name: manager ports: From 7c1b54cee624bfaafc9eacd489a58b80a7fe5ae1 Mon Sep 17 00:00:00 2001 From: mmahipal Date: Thu, 11 Nov 2021 16:56:30 +0530 Subject: [PATCH 043/628] Changed Image Name --- config/manager/kustomization.yaml | 2 +- oracle-database-operator.yaml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/config/manager/kustomization.yaml b/config/manager/kustomization.yaml index 24e25662..99b97c84 100644 --- a/config/manager/kustomization.yaml +++ b/config/manager/kustomization.yaml @@ -8,5 +8,5 @@ apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization images: - name: controller - newName: phx.ocir.io/oracassandra/OraOperator + newName: phx.ocir.io/oracassandra/oraoperator newTag: test-0.1.1 diff --git a/oracle-database-operator.yaml b/oracle-database-operator.yaml index 2c344eeb..4d240f04 100644 --- a/oracle-database-operator.yaml +++ b/oracle-database-operator.yaml @@ -1412,7 +1412,7 @@ spec: - --enable-leader-election command: - /manager - image: phx.ocir.io/oracassandra/OraOperator:test-0.1.1 + image: phx.ocir.io/oracassandra/oraoperator:test-0.1.1 imagePullPolicy: Always name: manager ports: From b203d9b31cc00b4c9325e112dce3849efb025a2b Mon Sep 17 00:00:00 2001 From: mmahipal Date: Thu, 11 Nov 2021 18:52:16 +0530 Subject: [PATCH 044/628] Updated table of Contents --- docs/sidb/README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 52cd8d69..973a8ad7 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -7,6 +7,8 @@ Oracle Database Operator for Kubernetes (the operator) includes the Single Insta * [Provision New Database](#provision-new-database) * [Clone Existing Database](#clone-existing-database) * [Patch/Rollback Database](#patchrollback-database) +* [Kind OracleRestDataService](#kind-oraclerestdataservice) +* [REST Enable Database](#rest-enable-database) ## Prerequisites From d7ab5137d6f966ed8d4c08595f449cd84c8970b4 Mon Sep 17 00:00:00 2001 From: mmahipal Date: Thu, 11 Nov 2021 18:55:55 +0530 Subject: [PATCH 045/628] Updated pre requisites for ords --- docs/sidb/SIDB_PREREQUISITES.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/sidb/SIDB_PREREQUISITES.md b/docs/sidb/SIDB_PREREQUISITES.md index 2c13b8e2..d7964bb9 100644 --- a/docs/sidb/SIDB_PREREQUISITES.md +++ b/docs/sidb/SIDB_PREREQUISITES.md @@ -8,6 +8,10 @@ To deploy Oracle Database Operator for Kubernetes on Oracle Docker images, compl Oracle Database Releases Supported: Oracle Database 19c Enterprise Edition or Standard Edition, and later releases. + Build OracleRestDataService Docker Images from source following the instructions at https://github.com/oracle/docker-images/tree/main/OracleRestDataServices . + Add 'database.api.enabled=true' entry in 'ords_params.properties.tmpl' file while building OracleRestDataService image . + OracleRestDataService version 20.4.1 onwards are supported + * ### Set Up Kubernetes and Volumes Set up an on-premises Kubernetes cluster, or subscribe to a managed Kubernetes service, such as Oracle Cloud Infrastructure Container Engine for Kubernetes, configured with persistent volumes. The persistent volumes are required for storage of the database files. From 392f87a9e44ada2c276a432c1e0669727a0638d3 Mon Sep 17 00:00:00 2001 From: mmahipal Date: Thu, 11 Nov 2021 18:59:16 +0530 Subject: [PATCH 046/628] Updated pre requisites for ords --- docs/sidb/SIDB_PREREQUISITES.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/sidb/SIDB_PREREQUISITES.md b/docs/sidb/SIDB_PREREQUISITES.md index d7964bb9..ce8f1c1f 100644 --- a/docs/sidb/SIDB_PREREQUISITES.md +++ b/docs/sidb/SIDB_PREREQUISITES.md @@ -6,12 +6,12 @@ To deploy Oracle Database Operator for Kubernetes on Oracle Docker images, compl Build SingleInstanceDatabase Docker Images from source, following the instructions at [https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance), or use the pre-built images available at [https://container-registry.oracle.com](https://container-registry.oracle.com) - Oracle Database Releases Supported: Oracle Database 19c Enterprise Edition or Standard Edition, and later releases. + Oracle Database Releases Supported: Oracle Database 19c Enterprise Edition or Standard Edition, and later releases. - Build OracleRestDataService Docker Images from source following the instructions at https://github.com/oracle/docker-images/tree/main/OracleRestDataServices . + Build OracleRestDataService Docker Images from source following the instructions at [https://github.com/oracle/docker-images/tree/main/OracleRestDataServices](https://github.com/oracle/docker-images/tree/main/OracleRestDataServices). Add 'database.api.enabled=true' entry in 'ords_params.properties.tmpl' file while building OracleRestDataService image . - OracleRestDataService version 20.4.1 onwards are supported - + OracleRestDataService version 20.4.1 onwards are supported. + * ### Set Up Kubernetes and Volumes Set up an on-premises Kubernetes cluster, or subscribe to a managed Kubernetes service, such as Oracle Cloud Infrastructure Container Engine for Kubernetes, configured with persistent volumes. The persistent volumes are required for storage of the database files. From faccc565419af00a730060d77d1956d292f54a72 Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Fri, 19 Nov 2021 16:56:34 +0530 Subject: [PATCH 047/628] Added support to clone from On Prem Database --- commons/database/utils.go | 8 +++++ config/manager/kustomization.yaml | 2 +- .../sidb/singleinstancedatabase_clone.yaml | 3 +- .../singleinstancedatabase_controller.go | 31 ++++++++++--------- oracle-database-operator.yaml | 2 +- 5 files changed, 29 insertions(+), 17 deletions(-) diff --git a/commons/database/utils.go b/commons/database/utils.go index 4e1a143d..f7fb051b 100644 --- a/commons/database/utils.go +++ b/commons/database/utils.go @@ -617,3 +617,11 @@ func GetSqlClient(edition string) string { } return "sqlplus -s / as sysdba" } + +// Is Source Database On same Cluster +func IsSourceDatabaseOnCluster(cloneFrom string) bool { + if strings.Contains(cloneFrom, ":") && strings.Contains(cloneFrom, "/") { + return false + } + return true +} diff --git a/config/manager/kustomization.yaml b/config/manager/kustomization.yaml index 99b97c84..6022352b 100644 --- a/config/manager/kustomization.yaml +++ b/config/manager/kustomization.yaml @@ -8,5 +8,5 @@ apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization images: - name: controller - newName: phx.ocir.io/oracassandra/oraoperator + newName: phx.ocir.io/oracassandra/cloneonprem newTag: test-0.1.1 diff --git a/config/samples/sidb/singleinstancedatabase_clone.yaml b/config/samples/sidb/singleinstancedatabase_clone.yaml index c324c1c3..6d19f243 100644 --- a/config/samples/sidb/singleinstancedatabase_clone.yaml +++ b/config/samples/sidb/singleinstancedatabase_clone.yaml @@ -12,7 +12,8 @@ spec: ## Use only alphanumeric characters for sid sid: ORCL1 - ## A source database ref to clone from, leave empty to create a fresh database + ## A source database ref to clone from. + ## If cloning from on prem source database, mention connect string `:/`. cloneFrom: "" ## Should refer to SourceDB secret diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index c5d8366a..f1d6b33f 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -387,9 +387,11 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab } if m.Spec.CloneFrom != "" { // Once a clone database has created , it has no link with its reference - if m.Status.DatafilesCreated == "true" { + if m.Status.DatafilesCreated == "true" || + !dbcommons.IsSourceDatabaseOnCluster(m.Spec.CloneFrom) { return requeueN, nil } + // Fetch the Clone database reference err = r.Get(ctx, types.NamespacedName{Namespace: req.Namespace, Name: m.Spec.CloneFrom}, n) if err != nil { @@ -493,9 +495,13 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns edition = "enterprise" } } else { - edition = n.Spec.Edition - if n.Spec.Edition == "" { - edition = "enterprise" + if !dbcommons.IsSourceDatabaseOnCluster(m.Spec.CloneFrom) { + edition = "" + } else { + edition = n.Spec.Edition + if n.Spec.Edition == "" { + edition = "enterprise" + } } } return []string{"-c", fmt.Sprintf(dbcommons.InitWalletCMD, edition)} @@ -639,16 +645,13 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns Value: "/opt/oracle/oradata/dbconfig/$(ORACLE_SID)/.wallet", }, { - Name: "PRIMARY_DB_CONN_STR", - Value: n.Name + ":1521/" + n.Spec.Sid, - }, - { - Name: "PRIMARY_SID", - Value: strings.ToUpper(n.Spec.Sid), - }, - { - Name: "PRIMARY_NAME", - Value: n.Name, + Name: "PRIMARY_DB_CONN_STR", + Value: func() string { + if dbcommons.IsSourceDatabaseOnCluster(m.Spec.CloneFrom) { + return n.Name + ":1521/" + n.Spec.Sid + } + return m.Spec.CloneFrom + }(), }, { Name: "ORACLE_HOSTNAME", diff --git a/oracle-database-operator.yaml b/oracle-database-operator.yaml index 4d240f04..65e704a3 100644 --- a/oracle-database-operator.yaml +++ b/oracle-database-operator.yaml @@ -1412,7 +1412,7 @@ spec: - --enable-leader-election command: - /manager - image: phx.ocir.io/oracassandra/oraoperator:test-0.1.1 + image: phx.ocir.io/oracassandra/cloneonprem:test-0.1.1 imagePullPolicy: Always name: manager ports: From bd80ff4cd8ec40c2f8ec181a30976cbd4270374c Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Mon, 22 Nov 2021 11:35:52 -0500 Subject: [PATCH 048/628] add backup, restore. change oci-go-sdk to v51 --- PROJECT | 23 +- .../v1alpha1/autonomousdatabase_types.go | 2 +- .../autonomousdatabasebackup_types.go | 9 +- .../autonomousdatabaserestore_types.go | 104 +++++++++ .../v1alpha1/zz_generated.deepcopy.go | 106 +++++++++ .../autonomousdatabase/adb_reconciler_util.go | 14 +- .../adb_restore_reconciler_util.go | 68 ++++++ commons/oci/database.go | 60 ++++- commons/oci/ociutil/time.go | 13 ++ commons/oci/provider.go | 4 +- commons/oci/vault.go | 4 +- commons/oci/wallet.go | 6 +- commons/sharding/scommon.go | 4 +- ....oracle.com_autonomousdatabasebackups.yaml | 2 +- ...oracle.com_autonomousdatabaserestores.yaml | 93 ++++++++ config/crd/kustomization.yaml | 3 + ...jection_in_autonomousdatabaserestores.yaml | 8 + ...webhook_in_autonomousdatabaserestores.yaml | 17 ++ config/manager/kustomization.yaml | 4 +- config/manager/manager.yaml | 2 +- ...autonomousdatabaserestore_editor_role.yaml | 24 ++ ...autonomousdatabaserestore_viewer_role.yaml | 20 ++ config/rbac/role.yaml | 20 ++ ...se_v1alpha1_autonomousdatabaserestore.yaml | 7 + config/samples/kustomization.yaml | 1 + .../database/autonomousdatabase_controller.go | 8 +- .../autonomousdatabasebackup_controller.go | 6 +- .../autonomousdatabaserestore_controller.go | 216 ++++++++++++++++++ .../database/shardingdatabase_controller.go | 4 +- controllers/database/suite_test.go | 3 + go.mod | 2 +- go.sum | 6 +- main.go | 8 + ...autonomousdatabase_controller_bind_test.go | 6 +- ...tonomousdatabase_controller_create_test.go | 4 +- test/e2e/behavior/shared_behaviors.go | 4 +- test/e2e/suite_test.go | 4 +- test/e2e/util/oci_config_util.go | 2 +- test/e2e/util/oci_db_request.go | 4 +- test/e2e/util/oci_vault_request.go | 6 +- test/e2e/util/oci_work_request.go | 4 +- test/e2e/util/util.go | 2 +- 42 files changed, 836 insertions(+), 71 deletions(-) create mode 100644 apis/database/v1alpha1/autonomousdatabaserestore_types.go create mode 100644 commons/autonomousdatabase/adb_restore_reconciler_util.go create mode 100644 commons/oci/ociutil/time.go create mode 100644 config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml create mode 100644 config/crd/patches/cainjection_in_autonomousdatabaserestores.yaml create mode 100644 config/crd/patches/webhook_in_autonomousdatabaserestores.yaml create mode 100644 config/rbac/autonomousdatabaserestore_editor_role.yaml create mode 100644 config/rbac/autonomousdatabaserestore_viewer_role.yaml create mode 100644 config/samples/database_v1alpha1_autonomousdatabaserestore.yaml create mode 100644 controllers/database/autonomousdatabaserestore_controller.go diff --git a/PROJECT b/PROJECT index 7ad79eeb..184a0a94 100644 --- a/PROJECT +++ b/PROJECT @@ -23,29 +23,38 @@ resources: controller: true domain: oracle.com group: database - kind: SingleInstanceDatabase + kind: AutonomousDatabaseBackup + path: github.com/oracle/oracle-database-operator/apis/database/v1alpha1 + version: v1alpha1 +- api: + crdVersion: v1beta1 + namespaced: true + controller: true + domain: oracle.com + group: database + kind: AutonomousDatabaseRestore path: github.com/oracle/oracle-database-operator/apis/database/v1alpha1 version: v1alpha1 - webhooks: - defaulting: true - validation: true - webhookVersion: v1 - api: crdVersion: v1 namespaced: true controller: true domain: oracle.com group: database - kind: ShardingDatabase + kind: SingleInstanceDatabase path: github.com/oracle/oracle-database-operator/apis/database/v1alpha1 version: v1alpha1 + webhooks: + defaulting: true + validation: true + webhookVersion: v1 - api: crdVersion: v1 namespaced: true controller: true domain: oracle.com group: database - kind: AutonomousDatabaseBackup + kind: ShardingDatabase path: github.com/oracle/oracle-database-operator/apis/database/v1alpha1 version: v1alpha1 version: "3" diff --git a/apis/database/v1alpha1/autonomousdatabase_types.go b/apis/database/v1alpha1/autonomousdatabase_types.go index 4916ad46..7ce6e538 100644 --- a/apis/database/v1alpha1/autonomousdatabase_types.go +++ b/apis/database/v1alpha1/autonomousdatabase_types.go @@ -42,7 +42,7 @@ import ( "encoding/json" "strconv" - "github.com/oracle/oci-go-sdk/v45/database" + "github.com/oracle/oci-go-sdk/v51/database" metaV1 "k8s.io/apimachinery/pkg/apis/meta/v1" "sigs.k8s.io/controller-runtime/pkg/client" diff --git a/apis/database/v1alpha1/autonomousdatabasebackup_types.go b/apis/database/v1alpha1/autonomousdatabasebackup_types.go index 227aeff0..50d93b42 100644 --- a/apis/database/v1alpha1/autonomousdatabasebackup_types.go +++ b/apis/database/v1alpha1/autonomousdatabasebackup_types.go @@ -41,7 +41,8 @@ package v1alpha1 import ( metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" - "github.com/oracle/oci-go-sdk/v45/database" + "github.com/oracle/oci-go-sdk/v51/database" + "github.com/oracle/oracle-database-operator/commons/oci/ociutil" ) // EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! @@ -76,7 +77,7 @@ type AutonomousDatabaseBackupStatus struct { //+kubebuilder:object:root=true //+kubebuilder:subresource:status // +kubebuilder:resource:shortName="adbbu";"adbbus" -// +kubebuilder:printcolumn:JSONPath=".spec.displayName",name="Display Name",type=string +// +kubebuilder:printcolumn:JSONPath=".status.displayName",name="Display Name",type=string // +kubebuilder:printcolumn:JSONPath=".status.lifecycleState",name="State",type=string // +kubebuilder:printcolumn:JSONPath=".status.type",name="Type",type=string // +kubebuilder:printcolumn:JSONPath=".status.timeStarted",name="Started",type=string @@ -101,10 +102,10 @@ func (backup *AutonomousDatabaseBackup) UpdateStatusFromAutonomousDatabaseBackup backup.Status.LifecycleState = resp.LifecycleState if resp.TimeStarted != nil { - backup.Status.TimeStarted = resp.TimeStarted.String() + backup.Status.TimeStarted = ociutil.FormatSDKTime(resp.TimeStarted.Time) } if resp.TimeEnded != nil { - backup.Status.TimeEnded = resp.TimeEnded.String() + backup.Status.TimeEnded = ociutil.FormatSDKTime(resp.TimeEnded.Time) } } diff --git a/apis/database/v1alpha1/autonomousdatabaserestore_types.go b/apis/database/v1alpha1/autonomousdatabaserestore_types.go new file mode 100644 index 00000000..f7b1a1ff --- /dev/null +++ b/apis/database/v1alpha1/autonomousdatabaserestore_types.go @@ -0,0 +1,104 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v1alpha1 + +import ( + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! +// NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized. + +// AutonomousDatabaseRestoreSpec defines the desired state of AutonomousDatabaseRestore +type AutonomousDatabaseRestoreSpec struct { + // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster + // Important: Run "make" to regenerate code after modifying this file + AutonomousDatabaseOCID string `json:"autonomousDatabaseOCID"` + Source AutonomousDatabaseRestoreSource `json:"source"` + + OCIConfig OCIConfigSpec `json:"ociConfig,omitempty"` +} + +type AutonomousDatabaseRestoreSource struct { + BackupName string `json:"backupName,omitempty"` + DateTime string `json:"datetime,omitempty"` +} + +// +kubebuilder:validation:Enum=New;InProgress;Completed;Failed +type RestoreLifecycleState string + +const ( + RestoreLifecycleStateNew RestoreLifecycleState = "NewCreated" + RestoreLifecycleStateInProgress RestoreLifecycleState = "InProgress" + RestoreLifecycleStateCompleted RestoreLifecycleState = "Completed" + RestoreLifecycleStateFailed RestoreLifecycleState = "Failed" +) + +// AutonomousDatabaseRestoreStatus defines the observed state of AutonomousDatabaseRestore +type AutonomousDatabaseRestoreStatus struct { + // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster + // Important: Run "make" to regenerate code after modifying this file + LifecycleState RestoreLifecycleState `json:"state"` +} + +//+kubebuilder:object:root=true +//+kubebuilder:subresource:status +//+kubebuilder:resource:shortName="adbr";"adbrs" + +// AutonomousDatabaseRestore is the Schema for the autonomousdatabaserestores API +type AutonomousDatabaseRestore struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + Spec AutonomousDatabaseRestoreSpec `json:"spec,omitempty"` + Status AutonomousDatabaseRestoreStatus `json:"status,omitempty"` +} + +//+kubebuilder:object:root=true + +// AutonomousDatabaseRestoreList contains a list of AutonomousDatabaseRestore +type AutonomousDatabaseRestoreList struct { + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata,omitempty"` + Items []AutonomousDatabaseRestore `json:"items"` +} + +func init() { + SchemeBuilder.Register(&AutonomousDatabaseRestore{}, &AutonomousDatabaseRestoreList{}) +} diff --git a/apis/database/v1alpha1/zz_generated.deepcopy.go b/apis/database/v1alpha1/zz_generated.deepcopy.go index 78806d2f..1ec4cd16 100644 --- a/apis/database/v1alpha1/zz_generated.deepcopy.go +++ b/apis/database/v1alpha1/zz_generated.deepcopy.go @@ -291,6 +291,112 @@ func (in *AutonomousDatabaseList) DeepCopyObject() runtime.Object { return nil } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousDatabaseRestore) DeepCopyInto(out *AutonomousDatabaseRestore) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + in.Spec.DeepCopyInto(&out.Spec) + out.Status = in.Status +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseRestore. +func (in *AutonomousDatabaseRestore) DeepCopy() *AutonomousDatabaseRestore { + if in == nil { + return nil + } + out := new(AutonomousDatabaseRestore) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *AutonomousDatabaseRestore) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousDatabaseRestoreList) DeepCopyInto(out *AutonomousDatabaseRestoreList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]AutonomousDatabaseRestore, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseRestoreList. +func (in *AutonomousDatabaseRestoreList) DeepCopy() *AutonomousDatabaseRestoreList { + if in == nil { + return nil + } + out := new(AutonomousDatabaseRestoreList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *AutonomousDatabaseRestoreList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousDatabaseRestoreSource) DeepCopyInto(out *AutonomousDatabaseRestoreSource) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseRestoreSource. +func (in *AutonomousDatabaseRestoreSource) DeepCopy() *AutonomousDatabaseRestoreSource { + if in == nil { + return nil + } + out := new(AutonomousDatabaseRestoreSource) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousDatabaseRestoreSpec) DeepCopyInto(out *AutonomousDatabaseRestoreSpec) { + *out = *in + out.Source = in.Source + in.OCIConfig.DeepCopyInto(&out.OCIConfig) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseRestoreSpec. +func (in *AutonomousDatabaseRestoreSpec) DeepCopy() *AutonomousDatabaseRestoreSpec { + if in == nil { + return nil + } + out := new(AutonomousDatabaseRestoreSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousDatabaseRestoreStatus) DeepCopyInto(out *AutonomousDatabaseRestoreStatus) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseRestoreStatus. +func (in *AutonomousDatabaseRestoreStatus) DeepCopy() *AutonomousDatabaseRestoreStatus { + if in == nil { + return nil + } + out := new(AutonomousDatabaseRestoreStatus) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *AutonomousDatabaseSpec) DeepCopyInto(out *AutonomousDatabaseSpec) { *out = *in diff --git a/commons/autonomousdatabase/adb_reconciler_util.go b/commons/autonomousdatabase/adb_reconciler_util.go index 11ce331b..1bf817aa 100644 --- a/commons/autonomousdatabase/adb_reconciler_util.go +++ b/commons/autonomousdatabase/adb_reconciler_util.go @@ -52,9 +52,9 @@ import ( "sigs.k8s.io/controller-runtime/pkg/client" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/database" - "github.com/oracle/oci-go-sdk/v45/secrets" + "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v51/database" + "github.com/oracle/oci-go-sdk/v51/secrets" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" "github.com/oracle/oracle-database-operator/commons/oci" @@ -212,14 +212,6 @@ func CreateBackupResources(logger logr.Logger, kubeClient client.Client, dbClien }, } - // fields with mandatory:"false" could be nil - if backupSummary.TimeStarted != nil { - backup.Status.TimeStarted = backupSummary.TimeStarted.String() - } - if backupSummary.TimeEnded != nil { - backup.Status.TimeEnded = backupSummary.TimeEnded.String() - } - if err := kubeClient.Create(context.TODO(), backup); err != nil { return err } diff --git a/commons/autonomousdatabase/adb_restore_reconciler_util.go b/commons/autonomousdatabase/adb_restore_reconciler_util.go new file mode 100644 index 00000000..ed0b1437 --- /dev/null +++ b/commons/autonomousdatabase/adb_restore_reconciler_util.go @@ -0,0 +1,68 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package autonomousdatabase + +import ( + "context" + + "k8s.io/apimachinery/pkg/types" + "k8s.io/client-go/util/retry" + "sigs.k8s.io/controller-runtime/pkg/client" + + dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" +) + +// UpdateAutonomousDatabaseBackupStatus updates the status subresource of AutonomousDatabaseBackup +func UpdateAutonomousDatabaseRestoreStatus(kubeClient client.Client, adbRestore *dbv1alpha1.AutonomousDatabaseRestore) error { + return retry.RetryOnConflict(retry.DefaultRetry, func() error { + curBackup := &dbv1alpha1.AutonomousDatabaseRestore{} + + namespacedName := types.NamespacedName{ + Namespace: adbRestore.GetNamespace(), + Name: adbRestore.GetName(), + } + + if err := kubeClient.Get(context.TODO(), namespacedName, curBackup); err != nil { + return err + } + + curBackup.Status = adbRestore.Status + return kubeClient.Status().Update(context.TODO(), curBackup) + }) +} diff --git a/commons/oci/database.go b/commons/oci/database.go index e29afc78..b6a12f45 100644 --- a/commons/oci/database.go +++ b/commons/oci/database.go @@ -39,23 +39,27 @@ package oci import ( + "bytes" "context" + "encoding/json" "errors" "fmt" "math" + "net/http" "time" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/database" - "github.com/oracle/oci-go-sdk/v45/secrets" - "github.com/oracle/oci-go-sdk/v45/workrequests" + "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v51/database" + "github.com/oracle/oci-go-sdk/v51/secrets" + "github.com/oracle/oci-go-sdk/v51/workrequests" "sigs.k8s.io/controller-runtime/pkg/client" corev1 "k8s.io/api/core/v1" "k8s.io/apimachinery/pkg/types" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + "github.com/oracle/oracle-database-operator/commons/oci/ociutil" ) // CreateAutonomousDatabase sends a request to OCI to provision a database and returns the AutonomousDatabase OCID. @@ -448,7 +452,6 @@ func stopAutonomousDatabase(dbClient database.DatabaseClient, adbOCID string) (r // DeleteAutonomousDatabase terminates an Autonomous Database in OCI func DeleteAutonomousDatabase(dbClient database.DatabaseClient, adbOCID string) (resp database.DeleteAutonomousDatabaseResponse, err error) { - deleteRequest := database.DeleteAutonomousDatabaseRequest{ AutonomousDatabaseId: common.String(adbOCID), } @@ -457,6 +460,53 @@ func DeleteAutonomousDatabase(dbClient database.DatabaseClient, adbOCID string) return } +func RestoreAutonomousDatabase(dbClient database.DatabaseClient, adbOCID string, dateTime time.Time) (opcID string, err error) { + endpoint := dbClient.Endpoint() + url := fmt.Sprintf("%s/20160918/autonomousDatabases/%s/actions/restore", endpoint, adbOCID) + + // build the body + reqBody, err := json.Marshal(map[string]string{ + "timestamp": ociutil.FormatSDKTime(dateTime), + }) + if err != nil { + return "", err + } + + // create request + request, err := http.NewRequest("POST", url, bytes.NewBuffer(reqBody)) + if err != nil { + return "", err + } + + // Set the header + request.Header.Set("Date", time.Now().UTC().Format(http.TimeFormat)) + request.Header.Set("Content-Type", "application/json") + + hash, err := common.GetBodyHash(request) + if err != nil { + return "", err + } + request.Header.Set("x-content-sha256", hash) + request.Header.Set("Content-Length", fmt.Sprint(request.ContentLength)) + + // Build the signer + signer := common.DefaultRequestSigner(*dbClient.ConfigurationProvider()) + + // Sign the request + signer.Sign(request) + + client := http.Client{} + + // Execute the request + resp, err := client.Do(request) + if err != nil { + return "", err + } + + opcRequestId := resp.Header.Get("opc-work-request-id") + return opcRequestId, nil +} + // ListAutonomousDatabaseBackups returns a list of Autonomous Database backups func ListAutonomousDatabaseBackups(dbClient database.DatabaseClient, adb *dbv1alpha1.AutonomousDatabase) (resp database.ListAutonomousDatabaseBackupsResponse, err error) { if adb.Spec.Details.AutonomousDatabaseOCID == nil { diff --git a/commons/oci/ociutil/time.go b/commons/oci/ociutil/time.go new file mode 100644 index 00000000..47e7cc88 --- /dev/null +++ b/commons/oci/ociutil/time.go @@ -0,0 +1,13 @@ +package ociutil + +import "time" + +const sdkFormat = "2006-01-02T15:04:05.999Z07:00" + +func FormatSDKTime(dateTime time.Time) string { + return dateTime.Format(sdkFormat) +} + +func ParseSDKTime(val string) (time.Time, error) { + return time.Parse(sdkFormat, val) +} diff --git a/commons/oci/provider.go b/commons/oci/provider.go index f5bd6cda..32188e81 100644 --- a/commons/oci/provider.go +++ b/commons/oci/provider.go @@ -42,8 +42,8 @@ import ( "context" "errors" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/common/auth" + "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v51/common/auth" "sigs.k8s.io/controller-runtime/pkg/client" diff --git a/commons/oci/vault.go b/commons/oci/vault.go index 6bf790aa..5df1cc04 100644 --- a/commons/oci/vault.go +++ b/commons/oci/vault.go @@ -42,8 +42,8 @@ import ( "context" "encoding/base64" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/secrets" + "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v51/secrets" ) func getValueFromVaultSecret(secretClient secrets.SecretsClient, vaultSecretOCID string) (string, error) { diff --git a/commons/oci/wallet.go b/commons/oci/wallet.go index 3faa4f16..268e66c5 100644 --- a/commons/oci/wallet.go +++ b/commons/oci/wallet.go @@ -49,9 +49,9 @@ import ( "time" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/database" - "github.com/oracle/oci-go-sdk/v45/secrets" + "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v51/database" + "github.com/oracle/oci-go-sdk/v51/secrets" "sigs.k8s.io/controller-runtime/pkg/client" "k8s.io/apimachinery/pkg/types" diff --git a/commons/sharding/scommon.go b/commons/sharding/scommon.go index 55fdd5dc..5af51390 100644 --- a/commons/sharding/scommon.go +++ b/commons/sharding/scommon.go @@ -48,8 +48,8 @@ import ( "strings" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/ons" + "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v51/ons" appsv1 "k8s.io/api/apps/v1" corev1 "k8s.io/api/core/v1" "k8s.io/apimachinery/pkg/api/errors" diff --git a/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml b/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml index d47fd202..4e6ae88b 100644 --- a/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml +++ b/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml @@ -20,7 +20,7 @@ spec: scope: Namespaced versions: - additionalPrinterColumns: - - jsonPath: .spec.displayName + - jsonPath: .status.displayName name: Display Name type: string - jsonPath: .status.lifecycleState diff --git a/config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml b/config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml new file mode 100644 index 00000000..f4671cce --- /dev/null +++ b/config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml @@ -0,0 +1,93 @@ + +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.6.1 + creationTimestamp: null + name: autonomousdatabaserestores.database.oracle.com +spec: + group: database.oracle.com + names: + kind: AutonomousDatabaseRestore + listKind: AutonomousDatabaseRestoreList + plural: autonomousdatabaserestores + shortNames: + - adbr + - adbrs + singular: autonomousdatabaserestore + scope: Namespaced + versions: + - name: v1alpha1 + schema: + openAPIV3Schema: + description: AutonomousDatabaseRestore is the Schema for the autonomousdatabaserestores + API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: AutonomousDatabaseRestoreSpec defines the desired state of + AutonomousDatabaseRestore + properties: + autonomousDatabaseOCID: + description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster + Important: Run "make" to regenerate code after modifying this file' + type: string + ociConfig: + properties: + configMapName: + type: string + secretName: + type: string + type: object + source: + properties: + backupName: + type: string + datetime: + type: string + type: object + required: + - autonomousDatabaseOCID + - source + type: object + status: + description: AutonomousDatabaseRestoreStatus defines the observed state + of AutonomousDatabaseRestore + properties: + state: + description: 'INSERT ADDITIONAL STATUS FIELD - define observed state + of cluster Important: Run "make" to regenerate code after modifying + this file' + enum: + - New + - InProgress + - Completed + - Failed + type: string + required: + - state + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] diff --git a/config/crd/kustomization.yaml b/config/crd/kustomization.yaml index c6d82e86..589028fb 100644 --- a/config/crd/kustomization.yaml +++ b/config/crd/kustomization.yaml @@ -10,6 +10,7 @@ resources: - bases/database.oracle.com_singleinstancedatabases.yaml - bases/database.oracle.com_shardingdatabases.yaml - bases/database.oracle.com_autonomousdatabasebackups.yaml +- bases/database.oracle.com_autonomousdatabaserestores.yaml # +kubebuilder:scaffold:crdkustomizeresource patchesStrategicMerge: @@ -20,6 +21,7 @@ patchesStrategicMerge: #- patches/webhook_in_singleinstancedatabases.yaml #- patches/webhook_in_shardingdatabases.yaml #- patches/webhook_in_autonomousdatabasebackups.yaml +#- patches/webhook_in_autonomousdatabaserestores.yaml # +kubebuilder:scaffold:crdkustomizewebhookpatch # [CERTMANAGER] To enable webhook, uncomment all the sections with [CERTMANAGER] prefix. @@ -29,6 +31,7 @@ patchesStrategicMerge: - patches/cainjection_in_singleinstancedatabases.yaml #- patches/cainjection_in_shardingdatabases.yaml #- patches/cainjection_in_autonomousdatabasebackups.yaml +#- patches/cainjection_in_autonomousdatabaserestores.yaml # +kubebuilder:scaffold:crdkustomizecainjectionpatch # the following config is for teaching kustomize how to do kustomization for CRDs. diff --git a/config/crd/patches/cainjection_in_autonomousdatabaserestores.yaml b/config/crd/patches/cainjection_in_autonomousdatabaserestores.yaml new file mode 100644 index 00000000..75894cbb --- /dev/null +++ b/config/crd/patches/cainjection_in_autonomousdatabaserestores.yaml @@ -0,0 +1,8 @@ +# The following patch adds a directive for certmanager to inject CA into the CRD +# CRD conversion requires k8s 1.13 or later. +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + annotations: + cert-manager.io/inject-ca-from: $(CERTIFICATE_NAMESPACE)/$(CERTIFICATE_NAME) + name: autonomousdatabaserestores.database.oracle.com diff --git a/config/crd/patches/webhook_in_autonomousdatabaserestores.yaml b/config/crd/patches/webhook_in_autonomousdatabaserestores.yaml new file mode 100644 index 00000000..0a0ed4ad --- /dev/null +++ b/config/crd/patches/webhook_in_autonomousdatabaserestores.yaml @@ -0,0 +1,17 @@ +# The following patch enables conversion webhook for CRD +# CRD conversion requires k8s 1.13 or later. +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: autonomousdatabaserestores.database.oracle.com +spec: + conversion: + strategy: Webhook + webhookClientConfig: + # this is "\n" used as a placeholder, otherwise it will be rejected by the apiserver for being blank, + # but we're going to set it later using the cert-manager (or potentially a patch if not using cert-manager) + caBundle: Cg== + service: + namespace: system + name: webhook-service + path: /convert diff --git a/config/manager/kustomization.yaml b/config/manager/kustomization.yaml index 30ed1f75..d8632b06 100644 --- a/config/manager/kustomization.yaml +++ b/config/manager/kustomization.yaml @@ -8,5 +8,5 @@ apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization images: - name: controller - newName: container-registry.oracle.com/database/operator - newTag: 0.1.0 + newName: local-oracle-db-operator + newTag: v0.0.1beta diff --git a/config/manager/manager.yaml b/config/manager/manager.yaml index 362aac61..259b79a5 100644 --- a/config/manager/manager.yaml +++ b/config/manager/manager.yaml @@ -34,7 +34,7 @@ spec: args: - --enable-leader-election image: controller:latest - imagePullPolicy: Always + imagePullPolicy: Never #Always name: manager resources: limits: diff --git a/config/rbac/autonomousdatabaserestore_editor_role.yaml b/config/rbac/autonomousdatabaserestore_editor_role.yaml new file mode 100644 index 00000000..6efd98ae --- /dev/null +++ b/config/rbac/autonomousdatabaserestore_editor_role.yaml @@ -0,0 +1,24 @@ +# permissions for end users to edit autonomousdatabaserestores. +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: autonomousdatabaserestore-editor-role +rules: +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabaserestores + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabaserestores/status + verbs: + - get diff --git a/config/rbac/autonomousdatabaserestore_viewer_role.yaml b/config/rbac/autonomousdatabaserestore_viewer_role.yaml new file mode 100644 index 00000000..66cc7f51 --- /dev/null +++ b/config/rbac/autonomousdatabaserestore_viewer_role.yaml @@ -0,0 +1,20 @@ +# permissions for end users to view autonomousdatabaserestores. +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: autonomousdatabaserestore-viewer-role +rules: +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabaserestores + verbs: + - get + - list + - watch +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabaserestores/status + verbs: + - get diff --git a/config/rbac/role.yaml b/config/rbac/role.yaml index bef8ac3b..e1011e9d 100644 --- a/config/rbac/role.yaml +++ b/config/rbac/role.yaml @@ -127,6 +127,26 @@ rules: - get - patch - update +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabaserestores + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabaserestores/status + verbs: + - get + - patch + - update - apiGroups: - database.oracle.com resources: diff --git a/config/samples/database_v1alpha1_autonomousdatabaserestore.yaml b/config/samples/database_v1alpha1_autonomousdatabaserestore.yaml new file mode 100644 index 00000000..4e1cb9a1 --- /dev/null +++ b/config/samples/database_v1alpha1_autonomousdatabaserestore.yaml @@ -0,0 +1,7 @@ +apiVersion: database.oracle.com/v1alpha1 +kind: AutonomousDatabaseRestore +metadata: + name: autonomousdatabaserestore-sample +spec: + # Add fields here + foo: bar diff --git a/config/samples/kustomization.yaml b/config/samples/kustomization.yaml index 1a6e104a..1dd6fee0 100644 --- a/config/samples/kustomization.yaml +++ b/config/samples/kustomization.yaml @@ -10,4 +10,5 @@ resources: - singleinstancedatabase.yaml - shardingdatabase.yaml - autonomousdatabasebackup.yaml +- database_v1alpha1_autonomousdatabaserestore.yaml # +kubebuilder:scaffold:manifestskustomizesamples diff --git a/controllers/database/autonomousdatabase_controller.go b/controllers/database/autonomousdatabase_controller.go index 0a2bfbe8..05b8baaa 100644 --- a/controllers/database/autonomousdatabase_controller.go +++ b/controllers/database/autonomousdatabase_controller.go @@ -44,9 +44,9 @@ import ( "reflect" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v45/database" - "github.com/oracle/oci-go-sdk/v45/secrets" - "github.com/oracle/oci-go-sdk/v45/workrequests" + "github.com/oracle/oci-go-sdk/v51/database" + "github.com/oracle/oci-go-sdk/v51/secrets" + "github.com/oracle/oci-go-sdk/v51/workrequests" apiErrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/runtime" @@ -509,7 +509,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R return ctrl.Result{}, err } - r.currentLogger.Info("AutonomousDatabase resoursce reconcile successfully") + r.currentLogger.Info("AutonomousDatabase resource reconcile successfully") return ctrl.Result{}, nil } diff --git a/controllers/database/autonomousdatabasebackup_controller.go b/controllers/database/autonomousdatabasebackup_controller.go index 966182f1..fa83a823 100644 --- a/controllers/database/autonomousdatabasebackup_controller.go +++ b/controllers/database/autonomousdatabasebackup_controller.go @@ -51,8 +51,8 @@ import ( "sigs.k8s.io/controller-runtime/pkg/event" "sigs.k8s.io/controller-runtime/pkg/predicate" - "github.com/oracle/oci-go-sdk/v45/database" - "github.com/oracle/oci-go-sdk/v45/workrequests" + "github.com/oracle/oci-go-sdk/v51/database" + "github.com/oracle/oci-go-sdk/v51/workrequests" databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" backupUtil "github.com/oracle/oracle-database-operator/commons/autonomousdatabase" @@ -160,7 +160,7 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req } /****************************************************************** - * Create a backup if the AutonomousDatabaseBackupOCID is empty, + * Create a backup if the Spec.AutonomousDatabaseBackupOCID is empty, * otherwise bind to an exisiting backup ******************************************************************/ if adbBackup.Spec.AutonomousDatabaseBackupOCID == "" { diff --git a/controllers/database/autonomousdatabaserestore_controller.go b/controllers/database/autonomousdatabaserestore_controller.go new file mode 100644 index 00000000..ed9dbb8e --- /dev/null +++ b/controllers/database/autonomousdatabaserestore_controller.go @@ -0,0 +1,216 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package controllers + +import ( + "context" + "time" + + "github.com/go-logr/logr" + apiErrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/types" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" + + "github.com/oracle/oci-go-sdk/v51/database" + "github.com/oracle/oci-go-sdk/v51/workrequests" + databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + restoreUtil "github.com/oracle/oracle-database-operator/commons/autonomousdatabase" + "github.com/oracle/oracle-database-operator/commons/oci" + "github.com/oracle/oracle-database-operator/commons/oci/ociutil" +) + +// AutonomousDatabaseRestoreReconciler reconciles a AutonomousDatabaseRestore object +type AutonomousDatabaseRestoreReconciler struct { + KubeClient client.Client + Log logr.Logger + Scheme *runtime.Scheme + + currentLogger logr.Logger +} + +//+kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabaserestores,verbs=get;list;watch;create;update;patch;delete +//+kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabaserestores/status,verbs=get;update;patch + +// Reconcile is part of the main kubernetes reconciliation loop which aims to +// move the current state of the cluster closer to the desired state. +// TODO(user): Modify the Reconcile function to compare the state specified by +// the AutonomousDatabaseRestore object against the actual cluster state, and then +// perform operations to make the cluster state reflect the state specified by +// the user. +// +// For more details, check Reconcile and its Result here: +// - https://pkg.go.dev/sigs.k8s.io/controller-runtime@v0.6.4/pkg/reconcile +func (r *AutonomousDatabaseRestoreReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + r.currentLogger = r.Log.WithValues("autonomousdatabaserestore", req.NamespacedName) + + restore := &dbv1alpha1.AutonomousDatabaseRestore{} + if err := r.KubeClient.Get(context.TODO(), req.NamespacedName, restore); err != nil { + // Ignore not-found errors, since they can't be fixed by an immediate requeue. + // No need to change the since we don't know if we obtain the object. + if !apiErrors.IsNotFound(err) { + return ctrl.Result{}, err + } + } + + /****************************************************************** + * Get OCI database client and work request client + ******************************************************************/ + authData := oci.APIKeyAuth{ + ConfigMapName: restore.Spec.OCIConfig.ConfigMapName, + SecretName: restore.Spec.OCIConfig.SecretName, + Namespace: restore.GetNamespace(), + } + provider, err := oci.GetOCIProvider(r.KubeClient, authData) + if err != nil { + r.currentLogger.Error(err, "Fail to get OCI provider") + + // Change the status to UNAVAILABLE + restore.Status.LifecycleState = dbv1alpha1.RestoreLifecycleStateFailed + if statusErr := restoreUtil.UpdateAutonomousDatabaseRestoreStatus(r.KubeClient, restore); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, nil + } + + dbClient, err := database.NewDatabaseClientWithConfigurationProvider(provider) + if err != nil { + r.currentLogger.Error(err, "Fail to get OCI database client") + + // Change the status to UNAVAILABLE + restore.Status.LifecycleState = dbv1alpha1.RestoreLifecycleStateFailed + if statusErr := restoreUtil.UpdateAutonomousDatabaseRestoreStatus(r.KubeClient, restore); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, nil + } + + workClient, err := workrequests.NewWorkRequestClientWithConfigurationProvider(provider) + if err != nil { + r.currentLogger.Error(err, "Fail to get OCI work request client") + + // Change the status to UNAVAILABLE + restore.Status.LifecycleState = dbv1alpha1.RestoreLifecycleStateFailed + if statusErr := restoreUtil.UpdateAutonomousDatabaseRestoreStatus(r.KubeClient, restore); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, nil + } + + /****************************************************************** + * Restore + ******************************************************************/ + if restore.Status.LifecycleState == "" || restore.Status.LifecycleState == dbv1alpha1.RestoreLifecycleStateNew { + var restoreTime time.Time + + if restore.Spec.Source.BackupName != "" { + backup := &dbv1alpha1.AutonomousDatabaseBackup{} + namespacedName := types.NamespacedName{Namespace: restore.Namespace, Name: restore.Spec.Source.BackupName} + if err := r.KubeClient.Get(context.TODO(), namespacedName, backup); err != nil { + return ctrl.Result{}, err + } + + restoreTime, err = ociutil.ParseSDKTime(backup.Status.TimeEnded) + if err != nil { + r.currentLogger.Error(err, "Fail to parse time "+backup.Status.TimeEnded) + + // Change the status to UNAVAILABLE + restore.Status.LifecycleState = dbv1alpha1.RestoreLifecycleStateFailed + if statusErr := restoreUtil.UpdateAutonomousDatabaseRestoreStatus(r.KubeClient, restore); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, nil + } + } else if restore.Spec.Source.DateTime != "" { + restoreTime, err = ociutil.ParseSDKTime(restore.Spec.Source.DateTime) + if err != nil { + r.currentLogger.Error(err, "Fail to parse time "+restore.Spec.Source.DateTime) + + // Change the status to UNAVAILABLE + restore.Status.LifecycleState = dbv1alpha1.RestoreLifecycleStateFailed + if statusErr := restoreUtil.UpdateAutonomousDatabaseRestoreStatus(r.KubeClient, restore); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, nil + } + } + + opcID, err := oci.RestoreAutonomousDatabase(dbClient, restore.Spec.AutonomousDatabaseOCID, restoreTime) + + if err != nil { + r.currentLogger.Error(err, "Fail to get OCI work request client") + + // Change the status to UNAVAILABLE + restore.Status.LifecycleState = dbv1alpha1.RestoreLifecycleStateFailed + if statusErr := restoreUtil.UpdateAutonomousDatabaseRestoreStatus(r.KubeClient, restore); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, nil + } + + if err := oci.WaitUntilWorkCompleted(r.currentLogger, workClient, &opcID); err != nil { + r.currentLogger.Error(err, "Fail to watch workrequest. Workrequest ID = "+opcID) + + // Change the status to UNAVAILABLE + restore.Status.LifecycleState = dbv1alpha1.RestoreLifecycleStateFailed + if statusErr := restoreUtil.UpdateAutonomousDatabaseRestoreStatus(r.KubeClient, restore); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, nil + } + + r.currentLogger.Info("Restore database completed") + + restore.Status.LifecycleState = dbv1alpha1.RestoreLifecycleStateCompleted + if statusErr := restoreUtil.UpdateAutonomousDatabaseRestoreStatus(r.KubeClient, restore); statusErr != nil { + return ctrl.Result{}, statusErr + } + } + + return ctrl.Result{}, nil +} + +// SetupWithManager sets up the controller with the Manager. +func (r *AutonomousDatabaseRestoreReconciler) SetupWithManager(mgr ctrl.Manager) error { + return ctrl.NewControllerManagedBy(mgr). + For(&databasev1alpha1.AutonomousDatabaseRestore{}). + Complete(r) +} diff --git a/controllers/database/shardingdatabase_controller.go b/controllers/database/shardingdatabase_controller.go index 73575df7..f30e714b 100644 --- a/controllers/database/shardingdatabase_controller.go +++ b/controllers/database/shardingdatabase_controller.go @@ -47,8 +47,8 @@ import ( "time" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/ons" + "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v51/ons" appsv1 "k8s.io/api/apps/v1" corev1 "k8s.io/api/core/v1" "k8s.io/apimachinery/pkg/api/errors" diff --git a/controllers/database/suite_test.go b/controllers/database/suite_test.go index 9b62324a..fdebedce 100644 --- a/controllers/database/suite_test.go +++ b/controllers/database/suite_test.go @@ -87,6 +87,9 @@ var _ = BeforeSuite(func(done Done) { err = databasev1alpha1.AddToScheme(scheme.Scheme) Expect(err).NotTo(HaveOccurred()) + err = databasev1alpha1.AddToScheme(scheme.Scheme) + Expect(err).NotTo(HaveOccurred()) + // +kubebuilder:scaffold:scheme k8sClient, err = client.New(cfg, client.Options{Scheme: scheme.Scheme}) diff --git a/go.mod b/go.mod index fada426a..4e19053e 100644 --- a/go.mod +++ b/go.mod @@ -6,7 +6,7 @@ require ( github.com/go-logr/logr v0.4.0 github.com/onsi/ginkgo v1.16.4 github.com/onsi/gomega v1.13.0 - github.com/oracle/oci-go-sdk/v45 v45.2.0 + github.com/oracle/oci-go-sdk/v51 v51.0.0 gopkg.in/yaml.v2 v2.4.0 k8s.io/api v0.21.2 k8s.io/apimachinery v0.21.2 diff --git a/go.sum b/go.sum index 67d9f3df..2b051eef 100644 --- a/go.sum +++ b/go.sum @@ -304,8 +304,8 @@ github.com/onsi/gomega v1.7.1/go.mod h1:XdKZgCCFLUoM/7CFJVPcG8C1xQ1AJ0vpAezJrB7J github.com/onsi/gomega v1.10.1/go.mod h1:iN09h71vgCQne3DLsj+A5owkum+a2tYe+TOCB1ybHNo= github.com/onsi/gomega v1.13.0 h1:7lLHu94wT9Ij0o6EWWclhu0aOh32VxhkwEJvzuWPeak= github.com/onsi/gomega v1.13.0/go.mod h1:lRk9szgn8TxENtWd0Tp4c3wjlRfMTMH27I+3Je41yGY= -github.com/oracle/oci-go-sdk/v45 v45.2.0 h1:vCPoQlE+DOrM2heJn66rvPU6fbsc/0Cxtzs2jnFut6U= -github.com/oracle/oci-go-sdk/v45 v45.2.0/go.mod h1:ZM6LGiRO5TPQJxTlrXbcHMbClE775wnGD5U/EerCsRw= +github.com/oracle/oci-go-sdk/v51 v51.0.0 h1:eDUVMsAzvf+jfq4xbtpQrxznYNjccwKKyIhmRvZLgwI= +github.com/oracle/oci-go-sdk/v51 v51.0.0/go.mod h1:d9KSNXwE64drofxoor+y/JWofJqLqRF9D1/AtfYIE10= github.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc= github.com/pelletier/go-toml v1.2.0/go.mod h1:5z9KED0ma1S8pY6P1sdut58dfprrGBbd/94hg7ilaic= github.com/peterbourgon/diskv v2.0.1+incompatible/go.mod h1:uqqh8zWWbv1HBMNONnaR/tNboyR3/BZd58JJSHlUSCU= @@ -355,6 +355,8 @@ github.com/sirupsen/logrus v1.7.0/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1:OnSkiWE9lh6wB0YB77sQom3nweQdgAjqCqsofrRNTgc= github.com/smartystreets/goconvey v1.6.4/go.mod h1:syvi0/a8iFYH4r/RixwvyeAJjdLS9QV7WQ/tjFTllLA= github.com/soheilhy/cmux v0.1.4/go.mod h1:IM3LyeVVIOuxMH7sFAkER9+bJ4dT7Ms6E4xg4kGIyLM= +github.com/sony/gobreaker v0.4.2-0.20210216022020-dd874f9dd33b h1:br+bPNZsJWKicw/5rALEo67QHs5weyD5tf8WST+4sJ0= +github.com/sony/gobreaker v0.4.2-0.20210216022020-dd874f9dd33b/go.mod h1:ZKptC7FHNvhBz7dN2LGjPVBz2sZJmc0/PkyDJOjmxWY= github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA= github.com/spf13/afero v1.1.2/go.mod h1:j4pytiNVoe2o6bmDsKpLACNPDBIoEAkihy7loJ1B0CQ= github.com/spf13/afero v1.2.2/go.mod h1:9ZxEEn6pIJ8Rxe320qSDBk6AsU0r9pR7Q4OcevTdifk= diff --git a/main.go b/main.go index 8ebfa939..3049c485 100644 --- a/main.go +++ b/main.go @@ -106,6 +106,14 @@ func main() { setupLog.Error(err, "unable to create controller", "controller", "AutonomousDatabaseBackup") os.Exit(1) } + if err = (&databasecontroller.AutonomousDatabaseRestoreReconciler{ + KubeClient: mgr.GetClient(), + Log: ctrl.Log.WithName("controllers").WithName("AutonomousDatabaseRestore"), + Scheme: mgr.GetScheme(), + }).SetupWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create controller", "controller", "AutonomousDatabaseRestore") + os.Exit(1) + } if err = (&databasecontroller.SingleInstanceDatabaseReconciler{ Client: mgr.GetClient(), Log: ctrl.Log.WithName("controllers").WithName("database").WithName("SingleInstanceDatabase"), diff --git a/test/e2e/autonomousdatabase_controller_bind_test.go b/test/e2e/autonomousdatabase_controller_bind_test.go index ff415da9..3ab132c9 100644 --- a/test/e2e/autonomousdatabase_controller_bind_test.go +++ b/test/e2e/autonomousdatabase_controller_bind_test.go @@ -44,9 +44,9 @@ import ( . "github.com/onsi/ginkgo" . "github.com/onsi/gomega" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/database" - "github.com/oracle/oci-go-sdk/v45/workrequests" + "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v51/database" + "github.com/oracle/oci-go-sdk/v51/workrequests" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/types" diff --git a/test/e2e/autonomousdatabase_controller_create_test.go b/test/e2e/autonomousdatabase_controller_create_test.go index 3ecd2804..874ada3a 100644 --- a/test/e2e/autonomousdatabase_controller_create_test.go +++ b/test/e2e/autonomousdatabase_controller_create_test.go @@ -44,8 +44,8 @@ import ( . "github.com/onsi/ginkgo" . "github.com/onsi/gomega" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/database" + "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v51/database" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/types" diff --git a/test/e2e/behavior/shared_behaviors.go b/test/e2e/behavior/shared_behaviors.go index 501a93b2..9adb7121 100644 --- a/test/e2e/behavior/shared_behaviors.go +++ b/test/e2e/behavior/shared_behaviors.go @@ -47,8 +47,8 @@ import ( . "github.com/onsi/ginkgo" . "github.com/onsi/gomega" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/database" + "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v51/database" corev1 "k8s.io/api/core/v1" k8sErrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/types" diff --git a/test/e2e/suite_test.go b/test/e2e/suite_test.go index 3fd5cd2e..7cfb9424 100644 --- a/test/e2e/suite_test.go +++ b/test/e2e/suite_test.go @@ -47,8 +47,8 @@ import ( . "github.com/onsi/ginkgo" . "github.com/onsi/gomega" "github.com/onsi/gomega/gexec" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/database" + "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v51/database" corev1 "k8s.io/api/core/v1" "k8s.io/client-go/kubernetes/scheme" "k8s.io/client-go/rest" diff --git a/test/e2e/util/oci_config_util.go b/test/e2e/util/oci_config_util.go index 70ee1cab..0c8b8ba9 100644 --- a/test/e2e/util/oci_config_util.go +++ b/test/e2e/util/oci_config_util.go @@ -48,7 +48,7 @@ import ( "regexp" "strings" - "github.com/oracle/oci-go-sdk/v45/common" + "github.com/oracle/oci-go-sdk/v51/common" corev1 "k8s.io/api/core/v1" ) diff --git a/test/e2e/util/oci_db_request.go b/test/e2e/util/oci_db_request.go index 5fcd3aa1..056e1c40 100644 --- a/test/e2e/util/oci_db_request.go +++ b/test/e2e/util/oci_db_request.go @@ -41,8 +41,8 @@ package e2eutil import ( "context" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/database" + "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v51/database" "time" ) diff --git a/test/e2e/util/oci_vault_request.go b/test/e2e/util/oci_vault_request.go index ef7a793c..9cdc210b 100644 --- a/test/e2e/util/oci_vault_request.go +++ b/test/e2e/util/oci_vault_request.go @@ -44,9 +44,9 @@ import ( "math" "time" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/keymanagement" - "github.com/oracle/oci-go-sdk/v45/vault" + "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v51/keymanagement" + "github.com/oracle/oci-go-sdk/v51/vault" ) func waitForVaultStatePolicy(state keymanagement.VaultLifecycleStateEnum) common.RetryPolicy { diff --git a/test/e2e/util/oci_work_request.go b/test/e2e/util/oci_work_request.go index 4721fa0a..450d9c3f 100644 --- a/test/e2e/util/oci_work_request.go +++ b/test/e2e/util/oci_work_request.go @@ -41,8 +41,8 @@ package e2eutil import ( "context" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/workrequests" + "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v51/workrequests" "time" ) diff --git a/test/e2e/util/util.go b/test/e2e/util/util.go index 885985f1..0e6c97a7 100644 --- a/test/e2e/util/util.go +++ b/test/e2e/util/util.go @@ -73,7 +73,7 @@ func unmarshalFromYamlBytes(bytes []byte, obj interface{}) error { return json.Unmarshal(jsonBytes, obj) } -// LoadTestFixture create an AutonomousDatabase resoursce from a test fixture +// LoadTestFixture create an AutonomousDatabase resource from a test fixture func LoadTestFixture(adb *dbv1alpha1.AutonomousDatabase, filename string) (*dbv1alpha1.AutonomousDatabase, error) { filePath := "./resource/" + filename yamlBytes, err := ioutil.ReadFile(filePath) From 1ea823a1bf4d125ca290523fb9486cd6d5f168d1 Mon Sep 17 00:00:00 2001 From: Gajanan Bhat Date: Thu, 25 Nov 2021 12:00:34 +0530 Subject: [PATCH 049/628] Added PDB and CDB Controller --- .gitignore | 5 + PROJECT | 26 + apis/database/v1alpha1/cdb_types.go | 168 ++++ apis/database/v1alpha1/cdb_webhook.go | 156 ++++ apis/database/v1alpha1/pdb_types.go | 177 ++++ apis/database/v1alpha1/pdb_webhook.go | 279 ++++++ .../v1alpha1/zz_generated.deepcopy.go | 415 +++++++++ .../crd/bases/database.oracle.com_cdbs.yaml | 235 +++++ .../crd/bases/database.oracle.com_pdbs.yaml | 260 ++++++ config/crd/kustomization.yaml | 6 + config/crd/patches/cainjection_in_cdbs.yaml | 8 + config/crd/patches/cainjection_in_pdbs.yaml | 8 + config/crd/patches/webhook_in_cdbs.yaml | 17 + config/crd/patches/webhook_in_pdbs.yaml | 17 + config/rbac/cdb_editor_role.yaml | 24 + config/rbac/cdb_viewer_role.yaml | 20 + config/rbac/pdb_editor_role.yaml | 24 + config/rbac/pdb_viewer_role.yaml | 20 + config/rbac/role.yaml | 89 ++ config/samples/kustomization.yaml | 2 + config/samples/onpremdb/cdb.yaml | 40 + config/samples/onpremdb/cdb_secret.yaml | 17 + config/samples/onpremdb/pdb.yaml | 27 + config/samples/onpremdb/pdb_clone.yaml | 27 + config/samples/onpremdb/pdb_delete.yaml | 16 + config/samples/onpremdb/pdb_plug.yaml | 19 + config/samples/onpremdb/pdb_secret.yaml | 13 + config/samples/onpremdb/pdb_unplug.yaml | 27 + config/webhook/manifests.yaml | 87 ++ controllers/database/cdb_controller.go | 671 ++++++++++++++ controllers/database/pdb_controller.go | 860 ++++++++++++++++++ docs/onpremdb/README.md | 189 ++++ main.go | 34 + ords/Dockerfile | 66 ++ ords/cdbadmin.properties.tmpl | 3 + ords/ords_params.properties.tmpl | 16 + ords/runOrds.sh | 75 ++ ords/setupwebuser.sh | 20 + ords/standalone.properties.tmpl | 8 + 39 files changed, 4171 insertions(+) create mode 100644 .gitignore create mode 100644 apis/database/v1alpha1/cdb_types.go create mode 100644 apis/database/v1alpha1/cdb_webhook.go create mode 100644 apis/database/v1alpha1/pdb_types.go create mode 100644 apis/database/v1alpha1/pdb_webhook.go create mode 100644 config/crd/bases/database.oracle.com_cdbs.yaml create mode 100644 config/crd/bases/database.oracle.com_pdbs.yaml create mode 100644 config/crd/patches/cainjection_in_cdbs.yaml create mode 100644 config/crd/patches/cainjection_in_pdbs.yaml create mode 100644 config/crd/patches/webhook_in_cdbs.yaml create mode 100644 config/crd/patches/webhook_in_pdbs.yaml create mode 100644 config/rbac/cdb_editor_role.yaml create mode 100644 config/rbac/cdb_viewer_role.yaml create mode 100644 config/rbac/pdb_editor_role.yaml create mode 100644 config/rbac/pdb_viewer_role.yaml create mode 100644 config/samples/onpremdb/cdb.yaml create mode 100644 config/samples/onpremdb/cdb_secret.yaml create mode 100644 config/samples/onpremdb/pdb.yaml create mode 100644 config/samples/onpremdb/pdb_clone.yaml create mode 100644 config/samples/onpremdb/pdb_delete.yaml create mode 100644 config/samples/onpremdb/pdb_plug.yaml create mode 100644 config/samples/onpremdb/pdb_secret.yaml create mode 100644 config/samples/onpremdb/pdb_unplug.yaml create mode 100644 controllers/database/cdb_controller.go create mode 100644 controllers/database/pdb_controller.go create mode 100644 docs/onpremdb/README.md create mode 100644 ords/Dockerfile create mode 100644 ords/cdbadmin.properties.tmpl create mode 100644 ords/ords_params.properties.tmpl create mode 100644 ords/runOrds.sh create mode 100644 ords/setupwebuser.sh create mode 100644 ords/standalone.properties.tmpl diff --git a/.gitignore b/.gitignore new file mode 100644 index 00000000..8a2f9064 --- /dev/null +++ b/.gitignore @@ -0,0 +1,5 @@ +operator.tgz +cover.out +bin +testbin/* +onpremtest/* diff --git a/PROJECT b/PROJECT index 70e34778..07cf84c7 100644 --- a/PROJECT +++ b/PROJECT @@ -38,4 +38,30 @@ resources: kind: ShardingDatabase path: github.com/oracle/oracle-database-operator/apis/database/v1alpha1 version: v1alpha1 +- api: + crdVersion: v1 + namespaced: true + controller: true + domain: oracle.com + group: database + kind: PDB + path: github.com/oracle/oracle-database-operator/apis/database/v1alpha1 + version: v1alpha1 + webhooks: + defaulting: true + validation: true + webhookVersion: v1beta1 +- api: + crdVersion: v1 + namespaced: true + controller: true + domain: oracle.com + group: database + kind: CDB + path: github.com/oracle/oracle-database-operator/apis/database/v1alpha1 + version: v1alpha1 + webhooks: + defaulting: true + validation: true + webhookVersion: v1beta1 version: "3" diff --git a/apis/database/v1alpha1/cdb_types.go b/apis/database/v1alpha1/cdb_types.go new file mode 100644 index 00000000..2268ff26 --- /dev/null +++ b/apis/database/v1alpha1/cdb_types.go @@ -0,0 +1,168 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v1alpha1 + +import ( + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! +// NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized. + +// CDBSpec defines the desired state of CDB +type CDBSpec struct { + // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster + // Important: Run "make" to regenerate code after modifying this file + + // Name of the CDB + CDBName string `json:"cdbName,omitempty"` + // Name of the CDB Service + ServiceName string `json:"serviceName,omitempty"` + // Password for the CDB System Administrator + SysAdminPwd CDBSysAdminPassword `json:"sysAdminPwd,omitempty"` + // User in the root container with sysdba priviledges to manage PDB lifecycle + CDBAdminUser CDBAdminUser `json:"cdbAdminUser,omitempty"` + // Password for the CDB Administrator to manage PDB lifecycle + CDBAdminPwd CDBAdminPassword `json:"cdbAdminPwd,omitempty"` + // Password for user ORDS_PUBLIC_USER + ORDSPwd ORDSPassword `json:"ordsPwd,omitempty"` + // ORDS server port. For now, keep it as 8888. TO BE USED IN FUTURE RELEASE. + ORDSPort int `json:"ordsPort,omitempty"` + // ORDS Image Name + ORDSImage string `json:"ordsImage,omitempty"` + // The name of the image pull secret in case of a private docker repository. + ORDSImagePullSecret string `json:"ordsImagePullSecret,omitempty"` + // ORDS Image Pull Policy + // +kubebuilder:validation:Enum=Always;Never + ORDSImagePullPolicy string `json:"ordsImagePullPolicy,omitempty"` + // Number of ORDS Containers to create + Replicas int `json:"replicas,omitempty"` + // Web Server User with SQL Administrator role to allow us to authenticate to the PDB Lifecycle Management REST endpoints + WebServerUser WebServerUser `json:"webServerUser,omitempty"` + // Password for the Web Server User + WebServerPwd WebServerPassword `json:"webServerPwd,omitempty"` + // SCAN Name + SCANName string `json:"scanName,omitempty"` + // Name of the DB server + DBServer string `json:"dbServer,omitempty"` + // DB server port + DBPort int `json:"dbPort,omitempty"` + // Node Selector for running the Pod + NodeSelector map[string]string `json:"nodeSelector,omitempty"` +} + +// CDBSecret defines the secretName +type CDBSecret struct { + SecretName string `json:"secretName"` + Key string `json:"key"` +} + +// CDBSysAdminPassword defines the secret containing SysAdmin Password mapped to key 'sysAdminPwd' for CDB +type CDBSysAdminPassword struct { + Secret CDBSecret `json:"secret"` +} + +// CDBAdminUser defines the secret containing CDB Administrator User mapped to key 'cdbAdminUser' to manage PDB lifecycle +type CDBAdminUser struct { + Secret CDBSecret `json:"secret"` +} + +// CDBAdminPassword defines the secret containing CDB Administrator Password mapped to key 'cdbAdminPwd' to manage PDB lifecycle +type CDBAdminPassword struct { + Secret CDBSecret `json:"secret"` +} + +// ORDSPassword defines the secret containing ORDS_PUBLIC_USER Password mapped to key 'ordsPwd' +type ORDSPassword struct { + Secret CDBSecret `json:"secret"` +} + +// WebServerUser defines the secret containing Web Server User mapped to key 'webServerUser' to manage PDB lifecycle +type WebServerUser struct { + Secret CDBSecret `json:"secret"` +} + +// WebServerPassword defines the secret containing password for Web Server User mapped to key 'webServerPwd' to manage PDB lifecycle +type WebServerPassword struct { + Secret CDBSecret `json:"secret"` +} + +// CDBStatus defines the observed state of CDB +type CDBStatus struct { + // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster + // Important: Run "make" to regenerate code after modifying this file + + // Phase of the CDB Resource + Phase string `json:"phase"` + // CDB Resource Status + Status bool `json:"status"` + // Message + Msg string `json:"msg,omitempty"` +} + +// +kubebuilder:object:root=true +// +kubebuilder:subresource:status +// +kubebuilder:printcolumn:JSONPath=".spec.cdbName",name="CDB Name",type="string",description="Name of the CDB" +// +kubebuilder:printcolumn:JSONPath=".spec.dbServer",name="DB Server",type="string",description=" Name of the DB Server" +// +kubebuilder:printcolumn:JSONPath=".spec.dbPort",name="DB Port",type="integer",description="DB server port" +// +kubebuilder:printcolumn:JSONPath=".spec.scanName",name="SCAN NAme",type="string",description="SCAN Name" +// +kubebuilder:printcolumn:JSONPath=".status.phase",name="Status",type="string",description="Status of the CDB Resource" +// +kubebuilder:printcolumn:JSONPath=".status.msg",name="Message",type="string",description="Error message, if any" + +// CDB is the Schema for the cdbs API +type CDB struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + Spec CDBSpec `json:"spec,omitempty"` + Status CDBStatus `json:"status,omitempty"` +} + +//+kubebuilder:object:root=true + +// CDBList contains a list of CDB +type CDBList struct { + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata,omitempty"` + Items []CDB `json:"items"` +} + +func init() { + SchemeBuilder.Register(&CDB{}, &CDBList{}) +} diff --git a/apis/database/v1alpha1/cdb_webhook.go b/apis/database/v1alpha1/cdb_webhook.go new file mode 100644 index 00000000..cd84c36f --- /dev/null +++ b/apis/database/v1alpha1/cdb_webhook.go @@ -0,0 +1,156 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v1alpha1 + +import ( + "reflect" + //"strings" + + apierrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/runtime/schema" + "k8s.io/apimachinery/pkg/util/validation/field" + ctrl "sigs.k8s.io/controller-runtime" + logf "sigs.k8s.io/controller-runtime/pkg/log" + "sigs.k8s.io/controller-runtime/pkg/webhook" +) + +// log is for logging in this package. +var cdblog = logf.Log.WithName("cdb-resource") + +func (r *CDB) SetupWebhookWithManager(mgr ctrl.Manager) error { + return ctrl.NewWebhookManagedBy(mgr). + For(r). + Complete() +} + +// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! + +//+kubebuilder:webhook:path=/mutate-database-oracle-com-v1alpha1-cdb,mutating=true,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=cdbs,verbs=create;update,versions=v1alpha1,name=mcdb.kb.io,admissionReviewVersions={v1,v1beta1} + +var _ webhook.Defaulter = &CDB{} + +// Default implements webhook.Defaulter so a webhook will be registered for the type +func (r *CDB) Default() { + cdblog.Info("Setting default values in CDB spec for : " + r.Name) + + if r.Spec.ORDSPort == 0 { + r.Spec.ORDSPort = 8888 + } + + if r.Spec.Replicas == 0 { + r.Spec.Replicas = 1 + } +} + +// TODO(user): change verbs to "verbs=create;update;delete" if you want to enable deletion validation. +//+kubebuilder:webhook:path=/validate-database-oracle-com-v1alpha1-cdb,mutating=false,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=cdbs,verbs=create;update,versions=v1alpha1,name=vcdb.kb.io,admissionReviewVersions={v1,v1beta1} + +var _ webhook.Validator = &CDB{} + +// ValidateCreate implements webhook.Validator so a webhook will be registered for the type +func (r *CDB) ValidateCreate() error { + cdblog.Info("Validating CDB spec for : " + r.Name) + cdblog.Info("ValidateCreate", "name", r.Name) + + var allErrs field.ErrorList + + if r.Spec.ServiceName == "" { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("serviceName"), "Please specify CDB Service name")) + } + if r.Spec.SCANName == "" { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("scanName"), "Please specify SCAN Name for CDB")) + } + if r.Spec.DBServer == "" { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("dbServer"), "Please specify Database Server Name or IP Address")) + } + if r.Spec.DBPort == 0 { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("dbPort"), "Please specify DB Server Port")) + } + if r.Spec.ORDSImage == "" { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("ordsImage"), "Please specify name of ORDS Image to be used")) + } + if reflect.ValueOf(r.Spec.CDBAdminUser).IsZero() { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("cdbAdminUser"), "Please specify user in the root container with sysdba priviledges to manage PDB lifecycle")) + } + if reflect.ValueOf(r.Spec.CDBAdminPwd).IsZero() { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("cdbAdminPwd"), "Please specify password for the CDB Administrator to manage PDB lifecycle")) + } + if reflect.ValueOf(r.Spec.ORDSPwd).IsZero() { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("ordsPwd"), "Please specify password for user ORDS_PUBLIC_USER")) + } + if reflect.ValueOf(r.Spec.WebServerUser).IsZero() { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("webServerUser"), "Please specify the Web Server User having SQL Administrator role")) + } + if reflect.ValueOf(r.Spec.WebServerPwd).IsZero() { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("webServerPwd"), "Please specify password for the Web Server User having SQL Administrator role")) + } + if len(allErrs) == 0 { + return nil + } + return apierrors.NewInvalid( + schema.GroupKind{Group: "database.oracle.com", Kind: "PDB"}, + r.Name, allErrs) +} + +// ValidateUpdate implements webhook.Validator so a webhook will be registered for the type +func (r *CDB) ValidateUpdate(old runtime.Object) error { + cdblog.Info("validate update", "name", r.Name) + + // TODO(user): fill in your validation logic upon object update. + return nil +} + +// ValidateDelete implements webhook.Validator so a webhook will be registered for the type +func (r *CDB) ValidateDelete() error { + cdblog.Info("validate delete", "name", r.Name) + + // TODO(user): fill in your validation logic upon object deletion. + return nil +} diff --git a/apis/database/v1alpha1/pdb_types.go b/apis/database/v1alpha1/pdb_types.go new file mode 100644 index 00000000..e04f94c5 --- /dev/null +++ b/apis/database/v1alpha1/pdb_types.go @@ -0,0 +1,177 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v1alpha1 + +import ( + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! +// NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized. + +// PDBSpec defines the desired state of PDB +type PDBSpec struct { + // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster + // Important: Run "make" to regenerate code after modifying this file + + // Name of the CDB Custom Resource that runs the ORDS container + CDBResName string `json:"cdbResName,omitempty"` + // Name of the CDB + CDBName string `json:"cdbName,omitempty"` + // The name of the new PDB. Relevant for both Create and Plug Actions. + PDBName string `json:"pdbName,omitempty"` + // Name of the Source PDB from which to clone + SrcPDBName string `json:"srcPdbName,omitempty"` + // The administrator username for the new PDB. This property is required when the Action property is Create. + AdminName PDBAdminName `json:"adminName,omitempty"` + // The administrator password for the new PDB. This property is required when the Action property is Create. + AdminPwd PDBAdminPassword `json:"adminPwd,omitempty"` + // Relevant for Create and Plug operations. As defined in the Oracle Multitenant Database documentation. Values can be a filename convert pattern or NONE. + FileNameConversions string `json:"fileNameConversions,omitempty"` + // This property is required when the Action property is Plug. As defined in the Oracle Multitenant Database documentation. Values can be a source filename convert pattern or NONE. + SourceFileNameConversions string `json:"sourceFileNameConversions,omitempty"` + // XML metadata filename to be used for Plug or Unplug operations + XMLFileName string `json:"xmlFileName,omitempty"` + // To copy files or not while cloning a PDB + // +kubebuilder:validation:Enum=COPY;NOCOPY;MOVE + CopyAction string `json:"copyAction,omitempty"` + // Specify if datafiles should be removed or not. The value can be INCLUDING or KEEP (default). + // +kubebuilder:validation:Enum=INCLUDING;KEEP + DropAction string `json:"dropAction,omitempty"` + // A Path specified for sparse clone snapshot copy. (Optional) + SparseClonePath string `json:"sparseClonePath,omitempty"` + // Whether to reuse temp file + ReuseTempFile *bool `json:"reuseTempFile,omitempty"` + // Relevant for Create and Plug operations. True for unlimited storage. Even when set to true, totalSize and tempSize MUST be specified in the request if Action is Create. + UnlimitedStorage *bool `json:"unlimitedStorage,omitempty"` + // Indicate if 'AS CLONE' option should be used in the command to plug in a PDB. This property is applicable when the Action property is PLUG but not required. + AsClone *bool `json:"asClone,omitempty"` + // Relevant for create and plug operations. Total size as defined in the Oracle Multitenant Database documentation. See size_clause description in Database SQL Language Reference documentation. + TotalSize string `json:"totalSize,omitempty"` + // Relevant for Create and Clone operations. Total size for temporary tablespace as defined in the Oracle Multitenant Database documentation. See size_clause description in Database SQL Language Reference documentation. + TempSize string `json:"tempSize,omitempty"` + // TDE import for plug operations + TDEImport *bool `json:"tdeImport,omitempty"` + // TDE export for unplug operations + TDEExport *bool `json:"tdeExport,omitempty"` + // TDE password if the tdeImport or tdeExport flag is set to true. Can be used in create, plug or unplug operations + TDEPassword TDEPwd `json:"tdePassword,omitempty"` + // TDE keystore path is required if the tdeImport or tdeExport flag is set to true. Can be used in plug or unplug operations. + TDEKeystorePath string `json:"tdeKeystorePath,omitempty"` + // TDE secret is required if the tdeImport or tdeExport flag is set to true. Can be used in plug or unplug operations. + TDESecret TDESecret `json:"tdeSecret,omitempty"` + // Whether you need the script only or execute the script + GetScript *bool `json:"getScript,omitempty"` + // Action to be taken: Create or Clone or Plug or Unplug + // +kubebuilder:validation:Enum=Create;Clone;Plug;Unplug;Delete + Action string `json:"action"` +} + +// PDBAdminName defines the secret containing Sys Admin User mapped to key 'adminName' for PDB +type PDBAdminName struct { + Secret PDBSecret `json:"secret"` +} + +// PDBAdminPassword defines the secret containing Sys Admin Password mapped to key 'adminPwd' for PDB +type PDBAdminPassword struct { + Secret PDBSecret `json:"secret"` +} + +// TDEPwd defines the secret containing TDE Wallet Password mapped to key 'tdePassword' for PDB +type TDEPwd struct { + Secret PDBSecret `json:"secret"` +} + +// TDESecret defines the secret containing TDE Secret to key 'tdeSecret' for PDB +type TDESecret struct { + Secret PDBSecret `json:"secret"` +} + +// PDBSecret defines the secretName +type PDBSecret struct { + SecretName string `json:"secretName"` + Key string `json:"key"` +} + +// PDBStatus defines the observed state of PDB +type PDBStatus struct { + // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster + // Important: Run "make" to regenerate code after modifying this file + + // PDB Connect String + ConnString string `json:"connString,omitempty"` + // Phase of the CDB Resource + Phase string `json:"phase"` + // CDB Resource Status + Status bool `json:"status"` + // Message + Msg string `json:"msg,omitempty"` + // Last Completed Action + Action string `json:"action,omitempty"` +} + +// +kubebuilder:object:root=true +// +kubebuilder:subresource:status +// +kubebuilder:printcolumn:JSONPath=".status.connString",name="Connect String",type="string",description="The connect string to be used" +// +kubebuilder:printcolumn:JSONPath=".spec.cdbName",name="CDB Name",type="string",description="Name of the CDB" +// +kubebuilder:printcolumn:JSONPath=".spec.pdbName",name="PDB Name",type="string",description="Name of the PDB" +// +kubebuilder:printcolumn:JSONPath=".spec.totalSize",name="PDB Size",type="string",description="Total Size of the PDB" +// +kubebuilder:printcolumn:JSONPath=".status.phase",name="Status",type="string",description="Status of the PDB" +// +kubebuilder:printcolumn:JSONPath=".status.msg",name="Message",type="string",description="Error message, if any" +// PDB is the Schema for the pdbs API +type PDB struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + Spec PDBSpec `json:"spec,omitempty"` + Status PDBStatus `json:"status,omitempty"` +} + +//+kubebuilder:object:root=true + +// PDBList contains a list of PDB +type PDBList struct { + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata,omitempty"` + Items []PDB `json:"items"` +} + +func init() { + SchemeBuilder.Register(&PDB{}, &PDBList{}) +} diff --git a/apis/database/v1alpha1/pdb_webhook.go b/apis/database/v1alpha1/pdb_webhook.go new file mode 100644 index 00000000..5d7e25e4 --- /dev/null +++ b/apis/database/v1alpha1/pdb_webhook.go @@ -0,0 +1,279 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v1alpha1 + +import ( + "reflect" + "strconv" + "strings" + + apierrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/runtime/schema" + "k8s.io/apimachinery/pkg/util/validation/field" + ctrl "sigs.k8s.io/controller-runtime" + logf "sigs.k8s.io/controller-runtime/pkg/log" + "sigs.k8s.io/controller-runtime/pkg/webhook" +) + +// log is for logging in this package. +var pdblog = logf.Log.WithName("pdb-resource") + +func (r *PDB) SetupWebhookWithManager(mgr ctrl.Manager) error { + return ctrl.NewWebhookManagedBy(mgr). + For(r). + Complete() +} + +// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! + +//+kubebuilder:webhook:path=/mutate-database-oracle-com-v1alpha1-pdb,mutating=true,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=pdbs;pdbs/status,verbs=create;update,versions=v1alpha1,name=mpdb.kb.io,admissionReviewVersions={v1,v1beta1} + +var _ webhook.Defaulter = &PDB{} + +// Default implements webhook.Defaulter so a webhook will be registered for the type +func (r *PDB) Default() { + pdblog.Info("Setting default values in PDB spec for : " + r.Name) + + if r.Spec.ReuseTempFile == nil { + r.Spec.ReuseTempFile = new(bool) + *r.Spec.ReuseTempFile = true + pdblog.Info(" - reuseTempFile : " + strconv.FormatBool(*(r.Spec.ReuseTempFile))) + } + if r.Spec.UnlimitedStorage == nil { + r.Spec.UnlimitedStorage = new(bool) + *r.Spec.UnlimitedStorage = true + pdblog.Info(" - unlimitedStorage : " + strconv.FormatBool(*(r.Spec.UnlimitedStorage))) + } + if r.Spec.GetScript == nil { + r.Spec.GetScript = new(bool) + *r.Spec.GetScript = false + pdblog.Info(" - getScript : " + strconv.FormatBool(*(r.Spec.GetScript))) + } + if r.Spec.AsClone == nil { + r.Spec.AsClone = new(bool) + *r.Spec.AsClone = false + pdblog.Info(" - asClone : " + strconv.FormatBool(*(r.Spec.AsClone))) + } + if r.Spec.TDEImport == nil { + r.Spec.TDEImport = new(bool) + *r.Spec.TDEImport = false + pdblog.Info(" - tdeImport : " + strconv.FormatBool(*(r.Spec.TDEImport))) + } + if r.Spec.TDEExport == nil { + r.Spec.TDEExport = new(bool) + *r.Spec.TDEExport = false + pdblog.Info(" - tdeExport : " + strconv.FormatBool(*(r.Spec.TDEExport))) + } +} + +// TODO(user): change verbs to "verbs=create;update;delete" if you want to enable deletion validation. +//+kubebuilder:webhook:path=/validate-database-oracle-com-v1alpha1-pdb,mutating=false,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=pdbs;pdbs/status,verbs=create;update;delete,versions=v1alpha1,name=vpdb.kb.io,admissionReviewVersions={v1,v1beta1} + +var _ webhook.Validator = &PDB{} + +// ValidateCreate implements webhook.Validator so a webhook will be registered for the type +func (r *PDB) ValidateCreate() error { + pdblog.Info("ValidateCreate-Validating PDB spec for : " + r.Name) + + var allErrs field.ErrorList + + r.validateCommon(&allErrs) + + r.validateAction(&allErrs) + + action := strings.ToUpper(r.Spec.Action) + + if len(allErrs) == 0 { + pdblog.Info("PDB Resource : " + r.Name + " successfully validated for Action : " + action) + return nil + } + return apierrors.NewInvalid( + schema.GroupKind{Group: "database.oracle.com", Kind: "PDB"}, + r.Name, allErrs) +} + +// Validate Action for required parameters +func (r *PDB) validateAction(allErrs *field.ErrorList) { + action := strings.ToUpper(r.Spec.Action) + + pdblog.Info("Valdiating PDB Resource Action : " + action) + + switch action { + case "CREATE": + if reflect.ValueOf(r.Spec.AdminName).IsZero() { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("adminName"), "Please specify PDB System Administrator user")) + } + if reflect.ValueOf(r.Spec.AdminPwd).IsZero() { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("adminPwd"), "Please specify PDB System Administrator Password")) + } + if r.Spec.FileNameConversions == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("fileNameConversions"), "Please specify a value for fileNameConversions. Values can be a filename convert pattern or NONE")) + } + if *(r.Spec.TDEImport) { + r.validateTDEInfo(allErrs) + } + case "CLONE": + // Sample Err: The PDB "pdb1-clone" is invalid: spec.srcPdbName: Required value: Please specify source PDB for Cloning + if r.Spec.SrcPDBName == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("srcPdbName"), "Please specify source PDB name for Cloning")) + } + if r.Spec.TotalSize == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("totalSize"), "Please specify size of the tablespace")) + } + if r.Spec.TempSize == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("tempSize"), "Please specify size of the temporary tablespace")) + } + case "PLUG": + if r.Spec.XMLFileName == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("xmlFileName"), "Please specify XML metadata filename")) + } + if r.Spec.FileNameConversions == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("fileNameConversions"), "Please specify a value for fileNameConversions. Values can be a filename convert pattern or NONE")) + } + if r.Spec.SourceFileNameConversions == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("sourceFileNameConversions"), "Please specify a value for sourceFileNameConversions. Values can be a filename convert pattern or NONE")) + } + if r.Spec.CopyAction == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("copyAction"), "Please specify a value for copyAction. Values can be COPY, NOCOPY or MOVE")) + } + if *(r.Spec.TDEImport) { + r.validateTDEInfo(allErrs) + } + case "UNPLUG": + if r.Spec.XMLFileName == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("xmlFileName"), "Please specify XML metadata filename")) + } + if *(r.Spec.TDEExport) { + r.validateTDEInfo(allErrs) + } + } +} + +// ValidateUpdate implements webhook.Validator so a webhook will be registered for the type +func (r *PDB) ValidateUpdate(old runtime.Object) error { + pdblog.Info("ValidateUpdate-Validating PDB spec for : " + r.Name) + + var allErrs field.ErrorList + + // Check Common Validations + r.validateCommon(&allErrs) + + // Validate required parameters for Action specified + r.validateAction(&allErrs) + + // Check TDE requirements + if *(r.Spec.TDEImport) || *(r.Spec.TDEExport) { + r.validateTDEInfo(&allErrs) + } + + // Check for updation errors + oldPDB, ok := old.(*PDB) + if !ok { + return nil + } + + if !strings.EqualFold(oldPDB.Spec.CDBResName, r.Spec.CDBResName) { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("cdbResName"), "cannot be changed")) + } + + if len(allErrs) == 0 { + return nil + } + return apierrors.NewInvalid( + schema.GroupKind{Group: "database.oracle.com", Kind: "PDB"}, + r.Name, allErrs) +} + +// ValidateDelete implements webhook.Validator so a webhook will be registered for the type +func (r *PDB) ValidateDelete() error { + pdblog.Info("ValidateDelete-Validating PDB spec for : " + r.Name) + + // TODO(user): fill in your validation logic upon object deletion. + return nil +} + +// Validate common specs needed for all PDB Actions +func (r *PDB) validateCommon(allErrs *field.ErrorList) { + pdblog.Info("validateCommon", "name", r.Name) + + if r.Spec.Action == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("action"), "Please specify PDB operation to be performed")) + } + if r.Spec.CDBResName == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("cdbResName"), "Please specify the name of the CDB Kubernetes resource to use for PDB operations")) + } + if r.Spec.PDBName == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("pdbName"), "Please specify name of the PDB to be created")) + } +} + +// Validate TDE information for Create, Plug and Unplug Actions +func (r *PDB) validateTDEInfo(allErrs *field.ErrorList) { + pdblog.Info("validateTDEInfo", "name", r.Name) + + if reflect.ValueOf(r.Spec.TDEPassword).IsZero() { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("tdePassword"), "Please specify a value for tdePassword.")) + } + if r.Spec.TDEKeystorePath == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("tdeKeystorePath"), "Please specify a value for tdeKeystorePath.")) + } + if reflect.ValueOf(r.Spec.TDESecret).IsZero() { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("tdeSecret"), "Please specify a value for tdeSecret.")) + } + +} diff --git a/apis/database/v1alpha1/zz_generated.deepcopy.go b/apis/database/v1alpha1/zz_generated.deepcopy.go index 2766d559..1fb813fb 100644 --- a/apis/database/v1alpha1/zz_generated.deepcopy.go +++ b/apis/database/v1alpha1/zz_generated.deepcopy.go @@ -238,6 +238,171 @@ func (in *AutonomousDatabaseStatus) DeepCopy() *AutonomousDatabaseStatus { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CDB) DeepCopyInto(out *CDB) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + in.Spec.DeepCopyInto(&out.Spec) + out.Status = in.Status +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDB. +func (in *CDB) DeepCopy() *CDB { + if in == nil { + return nil + } + out := new(CDB) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *CDB) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CDBAdminPassword) DeepCopyInto(out *CDBAdminPassword) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDBAdminPassword. +func (in *CDBAdminPassword) DeepCopy() *CDBAdminPassword { + if in == nil { + return nil + } + out := new(CDBAdminPassword) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CDBAdminUser) DeepCopyInto(out *CDBAdminUser) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDBAdminUser. +func (in *CDBAdminUser) DeepCopy() *CDBAdminUser { + if in == nil { + return nil + } + out := new(CDBAdminUser) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CDBList) DeepCopyInto(out *CDBList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]CDB, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDBList. +func (in *CDBList) DeepCopy() *CDBList { + if in == nil { + return nil + } + out := new(CDBList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *CDBList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CDBSecret) DeepCopyInto(out *CDBSecret) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDBSecret. +func (in *CDBSecret) DeepCopy() *CDBSecret { + if in == nil { + return nil + } + out := new(CDBSecret) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CDBSpec) DeepCopyInto(out *CDBSpec) { + *out = *in + out.SysAdminPwd = in.SysAdminPwd + out.CDBAdminUser = in.CDBAdminUser + out.CDBAdminPwd = in.CDBAdminPwd + out.ORDSPwd = in.ORDSPwd + out.WebServerUser = in.WebServerUser + out.WebServerPwd = in.WebServerPwd + if in.NodeSelector != nil { + in, out := &in.NodeSelector, &out.NodeSelector + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDBSpec. +func (in *CDBSpec) DeepCopy() *CDBSpec { + if in == nil { + return nil + } + out := new(CDBSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CDBStatus) DeepCopyInto(out *CDBStatus) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDBStatus. +func (in *CDBStatus) DeepCopy() *CDBStatus { + if in == nil { + return nil + } + out := new(CDBStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CDBSysAdminPassword) DeepCopyInto(out *CDBSysAdminPassword) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDBSysAdminPassword. +func (in *CDBSysAdminPassword) DeepCopy() *CDBSysAdminPassword { + if in == nil { + return nil + } + out := new(CDBSysAdminPassword) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *CatalogSpec) DeepCopyInto(out *CatalogSpec) { *out = *in @@ -417,6 +582,192 @@ func (in *OCIConfigSpec) DeepCopy() *OCIConfigSpec { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *ORDSPassword) DeepCopyInto(out *ORDSPassword) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ORDSPassword. +func (in *ORDSPassword) DeepCopy() *ORDSPassword { + if in == nil { + return nil + } + out := new(ORDSPassword) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PDB) DeepCopyInto(out *PDB) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + in.Spec.DeepCopyInto(&out.Spec) + out.Status = in.Status +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDB. +func (in *PDB) DeepCopy() *PDB { + if in == nil { + return nil + } + out := new(PDB) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *PDB) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PDBAdminName) DeepCopyInto(out *PDBAdminName) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBAdminName. +func (in *PDBAdminName) DeepCopy() *PDBAdminName { + if in == nil { + return nil + } + out := new(PDBAdminName) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PDBAdminPassword) DeepCopyInto(out *PDBAdminPassword) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBAdminPassword. +func (in *PDBAdminPassword) DeepCopy() *PDBAdminPassword { + if in == nil { + return nil + } + out := new(PDBAdminPassword) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PDBList) DeepCopyInto(out *PDBList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]PDB, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBList. +func (in *PDBList) DeepCopy() *PDBList { + if in == nil { + return nil + } + out := new(PDBList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *PDBList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PDBSecret) DeepCopyInto(out *PDBSecret) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBSecret. +func (in *PDBSecret) DeepCopy() *PDBSecret { + if in == nil { + return nil + } + out := new(PDBSecret) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PDBSpec) DeepCopyInto(out *PDBSpec) { + *out = *in + out.AdminName = in.AdminName + out.AdminPwd = in.AdminPwd + if in.ReuseTempFile != nil { + in, out := &in.ReuseTempFile, &out.ReuseTempFile + *out = new(bool) + **out = **in + } + if in.UnlimitedStorage != nil { + in, out := &in.UnlimitedStorage, &out.UnlimitedStorage + *out = new(bool) + **out = **in + } + if in.AsClone != nil { + in, out := &in.AsClone, &out.AsClone + *out = new(bool) + **out = **in + } + if in.TDEImport != nil { + in, out := &in.TDEImport, &out.TDEImport + *out = new(bool) + **out = **in + } + if in.TDEExport != nil { + in, out := &in.TDEExport, &out.TDEExport + *out = new(bool) + **out = **in + } + out.TDEPassword = in.TDEPassword + out.TDESecret = in.TDESecret + if in.GetScript != nil { + in, out := &in.GetScript, &out.GetScript + *out = new(bool) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBSpec. +func (in *PDBSpec) DeepCopy() *PDBSpec { + if in == nil { + return nil + } + out := new(PDBSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PDBStatus) DeepCopyInto(out *PDBStatus) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBStatus. +func (in *PDBStatus) DeepCopy() *PDBStatus { + if in == nil { + return nil + } + out := new(PDBStatus) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *PasswordSpec) DeepCopyInto(out *PasswordSpec) { *out = *in @@ -826,6 +1177,38 @@ func (in *SingleInstanceDatabaseStatus) DeepCopy() *SingleInstanceDatabaseStatus return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *TDEPwd) DeepCopyInto(out *TDEPwd) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new TDEPwd. +func (in *TDEPwd) DeepCopy() *TDEPwd { + if in == nil { + return nil + } + out := new(TDEPwd) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *TDESecret) DeepCopyInto(out *TDESecret) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new TDESecret. +func (in *TDESecret) DeepCopy() *TDESecret { + if in == nil { + return nil + } + out := new(TDESecret) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *WalletSpec) DeepCopyInto(out *WalletSpec) { *out = *in @@ -846,3 +1229,35 @@ func (in *WalletSpec) DeepCopy() *WalletSpec { in.DeepCopyInto(out) return out } + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *WebServerPassword) DeepCopyInto(out *WebServerPassword) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new WebServerPassword. +func (in *WebServerPassword) DeepCopy() *WebServerPassword { + if in == nil { + return nil + } + out := new(WebServerPassword) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *WebServerUser) DeepCopyInto(out *WebServerUser) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new WebServerUser. +func (in *WebServerUser) DeepCopy() *WebServerUser { + if in == nil { + return nil + } + out := new(WebServerUser) + in.DeepCopyInto(out) + return out +} diff --git a/config/crd/bases/database.oracle.com_cdbs.yaml b/config/crd/bases/database.oracle.com_cdbs.yaml new file mode 100644 index 00000000..bc87e42c --- /dev/null +++ b/config/crd/bases/database.oracle.com_cdbs.yaml @@ -0,0 +1,235 @@ + +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.6.1 + creationTimestamp: null + name: cdbs.database.oracle.com +spec: + group: database.oracle.com + names: + kind: CDB + listKind: CDBList + plural: cdbs + singular: cdb + scope: Namespaced + versions: + - additionalPrinterColumns: + - description: Name of the CDB + jsonPath: .spec.cdbName + name: CDB Name + type: string + - description: ' Name of the DB Server' + jsonPath: .spec.dbServer + name: DB Server + type: string + - description: DB server port + jsonPath: .spec.dbPort + name: DB Port + type: integer + - description: SCAN Name + jsonPath: .spec.scanName + name: SCAN NAme + type: string + - description: Status of the CDB Resource + jsonPath: .status.phase + name: Status + type: string + - description: Error message, if any + jsonPath: .status.msg + name: Message + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + description: CDB is the Schema for the cdbs API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: CDBSpec defines the desired state of CDB + properties: + cdbAdminPwd: + description: Password for the CDB Administrator to manage PDB lifecycle + properties: + secret: + description: CDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + cdbAdminUser: + description: User in the root container with sysdba priviledges to + manage PDB lifecycle + properties: + secret: + description: CDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + cdbName: + description: Name of the CDB + type: string + dbPort: + description: DB server port + type: integer + dbServer: + description: Name of the DB server + type: string + nodeSelector: + additionalProperties: + type: string + description: Node Selector for running the Pod + type: object + ordsImage: + description: ORDS Image Name + type: string + ordsImagePullPolicy: + description: ORDS Image Pull Policy + enum: + - Always + - Never + type: string + ordsImagePullSecret: + description: The name of the image pull secret in case of a private + docker repository. + type: string + ordsPort: + description: ORDS server port. For now, keep it as 8888. TO BE USED + IN FUTURE RELEASE. + type: integer + ordsPwd: + description: Password for user ORDS_PUBLIC_USER + properties: + secret: + description: CDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + replicas: + description: Number of ORDS Containers to create + type: integer + scanName: + description: SCAN Name + type: string + serviceName: + description: Name of the CDB Service + type: string + sysAdminPwd: + description: Password for the CDB System Administrator + properties: + secret: + description: CDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + webServerPwd: + description: Password for the Web Server User + properties: + secret: + description: CDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + webServerUser: + description: Web Server User with SQL Administrator role to allow + us to authenticate to the PDB Lifecycle Management REST endpoints + properties: + secret: + description: CDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + type: object + status: + description: CDBStatus defines the observed state of CDB + properties: + msg: + description: Message + type: string + phase: + description: Phase of the CDB Resource + type: string + status: + description: CDB Resource Status + type: boolean + required: + - phase + - status + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] diff --git a/config/crd/bases/database.oracle.com_pdbs.yaml b/config/crd/bases/database.oracle.com_pdbs.yaml new file mode 100644 index 00000000..0806fe85 --- /dev/null +++ b/config/crd/bases/database.oracle.com_pdbs.yaml @@ -0,0 +1,260 @@ + +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.6.1 + creationTimestamp: null + name: pdbs.database.oracle.com +spec: + group: database.oracle.com + names: + kind: PDB + listKind: PDBList + plural: pdbs + singular: pdb + scope: Namespaced + versions: + - additionalPrinterColumns: + - description: The connect string to be used + jsonPath: .status.connString + name: Connect String + type: string + - description: Name of the CDB + jsonPath: .spec.cdbName + name: CDB Name + type: string + - description: Name of the PDB + jsonPath: .spec.pdbName + name: PDB Name + type: string + - description: Total Size of the PDB + jsonPath: .spec.totalSize + name: PDB Size + type: string + - description: Status of the PDB + jsonPath: .status.phase + name: Status + type: string + - description: Error message, if any + jsonPath: .status.msg + name: Message + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + description: PDB is the Schema for the pdbs API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: PDBSpec defines the desired state of PDB + properties: + action: + description: 'Action to be taken: Create or Clone or Plug or Unplug' + enum: + - Create + - Clone + - Plug + - Unplug + - Delete + type: string + adminName: + description: The administrator username for the new PDB. This property + is required when the Action property is Create. + properties: + secret: + description: PDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + adminPwd: + description: The administrator password for the new PDB. This property + is required when the Action property is Create. + properties: + secret: + description: PDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + asClone: + description: Indicate if 'AS CLONE' option should be used in the command + to plug in a PDB. This property is applicable when the Action property + is PLUG but not required. + type: boolean + cdbName: + description: Name of the CDB + type: string + cdbResName: + description: Name of the CDB Custom Resource that runs the ORDS container + type: string + copyAction: + description: To copy files or not while cloning a PDB + enum: + - COPY + - NOCOPY + - MOVE + type: string + dropAction: + description: Specify if datafiles should be removed or not. The value + can be INCLUDING or KEEP (default). + enum: + - INCLUDING + - KEEP + type: string + fileNameConversions: + description: Relevant for Create and Plug operations. As defined in + the Oracle Multitenant Database documentation. Values can be a + filename convert pattern or NONE. + type: string + getScript: + description: Whether you need the script only or execute the script + type: boolean + pdbName: + description: The name of the new PDB. Relevant for both Create and + Plug Actions. + type: string + reuseTempFile: + description: Whether to reuse temp file + type: boolean + sourceFileNameConversions: + description: This property is required when the Action property is + Plug. As defined in the Oracle Multitenant Database documentation. + Values can be a source filename convert pattern or NONE. + type: string + sparseClonePath: + description: A Path specified for sparse clone snapshot copy. (Optional) + type: string + srcPdbName: + description: Name of the Source PDB from which to clone + type: string + tdeExport: + description: TDE export for unplug operations + type: boolean + tdeImport: + description: TDE import for plug operations + type: boolean + tdeKeystorePath: + description: TDE keystore path is required if the tdeImport or tdeExport + flag is set to true. Can be used in plug or unplug operations. + type: string + tdePassword: + description: TDE password if the tdeImport or tdeExport flag is set + to true. Can be used in create, plug or unplug operations + properties: + secret: + description: PDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + tdeSecret: + description: TDE secret is required if the tdeImport or tdeExport + flag is set to true. Can be used in plug or unplug operations. + properties: + secret: + description: PDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + tempSize: + description: Relevant for Create and Clone operations. Total size + for temporary tablespace as defined in the Oracle Multitenant Database + documentation. See size_clause description in Database SQL Language + Reference documentation. + type: string + totalSize: + description: Relevant for create and plug operations. Total size as + defined in the Oracle Multitenant Database documentation. See size_clause + description in Database SQL Language Reference documentation. + type: string + unlimitedStorage: + description: Relevant for Create and Plug operations. True for unlimited + storage. Even when set to true, totalSize and tempSize MUST be specified + in the request if Action is Create. + type: boolean + xmlFileName: + description: XML metadata filename to be used for Plug or Unplug operations + type: string + required: + - action + type: object + status: + description: PDBStatus defines the observed state of PDB + properties: + action: + description: Last Completed Action + type: string + connString: + description: PDB Connect String + type: string + msg: + description: Message + type: string + phase: + description: Phase of the CDB Resource + type: string + status: + description: CDB Resource Status + type: boolean + required: + - phase + - status + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] diff --git a/config/crd/kustomization.yaml b/config/crd/kustomization.yaml index 6b3d488e..7570a5c7 100644 --- a/config/crd/kustomization.yaml +++ b/config/crd/kustomization.yaml @@ -9,6 +9,8 @@ resources: - bases/database.oracle.com_autonomousdatabases.yaml - bases/database.oracle.com_singleinstancedatabases.yaml - bases/database.oracle.com_shardingdatabases.yaml +- bases/database.oracle.com_pdbs.yaml +- bases/database.oracle.com_cdbs.yaml # +kubebuilder:scaffold:crdkustomizeresource patchesStrategicMerge: @@ -18,6 +20,8 @@ patchesStrategicMerge: #- patches/webhook_in_autonomousdatabases.yaml #- patches/webhook_in_singleinstancedatabases.yaml #- patches/webhook_in_shardingdatabases.yaml +#- patches/webhook_in_pdbs.yaml +#- patches/webhook_in_cdbs.yaml # +kubebuilder:scaffold:crdkustomizewebhookpatch # [CERTMANAGER] To enable webhook, uncomment all the sections with [CERTMANAGER] prefix. @@ -26,6 +30,8 @@ patchesStrategicMerge: #- patches/cainjection_in_autonomousdatabases.yaml - patches/cainjection_in_singleinstancedatabases.yaml #- patches/cainjection_in_shardingdatabases.yaml +- patches/cainjection_in_pdbs.yaml +- patches/cainjection_in_cdbs.yaml # +kubebuilder:scaffold:crdkustomizecainjectionpatch # the following config is for teaching kustomize how to do kustomization for CRDs. diff --git a/config/crd/patches/cainjection_in_cdbs.yaml b/config/crd/patches/cainjection_in_cdbs.yaml new file mode 100644 index 00000000..5ab5efe9 --- /dev/null +++ b/config/crd/patches/cainjection_in_cdbs.yaml @@ -0,0 +1,8 @@ +# The following patch adds a directive for certmanager to inject CA into the CRD +# CRD conversion requires k8s 1.13 or later. +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + annotations: + cert-manager.io/inject-ca-from: $(CERTIFICATE_NAMESPACE)/$(CERTIFICATE_NAME) + name: cdbs.database.oracle.com diff --git a/config/crd/patches/cainjection_in_pdbs.yaml b/config/crd/patches/cainjection_in_pdbs.yaml new file mode 100644 index 00000000..3b1bdcbc --- /dev/null +++ b/config/crd/patches/cainjection_in_pdbs.yaml @@ -0,0 +1,8 @@ +# The following patch adds a directive for certmanager to inject CA into the CRD +# CRD conversion requires k8s 1.13 or later. +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + annotations: + cert-manager.io/inject-ca-from: $(CERTIFICATE_NAMESPACE)/$(CERTIFICATE_NAME) + name: pdbs.database.oracle.com diff --git a/config/crd/patches/webhook_in_cdbs.yaml b/config/crd/patches/webhook_in_cdbs.yaml new file mode 100644 index 00000000..d3700966 --- /dev/null +++ b/config/crd/patches/webhook_in_cdbs.yaml @@ -0,0 +1,17 @@ +# The following patch enables conversion webhook for CRD +# CRD conversion requires k8s 1.13 or later. +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: cdbs.database.oracle.com +spec: + conversion: + strategy: Webhook + webhookClientConfig: + # this is "\n" used as a placeholder, otherwise it will be rejected by the apiserver for being blank, + # but we're going to set it later using the cert-manager (or potentially a patch if not using cert-manager) + caBundle: Cg== + service: + namespace: system + name: webhook-service + path: /convert diff --git a/config/crd/patches/webhook_in_pdbs.yaml b/config/crd/patches/webhook_in_pdbs.yaml new file mode 100644 index 00000000..f0012581 --- /dev/null +++ b/config/crd/patches/webhook_in_pdbs.yaml @@ -0,0 +1,17 @@ +# The following patch enables conversion webhook for CRD +# CRD conversion requires k8s 1.13 or later. +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: pdbs.database.oracle.com +spec: + conversion: + strategy: Webhook + webhookClientConfig: + # this is "\n" used as a placeholder, otherwise it will be rejected by the apiserver for being blank, + # but we're going to set it later using the cert-manager (or potentially a patch if not using cert-manager) + caBundle: Cg== + service: + namespace: system + name: webhook-service + path: /convert diff --git a/config/rbac/cdb_editor_role.yaml b/config/rbac/cdb_editor_role.yaml new file mode 100644 index 00000000..244ddff2 --- /dev/null +++ b/config/rbac/cdb_editor_role.yaml @@ -0,0 +1,24 @@ +# permissions for end users to edit cdbs. +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: cdb-editor-role +rules: +- apiGroups: + - database.oracle.com + resources: + - cdbs + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - database.oracle.com + resources: + - cdbs/status + verbs: + - get diff --git a/config/rbac/cdb_viewer_role.yaml b/config/rbac/cdb_viewer_role.yaml new file mode 100644 index 00000000..78a84283 --- /dev/null +++ b/config/rbac/cdb_viewer_role.yaml @@ -0,0 +1,20 @@ +# permissions for end users to view cdbs. +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: cdb-viewer-role +rules: +- apiGroups: + - database.oracle.com + resources: + - cdbs + verbs: + - get + - list + - watch +- apiGroups: + - database.oracle.com + resources: + - cdbs/status + verbs: + - get diff --git a/config/rbac/pdb_editor_role.yaml b/config/rbac/pdb_editor_role.yaml new file mode 100644 index 00000000..7d668e4a --- /dev/null +++ b/config/rbac/pdb_editor_role.yaml @@ -0,0 +1,24 @@ +# permissions for end users to edit pdbs. +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: pdb-editor-role +rules: +- apiGroups: + - database.oracle.com + resources: + - pdbs + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - database.oracle.com + resources: + - pdbs/status + verbs: + - get diff --git a/config/rbac/pdb_viewer_role.yaml b/config/rbac/pdb_viewer_role.yaml new file mode 100644 index 00000000..5fcf68c9 --- /dev/null +++ b/config/rbac/pdb_viewer_role.yaml @@ -0,0 +1,20 @@ +# permissions for end users to view pdbs. +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: pdb-viewer-role +rules: +- apiGroups: + - database.oracle.com + resources: + - pdbs + verbs: + - get + - list + - watch +- apiGroups: + - database.oracle.com + resources: + - pdbs/status + verbs: + - get diff --git a/config/rbac/role.yaml b/config/rbac/role.yaml index 6d763212..2e49c9e6 100644 --- a/config/rbac/role.yaml +++ b/config/rbac/role.yaml @@ -6,6 +6,23 @@ metadata: creationTimestamp: null name: manager-role rules: +- apiGroups: + - "" + resources: + - configmaps + - events + - pods + - pods/exec + - pods/log + - services + verbs: + - create + - delete + - get + - list + - patch + - update + - watch - apiGroups: - "" resources: @@ -91,6 +108,22 @@ rules: - patch - update - watch +- apiGroups: + - "" + resources: + - configmaps + - namespaces + - pods + - secrets + - services + verbs: + - create + - delete + - get + - list + - patch + - update + - watch - apiGroups: - "" resources: @@ -116,6 +149,62 @@ rules: verbs: - patch - update +- apiGroups: + - database.oracle.com + resources: + - cdbs + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - database.oracle.com + resources: + - cdbs/finalizers + verbs: + - update +- apiGroups: + - database.oracle.com + resources: + - cdbs/status + verbs: + - get + - patch + - update +- apiGroups: + - database.oracle.com + resources: + - pdbs + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - database.oracle.com + resources: + - pdbs/finalizers + verbs: + - create + - delete + - get + - patch + - update +- apiGroups: + - database.oracle.com + resources: + - pdbs/status + verbs: + - get + - patch + - update - apiGroups: - database.oracle.com resources: diff --git a/config/samples/kustomization.yaml b/config/samples/kustomization.yaml index 4b419923..e6adda85 100644 --- a/config/samples/kustomization.yaml +++ b/config/samples/kustomization.yaml @@ -9,4 +9,6 @@ resources: - autonomousdatabase.yaml - singleinstancedatabase.yaml - shardingdatabase.yaml + - database_v1alpha1_pdb.yaml + - database_v1alpha1_cdb.yaml # +kubebuilder:scaffold:manifestskustomizesamples diff --git a/config/samples/onpremdb/cdb.yaml b/config/samples/onpremdb/cdb.yaml new file mode 100644 index 00000000..a7f71a7d --- /dev/null +++ b/config/samples/onpremdb/cdb.yaml @@ -0,0 +1,40 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: CDB +metadata: + name: cdb-dev + namespace: oracle-database-operator-system +spec: + cdbName: "devcdb" + scanName: "devdb" + dbServer: "172.17.0.4" + dbPort: 1521 + replicas: 2 + serviceName: "devdb.example.com" + sysAdminPwd: + secret: + secretName: "cdb1-secret" + key: "sysadmin_pwd" + ordsPwd: + secret: + secretName: "cdb1-secret" + key: "ords_pwd" + cdbAdminUser: + secret: + secretName: "cdb1-secret" + key: "cdbadmin_user" + cdbAdminPwd: + secret: + secretName: "cdb1-secret" + key: "cdbadmin_pwd" + webServerUser: + secret: + secretName: "cdb1-secret" + key: "webserver_user" + webServerPwd: + secret: + secretName: "cdb1-secret" + key: "webserver_pwd" \ No newline at end of file diff --git a/config/samples/onpremdb/cdb_secret.yaml b/config/samples/onpremdb/cdb_secret.yaml new file mode 100644 index 00000000..856a653b --- /dev/null +++ b/config/samples/onpremdb/cdb_secret.yaml @@ -0,0 +1,17 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: v1 +kind: Secret +metadata: + name: cdb1-secret + namespace: oracle-database-operator-system +type: Opaque +data: + ords_pwd: "d2VsY29tZTE=" + sysadmin_pwd: "V0VsY29tZV8xMiMj" + cdbadmin_user: "QyMjREJBUElfQ0RCX0FETUlO" + cdbadmin_pwd: "V0VsY29tZV8xMiMj" + webserver_user: "c3FsX2FkbWlu" + webserver_pwd: "d2VsY29tZTE=" \ No newline at end of file diff --git a/config/samples/onpremdb/pdb.yaml b/config/samples/onpremdb/pdb.yaml new file mode 100644 index 00000000..71641e9c --- /dev/null +++ b/config/samples/onpremdb/pdb.yaml @@ -0,0 +1,27 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbName: "devcdb" + pdbName: "pdbdev" + adminName: + secret: + secretName: "pdb1-secret" + key: "sysadmin_user" + adminPwd: + secret: + secretName: "pdb1-secret" + key: "sysadmin_pwd" + fileNameConversions: "NONE" + totalSize: "1G" + tempSize: "100M" + action: "Create" diff --git a/config/samples/onpremdb/pdb_clone.yaml b/config/samples/onpremdb/pdb_clone.yaml new file mode 100644 index 00000000..c88db391 --- /dev/null +++ b/config/samples/onpremdb/pdb_clone.yaml @@ -0,0 +1,27 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1-clone + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbName: "devcdb" + pdbName: "pdbdevclone" + adminName: + secret: + secretName: "pdb1-secret" + key: "sysadmin_user" + adminPwd: + secret: + secretName: "pdb1-secret" + key: "sysadmin_pwd" + fileNameConversions: "NONE" + totalSize: "1G" + tempSize: "100M" + action: "Clone" diff --git a/config/samples/onpremdb/pdb_delete.yaml b/config/samples/onpremdb/pdb_delete.yaml new file mode 100644 index 00000000..3532c023 --- /dev/null +++ b/config/samples/onpremdb/pdb_delete.yaml @@ -0,0 +1,16 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + pdbName: "pdbdev" + action: "Delete" + dropAction: "INCLUDING" \ No newline at end of file diff --git a/config/samples/onpremdb/pdb_plug.yaml b/config/samples/onpremdb/pdb_plug.yaml new file mode 100644 index 00000000..4a9fef98 --- /dev/null +++ b/config/samples/onpremdb/pdb_plug.yaml @@ -0,0 +1,19 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + pdbName: "pdbdev" + xmlFileName: "/opt/oracle/oradata/pdbdev.xml" + sourceFileNameConversions: "NONE" + fileNameConversions: "NONE" + copyAction: "NOCOPY" + action: "Plug" \ No newline at end of file diff --git a/config/samples/onpremdb/pdb_secret.yaml b/config/samples/onpremdb/pdb_secret.yaml new file mode 100644 index 00000000..65a5fabe --- /dev/null +++ b/config/samples/onpremdb/pdb_secret.yaml @@ -0,0 +1,13 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: v1 +kind: Secret +metadata: + name: pdb1-secret + namespace: oracle-database-operator-system +type: Opaque +data: + sysadmin_user: "cGRiYWRtaW4=" + sysadmin_pwd: "V0VsY29tZV8xMiMj" \ No newline at end of file diff --git a/config/samples/onpremdb/pdb_unplug.yaml b/config/samples/onpremdb/pdb_unplug.yaml new file mode 100644 index 00000000..89b4d086 --- /dev/null +++ b/config/samples/onpremdb/pdb_unplug.yaml @@ -0,0 +1,27 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + pdbName: "pdbdev" + xmlFileName: "/opt/oracle/oradata/demotest_pdb.xml" + action: "Unplug" + tdeExport: true + tdeSecret: + secret: + secretName: "pdb1-secret" + key: "tde_secret" + tdeKeystorePath: "/opt/oracle/test" + tdePassword: + secret: + secretName: "pdb1-secret" + key: "tde_pwd" + getScript: true \ No newline at end of file diff --git a/config/webhook/manifests.yaml b/config/webhook/manifests.yaml index 5cb37b30..7b3c4d6f 100644 --- a/config/webhook/manifests.yaml +++ b/config/webhook/manifests.yaml @@ -6,6 +6,49 @@ metadata: creationTimestamp: null name: mutating-webhook-configuration webhooks: +- admissionReviewVersions: + - v1 + - v1beta1 + clientConfig: + service: + name: webhook-service + namespace: system + path: /mutate-database-oracle-com-v1alpha1-cdb + failurePolicy: Fail + name: mcdb.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + resources: + - cdbs + sideEffects: None +- admissionReviewVersions: + - v1 + - v1beta1 + clientConfig: + service: + name: webhook-service + namespace: system + path: /mutate-database-oracle-com-v1alpha1-pdb + failurePolicy: Fail + name: mpdb.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + resources: + - pdbs + - pdbs/status + sideEffects: None - admissionReviewVersions: - v1 - v1beta1 @@ -35,6 +78,50 @@ metadata: creationTimestamp: null name: validating-webhook-configuration webhooks: +- admissionReviewVersions: + - v1 + - v1beta1 + clientConfig: + service: + name: webhook-service + namespace: system + path: /validate-database-oracle-com-v1alpha1-cdb + failurePolicy: Fail + name: vcdb.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + resources: + - cdbs + sideEffects: None +- admissionReviewVersions: + - v1 + - v1beta1 + clientConfig: + service: + name: webhook-service + namespace: system + path: /validate-database-oracle-com-v1alpha1-pdb + failurePolicy: Fail + name: vpdb.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + - DELETE + resources: + - pdbs + - pdbs/status + sideEffects: None - admissionReviewVersions: - v1 - v1beta1 diff --git a/controllers/database/cdb_controller.go b/controllers/database/cdb_controller.go new file mode 100644 index 00000000..02176927 --- /dev/null +++ b/controllers/database/cdb_controller.go @@ -0,0 +1,671 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package controllers + +import ( + "context" + "errors" + //"fmt" + "strconv" + "strings" + "time" + + "github.com/go-logr/logr" + corev1 "k8s.io/api/core/v1" + apierrors "k8s.io/apimachinery/pkg/api/errors" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/types" + "k8s.io/client-go/kubernetes" + "k8s.io/client-go/rest" + "k8s.io/client-go/tools/record" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/controller" + "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" + "sigs.k8s.io/controller-runtime/pkg/event" + "sigs.k8s.io/controller-runtime/pkg/predicate" + + dbapi "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + dbcommons "github.com/oracle/oracle-database-operator/commons/database" +) + +// CDBReconciler reconciles a CDB object +type CDBReconciler struct { + client.Client + Scheme *runtime.Scheme + Config *rest.Config + Log logr.Logger + Interval time.Duration + Recorder record.EventRecorder +} + +var ( + cdbPhaseInit = "Initializing" + cdbPhasePod = "CreatingPod" + cdbPhaseValPod = "ValidatingPod" + cdbPhaseService = "CreatingService" + cdbPhaseSecrets = "DeletingSecrets" + cdbPhaseReady = "Ready" + cdbPhaseDelete = "Deleting" +) + +const CDBFinalizer = "database.oracle.com/CDBfinalizer" + +//+kubebuilder:rbac:groups=database.oracle.com,resources=cdbs,verbs=get;list;watch;create;update;patch;delete +//+kubebuilder:rbac:groups=database.oracle.com,resources=cdbs/status,verbs=get;update;patch +//+kubebuilder:rbac:groups=database.oracle.com,resources=cdbs/finalizers,verbs=update +//+kubebuilder:rbac:groups="",resources=pods;pods/log;pods/exec;services;configmaps;events,verbs=create;delete;get;list;patch;update;watch +//+kubebuilder:rbac:groups=core,resources=pods;secrets;services;configmaps;namespaces,verbs=get;list;watch;create;update;patch;delete + +// Reconcile is part of the main kubernetes reconciliation loop which aims to +// move the current state of the cluster closer to the desired state. +// TODO(user): Modify the Reconcile function to compare the state specified by +// the CDB object against the actual cluster state, and then +// perform operations to make the cluster state reflect the state specified by +// the user. +// +// For more details, check Reconcile and its Result here: +// - https://pkg.go.dev/sigs.k8s.io/controller-runtime@v0.9.2/pkg/reconcile +func (r *CDBReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + + log := r.Log.WithValues("onpremdboperator", req.NamespacedName) + log.Info("Reconcile requested") + + reconcilePeriod := r.Interval * time.Second + requeueY := ctrl.Result{Requeue: true, RequeueAfter: reconcilePeriod} + requeueN := ctrl.Result{} + + var err error + cdb := &dbapi.CDB{} + + // Execute for every reconcile + defer func() { + //log.Info("DEFER", "Name", cdb.Name, "Phase", cdb.Status.Phase, "Status", strconv.FormatBool(cdb.Status.Status)) + if !cdb.Status.Status { + if err := r.Status().Update(ctx, cdb); err != nil { + log.Error(err, "Failed to update status for :"+cdb.Name, "err", err.Error()) + } + } + }() + + err = r.Client.Get(context.TODO(), req.NamespacedName, cdb) + if err != nil { + if apierrors.IsNotFound(err) { + log.Info("CDB Resource Not found", "Name", cdb.Name) + // Request object not found, could have been deleted after reconcile req. + // Owned objects are automatically garbage collected. For additional cleanup logic use finalizers. + // Return and don't requeue + cdb.Status.Status = true + return requeueN, nil + } + // Error reading the object - requeue the req. + return requeueY, err + } + + //log.Info("Res Status:", "Name", cdb.Name, "Phase", cdb.Status.Phase, "Status", strconv.FormatBool(cdb.Status.Status)) + + // Finalizer section + err = r.manageCDBDeletion(ctx, req, cdb) + if err != nil { + log.Info("Reconcile queued") + return requeueY, nil + } + + if !cdb.Status.Status { + phase := cdb.Status.Phase + log.Info("Current Phase:"+phase, "Name", cdb.Name) + + switch phase { + case cdbPhaseInit: + cdb.Status.Phase = cdbPhasePod + case cdbPhasePod: + // Create ORDS POD + err = r.createORDSPod(ctx, req, cdb) + if err != nil { + log.Info("Reconcile queued") + return requeueY, nil + } + cdb.Status.Phase = cdbPhaseValPod + case cdbPhaseValPod: + // Validate ORDS POD + err = r.validateORDSPod(ctx, req, cdb) + if err != nil { + log.Info("Reconcile queued") + return requeueY, nil + } + cdb.Status.Phase = cdbPhaseService + case cdbPhaseService: + // Create ORDS Service + err = r.createORDSSVC(ctx, req, cdb) + if err != nil { + log.Info("Reconcile queued") + return requeueY, nil + } + cdb.Status.Phase = cdbPhaseSecrets + case cdbPhaseSecrets: + // Delete CDB Secrets + //r.deleteSecrets(ctx, req, cdb) + cdb.Status.Phase = cdbPhaseReady + cdb.Status.Msg = "Success" + case cdbPhaseReady: + cdb.Status.Status = true + r.Status().Update(ctx, cdb) + return requeueN, nil + default: + cdb.Status.Phase = cdbPhaseInit + log.Info("DEFAULT:", "Name", cdb.Name, "Phase", phase, "Status", strconv.FormatBool(cdb.Status.Status)) + } + + if err := r.Status().Update(ctx, cdb); err != nil { + log.Error(err, "Failed to update status for :"+cdb.Name, "err", err.Error()) + } + return requeueY, nil + } + + log.Info("Reconcile completed") + return requeueN, nil +} + +/************************************************* + * Create a Pod based on the ORDS container + /************************************************/ +func (r *CDBReconciler) createORDSPod(ctx context.Context, req ctrl.Request, cdb *dbapi.CDB) error { + + log := r.Log.WithValues("createORDSPod", req.NamespacedName) + + for i := 0; i < cdb.Spec.Replicas; i++ { + pod := r.createPodSpec(cdb) + + log.Info("Creating a new Pod for :"+cdb.Name, "Pod.Namespace", pod.Namespace, "Pod.Name", pod.Name) + err := r.Create(ctx, pod) + if err != nil { + log.Error(err, "Failed to create new Pod for :"+cdb.Name, "Pod.Namespace", pod.Namespace, "Pod.Name", pod.Name) + return err + } + } + + log.Info("Created ORDS Pod(s) successfully") + r.Recorder.Eventf(cdb, corev1.EventTypeNormal, "CreatedORDSPod", "Created %s ORDS Pod(s) for %s", strconv.Itoa(cdb.Spec.Replicas), cdb.Name) + return nil +} + +/************************************************* + * Validate ORDS Pod. Check if there are any errors + /************************************************/ +func (r *CDBReconciler) validateORDSPod(ctx context.Context, req ctrl.Request, cdb *dbapi.CDB) error { + + log := r.Log.WithValues("validateORDSPod", req.NamespacedName) + + log.Info("Validating Pod creation for :" + cdb.Name) + + podName := cdb.Name + "-ords" + podList := &corev1.PodList{} + listOpts := []client.ListOption{client.InNamespace(req.Namespace), client.MatchingLabels{"name": podName}} + + // List retrieves list of objects for a given namespace and list options. + err := r.List(ctx, podList, listOpts...) + if err != nil { + log.Info("Failed to list pods of: "+podName, "Namespace", req.Namespace) + return err + } + + if len(podList.Items) == 0 { + log.Info("No pods found for: "+podName, "Namespace", req.Namespace) + cdb.Status.Msg = "Waiting for ORDS Pod(s) to start" + return errors.New("Waiting for ORDS pods to start") + } + + getORDSStatus := " curl -sSkv -k -X GET https://localhost:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/stable/metadata-catalog/ || curl -sSkv -X GET http://localhost:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/stable/metadata-catalog/ " + readyPods := 0 + for _, pod := range podList.Items { + if pod.Status.Phase == corev1.PodRunning { + // Get ORDS Status + out, err := dbcommons.ExecCommand(r, r.Config, pod.Name, pod.Namespace, "", ctx, req, false, "bash", "-c", getORDSStatus) + if strings.Contains(out, "HTTP/1.1 200 OK") || strings.Contains(strings.ToUpper(err.Error()), "HTTP/1.1 200 OK") { + readyPods++ + } + } + } + + if readyPods != cdb.Spec.Replicas { + log.Info("Replicas: "+strconv.Itoa(cdb.Spec.Replicas), "Ready Pods: ", readyPods) + cdb.Status.Msg = "Waiting for ORDS Pod(s) to be ready" + return errors.New("Waiting for ORDS pods to be ready") + } + + cdb.Status.Msg = "" + return nil +} + +/************************ + * Create Pod spec + /************************/ +func (r *CDBReconciler) createPodSpec(cdb *dbapi.CDB) *corev1.Pod { + + pod := &corev1.Pod{ + TypeMeta: metav1.TypeMeta{ + Kind: "Pod", + }, + ObjectMeta: metav1.ObjectMeta{ + Name: cdb.Name + "-ords-" + dbcommons.GenerateRandomString(5), + Namespace: cdb.Namespace, + Labels: map[string]string{ + "name": cdb.Name + "-ords", + }, + }, + Spec: corev1.PodSpec{ + Volumes: []corev1.Volume{{ + Name: "secrets", + VolumeSource: corev1.VolumeSource{ + Projected: &corev1.ProjectedVolumeSource{ + DefaultMode: func() *int32 { i := int32(0666); return &i }(), + Sources: []corev1.VolumeProjection{ + { + Secret: &corev1.SecretProjection{ + LocalObjectReference: corev1.LocalObjectReference{ + Name: cdb.Spec.SysAdminPwd.Secret.SecretName, + }, + }, + }, + { + Secret: &corev1.SecretProjection{ + LocalObjectReference: corev1.LocalObjectReference{ + Name: cdb.Spec.CDBAdminUser.Secret.SecretName, + }, + }, + }, + { + Secret: &corev1.SecretProjection{ + LocalObjectReference: corev1.LocalObjectReference{ + Name: cdb.Spec.CDBAdminPwd.Secret.SecretName, + }, + }, + }, + { + Secret: &corev1.SecretProjection{ + LocalObjectReference: corev1.LocalObjectReference{ + Name: cdb.Spec.ORDSPwd.Secret.SecretName, + }, + }, + }, + { + Secret: &corev1.SecretProjection{ + LocalObjectReference: corev1.LocalObjectReference{ + Name: cdb.Spec.WebServerUser.Secret.SecretName, + }, + }, + }, + { + Secret: &corev1.SecretProjection{ + LocalObjectReference: corev1.LocalObjectReference{ + Name: cdb.Spec.WebServerPwd.Secret.SecretName, + }, + }, + }, + }, + }, + }, + }}, + Containers: []corev1.Container{{ + Name: cdb.Name + "-ords", + Image: cdb.Spec.ORDSImage, + VolumeMounts: []corev1.VolumeMount{{ + MountPath: "/opt/oracle/ords/secrets", + Name: "secrets", + ReadOnly: true, + }}, + Env: func() []corev1.EnvVar { + return []corev1.EnvVar{ + { + Name: "ORACLE_HOST", + Value: cdb.Spec.DBServer, + }, + { + Name: "ORACLE_PORT", + Value: strconv.Itoa(cdb.Spec.DBPort), + }, + { + Name: "ORDS_PORT", + Value: strconv.Itoa(cdb.Spec.ORDSPort), + }, + { + Name: "ORACLE_SERVICE", + Value: cdb.Spec.ServiceName, + }, + { + Name: "ORACLE_PWD_KEY", + Value: cdb.Spec.SysAdminPwd.Secret.Key, + }, + { + Name: "CDBADMIN_USER_KEY", + Value: cdb.Spec.CDBAdminUser.Secret.Key, + }, + { + Name: "CDBADMIN_PWD_KEY", + Value: cdb.Spec.CDBAdminPwd.Secret.Key, + }, + { + Name: "ORDS_PWD_KEY", + Value: cdb.Spec.ORDSPwd.Secret.Key, + }, + { + Name: "WEBSERVER_USER_KEY", + Value: cdb.Spec.WebServerUser.Secret.Key, + }, + { + Name: "WEBSERVER_PASSWORD_KEY", + Value: cdb.Spec.WebServerPwd.Secret.Key, + }, + } + }(), + }}, + + NodeSelector: func() map[string]string { + ns := make(map[string]string) + if len(cdb.Spec.NodeSelector) != 0 { + for key, value := range cdb.Spec.NodeSelector { + ns[key] = value + } + } + return ns + }(), + }, + } + + if len(cdb.Spec.ORDSImagePullSecret) > 0 { + pod.Spec.ImagePullSecrets = []corev1.LocalObjectReference{ + { + Name: cdb.Spec.ORDSImagePullSecret, + }, + } + } + + pod.Spec.Containers[0].ImagePullPolicy = corev1.PullAlways + + if len(cdb.Spec.ORDSImagePullPolicy) > 0 { + if strings.ToUpper(cdb.Spec.ORDSImagePullPolicy) == "NEVER" { + pod.Spec.Containers[0].ImagePullPolicy = corev1.PullNever + } + } + + // Set CDB instance as the owner and controller + ctrl.SetControllerReference(cdb, pod, r.Scheme) + return pod +} + +/************************************************* + * Create a Cluster Service for ORDS CDB Pod + /************************************************/ +func (r *CDBReconciler) createORDSSVC(ctx context.Context, req ctrl.Request, cdb *dbapi.CDB) error { + + log := r.Log.WithValues("createORDSSVC", req.NamespacedName) + + svc := r.createSvcSpec(cdb) + + log.Info("Creating a new Cluster Service for: "+cdb.Name, "Svc.Namespace", svc.Namespace, "Service.Name", svc.Name) + err := r.Create(ctx, svc) + if err != nil { + log.Error(err, "Failed to create new Cluster Service for: "+cdb.Name, "Svc.Namespace", svc.Namespace, "Service.Name", svc.Name) + return err + } + + log.Info("Created ORDS Cluster Service successfully") + r.Recorder.Eventf(cdb, corev1.EventTypeNormal, "CreatedORDSService", "Created ORDS Service for %s", cdb.Name) + return nil +} + +/************************ + * Create Service spec + /************************/ +func (r *CDBReconciler) createSvcSpec(cdb *dbapi.CDB) *corev1.Service { + + svc := &corev1.Service{ + TypeMeta: metav1.TypeMeta{ + Kind: "Service", + }, + ObjectMeta: metav1.ObjectMeta{ + Name: cdb.Name + "-ords", + Namespace: cdb.Namespace, + }, + Spec: corev1.ServiceSpec{ + Selector: map[string]string{ + "name": cdb.Name + "-ords", + }, + ClusterIP: corev1.ClusterIPNone, + }, + } + // Set CDB instance as the owner and controller + ctrl.SetControllerReference(cdb, svc, r.Scheme) + return svc +} + +/************************************************* + * Check CDB deletion + /************************************************/ +func (r *CDBReconciler) manageCDBDeletion(ctx context.Context, req ctrl.Request, cdb *dbapi.CDB) error { + log := r.Log.WithValues("manageCDBDeletion", req.NamespacedName) + + // Check if the PDB instance is marked to be deleted, which is + // indicated by the deletion timestamp being set. + isCDBMarkedToBeDeleted := cdb.GetDeletionTimestamp() != nil + if isCDBMarkedToBeDeleted { + log.Info("Marked to be deleted") + cdb.Status.Phase = cdbPhaseDelete + cdb.Status.Status = true + r.Status().Update(ctx, cdb) + if controllerutil.ContainsFinalizer(cdb, CDBFinalizer) { + // Run finalization logic for CDBFinalizer. If the + // finalization logic fails, don't remove the finalizer so + // that we can retry during the next reconciliation. + err := r.deleteCDBInstance(ctx, req, cdb) + if err != nil { + log.Info("Could not delete CDB Resource", "CDB Name", cdb.Spec.CDBName, "err", err.Error()) + return err + } + + // Remove CDBFinalizer. Once all finalizers have been + // removed, the object will be deleted. + log.Info("Removing finalizer") + controllerutil.RemoveFinalizer(cdb, CDBFinalizer) + err = r.Update(ctx, cdb) + if err != nil { + log.Info("Could not remove finalizer", "err", err.Error()) + return err + } + + log.Info("Successfully removed CDB Resource") + return nil + } + } + + // Add finalizer for this CR + if !controllerutil.ContainsFinalizer(cdb, CDBFinalizer) { + log.Info("Adding finalizer") + + cdb.Status.Phase = cdbPhaseInit + cdb.Status.Status = false + controllerutil.AddFinalizer(cdb, CDBFinalizer) + err := r.Update(ctx, cdb) + if err != nil { + log.Info("Could not add finalizer", "err", err.Error()) + return err + } + } + return nil +} + +/************************************************* + * Delete CDB Resource + /************************************************/ +func (r *CDBReconciler) deleteCDBInstance(ctx context.Context, req ctrl.Request, cdb *dbapi.CDB) error { + + log := r.Log.WithValues("deleteCDBInstance", req.NamespacedName) + + k_client, err := kubernetes.NewForConfig(r.Config) + if err != nil { + log.Error(err, "Kubernetes Config Error") + } + + podName := cdb.Name + "-ords" + + podList := &corev1.PodList{} + listOpts := []client.ListOption{client.InNamespace(req.Namespace), client.MatchingLabels{"name": podName}} + + // List retrieves list of objects for a given namespace and list options. + err = r.List(ctx, podList, listOpts...) + if err != nil { + log.Error(err, "Failed to list pods of: "+podName, "Namespace", req.Namespace) + return err + } + + if len(podList.Items) == 0 { + log.Info("No pods found for: "+podName, "Namespace", req.Namespace) + return nil + } + + for _, pod := range podList.Items { + err = k_client.CoreV1().Pods(cdb.Namespace).Delete(context.TODO(), pod.Name, metav1.DeleteOptions{}) + if err != nil { + log.Info("Could not delete Pod", "Pod Name", pod.Name, "err", err.Error()) + if !strings.Contains(strings.ToUpper(err.Error()), "NOT FOUND") { + return err + } + } else { + log.Info("Successfully deleted ORDS Pod", "Pod Name", pod.Name) + } + } + + r.Recorder.Eventf(cdb, corev1.EventTypeNormal, "DeletedORDSPod", "Deleted ORDS Pod(s) for %s", cdb.Name) + + svcName := cdb.Name + "-ords" + + err = k_client.CoreV1().Services(cdb.Namespace).Delete(context.TODO(), svcName, metav1.DeleteOptions{}) + if err != nil { + log.Info("Could not delete Service", "Service Name", svcName, "err", err.Error()) + if !strings.Contains(strings.ToUpper(err.Error()), "NOT FOUND") { + return err + } + } else { + r.Recorder.Eventf(cdb, corev1.EventTypeNormal, "DeletedORDSService", "Deleted ORDS Service for %s", cdb.Name) + log.Info("Successfully deleted ORDS Service", "Service Name", svcName) + } + + log.Info("Successfully deleted CDB resource", "CDB Name", cdb.Spec.CDBName) + return nil +} + +/************************************************* + * Delete Secrets + /************************************************/ +func (r *CDBReconciler) deleteSecrets(ctx context.Context, req ctrl.Request, cdb *dbapi.CDB) { + + log := r.Log.WithValues("deleteSecrets", req.NamespacedName) + + log.Info("Deleting CDB secrets") + secret := &corev1.Secret{} + err := r.Get(ctx, types.NamespacedName{Name: cdb.Spec.SysAdminPwd.Secret.SecretName, Namespace: cdb.Namespace}, secret) + if err == nil { + err := r.Delete(ctx, secret) + if err == nil { + log.Info("Deleted the secret : " + cdb.Spec.SysAdminPwd.Secret.SecretName) + } + } + + err = r.Get(ctx, types.NamespacedName{Name: cdb.Spec.CDBAdminUser.Secret.SecretName, Namespace: cdb.Namespace}, secret) + if err == nil { + err := r.Delete(ctx, secret) + if err == nil { + log.Info("Deleted the secret : " + cdb.Spec.CDBAdminUser.Secret.SecretName) + } + } + + err = r.Get(ctx, types.NamespacedName{Name: cdb.Spec.CDBAdminPwd.Secret.SecretName, Namespace: cdb.Namespace}, secret) + if err == nil { + err := r.Delete(ctx, secret) + if err == nil { + log.Info("Deleted the secret : " + cdb.Spec.CDBAdminPwd.Secret.SecretName) + } + } + + err = r.Get(ctx, types.NamespacedName{Name: cdb.Spec.ORDSPwd.Secret.SecretName, Namespace: cdb.Namespace}, secret) + if err == nil { + err := r.Delete(ctx, secret) + if err == nil { + log.Info("Deleted the secret : " + cdb.Spec.ORDSPwd.Secret.SecretName) + } + } + + err = r.Get(ctx, types.NamespacedName{Name: cdb.Spec.WebServerUser.Secret.SecretName, Namespace: cdb.Namespace}, secret) + if err == nil { + err := r.Delete(ctx, secret) + if err == nil { + log.Info("Deleted the secret : " + cdb.Spec.WebServerUser.Secret.SecretName) + } + } + + err = r.Get(ctx, types.NamespacedName{Name: cdb.Spec.WebServerPwd.Secret.SecretName, Namespace: cdb.Namespace}, secret) + if err == nil { + err := r.Delete(ctx, secret) + if err == nil { + log.Info("Deleted the secret : " + cdb.Spec.WebServerPwd.Secret.SecretName) + } + } +} + +/************************************************************** + * SetupWithManager sets up the controller with the Manager. + /*************************************************************/ +func (r *CDBReconciler) SetupWithManager(mgr ctrl.Manager) error { + return ctrl.NewControllerManagedBy(mgr). + For(&dbapi.CDB{}). + Owns(&corev1.Pod{}). //Watch for deleted pods owned by this controller + WithEventFilter(predicate.Funcs{ + UpdateFunc: func(e event.UpdateEvent) bool { + // Ignore updates to CR status in which case metadata.Generation does not change + return e.ObjectOld.GetGeneration() != e.ObjectNew.GetGeneration() + }, + DeleteFunc: func(e event.DeleteEvent) bool { + // Evaluates to false if the object has been confirmed deleted. + return !e.DeleteStateUnknown + }, + }). + WithOptions(controller.Options{MaxConcurrentReconciles: 100}). + Complete(r) +} diff --git a/controllers/database/pdb_controller.go b/controllers/database/pdb_controller.go new file mode 100644 index 00000000..92938cf2 --- /dev/null +++ b/controllers/database/pdb_controller.go @@ -0,0 +1,860 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package controllers + +import ( + "bytes" + "context" + "encoding/json" + "errors" + "fmt" + "io/ioutil" + "net/http" + "strconv" + "strings" + "time" + + dbapi "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + + "github.com/go-logr/logr" + corev1 "k8s.io/api/core/v1" + apierrors "k8s.io/apimachinery/pkg/api/errors" + //metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/types" + "k8s.io/client-go/tools/record" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/controller" + "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" + "sigs.k8s.io/controller-runtime/pkg/event" + "sigs.k8s.io/controller-runtime/pkg/predicate" +) + +// PDBReconciler reconciles a PDB object +type PDBReconciler struct { + client.Client + Log logr.Logger + Scheme *runtime.Scheme + Interval time.Duration + Recorder record.EventRecorder +} + +type RESTSQLCollection struct { + Env struct { + DefaultTimeZone string `json:"defaultTimeZone,omitempty"` + } `json:"env"` + Items []SQLItem `json:"items"` +} + +type SQLItem struct { + StatementId int `json:"statementId,omitempty"` + Response []string `json:"response"` + ErrorCode int `json:"errorCode,omitempty"` + ErrorLine int `json:"errorLine,omitempty"` + ErrorColumn int `json:"errorColumn,omitempty"` + ErrorDetails string `json:"errorDetails,omitempty"` + Result int `json:"result,omitempty"` +} + +type ORDSError struct { + Code string `json:"code,omitempty"` + Message string `json:"message,omitempty"` + Type string `json:"type,omitempty"` + Instance string `json:"instance,omitempty"` +} + +var ( + pdbPhaseValidate = "Validating" + pdbPhaseCreate = "Creating" + pdbPhasePlug = "Plugging" + pdbPhaseUnplug = "Unplugging" + pdbPhaseClone = "Cloning" + pdbPhaseFinish = "Finishing" + pdbPhaseReady = "Ready" + pdbPhaseDelete = "Deleting" + pdbPhaseFail = "Failed" +) + +const PDBFinalizer = "database.oracle.com/PDBfinalizer" + +//+kubebuilder:rbac:groups=database.oracle.com,resources=pdbs,verbs=get;list;watch;create;update;patch;delete +//+kubebuilder:rbac:groups=database.oracle.com,resources=pdbs/status,verbs=get;update;patch +//+kubebuilder:rbac:groups=database.oracle.com,resources=pdbs/finalizers,verbs=get;create;update;patch;delete + +// Reconcile is part of the main kubernetes reconciliation loop which aims to +// move the current state of the cluster closer to the desired state. +// TODO(user): Modify the Reconcile function to compare the state specified by +// the PDB object against the actual cluster state, and then +// perform operations to make the cluster state reflect the state specified by +// the user. +// +// For more details, check Reconcile and its Result here: +// - https://pkg.go.dev/sigs.k8s.io/controller-runtime@v0.9.2/pkg/reconcile +func (r *PDBReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + //_ = log.FromContext(ctx) + //_ = r.Log.WithValues("onpremdboperator", req.NamespacedName) + //fmt.Printf("In Reconcile") + //ctx := context.Background() + + log := r.Log.WithValues("onpremdboperator", req.NamespacedName) + log.Info("Reconcile requested") + + reconcilePeriod := r.Interval * time.Second + requeueY := ctrl.Result{Requeue: true, RequeueAfter: reconcilePeriod} + requeueN := ctrl.Result{} + + var err error + pdb := &dbapi.PDB{} + + // Execute for every reconcile + defer func() { + //log.Info("DEFER PDB", "Name", pdb.Name, "Phase", pdb.Status.Phase, "Status", strconv.FormatBool(pdb.Status.Status)) + if !pdb.Status.Status { + if err := r.Status().Update(ctx, pdb); err != nil { + log.Error(err, "Failed to update status for :"+pdb.Name, "err", err.Error()) + } + } + }() + + err = r.Client.Get(context.TODO(), req.NamespacedName, pdb) + if err != nil { + if apierrors.IsNotFound(err) { + log.Info("PDB Resource Not found", "Name", pdb.Name) + // Request object not found, could have been deleted after reconcile req. + // Owned objects are automatically garbage collected. For additional cleanup logic use finalizers. + // Return and don't requeue + pdb.Status.Status = true + return requeueN, nil + } + // Error reading the object - requeue the req. + return requeueY, err + } + + // Finalizer section + err = r.managePDBDeletion(ctx, req, pdb) + if err != nil { + log.Info("Reconcile queued") + return requeueY, nil + } + + action := strings.ToUpper(pdb.Spec.Action) + if pdb.Status.Phase != pdbPhaseFail && pdb.Status.Action != "" && pdb.Status.Action != action { + pdb.Status.Status = false + } + + log.Info("PDB PHASE STATUS:", "Name", pdb.Name, "Phase", pdb.Status.Phase, "Status", strconv.FormatBool(pdb.Status.Status), "Last Action", pdb.Status.Action) + + if (pdb.Status.Phase != pdbPhaseReady && pdb.Status.Phase != pdbPhaseFail && !pdb.Status.Status) || (pdb.Status.Action != "" && pdb.Status.Action != action) { + r.validatePhase(ctx, req, pdb) + } + + if !pdb.Status.Status { + phase := pdb.Status.Phase + log.Info("PDB:", "Name", pdb.Name, "Phase", phase, "Status", strconv.FormatBool(pdb.Status.Status)) + + switch phase { + case pdbPhaseCreate: + err = r.createPDB(ctx, req, pdb) + if err != nil { + log.Info("Reconcile queued") + return requeueY, nil + } + case pdbPhaseClone: + err = r.clonePDB(ctx, req, pdb) + if err != nil { + log.Info("Reconcile queued") + return requeueY, nil + } + case pdbPhasePlug: + err = r.plugPDB(ctx, req, pdb) + if err != nil { + log.Info("Reconcile queued") + return requeueY, nil + } + case pdbPhaseUnplug: + err = r.unplugPDB(ctx, req, pdb) + if err != nil { + log.Info("Reconcile queued") + return requeueY, nil + } + case pdbPhaseDelete: + err = r.deletePDB(ctx, req, pdb) + if err != nil { + log.Info("Reconcile queued") + return requeueY, nil + } + case pdbPhaseReady, pdbPhaseFail: + pdb.Status.Status = true + default: + //pdb.Status.Phase = pdbPhaseValidate + log.Info("DEFAULT:", "Name", pdb.Name, "Phase", phase, "Status", strconv.FormatBool(pdb.Status.Status)) + return requeueN, nil + } + pdb.Status.Action = strings.ToUpper(pdb.Spec.Action) + pdb.Status.Phase = pdbPhaseReady + pdb.Status.Msg = "Success" + } + + log.Info("Reconcile completed") + return requeueN, nil +} + +/************************************************* + * Validate the PDB Spec + /************************************************/ +func (r *PDBReconciler) validatePhase(ctx context.Context, req ctrl.Request, pdb *dbapi.PDB) { + + log := r.Log.WithValues("validatePhase", req.NamespacedName) + + action := strings.ToUpper(pdb.Spec.Action) + + log.Info("Validating PDB phase for: "+pdb.Name, "Action", action) + + switch action { + case "CREATE": + pdb.Status.Phase = pdbPhaseCreate + case "CLONE": + pdb.Status.Phase = pdbPhaseClone + case "PLUG": + pdb.Status.Phase = pdbPhasePlug + case "UNPLUG": + pdb.Status.Phase = pdbPhaseUnplug + case "DELETE": + pdb.Status.Phase = pdbPhaseDelete + } + + log.Info("Validation complete") +} + +/**************************************************************** + * Get the Custom Resource for the CDB mentioned in the PDB Spec + /***************************************************************/ +func (r *PDBReconciler) getCDBResource(ctx context.Context, req ctrl.Request, pdb *dbapi.PDB) (dbapi.CDB, error) { + + log := r.Log.WithValues("getCDBResource", req.NamespacedName) + + var cdb dbapi.CDB // CDB CR corresponding to the CDB name specified in the PDB spec + + // Name of the CDB CR that holds the ORDS container + cdbResName := pdb.Spec.CDBResName + + // Get CDB CR corresponding to the CDB name specified in the PDB spec + err := r.Get(context.Background(), client.ObjectKey{ + Namespace: req.Namespace, + Name: cdbResName, + }, &cdb) + + if err != nil { + log.Info("Failed to get CRD for CDB", "Name", cdbResName, "Namespace", req.Namespace, "Error", err.Error()) + pdb.Status.Phase = pdbPhaseFail + pdb.Status.Msg = "Unable to get CRD for CDB : " + cdbResName + r.Status().Update(ctx, pdb) + return cdb, err + } + + log.Info("Found CR for CDB", "Name", cdbResName, "CR Name", cdb.Name) + return cdb, nil +} + +/**************************************************************** + * Get the ORDS Pod for the CDB mentioned in the PDB Spec + /***************************************************************/ +func (r *PDBReconciler) getORDSPod(ctx context.Context, req ctrl.Request, pdb *dbapi.PDB) (corev1.Pod, error) { + + log := r.Log.WithValues("getORDSPod", req.NamespacedName) + + var cdbPod corev1.Pod // ORDS Pod container with connection to the concerned CDB + + // Name of the CDB CR that holds the ORDS container + cdbResName := pdb.Spec.CDBResName + + // Get ORDS Pod associated with the CDB Name specified in the PDB Spec + err := r.Get(context.Background(), client.ObjectKey{ + Namespace: req.Namespace, + Name: cdbResName + "-ords", + }, &cdbPod) + + if err != nil { + log.Info("Failed to get Pod for CDB", "Name", cdbResName, "Namespace", req.Namespace, "Error", err.Error()) + pdb.Status.Phase = pdbPhaseFail + pdb.Status.Msg = "Unable to get ORDS Pod for CDB : " + cdbResName + r.Status().Update(ctx, pdb) + return cdbPod, err + } + + log.Info("Found ORDS Pod for CDB", "Name", cdbResName, "Pod Name", cdbPod.Name, "ORDS Container hostname", cdbPod.Spec.Hostname) + return cdbPod, nil +} + +/************************************************* + * Get Secret Key for a Secret Name + /************************************************/ +func (r *PDBReconciler) getSecret(ctx context.Context, req ctrl.Request, pdb *dbapi.PDB, secretName string, keyName string) (string, error) { + + log := r.Log.WithValues("getSecret", req.NamespacedName) + + secret := &corev1.Secret{} + err := r.Get(ctx, types.NamespacedName{Name: secretName, Namespace: pdb.Namespace}, secret) + if err != nil { + if apierrors.IsNotFound(err) { + log.Info("Secret not found:" + secretName) + pdb.Status.Phase = pdbPhaseFail + pdb.Status.Msg = "Secret not found:" + secretName + r.Status().Update(ctx, pdb) + return "", err + } + log.Error(err, "Unable to get the secret.") + return "", err + } + + return string(secret.Data[keyName]), nil +} + +/************************************************* + * Issue a REST API Call to the ORDS container + /************************************************/ +func (r *PDBReconciler) callAPI(ctx context.Context, req ctrl.Request, pdb *dbapi.PDB, url string, payload map[string]string, action string) error { + log := r.Log.WithValues("callAPI", req.NamespacedName) + + var err error + httpclient := &http.Client{} + + log.Info("Issuing REST call", "URL", url, "Action", action) + + cdb, err := r.getCDBResource(ctx, req, pdb) + if err != nil { + return err + } + + // Get Web Server User + secret := &corev1.Secret{} + err = r.Get(ctx, types.NamespacedName{Name: cdb.Spec.WebServerUser.Secret.SecretName, Namespace: cdb.Namespace}, secret) + if err != nil { + if apierrors.IsNotFound(err) { + log.Info("Secret not found:" + cdb.Spec.WebServerUser.Secret.SecretName) + return err + } + log.Error(err, "Unable to get the secret.") + return err + } + webUser := string(secret.Data[cdb.Spec.WebServerUser.Secret.Key]) + + // Get Web Server User Password + secret = &corev1.Secret{} + err = r.Get(ctx, types.NamespacedName{Name: cdb.Spec.WebServerPwd.Secret.SecretName, Namespace: cdb.Namespace}, secret) + if err != nil { + if apierrors.IsNotFound(err) { + log.Info("Secret not found:" + cdb.Spec.WebServerPwd.Secret.SecretName) + return err + } + log.Error(err, "Unable to get the secret.") + return err + } + webUserPwd := string(secret.Data[cdb.Spec.WebServerPwd.Secret.Key]) + + //fmt.Println("payload:", payload) + + jsonValue, _ := json.Marshal(payload) + httpreq, err := http.NewRequest(action, url, bytes.NewBuffer(jsonValue)) + if err != nil { + log.Info("Unable to create HTTP Request for PDB : "+pdb.Name, "err", err.Error()) + return err + } + + httpreq.Header.Add("Accept", "application/json") + httpreq.Header.Add("Content-Type", "application/json") + httpreq.SetBasicAuth(webUser, webUserPwd) + + resp, err := httpclient.Do(httpreq) + if err != nil { + log.Error(err, "Failed - Could not connect to ORDS Pod", "err", err.Error()) + pdb.Status.Phase = pdbPhaseFail + pdb.Status.Msg = "Could not connect to ORDS Pod" + pdb.Status.Status = true + if err := r.Status().Update(ctx, pdb); err != nil { + log.Error(err, "Failed to update status for :"+pdb.Name, "err", err.Error()) + } + r.Recorder.Eventf(pdb, corev1.EventTypeWarning, "ORDSError", "Failed: Could not connect to ORDS Pod") + return err + } + + if resp.StatusCode != http.StatusOK { + bb, _ := ioutil.ReadAll(resp.Body) + pdb.Status.Phase = pdbPhaseFail + pdb.Status.Msg = "ORDS Error - HTTP Status Code:" + strconv.Itoa(resp.StatusCode) + pdb.Status.Status = true + if err := r.Status().Update(ctx, pdb); err != nil { + log.Error(err, "Failed to update status for :"+pdb.Name, "err", err.Error()) + } + log.Info("ORDS Error - HTTP Status Code :"+strconv.Itoa(resp.StatusCode), "Err", string(bb)) + + var apiErr ORDSError + json.Unmarshal([]byte(bb), &apiErr) + r.Recorder.Eventf(pdb, corev1.EventTypeWarning, "ORDSError", "Failed: %s", apiErr.Message) + //fmt.Printf("%+v", apiErr) + //fmt.Println(string(bb)) + return errors.New("ORDS Error") + } + + defer resp.Body.Close() + + bodyBytes, err := ioutil.ReadAll(resp.Body) + if err != nil { + fmt.Print(err.Error()) + } + //fmt.Println(string(bodyBytes)) + + var apiResponse RESTSQLCollection + json.Unmarshal([]byte(bodyBytes), &apiResponse) + //fmt.Printf("%#v", apiResponse) + //fmt.Printf("%+v", apiResponse) + + errFound := false + for _, sqlItem := range apiResponse.Items { + if sqlItem.ErrorDetails != "" { + log.Info("ORDS Error - Oracle Error Code :" + strconv.Itoa(sqlItem.ErrorCode)) + if !errFound { + pdb.Status.Phase = pdbPhaseFail + pdb.Status.Msg = sqlItem.ErrorDetails + pdb.Status.Status = true + if err := r.Status().Update(ctx, pdb); err != nil { + log.Error(err, "Failed to update status for :"+pdb.Name, "err", err.Error()) + } + } + r.Recorder.Eventf(pdb, corev1.EventTypeWarning, "OraError", "%s", sqlItem.ErrorDetails) + errFound = true + } + } + + if errFound { + return errors.New("Oracle Error") + } + + return nil +} + +/************************************************* + * Create a PDB + /************************************************/ +func (r *PDBReconciler) createPDB(ctx context.Context, req ctrl.Request, pdb *dbapi.PDB) error { + + log := r.Log.WithValues("createPDB", req.NamespacedName) + + var err error + + cdb, err := r.getCDBResource(ctx, req, pdb) + if err != nil { + return err + } + + pdbAdminName, err := r.getSecret(ctx, req, pdb, pdb.Spec.AdminName.Secret.SecretName, pdb.Spec.AdminName.Secret.Key) + if err != nil { + return err + } + pdbAdminPwd, err := r.getSecret(ctx, req, pdb, pdb.Spec.AdminPwd.Secret.SecretName, pdb.Spec.AdminPwd.Secret.Key) + if err != nil { + return err + } + + values := map[string]string{ + "method": "CREATE", + "pdb_name": pdb.Spec.PDBName, + "adminName": pdbAdminName, + "adminPwd": pdbAdminPwd, + "fileNameConversions": pdb.Spec.FileNameConversions, + "reuseTempFile": strconv.FormatBool(*(pdb.Spec.ReuseTempFile)), + "unlimitedStorage": strconv.FormatBool(*(pdb.Spec.UnlimitedStorage)), + "totalSize": pdb.Spec.TotalSize, + "tempSize": pdb.Spec.TempSize, + "getScript": strconv.FormatBool(*(pdb.Spec.GetScript))} + + if *(pdb.Spec.TDEImport) { + tdePassword, err := r.getSecret(ctx, req, pdb, pdb.Spec.TDEPassword.Secret.SecretName, pdb.Spec.TDEPassword.Secret.Key) + if err != nil { + return err + } + tdeSecret, err := r.getSecret(ctx, req, pdb, pdb.Spec.TDESecret.Secret.SecretName, pdb.Spec.TDESecret.Secret.Key) + if err != nil { + return err + } + values["tdePassword"] = tdePassword + values["tdeKeystorePath"] = pdb.Spec.TDEKeystorePath + values["tdeSecret"] = tdeSecret + } + + url := "http://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + + pdb.Status.Phase = pdbPhaseCreate + pdb.Status.Msg = "Waiting for PDB to be created" + if err := r.Status().Update(ctx, pdb); err != nil { + log.Error(err, "Failed to update status for :"+pdb.Name, "err", err.Error()) + } + err = r.callAPI(ctx, req, pdb, url, values, "POST") + if err != nil { + return err + } + + r.Recorder.Eventf(pdb, corev1.EventTypeNormal, "Created", "PDB '%s' created successfully", pdb.Spec.PDBName) + + pdb.Status.ConnString = cdb.Spec.SCANName + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName + log.Info("Created PDB Resource", "PDB Name", pdb.Spec.PDBName) + return nil +} + +/************************************************* + * Clone a PDB + /************************************************/ +func (r *PDBReconciler) clonePDB(ctx context.Context, req ctrl.Request, pdb *dbapi.PDB) error { + + log := r.Log.WithValues("clonePDB", req.NamespacedName) + + var err error + + cdb, err := r.getCDBResource(ctx, req, pdb) + if err != nil { + return err + } + + values := map[string]string{ + "method": "CLONE", + "clonePDBName": pdb.Spec.PDBName, + "reuseTempFile": strconv.FormatBool(*(pdb.Spec.ReuseTempFile)), + "unlimitedStorage": strconv.FormatBool(*(pdb.Spec.UnlimitedStorage)), + "getScript": strconv.FormatBool(*(pdb.Spec.GetScript))} + + if pdb.Spec.SparseClonePath != "" { + values["sparseClonePath"] = pdb.Spec.SparseClonePath + } + if pdb.Spec.FileNameConversions != "" { + values["fileNameConversions"] = pdb.Spec.FileNameConversions + } + if pdb.Spec.TotalSize != "" { + values["totalSize"] = pdb.Spec.TotalSize + } + if pdb.Spec.TempSize != "" { + values["tempSize"] = pdb.Spec.TempSize + } + + url := "http://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdb.Spec.SrcPDBName + "/" + + pdb.Status.Phase = pdbPhaseClone + pdb.Status.Msg = "Waiting for PDB to be cloned" + if err := r.Status().Update(ctx, pdb); err != nil { + log.Error(err, "Failed to update status for :"+pdb.Name, "err", err.Error()) + } + err = r.callAPI(ctx, req, pdb, url, values, "POST") + if err != nil { + return err + } + + r.Recorder.Eventf(pdb, corev1.EventTypeNormal, "Created", "PDB '%s' cloned successfully", pdb.Spec.PDBName) + + pdb.Status.ConnString = cdb.Spec.SCANName + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName + log.Info("Cloned PDB successfully", "Source PDB Name", pdb.Spec.SrcPDBName, "Clone PDB Name", pdb.Spec.PDBName) + return nil +} + +/************************************************* + * Plug a PDB + /************************************************/ +func (r *PDBReconciler) plugPDB(ctx context.Context, req ctrl.Request, pdb *dbapi.PDB) error { + + log := r.Log.WithValues("plugPDB", req.NamespacedName) + + var err error + + cdb, err := r.getCDBResource(ctx, req, pdb) + if err != nil { + return err + } + + values := map[string]string{ + "method": "PLUG", + "xmlFileName": pdb.Spec.XMLFileName, + "pdb_name": pdb.Spec.PDBName, + //"adminName": pdbAdminName, + //"adminPwd": pdbAdminPwd, + "sourceFileNameConversions": pdb.Spec.SourceFileNameConversions, + "copyAction": pdb.Spec.CopyAction, + "fileNameConversions": pdb.Spec.FileNameConversions, + "unlimitedStorage": strconv.FormatBool(*(pdb.Spec.UnlimitedStorage)), + "reuseTempFile": strconv.FormatBool(*(pdb.Spec.ReuseTempFile)), + "totalSize": pdb.Spec.TotalSize, + "tempSize": pdb.Spec.TempSize, + "getScript": strconv.FormatBool(*(pdb.Spec.GetScript))} + + if *(pdb.Spec.TDEImport) { + tdePassword, err := r.getSecret(ctx, req, pdb, pdb.Spec.TDEPassword.Secret.SecretName, pdb.Spec.TDEPassword.Secret.Key) + if err != nil { + return err + } + tdeSecret, err := r.getSecret(ctx, req, pdb, pdb.Spec.TDESecret.Secret.SecretName, pdb.Spec.TDESecret.Secret.Key) + if err != nil { + return err + } + values["tdePassword"] = tdePassword + values["tdeKeystorePath"] = pdb.Spec.TDEKeystorePath + values["tdeSecret"] = tdeSecret + } + if *(pdb.Spec.AsClone) { + values["asClone"] = strconv.FormatBool(*(pdb.Spec.AsClone)) + } + + url := "http://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + + pdb.Status.Phase = pdbPhasePlug + pdb.Status.Msg = "Waiting for PDB to be plugged" + if err := r.Status().Update(ctx, pdb); err != nil { + log.Error(err, "Failed to update status for :"+pdb.Name, "err", err.Error()) + } + err = r.callAPI(ctx, req, pdb, url, values, "POST") + if err != nil { + return err + } + + r.Recorder.Eventf(pdb, corev1.EventTypeNormal, "Created", "PDB '%s' plugged successfully", pdb.Spec.PDBName) + + pdb.Status.ConnString = cdb.Spec.SCANName + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName + log.Info("Successfully plugged PDB", "PDB Name", pdb.Spec.PDBName) + return nil +} + +/************************************************* + * Unplug a PDB + /************************************************/ +func (r *PDBReconciler) unplugPDB(ctx context.Context, req ctrl.Request, pdb *dbapi.PDB) error { + + log := r.Log.WithValues("unplugPDB", req.NamespacedName) + + var err error + + cdb, err := r.getCDBResource(ctx, req, pdb) + if err != nil { + return err + } + + values := map[string]string{ + "method": "UNPLUG", + "xmlFileName": pdb.Spec.XMLFileName, + "getScript": strconv.FormatBool(*(pdb.Spec.GetScript))} + + if *(pdb.Spec.TDEExport) { + // Get the TDE Password + tdePassword, err := r.getSecret(ctx, req, pdb, pdb.Spec.TDEPassword.Secret.SecretName, pdb.Spec.TDEPassword.Secret.Key) + if err != nil { + return err + } + tdeSecret, err := r.getSecret(ctx, req, pdb, pdb.Spec.TDESecret.Secret.SecretName, pdb.Spec.TDESecret.Secret.Key) + if err != nil { + return err + } + values["tdePassword"] = tdePassword + values["tdeKeystorePath"] = pdb.Spec.TDEKeystorePath + values["tdeSecret"] = tdeSecret + } + + url := "http://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdb.Spec.PDBName + "/" + + pdb.Status.Phase = pdbPhaseUnplug + pdb.Status.Msg = "Waiting for PDB to be unplugged" + if err := r.Status().Update(ctx, pdb); err != nil { + log.Error(err, "Failed to update status for :"+pdb.Name, "err", err.Error()) + } + err = r.callAPI(ctx, req, pdb, url, values, "POST") + if err != nil { + return err + } + + if controllerutil.ContainsFinalizer(pdb, PDBFinalizer) { + log.Info("Removing finalizer") + controllerutil.RemoveFinalizer(pdb, PDBFinalizer) + err := r.Update(ctx, pdb) + if err != nil { + log.Info("Could not remove finalizer", "err", err.Error()) + return err + } + pdb.Status.Status = true + err = r.Delete(context.Background(), pdb, client.GracePeriodSeconds(1)) + if err != nil { + log.Info("Could not delete PDB resource", "err", err.Error()) + return err + } + } + + r.Recorder.Eventf(pdb, corev1.EventTypeNormal, "Unplugged", "PDB '%s' unplugged successfully", pdb.Spec.PDBName) + + log.Info("Successfully unplugged PDB resource") + return nil +} + +/************************************************* + * Delete a PDB + /************************************************/ +func (r *PDBReconciler) deletePDB(ctx context.Context, req ctrl.Request, pdb *dbapi.PDB) error { + + log := r.Log.WithValues("deletePDB", req.NamespacedName) + + err := r.deletePDBInstance(req, ctx, pdb) + if err != nil { + log.Info("Could not delete PDB", "PDB Name", pdb.Spec.PDBName, "err", err.Error()) + return err + } + + if controllerutil.ContainsFinalizer(pdb, PDBFinalizer) { + log.Info("Removing finalizer") + controllerutil.RemoveFinalizer(pdb, PDBFinalizer) + err := r.Update(ctx, pdb) + if err != nil { + log.Info("Could not remove finalizer", "err", err.Error()) + return err + } + pdb.Status.Status = true + err = r.Delete(context.Background(), pdb, client.GracePeriodSeconds(1)) + if err != nil { + log.Info("Could not delete PDB resource", "err", err.Error()) + return err + } + } + + r.Recorder.Eventf(pdb, corev1.EventTypeNormal, "Deleted", "PDB '%s' dropped successfully", pdb.Spec.PDBName) + + log.Info("Successfully deleted PDB resource") + return nil +} + +/************************************************* + * Check PDB deletion + /************************************************/ +func (r *PDBReconciler) managePDBDeletion(ctx context.Context, req ctrl.Request, pdb *dbapi.PDB) error { + log := r.Log.WithValues("managePDBDeletion", req.NamespacedName) + + // Check if the PDB instance is marked to be deleted, which is + // indicated by the deletion timestamp being set. + isPDBMarkedToBeDeleted := pdb.GetDeletionTimestamp() != nil + if isPDBMarkedToBeDeleted { + log.Info("Marked to be deleted") + if controllerutil.ContainsFinalizer(pdb, PDBFinalizer) { + // Remove PDBFinalizer. Once all finalizers have been + // removed, the object will be deleted. + log.Info("Removing finalizer") + controllerutil.RemoveFinalizer(pdb, PDBFinalizer) + err := r.Update(ctx, pdb) + if err != nil { + log.Info("Could not remove finalizer", "err", err.Error()) + return err + } + pdb.Status.Status = true + log.Info("Successfully removed PDB resource") + return nil + } + } + + // Add finalizer for this CR + if !controllerutil.ContainsFinalizer(pdb, PDBFinalizer) { + log.Info("Adding finalizer") + controllerutil.AddFinalizer(pdb, PDBFinalizer) + err := r.Update(ctx, pdb) + if err != nil { + log.Info("Could not add finalizer", "err", err.Error()) + return err + } + pdb.Status.Status = false + } + return nil +} + +/************************************************* + * Finalization logic for PDBFinalizer + /************************************************/ +func (r *PDBReconciler) deletePDBInstance(req ctrl.Request, ctx context.Context, pdb *dbapi.PDB) error { + + log := r.Log.WithValues("deletePDBInstance", req.NamespacedName) + + var err error + + cdb, err := r.getCDBResource(ctx, req, pdb) + if err != nil { + return err + } + + values := map[string]string{ + "method": "DELETE", + "getScript": strconv.FormatBool(*(pdb.Spec.GetScript))} + + if pdb.Spec.DropAction != "" { + values["action"] = pdb.Spec.DropAction + } + + pdbName := pdb.Spec.PDBName + url := "http://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdbName + "/" + + pdb.Status.Phase = pdbPhaseDelete + pdb.Status.Msg = "Waiting for PDB to be deleted" + if err := r.Status().Update(ctx, pdb); err != nil { + log.Error(err, "Failed to update status for :"+pdb.Name, "err", err.Error()) + } + err = r.callAPI(ctx, req, pdb, url, values, "DELETE") + if err != nil { + return err + } + + log.Info("Successfully dropped PDB", "PDB Name", pdbName) + return nil +} + +/************************************************************** + * SetupWithManager sets up the controller with the Manager. + /*************************************************************/ +func (r *PDBReconciler) SetupWithManager(mgr ctrl.Manager) error { + return ctrl.NewControllerManagedBy(mgr). + For(&dbapi.PDB{}). + WithEventFilter(predicate.Funcs{ + UpdateFunc: func(e event.UpdateEvent) bool { + // Ignore updates to CR status in which case metadata.Generation does not change + return e.ObjectOld.GetGeneration() != e.ObjectNew.GetGeneration() + }, + DeleteFunc: func(e event.DeleteEvent) bool { + // Evaluates to false if the object has been confirmed deleted. + return !e.DeleteStateUnknown + }, + }). + WithOptions(controller.Options{MaxConcurrentReconciles: 100}). + Complete(r) +} diff --git a/docs/onpremdb/README.md b/docs/onpremdb/README.md new file mode 100644 index 00000000..35d59e02 --- /dev/null +++ b/docs/onpremdb/README.md @@ -0,0 +1,189 @@ +# Oracle On-Premise Database Controller + +The On-Premise Database Controller enables provisioning of Oracle Databases (PDBs) both on a Kubernetes cluster or outside of a Kubernetes cluster. The following sections explain the setup and functionality of this controller: + +* [Prerequisites for On-Premise Database Controller](ORACLE_ONPREMDB_CONTROLLER_README.md#prerequisites-and-setup) +* [Kubernetes Secrets](ORACLE_ONPREMDB_CONTROLLER_README.md#kubernetes-secrets) +* [Kubernetes CRD for CDB](ORACLE_ONPREMDB_CONTROLLER_README.md#kubernetes-crd-for-cdb) +* [Kubernetes CRD for PDB](ORACLE_ONPREMDB_CONTROLLER_README.md#kubernetes-crd-for-pdb) +* [PDB Lifecycle Management Operations](ORACLE_ONPREMDB_CONTROLLER_README.md#pdb-lifecycl-management-operations) +* [Validation and Errors](ORACLE_ONPREMDB_CONTROLLER_README.md#validation-and-errors) + +## Prerequisites for On-Premise Database Controller + ++ ### Prepare CDB for PDB Lifecycme Management (PDB-LM) + + Pluggable Database management is performed in the Container Database (CDB) and includes create, clone, plug, unplug and delete operations. + You cannot have an ORDS enabled schema in the container database. To perform the PDB lifecycle management operations, the default CDB administrator credentials must be defined. + + To define the default CDB administrator credentials, perform the following steps on the target CDB(s) where PDB-LM operations are to be performed: + + Create the CDB administrator user and grant the SYSDBA privilege. In this example, the user is called C##DBAPI_CDB_ADMIN. However, any suitable common user name can be used. + + ```sh + CREATE USER C##DBAPI_CDB_ADMIN IDENTIFIED BY ; + GRANT SYSOPER TO C##DBAPI_CDB_ADMIN CONTAINER = ALL; + ``` ++ ### Building the Oracle REST Data Service (ORDS) Image + + Oracle On-Premise Database controller enhances the Oracle REST Data Services (ORDS) image to enable it for PDB Lifecycle Management. You can build this image by following the instructions below: + * After cloning the repository, go to the "ords" folder, and run: + ```sh + docker build -t oracle/ords-dboper:latest . + ``` + * Once the image is ready, you need to push it to your Docker Images Repository to pull it during CDB Resource creation. + ++ ### Install cert-manager + + Validating webhook is an endpoint Kubernetes can invoke prior to persisting resources in ETCD. This endpoint returns a structured response indicating whether the resource should be rejected or accepted and persisted to the datastore. + + Webhook requires a TLS certificate that the apiserver is configured to trust . Install the cert-manager with the following command: + + ```sh + kubectl apply -f https://github.com/jetstack/cert-manager/releases/latest/download/cert-manager.yaml + ``` + +## Kubernetes Secrets + + Both CDBs and PDBs make use of Kubernetes Secrets to store usernames and passwords. + + For CDB, create a secret file as shown here: [config/samples/onpremdb/cdb_secret.yaml](../../config/samples/onpremdb/cdb_secret.yaml) + + ```sh + $ kubectl apply -f cdb_secret.yaml + secret/cdb1-secret created + ``` + On successful creation of the CDB Resource, the CDB secrets would be deleted from the Kubernetes system. + + For PDB, create a secret file as shown here: [config/samples/onpremdb/pdb_secret.yaml](../../config/samples/onpremdb/pdb_secret.yaml) + + ```sh + $ kubectl apply -f pdb_secret.yaml + secret/pdb1-secret created + ``` + **Note:** Don't leave plaintext files containing sensitive data on disk. After loading the Secret, remove the plaintext file or move it to secure storage. + + Another option is to use "kubectl create secret" command as shown below for the PDB: + ```sh + $ kubectl create secret generic pdb1-secret --from-literal sysadmin_user=pdbadmin --from-literal sysdamin_pwd=WE2112#232# + secret/pdb1-secret created + ``` + +## Kubernetes CRD for CDB + + The Oracle Database Operator creates the CDB kind as a custom resource that models a target CDB as a native Kubernetes object. This is only used to create Pods to connect to the target CDB to perform PDB-LM operations. Each CDB resource follows the CDB CRD as defined here: [config/crd/bases/database.oracle.com_cdbs.yaml](../../config/crd/bases/database.oracle.com_cdbs.yaml) + + + ### CDB Sample YAML + + A sample .yaml file is available here: [config/samples/onpremdb/cdb.yaml](../../config/samples/onpremdb/cdb.yaml) + + **Note:** The password and username fields in the above `cdb.yaml` yaml are Kubernetes Secrets. Please see the section [Kubernetes Secrets](ORACLE_ONPREMDB_CONTROLLER_README.md#kubernetes-secrets) for more information. + + + ### Check the status of the all CDBs + ```sh + $ kubectl get cdbs -A + + NAMESPACE NAME CDB NAME DB SERVER DB PORT SCAN NAME STATUS MESSAGE + oracle-database-operator-system cdb-dev devdb 172.17.0.4 1521 devdb Ready Success + ``` + +## Kubernetes CRD for PDB + + The Oracle Database Operator creates the PDB kind as a custom resource that models a PDB as a native Kubernetes object. This PDB resource can be used to perform PDB-LM operations by specifying the action attribute in the PDB specs. Each PDB resource follows the PDB CRD as defined here: [config/crd/bases/database.oracle.com_pdbs.yaml](../../config/crd/bases/database.oracle.com_pdbs.yaml) + + + + ### PDB Sample YAML + + A sample .yaml file is available here: [config/samples/onpremdb/pdb.yaml](../../config/samples/onpremdb/pdb.yaml) + + **Note:** The password and username fields in the above `pdb.yaml` yaml are Kubernetes Secrets. Please see the section [Kubernetes Secrets](ORACLE_ONPREMDB_CONTROLLER_README.md#kubernetes-secrets) for more information. + + + ### Check the status of the all PDBs + ```sh + $ kubectl get pdbs -A + + NAMESPACE NAME CONNECT STRING CDB NAME PDB NAME PDB SIZE STATUS MESSAGE + oracle-database-operator-system pdb1 devdb:1521/pdbdev cdb-dev pdbdev 2G Ready Success + oracle-database-operator-system pdb2 testdb:1521/pdbtets cdb-test pdbtes 1G Ready Success + ``` + +## PDB Lifecycle Management Operations + + Using ORDS, you can perform the following PDB-LM operations: CREATE, CLONE, PLUG, UNPLUG and DELETE + ++ ### Create PDB + + A sample .yaml file is available here: [config/samples/onpremdb/pdb.yaml](../../config/samples/onpremdb/pdb.yaml) + ++ ### Clone PDB + + A sample .yaml file is available here: [config/samples/onpremdb/pdb_clone.yaml](../../config/samples/onpremdb/pdb_clone.yaml) + ++ ### Plug PDB + + A sample .yaml file is available here: [config/samples/onpremdb/pdb_plug.yaml](../../config/samples/onpremdb/pdb_plug.yaml) + ++ ### Unplug PDB + + A sample .yaml file is available here: [config/samples/onpremdb/pdb_unplug.yaml](../../config/samples/onpremdb/pdb_unplug.yaml) + ++ ### Delete PDB + + A sample .yaml file is available here: [config/samples/onpremdb/pdb_delete.yaml](../../config/samples/onpremdb/pdb_delete.yaml) + +## Validation and Errors + +You can check Kubernetes events for any errors or status updates as shown below: +```sh +$ kubectl get events -A +NAMESPACE LAST SEEN TYPE REASON OBJECT MESSAGE +oracle-database-operator-system 58m Warning Failed pod/cdb-dev-ords-qiigr Error: secret "cdb1-secret" not found +oracle-database-operator-system 56m Normal DeletedORDSPod cdb/cdb-dev Deleted ORDS Pod(s) for cdb-dev +oracle-database-operator-system 56m Normal DeletedORDSService cdb/cdb-dev Deleted ORDS Service for cdb-dev +... +oracle-database-operator-system 26m Warning OraError pdb/pdb1 ORA-65016: FILE_NAME_CONVERT must be specified... +oracle-database-operator-system 24m Warning OraError pdb/pdb2 ORA-65011: Pluggable database DEMOTEST does not exist. +... +oracle-database-operator-system 20m Normal Created pdb/pdb1 PDB 'demotest' created successfully +... +oracle-database-operator-system 17m Warning OraError pdb/pdb3 ORA-65012: Pluggable database DEMOTEST already exists... +``` + ++ ### CDB Validation and Errors + +Validation is done at the time of CDB resource creation as shown below: +```sh +$ kubectl apply -f cdb1.yaml +The PDB "cdb-dev" is invalid: +* spec.dbServer: Required value: Please specify Database Server Name or IP Address +* spec.dbPort: Required value: Please specify DB Server Port +* spec.ordsImage: Required value: Please specify name of ORDS Image to be used +``` + +Apart from events, listing of CDBs will also show the possible reasons why a particular CDB CR could not be created as shown below: +```sh + $ kubectl get cdbs -A + + NAMESPACE NAME CDB NAME DB SERVER DB PORT SCAN NAME STATUS MESSAGE + oracle-database-operator-system cdb-dev devdb 172.17.0.4 1521 devdb Failed Secret not found:cdb1-secret +``` + ++ ### PDB Validation and Errors + +Validation is done at the time of PDB resource creation as shown below: +```sh +$ kubectl apply -f pdb1.yaml +The PDB "pdb1" is invalid: +* spec.cdbResName: Required value: Please specify the name of the CDB Kubernetes resource to use for PDB operations +* spec.pdbName: Required value: Please specify name of the PDB to be created +* spec.adminPwd: Required value: Please specify PDB System Administrator Password +* spec.fileNameConversions: Required value: Please specify a value for fileNameConversions. Values can be a filename convert pattern or NONE +``` + +Similarly, for PDBs, listing of PDBs will also show the possible reasons why a particular PDB CR could not be created as shown below: +```sh +$ kubectl get pdbs -A +NAMESPACE NAME CONNECT STRING CDB NAME PDB NAME PDB SIZE STATUS MESSAGE +oracle-database-operator-system pdb1 democdb demotest1 Failed Secret not found:pdb12-secret +oracle-database-operator-system pdb2 democdb demotest2 Failed ORA-65016: FILE_NAME_CONVERT must be specified... +``` \ No newline at end of file diff --git a/main.go b/main.go index 9e68368f..cdb8b01d 100644 --- a/main.go +++ b/main.go @@ -42,6 +42,7 @@ import ( "flag" "os" "strconv" + "time" "k8s.io/apimachinery/pkg/runtime" utilruntime "k8s.io/apimachinery/pkg/util/runtime" @@ -131,6 +132,39 @@ func main() { setupLog.Error(err, "unable to create webhook", "webhook", "SingleInstanceDatabase") os.Exit(1) } + if err = (&databasev1alpha1.PDB{}).SetupWebhookWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create webhook", "webhook", "PDB") + os.Exit(1) + } + if err = (&databasev1alpha1.CDB{}).SetupWebhookWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create webhook", "webhook", "CDB") + os.Exit(1) + } + } + + // PDB Reconciler + if err = (&databasecontroller.PDBReconciler{ + Client: mgr.GetClient(), + Scheme: mgr.GetScheme(), + Log: ctrl.Log.WithName("controllers").WithName("PDB"), + Interval: time.Duration(i), + Recorder: mgr.GetEventRecorderFor("PDB"), + }).SetupWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create controller", "controller", "PDB") + os.Exit(1) + } + + // CDB Reconciler + if err = (&databasecontroller.CDBReconciler{ + Client: mgr.GetClient(), + Scheme: mgr.GetScheme(), + Config: mgr.GetConfig(), + Log: ctrl.Log.WithName("controllers").WithName("CDB"), + Interval: time.Duration(i), + Recorder: mgr.GetEventRecorderFor("CDB"), + }).SetupWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create controller", "controller", "CDB") + os.Exit(1) } // +kubebuilder:scaffold:builder diff --git a/ords/Dockerfile b/ords/Dockerfile new file mode 100644 index 00000000..eeff5cba --- /dev/null +++ b/ords/Dockerfile @@ -0,0 +1,66 @@ +# LICENSE UPL 1.0 +# +# Copyright (c) 1982-2017 Oracle and/or its affiliates. All rights reserved. +# +# ORACLE DOCKERFILES PROJECT +# -------------------------- +# This is the Dockerfile for Oracle Rest Data Services +# +# REQUIRED FILES TO BUILD THIS IMAGE +# ---------------------------------- +# (1) ords.3.0.10.165.06.53.zip +# Download Oracle Rest Data Services from +# http://www.oracle.com/technetwork/developer-tools/rest-data-services/downloads/index.html +# +# HOW TO BUILD THIS IMAGE +# ----------------------- +# Put the downloaded file in the same directory as this Dockerfile +# Run: +# $ docker build -t oracle/restdataservices:3.0.10 . +# +# Pull base image +# --------------- +FROM container-registry.oracle.com/java/serverjre:8 + +# Environment variables required for this build (do NOT change) +# ------------------------------------------------------------- +ENV ORDS_HOME=/opt/oracle/ords \ + INSTALL_FILE=ords*.zip \ + CONFIG_PROPS="ords_params.properties.tmpl" \ + STANDALONE_PROPS="standalone.properties.tmpl" \ + CDBADMIN_PROPS="cdbadmin.properties.tmpl" \ + SETUP_WEBUSER="setupwebuser.sh" \ + RUN_FILE="runOrds.sh" + +# Copy binaries +# ------------- +COPY $INSTALL_FILE $CONFIG_PROPS $STANDALONE_PROPS $CDBADMIN_PROPS $RUN_FILE $SETUP_WEBUSER $ORDS_HOME/ + +RUN yum -y install bind-utils net-tools zip unzip tar wget vim-minimal which sudo expect procps && \ + yum clean all + +# Setup filesystem and oracle user +# Adjust file permissions, go to /opt/oracle as user 'oracle' to proceed with ORDS installation +# ------------------------------------------------------------ +RUN mkdir -p $ORDS_HOME/doc_root && \ + chmod ug+x $ORDS_HOME/*.sh && \ + groupadd -g 54322 dba && \ + useradd -u 54321 -d /home/oracle -g dba -m -s /bin/bash oracle && \ + cd $ORDS_HOME && \ + jar -xf $INSTALL_FILE && \ + rm $INSTALL_FILE && \ + mkdir -p $ORDS_HOME/config/ords && \ + mkdir -p $ORDS_HOME/secrets && \ + java -jar $ORDS_HOME/ords.war configdir $ORDS_HOME/config && \ + chown -R oracle:dba $ORDS_HOME + +# Finalize setup +# ------------------- +USER oracle +WORKDIR /home/oracle + +VOLUME ["$ORDS_HOME/config/ords"] +EXPOSE 8888 + +# Define default command to start Oracle Database. +CMD $ORDS_HOME/$RUN_FILE diff --git a/ords/cdbadmin.properties.tmpl b/ords/cdbadmin.properties.tmpl new file mode 100644 index 00000000..b1a801e4 --- /dev/null +++ b/ords/cdbadmin.properties.tmpl @@ -0,0 +1,3 @@ +database.api.admin.enabled=true +db.cdb.adminUser=###CDBADMIN_USER### as SYSOPER +db.cdb.adminUser.password=###CDBADMIN_PWD### \ No newline at end of file diff --git a/ords/ords_params.properties.tmpl b/ords/ords_params.properties.tmpl new file mode 100644 index 00000000..d2aaf33b --- /dev/null +++ b/ords/ords_params.properties.tmpl @@ -0,0 +1,16 @@ +db.hostname=###ORACLE_HOST### +db.port=###ORACLE_PORT### +db.servicename=###ORACLE_SERVICE### +database.api.admin.enabled=true +database.api.enabled=true +feature.sdw=true +plsql.gateway.add=false +restEnabledSql.active=true +rest.services.apex.add=false +rest.services.ords.add=true +standalone.http.port=8888 +standalone.mode=false +standalone.use.https=true +sys.user=SYS +sys.password=###ORACLE_PWD### +user.public.password=###ORDS_PWD### \ No newline at end of file diff --git a/ords/runOrds.sh b/ords/runOrds.sh new file mode 100644 index 00000000..d6ee09cc --- /dev/null +++ b/ords/runOrds.sh @@ -0,0 +1,75 @@ +#!/bin/bash +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +# Since: June, 2017 +# Author: gerald.venzl@oracle.com +# Description: Setup and runs Oracle Rest Data Services. +# +# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS HEADER. +# +# Copyright (c) 2014-2017 Oracle and/or its affiliates. All rights reserved. +# +# MODIFIED (DD-Mon-YY) +# gdbhat 16-Aug-21 - Added CDB Admin properties. Commented out ORACLE_PWD check + +function setupOrds() { + + # Check whether the Oracle DB password has been specified + #if [ "$ORACLE_PWD" == "" ]; then + # echo "Error: No ORACLE_PWD specified!" + # echo "Please specify Oracle DB password using the ORACLE_PWD environment variable." + # exit 1; + #fi; + + # Defaults + ORACLE_SERVICE=${ORACLE_SERVICE:-"ORCLPDB1"} + ORACLE_HOST=${ORACLE_HOST:-"localhost"} + ORACLE_PORT=${ORACLE_PORT:-"1521"} + APEXI=${APEXI:-"$ORDS_HOME/doc_root/i"} + ORDS_PORT=${ORDS_PORT:-"8888"} + + ORDS_PWD=`cat $ORDS_HOME/secrets/$ORDS_PWD_KEY` + ORACLE_PWD=`cat $ORDS_HOME/secrets/$ORACLE_PWD_KEY` + CDBADMIN_USER=`cat $ORDS_HOME/secrets/$CDBADMIN_USER_KEY` + CDBADMIN_PWD=`cat $ORDS_HOME/secrets/$CDBADMIN_PWD_KEY` + + # Make standalone dir + mkdir -p $ORDS_HOME/config/ords/standalone + + # Copy template files + cp $ORDS_HOME/$CONFIG_PROPS $ORDS_HOME/params/ords_params.properties + cp $ORDS_HOME/$STANDALONE_PROPS $ORDS_HOME/config/ords/standalone/standalone.properties + cp $ORDS_HOME/$CDBADMIN_PROPS $ORDS_HOME/cdbadmin.properties + + # Replace DB related variables (ords_params.properties) + sed -i -e "s|###ORACLE_SERVICE###|$ORACLE_SERVICE|g" $ORDS_HOME/params/ords_params.properties + sed -i -e "s|###ORACLE_HOST###|$ORACLE_HOST|g" $ORDS_HOME/params/ords_params.properties + sed -i -e "s|###ORACLE_PORT###|$ORACLE_PORT|g" $ORDS_HOME/params/ords_params.properties + sed -i -e "s|###ORDS_PWD###|$ORDS_PWD|g" $ORDS_HOME/params/ords_params.properties + sed -i -e "s|###ORACLE_PWD###|$ORACLE_PWD|g" $ORDS_HOME/params/ords_params.properties + + # Replace standalone runtime variables (standalone.properties) + sed -i -e "s|###PORT###|$ORDS_PORT|g" $ORDS_HOME/config/ords/standalone/standalone.properties + sed -i -e "s|###DOC_ROOT###|$ORDS_HOME/doc_root|g" $ORDS_HOME/config/ords/standalone/standalone.properties + sed -i -e "s|###APEXI###|$APEXI|g" $ORDS_HOME/config/ords/standalone/standalone.properties + + # Replace CDB Admin runtime variables (cdbadmin.properties) + sed -i -e "s|###CDBADMIN_USER###|$CDBADMIN_USER|g" $ORDS_HOME/cdbadmin.properties + sed -i -e "s|###CDBADMIN_PWD###|$CDBADMIN_PWD|g" $ORDS_HOME/cdbadmin.properties + + # Start ORDS setup + java -jar $ORDS_HOME/ords.war install simple + + # Setup Web Server User + $ORDS_HOME/$SETUP_WEBUSER +} + +############# MAIN ################ + +# Check whether ords is already setup +if [ ! -f $ORDS_HOME/config/ords/standalone/standalone.properties ]; then + setupOrds; + java -jar $ORDS_HOME/ords.war set-properties --conf apex_pu $ORDS_HOME/cdbadmin.properties +fi; + +java -jar $ORDS_HOME/ords.war standalone diff --git a/ords/setupwebuser.sh b/ords/setupwebuser.sh new file mode 100644 index 00000000..c90deeda --- /dev/null +++ b/ords/setupwebuser.sh @@ -0,0 +1,20 @@ +#!/bin/bash +# Copyright (c) 2021, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. + +WEBSERVER_USER=`cat $ORDS_HOME/secrets/$WEBSERVER_USER_KEY` +WEBSERVER_PASSWORD=`cat $ORDS_HOME/secrets/$WEBSERVER_PASSWORD_KEY` + +export WEBSERVER_USER +export WEBSERVER_PASSWORD + +/usr/bin/expect -c ' + +spawn java -jar $env(ORDS_HOME)/ords.war user $env(WEBSERVER_USER) "SQL Administrator" +expect "Enter a password for user" +send "$env(WEBSERVER_PASSWORD)\n" +expect "Confirm password for user" +send "$env(WEBSERVER_PASSWORD)\n" +expect "Created user" + +' \ No newline at end of file diff --git a/ords/standalone.properties.tmpl b/ords/standalone.properties.tmpl new file mode 100644 index 00000000..689c62ae --- /dev/null +++ b/ords/standalone.properties.tmpl @@ -0,0 +1,8 @@ +jetty.port=###PORT### +standalone.access.log=/tmp/ords_log +standalone.context.path=/ords +standalone.doc.root=###DOC_ROOT### +standalone.scheme.do.not.prompt=true +standalone.static.context.path=/i +standalone.static.do.not.prompt=true +standalone.static.path=###APEXI### From 7e016b622f1a757fd64c9a4d959089b39108b0cc Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Thu, 25 Nov 2021 13:02:48 +0530 Subject: [PATCH 050/628] Added support for prebuilt db --- .../v1alpha1/singleinstancedatabase_types.go | 2 +- .../singleinstancedatabase_webhook.go | 12 ++++ ...se.oracle.com_singleinstancedatabases.yaml | 1 - config/manager/kustomization.yaml | 2 +- .../sidb/singleinstancedatabase_prov.yaml | 7 +- ...ngleinstancedatabase_prov_prebuilt_db.yaml | 16 +++++ .../singleinstancedatabase_controller.go | 65 +++++++++++++++---- oracle-database-operator.yaml | 3 +- 8 files changed, 87 insertions(+), 21 deletions(-) create mode 100644 config/samples/sidb/singleinstancedatabase_prov_prebuilt_db.yaml diff --git a/apis/database/v1alpha1/singleinstancedatabase_types.go b/apis/database/v1alpha1/singleinstancedatabase_types.go index 7afb7ae6..6787bccf 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_types.go +++ b/apis/database/v1alpha1/singleinstancedatabase_types.go @@ -75,7 +75,7 @@ type SingleInstanceDatabaseSpec struct { NodeSelector map[string]string `json:"nodeSelector,omitempty"` AdminPassword SingleInstanceDatabaseAdminPassword `json:"adminPassword"` Image SingleInstanceDatabaseImage `json:"image"` - Persistence SingleInstanceDatabasePersistence `json:"persistence"` + Persistence SingleInstanceDatabasePersistence `json:"persistence,omitempty"` InitParams SingleInstanceDatabaseInitParams `json:"initParams,omitempty"` } diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index 1f7b44bb..19173ffc 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -74,6 +74,12 @@ func (r *SingleInstanceDatabase) Default() { if r.Spec.Edition == "express" { r.Spec.Replicas = 1 } + + // Pre-built db should have 1 replica only + if r.Spec.Persistence.AccessMode == "" { + r.Spec.Replicas = 1 + } + // TODO(user): fill in your defaulting logic. } @@ -107,6 +113,12 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { field.Invalid(field.NewPath("spec").Child("pdbName"), r.Spec.Pdbname, "Express edition PDB must be XEPDB1")) } + //Edition must be passed when cloning from a source database other than same k8s cluster + if strings.Contains(r.Spec.CloneFrom, ":") && strings.Contains(r.Spec.CloneFrom, "/") && r.Spec.Edition == "" { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("edition"), r.Spec.CloneFrom, + "Edition must be passed when cloning from a source database other than same k8s cluster")) + } if len(allErrs) == 0 { return nil diff --git a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml index d7abe5e0..456ec309 100644 --- a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml +++ b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml @@ -156,7 +156,6 @@ spec: required: - adminPassword - image - - persistence - replicas type: object status: diff --git a/config/manager/kustomization.yaml b/config/manager/kustomization.yaml index 6022352b..ce2bd741 100644 --- a/config/manager/kustomization.yaml +++ b/config/manager/kustomization.yaml @@ -9,4 +9,4 @@ kind: Kustomization images: - name: controller newName: phx.ocir.io/oracassandra/cloneonprem - newTag: test-0.1.1 + newTag: test-0.1.2 diff --git a/config/samples/sidb/singleinstancedatabase_prov.yaml b/config/samples/sidb/singleinstancedatabase_prov.yaml index 7123b117..8d691fee 100644 --- a/config/samples/sidb/singleinstancedatabase_prov.yaml +++ b/config/samples/sidb/singleinstancedatabase_prov.yaml @@ -13,10 +13,11 @@ spec: sid: ORCL1 ## Secret containing SIDB password mapped to secretKey + ## This secret will be deleted after creation of the database unless keepSecret is set to true adminPassword: - secret: - secretName: - key: + secretName: + secretKey: + keepSecret: false ## Database image details image: diff --git a/config/samples/sidb/singleinstancedatabase_prov_prebuilt_db.yaml b/config/samples/sidb/singleinstancedatabase_prov_prebuilt_db.yaml new file mode 100644 index 00000000..cb856004 --- /dev/null +++ b/config/samples/sidb/singleinstancedatabase_prov_prebuilt_db.yaml @@ -0,0 +1,16 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: SingleInstanceDatabase +metadata: + name: singleinstancedatabase-sample + namespace: default +spec: + + ## Database image details + image: + version: + pullFrom: + pullSecrets: diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index f1d6b33f..890ef66f 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -446,16 +446,26 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, }, Spec: corev1.PodSpec{ - Volumes: []corev1.Volume{{ - Name: "datamount", - VolumeSource: corev1.VolumeSource{ - PersistentVolumeClaim: &corev1.PersistentVolumeClaimVolumeSource{ - ClaimName: m.Name, - ReadOnly: false, + Volumes: func() []corev1.Volume { + // No PVC for Pre-built db + if m.Spec.Persistence.AccessMode == "" { + return []corev1.Volume{{}} + } + return []corev1.Volume{{ + Name: "datamount", + VolumeSource: corev1.VolumeSource{ + PersistentVolumeClaim: &corev1.PersistentVolumeClaimVolumeSource{ + ClaimName: m.Name, + ReadOnly: false, + }, }, - }, - }}, + }} + }(), InitContainers: func() []corev1.Container { + // No InitContainer needed for Pre-built db + if m.Spec.Persistence.AccessMode == "" { + return []corev1.Container{{}} + } if m.Spec.Edition != "express" { return []corev1.Container{{ Name: "init-permissions", @@ -496,7 +506,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns } } else { if !dbcommons.IsSourceDatabaseOnCluster(m.Spec.CloneFrom) { - edition = "" + edition = m.Spec.Edition } else { edition = n.Spec.Edition if n.Spec.Edition == "" { @@ -559,11 +569,36 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }(), }, - VolumeMounts: []corev1.VolumeMount{{ - MountPath: "/opt/oracle/oradata", - Name: "datamount", - }}, + VolumeMounts: func() []corev1.VolumeMount { + if m.Spec.Persistence.AccessMode == "" { + return []corev1.VolumeMount{{}} + } + return []corev1.VolumeMount{{ + MountPath: "/opt/oracle/oradata", + Name: "datamount", + }} + }(), + // VolumeMounts: []corev1.VolumeMount{{ + // MountPath: "/opt/oracle/oradata", + // Name: "datamount", + // }}, Env: func() []corev1.EnvVar { + if m.Spec.Persistence.AccessMode != "" { + return []corev1.EnvVar{ + { + Name: "SVC_HOST", + Value: m.Name, + }, + { + Name: "SVC_PORT", + Value: "1521", + }, + { + Name: "SKIP_DATAPATCH", + Value: "true", + }, + } + } if m.Spec.CloneFrom == "" { return []corev1.EnvVar{ { @@ -800,6 +835,10 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePVCSpec(m *dbapi.SingleIns func (r *SingleInstanceDatabaseReconciler) createOrReplacePVC(ctx context.Context, req ctrl.Request, m *dbapi.SingleInstanceDatabase) (ctrl.Result, error) { + // No PVC for Pre-built db + if m.Spec.Persistence.AccessMode == "" { + return requeueN, nil + } log := r.Log.WithValues("createPVC", req.NamespacedName) pvcDeleted := false diff --git a/oracle-database-operator.yaml b/oracle-database-operator.yaml index 65e704a3..b547f929 100644 --- a/oracle-database-operator.yaml +++ b/oracle-database-operator.yaml @@ -897,7 +897,6 @@ spec: required: - adminPassword - image - - persistence - replicas type: object status: @@ -1412,7 +1411,7 @@ spec: - --enable-leader-election command: - /manager - image: phx.ocir.io/oracassandra/cloneonprem:test-0.1.1 + image: phx.ocir.io/oracassandra/cloneonprem:test-0.1.2 imagePullPolicy: Always name: manager ports: From 942882d2b8a399f8dbdb4d1ebd6496fdf5870f48 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Thu, 25 Nov 2021 15:54:36 +0530 Subject: [PATCH 051/628] Add gitlab for CI/CD --- .gitlab-ci.yml | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) create mode 100644 .gitlab-ci.yml diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml new file mode 100644 index 00000000..e4d80bb9 --- /dev/null +++ b/.gitlab-ci.yml @@ -0,0 +1,27 @@ +build-operator: + stage: build + variables: + IMAGE: "$DOCKER_REPO:$CI_COMMIT_BRANCH" + OP_YAML: oracle-database-operator.yaml + script: + - make docker-build IMG="$IMAGE" + - docker push "$IMAGE" + - newimage=$(docker inspect $IMAGE | python -c 'import json,sys; print json.load(sys.stdin)[0]["RepoDigests"][0]') + - echo $newimage + - docker rmi "$IMAGE" && docker system prune -f + - make operator-yaml IMG=$newimage + - curl -s -n $ARTIFACTORY_REPO/$CI_COMMIT_BRANCH/$OP_YAML -T ./$OP_YAML + only: + variables: + - $CI_COMMIT_MESSAGE =~ /\#run-pipeline/ + - $CI_COMMIT_BRANCH =~ /master/ + - $CI_MERGE_REQUEST_ID != "" + except: + variables: + - $CI_COMMIT_MESSAGE =~ /\#skip-pipeline/ + +cleanup: + stage: .post + script: + - echo "Clean up downloaded binaries" + - rm -rf bin/ From e573b741e27222fcc7424df4ead0a425262e3942 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Thu, 25 Nov 2021 15:54:36 +0530 Subject: [PATCH 052/628] Add gitlab for CI/CD --- .gitlab-ci.yml | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) create mode 100644 .gitlab-ci.yml diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml new file mode 100644 index 00000000..e4d80bb9 --- /dev/null +++ b/.gitlab-ci.yml @@ -0,0 +1,27 @@ +build-operator: + stage: build + variables: + IMAGE: "$DOCKER_REPO:$CI_COMMIT_BRANCH" + OP_YAML: oracle-database-operator.yaml + script: + - make docker-build IMG="$IMAGE" + - docker push "$IMAGE" + - newimage=$(docker inspect $IMAGE | python -c 'import json,sys; print json.load(sys.stdin)[0]["RepoDigests"][0]') + - echo $newimage + - docker rmi "$IMAGE" && docker system prune -f + - make operator-yaml IMG=$newimage + - curl -s -n $ARTIFACTORY_REPO/$CI_COMMIT_BRANCH/$OP_YAML -T ./$OP_YAML + only: + variables: + - $CI_COMMIT_MESSAGE =~ /\#run-pipeline/ + - $CI_COMMIT_BRANCH =~ /master/ + - $CI_MERGE_REQUEST_ID != "" + except: + variables: + - $CI_COMMIT_MESSAGE =~ /\#skip-pipeline/ + +cleanup: + stage: .post + script: + - echo "Clean up downloaded binaries" + - rm -rf bin/ From 16f332b9b56e15d6aad3d494dd26f036e500d704 Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Fri, 26 Nov 2021 11:14:36 +0530 Subject: [PATCH 053/628] Changed podspec structure --- .../v1alpha1/singleinstancedatabase_types.go | 5 +- ...se.oracle.com_singleinstancedatabases.yaml | 9 - .../sidb/singleinstancedatabase_prov.yaml | 3 +- ...ngleinstancedatabase_prov_prebuilt_db.yaml | 3 +- .../singleinstancedatabase_controller.go | 200 +++++++++++++----- oracle-database-operator.yaml | 9 - 6 files changed, 155 insertions(+), 74 deletions(-) diff --git a/apis/database/v1alpha1/singleinstancedatabase_types.go b/apis/database/v1alpha1/singleinstancedatabase_types.go index 6787bccf..f14bf8de 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_types.go +++ b/apis/database/v1alpha1/singleinstancedatabase_types.go @@ -73,7 +73,7 @@ type SingleInstanceDatabaseSpec struct { Replicas int `json:"replicas"` NodeSelector map[string]string `json:"nodeSelector,omitempty"` - AdminPassword SingleInstanceDatabaseAdminPassword `json:"adminPassword"` + AdminPassword SingleInstanceDatabaseAdminPassword `json:"adminPassword,omitempty"` Image SingleInstanceDatabaseImage `json:"image"` Persistence SingleInstanceDatabasePersistence `json:"persistence,omitempty"` InitParams SingleInstanceDatabaseInitParams `json:"initParams,omitempty"` @@ -84,8 +84,7 @@ type SingleInstanceDatabasePersistence struct { Size string `json:"size"` StorageClass string `json:"storageClass"` - // +kubebuilder:validation:Enum=ReadWriteOnce;ReadWriteMany - AccessMode string `json:"accessMode"` + AccessMode string `json:"accessMode,omitempty"` } // SingleInstanceDatabaseInitParams defines the Init Parameters diff --git a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml index 456ec309..28c77806 100644 --- a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml +++ b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml @@ -130,16 +130,12 @@ spec: size and class for PVC properties: accessMode: - enum: - - ReadWriteOnce - - ReadWriteMany type: string size: type: string storageClass: type: string required: - - accessMode - size - storageClass type: object @@ -154,7 +150,6 @@ spec: pattern: ^[a-zA-Z0-9]+$ type: string required: - - adminPassword - image - replicas type: object @@ -289,16 +284,12 @@ spec: size and class for PVC properties: accessMode: - enum: - - ReadWriteOnce - - ReadWriteMany type: string size: type: string storageClass: type: string required: - - accessMode - size - storageClass type: object diff --git a/config/samples/sidb/singleinstancedatabase_prov.yaml b/config/samples/sidb/singleinstancedatabase_prov.yaml index 8d691fee..b1a83e76 100644 --- a/config/samples/sidb/singleinstancedatabase_prov.yaml +++ b/config/samples/sidb/singleinstancedatabase_prov.yaml @@ -19,9 +19,8 @@ spec: secretKey: keepSecret: false - ## Database image details + ## Database Image image: - version: pullFrom: pullSecrets: diff --git a/config/samples/sidb/singleinstancedatabase_prov_prebuilt_db.yaml b/config/samples/sidb/singleinstancedatabase_prov_prebuilt_db.yaml index cb856004..3e502d84 100644 --- a/config/samples/sidb/singleinstancedatabase_prov_prebuilt_db.yaml +++ b/config/samples/sidb/singleinstancedatabase_prov_prebuilt_db.yaml @@ -9,8 +9,7 @@ metadata: namespace: default spec: - ## Database image details + ## Database Image image: - version: pullFrom: pullSecrets: diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 890ef66f..c93976f4 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -356,8 +356,8 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab return requeueN, err } - // Validating the secret - if m.Status.DatafilesCreated != "true" { + // Validating the secret. Pre-built db doesnt need secret + if m.Spec.Persistence.AccessMode != "" && m.Status.DatafilesCreated != "true" { secret := &corev1.Secret{} err = r.Get(ctx, types.NamespacedName{Name: m.Spec.AdminPassword.SecretName, Namespace: m.Namespace}, secret) if err != nil { @@ -432,7 +432,108 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab // Instantiate POD spec from SingleInstanceDatabase spec //############################################################################# func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleInstanceDatabase, n *dbapi.SingleInstanceDatabase) *corev1.Pod { + // Pre-built db + if m.Spec.Persistence.AccessMode == "" { + pod := &corev1.Pod{ + TypeMeta: metav1.TypeMeta{ + Kind: "Pod", + }, + ObjectMeta: metav1.ObjectMeta{ + Name: m.Name + "-" + dbcommons.GenerateRandomString(5), + Namespace: m.Namespace, + Labels: map[string]string{ + "app": m.Name, + "version": m.Spec.Image.Version, + }, + }, + Spec: corev1.PodSpec{ + Containers: []corev1.Container{{ + Name: m.Name, + Image: m.Spec.Image.PullFrom, + Lifecycle: &corev1.Lifecycle{ + PreStop: &corev1.Handler{ + Exec: &corev1.ExecAction{ + Command: []string{"/bin/sh", "-c", "/bin/echo -en 'shutdown abort;\n' | env ORACLE_SID=${ORACLE_SID^^} sqlplus -S / as sysdba"}, + }, + }, + }, + ImagePullPolicy: corev1.PullAlways, + Ports: []corev1.ContainerPort{{ContainerPort: 1521}, {ContainerPort: 5500}}, + ReadinessProbe: &corev1.Probe{ + Handler: corev1.Handler{ + Exec: &corev1.ExecAction{ + Command: []string{"/bin/sh", "-c", "if [ -f $ORACLE_BASE/checkDBLockStatus.sh ]; then $ORACLE_BASE/checkDBLockStatus.sh ; else $ORACLE_BASE/checkDBStatus.sh; fi "}, + }, + }, + InitialDelaySeconds: 20, + TimeoutSeconds: 20, + PeriodSeconds: func() int32 { + if m.Spec.ReadinessCheckPeriod > 0 { + return int32(m.Spec.ReadinessCheckPeriod) + } + return 30 + }(), + }, + Env: func() []corev1.EnvVar { + return []corev1.EnvVar{ + { + Name: "SVC_HOST", + Value: m.Name, + }, + { + Name: "SVC_PORT", + Value: "1521", + }, + { + Name: "SKIP_DATAPATCH", + Value: "true", + }, + } + }(), + }}, + + TerminationGracePeriodSeconds: func() *int64 { i := int64(30); return &i }(), + + NodeSelector: func() map[string]string { + ns := make(map[string]string) + if len(m.Spec.NodeSelector) != 0 { + for key, value := range m.Spec.NodeSelector { + ns[key] = value + } + } + return ns + }(), + + SecurityContext: &corev1.PodSecurityContext{ + RunAsUser: func() *int64 { + i := int64(0) + if m.Spec.Edition != "express" { + i = int64(dbcommons.ORACLE_UID) + } + return &i + }(), + RunAsGroup: func() *int64 { + i := int64(0) + if m.Spec.Edition != "express" { + i = int64(dbcommons.ORACLE_GUID) + } + return &i + }(), + }, + ImagePullSecrets: []corev1.LocalObjectReference{ + { + Name: m.Spec.Image.PullSecrets, + }, + }, + }, + } + + // Set SingleInstanceDatabase instance as the owner and controller + ctrl.SetControllerReference(m, pod, r.Scheme) + return pod + + } pod := &corev1.Pod{ TypeMeta: metav1.TypeMeta{ Kind: "Pod", @@ -446,26 +547,16 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, }, Spec: corev1.PodSpec{ - Volumes: func() []corev1.Volume { - // No PVC for Pre-built db - if m.Spec.Persistence.AccessMode == "" { - return []corev1.Volume{{}} - } - return []corev1.Volume{{ - Name: "datamount", - VolumeSource: corev1.VolumeSource{ - PersistentVolumeClaim: &corev1.PersistentVolumeClaimVolumeSource{ - ClaimName: m.Name, - ReadOnly: false, - }, + Volumes: []corev1.Volume{{ + Name: "datamount", + VolumeSource: corev1.VolumeSource{ + PersistentVolumeClaim: &corev1.PersistentVolumeClaimVolumeSource{ + ClaimName: m.Name, + ReadOnly: false, }, - }} - }(), + }, + }}, InitContainers: func() []corev1.Container { - // No InitContainer needed for Pre-built db - if m.Spec.Persistence.AccessMode == "" { - return []corev1.Container{{}} - } if m.Spec.Edition != "express" { return []corev1.Container{{ Name: "init-permissions", @@ -569,36 +660,11 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }(), }, - VolumeMounts: func() []corev1.VolumeMount { - if m.Spec.Persistence.AccessMode == "" { - return []corev1.VolumeMount{{}} - } - return []corev1.VolumeMount{{ - MountPath: "/opt/oracle/oradata", - Name: "datamount", - }} - }(), - // VolumeMounts: []corev1.VolumeMount{{ - // MountPath: "/opt/oracle/oradata", - // Name: "datamount", - // }}, + VolumeMounts: []corev1.VolumeMount{{ + MountPath: "/opt/oracle/oradata", + Name: "datamount", + }}, Env: func() []corev1.EnvVar { - if m.Spec.Persistence.AccessMode != "" { - return []corev1.EnvVar{ - { - Name: "SVC_HOST", - Value: m.Name, - }, - { - Name: "SVC_PORT", - Value: "1521", - }, - { - Name: "SKIP_DATAPATCH", - Value: "true", - }, - } - } if m.Spec.CloneFrom == "" { return []corev1.EnvVar{ { @@ -747,6 +813,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns // Set SingleInstanceDatabase instance as the owner and controller ctrl.SetControllerReference(m, pod, r.Scheme) return pod + } //############################################################################# @@ -1073,6 +1140,11 @@ func (r *SingleInstanceDatabaseReconciler) createWallet(m *dbapi.SingleInstanceD return requeueN, nil } + // No Wallet for Pre-built db + if m.Spec.Persistence.AccessMode == "" { + return requeueN, nil + } + // Listing all the pods readyPod, _, availableFinal, _, err := dbcommons.FindPods(r, m.Spec.Image.Version, m.Spec.Image.PullFrom, m.Name, m.Namespace, ctx, req) @@ -1390,6 +1462,11 @@ func (r *SingleInstanceDatabaseReconciler) deleteWallet(m *dbapi.SingleInstanceD return requeueN, nil } + // No Wallet for Pre-built db + if m.Spec.Persistence.AccessMode == "" { + return requeueN, nil + } + // Deleting the secret and then deleting the wallet // If the secret is not found it means that the secret and wallet both are deleted, hence no need to requeue if !m.Spec.AdminPassword.KeepSecret { @@ -1434,6 +1511,11 @@ func (r *SingleInstanceDatabaseReconciler) runDatapatch(m *dbapi.SingleInstanceD return requeueN, nil } + // No Patching for Pre-built db + if m.Spec.Persistence.AccessMode == "" { + return requeueN, nil + } + m.Status.Status = dbcommons.StatusPatching r.Status().Update(ctx, m) eventReason := "Datapatch Executing" @@ -1484,6 +1566,11 @@ func (r *SingleInstanceDatabaseReconciler) updateInitParameters(m *dbapi.SingleI readyPod corev1.Pod, ctx context.Context, req ctrl.Request) (ctrl.Result, error) { log := r.Log.WithValues("updateInitParameters", req.NamespacedName) + // No InitParameters Updation for Pre-built db + if m.Spec.Persistence.AccessMode == "" { + return requeueN, nil + } + if m.Status.InitParams == m.Spec.InitParams { return requeueN, nil } @@ -1529,6 +1616,11 @@ func (r *SingleInstanceDatabaseReconciler) updateDBConfig(m *dbapi.SingleInstanc log := r.Log.WithValues("updateDBConfig", req.NamespacedName) + // No updateDBConfig for Pre-built db + if m.Spec.Persistence.AccessMode == "" { + return requeueN, nil + } + m.Status.Status = dbcommons.StatusUpdating r.Status().Update(ctx, m) var forceLoggingStatus bool @@ -1722,6 +1814,11 @@ func (r *SingleInstanceDatabaseReconciler) installApex(m *dbapi.SingleInstanceDa readyPod corev1.Pod, ctx context.Context, req ctrl.Request) ctrl.Result { log := r.Log.WithValues("installApex", req.NamespacedName) + // No APEX for Pre-built db + if m.Spec.Persistence.AccessMode == "" { + return requeueN + } + if !m.Spec.InstallApex || m.Status.ApexInstalled { return requeueN } @@ -1802,6 +1899,11 @@ func (r *SingleInstanceDatabaseReconciler) uninstallApex(m *dbapi.SingleInstance readyPod corev1.Pod, ctx context.Context, req ctrl.Request) ctrl.Result { log := r.Log.WithValues("uninstallApex", req.NamespacedName) + // No APEX for Pre-built db + if m.Spec.Persistence.AccessMode == "" { + return requeueN + } + if m.Spec.InstallApex || !m.Status.ApexInstalled { return requeueN } diff --git a/oracle-database-operator.yaml b/oracle-database-operator.yaml index b547f929..4010aa78 100644 --- a/oracle-database-operator.yaml +++ b/oracle-database-operator.yaml @@ -872,16 +872,12 @@ spec: description: SingleInstanceDatabasePersistence defines the storage size and class for PVC properties: accessMode: - enum: - - ReadWriteOnce - - ReadWriteMany type: string size: type: string storageClass: type: string required: - - accessMode - size - storageClass type: object @@ -895,7 +891,6 @@ spec: pattern: ^[a-zA-Z0-9]+$ type: string required: - - adminPassword - image - replicas type: object @@ -1002,16 +997,12 @@ spec: description: SingleInstanceDatabasePersistence defines the storage size and class for PVC properties: accessMode: - enum: - - ReadWriteOnce - - ReadWriteMany type: string size: type: string storageClass: type: string required: - - accessMode - size - storageClass type: object From eccadda50470f0f063b4722014891262d4f392eb Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Thu, 2 Dec 2021 12:49:05 -0500 Subject: [PATCH 054/628] update oci-go-sdk v51 --- .../v1alpha1/zz_generated.deepcopy.go | 21 ++++++++++++------- commons/autonomousdatabase/reconciler_util.go | 6 +++--- commons/oci/database.go | 8 +++---- commons/oci/provider.go | 4 ++-- commons/oci/vault.go | 4 ++-- commons/oci/wallet.go | 6 +++--- commons/sharding/scommon.go | 4 ++-- .../database/autonomousdatabase_controller.go | 6 +++--- .../database/shardingdatabase_controller.go | 4 ++-- go.mod | 2 +- go.sum | 6 ++++-- ...autonomousdatabase_controller_bind_test.go | 6 +++--- ...tonomousdatabase_controller_create_test.go | 4 ++-- test/e2e/behavior/shared_behaviors.go | 4 ++-- test/e2e/suite_test.go | 4 ++-- test/e2e/util/oci_config_util.go | 2 +- test/e2e/util/oci_db_request.go | 4 ++-- test/e2e/util/oci_vault_request.go | 6 +++--- test/e2e/util/oci_work_request.go | 4 ++-- 19 files changed, 56 insertions(+), 49 deletions(-) diff --git a/apis/database/v1alpha1/zz_generated.deepcopy.go b/apis/database/v1alpha1/zz_generated.deepcopy.go index 2766d559..4d27bb59 100644 --- a/apis/database/v1alpha1/zz_generated.deepcopy.go +++ b/apis/database/v1alpha1/zz_generated.deepcopy.go @@ -134,18 +134,23 @@ func (in *AutonomousDatabaseDetails) DeepCopyInto(out *AutonomousDatabaseDetails *out = make([]string, len(*in)) copy(*out, *in) } - if in.PrivateEndpoint != nil { - in, out := &in.PrivateEndpoint, &out.PrivateEndpoint - *out = new(string) + if in.IsAccessControlEnabled != nil { + in, out := &in.IsAccessControlEnabled, &out.IsAccessControlEnabled + *out = new(bool) **out = **in } - if in.PrivateEndpointLabel != nil { - in, out := &in.PrivateEndpointLabel, &out.PrivateEndpointLabel - *out = new(string) + if in.WhitelistedIPs != nil { + in, out := &in.WhitelistedIPs, &out.WhitelistedIPs + *out = make([]string, len(*in)) + copy(*out, *in) + } + if in.IsMTLSConnectionRequired != nil { + in, out := &in.IsMTLSConnectionRequired, &out.IsMTLSConnectionRequired + *out = new(bool) **out = **in } - if in.PrivateEndpointIP != nil { - in, out := &in.PrivateEndpointIP, &out.PrivateEndpointIP + if in.PrivateEndpointLabel != nil { + in, out := &in.PrivateEndpointLabel, &out.PrivateEndpointLabel *out = new(string) **out = **in } diff --git a/commons/autonomousdatabase/reconciler_util.go b/commons/autonomousdatabase/reconciler_util.go index 2ceac5e6..2f820410 100644 --- a/commons/autonomousdatabase/reconciler_util.go +++ b/commons/autonomousdatabase/reconciler_util.go @@ -49,9 +49,9 @@ import ( "sigs.k8s.io/controller-runtime/pkg/client" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/database" - "github.com/oracle/oci-go-sdk/v45/secrets" + "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v51/database" + "github.com/oracle/oci-go-sdk/v51/secrets" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" "github.com/oracle/oracle-database-operator/commons/oci" diff --git a/commons/oci/database.go b/commons/oci/database.go index 985d299b..c1a8cad0 100644 --- a/commons/oci/database.go +++ b/commons/oci/database.go @@ -46,10 +46,10 @@ import ( "time" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/database" - "github.com/oracle/oci-go-sdk/v45/secrets" - "github.com/oracle/oci-go-sdk/v45/workrequests" + "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v51/database" + "github.com/oracle/oci-go-sdk/v51/secrets" + "github.com/oracle/oci-go-sdk/v51/workrequests" "sigs.k8s.io/controller-runtime/pkg/client" corev1 "k8s.io/api/core/v1" diff --git a/commons/oci/provider.go b/commons/oci/provider.go index f5bd6cda..32188e81 100644 --- a/commons/oci/provider.go +++ b/commons/oci/provider.go @@ -42,8 +42,8 @@ import ( "context" "errors" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/common/auth" + "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v51/common/auth" "sigs.k8s.io/controller-runtime/pkg/client" diff --git a/commons/oci/vault.go b/commons/oci/vault.go index 6bf790aa..5df1cc04 100644 --- a/commons/oci/vault.go +++ b/commons/oci/vault.go @@ -42,8 +42,8 @@ import ( "context" "encoding/base64" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/secrets" + "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v51/secrets" ) func getValueFromVaultSecret(secretClient secrets.SecretsClient, vaultSecretOCID string) (string, error) { diff --git a/commons/oci/wallet.go b/commons/oci/wallet.go index 3faa4f16..268e66c5 100644 --- a/commons/oci/wallet.go +++ b/commons/oci/wallet.go @@ -49,9 +49,9 @@ import ( "time" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/database" - "github.com/oracle/oci-go-sdk/v45/secrets" + "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v51/database" + "github.com/oracle/oci-go-sdk/v51/secrets" "sigs.k8s.io/controller-runtime/pkg/client" "k8s.io/apimachinery/pkg/types" diff --git a/commons/sharding/scommon.go b/commons/sharding/scommon.go index 55fdd5dc..5af51390 100644 --- a/commons/sharding/scommon.go +++ b/commons/sharding/scommon.go @@ -48,8 +48,8 @@ import ( "strings" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/ons" + "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v51/ons" appsv1 "k8s.io/api/apps/v1" corev1 "k8s.io/api/core/v1" "k8s.io/apimachinery/pkg/api/errors" diff --git a/controllers/database/autonomousdatabase_controller.go b/controllers/database/autonomousdatabase_controller.go index 2f250489..5eba7ae7 100644 --- a/controllers/database/autonomousdatabase_controller.go +++ b/controllers/database/autonomousdatabase_controller.go @@ -44,9 +44,9 @@ import ( "reflect" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v45/database" - "github.com/oracle/oci-go-sdk/v45/secrets" - "github.com/oracle/oci-go-sdk/v45/workrequests" + "github.com/oracle/oci-go-sdk/v51/database" + "github.com/oracle/oci-go-sdk/v51/secrets" + "github.com/oracle/oci-go-sdk/v51/workrequests" apiErrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/runtime" diff --git a/controllers/database/shardingdatabase_controller.go b/controllers/database/shardingdatabase_controller.go index 73575df7..f30e714b 100644 --- a/controllers/database/shardingdatabase_controller.go +++ b/controllers/database/shardingdatabase_controller.go @@ -47,8 +47,8 @@ import ( "time" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/ons" + "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v51/ons" appsv1 "k8s.io/api/apps/v1" corev1 "k8s.io/api/core/v1" "k8s.io/apimachinery/pkg/api/errors" diff --git a/go.mod b/go.mod index fada426a..4e19053e 100644 --- a/go.mod +++ b/go.mod @@ -6,7 +6,7 @@ require ( github.com/go-logr/logr v0.4.0 github.com/onsi/ginkgo v1.16.4 github.com/onsi/gomega v1.13.0 - github.com/oracle/oci-go-sdk/v45 v45.2.0 + github.com/oracle/oci-go-sdk/v51 v51.0.0 gopkg.in/yaml.v2 v2.4.0 k8s.io/api v0.21.2 k8s.io/apimachinery v0.21.2 diff --git a/go.sum b/go.sum index 67d9f3df..2b051eef 100644 --- a/go.sum +++ b/go.sum @@ -304,8 +304,8 @@ github.com/onsi/gomega v1.7.1/go.mod h1:XdKZgCCFLUoM/7CFJVPcG8C1xQ1AJ0vpAezJrB7J github.com/onsi/gomega v1.10.1/go.mod h1:iN09h71vgCQne3DLsj+A5owkum+a2tYe+TOCB1ybHNo= github.com/onsi/gomega v1.13.0 h1:7lLHu94wT9Ij0o6EWWclhu0aOh32VxhkwEJvzuWPeak= github.com/onsi/gomega v1.13.0/go.mod h1:lRk9szgn8TxENtWd0Tp4c3wjlRfMTMH27I+3Je41yGY= -github.com/oracle/oci-go-sdk/v45 v45.2.0 h1:vCPoQlE+DOrM2heJn66rvPU6fbsc/0Cxtzs2jnFut6U= -github.com/oracle/oci-go-sdk/v45 v45.2.0/go.mod h1:ZM6LGiRO5TPQJxTlrXbcHMbClE775wnGD5U/EerCsRw= +github.com/oracle/oci-go-sdk/v51 v51.0.0 h1:eDUVMsAzvf+jfq4xbtpQrxznYNjccwKKyIhmRvZLgwI= +github.com/oracle/oci-go-sdk/v51 v51.0.0/go.mod h1:d9KSNXwE64drofxoor+y/JWofJqLqRF9D1/AtfYIE10= github.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc= github.com/pelletier/go-toml v1.2.0/go.mod h1:5z9KED0ma1S8pY6P1sdut58dfprrGBbd/94hg7ilaic= github.com/peterbourgon/diskv v2.0.1+incompatible/go.mod h1:uqqh8zWWbv1HBMNONnaR/tNboyR3/BZd58JJSHlUSCU= @@ -355,6 +355,8 @@ github.com/sirupsen/logrus v1.7.0/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1:OnSkiWE9lh6wB0YB77sQom3nweQdgAjqCqsofrRNTgc= github.com/smartystreets/goconvey v1.6.4/go.mod h1:syvi0/a8iFYH4r/RixwvyeAJjdLS9QV7WQ/tjFTllLA= github.com/soheilhy/cmux v0.1.4/go.mod h1:IM3LyeVVIOuxMH7sFAkER9+bJ4dT7Ms6E4xg4kGIyLM= +github.com/sony/gobreaker v0.4.2-0.20210216022020-dd874f9dd33b h1:br+bPNZsJWKicw/5rALEo67QHs5weyD5tf8WST+4sJ0= +github.com/sony/gobreaker v0.4.2-0.20210216022020-dd874f9dd33b/go.mod h1:ZKptC7FHNvhBz7dN2LGjPVBz2sZJmc0/PkyDJOjmxWY= github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA= github.com/spf13/afero v1.1.2/go.mod h1:j4pytiNVoe2o6bmDsKpLACNPDBIoEAkihy7loJ1B0CQ= github.com/spf13/afero v1.2.2/go.mod h1:9ZxEEn6pIJ8Rxe320qSDBk6AsU0r9pR7Q4OcevTdifk= diff --git a/test/e2e/autonomousdatabase_controller_bind_test.go b/test/e2e/autonomousdatabase_controller_bind_test.go index ff415da9..3ab132c9 100644 --- a/test/e2e/autonomousdatabase_controller_bind_test.go +++ b/test/e2e/autonomousdatabase_controller_bind_test.go @@ -44,9 +44,9 @@ import ( . "github.com/onsi/ginkgo" . "github.com/onsi/gomega" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/database" - "github.com/oracle/oci-go-sdk/v45/workrequests" + "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v51/database" + "github.com/oracle/oci-go-sdk/v51/workrequests" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/types" diff --git a/test/e2e/autonomousdatabase_controller_create_test.go b/test/e2e/autonomousdatabase_controller_create_test.go index 3ecd2804..874ada3a 100644 --- a/test/e2e/autonomousdatabase_controller_create_test.go +++ b/test/e2e/autonomousdatabase_controller_create_test.go @@ -44,8 +44,8 @@ import ( . "github.com/onsi/ginkgo" . "github.com/onsi/gomega" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/database" + "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v51/database" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/types" diff --git a/test/e2e/behavior/shared_behaviors.go b/test/e2e/behavior/shared_behaviors.go index 501a93b2..9adb7121 100644 --- a/test/e2e/behavior/shared_behaviors.go +++ b/test/e2e/behavior/shared_behaviors.go @@ -47,8 +47,8 @@ import ( . "github.com/onsi/ginkgo" . "github.com/onsi/gomega" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/database" + "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v51/database" corev1 "k8s.io/api/core/v1" k8sErrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/types" diff --git a/test/e2e/suite_test.go b/test/e2e/suite_test.go index 3fd5cd2e..7cfb9424 100644 --- a/test/e2e/suite_test.go +++ b/test/e2e/suite_test.go @@ -47,8 +47,8 @@ import ( . "github.com/onsi/ginkgo" . "github.com/onsi/gomega" "github.com/onsi/gomega/gexec" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/database" + "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v51/database" corev1 "k8s.io/api/core/v1" "k8s.io/client-go/kubernetes/scheme" "k8s.io/client-go/rest" diff --git a/test/e2e/util/oci_config_util.go b/test/e2e/util/oci_config_util.go index 70ee1cab..0c8b8ba9 100644 --- a/test/e2e/util/oci_config_util.go +++ b/test/e2e/util/oci_config_util.go @@ -48,7 +48,7 @@ import ( "regexp" "strings" - "github.com/oracle/oci-go-sdk/v45/common" + "github.com/oracle/oci-go-sdk/v51/common" corev1 "k8s.io/api/core/v1" ) diff --git a/test/e2e/util/oci_db_request.go b/test/e2e/util/oci_db_request.go index 5fcd3aa1..056e1c40 100644 --- a/test/e2e/util/oci_db_request.go +++ b/test/e2e/util/oci_db_request.go @@ -41,8 +41,8 @@ package e2eutil import ( "context" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/database" + "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v51/database" "time" ) diff --git a/test/e2e/util/oci_vault_request.go b/test/e2e/util/oci_vault_request.go index ef7a793c..9cdc210b 100644 --- a/test/e2e/util/oci_vault_request.go +++ b/test/e2e/util/oci_vault_request.go @@ -44,9 +44,9 @@ import ( "math" "time" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/keymanagement" - "github.com/oracle/oci-go-sdk/v45/vault" + "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v51/keymanagement" + "github.com/oracle/oci-go-sdk/v51/vault" ) func waitForVaultStatePolicy(state keymanagement.VaultLifecycleStateEnum) common.RetryPolicy { diff --git a/test/e2e/util/oci_work_request.go b/test/e2e/util/oci_work_request.go index 4721fa0a..450d9c3f 100644 --- a/test/e2e/util/oci_work_request.go +++ b/test/e2e/util/oci_work_request.go @@ -41,8 +41,8 @@ package e2eutil import ( "context" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/workrequests" + "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v51/workrequests" "time" ) From f91a4e5223604eda32af6815cd938dd0a85d2c51 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Thu, 2 Dec 2021 12:49:05 -0500 Subject: [PATCH 055/628] update oci-go-sdk v51 --- .../v1alpha1/zz_generated.deepcopy.go | 21 ++++++++++++------- commons/autonomousdatabase/reconciler_util.go | 6 +++--- commons/oci/database.go | 8 +++---- commons/oci/provider.go | 4 ++-- commons/oci/vault.go | 4 ++-- commons/oci/wallet.go | 6 +++--- commons/sharding/scommon.go | 4 ++-- .../database/autonomousdatabase_controller.go | 6 +++--- .../database/shardingdatabase_controller.go | 4 ++-- go.mod | 2 +- go.sum | 6 ++++-- ...autonomousdatabase_controller_bind_test.go | 6 +++--- ...tonomousdatabase_controller_create_test.go | 4 ++-- test/e2e/behavior/shared_behaviors.go | 4 ++-- test/e2e/suite_test.go | 4 ++-- test/e2e/util/oci_config_util.go | 2 +- test/e2e/util/oci_db_request.go | 4 ++-- test/e2e/util/oci_vault_request.go | 6 +++--- test/e2e/util/oci_work_request.go | 4 ++-- 19 files changed, 56 insertions(+), 49 deletions(-) diff --git a/apis/database/v1alpha1/zz_generated.deepcopy.go b/apis/database/v1alpha1/zz_generated.deepcopy.go index 2766d559..4d27bb59 100644 --- a/apis/database/v1alpha1/zz_generated.deepcopy.go +++ b/apis/database/v1alpha1/zz_generated.deepcopy.go @@ -134,18 +134,23 @@ func (in *AutonomousDatabaseDetails) DeepCopyInto(out *AutonomousDatabaseDetails *out = make([]string, len(*in)) copy(*out, *in) } - if in.PrivateEndpoint != nil { - in, out := &in.PrivateEndpoint, &out.PrivateEndpoint - *out = new(string) + if in.IsAccessControlEnabled != nil { + in, out := &in.IsAccessControlEnabled, &out.IsAccessControlEnabled + *out = new(bool) **out = **in } - if in.PrivateEndpointLabel != nil { - in, out := &in.PrivateEndpointLabel, &out.PrivateEndpointLabel - *out = new(string) + if in.WhitelistedIPs != nil { + in, out := &in.WhitelistedIPs, &out.WhitelistedIPs + *out = make([]string, len(*in)) + copy(*out, *in) + } + if in.IsMTLSConnectionRequired != nil { + in, out := &in.IsMTLSConnectionRequired, &out.IsMTLSConnectionRequired + *out = new(bool) **out = **in } - if in.PrivateEndpointIP != nil { - in, out := &in.PrivateEndpointIP, &out.PrivateEndpointIP + if in.PrivateEndpointLabel != nil { + in, out := &in.PrivateEndpointLabel, &out.PrivateEndpointLabel *out = new(string) **out = **in } diff --git a/commons/autonomousdatabase/reconciler_util.go b/commons/autonomousdatabase/reconciler_util.go index 2ceac5e6..2f820410 100644 --- a/commons/autonomousdatabase/reconciler_util.go +++ b/commons/autonomousdatabase/reconciler_util.go @@ -49,9 +49,9 @@ import ( "sigs.k8s.io/controller-runtime/pkg/client" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/database" - "github.com/oracle/oci-go-sdk/v45/secrets" + "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v51/database" + "github.com/oracle/oci-go-sdk/v51/secrets" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" "github.com/oracle/oracle-database-operator/commons/oci" diff --git a/commons/oci/database.go b/commons/oci/database.go index 985d299b..c1a8cad0 100644 --- a/commons/oci/database.go +++ b/commons/oci/database.go @@ -46,10 +46,10 @@ import ( "time" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/database" - "github.com/oracle/oci-go-sdk/v45/secrets" - "github.com/oracle/oci-go-sdk/v45/workrequests" + "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v51/database" + "github.com/oracle/oci-go-sdk/v51/secrets" + "github.com/oracle/oci-go-sdk/v51/workrequests" "sigs.k8s.io/controller-runtime/pkg/client" corev1 "k8s.io/api/core/v1" diff --git a/commons/oci/provider.go b/commons/oci/provider.go index f5bd6cda..32188e81 100644 --- a/commons/oci/provider.go +++ b/commons/oci/provider.go @@ -42,8 +42,8 @@ import ( "context" "errors" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/common/auth" + "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v51/common/auth" "sigs.k8s.io/controller-runtime/pkg/client" diff --git a/commons/oci/vault.go b/commons/oci/vault.go index 6bf790aa..5df1cc04 100644 --- a/commons/oci/vault.go +++ b/commons/oci/vault.go @@ -42,8 +42,8 @@ import ( "context" "encoding/base64" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/secrets" + "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v51/secrets" ) func getValueFromVaultSecret(secretClient secrets.SecretsClient, vaultSecretOCID string) (string, error) { diff --git a/commons/oci/wallet.go b/commons/oci/wallet.go index 3faa4f16..268e66c5 100644 --- a/commons/oci/wallet.go +++ b/commons/oci/wallet.go @@ -49,9 +49,9 @@ import ( "time" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/database" - "github.com/oracle/oci-go-sdk/v45/secrets" + "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v51/database" + "github.com/oracle/oci-go-sdk/v51/secrets" "sigs.k8s.io/controller-runtime/pkg/client" "k8s.io/apimachinery/pkg/types" diff --git a/commons/sharding/scommon.go b/commons/sharding/scommon.go index 55fdd5dc..5af51390 100644 --- a/commons/sharding/scommon.go +++ b/commons/sharding/scommon.go @@ -48,8 +48,8 @@ import ( "strings" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/ons" + "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v51/ons" appsv1 "k8s.io/api/apps/v1" corev1 "k8s.io/api/core/v1" "k8s.io/apimachinery/pkg/api/errors" diff --git a/controllers/database/autonomousdatabase_controller.go b/controllers/database/autonomousdatabase_controller.go index 2f250489..5eba7ae7 100644 --- a/controllers/database/autonomousdatabase_controller.go +++ b/controllers/database/autonomousdatabase_controller.go @@ -44,9 +44,9 @@ import ( "reflect" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v45/database" - "github.com/oracle/oci-go-sdk/v45/secrets" - "github.com/oracle/oci-go-sdk/v45/workrequests" + "github.com/oracle/oci-go-sdk/v51/database" + "github.com/oracle/oci-go-sdk/v51/secrets" + "github.com/oracle/oci-go-sdk/v51/workrequests" apiErrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/runtime" diff --git a/controllers/database/shardingdatabase_controller.go b/controllers/database/shardingdatabase_controller.go index 73575df7..f30e714b 100644 --- a/controllers/database/shardingdatabase_controller.go +++ b/controllers/database/shardingdatabase_controller.go @@ -47,8 +47,8 @@ import ( "time" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/ons" + "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v51/ons" appsv1 "k8s.io/api/apps/v1" corev1 "k8s.io/api/core/v1" "k8s.io/apimachinery/pkg/api/errors" diff --git a/go.mod b/go.mod index fada426a..4e19053e 100644 --- a/go.mod +++ b/go.mod @@ -6,7 +6,7 @@ require ( github.com/go-logr/logr v0.4.0 github.com/onsi/ginkgo v1.16.4 github.com/onsi/gomega v1.13.0 - github.com/oracle/oci-go-sdk/v45 v45.2.0 + github.com/oracle/oci-go-sdk/v51 v51.0.0 gopkg.in/yaml.v2 v2.4.0 k8s.io/api v0.21.2 k8s.io/apimachinery v0.21.2 diff --git a/go.sum b/go.sum index 67d9f3df..2b051eef 100644 --- a/go.sum +++ b/go.sum @@ -304,8 +304,8 @@ github.com/onsi/gomega v1.7.1/go.mod h1:XdKZgCCFLUoM/7CFJVPcG8C1xQ1AJ0vpAezJrB7J github.com/onsi/gomega v1.10.1/go.mod h1:iN09h71vgCQne3DLsj+A5owkum+a2tYe+TOCB1ybHNo= github.com/onsi/gomega v1.13.0 h1:7lLHu94wT9Ij0o6EWWclhu0aOh32VxhkwEJvzuWPeak= github.com/onsi/gomega v1.13.0/go.mod h1:lRk9szgn8TxENtWd0Tp4c3wjlRfMTMH27I+3Je41yGY= -github.com/oracle/oci-go-sdk/v45 v45.2.0 h1:vCPoQlE+DOrM2heJn66rvPU6fbsc/0Cxtzs2jnFut6U= -github.com/oracle/oci-go-sdk/v45 v45.2.0/go.mod h1:ZM6LGiRO5TPQJxTlrXbcHMbClE775wnGD5U/EerCsRw= +github.com/oracle/oci-go-sdk/v51 v51.0.0 h1:eDUVMsAzvf+jfq4xbtpQrxznYNjccwKKyIhmRvZLgwI= +github.com/oracle/oci-go-sdk/v51 v51.0.0/go.mod h1:d9KSNXwE64drofxoor+y/JWofJqLqRF9D1/AtfYIE10= github.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc= github.com/pelletier/go-toml v1.2.0/go.mod h1:5z9KED0ma1S8pY6P1sdut58dfprrGBbd/94hg7ilaic= github.com/peterbourgon/diskv v2.0.1+incompatible/go.mod h1:uqqh8zWWbv1HBMNONnaR/tNboyR3/BZd58JJSHlUSCU= @@ -355,6 +355,8 @@ github.com/sirupsen/logrus v1.7.0/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1:OnSkiWE9lh6wB0YB77sQom3nweQdgAjqCqsofrRNTgc= github.com/smartystreets/goconvey v1.6.4/go.mod h1:syvi0/a8iFYH4r/RixwvyeAJjdLS9QV7WQ/tjFTllLA= github.com/soheilhy/cmux v0.1.4/go.mod h1:IM3LyeVVIOuxMH7sFAkER9+bJ4dT7Ms6E4xg4kGIyLM= +github.com/sony/gobreaker v0.4.2-0.20210216022020-dd874f9dd33b h1:br+bPNZsJWKicw/5rALEo67QHs5weyD5tf8WST+4sJ0= +github.com/sony/gobreaker v0.4.2-0.20210216022020-dd874f9dd33b/go.mod h1:ZKptC7FHNvhBz7dN2LGjPVBz2sZJmc0/PkyDJOjmxWY= github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA= github.com/spf13/afero v1.1.2/go.mod h1:j4pytiNVoe2o6bmDsKpLACNPDBIoEAkihy7loJ1B0CQ= github.com/spf13/afero v1.2.2/go.mod h1:9ZxEEn6pIJ8Rxe320qSDBk6AsU0r9pR7Q4OcevTdifk= diff --git a/test/e2e/autonomousdatabase_controller_bind_test.go b/test/e2e/autonomousdatabase_controller_bind_test.go index ff415da9..3ab132c9 100644 --- a/test/e2e/autonomousdatabase_controller_bind_test.go +++ b/test/e2e/autonomousdatabase_controller_bind_test.go @@ -44,9 +44,9 @@ import ( . "github.com/onsi/ginkgo" . "github.com/onsi/gomega" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/database" - "github.com/oracle/oci-go-sdk/v45/workrequests" + "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v51/database" + "github.com/oracle/oci-go-sdk/v51/workrequests" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/types" diff --git a/test/e2e/autonomousdatabase_controller_create_test.go b/test/e2e/autonomousdatabase_controller_create_test.go index 3ecd2804..874ada3a 100644 --- a/test/e2e/autonomousdatabase_controller_create_test.go +++ b/test/e2e/autonomousdatabase_controller_create_test.go @@ -44,8 +44,8 @@ import ( . "github.com/onsi/ginkgo" . "github.com/onsi/gomega" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/database" + "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v51/database" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/types" diff --git a/test/e2e/behavior/shared_behaviors.go b/test/e2e/behavior/shared_behaviors.go index 501a93b2..9adb7121 100644 --- a/test/e2e/behavior/shared_behaviors.go +++ b/test/e2e/behavior/shared_behaviors.go @@ -47,8 +47,8 @@ import ( . "github.com/onsi/ginkgo" . "github.com/onsi/gomega" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/database" + "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v51/database" corev1 "k8s.io/api/core/v1" k8sErrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/types" diff --git a/test/e2e/suite_test.go b/test/e2e/suite_test.go index 3fd5cd2e..7cfb9424 100644 --- a/test/e2e/suite_test.go +++ b/test/e2e/suite_test.go @@ -47,8 +47,8 @@ import ( . "github.com/onsi/ginkgo" . "github.com/onsi/gomega" "github.com/onsi/gomega/gexec" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/database" + "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v51/database" corev1 "k8s.io/api/core/v1" "k8s.io/client-go/kubernetes/scheme" "k8s.io/client-go/rest" diff --git a/test/e2e/util/oci_config_util.go b/test/e2e/util/oci_config_util.go index 70ee1cab..0c8b8ba9 100644 --- a/test/e2e/util/oci_config_util.go +++ b/test/e2e/util/oci_config_util.go @@ -48,7 +48,7 @@ import ( "regexp" "strings" - "github.com/oracle/oci-go-sdk/v45/common" + "github.com/oracle/oci-go-sdk/v51/common" corev1 "k8s.io/api/core/v1" ) diff --git a/test/e2e/util/oci_db_request.go b/test/e2e/util/oci_db_request.go index 5fcd3aa1..056e1c40 100644 --- a/test/e2e/util/oci_db_request.go +++ b/test/e2e/util/oci_db_request.go @@ -41,8 +41,8 @@ package e2eutil import ( "context" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/database" + "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v51/database" "time" ) diff --git a/test/e2e/util/oci_vault_request.go b/test/e2e/util/oci_vault_request.go index ef7a793c..9cdc210b 100644 --- a/test/e2e/util/oci_vault_request.go +++ b/test/e2e/util/oci_vault_request.go @@ -44,9 +44,9 @@ import ( "math" "time" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/keymanagement" - "github.com/oracle/oci-go-sdk/v45/vault" + "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v51/keymanagement" + "github.com/oracle/oci-go-sdk/v51/vault" ) func waitForVaultStatePolicy(state keymanagement.VaultLifecycleStateEnum) common.RetryPolicy { diff --git a/test/e2e/util/oci_work_request.go b/test/e2e/util/oci_work_request.go index 4721fa0a..450d9c3f 100644 --- a/test/e2e/util/oci_work_request.go +++ b/test/e2e/util/oci_work_request.go @@ -41,8 +41,8 @@ package e2eutil import ( "context" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/workrequests" + "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v51/workrequests" "time" ) From ce0bb23501da7f3c8118df7d1689214b037531f5 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Thu, 2 Dec 2021 12:49:43 -0500 Subject: [PATCH 056/628] update network access attrs --- .../v1alpha1/autonomousdatabase_types.go | 25 +++++++++++-------- 1 file changed, 15 insertions(+), 10 deletions(-) diff --git a/apis/database/v1alpha1/autonomousdatabase_types.go b/apis/database/v1alpha1/autonomousdatabase_types.go index 4916ad46..b8759348 100644 --- a/apis/database/v1alpha1/autonomousdatabase_types.go +++ b/apis/database/v1alpha1/autonomousdatabase_types.go @@ -42,7 +42,7 @@ import ( "encoding/json" "strconv" - "github.com/oracle/oci-go-sdk/v45/database" + "github.com/oracle/oci-go-sdk/v51/database" metaV1 "k8s.io/apimachinery/pkg/apis/meta/v1" "sigs.k8s.io/controller-runtime/pkg/client" @@ -81,13 +81,17 @@ type AutonomousDatabaseDetails struct { AdminPassword PasswordSpec `json:"adminPassword,omitempty"` IsAutoScalingEnabled *bool `json:"isAutoScalingEnabled,omitempty"` LifecycleState database.AutonomousDatabaseLifecycleStateEnum `json:"lifecycleState,omitempty"` - SubnetOCID *string `json:"subnetOCID,omitempty"` - NsgOCIDs []string `json:"nsgOCIDs,omitempty"` - PrivateEndpoint *string `json:"privateEndpoint,omitempty"` - PrivateEndpointLabel *string `json:"privateEndpointLabel,omitempty"` - PrivateEndpointIP *string `json:"privateEndpointIP,omitempty"` - FreeformTags map[string]string `json:"freeformTags,omitempty"` - Wallet WalletSpec `json:"wallet,omitempty"` + + SubnetOCID *string `json:"subnetOCID,omitempty"` + NsgOCIDs []string `json:"nsgOCIDs,omitempty"` + IsAccessControlEnabled *bool `json:"isAccessControlEnabled,omitempty"` + WhitelistedIPs []string `json:"whitelistedIPs,omitempty"` + IsMTLSConnectionRequired *bool `json:"isMTLSConnectionRequired,omitempty"` + PrivateEndpointLabel *string `json:"privateEndpointLabel,omitempty"` + + FreeformTags map[string]string `json:"freeformTags,omitempty"` + + Wallet WalletSpec `json:"wallet,omitempty"` } type WalletSpec struct { @@ -185,9 +189,10 @@ func (adb *AutonomousDatabase) UpdateAttrFromOCIAutonomousDatabase(ociObj databa adb.Spec.Details.SubnetOCID = ociObj.SubnetId adb.Spec.Details.NsgOCIDs = ociObj.NsgIds - adb.Spec.Details.PrivateEndpoint = ociObj.PrivateEndpoint + adb.Spec.Details.IsAccessControlEnabled = ociObj.IsAccessControlEnabled + adb.Spec.Details.WhitelistedIPs = ociObj.WhitelistedIps + adb.Spec.Details.IsMTLSConnectionRequired = ociObj.IsMtlsConnectionRequired adb.Spec.Details.PrivateEndpointLabel = ociObj.PrivateEndpointLabel - adb.Spec.Details.PrivateEndpointIP = ociObj.PrivateEndpointIp // update the subresource as well adb.Status.DisplayName = *ociObj.DisplayName From 9d88664451d65fcf6abf0523888824eb904275bd Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Thu, 2 Dec 2021 12:49:43 -0500 Subject: [PATCH 057/628] update network access attrs --- .../v1alpha1/autonomousdatabase_types.go | 25 +++++++++++-------- 1 file changed, 15 insertions(+), 10 deletions(-) diff --git a/apis/database/v1alpha1/autonomousdatabase_types.go b/apis/database/v1alpha1/autonomousdatabase_types.go index 4916ad46..b8759348 100644 --- a/apis/database/v1alpha1/autonomousdatabase_types.go +++ b/apis/database/v1alpha1/autonomousdatabase_types.go @@ -42,7 +42,7 @@ import ( "encoding/json" "strconv" - "github.com/oracle/oci-go-sdk/v45/database" + "github.com/oracle/oci-go-sdk/v51/database" metaV1 "k8s.io/apimachinery/pkg/apis/meta/v1" "sigs.k8s.io/controller-runtime/pkg/client" @@ -81,13 +81,17 @@ type AutonomousDatabaseDetails struct { AdminPassword PasswordSpec `json:"adminPassword,omitempty"` IsAutoScalingEnabled *bool `json:"isAutoScalingEnabled,omitempty"` LifecycleState database.AutonomousDatabaseLifecycleStateEnum `json:"lifecycleState,omitempty"` - SubnetOCID *string `json:"subnetOCID,omitempty"` - NsgOCIDs []string `json:"nsgOCIDs,omitempty"` - PrivateEndpoint *string `json:"privateEndpoint,omitempty"` - PrivateEndpointLabel *string `json:"privateEndpointLabel,omitempty"` - PrivateEndpointIP *string `json:"privateEndpointIP,omitempty"` - FreeformTags map[string]string `json:"freeformTags,omitempty"` - Wallet WalletSpec `json:"wallet,omitempty"` + + SubnetOCID *string `json:"subnetOCID,omitempty"` + NsgOCIDs []string `json:"nsgOCIDs,omitempty"` + IsAccessControlEnabled *bool `json:"isAccessControlEnabled,omitempty"` + WhitelistedIPs []string `json:"whitelistedIPs,omitempty"` + IsMTLSConnectionRequired *bool `json:"isMTLSConnectionRequired,omitempty"` + PrivateEndpointLabel *string `json:"privateEndpointLabel,omitempty"` + + FreeformTags map[string]string `json:"freeformTags,omitempty"` + + Wallet WalletSpec `json:"wallet,omitempty"` } type WalletSpec struct { @@ -185,9 +189,10 @@ func (adb *AutonomousDatabase) UpdateAttrFromOCIAutonomousDatabase(ociObj databa adb.Spec.Details.SubnetOCID = ociObj.SubnetId adb.Spec.Details.NsgOCIDs = ociObj.NsgIds - adb.Spec.Details.PrivateEndpoint = ociObj.PrivateEndpoint + adb.Spec.Details.IsAccessControlEnabled = ociObj.IsAccessControlEnabled + adb.Spec.Details.WhitelistedIPs = ociObj.WhitelistedIps + adb.Spec.Details.IsMTLSConnectionRequired = ociObj.IsMtlsConnectionRequired adb.Spec.Details.PrivateEndpointLabel = ociObj.PrivateEndpointLabel - adb.Spec.Details.PrivateEndpointIP = ociObj.PrivateEndpointIp // update the subresource as well adb.Status.DisplayName = *ociObj.DisplayName From 21514b1e821382ac86c6ed9ad323531bdae9e1cd Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Thu, 2 Dec 2021 12:54:12 -0500 Subject: [PATCH 058/628] isAttrChanged() returns true when last attr is nil --- commons/oci/database.go | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/commons/oci/database.go b/commons/oci/database.go index c1a8cad0..c726b17d 100644 --- a/commons/oci/database.go +++ b/commons/oci/database.go @@ -177,7 +177,7 @@ func isAttrChanged(lastSucObj interface{}, curObj interface{}) bool { } curIntPtr, ok := curObj.(*int) - if lastSucIntPtr != nil && curIntPtr != nil && *curIntPtr != 0 && *lastSucIntPtr != *curIntPtr { + if (lastSucIntPtr == nil && curIntPtr != nil) || (lastSucIntPtr != nil && curIntPtr != nil && *lastSucIntPtr != *curIntPtr) { return true } case *string: @@ -188,7 +188,7 @@ func isAttrChanged(lastSucObj interface{}, curObj interface{}) bool { } curStringPtr := curObj.(*string) - if lastSucStringPtr != nil && curStringPtr != nil && *curStringPtr != "" && *lastSucStringPtr != *curStringPtr { + if (lastSucStringPtr == nil && curStringPtr != nil) || (lastSucStringPtr != nil && curStringPtr != nil && *lastSucStringPtr != *curStringPtr) { return true } case *bool: @@ -200,7 +200,7 @@ func isAttrChanged(lastSucObj interface{}, curObj interface{}) bool { curBoolPtr := curObj.(*bool) // For boolean type, we don't have to check zero value - if lastSucBoolPtr != nil && curBoolPtr != nil && *lastSucBoolPtr != *curBoolPtr { + if (lastSucBoolPtr == nil && curBoolPtr != nil) || (lastSucBoolPtr != nil && curBoolPtr != nil && *lastSucBoolPtr != *curBoolPtr) { return true } case []string: From 2e875b737735288965609de1d14625877bef2e43 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Thu, 2 Dec 2021 12:54:12 -0500 Subject: [PATCH 059/628] isAttrChanged() returns true when last attr is nil --- commons/oci/database.go | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/commons/oci/database.go b/commons/oci/database.go index c1a8cad0..c726b17d 100644 --- a/commons/oci/database.go +++ b/commons/oci/database.go @@ -177,7 +177,7 @@ func isAttrChanged(lastSucObj interface{}, curObj interface{}) bool { } curIntPtr, ok := curObj.(*int) - if lastSucIntPtr != nil && curIntPtr != nil && *curIntPtr != 0 && *lastSucIntPtr != *curIntPtr { + if (lastSucIntPtr == nil && curIntPtr != nil) || (lastSucIntPtr != nil && curIntPtr != nil && *lastSucIntPtr != *curIntPtr) { return true } case *string: @@ -188,7 +188,7 @@ func isAttrChanged(lastSucObj interface{}, curObj interface{}) bool { } curStringPtr := curObj.(*string) - if lastSucStringPtr != nil && curStringPtr != nil && *curStringPtr != "" && *lastSucStringPtr != *curStringPtr { + if (lastSucStringPtr == nil && curStringPtr != nil) || (lastSucStringPtr != nil && curStringPtr != nil && *lastSucStringPtr != *curStringPtr) { return true } case *bool: @@ -200,7 +200,7 @@ func isAttrChanged(lastSucObj interface{}, curObj interface{}) bool { curBoolPtr := curObj.(*bool) // For boolean type, we don't have to check zero value - if lastSucBoolPtr != nil && curBoolPtr != nil && *lastSucBoolPtr != *curBoolPtr { + if (lastSucBoolPtr == nil && curBoolPtr != nil) || (lastSucBoolPtr != nil && curBoolPtr != nil && *lastSucBoolPtr != *curBoolPtr) { return true } case []string: From c8d918218ff6686fcf28f4fb7f5c7df475105a87 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Thu, 2 Dec 2021 12:59:30 -0500 Subject: [PATCH 060/628] add network settings to CreateAutonomousDatabase() --- commons/oci/database.go | 107 ++++++++++++++++++++++++++++++++++------ 1 file changed, 92 insertions(+), 15 deletions(-) diff --git a/commons/oci/database.go b/commons/oci/database.go index c726b17d..9b78e37e 100644 --- a/commons/oci/database.go +++ b/commons/oci/database.go @@ -77,8 +77,14 @@ func CreateAutonomousDatabase(logger logr.Logger, kubeClient client.Client, dbCl DbVersion: adb.Spec.Details.DbVersion, DbWorkload: database.CreateAutonomousDatabaseBaseDbWorkloadEnum( adb.Spec.Details.DbWorkload), - SubnetId: adb.Spec.Details.SubnetOCID, - NsgIds: adb.Spec.Details.NsgOCIDs, + + WhitelistedIps: adb.Spec.Details.WhitelistedIPs, + SubnetId: adb.Spec.Details.SubnetOCID, + NsgIds: adb.Spec.Details.NsgOCIDs, + PrivateEndpointLabel: adb.Spec.Details.PrivateEndpointLabel, + IsMtlsConnectionRequired: adb.Spec.Details.IsMTLSConnectionRequired, + + FreeformTags: adb.Spec.Details.FreeformTags, } createAutonomousDatabaseRequest := database.CreateAutonomousDatabaseRequest{ @@ -287,19 +293,6 @@ func UpdateGeneralAndPasswordAttributes(logger logr.Logger, kubeClient client.Cl shouldSendRequest = true } - if isAttrChanged(lastSucSpec.Details.FreeformTags, curADB.Spec.Details.FreeformTags) { - updateAutonomousDatabaseDetails.FreeformTags = curADB.Spec.Details.FreeformTags - shouldSendRequest = true - } - if isAttrChanged(lastSucSpec.Details.SubnetOCID, curADB.Spec.Details.SubnetOCID) { - updateAutonomousDatabaseDetails.SubnetId = curADB.Spec.Details.SubnetOCID - shouldSendRequest = true - } - if isAttrChanged(lastSucSpec.Details.NsgOCIDs, curADB.Spec.Details.NsgOCIDs) { - updateAutonomousDatabaseDetails.NsgIds = curADB.Spec.Details.NsgOCIDs - shouldSendRequest = true - } - if isAttrChanged(lastSucSpec.Details.AdminPassword.K8sSecretName, curADB.Spec.Details.AdminPassword.K8sSecretName) || isAttrChanged(lastSucSpec.Details.AdminPassword.OCISecretOCID, curADB.Spec.Details.AdminPassword.OCISecretOCID) { // Get the adminPassword @@ -375,6 +368,90 @@ func UpdateScaleAttributes(logger logr.Logger, kubeClient client.Client, dbClien return } +func UpdateOneWayTLSAttribute(logger logr.Logger, kubeClient client.Client, dbClient database.DatabaseClient, + curADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { + var shouldSendRequest = false + + lastSucSpec, err := curADB.GetLastSuccessfulSpec() + if err != nil { + return resp, err + } + + // Prepare the update request + updateAutonomousDatabaseDetails := database.UpdateAutonomousDatabaseDetails{} + + if isAttrChanged(lastSucSpec.Details.IsMTLSConnectionRequired, curADB.Spec.Details.IsMTLSConnectionRequired) { + updateAutonomousDatabaseDetails.IsMtlsConnectionRequired = curADB.Spec.Details.IsMTLSConnectionRequired + shouldSendRequest = true + } + + // Don't send the request if nothing is changed + if shouldSendRequest { + + logger.Info("Sending 1-way TLS attribute update request") + + updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ + AutonomousDatabaseId: curADB.Spec.Details.AutonomousDatabaseOCID, + UpdateAutonomousDatabaseDetails: updateAutonomousDatabaseDetails, + } + + resp, err = dbClient.UpdateAutonomousDatabase(context.TODO(), updateAutonomousDatabaseRequest) + } + + return +} + +func UpdateNetworkAttributes(logger logr.Logger, kubeClient client.Client, dbClient database.DatabaseClient, + curADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { + var shouldSendRequest = false + + lastSucSpec, err := curADB.GetLastSuccessfulSpec() + if err != nil { + return resp, err + } + + // Prepare the update request + updateAutonomousDatabaseDetails := database.UpdateAutonomousDatabaseDetails{} + + // Network settings + if isAttrChanged(lastSucSpec.Details.SubnetOCID, curADB.Spec.Details.SubnetOCID) { + updateAutonomousDatabaseDetails.SubnetId = curADB.Spec.Details.SubnetOCID + shouldSendRequest = true + } + if isAttrChanged(lastSucSpec.Details.NsgOCIDs, curADB.Spec.Details.NsgOCIDs) { + updateAutonomousDatabaseDetails.NsgIds = curADB.Spec.Details.NsgOCIDs + shouldSendRequest = true + } + if isAttrChanged(lastSucSpec.Details.IsAccessControlEnabled, curADB.Spec.Details.IsAccessControlEnabled) { + updateAutonomousDatabaseDetails.IsAccessControlEnabled = curADB.Spec.Details.IsAccessControlEnabled + shouldSendRequest = true + } + if isAttrChanged(lastSucSpec.Details.WhitelistedIPs, curADB.Spec.Details.WhitelistedIPs) { + updateAutonomousDatabaseDetails.WhitelistedIps = curADB.Spec.Details.WhitelistedIPs + shouldSendRequest = true + } + // PrivateEndpointLabel + if isAttrChanged(lastSucSpec.Details.PrivateEndpointLabel, curADB.Spec.Details.PrivateEndpointLabel) { + updateAutonomousDatabaseDetails.PrivateEndpointLabel = curADB.Spec.Details.PrivateEndpointLabel + shouldSendRequest = true + } + + // Don't send the request if nothing is changed + if shouldSendRequest { + + logger.Info("Sending network attributes update request") + + updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ + AutonomousDatabaseId: curADB.Spec.Details.AutonomousDatabaseOCID, + UpdateAutonomousDatabaseDetails: updateAutonomousDatabaseDetails, + } + + resp, err = dbClient.UpdateAutonomousDatabase(context.TODO(), updateAutonomousDatabaseRequest) + } + + return +} + // SetAutonomousDatabaseLifecycleState starts or stops AutonomousDatabase in OCI based on the LifeCycleState attribute func SetAutonomousDatabaseLifecycleState(logger logr.Logger, dbClient database.DatabaseClient, adb *dbv1alpha1.AutonomousDatabase) (resp interface{}, err error) { lastSucSpec, err := adb.GetLastSuccessfulSpec() From 2bab29c634db96e7464669c17d512f77e6583547 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Thu, 2 Dec 2021 12:59:30 -0500 Subject: [PATCH 061/628] add network settings to CreateAutonomousDatabase() --- commons/oci/database.go | 107 ++++++++++++++++++++++++++++++++++------ 1 file changed, 92 insertions(+), 15 deletions(-) diff --git a/commons/oci/database.go b/commons/oci/database.go index c726b17d..9b78e37e 100644 --- a/commons/oci/database.go +++ b/commons/oci/database.go @@ -77,8 +77,14 @@ func CreateAutonomousDatabase(logger logr.Logger, kubeClient client.Client, dbCl DbVersion: adb.Spec.Details.DbVersion, DbWorkload: database.CreateAutonomousDatabaseBaseDbWorkloadEnum( adb.Spec.Details.DbWorkload), - SubnetId: adb.Spec.Details.SubnetOCID, - NsgIds: adb.Spec.Details.NsgOCIDs, + + WhitelistedIps: adb.Spec.Details.WhitelistedIPs, + SubnetId: adb.Spec.Details.SubnetOCID, + NsgIds: adb.Spec.Details.NsgOCIDs, + PrivateEndpointLabel: adb.Spec.Details.PrivateEndpointLabel, + IsMtlsConnectionRequired: adb.Spec.Details.IsMTLSConnectionRequired, + + FreeformTags: adb.Spec.Details.FreeformTags, } createAutonomousDatabaseRequest := database.CreateAutonomousDatabaseRequest{ @@ -287,19 +293,6 @@ func UpdateGeneralAndPasswordAttributes(logger logr.Logger, kubeClient client.Cl shouldSendRequest = true } - if isAttrChanged(lastSucSpec.Details.FreeformTags, curADB.Spec.Details.FreeformTags) { - updateAutonomousDatabaseDetails.FreeformTags = curADB.Spec.Details.FreeformTags - shouldSendRequest = true - } - if isAttrChanged(lastSucSpec.Details.SubnetOCID, curADB.Spec.Details.SubnetOCID) { - updateAutonomousDatabaseDetails.SubnetId = curADB.Spec.Details.SubnetOCID - shouldSendRequest = true - } - if isAttrChanged(lastSucSpec.Details.NsgOCIDs, curADB.Spec.Details.NsgOCIDs) { - updateAutonomousDatabaseDetails.NsgIds = curADB.Spec.Details.NsgOCIDs - shouldSendRequest = true - } - if isAttrChanged(lastSucSpec.Details.AdminPassword.K8sSecretName, curADB.Spec.Details.AdminPassword.K8sSecretName) || isAttrChanged(lastSucSpec.Details.AdminPassword.OCISecretOCID, curADB.Spec.Details.AdminPassword.OCISecretOCID) { // Get the adminPassword @@ -375,6 +368,90 @@ func UpdateScaleAttributes(logger logr.Logger, kubeClient client.Client, dbClien return } +func UpdateOneWayTLSAttribute(logger logr.Logger, kubeClient client.Client, dbClient database.DatabaseClient, + curADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { + var shouldSendRequest = false + + lastSucSpec, err := curADB.GetLastSuccessfulSpec() + if err != nil { + return resp, err + } + + // Prepare the update request + updateAutonomousDatabaseDetails := database.UpdateAutonomousDatabaseDetails{} + + if isAttrChanged(lastSucSpec.Details.IsMTLSConnectionRequired, curADB.Spec.Details.IsMTLSConnectionRequired) { + updateAutonomousDatabaseDetails.IsMtlsConnectionRequired = curADB.Spec.Details.IsMTLSConnectionRequired + shouldSendRequest = true + } + + // Don't send the request if nothing is changed + if shouldSendRequest { + + logger.Info("Sending 1-way TLS attribute update request") + + updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ + AutonomousDatabaseId: curADB.Spec.Details.AutonomousDatabaseOCID, + UpdateAutonomousDatabaseDetails: updateAutonomousDatabaseDetails, + } + + resp, err = dbClient.UpdateAutonomousDatabase(context.TODO(), updateAutonomousDatabaseRequest) + } + + return +} + +func UpdateNetworkAttributes(logger logr.Logger, kubeClient client.Client, dbClient database.DatabaseClient, + curADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { + var shouldSendRequest = false + + lastSucSpec, err := curADB.GetLastSuccessfulSpec() + if err != nil { + return resp, err + } + + // Prepare the update request + updateAutonomousDatabaseDetails := database.UpdateAutonomousDatabaseDetails{} + + // Network settings + if isAttrChanged(lastSucSpec.Details.SubnetOCID, curADB.Spec.Details.SubnetOCID) { + updateAutonomousDatabaseDetails.SubnetId = curADB.Spec.Details.SubnetOCID + shouldSendRequest = true + } + if isAttrChanged(lastSucSpec.Details.NsgOCIDs, curADB.Spec.Details.NsgOCIDs) { + updateAutonomousDatabaseDetails.NsgIds = curADB.Spec.Details.NsgOCIDs + shouldSendRequest = true + } + if isAttrChanged(lastSucSpec.Details.IsAccessControlEnabled, curADB.Spec.Details.IsAccessControlEnabled) { + updateAutonomousDatabaseDetails.IsAccessControlEnabled = curADB.Spec.Details.IsAccessControlEnabled + shouldSendRequest = true + } + if isAttrChanged(lastSucSpec.Details.WhitelistedIPs, curADB.Spec.Details.WhitelistedIPs) { + updateAutonomousDatabaseDetails.WhitelistedIps = curADB.Spec.Details.WhitelistedIPs + shouldSendRequest = true + } + // PrivateEndpointLabel + if isAttrChanged(lastSucSpec.Details.PrivateEndpointLabel, curADB.Spec.Details.PrivateEndpointLabel) { + updateAutonomousDatabaseDetails.PrivateEndpointLabel = curADB.Spec.Details.PrivateEndpointLabel + shouldSendRequest = true + } + + // Don't send the request if nothing is changed + if shouldSendRequest { + + logger.Info("Sending network attributes update request") + + updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ + AutonomousDatabaseId: curADB.Spec.Details.AutonomousDatabaseOCID, + UpdateAutonomousDatabaseDetails: updateAutonomousDatabaseDetails, + } + + resp, err = dbClient.UpdateAutonomousDatabase(context.TODO(), updateAutonomousDatabaseRequest) + } + + return +} + // SetAutonomousDatabaseLifecycleState starts or stops AutonomousDatabase in OCI based on the LifeCycleState attribute func SetAutonomousDatabaseLifecycleState(logger logr.Logger, dbClient database.DatabaseClient, adb *dbv1alpha1.AutonomousDatabase) (resp interface{}, err error) { lastSucSpec, err := adb.GetLastSuccessfulSpec() From c63c3d3a3f28a8fe0308b81f50a3b5e52b834d38 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Thu, 2 Dec 2021 12:59:50 -0500 Subject: [PATCH 062/628] add network requests to controller --- .../database/autonomousdatabase_controller.go | 64 ++++++++++++++++++- 1 file changed, 63 insertions(+), 1 deletion(-) diff --git a/controllers/database/autonomousdatabase_controller.go b/controllers/database/autonomousdatabase_controller.go index 5eba7ae7..142e78c6 100644 --- a/controllers/database/autonomousdatabase_controller.go +++ b/controllers/database/autonomousdatabase_controller.go @@ -428,7 +428,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R } if err := oci.WaitUntilWorkCompleted(r.currentLogger, workClient, scaleResp.OpcWorkRequestId); err != nil { - r.currentLogger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*updateGenPassResp.OpcWorkRequestId) + r.currentLogger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*scaleResp.OpcWorkRequestId) // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable @@ -438,6 +438,68 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R } r.currentLogger.Info("Scale AutonomousDatabase " + *adb.Spec.Details.DbName + " succesfully") } + + oneWayTLSResp, err := oci.UpdateOneWayTLSAttribute(r.currentLogger, r.KubeClient, dbClient, adb) + if err != nil { + r.currentLogger.Error(err, "Fail to update Autonomous Database") + + // Change the status to UNAVAILABLE + adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + return ctrl.Result{}, statusErr + } + // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue + } + + if oneWayTLSResp.OpcWorkRequestId != nil { + // Update status.state + adb.Status.LifecycleState = oneWayTLSResp.AutonomousDatabase.LifecycleState + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + return ctrl.Result{}, statusErr + } + + if err := oci.WaitUntilWorkCompleted(r.currentLogger, workClient, oneWayTLSResp.OpcWorkRequestId); err != nil { + r.currentLogger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*oneWayTLSResp.OpcWorkRequestId) + + // Change the status to UNAVAILABLE + adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + return ctrl.Result{}, statusErr + } + } + r.currentLogger.Info("Update AutonomousDatabase " + *adb.Spec.Details.DbName + " 1-way TLS setting succesfully") + } + + networkResp, err := oci.UpdateNetworkAttributes(r.currentLogger, r.KubeClient, dbClient, adb) + if err != nil { + r.currentLogger.Error(err, "Fail to update Autonomous Database") + + // Change the status to UNAVAILABLE + adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + return ctrl.Result{}, statusErr + } + // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue + } + + if networkResp.OpcWorkRequestId != nil { + // Update status.state + adb.Status.LifecycleState = networkResp.AutonomousDatabase.LifecycleState + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + return ctrl.Result{}, statusErr + } + + if err := oci.WaitUntilWorkCompleted(r.currentLogger, workClient, networkResp.OpcWorkRequestId); err != nil { + r.currentLogger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*networkResp.OpcWorkRequestId) + + // Change the status to UNAVAILABLE + adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + return ctrl.Result{}, statusErr + } + } + r.currentLogger.Info("Update AutonomousDatabase " + *adb.Spec.Details.DbName + " network settings succesfully") + } } } From 1b47d0cdd5cadbd8d26f3afe2e728394f6ef6792 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Thu, 2 Dec 2021 12:59:50 -0500 Subject: [PATCH 063/628] add network requests to controller --- .../database/autonomousdatabase_controller.go | 64 ++++++++++++++++++- 1 file changed, 63 insertions(+), 1 deletion(-) diff --git a/controllers/database/autonomousdatabase_controller.go b/controllers/database/autonomousdatabase_controller.go index 5eba7ae7..142e78c6 100644 --- a/controllers/database/autonomousdatabase_controller.go +++ b/controllers/database/autonomousdatabase_controller.go @@ -428,7 +428,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R } if err := oci.WaitUntilWorkCompleted(r.currentLogger, workClient, scaleResp.OpcWorkRequestId); err != nil { - r.currentLogger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*updateGenPassResp.OpcWorkRequestId) + r.currentLogger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*scaleResp.OpcWorkRequestId) // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable @@ -438,6 +438,68 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R } r.currentLogger.Info("Scale AutonomousDatabase " + *adb.Spec.Details.DbName + " succesfully") } + + oneWayTLSResp, err := oci.UpdateOneWayTLSAttribute(r.currentLogger, r.KubeClient, dbClient, adb) + if err != nil { + r.currentLogger.Error(err, "Fail to update Autonomous Database") + + // Change the status to UNAVAILABLE + adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + return ctrl.Result{}, statusErr + } + // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue + } + + if oneWayTLSResp.OpcWorkRequestId != nil { + // Update status.state + adb.Status.LifecycleState = oneWayTLSResp.AutonomousDatabase.LifecycleState + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + return ctrl.Result{}, statusErr + } + + if err := oci.WaitUntilWorkCompleted(r.currentLogger, workClient, oneWayTLSResp.OpcWorkRequestId); err != nil { + r.currentLogger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*oneWayTLSResp.OpcWorkRequestId) + + // Change the status to UNAVAILABLE + adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + return ctrl.Result{}, statusErr + } + } + r.currentLogger.Info("Update AutonomousDatabase " + *adb.Spec.Details.DbName + " 1-way TLS setting succesfully") + } + + networkResp, err := oci.UpdateNetworkAttributes(r.currentLogger, r.KubeClient, dbClient, adb) + if err != nil { + r.currentLogger.Error(err, "Fail to update Autonomous Database") + + // Change the status to UNAVAILABLE + adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + return ctrl.Result{}, statusErr + } + // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue + } + + if networkResp.OpcWorkRequestId != nil { + // Update status.state + adb.Status.LifecycleState = networkResp.AutonomousDatabase.LifecycleState + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + return ctrl.Result{}, statusErr + } + + if err := oci.WaitUntilWorkCompleted(r.currentLogger, workClient, networkResp.OpcWorkRequestId); err != nil { + r.currentLogger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*networkResp.OpcWorkRequestId) + + // Change the status to UNAVAILABLE + adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + return ctrl.Result{}, statusErr + } + } + r.currentLogger.Info("Update AutonomousDatabase " + *adb.Spec.Details.DbName + " network settings succesfully") + } } } From 5ba753c1680b9a57402971687517c8e41ccc1f01 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Thu, 2 Dec 2021 13:47:50 -0500 Subject: [PATCH 064/628] remove unused attributes --- test/e2e/behavior/shared_behaviors.go | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/test/e2e/behavior/shared_behaviors.go b/test/e2e/behavior/shared_behaviors.go index 9adb7121..a54e4e3a 100644 --- a/test/e2e/behavior/shared_behaviors.go +++ b/test/e2e/behavior/shared_behaviors.go @@ -311,9 +311,7 @@ func AssertUpdate(k8sClient *client.Client, dbClient *database.DatabaseClient, a compartStringMap(adbDetails.FreeformTags, ociADBDetails.FreeformTags) && compartString(adbDetails.SubnetOCID, ociADBDetails.SubnetOCID) && reflect.DeepEqual(adbDetails.NsgOCIDs, ociADBDetails.NsgOCIDs) && - compartString(adbDetails.PrivateEndpoint, ociADBDetails.PrivateEndpoint) && - compartString(adbDetails.PrivateEndpointLabel, ociADBDetails.PrivateEndpointLabel) && - compartString(adbDetails.PrivateEndpointIP, ociADBDetails.PrivateEndpointIP) + compartString(adbDetails.PrivateEndpointLabel, ociADBDetails.PrivateEndpointLabel) return same, nil }, updateTimeout, updateInterval).Should(BeTrue()) From 6acd5c65f6a216d0ec2318fa0f72172b194791ee Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Thu, 2 Dec 2021 13:47:50 -0500 Subject: [PATCH 065/628] remove unused attributes --- test/e2e/behavior/shared_behaviors.go | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/test/e2e/behavior/shared_behaviors.go b/test/e2e/behavior/shared_behaviors.go index 9adb7121..a54e4e3a 100644 --- a/test/e2e/behavior/shared_behaviors.go +++ b/test/e2e/behavior/shared_behaviors.go @@ -311,9 +311,7 @@ func AssertUpdate(k8sClient *client.Client, dbClient *database.DatabaseClient, a compartStringMap(adbDetails.FreeformTags, ociADBDetails.FreeformTags) && compartString(adbDetails.SubnetOCID, ociADBDetails.SubnetOCID) && reflect.DeepEqual(adbDetails.NsgOCIDs, ociADBDetails.NsgOCIDs) && - compartString(adbDetails.PrivateEndpoint, ociADBDetails.PrivateEndpoint) && - compartString(adbDetails.PrivateEndpointLabel, ociADBDetails.PrivateEndpointLabel) && - compartString(adbDetails.PrivateEndpointIP, ociADBDetails.PrivateEndpointIP) + compartString(adbDetails.PrivateEndpointLabel, ociADBDetails.PrivateEndpointLabel) return same, nil }, updateTimeout, updateInterval).Should(BeTrue()) From fa35e44358b34ca24889ecdf50d63160e2b1321b Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Thu, 2 Dec 2021 13:59:18 -0500 Subject: [PATCH 066/628] remove controller-gen bin/ From b792d90484eb19c44db41c5e937e4262e3eab686 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Thu, 2 Dec 2021 14:17:22 -0500 Subject: [PATCH 067/628] bin/ From d5883407315c064066150c8d7110e5279d580945 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Thu, 2 Dec 2021 14:00:13 -0500 Subject: [PATCH 068/628] update base .yaml --- .../database.oracle.com_autonomousdatabases.yaml | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/config/crd/bases/database.oracle.com_autonomousdatabases.yaml b/config/crd/bases/database.oracle.com_autonomousdatabases.yaml index 51b0b6ab..dbce1bfa 100644 --- a/config/crd/bases/database.oracle.com_autonomousdatabases.yaml +++ b/config/crd/bases/database.oracle.com_autonomousdatabases.yaml @@ -101,10 +101,14 @@ spec: additionalProperties: type: string type: object + isAccessControlEnabled: + type: boolean isAutoScalingEnabled: type: boolean isDedicated: type: boolean + isMTLSConnectionRequired: + type: boolean lifecycleState: description: 'AutonomousDatabaseLifecycleStateEnum Enum with underlying type: string' @@ -113,10 +117,6 @@ spec: items: type: string type: array - privateEndpoint: - type: string - privateEndpointIP: - type: string privateEndpointLabel: type: string subnetOCID: @@ -133,6 +133,10 @@ spec: type: string type: object type: object + whitelistedIPs: + items: + type: string + type: array type: object hardLink: default: false From 320e3ffc9f33dc2f82368c715b8c1a5d7ac00a86 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Thu, 2 Dec 2021 14:00:13 -0500 Subject: [PATCH 069/628] update base .yaml --- .../database.oracle.com_autonomousdatabases.yaml | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/config/crd/bases/database.oracle.com_autonomousdatabases.yaml b/config/crd/bases/database.oracle.com_autonomousdatabases.yaml index 51b0b6ab..dbce1bfa 100644 --- a/config/crd/bases/database.oracle.com_autonomousdatabases.yaml +++ b/config/crd/bases/database.oracle.com_autonomousdatabases.yaml @@ -101,10 +101,14 @@ spec: additionalProperties: type: string type: object + isAccessControlEnabled: + type: boolean isAutoScalingEnabled: type: boolean isDedicated: type: boolean + isMTLSConnectionRequired: + type: boolean lifecycleState: description: 'AutonomousDatabaseLifecycleStateEnum Enum with underlying type: string' @@ -113,10 +117,6 @@ spec: items: type: string type: array - privateEndpoint: - type: string - privateEndpointIP: - type: string privateEndpointLabel: type: string subnetOCID: @@ -133,6 +133,10 @@ spec: type: string type: object type: object + whitelistedIPs: + items: + type: string + type: array type: object hardLink: default: false From 771f50cf0b5484ca1ca43342d59fb1e9c53be8de Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Mon, 6 Dec 2021 11:07:39 -0500 Subject: [PATCH 070/628] update incorrect function name --- .../database/autonomousdatabase_controller.go | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/controllers/database/autonomousdatabase_controller.go b/controllers/database/autonomousdatabase_controller.go index 8060147b..7f364e5d 100644 --- a/controllers/database/autonomousdatabase_controller.go +++ b/controllers/database/autonomousdatabase_controller.go @@ -446,7 +446,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue @@ -455,7 +455,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R if oneWayTLSResp.OpcWorkRequestId != nil { // Update status.state adb.Status.LifecycleState = oneWayTLSResp.AutonomousDatabase.LifecycleState - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } @@ -464,7 +464,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } } @@ -477,7 +477,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue @@ -486,7 +486,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R if networkResp.OpcWorkRequestId != nil { // Update status.state adb.Status.LifecycleState = networkResp.AutonomousDatabase.LifecycleState - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } @@ -495,7 +495,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } } From 96542945c8d0f7a003a5ccea55f32a46f9b50e0c Mon Sep 17 00:00:00 2001 From: Gajanan Bhat Date: Tue, 7 Dec 2021 11:23:09 +0530 Subject: [PATCH 071/628] Add ReplicaSet for CDB --- .gitignore | 1 + PROJECT | 4 +- apis/database/v1alpha1/cdb_webhook.go | 30 +- config/crd/patches/cainjection_in_cdbs.yaml | 2 +- config/crd/patches/cainjection_in_pdbs.yaml | 2 +- config/crd/patches/webhook_in_cdbs.yaml | 2 +- config/crd/patches/webhook_in_pdbs.yaml | 2 +- config/rbac/role.yaml | 13 + controllers/database/cdb_controller.go | 293 ++++++- oracle-database-operator.yaml | 900 ++++++++++++++++++-- ords/cdbadmin.properties.tmpl | 2 +- 11 files changed, 1178 insertions(+), 73 deletions(-) diff --git a/.gitignore b/.gitignore index 8a2f9064..fd8f5261 100644 --- a/.gitignore +++ b/.gitignore @@ -3,3 +3,4 @@ cover.out bin testbin/* onpremtest/* +ords/*zip diff --git a/PROJECT b/PROJECT index 07cf84c7..ec188cae 100644 --- a/PROJECT +++ b/PROJECT @@ -50,7 +50,7 @@ resources: webhooks: defaulting: true validation: true - webhookVersion: v1beta1 + webhookVersion: v1 - api: crdVersion: v1 namespaced: true @@ -63,5 +63,5 @@ resources: webhooks: defaulting: true validation: true - webhookVersion: v1beta1 + webhookVersion: v1 version: "3" diff --git a/apis/database/v1alpha1/cdb_webhook.go b/apis/database/v1alpha1/cdb_webhook.go index cd84c36f..d18cfc8a 100644 --- a/apis/database/v1alpha1/cdb_webhook.go +++ b/apis/database/v1alpha1/cdb_webhook.go @@ -40,7 +40,7 @@ package v1alpha1 import ( "reflect" - //"strings" + "strings" apierrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/runtime" @@ -77,6 +77,12 @@ func (r *CDB) Default() { if r.Spec.Replicas == 0 { r.Spec.Replicas = 1 } + + cdblog.Info("CDB phase : " + r.Status.Phase) + if r.Status.Phase == "Ready" { + r.Status.Status = false + r.Status.Phase = "CreatingPod" + } } // TODO(user): change verbs to "verbs=create;update;delete" if you want to enable deletion validation. @@ -143,8 +149,26 @@ func (r *CDB) ValidateCreate() error { func (r *CDB) ValidateUpdate(old runtime.Object) error { cdblog.Info("validate update", "name", r.Name) - // TODO(user): fill in your validation logic upon object update. - return nil + var allErrs field.ErrorList + + // Check for updation errors + oldCDB, ok := old.(*CDB) + if !ok { + return nil + } + + if !strings.EqualFold(oldCDB.Spec.ServiceName, r.Spec.ServiceName) { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("serviceName"), "cannot be changed")) + } + + if len(allErrs) == 0 { + return nil + } + + return apierrors.NewInvalid( + schema.GroupKind{Group: "database.oracle.com", Kind: "CDB"}, + r.Name, allErrs) } // ValidateDelete implements webhook.Validator so a webhook will be registered for the type diff --git a/config/crd/patches/cainjection_in_cdbs.yaml b/config/crd/patches/cainjection_in_cdbs.yaml index 5ab5efe9..8cb50343 100644 --- a/config/crd/patches/cainjection_in_cdbs.yaml +++ b/config/crd/patches/cainjection_in_cdbs.yaml @@ -1,6 +1,6 @@ # The following patch adds a directive for certmanager to inject CA into the CRD # CRD conversion requires k8s 1.13 or later. -apiVersion: apiextensions.k8s.io/v1beta1 +apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: diff --git a/config/crd/patches/cainjection_in_pdbs.yaml b/config/crd/patches/cainjection_in_pdbs.yaml index 3b1bdcbc..8c41010a 100644 --- a/config/crd/patches/cainjection_in_pdbs.yaml +++ b/config/crd/patches/cainjection_in_pdbs.yaml @@ -1,6 +1,6 @@ # The following patch adds a directive for certmanager to inject CA into the CRD # CRD conversion requires k8s 1.13 or later. -apiVersion: apiextensions.k8s.io/v1beta1 +apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: diff --git a/config/crd/patches/webhook_in_cdbs.yaml b/config/crd/patches/webhook_in_cdbs.yaml index d3700966..9283c020 100644 --- a/config/crd/patches/webhook_in_cdbs.yaml +++ b/config/crd/patches/webhook_in_cdbs.yaml @@ -1,6 +1,6 @@ # The following patch enables conversion webhook for CRD # CRD conversion requires k8s 1.13 or later. -apiVersion: apiextensions.k8s.io/v1beta1 +apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: name: cdbs.database.oracle.com diff --git a/config/crd/patches/webhook_in_pdbs.yaml b/config/crd/patches/webhook_in_pdbs.yaml index f0012581..2c41e439 100644 --- a/config/crd/patches/webhook_in_pdbs.yaml +++ b/config/crd/patches/webhook_in_pdbs.yaml @@ -1,6 +1,6 @@ # The following patch enables conversion webhook for CRD # CRD conversion requires k8s 1.13 or later. -apiVersion: apiextensions.k8s.io/v1beta1 +apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: name: pdbs.database.oracle.com diff --git a/config/rbac/role.yaml b/config/rbac/role.yaml index 2e49c9e6..117067cf 100644 --- a/config/rbac/role.yaml +++ b/config/rbac/role.yaml @@ -14,6 +14,7 @@ rules: - pods - pods/exec - pods/log + - replicasets - services verbs: - create @@ -66,6 +67,18 @@ rules: - patch - update - watch +- apiGroups: + - apps + resources: + - replicasets + verbs: + - create + - delete + - get + - list + - patch + - update + - watch - apiGroups: - apps resources: diff --git a/controllers/database/cdb_controller.go b/controllers/database/cdb_controller.go index 02176927..b3eb243c 100644 --- a/controllers/database/cdb_controller.go +++ b/controllers/database/cdb_controller.go @@ -47,6 +47,7 @@ import ( "time" "github.com/go-logr/logr" + appsv1 "k8s.io/api/apps/v1" corev1 "k8s.io/api/core/v1" apierrors "k8s.io/apimachinery/pkg/api/errors" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" @@ -91,8 +92,9 @@ const CDBFinalizer = "database.oracle.com/CDBfinalizer" //+kubebuilder:rbac:groups=database.oracle.com,resources=cdbs,verbs=get;list;watch;create;update;patch;delete //+kubebuilder:rbac:groups=database.oracle.com,resources=cdbs/status,verbs=get;update;patch //+kubebuilder:rbac:groups=database.oracle.com,resources=cdbs/finalizers,verbs=update -//+kubebuilder:rbac:groups="",resources=pods;pods/log;pods/exec;services;configmaps;events,verbs=create;delete;get;list;patch;update;watch +//+kubebuilder:rbac:groups="",resources=pods;pods/log;pods/exec;services;configmaps;events;replicasets,verbs=create;delete;get;list;patch;update;watch //+kubebuilder:rbac:groups=core,resources=pods;secrets;services;configmaps;namespaces,verbs=get;list;watch;create;update;patch;delete +//+kubebuilder:rbac:groups=apps,resources=replicasets,verbs=get;list;watch;create;update;patch;delete // Reconcile is part of the main kubernetes reconciliation loop which aims to // move the current state of the cluster closer to the desired state. @@ -148,6 +150,10 @@ func (r *CDBReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.R return requeueY, nil } + if (cdb.Status.Phase == cdbPhaseReady) && cdb.Status.Status { + r.evaluateSpecChange(ctx, req, cdb) + } + if !cdb.Status.Status { phase := cdb.Status.Phase log.Info("Current Phase:"+phase, "Name", cdb.Name) @@ -157,7 +163,8 @@ func (r *CDBReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.R cdb.Status.Phase = cdbPhasePod case cdbPhasePod: // Create ORDS POD - err = r.createORDSPod(ctx, req, cdb) + //err = r.createORDSPod(ctx, req, cdb) + err = r.createORDSReplicaSet(ctx, req, cdb) if err != nil { log.Info("Reconcile queued") return requeueY, nil @@ -178,7 +185,8 @@ func (r *CDBReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.R log.Info("Reconcile queued") return requeueY, nil } - cdb.Status.Phase = cdbPhaseSecrets + //cdb.Status.Phase = cdbPhaseSecrets + cdb.Status.Phase = cdbPhaseReady case cdbPhaseSecrets: // Delete CDB Secrets //r.deleteSecrets(ctx, req, cdb) @@ -206,7 +214,7 @@ func (r *CDBReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.R /************************************************* * Create a Pod based on the ORDS container /************************************************/ -func (r *CDBReconciler) createORDSPod(ctx context.Context, req ctrl.Request, cdb *dbapi.CDB) error { +/*func (r *CDBReconciler) createORDSPod(ctx context.Context, req ctrl.Request, cdb *dbapi.CDB) error { log := r.Log.WithValues("createORDSPod", req.NamespacedName) @@ -225,6 +233,72 @@ func (r *CDBReconciler) createORDSPod(ctx context.Context, req ctrl.Request, cdb r.Recorder.Eventf(cdb, corev1.EventTypeNormal, "CreatedORDSPod", "Created %s ORDS Pod(s) for %s", strconv.Itoa(cdb.Spec.Replicas), cdb.Name) return nil } +*/ +/********************************************************** + * Create a ReplicaSet for pods based on the ORDS container + /********************************************************/ +func (r *CDBReconciler) createORDSReplicaSet(ctx context.Context, req ctrl.Request, cdb *dbapi.CDB) error { + + log := r.Log.WithValues("createORDSReplicaSet", req.NamespacedName) + + replicas := int32(cdb.Spec.Replicas) + podSpec := r.createPodSpec(cdb) + + replicaSet := &appsv1.ReplicaSet{ + ObjectMeta: metav1.ObjectMeta{ + Name: cdb.Name + "-ords-rs", + Namespace: cdb.Namespace, + Labels: map[string]string{ + "name": cdb.Name + "-ords-rs", + }, + }, + Spec: appsv1.ReplicaSetSpec{ + Replicas: &replicas, + Template: corev1.PodTemplateSpec{ + ObjectMeta: metav1.ObjectMeta{ + Name: cdb.Name + "-ords", + Namespace: cdb.Namespace, + Labels: map[string]string{ + "name": cdb.Name + "-ords", + }, + }, + Spec: podSpec, + }, + Selector: &metav1.LabelSelector{ + MatchLabels: map[string]string{ + "name": cdb.Name + "-ords", + }, + }, + }, + } + + foundRS := &appsv1.ReplicaSet{} + err := r.Get(context.TODO(), types.NamespacedName{Name: replicaSet.Name, Namespace: cdb.Namespace}, foundRS) + if err != nil && apierrors.IsNotFound(err) { + log.Info("Creating ORDS Replicaset: " + replicaSet.Name) + err = r.Create(ctx, replicaSet) + if err != nil { + log.Error(err, "Failed to create ReplicaSet for :"+cdb.Name, "Namespace", replicaSet.Namespace, "Name", replicaSet.Name) + return err + } + } else { + log.Info("Found Replicas: " + strconv.Itoa(int(*(foundRS.Spec.Replicas))) + ", New Replicas: " + strconv.Itoa(int(*(replicaSet.Spec.Replicas)))) + foundRS.Spec = replicaSet.Spec + err = r.Update(ctx, foundRS) + //err = r.Update(ctx, replicaSet) + if err != nil { + log.Error(err, "Failed to update ReplicaSet for :"+cdb.Name, "Namespace", replicaSet.Namespace, "Name", replicaSet.Name) + return err + } + } + + // Set CDB instance as the owner and controller + ctrl.SetControllerReference(cdb, replicaSet, r.Scheme) + + log.Info("Created ORDS ReplicaSet successfully") + r.Recorder.Eventf(cdb, corev1.EventTypeNormal, "CreatedORDSReplicaSet", "Created ORDS Replicaset (Replicas - %s) for %s", strconv.Itoa(cdb.Spec.Replicas), cdb.Name) + return nil +} /************************************************* * Validate ORDS Pod. Check if there are any errors @@ -276,8 +350,168 @@ func (r *CDBReconciler) validateORDSPod(ctx context.Context, req ctrl.Request, c /************************ * Create Pod spec - /************************/ -func (r *CDBReconciler) createPodSpec(cdb *dbapi.CDB) *corev1.Pod { +/************************/ +func (r *CDBReconciler) createPodSpec(cdb *dbapi.CDB) corev1.PodSpec { + + podSpec := corev1.PodSpec{ + Volumes: []corev1.Volume{{ + Name: "secrets", + VolumeSource: corev1.VolumeSource{ + Projected: &corev1.ProjectedVolumeSource{ + DefaultMode: func() *int32 { i := int32(0666); return &i }(), + Sources: []corev1.VolumeProjection{ + { + Secret: &corev1.SecretProjection{ + LocalObjectReference: corev1.LocalObjectReference{ + Name: cdb.Spec.SysAdminPwd.Secret.SecretName, + }, + }, + }, + { + Secret: &corev1.SecretProjection{ + LocalObjectReference: corev1.LocalObjectReference{ + Name: cdb.Spec.CDBAdminUser.Secret.SecretName, + }, + }, + }, + { + Secret: &corev1.SecretProjection{ + LocalObjectReference: corev1.LocalObjectReference{ + Name: cdb.Spec.CDBAdminPwd.Secret.SecretName, + }, + }, + }, + { + Secret: &corev1.SecretProjection{ + LocalObjectReference: corev1.LocalObjectReference{ + Name: cdb.Spec.ORDSPwd.Secret.SecretName, + }, + }, + }, + { + Secret: &corev1.SecretProjection{ + LocalObjectReference: corev1.LocalObjectReference{ + Name: cdb.Spec.WebServerUser.Secret.SecretName, + }, + }, + }, + { + Secret: &corev1.SecretProjection{ + LocalObjectReference: corev1.LocalObjectReference{ + Name: cdb.Spec.WebServerPwd.Secret.SecretName, + }, + }, + }, + }, + }, + }, + }}, + Containers: []corev1.Container{{ + Name: cdb.Name + "-ords", + Image: cdb.Spec.ORDSImage, + VolumeMounts: []corev1.VolumeMount{{ + MountPath: "/opt/oracle/ords/secrets", + Name: "secrets", + ReadOnly: true, + }}, + Env: func() []corev1.EnvVar { + return []corev1.EnvVar{ + { + Name: "ORACLE_HOST", + Value: cdb.Spec.DBServer, + }, + { + Name: "ORACLE_PORT", + Value: strconv.Itoa(cdb.Spec.DBPort), + }, + { + Name: "ORDS_PORT", + Value: strconv.Itoa(cdb.Spec.ORDSPort), + }, + { + Name: "ORACLE_SERVICE", + Value: cdb.Spec.ServiceName, + }, + { + Name: "ORACLE_PWD_KEY", + Value: cdb.Spec.SysAdminPwd.Secret.Key, + }, + { + Name: "CDBADMIN_USER_KEY", + Value: cdb.Spec.CDBAdminUser.Secret.Key, + }, + { + Name: "CDBADMIN_PWD_KEY", + Value: cdb.Spec.CDBAdminPwd.Secret.Key, + }, + { + Name: "ORDS_PWD_KEY", + Value: cdb.Spec.ORDSPwd.Secret.Key, + }, + { + Name: "WEBSERVER_USER_KEY", + Value: cdb.Spec.WebServerUser.Secret.Key, + }, + { + Name: "WEBSERVER_PASSWORD_KEY", + Value: cdb.Spec.WebServerPwd.Secret.Key, + }, + } + }(), + }}, + + NodeSelector: func() map[string]string { + ns := make(map[string]string) + if len(cdb.Spec.NodeSelector) != 0 { + for key, value := range cdb.Spec.NodeSelector { + ns[key] = value + } + } + return ns + }(), + } + + if len(cdb.Spec.ORDSImagePullSecret) > 0 { + podSpec.ImagePullSecrets = []corev1.LocalObjectReference{ + { + Name: cdb.Spec.ORDSImagePullSecret, + }, + } + } + + podSpec.Containers[0].ImagePullPolicy = corev1.PullAlways + + if len(cdb.Spec.ORDSImagePullPolicy) > 0 { + if strings.ToUpper(cdb.Spec.ORDSImagePullPolicy) == "NEVER" { + podSpec.Containers[0].ImagePullPolicy = corev1.PullNever + } + } + + return podSpec +} + +/********************************************************** + * Evaluate change in Spec post creation and instantiation + /********************************************************/ +func (r *CDBReconciler) evaluateSpecChange(ctx context.Context, req ctrl.Request, cdb *dbapi.CDB) error { + log := r.Log.WithValues("evaluateSpecChange", req.NamespacedName) + + currCDB := &dbapi.CDB{} + + err := r.Get(context.TODO(), types.NamespacedName{Name: cdb.Name, Namespace: cdb.Namespace}, currCDB) + if err != nil && apierrors.IsNotFound(err) { + log.Info("Uanble to get curr CDB CR: " + cdb.Name) + return err + } + + log.Info("Existing replicas: " + strconv.Itoa(currCDB.Spec.Replicas) + ", New Replicas: " + strconv.Itoa(cdb.Spec.Replicas)) + return nil +} + +/************************ + * Create Pod spec +/************************/ +func (r *CDBReconciler) createPodSpec2(cdb *dbapi.CDB) *corev1.Pod { pod := &corev1.Pod{ TypeMeta: metav1.TypeMeta{ @@ -533,7 +767,7 @@ func (r *CDBReconciler) manageCDBDeletion(ctx context.Context, req ctrl.Request, /************************************************* * Delete CDB Resource - /************************************************/ +/************************************************/ func (r *CDBReconciler) deleteCDBInstance(ctx context.Context, req ctrl.Request, cdb *dbapi.CDB) error { log := r.Log.WithValues("deleteCDBInstance", req.NamespacedName) @@ -543,6 +777,49 @@ func (r *CDBReconciler) deleteCDBInstance(ctx context.Context, req ctrl.Request, log.Error(err, "Kubernetes Config Error") } + replicaSetName := cdb.Name + "-ords-rs" + + err = k_client.AppsV1().ReplicaSets(cdb.Namespace).Delete(context.TODO(), replicaSetName, metav1.DeleteOptions{}) + if err != nil { + log.Info("Could not delete ReplicaSet", "RS Name", replicaSetName, "err", err.Error()) + if !strings.Contains(strings.ToUpper(err.Error()), "NOT FOUND") { + return err + } + } else { + log.Info("Successfully deleted ORDS ReplicaSet", "RS Name", replicaSetName) + } + + r.Recorder.Eventf(cdb, corev1.EventTypeNormal, "DeletedORDSReplicaSet", "Deleted ORDS ReplicaSet for %s", cdb.Name) + + svcName := cdb.Name + "-ords" + + err = k_client.CoreV1().Services(cdb.Namespace).Delete(context.TODO(), svcName, metav1.DeleteOptions{}) + if err != nil { + log.Info("Could not delete Service", "Service Name", svcName, "err", err.Error()) + if !strings.Contains(strings.ToUpper(err.Error()), "NOT FOUND") { + return err + } + } else { + r.Recorder.Eventf(cdb, corev1.EventTypeNormal, "DeletedORDSService", "Deleted ORDS Service for %s", cdb.Name) + log.Info("Successfully deleted ORDS Service", "Service Name", svcName) + } + + log.Info("Successfully deleted CDB resource", "CDB Name", cdb.Spec.CDBName) + return nil +} + +/************************************************* + * Delete CDB Resource +/************************************************/ +func (r *CDBReconciler) deleteCDBInstance2(ctx context.Context, req ctrl.Request, cdb *dbapi.CDB) error { + + log := r.Log.WithValues("deleteCDBInstance", req.NamespacedName) + + k_client, err := kubernetes.NewForConfig(r.Config) + if err != nil { + log.Error(err, "Kubernetes Config Error") + } + podName := cdb.Name + "-ords" podList := &corev1.PodList{} @@ -655,7 +932,7 @@ func (r *CDBReconciler) deleteSecrets(ctx context.Context, req ctrl.Request, cdb func (r *CDBReconciler) SetupWithManager(mgr ctrl.Manager) error { return ctrl.NewControllerManagedBy(mgr). For(&dbapi.CDB{}). - Owns(&corev1.Pod{}). //Watch for deleted pods owned by this controller + Owns(&appsv1.ReplicaSet{}). //Watch for deleted RS owned by this controller WithEventFilter(predicate.Funcs{ UpdateFunc: func(e event.UpdateEvent) bool { // Ignore updates to CR status in which case metadata.Generation does not change diff --git a/oracle-database-operator.yaml b/oracle-database-operator.yaml index ae464a57..b13cc06b 100644 --- a/oracle-database-operator.yaml +++ b/oracle-database-operator.yaml @@ -49,21 +49,28 @@ spec: name: v1alpha1 schema: openAPIV3Schema: - description: AutonomousDatabase is the Schema for the autonomousdatabases API + description: AutonomousDatabase is the Schema for the autonomousdatabases + API properties: apiVersion: - description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: - description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: - description: 'AutonomousDatabaseSpec defines the desired state of AutonomousDatabase Important: Run "make" to regenerate code after modifying this file' + description: 'AutonomousDatabaseSpec defines the desired state of AutonomousDatabase + Important: Run "make" to regenerate code after modifying this file' properties: details: - description: AutonomousDatabaseDetails defines the detail information of AutonomousDatabase, corresponding to oci-go-sdk/database/AutonomousDatabase + description: AutonomousDatabaseDetails defines the detail information + of AutonomousDatabase, corresponding to oci-go-sdk/database/AutonomousDatabase properties: adminPassword: properties: @@ -85,7 +92,8 @@ spec: dbVersion: type: string dbWorkload: - description: 'AutonomousDatabaseDbWorkloadEnum Enum with underlying type: string' + description: 'AutonomousDatabaseDbWorkloadEnum Enum with underlying + type: string' enum: - OLTP - DW @@ -103,7 +111,8 @@ spec: isDedicated: type: boolean lifecycleState: - description: 'AutonomousDatabaseLifecycleStateEnum Enum with underlying type: string' + description: 'AutonomousDatabaseLifecycleStateEnum Enum with underlying + type: string' type: string nsgOCIDs: items: @@ -151,15 +160,19 @@ spec: dataStorageSizeInTBs: type: integer dbWorkload: - description: 'AutonomousDatabaseDbWorkloadEnum Enum with underlying type: string' + description: 'AutonomousDatabaseDbWorkloadEnum Enum with underlying + type: string' type: string displayName: - description: 'INSERT ADDITIONAL STATUS FIELD - define observed state of cluster Important: Run "make" to regenerate code after modifying this file' + description: 'INSERT ADDITIONAL STATUS FIELD - define observed state + of cluster Important: Run "make" to regenerate code after modifying + this file' type: string isDedicated: type: string lifecycleState: - description: 'AutonomousDatabaseLifecycleStateEnum Enum with underlying type: string' + description: 'AutonomousDatabaseLifecycleStateEnum Enum with underlying + type: string' type: string timeCreated: type: string @@ -178,6 +191,499 @@ status: --- apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition +metadata: + annotations: + cert-manager.io/inject-ca-from: oracle-database-operator-system/oracle-database-operator-serving-cert + controller-gen.kubebuilder.io/version: v0.6.1 + name: cdbs.database.oracle.com +spec: + group: database.oracle.com + names: + kind: CDB + listKind: CDBList + plural: cdbs + singular: cdb + scope: Namespaced + versions: + - additionalPrinterColumns: + - description: Name of the CDB + jsonPath: .spec.cdbName + name: CDB Name + type: string + - description: ' Name of the DB Server' + jsonPath: .spec.dbServer + name: DB Server + type: string + - description: DB server port + jsonPath: .spec.dbPort + name: DB Port + type: integer + - description: SCAN Name + jsonPath: .spec.scanName + name: SCAN NAme + type: string + - description: Status of the CDB Resource + jsonPath: .status.phase + name: Status + type: string + - description: Error message, if any + jsonPath: .status.msg + name: Message + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + description: CDB is the Schema for the cdbs API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: CDBSpec defines the desired state of CDB + properties: + cdbAdminPwd: + description: Password for the CDB Administrator to manage PDB lifecycle + properties: + secret: + description: CDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + cdbAdminUser: + description: User in the root container with sysdba priviledges to + manage PDB lifecycle + properties: + secret: + description: CDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + cdbName: + description: Name of the CDB + type: string + dbPort: + description: DB server port + type: integer + dbServer: + description: Name of the DB server + type: string + nodeSelector: + additionalProperties: + type: string + description: Node Selector for running the Pod + type: object + ordsImage: + description: ORDS Image Name + type: string + ordsImagePullPolicy: + description: ORDS Image Pull Policy + enum: + - Always + - Never + type: string + ordsImagePullSecret: + description: The name of the image pull secret in case of a private + docker repository. + type: string + ordsPort: + description: ORDS server port. For now, keep it as 8888. TO BE USED + IN FUTURE RELEASE. + type: integer + ordsPwd: + description: Password for user ORDS_PUBLIC_USER + properties: + secret: + description: CDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + replicas: + description: Number of ORDS Containers to create + type: integer + scanName: + description: SCAN Name + type: string + serviceName: + description: Name of the CDB Service + type: string + sysAdminPwd: + description: Password for the CDB System Administrator + properties: + secret: + description: CDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + webServerPwd: + description: Password for the Web Server User + properties: + secret: + description: CDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + webServerUser: + description: Web Server User with SQL Administrator role to allow + us to authenticate to the PDB Lifecycle Management REST endpoints + properties: + secret: + description: CDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + type: object + status: + description: CDBStatus defines the observed state of CDB + properties: + msg: + description: Message + type: string + phase: + description: Phase of the CDB Resource + type: string + status: + description: CDB Resource Status + type: boolean + required: + - phase + - status + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + cert-manager.io/inject-ca-from: oracle-database-operator-system/oracle-database-operator-serving-cert + controller-gen.kubebuilder.io/version: v0.6.1 + name: pdbs.database.oracle.com +spec: + group: database.oracle.com + names: + kind: PDB + listKind: PDBList + plural: pdbs + singular: pdb + scope: Namespaced + versions: + - additionalPrinterColumns: + - description: The connect string to be used + jsonPath: .status.connString + name: Connect String + type: string + - description: Name of the CDB + jsonPath: .spec.cdbName + name: CDB Name + type: string + - description: Name of the PDB + jsonPath: .spec.pdbName + name: PDB Name + type: string + - description: Total Size of the PDB + jsonPath: .spec.totalSize + name: PDB Size + type: string + - description: Status of the PDB + jsonPath: .status.phase + name: Status + type: string + - description: Error message, if any + jsonPath: .status.msg + name: Message + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + description: PDB is the Schema for the pdbs API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: PDBSpec defines the desired state of PDB + properties: + action: + description: 'Action to be taken: Create or Clone or Plug or Unplug' + enum: + - Create + - Clone + - Plug + - Unplug + - Delete + type: string + adminName: + description: The administrator username for the new PDB. This property + is required when the Action property is Create. + properties: + secret: + description: PDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + adminPwd: + description: The administrator password for the new PDB. This property + is required when the Action property is Create. + properties: + secret: + description: PDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + asClone: + description: Indicate if 'AS CLONE' option should be used in the command + to plug in a PDB. This property is applicable when the Action property + is PLUG but not required. + type: boolean + cdbName: + description: Name of the CDB + type: string + cdbResName: + description: Name of the CDB Custom Resource that runs the ORDS container + type: string + copyAction: + description: To copy files or not while cloning a PDB + enum: + - COPY + - NOCOPY + - MOVE + type: string + dropAction: + description: Specify if datafiles should be removed or not. The value + can be INCLUDING or KEEP (default). + enum: + - INCLUDING + - KEEP + type: string + fileNameConversions: + description: Relevant for Create and Plug operations. As defined in + the Oracle Multitenant Database documentation. Values can be a + filename convert pattern or NONE. + type: string + getScript: + description: Whether you need the script only or execute the script + type: boolean + pdbName: + description: The name of the new PDB. Relevant for both Create and + Plug Actions. + type: string + reuseTempFile: + description: Whether to reuse temp file + type: boolean + sourceFileNameConversions: + description: This property is required when the Action property is + Plug. As defined in the Oracle Multitenant Database documentation. + Values can be a source filename convert pattern or NONE. + type: string + sparseClonePath: + description: A Path specified for sparse clone snapshot copy. (Optional) + type: string + srcPdbName: + description: Name of the Source PDB from which to clone + type: string + tdeExport: + description: TDE export for unplug operations + type: boolean + tdeImport: + description: TDE import for plug operations + type: boolean + tdeKeystorePath: + description: TDE keystore path is required if the tdeImport or tdeExport + flag is set to true. Can be used in plug or unplug operations. + type: string + tdePassword: + description: TDE password if the tdeImport or tdeExport flag is set + to true. Can be used in create, plug or unplug operations + properties: + secret: + description: PDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + tdeSecret: + description: TDE secret is required if the tdeImport or tdeExport + flag is set to true. Can be used in plug or unplug operations. + properties: + secret: + description: PDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + tempSize: + description: Relevant for Create and Clone operations. Total size + for temporary tablespace as defined in the Oracle Multitenant Database + documentation. See size_clause description in Database SQL Language + Reference documentation. + type: string + totalSize: + description: Relevant for create and plug operations. Total size as + defined in the Oracle Multitenant Database documentation. See size_clause + description in Database SQL Language Reference documentation. + type: string + unlimitedStorage: + description: Relevant for Create and Plug operations. True for unlimited + storage. Even when set to true, totalSize and tempSize MUST be specified + in the request if Action is Create. + type: boolean + xmlFileName: + description: XML metadata filename to be used for Plug or Unplug operations + type: string + required: + - action + type: object + status: + description: PDBStatus defines the observed state of PDB + properties: + action: + description: Last Completed Action + type: string + connString: + description: PDB Connect String + type: string + msg: + description: Message + type: string + phase: + description: Phase of the CDB Resource + type: string + status: + description: CDB Resource Status + type: boolean + required: + - phase + - status + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition metadata: annotations: controller-gen.kubebuilder.io/version: v0.6.1 @@ -198,10 +704,14 @@ spec: description: ShardingDatabase is the Schema for the shardingdatabases API properties: apiVersion: - description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: - description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object @@ -214,7 +724,8 @@ spec: properties: envVars: items: - description: EnvironmentVariable represents a named variable accessible for containers. + description: EnvironmentVariable represents a named variable + accessible for containers. properties: name: type: string @@ -226,7 +737,8 @@ spec: type: object type: array imagePullPolicy: - description: PullPolicy describes a policy for if/when to pull a container image + description: PullPolicy describes a policy for if/when to pull + a container image type: string isDelete: type: boolean @@ -249,7 +761,8 @@ spec: pvcName: type: string resources: - description: ResourceRequirements describes the compute resource requirements. + description: ResourceRequirements describes the compute resource + requirements. properties: limits: additionalProperties: @@ -258,7 +771,8 @@ spec: - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true - description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + description: 'Limits describes the maximum amount of compute + resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' type: object requests: additionalProperties: @@ -267,7 +781,11 @@ spec: - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true - description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + description: 'Requests describes the minimum amount of compute + resources required. If Requests is omitted for a container, + it defaults to Limits if that is explicitly specified, + otherwise to an implementation-defined value. More info: + https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' type: object type: object storageSizeInGb: @@ -287,7 +805,8 @@ spec: properties: envVars: items: - description: EnvironmentVariable represents a named variable accessible for containers. + description: EnvironmentVariable represents a named variable + accessible for containers. properties: name: type: string @@ -299,7 +818,8 @@ spec: type: object type: array imagePullPolicy: - description: PullPolicy describes a policy for if/when to pull a container image + description: PullPolicy describes a policy for if/when to pull + a container image type: string isDelete: type: boolean @@ -321,7 +841,8 @@ spec: format: int32 type: integer resources: - description: ResourceRequirements describes the compute resource requirements. + description: ResourceRequirements describes the compute resource + requirements. properties: limits: additionalProperties: @@ -330,7 +851,8 @@ spec: - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true - description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + description: 'Limits describes the maximum amount of compute + resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' type: object requests: additionalProperties: @@ -339,7 +861,11 @@ spec: - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true - description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + description: 'Requests describes the minimum amount of compute + resources required. If Requests is omitted for a container, + it defaults to Limits if that is explicitly specified, + otherwise to an implementation-defined value. More info: + https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' type: object type: object storageSizeInGb: @@ -371,7 +897,8 @@ spec: type: string portMappings: items: - description: PortMapping is a specification of port mapping for an application deployment. + description: PortMapping is a specification of port mapping for + an application deployment. properties: port: format: int32 @@ -393,13 +920,16 @@ spec: secret: type: string shard: - description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster Important: Run "make" to regenerate code after modifying this file' + description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster + Important: Run "make" to regenerate code after modifying this file' items: - description: ShardSpec is a specification of Shards for an application deployment. + description: ShardSpec is a specification of Shards for an application + deployment. properties: envVars: items: - description: EnvironmentVariable represents a named variable accessible for containers. + description: EnvironmentVariable represents a named variable + accessible for containers. properties: name: type: string @@ -411,7 +941,8 @@ spec: type: object type: array imagePullPolicy: - description: PullPolicy describes a policy for if/when to pull a container image + description: PullPolicy describes a policy for if/when to pull + a container image type: string isDelete: type: boolean @@ -434,7 +965,8 @@ spec: pvcName: type: string resources: - description: ResourceRequirements describes the compute resource requirements. + description: ResourceRequirements describes the compute resource + requirements. properties: limits: additionalProperties: @@ -443,7 +975,8 @@ spec: - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true - description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + description: 'Limits describes the maximum amount of compute + resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' type: object requests: additionalProperties: @@ -452,7 +985,11 @@ spec: - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true - description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + description: 'Requests describes the minimum amount of compute + resources required. If Requests is omitted for a container, + it defaults to Limits if that is explicitly specified, + otherwise to an implementation-defined value. More info: + https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' type: object type: object storageSizeInGb: @@ -475,7 +1012,8 @@ spec: - shard type: object status: - description: To understand Metav1.Condition, please refer the link https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1 ShardingDatabaseStatus defines the observed state of ShardingDatabase + description: To understand Metav1.Condition, please refer the link https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1 + ShardingDatabaseStatus defines the observed state of ShardingDatabase properties: catalogs: additionalProperties: @@ -483,23 +1021,45 @@ spec: type: object conditions: items: - description: "Condition contains details for one aspect of the current state of this API Resource. --- This struct is intended for direct use as an array at the field path .status.conditions. For example, type FooStatus struct{ // Represents the observations of a foo's current state. // Known .status.conditions.type are: \"Available\", \"Progressing\", and \"Degraded\" // +patchMergeKey=type // +patchStrategy=merge // +listType=map // +listMapKey=type Conditions []metav1.Condition `json:\"conditions,omitempty\" patchStrategy:\"merge\" patchMergeKey:\"type\" protobuf:\"bytes,1,rep,name=conditions\"` \n // other fields }" + description: "Condition contains details for one aspect of the current + state of this API Resource. --- This struct is intended for direct + use as an array at the field path .status.conditions. For example, + type FooStatus struct{ // Represents the observations of a + foo's current state. // Known .status.conditions.type are: + \"Available\", \"Progressing\", and \"Degraded\" // +patchMergeKey=type + \ // +patchStrategy=merge // +listType=map // +listMapKey=type + \ Conditions []metav1.Condition `json:\"conditions,omitempty\" + patchStrategy:\"merge\" patchMergeKey:\"type\" protobuf:\"bytes,1,rep,name=conditions\"` + \n // other fields }" properties: lastTransitionTime: - description: lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + description: lastTransitionTime is the last time the condition + transitioned from one status to another. This should be when + the underlying condition changed. If that is not known, then + using the time when the API field changed is acceptable. format: date-time type: string message: - description: message is a human readable message indicating details about the transition. This may be an empty string. + description: message is a human readable message indicating + details about the transition. This may be an empty string. maxLength: 32768 type: string observedGeneration: - description: observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance. + description: observedGeneration represents the .metadata.generation + that the condition was set based upon. For instance, if .metadata.generation + is currently 12, but the .status.conditions[x].observedGeneration + is 9, the condition is out of date with respect to the current + state of the instance. format: int64 minimum: 0 type: integer reason: - description: reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty. + description: reason contains a programmatic identifier indicating + the reason for the condition's last transition. Producers + of specific condition types may define expected values and + meanings for this field, and whether the values are considered + a guaranteed API. The value should be a CamelCase string. + This field may not be empty. maxLength: 1024 minLength: 1 pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ @@ -512,7 +1072,11 @@ spec: - Unknown type: string type: - description: type of condition in CamelCase or in foo.example.com/CamelCase. --- Many .condition.type values are consistent across resources like Available, but because arbitrary conditions can be useful (see .node.status.conditions), the ability to deconflict is important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) + description: type of condition in CamelCase or in foo.example.com/CamelCase. + --- Many .condition.type values are consistent across resources + like Available, but because arbitrary conditions can be useful + (see .node.status.conditions), the ability to deconflict is + important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) maxLength: 316 pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ type: string @@ -606,13 +1170,18 @@ spec: name: v1alpha1 schema: openAPIV3Schema: - description: SingleInstanceDatabase is the Schema for the singleinstancedatabases API + description: SingleInstanceDatabase is the Schema for the singleinstancedatabases + API properties: apiVersion: - description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: - description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object @@ -620,7 +1189,8 @@ spec: description: SingleInstanceDatabaseSpec defines the desired state of SingleInstanceDatabase properties: adminPassword: - description: SingleInsatnceAdminPassword defines the secret containing Admin Password mapped to secretKey for Database + description: SingleInsatnceAdminPassword defines the secret containing + Admin Password mapped to secretKey for Database properties: keepSecret: type: boolean @@ -648,7 +1218,8 @@ spec: forceLog: type: boolean image: - description: SingleInstanceDatabaseImage defines the Image source and pullSecrets for POD + description: SingleInstanceDatabaseImage defines the Image source + and pullSecrets for POD properties: pullFrom: type: string @@ -682,7 +1253,8 @@ spec: pdbName: type: string persistence: - description: SingleInstanceDatabasePersistence defines the storage size and class for PVC + description: SingleInstanceDatabasePersistence defines the storage + size and class for PVC properties: accessMode: enum: @@ -704,7 +1276,8 @@ spec: minimum: 1 type: integer sid: - description: SID can only have a-z , A-Z, 0-9 . It cant have any special characters + description: SID can only have a-z , A-Z, 0-9 . It cant have any special + characters pattern: ^[a-zA-Z0-9]+$ type: string required: @@ -714,7 +1287,8 @@ spec: - replicas type: object status: - description: SingleInstanceDatabaseStatus defines the observed state of SingleInstanceDatabase + description: SingleInstanceDatabaseStatus defines the observed state of + SingleInstanceDatabase properties: apexInstalled: type: boolean @@ -728,23 +1302,45 @@ spec: type: string conditions: items: - description: "Condition contains details for one aspect of the current state of this API Resource. --- This struct is intended for direct use as an array at the field path .status.conditions. For example, type FooStatus struct{ // Represents the observations of a foo's current state. // Known .status.conditions.type are: \"Available\", \"Progressing\", and \"Degraded\" // +patchMergeKey=type // +patchStrategy=merge // +listType=map // +listMapKey=type Conditions []metav1.Condition `json:\"conditions,omitempty\" patchStrategy:\"merge\" patchMergeKey:\"type\" protobuf:\"bytes,1,rep,name=conditions\"` \n // other fields }" + description: "Condition contains details for one aspect of the current + state of this API Resource. --- This struct is intended for direct + use as an array at the field path .status.conditions. For example, + type FooStatus struct{ // Represents the observations of a + foo's current state. // Known .status.conditions.type are: + \"Available\", \"Progressing\", and \"Degraded\" // +patchMergeKey=type + \ // +patchStrategy=merge // +listType=map // +listMapKey=type + \ Conditions []metav1.Condition `json:\"conditions,omitempty\" + patchStrategy:\"merge\" patchMergeKey:\"type\" protobuf:\"bytes,1,rep,name=conditions\"` + \n // other fields }" properties: lastTransitionTime: - description: lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + description: lastTransitionTime is the last time the condition + transitioned from one status to another. This should be when + the underlying condition changed. If that is not known, then + using the time when the API field changed is acceptable. format: date-time type: string message: - description: message is a human readable message indicating details about the transition. This may be an empty string. + description: message is a human readable message indicating + details about the transition. This may be an empty string. maxLength: 32768 type: string observedGeneration: - description: observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance. + description: observedGeneration represents the .metadata.generation + that the condition was set based upon. For instance, if .metadata.generation + is currently 12, but the .status.conditions[x].observedGeneration + is 9, the condition is out of date with respect to the current + state of the instance. format: int64 minimum: 0 type: integer reason: - description: reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty. + description: reason contains a programmatic identifier indicating + the reason for the condition's last transition. Producers + of specific condition types may define expected values and + meanings for this field, and whether the values are considered + a guaranteed API. The value should be a CamelCase string. + This field may not be empty. maxLength: 1024 minLength: 1 pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ @@ -757,7 +1353,11 @@ spec: - Unknown type: string type: - description: type of condition in CamelCase or in foo.example.com/CamelCase. --- Many .condition.type values are consistent across resources like Available, but because arbitrary conditions can be useful (see .node.status.conditions), the ability to deconflict is important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) + description: type of condition in CamelCase or in foo.example.com/CamelCase. + --- Many .condition.type values are consistent across resources + like Available, but because arbitrary conditions can be useful + (see .node.status.conditions), the ability to deconflict is + important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) maxLength: 316 pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ type: string @@ -813,7 +1413,8 @@ spec: pdbName: type: string persistence: - description: SingleInstanceDatabasePersistence defines the storage size and class for PVC + description: SingleInstanceDatabasePersistence defines the storage + size and class for PVC properties: accessMode: enum: @@ -914,6 +1515,24 @@ metadata: creationTimestamp: null name: oracle-database-operator-manager-role rules: +- apiGroups: + - "" + resources: + - configmaps + - events + - pods + - pods/exec + - pods/log + - replicasets + - services + verbs: + - create + - delete + - get + - list + - patch + - update + - watch - apiGroups: - "" resources: @@ -957,6 +1576,18 @@ rules: - patch - update - watch +- apiGroups: + - apps + resources: + - replicasets + verbs: + - create + - delete + - get + - list + - patch + - update + - watch - apiGroups: - apps resources: @@ -999,6 +1630,22 @@ rules: - patch - update - watch +- apiGroups: + - "" + resources: + - configmaps + - namespaces + - pods + - secrets + - services + verbs: + - create + - delete + - get + - list + - patch + - update + - watch - apiGroups: - "" resources: @@ -1024,6 +1671,62 @@ rules: verbs: - patch - update +- apiGroups: + - database.oracle.com + resources: + - cdbs + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - database.oracle.com + resources: + - cdbs/finalizers + verbs: + - update +- apiGroups: + - database.oracle.com + resources: + - cdbs/status + verbs: + - get + - patch + - update +- apiGroups: + - database.oracle.com + resources: + - pdbs + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - database.oracle.com + resources: + - pdbs/finalizers + verbs: + - create + - delete + - get + - patch + - update +- apiGroups: + - database.oracle.com + resources: + - pdbs/status + verbs: + - get + - patch + - update - apiGroups: - database.oracle.com resources: @@ -1185,7 +1888,7 @@ metadata: name: oracle-database-operator-controller-manager namespace: oracle-database-operator-system spec: - replicas: 3 + replicas: 1 selector: matchLabels: control-plane: controller-manager @@ -1199,8 +1902,8 @@ spec: - --enable-leader-election command: - /manager - image: container-registry.oracle.com/database/operator:0.1.0 - imagePullPolicy: Always + image: phx.ocir.io/intsanjaysingh/gdb-repo/oracle/dboper-ords-go:master + imagePullPolicy: Never name: manager ports: - containerPort: 9443 @@ -1255,6 +1958,49 @@ metadata: cert-manager.io/inject-ca-from: oracle-database-operator-system/oracle-database-operator-serving-cert name: oracle-database-operator-mutating-webhook-configuration webhooks: +- admissionReviewVersions: + - v1 + - v1beta1 + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /mutate-database-oracle-com-v1alpha1-cdb + failurePolicy: Fail + name: mcdb.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + resources: + - cdbs + sideEffects: None +- admissionReviewVersions: + - v1 + - v1beta1 + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /mutate-database-oracle-com-v1alpha1-pdb + failurePolicy: Fail + name: mpdb.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + resources: + - pdbs + - pdbs/status + sideEffects: None - admissionReviewVersions: - v1 - v1beta1 @@ -1284,6 +2030,50 @@ metadata: cert-manager.io/inject-ca-from: oracle-database-operator-system/oracle-database-operator-serving-cert name: oracle-database-operator-validating-webhook-configuration webhooks: +- admissionReviewVersions: + - v1 + - v1beta1 + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /validate-database-oracle-com-v1alpha1-cdb + failurePolicy: Fail + name: vcdb.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + resources: + - cdbs + sideEffects: None +- admissionReviewVersions: + - v1 + - v1beta1 + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /validate-database-oracle-com-v1alpha1-pdb + failurePolicy: Fail + name: vpdb.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + - DELETE + resources: + - pdbs + - pdbs/status + sideEffects: None - admissionReviewVersions: - v1 - v1beta1 diff --git a/ords/cdbadmin.properties.tmpl b/ords/cdbadmin.properties.tmpl index b1a801e4..34bc56fd 100644 --- a/ords/cdbadmin.properties.tmpl +++ b/ords/cdbadmin.properties.tmpl @@ -1,3 +1,3 @@ database.api.admin.enabled=true -db.cdb.adminUser=###CDBADMIN_USER### as SYSOPER +db.cdb.adminUser=###CDBADMIN_USER### as SYSDBA db.cdb.adminUser.password=###CDBADMIN_PWD### \ No newline at end of file From 76d0de0543ab3f73e7e61b5fba514c937c7c0065 Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Tue, 7 Dec 2021 15:34:35 +0530 Subject: [PATCH 072/628] Added support for provisioning pre-built db --- .../v1alpha1/singleinstancedatabase_types.go | 3 +- .../singleinstancedatabase_webhook.go | 44 +++++++++++++++++++ commons/database/constants.go | 3 ++ commons/database/utils.go | 26 +++++++++++ ...ngleinstancedatabase_prov_prebuilt_db.yaml | 4 ++ .../singleinstancedatabase_controller.go | 19 ++++++-- 6 files changed, 93 insertions(+), 6 deletions(-) diff --git a/apis/database/v1alpha1/singleinstancedatabase_types.go b/apis/database/v1alpha1/singleinstancedatabase_types.go index f14bf8de..6f0e2be4 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_types.go +++ b/apis/database/v1alpha1/singleinstancedatabase_types.go @@ -83,8 +83,7 @@ type SingleInstanceDatabaseSpec struct { type SingleInstanceDatabasePersistence struct { Size string `json:"size"` StorageClass string `json:"storageClass"` - - AccessMode string `json:"accessMode,omitempty"` + AccessMode string `json:"accessMode,omitempty"` } // SingleInstanceDatabaseInitParams defines the Init Parameters diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index 19173ffc..66d4767c 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -93,6 +93,43 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { singleinstancedatabaselog.Info("validate create", "name", r.Name) var allErrs field.ErrorList + // Pre-built db + if r.Spec.Persistence.AccessMode == "" { + if r.Spec.AdminPassword.SecretName != "" { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("adminPassword"), r.Spec.AdminPassword, + "cannot change password for prebuilt db")) + } + if r.Spec.Sid != "" { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("sid"), r.Spec.Sid, + "cannot change sid for prebuilt db")) + } + if r.Spec.Pdbname != "" { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("pdbName"), r.Spec.Pdbname, + "cannot change pdbName for prebuilt db")) + } + if r.Spec.CloneFrom != "" { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("cloneFrom"), r.Spec.CloneFrom, + "cannot clone to create a prebuilt db")) + } + if len(allErrs) == 0 { + return nil + } + return apierrors.NewInvalid( + schema.GroupKind{Group: "database.oracle.com", Kind: "SingleInstanceDatabase"}, + r.Name, allErrs) + } + + if r.Spec.Persistence.AccessMode != "" && + r.Spec.Persistence.AccessMode != "ReadWriteMany" && r.Spec.Persistence.AccessMode != "ReadWriteOnce" { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("persistence"), r.Spec.Persistence.AccessMode, + "should be either \"ReadWriteOnce\" or \"ReadWriteMany\"")) + } + if r.Spec.Persistence.AccessMode == "ReadWriteOnce" && r.Spec.Replicas != 1 { allErrs = append(allErrs, field.Invalid(field.NewPath("spec").Child("replicas"), r.Spec.Replicas, @@ -146,11 +183,18 @@ func (r *SingleInstanceDatabase) ValidateUpdate(oldRuntimeObject runtime.Object) return err } } + + // Pre-built db + if r.Spec.Persistence.AccessMode == "" { + return nil + } + // Now check for updation errors old, ok := oldRuntimeObject.(*SingleInstanceDatabase) if !ok { return nil } + edition := r.Spec.Edition if r.Spec.Edition == "" { edition = "Enterprise" diff --git a/commons/database/constants.go b/commons/database/constants.go index 8b376051..5aad2bb8 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -420,3 +420,6 @@ const SetApexUsers string = "\numask 177" + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-properties --conf apex apexPublicUser" + "\nrm -f apexPublicUser" + "\numask 022" + +// Get Sid, Pdbname, Edition for prebuilt db +const GetSidPdbEditionCMD string = "if [ -f ${ORACLE_BASE}/oradata/dbconfig/$ORACLE_SID/.docker_standard ]; then echo \"$ORACLE_SID,$ORACLE_PDB,Standard\"; else echo \"$ORACLE_SID,$ORACLE_PDB,Enterprise\"; fi;" diff --git a/commons/database/utils.go b/commons/database/utils.go index f7fb051b..7f39c598 100644 --- a/commons/database/utils.go +++ b/commons/database/utils.go @@ -571,6 +571,32 @@ func GetNodeIp(r client.Reader, ctx context.Context, req ctrl.Request) string { return nodeip } +// GetSidPdbEdition to display sid, pdbname, edition in ConnectionString +func GetSidPdbEdition(r client.Reader, config *rest.Config, ctx context.Context, req ctrl.Request) (string, string, string) { + + log := ctrllog.FromContext(ctx).WithValues("GetNodeIp", req.NamespacedName) + + readyPod, _, _, _, err := FindPods(r, "", "", req.Name, req.Namespace, ctx, req) + if err != nil { + log.Error(err, err.Error()) + return "", "", "" + } + if readyPod.Name != "" { + out, err := ExecCommand(r, config, readyPod.Name, readyPod.Namespace, "", + ctx, req, false, "bash", "-c", fmt.Sprintf(GetSidPdbEditionCMD)) + if err != nil { + log.Error(err, err.Error()) + return "", "", "" + } + splitstr := strings.Split(out, ",") + if len(splitstr) == 3 { + return splitstr[0], splitstr[1], splitstr[2] + } + } + + return "", "", "" +} + // Get Datapatch Status func GetSqlpatchStatus(r client.Reader, config *rest.Config, readyPod corev1.Pod, ctx context.Context, req ctrl.Request) (string, string, string, error) { log := ctrllog.FromContext(ctx).WithValues("getSqlpatchStatus", req.NamespacedName) diff --git a/config/samples/sidb/singleinstancedatabase_prov_prebuilt_db.yaml b/config/samples/sidb/singleinstancedatabase_prov_prebuilt_db.yaml index 3e502d84..913e21ad 100644 --- a/config/samples/sidb/singleinstancedatabase_prov_prebuilt_db.yaml +++ b/config/samples/sidb/singleinstancedatabase_prov_prebuilt_db.yaml @@ -13,3 +13,7 @@ spec: image: pullFrom: pullSecrets: + + ## Type of service . Applicable on cloud enviroments only + ## if loadBalService : false, service type = "NodePort". else "LoadBalancer" + loadBalancer: false \ No newline at end of file diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index c93976f4..b2b69ede 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -314,6 +314,11 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab eventReason := "Spec Error" var eventMsgs []string + // Pre-built db + if m.Spec.Persistence.AccessMode == "" { + return requeueN, nil + } + // If Express Edition , Ensure Replicas=1 if m.Spec.Edition == "express" && m.Spec.Replicas != 1 { eventMsgs = append(eventMsgs, "XE supports only one replica") @@ -1000,24 +1005,30 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex m.Status.ConnectString = dbcommons.ValueUnavailable m.Status.PdbConnectString = dbcommons.ValueUnavailable m.Status.OemExpressUrl = dbcommons.ValueUnavailable + pdbName := "ORCLPDB1" + sid := m.Spec.Sid + if m.Spec.Persistence.AccessMode == "" { + sid, pdbName, m.Status.Edition = dbcommons.GetSidPdbEdition(r, r.Config, ctx, req) + } + if m.Spec.Pdbname != "" { pdbName = strings.ToUpper(m.Spec.Pdbname) } if m.Spec.LoadBalancer { - m.Status.ClusterConnectString = svc.Name + "." + svc.Namespace + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/" + strings.ToUpper(m.Spec.Sid) + m.Status.ClusterConnectString = svc.Name + "." + svc.Namespace + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/" + strings.ToUpper(sid) if len(svc.Status.LoadBalancer.Ingress) > 0 { - m.Status.ConnectString = svc.Status.LoadBalancer.Ingress[0].IP + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/" + strings.ToUpper(m.Spec.Sid) + m.Status.ConnectString = svc.Status.LoadBalancer.Ingress[0].IP + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/" + strings.ToUpper(sid) m.Status.PdbConnectString = svc.Status.LoadBalancer.Ingress[0].IP + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/" + strings.ToUpper(pdbName) m.Status.OemExpressUrl = "https://" + svc.Status.LoadBalancer.Ingress[0].IP + ":" + fmt.Sprint(svc.Spec.Ports[1].Port) + "/em" } return requeueN, nil } - m.Status.ClusterConnectString = svc.Name + "." + svc.Namespace + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/" + strings.ToUpper(m.Spec.Sid) + m.Status.ClusterConnectString = svc.Name + "." + svc.Namespace + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/" + strings.ToUpper(sid) nodeip := dbcommons.GetNodeIp(r, ctx, req) if nodeip != "" { - m.Status.ConnectString = nodeip + ":" + fmt.Sprint(svc.Spec.Ports[0].NodePort) + "/" + strings.ToUpper(m.Spec.Sid) + m.Status.ConnectString = nodeip + ":" + fmt.Sprint(svc.Spec.Ports[0].NodePort) + "/" + strings.ToUpper(sid) m.Status.PdbConnectString = nodeip + ":" + fmt.Sprint(svc.Spec.Ports[0].NodePort) + "/" + strings.ToUpper(pdbName) m.Status.OemExpressUrl = "https://" + nodeip + ":" + fmt.Sprint(svc.Spec.Ports[1].NodePort) + "/em" } From 4f36fdc0a2930a73e00835329e12a1fa6a8662ac Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Tue, 7 Dec 2021 15:37:24 +0530 Subject: [PATCH 073/628] Made replicas omitempty field --- apis/database/v1alpha1/singleinstancedatabase_types.go | 2 +- .../crd/bases/database.oracle.com_singleinstancedatabases.yaml | 1 - oracle-database-operator.yaml | 1 - 3 files changed, 1 insertion(+), 3 deletions(-) diff --git a/apis/database/v1alpha1/singleinstancedatabase_types.go b/apis/database/v1alpha1/singleinstancedatabase_types.go index 6f0e2be4..cdd02a6a 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_types.go +++ b/apis/database/v1alpha1/singleinstancedatabase_types.go @@ -70,7 +70,7 @@ type SingleInstanceDatabaseSpec struct { // +k8s:openapi-gen=true // +kubebuilder:validation:Minimum=1 - Replicas int `json:"replicas"` + Replicas int `json:"replicas,omitempty"` NodeSelector map[string]string `json:"nodeSelector,omitempty"` AdminPassword SingleInstanceDatabaseAdminPassword `json:"adminPassword,omitempty"` diff --git a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml index 28c77806..c19470eb 100644 --- a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml +++ b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml @@ -151,7 +151,6 @@ spec: type: string required: - image - - replicas type: object status: description: SingleInstanceDatabaseStatus defines the observed state of diff --git a/oracle-database-operator.yaml b/oracle-database-operator.yaml index 4010aa78..3a3e8c85 100644 --- a/oracle-database-operator.yaml +++ b/oracle-database-operator.yaml @@ -892,7 +892,6 @@ spec: type: string required: - image - - replicas type: object status: description: SingleInstanceDatabaseStatus defines the observed state of SingleInstanceDatabase From 3696084358dc85110f22039d2bc0f0de2ff3a1d8 Mon Sep 17 00:00:00 2001 From: Gajanan Bhat Date: Tue, 7 Dec 2021 22:35:00 +0530 Subject: [PATCH 074/628] Added scale up/down for replicas --- apis/database/v1alpha1/cdb_types.go | 3 +- apis/database/v1alpha1/cdb_webhook.go | 9 +- .../crd/bases/database.oracle.com_cdbs.yaml | 6 +- controllers/database/cdb_controller.go | 434 ++++++------------ oracle-database-operator.yaml | 8 +- 5 files changed, 158 insertions(+), 302 deletions(-) diff --git a/apis/database/v1alpha1/cdb_types.go b/apis/database/v1alpha1/cdb_types.go index 2268ff26..6b8c82f6 100644 --- a/apis/database/v1alpha1/cdb_types.go +++ b/apis/database/v1alpha1/cdb_types.go @@ -141,7 +141,8 @@ type CDBStatus struct { // +kubebuilder:printcolumn:JSONPath=".spec.cdbName",name="CDB Name",type="string",description="Name of the CDB" // +kubebuilder:printcolumn:JSONPath=".spec.dbServer",name="DB Server",type="string",description=" Name of the DB Server" // +kubebuilder:printcolumn:JSONPath=".spec.dbPort",name="DB Port",type="integer",description="DB server port" -// +kubebuilder:printcolumn:JSONPath=".spec.scanName",name="SCAN NAme",type="string",description="SCAN Name" +// +kubebuilder:printcolumn:JSONPath=".spec.scanName",name="SCAN Name",type="string",description="SCAN Name" +// +kubebuilder:printcolumn:JSONPath=".spec.replicas",name="Replicas",type="integer",description="Replicas" // +kubebuilder:printcolumn:JSONPath=".status.phase",name="Status",type="string",description="Status of the CDB Resource" // +kubebuilder:printcolumn:JSONPath=".status.msg",name="Message",type="string",description="Error message, if any" diff --git a/apis/database/v1alpha1/cdb_webhook.go b/apis/database/v1alpha1/cdb_webhook.go index d18cfc8a..9fcf7b07 100644 --- a/apis/database/v1alpha1/cdb_webhook.go +++ b/apis/database/v1alpha1/cdb_webhook.go @@ -77,12 +77,6 @@ func (r *CDB) Default() { if r.Spec.Replicas == 0 { r.Spec.Replicas = 1 } - - cdblog.Info("CDB phase : " + r.Status.Phase) - if r.Status.Phase == "Ready" { - r.Status.Status = false - r.Status.Phase = "CreatingPod" - } } // TODO(user): change verbs to "verbs=create;update;delete" if you want to enable deletion validation. @@ -92,7 +86,6 @@ var _ webhook.Validator = &CDB{} // ValidateCreate implements webhook.Validator so a webhook will be registered for the type func (r *CDB) ValidateCreate() error { - cdblog.Info("Validating CDB spec for : " + r.Name) cdblog.Info("ValidateCreate", "name", r.Name) var allErrs field.ErrorList @@ -159,7 +152,7 @@ func (r *CDB) ValidateUpdate(old runtime.Object) error { if !strings.EqualFold(oldCDB.Spec.ServiceName, r.Spec.ServiceName) { allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec").Child("serviceName"), "cannot be changed")) + field.Forbidden(field.NewPath("spec").Child("replics"), "cannot be changed")) } if len(allErrs) == 0 { diff --git a/config/crd/bases/database.oracle.com_cdbs.yaml b/config/crd/bases/database.oracle.com_cdbs.yaml index bc87e42c..e50888d0 100644 --- a/config/crd/bases/database.oracle.com_cdbs.yaml +++ b/config/crd/bases/database.oracle.com_cdbs.yaml @@ -31,8 +31,12 @@ spec: type: integer - description: SCAN Name jsonPath: .spec.scanName - name: SCAN NAme + name: SCAN Name type: string + - description: Replicas + jsonPath: .spec.replicas + name: Replicas + type: integer - description: Status of the CDB Resource jsonPath: .status.phase name: Status diff --git a/controllers/database/cdb_controller.go b/controllers/database/cdb_controller.go index b3eb243c..a21572d2 100644 --- a/controllers/database/cdb_controller.go +++ b/controllers/database/cdb_controller.go @@ -80,7 +80,7 @@ type CDBReconciler struct { var ( cdbPhaseInit = "Initializing" cdbPhasePod = "CreatingPod" - cdbPhaseValPod = "ValidatingPod" + cdbPhaseValPod = "ValidatingPods" cdbPhaseService = "CreatingService" cdbPhaseSecrets = "DeletingSecrets" cdbPhaseReady = "Ready" @@ -150,6 +150,7 @@ func (r *CDBReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.R return requeueY, nil } + // If post-creation, CDB spec is changed, check and take appropriate action if (cdb.Status.Phase == cdbPhaseReady) && cdb.Status.Status { r.evaluateSpecChange(ctx, req, cdb) } @@ -164,15 +165,15 @@ func (r *CDBReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.R case cdbPhasePod: // Create ORDS POD //err = r.createORDSPod(ctx, req, cdb) - err = r.createORDSReplicaSet(ctx, req, cdb) + err = r.createORDSInstances(ctx, req, cdb) if err != nil { log.Info("Reconcile queued") return requeueY, nil } cdb.Status.Phase = cdbPhaseValPod case cdbPhaseValPod: - // Validate ORDS POD - err = r.validateORDSPod(ctx, req, cdb) + // Validate ORDS PODs + err = r.validateORDSPods(ctx, req, cdb) if err != nil { log.Info("Reconcile queued") return requeueY, nil @@ -211,66 +212,14 @@ func (r *CDBReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.R return requeueN, nil } -/************************************************* - * Create a Pod based on the ORDS container - /************************************************/ -/*func (r *CDBReconciler) createORDSPod(ctx context.Context, req ctrl.Request, cdb *dbapi.CDB) error { - - log := r.Log.WithValues("createORDSPod", req.NamespacedName) - - for i := 0; i < cdb.Spec.Replicas; i++ { - pod := r.createPodSpec(cdb) - - log.Info("Creating a new Pod for :"+cdb.Name, "Pod.Namespace", pod.Namespace, "Pod.Name", pod.Name) - err := r.Create(ctx, pod) - if err != nil { - log.Error(err, "Failed to create new Pod for :"+cdb.Name, "Pod.Namespace", pod.Namespace, "Pod.Name", pod.Name) - return err - } - } - - log.Info("Created ORDS Pod(s) successfully") - r.Recorder.Eventf(cdb, corev1.EventTypeNormal, "CreatedORDSPod", "Created %s ORDS Pod(s) for %s", strconv.Itoa(cdb.Spec.Replicas), cdb.Name) - return nil -} -*/ /********************************************************** * Create a ReplicaSet for pods based on the ORDS container /********************************************************/ -func (r *CDBReconciler) createORDSReplicaSet(ctx context.Context, req ctrl.Request, cdb *dbapi.CDB) error { +func (r *CDBReconciler) createORDSInstances(ctx context.Context, req ctrl.Request, cdb *dbapi.CDB) error { - log := r.Log.WithValues("createORDSReplicaSet", req.NamespacedName) - - replicas := int32(cdb.Spec.Replicas) - podSpec := r.createPodSpec(cdb) + log := r.Log.WithValues("createORDSInstances", req.NamespacedName) - replicaSet := &appsv1.ReplicaSet{ - ObjectMeta: metav1.ObjectMeta{ - Name: cdb.Name + "-ords-rs", - Namespace: cdb.Namespace, - Labels: map[string]string{ - "name": cdb.Name + "-ords-rs", - }, - }, - Spec: appsv1.ReplicaSetSpec{ - Replicas: &replicas, - Template: corev1.PodTemplateSpec{ - ObjectMeta: metav1.ObjectMeta{ - Name: cdb.Name + "-ords", - Namespace: cdb.Namespace, - Labels: map[string]string{ - "name": cdb.Name + "-ords", - }, - }, - Spec: podSpec, - }, - Selector: &metav1.LabelSelector{ - MatchLabels: map[string]string{ - "name": cdb.Name + "-ords", - }, - }, - }, - } + replicaSet := r.createReplicaSetSpec(cdb) foundRS := &appsv1.ReplicaSet{} err := r.Get(context.TODO(), types.NamespacedName{Name: replicaSet.Name, Namespace: cdb.Namespace}, foundRS) @@ -281,15 +230,9 @@ func (r *CDBReconciler) createORDSReplicaSet(ctx context.Context, req ctrl.Reque log.Error(err, "Failed to create ReplicaSet for :"+cdb.Name, "Namespace", replicaSet.Namespace, "Name", replicaSet.Name) return err } - } else { - log.Info("Found Replicas: " + strconv.Itoa(int(*(foundRS.Spec.Replicas))) + ", New Replicas: " + strconv.Itoa(int(*(replicaSet.Spec.Replicas)))) - foundRS.Spec = replicaSet.Spec - err = r.Update(ctx, foundRS) - //err = r.Update(ctx, replicaSet) - if err != nil { - log.Error(err, "Failed to update ReplicaSet for :"+cdb.Name, "Namespace", replicaSet.Namespace, "Name", replicaSet.Name) - return err - } + } else if err != nil { + log.Error(err, "Replicaset : "+replicaSet.Name+" already exists.") + return err } // Set CDB instance as the owner and controller @@ -303,7 +246,7 @@ func (r *CDBReconciler) createORDSReplicaSet(ctx context.Context, req ctrl.Reque /************************************************* * Validate ORDS Pod. Check if there are any errors /************************************************/ -func (r *CDBReconciler) validateORDSPod(ctx context.Context, req ctrl.Request, cdb *dbapi.CDB) error { +func (r *CDBReconciler) validateORDSPods(ctx context.Context, req ctrl.Request, cdb *dbapi.CDB) error { log := r.Log.WithValues("validateORDSPod", req.NamespacedName) @@ -490,178 +433,141 @@ func (r *CDBReconciler) createPodSpec(cdb *dbapi.CDB) corev1.PodSpec { return podSpec } -/********************************************************** - * Evaluate change in Spec post creation and instantiation - /********************************************************/ -func (r *CDBReconciler) evaluateSpecChange(ctx context.Context, req ctrl.Request, cdb *dbapi.CDB) error { - log := r.Log.WithValues("evaluateSpecChange", req.NamespacedName) - - currCDB := &dbapi.CDB{} - - err := r.Get(context.TODO(), types.NamespacedName{Name: cdb.Name, Namespace: cdb.Namespace}, currCDB) - if err != nil && apierrors.IsNotFound(err) { - log.Info("Uanble to get curr CDB CR: " + cdb.Name) - return err - } - - log.Info("Existing replicas: " + strconv.Itoa(currCDB.Spec.Replicas) + ", New Replicas: " + strconv.Itoa(cdb.Spec.Replicas)) - return nil -} - /************************ - * Create Pod spec + * Create ReplicaSet spec /************************/ -func (r *CDBReconciler) createPodSpec2(cdb *dbapi.CDB) *corev1.Pod { +func (r *CDBReconciler) createReplicaSetSpec(cdb *dbapi.CDB) *appsv1.ReplicaSet { - pod := &corev1.Pod{ - TypeMeta: metav1.TypeMeta{ - Kind: "Pod", - }, + replicas := int32(cdb.Spec.Replicas) + podSpec := r.createPodSpec(cdb) + + replicaSet := &appsv1.ReplicaSet{ ObjectMeta: metav1.ObjectMeta{ - Name: cdb.Name + "-ords-" + dbcommons.GenerateRandomString(5), + Name: cdb.Name + "-ords-rs", Namespace: cdb.Namespace, Labels: map[string]string{ - "name": cdb.Name + "-ords", + "name": cdb.Name + "-ords-rs", }, }, - Spec: corev1.PodSpec{ - Volumes: []corev1.Volume{{ - Name: "secrets", - VolumeSource: corev1.VolumeSource{ - Projected: &corev1.ProjectedVolumeSource{ - DefaultMode: func() *int32 { i := int32(0666); return &i }(), - Sources: []corev1.VolumeProjection{ - { - Secret: &corev1.SecretProjection{ - LocalObjectReference: corev1.LocalObjectReference{ - Name: cdb.Spec.SysAdminPwd.Secret.SecretName, - }, - }, - }, - { - Secret: &corev1.SecretProjection{ - LocalObjectReference: corev1.LocalObjectReference{ - Name: cdb.Spec.CDBAdminUser.Secret.SecretName, - }, - }, - }, - { - Secret: &corev1.SecretProjection{ - LocalObjectReference: corev1.LocalObjectReference{ - Name: cdb.Spec.CDBAdminPwd.Secret.SecretName, - }, - }, - }, - { - Secret: &corev1.SecretProjection{ - LocalObjectReference: corev1.LocalObjectReference{ - Name: cdb.Spec.ORDSPwd.Secret.SecretName, - }, - }, - }, - { - Secret: &corev1.SecretProjection{ - LocalObjectReference: corev1.LocalObjectReference{ - Name: cdb.Spec.WebServerUser.Secret.SecretName, - }, - }, - }, - { - Secret: &corev1.SecretProjection{ - LocalObjectReference: corev1.LocalObjectReference{ - Name: cdb.Spec.WebServerPwd.Secret.SecretName, - }, - }, - }, - }, + Spec: appsv1.ReplicaSetSpec{ + Replicas: &replicas, + Template: corev1.PodTemplateSpec{ + ObjectMeta: metav1.ObjectMeta{ + Name: cdb.Name + "-ords", + Namespace: cdb.Namespace, + Labels: map[string]string{ + "name": cdb.Name + "-ords", }, }, - }}, - Containers: []corev1.Container{{ - Name: cdb.Name + "-ords", - Image: cdb.Spec.ORDSImage, - VolumeMounts: []corev1.VolumeMount{{ - MountPath: "/opt/oracle/ords/secrets", - Name: "secrets", - ReadOnly: true, - }}, - Env: func() []corev1.EnvVar { - return []corev1.EnvVar{ - { - Name: "ORACLE_HOST", - Value: cdb.Spec.DBServer, - }, - { - Name: "ORACLE_PORT", - Value: strconv.Itoa(cdb.Spec.DBPort), - }, - { - Name: "ORDS_PORT", - Value: strconv.Itoa(cdb.Spec.ORDSPort), - }, - { - Name: "ORACLE_SERVICE", - Value: cdb.Spec.ServiceName, - }, - { - Name: "ORACLE_PWD_KEY", - Value: cdb.Spec.SysAdminPwd.Secret.Key, - }, - { - Name: "CDBADMIN_USER_KEY", - Value: cdb.Spec.CDBAdminUser.Secret.Key, - }, - { - Name: "CDBADMIN_PWD_KEY", - Value: cdb.Spec.CDBAdminPwd.Secret.Key, - }, - { - Name: "ORDS_PWD_KEY", - Value: cdb.Spec.ORDSPwd.Secret.Key, - }, - { - Name: "WEBSERVER_USER_KEY", - Value: cdb.Spec.WebServerUser.Secret.Key, - }, - { - Name: "WEBSERVER_PASSWORD_KEY", - Value: cdb.Spec.WebServerPwd.Secret.Key, - }, - } - }(), - }}, - - NodeSelector: func() map[string]string { - ns := make(map[string]string) - if len(cdb.Spec.NodeSelector) != 0 { - for key, value := range cdb.Spec.NodeSelector { - ns[key] = value - } - } - return ns - }(), + Spec: podSpec, + }, + Selector: &metav1.LabelSelector{ + MatchLabels: map[string]string{ + "name": cdb.Name + "-ords", + }, + }, }, } - if len(cdb.Spec.ORDSImagePullSecret) > 0 { - pod.Spec.ImagePullSecrets = []corev1.LocalObjectReference{ - { - Name: cdb.Spec.ORDSImagePullSecret, - }, + return replicaSet +} + +/********************************************************** + * Evaluate change in Spec post creation and instantiation + /********************************************************/ +func (r *CDBReconciler) evaluateSpecChange(ctx context.Context, req ctrl.Request, cdb *dbapi.CDB) error { + log := r.Log.WithValues("evaluateSpecChange", req.NamespacedName) + + // List the Pods matching the PodTemplate Labels + podName := cdb.Name + "-ords" + podList := &corev1.PodList{} + listOpts := []client.ListOption{client.InNamespace(req.Namespace), client.MatchingLabels{"name": podName}} + + // List retrieves list of objects for a given namespace and list options. + err := r.List(ctx, podList, listOpts...) + if err != nil { + log.Info("Failed to list pods of: "+podName, "Namespace", req.Namespace) + return err + } + + var foundPod corev1.Pod + for _, pod := range podList.Items { + foundPod = pod + break + } + + ordsSpecChange := false + for _, envVar := range foundPod.Spec.Containers[0].Env { + if envVar.Name == "ORACLE_HOST" && envVar.Value != cdb.Spec.DBServer { + ordsSpecChange = true + } else if envVar.Name == "ORACLE_PORT" && envVar.Value != strconv.Itoa(cdb.Spec.DBPort) { + ordsSpecChange = true + } else if envVar.Name == "ORDS_PORT" && envVar.Value != strconv.Itoa(cdb.Spec.ORDSPort) { + ordsSpecChange = true } } - pod.Spec.Containers[0].ImagePullPolicy = corev1.PullAlways + if ordsSpecChange { + // Delete existing ReplicaSet + k_client, err := kubernetes.NewForConfig(r.Config) + if err != nil { + log.Error(err, "Kubernetes Config Error") + return err + } + replicaSetName := cdb.Name + "-ords-rs" + err = k_client.AppsV1().ReplicaSets(cdb.Namespace).Delete(context.TODO(), replicaSetName, metav1.DeleteOptions{}) + if err != nil { + log.Info("Could not delete ReplicaSet", "RS Name", replicaSetName, "err", err.Error()) + if !strings.Contains(strings.ToUpper(err.Error()), "NOT FOUND") { + return err + } + } else { + log.Info("Successfully deleted ORDS ReplicaSet", "RS Name", replicaSetName) + } + + // Create new ReplicaSet + replicaSet := r.createReplicaSetSpec(cdb) + log.Info("Re-Creating ORDS Replicaset: " + replicaSet.Name) + err = r.Create(ctx, replicaSet) + if err != nil { + log.Error(err, "Failed to re-create ReplicaSet for :"+cdb.Name, "Namespace", replicaSet.Namespace, "Name", replicaSet.Name) + return err + } + // Set CDB instance as the owner and controller + ctrl.SetControllerReference(cdb, replicaSet, r.Scheme) + log.Info("Successfully re-created ORDS ReplicaSet", "RS Name", replicaSetName) - if len(cdb.Spec.ORDSImagePullPolicy) > 0 { - if strings.ToUpper(cdb.Spec.ORDSImagePullPolicy) == "NEVER" { - pod.Spec.Containers[0].ImagePullPolicy = corev1.PullNever + cdb.Status.Phase = cdbPhaseValPod + cdb.Status.Status = false + r.Status().Update(ctx, cdb) + } else { + // If only the value of replicas is changed, update the RS only + replicaSetName := cdb.Name + "-ords-rs" + + foundRS := &appsv1.ReplicaSet{} + err := r.Get(context.TODO(), types.NamespacedName{Name: replicaSetName, Namespace: cdb.Namespace}, foundRS) + if err != nil { + log.Error(err, "Unable to get ORDS Replicaset: "+replicaSetName) + return err + } + + // Check if replicas have changed + replicas := int32(cdb.Spec.Replicas) + if cdb.Spec.Replicas != int(*(foundRS.Spec.Replicas)) { + log.Info("Existing Replicas: " + strconv.Itoa(int(*(foundRS.Spec.Replicas))) + ", New Replicas: " + strconv.Itoa(cdb.Spec.Replicas)) + foundRS.Spec.Replicas = &replicas + err = r.Update(ctx, foundRS) + if err != nil { + log.Error(err, "Failed to update ReplicaSet for :"+cdb.Name, "Namespace", cdb.Namespace, "Name", replicaSetName) + return err + } + cdb.Status.Phase = cdbPhaseValPod + cdb.Status.Status = false + r.Status().Update(ctx, cdb) } } - // Set CDB instance as the owner and controller - ctrl.SetControllerReference(cdb, pod, r.Scheme) - return pod + return nil } /************************************************* @@ -671,17 +577,24 @@ func (r *CDBReconciler) createORDSSVC(ctx context.Context, req ctrl.Request, cdb log := r.Log.WithValues("createORDSSVC", req.NamespacedName) - svc := r.createSvcSpec(cdb) + foundSvc := &corev1.Service{} + err := r.Get(context.TODO(), types.NamespacedName{Name: cdb.Name + "-ords", Namespace: cdb.Namespace}, foundSvc) + if err != nil && apierrors.IsNotFound(err) { + svc := r.createSvcSpec(cdb) - log.Info("Creating a new Cluster Service for: "+cdb.Name, "Svc.Namespace", svc.Namespace, "Service.Name", svc.Name) - err := r.Create(ctx, svc) - if err != nil { - log.Error(err, "Failed to create new Cluster Service for: "+cdb.Name, "Svc.Namespace", svc.Namespace, "Service.Name", svc.Name) - return err + log.Info("Creating a new Cluster Service for: "+cdb.Name, "Svc.Namespace", svc.Namespace, "Service.Name", svc.Name) + err := r.Create(ctx, svc) + if err != nil { + log.Error(err, "Failed to create new Cluster Service for: "+cdb.Name, "Svc.Namespace", svc.Namespace, "Service.Name", svc.Name) + return err + } + + log.Info("Created ORDS Cluster Service successfully") + r.Recorder.Eventf(cdb, corev1.EventTypeNormal, "CreatedORDSService", "Created ORDS Service for %s", cdb.Name) + } else { + log.Info("ORDS Cluster Service already exists") } - log.Info("Created ORDS Cluster Service successfully") - r.Recorder.Eventf(cdb, corev1.EventTypeNormal, "CreatedORDSService", "Created ORDS Service for %s", cdb.Name) return nil } @@ -808,66 +721,6 @@ func (r *CDBReconciler) deleteCDBInstance(ctx context.Context, req ctrl.Request, return nil } -/************************************************* - * Delete CDB Resource -/************************************************/ -func (r *CDBReconciler) deleteCDBInstance2(ctx context.Context, req ctrl.Request, cdb *dbapi.CDB) error { - - log := r.Log.WithValues("deleteCDBInstance", req.NamespacedName) - - k_client, err := kubernetes.NewForConfig(r.Config) - if err != nil { - log.Error(err, "Kubernetes Config Error") - } - - podName := cdb.Name + "-ords" - - podList := &corev1.PodList{} - listOpts := []client.ListOption{client.InNamespace(req.Namespace), client.MatchingLabels{"name": podName}} - - // List retrieves list of objects for a given namespace and list options. - err = r.List(ctx, podList, listOpts...) - if err != nil { - log.Error(err, "Failed to list pods of: "+podName, "Namespace", req.Namespace) - return err - } - - if len(podList.Items) == 0 { - log.Info("No pods found for: "+podName, "Namespace", req.Namespace) - return nil - } - - for _, pod := range podList.Items { - err = k_client.CoreV1().Pods(cdb.Namespace).Delete(context.TODO(), pod.Name, metav1.DeleteOptions{}) - if err != nil { - log.Info("Could not delete Pod", "Pod Name", pod.Name, "err", err.Error()) - if !strings.Contains(strings.ToUpper(err.Error()), "NOT FOUND") { - return err - } - } else { - log.Info("Successfully deleted ORDS Pod", "Pod Name", pod.Name) - } - } - - r.Recorder.Eventf(cdb, corev1.EventTypeNormal, "DeletedORDSPod", "Deleted ORDS Pod(s) for %s", cdb.Name) - - svcName := cdb.Name + "-ords" - - err = k_client.CoreV1().Services(cdb.Namespace).Delete(context.TODO(), svcName, metav1.DeleteOptions{}) - if err != nil { - log.Info("Could not delete Service", "Service Name", svcName, "err", err.Error()) - if !strings.Contains(strings.ToUpper(err.Error()), "NOT FOUND") { - return err - } - } else { - r.Recorder.Eventf(cdb, corev1.EventTypeNormal, "DeletedORDSService", "Deleted ORDS Service for %s", cdb.Name) - log.Info("Successfully deleted ORDS Service", "Service Name", svcName) - } - - log.Info("Successfully deleted CDB resource", "CDB Name", cdb.Spec.CDBName) - return nil -} - /************************************************* * Delete Secrets /************************************************/ @@ -940,7 +793,8 @@ func (r *CDBReconciler) SetupWithManager(mgr ctrl.Manager) error { }, DeleteFunc: func(e event.DeleteEvent) bool { // Evaluates to false if the object has been confirmed deleted. - return !e.DeleteStateUnknown + //return !e.DeleteStateUnknown + return false }, }). WithOptions(controller.Options{MaxConcurrentReconciles: 100}). diff --git a/oracle-database-operator.yaml b/oracle-database-operator.yaml index b13cc06b..d711eb38 100644 --- a/oracle-database-operator.yaml +++ b/oracle-database-operator.yaml @@ -220,8 +220,12 @@ spec: type: integer - description: SCAN Name jsonPath: .spec.scanName - name: SCAN NAme + name: SCAN Name type: string + - description: Replicas + jsonPath: .spec.replicas + name: Replicas + type: integer - description: Status of the CDB Resource jsonPath: .status.phase name: Status @@ -1903,7 +1907,7 @@ spec: command: - /manager image: phx.ocir.io/intsanjaysingh/gdb-repo/oracle/dboper-ords-go:master - imagePullPolicy: Never + imagePullPolicy: Always name: manager ports: - containerPort: 9443 From e052b839576e3883e9bfc23b6bdc88095d5dc52f Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Wed, 8 Dec 2021 11:41:03 +0530 Subject: [PATCH 075/628] extract sid,pdb,edition for prebuiltdb --- commons/database/constants.go | 2 +- commons/database/utils.go | 2 +- config/manager/kustomization.yaml | 2 +- controllers/database/singleinstancedatabase_controller.go | 3 +++ oracle-database-operator.yaml | 2 +- 5 files changed, 7 insertions(+), 4 deletions(-) diff --git a/commons/database/constants.go b/commons/database/constants.go index 5aad2bb8..2d84d23a 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -422,4 +422,4 @@ const SetApexUsers string = "\numask 177" + "\numask 022" // Get Sid, Pdbname, Edition for prebuilt db -const GetSidPdbEditionCMD string = "if [ -f ${ORACLE_BASE}/oradata/dbconfig/$ORACLE_SID/.docker_standard ]; then echo \"$ORACLE_SID,$ORACLE_PDB,Standard\"; else echo \"$ORACLE_SID,$ORACLE_PDB,Enterprise\"; fi;" +const GetSidPdbEditionCMD string = "if [ -f ${ORACLE_BASE}/oradata/dbconfig/$ORACLE_SID/.docker_standard ]; then echo \"$ORACLE_SID,$ORACLE_PDB,Standard,Edition\"; else echo \"$ORACLE_SID,$ORACLE_PDB,Enterprise,Edition\"; fi;" diff --git a/commons/database/utils.go b/commons/database/utils.go index 7f39c598..211a92d0 100644 --- a/commons/database/utils.go +++ b/commons/database/utils.go @@ -589,7 +589,7 @@ func GetSidPdbEdition(r client.Reader, config *rest.Config, ctx context.Context, return "", "", "" } splitstr := strings.Split(out, ",") - if len(splitstr) == 3 { + if len(splitstr) == 4 { return splitstr[0], splitstr[1], splitstr[2] } } diff --git a/config/manager/kustomization.yaml b/config/manager/kustomization.yaml index ce2bd741..11f7656c 100644 --- a/config/manager/kustomization.yaml +++ b/config/manager/kustomization.yaml @@ -8,5 +8,5 @@ apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization images: - name: controller - newName: phx.ocir.io/oracassandra/cloneonprem + newName: phx.ocir.io/oracassandra/operator/prebuiltdb newTag: test-0.1.2 diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index b2b69ede..c31bba53 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1010,6 +1010,9 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex sid := m.Spec.Sid if m.Spec.Persistence.AccessMode == "" { sid, pdbName, m.Status.Edition = dbcommons.GetSidPdbEdition(r, r.Config, ctx, req) + if sid == "" || pdbName == "" || m.Status.Edition == "" { + return requeueN, nil + } } if m.Spec.Pdbname != "" { diff --git a/oracle-database-operator.yaml b/oracle-database-operator.yaml index 3a3e8c85..ca9d678f 100644 --- a/oracle-database-operator.yaml +++ b/oracle-database-operator.yaml @@ -1401,7 +1401,7 @@ spec: - --enable-leader-election command: - /manager - image: phx.ocir.io/oracassandra/cloneonprem:test-0.1.2 + image: phx.ocir.io/oracassandra/operator/prebuiltdb:test-0.1.2 imagePullPolicy: Always name: manager ports: From 9cc996ffbe017ac9a2241877ce372086f679ba81 Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Wed, 8 Dec 2021 12:05:07 +0530 Subject: [PATCH 076/628] Made readme changes for prebuiltdb --- .../samples/sidb/oraclerestdataservice.yaml | 2 +- .../samples/sidb/singleinstancedatabase.yaml | 3 ++- .../sidb/singleinstancedatabase_clone.yaml | 2 +- .../sidb/singleinstancedatabase_patch.yaml | 2 +- .../sidb/singleinstancedatabase_prov.yaml | 2 +- ...ngleinstancedatabase_prov_prebuilt_db.yaml | 4 ++-- docs/sidb/README.md | 19 +++++++++++++++++++ docs/sidb/SIDB_PREREQUISITES.md | 2 ++ 8 files changed, 29 insertions(+), 7 deletions(-) diff --git a/config/samples/sidb/oraclerestdataservice.yaml b/config/samples/sidb/oraclerestdataservice.yaml index f5d58a25..559e26bc 100644 --- a/config/samples/sidb/oraclerestdataservice.yaml +++ b/config/samples/sidb/oraclerestdataservice.yaml @@ -5,7 +5,7 @@ apiVersion: database.oracle.com/v1alpha1 kind: OracleRestDataService metadata: - name: oraclerestdataservice-sample + name: ords-sample spec: ## Database ref. This can be of kind SingleInstanceDatabase. diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index 3e8b0ed7..218d0ed5 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -5,7 +5,7 @@ apiVersion: database.oracle.com/v1alpha1 kind: SingleInstanceDatabase metadata: - name: singleinstancedatabase-sample + name: sidb-sample namespace: default spec: @@ -13,6 +13,7 @@ spec: sid: ORCL1 ## A source database ref to clone from, leave empty to create a fresh database + ## If cloning from on prem source database, mention connect string `:/`. cloneFrom: "" ## NA if cloning from a SourceDB (cloneFrom is set) diff --git a/config/samples/sidb/singleinstancedatabase_clone.yaml b/config/samples/sidb/singleinstancedatabase_clone.yaml index 6d19f243..fe6d0556 100644 --- a/config/samples/sidb/singleinstancedatabase_clone.yaml +++ b/config/samples/sidb/singleinstancedatabase_clone.yaml @@ -5,7 +5,7 @@ apiVersion: database.oracle.com/v1alpha1 kind: SingleInstanceDatabase metadata: - name: singleinstancedatabase-sample + name: sidb-sample-clone namespace: default spec: diff --git a/config/samples/sidb/singleinstancedatabase_patch.yaml b/config/samples/sidb/singleinstancedatabase_patch.yaml index 4d2d03b0..dbcc7fc1 100644 --- a/config/samples/sidb/singleinstancedatabase_patch.yaml +++ b/config/samples/sidb/singleinstancedatabase_patch.yaml @@ -5,7 +5,7 @@ apiVersion: database.oracle.com/v1alpha1 kind: SingleInstanceDatabase metadata: - name: singleinstancedatabase-sample + name: sidb-sample-patch namespace: default spec: diff --git a/config/samples/sidb/singleinstancedatabase_prov.yaml b/config/samples/sidb/singleinstancedatabase_prov.yaml index b1a83e76..612a53d5 100644 --- a/config/samples/sidb/singleinstancedatabase_prov.yaml +++ b/config/samples/sidb/singleinstancedatabase_prov.yaml @@ -5,7 +5,7 @@ apiVersion: database.oracle.com/v1alpha1 kind: SingleInstanceDatabase metadata: - name: singleinstancedatabase-sample + name: sidb-sample namespace: default spec: diff --git a/config/samples/sidb/singleinstancedatabase_prov_prebuilt_db.yaml b/config/samples/sidb/singleinstancedatabase_prov_prebuilt_db.yaml index 913e21ad..e7b2b02b 100644 --- a/config/samples/sidb/singleinstancedatabase_prov_prebuilt_db.yaml +++ b/config/samples/sidb/singleinstancedatabase_prov_prebuilt_db.yaml @@ -5,11 +5,11 @@ apiVersion: database.oracle.com/v1alpha1 kind: SingleInstanceDatabase metadata: - name: singleinstancedatabase-sample + name: sidb-sample-prebuiltdb namespace: default spec: - ## Database Image + ## Pre-built Database Image image: pullFrom: pullSecrets: diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 973a8ad7..7494ec43 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -116,6 +116,25 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI singleinstancedatabase.database.oracle.com/sidb-sample created ``` +* ### Provision a Pre-built Database + + Provision a new Pre-built database instance by specifying appropriate values for the attributes in the the example `.yaml` file, and running the following command: + + ```sh + $ kubectl create -f singleinstancedatabase_prov_prebuilt_db.yaml + + singleinstancedatabase.database.oracle.com/sidb-sample created + ``` + + This Pre-built image includes an already setup database inside the image itself. Although the image size is larger, the startup time of the container includes only the database startup itself, which makes the container startup duration just a couple of seconds. + + This Pre-built database would be very useful in CI/CD scenarios, where database would be used for conducting tests, experiments and the workflow is simple. + + Some limitations are listed as follows: + + External volume can not be used for database persistence (as data files are inside the image itself). + Only the single replica mode (i.e. replicas=1) can be used. + * ### Creation Status Creating a new database instance takes a while. When the 'status' status returns the response "Healthy", the Database is open for connections. diff --git a/docs/sidb/SIDB_PREREQUISITES.md b/docs/sidb/SIDB_PREREQUISITES.md index ce8f1c1f..c6520aae 100644 --- a/docs/sidb/SIDB_PREREQUISITES.md +++ b/docs/sidb/SIDB_PREREQUISITES.md @@ -8,6 +8,8 @@ To deploy Oracle Database Operator for Kubernetes on Oracle Docker images, compl Oracle Database Releases Supported: Oracle Database 19c Enterprise Edition or Standard Edition, and later releases. + Prepare the pre-built database docker image by following the [instructions](https://github.com/oracle/docker-images/blob/main/OracleDatabase/SingleInstance/extensions/prebuiltdb/README.md) + Build OracleRestDataService Docker Images from source following the instructions at [https://github.com/oracle/docker-images/tree/main/OracleRestDataServices](https://github.com/oracle/docker-images/tree/main/OracleRestDataServices). Add 'database.api.enabled=true' entry in 'ords_params.properties.tmpl' file while building OracleRestDataService image . OracleRestDataService version 20.4.1 onwards are supported. From 6ff180af14e961e0e01e8efbd9f4de98ef2fa2a5 Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Wed, 8 Dec 2021 12:37:39 +0530 Subject: [PATCH 077/628] changed ords prerequisite --- config/samples/sidb/oraclerestdataservice.yaml | 2 -- docs/sidb/SIDB_PREREQUISITES.md | 1 - 2 files changed, 3 deletions(-) diff --git a/config/samples/sidb/oraclerestdataservice.yaml b/config/samples/sidb/oraclerestdataservice.yaml index 559e26bc..4c3400f6 100644 --- a/config/samples/sidb/oraclerestdataservice.yaml +++ b/config/samples/sidb/oraclerestdataservice.yaml @@ -34,8 +34,6 @@ spec: keepSecret: false ## ORDS image details - ## Add 'database.api.enabled=true' entry to below file while building ORDS image. - ## https://github.com/oracle/docker-images/blob/main/OracleRestDataServices/dockerfiles/ords_params.properties.tmpl image: pullFrom: pullSecrets: diff --git a/docs/sidb/SIDB_PREREQUISITES.md b/docs/sidb/SIDB_PREREQUISITES.md index c6520aae..e7b30bf2 100644 --- a/docs/sidb/SIDB_PREREQUISITES.md +++ b/docs/sidb/SIDB_PREREQUISITES.md @@ -11,7 +11,6 @@ To deploy Oracle Database Operator for Kubernetes on Oracle Docker images, compl Prepare the pre-built database docker image by following the [instructions](https://github.com/oracle/docker-images/blob/main/OracleDatabase/SingleInstance/extensions/prebuiltdb/README.md) Build OracleRestDataService Docker Images from source following the instructions at [https://github.com/oracle/docker-images/tree/main/OracleRestDataServices](https://github.com/oracle/docker-images/tree/main/OracleRestDataServices). - Add 'database.api.enabled=true' entry in 'ords_params.properties.tmpl' file while building OracleRestDataService image . OracleRestDataService version 20.4.1 onwards are supported. * ### Set Up Kubernetes and Volumes From fc75dc667e92329dbbffbc28dacb2f55d378d3e5 Mon Sep 17 00:00:00 2001 From: Gajanan Bhat Date: Thu, 9 Dec 2021 15:45:29 +0530 Subject: [PATCH 078/628] Fixed PDB state transitions --- apis/database/v1alpha1/cdb_types.go | 3 - apis/database/v1alpha1/cdb_webhook.go | 4 +- apis/database/v1alpha1/pdb_types.go | 3 - apis/database/v1alpha1/pdb_webhook.go | 97 ++++++++++++++------------ config/webhook/manifests.yaml | 2 - controllers/database/cdb_controller.go | 7 +- controllers/database/pdb_controller.go | 72 +++++-------------- oracle-database-operator.yaml | 4 +- 8 files changed, 73 insertions(+), 119 deletions(-) diff --git a/apis/database/v1alpha1/cdb_types.go b/apis/database/v1alpha1/cdb_types.go index 6b8c82f6..1d323c20 100644 --- a/apis/database/v1alpha1/cdb_types.go +++ b/apis/database/v1alpha1/cdb_types.go @@ -42,9 +42,6 @@ import ( metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" ) -// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! -// NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized. - // CDBSpec defines the desired state of CDB type CDBSpec struct { // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster diff --git a/apis/database/v1alpha1/cdb_webhook.go b/apis/database/v1alpha1/cdb_webhook.go index 9fcf7b07..529df5fb 100644 --- a/apis/database/v1alpha1/cdb_webhook.go +++ b/apis/database/v1alpha1/cdb_webhook.go @@ -52,7 +52,7 @@ import ( ) // log is for logging in this package. -var cdblog = logf.Log.WithName("cdb-resource") +var cdblog = logf.Log.WithName("cdb-webhook") func (r *CDB) SetupWebhookWithManager(mgr ctrl.Manager) error { return ctrl.NewWebhookManagedBy(mgr). @@ -60,8 +60,6 @@ func (r *CDB) SetupWebhookWithManager(mgr ctrl.Manager) error { Complete() } -// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! - //+kubebuilder:webhook:path=/mutate-database-oracle-com-v1alpha1-cdb,mutating=true,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=cdbs,verbs=create;update,versions=v1alpha1,name=mcdb.kb.io,admissionReviewVersions={v1,v1beta1} var _ webhook.Defaulter = &CDB{} diff --git a/apis/database/v1alpha1/pdb_types.go b/apis/database/v1alpha1/pdb_types.go index e04f94c5..7b4e6514 100644 --- a/apis/database/v1alpha1/pdb_types.go +++ b/apis/database/v1alpha1/pdb_types.go @@ -42,9 +42,6 @@ import ( metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" ) -// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! -// NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized. - // PDBSpec defines the desired state of PDB type PDBSpec struct { // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster diff --git a/apis/database/v1alpha1/pdb_webhook.go b/apis/database/v1alpha1/pdb_webhook.go index 5d7e25e4..afbaba99 100644 --- a/apis/database/v1alpha1/pdb_webhook.go +++ b/apis/database/v1alpha1/pdb_webhook.go @@ -53,7 +53,7 @@ import ( ) // log is for logging in this package. -var pdblog = logf.Log.WithName("pdb-resource") +var pdblog = logf.Log.WithName("pdb-webhook") func (r *PDB) SetupWebhookWithManager(mgr ctrl.Manager) error { return ctrl.NewWebhookManagedBy(mgr). @@ -61,9 +61,7 @@ func (r *PDB) SetupWebhookWithManager(mgr ctrl.Manager) error { Complete() } -// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! - -//+kubebuilder:webhook:path=/mutate-database-oracle-com-v1alpha1-pdb,mutating=true,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=pdbs;pdbs/status,verbs=create;update,versions=v1alpha1,name=mpdb.kb.io,admissionReviewVersions={v1,v1beta1} +//+kubebuilder:webhook:path=/mutate-database-oracle-com-v1alpha1-pdb,mutating=true,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=pdbs,verbs=create;update,versions=v1alpha1,name=mpdb.kb.io,admissionReviewVersions={v1,v1beta1} var _ webhook.Defaulter = &PDB{} @@ -71,40 +69,50 @@ var _ webhook.Defaulter = &PDB{} func (r *PDB) Default() { pdblog.Info("Setting default values in PDB spec for : " + r.Name) - if r.Spec.ReuseTempFile == nil { - r.Spec.ReuseTempFile = new(bool) - *r.Spec.ReuseTempFile = true - pdblog.Info(" - reuseTempFile : " + strconv.FormatBool(*(r.Spec.ReuseTempFile))) - } - if r.Spec.UnlimitedStorage == nil { - r.Spec.UnlimitedStorage = new(bool) - *r.Spec.UnlimitedStorage = true - pdblog.Info(" - unlimitedStorage : " + strconv.FormatBool(*(r.Spec.UnlimitedStorage))) + action := strings.ToUpper(r.Spec.Action) + + if action == "DELETE" { + if r.Spec.DropAction == "" { + r.Spec.DropAction = "KEEP" + pdblog.Info(" - dropAction : KEEP") + } + } else { + if r.Spec.ReuseTempFile == nil { + r.Spec.ReuseTempFile = new(bool) + *r.Spec.ReuseTempFile = true + pdblog.Info(" - reuseTempFile : " + strconv.FormatBool(*(r.Spec.ReuseTempFile))) + } + if r.Spec.UnlimitedStorage == nil { + r.Spec.UnlimitedStorage = new(bool) + *r.Spec.UnlimitedStorage = true + pdblog.Info(" - unlimitedStorage : " + strconv.FormatBool(*(r.Spec.UnlimitedStorage))) + } + if r.Spec.TDEImport == nil { + r.Spec.TDEImport = new(bool) + *r.Spec.TDEImport = false + pdblog.Info(" - tdeImport : " + strconv.FormatBool(*(r.Spec.TDEImport))) + } + if r.Spec.TDEExport == nil { + r.Spec.TDEExport = new(bool) + *r.Spec.TDEExport = false + pdblog.Info(" - tdeExport : " + strconv.FormatBool(*(r.Spec.TDEExport))) + } + if r.Spec.AsClone == nil { + r.Spec.AsClone = new(bool) + *r.Spec.AsClone = false + pdblog.Info(" - asClone : " + strconv.FormatBool(*(r.Spec.AsClone))) + } } + if r.Spec.GetScript == nil { r.Spec.GetScript = new(bool) *r.Spec.GetScript = false pdblog.Info(" - getScript : " + strconv.FormatBool(*(r.Spec.GetScript))) } - if r.Spec.AsClone == nil { - r.Spec.AsClone = new(bool) - *r.Spec.AsClone = false - pdblog.Info(" - asClone : " + strconv.FormatBool(*(r.Spec.AsClone))) - } - if r.Spec.TDEImport == nil { - r.Spec.TDEImport = new(bool) - *r.Spec.TDEImport = false - pdblog.Info(" - tdeImport : " + strconv.FormatBool(*(r.Spec.TDEImport))) - } - if r.Spec.TDEExport == nil { - r.Spec.TDEExport = new(bool) - *r.Spec.TDEExport = false - pdblog.Info(" - tdeExport : " + strconv.FormatBool(*(r.Spec.TDEExport))) - } } // TODO(user): change verbs to "verbs=create;update;delete" if you want to enable deletion validation. -//+kubebuilder:webhook:path=/validate-database-oracle-com-v1alpha1-pdb,mutating=false,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=pdbs;pdbs/status,verbs=create;update;delete,versions=v1alpha1,name=vpdb.kb.io,admissionReviewVersions={v1,v1beta1} +//+kubebuilder:webhook:path=/validate-database-oracle-com-v1alpha1-pdb,mutating=false,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=pdbs,verbs=create;update;delete,versions=v1alpha1,name=vpdb.kb.io,admissionReviewVersions={v1,v1beta1} var _ webhook.Validator = &PDB{} @@ -202,27 +210,24 @@ func (r *PDB) ValidateUpdate(old runtime.Object) error { pdblog.Info("ValidateUpdate-Validating PDB spec for : " + r.Name) var allErrs field.ErrorList + action := strings.ToUpper(r.Spec.Action) - // Check Common Validations - r.validateCommon(&allErrs) - - // Validate required parameters for Action specified - r.validateAction(&allErrs) + // If PDB CR has been created and in Ready state, only allow updates if the "action" value has changed as well + if (r.Status.Phase == "Ready") && (r.Status.Action == action) { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("action"), "New action also needs to be specified after PDB is in Ready state")) + } else { - // Check TDE requirements - if *(r.Spec.TDEImport) || *(r.Spec.TDEExport) { - r.validateTDEInfo(&allErrs) - } + // Check Common Validations + r.validateCommon(&allErrs) - // Check for updation errors - oldPDB, ok := old.(*PDB) - if !ok { - return nil - } + // Validate required parameters for Action specified + r.validateAction(&allErrs) - if !strings.EqualFold(oldPDB.Spec.CDBResName, r.Spec.CDBResName) { - allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec").Child("cdbResName"), "cannot be changed")) + // Check TDE requirements + if (action != "DELETE") && (*(r.Spec.TDEImport) || *(r.Spec.TDEExport)) { + r.validateTDEInfo(&allErrs) + } } if len(allErrs) == 0 { diff --git a/config/webhook/manifests.yaml b/config/webhook/manifests.yaml index 7b3c4d6f..7e27a61a 100644 --- a/config/webhook/manifests.yaml +++ b/config/webhook/manifests.yaml @@ -47,7 +47,6 @@ webhooks: - UPDATE resources: - pdbs - - pdbs/status sideEffects: None - admissionReviewVersions: - v1 @@ -120,7 +119,6 @@ webhooks: - DELETE resources: - pdbs - - pdbs/status sideEffects: None - admissionReviewVersions: - v1 diff --git a/controllers/database/cdb_controller.go b/controllers/database/cdb_controller.go index a21572d2..e331b31c 100644 --- a/controllers/database/cdb_controller.go +++ b/controllers/database/cdb_controller.go @@ -163,8 +163,7 @@ func (r *CDBReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.R case cdbPhaseInit: cdb.Status.Phase = cdbPhasePod case cdbPhasePod: - // Create ORDS POD - //err = r.createORDSPod(ctx, req, cdb) + // Create ORDS PODs err = r.createORDSInstances(ctx, req, cdb) if err != nil { log.Info("Reconcile queued") @@ -541,7 +540,7 @@ func (r *CDBReconciler) evaluateSpecChange(ctx context.Context, req ctrl.Request cdb.Status.Status = false r.Status().Update(ctx, cdb) } else { - // If only the value of replicas is changed, update the RS only + // Update the RS if the value of "replicas" is changed replicaSetName := cdb.Name + "-ords-rs" foundRS := &appsv1.ReplicaSet{} @@ -551,7 +550,7 @@ func (r *CDBReconciler) evaluateSpecChange(ctx context.Context, req ctrl.Request return err } - // Check if replicas have changed + // Check if number of replicas have changed replicas := int32(cdb.Spec.Replicas) if cdb.Spec.Replicas != int(*(foundRS.Spec.Replicas)) { log.Info("Existing Replicas: " + strconv.Itoa(int(*(foundRS.Spec.Replicas))) + ", New Replicas: " + strconv.Itoa(cdb.Spec.Replicas)) diff --git a/controllers/database/pdb_controller.go b/controllers/database/pdb_controller.go index 92938cf2..cc5fcc3f 100644 --- a/controllers/database/pdb_controller.go +++ b/controllers/database/pdb_controller.go @@ -128,11 +128,6 @@ const PDBFinalizer = "database.oracle.com/PDBfinalizer" // For more details, check Reconcile and its Result here: // - https://pkg.go.dev/sigs.k8s.io/controller-runtime@v0.9.2/pkg/reconcile func (r *PDBReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { - //_ = log.FromContext(ctx) - //_ = r.Log.WithValues("onpremdboperator", req.NamespacedName) - //fmt.Printf("In Reconcile") - //ctx := context.Background() - log := r.Log.WithValues("onpremdboperator", req.NamespacedName) log.Info("Reconcile requested") @@ -174,52 +169,34 @@ func (r *PDBReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.R return requeueY, nil } - action := strings.ToUpper(pdb.Spec.Action) - if pdb.Status.Phase != pdbPhaseFail && pdb.Status.Action != "" && pdb.Status.Action != action { + //action := strings.ToUpper(pdb.Spec.Action) + + /*if pdb.Status.Phase != pdbPhaseFail && pdb.Status.Action != "" && pdb.Status.Action != action { pdb.Status.Status = false - } + }*/ - log.Info("PDB PHASE STATUS:", "Name", pdb.Name, "Phase", pdb.Status.Phase, "Status", strconv.FormatBool(pdb.Status.Status), "Last Action", pdb.Status.Action) + //log.Info("PDB PHASE STATUS:", "Name", pdb.Name, "Phase", pdb.Status.Phase, "Status", strconv.FormatBool(pdb.Status.Status), "Last Action", pdb.Status.Action) - if (pdb.Status.Phase != pdbPhaseReady && pdb.Status.Phase != pdbPhaseFail && !pdb.Status.Status) || (pdb.Status.Action != "" && pdb.Status.Action != action) { - r.validatePhase(ctx, req, pdb) - } + //if (pdb.Status.Phase != pdbPhaseReady && pdb.Status.Phase != pdbPhaseFail && !pdb.Status.Status) || (pdb.Status.Action != "" && pdb.Status.Action != action) { + //r.validatePhase(ctx, req, pdb) + //} if !pdb.Status.Status { + r.validatePhase(ctx, req, pdb) phase := pdb.Status.Phase log.Info("PDB:", "Name", pdb.Name, "Phase", phase, "Status", strconv.FormatBool(pdb.Status.Status)) switch phase { case pdbPhaseCreate: err = r.createPDB(ctx, req, pdb) - if err != nil { - log.Info("Reconcile queued") - return requeueY, nil - } case pdbPhaseClone: err = r.clonePDB(ctx, req, pdb) - if err != nil { - log.Info("Reconcile queued") - return requeueY, nil - } case pdbPhasePlug: err = r.plugPDB(ctx, req, pdb) - if err != nil { - log.Info("Reconcile queued") - return requeueY, nil - } case pdbPhaseUnplug: err = r.unplugPDB(ctx, req, pdb) - if err != nil { - log.Info("Reconcile queued") - return requeueY, nil - } case pdbPhaseDelete: err = r.deletePDB(ctx, req, pdb) - if err != nil { - log.Info("Reconcile queued") - return requeueY, nil - } case pdbPhaseReady, pdbPhaseFail: pdb.Status.Status = true default: @@ -228,8 +205,12 @@ func (r *PDBReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.R return requeueN, nil } pdb.Status.Action = strings.ToUpper(pdb.Spec.Action) - pdb.Status.Phase = pdbPhaseReady - pdb.Status.Msg = "Success" + if err != nil { + pdb.Status.Phase = pdbPhaseFail + } else { + pdb.Status.Phase = pdbPhaseReady + pdb.Status.Msg = "Success" + } } log.Info("Reconcile completed") @@ -283,7 +264,6 @@ func (r *PDBReconciler) getCDBResource(ctx context.Context, req ctrl.Request, pd if err != nil { log.Info("Failed to get CRD for CDB", "Name", cdbResName, "Namespace", req.Namespace, "Error", err.Error()) - pdb.Status.Phase = pdbPhaseFail pdb.Status.Msg = "Unable to get CRD for CDB : " + cdbResName r.Status().Update(ctx, pdb) return cdb, err @@ -313,9 +293,7 @@ func (r *PDBReconciler) getORDSPod(ctx context.Context, req ctrl.Request, pdb *d if err != nil { log.Info("Failed to get Pod for CDB", "Name", cdbResName, "Namespace", req.Namespace, "Error", err.Error()) - pdb.Status.Phase = pdbPhaseFail pdb.Status.Msg = "Unable to get ORDS Pod for CDB : " + cdbResName - r.Status().Update(ctx, pdb) return cdbPod, err } @@ -335,9 +313,7 @@ func (r *PDBReconciler) getSecret(ctx context.Context, req ctrl.Request, pdb *db if err != nil { if apierrors.IsNotFound(err) { log.Info("Secret not found:" + secretName) - pdb.Status.Phase = pdbPhaseFail pdb.Status.Msg = "Secret not found:" + secretName - r.Status().Update(ctx, pdb) return "", err } log.Error(err, "Unable to get the secret.") @@ -405,24 +381,14 @@ func (r *PDBReconciler) callAPI(ctx context.Context, req ctrl.Request, pdb *dbap resp, err := httpclient.Do(httpreq) if err != nil { log.Error(err, "Failed - Could not connect to ORDS Pod", "err", err.Error()) - pdb.Status.Phase = pdbPhaseFail pdb.Status.Msg = "Could not connect to ORDS Pod" - pdb.Status.Status = true - if err := r.Status().Update(ctx, pdb); err != nil { - log.Error(err, "Failed to update status for :"+pdb.Name, "err", err.Error()) - } r.Recorder.Eventf(pdb, corev1.EventTypeWarning, "ORDSError", "Failed: Could not connect to ORDS Pod") return err } if resp.StatusCode != http.StatusOK { bb, _ := ioutil.ReadAll(resp.Body) - pdb.Status.Phase = pdbPhaseFail pdb.Status.Msg = "ORDS Error - HTTP Status Code:" + strconv.Itoa(resp.StatusCode) - pdb.Status.Status = true - if err := r.Status().Update(ctx, pdb); err != nil { - log.Error(err, "Failed to update status for :"+pdb.Name, "err", err.Error()) - } log.Info("ORDS Error - HTTP Status Code :"+strconv.Itoa(resp.StatusCode), "Err", string(bb)) var apiErr ORDSError @@ -451,12 +417,7 @@ func (r *PDBReconciler) callAPI(ctx context.Context, req ctrl.Request, pdb *dbap if sqlItem.ErrorDetails != "" { log.Info("ORDS Error - Oracle Error Code :" + strconv.Itoa(sqlItem.ErrorCode)) if !errFound { - pdb.Status.Phase = pdbPhaseFail pdb.Status.Msg = sqlItem.ErrorDetails - pdb.Status.Status = true - if err := r.Status().Update(ctx, pdb); err != nil { - log.Error(err, "Failed to update status for :"+pdb.Name, "err", err.Error()) - } } r.Recorder.Eventf(pdb, corev1.EventTypeWarning, "OraError", "%s", sqlItem.ErrorDetails) errFound = true @@ -852,7 +813,8 @@ func (r *PDBReconciler) SetupWithManager(mgr ctrl.Manager) error { }, DeleteFunc: func(e event.DeleteEvent) bool { // Evaluates to false if the object has been confirmed deleted. - return !e.DeleteStateUnknown + //return !e.DeleteStateUnknown + return false }, }). WithOptions(controller.Options{MaxConcurrentReconciles: 100}). diff --git a/oracle-database-operator.yaml b/oracle-database-operator.yaml index d711eb38..d87f4dd4 100644 --- a/oracle-database-operator.yaml +++ b/oracle-database-operator.yaml @@ -1907,7 +1907,7 @@ spec: command: - /manager image: phx.ocir.io/intsanjaysingh/gdb-repo/oracle/dboper-ords-go:master - imagePullPolicy: Always + imagePullPolicy: Never name: manager ports: - containerPort: 9443 @@ -2003,7 +2003,6 @@ webhooks: - UPDATE resources: - pdbs - - pdbs/status sideEffects: None - admissionReviewVersions: - v1 @@ -2076,7 +2075,6 @@ webhooks: - DELETE resources: - pdbs - - pdbs/status sideEffects: None - admissionReviewVersions: - v1 From ee54dd29b3faee525a250c47937eb2d52257b164 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Thu, 9 Dec 2021 09:36:02 -0500 Subject: [PATCH 079/628] removed dot imports and unused codes --- commons/oci/database.go | 11 +- commons/oci/provider.go | 4 +- ...autonomousdatabase_controller_bind_test.go | 3 - ...tonomousdatabase_controller_create_test.go | 7 - test/e2e/behavior/shared_behaviors.go | 17 +- test/e2e/suite_test.go | 31 ++- test/e2e/util/oci_config_util.go | 16 +- test/e2e/util/oci_db_request.go | 24 -- test/e2e/util/oci_vault_request.go | 255 ------------------ 9 files changed, 43 insertions(+), 325 deletions(-) delete mode 100644 test/e2e/util/oci_vault_request.go diff --git a/commons/oci/database.go b/commons/oci/database.go index 985d299b..843b57ff 100644 --- a/commons/oci/database.go +++ b/commons/oci/database.go @@ -175,7 +175,7 @@ func isAttrChanged(lastSucObj interface{}, curObj interface{}) bool { if !ok { return false } - curIntPtr, ok := curObj.(*int) + curIntPtr := curObj.(*int) if lastSucIntPtr != nil && curIntPtr != nil && *curIntPtr != 0 && *lastSucIntPtr != *curIntPtr { return true @@ -500,15 +500,6 @@ func getCompleteWorkRetryPolicy() common.RetryPolicy { return getRetryPolicy(shouldRetry) } -func getConflictRetryPolicy() common.RetryPolicy { - // retry for 409 conflict status code - shouldRetry := func(r common.OCIOperationResponse) bool { - return r.Error != nil && r.Response.HTTPResponse().StatusCode == 409 - } - - return getRetryPolicy(shouldRetry) -} - func getRetryPolicy(retryOperation func(common.OCIOperationResponse) bool) common.RetryPolicy { // maximum times of retry attempts := uint(10) diff --git a/commons/oci/provider.go b/commons/oci/provider.go index f5bd6cda..2fafdec9 100644 --- a/commons/oci/provider.go +++ b/commons/oci/provider.go @@ -77,8 +77,8 @@ func GetOCIProvider(kubeClient client.Client, authData APIKeyAuth) (common.Confi } else if authData.ConfigMapName == nil && authData.SecretName == nil { return auth.InstancePrincipalConfigurationProvider() } else { - return nil, errors.New("You have to provide both the OCI ConfigMap and the privateKey to authorize with API signing key, " + - "or leave them both empty to authorize with Instance Principal. Check if the spec configuration is correct.") + return nil, errors.New("both the OCI ConfigMap and the privateKey are required to authorize with API signing key; " + + "leave them both empty to authorize with Instance Principal") } } diff --git a/test/e2e/autonomousdatabase_controller_bind_test.go b/test/e2e/autonomousdatabase_controller_bind_test.go index ff415da9..f10d0685 100644 --- a/test/e2e/autonomousdatabase_controller_bind_test.go +++ b/test/e2e/autonomousdatabase_controller_bind_test.go @@ -42,8 +42,6 @@ import ( "context" "time" - . "github.com/onsi/ginkgo" - . "github.com/onsi/gomega" "github.com/oracle/oci-go-sdk/v45/common" "github.com/oracle/oci-go-sdk/v45/database" "github.com/oracle/oci-go-sdk/v45/workrequests" @@ -61,7 +59,6 @@ import ( // http://onsi.github.io/ginkgo/ to learn more about Ginkgo. var _ = Describe("test ADB binding with hardLink=true", func() { - const bindingHardLinkTestFileName = "bind_adb_hardLink.yaml" var adbLookupKey types.NamespacedName const downloadedWallet = "instance-wallet-secret-1" var adbID *string diff --git a/test/e2e/autonomousdatabase_controller_create_test.go b/test/e2e/autonomousdatabase_controller_create_test.go index 3ecd2804..680bd9f4 100644 --- a/test/e2e/autonomousdatabase_controller_create_test.go +++ b/test/e2e/autonomousdatabase_controller_create_test.go @@ -42,8 +42,6 @@ import ( "context" "time" - . "github.com/onsi/ginkgo" - . "github.com/onsi/gomega" "github.com/oracle/oci-go-sdk/v45/common" "github.com/oracle/oci-go-sdk/v45/database" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" @@ -59,11 +57,6 @@ import ( // http://onsi.github.io/ginkgo/ to learn more about Ginkgo. var _ = Describe("test ADB provisioning", func() { - const ( - changeStateTimeout = time.Second * 300 - changeStateInterval = time.Second * 10 - ) - AfterEach(func() { // IMPORTANT: The operator might have to call reconcile multiple times to finish an operation. // If we do the update immediately, the previous reconciliation will overwrite the changes. diff --git a/test/e2e/behavior/shared_behaviors.go b/test/e2e/behavior/shared_behaviors.go index 501a93b2..38501d38 100644 --- a/test/e2e/behavior/shared_behaviors.go +++ b/test/e2e/behavior/shared_behaviors.go @@ -45,8 +45,8 @@ import ( "reflect" "time" - . "github.com/onsi/ginkgo" - . "github.com/onsi/gomega" + "github.com/onsi/ginkgo" + "github.com/onsi/gomega" "github.com/oracle/oci-go-sdk/v45/common" "github.com/oracle/oci-go-sdk/v45/database" corev1 "k8s.io/api/core/v1" @@ -65,6 +65,19 @@ import ( * to the function, which is likely to be nil or zero value. **************************************************************/ +var ( + Describe = ginkgo.Describe + By = ginkgo.By + GinkgoWriter = ginkgo.GinkgoWriter + Expect = gomega.Expect + BeNil = gomega.BeNil + Eventually = gomega.Eventually + Equal = gomega.Equal + Succeed = gomega.Succeed + BeNumerically = gomega.BeNumerically + BeTrue = gomega.BeTrue +) + func AssertProvision(k8sClient *client.Client, adbLookupKey *types.NamespacedName) func() { return func() { // Set the timeout to 15 minutes. The provision operation might take up to 10 minutes diff --git a/test/e2e/suite_test.go b/test/e2e/suite_test.go index 3fd5cd2e..2ec0d361 100644 --- a/test/e2e/suite_test.go +++ b/test/e2e/suite_test.go @@ -44,8 +44,8 @@ import ( "testing" "time" - . "github.com/onsi/ginkgo" - . "github.com/onsi/gomega" + "github.com/onsi/ginkgo" + "github.com/onsi/gomega" "github.com/onsi/gomega/gexec" "github.com/oracle/oci-go-sdk/v45/common" "github.com/oracle/oci-go-sdk/v45/database" @@ -77,6 +77,21 @@ This test suite runs the integration test which checks the following scenario 5. Test ADB binding with hardLink=true **/ +// To avoid dot import +var ( + BeforeSuite = ginkgo.BeforeSuite + AfterSuite = ginkgo.AfterSuite + Describe = ginkgo.Describe + AfterEach = ginkgo.AfterEach + By = ginkgo.By + It = ginkgo.It + Expect = gomega.Expect + Succeed = gomega.Succeed + HaveOccurred = gomega.HaveOccurred + BeNil = gomega.BeNil + Equal = gomega.Equal +) + var cfg *rest.Config var k8sClient client.Client var configProvider common.ConfigurationProvider @@ -100,15 +115,15 @@ const SharedAdminPassSecretName string = "adb-admin-password" const SharedWalletPassSecretName = "adb-wallet-password" func TestAPIs(t *testing.T) { - RegisterFailHandler(Fail) + gomega.RegisterFailHandler(ginkgo.Fail) - RunSpecsWithDefaultAndCustomReporters(t, + ginkgo.RunSpecsWithDefaultAndCustomReporters(t, "Controller Suite", - []Reporter{printer.NewlineReporter{}}) + []ginkgo.Reporter{printer.NewlineReporter{}}) } -var _ = BeforeSuite(func(done Done) { - logf.SetLogger(zap.New(zap.WriteTo(GinkgoWriter), zap.UseDevMode(true))) +var _ = BeforeSuite(func(done ginkgo.Done) { + logf.SetLogger(zap.New(zap.WriteTo(ginkgo.GinkgoWriter), zap.UseDevMode(true))) By("bootstrapping test environment") testEnv = &envtest.Environment{ @@ -145,7 +160,7 @@ var _ = BeforeSuite(func(done Done) { Expect(err).ToNot(HaveOccurred()) go func() { - defer GinkgoRecover() + defer ginkgo.GinkgoRecover() err = k8sManager.Start(ctrl.SetupSignalHandler()) Expect(err).ToNot(HaveOccurred(), "failed to run manager") gexec.KillAndWait(4 * time.Second) diff --git a/test/e2e/util/oci_config_util.go b/test/e2e/util/oci_config_util.go index 70ee1cab..fcdb29d2 100644 --- a/test/e2e/util/oci_config_util.go +++ b/test/e2e/util/oci_config_util.go @@ -62,8 +62,6 @@ type configUtil struct { //ConfigFileInfo FileInfo *configFileInfo - - provider common.ConfigurationProvider } func (p configUtil) readAndParseConfigFile() (*configFileInfo, error) { @@ -125,17 +123,7 @@ func (p configUtil) CreateOCISecret(secretNamespace string, secretName string) ( } func (p configUtil) GetConfigProvider() (common.ConfigurationProvider, error) { - if p.provider != nil { - return p.provider, nil - } - - newProvider, err := common.ConfigurationProviderFromFileWithProfile(p.OCIConfigPath, p.Profile, "") - if err != nil { - return nil, nil - } - - p.provider = newProvider - return newProvider, nil + return common.ConfigurationProviderFromFileWithProfile(p.OCIConfigPath, p.Profile, "") } func openConfigFile(configFilePath string) (data []byte, err error) { @@ -193,7 +181,7 @@ func parseConfigFile(data []byte, profile string) (info *configFileInfo, err err //Look for profile for i, line := range splitContent { - if match := profileRegex.FindStringSubmatch(line); match != nil && len(match) > 1 && match[1] == profile { + if match := profileRegex.FindStringSubmatch(line); len(match) > 1 && match[1] == profile { start := i + 1 return parseConfigAtLine(start, splitContent) } diff --git a/test/e2e/util/oci_db_request.go b/test/e2e/util/oci_db_request.go index 5fcd3aa1..185d25e9 100644 --- a/test/e2e/util/oci_db_request.go +++ b/test/e2e/util/oci_db_request.go @@ -130,30 +130,6 @@ func generateRetryPolicy(retryFunc func(r common.OCIOperationResponse) bool) com return common.NewRetryPolicy(attempts, retryFunc, nextDuration) } -func NewDisplayNameRetryPolicy(name string) common.RetryPolicy { - shouldRetry := func(r common.OCIOperationResponse) bool { - if databaseResponse, ok := r.Response.(database.GetAutonomousDatabaseResponse); ok { - // do the retry until lifecycle state reaches the passed terminal state - return databaseResponse.LifecycleState != database.AutonomousDatabaseLifecycleStateAvailable || - *databaseResponse.DisplayName != name - } - return true - } - return generateRetryPolicy(shouldRetry) -} - -func NewCPUCoreCountRetryPolicy(count int) common.RetryPolicy { - shouldRetry := func(r common.OCIOperationResponse) bool { - if databaseResponse, ok := r.Response.(database.GetAutonomousDatabaseResponse); ok { - // do the retry until lifecycle state reaches the passed terminal state - return databaseResponse.LifecycleState != database.AutonomousDatabaseLifecycleStateAvailable || - *databaseResponse.CpuCoreCount != count - } - return true - } - return generateRetryPolicy(shouldRetry) -} - func NewLifecycleStateRetryPolicy(lifecycleState database.AutonomousDatabaseLifecycleStateEnum) common.RetryPolicy { shouldRetry := func(r common.OCIOperationResponse) bool { if databaseResponse, ok := r.Response.(database.GetAutonomousDatabaseResponse); ok { diff --git a/test/e2e/util/oci_vault_request.go b/test/e2e/util/oci_vault_request.go deleted file mode 100644 index ef7a793c..00000000 --- a/test/e2e/util/oci_vault_request.go +++ /dev/null @@ -1,255 +0,0 @@ -/* -** Copyright (c) 2021 Oracle and/or its affiliates. -** -** The Universal Permissive License (UPL), Version 1.0 -** -** Subject to the condition set forth below, permission is hereby granted to any -** person obtaining a copy of this software, associated documentation and/or data -** (collectively the "Software"), free of charge and under any and all copyright -** rights in the Software, and any and all patent rights owned or freely -** licensable by each licensor hereunder covering either (i) the unmodified -** Software as contributed to or provided by such licensor, or (ii) the Larger -** Works (as defined below), to deal in both -** -** (a) the Software, and -** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if -** one is included with the Software (each a "Larger Work" to which the Software -** is contributed by such licensors), -** -** without restriction, including without limitation the rights to copy, create -** derivative works of, display, perform, and distribute the Software and make, -** use, sell, offer for sale, import, export, have made, and have sold the -** Software and the Larger Work(s), and to sublicense the foregoing rights on -** either these or other terms. -** -** This license is subject to the following condition: -** The above copyright notice and either this complete permission notice or at -** a minimum a reference to the UPL must be included in all copies or -** substantial portions of the Software. -** -** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -** SOFTWARE. - */ - -package e2eutil - -import ( - "context" - "encoding/base64" - "math" - "time" - - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/keymanagement" - "github.com/oracle/oci-go-sdk/v45/vault" -) - -func waitForVaultStatePolicy(state keymanagement.VaultLifecycleStateEnum) common.RetryPolicy { - shouldRetry := func(r common.OCIOperationResponse) bool { - if _, isServiceError := common.IsServiceError(r.Error); isServiceError { - // not service error, could be network error or other errors which prevents - // request send to server, will do retry here - return true - } - - if vaultResponse, ok := r.Response.(keymanagement.GetVaultResponse); ok { - // do the retry until lifecycle state reaches the passed terminal state - return vaultResponse.Vault.LifecycleState != state - } - - return true - } - - return newRetryPolicy(shouldRetry) -} - -func CreateOCIVault(kmsVaultClient keymanagement.KmsVaultClient, compartmentID *string, vaultName *string) (*keymanagement.Vault, error) { - vaultDetails := keymanagement.CreateVaultDetails{ - CompartmentId: compartmentID, - DisplayName: vaultName, - VaultType: keymanagement.CreateVaultDetailsVaultTypeDefault, - } - - request := keymanagement.CreateVaultRequest{ - CreateVaultDetails: vaultDetails, - } - response, err := kmsVaultClient.CreateVault(context.TODO(), request) - if err != nil { - return nil, err - } - - return &response.Vault, nil -} - -func CreateOCIKey(vaultManagementClient keymanagement.KmsManagementClient, compartmentID *string, keyName *string) (*keymanagement.Key, error) { - keyLength := 32 - - keyShape := keymanagement.KeyShape{ - Algorithm: keymanagement.KeyShapeAlgorithmAes, - Length: &keyLength, - } - - createKeyDetails := keymanagement.CreateKeyDetails{ - CompartmentId: compartmentID, - KeyShape: &keyShape, - DisplayName: keyName, - } - - request := keymanagement.CreateKeyRequest{ - CreateKeyDetails: createKeyDetails, - } - response, err := vaultManagementClient.CreateKey(context.TODO(), request) - if err != nil { - return nil, err - } - - return &response.Key, nil -} - -func CreateOCISecret(vaultClient vault.VaultsClient, compartmentID *string, secretName *string, vaultID *string, keyID *string, content *string) (*string, error) { - encoded := base64.StdEncoding.EncodeToString([]byte(*content)) - - base64Content := vault.Base64SecretContentDetails{ - Name: secretName, - Content: common.String(encoded), - } - - details := vault.CreateSecretDetails{ - CompartmentId: compartmentID, - SecretContent: base64Content, - SecretName: secretName, - VaultId: vaultID, - KeyId: keyID, - } - - request := vault.CreateSecretRequest{ - CreateSecretDetails: details, - } - - // Send the request using the service client - response, err := vaultClient.CreateSecret(context.TODO(), request) - if err != nil { - return nil, err - } - - return response.Secret.Id, nil -} - -func getVault(ctx context.Context, client keymanagement.KmsVaultClient, retryPolicy *common.RetryPolicy, vaultID *string) error { - request := keymanagement.GetVaultRequest{ - VaultId: vaultID, - RequestMetadata: common.RequestMetadata{ - RetryPolicy: retryPolicy, - }, - } - if _, err := client.GetVault(ctx, request); err != nil { - return err - } - return nil -} - -func getKey(client keymanagement.KmsManagementClient, retryPolicy *common.RetryPolicy, keyID *string) error { - request := keymanagement.GetKeyRequest{ - KeyId: keyID, - RequestMetadata: common.RequestMetadata{ - RetryPolicy: retryPolicy, - }, - } - if _, err := client.GetKey(context.TODO(), request); err != nil { - return err - } - return nil -} - -func getSecret(client vault.VaultsClient, retryPolicy *common.RetryPolicy, secretID *string) error { - request := vault.GetSecretRequest{ - SecretId: secretID, - RequestMetadata: common.RequestMetadata{ - RetryPolicy: retryPolicy, - }, - } - if _, err := client.GetSecret(context.TODO(), request); err != nil { - return err - } - return nil -} - -func newRetryPolicy(retryOperation func(common.OCIOperationResponse) bool) common.RetryPolicy { - // maximum times of retry - attempts := uint(10) - - nextDuration := func(r common.OCIOperationResponse) time.Duration { - // you might want wait longer for next retry when your previous one failed - // this function will return the duration as: - // 1s, 2s, 4s, 8s, 16s, 32s, 64s etc... - return time.Duration(math.Pow(float64(2), float64(r.AttemptNumber-1))) * time.Second - } - - return common.NewRetryPolicy(attempts, retryOperation, nextDuration) -} - -func WaitForVaultState(client keymanagement.KmsVaultClient, vaultID *string, state keymanagement.VaultLifecycleStateEnum) error { - shouldRetry := func(r common.OCIOperationResponse) bool { - if vaultResponse, ok := r.Response.(keymanagement.GetVaultResponse); ok { - // do the retry until lifecycle state reaches the passed terminal state - return vaultResponse.Vault.LifecycleState != state - } - - return true - } - - lifecycleStateCheckRetryPolicy := newRetryPolicy(shouldRetry) - - return getVault(context.TODO(), client, &lifecycleStateCheckRetryPolicy, vaultID) -} - -func WaitForKeyState(client keymanagement.KmsManagementClient, keyID *string, state keymanagement.KeyLifecycleStateEnum) error { - shouldRetry := func(r common.OCIOperationResponse) bool { - if keyResponse, ok := r.Response.(keymanagement.GetKeyResponse); ok { - // do the retry until lifecycle state reaches the passed terminal state - return keyResponse.Key.LifecycleState != state - } - - return true - } - - lifecycleStateCheckRetryPolicy := newRetryPolicy(shouldRetry) - - return getKey(client, &lifecycleStateCheckRetryPolicy, keyID) -} - -func WaitForSecretState(client vault.VaultsClient, secretID *string, state vault.SecretLifecycleStateEnum) error { - shouldRetry := func(r common.OCIOperationResponse) bool { - if secretResponse, ok := r.Response.(vault.GetSecretResponse); ok { - // do the retry until lifecycle state reaches the passed terminal state - return secretResponse.Secret.LifecycleState != state - } - - return true - } - - lifecycleStateCheckRetryPolicy := newRetryPolicy(shouldRetry) - - return getSecret(client, &lifecycleStateCheckRetryPolicy, secretID) -} - -// CleanupVault deletes the vault -// Anything encrypted by the keys contained within this vault will be unusable or irretrievable after the vault has been deleted -func CleanupVault(kmsVaultClient keymanagement.KmsVaultClient, vaultID *string) error { - if vaultID == nil { - return nil - } - - request := keymanagement.ScheduleVaultDeletionRequest{ - VaultId: vaultID, - } - if _, err := kmsVaultClient.ScheduleVaultDeletion(context.TODO(), request); err != nil { - return err - } - return nil -} From 5f3531f9e9a79c3edfdeee040694e0676ac70658 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Thu, 9 Dec 2021 09:36:02 -0500 Subject: [PATCH 080/628] removed dot imports and unused codes --- commons/oci/database.go | 11 +- commons/oci/provider.go | 4 +- ...autonomousdatabase_controller_bind_test.go | 3 - ...tonomousdatabase_controller_create_test.go | 7 - test/e2e/behavior/shared_behaviors.go | 17 +- test/e2e/suite_test.go | 31 ++- test/e2e/util/oci_config_util.go | 16 +- test/e2e/util/oci_db_request.go | 24 -- test/e2e/util/oci_vault_request.go | 255 ------------------ 9 files changed, 43 insertions(+), 325 deletions(-) delete mode 100644 test/e2e/util/oci_vault_request.go diff --git a/commons/oci/database.go b/commons/oci/database.go index 985d299b..843b57ff 100644 --- a/commons/oci/database.go +++ b/commons/oci/database.go @@ -175,7 +175,7 @@ func isAttrChanged(lastSucObj interface{}, curObj interface{}) bool { if !ok { return false } - curIntPtr, ok := curObj.(*int) + curIntPtr := curObj.(*int) if lastSucIntPtr != nil && curIntPtr != nil && *curIntPtr != 0 && *lastSucIntPtr != *curIntPtr { return true @@ -500,15 +500,6 @@ func getCompleteWorkRetryPolicy() common.RetryPolicy { return getRetryPolicy(shouldRetry) } -func getConflictRetryPolicy() common.RetryPolicy { - // retry for 409 conflict status code - shouldRetry := func(r common.OCIOperationResponse) bool { - return r.Error != nil && r.Response.HTTPResponse().StatusCode == 409 - } - - return getRetryPolicy(shouldRetry) -} - func getRetryPolicy(retryOperation func(common.OCIOperationResponse) bool) common.RetryPolicy { // maximum times of retry attempts := uint(10) diff --git a/commons/oci/provider.go b/commons/oci/provider.go index f5bd6cda..2fafdec9 100644 --- a/commons/oci/provider.go +++ b/commons/oci/provider.go @@ -77,8 +77,8 @@ func GetOCIProvider(kubeClient client.Client, authData APIKeyAuth) (common.Confi } else if authData.ConfigMapName == nil && authData.SecretName == nil { return auth.InstancePrincipalConfigurationProvider() } else { - return nil, errors.New("You have to provide both the OCI ConfigMap and the privateKey to authorize with API signing key, " + - "or leave them both empty to authorize with Instance Principal. Check if the spec configuration is correct.") + return nil, errors.New("both the OCI ConfigMap and the privateKey are required to authorize with API signing key; " + + "leave them both empty to authorize with Instance Principal") } } diff --git a/test/e2e/autonomousdatabase_controller_bind_test.go b/test/e2e/autonomousdatabase_controller_bind_test.go index ff415da9..f10d0685 100644 --- a/test/e2e/autonomousdatabase_controller_bind_test.go +++ b/test/e2e/autonomousdatabase_controller_bind_test.go @@ -42,8 +42,6 @@ import ( "context" "time" - . "github.com/onsi/ginkgo" - . "github.com/onsi/gomega" "github.com/oracle/oci-go-sdk/v45/common" "github.com/oracle/oci-go-sdk/v45/database" "github.com/oracle/oci-go-sdk/v45/workrequests" @@ -61,7 +59,6 @@ import ( // http://onsi.github.io/ginkgo/ to learn more about Ginkgo. var _ = Describe("test ADB binding with hardLink=true", func() { - const bindingHardLinkTestFileName = "bind_adb_hardLink.yaml" var adbLookupKey types.NamespacedName const downloadedWallet = "instance-wallet-secret-1" var adbID *string diff --git a/test/e2e/autonomousdatabase_controller_create_test.go b/test/e2e/autonomousdatabase_controller_create_test.go index 3ecd2804..680bd9f4 100644 --- a/test/e2e/autonomousdatabase_controller_create_test.go +++ b/test/e2e/autonomousdatabase_controller_create_test.go @@ -42,8 +42,6 @@ import ( "context" "time" - . "github.com/onsi/ginkgo" - . "github.com/onsi/gomega" "github.com/oracle/oci-go-sdk/v45/common" "github.com/oracle/oci-go-sdk/v45/database" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" @@ -59,11 +57,6 @@ import ( // http://onsi.github.io/ginkgo/ to learn more about Ginkgo. var _ = Describe("test ADB provisioning", func() { - const ( - changeStateTimeout = time.Second * 300 - changeStateInterval = time.Second * 10 - ) - AfterEach(func() { // IMPORTANT: The operator might have to call reconcile multiple times to finish an operation. // If we do the update immediately, the previous reconciliation will overwrite the changes. diff --git a/test/e2e/behavior/shared_behaviors.go b/test/e2e/behavior/shared_behaviors.go index 501a93b2..38501d38 100644 --- a/test/e2e/behavior/shared_behaviors.go +++ b/test/e2e/behavior/shared_behaviors.go @@ -45,8 +45,8 @@ import ( "reflect" "time" - . "github.com/onsi/ginkgo" - . "github.com/onsi/gomega" + "github.com/onsi/ginkgo" + "github.com/onsi/gomega" "github.com/oracle/oci-go-sdk/v45/common" "github.com/oracle/oci-go-sdk/v45/database" corev1 "k8s.io/api/core/v1" @@ -65,6 +65,19 @@ import ( * to the function, which is likely to be nil or zero value. **************************************************************/ +var ( + Describe = ginkgo.Describe + By = ginkgo.By + GinkgoWriter = ginkgo.GinkgoWriter + Expect = gomega.Expect + BeNil = gomega.BeNil + Eventually = gomega.Eventually + Equal = gomega.Equal + Succeed = gomega.Succeed + BeNumerically = gomega.BeNumerically + BeTrue = gomega.BeTrue +) + func AssertProvision(k8sClient *client.Client, adbLookupKey *types.NamespacedName) func() { return func() { // Set the timeout to 15 minutes. The provision operation might take up to 10 minutes diff --git a/test/e2e/suite_test.go b/test/e2e/suite_test.go index 3fd5cd2e..2ec0d361 100644 --- a/test/e2e/suite_test.go +++ b/test/e2e/suite_test.go @@ -44,8 +44,8 @@ import ( "testing" "time" - . "github.com/onsi/ginkgo" - . "github.com/onsi/gomega" + "github.com/onsi/ginkgo" + "github.com/onsi/gomega" "github.com/onsi/gomega/gexec" "github.com/oracle/oci-go-sdk/v45/common" "github.com/oracle/oci-go-sdk/v45/database" @@ -77,6 +77,21 @@ This test suite runs the integration test which checks the following scenario 5. Test ADB binding with hardLink=true **/ +// To avoid dot import +var ( + BeforeSuite = ginkgo.BeforeSuite + AfterSuite = ginkgo.AfterSuite + Describe = ginkgo.Describe + AfterEach = ginkgo.AfterEach + By = ginkgo.By + It = ginkgo.It + Expect = gomega.Expect + Succeed = gomega.Succeed + HaveOccurred = gomega.HaveOccurred + BeNil = gomega.BeNil + Equal = gomega.Equal +) + var cfg *rest.Config var k8sClient client.Client var configProvider common.ConfigurationProvider @@ -100,15 +115,15 @@ const SharedAdminPassSecretName string = "adb-admin-password" const SharedWalletPassSecretName = "adb-wallet-password" func TestAPIs(t *testing.T) { - RegisterFailHandler(Fail) + gomega.RegisterFailHandler(ginkgo.Fail) - RunSpecsWithDefaultAndCustomReporters(t, + ginkgo.RunSpecsWithDefaultAndCustomReporters(t, "Controller Suite", - []Reporter{printer.NewlineReporter{}}) + []ginkgo.Reporter{printer.NewlineReporter{}}) } -var _ = BeforeSuite(func(done Done) { - logf.SetLogger(zap.New(zap.WriteTo(GinkgoWriter), zap.UseDevMode(true))) +var _ = BeforeSuite(func(done ginkgo.Done) { + logf.SetLogger(zap.New(zap.WriteTo(ginkgo.GinkgoWriter), zap.UseDevMode(true))) By("bootstrapping test environment") testEnv = &envtest.Environment{ @@ -145,7 +160,7 @@ var _ = BeforeSuite(func(done Done) { Expect(err).ToNot(HaveOccurred()) go func() { - defer GinkgoRecover() + defer ginkgo.GinkgoRecover() err = k8sManager.Start(ctrl.SetupSignalHandler()) Expect(err).ToNot(HaveOccurred(), "failed to run manager") gexec.KillAndWait(4 * time.Second) diff --git a/test/e2e/util/oci_config_util.go b/test/e2e/util/oci_config_util.go index 70ee1cab..fcdb29d2 100644 --- a/test/e2e/util/oci_config_util.go +++ b/test/e2e/util/oci_config_util.go @@ -62,8 +62,6 @@ type configUtil struct { //ConfigFileInfo FileInfo *configFileInfo - - provider common.ConfigurationProvider } func (p configUtil) readAndParseConfigFile() (*configFileInfo, error) { @@ -125,17 +123,7 @@ func (p configUtil) CreateOCISecret(secretNamespace string, secretName string) ( } func (p configUtil) GetConfigProvider() (common.ConfigurationProvider, error) { - if p.provider != nil { - return p.provider, nil - } - - newProvider, err := common.ConfigurationProviderFromFileWithProfile(p.OCIConfigPath, p.Profile, "") - if err != nil { - return nil, nil - } - - p.provider = newProvider - return newProvider, nil + return common.ConfigurationProviderFromFileWithProfile(p.OCIConfigPath, p.Profile, "") } func openConfigFile(configFilePath string) (data []byte, err error) { @@ -193,7 +181,7 @@ func parseConfigFile(data []byte, profile string) (info *configFileInfo, err err //Look for profile for i, line := range splitContent { - if match := profileRegex.FindStringSubmatch(line); match != nil && len(match) > 1 && match[1] == profile { + if match := profileRegex.FindStringSubmatch(line); len(match) > 1 && match[1] == profile { start := i + 1 return parseConfigAtLine(start, splitContent) } diff --git a/test/e2e/util/oci_db_request.go b/test/e2e/util/oci_db_request.go index 5fcd3aa1..185d25e9 100644 --- a/test/e2e/util/oci_db_request.go +++ b/test/e2e/util/oci_db_request.go @@ -130,30 +130,6 @@ func generateRetryPolicy(retryFunc func(r common.OCIOperationResponse) bool) com return common.NewRetryPolicy(attempts, retryFunc, nextDuration) } -func NewDisplayNameRetryPolicy(name string) common.RetryPolicy { - shouldRetry := func(r common.OCIOperationResponse) bool { - if databaseResponse, ok := r.Response.(database.GetAutonomousDatabaseResponse); ok { - // do the retry until lifecycle state reaches the passed terminal state - return databaseResponse.LifecycleState != database.AutonomousDatabaseLifecycleStateAvailable || - *databaseResponse.DisplayName != name - } - return true - } - return generateRetryPolicy(shouldRetry) -} - -func NewCPUCoreCountRetryPolicy(count int) common.RetryPolicy { - shouldRetry := func(r common.OCIOperationResponse) bool { - if databaseResponse, ok := r.Response.(database.GetAutonomousDatabaseResponse); ok { - // do the retry until lifecycle state reaches the passed terminal state - return databaseResponse.LifecycleState != database.AutonomousDatabaseLifecycleStateAvailable || - *databaseResponse.CpuCoreCount != count - } - return true - } - return generateRetryPolicy(shouldRetry) -} - func NewLifecycleStateRetryPolicy(lifecycleState database.AutonomousDatabaseLifecycleStateEnum) common.RetryPolicy { shouldRetry := func(r common.OCIOperationResponse) bool { if databaseResponse, ok := r.Response.(database.GetAutonomousDatabaseResponse); ok { diff --git a/test/e2e/util/oci_vault_request.go b/test/e2e/util/oci_vault_request.go deleted file mode 100644 index ef7a793c..00000000 --- a/test/e2e/util/oci_vault_request.go +++ /dev/null @@ -1,255 +0,0 @@ -/* -** Copyright (c) 2021 Oracle and/or its affiliates. -** -** The Universal Permissive License (UPL), Version 1.0 -** -** Subject to the condition set forth below, permission is hereby granted to any -** person obtaining a copy of this software, associated documentation and/or data -** (collectively the "Software"), free of charge and under any and all copyright -** rights in the Software, and any and all patent rights owned or freely -** licensable by each licensor hereunder covering either (i) the unmodified -** Software as contributed to or provided by such licensor, or (ii) the Larger -** Works (as defined below), to deal in both -** -** (a) the Software, and -** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if -** one is included with the Software (each a "Larger Work" to which the Software -** is contributed by such licensors), -** -** without restriction, including without limitation the rights to copy, create -** derivative works of, display, perform, and distribute the Software and make, -** use, sell, offer for sale, import, export, have made, and have sold the -** Software and the Larger Work(s), and to sublicense the foregoing rights on -** either these or other terms. -** -** This license is subject to the following condition: -** The above copyright notice and either this complete permission notice or at -** a minimum a reference to the UPL must be included in all copies or -** substantial portions of the Software. -** -** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -** SOFTWARE. - */ - -package e2eutil - -import ( - "context" - "encoding/base64" - "math" - "time" - - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/keymanagement" - "github.com/oracle/oci-go-sdk/v45/vault" -) - -func waitForVaultStatePolicy(state keymanagement.VaultLifecycleStateEnum) common.RetryPolicy { - shouldRetry := func(r common.OCIOperationResponse) bool { - if _, isServiceError := common.IsServiceError(r.Error); isServiceError { - // not service error, could be network error or other errors which prevents - // request send to server, will do retry here - return true - } - - if vaultResponse, ok := r.Response.(keymanagement.GetVaultResponse); ok { - // do the retry until lifecycle state reaches the passed terminal state - return vaultResponse.Vault.LifecycleState != state - } - - return true - } - - return newRetryPolicy(shouldRetry) -} - -func CreateOCIVault(kmsVaultClient keymanagement.KmsVaultClient, compartmentID *string, vaultName *string) (*keymanagement.Vault, error) { - vaultDetails := keymanagement.CreateVaultDetails{ - CompartmentId: compartmentID, - DisplayName: vaultName, - VaultType: keymanagement.CreateVaultDetailsVaultTypeDefault, - } - - request := keymanagement.CreateVaultRequest{ - CreateVaultDetails: vaultDetails, - } - response, err := kmsVaultClient.CreateVault(context.TODO(), request) - if err != nil { - return nil, err - } - - return &response.Vault, nil -} - -func CreateOCIKey(vaultManagementClient keymanagement.KmsManagementClient, compartmentID *string, keyName *string) (*keymanagement.Key, error) { - keyLength := 32 - - keyShape := keymanagement.KeyShape{ - Algorithm: keymanagement.KeyShapeAlgorithmAes, - Length: &keyLength, - } - - createKeyDetails := keymanagement.CreateKeyDetails{ - CompartmentId: compartmentID, - KeyShape: &keyShape, - DisplayName: keyName, - } - - request := keymanagement.CreateKeyRequest{ - CreateKeyDetails: createKeyDetails, - } - response, err := vaultManagementClient.CreateKey(context.TODO(), request) - if err != nil { - return nil, err - } - - return &response.Key, nil -} - -func CreateOCISecret(vaultClient vault.VaultsClient, compartmentID *string, secretName *string, vaultID *string, keyID *string, content *string) (*string, error) { - encoded := base64.StdEncoding.EncodeToString([]byte(*content)) - - base64Content := vault.Base64SecretContentDetails{ - Name: secretName, - Content: common.String(encoded), - } - - details := vault.CreateSecretDetails{ - CompartmentId: compartmentID, - SecretContent: base64Content, - SecretName: secretName, - VaultId: vaultID, - KeyId: keyID, - } - - request := vault.CreateSecretRequest{ - CreateSecretDetails: details, - } - - // Send the request using the service client - response, err := vaultClient.CreateSecret(context.TODO(), request) - if err != nil { - return nil, err - } - - return response.Secret.Id, nil -} - -func getVault(ctx context.Context, client keymanagement.KmsVaultClient, retryPolicy *common.RetryPolicy, vaultID *string) error { - request := keymanagement.GetVaultRequest{ - VaultId: vaultID, - RequestMetadata: common.RequestMetadata{ - RetryPolicy: retryPolicy, - }, - } - if _, err := client.GetVault(ctx, request); err != nil { - return err - } - return nil -} - -func getKey(client keymanagement.KmsManagementClient, retryPolicy *common.RetryPolicy, keyID *string) error { - request := keymanagement.GetKeyRequest{ - KeyId: keyID, - RequestMetadata: common.RequestMetadata{ - RetryPolicy: retryPolicy, - }, - } - if _, err := client.GetKey(context.TODO(), request); err != nil { - return err - } - return nil -} - -func getSecret(client vault.VaultsClient, retryPolicy *common.RetryPolicy, secretID *string) error { - request := vault.GetSecretRequest{ - SecretId: secretID, - RequestMetadata: common.RequestMetadata{ - RetryPolicy: retryPolicy, - }, - } - if _, err := client.GetSecret(context.TODO(), request); err != nil { - return err - } - return nil -} - -func newRetryPolicy(retryOperation func(common.OCIOperationResponse) bool) common.RetryPolicy { - // maximum times of retry - attempts := uint(10) - - nextDuration := func(r common.OCIOperationResponse) time.Duration { - // you might want wait longer for next retry when your previous one failed - // this function will return the duration as: - // 1s, 2s, 4s, 8s, 16s, 32s, 64s etc... - return time.Duration(math.Pow(float64(2), float64(r.AttemptNumber-1))) * time.Second - } - - return common.NewRetryPolicy(attempts, retryOperation, nextDuration) -} - -func WaitForVaultState(client keymanagement.KmsVaultClient, vaultID *string, state keymanagement.VaultLifecycleStateEnum) error { - shouldRetry := func(r common.OCIOperationResponse) bool { - if vaultResponse, ok := r.Response.(keymanagement.GetVaultResponse); ok { - // do the retry until lifecycle state reaches the passed terminal state - return vaultResponse.Vault.LifecycleState != state - } - - return true - } - - lifecycleStateCheckRetryPolicy := newRetryPolicy(shouldRetry) - - return getVault(context.TODO(), client, &lifecycleStateCheckRetryPolicy, vaultID) -} - -func WaitForKeyState(client keymanagement.KmsManagementClient, keyID *string, state keymanagement.KeyLifecycleStateEnum) error { - shouldRetry := func(r common.OCIOperationResponse) bool { - if keyResponse, ok := r.Response.(keymanagement.GetKeyResponse); ok { - // do the retry until lifecycle state reaches the passed terminal state - return keyResponse.Key.LifecycleState != state - } - - return true - } - - lifecycleStateCheckRetryPolicy := newRetryPolicy(shouldRetry) - - return getKey(client, &lifecycleStateCheckRetryPolicy, keyID) -} - -func WaitForSecretState(client vault.VaultsClient, secretID *string, state vault.SecretLifecycleStateEnum) error { - shouldRetry := func(r common.OCIOperationResponse) bool { - if secretResponse, ok := r.Response.(vault.GetSecretResponse); ok { - // do the retry until lifecycle state reaches the passed terminal state - return secretResponse.Secret.LifecycleState != state - } - - return true - } - - lifecycleStateCheckRetryPolicy := newRetryPolicy(shouldRetry) - - return getSecret(client, &lifecycleStateCheckRetryPolicy, secretID) -} - -// CleanupVault deletes the vault -// Anything encrypted by the keys contained within this vault will be unusable or irretrievable after the vault has been deleted -func CleanupVault(kmsVaultClient keymanagement.KmsVaultClient, vaultID *string) error { - if vaultID == nil { - return nil - } - - request := keymanagement.ScheduleVaultDeletionRequest{ - VaultId: vaultID, - } - if _, err := kmsVaultClient.ScheduleVaultDeletion(context.TODO(), request); err != nil { - return err - } - return nil -} From 8166b3f68004becf9020cfab60b9afc28b520a3f Mon Sep 17 00:00:00 2001 From: Gajanan Bhat Date: Mon, 13 Dec 2021 15:54:37 +0530 Subject: [PATCH 081/628] Updated MODIFY option --- apis/database/v1alpha1/pdb_types.go | 19 +- apis/database/v1alpha1/pdb_webhook.go | 15 +- .../crd/bases/database.oracle.com_pdbs.yaml | 32 +++- controllers/database/pdb_controller.go | 180 ++++++++++++++---- 4 files changed, 195 insertions(+), 51 deletions(-) diff --git a/apis/database/v1alpha1/pdb_types.go b/apis/database/v1alpha1/pdb_types.go index 7b4e6514..686ebbbe 100644 --- a/apis/database/v1alpha1/pdb_types.go +++ b/apis/database/v1alpha1/pdb_types.go @@ -96,8 +96,14 @@ type PDBSpec struct { // Whether you need the script only or execute the script GetScript *bool `json:"getScript,omitempty"` // Action to be taken: Create or Clone or Plug or Unplug - // +kubebuilder:validation:Enum=Create;Clone;Plug;Unplug;Delete + // +kubebuilder:validation:Enum=Create;Clone;Plug;Unplug;Delete;Modify Action string `json:"action"` + // Extra options for opening and closing a PDB + // +kubebuilder:validation:Enum=IMMEDIATE;NORMAL;READ ONLY;READ WRITE;RESTRICTED + ModifyOption string `json:"modifyOption,omitempty"` + // The target state of the PDB + // +kubebuilder:validation:Enum=OPEN;CLOSE + PDBState string `json:"pdbState,omitempty"` } // PDBAdminName defines the secret containing Sys Admin User mapped to key 'adminName' for PDB @@ -133,10 +139,14 @@ type PDBStatus struct { // PDB Connect String ConnString string `json:"connString,omitempty"` - // Phase of the CDB Resource + // Phase of the PDB Resource Phase string `json:"phase"` - // CDB Resource Status + // PDB Resource Status Status bool `json:"status"` + // Open mode of the PDB + OpenMode string `json:"openMode,omitempty"` + // Modify Option of the PDB + ModifyOption string `json:"modifyOption,omitempty"` // Message Msg string `json:"msg,omitempty"` // Last Completed Action @@ -148,8 +158,9 @@ type PDBStatus struct { // +kubebuilder:printcolumn:JSONPath=".status.connString",name="Connect String",type="string",description="The connect string to be used" // +kubebuilder:printcolumn:JSONPath=".spec.cdbName",name="CDB Name",type="string",description="Name of the CDB" // +kubebuilder:printcolumn:JSONPath=".spec.pdbName",name="PDB Name",type="string",description="Name of the PDB" +// +kubebuilder:printcolumn:JSONPath=".status.openMode",name="PDB State",type="string",description="PDB Open Mode" // +kubebuilder:printcolumn:JSONPath=".spec.totalSize",name="PDB Size",type="string",description="Total Size of the PDB" -// +kubebuilder:printcolumn:JSONPath=".status.phase",name="Status",type="string",description="Status of the PDB" +// +kubebuilder:printcolumn:JSONPath=".status.phase",name="Status",type="string",description="Status of the PDB Resource" // +kubebuilder:printcolumn:JSONPath=".status.msg",name="Message",type="string",description="Error message, if any" // PDB is the Schema for the pdbs API type PDB struct { diff --git a/apis/database/v1alpha1/pdb_webhook.go b/apis/database/v1alpha1/pdb_webhook.go index afbaba99..1c42ca85 100644 --- a/apis/database/v1alpha1/pdb_webhook.go +++ b/apis/database/v1alpha1/pdb_webhook.go @@ -76,7 +76,7 @@ func (r *PDB) Default() { r.Spec.DropAction = "KEEP" pdblog.Info(" - dropAction : KEEP") } - } else { + } else if action != "MODIFY" { if r.Spec.ReuseTempFile == nil { r.Spec.ReuseTempFile = new(bool) *r.Spec.ReuseTempFile = true @@ -202,6 +202,15 @@ func (r *PDB) validateAction(allErrs *field.ErrorList) { if *(r.Spec.TDEExport) { r.validateTDEInfo(allErrs) } + case "MODIFY": + if r.Spec.PDBState == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("pdbState"), "Please specify target state of PDB")) + } + if r.Spec.ModifyOption == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("modifyOption"), "Please specify an option for opening/closing a PDB")) + } } } @@ -213,7 +222,7 @@ func (r *PDB) ValidateUpdate(old runtime.Object) error { action := strings.ToUpper(r.Spec.Action) // If PDB CR has been created and in Ready state, only allow updates if the "action" value has changed as well - if (r.Status.Phase == "Ready") && (r.Status.Action == action) { + if (r.Status.Phase == "Ready") && (r.Status.Action != "MODIFY") && (r.Status.Action == action) { allErrs = append(allErrs, field.Required(field.NewPath("spec").Child("action"), "New action also needs to be specified after PDB is in Ready state")) } else { @@ -225,7 +234,7 @@ func (r *PDB) ValidateUpdate(old runtime.Object) error { r.validateAction(&allErrs) // Check TDE requirements - if (action != "DELETE") && (*(r.Spec.TDEImport) || *(r.Spec.TDEExport)) { + if action != "DELETE" && action != "MODIFY" && (*(r.Spec.TDEImport) || *(r.Spec.TDEExport)) { r.validateTDEInfo(&allErrs) } } diff --git a/config/crd/bases/database.oracle.com_pdbs.yaml b/config/crd/bases/database.oracle.com_pdbs.yaml index 0806fe85..951c95a7 100644 --- a/config/crd/bases/database.oracle.com_pdbs.yaml +++ b/config/crd/bases/database.oracle.com_pdbs.yaml @@ -29,11 +29,15 @@ spec: jsonPath: .spec.pdbName name: PDB Name type: string + - description: PDB Open Mode + jsonPath: .status.openMode + name: PDB State + type: string - description: Total Size of the PDB jsonPath: .spec.totalSize name: PDB Size type: string - - description: Status of the PDB + - description: Status of the PDB Resource jsonPath: .status.phase name: Status type: string @@ -69,6 +73,7 @@ spec: - Plug - Unplug - Delete + - Modify type: string adminName: description: The administrator username for the new PDB. This property @@ -139,10 +144,25 @@ spec: getScript: description: Whether you need the script only or execute the script type: boolean + modifyOption: + description: Extra options for opening and closing a PDB + enum: + - IMMEDIATE + - NORMAL + - READ ONLY + - READ WRITE + - RESTRICTED + type: string pdbName: description: The name of the new PDB. Relevant for both Create and Plug Actions. type: string + pdbState: + description: The target state of the PDB + enum: + - OPEN + - CLOSE + type: string reuseTempFile: description: Whether to reuse temp file type: boolean @@ -234,14 +254,20 @@ spec: connString: description: PDB Connect String type: string + modifyOption: + description: Modify Option of the PDB + type: string msg: description: Message type: string + openMode: + description: Open mode of the PDB + type: string phase: - description: Phase of the CDB Resource + description: Phase of the PDB Resource type: string status: - description: CDB Resource Status + description: PDB Resource Status type: boolean required: - phase diff --git a/controllers/database/pdb_controller.go b/controllers/database/pdb_controller.go index cc5fcc3f..778ae818 100644 --- a/controllers/database/pdb_controller.go +++ b/controllers/database/pdb_controller.go @@ -101,15 +101,15 @@ type ORDSError struct { } var ( - pdbPhaseValidate = "Validating" - pdbPhaseCreate = "Creating" - pdbPhasePlug = "Plugging" - pdbPhaseUnplug = "Unplugging" - pdbPhaseClone = "Cloning" - pdbPhaseFinish = "Finishing" - pdbPhaseReady = "Ready" - pdbPhaseDelete = "Deleting" - pdbPhaseFail = "Failed" + pdbPhaseCreate = "Creating" + pdbPhasePlug = "Plugging" + pdbPhaseUnplug = "Unplugging" + pdbPhaseClone = "Cloning" + pdbPhaseFinish = "Finishing" + pdbPhaseReady = "Ready" + pdbPhaseDelete = "Deleting" + pdbPhaseModify = "Modifying" + pdbPhaseFail = "Failed" ) const PDBFinalizer = "database.oracle.com/PDBfinalizer" @@ -142,6 +142,9 @@ func (r *PDBReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.R defer func() { //log.Info("DEFER PDB", "Name", pdb.Name, "Phase", pdb.Status.Phase, "Status", strconv.FormatBool(pdb.Status.Status)) if !pdb.Status.Status { + if pdb.Status.Phase == pdbPhaseReady { + pdb.Status.Status = true + } if err := r.Status().Update(ctx, pdb); err != nil { log.Error(err, "Failed to update status for :"+pdb.Name, "err", err.Error()) } @@ -169,17 +172,19 @@ func (r *PDBReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.R return requeueY, nil } - //action := strings.ToUpper(pdb.Spec.Action) - - /*if pdb.Status.Phase != pdbPhaseFail && pdb.Status.Action != "" && pdb.Status.Action != action { - pdb.Status.Status = false - }*/ - - //log.Info("PDB PHASE STATUS:", "Name", pdb.Name, "Phase", pdb.Status.Phase, "Status", strconv.FormatBool(pdb.Status.Status), "Last Action", pdb.Status.Action) + action := strings.ToUpper(pdb.Spec.Action) - //if (pdb.Status.Phase != pdbPhaseReady && pdb.Status.Phase != pdbPhaseFail && !pdb.Status.Status) || (pdb.Status.Action != "" && pdb.Status.Action != action) { - //r.validatePhase(ctx, req, pdb) - //} + if (pdb.Status.Phase == pdbPhaseReady) && (pdb.Status.Action != "") && (action == "MODIFY" || pdb.Status.Action != action) { + if action == "MODIFY" { + modOption := pdb.Spec.PDBState + "-" + pdb.Spec.ModifyOption + // To prevent Reconcile from Modifying again whenever the Operator gets re-started + if pdb.Status.ModifyOption != modOption { + pdb.Status.Status = false + } + } else { + pdb.Status.Status = false + } + } if !pdb.Status.Status { r.validatePhase(ctx, req, pdb) @@ -195,12 +200,11 @@ func (r *PDBReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.R err = r.plugPDB(ctx, req, pdb) case pdbPhaseUnplug: err = r.unplugPDB(ctx, req, pdb) + case pdbPhaseModify: + err = r.modifyPDB(ctx, req, pdb) case pdbPhaseDelete: err = r.deletePDB(ctx, req, pdb) - case pdbPhaseReady, pdbPhaseFail: - pdb.Status.Status = true default: - //pdb.Status.Phase = pdbPhaseValidate log.Info("DEFAULT:", "Name", pdb.Name, "Phase", phase, "Status", strconv.FormatBool(pdb.Status.Status)) return requeueN, nil } @@ -237,6 +241,8 @@ func (r *PDBReconciler) validatePhase(ctx context.Context, req ctrl.Request, pdb pdb.Status.Phase = pdbPhasePlug case "UNPLUG": pdb.Status.Phase = pdbPhaseUnplug + case "MODIFY": + pdb.Status.Phase = pdbPhaseModify case "DELETE": pdb.Status.Phase = pdbPhaseDelete } @@ -326,7 +332,7 @@ func (r *PDBReconciler) getSecret(ctx context.Context, req ctrl.Request, pdb *db /************************************************* * Issue a REST API Call to the ORDS container /************************************************/ -func (r *PDBReconciler) callAPI(ctx context.Context, req ctrl.Request, pdb *dbapi.PDB, url string, payload map[string]string, action string) error { +func (r *PDBReconciler) callAPI(ctx context.Context, req ctrl.Request, pdb *dbapi.PDB, url string, payload map[string]string, action string) (string, error) { log := r.Log.WithValues("callAPI", req.NamespacedName) var err error @@ -336,7 +342,7 @@ func (r *PDBReconciler) callAPI(ctx context.Context, req ctrl.Request, pdb *dbap cdb, err := r.getCDBResource(ctx, req, pdb) if err != nil { - return err + return "", err } // Get Web Server User @@ -345,10 +351,10 @@ func (r *PDBReconciler) callAPI(ctx context.Context, req ctrl.Request, pdb *dbap if err != nil { if apierrors.IsNotFound(err) { log.Info("Secret not found:" + cdb.Spec.WebServerUser.Secret.SecretName) - return err + return "", err } log.Error(err, "Unable to get the secret.") - return err + return "", err } webUser := string(secret.Data[cdb.Spec.WebServerUser.Secret.Key]) @@ -358,20 +364,25 @@ func (r *PDBReconciler) callAPI(ctx context.Context, req ctrl.Request, pdb *dbap if err != nil { if apierrors.IsNotFound(err) { log.Info("Secret not found:" + cdb.Spec.WebServerPwd.Secret.SecretName) - return err + return "", err } log.Error(err, "Unable to get the secret.") - return err + return "", err } webUserPwd := string(secret.Data[cdb.Spec.WebServerPwd.Secret.Key]) - //fmt.Println("payload:", payload) + var httpreq *http.Request + if action == "GET" { + httpreq, err = http.NewRequest(action, url, nil) + } else { + //fmt.Println("payload:", payload) + jsonValue, _ := json.Marshal(payload) + httpreq, err = http.NewRequest(action, url, bytes.NewBuffer(jsonValue)) + } - jsonValue, _ := json.Marshal(payload) - httpreq, err := http.NewRequest(action, url, bytes.NewBuffer(jsonValue)) if err != nil { log.Info("Unable to create HTTP Request for PDB : "+pdb.Name, "err", err.Error()) - return err + return "", err } httpreq.Header.Add("Accept", "application/json") @@ -383,7 +394,7 @@ func (r *PDBReconciler) callAPI(ctx context.Context, req ctrl.Request, pdb *dbap log.Error(err, "Failed - Could not connect to ORDS Pod", "err", err.Error()) pdb.Status.Msg = "Could not connect to ORDS Pod" r.Recorder.Eventf(pdb, corev1.EventTypeWarning, "ORDSError", "Failed: Could not connect to ORDS Pod") - return err + return "", err } if resp.StatusCode != http.StatusOK { @@ -396,7 +407,7 @@ func (r *PDBReconciler) callAPI(ctx context.Context, req ctrl.Request, pdb *dbap r.Recorder.Eventf(pdb, corev1.EventTypeWarning, "ORDSError", "Failed: %s", apiErr.Message) //fmt.Printf("%+v", apiErr) //fmt.Println(string(bb)) - return errors.New("ORDS Error") + return "", errors.New("ORDS Error") } defer resp.Body.Close() @@ -405,6 +416,7 @@ func (r *PDBReconciler) callAPI(ctx context.Context, req ctrl.Request, pdb *dbap if err != nil { fmt.Print(err.Error()) } + respData := string(bodyBytes) //fmt.Println(string(bodyBytes)) var apiResponse RESTSQLCollection @@ -425,10 +437,10 @@ func (r *PDBReconciler) callAPI(ctx context.Context, req ctrl.Request, pdb *dbap } if errFound { - return errors.New("Oracle Error") + return "", errors.New("Oracle Error") } - return nil + return respData, nil } /************************************************* @@ -487,7 +499,7 @@ func (r *PDBReconciler) createPDB(ctx context.Context, req ctrl.Request, pdb *db if err := r.Status().Update(ctx, pdb); err != nil { log.Error(err, "Failed to update status for :"+pdb.Name, "err", err.Error()) } - err = r.callAPI(ctx, req, pdb, url, values, "POST") + _, err = r.callAPI(ctx, req, pdb, url, values, "POST") if err != nil { return err } @@ -496,6 +508,7 @@ func (r *PDBReconciler) createPDB(ctx context.Context, req ctrl.Request, pdb *db pdb.Status.ConnString = cdb.Spec.SCANName + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName log.Info("Created PDB Resource", "PDB Name", pdb.Spec.PDBName) + r.getPDBState(ctx, req, pdb) return nil } @@ -540,7 +553,7 @@ func (r *PDBReconciler) clonePDB(ctx context.Context, req ctrl.Request, pdb *dba if err := r.Status().Update(ctx, pdb); err != nil { log.Error(err, "Failed to update status for :"+pdb.Name, "err", err.Error()) } - err = r.callAPI(ctx, req, pdb, url, values, "POST") + _, err = r.callAPI(ctx, req, pdb, url, values, "POST") if err != nil { return err } @@ -549,6 +562,7 @@ func (r *PDBReconciler) clonePDB(ctx context.Context, req ctrl.Request, pdb *dba pdb.Status.ConnString = cdb.Spec.SCANName + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName log.Info("Cloned PDB successfully", "Source PDB Name", pdb.Spec.SrcPDBName, "Clone PDB Name", pdb.Spec.PDBName) + r.getPDBState(ctx, req, pdb) return nil } @@ -593,6 +607,7 @@ func (r *PDBReconciler) plugPDB(ctx context.Context, req ctrl.Request, pdb *dbap values["tdePassword"] = tdePassword values["tdeKeystorePath"] = pdb.Spec.TDEKeystorePath values["tdeSecret"] = tdeSecret + values["tdeImport"] = strconv.FormatBool(*(pdb.Spec.TDEImport)) } if *(pdb.Spec.AsClone) { values["asClone"] = strconv.FormatBool(*(pdb.Spec.AsClone)) @@ -605,7 +620,7 @@ func (r *PDBReconciler) plugPDB(ctx context.Context, req ctrl.Request, pdb *dbap if err := r.Status().Update(ctx, pdb); err != nil { log.Error(err, "Failed to update status for :"+pdb.Name, "err", err.Error()) } - err = r.callAPI(ctx, req, pdb, url, values, "POST") + _, err = r.callAPI(ctx, req, pdb, url, values, "POST") if err != nil { return err } @@ -614,6 +629,7 @@ func (r *PDBReconciler) plugPDB(ctx context.Context, req ctrl.Request, pdb *dbap pdb.Status.ConnString = cdb.Spec.SCANName + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName log.Info("Successfully plugged PDB", "PDB Name", pdb.Spec.PDBName) + r.getPDBState(ctx, req, pdb) return nil } @@ -649,6 +665,7 @@ func (r *PDBReconciler) unplugPDB(ctx context.Context, req ctrl.Request, pdb *db values["tdePassword"] = tdePassword values["tdeKeystorePath"] = pdb.Spec.TDEKeystorePath values["tdeSecret"] = tdeSecret + values["tdeExport"] = strconv.FormatBool(*(pdb.Spec.TDEExport)) } url := "http://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdb.Spec.PDBName + "/" @@ -658,7 +675,7 @@ func (r *PDBReconciler) unplugPDB(ctx context.Context, req ctrl.Request, pdb *db if err := r.Status().Update(ctx, pdb); err != nil { log.Error(err, "Failed to update status for :"+pdb.Name, "err", err.Error()) } - err = r.callAPI(ctx, req, pdb, url, values, "POST") + _, err = r.callAPI(ctx, req, pdb, url, values, "POST") if err != nil { return err } @@ -685,6 +702,87 @@ func (r *PDBReconciler) unplugPDB(ctx context.Context, req ctrl.Request, pdb *db return nil } +/************************************************* + * Modify a PDB state + /************************************************/ +func (r *PDBReconciler) modifyPDB(ctx context.Context, req ctrl.Request, pdb *dbapi.PDB) error { + + log := r.Log.WithValues("modifyPDB", req.NamespacedName) + + var err error + + cdb, err := r.getCDBResource(ctx, req, pdb) + if err != nil { + return err + } + + values := map[string]string{ + "state": pdb.Spec.PDBState, + "modifyOption": pdb.Spec.ModifyOption, + "getScript": strconv.FormatBool(*(pdb.Spec.GetScript))} + + pdbName := pdb.Spec.PDBName + url := "http://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdbName + "/status" + + pdb.Status.Phase = pdbPhaseModify + pdb.Status.ModifyOption = pdb.Spec.PDBState + "-" + pdb.Spec.ModifyOption + pdb.Status.Msg = "Waiting for PDB to be modified" + if err := r.Status().Update(ctx, pdb); err != nil { + log.Error(err, "Failed to update status for :"+pdb.Name, "err", err.Error()) + } + _, err = r.callAPI(ctx, req, pdb, url, values, "POST") + if err != nil { + return err + } + + r.Recorder.Eventf(pdb, corev1.EventTypeNormal, "Modified", "PDB '%s' modified successfully", pdb.Spec.PDBName) + pdb.Status.ConnString = cdb.Spec.SCANName + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName + + log.Info("Successfully modified PDB state", "PDB Name", pdb.Spec.PDBName) + r.getPDBState(ctx, req, pdb) + return nil +} + +/************************************************* + * Get PDB State + /************************************************/ +func (r *PDBReconciler) getPDBState(ctx context.Context, req ctrl.Request, pdb *dbapi.PDB) error { + + log := r.Log.WithValues("getPDBState", req.NamespacedName) + + var err error + + cdb, err := r.getCDBResource(ctx, req, pdb) + if err != nil { + return err + } + + pdbName := pdb.Spec.PDBName + url := "http://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdbName + "/status" + + pdb.Status.Msg = "Getting PDB state" + if err := r.Status().Update(ctx, pdb); err != nil { + log.Error(err, "Failed to update status for :"+pdb.Name, "err", err.Error()) + } + + respData, err := r.callAPI(ctx, req, pdb, url, nil, "GET") + + if err != nil { + pdb.Status.OpenMode = "UNKNOWN" + return err + } + + var objmap map[string]interface{} + if err := json.Unmarshal([]byte(respData), &objmap); err != nil { + log.Error(err, "Failed to get state of PDB :"+pdbName, "err", err.Error()) + } + + pdb.Status.OpenMode = objmap["open_mode"].(string) + + log.Info("Successfully obtained PDB state", "PDB Name", pdb.Spec.PDBName) + return nil +} + /************************************************* * Delete a PDB /************************************************/ @@ -791,7 +889,7 @@ func (r *PDBReconciler) deletePDBInstance(req ctrl.Request, ctx context.Context, if err := r.Status().Update(ctx, pdb); err != nil { log.Error(err, "Failed to update status for :"+pdb.Name, "err", err.Error()) } - err = r.callAPI(ctx, req, pdb, url, values, "DELETE") + _, err = r.callAPI(ctx, req, pdb, url, values, "DELETE") if err != nil { return err } From 9f61eb7970158fdba866f79ac1c0d11a9dad360c Mon Sep 17 00:00:00 2001 From: Pablo Silberkasten <47338417+psilberk@users.noreply.github.com> Date: Tue, 14 Dec 2021 17:57:57 -0800 Subject: [PATCH 082/628] Update kustomization.yaml --- config/samples/kustomization.yaml | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/config/samples/kustomization.yaml b/config/samples/kustomization.yaml index 4b419923..be6907a3 100644 --- a/config/samples/kustomization.yaml +++ b/config/samples/kustomization.yaml @@ -5,8 +5,7 @@ ## Append samples you want in your CSV to this file as resources ## resources: - - sharding_v1alpha1_provshard.yaml - - autonomousdatabase.yaml - - singleinstancedatabase.yaml - - shardingdatabase.yaml + - adb/autonomousdatabase.yaml + - sidb/singleinstancedatabase.yaml + - sharding/shardingdatabase.yaml # +kubebuilder:scaffold:manifestskustomizesamples From b5d859e5063258ed0b819ae66332cbbd07ae729c Mon Sep 17 00:00:00 2001 From: Pablo Silberkasten <47338417+psilberk@users.noreply.github.com> Date: Tue, 14 Dec 2021 17:57:57 -0800 Subject: [PATCH 083/628] Update kustomization.yaml --- config/samples/kustomization.yaml | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/config/samples/kustomization.yaml b/config/samples/kustomization.yaml index 4b419923..be6907a3 100644 --- a/config/samples/kustomization.yaml +++ b/config/samples/kustomization.yaml @@ -5,8 +5,7 @@ ## Append samples you want in your CSV to this file as resources ## resources: - - sharding_v1alpha1_provshard.yaml - - autonomousdatabase.yaml - - singleinstancedatabase.yaml - - shardingdatabase.yaml + - adb/autonomousdatabase.yaml + - sidb/singleinstancedatabase.yaml + - sharding/shardingdatabase.yaml # +kubebuilder:scaffold:manifestskustomizesamples From 5f0be3daef2bcd908bd7b920a88f7f9717c558ce Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Thu, 16 Dec 2021 14:39:25 -0500 Subject: [PATCH 084/628] assert the local resource is AVAILABLE --- ...autonomousdatabase_controller_bind_test.go | 71 +--------- ...tonomousdatabase_controller_create_test.go | 4 +- test/e2e/behavior/shared_behaviors.go | 133 +++++++++--------- test/e2e/suite_test.go | 18 ++- 4 files changed, 88 insertions(+), 138 deletions(-) diff --git a/test/e2e/autonomousdatabase_controller_bind_test.go b/test/e2e/autonomousdatabase_controller_bind_test.go index f10d0685..fe84cfb3 100644 --- a/test/e2e/autonomousdatabase_controller_bind_test.go +++ b/test/e2e/autonomousdatabase_controller_bind_test.go @@ -132,7 +132,7 @@ var _ = Describe("test ADB binding with hardLink=true", func() { It("Should download an instance wallet using the password from K8s Secret "+SharedWalletPassSecretName, e2ebehavior.AssertWallet(&k8sClient, &adbLookupKey)) - It("should update ADB", e2ebehavior.AssertUpdate(&k8sClient, &dbClient, &adbLookupKey)) + It("should update ADB", e2ebehavior.UpdateAndAssertDetails(&k8sClient, &dbClient, &adbLookupKey)) It("Should stop ADB", e2ebehavior.UpdateAndAssertState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateStopped)) @@ -179,7 +179,7 @@ var _ = Describe("test ADB binding with hardLink=true", func() { It("Should download an instance wallet using the password from OCI Secret OCID "+SharedInstanceWalletPasswordOCID, e2ebehavior.AssertWallet(&k8sClient, &adbLookupKey)) - It("should update ADB", e2ebehavior.AssertUpdate(&k8sClient, &dbClient, &adbLookupKey)) + It("should update ADB", e2ebehavior.UpdateAndAssertDetails(&k8sClient, &dbClient, &adbLookupKey)) It("Should stop ADB", e2ebehavior.UpdateAndAssertState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateStopped)) @@ -225,71 +225,4 @@ var _ = Describe("test ADB binding with hardLink=true", func() { It("Should delete local resource", e2ebehavior.AssertSoftLinkDelete(&k8sClient, &adbLookupKey)) }) - - // Describe("Test ADB status", func() { - // var dbName string - // var backupID string - - // It("Should init the test", func() { - // By("creating a temp ADB in OCI for binding test") - // dbName = e2eutil.GenerateDBName() - // createResp, err := e2eutil.CreateAutonomousDatabase(dbClient, &SharedCompartmentOCID, &dbName, &SharedPlainTextAdminPassword) - // Expect(err).ShouldNot(HaveOccurred()) - // Expect(createResp.AutonomousDatabase.Id).ShouldNot(BeNil()) - - // By("Save the database ID for later use") - // adbID = createResp.AutonomousDatabase.Id - // backupID = *adbID - - // By("Wait until the work request is in SUCCEEDED status") - // workClient, err := workrequests.NewWorkRequestClientWithConfigurationProvider(configProvider) - // Expect(err).ShouldNot(HaveOccurred()) - - // err = e2eutil.WaitUntilWorkCompleted(workClient, createResp.OpcWorkRequestId) - // Expect(err).ShouldNot(HaveOccurred()) - // }) - - // It("Should create a AutonomousDatabase resource", func() { - // adb := &dbv1alpha1.AutonomousDatabase{ - // TypeMeta: metav1.TypeMeta{ - // APIVersion: "database.oracle.com/v1alpha1", - // Kind: "AutonomousDatabase", - // }, - // ObjectMeta: metav1.ObjectMeta{ - // Name: "bindadb", - // Namespace: ADBNamespace, - // }, - // Spec: dbv1alpha1.AutonomousDatabaseSpec{ - // Details: dbv1alpha1.AutonomousDatabaseDetails{ - // AutonomousDatabaseOCID: adbID, - // }, - // HardLink: common.Bool(true), - // OCIConfig: dbv1alpha1.OCIConfigSpec{ - // ConfigMapName: common.String(SharedOCIConfigMapName), - // SecretName: common.String(SharedOCISecretName), - // }, - // }, - // } - - // adbLookupKey = types.NamespacedName{Name: adb.Name, Namespace: adb.Namespace} - - // Expect(k8sClient.Create(context.TODO(), adb)).Should(Succeed()) - // }) - - // It("should bind to an ADB", e2ebehavior.AssertBind(&k8sClient, &adbLookupKey)) - - // It("should terminate ADB in a different routine", func() { - // err := e2eutil.DeleteAutonomousDatabase(dbClient, adbID) - // Expect(err).ToNot(HaveOccurred()) - // // By("Wait until the work request is in SUCCEEDED status") - // // workClient, err := workrequests.NewWorkRequestClientWithConfigurationProvider(configProvider) - // // Expect(err).ShouldNot(HaveOccurred()) - // // err = e2eutil.WaitUntilWorkCompleted(workClient, createResp.OpcWorkRequestId) - // // Expect(err).ShouldNot(HaveOccurred()) - // }) - - // It("should check for terminated state in OCI", e2ebehavior.AssertRemoteStateOCID(&k8sClient, &dbClient, &backupID, database.AutonomousDatabaseLifecycleStateTerminated)) - - // It("should check for terminated state in local resource", e2ebehavior.AssertLocalState(&k8sClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateTerminated)) - // }) }) diff --git a/test/e2e/autonomousdatabase_controller_create_test.go b/test/e2e/autonomousdatabase_controller_create_test.go index 680bd9f4..ac3a86df 100644 --- a/test/e2e/autonomousdatabase_controller_create_test.go +++ b/test/e2e/autonomousdatabase_controller_create_test.go @@ -153,7 +153,7 @@ var _ = Describe("test ADB provisioning", func() { It("Should download an instance wallet using the password from K8s Secret "+SharedWalletPassSecretName, e2ebehavior.AssertWallet(&k8sClient, &adbLookupKey)) - It("should update ADB", e2ebehavior.AssertUpdate(&k8sClient, &dbClient, &adbLookupKey)) + It("should update ADB", e2ebehavior.UpdateAndAssertDetails(&k8sClient, &dbClient, &adbLookupKey)) It("Should stop ADB", e2ebehavior.UpdateAndAssertState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateStopped)) @@ -214,7 +214,7 @@ var _ = Describe("test ADB provisioning", func() { It("Should download an instance wallet using the password from OCI Secret OCID "+SharedInstanceWalletPasswordOCID, e2ebehavior.AssertWallet(&k8sClient, &adbLookupKey)) - It("should update ADB", e2ebehavior.AssertUpdate(&k8sClient, &dbClient, &adbLookupKey)) + It("should update ADB", e2ebehavior.UpdateAndAssertDetails(&k8sClient, &dbClient, &adbLookupKey)) It("Should stop ADB", e2ebehavior.UpdateAndAssertState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateStopped)) diff --git a/test/e2e/behavior/shared_behaviors.go b/test/e2e/behavior/shared_behaviors.go index 38501d38..2aba3ad4 100644 --- a/test/e2e/behavior/shared_behaviors.go +++ b/test/e2e/behavior/shared_behaviors.go @@ -173,30 +173,6 @@ func AssertWallet(k8sClient *client.Client, adbLookupKey *types.NamespacedName) } } -func CleanupDB(k8sClient *client.Client, dbClient *database.DatabaseClient, namespace string) func() { - return func() { - - Expect(k8sClient).NotTo(BeNil()) - Expect(dbClient).NotTo(BeNil()) - - derefK8sClient := *k8sClient - derefDBClient := *dbClient - - adbList := &dbv1alpha1.AutonomousDatabaseList{} - options := &client.ListOptions{ - Namespace: namespace, - } - derefK8sClient.List(context.TODO(), adbList, options) - - for _, adb := range adbList.Items { - if adb.Spec.Details.AutonomousDatabaseOCID != nil { - By("Terminating database " + *adb.Spec.Details.DbName) - Expect(e2eutil.DeleteAutonomousDatabase(derefDBClient, adb.Spec.Details.AutonomousDatabaseOCID)).Should(Succeed()) - } - } - } -} - func compartInt(obj1 *int, obj2 *int) bool { if obj1 == nil && obj2 == nil { return true @@ -242,9 +218,9 @@ func compartStringMap(obj1 map[string]string, obj2 map[string]string) bool { return true } -// AssertUpdate changes the displayName from "foo" to "foo_new", and scale the cpuCoreCount to 2 -func AssertUpdate(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName) func() { - return func() { +// UpdateDetails updates spec.details from local resource and OCI +func UpdateDetails(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName) func() *dbv1alpha1.AutonomousDatabase { + return func() *dbv1alpha1.AutonomousDatabase { // Considering that there are at most two update requests will be sent during the update // From the observation per request takes ~3mins to finish updateTimeout := time.Minute * 7 @@ -279,7 +255,13 @@ func AssertUpdate(k8sClient *client.Client, dbClient *database.DatabaseClient, a // Update var newDisplayName = *expectedADB.Spec.Details.DisplayName + "_new" - var newCPUCoreCount = 2 + + var newCPUCoreCount int + if *expectedADB.Spec.Details.CPUCoreCount == 1 { + newCPUCoreCount = 2 + } else { + newCPUCoreCount = 1 + } By(fmt.Sprintf("Updating the ADB with newDisplayName = %s and newCPUCoreCount = %d\n", newDisplayName, newCPUCoreCount)) @@ -288,52 +270,71 @@ func AssertUpdate(k8sClient *client.Client, dbClient *database.DatabaseClient, a Expect(derefK8sClient.Update(context.TODO(), expectedADB)).To(Succeed()) - Eventually(func() (bool, error) { - // Get the current ADB resource and retry it's not in AVAILABLE state - currentADB := &dbv1alpha1.AutonomousDatabase{} - derefK8sClient.Get(context.TODO(), *adbLookupKey, currentADB) - if currentADB.Spec.Details.LifecycleState != database.AutonomousDatabaseLifecycleStateAvailable { - return false, nil - } + return expectedADB + } +} + +// AssertADBDetails asserts the changes in spec.details +func AssertADBDetails(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName, expectedADB *dbv1alpha1.AutonomousDatabase) func() { + return func() { + // Considering that there are at most two update requests will be sent during the update + // From the observation per request takes ~3mins to finish + updateTimeout := time.Minute * 7 + updateInterval := time.Second * 20 + + Expect(k8sClient).NotTo(BeNil()) + Expect(dbClient).NotTo(BeNil()) + Expect(adbLookupKey).NotTo(BeNil()) + derefDBClient := *dbClient + + Eventually(func() (bool, error) { // Fetch the ADB from OCI when it's in AVAILABLE state, and retry if its attributes doesn't match the new ADB's attributes retryPolicy := e2eutil.NewLifecycleStateRetryPolicy(database.AutonomousDatabaseLifecycleStateAvailable) - resp, err := e2eutil.GetAutonomousDatabase(derefDBClient, currentADB.Spec.Details.AutonomousDatabaseOCID, &retryPolicy) + resp, err := e2eutil.GetAutonomousDatabase(derefDBClient, expectedADB.Spec.Details.AutonomousDatabaseOCID, &retryPolicy) if err != nil { return false, err } - adbDetails := currentADB.Spec.Details - - ociADB := currentADB - ociADB = currentADB.UpdateAttrFromOCIAutonomousDatabase(resp.AutonomousDatabase) - ociADBDetails := ociADB.Spec.Details - - // Compare - same := compartString(adbDetails.AutonomousDatabaseOCID, ociADBDetails.AutonomousDatabaseOCID) && - compartString(adbDetails.CompartmentOCID, ociADBDetails.CompartmentOCID) && - compartString(adbDetails.DisplayName, ociADBDetails.DisplayName) && - compartString(adbDetails.DbName, ociADBDetails.DbName) && - adbDetails.DbWorkload == ociADBDetails.DbWorkload && - compartBool(adbDetails.IsDedicated, ociADBDetails.IsDedicated) && - compartString(adbDetails.DbVersion, ociADBDetails.DbVersion) && - compartInt(adbDetails.DataStorageSizeInTBs, ociADBDetails.DataStorageSizeInTBs) && - compartInt(adbDetails.CPUCoreCount, ociADBDetails.CPUCoreCount) && - compartBool(adbDetails.IsAutoScalingEnabled, ociADBDetails.IsAutoScalingEnabled) && - adbDetails.LifecycleState == ociADBDetails.LifecycleState && - compartStringMap(adbDetails.FreeformTags, ociADBDetails.FreeformTags) && - compartString(adbDetails.SubnetOCID, ociADBDetails.SubnetOCID) && - reflect.DeepEqual(adbDetails.NsgOCIDs, ociADBDetails.NsgOCIDs) && - compartString(adbDetails.PrivateEndpoint, ociADBDetails.PrivateEndpoint) && - compartString(adbDetails.PrivateEndpointLabel, ociADBDetails.PrivateEndpointLabel) && - compartString(adbDetails.PrivateEndpointIP, ociADBDetails.PrivateEndpointIP) + expectedADBDetails := expectedADB.Spec.Details + + // Compare each elements. Reflect.DeepEqual isn't used here because some parameters (e.g. adminPassword) + // may not match. + // We don't compare LifecycleState in this case. We only make sure that the ADB is in AVAIABLE state before + // proceeding to the next test. + same := compartString(expectedADBDetails.AutonomousDatabaseOCID, resp.AutonomousDatabase.Id) && + compartString(expectedADBDetails.CompartmentOCID, resp.AutonomousDatabase.CompartmentId) && + compartString(expectedADBDetails.DisplayName, resp.AutonomousDatabase.DisplayName) && + compartString(expectedADBDetails.DbName, resp.AutonomousDatabase.DbName) && + expectedADBDetails.DbWorkload == resp.AutonomousDatabase.DbWorkload && + compartBool(expectedADBDetails.IsDedicated, resp.AutonomousDatabase.IsDedicated) && + compartString(expectedADBDetails.DbVersion, resp.AutonomousDatabase.DbVersion) && + compartInt(expectedADBDetails.DataStorageSizeInTBs, resp.AutonomousDatabase.DataStorageSizeInTBs) && + compartInt(expectedADBDetails.CPUCoreCount, resp.AutonomousDatabase.CpuCoreCount) && + compartBool(expectedADBDetails.IsAutoScalingEnabled, resp.AutonomousDatabase.IsAutoScalingEnabled) && + compartStringMap(expectedADBDetails.FreeformTags, resp.AutonomousDatabase.FreeformTags) && + compartString(expectedADBDetails.SubnetOCID, resp.AutonomousDatabase.SubnetId) && + reflect.DeepEqual(expectedADBDetails.NsgOCIDs, resp.AutonomousDatabase.NsgIds) && + compartString(expectedADBDetails.PrivateEndpointLabel, resp.AutonomousDatabase.PrivateEndpointLabel) return same, nil }, updateTimeout, updateInterval).Should(BeTrue()) + + // IMPORTANT: make sure the local resource has finished reconciling, otherwise the changes will + // be conflicted with the next test and cause unknow result. + AssertLocalState(k8sClient, adbLookupKey, database.AutonomousDatabaseLifecycleStateAvailable)() + } +} + +// UpdateAndAssertDetails changes the displayName from "foo" to "foo_new", and scale the cpuCoreCount to 2 +func UpdateAndAssertDetails(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName) func() { + return func() { + expectedADB := UpdateDetails(k8sClient, dbClient, adbLookupKey)() + AssertADBDetails(k8sClient, dbClient, adbLookupKey, expectedADB)() } } -// Updates adb state and then asserts if change is propagated to OCI +// UpdateAndAssertState updates adb state and then asserts if change is propagated to OCI func UpdateAndAssertState(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName, state database.AutonomousDatabaseLifecycleStateEnum) func() { return func() { UpdateState(k8sClient, adbLookupKey, state)() @@ -341,7 +342,7 @@ func UpdateAndAssertState(k8sClient *client.Client, dbClient *database.DatabaseC } } -// Assert local and remote state +// AssertState asserts local and remote state func AssertState(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName, state database.AutonomousDatabaseLifecycleStateEnum) func() { return func() { // Waits longer for the local resource to reach the desired state @@ -352,6 +353,7 @@ func AssertState(k8sClient *client.Client, dbClient *database.DatabaseClient, ad } } +// AssertHardLinkDelete asserts the database is terminated in OCI when hardLink is set to true func AssertHardLinkDelete(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName) func() { return func() { changeStateTimeout := time.Second * 300 @@ -378,6 +380,7 @@ func AssertHardLinkDelete(k8sClient *client.Client, dbClient *database.DatabaseC } } +// AssertSoftLinkDelete asserts the database remains in OCI when hardLink is set to false func AssertSoftLinkDelete(k8sClient *client.Client, adbLookupKey *types.NamespacedName) func() { return func() { changeStateTimeout := time.Second * 300 @@ -406,6 +409,7 @@ func AssertSoftLinkDelete(k8sClient *client.Client, adbLookupKey *types.Namespac } } +// AssertLocalState asserts the lifecycle state of the local resource using adbLookupKey func AssertLocalState(k8sClient *client.Client, adbLookupKey *types.NamespacedName, state database.AutonomousDatabaseLifecycleStateEnum) func() { return func() { changeLocalStateTimeout := time.Second * 600 @@ -422,6 +426,7 @@ func AssertLocalState(k8sClient *client.Client, adbLookupKey *types.NamespacedNa } } +// AssertRemoteState asserts the lifecycle state in OCI using adbLookupKey func AssertRemoteState(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName, state database.AutonomousDatabaseLifecycleStateEnum) func() { return func() { @@ -438,7 +443,7 @@ func AssertRemoteState(k8sClient *client.Client, dbClient *database.DatabaseClie } } -// Assert remote state using adb OCID +// AssertRemoteStateOCID asserts the lifecycle state in OCI using autonomousDatabaseOCID func AssertRemoteStateOCID(k8sClient *client.Client, dbClient *database.DatabaseClient, adbID *string, state database.AutonomousDatabaseLifecycleStateEnum) func() { return func() { changeRemoteStateTimeout := time.Second * 300 @@ -460,7 +465,7 @@ func AssertRemoteStateOCID(k8sClient *client.Client, dbClient *database.Database } } -// Updates state from local resource and OCI +// UpdateState updates state from local resource and OCI func UpdateState(k8sClient *client.Client, adbLookupKey *types.NamespacedName, state database.AutonomousDatabaseLifecycleStateEnum) func() { return func() { Expect(k8sClient).NotTo(BeNil()) diff --git a/test/e2e/suite_test.go b/test/e2e/suite_test.go index 2ec0d361..113fbbe0 100644 --- a/test/e2e/suite_test.go +++ b/test/e2e/suite_test.go @@ -40,6 +40,7 @@ package e2etest import ( "context" + "fmt" "path/filepath" "testing" "time" @@ -61,7 +62,6 @@ import ( databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" controllers "github.com/oracle/oracle-database-operator/controllers/database" - "github.com/oracle/oracle-database-operator/test/e2e/behavior" "github.com/oracle/oracle-database-operator/test/e2e/util" // +kubebuilder:scaffold:imports ) @@ -246,6 +246,18 @@ var _ = AfterSuite(func() { Expect(err).ToNot(HaveOccurred()) */ - By("Deleting the resources that are created during the tests") - e2ebehavior.CleanupDB(&k8sClient, &dbClient, ADBNamespace) + By("Delete the resources that are created during the tests") + adbList := &databasev1alpha1.AutonomousDatabaseList{} + options := &client.ListOptions{ + Namespace: ADBNamespace, + } + k8sClient.List(context.TODO(), adbList, options) + By(fmt.Sprintf("Found %d AutonomousDatabase(s)", len(adbList.Items))) + + for _, adb := range adbList.Items { + if adb.Spec.Details.AutonomousDatabaseOCID != nil { + By("Terminating database " + *adb.Spec.Details.DbName) + Expect(e2eutil.DeleteAutonomousDatabase(dbClient, adb.Spec.Details.AutonomousDatabaseOCID)).Should(Succeed()) + } + } }) From 4409a6976601b053d6ea947f53a9d23e13155ad7 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Thu, 16 Dec 2021 14:39:25 -0500 Subject: [PATCH 085/628] assert the local resource is AVAILABLE --- ...autonomousdatabase_controller_bind_test.go | 71 +--------- ...tonomousdatabase_controller_create_test.go | 4 +- test/e2e/behavior/shared_behaviors.go | 133 +++++++++--------- test/e2e/suite_test.go | 18 ++- 4 files changed, 88 insertions(+), 138 deletions(-) diff --git a/test/e2e/autonomousdatabase_controller_bind_test.go b/test/e2e/autonomousdatabase_controller_bind_test.go index f10d0685..fe84cfb3 100644 --- a/test/e2e/autonomousdatabase_controller_bind_test.go +++ b/test/e2e/autonomousdatabase_controller_bind_test.go @@ -132,7 +132,7 @@ var _ = Describe("test ADB binding with hardLink=true", func() { It("Should download an instance wallet using the password from K8s Secret "+SharedWalletPassSecretName, e2ebehavior.AssertWallet(&k8sClient, &adbLookupKey)) - It("should update ADB", e2ebehavior.AssertUpdate(&k8sClient, &dbClient, &adbLookupKey)) + It("should update ADB", e2ebehavior.UpdateAndAssertDetails(&k8sClient, &dbClient, &adbLookupKey)) It("Should stop ADB", e2ebehavior.UpdateAndAssertState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateStopped)) @@ -179,7 +179,7 @@ var _ = Describe("test ADB binding with hardLink=true", func() { It("Should download an instance wallet using the password from OCI Secret OCID "+SharedInstanceWalletPasswordOCID, e2ebehavior.AssertWallet(&k8sClient, &adbLookupKey)) - It("should update ADB", e2ebehavior.AssertUpdate(&k8sClient, &dbClient, &adbLookupKey)) + It("should update ADB", e2ebehavior.UpdateAndAssertDetails(&k8sClient, &dbClient, &adbLookupKey)) It("Should stop ADB", e2ebehavior.UpdateAndAssertState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateStopped)) @@ -225,71 +225,4 @@ var _ = Describe("test ADB binding with hardLink=true", func() { It("Should delete local resource", e2ebehavior.AssertSoftLinkDelete(&k8sClient, &adbLookupKey)) }) - - // Describe("Test ADB status", func() { - // var dbName string - // var backupID string - - // It("Should init the test", func() { - // By("creating a temp ADB in OCI for binding test") - // dbName = e2eutil.GenerateDBName() - // createResp, err := e2eutil.CreateAutonomousDatabase(dbClient, &SharedCompartmentOCID, &dbName, &SharedPlainTextAdminPassword) - // Expect(err).ShouldNot(HaveOccurred()) - // Expect(createResp.AutonomousDatabase.Id).ShouldNot(BeNil()) - - // By("Save the database ID for later use") - // adbID = createResp.AutonomousDatabase.Id - // backupID = *adbID - - // By("Wait until the work request is in SUCCEEDED status") - // workClient, err := workrequests.NewWorkRequestClientWithConfigurationProvider(configProvider) - // Expect(err).ShouldNot(HaveOccurred()) - - // err = e2eutil.WaitUntilWorkCompleted(workClient, createResp.OpcWorkRequestId) - // Expect(err).ShouldNot(HaveOccurred()) - // }) - - // It("Should create a AutonomousDatabase resource", func() { - // adb := &dbv1alpha1.AutonomousDatabase{ - // TypeMeta: metav1.TypeMeta{ - // APIVersion: "database.oracle.com/v1alpha1", - // Kind: "AutonomousDatabase", - // }, - // ObjectMeta: metav1.ObjectMeta{ - // Name: "bindadb", - // Namespace: ADBNamespace, - // }, - // Spec: dbv1alpha1.AutonomousDatabaseSpec{ - // Details: dbv1alpha1.AutonomousDatabaseDetails{ - // AutonomousDatabaseOCID: adbID, - // }, - // HardLink: common.Bool(true), - // OCIConfig: dbv1alpha1.OCIConfigSpec{ - // ConfigMapName: common.String(SharedOCIConfigMapName), - // SecretName: common.String(SharedOCISecretName), - // }, - // }, - // } - - // adbLookupKey = types.NamespacedName{Name: adb.Name, Namespace: adb.Namespace} - - // Expect(k8sClient.Create(context.TODO(), adb)).Should(Succeed()) - // }) - - // It("should bind to an ADB", e2ebehavior.AssertBind(&k8sClient, &adbLookupKey)) - - // It("should terminate ADB in a different routine", func() { - // err := e2eutil.DeleteAutonomousDatabase(dbClient, adbID) - // Expect(err).ToNot(HaveOccurred()) - // // By("Wait until the work request is in SUCCEEDED status") - // // workClient, err := workrequests.NewWorkRequestClientWithConfigurationProvider(configProvider) - // // Expect(err).ShouldNot(HaveOccurred()) - // // err = e2eutil.WaitUntilWorkCompleted(workClient, createResp.OpcWorkRequestId) - // // Expect(err).ShouldNot(HaveOccurred()) - // }) - - // It("should check for terminated state in OCI", e2ebehavior.AssertRemoteStateOCID(&k8sClient, &dbClient, &backupID, database.AutonomousDatabaseLifecycleStateTerminated)) - - // It("should check for terminated state in local resource", e2ebehavior.AssertLocalState(&k8sClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateTerminated)) - // }) }) diff --git a/test/e2e/autonomousdatabase_controller_create_test.go b/test/e2e/autonomousdatabase_controller_create_test.go index 680bd9f4..ac3a86df 100644 --- a/test/e2e/autonomousdatabase_controller_create_test.go +++ b/test/e2e/autonomousdatabase_controller_create_test.go @@ -153,7 +153,7 @@ var _ = Describe("test ADB provisioning", func() { It("Should download an instance wallet using the password from K8s Secret "+SharedWalletPassSecretName, e2ebehavior.AssertWallet(&k8sClient, &adbLookupKey)) - It("should update ADB", e2ebehavior.AssertUpdate(&k8sClient, &dbClient, &adbLookupKey)) + It("should update ADB", e2ebehavior.UpdateAndAssertDetails(&k8sClient, &dbClient, &adbLookupKey)) It("Should stop ADB", e2ebehavior.UpdateAndAssertState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateStopped)) @@ -214,7 +214,7 @@ var _ = Describe("test ADB provisioning", func() { It("Should download an instance wallet using the password from OCI Secret OCID "+SharedInstanceWalletPasswordOCID, e2ebehavior.AssertWallet(&k8sClient, &adbLookupKey)) - It("should update ADB", e2ebehavior.AssertUpdate(&k8sClient, &dbClient, &adbLookupKey)) + It("should update ADB", e2ebehavior.UpdateAndAssertDetails(&k8sClient, &dbClient, &adbLookupKey)) It("Should stop ADB", e2ebehavior.UpdateAndAssertState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateStopped)) diff --git a/test/e2e/behavior/shared_behaviors.go b/test/e2e/behavior/shared_behaviors.go index 38501d38..2aba3ad4 100644 --- a/test/e2e/behavior/shared_behaviors.go +++ b/test/e2e/behavior/shared_behaviors.go @@ -173,30 +173,6 @@ func AssertWallet(k8sClient *client.Client, adbLookupKey *types.NamespacedName) } } -func CleanupDB(k8sClient *client.Client, dbClient *database.DatabaseClient, namespace string) func() { - return func() { - - Expect(k8sClient).NotTo(BeNil()) - Expect(dbClient).NotTo(BeNil()) - - derefK8sClient := *k8sClient - derefDBClient := *dbClient - - adbList := &dbv1alpha1.AutonomousDatabaseList{} - options := &client.ListOptions{ - Namespace: namespace, - } - derefK8sClient.List(context.TODO(), adbList, options) - - for _, adb := range adbList.Items { - if adb.Spec.Details.AutonomousDatabaseOCID != nil { - By("Terminating database " + *adb.Spec.Details.DbName) - Expect(e2eutil.DeleteAutonomousDatabase(derefDBClient, adb.Spec.Details.AutonomousDatabaseOCID)).Should(Succeed()) - } - } - } -} - func compartInt(obj1 *int, obj2 *int) bool { if obj1 == nil && obj2 == nil { return true @@ -242,9 +218,9 @@ func compartStringMap(obj1 map[string]string, obj2 map[string]string) bool { return true } -// AssertUpdate changes the displayName from "foo" to "foo_new", and scale the cpuCoreCount to 2 -func AssertUpdate(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName) func() { - return func() { +// UpdateDetails updates spec.details from local resource and OCI +func UpdateDetails(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName) func() *dbv1alpha1.AutonomousDatabase { + return func() *dbv1alpha1.AutonomousDatabase { // Considering that there are at most two update requests will be sent during the update // From the observation per request takes ~3mins to finish updateTimeout := time.Minute * 7 @@ -279,7 +255,13 @@ func AssertUpdate(k8sClient *client.Client, dbClient *database.DatabaseClient, a // Update var newDisplayName = *expectedADB.Spec.Details.DisplayName + "_new" - var newCPUCoreCount = 2 + + var newCPUCoreCount int + if *expectedADB.Spec.Details.CPUCoreCount == 1 { + newCPUCoreCount = 2 + } else { + newCPUCoreCount = 1 + } By(fmt.Sprintf("Updating the ADB with newDisplayName = %s and newCPUCoreCount = %d\n", newDisplayName, newCPUCoreCount)) @@ -288,52 +270,71 @@ func AssertUpdate(k8sClient *client.Client, dbClient *database.DatabaseClient, a Expect(derefK8sClient.Update(context.TODO(), expectedADB)).To(Succeed()) - Eventually(func() (bool, error) { - // Get the current ADB resource and retry it's not in AVAILABLE state - currentADB := &dbv1alpha1.AutonomousDatabase{} - derefK8sClient.Get(context.TODO(), *adbLookupKey, currentADB) - if currentADB.Spec.Details.LifecycleState != database.AutonomousDatabaseLifecycleStateAvailable { - return false, nil - } + return expectedADB + } +} + +// AssertADBDetails asserts the changes in spec.details +func AssertADBDetails(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName, expectedADB *dbv1alpha1.AutonomousDatabase) func() { + return func() { + // Considering that there are at most two update requests will be sent during the update + // From the observation per request takes ~3mins to finish + updateTimeout := time.Minute * 7 + updateInterval := time.Second * 20 + + Expect(k8sClient).NotTo(BeNil()) + Expect(dbClient).NotTo(BeNil()) + Expect(adbLookupKey).NotTo(BeNil()) + derefDBClient := *dbClient + + Eventually(func() (bool, error) { // Fetch the ADB from OCI when it's in AVAILABLE state, and retry if its attributes doesn't match the new ADB's attributes retryPolicy := e2eutil.NewLifecycleStateRetryPolicy(database.AutonomousDatabaseLifecycleStateAvailable) - resp, err := e2eutil.GetAutonomousDatabase(derefDBClient, currentADB.Spec.Details.AutonomousDatabaseOCID, &retryPolicy) + resp, err := e2eutil.GetAutonomousDatabase(derefDBClient, expectedADB.Spec.Details.AutonomousDatabaseOCID, &retryPolicy) if err != nil { return false, err } - adbDetails := currentADB.Spec.Details - - ociADB := currentADB - ociADB = currentADB.UpdateAttrFromOCIAutonomousDatabase(resp.AutonomousDatabase) - ociADBDetails := ociADB.Spec.Details - - // Compare - same := compartString(adbDetails.AutonomousDatabaseOCID, ociADBDetails.AutonomousDatabaseOCID) && - compartString(adbDetails.CompartmentOCID, ociADBDetails.CompartmentOCID) && - compartString(adbDetails.DisplayName, ociADBDetails.DisplayName) && - compartString(adbDetails.DbName, ociADBDetails.DbName) && - adbDetails.DbWorkload == ociADBDetails.DbWorkload && - compartBool(adbDetails.IsDedicated, ociADBDetails.IsDedicated) && - compartString(adbDetails.DbVersion, ociADBDetails.DbVersion) && - compartInt(adbDetails.DataStorageSizeInTBs, ociADBDetails.DataStorageSizeInTBs) && - compartInt(adbDetails.CPUCoreCount, ociADBDetails.CPUCoreCount) && - compartBool(adbDetails.IsAutoScalingEnabled, ociADBDetails.IsAutoScalingEnabled) && - adbDetails.LifecycleState == ociADBDetails.LifecycleState && - compartStringMap(adbDetails.FreeformTags, ociADBDetails.FreeformTags) && - compartString(adbDetails.SubnetOCID, ociADBDetails.SubnetOCID) && - reflect.DeepEqual(adbDetails.NsgOCIDs, ociADBDetails.NsgOCIDs) && - compartString(adbDetails.PrivateEndpoint, ociADBDetails.PrivateEndpoint) && - compartString(adbDetails.PrivateEndpointLabel, ociADBDetails.PrivateEndpointLabel) && - compartString(adbDetails.PrivateEndpointIP, ociADBDetails.PrivateEndpointIP) + expectedADBDetails := expectedADB.Spec.Details + + // Compare each elements. Reflect.DeepEqual isn't used here because some parameters (e.g. adminPassword) + // may not match. + // We don't compare LifecycleState in this case. We only make sure that the ADB is in AVAIABLE state before + // proceeding to the next test. + same := compartString(expectedADBDetails.AutonomousDatabaseOCID, resp.AutonomousDatabase.Id) && + compartString(expectedADBDetails.CompartmentOCID, resp.AutonomousDatabase.CompartmentId) && + compartString(expectedADBDetails.DisplayName, resp.AutonomousDatabase.DisplayName) && + compartString(expectedADBDetails.DbName, resp.AutonomousDatabase.DbName) && + expectedADBDetails.DbWorkload == resp.AutonomousDatabase.DbWorkload && + compartBool(expectedADBDetails.IsDedicated, resp.AutonomousDatabase.IsDedicated) && + compartString(expectedADBDetails.DbVersion, resp.AutonomousDatabase.DbVersion) && + compartInt(expectedADBDetails.DataStorageSizeInTBs, resp.AutonomousDatabase.DataStorageSizeInTBs) && + compartInt(expectedADBDetails.CPUCoreCount, resp.AutonomousDatabase.CpuCoreCount) && + compartBool(expectedADBDetails.IsAutoScalingEnabled, resp.AutonomousDatabase.IsAutoScalingEnabled) && + compartStringMap(expectedADBDetails.FreeformTags, resp.AutonomousDatabase.FreeformTags) && + compartString(expectedADBDetails.SubnetOCID, resp.AutonomousDatabase.SubnetId) && + reflect.DeepEqual(expectedADBDetails.NsgOCIDs, resp.AutonomousDatabase.NsgIds) && + compartString(expectedADBDetails.PrivateEndpointLabel, resp.AutonomousDatabase.PrivateEndpointLabel) return same, nil }, updateTimeout, updateInterval).Should(BeTrue()) + + // IMPORTANT: make sure the local resource has finished reconciling, otherwise the changes will + // be conflicted with the next test and cause unknow result. + AssertLocalState(k8sClient, adbLookupKey, database.AutonomousDatabaseLifecycleStateAvailable)() + } +} + +// UpdateAndAssertDetails changes the displayName from "foo" to "foo_new", and scale the cpuCoreCount to 2 +func UpdateAndAssertDetails(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName) func() { + return func() { + expectedADB := UpdateDetails(k8sClient, dbClient, adbLookupKey)() + AssertADBDetails(k8sClient, dbClient, adbLookupKey, expectedADB)() } } -// Updates adb state and then asserts if change is propagated to OCI +// UpdateAndAssertState updates adb state and then asserts if change is propagated to OCI func UpdateAndAssertState(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName, state database.AutonomousDatabaseLifecycleStateEnum) func() { return func() { UpdateState(k8sClient, adbLookupKey, state)() @@ -341,7 +342,7 @@ func UpdateAndAssertState(k8sClient *client.Client, dbClient *database.DatabaseC } } -// Assert local and remote state +// AssertState asserts local and remote state func AssertState(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName, state database.AutonomousDatabaseLifecycleStateEnum) func() { return func() { // Waits longer for the local resource to reach the desired state @@ -352,6 +353,7 @@ func AssertState(k8sClient *client.Client, dbClient *database.DatabaseClient, ad } } +// AssertHardLinkDelete asserts the database is terminated in OCI when hardLink is set to true func AssertHardLinkDelete(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName) func() { return func() { changeStateTimeout := time.Second * 300 @@ -378,6 +380,7 @@ func AssertHardLinkDelete(k8sClient *client.Client, dbClient *database.DatabaseC } } +// AssertSoftLinkDelete asserts the database remains in OCI when hardLink is set to false func AssertSoftLinkDelete(k8sClient *client.Client, adbLookupKey *types.NamespacedName) func() { return func() { changeStateTimeout := time.Second * 300 @@ -406,6 +409,7 @@ func AssertSoftLinkDelete(k8sClient *client.Client, adbLookupKey *types.Namespac } } +// AssertLocalState asserts the lifecycle state of the local resource using adbLookupKey func AssertLocalState(k8sClient *client.Client, adbLookupKey *types.NamespacedName, state database.AutonomousDatabaseLifecycleStateEnum) func() { return func() { changeLocalStateTimeout := time.Second * 600 @@ -422,6 +426,7 @@ func AssertLocalState(k8sClient *client.Client, adbLookupKey *types.NamespacedNa } } +// AssertRemoteState asserts the lifecycle state in OCI using adbLookupKey func AssertRemoteState(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName, state database.AutonomousDatabaseLifecycleStateEnum) func() { return func() { @@ -438,7 +443,7 @@ func AssertRemoteState(k8sClient *client.Client, dbClient *database.DatabaseClie } } -// Assert remote state using adb OCID +// AssertRemoteStateOCID asserts the lifecycle state in OCI using autonomousDatabaseOCID func AssertRemoteStateOCID(k8sClient *client.Client, dbClient *database.DatabaseClient, adbID *string, state database.AutonomousDatabaseLifecycleStateEnum) func() { return func() { changeRemoteStateTimeout := time.Second * 300 @@ -460,7 +465,7 @@ func AssertRemoteStateOCID(k8sClient *client.Client, dbClient *database.Database } } -// Updates state from local resource and OCI +// UpdateState updates state from local resource and OCI func UpdateState(k8sClient *client.Client, adbLookupKey *types.NamespacedName, state database.AutonomousDatabaseLifecycleStateEnum) func() { return func() { Expect(k8sClient).NotTo(BeNil()) diff --git a/test/e2e/suite_test.go b/test/e2e/suite_test.go index 2ec0d361..113fbbe0 100644 --- a/test/e2e/suite_test.go +++ b/test/e2e/suite_test.go @@ -40,6 +40,7 @@ package e2etest import ( "context" + "fmt" "path/filepath" "testing" "time" @@ -61,7 +62,6 @@ import ( databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" controllers "github.com/oracle/oracle-database-operator/controllers/database" - "github.com/oracle/oracle-database-operator/test/e2e/behavior" "github.com/oracle/oracle-database-operator/test/e2e/util" // +kubebuilder:scaffold:imports ) @@ -246,6 +246,18 @@ var _ = AfterSuite(func() { Expect(err).ToNot(HaveOccurred()) */ - By("Deleting the resources that are created during the tests") - e2ebehavior.CleanupDB(&k8sClient, &dbClient, ADBNamespace) + By("Delete the resources that are created during the tests") + adbList := &databasev1alpha1.AutonomousDatabaseList{} + options := &client.ListOptions{ + Namespace: ADBNamespace, + } + k8sClient.List(context.TODO(), adbList, options) + By(fmt.Sprintf("Found %d AutonomousDatabase(s)", len(adbList.Items))) + + for _, adb := range adbList.Items { + if adb.Spec.Details.AutonomousDatabaseOCID != nil { + By("Terminating database " + *adb.Spec.Details.DbName) + Expect(e2eutil.DeleteAutonomousDatabase(dbClient, adb.Spec.Details.AutonomousDatabaseOCID)).Should(Succeed()) + } + } }) From a9fb5ceb3b82562f596bd9ec242ea693ebbded48 Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Fri, 17 Dec 2021 11:58:18 +0530 Subject: [PATCH 086/628] Readme changes for supporting sidb on minikube --- README.md | 1 + .../sidb/singleinstancedatabase_minikube.yaml | 93 +++++++++++++++++++ .../sidb/singleinstancedatabase_prov.yaml | 7 +- docs/sidb/README.md | 13 +++ 4 files changed, 110 insertions(+), 4 deletions(-) create mode 100644 config/samples/sidb/singleinstancedatabase_minikube.yaml diff --git a/README.md b/README.md index cd72a209..029173a0 100644 --- a/README.md +++ b/README.md @@ -30,6 +30,7 @@ Upcoming releases will support new configurations, operations and capabilities. This release can be deployed on the following platforms: +* [Minikube](https://minikube.sigs.k8s.io/docs/) with version v1.21.0 or later * [Oracle Container Engine for Kubernetes (OKE)](https://www.oracle.com/cloud-native/container-engine-kubernetes/) with Kubernetes 1.17 or later * In an on-premises [Oracle Linux Cloud Native Environment(OLCNE)](https://docs.oracle.com/en/operating-systems/olcne/) 1.3 or later diff --git a/config/samples/sidb/singleinstancedatabase_minikube.yaml b/config/samples/sidb/singleinstancedatabase_minikube.yaml new file mode 100644 index 00000000..195f3ca8 --- /dev/null +++ b/config/samples/sidb/singleinstancedatabase_minikube.yaml @@ -0,0 +1,93 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +apiVersion: v1 +kind: Secret +metadata: + name: db-secret +type: Opaque +data: + oracle_pwd: "T3JhY2xlIzc=" # 'Oracle#7' base64 encoded + +--- + +apiVersion: v1 +kind: PersistentVolume +metadata: + name: pv1 +spec: + accessModes: + - ReadWriteMany + capacity: + storage: 5Gi + hostPath: + path: /data/pv1 + +--- + +apiVersion: database.oracle.com/v1alpha1 +kind: SingleInstanceDatabase +metadata: + name: sidb-sample + namespace: default +spec: + + ## Use only alphanumeric characters for sid + sid: ORCL1 + + ## A source database ref to clone from, leave empty to create a fresh database + cloneFrom: "" + + ## NA if cloning from a SourceDB (cloneFrom is set) + edition: enterprise + + ## Should refer to SourceDB secret if cloning from a SourceDB (cloneFrom is set) + ## Secret containing SIDB password mapped to secretKey + ## This secret will be deleted after creation of the database unless keepSecret is set to true + adminPassword: + secretName: db-secret + secretKey: oracle_pwd + keepSecret: false + + ## NA if cloning from a SourceDB (cloneFrom is set) + charset: AL32UTF8 + + ## NA if cloning from a SourceDB (cloneFrom is set) + pdbName: orclpdb1 + + ## Enable/Disable Flashback + flashBack: false + + ## Enable/Disable ArchiveLog + archiveLog: false + + ## Enable/Disable ForceLogging + forceLog: false + + ## NA if cloning from a SourceDB (cloneFrom is set) + ## Specify both sgaSize and pgaSize (in MB) or dont specify both + ## Specify Non-Zero value to use + initParams: + cpuCount: 0 + processes: 0 + sgaTarget: 0 + pgaAggregateTarget: 0 + + ## Database image details + ## Database can be patched by updating the RU version/image + ## Major version changes are not supported + image: + pullFrom: + pullSecrets: + + ## size : Minimum size of pvc | class : PVC storage Class + ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany + persistence: + size: 4Gi + storageClass: "" + accessMode: "ReadWriteMany" + + ## Count of Database Pods. Applicable only for "ReadWriteMany" AccessMode + replicas: 1 diff --git a/config/samples/sidb/singleinstancedatabase_prov.yaml b/config/samples/sidb/singleinstancedatabase_prov.yaml index 7123b117..a1f06471 100644 --- a/config/samples/sidb/singleinstancedatabase_prov.yaml +++ b/config/samples/sidb/singleinstancedatabase_prov.yaml @@ -14,13 +14,12 @@ spec: ## Secret containing SIDB password mapped to secretKey adminPassword: - secret: - secretName: - key: + secretName: + secretKey: + keepSecret: false ## Database image details image: - version: pullFrom: pullSecrets: diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 168a2395..abb6d8f8 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -5,6 +5,7 @@ Oracle Database Operator for Kubernetes (the operator) includes the Single Insta * [Prerequisites](#prerequisites) * [Kind SingleInstanceDatabase Resource](#kind-singleinstancedatabase-resource) * [Provision New Database](#provision-new-database) +* [Provision New Database on Minikube](#provision-new-database-on-minikube) * [Clone Existing Database](#clone-existing-database) * [Patch/Rollback Database](#patchrollback-database) @@ -114,6 +115,18 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI singleinstancedatabase.database.oracle.com/sidb-sample created ``` +* ### Provision New Database On Minikube + + Quickly Provision a new database instance on minikube using the [singleinstancedatabase_minikube.yaml](../../config/samples/sidb/singleinstancedatabase_minikube.yaml) + + ```sh + $ kubectl create -f singleinstancedatabase_minikube.yaml + + singleinstancedatabase.database.oracle.com/sidb-sample created + ``` + + + * ### Creation Status Creating a new database instance takes a while. When the 'status' status returns the response "Healthy", the Database is open for connections. From bea83160885add1be2977fc422166aa73fab4dc6 Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Fri, 17 Dec 2021 11:58:18 +0530 Subject: [PATCH 087/628] Readme changes for supporting sidb on minikube --- README.md | 1 + .../sidb/singleinstancedatabase_minikube.yaml | 93 +++++++++++++++++++ .../sidb/singleinstancedatabase_prov.yaml | 7 +- docs/sidb/README.md | 13 +++ 4 files changed, 110 insertions(+), 4 deletions(-) create mode 100644 config/samples/sidb/singleinstancedatabase_minikube.yaml diff --git a/README.md b/README.md index cd72a209..029173a0 100644 --- a/README.md +++ b/README.md @@ -30,6 +30,7 @@ Upcoming releases will support new configurations, operations and capabilities. This release can be deployed on the following platforms: +* [Minikube](https://minikube.sigs.k8s.io/docs/) with version v1.21.0 or later * [Oracle Container Engine for Kubernetes (OKE)](https://www.oracle.com/cloud-native/container-engine-kubernetes/) with Kubernetes 1.17 or later * In an on-premises [Oracle Linux Cloud Native Environment(OLCNE)](https://docs.oracle.com/en/operating-systems/olcne/) 1.3 or later diff --git a/config/samples/sidb/singleinstancedatabase_minikube.yaml b/config/samples/sidb/singleinstancedatabase_minikube.yaml new file mode 100644 index 00000000..195f3ca8 --- /dev/null +++ b/config/samples/sidb/singleinstancedatabase_minikube.yaml @@ -0,0 +1,93 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +apiVersion: v1 +kind: Secret +metadata: + name: db-secret +type: Opaque +data: + oracle_pwd: "T3JhY2xlIzc=" # 'Oracle#7' base64 encoded + +--- + +apiVersion: v1 +kind: PersistentVolume +metadata: + name: pv1 +spec: + accessModes: + - ReadWriteMany + capacity: + storage: 5Gi + hostPath: + path: /data/pv1 + +--- + +apiVersion: database.oracle.com/v1alpha1 +kind: SingleInstanceDatabase +metadata: + name: sidb-sample + namespace: default +spec: + + ## Use only alphanumeric characters for sid + sid: ORCL1 + + ## A source database ref to clone from, leave empty to create a fresh database + cloneFrom: "" + + ## NA if cloning from a SourceDB (cloneFrom is set) + edition: enterprise + + ## Should refer to SourceDB secret if cloning from a SourceDB (cloneFrom is set) + ## Secret containing SIDB password mapped to secretKey + ## This secret will be deleted after creation of the database unless keepSecret is set to true + adminPassword: + secretName: db-secret + secretKey: oracle_pwd + keepSecret: false + + ## NA if cloning from a SourceDB (cloneFrom is set) + charset: AL32UTF8 + + ## NA if cloning from a SourceDB (cloneFrom is set) + pdbName: orclpdb1 + + ## Enable/Disable Flashback + flashBack: false + + ## Enable/Disable ArchiveLog + archiveLog: false + + ## Enable/Disable ForceLogging + forceLog: false + + ## NA if cloning from a SourceDB (cloneFrom is set) + ## Specify both sgaSize and pgaSize (in MB) or dont specify both + ## Specify Non-Zero value to use + initParams: + cpuCount: 0 + processes: 0 + sgaTarget: 0 + pgaAggregateTarget: 0 + + ## Database image details + ## Database can be patched by updating the RU version/image + ## Major version changes are not supported + image: + pullFrom: + pullSecrets: + + ## size : Minimum size of pvc | class : PVC storage Class + ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany + persistence: + size: 4Gi + storageClass: "" + accessMode: "ReadWriteMany" + + ## Count of Database Pods. Applicable only for "ReadWriteMany" AccessMode + replicas: 1 diff --git a/config/samples/sidb/singleinstancedatabase_prov.yaml b/config/samples/sidb/singleinstancedatabase_prov.yaml index 7123b117..a1f06471 100644 --- a/config/samples/sidb/singleinstancedatabase_prov.yaml +++ b/config/samples/sidb/singleinstancedatabase_prov.yaml @@ -14,13 +14,12 @@ spec: ## Secret containing SIDB password mapped to secretKey adminPassword: - secret: - secretName: - key: + secretName: + secretKey: + keepSecret: false ## Database image details image: - version: pullFrom: pullSecrets: diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 168a2395..abb6d8f8 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -5,6 +5,7 @@ Oracle Database Operator for Kubernetes (the operator) includes the Single Insta * [Prerequisites](#prerequisites) * [Kind SingleInstanceDatabase Resource](#kind-singleinstancedatabase-resource) * [Provision New Database](#provision-new-database) +* [Provision New Database on Minikube](#provision-new-database-on-minikube) * [Clone Existing Database](#clone-existing-database) * [Patch/Rollback Database](#patchrollback-database) @@ -114,6 +115,18 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI singleinstancedatabase.database.oracle.com/sidb-sample created ``` +* ### Provision New Database On Minikube + + Quickly Provision a new database instance on minikube using the [singleinstancedatabase_minikube.yaml](../../config/samples/sidb/singleinstancedatabase_minikube.yaml) + + ```sh + $ kubectl create -f singleinstancedatabase_minikube.yaml + + singleinstancedatabase.database.oracle.com/sidb-sample created + ``` + + + * ### Creation Status Creating a new database instance takes a while. When the 'status' status returns the response "Healthy", the Database is open for connections. From cd83a64bc30cbda9389a45a0838facc6dad2cf91 Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Fri, 17 Dec 2021 11:59:58 +0530 Subject: [PATCH 088/628] Removed blank lines --- docs/sidb/README.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index abb6d8f8..309c518b 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -124,9 +124,7 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI singleinstancedatabase.database.oracle.com/sidb-sample created ``` - - - + * ### Creation Status Creating a new database instance takes a while. When the 'status' status returns the response "Healthy", the Database is open for connections. From d80f3bae2a88bca5a8cd8a71830b7836e42db8da Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Fri, 17 Dec 2021 11:59:58 +0530 Subject: [PATCH 089/628] Removed blank lines --- docs/sidb/README.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index abb6d8f8..309c518b 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -124,9 +124,7 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI singleinstancedatabase.database.oracle.com/sidb-sample created ``` - - - + * ### Creation Status Creating a new database instance takes a while. When the 'status' status returns the response "Healthy", the Database is open for connections. From c375ea9ff7f39df16bb4e7421ab11fd62dcbec9e Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Fri, 17 Dec 2021 17:43:59 +0530 Subject: [PATCH 090/628] Changed sanple yamls --- config/samples/sidb/singleinstancedatabase.yaml | 5 +++-- config/samples/sidb/singleinstancedatabase_clone.yaml | 5 +++-- .../samples/sidb/singleinstancedatabase_minikube.yaml | 11 ++++++----- config/samples/sidb/singleinstancedatabase_patch.yaml | 5 +++-- config/samples/sidb/singleinstancedatabase_prov.yaml | 9 +++++---- docs/sidb/README.md | 5 +++-- 6 files changed, 23 insertions(+), 17 deletions(-) diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index ac140361..dd5fe306 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -59,10 +59,11 @@ spec: ## size : Minimum size of pvc | class : PVC storage Class ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany + ## Change the storageClass and accessMode accordingly if you dont want to use oci block volume persistence: size: 100Gi - storageClass: "" - accessMode: "ReadWriteMany" + storageClass: "oci" + accessMode: "ReadWriteOnce" ## Type of service . Applicable on cloud enviroments only ## if loadBalService : false, service type = "NodePort". else "LoadBalancer" diff --git a/config/samples/sidb/singleinstancedatabase_clone.yaml b/config/samples/sidb/singleinstancedatabase_clone.yaml index c324c1c3..146d7c0c 100644 --- a/config/samples/sidb/singleinstancedatabase_clone.yaml +++ b/config/samples/sidb/singleinstancedatabase_clone.yaml @@ -32,10 +32,11 @@ spec: ## size : Minimum size of pvc | class : PVC storage Class ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany + ## Change the storageClass and accessMode accordingly if you dont want to use oci block volume persistence: size: 100Gi - storageClass: "" - accessMode: "ReadWriteMany" + storageClass: "oci" + accessMode: "ReadWriteOnce" ## Count of Database Pods. Applicable only for "ReadWriteMany" AccessMode replicas: 1 diff --git a/config/samples/sidb/singleinstancedatabase_minikube.yaml b/config/samples/sidb/singleinstancedatabase_minikube.yaml index 195f3ca8..2f17332a 100644 --- a/config/samples/sidb/singleinstancedatabase_minikube.yaml +++ b/config/samples/sidb/singleinstancedatabase_minikube.yaml @@ -8,22 +8,23 @@ kind: Secret metadata: name: db-secret type: Opaque -data: - oracle_pwd: "T3JhY2xlIzc=" # 'Oracle#7' base64 encoded +stringData: + oracle_pwd: "Change_On_Install_1" --- apiVersion: v1 kind: PersistentVolume metadata: - name: pv1 + name: sidb-pv spec: accessModes: - ReadWriteMany capacity: storage: 5Gi + storageClassName: sidb hostPath: - path: /data/pv1 + path: /data/oradata --- @@ -86,7 +87,7 @@ spec: ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany persistence: size: 4Gi - storageClass: "" + storageClass: "sidb" accessMode: "ReadWriteMany" ## Count of Database Pods. Applicable only for "ReadWriteMany" AccessMode diff --git a/config/samples/sidb/singleinstancedatabase_patch.yaml b/config/samples/sidb/singleinstancedatabase_patch.yaml index 4d2d03b0..db772386 100644 --- a/config/samples/sidb/singleinstancedatabase_patch.yaml +++ b/config/samples/sidb/singleinstancedatabase_patch.yaml @@ -27,10 +27,11 @@ spec: ## size : Minimum size of pvc | class : PVC storage Class ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany + ## Change the storageClass and accessMode accordingly if you dont want to use oci block volume persistence: size: 100Gi - storageClass: "" - accessMode: "ReadWriteMany" + storageClass: "oci" + accessMode: "ReadWriteOnce" ## Count of Database Pods. Applicable only for "ReadWriteMany" AccessMode replicas: 1 diff --git a/config/samples/sidb/singleinstancedatabase_prov.yaml b/config/samples/sidb/singleinstancedatabase_prov.yaml index a1f06471..7754d363 100644 --- a/config/samples/sidb/singleinstancedatabase_prov.yaml +++ b/config/samples/sidb/singleinstancedatabase_prov.yaml @@ -23,12 +23,13 @@ spec: pullFrom: pullSecrets: - ## size : Minimum size of pvc | class : PVC storage Class . - ## AccessMode can only accept one of ReadWriteOnce , ReadWriteMany + ## size : Minimum size of pvc | class : PVC storage Class + ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany + ## Change the storageClass and accessMode accordingly if you dont want to use oci block volume persistence: size: 100Gi - storageClass: "" - accessMode: "ReadWriteMany" + storageClass: "oci" + accessMode: "ReadWriteOnce" ## Count of Database Pods. Applicable only for "ReadWriteMany" AccessMode replicas: 1 diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 309c518b..6fa4505c 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -5,7 +5,6 @@ Oracle Database Operator for Kubernetes (the operator) includes the Single Insta * [Prerequisites](#prerequisites) * [Kind SingleInstanceDatabase Resource](#kind-singleinstancedatabase-resource) * [Provision New Database](#provision-new-database) -* [Provision New Database on Minikube](#provision-new-database-on-minikube) * [Clone Existing Database](#clone-existing-database) * [Patch/Rollback Database](#patchrollback-database) @@ -115,7 +114,9 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI singleinstancedatabase.database.oracle.com/sidb-sample created ``` -* ### Provision New Database On Minikube + **NOTE:** Make sure you have created the required `.spec.adminPassword` [secret](https://kubernetes.io/docs/tasks/configmap-secret/managing-secret-using-kubectl/)) and `.spec.persistence` [persistent volume](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) + +* ### Quickly Provision On Minikube Quickly Provision a new database instance on minikube using the [singleinstancedatabase_minikube.yaml](../../config/samples/sidb/singleinstancedatabase_minikube.yaml) From 4d3b8853ada55e6066b5375e451ddb6d77808d5b Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Fri, 17 Dec 2021 17:43:59 +0530 Subject: [PATCH 091/628] Changed sanple yamls --- config/samples/sidb/singleinstancedatabase.yaml | 5 +++-- config/samples/sidb/singleinstancedatabase_clone.yaml | 5 +++-- .../samples/sidb/singleinstancedatabase_minikube.yaml | 11 ++++++----- config/samples/sidb/singleinstancedatabase_patch.yaml | 5 +++-- config/samples/sidb/singleinstancedatabase_prov.yaml | 9 +++++---- docs/sidb/README.md | 5 +++-- 6 files changed, 23 insertions(+), 17 deletions(-) diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index ac140361..dd5fe306 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -59,10 +59,11 @@ spec: ## size : Minimum size of pvc | class : PVC storage Class ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany + ## Change the storageClass and accessMode accordingly if you dont want to use oci block volume persistence: size: 100Gi - storageClass: "" - accessMode: "ReadWriteMany" + storageClass: "oci" + accessMode: "ReadWriteOnce" ## Type of service . Applicable on cloud enviroments only ## if loadBalService : false, service type = "NodePort". else "LoadBalancer" diff --git a/config/samples/sidb/singleinstancedatabase_clone.yaml b/config/samples/sidb/singleinstancedatabase_clone.yaml index c324c1c3..146d7c0c 100644 --- a/config/samples/sidb/singleinstancedatabase_clone.yaml +++ b/config/samples/sidb/singleinstancedatabase_clone.yaml @@ -32,10 +32,11 @@ spec: ## size : Minimum size of pvc | class : PVC storage Class ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany + ## Change the storageClass and accessMode accordingly if you dont want to use oci block volume persistence: size: 100Gi - storageClass: "" - accessMode: "ReadWriteMany" + storageClass: "oci" + accessMode: "ReadWriteOnce" ## Count of Database Pods. Applicable only for "ReadWriteMany" AccessMode replicas: 1 diff --git a/config/samples/sidb/singleinstancedatabase_minikube.yaml b/config/samples/sidb/singleinstancedatabase_minikube.yaml index 195f3ca8..2f17332a 100644 --- a/config/samples/sidb/singleinstancedatabase_minikube.yaml +++ b/config/samples/sidb/singleinstancedatabase_minikube.yaml @@ -8,22 +8,23 @@ kind: Secret metadata: name: db-secret type: Opaque -data: - oracle_pwd: "T3JhY2xlIzc=" # 'Oracle#7' base64 encoded +stringData: + oracle_pwd: "Change_On_Install_1" --- apiVersion: v1 kind: PersistentVolume metadata: - name: pv1 + name: sidb-pv spec: accessModes: - ReadWriteMany capacity: storage: 5Gi + storageClassName: sidb hostPath: - path: /data/pv1 + path: /data/oradata --- @@ -86,7 +87,7 @@ spec: ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany persistence: size: 4Gi - storageClass: "" + storageClass: "sidb" accessMode: "ReadWriteMany" ## Count of Database Pods. Applicable only for "ReadWriteMany" AccessMode diff --git a/config/samples/sidb/singleinstancedatabase_patch.yaml b/config/samples/sidb/singleinstancedatabase_patch.yaml index 4d2d03b0..db772386 100644 --- a/config/samples/sidb/singleinstancedatabase_patch.yaml +++ b/config/samples/sidb/singleinstancedatabase_patch.yaml @@ -27,10 +27,11 @@ spec: ## size : Minimum size of pvc | class : PVC storage Class ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany + ## Change the storageClass and accessMode accordingly if you dont want to use oci block volume persistence: size: 100Gi - storageClass: "" - accessMode: "ReadWriteMany" + storageClass: "oci" + accessMode: "ReadWriteOnce" ## Count of Database Pods. Applicable only for "ReadWriteMany" AccessMode replicas: 1 diff --git a/config/samples/sidb/singleinstancedatabase_prov.yaml b/config/samples/sidb/singleinstancedatabase_prov.yaml index a1f06471..7754d363 100644 --- a/config/samples/sidb/singleinstancedatabase_prov.yaml +++ b/config/samples/sidb/singleinstancedatabase_prov.yaml @@ -23,12 +23,13 @@ spec: pullFrom: pullSecrets: - ## size : Minimum size of pvc | class : PVC storage Class . - ## AccessMode can only accept one of ReadWriteOnce , ReadWriteMany + ## size : Minimum size of pvc | class : PVC storage Class + ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany + ## Change the storageClass and accessMode accordingly if you dont want to use oci block volume persistence: size: 100Gi - storageClass: "" - accessMode: "ReadWriteMany" + storageClass: "oci" + accessMode: "ReadWriteOnce" ## Count of Database Pods. Applicable only for "ReadWriteMany" AccessMode replicas: 1 diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 309c518b..6fa4505c 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -5,7 +5,6 @@ Oracle Database Operator for Kubernetes (the operator) includes the Single Insta * [Prerequisites](#prerequisites) * [Kind SingleInstanceDatabase Resource](#kind-singleinstancedatabase-resource) * [Provision New Database](#provision-new-database) -* [Provision New Database on Minikube](#provision-new-database-on-minikube) * [Clone Existing Database](#clone-existing-database) * [Patch/Rollback Database](#patchrollback-database) @@ -115,7 +114,9 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI singleinstancedatabase.database.oracle.com/sidb-sample created ``` -* ### Provision New Database On Minikube + **NOTE:** Make sure you have created the required `.spec.adminPassword` [secret](https://kubernetes.io/docs/tasks/configmap-secret/managing-secret-using-kubectl/)) and `.spec.persistence` [persistent volume](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) + +* ### Quickly Provision On Minikube Quickly Provision a new database instance on minikube using the [singleinstancedatabase_minikube.yaml](../../config/samples/sidb/singleinstancedatabase_minikube.yaml) From 8e22bc1ec6d771e0cc1488f8a482d4da0219e1ed Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Fri, 17 Dec 2021 17:45:42 +0530 Subject: [PATCH 092/628] Changed sanple yamls --- config/samples/sidb/singleinstancedatabase.yaml | 2 +- config/samples/sidb/singleinstancedatabase_clone.yaml | 2 +- config/samples/sidb/singleinstancedatabase_patch.yaml | 2 +- config/samples/sidb/singleinstancedatabase_prov.yaml | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index dd5fe306..52330a28 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -5,7 +5,7 @@ apiVersion: database.oracle.com/v1alpha1 kind: SingleInstanceDatabase metadata: - name: singleinstancedatabase-sample + name: sidb-sample namespace: default spec: diff --git a/config/samples/sidb/singleinstancedatabase_clone.yaml b/config/samples/sidb/singleinstancedatabase_clone.yaml index 146d7c0c..c4cf8b0e 100644 --- a/config/samples/sidb/singleinstancedatabase_clone.yaml +++ b/config/samples/sidb/singleinstancedatabase_clone.yaml @@ -5,7 +5,7 @@ apiVersion: database.oracle.com/v1alpha1 kind: SingleInstanceDatabase metadata: - name: singleinstancedatabase-sample + name: sidb-sample-clone namespace: default spec: diff --git a/config/samples/sidb/singleinstancedatabase_patch.yaml b/config/samples/sidb/singleinstancedatabase_patch.yaml index db772386..dabf9719 100644 --- a/config/samples/sidb/singleinstancedatabase_patch.yaml +++ b/config/samples/sidb/singleinstancedatabase_patch.yaml @@ -5,7 +5,7 @@ apiVersion: database.oracle.com/v1alpha1 kind: SingleInstanceDatabase metadata: - name: singleinstancedatabase-sample + name: sidb-sample-patch namespace: default spec: diff --git a/config/samples/sidb/singleinstancedatabase_prov.yaml b/config/samples/sidb/singleinstancedatabase_prov.yaml index 7754d363..72d8beb7 100644 --- a/config/samples/sidb/singleinstancedatabase_prov.yaml +++ b/config/samples/sidb/singleinstancedatabase_prov.yaml @@ -5,7 +5,7 @@ apiVersion: database.oracle.com/v1alpha1 kind: SingleInstanceDatabase metadata: - name: singleinstancedatabase-sample + name: sidb-sample namespace: default spec: From 5d4e05e7e7b0b6e5810065bbd359ef4c41d52701 Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Fri, 17 Dec 2021 17:45:42 +0530 Subject: [PATCH 093/628] Changed sanple yamls --- config/samples/sidb/singleinstancedatabase.yaml | 2 +- config/samples/sidb/singleinstancedatabase_clone.yaml | 2 +- config/samples/sidb/singleinstancedatabase_patch.yaml | 2 +- config/samples/sidb/singleinstancedatabase_prov.yaml | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index dd5fe306..52330a28 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -5,7 +5,7 @@ apiVersion: database.oracle.com/v1alpha1 kind: SingleInstanceDatabase metadata: - name: singleinstancedatabase-sample + name: sidb-sample namespace: default spec: diff --git a/config/samples/sidb/singleinstancedatabase_clone.yaml b/config/samples/sidb/singleinstancedatabase_clone.yaml index 146d7c0c..c4cf8b0e 100644 --- a/config/samples/sidb/singleinstancedatabase_clone.yaml +++ b/config/samples/sidb/singleinstancedatabase_clone.yaml @@ -5,7 +5,7 @@ apiVersion: database.oracle.com/v1alpha1 kind: SingleInstanceDatabase metadata: - name: singleinstancedatabase-sample + name: sidb-sample-clone namespace: default spec: diff --git a/config/samples/sidb/singleinstancedatabase_patch.yaml b/config/samples/sidb/singleinstancedatabase_patch.yaml index db772386..dabf9719 100644 --- a/config/samples/sidb/singleinstancedatabase_patch.yaml +++ b/config/samples/sidb/singleinstancedatabase_patch.yaml @@ -5,7 +5,7 @@ apiVersion: database.oracle.com/v1alpha1 kind: SingleInstanceDatabase metadata: - name: singleinstancedatabase-sample + name: sidb-sample-patch namespace: default spec: diff --git a/config/samples/sidb/singleinstancedatabase_prov.yaml b/config/samples/sidb/singleinstancedatabase_prov.yaml index 7754d363..72d8beb7 100644 --- a/config/samples/sidb/singleinstancedatabase_prov.yaml +++ b/config/samples/sidb/singleinstancedatabase_prov.yaml @@ -5,7 +5,7 @@ apiVersion: database.oracle.com/v1alpha1 kind: SingleInstanceDatabase metadata: - name: singleinstancedatabase-sample + name: sidb-sample namespace: default spec: From 92c44d81f3f40c1167df6705620f1d1251d79abe Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Fri, 17 Dec 2021 18:53:49 +0530 Subject: [PATCH 094/628] Changed sanple yamls --- config/samples/sidb/singleinstancedatabase.yaml | 2 +- config/samples/sidb/singleinstancedatabase_clone.yaml | 2 +- config/samples/sidb/singleinstancedatabase_minikube.yaml | 2 +- config/samples/sidb/singleinstancedatabase_patch.yaml | 2 +- config/samples/sidb/singleinstancedatabase_prov.yaml | 2 +- docs/sidb/README.md | 7 +++++++ 6 files changed, 12 insertions(+), 5 deletions(-) diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index 52330a28..95142ab7 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -59,7 +59,7 @@ spec: ## size : Minimum size of pvc | class : PVC storage Class ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany - ## Change the storageClass and accessMode accordingly if you dont want to use oci block volume + ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types for persistent volumes. persistence: size: 100Gi storageClass: "oci" diff --git a/config/samples/sidb/singleinstancedatabase_clone.yaml b/config/samples/sidb/singleinstancedatabase_clone.yaml index c4cf8b0e..e465ffe3 100644 --- a/config/samples/sidb/singleinstancedatabase_clone.yaml +++ b/config/samples/sidb/singleinstancedatabase_clone.yaml @@ -32,7 +32,7 @@ spec: ## size : Minimum size of pvc | class : PVC storage Class ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany - ## Change the storageClass and accessMode accordingly if you dont want to use oci block volume + ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types for persistent volumes. persistence: size: 100Gi storageClass: "oci" diff --git a/config/samples/sidb/singleinstancedatabase_minikube.yaml b/config/samples/sidb/singleinstancedatabase_minikube.yaml index 2f17332a..749b8bc8 100644 --- a/config/samples/sidb/singleinstancedatabase_minikube.yaml +++ b/config/samples/sidb/singleinstancedatabase_minikube.yaml @@ -86,7 +86,7 @@ spec: ## size : Minimum size of pvc | class : PVC storage Class ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany persistence: - size: 4Gi + size: 5Gi storageClass: "sidb" accessMode: "ReadWriteMany" diff --git a/config/samples/sidb/singleinstancedatabase_patch.yaml b/config/samples/sidb/singleinstancedatabase_patch.yaml index dabf9719..9fb01955 100644 --- a/config/samples/sidb/singleinstancedatabase_patch.yaml +++ b/config/samples/sidb/singleinstancedatabase_patch.yaml @@ -27,7 +27,7 @@ spec: ## size : Minimum size of pvc | class : PVC storage Class ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany - ## Change the storageClass and accessMode accordingly if you dont want to use oci block volume + ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types for persistent volumes. persistence: size: 100Gi storageClass: "oci" diff --git a/config/samples/sidb/singleinstancedatabase_prov.yaml b/config/samples/sidb/singleinstancedatabase_prov.yaml index 72d8beb7..35d32209 100644 --- a/config/samples/sidb/singleinstancedatabase_prov.yaml +++ b/config/samples/sidb/singleinstancedatabase_prov.yaml @@ -25,7 +25,7 @@ spec: ## size : Minimum size of pvc | class : PVC storage Class ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany - ## Change the storageClass and accessMode accordingly if you dont want to use oci block volume + ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types for persistent volumes. persistence: size: 100Gi storageClass: "oci" diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 6fa4505c..308fe957 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -250,6 +250,13 @@ $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.status}" Patched Oracle Docker images can be built using this [patching extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/patching) + ```sh + kubectl --type=merge -p '{"spec":{"image":{"pullFrom":"patched-image:tag","pullSecrets":"pull-secret"}}}' patch singleinstancedatabase sidb-sample + + singleinstancedatabase.database.oracle.com/sidb-sample patched + + ``` + * ### Patch existing Database Edit and apply the `singleinstancedatabase.yaml` file of the database resource/object by specifying a new release update for image attributes. The database pods will be restarted with the new release update image. For minimum downtime, ensure that you have mutiple replicas of the database pods running. From f17be473d09befda8be2f36a52aceab95c6b30a7 Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Fri, 17 Dec 2021 18:53:49 +0530 Subject: [PATCH 095/628] Changed sanple yamls --- config/samples/sidb/singleinstancedatabase.yaml | 2 +- config/samples/sidb/singleinstancedatabase_clone.yaml | 2 +- config/samples/sidb/singleinstancedatabase_minikube.yaml | 2 +- config/samples/sidb/singleinstancedatabase_patch.yaml | 2 +- config/samples/sidb/singleinstancedatabase_prov.yaml | 2 +- docs/sidb/README.md | 7 +++++++ 6 files changed, 12 insertions(+), 5 deletions(-) diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index 52330a28..95142ab7 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -59,7 +59,7 @@ spec: ## size : Minimum size of pvc | class : PVC storage Class ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany - ## Change the storageClass and accessMode accordingly if you dont want to use oci block volume + ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types for persistent volumes. persistence: size: 100Gi storageClass: "oci" diff --git a/config/samples/sidb/singleinstancedatabase_clone.yaml b/config/samples/sidb/singleinstancedatabase_clone.yaml index c4cf8b0e..e465ffe3 100644 --- a/config/samples/sidb/singleinstancedatabase_clone.yaml +++ b/config/samples/sidb/singleinstancedatabase_clone.yaml @@ -32,7 +32,7 @@ spec: ## size : Minimum size of pvc | class : PVC storage Class ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany - ## Change the storageClass and accessMode accordingly if you dont want to use oci block volume + ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types for persistent volumes. persistence: size: 100Gi storageClass: "oci" diff --git a/config/samples/sidb/singleinstancedatabase_minikube.yaml b/config/samples/sidb/singleinstancedatabase_minikube.yaml index 2f17332a..749b8bc8 100644 --- a/config/samples/sidb/singleinstancedatabase_minikube.yaml +++ b/config/samples/sidb/singleinstancedatabase_minikube.yaml @@ -86,7 +86,7 @@ spec: ## size : Minimum size of pvc | class : PVC storage Class ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany persistence: - size: 4Gi + size: 5Gi storageClass: "sidb" accessMode: "ReadWriteMany" diff --git a/config/samples/sidb/singleinstancedatabase_patch.yaml b/config/samples/sidb/singleinstancedatabase_patch.yaml index dabf9719..9fb01955 100644 --- a/config/samples/sidb/singleinstancedatabase_patch.yaml +++ b/config/samples/sidb/singleinstancedatabase_patch.yaml @@ -27,7 +27,7 @@ spec: ## size : Minimum size of pvc | class : PVC storage Class ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany - ## Change the storageClass and accessMode accordingly if you dont want to use oci block volume + ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types for persistent volumes. persistence: size: 100Gi storageClass: "oci" diff --git a/config/samples/sidb/singleinstancedatabase_prov.yaml b/config/samples/sidb/singleinstancedatabase_prov.yaml index 72d8beb7..35d32209 100644 --- a/config/samples/sidb/singleinstancedatabase_prov.yaml +++ b/config/samples/sidb/singleinstancedatabase_prov.yaml @@ -25,7 +25,7 @@ spec: ## size : Minimum size of pvc | class : PVC storage Class ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany - ## Change the storageClass and accessMode accordingly if you dont want to use oci block volume + ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types for persistent volumes. persistence: size: 100Gi storageClass: "oci" diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 6fa4505c..308fe957 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -250,6 +250,13 @@ $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.status}" Patched Oracle Docker images can be built using this [patching extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/patching) + ```sh + kubectl --type=merge -p '{"spec":{"image":{"pullFrom":"patched-image:tag","pullSecrets":"pull-secret"}}}' patch singleinstancedatabase sidb-sample + + singleinstancedatabase.database.oracle.com/sidb-sample patched + + ``` + * ### Patch existing Database Edit and apply the `singleinstancedatabase.yaml` file of the database resource/object by specifying a new release update for image attributes. The database pods will be restarted with the new release update image. For minimum downtime, ensure that you have mutiple replicas of the database pods running. From 01c72760138d0572f585182302685f5942192e2c Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Fri, 17 Dec 2021 18:59:28 +0530 Subject: [PATCH 096/628] Changed sanple yamls --- docs/sidb/README.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 308fe957..682034b4 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -106,25 +106,25 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI ## Provision New Database - Provision a new database instance by specifying appropriate values for the attributes in the the example `.yaml` file, and running the following command: + - Quickly Provision a new database instance on **minikube** using the [singleinstancedatabase_minikube.yaml](../../config/samples/sidb/singleinstancedatabase_minikube.yaml) using following one command. ```sh - $ kubectl create -f singleinstancedatabase.yaml + $ kubectl create -f singleinstancedatabase_minikube.yaml singleinstancedatabase.database.oracle.com/sidb-sample created ``` - **NOTE:** Make sure you have created the required `.spec.adminPassword` [secret](https://kubernetes.io/docs/tasks/configmap-secret/managing-secret-using-kubectl/)) and `.spec.persistence` [persistent volume](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) - -* ### Quickly Provision On Minikube - - Quickly Provision a new database instance on minikube using the [singleinstancedatabase_minikube.yaml](../../config/samples/sidb/singleinstancedatabase_minikube.yaml) + - Provision a new database instance on any k8s cluster by specifying appropriate values for the attributes in the the example `.yaml` file, and running the following command: ```sh - $ kubectl create -f singleinstancedatabase_minikube.yaml + $ kubectl create -f singleinstancedatabase.yaml singleinstancedatabase.database.oracle.com/sidb-sample created ``` + + **NOTE:** Make sure you have created the required `.spec.adminPassword` [secret](https://kubernetes.io/docs/tasks/configmap-secret/managing-secret-using-kubectl/)) and `.spec.persistence` [persistent volume](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) + + * ### Creation Status From 778aaf6c9c5144217cd28aa92cad3d7e49091834 Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Fri, 17 Dec 2021 18:59:28 +0530 Subject: [PATCH 097/628] Changed sanple yamls --- docs/sidb/README.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 308fe957..682034b4 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -106,25 +106,25 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI ## Provision New Database - Provision a new database instance by specifying appropriate values for the attributes in the the example `.yaml` file, and running the following command: + - Quickly Provision a new database instance on **minikube** using the [singleinstancedatabase_minikube.yaml](../../config/samples/sidb/singleinstancedatabase_minikube.yaml) using following one command. ```sh - $ kubectl create -f singleinstancedatabase.yaml + $ kubectl create -f singleinstancedatabase_minikube.yaml singleinstancedatabase.database.oracle.com/sidb-sample created ``` - **NOTE:** Make sure you have created the required `.spec.adminPassword` [secret](https://kubernetes.io/docs/tasks/configmap-secret/managing-secret-using-kubectl/)) and `.spec.persistence` [persistent volume](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) - -* ### Quickly Provision On Minikube - - Quickly Provision a new database instance on minikube using the [singleinstancedatabase_minikube.yaml](../../config/samples/sidb/singleinstancedatabase_minikube.yaml) + - Provision a new database instance on any k8s cluster by specifying appropriate values for the attributes in the the example `.yaml` file, and running the following command: ```sh - $ kubectl create -f singleinstancedatabase_minikube.yaml + $ kubectl create -f singleinstancedatabase.yaml singleinstancedatabase.database.oracle.com/sidb-sample created ``` + + **NOTE:** Make sure you have created the required `.spec.adminPassword` [secret](https://kubernetes.io/docs/tasks/configmap-secret/managing-secret-using-kubectl/)) and `.spec.persistence` [persistent volume](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) + + * ### Creation Status From 845f396bc11a926744e5c092939c9a84f5bb8783 Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Fri, 17 Dec 2021 19:03:40 +0530 Subject: [PATCH 098/628] Changed sanple yamls --- README.md | 2 +- config/samples/sidb/singleinstancedatabase.yaml | 2 +- config/samples/sidb/singleinstancedatabase_clone.yaml | 2 +- config/samples/sidb/singleinstancedatabase_patch.yaml | 2 +- config/samples/sidb/singleinstancedatabase_prov.yaml | 2 +- docs/sidb/README.md | 3 +-- 6 files changed, 6 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index 029173a0..99845ab9 100644 --- a/README.md +++ b/README.md @@ -30,9 +30,9 @@ Upcoming releases will support new configurations, operations and capabilities. This release can be deployed on the following platforms: -* [Minikube](https://minikube.sigs.k8s.io/docs/) with version v1.21.0 or later * [Oracle Container Engine for Kubernetes (OKE)](https://www.oracle.com/cloud-native/container-engine-kubernetes/) with Kubernetes 1.17 or later * In an on-premises [Oracle Linux Cloud Native Environment(OLCNE)](https://docs.oracle.com/en/operating-systems/olcne/) 1.3 or later +* [Minikube](https://minikube.sigs.k8s.io/docs/) with version v1.21.0 or later In upcoming releases, the operator will be certified against third-party Kubernetes clusters. diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index 95142ab7..99e2ee3a 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -59,7 +59,7 @@ spec: ## size : Minimum size of pvc | class : PVC storage Class ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany - ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types for persistent volumes. + ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. persistence: size: 100Gi storageClass: "oci" diff --git a/config/samples/sidb/singleinstancedatabase_clone.yaml b/config/samples/sidb/singleinstancedatabase_clone.yaml index e465ffe3..a6d8237b 100644 --- a/config/samples/sidb/singleinstancedatabase_clone.yaml +++ b/config/samples/sidb/singleinstancedatabase_clone.yaml @@ -32,7 +32,7 @@ spec: ## size : Minimum size of pvc | class : PVC storage Class ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany - ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types for persistent volumes. + ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. persistence: size: 100Gi storageClass: "oci" diff --git a/config/samples/sidb/singleinstancedatabase_patch.yaml b/config/samples/sidb/singleinstancedatabase_patch.yaml index 9fb01955..fb55e6b2 100644 --- a/config/samples/sidb/singleinstancedatabase_patch.yaml +++ b/config/samples/sidb/singleinstancedatabase_patch.yaml @@ -27,7 +27,7 @@ spec: ## size : Minimum size of pvc | class : PVC storage Class ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany - ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types for persistent volumes. + ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. persistence: size: 100Gi storageClass: "oci" diff --git a/config/samples/sidb/singleinstancedatabase_prov.yaml b/config/samples/sidb/singleinstancedatabase_prov.yaml index 35d32209..6e6afde0 100644 --- a/config/samples/sidb/singleinstancedatabase_prov.yaml +++ b/config/samples/sidb/singleinstancedatabase_prov.yaml @@ -25,7 +25,7 @@ spec: ## size : Minimum size of pvc | class : PVC storage Class ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany - ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types for persistent volumes. + ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. persistence: size: 100Gi storageClass: "oci" diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 682034b4..0b450f14 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -114,7 +114,7 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI singleinstancedatabase.database.oracle.com/sidb-sample created ``` - - Provision a new database instance on any k8s cluster by specifying appropriate values for the attributes in the the example `.yaml` file, and running the following command: + - Provision a new database instance on any k8s cluster by specifying appropriate values for the attributes in the [singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml), and running the following command: ```sh $ kubectl create -f singleinstancedatabase.yaml @@ -124,7 +124,6 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI **NOTE:** Make sure you have created the required `.spec.adminPassword` [secret](https://kubernetes.io/docs/tasks/configmap-secret/managing-secret-using-kubectl/)) and `.spec.persistence` [persistent volume](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) - * ### Creation Status From 712acf14d158302ff80093ffaa39b77a8edd3614 Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Fri, 17 Dec 2021 19:03:40 +0530 Subject: [PATCH 099/628] Changed sanple yamls --- README.md | 2 +- config/samples/sidb/singleinstancedatabase.yaml | 2 +- config/samples/sidb/singleinstancedatabase_clone.yaml | 2 +- config/samples/sidb/singleinstancedatabase_patch.yaml | 2 +- config/samples/sidb/singleinstancedatabase_prov.yaml | 2 +- docs/sidb/README.md | 3 +-- 6 files changed, 6 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index 029173a0..99845ab9 100644 --- a/README.md +++ b/README.md @@ -30,9 +30,9 @@ Upcoming releases will support new configurations, operations and capabilities. This release can be deployed on the following platforms: -* [Minikube](https://minikube.sigs.k8s.io/docs/) with version v1.21.0 or later * [Oracle Container Engine for Kubernetes (OKE)](https://www.oracle.com/cloud-native/container-engine-kubernetes/) with Kubernetes 1.17 or later * In an on-premises [Oracle Linux Cloud Native Environment(OLCNE)](https://docs.oracle.com/en/operating-systems/olcne/) 1.3 or later +* [Minikube](https://minikube.sigs.k8s.io/docs/) with version v1.21.0 or later In upcoming releases, the operator will be certified against third-party Kubernetes clusters. diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index 95142ab7..99e2ee3a 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -59,7 +59,7 @@ spec: ## size : Minimum size of pvc | class : PVC storage Class ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany - ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types for persistent volumes. + ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. persistence: size: 100Gi storageClass: "oci" diff --git a/config/samples/sidb/singleinstancedatabase_clone.yaml b/config/samples/sidb/singleinstancedatabase_clone.yaml index e465ffe3..a6d8237b 100644 --- a/config/samples/sidb/singleinstancedatabase_clone.yaml +++ b/config/samples/sidb/singleinstancedatabase_clone.yaml @@ -32,7 +32,7 @@ spec: ## size : Minimum size of pvc | class : PVC storage Class ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany - ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types for persistent volumes. + ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. persistence: size: 100Gi storageClass: "oci" diff --git a/config/samples/sidb/singleinstancedatabase_patch.yaml b/config/samples/sidb/singleinstancedatabase_patch.yaml index 9fb01955..fb55e6b2 100644 --- a/config/samples/sidb/singleinstancedatabase_patch.yaml +++ b/config/samples/sidb/singleinstancedatabase_patch.yaml @@ -27,7 +27,7 @@ spec: ## size : Minimum size of pvc | class : PVC storage Class ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany - ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types for persistent volumes. + ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. persistence: size: 100Gi storageClass: "oci" diff --git a/config/samples/sidb/singleinstancedatabase_prov.yaml b/config/samples/sidb/singleinstancedatabase_prov.yaml index 35d32209..6e6afde0 100644 --- a/config/samples/sidb/singleinstancedatabase_prov.yaml +++ b/config/samples/sidb/singleinstancedatabase_prov.yaml @@ -25,7 +25,7 @@ spec: ## size : Minimum size of pvc | class : PVC storage Class ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany - ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types for persistent volumes. + ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. persistence: size: 100Gi storageClass: "oci" diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 682034b4..0b450f14 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -114,7 +114,7 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI singleinstancedatabase.database.oracle.com/sidb-sample created ``` - - Provision a new database instance on any k8s cluster by specifying appropriate values for the attributes in the the example `.yaml` file, and running the following command: + - Provision a new database instance on any k8s cluster by specifying appropriate values for the attributes in the [singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml), and running the following command: ```sh $ kubectl create -f singleinstancedatabase.yaml @@ -124,7 +124,6 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI **NOTE:** Make sure you have created the required `.spec.adminPassword` [secret](https://kubernetes.io/docs/tasks/configmap-secret/managing-secret-using-kubectl/)) and `.spec.persistence` [persistent volume](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) - * ### Creation Status From ab6ce67ba333e83268a89cea243f5aa013e20593 Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Fri, 17 Dec 2021 19:05:43 +0530 Subject: [PATCH 100/628] Changed sanple yamls --- docs/sidb/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 0b450f14..faef9aab 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -106,7 +106,7 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI ## Provision New Database - - Quickly Provision a new database instance on **minikube** using the [singleinstancedatabase_minikube.yaml](../../config/samples/sidb/singleinstancedatabase_minikube.yaml) using following one command. + - Easily provision a new database instance on **minikube** using [singleinstancedatabase_minikube.yaml](../../config/samples/sidb/singleinstancedatabase_minikube.yaml) by the following one command. ```sh $ kubectl create -f singleinstancedatabase_minikube.yaml From e7dd4f711d44ac68072aa7b37a93515bff271756 Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Fri, 17 Dec 2021 19:05:43 +0530 Subject: [PATCH 101/628] Changed sanple yamls --- docs/sidb/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 0b450f14..faef9aab 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -106,7 +106,7 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI ## Provision New Database - - Quickly Provision a new database instance on **minikube** using the [singleinstancedatabase_minikube.yaml](../../config/samples/sidb/singleinstancedatabase_minikube.yaml) using following one command. + - Easily provision a new database instance on **minikube** using [singleinstancedatabase_minikube.yaml](../../config/samples/sidb/singleinstancedatabase_minikube.yaml) by the following one command. ```sh $ kubectl create -f singleinstancedatabase_minikube.yaml From de798e2e55656354c2236e1d801a699628a1f7bd Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Fri, 17 Dec 2021 19:08:00 +0530 Subject: [PATCH 102/628] Changed sanple yamls --- docs/sidb/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index faef9aab..5d696c71 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -114,7 +114,7 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI singleinstancedatabase.database.oracle.com/sidb-sample created ``` - - Provision a new database instance on any k8s cluster by specifying appropriate values for the attributes in the [singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml), and running the following command: + - Provision a new database instance on any K8s cluster by specifying appropriate values for the attributes in the [singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml) and running the following command: ```sh $ kubectl create -f singleinstancedatabase.yaml From 48f4ecb95eda26d76afc9703960954943e0c1863 Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Fri, 17 Dec 2021 19:08:00 +0530 Subject: [PATCH 103/628] Changed sanple yamls --- docs/sidb/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index faef9aab..5d696c71 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -114,7 +114,7 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI singleinstancedatabase.database.oracle.com/sidb-sample created ``` - - Provision a new database instance on any k8s cluster by specifying appropriate values for the attributes in the [singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml), and running the following command: + - Provision a new database instance on any K8s cluster by specifying appropriate values for the attributes in the [singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml) and running the following command: ```sh $ kubectl create -f singleinstancedatabase.yaml From 8e002df2a977e1cee3ee5158467f87ad03011bd0 Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Fri, 17 Dec 2021 19:09:15 +0530 Subject: [PATCH 104/628] Changed sanple yamls --- docs/sidb/README.md | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 5d696c71..1c124e29 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -108,21 +108,21 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI - Easily provision a new database instance on **minikube** using [singleinstancedatabase_minikube.yaml](../../config/samples/sidb/singleinstancedatabase_minikube.yaml) by the following one command. - ```sh - $ kubectl create -f singleinstancedatabase_minikube.yaml - - singleinstancedatabase.database.oracle.com/sidb-sample created - ``` + ```sh + $ kubectl create -f singleinstancedatabase_minikube.yaml + + singleinstancedatabase.database.oracle.com/sidb-sample created + ``` - Provision a new database instance on any K8s cluster by specifying appropriate values for the attributes in the [singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml) and running the following command: - ```sh - $ kubectl create -f singleinstancedatabase.yaml - - singleinstancedatabase.database.oracle.com/sidb-sample created - ``` + ```sh + $ kubectl create -f singleinstancedatabase.yaml + + singleinstancedatabase.database.oracle.com/sidb-sample created + ``` - **NOTE:** Make sure you have created the required `.spec.adminPassword` [secret](https://kubernetes.io/docs/tasks/configmap-secret/managing-secret-using-kubectl/)) and `.spec.persistence` [persistent volume](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) + **NOTE:** Make sure you have created the required `.spec.adminPassword` [secret](https://kubernetes.io/docs/tasks/configmap-secret/managing-secret-using-kubectl/) and `.spec.persistence` [persistent volume](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) * ### Creation Status From 6d980510ed8b3c108b98edaa335761ef0147f4b7 Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Fri, 17 Dec 2021 19:09:15 +0530 Subject: [PATCH 105/628] Changed sanple yamls --- docs/sidb/README.md | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 5d696c71..1c124e29 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -108,21 +108,21 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI - Easily provision a new database instance on **minikube** using [singleinstancedatabase_minikube.yaml](../../config/samples/sidb/singleinstancedatabase_minikube.yaml) by the following one command. - ```sh - $ kubectl create -f singleinstancedatabase_minikube.yaml - - singleinstancedatabase.database.oracle.com/sidb-sample created - ``` + ```sh + $ kubectl create -f singleinstancedatabase_minikube.yaml + + singleinstancedatabase.database.oracle.com/sidb-sample created + ``` - Provision a new database instance on any K8s cluster by specifying appropriate values for the attributes in the [singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml) and running the following command: - ```sh - $ kubectl create -f singleinstancedatabase.yaml - - singleinstancedatabase.database.oracle.com/sidb-sample created - ``` + ```sh + $ kubectl create -f singleinstancedatabase.yaml + + singleinstancedatabase.database.oracle.com/sidb-sample created + ``` - **NOTE:** Make sure you have created the required `.spec.adminPassword` [secret](https://kubernetes.io/docs/tasks/configmap-secret/managing-secret-using-kubectl/)) and `.spec.persistence` [persistent volume](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) + **NOTE:** Make sure you have created the required `.spec.adminPassword` [secret](https://kubernetes.io/docs/tasks/configmap-secret/managing-secret-using-kubectl/) and `.spec.persistence` [persistent volume](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) * ### Creation Status From f87532db581f772a45990b7e67dca1aaf590ec9f Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Fri, 17 Dec 2021 11:25:59 -0500 Subject: [PATCH 106/628] add missing attrubutes to AssertADBDetails --- test/e2e/behavior/shared_behaviors.go | 39 ++++++++++++++------------- 1 file changed, 21 insertions(+), 18 deletions(-) diff --git a/test/e2e/behavior/shared_behaviors.go b/test/e2e/behavior/shared_behaviors.go index 6047cd8a..593bc378 100644 --- a/test/e2e/behavior/shared_behaviors.go +++ b/test/e2e/behavior/shared_behaviors.go @@ -173,7 +173,7 @@ func AssertWallet(k8sClient *client.Client, adbLookupKey *types.NamespacedName) } } -func compartInt(obj1 *int, obj2 *int) bool { +func compareInt(obj1 *int, obj2 *int) bool { if obj1 == nil && obj2 == nil { return true } @@ -183,7 +183,7 @@ func compartInt(obj1 *int, obj2 *int) bool { return *obj1 == *obj2 } -func compartBool(obj1 *bool, obj2 *bool) bool { +func compareBool(obj1 *bool, obj2 *bool) bool { if obj1 == nil && obj2 == nil { return true } @@ -193,7 +193,7 @@ func compartBool(obj1 *bool, obj2 *bool) bool { return *obj1 == *obj2 } -func compartString(obj1 *string, obj2 *string) bool { +func compareString(obj1 *string, obj2 *string) bool { if obj1 == nil && obj2 == nil { return true } @@ -203,7 +203,7 @@ func compartString(obj1 *string, obj2 *string) bool { return *obj1 == *obj2 } -func compartStringMap(obj1 map[string]string, obj2 map[string]string) bool { +func compareStringMap(obj1 map[string]string, obj2 map[string]string) bool { if len(obj1) != len(obj2) { return false } @@ -298,24 +298,27 @@ func AssertADBDetails(k8sClient *client.Client, dbClient *database.DatabaseClien expectedADBDetails := expectedADB.Spec.Details - // Compare each elements. Reflect.DeepEqual isn't used here because some parameters (e.g. adminPassword) - // may not match. + // Compare the elements one by one rather than doing reflect.DeelEqual(adb1, adb2), since some parameters + // (e.g. adminPassword, wallet) are missing from e2eutil.GetAutonomousDatabase(). // We don't compare LifecycleState in this case. We only make sure that the ADB is in AVAIABLE state before // proceeding to the next test. - same := compartString(expectedADBDetails.AutonomousDatabaseOCID, resp.AutonomousDatabase.Id) && - compartString(expectedADBDetails.CompartmentOCID, resp.AutonomousDatabase.CompartmentId) && - compartString(expectedADBDetails.DisplayName, resp.AutonomousDatabase.DisplayName) && - compartString(expectedADBDetails.DbName, resp.AutonomousDatabase.DbName) && + same := compareString(expectedADBDetails.AutonomousDatabaseOCID, resp.AutonomousDatabase.Id) && + compareString(expectedADBDetails.CompartmentOCID, resp.AutonomousDatabase.CompartmentId) && + compareString(expectedADBDetails.DisplayName, resp.AutonomousDatabase.DisplayName) && + compareString(expectedADBDetails.DbName, resp.AutonomousDatabase.DbName) && expectedADBDetails.DbWorkload == resp.AutonomousDatabase.DbWorkload && - compartBool(expectedADBDetails.IsDedicated, resp.AutonomousDatabase.IsDedicated) && - compartString(expectedADBDetails.DbVersion, resp.AutonomousDatabase.DbVersion) && - compartInt(expectedADBDetails.DataStorageSizeInTBs, resp.AutonomousDatabase.DataStorageSizeInTBs) && - compartInt(expectedADBDetails.CPUCoreCount, resp.AutonomousDatabase.CpuCoreCount) && - compartBool(expectedADBDetails.IsAutoScalingEnabled, resp.AutonomousDatabase.IsAutoScalingEnabled) && - compartStringMap(expectedADBDetails.FreeformTags, resp.AutonomousDatabase.FreeformTags) && - compartString(expectedADBDetails.SubnetOCID, resp.AutonomousDatabase.SubnetId) && + compareBool(expectedADBDetails.IsDedicated, resp.AutonomousDatabase.IsDedicated) && + compareString(expectedADBDetails.DbVersion, resp.AutonomousDatabase.DbVersion) && + compareInt(expectedADBDetails.DataStorageSizeInTBs, resp.AutonomousDatabase.DataStorageSizeInTBs) && + compareInt(expectedADBDetails.CPUCoreCount, resp.AutonomousDatabase.CpuCoreCount) && + compareBool(expectedADBDetails.IsAutoScalingEnabled, resp.AutonomousDatabase.IsAutoScalingEnabled) && + compareStringMap(expectedADBDetails.FreeformTags, resp.AutonomousDatabase.FreeformTags) && + compareString(expectedADBDetails.SubnetOCID, resp.AutonomousDatabase.SubnetId) && reflect.DeepEqual(expectedADBDetails.NsgOCIDs, resp.AutonomousDatabase.NsgIds) && - compartString(expectedADBDetails.PrivateEndpointLabel, resp.AutonomousDatabase.PrivateEndpointLabel) + compareBool(expectedADBDetails.IsAccessControlEnabled, resp.AutonomousDatabase.IsAccessControlEnabled) && + reflect.DeepEqual(expectedADBDetails.WhitelistedIPs, resp.AutonomousDatabase.WhitelistedIps) && + compareBool(expectedADBDetails.IsMTLSConnectionRequired, resp.AutonomousDatabase.IsMtlsConnectionRequired) && + compareString(expectedADBDetails.PrivateEndpointLabel, resp.AutonomousDatabase.PrivateEndpointLabel) return same, nil }, updateTimeout, updateInterval).Should(BeTrue()) From 185585868d484edeca1fa6c3db4f0f3421b2f3b2 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Fri, 17 Dec 2021 11:25:59 -0500 Subject: [PATCH 107/628] add missing attrubutes to AssertADBDetails --- test/e2e/behavior/shared_behaviors.go | 39 ++++++++++++++------------- 1 file changed, 21 insertions(+), 18 deletions(-) diff --git a/test/e2e/behavior/shared_behaviors.go b/test/e2e/behavior/shared_behaviors.go index 6047cd8a..593bc378 100644 --- a/test/e2e/behavior/shared_behaviors.go +++ b/test/e2e/behavior/shared_behaviors.go @@ -173,7 +173,7 @@ func AssertWallet(k8sClient *client.Client, adbLookupKey *types.NamespacedName) } } -func compartInt(obj1 *int, obj2 *int) bool { +func compareInt(obj1 *int, obj2 *int) bool { if obj1 == nil && obj2 == nil { return true } @@ -183,7 +183,7 @@ func compartInt(obj1 *int, obj2 *int) bool { return *obj1 == *obj2 } -func compartBool(obj1 *bool, obj2 *bool) bool { +func compareBool(obj1 *bool, obj2 *bool) bool { if obj1 == nil && obj2 == nil { return true } @@ -193,7 +193,7 @@ func compartBool(obj1 *bool, obj2 *bool) bool { return *obj1 == *obj2 } -func compartString(obj1 *string, obj2 *string) bool { +func compareString(obj1 *string, obj2 *string) bool { if obj1 == nil && obj2 == nil { return true } @@ -203,7 +203,7 @@ func compartString(obj1 *string, obj2 *string) bool { return *obj1 == *obj2 } -func compartStringMap(obj1 map[string]string, obj2 map[string]string) bool { +func compareStringMap(obj1 map[string]string, obj2 map[string]string) bool { if len(obj1) != len(obj2) { return false } @@ -298,24 +298,27 @@ func AssertADBDetails(k8sClient *client.Client, dbClient *database.DatabaseClien expectedADBDetails := expectedADB.Spec.Details - // Compare each elements. Reflect.DeepEqual isn't used here because some parameters (e.g. adminPassword) - // may not match. + // Compare the elements one by one rather than doing reflect.DeelEqual(adb1, adb2), since some parameters + // (e.g. adminPassword, wallet) are missing from e2eutil.GetAutonomousDatabase(). // We don't compare LifecycleState in this case. We only make sure that the ADB is in AVAIABLE state before // proceeding to the next test. - same := compartString(expectedADBDetails.AutonomousDatabaseOCID, resp.AutonomousDatabase.Id) && - compartString(expectedADBDetails.CompartmentOCID, resp.AutonomousDatabase.CompartmentId) && - compartString(expectedADBDetails.DisplayName, resp.AutonomousDatabase.DisplayName) && - compartString(expectedADBDetails.DbName, resp.AutonomousDatabase.DbName) && + same := compareString(expectedADBDetails.AutonomousDatabaseOCID, resp.AutonomousDatabase.Id) && + compareString(expectedADBDetails.CompartmentOCID, resp.AutonomousDatabase.CompartmentId) && + compareString(expectedADBDetails.DisplayName, resp.AutonomousDatabase.DisplayName) && + compareString(expectedADBDetails.DbName, resp.AutonomousDatabase.DbName) && expectedADBDetails.DbWorkload == resp.AutonomousDatabase.DbWorkload && - compartBool(expectedADBDetails.IsDedicated, resp.AutonomousDatabase.IsDedicated) && - compartString(expectedADBDetails.DbVersion, resp.AutonomousDatabase.DbVersion) && - compartInt(expectedADBDetails.DataStorageSizeInTBs, resp.AutonomousDatabase.DataStorageSizeInTBs) && - compartInt(expectedADBDetails.CPUCoreCount, resp.AutonomousDatabase.CpuCoreCount) && - compartBool(expectedADBDetails.IsAutoScalingEnabled, resp.AutonomousDatabase.IsAutoScalingEnabled) && - compartStringMap(expectedADBDetails.FreeformTags, resp.AutonomousDatabase.FreeformTags) && - compartString(expectedADBDetails.SubnetOCID, resp.AutonomousDatabase.SubnetId) && + compareBool(expectedADBDetails.IsDedicated, resp.AutonomousDatabase.IsDedicated) && + compareString(expectedADBDetails.DbVersion, resp.AutonomousDatabase.DbVersion) && + compareInt(expectedADBDetails.DataStorageSizeInTBs, resp.AutonomousDatabase.DataStorageSizeInTBs) && + compareInt(expectedADBDetails.CPUCoreCount, resp.AutonomousDatabase.CpuCoreCount) && + compareBool(expectedADBDetails.IsAutoScalingEnabled, resp.AutonomousDatabase.IsAutoScalingEnabled) && + compareStringMap(expectedADBDetails.FreeformTags, resp.AutonomousDatabase.FreeformTags) && + compareString(expectedADBDetails.SubnetOCID, resp.AutonomousDatabase.SubnetId) && reflect.DeepEqual(expectedADBDetails.NsgOCIDs, resp.AutonomousDatabase.NsgIds) && - compartString(expectedADBDetails.PrivateEndpointLabel, resp.AutonomousDatabase.PrivateEndpointLabel) + compareBool(expectedADBDetails.IsAccessControlEnabled, resp.AutonomousDatabase.IsAccessControlEnabled) && + reflect.DeepEqual(expectedADBDetails.WhitelistedIPs, resp.AutonomousDatabase.WhitelistedIps) && + compareBool(expectedADBDetails.IsMTLSConnectionRequired, resp.AutonomousDatabase.IsMtlsConnectionRequired) && + compareString(expectedADBDetails.PrivateEndpointLabel, resp.AutonomousDatabase.PrivateEndpointLabel) return same, nil }, updateTimeout, updateInterval).Should(BeTrue()) From eb4bb7461f9814969d2ad39752d688bfd3090f86 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Sat, 6 Nov 2021 16:18:26 -0400 Subject: [PATCH 108/628] Add AutonomousDatabaseBackup --- PROJECT | 12 +- .../autonomousdatabasebackup_types.go | 122 ++++++++++ .../v1alpha1/zz_generated.deepcopy.go | 90 +++++++ commons/autonomousdatabase/reconciler_util.go | 123 +++++++++- .../reconciler_util.go | 97 ++++++++ commons/oci/database.go | 36 +++ ....oracle.com_autonomousdatabasebackups.yaml | 124 ++++++++++ config/crd/kustomization.yaml | 3 + ...njection_in_autonomousdatabasebackups.yaml | 8 + .../webhook_in_autonomousdatabasebackups.yaml | 17 ++ .../autonomousdatabasebackup_editor_role.yaml | 24 ++ .../autonomousdatabasebackup_viewer_role.yaml | 20 ++ config/rbac/role.yaml | 30 +++ .../adb_backup/autonomousdatabasebackup.yaml | 7 + config/samples/kustomization.yaml | 1 + .../database/autonomousdatabase_controller.go | 16 ++ .../autonomousdatabasebackup_controller.go | 222 ++++++++++++++++++ main.go | 8 + 18 files changed, 958 insertions(+), 2 deletions(-) create mode 100644 apis/database/v1alpha1/autonomousdatabasebackup_types.go create mode 100644 commons/autonomousdatabase_backup/reconciler_util.go create mode 100644 config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml create mode 100644 config/crd/patches/cainjection_in_autonomousdatabasebackups.yaml create mode 100644 config/crd/patches/webhook_in_autonomousdatabasebackups.yaml create mode 100644 config/rbac/autonomousdatabasebackup_editor_role.yaml create mode 100644 config/rbac/autonomousdatabasebackup_viewer_role.yaml create mode 100644 config/samples/adb_backup/autonomousdatabasebackup.yaml create mode 100644 controllers/database/autonomousdatabasebackup_controller.go diff --git a/PROJECT b/PROJECT index 70e34778..7ad79eeb 100644 --- a/PROJECT +++ b/PROJECT @@ -3,7 +3,8 @@ layout: - go.kubebuilder.io/v2 multigroup: true plugins: - go.sdk.operatorframework.io/v2-alpha: {} + manifests.sdk.operatorframework.io/v2: {} + scorecard.sdk.operatorframework.io/v2: {} projectName: oracle-database-operator repo: github.com/oracle/oracle-database-operator resources: @@ -38,4 +39,13 @@ resources: kind: ShardingDatabase path: github.com/oracle/oracle-database-operator/apis/database/v1alpha1 version: v1alpha1 +- api: + crdVersion: v1 + namespaced: true + controller: true + domain: oracle.com + group: database + kind: AutonomousDatabaseBackup + path: github.com/oracle/oracle-database-operator/apis/database/v1alpha1 + version: v1alpha1 version: "3" diff --git a/apis/database/v1alpha1/autonomousdatabasebackup_types.go b/apis/database/v1alpha1/autonomousdatabasebackup_types.go new file mode 100644 index 00000000..227aeff0 --- /dev/null +++ b/apis/database/v1alpha1/autonomousdatabasebackup_types.go @@ -0,0 +1,122 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v1alpha1 + +import ( + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + + "github.com/oracle/oci-go-sdk/v45/database" +) + +// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! +// NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized. + +// AutonomousDatabaseBackupSpec defines the desired state of AutonomousDatabaseBackup +type AutonomousDatabaseBackupSpec struct { + // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster + // Important: Run "make" to regenerate code after modifying this file + DisplayName string `json:"displayName,omitempty"` + AutonomousDatabaseOCID string `json:"autonomousDatabaseOCID,omitempty"` + AutonomousDatabaseBackupOCID string `json:"autonomousDatabaseBackupOCID,omitempty"` + + OCIConfig OCIConfigSpec `json:"ociConfig,omitempty"` +} + +// AutonomousDatabaseBackupStatus defines the observed state of AutonomousDatabaseBackup +type AutonomousDatabaseBackupStatus struct { + // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster + // Important: Run "make" to regenerate code after modifying this file + AutonomousDatabaseBackupOCID string `json:"autonomousDatabaseBackupOCID"` + CompartmentOCID string `json:"compartmentOCID"` + AutonomousDatabaseOCID string `json:"autonomousDatabaseOCID"` + DisplayName string `json:"displayName"` + Type database.AutonomousDatabaseBackupTypeEnum `json:"type"` + IsAutomatic bool `json:"isAutomatic"` + LifecycleState database.AutonomousDatabaseBackupLifecycleStateEnum `json:"lifecycleState"` + TimeStarted string `json:"timeStarted,omitempty"` + TimeEnded string `json:"timeEnded,omitempty"` +} + +//+kubebuilder:object:root=true +//+kubebuilder:subresource:status +// +kubebuilder:resource:shortName="adbbu";"adbbus" +// +kubebuilder:printcolumn:JSONPath=".spec.displayName",name="Display Name",type=string +// +kubebuilder:printcolumn:JSONPath=".status.lifecycleState",name="State",type=string +// +kubebuilder:printcolumn:JSONPath=".status.type",name="Type",type=string +// +kubebuilder:printcolumn:JSONPath=".status.timeStarted",name="Started",type=string +// +kubebuilder:printcolumn:JSONPath=".status.timeEnded",name="Ended",type=string + +// AutonomousDatabaseBackup is the Schema for the autonomousdatabasebackups API +type AutonomousDatabaseBackup struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + Spec AutonomousDatabaseBackupSpec `json:"spec,omitempty"` + Status AutonomousDatabaseBackupStatus `json:"status,omitempty"` +} + +func (backup *AutonomousDatabaseBackup) UpdateStatusFromAutonomousDatabaseBackupResponse(resp database.GetAutonomousDatabaseBackupResponse) { + backup.Status.AutonomousDatabaseBackupOCID = *resp.Id + backup.Status.CompartmentOCID = *resp.CompartmentId + backup.Status.AutonomousDatabaseOCID = *resp.AutonomousDatabaseId + backup.Status.DisplayName = *resp.DisplayName + backup.Status.Type = resp.Type + backup.Status.IsAutomatic = *resp.IsAutomatic + backup.Status.LifecycleState = resp.LifecycleState + + if resp.TimeStarted != nil { + backup.Status.TimeStarted = resp.TimeStarted.String() + } + if resp.TimeEnded != nil { + backup.Status.TimeEnded = resp.TimeEnded.String() + } +} + +//+kubebuilder:object:root=true + +// AutonomousDatabaseBackupList contains a list of AutonomousDatabaseBackup +type AutonomousDatabaseBackupList struct { + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata,omitempty"` + Items []AutonomousDatabaseBackup `json:"items"` +} + +func init() { + SchemeBuilder.Register(&AutonomousDatabaseBackup{}, &AutonomousDatabaseBackupList{}) +} diff --git a/apis/database/v1alpha1/zz_generated.deepcopy.go b/apis/database/v1alpha1/zz_generated.deepcopy.go index 4d27bb59..caad1f43 100644 --- a/apis/database/v1alpha1/zz_generated.deepcopy.go +++ b/apis/database/v1alpha1/zz_generated.deepcopy.go @@ -75,6 +75,96 @@ func (in *AutonomousDatabase) DeepCopyObject() runtime.Object { return nil } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousDatabaseBackup) DeepCopyInto(out *AutonomousDatabaseBackup) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + in.Spec.DeepCopyInto(&out.Spec) + out.Status = in.Status +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseBackup. +func (in *AutonomousDatabaseBackup) DeepCopy() *AutonomousDatabaseBackup { + if in == nil { + return nil + } + out := new(AutonomousDatabaseBackup) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *AutonomousDatabaseBackup) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousDatabaseBackupList) DeepCopyInto(out *AutonomousDatabaseBackupList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]AutonomousDatabaseBackup, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseBackupList. +func (in *AutonomousDatabaseBackupList) DeepCopy() *AutonomousDatabaseBackupList { + if in == nil { + return nil + } + out := new(AutonomousDatabaseBackupList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *AutonomousDatabaseBackupList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousDatabaseBackupSpec) DeepCopyInto(out *AutonomousDatabaseBackupSpec) { + *out = *in + in.OCIConfig.DeepCopyInto(&out.OCIConfig) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseBackupSpec. +func (in *AutonomousDatabaseBackupSpec) DeepCopy() *AutonomousDatabaseBackupSpec { + if in == nil { + return nil + } + out := new(AutonomousDatabaseBackupSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousDatabaseBackupStatus) DeepCopyInto(out *AutonomousDatabaseBackupStatus) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseBackupStatus. +func (in *AutonomousDatabaseBackupStatus) DeepCopy() *AutonomousDatabaseBackupStatus { + if in == nil { + return nil + } + out := new(AutonomousDatabaseBackupStatus) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *AutonomousDatabaseDetails) DeepCopyInto(out *AutonomousDatabaseDetails) { *out = *in diff --git a/commons/autonomousdatabase/reconciler_util.go b/commons/autonomousdatabase/reconciler_util.go index 2f820410..80107909 100644 --- a/commons/autonomousdatabase/reconciler_util.go +++ b/commons/autonomousdatabase/reconciler_util.go @@ -41,8 +41,11 @@ package autonomousdatabase import ( "context" "fmt" + "regexp" + "strings" corev1 "k8s.io/api/core/v1" + apiErrors "k8s.io/apimachinery/pkg/api/errors" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/types" "k8s.io/client-go/util/retry" @@ -57,7 +60,7 @@ import ( "github.com/oracle/oracle-database-operator/commons/oci" ) -// SetStatus sets the status subresource. +// SetStatus sets the status subresource of AutonomousDatabase func SetStatus(kubeClient client.Client, adb *dbv1alpha1.AutonomousDatabase) error { return retry.RetryOnConflict(retry.DefaultRetry, func() error { curADB := &dbv1alpha1.AutonomousDatabase{} @@ -125,3 +128,121 @@ func CreateWalletSecret(logger logr.Logger, kubeClient client.Client, dbClient d logger.Info(fmt.Sprintf("Wallet is stored in the Secret %s", *walletName)) return nil } + +func getValidName(name string, usedNames map[string]bool) string { + returnedName := name + var i = 1 + + _, ok := usedNames[returnedName] + for ok { + returnedName = fmt.Sprintf("%s-%d", name, i) + _, ok = usedNames[returnedName] + i++ + } + + return returnedName +} + +func getOwnerReference(adb *dbv1alpha1.AutonomousDatabase) []metav1.OwnerReference { + ownerRef := []metav1.OwnerReference{ + { + Kind: adb.GroupVersionKind().Kind, + APIVersion: adb.APIVersion, + Name: adb.Name, + UID: types.UID(adb.UID), + }, + } + return ownerRef +} + +// CreateBackupResources creates the all the AutonomousDatabasBackups that appears in the ListAutonomousDatabaseBackups request +// The backup object will not be created if it already exists. +func CreateBackupResources(logger logr.Logger, kubeClient client.Client, dbClient database.DatabaseClient, adb *dbv1alpha1.AutonomousDatabase) error { + // Get the list of AutonomousDatabaseBackupOCID in the same namespace + backupList := &dbv1alpha1.AutonomousDatabaseBackupList{} + + if err := kubeClient.List(context.TODO(), backupList, &client.ListOptions{Namespace: adb.Namespace}); err != nil { + // Ignore not-found errors, since they can't be fixed by an immediate requeue. + // No need to change the since we don't know if we obtain the object. + if !apiErrors.IsNotFound(err) { + return err + } + } + + usedNames := make(map[string]bool) + usedBackupOCIDs := make(map[string]bool) + + for _, backup := range backupList.Items { + usedNames[backup.Name] = true + + // Add both Spec.AutonomousDatabaseBackupOCID and Status.AutonomousDatabaseBackupOCID. + // If it's a backup created from the operator, it won't have the OCID under the spec. + // if the backup isn't ready yet, it won't have the OCID under the status. + if backup.Spec.AutonomousDatabaseBackupOCID != "" { + usedBackupOCIDs[backup.Spec.AutonomousDatabaseBackupOCID] = true + } + if backup.Status.AutonomousDatabaseBackupOCID != "" { + usedBackupOCIDs[backup.Status.AutonomousDatabaseBackupOCID] = true + } + } + + resp, err := oci.ListAutonomousDatabaseBackups(dbClient, adb) + if err != nil { + return err + } + + var ownerRef []metav1.OwnerReference + ownerRef = append(ownerRef, metav1.OwnerReference{Kind: adb.GroupVersionKind().Kind, APIVersion: adb.APIVersion, Name: adb.Name, UID: types.UID(adb.UID)}) + + for _, backupSummary := range resp.Items { + // Create the resource if the AutonomousDatabaseBackupOCID doesn't exist + _, ok := usedBackupOCIDs[*backupSummary.Id] + if !ok { + // Convert the string to lowercase, and replace spaces, commas, and colons with hyphens + backupName := *backupSummary.DisplayName + backupName = strings.ToLower(backupName) + + re, err := regexp.Compile(`[^-a-zA-Z0-9]`) + if err != nil { + return err + } + backupName = re.ReplaceAllString(backupName, "-") + backupName = getValidName(backupName, usedNames) + + backup := &dbv1alpha1.AutonomousDatabaseBackup{ + ObjectMeta: metav1.ObjectMeta{ + Namespace: adb.GetNamespace(), + Name: backupName, + OwnerReferences: getOwnerReference(adb), + }, + Spec: dbv1alpha1.AutonomousDatabaseBackupSpec{ + AutonomousDatabaseBackupOCID: *backupSummary.Id, + OCIConfig: dbv1alpha1.OCIConfigSpec{ + ConfigMapName: adb.Spec.OCIConfig.ConfigMapName, + SecretName: adb.Spec.OCIConfig.SecretName, + }, + }, + } + + // fields with mandatory:"false" could be nil + if backupSummary.TimeStarted != nil { + backup.Status.TimeStarted = backupSummary.TimeStarted.String() + } + if backupSummary.TimeEnded != nil { + backup.Status.TimeEnded = backupSummary.TimeEnded.String() + } + + if err := kubeClient.Create(context.TODO(), backup); err != nil { + return err + } + + // Add the used names and ocids + usedNames[backupName] = true + usedBackupOCIDs[*backupSummary.AutonomousDatabaseId] = true + + logger.Info("Create AutonomousDatabaseBackup " + backupName) + } + } + + return nil +} diff --git a/commons/autonomousdatabase_backup/reconciler_util.go b/commons/autonomousdatabase_backup/reconciler_util.go new file mode 100644 index 00000000..c95a2608 --- /dev/null +++ b/commons/autonomousdatabase_backup/reconciler_util.go @@ -0,0 +1,97 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package autonomousdatabase_backup + +import ( + "context" + + "k8s.io/apimachinery/pkg/types" + "k8s.io/client-go/util/retry" + "sigs.k8s.io/controller-runtime/pkg/client" + + "github.com/go-logr/logr" + dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" +) + +func UpdateAutonomousDatabaseBackupResource(logger logr.Logger, kubeClient client.Client, backup *dbv1alpha1.AutonomousDatabaseBackup) error { + if err := retry.RetryOnConflict(retry.DefaultRetry, func() error { + curBackup := &dbv1alpha1.AutonomousDatabaseBackup{} + + namespacedName := types.NamespacedName{ + Namespace: backup.GetNamespace(), + Name: backup.GetName(), + } + + if err := kubeClient.Get(context.TODO(), namespacedName, curBackup); err != nil { + return err + } + + curBackup.Spec = backup.Spec + return kubeClient.Update(context.TODO(), curBackup) + }); err != nil { + return err + } + + // Update status + if statusErr := UpdateStatus(kubeClient, backup); statusErr != nil { + return statusErr + } + logger.Info("Update local resource AutonomousDatabase successfully") + + return nil +} + +// UpdateStatus sets the status subresource of AutonomousDatabaseBackup +func UpdateStatus(kubeClient client.Client, adbBackup *dbv1alpha1.AutonomousDatabaseBackup) error { + return retry.RetryOnConflict(retry.DefaultRetry, func() error { + curBackup := &dbv1alpha1.AutonomousDatabaseBackup{} + + namespacedName := types.NamespacedName{ + Namespace: adbBackup.GetNamespace(), + Name: adbBackup.GetName(), + } + + if err := kubeClient.Get(context.TODO(), namespacedName, curBackup); err != nil { + return err + } + + curBackup.Status = adbBackup.Status + return kubeClient.Status().Update(context.TODO(), curBackup) + }) +} diff --git a/commons/oci/database.go b/commons/oci/database.go index 3fdf7a00..1f8aaf77 100644 --- a/commons/oci/database.go +++ b/commons/oci/database.go @@ -534,6 +534,42 @@ func DeleteAutonomousDatabase(dbClient database.DatabaseClient, adbOCID string) return } +// ListAutonomousDatabaseBackups returns a list of Autonomous Database backups +func ListAutonomousDatabaseBackups(dbClient database.DatabaseClient, adb *dbv1alpha1.AutonomousDatabase) (resp database.ListAutonomousDatabaseBackupsResponse, err error) { + if adb.Spec.Details.AutonomousDatabaseOCID == nil { + return resp, nil + } + + listBackupRequest := database.ListAutonomousDatabaseBackupsRequest{ + AutonomousDatabaseId: adb.Spec.Details.AutonomousDatabaseOCID, + } + + return dbClient.ListAutonomousDatabaseBackups(context.TODO(), listBackupRequest) +} + +// CreateAutonomousDatabaseBackup creates an backup of Autonomous Database +func CreateAutonomousDatabaseBackup(logger logr.Logger, dbClient database.DatabaseClient, adbBackup *dbv1alpha1.AutonomousDatabaseBackup) (resp database.CreateAutonomousDatabaseBackupResponse, err error) { + logger.Info("Creating Autonomous Database backup " + adbBackup.Spec.DisplayName) + + createBackupRequest := database.CreateAutonomousDatabaseBackupRequest{ + CreateAutonomousDatabaseBackupDetails: database.CreateAutonomousDatabaseBackupDetails{ + DisplayName: &adbBackup.Spec.DisplayName, + AutonomousDatabaseId: &adbBackup.Spec.AutonomousDatabaseOCID, + }, + } + + return dbClient.CreateAutonomousDatabaseBackup(context.TODO(), createBackupRequest) +} + +// GetAutonomousDatabaseBackup returns the response of GetAutonomousDatabaseBackupRequest +func GetAutonomousDatabaseBackup(dbClient database.DatabaseClient, backupOCID *string) (resp database.GetAutonomousDatabaseBackupResponse, err error) { + getBackupRequest := database.GetAutonomousDatabaseBackupRequest{ + AutonomousDatabaseBackupId: backupOCID, + } + + return dbClient.GetAutonomousDatabaseBackup(context.TODO(), getBackupRequest) +} + func WaitUntilWorkCompleted(logger logr.Logger, workClient workrequests.WorkRequestClient, opcWorkRequestID *string) error { if opcWorkRequestID == nil { return nil diff --git a/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml b/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml new file mode 100644 index 00000000..d47fd202 --- /dev/null +++ b/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml @@ -0,0 +1,124 @@ + +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.6.1 + creationTimestamp: null + name: autonomousdatabasebackups.database.oracle.com +spec: + group: database.oracle.com + names: + kind: AutonomousDatabaseBackup + listKind: AutonomousDatabaseBackupList + plural: autonomousdatabasebackups + shortNames: + - adbbu + - adbbus + singular: autonomousdatabasebackup + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .spec.displayName + name: Display Name + type: string + - jsonPath: .status.lifecycleState + name: State + type: string + - jsonPath: .status.type + name: Type + type: string + - jsonPath: .status.timeStarted + name: Started + type: string + - jsonPath: .status.timeEnded + name: Ended + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + description: AutonomousDatabaseBackup is the Schema for the autonomousdatabasebackups + API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: AutonomousDatabaseBackupSpec defines the desired state of + AutonomousDatabaseBackup + properties: + autonomousDatabaseBackupOCID: + type: string + autonomousDatabaseOCID: + type: string + displayName: + description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster + Important: Run "make" to regenerate code after modifying this file' + type: string + ociConfig: + properties: + configMapName: + type: string + secretName: + type: string + type: object + type: object + status: + description: AutonomousDatabaseBackupStatus defines the observed state + of AutonomousDatabaseBackup + properties: + autonomousDatabaseBackupOCID: + description: 'INSERT ADDITIONAL STATUS FIELD - define observed state + of cluster Important: Run "make" to regenerate code after modifying + this file' + type: string + autonomousDatabaseOCID: + type: string + compartmentOCID: + type: string + displayName: + type: string + isAutomatic: + type: boolean + lifecycleState: + description: 'AutonomousDatabaseBackupLifecycleStateEnum Enum with + underlying type: string' + type: string + timeEnded: + type: string + timeStarted: + type: string + type: + description: 'AutonomousDatabaseBackupTypeEnum Enum with underlying + type: string' + type: string + required: + - autonomousDatabaseBackupOCID + - autonomousDatabaseOCID + - compartmentOCID + - displayName + - isAutomatic + - lifecycleState + - type + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] diff --git a/config/crd/kustomization.yaml b/config/crd/kustomization.yaml index 6b3d488e..c6d82e86 100644 --- a/config/crd/kustomization.yaml +++ b/config/crd/kustomization.yaml @@ -9,6 +9,7 @@ resources: - bases/database.oracle.com_autonomousdatabases.yaml - bases/database.oracle.com_singleinstancedatabases.yaml - bases/database.oracle.com_shardingdatabases.yaml +- bases/database.oracle.com_autonomousdatabasebackups.yaml # +kubebuilder:scaffold:crdkustomizeresource patchesStrategicMerge: @@ -18,6 +19,7 @@ patchesStrategicMerge: #- patches/webhook_in_autonomousdatabases.yaml #- patches/webhook_in_singleinstancedatabases.yaml #- patches/webhook_in_shardingdatabases.yaml +#- patches/webhook_in_autonomousdatabasebackups.yaml # +kubebuilder:scaffold:crdkustomizewebhookpatch # [CERTMANAGER] To enable webhook, uncomment all the sections with [CERTMANAGER] prefix. @@ -26,6 +28,7 @@ patchesStrategicMerge: #- patches/cainjection_in_autonomousdatabases.yaml - patches/cainjection_in_singleinstancedatabases.yaml #- patches/cainjection_in_shardingdatabases.yaml +#- patches/cainjection_in_autonomousdatabasebackups.yaml # +kubebuilder:scaffold:crdkustomizecainjectionpatch # the following config is for teaching kustomize how to do kustomization for CRDs. diff --git a/config/crd/patches/cainjection_in_autonomousdatabasebackups.yaml b/config/crd/patches/cainjection_in_autonomousdatabasebackups.yaml new file mode 100644 index 00000000..78280137 --- /dev/null +++ b/config/crd/patches/cainjection_in_autonomousdatabasebackups.yaml @@ -0,0 +1,8 @@ +# The following patch adds a directive for certmanager to inject CA into the CRD +# CRD conversion requires k8s 1.13 or later. +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + annotations: + cert-manager.io/inject-ca-from: $(CERTIFICATE_NAMESPACE)/$(CERTIFICATE_NAME) + name: autonomousdatabasebackups.database.oracle.com diff --git a/config/crd/patches/webhook_in_autonomousdatabasebackups.yaml b/config/crd/patches/webhook_in_autonomousdatabasebackups.yaml new file mode 100644 index 00000000..1a4eacb6 --- /dev/null +++ b/config/crd/patches/webhook_in_autonomousdatabasebackups.yaml @@ -0,0 +1,17 @@ +# The following patch enables conversion webhook for CRD +# CRD conversion requires k8s 1.13 or later. +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: autonomousdatabasebackups.database.oracle.com +spec: + conversion: + strategy: Webhook + webhookClientConfig: + # this is "\n" used as a placeholder, otherwise it will be rejected by the apiserver for being blank, + # but we're going to set it later using the cert-manager (or potentially a patch if not using cert-manager) + caBundle: Cg== + service: + namespace: system + name: webhook-service + path: /convert diff --git a/config/rbac/autonomousdatabasebackup_editor_role.yaml b/config/rbac/autonomousdatabasebackup_editor_role.yaml new file mode 100644 index 00000000..1d210196 --- /dev/null +++ b/config/rbac/autonomousdatabasebackup_editor_role.yaml @@ -0,0 +1,24 @@ +# permissions for end users to edit autonomousdatabasebackups. +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: autonomousdatabasebackup-editor-role +rules: +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabasebackups + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabasebackups/status + verbs: + - get diff --git a/config/rbac/autonomousdatabasebackup_viewer_role.yaml b/config/rbac/autonomousdatabasebackup_viewer_role.yaml new file mode 100644 index 00000000..3be0c5cb --- /dev/null +++ b/config/rbac/autonomousdatabasebackup_viewer_role.yaml @@ -0,0 +1,20 @@ +# permissions for end users to view autonomousdatabasebackups. +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: autonomousdatabasebackup-viewer-role +rules: +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabasebackups + verbs: + - get + - list + - watch +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabasebackups/status + verbs: + - get diff --git a/config/rbac/role.yaml b/config/rbac/role.yaml index 6d763212..bef8ac3b 100644 --- a/config/rbac/role.yaml +++ b/config/rbac/role.yaml @@ -97,6 +97,36 @@ rules: - pods/exec verbs: - create +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabaseBackups + verbs: + - create + - delete + - get + - list + - update +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabasebackups + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabasebackups/status + verbs: + - get + - patch + - update - apiGroups: - database.oracle.com resources: diff --git a/config/samples/adb_backup/autonomousdatabasebackup.yaml b/config/samples/adb_backup/autonomousdatabasebackup.yaml new file mode 100644 index 00000000..93da9f77 --- /dev/null +++ b/config/samples/adb_backup/autonomousdatabasebackup.yaml @@ -0,0 +1,7 @@ +apiVersion: database.oracle.com/v1alpha1 +kind: AutonomousDatabaseBackup +metadata: + name: autonomousdatabasebackup-sample +spec: + # Add fields here + foo: bar diff --git a/config/samples/kustomization.yaml b/config/samples/kustomization.yaml index 4b419923..1a6e104a 100644 --- a/config/samples/kustomization.yaml +++ b/config/samples/kustomization.yaml @@ -9,4 +9,5 @@ resources: - autonomousdatabase.yaml - singleinstancedatabase.yaml - shardingdatabase.yaml + - autonomousdatabasebackup.yaml # +kubebuilder:scaffold:manifestskustomizesamples diff --git a/controllers/database/autonomousdatabase_controller.go b/controllers/database/autonomousdatabase_controller.go index 142e78c6..102cddc4 100644 --- a/controllers/database/autonomousdatabase_controller.go +++ b/controllers/database/autonomousdatabase_controller.go @@ -112,6 +112,7 @@ func (r *AutonomousDatabaseReconciler) eventFilterPredicate() predicate.Predicat // +kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabases,verbs=get;list;watch;create;update;patch;delete // +kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabases/status,verbs=update;patch +// +kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabaseBackups,verbs=get;list;create;update;delete // +kubebuilder:rbac:groups=coordination.k8s.io,resources=leases,verbs=create;get;list;update // +kubebuilder:rbac:groups="",resources=configmaps;secrets,verbs=get;list;watch;create;update;patch;delete @@ -548,6 +549,21 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R } } + /***************************************************** + * AutonomousDatabase Backups + *****************************************************/ + if err := adbutil.CreateBackupResources(r.currentLogger, r.KubeClient, dbClient, adb); err != nil { + r.currentLogger.Error(err, "Fail to update Autonomous Database") + + // Change the status to UNAVAILABLE + adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + return ctrl.Result{}, statusErr + } + // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue + return ctrl.Result{}, nil + } + /***************************************************** * Update last succesful spec *****************************************************/ diff --git a/controllers/database/autonomousdatabasebackup_controller.go b/controllers/database/autonomousdatabasebackup_controller.go new file mode 100644 index 00000000..0cf425c6 --- /dev/null +++ b/controllers/database/autonomousdatabasebackup_controller.go @@ -0,0 +1,222 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package controllers + +import ( + "context" + "reflect" + + "github.com/go-logr/logr" + apiErrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/runtime" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/controller" + "sigs.k8s.io/controller-runtime/pkg/event" + "sigs.k8s.io/controller-runtime/pkg/predicate" + + "github.com/oracle/oci-go-sdk/v45/database" + "github.com/oracle/oci-go-sdk/v45/workrequests" + databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + backupUtil "github.com/oracle/oracle-database-operator/commons/autonomousdatabase_backup" + "github.com/oracle/oracle-database-operator/commons/oci" +) + +// AutonomousDatabaseBackupReconciler reconciles a AutonomousDatabaseBackup object +type AutonomousDatabaseBackupReconciler struct { + KubeClient client.Client + Log logr.Logger + Scheme *runtime.Scheme + + currentLogger logr.Logger +} + +// SetupWithManager sets up the controller with the Manager. +func (r *AutonomousDatabaseBackupReconciler) SetupWithManager(mgr ctrl.Manager) error { + return ctrl.NewControllerManagedBy(mgr). + For(&databasev1alpha1.AutonomousDatabaseBackup{}). + WithEventFilter(r.eventFilterPredicate()). + WithOptions(controller.Options{MaxConcurrentReconciles: 50}). // ReconcileHandler is never invoked concurrently with the same object. + Complete(r) +} + +func (r *AutonomousDatabaseBackupReconciler) eventFilterPredicate() predicate.Predicate { + pred := predicate.Funcs{ + UpdateFunc: func(e event.UpdateEvent) bool { + oldBackup := e.ObjectOld.DeepCopyObject().(*dbv1alpha1.AutonomousDatabaseBackup) + newBackup := e.ObjectNew.DeepCopyObject().(*dbv1alpha1.AutonomousDatabaseBackup) + + specChanged := reflect.DeepEqual(oldBackup.Spec, newBackup.Spec) + statusChanged := reflect.DeepEqual(oldBackup.Status, newBackup.Status) + if specChanged || statusChanged { + // Don't enqueue request + return false + } + // Enqueue request + return true + }, + DeleteFunc: func(e event.DeleteEvent) bool { + // Do not trigger reconciliation when the real object is deleted from the cluster. + return false + }, + } + + return pred +} + +//+kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabasebackups,verbs=get;list;watch;create;update;patch;delete +//+kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabasebackups/status,verbs=get;update;patch + +func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + r.currentLogger = r.Log.WithValues("autonomousdatabase_backup", req.NamespacedName) + + adbBackup := &dbv1alpha1.AutonomousDatabaseBackup{} + if err := r.KubeClient.Get(context.TODO(), req.NamespacedName, adbBackup); err != nil { + // Ignore not-found errors, since they can't be fixed by an immediate requeue. + // No need to change the since we don't know if we obtain the object. + if !apiErrors.IsNotFound(err) { + return ctrl.Result{}, err + } + } + + /****************************************************************** + * Get OCI database client and work request client + ******************************************************************/ + authData := oci.APIKeyAuth{ + ConfigMapName: adbBackup.Spec.OCIConfig.ConfigMapName, + SecretName: adbBackup.Spec.OCIConfig.SecretName, + Namespace: adbBackup.GetNamespace(), + } + provider, err := oci.GetOCIProvider(r.KubeClient, authData) + if err != nil { + r.currentLogger.Error(err, "Fail to get OCI provider") + + // Change the status to UNAVAILABLE + adbBackup.Status.LifecycleState = database.AutonomousDatabaseBackupLifecycleStateFailed + if statusErr := backupUtil.UpdateStatus(r.KubeClient, adbBackup); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, nil + } + + dbClient, err := database.NewDatabaseClientWithConfigurationProvider(provider) + if err != nil { + r.currentLogger.Error(err, "Fail to get OCI database client") + + // Change the status to UNAVAILABLE + adbBackup.Status.LifecycleState = database.AutonomousDatabaseBackupLifecycleStateFailed + if statusErr := backupUtil.UpdateStatus(r.KubeClient, adbBackup); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, nil + } + + workClient, err := workrequests.NewWorkRequestClientWithConfigurationProvider(provider) + if err != nil { + r.currentLogger.Error(err, "Fail to get OCI work request client") + + // Change the status to UNAVAILABLE + adbBackup.Status.LifecycleState = database.AutonomousDatabaseBackupLifecycleStateFailed + if statusErr := backupUtil.UpdateStatus(r.KubeClient, adbBackup); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, nil + } + + /****************************************************************** + * Create a backup if the AutonomousDatabaseBackupOCID is empty + ******************************************************************/ + if adbBackup.Spec.AutonomousDatabaseBackupOCID == "" { + resp, err := oci.CreateAutonomousDatabaseBackup(r.currentLogger, dbClient, adbBackup) + if err != nil { + r.currentLogger.Error(err, "Fail to create AutonomousDatabase Backup") + + // Change the status to UNAVAILABLE + adbBackup.Status.LifecycleState = database.AutonomousDatabaseBackupLifecycleStateFailed + if statusErr := backupUtil.UpdateStatus(r.KubeClient, adbBackup); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, nil + } + + // update the status + adbBackup.Status.AutonomousDatabaseBackupOCID = *resp.Id + adbBackup.Status.LifecycleState = resp.LifecycleState + backupUtil.UpdateStatus(r.KubeClient, adbBackup) + + // Wait until the work is done + if err := oci.WaitUntilWorkCompleted(r.currentLogger, workClient, resp.OpcWorkRequestId); err != nil { + r.currentLogger.Error(err, "Fail to watch the status of provision request. opcWorkRequestID = "+*resp.OpcWorkRequestId) + + // Change the status to UNAVAILABLE + adbBackup.Status.LifecycleState = database.AutonomousDatabaseBackupLifecycleStateFailed + if statusErr := backupUtil.UpdateStatus(r.KubeClient, adbBackup); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, nil + } + } else { + // Binding + // Add the Status.AutonomousDatabaseBackupOCID and update the status + adbBackup.Status.AutonomousDatabaseBackupOCID = adbBackup.Spec.AutonomousDatabaseBackupOCID + backupUtil.UpdateStatus(r.KubeClient, adbBackup) + } + + /****************************************************************** + * Update the status of the resource + ******************************************************************/ + resp, err := oci.GetAutonomousDatabaseBackup(dbClient, &adbBackup.Status.AutonomousDatabaseBackupOCID) + if err != nil { + // Change the status to UNAVAILABLE + adbBackup.Status.LifecycleState = database.AutonomousDatabaseBackupLifecycleStateFailed + if statusErr := backupUtil.UpdateStatus(r.KubeClient, adbBackup); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, err + } + + adbBackup.UpdateStatusFromAutonomousDatabaseBackupResponse(resp) + + backupUtil.UpdateStatus(r.KubeClient, adbBackup) + + r.currentLogger.Info("AutonomousDatabaseBackup reconcile successfully") + + return ctrl.Result{}, nil +} diff --git a/main.go b/main.go index 9e68368f..8ebfa939 100644 --- a/main.go +++ b/main.go @@ -98,6 +98,14 @@ func main() { setupLog.Error(err, "unable to create controller", "controller", "AutonomousDatabase") os.Exit(1) } + if err = (&databasecontroller.AutonomousDatabaseBackupReconciler{ + KubeClient: mgr.GetClient(), + Log: ctrl.Log.WithName("controllers").WithName("AutonomousDatabaseBackup"), + Scheme: mgr.GetScheme(), + }).SetupWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create controller", "controller", "AutonomousDatabaseBackup") + os.Exit(1) + } if err = (&databasecontroller.SingleInstanceDatabaseReconciler{ Client: mgr.GetClient(), Log: ctrl.Log.WithName("controllers").WithName("database").WithName("SingleInstanceDatabase"), From 1a167d96f9ba234ce2ffc9e53e610b466db39fb5 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Mon, 8 Nov 2021 09:58:22 -0500 Subject: [PATCH 109/628] Set AutonomousDatabase as the owner --- .../adb_backup_reconciler_util.go} | 48 +++++++- ...onciler_util.go => adb_reconciler_util.go} | 26 ++--- commons/autonomousdatabase/common_util.go | 103 ++++++++++++++++++ .../database/autonomousdatabase_controller.go | 48 ++++---- .../autonomousdatabasebackup_controller.go | 46 +++++--- 5 files changed, 205 insertions(+), 66 deletions(-) rename commons/{autonomousdatabase_backup/reconciler_util.go => autonomousdatabase/adb_backup_reconciler_util.go} (64%) rename commons/autonomousdatabase/{reconciler_util.go => adb_reconciler_util.go} (91%) create mode 100644 commons/autonomousdatabase/common_util.go diff --git a/commons/autonomousdatabase_backup/reconciler_util.go b/commons/autonomousdatabase/adb_backup_reconciler_util.go similarity index 64% rename from commons/autonomousdatabase_backup/reconciler_util.go rename to commons/autonomousdatabase/adb_backup_reconciler_util.go index c95a2608..d819ce71 100644 --- a/commons/autonomousdatabase_backup/reconciler_util.go +++ b/commons/autonomousdatabase/adb_backup_reconciler_util.go @@ -36,10 +36,11 @@ ** SOFTWARE. */ -package autonomousdatabase_backup +package autonomousdatabase import ( "context" + "fmt" "k8s.io/apimachinery/pkg/types" "k8s.io/client-go/util/retry" @@ -49,7 +50,41 @@ import ( dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" ) -func UpdateAutonomousDatabaseBackupResource(logger logr.Logger, kubeClient client.Client, backup *dbv1alpha1.AutonomousDatabaseBackup) error { +// Returns the first AutonomousDatabase resource that matches the AutonomousDatabaseOCID of the backup +// If the AutonomousDatabase doesn't exist, returns a nil +func getOwnerAutonomousDatabase(kubeClient client.Client, namespace string, adbOCID string) (*dbv1alpha1.AutonomousDatabase, error) { + adbList, err := fetchAutonomousDatabases(kubeClient, namespace) + if err != nil { + return nil, err + } + + for _, adb := range adbList.Items { + if adb.Spec.Details.AutonomousDatabaseOCID != nil && *adb.Spec.Details.AutonomousDatabaseOCID == adbOCID { + return &adb, nil + } + } + + return nil, nil +} + +// SetOwnerAutonomousDatabase sets the owner of the AutonomousDatabaseBackup if the AutonomousDatabase resource with the same database OCID is found +func SetOwnerAutonomousDatabase(logger logr.Logger, kubeClient client.Client, backup *dbv1alpha1.AutonomousDatabaseBackup) error { + adb, err := getOwnerAutonomousDatabase(kubeClient, backup.Namespace, backup.Status.AutonomousDatabaseOCID) + if err != nil { + return err + } + + if adb != nil { + backup.SetOwnerReferences(newOwnerReference(adb)) + updateAutonomousDatabaseBackupResource(logger, kubeClient, backup) + logger.Info(fmt.Sprintf("Set the owner of %s to %s", backup.Name, adb.Name)) + } + + return nil +} + +// update the spec and the objectMeta +func updateAutonomousDatabaseBackupResource(logger logr.Logger, kubeClient client.Client, backup *dbv1alpha1.AutonomousDatabaseBackup) error { if err := retry.RetryOnConflict(retry.DefaultRetry, func() error { curBackup := &dbv1alpha1.AutonomousDatabaseBackup{} @@ -62,14 +97,15 @@ func UpdateAutonomousDatabaseBackupResource(logger logr.Logger, kubeClient clien return err } - curBackup.Spec = backup.Spec + curBackup.Spec = *backup.Spec.DeepCopy() + curBackup.ObjectMeta = *backup.ObjectMeta.DeepCopy() return kubeClient.Update(context.TODO(), curBackup) }); err != nil { return err } // Update status - if statusErr := UpdateStatus(kubeClient, backup); statusErr != nil { + if statusErr := UpdateAutonomousDatabaseBackupStatus(kubeClient, backup); statusErr != nil { return statusErr } logger.Info("Update local resource AutonomousDatabase successfully") @@ -77,8 +113,8 @@ func UpdateAutonomousDatabaseBackupResource(logger logr.Logger, kubeClient clien return nil } -// UpdateStatus sets the status subresource of AutonomousDatabaseBackup -func UpdateStatus(kubeClient client.Client, adbBackup *dbv1alpha1.AutonomousDatabaseBackup) error { +// UpdateAutonomousDatabaseBackupStatus updates the status subresource of AutonomousDatabaseBackup +func UpdateAutonomousDatabaseBackupStatus(kubeClient client.Client, adbBackup *dbv1alpha1.AutonomousDatabaseBackup) error { return retry.RetryOnConflict(retry.DefaultRetry, func() error { curBackup := &dbv1alpha1.AutonomousDatabaseBackup{} diff --git a/commons/autonomousdatabase/reconciler_util.go b/commons/autonomousdatabase/adb_reconciler_util.go similarity index 91% rename from commons/autonomousdatabase/reconciler_util.go rename to commons/autonomousdatabase/adb_reconciler_util.go index 80107909..bc783aa7 100644 --- a/commons/autonomousdatabase/reconciler_util.go +++ b/commons/autonomousdatabase/adb_reconciler_util.go @@ -60,8 +60,8 @@ import ( "github.com/oracle/oracle-database-operator/commons/oci" ) -// SetStatus sets the status subresource of AutonomousDatabase -func SetStatus(kubeClient client.Client, adb *dbv1alpha1.AutonomousDatabase) error { +// UpdateAutonomousDatabaseStatus sets the status subresource of AutonomousDatabase +func UpdateAutonomousDatabaseStatus(kubeClient client.Client, adb *dbv1alpha1.AutonomousDatabase) error { return retry.RetryOnConflict(retry.DefaultRetry, func() error { curADB := &dbv1alpha1.AutonomousDatabase{} @@ -143,23 +143,14 @@ func getValidName(name string, usedNames map[string]bool) string { return returnedName } -func getOwnerReference(adb *dbv1alpha1.AutonomousDatabase) []metav1.OwnerReference { - ownerRef := []metav1.OwnerReference{ - { - Kind: adb.GroupVersionKind().Kind, - APIVersion: adb.APIVersion, - Name: adb.Name, - UID: types.UID(adb.UID), - }, - } - return ownerRef -} - // CreateBackupResources creates the all the AutonomousDatabasBackups that appears in the ListAutonomousDatabaseBackups request // The backup object will not be created if it already exists. func CreateBackupResources(logger logr.Logger, kubeClient client.Client, dbClient database.DatabaseClient, adb *dbv1alpha1.AutonomousDatabase) error { // Get the list of AutonomousDatabaseBackupOCID in the same namespace - backupList := &dbv1alpha1.AutonomousDatabaseBackupList{} + backupList, err := fetchAutonomousDatabaseBackups(kubeClient, adb.Namespace) + if err != nil { + return err + } if err := kubeClient.List(context.TODO(), backupList, &client.ListOptions{Namespace: adb.Namespace}); err != nil { // Ignore not-found errors, since they can't be fixed by an immediate requeue. @@ -191,9 +182,6 @@ func CreateBackupResources(logger logr.Logger, kubeClient client.Client, dbClien return err } - var ownerRef []metav1.OwnerReference - ownerRef = append(ownerRef, metav1.OwnerReference{Kind: adb.GroupVersionKind().Kind, APIVersion: adb.APIVersion, Name: adb.Name, UID: types.UID(adb.UID)}) - for _, backupSummary := range resp.Items { // Create the resource if the AutonomousDatabaseBackupOCID doesn't exist _, ok := usedBackupOCIDs[*backupSummary.Id] @@ -213,7 +201,7 @@ func CreateBackupResources(logger logr.Logger, kubeClient client.Client, dbClien ObjectMeta: metav1.ObjectMeta{ Namespace: adb.GetNamespace(), Name: backupName, - OwnerReferences: getOwnerReference(adb), + OwnerReferences: newOwnerReference(adb), }, Spec: dbv1alpha1.AutonomousDatabaseBackupSpec{ AutonomousDatabaseBackupOCID: *backupSummary.Id, diff --git a/commons/autonomousdatabase/common_util.go b/commons/autonomousdatabase/common_util.go new file mode 100644 index 00000000..a344d3f6 --- /dev/null +++ b/commons/autonomousdatabase/common_util.go @@ -0,0 +1,103 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package autonomousdatabase + +import ( + "context" + + apiErrors "k8s.io/apimachinery/pkg/api/errors" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "sigs.k8s.io/controller-runtime/pkg/client" + + dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" +) + +func newOwnerReference(owner client.Object) []metav1.OwnerReference { + ownerRef := []metav1.OwnerReference{ + { + Kind: owner.GetObjectKind().GroupVersionKind().Kind, + APIVersion: owner.GetObjectKind().GroupVersionKind().GroupVersion().String(), + Name: owner.GetName(), + UID: owner.GetUID(), + }, + } + return ownerRef +} + +// func newOwnerReference(adb *dbv1alpha1.AutonomousDatabase) []metav1.OwnerReference { +// ownerRef := []metav1.OwnerReference{ +// { +// Kind: adb.GroupVersionKind().Kind, +// APIVersion: adb.APIVersion, +// Name: adb.Name, +// UID: types.UID(adb.UID), +// }, +// } +// return ownerRef +// } + +func fetchAutonomousDatabases(kubeClient client.Client, namespace string) (*dbv1alpha1.AutonomousDatabaseList, error) { + // Get the list of AutonomousDatabaseBackupOCID in the same namespace + adbList := &dbv1alpha1.AutonomousDatabaseList{} + + if err := kubeClient.List(context.TODO(), adbList, &client.ListOptions{Namespace: namespace}); err != nil { + // Ignore not-found errors, since they can't be fixed by an immediate requeue. + // No need to change the since we don't know if we obtain the object. + if !apiErrors.IsNotFound(err) { + return adbList, err + } + } + + return adbList, nil +} + +func fetchAutonomousDatabaseBackups(kubeClient client.Client, namespace string) (*dbv1alpha1.AutonomousDatabaseBackupList, error) { + // Get the list of AutonomousDatabaseBackupOCID in the same namespace + backupList := &dbv1alpha1.AutonomousDatabaseBackupList{} + + if err := kubeClient.List(context.TODO(), backupList, &client.ListOptions{Namespace: namespace}); err != nil { + // Ignore not-found errors, since they can't be fixed by an immediate requeue. + // No need to change the since we don't know if we obtain the object. + if !apiErrors.IsNotFound(err) { + return backupList, err + } + } + + return backupList, nil +} diff --git a/controllers/database/autonomousdatabase_controller.go b/controllers/database/autonomousdatabase_controller.go index 102cddc4..5d8c3258 100644 --- a/controllers/database/autonomousdatabase_controller.go +++ b/controllers/database/autonomousdatabase_controller.go @@ -146,7 +146,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } return ctrl.Result{}, nil @@ -158,7 +158,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } return ctrl.Result{}, nil @@ -170,7 +170,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } return ctrl.Result{}, nil @@ -182,7 +182,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } return ctrl.Result{}, nil @@ -220,7 +220,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } } @@ -260,7 +260,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue @@ -271,7 +271,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Update status.state adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } @@ -280,7 +280,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } } @@ -297,7 +297,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } return ctrl.Result{}, nil @@ -320,7 +320,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } return ctrl.Result{}, nil @@ -348,7 +348,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } return ctrl.Result{}, nil @@ -356,7 +356,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Update status.state adb.Status.LifecycleState = lifecycleState - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } @@ -365,7 +365,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } } @@ -382,7 +382,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue @@ -392,7 +392,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R if updateGenPassResp.OpcWorkRequestId != nil { // Update status.state adb.Status.LifecycleState = updateGenPassResp.AutonomousDatabase.LifecycleState - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } @@ -401,7 +401,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } } @@ -414,7 +414,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue @@ -424,7 +424,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R if scaleResp.OpcWorkRequestId != nil { // Update status.state adb.Status.LifecycleState = scaleResp.AutonomousDatabase.LifecycleState - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } @@ -433,7 +433,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } } @@ -511,7 +511,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } return ctrl.Result{}, nil @@ -523,7 +523,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R if err := r.updateAutonomousDatabaseDetails(adb); err != nil { // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } return ctrl.Result{}, err @@ -542,7 +542,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R r.currentLogger.Error(err, "Fail to download Instance Wallet") // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } return ctrl.Result{}, nil @@ -557,7 +557,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue @@ -618,7 +618,7 @@ func (r *AutonomousDatabaseReconciler) updateAutonomousDatabaseDetails(adb *dbv1 } // Update status - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return statusErr } r.currentLogger.Info("Update local resource AutonomousDatabase successfully") diff --git a/controllers/database/autonomousdatabasebackup_controller.go b/controllers/database/autonomousdatabasebackup_controller.go index 0cf425c6..966182f1 100644 --- a/controllers/database/autonomousdatabasebackup_controller.go +++ b/controllers/database/autonomousdatabasebackup_controller.go @@ -55,7 +55,7 @@ import ( "github.com/oracle/oci-go-sdk/v45/workrequests" databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" - backupUtil "github.com/oracle/oracle-database-operator/commons/autonomousdatabase_backup" + backupUtil "github.com/oracle/oracle-database-operator/commons/autonomousdatabase" "github.com/oracle/oracle-database-operator/commons/oci" ) @@ -84,13 +84,12 @@ func (r *AutonomousDatabaseBackupReconciler) eventFilterPredicate() predicate.Pr newBackup := e.ObjectNew.DeepCopyObject().(*dbv1alpha1.AutonomousDatabaseBackup) specChanged := reflect.DeepEqual(oldBackup.Spec, newBackup.Spec) - statusChanged := reflect.DeepEqual(oldBackup.Status, newBackup.Status) - if specChanged || statusChanged { - // Don't enqueue request - return false + if specChanged { + // Enqueue request + return true } - // Enqueue request - return true + // Don't enqueue request + return false }, DeleteFunc: func(e event.DeleteEvent) bool { // Do not trigger reconciliation when the real object is deleted from the cluster. @@ -130,7 +129,7 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req // Change the status to UNAVAILABLE adbBackup.Status.LifecycleState = database.AutonomousDatabaseBackupLifecycleStateFailed - if statusErr := backupUtil.UpdateStatus(r.KubeClient, adbBackup); statusErr != nil { + if statusErr := backupUtil.UpdateAutonomousDatabaseBackupStatus(r.KubeClient, adbBackup); statusErr != nil { return ctrl.Result{}, statusErr } return ctrl.Result{}, nil @@ -142,7 +141,7 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req // Change the status to UNAVAILABLE adbBackup.Status.LifecycleState = database.AutonomousDatabaseBackupLifecycleStateFailed - if statusErr := backupUtil.UpdateStatus(r.KubeClient, adbBackup); statusErr != nil { + if statusErr := backupUtil.UpdateAutonomousDatabaseBackupStatus(r.KubeClient, adbBackup); statusErr != nil { return ctrl.Result{}, statusErr } return ctrl.Result{}, nil @@ -154,14 +153,15 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req // Change the status to UNAVAILABLE adbBackup.Status.LifecycleState = database.AutonomousDatabaseBackupLifecycleStateFailed - if statusErr := backupUtil.UpdateStatus(r.KubeClient, adbBackup); statusErr != nil { + if statusErr := backupUtil.UpdateAutonomousDatabaseBackupStatus(r.KubeClient, adbBackup); statusErr != nil { return ctrl.Result{}, statusErr } return ctrl.Result{}, nil } /****************************************************************** - * Create a backup if the AutonomousDatabaseBackupOCID is empty + * Create a backup if the AutonomousDatabaseBackupOCID is empty, + * otherwise bind to an exisiting backup ******************************************************************/ if adbBackup.Spec.AutonomousDatabaseBackupOCID == "" { resp, err := oci.CreateAutonomousDatabaseBackup(r.currentLogger, dbClient, adbBackup) @@ -170,7 +170,7 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req // Change the status to UNAVAILABLE adbBackup.Status.LifecycleState = database.AutonomousDatabaseBackupLifecycleStateFailed - if statusErr := backupUtil.UpdateStatus(r.KubeClient, adbBackup); statusErr != nil { + if statusErr := backupUtil.UpdateAutonomousDatabaseBackupStatus(r.KubeClient, adbBackup); statusErr != nil { return ctrl.Result{}, statusErr } return ctrl.Result{}, nil @@ -179,7 +179,7 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req // update the status adbBackup.Status.AutonomousDatabaseBackupOCID = *resp.Id adbBackup.Status.LifecycleState = resp.LifecycleState - backupUtil.UpdateStatus(r.KubeClient, adbBackup) + backupUtil.UpdateAutonomousDatabaseBackupStatus(r.KubeClient, adbBackup) // Wait until the work is done if err := oci.WaitUntilWorkCompleted(r.currentLogger, workClient, resp.OpcWorkRequestId); err != nil { @@ -187,7 +187,7 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req // Change the status to UNAVAILABLE adbBackup.Status.LifecycleState = database.AutonomousDatabaseBackupLifecycleStateFailed - if statusErr := backupUtil.UpdateStatus(r.KubeClient, adbBackup); statusErr != nil { + if statusErr := backupUtil.UpdateAutonomousDatabaseBackupStatus(r.KubeClient, adbBackup); statusErr != nil { return ctrl.Result{}, statusErr } return ctrl.Result{}, nil @@ -196,7 +196,7 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req // Binding // Add the Status.AutonomousDatabaseBackupOCID and update the status adbBackup.Status.AutonomousDatabaseBackupOCID = adbBackup.Spec.AutonomousDatabaseBackupOCID - backupUtil.UpdateStatus(r.KubeClient, adbBackup) + backupUtil.UpdateAutonomousDatabaseBackupStatus(r.KubeClient, adbBackup) } /****************************************************************** @@ -206,7 +206,7 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req if err != nil { // Change the status to UNAVAILABLE adbBackup.Status.LifecycleState = database.AutonomousDatabaseBackupLifecycleStateFailed - if statusErr := backupUtil.UpdateStatus(r.KubeClient, adbBackup); statusErr != nil { + if statusErr := backupUtil.UpdateAutonomousDatabaseBackupStatus(r.KubeClient, adbBackup); statusErr != nil { return ctrl.Result{}, statusErr } return ctrl.Result{}, err @@ -214,7 +214,19 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req adbBackup.UpdateStatusFromAutonomousDatabaseBackupResponse(resp) - backupUtil.UpdateStatus(r.KubeClient, adbBackup) + backupUtil.UpdateAutonomousDatabaseBackupStatus(r.KubeClient, adbBackup) + + /****************************************************************** + * Update the ownerReference if the AutonomousDatabase is found + ******************************************************************/ + if err := backupUtil.SetOwnerAutonomousDatabase(r.currentLogger, r.KubeClient, adbBackup); err != nil { + // Change the status to UNAVAILABLE + adbBackup.Status.LifecycleState = database.AutonomousDatabaseBackupLifecycleStateFailed + if statusErr := backupUtil.UpdateAutonomousDatabaseBackupStatus(r.KubeClient, adbBackup); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, err + } r.currentLogger.Info("AutonomousDatabaseBackup reconcile successfully") From e4a67b02d56b55de6e17ec3f749f883fab35b30f Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Mon, 22 Nov 2021 11:35:52 -0500 Subject: [PATCH 110/628] add backup, restore. change oci-go-sdk to v51 --- PROJECT | 23 +- .../autonomousdatabasebackup_types.go | 9 +- .../autonomousdatabaserestore_types.go | 104 +++++++++ .../v1alpha1/zz_generated.deepcopy.go | 106 +++++++++ .../autonomousdatabase/adb_reconciler_util.go | 8 - .../adb_restore_reconciler_util.go | 68 ++++++ commons/oci/database.go | 52 ++++- commons/oci/ociutil/time.go | 13 ++ ....oracle.com_autonomousdatabasebackups.yaml | 2 +- ...oracle.com_autonomousdatabaserestores.yaml | 93 ++++++++ config/crd/kustomization.yaml | 3 + ...jection_in_autonomousdatabaserestores.yaml | 8 + ...webhook_in_autonomousdatabaserestores.yaml | 17 ++ config/manager/kustomization.yaml | 4 +- config/manager/manager.yaml | 2 +- ...autonomousdatabaserestore_editor_role.yaml | 24 ++ ...autonomousdatabaserestore_viewer_role.yaml | 20 ++ config/rbac/role.yaml | 20 ++ ...se_v1alpha1_autonomousdatabaserestore.yaml | 7 + config/samples/kustomization.yaml | 1 + .../database/autonomousdatabase_controller.go | 2 +- .../autonomousdatabasebackup_controller.go | 6 +- .../autonomousdatabaserestore_controller.go | 216 ++++++++++++++++++ controllers/database/suite_test.go | 3 + main.go | 8 + test/e2e/util/util.go | 2 +- 26 files changed, 792 insertions(+), 29 deletions(-) create mode 100644 apis/database/v1alpha1/autonomousdatabaserestore_types.go create mode 100644 commons/autonomousdatabase/adb_restore_reconciler_util.go create mode 100644 commons/oci/ociutil/time.go create mode 100644 config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml create mode 100644 config/crd/patches/cainjection_in_autonomousdatabaserestores.yaml create mode 100644 config/crd/patches/webhook_in_autonomousdatabaserestores.yaml create mode 100644 config/rbac/autonomousdatabaserestore_editor_role.yaml create mode 100644 config/rbac/autonomousdatabaserestore_viewer_role.yaml create mode 100644 config/samples/database_v1alpha1_autonomousdatabaserestore.yaml create mode 100644 controllers/database/autonomousdatabaserestore_controller.go diff --git a/PROJECT b/PROJECT index 7ad79eeb..184a0a94 100644 --- a/PROJECT +++ b/PROJECT @@ -23,29 +23,38 @@ resources: controller: true domain: oracle.com group: database - kind: SingleInstanceDatabase + kind: AutonomousDatabaseBackup + path: github.com/oracle/oracle-database-operator/apis/database/v1alpha1 + version: v1alpha1 +- api: + crdVersion: v1beta1 + namespaced: true + controller: true + domain: oracle.com + group: database + kind: AutonomousDatabaseRestore path: github.com/oracle/oracle-database-operator/apis/database/v1alpha1 version: v1alpha1 - webhooks: - defaulting: true - validation: true - webhookVersion: v1 - api: crdVersion: v1 namespaced: true controller: true domain: oracle.com group: database - kind: ShardingDatabase + kind: SingleInstanceDatabase path: github.com/oracle/oracle-database-operator/apis/database/v1alpha1 version: v1alpha1 + webhooks: + defaulting: true + validation: true + webhookVersion: v1 - api: crdVersion: v1 namespaced: true controller: true domain: oracle.com group: database - kind: AutonomousDatabaseBackup + kind: ShardingDatabase path: github.com/oracle/oracle-database-operator/apis/database/v1alpha1 version: v1alpha1 version: "3" diff --git a/apis/database/v1alpha1/autonomousdatabasebackup_types.go b/apis/database/v1alpha1/autonomousdatabasebackup_types.go index 227aeff0..50d93b42 100644 --- a/apis/database/v1alpha1/autonomousdatabasebackup_types.go +++ b/apis/database/v1alpha1/autonomousdatabasebackup_types.go @@ -41,7 +41,8 @@ package v1alpha1 import ( metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" - "github.com/oracle/oci-go-sdk/v45/database" + "github.com/oracle/oci-go-sdk/v51/database" + "github.com/oracle/oracle-database-operator/commons/oci/ociutil" ) // EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! @@ -76,7 +77,7 @@ type AutonomousDatabaseBackupStatus struct { //+kubebuilder:object:root=true //+kubebuilder:subresource:status // +kubebuilder:resource:shortName="adbbu";"adbbus" -// +kubebuilder:printcolumn:JSONPath=".spec.displayName",name="Display Name",type=string +// +kubebuilder:printcolumn:JSONPath=".status.displayName",name="Display Name",type=string // +kubebuilder:printcolumn:JSONPath=".status.lifecycleState",name="State",type=string // +kubebuilder:printcolumn:JSONPath=".status.type",name="Type",type=string // +kubebuilder:printcolumn:JSONPath=".status.timeStarted",name="Started",type=string @@ -101,10 +102,10 @@ func (backup *AutonomousDatabaseBackup) UpdateStatusFromAutonomousDatabaseBackup backup.Status.LifecycleState = resp.LifecycleState if resp.TimeStarted != nil { - backup.Status.TimeStarted = resp.TimeStarted.String() + backup.Status.TimeStarted = ociutil.FormatSDKTime(resp.TimeStarted.Time) } if resp.TimeEnded != nil { - backup.Status.TimeEnded = resp.TimeEnded.String() + backup.Status.TimeEnded = ociutil.FormatSDKTime(resp.TimeEnded.Time) } } diff --git a/apis/database/v1alpha1/autonomousdatabaserestore_types.go b/apis/database/v1alpha1/autonomousdatabaserestore_types.go new file mode 100644 index 00000000..f7b1a1ff --- /dev/null +++ b/apis/database/v1alpha1/autonomousdatabaserestore_types.go @@ -0,0 +1,104 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v1alpha1 + +import ( + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! +// NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized. + +// AutonomousDatabaseRestoreSpec defines the desired state of AutonomousDatabaseRestore +type AutonomousDatabaseRestoreSpec struct { + // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster + // Important: Run "make" to regenerate code after modifying this file + AutonomousDatabaseOCID string `json:"autonomousDatabaseOCID"` + Source AutonomousDatabaseRestoreSource `json:"source"` + + OCIConfig OCIConfigSpec `json:"ociConfig,omitempty"` +} + +type AutonomousDatabaseRestoreSource struct { + BackupName string `json:"backupName,omitempty"` + DateTime string `json:"datetime,omitempty"` +} + +// +kubebuilder:validation:Enum=New;InProgress;Completed;Failed +type RestoreLifecycleState string + +const ( + RestoreLifecycleStateNew RestoreLifecycleState = "NewCreated" + RestoreLifecycleStateInProgress RestoreLifecycleState = "InProgress" + RestoreLifecycleStateCompleted RestoreLifecycleState = "Completed" + RestoreLifecycleStateFailed RestoreLifecycleState = "Failed" +) + +// AutonomousDatabaseRestoreStatus defines the observed state of AutonomousDatabaseRestore +type AutonomousDatabaseRestoreStatus struct { + // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster + // Important: Run "make" to regenerate code after modifying this file + LifecycleState RestoreLifecycleState `json:"state"` +} + +//+kubebuilder:object:root=true +//+kubebuilder:subresource:status +//+kubebuilder:resource:shortName="adbr";"adbrs" + +// AutonomousDatabaseRestore is the Schema for the autonomousdatabaserestores API +type AutonomousDatabaseRestore struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + Spec AutonomousDatabaseRestoreSpec `json:"spec,omitempty"` + Status AutonomousDatabaseRestoreStatus `json:"status,omitempty"` +} + +//+kubebuilder:object:root=true + +// AutonomousDatabaseRestoreList contains a list of AutonomousDatabaseRestore +type AutonomousDatabaseRestoreList struct { + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata,omitempty"` + Items []AutonomousDatabaseRestore `json:"items"` +} + +func init() { + SchemeBuilder.Register(&AutonomousDatabaseRestore{}, &AutonomousDatabaseRestoreList{}) +} diff --git a/apis/database/v1alpha1/zz_generated.deepcopy.go b/apis/database/v1alpha1/zz_generated.deepcopy.go index caad1f43..585cc2c0 100644 --- a/apis/database/v1alpha1/zz_generated.deepcopy.go +++ b/apis/database/v1alpha1/zz_generated.deepcopy.go @@ -296,6 +296,112 @@ func (in *AutonomousDatabaseList) DeepCopyObject() runtime.Object { return nil } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousDatabaseRestore) DeepCopyInto(out *AutonomousDatabaseRestore) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + in.Spec.DeepCopyInto(&out.Spec) + out.Status = in.Status +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseRestore. +func (in *AutonomousDatabaseRestore) DeepCopy() *AutonomousDatabaseRestore { + if in == nil { + return nil + } + out := new(AutonomousDatabaseRestore) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *AutonomousDatabaseRestore) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousDatabaseRestoreList) DeepCopyInto(out *AutonomousDatabaseRestoreList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]AutonomousDatabaseRestore, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseRestoreList. +func (in *AutonomousDatabaseRestoreList) DeepCopy() *AutonomousDatabaseRestoreList { + if in == nil { + return nil + } + out := new(AutonomousDatabaseRestoreList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *AutonomousDatabaseRestoreList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousDatabaseRestoreSource) DeepCopyInto(out *AutonomousDatabaseRestoreSource) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseRestoreSource. +func (in *AutonomousDatabaseRestoreSource) DeepCopy() *AutonomousDatabaseRestoreSource { + if in == nil { + return nil + } + out := new(AutonomousDatabaseRestoreSource) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousDatabaseRestoreSpec) DeepCopyInto(out *AutonomousDatabaseRestoreSpec) { + *out = *in + out.Source = in.Source + in.OCIConfig.DeepCopyInto(&out.OCIConfig) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseRestoreSpec. +func (in *AutonomousDatabaseRestoreSpec) DeepCopy() *AutonomousDatabaseRestoreSpec { + if in == nil { + return nil + } + out := new(AutonomousDatabaseRestoreSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousDatabaseRestoreStatus) DeepCopyInto(out *AutonomousDatabaseRestoreStatus) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseRestoreStatus. +func (in *AutonomousDatabaseRestoreStatus) DeepCopy() *AutonomousDatabaseRestoreStatus { + if in == nil { + return nil + } + out := new(AutonomousDatabaseRestoreStatus) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *AutonomousDatabaseSpec) DeepCopyInto(out *AutonomousDatabaseSpec) { *out = *in diff --git a/commons/autonomousdatabase/adb_reconciler_util.go b/commons/autonomousdatabase/adb_reconciler_util.go index bc783aa7..1bf817aa 100644 --- a/commons/autonomousdatabase/adb_reconciler_util.go +++ b/commons/autonomousdatabase/adb_reconciler_util.go @@ -212,14 +212,6 @@ func CreateBackupResources(logger logr.Logger, kubeClient client.Client, dbClien }, } - // fields with mandatory:"false" could be nil - if backupSummary.TimeStarted != nil { - backup.Status.TimeStarted = backupSummary.TimeStarted.String() - } - if backupSummary.TimeEnded != nil { - backup.Status.TimeEnded = backupSummary.TimeEnded.String() - } - if err := kubeClient.Create(context.TODO(), backup); err != nil { return err } diff --git a/commons/autonomousdatabase/adb_restore_reconciler_util.go b/commons/autonomousdatabase/adb_restore_reconciler_util.go new file mode 100644 index 00000000..ed0b1437 --- /dev/null +++ b/commons/autonomousdatabase/adb_restore_reconciler_util.go @@ -0,0 +1,68 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package autonomousdatabase + +import ( + "context" + + "k8s.io/apimachinery/pkg/types" + "k8s.io/client-go/util/retry" + "sigs.k8s.io/controller-runtime/pkg/client" + + dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" +) + +// UpdateAutonomousDatabaseBackupStatus updates the status subresource of AutonomousDatabaseBackup +func UpdateAutonomousDatabaseRestoreStatus(kubeClient client.Client, adbRestore *dbv1alpha1.AutonomousDatabaseRestore) error { + return retry.RetryOnConflict(retry.DefaultRetry, func() error { + curBackup := &dbv1alpha1.AutonomousDatabaseRestore{} + + namespacedName := types.NamespacedName{ + Namespace: adbRestore.GetNamespace(), + Name: adbRestore.GetName(), + } + + if err := kubeClient.Get(context.TODO(), namespacedName, curBackup); err != nil { + return err + } + + curBackup.Status = adbRestore.Status + return kubeClient.Status().Update(context.TODO(), curBackup) + }) +} diff --git a/commons/oci/database.go b/commons/oci/database.go index 1f8aaf77..fe8c352b 100644 --- a/commons/oci/database.go +++ b/commons/oci/database.go @@ -39,10 +39,13 @@ package oci import ( + "bytes" "context" + "encoding/json" "errors" "fmt" "math" + "net/http" "time" "github.com/go-logr/logr" @@ -56,6 +59,7 @@ import ( "k8s.io/apimachinery/pkg/types" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + "github.com/oracle/oracle-database-operator/commons/oci/ociutil" ) // CreateAutonomousDatabase sends a request to OCI to provision a database and returns the AutonomousDatabase OCID. @@ -525,7 +529,6 @@ func stopAutonomousDatabase(dbClient database.DatabaseClient, adbOCID string) (r // DeleteAutonomousDatabase terminates an Autonomous Database in OCI func DeleteAutonomousDatabase(dbClient database.DatabaseClient, adbOCID string) (resp database.DeleteAutonomousDatabaseResponse, err error) { - deleteRequest := database.DeleteAutonomousDatabaseRequest{ AutonomousDatabaseId: common.String(adbOCID), } @@ -534,6 +537,53 @@ func DeleteAutonomousDatabase(dbClient database.DatabaseClient, adbOCID string) return } +func RestoreAutonomousDatabase(dbClient database.DatabaseClient, adbOCID string, dateTime time.Time) (opcID string, err error) { + endpoint := dbClient.Endpoint() + url := fmt.Sprintf("%s/20160918/autonomousDatabases/%s/actions/restore", endpoint, adbOCID) + + // build the body + reqBody, err := json.Marshal(map[string]string{ + "timestamp": ociutil.FormatSDKTime(dateTime), + }) + if err != nil { + return "", err + } + + // create request + request, err := http.NewRequest("POST", url, bytes.NewBuffer(reqBody)) + if err != nil { + return "", err + } + + // Set the header + request.Header.Set("Date", time.Now().UTC().Format(http.TimeFormat)) + request.Header.Set("Content-Type", "application/json") + + hash, err := common.GetBodyHash(request) + if err != nil { + return "", err + } + request.Header.Set("x-content-sha256", hash) + request.Header.Set("Content-Length", fmt.Sprint(request.ContentLength)) + + // Build the signer + signer := common.DefaultRequestSigner(*dbClient.ConfigurationProvider()) + + // Sign the request + signer.Sign(request) + + client := http.Client{} + + // Execute the request + resp, err := client.Do(request) + if err != nil { + return "", err + } + + opcRequestId := resp.Header.Get("opc-work-request-id") + return opcRequestId, nil +} + // ListAutonomousDatabaseBackups returns a list of Autonomous Database backups func ListAutonomousDatabaseBackups(dbClient database.DatabaseClient, adb *dbv1alpha1.AutonomousDatabase) (resp database.ListAutonomousDatabaseBackupsResponse, err error) { if adb.Spec.Details.AutonomousDatabaseOCID == nil { diff --git a/commons/oci/ociutil/time.go b/commons/oci/ociutil/time.go new file mode 100644 index 00000000..47e7cc88 --- /dev/null +++ b/commons/oci/ociutil/time.go @@ -0,0 +1,13 @@ +package ociutil + +import "time" + +const sdkFormat = "2006-01-02T15:04:05.999Z07:00" + +func FormatSDKTime(dateTime time.Time) string { + return dateTime.Format(sdkFormat) +} + +func ParseSDKTime(val string) (time.Time, error) { + return time.Parse(sdkFormat, val) +} diff --git a/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml b/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml index d47fd202..4e6ae88b 100644 --- a/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml +++ b/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml @@ -20,7 +20,7 @@ spec: scope: Namespaced versions: - additionalPrinterColumns: - - jsonPath: .spec.displayName + - jsonPath: .status.displayName name: Display Name type: string - jsonPath: .status.lifecycleState diff --git a/config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml b/config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml new file mode 100644 index 00000000..f4671cce --- /dev/null +++ b/config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml @@ -0,0 +1,93 @@ + +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.6.1 + creationTimestamp: null + name: autonomousdatabaserestores.database.oracle.com +spec: + group: database.oracle.com + names: + kind: AutonomousDatabaseRestore + listKind: AutonomousDatabaseRestoreList + plural: autonomousdatabaserestores + shortNames: + - adbr + - adbrs + singular: autonomousdatabaserestore + scope: Namespaced + versions: + - name: v1alpha1 + schema: + openAPIV3Schema: + description: AutonomousDatabaseRestore is the Schema for the autonomousdatabaserestores + API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: AutonomousDatabaseRestoreSpec defines the desired state of + AutonomousDatabaseRestore + properties: + autonomousDatabaseOCID: + description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster + Important: Run "make" to regenerate code after modifying this file' + type: string + ociConfig: + properties: + configMapName: + type: string + secretName: + type: string + type: object + source: + properties: + backupName: + type: string + datetime: + type: string + type: object + required: + - autonomousDatabaseOCID + - source + type: object + status: + description: AutonomousDatabaseRestoreStatus defines the observed state + of AutonomousDatabaseRestore + properties: + state: + description: 'INSERT ADDITIONAL STATUS FIELD - define observed state + of cluster Important: Run "make" to regenerate code after modifying + this file' + enum: + - New + - InProgress + - Completed + - Failed + type: string + required: + - state + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] diff --git a/config/crd/kustomization.yaml b/config/crd/kustomization.yaml index c6d82e86..589028fb 100644 --- a/config/crd/kustomization.yaml +++ b/config/crd/kustomization.yaml @@ -10,6 +10,7 @@ resources: - bases/database.oracle.com_singleinstancedatabases.yaml - bases/database.oracle.com_shardingdatabases.yaml - bases/database.oracle.com_autonomousdatabasebackups.yaml +- bases/database.oracle.com_autonomousdatabaserestores.yaml # +kubebuilder:scaffold:crdkustomizeresource patchesStrategicMerge: @@ -20,6 +21,7 @@ patchesStrategicMerge: #- patches/webhook_in_singleinstancedatabases.yaml #- patches/webhook_in_shardingdatabases.yaml #- patches/webhook_in_autonomousdatabasebackups.yaml +#- patches/webhook_in_autonomousdatabaserestores.yaml # +kubebuilder:scaffold:crdkustomizewebhookpatch # [CERTMANAGER] To enable webhook, uncomment all the sections with [CERTMANAGER] prefix. @@ -29,6 +31,7 @@ patchesStrategicMerge: - patches/cainjection_in_singleinstancedatabases.yaml #- patches/cainjection_in_shardingdatabases.yaml #- patches/cainjection_in_autonomousdatabasebackups.yaml +#- patches/cainjection_in_autonomousdatabaserestores.yaml # +kubebuilder:scaffold:crdkustomizecainjectionpatch # the following config is for teaching kustomize how to do kustomization for CRDs. diff --git a/config/crd/patches/cainjection_in_autonomousdatabaserestores.yaml b/config/crd/patches/cainjection_in_autonomousdatabaserestores.yaml new file mode 100644 index 00000000..75894cbb --- /dev/null +++ b/config/crd/patches/cainjection_in_autonomousdatabaserestores.yaml @@ -0,0 +1,8 @@ +# The following patch adds a directive for certmanager to inject CA into the CRD +# CRD conversion requires k8s 1.13 or later. +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + annotations: + cert-manager.io/inject-ca-from: $(CERTIFICATE_NAMESPACE)/$(CERTIFICATE_NAME) + name: autonomousdatabaserestores.database.oracle.com diff --git a/config/crd/patches/webhook_in_autonomousdatabaserestores.yaml b/config/crd/patches/webhook_in_autonomousdatabaserestores.yaml new file mode 100644 index 00000000..0a0ed4ad --- /dev/null +++ b/config/crd/patches/webhook_in_autonomousdatabaserestores.yaml @@ -0,0 +1,17 @@ +# The following patch enables conversion webhook for CRD +# CRD conversion requires k8s 1.13 or later. +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: autonomousdatabaserestores.database.oracle.com +spec: + conversion: + strategy: Webhook + webhookClientConfig: + # this is "\n" used as a placeholder, otherwise it will be rejected by the apiserver for being blank, + # but we're going to set it later using the cert-manager (or potentially a patch if not using cert-manager) + caBundle: Cg== + service: + namespace: system + name: webhook-service + path: /convert diff --git a/config/manager/kustomization.yaml b/config/manager/kustomization.yaml index 30ed1f75..d8632b06 100644 --- a/config/manager/kustomization.yaml +++ b/config/manager/kustomization.yaml @@ -8,5 +8,5 @@ apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization images: - name: controller - newName: container-registry.oracle.com/database/operator - newTag: 0.1.0 + newName: local-oracle-db-operator + newTag: v0.0.1beta diff --git a/config/manager/manager.yaml b/config/manager/manager.yaml index 362aac61..259b79a5 100644 --- a/config/manager/manager.yaml +++ b/config/manager/manager.yaml @@ -34,7 +34,7 @@ spec: args: - --enable-leader-election image: controller:latest - imagePullPolicy: Always + imagePullPolicy: Never #Always name: manager resources: limits: diff --git a/config/rbac/autonomousdatabaserestore_editor_role.yaml b/config/rbac/autonomousdatabaserestore_editor_role.yaml new file mode 100644 index 00000000..6efd98ae --- /dev/null +++ b/config/rbac/autonomousdatabaserestore_editor_role.yaml @@ -0,0 +1,24 @@ +# permissions for end users to edit autonomousdatabaserestores. +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: autonomousdatabaserestore-editor-role +rules: +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabaserestores + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabaserestores/status + verbs: + - get diff --git a/config/rbac/autonomousdatabaserestore_viewer_role.yaml b/config/rbac/autonomousdatabaserestore_viewer_role.yaml new file mode 100644 index 00000000..66cc7f51 --- /dev/null +++ b/config/rbac/autonomousdatabaserestore_viewer_role.yaml @@ -0,0 +1,20 @@ +# permissions for end users to view autonomousdatabaserestores. +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: autonomousdatabaserestore-viewer-role +rules: +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabaserestores + verbs: + - get + - list + - watch +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabaserestores/status + verbs: + - get diff --git a/config/rbac/role.yaml b/config/rbac/role.yaml index bef8ac3b..e1011e9d 100644 --- a/config/rbac/role.yaml +++ b/config/rbac/role.yaml @@ -127,6 +127,26 @@ rules: - get - patch - update +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabaserestores + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabaserestores/status + verbs: + - get + - patch + - update - apiGroups: - database.oracle.com resources: diff --git a/config/samples/database_v1alpha1_autonomousdatabaserestore.yaml b/config/samples/database_v1alpha1_autonomousdatabaserestore.yaml new file mode 100644 index 00000000..4e1cb9a1 --- /dev/null +++ b/config/samples/database_v1alpha1_autonomousdatabaserestore.yaml @@ -0,0 +1,7 @@ +apiVersion: database.oracle.com/v1alpha1 +kind: AutonomousDatabaseRestore +metadata: + name: autonomousdatabaserestore-sample +spec: + # Add fields here + foo: bar diff --git a/config/samples/kustomization.yaml b/config/samples/kustomization.yaml index 1a6e104a..1dd6fee0 100644 --- a/config/samples/kustomization.yaml +++ b/config/samples/kustomization.yaml @@ -10,4 +10,5 @@ resources: - singleinstancedatabase.yaml - shardingdatabase.yaml - autonomousdatabasebackup.yaml +- database_v1alpha1_autonomousdatabaserestore.yaml # +kubebuilder:scaffold:manifestskustomizesamples diff --git a/controllers/database/autonomousdatabase_controller.go b/controllers/database/autonomousdatabase_controller.go index 5d8c3258..8060147b 100644 --- a/controllers/database/autonomousdatabase_controller.go +++ b/controllers/database/autonomousdatabase_controller.go @@ -571,7 +571,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R return ctrl.Result{}, err } - r.currentLogger.Info("AutonomousDatabase resoursce reconcile successfully") + r.currentLogger.Info("AutonomousDatabase resource reconcile successfully") return ctrl.Result{}, nil } diff --git a/controllers/database/autonomousdatabasebackup_controller.go b/controllers/database/autonomousdatabasebackup_controller.go index 966182f1..fa83a823 100644 --- a/controllers/database/autonomousdatabasebackup_controller.go +++ b/controllers/database/autonomousdatabasebackup_controller.go @@ -51,8 +51,8 @@ import ( "sigs.k8s.io/controller-runtime/pkg/event" "sigs.k8s.io/controller-runtime/pkg/predicate" - "github.com/oracle/oci-go-sdk/v45/database" - "github.com/oracle/oci-go-sdk/v45/workrequests" + "github.com/oracle/oci-go-sdk/v51/database" + "github.com/oracle/oci-go-sdk/v51/workrequests" databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" backupUtil "github.com/oracle/oracle-database-operator/commons/autonomousdatabase" @@ -160,7 +160,7 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req } /****************************************************************** - * Create a backup if the AutonomousDatabaseBackupOCID is empty, + * Create a backup if the Spec.AutonomousDatabaseBackupOCID is empty, * otherwise bind to an exisiting backup ******************************************************************/ if adbBackup.Spec.AutonomousDatabaseBackupOCID == "" { diff --git a/controllers/database/autonomousdatabaserestore_controller.go b/controllers/database/autonomousdatabaserestore_controller.go new file mode 100644 index 00000000..ed9dbb8e --- /dev/null +++ b/controllers/database/autonomousdatabaserestore_controller.go @@ -0,0 +1,216 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package controllers + +import ( + "context" + "time" + + "github.com/go-logr/logr" + apiErrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/types" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" + + "github.com/oracle/oci-go-sdk/v51/database" + "github.com/oracle/oci-go-sdk/v51/workrequests" + databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + restoreUtil "github.com/oracle/oracle-database-operator/commons/autonomousdatabase" + "github.com/oracle/oracle-database-operator/commons/oci" + "github.com/oracle/oracle-database-operator/commons/oci/ociutil" +) + +// AutonomousDatabaseRestoreReconciler reconciles a AutonomousDatabaseRestore object +type AutonomousDatabaseRestoreReconciler struct { + KubeClient client.Client + Log logr.Logger + Scheme *runtime.Scheme + + currentLogger logr.Logger +} + +//+kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabaserestores,verbs=get;list;watch;create;update;patch;delete +//+kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabaserestores/status,verbs=get;update;patch + +// Reconcile is part of the main kubernetes reconciliation loop which aims to +// move the current state of the cluster closer to the desired state. +// TODO(user): Modify the Reconcile function to compare the state specified by +// the AutonomousDatabaseRestore object against the actual cluster state, and then +// perform operations to make the cluster state reflect the state specified by +// the user. +// +// For more details, check Reconcile and its Result here: +// - https://pkg.go.dev/sigs.k8s.io/controller-runtime@v0.6.4/pkg/reconcile +func (r *AutonomousDatabaseRestoreReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + r.currentLogger = r.Log.WithValues("autonomousdatabaserestore", req.NamespacedName) + + restore := &dbv1alpha1.AutonomousDatabaseRestore{} + if err := r.KubeClient.Get(context.TODO(), req.NamespacedName, restore); err != nil { + // Ignore not-found errors, since they can't be fixed by an immediate requeue. + // No need to change the since we don't know if we obtain the object. + if !apiErrors.IsNotFound(err) { + return ctrl.Result{}, err + } + } + + /****************************************************************** + * Get OCI database client and work request client + ******************************************************************/ + authData := oci.APIKeyAuth{ + ConfigMapName: restore.Spec.OCIConfig.ConfigMapName, + SecretName: restore.Spec.OCIConfig.SecretName, + Namespace: restore.GetNamespace(), + } + provider, err := oci.GetOCIProvider(r.KubeClient, authData) + if err != nil { + r.currentLogger.Error(err, "Fail to get OCI provider") + + // Change the status to UNAVAILABLE + restore.Status.LifecycleState = dbv1alpha1.RestoreLifecycleStateFailed + if statusErr := restoreUtil.UpdateAutonomousDatabaseRestoreStatus(r.KubeClient, restore); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, nil + } + + dbClient, err := database.NewDatabaseClientWithConfigurationProvider(provider) + if err != nil { + r.currentLogger.Error(err, "Fail to get OCI database client") + + // Change the status to UNAVAILABLE + restore.Status.LifecycleState = dbv1alpha1.RestoreLifecycleStateFailed + if statusErr := restoreUtil.UpdateAutonomousDatabaseRestoreStatus(r.KubeClient, restore); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, nil + } + + workClient, err := workrequests.NewWorkRequestClientWithConfigurationProvider(provider) + if err != nil { + r.currentLogger.Error(err, "Fail to get OCI work request client") + + // Change the status to UNAVAILABLE + restore.Status.LifecycleState = dbv1alpha1.RestoreLifecycleStateFailed + if statusErr := restoreUtil.UpdateAutonomousDatabaseRestoreStatus(r.KubeClient, restore); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, nil + } + + /****************************************************************** + * Restore + ******************************************************************/ + if restore.Status.LifecycleState == "" || restore.Status.LifecycleState == dbv1alpha1.RestoreLifecycleStateNew { + var restoreTime time.Time + + if restore.Spec.Source.BackupName != "" { + backup := &dbv1alpha1.AutonomousDatabaseBackup{} + namespacedName := types.NamespacedName{Namespace: restore.Namespace, Name: restore.Spec.Source.BackupName} + if err := r.KubeClient.Get(context.TODO(), namespacedName, backup); err != nil { + return ctrl.Result{}, err + } + + restoreTime, err = ociutil.ParseSDKTime(backup.Status.TimeEnded) + if err != nil { + r.currentLogger.Error(err, "Fail to parse time "+backup.Status.TimeEnded) + + // Change the status to UNAVAILABLE + restore.Status.LifecycleState = dbv1alpha1.RestoreLifecycleStateFailed + if statusErr := restoreUtil.UpdateAutonomousDatabaseRestoreStatus(r.KubeClient, restore); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, nil + } + } else if restore.Spec.Source.DateTime != "" { + restoreTime, err = ociutil.ParseSDKTime(restore.Spec.Source.DateTime) + if err != nil { + r.currentLogger.Error(err, "Fail to parse time "+restore.Spec.Source.DateTime) + + // Change the status to UNAVAILABLE + restore.Status.LifecycleState = dbv1alpha1.RestoreLifecycleStateFailed + if statusErr := restoreUtil.UpdateAutonomousDatabaseRestoreStatus(r.KubeClient, restore); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, nil + } + } + + opcID, err := oci.RestoreAutonomousDatabase(dbClient, restore.Spec.AutonomousDatabaseOCID, restoreTime) + + if err != nil { + r.currentLogger.Error(err, "Fail to get OCI work request client") + + // Change the status to UNAVAILABLE + restore.Status.LifecycleState = dbv1alpha1.RestoreLifecycleStateFailed + if statusErr := restoreUtil.UpdateAutonomousDatabaseRestoreStatus(r.KubeClient, restore); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, nil + } + + if err := oci.WaitUntilWorkCompleted(r.currentLogger, workClient, &opcID); err != nil { + r.currentLogger.Error(err, "Fail to watch workrequest. Workrequest ID = "+opcID) + + // Change the status to UNAVAILABLE + restore.Status.LifecycleState = dbv1alpha1.RestoreLifecycleStateFailed + if statusErr := restoreUtil.UpdateAutonomousDatabaseRestoreStatus(r.KubeClient, restore); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, nil + } + + r.currentLogger.Info("Restore database completed") + + restore.Status.LifecycleState = dbv1alpha1.RestoreLifecycleStateCompleted + if statusErr := restoreUtil.UpdateAutonomousDatabaseRestoreStatus(r.KubeClient, restore); statusErr != nil { + return ctrl.Result{}, statusErr + } + } + + return ctrl.Result{}, nil +} + +// SetupWithManager sets up the controller with the Manager. +func (r *AutonomousDatabaseRestoreReconciler) SetupWithManager(mgr ctrl.Manager) error { + return ctrl.NewControllerManagedBy(mgr). + For(&databasev1alpha1.AutonomousDatabaseRestore{}). + Complete(r) +} diff --git a/controllers/database/suite_test.go b/controllers/database/suite_test.go index 9b62324a..fdebedce 100644 --- a/controllers/database/suite_test.go +++ b/controllers/database/suite_test.go @@ -87,6 +87,9 @@ var _ = BeforeSuite(func(done Done) { err = databasev1alpha1.AddToScheme(scheme.Scheme) Expect(err).NotTo(HaveOccurred()) + err = databasev1alpha1.AddToScheme(scheme.Scheme) + Expect(err).NotTo(HaveOccurred()) + // +kubebuilder:scaffold:scheme k8sClient, err = client.New(cfg, client.Options{Scheme: scheme.Scheme}) diff --git a/main.go b/main.go index 8ebfa939..3049c485 100644 --- a/main.go +++ b/main.go @@ -106,6 +106,14 @@ func main() { setupLog.Error(err, "unable to create controller", "controller", "AutonomousDatabaseBackup") os.Exit(1) } + if err = (&databasecontroller.AutonomousDatabaseRestoreReconciler{ + KubeClient: mgr.GetClient(), + Log: ctrl.Log.WithName("controllers").WithName("AutonomousDatabaseRestore"), + Scheme: mgr.GetScheme(), + }).SetupWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create controller", "controller", "AutonomousDatabaseRestore") + os.Exit(1) + } if err = (&databasecontroller.SingleInstanceDatabaseReconciler{ Client: mgr.GetClient(), Log: ctrl.Log.WithName("controllers").WithName("database").WithName("SingleInstanceDatabase"), diff --git a/test/e2e/util/util.go b/test/e2e/util/util.go index 885985f1..0e6c97a7 100644 --- a/test/e2e/util/util.go +++ b/test/e2e/util/util.go @@ -73,7 +73,7 @@ func unmarshalFromYamlBytes(bytes []byte, obj interface{}) error { return json.Unmarshal(jsonBytes, obj) } -// LoadTestFixture create an AutonomousDatabase resoursce from a test fixture +// LoadTestFixture create an AutonomousDatabase resource from a test fixture func LoadTestFixture(adb *dbv1alpha1.AutonomousDatabase, filename string) (*dbv1alpha1.AutonomousDatabase, error) { filePath := "./resource/" + filename yamlBytes, err := ioutil.ReadFile(filePath) From b7c97cca03430a69e91e596907b4cf17ae516099 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Mon, 6 Dec 2021 11:07:39 -0500 Subject: [PATCH 111/628] update incorrect function name --- .../database/autonomousdatabase_controller.go | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/controllers/database/autonomousdatabase_controller.go b/controllers/database/autonomousdatabase_controller.go index 8060147b..7f364e5d 100644 --- a/controllers/database/autonomousdatabase_controller.go +++ b/controllers/database/autonomousdatabase_controller.go @@ -446,7 +446,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue @@ -455,7 +455,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R if oneWayTLSResp.OpcWorkRequestId != nil { // Update status.state adb.Status.LifecycleState = oneWayTLSResp.AutonomousDatabase.LifecycleState - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } @@ -464,7 +464,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } } @@ -477,7 +477,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue @@ -486,7 +486,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R if networkResp.OpcWorkRequestId != nil { // Update status.state adb.Status.LifecycleState = networkResp.AutonomousDatabase.LifecycleState - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } @@ -495,7 +495,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } } From eab6b3b0d3299ffe384a73e9c493172d5df41055 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Fri, 17 Dec 2021 15:04:39 -0500 Subject: [PATCH 112/628] restore imagePullPolicy --- config/manager/manager.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/config/manager/manager.yaml b/config/manager/manager.yaml index 259b79a5..362aac61 100644 --- a/config/manager/manager.yaml +++ b/config/manager/manager.yaml @@ -34,7 +34,7 @@ spec: args: - --enable-leader-election image: controller:latest - imagePullPolicy: Never #Always + imagePullPolicy: Always name: manager resources: limits: From efc7ff084c9caaacd5e6d3498a0100bb44d982f8 Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Mon, 20 Dec 2021 12:50:15 +0530 Subject: [PATCH 113/628] Added link for pv creation in oci --- docs/sidb/README.md | 168 ++++++++++++++++---------------- docs/sidb/SIDB_PREREQUISITES.md | 2 + 2 files changed, 86 insertions(+), 84 deletions(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 1c124e29..40ca9a06 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -125,103 +125,103 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI **NOTE:** Make sure you have created the required `.spec.adminPassword` [secret](https://kubernetes.io/docs/tasks/configmap-secret/managing-secret-using-kubectl/) and `.spec.persistence` [persistent volume](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) -* ### Creation Status + * ### Creation Status + + Creating a new database instance takes a while. When the 'status' status returns the response "Healthy", the Database is open for connections. - Creating a new database instance takes a while. When the 'status' status returns the response "Healthy", the Database is open for connections. - - ```sh -$ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.status}" - - Healthy -``` + ```sh + $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.status}" + + Healthy + ``` -* ### Connection Information + * ### Connection Information - External and internal (running in Kubernetes pods) clients can connect to the database using .status.connectString and .status.clusterConnectString - respectively in the following command + External and internal (running in Kubernetes pods) clients can connect to the database using .status.connectString and .status.clusterConnectString + respectively in the following command - ```sh - $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.connectString}" + ```sh + $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.connectString}" - 144.25.10.119:1521/ORCL - ``` + 144.25.10.119:1521/ORCL + ``` - The Oracle Database inside the container also has Oracle Enterprise Manager Express configured. To access OEM Express, start the browser and follow the URL: + The Oracle Database inside the container also has Oracle Enterprise Manager Express configured. To access OEM Express, start the browser and follow the URL: - ```sh - $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.oemExpressUrl}" + ```sh + $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.oemExpressUrl}" - https://144.25.10.119:5500/em - ``` + https://144.25.10.119:5500/em + ``` -* ### Update Database Config - - The following database parameters can be updated post database creation: flashBack, archiveLog, forceLog. Change their attribute values and apply using - kubectl apply or edit/patch commands . Enable archiveLog before turning ON flashBack . Turn OFF flashBack before disabling the archiveLog + * ### Update Database Config + + The following database parameters can be updated post database creation: flashBack, archiveLog, forceLog. Change their attribute values and apply using + kubectl apply or edit/patch commands . Enable archiveLog before turning ON flashBack . Turn OFF flashBack before disabling the archiveLog - ```sh - $ kubectl patch singleinstancedatabase sidb-sample --type merge -p '{"spec":{"forceLog": true}}' + ```sh + $ kubectl patch singleinstancedatabase sidb-sample --type merge -p '{"spec":{"forceLog": true}}' - singleinstancedatabase.database.oracle.com/sidb-sample patched - ``` + singleinstancedatabase.database.oracle.com/sidb-sample patched + ``` -* #### Database Config Status + * #### Database Config Status - Check the Database Config Status using the following command + Check the Database Config Status using the following command - ```sh - $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath=[{.status.archiveLog}, {.status.flashBack}, {.status.forceLog}]" + ```sh + $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath=[{.status.archiveLog}, {.status.flashBack}, {.status.forceLog}]" - [true, true, true] - ``` + [true, true, true] + ``` -* ### Update Initialization Parameters + * ### Update Initialization Parameters - The following database initialization parameters can be updated post database creation: `sgaTarget, pgaAggregateTarget, cpuCount, processes`. Change their attribute values and apply using kubectl apply or edit/patch commands. + The following database initialization parameters can be updated post database creation: `sgaTarget, pgaAggregateTarget, cpuCount, processes`. Change their attribute values and apply using kubectl apply or edit/patch commands. - **NOTE** - * `sgaTarget` should be in range [sga_min_size, sga_max_size], else initialization parameter `sga_target` would not be updated to specified `sgaTarget`. + **NOTE** + * `sgaTarget` should be in range [sga_min_size, sga_max_size], else initialization parameter `sga_target` would not be updated to specified `sgaTarget`. -* ### Multiple Replicas - - Multiple database pod replicas can be provisioned when the persistent volume access mode is ReadWriteMany. Database is open and mounted by one of the replicas. Other replicas will have instance started but not mounted and serve to provide quick cold fail-over in case the active pod dies. Update the replica attribute in the .yaml and apply using the kubectl apply command or edit/patch commands + * ### Multiple Replicas + + Multiple database pod replicas can be provisioned when the persistent volume access mode is ReadWriteMany. Database is open and mounted by one of the replicas. Other replicas will have instance started but not mounted and serve to provide quick cold fail-over in case the active pod dies. Update the replica attribute in the .yaml and apply using the kubectl apply command or edit/patch commands - Note: This functionality requires the [K8s extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/k8s) - Pre-built images from container-registry.oracle.com include the K8s extension + Note: This functionality requires the [K8s extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/k8s) + Pre-built images from container-registry.oracle.com include the K8s extension -* ### Patch Attributes + * ### Patch Attributes - The following attributes cannot be patched post SingleInstanceDatabase instance Creation : sid, edition, charset, pdbName, cloneFrom. + The following attributes cannot be patched post SingleInstanceDatabase instance Creation : sid, edition, charset, pdbName, cloneFrom. - ```sh - $ kubectl --type=merge -p '{"spec":{"sid":"ORCL1"}}' patch singleinstancedatabase sidb-sample + ```sh + $ kubectl --type=merge -p '{"spec":{"sid":"ORCL1"}}' patch singleinstancedatabase sidb-sample - The SingleInstanceDatabase "sidb-sample" is invalid: spec.sid: Forbidden: cannot be changed - ``` + The SingleInstanceDatabase "sidb-sample" is invalid: spec.sid: Forbidden: cannot be changed + ``` -* #### Patch Persistence Volume Claim + * #### Patch Persistence Volume Claim - Persistence Volume Claim (PVC) can be patched post SingleInstanceDatabase instance Creation . This will **delete all the database pods, PVC** and new database pods are created using the new PVC . + Persistence Volume Claim (PVC) can be patched post SingleInstanceDatabase instance Creation . This will **delete all the database pods, PVC** and new database pods are created using the new PVC . - ```sh - $ kubectl --type=merge -p '{"spec":{"persistence":{"accessMode":"ReadWriteMany","size":"110Gi","storageClass":""}}}' patch singleinstancedatabase sidb-sample + ```sh + $ kubectl --type=merge -p '{"spec":{"persistence":{"accessMode":"ReadWriteMany","size":"110Gi","storageClass":""}}}' patch singleinstancedatabase sidb-sample - singleinstancedatabase.database.oracle.com/sidb-sample patched - ``` + singleinstancedatabase.database.oracle.com/sidb-sample patched + ``` -* #### Patch Service + * #### Patch Service - Service can be patched post SingleInstanceDatabase instance Creation . This will **replace the Service with a new type** . - * NodePort - '{"spec":{"loadBalancer": false}}' - * LoadBalancer - '{"spec":{"loadBalancer": true }}' + Service can be patched post SingleInstanceDatabase instance Creation . This will **replace the Service with a new type** . + * NodePort - '{"spec":{"loadBalancer": false}}' + * LoadBalancer - '{"spec":{"loadBalancer": true }}' - ```sh - $ kubectl --type=merge -p '{"spec":{"loadBalancer": false}}' patch singleinstancedatabase sidb-sample + ```sh + $ kubectl --type=merge -p '{"spec":{"loadBalancer": false}}' patch singleinstancedatabase sidb-sample - singleinstancedatabase.database.oracle.com/sidb-sample patched - ``` + singleinstancedatabase.database.oracle.com/sidb-sample patched + ``` ## Clone Existing Database @@ -249,33 +249,33 @@ $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.status}" Patched Oracle Docker images can be built using this [patching extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/patching) - ```sh - kubectl --type=merge -p '{"spec":{"image":{"pullFrom":"patched-image:tag","pullSecrets":"pull-secret"}}}' patch singleinstancedatabase sidb-sample + * ### Patch existing Database - singleinstancedatabase.database.oracle.com/sidb-sample patched + Edit and apply the `singleinstancedatabase.yaml` file of the database resource/object by specifying a new release update for image attributes. The database pods will be restarted with the new release update image. For minimum downtime, ensure that you have mutiple replicas of the database pods running. + + ```sh + kubectl --type=merge -p '{"spec":{"image":{"pullFrom":"patched-image:tag","pullSecrets":"pull-secret"}}}' patch singleinstancedatabase sidb-sample - ``` + singleinstancedatabase.database.oracle.com/sidb-sample patched -* ### Patch existing Database + ``` - Edit and apply the `singleinstancedatabase.yaml` file of the database resource/object by specifying a new release update for image attributes. The database pods will be restarted with the new release update image. For minimum downtime, ensure that you have mutiple replicas of the database pods running. - -* ### Clone and Patch Database - - Clone your source database using the method of [cloning existing database](README.md#clone-existing-database) and specify a new release image for the cloned database. Use this method to enusure there are no patching related issues impacting your database performance/functionality - -* ### Datapatch status + * ### Clone and Patch Database + + Clone your source database using the method of [cloning existing database](README.md#clone-existing-database) and specify a new release image for the cloned database. Use this method to enusure there are no patching related issues impacting your database performance/functionality + + * ### Datapatch status - Patching/Rollback operations are complete when the datapatch tool completes patching or rollback of the data files. Check the data files patching status - and current release update version using the following commands + Patching/Rollback operations are complete when the datapatch tool completes patching or rollback of the data files. Check the data files patching status + and current release update version using the following commands - ```sh - $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.datafilesPatched}" + ```sh + $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.datafilesPatched}" - true - - $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.releaseUpdate}" + true + + $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.releaseUpdate}" - 19.3.0.0.0 (29517242) - ``` + 19.3.0.0.0 (29517242) + ``` diff --git a/docs/sidb/SIDB_PREREQUISITES.md b/docs/sidb/SIDB_PREREQUISITES.md index 2c13b8e2..46a42808 100644 --- a/docs/sidb/SIDB_PREREQUISITES.md +++ b/docs/sidb/SIDB_PREREQUISITES.md @@ -13,3 +13,5 @@ To deploy Oracle Database Operator for Kubernetes on Oracle Docker images, compl Set up an on-premises Kubernetes cluster, or subscribe to a managed Kubernetes service, such as Oracle Cloud Infrastructure Container Engine for Kubernetes, configured with persistent volumes. The persistent volumes are required for storage of the database files. More info on creating persistent volumes available at [https://kubernetes.io/docs/concepts/storage/persistent-volumes/](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) + + More info on creating persistent volumes on oci is available at [https://docs.oracle.com/en-us/iaas/Content/ContEng/Tasks/contengcreatingpersistentvolumeclaim.htm](https://docs.oracle.com/en-us/iaas/Content/ContEng/Tasks/contengcreatingpersistentvolumeclaim.htm) From 5144687bc21b6bd7f31532707854747cf994980a Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Mon, 20 Dec 2021 12:50:15 +0530 Subject: [PATCH 114/628] Added link for pv creation in oci --- docs/sidb/README.md | 168 ++++++++++++++++---------------- docs/sidb/SIDB_PREREQUISITES.md | 2 + 2 files changed, 86 insertions(+), 84 deletions(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 1c124e29..40ca9a06 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -125,103 +125,103 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI **NOTE:** Make sure you have created the required `.spec.adminPassword` [secret](https://kubernetes.io/docs/tasks/configmap-secret/managing-secret-using-kubectl/) and `.spec.persistence` [persistent volume](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) -* ### Creation Status + * ### Creation Status + + Creating a new database instance takes a while. When the 'status' status returns the response "Healthy", the Database is open for connections. - Creating a new database instance takes a while. When the 'status' status returns the response "Healthy", the Database is open for connections. - - ```sh -$ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.status}" - - Healthy -``` + ```sh + $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.status}" + + Healthy + ``` -* ### Connection Information + * ### Connection Information - External and internal (running in Kubernetes pods) clients can connect to the database using .status.connectString and .status.clusterConnectString - respectively in the following command + External and internal (running in Kubernetes pods) clients can connect to the database using .status.connectString and .status.clusterConnectString + respectively in the following command - ```sh - $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.connectString}" + ```sh + $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.connectString}" - 144.25.10.119:1521/ORCL - ``` + 144.25.10.119:1521/ORCL + ``` - The Oracle Database inside the container also has Oracle Enterprise Manager Express configured. To access OEM Express, start the browser and follow the URL: + The Oracle Database inside the container also has Oracle Enterprise Manager Express configured. To access OEM Express, start the browser and follow the URL: - ```sh - $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.oemExpressUrl}" + ```sh + $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.oemExpressUrl}" - https://144.25.10.119:5500/em - ``` + https://144.25.10.119:5500/em + ``` -* ### Update Database Config - - The following database parameters can be updated post database creation: flashBack, archiveLog, forceLog. Change their attribute values and apply using - kubectl apply or edit/patch commands . Enable archiveLog before turning ON flashBack . Turn OFF flashBack before disabling the archiveLog + * ### Update Database Config + + The following database parameters can be updated post database creation: flashBack, archiveLog, forceLog. Change their attribute values and apply using + kubectl apply or edit/patch commands . Enable archiveLog before turning ON flashBack . Turn OFF flashBack before disabling the archiveLog - ```sh - $ kubectl patch singleinstancedatabase sidb-sample --type merge -p '{"spec":{"forceLog": true}}' + ```sh + $ kubectl patch singleinstancedatabase sidb-sample --type merge -p '{"spec":{"forceLog": true}}' - singleinstancedatabase.database.oracle.com/sidb-sample patched - ``` + singleinstancedatabase.database.oracle.com/sidb-sample patched + ``` -* #### Database Config Status + * #### Database Config Status - Check the Database Config Status using the following command + Check the Database Config Status using the following command - ```sh - $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath=[{.status.archiveLog}, {.status.flashBack}, {.status.forceLog}]" + ```sh + $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath=[{.status.archiveLog}, {.status.flashBack}, {.status.forceLog}]" - [true, true, true] - ``` + [true, true, true] + ``` -* ### Update Initialization Parameters + * ### Update Initialization Parameters - The following database initialization parameters can be updated post database creation: `sgaTarget, pgaAggregateTarget, cpuCount, processes`. Change their attribute values and apply using kubectl apply or edit/patch commands. + The following database initialization parameters can be updated post database creation: `sgaTarget, pgaAggregateTarget, cpuCount, processes`. Change their attribute values and apply using kubectl apply or edit/patch commands. - **NOTE** - * `sgaTarget` should be in range [sga_min_size, sga_max_size], else initialization parameter `sga_target` would not be updated to specified `sgaTarget`. + **NOTE** + * `sgaTarget` should be in range [sga_min_size, sga_max_size], else initialization parameter `sga_target` would not be updated to specified `sgaTarget`. -* ### Multiple Replicas - - Multiple database pod replicas can be provisioned when the persistent volume access mode is ReadWriteMany. Database is open and mounted by one of the replicas. Other replicas will have instance started but not mounted and serve to provide quick cold fail-over in case the active pod dies. Update the replica attribute in the .yaml and apply using the kubectl apply command or edit/patch commands + * ### Multiple Replicas + + Multiple database pod replicas can be provisioned when the persistent volume access mode is ReadWriteMany. Database is open and mounted by one of the replicas. Other replicas will have instance started but not mounted and serve to provide quick cold fail-over in case the active pod dies. Update the replica attribute in the .yaml and apply using the kubectl apply command or edit/patch commands - Note: This functionality requires the [K8s extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/k8s) - Pre-built images from container-registry.oracle.com include the K8s extension + Note: This functionality requires the [K8s extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/k8s) + Pre-built images from container-registry.oracle.com include the K8s extension -* ### Patch Attributes + * ### Patch Attributes - The following attributes cannot be patched post SingleInstanceDatabase instance Creation : sid, edition, charset, pdbName, cloneFrom. + The following attributes cannot be patched post SingleInstanceDatabase instance Creation : sid, edition, charset, pdbName, cloneFrom. - ```sh - $ kubectl --type=merge -p '{"spec":{"sid":"ORCL1"}}' patch singleinstancedatabase sidb-sample + ```sh + $ kubectl --type=merge -p '{"spec":{"sid":"ORCL1"}}' patch singleinstancedatabase sidb-sample - The SingleInstanceDatabase "sidb-sample" is invalid: spec.sid: Forbidden: cannot be changed - ``` + The SingleInstanceDatabase "sidb-sample" is invalid: spec.sid: Forbidden: cannot be changed + ``` -* #### Patch Persistence Volume Claim + * #### Patch Persistence Volume Claim - Persistence Volume Claim (PVC) can be patched post SingleInstanceDatabase instance Creation . This will **delete all the database pods, PVC** and new database pods are created using the new PVC . + Persistence Volume Claim (PVC) can be patched post SingleInstanceDatabase instance Creation . This will **delete all the database pods, PVC** and new database pods are created using the new PVC . - ```sh - $ kubectl --type=merge -p '{"spec":{"persistence":{"accessMode":"ReadWriteMany","size":"110Gi","storageClass":""}}}' patch singleinstancedatabase sidb-sample + ```sh + $ kubectl --type=merge -p '{"spec":{"persistence":{"accessMode":"ReadWriteMany","size":"110Gi","storageClass":""}}}' patch singleinstancedatabase sidb-sample - singleinstancedatabase.database.oracle.com/sidb-sample patched - ``` + singleinstancedatabase.database.oracle.com/sidb-sample patched + ``` -* #### Patch Service + * #### Patch Service - Service can be patched post SingleInstanceDatabase instance Creation . This will **replace the Service with a new type** . - * NodePort - '{"spec":{"loadBalancer": false}}' - * LoadBalancer - '{"spec":{"loadBalancer": true }}' + Service can be patched post SingleInstanceDatabase instance Creation . This will **replace the Service with a new type** . + * NodePort - '{"spec":{"loadBalancer": false}}' + * LoadBalancer - '{"spec":{"loadBalancer": true }}' - ```sh - $ kubectl --type=merge -p '{"spec":{"loadBalancer": false}}' patch singleinstancedatabase sidb-sample + ```sh + $ kubectl --type=merge -p '{"spec":{"loadBalancer": false}}' patch singleinstancedatabase sidb-sample - singleinstancedatabase.database.oracle.com/sidb-sample patched - ``` + singleinstancedatabase.database.oracle.com/sidb-sample patched + ``` ## Clone Existing Database @@ -249,33 +249,33 @@ $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.status}" Patched Oracle Docker images can be built using this [patching extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/patching) - ```sh - kubectl --type=merge -p '{"spec":{"image":{"pullFrom":"patched-image:tag","pullSecrets":"pull-secret"}}}' patch singleinstancedatabase sidb-sample + * ### Patch existing Database - singleinstancedatabase.database.oracle.com/sidb-sample patched + Edit and apply the `singleinstancedatabase.yaml` file of the database resource/object by specifying a new release update for image attributes. The database pods will be restarted with the new release update image. For minimum downtime, ensure that you have mutiple replicas of the database pods running. + + ```sh + kubectl --type=merge -p '{"spec":{"image":{"pullFrom":"patched-image:tag","pullSecrets":"pull-secret"}}}' patch singleinstancedatabase sidb-sample - ``` + singleinstancedatabase.database.oracle.com/sidb-sample patched -* ### Patch existing Database + ``` - Edit and apply the `singleinstancedatabase.yaml` file of the database resource/object by specifying a new release update for image attributes. The database pods will be restarted with the new release update image. For minimum downtime, ensure that you have mutiple replicas of the database pods running. - -* ### Clone and Patch Database - - Clone your source database using the method of [cloning existing database](README.md#clone-existing-database) and specify a new release image for the cloned database. Use this method to enusure there are no patching related issues impacting your database performance/functionality - -* ### Datapatch status + * ### Clone and Patch Database + + Clone your source database using the method of [cloning existing database](README.md#clone-existing-database) and specify a new release image for the cloned database. Use this method to enusure there are no patching related issues impacting your database performance/functionality + + * ### Datapatch status - Patching/Rollback operations are complete when the datapatch tool completes patching or rollback of the data files. Check the data files patching status - and current release update version using the following commands + Patching/Rollback operations are complete when the datapatch tool completes patching or rollback of the data files. Check the data files patching status + and current release update version using the following commands - ```sh - $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.datafilesPatched}" + ```sh + $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.datafilesPatched}" - true - - $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.releaseUpdate}" + true + + $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.releaseUpdate}" - 19.3.0.0.0 (29517242) - ``` + 19.3.0.0.0 (29517242) + ``` diff --git a/docs/sidb/SIDB_PREREQUISITES.md b/docs/sidb/SIDB_PREREQUISITES.md index 2c13b8e2..46a42808 100644 --- a/docs/sidb/SIDB_PREREQUISITES.md +++ b/docs/sidb/SIDB_PREREQUISITES.md @@ -13,3 +13,5 @@ To deploy Oracle Database Operator for Kubernetes on Oracle Docker images, compl Set up an on-premises Kubernetes cluster, or subscribe to a managed Kubernetes service, such as Oracle Cloud Infrastructure Container Engine for Kubernetes, configured with persistent volumes. The persistent volumes are required for storage of the database files. More info on creating persistent volumes available at [https://kubernetes.io/docs/concepts/storage/persistent-volumes/](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) + + More info on creating persistent volumes on oci is available at [https://docs.oracle.com/en-us/iaas/Content/ContEng/Tasks/contengcreatingpersistentvolumeclaim.htm](https://docs.oracle.com/en-us/iaas/Content/ContEng/Tasks/contengcreatingpersistentvolumeclaim.htm) From dd53a97b9d182ac508d7d021e325309013d9621e Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Mon, 20 Dec 2021 12:52:36 +0530 Subject: [PATCH 115/628] Added link for pv creation in oci --- docs/sidb/README.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 40ca9a06..c3bdd997 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -127,13 +127,13 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI * ### Creation Status - Creating a new database instance takes a while. When the 'status' status returns the response "Healthy", the Database is open for connections. - - ```sh - $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.status}" + Creating a new database instance takes a while. When the 'status' status returns the response "Healthy", the Database is open for connections. - Healthy - ``` + ```sh + $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.status}" + + Healthy + ``` From c062b5efc83896a593e000d175bec6a634e0ccb8 Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Mon, 20 Dec 2021 12:52:36 +0530 Subject: [PATCH 116/628] Added link for pv creation in oci --- docs/sidb/README.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 40ca9a06..c3bdd997 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -127,13 +127,13 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI * ### Creation Status - Creating a new database instance takes a while. When the 'status' status returns the response "Healthy", the Database is open for connections. - - ```sh - $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.status}" + Creating a new database instance takes a while. When the 'status' status returns the response "Healthy", the Database is open for connections. - Healthy - ``` + ```sh + $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.status}" + + Healthy + ``` From 16e9a1e0311daeaf6814ff7d80909a021cc85c62 Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Mon, 20 Dec 2021 14:51:08 +0530 Subject: [PATCH 117/628] Added command to create image pull secret --- config/samples/sidb/singleinstancedatabase_minikube.yaml | 4 ++-- docs/sidb/README.md | 5 ++++- docs/sidb/SIDB_PREREQUISITES.md | 2 +- 3 files changed, 7 insertions(+), 4 deletions(-) diff --git a/config/samples/sidb/singleinstancedatabase_minikube.yaml b/config/samples/sidb/singleinstancedatabase_minikube.yaml index 749b8bc8..f4a200f8 100644 --- a/config/samples/sidb/singleinstancedatabase_minikube.yaml +++ b/config/samples/sidb/singleinstancedatabase_minikube.yaml @@ -80,8 +80,8 @@ spec: ## Database can be patched by updating the RU version/image ## Major version changes are not supported image: - pullFrom: - pullSecrets: + pullFrom: container-registry.oracle.com/database/enterprise:latest + pullSecrets: oracle-container-registry-secret ## size : Minimum size of pvc | class : PVC storage Class ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany diff --git a/docs/sidb/README.md b/docs/sidb/README.md index c3bdd997..7194c9f9 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -109,6 +109,7 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI - Easily provision a new database instance on **minikube** using [singleinstancedatabase_minikube.yaml](../../config/samples/sidb/singleinstancedatabase_minikube.yaml) by the following one command. ```sh + $ kubectl create secret docker-registry oracle-container-registry-secret --docker-server=container-registry.oracle.com --docker-username='' --docker-password='' --docker-email='' $ kubectl create -f singleinstancedatabase_minikube.yaml singleinstancedatabase.database.oracle.com/sidb-sample created @@ -251,7 +252,7 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI * ### Patch existing Database - Edit and apply the `singleinstancedatabase.yaml` file of the database resource/object by specifying a new release update for image attributes. The database pods will be restarted with the new release update image. For minimum downtime, ensure that you have mutiple replicas of the database pods running. + Edit and apply the `singleinstancedatabase.yaml` file of the database resource/object by specifying a new release update for image attributes. ```sh kubectl --type=merge -p '{"spec":{"image":{"pullFrom":"patched-image:tag","pullSecrets":"pull-secret"}}}' patch singleinstancedatabase sidb-sample @@ -260,6 +261,8 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI ``` + The database pods will be restarted with the new release update image. For minimum downtime, ensure that you have mutiple replicas of the database pods running. + * ### Clone and Patch Database Clone your source database using the method of [cloning existing database](README.md#clone-existing-database) and specify a new release image for the cloned database. Use this method to enusure there are no patching related issues impacting your database performance/functionality diff --git a/docs/sidb/SIDB_PREREQUISITES.md b/docs/sidb/SIDB_PREREQUISITES.md index 46a42808..b17ff7ea 100644 --- a/docs/sidb/SIDB_PREREQUISITES.md +++ b/docs/sidb/SIDB_PREREQUISITES.md @@ -4,7 +4,7 @@ To deploy Oracle Database Operator for Kubernetes on Oracle Docker images, compl * ### Prepare Oracle Docker Images Build SingleInstanceDatabase Docker Images from source, following the instructions at [https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance), or - use the pre-built images available at [https://container-registry.oracle.com](https://container-registry.oracle.com) + use the pre-built images available at [https://container-registry.oracle.com](https://container-registry.oracle.com) by signing in and accepting the required license agreement. Oracle Database Releases Supported: Oracle Database 19c Enterprise Edition or Standard Edition, and later releases. From 5632275c8a6218f535aae71d7377820cdbe1a1c7 Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Mon, 20 Dec 2021 14:51:08 +0530 Subject: [PATCH 118/628] Added command to create image pull secret --- config/samples/sidb/singleinstancedatabase_minikube.yaml | 4 ++-- docs/sidb/README.md | 5 ++++- docs/sidb/SIDB_PREREQUISITES.md | 2 +- 3 files changed, 7 insertions(+), 4 deletions(-) diff --git a/config/samples/sidb/singleinstancedatabase_minikube.yaml b/config/samples/sidb/singleinstancedatabase_minikube.yaml index 749b8bc8..f4a200f8 100644 --- a/config/samples/sidb/singleinstancedatabase_minikube.yaml +++ b/config/samples/sidb/singleinstancedatabase_minikube.yaml @@ -80,8 +80,8 @@ spec: ## Database can be patched by updating the RU version/image ## Major version changes are not supported image: - pullFrom: - pullSecrets: + pullFrom: container-registry.oracle.com/database/enterprise:latest + pullSecrets: oracle-container-registry-secret ## size : Minimum size of pvc | class : PVC storage Class ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany diff --git a/docs/sidb/README.md b/docs/sidb/README.md index c3bdd997..7194c9f9 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -109,6 +109,7 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI - Easily provision a new database instance on **minikube** using [singleinstancedatabase_minikube.yaml](../../config/samples/sidb/singleinstancedatabase_minikube.yaml) by the following one command. ```sh + $ kubectl create secret docker-registry oracle-container-registry-secret --docker-server=container-registry.oracle.com --docker-username='' --docker-password='' --docker-email='' $ kubectl create -f singleinstancedatabase_minikube.yaml singleinstancedatabase.database.oracle.com/sidb-sample created @@ -251,7 +252,7 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI * ### Patch existing Database - Edit and apply the `singleinstancedatabase.yaml` file of the database resource/object by specifying a new release update for image attributes. The database pods will be restarted with the new release update image. For minimum downtime, ensure that you have mutiple replicas of the database pods running. + Edit and apply the `singleinstancedatabase.yaml` file of the database resource/object by specifying a new release update for image attributes. ```sh kubectl --type=merge -p '{"spec":{"image":{"pullFrom":"patched-image:tag","pullSecrets":"pull-secret"}}}' patch singleinstancedatabase sidb-sample @@ -260,6 +261,8 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI ``` + The database pods will be restarted with the new release update image. For minimum downtime, ensure that you have mutiple replicas of the database pods running. + * ### Clone and Patch Database Clone your source database using the method of [cloning existing database](README.md#clone-existing-database) and specify a new release image for the cloned database. Use this method to enusure there are no patching related issues impacting your database performance/functionality diff --git a/docs/sidb/SIDB_PREREQUISITES.md b/docs/sidb/SIDB_PREREQUISITES.md index 46a42808..b17ff7ea 100644 --- a/docs/sidb/SIDB_PREREQUISITES.md +++ b/docs/sidb/SIDB_PREREQUISITES.md @@ -4,7 +4,7 @@ To deploy Oracle Database Operator for Kubernetes on Oracle Docker images, compl * ### Prepare Oracle Docker Images Build SingleInstanceDatabase Docker Images from source, following the instructions at [https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance), or - use the pre-built images available at [https://container-registry.oracle.com](https://container-registry.oracle.com) + use the pre-built images available at [https://container-registry.oracle.com](https://container-registry.oracle.com) by signing in and accepting the required license agreement. Oracle Database Releases Supported: Oracle Database 19c Enterprise Edition or Standard Edition, and later releases. From d36a406ed24203831190978d2c6480e49d18f054 Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Mon, 20 Dec 2021 14:55:28 +0530 Subject: [PATCH 119/628] Added command to create image pull secret --- docs/sidb/README.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 7194c9f9..b0b680fe 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -110,6 +110,9 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI ```sh $ kubectl create secret docker-registry oracle-container-registry-secret --docker-server=container-registry.oracle.com --docker-username='' --docker-password='' --docker-email='' + + secret/oracle-container-registry-secret created + $ kubectl create -f singleinstancedatabase_minikube.yaml singleinstancedatabase.database.oracle.com/sidb-sample created From 0825eb9c3ed9a6eb45eb848e1c4b905dfbcccfa3 Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Mon, 20 Dec 2021 14:55:28 +0530 Subject: [PATCH 120/628] Added command to create image pull secret --- docs/sidb/README.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 7194c9f9..b0b680fe 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -110,6 +110,9 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI ```sh $ kubectl create secret docker-registry oracle-container-registry-secret --docker-server=container-registry.oracle.com --docker-username='' --docker-password='' --docker-email='' + + secret/oracle-container-registry-secret created + $ kubectl create -f singleinstancedatabase_minikube.yaml singleinstancedatabase.database.oracle.com/sidb-sample created From c75af5536f07ba632c0fa08cbec1b25b19164e1a Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Mon, 20 Dec 2021 15:01:30 +0530 Subject: [PATCH 121/628] Added command to create image pull secret --- docs/sidb/README.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index b0b680fe..d2504cff 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -106,13 +106,21 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI ## Provision New Database - - Easily provision a new database instance on **minikube** using [singleinstancedatabase_minikube.yaml](../../config/samples/sidb/singleinstancedatabase_minikube.yaml) by the following one command. + - Easily provision a new database instance on **minikube** using [singleinstancedatabase_minikube.yaml](../../config/samples/sidb/singleinstancedatabase_minikube.yaml). + + Sign into [Oracle Container Registry](https://container-registry.oracle.com/ords/f?p=113:4:7154182141811:::4:P4_REPOSITORY,AI_REPOSITORY,AI_REPOSITORY_NAME,P4_REPOSITORY_NAME,P4_EULA_ID,P4_BUSINESS_AREA_ID:9,9,Oracle%20Database%20Enterprise%20Edition,Oracle%20Database%20Enterprise%20Edition,1,0&cs=3Y_90hkCQLfJzrvTLiEipIGgWGUytfrtAPuHFocuWd0NDSacbBPlamohfLuiJA-bAsVL6Z_yKEMsTbb52bm6IRA) and accept the license agreement for the Database image, ignore if accepted already. + + Create an image pull secret for Oracle Container Registry, ignore if you have created already: ```sh $ kubectl create secret docker-registry oracle-container-registry-secret --docker-server=container-registry.oracle.com --docker-username='' --docker-password='' --docker-email='' secret/oracle-container-registry-secret created + ``` + Now, Easily provision a new database instance on minikube by using following one command. + + ```sh $ kubectl create -f singleinstancedatabase_minikube.yaml singleinstancedatabase.database.oracle.com/sidb-sample created From 4817912c9905cf985cd60ba0f2537f7ec05eb918 Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Mon, 20 Dec 2021 15:01:30 +0530 Subject: [PATCH 122/628] Added command to create image pull secret --- docs/sidb/README.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index b0b680fe..d2504cff 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -106,13 +106,21 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI ## Provision New Database - - Easily provision a new database instance on **minikube** using [singleinstancedatabase_minikube.yaml](../../config/samples/sidb/singleinstancedatabase_minikube.yaml) by the following one command. + - Easily provision a new database instance on **minikube** using [singleinstancedatabase_minikube.yaml](../../config/samples/sidb/singleinstancedatabase_minikube.yaml). + + Sign into [Oracle Container Registry](https://container-registry.oracle.com/ords/f?p=113:4:7154182141811:::4:P4_REPOSITORY,AI_REPOSITORY,AI_REPOSITORY_NAME,P4_REPOSITORY_NAME,P4_EULA_ID,P4_BUSINESS_AREA_ID:9,9,Oracle%20Database%20Enterprise%20Edition,Oracle%20Database%20Enterprise%20Edition,1,0&cs=3Y_90hkCQLfJzrvTLiEipIGgWGUytfrtAPuHFocuWd0NDSacbBPlamohfLuiJA-bAsVL6Z_yKEMsTbb52bm6IRA) and accept the license agreement for the Database image, ignore if accepted already. + + Create an image pull secret for Oracle Container Registry, ignore if you have created already: ```sh $ kubectl create secret docker-registry oracle-container-registry-secret --docker-server=container-registry.oracle.com --docker-username='' --docker-password='' --docker-email='' secret/oracle-container-registry-secret created + ``` + Now, Easily provision a new database instance on minikube by using following one command. + + ```sh $ kubectl create -f singleinstancedatabase_minikube.yaml singleinstancedatabase.database.oracle.com/sidb-sample created From 84ff900f5df4096f0cdca85eaf5b0a25fad263b5 Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Mon, 20 Dec 2021 15:03:32 +0530 Subject: [PATCH 123/628] Added command to create image pull secret --- docs/sidb/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index d2504cff..f87b16e7 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -108,7 +108,7 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI - Easily provision a new database instance on **minikube** using [singleinstancedatabase_minikube.yaml](../../config/samples/sidb/singleinstancedatabase_minikube.yaml). - Sign into [Oracle Container Registry](https://container-registry.oracle.com/ords/f?p=113:4:7154182141811:::4:P4_REPOSITORY,AI_REPOSITORY,AI_REPOSITORY_NAME,P4_REPOSITORY_NAME,P4_EULA_ID,P4_BUSINESS_AREA_ID:9,9,Oracle%20Database%20Enterprise%20Edition,Oracle%20Database%20Enterprise%20Edition,1,0&cs=3Y_90hkCQLfJzrvTLiEipIGgWGUytfrtAPuHFocuWd0NDSacbBPlamohfLuiJA-bAsVL6Z_yKEMsTbb52bm6IRA) and accept the license agreement for the Database image, ignore if accepted already. + Sign into [Oracle Container Registry](https://container-registry.oracle.com/ords/f?p=113:4:7154182141811:::4:P4_REPOSITORY,AI_REPOSITORY,AI_REPOSITORY_NAME,P4_REPOSITORY_NAME,P4_EULA_ID,P4_BUSINESS_AREA_ID:9,9,Oracle%20Database%20Enterprise%20Edition,Oracle%20Database%20Enterprise%20Edition,1,0&cs=3Y_90hkCQLfJzrvTLiEipIGgWGUytfrtAPuHFocuWd0NDSacbBPlamohfLuiJA-bAsVL6Z_yKEMsTbb52bm6IRA) and accept the license agreement for the Database image, ignore if you have accepted already. Create an image pull secret for Oracle Container Registry, ignore if you have created already: From 35a7e6f73b63b716d5159630f6165c8b088568f8 Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Mon, 20 Dec 2021 15:03:32 +0530 Subject: [PATCH 124/628] Added command to create image pull secret --- docs/sidb/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index d2504cff..f87b16e7 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -108,7 +108,7 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI - Easily provision a new database instance on **minikube** using [singleinstancedatabase_minikube.yaml](../../config/samples/sidb/singleinstancedatabase_minikube.yaml). - Sign into [Oracle Container Registry](https://container-registry.oracle.com/ords/f?p=113:4:7154182141811:::4:P4_REPOSITORY,AI_REPOSITORY,AI_REPOSITORY_NAME,P4_REPOSITORY_NAME,P4_EULA_ID,P4_BUSINESS_AREA_ID:9,9,Oracle%20Database%20Enterprise%20Edition,Oracle%20Database%20Enterprise%20Edition,1,0&cs=3Y_90hkCQLfJzrvTLiEipIGgWGUytfrtAPuHFocuWd0NDSacbBPlamohfLuiJA-bAsVL6Z_yKEMsTbb52bm6IRA) and accept the license agreement for the Database image, ignore if accepted already. + Sign into [Oracle Container Registry](https://container-registry.oracle.com/ords/f?p=113:4:7154182141811:::4:P4_REPOSITORY,AI_REPOSITORY,AI_REPOSITORY_NAME,P4_REPOSITORY_NAME,P4_EULA_ID,P4_BUSINESS_AREA_ID:9,9,Oracle%20Database%20Enterprise%20Edition,Oracle%20Database%20Enterprise%20Edition,1,0&cs=3Y_90hkCQLfJzrvTLiEipIGgWGUytfrtAPuHFocuWd0NDSacbBPlamohfLuiJA-bAsVL6Z_yKEMsTbb52bm6IRA) and accept the license agreement for the Database image, ignore if you have accepted already. Create an image pull secret for Oracle Container Registry, ignore if you have created already: From eb866da9703aa7b356541d314b2cfe66fd37f5ff Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Mon, 20 Dec 2021 15:05:06 +0530 Subject: [PATCH 125/628] Added command to create image pull secret --- docs/sidb/SIDB_PREREQUISITES.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/sidb/SIDB_PREREQUISITES.md b/docs/sidb/SIDB_PREREQUISITES.md index b17ff7ea..d56c801c 100644 --- a/docs/sidb/SIDB_PREREQUISITES.md +++ b/docs/sidb/SIDB_PREREQUISITES.md @@ -14,4 +14,4 @@ To deploy Oracle Database Operator for Kubernetes on Oracle Docker images, compl More info on creating persistent volumes available at [https://kubernetes.io/docs/concepts/storage/persistent-volumes/](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) - More info on creating persistent volumes on oci is available at [https://docs.oracle.com/en-us/iaas/Content/ContEng/Tasks/contengcreatingpersistentvolumeclaim.htm](https://docs.oracle.com/en-us/iaas/Content/ContEng/Tasks/contengcreatingpersistentvolumeclaim.htm) + More info on creating persistent volumes on OCI is available at [https://docs.oracle.com/en-us/iaas/Content/ContEng/Tasks/contengcreatingpersistentvolumeclaim.htm](https://docs.oracle.com/en-us/iaas/Content/ContEng/Tasks/contengcreatingpersistentvolumeclaim.htm) From d571189715fc57329b8b9bbca0ae35a77f9a7187 Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Mon, 20 Dec 2021 15:05:06 +0530 Subject: [PATCH 126/628] Added command to create image pull secret --- docs/sidb/SIDB_PREREQUISITES.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/sidb/SIDB_PREREQUISITES.md b/docs/sidb/SIDB_PREREQUISITES.md index b17ff7ea..d56c801c 100644 --- a/docs/sidb/SIDB_PREREQUISITES.md +++ b/docs/sidb/SIDB_PREREQUISITES.md @@ -14,4 +14,4 @@ To deploy Oracle Database Operator for Kubernetes on Oracle Docker images, compl More info on creating persistent volumes available at [https://kubernetes.io/docs/concepts/storage/persistent-volumes/](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) - More info on creating persistent volumes on oci is available at [https://docs.oracle.com/en-us/iaas/Content/ContEng/Tasks/contengcreatingpersistentvolumeclaim.htm](https://docs.oracle.com/en-us/iaas/Content/ContEng/Tasks/contengcreatingpersistentvolumeclaim.htm) + More info on creating persistent volumes on OCI is available at [https://docs.oracle.com/en-us/iaas/Content/ContEng/Tasks/contengcreatingpersistentvolumeclaim.htm](https://docs.oracle.com/en-us/iaas/Content/ContEng/Tasks/contengcreatingpersistentvolumeclaim.htm) From 4b265dfb641c69262df5875f36cf6f218d9088ca Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Mon, 20 Dec 2021 15:23:10 +0530 Subject: [PATCH 127/628] Added command to create image pull secret --- docs/sidb/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index f87b16e7..7500e212 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -263,7 +263,7 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI * ### Patch existing Database - Edit and apply the `singleinstancedatabase.yaml` file of the database resource/object by specifying a new release update for image attributes. + Edit and apply the `singleinstancedatabase.yaml` file of the database resource/object by specifying a new release update for image attributes or run the following command. ```sh kubectl --type=merge -p '{"spec":{"image":{"pullFrom":"patched-image:tag","pullSecrets":"pull-secret"}}}' patch singleinstancedatabase sidb-sample From 0d97ed2867f3c1ec4614cf9f37338cb04a4648a5 Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Mon, 20 Dec 2021 15:23:10 +0530 Subject: [PATCH 128/628] Added command to create image pull secret --- docs/sidb/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index f87b16e7..7500e212 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -263,7 +263,7 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI * ### Patch existing Database - Edit and apply the `singleinstancedatabase.yaml` file of the database resource/object by specifying a new release update for image attributes. + Edit and apply the `singleinstancedatabase.yaml` file of the database resource/object by specifying a new release update for image attributes or run the following command. ```sh kubectl --type=merge -p '{"spec":{"image":{"pullFrom":"patched-image:tag","pullSecrets":"pull-secret"}}}' patch singleinstancedatabase sidb-sample From 6aecfe0e3e3a906b0d611fd0de90396e8797c086 Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Wed, 22 Dec 2021 10:51:16 +0530 Subject: [PATCH 129/628] Defaulted keepSecret to true --- config/samples/sidb/singleinstancedatabase.yaml | 2 +- config/samples/sidb/singleinstancedatabase_clone.yaml | 2 +- config/samples/sidb/singleinstancedatabase_minikube.yaml | 2 +- config/samples/sidb/singleinstancedatabase_patch.yaml | 2 +- config/samples/sidb/singleinstancedatabase_prov.yaml | 2 +- 5 files changed, 5 insertions(+), 5 deletions(-) diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index 99e2ee3a..e39e3208 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -24,7 +24,7 @@ spec: adminPassword: secretName: secretKey: - keepSecret: false + keepSecret: true ## NA if cloning from a SourceDB (cloneFrom is set) charset: AL32UTF8 diff --git a/config/samples/sidb/singleinstancedatabase_clone.yaml b/config/samples/sidb/singleinstancedatabase_clone.yaml index a6d8237b..2881b4f9 100644 --- a/config/samples/sidb/singleinstancedatabase_clone.yaml +++ b/config/samples/sidb/singleinstancedatabase_clone.yaml @@ -21,7 +21,7 @@ spec: adminPassword: secretName: secretKey: - keepSecret: false + keepSecret: true ## Database image details ## Database can be patched out of place by updating the RU version/image diff --git a/config/samples/sidb/singleinstancedatabase_minikube.yaml b/config/samples/sidb/singleinstancedatabase_minikube.yaml index f4a200f8..a3a4824c 100644 --- a/config/samples/sidb/singleinstancedatabase_minikube.yaml +++ b/config/samples/sidb/singleinstancedatabase_minikube.yaml @@ -50,7 +50,7 @@ spec: adminPassword: secretName: db-secret secretKey: oracle_pwd - keepSecret: false + keepSecret: true ## NA if cloning from a SourceDB (cloneFrom is set) charset: AL32UTF8 diff --git a/config/samples/sidb/singleinstancedatabase_patch.yaml b/config/samples/sidb/singleinstancedatabase_patch.yaml index fb55e6b2..8fb0ddb6 100644 --- a/config/samples/sidb/singleinstancedatabase_patch.yaml +++ b/config/samples/sidb/singleinstancedatabase_patch.yaml @@ -17,7 +17,7 @@ spec: adminPassword: secretName: secretKey: - keepSecret: false + keepSecret: true ## Patch the database by updating the RU version/image ## Major version changes are not supported diff --git a/config/samples/sidb/singleinstancedatabase_prov.yaml b/config/samples/sidb/singleinstancedatabase_prov.yaml index 6e6afde0..dbee637c 100644 --- a/config/samples/sidb/singleinstancedatabase_prov.yaml +++ b/config/samples/sidb/singleinstancedatabase_prov.yaml @@ -16,7 +16,7 @@ spec: adminPassword: secretName: secretKey: - keepSecret: false + keepSecret: true ## Database image details image: From f502da94c08b3f0b38b37ed84692a2885078fa74 Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Wed, 22 Dec 2021 10:51:16 +0530 Subject: [PATCH 130/628] Defaulted keepSecret to true --- config/samples/sidb/singleinstancedatabase.yaml | 2 +- config/samples/sidb/singleinstancedatabase_clone.yaml | 2 +- config/samples/sidb/singleinstancedatabase_minikube.yaml | 2 +- config/samples/sidb/singleinstancedatabase_patch.yaml | 2 +- config/samples/sidb/singleinstancedatabase_prov.yaml | 2 +- 5 files changed, 5 insertions(+), 5 deletions(-) diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index 99e2ee3a..e39e3208 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -24,7 +24,7 @@ spec: adminPassword: secretName: secretKey: - keepSecret: false + keepSecret: true ## NA if cloning from a SourceDB (cloneFrom is set) charset: AL32UTF8 diff --git a/config/samples/sidb/singleinstancedatabase_clone.yaml b/config/samples/sidb/singleinstancedatabase_clone.yaml index a6d8237b..2881b4f9 100644 --- a/config/samples/sidb/singleinstancedatabase_clone.yaml +++ b/config/samples/sidb/singleinstancedatabase_clone.yaml @@ -21,7 +21,7 @@ spec: adminPassword: secretName: secretKey: - keepSecret: false + keepSecret: true ## Database image details ## Database can be patched out of place by updating the RU version/image diff --git a/config/samples/sidb/singleinstancedatabase_minikube.yaml b/config/samples/sidb/singleinstancedatabase_minikube.yaml index f4a200f8..a3a4824c 100644 --- a/config/samples/sidb/singleinstancedatabase_minikube.yaml +++ b/config/samples/sidb/singleinstancedatabase_minikube.yaml @@ -50,7 +50,7 @@ spec: adminPassword: secretName: db-secret secretKey: oracle_pwd - keepSecret: false + keepSecret: true ## NA if cloning from a SourceDB (cloneFrom is set) charset: AL32UTF8 diff --git a/config/samples/sidb/singleinstancedatabase_patch.yaml b/config/samples/sidb/singleinstancedatabase_patch.yaml index fb55e6b2..8fb0ddb6 100644 --- a/config/samples/sidb/singleinstancedatabase_patch.yaml +++ b/config/samples/sidb/singleinstancedatabase_patch.yaml @@ -17,7 +17,7 @@ spec: adminPassword: secretName: secretKey: - keepSecret: false + keepSecret: true ## Patch the database by updating the RU version/image ## Major version changes are not supported diff --git a/config/samples/sidb/singleinstancedatabase_prov.yaml b/config/samples/sidb/singleinstancedatabase_prov.yaml index 6e6afde0..dbee637c 100644 --- a/config/samples/sidb/singleinstancedatabase_prov.yaml +++ b/config/samples/sidb/singleinstancedatabase_prov.yaml @@ -16,7 +16,7 @@ spec: adminPassword: secretName: secretKey: - keepSecret: false + keepSecret: true ## Database image details image: From 9ab832cceba92b6968fc21e68d3c4d80a3c8a15c Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Thu, 23 Dec 2021 13:51:40 -0500 Subject: [PATCH 131/628] update Autonomous Database Backup --- .../autonomousdatabasebackup_types.go | 6 +- .../autonomousdatabaserestore_types.go | 10 +-- .../v1alpha1/zz_generated.deepcopy.go | 33 ++++----- commons/oci/database.go | 70 ++++--------------- commons/oci/ociutil/time.go | 20 ++++-- ....oracle.com_autonomousdatabasebackups.yaml | 10 +-- ...oracle.com_autonomousdatabaserestores.yaml | 16 ++--- .../adb/autonomousdatabase_backup.yaml | 11 +++ .../adb/autonomousdatabase_restore.yaml | 16 +++++ .../adb_backup/autonomousdatabasebackup.yaml | 7 -- ...se_v1alpha1_autonomousdatabaserestore.yaml | 7 -- .../autonomousdatabaserestore_controller.go | 25 ++++--- 12 files changed, 102 insertions(+), 129 deletions(-) create mode 100644 config/samples/adb/autonomousdatabase_backup.yaml create mode 100644 config/samples/adb/autonomousdatabase_restore.yaml delete mode 100644 config/samples/adb_backup/autonomousdatabasebackup.yaml delete mode 100644 config/samples/database_v1alpha1_autonomousdatabaserestore.yaml diff --git a/apis/database/v1alpha1/autonomousdatabasebackup_types.go b/apis/database/v1alpha1/autonomousdatabasebackup_types.go index 50d93b42..b909fa77 100644 --- a/apis/database/v1alpha1/autonomousdatabasebackup_types.go +++ b/apis/database/v1alpha1/autonomousdatabasebackup_types.go @@ -52,8 +52,7 @@ import ( type AutonomousDatabaseBackupSpec struct { // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster // Important: Run "make" to regenerate code after modifying this file - DisplayName string `json:"displayName,omitempty"` - AutonomousDatabaseOCID string `json:"autonomousDatabaseOCID,omitempty"` + AutonomousDatabaseOCID string `json:"autonomousDatabaseOCID"` AutonomousDatabaseBackupOCID string `json:"autonomousDatabaseBackupOCID,omitempty"` OCIConfig OCIConfigSpec `json:"ociConfig,omitempty"` @@ -66,7 +65,6 @@ type AutonomousDatabaseBackupStatus struct { AutonomousDatabaseBackupOCID string `json:"autonomousDatabaseBackupOCID"` CompartmentOCID string `json:"compartmentOCID"` AutonomousDatabaseOCID string `json:"autonomousDatabaseOCID"` - DisplayName string `json:"displayName"` Type database.AutonomousDatabaseBackupTypeEnum `json:"type"` IsAutomatic bool `json:"isAutomatic"` LifecycleState database.AutonomousDatabaseBackupLifecycleStateEnum `json:"lifecycleState"` @@ -77,7 +75,6 @@ type AutonomousDatabaseBackupStatus struct { //+kubebuilder:object:root=true //+kubebuilder:subresource:status // +kubebuilder:resource:shortName="adbbu";"adbbus" -// +kubebuilder:printcolumn:JSONPath=".status.displayName",name="Display Name",type=string // +kubebuilder:printcolumn:JSONPath=".status.lifecycleState",name="State",type=string // +kubebuilder:printcolumn:JSONPath=".status.type",name="Type",type=string // +kubebuilder:printcolumn:JSONPath=".status.timeStarted",name="Started",type=string @@ -96,7 +93,6 @@ func (backup *AutonomousDatabaseBackup) UpdateStatusFromAutonomousDatabaseBackup backup.Status.AutonomousDatabaseBackupOCID = *resp.Id backup.Status.CompartmentOCID = *resp.CompartmentId backup.Status.AutonomousDatabaseOCID = *resp.AutonomousDatabaseId - backup.Status.DisplayName = *resp.DisplayName backup.Status.Type = resp.Type backup.Status.IsAutomatic = *resp.IsAutomatic backup.Status.LifecycleState = resp.LifecycleState diff --git a/apis/database/v1alpha1/autonomousdatabaserestore_types.go b/apis/database/v1alpha1/autonomousdatabaserestore_types.go index f7b1a1ff..b461b8ab 100644 --- a/apis/database/v1alpha1/autonomousdatabaserestore_types.go +++ b/apis/database/v1alpha1/autonomousdatabaserestore_types.go @@ -49,18 +49,18 @@ import ( type AutonomousDatabaseRestoreSpec struct { // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster // Important: Run "make" to regenerate code after modifying this file - AutonomousDatabaseOCID string `json:"autonomousDatabaseOCID"` - Source AutonomousDatabaseRestoreSource `json:"source"` + AutonomousDatabaseOCID string `json:"autonomousDatabaseOCID"` + Destination AutonomousDatabaseRestoreDestination `json:"destination"` OCIConfig OCIConfigSpec `json:"ociConfig,omitempty"` } -type AutonomousDatabaseRestoreSource struct { +type AutonomousDatabaseRestoreDestination struct { BackupName string `json:"backupName,omitempty"` - DateTime string `json:"datetime,omitempty"` + TimeStamp string `json:"timeStamp,omitempty"` } -// +kubebuilder:validation:Enum=New;InProgress;Completed;Failed +// +kubebuilder:validation:Enum=NewCreated;InProgress;Completed;Failed type RestoreLifecycleState string const ( diff --git a/apis/database/v1alpha1/zz_generated.deepcopy.go b/apis/database/v1alpha1/zz_generated.deepcopy.go index 585cc2c0..f1aa816c 100644 --- a/apis/database/v1alpha1/zz_generated.deepcopy.go +++ b/apis/database/v1alpha1/zz_generated.deepcopy.go @@ -1,3 +1,4 @@ +//go:build !ignore_autogenerated // +build !ignore_autogenerated /* @@ -323,6 +324,21 @@ func (in *AutonomousDatabaseRestore) DeepCopyObject() runtime.Object { return nil } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousDatabaseRestoreDestination) DeepCopyInto(out *AutonomousDatabaseRestoreDestination) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseRestoreDestination. +func (in *AutonomousDatabaseRestoreDestination) DeepCopy() *AutonomousDatabaseRestoreDestination { + if in == nil { + return nil + } + out := new(AutonomousDatabaseRestoreDestination) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *AutonomousDatabaseRestoreList) DeepCopyInto(out *AutonomousDatabaseRestoreList) { *out = *in @@ -355,25 +371,10 @@ func (in *AutonomousDatabaseRestoreList) DeepCopyObject() runtime.Object { return nil } -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *AutonomousDatabaseRestoreSource) DeepCopyInto(out *AutonomousDatabaseRestoreSource) { - *out = *in -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseRestoreSource. -func (in *AutonomousDatabaseRestoreSource) DeepCopy() *AutonomousDatabaseRestoreSource { - if in == nil { - return nil - } - out := new(AutonomousDatabaseRestoreSource) - in.DeepCopyInto(out) - return out -} - // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *AutonomousDatabaseRestoreSpec) DeepCopyInto(out *AutonomousDatabaseRestoreSpec) { *out = *in - out.Source = in.Source + out.Destination = in.Destination in.OCIConfig.DeepCopyInto(&out.OCIConfig) } diff --git a/commons/oci/database.go b/commons/oci/database.go index fe8c352b..06ab3a3c 100644 --- a/commons/oci/database.go +++ b/commons/oci/database.go @@ -39,13 +39,10 @@ package oci import ( - "bytes" "context" - "encoding/json" "errors" "fmt" "math" - "net/http" "time" "github.com/go-logr/logr" @@ -59,7 +56,6 @@ import ( "k8s.io/apimachinery/pkg/types" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" - "github.com/oracle/oracle-database-operator/commons/oci/ociutil" ) // CreateAutonomousDatabase sends a request to OCI to provision a database and returns the AutonomousDatabase OCID. @@ -513,7 +509,7 @@ func startAutonomousDatabase(dbClient database.DatabaseClient, adbOCID string) ( AutonomousDatabaseId: common.String(adbOCID), } - resp, err = dbClient.StartAutonomousDatabase(context.Background(), startRequest) + resp, err = dbClient.StartAutonomousDatabase(context.TODO(), startRequest) return } @@ -523,7 +519,7 @@ func stopAutonomousDatabase(dbClient database.DatabaseClient, adbOCID string) (r AutonomousDatabaseId: common.String(adbOCID), } - resp, err = dbClient.StopAutonomousDatabase(context.Background(), stopRequest) + resp, err = dbClient.StopAutonomousDatabase(context.TODO(), stopRequest) return } @@ -533,55 +529,17 @@ func DeleteAutonomousDatabase(dbClient database.DatabaseClient, adbOCID string) AutonomousDatabaseId: common.String(adbOCID), } - resp, err = dbClient.DeleteAutonomousDatabase(context.Background(), deleteRequest) - return + return dbClient.DeleteAutonomousDatabase(context.TODO(), deleteRequest) } -func RestoreAutonomousDatabase(dbClient database.DatabaseClient, adbOCID string, dateTime time.Time) (opcID string, err error) { - endpoint := dbClient.Endpoint() - url := fmt.Sprintf("%s/20160918/autonomousDatabases/%s/actions/restore", endpoint, adbOCID) - - // build the body - reqBody, err := json.Marshal(map[string]string{ - "timestamp": ociutil.FormatSDKTime(dateTime), - }) - if err != nil { - return "", err - } - - // create request - request, err := http.NewRequest("POST", url, bytes.NewBuffer(reqBody)) - if err != nil { - return "", err - } - - // Set the header - request.Header.Set("Date", time.Now().UTC().Format(http.TimeFormat)) - request.Header.Set("Content-Type", "application/json") - - hash, err := common.GetBodyHash(request) - if err != nil { - return "", err - } - request.Header.Set("x-content-sha256", hash) - request.Header.Set("Content-Length", fmt.Sprint(request.ContentLength)) - - // Build the signer - signer := common.DefaultRequestSigner(*dbClient.ConfigurationProvider()) - - // Sign the request - signer.Sign(request) - - client := http.Client{} - - // Execute the request - resp, err := client.Do(request) - if err != nil { - return "", err +func RestoreAutonomousDatabase(dbClient database.DatabaseClient, adbOCID string, sdkTime *common.SDKTime) (resp database.RestoreAutonomousDatabaseResponse, err error) { + request := database.RestoreAutonomousDatabaseRequest{ + AutonomousDatabaseId: common.String(adbOCID), + RestoreAutonomousDatabaseDetails: database.RestoreAutonomousDatabaseDetails{ + Timestamp: sdkTime, + }, } - - opcRequestId := resp.Header.Get("opc-work-request-id") - return opcRequestId, nil + return dbClient.RestoreAutonomousDatabase(context.TODO(), request) } // ListAutonomousDatabaseBackups returns a list of Autonomous Database backups @@ -599,11 +557,11 @@ func ListAutonomousDatabaseBackups(dbClient database.DatabaseClient, adb *dbv1al // CreateAutonomousDatabaseBackup creates an backup of Autonomous Database func CreateAutonomousDatabaseBackup(logger logr.Logger, dbClient database.DatabaseClient, adbBackup *dbv1alpha1.AutonomousDatabaseBackup) (resp database.CreateAutonomousDatabaseBackupResponse, err error) { - logger.Info("Creating Autonomous Database backup " + adbBackup.Spec.DisplayName) + logger.Info("Creating Autonomous Database backup " + adbBackup.GetName()) createBackupRequest := database.CreateAutonomousDatabaseBackupRequest{ CreateAutonomousDatabaseBackupDetails: database.CreateAutonomousDatabaseBackupDetails{ - DisplayName: &adbBackup.Spec.DisplayName, + DisplayName: common.String(adbBackup.GetName()), AutonomousDatabaseId: &adbBackup.Spec.AutonomousDatabaseOCID, }, } @@ -654,7 +612,9 @@ func getCompleteWorkRetryPolicy() common.RetryPolicy { if converted, ok := r.Response.(workrequests.GetWorkRequestResponse); ok { // do the retry until WorkReqeut Status is Succeeded - ignore case (BMI-2652) - return converted.Status != workrequests.WorkRequestStatusSucceeded + return converted.Status != workrequests.WorkRequestStatusSucceeded && + converted.Status != workrequests.WorkRequestStatusFailed && + converted.Status != workrequests.WorkRequestStatusCanceled } return true diff --git a/commons/oci/ociutil/time.go b/commons/oci/ociutil/time.go index 47e7cc88..cfd375d5 100644 --- a/commons/oci/ociutil/time.go +++ b/commons/oci/ociutil/time.go @@ -1,13 +1,23 @@ package ociutil -import "time" +import ( + "time" -const sdkFormat = "2006-01-02T15:04:05.999Z07:00" + "github.com/oracle/oci-go-sdk/v51/common" +) + +// Follow the format of the Display Name +const displayFormat = "Jan 02, 2006 15:04:05 MST" func FormatSDKTime(dateTime time.Time) string { - return dateTime.Format(sdkFormat) + return dateTime.Format(displayFormat) } -func ParseSDKTime(val string) (time.Time, error) { - return time.Parse(sdkFormat, val) +func ParseDisplayTime(val string) (*common.SDKTime, error) { + parsedTime, err := time.Parse(displayFormat, val) + if err != nil { + return nil, err + } + sdkTime := common.SDKTime{Time: parsedTime} + return &sdkTime, nil } diff --git a/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml b/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml index 4e6ae88b..68d26754 100644 --- a/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml +++ b/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml @@ -20,9 +20,6 @@ spec: scope: Namespaced versions: - additionalPrinterColumns: - - jsonPath: .status.displayName - name: Display Name - type: string - jsonPath: .status.lifecycleState name: State type: string @@ -60,8 +57,6 @@ spec: autonomousDatabaseBackupOCID: type: string autonomousDatabaseOCID: - type: string - displayName: description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster Important: Run "make" to regenerate code after modifying this file' type: string @@ -72,6 +67,8 @@ spec: secretName: type: string type: object + required: + - autonomousDatabaseOCID type: object status: description: AutonomousDatabaseBackupStatus defines the observed state @@ -86,8 +83,6 @@ spec: type: string compartmentOCID: type: string - displayName: - type: string isAutomatic: type: boolean lifecycleState: @@ -106,7 +101,6 @@ spec: - autonomousDatabaseBackupOCID - autonomousDatabaseOCID - compartmentOCID - - displayName - isAutomatic - lifecycleState - type diff --git a/config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml b/config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml index f4671cce..9534f3b6 100644 --- a/config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml +++ b/config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml @@ -45,23 +45,23 @@ spec: description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster Important: Run "make" to regenerate code after modifying this file' type: string - ociConfig: + destination: properties: - configMapName: + backupName: type: string - secretName: + timeStamp: type: string type: object - source: + ociConfig: properties: - backupName: + configMapName: type: string - datetime: + secretName: type: string type: object required: - autonomousDatabaseOCID - - source + - destination type: object status: description: AutonomousDatabaseRestoreStatus defines the observed state @@ -72,7 +72,7 @@ spec: of cluster Important: Run "make" to regenerate code after modifying this file' enum: - - New + - NewCreated - InProgress - Completed - Failed diff --git a/config/samples/adb/autonomousdatabase_backup.yaml b/config/samples/adb/autonomousdatabase_backup.yaml new file mode 100644 index 00000000..3e303880 --- /dev/null +++ b/config/samples/adb/autonomousdatabase_backup.yaml @@ -0,0 +1,11 @@ +apiVersion: database.oracle.com/v1alpha1 +kind: AutonomousDatabaseBackup +metadata: + name: autonomousdatabasebackup-sample +spec: + autonomousDatabaseOCID: ocid1.autonomousdatabase... + + # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey \ No newline at end of file diff --git a/config/samples/adb/autonomousdatabase_restore.yaml b/config/samples/adb/autonomousdatabase_restore.yaml new file mode 100644 index 00000000..52c637c8 --- /dev/null +++ b/config/samples/adb/autonomousdatabase_restore.yaml @@ -0,0 +1,16 @@ +apiVersion: database.oracle.com/v1alpha1 +kind: AutonomousDatabaseRestore +metadata: + name: autonomousdatabaserestore-sample +spec: + autonomousDatabaseOCID: ocid1.autonomousdatabase... + destination: + backupName: autonomousdatabasebackup-sample + timeStamp: Oct 23, 2021 11:03:13 UTC + + # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey + + \ No newline at end of file diff --git a/config/samples/adb_backup/autonomousdatabasebackup.yaml b/config/samples/adb_backup/autonomousdatabasebackup.yaml deleted file mode 100644 index 93da9f77..00000000 --- a/config/samples/adb_backup/autonomousdatabasebackup.yaml +++ /dev/null @@ -1,7 +0,0 @@ -apiVersion: database.oracle.com/v1alpha1 -kind: AutonomousDatabaseBackup -metadata: - name: autonomousdatabasebackup-sample -spec: - # Add fields here - foo: bar diff --git a/config/samples/database_v1alpha1_autonomousdatabaserestore.yaml b/config/samples/database_v1alpha1_autonomousdatabaserestore.yaml deleted file mode 100644 index 4e1cb9a1..00000000 --- a/config/samples/database_v1alpha1_autonomousdatabaserestore.yaml +++ /dev/null @@ -1,7 +0,0 @@ -apiVersion: database.oracle.com/v1alpha1 -kind: AutonomousDatabaseRestore -metadata: - name: autonomousdatabaserestore-sample -spec: - # Add fields here - foo: bar diff --git a/controllers/database/autonomousdatabaserestore_controller.go b/controllers/database/autonomousdatabaserestore_controller.go index ed9dbb8e..d666fc1f 100644 --- a/controllers/database/autonomousdatabaserestore_controller.go +++ b/controllers/database/autonomousdatabaserestore_controller.go @@ -40,7 +40,6 @@ package controllers import ( "context" - "time" "github.com/go-logr/logr" apiErrors "k8s.io/apimachinery/pkg/api/errors" @@ -49,6 +48,7 @@ import ( ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" + "github.com/oracle/oci-go-sdk/v51/common" "github.com/oracle/oci-go-sdk/v51/database" "github.com/oracle/oci-go-sdk/v51/workrequests" databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" @@ -139,16 +139,16 @@ func (r *AutonomousDatabaseRestoreReconciler) Reconcile(ctx context.Context, req * Restore ******************************************************************/ if restore.Status.LifecycleState == "" || restore.Status.LifecycleState == dbv1alpha1.RestoreLifecycleStateNew { - var restoreTime time.Time + var restoreTime *common.SDKTime - if restore.Spec.Source.BackupName != "" { + if restore.Spec.Destination.BackupName != "" { backup := &dbv1alpha1.AutonomousDatabaseBackup{} - namespacedName := types.NamespacedName{Namespace: restore.Namespace, Name: restore.Spec.Source.BackupName} + namespacedName := types.NamespacedName{Namespace: restore.Namespace, Name: restore.Spec.Destination.BackupName} if err := r.KubeClient.Get(context.TODO(), namespacedName, backup); err != nil { return ctrl.Result{}, err } - restoreTime, err = ociutil.ParseSDKTime(backup.Status.TimeEnded) + restoreTime, err = ociutil.ParseDisplayTime(backup.Status.TimeEnded) if err != nil { r.currentLogger.Error(err, "Fail to parse time "+backup.Status.TimeEnded) @@ -159,10 +159,10 @@ func (r *AutonomousDatabaseRestoreReconciler) Reconcile(ctx context.Context, req } return ctrl.Result{}, nil } - } else if restore.Spec.Source.DateTime != "" { - restoreTime, err = ociutil.ParseSDKTime(restore.Spec.Source.DateTime) + } else if restore.Spec.Destination.TimeStamp != "" { + restoreTime, err = ociutil.ParseDisplayTime(restore.Spec.Destination.TimeStamp) if err != nil { - r.currentLogger.Error(err, "Fail to parse time "+restore.Spec.Source.DateTime) + r.currentLogger.Error(err, "Fail to parse time "+restore.Spec.Destination.TimeStamp) // Change the status to UNAVAILABLE restore.Status.LifecycleState = dbv1alpha1.RestoreLifecycleStateFailed @@ -173,10 +173,9 @@ func (r *AutonomousDatabaseRestoreReconciler) Reconcile(ctx context.Context, req } } - opcID, err := oci.RestoreAutonomousDatabase(dbClient, restore.Spec.AutonomousDatabaseOCID, restoreTime) - + resp, err := oci.RestoreAutonomousDatabase(dbClient, restore.Spec.AutonomousDatabaseOCID, restoreTime) if err != nil { - r.currentLogger.Error(err, "Fail to get OCI work request client") + r.currentLogger.Error(err, "Fail to restore database") // Change the status to UNAVAILABLE restore.Status.LifecycleState = dbv1alpha1.RestoreLifecycleStateFailed @@ -186,8 +185,8 @@ func (r *AutonomousDatabaseRestoreReconciler) Reconcile(ctx context.Context, req return ctrl.Result{}, nil } - if err := oci.WaitUntilWorkCompleted(r.currentLogger, workClient, &opcID); err != nil { - r.currentLogger.Error(err, "Fail to watch workrequest. Workrequest ID = "+opcID) + if err := oci.WaitUntilWorkCompleted(r.currentLogger, workClient, resp.OpcWorkRequestId); err != nil { + r.currentLogger.Error(err, "Fail to watch workrequest. Workrequest ID = "+*resp.OpcWorkRequestId) // Change the status to UNAVAILABLE restore.Status.LifecycleState = dbv1alpha1.RestoreLifecycleStateFailed From aa478cd390f2e547a728a29ceb644d3a08117e6b Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Thu, 23 Dec 2021 14:52:55 -0500 Subject: [PATCH 132/628] resolve merge conflicts --- commons/oci/database.go | 87 ----------------------------------------- 1 file changed, 87 deletions(-) diff --git a/commons/oci/database.go b/commons/oci/database.go index 27dced4b..06ab3a3c 100644 --- a/commons/oci/database.go +++ b/commons/oci/database.go @@ -39,13 +39,10 @@ package oci import ( - "bytes" "context" - "encoding/json" "errors" "fmt" "math" - "net/http" "time" "github.com/go-logr/logr" @@ -59,7 +56,6 @@ import ( "k8s.io/apimachinery/pkg/types" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" - "github.com/oracle/oracle-database-operator/commons/oci/ociutil" ) // CreateAutonomousDatabase sends a request to OCI to provision a database and returns the AutonomousDatabase OCID. @@ -582,89 +578,6 @@ func GetAutonomousDatabaseBackup(dbClient database.DatabaseClient, backupOCID *s return dbClient.GetAutonomousDatabaseBackup(context.TODO(), getBackupRequest) } -func RestoreAutonomousDatabase(dbClient database.DatabaseClient, adbOCID string, dateTime time.Time) (opcID string, err error) { - endpoint := dbClient.Endpoint() - url := fmt.Sprintf("%s/20160918/autonomousDatabases/%s/actions/restore", endpoint, adbOCID) - - // build the body - reqBody, err := json.Marshal(map[string]string{ - "timestamp": ociutil.FormatSDKTime(dateTime), - }) - if err != nil { - return "", err - } - - // create request - request, err := http.NewRequest("POST", url, bytes.NewBuffer(reqBody)) - if err != nil { - return "", err - } - - // Set the header - request.Header.Set("Date", time.Now().UTC().Format(http.TimeFormat)) - request.Header.Set("Content-Type", "application/json") - - hash, err := common.GetBodyHash(request) - if err != nil { - return "", err - } - request.Header.Set("x-content-sha256", hash) - request.Header.Set("Content-Length", fmt.Sprint(request.ContentLength)) - - // Build the signer - signer := common.DefaultRequestSigner(*dbClient.ConfigurationProvider()) - - // Sign the request - signer.Sign(request) - - client := http.Client{} - - // Execute the request - resp, err := client.Do(request) - if err != nil { - return "", err - } - - opcRequestId := resp.Header.Get("opc-work-request-id") - return opcRequestId, nil -} - -// ListAutonomousDatabaseBackups returns a list of Autonomous Database backups -func ListAutonomousDatabaseBackups(dbClient database.DatabaseClient, adb *dbv1alpha1.AutonomousDatabase) (resp database.ListAutonomousDatabaseBackupsResponse, err error) { - if adb.Spec.Details.AutonomousDatabaseOCID == nil { - return resp, nil - } - - listBackupRequest := database.ListAutonomousDatabaseBackupsRequest{ - AutonomousDatabaseId: adb.Spec.Details.AutonomousDatabaseOCID, - } - - return dbClient.ListAutonomousDatabaseBackups(context.TODO(), listBackupRequest) -} - -// CreateAutonomousDatabaseBackup creates an backup of Autonomous Database -func CreateAutonomousDatabaseBackup(logger logr.Logger, dbClient database.DatabaseClient, adbBackup *dbv1alpha1.AutonomousDatabaseBackup) (resp database.CreateAutonomousDatabaseBackupResponse, err error) { - logger.Info("Creating Autonomous Database backup " + adbBackup.Spec.DisplayName) - - createBackupRequest := database.CreateAutonomousDatabaseBackupRequest{ - CreateAutonomousDatabaseBackupDetails: database.CreateAutonomousDatabaseBackupDetails{ - DisplayName: &adbBackup.Spec.DisplayName, - AutonomousDatabaseId: &adbBackup.Spec.AutonomousDatabaseOCID, - }, - } - - return dbClient.CreateAutonomousDatabaseBackup(context.TODO(), createBackupRequest) -} - -// GetAutonomousDatabaseBackup returns the response of GetAutonomousDatabaseBackupRequest -func GetAutonomousDatabaseBackup(dbClient database.DatabaseClient, backupOCID *string) (resp database.GetAutonomousDatabaseBackupResponse, err error) { - getBackupRequest := database.GetAutonomousDatabaseBackupRequest{ - AutonomousDatabaseBackupId: backupOCID, - } - - return dbClient.GetAutonomousDatabaseBackup(context.TODO(), getBackupRequest) -} - func WaitUntilWorkCompleted(logger logr.Logger, workClient workrequests.WorkRequestClient, opcWorkRequestID *string) error { if opcWorkRequestID == nil { return nil From d6d2329f48684a0830a4600ce51d8b46fe9aea22 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Thu, 23 Dec 2021 17:53:18 -0500 Subject: [PATCH 133/628] Update time format and fix unexpected reconcile --- commons/oci/ociutil/time.go | 2 +- config/manager/manager.yaml | 2 +- .../adb/autonomousdatabase_backup.yaml | 2 + .../adb/autonomousdatabase_restore.yaml | 5 +- .../autonomousdatabasebackup_controller.go | 51 +++++++++++-------- .../autonomousdatabaserestore_controller.go | 1 - test/e2e/behavior/shared_behaviors.go | 4 +- 7 files changed, 39 insertions(+), 28 deletions(-) diff --git a/commons/oci/ociutil/time.go b/commons/oci/ociutil/time.go index cfd375d5..49cd9894 100644 --- a/commons/oci/ociutil/time.go +++ b/commons/oci/ociutil/time.go @@ -7,7 +7,7 @@ import ( ) // Follow the format of the Display Name -const displayFormat = "Jan 02, 2006 15:04:05 MST" +const displayFormat = "2006-01-02 15:04:05 MST" func FormatSDKTime(dateTime time.Time) string { return dateTime.Format(displayFormat) diff --git a/config/manager/manager.yaml b/config/manager/manager.yaml index 259b79a5..362aac61 100644 --- a/config/manager/manager.yaml +++ b/config/manager/manager.yaml @@ -34,7 +34,7 @@ spec: args: - --enable-leader-election image: controller:latest - imagePullPolicy: Never #Always + imagePullPolicy: Always name: manager resources: limits: diff --git a/config/samples/adb/autonomousdatabase_backup.yaml b/config/samples/adb/autonomousdatabase_backup.yaml index 3e303880..bd671910 100644 --- a/config/samples/adb/autonomousdatabase_backup.yaml +++ b/config/samples/adb/autonomousdatabase_backup.yaml @@ -3,6 +3,8 @@ kind: AutonomousDatabaseBackup metadata: name: autonomousdatabasebackup-sample spec: + # Before you can create manual backups, you must have an Object Storage bucket and your database must be configured to connect to it. This is a one-time operation. + # See https://docs.oracle.com/en-us/iaas/Content/Database/Tasks/adbbackingup.htm#creatingbucket autonomousDatabaseOCID: ocid1.autonomousdatabase... # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. diff --git a/config/samples/adb/autonomousdatabase_restore.yaml b/config/samples/adb/autonomousdatabase_restore.yaml index 52c637c8..2f9dc623 100644 --- a/config/samples/adb/autonomousdatabase_restore.yaml +++ b/config/samples/adb/autonomousdatabase_restore.yaml @@ -4,9 +4,12 @@ metadata: name: autonomousdatabaserestore-sample spec: autonomousDatabaseOCID: ocid1.autonomousdatabase... + # Restore the database either from a backup or using point-in-time restore destination: + # The name of your AutonomousDatabaseBackup backupName: autonomousdatabasebackup-sample - timeStamp: Oct 23, 2021 11:03:13 UTC + # The timestamp must follow this format: YYYY-MM-DD HH:MM:SS GMT + # timeStamp: 2021-10-23 11:03:13 UTC # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. ociConfig: diff --git a/controllers/database/autonomousdatabasebackup_controller.go b/controllers/database/autonomousdatabasebackup_controller.go index fa83a823..a4aa3c6f 100644 --- a/controllers/database/autonomousdatabasebackup_controller.go +++ b/controllers/database/autonomousdatabasebackup_controller.go @@ -160,10 +160,11 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req } /****************************************************************** - * Create a backup if the Spec.AutonomousDatabaseBackupOCID is empty, - * otherwise bind to an exisiting backup + * Create a backup if the Spec.AutonomousDatabaseBackupOCID is + * empty and the LifecycleState is never assigned, or bind to an exisiting + * backup if the Spec.AutonomousDatabaseBackupOCID isn't empty. ******************************************************************/ - if adbBackup.Spec.AutonomousDatabaseBackupOCID == "" { + if adbBackup.Spec.AutonomousDatabaseBackupOCID == "" && adbBackup.Status.LifecycleState == "" { resp, err := oci.CreateAutonomousDatabaseBackup(r.currentLogger, dbClient, adbBackup) if err != nil { r.currentLogger.Error(err, "Fail to create AutonomousDatabase Backup") @@ -192,7 +193,7 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req } return ctrl.Result{}, nil } - } else { + } else if adbBackup.Spec.AutonomousDatabaseBackupOCID != "" { // Binding // Add the Status.AutonomousDatabaseBackupOCID and update the status adbBackup.Status.AutonomousDatabaseBackupOCID = adbBackup.Spec.AutonomousDatabaseBackupOCID @@ -200,32 +201,38 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req } /****************************************************************** - * Update the status of the resource + * Update the status of the resource if the + * Status.AutonomousDatabaseOCID isn't empty. ******************************************************************/ - resp, err := oci.GetAutonomousDatabaseBackup(dbClient, &adbBackup.Status.AutonomousDatabaseBackupOCID) - if err != nil { - // Change the status to UNAVAILABLE - adbBackup.Status.LifecycleState = database.AutonomousDatabaseBackupLifecycleStateFailed - if statusErr := backupUtil.UpdateAutonomousDatabaseBackupStatus(r.KubeClient, adbBackup); statusErr != nil { - return ctrl.Result{}, statusErr + if adbBackup.Status.AutonomousDatabaseOCID != "" { + resp, err := oci.GetAutonomousDatabaseBackup(dbClient, &adbBackup.Status.AutonomousDatabaseBackupOCID) + if err != nil { + // Change the status to UNAVAILABLE + adbBackup.Status.LifecycleState = database.AutonomousDatabaseBackupLifecycleStateFailed + if statusErr := backupUtil.UpdateAutonomousDatabaseBackupStatus(r.KubeClient, adbBackup); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, err } - return ctrl.Result{}, err - } - adbBackup.UpdateStatusFromAutonomousDatabaseBackupResponse(resp) + adbBackup.UpdateStatusFromAutonomousDatabaseBackupResponse(resp) - backupUtil.UpdateAutonomousDatabaseBackupStatus(r.KubeClient, adbBackup) + backupUtil.UpdateAutonomousDatabaseBackupStatus(r.KubeClient, adbBackup) + } /****************************************************************** - * Update the ownerReference if the AutonomousDatabase is found + * Look up the owner AutonomousDatabase and set the ownerReference + * if the owner hasn't been set. ******************************************************************/ - if err := backupUtil.SetOwnerAutonomousDatabase(r.currentLogger, r.KubeClient, adbBackup); err != nil { - // Change the status to UNAVAILABLE - adbBackup.Status.LifecycleState = database.AutonomousDatabaseBackupLifecycleStateFailed - if statusErr := backupUtil.UpdateAutonomousDatabaseBackupStatus(r.KubeClient, adbBackup); statusErr != nil { - return ctrl.Result{}, statusErr + if len(adbBackup.GetOwnerReferences()) > 0 && adbBackup.Status.AutonomousDatabaseOCID != "" { + if err := backupUtil.SetOwnerAutonomousDatabase(r.currentLogger, r.KubeClient, adbBackup); err != nil { + // Change the status to UNAVAILABLE + adbBackup.Status.LifecycleState = database.AutonomousDatabaseBackupLifecycleStateFailed + if statusErr := backupUtil.UpdateAutonomousDatabaseBackupStatus(r.KubeClient, adbBackup); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, err } - return ctrl.Result{}, err } r.currentLogger.Info("AutonomousDatabaseBackup reconcile successfully") diff --git a/controllers/database/autonomousdatabaserestore_controller.go b/controllers/database/autonomousdatabaserestore_controller.go index 99d435db..d666fc1f 100644 --- a/controllers/database/autonomousdatabaserestore_controller.go +++ b/controllers/database/autonomousdatabaserestore_controller.go @@ -103,7 +103,6 @@ func (r *AutonomousDatabaseRestoreReconciler) Reconcile(ctx context.Context, req if err != nil { r.currentLogger.Error(err, "Fail to get OCI provider") - // Change the status to UNAVAILABLE restore.Status.LifecycleState = dbv1alpha1.RestoreLifecycleStateFailed if statusErr := restoreUtil.UpdateAutonomousDatabaseRestoreStatus(r.KubeClient, restore); statusErr != nil { diff --git a/test/e2e/behavior/shared_behaviors.go b/test/e2e/behavior/shared_behaviors.go index 593bc378..ec4041b9 100644 --- a/test/e2e/behavior/shared_behaviors.go +++ b/test/e2e/behavior/shared_behaviors.go @@ -298,7 +298,7 @@ func AssertADBDetails(k8sClient *client.Client, dbClient *database.DatabaseClien expectedADBDetails := expectedADB.Spec.Details - // Compare the elements one by one rather than doing reflect.DeelEqual(adb1, adb2), since some parameters + // Compare the elements one by one rather than doing reflect.DeelEqual(adb1, adb2), since some parameters // (e.g. adminPassword, wallet) are missing from e2eutil.GetAutonomousDatabase(). // We don't compare LifecycleState in this case. We only make sure that the ADB is in AVAIABLE state before // proceeding to the next test. @@ -317,7 +317,7 @@ func AssertADBDetails(k8sClient *client.Client, dbClient *database.DatabaseClien reflect.DeepEqual(expectedADBDetails.NsgOCIDs, resp.AutonomousDatabase.NsgIds) && compareBool(expectedADBDetails.IsAccessControlEnabled, resp.AutonomousDatabase.IsAccessControlEnabled) && reflect.DeepEqual(expectedADBDetails.WhitelistedIPs, resp.AutonomousDatabase.WhitelistedIps) && - compareBool(expectedADBDetails.IsMTLSConnectionRequired, resp.AutonomousDatabase.IsMtlsConnectionRequired) && + compareBool(expectedADBDetails.IsMTLSConnectionRequired, resp.AutonomousDatabase.IsMtlsConnectionRequired) && compareString(expectedADBDetails.PrivateEndpointLabel, resp.AutonomousDatabase.PrivateEndpointLabel) return same, nil From 30ed9c6012e135284025de35d46c3af7f315e166 Mon Sep 17 00:00:00 2001 From: Gajanan Bhat Date: Mon, 3 Jan 2022 12:50:52 +0530 Subject: [PATCH 134/628] Added Status and Modify for PDBS --- .gitignore | 13 +- apis/database/v1alpha1/pdb_types.go | 2 +- apis/database/v1alpha1/pdb_webhook.go | 6 +- .../crd/bases/database.oracle.com_pdbs.yaml | 1 + controllers/database/cdb_controller.go | 132 ++++-- controllers/database/pdb_controller.go | 39 +- ords/runOrds.sh | 2 +- ...autonomousdatabase_controller_bind_test.go | 404 +++++++++--------- ...utonomousdatabase_controller_suite_test.go | 145 +++++++ test/e2e/suite_test.go | 103 ++--- 10 files changed, 519 insertions(+), 328 deletions(-) create mode 100644 test/e2e/autonomousdatabase_controller_suite_test.go diff --git a/.gitignore b/.gitignore index fd8f5261..a78b60d5 100644 --- a/.gitignore +++ b/.gitignore @@ -1,6 +1,7 @@ -operator.tgz -cover.out -bin -testbin/* -onpremtest/* -ords/*zip +operator.tgz +cover.out +bin +testbin/* +onpremtest/* +ords/*zip +.gitattributes diff --git a/apis/database/v1alpha1/pdb_types.go b/apis/database/v1alpha1/pdb_types.go index 686ebbbe..bfa9897d 100644 --- a/apis/database/v1alpha1/pdb_types.go +++ b/apis/database/v1alpha1/pdb_types.go @@ -96,7 +96,7 @@ type PDBSpec struct { // Whether you need the script only or execute the script GetScript *bool `json:"getScript,omitempty"` // Action to be taken: Create or Clone or Plug or Unplug - // +kubebuilder:validation:Enum=Create;Clone;Plug;Unplug;Delete;Modify + // +kubebuilder:validation:Enum=Create;Clone;Plug;Unplug;Delete;Modify;Status Action string `json:"action"` // Extra options for opening and closing a PDB // +kubebuilder:validation:Enum=IMMEDIATE;NORMAL;READ ONLY;READ WRITE;RESTRICTED diff --git a/apis/database/v1alpha1/pdb_webhook.go b/apis/database/v1alpha1/pdb_webhook.go index 1c42ca85..a0e988ab 100644 --- a/apis/database/v1alpha1/pdb_webhook.go +++ b/apis/database/v1alpha1/pdb_webhook.go @@ -76,7 +76,7 @@ func (r *PDB) Default() { r.Spec.DropAction = "KEEP" pdblog.Info(" - dropAction : KEEP") } - } else if action != "MODIFY" { + } else if action != "MODIFY" && action != "STATUS" { if r.Spec.ReuseTempFile == nil { r.Spec.ReuseTempFile = new(bool) *r.Spec.ReuseTempFile = true @@ -222,7 +222,7 @@ func (r *PDB) ValidateUpdate(old runtime.Object) error { action := strings.ToUpper(r.Spec.Action) // If PDB CR has been created and in Ready state, only allow updates if the "action" value has changed as well - if (r.Status.Phase == "Ready") && (r.Status.Action != "MODIFY") && (r.Status.Action == action) { + if (r.Status.Phase == "Ready") && (r.Status.Action != "MODIFY") && (r.Status.Action != "STATUS") && (r.Status.Action == action) { allErrs = append(allErrs, field.Required(field.NewPath("spec").Child("action"), "New action also needs to be specified after PDB is in Ready state")) } else { @@ -234,7 +234,7 @@ func (r *PDB) ValidateUpdate(old runtime.Object) error { r.validateAction(&allErrs) // Check TDE requirements - if action != "DELETE" && action != "MODIFY" && (*(r.Spec.TDEImport) || *(r.Spec.TDEExport)) { + if (action != "DELETE") && (action != "MODIFY") && (action != "STATUS") && (*(r.Spec.TDEImport) || *(r.Spec.TDEExport)) { r.validateTDEInfo(&allErrs) } } diff --git a/config/crd/bases/database.oracle.com_pdbs.yaml b/config/crd/bases/database.oracle.com_pdbs.yaml index 951c95a7..d80a4fcc 100644 --- a/config/crd/bases/database.oracle.com_pdbs.yaml +++ b/config/crd/bases/database.oracle.com_pdbs.yaml @@ -74,6 +74,7 @@ spec: - Unplug - Delete - Modify + - Status type: string adminName: description: The administrator username for the new PDB. This property diff --git a/controllers/database/cdb_controller.go b/controllers/database/cdb_controller.go index e331b31c..11bc68ca 100644 --- a/controllers/database/cdb_controller.go +++ b/controllers/database/cdb_controller.go @@ -85,6 +85,7 @@ var ( cdbPhaseSecrets = "DeletingSecrets" cdbPhaseReady = "Ready" cdbPhaseDelete = "Deleting" + cdbPhaseFail = "Failed" ) const CDBFinalizer = "database.oracle.com/CDBfinalizer" @@ -119,7 +120,7 @@ func (r *CDBReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.R // Execute for every reconcile defer func() { - //log.Info("DEFER", "Name", cdb.Name, "Phase", cdb.Status.Phase, "Status", strconv.FormatBool(cdb.Status.Status)) + log.Info("DEFER", "Name", cdb.Name, "Phase", cdb.Status.Phase, "Status", strconv.FormatBool(cdb.Status.Status)) if !cdb.Status.Status { if err := r.Status().Update(ctx, cdb); err != nil { log.Error(err, "Failed to update status for :"+cdb.Name, "err", err.Error()) @@ -141,7 +142,7 @@ func (r *CDBReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.R return requeueY, err } - //log.Info("Res Status:", "Name", cdb.Name, "Phase", cdb.Status.Phase, "Status", strconv.FormatBool(cdb.Status.Status)) + log.Info("Res Status:", "Name", cdb.Name, "Phase", cdb.Status.Phase, "Status", strconv.FormatBool(cdb.Status.Status)) // Finalizer section err = r.manageCDBDeletion(ctx, req, cdb) @@ -161,6 +162,11 @@ func (r *CDBReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.R switch phase { case cdbPhaseInit: + err = r.verifySecrets(ctx, req, cdb) + if err != nil { + cdb.Status.Phase = cdbPhaseFail + return requeueN, nil + } cdb.Status.Phase = cdbPhasePod case cdbPhasePod: // Create ORDS PODs @@ -174,6 +180,9 @@ func (r *CDBReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.R // Validate ORDS PODs err = r.validateORDSPods(ctx, req, cdb) if err != nil { + if cdb.Status.Phase == cdbPhaseFail { + return requeueN, nil + } log.Info("Reconcile queued") return requeueY, nil } @@ -276,6 +285,17 @@ func (r *CDBReconciler) validateORDSPods(ctx context.Context, req ctrl.Request, out, err := dbcommons.ExecCommand(r, r.Config, pod.Name, pod.Namespace, "", ctx, req, false, "bash", "-c", getORDSStatus) if strings.Contains(out, "HTTP/1.1 200 OK") || strings.Contains(strings.ToUpper(err.Error()), "HTTP/1.1 200 OK") { readyPods++ + } else if strings.Contains(out, "HTTP/1.1 404 Not Found") || strings.Contains(strings.ToUpper(err.Error()), "HTTP/1.1 404 NOT FOUND") { + // Check if DB connection parameters are correct + getORDSInstallStatus := " grep -q 'Failed to' /tmp/ords_install.log; echo $?;" + out, _ := dbcommons.ExecCommand(r, r.Config, pod.Name, pod.Namespace, "", ctx, req, false, "bash", "-c", getORDSInstallStatus) + if strings.TrimSpace(out) == "0" { + cdb.Status.Msg = "Check DB connection parameters" + cdb.Status.Phase = cdbPhaseFail + // Delete existing ReplicaSet + r.deleteReplicaSet(ctx, req, cdb) + return errors.New("Check DB connection parameters") + } } } } @@ -471,6 +491,32 @@ func (r *CDBReconciler) createReplicaSetSpec(cdb *dbapi.CDB) *appsv1.ReplicaSet return replicaSet } +/********************************************************** + * Evaluate change in Spec post creation and instantiation + /********************************************************/ +func (r *CDBReconciler) deleteReplicaSet(ctx context.Context, req ctrl.Request, cdb *dbapi.CDB) error { + log := r.Log.WithValues("deleteReplicaSet", req.NamespacedName) + + k_client, err := kubernetes.NewForConfig(r.Config) + if err != nil { + log.Error(err, "Kubernetes Config Error") + return err + } + + replicaSetName := cdb.Name + "-ords-rs" + err = k_client.AppsV1().ReplicaSets(cdb.Namespace).Delete(context.TODO(), replicaSetName, metav1.DeleteOptions{}) + if err != nil { + log.Info("Could not delete ReplicaSet", "RS Name", replicaSetName, "err", err.Error()) + if !strings.Contains(strings.ToUpper(err.Error()), "NOT FOUND") { + return err + } + } else { + log.Info("Successfully deleted ORDS ReplicaSet", "RS Name", replicaSetName) + } + + return nil +} + /********************************************************** * Evaluate change in Spec post creation and instantiation /********************************************************/ @@ -503,40 +549,19 @@ func (r *CDBReconciler) evaluateSpecChange(ctx context.Context, req ctrl.Request ordsSpecChange = true } else if envVar.Name == "ORDS_PORT" && envVar.Value != strconv.Itoa(cdb.Spec.ORDSPort) { ordsSpecChange = true + } else if envVar.Name == "ORACLE_SERVICE" && envVar.Value != cdb.Spec.ServiceName { + ordsSpecChange = true } } if ordsSpecChange { // Delete existing ReplicaSet - k_client, err := kubernetes.NewForConfig(r.Config) - if err != nil { - log.Error(err, "Kubernetes Config Error") - return err - } - replicaSetName := cdb.Name + "-ords-rs" - err = k_client.AppsV1().ReplicaSets(cdb.Namespace).Delete(context.TODO(), replicaSetName, metav1.DeleteOptions{}) - if err != nil { - log.Info("Could not delete ReplicaSet", "RS Name", replicaSetName, "err", err.Error()) - if !strings.Contains(strings.ToUpper(err.Error()), "NOT FOUND") { - return err - } - } else { - log.Info("Successfully deleted ORDS ReplicaSet", "RS Name", replicaSetName) - } - - // Create new ReplicaSet - replicaSet := r.createReplicaSetSpec(cdb) - log.Info("Re-Creating ORDS Replicaset: " + replicaSet.Name) - err = r.Create(ctx, replicaSet) + err = r.deleteReplicaSet(ctx, req, cdb) if err != nil { - log.Error(err, "Failed to re-create ReplicaSet for :"+cdb.Name, "Namespace", replicaSet.Namespace, "Name", replicaSet.Name) return err } - // Set CDB instance as the owner and controller - ctrl.SetControllerReference(cdb, replicaSet, r.Scheme) - log.Info("Successfully re-created ORDS ReplicaSet", "RS Name", replicaSetName) - cdb.Status.Phase = cdbPhaseValPod + cdb.Status.Phase = cdbPhaseInit cdb.Status.Status = false r.Status().Update(ctx, cdb) } else { @@ -720,6 +745,59 @@ func (r *CDBReconciler) deleteCDBInstance(ctx context.Context, req ctrl.Request, return nil } +/************************************************* + * Get Secret Key for a Secret Name + /************************************************/ +func (r *CDBReconciler) verifySecrets(ctx context.Context, req ctrl.Request, cdb *dbapi.CDB) error { + + log := r.Log.WithValues("verifySecrets", req.NamespacedName) + + if err := r.checkSecret(ctx, req, cdb, cdb.Spec.SysAdminPwd.Secret.SecretName); err != nil { + return err + } + if err := r.checkSecret(ctx, req, cdb, cdb.Spec.CDBAdminUser.Secret.SecretName); err != nil { + return err + } + if err := r.checkSecret(ctx, req, cdb, cdb.Spec.CDBAdminPwd.Secret.SecretName); err != nil { + return err + } + if err := r.checkSecret(ctx, req, cdb, cdb.Spec.ORDSPwd.Secret.SecretName); err != nil { + return err + } + if err := r.checkSecret(ctx, req, cdb, cdb.Spec.WebServerUser.Secret.SecretName); err != nil { + return err + } + if err := r.checkSecret(ctx, req, cdb, cdb.Spec.WebServerPwd.Secret.SecretName); err != nil { + return err + } + + cdb.Status.Msg = "" + log.Info("Verified secrets successfully") + return nil +} + +/************************************************* + * Get Secret Key for a Secret Name + /************************************************/ +func (r *CDBReconciler) checkSecret(ctx context.Context, req ctrl.Request, cdb *dbapi.CDB, secretName string) error { + + log := r.Log.WithValues("checkSecret", req.NamespacedName) + + secret := &corev1.Secret{} + err := r.Get(ctx, types.NamespacedName{Name: secretName, Namespace: cdb.Namespace}, secret) + if err != nil { + if apierrors.IsNotFound(err) { + log.Info("Secret not found:" + secretName) + cdb.Status.Msg = "Secret not found:" + secretName + return err + } + log.Error(err, "Unable to get the secret.") + return err + } + + return nil +} + /************************************************* * Delete Secrets /************************************************/ diff --git a/controllers/database/pdb_controller.go b/controllers/database/pdb_controller.go index 778ae818..1941be21 100644 --- a/controllers/database/pdb_controller.go +++ b/controllers/database/pdb_controller.go @@ -109,6 +109,7 @@ var ( pdbPhaseReady = "Ready" pdbPhaseDelete = "Deleting" pdbPhaseModify = "Modifying" + pdbPhaseStatus = "CheckingState" pdbPhaseFail = "Failed" ) @@ -174,16 +175,8 @@ func (r *PDBReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.R action := strings.ToUpper(pdb.Spec.Action) - if (pdb.Status.Phase == pdbPhaseReady) && (pdb.Status.Action != "") && (action == "MODIFY" || pdb.Status.Action != action) { - if action == "MODIFY" { - modOption := pdb.Spec.PDBState + "-" + pdb.Spec.ModifyOption - // To prevent Reconcile from Modifying again whenever the Operator gets re-started - if pdb.Status.ModifyOption != modOption { - pdb.Status.Status = false - } - } else { - pdb.Status.Status = false - } + if (pdb.Status.Phase == pdbPhaseReady) && (pdb.Status.Action != "") && (action == "MODIFY" || action == "STATUS" || pdb.Status.Action != action) { + pdb.Status.Status = false } if !pdb.Status.Status { @@ -204,6 +197,8 @@ func (r *PDBReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.R err = r.modifyPDB(ctx, req, pdb) case pdbPhaseDelete: err = r.deletePDB(ctx, req, pdb) + case pdbPhaseStatus: + err = r.getPDBState(ctx, req, pdb) default: log.Info("DEFAULT:", "Name", pdb.Name, "Phase", phase, "Status", strconv.FormatBool(pdb.Status.Status)) return requeueN, nil @@ -245,6 +240,8 @@ func (r *PDBReconciler) validatePhase(ctx context.Context, req ctrl.Request, pdb pdb.Status.Phase = pdbPhaseModify case "DELETE": pdb.Status.Phase = pdbPhaseDelete + case "STATUS": + pdb.Status.Phase = pdbPhaseStatus } log.Info("Validation complete") @@ -399,7 +396,13 @@ func (r *PDBReconciler) callAPI(ctx context.Context, req ctrl.Request, pdb *dbap if resp.StatusCode != http.StatusOK { bb, _ := ioutil.ReadAll(resp.Body) - pdb.Status.Msg = "ORDS Error - HTTP Status Code:" + strconv.Itoa(resp.StatusCode) + + if resp.StatusCode == 404 { + pdb.Status.ConnString = "" + pdb.Status.Msg = pdb.Spec.PDBName + " not found" + } else { + pdb.Status.Msg = "ORDS Error - HTTP Status Code:" + strconv.Itoa(resp.StatusCode) + } log.Info("ORDS Error - HTTP Status Code :"+strconv.Itoa(resp.StatusCode), "Err", string(bb)) var apiErr ORDSError @@ -711,6 +714,17 @@ func (r *PDBReconciler) modifyPDB(ctx context.Context, req ctrl.Request, pdb *db var err error + err = r.getPDBState(ctx, req, pdb) + if err != nil { + return err + } + + // To prevent Reconcile from Modifying again whenever the Operator gets re-started + modOption := pdb.Spec.PDBState + "-" + pdb.Spec.ModifyOption + if pdb.Status.ModifyOption == modOption { + return nil + } + cdb, err := r.getCDBResource(ctx, req, pdb) if err != nil { return err @@ -779,7 +793,7 @@ func (r *PDBReconciler) getPDBState(ctx context.Context, req ctrl.Request, pdb * pdb.Status.OpenMode = objmap["open_mode"].(string) - log.Info("Successfully obtained PDB state", "PDB Name", pdb.Spec.PDBName) + log.Info("Successfully obtained PDB state", "PDB Name", pdb.Spec.PDBName, "State", objmap["open_mode"].(string)) return nil } @@ -891,6 +905,7 @@ func (r *PDBReconciler) deletePDBInstance(req ctrl.Request, ctx context.Context, } _, err = r.callAPI(ctx, req, pdb, url, values, "DELETE") if err != nil { + pdb.Status.ConnString = "" return err } diff --git a/ords/runOrds.sh b/ords/runOrds.sh index d6ee09cc..60bab949 100644 --- a/ords/runOrds.sh +++ b/ords/runOrds.sh @@ -58,7 +58,7 @@ function setupOrds() { sed -i -e "s|###CDBADMIN_PWD###|$CDBADMIN_PWD|g" $ORDS_HOME/cdbadmin.properties # Start ORDS setup - java -jar $ORDS_HOME/ords.war install simple + java -jar $ORDS_HOME/ords.war install simple 2>&1 | tee /tmp/ords_install.log # Setup Web Server User $ORDS_HOME/$SETUP_WEBUSER diff --git a/test/e2e/autonomousdatabase_controller_bind_test.go b/test/e2e/autonomousdatabase_controller_bind_test.go index ff415da9..a0ff1104 100644 --- a/test/e2e/autonomousdatabase_controller_bind_test.go +++ b/test/e2e/autonomousdatabase_controller_bind_test.go @@ -60,239 +60,245 @@ import ( // These tests use Ginkgo (BDD-style Go testing framework). Refer to // http://onsi.github.io/ginkgo/ to learn more about Ginkgo. -var _ = Describe("test ADB binding with hardLink=true", func() { - const bindingHardLinkTestFileName = "bind_adb_hardLink.yaml" - var adbLookupKey types.NamespacedName - const downloadedWallet = "instance-wallet-secret-1" - var adbID *string - var terminatedAdbID string - - AfterEach(func() { - // IMPORTANT: The operator might have to call reconcile multiple times to finish an operation. - // If we do the update immediately, the previous reconciliation will overwrite the changes. - By("Sleeping 20 seconds to wait for reconciliation to finish") - time.Sleep(time.Second * 20) - }) +//if os.Getenv("ENABLE_WEBHOOKS") != "false" { +// ADBBindTests() +//} + +func ADBBindTests() { + var _ = Describe("test ADB binding with hardLink=true", func() { + const bindingHardLinkTestFileName = "bind_adb_hardLink.yaml" + var adbLookupKey types.NamespacedName + const downloadedWallet = "instance-wallet-secret-1" + var adbID *string + var terminatedAdbID string + + AfterEach(func() { + // IMPORTANT: The operator might have to call reconcile multiple times to finish an operation. + // If we do the update immediately, the previous reconciliation will overwrite the changes. + By("Sleeping 20 seconds to wait for reconciliation to finish") + time.Sleep(time.Second * 20) + }) - It("should init the test", func() { - By("creating a temp ADB in OCI for binding test") - dbName := e2eutil.GenerateDBName() - createResp, err := e2eutil.CreateAutonomousDatabase(dbClient, &SharedCompartmentOCID, &dbName, &SharedPlainTextAdminPassword) - Expect(err).ShouldNot(HaveOccurred()) - Expect(createResp.AutonomousDatabase.Id).ShouldNot(BeNil()) + It("should init the test", func() { + By("creating a temp ADB in OCI for binding test") + dbName := e2eutil.GenerateDBName() + createResp, err := e2eutil.CreateAutonomousDatabase(dbClient, &SharedCompartmentOCID, &dbName, &SharedPlainTextAdminPassword) + Expect(err).ShouldNot(HaveOccurred()) + Expect(createResp.AutonomousDatabase.Id).ShouldNot(BeNil()) - By("Save the database ID for later use") - adbID = createResp.AutonomousDatabase.Id - terminatedAdbID = *adbID + By("Save the database ID for later use") + adbID = createResp.AutonomousDatabase.Id + terminatedAdbID = *adbID - By("Wait until the work request is in SUCCEEDED status") - workClient, err := workrequests.NewWorkRequestClientWithConfigurationProvider(configProvider) - Expect(err).ShouldNot(HaveOccurred()) + By("Wait until the work request is in SUCCEEDED status") + workClient, err := workrequests.NewWorkRequestClientWithConfigurationProvider(configProvider) + Expect(err).ShouldNot(HaveOccurred()) - err = e2eutil.WaitUntilWorkCompleted(workClient, createResp.OpcWorkRequestId) - Expect(err).ShouldNot(HaveOccurred()) + err = e2eutil.WaitUntilWorkCompleted(workClient, createResp.OpcWorkRequestId) + Expect(err).ShouldNot(HaveOccurred()) - // listResp, err := e2eutil.ListAutonomousDatabases(dbClient, &SharedCompartmentOCID, &dbName) - // Expect(err).ShouldNot(HaveOccurred()) - // fmt.Printf("List request DB %s is in %s state \n", *listResp.Items[0].DisplayName, listResp.Items[0].LifecycleState) - }) + // listResp, err := e2eutil.ListAutonomousDatabases(dbClient, &SharedCompartmentOCID, &dbName) + // Expect(err).ShouldNot(HaveOccurred()) + // fmt.Printf("List request DB %s is in %s state \n", *listResp.Items[0].DisplayName, listResp.Items[0].LifecycleState) + }) - Describe("ADB binding with HardLink = false using Wallet Password Secret", func() { - It("Should create a AutonomousDatabase resource with HardLink = false", func() { - adb := &dbv1alpha1.AutonomousDatabase{ - TypeMeta: metav1.TypeMeta{ - APIVersion: "database.oracle.com/v1alpha1", - Kind: "AutonomousDatabase", - }, - ObjectMeta: metav1.ObjectMeta{ - Name: "bindadb", - Namespace: ADBNamespace, - }, - Spec: dbv1alpha1.AutonomousDatabaseSpec{ - Details: dbv1alpha1.AutonomousDatabaseDetails{ - AutonomousDatabaseOCID: adbID, - Wallet: dbv1alpha1.WalletSpec{ - Name: common.String(downloadedWallet), - Password: dbv1alpha1.PasswordSpec{ - K8sSecretName: common.String(SharedWalletPassSecretName), + Describe("ADB binding with HardLink = false using Wallet Password Secret", func() { + It("Should create a AutonomousDatabase resource with HardLink = false", func() { + adb := &dbv1alpha1.AutonomousDatabase{ + TypeMeta: metav1.TypeMeta{ + APIVersion: "database.oracle.com/v1alpha1", + Kind: "AutonomousDatabase", + }, + ObjectMeta: metav1.ObjectMeta{ + Name: "bindadb", + Namespace: ADBNamespace, + }, + Spec: dbv1alpha1.AutonomousDatabaseSpec{ + Details: dbv1alpha1.AutonomousDatabaseDetails{ + AutonomousDatabaseOCID: adbID, + Wallet: dbv1alpha1.WalletSpec{ + Name: common.String(downloadedWallet), + Password: dbv1alpha1.PasswordSpec{ + K8sSecretName: common.String(SharedWalletPassSecretName), + }, }, }, + HardLink: common.Bool(false), + OCIConfig: dbv1alpha1.OCIConfigSpec{ + ConfigMapName: common.String(SharedOCIConfigMapName), + SecretName: common.String(SharedOCISecretName), + }, }, - HardLink: common.Bool(false), - OCIConfig: dbv1alpha1.OCIConfigSpec{ - ConfigMapName: common.String(SharedOCIConfigMapName), - SecretName: common.String(SharedOCISecretName), - }, - }, - } + } - adbLookupKey = types.NamespacedName{Name: adb.Name, Namespace: adb.Namespace} + adbLookupKey = types.NamespacedName{Name: adb.Name, Namespace: adb.Namespace} - Expect(k8sClient.Create(context.TODO(), adb)).Should(Succeed()) - }) + Expect(k8sClient.Create(context.TODO(), adb)).Should(Succeed()) + }) - It("should bind to an ADB", e2ebehavior.AssertBind(&k8sClient, &adbLookupKey)) + It("should bind to an ADB", e2ebehavior.AssertBind(&k8sClient, &adbLookupKey)) - It("Should download an instance wallet using the password from K8s Secret "+SharedWalletPassSecretName, e2ebehavior.AssertWallet(&k8sClient, &adbLookupKey)) + It("Should download an instance wallet using the password from K8s Secret "+SharedWalletPassSecretName, e2ebehavior.AssertWallet(&k8sClient, &adbLookupKey)) - It("should update ADB", e2ebehavior.AssertUpdate(&k8sClient, &dbClient, &adbLookupKey)) + It("should update ADB", e2ebehavior.AssertUpdate(&k8sClient, &dbClient, &adbLookupKey)) - It("Should stop ADB", e2ebehavior.UpdateAndAssertState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateStopped)) + It("Should stop ADB", e2ebehavior.UpdateAndAssertState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateStopped)) - It("Should restart ADB", e2ebehavior.UpdateAndAssertState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateAvailable)) + It("Should restart ADB", e2ebehavior.UpdateAndAssertState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateAvailable)) - It("Should delete the resource in cluster but not terminate the database in OCI", e2ebehavior.AssertSoftLinkDelete(&k8sClient, &adbLookupKey)) - }) + It("Should delete the resource in cluster but not terminate the database in OCI", e2ebehavior.AssertSoftLinkDelete(&k8sClient, &adbLookupKey)) + }) - Describe("ADB binding with HardLink = true using Wallet Password OCID", func() { - It("Should create a AutonomousDatabase resource with HardLink = true", func() { - adb := &dbv1alpha1.AutonomousDatabase{ - TypeMeta: metav1.TypeMeta{ - APIVersion: "database.oracle.com/v1alpha1", - Kind: "AutonomousDatabase", - }, - ObjectMeta: metav1.ObjectMeta{ - Name: "bindadb", - Namespace: ADBNamespace, - }, - Spec: dbv1alpha1.AutonomousDatabaseSpec{ - Details: dbv1alpha1.AutonomousDatabaseDetails{ - AutonomousDatabaseOCID: adbID, - Wallet: dbv1alpha1.WalletSpec{ - Name: common.String(downloadedWallet), - Password: dbv1alpha1.PasswordSpec{ - OCISecretOCID: common.String(SharedInstanceWalletPasswordOCID), + Describe("ADB binding with HardLink = true using Wallet Password OCID", func() { + It("Should create a AutonomousDatabase resource with HardLink = true", func() { + adb := &dbv1alpha1.AutonomousDatabase{ + TypeMeta: metav1.TypeMeta{ + APIVersion: "database.oracle.com/v1alpha1", + Kind: "AutonomousDatabase", + }, + ObjectMeta: metav1.ObjectMeta{ + Name: "bindadb", + Namespace: ADBNamespace, + }, + Spec: dbv1alpha1.AutonomousDatabaseSpec{ + Details: dbv1alpha1.AutonomousDatabaseDetails{ + AutonomousDatabaseOCID: adbID, + Wallet: dbv1alpha1.WalletSpec{ + Name: common.String(downloadedWallet), + Password: dbv1alpha1.PasswordSpec{ + OCISecretOCID: common.String(SharedInstanceWalletPasswordOCID), + }, }, }, + HardLink: common.Bool(true), + OCIConfig: dbv1alpha1.OCIConfigSpec{ + ConfigMapName: common.String(SharedOCIConfigMapName), + SecretName: common.String(SharedOCISecretName), + }, }, - HardLink: common.Bool(true), - OCIConfig: dbv1alpha1.OCIConfigSpec{ - ConfigMapName: common.String(SharedOCIConfigMapName), - SecretName: common.String(SharedOCISecretName), - }, - }, - } + } - adbLookupKey = types.NamespacedName{Name: adb.Name, Namespace: adb.Namespace} + adbLookupKey = types.NamespacedName{Name: adb.Name, Namespace: adb.Namespace} - Expect(k8sClient.Create(context.TODO(), adb)).Should(Succeed()) - }) + Expect(k8sClient.Create(context.TODO(), adb)).Should(Succeed()) + }) - It("should bind to an ADB", e2ebehavior.AssertBind(&k8sClient, &adbLookupKey)) + It("should bind to an ADB", e2ebehavior.AssertBind(&k8sClient, &adbLookupKey)) - It("Should download an instance wallet using the password from OCI Secret OCID "+SharedInstanceWalletPasswordOCID, e2ebehavior.AssertWallet(&k8sClient, &adbLookupKey)) + It("Should download an instance wallet using the password from OCI Secret OCID "+SharedInstanceWalletPasswordOCID, e2ebehavior.AssertWallet(&k8sClient, &adbLookupKey)) - It("should update ADB", e2ebehavior.AssertUpdate(&k8sClient, &dbClient, &adbLookupKey)) + It("should update ADB", e2ebehavior.AssertUpdate(&k8sClient, &dbClient, &adbLookupKey)) - It("Should stop ADB", e2ebehavior.UpdateAndAssertState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateStopped)) + It("Should stop ADB", e2ebehavior.UpdateAndAssertState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateStopped)) - It("Should restart ADB", e2ebehavior.UpdateAndAssertState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateAvailable)) + It("Should restart ADB", e2ebehavior.UpdateAndAssertState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateAvailable)) - It("Should delete the resource in cluster and terminate the database in OCI", e2ebehavior.AssertHardLinkDelete(&k8sClient, &dbClient, &adbLookupKey)) - }) + It("Should delete the resource in cluster and terminate the database in OCI", e2ebehavior.AssertHardLinkDelete(&k8sClient, &dbClient, &adbLookupKey)) + }) - //Bind to terminated adb from previous test - Describe("bind to a terminated adb", func() { - - //Wait until remote state is terminated - It("Should check that OCI adb state is terminated", e2ebehavior.AssertRemoteStateOCID(&k8sClient, &dbClient, &terminatedAdbID, database.AutonomousDatabaseLifecycleStateTerminated)) - - It("Should create a AutonomousDatabase resource", func() { - adb := &dbv1alpha1.AutonomousDatabase{ - TypeMeta: metav1.TypeMeta{ - APIVersion: "database.oracle.com/v1alpha1", - Kind: "AutonomousDatabase", - }, - ObjectMeta: metav1.ObjectMeta{ - Name: "bindadb", - Namespace: ADBNamespace, - }, - Spec: dbv1alpha1.AutonomousDatabaseSpec{ - Details: dbv1alpha1.AutonomousDatabaseDetails{ - AutonomousDatabaseOCID: &terminatedAdbID, + //Bind to terminated adb from previous test + Describe("bind to a terminated adb", func() { + + //Wait until remote state is terminated + It("Should check that OCI adb state is terminated", e2ebehavior.AssertRemoteStateOCID(&k8sClient, &dbClient, &terminatedAdbID, database.AutonomousDatabaseLifecycleStateTerminated)) + + It("Should create a AutonomousDatabase resource", func() { + adb := &dbv1alpha1.AutonomousDatabase{ + TypeMeta: metav1.TypeMeta{ + APIVersion: "database.oracle.com/v1alpha1", + Kind: "AutonomousDatabase", }, - HardLink: common.Bool(true), - OCIConfig: dbv1alpha1.OCIConfigSpec{ - ConfigMapName: common.String(SharedOCIConfigMapName), - SecretName: common.String(SharedOCISecretName), + ObjectMeta: metav1.ObjectMeta{ + Name: "bindadb", + Namespace: ADBNamespace, }, - }, - } + Spec: dbv1alpha1.AutonomousDatabaseSpec{ + Details: dbv1alpha1.AutonomousDatabaseDetails{ + AutonomousDatabaseOCID: &terminatedAdbID, + }, + HardLink: common.Bool(true), + OCIConfig: dbv1alpha1.OCIConfigSpec{ + ConfigMapName: common.String(SharedOCIConfigMapName), + SecretName: common.String(SharedOCISecretName), + }, + }, + } - adbLookupKey = types.NamespacedName{Name: adb.Name, Namespace: adb.Namespace} + adbLookupKey = types.NamespacedName{Name: adb.Name, Namespace: adb.Namespace} - Expect(k8sClient.Create(context.TODO(), adb)).Should(Succeed()) - }) + Expect(k8sClient.Create(context.TODO(), adb)).Should(Succeed()) + }) - It("Should check for TERMINATED state in local resource", e2ebehavior.AssertLocalState(&k8sClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateTerminated)) + It("Should check for TERMINATED state in local resource", e2ebehavior.AssertLocalState(&k8sClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateTerminated)) - It("Should delete local resource", e2ebehavior.AssertSoftLinkDelete(&k8sClient, &adbLookupKey)) - }) + It("Should delete local resource", e2ebehavior.AssertSoftLinkDelete(&k8sClient, &adbLookupKey)) + }) - // Describe("Test ADB status", func() { - // var dbName string - // var backupID string - - // It("Should init the test", func() { - // By("creating a temp ADB in OCI for binding test") - // dbName = e2eutil.GenerateDBName() - // createResp, err := e2eutil.CreateAutonomousDatabase(dbClient, &SharedCompartmentOCID, &dbName, &SharedPlainTextAdminPassword) - // Expect(err).ShouldNot(HaveOccurred()) - // Expect(createResp.AutonomousDatabase.Id).ShouldNot(BeNil()) - - // By("Save the database ID for later use") - // adbID = createResp.AutonomousDatabase.Id - // backupID = *adbID - - // By("Wait until the work request is in SUCCEEDED status") - // workClient, err := workrequests.NewWorkRequestClientWithConfigurationProvider(configProvider) - // Expect(err).ShouldNot(HaveOccurred()) - - // err = e2eutil.WaitUntilWorkCompleted(workClient, createResp.OpcWorkRequestId) - // Expect(err).ShouldNot(HaveOccurred()) - // }) - - // It("Should create a AutonomousDatabase resource", func() { - // adb := &dbv1alpha1.AutonomousDatabase{ - // TypeMeta: metav1.TypeMeta{ - // APIVersion: "database.oracle.com/v1alpha1", - // Kind: "AutonomousDatabase", - // }, - // ObjectMeta: metav1.ObjectMeta{ - // Name: "bindadb", - // Namespace: ADBNamespace, - // }, - // Spec: dbv1alpha1.AutonomousDatabaseSpec{ - // Details: dbv1alpha1.AutonomousDatabaseDetails{ - // AutonomousDatabaseOCID: adbID, - // }, - // HardLink: common.Bool(true), - // OCIConfig: dbv1alpha1.OCIConfigSpec{ - // ConfigMapName: common.String(SharedOCIConfigMapName), - // SecretName: common.String(SharedOCISecretName), - // }, - // }, - // } - - // adbLookupKey = types.NamespacedName{Name: adb.Name, Namespace: adb.Namespace} - - // Expect(k8sClient.Create(context.TODO(), adb)).Should(Succeed()) - // }) - - // It("should bind to an ADB", e2ebehavior.AssertBind(&k8sClient, &adbLookupKey)) - - // It("should terminate ADB in a different routine", func() { - // err := e2eutil.DeleteAutonomousDatabase(dbClient, adbID) - // Expect(err).ToNot(HaveOccurred()) - // // By("Wait until the work request is in SUCCEEDED status") - // // workClient, err := workrequests.NewWorkRequestClientWithConfigurationProvider(configProvider) - // // Expect(err).ShouldNot(HaveOccurred()) - // // err = e2eutil.WaitUntilWorkCompleted(workClient, createResp.OpcWorkRequestId) - // // Expect(err).ShouldNot(HaveOccurred()) - // }) - - // It("should check for terminated state in OCI", e2ebehavior.AssertRemoteStateOCID(&k8sClient, &dbClient, &backupID, database.AutonomousDatabaseLifecycleStateTerminated)) - - // It("should check for terminated state in local resource", e2ebehavior.AssertLocalState(&k8sClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateTerminated)) - // }) -}) + // Describe("Test ADB status", func() { + // var dbName string + // var backupID string + + // It("Should init the test", func() { + // By("creating a temp ADB in OCI for binding test") + // dbName = e2eutil.GenerateDBName() + // createResp, err := e2eutil.CreateAutonomousDatabase(dbClient, &SharedCompartmentOCID, &dbName, &SharedPlainTextAdminPassword) + // Expect(err).ShouldNot(HaveOccurred()) + // Expect(createResp.AutonomousDatabase.Id).ShouldNot(BeNil()) + + // By("Save the database ID for later use") + // adbID = createResp.AutonomousDatabase.Id + // backupID = *adbID + + // By("Wait until the work request is in SUCCEEDED status") + // workClient, err := workrequests.NewWorkRequestClientWithConfigurationProvider(configProvider) + // Expect(err).ShouldNot(HaveOccurred()) + + // err = e2eutil.WaitUntilWorkCompleted(workClient, createResp.OpcWorkRequestId) + // Expect(err).ShouldNot(HaveOccurred()) + // }) + + // It("Should create a AutonomousDatabase resource", func() { + // adb := &dbv1alpha1.AutonomousDatabase{ + // TypeMeta: metav1.TypeMeta{ + // APIVersion: "database.oracle.com/v1alpha1", + // Kind: "AutonomousDatabase", + // }, + // ObjectMeta: metav1.ObjectMeta{ + // Name: "bindadb", + // Namespace: ADBNamespace, + // }, + // Spec: dbv1alpha1.AutonomousDatabaseSpec{ + // Details: dbv1alpha1.AutonomousDatabaseDetails{ + // AutonomousDatabaseOCID: adbID, + // }, + // HardLink: common.Bool(true), + // OCIConfig: dbv1alpha1.OCIConfigSpec{ + // ConfigMapName: common.String(SharedOCIConfigMapName), + // SecretName: common.String(SharedOCISecretName), + // }, + // }, + // } + + // adbLookupKey = types.NamespacedName{Name: adb.Name, Namespace: adb.Namespace} + + // Expect(k8sClient.Create(context.TODO(), adb)).Should(Succeed()) + // }) + + // It("should bind to an ADB", e2ebehavior.AssertBind(&k8sClient, &adbLookupKey)) + + // It("should terminate ADB in a different routine", func() { + // err := e2eutil.DeleteAutonomousDatabase(dbClient, adbID) + // Expect(err).ToNot(HaveOccurred()) + // // By("Wait until the work request is in SUCCEEDED status") + // // workClient, err := workrequests.NewWorkRequestClientWithConfigurationProvider(configProvider) + // // Expect(err).ShouldNot(HaveOccurred()) + // // err = e2eutil.WaitUntilWorkCompleted(workClient, createResp.OpcWorkRequestId) + // // Expect(err).ShouldNot(HaveOccurred()) + // }) + + // It("should check for terminated state in OCI", e2ebehavior.AssertRemoteStateOCID(&k8sClient, &dbClient, &backupID, database.AutonomousDatabaseLifecycleStateTerminated)) + + // It("should check for terminated state in local resource", e2ebehavior.AssertLocalState(&k8sClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateTerminated)) + // }) + }) +} diff --git a/test/e2e/autonomousdatabase_controller_suite_test.go b/test/e2e/autonomousdatabase_controller_suite_test.go new file mode 100644 index 00000000..adc6706f --- /dev/null +++ b/test/e2e/autonomousdatabase_controller_suite_test.go @@ -0,0 +1,145 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package e2etest + +import ( + "context" + + . "github.com/onsi/ginkgo" + . "github.com/onsi/gomega" + "github.com/oracle/oci-go-sdk/v45/common" + "github.com/oracle/oci-go-sdk/v45/database" + "sigs.k8s.io/controller-runtime/pkg/client" + + "github.com/oracle/oracle-database-operator/test/e2e/behavior" + "github.com/oracle/oracle-database-operator/test/e2e/util" + // +kubebuilder:scaffold:imports +) + +// These tests use Ginkgo (BDD-style Go testing framework). Refer to +// http://onsi.github.io/ginkgo/ to learn more about Ginkgo. + +/** +This test suite runs the integration test which checks the following scenario +1. Init the test by creating utililty variables and objects +2. Test ADB provisioned using adminPasswordOCID with hardLink=true +3. Test ADB provisioned using adminPasswordSecret with hardLink=true +5. Test ADB binding with hardLink=true +**/ + +var dbClient database.DatabaseClient +var configProvider common.ConfigurationProvider + +const configFileName = "test_config.yaml" +const ADBNamespace string = "default" + +var SharedOCIConfigMapName = "oci-cred" +var SharedOCISecretName = "oci-privatekey" +var SharedPlainTextAdminPassword = "Welcome_1234" +var SharedPlainTextWalletPassword = "Welcome_1234" +var SharedCompartmentOCID string + +var SharedKeyOCID string +var SharedAdminPasswordOCID string +var SharedInstanceWalletPasswordOCID string + +const SharedAdminPassSecretName string = "adb-admin-password" +const SharedWalletPassSecretName = "adb-wallet-password" + +func ADBSetup(k8sClient client.Client) { + /************************************************** + * Custom codes for autonomousdatabase controller + **************************************************/ + By("init the test by creating utililty variables and objects") + testConfig, err := e2eutil.GetTestConfig(configFileName) + Expect(err).ToNot(HaveOccurred()) + Expect(testConfig).ToNot(BeNil()) + + SharedCompartmentOCID = testConfig.CompartmentOCID + SharedAdminPasswordOCID = testConfig.AdminPasswordOCID + SharedInstanceWalletPasswordOCID = testConfig.InstanceWalletPasswordOCID + + By("checking if the required parameters exist") + Expect(testConfig.OCIConfigFile).ToNot(Equal("")) + Expect(testConfig.CompartmentOCID).ToNot(Equal("")) + Expect(testConfig.AdminPasswordOCID).ToNot(Equal("")) + Expect(testConfig.InstanceWalletPasswordOCID).ToNot(Equal("")) + + By("getting OCI provider") + ociConfigUtil, err := e2eutil.GetOCIConfigUtil(testConfig.OCIConfigFile, testConfig.Profile) + Expect(err).ToNot(HaveOccurred()) + configProvider, err = ociConfigUtil.GetConfigProvider() + Expect(err).ToNot(HaveOccurred()) + + By("creating a OCI DB client") + dbClient, err = database.NewDatabaseClientWithConfigurationProvider(configProvider) + Expect(err).ToNot(HaveOccurred()) + + By("creating a configMap for calling OCI") + ociConfigMap, err := ociConfigUtil.CreateOCIConfigMap(ADBNamespace, SharedOCIConfigMapName) + Expect(err).ToNot(HaveOccurred()) + Expect(k8sClient.Create(context.TODO(), ociConfigMap)).To(Succeed()) + + By("creating a secret for calling OCI") + ociSecret, err := ociConfigUtil.CreateOCISecret(ADBNamespace, SharedOCISecretName) + Expect(err).ToNot(HaveOccurred()) + Expect(k8sClient.Create(context.TODO(), ociSecret)).To(Succeed()) + + By("Creating a k8s secret to hold admin password", func() { + data := map[string]string{ + SharedAdminPassSecretName: SharedPlainTextAdminPassword, + } + adminSecret, err := e2eutil.CreateKubeSecret(ADBNamespace, SharedAdminPassSecretName, data) + Expect(err).ToNot(HaveOccurred()) + Expect(k8sClient.Create(context.TODO(), adminSecret)).To(Succeed()) + }) + + By("Creating a k8s secret to hold wallet password", func() { + data := map[string]string{ + SharedWalletPassSecretName: SharedPlainTextWalletPassword, + } + walletSecret, err := e2eutil.CreateKubeSecret(ADBNamespace, SharedWalletPassSecretName, data) + Expect(err).ToNot(HaveOccurred()) + Expect(k8sClient.Create(context.TODO(), walletSecret)).To(Succeed()) + }) +} + +func CleanupADB(k8sClient *client.Client) { + e2ebehavior.CleanupDB(k8sClient, &dbClient, ADBNamespace) +} diff --git a/test/e2e/suite_test.go b/test/e2e/suite_test.go index 3fd5cd2e..060d1ed7 100644 --- a/test/e2e/suite_test.go +++ b/test/e2e/suite_test.go @@ -39,7 +39,7 @@ package e2etest import ( - "context" + "os" "path/filepath" "testing" "time" @@ -47,8 +47,6 @@ import ( . "github.com/onsi/ginkgo" . "github.com/onsi/gomega" "github.com/onsi/gomega/gexec" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/database" corev1 "k8s.io/api/core/v1" "k8s.io/client-go/kubernetes/scheme" "k8s.io/client-go/rest" @@ -61,8 +59,6 @@ import ( databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" controllers "github.com/oracle/oracle-database-operator/controllers/database" - "github.com/oracle/oracle-database-operator/test/e2e/behavior" - "github.com/oracle/oracle-database-operator/test/e2e/util" // +kubebuilder:scaffold:imports ) @@ -79,26 +75,8 @@ This test suite runs the integration test which checks the following scenario var cfg *rest.Config var k8sClient client.Client -var configProvider common.ConfigurationProvider -var dbClient database.DatabaseClient var testEnv *envtest.Environment -const configFileName = "test_config.yaml" -const ADBNamespace string = "default" - -var SharedOCIConfigMapName = "oci-cred" -var SharedOCISecretName = "oci-privatekey" -var SharedPlainTextAdminPassword = "Welcome_1234" -var SharedPlainTextWalletPassword = "Welcome_1234" -var SharedCompartmentOCID string - -var SharedKeyOCID string -var SharedAdminPasswordOCID string -var SharedInstanceWalletPasswordOCID string - -const SharedAdminPassSecretName string = "adb-admin-password" -const SharedWalletPassSecretName = "adb-wallet-password" - func TestAPIs(t *testing.T) { RegisterFailHandler(Fail) @@ -111,8 +89,16 @@ var _ = BeforeSuite(func(done Done) { logf.SetLogger(zap.New(zap.WriteTo(GinkgoWriter), zap.UseDevMode(true))) By("bootstrapping test environment") - testEnv = &envtest.Environment{ - CRDDirectoryPaths: []string{filepath.Join("../..", "config", "crd", "bases")}, + t := true + + if os.Getenv("TEST_USE_EXISTING_CLUSTER") == "true" { + testEnv = &envtest.Environment{ + UseExistingCluster: &t, + } + } else { + testEnv = &envtest.Environment{ + CRDDirectoryPaths: []string{filepath.Join("../..", "config", "crd", "bases")}, + } } var err error @@ -144,6 +130,17 @@ var _ = BeforeSuite(func(done Done) { }).SetupWithManager(k8sManager) Expect(err).ToNot(HaveOccurred()) + // CDB Reconciler + err = (&controllers.CDBReconciler{ + Client: k8sManager.GetClient(), + Scheme: k8sManager.GetScheme(), + Config: k8sManager.GetConfig(), + Log: ctrl.Log.WithName("controllers").WithName("CDB_test"), + Interval: time.Duration(15), + Recorder: k8sManager.GetEventRecorderFor("CDB"), + }).SetupWithManager(k8sManager) + Expect(err).ToNot(HaveOccurred()) + go func() { defer GinkgoRecover() err = k8sManager.Start(ctrl.SetupSignalHandler()) @@ -160,59 +157,7 @@ var _ = BeforeSuite(func(done Done) { /************************************************** * Custom codes for autonomousdatabase controller **************************************************/ - By("init the test by creating utililty variables and objects") - testConfig, err := e2eutil.GetTestConfig(configFileName) - Expect(err).ToNot(HaveOccurred()) - Expect(testConfig).ToNot(BeNil()) - - SharedCompartmentOCID = testConfig.CompartmentOCID - SharedAdminPasswordOCID = testConfig.AdminPasswordOCID - SharedInstanceWalletPasswordOCID = testConfig.InstanceWalletPasswordOCID - - By("checking if the required parameters exist") - Expect(testConfig.OCIConfigFile).ToNot(Equal("")) - Expect(testConfig.CompartmentOCID).ToNot(Equal("")) - Expect(testConfig.AdminPasswordOCID).ToNot(Equal("")) - Expect(testConfig.InstanceWalletPasswordOCID).ToNot(Equal("")) - - By("getting OCI provider") - ociConfigUtil, err := e2eutil.GetOCIConfigUtil(testConfig.OCIConfigFile, testConfig.Profile) - Expect(err).ToNot(HaveOccurred()) - configProvider, err = ociConfigUtil.GetConfigProvider() - Expect(err).ToNot(HaveOccurred()) - - By("creating a OCI DB client") - dbClient, err = database.NewDatabaseClientWithConfigurationProvider(configProvider) - Expect(err).ToNot(HaveOccurred()) - - By("creating a configMap for calling OCI") - ociConfigMap, err := ociConfigUtil.CreateOCIConfigMap(ADBNamespace, SharedOCIConfigMapName) - Expect(err).ToNot(HaveOccurred()) - Expect(k8sClient.Create(context.TODO(), ociConfigMap)).To(Succeed()) - - By("creating a secret for calling OCI") - ociSecret, err := ociConfigUtil.CreateOCISecret(ADBNamespace, SharedOCISecretName) - Expect(err).ToNot(HaveOccurred()) - Expect(k8sClient.Create(context.TODO(), ociSecret)).To(Succeed()) - - By("Creating a k8s secret to hold admin password", func() { - data := map[string]string{ - SharedAdminPassSecretName: SharedPlainTextAdminPassword, - } - adminSecret, err := e2eutil.CreateKubeSecret(ADBNamespace, SharedAdminPassSecretName, data) - Expect(err).ToNot(HaveOccurred()) - Expect(k8sClient.Create(context.TODO(), adminSecret)).To(Succeed()) - }) - - By("Creating a k8s secret to hold wallet password", func() { - data := map[string]string{ - SharedWalletPassSecretName: SharedPlainTextWalletPassword, - } - walletSecret, err := e2eutil.CreateKubeSecret(ADBNamespace, SharedWalletPassSecretName, data) - Expect(err).ToNot(HaveOccurred()) - Expect(k8sClient.Create(context.TODO(), walletSecret)).To(Succeed()) - }) - + //ADBSetup(k8sClient) close(done) }, 60) @@ -232,5 +177,5 @@ var _ = AfterSuite(func() { */ By("Deleting the resources that are created during the tests") - e2ebehavior.CleanupDB(&k8sClient, &dbClient, ADBNamespace) + //CleanupADB(&k8sClient) }) From dcc75374ca1f926c858de5b50908d98415013a26 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Tue, 4 Jan 2022 13:50:13 +0000 Subject: [PATCH 135/628] Deleted config/samples/adb_backup/autonomousdatabasebackup.yaml, config/samples/database_v1alpha1_autonomousdatabaserestore.yaml files --- config/samples/adb_backup/autonomousdatabasebackup.yaml | 7 ------- .../database_v1alpha1_autonomousdatabaserestore.yaml | 7 ------- 2 files changed, 14 deletions(-) delete mode 100644 config/samples/adb_backup/autonomousdatabasebackup.yaml delete mode 100644 config/samples/database_v1alpha1_autonomousdatabaserestore.yaml diff --git a/config/samples/adb_backup/autonomousdatabasebackup.yaml b/config/samples/adb_backup/autonomousdatabasebackup.yaml deleted file mode 100644 index 93da9f77..00000000 --- a/config/samples/adb_backup/autonomousdatabasebackup.yaml +++ /dev/null @@ -1,7 +0,0 @@ -apiVersion: database.oracle.com/v1alpha1 -kind: AutonomousDatabaseBackup -metadata: - name: autonomousdatabasebackup-sample -spec: - # Add fields here - foo: bar diff --git a/config/samples/database_v1alpha1_autonomousdatabaserestore.yaml b/config/samples/database_v1alpha1_autonomousdatabaserestore.yaml deleted file mode 100644 index 4e1cb9a1..00000000 --- a/config/samples/database_v1alpha1_autonomousdatabaserestore.yaml +++ /dev/null @@ -1,7 +0,0 @@ -apiVersion: database.oracle.com/v1alpha1 -kind: AutonomousDatabaseRestore -metadata: - name: autonomousdatabaserestore-sample -spec: - # Add fields here - foo: bar From f045a52d26b92564c8541df363dee33795e8f1a3 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Wed, 5 Jan 2022 16:00:50 -0500 Subject: [PATCH 136/628] resturcture network apis --- .../v1alpha1/autonomousdatabase_types.go | 41 +++++--- .../v1alpha1/zz_generated.deepcopy.go | 93 +++++++++++++------ commons/oci/database.go | 37 ++++---- ...tabase.oracle.com_autonomousdatabases.yaml | 42 +++++---- test/e2e/behavior/shared_behaviors.go | 12 +-- 5 files changed, 144 insertions(+), 81 deletions(-) diff --git a/apis/database/v1alpha1/autonomousdatabase_types.go b/apis/database/v1alpha1/autonomousdatabase_types.go index b8759348..67bfa043 100644 --- a/apis/database/v1alpha1/autonomousdatabase_types.go +++ b/apis/database/v1alpha1/autonomousdatabase_types.go @@ -82,12 +82,7 @@ type AutonomousDatabaseDetails struct { IsAutoScalingEnabled *bool `json:"isAutoScalingEnabled,omitempty"` LifecycleState database.AutonomousDatabaseLifecycleStateEnum `json:"lifecycleState,omitempty"` - SubnetOCID *string `json:"subnetOCID,omitempty"` - NsgOCIDs []string `json:"nsgOCIDs,omitempty"` - IsAccessControlEnabled *bool `json:"isAccessControlEnabled,omitempty"` - WhitelistedIPs []string `json:"whitelistedIPs,omitempty"` - IsMTLSConnectionRequired *bool `json:"isMTLSConnectionRequired,omitempty"` - PrivateEndpointLabel *string `json:"privateEndpointLabel,omitempty"` + NetworkAccess NetworkAccessSpec `json:"networkAccess,omitempty"` FreeformTags map[string]string `json:"freeformTags,omitempty"` @@ -104,6 +99,28 @@ type PasswordSpec struct { OCISecretOCID *string `json:"ociSecretOCID,omitempty"` } +type NetworkAccessTypeEnum string + +const ( + NetworkAccessTypePublic NetworkAccessTypeEnum = "PUBLIC" + NetworkAccessTypeRestricted NetworkAccessTypeEnum = "RESTRICTED" + NetworkAccessTypePrivate NetworkAccessTypeEnum = "PRIVATE" +) + +type NetworkAccessSpec struct { + AccessType NetworkAccessTypeEnum `json:"accessType"` + IsAccessControlEnabled *bool `json:"isAccessControlEnabled,omitempty"` + AccessControlList []string `json:"accessControlList,omitempty"` + PrivateEndpoint PrivateEndpointSpec `json:"privateEndpoint,omitempty"` + IsMTLSConnectionRequired *bool `json:"isMTLSConnectionRequired,omitempty"` +} + +type PrivateEndpointSpec struct { + SubnetOCID *string `json:"subnetOCID,omitempty"` + NsgOCIDs []string `json:"nsgOCIDs,omitempty"` + HostnamePrefix *string `json:"hostnamePrefix,omitempty"` +} + // AutonomousDatabaseStatus defines the observed state of AutonomousDatabase type AutonomousDatabaseStatus struct { // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster @@ -187,12 +204,12 @@ func (adb *AutonomousDatabase) UpdateAttrFromOCIAutonomousDatabase(ociObj databa adb.Spec.Details.LifecycleState = ociObj.LifecycleState adb.Spec.Details.FreeformTags = ociObj.FreeformTags - adb.Spec.Details.SubnetOCID = ociObj.SubnetId - adb.Spec.Details.NsgOCIDs = ociObj.NsgIds - adb.Spec.Details.IsAccessControlEnabled = ociObj.IsAccessControlEnabled - adb.Spec.Details.WhitelistedIPs = ociObj.WhitelistedIps - adb.Spec.Details.IsMTLSConnectionRequired = ociObj.IsMtlsConnectionRequired - adb.Spec.Details.PrivateEndpointLabel = ociObj.PrivateEndpointLabel + adb.Spec.Details.NetworkAccess.IsAccessControlEnabled = ociObj.IsAccessControlEnabled + adb.Spec.Details.NetworkAccess.AccessControlList = ociObj.WhitelistedIps + adb.Spec.Details.NetworkAccess.IsMTLSConnectionRequired = ociObj.IsMtlsConnectionRequired + adb.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID = ociObj.SubnetId + adb.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs = ociObj.NsgIds + adb.Spec.Details.NetworkAccess.PrivateEndpoint.HostnamePrefix = ociObj.PrivateEndpointLabel // update the subresource as well adb.Status.DisplayName = *ociObj.DisplayName diff --git a/apis/database/v1alpha1/zz_generated.deepcopy.go b/apis/database/v1alpha1/zz_generated.deepcopy.go index 4d27bb59..e60ce721 100644 --- a/apis/database/v1alpha1/zz_generated.deepcopy.go +++ b/apis/database/v1alpha1/zz_generated.deepcopy.go @@ -1,3 +1,4 @@ +//go:build !ignore_autogenerated // +build !ignore_autogenerated /* @@ -124,36 +125,7 @@ func (in *AutonomousDatabaseDetails) DeepCopyInto(out *AutonomousDatabaseDetails *out = new(bool) **out = **in } - if in.SubnetOCID != nil { - in, out := &in.SubnetOCID, &out.SubnetOCID - *out = new(string) - **out = **in - } - if in.NsgOCIDs != nil { - in, out := &in.NsgOCIDs, &out.NsgOCIDs - *out = make([]string, len(*in)) - copy(*out, *in) - } - if in.IsAccessControlEnabled != nil { - in, out := &in.IsAccessControlEnabled, &out.IsAccessControlEnabled - *out = new(bool) - **out = **in - } - if in.WhitelistedIPs != nil { - in, out := &in.WhitelistedIPs, &out.WhitelistedIPs - *out = make([]string, len(*in)) - copy(*out, *in) - } - if in.IsMTLSConnectionRequired != nil { - in, out := &in.IsMTLSConnectionRequired, &out.IsMTLSConnectionRequired - *out = new(bool) - **out = **in - } - if in.PrivateEndpointLabel != nil { - in, out := &in.PrivateEndpointLabel, &out.PrivateEndpointLabel - *out = new(string) - **out = **in - } + in.NetworkAccess.DeepCopyInto(&out.NetworkAccess) if in.FreeformTags != nil { in, out := &in.FreeformTags, &out.FreeformTags *out = make(map[string]string, len(*in)) @@ -397,6 +369,37 @@ func (in *GsmStatusDetails) DeepCopy() *GsmStatusDetails { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *NetworkAccessSpec) DeepCopyInto(out *NetworkAccessSpec) { + *out = *in + if in.IsAccessControlEnabled != nil { + in, out := &in.IsAccessControlEnabled, &out.IsAccessControlEnabled + *out = new(bool) + **out = **in + } + if in.AccessControlList != nil { + in, out := &in.AccessControlList, &out.AccessControlList + *out = make([]string, len(*in)) + copy(*out, *in) + } + in.PrivateEndpoint.DeepCopyInto(&out.PrivateEndpoint) + if in.IsMTLSConnectionRequired != nil { + in, out := &in.IsMTLSConnectionRequired, &out.IsMTLSConnectionRequired + *out = new(bool) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new NetworkAccessSpec. +func (in *NetworkAccessSpec) DeepCopy() *NetworkAccessSpec { + if in == nil { + return nil + } + out := new(NetworkAccessSpec) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *OCIConfigSpec) DeepCopyInto(out *OCIConfigSpec) { *out = *in @@ -462,6 +465,36 @@ func (in *PortMapping) DeepCopy() *PortMapping { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PrivateEndpointSpec) DeepCopyInto(out *PrivateEndpointSpec) { + *out = *in + if in.SubnetOCID != nil { + in, out := &in.SubnetOCID, &out.SubnetOCID + *out = new(string) + **out = **in + } + if in.NsgOCIDs != nil { + in, out := &in.NsgOCIDs, &out.NsgOCIDs + *out = make([]string, len(*in)) + copy(*out, *in) + } + if in.HostnamePrefix != nil { + in, out := &in.HostnamePrefix, &out.HostnamePrefix + *out = new(string) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PrivateEndpointSpec. +func (in *PrivateEndpointSpec) DeepCopy() *PrivateEndpointSpec { + if in == nil { + return nil + } + out := new(PrivateEndpointSpec) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *ShardSpec) DeepCopyInto(out *ShardSpec) { *out = *in diff --git a/commons/oci/database.go b/commons/oci/database.go index 3fdf7a00..d38aff1c 100644 --- a/commons/oci/database.go +++ b/commons/oci/database.go @@ -78,11 +78,12 @@ func CreateAutonomousDatabase(logger logr.Logger, kubeClient client.Client, dbCl DbWorkload: database.CreateAutonomousDatabaseBaseDbWorkloadEnum( adb.Spec.Details.DbWorkload), - WhitelistedIps: adb.Spec.Details.WhitelistedIPs, - SubnetId: adb.Spec.Details.SubnetOCID, - NsgIds: adb.Spec.Details.NsgOCIDs, - PrivateEndpointLabel: adb.Spec.Details.PrivateEndpointLabel, - IsMtlsConnectionRequired: adb.Spec.Details.IsMTLSConnectionRequired, + IsAccessControlEnabled: adb.Spec.Details.NetworkAccess.IsAccessControlEnabled, + WhitelistedIps: adb.Spec.Details.NetworkAccess.AccessControlList, + IsMtlsConnectionRequired: adb.Spec.Details.NetworkAccess.IsMTLSConnectionRequired, + SubnetId: adb.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID, + NsgIds: adb.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs, + PrivateEndpointLabel: adb.Spec.Details.NetworkAccess.PrivateEndpoint.HostnamePrefix, FreeformTags: adb.Spec.Details.FreeformTags, } @@ -368,6 +369,8 @@ func UpdateScaleAttributes(logger logr.Logger, kubeClient client.Client, dbClien return } + + func UpdateOneWayTLSAttribute(logger logr.Logger, kubeClient client.Client, dbClient database.DatabaseClient, curADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { var shouldSendRequest = false @@ -380,8 +383,8 @@ func UpdateOneWayTLSAttribute(logger logr.Logger, kubeClient client.Client, dbCl // Prepare the update request updateAutonomousDatabaseDetails := database.UpdateAutonomousDatabaseDetails{} - if isAttrChanged(lastSucSpec.Details.IsMTLSConnectionRequired, curADB.Spec.Details.IsMTLSConnectionRequired) { - updateAutonomousDatabaseDetails.IsMtlsConnectionRequired = curADB.Spec.Details.IsMTLSConnectionRequired + if isAttrChanged(lastSucSpec.Details.NetworkAccess.IsMTLSConnectionRequired, curADB.Spec.Details.NetworkAccess.IsMTLSConnectionRequired) { + updateAutonomousDatabaseDetails.IsMtlsConnectionRequired = curADB.Spec.Details.NetworkAccess.IsMTLSConnectionRequired shouldSendRequest = true } @@ -414,25 +417,25 @@ func UpdateNetworkAttributes(logger logr.Logger, kubeClient client.Client, dbCli updateAutonomousDatabaseDetails := database.UpdateAutonomousDatabaseDetails{} // Network settings - if isAttrChanged(lastSucSpec.Details.SubnetOCID, curADB.Spec.Details.SubnetOCID) { - updateAutonomousDatabaseDetails.SubnetId = curADB.Spec.Details.SubnetOCID + if isAttrChanged(lastSucSpec.Details.NetworkAccess.IsAccessControlEnabled, curADB.Spec.Details.NetworkAccess.IsAccessControlEnabled) { + updateAutonomousDatabaseDetails.IsAccessControlEnabled = curADB.Spec.Details.NetworkAccess.IsAccessControlEnabled shouldSendRequest = true } - if isAttrChanged(lastSucSpec.Details.NsgOCIDs, curADB.Spec.Details.NsgOCIDs) { - updateAutonomousDatabaseDetails.NsgIds = curADB.Spec.Details.NsgOCIDs + if isAttrChanged(lastSucSpec.Details.NetworkAccess.AccessControlList, curADB.Spec.Details.NetworkAccess.AccessControlList) { + updateAutonomousDatabaseDetails.WhitelistedIps = curADB.Spec.Details.NetworkAccess.AccessControlList shouldSendRequest = true } - if isAttrChanged(lastSucSpec.Details.IsAccessControlEnabled, curADB.Spec.Details.IsAccessControlEnabled) { - updateAutonomousDatabaseDetails.IsAccessControlEnabled = curADB.Spec.Details.IsAccessControlEnabled + if isAttrChanged(lastSucSpec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID, curADB.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID) { + updateAutonomousDatabaseDetails.SubnetId = curADB.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID shouldSendRequest = true } - if isAttrChanged(lastSucSpec.Details.WhitelistedIPs, curADB.Spec.Details.WhitelistedIPs) { - updateAutonomousDatabaseDetails.WhitelistedIps = curADB.Spec.Details.WhitelistedIPs + if isAttrChanged(lastSucSpec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs, curADB.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs) { + updateAutonomousDatabaseDetails.NsgIds = curADB.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs shouldSendRequest = true } // PrivateEndpointLabel - if isAttrChanged(lastSucSpec.Details.PrivateEndpointLabel, curADB.Spec.Details.PrivateEndpointLabel) { - updateAutonomousDatabaseDetails.PrivateEndpointLabel = curADB.Spec.Details.PrivateEndpointLabel + if isAttrChanged(lastSucSpec.Details.NetworkAccess.PrivateEndpoint.HostnamePrefix, curADB.Spec.Details.NetworkAccess.PrivateEndpoint.HostnamePrefix) { + updateAutonomousDatabaseDetails.PrivateEndpointLabel = curADB.Spec.Details.NetworkAccess.PrivateEndpoint.HostnamePrefix shouldSendRequest = true } diff --git a/config/crd/bases/database.oracle.com_autonomousdatabases.yaml b/config/crd/bases/database.oracle.com_autonomousdatabases.yaml index dbce1bfa..0f428fe4 100644 --- a/config/crd/bases/database.oracle.com_autonomousdatabases.yaml +++ b/config/crd/bases/database.oracle.com_autonomousdatabases.yaml @@ -101,26 +101,40 @@ spec: additionalProperties: type: string type: object - isAccessControlEnabled: - type: boolean isAutoScalingEnabled: type: boolean isDedicated: type: boolean - isMTLSConnectionRequired: - type: boolean lifecycleState: description: 'AutonomousDatabaseLifecycleStateEnum Enum with underlying type: string' type: string - nsgOCIDs: - items: - type: string - type: array - privateEndpointLabel: - type: string - subnetOCID: - type: string + networkAccess: + properties: + accessControlList: + items: + type: string + type: array + accessType: + type: string + isAccessControlEnabled: + type: boolean + isMTLSConnectionRequired: + type: boolean + privateEndpoint: + properties: + hostnamePrefix: + type: string + nsgOCIDs: + items: + type: string + type: array + subnetOCID: + type: string + type: object + required: + - accessType + type: object wallet: properties: name: @@ -133,10 +147,6 @@ spec: type: string type: object type: object - whitelistedIPs: - items: - type: string - type: array type: object hardLink: default: false diff --git a/test/e2e/behavior/shared_behaviors.go b/test/e2e/behavior/shared_behaviors.go index 593bc378..5f0ef7d7 100644 --- a/test/e2e/behavior/shared_behaviors.go +++ b/test/e2e/behavior/shared_behaviors.go @@ -313,12 +313,12 @@ func AssertADBDetails(k8sClient *client.Client, dbClient *database.DatabaseClien compareInt(expectedADBDetails.CPUCoreCount, resp.AutonomousDatabase.CpuCoreCount) && compareBool(expectedADBDetails.IsAutoScalingEnabled, resp.AutonomousDatabase.IsAutoScalingEnabled) && compareStringMap(expectedADBDetails.FreeformTags, resp.AutonomousDatabase.FreeformTags) && - compareString(expectedADBDetails.SubnetOCID, resp.AutonomousDatabase.SubnetId) && - reflect.DeepEqual(expectedADBDetails.NsgOCIDs, resp.AutonomousDatabase.NsgIds) && - compareBool(expectedADBDetails.IsAccessControlEnabled, resp.AutonomousDatabase.IsAccessControlEnabled) && - reflect.DeepEqual(expectedADBDetails.WhitelistedIPs, resp.AutonomousDatabase.WhitelistedIps) && - compareBool(expectedADBDetails.IsMTLSConnectionRequired, resp.AutonomousDatabase.IsMtlsConnectionRequired) && - compareString(expectedADBDetails.PrivateEndpointLabel, resp.AutonomousDatabase.PrivateEndpointLabel) + compareBool(expectedADBDetails.NetworkAccess.IsAccessControlEnabled, resp.AutonomousDatabase.IsAccessControlEnabled) && + reflect.DeepEqual(expectedADBDetails.NetworkAccess.AccessControlList, resp.AutonomousDatabase.WhitelistedIps) && + compareBool(expectedADBDetails.NetworkAccess.IsMTLSConnectionRequired, resp.AutonomousDatabase.IsMtlsConnectionRequired) && + compareString(expectedADBDetails.NetworkAccess.PrivateEndpoint.SubnetOCID, resp.AutonomousDatabase.SubnetId) && + reflect.DeepEqual(expectedADBDetails.NetworkAccess.PrivateEndpoint.NsgOCIDs, resp.AutonomousDatabase.NsgIds) && + compareString(expectedADBDetails.NetworkAccess.PrivateEndpoint.HostnamePrefix, resp.AutonomousDatabase.PrivateEndpointLabel) return same, nil }, updateTimeout, updateInterval).Should(BeTrue()) From b9a72f6151ef9799a29d11d07a38933546f28c29 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Thu, 6 Jan 2022 15:03:56 -0500 Subject: [PATCH 137/628] update new crds --- config/manager/kustomization.yaml | 4 +- oracle-database-operator.yaml | 249 +++++++++++++++++++++++++++++- 2 files changed, 247 insertions(+), 6 deletions(-) diff --git a/config/manager/kustomization.yaml b/config/manager/kustomization.yaml index d8632b06..30ed1f75 100644 --- a/config/manager/kustomization.yaml +++ b/config/manager/kustomization.yaml @@ -8,5 +8,5 @@ apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization images: - name: controller - newName: local-oracle-db-operator - newTag: v0.0.1beta + newName: container-registry.oracle.com/database/operator + newTag: 0.1.0 diff --git a/oracle-database-operator.yaml b/oracle-database-operator.yaml index ae464a57..2b12d725 100644 --- a/oracle-database-operator.yaml +++ b/oracle-database-operator.yaml @@ -7,6 +7,193 @@ metadata: --- apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.6.1 + creationTimestamp: null + name: autonomousdatabasebackups.database.oracle.com +spec: + group: database.oracle.com + names: + kind: AutonomousDatabaseBackup + listKind: AutonomousDatabaseBackupList + plural: autonomousdatabasebackups + shortNames: + - adbbu + - adbbus + singular: autonomousdatabasebackup + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .status.lifecycleState + name: State + type: string + - jsonPath: .status.type + name: Type + type: string + - jsonPath: .status.timeStarted + name: Started + type: string + - jsonPath: .status.timeEnded + name: Ended + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + description: AutonomousDatabaseBackup is the Schema for the autonomousdatabasebackups API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: AutonomousDatabaseBackupSpec defines the desired state of AutonomousDatabaseBackup + properties: + autonomousDatabaseBackupOCID: + type: string + autonomousDatabaseOCID: + description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster Important: Run "make" to regenerate code after modifying this file' + type: string + ociConfig: + properties: + configMapName: + type: string + secretName: + type: string + type: object + required: + - autonomousDatabaseOCID + type: object + status: + description: AutonomousDatabaseBackupStatus defines the observed state of AutonomousDatabaseBackup + properties: + autonomousDatabaseBackupOCID: + description: 'INSERT ADDITIONAL STATUS FIELD - define observed state of cluster Important: Run "make" to regenerate code after modifying this file' + type: string + autonomousDatabaseOCID: + type: string + compartmentOCID: + type: string + isAutomatic: + type: boolean + lifecycleState: + description: 'AutonomousDatabaseBackupLifecycleStateEnum Enum with underlying type: string' + type: string + timeEnded: + type: string + timeStarted: + type: string + type: + description: 'AutonomousDatabaseBackupTypeEnum Enum with underlying type: string' + type: string + required: + - autonomousDatabaseBackupOCID + - autonomousDatabaseOCID + - compartmentOCID + - isAutomatic + - lifecycleState + - type + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.6.1 + creationTimestamp: null + name: autonomousdatabaserestores.database.oracle.com +spec: + group: database.oracle.com + names: + kind: AutonomousDatabaseRestore + listKind: AutonomousDatabaseRestoreList + plural: autonomousdatabaserestores + shortNames: + - adbr + - adbrs + singular: autonomousdatabaserestore + scope: Namespaced + versions: + - name: v1alpha1 + schema: + openAPIV3Schema: + description: AutonomousDatabaseRestore is the Schema for the autonomousdatabaserestores API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: AutonomousDatabaseRestoreSpec defines the desired state of AutonomousDatabaseRestore + properties: + autonomousDatabaseOCID: + description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster Important: Run "make" to regenerate code after modifying this file' + type: string + destination: + properties: + backupName: + type: string + timeStamp: + type: string + type: object + ociConfig: + properties: + configMapName: + type: string + secretName: + type: string + type: object + required: + - autonomousDatabaseOCID + - destination + type: object + status: + description: AutonomousDatabaseRestoreStatus defines the observed state of AutonomousDatabaseRestore + properties: + state: + description: 'INSERT ADDITIONAL STATUS FIELD - define observed state of cluster Important: Run "make" to regenerate code after modifying this file' + enum: + - NewCreated + - InProgress + - Completed + - Failed + type: string + required: + - state + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition metadata: annotations: controller-gen.kubebuilder.io/version: v0.6.1 @@ -98,10 +285,14 @@ spec: additionalProperties: type: string type: object + isAccessControlEnabled: + type: boolean isAutoScalingEnabled: type: boolean isDedicated: type: boolean + isMTLSConnectionRequired: + type: boolean lifecycleState: description: 'AutonomousDatabaseLifecycleStateEnum Enum with underlying type: string' type: string @@ -109,10 +300,6 @@ spec: items: type: string type: array - privateEndpoint: - type: string - privateEndpointIP: - type: string privateEndpointLabel: type: string subnetOCID: @@ -129,6 +316,10 @@ spec: type: string type: object type: object + whitelistedIPs: + items: + type: string + type: array type: object hardLink: default: false @@ -1005,6 +1196,56 @@ rules: - pods/exec verbs: - create +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabaseBackups + verbs: + - create + - delete + - get + - list + - update +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabasebackups + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabasebackups/status + verbs: + - get + - patch + - update +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabaserestores + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabaserestores/status + verbs: + - get + - patch + - update - apiGroups: - database.oracle.com resources: From a8709aab23cff5247074726715d868c731e661ec Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Fri, 7 Jan 2022 11:05:01 -0500 Subject: [PATCH 138/628] update latest versions --- Dockerfile | 2 +- .../v1alpha1/autonomousdatabase_types.go | 2 +- commons/autonomousdatabase/reconciler_util.go | 6 +- commons/oci/database.go | 8 +- commons/oci/provider.go | 4 +- commons/oci/vault.go | 4 +- commons/oci/wallet.go | 6 +- commons/sharding/catalog.go | 2 +- commons/sharding/gsm.go | 2 +- commons/sharding/scommon.go | 4 +- commons/sharding/shard.go | 2 +- .../database/autonomousdatabase_controller.go | 6 +- .../database/shardingdatabase_controller.go | 4 +- .../singleinstancedatabase_controller.go | 4 +- go.mod | 73 ++- go.sum | 458 +++++++++++++----- ...autonomousdatabase_controller_bind_test.go | 6 +- ...tonomousdatabase_controller_create_test.go | 4 +- test/e2e/behavior/shared_behaviors.go | 4 +- test/e2e/suite_test.go | 4 +- test/e2e/util/oci_config_util.go | 2 +- test/e2e/util/oci_db_request.go | 4 +- test/e2e/util/oci_work_request.go | 4 +- 23 files changed, 444 insertions(+), 171 deletions(-) diff --git a/Dockerfile b/Dockerfile index 0c181a48..dc6742f9 100644 --- a/Dockerfile +++ b/Dockerfile @@ -3,7 +3,7 @@ # # Build the manager binary -FROM golang:1.16 as builder +FROM golang:1.17 as builder WORKDIR /workspace # Copy the Go Modules manifests diff --git a/apis/database/v1alpha1/autonomousdatabase_types.go b/apis/database/v1alpha1/autonomousdatabase_types.go index b8759348..c9b524e3 100644 --- a/apis/database/v1alpha1/autonomousdatabase_types.go +++ b/apis/database/v1alpha1/autonomousdatabase_types.go @@ -42,7 +42,7 @@ import ( "encoding/json" "strconv" - "github.com/oracle/oci-go-sdk/v51/database" + "github.com/oracle/oci-go-sdk/v54/database" metaV1 "k8s.io/apimachinery/pkg/apis/meta/v1" "sigs.k8s.io/controller-runtime/pkg/client" diff --git a/commons/autonomousdatabase/reconciler_util.go b/commons/autonomousdatabase/reconciler_util.go index 2f820410..d081e43c 100644 --- a/commons/autonomousdatabase/reconciler_util.go +++ b/commons/autonomousdatabase/reconciler_util.go @@ -49,9 +49,9 @@ import ( "sigs.k8s.io/controller-runtime/pkg/client" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v51/common" - "github.com/oracle/oci-go-sdk/v51/database" - "github.com/oracle/oci-go-sdk/v51/secrets" + "github.com/oracle/oci-go-sdk/v54/common" + "github.com/oracle/oci-go-sdk/v54/database" + "github.com/oracle/oci-go-sdk/v54/secrets" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" "github.com/oracle/oracle-database-operator/commons/oci" diff --git a/commons/oci/database.go b/commons/oci/database.go index 3fdf7a00..1d223f79 100644 --- a/commons/oci/database.go +++ b/commons/oci/database.go @@ -46,10 +46,10 @@ import ( "time" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v51/common" - "github.com/oracle/oci-go-sdk/v51/database" - "github.com/oracle/oci-go-sdk/v51/secrets" - "github.com/oracle/oci-go-sdk/v51/workrequests" + "github.com/oracle/oci-go-sdk/v54/common" + "github.com/oracle/oci-go-sdk/v54/database" + "github.com/oracle/oci-go-sdk/v54/secrets" + "github.com/oracle/oci-go-sdk/v54/workrequests" "sigs.k8s.io/controller-runtime/pkg/client" corev1 "k8s.io/api/core/v1" diff --git a/commons/oci/provider.go b/commons/oci/provider.go index ec653d09..02fb09bb 100644 --- a/commons/oci/provider.go +++ b/commons/oci/provider.go @@ -42,8 +42,8 @@ import ( "context" "errors" - "github.com/oracle/oci-go-sdk/v51/common" - "github.com/oracle/oci-go-sdk/v51/common/auth" + "github.com/oracle/oci-go-sdk/v54/common" + "github.com/oracle/oci-go-sdk/v54/common/auth" "sigs.k8s.io/controller-runtime/pkg/client" diff --git a/commons/oci/vault.go b/commons/oci/vault.go index 5df1cc04..be48d176 100644 --- a/commons/oci/vault.go +++ b/commons/oci/vault.go @@ -42,8 +42,8 @@ import ( "context" "encoding/base64" - "github.com/oracle/oci-go-sdk/v51/common" - "github.com/oracle/oci-go-sdk/v51/secrets" + "github.com/oracle/oci-go-sdk/v54/common" + "github.com/oracle/oci-go-sdk/v54/secrets" ) func getValueFromVaultSecret(secretClient secrets.SecretsClient, vaultSecretOCID string) (string, error) { diff --git a/commons/oci/wallet.go b/commons/oci/wallet.go index 268e66c5..289b5bbf 100644 --- a/commons/oci/wallet.go +++ b/commons/oci/wallet.go @@ -49,9 +49,9 @@ import ( "time" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v51/common" - "github.com/oracle/oci-go-sdk/v51/database" - "github.com/oracle/oci-go-sdk/v51/secrets" + "github.com/oracle/oci-go-sdk/v54/common" + "github.com/oracle/oci-go-sdk/v54/database" + "github.com/oracle/oci-go-sdk/v54/secrets" "sigs.k8s.io/controller-runtime/pkg/client" "k8s.io/apimachinery/pkg/types" diff --git a/commons/sharding/catalog.go b/commons/sharding/catalog.go index a24abeda..406250e9 100644 --- a/commons/sharding/catalog.go +++ b/commons/sharding/catalog.go @@ -221,7 +221,7 @@ func buildContainerSpecForCatalog(instance *databasev1alpha1.ShardingDatabase, O PeriodSeconds: int32(240), InitialDelaySeconds: int32(300), TimeoutSeconds: int32(60), - Handler: corev1.Handler{ + ProbeHandler: corev1.ProbeHandler{ Exec: &corev1.ExecAction{ Command: getLivenessCmd("CATALOG"), }, diff --git a/commons/sharding/gsm.go b/commons/sharding/gsm.go index 720c00df..5ec9be10 100644 --- a/commons/sharding/gsm.go +++ b/commons/sharding/gsm.go @@ -233,7 +233,7 @@ func buildContainerSpecForGsm(instance *databasev1alpha1.ShardingDatabase, OraGs PeriodSeconds: int32(240), InitialDelaySeconds: int32(300), TimeoutSeconds: int32(60), - Handler: corev1.Handler{ + ProbeHandler: corev1.ProbeHandler{ Exec: &corev1.ExecAction{ Command: getLivenessCmd("GSM"), }, diff --git a/commons/sharding/scommon.go b/commons/sharding/scommon.go index 5af51390..bc9030ec 100644 --- a/commons/sharding/scommon.go +++ b/commons/sharding/scommon.go @@ -48,8 +48,8 @@ import ( "strings" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v51/common" - "github.com/oracle/oci-go-sdk/v51/ons" + "github.com/oracle/oci-go-sdk/v54/common" + "github.com/oracle/oci-go-sdk/v54/ons" appsv1 "k8s.io/api/apps/v1" corev1 "k8s.io/api/core/v1" "k8s.io/apimachinery/pkg/api/errors" diff --git a/commons/sharding/shard.go b/commons/sharding/shard.go index 08ba490f..e1063ba5 100644 --- a/commons/sharding/shard.go +++ b/commons/sharding/shard.go @@ -222,7 +222,7 @@ func buildContainerSpecForShard(instance *databasev1alpha1.ShardingDatabase, Ora PeriodSeconds: int32(240), InitialDelaySeconds: int32(300), TimeoutSeconds: int32(120), - Handler: corev1.Handler{ + ProbeHandler: corev1.ProbeHandler{ Exec: &corev1.ExecAction{ Command: getLivenessCmd("SHARD"), }, diff --git a/controllers/database/autonomousdatabase_controller.go b/controllers/database/autonomousdatabase_controller.go index 142e78c6..d625968b 100644 --- a/controllers/database/autonomousdatabase_controller.go +++ b/controllers/database/autonomousdatabase_controller.go @@ -44,9 +44,9 @@ import ( "reflect" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v51/database" - "github.com/oracle/oci-go-sdk/v51/secrets" - "github.com/oracle/oci-go-sdk/v51/workrequests" + "github.com/oracle/oci-go-sdk/v54/database" + "github.com/oracle/oci-go-sdk/v54/secrets" + "github.com/oracle/oci-go-sdk/v54/workrequests" apiErrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/runtime" diff --git a/controllers/database/shardingdatabase_controller.go b/controllers/database/shardingdatabase_controller.go index f30e714b..aa37750d 100644 --- a/controllers/database/shardingdatabase_controller.go +++ b/controllers/database/shardingdatabase_controller.go @@ -47,8 +47,8 @@ import ( "time" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v51/common" - "github.com/oracle/oci-go-sdk/v51/ons" + "github.com/oracle/oci-go-sdk/v54/common" + "github.com/oracle/oci-go-sdk/v54/ons" appsv1 "k8s.io/api/apps/v1" corev1 "k8s.io/api/core/v1" "k8s.io/apimachinery/pkg/api/errors" diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index c69ba5b3..b5d5c567 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -513,7 +513,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns Name: m.Name, Image: m.Spec.Image.PullFrom, Lifecycle: &corev1.Lifecycle{ - PreStop: &corev1.Handler{ + PreStop: &corev1.LifecycleHandler{ Exec: &corev1.ExecAction{ Command: []string{"/bin/sh", "-c", "/bin/echo -en 'shutdown abort;\n' | env ORACLE_SID=${ORACLE_SID^^} sqlplus -S / as sysdba"}, }, @@ -523,7 +523,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns Ports: []corev1.ContainerPort{{ContainerPort: 1521}, {ContainerPort: 5500}}, ReadinessProbe: &corev1.Probe{ - Handler: corev1.Handler{ + ProbeHandler: corev1.ProbeHandler{ Exec: &corev1.ExecAction{ Command: []string{"/bin/sh", "-c", "if [ -f $ORACLE_BASE/checkDBLockStatus.sh ]; then $ORACLE_BASE/checkDBLockStatus.sh ; else $ORACLE_BASE/checkDBStatus.sh; fi "}, }, diff --git a/go.mod b/go.mod index 4e19053e..812af66f 100644 --- a/go.mod +++ b/go.mod @@ -1,16 +1,69 @@ module github.com/oracle/oracle-database-operator -go 1.16 +go 1.17 require ( - github.com/go-logr/logr v0.4.0 - github.com/onsi/ginkgo v1.16.4 - github.com/onsi/gomega v1.13.0 - github.com/oracle/oci-go-sdk/v51 v51.0.0 + github.com/go-logr/logr v1.2.2 + github.com/onsi/ginkgo v1.16.5 + github.com/onsi/gomega v1.17.0 + github.com/oracle/oci-go-sdk/v54 v54.0.0 gopkg.in/yaml.v2 v2.4.0 - k8s.io/api v0.21.2 - k8s.io/apimachinery v0.21.2 - k8s.io/client-go v0.21.2 - sigs.k8s.io/controller-runtime v0.9.2 - sigs.k8s.io/yaml v1.2.0 + k8s.io/api v0.23.1 + k8s.io/apimachinery v0.23.1 + k8s.io/client-go v0.23.1 + sigs.k8s.io/controller-runtime v0.11.0 + sigs.k8s.io/yaml v1.3.0 +) + +require ( + cloud.google.com/go v0.81.0 // indirect + github.com/beorn7/perks v1.0.1 // indirect + github.com/cespare/xxhash/v2 v2.1.1 // indirect + github.com/davecgh/go-spew v1.1.1 // indirect + github.com/evanphx/json-patch v4.12.0+incompatible // indirect + github.com/fsnotify/fsnotify v1.5.1 // indirect + github.com/go-logr/zapr v1.2.0 // indirect + github.com/gogo/protobuf v1.3.2 // indirect + github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect + github.com/golang/protobuf v1.5.2 // indirect + github.com/google/go-cmp v0.5.5 // indirect + github.com/google/gofuzz v1.1.0 // indirect + github.com/google/uuid v1.1.2 // indirect + github.com/googleapis/gnostic v0.5.5 // indirect + github.com/imdario/mergo v0.3.12 // indirect + github.com/json-iterator/go v1.1.12 // indirect + github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 // indirect + github.com/moby/spdystream v0.2.0 // indirect + github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect + github.com/modern-go/reflect2 v1.0.2 // indirect + github.com/nxadm/tail v1.4.8 // indirect + github.com/pkg/errors v0.9.1 // indirect + github.com/prometheus/client_golang v1.11.0 // indirect + github.com/prometheus/client_model v0.2.0 // indirect + github.com/prometheus/common v0.28.0 // indirect + github.com/prometheus/procfs v0.6.0 // indirect + github.com/sony/gobreaker v0.4.2-0.20210216022020-dd874f9dd33b // indirect + github.com/spf13/pflag v1.0.5 // indirect + go.uber.org/atomic v1.7.0 // indirect + go.uber.org/multierr v1.6.0 // indirect + go.uber.org/zap v1.19.1 // indirect + golang.org/x/net v0.0.0-20211209124913-491a49abca63 // indirect + golang.org/x/oauth2 v0.0.0-20210819190943-2bc19b11175f // indirect + golang.org/x/sys v0.0.0-20211029165221-6e7872819dc8 // indirect + golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b // indirect + golang.org/x/text v0.3.7 // indirect + golang.org/x/time v0.0.0-20210723032227-1f47c861a9ac // indirect + gomodules.xyz/jsonpatch/v2 v2.2.0 // indirect + google.golang.org/appengine v1.6.7 // indirect + google.golang.org/protobuf v1.27.1 // indirect + gopkg.in/inf.v0 v0.9.1 // indirect + gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 // indirect + gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b // indirect + k8s.io/apiextensions-apiserver v0.23.0 // indirect + k8s.io/component-base v0.23.0 // indirect + k8s.io/klog/v2 v2.30.0 // indirect + k8s.io/kube-openapi v0.0.0-20211115234752-e816edb12b65 // indirect + k8s.io/utils v0.0.0-20210930125809-cb0fa318a74b // indirect + sigs.k8s.io/json v0.0.0-20211020170558-c049b76a60c6 // indirect + sigs.k8s.io/structured-merge-diff/v4 v4.2.0 // indirect ) diff --git a/go.sum b/go.sum index 2b051eef..42369280 100644 --- a/go.sum +++ b/go.sum @@ -8,28 +8,44 @@ cloud.google.com/go v0.46.3/go.mod h1:a6bKKbmY7er1mI7TEI4lsAkts/mkhTSZK8w33B4RAg cloud.google.com/go v0.50.0/go.mod h1:r9sluTvynVuxRIOHXQEHMFffphuXHOMZMycpNR5e6To= cloud.google.com/go v0.52.0/go.mod h1:pXajvRH/6o3+F9jDHZWQ5PbGhn+o8w9qiu/CffaVdO4= cloud.google.com/go v0.53.0/go.mod h1:fp/UouUEsRkN6ryDKNW/Upv/JBKnv6WDthjR6+vze6M= -cloud.google.com/go v0.54.0 h1:3ithwDMr7/3vpAMXiH+ZQnYbuIsh+OPhUPMFC9enmn0= cloud.google.com/go v0.54.0/go.mod h1:1rq2OEkV3YMf6n/9ZvGWI3GWw0VoqH/1x2nd8Is/bPc= +cloud.google.com/go v0.56.0/go.mod h1:jr7tqZxxKOVYizybht9+26Z/gUq7tiRzu+ACVAMbKVk= +cloud.google.com/go v0.57.0/go.mod h1:oXiQ6Rzq3RAkkY7N6t3TcE6jE+CIBBbA36lwQ1JyzZs= +cloud.google.com/go v0.62.0/go.mod h1:jmCYTdRCQuc1PHIIJ/maLInMho30T/Y0M4hTdTShOYc= +cloud.google.com/go v0.65.0/go.mod h1:O5N8zS7uWy9vkA9vayVHs65eM1ubvY4h553ofrNHObY= +cloud.google.com/go v0.72.0/go.mod h1:M+5Vjvlc2wnp6tjzE102Dw08nGShTscUx2nZMufOKPI= +cloud.google.com/go v0.74.0/go.mod h1:VV1xSbzvo+9QJOxLDaJfTjx5e+MePCpCWwvftOeQmWk= +cloud.google.com/go v0.78.0/go.mod h1:QjdrLG0uq+YwhjoVOLsS1t7TW8fs36kLs4XO5R5ECHg= +cloud.google.com/go v0.79.0/go.mod h1:3bzgcEeQlzbuEAYu4mrWhKqWjmpprinYgKJLgKHnbb8= +cloud.google.com/go v0.81.0 h1:at8Tk2zUz63cLPR0JPWm5vp77pEZmzxEQBEfRKn1VV8= +cloud.google.com/go v0.81.0/go.mod h1:mk/AM35KwGk/Nm2YSeZbxXdrNK3KZOYHmLkOqC2V6E0= cloud.google.com/go/bigquery v1.0.1/go.mod h1:i/xbL2UlR5RvWAURpBYZTtm/cXjCha9lbfbpx4poX+o= cloud.google.com/go/bigquery v1.3.0/go.mod h1:PjpwJnslEMmckchkHFfq+HTD2DmtT67aNFKH1/VBDHE= cloud.google.com/go/bigquery v1.4.0/go.mod h1:S8dzgnTigyfTmLBfrtrhyYhwRxG72rYxvftPBK2Dvzc= +cloud.google.com/go/bigquery v1.5.0/go.mod h1:snEHRnqQbz117VIFhE8bmtwIDY80NLUZUMb4Nv6dBIg= +cloud.google.com/go/bigquery v1.7.0/go.mod h1://okPTzCYNXSlb24MZs83e2Do+h+VXtc4gLoIoXIAPc= +cloud.google.com/go/bigquery v1.8.0/go.mod h1:J5hqkt3O0uAFnINi6JXValWIb1v0goeZM77hZzJN/fQ= cloud.google.com/go/datastore v1.0.0/go.mod h1:LXYbyblFSglQ5pkeyhO+Qmw7ukd3C+pD7TKLgZqpHYE= cloud.google.com/go/datastore v1.1.0/go.mod h1:umbIZjpQpHh4hmRpGhH4tLFup+FVzqBi1b3c64qFpCk= cloud.google.com/go/firestore v1.1.0/go.mod h1:ulACoGHTpvq5r8rxGJ4ddJZBZqakUQqClKRT5SZwBmk= cloud.google.com/go/pubsub v1.0.1/go.mod h1:R0Gpsv3s54REJCy4fxDixWD93lHJMoZTyQ2kNxGRt3I= cloud.google.com/go/pubsub v1.1.0/go.mod h1:EwwdRX2sKPjnvnqCa270oGRyludottCI76h+R3AArQw= cloud.google.com/go/pubsub v1.2.0/go.mod h1:jhfEVHT8odbXTkndysNHCcx0awwzvfOlguIAii9o8iA= +cloud.google.com/go/pubsub v1.3.1/go.mod h1:i+ucay31+CNRpDW4Lu78I4xXG+O1r/MAHgjpRVR+TSU= cloud.google.com/go/storage v1.0.0/go.mod h1:IhtSnM/ZTZV8YYJWCY8RULGVqBDmpoyjwiyrjsg+URw= cloud.google.com/go/storage v1.5.0/go.mod h1:tpKbwo567HUNpVclU5sGELwQWBDZ8gh0ZeosJ0Rtdos= cloud.google.com/go/storage v1.6.0/go.mod h1:N7U0C8pVQ/+NIKOBQyamJIeKQKkZ+mxpohlUTyfDhBk= +cloud.google.com/go/storage v1.8.0/go.mod h1:Wv1Oy7z6Yz3DshWRJFhqM/UCfaWIRTdp0RXyy7KQOVs= +cloud.google.com/go/storage v1.10.0/go.mod h1:FLPqc6j+Ki4BU591ie1oL6qBQGu2Bl/tZ9ullr3+Kg0= dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU= -github.com/Azure/go-ansiterm v0.0.0-20170929234023-d6e3b3328b78/go.mod h1:LmzpDX56iTiv29bbRTIsUNlaFfuhWRQBWjQdVyAevI8= +github.com/Azure/go-ansiterm v0.0.0-20210608223527-2377c96fe795/go.mod h1:LmzpDX56iTiv29bbRTIsUNlaFfuhWRQBWjQdVyAevI8= +github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1/go.mod h1:xomTg63KZ2rFqZQzSB4Vz2SUXa1BpHTVz9L5PTmPC4E= github.com/Azure/go-autorest v14.2.0+incompatible/go.mod h1:r+4oMnoxhatjLLJ6zxSWATqVooLgysK6ZNox3g/xq24= -github.com/Azure/go-autorest/autorest v0.11.12/go.mod h1:eipySxLmqSyC5s5k1CLupqet0PSENBEDP93LQ9a8QYw= -github.com/Azure/go-autorest/autorest/adal v0.9.5/go.mod h1:B7KF7jKIeC9Mct5spmyCB/A8CG/sEz1vwIRGv/bbw7A= +github.com/Azure/go-autorest/autorest v0.11.18/go.mod h1:dSiJPy22c3u0OtOKDNttNgqpNFY/GeWa7GH/Pz56QRA= +github.com/Azure/go-autorest/autorest/adal v0.9.13/go.mod h1:W/MM4U6nLxnIskrw4UwWzlHfGjwUS50aOsc/I3yuU8M= github.com/Azure/go-autorest/autorest/date v0.3.0/go.mod h1:BI0uouVdmngYNUzGWeSYnokU+TrmwEsOqdt8Y6sso74= github.com/Azure/go-autorest/autorest/mocks v0.4.1/go.mod h1:LTp+uSrOhSkaKrUy935gNZuuIPPVsHlr9DSOxSayd+k= -github.com/Azure/go-autorest/logger v0.2.0/go.mod h1:T9E3cAhj2VqvPOtCYAvby9aBXkZmbF5NWuPV8+WeEW8= +github.com/Azure/go-autorest/logger v0.2.1/go.mod h1:T9E3cAhj2VqvPOtCYAvby9aBXkZmbF5NWuPV8+WeEW8= github.com/Azure/go-autorest/tracing v0.6.0/go.mod h1:+vhtPC754Xsa23ID7GlGsrdKBpUA79WCAKPPZVC2DeU= github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU= github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo= @@ -43,18 +59,26 @@ github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuy github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0= github.com/alecthomas/units v0.0.0-20190717042225-c3de453c63f4/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0= github.com/alecthomas/units v0.0.0-20190924025748-f65c72e2690d/go.mod h1:rBZYJk541a8SKzHPHnH3zbiI+7dagKZ0cgpgrD7Fyho= +github.com/antihax/optional v1.0.0/go.mod h1:uupD/76wgC+ih3iEmQUL+0Ugr19nfwCT1kdvxnR2qWY= +github.com/antlr/antlr4/runtime/Go/antlr v0.0.0-20210826220005-b48c857c3a0e/go.mod h1:F7bn7fEU90QkQ3tnmaTx3LTKLEDqnwWODIYppRQ5hnY= github.com/armon/circbuf v0.0.0-20150827004946-bbbad097214e/go.mod h1:3U/XgcO3hCbHZ8TKRvWD2dDTCfh9M9ya+I9JpbB7O8o= github.com/armon/go-metrics v0.0.0-20180917152333-f0300d1749da/go.mod h1:Q73ZrmVTwzkszR9V5SSuryQ31EELlFMUz1kKyl939pY= github.com/armon/go-radix v0.0.0-20180808171621-7fddfc383310/go.mod h1:ufUuZ+zHj4x4TnLV4JWEpy2hxWSpsRywHrMgIH9cCH8= github.com/asaskevich/govalidator v0.0.0-20190424111038-f61b66f89f4a/go.mod h1:lB+ZfQJz7igIIfQNfa7Ml4HSf2uFQQRzpGGRXenZAgY= +github.com/benbjohnson/clock v1.0.3/go.mod h1:bGMdMPoPVvcYyt1gHDf4J2KE153Yf9BuiUKYMaxlTDM= +github.com/benbjohnson/clock v1.1.0 h1:Q92kusRqC1XV2MjkWETPvjJVqKetz1OzxZB7mHJLju8= +github.com/benbjohnson/clock v1.1.0/go.mod h1:J11/hYXuz8f4ySSvYwY0FKfm+ezbsZBKZxNJlLklBHA= github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q= github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8= github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= github.com/bgentry/speakeasy v0.1.0/go.mod h1:+zsyZBPWlz7T6j88CTgSN5bM796AkVf0kBD4zp0CCIs= github.com/bketelsen/crypt v0.0.3-0.20200106085610-5cbc8cc4026c/go.mod h1:MKsuJmJgSg28kpZDP6UIiPt0e0Oz0kqKNGyRaWEPv84= +github.com/bketelsen/crypt v0.0.4/go.mod h1:aI6NrJ0pMGgvZKL1iVgXLnfIFJtfV+bKCoqOes/6LfM= github.com/blang/semver v3.5.1+incompatible/go.mod h1:kRBLl5iJ+tD4TcOOxsy/0fnwebNt5EWlYSAyrTnjyyk= github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU= +github.com/certifi/gocertifi v0.0.0-20191021191039-0944d244cd40/go.mod h1:sGbDF6GwGcLpkNXPUTkMRoywsNa/ol15pxFe6ERfguA= +github.com/certifi/gocertifi v0.0.0-20200922220541-2c3bb06c6054/go.mod h1:sGbDF6GwGcLpkNXPUTkMRoywsNa/ol15pxFe6ERfguA= github.com/cespare/xxhash v1.1.0 h1:a6HrQnmkObjyL+Gs60czilIUGqrzKutQD6XZog3p+ko= github.com/cespare/xxhash v1.1.0/go.mod h1:XrSqR1VqqWfGrhpAt58auRo0WTKS1nRRg3ghfAqPWnc= github.com/cespare/xxhash/v2 v2.1.1 h1:6MnRN8NT7+YBpUIWxHtefFZOKTAPgGjpQSxqLNn0+qY= @@ -63,18 +87,21 @@ github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWR github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI= github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU= github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw= -github.com/cockroachdb/datadriven v0.0.0-20190809214429-80d97fb3cbaa/go.mod h1:zn76sxSg3SzpJ0PPJaLDCu+Bu0Lg3sKTORVIj19EIF8= +github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc= +github.com/cncf/udpa/go v0.0.0-20200629203442-efcf912fb354/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk= +github.com/cncf/udpa/go v0.0.0-20201120205902-5459f2c99403/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk= +github.com/cncf/xds/go v0.0.0-20210312221358-fbca930ec8ed/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs= +github.com/cockroachdb/datadriven v0.0.0-20200714090401-bf6692d28da5/go.mod h1:h6jFvWxBdQXxjopDMZyH2UVceIRfR84bdzbkoKrsWNo= +github.com/cockroachdb/errors v1.2.4/go.mod h1:rQD95gz6FARkaKkQXUksEje/d9a6wBJoCr5oaCLELYA= +github.com/cockroachdb/logtags v0.0.0-20190617123548-eb05cc24525f/go.mod h1:i/u985jwjWRlyHXQbwatDASoW0RMlZ/3i9yJHE2xLkI= github.com/coreos/bbolt v1.3.2/go.mod h1:iRUV2dpdMOn7Bo10OQBFzIJO9kkE559Wcmn+qkEiiKk= github.com/coreos/etcd v3.3.13+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE= github.com/coreos/go-oidc v2.1.0+incompatible/go.mod h1:CgnwVTmzoESiwO9qyAFEMiHoZ1nMCKZlZ9V6mm3/LKc= -github.com/coreos/go-semver v0.2.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk= github.com/coreos/go-semver v0.3.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk= -github.com/coreos/go-systemd v0.0.0-20180511133405-39ca1b05acc7/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4= github.com/coreos/go-systemd v0.0.0-20190321100706-95778dfbb74e/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4= -github.com/coreos/pkg v0.0.0-20160727233714-3ac0863d7acf/go.mod h1:E3G3o1h8I7cfcXa63jLwjI0eiQQMgzzUDFVpN/nH/eA= +github.com/coreos/go-systemd/v22 v22.3.2/go.mod h1:Y58oyj3AT4RCenI/lSvhwexgC+NSVTIJ3seZv2GcEnc= github.com/coreos/pkg v0.0.0-20180928190104-399ea9e2e55f/go.mod h1:E3G3o1h8I7cfcXa63jLwjI0eiQQMgzzUDFVpN/nH/eA= github.com/cpuguy83/go-md2man/v2 v2.0.0/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU= -github.com/creack/pty v1.1.7/go.mod h1:lj5s0c3V2DBrqTV7llrYr5NG6My20zk30Fl46Y7DoTY= github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= github.com/creack/pty v1.1.11/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= @@ -83,23 +110,32 @@ github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSs github.com/dgrijalva/jwt-go v3.2.0+incompatible/go.mod h1:E3ru+11k8xSBh+hMPgOLZmtrrCbhqsmaPHjLKYnJCaQ= github.com/dgryski/go-sip13 v0.0.0-20181026042036-e10d5fee7954/go.mod h1:vAd38F8PWV+bWy6jNmig1y/TA+kYO4g3RSRF0IAv0no= github.com/docopt/docopt-go v0.0.0-20180111231733-ee0de3bc6815/go.mod h1:WwZ+bS3ebgob9U8Nd0kOddGdZWjyMGR8Wziv+TBNwSE= -github.com/dustin/go-humanize v0.0.0-20171111073723-bb3d318650d4/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk= github.com/dustin/go-humanize v1.0.0/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk= github.com/elazarl/goproxy v0.0.0-20180725130230-947c36da3153 h1:yUdfgN0XgIJw7foRItutHYUIhlcKzcSf5vDpdhQAKTc= github.com/elazarl/goproxy v0.0.0-20180725130230-947c36da3153/go.mod h1:/Zj4wYkgs4iZTTu3o/KG3Itv/qCCa8VVMlb3i9OVuzc= github.com/emicklei/go-restful v0.0.0-20170410110728-ff4f55a20633/go.mod h1:otzb+WCGbkyDHkqmQmT5YD2WR4BBwUdeQoFo8l/7tVs= github.com/emicklei/go-restful v2.9.5+incompatible/go.mod h1:otzb+WCGbkyDHkqmQmT5YD2WR4BBwUdeQoFo8l/7tVs= +github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4= github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4= +github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98= +github.com/envoyproxy/go-control-plane v0.9.7/go.mod h1:cwu0lG7PUMfa9snN8LXBig5ynNVH9qI8YYLbd1fK2po= +github.com/envoyproxy/go-control-plane v0.9.9-0.20201210154907-fd9021fe5dad/go.mod h1:cXg6YxExXjJnVBQHBLXeUAgxn2UodCpnH306RInaBQk= +github.com/envoyproxy/go-control-plane v0.9.9-0.20210217033140-668b12f5399d/go.mod h1:cXg6YxExXjJnVBQHBLXeUAgxn2UodCpnH306RInaBQk= +github.com/envoyproxy/go-control-plane v0.9.9-0.20210512163311-63b5d3c536b0/go.mod h1:hliV/p42l8fGbc6Y9bQ70uLwIvmJyVE5k4iMKlh8wCQ= github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c= github.com/evanphx/json-patch v0.5.2/go.mod h1:ZWS5hhDbVDyob71nXKNL0+PWn6ToqBHMikGIFbs31qQ= -github.com/evanphx/json-patch v4.9.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk= -github.com/evanphx/json-patch v4.11.0+incompatible h1:glyUF9yIYtMHzn8xaKw5rMhdWcwsYV8dZHIq5567/xs= -github.com/evanphx/json-patch v4.11.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk= +github.com/evanphx/json-patch v4.12.0+incompatible h1:4onqiflcdA9EOZ4RxV643DvftH5pOlLGNtQ5lPWQu84= +github.com/evanphx/json-patch v4.12.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk= github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4= +github.com/felixge/httpsnoop v1.0.1/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U= github.com/form3tech-oss/jwt-go v3.2.2+incompatible/go.mod h1:pbq4aXjuKjdthFRnoDwaVPLA+WlJuPGy+QneDUgJi2k= +github.com/form3tech-oss/jwt-go v3.2.3+incompatible/go.mod h1:pbq4aXjuKjdthFRnoDwaVPLA+WlJuPGy+QneDUgJi2k= github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo= -github.com/fsnotify/fsnotify v1.4.9 h1:hsms1Qyu0jgnwNXIxa+/V/PDsU6CfLf6CNO8H7IWoS4= github.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ= +github.com/fsnotify/fsnotify v1.5.1 h1:mZcQUHVQUQWoPXXtuf9yuEXKudkV2sx1E06UadKWpgI= +github.com/fsnotify/fsnotify v1.5.1/go.mod h1:T3375wBYaZdLLcVNkcVbzGHY7f1l/uK5T5Ai1i3InKU= +github.com/getkin/kin-openapi v0.76.0/go.mod h1:660oXbgy5JFMKreazJaQTw7o+X00qeSyhcnluiMv+Xg= +github.com/getsentry/raven-go v0.2.0/go.mod h1:KungGk8q33+aIAZUIVWZDr2OfAEBsO49PX4NzFV5kcQ= github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04= github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU= github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8= @@ -112,41 +148,47 @@ github.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V github.com/go-logfmt/logfmt v0.5.0/go.mod h1:wCYkCAKZfumFQihp8CzCvQ3paCTfi41vtzG1KdI/P7A= github.com/go-logr/logr v0.1.0/go.mod h1:ixOQHD9gLJUVQQ2ZOR7zLEifBX6tGkNJF4QyIY7sIas= github.com/go-logr/logr v0.2.0/go.mod h1:z6/tIYblkpsD+a4lm/fGIIU9mZ+XfAiaFtq7xTgseGU= -github.com/go-logr/logr v0.4.0 h1:K7/B1jt6fIBQVd4Owv2MqGQClcgf0R266+7C/QjRcLc= -github.com/go-logr/logr v0.4.0/go.mod h1:z6/tIYblkpsD+a4lm/fGIIU9mZ+XfAiaFtq7xTgseGU= -github.com/go-logr/zapr v0.4.0 h1:uc1uML3hRYL9/ZZPdgHS/n8Nzo+eaYL/Efxkkamf7OM= -github.com/go-logr/zapr v0.4.0/go.mod h1:tabnROwaDl0UNxkVeFRbY8bwB37GwRv0P8lg6aAiEnk= -github.com/go-openapi/jsonpointer v0.19.2/go.mod h1:3akKfEdA7DF1sugOqz1dVQHBcuDBPKZGEoHC/NkiQRg= +github.com/go-logr/logr v1.2.0/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A= +github.com/go-logr/logr v1.2.2 h1:ahHml/yUpnlb96Rp8HCvtYVPY8ZYpxq3g7UYchIYwbs= +github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A= +github.com/go-logr/zapr v1.2.0 h1:n4JnPI1T3Qq1SFEi/F8rwLrZERp2bso19PJZDB9dayk= +github.com/go-logr/zapr v1.2.0/go.mod h1:Qa4Bsj2Vb+FAVeAKsLD8RLQ+YRJB8YDmOAKxaBQf7Ro= github.com/go-openapi/jsonpointer v0.19.3/go.mod h1:Pl9vOtqEWErmShwVjC8pYs9cog34VGT37dQOVbmoatg= -github.com/go-openapi/jsonreference v0.19.2/go.mod h1:jMjeRr2HHw6nAVajTXJ4eiUwohSTlpa0o73RUL1owJc= +github.com/go-openapi/jsonpointer v0.19.5/go.mod h1:Pl9vOtqEWErmShwVjC8pYs9cog34VGT37dQOVbmoatg= github.com/go-openapi/jsonreference v0.19.3/go.mod h1:rjx6GuL8TTa9VaixXglHmQmIL98+wF9xc8zWvFonSJ8= -github.com/go-openapi/spec v0.19.3/go.mod h1:FpwSN1ksY1eteniUU7X0N/BgJ7a4WvBFVA8Lj9mJglo= -github.com/go-openapi/spec v0.19.5/go.mod h1:Hm2Jr4jv8G1ciIAo+frC/Ft+rR2kQDh8JHKHb3gWUSk= -github.com/go-openapi/swag v0.19.2/go.mod h1:POnQmlKehdgb5mhVOsnJFsivZCEZ/vjK9gh66Z9tfKk= +github.com/go-openapi/jsonreference v0.19.5/go.mod h1:RdybgQwPxbL4UEjuAruzK1x3nE69AqPYEJeo/TWfEeg= github.com/go-openapi/swag v0.19.5/go.mod h1:POnQmlKehdgb5mhVOsnJFsivZCEZ/vjK9gh66Z9tfKk= +github.com/go-openapi/swag v0.19.14/go.mod h1:QYRuS/SOXUCsnplDa677K7+DxSOj6IPNl/eQntq43wQ= github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY= github.com/go-task/slim-sprig v0.0.0-20210107165309-348f09dbbbc0/go.mod h1:fyg7847qk6SyHyPtNmDHnmrv/HOrqktSC+C9fM+CJOE= +github.com/godbus/dbus/v5 v5.0.4/go.mod h1:xhWf0FNVPg57R7Z0UbKHbJfkEywrmjJnf7w5xrFpKfA= github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ= github.com/gogo/protobuf v1.2.1/go.mod h1:hp+jE20tsWTFYpLwKvXlhS1hjn+gTNwPg2I6zVXpSg4= +github.com/gogo/protobuf v1.3.1/go.mod h1:SlYgWuQ5SjCEi6WLHjHCa1yvBfUnHcTbrrZtXPKa29o= github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q= github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q= github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q= -github.com/golang/groupcache v0.0.0-20160516000752-02826c3e7903/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= +github.com/golang/glog v1.0.0/go.mod h1:EWib/APOK0SL3dFbYqvxE3UYd8E6s1ouQ7iEp/0LWV4= github.com/golang/groupcache v0.0.0-20190129154638-5b532d6fd5ef/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= -github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e h1:1r7pUrabqp18hOBcwBwiTsbnFeTZHV9eER/QT5JVZxY= github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= +github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da h1:oI5xCqsCo564l8iNU+DwB5epxmsaqB+rhGL0m5jtYqE= +github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A= github.com/golang/mock v1.2.0/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A= github.com/golang/mock v1.3.1/go.mod h1:sBzyDLLjw3U8JLTeZvSv8jJB+tU5PVekmnlKIyFUx0Y= github.com/golang/mock v1.4.0/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw= github.com/golang/mock v1.4.1/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw= +github.com/golang/mock v1.4.3/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw= +github.com/golang/mock v1.4.4/go.mod h1:l3mdAwkq5BuhzHwde/uurv3sEJeZMXNpwsxVWU71h+4= +github.com/golang/mock v1.5.0/go.mod h1:CWnOUgYIOo4TcNZ0wHX3YZCqsaM1I1Jvs6v3mP3KVu8= github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= github.com/golang/protobuf v1.3.3/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw= github.com/golang/protobuf v1.3.4/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw= +github.com/golang/protobuf v1.3.5/go.mod h1:6O5/vntMXwX2lRkT1hjjk0nAC1IDOTvTlVgjlRvqsdk= github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8= github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA= github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs= @@ -156,15 +198,23 @@ github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QD github.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI= github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI= github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk= +github.com/golang/protobuf v1.5.1/go.mod h1:DopwsBzvsk0Fs44TXzsVbJyPhcCPeIwnvohx4u74HPM= github.com/golang/protobuf v1.5.2 h1:ROPKBNFfQgOUMifHyP+KYbvpjbdoFNs+aK7DXlji0Tw= github.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY= github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= +github.com/google/btree v1.0.1/go.mod h1:xXMiIv4Fb/0kKde4SpL7qlzvu5cMJDRkFDxJfI9uaxA= +github.com/google/cel-go v0.9.0/go.mod h1:U7ayypeSkw23szu4GaQTPJGx66c20mx8JklMSxrmI1w= +github.com/google/cel-spec v0.6.0/go.mod h1:Nwjgxy5CbjlPrtCWjeDjUyKMl8w41YBYGjsyDdqk0xA= github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M= github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= +github.com/google/go-cmp v0.4.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= +github.com/google/go-cmp v0.5.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= +github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= +github.com/google/go-cmp v0.5.3/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.5 h1:Khx7svrCpmxxtHBq5j2mp/xVjsi8hQMfNLvJFAlrGgU= github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= @@ -172,30 +222,36 @@ github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/ github.com/google/gofuzz v1.1.0 h1:Hsa8mG0dQ46ij8Sl2AYJDUv1oA9/d6Vk+3LG99Oe02g= github.com/google/gofuzz v1.1.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs= +github.com/google/martian/v3 v3.0.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0= +github.com/google/martian/v3 v3.1.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0= github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc= github.com/google/pprof v0.0.0-20190515194954-54271f7e092f/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc= github.com/google/pprof v0.0.0-20191218002539-d4f498aebedc/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= github.com/google/pprof v0.0.0-20200212024743-f11f1df84d12/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= github.com/google/pprof v0.0.0-20200229191704-1ebb73c60ed3/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= +github.com/google/pprof v0.0.0-20200430221834-fc25d7d30c6d/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= +github.com/google/pprof v0.0.0-20200708004538-1a94d8640e99/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= +github.com/google/pprof v0.0.0-20201023163331-3e6fc7fc9c4c/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= +github.com/google/pprof v0.0.0-20201203190320-1bf35d6f28c2/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= +github.com/google/pprof v0.0.0-20210122040257-d980be63207e/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= +github.com/google/pprof v0.0.0-20210226084205-cbba55b83ad5/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI= -github.com/google/uuid v1.0.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= -github.com/google/uuid v1.1.1/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= github.com/google/uuid v1.1.2 h1:EVhdT+1Kseyi1/pUmXKaFxYsDNy9RQYkMWRH68J/W7Y= github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= github.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg= github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk= -github.com/googleapis/gnostic v0.4.1/go.mod h1:LRhVm6pbyptWbWbuZ38d1eyptfvIytN3ir6b65WBswg= +github.com/googleapis/gnostic v0.5.1/go.mod h1:6U4PtQXGIEt/Z3h5MAT7FNofLnw9vXk2cUuW7uA/OeU= github.com/googleapis/gnostic v0.5.5 h1:9fHAtK0uDfpveeqqo1hkEZJcFvYXAiCN3UutL8F9xHw= github.com/googleapis/gnostic v0.5.5/go.mod h1:7+EbHbldMins07ALC74bsA81Ovc97DwqyJO1AENw9kA= github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY= -github.com/gorilla/websocket v0.0.0-20170926233335-4201258b820c/go.mod h1:E7qHFY5m1UJ88s3WnNqhKjPHQ0heANvMoAMk2YaljkQ= +github.com/gorilla/mux v1.8.0/go.mod h1:DVbg23sWSpFRCP0SfiEN6jmj59UnW/n46BH5rLB71So= github.com/gorilla/websocket v1.4.2/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE= github.com/gregjones/httpcache v0.0.0-20180305231024-9cad4c3443a7/go.mod h1:FecbI9+v66THATjSRHfNgh1IVFe/9kFxbXtjV0ctIMA= github.com/grpc-ecosystem/go-grpc-middleware v1.0.0/go.mod h1:FiyG127CGDf3tlThmgyCl78X/SZQqEOJBCDaAfeWzPs= -github.com/grpc-ecosystem/go-grpc-middleware v1.0.1-0.20190118093823-f849b5445de4/go.mod h1:FiyG127CGDf3tlThmgyCl78X/SZQqEOJBCDaAfeWzPs= +github.com/grpc-ecosystem/go-grpc-middleware v1.3.0/go.mod h1:z0ButlSOZa5vEBq9m2m2hlwIgKw+rp3sdCBRoJY+30Y= github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0/go.mod h1:8NvIoxWQoOIhqOTXgfV/d3M/q6VIi02HzZEHgUlZvzk= github.com/grpc-ecosystem/grpc-gateway v1.9.0/go.mod h1:vNeuVxBJEsws4ogUvrchl83t/GYV9WGTSLVdBhOQFDY= -github.com/grpc-ecosystem/grpc-gateway v1.9.5/go.mod h1:vNeuVxBJEsws4ogUvrchl83t/GYV9WGTSLVdBhOQFDY= +github.com/grpc-ecosystem/grpc-gateway v1.16.0/go.mod h1:BDjrQk3hbvj6Nolgz8mAMFbcEtjT1g+wF4CSlocrBnw= github.com/hashicorp/consul/api v1.1.0/go.mod h1:VmuI/Lkw1nC05EYQWNKwWGbkg+FbDBtguAZLlVdkD9Q= github.com/hashicorp/consul/sdk v0.1.1/go.mod h1:VKf9jXwCTEY1QZP2MOLRhb5i/I/ssyNV1vwHyQBF0x8= github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4= @@ -211,8 +267,6 @@ github.com/hashicorp/go-uuid v1.0.1/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/b github.com/hashicorp/go.net v0.0.1/go.mod h1:hjKkEWcCURg++eb33jQU7oqQcI9XDCnUzHA0oac0k90= github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8= github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8= -github.com/hashicorp/golang-lru v0.5.4 h1:YDjusn29QI/Das2iO9M0BHnIbxPeyuCHsjMW+lJfyTc= -github.com/hashicorp/golang-lru v0.5.4/go.mod h1:iADmTwqILo4mZ8BN3D2Q6+9jd8WM5uGBxy+E8yxSoD4= github.com/hashicorp/hcl v1.0.0/go.mod h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ= github.com/hashicorp/logutils v1.0.0/go.mod h1:QIAnNjmIWmVIIkWDTG1z5v++HQmx9WQRO+LraFDTW64= github.com/hashicorp/mdns v1.0.0/go.mod h1:tL+uN++7HEJ6SQLQ2/p+z2pH24WQKWjBPkE0mNTz8vQ= @@ -220,44 +274,47 @@ github.com/hashicorp/memberlist v0.1.3/go.mod h1:ajVTdAv/9Im8oMAAj5G31PhhMCZJV2p github.com/hashicorp/serf v0.8.2/go.mod h1:6hOLApaqBFA1NXqRQAsxw9QxuDEvNxSQRwA/JwenrHc= github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU= github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc= +github.com/ianlancetaylor/demangle v0.0.0-20200824232613-28f6c0f3b639/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc= github.com/imdario/mergo v0.3.5/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA= github.com/imdario/mergo v0.3.12 h1:b6R2BslTbIEToALKP7LxUvijTsNI9TAe80pLWN2g/HU= github.com/imdario/mergo v0.3.12/go.mod h1:jmQim1M+e3UYxmgPu/WyfjB3N3VflVyUjjjwH0dnCYA= github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8= github.com/jessevdk/go-flags v1.4.0/go.mod h1:4FA24M0QyGHXBuZZK/XkWh8h0e1EYbRYJSGM75WSRxI= github.com/jonboulle/clockwork v0.1.0/go.mod h1:Ii8DK3G1RaLaWxj9trq07+26W01tbo22gdxWY5EU2bo= +github.com/jonboulle/clockwork v0.2.2/go.mod h1:Pkfl5aHPm1nk2H9h0bjmnJD/BcgbGXUBGnn1kMkgxc8= +github.com/josharian/intern v1.0.0/go.mod h1:5DoeVV0s6jJacbCEi61lwdGj/aVlrQvzHFFd8Hwg//Y= github.com/jpillora/backoff v1.0.0/go.mod h1:J/6gKK9jxlEcS3zixgDgUAsiuZ7yrSoa/FX5e0EB2j4= github.com/json-iterator/go v1.1.6/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU= -github.com/json-iterator/go v1.1.7/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4= github.com/json-iterator/go v1.1.10/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4= -github.com/json-iterator/go v1.1.11 h1:uVUAXhF2To8cbw/3xN3pxj6kk7TYKs98NIrTqPlMWAQ= github.com/json-iterator/go v1.1.11/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4= +github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM= +github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo= github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU= github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk= github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU= github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w= github.com/julienschmidt/httprouter v1.3.0/go.mod h1:JR6WtHb+2LUe8TCKY3cZOxFyyO8IZAc4RVcycCCAKdM= github.com/kisielk/errcheck v1.1.0/go.mod h1:EZBBE59ingxPouuu3KfxchcWSUPOHkagtvWXihfKN4Q= +github.com/kisielk/errcheck v1.2.0/go.mod h1:/BMXB+zMLi60iA8Vv6Ksmxu/1UDYcXs4uQLJ+jE2L00= github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8= github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck= github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= github.com/konsorten/go-windows-terminal-sequences v1.0.3/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= +github.com/kr/fs v0.1.0/go.mod h1:FFnZGqtBN9Gxj7eW1uZ42v5BccTP0vu6NEaFoC2HwRg= github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc= github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo= github.com/kr/pretty v0.2.0/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI= github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ= -github.com/kr/pty v1.1.5/go.mod h1:9r2w37qlBe7rQ6e1fg1S/9xpWHSnaqNdHD3WcMdbPDA= github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI= github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= github.com/magiconair/properties v1.8.1/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ= +github.com/magiconair/properties v1.8.5/go.mod h1:y3VJvCyxH9uVvJTWEGAELF3aiYNyPKd5NZ3oSwXrF60= github.com/mailru/easyjson v0.0.0-20190614124828-94de47d64c63/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc= github.com/mailru/easyjson v0.0.0-20190626092158-b2ccc519800e/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc= -github.com/mailru/easyjson v0.7.0/go.mod h1:KAzv3t3aY1NaHWoQz1+4F1ccyAH66Jk7yos7ldAVICs= +github.com/mailru/easyjson v0.7.6/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc= github.com/mattn/go-colorable v0.0.9/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaOChaDxuIBZU= github.com/mattn/go-isatty v0.0.3/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4= -github.com/mattn/go-isatty v0.0.4/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4= -github.com/mattn/go-runewidth v0.0.2/go.mod h1:LwmH8dsx7+W8Uxz3IHJYH5QSwggIsqBzpuz5H//U1FU= github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0= github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 h1:I0XW9+e1XWDxdcEniV4rQAIOPUGDq67JSCiRCgGCZLI= github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369/go.mod h1:BSXmuO+STAnVfrANrmjBb36TMTDstsz7MSK+HVaYKv4= @@ -270,15 +327,17 @@ github.com/mitchellh/gox v0.4.0/go.mod h1:Sd9lOJ0+aimLBi73mGofS1ycjY8lL3uZM3JPS4 github.com/mitchellh/iochan v1.0.0/go.mod h1:JwYml1nuB7xOzsp52dPpHFffvOCDupsG0QubkSMEySY= github.com/mitchellh/mapstructure v0.0.0-20160808181253-ca63d7c062ee/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y= github.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y= +github.com/mitchellh/mapstructure v1.4.1/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo= github.com/moby/spdystream v0.2.0 h1:cjW1zVyyoiM0T7b6UoySUFqzXMoqRckQtXwGPiBhOM8= github.com/moby/spdystream v0.2.0/go.mod h1:f7i0iNDQJ059oMTcWxx8MA/zKFIuD/lY+0GqbN2Wy8c= -github.com/moby/term v0.0.0-20201216013528-df9cb8a40635/go.mod h1:FBS0z0QWA44HXygs7VXDUOGoN/1TV3RuWkLO04am3wc= +github.com/moby/term v0.0.0-20210610120745-9d4ed1856297/go.mod h1:vgPCkQMyxTZ7IDy8SXRufE172gr8+K/JE/7hHFxHW3A= github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg= github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0= -github.com/modern-go/reflect2 v1.0.1 h1:9f412s+6RmYXLWZSEzVVgPGK7C2PphHj5RJrvfx9AWI= github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0= +github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M= +github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk= github.com/munnerz/goautoneg v0.0.0-20120707110453-a547fc61f48d/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ= github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ= github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U= @@ -290,29 +349,30 @@ github.com/nxadm/tail v1.4.4/go.mod h1:kenIhsEOeOJmVchQTgglprH7qJGnHDVpk1VPCcaMI github.com/nxadm/tail v1.4.8 h1:nPr65rt6Y5JFSKQO7qToXr7pePgD6Gwiw05lkbyAQTE= github.com/nxadm/tail v1.4.8/go.mod h1:+ncqLTQzXmGhMZNUePPaPqPvBxHAIsmXswZKocGu+AU= github.com/oklog/ulid v1.3.1/go.mod h1:CirwcVhetQ6Lv90oh/F+FBtV6XMibvdAFo93nm5qn4U= -github.com/olekukonko/tablewriter v0.0.0-20170122224234-a0225b3f23b5/go.mod h1:vsDQFd/mU46D+Z4whnwzcISnGGzXWMclvtLoiIKAKIo= github.com/onsi/ginkgo v0.0.0-20170829012221-11459a886d9c/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE= github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE= -github.com/onsi/ginkgo v1.11.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE= github.com/onsi/ginkgo v1.12.1/go.mod h1:zj2OWP4+oCPe1qIXoGWkgMRwljMUYCdkwsT2108oapk= -github.com/onsi/ginkgo v1.16.2/go.mod h1:CObGmKUOKaSC0RjmoAK7tKyn4Azo5P2IWuoMnvwxz1E= -github.com/onsi/ginkgo v1.16.4 h1:29JGrr5oVBm5ulCWet69zQkzWipVXIol6ygQUe/EzNc= +github.com/onsi/ginkgo v1.14.0/go.mod h1:iSB4RoI2tjJc9BBv4NKIKWKya62Rps+oPG/Lv9klQyY= github.com/onsi/ginkgo v1.16.4/go.mod h1:dX+/inL/fNMqNlz0e9LfyB9TswhZpCVdJM/Z6Vvnwo0= +github.com/onsi/ginkgo v1.16.5 h1:8xi0RTUf59SOSfEtZMvwTvXYMzG4gV23XVHOZiXNtnE= +github.com/onsi/ginkgo v1.16.5/go.mod h1:+E8gABHa3K6zRBolWtd+ROzc/U5bkGt0FwiG042wbpU= github.com/onsi/gomega v0.0.0-20170829124025-dcabb60a477c/go.mod h1:C1qb7wdrVGGVU+Z6iS04AVkA3Q65CEZX59MT0QO5uiA= -github.com/onsi/gomega v1.7.0/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY= github.com/onsi/gomega v1.7.1/go.mod h1:XdKZgCCFLUoM/7CFJVPcG8C1xQ1AJ0vpAezJrB7JYyY= github.com/onsi/gomega v1.10.1/go.mod h1:iN09h71vgCQne3DLsj+A5owkum+a2tYe+TOCB1ybHNo= -github.com/onsi/gomega v1.13.0 h1:7lLHu94wT9Ij0o6EWWclhu0aOh32VxhkwEJvzuWPeak= -github.com/onsi/gomega v1.13.0/go.mod h1:lRk9szgn8TxENtWd0Tp4c3wjlRfMTMH27I+3Je41yGY= -github.com/oracle/oci-go-sdk/v51 v51.0.0 h1:eDUVMsAzvf+jfq4xbtpQrxznYNjccwKKyIhmRvZLgwI= -github.com/oracle/oci-go-sdk/v51 v51.0.0/go.mod h1:d9KSNXwE64drofxoor+y/JWofJqLqRF9D1/AtfYIE10= +github.com/onsi/gomega v1.17.0 h1:9Luw4uT5HTjHTN8+aNcSThgH1vdXnmdJ8xIfZ4wyTRE= +github.com/onsi/gomega v1.17.0/go.mod h1:HnhC7FXeEQY45zxNK3PPoIUhzk/80Xly9PcubAlGdZY= +github.com/opentracing/opentracing-go v1.1.0/go.mod h1:UkNAQd3GIcIGf0SeVgPpRdFStlNbqXla1AfSYxPUl2o= +github.com/oracle/oci-go-sdk/v54 v54.0.0 h1:CDLjeSejv2aDpElAJrhKpi6zvT/zhZCZuXchUUZ+LS4= +github.com/oracle/oci-go-sdk/v54 v54.0.0/go.mod h1:+t+yvcFGVp+3ZnztnyxqXfQDsMlq8U25faBLa+mqCMc= github.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc= github.com/pelletier/go-toml v1.2.0/go.mod h1:5z9KED0ma1S8pY6P1sdut58dfprrGBbd/94hg7ilaic= +github.com/pelletier/go-toml v1.9.3/go.mod h1:u1nR/EPcESfeI/szUZKdtJ0xRNbUoANCkoOuaOx1Y+c= github.com/peterbourgon/diskv v2.0.1+incompatible/go.mod h1:uqqh8zWWbv1HBMNONnaR/tNboyR3/BZd58JJSHlUSCU= github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4= github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= +github.com/pkg/sftp v1.10.1/go.mod h1:lYOWFsE0bwd1+KfKJaKeuokY15vzFx25BLbzYYoAxZI= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/posener/complete v1.1.1/go.mod h1:em0nMJCgc9GFtwrmVmEMR/ZL6WyhyjMBndrE9hABlRI= @@ -332,17 +392,18 @@ github.com/prometheus/common v0.0.0-20181113130724-41aa239b4cce/go.mod h1:daVV7q github.com/prometheus/common v0.4.0/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4= github.com/prometheus/common v0.4.1/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4= github.com/prometheus/common v0.10.0/go.mod h1:Tlit/dnDKsSWFlCLTWaA1cyBgKHSMdTB80sz/V91rCo= -github.com/prometheus/common v0.26.0 h1:iMAkS2TDoNWnKM+Kopnx/8tnEStIfpYA0ur0xQzzhMQ= github.com/prometheus/common v0.26.0/go.mod h1:M7rCNAaPfAosfx8veZJCuw84e35h3Cfd9VFqTh1DIvc= +github.com/prometheus/common v0.28.0 h1:vGVfV9KrDTvWt5boZO0I19g2E3CsWfpPPKZM9dt3mEw= +github.com/prometheus/common v0.28.0/go.mod h1:vu+V0TpY+O6vW9J44gczi3Ap/oXXR10b+M/gUGO4Hls= github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk= github.com/prometheus/procfs v0.0.0-20190507164030-5867b95ac084/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA= github.com/prometheus/procfs v0.0.2/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA= github.com/prometheus/procfs v0.1.3/go.mod h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4OA4YeYWdaU= -github.com/prometheus/procfs v0.2.0/go.mod h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4OA4YeYWdaU= github.com/prometheus/procfs v0.6.0 h1:mxy4L2jP6qMonqmq+aTtOx1ifVWUgG/TAmntgbh3xv4= github.com/prometheus/procfs v0.6.0/go.mod h1:cz+aTbrPOrUb4q7XlbU9ygM+/jj0fzG6c1xBZuNvfVA= github.com/prometheus/tsdb v0.7.1/go.mod h1:qhTCs0VvXwvX/y3TZrWD7rabWM+ijKTux40TwIPHuXU= github.com/rogpeppe/fastuuid v0.0.0-20150106093220-6724a57986af/go.mod h1:XWv6SoW27p1b0cqNHllgS5HIMJraePCO15w5zCzIWYg= +github.com/rogpeppe/fastuuid v1.2.0/go.mod h1:jVj6XXZzXRy/MSR5jhDC/2q6DgLz+nrA6LYCDYWNEvQ= github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4= github.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM= github.com/ryanuber/columnize v0.0.0-20160712163229-9b3edd62028f/go.mod h1:sm1tb6uqfes/u+d4ooFouqFdy9/2g9QGwK3SQygK0Ts= @@ -352,29 +413,32 @@ github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPx github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE= github.com/sirupsen/logrus v1.6.0/go.mod h1:7uNnSEd1DgxDLC74fIahvMZmmYsHGZGEOFrfsX/uA88= github.com/sirupsen/logrus v1.7.0/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0= +github.com/sirupsen/logrus v1.8.1/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0= github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1:OnSkiWE9lh6wB0YB77sQom3nweQdgAjqCqsofrRNTgc= github.com/smartystreets/goconvey v1.6.4/go.mod h1:syvi0/a8iFYH4r/RixwvyeAJjdLS9QV7WQ/tjFTllLA= github.com/soheilhy/cmux v0.1.4/go.mod h1:IM3LyeVVIOuxMH7sFAkER9+bJ4dT7Ms6E4xg4kGIyLM= +github.com/soheilhy/cmux v0.1.5/go.mod h1:T7TcVDs9LWfQgPlPsdngu6I6QIoyIFZDDC6sNE1GqG0= github.com/sony/gobreaker v0.4.2-0.20210216022020-dd874f9dd33b h1:br+bPNZsJWKicw/5rALEo67QHs5weyD5tf8WST+4sJ0= github.com/sony/gobreaker v0.4.2-0.20210216022020-dd874f9dd33b/go.mod h1:ZKptC7FHNvhBz7dN2LGjPVBz2sZJmc0/PkyDJOjmxWY= github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA= github.com/spf13/afero v1.1.2/go.mod h1:j4pytiNVoe2o6bmDsKpLACNPDBIoEAkihy7loJ1B0CQ= github.com/spf13/afero v1.2.2/go.mod h1:9ZxEEn6pIJ8Rxe320qSDBk6AsU0r9pR7Q4OcevTdifk= +github.com/spf13/afero v1.6.0/go.mod h1:Ai8FlHk4v/PARR026UzYexafAt9roJ7LcLMAmO6Z93I= github.com/spf13/cast v1.3.0/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE= -github.com/spf13/cobra v0.0.3/go.mod h1:1l0Ry5zgKvJasoi3XT1TypsSe7PqH0Sj9dhYf7v3XqQ= -github.com/spf13/cobra v1.1.1/go.mod h1:WnodtKOvamDL/PwE2M4iKs8aMDBZ5Q5klgD3qfVJQMI= +github.com/spf13/cast v1.3.1/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE= +github.com/spf13/cobra v1.1.3/go.mod h1:pGADOWyqRD/YMrPZigI/zbliZ2wVD/23d+is3pSWzOo= +github.com/spf13/cobra v1.2.1/go.mod h1:ExllRjgxM/piMAM+3tAZvg8fsklGAf3tPfi+i8t68Nk= github.com/spf13/jwalterweatherman v1.0.0/go.mod h1:cQK4TGJAtQXfYWX+Ddv3mKDzgVb68N+wFjFa4jdeBTo= -github.com/spf13/pflag v0.0.0-20170130214245-9ff6c6923cff/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4= -github.com/spf13/pflag v1.0.1/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4= +github.com/spf13/jwalterweatherman v1.1.0/go.mod h1:aNWZUN0dPAAO/Ljvb5BEdw96iTZ0EXowPYD95IqWIGo= github.com/spf13/pflag v1.0.3/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4= github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA= github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= github.com/spf13/viper v1.7.0/go.mod h1:8WkrPz2fc9jxqZNCJI/76HCieCp4Q8HaLFoCha5qpdg= +github.com/spf13/viper v1.8.1/go.mod h1:o0Pch8wJ9BVSWGQMbra6iw0oQ5oktSIBaujf1rJH9Ns= github.com/stoewer/go-strcase v1.2.0/go.mod h1:IBiWB2sKIp3wVVQ3Y035++gc+knqhUQag1KpM8ahLw8= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= +github.com/stretchr/objx v0.1.1 h1:2vfRuCMp5sSVIDSqO8oNnWJq7mPa6KVP3iPIwFBuy8A= github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= -github.com/stretchr/objx v0.2.0 h1:Hbg2NidpLE8veEBkEZTL3CvlkUIVzuU9jDplZO54c48= -github.com/stretchr/objx v0.2.0/go.mod h1:qt09Ya8vawLte6SNmTgCsAVtYtaKzEcn8ATUoHMkEqE= github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs= github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4= @@ -383,42 +447,68 @@ github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/ github.com/stretchr/testify v1.7.0 h1:nwc3DEeHmmLAfoZucVR881uASk0Mfjw8xYJ99tb5CcY= github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= github.com/subosito/gotenv v1.2.0/go.mod h1:N0PQaV/YGNqwC0u51sEeR/aUtSLEXKX9iv69rRypqCw= -github.com/tmc/grpc-websocket-proxy v0.0.0-20170815181823-89b8d40f7ca8/go.mod h1:ncp9v5uamzpCO7NfCPTXjqaC+bZgJeR0sMTm6dMHP7U= github.com/tmc/grpc-websocket-proxy v0.0.0-20190109142713-0ad062ec5ee5/go.mod h1:ncp9v5uamzpCO7NfCPTXjqaC+bZgJeR0sMTm6dMHP7U= -github.com/urfave/cli v1.20.0/go.mod h1:70zkFmudgCuE/ngEzBv17Jvp/497gISqfk5gWijbERA= +github.com/tmc/grpc-websocket-proxy v0.0.0-20201229170055-e5319fda7802/go.mod h1:ncp9v5uamzpCO7NfCPTXjqaC+bZgJeR0sMTm6dMHP7U= github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2/go.mod h1:UETIi67q53MR2AWcXfiuqkDkRtnGDLqkBTpCHuJHxtU= +github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= +github.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= +github.com/yuin/goldmark v1.3.5/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k= +github.com/yuin/goldmark v1.4.0/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k= go.etcd.io/bbolt v1.3.2/go.mod h1:IbVyRI1SCnLcuJnV2u8VeU0CEYM7e686BmAb1XKL+uU= -go.etcd.io/bbolt v1.3.3/go.mod h1:IbVyRI1SCnLcuJnV2u8VeU0CEYM7e686BmAb1XKL+uU= -go.etcd.io/bbolt v1.3.5/go.mod h1:G5EMThwa9y8QZGBClrRx5EY+Yw9kAhnjy3bSjsnlVTQ= -go.etcd.io/etcd v0.5.0-alpha.5.0.20200910180754-dd1b699fc489/go.mod h1:yVHk9ub3CSBatqGNg7GRmsnfLWtoW60w4eDYfh7vHDg= +go.etcd.io/bbolt v1.3.6/go.mod h1:qXsaaIqmgQH0T+OPdb99Bf+PKfBBQVAdyD6TY9G8XM4= +go.etcd.io/etcd/api/v3 v3.5.0/go.mod h1:cbVKeC6lCfl7j/8jBhAK6aIYO9XOjdptoxU/nLQcPvs= +go.etcd.io/etcd/client/pkg/v3 v3.5.0/go.mod h1:IJHfcCEKxYu1Os13ZdwCwIUTUVGYTSAM3YSwc9/Ac1g= +go.etcd.io/etcd/client/v2 v2.305.0/go.mod h1:h9puh54ZTgAKtEbut2oe9P4L/oqKCVB6xsXlzd7alYQ= +go.etcd.io/etcd/client/v3 v3.5.0/go.mod h1:AIKXXVX/DQXtfTEqBryiLTUXwON+GuvO6Z7lLS/oTh0= +go.etcd.io/etcd/pkg/v3 v3.5.0/go.mod h1:UzJGatBQ1lXChBkQF0AuAtkRQMYnHubxAEYIrC3MSsE= +go.etcd.io/etcd/raft/v3 v3.5.0/go.mod h1:UFOHSIvO/nKwd4lhkwabrTD3cqW5yVyYYf/KlD00Szc= +go.etcd.io/etcd/server/v3 v3.5.0/go.mod h1:3Ah5ruV+M+7RZr0+Y/5mNLwC+eQlni+mQmOVdCRJoS4= go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU= go.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8= go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= go.opencensus.io v0.22.3/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= -go.uber.org/atomic v1.3.2/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE= +go.opencensus.io v0.22.4/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= +go.opencensus.io v0.22.5/go.mod h1:5pWMHQbX5EPX2/62yrJeAkowc+lfs/XD7Uxpq3pI6kk= +go.opencensus.io v0.23.0/go.mod h1:XItmlyltB5F7CS4xOC1DcqMoFqwtC6OG2xF7mCv7P7E= +go.opentelemetry.io/contrib v0.20.0/go.mod h1:G/EtFaa6qaN7+LxqfIAT3GiZa7Wv5DTBUzl5H4LY0Kc= +go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.20.0/go.mod h1:oVGt1LRbBOBq1A5BQLlUg9UaU/54aiHw8cgjV3aWZ/E= +go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.20.0/go.mod h1:2AboqHi0CiIZU0qwhtUfCYD1GeUzvvIXWNkhDt7ZMG4= +go.opentelemetry.io/otel v0.20.0/go.mod h1:Y3ugLH2oa81t5QO+Lty+zXf8zC9L26ax4Nzoxm/dooo= +go.opentelemetry.io/otel/exporters/otlp v0.20.0/go.mod h1:YIieizyaN77rtLJra0buKiNBOm9XQfkPEKBeuhoMwAM= +go.opentelemetry.io/otel/metric v0.20.0/go.mod h1:598I5tYlH1vzBjn+BTuhzTCSb/9debfNp6R3s7Pr1eU= +go.opentelemetry.io/otel/oteltest v0.20.0/go.mod h1:L7bgKf9ZB7qCwT9Up7i9/pn0PWIa9FqQ2IQ8LoxiGnw= +go.opentelemetry.io/otel/sdk v0.20.0/go.mod h1:g/IcepuwNsoiX5Byy2nNV0ySUF1em498m7hBWC279Yc= +go.opentelemetry.io/otel/sdk/export/metric v0.20.0/go.mod h1:h7RBNMsDJ5pmI1zExLi+bJK+Dr8NQCh0qGhm1KDnNlE= +go.opentelemetry.io/otel/sdk/metric v0.20.0/go.mod h1:knxiS8Xd4E/N+ZqKmUPf3gTTZ4/0TjTXukfxjzSTpHE= +go.opentelemetry.io/otel/trace v0.20.0/go.mod h1:6GjCW8zgDjwGHGa6GkyeB8+/5vjT16gUEi0Nf1iBdgw= +go.opentelemetry.io/proto/otlp v0.7.0/go.mod h1:PqfVotwruBrMGOCsRd/89rSnXhoiJIqeYNgFYFoEGnI= go.uber.org/atomic v1.4.0/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE= go.uber.org/atomic v1.7.0 h1:ADUqmZGgLDDfbSL9ZmPxKTybcoEYHgpYfELNoN+7hsw= go.uber.org/atomic v1.7.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc= -go.uber.org/goleak v1.1.10 h1:z+mqJhf6ss6BSfSM671tgKyZBFPTTJM+HLxnhPC3wu0= go.uber.org/goleak v1.1.10/go.mod h1:8a7PlsEVH3e/a/GLqe5IIrQx6GzcnRmZEufDUTk4A7A= +go.uber.org/goleak v1.1.11-0.20210813005559-691160354723/go.mod h1:cwTWslyiVhfpKIDGSZEM2HlOvcqm+tG4zioyIeLoqMQ= +go.uber.org/goleak v1.1.12 h1:gZAh5/EyT/HQwlpkCy6wTpqfH9H8Lz8zbm3dZh+OyzA= +go.uber.org/goleak v1.1.12/go.mod h1:cwTWslyiVhfpKIDGSZEM2HlOvcqm+tG4zioyIeLoqMQ= go.uber.org/multierr v1.1.0/go.mod h1:wR5kodmAFQ0UK8QlbwjlSNy0Z68gJhDJUG5sjR94q/0= go.uber.org/multierr v1.6.0 h1:y6IPFStTAIT5Ytl7/XYmHvzXQ7S3g/IeZW9hyZ5thw4= go.uber.org/multierr v1.6.0/go.mod h1:cdWPpRnG4AhwMwsgIHip0KRBQjJy5kYEpYjJxpXp9iU= go.uber.org/zap v1.10.0/go.mod h1:vwi/ZaCAaUcBkycHslxD9B2zi4UTXhF60s6SWpuDF0Q= -go.uber.org/zap v1.17.0 h1:MTjgFu6ZLKvY6Pvaqk97GlxNBuMpV4Hy/3P6tRGlI2U= go.uber.org/zap v1.17.0/go.mod h1:MXVU+bhUf/A7Xi2HNOnopQOrmycQ5Ih87HtOu4q5SSo= +go.uber.org/zap v1.19.0/go.mod h1:xg/QME4nWcxGxrpdeYfq7UvYrLh66cuVKdrbD1XF/NI= +go.uber.org/zap v1.19.1 h1:ue41HOKd1vGURxrmeKIgELGb3jPW9DMUDGtsinblHwI= +go.uber.org/zap v1.19.1/go.mod h1:j3DNczoxDZroyBnOT1L/Q79cfUMGZxlv/9dzN7SM1rI= golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= golang.org/x/crypto v0.0.0-20181029021203-45a5f77698d3/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= -golang.org/x/crypto v0.0.0-20190611184440-5c40567a22f8/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= +golang.org/x/crypto v0.0.0-20190820162420-60c769a6c586/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= golang.org/x/crypto v0.0.0-20201002170205-7f63de1d35b0/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= -golang.org/x/crypto v0.0.0-20210220033148-5ea612d1eb83/go.mod h1:jdWPYTVW3xRLrWPugEBEK3UY2ZEsg3UU495nc5E+M+I= +golang.org/x/crypto v0.0.0-20210817164053-32db794688a5/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc= golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8= @@ -440,8 +530,9 @@ golang.org/x/lint v0.0.0-20190909230951-414d861bb4ac/go.mod h1:6SW0HCj/g11FgYtHl golang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= golang.org/x/lint v0.0.0-20191125180803-fdd1cda4f05f/go.mod h1:5qLYkcX4OjUUV8bRuDixDT3tpyyb+LUpUlRWLxfhWrs= golang.org/x/lint v0.0.0-20200130185559-910be7a94367/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY= -golang.org/x/lint v0.0.0-20200302205851-738671d3881b h1:Wh+f8QHJXR411sJR8/vRBTZ7YapZaRvUcLFFJhusH0k= golang.org/x/lint v0.0.0-20200302205851-738671d3881b/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY= +golang.org/x/lint v0.0.0-20201208152925-83fdc39ff7b5/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY= +golang.org/x/lint v0.0.0-20210508222113-6edffad5e616/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY= golang.org/x/mobile v0.0.0-20190312151609-d3739f865fa6/go.mod h1:z+o9i4GpDbdi3rU15maQ/Ox0txvL9dWGYEHz965HBQE= golang.org/x/mobile v0.0.0-20190719004257-d2bd2a29d028/go.mod h1:E/iHnbuqvinMTCcRqshq8CkpyQDoeVncDDYHnLhea+o= golang.org/x/mod v0.0.0-20190513183733-4bf6d317e70e/go.mod h1:mXi4GBBbnImb6dmsKGUJ2LatrhH/nqhxcFungHvyanc= @@ -450,7 +541,9 @@ golang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzB golang.org/x/mod v0.1.1-0.20191107180719-034126e5016b/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg= golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= -golang.org/x/mod v0.3.1-0.20200828183125-ce943fd02449/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= +golang.org/x/mod v0.4.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= +golang.org/x/mod v0.4.1/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= +golang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= @@ -467,8 +560,8 @@ golang.org/x/net v0.0.0-20190503192946-f4e77d36d62c/go.mod h1:t9HGtf8HONx5eT2rtn golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks= golang.org/x/net v0.0.0-20190613194153-d28f0bde5980/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20190628185345-da137c7871d7/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20190724013045-ca1201d0de80/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= -golang.org/x/net v0.0.0-20190813141303-74dc4d7220e7/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20190827160401-ba9fcec4b297/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20200114155413-6afb5195e5aa/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= @@ -477,26 +570,55 @@ golang.org/x/net v0.0.0-20200222125558-5a598a2470a0/go.mod h1:z5CRVTTTmAJ677TzLL golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20200301022130-244492dfa37a/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20200324143707-d3edc9973b7e/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= +golang.org/x/net v0.0.0-20200501053045-e0ff5e5a1de5/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= +golang.org/x/net v0.0.0-20200506145744-7e3656a0809f/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= +golang.org/x/net v0.0.0-20200513185701-a91f0712d120/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= golang.org/x/net v0.0.0-20200520004742-59133d7f0dd7/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= +golang.org/x/net v0.0.0-20200520182314-0ba52f642ac2/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= golang.org/x/net v0.0.0-20200625001655-4c5254603344/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= +golang.org/x/net v0.0.0-20200707034311-ab3426394381/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= +golang.org/x/net v0.0.0-20200822124328-c89045814202/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= -golang.org/x/net v0.0.0-20210224082022-3d97a244fca7/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= -golang.org/x/net v0.0.0-20210428140749-89ef3d95e781 h1:DzZ89McO9/gWPsQXS/FVKAlG02ZjaQ6AlZRBimEYOd0= +golang.org/x/net v0.0.0-20201031054903-ff519b6c9102/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= +golang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= +golang.org/x/net v0.0.0-20201202161906-c7110b5ffcbb/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= +golang.org/x/net v0.0.0-20201209123823-ac852fbbde11/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= +golang.org/x/net v0.0.0-20210119194325-5f4716e94777/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= +golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= +golang.org/x/net v0.0.0-20210316092652-d523dce5a7f4/go.mod h1:RBQZq4jEuRlivfhVLdyRGr576XBO4/greRjx4P4O3yc= +golang.org/x/net v0.0.0-20210405180319-a5a99cb37ef4/go.mod h1:p54w0d4576C0XHj96bSt6lcn1PtDYWL6XObtHCRCNQM= golang.org/x/net v0.0.0-20210428140749-89ef3d95e781/go.mod h1:OJAsFXCWl8Ukc7SiCT/9KSuxbyM7479/AVlXFRxuMCk= +golang.org/x/net v0.0.0-20210525063256-abc453219eb5/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= +golang.org/x/net v0.0.0-20210805182204-aaa1db679c0d/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= +golang.org/x/net v0.0.0-20210825183410-e898025ed96a/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= +golang.org/x/net v0.0.0-20211209124913-491a49abca63 h1:iocB37TsdFuN6IBRZ+ry36wrkoV51/tl5vOWqkcPGvY= +golang.org/x/net v0.0.0-20211209124913-491a49abca63/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U= golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= golang.org/x/oauth2 v0.0.0-20191202225959-858c2ad4c8b6/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= -golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d h1:TzXSXBo42m9gQenoE3b9BGiEpg5IG2JkU5FkPIawgtw= golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= +golang.org/x/oauth2 v0.0.0-20200902213428-5d25da1a8d43/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= +golang.org/x/oauth2 v0.0.0-20201109201403-9fd604954f58/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= +golang.org/x/oauth2 v0.0.0-20201208152858-08078c50e5b5/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= +golang.org/x/oauth2 v0.0.0-20210218202405-ba52d332ba99/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= +golang.org/x/oauth2 v0.0.0-20210220000619-9bb904979d93/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= +golang.org/x/oauth2 v0.0.0-20210313182246-cd4f82c27b84/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= +golang.org/x/oauth2 v0.0.0-20210402161424-2e8d93401602/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= +golang.org/x/oauth2 v0.0.0-20210514164344-f6687ab2804c/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= +golang.org/x/oauth2 v0.0.0-20210819190943-2bc19b11175f h1:Qmd2pbz05z7z6lm0DrgQVVPuBm92jqujBKMHMOlOQEw= +golang.org/x/oauth2 v0.0.0-20210819190943-2bc19b11175f/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.0.0-20200317015054-43a5402ce75a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= @@ -511,10 +633,8 @@ golang.org/x/sys v0.0.0-20190422165155-953cdadca894/go.mod h1:h1NjWce9XRLGQEsW7w golang.org/x/sys v0.0.0-20190502145724-3ef323f4f1fd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190606165138-5da285871e9c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20190616124812-15dcb6c0061f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190624142023-c5567b49c5d0/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20190826190057-c7b8b68b1456/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190904154756-749cb33beabd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20191001151750-bb3f8db39f24/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20191005200804-aed5e4c7ecf9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= @@ -530,39 +650,63 @@ golang.org/x/sys v0.0.0-20200212091648-12a6c2dcc1e4/go.mod h1:h1NjWce9XRLGQEsW7w golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200302150141-5c8b2ff67527/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200331124033-c3d80250170d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200501052902-10377860bb8e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200511232937-7e40ca221e25/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200515095857-1151b9dac4a9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200519105757-fe76b779f299/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200523222454-059865788121/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200615200032-f1bc736245b1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200625212154-ddb9806d33ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200803210538-64077c9b5642/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200831180312-196b9ba8737a/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200905004654-be1d3432aa8f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200923182605-d9f96fdee20d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20201201145000-ef89a241ccb3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20210104204734-6f8348627aad/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210112080510-489259a85091/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210119212857-b64e53b001e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20210220050731-9a76102bfb43/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20210305230114-8fe3ee5dd75b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20210315160823-c6e025ad8005/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20210320140829-1e4c9ba3b0c4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20210403161142-5e06dd20ab57/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20210426230700-d19ff857e887/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20210603081109-ebe580a85c40 h1:JWgyZ1qgdTaF3N3oxC+MdTV7qvEEgHo3otj+HB5CM7Q= +golang.org/x/sys v0.0.0-20210510120138-977fb7262007/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20210603081109-ebe580a85c40/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw= +golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20210616094352-59db8d763f22/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20210809222454-d867a43fc93e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20210831042530-f4d43177bf5e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20211029165221-6e7872819dc8 h1:M69LAlWZCshgp0QSzyDcSsSIejIEeuaCVpmwcKwyLMk= +golang.org/x/sys v0.0.0-20211029165221-6e7872819dc8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= -golang.org/x/term v0.0.0-20210220032956-6a3ed077a48d h1:SZxvLBoTP5yHO3Frd4z4vrF+DBX9vMVanchswa69toE= -golang.org/x/term v0.0.0-20210220032956-6a3ed077a48d/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= +golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b h1:9zKuko04nR4gjZ4+DNjHqRlAJqbJETHwiNKDqTfOjfE= +golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8= golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk= golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= -golang.org/x/text v0.3.6 h1:aRYxNxv6iGQlyVaZmk6ZgYEDa+Jg18DxebPSrd6bg1M= +golang.org/x/text v0.3.5/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= -golang.org/x/time v0.0.0-20180412165947-fbb02b2291d2/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= +golang.org/x/text v0.3.7 h1:olpwvP2KacW1ZWvsR7uQhoyTYvKAupfQrRGBFM352Gk= +golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ= golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/time v0.0.0-20210220033141-f8bda1e9f3ba/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= -golang.org/x/time v0.0.0-20210611083556-38a9dc6acbc6 h1:Vv0JUPWTyeqUq42B2WJ1FeIDjjvGKoA2Ss+Ts0lAVbs= -golang.org/x/time v0.0.0-20210611083556-38a9dc6acbc6/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= +golang.org/x/time v0.0.0-20210723032227-1f47c861a9ac h1:7zkz7BUtwNFFqcowJ+RIgu2MaV/MapERkDIy+mwPyjs= +golang.org/x/time v0.0.0-20210723032227-1f47c861a9ac/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/tools v0.0.0-20180221164845-07fd8470d635/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= +golang.org/x/tools v0.0.0-20181030221726-6c7e314b6563/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY= golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= @@ -573,7 +717,6 @@ golang.org/x/tools v0.0.0-20190425150028-36563e24a262/go.mod h1:RgjU9mgBXZiqYHBn golang.org/x/tools v0.0.0-20190506145303-2d16b83fe98c/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= golang.org/x/tools v0.0.0-20190606124116-d0a3d012864b/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= -golang.org/x/tools v0.0.0-20190614205625-5aca471b1d59/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= golang.org/x/tools v0.0.0-20190621195816-6e04913cbbac/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= golang.org/x/tools v0.0.0-20190624222133-a101b041ded4/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= golang.org/x/tools v0.0.0-20190628153133-6cdbf07be9d0/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= @@ -596,13 +739,30 @@ golang.org/x/tools v0.0.0-20200204074204-1cc6d1ef6c74/go.mod h1:TB2adYChydJhpapK golang.org/x/tools v0.0.0-20200207183749-b753a1ba74fa/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= golang.org/x/tools v0.0.0-20200212150539-ea181f53ac56/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= golang.org/x/tools v0.0.0-20200224181240-023911ca70b2/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= +golang.org/x/tools v0.0.0-20200227222343-706bc42d1f0d/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= golang.org/x/tools v0.0.0-20200304193943-95d2e580d8eb/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw= +golang.org/x/tools v0.0.0-20200312045724-11d5b4c81c7d/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw= +golang.org/x/tools v0.0.0-20200331025713-a30bf2db82d4/go.mod h1:Sl4aGygMT6LrqrWclx+PTx3U+LnKx/seiNR+3G19Ar8= +golang.org/x/tools v0.0.0-20200501065659-ab2804fb9c9d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= golang.org/x/tools v0.0.0-20200505023115-26f46d2f7ef8/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= +golang.org/x/tools v0.0.0-20200512131952-2bc93b1c0c88/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= +golang.org/x/tools v0.0.0-20200515010526-7d3b6ebf133d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= +golang.org/x/tools v0.0.0-20200618134242-20370b0cb4b2/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= +golang.org/x/tools v0.0.0-20200729194436-6467de6f59a7/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA= +golang.org/x/tools v0.0.0-20200804011535-6c149bb5ef0d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA= +golang.org/x/tools v0.0.0-20200825202427-b303f430e36d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA= +golang.org/x/tools v0.0.0-20200904185747-39188db58858/go.mod h1:Cj7w3i3Rnn0Xh82ur9kSqwfTHTeVxaDqrfMjpcNT6bE= +golang.org/x/tools v0.0.0-20201110124207-079ba7bd75cd/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= +golang.org/x/tools v0.0.0-20201201161351-ac6f37ff4c2a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= +golang.org/x/tools v0.0.0-20201208233053-a543418bbed2/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= golang.org/x/tools v0.0.0-20201224043029-2b0845dc783e/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= +golang.org/x/tools v0.0.0-20210105154028-b0ab187a4818/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= -golang.org/x/tools v0.1.0 h1:po9/4sTYwZU9lPhi1tOrb4hCv3qrhiQ77LZfGa2OjwY= golang.org/x/tools v0.1.0/go.mod h1:xkSsbof2nBLbhDlRMhhhyNLN/zl3eTqcnHD5viDpcZ0= +golang.org/x/tools v0.1.2/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= +golang.org/x/tools v0.1.5/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= +golang.org/x/tools v0.1.6-0.20210820212750-d4cc65f0b2ff/go.mod h1:YD9qOF0M9xpSpdWTBbzEl5e/RnCefISl8E5Noe10jFM= golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= @@ -619,12 +779,25 @@ google.golang.org/api v0.14.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsb google.golang.org/api v0.15.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI= google.golang.org/api v0.17.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= google.golang.org/api v0.18.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= +google.golang.org/api v0.19.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= google.golang.org/api v0.20.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= +google.golang.org/api v0.22.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= +google.golang.org/api v0.24.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE= +google.golang.org/api v0.28.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE= +google.golang.org/api v0.29.0/go.mod h1:Lcubydp8VUV7KeIHD9z2Bys/sm/vGKnG1UHuDBSrHWM= +google.golang.org/api v0.30.0/go.mod h1:QGmEvQ87FHZNiUVJkT14jQNYJ4ZJjdRF23ZXz5138Fc= +google.golang.org/api v0.35.0/go.mod h1:/XrVsuzM0rZmrsbjJutiuftIzeuTQcEeaYcSk/mQ1dg= +google.golang.org/api v0.36.0/go.mod h1:+z5ficQTmoYpPn8LCUNVpK5I7hwkpjbcgqA7I34qYtE= +google.golang.org/api v0.40.0/go.mod h1:fYKFpnQN0DsDSKRVRcQSDQNtqWPfM9i+zNPxepjRCQ8= +google.golang.org/api v0.41.0/go.mod h1:RkxM5lITDfTzmyKFPt+wGrCJbVfniCr2ool8kTBzRTU= +google.golang.org/api v0.43.0/go.mod h1:nQsDGjRXMo4lvh5hP0TKqF244gqhGcr/YSIykhUk/94= +google.golang.org/api v0.44.0/go.mod h1:EBOGZqzyhtvMDoxwS97ctnh0zUmYY6CxqXsc1AvkYD8= google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM= google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4= google.golang.org/appengine v1.5.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4= google.golang.org/appengine v1.6.1/go.mod h1:i06prIuMbXzDqacNJfV5OdTW448YApPu5ww/cMBSeb0= google.golang.org/appengine v1.6.5/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc= +google.golang.org/appengine v1.6.6/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc= google.golang.org/appengine v1.6.7 h1:FZR1q0exgwxzPzp/aF+VccGrSfxfPpkBqjIIEq3ru6c= google.golang.org/appengine v1.6.7/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc= google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc= @@ -644,17 +817,56 @@ google.golang.org/genproto v0.0.0-20200122232147-0452cf42e150/go.mod h1:n3cpQtvx google.golang.org/genproto v0.0.0-20200204135345-fa8e72b47b90/go.mod h1:GmwEX6Z4W5gMy59cAlVYjN9JhxgbQH6Gn+gFDQe2lzA= google.golang.org/genproto v0.0.0-20200212174721-66ed5ce911ce/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= google.golang.org/genproto v0.0.0-20200224152610-e50cd9704f63/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= +google.golang.org/genproto v0.0.0-20200228133532-8c2c7df3a383/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= google.golang.org/genproto v0.0.0-20200305110556-506484158171/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= +google.golang.org/genproto v0.0.0-20200312145019-da6875a35672/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= +google.golang.org/genproto v0.0.0-20200331122359-1ee6d9798940/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= +google.golang.org/genproto v0.0.0-20200423170343-7949de9c1215/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= +google.golang.org/genproto v0.0.0-20200430143042-b979b6f78d84/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= +google.golang.org/genproto v0.0.0-20200511104702-f5ebc3bea380/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= +google.golang.org/genproto v0.0.0-20200513103714-09dca8ec2884/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= +google.golang.org/genproto v0.0.0-20200515170657-fc4c6c6a6587/go.mod h1:YsZOwe1myG/8QRHRsmBRE1LrgQY60beZKjly0O1fX9U= google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo= +google.golang.org/genproto v0.0.0-20200618031413-b414f8b61790/go.mod h1:jDfRM7FcilCzHH/e9qn6dsT145K34l5v+OpcnNgKAAA= +google.golang.org/genproto v0.0.0-20200729003335-053ba62fc06f/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= +google.golang.org/genproto v0.0.0-20200804131852-c06518451d9c/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= +google.golang.org/genproto v0.0.0-20200825200019-8632dd797987/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= +google.golang.org/genproto v0.0.0-20200904004341-0bd0a958aa1d/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= google.golang.org/genproto v0.0.0-20201019141844-1ed22bb0c154/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= -google.golang.org/genproto v0.0.0-20201110150050-8816d57aaa9a/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= +google.golang.org/genproto v0.0.0-20201102152239-715cce707fb0/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= +google.golang.org/genproto v0.0.0-20201109203340-2640f1f9cdfb/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= +google.golang.org/genproto v0.0.0-20201201144952-b05cb90ed32e/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= +google.golang.org/genproto v0.0.0-20201210142538-e3217bee35cc/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= +google.golang.org/genproto v0.0.0-20201214200347-8c77b98c765d/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= +google.golang.org/genproto v0.0.0-20210222152913-aa3ee6e6a81c/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= +google.golang.org/genproto v0.0.0-20210303154014-9728d6b83eeb/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= +google.golang.org/genproto v0.0.0-20210310155132-4ce2db91004e/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= +google.golang.org/genproto v0.0.0-20210319143718-93e7006c17a6/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= +google.golang.org/genproto v0.0.0-20210402141018-6c239bbf2bb1/go.mod h1:9lPAdzaEmUacj36I+k7YKbEc5CXzPIeORRgDAUOu28A= +google.golang.org/genproto v0.0.0-20210602131652-f16073e35f0c/go.mod h1:UODoCrxHCcBojKKwX1terBiRUaqAsFqJiF615XL43r0= +google.golang.org/genproto v0.0.0-20210831024726-fe130286e0e2/go.mod h1:eFjDcFEctNawg4eG61bRv87N7iHBWyVhJu7u1kqDUXY= google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c= google.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38= google.golang.org/grpc v1.21.1/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM= google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg= +google.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY= google.golang.org/grpc v1.26.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk= google.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk= google.golang.org/grpc v1.27.1/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk= +google.golang.org/grpc v1.28.0/go.mod h1:rpkK4SK4GF4Ach/+MFLZUBavHOvF2JJB5uozKKal+60= +google.golang.org/grpc v1.29.1/go.mod h1:itym6AZVZYACWQqET3MqgPpjcuV5QH3BxFS3IjizoKk= +google.golang.org/grpc v1.30.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak= +google.golang.org/grpc v1.31.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak= +google.golang.org/grpc v1.31.1/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak= +google.golang.org/grpc v1.33.1/go.mod h1:fr5YgcSWrqhRRxogOsw7RzIpsmvOZ6IcH4kBYTpR3n0= +google.golang.org/grpc v1.33.2/go.mod h1:JMHMWHQWaTccqQQlmk3MJZS+GWXOdAesneDmEnv2fbc= +google.golang.org/grpc v1.34.0/go.mod h1:WotjhfgOW/POjDeRt8vscBtXq+2VjORFy659qA51WJ8= +google.golang.org/grpc v1.35.0/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU= +google.golang.org/grpc v1.36.0/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU= +google.golang.org/grpc v1.36.1/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU= +google.golang.org/grpc v1.37.0/go.mod h1:NREThFqKR1f3iQ6oBuvc5LadQuXVGo9rkm5ZGrQdJfM= +google.golang.org/grpc v1.38.0/go.mod h1:NREThFqKR1f3iQ6oBuvc5LadQuXVGo9rkm5ZGrQdJfM= +google.golang.org/grpc v1.40.0/go.mod h1:ogyxbiOoUXAkP+4+xa6PZSE9DZgIHtSpzjDTB9KAK34= google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8= google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0= google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM= @@ -666,20 +878,21 @@ google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpAD google.golang.org/protobuf v1.24.0/go.mod h1:r/3tXBNzIEhYS9I1OUVjXDlt8tc493IdKGjtUeSXeh4= google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c= google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw= -google.golang.org/protobuf v1.26.0 h1:bxAC2xTBsZGibn2RTntX0oH50xLsqy1OxA9tTL3p/lk= google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc= +google.golang.org/protobuf v1.27.1 h1:SnqbnDw1V7RiZcXPx5MEeqPv2s79L9i7BJUlG/+RurQ= +google.golang.org/protobuf v1.27.1/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc= gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f h1:BLraFXnmrev5lT+xlilqcH8XK9/i0At2xKjWk4p6zsU= gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= -gopkg.in/cheggaaa/pb.v1 v1.0.25/go.mod h1:V/YB90LKu/1FcN3WVnfiiE5oMCibMjukxqG/qStrOgw= gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI= gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys= gopkg.in/inf.v0 v0.9.1 h1:73M5CoZyi3ZLMOyDlQh031Cx6N9NDJ2Vvfl76EDAgDc= gopkg.in/inf.v0 v0.9.1/go.mod h1:cWUDdTG/fYaXco+Dcufb5Vnc6Gp2YChqWtbxRZE0mXw= gopkg.in/ini.v1 v1.51.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k= +gopkg.in/ini.v1 v1.62.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k= gopkg.in/natefinch/lumberjack.v2 v2.0.0/go.mod h1:l0ndWWf7gzL7RNwBG7wST/UCcT4T24xpD6X8LsfU/+k= gopkg.in/resty.v1 v1.12.0/go.mod h1:mDo4pnntr5jdWRML875a/NmxYqAlA73dVijT2AXvQQo= gopkg.in/square/go-jose.v2 v2.2.2/go.mod h1:M9dMgbHiYLoDGQrXy7OpJDJWiKiU//h+vD76mk0e1AI= @@ -688,6 +901,7 @@ gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWD gopkg.in/yaml.v2 v2.0.0-20170812160011-eb3733d160e7/go.mod h1:JAlM8MvJe8wmxCU4Bli9HhUf9+ttbYbLASfIpnQbh74= gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= +gopkg.in/yaml.v2 v2.2.3/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.2.5/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= @@ -706,38 +920,44 @@ honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWh honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg= honnef.co/go/tools v0.0.1-2020.1.3/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k= -k8s.io/api v0.21.2 h1:vz7DqmRsXTCSa6pNxXwQ1IYeAZgdIsua+DZU+o+SX3Y= -k8s.io/api v0.21.2/go.mod h1:Lv6UGJZ1rlMI1qusN8ruAp9PUBFyBwpEHAdG24vIsiU= -k8s.io/apiextensions-apiserver v0.21.2 h1:+exKMRep4pDrphEafRvpEi79wTnCFMqKf8LBtlA3yrE= -k8s.io/apiextensions-apiserver v0.21.2/go.mod h1:+Axoz5/l3AYpGLlhJDfcVQzCerVYq3K3CvDMvw6X1RA= -k8s.io/apimachinery v0.21.2 h1:vezUc/BHqWlQDnZ+XkrpXSmnANSLbpnlpwo0Lhk0gpc= -k8s.io/apimachinery v0.21.2/go.mod h1:CdTY8fU/BlvAbJ2z/8kBwimGki5Zp8/fbVuLY8gJumM= -k8s.io/apiserver v0.21.2/go.mod h1:lN4yBoGyiNT7SC1dmNk0ue6a5Wi6O3SWOIw91TsucQw= -k8s.io/client-go v0.21.2 h1:Q1j4L/iMN4pTw6Y4DWppBoUxgKO8LbffEMVEV00MUp0= -k8s.io/client-go v0.21.2/go.mod h1:HdJ9iknWpbl3vMGtib6T2PyI/VYxiZfq936WNVHBRrA= -k8s.io/code-generator v0.21.2/go.mod h1:8mXJDCB7HcRo1xiEQstcguZkbxZaqeUOrO9SsicWs3U= -k8s.io/component-base v0.21.2 h1:EsnmFFoJ86cEywC0DoIkAUiEV6fjgauNugiw1lmIjs4= -k8s.io/component-base v0.21.2/go.mod h1:9lvmIThzdlrJj5Hp8Z/TOgIkdfsNARQ1pT+3PByuiuc= -k8s.io/gengo v0.0.0-20200413195148-3a45101e95ac/go.mod h1:ezvh/TsK7cY6rbqRK0oQQ8IAqLxYwwyPxAX1Pzy0ii0= -k8s.io/gengo v0.0.0-20201214224949-b6c5ce23f027/go.mod h1:FiNAH4ZV3gBg2Kwh89tzAEV2be7d5xI0vBa/VySYy3E= +honnef.co/go/tools v0.0.1-2020.1.4/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k= +k8s.io/api v0.23.0/go.mod h1:8wmDdLBHBNxtOIytwLstXt5E9PddnZb0GaMcqsvDBpg= +k8s.io/api v0.23.1 h1:ncu/qfBfUoClqwkTGbeRqqOqBCRoUAflMuOaOD7J0c8= +k8s.io/api v0.23.1/go.mod h1:WfXnOnwSqNtG62Y1CdjoMxh7r7u9QXGCkA1u0na2jgo= +k8s.io/apiextensions-apiserver v0.23.0 h1:uii8BYmHYiT2ZTAJxmvc3X8UhNYMxl2A0z0Xq3Pm+WY= +k8s.io/apiextensions-apiserver v0.23.0/go.mod h1:xIFAEEDlAZgpVBl/1VSjGDmLoXAWRG40+GsWhKhAxY4= +k8s.io/apimachinery v0.23.0/go.mod h1:fFCTTBKvKcwTPFzjlcxp91uPFZr+JA0FubU4fLzzFYc= +k8s.io/apimachinery v0.23.1 h1:sfBjlDFwj2onG0Ijx5C+SrAoeUscPrmghm7wHP+uXlo= +k8s.io/apimachinery v0.23.1/go.mod h1:SADt2Kl8/sttJ62RRsi9MIV4o8f5S3coArm0Iu3fBno= +k8s.io/apiserver v0.23.0/go.mod h1:Cec35u/9zAepDPPFyT+UMrgqOCjgJ5qtfVJDxjZYmt4= +k8s.io/client-go v0.23.0/go.mod h1:hrDnpnK1mSr65lHHcUuIZIXDgEbzc7/683c6hyG4jTA= +k8s.io/client-go v0.23.1 h1:Ma4Fhf/p07Nmj9yAB1H7UwbFHEBrSPg8lviR24U2GiQ= +k8s.io/client-go v0.23.1/go.mod h1:6QSI8fEuqD4zgFK0xbdwfB/PthBsIxCJMa3s17WlcO0= +k8s.io/code-generator v0.23.0/go.mod h1:vQvOhDXhuzqiVfM/YHp+dmg10WDZCchJVObc9MvowsE= +k8s.io/component-base v0.23.0 h1:UAnyzjvVZ2ZR1lF35YwtNY6VMN94WtOnArcXBu34es8= +k8s.io/component-base v0.23.0/go.mod h1:DHH5uiFvLC1edCpvcTDV++NKULdYYU6pR9Tt3HIKMKI= +k8s.io/gengo v0.0.0-20210813121822-485abfe95c7c/go.mod h1:FiNAH4ZV3gBg2Kwh89tzAEV2be7d5xI0vBa/VySYy3E= k8s.io/klog/v2 v2.0.0/go.mod h1:PBfzABfn139FHAV07az/IF9Wp1bkk3vpT2XSJ76fSDE= k8s.io/klog/v2 v2.2.0/go.mod h1:Od+F08eJP+W3HUb4pSrPpgp9DGU4GzlpG/TmITuYh/Y= -k8s.io/klog/v2 v2.8.0 h1:Q3gmuM9hKEjefWFFYF0Mat+YyFJvsUyYuwyNNJ5C9Ts= -k8s.io/klog/v2 v2.8.0/go.mod h1:hy9LJ/NvuK+iVyP4Ehqva4HxZG/oXyIS3n3Jmire4Ec= -k8s.io/kube-openapi v0.0.0-20210305001622-591a79e4bda7 h1:vEx13qjvaZ4yfObSSXW7BrMc/KQBBT/Jyee8XtLf4x0= -k8s.io/kube-openapi v0.0.0-20210305001622-591a79e4bda7/go.mod h1:wXW5VT87nVfh/iLV8FpR2uDvrFyomxbtb1KivDbvPTE= -k8s.io/utils v0.0.0-20201110183641-67b214c5f920/go.mod h1:jPW/WVKK9YHAvNhRxK0md/EJ228hCsBRufyofKtW8HA= -k8s.io/utils v0.0.0-20210527160623-6fdb442a123b h1:MSqsVQ3pZvPGTqCjptfimO2WjG7A9un2zcpiHkA6M/s= -k8s.io/utils v0.0.0-20210527160623-6fdb442a123b/go.mod h1:jPW/WVKK9YHAvNhRxK0md/EJ228hCsBRufyofKtW8HA= +k8s.io/klog/v2 v2.30.0 h1:bUO6drIvCIsvZ/XFgfxoGFQU/a4Qkh0iAlvUR7vlHJw= +k8s.io/klog/v2 v2.30.0/go.mod h1:y1WjHnz7Dj687irZUWR/WLkLc5N1YHtjLdmgWjndZn0= +k8s.io/kube-openapi v0.0.0-20211115234752-e816edb12b65 h1:E3J9oCLlaobFUqsjG9DfKbP2BmgwBL2p7pn0A3dG9W4= +k8s.io/kube-openapi v0.0.0-20211115234752-e816edb12b65/go.mod h1:sX9MT8g7NVZM5lVL/j8QyCCJe8YSMW30QvGZWaCIDIk= +k8s.io/utils v0.0.0-20210802155522-efc7438f0176/go.mod h1:jPW/WVKK9YHAvNhRxK0md/EJ228hCsBRufyofKtW8HA= +k8s.io/utils v0.0.0-20210930125809-cb0fa318a74b h1:wxEMGetGMur3J1xuGLQY7GEQYg9bZxKn3tKo5k/eYcs= +k8s.io/utils v0.0.0-20210930125809-cb0fa318a74b/go.mod h1:jPW/WVKK9YHAvNhRxK0md/EJ228hCsBRufyofKtW8HA= rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8= rsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0= rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA= -sigs.k8s.io/apiserver-network-proxy/konnectivity-client v0.0.19/go.mod h1:LEScyzhFmoF5pso/YSeBstl57mOzx9xlU9n85RGrDQg= -sigs.k8s.io/controller-runtime v0.9.2 h1:MnCAsopQno6+hI9SgJHKddzXpmv2wtouZz6931Eax+Q= -sigs.k8s.io/controller-runtime v0.9.2/go.mod h1:TxzMCHyEUpaeuOiZx/bIdc2T81vfs/aKdvJt9wuu0zk= +sigs.k8s.io/apiserver-network-proxy/konnectivity-client v0.0.25/go.mod h1:Mlj9PNLmG9bZ6BHFwFKDo5afkpWyUISkb9Me0GnK66I= +sigs.k8s.io/controller-runtime v0.11.0 h1:DqO+c8mywcZLFJWILq4iktoECTyn30Bkj0CwgqMpZWQ= +sigs.k8s.io/controller-runtime v0.11.0/go.mod h1:KKwLiTooNGu+JmLZGn9Sl3Gjmfj66eMbCQznLP5zcqA= +sigs.k8s.io/json v0.0.0-20211020170558-c049b76a60c6 h1:fD1pz4yfdADVNfFmcP2aBEtudwUQ1AlLnRBALr33v3s= +sigs.k8s.io/json v0.0.0-20211020170558-c049b76a60c6/go.mod h1:p4QtZmO4uMYipTQNzagwnNoseA6OxSUutVw05NhYDRs= sigs.k8s.io/structured-merge-diff/v4 v4.0.2/go.mod h1:bJZC9H9iH24zzfZ/41RGcq60oK1F7G282QMXDPYydCw= -sigs.k8s.io/structured-merge-diff/v4 v4.1.0 h1:C4r9BgJ98vrKnnVCjwCSXcWjWe0NKcUQkmzDXZXGwH8= -sigs.k8s.io/structured-merge-diff/v4 v4.1.0/go.mod h1:bJZC9H9iH24zzfZ/41RGcq60oK1F7G282QMXDPYydCw= -sigs.k8s.io/yaml v1.1.0/go.mod h1:UJmg0vDUVViEyp3mgSv9WPwZCDxu4rQW1olrI1uml+o= -sigs.k8s.io/yaml v1.2.0 h1:kr/MCeFWJWTwyaHoR9c8EjH9OumOmoF9YGiZd7lFm/Q= +sigs.k8s.io/structured-merge-diff/v4 v4.1.2/go.mod h1:j/nl6xW8vLS49O8YvXW1ocPhZawJtm+Yrr7PPRQ0Vg4= +sigs.k8s.io/structured-merge-diff/v4 v4.2.0 h1:kDvPBbnPk+qYmkHmSo8vKGp438IASWofnbbUKDE/bv0= +sigs.k8s.io/structured-merge-diff/v4 v4.2.0/go.mod h1:j/nl6xW8vLS49O8YvXW1ocPhZawJtm+Yrr7PPRQ0Vg4= sigs.k8s.io/yaml v1.2.0/go.mod h1:yfXDCHCao9+ENCvLSE62v9VSji2MKu5jeNfTrofGhJc= +sigs.k8s.io/yaml v1.3.0 h1:a2VclLzOGrwOHDiV8EfBGhvjHvP46CtW5j6POvhYGGo= +sigs.k8s.io/yaml v1.3.0/go.mod h1:GeOyir5tyXNByN85N/dRIT9es5UQNerPYEKK56eTBm8= diff --git a/test/e2e/autonomousdatabase_controller_bind_test.go b/test/e2e/autonomousdatabase_controller_bind_test.go index ebd6c4f0..3868399b 100644 --- a/test/e2e/autonomousdatabase_controller_bind_test.go +++ b/test/e2e/autonomousdatabase_controller_bind_test.go @@ -42,9 +42,9 @@ import ( "context" "time" - "github.com/oracle/oci-go-sdk/v51/common" - "github.com/oracle/oci-go-sdk/v51/database" - "github.com/oracle/oci-go-sdk/v51/workrequests" + "github.com/oracle/oci-go-sdk/v54/common" + "github.com/oracle/oci-go-sdk/v54/database" + "github.com/oracle/oci-go-sdk/v54/workrequests" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/types" diff --git a/test/e2e/autonomousdatabase_controller_create_test.go b/test/e2e/autonomousdatabase_controller_create_test.go index 74b0c09e..b0b9c91a 100644 --- a/test/e2e/autonomousdatabase_controller_create_test.go +++ b/test/e2e/autonomousdatabase_controller_create_test.go @@ -42,8 +42,8 @@ import ( "context" "time" - "github.com/oracle/oci-go-sdk/v51/common" - "github.com/oracle/oci-go-sdk/v51/database" + "github.com/oracle/oci-go-sdk/v54/common" + "github.com/oracle/oci-go-sdk/v54/database" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/types" diff --git a/test/e2e/behavior/shared_behaviors.go b/test/e2e/behavior/shared_behaviors.go index 593bc378..6253bd17 100644 --- a/test/e2e/behavior/shared_behaviors.go +++ b/test/e2e/behavior/shared_behaviors.go @@ -47,8 +47,8 @@ import ( "github.com/onsi/ginkgo" "github.com/onsi/gomega" - "github.com/oracle/oci-go-sdk/v51/common" - "github.com/oracle/oci-go-sdk/v51/database" + "github.com/oracle/oci-go-sdk/v54/common" + "github.com/oracle/oci-go-sdk/v54/database" corev1 "k8s.io/api/core/v1" k8sErrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/types" diff --git a/test/e2e/suite_test.go b/test/e2e/suite_test.go index 5f763e58..4c49c990 100644 --- a/test/e2e/suite_test.go +++ b/test/e2e/suite_test.go @@ -48,8 +48,8 @@ import ( "github.com/onsi/ginkgo" "github.com/onsi/gomega" "github.com/onsi/gomega/gexec" - "github.com/oracle/oci-go-sdk/v51/common" - "github.com/oracle/oci-go-sdk/v51/database" + "github.com/oracle/oci-go-sdk/v54/common" + "github.com/oracle/oci-go-sdk/v54/database" corev1 "k8s.io/api/core/v1" "k8s.io/client-go/kubernetes/scheme" "k8s.io/client-go/rest" diff --git a/test/e2e/util/oci_config_util.go b/test/e2e/util/oci_config_util.go index b7f33a5b..f6f2abd9 100644 --- a/test/e2e/util/oci_config_util.go +++ b/test/e2e/util/oci_config_util.go @@ -48,7 +48,7 @@ import ( "regexp" "strings" - "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v54/common" corev1 "k8s.io/api/core/v1" ) diff --git a/test/e2e/util/oci_db_request.go b/test/e2e/util/oci_db_request.go index ff73bf77..f02c2759 100644 --- a/test/e2e/util/oci_db_request.go +++ b/test/e2e/util/oci_db_request.go @@ -41,8 +41,8 @@ package e2eutil import ( "context" - "github.com/oracle/oci-go-sdk/v51/common" - "github.com/oracle/oci-go-sdk/v51/database" + "github.com/oracle/oci-go-sdk/v54/common" + "github.com/oracle/oci-go-sdk/v54/database" "time" ) diff --git a/test/e2e/util/oci_work_request.go b/test/e2e/util/oci_work_request.go index 450d9c3f..d15a1402 100644 --- a/test/e2e/util/oci_work_request.go +++ b/test/e2e/util/oci_work_request.go @@ -41,8 +41,8 @@ package e2eutil import ( "context" - "github.com/oracle/oci-go-sdk/v51/common" - "github.com/oracle/oci-go-sdk/v51/workrequests" + "github.com/oracle/oci-go-sdk/v54/common" + "github.com/oracle/oci-go-sdk/v54/workrequests" "time" ) From a1a971abe907fac57b487664f27e0b3b8f689aa5 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Fri, 7 Jan 2022 11:45:33 -0500 Subject: [PATCH 139/628] upgrade thrid party dependencies --- apis/database/v1alpha1/zz_generated.deepcopy.go | 1 + 1 file changed, 1 insertion(+) diff --git a/apis/database/v1alpha1/zz_generated.deepcopy.go b/apis/database/v1alpha1/zz_generated.deepcopy.go index 4d27bb59..08610d34 100644 --- a/apis/database/v1alpha1/zz_generated.deepcopy.go +++ b/apis/database/v1alpha1/zz_generated.deepcopy.go @@ -1,3 +1,4 @@ +//go:build !ignore_autogenerated // +build !ignore_autogenerated /* From 32dfa771144c079e53366f8a391df12be5b307a5 Mon Sep 17 00:00:00 2001 From: Gajanan Bhat Date: Fri, 21 Jan 2022 11:07:55 +0530 Subject: [PATCH 140/628] Update operator.yaml --- oracle-database-operator.yaml | 35 +++++++++++++++++++++++++++++++---- 1 file changed, 31 insertions(+), 4 deletions(-) diff --git a/oracle-database-operator.yaml b/oracle-database-operator.yaml index d87f4dd4..c2bf643b 100644 --- a/oracle-database-operator.yaml +++ b/oracle-database-operator.yaml @@ -456,11 +456,15 @@ spec: jsonPath: .spec.pdbName name: PDB Name type: string + - description: PDB Open Mode + jsonPath: .status.openMode + name: PDB State + type: string - description: Total Size of the PDB jsonPath: .spec.totalSize name: PDB Size type: string - - description: Status of the PDB + - description: Status of the PDB Resource jsonPath: .status.phase name: Status type: string @@ -496,6 +500,8 @@ spec: - Plug - Unplug - Delete + - Modify + - Status type: string adminName: description: The administrator username for the new PDB. This property @@ -566,10 +572,25 @@ spec: getScript: description: Whether you need the script only or execute the script type: boolean + modifyOption: + description: Extra options for opening and closing a PDB + enum: + - IMMEDIATE + - NORMAL + - READ ONLY + - READ WRITE + - RESTRICTED + type: string pdbName: description: The name of the new PDB. Relevant for both Create and Plug Actions. type: string + pdbState: + description: The target state of the PDB + enum: + - OPEN + - CLOSE + type: string reuseTempFile: description: Whether to reuse temp file type: boolean @@ -661,14 +682,20 @@ spec: connString: description: PDB Connect String type: string + modifyOption: + description: Modify Option of the PDB + type: string msg: description: Message type: string + openMode: + description: Open mode of the PDB + type: string phase: - description: Phase of the CDB Resource + description: Phase of the PDB Resource type: string status: - description: CDB Resource Status + description: PDB Resource Status type: boolean required: - phase @@ -1907,7 +1934,7 @@ spec: command: - /manager image: phx.ocir.io/intsanjaysingh/gdb-repo/oracle/dboper-ords-go:master - imagePullPolicy: Never + imagePullPolicy: Always name: manager ports: - containerPort: 9443 From 61e608b7d501f98cd2933747aefc603bb196651a Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Fri, 21 Jan 2022 16:44:09 -0500 Subject: [PATCH 141/628] restructure network apis and add webhooks --- PROJECT | 3 + .../v1alpha1/autonomousdatabase_types.go | 27 +- .../v1alpha1/autonomousdatabase_webhook.go | 206 ++++++++++++++ .../v1alpha1/zz_generated.deepcopy.go | 5 + commons/autonomousdatabase/reconciler_util.go | 253 ++++++++++++++++++ commons/oci/database.go | 168 +++++++----- ...tabase.oracle.com_autonomousdatabases.yaml | 9 +- .../adb/autonomousdatabase_create.yaml | 37 ++- ...nomousdatabase_update_admin_password.yaml} | 0 .../adb/autonomousdatabase_update_mtls.yaml | 20 ++ ...onomousdatabase_update_network_access.yaml | 47 ++++ config/webhook/manifests.yaml | 41 +++ .../database/autonomousdatabase_controller.go | 158 +---------- docs/adb/NETWORK_ACCESS_OPTIONS.md | 244 +++++++++++++++++ docs/adb/README.md | 10 +- main.go | 4 + test/e2e/behavior/shared_behaviors.go | 2 +- 17 files changed, 1001 insertions(+), 233 deletions(-) create mode 100644 apis/database/v1alpha1/autonomousdatabase_webhook.go rename config/samples/adb/{autonomousdatabase_change_admin_password.yaml => autonomousdatabase_update_admin_password.yaml} (100%) create mode 100644 config/samples/adb/autonomousdatabase_update_mtls.yaml create mode 100644 config/samples/adb/autonomousdatabase_update_network_access.yaml create mode 100644 docs/adb/NETWORK_ACCESS_OPTIONS.md diff --git a/PROJECT b/PROJECT index 70e34778..79a0d27b 100644 --- a/PROJECT +++ b/PROJECT @@ -16,6 +16,9 @@ resources: kind: AutonomousDatabase path: github.com/oracle/oracle-database-operator/apis/database/v1alpha1 version: v1alpha1 + webhooks: + validation: true + webhookVersion: v1 - api: crdVersion: v1 namespaced: true diff --git a/apis/database/v1alpha1/autonomousdatabase_types.go b/apis/database/v1alpha1/autonomousdatabase_types.go index 67bfa043..3f22e01d 100644 --- a/apis/database/v1alpha1/autonomousdatabase_types.go +++ b/apis/database/v1alpha1/autonomousdatabase_types.go @@ -68,11 +68,12 @@ type OCIConfigSpec struct { // AutonomousDatabaseDetails defines the detail information of AutonomousDatabase, corresponding to oci-go-sdk/database/AutonomousDatabase type AutonomousDatabaseDetails struct { - AutonomousDatabaseOCID *string `json:"autonomousDatabaseOCID,omitempty"` - CompartmentOCID *string `json:"compartmentOCID,omitempty"` - DisplayName *string `json:"displayName,omitempty"` - DbName *string `json:"dbName,omitempty"` - // +kubebuilder:validation:Enum:=OLTP;DW;AJD;APEX + AutonomousDatabaseOCID *string `json:"autonomousDatabaseOCID,omitempty"` + CompartmentOCID *string `json:"compartmentOCID,omitempty"` + AutonomousContainerDatabaseOCID *string `json:"autonomousContainerDatabaseOCID,omitempty"` + DisplayName *string `json:"displayName,omitempty"` + DbName *string `json:"dbName,omitempty"` + // +kubebuilder:validation:Enum:="OLTP";"DW";"AJD";"APEX" DbWorkload database.AutonomousDatabaseDbWorkloadEnum `json:"dbWorkload,omitempty"` IsDedicated *bool `json:"isDedicated,omitempty"` DbVersion *string `json:"dbVersion,omitempty"` @@ -108,7 +109,8 @@ const ( ) type NetworkAccessSpec struct { - AccessType NetworkAccessTypeEnum `json:"accessType"` + // +kubebuilder:validation:Enum:="";"PUBLIC";"RESTRICTED";"PRIVATE" + AccessType NetworkAccessTypeEnum `json:"accessType,omitempty"` IsAccessControlEnabled *bool `json:"isAccessControlEnabled,omitempty"` AccessControlList []string `json:"accessControlList,omitempty"` PrivateEndpoint PrivateEndpointSpec `json:"privateEndpoint,omitempty"` @@ -193,6 +195,7 @@ func (adb *AutonomousDatabase) UpdateLastSuccessfulSpec(kubeClient client.Client func (adb *AutonomousDatabase) UpdateAttrFromOCIAutonomousDatabase(ociObj database.AutonomousDatabase) *AutonomousDatabase { adb.Spec.Details.AutonomousDatabaseOCID = ociObj.Id adb.Spec.Details.CompartmentOCID = ociObj.CompartmentId + adb.Spec.Details.AutonomousContainerDatabaseOCID = ociObj.AutonomousContainerDatabaseId adb.Spec.Details.DisplayName = ociObj.DisplayName adb.Spec.Details.DbName = ociObj.DbName adb.Spec.Details.DbWorkload = ociObj.DbWorkload @@ -204,6 +207,18 @@ func (adb *AutonomousDatabase) UpdateAttrFromOCIAutonomousDatabase(ociObj databa adb.Spec.Details.LifecycleState = ociObj.LifecycleState adb.Spec.Details.FreeformTags = ociObj.FreeformTags + if *ociObj.IsDedicated { + adb.Spec.Details.NetworkAccess.AccessType = NetworkAccessTypePrivate + } else { + if ociObj.NsgIds != nil { + adb.Spec.Details.NetworkAccess.AccessType = NetworkAccessTypePrivate + } else if ociObj.WhitelistedIps != nil { + adb.Spec.Details.NetworkAccess.AccessType = NetworkAccessTypeRestricted + } else { + adb.Spec.Details.NetworkAccess.AccessType = NetworkAccessTypePublic + } + } + adb.Spec.Details.NetworkAccess.IsAccessControlEnabled = ociObj.IsAccessControlEnabled adb.Spec.Details.NetworkAccess.AccessControlList = ociObj.WhitelistedIps adb.Spec.Details.NetworkAccess.IsMTLSConnectionRequired = ociObj.IsMtlsConnectionRequired diff --git a/apis/database/v1alpha1/autonomousdatabase_webhook.go b/apis/database/v1alpha1/autonomousdatabase_webhook.go new file mode 100644 index 00000000..1c137eed --- /dev/null +++ b/apis/database/v1alpha1/autonomousdatabase_webhook.go @@ -0,0 +1,206 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v1alpha1 + +import ( + "fmt" + + apierrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/runtime/schema" + "k8s.io/apimachinery/pkg/util/validation/field" + ctrl "sigs.k8s.io/controller-runtime" + logf "sigs.k8s.io/controller-runtime/pkg/log" + "sigs.k8s.io/controller-runtime/pkg/webhook" +) + +// log is for logging in this package. +var autonomousdatabaselog = logf.Log.WithName("autonomousdatabase-resource") + +func (r *AutonomousDatabase) SetupWebhookWithManager(mgr ctrl.Manager) error { + return ctrl.NewWebhookManagedBy(mgr). + For(r). + Complete() +} + +// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! + +//+kubebuilder:webhook:verbs=create;update,path=/mutate-database-oracle-com-v1alpha1-autonomousdatabase,mutating=true,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=autonomousdatabase,versions=v1alpha1,name=mautonomousdatabase.kb.io,admissionReviewVersions={v1} + +var _ webhook.Defaulter = &AutonomousDatabase{} + +// Default implements webhook.Defaulter so a webhook will be registered for the type +func (r *AutonomousDatabase) Default() { + autonomousdatabaselog.Info("default", "name", r.Name) + + if !isDedicated(r) { // Shared database + // AccessType is PUBLIC by default + if r.Spec.Details.NetworkAccess.AccessType == "" { + r.Spec.Details.NetworkAccess.AccessType = NetworkAccessTypePublic + } + + // IsAccessControlEnabled is not applicable only to a shared database + if r.Spec.Details.NetworkAccess.IsAccessControlEnabled != nil { + r.Spec.Details.NetworkAccess.IsAccessControlEnabled = nil + } + } else { // Dedicated database + // AccessType can only be PRIVATE for a dedicated database + r.Spec.Details.NetworkAccess.AccessType = NetworkAccessTypePrivate + } + +} + +// TODO(user): change verbs to "verbs=create;update;delete" if you want to enable deletion validation. +//+kubebuilder:webhook:verbs=create;update;delete,path=/validate-database-oracle-com-v1alpha1-autonomousdatabase,mutating=false,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=autonomousdatabases,versions=v1alpha1,name=vautonomousdatabase.kb.io,admissionReviewVersions={v1} + +var _ webhook.Validator = &AutonomousDatabase{} + +// ValidateCreate implements webhook.Validator so a webhook will be registered for the type +// ValidateCreate checks if the spec is valid for a provisioning or a binding operation +func (r *AutonomousDatabase) ValidateCreate() error { + + var allErrs field.ErrorList + + autonomousdatabaselog.Info("validate create", "name", r.Name) + + if r.Spec.Details.AutonomousDatabaseOCID == nil { // provisioning operation + validateNetworkAcces(r, allErrs) + } else { // binding operation + } + + if len(allErrs) == 0 { + return nil + } + return apierrors.NewInvalid( + schema.GroupKind{Group: "database.oracle.com", Kind: "AutonomousDatabase"}, + r.Name, allErrs) +} + +// ValidateUpdate implements webhook.Validator so a webhook will be registered for the type +func (r *AutonomousDatabase) ValidateUpdate(old runtime.Object) error { + // Do ValidateCreate instead if there is no last successful spec, i.e. the database isn't provisioned or bound yet + lastSucSpec, err := r.GetLastSuccessfulSpec() + if err != nil { + return err + } + if lastSucSpec == nil { + return r.ValidateCreate() + } + + var allErrs field.ErrorList + + autonomousdatabaselog.Info("validate update", "name", r.Name) + + validateNetworkAcces(r, allErrs) + + if len(allErrs) == 0 { + return nil + } + return apierrors.NewInvalid( + schema.GroupKind{Group: "database.oracle.com", Kind: "AutonomousDatabase"}, + r.Name, allErrs) +} + +func validateNetworkAcces(adb *AutonomousDatabase, allErrs field.ErrorList) { + if !isDedicated(adb) { + // Shared database + if adb.Spec.Details.NetworkAccess.AccessType == NetworkAccessTypePublic { + if adb.Spec.Details.NetworkAccess.AccessControlList != nil || + adb.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID != nil || + adb.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs != nil { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("details").Child("networkAccess").Child("accessControlList"), + fmt.Sprintf("accessControlList, subnetOCID, nsgOCIDs cannot be provided when the network access type is %s", NetworkAccessTypePublic))) + } + + if adb.Spec.Details.NetworkAccess.IsMTLSConnectionRequired != nil && !*adb.Spec.Details.NetworkAccess.IsMTLSConnectionRequired { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("details").Child("networkAccess").Child("accessControlList"), + fmt.Sprintf("isMTLSConnectionRequired cannot be false when the network access type is %s", NetworkAccessTypePublic))) + } + } else if adb.Spec.Details.NetworkAccess.AccessType == NetworkAccessTypeRestricted { + if adb.Spec.Details.NetworkAccess.AccessControlList == nil { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("details").Child("networkAccess").Child("accessControlList"), + fmt.Sprintf("accessControlList cannot be empty when the network access type is %s", NetworkAccessTypeRestricted))) + } + } else { // the accessType is PRIVATE + if adb.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID == nil { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("details").Child("networkAccess").Child("privateEndpoint").Child("subnetOCID"), + fmt.Sprintf("subnetOCID cannot be empty when the network access type is %s", NetworkAccessTypePrivate))) + } + + if adb.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs == nil { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("details").Child("networkAccess").Child("privateEndpoint").Child("nsgOCIDs"), + fmt.Sprintf("nsgOCIDs cannot be empty when the network access type is %s", NetworkAccessTypePrivate))) + } + } + } else { + // Dedicated database + + // accessControlList cannot be provided when Autonomous Database's access control is disabled + if !*adb.Spec.Details.NetworkAccess.IsAccessControlEnabled && adb.Spec.Details.NetworkAccess.AccessControlList != nil { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("details").Child("networkAccess").Child("accessControlList"), + "access control list cannot be provided when Autonomous Database's access control is disabled")) + } + + // IsMTLSConnectionRequired is not supported by dedicated database + if adb.Spec.Details.NetworkAccess.IsMTLSConnectionRequired != nil { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("details").Child("networkAccess").Child("isMTLSConnectionRequired"), "isMTLSConnectionRequired is not supported on a dedicated database")) + } + } +} + +// ValidateDelete implements webhook.Validator so a webhook will be registered for the type +func (r *AutonomousDatabase) ValidateDelete() error { + autonomousdatabaselog.Info("validate delete", "name", r.Name) + + // TODO(user): fill in your validation logic upon object deletion. + return nil +} + +// Returns true if AutonomousContainerDatabaseOCID has value. +// We don't use Details.IsDedicated because the parameter might be null when it's a provision operation. +func isDedicated(adb *AutonomousDatabase) bool { + return adb.Spec.Details.AutonomousContainerDatabaseOCID != nil +} diff --git a/apis/database/v1alpha1/zz_generated.deepcopy.go b/apis/database/v1alpha1/zz_generated.deepcopy.go index e60ce721..62c78f84 100644 --- a/apis/database/v1alpha1/zz_generated.deepcopy.go +++ b/apis/database/v1alpha1/zz_generated.deepcopy.go @@ -89,6 +89,11 @@ func (in *AutonomousDatabaseDetails) DeepCopyInto(out *AutonomousDatabaseDetails *out = new(string) **out = **in } + if in.AutonomousContainerDatabaseOCID != nil { + in, out := &in.AutonomousContainerDatabaseOCID, &out.AutonomousContainerDatabaseOCID + *out = new(string) + **out = **in + } if in.DisplayName != nil { in, out := &in.DisplayName, &out.DisplayName *out = new(string) diff --git a/commons/autonomousdatabase/reconciler_util.go b/commons/autonomousdatabase/reconciler_util.go index 2f820410..f8095309 100644 --- a/commons/autonomousdatabase/reconciler_util.go +++ b/commons/autonomousdatabase/reconciler_util.go @@ -43,20 +43,273 @@ import ( "fmt" corev1 "k8s.io/api/core/v1" + apiErrors "k8s.io/apimachinery/pkg/api/errors" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/types" "k8s.io/client-go/util/retry" + ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" "github.com/go-logr/logr" "github.com/oracle/oci-go-sdk/v51/common" "github.com/oracle/oci-go-sdk/v51/database" "github.com/oracle/oci-go-sdk/v51/secrets" + "github.com/oracle/oci-go-sdk/v51/workrequests" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" "github.com/oracle/oracle-database-operator/commons/oci" ) +func DetermineReturn(logger logr.Logger, err error) (ctrl.Result, error) { + if apiErrors.IsConflict(err) { + return ctrl.Result{}, err + } + + logger.Error(err, "Reconcile failed") + return ctrl.Result{}, nil +} + +func UpdateGeneralAndPasswordAttributesAndWait(logger logr.Logger, kubeClient client.Client, + dbClient database.DatabaseClient, + secretClient secrets.SecretsClient, + workClient workrequests.WorkRequestClient, + adb *dbv1alpha1.AutonomousDatabase) error { + + updateGenPassResp, err := oci.UpdateGeneralAndPasswordAttributes(logger, kubeClient, dbClient, secretClient, adb) + if err != nil { + logger.Error(err, "Fail to update Autonomous Database") + + // Change the status to UNAVAILABLE + adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable + if statusErr := SetStatus(kubeClient, adb); statusErr != nil { + return statusErr + } + // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue + return err + } + + // Wait for the work finish if a request is sent. Note that some of the requests (e.g. update displayName) won't return a work request ID. + if updateGenPassResp.OpcWorkRequestId != nil { + if err := UpdateStatusAndWait(logger, kubeClient, workClient, adb, updateGenPassResp.AutonomousDatabase.LifecycleState, updateGenPassResp.OpcWorkRequestId); err != nil { + logger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*updateGenPassResp.OpcWorkRequestId) + } + + logger.Info("Update AutonomousDatabase " + *adb.Spec.Details.DbName + " succesfully") + } + + return nil +} + +func UpdateScaleAttributesAndWait(logger logr.Logger, kubeClient client.Client, + dbClient database.DatabaseClient, + workClient workrequests.WorkRequestClient, + adb *dbv1alpha1.AutonomousDatabase) error { + + scaleResp, err := oci.UpdateScaleAttributes(logger, kubeClient, dbClient, adb) + if err != nil { + logger.Error(err, "Fail to update Autonomous Database") + + // Change the status to UNAVAILABLE + adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable + if statusErr := SetStatus(kubeClient, adb); statusErr != nil { + return statusErr + } + // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue + return err + } + + if scaleResp.OpcWorkRequestId != nil { + if err := UpdateStatusAndWait(logger, kubeClient, workClient, adb, scaleResp.AutonomousDatabase.LifecycleState, scaleResp.OpcWorkRequestId); err != nil { + logger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*scaleResp.OpcWorkRequestId) + } + + logger.Info("Scale AutonomousDatabase " + *adb.Spec.Details.DbName + " succesfully") + } + + return nil +} + +// The logic of updating the network access configurations is as follows: +// 1. Shared databases: +// If the network access type changes +// a. to PUBLIC: +// was RESTRICTED: re-enable IsMTLSConnectionRequired if it's not. Then set WhitelistedIps to an array with a single empty string entry. +// was PRIVATE: re-enable IsMTLSConnectionRequired if it's not. Then set PrivateEndpointLabel to an emtpy string. +// b. to RESTRICTED: +// was PUBLIC: set WhitelistedIps to desired IPs/CIDR blocks/VCN OCID. Configure the IsMTLSConnectionRequired settings if it is set to disabled. +// was PRIVATE: re-enable IsMTLSConnectionRequired if it's not. Set the type to PUBLIC first, and then configure the WhitelistedIps. Finally resume the IsMTLSConnectionRequired settings if it was, or is configured as disabled. +// c. to PRIVATE: +// was PUBLIC: set subnetOCID and nsgOCIDs. Configure the IsMTLSConnectionRequired settings if it is set. +// was RESTRICTED: set subnetOCID and nsgOCIDs. Configure the IsMTLSConnectionRequired settings if it is set. +// +// Otherwise, if the network access type remains the same, apply the network configuration, and then set the IsMTLSConnectionRequired. +// +// 2. Dedicated databases: +// Apply the configs directly +func UpdateNetworkAttributes(logger logr.Logger, + kubeClient client.Client, + dbClient database.DatabaseClient, + workClient workrequests.WorkRequestClient, + curADB *dbv1alpha1.AutonomousDatabase) error { + + lastSucSpec, err := curADB.GetLastSuccessfulSpec() + if err != nil { + return err + } + + if !*curADB.Spec.Details.IsDedicated { + var lastAccessType = lastSucSpec.Details.NetworkAccess.AccessType + var curAccessType = curADB.Spec.Details.NetworkAccess.AccessType + + if oci.IsAttrChanged(lastAccessType, curAccessType) { + switch curAccessType { + case dbv1alpha1.NetworkAccessTypePublic: + // OCI validation requires IsMTLSConnectionRequired to be enabled before changing the network access type to PUBLIC + if !*lastSucSpec.Details.NetworkAccess.IsMTLSConnectionRequired { + curADB.Spec.Details.NetworkAccess.IsMTLSConnectionRequired = common.Bool(true) + if err := updateMTLSAndWait(logger, kubeClient, dbClient, workClient, curADB); err != nil { + return err + } + } + + if err := setNetworkAccessPublicAndWait(logger, kubeClient, dbClient, workClient, curADB); err != nil { + return err + } + case dbv1alpha1.NetworkAccessTypeRestricted: + // If the access type was PRIVATE, then OCI validation requires IsMTLSConnectionRequired + // to be enabled before setting ACL. Also we can only change the network access type from + // PRIVATE to PUBLIC. + if lastAccessType == dbv1alpha1.NetworkAccessTypePrivate { + if !*lastSucSpec.Details.NetworkAccess.IsMTLSConnectionRequired { + var oldMTLS bool = *curADB.Spec.Details.NetworkAccess.IsMTLSConnectionRequired + curADB.Spec.Details.NetworkAccess.IsMTLSConnectionRequired = common.Bool(true) + if err := updateMTLSAndWait(logger, kubeClient, dbClient, workClient, curADB); err != nil { + return err + } + // restore IsMTLSConnectionRequired + curADB.Spec.Details.NetworkAccess.IsMTLSConnectionRequired = &oldMTLS + } + + if err := setNetworkAccessPublicAndWait(logger, kubeClient, dbClient, workClient, curADB); err != nil { + return err + } + } + + if err := updateNetworkAccessAttributesAndWait(logger, kubeClient, dbClient, workClient, curADB); err != nil { + return err + } + + if err := updateMTLSAndWait(logger, kubeClient, dbClient, workClient, curADB); err != nil { + return err + } + case dbv1alpha1.NetworkAccessTypePrivate: + if err := updateNetworkAccessAttributesAndWait(logger, kubeClient, dbClient, workClient, curADB); err != nil { + return err + } + + if err := updateMTLSAndWait(logger, kubeClient, dbClient, workClient, curADB); err != nil { + return err + } + } + } else { + // Access type doesn't change + if err := updateNetworkAccessAttributesAndWait(logger, kubeClient, dbClient, workClient, curADB); err != nil { + return err + } + + if err := updateMTLSAndWait(logger, kubeClient, dbClient, workClient, curADB); err != nil { + return err + } + } + } else { + // Dedicated database + if err := updateNetworkAccessAttributesAndWait(logger, kubeClient, dbClient, workClient, curADB); err != nil { + return err + } + } + + return nil +} + +func updateMTLSAndWait(logger logr.Logger, + kubeClient client.Client, + dbClient database.DatabaseClient, + workClient workrequests.WorkRequestClient, + curADB *dbv1alpha1.AutonomousDatabase) error { + + resp, err := oci.UpdateMTLSConnectionRequired(logger, dbClient, curADB) + if err != nil { + return err + } + + if resp.OpcWorkRequestId != nil { + if err := UpdateStatusAndWait(logger, kubeClient, workClient, curADB, resp.AutonomousDatabase.LifecycleState, resp.OpcWorkRequestId); err != nil { + logger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*resp.OpcWorkRequestId) + } + } + + return nil +} + +func setNetworkAccessPublicAndWait(logger logr.Logger, + kubeClient client.Client, + dbClient database.DatabaseClient, + workClient workrequests.WorkRequestClient, + curADB *dbv1alpha1.AutonomousDatabase) error { + + resp, err := oci.SetNetworkAccessPublic(logger, dbClient, curADB) + if err != nil { + return err + } + + if resp.OpcWorkRequestId != nil { + if err := UpdateStatusAndWait(logger, kubeClient, workClient, curADB, resp.AutonomousDatabase.LifecycleState, resp.OpcWorkRequestId); err != nil { + logger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*resp.OpcWorkRequestId) + } + } + + return nil +} + +func updateNetworkAccessAttributesAndWait(logger logr.Logger, + kubeClient client.Client, + dbClient database.DatabaseClient, + workClient workrequests.WorkRequestClient, + curADB *dbv1alpha1.AutonomousDatabase) error { + + resp, err := oci.UpdateNetworkAccessAttributes(logger, dbClient, curADB) + if err != nil { + return err + } + + if resp.OpcWorkRequestId != nil { + if err := UpdateStatusAndWait(logger, kubeClient, workClient, curADB, resp.AutonomousDatabase.LifecycleState, resp.OpcWorkRequestId); err != nil { + logger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*resp.OpcWorkRequestId) + } + } + + return nil +} + +func UpdateStatusAndWait(logger logr.Logger, kubeClient client.Client, + workClient workrequests.WorkRequestClient, + adb *dbv1alpha1.AutonomousDatabase, + desiredLifecycleState database.AutonomousDatabaseLifecycleStateEnum, + opcWorkRequestID *string) error { + + // Update status.state + adb.Status.LifecycleState = desiredLifecycleState + if statusErr := SetStatus(kubeClient, adb); statusErr != nil { + return statusErr + } + + if err := oci.WaitUntilWorkCompleted(logger, workClient, opcWorkRequestID); err != nil { + return err + } + + return nil +} + // SetStatus sets the status subresource. func SetStatus(kubeClient client.Client, adb *dbv1alpha1.AutonomousDatabase) error { return retry.RetryOnConflict(retry.DefaultRetry, func() error { diff --git a/commons/oci/database.go b/commons/oci/database.go index d38aff1c..57a0e5d3 100644 --- a/commons/oci/database.go +++ b/commons/oci/database.go @@ -66,15 +66,16 @@ func CreateAutonomousDatabase(logger logr.Logger, kubeClient client.Client, dbCl } createAutonomousDatabaseDetails := database.CreateAutonomousDatabaseDetails{ - CompartmentId: adb.Spec.Details.CompartmentOCID, - DbName: adb.Spec.Details.DbName, - CpuCoreCount: adb.Spec.Details.CPUCoreCount, - DataStorageSizeInTBs: adb.Spec.Details.DataStorageSizeInTBs, - AdminPassword: common.String(adminPassword), - DisplayName: adb.Spec.Details.DisplayName, - IsAutoScalingEnabled: adb.Spec.Details.IsAutoScalingEnabled, - IsDedicated: adb.Spec.Details.IsDedicated, - DbVersion: adb.Spec.Details.DbVersion, + CompartmentId: adb.Spec.Details.CompartmentOCID, + AutonomousContainerDatabaseId: adb.Spec.Details.AutonomousContainerDatabaseOCID, + DbName: adb.Spec.Details.DbName, + CpuCoreCount: adb.Spec.Details.CPUCoreCount, + DataStorageSizeInTBs: adb.Spec.Details.DataStorageSizeInTBs, + AdminPassword: common.String(adminPassword), + DisplayName: adb.Spec.Details.DisplayName, + IsAutoScalingEnabled: adb.Spec.Details.IsAutoScalingEnabled, + IsDedicated: adb.Spec.Details.IsDedicated, + DbVersion: adb.Spec.Details.DbVersion, DbWorkload: database.CreateAutonomousDatabaseBaseDbWorkloadEnum( adb.Spec.Details.DbWorkload), @@ -160,12 +161,23 @@ func GetAutonomousDatabaseResource(logger logr.Logger, dbClient database.Databas return returnedADB, nil } -// isAttrChanged checks if the values of last successful object and current object are different. +// IsAttrChanged checks if the values of last successful object and current object are different. // The function returns false if the types are mismatch or unknown. // The function returns false if the current object has zero value (not applicable for boolean type). -func isAttrChanged(lastSucObj interface{}, curObj interface{}) bool { +func IsAttrChanged(lastSucObj interface{}, curObj interface{}) bool { switch curObj.(type) { - case string: // Enum + case dbv1alpha1.NetworkAccessTypeEnum: + lastSucAccessType, ok := lastSucObj.(dbv1alpha1.NetworkAccessTypeEnum) + if !ok { + return false + } + curSucAccessType, ok := curObj.(dbv1alpha1.NetworkAccessTypeEnum) + if !ok { + return false + } + + return lastSucAccessType != curSucAccessType + case string: // type check lastSucString, ok := lastSucObj.(string) if !ok { @@ -272,30 +284,30 @@ func UpdateGeneralAndPasswordAttributes(logger logr.Logger, kubeClient client.Cl // Prepare the update request updateAutonomousDatabaseDetails := database.UpdateAutonomousDatabaseDetails{} - if isAttrChanged(lastSucSpec.Details.DisplayName, curADB.Spec.Details.DisplayName) { + if IsAttrChanged(lastSucSpec.Details.DisplayName, curADB.Spec.Details.DisplayName) { updateAutonomousDatabaseDetails.DisplayName = curADB.Spec.Details.DisplayName shouldSendRequest = true } - if isAttrChanged(lastSucSpec.Details.DbName, curADB.Spec.Details.DbName) { + if IsAttrChanged(lastSucSpec.Details.DbName, curADB.Spec.Details.DbName) { updateAutonomousDatabaseDetails.DbName = curADB.Spec.Details.DbName shouldSendRequest = true } - if isAttrChanged(lastSucSpec.Details.DbWorkload, curADB.Spec.Details.DbWorkload) { + if IsAttrChanged(lastSucSpec.Details.DbWorkload, curADB.Spec.Details.DbWorkload) { updateAutonomousDatabaseDetails.DbWorkload = database.UpdateAutonomousDatabaseDetailsDbWorkloadEnum(curADB.Spec.Details.DbWorkload) shouldSendRequest = true } - if isAttrChanged(lastSucSpec.Details.DbVersion, curADB.Spec.Details.DbVersion) { + if IsAttrChanged(lastSucSpec.Details.DbVersion, curADB.Spec.Details.DbVersion) { updateAutonomousDatabaseDetails.DbVersion = curADB.Spec.Details.DbVersion shouldSendRequest = true } - if isAttrChanged(lastSucSpec.Details.FreeformTags, curADB.Spec.Details.FreeformTags) { + if IsAttrChanged(lastSucSpec.Details.FreeformTags, curADB.Spec.Details.FreeformTags) { updateAutonomousDatabaseDetails.FreeformTags = curADB.Spec.Details.FreeformTags shouldSendRequest = true } - if isAttrChanged(lastSucSpec.Details.AdminPassword.K8sSecretName, curADB.Spec.Details.AdminPassword.K8sSecretName) || - isAttrChanged(lastSucSpec.Details.AdminPassword.OCISecretOCID, curADB.Spec.Details.AdminPassword.OCISecretOCID) { + if IsAttrChanged(lastSucSpec.Details.AdminPassword.K8sSecretName, curADB.Spec.Details.AdminPassword.K8sSecretName) || + IsAttrChanged(lastSucSpec.Details.AdminPassword.OCISecretOCID, curADB.Spec.Details.AdminPassword.OCISecretOCID) { // Get the adminPassword var adminPassword string @@ -314,7 +326,6 @@ func UpdateGeneralAndPasswordAttributes(logger logr.Logger, kubeClient client.Cl logger.Info("Sending general attributes and ADMIN password update request") updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ - // AutonomousDatabaseId: common.String(curADB.Spec.Details.AutonomousDatabaseOCID), AutonomousDatabaseId: curADB.Spec.Details.AutonomousDatabaseOCID, UpdateAutonomousDatabaseDetails: updateAutonomousDatabaseDetails, } @@ -339,15 +350,15 @@ func UpdateScaleAttributes(logger logr.Logger, kubeClient client.Client, dbClien // Prepare the update request updateAutonomousDatabaseDetails := database.UpdateAutonomousDatabaseDetails{} - if isAttrChanged(lastSucSpec.Details.DataStorageSizeInTBs, curADB.Spec.Details.DataStorageSizeInTBs) { + if IsAttrChanged(lastSucSpec.Details.DataStorageSizeInTBs, curADB.Spec.Details.DataStorageSizeInTBs) { updateAutonomousDatabaseDetails.DataStorageSizeInTBs = curADB.Spec.Details.DataStorageSizeInTBs shouldSendRequest = true } - if isAttrChanged(lastSucSpec.Details.CPUCoreCount, curADB.Spec.Details.CPUCoreCount) { + if IsAttrChanged(lastSucSpec.Details.CPUCoreCount, curADB.Spec.Details.CPUCoreCount) { updateAutonomousDatabaseDetails.CpuCoreCount = curADB.Spec.Details.CPUCoreCount shouldSendRequest = true } - if isAttrChanged(lastSucSpec.Details.IsAutoScalingEnabled, curADB.Spec.Details.IsAutoScalingEnabled) { + if IsAttrChanged(lastSucSpec.Details.IsAutoScalingEnabled, curADB.Spec.Details.IsAutoScalingEnabled) { updateAutonomousDatabaseDetails.IsAutoScalingEnabled = curADB.Spec.Details.IsAutoScalingEnabled shouldSendRequest = true } @@ -358,7 +369,6 @@ func UpdateScaleAttributes(logger logr.Logger, kubeClient client.Client, dbClien logger.Info("Sending scale attributes update request") updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ - // AutonomousDatabaseId: common.String(curADB.Spec.Details.AutonomousDatabaseOCID), AutonomousDatabaseId: curADB.Spec.Details.AutonomousDatabaseOCID, UpdateAutonomousDatabaseDetails: updateAutonomousDatabaseDetails, } @@ -369,80 +379,96 @@ func UpdateScaleAttributes(logger logr.Logger, kubeClient client.Client, dbClien return } - - -func UpdateOneWayTLSAttribute(logger logr.Logger, kubeClient client.Client, dbClient database.DatabaseClient, +func UpdateMTLSConnectionRequired(logger logr.Logger, + dbClient database.DatabaseClient, curADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { - var shouldSendRequest = false lastSucSpec, err := curADB.GetLastSuccessfulSpec() if err != nil { return resp, err } - // Prepare the update request - updateAutonomousDatabaseDetails := database.UpdateAutonomousDatabaseDetails{} - - if isAttrChanged(lastSucSpec.Details.NetworkAccess.IsMTLSConnectionRequired, curADB.Spec.Details.NetworkAccess.IsMTLSConnectionRequired) { - updateAutonomousDatabaseDetails.IsMtlsConnectionRequired = curADB.Spec.Details.NetworkAccess.IsMTLSConnectionRequired - shouldSendRequest = true - } - - // Don't send the request if nothing is changed - if shouldSendRequest { - - logger.Info("Sending 1-way TLS attribute update request") + if IsAttrChanged(lastSucSpec.Details.NetworkAccess.IsMTLSConnectionRequired, curADB.Spec.Details.NetworkAccess.IsMTLSConnectionRequired) { + logger.Info(fmt.Sprintf("Sending request to set mTLSRequired to %t", *curADB.Spec.Details.NetworkAccess.IsMTLSConnectionRequired)) updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ - AutonomousDatabaseId: curADB.Spec.Details.AutonomousDatabaseOCID, - UpdateAutonomousDatabaseDetails: updateAutonomousDatabaseDetails, + AutonomousDatabaseId: curADB.Spec.Details.AutonomousDatabaseOCID, + UpdateAutonomousDatabaseDetails: database.UpdateAutonomousDatabaseDetails{ + IsMtlsConnectionRequired: curADB.Spec.Details.NetworkAccess.IsMTLSConnectionRequired, + }, } - resp, err = dbClient.UpdateAutonomousDatabase(context.TODO(), updateAutonomousDatabaseRequest) } return } -func UpdateNetworkAttributes(logger logr.Logger, kubeClient client.Client, dbClient database.DatabaseClient, +func SetNetworkAccessPublic(logger logr.Logger, dbClient database.DatabaseClient, curADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { - var shouldSendRequest = false - lastSucSpec, err := curADB.GetLastSuccessfulSpec() if err != nil { return resp, err } - // Prepare the update request + logger.Info("Sending request to configure Network Access type to PUBLIC") + updateAutonomousDatabaseDetails := database.UpdateAutonomousDatabaseDetails{} - // Network settings - if isAttrChanged(lastSucSpec.Details.NetworkAccess.IsAccessControlEnabled, curADB.Spec.Details.NetworkAccess.IsAccessControlEnabled) { - updateAutonomousDatabaseDetails.IsAccessControlEnabled = curADB.Spec.Details.NetworkAccess.IsAccessControlEnabled - shouldSendRequest = true - } - if isAttrChanged(lastSucSpec.Details.NetworkAccess.AccessControlList, curADB.Spec.Details.NetworkAccess.AccessControlList) { - updateAutonomousDatabaseDetails.WhitelistedIps = curADB.Spec.Details.NetworkAccess.AccessControlList - shouldSendRequest = true + if lastSucSpec.Details.NetworkAccess.AccessType == dbv1alpha1.NetworkAccessTypeRestricted { + updateAutonomousDatabaseDetails.WhitelistedIps = []string{""} + } else if lastSucSpec.Details.NetworkAccess.AccessType == dbv1alpha1.NetworkAccessTypePrivate { + updateAutonomousDatabaseDetails.PrivateEndpointLabel = common.String("") } - if isAttrChanged(lastSucSpec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID, curADB.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID) { - updateAutonomousDatabaseDetails.SubnetId = curADB.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID - shouldSendRequest = true - } - if isAttrChanged(lastSucSpec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs, curADB.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs) { - updateAutonomousDatabaseDetails.NsgIds = curADB.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs - shouldSendRequest = true + + updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ + AutonomousDatabaseId: curADB.Spec.Details.AutonomousDatabaseOCID, + UpdateAutonomousDatabaseDetails: updateAutonomousDatabaseDetails, } - // PrivateEndpointLabel - if isAttrChanged(lastSucSpec.Details.NetworkAccess.PrivateEndpoint.HostnamePrefix, curADB.Spec.Details.NetworkAccess.PrivateEndpoint.HostnamePrefix) { - updateAutonomousDatabaseDetails.PrivateEndpointLabel = curADB.Spec.Details.NetworkAccess.PrivateEndpoint.HostnamePrefix - shouldSendRequest = true + + return dbClient.UpdateAutonomousDatabase(context.TODO(), updateAutonomousDatabaseRequest) +} + +// UpdateNetworkAccessAttributes determines if any of the network access attributes changes and send the update request +func UpdateNetworkAccessAttributes(logger logr.Logger, dbClient database.DatabaseClient, + curADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { + + lastSucSpec, err := curADB.GetLastSuccessfulSpec() + if err != nil { + return resp, err } - // Don't send the request if nothing is changed - if shouldSendRequest { + var lastNetworkAccess = lastSucSpec.Details.NetworkAccess + var curNetworkAccess = curADB.Spec.Details.NetworkAccess - logger.Info("Sending network attributes update request") + if IsAttrChanged(lastNetworkAccess.IsAccessControlEnabled, curNetworkAccess.IsAccessControlEnabled) || + IsAttrChanged(lastNetworkAccess.AccessControlList, curNetworkAccess.AccessControlList) || + IsAttrChanged(lastNetworkAccess.PrivateEndpoint.SubnetOCID, curNetworkAccess.PrivateEndpoint.SubnetOCID) || + IsAttrChanged(lastNetworkAccess.PrivateEndpoint.NsgOCIDs, curNetworkAccess.PrivateEndpoint.NsgOCIDs) || + IsAttrChanged(lastNetworkAccess.PrivateEndpoint.HostnamePrefix, curNetworkAccess.PrivateEndpoint.HostnamePrefix) { + + logger.Info("Sending request to configure Network Access") + + updateAutonomousDatabaseDetails := database.UpdateAutonomousDatabaseDetails{} + + if IsAttrChanged(lastNetworkAccess.IsAccessControlEnabled, curNetworkAccess.IsAccessControlEnabled) { + updateAutonomousDatabaseDetails.IsAccessControlEnabled = curNetworkAccess.IsAccessControlEnabled + } + + if IsAttrChanged(lastNetworkAccess.AccessControlList, curNetworkAccess.AccessControlList) { + updateAutonomousDatabaseDetails.WhitelistedIps = curNetworkAccess.AccessControlList + } + + if IsAttrChanged(lastNetworkAccess.PrivateEndpoint.SubnetOCID, curNetworkAccess.PrivateEndpoint.SubnetOCID) { + updateAutonomousDatabaseDetails.SubnetId = curNetworkAccess.PrivateEndpoint.SubnetOCID + } + + if IsAttrChanged(lastNetworkAccess.PrivateEndpoint.NsgOCIDs, curNetworkAccess.PrivateEndpoint.NsgOCIDs) { + updateAutonomousDatabaseDetails.NsgIds = curNetworkAccess.PrivateEndpoint.NsgOCIDs + } + + if IsAttrChanged(lastNetworkAccess.PrivateEndpoint.HostnamePrefix, curNetworkAccess.PrivateEndpoint.HostnamePrefix) { + updateAutonomousDatabaseDetails.PrivateEndpointLabel = curNetworkAccess.PrivateEndpoint.HostnamePrefix + } updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ AutonomousDatabaseId: curADB.Spec.Details.AutonomousDatabaseOCID, @@ -538,10 +564,6 @@ func DeleteAutonomousDatabase(dbClient database.DatabaseClient, adbOCID string) } func WaitUntilWorkCompleted(logger logr.Logger, workClient workrequests.WorkRequestClient, opcWorkRequestID *string) error { - if opcWorkRequestID == nil { - return nil - } - logger.Info("Waiting for the work request to finish. opcWorkRequestID = " + *opcWorkRequestID) retryPolicy := getCompleteWorkRetryPolicy() diff --git a/config/crd/bases/database.oracle.com_autonomousdatabases.yaml b/config/crd/bases/database.oracle.com_autonomousdatabases.yaml index 0f428fe4..2fab5d0e 100644 --- a/config/crd/bases/database.oracle.com_autonomousdatabases.yaml +++ b/config/crd/bases/database.oracle.com_autonomousdatabases.yaml @@ -74,6 +74,8 @@ spec: ociSecretOCID: type: string type: object + autonomousContainerDatabaseOCID: + type: string autonomousDatabaseOCID: type: string compartmentOCID: @@ -116,6 +118,11 @@ spec: type: string type: array accessType: + enum: + - "" + - PUBLIC + - RESTRICTED + - PRIVATE type: string isAccessControlEnabled: type: boolean @@ -132,8 +139,6 @@ spec: subnetOCID: type: string type: object - required: - - accessType type: object wallet: properties: diff --git a/config/samples/adb/autonomousdatabase_create.yaml b/config/samples/adb/autonomousdatabase_create.yaml index fb58b656..2a3ffe82 100644 --- a/config/samples/adb/autonomousdatabase_create.yaml +++ b/config/samples/adb/autonomousdatabase_create.yaml @@ -9,7 +9,7 @@ metadata: spec: details: # Update compartmentOCID with your compartment OCID. - compartmentOCID: ocid1.compartment... + compartmentOCID: ocid1.compartment... OR ocid1.tenancy... # The dbName must begin with an alphabetic character and can contain a maximum of 14 alphanumeric characters. Special characters are not permitted. The database name must be unique in the tenancy. dbName: NewADB displayName: NewADB @@ -20,6 +20,41 @@ spec: # The OCID of the OCI Secret that holds the password of the ADMIN account. It should start with ocid1.vaultsecret... . # ociSecretOCID: ocid1.vaultsecret... dataStorageSizeInTBs: 1 + + # networkAccess: + # # Uncomment this block to configure the network access type with the PUBLIC option, which allows secure access from everywhere. + # accessType: PUBLIC + + # # Uncomment this block to configure the network access type with the RESTRICTED option. + # # This option lets you restrict access by defining access control rules in an Access Control List (ACL). + # # By specifying an ACL, the database will be accessible from a whitelisted set of IP addresses, CIDR (Classless Inter-Domain Routing) blocks, or VCNs. + # # Use a semicolon (;) as a deliminator between the VCN-specific subnets or IPs. + # accessType: RESTRICTED + # accessControlList: + # - 1.1.1.1 + # - 1.1.0.0/16 + # - ocid1.vcn... + # - ocid1.vcn...;1.1.1.1 + # - ocid1.vcn...;1.1.0.0/16 + # isMTLSConnectionRequired: true + + # # Uncomment this block to configure the network access type with the PRIVATE option. + # # This option assigns a private endpoint, private IP, and hostname to your database. + # # Specifying this option allows traffic only from the VCN you specify. + # # This allows you to define security rules, ingress/egress, at the Network Security Group (NSG) level and to control traffic to your Autonomous Database. + # accessType: PRIVATE + # privateEndpoint: + # subnetOCID: ocid1.subnet... + # nsgOCIDs: + # - ocid1.networksecuritygroup... + # isMTLSConnectionRequired: true + + # # Uncomment this block to configure the network access of an dedicated Autonomous Database (ADB-D) with an access control list. + # isAccessControlEnabled: true + # accessControlList: + # - 1.1.1.1 + # - 1.1.0.0/16 + # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. ociConfig: configMapName: oci-cred diff --git a/config/samples/adb/autonomousdatabase_change_admin_password.yaml b/config/samples/adb/autonomousdatabase_update_admin_password.yaml similarity index 100% rename from config/samples/adb/autonomousdatabase_change_admin_password.yaml rename to config/samples/adb/autonomousdatabase_update_admin_password.yaml diff --git a/config/samples/adb/autonomousdatabase_update_mtls.yaml b/config/samples/adb/autonomousdatabase_update_mtls.yaml new file mode 100644 index 00000000..f067f4c7 --- /dev/null +++ b/config/samples/adb/autonomousdatabase_update_mtls.yaml @@ -0,0 +1,20 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: AutonomousDatabase +metadata: + name: autonomousdatabase-sample +spec: + details: + autonomousDatabaseOCID: ocid1.autonomousdatabase... + networkAccess: + # Set the patameter to false to allow both TLS and mutual TLS (mTLS) authentication. + # Avaiable when the networkAccessType is RESTRICTED or PRIVATE on shared Autnomous Database. + isMTLSConnectionRequired: false + + # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey \ No newline at end of file diff --git a/config/samples/adb/autonomousdatabase_update_network_access.yaml b/config/samples/adb/autonomousdatabase_update_network_access.yaml new file mode 100644 index 00000000..ef7145eb --- /dev/null +++ b/config/samples/adb/autonomousdatabase_update_network_access.yaml @@ -0,0 +1,47 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: AutonomousDatabase +metadata: + name: autonomousdatabase-sample +spec: + details: + autonomousDatabaseOCID: ocid1.autonomousdatabase... + networkAccess: + # Allow secure access from everywhere. + accessType: PUBLIC + + # # Uncomment this block to configure the network access type with the RESTRICTED option. + # # This option lets you restrict access by defining access control rules in an Access Control List (ACL). + # # By specifying an ACL, the database will be accessible from a whitelisted set of IP addresses, CIDR (Classless Inter-Domain Routing) blocks, or VCNs. + # # Use a semicolon (;) as a deliminator between the VCN-specific subnets or IPs. + # accessType: RESTRICTED + # accessControlList: + # - 1.1.1.1 + # - 1.1.0.0/16 + # - ocid1.vcn... + # - ocid1.vcn...;1.1.1.1 + # - ocid1.vcn...;1.1.0.0/16 + + # # Uncomment this block to configure the network access type with the PRIVATE option. + # # This option assigns a private endpoint, private IP, and hostname to your database. + # # Specifying this option allows traffic only from the VCN you specify. + # # This allows you to define security rules, ingress/egress, at the Network Security Group (NSG) level and to control traffic to your Autonomous Database. + # accessType: PRIVATE + # privateEndpoint: + # subnetOCID: ocid1.subnet... + # nsgOCIDs: + # - ocid1.networksecuritygroup... + + # # Uncomment this block to configure the network access of an dedicated Autonomous Database (ADB-D) with an access control list. + # isAccessControlEnabled: true + # accessControlList: + # - 1.1.1.1 + # - 1.1.0.0/16 + + # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey \ No newline at end of file diff --git a/config/webhook/manifests.yaml b/config/webhook/manifests.yaml index 5cb37b30..c635bcc2 100644 --- a/config/webhook/manifests.yaml +++ b/config/webhook/manifests.yaml @@ -6,6 +6,26 @@ metadata: creationTimestamp: null name: mutating-webhook-configuration webhooks: +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: webhook-service + namespace: system + path: /mutate-database-oracle-com-v1alpha1-autonomousdatabase + failurePolicy: Fail + name: mautonomousdatabase.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + resources: + - autonomousdatabase + sideEffects: None - admissionReviewVersions: - v1 - v1beta1 @@ -35,6 +55,27 @@ metadata: creationTimestamp: null name: validating-webhook-configuration webhooks: +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: webhook-service + namespace: system + path: /validate-database-oracle-com-v1alpha1-autonomousdatabase + failurePolicy: Fail + name: vautonomousdatabase.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + - DELETE + resources: + - autonomousdatabases + sideEffects: None - admissionReviewVersions: - v1 - v1beta1 diff --git a/controllers/database/autonomousdatabase_controller.go b/controllers/database/autonomousdatabase_controller.go index 142e78c6..81d80419 100644 --- a/controllers/database/autonomousdatabase_controller.go +++ b/controllers/database/autonomousdatabase_controller.go @@ -268,20 +268,9 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R adb.Spec.Details.AutonomousDatabaseOCID = resp.AutonomousDatabase.Id - // Update status.state - adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr - } - - if err := oci.WaitUntilWorkCompleted(r.currentLogger, workClient, resp.OpcWorkRequestId); err != nil { + if err := adbutil.UpdateStatusAndWait(r.currentLogger, r.KubeClient, workClient, adb, + resp.AutonomousDatabase.LifecycleState, resp.OpcWorkRequestId); err != nil { r.currentLogger.Error(err, "Fail to watch the status of provision request. opcWorkRequestID = "+*resp.OpcWorkRequestId) - - // Change the status to UNAVAILABLE - adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr - } } r.currentLogger.Info("AutonomousDatabase " + *adb.Spec.Details.DbName + " provisioned succesfully") @@ -353,152 +342,27 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R return ctrl.Result{}, nil } - // Update status.state - adb.Status.LifecycleState = lifecycleState - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr - } - - if err := oci.WaitUntilWorkCompleted(r.currentLogger, workClient, opcWorkRequestID); err != nil { + if err := adbutil.UpdateStatusAndWait(r.currentLogger, r.KubeClient, workClient, adb, lifecycleState, opcWorkRequestID); err != nil { r.currentLogger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*opcWorkRequestID) - - // Change the status to UNAVAILABLE - adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr - } } - r.currentLogger.Info(fmt.Sprintf("Set AutonomousDatabase %s lifecycle state to %s successfully\n", + + r.currentLogger.Info(fmt.Sprintf("Change AutonomousDatabase %s lifecycle state to %s successfully\n", *adb.Spec.Details.DbName, adb.Spec.Details.LifecycleState)) } // Update the database in OCI from the local resource. // The local resource will be synchronized again later. - updateGenPassResp, err := oci.UpdateGeneralAndPasswordAttributes(r.currentLogger, r.KubeClient, dbClient, secretClient, adb) - if err != nil { - r.currentLogger.Error(err, "Fail to update Autonomous Database") - - // Change the status to UNAVAILABLE - adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr - } - // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue - return ctrl.Result{}, nil - } - - if updateGenPassResp.OpcWorkRequestId != nil { - // Update status.state - adb.Status.LifecycleState = updateGenPassResp.AutonomousDatabase.LifecycleState - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr - } - - if err := oci.WaitUntilWorkCompleted(r.currentLogger, workClient, updateGenPassResp.OpcWorkRequestId); err != nil { - r.currentLogger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*updateGenPassResp.OpcWorkRequestId) - - // Change the status to UNAVAILABLE - adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr - } - } - r.currentLogger.Info("Update AutonomousDatabase " + *adb.Spec.Details.DbName + " succesfully") - } - - scaleResp, err := oci.UpdateScaleAttributes(r.currentLogger, r.KubeClient, dbClient, adb) - if err != nil { - r.currentLogger.Error(err, "Fail to update Autonomous Database") - - // Change the status to UNAVAILABLE - adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr - } - // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue - return ctrl.Result{}, nil + if err := adbutil.UpdateGeneralAndPasswordAttributesAndWait(r.currentLogger, r.KubeClient, dbClient, secretClient, workClient, adb); err != nil { + return adbutil.DetermineReturn(r.currentLogger, err) } - if scaleResp.OpcWorkRequestId != nil { - // Update status.state - adb.Status.LifecycleState = scaleResp.AutonomousDatabase.LifecycleState - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr - } - - if err := oci.WaitUntilWorkCompleted(r.currentLogger, workClient, scaleResp.OpcWorkRequestId); err != nil { - r.currentLogger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*scaleResp.OpcWorkRequestId) - - // Change the status to UNAVAILABLE - adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr - } - } - r.currentLogger.Info("Scale AutonomousDatabase " + *adb.Spec.Details.DbName + " succesfully") - } - - oneWayTLSResp, err := oci.UpdateOneWayTLSAttribute(r.currentLogger, r.KubeClient, dbClient, adb) - if err != nil { - r.currentLogger.Error(err, "Fail to update Autonomous Database") - - // Change the status to UNAVAILABLE - adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr - } - // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue + if err := adbutil.UpdateScaleAttributesAndWait(r.currentLogger, r.KubeClient, dbClient, workClient, adb); err != nil { + return adbutil.DetermineReturn(r.currentLogger, err) } - if oneWayTLSResp.OpcWorkRequestId != nil { - // Update status.state - adb.Status.LifecycleState = oneWayTLSResp.AutonomousDatabase.LifecycleState - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr - } - - if err := oci.WaitUntilWorkCompleted(r.currentLogger, workClient, oneWayTLSResp.OpcWorkRequestId); err != nil { - r.currentLogger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*oneWayTLSResp.OpcWorkRequestId) - - // Change the status to UNAVAILABLE - adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr - } - } - r.currentLogger.Info("Update AutonomousDatabase " + *adb.Spec.Details.DbName + " 1-way TLS setting succesfully") - } - - networkResp, err := oci.UpdateNetworkAttributes(r.currentLogger, r.KubeClient, dbClient, adb) - if err != nil { - r.currentLogger.Error(err, "Fail to update Autonomous Database") - - // Change the status to UNAVAILABLE - adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr - } - // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue - } - - if networkResp.OpcWorkRequestId != nil { - // Update status.state - adb.Status.LifecycleState = networkResp.AutonomousDatabase.LifecycleState - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr - } - - if err := oci.WaitUntilWorkCompleted(r.currentLogger, workClient, networkResp.OpcWorkRequestId); err != nil { - r.currentLogger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*networkResp.OpcWorkRequestId) - - // Change the status to UNAVAILABLE - adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr - } - } - r.currentLogger.Info("Update AutonomousDatabase " + *adb.Spec.Details.DbName + " network settings succesfully") + if err := adbutil.UpdateNetworkAttributes(r.currentLogger, r.KubeClient, dbClient, workClient, adb); err != nil { + return adbutil.DetermineReturn(r.currentLogger, err) } } } diff --git a/docs/adb/NETWORK_ACCESS_OPTIONS.md b/docs/adb/NETWORK_ACCESS_OPTIONS.md new file mode 100644 index 00000000..908fde32 --- /dev/null +++ b/docs/adb/NETWORK_ACCESS_OPTIONS.md @@ -0,0 +1,244 @@ +# Configuring Network Access of Autonomous Database + +This documentation describes how to configure network access with public access, access control rules (ACLs), or private endpoints. Also describes how to configure the TLS connections (require mutual TLS only or allow both 1 way TLS and mutual TLS). For more information, please visit [this page](https://docs.oracle.com/en/cloud/paas/autonomous-database/adbsa/autonomous-network-access.html#GUID-D2D468C3-CA2D-411E-92BC-E122F795A413). + +## Supported Features + +### Types of Network Access + +There are three types of network access supported by Autonomous Database: + +* **PUBLIC**: + + This option allows secure access from anywhere. The network access type is PUBLIC if no option is specified in the spec. With this option, mutual TLS (mTLS) authentication is always required to connect to the database. This option is available only for databases on shared Exadata infrastructure. + +* **RESTRICTED**: + + This option restricts connections to the database according to the access control lists (ACLs) you specify. This option is available only for databases on shared Exadata infrastructure. + + You can add the following to your ACL: + * **IP Address**: Specify one or more individual public IP address. Use commas to separate your addresses in the input field. + * **CIDR Block**: Specify one or more ranges of public IP addresses using CIDR notation. Use commas to separate your CIDR block entries in the input field. + * **Virtual Cloud Network (OCID)** (applies to Autonomous Databases on shared Exadata infrastructure): Specify the OCID of a virtual cloud network (VCN). If you want to specify multiple IP addresses or CIDR ranges within the same VCN, then do not create multiple access control list entries. Use one access control list entry with the values for the multiple IP addresses or CIDR ranges separated by commas. + +* **PRIVATE**: + + This option creates a private endpoint for your database within a specified VCN. This option is available for databases on shared Exadata infrastructure and is the only available option for databases on dedicated Exadata infrastructure. + + * **Autonomous Databases on shared Exadata infrastructure**: + + This option allows the access through private enpoints by specifying the OCIDs of a subnet and network security groups (NSGs) under the same VCN in the spec. + + * **Autonomous Databases on dedicated Exadata infrastructure**: + + The network path to a dedicated Autonomous Database is through a VCN and subnet defined by the dedicated infrastucture hosting the database. Usually, the subnet is defined as private, meaning that there is no public Internet access to databases. + + Autonomous Database supports restricted access using a ACL. You can optionally enabling an ACL by setting the `isAccessControlEnabled` parameter. If disabled, database access is defined by the network security rules. If enabled, database access is restricted to the IP addresses and CIDR blocks defined in the ACL. Note that enabling an ACL with an empty list of IP addresses makes the database inaccessible. See [Autonomous Database with Private Endpoint](https://docs.oracle.com/en-us/iaas/Content/Database/Concepts/adbsprivateaccess.htm) for overview and examples for private endpoint. + +### Allowing TLS or Require Only Mutual TLS (mTLS) Authentication + +If your Autonomous Database instance is configured to only allow mTLS connections, you can update the instance to allow both mTLS and TLS connections. When you update your configuration to allow both mTLS and TLS, you can use both authentication types at the same time and connections are no longer restricted to require mTLS authentication. + +This option only applies to Autonomous Databases on shared Exadata infrastructure, and you can allow TLS connections when network access type is configured as follows: + +* **RESTRICTED**: with ACLs defined. +* **PRIVATE**: with a private endpoint defined. + +## Sample YAML + +You can always configure the network access options when you create an Autonomous Database, or update the settings after the creation. Following are some sample YAMLs which configure the networking with different newtwork access options. + +For Autonomous Databases on shared Exadata infrastructure, you can: + +* Configure network access [with PUBLIC access type](#autonomous-database-with-public-access-type-on-shared-exadata-infrastructure) +* Configure network access [with RESTRICTED access type](#autonomous-database-with-restricted-access-type-on-shared-exadata-infrastructure) +* Configure network access [with PRIVATE access type](#autonomous-database-with-private-access-type-on-shared-exadata-infrastructure) +* [Change the mutual TLS (mTLS) authentication setting](#allow-both-tls-and-mutual-tls-mtls-authentication-of-autonomous-database-on-shared-exadata-infrastructure) + +For Autonomous Databases on dedicated Exadata infrastructure, you can: + +* Configure network access [with access control list enabled](#autonomous-database-with-access-control-list-enabled-on-dedicated-exadata-infrastructure) + +> Note: +> +> * The above operations require an `AutonomousDatabase` object to be in your cluster. This example assumes either the provision operation or the bind operation has been done by the users and the operator is authorized with API Key Authentication. +> * If you are creating an Autonomous Database, see step 4 of [Provision an Autonomous Database](./README.md#provision-an-autonomous-database) in [Managing Oracle Autonomous Databases with Oracle Database Operator for Kubernetes](./README.md) topic to return to provisioning instructions. + +### Autonomous Database with PUBLIC access type on shared Exadata infrastructure + +Follow the steps to configure the network with PUBLIC access type. + +1. Add the following parameters to the spec. An example file is availble here: [`config/samples/adb/autonomousdatabase_update_network_access.yaml`](./../../config/samples/adb/autonomousdatabase_update_network_access.yaml): + + | Attribute | Type | Description | Required? | + |----|----|----|----| + | `networkAccess.accessType` | string | An enum value which defines how the database can be accessed. The value can be PUBLIC, RESTRICTED or PRIVATE. See [Types of Network Access](#types-of-network-access) for more descriptions. | Yes | + + ```yaml + --- + apiVersion: database.oracle.com/v1alpha1 + kind: AutonomousDatabase + metadata: + name: autonomousdatabase-sample + spec: + details: + autonomousDatabaseOCID: ocid1.autonomousdatabase... + networkAccess: + # Allow secure access from everywhere. + accessType: PUBLIC + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey + ``` + +2. Apply the yaml: + + ```sh + kubectl apply -f config/samples/adb/autonomousdatabase_update_network_access.yaml + autonomousdatabase.database.oracle.com/autonomousdatabase-sample configured + ``` + +### Autonomous Database with RESTRICTED access type on shared Exadata infrastructure + +Follow the steps to configure the network with RESTRICTED access type. + +1. Add the following parameters to the spec. An example file is availble here: [`config/samples/adb/autonomousdatabase_update_network_access.yaml`](./../../config/samples/adb/autonomousdatabase_update_network_access.yaml): + + | Attribute | Type | Description | Required? | + |----|----|----|----| + | `networkAccess.accessType` | string | An enum value which defines how the database can be accessed. The value can be PUBLIC, RESTRICTED or PRIVATE. See [Types of Network Access](#types-of-network-access) for more descriptions. | Yes | + | `networkAccess.accessControlList` | []string | The client IP access control list (ACL). This feature is available for autonomous databases on [shared Exadata infrastructure](https://docs.cloud.oracle.com/Content/Database/Concepts/adboverview.htm#AEI) and on Exadata Cloud@Customer.
Only clients connecting from an IP address included in the ACL may access the Autonomous Database instance.

For shared Exadata infrastructure, this is an array of CIDR (Classless Inter-Domain Routing) notations for a subnet or VCN OCID.
Use a semicolon (;) as a deliminator between the VCN-specific subnets or IPs.
Example: `["1.1.1.1","1.1.1.0/24","ocid1.vcn.oc1.sea.","ocid1.vcn.oc1.sea.;1.1.1.1","ocid1.vcn.oc1.sea.;1.1.0.0/16"]`

For Exadata Cloud@Customer, this is an array of IP addresses or CIDR (Classless Inter-Domain Routing) notations.
Example: `["1.1.1.1","1.1.1.0/24","1.1.2.25"]`

For an update operation, if you want to delete all the IPs in the ACL, use an array with a single empty string entry. | Yes | + + ```yaml + --- + apiVersion: database.oracle.com/v1alpha1 + kind: AutonomousDatabase + metadata: + name: autonomousdatabase-sample + spec: + details: + autonomousDatabaseOCID: ocid1.autonomousdatabase... + networkAccess: + # Restrict access by defining access control rules in an Access Control List (ACL). + accessType: RESTRICTED + accessControlList: + - 1.1.1.1 + - 1.1.0.0/16 + - ocid1.vcn... + - ocid1.vcn...;1.1.1.1 + - ocid1.vcn...;1.1.0.0/16 + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey + ``` + +2. Apply the yaml: + + ```sh + kubectl apply -f config/samples/adb/autonomousdatabase_update_network_access.yaml + autonomousdatabase.database.oracle.com/autonomousdatabase-sample configured + +### Autonomous Database with PRIVATE access type on shared Exadata infrastructure + +Follow the steps to configure the network with RESTRICTED access type. + +1. Visit [Overview of VCNs and Subnets](https://docs.oracle.com/en-us/iaas/Content/Network/Tasks/managingVCNs_topic-Overview_of_VCNs_and_Subnets.htm#console) and [Network Security Groups](https://docs.oracle.com/en-us/iaas/Content/Network/Concepts/networksecuritygroups.htm#working) to see how to create VCNs, subnets, and network security groups (NSGs) if you haven't created them yet. The subnet and the NSG has to be in the same VCN. + +2. Copy and paste the OCIDs of the subnet and NSG to the corresponding parameters. An example file is availble here: [`config/samples/adb/autonomousdatabase_update_network_access.yaml`](./../../config/samples/adb/autonomousdatabase_update_network_access.yaml): + + | Attribute | Type | Description | Required? | + |----|----|----|----| + | `networkAccess.accessType` | string | An enum value which defines how the database can be accessed. The value can be PUBLIC, RESTRICTED or PRIVATE. See [Types of Network Access](#types-of-network-access) for more descriptions. | Yes | + | `networkAccess.privateEndpoint.subnetOCID` | string | The [OCID](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm) of the subnet the resource is associated with.

**Subnet Restrictions:**
- For bare metal DB systems and for single node virtual machine DB systems, do not use a subnet that overlaps with 192.168.16.16/28.
- For Exadata and virtual machine 2-node RAC systems, do not use a subnet that overlaps with 192.168.128.0/20.
- For Autonomous Database, setting this will disable public secure access to the database.
These subnets are used by the Oracle Clusterware private interconnect on the database instance.
Specifying an overlapping subnet will cause the private interconnect to malfunction.
This restriction applies to both the client subnet and the backup subnet. | Yes | + | `networkAccess.privateEndpoint.nsgOCIDs` | string[] | A list of the [OCIDs](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm) of the network security groups (NSGs) that this resource belongs to. Setting this to an empty array after the list is created removes the resource from all NSGs. For more information about NSGs, see [Security Rules](https://docs.cloud.oracle.com/Content/Network/Concepts/securityrules.htm).

**NsgOCIDs restrictions:**
- Autonomous Databases with private access require at least 1 Network Security Group (NSG). The nsgOCIDs array cannot be empty. | Yes | + | `networkAccess.privateEndpoint.hostnamePrefix` | string | The hostname prefix for the resource. | No | + + ```yaml + --- + apiVersion: database.oracle.com/v1alpha1 + kind: AutonomousDatabase + metadata: + name: autonomousdatabase-sample + spec: + details: + autonomousDatabaseOCID: ocid1.autonomousdatabase... + networkAccess: + # Assigns a private endpoint, private IP, and hostname to your database. + accessType: PRIVATE + privateEndpoint: + subnetOCID: ocid1.subnet... + nsgOCIDs: + - ocid1.networksecuritygroup... + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey + ``` + +### Allow both TLS and mutual TLS (mTLS) authentication of Autonomous Database on shared Exadata infrastructure + +If you are using the RESTRICTED or the PRIVATE network access option, you can choose whether to allow both TLS and mutual TLS (mTLS) authentication, or to allow only mTLS authentication. Follow the steps to change the mTLS authentication setting. + +1. Add the following parameters to the spec. An example file is availble here: [`config/samples/adb/autonomousdatabase_update_mtls.yaml`](./../../config/samples/adb/autonomousdatabase_update_mtls.yaml): + + | Attribute | Type | Description | Required? | + |----|----|----|----| + | `networkAccess.isMTLSConnectionRequired` | boolean| Indicates whether the Autonomous Database requires mTLS connections. | Yes | + + ```yaml + --- + apiVersion: database.oracle.com/v1alpha1 + kind: AutonomousDatabase + metadata: + name: autonomousdatabase-sample + spec: + details: + autonomousDatabaseOCID: ocid1.autonomousdatabase... + networkAccess: + isMTLSConnectionRequired: false + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey + ``` + +2. Apply the yaml: + + ```sh + kubectl apply -f config/samples/adb/autonomousdatabase_update_mtls.yaml + autonomousdatabase.database.oracle.com/autonomousdatabase-sample configured + ``` + +### Autonomous Database with access control list enabled on dedicated Exadata infrastructure + +Follow the steps to configure the network with RESTRICTED access type. + +1. Add the following parameters to the spec. An example file is availble here: [`config/samples/adb/autonomousdatabase_update_network_access.yaml`](./../../config/samples/adb/autonomousdatabase_update_network_access.yaml): + + | Attribute | Type | Description | Required? | + |----|----|----|----| + | `networkAccess.isAccessControlEnabled` | boolean | Indicates if the database-level access control is enabled.

If disabled, database access is defined by the network security rules.

If enabled, database access is restricted to the IP addresses defined by the rules specified with the `accessControlList` property. While specifying `accessControlList` rules is optional, if database-level access control is enabled and no rules are specified, the database will become inaccessible. The rules can be added later using the `UpdateAutonomousDatabase` API operation or edit option in console.

When creating a database clone, the desired access control setting should be specified. By default, database-level access control will be disabled for the clone.
This property is applicable only to Autonomous Databases on the Exadata Cloud@Customer platform. | Yes | + | `networkAccess.accessControlList` | []string | The client IP access control list (ACL). This feature is available for autonomous databases on [shared Exadata infrastructure](https://docs.cloud.oracle.com/Content/Database/Concepts/adboverview.htm#AEI) and on Exadata Cloud@Customer.
Only clients connecting from an IP address included in the ACL may access the Autonomous Database instance.

For shared Exadata infrastructure, this is an array of CIDR (Classless Inter-Domain Routing) notations for a subnet or VCN OCID.
Use a semicolon (;) as a deliminator between the VCN-specific subnets or IPs.
Example: `["1.1.1.1","1.1.1.0/24","ocid1.vcn.oc1.sea.","ocid1.vcn.oc1.sea.;1.1.1.1","ocid1.vcn.oc1.sea.;1.1.0.0/16"]`

For Exadata Cloud@Customer, this is an array of IP addresses or CIDR (Classless Inter-Domain Routing) notations.
Example: `["1.1.1.1","1.1.1.0/24","1.1.2.25"]`

For an update operation, if you want to delete all the IPs in the ACL, use an array with a single empty string entry. | Yes | + + ```yaml + --- + apiVersion: database.oracle.com/v1alpha1 + kind: AutonomousDatabase + metadata: + name: autonomousdatabase-sample + spec: + details: + autonomousDatabaseOCID: ocid1.autonomousdatabase... + networkAccess: + isAccessControlEnabled: true + accessControlList: + - 1.1.1.1 + - 1.1.0.0/16 + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey + ``` + +2. Apply the yaml: + + ```sh + kubectl apply -f config/samples/adb/autonomousdatabase_update_network_access.yaml + autonomousdatabase.database.oracle.com/autonomousdatabase-sample configured diff --git a/docs/adb/README.md b/docs/adb/README.md index 6faaf2a0..55778566 100644 --- a/docs/adb/README.md +++ b/docs/adb/README.md @@ -88,7 +88,11 @@ Follow the steps to provision an Autonomous Database that will bind objects in y secretName: oci-privatekey ``` -4. Apply the yaml: +4. Choose the type of network access (optional): + + By default, the network access type is set to PUBLIC, which allows secure connections from anywhere. Uncomment the code block if you want configure the netowrk acess. See [Configuring Network Access of Autonomous Database](./NETWORK_ACCESS_OPTIONS.md) for more information. + +5. Apply the yaml: ```sh kubectl apply -f config/samples/adb/autonomousdatabase_create.yaml @@ -221,7 +225,7 @@ You can rename the database by changing the values of the `dbName` and `displayN \* The password must be between 12 and 30 characters long, and must contain at least 1 uppercase, 1 lowercase, and 1 numeric character. It cannot contain the double quote symbol (") or the username "admin", regardless of casing. -2. Update the example [config/samples/adb/autonomousdatabase_change_admin_password.yaml](./../../config/samples/adb/autonomousdatabase_change_admin_password.yaml) +2. Update the example [config/samples/adb/autonomousdatabase_update_admin_password.yaml](./../../config/samples/adb/autonomousdatabase_update_admin_password.yaml) ```yaml --- @@ -244,7 +248,7 @@ You can rename the database by changing the values of the `dbName` and `displayN 3. Apply the YAML. ```sh - kubectl apply -f config/samples/adb/autonomousdatabase_change_admin_password.yaml + kubectl apply -f config/samples/adb/autonomousdatabase_update_admin_password.yaml autonomousdatabase.database.oracle.com/autonomousdatabase-sample configured ``` diff --git a/main.go b/main.go index 9e68368f..25a16470 100644 --- a/main.go +++ b/main.go @@ -133,6 +133,10 @@ func main() { } } + if err = (&databasev1alpha1.AutonomousDatabase{}).SetupWebhookWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create webhook", "webhook", "AutonomousDatabase") + os.Exit(1) + } // +kubebuilder:scaffold:builder setupLog.Info("starting manager") diff --git a/test/e2e/behavior/shared_behaviors.go b/test/e2e/behavior/shared_behaviors.go index 5f0ef7d7..439466bb 100644 --- a/test/e2e/behavior/shared_behaviors.go +++ b/test/e2e/behavior/shared_behaviors.go @@ -298,7 +298,7 @@ func AssertADBDetails(k8sClient *client.Client, dbClient *database.DatabaseClien expectedADBDetails := expectedADB.Spec.Details - // Compare the elements one by one rather than doing reflect.DeelEqual(adb1, adb2), since some parameters + // Compare the elements one by one rather than doing reflect.DeelEqual(adb1, adb2), since some parameters // (e.g. adminPassword, wallet) are missing from e2eutil.GetAutonomousDatabase(). // We don't compare LifecycleState in this case. We only make sure that the ADB is in AVAIABLE state before // proceeding to the next test. From 6090fdd2ada316d6361c5e317cde507af24c3fdc Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Wed, 5 Jan 2022 16:00:50 -0500 Subject: [PATCH 142/628] resturcture network apis --- .../v1alpha1/autonomousdatabase_types.go | 41 ++++++--- .../v1alpha1/zz_generated.deepcopy.go | 92 +++++++++++++------ commons/oci/database.go | 37 ++++---- ...tabase.oracle.com_autonomousdatabases.yaml | 42 +++++---- test/e2e/behavior/shared_behaviors.go | 12 +-- 5 files changed, 143 insertions(+), 81 deletions(-) diff --git a/apis/database/v1alpha1/autonomousdatabase_types.go b/apis/database/v1alpha1/autonomousdatabase_types.go index b8759348..67bfa043 100644 --- a/apis/database/v1alpha1/autonomousdatabase_types.go +++ b/apis/database/v1alpha1/autonomousdatabase_types.go @@ -82,12 +82,7 @@ type AutonomousDatabaseDetails struct { IsAutoScalingEnabled *bool `json:"isAutoScalingEnabled,omitempty"` LifecycleState database.AutonomousDatabaseLifecycleStateEnum `json:"lifecycleState,omitempty"` - SubnetOCID *string `json:"subnetOCID,omitempty"` - NsgOCIDs []string `json:"nsgOCIDs,omitempty"` - IsAccessControlEnabled *bool `json:"isAccessControlEnabled,omitempty"` - WhitelistedIPs []string `json:"whitelistedIPs,omitempty"` - IsMTLSConnectionRequired *bool `json:"isMTLSConnectionRequired,omitempty"` - PrivateEndpointLabel *string `json:"privateEndpointLabel,omitempty"` + NetworkAccess NetworkAccessSpec `json:"networkAccess,omitempty"` FreeformTags map[string]string `json:"freeformTags,omitempty"` @@ -104,6 +99,28 @@ type PasswordSpec struct { OCISecretOCID *string `json:"ociSecretOCID,omitempty"` } +type NetworkAccessTypeEnum string + +const ( + NetworkAccessTypePublic NetworkAccessTypeEnum = "PUBLIC" + NetworkAccessTypeRestricted NetworkAccessTypeEnum = "RESTRICTED" + NetworkAccessTypePrivate NetworkAccessTypeEnum = "PRIVATE" +) + +type NetworkAccessSpec struct { + AccessType NetworkAccessTypeEnum `json:"accessType"` + IsAccessControlEnabled *bool `json:"isAccessControlEnabled,omitempty"` + AccessControlList []string `json:"accessControlList,omitempty"` + PrivateEndpoint PrivateEndpointSpec `json:"privateEndpoint,omitempty"` + IsMTLSConnectionRequired *bool `json:"isMTLSConnectionRequired,omitempty"` +} + +type PrivateEndpointSpec struct { + SubnetOCID *string `json:"subnetOCID,omitempty"` + NsgOCIDs []string `json:"nsgOCIDs,omitempty"` + HostnamePrefix *string `json:"hostnamePrefix,omitempty"` +} + // AutonomousDatabaseStatus defines the observed state of AutonomousDatabase type AutonomousDatabaseStatus struct { // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster @@ -187,12 +204,12 @@ func (adb *AutonomousDatabase) UpdateAttrFromOCIAutonomousDatabase(ociObj databa adb.Spec.Details.LifecycleState = ociObj.LifecycleState adb.Spec.Details.FreeformTags = ociObj.FreeformTags - adb.Spec.Details.SubnetOCID = ociObj.SubnetId - adb.Spec.Details.NsgOCIDs = ociObj.NsgIds - adb.Spec.Details.IsAccessControlEnabled = ociObj.IsAccessControlEnabled - adb.Spec.Details.WhitelistedIPs = ociObj.WhitelistedIps - adb.Spec.Details.IsMTLSConnectionRequired = ociObj.IsMtlsConnectionRequired - adb.Spec.Details.PrivateEndpointLabel = ociObj.PrivateEndpointLabel + adb.Spec.Details.NetworkAccess.IsAccessControlEnabled = ociObj.IsAccessControlEnabled + adb.Spec.Details.NetworkAccess.AccessControlList = ociObj.WhitelistedIps + adb.Spec.Details.NetworkAccess.IsMTLSConnectionRequired = ociObj.IsMtlsConnectionRequired + adb.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID = ociObj.SubnetId + adb.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs = ociObj.NsgIds + adb.Spec.Details.NetworkAccess.PrivateEndpoint.HostnamePrefix = ociObj.PrivateEndpointLabel // update the subresource as well adb.Status.DisplayName = *ociObj.DisplayName diff --git a/apis/database/v1alpha1/zz_generated.deepcopy.go b/apis/database/v1alpha1/zz_generated.deepcopy.go index f1aa816c..a5087592 100644 --- a/apis/database/v1alpha1/zz_generated.deepcopy.go +++ b/apis/database/v1alpha1/zz_generated.deepcopy.go @@ -215,36 +215,7 @@ func (in *AutonomousDatabaseDetails) DeepCopyInto(out *AutonomousDatabaseDetails *out = new(bool) **out = **in } - if in.SubnetOCID != nil { - in, out := &in.SubnetOCID, &out.SubnetOCID - *out = new(string) - **out = **in - } - if in.NsgOCIDs != nil { - in, out := &in.NsgOCIDs, &out.NsgOCIDs - *out = make([]string, len(*in)) - copy(*out, *in) - } - if in.IsAccessControlEnabled != nil { - in, out := &in.IsAccessControlEnabled, &out.IsAccessControlEnabled - *out = new(bool) - **out = **in - } - if in.WhitelistedIPs != nil { - in, out := &in.WhitelistedIPs, &out.WhitelistedIPs - *out = make([]string, len(*in)) - copy(*out, *in) - } - if in.IsMTLSConnectionRequired != nil { - in, out := &in.IsMTLSConnectionRequired, &out.IsMTLSConnectionRequired - *out = new(bool) - **out = **in - } - if in.PrivateEndpointLabel != nil { - in, out := &in.PrivateEndpointLabel, &out.PrivateEndpointLabel - *out = new(string) - **out = **in - } + in.NetworkAccess.DeepCopyInto(&out.NetworkAccess) if in.FreeformTags != nil { in, out := &in.FreeformTags, &out.FreeformTags *out = make(map[string]string, len(*in)) @@ -594,6 +565,37 @@ func (in *GsmStatusDetails) DeepCopy() *GsmStatusDetails { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *NetworkAccessSpec) DeepCopyInto(out *NetworkAccessSpec) { + *out = *in + if in.IsAccessControlEnabled != nil { + in, out := &in.IsAccessControlEnabled, &out.IsAccessControlEnabled + *out = new(bool) + **out = **in + } + if in.AccessControlList != nil { + in, out := &in.AccessControlList, &out.AccessControlList + *out = make([]string, len(*in)) + copy(*out, *in) + } + in.PrivateEndpoint.DeepCopyInto(&out.PrivateEndpoint) + if in.IsMTLSConnectionRequired != nil { + in, out := &in.IsMTLSConnectionRequired, &out.IsMTLSConnectionRequired + *out = new(bool) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new NetworkAccessSpec. +func (in *NetworkAccessSpec) DeepCopy() *NetworkAccessSpec { + if in == nil { + return nil + } + out := new(NetworkAccessSpec) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *OCIConfigSpec) DeepCopyInto(out *OCIConfigSpec) { *out = *in @@ -659,6 +661,36 @@ func (in *PortMapping) DeepCopy() *PortMapping { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PrivateEndpointSpec) DeepCopyInto(out *PrivateEndpointSpec) { + *out = *in + if in.SubnetOCID != nil { + in, out := &in.SubnetOCID, &out.SubnetOCID + *out = new(string) + **out = **in + } + if in.NsgOCIDs != nil { + in, out := &in.NsgOCIDs, &out.NsgOCIDs + *out = make([]string, len(*in)) + copy(*out, *in) + } + if in.HostnamePrefix != nil { + in, out := &in.HostnamePrefix, &out.HostnamePrefix + *out = new(string) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PrivateEndpointSpec. +func (in *PrivateEndpointSpec) DeepCopy() *PrivateEndpointSpec { + if in == nil { + return nil + } + out := new(PrivateEndpointSpec) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *ShardSpec) DeepCopyInto(out *ShardSpec) { *out = *in diff --git a/commons/oci/database.go b/commons/oci/database.go index 06ab3a3c..e43d6e66 100644 --- a/commons/oci/database.go +++ b/commons/oci/database.go @@ -78,11 +78,12 @@ func CreateAutonomousDatabase(logger logr.Logger, kubeClient client.Client, dbCl DbWorkload: database.CreateAutonomousDatabaseBaseDbWorkloadEnum( adb.Spec.Details.DbWorkload), - WhitelistedIps: adb.Spec.Details.WhitelistedIPs, - SubnetId: adb.Spec.Details.SubnetOCID, - NsgIds: adb.Spec.Details.NsgOCIDs, - PrivateEndpointLabel: adb.Spec.Details.PrivateEndpointLabel, - IsMtlsConnectionRequired: adb.Spec.Details.IsMTLSConnectionRequired, + IsAccessControlEnabled: adb.Spec.Details.NetworkAccess.IsAccessControlEnabled, + WhitelistedIps: adb.Spec.Details.NetworkAccess.AccessControlList, + IsMtlsConnectionRequired: adb.Spec.Details.NetworkAccess.IsMTLSConnectionRequired, + SubnetId: adb.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID, + NsgIds: adb.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs, + PrivateEndpointLabel: adb.Spec.Details.NetworkAccess.PrivateEndpoint.HostnamePrefix, FreeformTags: adb.Spec.Details.FreeformTags, } @@ -368,6 +369,8 @@ func UpdateScaleAttributes(logger logr.Logger, kubeClient client.Client, dbClien return } + + func UpdateOneWayTLSAttribute(logger logr.Logger, kubeClient client.Client, dbClient database.DatabaseClient, curADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { var shouldSendRequest = false @@ -380,8 +383,8 @@ func UpdateOneWayTLSAttribute(logger logr.Logger, kubeClient client.Client, dbCl // Prepare the update request updateAutonomousDatabaseDetails := database.UpdateAutonomousDatabaseDetails{} - if isAttrChanged(lastSucSpec.Details.IsMTLSConnectionRequired, curADB.Spec.Details.IsMTLSConnectionRequired) { - updateAutonomousDatabaseDetails.IsMtlsConnectionRequired = curADB.Spec.Details.IsMTLSConnectionRequired + if isAttrChanged(lastSucSpec.Details.NetworkAccess.IsMTLSConnectionRequired, curADB.Spec.Details.NetworkAccess.IsMTLSConnectionRequired) { + updateAutonomousDatabaseDetails.IsMtlsConnectionRequired = curADB.Spec.Details.NetworkAccess.IsMTLSConnectionRequired shouldSendRequest = true } @@ -414,25 +417,25 @@ func UpdateNetworkAttributes(logger logr.Logger, kubeClient client.Client, dbCli updateAutonomousDatabaseDetails := database.UpdateAutonomousDatabaseDetails{} // Network settings - if isAttrChanged(lastSucSpec.Details.SubnetOCID, curADB.Spec.Details.SubnetOCID) { - updateAutonomousDatabaseDetails.SubnetId = curADB.Spec.Details.SubnetOCID + if isAttrChanged(lastSucSpec.Details.NetworkAccess.IsAccessControlEnabled, curADB.Spec.Details.NetworkAccess.IsAccessControlEnabled) { + updateAutonomousDatabaseDetails.IsAccessControlEnabled = curADB.Spec.Details.NetworkAccess.IsAccessControlEnabled shouldSendRequest = true } - if isAttrChanged(lastSucSpec.Details.NsgOCIDs, curADB.Spec.Details.NsgOCIDs) { - updateAutonomousDatabaseDetails.NsgIds = curADB.Spec.Details.NsgOCIDs + if isAttrChanged(lastSucSpec.Details.NetworkAccess.AccessControlList, curADB.Spec.Details.NetworkAccess.AccessControlList) { + updateAutonomousDatabaseDetails.WhitelistedIps = curADB.Spec.Details.NetworkAccess.AccessControlList shouldSendRequest = true } - if isAttrChanged(lastSucSpec.Details.IsAccessControlEnabled, curADB.Spec.Details.IsAccessControlEnabled) { - updateAutonomousDatabaseDetails.IsAccessControlEnabled = curADB.Spec.Details.IsAccessControlEnabled + if isAttrChanged(lastSucSpec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID, curADB.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID) { + updateAutonomousDatabaseDetails.SubnetId = curADB.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID shouldSendRequest = true } - if isAttrChanged(lastSucSpec.Details.WhitelistedIPs, curADB.Spec.Details.WhitelistedIPs) { - updateAutonomousDatabaseDetails.WhitelistedIps = curADB.Spec.Details.WhitelistedIPs + if isAttrChanged(lastSucSpec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs, curADB.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs) { + updateAutonomousDatabaseDetails.NsgIds = curADB.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs shouldSendRequest = true } // PrivateEndpointLabel - if isAttrChanged(lastSucSpec.Details.PrivateEndpointLabel, curADB.Spec.Details.PrivateEndpointLabel) { - updateAutonomousDatabaseDetails.PrivateEndpointLabel = curADB.Spec.Details.PrivateEndpointLabel + if isAttrChanged(lastSucSpec.Details.NetworkAccess.PrivateEndpoint.HostnamePrefix, curADB.Spec.Details.NetworkAccess.PrivateEndpoint.HostnamePrefix) { + updateAutonomousDatabaseDetails.PrivateEndpointLabel = curADB.Spec.Details.NetworkAccess.PrivateEndpoint.HostnamePrefix shouldSendRequest = true } diff --git a/config/crd/bases/database.oracle.com_autonomousdatabases.yaml b/config/crd/bases/database.oracle.com_autonomousdatabases.yaml index dbce1bfa..0f428fe4 100644 --- a/config/crd/bases/database.oracle.com_autonomousdatabases.yaml +++ b/config/crd/bases/database.oracle.com_autonomousdatabases.yaml @@ -101,26 +101,40 @@ spec: additionalProperties: type: string type: object - isAccessControlEnabled: - type: boolean isAutoScalingEnabled: type: boolean isDedicated: type: boolean - isMTLSConnectionRequired: - type: boolean lifecycleState: description: 'AutonomousDatabaseLifecycleStateEnum Enum with underlying type: string' type: string - nsgOCIDs: - items: - type: string - type: array - privateEndpointLabel: - type: string - subnetOCID: - type: string + networkAccess: + properties: + accessControlList: + items: + type: string + type: array + accessType: + type: string + isAccessControlEnabled: + type: boolean + isMTLSConnectionRequired: + type: boolean + privateEndpoint: + properties: + hostnamePrefix: + type: string + nsgOCIDs: + items: + type: string + type: array + subnetOCID: + type: string + type: object + required: + - accessType + type: object wallet: properties: name: @@ -133,10 +147,6 @@ spec: type: string type: object type: object - whitelistedIPs: - items: - type: string - type: array type: object hardLink: default: false diff --git a/test/e2e/behavior/shared_behaviors.go b/test/e2e/behavior/shared_behaviors.go index ec4041b9..439466bb 100644 --- a/test/e2e/behavior/shared_behaviors.go +++ b/test/e2e/behavior/shared_behaviors.go @@ -313,12 +313,12 @@ func AssertADBDetails(k8sClient *client.Client, dbClient *database.DatabaseClien compareInt(expectedADBDetails.CPUCoreCount, resp.AutonomousDatabase.CpuCoreCount) && compareBool(expectedADBDetails.IsAutoScalingEnabled, resp.AutonomousDatabase.IsAutoScalingEnabled) && compareStringMap(expectedADBDetails.FreeformTags, resp.AutonomousDatabase.FreeformTags) && - compareString(expectedADBDetails.SubnetOCID, resp.AutonomousDatabase.SubnetId) && - reflect.DeepEqual(expectedADBDetails.NsgOCIDs, resp.AutonomousDatabase.NsgIds) && - compareBool(expectedADBDetails.IsAccessControlEnabled, resp.AutonomousDatabase.IsAccessControlEnabled) && - reflect.DeepEqual(expectedADBDetails.WhitelistedIPs, resp.AutonomousDatabase.WhitelistedIps) && - compareBool(expectedADBDetails.IsMTLSConnectionRequired, resp.AutonomousDatabase.IsMtlsConnectionRequired) && - compareString(expectedADBDetails.PrivateEndpointLabel, resp.AutonomousDatabase.PrivateEndpointLabel) + compareBool(expectedADBDetails.NetworkAccess.IsAccessControlEnabled, resp.AutonomousDatabase.IsAccessControlEnabled) && + reflect.DeepEqual(expectedADBDetails.NetworkAccess.AccessControlList, resp.AutonomousDatabase.WhitelistedIps) && + compareBool(expectedADBDetails.NetworkAccess.IsMTLSConnectionRequired, resp.AutonomousDatabase.IsMtlsConnectionRequired) && + compareString(expectedADBDetails.NetworkAccess.PrivateEndpoint.SubnetOCID, resp.AutonomousDatabase.SubnetId) && + reflect.DeepEqual(expectedADBDetails.NetworkAccess.PrivateEndpoint.NsgOCIDs, resp.AutonomousDatabase.NsgIds) && + compareString(expectedADBDetails.NetworkAccess.PrivateEndpoint.HostnamePrefix, resp.AutonomousDatabase.PrivateEndpointLabel) return same, nil }, updateTimeout, updateInterval).Should(BeTrue()) From f749712cb098c2deddccf6be2a3fa5f3ba478ee9 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Fri, 21 Jan 2022 16:44:09 -0500 Subject: [PATCH 143/628] restructure network apis and add webhooks --- PROJECT | 3 + .../v1alpha1/autonomousdatabase_types.go | 27 +- .../v1alpha1/autonomousdatabase_webhook.go | 206 ++++++++++++++ .../v1alpha1/zz_generated.deepcopy.go | 5 + .../autonomousdatabase/adb_reconciler_util.go | 256 +++++++++++++++++- commons/oci/database.go | 168 +++++++----- ...tabase.oracle.com_autonomousdatabases.yaml | 9 +- .../adb/autonomousdatabase_create.yaml | 37 ++- ...nomousdatabase_update_admin_password.yaml} | 0 .../adb/autonomousdatabase_update_mtls.yaml | 20 ++ ...onomousdatabase_update_network_access.yaml | 47 ++++ config/webhook/manifests.yaml | 41 +++ .../database/autonomousdatabase_controller.go | 186 ++----------- docs/adb/NETWORK_ACCESS_OPTIONS.md | 244 +++++++++++++++++ docs/adb/README.md | 10 +- main.go | 4 + 16 files changed, 1015 insertions(+), 248 deletions(-) create mode 100644 apis/database/v1alpha1/autonomousdatabase_webhook.go rename config/samples/adb/{autonomousdatabase_change_admin_password.yaml => autonomousdatabase_update_admin_password.yaml} (100%) create mode 100644 config/samples/adb/autonomousdatabase_update_mtls.yaml create mode 100644 config/samples/adb/autonomousdatabase_update_network_access.yaml create mode 100644 docs/adb/NETWORK_ACCESS_OPTIONS.md diff --git a/PROJECT b/PROJECT index 184a0a94..6cc088f1 100644 --- a/PROJECT +++ b/PROJECT @@ -17,6 +17,9 @@ resources: kind: AutonomousDatabase path: github.com/oracle/oracle-database-operator/apis/database/v1alpha1 version: v1alpha1 + webhooks: + validation: true + webhookVersion: v1 - api: crdVersion: v1 namespaced: true diff --git a/apis/database/v1alpha1/autonomousdatabase_types.go b/apis/database/v1alpha1/autonomousdatabase_types.go index 67bfa043..3f22e01d 100644 --- a/apis/database/v1alpha1/autonomousdatabase_types.go +++ b/apis/database/v1alpha1/autonomousdatabase_types.go @@ -68,11 +68,12 @@ type OCIConfigSpec struct { // AutonomousDatabaseDetails defines the detail information of AutonomousDatabase, corresponding to oci-go-sdk/database/AutonomousDatabase type AutonomousDatabaseDetails struct { - AutonomousDatabaseOCID *string `json:"autonomousDatabaseOCID,omitempty"` - CompartmentOCID *string `json:"compartmentOCID,omitempty"` - DisplayName *string `json:"displayName,omitempty"` - DbName *string `json:"dbName,omitempty"` - // +kubebuilder:validation:Enum:=OLTP;DW;AJD;APEX + AutonomousDatabaseOCID *string `json:"autonomousDatabaseOCID,omitempty"` + CompartmentOCID *string `json:"compartmentOCID,omitempty"` + AutonomousContainerDatabaseOCID *string `json:"autonomousContainerDatabaseOCID,omitempty"` + DisplayName *string `json:"displayName,omitempty"` + DbName *string `json:"dbName,omitempty"` + // +kubebuilder:validation:Enum:="OLTP";"DW";"AJD";"APEX" DbWorkload database.AutonomousDatabaseDbWorkloadEnum `json:"dbWorkload,omitempty"` IsDedicated *bool `json:"isDedicated,omitempty"` DbVersion *string `json:"dbVersion,omitempty"` @@ -108,7 +109,8 @@ const ( ) type NetworkAccessSpec struct { - AccessType NetworkAccessTypeEnum `json:"accessType"` + // +kubebuilder:validation:Enum:="";"PUBLIC";"RESTRICTED";"PRIVATE" + AccessType NetworkAccessTypeEnum `json:"accessType,omitempty"` IsAccessControlEnabled *bool `json:"isAccessControlEnabled,omitempty"` AccessControlList []string `json:"accessControlList,omitempty"` PrivateEndpoint PrivateEndpointSpec `json:"privateEndpoint,omitempty"` @@ -193,6 +195,7 @@ func (adb *AutonomousDatabase) UpdateLastSuccessfulSpec(kubeClient client.Client func (adb *AutonomousDatabase) UpdateAttrFromOCIAutonomousDatabase(ociObj database.AutonomousDatabase) *AutonomousDatabase { adb.Spec.Details.AutonomousDatabaseOCID = ociObj.Id adb.Spec.Details.CompartmentOCID = ociObj.CompartmentId + adb.Spec.Details.AutonomousContainerDatabaseOCID = ociObj.AutonomousContainerDatabaseId adb.Spec.Details.DisplayName = ociObj.DisplayName adb.Spec.Details.DbName = ociObj.DbName adb.Spec.Details.DbWorkload = ociObj.DbWorkload @@ -204,6 +207,18 @@ func (adb *AutonomousDatabase) UpdateAttrFromOCIAutonomousDatabase(ociObj databa adb.Spec.Details.LifecycleState = ociObj.LifecycleState adb.Spec.Details.FreeformTags = ociObj.FreeformTags + if *ociObj.IsDedicated { + adb.Spec.Details.NetworkAccess.AccessType = NetworkAccessTypePrivate + } else { + if ociObj.NsgIds != nil { + adb.Spec.Details.NetworkAccess.AccessType = NetworkAccessTypePrivate + } else if ociObj.WhitelistedIps != nil { + adb.Spec.Details.NetworkAccess.AccessType = NetworkAccessTypeRestricted + } else { + adb.Spec.Details.NetworkAccess.AccessType = NetworkAccessTypePublic + } + } + adb.Spec.Details.NetworkAccess.IsAccessControlEnabled = ociObj.IsAccessControlEnabled adb.Spec.Details.NetworkAccess.AccessControlList = ociObj.WhitelistedIps adb.Spec.Details.NetworkAccess.IsMTLSConnectionRequired = ociObj.IsMtlsConnectionRequired diff --git a/apis/database/v1alpha1/autonomousdatabase_webhook.go b/apis/database/v1alpha1/autonomousdatabase_webhook.go new file mode 100644 index 00000000..1c137eed --- /dev/null +++ b/apis/database/v1alpha1/autonomousdatabase_webhook.go @@ -0,0 +1,206 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v1alpha1 + +import ( + "fmt" + + apierrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/runtime/schema" + "k8s.io/apimachinery/pkg/util/validation/field" + ctrl "sigs.k8s.io/controller-runtime" + logf "sigs.k8s.io/controller-runtime/pkg/log" + "sigs.k8s.io/controller-runtime/pkg/webhook" +) + +// log is for logging in this package. +var autonomousdatabaselog = logf.Log.WithName("autonomousdatabase-resource") + +func (r *AutonomousDatabase) SetupWebhookWithManager(mgr ctrl.Manager) error { + return ctrl.NewWebhookManagedBy(mgr). + For(r). + Complete() +} + +// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! + +//+kubebuilder:webhook:verbs=create;update,path=/mutate-database-oracle-com-v1alpha1-autonomousdatabase,mutating=true,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=autonomousdatabase,versions=v1alpha1,name=mautonomousdatabase.kb.io,admissionReviewVersions={v1} + +var _ webhook.Defaulter = &AutonomousDatabase{} + +// Default implements webhook.Defaulter so a webhook will be registered for the type +func (r *AutonomousDatabase) Default() { + autonomousdatabaselog.Info("default", "name", r.Name) + + if !isDedicated(r) { // Shared database + // AccessType is PUBLIC by default + if r.Spec.Details.NetworkAccess.AccessType == "" { + r.Spec.Details.NetworkAccess.AccessType = NetworkAccessTypePublic + } + + // IsAccessControlEnabled is not applicable only to a shared database + if r.Spec.Details.NetworkAccess.IsAccessControlEnabled != nil { + r.Spec.Details.NetworkAccess.IsAccessControlEnabled = nil + } + } else { // Dedicated database + // AccessType can only be PRIVATE for a dedicated database + r.Spec.Details.NetworkAccess.AccessType = NetworkAccessTypePrivate + } + +} + +// TODO(user): change verbs to "verbs=create;update;delete" if you want to enable deletion validation. +//+kubebuilder:webhook:verbs=create;update;delete,path=/validate-database-oracle-com-v1alpha1-autonomousdatabase,mutating=false,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=autonomousdatabases,versions=v1alpha1,name=vautonomousdatabase.kb.io,admissionReviewVersions={v1} + +var _ webhook.Validator = &AutonomousDatabase{} + +// ValidateCreate implements webhook.Validator so a webhook will be registered for the type +// ValidateCreate checks if the spec is valid for a provisioning or a binding operation +func (r *AutonomousDatabase) ValidateCreate() error { + + var allErrs field.ErrorList + + autonomousdatabaselog.Info("validate create", "name", r.Name) + + if r.Spec.Details.AutonomousDatabaseOCID == nil { // provisioning operation + validateNetworkAcces(r, allErrs) + } else { // binding operation + } + + if len(allErrs) == 0 { + return nil + } + return apierrors.NewInvalid( + schema.GroupKind{Group: "database.oracle.com", Kind: "AutonomousDatabase"}, + r.Name, allErrs) +} + +// ValidateUpdate implements webhook.Validator so a webhook will be registered for the type +func (r *AutonomousDatabase) ValidateUpdate(old runtime.Object) error { + // Do ValidateCreate instead if there is no last successful spec, i.e. the database isn't provisioned or bound yet + lastSucSpec, err := r.GetLastSuccessfulSpec() + if err != nil { + return err + } + if lastSucSpec == nil { + return r.ValidateCreate() + } + + var allErrs field.ErrorList + + autonomousdatabaselog.Info("validate update", "name", r.Name) + + validateNetworkAcces(r, allErrs) + + if len(allErrs) == 0 { + return nil + } + return apierrors.NewInvalid( + schema.GroupKind{Group: "database.oracle.com", Kind: "AutonomousDatabase"}, + r.Name, allErrs) +} + +func validateNetworkAcces(adb *AutonomousDatabase, allErrs field.ErrorList) { + if !isDedicated(adb) { + // Shared database + if adb.Spec.Details.NetworkAccess.AccessType == NetworkAccessTypePublic { + if adb.Spec.Details.NetworkAccess.AccessControlList != nil || + adb.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID != nil || + adb.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs != nil { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("details").Child("networkAccess").Child("accessControlList"), + fmt.Sprintf("accessControlList, subnetOCID, nsgOCIDs cannot be provided when the network access type is %s", NetworkAccessTypePublic))) + } + + if adb.Spec.Details.NetworkAccess.IsMTLSConnectionRequired != nil && !*adb.Spec.Details.NetworkAccess.IsMTLSConnectionRequired { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("details").Child("networkAccess").Child("accessControlList"), + fmt.Sprintf("isMTLSConnectionRequired cannot be false when the network access type is %s", NetworkAccessTypePublic))) + } + } else if adb.Spec.Details.NetworkAccess.AccessType == NetworkAccessTypeRestricted { + if adb.Spec.Details.NetworkAccess.AccessControlList == nil { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("details").Child("networkAccess").Child("accessControlList"), + fmt.Sprintf("accessControlList cannot be empty when the network access type is %s", NetworkAccessTypeRestricted))) + } + } else { // the accessType is PRIVATE + if adb.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID == nil { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("details").Child("networkAccess").Child("privateEndpoint").Child("subnetOCID"), + fmt.Sprintf("subnetOCID cannot be empty when the network access type is %s", NetworkAccessTypePrivate))) + } + + if adb.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs == nil { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("details").Child("networkAccess").Child("privateEndpoint").Child("nsgOCIDs"), + fmt.Sprintf("nsgOCIDs cannot be empty when the network access type is %s", NetworkAccessTypePrivate))) + } + } + } else { + // Dedicated database + + // accessControlList cannot be provided when Autonomous Database's access control is disabled + if !*adb.Spec.Details.NetworkAccess.IsAccessControlEnabled && adb.Spec.Details.NetworkAccess.AccessControlList != nil { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("details").Child("networkAccess").Child("accessControlList"), + "access control list cannot be provided when Autonomous Database's access control is disabled")) + } + + // IsMTLSConnectionRequired is not supported by dedicated database + if adb.Spec.Details.NetworkAccess.IsMTLSConnectionRequired != nil { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("details").Child("networkAccess").Child("isMTLSConnectionRequired"), "isMTLSConnectionRequired is not supported on a dedicated database")) + } + } +} + +// ValidateDelete implements webhook.Validator so a webhook will be registered for the type +func (r *AutonomousDatabase) ValidateDelete() error { + autonomousdatabaselog.Info("validate delete", "name", r.Name) + + // TODO(user): fill in your validation logic upon object deletion. + return nil +} + +// Returns true if AutonomousContainerDatabaseOCID has value. +// We don't use Details.IsDedicated because the parameter might be null when it's a provision operation. +func isDedicated(adb *AutonomousDatabase) bool { + return adb.Spec.Details.AutonomousContainerDatabaseOCID != nil +} diff --git a/apis/database/v1alpha1/zz_generated.deepcopy.go b/apis/database/v1alpha1/zz_generated.deepcopy.go index a5087592..68045e84 100644 --- a/apis/database/v1alpha1/zz_generated.deepcopy.go +++ b/apis/database/v1alpha1/zz_generated.deepcopy.go @@ -179,6 +179,11 @@ func (in *AutonomousDatabaseDetails) DeepCopyInto(out *AutonomousDatabaseDetails *out = new(string) **out = **in } + if in.AutonomousContainerDatabaseOCID != nil { + in, out := &in.AutonomousContainerDatabaseOCID, &out.AutonomousContainerDatabaseOCID + *out = new(string) + **out = **in + } if in.DisplayName != nil { in, out := &in.DisplayName, &out.DisplayName *out = new(string) diff --git a/commons/autonomousdatabase/adb_reconciler_util.go b/commons/autonomousdatabase/adb_reconciler_util.go index 1bf817aa..5b3f5d99 100644 --- a/commons/autonomousdatabase/adb_reconciler_util.go +++ b/commons/autonomousdatabase/adb_reconciler_util.go @@ -49,19 +49,271 @@ import ( metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/types" "k8s.io/client-go/util/retry" + ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" "github.com/go-logr/logr" "github.com/oracle/oci-go-sdk/v51/common" "github.com/oracle/oci-go-sdk/v51/database" "github.com/oracle/oci-go-sdk/v51/secrets" + "github.com/oracle/oci-go-sdk/v51/workrequests" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" "github.com/oracle/oracle-database-operator/commons/oci" ) -// UpdateAutonomousDatabaseStatus sets the status subresource of AutonomousDatabase -func UpdateAutonomousDatabaseStatus(kubeClient client.Client, adb *dbv1alpha1.AutonomousDatabase) error { +func DetermineReturn(logger logr.Logger, err error) (ctrl.Result, error) { + if apiErrors.IsConflict(err) { + return ctrl.Result{}, err + } + + logger.Error(err, "Reconcile failed") + return ctrl.Result{}, nil +} + +func UpdateGeneralAndPasswordAttributesAndWait(logger logr.Logger, kubeClient client.Client, + dbClient database.DatabaseClient, + secretClient secrets.SecretsClient, + workClient workrequests.WorkRequestClient, + adb *dbv1alpha1.AutonomousDatabase) error { + + updateGenPassResp, err := oci.UpdateGeneralAndPasswordAttributes(logger, kubeClient, dbClient, secretClient, adb) + if err != nil { + logger.Error(err, "Fail to update Autonomous Database") + + // Change the status to UNAVAILABLE + adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable + if statusErr := SetStatus(kubeClient, adb); statusErr != nil { + return statusErr + } + // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue + return err + } + + // Wait for the work finish if a request is sent. Note that some of the requests (e.g. update displayName) won't return a work request ID. + if updateGenPassResp.OpcWorkRequestId != nil { + if err := UpdateStatusAndWait(logger, kubeClient, workClient, adb, updateGenPassResp.AutonomousDatabase.LifecycleState, updateGenPassResp.OpcWorkRequestId); err != nil { + logger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*updateGenPassResp.OpcWorkRequestId) + } + + logger.Info("Update AutonomousDatabase " + *adb.Spec.Details.DbName + " succesfully") + } + + return nil +} + +func UpdateScaleAttributesAndWait(logger logr.Logger, kubeClient client.Client, + dbClient database.DatabaseClient, + workClient workrequests.WorkRequestClient, + adb *dbv1alpha1.AutonomousDatabase) error { + + scaleResp, err := oci.UpdateScaleAttributes(logger, kubeClient, dbClient, adb) + if err != nil { + logger.Error(err, "Fail to update Autonomous Database") + + // Change the status to UNAVAILABLE + adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable + if statusErr := SetStatus(kubeClient, adb); statusErr != nil { + return statusErr + } + // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue + return err + } + + if scaleResp.OpcWorkRequestId != nil { + if err := UpdateStatusAndWait(logger, kubeClient, workClient, adb, scaleResp.AutonomousDatabase.LifecycleState, scaleResp.OpcWorkRequestId); err != nil { + logger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*scaleResp.OpcWorkRequestId) + } + + logger.Info("Scale AutonomousDatabase " + *adb.Spec.Details.DbName + " succesfully") + } + + return nil +} + +// The logic of updating the network access configurations is as follows: +// 1. Shared databases: +// If the network access type changes +// a. to PUBLIC: +// was RESTRICTED: re-enable IsMTLSConnectionRequired if it's not. Then set WhitelistedIps to an array with a single empty string entry. +// was PRIVATE: re-enable IsMTLSConnectionRequired if it's not. Then set PrivateEndpointLabel to an emtpy string. +// b. to RESTRICTED: +// was PUBLIC: set WhitelistedIps to desired IPs/CIDR blocks/VCN OCID. Configure the IsMTLSConnectionRequired settings if it is set to disabled. +// was PRIVATE: re-enable IsMTLSConnectionRequired if it's not. Set the type to PUBLIC first, and then configure the WhitelistedIps. Finally resume the IsMTLSConnectionRequired settings if it was, or is configured as disabled. +// c. to PRIVATE: +// was PUBLIC: set subnetOCID and nsgOCIDs. Configure the IsMTLSConnectionRequired settings if it is set. +// was RESTRICTED: set subnetOCID and nsgOCIDs. Configure the IsMTLSConnectionRequired settings if it is set. +// +// Otherwise, if the network access type remains the same, apply the network configuration, and then set the IsMTLSConnectionRequired. +// +// 2. Dedicated databases: +// Apply the configs directly +func UpdateNetworkAttributes(logger logr.Logger, + kubeClient client.Client, + dbClient database.DatabaseClient, + workClient workrequests.WorkRequestClient, + curADB *dbv1alpha1.AutonomousDatabase) error { + + lastSucSpec, err := curADB.GetLastSuccessfulSpec() + if err != nil { + return err + } + + if !*curADB.Spec.Details.IsDedicated { + var lastAccessType = lastSucSpec.Details.NetworkAccess.AccessType + var curAccessType = curADB.Spec.Details.NetworkAccess.AccessType + + if oci.IsAttrChanged(lastAccessType, curAccessType) { + switch curAccessType { + case dbv1alpha1.NetworkAccessTypePublic: + // OCI validation requires IsMTLSConnectionRequired to be enabled before changing the network access type to PUBLIC + if !*lastSucSpec.Details.NetworkAccess.IsMTLSConnectionRequired { + curADB.Spec.Details.NetworkAccess.IsMTLSConnectionRequired = common.Bool(true) + if err := updateMTLSAndWait(logger, kubeClient, dbClient, workClient, curADB); err != nil { + return err + } + } + + if err := setNetworkAccessPublicAndWait(logger, kubeClient, dbClient, workClient, curADB); err != nil { + return err + } + case dbv1alpha1.NetworkAccessTypeRestricted: + // If the access type was PRIVATE, then OCI validation requires IsMTLSConnectionRequired + // to be enabled before setting ACL. Also we can only change the network access type from + // PRIVATE to PUBLIC. + if lastAccessType == dbv1alpha1.NetworkAccessTypePrivate { + if !*lastSucSpec.Details.NetworkAccess.IsMTLSConnectionRequired { + var oldMTLS bool = *curADB.Spec.Details.NetworkAccess.IsMTLSConnectionRequired + curADB.Spec.Details.NetworkAccess.IsMTLSConnectionRequired = common.Bool(true) + if err := updateMTLSAndWait(logger, kubeClient, dbClient, workClient, curADB); err != nil { + return err + } + // restore IsMTLSConnectionRequired + curADB.Spec.Details.NetworkAccess.IsMTLSConnectionRequired = &oldMTLS + } + + if err := setNetworkAccessPublicAndWait(logger, kubeClient, dbClient, workClient, curADB); err != nil { + return err + } + } + + if err := updateNetworkAccessAttributesAndWait(logger, kubeClient, dbClient, workClient, curADB); err != nil { + return err + } + + if err := updateMTLSAndWait(logger, kubeClient, dbClient, workClient, curADB); err != nil { + return err + } + case dbv1alpha1.NetworkAccessTypePrivate: + if err := updateNetworkAccessAttributesAndWait(logger, kubeClient, dbClient, workClient, curADB); err != nil { + return err + } + + if err := updateMTLSAndWait(logger, kubeClient, dbClient, workClient, curADB); err != nil { + return err + } + } + } else { + // Access type doesn't change + if err := updateNetworkAccessAttributesAndWait(logger, kubeClient, dbClient, workClient, curADB); err != nil { + return err + } + + if err := updateMTLSAndWait(logger, kubeClient, dbClient, workClient, curADB); err != nil { + return err + } + } + } else { + // Dedicated database + if err := updateNetworkAccessAttributesAndWait(logger, kubeClient, dbClient, workClient, curADB); err != nil { + return err + } + } + + return nil +} + +func updateMTLSAndWait(logger logr.Logger, + kubeClient client.Client, + dbClient database.DatabaseClient, + workClient workrequests.WorkRequestClient, + curADB *dbv1alpha1.AutonomousDatabase) error { + + resp, err := oci.UpdateMTLSConnectionRequired(logger, dbClient, curADB) + if err != nil { + return err + } + + if resp.OpcWorkRequestId != nil { + if err := UpdateStatusAndWait(logger, kubeClient, workClient, curADB, resp.AutonomousDatabase.LifecycleState, resp.OpcWorkRequestId); err != nil { + logger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*resp.OpcWorkRequestId) + } + } + + return nil +} + +func setNetworkAccessPublicAndWait(logger logr.Logger, + kubeClient client.Client, + dbClient database.DatabaseClient, + workClient workrequests.WorkRequestClient, + curADB *dbv1alpha1.AutonomousDatabase) error { + + resp, err := oci.SetNetworkAccessPublic(logger, dbClient, curADB) + if err != nil { + return err + } + + if resp.OpcWorkRequestId != nil { + if err := UpdateStatusAndWait(logger, kubeClient, workClient, curADB, resp.AutonomousDatabase.LifecycleState, resp.OpcWorkRequestId); err != nil { + logger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*resp.OpcWorkRequestId) + } + } + + return nil +} + +func updateNetworkAccessAttributesAndWait(logger logr.Logger, + kubeClient client.Client, + dbClient database.DatabaseClient, + workClient workrequests.WorkRequestClient, + curADB *dbv1alpha1.AutonomousDatabase) error { + + resp, err := oci.UpdateNetworkAccessAttributes(logger, dbClient, curADB) + if err != nil { + return err + } + + if resp.OpcWorkRequestId != nil { + if err := UpdateStatusAndWait(logger, kubeClient, workClient, curADB, resp.AutonomousDatabase.LifecycleState, resp.OpcWorkRequestId); err != nil { + logger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*resp.OpcWorkRequestId) + } + } + + return nil +} + +func UpdateStatusAndWait(logger logr.Logger, kubeClient client.Client, + workClient workrequests.WorkRequestClient, + adb *dbv1alpha1.AutonomousDatabase, + desiredLifecycleState database.AutonomousDatabaseLifecycleStateEnum, + opcWorkRequestID *string) error { + + // Update status.state + adb.Status.LifecycleState = desiredLifecycleState + if statusErr := SetStatus(kubeClient, adb); statusErr != nil { + return statusErr + } + + if err := oci.WaitUntilWorkCompleted(logger, workClient, opcWorkRequestID); err != nil { + return err + } + + return nil +} + +// SetStatus sets the status subresource. +func SetStatus(kubeClient client.Client, adb *dbv1alpha1.AutonomousDatabase) error { return retry.RetryOnConflict(retry.DefaultRetry, func() error { curADB := &dbv1alpha1.AutonomousDatabase{} diff --git a/commons/oci/database.go b/commons/oci/database.go index e43d6e66..71fe5c50 100644 --- a/commons/oci/database.go +++ b/commons/oci/database.go @@ -66,15 +66,16 @@ func CreateAutonomousDatabase(logger logr.Logger, kubeClient client.Client, dbCl } createAutonomousDatabaseDetails := database.CreateAutonomousDatabaseDetails{ - CompartmentId: adb.Spec.Details.CompartmentOCID, - DbName: adb.Spec.Details.DbName, - CpuCoreCount: adb.Spec.Details.CPUCoreCount, - DataStorageSizeInTBs: adb.Spec.Details.DataStorageSizeInTBs, - AdminPassword: common.String(adminPassword), - DisplayName: adb.Spec.Details.DisplayName, - IsAutoScalingEnabled: adb.Spec.Details.IsAutoScalingEnabled, - IsDedicated: adb.Spec.Details.IsDedicated, - DbVersion: adb.Spec.Details.DbVersion, + CompartmentId: adb.Spec.Details.CompartmentOCID, + AutonomousContainerDatabaseId: adb.Spec.Details.AutonomousContainerDatabaseOCID, + DbName: adb.Spec.Details.DbName, + CpuCoreCount: adb.Spec.Details.CPUCoreCount, + DataStorageSizeInTBs: adb.Spec.Details.DataStorageSizeInTBs, + AdminPassword: common.String(adminPassword), + DisplayName: adb.Spec.Details.DisplayName, + IsAutoScalingEnabled: adb.Spec.Details.IsAutoScalingEnabled, + IsDedicated: adb.Spec.Details.IsDedicated, + DbVersion: adb.Spec.Details.DbVersion, DbWorkload: database.CreateAutonomousDatabaseBaseDbWorkloadEnum( adb.Spec.Details.DbWorkload), @@ -160,12 +161,23 @@ func GetAutonomousDatabaseResource(logger logr.Logger, dbClient database.Databas return returnedADB, nil } -// isAttrChanged checks if the values of last successful object and current object are different. +// IsAttrChanged checks if the values of last successful object and current object are different. // The function returns false if the types are mismatch or unknown. // The function returns false if the current object has zero value (not applicable for boolean type). -func isAttrChanged(lastSucObj interface{}, curObj interface{}) bool { +func IsAttrChanged(lastSucObj interface{}, curObj interface{}) bool { switch curObj.(type) { - case string: // Enum + case dbv1alpha1.NetworkAccessTypeEnum: + lastSucAccessType, ok := lastSucObj.(dbv1alpha1.NetworkAccessTypeEnum) + if !ok { + return false + } + curSucAccessType, ok := curObj.(dbv1alpha1.NetworkAccessTypeEnum) + if !ok { + return false + } + + return lastSucAccessType != curSucAccessType + case string: // type check lastSucString, ok := lastSucObj.(string) if !ok { @@ -272,30 +284,30 @@ func UpdateGeneralAndPasswordAttributes(logger logr.Logger, kubeClient client.Cl // Prepare the update request updateAutonomousDatabaseDetails := database.UpdateAutonomousDatabaseDetails{} - if isAttrChanged(lastSucSpec.Details.DisplayName, curADB.Spec.Details.DisplayName) { + if IsAttrChanged(lastSucSpec.Details.DisplayName, curADB.Spec.Details.DisplayName) { updateAutonomousDatabaseDetails.DisplayName = curADB.Spec.Details.DisplayName shouldSendRequest = true } - if isAttrChanged(lastSucSpec.Details.DbName, curADB.Spec.Details.DbName) { + if IsAttrChanged(lastSucSpec.Details.DbName, curADB.Spec.Details.DbName) { updateAutonomousDatabaseDetails.DbName = curADB.Spec.Details.DbName shouldSendRequest = true } - if isAttrChanged(lastSucSpec.Details.DbWorkload, curADB.Spec.Details.DbWorkload) { + if IsAttrChanged(lastSucSpec.Details.DbWorkload, curADB.Spec.Details.DbWorkload) { updateAutonomousDatabaseDetails.DbWorkload = database.UpdateAutonomousDatabaseDetailsDbWorkloadEnum(curADB.Spec.Details.DbWorkload) shouldSendRequest = true } - if isAttrChanged(lastSucSpec.Details.DbVersion, curADB.Spec.Details.DbVersion) { + if IsAttrChanged(lastSucSpec.Details.DbVersion, curADB.Spec.Details.DbVersion) { updateAutonomousDatabaseDetails.DbVersion = curADB.Spec.Details.DbVersion shouldSendRequest = true } - if isAttrChanged(lastSucSpec.Details.FreeformTags, curADB.Spec.Details.FreeformTags) { + if IsAttrChanged(lastSucSpec.Details.FreeformTags, curADB.Spec.Details.FreeformTags) { updateAutonomousDatabaseDetails.FreeformTags = curADB.Spec.Details.FreeformTags shouldSendRequest = true } - if isAttrChanged(lastSucSpec.Details.AdminPassword.K8sSecretName, curADB.Spec.Details.AdminPassword.K8sSecretName) || - isAttrChanged(lastSucSpec.Details.AdminPassword.OCISecretOCID, curADB.Spec.Details.AdminPassword.OCISecretOCID) { + if IsAttrChanged(lastSucSpec.Details.AdminPassword.K8sSecretName, curADB.Spec.Details.AdminPassword.K8sSecretName) || + IsAttrChanged(lastSucSpec.Details.AdminPassword.OCISecretOCID, curADB.Spec.Details.AdminPassword.OCISecretOCID) { // Get the adminPassword var adminPassword string @@ -314,7 +326,6 @@ func UpdateGeneralAndPasswordAttributes(logger logr.Logger, kubeClient client.Cl logger.Info("Sending general attributes and ADMIN password update request") updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ - // AutonomousDatabaseId: common.String(curADB.Spec.Details.AutonomousDatabaseOCID), AutonomousDatabaseId: curADB.Spec.Details.AutonomousDatabaseOCID, UpdateAutonomousDatabaseDetails: updateAutonomousDatabaseDetails, } @@ -339,15 +350,15 @@ func UpdateScaleAttributes(logger logr.Logger, kubeClient client.Client, dbClien // Prepare the update request updateAutonomousDatabaseDetails := database.UpdateAutonomousDatabaseDetails{} - if isAttrChanged(lastSucSpec.Details.DataStorageSizeInTBs, curADB.Spec.Details.DataStorageSizeInTBs) { + if IsAttrChanged(lastSucSpec.Details.DataStorageSizeInTBs, curADB.Spec.Details.DataStorageSizeInTBs) { updateAutonomousDatabaseDetails.DataStorageSizeInTBs = curADB.Spec.Details.DataStorageSizeInTBs shouldSendRequest = true } - if isAttrChanged(lastSucSpec.Details.CPUCoreCount, curADB.Spec.Details.CPUCoreCount) { + if IsAttrChanged(lastSucSpec.Details.CPUCoreCount, curADB.Spec.Details.CPUCoreCount) { updateAutonomousDatabaseDetails.CpuCoreCount = curADB.Spec.Details.CPUCoreCount shouldSendRequest = true } - if isAttrChanged(lastSucSpec.Details.IsAutoScalingEnabled, curADB.Spec.Details.IsAutoScalingEnabled) { + if IsAttrChanged(lastSucSpec.Details.IsAutoScalingEnabled, curADB.Spec.Details.IsAutoScalingEnabled) { updateAutonomousDatabaseDetails.IsAutoScalingEnabled = curADB.Spec.Details.IsAutoScalingEnabled shouldSendRequest = true } @@ -358,7 +369,6 @@ func UpdateScaleAttributes(logger logr.Logger, kubeClient client.Client, dbClien logger.Info("Sending scale attributes update request") updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ - // AutonomousDatabaseId: common.String(curADB.Spec.Details.AutonomousDatabaseOCID), AutonomousDatabaseId: curADB.Spec.Details.AutonomousDatabaseOCID, UpdateAutonomousDatabaseDetails: updateAutonomousDatabaseDetails, } @@ -369,80 +379,96 @@ func UpdateScaleAttributes(logger logr.Logger, kubeClient client.Client, dbClien return } - - -func UpdateOneWayTLSAttribute(logger logr.Logger, kubeClient client.Client, dbClient database.DatabaseClient, +func UpdateMTLSConnectionRequired(logger logr.Logger, + dbClient database.DatabaseClient, curADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { - var shouldSendRequest = false lastSucSpec, err := curADB.GetLastSuccessfulSpec() if err != nil { return resp, err } - // Prepare the update request - updateAutonomousDatabaseDetails := database.UpdateAutonomousDatabaseDetails{} - - if isAttrChanged(lastSucSpec.Details.NetworkAccess.IsMTLSConnectionRequired, curADB.Spec.Details.NetworkAccess.IsMTLSConnectionRequired) { - updateAutonomousDatabaseDetails.IsMtlsConnectionRequired = curADB.Spec.Details.NetworkAccess.IsMTLSConnectionRequired - shouldSendRequest = true - } - - // Don't send the request if nothing is changed - if shouldSendRequest { - - logger.Info("Sending 1-way TLS attribute update request") + if IsAttrChanged(lastSucSpec.Details.NetworkAccess.IsMTLSConnectionRequired, curADB.Spec.Details.NetworkAccess.IsMTLSConnectionRequired) { + logger.Info(fmt.Sprintf("Sending request to set mTLSRequired to %t", *curADB.Spec.Details.NetworkAccess.IsMTLSConnectionRequired)) updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ - AutonomousDatabaseId: curADB.Spec.Details.AutonomousDatabaseOCID, - UpdateAutonomousDatabaseDetails: updateAutonomousDatabaseDetails, + AutonomousDatabaseId: curADB.Spec.Details.AutonomousDatabaseOCID, + UpdateAutonomousDatabaseDetails: database.UpdateAutonomousDatabaseDetails{ + IsMtlsConnectionRequired: curADB.Spec.Details.NetworkAccess.IsMTLSConnectionRequired, + }, } - resp, err = dbClient.UpdateAutonomousDatabase(context.TODO(), updateAutonomousDatabaseRequest) } return } -func UpdateNetworkAttributes(logger logr.Logger, kubeClient client.Client, dbClient database.DatabaseClient, +func SetNetworkAccessPublic(logger logr.Logger, dbClient database.DatabaseClient, curADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { - var shouldSendRequest = false - lastSucSpec, err := curADB.GetLastSuccessfulSpec() if err != nil { return resp, err } - // Prepare the update request + logger.Info("Sending request to configure Network Access type to PUBLIC") + updateAutonomousDatabaseDetails := database.UpdateAutonomousDatabaseDetails{} - // Network settings - if isAttrChanged(lastSucSpec.Details.NetworkAccess.IsAccessControlEnabled, curADB.Spec.Details.NetworkAccess.IsAccessControlEnabled) { - updateAutonomousDatabaseDetails.IsAccessControlEnabled = curADB.Spec.Details.NetworkAccess.IsAccessControlEnabled - shouldSendRequest = true - } - if isAttrChanged(lastSucSpec.Details.NetworkAccess.AccessControlList, curADB.Spec.Details.NetworkAccess.AccessControlList) { - updateAutonomousDatabaseDetails.WhitelistedIps = curADB.Spec.Details.NetworkAccess.AccessControlList - shouldSendRequest = true + if lastSucSpec.Details.NetworkAccess.AccessType == dbv1alpha1.NetworkAccessTypeRestricted { + updateAutonomousDatabaseDetails.WhitelistedIps = []string{""} + } else if lastSucSpec.Details.NetworkAccess.AccessType == dbv1alpha1.NetworkAccessTypePrivate { + updateAutonomousDatabaseDetails.PrivateEndpointLabel = common.String("") } - if isAttrChanged(lastSucSpec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID, curADB.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID) { - updateAutonomousDatabaseDetails.SubnetId = curADB.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID - shouldSendRequest = true - } - if isAttrChanged(lastSucSpec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs, curADB.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs) { - updateAutonomousDatabaseDetails.NsgIds = curADB.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs - shouldSendRequest = true + + updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ + AutonomousDatabaseId: curADB.Spec.Details.AutonomousDatabaseOCID, + UpdateAutonomousDatabaseDetails: updateAutonomousDatabaseDetails, } - // PrivateEndpointLabel - if isAttrChanged(lastSucSpec.Details.NetworkAccess.PrivateEndpoint.HostnamePrefix, curADB.Spec.Details.NetworkAccess.PrivateEndpoint.HostnamePrefix) { - updateAutonomousDatabaseDetails.PrivateEndpointLabel = curADB.Spec.Details.NetworkAccess.PrivateEndpoint.HostnamePrefix - shouldSendRequest = true + + return dbClient.UpdateAutonomousDatabase(context.TODO(), updateAutonomousDatabaseRequest) +} + +// UpdateNetworkAccessAttributes determines if any of the network access attributes changes and send the update request +func UpdateNetworkAccessAttributes(logger logr.Logger, dbClient database.DatabaseClient, + curADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { + + lastSucSpec, err := curADB.GetLastSuccessfulSpec() + if err != nil { + return resp, err } - // Don't send the request if nothing is changed - if shouldSendRequest { + var lastNetworkAccess = lastSucSpec.Details.NetworkAccess + var curNetworkAccess = curADB.Spec.Details.NetworkAccess - logger.Info("Sending network attributes update request") + if IsAttrChanged(lastNetworkAccess.IsAccessControlEnabled, curNetworkAccess.IsAccessControlEnabled) || + IsAttrChanged(lastNetworkAccess.AccessControlList, curNetworkAccess.AccessControlList) || + IsAttrChanged(lastNetworkAccess.PrivateEndpoint.SubnetOCID, curNetworkAccess.PrivateEndpoint.SubnetOCID) || + IsAttrChanged(lastNetworkAccess.PrivateEndpoint.NsgOCIDs, curNetworkAccess.PrivateEndpoint.NsgOCIDs) || + IsAttrChanged(lastNetworkAccess.PrivateEndpoint.HostnamePrefix, curNetworkAccess.PrivateEndpoint.HostnamePrefix) { + + logger.Info("Sending request to configure Network Access") + + updateAutonomousDatabaseDetails := database.UpdateAutonomousDatabaseDetails{} + + if IsAttrChanged(lastNetworkAccess.IsAccessControlEnabled, curNetworkAccess.IsAccessControlEnabled) { + updateAutonomousDatabaseDetails.IsAccessControlEnabled = curNetworkAccess.IsAccessControlEnabled + } + + if IsAttrChanged(lastNetworkAccess.AccessControlList, curNetworkAccess.AccessControlList) { + updateAutonomousDatabaseDetails.WhitelistedIps = curNetworkAccess.AccessControlList + } + + if IsAttrChanged(lastNetworkAccess.PrivateEndpoint.SubnetOCID, curNetworkAccess.PrivateEndpoint.SubnetOCID) { + updateAutonomousDatabaseDetails.SubnetId = curNetworkAccess.PrivateEndpoint.SubnetOCID + } + + if IsAttrChanged(lastNetworkAccess.PrivateEndpoint.NsgOCIDs, curNetworkAccess.PrivateEndpoint.NsgOCIDs) { + updateAutonomousDatabaseDetails.NsgIds = curNetworkAccess.PrivateEndpoint.NsgOCIDs + } + + if IsAttrChanged(lastNetworkAccess.PrivateEndpoint.HostnamePrefix, curNetworkAccess.PrivateEndpoint.HostnamePrefix) { + updateAutonomousDatabaseDetails.PrivateEndpointLabel = curNetworkAccess.PrivateEndpoint.HostnamePrefix + } updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ AutonomousDatabaseId: curADB.Spec.Details.AutonomousDatabaseOCID, @@ -582,10 +608,6 @@ func GetAutonomousDatabaseBackup(dbClient database.DatabaseClient, backupOCID *s } func WaitUntilWorkCompleted(logger logr.Logger, workClient workrequests.WorkRequestClient, opcWorkRequestID *string) error { - if opcWorkRequestID == nil { - return nil - } - logger.Info("Waiting for the work request to finish. opcWorkRequestID = " + *opcWorkRequestID) retryPolicy := getCompleteWorkRetryPolicy() diff --git a/config/crd/bases/database.oracle.com_autonomousdatabases.yaml b/config/crd/bases/database.oracle.com_autonomousdatabases.yaml index 0f428fe4..2fab5d0e 100644 --- a/config/crd/bases/database.oracle.com_autonomousdatabases.yaml +++ b/config/crd/bases/database.oracle.com_autonomousdatabases.yaml @@ -74,6 +74,8 @@ spec: ociSecretOCID: type: string type: object + autonomousContainerDatabaseOCID: + type: string autonomousDatabaseOCID: type: string compartmentOCID: @@ -116,6 +118,11 @@ spec: type: string type: array accessType: + enum: + - "" + - PUBLIC + - RESTRICTED + - PRIVATE type: string isAccessControlEnabled: type: boolean @@ -132,8 +139,6 @@ spec: subnetOCID: type: string type: object - required: - - accessType type: object wallet: properties: diff --git a/config/samples/adb/autonomousdatabase_create.yaml b/config/samples/adb/autonomousdatabase_create.yaml index fb58b656..2a3ffe82 100644 --- a/config/samples/adb/autonomousdatabase_create.yaml +++ b/config/samples/adb/autonomousdatabase_create.yaml @@ -9,7 +9,7 @@ metadata: spec: details: # Update compartmentOCID with your compartment OCID. - compartmentOCID: ocid1.compartment... + compartmentOCID: ocid1.compartment... OR ocid1.tenancy... # The dbName must begin with an alphabetic character and can contain a maximum of 14 alphanumeric characters. Special characters are not permitted. The database name must be unique in the tenancy. dbName: NewADB displayName: NewADB @@ -20,6 +20,41 @@ spec: # The OCID of the OCI Secret that holds the password of the ADMIN account. It should start with ocid1.vaultsecret... . # ociSecretOCID: ocid1.vaultsecret... dataStorageSizeInTBs: 1 + + # networkAccess: + # # Uncomment this block to configure the network access type with the PUBLIC option, which allows secure access from everywhere. + # accessType: PUBLIC + + # # Uncomment this block to configure the network access type with the RESTRICTED option. + # # This option lets you restrict access by defining access control rules in an Access Control List (ACL). + # # By specifying an ACL, the database will be accessible from a whitelisted set of IP addresses, CIDR (Classless Inter-Domain Routing) blocks, or VCNs. + # # Use a semicolon (;) as a deliminator between the VCN-specific subnets or IPs. + # accessType: RESTRICTED + # accessControlList: + # - 1.1.1.1 + # - 1.1.0.0/16 + # - ocid1.vcn... + # - ocid1.vcn...;1.1.1.1 + # - ocid1.vcn...;1.1.0.0/16 + # isMTLSConnectionRequired: true + + # # Uncomment this block to configure the network access type with the PRIVATE option. + # # This option assigns a private endpoint, private IP, and hostname to your database. + # # Specifying this option allows traffic only from the VCN you specify. + # # This allows you to define security rules, ingress/egress, at the Network Security Group (NSG) level and to control traffic to your Autonomous Database. + # accessType: PRIVATE + # privateEndpoint: + # subnetOCID: ocid1.subnet... + # nsgOCIDs: + # - ocid1.networksecuritygroup... + # isMTLSConnectionRequired: true + + # # Uncomment this block to configure the network access of an dedicated Autonomous Database (ADB-D) with an access control list. + # isAccessControlEnabled: true + # accessControlList: + # - 1.1.1.1 + # - 1.1.0.0/16 + # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. ociConfig: configMapName: oci-cred diff --git a/config/samples/adb/autonomousdatabase_change_admin_password.yaml b/config/samples/adb/autonomousdatabase_update_admin_password.yaml similarity index 100% rename from config/samples/adb/autonomousdatabase_change_admin_password.yaml rename to config/samples/adb/autonomousdatabase_update_admin_password.yaml diff --git a/config/samples/adb/autonomousdatabase_update_mtls.yaml b/config/samples/adb/autonomousdatabase_update_mtls.yaml new file mode 100644 index 00000000..f067f4c7 --- /dev/null +++ b/config/samples/adb/autonomousdatabase_update_mtls.yaml @@ -0,0 +1,20 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: AutonomousDatabase +metadata: + name: autonomousdatabase-sample +spec: + details: + autonomousDatabaseOCID: ocid1.autonomousdatabase... + networkAccess: + # Set the patameter to false to allow both TLS and mutual TLS (mTLS) authentication. + # Avaiable when the networkAccessType is RESTRICTED or PRIVATE on shared Autnomous Database. + isMTLSConnectionRequired: false + + # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey \ No newline at end of file diff --git a/config/samples/adb/autonomousdatabase_update_network_access.yaml b/config/samples/adb/autonomousdatabase_update_network_access.yaml new file mode 100644 index 00000000..ef7145eb --- /dev/null +++ b/config/samples/adb/autonomousdatabase_update_network_access.yaml @@ -0,0 +1,47 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: AutonomousDatabase +metadata: + name: autonomousdatabase-sample +spec: + details: + autonomousDatabaseOCID: ocid1.autonomousdatabase... + networkAccess: + # Allow secure access from everywhere. + accessType: PUBLIC + + # # Uncomment this block to configure the network access type with the RESTRICTED option. + # # This option lets you restrict access by defining access control rules in an Access Control List (ACL). + # # By specifying an ACL, the database will be accessible from a whitelisted set of IP addresses, CIDR (Classless Inter-Domain Routing) blocks, or VCNs. + # # Use a semicolon (;) as a deliminator between the VCN-specific subnets or IPs. + # accessType: RESTRICTED + # accessControlList: + # - 1.1.1.1 + # - 1.1.0.0/16 + # - ocid1.vcn... + # - ocid1.vcn...;1.1.1.1 + # - ocid1.vcn...;1.1.0.0/16 + + # # Uncomment this block to configure the network access type with the PRIVATE option. + # # This option assigns a private endpoint, private IP, and hostname to your database. + # # Specifying this option allows traffic only from the VCN you specify. + # # This allows you to define security rules, ingress/egress, at the Network Security Group (NSG) level and to control traffic to your Autonomous Database. + # accessType: PRIVATE + # privateEndpoint: + # subnetOCID: ocid1.subnet... + # nsgOCIDs: + # - ocid1.networksecuritygroup... + + # # Uncomment this block to configure the network access of an dedicated Autonomous Database (ADB-D) with an access control list. + # isAccessControlEnabled: true + # accessControlList: + # - 1.1.1.1 + # - 1.1.0.0/16 + + # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey \ No newline at end of file diff --git a/config/webhook/manifests.yaml b/config/webhook/manifests.yaml index 5cb37b30..c635bcc2 100644 --- a/config/webhook/manifests.yaml +++ b/config/webhook/manifests.yaml @@ -6,6 +6,26 @@ metadata: creationTimestamp: null name: mutating-webhook-configuration webhooks: +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: webhook-service + namespace: system + path: /mutate-database-oracle-com-v1alpha1-autonomousdatabase + failurePolicy: Fail + name: mautonomousdatabase.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + resources: + - autonomousdatabase + sideEffects: None - admissionReviewVersions: - v1 - v1beta1 @@ -35,6 +55,27 @@ metadata: creationTimestamp: null name: validating-webhook-configuration webhooks: +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: webhook-service + namespace: system + path: /validate-database-oracle-com-v1alpha1-autonomousdatabase + failurePolicy: Fail + name: vautonomousdatabase.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + - DELETE + resources: + - autonomousdatabases + sideEffects: None - admissionReviewVersions: - v1 - v1beta1 diff --git a/controllers/database/autonomousdatabase_controller.go b/controllers/database/autonomousdatabase_controller.go index 7f364e5d..e5d1e158 100644 --- a/controllers/database/autonomousdatabase_controller.go +++ b/controllers/database/autonomousdatabase_controller.go @@ -146,7 +146,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } return ctrl.Result{}, nil @@ -158,7 +158,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } return ctrl.Result{}, nil @@ -170,7 +170,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } return ctrl.Result{}, nil @@ -182,7 +182,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } return ctrl.Result{}, nil @@ -220,7 +220,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } } @@ -260,7 +260,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue @@ -269,20 +269,9 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R adb.Spec.Details.AutonomousDatabaseOCID = resp.AutonomousDatabase.Id - // Update status.state - adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr - } - - if err := oci.WaitUntilWorkCompleted(r.currentLogger, workClient, resp.OpcWorkRequestId); err != nil { + if err := adbutil.UpdateStatusAndWait(r.currentLogger, r.KubeClient, workClient, adb, + resp.AutonomousDatabase.LifecycleState, resp.OpcWorkRequestId); err != nil { r.currentLogger.Error(err, "Fail to watch the status of provision request. opcWorkRequestID = "+*resp.OpcWorkRequestId) - - // Change the status to UNAVAILABLE - adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr - } } r.currentLogger.Info("AutonomousDatabase " + *adb.Spec.Details.DbName + " provisioned succesfully") @@ -297,7 +286,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } return ctrl.Result{}, nil @@ -320,7 +309,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } return ctrl.Result{}, nil @@ -348,158 +337,33 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } return ctrl.Result{}, nil } - // Update status.state - adb.Status.LifecycleState = lifecycleState - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr - } - - if err := oci.WaitUntilWorkCompleted(r.currentLogger, workClient, opcWorkRequestID); err != nil { + if err := adbutil.UpdateStatusAndWait(r.currentLogger, r.KubeClient, workClient, adb, lifecycleState, opcWorkRequestID); err != nil { r.currentLogger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*opcWorkRequestID) - - // Change the status to UNAVAILABLE - adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr - } } - r.currentLogger.Info(fmt.Sprintf("Set AutonomousDatabase %s lifecycle state to %s successfully\n", + + r.currentLogger.Info(fmt.Sprintf("Change AutonomousDatabase %s lifecycle state to %s successfully\n", *adb.Spec.Details.DbName, adb.Spec.Details.LifecycleState)) } // Update the database in OCI from the local resource. // The local resource will be synchronized again later. - updateGenPassResp, err := oci.UpdateGeneralAndPasswordAttributes(r.currentLogger, r.KubeClient, dbClient, secretClient, adb) - if err != nil { - r.currentLogger.Error(err, "Fail to update Autonomous Database") - - // Change the status to UNAVAILABLE - adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr - } - // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue - return ctrl.Result{}, nil - } - - if updateGenPassResp.OpcWorkRequestId != nil { - // Update status.state - adb.Status.LifecycleState = updateGenPassResp.AutonomousDatabase.LifecycleState - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr - } - - if err := oci.WaitUntilWorkCompleted(r.currentLogger, workClient, updateGenPassResp.OpcWorkRequestId); err != nil { - r.currentLogger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*updateGenPassResp.OpcWorkRequestId) - - // Change the status to UNAVAILABLE - adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr - } - } - r.currentLogger.Info("Update AutonomousDatabase " + *adb.Spec.Details.DbName + " succesfully") - } - - scaleResp, err := oci.UpdateScaleAttributes(r.currentLogger, r.KubeClient, dbClient, adb) - if err != nil { - r.currentLogger.Error(err, "Fail to update Autonomous Database") - - // Change the status to UNAVAILABLE - adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr - } - // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue - return ctrl.Result{}, nil + if err := adbutil.UpdateGeneralAndPasswordAttributesAndWait(r.currentLogger, r.KubeClient, dbClient, secretClient, workClient, adb); err != nil { + return adbutil.DetermineReturn(r.currentLogger, err) } - if scaleResp.OpcWorkRequestId != nil { - // Update status.state - adb.Status.LifecycleState = scaleResp.AutonomousDatabase.LifecycleState - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr - } - - if err := oci.WaitUntilWorkCompleted(r.currentLogger, workClient, scaleResp.OpcWorkRequestId); err != nil { - r.currentLogger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*scaleResp.OpcWorkRequestId) - - // Change the status to UNAVAILABLE - adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr - } - } - r.currentLogger.Info("Scale AutonomousDatabase " + *adb.Spec.Details.DbName + " succesfully") - } - - oneWayTLSResp, err := oci.UpdateOneWayTLSAttribute(r.currentLogger, r.KubeClient, dbClient, adb) - if err != nil { - r.currentLogger.Error(err, "Fail to update Autonomous Database") - - // Change the status to UNAVAILABLE - adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr - } - // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue + if err := adbutil.UpdateScaleAttributesAndWait(r.currentLogger, r.KubeClient, dbClient, workClient, adb); err != nil { + return adbutil.DetermineReturn(r.currentLogger, err) } - if oneWayTLSResp.OpcWorkRequestId != nil { - // Update status.state - adb.Status.LifecycleState = oneWayTLSResp.AutonomousDatabase.LifecycleState - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr - } - - if err := oci.WaitUntilWorkCompleted(r.currentLogger, workClient, oneWayTLSResp.OpcWorkRequestId); err != nil { - r.currentLogger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*oneWayTLSResp.OpcWorkRequestId) - - // Change the status to UNAVAILABLE - adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr - } - } - r.currentLogger.Info("Update AutonomousDatabase " + *adb.Spec.Details.DbName + " 1-way TLS setting succesfully") - } - - networkResp, err := oci.UpdateNetworkAttributes(r.currentLogger, r.KubeClient, dbClient, adb) - if err != nil { - r.currentLogger.Error(err, "Fail to update Autonomous Database") - - // Change the status to UNAVAILABLE - adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr - } - // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue - } - - if networkResp.OpcWorkRequestId != nil { - // Update status.state - adb.Status.LifecycleState = networkResp.AutonomousDatabase.LifecycleState - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr - } - - if err := oci.WaitUntilWorkCompleted(r.currentLogger, workClient, networkResp.OpcWorkRequestId); err != nil { - r.currentLogger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*networkResp.OpcWorkRequestId) - - // Change the status to UNAVAILABLE - adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr - } - } - r.currentLogger.Info("Update AutonomousDatabase " + *adb.Spec.Details.DbName + " network settings succesfully") + if err := adbutil.UpdateNetworkAttributes(r.currentLogger, r.KubeClient, dbClient, workClient, adb); err != nil { + return adbutil.DetermineReturn(r.currentLogger, err) } } } @@ -511,7 +375,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } return ctrl.Result{}, nil @@ -523,7 +387,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R if err := r.updateAutonomousDatabaseDetails(adb); err != nil { // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } return ctrl.Result{}, err @@ -542,7 +406,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R r.currentLogger.Error(err, "Fail to download Instance Wallet") // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } return ctrl.Result{}, nil @@ -557,7 +421,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { return ctrl.Result{}, statusErr } // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue @@ -618,7 +482,7 @@ func (r *AutonomousDatabaseReconciler) updateAutonomousDatabaseDetails(adb *dbv1 } // Update status - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { + if statusErr := adbutil.SetStatus(r.KubeClient, adb); statusErr != nil { return statusErr } r.currentLogger.Info("Update local resource AutonomousDatabase successfully") diff --git a/docs/adb/NETWORK_ACCESS_OPTIONS.md b/docs/adb/NETWORK_ACCESS_OPTIONS.md new file mode 100644 index 00000000..908fde32 --- /dev/null +++ b/docs/adb/NETWORK_ACCESS_OPTIONS.md @@ -0,0 +1,244 @@ +# Configuring Network Access of Autonomous Database + +This documentation describes how to configure network access with public access, access control rules (ACLs), or private endpoints. Also describes how to configure the TLS connections (require mutual TLS only or allow both 1 way TLS and mutual TLS). For more information, please visit [this page](https://docs.oracle.com/en/cloud/paas/autonomous-database/adbsa/autonomous-network-access.html#GUID-D2D468C3-CA2D-411E-92BC-E122F795A413). + +## Supported Features + +### Types of Network Access + +There are three types of network access supported by Autonomous Database: + +* **PUBLIC**: + + This option allows secure access from anywhere. The network access type is PUBLIC if no option is specified in the spec. With this option, mutual TLS (mTLS) authentication is always required to connect to the database. This option is available only for databases on shared Exadata infrastructure. + +* **RESTRICTED**: + + This option restricts connections to the database according to the access control lists (ACLs) you specify. This option is available only for databases on shared Exadata infrastructure. + + You can add the following to your ACL: + * **IP Address**: Specify one or more individual public IP address. Use commas to separate your addresses in the input field. + * **CIDR Block**: Specify one or more ranges of public IP addresses using CIDR notation. Use commas to separate your CIDR block entries in the input field. + * **Virtual Cloud Network (OCID)** (applies to Autonomous Databases on shared Exadata infrastructure): Specify the OCID of a virtual cloud network (VCN). If you want to specify multiple IP addresses or CIDR ranges within the same VCN, then do not create multiple access control list entries. Use one access control list entry with the values for the multiple IP addresses or CIDR ranges separated by commas. + +* **PRIVATE**: + + This option creates a private endpoint for your database within a specified VCN. This option is available for databases on shared Exadata infrastructure and is the only available option for databases on dedicated Exadata infrastructure. + + * **Autonomous Databases on shared Exadata infrastructure**: + + This option allows the access through private enpoints by specifying the OCIDs of a subnet and network security groups (NSGs) under the same VCN in the spec. + + * **Autonomous Databases on dedicated Exadata infrastructure**: + + The network path to a dedicated Autonomous Database is through a VCN and subnet defined by the dedicated infrastucture hosting the database. Usually, the subnet is defined as private, meaning that there is no public Internet access to databases. + + Autonomous Database supports restricted access using a ACL. You can optionally enabling an ACL by setting the `isAccessControlEnabled` parameter. If disabled, database access is defined by the network security rules. If enabled, database access is restricted to the IP addresses and CIDR blocks defined in the ACL. Note that enabling an ACL with an empty list of IP addresses makes the database inaccessible. See [Autonomous Database with Private Endpoint](https://docs.oracle.com/en-us/iaas/Content/Database/Concepts/adbsprivateaccess.htm) for overview and examples for private endpoint. + +### Allowing TLS or Require Only Mutual TLS (mTLS) Authentication + +If your Autonomous Database instance is configured to only allow mTLS connections, you can update the instance to allow both mTLS and TLS connections. When you update your configuration to allow both mTLS and TLS, you can use both authentication types at the same time and connections are no longer restricted to require mTLS authentication. + +This option only applies to Autonomous Databases on shared Exadata infrastructure, and you can allow TLS connections when network access type is configured as follows: + +* **RESTRICTED**: with ACLs defined. +* **PRIVATE**: with a private endpoint defined. + +## Sample YAML + +You can always configure the network access options when you create an Autonomous Database, or update the settings after the creation. Following are some sample YAMLs which configure the networking with different newtwork access options. + +For Autonomous Databases on shared Exadata infrastructure, you can: + +* Configure network access [with PUBLIC access type](#autonomous-database-with-public-access-type-on-shared-exadata-infrastructure) +* Configure network access [with RESTRICTED access type](#autonomous-database-with-restricted-access-type-on-shared-exadata-infrastructure) +* Configure network access [with PRIVATE access type](#autonomous-database-with-private-access-type-on-shared-exadata-infrastructure) +* [Change the mutual TLS (mTLS) authentication setting](#allow-both-tls-and-mutual-tls-mtls-authentication-of-autonomous-database-on-shared-exadata-infrastructure) + +For Autonomous Databases on dedicated Exadata infrastructure, you can: + +* Configure network access [with access control list enabled](#autonomous-database-with-access-control-list-enabled-on-dedicated-exadata-infrastructure) + +> Note: +> +> * The above operations require an `AutonomousDatabase` object to be in your cluster. This example assumes either the provision operation or the bind operation has been done by the users and the operator is authorized with API Key Authentication. +> * If you are creating an Autonomous Database, see step 4 of [Provision an Autonomous Database](./README.md#provision-an-autonomous-database) in [Managing Oracle Autonomous Databases with Oracle Database Operator for Kubernetes](./README.md) topic to return to provisioning instructions. + +### Autonomous Database with PUBLIC access type on shared Exadata infrastructure + +Follow the steps to configure the network with PUBLIC access type. + +1. Add the following parameters to the spec. An example file is availble here: [`config/samples/adb/autonomousdatabase_update_network_access.yaml`](./../../config/samples/adb/autonomousdatabase_update_network_access.yaml): + + | Attribute | Type | Description | Required? | + |----|----|----|----| + | `networkAccess.accessType` | string | An enum value which defines how the database can be accessed. The value can be PUBLIC, RESTRICTED or PRIVATE. See [Types of Network Access](#types-of-network-access) for more descriptions. | Yes | + + ```yaml + --- + apiVersion: database.oracle.com/v1alpha1 + kind: AutonomousDatabase + metadata: + name: autonomousdatabase-sample + spec: + details: + autonomousDatabaseOCID: ocid1.autonomousdatabase... + networkAccess: + # Allow secure access from everywhere. + accessType: PUBLIC + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey + ``` + +2. Apply the yaml: + + ```sh + kubectl apply -f config/samples/adb/autonomousdatabase_update_network_access.yaml + autonomousdatabase.database.oracle.com/autonomousdatabase-sample configured + ``` + +### Autonomous Database with RESTRICTED access type on shared Exadata infrastructure + +Follow the steps to configure the network with RESTRICTED access type. + +1. Add the following parameters to the spec. An example file is availble here: [`config/samples/adb/autonomousdatabase_update_network_access.yaml`](./../../config/samples/adb/autonomousdatabase_update_network_access.yaml): + + | Attribute | Type | Description | Required? | + |----|----|----|----| + | `networkAccess.accessType` | string | An enum value which defines how the database can be accessed. The value can be PUBLIC, RESTRICTED or PRIVATE. See [Types of Network Access](#types-of-network-access) for more descriptions. | Yes | + | `networkAccess.accessControlList` | []string | The client IP access control list (ACL). This feature is available for autonomous databases on [shared Exadata infrastructure](https://docs.cloud.oracle.com/Content/Database/Concepts/adboverview.htm#AEI) and on Exadata Cloud@Customer.
Only clients connecting from an IP address included in the ACL may access the Autonomous Database instance.

For shared Exadata infrastructure, this is an array of CIDR (Classless Inter-Domain Routing) notations for a subnet or VCN OCID.
Use a semicolon (;) as a deliminator between the VCN-specific subnets or IPs.
Example: `["1.1.1.1","1.1.1.0/24","ocid1.vcn.oc1.sea.","ocid1.vcn.oc1.sea.;1.1.1.1","ocid1.vcn.oc1.sea.;1.1.0.0/16"]`

For Exadata Cloud@Customer, this is an array of IP addresses or CIDR (Classless Inter-Domain Routing) notations.
Example: `["1.1.1.1","1.1.1.0/24","1.1.2.25"]`

For an update operation, if you want to delete all the IPs in the ACL, use an array with a single empty string entry. | Yes | + + ```yaml + --- + apiVersion: database.oracle.com/v1alpha1 + kind: AutonomousDatabase + metadata: + name: autonomousdatabase-sample + spec: + details: + autonomousDatabaseOCID: ocid1.autonomousdatabase... + networkAccess: + # Restrict access by defining access control rules in an Access Control List (ACL). + accessType: RESTRICTED + accessControlList: + - 1.1.1.1 + - 1.1.0.0/16 + - ocid1.vcn... + - ocid1.vcn...;1.1.1.1 + - ocid1.vcn...;1.1.0.0/16 + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey + ``` + +2. Apply the yaml: + + ```sh + kubectl apply -f config/samples/adb/autonomousdatabase_update_network_access.yaml + autonomousdatabase.database.oracle.com/autonomousdatabase-sample configured + +### Autonomous Database with PRIVATE access type on shared Exadata infrastructure + +Follow the steps to configure the network with RESTRICTED access type. + +1. Visit [Overview of VCNs and Subnets](https://docs.oracle.com/en-us/iaas/Content/Network/Tasks/managingVCNs_topic-Overview_of_VCNs_and_Subnets.htm#console) and [Network Security Groups](https://docs.oracle.com/en-us/iaas/Content/Network/Concepts/networksecuritygroups.htm#working) to see how to create VCNs, subnets, and network security groups (NSGs) if you haven't created them yet. The subnet and the NSG has to be in the same VCN. + +2. Copy and paste the OCIDs of the subnet and NSG to the corresponding parameters. An example file is availble here: [`config/samples/adb/autonomousdatabase_update_network_access.yaml`](./../../config/samples/adb/autonomousdatabase_update_network_access.yaml): + + | Attribute | Type | Description | Required? | + |----|----|----|----| + | `networkAccess.accessType` | string | An enum value which defines how the database can be accessed. The value can be PUBLIC, RESTRICTED or PRIVATE. See [Types of Network Access](#types-of-network-access) for more descriptions. | Yes | + | `networkAccess.privateEndpoint.subnetOCID` | string | The [OCID](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm) of the subnet the resource is associated with.

**Subnet Restrictions:**
- For bare metal DB systems and for single node virtual machine DB systems, do not use a subnet that overlaps with 192.168.16.16/28.
- For Exadata and virtual machine 2-node RAC systems, do not use a subnet that overlaps with 192.168.128.0/20.
- For Autonomous Database, setting this will disable public secure access to the database.
These subnets are used by the Oracle Clusterware private interconnect on the database instance.
Specifying an overlapping subnet will cause the private interconnect to malfunction.
This restriction applies to both the client subnet and the backup subnet. | Yes | + | `networkAccess.privateEndpoint.nsgOCIDs` | string[] | A list of the [OCIDs](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm) of the network security groups (NSGs) that this resource belongs to. Setting this to an empty array after the list is created removes the resource from all NSGs. For more information about NSGs, see [Security Rules](https://docs.cloud.oracle.com/Content/Network/Concepts/securityrules.htm).

**NsgOCIDs restrictions:**
- Autonomous Databases with private access require at least 1 Network Security Group (NSG). The nsgOCIDs array cannot be empty. | Yes | + | `networkAccess.privateEndpoint.hostnamePrefix` | string | The hostname prefix for the resource. | No | + + ```yaml + --- + apiVersion: database.oracle.com/v1alpha1 + kind: AutonomousDatabase + metadata: + name: autonomousdatabase-sample + spec: + details: + autonomousDatabaseOCID: ocid1.autonomousdatabase... + networkAccess: + # Assigns a private endpoint, private IP, and hostname to your database. + accessType: PRIVATE + privateEndpoint: + subnetOCID: ocid1.subnet... + nsgOCIDs: + - ocid1.networksecuritygroup... + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey + ``` + +### Allow both TLS and mutual TLS (mTLS) authentication of Autonomous Database on shared Exadata infrastructure + +If you are using the RESTRICTED or the PRIVATE network access option, you can choose whether to allow both TLS and mutual TLS (mTLS) authentication, or to allow only mTLS authentication. Follow the steps to change the mTLS authentication setting. + +1. Add the following parameters to the spec. An example file is availble here: [`config/samples/adb/autonomousdatabase_update_mtls.yaml`](./../../config/samples/adb/autonomousdatabase_update_mtls.yaml): + + | Attribute | Type | Description | Required? | + |----|----|----|----| + | `networkAccess.isMTLSConnectionRequired` | boolean| Indicates whether the Autonomous Database requires mTLS connections. | Yes | + + ```yaml + --- + apiVersion: database.oracle.com/v1alpha1 + kind: AutonomousDatabase + metadata: + name: autonomousdatabase-sample + spec: + details: + autonomousDatabaseOCID: ocid1.autonomousdatabase... + networkAccess: + isMTLSConnectionRequired: false + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey + ``` + +2. Apply the yaml: + + ```sh + kubectl apply -f config/samples/adb/autonomousdatabase_update_mtls.yaml + autonomousdatabase.database.oracle.com/autonomousdatabase-sample configured + ``` + +### Autonomous Database with access control list enabled on dedicated Exadata infrastructure + +Follow the steps to configure the network with RESTRICTED access type. + +1. Add the following parameters to the spec. An example file is availble here: [`config/samples/adb/autonomousdatabase_update_network_access.yaml`](./../../config/samples/adb/autonomousdatabase_update_network_access.yaml): + + | Attribute | Type | Description | Required? | + |----|----|----|----| + | `networkAccess.isAccessControlEnabled` | boolean | Indicates if the database-level access control is enabled.

If disabled, database access is defined by the network security rules.

If enabled, database access is restricted to the IP addresses defined by the rules specified with the `accessControlList` property. While specifying `accessControlList` rules is optional, if database-level access control is enabled and no rules are specified, the database will become inaccessible. The rules can be added later using the `UpdateAutonomousDatabase` API operation or edit option in console.

When creating a database clone, the desired access control setting should be specified. By default, database-level access control will be disabled for the clone.
This property is applicable only to Autonomous Databases on the Exadata Cloud@Customer platform. | Yes | + | `networkAccess.accessControlList` | []string | The client IP access control list (ACL). This feature is available for autonomous databases on [shared Exadata infrastructure](https://docs.cloud.oracle.com/Content/Database/Concepts/adboverview.htm#AEI) and on Exadata Cloud@Customer.
Only clients connecting from an IP address included in the ACL may access the Autonomous Database instance.

For shared Exadata infrastructure, this is an array of CIDR (Classless Inter-Domain Routing) notations for a subnet or VCN OCID.
Use a semicolon (;) as a deliminator between the VCN-specific subnets or IPs.
Example: `["1.1.1.1","1.1.1.0/24","ocid1.vcn.oc1.sea.","ocid1.vcn.oc1.sea.;1.1.1.1","ocid1.vcn.oc1.sea.;1.1.0.0/16"]`

For Exadata Cloud@Customer, this is an array of IP addresses or CIDR (Classless Inter-Domain Routing) notations.
Example: `["1.1.1.1","1.1.1.0/24","1.1.2.25"]`

For an update operation, if you want to delete all the IPs in the ACL, use an array with a single empty string entry. | Yes | + + ```yaml + --- + apiVersion: database.oracle.com/v1alpha1 + kind: AutonomousDatabase + metadata: + name: autonomousdatabase-sample + spec: + details: + autonomousDatabaseOCID: ocid1.autonomousdatabase... + networkAccess: + isAccessControlEnabled: true + accessControlList: + - 1.1.1.1 + - 1.1.0.0/16 + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey + ``` + +2. Apply the yaml: + + ```sh + kubectl apply -f config/samples/adb/autonomousdatabase_update_network_access.yaml + autonomousdatabase.database.oracle.com/autonomousdatabase-sample configured diff --git a/docs/adb/README.md b/docs/adb/README.md index 6faaf2a0..55778566 100644 --- a/docs/adb/README.md +++ b/docs/adb/README.md @@ -88,7 +88,11 @@ Follow the steps to provision an Autonomous Database that will bind objects in y secretName: oci-privatekey ``` -4. Apply the yaml: +4. Choose the type of network access (optional): + + By default, the network access type is set to PUBLIC, which allows secure connections from anywhere. Uncomment the code block if you want configure the netowrk acess. See [Configuring Network Access of Autonomous Database](./NETWORK_ACCESS_OPTIONS.md) for more information. + +5. Apply the yaml: ```sh kubectl apply -f config/samples/adb/autonomousdatabase_create.yaml @@ -221,7 +225,7 @@ You can rename the database by changing the values of the `dbName` and `displayN \* The password must be between 12 and 30 characters long, and must contain at least 1 uppercase, 1 lowercase, and 1 numeric character. It cannot contain the double quote symbol (") or the username "admin", regardless of casing. -2. Update the example [config/samples/adb/autonomousdatabase_change_admin_password.yaml](./../../config/samples/adb/autonomousdatabase_change_admin_password.yaml) +2. Update the example [config/samples/adb/autonomousdatabase_update_admin_password.yaml](./../../config/samples/adb/autonomousdatabase_update_admin_password.yaml) ```yaml --- @@ -244,7 +248,7 @@ You can rename the database by changing the values of the `dbName` and `displayN 3. Apply the YAML. ```sh - kubectl apply -f config/samples/adb/autonomousdatabase_change_admin_password.yaml + kubectl apply -f config/samples/adb/autonomousdatabase_update_admin_password.yaml autonomousdatabase.database.oracle.com/autonomousdatabase-sample configured ``` diff --git a/main.go b/main.go index 3049c485..09668a91 100644 --- a/main.go +++ b/main.go @@ -149,6 +149,10 @@ func main() { } } + if err = (&databasev1alpha1.AutonomousDatabase{}).SetupWebhookWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create webhook", "webhook", "AutonomousDatabase") + os.Exit(1) + } // +kubebuilder:scaffold:builder setupLog.Info("starting manager") From 8d6a2f52b18040de4e5eafbab63304a3675dae78 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Fri, 21 Jan 2022 18:06:51 -0500 Subject: [PATCH 144/628] remove spaces --- commons/autonomousdatabase/adb_restore_reconciler_util.go | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/commons/autonomousdatabase/adb_restore_reconciler_util.go b/commons/autonomousdatabase/adb_restore_reconciler_util.go index 2adbb92b..cf70f244 100644 --- a/commons/autonomousdatabase/adb_restore_reconciler_util.go +++ b/commons/autonomousdatabase/adb_restore_reconciler_util.go @@ -47,7 +47,7 @@ import ( dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" ) - + // UpdateAutonomousDatabaseBackupStatus updates the status subresource of AutonomousDatabaseBackup func UpdateAutonomousDatabaseRestoreStatus(kubeClient client.Client, adbRestore *dbv1alpha1.AutonomousDatabaseRestore) error { return retry.RetryOnConflict(retry.DefaultRetry, func() error { @@ -65,5 +65,4 @@ func UpdateAutonomousDatabaseRestoreStatus(kubeClient client.Client, adbRestore curBackup.Status = adbRestore.Status return kubeClient.Status().Update(context.TODO(), curBackup) }) -} - \ No newline at end of file +} \ No newline at end of file From 36788e0c11360d9aeb336cd47ac676f67a4560b2 Mon Sep 17 00:00:00 2001 From: Matteo Baccan Date: Tue, 25 Jan 2022 20:03:38 +0100 Subject: [PATCH 145/628] Fixed typo error Fixed unknow -> unknown --- set_ocicredentials.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/set_ocicredentials.sh b/set_ocicredentials.sh index fe2e5a5c..b21742f4 100755 --- a/set_ocicredentials.sh +++ b/set_ocicredentials.sh @@ -53,7 +53,7 @@ EOF exit 0 ;; *) # unknown command - echo "Unknow command. Use [set_ocicredentials -h] for help." + echo "Unknown command. Use [set_ocicredentials -h] for help." exit 1 shift # past argument ;; From b3bc4bdfed334307aa7dd4765927b18606ec9585 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Tue, 25 Jan 2022 17:27:14 -0500 Subject: [PATCH 146/628] add connection strings --- .../v1alpha1/autonomousdatabase_types.go | 57 ++++++++++++++++++- .../v1alpha1/zz_generated.deepcopy.go | 31 +++++++++- ...tabase.oracle.com_autonomousdatabases.yaml | 13 +++++ 3 files changed, 99 insertions(+), 2 deletions(-) diff --git a/apis/database/v1alpha1/autonomousdatabase_types.go b/apis/database/v1alpha1/autonomousdatabase_types.go index 3f22e01d..eb3eceb3 100644 --- a/apis/database/v1alpha1/autonomousdatabase_types.go +++ b/apis/database/v1alpha1/autonomousdatabase_types.go @@ -134,6 +134,19 @@ type AutonomousDatabaseStatus struct { DataStorageSizeInTBs int `json:"dataStorageSizeInTBs,omitempty"` DbWorkload database.AutonomousDatabaseDbWorkloadEnum `json:"dbWorkload,omitempty"` TimeCreated string `json:"timeCreated,omitempty"` + AllConnectionStrings []ConnectionStringsSet `json:"allConnectionStrings,omitempty"` +} + +type TLSAuthenticationEnum string + +const ( + TLSAuthenticationTLS TLSAuthenticationEnum = "TLS" + TLSAuthenticationmTLS TLSAuthenticationEnum = "Mutual TLS" +) + +type ConnectionStringsSet struct { + TLSAuthentication TLSAuthenticationEnum `json:"tlsAuthentication,omitempty"` + ConnectionStrings map[string]string `json:"connectionStrings"` } // AutonomousDatabase is the Schema for the autonomousdatabases API @@ -193,6 +206,9 @@ func (adb *AutonomousDatabase) UpdateLastSuccessfulSpec(kubeClient client.Client // UpdateAttrFromOCIAutonomousDatabase updates the attributes from database.AutonomousDatabase object and returns the resource func (adb *AutonomousDatabase) UpdateAttrFromOCIAutonomousDatabase(ociObj database.AutonomousDatabase) *AutonomousDatabase { + /*********************************** + * update the spec + ***********************************/ adb.Spec.Details.AutonomousDatabaseOCID = ociObj.Id adb.Spec.Details.CompartmentOCID = ociObj.CompartmentId adb.Spec.Details.AutonomousContainerDatabaseOCID = ociObj.AutonomousContainerDatabaseId @@ -226,7 +242,9 @@ func (adb *AutonomousDatabase) UpdateAttrFromOCIAutonomousDatabase(ociObj databa adb.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs = ociObj.NsgIds adb.Spec.Details.NetworkAccess.PrivateEndpoint.HostnamePrefix = ociObj.PrivateEndpointLabel - // update the subresource as well + /*********************************** + * update the status subresource + ***********************************/ adb.Status.DisplayName = *ociObj.DisplayName adb.Status.LifecycleState = ociObj.LifecycleState adb.Status.IsDedicated = strconv.FormatBool(*ociObj.IsDedicated) @@ -235,6 +253,43 @@ func (adb *AutonomousDatabase) UpdateAttrFromOCIAutonomousDatabase(ociObj databa adb.Status.DbWorkload = ociObj.DbWorkload adb.Status.TimeCreated = ociObj.TimeCreated.String() + var curAlllConns []ConnectionStringsSet + if *ociObj.IsDedicated { + connSet := ConnectionStringsSet{ConnectionStrings: ociObj.ConnectionStrings.AllConnectionStrings} + curAlllConns = append(curAlllConns, connSet) + + } else { + mTLSStrings := make(map[string]string) + tlsStrings := make(map[string]string) + + for _, profile := range ociObj.ConnectionStrings.Profiles { + if profile.TlsAuthentication == database.DatabaseConnectionStringProfileTlsAuthenticationMutual { + mTLSStrings[*profile.DisplayName] = *profile.Value + } else { + tlsStrings[*profile.DisplayName] = *profile.Value + } + } + + if len(mTLSStrings) > 0 { + mTLSConnSet := ConnectionStringsSet{ + TLSAuthentication: TLSAuthenticationmTLS, + ConnectionStrings: mTLSStrings, + } + + curAlllConns = append(curAlllConns, mTLSConnSet) + } + + if len(tlsStrings) > 0 { + tlsConnSet := ConnectionStringsSet{ + TLSAuthentication: TLSAuthenticationTLS, + ConnectionStrings: tlsStrings, + } + + curAlllConns = append(curAlllConns, tlsConnSet) + } + } + adb.Status.AllConnectionStrings = curAlllConns + return adb } diff --git a/apis/database/v1alpha1/zz_generated.deepcopy.go b/apis/database/v1alpha1/zz_generated.deepcopy.go index 68045e84..a0366f5d 100644 --- a/apis/database/v1alpha1/zz_generated.deepcopy.go +++ b/apis/database/v1alpha1/zz_generated.deepcopy.go @@ -55,7 +55,7 @@ func (in *AutonomousDatabase) DeepCopyInto(out *AutonomousDatabase) { out.TypeMeta = in.TypeMeta in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) in.Spec.DeepCopyInto(&out.Spec) - out.Status = in.Status + in.Status.DeepCopyInto(&out.Status) } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabase. @@ -404,6 +404,13 @@ func (in *AutonomousDatabaseSpec) DeepCopy() *AutonomousDatabaseSpec { // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *AutonomousDatabaseStatus) DeepCopyInto(out *AutonomousDatabaseStatus) { *out = *in + if in.AllConnectionStrings != nil { + in, out := &in.AllConnectionStrings, &out.AllConnectionStrings + *out = make([]ConnectionStringsSet, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseStatus. @@ -467,6 +474,28 @@ func (in *CatalogSpec) DeepCopy() *CatalogSpec { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *ConnectionStringsSet) DeepCopyInto(out *ConnectionStringsSet) { + *out = *in + if in.ConnectionStrings != nil { + in, out := &in.ConnectionStrings, &out.ConnectionStrings + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ConnectionStringsSet. +func (in *ConnectionStringsSet) DeepCopy() *ConnectionStringsSet { + if in == nil { + return nil + } + out := new(ConnectionStringsSet) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *EnvironmentVariable) DeepCopyInto(out *EnvironmentVariable) { *out = *in diff --git a/config/crd/bases/database.oracle.com_autonomousdatabases.yaml b/config/crd/bases/database.oracle.com_autonomousdatabases.yaml index 2fab5d0e..922b4897 100644 --- a/config/crd/bases/database.oracle.com_autonomousdatabases.yaml +++ b/config/crd/bases/database.oracle.com_autonomousdatabases.yaml @@ -169,6 +169,19 @@ spec: status: description: AutonomousDatabaseStatus defines the observed state of AutonomousDatabase properties: + allConnectionStrings: + items: + properties: + connectionStrings: + additionalProperties: + type: string + type: object + tlsAuthentication: + type: string + required: + - connectionStrings + type: object + type: array cpuCoreCount: type: integer dataStorageSizeInTBs: From 0dc238715a7db07b923dfc4aa14e91f5bac31314 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Wed, 26 Jan 2022 13:00:42 -0500 Subject: [PATCH 147/628] fix webhook not working --- apis/database/v1alpha1/autonomousdatabase_webhook.go | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/apis/database/v1alpha1/autonomousdatabase_webhook.go b/apis/database/v1alpha1/autonomousdatabase_webhook.go index 1c137eed..14779904 100644 --- a/apis/database/v1alpha1/autonomousdatabase_webhook.go +++ b/apis/database/v1alpha1/autonomousdatabase_webhook.go @@ -100,7 +100,7 @@ func (r *AutonomousDatabase) ValidateCreate() error { autonomousdatabaselog.Info("validate create", "name", r.Name) if r.Spec.Details.AutonomousDatabaseOCID == nil { // provisioning operation - validateNetworkAcces(r, allErrs) + allErrs = validateNetworkAcces(r, allErrs) } else { // binding operation } @@ -127,7 +127,7 @@ func (r *AutonomousDatabase) ValidateUpdate(old runtime.Object) error { autonomousdatabaselog.Info("validate update", "name", r.Name) - validateNetworkAcces(r, allErrs) + allErrs = validateNetworkAcces(r, allErrs) if len(allErrs) == 0 { return nil @@ -137,7 +137,7 @@ func (r *AutonomousDatabase) ValidateUpdate(old runtime.Object) error { r.Name, allErrs) } -func validateNetworkAcces(adb *AutonomousDatabase, allErrs field.ErrorList) { +func validateNetworkAcces(adb *AutonomousDatabase, allErrs field.ErrorList) field.ErrorList { if !isDedicated(adb) { // Shared database if adb.Spec.Details.NetworkAccess.AccessType == NetworkAccessTypePublic { @@ -145,13 +145,13 @@ func validateNetworkAcces(adb *AutonomousDatabase, allErrs field.ErrorList) { adb.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID != nil || adb.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs != nil { allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec").Child("details").Child("networkAccess").Child("accessControlList"), + field.Forbidden(field.NewPath("spec").Child("details").Child("networkAccess"), fmt.Sprintf("accessControlList, subnetOCID, nsgOCIDs cannot be provided when the network access type is %s", NetworkAccessTypePublic))) } if adb.Spec.Details.NetworkAccess.IsMTLSConnectionRequired != nil && !*adb.Spec.Details.NetworkAccess.IsMTLSConnectionRequired { allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec").Child("details").Child("networkAccess").Child("accessControlList"), + field.Forbidden(field.NewPath("spec").Child("details").Child("networkAccess").Child("isMTLSConnectionRequired"), fmt.Sprintf("isMTLSConnectionRequired cannot be false when the network access type is %s", NetworkAccessTypePublic))) } } else if adb.Spec.Details.NetworkAccess.AccessType == NetworkAccessTypeRestricted { @@ -189,6 +189,8 @@ func validateNetworkAcces(adb *AutonomousDatabase, allErrs field.ErrorList) { field.Forbidden(field.NewPath("spec").Child("details").Child("networkAccess").Child("isMTLSConnectionRequired"), "isMTLSConnectionRequired is not supported on a dedicated database")) } } + + return allErrs } // ValidateDelete implements webhook.Validator so a webhook will be registered for the type From e074d5abd9bcadd0f3c65e43701558136ad134a6 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Tue, 1 Feb 2022 15:22:06 -0500 Subject: [PATCH 148/628] bug 33740012, 33740073 --- .../autonomousdatabasebackup_types.go | 25 +-- .../autonomousdatabaserestore_types.go | 6 +- .../v1alpha1/zz_generated.deepcopy.go | 32 ++-- .../autonomousdatabase/adb_reconciler_util.go | 109 +++++++----- commons/oci/database.go | 27 ++- ....oracle.com_autonomousdatabasebackups.yaml | 5 + ...oracle.com_autonomousdatabaserestores.yaml | 14 +- config/rbac/role.yaml | 4 - .../adb/autonomousdatabase_restore.yaml | 2 +- .../database/autonomousdatabase_controller.go | 164 ++++++------------ .../autonomousdatabasebackup_controller.go | 84 ++++----- .../autonomousdatabaserestore_controller.go | 32 ++-- 12 files changed, 233 insertions(+), 271 deletions(-) diff --git a/apis/database/v1alpha1/autonomousdatabasebackup_types.go b/apis/database/v1alpha1/autonomousdatabasebackup_types.go index b909fa77..63246eb3 100644 --- a/apis/database/v1alpha1/autonomousdatabasebackup_types.go +++ b/apis/database/v1alpha1/autonomousdatabasebackup_types.go @@ -53,6 +53,7 @@ type AutonomousDatabaseBackupSpec struct { // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster // Important: Run "make" to regenerate code after modifying this file AutonomousDatabaseOCID string `json:"autonomousDatabaseOCID"` + DisplayName string `json:"displayName,omitempty"` AutonomousDatabaseBackupOCID string `json:"autonomousDatabaseBackupOCID,omitempty"` OCIConfig OCIConfigSpec `json:"ociConfig,omitempty"` @@ -65,6 +66,7 @@ type AutonomousDatabaseBackupStatus struct { AutonomousDatabaseBackupOCID string `json:"autonomousDatabaseBackupOCID"` CompartmentOCID string `json:"compartmentOCID"` AutonomousDatabaseOCID string `json:"autonomousDatabaseOCID"` + DisplayName string `json:"displayName"` Type database.AutonomousDatabaseBackupTypeEnum `json:"type"` IsAutomatic bool `json:"isAutomatic"` LifecycleState database.AutonomousDatabaseBackupLifecycleStateEnum `json:"lifecycleState"` @@ -89,19 +91,20 @@ type AutonomousDatabaseBackup struct { Status AutonomousDatabaseBackupStatus `json:"status,omitempty"` } -func (backup *AutonomousDatabaseBackup) UpdateStatusFromAutonomousDatabaseBackupResponse(resp database.GetAutonomousDatabaseBackupResponse) { - backup.Status.AutonomousDatabaseBackupOCID = *resp.Id - backup.Status.CompartmentOCID = *resp.CompartmentId - backup.Status.AutonomousDatabaseOCID = *resp.AutonomousDatabaseId - backup.Status.Type = resp.Type - backup.Status.IsAutomatic = *resp.IsAutomatic - backup.Status.LifecycleState = resp.LifecycleState +func (backup *AutonomousDatabaseBackup) UpdateStatusFromAutonomousDatabaseBackupResponse(ociBackup database.AutonomousDatabaseBackup) { + backup.Status.AutonomousDatabaseBackupOCID = *ociBackup.Id + backup.Status.CompartmentOCID = *ociBackup.CompartmentId + backup.Status.AutonomousDatabaseOCID = *ociBackup.AutonomousDatabaseId + backup.Status.DisplayName = *ociBackup.DisplayName + backup.Status.Type = ociBackup.Type + backup.Status.IsAutomatic = *ociBackup.IsAutomatic + backup.Status.LifecycleState = ociBackup.LifecycleState - if resp.TimeStarted != nil { - backup.Status.TimeStarted = ociutil.FormatSDKTime(resp.TimeStarted.Time) + if ociBackup.TimeStarted != nil { + backup.Status.TimeStarted = ociutil.FormatSDKTime(ociBackup.TimeStarted.Time) } - if resp.TimeEnded != nil { - backup.Status.TimeEnded = ociutil.FormatSDKTime(resp.TimeEnded.Time) + if ociBackup.TimeEnded != nil { + backup.Status.TimeEnded = ociutil.FormatSDKTime(ociBackup.TimeEnded.Time) } } diff --git a/apis/database/v1alpha1/autonomousdatabaserestore_types.go b/apis/database/v1alpha1/autonomousdatabaserestore_types.go index b461b8ab..6d37655e 100644 --- a/apis/database/v1alpha1/autonomousdatabaserestore_types.go +++ b/apis/database/v1alpha1/autonomousdatabaserestore_types.go @@ -49,13 +49,13 @@ import ( type AutonomousDatabaseRestoreSpec struct { // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster // Important: Run "make" to regenerate code after modifying this file - AutonomousDatabaseOCID string `json:"autonomousDatabaseOCID"` - Destination AutonomousDatabaseRestoreDestination `json:"destination"` + AutonomousDatabaseOCID string `json:"autonomousDatabaseOCID"` + Source AutonomousDatabaseRestoreSource `json:"source"` OCIConfig OCIConfigSpec `json:"ociConfig,omitempty"` } -type AutonomousDatabaseRestoreDestination struct { +type AutonomousDatabaseRestoreSource struct { BackupName string `json:"backupName,omitempty"` TimeStamp string `json:"timeStamp,omitempty"` } diff --git a/apis/database/v1alpha1/zz_generated.deepcopy.go b/apis/database/v1alpha1/zz_generated.deepcopy.go index f1aa816c..9fc088ed 100644 --- a/apis/database/v1alpha1/zz_generated.deepcopy.go +++ b/apis/database/v1alpha1/zz_generated.deepcopy.go @@ -324,21 +324,6 @@ func (in *AutonomousDatabaseRestore) DeepCopyObject() runtime.Object { return nil } -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *AutonomousDatabaseRestoreDestination) DeepCopyInto(out *AutonomousDatabaseRestoreDestination) { - *out = *in -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseRestoreDestination. -func (in *AutonomousDatabaseRestoreDestination) DeepCopy() *AutonomousDatabaseRestoreDestination { - if in == nil { - return nil - } - out := new(AutonomousDatabaseRestoreDestination) - in.DeepCopyInto(out) - return out -} - // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *AutonomousDatabaseRestoreList) DeepCopyInto(out *AutonomousDatabaseRestoreList) { *out = *in @@ -371,10 +356,25 @@ func (in *AutonomousDatabaseRestoreList) DeepCopyObject() runtime.Object { return nil } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousDatabaseRestoreSource) DeepCopyInto(out *AutonomousDatabaseRestoreSource) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseRestoreSource. +func (in *AutonomousDatabaseRestoreSource) DeepCopy() *AutonomousDatabaseRestoreSource { + if in == nil { + return nil + } + out := new(AutonomousDatabaseRestoreSource) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *AutonomousDatabaseRestoreSpec) DeepCopyInto(out *AutonomousDatabaseRestoreSpec) { *out = *in - out.Destination = in.Destination + out.Source = in.Source in.OCIConfig.DeepCopyInto(&out.OCIConfig) } diff --git a/commons/autonomousdatabase/adb_reconciler_util.go b/commons/autonomousdatabase/adb_reconciler_util.go index 1bf817aa..fb57984f 100644 --- a/commons/autonomousdatabase/adb_reconciler_util.go +++ b/commons/autonomousdatabase/adb_reconciler_util.go @@ -45,7 +45,6 @@ import ( "strings" corev1 "k8s.io/api/core/v1" - apiErrors "k8s.io/apimachinery/pkg/api/errors" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/types" "k8s.io/client-go/util/retry" @@ -60,6 +59,34 @@ import ( "github.com/oracle/oracle-database-operator/commons/oci" ) +func UpdateAutonomousDatabaseDetails(logger logr.Logger, kubeClient client.Client, adb *dbv1alpha1.AutonomousDatabase) error { + if err := retry.RetryOnConflict(retry.DefaultRetry, func() error { + curADB := &dbv1alpha1.AutonomousDatabase{} + + namespacedName := types.NamespacedName{ + Namespace: adb.GetNamespace(), + Name: adb.GetName(), + } + + if err := kubeClient.Get(context.TODO(), namespacedName, curADB); err != nil { + return err + } + + curADB.Spec.Details = adb.Spec.Details + return kubeClient.Update(context.TODO(), curADB) + }); err != nil { + return err + } + + // Update status + if statusErr := UpdateAutonomousDatabaseStatus(kubeClient, adb); statusErr != nil { + return statusErr + } + logger.Info("Update local resource AutonomousDatabase successfully") + + return nil +} + // UpdateAutonomousDatabaseStatus sets the status subresource of AutonomousDatabase func UpdateAutonomousDatabaseStatus(kubeClient client.Client, adb *dbv1alpha1.AutonomousDatabase) error { return retry.RetryOnConflict(retry.DefaultRetry, func() error { @@ -143,37 +170,23 @@ func getValidName(name string, usedNames map[string]bool) string { return returnedName } -// CreateBackupResources creates the all the AutonomousDatabasBackups that appears in the ListAutonomousDatabaseBackups request -// The backup object will not be created if it already exists. -func CreateBackupResources(logger logr.Logger, kubeClient client.Client, dbClient database.DatabaseClient, adb *dbv1alpha1.AutonomousDatabase) error { +// SyncBackupResources get the list of AutonomousDatabasBackups and +// create a backup object if it's not found in the same namespace +func SyncBackupResources(logger logr.Logger, kubeClient client.Client, dbClient database.DatabaseClient, adb *dbv1alpha1.AutonomousDatabase) error { // Get the list of AutonomousDatabaseBackupOCID in the same namespace backupList, err := fetchAutonomousDatabaseBackups(kubeClient, adb.Namespace) if err != nil { return err } - if err := kubeClient.List(context.TODO(), backupList, &client.ListOptions{Namespace: adb.Namespace}); err != nil { - // Ignore not-found errors, since they can't be fixed by an immediate requeue. - // No need to change the since we don't know if we obtain the object. - if !apiErrors.IsNotFound(err) { - return err - } - } - - usedNames := make(map[string]bool) - usedBackupOCIDs := make(map[string]bool) + curBackupNames := make(map[string]bool) + curBackupOCIDs := make(map[string]bool) for _, backup := range backupList.Items { - usedNames[backup.Name] = true + curBackupNames[backup.Name] = true - // Add both Spec.AutonomousDatabaseBackupOCID and Status.AutonomousDatabaseBackupOCID. - // If it's a backup created from the operator, it won't have the OCID under the spec. - // if the backup isn't ready yet, it won't have the OCID under the status. - if backup.Spec.AutonomousDatabaseBackupOCID != "" { - usedBackupOCIDs[backup.Spec.AutonomousDatabaseBackupOCID] = true - } if backup.Status.AutonomousDatabaseBackupOCID != "" { - usedBackupOCIDs[backup.Status.AutonomousDatabaseBackupOCID] = true + curBackupOCIDs[backup.Status.AutonomousDatabaseBackupOCID] = true } } @@ -184,7 +197,7 @@ func CreateBackupResources(logger logr.Logger, kubeClient client.Client, dbClien for _, backupSummary := range resp.Items { // Create the resource if the AutonomousDatabaseBackupOCID doesn't exist - _, ok := usedBackupOCIDs[*backupSummary.Id] + _, ok := curBackupOCIDs[*backupSummary.Id] if !ok { // Convert the string to lowercase, and replace spaces, commas, and colons with hyphens backupName := *backupSummary.DisplayName @@ -195,30 +208,15 @@ func CreateBackupResources(logger logr.Logger, kubeClient client.Client, dbClien return err } backupName = re.ReplaceAllString(backupName, "-") - backupName = getValidName(backupName, usedNames) - - backup := &dbv1alpha1.AutonomousDatabaseBackup{ - ObjectMeta: metav1.ObjectMeta{ - Namespace: adb.GetNamespace(), - Name: backupName, - OwnerReferences: newOwnerReference(adb), - }, - Spec: dbv1alpha1.AutonomousDatabaseBackupSpec{ - AutonomousDatabaseBackupOCID: *backupSummary.Id, - OCIConfig: dbv1alpha1.OCIConfigSpec{ - ConfigMapName: adb.Spec.OCIConfig.ConfigMapName, - SecretName: adb.Spec.OCIConfig.SecretName, - }, - }, - } + backupName = getValidName(backupName, curBackupNames) - if err := kubeClient.Create(context.TODO(), backup); err != nil { + if err := createBackupResource(kubeClient, backupName, backupSummary, adb); err != nil { return err } // Add the used names and ocids - usedNames[backupName] = true - usedBackupOCIDs[*backupSummary.AutonomousDatabaseId] = true + curBackupNames[backupName] = true + curBackupOCIDs[*backupSummary.AutonomousDatabaseId] = true logger.Info("Create AutonomousDatabaseBackup " + backupName) } @@ -226,3 +224,30 @@ func CreateBackupResources(logger logr.Logger, kubeClient client.Client, dbClien return nil } + +func createBackupResource(kubeClient client.Client, + backupName string, + backupSummary database.AutonomousDatabaseBackupSummary, + adb *dbv1alpha1.AutonomousDatabase) error { + + backup := &dbv1alpha1.AutonomousDatabaseBackup{ + ObjectMeta: metav1.ObjectMeta{ + Namespace: adb.GetNamespace(), + Name: backupName, + OwnerReferences: newOwnerReference(adb), + }, + Spec: dbv1alpha1.AutonomousDatabaseBackupSpec{ + AutonomousDatabaseBackupOCID: *backupSummary.Id, + OCIConfig: dbv1alpha1.OCIConfigSpec{ + ConfigMapName: adb.Spec.OCIConfig.ConfigMapName, + SecretName: adb.Spec.OCIConfig.SecretName, + }, + }, + } + + if err := kubeClient.Create(context.TODO(), backup); err != nil { + return err + } + + return nil +} diff --git a/commons/oci/database.go b/commons/oci/database.go index 06ab3a3c..f319ed8d 100644 --- a/commons/oci/database.go +++ b/commons/oci/database.go @@ -561,18 +561,25 @@ func CreateAutonomousDatabaseBackup(logger logr.Logger, dbClient database.Databa createBackupRequest := database.CreateAutonomousDatabaseBackupRequest{ CreateAutonomousDatabaseBackupDetails: database.CreateAutonomousDatabaseBackupDetails{ - DisplayName: common.String(adbBackup.GetName()), AutonomousDatabaseId: &adbBackup.Spec.AutonomousDatabaseOCID, }, } + // Use the spec.displayName as the displayName of the backup if is provided, + // otherwise use the resource name as the displayName. + if adbBackup.Spec.DisplayName != "" { + createBackupRequest.DisplayName = common.String(adbBackup.Spec.DisplayName) + } else { + createBackupRequest.DisplayName = common.String(adbBackup.GetName()) + } + return dbClient.CreateAutonomousDatabaseBackup(context.TODO(), createBackupRequest) } // GetAutonomousDatabaseBackup returns the response of GetAutonomousDatabaseBackupRequest -func GetAutonomousDatabaseBackup(dbClient database.DatabaseClient, backupOCID *string) (resp database.GetAutonomousDatabaseBackupResponse, err error) { +func GetAutonomousDatabaseBackup(dbClient database.DatabaseClient, backupOCID string) (resp database.GetAutonomousDatabaseBackupResponse, err error) { getBackupRequest := database.GetAutonomousDatabaseBackupRequest{ - AutonomousDatabaseBackupId: backupOCID, + AutonomousDatabaseBackupId: common.String(backupOCID), } return dbClient.GetAutonomousDatabaseBackup(context.TODO(), getBackupRequest) @@ -624,14 +631,18 @@ func getCompleteWorkRetryPolicy() common.RetryPolicy { } func getRetryPolicy(retryOperation func(common.OCIOperationResponse) bool) common.RetryPolicy { - // maximum times of retry - attempts := uint(10) + // maximum times of retry (~15mins) + attempts := uint(33) nextDuration := func(r common.OCIOperationResponse) time.Duration { - // you might want wait longer for next retry when your previous one failed + // Wait longer for next retry when your previous one failed // this function will return the duration as: - // 1s, 2s, 4s, 8s, 16s, 32s, 64s etc... - return time.Duration(math.Pow(float64(2), float64(r.AttemptNumber-1))) * time.Second + // 1s, 2s, 4s, 8s, 16s, 30s, 30s etc... + if r.AttemptNumber <= 5 { + return time.Duration(math.Pow(float64(2), float64(r.AttemptNumber-1))) * time.Second + } else { + return time.Duration(30) * time.Second + } } return common.NewRetryPolicy(attempts, retryOperation, nextDuration) diff --git a/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml b/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml index 68d26754..2eae5408 100644 --- a/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml +++ b/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml @@ -60,6 +60,8 @@ spec: description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster Important: Run "make" to regenerate code after modifying this file' type: string + displayName: + type: string ociConfig: properties: configMapName: @@ -83,6 +85,8 @@ spec: type: string compartmentOCID: type: string + displayName: + type: string isAutomatic: type: boolean lifecycleState: @@ -101,6 +105,7 @@ spec: - autonomousDatabaseBackupOCID - autonomousDatabaseOCID - compartmentOCID + - displayName - isAutomatic - lifecycleState - type diff --git a/config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml b/config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml index 9534f3b6..e852e906 100644 --- a/config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml +++ b/config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml @@ -45,23 +45,23 @@ spec: description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster Important: Run "make" to regenerate code after modifying this file' type: string - destination: + ociConfig: properties: - backupName: + configMapName: type: string - timeStamp: + secretName: type: string type: object - ociConfig: + source: properties: - configMapName: + backupName: type: string - secretName: + timeStamp: type: string type: object required: - autonomousDatabaseOCID - - destination + - source type: object status: description: AutonomousDatabaseRestoreStatus defines the observed state diff --git a/config/rbac/role.yaml b/config/rbac/role.yaml index e1011e9d..91a27ce9 100644 --- a/config/rbac/role.yaml +++ b/config/rbac/role.yaml @@ -116,8 +116,6 @@ rules: - delete - get - list - - patch - - update - watch - apiGroups: - database.oracle.com @@ -136,8 +134,6 @@ rules: - delete - get - list - - patch - - update - watch - apiGroups: - database.oracle.com diff --git a/config/samples/adb/autonomousdatabase_restore.yaml b/config/samples/adb/autonomousdatabase_restore.yaml index 2f9dc623..1485bec7 100644 --- a/config/samples/adb/autonomousdatabase_restore.yaml +++ b/config/samples/adb/autonomousdatabase_restore.yaml @@ -5,7 +5,7 @@ metadata: spec: autonomousDatabaseOCID: ocid1.autonomousdatabase... # Restore the database either from a backup or using point-in-time restore - destination: + source: # The name of your AutonomousDatabaseBackup backupName: autonomousdatabasebackup-sample # The timestamp must follow this format: YYYY-MM-DD HH:MM:SS GMT diff --git a/controllers/database/autonomousdatabase_controller.go b/controllers/database/autonomousdatabase_controller.go index 7f364e5d..dc67a3d2 100644 --- a/controllers/database/autonomousdatabase_controller.go +++ b/controllers/database/autonomousdatabase_controller.go @@ -50,8 +50,6 @@ import ( apiErrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/runtime" - "k8s.io/apimachinery/pkg/types" - "k8s.io/client-go/util/retry" ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/controller-runtime/pkg/controller" @@ -69,8 +67,6 @@ type AutonomousDatabaseReconciler struct { KubeClient client.Client Log logr.Logger Scheme *runtime.Scheme - - currentLogger logr.Logger } // SetupWithManager function @@ -120,7 +116,7 @@ func (r *AutonomousDatabaseReconciler) eventFilterPredicate() predicate.Predicat // It go to the beggining of the reconcile if an error is returned. We won't return a error if it is related // to OCI, because the issues cannot be solved by re-run the reconcile. func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { - r.currentLogger = r.Log.WithValues("Namespaced/Name", req.NamespacedName) + currentLogger := r.Log.WithValues("Namespaced/Name", req.NamespacedName) // Get the autonomousdatabase instance from the cluster adb := &dbv1alpha1.AutonomousDatabase{} @@ -142,7 +138,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R } provider, err := oci.GetOCIProvider(r.KubeClient, authData) if err != nil { - r.currentLogger.Error(err, "Fail to get OCI provider") + currentLogger.Error(err, "Fail to get OCI provider") // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable @@ -154,7 +150,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R dbClient, err := database.NewDatabaseClientWithConfigurationProvider(provider) if err != nil { - r.currentLogger.Error(err, "Fail to get OCI database client") + currentLogger.Error(err, "Fail to get OCI database client") // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable @@ -166,7 +162,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R secretClient, err := secrets.NewSecretsClientWithConfigurationProvider(provider) if err != nil { - r.currentLogger.Error(err, "Fail to get OCI secret client") + currentLogger.Error(err, "Fail to get OCI secret client") // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable @@ -178,7 +174,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R workClient, err := workrequests.NewWorkRequestClientWithConfigurationProvider(provider) if err != nil { - r.currentLogger.Error(err, "Fail to get OCI work request client") + currentLogger.Error(err, "Fail to get OCI work request client") // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable @@ -188,7 +184,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R return ctrl.Result{}, nil } - r.currentLogger.Info("OCI provider configured succesfully") + currentLogger.Info("OCI provider configured succesfully") /****************************************************************** * Register/unregister finalizer @@ -201,22 +197,22 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // The object is not being deleted if *adb.Spec.HardLink && !finalizer.HasFinalizer(adb) { finalizer.Register(r.KubeClient, adb) - r.currentLogger.Info("Finalizer registered successfully.") + currentLogger.Info("Finalizer registered successfully.") } else if !*adb.Spec.HardLink && finalizer.HasFinalizer(adb) { finalizer.Unregister(r.KubeClient, adb) - r.currentLogger.Info("Finalizer unregistered successfully.") + currentLogger.Info("Finalizer unregistered successfully.") } } else { // The object is being deleted if adb.Spec.Details.AutonomousDatabaseOCID == nil { - r.currentLogger.Info("Autonomous Database OCID is missing. Remove the resource only.") + currentLogger.Info("Autonomous Database OCID is missing. Remove the resource only.") } else if adb.Status.LifecycleState != database.AutonomousDatabaseLifecycleStateTerminating && adb.Status.LifecycleState != database.AutonomousDatabaseLifecycleStateTerminated { // Don't send terminate request if the database is terminating or already terminated - r.currentLogger.Info("Terminate Autonomous Database: " + *adb.Spec.Details.DbName) + currentLogger.Info("Terminate Autonomous Database: " + *adb.Spec.Details.DbName) if _, err := oci.DeleteAutonomousDatabase(dbClient, *adb.Spec.Details.AutonomousDatabaseOCID); err != nil { - r.currentLogger.Error(err, "Fail to terminate Autonomous Database") + currentLogger.Error(err, "Fail to terminate Autonomous Database") // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable @@ -227,7 +223,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R } finalizer.Unregister(r.KubeClient, adb) - r.currentLogger.Info("Finalizer unregistered successfully.") + currentLogger.Info("Finalizer unregistered successfully.") // Stop reconciliation as the item is being deleted return ctrl.Result{}, nil } @@ -251,12 +247,12 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R if adb.Spec.Details.AutonomousDatabaseOCID == nil && lastSucSpec == nil { // If no AutonomousDatabaseOCID specified, create a database // Update from yaml file might not have an AutonomousDatabaseOCID. Don't create a database if it already has last successful spec. - r.currentLogger.Info("AutonomousDatabase provisioning") + currentLogger.Info("AutonomousDatabase provisioning") - resp, err := oci.CreateAutonomousDatabase(r.currentLogger, r.KubeClient, dbClient, secretClient, adb) + resp, err := oci.CreateAutonomousDatabase(currentLogger, r.KubeClient, dbClient, secretClient, adb) if err != nil { - r.currentLogger.Error(err, "Fail to provision and get Autonomous Database OCID") + currentLogger.Error(err, "Fail to provision and get Autonomous Database OCID") // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable @@ -275,8 +271,8 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R return ctrl.Result{}, statusErr } - if err := oci.WaitUntilWorkCompleted(r.currentLogger, workClient, resp.OpcWorkRequestId); err != nil { - r.currentLogger.Error(err, "Fail to watch the status of provision request. opcWorkRequestID = "+*resp.OpcWorkRequestId) + if err := oci.WaitUntilWorkCompleted(currentLogger, workClient, resp.OpcWorkRequestId); err != nil { + currentLogger.Error(err, "Fail to watch the status of provision request. opcWorkRequestID = "+*resp.OpcWorkRequestId) // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable @@ -285,15 +281,15 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R } } - r.currentLogger.Info("AutonomousDatabase " + *adb.Spec.Details.DbName + " provisioned succesfully") + currentLogger.Info("AutonomousDatabase " + *adb.Spec.Details.DbName + " provisioned succesfully") } else if adb.Spec.Details.AutonomousDatabaseOCID != nil && lastSucSpec == nil { // Binding operation. We have the database ID but hasn't gotten complete infromation from OCI. // The next step is to get AutonomousDatabse details from a remote instance. - adb, err = oci.GetAutonomousDatabaseResource(r.currentLogger, dbClient, adb) + adb, err = oci.GetAutonomousDatabaseResource(currentLogger, dbClient, adb) if err != nil { - r.currentLogger.Error(err, "Fail to get Autonomous Database") + currentLogger.Error(err, "Fail to get Autonomous Database") // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable @@ -314,9 +310,9 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R } // Start/Stop/Terminate - setStateResp, err := oci.SetAutonomousDatabaseLifecycleState(r.currentLogger, dbClient, adb) + setStateResp, err := oci.SetAutonomousDatabaseLifecycleState(currentLogger, dbClient, adb) if err != nil { - r.currentLogger.Error(err, "Fail to set the Autonomous Database lifecycle state") + currentLogger.Error(err, "Fail to set the Autonomous Database lifecycle state") // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable @@ -344,7 +340,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R opcWorkRequestID = deleteResp.OpcWorkRequestId } else { - r.currentLogger.Error(err, "Unknown response type") + currentLogger.Error(err, "Unknown response type") // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable @@ -360,8 +356,8 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R return ctrl.Result{}, statusErr } - if err := oci.WaitUntilWorkCompleted(r.currentLogger, workClient, opcWorkRequestID); err != nil { - r.currentLogger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*opcWorkRequestID) + if err := oci.WaitUntilWorkCompleted(currentLogger, workClient, opcWorkRequestID); err != nil { + currentLogger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*opcWorkRequestID) // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable @@ -369,16 +365,16 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R return ctrl.Result{}, statusErr } } - r.currentLogger.Info(fmt.Sprintf("Set AutonomousDatabase %s lifecycle state to %s successfully\n", + currentLogger.Info(fmt.Sprintf("Set AutonomousDatabase %s lifecycle state to %s successfully\n", *adb.Spec.Details.DbName, adb.Spec.Details.LifecycleState)) } // Update the database in OCI from the local resource. // The local resource will be synchronized again later. - updateGenPassResp, err := oci.UpdateGeneralAndPasswordAttributes(r.currentLogger, r.KubeClient, dbClient, secretClient, adb) + updateGenPassResp, err := oci.UpdateGeneralAndPasswordAttributes(currentLogger, r.KubeClient, dbClient, secretClient, adb) if err != nil { - r.currentLogger.Error(err, "Fail to update Autonomous Database") + currentLogger.Error(err, "Fail to update Autonomous Database") // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable @@ -396,8 +392,8 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R return ctrl.Result{}, statusErr } - if err := oci.WaitUntilWorkCompleted(r.currentLogger, workClient, updateGenPassResp.OpcWorkRequestId); err != nil { - r.currentLogger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*updateGenPassResp.OpcWorkRequestId) + if err := oci.WaitUntilWorkCompleted(currentLogger, workClient, updateGenPassResp.OpcWorkRequestId); err != nil { + currentLogger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*updateGenPassResp.OpcWorkRequestId) // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable @@ -405,12 +401,12 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R return ctrl.Result{}, statusErr } } - r.currentLogger.Info("Update AutonomousDatabase " + *adb.Spec.Details.DbName + " succesfully") + currentLogger.Info("Update AutonomousDatabase " + *adb.Spec.Details.DbName + " succesfully") } - scaleResp, err := oci.UpdateScaleAttributes(r.currentLogger, r.KubeClient, dbClient, adb) + scaleResp, err := oci.UpdateScaleAttributes(currentLogger, r.KubeClient, dbClient, adb) if err != nil { - r.currentLogger.Error(err, "Fail to update Autonomous Database") + currentLogger.Error(err, "Fail to update Autonomous Database") // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable @@ -428,8 +424,8 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R return ctrl.Result{}, statusErr } - if err := oci.WaitUntilWorkCompleted(r.currentLogger, workClient, scaleResp.OpcWorkRequestId); err != nil { - r.currentLogger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*scaleResp.OpcWorkRequestId) + if err := oci.WaitUntilWorkCompleted(currentLogger, workClient, scaleResp.OpcWorkRequestId); err != nil { + currentLogger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*scaleResp.OpcWorkRequestId) // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable @@ -437,12 +433,12 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R return ctrl.Result{}, statusErr } } - r.currentLogger.Info("Scale AutonomousDatabase " + *adb.Spec.Details.DbName + " succesfully") + currentLogger.Info("Scale AutonomousDatabase " + *adb.Spec.Details.DbName + " succesfully") } - oneWayTLSResp, err := oci.UpdateOneWayTLSAttribute(r.currentLogger, r.KubeClient, dbClient, adb) + oneWayTLSResp, err := oci.UpdateOneWayTLSAttribute(currentLogger, r.KubeClient, dbClient, adb) if err != nil { - r.currentLogger.Error(err, "Fail to update Autonomous Database") + currentLogger.Error(err, "Fail to update Autonomous Database") // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable @@ -459,8 +455,8 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R return ctrl.Result{}, statusErr } - if err := oci.WaitUntilWorkCompleted(r.currentLogger, workClient, oneWayTLSResp.OpcWorkRequestId); err != nil { - r.currentLogger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*oneWayTLSResp.OpcWorkRequestId) + if err := oci.WaitUntilWorkCompleted(currentLogger, workClient, oneWayTLSResp.OpcWorkRequestId); err != nil { + currentLogger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*oneWayTLSResp.OpcWorkRequestId) // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable @@ -468,12 +464,12 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R return ctrl.Result{}, statusErr } } - r.currentLogger.Info("Update AutonomousDatabase " + *adb.Spec.Details.DbName + " 1-way TLS setting succesfully") + currentLogger.Info("Update AutonomousDatabase " + *adb.Spec.Details.DbName + " 1-way TLS setting succesfully") } - networkResp, err := oci.UpdateNetworkAttributes(r.currentLogger, r.KubeClient, dbClient, adb) + networkResp, err := oci.UpdateNetworkAttributes(currentLogger, r.KubeClient, dbClient, adb) if err != nil { - r.currentLogger.Error(err, "Fail to update Autonomous Database") + currentLogger.Error(err, "Fail to update Autonomous Database") // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable @@ -490,8 +486,8 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R return ctrl.Result{}, statusErr } - if err := oci.WaitUntilWorkCompleted(r.currentLogger, workClient, networkResp.OpcWorkRequestId); err != nil { - r.currentLogger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*networkResp.OpcWorkRequestId) + if err := oci.WaitUntilWorkCompleted(currentLogger, workClient, networkResp.OpcWorkRequestId); err != nil { + currentLogger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*networkResp.OpcWorkRequestId) // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable @@ -499,15 +495,15 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R return ctrl.Result{}, statusErr } } - r.currentLogger.Info("Update AutonomousDatabase " + *adb.Spec.Details.DbName + " network settings succesfully") + currentLogger.Info("Update AutonomousDatabase " + *adb.Spec.Details.DbName + " network settings succesfully") } } } // Get the information from OCI - updatedADB, err := oci.GetAutonomousDatabaseResource(r.currentLogger, dbClient, adb) + updatedADB, err := oci.GetAutonomousDatabaseResource(currentLogger, dbClient, adb) if err != nil { - r.currentLogger.Error(err, "Fail to get Autonomous Database") + currentLogger.Error(err, "Fail to get Autonomous Database") // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable @@ -520,7 +516,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R adb = updatedADB // Update local object and the status - if err := r.updateAutonomousDatabaseDetails(adb); err != nil { + if err := adbutil.UpdateAutonomousDatabaseDetails(currentLogger, r.KubeClient, adb); err != nil { // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { @@ -538,8 +534,8 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R (lastSucSpec != nil && lastSucSpec.Details.Wallet.Password.OCISecretOCID != adb.Spec.Details.Wallet.Password.OCISecretOCID) if (passwordSecretUpdate || passwordOCIDUpdate) && adb.Status.LifecycleState == database.AutonomousDatabaseLifecycleStateAvailable { - if err := adbutil.CreateWalletSecret(r.currentLogger, r.KubeClient, dbClient, secretClient, adb); err != nil { - r.currentLogger.Error(err, "Fail to download Instance Wallet") + if err := adbutil.CreateWalletSecret(currentLogger, r.KubeClient, dbClient, secretClient, adb); err != nil { + currentLogger.Error(err, "Fail to download Instance Wallet") // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { @@ -550,10 +546,10 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R } /***************************************************** - * AutonomousDatabase Backups + * Sync AutonomousDatabase Backups *****************************************************/ - if err := adbutil.CreateBackupResources(r.currentLogger, r.KubeClient, dbClient, adb); err != nil { - r.currentLogger.Error(err, "Fail to update Autonomous Database") + if err := adbutil.SyncBackupResources(currentLogger, r.KubeClient, dbClient, adb); err != nil { + currentLogger.Error(err, "Fail to sync Autonomous Database backups") // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable @@ -571,57 +567,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R return ctrl.Result{}, err } - r.currentLogger.Info("AutonomousDatabase resource reconcile successfully") + currentLogger.Info("AutonomousDatabase resource reconcile successfully") return ctrl.Result{}, nil } - -// type patchValue struct { -// Op string `json:"op"` -// Path string `json:"path"` -// Value interface{} `json:"value"` -// } - -func (r *AutonomousDatabaseReconciler) updateAutonomousDatabaseDetails(adb *dbv1alpha1.AutonomousDatabase) error { - // Patch the AutonomousDatabase Kubernetes resource to avoid resource version conflicts. - // payload := []patchValue{{ - // Op: "replace", - // Path: "/spec/details", - // Value: adb.Spec.Details, - // }} - // payloadBytes, err := json.Marshal(payload) - // if err != nil { - // return err - // } - - // patch := client.RawPatch(types.JSONPatchType, payloadBytes) - // if err := r.KubeClient.Patch(context.TODO(), adb, patch); err != nil { - // return err - // } - - if err := retry.RetryOnConflict(retry.DefaultRetry, func() error { - curADB := &dbv1alpha1.AutonomousDatabase{} - - namespacedName := types.NamespacedName{ - Namespace: adb.GetNamespace(), - Name: adb.GetName(), - } - - if err := r.KubeClient.Get(context.TODO(), namespacedName, curADB); err != nil { - return err - } - - curADB.Spec.Details = adb.Spec.Details - return r.KubeClient.Update(context.TODO(), curADB) - }); err != nil { - return err - } - - // Update status - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { - return statusErr - } - r.currentLogger.Info("Update local resource AutonomousDatabase successfully") - - return nil -} diff --git a/controllers/database/autonomousdatabasebackup_controller.go b/controllers/database/autonomousdatabasebackup_controller.go index a4aa3c6f..df96d434 100644 --- a/controllers/database/autonomousdatabasebackup_controller.go +++ b/controllers/database/autonomousdatabasebackup_controller.go @@ -64,8 +64,6 @@ type AutonomousDatabaseBackupReconciler struct { KubeClient client.Client Log logr.Logger Scheme *runtime.Scheme - - currentLogger logr.Logger } // SetupWithManager sets up the controller with the Manager. @@ -73,17 +71,20 @@ func (r *AutonomousDatabaseBackupReconciler) SetupWithManager(mgr ctrl.Manager) return ctrl.NewControllerManagedBy(mgr). For(&databasev1alpha1.AutonomousDatabaseBackup{}). WithEventFilter(r.eventFilterPredicate()). - WithOptions(controller.Options{MaxConcurrentReconciles: 50}). // ReconcileHandler is never invoked concurrently with the same object. + WithOptions(controller.Options{MaxConcurrentReconciles: 100}). // ReconcileHandler is never invoked concurrently with the same object. Complete(r) } func (r *AutonomousDatabaseBackupReconciler) eventFilterPredicate() predicate.Predicate { pred := predicate.Funcs{ + CreateFunc: func(e event.CreateEvent) bool { + return true + }, UpdateFunc: func(e event.UpdateEvent) bool { oldBackup := e.ObjectOld.DeepCopyObject().(*dbv1alpha1.AutonomousDatabaseBackup) newBackup := e.ObjectNew.DeepCopyObject().(*dbv1alpha1.AutonomousDatabaseBackup) - specChanged := reflect.DeepEqual(oldBackup.Spec, newBackup.Spec) + specChanged := !reflect.DeepEqual(oldBackup.Spec, newBackup.Spec) if specChanged { // Enqueue request return true @@ -100,11 +101,11 @@ func (r *AutonomousDatabaseBackupReconciler) eventFilterPredicate() predicate.Pr return pred } -//+kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabasebackups,verbs=get;list;watch;create;update;patch;delete +//+kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabasebackups,verbs=get;list;watch;create;delete //+kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabasebackups/status,verbs=get;update;patch func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { - r.currentLogger = r.Log.WithValues("autonomousdatabase_backup", req.NamespacedName) + currentLogger := r.Log.WithValues("Namespaced/Name", req.NamespacedName) adbBackup := &dbv1alpha1.AutonomousDatabaseBackup{} if err := r.KubeClient.Get(context.TODO(), req.NamespacedName, adbBackup); err != nil { @@ -125,7 +126,7 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req } provider, err := oci.GetOCIProvider(r.KubeClient, authData) if err != nil { - r.currentLogger.Error(err, "Fail to get OCI provider") + currentLogger.Error(err, "Fail to get OCI provider") // Change the status to UNAVAILABLE adbBackup.Status.LifecycleState = database.AutonomousDatabaseBackupLifecycleStateFailed @@ -137,7 +138,7 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req dbClient, err := database.NewDatabaseClientWithConfigurationProvider(provider) if err != nil { - r.currentLogger.Error(err, "Fail to get OCI database client") + currentLogger.Error(err, "Fail to get OCI database client") // Change the status to UNAVAILABLE adbBackup.Status.LifecycleState = database.AutonomousDatabaseBackupLifecycleStateFailed @@ -149,7 +150,7 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req workClient, err := workrequests.NewWorkRequestClientWithConfigurationProvider(provider) if err != nil { - r.currentLogger.Error(err, "Fail to get OCI work request client") + currentLogger.Error(err, "Fail to get OCI work request client") // Change the status to UNAVAILABLE adbBackup.Status.LifecycleState = database.AutonomousDatabaseBackupLifecycleStateFailed @@ -160,82 +161,59 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req } /****************************************************************** - * Create a backup if the Spec.AutonomousDatabaseBackupOCID is - * empty and the LifecycleState is never assigned, or bind to an exisiting - * backup if the Spec.AutonomousDatabaseBackupOCID isn't empty. + * If the Spec.AutonomousDatabaseBackupOCID is empty and the LifecycleState is never assigned , create a backup. + * LifecycleState is checked to avoid sending a duplicated backup request when the backup is creating. + * Otherwise, bind to an exisiting backup if the Spec.AutonomousDatabaseBackupOCID isn't empty. ******************************************************************/ if adbBackup.Spec.AutonomousDatabaseBackupOCID == "" && adbBackup.Status.LifecycleState == "" { - resp, err := oci.CreateAutonomousDatabaseBackup(r.currentLogger, dbClient, adbBackup) + // Create a new backup + resp, err := oci.CreateAutonomousDatabaseBackup(currentLogger, dbClient, adbBackup) if err != nil { - r.currentLogger.Error(err, "Fail to create AutonomousDatabase Backup") - - // Change the status to UNAVAILABLE - adbBackup.Status.LifecycleState = database.AutonomousDatabaseBackupLifecycleStateFailed - if statusErr := backupUtil.UpdateAutonomousDatabaseBackupStatus(r.KubeClient, adbBackup); statusErr != nil { - return ctrl.Result{}, statusErr - } + currentLogger.Error(err, "Fail to create AutonomousDatabase Backup") return ctrl.Result{}, nil } // update the status - adbBackup.Status.AutonomousDatabaseBackupOCID = *resp.Id - adbBackup.Status.LifecycleState = resp.LifecycleState + adbBackup.UpdateStatusFromAutonomousDatabaseBackupResponse(resp.AutonomousDatabaseBackup) backupUtil.UpdateAutonomousDatabaseBackupStatus(r.KubeClient, adbBackup) // Wait until the work is done - if err := oci.WaitUntilWorkCompleted(r.currentLogger, workClient, resp.OpcWorkRequestId); err != nil { - r.currentLogger.Error(err, "Fail to watch the status of provision request. opcWorkRequestID = "+*resp.OpcWorkRequestId) - - // Change the status to UNAVAILABLE - adbBackup.Status.LifecycleState = database.AutonomousDatabaseBackupLifecycleStateFailed - if statusErr := backupUtil.UpdateAutonomousDatabaseBackupStatus(r.KubeClient, adbBackup); statusErr != nil { - return ctrl.Result{}, statusErr - } + if err := oci.WaitUntilWorkCompleted(currentLogger, workClient, resp.OpcWorkRequestId); err != nil { + currentLogger.Error(err, "Fail to watch the status of provision request. opcWorkRequestID = "+*resp.OpcWorkRequestId) return ctrl.Result{}, nil } + + currentLogger.Info("AutonomousBackup " + *resp.DisplayName + " created successfully") + } else if adbBackup.Spec.AutonomousDatabaseBackupOCID != "" { - // Binding - // Add the Status.AutonomousDatabaseBackupOCID and update the status + // Bind to an existing backup adbBackup.Status.AutonomousDatabaseBackupOCID = adbBackup.Spec.AutonomousDatabaseBackupOCID - backupUtil.UpdateAutonomousDatabaseBackupStatus(r.KubeClient, adbBackup) } /****************************************************************** * Update the status of the resource if the * Status.AutonomousDatabaseOCID isn't empty. ******************************************************************/ - if adbBackup.Status.AutonomousDatabaseOCID != "" { - resp, err := oci.GetAutonomousDatabaseBackup(dbClient, &adbBackup.Status.AutonomousDatabaseBackupOCID) + if adbBackup.Status.AutonomousDatabaseBackupOCID != "" { + resp, err := oci.GetAutonomousDatabaseBackup(dbClient, adbBackup.Status.AutonomousDatabaseBackupOCID) if err != nil { - // Change the status to UNAVAILABLE - adbBackup.Status.LifecycleState = database.AutonomousDatabaseBackupLifecycleStateFailed - if statusErr := backupUtil.UpdateAutonomousDatabaseBackupStatus(r.KubeClient, adbBackup); statusErr != nil { - return ctrl.Result{}, statusErr - } - return ctrl.Result{}, err + currentLogger.Error(err, "Fail to get AutonomousDatabase Backup. The AutonomousDatabase Backup OCID = "+adbBackup.Status.AutonomousDatabaseBackupOCID) + return ctrl.Result{}, nil } - adbBackup.UpdateStatusFromAutonomousDatabaseBackupResponse(resp) - + adbBackup.UpdateStatusFromAutonomousDatabaseBackupResponse(resp.AutonomousDatabaseBackup) backupUtil.UpdateAutonomousDatabaseBackupStatus(r.KubeClient, adbBackup) } /****************************************************************** * Look up the owner AutonomousDatabase and set the ownerReference - * if the owner hasn't been set. + * if the owner hasn't been set yet. ******************************************************************/ - if len(adbBackup.GetOwnerReferences()) > 0 && adbBackup.Status.AutonomousDatabaseOCID != "" { - if err := backupUtil.SetOwnerAutonomousDatabase(r.currentLogger, r.KubeClient, adbBackup); err != nil { - // Change the status to UNAVAILABLE - adbBackup.Status.LifecycleState = database.AutonomousDatabaseBackupLifecycleStateFailed - if statusErr := backupUtil.UpdateAutonomousDatabaseBackupStatus(r.KubeClient, adbBackup); statusErr != nil { - return ctrl.Result{}, statusErr - } + if len(adbBackup.GetOwnerReferences()) == 0 && adbBackup.Status.AutonomousDatabaseOCID != "" { + if err := backupUtil.SetOwnerAutonomousDatabase(currentLogger, r.KubeClient, adbBackup); err != nil { return ctrl.Result{}, err } } - r.currentLogger.Info("AutonomousDatabaseBackup reconcile successfully") - return ctrl.Result{}, nil } diff --git a/controllers/database/autonomousdatabaserestore_controller.go b/controllers/database/autonomousdatabaserestore_controller.go index d666fc1f..e393bc5d 100644 --- a/controllers/database/autonomousdatabaserestore_controller.go +++ b/controllers/database/autonomousdatabaserestore_controller.go @@ -63,11 +63,9 @@ type AutonomousDatabaseRestoreReconciler struct { KubeClient client.Client Log logr.Logger Scheme *runtime.Scheme - - currentLogger logr.Logger } -//+kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabaserestores,verbs=get;list;watch;create;update;patch;delete +//+kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabaserestores,verbs=get;list;watch;create;delete //+kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabaserestores/status,verbs=get;update;patch // Reconcile is part of the main kubernetes reconciliation loop which aims to @@ -80,7 +78,7 @@ type AutonomousDatabaseRestoreReconciler struct { // For more details, check Reconcile and its Result here: // - https://pkg.go.dev/sigs.k8s.io/controller-runtime@v0.6.4/pkg/reconcile func (r *AutonomousDatabaseRestoreReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { - r.currentLogger = r.Log.WithValues("autonomousdatabaserestore", req.NamespacedName) + currentLogger := r.Log.WithValues("Namespaced/Name", req.NamespacedName) restore := &dbv1alpha1.AutonomousDatabaseRestore{} if err := r.KubeClient.Get(context.TODO(), req.NamespacedName, restore); err != nil { @@ -101,7 +99,7 @@ func (r *AutonomousDatabaseRestoreReconciler) Reconcile(ctx context.Context, req } provider, err := oci.GetOCIProvider(r.KubeClient, authData) if err != nil { - r.currentLogger.Error(err, "Fail to get OCI provider") + currentLogger.Error(err, "Fail to get OCI provider") // Change the status to UNAVAILABLE restore.Status.LifecycleState = dbv1alpha1.RestoreLifecycleStateFailed @@ -113,7 +111,7 @@ func (r *AutonomousDatabaseRestoreReconciler) Reconcile(ctx context.Context, req dbClient, err := database.NewDatabaseClientWithConfigurationProvider(provider) if err != nil { - r.currentLogger.Error(err, "Fail to get OCI database client") + currentLogger.Error(err, "Fail to get OCI database client") // Change the status to UNAVAILABLE restore.Status.LifecycleState = dbv1alpha1.RestoreLifecycleStateFailed @@ -125,7 +123,7 @@ func (r *AutonomousDatabaseRestoreReconciler) Reconcile(ctx context.Context, req workClient, err := workrequests.NewWorkRequestClientWithConfigurationProvider(provider) if err != nil { - r.currentLogger.Error(err, "Fail to get OCI work request client") + currentLogger.Error(err, "Fail to get OCI work request client") // Change the status to UNAVAILABLE restore.Status.LifecycleState = dbv1alpha1.RestoreLifecycleStateFailed @@ -141,16 +139,16 @@ func (r *AutonomousDatabaseRestoreReconciler) Reconcile(ctx context.Context, req if restore.Status.LifecycleState == "" || restore.Status.LifecycleState == dbv1alpha1.RestoreLifecycleStateNew { var restoreTime *common.SDKTime - if restore.Spec.Destination.BackupName != "" { + if restore.Spec.Source.BackupName != "" { backup := &dbv1alpha1.AutonomousDatabaseBackup{} - namespacedName := types.NamespacedName{Namespace: restore.Namespace, Name: restore.Spec.Destination.BackupName} + namespacedName := types.NamespacedName{Namespace: restore.Namespace, Name: restore.Spec.Source.BackupName} if err := r.KubeClient.Get(context.TODO(), namespacedName, backup); err != nil { return ctrl.Result{}, err } restoreTime, err = ociutil.ParseDisplayTime(backup.Status.TimeEnded) if err != nil { - r.currentLogger.Error(err, "Fail to parse time "+backup.Status.TimeEnded) + currentLogger.Error(err, "Fail to parse time "+backup.Status.TimeEnded) // Change the status to UNAVAILABLE restore.Status.LifecycleState = dbv1alpha1.RestoreLifecycleStateFailed @@ -159,10 +157,10 @@ func (r *AutonomousDatabaseRestoreReconciler) Reconcile(ctx context.Context, req } return ctrl.Result{}, nil } - } else if restore.Spec.Destination.TimeStamp != "" { - restoreTime, err = ociutil.ParseDisplayTime(restore.Spec.Destination.TimeStamp) + } else if restore.Spec.Source.TimeStamp != "" { + restoreTime, err = ociutil.ParseDisplayTime(restore.Spec.Source.TimeStamp) if err != nil { - r.currentLogger.Error(err, "Fail to parse time "+restore.Spec.Destination.TimeStamp) + currentLogger.Error(err, "Fail to parse time "+restore.Spec.Source.TimeStamp) // Change the status to UNAVAILABLE restore.Status.LifecycleState = dbv1alpha1.RestoreLifecycleStateFailed @@ -175,7 +173,7 @@ func (r *AutonomousDatabaseRestoreReconciler) Reconcile(ctx context.Context, req resp, err := oci.RestoreAutonomousDatabase(dbClient, restore.Spec.AutonomousDatabaseOCID, restoreTime) if err != nil { - r.currentLogger.Error(err, "Fail to restore database") + currentLogger.Error(err, "Fail to restore database") // Change the status to UNAVAILABLE restore.Status.LifecycleState = dbv1alpha1.RestoreLifecycleStateFailed @@ -185,8 +183,8 @@ func (r *AutonomousDatabaseRestoreReconciler) Reconcile(ctx context.Context, req return ctrl.Result{}, nil } - if err := oci.WaitUntilWorkCompleted(r.currentLogger, workClient, resp.OpcWorkRequestId); err != nil { - r.currentLogger.Error(err, "Fail to watch workrequest. Workrequest ID = "+*resp.OpcWorkRequestId) + if err := oci.WaitUntilWorkCompleted(currentLogger, workClient, resp.OpcWorkRequestId); err != nil { + currentLogger.Error(err, "Fail to watch workrequest. Workrequest ID = "+*resp.OpcWorkRequestId) // Change the status to UNAVAILABLE restore.Status.LifecycleState = dbv1alpha1.RestoreLifecycleStateFailed @@ -196,7 +194,7 @@ func (r *AutonomousDatabaseRestoreReconciler) Reconcile(ctx context.Context, req return ctrl.Result{}, nil } - r.currentLogger.Info("Restore database completed") + currentLogger.Info("Restore database completed") restore.Status.LifecycleState = dbv1alpha1.RestoreLifecycleStateCompleted if statusErr := restoreUtil.UpdateAutonomousDatabaseRestoreStatus(r.KubeClient, restore); statusErr != nil { From 06aebdb0c5ae0b14aa03a1837ae4408744bd4abd Mon Sep 17 00:00:00 2001 From: Gajanan Bhat Date: Wed, 2 Feb 2022 12:55:59 +0530 Subject: [PATCH 149/628] CDB Checks --- apis/database/v1alpha1/cdb_webhook.go | 14 +- .../v1alpha1/zz_generated.deepcopy.go | 1 - go.mod | 1 + go.sum | 2 + ...autonomousdatabase_controller_bind_test.go | 320 +++++++----------- test/e2e/suite_test.go | 122 +++++-- 6 files changed, 221 insertions(+), 239 deletions(-) diff --git a/apis/database/v1alpha1/cdb_webhook.go b/apis/database/v1alpha1/cdb_webhook.go index 529df5fb..34983b78 100644 --- a/apis/database/v1alpha1/cdb_webhook.go +++ b/apis/database/v1alpha1/cdb_webhook.go @@ -104,6 +104,18 @@ func (r *CDB) ValidateCreate() error { allErrs = append(allErrs, field.Required(field.NewPath("spec").Child("dbPort"), "Please specify DB Server Port")) } + if r.Spec.DBPort < 0 { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("dbPort"), "Please specify a valid DB Server Port")) + } + if r.Spec.ORDSPort < 0 { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("dbPort"), "Please specify a valid ORDS Port")) + } + if r.Spec.Replicas < 0 { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("dbPort"), "Please specify a valid value for Replicas")) + } if r.Spec.ORDSImage == "" { allErrs = append(allErrs, field.Required(field.NewPath("spec").Child("ordsImage"), "Please specify name of ORDS Image to be used")) @@ -132,7 +144,7 @@ func (r *CDB) ValidateCreate() error { return nil } return apierrors.NewInvalid( - schema.GroupKind{Group: "database.oracle.com", Kind: "PDB"}, + schema.GroupKind{Group: "database.oracle.com", Kind: "CDB"}, r.Name, allErrs) } diff --git a/apis/database/v1alpha1/zz_generated.deepcopy.go b/apis/database/v1alpha1/zz_generated.deepcopy.go index 30887666..c59ed069 100644 --- a/apis/database/v1alpha1/zz_generated.deepcopy.go +++ b/apis/database/v1alpha1/zz_generated.deepcopy.go @@ -1,4 +1,3 @@ -//go:build !ignore_autogenerated // +build !ignore_autogenerated /* diff --git a/go.mod b/go.mod index 4e19053e..be66c0ff 100644 --- a/go.mod +++ b/go.mod @@ -6,6 +6,7 @@ require ( github.com/go-logr/logr v0.4.0 github.com/onsi/ginkgo v1.16.4 github.com/onsi/gomega v1.13.0 + github.com/oracle/oci-go-sdk/v45 v45.2.0 // indirect github.com/oracle/oci-go-sdk/v51 v51.0.0 gopkg.in/yaml.v2 v2.4.0 k8s.io/api v0.21.2 diff --git a/go.sum b/go.sum index 2b051eef..d3f2ec71 100644 --- a/go.sum +++ b/go.sum @@ -304,6 +304,8 @@ github.com/onsi/gomega v1.7.1/go.mod h1:XdKZgCCFLUoM/7CFJVPcG8C1xQ1AJ0vpAezJrB7J github.com/onsi/gomega v1.10.1/go.mod h1:iN09h71vgCQne3DLsj+A5owkum+a2tYe+TOCB1ybHNo= github.com/onsi/gomega v1.13.0 h1:7lLHu94wT9Ij0o6EWWclhu0aOh32VxhkwEJvzuWPeak= github.com/onsi/gomega v1.13.0/go.mod h1:lRk9szgn8TxENtWd0Tp4c3wjlRfMTMH27I+3Je41yGY= +github.com/oracle/oci-go-sdk/v45 v45.2.0 h1:vCPoQlE+DOrM2heJn66rvPU6fbsc/0Cxtzs2jnFut6U= +github.com/oracle/oci-go-sdk/v45 v45.2.0/go.mod h1:ZM6LGiRO5TPQJxTlrXbcHMbClE775wnGD5U/EerCsRw= github.com/oracle/oci-go-sdk/v51 v51.0.0 h1:eDUVMsAzvf+jfq4xbtpQrxznYNjccwKKyIhmRvZLgwI= github.com/oracle/oci-go-sdk/v51 v51.0.0/go.mod h1:d9KSNXwE64drofxoor+y/JWofJqLqRF9D1/AtfYIE10= github.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc= diff --git a/test/e2e/autonomousdatabase_controller_bind_test.go b/test/e2e/autonomousdatabase_controller_bind_test.go index 9b03e2b6..ebd6c4f0 100644 --- a/test/e2e/autonomousdatabase_controller_bind_test.go +++ b/test/e2e/autonomousdatabase_controller_bind_test.go @@ -71,240 +71,158 @@ var _ = Describe("test ADB binding with hardLink=true", func() { time.Sleep(time.Second * 20) }) - It("should init the test", func() { - By("creating a temp ADB in OCI for binding test") - dbName := e2eutil.GenerateDBName() - createResp, err := e2eutil.CreateAutonomousDatabase(dbClient, &SharedCompartmentOCID, &dbName, &SharedPlainTextAdminPassword) - Expect(err).ShouldNot(HaveOccurred()) - Expect(createResp.AutonomousDatabase.Id).ShouldNot(BeNil()) - - By("Save the database ID for later use") - adbID = createResp.AutonomousDatabase.Id - terminatedAdbID = *adbID - - By("Wait until the work request is in SUCCEEDED status") - workClient, err := workrequests.NewWorkRequestClientWithConfigurationProvider(configProvider) - Expect(err).ShouldNot(HaveOccurred()) - - err = e2eutil.WaitUntilWorkCompleted(workClient, createResp.OpcWorkRequestId) - Expect(err).ShouldNot(HaveOccurred()) - - // listResp, err := e2eutil.ListAutonomousDatabases(dbClient, &SharedCompartmentOCID, &dbName) - // Expect(err).ShouldNot(HaveOccurred()) - // fmt.Printf("List request DB %s is in %s state \n", *listResp.Items[0].DisplayName, listResp.Items[0].LifecycleState) - }) + It("should init the test", func() { + By("creating a temp ADB in OCI for binding test") + dbName := e2eutil.GenerateDBName() + createResp, err := e2eutil.CreateAutonomousDatabase(dbClient, &SharedCompartmentOCID, &dbName, &SharedPlainTextAdminPassword) + Expect(err).ShouldNot(HaveOccurred()) + Expect(createResp.AutonomousDatabase.Id).ShouldNot(BeNil()) + + By("Save the database ID for later use") + adbID = createResp.AutonomousDatabase.Id + terminatedAdbID = *adbID + + By("Wait until the work request is in SUCCEEDED status") + workClient, err := workrequests.NewWorkRequestClientWithConfigurationProvider(configProvider) + Expect(err).ShouldNot(HaveOccurred()) + + err = e2eutil.WaitUntilWorkCompleted(workClient, createResp.OpcWorkRequestId) + Expect(err).ShouldNot(HaveOccurred()) + + // listResp, err := e2eutil.ListAutonomousDatabases(dbClient, &SharedCompartmentOCID, &dbName) + // Expect(err).ShouldNot(HaveOccurred()) + // fmt.Printf("List request DB %s is in %s state \n", *listResp.Items[0].DisplayName, listResp.Items[0].LifecycleState) + }) - Describe("ADB binding with HardLink = false using Wallet Password Secret", func() { - It("Should create a AutonomousDatabase resource with HardLink = false", func() { - adb := &dbv1alpha1.AutonomousDatabase{ - TypeMeta: metav1.TypeMeta{ - APIVersion: "database.oracle.com/v1alpha1", - Kind: "AutonomousDatabase", - }, - ObjectMeta: metav1.ObjectMeta{ - Name: "bindadb", - Namespace: ADBNamespace, - }, - Spec: dbv1alpha1.AutonomousDatabaseSpec{ - Details: dbv1alpha1.AutonomousDatabaseDetails{ - AutonomousDatabaseOCID: adbID, - Wallet: dbv1alpha1.WalletSpec{ - Name: common.String(downloadedWallet), - Password: dbv1alpha1.PasswordSpec{ - K8sSecretName: common.String(SharedWalletPassSecretName), - }, + Describe("ADB binding with HardLink = false using Wallet Password Secret", func() { + It("Should create a AutonomousDatabase resource with HardLink = false", func() { + adb := &dbv1alpha1.AutonomousDatabase{ + TypeMeta: metav1.TypeMeta{ + APIVersion: "database.oracle.com/v1alpha1", + Kind: "AutonomousDatabase", + }, + ObjectMeta: metav1.ObjectMeta{ + Name: "bindadb", + Namespace: ADBNamespace, + }, + Spec: dbv1alpha1.AutonomousDatabaseSpec{ + Details: dbv1alpha1.AutonomousDatabaseDetails{ + AutonomousDatabaseOCID: adbID, + Wallet: dbv1alpha1.WalletSpec{ + Name: common.String(downloadedWallet), + Password: dbv1alpha1.PasswordSpec{ + K8sSecretName: common.String(SharedWalletPassSecretName), }, }, - HardLink: common.Bool(false), - OCIConfig: dbv1alpha1.OCIConfigSpec{ - ConfigMapName: common.String(SharedOCIConfigMapName), - SecretName: common.String(SharedOCISecretName), - }, }, - } + HardLink: common.Bool(false), + OCIConfig: dbv1alpha1.OCIConfigSpec{ + ConfigMapName: common.String(SharedOCIConfigMapName), + SecretName: common.String(SharedOCISecretName), + }, + }, + } - adbLookupKey = types.NamespacedName{Name: adb.Name, Namespace: adb.Namespace} + adbLookupKey = types.NamespacedName{Name: adb.Name, Namespace: adb.Namespace} - Expect(k8sClient.Create(context.TODO(), adb)).Should(Succeed()) - }) + Expect(k8sClient.Create(context.TODO(), adb)).Should(Succeed()) + }) - It("should bind to an ADB", e2ebehavior.AssertBind(&k8sClient, &adbLookupKey)) + It("should bind to an ADB", e2ebehavior.AssertBind(&k8sClient, &adbLookupKey)) - It("Should download an instance wallet using the password from K8s Secret "+SharedWalletPassSecretName, e2ebehavior.AssertWallet(&k8sClient, &adbLookupKey)) + It("Should download an instance wallet using the password from K8s Secret "+SharedWalletPassSecretName, e2ebehavior.AssertWallet(&k8sClient, &adbLookupKey)) -<<<<<<< HEAD - It("should update ADB", e2ebehavior.AssertUpdate(&k8sClient, &dbClient, &adbLookupKey)) -======= It("should update ADB", e2ebehavior.UpdateAndAssertDetails(&k8sClient, &dbClient, &adbLookupKey)) ->>>>>>> feff58aab50838a39c03d8e22fdede11da02aa14 - It("Should stop ADB", e2ebehavior.UpdateAndAssertState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateStopped)) + It("Should stop ADB", e2ebehavior.UpdateAndAssertState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateStopped)) - It("Should restart ADB", e2ebehavior.UpdateAndAssertState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateAvailable)) + It("Should restart ADB", e2ebehavior.UpdateAndAssertState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateAvailable)) - It("Should delete the resource in cluster but not terminate the database in OCI", e2ebehavior.AssertSoftLinkDelete(&k8sClient, &adbLookupKey)) - }) + It("Should delete the resource in cluster but not terminate the database in OCI", e2ebehavior.AssertSoftLinkDelete(&k8sClient, &adbLookupKey)) + }) - Describe("ADB binding with HardLink = true using Wallet Password OCID", func() { - It("Should create a AutonomousDatabase resource with HardLink = true", func() { - adb := &dbv1alpha1.AutonomousDatabase{ - TypeMeta: metav1.TypeMeta{ - APIVersion: "database.oracle.com/v1alpha1", - Kind: "AutonomousDatabase", - }, - ObjectMeta: metav1.ObjectMeta{ - Name: "bindadb", - Namespace: ADBNamespace, - }, - Spec: dbv1alpha1.AutonomousDatabaseSpec{ - Details: dbv1alpha1.AutonomousDatabaseDetails{ - AutonomousDatabaseOCID: adbID, - Wallet: dbv1alpha1.WalletSpec{ - Name: common.String(downloadedWallet), - Password: dbv1alpha1.PasswordSpec{ - OCISecretOCID: common.String(SharedInstanceWalletPasswordOCID), - }, + Describe("ADB binding with HardLink = true using Wallet Password OCID", func() { + It("Should create a AutonomousDatabase resource with HardLink = true", func() { + adb := &dbv1alpha1.AutonomousDatabase{ + TypeMeta: metav1.TypeMeta{ + APIVersion: "database.oracle.com/v1alpha1", + Kind: "AutonomousDatabase", + }, + ObjectMeta: metav1.ObjectMeta{ + Name: "bindadb", + Namespace: ADBNamespace, + }, + Spec: dbv1alpha1.AutonomousDatabaseSpec{ + Details: dbv1alpha1.AutonomousDatabaseDetails{ + AutonomousDatabaseOCID: adbID, + Wallet: dbv1alpha1.WalletSpec{ + Name: common.String(downloadedWallet), + Password: dbv1alpha1.PasswordSpec{ + OCISecretOCID: common.String(SharedInstanceWalletPasswordOCID), }, }, - HardLink: common.Bool(true), - OCIConfig: dbv1alpha1.OCIConfigSpec{ - ConfigMapName: common.String(SharedOCIConfigMapName), - SecretName: common.String(SharedOCISecretName), - }, }, - } + HardLink: common.Bool(true), + OCIConfig: dbv1alpha1.OCIConfigSpec{ + ConfigMapName: common.String(SharedOCIConfigMapName), + SecretName: common.String(SharedOCISecretName), + }, + }, + } - adbLookupKey = types.NamespacedName{Name: adb.Name, Namespace: adb.Namespace} + adbLookupKey = types.NamespacedName{Name: adb.Name, Namespace: adb.Namespace} - Expect(k8sClient.Create(context.TODO(), adb)).Should(Succeed()) - }) + Expect(k8sClient.Create(context.TODO(), adb)).Should(Succeed()) + }) - It("should bind to an ADB", e2ebehavior.AssertBind(&k8sClient, &adbLookupKey)) + It("should bind to an ADB", e2ebehavior.AssertBind(&k8sClient, &adbLookupKey)) - It("Should download an instance wallet using the password from OCI Secret OCID "+SharedInstanceWalletPasswordOCID, e2ebehavior.AssertWallet(&k8sClient, &adbLookupKey)) + It("Should download an instance wallet using the password from OCI Secret OCID "+SharedInstanceWalletPasswordOCID, e2ebehavior.AssertWallet(&k8sClient, &adbLookupKey)) -<<<<<<< HEAD - It("should update ADB", e2ebehavior.AssertUpdate(&k8sClient, &dbClient, &adbLookupKey)) -======= It("should update ADB", e2ebehavior.UpdateAndAssertDetails(&k8sClient, &dbClient, &adbLookupKey)) ->>>>>>> feff58aab50838a39c03d8e22fdede11da02aa14 - - It("Should stop ADB", e2ebehavior.UpdateAndAssertState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateStopped)) - - It("Should restart ADB", e2ebehavior.UpdateAndAssertState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateAvailable)) - It("Should delete the resource in cluster and terminate the database in OCI", e2ebehavior.AssertHardLinkDelete(&k8sClient, &dbClient, &adbLookupKey)) - }) + It("Should stop ADB", e2ebehavior.UpdateAndAssertState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateStopped)) - //Bind to terminated adb from previous test - Describe("bind to a terminated adb", func() { + It("Should restart ADB", e2ebehavior.UpdateAndAssertState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateAvailable)) - //Wait until remote state is terminated - It("Should check that OCI adb state is terminated", e2ebehavior.AssertRemoteStateOCID(&k8sClient, &dbClient, &terminatedAdbID, database.AutonomousDatabaseLifecycleStateTerminated)) + It("Should delete the resource in cluster and terminate the database in OCI", e2ebehavior.AssertHardLinkDelete(&k8sClient, &dbClient, &adbLookupKey)) + }) - It("Should create a AutonomousDatabase resource", func() { - adb := &dbv1alpha1.AutonomousDatabase{ - TypeMeta: metav1.TypeMeta{ - APIVersion: "database.oracle.com/v1alpha1", - Kind: "AutonomousDatabase", - }, - ObjectMeta: metav1.ObjectMeta{ - Name: "bindadb", - Namespace: ADBNamespace, + //Bind to terminated adb from previous test + Describe("bind to a terminated adb", func() { + + //Wait until remote state is terminated + It("Should check that OCI adb state is terminated", e2ebehavior.AssertRemoteStateOCID(&k8sClient, &dbClient, &terminatedAdbID, database.AutonomousDatabaseLifecycleStateTerminated)) + + It("Should create a AutonomousDatabase resource", func() { + adb := &dbv1alpha1.AutonomousDatabase{ + TypeMeta: metav1.TypeMeta{ + APIVersion: "database.oracle.com/v1alpha1", + Kind: "AutonomousDatabase", + }, + ObjectMeta: metav1.ObjectMeta{ + Name: "bindadb", + Namespace: ADBNamespace, + }, + Spec: dbv1alpha1.AutonomousDatabaseSpec{ + Details: dbv1alpha1.AutonomousDatabaseDetails{ + AutonomousDatabaseOCID: &terminatedAdbID, }, - Spec: dbv1alpha1.AutonomousDatabaseSpec{ - Details: dbv1alpha1.AutonomousDatabaseDetails{ - AutonomousDatabaseOCID: &terminatedAdbID, - }, - HardLink: common.Bool(true), - OCIConfig: dbv1alpha1.OCIConfigSpec{ - ConfigMapName: common.String(SharedOCIConfigMapName), - SecretName: common.String(SharedOCISecretName), - }, + HardLink: common.Bool(true), + OCIConfig: dbv1alpha1.OCIConfigSpec{ + ConfigMapName: common.String(SharedOCIConfigMapName), + SecretName: common.String(SharedOCISecretName), }, - } - - adbLookupKey = types.NamespacedName{Name: adb.Name, Namespace: adb.Namespace} + }, + } - Expect(k8sClient.Create(context.TODO(), adb)).Should(Succeed()) - }) + adbLookupKey = types.NamespacedName{Name: adb.Name, Namespace: adb.Namespace} - It("Should check for TERMINATED state in local resource", e2ebehavior.AssertLocalState(&k8sClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateTerminated)) - -<<<<<<< HEAD - It("Should delete local resource", e2ebehavior.AssertSoftLinkDelete(&k8sClient, &adbLookupKey)) + Expect(k8sClient.Create(context.TODO(), adb)).Should(Succeed()) }) - // Describe("Test ADB status", func() { - // var dbName string - // var backupID string - - // It("Should init the test", func() { - // By("creating a temp ADB in OCI for binding test") - // dbName = e2eutil.GenerateDBName() - // createResp, err := e2eutil.CreateAutonomousDatabase(dbClient, &SharedCompartmentOCID, &dbName, &SharedPlainTextAdminPassword) - // Expect(err).ShouldNot(HaveOccurred()) - // Expect(createResp.AutonomousDatabase.Id).ShouldNot(BeNil()) - - // By("Save the database ID for later use") - // adbID = createResp.AutonomousDatabase.Id - // backupID = *adbID - - // By("Wait until the work request is in SUCCEEDED status") - // workClient, err := workrequests.NewWorkRequestClientWithConfigurationProvider(configProvider) - // Expect(err).ShouldNot(HaveOccurred()) - - // err = e2eutil.WaitUntilWorkCompleted(workClient, createResp.OpcWorkRequestId) - // Expect(err).ShouldNot(HaveOccurred()) - // }) - - // It("Should create a AutonomousDatabase resource", func() { - // adb := &dbv1alpha1.AutonomousDatabase{ - // TypeMeta: metav1.TypeMeta{ - // APIVersion: "database.oracle.com/v1alpha1", - // Kind: "AutonomousDatabase", - // }, - // ObjectMeta: metav1.ObjectMeta{ - // Name: "bindadb", - // Namespace: ADBNamespace, - // }, - // Spec: dbv1alpha1.AutonomousDatabaseSpec{ - // Details: dbv1alpha1.AutonomousDatabaseDetails{ - // AutonomousDatabaseOCID: adbID, - // }, - // HardLink: common.Bool(true), - // OCIConfig: dbv1alpha1.OCIConfigSpec{ - // ConfigMapName: common.String(SharedOCIConfigMapName), - // SecretName: common.String(SharedOCISecretName), - // }, - // }, - // } - - // adbLookupKey = types.NamespacedName{Name: adb.Name, Namespace: adb.Namespace} - - // Expect(k8sClient.Create(context.TODO(), adb)).Should(Succeed()) - // }) - - // It("should bind to an ADB", e2ebehavior.AssertBind(&k8sClient, &adbLookupKey)) - - // It("should terminate ADB in a different routine", func() { - // err := e2eutil.DeleteAutonomousDatabase(dbClient, adbID) - // Expect(err).ToNot(HaveOccurred()) - // // By("Wait until the work request is in SUCCEEDED status") - // // workClient, err := workrequests.NewWorkRequestClientWithConfigurationProvider(configProvider) - // // Expect(err).ShouldNot(HaveOccurred()) - // // err = e2eutil.WaitUntilWorkCompleted(workClient, createResp.OpcWorkRequestId) - // // Expect(err).ShouldNot(HaveOccurred()) - // }) - - // It("should check for terminated state in OCI", e2ebehavior.AssertRemoteStateOCID(&k8sClient, &dbClient, &backupID, database.AutonomousDatabaseLifecycleStateTerminated)) - - // It("should check for terminated state in local resource", e2ebehavior.AssertLocalState(&k8sClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateTerminated)) - // }) - }) -} -======= + It("Should check for TERMINATED state in local resource", e2ebehavior.AssertLocalState(&k8sClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateTerminated)) + It("Should delete local resource", e2ebehavior.AssertSoftLinkDelete(&k8sClient, &adbLookupKey)) }) }) ->>>>>>> feff58aab50838a39c03d8e22fdede11da02aa14 diff --git a/test/e2e/suite_test.go b/test/e2e/suite_test.go index 5b3dcb0b..6a2cfd81 100644 --- a/test/e2e/suite_test.go +++ b/test/e2e/suite_test.go @@ -39,7 +39,6 @@ package e2etest import ( - "os" "context" "fmt" "path/filepath" @@ -95,8 +94,26 @@ var ( var cfg *rest.Config var k8sClient client.Client +var configProvider common.ConfigurationProvider +var dbClient database.DatabaseClient var testEnv *envtest.Environment +const configFileName = "test_config.yaml" +const ADBNamespace string = "default" + +var SharedOCIConfigMapName = "oci-cred" +var SharedOCISecretName = "oci-privatekey" +var SharedPlainTextAdminPassword = "Welcome_1234" +var SharedPlainTextWalletPassword = "Welcome_1234" +var SharedCompartmentOCID string + +var SharedKeyOCID string +var SharedAdminPasswordOCID string +var SharedInstanceWalletPasswordOCID string + +const SharedAdminPassSecretName string = "adb-admin-password" +const SharedWalletPassSecretName = "adb-wallet-password" + func TestAPIs(t *testing.T) { gomega.RegisterFailHandler(ginkgo.Fail) @@ -109,16 +126,8 @@ var _ = BeforeSuite(func(done ginkgo.Done) { logf.SetLogger(zap.New(zap.WriteTo(ginkgo.GinkgoWriter), zap.UseDevMode(true))) By("bootstrapping test environment") - t := true - - if os.Getenv("TEST_USE_EXISTING_CLUSTER") == "true" { - testEnv = &envtest.Environment{ - UseExistingCluster: &t, - } - } else { - testEnv = &envtest.Environment{ - CRDDirectoryPaths: []string{filepath.Join("../..", "config", "crd", "bases")}, - } + testEnv = &envtest.Environment{ + CRDDirectoryPaths: []string{filepath.Join("../..", "config", "crd", "bases")}, } var err error @@ -150,17 +159,6 @@ var _ = BeforeSuite(func(done ginkgo.Done) { }).SetupWithManager(k8sManager) Expect(err).ToNot(HaveOccurred()) - // CDB Reconciler - err = (&controllers.CDBReconciler{ - Client: k8sManager.GetClient(), - Scheme: k8sManager.GetScheme(), - Config: k8sManager.GetConfig(), - Log: ctrl.Log.WithName("controllers").WithName("CDB_test"), - Interval: time.Duration(15), - Recorder: k8sManager.GetEventRecorderFor("CDB"), - }).SetupWithManager(k8sManager) - Expect(err).ToNot(HaveOccurred()) - go func() { defer ginkgo.GinkgoRecover() err = k8sManager.Start(ctrl.SetupSignalHandler()) @@ -175,25 +173,77 @@ var _ = BeforeSuite(func(done ginkgo.Done) { }() /************************************************** - * Custom codes for autonomousdatabase controller - **************************************************/ - //ADBSetup(k8sClient) + * Custom codes for autonomousdatabase controller + **************************************************/ + By("init the test by creating utililty variables and objects") + testConfig, err := e2eutil.GetTestConfig(configFileName) + Expect(err).ToNot(HaveOccurred()) + Expect(testConfig).ToNot(BeNil()) + + SharedCompartmentOCID = testConfig.CompartmentOCID + SharedAdminPasswordOCID = testConfig.AdminPasswordOCID + SharedInstanceWalletPasswordOCID = testConfig.InstanceWalletPasswordOCID + + By("checking if the required parameters exist") + Expect(testConfig.OCIConfigFile).ToNot(Equal("")) + Expect(testConfig.CompartmentOCID).ToNot(Equal("")) + Expect(testConfig.AdminPasswordOCID).ToNot(Equal("")) + Expect(testConfig.InstanceWalletPasswordOCID).ToNot(Equal("")) + + By("getting OCI provider") + ociConfigUtil, err := e2eutil.GetOCIConfigUtil(testConfig.OCIConfigFile, testConfig.Profile) + Expect(err).ToNot(HaveOccurred()) + configProvider, err = ociConfigUtil.GetConfigProvider() + Expect(err).ToNot(HaveOccurred()) + + By("creating a OCI DB client") + dbClient, err = database.NewDatabaseClientWithConfigurationProvider(configProvider) + Expect(err).ToNot(HaveOccurred()) + + By("creating a configMap for calling OCI") + ociConfigMap, err := ociConfigUtil.CreateOCIConfigMap(ADBNamespace, SharedOCIConfigMapName) + Expect(err).ToNot(HaveOccurred()) + Expect(k8sClient.Create(context.TODO(), ociConfigMap)).To(Succeed()) + + By("creating a secret for calling OCI") + ociSecret, err := ociConfigUtil.CreateOCISecret(ADBNamespace, SharedOCISecretName) + Expect(err).ToNot(HaveOccurred()) + Expect(k8sClient.Create(context.TODO(), ociSecret)).To(Succeed()) + + By("Creating a k8s secret to hold admin password", func() { + data := map[string]string{ + SharedAdminPassSecretName: SharedPlainTextAdminPassword, + } + adminSecret, err := e2eutil.CreateKubeSecret(ADBNamespace, SharedAdminPassSecretName, data) + Expect(err).ToNot(HaveOccurred()) + Expect(k8sClient.Create(context.TODO(), adminSecret)).To(Succeed()) + }) + + By("Creating a k8s secret to hold wallet password", func() { + data := map[string]string{ + SharedWalletPassSecretName: SharedPlainTextWalletPassword, + } + walletSecret, err := e2eutil.CreateKubeSecret(ADBNamespace, SharedWalletPassSecretName, data) + Expect(err).ToNot(HaveOccurred()) + Expect(k8sClient.Create(context.TODO(), walletSecret)).To(Succeed()) + }) + close(done) }, 60) var _ = AfterSuite(func() { /* - From Kubernetes 1.21+, when it tries to cleanup the test environment, there is - a clash if a custom controller is created during testing. It would seem that - the controller is still running and kube-apiserver will not respond to shutdown. - This is the reason why teardown happens in BeforeSuite() after controller has stopped. - The error shown is as documented in: - https://github.com/kubernetes-sigs/controller-runtime/issues/1571 - /* - /* - By("tearing down the test environment") - err := testEnv.Stop() - Expect(err).ToNot(HaveOccurred()) + From Kubernetes 1.21+, when it tries to cleanup the test environment, there is + a clash if a custom controller is created during testing. It would seem that + the controller is still running and kube-apiserver will not respond to shutdown. + This is the reason why teardown happens in BeforeSuite() after controller has stopped. + The error shown is as documented in: + https://github.com/kubernetes-sigs/controller-runtime/issues/1571 + /* + /* + By("tearing down the test environment") + err := testEnv.Stop() + Expect(err).ToNot(HaveOccurred()) */ By("Delete the resources that are created during the tests") From 96e5ec450245671fe5757f278c8b615ad15c47f2 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Wed, 9 Feb 2022 15:41:38 -0500 Subject: [PATCH 150/628] update the versions of the dependencies --- go.mod | 60 ++++++++++---------- go.sum | 172 ++++++++++++++++++++++++++++++++++++++++++++++----------- 2 files changed, 169 insertions(+), 63 deletions(-) diff --git a/go.mod b/go.mod index 812af66f..ac3012cd 100644 --- a/go.mod +++ b/go.mod @@ -5,30 +5,30 @@ go 1.17 require ( github.com/go-logr/logr v1.2.2 github.com/onsi/ginkgo v1.16.5 - github.com/onsi/gomega v1.17.0 + github.com/onsi/gomega v1.18.1 github.com/oracle/oci-go-sdk/v54 v54.0.0 gopkg.in/yaml.v2 v2.4.0 - k8s.io/api v0.23.1 - k8s.io/apimachinery v0.23.1 - k8s.io/client-go v0.23.1 + k8s.io/api v0.23.3 + k8s.io/apimachinery v0.23.3 + k8s.io/client-go v0.23.3 sigs.k8s.io/controller-runtime v0.11.0 sigs.k8s.io/yaml v1.3.0 ) require ( - cloud.google.com/go v0.81.0 // indirect + cloud.google.com/go/compute v1.2.0 // indirect github.com/beorn7/perks v1.0.1 // indirect - github.com/cespare/xxhash/v2 v2.1.1 // indirect + github.com/cespare/xxhash/v2 v2.1.2 // indirect github.com/davecgh/go-spew v1.1.1 // indirect - github.com/evanphx/json-patch v4.12.0+incompatible // indirect + github.com/evanphx/json-patch v5.6.0+incompatible // indirect github.com/fsnotify/fsnotify v1.5.1 // indirect - github.com/go-logr/zapr v1.2.0 // indirect + github.com/go-logr/zapr v1.2.2 // indirect github.com/gogo/protobuf v1.3.2 // indirect github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect github.com/golang/protobuf v1.5.2 // indirect - github.com/google/go-cmp v0.5.5 // indirect - github.com/google/gofuzz v1.1.0 // indirect - github.com/google/uuid v1.1.2 // indirect + github.com/google/go-cmp v0.5.7 // indirect + github.com/google/gofuzz v1.2.0 // indirect + github.com/google/uuid v1.3.0 // indirect github.com/googleapis/gnostic v0.5.5 // indirect github.com/imdario/mergo v0.3.12 // indirect github.com/json-iterator/go v1.1.12 // indirect @@ -38,32 +38,32 @@ require ( github.com/modern-go/reflect2 v1.0.2 // indirect github.com/nxadm/tail v1.4.8 // indirect github.com/pkg/errors v0.9.1 // indirect - github.com/prometheus/client_golang v1.11.0 // indirect + github.com/prometheus/client_golang v1.12.1 // indirect github.com/prometheus/client_model v0.2.0 // indirect - github.com/prometheus/common v0.28.0 // indirect - github.com/prometheus/procfs v0.6.0 // indirect - github.com/sony/gobreaker v0.4.2-0.20210216022020-dd874f9dd33b // indirect + github.com/prometheus/common v0.32.1 // indirect + github.com/prometheus/procfs v0.7.3 // indirect + github.com/sony/gobreaker v0.5.0 // indirect github.com/spf13/pflag v1.0.5 // indirect - go.uber.org/atomic v1.7.0 // indirect - go.uber.org/multierr v1.6.0 // indirect - go.uber.org/zap v1.19.1 // indirect - golang.org/x/net v0.0.0-20211209124913-491a49abca63 // indirect - golang.org/x/oauth2 v0.0.0-20210819190943-2bc19b11175f // indirect - golang.org/x/sys v0.0.0-20211029165221-6e7872819dc8 // indirect - golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b // indirect + go.uber.org/atomic v1.9.0 // indirect + go.uber.org/multierr v1.7.0 // indirect + go.uber.org/zap v1.21.0 // indirect + golang.org/x/net v0.0.0-20220127200216-cd36cc0744dd // indirect + golang.org/x/oauth2 v0.0.0-20211104180415-d3ed0bb246c8 // indirect + golang.org/x/sys v0.0.0-20220207234003-57398862261d // indirect + golang.org/x/term v0.0.0-20210927222741-03fcf44c2211 // indirect golang.org/x/text v0.3.7 // indirect - golang.org/x/time v0.0.0-20210723032227-1f47c861a9ac // indirect + golang.org/x/time v0.0.0-20211116232009-f0f3c7e86c11 // indirect gomodules.xyz/jsonpatch/v2 v2.2.0 // indirect google.golang.org/appengine v1.6.7 // indirect google.golang.org/protobuf v1.27.1 // indirect gopkg.in/inf.v0 v0.9.1 // indirect gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 // indirect gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b // indirect - k8s.io/apiextensions-apiserver v0.23.0 // indirect - k8s.io/component-base v0.23.0 // indirect - k8s.io/klog/v2 v2.30.0 // indirect - k8s.io/kube-openapi v0.0.0-20211115234752-e816edb12b65 // indirect - k8s.io/utils v0.0.0-20210930125809-cb0fa318a74b // indirect - sigs.k8s.io/json v0.0.0-20211020170558-c049b76a60c6 // indirect - sigs.k8s.io/structured-merge-diff/v4 v4.2.0 // indirect + k8s.io/apiextensions-apiserver v0.23.3 // indirect + k8s.io/component-base v0.23.3 // indirect + k8s.io/klog/v2 v2.40.1 // indirect + k8s.io/kube-openapi v0.0.0-20220124234850-424119656bbf // indirect + k8s.io/utils v0.0.0-20220127004650-9b3446523e65 // indirect + sigs.k8s.io/json v0.0.0-20211208200746-9f7c6b3444d2 // indirect + sigs.k8s.io/structured-merge-diff/v4 v4.2.1 // indirect ) diff --git a/go.sum b/go.sum index 42369280..0046074c 100644 --- a/go.sum +++ b/go.sum @@ -17,14 +17,26 @@ cloud.google.com/go v0.72.0/go.mod h1:M+5Vjvlc2wnp6tjzE102Dw08nGShTscUx2nZMufOKP cloud.google.com/go v0.74.0/go.mod h1:VV1xSbzvo+9QJOxLDaJfTjx5e+MePCpCWwvftOeQmWk= cloud.google.com/go v0.78.0/go.mod h1:QjdrLG0uq+YwhjoVOLsS1t7TW8fs36kLs4XO5R5ECHg= cloud.google.com/go v0.79.0/go.mod h1:3bzgcEeQlzbuEAYu4mrWhKqWjmpprinYgKJLgKHnbb8= -cloud.google.com/go v0.81.0 h1:at8Tk2zUz63cLPR0JPWm5vp77pEZmzxEQBEfRKn1VV8= cloud.google.com/go v0.81.0/go.mod h1:mk/AM35KwGk/Nm2YSeZbxXdrNK3KZOYHmLkOqC2V6E0= +cloud.google.com/go v0.83.0/go.mod h1:Z7MJUsANfY0pYPdw0lbnivPx4/vhy/e2FEkSkF7vAVY= +cloud.google.com/go v0.84.0/go.mod h1:RazrYuxIK6Kb7YrzzhPoLmCVzl7Sup4NrbKPg8KHSUM= +cloud.google.com/go v0.87.0/go.mod h1:TpDYlFy7vuLzZMMZ+B6iRiELaY7z/gJPaqbMx6mlWcY= +cloud.google.com/go v0.90.0/go.mod h1:kRX0mNRHe0e2rC6oNakvwQqzyDmg57xJ+SZU1eT2aDQ= +cloud.google.com/go v0.93.3/go.mod h1:8utlLll2EF5XMAV15woO4lSbWQlk8rer9aLOfLh7+YI= +cloud.google.com/go v0.94.1/go.mod h1:qAlAugsXlC+JWO+Bke5vCtc9ONxjQT3drlTTnAplMW4= +cloud.google.com/go v0.97.0/go.mod h1:GF7l59pYBVlXQIBLx3a761cZ41F9bBH3JUlihCt2Udc= +cloud.google.com/go v0.99.0/go.mod h1:w0Xx2nLzqWJPuozYQX+hFfCSI8WioryfRDzkoI/Y2ZA= +cloud.google.com/go v0.100.2 h1:t9Iw5QH5v4XtlEQaCtUY7x6sCABps8sW0acw7e2WQ6Y= +cloud.google.com/go v0.100.2/go.mod h1:4Xra9TjzAeYHrl5+oeLlzbM2k3mjVhZh4UqTZ//w99A= cloud.google.com/go/bigquery v1.0.1/go.mod h1:i/xbL2UlR5RvWAURpBYZTtm/cXjCha9lbfbpx4poX+o= cloud.google.com/go/bigquery v1.3.0/go.mod h1:PjpwJnslEMmckchkHFfq+HTD2DmtT67aNFKH1/VBDHE= cloud.google.com/go/bigquery v1.4.0/go.mod h1:S8dzgnTigyfTmLBfrtrhyYhwRxG72rYxvftPBK2Dvzc= cloud.google.com/go/bigquery v1.5.0/go.mod h1:snEHRnqQbz117VIFhE8bmtwIDY80NLUZUMb4Nv6dBIg= cloud.google.com/go/bigquery v1.7.0/go.mod h1://okPTzCYNXSlb24MZs83e2Do+h+VXtc4gLoIoXIAPc= cloud.google.com/go/bigquery v1.8.0/go.mod h1:J5hqkt3O0uAFnINi6JXValWIb1v0goeZM77hZzJN/fQ= +cloud.google.com/go/compute v0.1.0/go.mod h1:GAesmwr110a34z04OlxYkATPBEfVhkymfTBXtfbBFow= +cloud.google.com/go/compute v1.2.0 h1:EKki8sSdvDU0OO9mAXGwPXOTOgPz2l08R0/IutDH11I= +cloud.google.com/go/compute v1.2.0/go.mod h1:xlogom/6gr8RJGBe7nT2eGsQYAFUbbv8dbC29qE3Xmw= cloud.google.com/go/datastore v1.0.0/go.mod h1:LXYbyblFSglQ5pkeyhO+Qmw7ukd3C+pD7TKLgZqpHYE= cloud.google.com/go/datastore v1.1.0/go.mod h1:umbIZjpQpHh4hmRpGhH4tLFup+FVzqBi1b3c64qFpCk= cloud.google.com/go/firestore v1.1.0/go.mod h1:ulACoGHTpvq5r8rxGJ4ddJZBZqakUQqClKRT5SZwBmk= @@ -81,8 +93,9 @@ github.com/certifi/gocertifi v0.0.0-20191021191039-0944d244cd40/go.mod h1:sGbDF6 github.com/certifi/gocertifi v0.0.0-20200922220541-2c3bb06c6054/go.mod h1:sGbDF6GwGcLpkNXPUTkMRoywsNa/ol15pxFe6ERfguA= github.com/cespare/xxhash v1.1.0 h1:a6HrQnmkObjyL+Gs60czilIUGqrzKutQD6XZog3p+ko= github.com/cespare/xxhash v1.1.0/go.mod h1:XrSqR1VqqWfGrhpAt58auRo0WTKS1nRRg3ghfAqPWnc= -github.com/cespare/xxhash/v2 v2.1.1 h1:6MnRN8NT7+YBpUIWxHtefFZOKTAPgGjpQSxqLNn0+qY= github.com/cespare/xxhash/v2 v2.1.1/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= +github.com/cespare/xxhash/v2 v2.1.2 h1:YRXhKfTDauu4ajMg1TPgFO5jnlC2HCbmLXMcTG5cbYE= +github.com/cespare/xxhash/v2 v2.1.2/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI= github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI= github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU= @@ -124,8 +137,9 @@ github.com/envoyproxy/go-control-plane v0.9.9-0.20210217033140-668b12f5399d/go.m github.com/envoyproxy/go-control-plane v0.9.9-0.20210512163311-63b5d3c536b0/go.mod h1:hliV/p42l8fGbc6Y9bQ70uLwIvmJyVE5k4iMKlh8wCQ= github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c= github.com/evanphx/json-patch v0.5.2/go.mod h1:ZWS5hhDbVDyob71nXKNL0+PWn6ToqBHMikGIFbs31qQ= -github.com/evanphx/json-patch v4.12.0+incompatible h1:4onqiflcdA9EOZ4RxV643DvftH5pOlLGNtQ5lPWQu84= github.com/evanphx/json-patch v4.12.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk= +github.com/evanphx/json-patch v5.6.0+incompatible h1:jBYDEEiFBPxA0v50tFdvOzQQTCvpL6mnFh5mB2/l16U= +github.com/evanphx/json-patch v5.6.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk= github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4= github.com/felixge/httpsnoop v1.0.1/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U= github.com/form3tech-oss/jwt-go v3.2.2+incompatible/go.mod h1:pbq4aXjuKjdthFRnoDwaVPLA+WlJuPGy+QneDUgJi2k= @@ -151,8 +165,9 @@ github.com/go-logr/logr v0.2.0/go.mod h1:z6/tIYblkpsD+a4lm/fGIIU9mZ+XfAiaFtq7xTg github.com/go-logr/logr v1.2.0/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A= github.com/go-logr/logr v1.2.2 h1:ahHml/yUpnlb96Rp8HCvtYVPY8ZYpxq3g7UYchIYwbs= github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A= -github.com/go-logr/zapr v1.2.0 h1:n4JnPI1T3Qq1SFEi/F8rwLrZERp2bso19PJZDB9dayk= github.com/go-logr/zapr v1.2.0/go.mod h1:Qa4Bsj2Vb+FAVeAKsLD8RLQ+YRJB8YDmOAKxaBQf7Ro= +github.com/go-logr/zapr v1.2.2 h1:5YNlIL6oZLydaV4dOFjL8YpgXF/tPeTbnpatnu3cq6o= +github.com/go-logr/zapr v1.2.2/go.mod h1:eIauM6P8qSvTw5o2ez6UEAfGjQKrxQTl5EoK+Qa2oG4= github.com/go-openapi/jsonpointer v0.19.3/go.mod h1:Pl9vOtqEWErmShwVjC8pYs9cog34VGT37dQOVbmoatg= github.com/go-openapi/jsonpointer v0.19.5/go.mod h1:Pl9vOtqEWErmShwVjC8pYs9cog34VGT37dQOVbmoatg= github.com/go-openapi/jsonreference v0.19.3/go.mod h1:rjx6GuL8TTa9VaixXglHmQmIL98+wF9xc8zWvFonSJ8= @@ -183,6 +198,7 @@ github.com/golang/mock v1.4.1/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt github.com/golang/mock v1.4.3/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw= github.com/golang/mock v1.4.4/go.mod h1:l3mdAwkq5BuhzHwde/uurv3sEJeZMXNpwsxVWU71h+4= github.com/golang/mock v1.5.0/go.mod h1:CWnOUgYIOo4TcNZ0wHX3YZCqsaM1I1Jvs6v3mP3KVu8= +github.com/golang/mock v1.6.0/go.mod h1:p6yTPP+5HYm5mzsMV8JkE6ZKdX+/wYM6Hr+LicevLPs= github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= @@ -201,6 +217,7 @@ github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaS github.com/golang/protobuf v1.5.1/go.mod h1:DopwsBzvsk0Fs44TXzsVbJyPhcCPeIwnvohx4u74HPM= github.com/golang/protobuf v1.5.2 h1:ROPKBNFfQgOUMifHyP+KYbvpjbdoFNs+aK7DXlji0Tw= github.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY= +github.com/golang/snappy v0.0.3/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q= github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= github.com/google/btree v1.0.1/go.mod h1:xXMiIv4Fb/0kKde4SpL7qlzvu5cMJDRkFDxJfI9uaxA= @@ -216,14 +233,18 @@ github.com/google/go-cmp v0.5.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/ github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.3/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= -github.com/google/go-cmp v0.5.5 h1:Khx7svrCpmxxtHBq5j2mp/xVjsi8hQMfNLvJFAlrGgU= github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= +github.com/google/go-cmp v0.5.6/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= +github.com/google/go-cmp v0.5.7 h1:81/ik6ipDQS2aGcBfIN5dHDB36BwrStyeAQquSYCV4o= +github.com/google/go-cmp v0.5.7/go.mod h1:n+brtR0CgQNWTVd5ZUFpTBC8YFBDLK/h/bpaJ8/DtOE= github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= -github.com/google/gofuzz v1.1.0 h1:Hsa8mG0dQ46ij8Sl2AYJDUv1oA9/d6Vk+3LG99Oe02g= github.com/google/gofuzz v1.1.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= +github.com/google/gofuzz v1.2.0 h1:xRy4A+RhZaiKjJ1bPfwQ8sedCA+YS2YcCHW6ec7JMi0= +github.com/google/gofuzz v1.2.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs= github.com/google/martian/v3 v3.0.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0= github.com/google/martian/v3 v3.1.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0= +github.com/google/martian/v3 v3.2.1/go.mod h1:oBOf6HBosgwRXnUGWUB05QECsc6uvmMiJ3+6W4l/CUk= github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc= github.com/google/pprof v0.0.0-20190515194954-54271f7e092f/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc= github.com/google/pprof v0.0.0-20191218002539-d4f498aebedc/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= @@ -235,11 +256,18 @@ github.com/google/pprof v0.0.0-20201023163331-3e6fc7fc9c4c/go.mod h1:kpwsk12EmLe github.com/google/pprof v0.0.0-20201203190320-1bf35d6f28c2/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= github.com/google/pprof v0.0.0-20210122040257-d980be63207e/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= github.com/google/pprof v0.0.0-20210226084205-cbba55b83ad5/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= +github.com/google/pprof v0.0.0-20210407192527-94a9f03dee38/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= +github.com/google/pprof v0.0.0-20210601050228-01bbb1931b22/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= +github.com/google/pprof v0.0.0-20210609004039-a478d1d731e9/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= +github.com/google/pprof v0.0.0-20210720184732-4bb14d4b1be1/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI= -github.com/google/uuid v1.1.2 h1:EVhdT+1Kseyi1/pUmXKaFxYsDNy9RQYkMWRH68J/W7Y= github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= +github.com/google/uuid v1.3.0 h1:t6JiXgmwXMjEs8VusXIJk2BXHsn+wx8BZdTaoZ5fu7I= +github.com/google/uuid v1.3.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= github.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg= github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk= +github.com/googleapis/gax-go/v2 v2.1.0/go.mod h1:Q3nei7sK6ybPYH7twZdmQpAd1MKb7pfu6SK+H1/DsU0= +github.com/googleapis/gax-go/v2 v2.1.1/go.mod h1:hddJymUZASv3XPyGkUpKj8pPO47Rmb0eJc8R6ouapiM= github.com/googleapis/gnostic v0.5.1/go.mod h1:6U4PtQXGIEt/Z3h5MAT7FNofLnw9vXk2cUuW7uA/OeU= github.com/googleapis/gnostic v0.5.5 h1:9fHAtK0uDfpveeqqo1hkEZJcFvYXAiCN3UutL8F9xHw= github.com/googleapis/gnostic v0.5.5/go.mod h1:7+EbHbldMins07ALC74bsA81Ovc97DwqyJO1AENw9kA= @@ -356,11 +384,14 @@ github.com/onsi/ginkgo v1.14.0/go.mod h1:iSB4RoI2tjJc9BBv4NKIKWKya62Rps+oPG/Lv9k github.com/onsi/ginkgo v1.16.4/go.mod h1:dX+/inL/fNMqNlz0e9LfyB9TswhZpCVdJM/Z6Vvnwo0= github.com/onsi/ginkgo v1.16.5 h1:8xi0RTUf59SOSfEtZMvwTvXYMzG4gV23XVHOZiXNtnE= github.com/onsi/ginkgo v1.16.5/go.mod h1:+E8gABHa3K6zRBolWtd+ROzc/U5bkGt0FwiG042wbpU= +github.com/onsi/ginkgo/v2 v2.0.0 h1:CcuG/HvWNkkaqCUpJifQY8z7qEMBJya6aLPx6ftGyjQ= +github.com/onsi/ginkgo/v2 v2.0.0/go.mod h1:vw5CSIxN1JObi/U8gcbwft7ZxR2dgaR70JSE3/PpL4c= github.com/onsi/gomega v0.0.0-20170829124025-dcabb60a477c/go.mod h1:C1qb7wdrVGGVU+Z6iS04AVkA3Q65CEZX59MT0QO5uiA= github.com/onsi/gomega v1.7.1/go.mod h1:XdKZgCCFLUoM/7CFJVPcG8C1xQ1AJ0vpAezJrB7JYyY= github.com/onsi/gomega v1.10.1/go.mod h1:iN09h71vgCQne3DLsj+A5owkum+a2tYe+TOCB1ybHNo= -github.com/onsi/gomega v1.17.0 h1:9Luw4uT5HTjHTN8+aNcSThgH1vdXnmdJ8xIfZ4wyTRE= github.com/onsi/gomega v1.17.0/go.mod h1:HnhC7FXeEQY45zxNK3PPoIUhzk/80Xly9PcubAlGdZY= +github.com/onsi/gomega v1.18.1 h1:M1GfJqGRrBrrGGsbxzV5dqM2U2ApXefZCQpkukxYRLE= +github.com/onsi/gomega v1.18.1/go.mod h1:0q+aL8jAiMXy9hbwj2mr5GziHiwhAIQpFmmtT5hitRs= github.com/opentracing/opentracing-go v1.1.0/go.mod h1:UkNAQd3GIcIGf0SeVgPpRdFStlNbqXla1AfSYxPUl2o= github.com/oracle/oci-go-sdk/v54 v54.0.0 h1:CDLjeSejv2aDpElAJrhKpi6zvT/zhZCZuXchUUZ+LS4= github.com/oracle/oci-go-sdk/v54 v54.0.0/go.mod h1:+t+yvcFGVp+3ZnztnyxqXfQDsMlq8U25faBLa+mqCMc= @@ -381,8 +412,9 @@ github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXP github.com/prometheus/client_golang v0.9.3/go.mod h1:/TN21ttK/J9q6uSwhBd54HahCDft0ttaMvbicHlPoso= github.com/prometheus/client_golang v1.0.0/go.mod h1:db9x61etRT2tGnBNRi70OPL5FsnadC4Ky3P0J6CfImo= github.com/prometheus/client_golang v1.7.1/go.mod h1:PY5Wy2awLA44sXw4AOSfFBetzPP4j5+D6mVACh+pe2M= -github.com/prometheus/client_golang v1.11.0 h1:HNkLOAEQMIDv/K+04rukrLx6ch7msSRwf3/SASFAGtQ= github.com/prometheus/client_golang v1.11.0/go.mod h1:Z6t4BnS23TR94PD6BsDNk8yVqroYurpAkEiz0P2BEV0= +github.com/prometheus/client_golang v1.12.1 h1:ZiaPsmm9uiBeaSMRznKsCDNtPCS0T3JVDGF+06gjBzk= +github.com/prometheus/client_golang v1.12.1/go.mod h1:3Z9XVyYiZYEO+YQWt3RD2R3jrbd179Rt297l4aS6nDY= github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo= github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= @@ -393,14 +425,16 @@ github.com/prometheus/common v0.4.0/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y8 github.com/prometheus/common v0.4.1/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4= github.com/prometheus/common v0.10.0/go.mod h1:Tlit/dnDKsSWFlCLTWaA1cyBgKHSMdTB80sz/V91rCo= github.com/prometheus/common v0.26.0/go.mod h1:M7rCNAaPfAosfx8veZJCuw84e35h3Cfd9VFqTh1DIvc= -github.com/prometheus/common v0.28.0 h1:vGVfV9KrDTvWt5boZO0I19g2E3CsWfpPPKZM9dt3mEw= github.com/prometheus/common v0.28.0/go.mod h1:vu+V0TpY+O6vW9J44gczi3Ap/oXXR10b+M/gUGO4Hls= +github.com/prometheus/common v0.32.1 h1:hWIdL3N2HoUx3B8j3YN9mWor0qhY/NlEKZEaXxuIRh4= +github.com/prometheus/common v0.32.1/go.mod h1:vu+V0TpY+O6vW9J44gczi3Ap/oXXR10b+M/gUGO4Hls= github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk= github.com/prometheus/procfs v0.0.0-20190507164030-5867b95ac084/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA= github.com/prometheus/procfs v0.0.2/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA= github.com/prometheus/procfs v0.1.3/go.mod h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4OA4YeYWdaU= -github.com/prometheus/procfs v0.6.0 h1:mxy4L2jP6qMonqmq+aTtOx1ifVWUgG/TAmntgbh3xv4= github.com/prometheus/procfs v0.6.0/go.mod h1:cz+aTbrPOrUb4q7XlbU9ygM+/jj0fzG6c1xBZuNvfVA= +github.com/prometheus/procfs v0.7.3 h1:4jVXhlkAyzOScmCkXBTOLRLTz8EeU+eyjrwB/EPq0VU= +github.com/prometheus/procfs v0.7.3/go.mod h1:cz+aTbrPOrUb4q7XlbU9ygM+/jj0fzG6c1xBZuNvfVA= github.com/prometheus/tsdb v0.7.1/go.mod h1:qhTCs0VvXwvX/y3TZrWD7rabWM+ijKTux40TwIPHuXU= github.com/rogpeppe/fastuuid v0.0.0-20150106093220-6724a57986af/go.mod h1:XWv6SoW27p1b0cqNHllgS5HIMJraePCO15w5zCzIWYg= github.com/rogpeppe/fastuuid v1.2.0/go.mod h1:jVj6XXZzXRy/MSR5jhDC/2q6DgLz+nrA6LYCDYWNEvQ= @@ -418,8 +452,9 @@ github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1 github.com/smartystreets/goconvey v1.6.4/go.mod h1:syvi0/a8iFYH4r/RixwvyeAJjdLS9QV7WQ/tjFTllLA= github.com/soheilhy/cmux v0.1.4/go.mod h1:IM3LyeVVIOuxMH7sFAkER9+bJ4dT7Ms6E4xg4kGIyLM= github.com/soheilhy/cmux v0.1.5/go.mod h1:T7TcVDs9LWfQgPlPsdngu6I6QIoyIFZDDC6sNE1GqG0= -github.com/sony/gobreaker v0.4.2-0.20210216022020-dd874f9dd33b h1:br+bPNZsJWKicw/5rALEo67QHs5weyD5tf8WST+4sJ0= github.com/sony/gobreaker v0.4.2-0.20210216022020-dd874f9dd33b/go.mod h1:ZKptC7FHNvhBz7dN2LGjPVBz2sZJmc0/PkyDJOjmxWY= +github.com/sony/gobreaker v0.5.0 h1:dRCvqm0P490vZPmy7ppEk2qCnCieBooFJ+YoXGYB+yg= +github.com/sony/gobreaker v0.5.0/go.mod h1:ZKptC7FHNvhBz7dN2LGjPVBz2sZJmc0/PkyDJOjmxWY= github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA= github.com/spf13/afero v1.1.2/go.mod h1:j4pytiNVoe2o6bmDsKpLACNPDBIoEAkihy7loJ1B0CQ= github.com/spf13/afero v1.2.2/go.mod h1:9ZxEEn6pIJ8Rxe320qSDBk6AsU0r9pR7Q4OcevTdifk= @@ -485,20 +520,24 @@ go.opentelemetry.io/otel/sdk/metric v0.20.0/go.mod h1:knxiS8Xd4E/N+ZqKmUPf3gTTZ4 go.opentelemetry.io/otel/trace v0.20.0/go.mod h1:6GjCW8zgDjwGHGa6GkyeB8+/5vjT16gUEi0Nf1iBdgw= go.opentelemetry.io/proto/otlp v0.7.0/go.mod h1:PqfVotwruBrMGOCsRd/89rSnXhoiJIqeYNgFYFoEGnI= go.uber.org/atomic v1.4.0/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE= -go.uber.org/atomic v1.7.0 h1:ADUqmZGgLDDfbSL9ZmPxKTybcoEYHgpYfELNoN+7hsw= go.uber.org/atomic v1.7.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc= +go.uber.org/atomic v1.9.0 h1:ECmE8Bn/WFTYwEW/bpKD3M8VtR/zQVbavAoalC1PYyE= +go.uber.org/atomic v1.9.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc= go.uber.org/goleak v1.1.10/go.mod h1:8a7PlsEVH3e/a/GLqe5IIrQx6GzcnRmZEufDUTk4A7A= go.uber.org/goleak v1.1.11-0.20210813005559-691160354723/go.mod h1:cwTWslyiVhfpKIDGSZEM2HlOvcqm+tG4zioyIeLoqMQ= +go.uber.org/goleak v1.1.11/go.mod h1:cwTWslyiVhfpKIDGSZEM2HlOvcqm+tG4zioyIeLoqMQ= go.uber.org/goleak v1.1.12 h1:gZAh5/EyT/HQwlpkCy6wTpqfH9H8Lz8zbm3dZh+OyzA= go.uber.org/goleak v1.1.12/go.mod h1:cwTWslyiVhfpKIDGSZEM2HlOvcqm+tG4zioyIeLoqMQ= go.uber.org/multierr v1.1.0/go.mod h1:wR5kodmAFQ0UK8QlbwjlSNy0Z68gJhDJUG5sjR94q/0= -go.uber.org/multierr v1.6.0 h1:y6IPFStTAIT5Ytl7/XYmHvzXQ7S3g/IeZW9hyZ5thw4= go.uber.org/multierr v1.6.0/go.mod h1:cdWPpRnG4AhwMwsgIHip0KRBQjJy5kYEpYjJxpXp9iU= +go.uber.org/multierr v1.7.0 h1:zaiO/rmgFjbmCXdSYJWQcdvOCsthmdaHfr3Gm2Kx4Ec= +go.uber.org/multierr v1.7.0/go.mod h1:7EAYxJLBy9rStEaz58O2t4Uvip6FSURkq8/ppBp95ak= go.uber.org/zap v1.10.0/go.mod h1:vwi/ZaCAaUcBkycHslxD9B2zi4UTXhF60s6SWpuDF0Q= go.uber.org/zap v1.17.0/go.mod h1:MXVU+bhUf/A7Xi2HNOnopQOrmycQ5Ih87HtOu4q5SSo= go.uber.org/zap v1.19.0/go.mod h1:xg/QME4nWcxGxrpdeYfq7UvYrLh66cuVKdrbD1XF/NI= -go.uber.org/zap v1.19.1 h1:ue41HOKd1vGURxrmeKIgELGb3jPW9DMUDGtsinblHwI= go.uber.org/zap v1.19.1/go.mod h1:j3DNczoxDZroyBnOT1L/Q79cfUMGZxlv/9dzN7SM1rI= +go.uber.org/zap v1.21.0 h1:WefMeulhovoZ2sYXz7st6K0sLj7bBhpiFaud4r4zST8= +go.uber.org/zap v1.21.0/go.mod h1:wjWOCqI0f2ZZrJF/UufIOkiC8ii6tm1iqIsLo76RfJw= golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= golang.org/x/crypto v0.0.0-20181029021203-45a5f77698d3/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= @@ -588,11 +627,13 @@ golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v golang.org/x/net v0.0.0-20210316092652-d523dce5a7f4/go.mod h1:RBQZq4jEuRlivfhVLdyRGr576XBO4/greRjx4P4O3yc= golang.org/x/net v0.0.0-20210405180319-a5a99cb37ef4/go.mod h1:p54w0d4576C0XHj96bSt6lcn1PtDYWL6XObtHCRCNQM= golang.org/x/net v0.0.0-20210428140749-89ef3d95e781/go.mod h1:OJAsFXCWl8Ukc7SiCT/9KSuxbyM7479/AVlXFRxuMCk= +golang.org/x/net v0.0.0-20210503060351-7fd8e65b6420/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= golang.org/x/net v0.0.0-20210525063256-abc453219eb5/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= golang.org/x/net v0.0.0-20210805182204-aaa1db679c0d/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= golang.org/x/net v0.0.0-20210825183410-e898025ed96a/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= -golang.org/x/net v0.0.0-20211209124913-491a49abca63 h1:iocB37TsdFuN6IBRZ+ry36wrkoV51/tl5vOWqkcPGvY= golang.org/x/net v0.0.0-20211209124913-491a49abca63/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= +golang.org/x/net v0.0.0-20220127200216-cd36cc0744dd h1:O7DYs+zxREGLKzKoMQrtrEacpb0ZVXA5rIwylE2Xchk= +golang.org/x/net v0.0.0-20220127200216-cd36cc0744dd/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk= golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U= golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= @@ -606,8 +647,11 @@ golang.org/x/oauth2 v0.0.0-20210220000619-9bb904979d93/go.mod h1:KelEdhl1UZF7XfJ golang.org/x/oauth2 v0.0.0-20210313182246-cd4f82c27b84/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= golang.org/x/oauth2 v0.0.0-20210402161424-2e8d93401602/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= golang.org/x/oauth2 v0.0.0-20210514164344-f6687ab2804c/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= -golang.org/x/oauth2 v0.0.0-20210819190943-2bc19b11175f h1:Qmd2pbz05z7z6lm0DrgQVVPuBm92jqujBKMHMOlOQEw= +golang.org/x/oauth2 v0.0.0-20210628180205-a41e5a781914/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= +golang.org/x/oauth2 v0.0.0-20210805134026-6f1e6394065a/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= golang.org/x/oauth2 v0.0.0-20210819190943-2bc19b11175f/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= +golang.org/x/oauth2 v0.0.0-20211104180415-d3ed0bb246c8 h1:RerP+noqYHUQ8CMRcPlC2nvTa4dcBIjegkuWdcUDuqg= +golang.org/x/oauth2 v0.0.0-20211104180415-d3ed0bb246c8/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= @@ -677,17 +721,28 @@ golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44/go.mod h1:h1NjWce9XRLGQEsW7w golang.org/x/sys v0.0.0-20210403161142-5e06dd20ab57/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210510120138-977fb7262007/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20210514084401-e8d321eab015/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20210603081109-ebe580a85c40/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20210603125802-9665404d3644/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20210616094352-59db8d763f22/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20210806184541-e5e7981a1069/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20210809222454-d867a43fc93e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20210823070655-63515b42dcdf/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20210831042530-f4d43177bf5e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20211029165221-6e7872819dc8 h1:M69LAlWZCshgp0QSzyDcSsSIejIEeuaCVpmwcKwyLMk= +golang.org/x/sys v0.0.0-20210908233432-aa78b53d3365/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20211029165221-6e7872819dc8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20211124211545-fe61309f8881/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20211210111614-af8b64212486/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20220114195835-da31bd327af9/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20220207234003-57398862261d h1:Bm7BNOQt2Qv7ZqysjeLjgCBanX+88Z/OtdvsrEv1Djc= +golang.org/x/sys v0.0.0-20220207234003-57398862261d/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= -golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b h1:9zKuko04nR4gjZ4+DNjHqRlAJqbJETHwiNKDqTfOjfE= golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8= +golang.org/x/term v0.0.0-20210927222741-03fcf44c2211 h1:JGgROgKl9N8DuW20oFS5gxc+lE67/N3FcwmBPMe7ArY= +golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8= golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= @@ -702,8 +757,9 @@ golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxb golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/time v0.0.0-20210220033141-f8bda1e9f3ba/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= -golang.org/x/time v0.0.0-20210723032227-1f47c861a9ac h1:7zkz7BUtwNFFqcowJ+RIgu2MaV/MapERkDIy+mwPyjs= golang.org/x/time v0.0.0-20210723032227-1f47c861a9ac/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= +golang.org/x/time v0.0.0-20211116232009-f0f3c7e86c11 h1:GZokNIeuVkl3aZHJchRrr13WCsols02MLUcz1U9is6M= +golang.org/x/time v0.0.0-20211116232009-f0f3c7e86c11/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/tools v0.0.0-20180221164845-07fd8470d635/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20181030221726-6c7e314b6563/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= @@ -760,7 +816,10 @@ golang.org/x/tools v0.0.0-20201224043029-2b0845dc783e/go.mod h1:emZCQorbCU4vsT4f golang.org/x/tools v0.0.0-20210105154028-b0ab187a4818/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= golang.org/x/tools v0.1.0/go.mod h1:xkSsbof2nBLbhDlRMhhhyNLN/zl3eTqcnHD5viDpcZ0= +golang.org/x/tools v0.1.1/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= golang.org/x/tools v0.1.2/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= +golang.org/x/tools v0.1.3/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= +golang.org/x/tools v0.1.4/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= golang.org/x/tools v0.1.5/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= golang.org/x/tools v0.1.6-0.20210820212750-d4cc65f0b2ff/go.mod h1:YD9qOF0M9xpSpdWTBbzEl5e/RnCefISl8E5Noe10jFM= golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= @@ -792,6 +851,17 @@ google.golang.org/api v0.40.0/go.mod h1:fYKFpnQN0DsDSKRVRcQSDQNtqWPfM9i+zNPxepjR google.golang.org/api v0.41.0/go.mod h1:RkxM5lITDfTzmyKFPt+wGrCJbVfniCr2ool8kTBzRTU= google.golang.org/api v0.43.0/go.mod h1:nQsDGjRXMo4lvh5hP0TKqF244gqhGcr/YSIykhUk/94= google.golang.org/api v0.44.0/go.mod h1:EBOGZqzyhtvMDoxwS97ctnh0zUmYY6CxqXsc1AvkYD8= +google.golang.org/api v0.47.0/go.mod h1:Wbvgpq1HddcWVtzsVLyfLp8lDg6AA241LmgIL59tHXo= +google.golang.org/api v0.48.0/go.mod h1:71Pr1vy+TAZRPkPs/xlCf5SsU8WjuAWv1Pfjbtukyy4= +google.golang.org/api v0.50.0/go.mod h1:4bNT5pAuq5ji4SRZm+5QIkjny9JAyVD/3gaSihNefaw= +google.golang.org/api v0.51.0/go.mod h1:t4HdrdoNgyN5cbEfm7Lum0lcLDLiise1F8qDKX00sOU= +google.golang.org/api v0.54.0/go.mod h1:7C4bFFOvVDGXjfDTAsgGwDgAxRDeQ4X8NvUedIt6z3k= +google.golang.org/api v0.55.0/go.mod h1:38yMfeP1kfjsl8isn0tliTjIb1rJXcQi4UXlbqivdVE= +google.golang.org/api v0.56.0/go.mod h1:38yMfeP1kfjsl8isn0tliTjIb1rJXcQi4UXlbqivdVE= +google.golang.org/api v0.57.0/go.mod h1:dVPlbZyBo2/OjBpmvNdpn2GRm6rPy75jyU7bmhdrMgI= +google.golang.org/api v0.61.0/go.mod h1:xQRti5UdCmoCEqFxcz93fTl338AVqDgyaDRuOZ3hg9I= +google.golang.org/api v0.63.0/go.mod h1:gs4ij2ffTRXwuzzgJl/56BdwJaA194ijkfn++9tDuPo= +google.golang.org/api v0.66.0/go.mod h1:I1dmXYpX7HGwz/ejRxwQp2qj5bFAz93HiCU1C1oYd9M= google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM= google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4= google.golang.org/appengine v1.5.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4= @@ -843,8 +913,28 @@ google.golang.org/genproto v0.0.0-20210303154014-9728d6b83eeb/go.mod h1:FWY/as6D google.golang.org/genproto v0.0.0-20210310155132-4ce2db91004e/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= google.golang.org/genproto v0.0.0-20210319143718-93e7006c17a6/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= google.golang.org/genproto v0.0.0-20210402141018-6c239bbf2bb1/go.mod h1:9lPAdzaEmUacj36I+k7YKbEc5CXzPIeORRgDAUOu28A= +google.golang.org/genproto v0.0.0-20210513213006-bf773b8c8384/go.mod h1:P3QM42oQyzQSnHPnZ/vqoCdDmzH28fzWByN9asMeM8A= google.golang.org/genproto v0.0.0-20210602131652-f16073e35f0c/go.mod h1:UODoCrxHCcBojKKwX1terBiRUaqAsFqJiF615XL43r0= +google.golang.org/genproto v0.0.0-20210604141403-392c879c8b08/go.mod h1:UODoCrxHCcBojKKwX1terBiRUaqAsFqJiF615XL43r0= +google.golang.org/genproto v0.0.0-20210608205507-b6d2f5bf0d7d/go.mod h1:UODoCrxHCcBojKKwX1terBiRUaqAsFqJiF615XL43r0= +google.golang.org/genproto v0.0.0-20210624195500-8bfb893ecb84/go.mod h1:SzzZ/N+nwJDaO1kznhnlzqS8ocJICar6hYhVyhi++24= +google.golang.org/genproto v0.0.0-20210713002101-d411969a0d9a/go.mod h1:AxrInvYm1dci+enl5hChSFPOmmUF1+uAa/UsgNRWd7k= +google.golang.org/genproto v0.0.0-20210716133855-ce7ef5c701ea/go.mod h1:AxrInvYm1dci+enl5hChSFPOmmUF1+uAa/UsgNRWd7k= +google.golang.org/genproto v0.0.0-20210728212813-7823e685a01f/go.mod h1:ob2IJxKrgPT52GcgX759i1sleT07tiKowYBGbczaW48= +google.golang.org/genproto v0.0.0-20210805201207-89edb61ffb67/go.mod h1:ob2IJxKrgPT52GcgX759i1sleT07tiKowYBGbczaW48= +google.golang.org/genproto v0.0.0-20210813162853-db860fec028c/go.mod h1:cFeNkxwySK631ADgubI+/XFU/xp8FD5KIVV4rj8UC5w= +google.golang.org/genproto v0.0.0-20210821163610-241b8fcbd6c8/go.mod h1:eFjDcFEctNawg4eG61bRv87N7iHBWyVhJu7u1kqDUXY= +google.golang.org/genproto v0.0.0-20210828152312-66f60bf46e71/go.mod h1:eFjDcFEctNawg4eG61bRv87N7iHBWyVhJu7u1kqDUXY= google.golang.org/genproto v0.0.0-20210831024726-fe130286e0e2/go.mod h1:eFjDcFEctNawg4eG61bRv87N7iHBWyVhJu7u1kqDUXY= +google.golang.org/genproto v0.0.0-20210903162649-d08c68adba83/go.mod h1:eFjDcFEctNawg4eG61bRv87N7iHBWyVhJu7u1kqDUXY= +google.golang.org/genproto v0.0.0-20210909211513-a8c4777a87af/go.mod h1:eFjDcFEctNawg4eG61bRv87N7iHBWyVhJu7u1kqDUXY= +google.golang.org/genproto v0.0.0-20210924002016-3dee208752a0/go.mod h1:5CzLGKJ67TSI2B9POpiiyGha0AjJvZIUgRMt1dSmuhc= +google.golang.org/genproto v0.0.0-20211118181313-81c1377c94b1/go.mod h1:5CzLGKJ67TSI2B9POpiiyGha0AjJvZIUgRMt1dSmuhc= +google.golang.org/genproto v0.0.0-20211206160659-862468c7d6e0/go.mod h1:5CzLGKJ67TSI2B9POpiiyGha0AjJvZIUgRMt1dSmuhc= +google.golang.org/genproto v0.0.0-20211208223120-3a66f561d7aa/go.mod h1:5CzLGKJ67TSI2B9POpiiyGha0AjJvZIUgRMt1dSmuhc= +google.golang.org/genproto v0.0.0-20211221195035-429b39de9b1c/go.mod h1:5CzLGKJ67TSI2B9POpiiyGha0AjJvZIUgRMt1dSmuhc= +google.golang.org/genproto v0.0.0-20220114231437-d2e6a121cae0/go.mod h1:5CzLGKJ67TSI2B9POpiiyGha0AjJvZIUgRMt1dSmuhc= +google.golang.org/genproto v0.0.0-20220201184016-50beb8ab5c44/go.mod h1:5CzLGKJ67TSI2B9POpiiyGha0AjJvZIUgRMt1dSmuhc= google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c= google.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38= google.golang.org/grpc v1.21.1/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM= @@ -865,8 +955,13 @@ google.golang.org/grpc v1.35.0/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAG google.golang.org/grpc v1.36.0/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU= google.golang.org/grpc v1.36.1/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU= google.golang.org/grpc v1.37.0/go.mod h1:NREThFqKR1f3iQ6oBuvc5LadQuXVGo9rkm5ZGrQdJfM= +google.golang.org/grpc v1.37.1/go.mod h1:NREThFqKR1f3iQ6oBuvc5LadQuXVGo9rkm5ZGrQdJfM= google.golang.org/grpc v1.38.0/go.mod h1:NREThFqKR1f3iQ6oBuvc5LadQuXVGo9rkm5ZGrQdJfM= +google.golang.org/grpc v1.39.0/go.mod h1:PImNr+rS9TWYb2O4/emRugxiyHZ5JyHW5F+RPnDzfrE= +google.golang.org/grpc v1.39.1/go.mod h1:PImNr+rS9TWYb2O4/emRugxiyHZ5JyHW5F+RPnDzfrE= google.golang.org/grpc v1.40.0/go.mod h1:ogyxbiOoUXAkP+4+xa6PZSE9DZgIHtSpzjDTB9KAK34= +google.golang.org/grpc v1.40.1/go.mod h1:ogyxbiOoUXAkP+4+xa6PZSE9DZgIHtSpzjDTB9KAK34= +google.golang.org/grpc/cmd/protoc-gen-go-grpc v1.1.0/go.mod h1:6Kw0yEErY5E/yWrBtf03jp27GLLJujG4z/JK95pnjjw= google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8= google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0= google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM= @@ -922,42 +1017,53 @@ honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt honnef.co/go/tools v0.0.1-2020.1.3/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k= honnef.co/go/tools v0.0.1-2020.1.4/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k= k8s.io/api v0.23.0/go.mod h1:8wmDdLBHBNxtOIytwLstXt5E9PddnZb0GaMcqsvDBpg= -k8s.io/api v0.23.1 h1:ncu/qfBfUoClqwkTGbeRqqOqBCRoUAflMuOaOD7J0c8= -k8s.io/api v0.23.1/go.mod h1:WfXnOnwSqNtG62Y1CdjoMxh7r7u9QXGCkA1u0na2jgo= -k8s.io/apiextensions-apiserver v0.23.0 h1:uii8BYmHYiT2ZTAJxmvc3X8UhNYMxl2A0z0Xq3Pm+WY= +k8s.io/api v0.23.3 h1:KNrME8KHGr12Ozjf8ytOewKzZh6hl/hHUZeHddT3a38= +k8s.io/api v0.23.3/go.mod h1:w258XdGyvCmnBj/vGzQMj6kzdufJZVUwEM1U2fRJwSQ= k8s.io/apiextensions-apiserver v0.23.0/go.mod h1:xIFAEEDlAZgpVBl/1VSjGDmLoXAWRG40+GsWhKhAxY4= +k8s.io/apiextensions-apiserver v0.23.3 h1:JvPJA7hSEAqMRteveq4aj9semilAZYcJv+9HHFWfUdM= +k8s.io/apiextensions-apiserver v0.23.3/go.mod h1:/ZpRXdgKZA6DvIVPEmXDCZJN53YIQEUDF+hrpIQJL38= k8s.io/apimachinery v0.23.0/go.mod h1:fFCTTBKvKcwTPFzjlcxp91uPFZr+JA0FubU4fLzzFYc= -k8s.io/apimachinery v0.23.1 h1:sfBjlDFwj2onG0Ijx5C+SrAoeUscPrmghm7wHP+uXlo= -k8s.io/apimachinery v0.23.1/go.mod h1:SADt2Kl8/sttJ62RRsi9MIV4o8f5S3coArm0Iu3fBno= +k8s.io/apimachinery v0.23.3 h1:7IW6jxNzrXTsP0c8yXz2E5Yx/WTzVPTsHIx/2Vm0cIk= +k8s.io/apimachinery v0.23.3/go.mod h1:BEuFMMBaIbcOqVIJqNZJXGFTP4W6AycEpb5+m/97hrM= k8s.io/apiserver v0.23.0/go.mod h1:Cec35u/9zAepDPPFyT+UMrgqOCjgJ5qtfVJDxjZYmt4= +k8s.io/apiserver v0.23.3/go.mod h1:3HhsTmC+Pn+Jctw+Ow0LHA4dQ4oXrQ4XJDzrVDG64T4= k8s.io/client-go v0.23.0/go.mod h1:hrDnpnK1mSr65lHHcUuIZIXDgEbzc7/683c6hyG4jTA= -k8s.io/client-go v0.23.1 h1:Ma4Fhf/p07Nmj9yAB1H7UwbFHEBrSPg8lviR24U2GiQ= -k8s.io/client-go v0.23.1/go.mod h1:6QSI8fEuqD4zgFK0xbdwfB/PthBsIxCJMa3s17WlcO0= +k8s.io/client-go v0.23.3 h1:23QYUmCQ/W6hW78xIwm3XqZrrKZM+LWDqW2zfo+szJs= +k8s.io/client-go v0.23.3/go.mod h1:47oMd+YvAOqZM7pcQ6neJtBiFH7alOyfunYN48VsmwE= k8s.io/code-generator v0.23.0/go.mod h1:vQvOhDXhuzqiVfM/YHp+dmg10WDZCchJVObc9MvowsE= -k8s.io/component-base v0.23.0 h1:UAnyzjvVZ2ZR1lF35YwtNY6VMN94WtOnArcXBu34es8= +k8s.io/code-generator v0.23.3/go.mod h1:S0Q1JVA+kSzTI1oUvbKAxZY/DYbA/ZUb4Uknog12ETk= k8s.io/component-base v0.23.0/go.mod h1:DHH5uiFvLC1edCpvcTDV++NKULdYYU6pR9Tt3HIKMKI= +k8s.io/component-base v0.23.3 h1:q+epprVdylgecijVGVdf4MbizEL2feW4ssd7cdo6LVY= +k8s.io/component-base v0.23.3/go.mod h1:1Smc4C60rWG7d3HjSYpIwEbySQ3YWg0uzH5a2AtaTLg= k8s.io/gengo v0.0.0-20210813121822-485abfe95c7c/go.mod h1:FiNAH4ZV3gBg2Kwh89tzAEV2be7d5xI0vBa/VySYy3E= k8s.io/klog/v2 v2.0.0/go.mod h1:PBfzABfn139FHAV07az/IF9Wp1bkk3vpT2XSJ76fSDE= k8s.io/klog/v2 v2.2.0/go.mod h1:Od+F08eJP+W3HUb4pSrPpgp9DGU4GzlpG/TmITuYh/Y= -k8s.io/klog/v2 v2.30.0 h1:bUO6drIvCIsvZ/XFgfxoGFQU/a4Qkh0iAlvUR7vlHJw= k8s.io/klog/v2 v2.30.0/go.mod h1:y1WjHnz7Dj687irZUWR/WLkLc5N1YHtjLdmgWjndZn0= -k8s.io/kube-openapi v0.0.0-20211115234752-e816edb12b65 h1:E3J9oCLlaobFUqsjG9DfKbP2BmgwBL2p7pn0A3dG9W4= +k8s.io/klog/v2 v2.40.1 h1:P4RRucWk/lFOlDdkAr3mc7iWFkgKrZY9qZMAgek06S4= +k8s.io/klog/v2 v2.40.1/go.mod h1:y1WjHnz7Dj687irZUWR/WLkLc5N1YHtjLdmgWjndZn0= k8s.io/kube-openapi v0.0.0-20211115234752-e816edb12b65/go.mod h1:sX9MT8g7NVZM5lVL/j8QyCCJe8YSMW30QvGZWaCIDIk= +k8s.io/kube-openapi v0.0.0-20220124234850-424119656bbf h1:M9XBsiMslw2lb2ZzglC0TOkBPK5NQi0/noUrdnoFwUg= +k8s.io/kube-openapi v0.0.0-20220124234850-424119656bbf/go.mod h1:sX9MT8g7NVZM5lVL/j8QyCCJe8YSMW30QvGZWaCIDIk= k8s.io/utils v0.0.0-20210802155522-efc7438f0176/go.mod h1:jPW/WVKK9YHAvNhRxK0md/EJ228hCsBRufyofKtW8HA= -k8s.io/utils v0.0.0-20210930125809-cb0fa318a74b h1:wxEMGetGMur3J1xuGLQY7GEQYg9bZxKn3tKo5k/eYcs= k8s.io/utils v0.0.0-20210930125809-cb0fa318a74b/go.mod h1:jPW/WVKK9YHAvNhRxK0md/EJ228hCsBRufyofKtW8HA= +k8s.io/utils v0.0.0-20211116205334-6203023598ed/go.mod h1:jPW/WVKK9YHAvNhRxK0md/EJ228hCsBRufyofKtW8HA= +k8s.io/utils v0.0.0-20220127004650-9b3446523e65 h1:ONWS0Wgdg5wRiQIAui7L/023aC9+IxrIrydY7l8llsE= +k8s.io/utils v0.0.0-20220127004650-9b3446523e65/go.mod h1:jPW/WVKK9YHAvNhRxK0md/EJ228hCsBRufyofKtW8HA= rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8= rsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0= rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA= sigs.k8s.io/apiserver-network-proxy/konnectivity-client v0.0.25/go.mod h1:Mlj9PNLmG9bZ6BHFwFKDo5afkpWyUISkb9Me0GnK66I= +sigs.k8s.io/apiserver-network-proxy/konnectivity-client v0.0.27/go.mod h1:tq2nT0Kx7W+/f2JVE+zxYtUhdjuELJkVpNz+x/QN5R4= sigs.k8s.io/controller-runtime v0.11.0 h1:DqO+c8mywcZLFJWILq4iktoECTyn30Bkj0CwgqMpZWQ= sigs.k8s.io/controller-runtime v0.11.0/go.mod h1:KKwLiTooNGu+JmLZGn9Sl3Gjmfj66eMbCQznLP5zcqA= -sigs.k8s.io/json v0.0.0-20211020170558-c049b76a60c6 h1:fD1pz4yfdADVNfFmcP2aBEtudwUQ1AlLnRBALr33v3s= sigs.k8s.io/json v0.0.0-20211020170558-c049b76a60c6/go.mod h1:p4QtZmO4uMYipTQNzagwnNoseA6OxSUutVw05NhYDRs= +sigs.k8s.io/json v0.0.0-20211208200746-9f7c6b3444d2 h1:kDi4JBNAsJWfz1aEXhO8Jg87JJaPNLh5tIzYHgStQ9Y= +sigs.k8s.io/json v0.0.0-20211208200746-9f7c6b3444d2/go.mod h1:B+TnT182UBxE84DiCz4CVE26eOSDAeYCpfDnC2kdKMY= sigs.k8s.io/structured-merge-diff/v4 v4.0.2/go.mod h1:bJZC9H9iH24zzfZ/41RGcq60oK1F7G282QMXDPYydCw= sigs.k8s.io/structured-merge-diff/v4 v4.1.2/go.mod h1:j/nl6xW8vLS49O8YvXW1ocPhZawJtm+Yrr7PPRQ0Vg4= -sigs.k8s.io/structured-merge-diff/v4 v4.2.0 h1:kDvPBbnPk+qYmkHmSo8vKGp438IASWofnbbUKDE/bv0= sigs.k8s.io/structured-merge-diff/v4 v4.2.0/go.mod h1:j/nl6xW8vLS49O8YvXW1ocPhZawJtm+Yrr7PPRQ0Vg4= +sigs.k8s.io/structured-merge-diff/v4 v4.2.1 h1:bKCqE9GvQ5tiVHn5rfn1r+yao3aLQEaLzkkmAkf+A6Y= +sigs.k8s.io/structured-merge-diff/v4 v4.2.1/go.mod h1:j/nl6xW8vLS49O8YvXW1ocPhZawJtm+Yrr7PPRQ0Vg4= sigs.k8s.io/yaml v1.2.0/go.mod h1:yfXDCHCao9+ENCvLSE62v9VSji2MKu5jeNfTrofGhJc= sigs.k8s.io/yaml v1.3.0 h1:a2VclLzOGrwOHDiV8EfBGhvjHvP46CtW5j6POvhYGGo= sigs.k8s.io/yaml v1.3.0/go.mod h1:GeOyir5tyXNByN85N/dRIT9es5UQNerPYEKK56eTBm8= From f43fb1f9b64179d394abf25aaed571a7f59bc498 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Wed, 9 Feb 2022 16:17:57 -0500 Subject: [PATCH 151/628] change OCI-Go-SDK from v51 to v54 --- apis/database/v1alpha1/autonomousdatabasebackup_types.go | 2 +- commons/oci/ociutil/time.go | 2 +- controllers/database/autonomousdatabasebackup_controller.go | 4 ++-- .../database/autonomousdatabaserestore_controller.go | 6 +++--- 4 files changed, 7 insertions(+), 7 deletions(-) diff --git a/apis/database/v1alpha1/autonomousdatabasebackup_types.go b/apis/database/v1alpha1/autonomousdatabasebackup_types.go index 63246eb3..d9210315 100644 --- a/apis/database/v1alpha1/autonomousdatabasebackup_types.go +++ b/apis/database/v1alpha1/autonomousdatabasebackup_types.go @@ -41,7 +41,7 @@ package v1alpha1 import ( metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" - "github.com/oracle/oci-go-sdk/v51/database" + "github.com/oracle/oci-go-sdk/v54/database" "github.com/oracle/oracle-database-operator/commons/oci/ociutil" ) diff --git a/commons/oci/ociutil/time.go b/commons/oci/ociutil/time.go index 49cd9894..921dfe3d 100644 --- a/commons/oci/ociutil/time.go +++ b/commons/oci/ociutil/time.go @@ -3,7 +3,7 @@ package ociutil import ( "time" - "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v54/common" ) // Follow the format of the Display Name diff --git a/controllers/database/autonomousdatabasebackup_controller.go b/controllers/database/autonomousdatabasebackup_controller.go index df96d434..ab0ab904 100644 --- a/controllers/database/autonomousdatabasebackup_controller.go +++ b/controllers/database/autonomousdatabasebackup_controller.go @@ -51,8 +51,8 @@ import ( "sigs.k8s.io/controller-runtime/pkg/event" "sigs.k8s.io/controller-runtime/pkg/predicate" - "github.com/oracle/oci-go-sdk/v51/database" - "github.com/oracle/oci-go-sdk/v51/workrequests" + "github.com/oracle/oci-go-sdk/v54/database" + "github.com/oracle/oci-go-sdk/v54/workrequests" databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" backupUtil "github.com/oracle/oracle-database-operator/commons/autonomousdatabase" diff --git a/controllers/database/autonomousdatabaserestore_controller.go b/controllers/database/autonomousdatabaserestore_controller.go index e393bc5d..358cae90 100644 --- a/controllers/database/autonomousdatabaserestore_controller.go +++ b/controllers/database/autonomousdatabaserestore_controller.go @@ -48,9 +48,9 @@ import ( ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" - "github.com/oracle/oci-go-sdk/v51/common" - "github.com/oracle/oci-go-sdk/v51/database" - "github.com/oracle/oci-go-sdk/v51/workrequests" + "github.com/oracle/oci-go-sdk/v54/common" + "github.com/oracle/oci-go-sdk/v54/database" + "github.com/oracle/oci-go-sdk/v54/workrequests" databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" restoreUtil "github.com/oracle/oracle-database-operator/commons/autonomousdatabase" From b0083e1ca6cd9323659204796711a7d3ab7891e1 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Wed, 9 Feb 2022 23:13:46 -0500 Subject: [PATCH 152/628] update the restore feature --- PROJECT | 7 + .../v1alpha1/autonomousdatabase_webhook.go | 15 +- .../autonomousdatabasebackup_types.go | 18 ++- .../autonomousdatabasebackup_webhook.go | 145 ++++++++++++++++++ .../autonomousdatabaserestore_types.go | 32 ++-- .../autonomousdatabaserestore_webhook.go | 139 +++++++++++++++++ .../v1alpha1/zz_generated.deepcopy.go | 32 ++-- .../autonomousdatabase/adb_reconciler_util.go | 31 ++-- .../adb_restore_reconciler_util.go | 73 +++++++++ commons/oci/database.go | 50 +++--- ....oracle.com_autonomousdatabasebackups.yaml | 19 ++- ...oracle.com_autonomousdatabaserestores.yaml | 40 +++-- config/manager/kustomization.yaml | 4 +- config/manager/manager.yaml | 2 +- .../adb/autonomousdatabase_restore.yaml | 14 +- config/webhook/manifests.yaml | 60 ++++++++ .../database/autonomousdatabase_controller.go | 28 ++-- .../autonomousdatabasebackup_controller.go | 32 ++-- .../autonomousdatabaserestore_controller.go | 80 ++-------- main.go | 8 + test/e2e/resource/test_config.yaml | 15 +- 21 files changed, 638 insertions(+), 206 deletions(-) create mode 100644 apis/database/v1alpha1/autonomousdatabasebackup_webhook.go create mode 100644 apis/database/v1alpha1/autonomousdatabaserestore_webhook.go diff --git a/PROJECT b/PROJECT index 6cc088f1..455e938d 100644 --- a/PROJECT +++ b/PROJECT @@ -29,6 +29,10 @@ resources: kind: AutonomousDatabaseBackup path: github.com/oracle/oracle-database-operator/apis/database/v1alpha1 version: v1alpha1 + webhooks: + defaulting: true + validation: true + webhookVersion: v1beta1 - api: crdVersion: v1beta1 namespaced: true @@ -38,6 +42,9 @@ resources: kind: AutonomousDatabaseRestore path: github.com/oracle/oracle-database-operator/apis/database/v1alpha1 version: v1alpha1 + webhooks: + validation: true + webhookVersion: v1beta1 - api: crdVersion: v1 namespaced: true diff --git a/apis/database/v1alpha1/autonomousdatabase_webhook.go b/apis/database/v1alpha1/autonomousdatabase_webhook.go index 14779904..df6d2f59 100644 --- a/apis/database/v1alpha1/autonomousdatabase_webhook.go +++ b/apis/database/v1alpha1/autonomousdatabase_webhook.go @@ -100,7 +100,7 @@ func (r *AutonomousDatabase) ValidateCreate() error { autonomousdatabaselog.Info("validate create", "name", r.Name) if r.Spec.Details.AutonomousDatabaseOCID == nil { // provisioning operation - allErrs = validateNetworkAcces(r, allErrs) + allErrs = validateNetworkAccess(r, allErrs) } else { // binding operation } @@ -124,10 +124,17 @@ func (r *AutonomousDatabase) ValidateUpdate(old runtime.Object) error { } var allErrs field.ErrorList - autonomousdatabaselog.Info("validate update", "name", r.Name) - allErrs = validateNetworkAcces(r, allErrs) + allErrs = validateNetworkAccess(r, allErrs) + + if r.Spec.Details.AutonomousDatabaseOCID != nil && + old.(*AutonomousDatabase).Spec.Details.AutonomousDatabaseOCID != nil && + *r.Spec.Details.AutonomousDatabaseOCID != *old.(*AutonomousDatabase).Spec.Details.AutonomousDatabaseOCID { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("details").Child("autonomousDatabaseOCID"), + "autonomousDatabaseOCID cannot be modified")) + } if len(allErrs) == 0 { return nil @@ -137,7 +144,7 @@ func (r *AutonomousDatabase) ValidateUpdate(old runtime.Object) error { r.Name, allErrs) } -func validateNetworkAcces(adb *AutonomousDatabase, allErrs field.ErrorList) field.ErrorList { +func validateNetworkAccess(adb *AutonomousDatabase, allErrs field.ErrorList) field.ErrorList { if !isDedicated(adb) { // Shared database if adb.Spec.Details.NetworkAccess.AccessType == NetworkAccessTypePublic { diff --git a/apis/database/v1alpha1/autonomousdatabasebackup_types.go b/apis/database/v1alpha1/autonomousdatabasebackup_types.go index d9210315..bb5ec00a 100644 --- a/apis/database/v1alpha1/autonomousdatabasebackup_types.go +++ b/apis/database/v1alpha1/autonomousdatabasebackup_types.go @@ -52,7 +52,7 @@ import ( type AutonomousDatabaseBackupSpec struct { // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster // Important: Run "make" to regenerate code after modifying this file - AutonomousDatabaseOCID string `json:"autonomousDatabaseOCID"` + AutonomousDatabaseOCID string `json:"autonomousDatabaseOCID,omitempty"` DisplayName string `json:"displayName,omitempty"` AutonomousDatabaseBackupOCID string `json:"autonomousDatabaseBackupOCID,omitempty"` @@ -63,21 +63,24 @@ type AutonomousDatabaseBackupSpec struct { type AutonomousDatabaseBackupStatus struct { // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster // Important: Run "make" to regenerate code after modifying this file + LifecycleState database.AutonomousDatabaseBackupLifecycleStateEnum `json:"lifecycleState"` AutonomousDatabaseBackupOCID string `json:"autonomousDatabaseBackupOCID"` - CompartmentOCID string `json:"compartmentOCID"` - AutonomousDatabaseOCID string `json:"autonomousDatabaseOCID"` DisplayName string `json:"displayName"` Type database.AutonomousDatabaseBackupTypeEnum `json:"type"` IsAutomatic bool `json:"isAutomatic"` - LifecycleState database.AutonomousDatabaseBackupLifecycleStateEnum `json:"lifecycleState"` TimeStarted string `json:"timeStarted,omitempty"` TimeEnded string `json:"timeEnded,omitempty"` + CompartmentOCID string `json:"compartmentOCID"` + AutonomousDatabaseOCID string `json:"autonomousDatabaseOCID"` + DBName string `json:"dbName"` + DBDisplayName string `json:"dbDisplayName"` } //+kubebuilder:object:root=true //+kubebuilder:subresource:status // +kubebuilder:resource:shortName="adbbu";"adbbus" // +kubebuilder:printcolumn:JSONPath=".status.lifecycleState",name="State",type=string +// +kubebuilder:printcolumn:JSONPath=".status.dbDisplayName",name="DB DisplayName",type=string // +kubebuilder:printcolumn:JSONPath=".status.type",name="Type",type=string // +kubebuilder:printcolumn:JSONPath=".status.timeStarted",name="Started",type=string // +kubebuilder:printcolumn:JSONPath=".status.timeEnded",name="Ended",type=string @@ -91,10 +94,9 @@ type AutonomousDatabaseBackup struct { Status AutonomousDatabaseBackupStatus `json:"status,omitempty"` } -func (backup *AutonomousDatabaseBackup) UpdateStatusFromAutonomousDatabaseBackupResponse(ociBackup database.AutonomousDatabaseBackup) { +func (backup *AutonomousDatabaseBackup) UpdateStatusFromAutonomousDatabaseBackupResponse(ociBackup database.AutonomousDatabaseBackup, ociADB database.AutonomousDatabase) { backup.Status.AutonomousDatabaseBackupOCID = *ociBackup.Id backup.Status.CompartmentOCID = *ociBackup.CompartmentId - backup.Status.AutonomousDatabaseOCID = *ociBackup.AutonomousDatabaseId backup.Status.DisplayName = *ociBackup.DisplayName backup.Status.Type = ociBackup.Type backup.Status.IsAutomatic = *ociBackup.IsAutomatic @@ -106,6 +108,10 @@ func (backup *AutonomousDatabaseBackup) UpdateStatusFromAutonomousDatabaseBackup if ociBackup.TimeEnded != nil { backup.Status.TimeEnded = ociutil.FormatSDKTime(ociBackup.TimeEnded.Time) } + + backup.Status.AutonomousDatabaseOCID = *ociBackup.AutonomousDatabaseId + backup.Status.DBDisplayName = *ociADB.DisplayName + backup.Status.DBName = *ociADB.DbName } //+kubebuilder:object:root=true diff --git a/apis/database/v1alpha1/autonomousdatabasebackup_webhook.go b/apis/database/v1alpha1/autonomousdatabasebackup_webhook.go new file mode 100644 index 00000000..6b2c07df --- /dev/null +++ b/apis/database/v1alpha1/autonomousdatabasebackup_webhook.go @@ -0,0 +1,145 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v1alpha1 + +import ( + apierrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/runtime/schema" + "k8s.io/apimachinery/pkg/util/validation/field" + ctrl "sigs.k8s.io/controller-runtime" + logf "sigs.k8s.io/controller-runtime/pkg/log" + "sigs.k8s.io/controller-runtime/pkg/webhook" +) + +// log is for logging in this package. +var autonomousdatabasebackuplog = logf.Log.WithName("autonomousdatabasebackup-resource") + +func (r *AutonomousDatabaseBackup) SetupWebhookWithManager(mgr ctrl.Manager) error { + return ctrl.NewWebhookManagedBy(mgr). + For(r). + Complete() +} + +// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! + +//+kubebuilder:webhook:path=/mutate-database-oracle-com-v1alpha1-autonomousdatabasebackup,mutating=true,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=autonomousdatabasebackups,verbs=create;update,versions=v1alpha1,name=mautonomousdatabasebackup.kb.io,admissionReviewVersions={v1} + +var _ webhook.Defaulter = &AutonomousDatabaseBackup{} + +// Default implements webhook.Defaulter so a webhook will be registered for the type +func (r *AutonomousDatabaseBackup) Default() { + autonomousdatabasebackuplog.Info("default", "name", r.Name) + + // TODO(user): fill in your defaulting logic. +} + +// TODO(user): change verbs to "verbs=create;update;delete" if you want to enable deletion validation. +//+kubebuilder:webhook:verbs=create;update,path=/validate-database-oracle-com-v1alpha1-autonomousdatabasebackup,mutating=false,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=autonomousdatabasebackups,versions=v1alpha1,name=vautonomousdatabasebackup.kb.io,admissionReviewVersions={v1} + +var _ webhook.Validator = &AutonomousDatabaseBackup{} + +// ValidateCreate implements webhook.Validator so a webhook will be registered for the type +func (r *AutonomousDatabaseBackup) ValidateCreate() error { + autonomousdatabasebackuplog.Info("validate create", "name", r.Name) + + var allErrs field.ErrorList + + if r.Spec.AutonomousDatabaseBackupOCID != "" && r.Spec.AutonomousDatabaseOCID != "" { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("autonomousDatabaseBackupOCID"), + "cannot apply autonomousDatabaseBackupOCID and autonomousDatabaseOCID to the backup at the same time")) + } + + if r.Spec.DisplayName != "" && r.Spec.AutonomousDatabaseOCID == "" { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("displayName"), + "autonomousDatabaseOCID cannot be empty")) + } + + if len(allErrs) == 0 { + return nil + } + return apierrors.NewInvalid( + schema.GroupKind{Group: "database.oracle.com", Kind: "AutonomousDatabaseBackup"}, + r.Name, allErrs) +} + +// ValidateUpdate implements webhook.Validator so a webhook will be registered for the type +func (r *AutonomousDatabaseBackup) ValidateUpdate(old runtime.Object) error { + autonomousdatabasebackuplog.Info("validate update", "name", r.Name) + + var allErrs field.ErrorList + + if r.Spec.AutonomousDatabaseBackupOCID != "" && + old.(*AutonomousDatabaseBackup).Status.AutonomousDatabaseBackupOCID != "" && + r.Spec.AutonomousDatabaseBackupOCID != old.(*AutonomousDatabaseBackup).Status.AutonomousDatabaseBackupOCID { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("autonomousDatabaseBackupOCID"), "cannot assign a new autonomousDatabaseBackupOCID to this backup")) + } + + if r.Spec.AutonomousDatabaseOCID != "" && + old.(*AutonomousDatabaseBackup).Status.AutonomousDatabaseOCID != "" && + r.Spec.AutonomousDatabaseOCID != old.(*AutonomousDatabaseBackup).Status.AutonomousDatabaseOCID { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("autonomousDatabaseOCID"), "cannot assign a new autonomousDatabaseOCID to this backup")) + } + + if r.Spec.DisplayName != "" && + old.(*AutonomousDatabaseBackup).Status.DisplayName != "" && + r.Spec.DisplayName != old.(*AutonomousDatabaseBackup).Status.DisplayName { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("displayName"), "cannot assign a new displayName to this backup")) + } + + if len(allErrs) == 0 { + return nil + } + return apierrors.NewInvalid( + schema.GroupKind{Group: "database.oracle.com", Kind: "AutonomousDatabaseBackup"}, + r.Name, allErrs) +} + +// ValidateDelete implements webhook.Validator so a webhook will be registered for the type +func (r *AutonomousDatabaseBackup) ValidateDelete() error { + autonomousdatabasebackuplog.Info("validate delete", "name", r.Name) + + // TODO(user): fill in your validation logic upon object deletion. + return nil +} diff --git a/apis/database/v1alpha1/autonomousdatabaserestore_types.go b/apis/database/v1alpha1/autonomousdatabaserestore_types.go index 6d37655e..a4c1652a 100644 --- a/apis/database/v1alpha1/autonomousdatabaserestore_types.go +++ b/apis/database/v1alpha1/autonomousdatabaserestore_types.go @@ -39,6 +39,7 @@ package v1alpha1 import ( + "github.com/oracle/oci-go-sdk/v51/workrequests" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" ) @@ -49,37 +50,32 @@ import ( type AutonomousDatabaseRestoreSpec struct { // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster // Important: Run "make" to regenerate code after modifying this file - AutonomousDatabaseOCID string `json:"autonomousDatabaseOCID"` - Source AutonomousDatabaseRestoreSource `json:"source"` - - OCIConfig OCIConfigSpec `json:"ociConfig,omitempty"` + BackupName string `json:"backupName,omitempty"` + PointInTime PITSource `json:"pointInTime,omitempty"` + OCIConfig OCIConfigSpec `json:"ociConfig,omitempty"` } -type AutonomousDatabaseRestoreSource struct { - BackupName string `json:"backupName,omitempty"` - TimeStamp string `json:"timeStamp,omitempty"` +type PITSource struct { + AutonomousDatabaseOCID string `json:"autonomousDatabaseOCID,omitempty"` + TimeStamp string `json:"timeStamp,omitempty"` } -// +kubebuilder:validation:Enum=NewCreated;InProgress;Completed;Failed -type RestoreLifecycleState string - -const ( - RestoreLifecycleStateNew RestoreLifecycleState = "NewCreated" - RestoreLifecycleStateInProgress RestoreLifecycleState = "InProgress" - RestoreLifecycleStateCompleted RestoreLifecycleState = "Completed" - RestoreLifecycleStateFailed RestoreLifecycleState = "Failed" -) - // AutonomousDatabaseRestoreStatus defines the observed state of AutonomousDatabaseRestore type AutonomousDatabaseRestoreStatus struct { // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster // Important: Run "make" to regenerate code after modifying this file - LifecycleState RestoreLifecycleState `json:"state"` + DisplayName string `json:"displayName"` + DbName string `json:"dbName"` + AutonomousDatabaseOCID string `json:"autonomousDatabaseOCID"` + LifecycleState workrequests.WorkRequestStatusEnum `json:"lifecycleState"` } //+kubebuilder:object:root=true //+kubebuilder:subresource:status //+kubebuilder:resource:shortName="adbr";"adbrs" +// +kubebuilder:printcolumn:JSONPath=".status.lifecycleState",name="State",type=string +// +kubebuilder:printcolumn:JSONPath=".status.displayName",name="DisplayName",type=string +// +kubebuilder:printcolumn:JSONPath=".status.dbName",name="DbName",type=string // AutonomousDatabaseRestore is the Schema for the autonomousdatabaserestores API type AutonomousDatabaseRestore struct { diff --git a/apis/database/v1alpha1/autonomousdatabaserestore_webhook.go b/apis/database/v1alpha1/autonomousdatabaserestore_webhook.go new file mode 100644 index 00000000..709a8d93 --- /dev/null +++ b/apis/database/v1alpha1/autonomousdatabaserestore_webhook.go @@ -0,0 +1,139 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v1alpha1 + +import ( + "reflect" + + "github.com/oracle/oracle-database-operator/commons/oci/ociutil" + apierrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/runtime/schema" + "k8s.io/apimachinery/pkg/util/validation/field" + ctrl "sigs.k8s.io/controller-runtime" + logf "sigs.k8s.io/controller-runtime/pkg/log" + "sigs.k8s.io/controller-runtime/pkg/webhook" +) + +// log is for logging in this package. +var autonomousdatabaserestorelog = logf.Log.WithName("autonomousdatabaserestore-resource") + +func (r *AutonomousDatabaseRestore) SetupWebhookWithManager(mgr ctrl.Manager) error { + return ctrl.NewWebhookManagedBy(mgr). + For(r). + Complete() +} + +// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! + +// TODO(user): change verbs to "verbs=create;update;delete" if you want to enable deletion validation. +//+kubebuilder:webhook:verbs=create;update,path=/validate-database-oracle-com-v1alpha1-autonomousdatabaserestore,mutating=false,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=autonomousdatabaserestores,versions=v1alpha1,name=vautonomousdatabaserestore.kb.io,admissionReviewVersions={v1} + +var _ webhook.Validator = &AutonomousDatabaseRestore{} + +// ValidateCreate implements webhook.Validator so a webhook will be registered for the type +func (r *AutonomousDatabaseRestore) ValidateCreate() error { + autonomousdatabaserestorelog.Info("validate create", "name", r.Name) + + var allErrs field.ErrorList + + // Validate the restore source + if r.Spec.BackupName == "" && + r.Spec.PointInTime.AutonomousDatabaseOCID == "" && + r.Spec.PointInTime.TimeStamp == "" { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec"), "no retore source is chosen")) + } + + if r.Spec.BackupName != "" && + (r.Spec.PointInTime.AutonomousDatabaseOCID != "" || r.Spec.PointInTime.TimeStamp != "") { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec"), "cannot apply backupName and the PITR parameters at the same time")) + } + + if (r.Spec.PointInTime.AutonomousDatabaseOCID == "" && r.Spec.PointInTime.TimeStamp != "") || + (r.Spec.PointInTime.AutonomousDatabaseOCID != "" && r.Spec.PointInTime.TimeStamp == "") { + field.Forbidden(field.NewPath("spec").Child("pointInTime"), "autonomousDatabaseOCID or timeStamp cannot be empty") + } + + // Verify the timestamp format if it's PITR + if r.Spec.PointInTime.TimeStamp != "" { + _, err := ociutil.ParseDisplayTime(r.Spec.PointInTime.TimeStamp) + if err != nil { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec"), "invalid timestamp format")) + } + } + + if len(allErrs) == 0 { + return nil + } + return apierrors.NewInvalid( + schema.GroupKind{Group: "database.oracle.com", Kind: "AutonomousDatabaseRestore"}, + r.Name, allErrs) +} + +// ValidateUpdate implements webhook.Validator so a webhook will be registered for the type +func (r *AutonomousDatabaseRestore) ValidateUpdate(old runtime.Object) error { + autonomousdatabaserestorelog.Info("validate update", "name", r.Name) + + var allErrs field.ErrorList + + if old.(*AutonomousDatabaseRestore).Status.LifecycleState != "" && + !reflect.DeepEqual(r.Spec, old.(*AutonomousDatabaseRestore).Spec) { + + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec"), "the AutonomousDatabaseRestore resource cannot be modified")) + } + + if len(allErrs) == 0 { + return nil + } + return apierrors.NewInvalid( + schema.GroupKind{Group: "database.oracle.com", Kind: "AutonomousDatabaseRestore"}, + r.Name, allErrs) +} + +// ValidateDelete implements webhook.Validator so a webhook will be registered for the type +func (r *AutonomousDatabaseRestore) ValidateDelete() error { + autonomousdatabaserestorelog.Info("validate delete", "name", r.Name) + + // TODO(user): fill in your validation logic upon object deletion. + return nil +} diff --git a/apis/database/v1alpha1/zz_generated.deepcopy.go b/apis/database/v1alpha1/zz_generated.deepcopy.go index fba5ba12..0e63b223 100644 --- a/apis/database/v1alpha1/zz_generated.deepcopy.go +++ b/apis/database/v1alpha1/zz_generated.deepcopy.go @@ -332,25 +332,10 @@ func (in *AutonomousDatabaseRestoreList) DeepCopyObject() runtime.Object { return nil } -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *AutonomousDatabaseRestoreSource) DeepCopyInto(out *AutonomousDatabaseRestoreSource) { - *out = *in -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseRestoreSource. -func (in *AutonomousDatabaseRestoreSource) DeepCopy() *AutonomousDatabaseRestoreSource { - if in == nil { - return nil - } - out := new(AutonomousDatabaseRestoreSource) - in.DeepCopyInto(out) - return out -} - // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *AutonomousDatabaseRestoreSpec) DeepCopyInto(out *AutonomousDatabaseRestoreSpec) { *out = *in - out.Source = in.Source + out.PointInTime = in.PointInTime in.OCIConfig.DeepCopyInto(&out.OCIConfig) } @@ -655,6 +640,21 @@ func (in *OCIConfigSpec) DeepCopy() *OCIConfigSpec { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PITSource) DeepCopyInto(out *PITSource) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PITSource. +func (in *PITSource) DeepCopy() *PITSource { + if in == nil { + return nil + } + out := new(PITSource) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *PasswordSpec) DeepCopyInto(out *PasswordSpec) { *out = *in diff --git a/commons/autonomousdatabase/adb_reconciler_util.go b/commons/autonomousdatabase/adb_reconciler_util.go index d3b3761a..eb4ecc59 100644 --- a/commons/autonomousdatabase/adb_reconciler_util.go +++ b/commons/autonomousdatabase/adb_reconciler_util.go @@ -87,7 +87,8 @@ func UpdateAutonomousDatabaseDetails(logger logr.Logger, kubeClient client.Clien return nil } -func UpdateGeneralAndPasswordAttributesAndWait(logger logr.Logger, kubeClient client.Client, +func UpdateGeneralAndPasswordAttributesAndWait(logger logr.Logger, + kubeClient client.Client, dbClient database.DatabaseClient, secretClient secrets.SecretsClient, workClient workrequests.WorkRequestClient, @@ -103,13 +104,13 @@ func UpdateGeneralAndPasswordAttributesAndWait(logger logr.Logger, kubeClient cl return statusErr } // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue - return err + return nil } // Wait for the work finish if a request is sent. Note that some of the requests (e.g. update displayName) won't return a work request ID. if updateGenPassResp.OpcWorkRequestId != nil { - if err := UpdateStatusAndWait(logger, kubeClient, workClient, adb, updateGenPassResp.AutonomousDatabase.LifecycleState, updateGenPassResp.OpcWorkRequestId); err != nil { - logger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*updateGenPassResp.OpcWorkRequestId) + if err := UpdateAutonomousDatabaseStatusAndWait(logger, kubeClient, workClient, adb, updateGenPassResp.AutonomousDatabase.LifecycleState, updateGenPassResp.OpcWorkRequestId); err != nil { + logger.Error(err, "Fail to update Autonomous Database. opcWorkRequestID = "+*updateGenPassResp.OpcWorkRequestId) } logger.Info("Update AutonomousDatabase " + *adb.Spec.Details.DbName + " succesfully") @@ -125,8 +126,6 @@ func UpdateScaleAttributesAndWait(logger logr.Logger, kubeClient client.Client, scaleResp, err := oci.UpdateScaleAttributes(logger, kubeClient, dbClient, adb) if err != nil { - logger.Error(err, "Fail to update Autonomous Database") - // Change the status to UNAVAILABLE adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable if statusErr := UpdateAutonomousDatabaseStatus(kubeClient, adb); statusErr != nil { @@ -137,8 +136,8 @@ func UpdateScaleAttributesAndWait(logger logr.Logger, kubeClient client.Client, } if scaleResp.OpcWorkRequestId != nil { - if err := UpdateStatusAndWait(logger, kubeClient, workClient, adb, scaleResp.AutonomousDatabase.LifecycleState, scaleResp.OpcWorkRequestId); err != nil { - logger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*scaleResp.OpcWorkRequestId) + if err := UpdateAutonomousDatabaseStatusAndWait(logger, kubeClient, workClient, adb, scaleResp.AutonomousDatabase.LifecycleState, scaleResp.OpcWorkRequestId); err != nil { + logger.Error(err, "Fail to scale the Autonomous Database. opcWorkRequestID = "+*scaleResp.OpcWorkRequestId) } logger.Info("Scale AutonomousDatabase " + *adb.Spec.Details.DbName + " succesfully") @@ -261,8 +260,8 @@ func updateMTLSAndWait(logger logr.Logger, } if resp.OpcWorkRequestId != nil { - if err := UpdateStatusAndWait(logger, kubeClient, workClient, curADB, resp.AutonomousDatabase.LifecycleState, resp.OpcWorkRequestId); err != nil { - logger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*resp.OpcWorkRequestId) + if err := UpdateAutonomousDatabaseStatusAndWait(logger, kubeClient, workClient, curADB, resp.AutonomousDatabase.LifecycleState, resp.OpcWorkRequestId); err != nil { + logger.Error(err, "Fail to update Autonomous Database MTLS. opcWorkRequestID = "+*resp.OpcWorkRequestId) } } @@ -281,8 +280,8 @@ func setNetworkAccessPublicAndWait(logger logr.Logger, } if resp.OpcWorkRequestId != nil { - if err := UpdateStatusAndWait(logger, kubeClient, workClient, curADB, resp.AutonomousDatabase.LifecycleState, resp.OpcWorkRequestId); err != nil { - logger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*resp.OpcWorkRequestId) + if err := UpdateAutonomousDatabaseStatusAndWait(logger, kubeClient, workClient, curADB, resp.AutonomousDatabase.LifecycleState, resp.OpcWorkRequestId); err != nil { + logger.Error(err, "Fail to update the network access options to public. opcWorkRequestID = "+*resp.OpcWorkRequestId) } } @@ -301,15 +300,15 @@ func updateNetworkAccessAttributesAndWait(logger logr.Logger, } if resp.OpcWorkRequestId != nil { - if err := UpdateStatusAndWait(logger, kubeClient, workClient, curADB, resp.AutonomousDatabase.LifecycleState, resp.OpcWorkRequestId); err != nil { - logger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*resp.OpcWorkRequestId) + if err := UpdateAutonomousDatabaseStatusAndWait(logger, kubeClient, workClient, curADB, resp.AutonomousDatabase.LifecycleState, resp.OpcWorkRequestId); err != nil { + logger.Error(err, "Fail to update the network access of Autonomous Database. opcWorkRequestID = "+*resp.OpcWorkRequestId) } } return nil } -func UpdateStatusAndWait(logger logr.Logger, kubeClient client.Client, +func UpdateAutonomousDatabaseStatusAndWait(logger logr.Logger, kubeClient client.Client, workClient workrequests.WorkRequestClient, adb *dbv1alpha1.AutonomousDatabase, desiredLifecycleState database.AutonomousDatabaseLifecycleStateEnum, @@ -321,7 +320,7 @@ func UpdateStatusAndWait(logger logr.Logger, kubeClient client.Client, return statusErr } - if err := oci.WaitUntilWorkCompleted(logger, workClient, opcWorkRequestID); err != nil { + if _, err := oci.GetWorkStatusAndWait(logger, workClient, opcWorkRequestID); err != nil { return err } diff --git a/commons/autonomousdatabase/adb_restore_reconciler_util.go b/commons/autonomousdatabase/adb_restore_reconciler_util.go index cf70f244..6083575f 100644 --- a/commons/autonomousdatabase/adb_restore_reconciler_util.go +++ b/commons/autonomousdatabase/adb_restore_reconciler_util.go @@ -40,14 +40,87 @@ package autonomousdatabase import ( "context" + "errors" "k8s.io/apimachinery/pkg/types" "k8s.io/client-go/util/retry" "sigs.k8s.io/controller-runtime/pkg/client" + "github.com/go-logr/logr" + "github.com/oracle/oci-go-sdk/v51/common" + "github.com/oracle/oci-go-sdk/v51/database" + "github.com/oracle/oci-go-sdk/v51/workrequests" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + "github.com/oracle/oracle-database-operator/commons/oci" + "github.com/oracle/oracle-database-operator/commons/oci/ociutil" ) +func RestoreAutonomousDatabaseAndWait(logger logr.Logger, + kubeClient client.Client, + dbClient database.DatabaseClient, + workClient workrequests.WorkRequestClient, + restore *dbv1alpha1.AutonomousDatabaseRestore) (lifecycleState workrequests.WorkRequestStatusEnum, err error) { + + var restoreTime *common.SDKTime + var adbOCID string + + if restore.Spec.BackupName != "" { + backup := &dbv1alpha1.AutonomousDatabaseBackup{} + namespacedName := types.NamespacedName{Namespace: restore.Namespace, Name: restore.Spec.BackupName} + if err := kubeClient.Get(context.TODO(), namespacedName, backup); err != nil { + return "", err + } + + if backup.Status.TimeEnded == "" { + return "", errors.New("broken backup: ended time is missing in the AutonomousDatabaseBackup " + backup.GetName()) + } + restoreTime, err = ociutil.ParseDisplayTime(backup.Status.TimeEnded) + if err != nil { + return "", err + } + + adbOCID = backup.Status.AutonomousDatabaseOCID + + } else if restore.Spec.PointInTime.TimeStamp != "" { + // The validation of the pitr.timestamp has been handled by the webhook, so the error return is ignored + restoreTime, _ = ociutil.ParseDisplayTime(restore.Spec.PointInTime.TimeStamp) + adbOCID = restore.Spec.PointInTime.AutonomousDatabaseOCID + } + + resp, err := oci.RestoreAutonomousDatabase(dbClient, adbOCID, restoreTime) + if err != nil { + logger.Error(err, "Fail to restore Autonomous Database") + return "", nil + } + + // Update status and wait for the work finish if a request is sent. Note that some of the requests (e.g. update displayName) won't return a work request ID. + // It's important to update the status by reference otherwise Reconcile() won't be able to get the latest values + status := &restore.Status + status.DisplayName = *resp.AutonomousDatabase.DisplayName + status.DbName = *resp.AutonomousDatabase.DbName + status.AutonomousDatabaseOCID = *resp.AutonomousDatabase.Id + + workResp, err := oci.GetWorkRequest(workClient, resp.OpcWorkRequestId, nil) + if err != nil { + logger.Error(err, "Fail to get the work status. opcWorkRequestID = "+*resp.OpcWorkRequestId) + return workResp.Status, nil + } + status.LifecycleState = workResp.Status + + UpdateAutonomousDatabaseRestoreStatus(kubeClient, restore) + + // Wait until the work is done + lifecycleState, err = oci.GetWorkStatusAndWait(logger, workClient, resp.OpcWorkRequestId) + if err != nil { + logger.Error(err, "Fail to restore Autonomous Database. opcWorkRequestID = "+*resp.OpcWorkRequestId) + return lifecycleState, nil + } + + logger.Info("Restoration of Autonomous Database" + *resp.DisplayName + " finished") + + return lifecycleState, nil +} + // UpdateAutonomousDatabaseBackupStatus updates the status subresource of AutonomousDatabaseBackup func UpdateAutonomousDatabaseRestoreStatus(kubeClient client.Client, adbRestore *dbv1alpha1.AutonomousDatabaseRestore) error { return retry.RetryOnConflict(retry.DefaultRetry, func() error { diff --git a/commons/oci/database.go b/commons/oci/database.go index 91769eec..46d33a39 100644 --- a/commons/oci/database.go +++ b/commons/oci/database.go @@ -146,11 +146,7 @@ func getValueFromKubeSecret(kubeClient client.Client, namespacedName types.Names // GetAutonomousDatabaseResource gets Autonomous Database information from a remote instance // and return an AutonomousDatabase object func GetAutonomousDatabaseResource(logger logr.Logger, dbClient database.DatabaseClient, adb *dbv1alpha1.AutonomousDatabase) (*dbv1alpha1.AutonomousDatabase, error) { - getAutonomousDatabaseRequest := database.GetAutonomousDatabaseRequest{ - AutonomousDatabaseId: adb.Spec.Details.AutonomousDatabaseOCID, - } - - response, err := dbClient.GetAutonomousDatabase(context.TODO(), getAutonomousDatabaseRequest) + response, err := GetAutonomousDatabase(dbClient, adb.Spec.Details.AutonomousDatabaseOCID) if err != nil { return nil, err } @@ -161,6 +157,14 @@ func GetAutonomousDatabaseResource(logger logr.Logger, dbClient database.Databas return returnedADB, nil } +func GetAutonomousDatabase(dbClient database.DatabaseClient, adbOCID *string) (database.GetAutonomousDatabaseResponse, error) { + getAutonomousDatabaseRequest := database.GetAutonomousDatabaseRequest{ + AutonomousDatabaseId: adbOCID, + } + + return dbClient.GetAutonomousDatabase(context.TODO(), getAutonomousDatabaseRequest) +} + // IsAttrChanged checks if the values of last successful object and current object are different. // The function returns false if the types are mismatch or unknown. // The function returns false if the current object has zero value (not applicable for boolean type). @@ -614,24 +618,35 @@ func GetAutonomousDatabaseBackup(dbClient database.DatabaseClient, backupOCID st return dbClient.GetAutonomousDatabaseBackup(context.TODO(), getBackupRequest) } -func WaitUntilWorkCompleted(logger logr.Logger, workClient workrequests.WorkRequestClient, opcWorkRequestID *string) error { +func GetWorkStatusAndWait(logger logr.Logger, workClient workrequests.WorkRequestClient, opcWorkRequestID *string) (workrequests.WorkRequestStatusEnum, error) { logger.Info("Waiting for the work request to finish. opcWorkRequestID = " + *opcWorkRequestID) + // retries until the work status is SUCCEEDED, FAILED or CANCELED retryPolicy := getCompleteWorkRetryPolicy() - // Apply wait until work complete retryPolicy + + resp, err := GetWorkRequest(workClient, opcWorkRequestID, &retryPolicy) + if err != nil { + return resp.Status, err + } + + return resp.Status, nil +} + +func GetWorkRequest(workClient workrequests.WorkRequestClient, + opcWorkRequestID *string, + retryPolicy *common.RetryPolicy) (response workrequests.GetWorkRequestResponse, err error) { + workRequest := workrequests.GetWorkRequestRequest{ WorkRequestId: opcWorkRequestID, - RequestMetadata: common.RequestMetadata{ - RetryPolicy: &retryPolicy, - }, } - // GetWorkRequest retries until the work status is SUCCEEDED - if _, err := workClient.GetWorkRequest(context.TODO(), workRequest); err != nil { - return err + if (retryPolicy != nil) { + workRequest.RequestMetadata = common.RequestMetadata{ + RetryPolicy: retryPolicy, + } } - return nil + return workClient.GetWorkRequest(context.TODO(), workRequest) } func getCompleteWorkRetryPolicy() common.RetryPolicy { @@ -656,8 +671,8 @@ func getCompleteWorkRetryPolicy() common.RetryPolicy { } func getRetryPolicy(retryOperation func(common.OCIOperationResponse) bool) common.RetryPolicy { - // maximum times of retry (~15mins) - attempts := uint(33) + // maximum times of retry (~30mins) + attempts := uint(63) nextDuration := func(r common.OCIOperationResponse) time.Duration { // Wait longer for next retry when your previous one failed @@ -665,9 +680,8 @@ func getRetryPolicy(retryOperation func(common.OCIOperationResponse) bool) commo // 1s, 2s, 4s, 8s, 16s, 30s, 30s etc... if r.AttemptNumber <= 5 { return time.Duration(math.Pow(float64(2), float64(r.AttemptNumber-1))) * time.Second - } else { - return time.Duration(30) * time.Second } + return time.Duration(30) * time.Second } return common.NewRetryPolicy(attempts, retryOperation, nextDuration) diff --git a/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml b/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml index 2eae5408..200c8ccb 100644 --- a/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml +++ b/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml @@ -23,6 +23,9 @@ spec: - jsonPath: .status.lifecycleState name: State type: string + - jsonPath: .status.dbDisplayName + name: DB DisplayName + type: string - jsonPath: .status.type name: Type type: string @@ -69,29 +72,29 @@ spec: secretName: type: string type: object - required: - - autonomousDatabaseOCID type: object status: description: AutonomousDatabaseBackupStatus defines the observed state of AutonomousDatabaseBackup properties: autonomousDatabaseBackupOCID: - description: 'INSERT ADDITIONAL STATUS FIELD - define observed state - of cluster Important: Run "make" to regenerate code after modifying - this file' type: string autonomousDatabaseOCID: type: string compartmentOCID: type: string + dbDisplayName: + type: string + dbName: + type: string displayName: type: string isAutomatic: type: boolean lifecycleState: - description: 'AutonomousDatabaseBackupLifecycleStateEnum Enum with - underlying type: string' + description: 'INSERT ADDITIONAL STATUS FIELD - define observed state + of cluster Important: Run "make" to regenerate code after modifying + this file' type: string timeEnded: type: string @@ -105,6 +108,8 @@ spec: - autonomousDatabaseBackupOCID - autonomousDatabaseOCID - compartmentOCID + - dbDisplayName + - dbName - displayName - isAutomatic - lifecycleState diff --git a/config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml b/config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml index e852e906..5b298640 100644 --- a/config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml +++ b/config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml @@ -19,7 +19,17 @@ spec: singular: autonomousdatabaserestore scope: Namespaced versions: - - name: v1alpha1 + - additionalPrinterColumns: + - jsonPath: .status.lifecycleState + name: State + type: string + - jsonPath: .status.displayName + name: DisplayName + type: string + - jsonPath: .status.dbName + name: DbName + type: string + name: v1alpha1 schema: openAPIV3Schema: description: AutonomousDatabaseRestore is the Schema for the autonomousdatabaserestores @@ -41,7 +51,7 @@ spec: description: AutonomousDatabaseRestoreSpec defines the desired state of AutonomousDatabaseRestore properties: - autonomousDatabaseOCID: + backupName: description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster Important: Run "make" to regenerate code after modifying this file' type: string @@ -52,33 +62,35 @@ spec: secretName: type: string type: object - source: + pointInTime: properties: - backupName: + autonomousDatabaseOCID: type: string timeStamp: type: string type: object - required: - - autonomousDatabaseOCID - - source type: object status: description: AutonomousDatabaseRestoreStatus defines the observed state of AutonomousDatabaseRestore properties: - state: + autonomousDatabaseOCID: + type: string + dbName: + type: string + displayName: description: 'INSERT ADDITIONAL STATUS FIELD - define observed state of cluster Important: Run "make" to regenerate code after modifying this file' - enum: - - NewCreated - - InProgress - - Completed - - Failed + type: string + lifecycleState: + description: 'WorkRequestStatusEnum Enum with underlying type: string' type: string required: - - state + - autonomousDatabaseOCID + - dbName + - displayName + - lifecycleState type: object type: object served: true diff --git a/config/manager/kustomization.yaml b/config/manager/kustomization.yaml index 30ed1f75..d8632b06 100644 --- a/config/manager/kustomization.yaml +++ b/config/manager/kustomization.yaml @@ -8,5 +8,5 @@ apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization images: - name: controller - newName: container-registry.oracle.com/database/operator - newTag: 0.1.0 + newName: local-oracle-db-operator + newTag: v0.0.1beta diff --git a/config/manager/manager.yaml b/config/manager/manager.yaml index 362aac61..259b79a5 100644 --- a/config/manager/manager.yaml +++ b/config/manager/manager.yaml @@ -34,7 +34,7 @@ spec: args: - --enable-leader-election image: controller:latest - imagePullPolicy: Always + imagePullPolicy: Never #Always name: manager resources: limits: diff --git a/config/samples/adb/autonomousdatabase_restore.yaml b/config/samples/adb/autonomousdatabase_restore.yaml index 1485bec7..401b03a2 100644 --- a/config/samples/adb/autonomousdatabase_restore.yaml +++ b/config/samples/adb/autonomousdatabase_restore.yaml @@ -3,13 +3,15 @@ kind: AutonomousDatabaseRestore metadata: name: autonomousdatabaserestore-sample spec: - autonomousDatabaseOCID: ocid1.autonomousdatabase... # Restore the database either from a backup or using point-in-time restore - source: - # The name of your AutonomousDatabaseBackup - backupName: autonomousdatabasebackup-sample - # The timestamp must follow this format: YYYY-MM-DD HH:MM:SS GMT - # timeStamp: 2021-10-23 11:03:13 UTC + # The name of your AutonomousDatabaseBackup resource + backupName: autonomousdatabasebackup-sample + + # Uncomment the following block to perform point-in-time restore + # pointInTime: + # autonomousDatabaseOCID: ocid1.autonomousdatabase... + # # The timestamp must follow this format: YYYY-MM-DD HH:MM:SS GMT + # timeStamp: 2021-10-23 11:03:13 UTC # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. ociConfig: diff --git a/config/webhook/manifests.yaml b/config/webhook/manifests.yaml index c635bcc2..be41e601 100644 --- a/config/webhook/manifests.yaml +++ b/config/webhook/manifests.yaml @@ -26,6 +26,26 @@ webhooks: resources: - autonomousdatabase sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: webhook-service + namespace: system + path: /mutate-database-oracle-com-v1alpha1-autonomousdatabasebackup + failurePolicy: Fail + name: mautonomousdatabasebackup.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + resources: + - autonomousdatabasebackups + sideEffects: None - admissionReviewVersions: - v1 - v1beta1 @@ -76,6 +96,46 @@ webhooks: resources: - autonomousdatabases sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: webhook-service + namespace: system + path: /validate-database-oracle-com-v1alpha1-autonomousdatabasebackup + failurePolicy: Fail + name: vautonomousdatabasebackup.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + resources: + - autonomousdatabasebackups + sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: webhook-service + namespace: system + path: /validate-database-oracle-com-v1alpha1-autonomousdatabaserestore + failurePolicy: Fail + name: vautonomousdatabaserestore.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + resources: + - autonomousdatabaserestores + sideEffects: None - admissionReviewVersions: - v1 - v1beta1 diff --git a/controllers/database/autonomousdatabase_controller.go b/controllers/database/autonomousdatabase_controller.go index 04be67ee..7d552d4b 100644 --- a/controllers/database/autonomousdatabase_controller.go +++ b/controllers/database/autonomousdatabase_controller.go @@ -81,8 +81,8 @@ func (r *AutonomousDatabaseReconciler) SetupWithManager(mgr ctrl.Manager) error func (r *AutonomousDatabaseReconciler) eventFilterPredicate() predicate.Predicate { pred := predicate.Funcs{ UpdateFunc: func(e event.UpdateEvent) bool { - oldADB := e.ObjectOld.DeepCopyObject().(*dbv1alpha1.AutonomousDatabase) - newADB := e.ObjectNew.DeepCopyObject().(*dbv1alpha1.AutonomousDatabase) + oldADB := e.ObjectOld.(*dbv1alpha1.AutonomousDatabase) + newADB := e.ObjectNew.(*dbv1alpha1.AutonomousDatabase) // Reconciliation should NOT happen if the lastSuccessfulSpec annotation or status.state changes. oldSucSpec := oldADB.GetAnnotations()[dbv1alpha1.LastSuccessfulSpec] @@ -123,9 +123,10 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R if err := r.KubeClient.Get(context.TODO(), req.NamespacedName, adb); err != nil { // Ignore not-found errors, since they can't be fixed by an immediate requeue. // No need to change the since we don't know if we obtain the object. - if !apiErrors.IsNotFound(err) { - return ctrl.Result{}, err + if apiErrors.IsNotFound(err) { + return ctrl.Result{}, nil } + return ctrl.Result{}, err } /****************************************************************** @@ -265,9 +266,9 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R adb.Spec.Details.AutonomousDatabaseOCID = resp.AutonomousDatabase.Id - if err := adbutil.UpdateStatusAndWait(currentLogger, r.KubeClient, workClient, adb, + if err := adbutil.UpdateAutonomousDatabaseStatusAndWait(currentLogger, r.KubeClient, workClient, adb, resp.AutonomousDatabase.LifecycleState, resp.OpcWorkRequestId); err != nil { - currentLogger.Error(err, "Fail to watch the status of provision request. opcWorkRequestID = "+*resp.OpcWorkRequestId) + currentLogger.Error(err, "Work request faied. opcWorkRequestID = "+*resp.OpcWorkRequestId) } currentLogger.Info("AutonomousDatabase " + *adb.Spec.Details.DbName + " provisioned succesfully") @@ -339,8 +340,8 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R return ctrl.Result{}, nil } - if err := adbutil.UpdateStatusAndWait(currentLogger, r.KubeClient, workClient, adb, lifecycleState, opcWorkRequestID); err != nil { - currentLogger.Error(err, "Fail to watch the status of work request. opcWorkRequestID = "+*opcWorkRequestID) + if err := adbutil.UpdateAutonomousDatabaseStatusAndWait(currentLogger, r.KubeClient, workClient, adb, lifecycleState, opcWorkRequestID); err != nil { + currentLogger.Error(err, "Fail to update the status of Autonomous Database. opcWorkRequestID = "+*opcWorkRequestID) } currentLogger.Info(fmt.Sprintf("Change AutonomousDatabase %s lifecycle state to %s successfully\n", @@ -351,18 +352,15 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Update the database in OCI from the local resource. // The local resource will be synchronized again later. if err := adbutil.UpdateGeneralAndPasswordAttributesAndWait(currentLogger, r.KubeClient, dbClient, secretClient, workClient, adb); err != nil { - currentLogger.Error(err, "Fail to update Autonomous Database") - return ctrl.Result{}, nil + currentLogger.Error(err, "Fail to update Autonomous Database general and password attributes") } if err := adbutil.UpdateScaleAttributesAndWait(currentLogger, r.KubeClient, dbClient, workClient, adb); err != nil { - currentLogger.Error(err, "Fail to update Autonomous Database") - return ctrl.Result{}, nil + currentLogger.Error(err, "Fail to scale Autonomous Database") } if err := adbutil.UpdateNetworkAttributes(currentLogger, r.KubeClient, dbClient, workClient, adb); err != nil { - currentLogger.Error(err, "Fail to update Autonomous Database") - return ctrl.Result{}, nil + currentLogger.Error(err, "Fail to update Autonomous Database network access attributes") } } } @@ -434,7 +432,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R return ctrl.Result{}, err } - currentLogger.Info("AutonomousDatabase resource reconcile successfully") + currentLogger.Info("AutonomousDatabase reconciles successfully") return ctrl.Result{}, nil } diff --git a/controllers/database/autonomousdatabasebackup_controller.go b/controllers/database/autonomousdatabasebackup_controller.go index ab0ab904..30790226 100644 --- a/controllers/database/autonomousdatabasebackup_controller.go +++ b/controllers/database/autonomousdatabasebackup_controller.go @@ -111,9 +111,10 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req if err := r.KubeClient.Get(context.TODO(), req.NamespacedName, adbBackup); err != nil { // Ignore not-found errors, since they can't be fixed by an immediate requeue. // No need to change the since we don't know if we obtain the object. - if !apiErrors.IsNotFound(err) { - return ctrl.Result{}, err + if apiErrors.IsNotFound(err) { + return ctrl.Result{}, nil } + return ctrl.Result{}, err } /****************************************************************** @@ -167,23 +168,29 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req ******************************************************************/ if adbBackup.Spec.AutonomousDatabaseBackupOCID == "" && adbBackup.Status.LifecycleState == "" { // Create a new backup - resp, err := oci.CreateAutonomousDatabaseBackup(currentLogger, dbClient, adbBackup) + backupResp, err := oci.CreateAutonomousDatabaseBackup(currentLogger, dbClient, adbBackup) if err != nil { currentLogger.Error(err, "Fail to create AutonomousDatabase Backup") return ctrl.Result{}, nil } + adbResp, err := oci.GetAutonomousDatabase(dbClient, backupResp.AutonomousDatabaseId) + if err != nil { + currentLogger.Error(err, "Fail to get AutonomousDatabase. The AutonomousDatabase OCID = "+*backupResp.AutonomousDatabaseId) + return ctrl.Result{}, nil + } + // update the status - adbBackup.UpdateStatusFromAutonomousDatabaseBackupResponse(resp.AutonomousDatabaseBackup) + adbBackup.UpdateStatusFromAutonomousDatabaseBackupResponse(backupResp.AutonomousDatabaseBackup, adbResp.AutonomousDatabase) backupUtil.UpdateAutonomousDatabaseBackupStatus(r.KubeClient, adbBackup) // Wait until the work is done - if err := oci.WaitUntilWorkCompleted(currentLogger, workClient, resp.OpcWorkRequestId); err != nil { - currentLogger.Error(err, "Fail to watch the status of provision request. opcWorkRequestID = "+*resp.OpcWorkRequestId) + if _, err := oci.GetWorkStatusAndWait(currentLogger, workClient, backupResp.OpcWorkRequestId); err != nil { + currentLogger.Error(err, "Work request faied. opcWorkRequestID = "+*backupResp.OpcWorkRequestId) return ctrl.Result{}, nil } - currentLogger.Info("AutonomousBackup " + *resp.DisplayName + " created successfully") + currentLogger.Info("AutonomousDatabaseBackup " + *backupResp.DisplayName + " created successfully") } else if adbBackup.Spec.AutonomousDatabaseBackupOCID != "" { // Bind to an existing backup @@ -195,13 +202,20 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req * Status.AutonomousDatabaseOCID isn't empty. ******************************************************************/ if adbBackup.Status.AutonomousDatabaseBackupOCID != "" { - resp, err := oci.GetAutonomousDatabaseBackup(dbClient, adbBackup.Status.AutonomousDatabaseBackupOCID) + backupResp, err := oci.GetAutonomousDatabaseBackup(dbClient, adbBackup.Status.AutonomousDatabaseBackupOCID) if err != nil { currentLogger.Error(err, "Fail to get AutonomousDatabase Backup. The AutonomousDatabase Backup OCID = "+adbBackup.Status.AutonomousDatabaseBackupOCID) return ctrl.Result{}, nil } - adbBackup.UpdateStatusFromAutonomousDatabaseBackupResponse(resp.AutonomousDatabaseBackup) + adbResp, err := oci.GetAutonomousDatabase(dbClient, backupResp.AutonomousDatabaseId) + if err != nil { + currentLogger.Error(err, "Fail to get AutonomousDatabase. The AutonomousDatabase OCID = "+*backupResp.AutonomousDatabaseId) + return ctrl.Result{}, nil + } + + + adbBackup.UpdateStatusFromAutonomousDatabaseBackupResponse(backupResp.AutonomousDatabaseBackup, adbResp.AutonomousDatabase) backupUtil.UpdateAutonomousDatabaseBackupStatus(r.KubeClient, adbBackup) } diff --git a/controllers/database/autonomousdatabaserestore_controller.go b/controllers/database/autonomousdatabaserestore_controller.go index 358cae90..0b318155 100644 --- a/controllers/database/autonomousdatabaserestore_controller.go +++ b/controllers/database/autonomousdatabaserestore_controller.go @@ -44,18 +44,15 @@ import ( "github.com/go-logr/logr" apiErrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/runtime" - "k8s.io/apimachinery/pkg/types" ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" - "github.com/oracle/oci-go-sdk/v54/common" "github.com/oracle/oci-go-sdk/v54/database" "github.com/oracle/oci-go-sdk/v54/workrequests" databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" restoreUtil "github.com/oracle/oracle-database-operator/commons/autonomousdatabase" "github.com/oracle/oracle-database-operator/commons/oci" - "github.com/oracle/oracle-database-operator/commons/oci/ociutil" ) // AutonomousDatabaseRestoreReconciler reconciles a AutonomousDatabaseRestore object @@ -83,10 +80,11 @@ func (r *AutonomousDatabaseRestoreReconciler) Reconcile(ctx context.Context, req restore := &dbv1alpha1.AutonomousDatabaseRestore{} if err := r.KubeClient.Get(context.TODO(), req.NamespacedName, restore); err != nil { // Ignore not-found errors, since they can't be fixed by an immediate requeue. - // No need to change the since we don't know if we obtain the object. - if !apiErrors.IsNotFound(err) { - return ctrl.Result{}, err + // No need to change since we don't know if we obtain the object. + if apiErrors.IsNotFound(err) { + return ctrl.Result{}, nil } + return ctrl.Result{}, err } /****************************************************************** @@ -102,7 +100,7 @@ func (r *AutonomousDatabaseRestoreReconciler) Reconcile(ctx context.Context, req currentLogger.Error(err, "Fail to get OCI provider") // Change the status to UNAVAILABLE - restore.Status.LifecycleState = dbv1alpha1.RestoreLifecycleStateFailed + restore.Status.LifecycleState = workrequests.WorkRequestStatusFailed if statusErr := restoreUtil.UpdateAutonomousDatabaseRestoreStatus(r.KubeClient, restore); statusErr != nil { return ctrl.Result{}, statusErr } @@ -114,7 +112,7 @@ func (r *AutonomousDatabaseRestoreReconciler) Reconcile(ctx context.Context, req currentLogger.Error(err, "Fail to get OCI database client") // Change the status to UNAVAILABLE - restore.Status.LifecycleState = dbv1alpha1.RestoreLifecycleStateFailed + restore.Status.LifecycleState = workrequests.WorkRequestStatusFailed if statusErr := restoreUtil.UpdateAutonomousDatabaseRestoreStatus(r.KubeClient, restore); statusErr != nil { return ctrl.Result{}, statusErr } @@ -126,7 +124,7 @@ func (r *AutonomousDatabaseRestoreReconciler) Reconcile(ctx context.Context, req currentLogger.Error(err, "Fail to get OCI work request client") // Change the status to UNAVAILABLE - restore.Status.LifecycleState = dbv1alpha1.RestoreLifecycleStateFailed + restore.Status.LifecycleState = workrequests.WorkRequestStatusFailed if statusErr := restoreUtil.UpdateAutonomousDatabaseRestoreStatus(r.KubeClient, restore); statusErr != nil { return ctrl.Result{}, statusErr } @@ -136,72 +134,24 @@ func (r *AutonomousDatabaseRestoreReconciler) Reconcile(ctx context.Context, req /****************************************************************** * Restore ******************************************************************/ - if restore.Status.LifecycleState == "" || restore.Status.LifecycleState == dbv1alpha1.RestoreLifecycleStateNew { - var restoreTime *common.SDKTime - - if restore.Spec.Source.BackupName != "" { - backup := &dbv1alpha1.AutonomousDatabaseBackup{} - namespacedName := types.NamespacedName{Namespace: restore.Namespace, Name: restore.Spec.Source.BackupName} - if err := r.KubeClient.Get(context.TODO(), namespacedName, backup); err != nil { - return ctrl.Result{}, err - } - - restoreTime, err = ociutil.ParseDisplayTime(backup.Status.TimeEnded) - if err != nil { - currentLogger.Error(err, "Fail to parse time "+backup.Status.TimeEnded) - - // Change the status to UNAVAILABLE - restore.Status.LifecycleState = dbv1alpha1.RestoreLifecycleStateFailed - if statusErr := restoreUtil.UpdateAutonomousDatabaseRestoreStatus(r.KubeClient, restore); statusErr != nil { - return ctrl.Result{}, statusErr - } - return ctrl.Result{}, nil - } - } else if restore.Spec.Source.TimeStamp != "" { - restoreTime, err = ociutil.ParseDisplayTime(restore.Spec.Source.TimeStamp) - if err != nil { - currentLogger.Error(err, "Fail to parse time "+restore.Spec.Source.TimeStamp) - - // Change the status to UNAVAILABLE - restore.Status.LifecycleState = dbv1alpha1.RestoreLifecycleStateFailed - if statusErr := restoreUtil.UpdateAutonomousDatabaseRestoreStatus(r.KubeClient, restore); statusErr != nil { - return ctrl.Result{}, statusErr - } - return ctrl.Result{}, nil - } - } - - resp, err := oci.RestoreAutonomousDatabase(dbClient, restore.Spec.AutonomousDatabaseOCID, restoreTime) + if restore.Status.LifecycleState == "" { + lifecycleState, err := restoreUtil.RestoreAutonomousDatabaseAndWait(currentLogger, r.KubeClient, dbClient, workClient, restore) if err != nil { currentLogger.Error(err, "Fail to restore database") - - // Change the status to UNAVAILABLE - restore.Status.LifecycleState = dbv1alpha1.RestoreLifecycleStateFailed - if statusErr := restoreUtil.UpdateAutonomousDatabaseRestoreStatus(r.KubeClient, restore); statusErr != nil { - return ctrl.Result{}, statusErr - } return ctrl.Result{}, nil } - if err := oci.WaitUntilWorkCompleted(currentLogger, workClient, resp.OpcWorkRequestId); err != nil { - currentLogger.Error(err, "Fail to watch workrequest. Workrequest ID = "+*resp.OpcWorkRequestId) + restore.Status.LifecycleState = lifecycleState - // Change the status to UNAVAILABLE - restore.Status.LifecycleState = dbv1alpha1.RestoreLifecycleStateFailed - if statusErr := restoreUtil.UpdateAutonomousDatabaseRestoreStatus(r.KubeClient, restore); statusErr != nil { - return ctrl.Result{}, statusErr - } - return ctrl.Result{}, nil - } - - currentLogger.Info("Restore database completed") - - restore.Status.LifecycleState = dbv1alpha1.RestoreLifecycleStateCompleted if statusErr := restoreUtil.UpdateAutonomousDatabaseRestoreStatus(r.KubeClient, restore); statusErr != nil { - return ctrl.Result{}, statusErr + // No need to return the error since it will re-execute the Reconcile() + currentLogger.Error(err, "Fail to update the status of AutonomousDatabaseRestore") + return ctrl.Result{}, nil } } + currentLogger.Info("AutonomousDatabaseRestore reconciles successfully") + return ctrl.Result{}, nil } diff --git a/main.go b/main.go index 09668a91..5da93064 100644 --- a/main.go +++ b/main.go @@ -153,6 +153,14 @@ func main() { setupLog.Error(err, "unable to create webhook", "webhook", "AutonomousDatabase") os.Exit(1) } + if err = (&databasev1alpha1.AutonomousDatabaseBackup{}).SetupWebhookWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create webhook", "webhook", "AutonomousDatabaseBackup") + os.Exit(1) + } + if err = (&databasev1alpha1.AutonomousDatabaseRestore{}).SetupWebhookWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create webhook", "webhook", "AutonomousDatabaseRestore") + os.Exit(1) + } // +kubebuilder:scaffold:builder setupLog.Info("starting manager") diff --git a/test/e2e/resource/test_config.yaml b/test/e2e/resource/test_config.yaml index 4a5f8027..67e5777c 100644 --- a/test/e2e/resource/test_config.yaml +++ b/test/e2e/resource/test_config.yaml @@ -2,15 +2,12 @@ # Copyright (c) 2021, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # - -# The path of your OCI configuration file, which usually is ~/.oci/config ociConfigFile: ~/.oci/config -# The profile you want to use in the test (Optional) profile: DEFAULT -# Compartment OCID where the database creates -compartmentOCID: ocid1.compartment.. -# The OCID of the OCI Vault Secret that holds the password of the ADMIN account (should start with ocid1.vaultsecret...) -adminPasswordOCID: ocid1.vaultsecret... -# The OCID of the OCI Vault Secret that holds the password of the wallet (should start with ocid1.vaultsecret...) -instanceWalletPasswordOCID: ocid1.vaultsecret... \ No newline at end of file +plainTextAdminPassword: Welcome_1234 +plainTextWalletPassword: Welcome_1234 + +compartmentOCID: ocid1.compartment.oc1..aaaaaaaa4rwyyueehc64fgs5q5tliyi44xjo3tm7zsuezvhyanuboztlce7q +adminPasswordOCID: ocid1.vaultsecret.oc1.phx.amaaaaaadxiv6saan2rk3uhki5zq5byxkmz7ac3jale2mr5fmcuyjatg34ya +instanceWalletPasswordOCID: ocid1.vaultsecret.oc1.phx.amaaaaaadxiv6saan2rk3uhki5zq5byxkmz7ac3jale2mr5fmcuyjatg34ya \ No newline at end of file From 1a76b9267285cfca1a77edde9f5f85584e769faf Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Thu, 10 Feb 2022 11:04:05 -0500 Subject: [PATCH 153/628] restructure network apis and add webhooks --- .../v1alpha1/autonomousdatabaserestore_types.go | 8 ++++---- .../adb_restore_reconciler_util.go | 12 ++++++------ commons/oci/database.go | 2 +- .../autonomousdatabasebackup_controller.go | 1 - test/e2e/resource/test_config.yaml | 15 +++++++++------ 5 files changed, 20 insertions(+), 18 deletions(-) diff --git a/apis/database/v1alpha1/autonomousdatabaserestore_types.go b/apis/database/v1alpha1/autonomousdatabaserestore_types.go index a4c1652a..6ea2b852 100644 --- a/apis/database/v1alpha1/autonomousdatabaserestore_types.go +++ b/apis/database/v1alpha1/autonomousdatabaserestore_types.go @@ -39,7 +39,7 @@ package v1alpha1 import ( - "github.com/oracle/oci-go-sdk/v51/workrequests" + "github.com/oracle/oci-go-sdk/v54/workrequests" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" ) @@ -64,9 +64,9 @@ type PITSource struct { type AutonomousDatabaseRestoreStatus struct { // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster // Important: Run "make" to regenerate code after modifying this file - DisplayName string `json:"displayName"` - DbName string `json:"dbName"` - AutonomousDatabaseOCID string `json:"autonomousDatabaseOCID"` + DisplayName string `json:"displayName"` + DbName string `json:"dbName"` + AutonomousDatabaseOCID string `json:"autonomousDatabaseOCID"` LifecycleState workrequests.WorkRequestStatusEnum `json:"lifecycleState"` } diff --git a/commons/autonomousdatabase/adb_restore_reconciler_util.go b/commons/autonomousdatabase/adb_restore_reconciler_util.go index 6083575f..447d8289 100644 --- a/commons/autonomousdatabase/adb_restore_reconciler_util.go +++ b/commons/autonomousdatabase/adb_restore_reconciler_util.go @@ -47,9 +47,9 @@ import ( "sigs.k8s.io/controller-runtime/pkg/client" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v51/common" - "github.com/oracle/oci-go-sdk/v51/database" - "github.com/oracle/oci-go-sdk/v51/workrequests" + "github.com/oracle/oci-go-sdk/v54/common" + "github.com/oracle/oci-go-sdk/v54/database" + "github.com/oracle/oci-go-sdk/v54/workrequests" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" "github.com/oracle/oracle-database-operator/commons/oci" "github.com/oracle/oracle-database-operator/commons/oci/ociutil" @@ -99,14 +99,14 @@ func RestoreAutonomousDatabaseAndWait(logger logr.Logger, status.DisplayName = *resp.AutonomousDatabase.DisplayName status.DbName = *resp.AutonomousDatabase.DbName status.AutonomousDatabaseOCID = *resp.AutonomousDatabase.Id - + workResp, err := oci.GetWorkRequest(workClient, resp.OpcWorkRequestId, nil) if err != nil { logger.Error(err, "Fail to get the work status. opcWorkRequestID = "+*resp.OpcWorkRequestId) return workResp.Status, nil } status.LifecycleState = workResp.Status - + UpdateAutonomousDatabaseRestoreStatus(kubeClient, restore) // Wait until the work is done @@ -138,4 +138,4 @@ func UpdateAutonomousDatabaseRestoreStatus(kubeClient client.Client, adbRestore curBackup.Status = adbRestore.Status return kubeClient.Status().Update(context.TODO(), curBackup) }) -} \ No newline at end of file +} diff --git a/commons/oci/database.go b/commons/oci/database.go index 46d33a39..1fc35e94 100644 --- a/commons/oci/database.go +++ b/commons/oci/database.go @@ -640,7 +640,7 @@ func GetWorkRequest(workClient workrequests.WorkRequestClient, WorkRequestId: opcWorkRequestID, } - if (retryPolicy != nil) { + if retryPolicy != nil { workRequest.RequestMetadata = common.RequestMetadata{ RetryPolicy: retryPolicy, } diff --git a/controllers/database/autonomousdatabasebackup_controller.go b/controllers/database/autonomousdatabasebackup_controller.go index 30790226..ce977b67 100644 --- a/controllers/database/autonomousdatabasebackup_controller.go +++ b/controllers/database/autonomousdatabasebackup_controller.go @@ -214,7 +214,6 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req return ctrl.Result{}, nil } - adbBackup.UpdateStatusFromAutonomousDatabaseBackupResponse(backupResp.AutonomousDatabaseBackup, adbResp.AutonomousDatabase) backupUtil.UpdateAutonomousDatabaseBackupStatus(r.KubeClient, adbBackup) } diff --git a/test/e2e/resource/test_config.yaml b/test/e2e/resource/test_config.yaml index 67e5777c..4a5f8027 100644 --- a/test/e2e/resource/test_config.yaml +++ b/test/e2e/resource/test_config.yaml @@ -2,12 +2,15 @@ # Copyright (c) 2021, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # + +# The path of your OCI configuration file, which usually is ~/.oci/config ociConfigFile: ~/.oci/config +# The profile you want to use in the test (Optional) profile: DEFAULT -plainTextAdminPassword: Welcome_1234 -plainTextWalletPassword: Welcome_1234 - -compartmentOCID: ocid1.compartment.oc1..aaaaaaaa4rwyyueehc64fgs5q5tliyi44xjo3tm7zsuezvhyanuboztlce7q -adminPasswordOCID: ocid1.vaultsecret.oc1.phx.amaaaaaadxiv6saan2rk3uhki5zq5byxkmz7ac3jale2mr5fmcuyjatg34ya -instanceWalletPasswordOCID: ocid1.vaultsecret.oc1.phx.amaaaaaadxiv6saan2rk3uhki5zq5byxkmz7ac3jale2mr5fmcuyjatg34ya \ No newline at end of file +# Compartment OCID where the database creates +compartmentOCID: ocid1.compartment.. +# The OCID of the OCI Vault Secret that holds the password of the ADMIN account (should start with ocid1.vaultsecret...) +adminPasswordOCID: ocid1.vaultsecret... +# The OCID of the OCI Vault Secret that holds the password of the wallet (should start with ocid1.vaultsecret...) +instanceWalletPasswordOCID: ocid1.vaultsecret... \ No newline at end of file From 364befc1cad5f732bd85c36779704f547677bcc7 Mon Sep 17 00:00:00 2001 From: Gajanan Bhat Date: Fri, 11 Feb 2022 16:21:24 +0530 Subject: [PATCH 154/628] PDB Webhook changes --- apis/database/v1alpha1/cdb_webhook.go | 21 ++- apis/database/v1alpha1/pdb_webhook.go | 7 +- config/webhook/manifests.yaml | 1 - controllers/database/pdb_controller.go | 15 +- oracle-database-operator.yaml | 80 ++++++---- ...utonomousdatabase_controller_suite_test.go | 145 ------------------ 6 files changed, 89 insertions(+), 180 deletions(-) delete mode 100644 test/e2e/autonomousdatabase_controller_suite_test.go diff --git a/apis/database/v1alpha1/cdb_webhook.go b/apis/database/v1alpha1/cdb_webhook.go index 34983b78..49922a93 100644 --- a/apis/database/v1alpha1/cdb_webhook.go +++ b/apis/database/v1alpha1/cdb_webhook.go @@ -110,11 +110,11 @@ func (r *CDB) ValidateCreate() error { } if r.Spec.ORDSPort < 0 { allErrs = append(allErrs, - field.Required(field.NewPath("spec").Child("dbPort"), "Please specify a valid ORDS Port")) + field.Required(field.NewPath("spec").Child("ordsPort"), "Please specify a valid ORDS Port")) } if r.Spec.Replicas < 0 { allErrs = append(allErrs, - field.Required(field.NewPath("spec").Child("dbPort"), "Please specify a valid value for Replicas")) + field.Required(field.NewPath("spec").Child("replicas"), "Please specify a valid value for Replicas")) } if r.Spec.ORDSImage == "" { allErrs = append(allErrs, @@ -152,6 +152,11 @@ func (r *CDB) ValidateCreate() error { func (r *CDB) ValidateUpdate(old runtime.Object) error { cdblog.Info("validate update", "name", r.Name) + isCDBMarkedToBeDeleted := r.GetDeletionTimestamp() != nil + if isCDBMarkedToBeDeleted { + return nil + } + var allErrs field.ErrorList // Check for updation errors @@ -160,6 +165,18 @@ func (r *CDB) ValidateUpdate(old runtime.Object) error { return nil } + if r.Spec.DBPort < 0 { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("dbPort"), "Please specify a valid DB Server Port")) + } + if r.Spec.ORDSPort < 0 { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("ordsPort"), "Please specify a valid ORDS Port")) + } + if r.Spec.Replicas < 0 { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("replicas"), "Please specify a valid value for Replicas")) + } if !strings.EqualFold(oldCDB.Spec.ServiceName, r.Spec.ServiceName) { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("replics"), "cannot be changed")) diff --git a/apis/database/v1alpha1/pdb_webhook.go b/apis/database/v1alpha1/pdb_webhook.go index a0e988ab..12ea899a 100644 --- a/apis/database/v1alpha1/pdb_webhook.go +++ b/apis/database/v1alpha1/pdb_webhook.go @@ -112,7 +112,7 @@ func (r *PDB) Default() { } // TODO(user): change verbs to "verbs=create;update;delete" if you want to enable deletion validation. -//+kubebuilder:webhook:path=/validate-database-oracle-com-v1alpha1-pdb,mutating=false,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=pdbs,verbs=create;update;delete,versions=v1alpha1,name=vpdb.kb.io,admissionReviewVersions={v1,v1beta1} +//+kubebuilder:webhook:path=/validate-database-oracle-com-v1alpha1-pdb,mutating=false,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=pdbs,verbs=create;update,versions=v1alpha1,name=vpdb.kb.io,admissionReviewVersions={v1,v1beta1} var _ webhook.Validator = &PDB{} @@ -218,6 +218,11 @@ func (r *PDB) validateAction(allErrs *field.ErrorList) { func (r *PDB) ValidateUpdate(old runtime.Object) error { pdblog.Info("ValidateUpdate-Validating PDB spec for : " + r.Name) + isPDBMarkedToBeDeleted := r.GetDeletionTimestamp() != nil + if isPDBMarkedToBeDeleted { + return nil + } + var allErrs field.ErrorList action := strings.ToUpper(r.Spec.Action) diff --git a/config/webhook/manifests.yaml b/config/webhook/manifests.yaml index 7e27a61a..200e81fb 100644 --- a/config/webhook/manifests.yaml +++ b/config/webhook/manifests.yaml @@ -116,7 +116,6 @@ webhooks: operations: - CREATE - UPDATE - - DELETE resources: - pdbs sideEffects: None diff --git a/controllers/database/pdb_controller.go b/controllers/database/pdb_controller.go index 1941be21..b96b3f4a 100644 --- a/controllers/database/pdb_controller.go +++ b/controllers/database/pdb_controller.go @@ -175,8 +175,19 @@ func (r *PDBReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.R action := strings.ToUpper(pdb.Spec.Action) - if (pdb.Status.Phase == pdbPhaseReady) && (pdb.Status.Action != "") && (action == "MODIFY" || action == "STATUS" || pdb.Status.Action != action) { - pdb.Status.Status = false + if pdb.Status.Phase == pdbPhaseReady { + if (pdb.Status.Action != "") && (action == "MODIFY" || action == "STATUS" || pdb.Status.Action != action) { + pdb.Status.Status = false + } else { + err = r.getPDBState(ctx, req, pdb) + if err != nil { + pdb.Status.Phase = pdbPhaseFail + } else { + pdb.Status.Phase = pdbPhaseReady + pdb.Status.Msg = "Success" + } + r.Status().Update(ctx, pdb) + } } if !pdb.Status.Status { diff --git a/oracle-database-operator.yaml b/oracle-database-operator.yaml index 3f113e15..72d50548 100644 --- a/oracle-database-operator.yaml +++ b/oracle-database-operator.yaml @@ -40,23 +40,32 @@ spec: name: v1alpha1 schema: openAPIV3Schema: - description: AutonomousDatabaseBackup is the Schema for the autonomousdatabasebackups API + description: AutonomousDatabaseBackup is the Schema for the autonomousdatabasebackups + API properties: apiVersion: - description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: - description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: - description: AutonomousDatabaseBackupSpec defines the desired state of AutonomousDatabaseBackup + description: AutonomousDatabaseBackupSpec defines the desired state of + AutonomousDatabaseBackup properties: autonomousDatabaseBackupOCID: type: string autonomousDatabaseOCID: - description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster Important: Run "make" to regenerate code after modifying this file' + description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster + Important: Run "make" to regenerate code after modifying this file' + type: string + displayName: type: string ociConfig: properties: @@ -69,31 +78,39 @@ spec: - autonomousDatabaseOCID type: object status: - description: AutonomousDatabaseBackupStatus defines the observed state of AutonomousDatabaseBackup + description: AutonomousDatabaseBackupStatus defines the observed state + of AutonomousDatabaseBackup properties: autonomousDatabaseBackupOCID: - description: 'INSERT ADDITIONAL STATUS FIELD - define observed state of cluster Important: Run "make" to regenerate code after modifying this file' + description: 'INSERT ADDITIONAL STATUS FIELD - define observed state + of cluster Important: Run "make" to regenerate code after modifying + this file' type: string autonomousDatabaseOCID: type: string compartmentOCID: type: string + displayName: + type: string isAutomatic: type: boolean lifecycleState: - description: 'AutonomousDatabaseBackupLifecycleStateEnum Enum with underlying type: string' + description: 'AutonomousDatabaseBackupLifecycleStateEnum Enum with + underlying type: string' type: string timeEnded: type: string timeStarted: type: string type: - description: 'AutonomousDatabaseBackupTypeEnum Enum with underlying type: string' + description: 'AutonomousDatabaseBackupTypeEnum Enum with underlying + type: string' type: string required: - autonomousDatabaseBackupOCID - autonomousDatabaseOCID - compartmentOCID + - displayName - isAutomatic - lifecycleState - type @@ -132,45 +149,55 @@ spec: - name: v1alpha1 schema: openAPIV3Schema: - description: AutonomousDatabaseRestore is the Schema for the autonomousdatabaserestores API + description: AutonomousDatabaseRestore is the Schema for the autonomousdatabaserestores + API properties: apiVersion: - description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: - description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: - description: AutonomousDatabaseRestoreSpec defines the desired state of AutonomousDatabaseRestore + description: AutonomousDatabaseRestoreSpec defines the desired state of + AutonomousDatabaseRestore properties: autonomousDatabaseOCID: - description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster Important: Run "make" to regenerate code after modifying this file' + description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster + Important: Run "make" to regenerate code after modifying this file' type: string - destination: + ociConfig: properties: - backupName: + configMapName: type: string - timeStamp: + secretName: type: string type: object - ociConfig: + source: properties: - configMapName: + backupName: type: string - secretName: + timeStamp: type: string type: object required: - autonomousDatabaseOCID - - destination + - source type: object status: - description: AutonomousDatabaseRestoreStatus defines the observed state of AutonomousDatabaseRestore + description: AutonomousDatabaseRestoreStatus defines the observed state + of AutonomousDatabaseRestore properties: state: - description: 'INSERT ADDITIONAL STATUS FIELD - define observed state of cluster Important: Run "make" to regenerate code after modifying this file' + description: 'INSERT ADDITIONAL STATUS FIELD - define observed state + of cluster Important: Run "make" to regenerate code after modifying + this file' enum: - NewCreated - InProgress @@ -1893,8 +1920,6 @@ rules: - delete - get - list - - patch - - update - watch - apiGroups: - database.oracle.com @@ -1913,8 +1938,6 @@ rules: - delete - get - list - - patch - - update - watch - apiGroups: - database.oracle.com @@ -2175,7 +2198,7 @@ spec: command: - /manager image: phx.ocir.io/intsanjaysingh/gdb-repo/oracle/dboper-ords-go:master - imagePullPolicy: Always + imagePullPolicy: Never name: manager ports: - containerPort: 9443 @@ -2340,7 +2363,6 @@ webhooks: operations: - CREATE - UPDATE - - DELETE resources: - pdbs sideEffects: None diff --git a/test/e2e/autonomousdatabase_controller_suite_test.go b/test/e2e/autonomousdatabase_controller_suite_test.go deleted file mode 100644 index adc6706f..00000000 --- a/test/e2e/autonomousdatabase_controller_suite_test.go +++ /dev/null @@ -1,145 +0,0 @@ -/* -** Copyright (c) 2021 Oracle and/or its affiliates. -** -** The Universal Permissive License (UPL), Version 1.0 -** -** Subject to the condition set forth below, permission is hereby granted to any -** person obtaining a copy of this software, associated documentation and/or data -** (collectively the "Software"), free of charge and under any and all copyright -** rights in the Software, and any and all patent rights owned or freely -** licensable by each licensor hereunder covering either (i) the unmodified -** Software as contributed to or provided by such licensor, or (ii) the Larger -** Works (as defined below), to deal in both -** -** (a) the Software, and -** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if -** one is included with the Software (each a "Larger Work" to which the Software -** is contributed by such licensors), -** -** without restriction, including without limitation the rights to copy, create -** derivative works of, display, perform, and distribute the Software and make, -** use, sell, offer for sale, import, export, have made, and have sold the -** Software and the Larger Work(s), and to sublicense the foregoing rights on -** either these or other terms. -** -** This license is subject to the following condition: -** The above copyright notice and either this complete permission notice or at -** a minimum a reference to the UPL must be included in all copies or -** substantial portions of the Software. -** -** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -** SOFTWARE. - */ - -package e2etest - -import ( - "context" - - . "github.com/onsi/ginkgo" - . "github.com/onsi/gomega" - "github.com/oracle/oci-go-sdk/v45/common" - "github.com/oracle/oci-go-sdk/v45/database" - "sigs.k8s.io/controller-runtime/pkg/client" - - "github.com/oracle/oracle-database-operator/test/e2e/behavior" - "github.com/oracle/oracle-database-operator/test/e2e/util" - // +kubebuilder:scaffold:imports -) - -// These tests use Ginkgo (BDD-style Go testing framework). Refer to -// http://onsi.github.io/ginkgo/ to learn more about Ginkgo. - -/** -This test suite runs the integration test which checks the following scenario -1. Init the test by creating utililty variables and objects -2. Test ADB provisioned using adminPasswordOCID with hardLink=true -3. Test ADB provisioned using adminPasswordSecret with hardLink=true -5. Test ADB binding with hardLink=true -**/ - -var dbClient database.DatabaseClient -var configProvider common.ConfigurationProvider - -const configFileName = "test_config.yaml" -const ADBNamespace string = "default" - -var SharedOCIConfigMapName = "oci-cred" -var SharedOCISecretName = "oci-privatekey" -var SharedPlainTextAdminPassword = "Welcome_1234" -var SharedPlainTextWalletPassword = "Welcome_1234" -var SharedCompartmentOCID string - -var SharedKeyOCID string -var SharedAdminPasswordOCID string -var SharedInstanceWalletPasswordOCID string - -const SharedAdminPassSecretName string = "adb-admin-password" -const SharedWalletPassSecretName = "adb-wallet-password" - -func ADBSetup(k8sClient client.Client) { - /************************************************** - * Custom codes for autonomousdatabase controller - **************************************************/ - By("init the test by creating utililty variables and objects") - testConfig, err := e2eutil.GetTestConfig(configFileName) - Expect(err).ToNot(HaveOccurred()) - Expect(testConfig).ToNot(BeNil()) - - SharedCompartmentOCID = testConfig.CompartmentOCID - SharedAdminPasswordOCID = testConfig.AdminPasswordOCID - SharedInstanceWalletPasswordOCID = testConfig.InstanceWalletPasswordOCID - - By("checking if the required parameters exist") - Expect(testConfig.OCIConfigFile).ToNot(Equal("")) - Expect(testConfig.CompartmentOCID).ToNot(Equal("")) - Expect(testConfig.AdminPasswordOCID).ToNot(Equal("")) - Expect(testConfig.InstanceWalletPasswordOCID).ToNot(Equal("")) - - By("getting OCI provider") - ociConfigUtil, err := e2eutil.GetOCIConfigUtil(testConfig.OCIConfigFile, testConfig.Profile) - Expect(err).ToNot(HaveOccurred()) - configProvider, err = ociConfigUtil.GetConfigProvider() - Expect(err).ToNot(HaveOccurred()) - - By("creating a OCI DB client") - dbClient, err = database.NewDatabaseClientWithConfigurationProvider(configProvider) - Expect(err).ToNot(HaveOccurred()) - - By("creating a configMap for calling OCI") - ociConfigMap, err := ociConfigUtil.CreateOCIConfigMap(ADBNamespace, SharedOCIConfigMapName) - Expect(err).ToNot(HaveOccurred()) - Expect(k8sClient.Create(context.TODO(), ociConfigMap)).To(Succeed()) - - By("creating a secret for calling OCI") - ociSecret, err := ociConfigUtil.CreateOCISecret(ADBNamespace, SharedOCISecretName) - Expect(err).ToNot(HaveOccurred()) - Expect(k8sClient.Create(context.TODO(), ociSecret)).To(Succeed()) - - By("Creating a k8s secret to hold admin password", func() { - data := map[string]string{ - SharedAdminPassSecretName: SharedPlainTextAdminPassword, - } - adminSecret, err := e2eutil.CreateKubeSecret(ADBNamespace, SharedAdminPassSecretName, data) - Expect(err).ToNot(HaveOccurred()) - Expect(k8sClient.Create(context.TODO(), adminSecret)).To(Succeed()) - }) - - By("Creating a k8s secret to hold wallet password", func() { - data := map[string]string{ - SharedWalletPassSecretName: SharedPlainTextWalletPassword, - } - walletSecret, err := e2eutil.CreateKubeSecret(ADBNamespace, SharedWalletPassSecretName, data) - Expect(err).ToNot(HaveOccurred()) - Expect(k8sClient.Create(context.TODO(), walletSecret)).To(Succeed()) - }) -} - -func CleanupADB(k8sClient *client.Client) { - e2ebehavior.CleanupDB(k8sClient, &dbClient, ADBNamespace) -} From 94df4a51e27fe66eb66bfb1e6bb8c65f0b5e40fe Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Fri, 11 Feb 2022 13:56:39 -0500 Subject: [PATCH 155/628] undo the changes to ImagePullPolicy --- config/manager/kustomization.yaml | 4 ++-- config/manager/manager.yaml | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/config/manager/kustomization.yaml b/config/manager/kustomization.yaml index d8632b06..c42eb919 100644 --- a/config/manager/kustomization.yaml +++ b/config/manager/kustomization.yaml @@ -8,5 +8,5 @@ apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization images: - name: controller - newName: local-oracle-db-operator - newTag: v0.0.1beta + newName: container-registry.oracle.com/database/operator + newTag: 0.1.0 \ No newline at end of file diff --git a/config/manager/manager.yaml b/config/manager/manager.yaml index 259b79a5..362aac61 100644 --- a/config/manager/manager.yaml +++ b/config/manager/manager.yaml @@ -34,7 +34,7 @@ spec: args: - --enable-leader-election image: controller:latest - imagePullPolicy: Never #Always + imagePullPolicy: Always name: manager resources: limits: From ed5ca9123d9e1ac1cdfd0fb451915e7cf140c965 Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Sat, 12 Feb 2022 21:44:44 +0530 Subject: [PATCH 156/628] Added openshift support --- .../v1alpha1/oraclerestdataservice_types.go | 21 +++--- .../v1alpha1/singleinstancedatabase_types.go | 1 + ...ase.oracle.com_oraclerestdataservices.yaml | 2 + ...se.oracle.com_singleinstancedatabases.yaml | 2 + config/samples/sidb/openshift_rbac.yaml | 70 +++++++++++++++++++ .../samples/sidb/oraclerestdataservice.yaml | 3 + .../samples/sidb/singleinstancedatabase.yaml | 3 + .../sidb/singleinstancedatabase_clone.yaml | 3 + .../sidb/singleinstancedatabase_patch.yaml | 3 + .../sidb/singleinstancedatabase_prov.yaml | 3 + ...ngleinstancedatabase_prov_prebuilt_db.yaml | 3 + .../oraclerestdataservice_controller.go | 7 +- .../singleinstancedatabase_controller.go | 7 +- 13 files changed, 116 insertions(+), 12 deletions(-) create mode 100644 config/samples/sidb/openshift_rbac.yaml diff --git a/apis/database/v1alpha1/oraclerestdataservice_types.go b/apis/database/v1alpha1/oraclerestdataservice_types.go index 334005a7..bc26d46c 100644 --- a/apis/database/v1alpha1/oraclerestdataservice_types.go +++ b/apis/database/v1alpha1/oraclerestdataservice_types.go @@ -50,16 +50,17 @@ type OracleRestDataServiceSpec struct { // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster // Important: Run "make" to regenerate code after modifying this file - DatabaseRef string `json:"databaseRef"` - LoadBalancer bool `json:"loadBalancer,omitempty"` - NodeSelector map[string]string `json:"nodeSelector,omitempty"` - Image OracleRestDataServiceImage `json:"image,omitempty"` - OrdsPassword OracleRestDataServicePassword `json:"ordsPassword"` - ApexPassword OracleRestDataServicePassword `json:"apexPassword,omitempty"` - AdminPassword OracleRestDataServicePassword `json:"adminPassword"` - OrdsUser string `json:"ordsUser,omitempty"` - RestEnableSchemas []OracleRestDataServiceRestEnableSchemas `json:"restEnableSchemas,omitempty"` - OracleService string `json:"oracleService,omitempty"` + DatabaseRef string `json:"databaseRef"` + LoadBalancer bool `json:"loadBalancer,omitempty"` + NodeSelector map[string]string `json:"nodeSelector,omitempty"` + Image OracleRestDataServiceImage `json:"image,omitempty"` + OrdsPassword OracleRestDataServicePassword `json:"ordsPassword"` + ApexPassword OracleRestDataServicePassword `json:"apexPassword,omitempty"` + AdminPassword OracleRestDataServicePassword `json:"adminPassword"` + OrdsUser string `json:"ordsUser,omitempty"` + RestEnableSchemas []OracleRestDataServiceRestEnableSchemas `json:"restEnableSchemas,omitempty"` + OracleService string `json:"oracleService,omitempty"` + ServiceAccountName string `json:"serviceAccountName,omitempty"` // +k8s:openapi-gen=true // +kubebuilder:validation:Minimum=1 diff --git a/apis/database/v1alpha1/singleinstancedatabase_types.go b/apis/database/v1alpha1/singleinstancedatabase_types.go index cdd02a6a..5415646d 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_types.go +++ b/apis/database/v1alpha1/singleinstancedatabase_types.go @@ -67,6 +67,7 @@ type SingleInstanceDatabaseSpec struct { CloneFrom string `json:"cloneFrom,omitempty"` ReadinessCheckPeriod int `json:"readinessCheckPeriod,omitempty"` + ServiceAccountName string `json:"serviceAccountName,omitempty"` // +k8s:openapi-gen=true // +kubebuilder:validation:Minimum=1 diff --git a/config/crd/bases/database.oracle.com_oraclerestdataservices.yaml b/config/crd/bases/database.oracle.com_oraclerestdataservices.yaml index 418b9370..f1a49357 100644 --- a/config/crd/bases/database.oracle.com_oraclerestdataservices.yaml +++ b/config/crd/bases/database.oracle.com_oraclerestdataservices.yaml @@ -139,6 +139,8 @@ spec: - schema type: object type: array + serviceAccountName: + type: string required: - adminPassword - databaseRef diff --git a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml index c19470eb..f1ee3a4e 100644 --- a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml +++ b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml @@ -144,6 +144,8 @@ spec: replicas: minimum: 1 type: integer + serviceAccountName: + type: string sid: description: SID can only have a-z , A-Z, 0-9 . It cant have any special characters diff --git a/config/samples/sidb/openshift_rbac.yaml b/config/samples/sidb/openshift_rbac.yaml new file mode 100644 index 00000000..30fde8fb --- /dev/null +++ b/config/samples/sidb/openshift_rbac.yaml @@ -0,0 +1,70 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +--- +# Create a Security Context Contraint + + kind: SecurityContextConstraints + apiVersion: v1 + metadata: + name: sidb-scc + namespace: default + allowPrivilegedContainer: false + runAsUser: + type: MustRunAsRange + uidRangeMin: 0 + uidRangeMax: 60000 + seLinuxContext: + type: RunAsAny + fsGroup: + type: MustRunAs + ranges: + - min: 0 + max: 60000 + supplementalGroups: + type: MustRunAs + ranges: + - min: 0 + max: 60000 + +--- +# Create Service Account + +apiVersion: v1 +kind: ServiceAccount +metadata: + name: sidb-sa + namespace: default + +--- +# Create a rbac role + + kind: Role + apiVersion: rbac.authorization.k8s.io/v1 + metadata: + name: use-sidb-scc + namespace: default + rules: + - apiGroups: ["security.openshift.io"] + resources: ["securitycontextconstraints"] + resourceNames: ["sidb-scc"] + verbs: ["use"] + +--- +# Create a rbac role binding + + kind: RoleBinding + apiVersion: rbac.authorization.k8s.io/v1 + metadata: + name: use-sidb-scc + namespace: default + subjects: + - kind: ServiceAccount + name: sidb-sa + roleRef: + kind: Role + name: use-sidb-scc + apiGroup: rbac.authorization.k8s.io + \ No newline at end of file diff --git a/config/samples/sidb/oraclerestdataservice.yaml b/config/samples/sidb/oraclerestdataservice.yaml index 4c3400f6..e916e93f 100644 --- a/config/samples/sidb/oraclerestdataservice.yaml +++ b/config/samples/sidb/oraclerestdataservice.yaml @@ -49,3 +49,6 @@ spec: enable: true urlMapping: pdb: + + ## If deploying on OpenShift, change service account name to 'sidb-sa' after you run `$ oc apply -f openshift_rbac.yaml` + ServiceAccountname: default \ No newline at end of file diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index 218d0ed5..b8e69869 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -82,5 +82,8 @@ spec: # failure-domain.beta.kubernetes.io/zone: bVCG:PHX-AD-1 # pool: sidb + ## If deploying on OpenShift, change service account name to 'sidb-sa' after you run `$ oc apply -f openshift_rbac.yaml` + ServiceAccountname: default + ## Count of Database Pods. Applicable only for "ReadWriteMany" AccessMode replicas: 1 diff --git a/config/samples/sidb/singleinstancedatabase_clone.yaml b/config/samples/sidb/singleinstancedatabase_clone.yaml index fe6d0556..fe25fe7f 100644 --- a/config/samples/sidb/singleinstancedatabase_clone.yaml +++ b/config/samples/sidb/singleinstancedatabase_clone.yaml @@ -38,5 +38,8 @@ spec: storageClass: "" accessMode: "ReadWriteMany" + ## If deploying on OpenShift, change service account name to 'sidb-sa' after you run `$ oc apply -f openshift_rbac.yaml` + ServiceAccountname: default + ## Count of Database Pods. Applicable only for "ReadWriteMany" AccessMode replicas: 1 diff --git a/config/samples/sidb/singleinstancedatabase_patch.yaml b/config/samples/sidb/singleinstancedatabase_patch.yaml index dbcc7fc1..3ae0e085 100644 --- a/config/samples/sidb/singleinstancedatabase_patch.yaml +++ b/config/samples/sidb/singleinstancedatabase_patch.yaml @@ -32,5 +32,8 @@ spec: storageClass: "" accessMode: "ReadWriteMany" + ## If deploying on OpenShift, change service account name to 'sidb-sa' after you run `$ oc apply -f openshift_rbac.yaml` + ServiceAccountname: default + ## Count of Database Pods. Applicable only for "ReadWriteMany" AccessMode replicas: 1 diff --git a/config/samples/sidb/singleinstancedatabase_prov.yaml b/config/samples/sidb/singleinstancedatabase_prov.yaml index 612a53d5..cbed7221 100644 --- a/config/samples/sidb/singleinstancedatabase_prov.yaml +++ b/config/samples/sidb/singleinstancedatabase_prov.yaml @@ -31,5 +31,8 @@ spec: storageClass: "" accessMode: "ReadWriteMany" + ## If deploying on OpenShift, change service account name to 'sidb-sa' after you run `$ oc apply -f openshift_rbac.yaml` + ServiceAccountname: default + ## Count of Database Pods. Applicable only for "ReadWriteMany" AccessMode replicas: 1 diff --git a/config/samples/sidb/singleinstancedatabase_prov_prebuilt_db.yaml b/config/samples/sidb/singleinstancedatabase_prov_prebuilt_db.yaml index e7b2b02b..f8eaa587 100644 --- a/config/samples/sidb/singleinstancedatabase_prov_prebuilt_db.yaml +++ b/config/samples/sidb/singleinstancedatabase_prov_prebuilt_db.yaml @@ -14,6 +14,9 @@ spec: pullFrom: pullSecrets: + ## If deploying on OpenShift, change service account name to 'sidb-sa' after you run `$ oc apply -f openshift_rbac.yaml` + ServiceAccountname: default + ## Type of service . Applicable on cloud enviroments only ## if loadBalService : false, service type = "NodePort". else "LoadBalancer" loadBalancer: false \ No newline at end of file diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 0759f13e..310e29f0 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -516,7 +516,12 @@ func (r *OracleRestDataServiceReconciler) instantiatePodSpec(m *dbapi.OracleRest } return ns }(), - + ServiceAccountName: func() string { + if m.Spec.ServiceAccountName != "" { + return m.Spec.ServiceAccountName + } + return "default" + }(), SecurityContext: &corev1.PodSecurityContext{ RunAsUser: func() *int64 { i := int64(dbcommons.ORACLE_UID); return &i }(), RunAsGroup: func() *int64 { i := int64(dbcommons.DBA_GUID); return &i }(), diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index c31bba53..c7e535b0 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -509,7 +509,12 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns } return ns }(), - + ServiceAccountName: func() string { + if m.Spec.ServiceAccountName != "" { + return m.Spec.ServiceAccountName + } + return "default" + }(), SecurityContext: &corev1.PodSecurityContext{ RunAsUser: func() *int64 { i := int64(0) From ffd9ce1427811aa8b30bb7dc402fdff4b38bf369 Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Sat, 12 Feb 2022 21:48:24 +0530 Subject: [PATCH 157/628] Added gitlab-ci file --- .gitlab-ci.yml | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) create mode 100644 .gitlab-ci.yml diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml new file mode 100644 index 00000000..e4d80bb9 --- /dev/null +++ b/.gitlab-ci.yml @@ -0,0 +1,27 @@ +build-operator: + stage: build + variables: + IMAGE: "$DOCKER_REPO:$CI_COMMIT_BRANCH" + OP_YAML: oracle-database-operator.yaml + script: + - make docker-build IMG="$IMAGE" + - docker push "$IMAGE" + - newimage=$(docker inspect $IMAGE | python -c 'import json,sys; print json.load(sys.stdin)[0]["RepoDigests"][0]') + - echo $newimage + - docker rmi "$IMAGE" && docker system prune -f + - make operator-yaml IMG=$newimage + - curl -s -n $ARTIFACTORY_REPO/$CI_COMMIT_BRANCH/$OP_YAML -T ./$OP_YAML + only: + variables: + - $CI_COMMIT_MESSAGE =~ /\#run-pipeline/ + - $CI_COMMIT_BRANCH =~ /master/ + - $CI_MERGE_REQUEST_ID != "" + except: + variables: + - $CI_COMMIT_MESSAGE =~ /\#skip-pipeline/ + +cleanup: + stage: .post + script: + - echo "Clean up downloaded binaries" + - rm -rf bin/ From 261f3d8b365e3f88c3dc7e2f7c7b5c24a4a28d93 Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Sat, 12 Feb 2022 21:57:57 +0530 Subject: [PATCH 158/628] change keepSecrets to true --- config/samples/sidb/oraclerestdataservice.yaml | 6 +++--- config/samples/sidb/singleinstancedatabase.yaml | 2 +- config/samples/sidb/singleinstancedatabase_clone.yaml | 2 +- config/samples/sidb/singleinstancedatabase_patch.yaml | 2 +- config/samples/sidb/singleinstancedatabase_prov.yaml | 4 ++-- 5 files changed, 8 insertions(+), 8 deletions(-) diff --git a/config/samples/sidb/oraclerestdataservice.yaml b/config/samples/sidb/oraclerestdataservice.yaml index e916e93f..19da6a11 100644 --- a/config/samples/sidb/oraclerestdataservice.yaml +++ b/config/samples/sidb/oraclerestdataservice.yaml @@ -16,14 +16,14 @@ spec: adminPassword: secretName: secretKey: - keepSecret: false + keepSecret: true ## Secret containing ORDS_PUBLIC_USER password mapped to secretKey. ## This secret will be deleted after ORDS Installation unless keepSecret set to true. ordsPassword: secretName: secretKey: - keepSecret: false + keepSecret: true ## Secret containing a common password for APEX_PUBLIC_USER, APEX_REST_PUBLIC_USER, APEX_LISTENER and Apex administrator (username: ADMIN) mapped to secretKey ## Mention a non-null string for apexPassword.secretName to configure APEX with ORDS. @@ -31,7 +31,7 @@ spec: apexPassword: secretName: secretKey: - keepSecret: false + keepSecret: true ## ORDS image details image: diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index b8e69869..c0cfa358 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -25,7 +25,7 @@ spec: adminPassword: secretName: secretKey: - keepSecret: false + keepSecret: true ## NA if cloning from a SourceDB (cloneFrom is set) charset: AL32UTF8 diff --git a/config/samples/sidb/singleinstancedatabase_clone.yaml b/config/samples/sidb/singleinstancedatabase_clone.yaml index fe25fe7f..4fd75582 100644 --- a/config/samples/sidb/singleinstancedatabase_clone.yaml +++ b/config/samples/sidb/singleinstancedatabase_clone.yaml @@ -22,7 +22,7 @@ spec: adminPassword: secretName: secretKey: - keepSecret: false + keepSecret: true ## Database image details ## Database can be patched out of place by updating the RU version/image diff --git a/config/samples/sidb/singleinstancedatabase_patch.yaml b/config/samples/sidb/singleinstancedatabase_patch.yaml index 3ae0e085..54c015d5 100644 --- a/config/samples/sidb/singleinstancedatabase_patch.yaml +++ b/config/samples/sidb/singleinstancedatabase_patch.yaml @@ -17,7 +17,7 @@ spec: adminPassword: secretName: secretKey: - keepSecret: false + keepSecret: true ## Patch the database by updating the RU version/image ## Major version changes are not supported diff --git a/config/samples/sidb/singleinstancedatabase_prov.yaml b/config/samples/sidb/singleinstancedatabase_prov.yaml index cbed7221..5ae6638f 100644 --- a/config/samples/sidb/singleinstancedatabase_prov.yaml +++ b/config/samples/sidb/singleinstancedatabase_prov.yaml @@ -17,7 +17,7 @@ spec: adminPassword: secretName: secretKey: - keepSecret: false + keepSecret: true ## Database Image image: @@ -33,6 +33,6 @@ spec: ## If deploying on OpenShift, change service account name to 'sidb-sa' after you run `$ oc apply -f openshift_rbac.yaml` ServiceAccountname: default - + ## Count of Database Pods. Applicable only for "ReadWriteMany" AccessMode replicas: 1 From 2f5cac5a63fc05e0600819f12852abd185645749 Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Sat, 12 Feb 2022 22:40:34 +0530 Subject: [PATCH 159/628] updated go version --- .../v1alpha1/zz_generated.deepcopy.go | 1 - config/manager/kustomization.yaml | 4 +- oracle-database-operator.yaml | 197 +----------------- 3 files changed, 7 insertions(+), 195 deletions(-) diff --git a/apis/database/v1alpha1/zz_generated.deepcopy.go b/apis/database/v1alpha1/zz_generated.deepcopy.go index 42bfd573..3e6716ac 100644 --- a/apis/database/v1alpha1/zz_generated.deepcopy.go +++ b/apis/database/v1alpha1/zz_generated.deepcopy.go @@ -1,4 +1,3 @@ -//go:build !ignore_autogenerated // +build !ignore_autogenerated /* diff --git a/config/manager/kustomization.yaml b/config/manager/kustomization.yaml index 11f7656c..30ed1f75 100644 --- a/config/manager/kustomization.yaml +++ b/config/manager/kustomization.yaml @@ -8,5 +8,5 @@ apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization images: - name: controller - newName: phx.ocir.io/oracassandra/operator/prebuiltdb - newTag: test-0.1.2 + newName: container-registry.oracle.com/database/operator + newTag: 0.1.0 diff --git a/oracle-database-operator.yaml b/oracle-database-operator.yaml index a9d2f5d4..48117e04 100644 --- a/oracle-database-operator.yaml +++ b/oracle-database-operator.yaml @@ -7,193 +7,6 @@ metadata: --- apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition -metadata: - annotations: - controller-gen.kubebuilder.io/version: v0.6.1 - creationTimestamp: null - name: autonomousdatabasebackups.database.oracle.com -spec: - group: database.oracle.com - names: - kind: AutonomousDatabaseBackup - listKind: AutonomousDatabaseBackupList - plural: autonomousdatabasebackups - shortNames: - - adbbu - - adbbus - singular: autonomousdatabasebackup - scope: Namespaced - versions: - - additionalPrinterColumns: - - jsonPath: .status.lifecycleState - name: State - type: string - - jsonPath: .status.type - name: Type - type: string - - jsonPath: .status.timeStarted - name: Started - type: string - - jsonPath: .status.timeEnded - name: Ended - type: string - name: v1alpha1 - schema: - openAPIV3Schema: - description: AutonomousDatabaseBackup is the Schema for the autonomousdatabasebackups API - properties: - apiVersion: - description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' - type: string - kind: - description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' - type: string - metadata: - type: object - spec: - description: AutonomousDatabaseBackupSpec defines the desired state of AutonomousDatabaseBackup - properties: - autonomousDatabaseBackupOCID: - type: string - autonomousDatabaseOCID: - description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster Important: Run "make" to regenerate code after modifying this file' - type: string - ociConfig: - properties: - configMapName: - type: string - secretName: - type: string - type: object - required: - - autonomousDatabaseOCID - type: object - status: - description: AutonomousDatabaseBackupStatus defines the observed state of AutonomousDatabaseBackup - properties: - autonomousDatabaseBackupOCID: - description: 'INSERT ADDITIONAL STATUS FIELD - define observed state of cluster Important: Run "make" to regenerate code after modifying this file' - type: string - autonomousDatabaseOCID: - type: string - compartmentOCID: - type: string - isAutomatic: - type: boolean - lifecycleState: - description: 'AutonomousDatabaseBackupLifecycleStateEnum Enum with underlying type: string' - type: string - timeEnded: - type: string - timeStarted: - type: string - type: - description: 'AutonomousDatabaseBackupTypeEnum Enum with underlying type: string' - type: string - required: - - autonomousDatabaseBackupOCID - - autonomousDatabaseOCID - - compartmentOCID - - isAutomatic - - lifecycleState - - type - type: object - type: object - served: true - storage: true - subresources: - status: {} -status: - acceptedNames: - kind: "" - plural: "" - conditions: [] - storedVersions: [] ---- -apiVersion: apiextensions.k8s.io/v1 -kind: CustomResourceDefinition -metadata: - annotations: - controller-gen.kubebuilder.io/version: v0.6.1 - creationTimestamp: null - name: autonomousdatabaserestores.database.oracle.com -spec: - group: database.oracle.com - names: - kind: AutonomousDatabaseRestore - listKind: AutonomousDatabaseRestoreList - plural: autonomousdatabaserestores - shortNames: - - adbr - - adbrs - singular: autonomousdatabaserestore - scope: Namespaced - versions: - - name: v1alpha1 - schema: - openAPIV3Schema: - description: AutonomousDatabaseRestore is the Schema for the autonomousdatabaserestores API - properties: - apiVersion: - description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' - type: string - kind: - description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' - type: string - metadata: - type: object - spec: - description: AutonomousDatabaseRestoreSpec defines the desired state of AutonomousDatabaseRestore - properties: - autonomousDatabaseOCID: - description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster Important: Run "make" to regenerate code after modifying this file' - type: string - destination: - properties: - backupName: - type: string - timeStamp: - type: string - type: object - ociConfig: - properties: - configMapName: - type: string - secretName: - type: string - type: object - required: - - autonomousDatabaseOCID - - destination - type: object - status: - description: AutonomousDatabaseRestoreStatus defines the observed state of AutonomousDatabaseRestore - properties: - state: - description: 'INSERT ADDITIONAL STATUS FIELD - define observed state of cluster Important: Run "make" to regenerate code after modifying this file' - enum: - - NewCreated - - InProgress - - Completed - - Failed - type: string - required: - - state - type: object - type: object - served: true - storage: true - subresources: - status: {} -status: - acceptedNames: - kind: "" - plural: "" - conditions: [] - storedVersions: [] ---- -apiVersion: apiextensions.k8s.io/v1 -kind: CustomResourceDefinition metadata: annotations: controller-gen.kubebuilder.io/version: v0.6.1 @@ -496,6 +309,8 @@ spec: - schema type: object type: array + serviceAccountName: + type: string required: - adminPassword - databaseRef @@ -1077,6 +892,8 @@ spec: replicas: minimum: 1 type: integer + serviceAccountName: + type: string sid: description: SID can only have a-z , A-Z, 0-9 . It cant have any special characters pattern: ^[a-zA-Z0-9]+$ @@ -1391,8 +1208,6 @@ rules: - delete - get - list - - patch - - update - watch - apiGroups: - database.oracle.com @@ -1411,8 +1226,6 @@ rules: - delete - get - list - - patch - - update - watch - apiGroups: - database.oracle.com @@ -1642,7 +1455,7 @@ spec: - --enable-leader-election command: - /manager - image: phx.ocir.io/oracassandra/operator/prebuiltdb:test-0.1.2 + image: container-registry.oracle.com/database/operator:0.1.0 imagePullPolicy: Always name: manager ports: From 66a728250c1af98b50f71c4938da99b554306234 Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Sat, 12 Feb 2022 22:57:34 +0530 Subject: [PATCH 160/628] updated go version --- controllers/database/singleinstancedatabase_controller.go | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index ac2e8abb..68849a35 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -456,7 +456,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns Name: m.Name, Image: m.Spec.Image.PullFrom, Lifecycle: &corev1.Lifecycle{ - PreStop: &corev1.Handler{ + PreStop: &corev1.LifecycleHandler{ Exec: &corev1.ExecAction{ Command: []string{"/bin/sh", "-c", "/bin/echo -en 'shutdown abort;\n' | env ORACLE_SID=${ORACLE_SID^^} sqlplus -S / as sysdba"}, }, @@ -466,7 +466,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns Ports: []corev1.ContainerPort{{ContainerPort: 1521}, {ContainerPort: 5500}}, ReadinessProbe: &corev1.Probe{ - Handler: corev1.Handler{ + ProbeHandler: corev1.ProbeHandler{ Exec: &corev1.ExecAction{ Command: []string{"/bin/sh", "-c", "if [ -f $ORACLE_BASE/checkDBLockStatus.sh ]; then $ORACLE_BASE/checkDBLockStatus.sh ; else $ORACLE_BASE/checkDBStatus.sh; fi "}, }, From e918b3f63674e76d7c22672afccd77b850fc12ba Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Sat, 12 Feb 2022 18:44:03 -0500 Subject: [PATCH 161/628] update ADB_PREREQUISITES.md --- docs/adb/ADB_PREREQUISITES.md | 80 +++++++++++++++++----------- images/adb/adb-id-1.png | Bin 54775 -> 44691 bytes images/adb/instance-principal-2.png | Bin 62643 -> 52023 bytes images/adb/instance-principal-3.png | Bin 76392 -> 47182 bytes images/adb/instance-principal-4.png | Bin 0 -> 20197 bytes images/adb/instance-principal-5.png | Bin 0 -> 76392 bytes 6 files changed, 50 insertions(+), 30 deletions(-) create mode 100644 images/adb/instance-principal-4.png create mode 100644 images/adb/instance-principal-5.png diff --git a/docs/adb/ADB_PREREQUISITES.md b/docs/adb/ADB_PREREQUISITES.md index 057a8687..56c47445 100644 --- a/docs/adb/ADB_PREREQUISITES.md +++ b/docs/adb/ADB_PREREQUISITES.md @@ -2,7 +2,7 @@ ## Oracle Autonomous Database (ADB) Prerequisites -Oracle Database Operator for Kubernetes must have access to OCI services. +Oracle Database Operator for Kubernetes must have access to OCI services. To provide access, choose **one of the following approaches**: @@ -10,12 +10,11 @@ To provide access, choose **one of the following approaches**: * The Kubernetes cluster nodes are [granted with Instance Principal](#authorized-with-instance-principal) -### Authorized with API Key Authentication +## Authorized with API Key Authentication -By default, all pods in the Oracle Container Engine for Kubernetes (OKE) are able to access the instance principal certificates, so that the operator calls OCI REST endpoints without any extra step. If you're using OKE, then please proceed to the installation. -If the operator is deployed in a third-party Kubernetes cluster, then the credentials of the Oracle Cloud Infrastructure (OCI) user are needed. The operator reads these credentials from a ConfigMap and a Secret. +API keys are supplied by users to authenticate the operator accessing Oracle Cloud Infrastructure (OCI) services. The operator reads the credintials of the OCI user from a ConfigMap and a Secret. If you're using Oracle Container Engine for Kubernetes (OKE), you may alternatively use [Instance Principal](#authorized-with-instance-principal) to avoid the need to configure user credentails or a configuration file. If the operator is deployed in a third-party Kubernetes cluster, then the credentials or a configuration file are needed, since Instance principal authorization applies only to instances that are running in the OCI. -Oracle recommends using the helper script `set_ocicredentials.sh` in the root directory of the repository; This script will generate a ConfigMap and a Secret with the OCI credentials. By default, the script parses the **DEFAULT** profile in `~/.oci/config`. The default names of the ConfigMap and the Secret are, respectively: `oci-cred` and `oci-privatekey`. +Oracle recommends using the helper script `set_ocicredentials.sh` in the root directory of the repository; this script will generate a ConfigMap and a Secret with the OCI credentials. By default, the script parses the **DEFAULT** profile in `~/.oci/config`. The default names of the ConfigMap and the Secret are, respectively: `oci-cred` and `oci-privatekey`. ```sh ./set_ocicredentials.sh run @@ -45,56 +44,77 @@ kubectl create secret generic oci-privatekey \ After creating the ConfigMap and the Secret, use their names as the values of `ociConfigMap` and `ociSecret` attributes in the yaml files for provisioning, binding, and other operations. -### Authorized with Instance Principal +## Authorized with Instance Principal -Instance principal authorization enables the operator to make API calls from an instance (that is, a node) without requiring the `ociConfigMap`, and `ociSecret` attributes in the `.yaml` file. +Instance principal authorization enables the operator to make API calls from an instance (that is, a node) without requiring the `ociConfigMap`, and `ociSecret` attributes in the `.yaml` file. This approach applies only to instances that are running in the Oracle Cloud Infrastructure (OCI). In general, you will have to: -> Note: Instance principal authorization applies only to instances that are running in the Oracle Cloud Infrastructure (OCI). +* [Define dynamic group that includes the nodes in which the operator runs](#define-dynamic-group) +* [Define policies that grant to the dynamic group the required permissions for the operator to its OCI interactions](#define-policies) -To set up Instance Principle authorization: +### Define Dynamic Group -1. Get the `compartment OCID`: +1. Go to the **Dynamic Groups** page, and click **Create Dynamic Group**. - Log in to the cloud console, and click **Compartment**. - - ![compartment-1](/images/adb/compartment-1.png) - - Choose the compartment where the cluster creates instances, and **copy** the OCID in the details page. + ![instance-principal-1](/images/adb/instance-principal-1.png) - ![compartment-2](/images/adb/compartment-2.png) +2. In the **Matching Rules** section, write rules the to include the OKE nodes in the dynamic group. -2. Create a dynamic group and matching rules: + Example 1 : enables **all** the resources, including OKE nodes in the compartment, to be members of the dynamic group. - Go to the **Dynamic Groups** page, and click **Create Dynamic Group**. + ```sh + All {instance.compartment.id = ''} + ``` - ![instance-principal-1](/images/adb/instance-principal-1.png) + ![instance-principal-2](/images/adb/instance-principal-2.png) - In the **Matching Rules** section, write the following rule. Change `compartment-OCID` to the OCID of your compartment. This rule enables all the resources, including **nodes** in the compartment, to be members of the dynamic group. + Example 2 : enables the OKE nodes in the compartment, to be members of the dynamic group. ```sh - All {instance.compartment.id = 'compartment-OCID'} + Any {instance.compartment.id = '', instance.compartment.id = '', instance.compartment.id = ''} ``` - ![instance-principal-2](/images/adb/instance-principal-2.png) + ![instance-principal-3](/images/adb/instance-principal-3.png) + +3. To apply the rules, click **Create**. + +### Define Policies - To apply the rules, click **Create**. +1. Get the `compartment name` where the database resides: -3. Set up policies for dynamic groups: + > Note: You may skip this step if the database is in the root compartment. + + Go to **Autonomous Database** in the Cloud Console. + + ![adb-id-1](/images/adb/adb-id-1.png) + + Copy the name of the compartment in the details page. + + ![instance-principal-4](/images/adb/instance-principal-4.png) + +2. Set up policies for dynamic groups to grant access to its OCI interactions. Use the dynamic group name is from the [Define Dynamic Group](#define-dynamic-group) section, and the compartment name from the previous step: Go to **Policies**, and click **Create Policy**. - ![instance-principal-3](/images/adb/instance-principal-3.png) + ![instance-principal-5](/images/adb/instance-principal-5.png) - This example enables the dynamic group to manage all the resources in your tenancy: + Example 1: enable the dynamic group to manage **all** the resources in a compartment ```sh - Allow dynamic-group to manage all-resources in tenancy + Allow dynamic-group to manage all-resources in compartment ``` - You can also specify a particular resouce access for the dynamic group. This example enables the dynamic group to manage Oracle Autonomous Database in a given compartment: + Example 2: enable the dynamic group to manage **all** the resources in your tenancy (root compartment). ```sh - Allow dynamic-group to manage autonomous-database-family in compartment + Allow dynamic-group to manage all-resources in tenancy ``` -At this stage, the operator has been granted sufficient permissions to call OCI services. You can now proceed to the installation. + Example 3: enable a particular resouce access for the dynamic group to manage Oracle Autonomous Database in a given compartment + + ```sh + Allow dynamic-group to manage autonomous-database-family in compartment + ``` + +3. To apply the policy, click Create. + +At this stage, the instances where the operator deploys have been granted sufficient permissions to call OCI services. You can now proceed to the installation. diff --git a/images/adb/adb-id-1.png b/images/adb/adb-id-1.png index 4cbf8c5a730b4cad2496efccce19b24f1daaccc2..7c2ab1f181bd0e218e573279738fcd597eca2631 100644 GIT binary patch literal 44691 zcmcG$bx>T*w=RrZa6-_agS-3SGJ}TT?gV%Dkl;F4upq&L1$PbZ$>8n~+=APkyl;Nz zocg}H_n%vrs$J8)cdxe9`mARWsiq>2g+Yn|2M31*Qjk%HgM)8|J)&qRus4QG6VkAV zq>Z$+8c152O3l^D%EsOj4vsO=+{6SQ#L75mYHDIKIL5++;p(Lx5fP(q64c+-+lBtY z9`jY?P#4wI9V(7(063oW1H;Qgp@1K$$#O79e?_}Q%0wK#g^)BYPcZLqBi zcfu25@7Ez@SvpM27$%7ZS0503#4@}^^h@~H$QBdu)sxhRSX7?)9|Dld=>dZzgB|b| zF96m6gfSC;c#8u7b!0JZb>7u`xW@R__xdIA@$bW5z|TV9!Rjnvi8{AHNy|4@zH*W8 zX*rYC2}Lg;m+miPFk@nfi6u<2-#p%;2(ps&3F4Z4ln~eR;pXJrrs>5;ASU+ohw~w% zE-LNm>T=5pS(mVvG_vizc)^yt;shs{X~fEmL2(1?Q!Ps!kd?AB95XD81_vK*1BV0)!NVR>*aHWL2nmBj zh5h2g9+_N(|9pjS&PDvMu;`zJl5eF!AlUC)b5~1CN4IxQ?q9h{I$&AN*l6gu>nJM; zn>#tMn_4)TS+aXOIR6oW6ZIB`1syEiO{u&c>>b^Ny~Sw$Ng)gi|9QRsH=sQu)2)gUvXGWjOLxYyR$F{hnJTZyB80;ldClc zmynPU2PZcNH#ZwB1)H0Xqr0g$o1+`;znT0e9~nzGb5|Q@cN-^1sz3RfnmKv6i_y^h zDd@lc{;j8_x6OZ-p`UwOksMgP1NR)>HmLU{=39~B{khFU8S8IV1n+z|L)(v!vFK*|5xyzB6a_}NFFZ!|5@b!`0`g$ zl;cm!|3^>!+s*&Hh4nKKLzLscbOywLYY@eSgOh**$w+E=!yg)=nrQYlEbp}wpz)!% ze^~KLqKROPGJ|h5^Vxr!vuadz{WN8L(*7PJ>g8Vh><@HYDFxd0r}~|pE@g#}A2L6P zksi12DjB+sm(TlM9l9_};9^Au)8wKtB1B2Nmg#E7ZUAK#{E7W}NsN@d^;=R6=_S0Z z1YSuk8spIPeC2c))4yLChxT~>5e%kLv&K_{l+_xRVZHq~u^MF3S_1DM$=q?8T!6N9 zs}@<&zxl4|;h~*p?`r}w`rjbE3~|Nv_FBGiHGA1=xX-S|z=T_DmZ~(*(tuQ)Y|M1wkfc4doh8LpXr5~9tKWz2!D1@nlD?hhTU*zV?B5=vS1qlG z*)WnaGBMMpewW*;KHQ^S+mUVYv^c8R%X|4)n*J!;T6u-JXK)ZFjhf8^e% z%SI8s_Uk!xFF>~IC)&+itYJroV_aoi>Nm73O*D;+Ek{fz0GDHmiswS^j)k>?-Z|dr z4Y_np@iN@lTDeRv$&`t4(N>h$&+HV5cV7tMQK{9K8uDMtzmIkizo`A0^`2a4=MbQx zQab1Sqc*?qGd_Z+*%DLXtPD&!&cbLVI_|>1Z1iBNf%S6;X>2gWAoiG|BI{pmcI{E+ z0=)ju3DlT+k|voHCYceUy7p)jj1xF~ZY_@tfREg~sIs~2)G)oJ`MzCLFUA{0ZT=V* z2A-skrOWY?n|7{Mxo+gIw8e#Kos^u9NH$jwk1-fYyE8B#a{q_Xd&Gy{o*qQn zT)Q0)1bDh;;h(V!5iM~N>;+9%4*H6?XyYRz1mi)!BGs>BW-bF~ud;y%w%)FH#J1Q` zUFni=aqk1#+A+Jc(-G=l%I03t!s>E=ucq((idq6)B^1_14RTnnQRanQgnve-8j>vu z0@$^_yr-_^?|>$hm%vhGj!Wqm4MqEr!HHAaytywVxZ~NK!-G>*_aa6f>vJt)TyVaJ zxg-!%;+LR7ti++OY$r1UA8N=XjgYq6mI|Ssw`tw**q9jYuTPVh%z7S}`2(o>tqYpW zI7KXUl&!4sL6$3`&LJ$E`eyq@{jlfARF7i}3KEQigC9dEG~zTuo%m)ByxurIq~_ zv0m;Dk%ZFZr}NrRsTmj;43ATW2AR3yqs$%Q2mLPD3jQEuSB&(t$;F)HM{)et)>d** zUG)&d8-B+X`m-~6MS${sRGmEqQ%^W{G9s$E<;r_m*F&As$0M3mp*7!)!9A2q=|auu z9sHNXCX1{FfSE{2KzM9s+G&xp#Zb+5jaQIT^QCZN^iifj`6EpRU(!w@ z-p2ZITt;pl9a1RC@#ER9J!oQ(F$MSG!QWD16}Yi`K)d+GYq(O(Z|n?AP2CM;Ds8W` zS(bXr;!%Y_AXq!Vl@5G-e2a~$t*CK*aEVzVM{`pP@u*n-GE24=!5r>dtqv4P{mg|G z!Q-p3O8S<+gbbow%r_`%GBbm*v2m30G_>cisl@ZBz-z*BV{gx9kcma4b+l%QN|V=i zPGX_KkuHMz>S zVWy_sW;eKRJFTd4DGWtM#G_NeMk_-Qj7?NjRM4uC7=T;UAj>cdz{4{_kN2g$FO;Y9 zwX#C;k%%Y8!3mD~S7Y&Bdv6*X_`{a>vz4nl^xe&p zJU>v621h26i~}>7zV6v<=#_-Us469h`CSj^J_T6Lb-|w>P}bUeujvkPNJ8o6_0(z0 zHZ=^!bm#N|s~aIU(Z&K>u{GyEhBQ9uDO>5xBs0q%V;g1WtZJv}ly z%9|u7&={310E*0-h;D3bOfS%o&*q&NrtJdB3x6Pv!|@NZ^%J~&1s7qP*p%SpfrKrW zou6+Z(P^|(xLWO&cDT}t6Y5x`%%G#5NBq2W_MP7WI;g;$47iaR&k#cmHT+D?1jPO7 zP4$XHpW3In@cyOyfMjijeuD&X9r{gG^>%$$XK0AMG-KOj2;S^h?o-&Bpx|po7) z__Jd}zIsIOaND24y)_0GE#`YkY$&t08zh`OeL|GoHzq%~`ULkO|8?i+Cnl1Ft0e;! z(a(DaC{2R(JxgL+SQOEMwkC&OZcBY~>Dy;FE(@ie7n;ZHwn6nGV_nQ|lA zv9q{Z4k6E(9cQzbU&ksKi;n^EYikm;-Pc=UOzmj4 zHItgXRAQGyk0zFkA4#Y&i5l^Kh-j4p|6Wpv2w%ym939zwIm+YH^#slQ#}Il{B<(5D zFA7}z(&#*Bnz?7PNu*!PidjBU7n58oBgA2;fBw*S@UX?#-w4&Q+nMp; zPjT_d4yC==&6%EUGxA61Yg34z_)GyUQqJ5wQ)JF@NvAv*z8{@<5;7Ok(v2B;s)ANI zUOB8g-CxzNXc_fQ>KxTo>K|1F){@`X+-5_0P8QA03(e=-9H*x#{8&``O{-MRTV2mP zuWX_ktLt3nJp1eSyH2$9QcVhJ*&P-dxb*mDCOcUBYro~Bgv~12C269Fk6;v<4&2p{ z?>J3odz8Er^ngfPTWxGJwJs1=gt1YOf%s?=z8QRXHA@m*FmL#UU_%98+EPzEr{Jt% znm6bw&5kn8r}j21?1O@g_uIYsifH@beJF;8gV21t{fx)GDn1)(=I=U8X|M+J94eSH zF~Oqg)j&A)PM0LH?qNb*ox9EZ`kr98pf9DFLVj^gtu53lKm*ZRGMkUL_Wc>_{0hYy zC%VkQ`-if9KZ=O@H$9xS#{;NN#}BgjUc5#IYL3kz_;0xC^HRcYK6DFghQ(%%DOwtG z>?Lhk_O!j*-Co;1oS_B_hXz|eoE87j`PgllT^2(U5SPW{9?@(RoYsdUb0k%!APUF2 zh%x04o$Fg3n-MoS*lJ<9(5sE>XXD>v12F2Km~V8~_b>+#ZVH~V8HV1bU_O%`&iG3xASy97$!fIxV`3 zy7f9UqKa8FuA|hhlzF}5UQcA`m<5wWoxGAR{+3;%k7<8Yz9MNO(!RJlXYuS!ZIG5r2_rN&8~%!1K06LWbeH)LTm&a7A*EBe7%kqrZLOQcV{Z} zm0t_ZgF;_++D>O=6R9P=7zUWPFvR`aa=7#e0POPft=N4|v4Y_(A&QWu5acyX<0Ze1R z#}tfOn~25K&Y@h1BW_NvH(p{^WgmtgFSlk|hj;*9$(u%Z)!bm-+}(f=_$!ntt0_dh zj(IH;iAluC73s-|ZtTnT8^@J|-@kuf;BhchWIyw*t}7Dpo`LWR)!>BQ6CNJEC}^Jk zY8syPLT TBv9#)irIaC?Ge956kH|zIHhX*d(>=hmT<_n6DdbU7y zuwWGGLhbtll+gr4=AHOoH;-~2LnLo-DGj!)Dd+YIk%7*V-aa;f?!X2Vlm1<*gS}(w zY!MG!j!{RR)BFkjCSr>n=IVs2cev1%Vm?-E+5(3%t(w;sa^Urmo)*r-rFJqTvkt8T zae#JZ_Zjkf=<@4>k76wzR6$c`$9-DNjZ&SlAK1C@cz0E@x+55w-{24CM>3gcG=taT|s zHsOfzOle~aqtm{37N-qm?kOAs*c!OdOYdH(Jr76+w8K9Nn5@A~7HxC3VFY)-TAT=R z--8x%n*<+G%wh|9a#jHWW=}hmixZMrmsa;g97gu%v?#2hAAtTh&O)wR5lo$LzR1zP zKKL$N6MvK4)bPDUyqe(^Y`^56JzTU6u)gh$KLJkSnqR>mC_?8$hPYg|7p^{w`6-D$ zSuK;KC4E?1Gm(A=;DnH`5^jt)R2h=5vK{JDrdnPK+~)v#_|Hge(HsaC9mP}u!VQ@L zeL&OKHwJaz=PcWkwDikW#0Iv97IozW1-`D5r&u?m&JK6W33M* zP@cIC`3n|LfI#)It9H6cz_yNxtQdFKa?;9yVTC9;2|&d>)x3Hkc>9h2I1q?sZeFHb;wzL zmVpTh3c<;9I^tj;d6y1Ks5tLSvKdFg4$TkOjCkt%Yh896Se#Bf(oz6CH(U;9_Jqpi z)(o#$hxObEZ-Amed2DSQ`LB84OcEmJED8*|utCc~eB7TdQ+}wSuPhHwE1q?U426=7 zP{Ai`-DE?UN9JQe@6b{io1O+mNz!KL=vS;)@ra(d3|&SK&&(+ke+jPfbfln8CZ4jP zXFFJTk^|j!XlQ6gFC-8)c)kleLnP55Q05DZplb2k?CbQ;bEnro@}$R{%;ceFYYaK? zI|G6bn;#-P%+`9pNQR<>4bD4ln+^@(-e(YSsLVuU6AFGUdg%mO@egDsgW=6qW{iyIzdvT_d&e{W~cmLX4!NC3&Oyxhgq z+71ROj)#VuZRdPc97XKwP7Af zPI1o>1Z>J~TEhW?Y2^tD>O5b*{E@*sS3GgpG;^K|y^5#tT0ag_P?|z&6B8$p}OORQ4;9 z>C^GbaVv@XDGzc3A)PlZJcw*-xfrpt#DY4BN($IB^%P>h#Q25#aiM&OCyD-yptIIT zxB-9Nj?00NPkT3+*KO6$KgK4rb|eK%!q>vw-JUscP{Z*tBf>75U^LZCML?+((V9+im1mA?Na+%I8R9z z`&3aBJ;3&#a1hbc9#7VHOczF0F%NlmWVtO&D0)Ky<5%YbCmhBd4>ti2y!+_O^$&xk z#z+b?Pb8e^hfmeR8y9BEpLD_q=ful7TrA0$2~@JTnBPW)ZmKWL!*7&O>fr?C^~(f3 zcOzOkl+kF?&H+DFyNJad_<|Bp-d4rt<`DomTDv4e{TAZ-85C5<0bF)lJ#`%PuXxI7 z!`_+yIP>RZP4Wb80R&0oopFt-ec5Jw?>4CONm|*Q=eE6oXb*OAVu7uBGZCR5DMgkE zn8#JJ^FL^%katzIy-3pDLv4@i{Sv_2b=ud{Ne2D7psY`=`Q3i0)cFkmCmdMP|3>je z`NbPd=F5YR!&io_iqZ7*oTYhqXmlTR>Si$jgIP)no!vSH13`4iSMf)d^0HXD3GU_@&om=){}lQDyvQRc4tgbAB_0TbU9$T1KP!?IwK;` zq)lQb#QUvHL|LlZh318JNIQA3QDJ9z!qq6dfK*S|EEO0EhGMQFqu@&bSyPUx>%WCY zY8+Co>{6n-7k#mfIbj>I3Vap$M(;~TY>%E#zVU2hBgtMs|;*r0f zkc0*yn2u5E@sKK7_qggz2|G(i4ZR#X>5W1HrO;j2M4iQ6ljMZ#%?khmcSii|#W!m) z`^_^e&|hxyP$1H20qy+bBH# zpz_)0u9Apv&rE)pRh2$9b(0UKXuB4kByBY~_R+8z0l+!D>rQB#HW(9^q@)4KzvZRF zl-x^2G_sYXFFd`wZU(Xfm_=HXgBxfiSu;<<~QI%3Y zqpAznq@KOjs?-gyFOV5!jR(!DympbW61kX7%%PI%UO{g;#(!!>TBb zY&NRK>FSGr@d1B-)Y%NJH3oXiZe~4*;BW+CHXx|I&vO^43ZmSbNt6@gohY__k*e1B ztY+O-M1N9KP%@_9KmroP#jkxF8T8~n)e9(*XfiaT#SD5k+j>N&Dk&1m3GXr><((KzlyUa((Dbc#@OAj(8eDHk z=7+wJ)KXe(?kO5I#0_korg(b9I}c!w<~e6kZR z;DMxQ1WEN5_!mRrKCWzoH1PK=T$HH;!fMRsz!=5zbuoNhTi?=K6-=gWPH{6;P{eIg zlb~)cZ~xtS5~A3Fl`(w~4_5Vt9@$Wo%k7XbkFj*=FKe@*r>&EXkQ3I;3*Z?8*8*zU zN^c~J!r_Twt0#%S4n^cRB*&t|Dt$xRtY9_O zdTnf9d z*M}>z*uCW3N$~e7v_?xN6GH0OmDv{E=P4T>br}&+KT}~!B=2(PNHIRSi^Zueu;zXh@H#}H;0&S z-YS&f@)(1NQGqU`U-SlUaST3%$NGv0J@LhJ%z$KAD1IjoPMLUmn9N|h2o|}Z^ltkr zHoZ^0503H00SgRKI9Dm%tTKR6f)t{Tv(GFArx#W%JoU^*( zTaS4bM|+(r1Mjb1eIszWf3fe5 zif}OHWKDXn1lB-;aiBWExBO@4Uknj+AG6Bcg;H^&@9B~hK{ipw<}1qZTed{8@)3ivkMlmc&FntK8lGwVa}7Eejm73sdrpB_scJ!He7rc%v3GK32>| zGIZCeDr&0{2~sP8i&w1u(sj=_RKvgNkb#2+Ne4^Sq18Vg*3(G5Nz_euR)}!29pIY^ z+^=SD5ZK^AM|1ULe&12R0JPcyqAnm-CxLi_>G%V3e-rJGiB%<1{{YptU>N+#e_uj|&VK(cE3L&FVQnh0{EJ(58 z)4M(E2G83!a6RSq(JnRy46vIgKuaV)W%1Z^E`2ykkR8z`4*g8kFn9P>o4d|d;4L#**V^ZOv0gI??r9V%BeTVzva zx9sqDK)E?#hb8r;{YhLvE6aH{Y52Nyn8KtS!;~6bA?^5~BDOm@qhQxU=SG_*n_Qq zHwj@Aiad8Mt{SfQjRPPozPBKRk)7oTbxD;e69YTnbG0%(L9>&o+0``b;ITuRI~1nv z?i(tqs{^uG4O)8bY2QtkeoPc~;P^R|VwR0>D%XzfqqZz3s&dmJD%271x};?uRdXAt zu*AC}6xI-MHRe?!O{T-fANocWj^do4mgm0Ce_m%I%RnzA^IQN{BEo%$<+GR2sQ4l~0bxj(H1y=JTx6 zf?0!Qm5~d&yd^wl@SPOaa=V@Ur>P!JDq6)Ssi%$(&N>M(5(W0GC1-Um$5F=G+w6nl z-z(hKD+s@>8NP(^CE9g~`}g0HzrpG|=|e?-YCuuC_j?$ZEY0tZa>|jHolGT+_h9*K zeYR>@*m&%MTeo0iSPN8aHNwB;#2RT-eVcf9>cc_tsj}c8M!T&eM$v(Uj!gw{^FV}i zxVwQgAb7Q{QBc`;!FJ?jGZC&C4;ije?cs;4m9)@trM?t&)FlXpoYz8=Jtm^e13sq$ ztSw#Xfn`#V+I>)D2w88lin$#4Kn^{J_c^y3S$RpwWC7JvgC0e)Qf@x8d~smi%M)$( zId_!HuWO0=HD?)X0wB~!!MP$oL!IFKj@h-E zGXz2h-LD`ll;b`llO4$vhJ&|yRM2DZU>39aVgFw~z01?Dqc5GYgC)`|Z4!<}P3mn` zA*`84J19OZrvi4HIG}!#12@osiA#88V?XBQq!UMDTGWS*I@cxh*x^p}xuve2Y@F zBVTMSWF0Rn9 z*O(q5YjfBtz~-mntE?gZ>=b}}7vGdWQcQ1H+M|ZF>|`4KM?#Lq{98iR;!eVlw~}j0 z_}yQpPBfqizsB85mg=6BW>rT1LmEI*L4?_KC*f3+U>4~=1cWJj7z4r9^ZDJ;%Mcp0 zDOp0TM3j9T+1x0+AAHTsC>phIkxCKaNofjxwyI@tEPeGMDQn$ll)unG+9!osgFPzb z>%~0q;ak9qZ60LgoNKH#^r8zNfWmx#=ng(;8c|zqrX-UE&Q97sB-)pcO>OcqJ@o8n!%wWXiKfacEXw zxURI{!Hm!Z2&p0*W#h&BGb7*$)eH7H;~ytcUAhqx&_sgbgyiW5FvxW@0-A*avspWn zBUp8Q-i(@s5QgY|3^wnr(L+wu()vrAXrNJp@eP`(j553q4}S1#6`p{@`DI>c|4XnC zG)F4|c=hMby#I%sp@uZh3u~}7sSBIxzw&D_F!TNYCP4`tLW^=P0+kKSvW|eJF?d>l zaI4|>QUHe%-EpaAVdN*mOZafw;GkwfJ;0s~>Oq~j}l9n31_36>SW_j|+U`Q72&)J&4@lryVifun? z^ZadyGXPfaI2q#+t-mxAEttwn35~`K{%uEBB8+w-gdf1B|F>WC{!mS{y!qVzB|gB* z!{{Zu3cu^^{}Lf=WhFqB|8FZ$a`na2TSLLqG4AEocC?_Toj$LdY?lk#nhv?PfPjm~ z&x=*^S#-S`8fR{VX|OAx2`(t8c`N@#fzJinP!EYN>C2Foh~U=NR{!fqjv9+T3>UYG znb}hI@5K)4NTpaIwgF`A?d=xdZ(LlD2Th@016W*s>N?hY6j&YVAYu3QqesROv|6%g zN7*?5gcZIVcdUqhtyedx857zg`7x!Tq`8S;@4ex}&L?{N<1`tcc%rXwx_6Vh23GD{ z$y`0q_`raCe~y6qLVc9Rdy$ltpS<-bA+Xazp*9hN5IGUUi!|1X6hK;LHk$xPHoMQf z(8k8G7Cn5mc~(|7odAnIPxcDH%Zz^`$@_7 z`_pNk3>T4Etu1{Xcqs9WoI}H)b+YKW257jV)975Pu1cU;8DzH7yss}j6$@tyDN#D zZU`$qkzUFrJWeuGhtBFkqK`T!&z@MR!8~TLL!0fC6xEsE<&q;Fw|~ zN-z%1r$kyyjHdJhKmuaIPUJij(5&`AiujgwJ4_tIphHAhYR@Ro_#;2a` z?{B-Uh7+m@dc`?=r>NbmyyGVwd=jG;OByUKEHB(Xt+|G=P8sM<9?&gurdW+CzflcfLkY_;G;5DL+&d zEXdp$beYGRdEeUP=dA@=WZwI_mWiq?b0FPOvV7 z*JeeJ;(4|4d}(=Badf;w;lu({v){AUA*4rjfjFXol%{~JoHroZzE>)xEB)u>;!8FcP? zz7;!goLsNCljqHVZbwQju<~5@o6@J_0c}LJRJiqulG#DzGPxOsn15Rhd;@*^R!LLe zYoEOlcn84@E&MgEe6am)sXIxb**o)hqd9I#H6Sf9kvdbrDQrbVb)n9I7G~p%lT=XD zL>dPpy~~3~d_k08s2A4j5b;WYIRBFeBkt1P8*)Lk8A!J8Wm=rrT}lMI61aC{Ank?O z4zRE#^|i3#MZ?i{C{a!VjLh+#%7rpyf4DR7K}pZ!klFAK8*;ULKU~Rn5QD)@usQg9 z;yGMLWh2a^x2?@!Y0(7D@o`wT#^I6q?6h}9B00^U*Qlh@ezjhtKte!TZ`aVJ{cx^s zri6Ynj&2c+M026qrQ+fD&!FwQ`yx;%{TSv@H9M&-1p;5(Rvfty&18{s(yybR-VqKn zhC*|V1h_A$;zb`K>TD*9cJfO2)RFcX5gffyi8_BL>V~>qIukwjn{Mo**92Q`?;(*o zK0IB@*O*@jn+vLoek0!!$F?2vH1u14)j-qi2X3#DgM=MnWNNX7NsdN7A^{{^*GX|H zH{D_#P^Y5UseV1G8SZRZP^8to(@1(zHkP$yFKq-6*LUVdVX|Qw+9{6CWYI|cjE}#d zW_Ep&*q_K&=yIXeWDQF7u~JS*43+MQu}#+c@=t4`wo*Ct*=G&>VlhdCWhi{M@X#@^ zxpAz`%<@OmdF1=!DV@+xA&Tr@eH3hCUxo;W!41)h@F|<6R^a9Q@KuMa03WuD_3uN} zSN4#6MT8<^*IFCO!#gLE!t+>l#3+PbB0~2;WkKdgBpTn~D_;lmhP6lZA%NoT+>?2G zGQxVMI=fZk`je`HjFh{#%d zzQ(Fq_lq0b#uNgSb_U%hz=xITXNlyI+s_jipOCax^%!@kJX{+$EH)=^}HlJ*YKj8p@05OyrHo*v&8O%^0blzR=-7 z$+7f!r<9ON*Fb`gVUJJuC)_4o!5G$4W!4<<*J%?vu)dDm@W(a}QfdsxhP=?%fnB z8<$+8Z*FfjnR9Artfkv)9Up~l=4%S4z8QCxm-wpUl_a8hhocPru<=spp?D4O?KjYU ze7aWCq$k!iu|34wbJ>*H^SDphSS?RXbZ1^f^W(G7wW;t<-n!{497~+bK7ZU zmbhi<{O{N1Wp33lB4nY{hSJL%h_FJ0O`WQ2&%)3Xm z`6^TCkDK~<@f2dr)z#JKzb92Q6+0iV$jJAtNFj^O=EbbKEizKl(w2h>)En?=vdMQa zL!33BBk=JKRHjv7F_ytM2gsoHyFTJObFI3z>VA(LCK!%IM*k!zKCwJz*x;~8pULk4 zIX!hY)x(OCaF7fJ$iLTlc(|{zyE5CH?}fzR=X;I3w_Z)+&-4P#+HnZeLB6;|Z~(Zp zo3@$xf;p8$JK=`Li#!Ty-%vu_U0rJ|PCrE#%qAp4kZPaT?o6c`4aNOW!$z~8>W5~n zyf-i-M+WNpI>>$5xVAI-`2DQjS7SW6UFX&pIS5%yT#cRgkxdNyXfFjHA*hm?6XQFE z_QzU|L1Y;(JU~(LZ#@ddy7&(LVyT*e?e~L+^R+CPV|k(B7B^e#gmF%?TSg+Ps;WK* z6^$0BYu((QhdLJ!*TwJOCWH7oXDjp>qF=G$3r?zhz-C)0hupUB$Cb$mH!6YGW#)-58U({?B76Yg(b!9SP=lV zzep}D_C_cb8-$QjQm?KcaTWXIQy0}J9wytqw*mX+(6cC)LWm|PR#`<9L%rrUjFl;< z$kY%VC2#&IrcOiJ40YWqTI&-p<~ktBNB6cm(1l7~--uzM>%J~ysS~lXGpat;vIs!m zMLaqfLK7LFgYxMr$KH7D|ozV6!4>aU>lu;knq@bn2*Wft0P(rO+JZt1Prr@~r zP5zP&akAC?EvvLqo=?qLXLwq7l(c}f<0nFHZ|6-`BYfE8QkD;1MA#8OOSLdRTF^^l z{x)u-Q>K4DEN>O+?c;KEq7N_&swg-q?5nNR_nRzVz4AHL1I>~Y*Bult*KUwW(RI($ zEC4T8Nn<5qsf1{LpO0{4Ue*jy_$evK^&u9PJU^ZUonK6dj~q3w8tq8yiB??tovgH< z_mepd!ZbFnZ+-;!Q$uph3`= z?P8DBDD%LJE_qnZ*(BC9G@ObdMXx0CG|OqxrqDd-staEJp4K)Va%fz(n)0KSi{WGj zfbSF zaxve!W^tc$i`&h3@dCW79rwezpHqoiN!{HI0m2#}Xo?akdt-K9-)prncT;zt54C<%4CiNK&Px5P3Vb+~o^SCetG3-D zB}NbJCz~}6YUe^q=eCT}m8WG&{LRaVzy}ZC0d{J;ZWg`xKC*#~SUiayOByy*`MS9C z>#h5Kt_@*GMvTO0+$CYDQEcK?`GTuqOh2`*V8R)UF=EvoqZMmp#BV(obG=ubLz=RE zy53{?^l%F;*X?Vh)Y<-3{IA1?#Ik&De_|H0&~zFe9^T9ZO%9+!e@ zpW0jo@);h)@jT-`(c$3DEaGTyFcf;xAHJKcI4uV!1LS2 zjTj<+u5Ds$bCJs^Td-_yKTYtihya6r;KQE4t;3AHi&uR%osRiI8R1)`aT=u3l&##d zK^~3{dQk-pp<`Cxn(>+<*CQbUl|Hwc+G(9F)&oBW$1%BFUr?L_>QBql0#hs) zq!%oqNt%;|>}*3o7-B0gLHIO{UH$5>Khzo{48$V;*p^!Mw{MSNAeN>`uijr*tKcuN zzIJREUFOa-WWW+qLD;S9Od=?Z2TgPu7 z4{+?o%cDPb4&PBV``q3rD1`P8^1OxN4rLAIvF8^Tg>Q47p!p8+^Hr{i_WG?-1_ovB zO{`^KeZOJg{V*z%;^bq2O$ONbdDqj=Z3TS^QU)g-X0eX?IWT}>>;K@Ycn}lGo{gLo zP2MSzyEZxhfrdA}ucKpJ^8j!1oB@^Q_UK3;_7$Gj0RqbCR#$Or2T*07{ecN$Y4wIn zBDF_~Qlbl2BQ7i+&yj2V7U{R9HGDBCsO51ezn{cH{pOl$%qNA>o#wR=r1pmzojL({ z4*3NGU3dwj23uW~@6w;|uQzs%(ZByrW)B!o**Ox)_#M)H`DWK$+P5CVyTTq1HpZj6 z^8PGLxnR651lnnT_JkUHyLT%1L4tBp-$_U6pM4B6LSU{;{(wuJ8utTnr4{erDmFj! zqLn(10?)88Au&_t8~exXY$EO`HV&zA?RS~`$l?46FDf&bLFn=p4=?@cDN>g81;Fn% z>0OTtwMQB}$^ay@f}rR$|&D`YL;CN8Tu*HOCJ%$Vu@e zZx?(};+QbyJ5=r@p0Y@fnQWNQLK-HJ_J&9mFX&mq<@!7o=W~$GafUlKpxGP-(7m1S zFeMy(m$7AGIn;nvwmlvx9g8tJwmeiK68VA4lS#u6qLEEqfgJW>_*yOMMIsF8|+k53y=e>}Hb!ipN(tac|2yQzlaR~AqA2(L`!JP_G$ zv$luDr7AkMjVNgGYOT}g1;RV1+RTQXL>P+-icH=e^=fKsd+D}XbUd7P+l8UgF3Oq4 zvdR9z@`gT2V$Pnw6PhPSp*N+1!d$)vpO$|@UIyvRux4;9qhgCBrU#a#z;=AdNS{5% zM0t}L-kgpp{IG#&mw!StC#I}?1(G&P1zOe-T2o6iG355VVmJ3m1Vu1QLazhbVdtcv z>{#P8!2jUTC#;{K`z2Zgmv&dRS(n}*;7hpyr}56id3{WqcMX=D$ay|B*ii+0emv$i zu;WRt)NYO!8bkK#xWS|GXaf`K?&-+|A)^y2<-s{xq*Kmlj}wq%+R7VmM;#U-s7&tQlo}f0Fpw7k3N*x00xH?c(nwOa5<~AT zpFh7dx8Dfk>x`YFukyOLvU!2-_G1p6+k3PR%3K zI}g7uDWZe|@D+CFU?K5+Duf387A&u|9D#HOCGkb?MloolwTHz*%AOw+h(7|LzLMWD zu(6Un6nDvJQ>?}oNqbV{9h-C-D)m{8THF&oms=#5mmC0`jxTbx1{Vj0%1`i^Qtx}h zod%OdC{v>=Tjxtua41DtS07I-LBhT=fzLbA>8=CPOMbT-N7bgJVDQW8F?oesgsjtga%OXm4a5I<2NEsbS&c@7;TqZ)h5 zx$mvV4(m`~*m+LplSY3$36pWqbF_iaWW`2gI2#rRf;+y|Tc(P7|0S`Q%+*)=q(DA@(Ol!%wo$Z|ZZY*u7asK7rTEK076b zo?%gyP%}nqDYE)9O7)w?C?e$K)uD6 zu!2%9F+OA$bvi0Wq~7h1d~FCIL41}K1xk%?z_t>L3=eVbm={8GtSU@D#hX4C&?=)q zT}!@jVX(11=2DCe1f}ooxyF=&W60PO1RvEXpSyhLdeUsKCS8$2avFn{e0=)nT zTp}~tCEnp+*K%qSejFT1N{R70#^v>#&FEnRgPlh!<&42-RUo~XpL>$I_piXmO(!4w zg|{1sgagvoDk*{OehR;jm$}V;nScf;vf{<>S3VEu_4u0CeY5r6AC&jWzzB1hcB=& zMJG!)p?tyBgQwt-uG$&3@6H+Owz$`lEo({sLVdL(h}M4N!R^iqT44GK0H6oygx}J^^ktDP|(yF2yo#Qu%Mz~L0 zNM0wjszcp#J?G;r6JRxy;JSA!-*J~#0=QgPzf26S!3S*h+kHzOVegQj@cs<~ee~@? z-bep)H`Op6X(9>g(U5lalQI4G2!;EK(IrJv>^lp)Iu!gYgf6?~N z?{T%?8)%b;O&T}0ZQGeRjWMxp+i7eYjcwa%(Ac)kGkw3GbN++#+swYM*)#i^{j9z2 zb>kt0)`7zBytIkcFr7<0K$e|6T=PXA_wTQ*dt4PeR5#e*T6}T&}?s7YnF=Xil z8GtKi+g#c8B=N_S6aOk_;%Qn&8OG-~$p0Ddskg#nB5QNfs3wU%pZ;q25TQk3|)XY79Gqg8lEc{>ftOV4l)8`d`}Vs1Mke)SdvyIIrKApY2M5N|jm6k1QhDfV`?ZM{l zp57bs^{@{9|ABD`R}pnP+3Md5pt&G3#72ZHr*XbxiA^*_N?JCUqi@FT!d;|HMi3N8 zw_j_0wAcAw8)~$(hqE0`p#pV(8rawfqtR;8DHFt^lC5@lpwkjs6}wiaZ?&`S@6G%c zCE$ug(a%)a?KT!whq$-YX=w^l9!arVFH<+azw%%IEJ&dCBxED`56M}414l^OTdR2_ zqvw|kAX!!Fg<IKyxz?k&NWp}mJo{shXA)PhK1u5EFf4_^4je7LX(oZd>k zxUBot=?nboOEKdgQkCs2a#n$`paLJ_5}~W~ zR8h&mT>4Z|>6(zZro)@mG<8$-P`Zo$Cn8~m;Cy=7@@nxQB68RRbQl?#Bx;G-yp~!9 zx_pVgf*l}`*WMllArFoC)Kp51t9Gpgd#R}@InJA57ty%oM=f~1oxSa2qI;3kanNAv z1lMLT-BU`SJaNzLISu4q2 z-0NkWsji1o*V9P`bH)Hzw*W5x-p%$19^Dg^Yf3k&vHw`J}uZza~CB`Q?;HxnX&ttMOKv6Bc#7q{O4z zm1k$8^=)gYkIwwd6yr%si>B0xrf%8u;4=|ZvVZl%i)?AtNk!CFcl)S8oDhNscd1#r zk8`#x!oeoj7D!-2AAfe;dpz&eGYtIw%l^Edd;W{B&M7=QA&X|2Yk3DWyeklgXOY8A zrq$L*((3?nS{I{b3R6gj@0=$1p*Flw`DU@xr=(r6M~UfhdnfBSsqCpQS3XS%n&b8 zzanyu>2V*7x}-OeR28RCJfJSx%vLBq*zfeWu?;yzS6bGVt}af$l;waPbPC82j92r) zJzx!zyF+#j8|ZM(@=UC$Ep02z?+ z?q|msM(;0CSMht)G5q$`cb9Y0@mqMuldb}Z&!4o3c_L8b#wj{sEK~_at}#_NuQ|Ts zaV-V0byFOqw;K{UN6Dh8sS2%RKq+xVOLScdNBjo8w^F#|i4Q#HT_H>#B+(ob6T zAXvdznXWe!fWf&vgV06isqSo}l7u7mw_EiVyVAGADXqNl3X02)>v)+~n=+v5BwCu& zH6$qJfo@!_GD!m8_koj$?{%+#_O}Wg7kuLt%QWc1n`}8a`T5TEk5|h>Fps(A<*-_!jS;xbCnY08H;?%Hd*w#s z9xX}(tmJcmKv(@;3H}bN0b=S;l;Yc`Cqj$>1p1O(_eMi9~PHgyC$u- zx*RyaE1GL`U+$+JrIa)o4?u#+Yz z&Q-^E=IP7FQGIoh{=0(vcIoio@VJ(V*JrSkzx<}IZwA?VyK7vtt|ypfS)-OhsiNdm zm&koPs61z`!#w%*kPCTeeK{uk8o$7v3C`^Bgk=6rxIpU9MvpBIvn^zJj17NKL zqx<-C-m_SmYn5zF!Un1{=aM)TQqP7b29 zll5cIvfUnjxL+qNMj0{RE?!2?w}8SOxFVGRN3nd+WkdX(ZQ~5x?^M&gkMT~Y3o09s zX;HXL${tR~l!gp#itJWvq&dF#l@poQS$6}=$wPucKN`US5d%qZ_e6Mu&`^-0t^yy# zzcAv2a5%aqe&!Bf?B@1DyHNgID*LOL0i?{zNbM7T~RcoqcU3 z&Ng`=9tG?bV(dLmQz{!ZDE+NaNLQCYgg&y{RyWeVW{Qoq%J}bBWX3(g% z=&1PJvt?T!2eXwkbiK$0C`@k$zK;G?lSxEFx;}SZk8x@bN5WG{W<|?uG}5uj?hW2F z6b!Sr-&>4Y)PaSYFEDFn!JS&ArLH6y2K@f0nd;CxqfB$_br-)sCfJJcSlzLobXd&p z{5@$O8x&LYTL@1R@#%S7`!k(dO+Gv5VRe3s-i+aIX7jpJM=;>CaLRXJa}@;qppQFQO9q1$v(+H$bc1rU2-)%*K53CcXFe zE5Z17!K|V)3dL7I_w5-VjhpdfrXnpeU?r+dObt_+G032EY;#k-$!e9J&(kF-8gB*X zFX7^5z7ResJnFY7;z(*FQh+2K508X6JOQH_XLV_r zFZMbvf1?~8-(#fa#J}qkt*ipd$#4670by}~)i!4utN<6@k3XGex(&FIe6-DLZc$$M z=hMsxnBLd-k0I(_dO`gzFZysH423(Zwp}_WB8{A|`)fXrU+XYX(!0B>L34}uo$K8~ zkRul4e-vowjH>`PYW~iPbUx)6bw{O7`Rsa)EB%ErAFn3g3Qy5Kkqk$lMx5jG5fuqt zT1yXaD`bHi-J-KXsr_wo(R-G(=+hDr!uvf!5id4h}|!5yhHC{1MnpX0Pzwz9yZC&N+ysM-k2Rj|A0O%_ zsmHDj$!bOEPhAmxjZ~JFo~BX*5(~V);moJV>KomDi!?A9F)LBpbbHpA+ZYKmxv9>m zPWa?~YH`Q^c(2B(mE+KK^FGLNRJgon_U%Q%e7%dS=J%8jAbe(8^B$&)2Uj_^WZ?aE zKXT8AU_ly@QYE^&*jIAM;{I_gvENo+9a{r+zdy&&%>qS3u21(q44VbyB%^MJoaA|+ z;|bVM`6r|i$ps~QEH{$0g|%6NfGPRsiUYG-dy}=Epu>ndYJ3mp{2x+2W?t|R((B)J zp!SpvWtIuPjb0%}OhQ708~!ah`a4c2)~_?9g%L9tN3?rq=s5OZr=>b$M}W&~9e1E86 zl&BvrXR+?k659v|;gsDDz$Rlpx@oYV! z5?r=ONWJ9eT-z>Z-4}p^R`X@-Kbe5D(N}P@sY&_=D%;g$NF+Uu}v8DtuoM5whpY!`Z4Ssn#3}^oOA@vs*oSl#Hx_OGULp zRN@{rM1r^?#1tsa%6|`Fu|Q}WlkJ?9!+Nkq@rhaS^w9mh{^23Hnby?c)?pf4w}aL_ z+-CniNgdSCoEfAo_#H)-*Sl*Ce+^?J2C zSFU1RDLN_|G$EnO~noq0<7RMHeBX%AuZlv3Nh&$;_uH8UCu(QE~sZj-5OHS;w z@rLVK!@=v+XJ)?fQ7wvyOA{6HHh5fCJ;}j9Ob=pb`Pt%{FCun1ACnOn&3LB*=)!m|aykcLmt7azzIOrw~k} znD!q6Wr;Q|Y&-ssk`ts8@7X3zLEzwAlx+p#w>`R$49-B~N)|bn1F*hI=a<`dX`$0= zn*f5VJX?xG+Dyjh#~m;27y|94=+#|Bz;0~aP?+y>f6kn{`wo9Vs zo4VQG|5(`)oPWUDK}e=juk^0ER1GKK)$&5veXkT_^5V6a;8_6cPjfLy#l1dznRz|b zcJiC2niU_m`9ME zffgKrthM6}Xmw*{%I>X5?D;atp@$F8Vtz*DZntvFh#bvoQc1-4{o|;cAy6hix`54M ziBQB%z8CrUs`uUaDfUAgoK+F~2Aj_-ciFaY6F!T(A#MgkE;%9>rIFX_mU|qJxh4ka zw(LBcXRXEcx#qY>z!|&$fy~X*`4qr(mDjuo@E@u~oCSWZhH^@pD^f}Xx@`bV4EtIS z;n}khH(vKxo(gXp=uiJgx%&ko*_8|@fMaIZ`NFCMqj|C0u3O9pRVmQS(oJvPVvM!P zY8ARZ+JtO1iC@NVAIb)po~cERr`?KwPoQ+e&n=7J-eu_es-bPG9O{I6x57g(78oPI zP+EFCH0OuG9?u;6bm4v^8M~2W5@=3|;FQT~501b%!TgBa|DXYgb=>;$u%dP0Z#dt7 z;GsTVRrd`?RMIY^HqUwfiDoZ>!IyFNC@DpjQBGSj`yV6mFXa$7!cT;ZEB<>--xinj z-j6a2C3|tS#g-EAv(5HmE=y{-6NwN3dpo_msjNEvTXgrQPaBTCC1Pf9Qqxq&4uF_lbr>9Tu1wBB}w#-KbRn79{NnX1jesGXvj8nhk!Aa{}m8FL(tth+8bbhn_S84IVGIN=+m%f z_m|XhA)z{^*(p>>Iiy(E15UJM&`ZGg&Q3Da_56SnF}xUdPX8qQuciIXm7U{ee$uX2 z5L_~eDr(+U4ou7D^P}~mVWoA>-KjAT!AXwTxZVJYnE}x20~0pLG7tHR#Ff`irIM{x z;<8&ywL;T$;r3`I!}IM;DD%#-na!iw-q@&AJf?V&AzbhW5hFF7(h-<&AV_LK*kk&k z94I0gT{!Ud6W{m3Se<%keEzg5(NRy(Ab4@}1K^YX0bN+<<$m+bQ8T%v>oERZY2sX@ z*52Xq>s)TS%vYl2LMDomA6RbcBt>laMDhmT`jCG#=@kg!yV`G1;xOI3EN*^Mv)Jv} z6_+4{a2^LX8D z3gpz899+Tv`k4E?GxCd^QCYGNOM(|Q-17{el?{-nUcZYKRw0fl(i;IK5JDVB6_5d3 zXie951SahX9(nJAT8B`>T3zXBXy%Zra;kG!5p9WU6?lH*N;6qZT45G4BEZH-Xxe>` zJ5;Qtb(Kf_HGqv9rD)a?@6T-xB1|#->%7#mZT@p@O~NQI2-Snu5S$ry?w)})9i;DF z=A>TQhnD?KjrfqKBq7JEC(wA@uQr^o@z2!&zj`;_&8~-*hX*++%j&|HA>(Y)%EUw? zjruM#tOairyw5WcsE!cZm5xS0*4gE$t5|LsQjN|<*;sYk?QswtdyIfvH7*UX+r%AM z<^!7L7_Tmzi$%7|HPUC-q$2caz6)i#AmJkuf$(=v<}rFiLZ3UfKa^Omcx%OyC68{EV- z(;dC&Uupg+oBD^coAtbqVBh(sEAFp8H@b#B#ll~gEfEwV zI0Ksm zm_@X|bKJH$uWxUYZl`e|F}!PUE>0Q7xOU0Ip%xS zv~60>@pns0JQt^G%)0tAL%g5al#C~WQ^tQ+&h#nO5M)2lD(C&Wd6Tvs0mVNKUDC-< zbRD;q3zkqZ?}upZ2S`_kIN#~G@65c)rYIr7@T?|!7~`z z7eCvmf!HDmR+dIqI8l8IwrjNiQur)Hs&edkknma^Gp47q+GSqKg>>#B=exAEcCW)M zBwRV;YAVmUB&7$6UmR)Rf$A~;V%5w$FOq)CV)tI2@hgb~fthFirukJ_1xC{!n9mQP zy8A2T^eI#+@(}t%qq_BntLPUNs7IQM2}lZ(7o>o@08pc;`~&>?iY6mKNid#BWrlmj z?Q5SRr%H;B4t_`?F$uAZy^YMVKL0*cHdE8PYf;c!IN{$(4lBL5|S08hYT$r^)L`r3>93M0eY@Lo- z7_Zl+wt9JeD_Xo7!h#C?60*;-<2TELeO;1fF<=C`~YOq}uuWEIO z|0V1mt%w0a0hh~+-=42iux&$h-b?wuLN?q zfxG9$%~4-n?KzxxBz@)C{YhbwO9Ou7jI3&!URAOXtj0HbkNR8}k+6kgQ z1=*?L85O59y+$%vFm2uQ!Ov@rtis1T zEn*IUPA`ma*YF1v`fLAkbpf3-Mq5G{Rjt`Tivpt4pQU#9!^%b{Gn&)ti3^Z6G30Q4 z-u5}%KlT@iU_FSVGidwT2&RiF2(T?8e}d$G7@*u_+AN+%4hgIGdNV4(9i2$LYUebk zpQc_K%$oD-r13_x%>qmeg{twL5~w6t!6fco7dGkllhm|W4}aD1ujsT<#W7nxf)_z8 zBX$(>a2nWxoEB3SYwK}{$B%W35fwc=SC%jlVmsd6j^^YFVXSQAyNEFay1F7N;a4)0 z_OrSR#TSZ^N9m;^M#+@~81d6?-T((=_chWqgb%M0`RfVaTUnxQD4W|VhP;Q))OGW? zOWW#%8O=#SeN+i8xKqB|4mFj6)NVQs8B^O>J@~IM|3^w8cTZ2P<{;65Q&rrJTs2gk z=793kUr&*tM+aw;r(JM2PE?V$H{fZS{zem)GDbPY&bk7Y6d-wEWBq2sLbzb1#Fp~B zUy`g+ONW|%1$BmRFv~(1f{0)x4IWe{`u|I@t^cFkIQr7N{||xr_5TwrV%}-i{11^i z{}U{(!&rX&PoM}of#j6_AIskVz5n|!j@+4~;+f|E^r@Wi{^?>4cLz!u8M>iD#gP9y zJnsa7+a({35A(K#>%0Sbxcy~7$K>{C;hbkbgTOcLeAN~6Env$mX&W=M;ox8|_3|a= ze?N#=0UsQ}`BW^xF|;|bTEIuEZM!7`zGXHqBD?2{uhOM`LRB1_CCA7KB+GUuueMF+ zArY=Y%d#6FSHj*A$Y(HnA5FtcVMk($06^l0i^R$8c;f{&XK4+R@E@&&_F{du_Q18V z?9?MH@acderxqgo=|fWy1L4Mdf9-&Lv?t=Z^ey*gHj%8K^*H{T+Y1qUAO!`X8?mkN z+A;6L9ZjA7xN7TpNV)OENS?#~AchnK0&@j{rhGhK^!AD!$YslLeSPgP^{4m1it?p( zwbQkmR-;LAc6QFF@7eJC^7WU&XuGL4yKCm%>ki!<`~xQge)NvvK&*uL_(^-*LwGz& zu#b-eBYphuJBNo9-5=~q%tCFQ9!@OTv!!CjT(=(@M$xcb?F0XiZ7arvH4sf^)y`z8u|s+$7W4$R?~o+eUthU|?dBV(bK6PItd8#FiytH1a{Fw&*x*KhG99O2GC&xZEyS z-}KAR*`Ej+B+0A0hxZLQN^B(t8arHV3B@)?qpIq1p3LR(^(-UA%siQ1JIsCbetj8v zcU>or+4!_gd-%gAK6}FA)zOOK$thRW^-O>Je)9_hGr;q%G-ZFG9iLTa{Icylf}M8s&a0}t%-f~4Y1%f1ns{+Y{~-t@d&v7=Z72yn^*fr$-=RPk$zt?i97cK$oHmS!CAu=qKtJR8 zR;2ujv;ErqaQyI8N=oq~m>Od7y{Rz`!hwG@i7%mcRMH_KQKia*FE9ikC7MDjNgT8n zel_tNfyt)NOe`MeM@Flp+omo=Du)z94K8UR&*adjd}sVq>~90N_XDof51OFuuO5Uy z`&xc?AEFT?kV*pXB?+xI9lzB3YK5_uD&~uB2UJk}fG$m-kYF^S#6WJgp7~=$=9+l* zC7wb$QdX-Ohi9w!wd^d3790!?Oj4E(+c$7#w>Z}`$G=6s-J=vmjL|^BR+&-@Wmt~W zVrUd?%gW?>tad1Nb4xsz;W49)%_+pG5kfn4Qq@{eDFK#&!}@xXX-z zq8a|b!vvCPjnlF@65?C4~ zNXOrZP^;3POU7GjeEq`5FQ59NY*PGgh+G5n6fW3U_QNhYVrF#CchEZ>ts(c_`hz&K zm}-Z}vYG4LPA9V)6bjp$z7J=Av4nrN+JyeMT$Pn=$LFJz?qD)gbhzGgzNf`t;tH%p zp(-(MDV@XI4;8v;=k2-^lfNcq`O=9Vm8odw&&L`=&(&Ttc&GOrE*rniSU|Zr;w*GOz^3m!p#@QjK z7Q1aEdu960QrBcG$-vXkz+)V{O*NUbX!#hwocK&to*E!XLN$D1{xcAxT>vP4Vl zhSr=JE9a&!kM!hw6dwQmy(S_^y>^I0y~BtLd!e*f%#Kks1|M=?3WH@2FVJGt)GvUcf7cf`|Gp?tc8zhsHJx5VFd@ZIvPEQ~luBS|;5X+kI|-H5yO!m$N#(Rw}k(7#>W(BNDrVP~n^bk2kC<{1q!;OHT^G zB=85n{qn(JZZkTYWVTsK1BG}L6_wGXKbwg4JN$VW4&!Qg?Rbi4NzK>q-z`hpddguN z&-r3Fl=&Qn$5&>*DIi~Zx_8XNX}1;YK9Vt`K`NJ-8q)G$zp5&wcgkTS?t~q!tsS`a zR657$b`<7cFU5zIqb)<&TJ~zQ+KS3{T$8U-WN<2%!(*E%X%i=kN+$D#D|h6)MQf7? z28SIDd$CfoaqufiscgeVF_gxP_ zGA$K*LWfMr7YV)^TMiIMh(PnH^M1m2h3gQOcDq6(jQaDqqn*{-?dI>gGIq*x-?$Jr z!i87x8g3SpLCFjoE{_Y!{&=G3XD><1QfIJ5BMv64?UP_U$><3I$ERS6#EsLgt^a(c z{*#tMv59ekIX06^{_{Ddc1I17GV(!h!SGW%GOm!$z&3Z)aoDnRS9A5pS1|SKRWVhh$hoaBZIZqEmCu2ru5|yy%(_znO_ZVLn% z<>dX*N}R>8vsurQCR?$r++K-7o_YO2KGL~WeR0fiPhNp6t@XXsejL556fLj=KBXmB zwOq5bf@tzYWejIH4rZG~VHFkb^vUyP>qDf_q$F_1IqMfSX&QC3>dC~7v2?D@IX3Xi zWwEK7^jaSJ2>WNCUrs`1?w_wz9Eiv<7)5m64{~I3b+-r^{d5DFci~3+ghZF7b40-d zcW=S9Q|)Ljp+je0gJT6gStkG_4g_l(j_bp^B=ts{1gy|1ic4wJdWb{OzAjCO4?KUz z`E)cYZAinA`+hwsTMGwv%bDMN$#lBEES%`$R&JmYn3XG3W8}!*87*(jPzhE`EeUYB zTyPC3qYZ)}!a05ePgyuq=qh!x!c85PX2$w%WbBG}7#0{v2 zlK}hv*9^mkE$z`l1a66Lwy&WvV)Y@;CCL-NPT#(}O`J<7_`$g(<_9R*G zyF#TPo$GKWFm$Mkb9a#ofFwY_!sI2Sn3oB(y48hwGS_7I{1P0dVjo0k#Jr;( zmzOiFp70>rt-zcK01*4G@84#fOv+)mNZjeOYSAVCHazn9zEe%}*pGmUmGb93tT zb^QeerAk)aC(rA<#1D(MW?z0raNCT9(+G-I#K`vZDdH<3q$eO?3N~a!?sUZWb(gI(Lh~;lc`03hpyYg_~Pjb~n zddM)JMDX!o9rLfv^c5C_xW~4A=ImIM9CDA9hbZ=r=GP??J>;tHF0H&UuFdJB7aFXjv`Sga8$9u!tTC!JhaviZ9DLeZ+d8zC<134GK)!!6&f2 zvrMe#ze8T&5VWJ-;IZdq|9!$wtj89qhV=0N&y!pg5pT&v+%sK%Dgk#5qc%YQ1!7M>cf_?HdxauKa zrNEQ`W$(;>s$;TY4x5facHkZ=C0Dx-NUbTdl~Mae|1D>J7cer%SAt2)Y3)s1vrimw zpSaNw8{olS*F?61G#I>o0#g>r(p%8-gXC#9u^ zyR8mxv7utylBUy!f9++r83b!{j(h3AR5?U(n=QyDF0_gsg-m9FcDB-qf!BUFFqy%D z9DcvY?Q9mAsZp=@*!6Blf=s8=1nzh+Wg2|>X4XesS|y}iYq`kp76ju@;Cm5Jk;YuN zbWGP^wyrhOdXxAOqo@s8xfDoBD??rhYMk`deiOa z=vKr>8UJaLhmi~(o;CA`@hp~dsRhG)R_|_}CV7{91CJ?HBHsBjpf3Pb?bdm(y(<9b2)~{<(#WMxz^uxGR!K zuY;L1p49Uupqi_H%dYIsSq9DHpR6fNBbv1sLeppQLz=|$L$SqfJAW6cl=Nw;HQZ?- z_~yL+(dT}{J#!;MkjPA~Q26bRu!=G4)pOt{RYzrt$i2kHy*l=oNc^1}D- z4?4PGauiltRMaoUvUvDDV1%;bv)XL#DNAmI5vkEyU| z*5ebb*K#Zxv+;-8~?z*WyKeJW{!fP;s zFo5d3!HN$q30llV#my-U$Lnui)qz*%cz>#TK>xo?4d8=%0Zuf zmv*bm_s5I%(1>!lDF=Sb_D6D__1=C7bGg_`I;3-FMA`1yzD!={WD`UTUeA}vvN4g>&LBF&$Q$ePlLx6O%he~b3}?T#1?4tGa-d$lOL>4ZPd;WL`3&{ePlF$T>k(X zEbr-xDuNF5vw1DPMCb5+mzN4C;0-df32dQIqGU(A%uzmnkLeOfDPBq9#I4j~%qxA~}Du@egt=fk75E2GNH* z(QX^OqnRXFCDV5(*|9SX=G0$U{K9UuSsakVZaSjzdL;_@q*e^@_)K75L!yPvs{mST zrJ*p}ed)h^_l{tL$o+jbYoS#?I#=$tJ8WMqkYSc!Ac(hHh)p7oNQ}*dXEZ0m4MN7K6+qGOJmc+EOW$76&)usOK5e*kHb54zGy# z;7&$kBygVJgx8&1O@(=T#LQ`cqs-{M4c6z|Z;0Mmi6k}026R@t-u;PeEE?5HksKisg<2~~QBdWNYJxgU zj2QmFcyEml!okOyEfRj76{|^q@O#k(8C*H6?np_Je6%aUFVk$n_dU@zSHbO@s!ETQnjfmu)__jwlwOP#f;PCVlCJOuRY&&5n)7x zXWiREWFB;;z~G3KT!oJgO!i!8HpZAqgFg`uuM&3yiKD`^+Oc{YGT5Zu3YqQILdClzUIzMg6LFQ&-1TpED)A`i(@l9M=xfjkKiN@g zQZg7w#8~hjPG$d!R}+Xg&-Biv~6cwQRExUDKBj||3p zHSE6Ww86sY$Nbm&`Ye7&gs&k{OsEzILHe01{;{M#!3Rtx{!FdEGI{K(IjW^+8@>BoTkdadHJ($pdFvJwv z3A&!Cd=vz2BZqH)M8st~B)Zn>ISmMGe>fAW_6ajgO2lcNblKcqZAh1;{B72)K^{ z#Isf!y&&D=;?k|um;mC$n5Ar03dNV(+e^jov%3OrPhVDD&V<@nsV2T%qfyBJArvwX ziw@&vqnT56#qoT=WkgAznju9>8)bKSf04CjX6NY~b97jLnQWO>Ihr%AmV;S_`R1F1 zGp_nJDOpjVYi-H^?EAg~Nmj&_VcURI^VuATMyFK8kQ~^|M`{XcA?McRKhetkM(x~@kNl!_Z#U?gvYZz+hGHsf` z91$x-k1@q0&UN|x;Jc;v%rw}aN|5=%Q@j~taThKp$#68B>8Zh7C{ldkZGV5lRzS{1 zu`G~72fo3&3_dSZlbrp$6r6bj5V*-v8U0Qs>&z>5^l<0!poN9pKHEC8RTv zZsx=shj-o_?Ayib`{_@jUI!E{-1^g_tf4b>!S`yHfRY0)eY-aj$*62DN(^f9_uR-(RJxi(-VgQ9qvrq7fib$ZnAZ0q5iN8C6)8pW`( z?1xOI0__S-hudb81_ZwZ1S`kn+YGM9^>p(q8dcNj6`bMfP}<-&_{zbY@oV^=GX^vu z`G>`PKWMp8c4`aHsh-2?;n|}2Mv*56bCK;Ch19ld0y?`%ZU-m3} zg9FLph5${K%;+GE3jFOYf%3-nN1b@7~?3Ju_^(K&Vn-%K*#t~HjIn7di!V_)+DKGD;@(~hC=C&d>RJ7hYQb`L2;!I2PMa`Wzl)S7&8Mbe zrUUEzzGlOZ6C2yP-Ml zShmBM@UjXeefS)L9CG(%yT8e7qd_2C`*6cfO47qs}P=+0nQg(NNOeQf8Adn z2zJ;AcHsY#JW6&c9of6@9oZ*j170bvIzZu$N@C9g6{0^7TP}O_hc1AF&G(k+lC(*s zfEo6Po|H(xFp0uFB~+&4O<(gWpwmf?wJtumf635oZ~on<;DBu0{wIfzA6;9=sQz%* zot-pU2|US|=GDZwzG!lN=$RoiTKi911}eFP-ZUNQS5Xv;bch>eWtlaWGWepdMAj0= zj>|<)W3?G);n=(RN0O7vMsBp_BQLhYRMOtja%rAGXgC{EZq$RGX<|MJO$JF%A-75i zakdlOoEMa_nA%z}(g%0uHQt9hUUL=St7eBvUYjS0k=Ue+a0NW}ioBPI z4L|_0BfMw1pj~%;Yn%dvXLN`5Pfg->LRDEXvng7=zIjLrS66AOP&0NAwZ>gEU9O0X z+S1Eq@ms=9E!w`1uj)%)jK5Bu<8|YJ=g^II;=q4gnZ=hD%1WR+8wXx7Q_K5uD)NvD zi1x&15(L;X;1MFZX~ow|oyfmCE24P62sgfzGFVre{s&~cSRvk2V0u%ax;k|uK&TZ2 zW}jMM)Z<@Uo>n&K{()c6K|~O)JPh4hmDc(n9v&70;o;%I#Rr@JPC^Ur6Xxg4do6Nx zUPK3;!vp?p+I~Lambo_n+viX4{hkTJyWa;g*%g@ciy0`ilwvt`Nbe~o@ue*E3&`99 zoN5bcx5wNYuOFCs(i^YMmjd|UpXXH1Zo9R7|GqE>8C*XNqe-&hMZM9A=)p!i0@Cq! zZKRx>h*#J8?|2bb>$mSG^JO1Wj2&U*jFPqroA`hnemu#OoHBy0`}-G{;LE+~6dbq% zxqgt#uhwPXfVD#HHOl*#8w8ITe`V`lPWwE3g0A`Yb2APvPmEBIcDR7m5d_;I+kE$Jn@Q;V?j@3&rStwwr^MU(*0~=o_{R_b zr%}|W#XRpidolnL0p!j8z_hl9;lNR?Ox|N+^=^&FAaNqCDkh8RwAfNjHXx3-lq&wL z1lHA$%wZ~_FdKVkPEXA0c&^ykejbTDD_c29YMJ6DqW7(-RFRYe!sTd$3D>9*9HgW} z+72aWVEX8WGjLxARctP~bW$Y=gLO2_8CbeJn_;dER z+^?gb`CI4_jre*StEas5LLHni807E{hZ7-#(#f>|2hu{xtngoJ+V*!#u=uV=3ELJ7 zcQ!B`Y@?-GWFP~i-1{of5F$RW zaHlw3S|B^7BpG#=?K*Do?P0GFIjejoA%1fGnBc42dX$r?2T^usyR&drrbceOR;>o$ z?|ceruh`*pKlFo87!r|868L_p^`AQ)5kaA=NPRd3 zu9;{{a!92MrG+**Hg5p3S%0+x6GHA43Z&f0LpDQ$u6LPdUSN|yumU5oX46S;KojMG zT<8{B+D_O1L2ezI>ssl@t|HE(H6~Y+Q(yPsx6-HNRDTW4A;h8(Ukd^A3$0 zlgs9{MrnIIU;crmVR6btHZfN`t;ZXIS*{5vC*a=sGh=9`+>HM1rlYT{Hdog_<X%dOpP@ZGahar?$ZMH3d2gFpOUN1!Z(!iW zT&qU)9#e{nYLTpH>_E=7c2A65_0ON|t>EsCwSx8@h8*n^hM;Gt)J>}mbwDPcCq9!T zu#BL1j9`>ff1TpF;>nUt+Gd><;$Js6=eyES3P|5(jA~-mmdB4j)>f)j-Ln81a?vBN z?T@F}pRar`Q0;I#4v*XD3q~}J&g!;fTCiP-tWg$mv0B7-^wBl4;C*;2>iiisHoDI| zuFq#u6t7j``1mw~3hZ+D0m2^ae-+51@ZWCQm?-I14g`=v|Ltu~vz~-8s60MNKqOZK zqr#1)i3KFBH!k5MmCKRB*hn*2OAIaOHcZLv+;yRfPZS}@ilaemR0HCfJVq%@kZ?h_ z!wg1Babr+A1&K!$zKmi$vUl%n*>OoIL;X0ME$UXHDBSoubv9?ruoReb=5Z3Zz*Eq) zCMP)+MXFWh1Z%Zf8!@}5DpF{z$;R@~r%aVl;LsY6Db0+}X#{a|F#3z=-6NGhsCuZX zPP=$2b69erazZH=ZTeAobVqCpzpr1w2hv66-$TS3{vU-E6VRhuzMwY~zu!>$9+|n! zoPaqMs~1j!qLMN!mpzbd&107c!RfwcH<8`kbH?Lyg9h3H-h`Jj5DG55X#BG;$#Rj= zNKk&lh-n995Dq)vL7%dxAlJ3i0zeb;*D&spa%_c@$3bMNc=Tzjubp_5GX;Bu5GS0T+l zOw7!Og@qNyl%ENAg0SJ#eVA|SGi7@Yv3Sj#fkI+Y%CyeOq)UEwD=qdXSvh`oCUQBGv~GKWplvRq}|V$9~<}0Pk3GT#S=y;WBe`qnqFcU zwf>2jE_0%5qRJW_Tcf|&7yQy9EfGU4A=X%1X3@YK+EE!_tk>{#uKKX^6}I_`*m1;Z z-{4G(aeKCfeb&;KUdy5Sa}W!KzquHxpf7gAAe^6c?&@At@n~4}f5^ps&Bm7Lxu_9T~*Vpy&d~&S|@eY|$lYGnOVC8LO%)!H} zh}qD@G1NnwxnFtV!N(gAI=iw~XHyNikzniLV+6F0ZVuo64ZYVYr!NHB#hun@htjsG zqBu-sPK#$0O(AT*dGHoFBQEEP-A0u!*;J0g6h9_DQIS-*QxnxbcLR60uBRxgzQDf!HE-|c#5Fg_M`6L8+$;Au=n6}? z#ESA-X!6xEO_9eGIZ9%!J z%cpqefFf`O7+Rv$;oIQ{DYD>I-73RhoD()DsUDA%6cZgJ@U{%&ptJVS$=$J=9)0`W zjUg`AVy8++E2RdWrgq09_3Z3-rV#f*Xgb4-Joe(cve~j0Q*IGZOYt z&fsDJnHarBjch3_wjzCkzxMQc!DNT2XNb>di{l6`Yd39V0VJfyYLvU0d*bmq(o+`E z%VNQ$>K56hi;s^1t-+I3_rbTY18=1zmfm=9p|av$?yz@HB>zN?pyUCm7Heo3M^yfB zwC5Peb+RCmNwDtn1hL6kf{gh6t9nXPji$rczObQ4uMSLBsKPwGWE$Q_OY9WvCV|%%ssj(I&NAnSXX9SEIX(3)7Yz$=+ za4J%(Q1f$KXHuCIM7VvhUC6p|<;=1?cU@9RZXVA&ve#SkNGqY^)6nq@f ze0LVFz*$SG#ZH9-jACUwC+ESx4%HNDQ-P0VEwRReS?q@2^(Ik1)As&E&O~q*x_sZT zI(jilI9&)1#k)h@jUVn=-nX)M?;##chqMvKO=_l0m$5k-ysyx`HB)>$l#tx%-fzW| zY~(OL+S_QbA?7r{7{N3eIu#@HtA^8Y9(Rpd<0+zoptGnBZOAZUN1Y3#34-bsY4gpo zkwsc;H(JQC;XX4CJ`?U=3-Jg?3yK+)Pf-dwOAzdE(2aoZr=NzuOp5gu2|j6^&hT;Y zzVzJFYZ;s`JOFk+qgOdTpZ4bN;_D*yyj(2pP&n9{TJVRG?oMvP%oWA87nf8wUc`mE zUc`-=Emf!F~@o`;Gwtu!vC?4x#=2ijxyCl};Oo5g5`Fw$erauRn|-LYD!nY$xP;5jkm z0-*+WRxS+PdQDc0ZG1~L#(?x@WA8VwF9E6K@9*vi-qV( zWHi18`kBhlE7%u+X+mgalYj|S6Y7wYiOf--I?n}KseWHokZkzkvmj9nn){&ei~wo; zdZj})3}zmUDYx0BgQEwik_@w8p+hB9Z3(wFp1nxI9 z^%%>3TnBiWI4|Nx_oq8#JX=aZbCZ?3^H^ciwRUAJlR9q(68x?Iz_zH@IT_LF!I6-K99^D(hTbUF&<TA1fR>TtMQyEA-+{3Kx>}w^{=GGi}~h$A7ryZDBb>gcY<~1EsIWVrBetgqy;YR*0#?77fVtk%s?AO^zE4XHd*fF zES*cX@0Pemzhu0Crs0I;bm&fJz{gtWuQB7!^SjI#zfetw60lsXG>n@YzMEaHm_jKr zW$m*SW3d5$NAx7zW7KQ3S5uM3qTfLr+k)6ksaKpTTw+BbzKL~*)5amJb+|+A}BU6UpGw9>!-lDUqIC!$) z`!{`jXT8vjRu}eNa(y-;qTXpU?OyFREg%fOf*$nG<8pLO`6myS;;U1-P%F%_MCg25 z_Zpkc>e}vCPqV+{t-hynfk!X7(M$5M7iY;uwLDc~gW0UcD=4!y*hwQQ(Ee2D(9KX> z5xdbj`D65w$ZVF$fJ&^VLw}J6z4jK9j5wCP!5pKg5UdivVlhrty}&qi9KJYJr1Jaj zmlp?u=N6MhexxsMqeA2f?bt=01A>9FIH_0Brv0cnfzOF7RhZP{JP2dUj2&U6A8qj1 zIoH?VXb=^d8*MCwiAlX+snb7OqVd(W&XEQcoM@wVI+JQhsO)azfsjBYPt(h6XNYb) zh!Tv?k}E@!8VRgs!XFkviIhhB`qm30;l6#4^B;K!@IU|Lf#`(bUkz3!h$Lhzy4Z=Z zr~)~-!|ra-3H@58O1Opz2}*U?IC-(s3aymna)$N_`DWzVI(pR<$#UhHL6eau35}0` z1d^)ni56ySghO%rjUJ+Bx@*c=Cb)UNxpPk>XqK#A^~L5-{($hm{zSp|>onhP=I(p> zM2{2fO9caU2ayb4-sfjdvoB0H=b8@RNJOUl13o{F_+hM=P*qX zS`WWtQ4@_!<6gkkKKtS0aNo@w^u^IE!xQZJK__SSh;~1w|Mw5;O<;q@K2--qGAYXn z!xMMq)U!)wnwH+2WppyFG{~tRKFx)wjB6ph=Y9pznd3pm^Y^oCeqOd>Irl@pd>3?< zu;Qui499=InXv8i_IQi;b8VDx>F{lJ=|2J(#Sg%1824o0i1aU1zYxLhb%CXN}P+vxep3I(G;WmH}0$gdynA`#!T?bz=h(hZgNmYLQj6MxVVV28h zt@XJ5PSg;Ir*T7f`{ST%lH+HYWWTKHCIfs%6qZe@rp5qaJzTOlow%-U5=7!Gp1L*u z>;*`Z%t%iei{owkuc?_~%4ot|KIS;egzzM;s-)^|$s2)J3eKRu4O8lvz8Ddeh~9NL zF8}<&{WJv#*upx@udiaZ02gy^Hy|NAYb}8551f9$zx%B188zBnA+9_*RDm?yE9XAWAp9~wUD0cO)RLs zbIQ$WG5IFYrOXnwATDaYqcQK?^PU@$%8)t(wK#M-fYvR>+efAsGU>m()@{0b*uR3# z9X3A9k!|uV)=WWndu}^#YK&c8r$;K2WAvYT;0mO!@}0=vru0l}hP4I@QE(Pw#4)H3 zf()XkPEC_YtIHzifT=MMz+;0+xpVd#kbFQD$7K>;2U)Mq2Jm}QVbK5fDz}!gc zE}hb%zsTS-8TFn<(G1ABk{{l3*zHV~Sjb$y%c+?Dm8U)3Y(eWGNYc%u#j$}KRB13q zZq|0agQbGY51xr3<8v2_I&`DviK*o!XyLNRRSKe+6F^5gykPcDu$tBT`-|}YiQmUF zRNEm(dQ%x(@jCXdu?8>6cM1H@m!MPgYN+qy1Th)a>8-LE)Bi+!wy0M~(CgJc34_%e zcmDCE*UtVyiJcw^l^ZWmFUFR^WjG795bg;T$`lU^wN0JP!0X8#7gXpQor$+w+n#?+ zP%+A~v&`Z{{KHu`{hhbK@_6S168AW~2_e^Aj9xS?^ zbbfrb{EV7kJp4^sDZ4Ru_WP_ccghgRa0YRVrA$0U!PX_CY2T-M+hWTXFzzzrgaRrr z2Ic;e65nv2R7SZ(*_0`06ZHDohtK1)Ca+Y4HTN|?j)E_k=bEi|h$~i{X+pL+)mWrN z0ZLr~r{h^!F=XU=As^Gmsk^s8rJj2Da-T-dT>oOy9KgP=!^0LVfs62df*fA3^*n?x zH=|ae1AqExipr!f&X4E|jDnXYy?UQF%cu@^_F!WTs+NssN#>rUqG8H_Mh(}>_pr?@ zg0p2KoWsT8r@v#Ud)Sj$tiI}$*V z(&lL+j;yg=;H>7(X+}H1M~5UX9>ilT8TVJ}D>u1PlbZSm6RI)GAcuAazH&TCxSSJ7 zF<%K_0B;&R&uNu-ZMjNltj9EQ^&&QJL?WVpJpoIKC|nQz=Yt-Ce;}UO)W}#;F~NUw zmy62$;vK0d?&?I&+TreR8+PQP{$)sQsYQpU+*mw{pv#^gIqx~^AwNGg(cv1a^B+d# zqG-z6bfW5CKym>9rqiluN{!G(zpBw%P3Ip=C9)H#fD}XV1)aq@Hf$27IQdT2`fWjw zm{pD3t=Q~>#s+wQe7C>g_hqVAzIcVvnB-u;OPhVi!_!8{NEIvv=M`Y&0Z)T7awB07 zH9V7@pf5yhr>*_@N#Y$s&~4jR4cfmLo22D8fqPj1TH%#K$s=5<1DXTD{;=JZnLyat z!79uC9t;aPtk~_(M9|+~$`K*jn}a?fc(32K{mWdc zo`i7aOfg0SY;e0NNNGS*_lTU#-v_ux?j*=!tf(U~R*wMrsZ{Q+jFawpb%Z6^tc4x> z?Hx)7N*q+9nxXD83`L)_JRX52Ev;0|LF1FJTY(y9WVfzw6^?o$fRnSUT{__4_@}ou zrS@+m;jri;BXH*ygUsk*t|=FDJw>;B4Kw01*L-Iv`3$>xYmFKW^4@vu(cgNHuyqBP7Y$IE&q*=_MM>B2j#)jZ@&D#_zx zRgj5pr~mayf_Je+f@Br8jUAA*Xd9e49j#Ue$4+}cw9}k@eCLNHdSm=r-Q|4uw^4Bi z#d}^S)XxQiiCk_Ns_WDiD}h%wW7x6td`{hGv>f`=tQqK#QTW<=ouef}8$l{P@qDPo zs%Iah8$_I9ppoI%)G+=D+;Uu-!flGb@%|weE?iD5UFsUqA2iXNRp786Bu_8ikOWH% z^H$5Qb?!6j?!I)2*E54|V~fj4P^;g38$9z=QO#9MPI4$}lgm60e6>PNV-r1By>FzE z)ewm#bLPyN6l6NB4)f7H~KTQ)IWAtE_qSFw$aO_h^ORvJ3 zu1r)w{7tm|l}U<1+^x&%_u|ie6VQIahNGx;di4oq$>Zrh)!Yx75e4uFmT3Oe07U(&o z;%hIZnDC!E$8{m5ho3c>6QG_jGiPKhKX~y;6#}*_l}<2`|HLIywJYz;2Nr0{)}))j zYWP|>B-^>M<7@hk>;V@g>g1DS}We<;3SXmtQgxz;hC*Wq8-I#`l|6?#{y+Z3w3dtrMkp70pG5lonOxjT3H82N>!4%*X8X889@SPa>O zuEAlI=R5PKzCxv)3>ig6Nm&$fth4v|RT$Gwvxio#&Y*)h(X6?_Y{(bLVTwd0g^5kMlSOy?O>$yU++acLX<=drMrVD=cULQPB)Z@)M_x0g8gsJ&$26 z!(0rDy3D(48y<{9ov>$f8ixJT8MH4`Vt=T>gv=F4L#TOw8?_aMd?RJcMC=tJh5IzP zb{<49*}G4@02Ee|by++vk4;Cg!)|&@#fYv|nN^45+1A7&uB4hkqjwQjj<0jmb1>aX zVPoCV7i407OSN!vfj3??cM+nzzYXw96XTv^y;Tnse2BLy{Wn_B?2LSuvepQF$_S@C zU#^cxmW!Z^Q7JbG`H0R&8g#Mi&&`$vU~8`^MT3N9U|)}X2JEmHjIQFx;r-o`QTWeK zb0f^U55;O-?}KLIxy&`3ZKwLMK$IMpO`y~T zfaMVEZqeP`bLDC=wn~fN(LS`@IkKAaK<_lTOqN1%0gwY=dU1K+uhtjKs<=d*EuTvI ztlmW^_iX!U)lELZk3UbjL$Adl7G@z)Y34uk4U4LgwBiz=IRGGp1SzSb^zoMP^6G{| z@9~bU#Nrm65akL{KhE_{O>>*7BDxSH5%|*(EA7$skT)>k-58tWo!Z&%*i*#EYAR%) zLAmAeGYQOXQ42#$n#p1L_?(77PC&)71)OvDh4Dt-p!6fj?cVB6MvkP`o!# zY8x7o`Qk9z{SsrjK7(6)cE4*G+_>IqZ^=wPR$n+KzN2#VD>Nz`&ULbzuQlChM8lye z%uVh#oQn_s(n19(HP^#PCf>J|`Artb6kz`p( zXQZBwC#wUeCL_GgYuvemDZvGJvRNIML-Jg_A769hf}f3hZ$@v`8_a56z%R}?vc7TC1Y4h=ui(_z?0poy`qf^O5GMnRCub}u1!rB)@Nk433K$GP=0$m5xj8WLH(yxibZ=(N-Bl*DSNJLgvhm1GCZZfU(-Zb9D((y z^T4&kVG$spJFBWKQ(%4#zlekBKb`H^nJfx7ei!<&-cr|Fw~A_40B_ylk(3lH@d3v+ z`h&8FYJ}!_ps4-i%ufZE&M*=*=2@5S#9y1NUj0%PH)vflodpt&G4KH1bHXn>Mc64( z*%#q~u2Jxy$?v7w!Oa1ZR5qcwYIgYTGfo6@br>^ymK>i$Or&vEcxN$NiZyYVJQS&H zND$^fDxd3SIBG*l(}UU}JVbw`y}}&6%YQ zVyMN97HV%k{`~xEa|v8=`NcjQ-bXh|!o#`SM+d_-D< z{nO$k&JU+bn;}Bv0^Y5fl7ekpiTmy1MFn5N>~*EX*&%$Z&^9f6$rd8aO2wY0wvIWr zH@8zQRDU)>dBKV;7oIi?MFL5D1<$3`ZM}bj&YkzcC_}^(Y**rjuOhW487UO@2sbLM zlrWZ(QH_Qx*`^yk@@DAHjk?NdqS<^bPqlD_F`JXDt#BmHf?tNnGe(jWQC69Cn0+YF zT-Y0Fwcg4J!~bw79^pvA&}I~@EZrWu3~1cpGj7)>>cEiXDJe@Li@Dv=Y6#6}z7C|` z$zNA&KM5Wkk?|UKz@pP=JzVFsS)#5<4BAT{A@|#??GXPnyx>*lb*RU~|B|+kkNvB} zDKHf2Yf-yFql9<3emRN;2p8xj3ct} z)i*Bjd>4a%~%|nq5kzZ8seBUD?K4YuiT~@vB{6X7w zuv6NYkUKTo4k>+a=*=Ke_TUMg@jTTZp#AMcgJ3jm8!3+e8oOaquUY7IQufIf_;UiK zECSnZw6}Ou)rYZ!MaKWb2Dfx5R(miP<7>?+3JIGGyg2~&{WI*dsns{7{6^Ulcji3J zc%$*!StZ1zcc;B8jP_#q>%HqC7h$T`g(Rt&D#T$YE@a^sclG}K*Fld^3Ya;_SU>Lg z(+<%-P$?bJQ2l_o=yO^Pvq|d}L$4C6RwKtrAjOH)+Fu@;4cK1MzI=gBB<|G8k8ZFM zG->slnouz@p1p$YU-zOUM4Z<+PG5e=DoPVm@h*%XR^mF1hTJzf-2M zehV^koQ7~UtF)PCgaq6W*^*Zw9ik`_5_l1{!A35}r8Z)l^E!B?eJRMuxMiS>$GV z{Go#l)i5{rEK~Q$xq=`*P1Y%;D##Xo+gR-#AYXeoOkud0L4_`*H0o+hphmN`v}lPq z+B#QKybZ7G%II=O?cj5}7$lD}j{(=aYWr9Gh=HRT+Mi`4{R4B=e|c{p#7V%z>j;=T zf8!tFe$+qUQz{YyyBt!i(rv%C^(z4d(^lfC_;TpLSJ~!FQ?^l&%3Eunrh1*Tm zm3LJ=WMtKe1-lj>CzO>cZ{6jH?E?9oHZq=y1c@Lty#9{QUg2*$$|mpNftTMHE-!$j z77#Vbk`L?drj)-QOgiPQKAbkS4*Clm@$rm|W{5|!3szdTXX7SEyv*knePfMQi-Mdv ztB_O)wQ*22xt=oKd-_kACrrVZnjcBde=^Rn+i`Yd;pJeABCOWkzymc*Q?j9VYUPlT zFRu#*J?)Ob>B)aIwQ|dG3OyMBqBQfMFK_BnXtu>p&a$=Hs7k|4vyD{*#vKE24$y+kIQTD4__rcGi~>sB%bwATf6o9| zU|iswhqJJL?-+pn)Z8Cju4gOy>>I^LS`I+5ll+w}ZxBsh- zM*&=%V6;V;*Y|(-0S{PSvxu?Vf1e~I1guU+=<&t=)&CH4&A+z+HDcj!(f;?Sr*Hqp qlkZ6?SN|)*pX&dG(EtC1JiMfUB^Fx}vf4fYUee+UVkII5{{IJx|69ob literal 54775 zcmZU(1yo!=^FNF)6o(dfr?|Tnic67F+}+(BTHM{;THM`jk;UB|ihE&^cl*fq`JMN_ z=j7bn$w)G@ndBy)@Gpu|sP72gK|w*G%1BG7KtVwtKnf)y0)%qdR4@X05w{c<{~{wU zPWHvg-pta*6bgzi-q^qZONNnd#L&>dU}S=U{+*M%N?2Hwih=)V&tT6`Pj4QOEHh0< zXA2Kw3s$WUs!+MN*%~E^?5U1id8tV$mjP%qeBjdm;=0;=T(zdyj3&$SQePf(76C09qhElj3`g;iz##7G_xXgCar`-W-slOF{| za2OvCZ2*S~hq{lr=TCGo@5B;f-@yi`r-%RS?QL=!1?4mY1;xMm?d{DQ{PyOb%ErpN z3G@2}0V>r}PdC)}!U^J4byE!)GX({xkB~Yd6m+O16da@m4JibW0tE$|@B<0}d166| zL>|n4TcHo~VE?I8{$&(X6_=5LJXMXIOik^aE$m%7m{)fosunEOG+Z@4i;Ob`wxXHPp9Lk}i9XUcyo`L7-cQ)go*O9vNAdpokfdJT>2U0sAIDE=Dy=lAb^ zntE9N&yt<}(XGtZ);e_FYQC*eaV?KbJTBU%Pmf#_VcGk2(wUidZGdWxFd?qT z`!>P5`y%Q=@X+Uw@4|b-SvdA1+;koy9ZW<3#otFI4D_Tr`S*AE|05wlQ=6S3{--@c zl;M+Tc%8su9{v-T8P(367%bCFn<-=!)u76XF7KH!lU;_nQhi-E5w;i(OE}_Bs zBLXD|8$r0}Fzcpd$(+ew16r6PK;F|ji=S-mD9Ms)|Ls=>DRe5=!Aik<&m3XJYOLo_ zcYBXH99e$`7~A)OqQ1%UgtmMiv@tEb;0P4P-QASp#JhwQSf*v1ZZ9w63Oa0n6Ee6; zK8tyX4X$uT3=$ha25joXrj3M;-4s)th@25=YmsFR0Le2pl5cLSW;mf>J1k1VJHK6kd9HbNE}EYoz!O z5|whebSA)!j<3X3R2ocj*NWvb6Bghnc?wqsm4xHH{1&?MM|kMI`7Px6*|Z)u1of5XVA88k(kLKNkLHP@2Ji2dd2p%FxUI z@b%x3;k89<*?unOZkSz=HMi{DUqVyU( z*~qJ+F6nR2LsE>Bs)7s&Z(vi1=xj8NWWJqIWg9&1oX!eXM5~N14MOS+=>`54PIC}B zje)ZM@|~%hN_y6BRiONo&-ti&`)OiII9yfI_VzZtQ2XG>h%e#mf(W&Inrrg(jn9c>H^9|uHv0@57 zJ=*Q!EVo~fDLVcv%YWfZoqYCwkdBjM$zYa>Eg8er_hm}gTm~sy`Iu)1!N5?_wEaRk zT9gV34i;^?Q%aGYvKv#cNKkps=CzB0qc|&d?oMhj*0k#+8$iymGC4%V7p%z*`Qc_$ z17JO$+5ctou<>%Ya~7Fe_$7yAeaCdTN~q4{_+2p54GF(V-mTmzwX<+0t)(S?$t!tp z?+jl@BDuZMR)xYMR9U!xO-0<6X!vK+pB0>{nKaB|^{SAF)nf+Ikelg?kb0g~KJ0%_ z-Lzpx>Ff}srlTv{=mo32+X+}f3x4!W912cq$H%4z0x_(0E8f`^njKtRm`h(|^}_C- z433#hcEPHksT_dwDf{|G7xq;GUrNm4s%S~A{B+wS3|M1ROvk>BWn6I2@s%Z>wg;Uu zgL_eul*y;3O#AKKi>-ATKj*@i zkrIO>W10;&3LIHdK7`7p^E~WV+WTUa@dt{-%mdNV{d|KVkv1wBc~V4a!_Pe?DJf-# zL}A$m{s%TZ2aDd}haF|Gtx3o57u@(8y|a^!A@oGgPiN-g@|g)u3J3>M;G?et+?`e zRPmLoO>$17nDk85qQ>*O6o8GXm^0JM0Kb`9V*uYUHX%R4LO$I^FYde_xIzO`%+j z=4?$jn6m|hkBRec;>ic;Y)b6<2H9jc#nx%hB7;1kZI7(M(vtT4#zQJ)SJ7ZN2QJ|> zb!io2FU7W<`f&=AKHzg+mExqeX8LkMcILOjt|s{f$|~z3l)-_+N_UY2@i-BvQr+dP zvt!c=YzFn}#4cYinaT5Ylba?g@Ae^mk`c^>u55|VZftj>qoCwqsTi1VaUb@`hQy#> z&5Q;C^L&>iv_zjjQLQSc8sWfKtD@g^t=Gej=6@4?;6|sn>E%|h8H|ek-mK5= zK$&|6yk+@LXPy|~xh77zJ7~By_BX9G2?C zUJ5LQ30W<1;EzC|@ky%HQc<6LsUoG>>XCqMwPT-6u5W#B9SPHdQA}?4iumXqi zNwQD0qfiQ^Kg6KmRK!DCRyHr01fA%pM4tECN+9ob-uH3s8ob2dV6xgDoYWPX##nv) z^ZLfxeGhL59bT?VaRYinr>spn?VLxl6Kf~vm zm?Sb#u_{eQlYS(Z^^O7TewynQ%Hv+-V?uf&C3Q^*O30&6-;%P$rW!rVuQr3&q45Qedi4iOE`XS1FH8Ud3L3=YQzEl#j}qbndtRlLA$;=VRfeqq>( ztOLZE<-<<0yV?>piB1b2)sGYyJ$3>L9Fq#2Y64p9$|TZ$9j;Vj9x^9kjP*Zr!n`qg zlpn66OUyR7%Bb(?nEaxT-*{p?w&m(~YOhT)=E$7td6VHR%s4yR?}DQr8JQfu_-JV0 z-D0%v&`dI(%YJDLTial&x%|hwCjc5N$T8$0`^0#6F(tD*J>~&F?Z-9SWgAn!PT3); zw(!2OFphnb(5&`j_KmSlit%hstUzoy=rddAM6hiiQhxFxAL+ti6aMTXb!xN{J`+w- z|G_ZlGpHMfP`jyKo)&zLz_utfwF2{zj836S;e*9}QTX#8j(N4^cM6T-28mA|G=Pl+Cn>RI+vPO5(lrH~N|xFoIZ4n*m})QJzndGey)Hr~*S z9gd8GI97-NZ0G~2yHm-9%DaW_vi=tNr+Yy9P}7N$N-Xq=YN<~h(+ezV{F)mXH{oyv zyDDR+SS0UMy=YZq@50)Nv0=UFc&LJt!{Jub0*v;v@?rV-M-?ruw43Yz^r?7t&64|1 z*ZY(@tMcP5Iyhl5F$s+)+*{xJ&*F08Ek}2PcZj4%v87@;Po^$>|ImkFC(=k<-Q6Wx zS9M5~7(DLGdg{ae`Y7+iKU!;?zIkFSK}F=@W$g4h;XK$aC&ESFr%@EGn22j|n$LFT z58F()zEL3!H;A2Q-W?5xC=Ux8JNmxIPa(P$BgTn{@ejFUsgYB!-{P-tER5a*X}z|G zplr$6gC2>>Li^1wjkc@Q4-D#QEt*rDyatI|=p=%4-uI_$!5R>@V*PHvl6R-oQih*3 zx_nycBb~kwHrWn9TP;@U_SG#Nrn_Hkbbj}=S#O)yaG=9&JPHYcQ%xn7eK=pY zNOZ{hAiRdMHqODpp~`KuWKflOIvj;>Y-0B4>FFuC0m)5$ixf*E&1Mq>st6NlN$HPl zMaDs&QFE|6(Q0_3@!0xCva+&l&hIWZH`8P^N%o0E{5pE$>#*d0l&h9zAOW~|4#X=Y zV~8wAf@!=0oEEM4GId?bhWgh_e|P9TE7g7$Sa|yklqXJ^l{Vtu#vhay~n@bUs}?b)ld78=J&Af zUK0C7_28uqMkeuYwcEejGScmA1wWl>$OikKJZ0G1#`|Eh1`>zgb(9noQCuVU(Dz79 zY-75OR>|?vrnR}(>AM`C#w{ufo|7JGhe>|P4e(dDij8vtL=V?w^8=qRhnBC$z!yFH zPrH1jmX`u`f9(3zTV3wa6{PG~t1@|nIt0hPSfSV5x8kmLDkG@~MpLYtfm2@IeQdut zq_4jyP?h(Mu(w*td0=I+y=amLK2R-Re& zh=}?K#dxWuo>i6iXYHCF~ ze#f8ZVI6#dt<0G@e)3(qjR+aOVyJw4Vu;8{F!)}EGYP7@db*E}%r~6@g8T+sguz_D zA4HzyXE^Fvx1B5~;lIOQzh+H}fl_7NE6VLRx(I=>O0vI4#y~$hr&3_Sw6ZfX*^;|W ziwwU;x6NzR>^iS95QtB^XpK4awt6t20Ky4S00S&NDS6@3o(bqMs=0b|rNVVc@>HGO z+UgaH(|XtK9A_kmb|gPru1cGa3Vg}x zjwKc_3kwM5mPvGS(EJ{Z2atU=BS3TTQEax~w7WJ*2Ti^MsG@lgneYM0EE%;LMo@Z^ zgs7=MbK9+1ASUb$uxrpDg=)xWZ~{?AvdCEN7jo|&4>IjduJ?c?HW}`=nTfm_H2$h+ z2^uCQMKWVg>`F!ot$bx1o3+Nn1o&ZWrc{o~Gznz;&uhs5W*$VPHU8?Qh>SaIqiGJeq=hASt zi^@@vGwsKpn(U=eO&W!hIVXBpqy#(`?WXbDuSTPS7L~Oo)@;Tv;q)RlwD+paH%r0; z+>dU{%~m~Ye!+hiKOojL?skjFs0`H7S8(xB(AoGj_jK#J` zHLHkLX+sM>fh-8Q$)udCsd~QJuP=QJcT!F_F|?do>LpbOt{lGzrY+6(J^Rz~p2xOK z*V9B*j7rNHrpoD2ioDRq*G?@O_Ydv80>MU5Md@7du-WyLYDChMweBp5cEFkBgh-}O zYwdFHG&{|*J?k-uCnS3-6zaXq{Br&nSI)V~DkfYOGEZkb6F3_9u6&2*tAqo+7$Az!HW z0|Q$((cH#!ALCuhWLQPy#s?CMdaq?nlA`_bQ&+NeNvT?$3`_(^C-?`4hchplh$87v zB@zY>6;Y{00XOVl>64Y{pIeY zV7T5o4HmT5@z6S_I&q3T!7L(-T*Lk!>EW=AbXeRc^7v^fi7=yMoLM?oTDpOC7)nEi z%M}2BZ70-sXzaEVU@GJUwnm*MY?*XCY}g%Hqr&@PL3#i!JC62&)b+lQ;+Kl3tfXac zn_qAkXg6jr(&twelncRl*TP8YS0}&FtLue^q6y7MckC4jr0*FsJ)Mnu!=Zt3>^CIg zu@BOD0~wxE5ntMgQWQ`$^qW{thPi1USG|b@mQy{mu|M8x-uo^zTDvS-jW2bF;g03m z+`->3TF#c>XW_@jqDm_5;AOUJaU(Fo+Dqnj!+JlsH?DK7jb zo#)@^VY>gJ4oOj0l_n=rjO@!0`CBV8j$u=peOCI?GOw!X;}QP+amFrkJI!LsV;(7A z#yi@k{-YYy)TG!@2FqdQy!#i?0>!bRacobglUeS$_3h}FmlnkeZ2;_>vp?)P;b!%>9&Iqa;W_EOrsDvn zEdUmR57Z(BYPY$x@XHL9e-)sloNB}jj-(8CGlt$RHCQH=1IC=I04GwY8J6iHOo;OR zJ?!Z$)z0kw=(J`{m0?K0tiUVjkn!BBbO12=40Q2~L7IREyITXY0q{pp^Y@~I;kkyN zV8#p|=hWztVNZJM@HEhOMmq2pGo~w(L%R9c@=ph<-D>LUSyo^Bz3QX!mhPWH4OuO< zo3+u>JryFrwC?m2S3~dk8p3yw-TBx-{Xhr3z{!@P1(uOpY0oTiD)`3(sdUP#FSv(M z5YamveQruaiH5Y(7d$ZBBf#?>93M$V?Cp0ZYB!r;vJfoFQG>>Byka8gux*j{jyHlB zKK?XSEzXfR{*eBn_8`7P1I>aTj!ZLzclrH8$_SI8mh8oYeRw} z=%XVd5~j(|HM*i@=}Mw-`F-En))Q($!!kj?-mcSR9~bYetLv06ep}1Wx5SOq-lh}= zI6OEMW-z0o@4!18&5DkFVh?-6GQbYo>?c;_1xrdG#mhFeGGa3pGbJc5N>WCHaP~x! z_z}Z>_FSUIiSvL3@ZQRZ%J7~tfd&>SBhYpzFhOEWl;}@(QEy=U$_JzfEfd{=o?9ZD z(|{r9j9{upJOPW0sO(q6j2|#J#+uJd&*`Mqs|~I|6#=Co^2bY-MO|Azq888WNz6uV ztaoe$>DdId`%M`byO+YVD?mhm>Fa^PSnulSz_n*~-~(RnBZjKch>IJ>p|2K`^gDT_ zH4Wx!Y-hAT4Q3ebXnPI3HLoa9Cq!v6I50zp5)mnw3&BMr=v;0khI|$6$?=|-0@d|8 z8FtGDRDF=GEoHMaUsCd!&1&eC1`YWA4Vh&vOD$uJP*UcjYJW6jr}+{@W4R-iZnF%! z+vaMInG}{oDWAyk5LPm6ExP3I#5Iag8S{VG$4MFCgMI47v-N z)c#J0m*Onxljk{{ZSC{0@$gdj_Vxyr*a}cb)^l+(!gBTGAUQ4}uT643JwPhmE0W&_dyEHX2N5SFA~%6km}E*4UHXd*-!-v576>kc*DY4X-q@#{MX?KH z-@H6^%_>`e$ZF^iZO|=jGtj?!e@wK}I~NKElo2~aXPRe>&258O4E`3pm9DSk0m8mn z2?W_qXZIH~C_ZV$ifdUMGzX%6zD+2l)M35PTFhbHX5%5aVn zwH&{SLy`t$5dJ1`>#%ELd}U&(4l(x)xhbA~%Jz*45ii*1kT%-t8SV|vAOi?z=q$T0 zy|s3RoAhz>U<;t|Q+>wb+nqxj2%+kM&L+T|*y2o`J=O8PEPrWt)4S&}El`(eO=scq zIv5Mw%9Z_1N-Q5^U_p80++9Y|j8BD81Cgn1HkhI?Lc)4Ylp*xi=C}PIR#Y=$UtM zHqKIkd}Q85^?Swfws7`1(J{!xxRw)Z*jp7ReIJm(&xQrVrs9WTuBPH)kd8{=F9Vh& zJ0|@GytDGyx(6e1Gp%hzMy-L-z9u3mzFA=S{jJc)8s<~)@x@jU$my!x)_k>uF_Blx zI^5Z3$Z&lrm+yGHE+{$hacZUKX_hCzcCwo>v#vg-fx8xU>3$&FlAd;0w~5Ko@Qp&x z75=bo!L>Wzdegqq%gDK=lEp#yrYX~GOk4yU4^7~ar6Q^gY(v%F+xcb36%3>|RhKfd z^VDoM#O{ayWU6NR-S!i1XZuRmiKS#=*MuyE$w-wJ%0;CK5temo~i_zti=pzf&CX@VaN zxJx8)kp2}db|$EitIG?2wmaFKgj}K!?@7kiFWhq%(pmFY||)6F7SiJ4nxcCc>@R*#4x z5uVe~vJpty^;7d$a>Np0?y-FBPZ#E5O(0t2!RL*e5=e8pA<1@bQ6Cnar5B+h*DC~Y za*?8~H03;7ch`58EyvwzB2*{F?dj%O~m-cLMtKVQp%CVz5iq$r_gN({+BpG@AlEF_!A z_#`wLRtvU~0i3YJ&P4I?s=r<64Y^E+WrmEnDkwreqX7<8ld1=hw8`*Zpg0K(G`0%NX<3N0uvk3 zNoV7vHI@9TiPK_8u_#TqiUJe!71P472s&1nLZLBpHb&6RUr+{ND z4zIO(pC;JZ>He%o_P2cT(2Tle^d(^`8>3-<4m8sy_DgSFS62A;%UA%rQ@ErzHCo>| z)u4$coC8Z=7wCN{Lh;b}vbHKbdJ<;9_s5l;+Pd$p>AYn-7KI9pb45OvTX9sD5yeBN zk^Ra?3NJ6`TVyBBofr%y3rtH%C#LE+Nw8d$=QI}G)){8iRT;~jrsEH(*EtY^?r>XOsS zTvFL6LGkt})k%#tX>no3tkW4>Hx!!i!&MVKSsti^Do|HM0#+>@_Xq7xaARosXP@@! z|hUNp2H(U$Tqjmr< zZTDBxEmwrmz8?O|9B04rU?)r|gcKl{8=Zmnf{k>;d>PboNpO>&Vsm0X)`WM(GiKoT zEbM-+v-|!td!Y{Z5`R4kL-54dLU6ZjSCiti_3m2r<-ew4FdY;`;fjyrI~nZ>d_taMI}+n?O)F7c~HXyOnga*jX|8Tud7WH> zxpO)#BYhez%564l3G2`BEIyHw9c0uk`S8vv^wR6LqxXTic%f)hY*W#uvUHGioOQ#H z!m3k^r}VDYCB^5~^E%yo$6LEZQCjW)u*z3t5pi)^k^%3Jmvx_vL>}^9muTfs*hC%l zA84Hq`|I0lye{~fzwvSUqe$aP-&2A@N&N%E7r)rZ^>d>L3JTiVFvoDLM{phI?b6Pj zeeS5nVGK;p6A|1#p3p@YlZo)DcS2n#?{~<+M-bUdg{+0>`JJ&L3qDs-H(P0HQP5*@ zIKI0}EbYGLxY*TQUd?;10<6oL9mG_0u(`CaW&)S(#H^>T0vP;bCs`YM&BuQ_csJc& zK`o8`tG7whqd@B>AG8D_i)?XUPu^1J82cM15{&HK>o(`*j>G46H_vBVfX$&3BLZmA zM!3;2Up__j_h_!?7v>F01UHq>Of7CchfmAb6)2$Y?Adv0hS|9;yBjx9tmi_N(pdxO zk=MVOomF2^I*W;|2}Qev!FQ6+KC{YAVAUu4ZN&x0_|W^#aEOZ+=CjI*F-eIedyVQBKe_)R4>AD5q2MDSDclC!rqNer;4v8UBW>jSUy zc3WHak4u7L`lgsF>8>6l!G03(;aU;CTdH0h~*_GQ3 z7UQ-bY=-G|ORJ8Z4TLqzVm%1KZK4T@H<~j|K5~EV-NpSM*PM|SvysGfAC+-fFf>)+ zAuC-%J_d~_;qRX;mG2svkni20eGxEz|H{oC-@|>M;n6g{a%NdulAal8dmP+vg=Yb> z2ltnjN4z>`V3J|Zu6dQjv2PcB*pmOLBLZZj2GP=Cumt`iup0UY1 zXGFl~E%~V=Tirho^1ou8*E4q88=qU_nt#JF2NWs)1MN`#)Prgy?7fJfh-9cxfyns* z#{B9@M^&usk?t=%f+wt%NlnUB4GyFOcVwad!#3Q!9(E#qwM~oT^A;RT+mH_o@=FO~ zgBIM;)bO_?$sNW^x3~oFA!4{nIr++_X*Jf*dGUyJY&=-J3J^9b7^;K!r|o(%XS%yX zQ3K)5#Y;-%MQaJRd8YI*Oy42M8aeXHeF$JA04i3)*B(@=5SM877wN+k-UGReK5=rC z9cLCV#{3Wnfuz_8SrVV+snEepd=KFF=fbVasQx#M1A?(Zgj<_RQ>rkB-wJEA$fES& zhauMc!{6_wp}!mcl>h?Ch!M3}w~w|ynOAv)U`8N)Y=In}YhJbrfeAGC%4v6(r#KJi zlaA(Y=84`1&IShf{+c_c(;oW^=U51ZfJAguJ**}O{_94F220E-u7i?6Rc8o(;NL(a z2-bv$^Wfw09G?b7fw$4y-m&)kU4nYgv)x)sFPIs6u1*@eMnoe2`CL$jwk>kGL4;^M z#TGjo7~qQWhBdrmS1;?5FwSS@3H~GFAPpFp`+AVrQ{22t2IEqX(&wFfgUtWK<-)WbzTJo8{-?U;DasE?E; zPO2|Ue@vp7ib8CWAE(;;Xrp{>0q>o36hHCCNe5$20h3I25OICIdr>p*=SS02yC8#U zfe6i$QtNWCff48R7uEEyVOqR{TO&Itp;Ns1FB%Hsp&u9y!D;hF3;z$J8J8&!{8mQj zKXJlhsb zIy|;|G5`;m5)ky5DCJ*2WcDPXI(yi-d^^ruk{cakxSfxVwqRA5;m5qZzfLHWnl+W1 zaqA%YfUR@~w8%y+R-uQng@R1H$GPloibDP|U6D4Kr-x(uS3kS=>j(jAn3HyJqobpy zo4xNnuNogrUVcmrZx*lNu$3psDrRKUfIv9RU9s!@L0%r7{htEnS{BvR}O_1d*oE zI)smfvZJzc3SckrAF^20YG+D#zLUg8B7q$scv-hu&M4PzhK=f|0-y)Y+mCVikf?`L z>jYN#2Bhp3VLZmix2+2LN`{Kgq7+WZAz>tiiqjouaD{rEswlK;0AB|4<|VBt`}2Lt zA!2G#!lGS*_%9wAF|QI2{A;JuQ3E4%mh|gjxuS2~VpxlX%^ra$D)$%h+1lK|r>~nM z)4_moE_Wx}pDi208&2n*xH8}vVw3GdQdwogZF*B(Wj$tWbDbY;Xq-!5%@Oca&}qkB z*H=w9W2opy#`uc)zo{o>J`;*p_`L=?O(*lnx~Z2`=sXvvbABBw*VlRPJ-iiU5L^^zy0O(bm-@99?2L}0=(8HcaIQJv6{izx0TeVDV z09x{^xYTGALv>>6+pE-ai&uK2C$}vpDX9oPm*>X@Hzm!stw&{dxcnNS1WG`T8}XtvLXM}pUUBYSYqAq)?znHDx8mC!z?$5q;(&TpBQH*r2 zJ>TEmq>6*xfcY=&`$ze(PW$K=2`B!~9Lr69m3#dIaub~>10XJa{w4fMY{KLW&l5*w8YOiOAgn zqE}LF9jA=qW>pz6V_$||g;uFgG%`GlC{q3QYEIW~vl(B0W6K<^f2#stO`7htAk|sj zQfby&RB2k%79`CFlKw{cRI0A6qGoZuMZmTHk%_*4uS9KBxmiWtX;e0HaFx(Dtf|2X z47JzXjg@*vRGi@ zsn*F{30gqPnRx5($Zt#xD|~b<*_Pf$icy-2_2eyz6P?QaH1kw+B2x;4oJnkrKB3u#6IwhJAq6VsvIzu%Lwa;h8meTyb2*-j{+ z7$9VNF>E@6i=G$Z)o|jdH@S$hB^yz)wv19|mxjMawp_$K+4&~zVNwLwC+om?S4PyU z^=IZWHzT|UNW4%}jOI$UKvRW1c`CDFV-kIYMPF~n0_|_%?1Z}LjBPY? z$fz@Cn2(>k1!3a7r+}QNEhViBbINPn$M3<`Spj{3@*)w!&IR3B7zg7|2as!Bkx0RM zvgs8_(j|o%YDKnAa(Rj%bGhhM%-8HcD>1XkyR;uj3&y=2U^>B%^XvWD3bXGMdZ4Bc`}n}iUUlsAoP~Ds zR2!|1L-=^dHuo_WF+({$ptzw-1BA~ZCaIW4A%q1UjZ_j+X99+^=-^&3J_tXVhszw1 zYo;G3K-)HN&P9gExv;H!k9xBRq4vS4M{{=07}eM zSYLD~73>&MGYE;h*>Aq%1B{GZsTP}eLth_;p2ypDf`LWI=IOvd4|U*?%xu&Ucs}b* z28?Ft^r+yTVf{LF>u$FJHN)JS@c>D8jz0w-_6qA9=bKA@&P9Mx@S^zLQb&$F4ALny z9ZXI|TzG%;>0$v;6U|A?SM%2|)19u8uY-S8rN$WMlkEy6SnUzB6w)XR!jZ`zbj0kV zVt6RY7x>+7*%upIgxIV$Ake(1V4HL9xQceLYw*Qy+1ZiM2ze{4 z!u43q5;}E0u5IpVti*HZIlHB&xoRJTOR06!|J{7AVN9-rmWX2QYcw z*`iB*%l^7O>4(Z>+u?#33DsLfaB(45)&80PvqaF;d9v+hCAXt-`(2XuxwrI?Lc4U= zi&xGmpjH521#W)XEF^o?c-#r*+27^S3ljVwdGgY|DFQFo^SRD^SY7k_k;uY@8T}XN z^HzH(Td8^Q?XQpC(zq7CT($_jH{fUZ>V7@npOYH()uKN9H1K;!Jvo0Ck!NR4ssY9K z7}l=wS=Z@!omCF?T}BF2B|NV&#yQgywGEB(%9C;T#GgNk2wBNrQEdfM4vnH;_-mAm zry&{X9mK&WRtNNurL~Lil}nwcGM%VnStU)wJNm;9*M+A+jVD!$c%rX(lBRpSq8H(f z!HaYmBaDu5+rIxmh`#1et;*6lgHd0t4@CntOI73x@H{ndhFztyIHJf;^H_U15dpe1 z9P+U4o0pW14%XYJ_yteKQ#*^qKj?Wb%xxkcTv?yDpx?EjkbntIducD;RU(=@IkW=i zFW{y-E*)6uLisAw>YZ*e{9U9;)4HN?&2MljTNz-?Kft&=(@(I(5$iL?7fs?6p5qDg z1%28rN6X45%`H~l-G8e26$mQ~ILu=70<-V_2r8XoNJBG>A|P2F6dNBN^E2`zMk0L=Y14cu(1CnUU4w z3pJ{CmEf3A{yhIm#K4$%mS7DvQ<1KB#f#Ud0E^)6R2eyOD)s%R8ydYUja4@^KMeu@ z<8oS)*^K@9)L5E})Cc{?=C#MIi@li>`{4eKxA7qRHe{sGu8JllB<`+{;`W?bEN*vANh?~#c0 zVtl@ZjoU<9j%_qEuQp3C+{~xkno@%)Z7`8%=Kn+}MH~a*H@4O79@--&ntA1ATWvpo zqfTd63z}lj-+u$6cc&QfdN~^Hl46{fWmb#G-@Z(EHKPBvnp@$I{FxAK-IXlm=hHcz zC*bxDSEO6C-cBCZX(eB4LSf5ETt^MKRHSH&pplYiO0g!>?|Jf9jL z50zU_G$Wf%)WNeFr^-HTzzz_CbdSj#>~IU|*AYe=UHb`Ie;;UD8`xjuTKcb(9VAXT zWTFvR0YlMACZ@2; z-@}Q{)(2XaUpZcIFPef;iOo2by>oZ56n57TWD|swzd+ycD$FvT&=KWWd^!}qU`s3& zhb5-dZ5uK=!$dS;OiUChqz4t zCpiyh^dARpdN`Bdr8Em?XZGzWuCFD!Udi+UC0+-+OZIw+E_i`Bd{sbowV*WXa0zF| zSQ=d<=ZBby6YB~}j!h>!)xKWx+nFi_i^e*uE!u_ce8<`LbnH~h(EEOQQU;OdZvEhj zA7l3unWoD@llwh6x(Vn(M7%yx&j$C=sC%JTSTsgP4(qWO!y$BEx*o4pHU9uZr(|RC zV!;e{kC;RgRvC2@DMc6k#-rrtC?70>H7eLPZ%%00G{X`;$l>%QT$Nnq(0J;*?4z=u zKjYIdzir<)^W;26`n()_lU-Wy2BRkRQi77VHQq?QH(V+n-sa?CEWgXFG>9$qT}>`) z=X$15Dojh_mrMP*i9)l(a28^kAGm}w;=cW}tCXhm_IyZ-N{+k0k5&!a!SDYD-UL=? zrjF^qJtaLK@Wc|Zh`WC#2l6axaow0%R6Va({u=rQr}#tE(=oim?O1olcm>ayu~a0h z8%6+*kJAVxZEnX&bq2)}*K{8U@*FdH=u2k5j6gQIY{boB^ghxjsq;QD1BN`@R1jXo zeb4oUnzhjt$H>aeqeq9j3UNfeIqt?|TEG$g)D*Z`u!H4b0mSfpN%zN_sCfTMr!!ns z;sII$lF<&=kKA3(i37H_zRi511v&`$+~ zP40#c)P+GTFG zt6^u_?Iy5U6Gx&vXdvU;5>nX18m7(4z9Qi29GFnPs)b6~7NsX7k2c1x*qMa52Xy;o zQPpwfw>*0x;TyJ+AbI){R&PJtZwl_=@g{ja&wbPIAesGM04Zzc)=7l6z9}9L0DSgb z)3g`rin$v8JMseLx6e<1cYP zL#1>mJ2ONMNUb8iPQBMfnhQu_dHyt{)ICx$yz$W9Jz5_t5(T{}mc@;3_l};Rs>Tux7VS?tma>PEaas=>wx>>4Df9TBwjYSG%`K?byp-cb;Dl#_D;}t9 z2l>6)Bzh!rQFO*w4UZOnu{6!!Nvx7ewDbKQYrL}dcj-=V@VPyZo4?K}*IzsO3_>@* z>FLEbOJUKsTS$c6;*86k6bf_&2;*TMQBg{w@L19)<=7A*{*KYT`H18Rk|IriE{{gX zV-;m?JEFL$iB~%dHVWU+GiJUQYg$}-#G8EN=`J-8U2Zdt9M2J2wRqvyQ;@k$rI>n$ z(X*u)F2jnBc7=+Yt1!!5j*Xte-naM$_tLXE zlj4$!76l|`Dxyl;mArvb!{OI1Rg)WHd}Ms*80o&?&Nu<6gT=u(zUZEL>YuHW*j$Sq z$k*2c$h&Y4XOadCYTe|NCOfxs8p zQ)8NGn%g5v>#bYKzQV6$bW8dLM%XBZPO*37yrBDjXlg;9kkDw40#uK;(M*3vF&(*3 znl-+UXdsQ4`dWy^eJdh|1pPD??Zql6zYi^>OyAa?DJ#CZCmp_CmX(hg+Q<>|n0{o% zw+u@^*(J{VuJ5;-jkQ_HBR{!6gKYik5~Yofnocqg1iKNMHyCMfJm)2&6AS3|3Ahh6 zHGUmZ6MD$zsTqD&&q8yjeFDmvA>*)-h@ig0`g-*{p2Et1ar!;@Am6$TIdRxr&UlA{ zb(6tmUF>e(Ro&`0jG?WEzw^aduiY&>%m4L}8T)3#7d%BCym#Jy!TtY{bCIeWBb&suAL)^~C? zm^yFKvs4hG_B#OC+#DQgq>gopIs{p0N-zrzp4r-sUJvKCQrx&y2zy&wMuPD0@O9>s zMBHVjQsypokB^U5Ki8vNN$l>9{}hB@kv5yypXB*t1cfv>VITrQ4)Co!xZ&CEN z-&+zei(V6Q+a$ei4J_0)Myyzj3ab!0>QH3W2AhzhDH-wW>mP3y6 z*SJ&370`){v5m`q_5hQ{Bp4`7_H8#Hs*3K(pt=_fcF})70E~Uvyw3C^7xH{muFwYr z?^T6u(*3IiK>rEW%7txxpjzPNTYVTVoFti$9Jdsw&QNjc1Kr1C;;O^Lr0KOd+qfZN zcs_}w=*-Q~@>$)J{_u76LC|NzKG&2<1O$u2CqjgwO4xU>8GZ>pzMuOC2DW(X3);ei zpi~>&_%&JK_8M))(s_R)1@qQK&nLk!!)klpdzGOyoo{&^>yY8Yx47BJXh81vMY zF?f$0CWBG&NUQ6SPJ>ti_}1+&n)BHGQnPl%BPT_=`RUS`;P6`2=5E=Jp3p^9yVfGm zrVlE>^?f^dVe4vbfV&-uP=F}$hd|hAKN%lz*;ICX#Df3sbeF0E_xtjx5Yg1G!aIAu zmvQk#f(v(uLw3G0_&~>NZb_lPB0uw89HacG^T9;2dG1^DSSDAqNOU@DgEeU8TaHK3 zuKkQNo##d+bs_yBFzo;YS7iY~&SlXjUI_E<)sz86sQJeGvjH22SMH{D^i*kgSKZEnesu=iq-0B{WS35fSyXIufVB!XUe5m1E{ ztJ|rD*kW{_B1WSZ)Zw@{=HxKpgHeE7D3voAWOmf{j*-d1L`Jv{J8OF)735Wj3vy~c zmp)~%`D9$es?P{d67AB+uR&)nE?g>xz>e1U&v1%1ME#45v^vY;5>iqoR7>4P!4;1O z_#RIKM5)8+5RUSxT9g9OJlr#%GG1(FR8hoaM_H=<3n@=e!G3!LU;Tv5$nb;{iNmaU zrq%JA_h+@yR$%k7d?IM~%-$f%SQ2G(?3;C+{nc*cY?7E+&~N8P?|VX%6o_WS$gkhH zryW-`1@Ge|-A^d8H%SyiBQkgtrDc~}V9>cKWg6(61eW91hb(5&9cEV=T_NwO0V#qW zxUiFZ`-TjXTf^9)WNH`=qM90CLYC$S;lCc|i&-RIjA^bOn!p(I{<;cbNg<56-}2%k z>}o;m6_Z+~r2(ueoa%hCL%rD}-ZvhX89R+L`@oaq8{iJev2H*LWp?>`F2CCD9;@x0 z@nk?*Aa=Um3A<9=gv;=74P@dn8Rt0H>+~A&@^DXhG@r<#k&GqcDD1G$70b*Kc(Hmq zt)JXmbsoaAefm|BlYq-+mUeNJ_GcP#^7UJl21)a()2HF*Je$QDk-=y_9F+YC+s{aI z@)wN)P|Nfz7*?oY4`lA&Ba+hMgUQW)3&Z!&X?a4KYs7wDExy6=p{j{4VmnYN{D4g(ts=wpOB;{BHJ*YiV`5+g5wGPFgC{d-AjX?&(2K=6w z*p1{`WA2I^gikDaFMw;HhWLCrT60F@J0v)==qk+Fb!X{%H7GL(3mXj9=E~7pBjkMP zrc)7fK8RDXVHD65NTtT}(X^oW>e>Oe{rduAgP;(`7Tt5v;HJ+1%=O*W2gO4*(d3C` zAQhJ7!K6NM_v@b6_47%!7#R|Hgpl3* z&sOAZ8RAwHJpM%wRaSpLO1U(Ptj;hbe5*Q-z2t9uo4qEWO4-yZ-F7^Nv}@)mnkdf@ zF}D3$-_>aj*W<6w7k3LA0shj-ETOQy&UF|%uRIZ(;M6+Clt=#uQQ(R8M04!#yT3bO zC!L^ba@XH&eE2EcpRH)yUK9Tj7Z)GZ6>7!*L$~*gZ^wSp^zk%#Ggv79lV6j#Het9m zM||2ON6x(ZhO2v}BTG6LWbxxP0Qn0uLTG5-TtNwj<%-sT?;z5bl!k9eC_T!n`;4TA z(=KsB-xkFxSp&_&8IFAh&lzYr_9d%um}6c4?bxsYt)#HffT2|8=T?^Jq(lM${$lQ% ze*DY%3+{Yobt>1o$68zi1@Kub=Akk%;tkj`v5-B(J5;;CV7ap4IzWQP#=Ma@Vsi7R z`yV32oC~6A$v;q8wvvU93pOGeU{&JMWklSoDao{6V16(3{+?8!S3gLh;x?h*P4_Rp zQi=4JRH0Z%JAwEm7@fGV(odHO8#RklQuTV9@h`kGg@@*__Ec`luw-XkRCvm1lJQr= z2t=&pVUcd~vh68(D$P!NpXraL-FNT3x{L2#Ev$badk%~LTD0JCqYQG$0lkuXySc>cOKfBPfA z8txKz3-Nx|ymr+Tl@KGmLP3CO!DK<>@1M2)qB@zP2KF6pm8GShnXuhEdVkSe&+>!N zsQ@s=e7b>8M32Qki#JsiJGf0RCUh(Ft!V$AMFTU42D=^^KL}pAFCvIGFv# zsB-JD0}ussn|t!OPquH0Ff+te(}pI;%>HOKazOc9LPXx-H;=Ad_=PM)U~`3Xv{#Dc zUDJY~9SjoCpShqTu*J$BHwz~B-gt7k;vX}jNvFI}nG?Ug7lh@_&rs7wkWRs%{XJ2e zhTa}im>N#FlphuU3QnPG!_-H<&gs|F^@y z|8B>z|NnKT$+3@Zi_0MQU)%Y+TT>%-EkU}MaCNPJ>?!o`*_2=QN&K~dfAp-TI{uC- zUNL4joFV&F|DGKx_`>99l}fZ`dlYEHmRMmmwa~BXMz1{|L>^S$vRxB(W$5;`!NUxi z#V>G{2;zB^AlRzDi?5Z^;aL{6H-#8sOUHPZh?wLFUE_hFg>Nb-6u_N(BI=H zt=kdb9Nm7LTIn%?%0ezjLgzk%l%hobf><2;>eEKFVWd+MsMJgh$+Gq>x)M>j^t~i! z1E0QVepMn~;jD;X$a#{jx2jcBOP;P0qiQ{FYA@2R{?dBU78$5@qX4xmRbQG0BCJv) zH2GZdE#$G3(JUz7l2R~7DC+H#KY`{yy58TNb~nLkplF8f4rz8u^yvltz_0sq{UH~y z6nDBr&r*!##64Sat)#YnV0}!*DdaYpCHkQztUEL{mCsv3tdo;l{IeVcI@40A&ebw} zencXeKFp7+q^0T;mLwFP?O0Z5t~pjBJoWFv9A%pdi~cGtWj6C*vgZnurJMyl#^~Uw z-WgqDt?_WytRTWcEa3NNG+Z*VAqCWr9^t3wdYd5V|+uH*mwcHrqQf#A7tL{ zlp{#)0>lE(NJ(>%2fzkkReaE@jZ%Gh$J_Qyo#&YoF1JyIj`xb9=^odoykFr`Uwwis z12^5yb`z}al9e$^;Hi+yQy7wXs`;5fX}?OjHizPl+gal}*Eg-FwQjxS?F3QJg8~1* zB;a0I=0!qW4B=ipKo=IH11PqyIe)CRRtz%5YMTi?j{nH5Ij}-;tA|Lq#OH0gfvQvJ z039Ge3JN;XUpT<0*(buJkdPx}q5gvS>iRYT1TRRvAm_81+?R zwSBfjw!-C41uyVCe1`I>wcYlR1Y&~7X*u;6yPt*gy>WP8Km3^IEk`mIuC( zDp1wO5Jq_T!XPp)*tdS5iS>-UxXbG4RmpYVTD4xC>l4SiL|ov~382Hu(R zg%goE&CbML9;DE|^D;GmHPi9Tb_s*!Soq%k=2fNR^@f7vh)lqu1rtj^x3_1SkQ{%m z0g!l~{oVa~DPN&`(U0dA0HRtbS+3UhQoI42)-#sjD6*{A2g@AIR<01gKp7_~sdi+i zD-JG0!dAJue45lGY%(UHCFF2+bjPncTx17usOz3`V0k6@32pTjk=pk~Z=uys>C{}-d)lw&Iemh-2 z|D@`|{r#a1|2%H$azaI2hjeAWIzH>vxQ@%R{6#N8A_Z@e*TLXE=@! zB~%n0DmFs{VMn%EHy9;tbpw;u4FUOTx{tyb@8&R_#^)b@y5=rSyD8#15@k8fAgGQ_ zuSvO*;Fh83aDz|)npFQGyn}YhjCcMTyV1LBYoGS3eUwbG6D%4d7%s)x>y2LP<>*O~u0<(@x{>b+zHxL(h3<1>ziRtcfQ5{B4+E4A<^I0gJ1MyPF(#!@ z>yE?`=CTMffQOXN{eq2c4rs1NeGr33E8zWmgz$l%{QZ3uJurK2C|9n^*b!!Yv%)g- zrDg0TYK%RmWe5p+sCvmGCu%4b24n%;6i}rjb~^PsIj;kqi?TkcA$^9C8FnIiIX;)m zH+8-e68LVd`3ol-3AfW>fJreRS&BQ3P*zcVHTb(4g`&{?r`C5W zp>VhLnJBuKt*y7~ja+7oP%+s1Vf6~>nbyNPBekP{Y5QbDD7=`WHo#UqSy9#Rf*w}&|fW^F|p_z3;G;Z?z$a`uO%*8N}xvygbGrc9wrV} znyV?3y}perExUK5XSVe;rGcpK4__b__2f)H&}OF2)Y&Hg8@Jh+*;)owoa1-~E!RQ` zu7{0-dPkgbx&|)=T#yHx%itd;0!V%6(G4Y%$>2p2?&YXvkp^MqrV^6JlB{1Cn&+hl!dXh1uVNvqC%T?@Wm7y=~d32Y;w zvx}hsc%|&J1XSzlo%gD8KO|Co+O^QV-BbDOTh&SPEnl9!WE~vxDs<{aa$YX9(F4Mx z3ttcX-zDcOR^S-;G$=)01ns9=S#N$?)*1F=w?3zMlUp7TiB0cJ5-z=#TEqY7(F_k@ z;pM8Emlt?uclC>|NTF1PI{xdD6{r8AfNV5KU=&*7@hrArsvtwbKjx<2?6&V>y1`*g ze(`_uc4new)d6XG7G+63lJMWI2mxUMlN=il=X~EU^haNEx&6Y!Lq|8keaAZmJH1D9 z0WU?jCo?mzL{rT;SyNUMS>s5cx6waTz6Cgx)v_}leT}Jk`egTYvCrcbe}zEuh<5d> zY;hF8wy=IeH#Br$kxY}s^9g#Nm+QkjlWU%vHajBvS`OX!CTDvq@x1w{-4O4Nz- ztAzlgvZ;xQtb&5zd?{otO!kEXwzEmW2PhQyuN%zpBLIl!EGxD5Y9(!70siFi(qs5r zvLlAiZkrpr=wxIp&x^SdPfb>~`rWx^!Qa^BUrz&(Sr!hNjobyHcZhux>gGz{ z9OHTyB2z{Roh3(FYROEt%|Gd96`0;DM75wYbvy(yC*`&4+tQ5s? zthmuI*FJ8Yu95^l%Yx})G<%-7Nly+lm-r&#O)2*E_eV9~V(Y}hp`^b1dS{-g;m#(Z z6c48q5}K{3c7L%c=xh1~9hpL!3x><$^L+`S(eZ{8e&F3=eR6Cp<$}{j^!an5JDEqhXAlEl2)t4Zotp_uifrj-HlU^KEFB>E8&c$5NyxqESJ;i)}5NzH0s3 zzojIYfrG?<^=~P?<^Ok$sUgn)v#$DE6Q8KH&i_I{xxdl5y6S!UGJJ#HKOI-X{*CuI z_UC78K!N6eg0Y@-kRATkIeYRcCe6Q7`FBrR;Go#*CL*y#D{z)J=-;@_A?$Aiyy9Fu zsYKAA_3zONmkWcQSi7SI!6pQkuWVV4p(&p#NV0O9;-T_u|ncR&)ot zNT92u6tPgIe${#CG1OL)ip=9`tU-abx5r_3<=UK+qsLLjF%bLXn)RklhsidwIG|qh z+Tn>t`JmdYSD#G{YQhGpjJC*H$35a~UJOJX4R`6TC~YKqC#aL?iu0@|`lZCR`U}V1 z67myZ;CyCre0HQb_GjPAD?D8>-n_oFSZq1!DPQgQ)Fn$r7o$l-`)bsrtEcSD>A9T0`)6x8SJ9iT6s!_I(8(h3 z;p4p;efd{C`NamqJSZ-{l_DT-UVo^m&|%dVeA%E=(bPPW5XXOU zm{<7r(u_|!i`)4A${Q96oXmZ!-`&H88`2^Ql#AwhyF~Lmx`+jy+u~ar?q_{Vz9nQ; zIrrJ*Vm-LS**j5O$6^8isNfZTS5}n9@c~{LuI1y+KOX6LYOEz8)uS+uvRL~Uo~st3 zin{S)L3eHSk#tp7dYqrLtz5_Zu4uYj5#D5$6w(n43q#dbhp3@-#$n%u)Nmj-W0n6##4#Fv3VSvgS@u`z;GKFI;N z2tF!0u~Fb?q3Qv(+@Jsj8=1UO+g;LOh~2d3w}#dnug5&4a>O%C5~3+uy3x@+jb*e# zro^62x9KWhNgCR*)POvLtLy88pjZA8!4x~fylKidgft3YG1SjXvD?*H5P_Z2af^kCkS2cBMJ%yAf5APQ#kJKs^ zov#+#2^#74nwq~%ic4gOO{_Lp$&8^2%925y(P^EIyRc+@vtelQ5-m8he#h}SZvT(cA>0G1C*f;E$uK@BFVPl z4w0;o6}yQRDYfy-ltaW1x?sO8!)rr)&mI>A`e4yzp6TQiX}5b8i^W#-@78C0A-FoF z^m_+pna>y|Ow=ldTe=RD-$vsp*~qYLwSMrGb*3pC?hAn^qZ>#a9UW2}xlC%s_j8=- zE?u}Xl~CB9GBEciWZ!hW0(D$x1QHs2OGx@y*}LYprRN0?vtDhhod-w4BmAH8)D4P< z!y~e1ak9fvjBsI~75N+7IL{+Ix&)CyBB_hNzP!Hr#8+>U&;@(`lIcJ@C<8Jjy!bnn z^Ir}F_Rbyz2O$Xq6g2E$aErG#ws-8{HowF+{w?py^^qc)JA=y{91zr+!7hFzzQ-n=vR*oGVAv zVw$4kyVA*uXSNu3FOraxLJ}$XrX=Y9fQ8Fz_EX1sWv$MC68UJ!ow&kQu+I0!%kwVR zKOqUy+hjYme)lH`PA^TxJQW;U0oxvcdsEgIOw1_b1i6{oisE`B8~r3Xi+C zj`j>G@h>_a8+J|1Ksgh4&r1}=0);okBUh++J>Q|u=}L2eSl;i;0xo@Qe`+R>B6jw! z+1~K1Y!&MsmF=s7*_ubV%SrP0h^W+F?{iptAeDV*$aPtOA#Cf>IV>~J@sAP7uCb3@ zU(b(Zf0B!LR($RUxl<;h^_;X-fo%e@ae~5Za z9{>GDYHw^xa@}1GuduO?LxJ>{-b}MTHC>UnZC22z@9eCKXl?)!<{5^3ShbV#>`x*0 zJXQSYvzgK+dV12GW~wcDz54SZwxiEf%kqcy)0Z`@?7X}@;d$FtT793VMwcT361HOb zlpbOKk#S(EU}sHLs6w6J4mR$Tkg`#nKS-$A^JFcyP_h|{$9`Lb2ToO)pZ&HcPk5=t zvV2Up28s|=aMbEl{57EC19rPeGy5sb9U*yc!#0VIv2_Z}XC!onoQzGjOYgN(E|XtD z0trAWDAjKM;OF@gtf*rL6zo!ccr_6DNmL@=9y-<_#OO6Tu?;>PqJvOV{vUnz&AF7N-Hj+V0d?aEUD;d5Ziwo z0H|X#@=Gc-*!5!wp8fBuAeeucL|~@~M?t{{Zk~e{hnDtsfP#a2_@*1^7ihrqLb>#UPa01Fc9Lfwi+}isSI&#wrvra-sGBs_H8=cuA(w?rA*?Z-%^X}DUU1&cBs=H zrk7NUq(YBT^JIWWJFpZAKMT>hq|tGO@~93Sm9<2er4o1{p-O-W=nE6;y_$pfCkP}| zKTJkdmHH%q(EiyW7Y3+-N9HC(poTf&Fod?wC9@5o0I!0*E6dsZ-$cNL*!BY^lI1Ho zlwY|=U%5Yse8Yi6%pz8~X9CZ%eUG(!gX4VYfE&Qt^8>FMtFp0!Ge7f*f)5r0eNyLy|3pQDRQ zoGssjRJN5Vv1eBT1#qGCi&oQDk_bY9vft~UL6xWZg+KN~cd6@B-7nxPgBp|PFM5>7m=xor**0BmoNF{d9M@46i9tBIEl_b_;W z(j$TE9uT~I+!9-x`jlGvb9u4L7C(=HD=riDCs&0A+_(#&!E%jCp^mG?tfuKo8u+Q4tp8ogh|eUtdLVX6)tq@oW+FaS4;YwQg?ZHoc1 z_QrVnryoDcGc$kFDt)z-h4H=`gKzcVBr@0~b~-w>Y;}AJLOhz5sl%pc+*<7u12P~F z^sk5GGNZC@@zvQ~385l4$)1KtQ&=9~qH>~-Ix$;aluRZmU{aTEsnYB;dOl6#TYE7) zeUuS1bdMKb>2<_?D^<3ULJIs+L$5i&9}93l3SgsiVs@^tn=a|tf_ z^syoML}}n^=0(!h8m5+(4tcTIiqb@YKbICuBt%ia&loSS?JhGfXPHcs%j>J)B<7vN z>z&`i9iC(a?JVmemGUJvSOiGKXIb2Ev?{rQfn7bbYp>a5)hRmWxc$3Ec)YEzIiSQ! zs#&oAc5!;(lN9#?Gxo^BP%s821cdgR8KKE>1@!3ciZ@~srEQy3|8g>x%mtfA!>rA! z!nv3dwd&8fmJp|2!phjP9BVy9=yybfj)5Bk$ZE1%+61^JSAtgSWd(nZb zSB*54xE>J-xEuyPZz_IgY=OXiDumtTN1{o?&7R&QmM*A*Pa%@We}Z40D_~)bNq1mT zQa&9hlp)Sf3#pw zylx=Ki$oxC;b5a|AhkXkkPGo%9G$w)B07S}7-Wck`iI4Y+V|YjU7*fBJnHn1o&tmx z$^{WF(eWSpVUc~EW7jI{*IV%i#0vQRR&rX@j^vXCMrm7a+dVpE?E78BGG#I3DMxOu zs`zMs{l0U(=F0G0*pQsh%T){^WUMl|(bm5Am?z}@1H4OF;K?}Ds+tI?)s!@>X37~g z-}G`L%@s#lz=`UdZ?qiY@zqxiKPDeip?{VDP%|M>QTT{q6-h#}g?x2LT;F;-9y1}{ z4aVNjSM;f-zymYR=sH)vh=w9qUJVY8=Q8Wr{8BDzn61!~F!~bv^GyLkR+|mKG04Nwc#8z3O0Ls!t&G(xFB}2kRY2UR&!hTj zo?R?ZC=BBEH5$5viPyU~0#ppB1qzV~kfH=JV$Ef6v2m1kAvLw!6FrD0Z zB51m7K?#26n6?BNFfTwY$IW#bQEft;cM}op-B;}M&Cr7mxYzWhgyCWM&sJfWajy%k zn+5V=C#-=Z!Ap#%Y8*V)yxKOEbkPC{uYrG)HdobVJ7&OIuC#s64hD?Nx85J#tcGtt zTr)*1JZZ;iF$1%GT1n9ix4uexbFb7wdKY@%EDPgPj?_!Wnr_@j(;gni?hoi=Ua}>P zN8-jst8DzL$Jx|;B2aRMvz*QgLnLBR&aTvAucA0orAmcuStJ{jo3{*caRQt1hCJkJ z@GJgp<+{C%6Ow;aIw0{Y?oFG*u9FQM`r=2qsIWIVX72M1TI^`DfJiDH=Aj6pxVv z*@I^_LqH5oQQ4f@3azio1Odn}_Z3sW&`fVFK|}gJble;v*?po?^iaH5@6bvxA4V+Y z430|gMxK1oW^VAVb9jbgK0PpC>?Qx4CDO{}5MEQ8A2&XU9fl}rH&?a5l+9$h<&nZH zZiUO~9s2GEaJW2$Crj3xUZR!GXqfiyBX4!SLCcb|HKc1wmW(|Y6MIO=7+;Z{_Yv=5 zFBp;MmemzGT=W(kS-_Or(g{bdetf>!t>Wj?4@Wp$4Wpp?k{|@#!#AbOIxdfKeo|t6{%^6ud!1QM(;O zDxgQtSL3|<=Enkj8jinm4({4kbakwu0j^?Co>yxa6zeXK(UeMt)S0-`)T@&%#PO_ zxlxG2_VzaY3o=1`{Q=W2u^&i8DU^@V=*AYWX14QPvc2GE&(WDqJM*>?=p%&d*|B?q zeI;39ywty4ABQ6zGDGilZWfd_-All7_t=JzM|SfeYeWEln8{fMt>R8nCK8>gVrg~+ z5Qiwej4=>8BU-e)B_(}}JM*N4g3Wjwm6+xZOP_Q*Jru%R?Mo)Un$U(6$6joZHCO4x zt2V)IE^#T4(s?_&307;>1mbb=T%TTXLsQ@{6bM$oH9m4x z>-UwvgvjBu!D($_L_>BV#-9J_gw;ANG&nSVaZ^TduxcDDceva3)W> z*d9Yi;ymDpEFQ(;j!vT1SF8gGhx)cPCi`Aa>(v=`y4b8KW%sf-z;_CSR~AX*qiEK8 z1>g2lD%DxMKNxFl{8mVjXEl`~+X>e!oCmq3qb7@4u*4J0q#a$AmCEIebr3}B#kdHM zsSvX;1ribDC7bdu<(#al^8N5In;2H}7gt${f0(1hyi@_4 zgvTYqCfjVE55p&vbucnX-F2c|qji`BVZjp%40;~sukN6HMK-29Dt!;X9ZnbtH4*!_ z`MnBzbt!I)_-%xGIN4>9b&ge@eZ&qB5EEXmcVnRxX3*3MgN~O{K!rK$WYTnCw#9=^gv!Kio2roeGS>UA8x9bAM=e~t*l*GWr z)L%2J35|^9gM4)NMu1F1LOj6RTac-@_N{tN$=;r~A>E96Qy<|KIcllfnsW#r;q@Hz z62eNnkoFLlKN2+JKkYpl!q05Xudn^Xmi^=XJow1(e%Alxt)A-7o6z^ub$=zVTL4M; zRt2;|*@yqV!$-{4#*PqQ>C?1Q&wtvcVo^XYE}Jb*w`{@N&SI_V@U5$yKy$k87`qtw zKP@mf5aNAP3VSFl*Km5n2-$_XmN(re@?8!P2Jvs(2!a5B=87AeHzbYU7m6|<|2;rN zXg^fw*Riz9+IAs}95apo>s2V|@1d(R?F$9bn#KRyQ5b~K_rL2m7<7(d(fSarsSW;V zmuyNslTqodrSfc;%>Qd}8VHZ+EH6Zd_%#^po%jf!|7ow%AjiO%)06GJaHX$e?%&gu zR0j32#O?o9Dr+*x@lP+0iPV=vl%Jmx8S6C5^Lt`{f|aAThJGp;u(w-lyHOs$ot%Wn zidCvT#6lQ(MMf^ExFH%H5Vvk~?xc^qK`f#M2>2jY6F#3k=Cnk(80h({&pbC{6Ex=P zD)n8RuXzvOd_e5s1^)y?kx~2^vM(@?*`s0D7t+fgG>Vi|_BP$p6+IBA+3Zkt4Rr@g z-w5H8+ivfhcJU?==-<8VcQQm^`kyOs2@kFDljXY{5t{~O%x7Yj0?GO$f!Ezq6hJS( zXt>D8g46bQ-u`Z08Oi+L0zMwNv+a>%8SFB}(SRS_WTqY;L!V{6hob~W9m>1#Ssk-OSyY<;wH zW?%XJp^HzbYpc;ftvjLsMrvJ0vvD~GpvV#K3l066&*CqBv5yJ5$$>&C5cCmR)c%YS zI9w^LOE#LU!@d2Izb6>eSU;sGHj1}?ou{T`Gdwd~s4*%U=;+f~M6t$FwKnC| z`)ci{FEbBXoCs)F92x1cWaa0u4`7hY(wT%{4}6pdCSF>bF9}$u4m=bG#iLodyNo-H zIDJB$gJl{@<4ul6UF-a2I32&VF4y^ECI1og0_tN~jZ}5QHXwIQ!1KInn)817j;eK4 zC4ckm^fE!nI~L28=d<8G+9+^=fV1AuxbG#*mn+tOcZn^lnFf>z&5tMsfecOCFg= zC*-V5hL{%&Yura&W6sjhJ0jNqj25rL;C3AxWl5>Bsnw%dPmvVv_j21rBpsryV)-2N zK@x}18eB4kQmifZTr_Dyv9}k0L{73Fg3NFzoXAHKXe!4tc|`7QG-(RL?-D<={5&4^ z*{Ie1q}jX;SZZ-1IWEUhg^@2_DlpbwYO;g(%Pf(L8)Gw%p=Y%o)!1tB=B!4Ijj=6M ze#@wHJu}jb-9gv~7Tx>+aK*^sa>>P}b9hFPgR_xg;7zrcvzO}14{MNyxF?dl@sq0* z!lCo2%cTn$2N5iGNu*5VQbwQ;ehA~9ryJ;)F#q-H#Bmkdgzx^D{fIS=v-SIms5OUi_2Z$(5K5zavz ztMRs!<=5t7`W!wiJa?U=nabc(n96eGH6Z4i&k?$~MI+olZ(fa5D3W zI6YDcI4YhxGq7EJO}7g^YAj$%+#4 z@7aX^U_w6*NxMmQ%tASD$t)G5yAH&f_0w-MiryVhS93B2(uX4hRKxtPcQxU0=Ed9G zEa34SsndCP0FF)#sl0P^GK1#f+3Lm&TvH_pcD^-AbVvrlp^6;CWhgbti!`9TOY=;cbr^4JycMD$&{tI&!krs z<+q!sm%#1lGxt+4ng81Y~chvg|*XXwl zCz;#nAWbQkL*eOUnmEF?FBLEPG=R2jd>z#a&eNS@TFyC~F)%}s;0@(koWX2b%x=G+ z#%e0Vl$dX+LN=DnkSLeV^9_&VLt8Z1kUQ47&gK@M6WAXfTPQN)w!QbJ6BEOdD9K<+l1^+4$Qsf~snlbl!l_(24CsDZBlcBZ=dNic4$tfHlA)$^Ju!`qNa4nBwcXu zd!BP<_N8u`bjfwwW?5}lmEcj@4#JpOZ0~(67b~(`dX~mVdX>x6US-uQ$%60x2uqN{ zDdu|8yaMJq5I67UsC&J?gnl30`%sbS^u7>jMKGjLt0PxwG@?0LHtz8rUX`@(KGXJR z-oNN4@Ch%d}*kwwo_1<3;}z#-r$|U zg8Z3RYcz$Hl7|G9JjhJO`_iaqZneaR(`vb)sN3^RU8grnC18snxCb4iTS&yMs%+5e zcY|$H-HRy>2Ln_3rN0~!H55}CL-yCU`0j1%&Gz9TIVh{EGI(zo)$DY4jHH%lxv=59 zStB|LRIh|2U(?IqY0CBv$_M2rqK?P`?03Piv9tw6zL{qOk*TOZ6%73HDVR?muLA5Y zd8f_%uBNfB!XRAE^S~?KfKKSOOF-y*c}?F%dij zp&O8DniXM!W5H%59u#a1^pR;_SOCkl(z!qOWR7bwb@aNuj449z@2EOJ-l+C4ehqp4 zTM(*pk<$Q$-|1(MLz~^@@s&ESHFaDbzaPBTNz}@4TvMNVIK6kliC8=NAl$Ty-T%7A z{z?JJat%n7@(D_`b5`_J3OVYB?7quM0mbb!kYtI`0HG#wnItegYl)uv80>I*n_ZHc zBgr4TjLki>{bc+lN`b3RkHd~M>eRU%P&XcznsN(;JES2y=sAkm#utbJg~hOC%^XYpptW$8t1DB@nB} z;_k0O$xd0Lu~Y>c!@TIke2xlTakn!e@;8{6xMl8Y*3Ss(bR0(3-q2x~Sohbv22~0% z$N^FHGAccNM7}V+keyMymUGt2>Y))H0S^p#^jT$7D1jgX>l^AupAdq-kqX_sk7vvV#v0tcH?z6oMzr_0>baGkGNZ+3d(UZa4 zW126f;^s)~k8os50`#UVPyhmq!C|p#u@_}nGS<}CJRf{|pLqLd5N`{KogdZ(XyTYk zcw3YghY`5na-U>pN0mE@zsz?M}DMX6WxClJm2f_Ad2S^0ci~bHCfNSsfckG?ZttNQ0g6YnI zzJii8oWDk-6fkMzV-XvhS34aBJc!${U8|Lx5S6t>D4}BoBHLz+Zu2Ax+O&C?DP{{) z%efKTB*`zkq_loY9cAX$TI+IF%+eM8Az3YAFg=u46iObco!C#3&De+N&6r!rtC`P#MP-T{2oCbSKxA~xYKQCo=~*ihL4+iu1sA- zXLFo?!Wm|BHL~Vx0*kkhA>%CKNsAo93;iOTA#JPxqQPt;B7?ZI&6kt4y7M=5KMx(FaeWJN5$9^)_l9+4su0xyvtKN z#pgVn)9aWiza6QB5Jv_?0;pEsm#SN+B*B_*<|FAG7$GTnbw=Ybp^e!h823Pl*@R(A z%f$v)oW+NiVNveHtCyXUiBg{Jk;D>&f>``@MY~?-EWL*VmR!R22ib*t1_S?`3ODPC zL`t|A4&Oc{%~pdjFMABXk{*+yR^(UMj-HL6&sW{ex2-bA&;{k*!tA1pPtN9vdfD3G zWsY&9t{MH2j?L_4-{do9D@kusDiWq96cQ+mwU89KUH3a?ZI{87M%j*HJ`qqUAcjWF zS8b36OL~@l<+5Ehuz|vH-029V%u$e0D?>S=5BwTLNQOz|WG`iAeMNkC%&)hH@vjzu zkc#eyY|m1@(-V3?K0O{!Hs!r&+3lf#M$!DTnr!*4l!p2CEKi+Vzj|mrL=aK&q);k^ z$!Fz)FA1t83a>xEctnm1LTzpmTm7iBV+r9nBOy@T@j^aaLepa9FhbbEj7x6`GGry` z6>fb>%uhT*XN@6X`N>Mcgj7Ug?5a{akI$)Mw~yJDYvf!4CVa@{Vu9oNY4H)eFS@i7 zz0OQ1b`xyC^YU{TKMVQys1Rgb#T@>btHTbgb&f2ZPDfp_M7XQ#&ytJsWNqy<1qDs= z#r<860=TEX7T2=`5wDu~`PzK#n-`%db2663gS>FI&gaO;$Oz5O`uXUnq#>1kSA|&l zZwJC~OGXlrw5ip4a~!Nt$Lnjb240F+z;Jnn<06U-C5?61`yh@){Ct#ruRUobI2kTz_5&yKx3vRp8DH>Ic2g zrcnCJ1S`-0ej{#2zwLx3UrG$el8i8@qNoaBU15DNByb!+>IU3GHgQWgDaX-uws!=L z0?G-SrlYD2=m|K8zpwJ|z{wi=@IBuXK&4iy`aE6k|BtD6aI5r<`?a%OlWp6!YqH(S zw%uggwr$sByCzPajLFz*e$U?T^B(IDSa(PFx^RB*oskNC?zRzv*l3@}JJ>Il@&-p? zv|cRB-jZ<7p(%blSGYrS*$t(VnOvK1NCzmAI*;EOYaUR&wo1t&d2Ke-#GIE`oHQAUif$08~q3m)}934?p}84 zPu`xmcSrZ0Z2JWdT5E=%G{2aSJv4B4`QC1#Qv-?mUec~J)XlN>Vu37nipNu2MMoB! zsjbCUP0rmxE25(-1C7I&lHd-%;G$(Cx_V}m!?aY z!libhpjFnUeAXZ^`csldxscH)KiOOQB15^5#iPr{oCFMl|Wj$|^>>@5%@UGmbOC8Px-hUXP>6bO4RND>?8WqyiSW-a&(Dv~=#b~< zg?4{G{5yrr^toH#z9*iYcHiq?*zY^-yM0H`{Thb;Z7k3aO_py8QO|>_7L3ATxito{ zGN61G5(xOcZ?8Tr$t^tCEpbyY+_~BoTVx%7EH;%MFLjT9*0zNbs)9{6-e&p)+KI&ED7wWz3ptO74%<*Fm#c@V zNK~xDaVc#-e*vqy;Ij4qgL@7lpaGWCpmZzDJ;Xj~4}md0GE3eo`z)T~o9?GH(P{N7 zZi_BT0xdVP45{-^z$5?P_J#(k1;$GRycT#d%6WU@hrrmnx7GO{n6Wmp6Gbnm{%*Rm znnA}t&moA{v)kd`zG*R)6cSM*N#F(vwk zzQq$70d?~K_xU9I$CIGxQxy0v4VX>6kt9VUhH_|zTWMt>042tz!u$j{>!*P%?v~%r zwrnDJ-6i4<8SgFPl2IByBt;zs3Ht1b%@-|-mmmqS*7fwoWNt~lC~(oB8=P#R#!hB3 zWhTGWwcw5?%p)U599&^|K<4B)%^9q9`f>3&^eEVaPaTG)5@LSgx)(xc{;7$#hx7R2ZY-+I_HTWy-DSHixM4iJmCq9N45ey0Nk@Y}JCntj{17Y@#$bM| zjfl2i>O=;+_%4DjX`l-OX1g(OY4X+pU>O8Ib0$VyzSor&0zmb`3i5x->6|&x@O65!9efN)1%SxV9u$Li!Fp&Y2Y z|9|K@upm$f+hss>BcCx2w=O`*gz$PzlJd=}JD9(@6}NRs%X(Cl&2NikU^9OI`AC)R zbE`A|(M<4vIkGPU> zaL1=_J= zIbYbeSrSN3W5MW6_M${#=xfKV9b;miotwpVnb8Y=w%$)keOdR1dBBHn!BE2mk)q2- zjp&SWR)md7r-YC9e_{7KZP%n6Sys9M}Dj~Q&c7oe~@}6(3C^)6(f2DPw z`$fHHF^WHLM)bjD=h6aP2?yrB2erf4A<(OiA}Jp)bf{DDIQ>asO`DthU=Sr2v(ewt z{-RHzs>s2Ze?b=%iVu8AGXr2v<1>j#DWve+W6H6mMAK`xfP}~>RdIIE>vm%PjSQb^ zwOOO!b!K1}sT{u;7nX*4Mhl)y`gRU@u7K5%s8;=DF0GC)VZH1S2as)FAu6N(qi!*g zprhk)%J`&JO}N|)k19H)Wj>LiXhXK7=Ya>dD0H7bnRHa$%6rv;PO}s_9OZ(L%8co_2$0XWEYk0`=BBbY|BN0-wKZrk?~@4yynlVnK1l zFrwW8dvS)lUv*iU&EqJ>=XJ(-ISQdkHli+Jz($)QVuc39QQ}@}SZq{4=Qg%adP63X zHhPz$6sb*^&6p+{=r^Et77Kekp3!BP7*}VGM9IdQgkU8C`!ES6kpi9b`q9}ys{%x@0o*KJ(gn8gtN}aB?;ABUb5Om zlJwiE!U%^eb>(hLEv118N8f8}^~7$j&mZp!obF(MK{6>YXjpR81jJhiYg`}2+}am> zXNxKFUS-{^hb(rxvc(@9m6e(IOs(fW^QYNdH55yzj9sQ{_GUx$7BUv=SRESlya4o7 z$*t|-bBDCW_!66WkNw)9764@;$YR<9m7TQ9~)6a~3$t zA9!=5kUX9)O6mmDNW|hRTd!4wviIyC?G+ZY0+u~HJ#V98@R#BLCOOEMV|0?-<8#=v zvQMt)P<80OttP?lCU1Pt;*>jP(4pjMvPg<5l><@iXQD}_O60)fw7EPqnP-JXMCxo1zJp zv@=Co7ZFj8&l^h0lT$h;Snj2n3YN zrgN%cQ-MbTVwNJQCA}};qpaZ0V7Sw!)6txB+9m=s0krZ8LBX$;t@|h|L31Pux1$Wd zQnasjx(!<|MPHo9{{erZJ5IHFHN~MYM1~Va1GIPp@KXJS;7MsY;o^S(^EP_jMoGw! z(DBh{2+_%h%zFUI=}+8sis)}CKrv+amb1zAh!-cMNaVYcL#O==txAowzmvFNqyRPP za=^GH^X|#~9^M&;=fiWOd}7%!X{H{O$vpJ<;6B-yT0>La(f8Lk zuYH#|2tbfO;!aySG8(azV#S&R0Nv!YYv)aU}=I`K$%#|aNcyg`f4_2zOTOEz+ zF3ygQ3BE>>)2CznPs>elgnSe@OQmTfA6+us%rT{b`r1Zi^-}-PbyLarY+2luN?K?n zd=;o)o%(EtLmo*BVz2*DwAt6-a0R`G9Au#U*^&%7^WzpzvviG?vU1zs3N@Oq^cG7! z%|`u==xSt+dk;sFY*Er26hC#l)b$l*6J>ImX9<|havSwNM$5jzGezVQ;l z>aN&tQw3Fv8FVbmnh&2Ato+DmK`C16P@ru<3EOL~>>I1sJ2UfE@M5J;Kql`!kxUkO zpLH6U)OQsHEe@$uiM%Cqx?+=+ZsMSj@l#mUqmc^K58-ucLrB+Rv!)5zqe>j6|N;I%o0PF3{oXqM4VmmLJxQ5zUMt~`!puS}8whhdk1 zJ5JNS$DB}S-c!8HlI5RwINTFd!9IoiQJrA4Xd3pQAu^)VahiPe{z&vOo1uR7dULPRa;gs}5+yPLA!U~({>|=9 zZes?O8kH?|rzCqDug`rp^`H&SJLq#0Y^4&v6c>H=9NkQ4b%>c3|4OC|l~ze#`wy2W z-y@;nrk)oG;~_n4*h&R~I0e?Mtv@!1Cs%Pn!P9c}V_ zp^YS>N^||}&b-*@k&nWdqh~}qYza;sfn7fP4=t&GO{(j_{r25ly*1Pj1%vbEZF;~|p7(BXP z@asI+y8$4vt4OBMX-g;cJlBTkpFhyqe!#RNeMeM3f%oCQ+_}i- zT%QfRl zd*Ld92*ltS&aAfc>VMc%52zF?lQH+t2mQKa!E7fHRwAL-*+qgok{DFIot#~l@m~P` z^t-ap&#RZeCNb_$W=N#s^HPRhTFzglmR37RbrdXXjbgdE&gx-8jdG8j{* zvznD!*$L(A0p}VH3~RE8jMUg;m24*|8$ib_D1rkJ`=)8j&FAs>h~$OHG+)>RKi`te zmAN|h!Id_+f9R+es1t^U38-Bjw!9=I0+XY0R?1{jNomv(E}!odP9mlJZNTS{e!x~P z^$N*?;kZizZ^&&P<-cqN#6+jnCf$`rM@x;Z(#mf^5Rghn5Z7j{hG@RwY&7|`r_ibE zDH~*gzKU>5=ks=MR&d$x)R{i}7Mf<+tTRh<;TvWe3_;wMx=su&_RgX7O3I*(rCYmY zgEn>$fAI5yln%6lTo7OnNo#Q@qYg3YR#nL7vS^NaDkkfsP7T}iaMxvqz==~BOYO8; ztDEDOIF7)#0SL6Fq zR71rFT_zFf;Z`Y>cUoCzP-^=d^`f_O{jtM*y!-Z=wSoiZ>jCF@ppZbEH+pEl+lqy7 zQfV!&JE(LTx{QEeMcL!`6_$Prgs?j{6NfF!CKDJ?4}ZvKHr-QSV1DB8G?Z_F#himZ zH$!Jk0A)xJf$&h7l_q1h9ZrlEv(ZydyCro=#Is0$D$#00D5)B@z;p{4d*Lkrcj)@d z+{A>4*3alXwg+rV^xY1#@X#LzD&ecKrAW53g`zyeV%}*v-Hsw&ZqLZiCWMXZW0j$PUYv7N(xdz^H|BGXuS8_|wm8MrTwVn&FfPu#zxz+}!oL zfmLf{2144Re>&l8jsVk9)oS>5Yx;C1zpOnd9HnXLlExCJe#%*R1ALt*RO@6qinDaJ z+mv~9^UzjCM!Y0R8AxdMh4vzy=q-Z#J_*dm9fW=oc~rf8Bh~8YLB$HId)5x{&Ckaq z#dn?FY_RH9V>FZxePmBf_3P-T*f+2NpqS9MKfv7>*+1o zN>@e;Sa^jC2FVdhprUrwUwlrdJgF0ALb2M1&L@(aL$!TO=wG4RK*_k!4tnYKbhmsxBi6eQxI6Ngs-A9o$ zH_2rpaoSWQ8(^I3gUwyK$9`AIrWl#jI3K&`ON;RX6ZsLFdPA)=5{Ifl$nQ;VF{`I! zK}(8Ml1bKL%qp`QO*JjQ{)A)F4g2xSv^VrIKLl!Z)YOZS!vRjpR@Oy$Y_Q;`jaYwZ zl~>*qe5jC!YCCudA)AgG4|4p)Ku~amsx-9PUN;G+(INsYS(nJ%UrXg8V!_wG#U;p_ zp7eAs%hOV6srC87APfjqK8`a5Y$1AI2x+NDIq|^$<}JqL0`a)hI>J-U`rUb($pbIM zArc(K-q!2EL*e?@-O88pi(=#H{-sndn->P4j?HHNr3dBA+z<2SFEA@)8Zgt?H#6LR zU2A&d(Kc86SFZmyC*VAL?0pSW$rM7Z|I4j+_sjQ-**rjaBfq<_@9Xz;#J{VxP#zeT zV&2znGBX=+%lkonL#c!f`@Z^H`^#bJ5qPU(w8a_vW{eIO@vvA@4D}07n8~%|23Uf` zKu;V{E-6-V`gMGdL|MdCywXTcd}92{iSe9>;Rz|75qJjwd-lKgJUgAy*4?lC^+L#bK(A!~`?_)US^BDPUz??dN?}ku4UVuVZ zVo(`@$%>5F$CaOU$VDEe`aKd=m>1>hT*xp63=ixAFFuoGd&I5iank>Noo^nP)n2(0 z`0Z~bqgj7DpiM5nJ@(lxBS0t)*D`~-uw-gifgB$n3JIT=bmdRWGWyh~x_H#_91T!2 zNMIBQ4|M8x0o*JwWI9_iUvI8V%GOvWmdb8x;PRauBskoq&0^q(Xj|)e{0YHuwc$8U z1QW-O*)4XFYy1(YT;)9M5qa1@(NKmC1UYeF%PA8n?RcZ_OZs?hh56B7fay=Ge`$=f z)vqy?G-7snrk~6SLu{l+2AzvPQR~=Zr2ekmirx1kZx(5lg7V+G4Pz#W*t3 zzC%Lb&ymAk#P6i(wq2uAd8(o%ojlp*keR}6oR+@Oh|lfTek>gupg+7#>h)m>zkx+UwN;TS`#?b!?YZmDgwm|=n4Wz(r1 zH~w0kMK9Yk?dsAXcq2wQC`Aw^{B;u8Wy4`bb(=U*W_lh5X3I~Yv#L4T?f#(= z8K>s#K2MxcX>plkHF;65*UOj*bp0R+c#eqUgyDA}bf3mqv*Ik4&d>kLuE^A2F92#z z5DvfPm0Q8Pq1>BL-@q#e?u%itGcfhfkIbkcLiT-wEC^LbZ4QI=S7-S2Rtn>Pbg@+g{NPuE4$w&!tv(qEW{T^^Pse&I0->)p?drk2OKOA)Xw z&>;YKl2KtvN0*g;`P6irjX)Tm@G%b{B~A=}X_hPJj6EVzdA{5Z&`Mr>I$am#u+|;f zVae!&x%*^|yLql`zMiT&n$A*OBt1~X20UbPUlL`8A&52ehg-jswBC%58;q#A+K=bC z;5Y@Ll7*@s3;BGaS#30gu^!z^8$Thi{NTZUM+O}kT=*2ypRX_P;8{}I>sJnB8DN}c zltw|-JJ6~XeGE8X$+7#-55r26_%9w9n9u+Er*|v(_SI?||L(1|F|c8yDut;qRsUI# z+GuBpT77gh@NMRohZPvOe_W*aslaP|Ae3J;QK@_?6sz@b1iySvbrnFbO!;-E*P%pe z<_N6m`AH|dhO|E5rO=hl?bw$_x3+LvzXv!_s8&t3Ct)D)a_A@Be28jkT&c@H#Srw4@SKl;*r6VtC#~hEI~R>?9Hi};M~u?XW_$e2ntegV zg4XP1SVWkQjS0A#fkGCLMKq&lG}$?FoGXo&SdG84u|FqYfl7D`(q3K?jF3POB=jrt z{|z5~cLVpm%xm=6HO^#qi&Lw$avg&y-fgJ}Vj}bnb+}t*0j#ssJGChZ z2BjCaUPUGQe*}Ucp{#%2Wy7(j4;fve5G53DI0l!NlbWa89x!(yk3r{;cg+RgCbz%% zBx3Txp)cw6dhO&D0a$##fD>AoGy~JMpBOuRL(m->}sYX zV5WicJ4-xN3M zFGky8df?Vv!Go@&BA`mj!|j|HL8R)8Mq!dlnKws@j9~Mwa&YUKQ{pgMPFc*9`^5GsnG$E2OHq@= zH`FbK*J7j$gE>>6^i_=eq_-(v>xvZXs9NN|!+fsYe}BEDbw&q}rUD4IocfBhXrB%R zQ~Co-h_6Au_C0rWYkrrucKKmwz98#Etyg|hn<%5LlwS38rnmc$Ssfl|sz zF_iT`fb~rG*RT&q(f3roWU|cF91qDdz2}9N*FJNI7C*x-2^qgFyxk zh$uaMlV@eNC+Ex?!!}I|>5eyYw1)C6&_at-x3vxV)w7nTo%zFQ5zv&gq^jmSxOd$O zTNlD_xlQbE{fp^3U5n5GoAbZ;D-OJhZG_a4Hce)wk z^^#0I;KN5dbfuWfdz(!ERvQ|FDWcN)EN4p~I(}f|Vdi|>f5eiPsB^W?J_w`0d4)>q z-%@z*b$EZm;~gltr01zJC9d+ZIE{=cfe@Q4DUl!?tDgzy2@d{algdFojq%Cx7|m#f4V& zvQ+cX)N)j9CD)5mGs`DVjnb~Jp{I)$9!?`!#&x;EoEVV~DU$ro=C~3;h@UV250LyB zUg-SY+uk{+m&_q3JhPLl4@v0|om6s#DDuFi-2u!#!?Let=|>_#lZ+;niJo_Mi~*NH z3pVP|r8OplNI!0iTwE8pp3&pz@~dW5Ggr~a>9fQ#&MvqaAni;6UW&js;Hu_u(7%){ z&EWK}Png(8Ccl+}WiMAV{z|Ag*)I;)X?}tXHHABuCkL3~A%MA}d`p{X(GJm0e%$MI zGrS2-n9&)x6(4yPfIP#>pqo*`XMzd!lK<*8gqFjfrNwGwAAgE1E5 z?wPHJG5uD;c*p_Qlp02|ZldmqHo5K-*4sZSa3 zY9gc)saPJ?gn<`1jVXWPRK>VBVQM+)b8|N!DMZTB$UOh8*CDx>y%sie!u#MY0(!9J z)q{8z)!)NO?=dmzcH@1seo~N&_-MUD_drrKB<@WM7Wu)+d} zwJwWLll%lXcaWjB?1_Avci>H)ElxvoY~%C=1&yt_O-zb1~sDcN_nKWbD@3% zE&FRlwm9P2wT3$6xp$*H0irr*a){XBbRP#aU1IzVpPPyaz^u9lbgD?0QQ>n`t5KKY zIo(sbm1utLM|^lJ31+zLJx8+L%w&+suOBQ_e#>EVkf{Qm*0mM#TZLEcx1wTy>1L5g z3I92i8NrLm5Tv17vlpHXR8y%@B0LoGtms5(yIe1x_qNaI9d9U~$)T!VwOj3hUZG7@ z9{Ei>n#t{kxd=`qmQ)yqJIDgGnv+9E5vNY3B>)R6jLF&)N8PHehloj|G%MX7%$(fcgSpJ`2C(sZ9MZD8Fo9s4dvWEm*pC_rmsz=QHbl*IHah;00t zStm93PwJHPF@n{?n4C6J`3eCFChdWibz5&&TjKF0zY(iEX?Pr=W|&&5FO5tddm=wX zRJ-E@t#*qm1uF|CmWKydWbiw})zy9~BYOEXgmt9fIr^s~ig{10ZdOKn2 zJZn)2Xhc0TSec~fW@}O1ebiMiiIp=q#9iGy8Tt{2gKgwe)4=0CuNj=LST4?C`>Y_>&d)%c*(3) zHUTd5Byvc*|3HnMY1fo0H4T&ulShyS*?k)Jos4F(JApBGX-WTX);VU-3!1x4+-BtE z9870EARipXQsC0s{HV}YrKVW&i*%VYIgpQ>>2&EfM?-@320BbhtUzJgZIliD*iZT`De=4m8 z2+#G@6f8MHQJH$38jDOK(1&@pN_aq-5}Vy#T`vtxN+d*3m?HJS}1t&>HJ9ZFZnM*GDfRmlL&X2qLms~jm=FmF$% z1(gx$^*U6_q+N3NlC5ek#qn~ydfn6|1r{ zsSRL+n!@=w3o#Oci(Qn>4$P@6Bn$#7S=Yo+X*Fsrt_Cn4Uz~|~uErGa zMVRFGh6FmqNNXmGIn;}8fJPNny@A7KArEfMzy%RgTt(b!w_N8MjdI3-{F8a@)5Rkj zR+}r635YT-tOcabW(#F=-3~R5m4{V_O-9vG1T`pBglX5=_fl$WHW)k$*|UQM4Z3+V z$ot%Q2od0gOTD}qh|j6fcc2gxgB`=`{8l+4Y8ilOpRS3M*-RX~_G}C~ zO*uwVvfWpOUn4GR0g(`w3AlxG0`qr-pLBs5ur5i)-$NfI%RX>zzS`;ZIEVewVMHWieS(XyN6Bq7E@jxZ<1Zx4$N{ zp^x1jCZI)WU7WMgsEnF*dg|fZnDeul_>(TnAl#Ss%Qn&{4Ms*Ioh|^8*>Kgs4ntYZ zXL9Dm@2#pZAOBCd$Ge zR{~zj$gr3Um9Yivst&KqC%MoA2Vw%t^m7Jh<;d?y=z z^>{nYxxO8oi+LF?ei0}$8~vB;d?vvdIDUM-Q{ZWLO&?)MmI&TpbBcp3qg3_39g#zd zH7ry00q5J+6;2-bq_bV19LHI_A!sbq+M7VfidhDbP$Pmfbu{qjfP&J3P$VRE>< z6SU-o`wQA6oLpmA9R)|bgd%Zwddv1wttB#%%~!n&Vzk>*7#6rYLohHyzln};U$!S2 z$|z_%Z7b5ieEzAJkNa8N85FDeiPoZxCA=fg(+ZXr6? z{fMumc0`SRX?hq$ZNS~ndw2DtO<^#UCOPvs<72+h2hYE8kZ#4TxHQ(>>*;9mis$KV4Q)vOj%tIAzH6l+l309{%$Pp2>CZA zMK>2P58f7q6slI2#wNSl$Hy_Ov{{4OrN*8lI5~gX&W;<5a{wz{42Ki=d0QZIbk$lH zfUyhd@~aO}@L$%OhHxlYpx@ZAzRsFY$k|n3p8hOKc8TDXf~TMlb?O<@8l6((L1JxA zT8)(+;M9ZPi^zi|!$&vu1PM(lJ_n_Fvl>2B4`82zU7SfR7e)_Z{s4!=Mfh2?L^Oj# zwQrgLrNcyG$LqGh=}&OOE3i5R=6?L){CeS4saag)9iLptxBhE5Pf!-emqGY<50;j6 z#C#VOt3(>3G7RUr9-J<^YFo+;QaP_4QIds=%o{KK?U=4F7pAJ^n!?lK`nJeMAswa9&VhkFWj7> zuqA_qhv6YQUN#W9y@MkCDib#n_6391&h0UN7_XyWx^zDgs`;3>9TpgZnQQmEQ{Zs* zAY*4zL?XY;LJG%ibyS)?h2F1OGglD?N;quy70a7(6K|k9<^YgF+EXxZgam@lMn;@` z9tbG)jl&7J5I%+;XZ|?3fO;*FsF;DDOX(z+b& zYe^vbUJV^NqsDmcFRCwA$=KmUzo?i3$k>E@Zb{iYqf{X9xW#%qBzbdRY_UbT{QRGA zkzAMitf>6&SBoMS>J(=>crOObfL(O@$T$c%qI&~rcAHTc%mHAnfBgmc0tGYT zfhU1hX0rW2Vz~=9>t*x@LkEd~Cysd;aj{HW07q~hcyV52%Wv>X8AoM7_H+A}69 zavu*221~y?^Nj2W)Lh`PW9=(5CY9;)G&&(L>Tdc>ez0jC4o$${K_HYJTl~)WmIryD z%AJ`&?_tJnO%b<#TG_~~_7gIjx;LSo*g{}9U8^kBv6+KpqBuPMn}*pqGC{m$2qpMI z+2pe><<^0Y3e?dH z@@}kD*(EMz%vO&d-a!zH@Dy3{3D0r?lVNXJ0^f-g$qzhl;JI-nW5m`!el=cZJ^%KgO|1pSEm*I+IrSSt6X(>!D~Dn<(u z@w}_=;i-RwEeO=xgD@*}*D-Qhm_e;i;a9E$8h_q!#Ml@v=3G8g)$YqrZfGNWMkRJ{wK80Bd{3*ggDxn<>@3_#%Y-`X>`$9 zGaw%y&r3exO&_6CF4xX__>ho0T6=>45Btfa(D@>eFpgSaEYESVCtADbSb_$=pO5<= zPZtMrrrQ4ZEZgy`chxgb!@Sd3GST1bccEXO-)BzVm= zq6Jx^^t_bMY416dO;c2P2`aKIO`~)K{cm?1_k(Xh`@#yhE0bx8YN~9DvsnP zjn)y{Kvu_NJll!`eMgBYnbhxJ(ZE0t2^-S&;?d=jNCh1z@v$($>Lz#ke(;A}7S6Th zqLl%RVUd3+2*q?m&nrN!*SmvsWrXhC)?m}YvLef&VV^40_9uRso}Hv zRp!j-OpX&o4i@=m4Ml3?!Run30c%enqsUl68$`>J5skue#sTHeg6Wb*qu78S0OAXo z*HymFq11}c{~nY%nr zk4OGUje)Wr+|f7W5w<)C`4n81P`Z6)R`i6TIS%OAJnf=kd+v*8nvKA0GWdf0C79nt zkqxl`n3;yTWPN_-a=C>f**ho>03$!a$v&@HIKvHvD4RE_`7-Qn8tN2AGUzyF3wVR+ zs75Ta168KuT?#-rfD?Ndys^du^*E`*-7GVvW1Qmq-gA)61G9|i8FW?`S#71wZSW%A zP605GA{YK=+||PrA0k=250e6BP~hI#90`5ZYtv{V zz4*Ff#!1`;dGm0X)eCgS7oW`**t_0xN{LtAT%Ujg5-?8vErRR!_DEEJ%||V z-MkXujUqyfFHWYo0sO<=bac@V;{t-NF~Qq8OiaI*2hG6qdKXGv1O`bp$ z5W7)}S_%1$=Yoc5Enoi%9{YVC%qj~XThBk$MTOnj`C)9`K;paGnfe#?^|#^f&t0ax z$7`WQ$o$yRzg8S@lcBPmW^*av&V=@_)j0$JWX{TQ$MpPD08^o{r%>3Bnvq5-QJ!&% zlImi8i?HYrdMQtF9ZzV*5Qg91;=19alEfO~ZJpgyhK($Z4LZj|6Ohng@g@woCFqBC z<(_>J54MJnS+K@xhi_$^b^y@i%+N0V5ufgRyNuHZlne>_t;Y_9(Pe_5l^V4|e1kPyA z8|XZM8WnuG391nxdb=t*X0=sEH?n~X@0W7ia^`Nx8RD%UXuE|7o}i!h`02tV6B3|G zfPx;GeMlP``tfWB@6Y!*J7^5{8XeVoHI#*AqM+GueBRK>yt3>2-Uiy1EUwpFOkk_X z=?r?l{kW+6XcSkDLmJ4Th;T`LyT9B(c%OlYx9(T-f_b5f2eax6DvD*CUwO*(QjL`a`|*A3@59iJAEJ%m=V@ZiFh9~BH`f+C`n-o{ zHnB6zy?&JG?0$PS%nxI&ar~Ip&GXEX0p9Z8yv_QfZ@tgv~-F4;6WsxojTB1W*V&Fiy8~B~Y265aVH|Dk3f!#|u>|~vuk&K4vtq4rVZHmlCPK$fJpYmagS8M1!I5usI??7JtXc z^I`G8AxrLCRyiVvL(O!yBM!L-?#)Zy$>xw#C>Y2Ae#m~WgI|LXr{UXxZ-!gM{lPPT zoS&2JZ2XJ0P_Ba#cL4vGlkK3>mYlS7&i$Al@!Gj&&4$DM&nybv_E;68&HzF)+ z*Vd-1d@s9(mZs)Ivld#-R{tD1eX4=-ncrdaj|LnVZuk3$fc3ot*lg9Txu7*t)IRR5 z!K@|LiW}+uxez}vP5ybq2To`PLg6sjp&;7RogD>VGI8!tG>!WHd020N%~xjApMgyb z^DPeG{x3dXmA8ghQ}5Q)H$EzX-(mON!#omrzNRlNmVXDBduMjy!0GWe{+biNVM99F zwIe;{TXP-?i#I-Dn*pY#_NFjyykEd3XsagwF8QFsM{V{0=eT)K{8uB@2EqE0E19JX z8LoQM?w8&@UuXOnDxkFm_<8j_XE` zUHhs7MaYK=85{~B%W@6VWblFO4-1-ZV5y;1X+}Cse353I10vGa9l}0m{n-M7@c>~Z zgwU9yObCi}x@MWUvF8uyG^Zu=XaT|OM<3Nap3sFq-zG^toT#)D8waj?wVHYVWw8ut z_K-&}3}Y|B!6hIP^Kn}>xzw05x$Nlwv{AmE;sa@veg!?+fzTV}bY`cEcoS(K}J-u@XAE7rfq2S@a1tFF#5t8ml1iUJ)X1aMh$=~_F#H5 z!o*!|J<`EXG^PcNCuDWxbC?nd`clW_oAtC=0lU0l8^87rJacS-&5rb5A~+3=YR^(i_g@_P4<97;>&7kx>xluvOA5QmLltZq zUO8ev(<}T;WsajNa)lgW%mVTa4#0@XwxSK6_eDuc^$vEcG?U!XU1^E>0q<`ph7>!p zMUqwPEvQs|`lF4Grun$I#cEx?N{vq@fiQJT{#QicXu~pE7hsUL8JGp9v^$zEL{EbL z(Vae*jSzOu2((MB;+BQHxu$jmPHbd6^ed$AB#~;H@-=YQ6`e3>97JawU>NQvtA(`qJPVI-!H)QzSdg8;q! z2bXeX-8d9M{<}miqdli&^9X%ztVURLm9%=jVzZHq-}Kr^p+?+s)Ho+K@?PL~^!fdc zGMRUvhiQ!JrG5U2f&7c=3H>| zS^g9$F;8V3LbF@W+Bjh-ImZ`tkKzYqorm(uNT!USI!bIyl=r$C`liyU=)zznXm{`60G>ZQ z1>8ycnJKgtPd^bq0b9?%tp&MAIM8Yu&|)iMMAwY`YWAzv?Q*&iFO*Ql5?OgJIe}g^ zxIa+fxi_8b)wD{cZMGL`vc$^e=8dj1lDF7Ol`47hC!7rSVGCvGIgF1!uNG|B zzVs_W+4|Rqp^qjQftlM6COK&^V^s8?w9;A#(y^kTsN}GTey9?(n^6+V-UV07;1yC) z%c3?%5xNXxrBUh_t)!&X6{sXA#3+a)Bnc3rq{N{KW-xMT{=2#I_$I_`_{?I*26LgA zUf2ECJJ%nxAG_}TEM(ymYc{JBhm)wBA-i^_l2x-2ar=`86IMoOh4J|KbaZ_&=+ht8 zUVnwGu`UFmiooH@!%z6{gE2@PTx?KiwqRM|;MS1RNlu@jU%1!ilKZ|VexuW%VLBz` z$4E%=0~_b1&dEBCc1GB0jyGEHFfhrwZC*D&z6D})d`DYauv`X#a`;_Tcu zbLM$hs}e@6lurEJh96r+(BQFii`^u`pD3y{5OvsoV0cpEHBT7=#MP_ggvDuaSxX+_JQA#Hf|7w^<*i0CJ{d-7Dn*QR z>PoIDnY7unQ|Q6@pW!Jj4+#FXC70w*UkBhzM-FV|6S2e4XIVb!jJd7?-q-#Oej=j# z2dTNVDeS!@tA|s}eu`+2$z2kG7D$i#plm+h<5CQ^%IsqsR}n$Q3T9Dmey^B~SJFs+pcG5s;r6UVIjc+2YU#e3m9$Bpbgae&q^ zX+39&XQEte*E?lcFIed?VO@edR@XVX7W6@c|JL1fpo4l&P;7edBbkJs^OCZ;%sF7b_Dy+7Db=Dd>71V`d@Q|BM?YdAO^K~V;{ z)po-a%ab#bpOx~s8AeC!f@&wWrj)<6y;kh|!XeLa#0iiCR(KUI@SB*(Xx}5SZu-RF z`HOKbMSkng!ESSLXi_nyWan_|Lf#3_SX{itfEWv4Ehdn+`w52~7-*nK{d_=X=SWU5 zj@+nzqPu({C^?wd|9noBY8C1v3sNSlX0GD;kl4|}PPQJa6gM(>1KQehJB~0d2dLrd ze7!ZTZajD~)uRRp?6xTwF=VSnq*@E_-3P`iVG2(+kASMh2DngLjS;(I47EvuZ=3k-kUJeVkWtHLAS~nP+@0qS9tfkyne0Dn3x4{WOF;aJy>gPt z;FYg~K|bd4Ik}KB=_9=j3D9y|joEifHiud+x&0KQ3bTd+Exl!G_+d+n*jXsfCFPwc zeY`H!wLR9vXc`wwAUi@*Kjcv2Zk2Vbu~IMHII5-ZM!b6+P(V=&yvIF$+K3d19S6Tt zUkNTi)?RfbLXGX1pG}uTl#koe&pqS;k_~_gh%w6@pPD0Om2X;bG ztBfhNbpKro!fRMWML{JF_7X9t&Y=BBQ8W@lq~h+Q-c0r#h4296{-XGBn*sV%w;O4m z`0K&h^L+kwSlNC|tao0xwnoqL5Z?lh-8UM4&RKLI8b1D(0$+aDLY;CStwq3Lmuf0r zE#O0+J6}$rY@s8g^vwutk3O(9^Mr{2p$X~c$@Ym5o-Y}U!lxwmK7uk~Bg5CvtD;o4|e7t}klY&dVls{3j>-mnkl3;cln-fD8P<=~Cmg3eJiZ z6B$sS$(^Mb`C0}g_-QN0k)A7B%v5E)8UOR4?V5o*`^>D0Yv>5eN&u4~AenTfxlM6z z5Qk>EiQb~`8OFel(Ec&;Qxo5ju_*#JX6!a{eyE&xiYCEHEy8=dwtXPxJr7P!&%djuMe?Oph?jSy7YseiCbZPBgdb7n{BnMsux-)b16l ziCx)T$@kCC{oHc%7m+|O&TPIs7);2#+G-)&*K6ot$qEEC{Qd;6cFuP9Ym$Yzl~`lx zd*Z(pfEz$G=t*|p*1wFUE&qKfe4Eopu|i?(On{&)3D!F4OCc1yl?(V*#Ggv75FEW~ z`hBM+E5lIGus*}>+T}S(+2OR|NotRx`*`NH4ITYz$)dz?31m5OPFQJ9+?+}f*g)(ol2*m+`0MDwKBM_l*Dj%?WD zZYNoSmg_?FQ-@GF5A${C59PBCf|h5BeWC}Z2liPo`7{%myjihy^uaBt8fE^c$}Z>0 zF`oDaWtByArN9p`X$#AcRqCY2E+=hlsiGZaMXwqR)BW*Ksrvg#6b?J*QACfNsH0INTL>yx6n^u^)bhO`mX?+;B zVfd>>tsG3b?TSugxFYhgQ}a1MNT6ui}FA0QI{PKnvGkkW@dG#c9M zE&mEvlG8aTrtL20p?C&ruogzw*;W&)3UHo{b&%W@L^E28bY?8ia~%W9&5*ej&u!iL zUPH)7i`qCQGh6zkLJR@o#8ElFqjO4NN{M!>Z!qIG98jq%#ssR$Cd{b{jVN=I$bQeuL1Khs9@ z3ik(QTl{@_!S!hRYth#2#&Q_#%#74S+3}RqLLGr=@e-^VqG#%n@Wf~3^aOiNnEp^P z+-U3NI)zFr_p2&`@~Kj1nZzFzIn--dQar2N3Y2v7y8s{N;jFcZd7@nEo&EdG4;jWg zenyeK?k0JxlFCCfd>UePc}r_Fwp<31;$na2i*0Rmi7>)Li-Y=Bu3xP zivzteO&=CxJO5APBLaTU)@9PGm+L!Z{ESBvymILL@xY!Wsa4xaZASo6X?SEtgGB6! zA0|C4`#j{a6IPI2ls~IDnIy9U5;(jtB#rRz>FcMJKWK4h*39IcPO`apO7M7DFa5i~ z%8U1pGw*?D(R@F-zP{d7KLd@`FO}$Z*@(~`8sWz;)yo_jc5V7haExf2U+&q9 zt@%YJKJPAW$dgo%mltH3nOF@AdhB5Ks!sx9>Nb(K2u$Rt+?cxD-BEoHw6wwx{6U~KAdkb>F z1~C>wkVVQ96WKf&*DlCG7t!I_c6adMAuEjiTKXeut-dto=)7IYup5=@t0n zg|0W-g3PS^*!idU9FZh3<5cW7q39CrdeO~@wFzE;sfV|1awcEXj zKPqG$0qj=^bej2)*|j=@tYOK|n>Mcck~C6hVQ| zq>Do6E%Y;a-o2l_@g3*K`E|xOJ_ih9veuk)&9&w|@2lK-d{m@uXR~in45R? z@^F*7dTFCjvD)TA9ZhXbB;g2hy2st^7vgboW#u`^1uxV3*LE}!X9+Ba_DEKWP7LlE zuFKv|x2$!|iTRPTbPq@G)*~0;#1s6s-0ll1oaE%O-13#KKI*urN!*7qmvEUcJnY9U z`NoE!fO%dF4c)jNSxDS`FuszPfD4K@0ZkPsS0 zhCA{U_m(!#E%|b{Kn3f2HomHlLpcPWYtu?^J>7A?6iXf(3x&#CP~JP>!FPrVrM@wfQi{6ONsUzze=CuPBcIn}k+SGT!y2j@C? zO^kyZWsgGuUg3h@%itGG|I_7z_C3SW1SI5fL z+S$7#*TNF%;V#3@e%8@{|NYN6t-bC4>B-6Mugd}#C~)?PfDpf+z<=8Y zr%Io_D|y%6+Zv&-Z0`uB2lOE;4i}dG&+-53lYe^rpEDo0S-UDB9l?q2vj5EQUnl?j z!#_{_PnQP&bSWYt_U|tL^~qmnN(-D_`M+l3f2R3A?}B-jC6yNV?>&+IFVU3H;!?NCH_Mk4>J>$Y?6np;cv%IrTf+2- zL(zlp28nNa?RllskGS-wS~?TDV+8_RC2ZG{`lFA7^wOR=*!HBR5u<|7{k&Cz^}paV z8vImX#16yz_s=1Ii2wfPJEIO9NEX2`SkJm8~;`v7BN%oy}^F?D_dc=n~T+`!GE7u zD_webt6#?vyKb?Oget~sc5t4NiB$FOckpIC+ITs<^MSSgkGqaNgxs`%V z+u!|C)QASp?<}Wjt!KfV*OKjX=t@VUqnaOoJm*INB=hP$V|}ls@?94`>lPYIERPpu z>|lJ{5E0W_W=fIdl_gf4`k;BolcRmluL?UWZ=Y#a$_AL0ShhzSJYiRH+{GA_tS?v* z-jr>OUJ8kzE-zY*;()Ydr)7J$gng1wYEtTRz#(q=KLd$9wE@qhbOZ2 ziKn@!N|`xAC0rk3K4-j8n(K&LsaY>*H&Nw57GIBW z?s_%w8I}uHE)`2uURYAfdyx0SW9j=SfAyQZJ1=i^zrXY9YO94=72Z36b4!>3aauZjC(v>mey>DMIb9&VRQi{hCT*C z>4MB|eoj3XmoRGt+EB`IE20USdovY*T?=V_*C|Ts)ZK$h1J@M1c zQYJBK3=DJ8wZNcDN|M3~N*Y&K;`fXy*F6erDIXxm%DL$lZ5+lc>vz^pJu!&3I(@W< zXA|zbf_ts1hknc-j?Tr1^(wAJ9jX*UDu7Guyk(~BS_mt8ZBB0PsQG4l@#2B z()#CVH-}y*@ZPvsvPBk?e?M3Iq@n0oe)f_J6J zLiaO)7n-HKZQq6%&&R2ddL^8d^z_>n@VAQ+6RkLC4=0Neq;UA&OrUDfz&gYJP{1E2 zV7L;W%p_XoT{O5AeT*`~Oe^j#YI6Vi#B&h86=bS(nzYfe=`9m=7&+n~eWUS55I^$3 z>ty!OuX*wo{qe^BB7<&;zd@dUCZYz{L+78{e0IytPuUVNMAPuRVu{^3r10h448k{0 z&$ZUH3b|&{f{Pi_6jDlJ6`{vJHhdjNa-3K9VeG;94C;PTWWj1jE zJ41TI#TH6_yQ8*~U%ls_sA0Zo$yM+fRqDSIbHofAAuANZ2z`z~z^_z)cGY09@3kXX zpt}QVtOmaa->tk$Hg(Nj?B>cSr_qErgXsC&It}iUMa!%I)pr+TUtml{y$aLYHWc;tyG=^SgNlEW-CPZd;6KBE1s zx9%^!{Z&c#-7Q+Jt);x8%7%c$v37bh1PE~@_p()eQm=J3p6nD|`n?=_K1NXBrJB0P zPka3yY;QlZ%?S8eWWxXJV!uNO-UUq1>0yTBcx5j|ILK1Qz6dlT_9neDC>Vs>#1Lrn zwfa5Ni6HN?z++z}c{8s`5B>v(?;mDnYeB@*%~ZS~&a{5GTDw;BhF|+FiV+_Qfs*Lt zXjTYi2bD62*g)+mnc!b#Pq0?v{DQJ#j$`wIg6u#1x9<4OKIY0d{^Tm=k9C$>Z^ECH z+lVUAMbv&YtoTU5B8~pCakSO%CKZvL55n(8T{&{t;8XQLX0+ z=7-;!Xl$sK?0#EZcqgmh#GGM+|0l<(hRLa|TR&W&238%hB_pPOy{0}3v;tqESOb6U zZD*XH8ppG#fr>k&EqqB9Xm}S@AcJ}oE@gRAIqu|E@}n^*==QB!rKuveW0=u0lN2j= zSdW1^b~=p8zcK@l)oa4Vvx_atyRN{n!j%xUPC|3FI&+}j&x=veJovTem_7g7)F-x) z8G=1O<}Tl`GvA^=T8$*nW0N~t4~t`x7z{k#(QC|*@^S*1V`_{izj;cU`=ENd&9kd_ z{1<}DD2|?J5gapnO<1%?)4BzPtDvXczugI34c)8y;hM*1P}-))5lzRplt-G+KYeX3 zGm7QW^DsW+8QHx)$z@)%Cv23Pv>r457kqvV7rZKpbY>-lK7z8jo(mVnRU}vR3TmH7 z@(>~vqH9_r$QXGDQ6{S;6exoLrV*hV>#eMK8IDrK7^`@Rj?<;@g>!_8>C!=CLQrOe zt>pq9uI5CxT7oBAKPm2r(EU)Q;JGM8Pm;lfgss_wMQF?^t$ZMAquDrS4;hMdXItraIW6z>08o&3{`;mDEGUU z$PU;g8K-1#xOHgAjWng7d~#bf%vOyvKAIX{7UM~53^wteiD=ktk`3rnF!mUFz~3=i zmVjgX4wg@sc*DpIUL-$}#G|)coPQtonYtHY;5!{cg!#-S@mSg_nbj+b>r|hCY z-Fhe^|8K%u)sLG)QMMitZWKE2H8$MH(!zeBOROfu1J@5Hy{0~~wSByeUb?h*$qyf2 z_bQ!$DP}zTs%FYB)l0TwSSycg-<_ofys#$`t#%f#?!)z({d8sK&&51*Ge}qYOw;gd zztvBjzQzNswrvI>zb0#|Tq;A5_s5C@!}E-GBf{;2xgD(_dE}N=hW(VpsHsg z8MU8RcyG*h_*|g8Q1&!9xWimNjU!7)13lN@Y|S)P**5cYx{GM!CCkKj-yh$DXLs@w#)`JG=490!AH#0lQYcB_Un>vU7`89$Kuxjz!{1d}fQ@pruBM ziMr;wy=M19h_O)S<mwf8VxE9BR?Jrel6ZnK_Ry-xxxu7HA;ae9a;$ zq#v5=OC3f#`%QJR};2m>ov9Ypuk>Yb~;hq=p(k~e!~TBrp>DwHz;Jw__zn`rhOlE zT;kzhw~A2wV{Sx4WK3PquNYXx5GnmwH<-kuSWK_<+hn>$G3+FvE*$<068mDi<)Qo9 z;acOK8}n}leunnU!*gpn@d3dJb)=Fmt?tGjISz&)L?eiKIh`~6?zuByc&86Rx^U86 zky97*Z}ie+$`0}cfe>BcbH}f1T}^Cp#!9iso^avo{!sZ>imMFcg530kzr^b`Dem%I z7348cG{b*Mb6wy1954-;WV>6~gZtYTq;c!nHed7#E8|{?Wx>=g63exu>CR}Ux!nAcU3hNr!Ni!xv#eNMUHvR8hEvO)(okwFeop$b zx!Aw9-9Lx#ILITyt2lu@%#qw|sNK(z2^Y3~U*~t3U&J0$Xtw0&^|`k8 z#8Z4yW=G9*FML1!eU9@#^N~mmpkr%_kQD+X6&(gd;a~Qqf{#-%5qgR%}&I&PvhRzl%7M!bVqb?b6l>A zRp-hJ=pE_-*@oaZr!!Hkh|kZt8Gf?jN+HmKM&)+*NYj7Q;tz)dIO*g2WB0Snrf|@S zKfr}W!o2u9Qoh@E$WuQ~H~TA!jUackxa{R&@XtEjl|37DrQ^8qdVy*En949%*odv4 zCPeSNFldX3MbQD5faT$W4ZmiA5$XxJ=|*eC^m?*)+~I)pjN7B%7tctL33K>XkI2gA z7lA5(6URmRrRRLudLik1Z6O=MF+X`Oi$^vCs5PN=1GBp_?zs89+vz!EcPlfaGs5dT zEAHb22e@fA;>dEoF|i0jR7K3XN&h1XzR_9#A*dz1AKSWdnSyaaquyzvT4=~y+IhZF z)L|G-^_g{+%!kZ}%09xCeV9aToXu>alk=xk`0X!RH@TiN2QjupY_}XY!NOZv_xVj~ z%E3uJ-xdum;f2(m-+tby1Sw2Hd_R`VCE`X_5gStx+_cqfT@4FS1GrmxtGPb$>Q1YS z@iiS(La{N~UX~BJOZZ_kRR#&YY}EF*P1d+?_`l`;ju9cp>GJ8ycs)N@UD~L{?7qU5 zJ?~Xv$@we~K*r|25rVe+_C79&*I+PAB=4x1h!v~LW>cmmwQ_?|_#rBWlb|bvh{djx z*(0yPsRy`JmWb0i@q4IgGs` zcpw}HRS;5K)(2VbBE|RKBaoN^wqmBk;@L&cTiy&bTUty#{H;(isC;(NWyhrJdZl4hvS`!5eQT+WHkyE) zs2iZ0pVlhR%Pu<4M9}+wYCPUG3gUZE`h2KBZv;zTwKSAB{ykU61i=o~l32LZk7xPx zK}dTHeeF#dOmS zg999(+0HM@^jzX#Li-YtF4-#&&0ExNOk+)_Cc= zFNpocldS?Z9mCdC_UJ&>0uCS}P$>OJB| zZRG5^E^{5Zx|e=)L<`zEqxfx7gy>Qpbi(D0b;LG9mAWF5qB2xZ9|GRxanz!$wURlz zEVVgt%Vp6=-uu^rax`B{uc4-z!-x)Llx2-Hx~M`S_inwMXTA61!#$2H%^8S~^RK~o zG09Jak>e+ew2j$#&udul7Urp$O+J8DsYWmA(;5K4c3CKS`9bT%L9O@DwDT zp~NRAgtrpJ3%=>KAipa%Cgd_0pG!kN0#ICSi5klXqoV;$L3Y6`9g+8fih{QUDT&p5 zt}$BVegSY#EKrPC^JvDqoyxie#>NT_-ljSr$g>9cx{y(-+>Bgo?CSn006{XEpW`zx z#ZqqSHwFeEmj$oNExz_jXS@VZ=hmjz%GeMo-L+qY)W_MraZ5r-!E_Ln`0@5?;lpXJq1CcLEwbkKQFw+409W6(5S^J+@Owizv`Q!DKnS zQ&gA*&E+-VNcRn8_$r6KASpx^#i1;Z>d5E{5`*gav)(`KxX0w}7 zsh+xug7YZGPQO(hS<`O(P^xs|ho2)en|Oso(;V1jV-*RmiR>Y2Er-Hg!`+omsv}E+ z$#~?RXB=e9Quks>R>BVCx3#I(xnx>qGKAM}NnE~xfKaJTOk^{4W;oXk6&Q`!F4>Db z7ZTFKs(H7TDejXxbtQ70d=MaK9(?ckbgzjyEXujO#9VqaUAmREFuf{e=JRK&MR_}~ zx2(PR)>#4JzDnLSEOP6p1J0ruH}n1qd`k2V2Tw5@)jGQ)(?-id@pGg+$LllnZP*@rWssCoj#xDOl0AD{HzT{# zV*Km$J(#k=fP6Tcwvud1l^8YAzAX>!ehvbqpo`z~naKXBsdPd~#>!Qv%9P>Hrd27Y`JBd-E z;CmEJr$gyc`=iQ|O z>q>_6B-E3ICQ`P&2x$+NHt|58eleD@zUqxR|0LYj2stRPSwoKseB(mYy->6b57SDD zqfY+SBmuo26a?hFgUvAP&sRu)&-5YE!BE;gyIOdm1uT)Mr8#z&X^Hc#`oVW89>Lqm z{5z7Ih#O{gYHzlPq{c2gM(&aX4Q3jbA5Ysg=J(BHtSC26!}xWX1-++lzIs>RoLH&M zOwgB^P-1x>7Sy0`_~{5|f6)4Nz-?wGJDU6-%7D$;9FPx&7mB~fMh+6?Ns_9a^UowR6(xM^AUJ1w-{yvCf zzH(2hNawd15q6RRB{+eENPk<*`bU$z$_E9yL&qGysF&g4P@`9Z=8#chUS%%p87hUi zYf8o&4R@Orbu%q5WK^iUv4F8%595%Xy(-u9lf6dJ`=*GOwiEqplyVzi=uFh8{SC|Xs~Hwrql^8o)_7!yEvic`FMbY{TOy#o zDZMK9f-v&(Y_Xw6^<2hwuE4C~cbW$C%+S>+-#2OBx$)8@-$@fYR36@KY#yLY`SiXa zwO#FdAXo;TeG-dst60MDYaNuTR~MO`=Q*L;5w81zi##DPRi`JiTbLHm>46-LUUtNQ5CJ0prjC;P4aCqNeNfVTsLanqf z^wL~QKZT<@S3LINk{>3eL|3@7o=9#!Y*TwNJWRxqw$l%+2Ew-1kIX(<{J)K!zpDPA zwDi1*)ZKHN_%Q4)ACcDg)4F)w9{l{zp zOB{NI6Q;;{s@=BkO1{l_Hbq0{i)>)!vA!%`I8JFV-qxB_cY^e%k2o9@tqrGl<& zVvbtx@nccq@YT62d(Ob0EWbA{2e9sRdKh;pd7B6Zg(UIm4UdK&*3HUQlz)1mUub8L zZK1?8Fk*MPPRqSn;J3Lq?9pgT3pieUO`V?l%RW6h@ovkvt~0(ih9PsKK#;Ak4X(J4 z;=w*{pc-l6*@H|p%g>5r9RZB9yCni8!!A%I8y*e#4R~*`u3ktVkfd1nNy|aXf!j(b z*Bs$|uaT{9I%=ft(7z+%C|s`*+ZptDM6r&Izb;~LV8of(rn*f!vcEaQFZwtc8g!%2 zSpYkf_cz@q-^T%Yjnyb{=O&N(E^WPgIW)qN@=81wDOwPhV3H^D4KKj!X z;hn$EsG8C(C*}V~p9+F>iM1+(|99R04|(KD3D|#$9R3Fv|HdT$kqo-T0GuzSC}2NN z#-}L(n!hjic9QZxND#DB8W&6+g;2^voogJn2myM7+mbftKjQ|ChaiB2z&8H%y5;#E zB?unE=hJHLtDj@FVByZ9x! zUkv#J6R9kCRbLc_0gNRdP!JC}UI9*`;bpP?xA*$hN8PTPXpiOLF#r%PDK6jGPNI+w z$ND6&4jvuvMTkwm_Nt!mXaO`Tvl#=4eHoqtdYV|xw2Nm%$+R_~{+y%)o4Ay-jJE7)?93SXpVx-xyIb$>VOzXZz)AHDW9J3*b-()PZojeBFuz$63 z((cuRBGXn$cQ}t{V|CnR(ZRKn_7AFYmpz*=n5;b0l8q-hm=Lq-(8$v-vFt&krvqc? z1)Nv|Zgkyxe$9#Iss`$vSrbU9FKy{`6Owy(Qal5wc0|%kwi85SwLF8e_l#H98c)*! z6d0%y5_RxIy>|VaTj{&=5q+y;S@)Bx^(#q_Tf7O@nEWzH!&G7GQjhVWRf}YSa}OHU zmKvzE^1Bj!O{rlfi zrHZG;x7nZgt`BXswb5AwYIBsP`rM z^lNQ&iw;134uk{ZHjT(BfW=!w5s8Gqo#kJyrDfZU z^~s{dZoqPRs<9DB8ZMQsjkxy7j*7-*HoejNd>3yA#a8;F?Lu)B*PA$r+ zLrRwe3g!AVF~T;A3WASZ7k@G)TSlt!YR-15v_8{ZMPDeK(K=71Dx2?-ysepSVw(f_ zbgXJB&`O{ItB7D6)bs3Ma4bNKx;BIgIvy0AC5J{e?zt|znerh=yAy8xZ9oM_6EbL=hiV{a9LbIn!|TV1O)GvCL>4 zsnw;Z^zn{OX*FbW0{nR;&?P!fevVSf9hrkw?iR-NlDI>~_qXctQvDwh#TAcRmkCYU zW)KR7+G4vWf%UiF`;wm-v^+43mWbTSM@}{DHsSB>8-x{_2y7-j7IW+?24qnimAD?b z3LAN*8A9b8P1|w6Djo&z05ZZx)M2RdF;FPAFDK~N3fYHVu$Kt; zt!9lSIf^uM(Y&-cbC40})=m+bs0!Iib2K-)Xra%IdaX}vIw4PdQjN9s54sF9CL77# zZwu-oT3a}`r^yottHIi%@tcKM;Ffh&%*$j@mT}A9U;gm@U}W%AETF62P2Y}O^4rf604A*)gU%_20 zOP6y4^np5mrt4J#67aiIPWA5gH?47?Rv!BT-p;Wrk&XKZ(;eZrRzLCs*aBooYC0_d z_N5W{!lJfGMS7FDac9>^_3&oDiEB3}5TeVs>o5cLdq4d8`D=c$Q;kGOY(G9;QLJ*B^vq*jLmFo-FiX1IRhEw=b}WhJtpG-!f_3NmUl;9{B(DuZ zrLUTfarML02f6pze7Q#t&l~^{x14m}kH-hgMY}*%9IY5Jb@&hK3t+0}_JGE0A10&TaQ0Z;7WOkm%vC91b((&}35}uTmoCyGYkEG@km16hK zlAL6-z)_ptm)2m*F0J#~A}s2_KS=jmEdgLpA8ZsWwQCJ!U|$BCUKnOBL2h>skgQ&( zW9S`Giz)B93_URY_fkbBWQKsu-H5#$K<^R1INS}M;$>%?r^p_%A-IsumX~8>Bk3ohTTs&=N2fDfd{|2`UQR`5>9{=&JpBg zW!=xX#=%vcnIU#}$C0(v;c}ag{eU~60Q7cdg#$Lc*Y#U{Ql@%S{#$W;W(^Y&-SlSu zPi)4({=F?3=>hz9(#*5RD^*j#Ou2~joSG>-2yCna^uZ{Az%_87bBAqUJ93HbO?fohYdfZhLb@*x}C(jVi5 z3&f5YfuhdS@{JWb^*y#>HR*9 zm3mI4mhp!F%&UW(Do+-&>hPYjLI7DUGDixCAV{KD;*Pt*+NNVHE9nh!3ZJW0q~1syS3)%&0OEoJT(g_@oGF00!cPYmxFi?jNS_i1V4ZIjWyl&cIGmWIE@jK#^52GsJ zJ+}P@_VR5TII8Z&OTc?Vg-*3iw;!)`ECWs;E`w-dR+c8|c{%Jfe$ZVYsn0+}oS*J5 z%=F7sLzgnHF^&9OXYr%7g`S5As zh0(+{`2PGelVB6KUQrf`EasUI;E*XGqyE7$SXezvN-sb>0^E#)D%5*f@!KCb$X}mX zkRs@37h^^GoS$^ux_`L4Qcuxy>{vMI+)5Fd>4u$&8V3t!wI%texf&3;{KPW}O;;%i zpg=5mQVhg2abor4g9uR`9k^}{#4KFN*s}ae4RCdhtFWo!d4&7|oVr$471nk(=ct@} zjNUOt`fgM?b@-z$GceaC;iY&)QsluOvg4_9pzOptW0wiVJcON&;C`Uv$-_^J>B}Ab zgybvu7V1IGH**RS6DVyxos|e>p=6oN9EtWsD4#J(Lh;axVTm=ezu!Kq>yh!>-7Y`b z$~g6wav1uUbeI!#(X6rRKtkZ_iLh9&z& zVBC|i=1_DL2m1lQ3|m*sLlrJE)j>H3saP<^_FD=~b5SLs?lXQMiqw2%74uJ7{G^t^ zlCJxu)oAR@o&(^6o9=c1xyAw8$}Ui$oGYO@dS(F?Tz7)jfQG?XCiphpc|Ef*UV8H; zl5stRrh|sJi#0Su_O=Nym9|~gO2zohUsffZV^JmeZFS9peAX^Q^wD?pgHC2iZ&R*2 zFXuM~(*;eh(O9>`-!VVPzbh0Gf_d^^b*D=e&pt0l=H~M#i57@LFXkGRClemgKya}F zukf?3R+f?;-K1Z)6d^psTsG|n73R86N1bO$%DRoM2oIj?XqJ>(f2m1q>iPvBa!xmf zrR=b0+vLpZ<(%@MZhP?dB=UZ^pw#2$uoQ`v@^Aimxa=pTU--9Hc!3UCPo}M`y>ofU z1S0Y*L0(=O=5&p<>}!5qV$tt!IpY|KvSZn@%{Hx3YO{&8@43%Jn4s4hN^I#4pEro)YF>SS{T$pB4WmChto9?l-F&s~k3jS1*r zu!bxf}OUL&HLQNukWcb`A+7+5t1CtSOU#*+{!UF!I|*pJ3-kdmJl7!So&D5 z6m7L5HjQr94%i8G;d->>nEuTe>8IjM8l7>B4h4qHiy0(pEJDTVVViUMe8E(_kk6To zPq>WB?>wc8m(?&4DO6_WO%cSd1)VPMUy;j~J%c22R{67#b*OJSm9G4+sBO71QN;|g z7~J+-*WhyRp651IJ-|5; z?8yy^HJfdMn2Vx>C^;e}+ooI4Rz2oB7dWW)fEThew!WK%HSgx)>&7qBo39SE{z@5l zFNW}nGI^spG+l`KY{c;e57hdkQtrUc%JF9wh5yYLao|ysOcw4 zI++}*CVXR(xeWz~>v~^RRc9*s0P>e3MR1%93L^1$L11P{Js zC)E~LqT>7A8eK@UlRWIVsRV~>ufaddADTPRt>EIuu_|+oWXCVBXB~rdw7a!*~vGLEN>osh&MdV_Wh;vziWi;*bG9byeczX%%$w#vJWRt(=zJ$!dMAa2l| z)eE6o$Yc@fo9VV4yJu*BmwrI3Nw+4B@ma?0tU^df{`XiGk+F8s4qoYOk1t}Pm3cGT zw-cm=buxbaYW+?`p8M*|*;g+}0&zq(r@Tc@Q)S~4Q5}f$kEvZ_%#&@lxc~Jh^)A#D z+LwP_n~v!?RTv1{o2_P8rlnAa`3Uz7HO9B6-;xdNNgPqT_xZ%)odrHu-{*!RH>*hm zxc*9!4@2c&T&5hryCOT`4Ph@&ahW?#8+F6fEEjDt_=jtMXM?4115jFPf-Re(pXsC{ z(xuOeuMWMf@hnOt>v<|-H*C%e5bZ9w&=$7m5Q*j#`QnYlP9csyStoJ`!G#kP*V}fU zeHB6BSD$;t@LoF-&or%Q9&RZloWgh&4+ivRRfA{ON2-u!Q~cC?;^XBIqh=h7ju(y$ zZE>)6wx5vrdr{LCx(xK2jTXm*^lQdS#%#u+R`21&1?B<-f__k88uUemd#)|`n%ZLQrm5YJ-cT)cROAovAi;lobbK|LANQc z${i98Nb_UwNfUZUGd7J6a{Av2_3tRatO&k#WXErE@GGU}s#Ug_zoDjzBzGixf<9BH zx_ZQZUzGEp>w`6k6W<#oc4=Xo7kPBaBJr;Hcl?!Sse6xtP}-G4Y=P9D5!6U*bybflcUOtk2o|3ndm;0mo)dnS_eX4kT3?=2IoCS68()au3B~<1|_r&enw4UsUp99wQI* zw&uYdzYP(Eaw9erQl?WTtzU@c<6+zHpD4k?M-{^<^O#VHuuIw`PiHTl2x+1TBd&$g zi=?p$u;G*~QORQ8tG$TjZUe=kOsvXW1N;(&Me4H=oOhFy(y?D8WPgJem4cWd)%im@mCEqb^ zfs|F0=1}w2l@Pq~X!zrAOX+)Ud}fssw$G`t&^*Mw?%0^i>I2RwquoZ1b-0Y1QQ#)Lc^JcB6lxy*%jN-LUp}lvI}t5@ zGr>^}N>+Yni4h;yD%ii+F;ZGkNy{VwD%7&2^~$0nU5sf`@uq*O$2QM_pa-L_EIbB} zi^=GPom&3KV&=zNMDRZG@IV^VW7T=NST}MTI7BXz%P~rFM)EjC(vpliKwNZ=Y}9_w6V{2P(8cYf104uo z>`&-XYpDP9H6+%Pg`8=28b=^BV$|#NZRxkv9>l`43$(3~j>ahBHU`TpaPq6W4sP+G zerXv^JFAo9dzxhljBF#ochVSN%`Ut9y_FdM9S{iv1&*SGI?}O%Jep(k%$xz~xdO_1EDKuckkk$~Pl-q`v{n}_uU)2IcTu~+;z5eoL@jQ>~W2y~?rN`Ck;^jt^O?!2p za>(TRsKe_w*{D$RvpMzQyTo#1q%E)Uzm$uT>-7<^c?zwz(20@@9Sy!`@%^PR^)rmM z0gqv~I6}fY$0=*v9q!jWit{*)n0 ziq_~etA3vz{FhGJMiX4Y+zpa?a%VrwgShG|6*savYPwS{AZ$CAzEam@_*fWe3tif# z?z3~D3u$2OAYDp!iNqhBPo?cKm;PF~L8em};duLb*p>p*5f|&*INUp|k1rF(N(gp* zK+rB1Z3k4UT#bKxPxZgn7K1x5J%o)^Lqa6cI$FIE{e;hxq&Z&`lR?K?c7ouv(0!5W z7mLC;J5%9VdCl6Yrt^t|kYcUiUUn;#cvL5<*m9R;20A%cpR=*D;+$%oLc2dsZ*Qa0 zP$nE-4u$zj=*NbXN{#oG3Tp*%@&_hIKFIy>ij#aQwGl7rcfkoz)5{U!gk3>cQ~1DWZ%++0RjW;D~OLD0edcc#R#Z7Z8xk3S&rCDxyq_l3(P-Fi%CiK>M+Lb+#|4am|6frsdU*s|C#hdnU zOcdn)!#|#uZIUkMn;8kKf9Q#AK)$^za?ZMqOWlJM7$~p@PYYD*j!tb|w~+ zgJ!!RPo)1aqJu^m{||0dKAk)O6A}d|+g_WV6pN(m4@!Z5`aMG{K>OC946gKN?<*rv z@%U{!+1mI6qOIVFCH!=n@P(>kkZ~7($Y0Cu0*8f zs2F6$yMyW~3zZnU3P60SBHtYb0vcS*2?OUbD$EDAaXypAKkjTQXChx@DomXzwJCwT~kkCjlY+5D>U`Ry;qB0D=Q?CJhGw zGFlgm?fx-2tL#K!&C>Nlz_@-B_130H>y9~NSI>B*Gpq2K_PH|g>6;bsBi38oZ~6&X z;$mQWUDfRYLoNH_XY9l*@3}5)=}_?*R7SNP@b_ap!9B&4f75isWpN6>Q3dLSX}C0Rga;Tp zfdWCG7H5?M^+I!k=(%JQFr+PakP?zjwA>;fzgQ)#!a?E_V|sfxgBE(zFoQV(41d*kY zqBSEY{_1JGGbz4AGQ=uI8>j=v$GzfH7)wTl1gqQk!4;kfFCWCCFZkgUVQyYM&gFe%6 zKtWvTnE-NDd3UB)XhXSJT;>~%Kk3NjG%w+frwF=(GSX!pxIp8P>y@_*`!|t1G#!ed z;5n@uc%fJGZseO2gyDzHobhqeWQD|zwR8LR{T9YccUD;X&BucWzjFa-Yc z`T9n|WvmBH1!_Ej17W;sh=4jK3XtyTbnjV`Wl&ZXu-;4rs_`?>5{qe8XA%n|YqE)F zDyT#5j}TZRB8Qp-86dC=4#mSKqTDV{qw)qtp)w7L*nERB>2%qE1FS(*x{ROi%e98X zLO%=6uq{H1<@=!UeL^m>-w0W64V8vcAvk84@SH%Q^)OJD=qFhNv6-DC0Mf~2+y&-( z_bOHDKtLKc;I%MZASckZGH+UH2bG+$^a){gSDchAmudI!^gfa6lS(Ao6n)qQ0i^M= z{{_3QX*L#uIh$R~sMCDYOJRPXcmQ%Y`g_)Q(SJqtkmD5j$~2c?E<#+zvlB)CO24}n{YjkcSbr@JhFmhIU1-ph>v2K@`#|MkNKf&xh7gGdz|YDnMz|& z{@kR9a$(nSxie{^u3d%RpuveL z@f1YRaUdT#iU07W(D2h+h3}VVZo1iL_0lMV`(TYxFmsx;EqCCYSTMw7`7LL9cEz(z zOXPWSN5h0s=>0jyT(s*D;CLKB@eQk(7M(upzh`&X2#-N4#uoQ-!)w$7T*}E!+JfAb zNl>pq%<(XL+ztB^7VhTm$;`nH#YI`!bW2DzV|<(u95tZyW76zV9(nVWnGYvxKGfO! zr5jyyiEB&9dRu|c%*kpU9gi;W$xtEm+Vf;2k~F?3;7sss?YvZkP?~kn-#3Y#78Hg% z_hz~>MP;vnudu-Cjg%(EAVWj-z+AzOWRBP~&^pyD!)7=w z;DFdywv2WWs8M^{g3?(&q^aibIMM`~5|A5@7l`OTQy*>KGL`8UYJ@keJ~s>bxonY| zOT}~H?H7T2ISZbMsifgu=Gq2Lu>sbFdB%9`d(J+6bN7;@4{nQnC!*!~my5@!buG=+ zf3%XeQL8b{5H+kdt_8|*wIG#?g+p0mFEDuw%x$u;a1nKk4lAcilnnsC(PM)_+dsmU zHl~WwSM5uq&lVkzcJ#JD23ukqnbI=k} z#Ii{?6mVB0MfvY<`5356A-<5>uSaL97(+FS<$Ioq&vFJd(CvaSx$Q9jM1;0YsX;Ln zNYjo$mEZJBYY(Q4dPfCne6&7-@9;PgErl{eDWKP&SFt2JbiL1h{JU}ylr6fp2kKea zmSgiq^2=PUZ>tHc?ChcD+FILD!q_F5TK8B zB5!@~Pp_N$R{E~v#!K0~ncCXeY>OS!f7v)-i!^8+TPEQcp#eLXzYT$-&5?~5^6`m* z5QFHW=O*CG55#sBbH#w@IwjnmN@lZGHfYzy8)avzWA)NI2}fNCmhC2 zIUs?d#Tz}4qV#)55cEtL{gdYS@@S)a<#UREJuCo&Tm+qpP#S0ptNejb6Rm+0^&!J8 zl`H3vt3>h{)HABOe*TM79x|K_C{mmH9AK31;{+zx#o>ZmSI;$K08BJWFEZ~d{{h?1 zP|hchyz^iEBmd&;+ncPf(LOnMdGp2qin=;YetdYok^eu2{V--m(7Xx959}de7dI^v z|9|Lu^LQx3x9>mekYPwMvW%rHk!C0cV_$}dq(s7uU0Jha-x)GlB1`tAD3YyGgt5%n zvX_0|hLA1G*q+Px_j~Twec!Lw^PfKkC4a7;O)T+XanGfTixv&<+Z zy9Q6`&2CAzfQ}vl1ZV&D*4KIxQcY1~0w4{Z0HkU8S#}l(iokv>`~{9+f1s^W>F5~0{Ax7diKszF4^j>q!{T=mssQoDP;Ev}`6~MAk7P{ho#kj9^+mE)N z)z*K$eEb4D?8>}><;3utgKTGn~=Cbx1>z>Cs83-&{;^9GEZWCLJ{(5-p%E#3jL z5&*Et=}c*iQ-D1r?(2I1aFs7mma|P6q!Bpq z_8@F|pMG5{t6je>dkR!VH!~Q@5{yMP7sO1$dQM6maWc>2|*}V@}0rH3=fmSWp(q5LcS|>kR zcEZzWT(oHFtzCdn6gk#^7H9+t{N?t)AamL7`52q6IS?GaNmgD|^tErA2iT74&rgSl z+s8mnygyIN&JJ)q_m2F0!nN-_UDb&nJKNd<*V8dtxb>+Y*2t%OKC-0-IGwZ)Jly?f z!WtJy*=fIzmmPNt$7cRzMq3It#KgJp0tMY}r^jMf`=Ch{^x^E1hQ&C^L~T&=QFWW& zRG$Ju`+E*oe1FD8U0FAdNxhGc)p0DhgvBrL&Yl5GI(3=rOF-wIe?ZH@Dx&c4tUC0# z`T0ngGL_V`a|j5i{O(dtvZI5Dj1+0-FrA0^+j4CC3K!Zqn)3h~f;V7q8M~VL*y=22 znKf|0GlvaRkH$QhgTjGAHz+tq*409{BWwO|dZ@z5{@hvwyMA=fWB4DHOkfl6l~2*H z@S0uP1Hkf7aLajg@S*&9PD!V4&;DMR0P2oRUhiQnz(sTJtH03?0${FTfH=Z-eE@>c zLv&0K_t{_o7bFp=ealq$<_|4s(PsMJTE#E{0dABzLLytn0muf z$sgExe+0JZTH^#}z?Jgoi_`K@4zN>c2mn6qv9s+2AdO*}oc(lqICUEExzN1UyL%HT z&a2OMT5?NA?QcddLxBWxBkOAd!ebJUKJA@SxIfiG7X=1^JG5pR`&txweWLdHSAc$k zI?h=5%+Q_LMzVldDIf@&XZ`qZb!31=apx{TDVqg)<5b^JU1}&V!oQcp-}aea`tzy5 z`^MMtXR$p^eNo!n1+P#rz(7F^7wbcin_h#Ic}xj@z?S(VKy_Kink{J^}3pSIcV9sNh

%Z0~yl ztYEwQ9mF#{Zlf9U6(HWZ=u!FjeUs2DYmMnW8>>Lrlw@F;F7NfBq3F&eaA3{{3KZCg zv%j)SuQVvPX@~D;%^|60Ev(Y8Vz&GV~giHqLF-)?f7w83@ zahss^9jak?*>wDFFMvb-Meigg3ZeZ>Or$xo09VY}u2|&Hr)u=DQL1WT!f#-0ULIoG z&U*=Or?&QK6qQd`tre84Y5HK({)#vZ+(IWq+eZK_pfD*3b&Z!>gf?{V>mDP@{sWA~ zsVh-u0BGL8n35w9PF|2q0T!-5z67`PaZBf|Awg={j>>uXL6P;-lfS2r2Q2_bq}@1} zTr|k-MNq^)B(9M7{JYXm{#c%#*ZBt9asx_2=$!r&t7Io#VZ!J--5d zlgKxK7wKQ!`=UU(Bi-7KCZ@n6+qyghBRk@w*`i~%o-?U{HH{UrtEK8iAP`{pQyO@a zTktX=73iY*a2LSOHsA3{C|owMGG)kFzxxa*g2r?nS>K{pjTVV^pUrpLc@A7b!)%mq z&FXO+($OB(Nd_TDxsGH1RL8?-2*^8!bLp)fpx~Vx4N_Xuu)c+Z5t7mb;0UXKXVmLz zTq-KwS~diE1Hhx7Vy-6Ca@l%sA~?r+9$`qKcqsFt>$20%h{n@~xxp)I`=Vuo+?`vnxHX&8b^416ll*fen zY6Gwt9um#SWLVXu{#GUAE`NyUlCEPHl2AT8l7gN2{17GCV{GByct z_>?D3o$-T@z^l<#^cE|Oi7~mRK$DT%mO+tTyNUlA+@+~mG8yN0#dSl_lEI6|3l}nr zy$@*ZV5Pp~?ea5?Pu)ZW(#JO8DsLLLe6u_^YxAxu`4uGXE^Z@&*DEGhqx6-`D#O@u z<+_EpU2Syg7xiPSi(Auk0{3dYogoHhIH6T-ap;%FA}+H9aiSu`6(A5=cf83OBTqR; z1TDOq5MX6%sLsi7scVPbFK3CvF{vfWcmbi0{qU%*vkqLdPXo_;D z>+AAS30CiI3U`i)_M^QEjV}gbs}b#B9$&k0E|gC>Db!HUqTgeXJ8Vh$^dO!oVdqHe z?{$;%#@2JoiYF~|lEw}>(JI6%#$r(Qpv%WeUwMt|r&UMT)S*rAd{-fDF4^!98y>s5 zSwQ31y&a@Fimzzl1>YIH94EuTIg5>qm1(xuxO9*kpQ!Zj63-lBHUZW56{?PI!5;&C z(ow*gu+l?CVbua&GSUA9&Weae@H@maNLds}R@;vNd8`gSf=;n`xt1h@X>17t^M%)sp0$*^4Ev4F*?_}@PTalAeH%dCxu|gHb zAnk2qD#9z66ZPsb&Eihbr5U0kqXsPwQT7UZWh=bVKi- z5{;pn6VvhZ{-;N?v3V9Q6uCGA(t)Xl72=$_bdfp>k&QBx-FWM2kRvmZ9kk*I>DhiG zpZ9nBXu%g}IedWw&D)pKD=;6gBSn0KN3AdmrJZzD#rwTu&Xsd$NIw2%pR?`Xf=g`j z8od$7W=CL4aerwJs;1&uOF(4Dn98vYr_xroy5$ZaYA`z6)7WVnx={U0g}oe|t&sSj zcj~(_%x+zHgf^l)Sw_b=u6l631ziCQ~3WjEY47CyIqLNYe?x!1Xhs9gV7! z%llpLD$d^n%qE;eV9zrn7f2c6PrnWnZGBsdovsBW{4N2K*tE*pl#pJnfi_KV$GE9i zBii6_BMZJ1RKhoz`8feevJb|{n{T=cV|{Zo9d|2?$++)(^lxT}bPFGi(B=jviY|(g znoTt(v~3j^&4WyThZuVkoZgtC#uH>Jr&U*}(&itQVJxZnQg zW=BMwwaEpT_|5$y7B<>waXx6Z&eol*yST1?Mc|yl!UeCR)SO9CvAKI1owv~G?VR-< z(2Z{A2MYXVW1CN^6f6T&C1xd$-hNO{5Mv6vSYNHU5(WbK=B{m=i<75-A>T9nCs&=r zfw1otXyi|b^Lms7K)Ve>ns5&i_(P`e_|3yM+3hx;i7f0&y>g`mmmr`HF;%9e)psqU zZyyO-u-!w|uua||j*7;CpizPlN4~VKj$g%w&7F}*L02fH^~K*OxB!bPzn6Ok#(!A1 zp$FyR(g$zX$QxS#wXNS>cGovUllx?6NP2#QeUs#pQY1#(>Tv>} zZC-=afd1$pn=`Aio*P07{(A<8$lSv#_n;2-jxNv9ALA+__kA=YnzIo(Ph+I}lX?bky zAXahMt?PnNTK<^rkycZ%r>DGNKD4iKx)~E!j-KD>(4(z#^c0m>rnwr26A1SiM5IYY zONmNxs|M_&ue_qqIneb)-E7KL7a{dkuq}W+Bi`as7xUvl{3Rf8NVE*C`0|CQ5vwL? zBprHu-&{Z9ib17q{3z0H_yFAp$dJ zKtlM=6A;sztYDtze`NdF#k`P1px8`uUlfR9z_PZe2!4vt+(|-zF|W7~hRKzvj&z_z zBj{b}SE86N6Sq}S;2q>jf_LZ)nrBct+;cEdn8wWw70|4A30vF)SLa+wz)?Xitw={R z^k1EF149HOWaIwiDc-ecGAycQUP-S^;L!7F+sdZ2Y&JHMo7$|6U;bg(p4N^>E&lvk z^20rACpyEsEAZq3uVcin>i!0XWleT}Nm?)yha$@U3&p71#eLT=foz^r_q1vheCIp8 z&2MA(dT08act3-0ON|D`5XX?ptMD8N8I@rPj`8ByV%yUD(R_}iNNYWqXgN&FnH3Yx z2npnIE&`%Rgp8$}@)&#=DD7Two?@a*&~k~RoUwy8hz%zcCwo}$=-lm)WSnqu2zz0T zyU^fJE&8>j(I|Z&vh5H;FR69m1RLL6j@hkwP?}|Eyb_6;X1!yG4#r)YDxvQ4e8|Mq- zaqmrauY>&K9fdfv%ewb#^oT42NnkM-T0>GarUPvp{c3wbvN zV4G+Q26ivK#)^OR&9O_4(>mYtEaBaE_Za+|j83s#@kGs2(l90^>p}X)(TkMSrJL*? zg}MsiSC2v2^Eo}qN#8>+NmffocsqQWr%}IA0c^iiNw>M@WWD$v?*!J+KcCI7tBxTf zRbg}Tj;u2BCtAmeuA^){{MBH;h|QObf;=+?2UjGsJIh&o1(k?_6iA7d9!sw!`@<2n zdPk66&DN3D!(6dxNfGM9H)nk!&x`UlKIL!|ggCqWU7vH-gl*2UfzXmxdNe)M?xuT} z%QI&Ur)6$cVi$xu-5J~!g1ecP#k3^KK;mzckdc0X+D{Y9EtpFS6zFz9?{U9B`szU{ zqr#aA%v!&{Ks4cS4?}a1D-L>aaej_pA1clNX))c~d z9P6SI5N!b9FgP|!xZFl2@_UnkN%sP_yZ`1VaA}vr7O6{QoK=lbVT%L7Xs(3&iQf36 zT9^vxR3f#8C!5u8G4!%RJ_$J0a2)H6<+E^w6nM4$Yw=^8DR951#8(z=yOQRud*YLJ z_iL#Otq>JkdYW>Jd_JyHaKooDDV6_PG}B~ZDBNBq1w>cZK4U#Y|2%<=IYRg<;THk*)WfFe;Tsby%Q5Jd5Ofop3e8WQ1&aZ)jONn%({m{gC^@K?Ga|LNicX&|a2Y7g)B$+c> z%mT}Wf59qJ)v;ltg+jI_Ygb2YF&!8ho+Y>-}vl{j^QLFh-J)=#{;0x4WUOwan^4jG@?l1BE zDJIvWL1D_Xv+26PW3`EJ?xg%(R{2)+oG+ToXICm+ zZIVmbhoiSa6x0LRYOab!^*wigOmb_*@berL2lwWPf856&)h{dpfu$m597=HAcYV&G zYS7nhV6=(xhHyq(a9p)cDu|DU^$^?69nQC4mm5MuE#dw-3bj65|CIiXvj&uRJ_$Lp6_$YY z=3ehwCA`^!dF>d=OixfLS?l~LPfZ6=(TLE9LO^z(kRGgdnQb1$26i90P`>=|bHY~_ zc0ppg9i<=Oot@L#ymw`GHH|o1?i?WQ<+D(cJ}}TA1rr0{jrZab7J-xWMcry}HL4I= zJ~DaHYtXz{w+#I!As|6!iCc|#E99MG1$C2pEY)T78^iDfq`HOA^ad-DbrH`v#^BJ< zwP=X$A`9&L`t)%Wc-@)cuXcb9ia9Ma5eT@CH>0odb+6u8Qq>>J{rO6yBY{j7Vo;jh zmD$9>I@tOlpQ3qo|Gf$x)wY5O8aQ2Tg4)R7Wx_=nmj#u`ITI)FyWg_g&i{s1Ku0p4 zVt*%&pnVb{>mbgpR{OGMxyvHQ^nBG}yLQCn*P=vD9xsit5Slq=FUPSR^x#6 zbDwt%>~Rg}=xA9%yy#5@mHO+7_;k}1b|pYJ(su(3c@8#0_R59{1=8}e z!=Y3kt)i|RUkax|gyrU4H*!gP7|V5G#}Uj176UVPoF6Dw7Zu#vz+uyeB}KW3H*E=! zgZw}?7&{|gldvE2+dY{xGoM;~ydaBTINe>OCpq5x+>I~0y#`oS-m0~T`kplPjJyCs zqU78;GqV{k=bZVR9dZGb7@$MFTu#jg&SQlvR|7fO24|5VZJ+mpBP*Q*(c8$yrxX56hDHYGyE;^cFu%0H$A z+*=xC_N4&BD=NN>_Z=HHz*`BjqfuBUgQosHE`LnRFg+6ybn|=?&-~Kqm;%$ek1xvN zb8MH_U(qvdKVUOkZF13E4fy!*c(rkzrM#lHqL#@FA*K_){;2*iV?m{P1>&f$mGbXXISAGIl z>Z&Z|8dbdbDZY`>_Fi1Vkm=razQ`L+UkxOTj5EwyfCR z7Ji#1!Hr574PDa3O{hj&lAjSr_zhA>LQ)Jr%Q#8JgMRR3Dux?y*cls%#lJ>>F&0a$ z(IbO{24m!#lBWjyzv-S-z`!)dMws~<4fkAY23X1kry1bLB#}(6G)MGopZ7QpVM};T z#bF0+n&&n+X^U2xrv2DJpa@vv8MneSxb3SI*ag{nZHZd4Qmtz(&u1^h?+nInaji;! z76@0a6qJ>Z$csJ^6xlLT#@`f1WJ|xFLES@tQt-MW>5Cc8v&{VXj#-hP-LVht8S>yW zl3(b!tl!+TqB`a|uGV%;mGgt(*$=*B=jLms^P> zQ5qR0uug(EC^ww(s`wyE%?lnk)hS2Va2}d=7E^7=Lb_MX6xeQx;6uptrjVtMv!!Ns z{yntBSw^V%Z|fWIkU4j#Y?`2j+1+VRq3~`gK0^6}PUGMCwy_(3Dr=pGe@21%E=*ok z7ZK}GAIBvyDymz|llnZ4H&4)JeK*>Le#6q5A`xKoyme$R^wd4%pftUz)M!Z`8xTi& zGMXUQF_5HejsN;E6XI3qoDlwO`A;>B$AObtDk)`GvuD`=sXX6lBJbX$@n`tGH8gg* zBY=<=J(*{=Baqf_*{8ade}cDH`fbGGaiP?tSA}B-znd>fyT#v&E>`Y|Mh9)0{0yYr z+7_3`^CatX_bvq;FFY(U`{dG#UW6}75zS1DgpZR9c2alMd!W*NbdzQny*z!l&`aH- z<7sonsUTz?sZIT`koV%ip+gVwzVeb?i@$4P+>kr~!N9`+i% z$V0car-f$X{daCqn2zI5zy_<-Y4D+{ zu?~bXQQ3*BW5e4+l5#Qht!D0H9^Bijbsd1@%0!+vt<6-?rgFBXtMY;ye%Kj{!#%f6 zyE{Y(@Bs@+!aP&j0^2f#kovp1-AEBK*9#dibK{e`857bRzISl%s}QDa`8W3M{o#3d8gP3VqL>-q$%PzODb| zS?Aixmy)5c592#d!|fJ~qow|*k?ay)1^71CQ~8k(!usl(k#gJn>ciVf4UH!IJ%bOE zQP=*bf8-hs)k%t>dtTS|fVgY_dxDFEC2hyJyGi{m+a4?l}g#D#nQuOtN~k#-&_aUoIgfAQ{T zm(}cX;mlv z@qqNNcsbmkm_*gb{@CiYdFsCj0em3=ogd!>qn1N^lA!J1-cC+r41+{IJt3j~6)8%& z)x}9(kAGAw|3BW;HtjVV#+et*&vOlX5O&x3{`hME2rFs?*KC{ki?S zGU)Ss^u0^|8&#WMh_97L9L;3DIzKG2e5bO#wIv}dD?112NiiE>pq&zon;h{i2dHVGy1NxXN8EfF8-xUNt1G<96O>iE_3jnlKjze&%9&2>m- zZ1t%T+pnH!f4y~Zkk(n{>Gs2P{NSdM@3NFs;NCN7MWG%O`xmhpV#k$kH`xDFCYPW* z8Pho*LOLp4VmnGuJ6QLB$c)m~&jyb|2jIx0`|8{D@-FDx>KK6p9;DXQGfhZ z6#Nx@gJ<%3h^DV?QmGCrwYDUigJqpBXGYk0eTd}6jX#l-ahl~(__NUG-Kzi5X%e-2 ztwZ>JiSf^F0EYZCQdP88GZ*(|Kwlv;2N+p1;q&Dt^~?P;XdPfE8%&s^@c@ib@wDt4 zk+QpHW+xbY(3QLC75d{0R~l{M!=4e)b25KU2;K z*a%wxwzLYsuafR>oP1v|9iN%9UA`~BlghZy1pz?zu-q!!t9t;;;R2A`rfgim?cJ@= z11P?^JyZUkr4@c`61npnMne$OL~d7!`~gXB`Pmu%(9$z5mm8RS=Ab~C z2i~VEz*Ibf;(X(-HwUPnw!9;_=q(vN9zmI{#L*})J+1!aJXZR+=VWhOyW|y;$wmK8 zx^xeVzfFzv)ZgdaNz6wV^q6vPkYsm-;O5ly8n|`0kERliQv;?Dt^`;DUjrVs3re*TH(bFTU+^BVwu8sl3~ zbM_sBVtErW(w^kr>0x;Lvn(;5p#2$Rz$Jd~GtJh&;M|5E07($(*=W1*td1{zWnoX7 z9n|pGan3#{>$*Ng$v2p?T;$!S$w|iouwv)B+$s#+pA=Mflqqj;!1y3~_3nlu<-XHfzfGED07Df#^qYkHLE)y< z++jF9b0rYd_==hByhs|LNGmDs|3DhV=hQDb9!cx}*{gi}PfGEQ`z9)`XtPdwVkvtH zhIGTgeP;Jh@CMXgvmz|ZNYtDKMQI9g(+xb}TW&IdxJ6_O24=kM0{>jZ*4O)&e>6s0 zFO7wF8ce!J+g>#X{->P)$K{Ra*&bglOF+K(#Kg_I{x3iYr71HB7D=M6+maTDdvjOD z_w1LS#;2|+^(X>46o1-==YYrW;v3LWI%ykqq|j3}v3-~=n}cLNBlp{-=ndGH3|asY zN_6UO9H#X|OnP4H(J9_aj%&1;_ zFk5+{XYwMzo7L0wL?J8sSM1rm9$@MM7O-&nnh;tO2))*?CIOb??3QH`C-VFyH2ahD za87`nERO(u8rA?#U;nGrRNC`4xWHxN`v{Quc&v>z`HB4(u9FLZU1tD&1mmUf^K<_` zl$4#nJ@YGnbMxAenX4k;-%9%ds903<$mM1(-)!$*UTI3Xcu@J|5Qt9Exqb%H4td%0 zu@7HV`ttwy`@IfGNnovl0p29cFm@IB`1GJBxJhFRI1U*D%w@c3jTcNC_Gg*Srf{i& z^7)>A{A_IU+Da2#4@5d1Vx9vdt%esfT=|SeH}ez2Lj!;Wk~c8Of!>-|ZiKGs-1E5z zLc`OcGH(i3k=dd58k3lkc)?)SYyGqVdH`=iY{e^xcr7Sw6{>fI*_S&2qavf}bys48 z2FuLHfV>x^xfq=~e1IJ~_m12OsF%1#3;p-j3H$liq1s{|Szf=zO!^1{xk0O78>Z3= z;{bGp6(kB+KZw@h`T}N6Zht}(t_J+f<`vXG%)hU|M?**$$KrTb&!D8H7qknkFu6Y0 z(RBcYXK?OXrw>2RE}?J>`mef~3dF-IOl5q%*Kzn&oKhbG!IR1T7dHKdNcvFpEGXq}MZco<-5xVuOS@ z*pBda~Eh+l^epAq;UfjZcVr4 z1JA|>RRHL%LTP3*c5lM_YtZxEDuAMk{B`R5bwUQ7kD{<74FC^Kn2q;{PgFs3#7|^+ zeP3CN;!>uETCm_cAarwqcHxK)$MvW64NF7R7G$E4h`CnX=xQ|L3c~8dN0qrmjH@93CS$A8sOWonTd1{X$3?%lx7`~ESuy`AHWrrUf(ht zY0-y!Xzul^cO9Rzx?ky`18a;?S0c)C)U|O2;nP=-8biNDS5V{d&qQfRcsa1ekoj}- z1g1bc+2~zc9##FtTO2(AZOH|G0v&(Wc^&)wqGEw-UwmQ@O~BrWeNJu_-W_1U0^EdU zhcV%5-={azzxIcE-}w}9@8N#GY}?RMve%HgESYBRhF9(>V4sF!+IfxuDIreoyhzGs zR3wZjuwuadO5`9n2Vay&@p6B@2tZ{&OM|7Wb6mn43>U*Ifd~!TS>d2DN2DtE@0Bo zKrdT&iQsd{g4W9N9W~9FDn0<6t8Qk^lrP@(*-gteG10pqI3Y?d`KX-NBw3TQ?Qenw zZg^xVlegxT$nhJc7|C+(PM+3)9#mLaW7q_e$UzjFSNp^jzl?5+J2CemL}CLl_*~|6 zSqF$`NJwcc`-?|QlwtQfjB9-+TG~#RHmF}$kmDYsc`<)bEx?% zmJJk-@6MC&^`8|#n+XiAugM9q-fA`Z2a{*hV$VvAsR^!()Nhv3{*emp@w}(B4Ar|w zD@-GNBRM)IzMY2V&0hn>gXB9|O4my>6XH;g@IzsxuFdw?D1(;C2DVZ%g%=}q<#cbo7xlXWH%aG|04@l>}lPvcd2 zg`&26SYS$As|n$%o7uI08QJe^S5L#}C0Ed(x7L-Tafy+JDJ)Qj>PzfjPB#=6Ro?(` zK4>ZO=_p?Rjv|ms(O9lCK%Hp7^&F(}e4`t{SfcacC@n=AId(u4d8}63#QpQ)`N{FN67n!Ryld(%&XW{&wdw@fdj? z?As8q@e_LtkJE3OUKeZ}r} z?C}z#bOlMkJ#6%FSV&=ER=Y9f`|LRU{RQ~VuRUMjYF~MkrZVQ)l2Rt7d%xWv;d=Vh z=x_usQSBT2U7#a7k%>buP)vI%Xr55zN`owbN<2{ zm|k(tnobcbsS-qFBR(ER827mVC_bi*GhJq--VhqhrOx#g&TlFqe^N6S?$jM_u=7$V z)iU5jGvQxKhAX3&`a^S0l+uC~l+Z|eOngGg+oPL(#>_OIpS{EFgDKC$;nB3jTP9YR z3I@$A?LU;4b1@eEpkAt}2@w_wowUh2nx17Qyt^YWmItG+A?xyE;WwPcy7!)JoDO0N zPQvO9?4^@Tx@MCa(|#h+3Inq6BeG?3a|fJ7@_YE|Fo;ZdAT8U+LKgW)LPjahF-Xlc z=8EhvE#~fB@TW;n6A_K~w!^E+j7CW%(%FY7y=3e536-ok*Q7{fqsBv-dfA`_Njs*Q zCR^CY_I3I3i7A!6=AKdCuCvc2Y=#B<=>M<`T>I5XWv(egPQo`xoXZ8ZRRcl*x}uZG z!*yo)u*bvPE(Qp`fnWz^n}DMjQg}eQT@cZsbBQoCCDA7NJw)XVGb z$1pfV<53$=>f#*aoE+rB<@h0bjD$S|Mb0P09whjtRoGfg)=>vuwH%E?Ser=2OKHK0 zxl_ZuT?9O7zCG<_cJE}{6ksLiR&uKOnXV|Z^NIPpGi!^DM3EA5>m|1=i+oE>v>slO z?BLim`;Ggpm;kSAy20ha4_0&UdUTUNY^X238Sw~g(owA9$v_5{-TIIfoe6vE7SVb{ zkHTN0_or71^SAcYKp~Cuzf55dH}I-2qL#SWxsy0U=~dj9ZNRH~?|i%?qF!De#XBBE z$@pN_oiV@c5Dqo=(X5gN~GP%lc&F0{3exGYc2PA^Uc9xf%%IZEZ` zcZo8K^NS+IwV>NjFTOVRxiFnLswkDEC0u1<0WeT@Mu|p}qypD_57I&sSxQ3PFm44B zWe6#;ctR8*`B^cHH>AL-W$b%LZ1Vqg0RS1-=6M|@Phk_paGwlP7(58{9eknMi8RGsj+C3@I0m(4)vk=T%;Ynq|KO2+k?ijc0O@_U4kDEDVw2s9`>C4jt9 z8s_#g2<|R~UXsjG|MA}Pv_&kE%AMgP{k;>+C&oajR50Pj8oE~cE`m$i5zABFK@3qj zG%{G3uVp6RpsC{4AoDrn5G+hV(eXivoB^hq&G4T0uh@4kDi)zeFMwWOx+V;spS!H6 z3Q0^qAux{#1ZM4NzY>0ku_Ko_D{2#~NW7rG1Wn>O^=s#O(n+I=V zq|Gq-`Hzk7&*(lLP5tEqhT~Bblzg1U-C!OSVb6W6PXeFIWIS*{Vq$0o-f)i$QVA{$ zo)dk=LBm0xtcabBXv~ON%5ZI96l^|^6Z(xVi-Cw_((R;D`uU)2w8%J^tP+$5%9wn| zY=_(@G?<8zT8MF1YX}vvg)d8C38CJFrwApf?7Oa+KoX}c5eF;d6sbW~3mxs-j5k76 zz9z;VLYXB-a|_^h;5qgKxCR2w7c@W9D>Cv+RoIh_CQZdc?+9qX==6WVftGN{`mp`B zf_PdjiE36TQ+oNbYl~1Fpb=IDk=e^hPyrVCXTw+=9X(mJB-v%W;1th^G^^%X(+k*A zLAP;_q>aDlCUKLSp==TieKJt)YTVw{v~M0Uneg!DBOCXZuYWVwx`xd0cdS&s{QyEN+Wo8>`WoyUSAPufh^UA9M@em5O(d0N5(JtX-dg z{DPf#;XGl3-rOItKxpfI6-JoP^G`ZEMBi9_fG;$@au4>y|1`VAsV zlbzUIlesU?z?X_I=$h`9R>mbeXhB^OMqyUTD+Rwd7+zyLvwex_?4wucelC`q6 z!Ww#PCU=AP&}OH&51WZ0aUJwN1V5#oQ}iA9T9=hbLzCr=Sr=(M1Q>O2ZBbh@RDA+& zBL*|x_=i%pueGo>8aS2S}W zl_gCgb8kDygi7VfyBthP4Fl%BA(YI<;ksbGf> z=kwTzFZ@;jJT3h(7po3uryi=G)FiT;TjBosusYzG&i)P%mdwFq>1=-73@o`b?MomZ{Ui$qMS>}X4fq*ftId) zKI!dBho@U^Bhl&yCjpE2^efEz(SML$rKZ-{Kpz$}>H{CSYc#FGs0ZQq#Ng@V1JApGMZHk@*Q=g!X zcM!I^$JvI;@$7t(`yk#teN@4EThnvnNz+TM8+oTzcl(wC6UDomKk2}5E5Ei#eCvrK zzC!F;f@?p|UuRL;OANPfW0Q{C-VQk8+w4E?8S9(#x!kdZNHqXEEynBVY161@@=?1# z8G6s_`RdoboH>-U-@5Aaz`8vBber>B=u@Apz6w2gHPc@oBs1p{31cGzuXq>aNTP?0 zgI@geuZNimt$MIUvG;`CF z%i`yJJfWjEX~id;1`ejpFHd_q2rEM7W1g?v4!clwyMw<&Mz-WuDQ$uaIpN5h?xB{F z>C2Wmo7QM&ogkC9n1u{`+K0k7k_^CsvzUNBF-3!H#5s6z`S+;39G2yy{qEgs;zvm4 zmqrR5lM=GO6KGA3Gq1AnmFRBF9Np?!4#>RT$_6}2 zihd0L7Ghbb5KLteLgT?SkdoMwW|ev8S8+XiKu9#=mtEO=As$nzTqFO(dm?zKw@m^~ zna^%1xo;$K8JI1=^lTGlG_6$fbMU&6_6t#la+}Y*1&rY-ubse_!OOq@G_!%|*}(Va zHx6x*T|o8HgsN9jahSAWfxp)>-YffUMMxYefxq%}r^ zXgni-@k||v7WalR&B6^6Qa!@tR!x582Y^|2czqpdC^QRj#&RpNR&(PG5WO&J?#(Yx z9!&~o5{oY(+??9j+P!7u9&sd+8yJk(seeKIK~PuQKjd|Pih@^5MS=ccXa#Lnu3o(n z#PX#Uhqtb1m2>SE>}D8}b`)qXap2A7m#jIpIJTO@-LJ0SD6JOYB(gXTN99KCN1bM>6?(n;MTbatuFOr zd5R~oX^z0{y@uoC5+EgB)&3-#A+Y(|lP;=*D80Gdp;8NWlgQw!BVPP>i<&Qp@XXN> zOV^vC9=O^JI~1(bMLp0?Sb*(ZklB&$H-Zrn`?GaY-IWO~NG)zbX=V4uUQrzt-;rqc6}xB#%gZI<)Zn8Sm=Cf6Mw_ige5Ynw%F0dLfE;nrX6{wtx%{Ml(|nQ zIIm;A+&S@mVb8Xwq{aZckKkAoj%(|+Ow17y={m+M45n^gP2uFq5*vy=KOcIBFnb>D zjQp*6IwNAVfpVv297f_(1MKT;vNv~a>#g0XNSD6=MACV^;R>78ojR%4>e=f@&D<o-xS9IB@EB{Uaip3vC`I(if%r}s; zEF_`Us|?*<&LFeLws+;w_7l>kb7pi`U}?@-t6;a%KAIzPcskK~r5n`&RjC!Ak#7nU zTkEMI2?{9O?}{JHDhR_XkdhRQx$aI=!(s{cg-xo zghwY1KY!-^{lwwZX|m{}UjYmDLV4J&)I!E@i5@CR)lcs=dz=5L_i8v9+W2FpUMRJX z>u#)NaH8_J*YMqxT6S{!OA&b)Q8CX8xoVwT|!+62;4w3#O&;EZZ0p9uc-s9*DJlEJ>n*e$<3PspOC>N z@2lT}I=llH`HqtauMAB3<0fRt9GQM!9%Au?sAiS1UV5_frUQtAbRPYaU_rnTzD>W8 z=ZB$H9MmbozImU>P_I_jNM5j8B^Oh4Cnm11X#|6R<0Y+lEIQT4WPaT=+7P~2Si?SNf6 zFv@!y0Wys?z>E_d1X=b>T2%J>bc`*5R*L~%KhE--OCpC>c+lmGlyg9+5dRb`NbS4L zt3_50%zQR;UfGkwLX!QTiUl>=4QZvT0OmVF_v2icqClH7=Ruh|o;-@0mR53YH0m*t zu{FnHv4`#rB9fv0)nfE-_IN2*DJooWEm*pU0}~IL$HgWe?9(qCc=(UmgU(w2G0Q)X zV-f}G7hq{vnnGiJDX#ay*G!9 z%p-gKu5;ht`~Kbf{q^gg&f`3ek$S%G{x$eMCJn=I`K45YGE?L^U~uvcz4gr5nVH%480(uTbzJi@RlZy3zu zNp|d4{BDnp+mlj1-Rq=x8vZN`X}ZR0hV+%1NlcsTfrT0rUvG5z4!J)8kMMfru^`7& z*)|)@4zk(0K*EpVMFab>_=+-kjFq}omtX!S6zPxWclTSbPa#U|f2yvjAEO^C(bB)e;- zp54P;K-oOGcJfsIcs(rbS>fZJX^}VwtDB+(ffUD=HfXWO{0#-t)33Ie2NK=K72W#6(xS~tktKXH`Fvoq{Gqpx}a$h7;5+83j zUACHnJXlJ^d;2nqhe5Bq(xcN@)5kZ!cg&m(F@kkd9zRu}L7m%Qm5ugxbY;Fwqy@B{ za%|-;w9Z`CKm40;X!|I$xQ} zv~RSq+j%Xxk{gFZvmMXELP1*4EJ}My_k!z z0rg@!4Y3PcH{k}me^Euok~AGpk~`ef6n_w>%JSKQ&+S7Bre@!_(>RB&x*OSxW-=L5 z?Oo>VsaJSePi?Fw@E#~k?VynxiQT!d0r_2Qr^l*{* z(2{v)^l_0aPp?9KeBl@GeGk5Nm|G`Y`-r~VZ=Yl#^GiQt`_EHESt{9L3uonCdFYn` z^j6sRHUDN5^03+?rum*^i;`jV7bF~q#FZ%3_Mjx;TrHJ8peXzo2|MwNH1}J62aLh5 zerhlUpVWm?AQuh3hgys|2JErhaH=LhJ$k=+QgjsY#781C57&&DXM7BK&#@O7ul>9S zInkH~pV^rgC?pt}?rzyf&zCDAd#*+B&pw*s6A>5Rf1AcAec@)`^Klz3y zhdvKasA`*A+l@BfPI-b^nl3rl1=Z^$>tpP2Hh8-BtsIH5Hj#ca`iPa8GBV#2eeODk zN9~SG&ZFM7I}N0ySB%=bJXd`mC8~sSo6SEGbICa0c2Rzo<1es%T!gwtDC#4jwar<# zSGjF5bkNmb{ak47hxC)Roc;6SV6tB?&i9*c9mDT(K#A7y7-bFSrMJFmqG__oX0qNo z**-(=eY1U$7`oh9{V|LIwoPj!oFey?7u}#h(J{?{ZYNl)}yK zDR~MNUcQtMEi!3Yl@&q6x4xw25Oc|!<`eTi4;y&Q-aTR8vxywZGQN7#l!tSKni1=grREQs^;l7j+l%EB$q^8OhtBua+VeQ(=Y3eRsy0 z`kJ+s&HOfa4~nwqOI5@Md9T1ovm@j7>;9ZwKC{RZR8YD1zWsp;7aOYa^5XDczU2MH`#$8(WztYfdu7k zWTmax>^f`9b?)^{Oc`9Cfhrs=tL&!u+w;`$x&$i|D=wefG{U~<<8P=3mz^E# zw4SmWriIq(^@O55xcA)Q&_YxoTLdZ%uD6k`V>S>qs@4_rH2I6+KsPLl`w8&BZCCzu zTd+j&pQL)DIqQD9i)=cIB6|c{ml$s#6lj-CtzJ+fySj%r;EXH4?6V;2-sai0!^Y59 zze5{=?B+-@^!j|_EnZOZslpg;X`<5}SSgX?<~7N)a?i5QfAUUMU@|Tb`xA*VmnSx2 z!`mrWN7&C*=DcYQVI2)Bw@YH!7T7qtJ?C*t-^HU=hjk9n@+bMMBk|w(1l`O9+(YiX z1@5=V1?`7l?>G-}7g$a0b%fa2^YDy?f7JgP;ihm>lfF_NhMIH~j8`fU{Heaju~{)2WO=}yQ(^y0gpI^(}%nCKg_ zAJvN?k+*(MN+8OQw$PLPhsaLIm1}o`d?rlS&f*5^Zj&kg68gx52%8_|UYCeiMT+y#?{sKM`ica zGC$N1eTutlUiv*`Fwn}p6!k>!V`S#TVyRj(K5LuQsl$$X=cVBDYbPPYTlB^6OqDd> z(d6k0A6>FDYLu(J4Y}HMUrT=i|8YQx*ok-`HtLPD3gupSYkpqLS2Ch@=#Z<-E zFdt~}!(k(eKkUYS;O&s~LjDWE$m#FMBT;G{bjtG=VA)b~z-FvQKB$><0r#fCk8QpI zWA(<;`*V=OENVXyvf<`Gd1N^)xUyF-c!s6$C~i3joX@A|PSEm%mE7qt^dzkzQ`)hu zzFEWI@Uy#uwsdKHP-pz;qtv&@egE|^$A(Fjx>dD4OcpS03zMPV%j$SiAh(m#%GS9T z8MZ}vgPg6vqBB{a| zFU;SE5C;6&Dl%#Rof>$p0}So$A@XGZMS6nQ$v#j5_6*ev=J%_<`CxoJ3hs(0L(9)3 zPbM~w77Xd{I_Ef9>Nb;ZPPcSV=43B+>cf>mBS0$~uh}T!vBMxFNyIZ~C{Mp@ykKx# z#Q8<(=;7`E_@b{Vr0Xdk*ediC4Sk_A%asss!au#&^?C8<=9Tfm;oAbaUp~C{^v=Gj=k(5ej#!$aN_Fbnk1|c>qaWLzje+VFbhRz@sZ32Vp1^8TrVtDEEcW!pTWjO(CPTI>G z7%MD|o}KTHWZ}}@Q!5O-iDm+PL5cYADIPAHO0pG`_jUo;{aZYpE^wdL&`dEjqx~2p7DE~xVRrK#-pCe z6*74)sMi3?ZRn)oAj#1wtwpl&GOOnSaI@-ZykqX^`+^}F1(7o7E{x4jy8x%*I*!%n&cLM<#P{n%D0e5_mn<^i47QyBHmoahX#lj9;yiBI29z*RXc+^3W9|Os zkbgpiZnCLyBbvc}`|Hd*{h-O0noS$nPi$kMN_9)4AK#W1>^I9=cjmhW2DIoTh`T%U zs+l0Mn#;i#1^KS~c8H}u;1k)v@Za>d4#5GV*6T?T65{*F7QDHA>POhu8bCw|dL8|{ z1YQkuqJnvC4U7+~6vQIBKnGHVvhP9rz1JI)en@VM#_$Df6bVaSOy4=QU{|1m@o3lUI!vlY#>l%48zS z^wlcY?knr}8h$G{$g&;Cgi)B!~~) zbEXca4doVe&0{s2I*;3~4_%F^(11+N!RrMe+QtBgw;QJU)w+C)4T7Ej(i>pk z%2#rdf?QN34zsL_8u5+!C4mR;O0o}N>Wu+oHjUO-@R=FrAa2T4M)x)2n)UJ|YhYb> z?&7KlAle5s-cm|cF}^HQxyYe*jIX_X)FjkTr05;S;!NP*Y-`{&q=`v##ING6J21fj z>rd6Svs+%|fpukaqDi#EyMdwSO#qao-%0)`%YZZi!?o2K5FQN(`KOnOnR|$JoTpuv zu96*&erx~ngCqh&ooo062(gU^$AER-v_Xz)wW8c6ScOJb`8mB4x7VO1f8! zzx(TX{>x_k-F)?CeEC{OZST`ve~BYc#|&^djoD46Cq4XQ^lu$eS-VZbk$A!LARDy| z{9SHLA}9%WR>X{#7Z?MpWl|Uw$ecDtJNWFwDQ`#eg?j{<)c+vwB3GXf6 zbjq@Q7dPa7#8_jz%n>Bw`Oc!f)ujQOU@nN>hS0Qi+I^FuEf|Z3gm52>e8f-emOonY z9Fy@0+83-rcS5E7|7<#Wto&_JJfY8~Dh$p!q$d+5IEhF8VSAw6~6I`$6>Ply? zpomTp#Ze_MIM7ER8d!D`jJ7I$reQa~wEOb@?mschX3T1%{ax2GWA-6(KBf9VsX}D2^L2?B<(y#SMRYlIuq`3CMh$5PWzLgb(04iwT%qcp)p> zm$~G5IeZ~xxM4WN1f3Z+e>uMh^_+=MEmB?Cln)l@2|62kS*E~pdS&RbE!R%gyaqhW z#QgJ6BVywO3`g(^J`=LuPQ=5B0%s-Jp_>T*dfOSe?>vXXeOncnJg=cWfdd*l04fE>EV&Gjqn(h;Vl)&ym)!`PwzahL}{i@m# ztB9A}J zUZ$&FQZZ>~Jq$)!f!ROrZ?!~*)o3(evE3G6h1W;Z&TkCI2?Cbk#B4P zB&*S;1u!SAz%u~pk=H~)#h6e04g&##J_EJj+Z;N85cocig)gblYQo-Wp6LNy1YH9x z?f8b&{G(My$t@YV5vA!@nU-2s8v8zc8#Di45F_V3Hw`W;)FBYi8sN*K0zLZhSCb*; ziHt?f(Us%bcZ%ur!!HYO`u$Lpt`0vZ3fvd#^IcaE1?wRr$e_27dXop}4OtN*k0QGE zJA!%M%4TbvMdK@N{PMT(#qoKOx+f2On09IT%^oC}@wE^s4`-QwSc`;_bLo~5?gK>Z zu3yRD#v7UHe)TS(*t=_TR4gA`-Wmtbld2(#iplUSUwsRpa)Q(HxTFkR_!xY zN*1U}1t>4!ybo3&xy+R3Vlhbf)&eZPOb=qN&gV~AtkY2swo`GjGJU29(8#{)T=?`E zAXlaF%}1>Q-9}PG7ZwXKu&7k$PqwnDBtX#DRf&1#XE}}4`q}C_JNYg`A-VV{>#fF2wc}kR;S5U5N2jP9#M>U>e z`U)geUtVXV;6$D)Ckcx4@rLg2yQ&64DM05^gWf58n5=ewMQSn zK6uB!v`Yn7o}b?|EYp@f+0L){S1|XyQOOf>g58q)k zghO^y@rZsW_g)`6wCqjy&FI;PX`rj>8(T{C=3X;#@y@Zn#X|a9xQ$iva^ML}0*2F$ zgfzM|y4;sUw-U8+L#3Qw5r*lkHHn>{?D(?xyx4SJA5V-R2s2*Gzw^Hbaw$Q~U3+UH z`V1l@T00OK4Sia@^|jbpv1G!IK_U7bQFNr$eeI;2q0sgg){gsK#h0}964h!PS17cE zK@rxGH(dvgVexx{t{XhkKzj15w>?-6Q2$NkWX52>DjM%4Miu;hSYfjRLh-8I<4Ks5 zJa*czg8Ed?-`fh#;ICE4HbJ0;-8=0c zEtg~pVTQ!7(L1x7bZQpPcy@1i>4asPYDn^-7MBO762Iq0z4MW6Cg%@x?7U8<6$jVK z0mWdh=yO6#7QRX4v$wzA!ezfbP}LXiZgl9pFCm$Or4S}LPvIgjMQyNUo=u;*(i93P zg{fX`qw6~A2A>O4%I=A}p4rG=iarWnYEdF^q%)0jHd-Q7pqRR$@kL!6=Co#&0jCKF)ON7E|=ec>3u7Z9JSV>Fv>MfRJ@@YXD&lf6=m3>gE6Kz9Pe=ww2!ZHDLI~P6HeY>0t%^e(>!Bf;#Qe_yL_uh+ zTTw*b0neVt-V_3^9lh6MUrl1)qZAlH#~yJ|Hj{L-t3xoNuJ!fwqx1rBg7w&a4^O$J zH70J`!*i-A0V<~yu+ufR{$M4FS~BtqkfvZv6ZXoPF{%S+r1s-?bwNiWaT5jHF`ckk z7y3mMVdeyUR+wW0Mh*3rSu(;VTm24ac8q6H!*t!*vCaP;D_jX)yS!yXpg`E+uTOBK zvAoGsPchx&H)9@es5L9*?lqXghnS*6AI;NekC%X#0QC8@DUq8( z+^8}=+wtPUspG|ZK}!)H<)5rUV4?J|`wsQSk$fIA4R$VjuSs*nrECM?hO#sm8@;pO zH?+)-_493F?N$nGnd9U^l9388L>f76csHC0&i9bdnLj7iCuZ`}WTw0u%`VR#bcKiH zm=C*zu_U2^ooK<<@0_v#vpE26CZ&&6YbgB*&@3$NG3?_b8H}NbeRjLQXppQ zwmdE-+%P4iB=ZM(NM;r0+V-`+rGpRxh~-+4QQ6jGHXheD{FcKEWwT7EHIL`Jx2C%KS89P;{pd`_R{@3*h9uziVMmLAp4l& zTRKzk-WsbKL1!lM5v|){zYDSNT&<3^=3;E zhWC^b!|>KL(xYo>8o9Bpbk>8p6qdF0&4+k{)R~}03-d3)DZT}Yp68=O(g?;5bv}ez z@hg0bqe}`QVnx7e5EUfRP(uszgzC(HBRHbILL(bQN>o!^MvwfkOg~8P#hc6UiGE!< zy>iM%cf zdh4H|c81$2OAyc6)1Sj-eS&8HO!2pJ6=#mUI1f`ZK2vFaw0UlQDoIac5b!evAHkkd zYOVB2P2l`#Aa|G1LuPt~H`?mv*@z)KJaTPaTyC648-6m{@-@@aRqx!Er z=}Q$0CPVdgQPcLqY;NI!@n!pDf4nffq7OiXEc#GhyUKN8Hrdw!R^wXHHO&jd)6xKw z(He|<`t8DO(y!s(DM0ahpPf9`cye)~Qn+yeQdKZg(6yP3>E0PGwNS-q1#5B7Y>Gj~ z+%*n%%ZDi9g@uJt5TIim&GzVypy8i-tF}G1X1rMF2jTC@!ULdLU*y%$=hz?bg}VxC z&O~bVr`^{9VEPuFYhwz4{9kheN|0k7=qJr^KLc9~gJSUOhJl>7`&gJh)! zdmOY~J`ZMe`}imsR{S6wvr9!nJ;R1TVnMsW)F0T5TkhD4B`98?t$_{je!gNf8;Bn; z!?d8TEoVjmaZrpNgwXs*5M~)P+shvd+PTW4E&|k+X7Vi1`PX8^6h8qWiwCEwt+hj(qKC(%C_X?s+~wp$|Q&;L-hHK4xsu1McZh1%=u8P*f! zps}*7D9^`q3ydOI0c2@#!E)}?Io||neYQ_>l#5eq_R44lo{s_o_QDB+CgK!R)sVmM1r?q$IQIGwPA6FtX0P!lr)BZcY?}c? z2-gQ?hG482f zOqvR~_tV3OY7gc^L2lcZSa@OQznA3nTB2pS05n_v3AmPACI?>K&zfJ9hTxTfT$Td8H zCnMh)NMrBSAS3;7K)b!XXvRE#2SQi?9UD$r1omoc$)G_veoJdtA>9nd*1GpELSLC7|v`)H!53VQK*4<{4 zZFM&Y90>i5|?#Z(K#_ePsR@3)ZX?_SyUoFtn&Z;-8T*;64$OF-Iw1^5b; zT%RZ(-*WiR(fa%s0OtVU5z@mM?^UPw1L{VKF%0bw_a~1b#fbEn@Rnhakum`knt=$V z-=~gbrM9JqNM;ZQxKFR+rZ?z6WjZxx_pgGXOZTqEuyE>(JF@PeoIUS_M z#+`y+H4OT%Crm=Qnm#XK?k|`;qHrcjiF|F_s>QgyWtIrtbAqGbpFVdaAdx2{m|7AzgqEP!oU}&N z!{X}+XuKvvt9QeZ3^ZY_KuVn`AA3$u&ub+0aUeV1;N?KhqboBY3X?)Bb?gRu3mn{M zKi=Of0MgSQQ0(dQM%GlUK(XP*Hzdcp1^93Qz&YF)o7k+i7MgN-5}D{W1uAF0DWU@4 zeqO>=KH1a_J0^0Zz_$kg+ zUulwhOd3tzhFul{tfgR(CD#eN6JXH0S2hB2nxM}8ZzjOSJ-vs7Uhr=6A~*mXc4MG@ z-!JM6ockcVRh-&!6B82zZj^K)?j1MdajOs#N6YD(3?hz%C5Mu0#CjmnS6X5NWIuI- zz~9fakY1)SPpP!Y+Y0-A49Qn!vNOer2J1oImh{X4f_RNO)qDET*GbPA3`NHCZSKE$ z+G>Yj2_kTU_&H)Q@D8}yr@L6DWx8KqHWJc`I$J#uv6+OKR$$8}4QUWF5SLh_xeo)W zcLYfF;;*J_tH1=l+kiTU-J`?Z)uB>wH-FYan?wN;SKfeJxXp=WaA=g1G@L zISkD^+YKljc4@yMxsbS+QxM!U!4e@x5!Ntb=>1LYs~s=zHQ`kMIZx#JYg}5m2CPZ? zh~BoA4-yK>BhR$q=O8M@&wmFza|Ag;(jhQlSpU?EYx@ENsK?20cNTvsQ^E`*=X?uZn56EQl$-ka z3dE!m6GSm@tfzDsnk z@{N&D2W2}__{lUly=RLi?M_LghtE%zBr^(OPM~g_-AI-4;WU!|d5mIR8A~^`x&dAU z$~kvnxkW}6nV<^*anr-(0Cg~TABkmgg#gR5IinHlK`^M=-iB$epyZd0pmqtfvPKn6 zIG&fs^0W#*vnY|u{qxYN_LfkifKW%rsdY5v-YYF_PozuJt(IkdjfMlWz<3|yMUJ%m zPE5cQ5P{c??wZ^+Wpzki`Is4TIWISVzO?#b!*+uds}ZDhJQORUF^n7m55W3yN7C1n z9yE^L&lfbUd?IL3e`vNy<4-J0A$vJ9!4wMlj*B)W@ZH&m(L;juP!A&6@x?g0PK;bI zLBQqXFY^L_flG=oWCUD>y?C4Wd@yQuk^Q5(!6*h&Yz1ZBJHmzj#^HRSrU8v%rk4aN zOKTd6DOSU!74*-Ke(}5lReK}3l+kYum)O0-G%Uig!g2h`BW_%#K~ywcQ@DzA%?1OA zi6kNj5E)>EOHq6AvKeEj7rkD*nGF5D<+Awk%@c3*6O?@!5*`Un-7WelEEY*a5PpmS zcsyw96j8Y(s)do+?bQelP+1gYv}3+-Qpm4qgSCgi2*?tYLCgdtghew>1(^2ey9c3X zFIiL$oPkO-asFE*q8H?;(az8HZYx~^Y2VOsN?IZqA!KIR4Ah>86#{&`W-HYRu+ei1 z>6r`u(?zD(4y#e5TV_vvB3E2RyvjFWeuG>Lq)G^yc{`NyW1)W^b$mPJ(q){Xypy~S zbpjV(kmsf3yTr2|gqVwP%ah^X!D@c9iHpcFf|1bOy#*=F-^jtqtB3MQvmug=wxSqO zeK>7*_IbrjWn)Wt=Xt-|>GBTqzT-P{XALUqk`@W$n|HaM5`6F;mTyIA!jfozn}=a? z6eZE~7THs^l==!lw7mTrySF*R92Occ?cL>OvaCYt@evKz#T784^#rhsvTz2CI zg2^UkmZ@_QNn3fclEo0s@@P%Ih_Zof1hU_Ki-9SqkW!3=QL^@cz5-`GtXS!EzSbnM z5+R#UuQyN!^)^Lb4#u;{0#ECWmJK)FkSsbH@Hr&&kag29;aD^~UCE_did!m%a=Rx> zu0&eqF+RhyAWF15vU`bYP@!87k$osRi?;z|nzy$i3zm$FKjvm1!F67>a~{A#QZWQ* ziaZi2towII%ju3wty`$1C`fFjWvYX>L|j48Jz0uQ6?}M7By|8kYE z{Z|laMcodGPh5YWDZR#g6+@LPde#soN304-MxCqmqK|C`-Mz{j6tSIJ2rX#l zKyDwQqMa9CQ|Sr;XP+&$6$ad5w|>4geTDQM)b5}*tq7&yaddg}e2bfO{eJq)9Ot~Y z%XDg*H#3>2`I^giG$~X8Rl(WVw0na=;UBL8*s<$lD^0`Y^~-BIw=mbSQ= z#B$#QxxDYA9?;jtchi|Nk_J`qC@Hpy{Pt>d6m=L6!qR9Bu0enbWgvl48%VWYZ%#)N zigF59`!;ro(yo!(i4pi;M!p$xBz|nRcXh{ThV!Nyxi zarhm)82p6&p;l-yxt2oH6}Ab$B;rh*I-lT3;|+ywjkffgr6 zss;__n&VGXqtZ|kwMkg<%H1S~Td!n=EjCFBoY)KLms;N9D7=PI`)3l8jqig8cj6Vk zy1pw1LNC-LU1#RT(^dIQ9f|RAJ44duBH?BT7E9P;m^w~9^aV1_&ZBbXhU*Pled!tA zx=(oo<8k|vqZ!u?z44hwEao3G@)tdmrc0(cA|<_b;z}fJ!}B|O&yUP9k_;J_@*ZZF z#(?iEXVj!FZFZl4=r;kPdpil)9?CTSW?!Fh8Oz*ND%TZ(qbxaFX%L)swjxD$#NhVc zIjbMOtN+jTlXJLG$LX`q`!{O`^>CcE?dSQpsI5v771q zDOqTm#VGwxMzfFO(Y&~!D2?|}OM>axt=Fm#qsuERVF{4YL2**ZA}{jOtw#-(Ep`?o zcxw$)HX z=}(&TTBh@ZO6R{@@$?fyjdoE|C0zzBHQ{l}FE%vs?D#B8<)yTsD*`fU(|)Yf(z>W< zkVq_}Wne1H|Aes}W!~CTWk{FnTc&BBcP%E~na-vAGOr2_mcIbK|J96`bG7MrbkXpr z1+!`8oi$SRkWQx z3;gJF?cV1h>)wu+D#=DW-jB|vAG~uuQ()5m+M;MV{vq#2eV(DwlX`WUF44u76OY^O z!RyopUy{%5oeh3aD;_;R+^;EAS2LI{thuJwBl*iw)?nQ33}=67CXIjWC%`;Zrk*$N zpI`-@{K`$^yk&aUN<*c$w4Qb@q>Amoi;f6!GB7vI8y4|aVE*zb*bj{^9~IL>SqarI zdn6g&G^{%mGKgnlhbJ%9Z|k1zZW%w94G#EqXj^aIA*gt^%Ow<40#}0t1yMGsvpZ`) zbkpE4OZWcQml#I`3M8GDY1;4pE|6R?oDEN-dM*C4$?DjAkQg-O6W7=UYqiSh&s!w_ z_*sd|;}(~FqqfCQQwkr32Qf{adfG0NzTwfeQkAr5=aHk1Mkt{9Uw41bOZ=LjG&$2} zKdf>_sNZF_WOn?6)5728IoErgZuwVilJrUUR5rf!P1_5Oac8j-Z9Num)D7QzS z-7hE)=VEtVfe|3{4eYZlY8EQ|h7_Iez5MYtdEx07-RKtbPXau*O<#9qc+#^o?X4kP>F$EN2sQ4eVZD+rZlz#u` zG#-Gz67|~$$(R(i;qGa@5 zB6{zBJJ0ido^y`#UB9*7cfEhU)>!uJa^L$d`*UBP>$bR^S43*Du-~u zJ^Clf(H#;|yjBU!mt>?%cVEw9l;4iwC-pVwX~ZaZiHT|-jz&N8ka|DarQh;sq)Wb~ zE%3c{4+cvC1q=Oza8qVE8^3rCtOZ1+RDhoj?|CPJFLy^S=fHr5%{ZV z>S$*6*2&V&xxqgpA1G?f>bbVF_EQxRQ#)HOkJWK`}b~cM*4r9;%qI>sQvUAy^Nis8NC44V=f*>34D5bdNIe><{}y} zx&OHw_$JP1>FjJT!p-gG=Emj5&t>Om!ObfyEX>Wr$IZw02sq=B6a1~SvHPR9PE7wU zF#r}2vKaTuaP+35Fs}>bd)^jWnE8iqXR^;?jc|Q4NZZlQj5j507T*LOSn9gs{k&DuxTZxsZ?0 zFXa$?5{FcxfJHsfe-}G*Py1x>cuwB}Vn)$u;_n}FD5fkyY6qV-HtfmYKh1aw->evKoQ`;Mi-lfF>j{pkdD!0zLcGULTe-Ol zm;4zfdx)#uOueu}Y~w$!z}+yz>dB=OC@p#%gWvC?I%ar%xmQq zAHo02jVXLv?lX>7|Hl#WTX?k_)a=1GZ+J-=lhpbDZ_59ZQHf%b(#nS6ap;u44k99d z^lGGvf?b2T&D9I^k15wpgE@|0yU-Cf`TiEx#{QsW1^o<(6Tv;sQtrkzTkEmH9vbD* z(3@j|cyj0WWhNb=<7-2+<53L$o4-neKgusBaOs=K1bbWm8Y@yRA!!Ll6SPLsdJ@ep zo=eXhv^II;PRb;=U!HN@oMMcnX-`moNE_X%w}(@g=g0g3U`9LL%11U9tR@kb)BVZ( zHlu&M_}R)OATX+-DKvqyowS*$hoihN-!ZLM8Mocht+veaIqLN(GTmSKs@_R;XoXTb|{%nc%)nC zv^wCr*OR9mNe6+~Xyhu5Pt`aTz522~Ru6Zgk@QZLKYsW;-^guhmy$!jwlad|*x&1D zJdf)ve{n=ptwJ?Tgl3cD=ELzG_cilQ%EP{D@%c$XLqYh}Ddj|=4!;-P19cHnWhSj+ zJ*(IUPb&$y>%XOY?#+&U`Be}NrLp*|RdU2eb3Q1#?Rh>qyHybvXCmmbuIAeJs0!V1 zJhKXapqfe|dzJ92$=j<)CfIZ{g!Rf+OW!5paBI5n)6MLh6sT#|1^1nk^`o;Q(c8%k*ZEFC8 z36zxc#3T!%a_dImW7ETSoeqIst&^nE8@lc>$(6^ch1Bi~VP!?sc;NvG`+-_QuX4ac5-CAyc-ROGPuV2D)p zQH^udncB@(n_#;teN!)1iht;OFw})EH|xu`oVa61Bmeli?!A-bXm@r9|!vsz1|EOt#?fbJJn|6O z%b*=C5<>?bn?=P07Nt0a*?Rc?14I0tVGa=d;_fZ=y1nH-+3e!fS<^0wbb}r~6^UW; z)~JgM=UXAQ@Me6Mia4?s%Jq4=+Svr_PeMh+|EOgRH?x3L6N#;F!r(=Dqs_NK-MW}P z*!su>J(CCO0xUO|ITGIs8G2nQt;eK}zl1)WVLcO+mz1S9PV~8>mT+T@A*aJ6Tpu0J zs?fm~X`eh>qpM1;poAS)I}GXuO>Vm8tI*tB%%5QIA)h4p6b!%F{W$x z1%N4hTEC`~dxMA}Z{;Cg&Ih_)vaX+!_HN#8^II~7@y+`t6gJbfqreLAhDOk)6bP0& z5%&cOEr&KgL?6??ds+0#EJer;^~a0PXQ+4%t^1QoacsswP_F%gafj?Pp z7Pi`9zOGy6S~NLu1+)X692ZBfTWJaExDef>wQs(4)Qm-ZB6uZJ2|PRfzc{ptQ;%7_ zEMkbmD7jq?;vc@)<6G`aJicOjS#~>}g5#yL?wZ@1H-E&lYjz&1)4F~a>K&@$xg%gb z_E@_G?;a3`RaXftZp><5KQ(%-+tZW{O{G5!e7@c$$=(R%C@!rSln_lL^+Q-r`x8r8 z9AYy2n7R^UsBaC_x;; z`%8C8td3`Wni7?RNzQrf=boHz`&|EmPoivpxf=brI^VWiO?5&hdYx}pZT51M9Jrnx z{#v!A-zly=dnkJPF?RFvc-HL`+V7jpfxhQ%hupNd$5CaLOmOY9RB;bf7}_*aeC+C^ zOHw2!iT#Z2?-TJE2Hw&{Oka)D>V)LgDKX_Ui1$fWSdGu+FTim>uYhQY zb?2`AuJrTu6uZVUU?qID+#7$%kGVIhW%#A(a?|dp1qT9aeOp6FBiNtgIw)~om?*x} zj3rJSEfjG7$YeH*yV3H%Y9wEsi}d^w}Xm|a-VV~X)eN7R97WhvaA=+()h{(!is@%U}_Ay~q_UPN&;qWN%h@&^Ll!pUiEW=JyV*(tHLExK7btOwgaJs4JV zo5xf8#}oOK{s%VgP@#y+`s+;fs<--)1`K@FRpD*{q{?8Bl5!&Fp;-jK#enm!lri?b zW1nbASb8-i@_fG9C)|rc9kOtDUK$(eF+h?~8OHPGdu7PiRDt0f=Xv(HrH&v*+0b|7 z8`1J~yCNYgUo3}Oa~wz1Ql#>C(^yhCU6|D=ZM(=OOOx~U6eu1(FU>)U^d`c-C(^GW z=t+IsW}@=Bz3pghnN|?x44&|O6@>$a(&d>N_t43}r&HWisFnZ^a-E=p8{zud4&Ks- z%+LCDC~nPgrui9u#+SGR2@(>Q8~jHzqT8>Ix2E;iM(j$(&pKis*7!u6Z)r0k(2KG< zq#(9-BF3=XC>)4qX&89`p|ZG4KqwonG8yOvY)PRqjZ-~gIgN9)9zen2I7voY?rV6< z3ucS6i<2_bhzA?Qeca<$!)lW7$1yBvQ&snRhlnSPL(CMWCWGw>7C3D3W^om_1Yo#~ z2W#6Kmh2qI(+uP?4RKOzK_cDq!e{rt``sqsLoAf)-$X07BcBvc%XF112DH)G)z6cY zD060POnmt)-$z2tq7izICE=dE1jNQg#t4@8upY#R-EX9Hq~*511S#clAqEVNfeId* zUw-JdgNha=)kkZT3rdlW(C%1G-Q}OS>AajnaTR^I7Pdfdm zyX^@A%us(a;AM>vPEcZuLzql^pKp%ksX+{9x;0sJRL6_;MeFtlL^r=HbAJhnxc`8q zPuz?(%{j(_TeoZNN+{vYo-mhRwOV*^E`5SRy{SyM|06^uJ_QGQIZ^+uKMt|r*3-JH zvtO6D*bon~8IFQP@`IP75qzYWu_D?*J%R}WG1c6gOR;K49<%Q2EYYo>i)(w2So z``TV0#r6~;8_Sl+Bw+IkeY_y&gG;33tg$@UAs}oXfZ#Q>8;eP0-tC!WGjZ#XU{e{On5JtC4I`GOWaTCMw#dLe8ni8J@5^eAI3*`pj-W) zA3YbX(cxCsfaCUuE^~y>X0=^SksELpO_dtU1m)(`ST(KRNCW_CXV0@|d1kZt)lInU z>lOT@Se)E!Rw_IFpBNDUk6JvPiZA~Gx`+UZ#q~+{Ke(S~Jh%$qD8e0L;%_vAu@uXf z^Y--1Npp(6_%FGA36o>tl3Gu!^}TcDYl_!(Z#;(=yLGuB!MY&yk$&ySc~Y!IbAWJr{RG8O z)Q1nXnw!h*pQifB&bsvaihX3m?z!As`l=Qp0p#4RFG~#4Y$1EG)8(>%b_hNhFnY8| zHpZo*s+n>CoPQ6c;8;OUAN(BNoV06lQTGFK($|VUWsGK;d-Uq{Vpl|!_vP6*KzoEcqDj_O z6&1?>SUgE-=xOXpgwkNi@Q;8t@=yyqeoQ1p1$aLb+GIE>D|9uS|e3}$SbAdWC&-Qg&l6eC{iG0tt>^7_ieByLm?!x9<#Osx|Iy zf29ua@qEGJ$>n^!&3yL+vBI(r@=T_2w|QlT49{Zl#pm1~!t{7ZsCA=eRAOzJ7R> z%x6~LXd=rbFJe}ga{<8V+GY;T+He<%#tPAPU^zi`M$k+G%-iMp`F4|?MXEzk#TkHE z>%~u+-=Ry5f0H3VLYKb(@w$LUb2HPQxcu^Hl6zW*$ca4fxNf`A=`D6vrsA5N4ro6w)OH9iYj*8z_1ZW#EmEBpaqVUZ*1q<>Eb0rm*+q8GCYfSR8N_Z(AHHP z*Y9^+>421MAW~#8MEH)7jjX-En~`oJku~~~;L{^uwA`S)9qw=%%C1N!{Gu_(F@pbRaB>`^@(< ziIDh0f1v$&Y`3rHv`n_$V%1n)eY>v?#FIs z8D>eafC<8s-}62iw`eSk8|0MCwL=(yEdy{Cmg=_UWIb2WQ4BJAANx| zOC}|w5UfZhBGZlQvtK3pBg_;1ZYQQ)AbGm^g#gqFcha^A2o=t*tW)N3R5(6{vemWA z*p*LWU?ah;9IbP++r}|7|C(+_i`<-ZrOyNz>@?3b``Y6EVZ`bsbHV%o&5)4XhpV-M z$35}M4$%hCs>1wxMmT~$KI;zNrteb%AxE6hS;6pGDvPtPo^t*4uZ-lfOAcrs=yA5l z*&y?HX8NPCh{e9iN~k--xG81L`%{g-ycyfu%>>&r{XtR}4l)VJEiP4DY6Tm4+ZzX& zhHRJ8iNA9W66Sp~qClyIg+p{8BJ4`ql@>-nB72>*i0-}}rzUwh5^0DesAe+n=~}&W zF0fNGC zuF)?9EJjv@>GvgE0wmCOv@-3rtUH6^mjYuC&sm>^lgl;Wg`hoxuScCdtLcv|)e@2~0d$ipdZY zE{iQ=j)6_0VW$8zsBNC(KTh6#XxLaE571$^PY8 zhludol}5mx6G#o#vznyne5B^>bl+&YzLNNao1V;X{tAdEUUVaN&t_K*=jb}qw~bYI z=vFZ`*XPl%RgyGe7gUC4UC_?eb{&hf+_Ui7w1SXc03jITeJ)yVgHIu|k zv~)%(h$ab#B{WfCTYjxA;gJT$Di>af%{a>Vq?#1B~dE1G?;41F#SB?e%XMiH000hc4nY0O8a zl$4Mp58S?f(#<`_UAk3i@eyU(4T43o6**5;9<81*Xd4l*IW_CP7_lbZcjQ4?kXV~P z963wc6d`Oj-y?x7;_@li)}?N)d-tp3D{OsNJ-%NRs6Pen=0+6&hZ9+Z1?&&ZvQvRG}C+KEl>Q-f zKk)8sA#TN%zdw&G%2J*AsrgH=XnLiTnC2(t@ear#_Zp@|Z8yOhbGyo2$tJ}s95lp3 z2OeH%zL%b6%gLV0ec*2iY6ah-CUt4anfE5VOv``vY9hgTm)wpcS38t|h0(V~b;ZQo z)kJ8MpEbmU(XnQ&`8E#Ki=tO7<(pb^O#@gqpPn)*c^-L8W|!qW?d1{N`M~QHbR5i* z$(-QesN#@8--TuS-J(;h-}J0U&qB28P8X>%f5@K0MxduJ;dZ~ko~b+)KdP(dqpV4i z$S0Nb1$fdgFodF4OF@J+R8GAzDkegdQocHac(U5!2+$IpqQ92EO33D8MPM@(%Sfwy z7h0h=tv3Jqriz!J>ws0qN5<>jpINN0SK5;@oNV%KW+YqzssznJ;G`;p6~SEDczWWj z4}CRz8=k-#s+9&wYwSB{E$*$xA?T1=Z9x%?yBedk+Nj`jCspBPa<>hVP-s8Y8Cp8k zCM(Lh+!`Wz5G6C6T&({Ybr_Ohrp^k&Oz@m1sqhSAaW#g6haQ71ur=-VK%CH}kr0xP zNd7^w;WVXGi?hCRSvsW_i?aZ>mR;5l?_&wD0$9&wq%DW;^ih@tE}P?a45W$HWwwQ+ zi@1Cs0G*U^g4Eyh%%@AjNNwp&bk5`+lb~RaNp2lPAjnMKZ$o=Li58N#ITt@R;S#nJ zgG+tOp;seM>5j+q1Hk;8P zKF3sJtU~u)Fo+EDy|H12fM6?Ps~8h9u=2{2_bWKn)8JuT%j`||q1(5qo)L{XaHTpW zH5rdvt(E*`aWFDudGC{y6bd6U3Hkz`a*Io{Ct93zfGHvfu_;@LS(68$@ZA7hdPvA1 z)C=l3&z>+MOAS^f9bn~NmnjT5^JnT4-1|;lytS`-Ins7KS6?1+BrR@mT5_$PuEi!# zm%3Ccj&3_P!n*A0X%A`7>gtIg5ZU`KDEN(Q8$BR;dZI$NzQPYb1#b|xJ-LF~^+ur~ zXDS~FcSD%KHn^!h@@bV)mkzr5!Dj{h2h_f#oe6vgi6WS4sFn~B_%5~R2{95)%9pJ=MdS&JZ7k@X zL0q0uX0TI2-1dSbq9V6jCx~Nha>aj~gsx#hHx5>&e2sWz^ z`SkFwHVN{#m>&AyZePszdYl+Mn{hZfM&SmI4&V;?wF;sYAG5)R{M;ud_s#68nz$p~ zsNDW?>ww;>MxUHH=>Rf6I^1w20+0A5?fED9>yn7vjy84`tO!SvZ!gZIb9>ZXLc=yr zteXBJr9PQw+?h08$Jc5a)%_FY{@bj;wgV7u_!lI9VBO%=0^$fVM z*$numICf<6iEglZnTLyclATC3@%4kQMc zrZ6P~)^7^P#=pI5+VTdM_x?`Q{eRjB_Hh?aKpuSg$8&ZLjgJcNUZQNr%R9aUKUJGO z!~kynve2Q72G}qgK;Y34eDP6XsQcxW@BrQTXE5*jVBCRS;TwZnGeOEk(fk6Cl@(ASJJ`KkHPy{InVEcDiTC zX*>DyvwBhOBb{r-L5?>j8;b*<0zwf&(AMD zC%h=s9`a7-?=tY%d_nWAsRXbHS*fApvj*Z!ItrbCmPfmDCK!7xB$Bcyp(zay%D3zq zL9#1K?pJ(fO1er)Wh$~pCba)FINc6>f&z(%l)L;A{B4xcw{Hn-?2pM!Xz(~jN%;qB z_r0_&blse2#CAH}TShtm{LW@a0kAmlm_KTKuNusd6e2YP_0dx^5CQLv$K;H|b9 z*U+tf9r7Yyy9Zz)Oj(o@%7HGIRpty6Z9Cmx9rr#Ls5C_icLLduB_MfM0^Rl7hhwA4 z%&DBfNge;0W>kNI$??|@J4Ax^ThwiWd)fda^qYw(!>4%Yu5UB@cF}~xuu+=%nY9Nz z!GR0llOD<)mgrPW0ExIM;r+*G*$AEUqfOfl zG_Rm(=iSIDLfW$;LOL-hsoFB3x(h(wpcPruY<>%o5?+l6m|>bicu{F@{AY9rrJ)Jn zktqXuLC(b!;#K_S^UI>p1be{>Wv-h3nRlOcna_IVTHm-Z*Y{3Ne6h~SjrH1eYpNUQ zj5u#eG#o3?WFwMe0g8)Dzf|94bd)R3 z8!S1n-Ins$Duxgu(f_u3{(<#ScHq+!hkC-W#u9)DW`YA`SXmfX?FbQc4kJb6kkyB+ z_b50uu5@JNx%Kd;B9}G$!mj{h(J;R0LL>ezPT%{K;%MEPTOQPu``a2e0T6f%db?{a zY_Y6rp#gJMu7{&dPIcxg8OGDI?}j@gA&}A0i5v&m`c}>AOnyVbSkWX6;7{bD_^%MNM2T6)yp}z=E~X*R9PTk<-k(HwXp!>t)T;bxIv=}uOcnH%2)q@K72WLX zzY(uxfZAK`)eXbag{jx}9(@%u^r$R`@2{xG+2#mLUSn9J^UJt%nenyrRK-P4=KRGc zI%7Fx7EU>)AZ~y!%YCU(?(6-bnEp$f`Bqu31I~B2D|WjT82<;(3GBgGiCjSGwwh<342F#SzW zjB>hIjpr#orRJ1Smn$|Eoa8xdwb~FF{3+FCB^f15>NA}8v_i`NPOY}|Yaqyt`#wux zNtBdR7fBG0sosuYN$(-`;}+A8MWG1#%F|tRnK{c`Jn*;8bFcug5>6QzG5_dTEzG9C#`01UuS)Mrk)~^#rEbvCR^2x2_>-3n4xM}Eb$GKc4$4{m zV0eW6Wr^zLngr_9yXk)RFMw$Nq9%Dh@Ew}X9%UO-4PoU{+UD+@+(1FR_mbvvzLdQ{$r zqgq!+vBmmoyLoDF!@T9Wa;i;!E*&k$+N@UXK6`UKRZPH_G5Sd_ zy?%AZBVLEWS7Ih|pak(f{E%)MWnMTRi(gmxGR%OXFm37g1wECF5$En;QyFBO;ja~XB*jG4{l$@;EK z^YJO2&Bw?QP{QkSXXn0z%C&Plue)BO|8S`;3EU#KVR`%^2UlO!TMW$59)=SdM}1n0&!_BCwZ3Yi2`Dsd@c;+Z#vjVUyvk>SGA+%G9-2+Ceq3 zL6@(L&0X6khyOzU{Tmi_H({6SJNC#ZC6wsdDkm19={hD-MA%+|Up>y%X6DLaeZ7a> z-(aPDMXRav;P0-VQeskQwe4rxckyNc$nL2k|i3|TPg+D3As|8kg zs*+sg#7|AFD1ca+09<>q-)AbVVvF=@_?262OrQ49QMXgorG-`MZO+da+syQ1q_R`5g! zYr3UHnoC03yY+%2;IWugm#2-d44d^nFPe59eyVuPG6@CjP<1B@VX>RN&bou?;+ufL zbbJZ@4*1w=FM-IDyfPb43pjxOxLoG;8IRAddAR~S-%jdj))e=4Lho_uSx(ow^r0wm z^MAYPjsWS(JjKn8ul{r;<)};YYUp!wF<|u)wsN^XpS`vLTz8Gz8x?~nz>N0bYhJ`8 z-qPp^#hsZ{2n0}k#D1+WvBws99B;wL((nJzH1PF`exmf&<;`!I>!SjQM%1-X{B$YS zb<(z~rZndYaOdd&Sn1;Ox1kJR?Z)$Z)@}x@oG`=X- z8$W&i$MYVYZ}&>@$R4;i{C0l)_FUcL)HWI!0+bDomxW1}SI13mH2EY_JBLvvw&UD= zsk=bL`?bk@B`M0GjZbm3{N~|LKRoV%(*cU<7+|FktP=+)%`Xh!??1LQ z0vxE7t6ARb232p<-c&d%=INo}Fs z@a`~GXg_J!B9%&L+PnQ+Se1j>5F-=c7VX5JZi9%{y^$OG_sk0#qvchZJ@j4w*VYEW;? z$VaEj#|H^EetpaGAKG?bs4sOM|9&9cre4S=~fqXnK+VYrS-s4UO7uN^#1vRmr`el#}jSO zq4UQ_h;lz~r&&iR1x=>)hblCK-)pql@U%4lKC4myNJeK*j)?@jl@uBZleE8E|^{Cr<{ zaFW*lAPo=Vry1lQ%+tbv%ZP`A&3cO!c?qEJUCFbhmYN5z9Ij4R>zMi^z)!%VSC{7= zBiu9OCfga;7kT;s>0)p+BoA>}^#m-8M}RK;>PNBu^boO`{rCJ}?|FJU&z+Vdz>`&z zDFY*<6&y*cj8&5ummRvkoW0g*^1hr38kD@A>Z89SbDd9lP6uBCaRIqxo+RKf4PW(TQCAC~9pExO0RrI}m4@WN&a~U?rYUUl`s_!O zXT8Hg&`ZV&GLLP3vL=SPF^L?4CL{UZd-~gwK+yU{V*jmsZ1fwhHaerFV)nN&js=fWJ$F}&dk%HKL3%dDyJ|uVJN%kOd>eM&24OvSw zA<3V^fX=gOipRRbtIEG9=6&JrR;UJ$zmzn%blHC}@X~<^(BYXm#0T+Z=EB@W0!KU& z&}Eq5Tpu_9r!}J6-LqIjs0)9JBn)xaW*soAz7r1MTYh)EZfjS68bqLF^PU*R$Vqh$ zAoWvOyqPA*c|)501>Rf8{s+@!csPa#U}3KRI@91W$x%|j;767v>+`l|`Cf+5&Z_nk z9wETLD{-+Svn{&s47hy<-i!Q%L_clER6nFw1=TV*i1B&%+qExOT*JrU_A^;_sgWcb zyX~h7N0W9{vzJ?Ljmf;(k(E7S##fA`$;#pIHaSAVA(iJ84qa4HNhM$$#1cnwb?doW zQ_b3wCh6?!LKwZ>nn(N}FT@4`yM#3`i|Q@Rkn~2$VW}kpqT6B_Kl)`?FjpTyIStCu zt0JqQc50_fVBWZ?cR4(10^XAO$??E4GM~m}m=(#tg4=b^M)GWpDTB;PcQ}>CG}6d+ z>SI|(PARbzF~NW&Zkb{%E8Ji}2iJUsW0iD;fzlbuT}F5()sA}hHUZ^H*`QdVejReq zs<#bWy|?2LUM-L5!GGS<5@wCE2GxFJ9x!wdw5@l_P5s;lQzBvKY(m(abZbF-bgM6ew4mF7Ufu2|ng(t?{Vql6 zY7HDg6Ul)G0}vvCzO6Z5jJB+nf-d8R&B!YF!RCI%2^+)WV$kQRV!wqNC^Ms+=52LL;O%oNRv3ed{e8&0 zHHHt)Qn=PIS|^ga`Cu7*yKL2dU!RLb$X55Nvp4BeKPIQUtY8L&=GXeZhalh9muLf* z(H92k@LPF+Id(XZU7FS+86IFOH0xY3NR6a~;V$?KL(~w$E4a~4XFV77KRZ*t_f0zu zigSh6RJX`dtza>RxG|0+n9J6&OHPWPnx0`Xvwg|O}+ z;HuR=y@FVR`{=;6ML!I<4rX7CU*px~w$E`*uPn*R9Z;D&{$kgZY!r|Ou+Wt)otV!s z$t~;?&UZS6)wDXCX5MaVuN%_kW$ z_)+dP?6eYy9;CsiQBQR#b$RB++IG9aPr4nwGY!E){qi=;?qX$qA+A@qSL=+J9u7MA z6CPYHgmIycaK0R$c2YU02h>DXOE(Jwqt5-lIX<2!{E_X9)3Nwiy{yyq0C)kql-ea6 z2oHG#nArCnsSNBZ9!f%kJOun@AVrR zK|D`i>u3HCn}8s?F>xgr{OAOhf~Itz}gs69X7!G+jGZv>-UXI*H)a-?ZOR zA)6gnU~6`fPO$id^#S-USTTT(hzdnA&EV)C8@%YpFZUV)Cnz6N6?Z^f8KXvIfZ{mH z{;@-Ob?ehYr=vD3AA)$*i$O4YVJ=S|&V29qA zif?Xb2&mCG%g}vn)l#e@v@|(}JTzP}KjTc{z~8Z60WI6WNgXDNM)#L1IpjkIX!POONkO1hGY7d*?#7M%$fQWXuSVh!L&iuH8_MN)04nvgi5uOFvu?j)WpZabUG=_888EL1{~T_@ttmgLU|tiaRwh8 zEV_PwB|)j}*p>7EUu50%^nnuhDcG5)3hqGg2`8nQ;L zic#Xe@CQ%dsy(B_g0@mzJb!qOuYarKlWcWpB23;f#$Ij#PnX0An%+|LaUSOZ&iwn>P=9}!K(hP2&ny(6{`HoL%i56=XWZN}*xb-JON2)Ojy20 zFtxmo|M7qR7l7Gb$9ZTH6E6-)8$)#PZrkZc+5jWVNVlVXy4g;P_Pax9_1Or=c?9&S zxu^?{Xsb6BNmBXW{%m2v~mt7~9q!F|rVp&dru83St% zu9cylVGl;oi(CQAYPIE@yEKd}S;nTrHj@rH^cpi*BDj5({?mA;UPu`43U6VXSo=Mj zWm*t{5SJXiL=Am&2Mvt&IK?bsjRti`HLIj&&uN=W-6FdQ-4b{UDZBB>hk^H)L~z%X zZ$sHrzK~ft`Oph+`FRg98U!5~(M=Gn-aTtw{Z{2pEioKyf3cUg7<)yZo)ilkMSvYu z30#~u+6M_Mqm!s2-Ic_yjfkCmoftC;{9UWrFytG%A`SNfB$%zaXA&g-%U%Ey*P`M2 z#{TvYc&l+OE4r9{}dN=LzBiB$|Z7gyFr4G?L zbO(an@9^K?AcS>zx86)H%s|nuk=`6BdXp?N@YGsLsMnLuIHN zO|fjhxr#ZAW^_NR&WY2CDx{vLh;cjX;p?x4=IJHTWa%wj5KQuWSX?9%q%t9-`GPoynD-KOAPc~|*B-m& zXPo~b$w+SdIMQJHq@H3m+5WQAdN+O#EHPnfc!Ff2lpwQhYhilWe5nK#;LKH4VUQeP zIwqg@EPEW|>K9AR?N4u-%&f{b5H(4;Oo%t+OFOmgJKWh|4NV{U%9q;?TZ&q?6>G+v z@42+$bRihvP=zR;S93q$`bavN`xF(=tu@2MD$_)}e#o$qAI7?S+ryu5X_bPMGe5Z9 zB9(uuqvh26C)@FzGL*D*g9l6)d8D?)P zX*NC<^HdqN`rkub(W|;!NStak<_~ga$uyUb@b|0Dv?XEJwMsNPIWW|nhAa|GuGks* z(l_r0KjJ3c3>a*j|8dRMcVR5dWU3%xz_3e=>UTo(byl|VX>5sx$tOrtPh>%u6UJB) z7H32P?g~0(ko95At2~Zm@rJI%8RM~ck#ylKq@1>CJnO`~rJVf3z~k%Y(@xUep0ZYV zdQX5YTFwSX2kppqJAj*ZJ82p(Vh9M(tqq>3WGDUFtYLnE60N~vSxpDR8;ee2>ONfx z&~nZMTyqi#tgl9%j1Juk4>3y)Cvb|Zkue+}2u!h`a-!?uG#hn;v3w?56IJtqAwZAJ zxH?vGy+V&8wv8pd=H2IK1lH%ltzQ0=hK^lbBF1!+) zKcA-hR1R`5OYlR08Ae%}AxygEt0I>8j7vfeW+N;}HETbB>+s|9D<76|VAYCze z%j!rG?N{XB!e>DqBP3*E81LEcY(v$dC(K)e4EKHc`}g=$q1n7~i?{F?4D*TQ!@6lf zgsBqktpY#vOyfCTpvG)pc?g18&RW{Q3J1}-{?G;JATxMgWN3H>`O`jpf9*$8#$?Z^ zEvcmNdzq&Jj3dg)?_7S3U*T+Q!{?N+11%|t!1W0C0FM1BN&tCPXP+x7eVMr}W?K~at zc6-S+?WeaS95)MsR%ZPF{L&CP@UNVNLn67{rBNl7w-IHKte0R4kds z##CjEJG|PEn75rz0_sRi(HaPE4#YlduAV<}jC`;kk#GNUpDYD2wbM+^n1o>1+BcOT zd66?Kh8-Y`CEBhcHWX|k8#a!JW~v7>`iU9qfS0u!cSmV{ttU^{TF4|_H8SIl(dw@(5N7(bc3{{ zgdklaT|f(X*x-Hk|hcMUbvyST6Gyq^2=Jm0_Id4BPu4$K^L%sP%)Yu~r; zw%F)kp#h;obnBc0-zM5FA};#cBgHXDoL3&r_2KRHi}JAzgkW^ux8q>!_9F(D9`V^8 zmF*@8^9V>+UD;ueCCgL`hE?}-$08?NheV`#u#kJC24^~w3etqVTcfi3e7fk5vbryf zJ(jBYf2Kq$fkKg8rspr-;^Q|bqUAcR%!Hlmd6!$x^*xW7_YhQ`h@ak1i0s4Z*@d3k zyDvWHKWknLEZt#Z9w|5weYB8X+7^>{gaRqpIUdDW%s{O*;8oiXnpn73>c&<%S^(p& zw3wKML(8#hHJI34?P?qPrn=>)ah|+46k=gpN4H(#q890hwm+s?wQpE?siQrS&++e0 zIpCfSrtx@AZL!XfSdm2v)CEs|DU;JKZN@prLj541U-phVe6ePve8EiWkNm3vY z&74nyrY!W^VD%W(a;!AZePYZuaWZB4#OSx2$JBr_Uv)(D(V}-M*MP`lPG)%2=tb11 zo3mc&)W1y|Uhc6%4M9V@44-hs86EXE71Uc|(=I!X#Nud6`aaWsn!#&lE1QvKto1>~ z%!xu-p+{g;Gpzogjh4Yb=1FGa`Z<3K|6j0%*OffQyx7K(;E&{YUHG1R=+B;_;&htH z)9z)Up1s=5<#=0&vV-PzF#WMsiK8)g*d}gXT|9MIe*O6K=xFF8R{8C_gFoc``ADFU$DJ$!A}-AF zXXB}f2|FTv8tKDxs=ww3e?dYQUVuOHAVQk_>p%W;o@#}Jye$dSa|8c;ofTwosxR~& z8~XhD$PR)R>8-h!M}Q^a#*e@K-i|QBBzn!pS=+dkY&k$#CVpF(x(+>7dGY5lQAH4Y zU2l!g>Z!$fhe7ak8uhY%zN>`>c>hDz5csS>SC8d|!!cc+LPH%sL-zFU>7GKa!++CJ zz$$VF2X)rlt@gwfFSVEUe&9(e^QP+ITqB{?8yMI58KWg@ClAp+vpo z&lCI!h3h^l&l|))U-fDF9rv}-)NTIfBN6GNr2oFW{u(0KUI@~F;rfJ`-1i50g$Zn^7(M@Y}@}n3{mciQ}%Z)Oa+aI@KXCHSpZ;<}tJO z%;(3?qEvprpC>%0Q4JxRDCK>Yb3LEJ@mBS(;>*8$2)bt|E4k@1Z{FO=N=7`c4kgT{ z%5ni*g_QWY<}RHHLhHMi)0I|MpvBZ8cVZ7fvNz^du7M$ zIwD1X*EVB+Aqc{JcacObn|6FvpgEE~UOf%dT?QbH7OZ1iqHnA~YZ{a9K*$WV2`VyQ zYR3)!x(2|3dijsBofu{LZu=12-pxNc`a*NKUCP3hHp}9dpuefb2;lCq*Pzcj7Io@0 zlrDS<*of+c8c?nMd~hG3Pc~{Ma4kh0Q%L1jW<)NPD`lU zBj03F(P{utGz9{n7I^e>_BN#9UBy;#*rsZ|i$g6iJM7HZ4VD8|ZH1^A1mkufrR+niD#Vo+ZNsJ7b?`{rBxir~t1EPFL~XYQVfV z?!{gHk&vjc@z7<2;7&i0#G>b&;sM`G6*SHMuncmz<{dEN4yA#%HYJ(8(cX{Mt6O7v z7Qhk~^qSobE7SN6@&nk=qsw#e2;S8g<$Sk`BVI=LM~UlwiFg9W@URzwLn%@yTyV7< zIWk~*NgXTHnb!dUBUWEy(8IMDUV04(wh&p1>BLhJ?Va}xZqDalVD6xh2)e0d=Et-z z@;2bXxTzx6y)kh_5H|qd&AN9hQbn5$94qs$pPYR=v<_>76?$Ix0z=7WFQxpR zOG&xK^l!lY2pHs2qI!W|FTGVjU0}*$Ye0?~y9?|Cv+IyBva>+udn|!-UHHwx{o=L} z(LGTEDca!%_5rR|a4vEp+O^#dU*AP1-+)~2G9Ur(d&)c)D=pO`ueUV zk|tdNx_-~j#`PE;1)@+T+#aXhI&n}r-UXblSx%8JN7HT$?|6irFy6g`E;=J@-#@>e zs)3AYKLg#Enx`tf0Hj&-9rx!>LEnJ=TA&)x5Ivyxxt2+pct8aDQ7V+ z7@6u0M3#kL9H?+PXEnR52|h?p1-4nyWAM#V>{_x%DzeUFOb`fx>FGEsF) zxB~j%(6t9!`^x@-jGR^7VTU~{;nrm<wBMS7sz5jPE$t5~o4QJ3w& zj)x3$Oz%!g163K$mxD*i*!45(6QIZINx6j`&e#1oqo-BBs<9B#1g##JwE;()y%dq!H|A_i9 zIvNq-t8Qy+EaLF9_0-z7Hg_mc#%+F&mwcN^bi|PvAZ0mSa~%{FbfOT(f&J~!_dJ79 zKN7@U4Rei30`bpgS819f=8rFXb#9itBP*N_V|u-O9!`PLW6XF`2MD`s&XbZKcu7&T z=4;KYjGczn&wIH{sc8hL2XK1JUblroA+9i12J%o!DgwojD!#4nSt6|e1>l2^a0S7u zo^Hc?_f8nXe|U~Z{&Pv9=AC8Gahh!M;ZXFnWo^GY6jYx&N$oSY5)Yh6*$T4z8o!) zn~@05qeL==5`%otp_#6X#56rI33D#nFFgBh197PNY~!jtO&U7=G)rs)Y4#WZJOI3C z0gS+IIanqbl$gtq`1DS&DA8F}7z&kaSN>ePfib`AP;Aeh7)U4Iukq?9z`()Xmo_}L zqzw18uJlnf3YoXf-b{7J(K`XWPy?wKSZ@{s#&eTa%NC`+9#)+Eue|r&>PmMUWye7{ z;D9Tm7nKRo`rNtA9hk48PyN3|FtV@QFvc9Yc`RqDV9%=y3qTEwm+?e)um}&b5IBK`P~Z}} z)xZ}k`GL>omV%?b-O5c+{)+Fok3IB*&eEXl!7oMfkD_lQza_dEqgqI!2*<@9jiP>& zwg`v${qHo{zpe^?gzoW&d)g_yDqmKEOw!ydKq}3ZTye?dAsZe63dNq;a!?T8Sd?pp zKPK5$TD3`N&f6n<2@m$L%Ux`oK0 zM{X}*MO$2qT@cAj(x`62e_<^*y>cGv>i;b*=ydCkNn+CgPS+?3*%AU}Q>#Qvt2nk5 zV=3S1gu%5$gL9XJWMI8N1_@lAv?w$n(Ylb-f#TPB3VYA7=*rb^Qbjtkhq-{Gy(z-> zC70n=`>DM;{SGbgli9EDk+=Ex&m1YQ0+qCEH)&G?pw3Zr4m9FdflqNM8C7ax)@ji? z9QW9%56O~(KDOiGy!NQX5PpKlK=lLA@(t!OM(Ly3Z%U_c?P~#J;^5(_VH_#Fg~6du zpB7C^Wxyi(o|2&We(_kbfyC_vqKD4uEZgz&0QvO8ZF7+=`AQ8rFq^s^X|fSd(A{s{#>M}kWB300ETB)wzb!X2yHXX~zY5%Py5`;$WxF|aIj{RT_MFg}mzOT+WaE#I zMaUKbNqmTwvr*TIAMhi5BBxiLX=khHFypK9!y6KNlsuqKsit^7Q6F>i$NbOr^C!gF zRH`H&7iazT6vsK!Bo3>Rj?gEUC6eT4gGw@S;?RvDO62mVf0!PH?(#d8@e-*ycD9Yg zf-dA9Tq&2ossi5pjovMx0`&s?6uzm>qn-o9H{>n2jm*s9vI^{f90^i{&`~EfK~vac z!)P4qZgK^sh)bI3KVa@tJ}h@TMYoBP8YF` z^~5k)031(dz9A0+mSjd?1Ck&9JcyAuAyx?Yv9nAM@p`2irU0=JD9Yb3m2PQW(N zdbN_Q2#iFU#rl4L!R_4|FWjGBCME3-vF16Z~W9Cx0r!r0fb*BpRL$FVvrxgwTv z9`LVsL7fin3_vY*mptyWNaC<40c(yH3oEczc7br+dfZ;0{RR?V5-#iXaOR!mpk3hc zyQ6PAIoHpRfqLl z@O8o4&Gp9vf*1T2zQKMYP>)>|^&BlZ`9S*u`XQvd~sO_{wu9IE^O{DSDfs1%_bee(I_Y=~xz#nvK z^JXS>bRVS!Ovs-0?o`j-_H6}QVWrK3i4>SGB}|`z8qVb16hPEdbgqxb3vcj(tFWE) z)S7&e3h~*nTv(^3Wp(=*C0yWQ1@A7>;yKuV3;LLr6|9a`={$BFvaVaXIndnh6#g8V zQlt1M1KA2C(9Z-fwg~IQ!^!JKP1hb0qJtXHG1;k6xt?0Dk&21i0R1+y5&LwiN`2yP z?d{E_mQ%5yh0K)Yb;`z}4c2m2*WWUTq1h$;oD>1d2|2;*jRcuyR~e* zhTbh47Sxujp0ti>W$J+Nnr+qMM^-gh^_FU#(;-aI7mrT12xQsX2DsNrTdUDg>Hh~) zzYC`;mw@qB#L&v?8F6RAM* zqJ=WE8m`Sopav-W@O!jyf$7(7NiS0=@u%4WIl z(e804erICzOZsnMmNi(ogKM-sniGCU^v_|c^^0a&PVEDv^LiO_YaI}68z|(v0Ti+^ z1L17H`XQE?WL&l($li&9@H^^%x-V7)SIyc+Oc$Z_u4!5s;C_2C=;=1EqNZh}oMYj* zMJ!nG1-BY?6O=ykSX1>NRPeh6F+;#sYqNikYwT}CiTD8#fJCy_={E+tP-M=$a&SLP&`CCv6Sy#R1($mY z@(EidQ6u~YbUkn6K%mRr%;iq_Qms|*SvVn*< zb00p(lwz1u5w3X$gOm36mLk9pQDwpxvup8HpSQa5sybXy~$`f&vWmhg)*P`mRGjygu;UmG$UuVK>hWv2{}eNeB?dFdlzz7g`zGPszGo$}8btM=Chs z(T=Z1wB!d}_O#ebrEBB6kmWsKKY@ zm!x>_Db|wl)*dqB0= zjl zit6y@c4rp(lh*4oJQXX}dm#Mwobl=N*VyP2_+qag zn^T}n2q{oz!sbLMU<4u}E)ADStBjnS`xYE}rHtLA?#}yjysLelaw&$ z$IHvT-+oj!PB7%jkTDzO}~*?P}*Kn^2hHUQsLN&t7fRoh?#}5)_2uXjY4S1}E{0YX0m(Vd%i3O$UixWFdNYKX5tc(RTqv zX9O>x^nOaBVPH~M0q*#2h(EHVzY;IfO)BcmaBE8}aLIRQM)b4Dq$4UJI5o!`Ua_ZL z$nt=oqV0iR7#IITErc=vj^sQz6{O%59n^g*{{sn{>}R4%!HuO}H3#Y;qtEm5qkQP? zZX&}yXi6EdD{TiOQ84?0SMG^NsoNFo1GjkXDw-$EfE-J3=OEdzNv839xIjoZRz%ncAcexJe z2G>7%jndqP=~|SJsg{yn&YI?=)TCOJ_Hz*9x$xVrbnyw)(6PKer(9kUiOR-mt~->H z)r@RwLip@#<2F&(3`|L!eO&NR%c>iH#zGdKZ@yi_Lq2ov3aRY(*{7@;uS*(6Px?8g z8BtTo&U{{k9DX22jOdXj@8;*O1ML!95?t4ySI!+e%24X6KEFKAzTBpFnGETa@!n3$ z!<;0xN6+Zjv&wr)tmF`ijJ%npxLoaSH!kh4@pelEt^My$f6(r(pb=A`UWl#h3k#9Z zB=CdI*mT+t6b>Y6#U`dk6GIUo9kkYjO;H$pk!nsOCN8v#*{xxVRu!AZ&PJzpVZw}VabD;gXh=hr}JPQqo$$dJiFX|{z$vs8=vE7Anne41bg6LO2 z#mh;sMWVmG-2q;s3&{m>5{sA$FneO;-Fd|;M%bO&PrZzt9Kah<)OYu{{Y^~o5Txm7_P25Oj5REj2sRZs*bg43%{`U zB``LBol+;Dd;hUuZKK{Y+O`^LQ^x^+l(CoVg#1mgpuS*eY(p{>ZaV3Cl(F*3LtXY2{ThTR3bCR_N*`l& z?rM#5)fSeJ@|qx_8~fJElJ+gG($Q;zeg}3+$Nu=D`l({w=7wQq_uSfc0!_o(A1@5> z7>I%|V(gm&Eqj&lx4Zk3mKCLVtaMk4q6qyB#LVszZZz}1XY)9! zbuV%#4%mcrt#yvL3^ORZw=z`JgfSjSni3-F4rrjJDtMe7ZeZeOS_q`-0 zk=iwS53S09{yCDs0bUtu;G$Lz{QVYIMKbk+{z=pi2W>$b_!U$r%)*4Ei`3__`Cmx* zs`mL28@X%fY?RhCT2{_32$R!jI-G1?MB=+y8?LQ)QHRNz{j67&n?WLoLbfA+Y2DNm z;+U_oge>Qcv5zISfOO+@-HD&V(Ef&xoBQaQdW&GEboNi%CxV@L5|&(2nE0t(>PQy| zLSB)+K5|GlsR%gdE?`e9Tr57*PTqvA&cpTZ>(lDob^~bw7T`T{M~+}kQk*5TaDR`c zkw40C$9$(?bB72xb>EODqV`A z0&gyVc&@t==q^}w{&Ke;8&g}=Rp3w1B#Kz@pCjSSM`E-`4Ts|=OLoQ5&YvIbR#PzA z^BJnKh)5uD?+8aUMKNif3lkVpb=mynuj%Aa$4)0%3*#HoYtG0T^$B=1U^IrRL>NhZ z#AvN7Ptrk!ijM-7uzya#LbDH#6XTpeG#wW|SV;OwF>Rg6+u;WKRw{NJw99X_H>!}H zb&pPM=claW2!LHT9Dch>mE0p<^dTGKXv~aK z5xr+M(DjBhh&OY+6dhGLGulY0{rfa68rMopBnHZf0MUQhT3M33!LDni-o=UuXq=#iOxi zo8RGf=gdCfUPJI)iz(DsJGX7Q6K=7&#(vKxwP!~ zC%Gr%i`toz7ZyPm-+TGeypWF=22kg9Q+I5s8ocWXU5l%s)032lylh?}iX`S=xMXef z8u?FWsNGM_2gDLEjDvCJXsZM$Yj^H($IMig;1wws%Iy(1sz1m+@ZrCB{uVm{g1s^7 zEBA?(s|Zgt*F-igvY_hdbN+bkR{nadzHy8??D|jk?r6 z@=6y@$7|cPJ2*G6cjFQl@7N-IV5Bm!MK|IvpTd=mG8dypXyp`YK=cTWp6cA{x5>_~ z28&YiiKE=D9J~oxFv3T~UqV;lbhCKUS%PXqvoV<_ps#uQWD$|(VKUjrcB=z&e@$M} zs0B%i9-KUS$tFA>OzQ3j0o0N{8NR(-w06R(r(8=sdx!Zq{lgNOhD_YP307gB*^7L- ziqIn#KOLBA6PbjGAkaS3-EsdyBYD05>b9Su{@h`u8-FHK zc_nbaA~+(w&odX|rM&V9Nsa*Hiy=7;Vsk)o!aAH4pRb(iYX&v9(kMih7NyT z+gu)Y3@QbZaK|HdRN|2|YF=~IuGAVQtRqrm81;E{SBJi0%<-GkEvUpjO-P=A2vZV# zfjBYv^%s}glle7mTO1l)Hsq`yLRGxDP1>qRzrAJIMAo}goWp&qYT+j)2g4>CJ|;WY zV3Q)_>a1y8LpSNZiH(Sq2nbgsF*>?EiWRgXf?I#vhlY5snNU3P(?}YQ3Y+f^LP`Bv z3h8rZqOYP(I7ONp)IL$jC6k}M1y2r@(;D?!`8o6y4|8qG?N!i@lgC=3iSc#ZN-6srD%Y?i z=_MMsH9_&WQ{PZ zK(kt-UVEDu7$4&R@m%|@pY1zWS5&|BV$i!wCqe+mkV`O))Q(VXwrpiYp9y~1;7U~)7 zVk$nzovdb6fAqz-LlGm$Z-tRm<`-epub}(m;aCDRw0o>B@TFv;dFb19R??`FExQqf zSD4Zyqy}8OALNCe5)sC1jDIfA`VeNs;!md1qU%>?>3g!=>ZEaPFD7nn)-I*+PLSj6 zIx`&hQ(zstjfI4{6cX&RW(u26k{!e#on>uc`+4!ckwN6Nm)gwq!tS}?LQTMx?} zfo4~o_2;Mg_^BN~Jy*Akwj0Mn&D;X{ zGx$FJaw2M-;`(*kUS#kP$^07M6zr!tpCj4eFN_Q8LhZqXvAk)F)g)Nbu}Aa+Qlm+X z>mla*aPQloh2E0p28Uxw;4@f7l;MKkaJ zC~h}jo^#y=SFhG1qpaKSYPXolGow1C@@JubVav=rSv1*MM>tcKaPfoyQs{VPg-^&& zZ33Nw!%;kv-1oDVRj0`Eq-GKhX?X4(j`j1<(`aTtui`P(G&$lnDLnLuwZ;LNm`ap3 zD-E%d?@j(wilS4+EWrp?5&_-uW<=wJtg)p+LdNZAD}AV$T*j8NTMtcJi6Z75c+>Mu zynI-t5A-o7O@aj6Kd3Onzk5P7cL%sF79Iz^CPBZqB@R&}2tVQ2KFTpClW-e%zm_f0 zeDQm{u)MJ0G=;ACsp%Qv8i83>{>>McZw2e{uqxEwOXScBUfv8#g%H0v^_M2}7f@({ zg0QZKP&)h8UTdU+tT;uN6$5t|2HM`w4&RLBUr2Bh4+pplEDe&*c?J%j8-A)vH&9!v z{l-a0`Aqxt;270sKX7l*^YpmRV8bcFfa|2`8QkV=(6Hb_(*xU~j|Gdw?h@QVm0}Y! z(F~ZK1v$$QJJH>P7{l@}0Z{h@iO#3ALL0v|E&IKaRkVct{qEJZJk+*H{yG=ndSmTx z#WlfSRBBOPDF@YgK^g)1YSH;q-z0a=xIT40b<8Jj?HFFp{j0HQ^1ai(3R_O-25*l{ z5v1zYtBz_f)wi!sOb3BXZc0CFZ0c2BN?|%(%Gc4Z26G@TZBGpbt5-oa7qa z1&&Hdr!89*v4>nPD@0UonvsU1RK8?1VgdT}LqyxPSKfFDMU)*B+@bBHaI=8XFanPl zBiUZywoiNuY0b`I%!VI4tg zAgjxo{LtBnE(UMZ#Kx_8HKU}SQ}ypz^a$sjX-<5-=;USc0bSVd=&9dO$(($uxR@!Y zQM`@f4#uChthA_bUBP@!;xL^IfgM1WzhbQn3HFu7iRLl{bn`majLEnZ^DPH51 zNpFN%OhMC}F`UgnR2a@-;LT#s+VH08$;2nRcP}rl)&2yH!33o(B1|w3g()$Ic+zv93OFbjooRPd|N)(Gne>N7~_ehfOhUC{* zhW$E}dp`NmgkHpdJHIXA9NNi87V9YXm{^eUMD07pf$i(`(OwwNnR2#Vx7PA9n(` zxWeE9s;;n6#hrSFRPdHk@%gxblJAP-tBY4(R%4EeMILbfX7Q3XN7|p*nb* z5E*pLGKJm58XbhHrRV5`4P&N%8tLIQ=)#uyns1M$5UUAVO}5r;-}jVg%T#4{Yt)Sh z`H(%PD$IiOziJ5oVxBp)I&0%|A@qI_@#}+f%d?4h0Xe(S$%t=fN@&oSxo>%K$=3gF^d_vS5wgj_8Ve;E!%&*!KB*WSBJ&V)BEb|0U zX;=>I&3*YMyRkPG`W2;#7AfuflN04#sD<&>xk@wy39^_=GGCyU&^$>)Ag(P&Va9p9 zMpEvHm|z%S@G|Y>@R_(oi{9mKU{poEK6(Nonz!wJvy_lS9|&Amu4nRr0i}c&XCii& zhU9lhJL^q^#cr+-eKU@GrBdHq1agUCr4md2d+ckLO5xP%9ILa?2OK+UTF`!ZyH&MD zd#QU;4f#jt6q8>;-{FBoO^A@~&}FpAkmF;jklW*%c#%jk%&?{iK!%!K7Bx zuuFY+j<}JdyTHI5LDHgA{bZX{c41eGZJ&SAv^fANhs<$`6?tq8@$^73;AUczA|R{_ zjWpDYNn_|^)DgkVuT{JQ9JY;VqU&#lN4-p_2mOVg=30_%N?GT^%bHfgWr@`k?B2gg z?q-L)hSXziVu^hq88rV^{c)9~l}`HJj$t${(OCEU2=?6(bfRDD%!YH_k*O1s$03C1 z`3LQlZKLNe=*Fght97BLbz*YWNMcT~?eo35#&P#^T(Dv^yq8FV{@VP0BFj8fx7-xd zM;B`BAno;jF#t34$JIBahgVDJ#hg)g`($S5#YwO{Iapt2j26prPxi1Kz2CVcl+~$) zv+huKob1}pH_qM1Pk?{fb%W{l=Zn*s1V)Wvd4be7h2zSNQXMW}_X-nd&T;dHBuh_A zP9^G0#N;L>L>~)0(Vhz!r{j;ckK*MjdI5G0GpS%Ur+3EiT;}iKfq1mqU#!LrvQe3L~cXOvT%?~lx^vXSDj(eGQ12=`4@;PW_uEQ&= zKX`Sr)4Ok|ky@zaC(1@9&6#6yoz9BA9`d(l*@DLG>k=^ z)o|0=v=OX`-N#nrjUb#>0!35RF*xaKM3fL}3fyE5(|t-`40}{a`q*H=RFdgr-6Q%y zca=$%HdL5*SkT9So-#SAI;zb&)w=G`pX6BPm{dSwUmb~DH%M@o-P^%Cz9xi}jlg{) zez=pl^J~au7#C5R>p4sKT9>;;bGQ37wVYIDwiSb$H|laTo-7w#V`REgI=y z*a@n>Q#>2F8!7edPUTc~`*YAVOB>B?AKRX`F8uy_$OgtMz6jQZ9LhVx5GkVlT zg_vJ%6?VdFzNu$AHpb)lsys*QDp5vE4n{UU^@)CF!>8hevvdb)NS^u2*P%5&nhhH@Vh=$LDcdEC8$tHtPsXA%`-He|aW@?y21 zX&%Ge2@z&+1Jf;^&<4QQ>sM zyEcPiy#@iSiKw%BNe3Ti_=_W_jH3@jAa*FD37$^Pn9bc>@6m_GnipQEOj0ceL)a9IM^d1T7!PaR+rG7H=o#Vl+f4OiOv5Pogi&Qeu@C zn^!--<8?uQ@!@v3n^VVWDTX12z#2IeK1_E(dc2UZL*ItMGDJU(MY0NmKt;3!UZ=fD zGHO=3K5w-!44FAdb4cmyi}yrBV8%m)zs%T{ktG^OQe^F|x>V|cHx$k#%+Gzus+73V zcUl_S?XsRbR2y?iYMlEZ&tlj)$0}6Y3dKR>n1kRwF3;j_7Eh-tD&}puIWZxbh_*TT zD$6^w2J_H4eU(!edvuKHHJ5reXK6JT>ULvVos*#XQr~V@gZGfyh7nYnbg&Z~l0FC! zzw^X2T%&C>nmNXujc?ff_W?vYvGTn$Dt|&E5$&2(Irni;E^-+|Ds47)x3{5YIg4*w6Vb(W&b79_;q~(Bq^3OU*BU3_@94j{Lka%Pt2>;{^{X0 zdPHdvPQiX-VwwN^%731rNQhDaw+88BO|LHce?RNpk6fU7MS&MCefLi<%&348<7?^l zWshy*e;v%fpIO&K1l03Ed4m;iWU+U5a#CJeGTT%4&6;7nktV-~*+4qg*CMMkm%!od!m}32 zki85&g8#B9{~#38}49$j9(_i5&Xjt@D@s=03$@etJ?k_2cHWB z43QIo>~HP=KX3WbUbMDLM)W4T(wD`wO%((7A674Qg``!ad2|;(y}fQ!_@_fsPset8 zM`M3fg%PjGMOHU^PQ{I_Lr~y1^@Dhe4z)`?CHeUs9!G>GBqD+{(bDj zwtWZ0XP&eYNJ}(A{mKT{WO8KoiQ)%{F2vvm&RNHO7Nu}~)A*A(SvArMZTRkqd=B2t zk>rz;Un_MsYS|V4^f~nHqrj$pEnUiAR1SX*{+32oC0=v3DP?-8Y$fj~@RAjS=q1IA z@~(6~=J7c}O7~kx_%{bN4Vsjyv*(*LXZeYk@b(S+-c~S-Z6B7-_v|%!Js5>;4h#_d z!>HXtd32!h?lt?;t5SF?LF45rwz5htkT zLU+QZ?N+bemBV*aO?uutO;^2_(6Z=52Tn&I^mTp~9ka069+r6TwI-!3b4CAmhy`fZ zFJGv_@CMgmn%8q{Hge7p$xB)I?dQZWrhbYCk$pP?XXd(|!fSU!)MFcrB?p-{ zt-g9!0pHd3P{q)G=kRMx+zKxp0Do|M;8@KDU7@dlKH%~M8`P!45(nMr0F(Z;vUW$& z6>$lWVDezMKJjq+29;LC4ZzO5nLqE~9KrHDZ+)rP;;Y)ewv}4p4ydat*zCRsC;BGH z7#-UKylN1C5P*YnVB2PS7f}3NOH|zH76Lny55WAG-6%nx(tpJXTBrr!turUS~^5#1yZg+ zQV)WZH?e8$GqpF{I=5dyV(m*3!PEC3R5UxqzU%~Gq0#upt5tgdlsa}QO`A+r0l-Vg zwst(O*#kxa+XQIGb~GO4+8QuK_S=e)Z;6F@T=eQ>gJjs*q}s2vTuz|BgpCz?I1p$m zF>}_=8!BE_&bgRr*3N2?7+wNwrTa*3aLZTXtnDEd*g(+ZYu%;i<@9{XPOS&`C`dIV z-EBK4K-^LnsMF;-kprl<#n;_RX`hC&@oJ7v3O)G&;&*GNtiNYvb_u#*vUfX>uS-i9 zj~|mc&J1&qy8&3Uc1ifSZ9duUsCyIW`+F*ynUXJiQov5+la*Hhz8lyzt)z#K?c>(C z{jwcEhn}yxUo2O;KJEFQwI{AZGPU(4lLcWO#7g<= zw^RzHt(NC(Slg~7?Xi5d{N+)4D#0mW5S0VfQBEjRJB;fnOWbqgz3P4eR(c?DnM!0xH< z^XDwDeyf&2&fK z?HcWMkXr1O8V-KL^Q@c(rM!ij$LcHuXWF*+Mb-r45gm*BQ8&FEL`~b-h=YgVDWb?v zNkKsDSXd&Tp0IXWooMhY$Fmz00T^@^x|ccsq40)ma4K~C*u}YVTCp^l!$*Vv_$oWe zvL~1xSaHSF6m$v+{ha}eA=es592YKA){;3H&Nh|Ni8XU8bVFvO;m&!XOb0*IYQ%i> z#Q{d>@MZAJ_>|dPu-V|o3V`)~;q!Hto~w#`o|v62)sR6BNO|l(XxR0?8Wll*5px*A zbF%N^zMEOKPBPYeNp@TTfQhf**X-k}l>+1>=Y|2%HlM6cZr8`U33guIe8ND^{h?cx zSIHP?flaeQIeOg3ioYM@pKPC9iEpV3xh#M#=Y+;&CDb64BQr&H*Y)>agNeh(xL}W1 zQ-NTY{>z^Qu_&C25GwjE&KzJd&9 z;p)=f$Go3$s@kTj@8vF=^IQnzFJ{##4&3BWt**AgjVU3I<6=CGO?cSTEqd^d8*;F> zH{@4lN;O5dBDm-XBTMR}aOMNI`BB2g^LF#ztl2e3JROUNuP+a0_Zrsw539r#E5WDm z#UaL2O@goav|=l74tniqnU93E3s)cnakBUiy^g#M{APYRN4R_F%ylgED1ztBST*_& z9cr?Sb*3a()n8;(<)$7B^O$5z3GqC7|WSONX1!MAuJR08k(FN-#3 zvySrMh7UQikn$Qsh3vlSw(~9nwodv1la;PWt-K&r8G*~m!R%5`8>q`FgHGZcP;{=R zw^U(ruSE0(9DXo zoLb@&0GMdb&x)z%cewE;0BieeA<13)9%r?%;&69HEtk!2lT8!5uDfPBNgVTT$FM~# zPp;#fQ0j0#wrfDrOYJ)kx{d(Bc0AY%EG7{ePs9A!WRU+%790|IS?alb*XHzb^k z@2j%zCkF^oJH5SqRp|VNDW?x?XjkZki(FfYGE+Mc_&(sl)1r$a7oSNz=yKvMZ_ z6=VqVRXvZQ(_{;s+8NkDn3md^8$ha~5&}n&1+i?$rj{E47eQo$SRMyH-TFxUEE4m8 zfnpb&qL*LOAqz%a4B@B712rZ!a+3`WI|}RNdz(yr-H;paF22Ncvyj4BGv7)Or>70e z5a3jluU!keYTIMvH$vAHZlE1Lhb`6}{7AEM-`2dCYZ9JKGCuioWNOp9RTupL;hmfM zg$aE!qk_cciZX&72zZ+aP;Xo>BYE~!gkUD>K&lqhsIvIG`*Q?Na{1Qv(x@g6`OLAk zig>F5VrEpWYX`GhY}#U|R_y2u`#OtGbm9K{orw`QRqKi-ac3EKeGne~8ni|AX{+kg z>(U9j$?Sc2OE^uiTnCy7UMWg?XfR)HRtnhZm7+Jc#!!FZ)4&DAB_PXfe}|*c=+B9I zF4bK8w5a*81GVms_SoqtA>0wH7COHjiEoJ6C9n6!((ZN*8BUnwtlhM78BH1ix7IIn zCk2kHS`V+kFaLT-Mx%LW35f3uz%nRjmtXI*+UaPWV#09ud7<^TLFzoZuHkZae<1iZ z1U?i{QtvPyK$fXitBRfiRh=E>fQT^Nk@?q2(G&zY-M3(xli--J@UdYMwL9{#@i5e? zuY&5FMFKu&UfMrGGI!w5bg}ZFi|aD8foz$fu!?P))zhHa3&Tdq_o@RG?f4#b!jNfQ z)GOXfmZSc{0MBLmvY!LU0ta3@IhngU*IR1N8RH4thLq>2uKV543HOGVuwV?7+1&BO z(RsrmTle;$fo=0p5YnZoIi#>=+Mgp6Q{FPGgiFo;fIO0XN|Wh4)@aZIuB7F z95H)fS01=`P;f+tlmss!aJ;~MTXnE)=sg=;)wjet>7B#%!W=ENoo~evOmhYu4%#8F z+^J`AM}I97qE|Rv{d$E(U>Ty7-)>yBKZ+)Azy?XxWYGZf+y_0Dje2ba#dZhW0LUJ7 z&)V<3Dk1}0T$pzR&qUC|e}qHfohD8A{!{F7ymD7n+}LKt2CyW|&w^dP7ba$9p&Kpf zUMY=-fr2%-heft)$(T$1ud0_cCpg%epc_6ncb<1G-rluenE}w{Hs6HtnPHrpR`z9j zys1+?`OTmk4LyS0LrB~(`}ONJ4sZ$>1Y>dR^g?@vc)JS zO58+saLG-L*~wyK|AF+?BCk=K+ZJ~I&ILA~q3IFSyvg92pDf7mw(;0AsX)-bRpp=Y%d@rL&D z{gfQHRodywGonv!($w6 z&$kCGx6#QFON=4~X`>_r*!-l5qXmrvFxHet6nzeSSOi%%^9GgZDWzgbhrcb8$h_#? zZ`v?(XQ&DD4 z(2YZS@WaO`9)z0PS{AYA$3Xka7JaLXha^A4B*|+S-$r|({@h`l|2VY6t!GeRH{9FK zI_iSJ%_m_RU3dCJs7{$&e!$RASouduRAqT3^e;`anYY9F>OxSFP6WZQx;`Xb)T)~t8d%=&I2=&v1)aDnH{9Rv{KJFk}`1{PH$~uPh0A!#Um^fV6 zIgT_IL*w-qI&jld0x{91cHb+~f^av=#|`T!uHzV+XR}5GCoc%RPXqwdDV6$E-+~qkqz!4=Y0QnsJnBg58NyC;nnqzeQd*s)tj!@GO zo)fmPW&Z&n*s7xe7SKMk38RoodBCoK0djtfp2G#MCU)O3$_hGSEFm?9`bAo zes6V6xu~yjn9yca6a^BDa1eyRm{L`%L~wmSyPI(r&KTcBIijAYZ?=IfLF65l4Y~^n zGq00(G925+uQ@kD2TfFFoU5dK7{tTqPNc33WVue|;5YPal-SvJ_f*S%)EWM6ytUh8 zo~CyOC>bPN-c;IlJtp6T5@o(BJ!N`QajO}5y$UP3I~pS(~ey7uwSgt%SXA$Xud|UZBu+P7LPBWCSh%< zv5=T7OQH%)XG3Zq`$%shmLu3gL{hAp{(MYOjCw~T{VD)UAUPpj2@QE8q}C>q2*C-# zZYVq#J2|;}QXT9~i@g5jZQG>e&~wF*Dz-Lx!;6|}jwTqkb0=@F0FbiCgc-W-&lCGv z`Uc#7w;v~-5HR=S#YOb#K6*Zj*#&cPEU7mNK8fVLnv{@}s2dG=EO}>ZBi(jHX=8VD zBkm_c=HOpSD`GfXA;T}|OQO}!xH$n9y$ze^;ig9@vqv_Hn~d$<4rROZOSnHhRGSDP z85D*qauMVdh?JJ`I z1B&2=>c}_O2k1}`GOtJpC+PBk-_w`@|8pOorwHevewS|6TwD!R@iGhdQ!bFOG#54b z0s6f7V;_6OtlRZqc_ncaCr!8UpGti}U<1&tNbiqAOoPB_C(TmyjdwT6jsSh--G*ed zu^;&ecoFP|+~F0iS*QBS+IEJG^R~e3fuq$UN!?6t=hXuz86<{L81_cii%|&6EDwB6 zcy`#lNSM%r0-qHMr6L}rZO9+T0r}{BkDSrDNi!2qynWvS%=vn~)bS|E#)38Wk@^C9N-%a)plA5Cx%~&G_XM3_nUbR<|2<`AC-VmUx|= zoNL+9DHF`L)Xz0mY*V*LtLxSGTl1=hz=<$fCPcqfAUT=Xsc=}$ZEP_FcR*%9>3Lq7 zFSVrMXGcsb*5aETz_dqq3N768fRR6vf33Gf#m&Sw6-MB_CT18&na^gomqNuF-?&oz zjrE{`aD+d0Jle(Q6Pv9Pv4Wv>B@!PyA5KBg;`b=1R+4&MB+1#`Q=Sk!qRCFxw!OJ- z_cJNGAPdX9`TDoBnO+HYve)WiDXY{&7xMUJ?MJYZyEV)+G`h-&?HiLG%GChvm4&dL z&Qja9yzBR1F59)58XzTcJN2bY?dV!UL8C4Edu3WlX<@MrQQ~A6z7H8;ji<3#}JYYBjRKwWa zN_1gVq2J{q(cBc=xIZ-$WOy-U-l;~d>uX?dn(7mgevKb1BHpOS!%>B>PS?_Y@- zR~qsbRmk>s%8Mwnu{R-f4^3w+Ch#k#p0Ex1%(k2!gg2M{l2b#-$8`G#x%NT?i`iCc ztb;!4SBz)K;w*uF9rZF>VXDl3IZBrd8c zwht8%7gq#%EXiumLM#Cq?ihiOu9c^)bSog_!bG=m^BVN#%!ojCgyIWs$O}FGo~SBT zwatU)U^OC3=u(&_wNZa^?n;!CYZR`_uZV~dti!M4uQ1SCK<>R^$sNN0`~j=QVI;&@ z8;I{ryD|vZFHBdQ_{Lj(H-&$@cE8rDDuS7LVuK)2RgjKFWXVKWvxnqsobxUV5#d~B z7&L!EU~ZYVzTpOnBZvy@EwSt-HH(W27-SsW!xr=z zUsLL8k<}5%#xhUQ`qz3*GM3faPX*}mz<#eO=l5jaB3CImOJ5*M7!M~07?!vNu=;jG z=RX^y#XJjLKuJ+Csj-XX<(=w%sLa9sBGa<_rK8_F8FXaK4xKLKo8M$(VMBp9Q73OP zTZf&+Di8OiTV18e5hCds-NR1NwA^BX&Sw&S_gu}}WHH^WBhP+~F@cAb*7M=ORe5;J zgp4s@)Z^CVy2j`J3%DKh>qZR5LCjPH_>L!mD6r>T?W=+=d+|m0EBs5zSi@cyvhCV2 zS34pPt)~1V?YtIo0!ceS5R4@2{_}YoD}~1Raz)ZnrJYxW_*X0~gIfjgrY5zR|0j@C z-8)3X>zN--7!6!}Y(nUa9Msn+1oyKUEMR=PK2A2wma0n}I>H$B>u7hwp(POYN)Cci~F~MUngZSWcJ-DN}4FD#sfZLDmA++AbAhx1RCzS9;VRVl8Ci+IkO>jL3QiPq2)i z3-KM*7)@b$*Nh8Y{^l3i>LUpz>%t-;i0GQ`OtO}FerUMgZH00X-5RTUcDXlh($ug} z^+$s1R(n}wE=r?U;C`LvqFdlli~l>_QStEhc}(fsxns;~dCo=Z`tkQL}Rs_j72#5Yt||sX`F#`Tr0BV=qOz{%y$y=Pc{K<;pDT_>yk6x zs={w&?eSv_7nJ4l)^8Q$s_e)khs0Skaj589U#BZ=xsj4qA~Kr4WOE;*vJ3npC%lphv6B{cpE@NRh6! zlzOwy1m$0N33Lbh@B)U!Z`fQRf*>!ow7wTFNFBk5$2XDG{IRz?R3{Nu%O^dqy(z3{ zv^>Z+tp1}|nS{27lDf-V+=wF6{TJ&EL(&a{fxw*q7ggtRwe%k+B?@`%{ShcoyP2RrCp6 zbh;TgTKNY2UC#~pUG_qKCV+^!)8~5ZK{f|ck5^vJ$MvMl^qrADqr`Q~Wv{|`x77WI z$Y9ZE;r6{ZNVwCNSKnOEHilgNS1GYMxDN8OsT2}_4ktB|dt3eiB06QT<1V#f?EBNb zRGl=V4PIkjoOD|9!F`=dSdaIiq(ARf%1{QLqj5o?Z(Im!*V;}ATy!{>^xEHbz3jSR zfMGhJCC;EOurp=jehF*St++p9USn}oP5=1A2SW~ekYwQO7MkUNGx|M^4T&=t6OtX%YXPllc z1eb)cexm2+dk?1Pv6Q-G-Qg>%l^?oAx@p8OmsFN4V8oEo$}o-Xye#FVY~S> za@9I`A1GE+O_lSeCjQ&Avc{>@M&BQ3Et5LMB#~40ZitpHr|;h3>;#Jz=Q5&wJ=Bw` zB0)Z_jFn6oe(b(&Y^y2_>Bf_G6B7HRzgesR`|+%<^-!S(cfo^{`;L2>@g_lI_Q(1! zlD-ZhWXz!ElDWmxfiRN$Qr^kC!hADYL+KT2?oaBTOI+N!9#N_EQ=|#=VnO9X|bnBKhS?N{Rh!@+aKYrtfQy5PueBDr3Syn=R=4L{$ zhgq{YvnH6yFgLTv&e>jilZNF@;beCD70H>r90&2-gzKyIT#oQYe==qC0N4C|t~UY$ z=?NxMOAHKxt3%BE@RFz2-t_a*+DE@!Kxo+OxdrC(;F1haoaxLJGiF1ALF?K~u#L+iqy>BXWwt@wM35frIQ-tXL zia<+IZ(c!^U*FtsLHYgs-JdIq;b3`=cKmHoNk^*c+a40AC-a_W&AdsH?pcw*qeaf> zS_qG_+KABrl+2iw(O-PI=fLcl-Gir&^6_$<{MgRL%l zwqZLBGe(Myu&v83Tz2c-T*>t5{1G|)upx!q=*8Vuil;TZ+{wNhO$wj~V}v~Fa%bMQ zYtt-vxliP7z_+@jG)u(hxOkcINQCcoj=Y9&)Dk5b)-+y~V9R2~WXttkUQPO5del~H=e0(Ad5e2Gg{Q6;b1bn~G^;7S ze0f8|!d?J*=UrXd-#N;c;}tYh6k4$84B>~e%bHDoJ0Y#)#`F2jACmM6zDjKhL|Bj^ z=Etp9Ls9~VuTse!ClwwY#RP6p$~PkAQiaIRpZhu*Fy$+zWg6k52dvd-wB!V{fFU@DVBAEdMFIg{p2{D|4e?`Z^Qhakx@Xjw;VBAmY(2HW}l z2E|A1+T7p(LOP)J%plEdTTyRjuCV?LQ0Cxv6NY`RWp~}vR+m4XanyiK`euF%_}ZeM zjq(|7nQ|C>r(0m5Vw(3*S4dk3GkP~SqGlH-Lonhw%I?qUt#Mdq(*FfLd#WVT`xA$h zCN~YH-gN}e zgn~%ti}h?-l_~zc6d*!4(zjXaQebse13 z0y{GpNI$Gs{w{ChIX$%CqWV=Uy0uelqKusLEq3mN+ z5{D-srKv@f+0mva=(FFAnP(&n8{z8yW5CKW)ahwTqugE@8`K_Q_Pd$=#?3iiloX1C zDNi=&0{V6n|HXqmy3^>3;nC^HyY>vdKP`L1_tCOW+dscFPGNa{L&{&W`9kAPlZ7M5 zO&qt*?N8_gNSdPN+WL5Jzkv{ zi9YLn=&KdK=$A-cf9~IJ$i!8XtfF`JMYd4rG`n4QqM#TL_2wt2o^40XXo%EBQ)%Os{LH-)z7j*-c=rj6&3f$eau_jbTTI z@$Zr|jWWxvNuo7}nPJUbx zjpsqzknkmzad=NWv(JE!4F&OJE0PK0Tc?J%XjIMm9J_~^Na*n^>kcgDW1s0}Cs%xA z{(wrMvX!Waw^52xW5{HYGC^&isJS3!99!qd5t8=ZD27 zJsdIL&ViQB3n(m`e=Nz+_U}U)$7gg9ANzv{(=|beR!icToJ6SUhwl zYaG&?@>TVkT{~nq^%(`BcH!ZZE%D&fgS$v?_N%pb$2MI*GvF_EEzJ`%I`B*{-M=Gc zG>zXfakiS!4Ww269<}3^&t&6?T5Vt^O^Uobb4YPL?`iLf8Y-!Wd))FyB88K$WMcs< z&V3{8yb7%=905mAB*IVx8Ou3PNb(*e)FxXSlIPvfk>FL^HQ?_~GHFYIHI}2G?O`$X z!Ka^p)Wy47hp#N)p7SVkimcay906$yv_pD=MHR0bq@Zq&p{x-K)E zQfq1DugrY>D3WSCAdJ^)x`;o$r}rqP=cs=oeng<%E@mA1?cF6N%@daFHm#=dmK!1B z&Q0oQvgeDUmld${f?@@f$zMjd)kp|m8f<=3GAd~`*CmkNOo(yL)FgGUn9m{aN1$7$ zb&Uvh+*3ABcC$?iU-OilIrTpY;4(5gATp%q>6d;Rv23?lgJ+~7)eK{onu~HDt8+94 z3m9FDMxMFV#=hXwNS=myt0xi|VH;Fsu%QyFGiVqzr5y5&yxtymq^$TDm7v&EBV7B!3x;;mH7?PdKWsbMk zez;&aL76>^nVV|j(c9n?gM+u$fo2s!A3P35dmN)_xv>|zIS+!aPQPQpzIzXQ@A1A2 z?!fc4S5J)SerY!+Jjbryo1B{0as@I*q?iH4(6~&h3=ysb?+r(=I-_tJv)@ov3)=gC ztNbg9hf8WB;>eu5+%JoFt2cereP%B}A$YMn_%>g~KQl)bYjC(08LKLnp9hU$FbICJ zF31M$AunEXD#L$2*ea+rUq2RRqH(Q{2no^gT=F^7KyhXorUX4rG0IMl>#tr} z_eMg#5f61@-3A)sMTSeX|ZUnT%e2Z3h{F2r2k}PvIr_9VPu-5ZJBCS8j$FhfT&}=7|bAWwOY3PGo z&3a~=A-o%Exm+F1czucNlpKS*TsCOc`^~*7mkQevEa2C6w}tu$X`i6U&0J6M0xkJ{ zn4KzfAT-*^Z!*}gweq2a8-P!UG@e8Eir3d&2P|$Rr|rA5Sod=yiEc+}Jzpk8+ZbUq zS_c(W5@wkhGr>OC5?Y}!xwIQH+)!+1uA@BnbKePLV;L{IN(h!xd5q8`?rn+l8^JT; zRP!)%=%lQy+KXmB4{y)c>yp&op!NC+S)f|J3BvU{F>aCely0|YC7Ao(Bgh&B{jOW! z3Vg=qe#29+XE8MzdCos)pjM9z`&>_k^&9~8lLHpG>XNe>%7xO16goA&ys;9ow^VMI z+hKXv2RmxB%FCG66gv`!Xtf??>0p=1e)o;GNZy zI_^%(gcHp1m%Y!nrm$i?weu;ac}yH!Xn$(}v222*;MSe4|DI0@^uy0k7VkRNsTR7|56Ve!#1Zd-f^;Lfv=|Me z(eVJNf*A|6Q#)zbGYvyu?R!yvy;t(Ru1dgrkY0PcH-4jv+Q})njg-AdTFEzgypdR< z@IyH^1@Y2lwV29=v0mE0=m`e>7+9@dva|RlKpt*u+;r$0~$}A0!FHE~lLbfAWZ> zZCQZq$eEI;{50ErkT~&=I&#||HbRB}aRn0RN67p%@SJP%+NtSpD!8dxc0$LkgqqIs z+9_T&GcVr8!Mm(h*MCTSk^g@E@Q&5Z0(cOBTxs=}u} z>wYLyGa);J{jASHal|4V}W3q*7r#0SBg@V;K(S2T~4{kJYyOp;n{|H?58zj`~bCD2_M|F0K%zJ$9LS1iWA{`xOJ^%n+eM$;X(fB)lOOWVZ&?&HDs z;|sBW)cS-}m_0!wX50zN|602IbYDJPG(kE4ieIpUU<^6LI+q;!s|Xg*h)hPNHM=zqR_p7xcVcl>z=R zQtd2BuN)NJ{$S|19N(JawXKz+JIxSJ1D6c*$U-BJe zXS92`c^ouwcnpYH9I&sSb$n&2HP-S@NEjOvOGM1?-y z9oB=Ns0UMbA0S=4Kn#&{?k4gbDg1c} zrnNZ_lRxXRyXIHPml-cmR1~^j#TpM5yvVr#q@;&0+`0nMcH>nPWB^+YIS+tsF!Pvd zA>@NtKc#@D)|)lak_!Pjzhi*06#w`GPoqA2q#$gk#0WqwQl4-STbJ+6b=yBM#t6KY zoX6!CE2CF@p+shR*Dc!uyXx(*&oCf2D&+}Su-(77c*A#5YUfQ~DeIETVWAA$WcEz~ zOkTQRhqG`W07@7SrOTY5Y8IG>e*7;OI1r3_VIg{b!mcvr>Fe#=w9-^CHXfJ%qi3F%^57>H? zw&J%;&FhW$$X3|3u4i&oD6bvX&+(VmbX8fzNZ=0vtNi)Fkmr%?coFEk4B*%0$_t<~ zV^5h+aR+1D%fRs1g%aDPA0iF{lG>@b$)@v{R!#~E&UylJSd2su@hiiyc|v|+zSMKm=iTGF zE)d`(Nu2`wY~^(v0c^x!BA`i~egSlCbG7K@F>tt?sit8NWW34O+y8J_znGcnb{0AT zWDz|j&y)N@-^&Ex6KXUqz%ngv(zDNhLzuE^hi~t8@E!c zOZsKM*Wk%S`KW=tFF621k%i?kS>}wRhV{sSDIn=RF#t#y+%IqUQGZ6$@RyK}R~Z4H zf!fHVHt^7$9)ZVioXX*#to0O{z;a_xoK(}c!OtwIK~E&mGS3;PZ)UL+HfOXn%i

&Qv*fZdIIkI`?_Vz>O>alEzWuK zR_94yJ84CM2RSWB%8+|hm$6NnOXyJomk(UY_2EN7`kaDJgPq zjwjhg-DGz{n4I2hZ|#&8{eHkAyVn@lrmEPdJKkHzFqEbWlcDWqtr2`b^1*vVlk*qJ zjig%xAR=B(%7O)*2pcGN?yG)UIgeLV3QF{u*hi{&SJZ_=ajOHo0p+ zfS(f5!T*=hf42(*cBmJcqGql~>eH%H#qWb7shwnBk|j^+7{qY6??rIzoyzPsS|%|G zmG-BfBn6KVp*X*tlkkt5ZrS?aZlY;V*ReW*Y>{T$WZ|(9Bju=hO|prTjH(X@1fI8a zc|-y;`1{76^7b!66}>3Q>1wL!q~qgTVUy(wPlC!$(N2Iu^M;X_KihDLIiNv%rF(zl z0PsFrZ6Hj4JcO8aL&Pv`VZFP#>8-`UCngWjG~btbMc*)5wz4M+hAFGh z`!_y;8oR(mX(dSZ)HB<<=zIqLl1885fI7VWU;H;g7Oxw;0GwhFA(j)w2fpoBcVw|{ zK{Rhc`?Zk7f{fO6kjy>z{w*#ynm82#`&7C2p%qaY;D$h1fA}{3>;yXkkQ7$feIls@ z5#M+6Jtb-v>s=P*@!!+bM!KdTd3^*SheMAE)1ED0%ibj8iflXqI`gQ4>+=nVYsp~0 z^cD52$dHe*?LhlYGeIT(ZhX~vHAG=g(Uu$;y(YO-Z=1t~toC5E=411lBKF4RzSh{7zMO3@c zZc)p0-NLaq4C3TD`-Xc*0T}MCr8=MA+u2wh@iSeEc`*M zji6l`G8v#RqCn$>qW}x3f2v6ntdEte;q2K`|mL6|*qXFcVDrn?&`&@3je2 zll#>5M#bBc>#}s}9bmM`axKo>mW2cY;#hDevY-U}(6H+d(XLG`>onX1_B^o{05m9w z;-;3?nq`8lBYO5kf6}q}zqCQ!ihd?iH7>UUvI{1MxbZ$E&pA@sk8GWoeBIt>ALeJT zKL@VK-p)=f<8kb{oCeav;q8o?bP5&C0I~Xik_o3!>7su2ZAJ3SJL%UOAkCCSIjtWA z7c-hT9Z&WGy$CtaWIFQX5z+(875rt;)}IKL_ajRJLOAAcTGBm_8Z7`lU0J}?&aJH; z*f1H(`2f9E@I(sk0^;sNov+&{`)XtxHWRhCNI!?>Q4Ev3;P|HND=qxC;u0Puj++%} z_Zgct5{i{JezN z)lXlp%m}qkgag@~rme3kwn0r=z1hPX3m;m;k)xjGA0EFq>svDmV=AGojd3H|+|dUb zGnK{02qKPVInQErk0(Zd^Uj*YzTQzpyzm8>nBA}YcG79>hUXrYiS3NB(jGpr@hnq; z3VXly5lvo$9`sSVCRr3}MrT=n@nISpsNv`eZm=(9tM_vKHt1_ctTAKlgtG1f-*XtS z(GZBz1|N)SFP$YW2x=n{9LgG#{K?NHfbVvdu;c4d=ipmT1l8K1l?S@-jO4g8WL6hC z-}h2Q$vrVmTiTS8!T5UStLX!)-5SSj`$x_vApz`Ix)Fkp-G*t#^LKHuJ76Tl9r9?K zlcik+Wq6IJElvEWu86VLW)OpMf$?`%EQWZC0*6GS>Zns#<^f4uvsohV%pYf;|FdyP z5RVw*Eq1_ zn6&04$Ep$wKEd_Nv1;F+b~q06)rq+!ICzmfQQdD_uh>P48ZiC@_>pK75`YvD)H4V= z(}d+e83%VNB-LHov_xvp`AtM+H_F5osvn*7d(DUOTE}!~3y4tqfEJ~dk&~pi7I2ey z+hw+}Bj+S;Dv5s{&R}ET1nHpO9H4edXrgsL;Y(#k)?3Nxgav^^M`6ZYMhwH z$OnA$qI`$5_@l>XN~BqR&Sg$}&9-rG*z{S~pD1mGGKQ9!A0M!`#pYw@6{oB_9biPlRc~5k8>)y{LjmYSI!{hM>RBL1u4&!C; zX#)CJPRsa?i{P-GG2JFYSWesWg)mz%gPe?NZMTm`vg0q(WBd+%S)nH`(q%Sxa#}+m zE7_c^nS;}&lGz!JhgE;5hMvfzIPK~;xi%Xa%=~)FUokp}zqxL2R0b{Gs<~A`e8z!; z)(HnfxtDnSpAX{!=BJ%Sy?O0Er}NGVM;Yu6Pfv=7#aN&B3=1p&_^{dXFLsC}?|u>8 z*cU*@bkd@%X740O%aGx1G$C60K^tlr4V^#buhTo(a6}%djDwFds(Z}$Qu;JMujSH2 zAg(3&^thY+xakw+xh;X>vv-VO5xYD;JKJ3mXhb7#YblMwjK`g9 z#3=#^5G>*fetcz0Kq3;U6o%eV*CJ{md=G?vY^7H91dK|&A3D(#MR54@$Pc-d-%+qb zXI6`oqC8GZ^;+(}3r)Atctmo%$gW)%;62_)YuZD|hkzyrS4O;TWWMIMSlOVxN=S-6 z297vy8=t9WxBOa@-LT?EWujRTL9{dqb;o@*hnG;}b74d-Pmt~_GR~cm*BqRbJ5QrN1zp7t-Zg)F+j5h@9le{YL1A#!M3V6YtrRkz+-fulJ0xb>5Z3S zcsR>lG*5dzgB~_`ouy_yyr)HA6?}&?xsctN2@8S8T)s%|>B1#C%~D*wKJ3MdY1-Y? zGf3}LNvyIA9+A%q9A#{hs$7B2`!LUXd=FhdT%E6H@HZx3+}pW2dNUH1x@T2OP5UmS zisk3H?P74sM@3egH(9;>T3A-8U z`_-T!ED{!$;WQQEtvS&_!1gjj(PAcJBEKoeODQ&Xma*CBNuj$|w1XHmI>JafT&}29tX!UxtTr6MXm>=XkGMoSdpG z+G@y)PhwftEuQ<`;2y{}=tq`}@im!AET_-pydu1le|`XQdqh$67w;@wFFB=KDsNZ| zYuV5C-DqWOCU%>#4|sW=-sA(2!?P}V90WSg9&=>>VCb?5-0vJbPCOt@-Z^y_Rpt1W zuEaY3YBsehuZAQE#tAq{F@ZK?$-FM|G+WKqMO5FG_^yuvnVvpK zHIXFc;ewC3gmR&mYg9Bh#_8Ed#xr$TUrFg3 zI^$XBG9OQ_`Y63Zb7;^*(i&JU(HI#SnRq3k%g=K)Fs0oa9ysdj*!g`DU)wh3Fe7&= z()OrnzHAoS`KNUT5MS?q3VHuO%_Bh62#9QI`}B40ez6Hw|FfH6H}=dyxfbtw=jPpR zD4L2&Lbpk*)JsA(D$?_&6!j*)EC`&wEeA0Y1P3qx@mi+oP_NWJB*J!?@R=FD>? z^;vwLaB`1cvV+FyOwF&<0kO%|L%j6N_!EDo=ELp=J@N4u!gXyPHL)BCUT>0~UZD8( z(mzZV=*k{DeN5CD_hi!&3M(P)k)Hfu%lVrUO@KBFhej%r(W^26$Moj+>A;RqTR{0% z{WiX8OVsuvhlhoA+h7o&koCLznZFb){IP;~G!w#rG5UtLAOduL08Y=it(Bb}Y6D@h zg!)__a(s1<+)(P;Hs5O6FG<5yhm0$<u^G)o8Z@1^M|fE3 zx)g_B44ItLb-Mg$YWZUErSxG%U4R<+-dtT_T>kbjzgd&4(8@Z7x@Ny%@xe5}g>S-3 zt=+b#nwoCOqBzM8e)LZ0(&KQirWss5zju9b@l&tBr=`)So$pSWdAiJ}Rqj^zKYK%D z1V2IfooDY0`Cx0oV_;i%S^>S*PuFsn)66EWk;Y7X&Yxf#W%T*ivwdk%eXNQ-k@rgY zz9rdD(u`qWR_-F3k7S2cdzRERd>Z#Obhdqw?7MEZ$EYS92685Si}YJ-)s8=2=#{NA zcoF>X$oNh<>V>-~f$;sYAKLEPe9sSU^0Z|9zc^TI8qX*#vg8vL=wEuTW>mc8YxllW znTP+C_INLP&$YBw&zgA3pn)3U2@&nRZ>eF!y9eO;u@DGwC zviZ-1^0XqgiLKDc?!atAeiZy?6Z5~Mx>$TEN6dU*o4^ChBZ~fa6u#rJ-j#OcGrl^Z zAR1a4dQ}*MKf>$q?5fbW{cB9U-`|>lu9>X+2!yb>O4KsLv}`{A#JAdyY&O^Z1QNj) zk?44dY=Zr2;{6>CV&7p)oZO9!EyWvqC!;p^%#EeH$Le%N{@wri9}hU7>K2|9R{W!Y6alTN%BVc?Z|z6D1XcaL*ZY4Im`DlxmmSO$15NoKU*Z4!e~SmyuDq&Z@Q(sMz?{^i zM~^Z5%m2M&;liidUu3=bN5L$1bT&QTmF3^umxVzeC4kyhZ&Wq@uVbn(s6P_ul-Wx4 zKYueorTp|BsQpJ*LDxSDAV8<|N49F=|KE1J@>71Q9ZwMN9|cuZ(TTnJ^|p(a#9X!) z@=!xoSs9s1z#CH=u)$>f;IcjEc)m3#sF?8n^G9no6N>`nA0KI$ivQlXl+h(1ocB9` zaFQk9fnGgXsy!XfF0NB;I#>c2^i7q6+cVgj?v9#Em2xGYyel)d!d=e+MTe?3?>nvS zp^Vm$h$N}$Cp+r@{g>T>E&VfFyWOXCvQVW5@HO6&(1k$i0EwwVK)k`ARJ*-}s+KGH5_Zg~SM$SFRaFOoXJFMEv9B6hTB%^RKDNQh zV)eWI5KhqI7X+k|&&6tOUN;GGaYm@Uz@2KfdR%syw1)j{d~_&&?HJ-JJ6yG;gwBWs zyWIgYTo3zz{pvxI2xNEcH_#kocGH21fdRSYGr&vr0FY6s1{}c;DpVZ--5$N4FG;vZ z08KRgiAuxYVWcGzJD11H%YePLC7>L}3d)TAzr*nh>U}&aHHTj)h;F|`#jLw~?p$C~Y0bCIa4$oeHR4CMG zaxSdQIs$@T=bNl)Kfy|#yEr3^AKv_ZAeo2)k0X){$3-mpv>z)t$KGS}{`TB^B2vgk zSo~X{y>_*!JUokIc!vQ-Ru_JpU&a3qY4y0qLlu1JWk=CT~Q(qLWTNUEyiiVip`M2 zmQT5)a}+r0tsXA>fH`J695~yVS!i^6V3SiQw4SdkRk{KqQdi%jHdBBTzM9kZ9%Ag) zaOO6*0^c)zz=jfr<3KYL3+8w94wl1mDS^A`YO%;8xDutA$s_{5- z_bu&lpHsGn05QUs%s(reQECB>8i>o^wCjg}P&j$9YT+2>;f|OAqISJ)EifSgGH(wrmKV^~ z&C$2pv}!GbEWv=y@gd;xr&g}lDtZmP{>EV6hgwqLTsP&gS93eMzq3E-M7d+|TGnI&%xI0~>jobN|0oJpQcUdZ!|#smfZ4C_kt{CBmlzPH zdX>g%^p)A^{hInYL9|YK=?3)z`Vayf-T!X>d|<`zd;_RgS%AGxB`W_=i$PJyy++#Z z&;A6auo?bB6}!hzxUr4=>q9?qq{`_D-1p%vL1sO0)ZPsTgbBBpy^XB_Z&eFlKqc=0 zkk_jPqyzsPwLkhaU(Hz^0>{K>pM`EW;;w)yW`U*l_)i9)%m!aZ*n`b4!%YE+uPk!r zU;LpC&A{|~?@5O7i2%zVa@`qKF5Uj$<(9$^6w4`EZ}JBp*a)QS0y=Y7PjeLzojvzr z5ZAG9xOwv)m)R2VkCuEih65W?)+N`Sk2UPwziu>L@PVN2b30=XIS84ggQD?qi&UF!OvdiZLg zAu=w)oVJVAZi%0q^v!pX8cHgvZuHDopWnnHkbl?OkiuZa7Z%6b`T$lHzgEx>@;B$%A6lsiU;NxQhEXqH5 zQ#Rw+k;kV4RhkCeg0y!Q6 ztWgC{ybj*TGX{R?*q5(&@ZsmTJa0cBb4;9dyR+M%SvK&-`AtAujnlHsZ+{0ERTYJW ztl&K{^h#k#gPo#zmJ>M@9iU3&W6HapG6lPu(9-|t+uNZ@F%V@v(uISvBWK@N9VgXK z4{qfU7ieYXnQn`rDAoe5$a^;_fC`r-pwCpK`^-Duaj?4Staocfuu84_^5Tzow+j;` zbXb=FR-&@tR^B|sgZk*O(I*hamV2wq``8tg!q@AUbkDV#0jtiI7ea~3zZDllvcnufbtGM~e^snb0 zlCOY4z!4oSyaH_N%D}^(>ELGH|~#*fSWhX+f7wW8nt~=y<-B+ySIn=a;^) z(T)OnL{=E?ULXhKF|RK8UU&Sb}%Y#>Co|m^XKQ=6xNkn&o55 z&Tse%y>;d4XEc_E9?2O53A~P_BXl!g+38%v^Ibf0nKnCd#RpI_e1>|u*ta_CzfoxB zC8aU8%UeCpC^mSMSs>%H<=maza7)-q&9eTGoTsv#n}DYoYx$n!7B#uokN)U0&cVAA z-EQGf@B)Dy5m?&*sE*;}Z)6Z;zegRkEu(y}W#uG_V~4DgCjkrh4L1s)7Z^6`GS6)~ z;}7^BD#wRz<#8hsj*g>jD%Z4rVA}cSY)?_O?^&JN=J8wmDeQ-^Ho(Un$HGniz)wCs;OJkt3mi^BmKYkO;h@#xnu1h7b#asbZP!d>?^oc zR>S;KU6L;rY;Tcy+j$VfQ|UB57O9!BGfigZ=UYQdKc|0C!xO!51uh-vF@Qr#E1}6tYUk8rTbtO@m!&{>=?jN=yEX@Y3RA~pc`|=! zjwDL-tL5FR(lm&WUtl57qN%H(K86vYCiEg$HYP+Id>E=9ye~J5gxs&Ro#eLdsDPgq zmm5K{12bg&M<5-~!uqWhART6d`uMzGdn~{5VsEvc73aSBO8K>6kXlL^)HOUdfYT9X zJIHw1^Ot#d1+W-Go%z?)E}{KgviXX@iuCHVh`8(7qH^mPI;7C@i&FMtJcO1mn=0e@ zEezMihF^Tp4PXQ5x5@EseQ&Nd12fO2j$1?So@Y#rO*Z47LOx(mg$UF8VugLL^!0!` zU|Zk>NmV{+A@Z5sBRl%;keUEFwb(M1C@J+yDYiiuyj0r^qhm*TRBXP`ZX0HYf9)!A zz|&b3eVR(At`FyDu%la9bhpHGOnDAreFHJeIEZrbIYu~NOp!vK?Bpia|&HP)=W)1fK)ENyOm3rxs!5R`%i2JoV;lQ-;j+k0cG&hT4#kJ zVIUh`tw?K!x@$9StJpY!$!ZN2l&9g;K&AFA()FqHuteuOF6CY^s*Fqm(b6c10-KTy zQ%^ABGePK8yQ|D_6F*ie_cU`CEQs#47COLNE+2mTNRW5m^(ItLZcMIaFmOwtHrSSt zK#gXAhWn~EF*$q@N7F95uZ#@wx(3F+@t13o+a3(T%*eZ!C0vy2mcx%|-JHNe$<$V8 z#%*vd@-hZWVB2Jtf}KP*Ytr7A%RjMO8o9I=3zA|Zz9rG$U?xG%7F;OMq>Wb>0UHgX zv!KsHU?9e`bj29f;uoc$;!Dulr)9r3Wb>dCtnDN3lGKllE#ugpED^h0Q z_fahemp3UJlA#HL5>{Gu@Jwq4!EJL5gXASlS2mghbS^)Pr1|Ju=muwf^_RCn41}-c zCU@vQYyM*hw+SJqh6eQ+MZ+W%o8k~6#r_ePcO8c zj!9WZBGYqH?7as@(_&Ugq+=*K$jcGq1Ac1Ap+Eax_X4biq6Z}6DTQPrf7vY zz7;yEV3>}-vz_`$Xp|4(Sq-o9EtLyo+4>k8m{+OFs~{HLnNv3GR{}A7JDPpr$ot5V zYw;(my5;!hFi`$e&4c_A;4B{|UlDBuLVfA`v<%UX8Xnq*mp~*4Xi!h`IK<%Bv#sj} zXE|x?=UA;+Ta}{(w+k3gTyYM$__{bKGiM-&Ye79fc(xv|(@IAbot4AQJZCtT$Pob&_|c0VlUm^!N!y&C zi4$^+$%XGE3-gUtgd~(plDW9e`wQa14KN}3ikDuI=XUmhvw66~#LaSl3~%B*e;@dX zTnYat00FxLcyLZ))#|HxAAj^*?x{?dAQ$F)p<(8q5f3lG^fCdQdUnhlUu?B@D-J^Lpph2 z**08)hgjT=t$3>pt_t-}$B!F*BG*{ZG8;ornr%;71NI(9q|-Sq|;fPXZSAsa7Gf8ZH%-=)Wltt zu)5wCMR#matun=yJx(%licQvn#R>w&lLPZb_GcAZf{)G7Fa&DPNWba3_G8@bef4@! zz4KJlaS#uN)?<12CC;xAM$KqZRk2gu1{!C{j^kG&in=r!M(gxNTW9i%bfua5i+Zz$ z!>b;T4y*L=$>Uo>Vli9%SWGnAhk?X;#gPC6=n>$6DG1U@LI@N9$2Mh^frNiR{1gU( z^eot7i))_q;)8&J?ZGY?*S@PHfiT-MC?P_OCkg>!OT(8UBYoba5l8l)yf#g%caC|n zS`ZyL_ zcsyg1e2wTeAQm5#euKI^xrL6GDS~#@J>FKWE<51hQ zXXj96xht_?J-igUUt2hR2Ihw(CgT*-_zA56>n8?ZSY?p+)!SX;;okdaC75L-=PJ8F zd%T_#REJ7g{&0?migFjEO{eNKu#))qo?cJ>3OcCL#)qrs-Zzdo$I7Kuyn4ykZH}#Q z1*_LYf4vf4Jl43ndhSQ1*h5a^*2v-SBahsnC?dqNfO|R2;G?OD6Ag8wwLelm$adOj zkdt-o7ATr0niJRS?CKSinV2z`UI(6}$2EYBF``^*n+lVIlRo8?D8<3xiI$<3$p_s% zvZ4*EOi~0zG?mvY8OOz>Fo>YQr02Ax8|ZaF4?ndQ_Jo!q95F4uTi|EBk)!g4O7E*7 z*t&_M)$0pHNKK73vMSINAP6LU9mQkFmyFq2<={j(hq9dd zNk@X*;=(jtcwVZ}vlel6DDhLL#LpblM5A)9=z~@DX47FcgU^?UUr+AloToeNQQicP zLbLn7YxvpHG(|e|NSX)OAUU=|a?^cUmC?c_QJgr}ZD~jpw@i;#YsPj{bzD*njSBjN zH=HBYIX_%>Y`@vP6;zj_QaV0FITrZsOZfQgac+N8ce*LqcVr+e0EmI(ZyuL$pn zo*$jhN##|;l*b1fPpKR7?l&x)=zYZ-Zy`FmXS9Osu_!K48qKJ2;!a}WdhxEfD`(W> z?K@6;{m|4rl{sutL@JF|oqJ*30ncP2S4Gyx_$4iF*>R&fUj1553Ju8V5l7cS;>s{p zak>|($GX_mI&YI`tu*4=X3$onx#3Hs7k(gGbFlz*{kGd4)6$co@2+aq;Y;1+x}5L7 zz7~6Lu{@&UKOs7wgiDpcii3hMmq}}WO-u`zNh~ptsFRV5K;QN zZkszZ!=WZu;du`;eznk0qfBo99nTDWBQ%j=Ns3KV=Veq8zXra)y6jOVcJH3T%!Qv} zg}i25brVe?Wjs0TiIoeF=RLCd=CHk=sHnEhx2{ti zE8`N0qx0}gL7R>lL-$?xvNUa{-R_}zm9oIPXi@4 z99jzUuw6jPh>XXKZy9kq`uXhj)Oha_kCNrtkq^XZZc7A^CN{S8knA;KL$#0uhm zxq5o#JdwZ2icY+NGPTH!8OzJB@WV%6_bH;vgv1jd*{c7mbsSxL?$_Jt!S))w)*iDtz+V=x+So zV-jy$x)RSM*7u(@Y+f!T7ewrSI}=@?pxeq_a4#fBY*eVEcNB-x@xXD6iZnSshD`Fj z`?eXBKKrh#{z6CNnKzP$_iBy%wDY|*$o~j%TcZF)$JfDd@5ftuB0N-OB0PB}b837c z*uBDY+bs0C^=${2GJ7|T(;FI;9BgXdNhjGh#Nu|(OR=ji-2U^N1B6emE(>h@YM1-_ z2me8Ro__@}j|7J&T7Q1^Pf{-as6ZbS6UFj7pODRPwdYKyrpMoCeXbBXRw$p+VcABU4iRD`{!s&%2>TFPB~?X|L@1-G{^N zvUiPU?H*M*Pv6;{s=JQsh`x7CTU6ig-)*LXcowmQ7KmRDCm>9w+ws`kL#qjs? zBo$+8j@u&lUTA)&sENMQ|GlvyY=LO{S_Ae7h1{p7?Z0{Qrpo)je?L~QOQ?ys+}(Ho zsG7kXT77b3JI`!E#~(AOLn06juYosuKB)PfaTEZGxFX`~N0Y4oj_3Rl*tzKk%(j=~ zWiMubD>;(}Ox)v{#}*s@eK*x@K_Dr>uML&>8)0wY3~=mce5mtxf4kFclIrVQ;y9SIa72Dcxh{(ki}}| GLH_|(h*d`b diff --git a/images/adb/instance-principal-3.png b/images/adb/instance-principal-3.png index 602d209a804a8df994162a4b3236bd3ace6257ca..bc1a3160c0ffc053aa74803bc77b26112c699377 100644 GIT binary patch literal 47182 zcmd?RWmr_-+cr#x#1JYCLyJgD*U$|DDo73`A|28_bPf&DEuyq^3_XC-3eqVpokKj^ z`~HW&=l%M8dpV9_Z}whmueGkd)^(lhI?o~UnWhraJ?eXCXlO(#%JMpBXc$$%`8FO7 z@Y`V6ofr)bUB*UE?wN|59Qc`w<7*o`D>Sr62^LT&naYz#NHa4i6#0#ly1>61JBP9GZaeH@1JB&b73?L59 zukP+T8t(4yJg6`+kGb*|9MMEFO`bfyM}Gm>^re-)%4-b`w8y|X9vXVM4H_13h7NqF zfe#uQW?~o`E^s6RKJs}O|GtY+m52H7bF}S01!Z*QR8)YYu7!)0m4oXWN4N4php)h+ ze%L(MchlEU7qf7*=Qgu+G`HgRw0HVb1x>uQnKjr3SlDY>5gC$%nUyJF;EB>oG@Ruah8#gy6F&-Wd z4-ak+0d7YZYaTvPQBfXVeja{)E}#ULtCxeDnJ1TnEA#(8`v2k*mvJV1PV-?(pz&^YZ-X*+5l^KUc+`*?3yn z>C4;L1LgtRkP;FRk@%nT|DQYmZSj9=zHqg2k#n>MD!NJi*S`O%{P)fOyW;;e`Rcz- z@(VouyUD-q{Hvw}&!3V1Wr_ba^Z#50>@0Opg6BW)OzNILZgnjh8U#&6Ugo(c`ffI^ zZ?Xb;A1($yxXnXW$y>W>G@&3k>a>>g>|Oy>q1po52V1){DmeI2L0~~d;3H_E-d0a4?)UUF$hH_^Ns(u{HH1vH9B(@RkN46$p1e}yU3d;4|_ej zpD5{brhK;hlhv@^PCJQ(g=JH6rRZIruawHV-v?AUC+J;i3^XD#54~LNv(ui!=}3 z)H}=@q;eUiK2MgPO5-u*?zp>c@eV5VlG3X%;&A=-RprRP;H!>jjq8Ty<$9WFk^OAF zPWel&(;nGm>B}|prpsTMR(msFRGmO6T<_9_>}T=|e)|1$(%@m~tOv2iKYsBMSz+9g z`~l~9uGOnHhFVZ{FqMm@l~&56+C0>5x{A?v7~K%vWi4X3n8RYVFE(-oMIIq=K2>SD z&%=@5x}IvdRy!gk9;i=_ADr=Azrn#+r-95kysF-IR@0$z8He>CIb7+BcA?@?AB`hR zROWb+vR&|&-e-dpmWQ35?sk8zFGO_==(n|`JkM_cB+ zGgZmMp5-LV#3XL}<4KubWzL|{X9K~ElN}b;CAHKihrM(MZQ^F<=u=S?JQa;j7GKl( z#v7ei<(+@G>`RrssC`*rdy7?hrox_GA>X_Oomw zlIK|qVT34qB9eJ7!}h3kaw3o!yW+5334QI+c!vYj(YPrOPZztoqRh;&e``io7 z_WI&bm6-7h$-2+)CZ(hm)-YagEhkI=tX04EyIa)h>b~wU0Z%LZu{nMK#NV%34APFSc)5cc1ztA+_9Tmg!{TQ{MZrhZ?%X&8N?A99I-b2@mPSLEj{2+B??w;r!Y%mStN3YC|; zLc7l!@djqwPzWo(BV@L?M|vw+nrp;U6CI?>n$Fz-doHaY&p4U;?sPuARlUeFHmieY zv!5FlKF+cuSXh^VNvF%hJ&P3_v=g$>=A%|=(pKbsy50OOn2Th-`5l4c7mI)Nnh+yg z16tYii<-4!m%{J#yI#sy#e)7Lv5@kByW8gb!+s{UDbJC)(jAFCOMVN_5jB$McsbKo zB5-A?v98Vq3d1UBC+31b#F*(A^h-_DQ^j_wg0B554ZhUH5+>EtH}J%(H(Nh{k#H?3 zge52Dcwa5yi~4wW++NwXg?rG92vqI{`)&**^(!?v{-a-57-b8XJ7Z>-+2NnSAK;b1 zs{OurFQB&nG|P1uKS|HwYo@Rk!`x9q@LCdXji&-dW!DjUR~Y-0BjdpoXoDs=q9Nro zZf65|Rhp2VQN?{LBK%&?T~q9rAD-o7wtfw`K2&;cPWZa@Va(@;aC<3|^=6ggUFqr3bK z?baSs;t}@Gn_JZWdK6~`yg(s(yM0`ZyGj1MMs|9-bg<>6*NX?UMAPMiQZFLRTx6fM z7p;(&`k~I6jnS#Rc%#mzj-sll@C6c8Q`g>E!aS3Bd<+=tJ{x=)Z(fo)oh4!tfaY?h zSKS1M4}i}n)6Tb62CPt6p!e04l<|Cqg43Yrsi}Vem{&wWlnos<9;%!~Wh*J#w=Z7x zMp5WGKHir*#d=7FOUgp*qyFk@0XK0;N1(dU^|+d7kgqBdNv=gvTglFA!J%+(;TvZkEd+$MRbjqF@#8 zh5KlciVh2poeVN3PVCWIiDQmDS@kUu_nA-`F5gz0I@h?47JqH@S0-PH_K%UvQBRDS}2{vvm+-;l!F%qC(6NEJ(rQE{iGma zsxExi<>`L_9J<_66)EzFUOygO*?Dz3W%_Ow(?{EjEA1a{f-op12r`OkizQ@H7pY?Q zS0oH%bRxkD#yYDqEqn)e*-ibw0pu?{${-C_WiGkD*9l?Ji97<&&bz?Y>u(!C@rpz- z6rmHeIzyv>OWoIqc@QE z%up<$Z;W3E|7p8%asaGuMOLia|6^S6|5sgC<_jVUmEGbPFd{20mh^L%pVb!OgtQ_D@IxJPMvk94NqVoB)=cu?HS9&7(-)%paxuX$wcoJ}XX?;PDD;G|jGLXd1-t7J%Ni$n)Af3;G_`zP) zZ)wzS?GfOU=w8xqWnYXbF{Bs>+H%w4vk#^5OgZwNFL#5alYz+cJ(q2# z#d;)XAS&%g!h+WZU521->T93nPzp9qA2&QnwaSBEnT~@11Y#Xe1wgN)!CI1*6vt|R zyw=1@=W;YpJN!lY%K^a6rhIDk*qdSXSoHI#St+qDD$Vsb@IL>Y{Q+;c3&1z`e@w{> zGi#P;WR^HHbF-sP_8Jy{G$!n&>=G8p?|v^&`()Z-vOnkCPbcg#Xs2f9KD zx-WfC#+2U23pd}P7Vp-kOaoptxoiw`@$vB;{j|<4QwJO>84k#F0VI75X3zDlfeUR` zhXyV>brZc!7IIwpWSQPCXtv+nWvbw4bRON+8@xKA?Z_Zs;dXDP*psiGskyrAW6vz5oqEzrcmR3`c|XCWnJ|<>2{6#o`L+kpFIV|A{{^NLHmdTJLKF zCg_7n>&dv9H~qB|4U;f~GS1pzD22(h8dqQ|@b>u$b++e9B>M?%K`8N6U>vFPNYTT& z&hux>@zS^ORa<%&r3nzg!nHQ%dvmny!WhR+)jS33@n<8mpk3 za3Wcyz@ltm3VZLR=$VX3IJO=t=EoOyNZHNSYrGNP9NcC zu!(c?erb=6JYn}|t?eRYq^xDCLk5OZ=09Hdu+U!|gGyE@XL~MWrBT2(=0wOw+~fDB>3ltBXJ>D^pVj4n z*UQ}oQ&7phA=o`L4Y(;jYz&hE3s~e-hA+&e;h^HVM+5Ndi0oyc7@gCRzei0r@%9uh zV~m0I=I6O<&L3|7cA>SY#G0_Mg2Wo*@~QpI^Bs|ZTi@@tS48&FSxChNU~7dR|0Uq= zrlBMo*bJ8_Q1r#plulJeW3{WuwV)7uQ%@){l$qFeFp3aKWtcb zUlBbVCL=g(hPK2_upDdT1aM z0dE1^)>~R$#Ud;4v87pG3{{lE<}S(k<*5zrsCQS$!gplFjP;}uk_<{kPInkQ& zh~M+v|3kxqg_Ak7xPy>JNL|t(iA{s-iW(*aV@%f(Iv+a+_Hl!8d*aDiqdbVA<>e7a0o5vmsN>B;Tj43pycxl zATB1?X$!ldi&8?j;x~!*G^I;M#^XoHcd?2hBB`es@zm|!h~t5U(C14)0*A%^o6`2$ zE$;TcXyVw1$71pJX0n|{sjXnqPVaFFz+-r7L=#!AbPayDmXp%;3C$_MB(Yx0p9g!x zo!y8ekI|1JpVW<5rJ0oQzuR9hM$!8vIo%LVmD_FNhJ>AU{|H2XXKmi)ak~AT+)F6K zlazt3|A)EOC=Z^<6(R97twh2m(FOJ%n-z@jCdofG4$98>Ox@uFGr7a=;~N6M{?|?n zT@l~p&#N&>`ez+FH1H`e3k)}(2+_7#@iS~~9*D`&VI!muy%*cVU{uuuu``ia&RmJn{%qj)hLf8J5{ zO~>iicC_dcRbyBOvcfhNGtrSE-jT%6^3ef9$EFvn?MFwP&8nS^-S1{H_`0>&s+BgM zSl}fIv)!K-&f;B&V_f_)WmIl~N6v3|UrebDk=7pCj*D3zb5wN-{oSwXHW$q3RP-m; zF`rbWMD(B!RTwX=Bd5gP|IB)*-J=O)u{*$-Vs;S7GCD>D2D5#A-ycU+jw#vm84`rC z>}_!{K)!GD{o{*n%+5xJK9zw))|5w(_ml{#N4yB&GJy{+FOFL?uzJhQ!l1lMe6E&x z{nJD-`rF~HP%y6q2{WExPpw&Z*kcM7JWStT8P*?zI`x-Q%n^Oh%&r+LDI*47JPzOK zLhlPcq2v}5$K*2E2DTtJbdNAGVObxe0O1(|1_XNDL7^ z$R*lgOCgGGMss~IWg;len~8wmf@JE`tHO1DX9zoQ#3aLbSXwlCL{M-&Hu>6yR5Jn$ z3ajIT(Vqs{SwxKCRw?EJDX?<>B|ezlvneQqfp;9*<-!UBdmZG{R(51DtNHr2l06u- zs*&A4kE(@iYMhbRu*r%brn}mv(u)k;bGDwF!l3 zQg9f^vx|C`mp?BnunfKuKtgEK2~#+96(`2}PC#r_f3mNwuIt0a>2gm$aKZ$=w=qo~ z4faIGUN32(xjJ?3d;^bwk%+I5_?%0r{o{Ol{7C{bNI>|7C#WW#;=1H6uN-bV^Qi2z zvJ{ckfy@Tu)0D?A#$_^P80fm|M}x55nb+;jzqc|j*C-nOd_OK&E)_msRDLyBpgX|aA)dYfXC!e9vMFU%5=xa1|7y?qB zcCetcuI^Xm?p$2B;OdZOEr+c3~3UJ=Q7h(i#F9$w6 zkw*k=5Rkb0_6NTZK6GO_95#>11pYDhD(eD-NQ6cqD2T$6f``M|`(!i!W+rVwconUj zd|DDIL>Zh0nxr$7Wq7S#)RvySk_FbfI(+}d_NjNbBGLEMJSC)JW_1^)`=m!+?U=|@ zO4|<|u z1n(K;<89hx#<vKy;$Q;0BzZRKADxaOtApAQEPc>T zyj7H3zXg)QuOSf}Nw0)zy(3s)gonO)!%x2b)|~#_M=*!CgiqY!vG=kDl0&FU>4u}k z5bTW*rF z!!aQeBb*y3+Q@WP^|IR9#_UKyxT7L1?*%EM8+SZ#FAs#nk@E>W%Huoz#Ye9(&Zy9j zgWORBi@3mT{>zk&+tH~PjQwKtHT*P=PW-V7crU+-P2fm|;`$|hJq&9$vvrJ=0hxvn z)N+IAGYZ=)GrRj33D!a*Xsw$>*NGfHK2)nB0Y|WMg5lNB0~co^ zT8TcW+%4gS=MtEf%UBwWHL)4$ch}18wEZsL;;0}%)-8P(OmiA`{{g3g`p6c|i8fwp zrGSkeP*>BRULY>A!*AA|_4~Aj@(U;F%3lrE9rFqgk^DywIgENJ@)tXs85xpfkBYc9 zTQ)y5Sg?RWFVPW^^_|>TLPu1q-z;lX+jtreFjCU4pyY4F-Dv1aXxYpcsARm6yyVa7 zO{*0!Gi9!!2m0`|Dg3pSZo!$ZSr;JAxK+63E=^z^_t0PNp~c8a1qyebr$Jl5X;U66 zo=-TGW)WYK_7~^ad9IkTl2oMj7&IQP1h=H6x#CGCiL+S}(2DkA?xOI{{UB^?D=``B zi3as=wj6&pFgnSTOiN#tREZG6L7qkjU3|5#`xK#IeD^5gQjgK%4{F?P^5>-0Sg?lU zqBkin%E}(_FTLmnTlglDZ`ys0;)^KiEhM3$%;S|x z{>E2CR;gAr#zKb^95=)dQp8}2M_(3W;pg{U?E$M%c`m~-Wij|HNqSI$W)^7BJSaz& zm_^_LRDhQ`-n5>G%-@uL=+fVmBZq$mpIHU9R0ew%Iu{7DMkj;mp=0h3;9gew_NwPx z5LS^qn;DgxVkyf9GszX7?-$-P>HOjoHed+^y;IWR*8gF@6c~f@AGT5pv103ZZmgC zI$+p1;;kFdUVbK6$s)O?*a_CW0uGysr(&x^b$DE{&XrPKpym>XS!AWl`rtM;A+i40 z_=PQWI3#9(6n+1>lTN3JC8J)u$Q6a?;)dCFjEr90Auv7RruwKR&c6LZ-K$kyM>esO zcl%>u5QZmMeDeJXX;16$QKAV%iBcX5M%lFEi(&pk&rkW4qkO40?ct5s#~T-|$0NG& z0$618$9wADS60hRpS}rq2y`F)jl?FOv0aGv?@8 z;t=489P!bUfCoLz@IBu%M>5m9Tw&XXa+NX&q*mL*5dLo#8*YAZ^X4s*9W>$&d@_l- z!2JQ&ddWKN9lX~~OfouTKYRW=3CER9YLbqwD`{{I)Q>~V-rD9_4Fj;ea9yVLnJ_;9crA|#WL-|m|(heV#GKePUN~!tNasO3n%cSCmK++*_GnqwzFy%4x|!eNPt8#tm-{7%Dk z)FmF&5FZysNC@rm0XRNC?PbSSf(P0C!4mMhsG)PItWfI_&?jUNx|uX-U%X6&BOV^i zaDa^U`;zl-Ogg&m2tLcI{tvv3W*6}FGAS*U@H09$0-PNE6+X&Z^+7&MhC;Dg- zhjW+B54m)nM>7#EfVYtdg#}8+2%!a>nV@yP)Er7XIJ!BlQH|xxZzj(NOH2qOD0!6y z(d~Yg1m?3D=a!;a-hX;lavsRvblB)jHwSJfyg(vdEYvpqg6%x1c(ZQPg}Pn8k%^kT zkF$1v9)k!vd;Kh*4zYac@6#ab$wSasZA~I7`kEKtn~Yl6=F>SG)7X-ih4%t?)j75^ z%w2{%_2i8j1}69_^j62Z>dby96~A@1GmsFeCbb#WkDvGrvV&=L(Mjn=8L?^D1EUmp zv-+aXV%;j=4-pdB9wbQ5coJp>NuAuvRDYOZhl4p3wuvXgzsXE74WkKa|Aq+D9G#kW zmxbsZfMs_NX2{d=T939lRUeM*HhTQbks%INzdPpVA1ef*0>3WdkL8&lV?8)2H+?qA z&wZsPrKO}ycB1C$gr_9%?Y}ptkZDM$c(}F59sL?AcQ1DAXrHMtnL-fEmn^6!R|`V# zm6be+5I-$#W1O|aMdtE!)bp*^Oz~1S8_g@U-Ko=m$V^LCN zqG?BT`gSqDZAPi@;Y@~KClcbP&kF5|H~GVCM^bx3poV158E941;q-ybR{kOva$8P{ z;h0y{CKgxd=~ecj?F#S9v(oMp`n0H!y1$xjaHspuQ!lmjp!m+BJYAo3lVS70SycTo z`t>DL7(qO%!Y(>s-Tvcv>-6ZGW8t^}pPU+0)R2|PE`TzssJ zD`x!`5e<|z-cK=1AV@A4nak38T#(itV(@v-l^?`%QN{*kM!-bxDN*uCn}jkHhd|NN~bqkexyIn>E4Dq#FMIO3{`qrd?mD1b|k& zx@$e_keqm{&i$OQDO^W&9Y)d9t{vZ}D#k9B>L2~fn*57~)M8^&`$&{QSo-*uNA_!i zlA~WFK%tPZ0|dRQa*N|X8G|pRc-~O^c`4hS;O`56F&IJW;Q&#gK4r;P^*1B&iUuGa zUbWnOKKq;2uoT16IVG8zH|0*&7Wkn`d|uIZLOkmcZrMNFvwy3$&(9U`PcEfSDbKP~ zpK7YD0>TIiS13tGoZWg?OGx_W_w>?JZP+)QSrw1GI<$$;TYvM9{yNE%rZ@RU*ZD1L zQB8b4GfT-d#}GFg?qgOoDf@Yr?L&NVn$rETnpNm8wg^EY@)++qPli$nm&gSVF`H7= z{TA)o7So%me@TY^b11?Is+$gaZ;p{kEyET2_qc)4!e9R3g*aKb82&jH=3u~3%ow!~ z%>U8T|7XP^QHz+FI%V$r_$pl$aOGfB1ggg}$xgIJ?Gvj0KVAQq;-Tn%kQ;LqWB-d| zkphE`xBgA-UK}=i?F4Isj2708$)J)?$qBl8L$A-ttIrI6CJSZs3jbv(uq`v3+Yk?2 z`C^WT>)Is%Q?yCd-)rXP+HQWKLuIRlZO}M(`1@k}O}mn$XqyS?&t-p`A=zzGKn|qh zXRc=RmjNJb@?c{;yUsaL%L^{S^}6 zo3!{r!R@-Y-#>P9`v4l9kGSEPq|@>{{qzEB>vCRF@;el_^xky<48uGbTZ})`FfGwy z*g>Bg7lVp3&&JzMzgrCO%{Za>UCc5%a={|(X&hNMf{bW8zY57Inq*_s3+s-h;6N?^ zQu5x`%GRZkaF~umx@*|y*d!3Y8t=*}4tJHi}aI6nd??+EE7^yz(et1-uMhN5Zp8^i6avz)`!vqboe zwvhah{erSpzPW+6AjuOrt;d{mjnlF$m8eS*Sl3dj2V}D9dvkRPFntY9x1Ff=xh26P zR@!Z98<^Qxfx^+q;@zWLDK4*UfPerPkQwMCj6P}qoABbNQ|5w^Xh$AH%X!fNKtM8i zfW32aHtR^xOy)9#1?0lN7y;>0>XD6A>_1{Gw?xt>ocbmQVm5DJ02wFvxMU90w=r$; zpp||PXp2#wV*HBaYj`vC;E(X2X}=w{mSLTPzbfzh8yKNsiB@jK0L-Rv-D8NW;bKU; zgL%SLlS-y7md%6Lc#RKR!GsxfWIt7rP5e5+>&kWT)QVi%K+2*&F6CA2Ya9QBbHAUy zDBpz=0XTRId{Xc>S z+|D;POzTW^j7QBTZg;_#SGKp}p8L>C`96Z2B)x+y3(Lq>O=G5=i3Z0-%yGqEy`qdY z{+;fGB>-6(=nH7H-dD1@-K9gWua8G?leO-0#NK_3cSkx}UI6^(JN0|1Tk(E4w^@$c z*8ok4yuLg>t?vkz`vu&bLKP~;CIWEk!J&TE2%_16a($@Z22J)iId0Lv|{gL?oAMw;5 z5;c*>REkxNq@GnRRXLoP?l~3TV<6+3WboxJN5e9Idh$aqzaVi~A>|iu2>MpzlxR-$teDXCwiIob7 znVxL}%Cr?NOH*`=@_FU=P z3%PZ~>};d6h3HmM(g}5qE{IiXG;gj}H48cE0>!QjbGE<h{L&>+XUgT`jMD*yWCeDVG7_qG)YfatP(0j`zCZM>H4v5W1gOc9bI0Z_HbG7J>!X4;O{q&uR$Am?=Mt!n8vxRdONbzsE_`+*K=*#8w>vsV zZ!-^sfPBt>f3AU#L@YBuzk4ra|IuEKdRtj4A&1~MpYjBIqmrqNw6Yp1X%XNdKUie} zZ~1WtkMz-OY3U+l5)c0654C%mdW&Xe(VGvj%lPN%C7^T{#5Y^Wa;vX|v)1ba$xFK#i093KI z3k~z`nKL9SV_H@Do17QHRC}_9DEAwHbg64Uu|2`_EmhB8I{r1RYwQVl^!0!{RPGGR zU_ZSXXv}cMd;|~z`hI5MXPN@!Y2`Hqu5j&{1gR*8u^#|O8a9K`J{I5P`s*X%HD;}- z>#t4GcNp~kc{JD&iqY)AbLo4^vP zE@Nub(Hl)sZd`B&_3)L*5;!rngk~Lkt9W1GdRZ{sUT!cb1RqtGZjzm)Ko&cHB>jS- z+l}FlO96Bn^@Gdb~b=!5A1QP{na?&-9A(zQ^&!hurDRzHo0s zW0K~Z2`|Q`+kU4kiG~=EBEuNS(iOfWCe=9*?1-00)V}3eU60Luv8H+~VfKXtfDNyU1Jwmf_5)?@J9= z#@zs{MASq~?h>LffwS&`>4kjf-u7F}^}-(z*T^~iqI2n!SW6aP6HEL7!y6_EF=8j` zXO+fyw>C|MDaqDGGbbAdv6aly9=#IdFFwrn5L;`FW1Q-g1GEKrqN`LcMZzJiYpY`V zn}zXG_e2W|PMztVtl`L~z$~glR7G4C=;#i51LqTkzCJ!a;%N0V~6d~&kWV=4!z zGl_#xKqRMfft3xe5!Np7=v8FZ>jAu!+(e1SgwHS}WWi@&vKJOGc^=80GwNzeycib~ zItQk9S-2+IllSWGprv`>^c{$N2BfN$)1ka2Kq1Ar&igp0s@9d|w_aZ4dF$3!fqC@#WJ>Ozj*iy$guR64j%@q5C36N8_04(-!XVMp=?QVB`*h2#pq zWO($gDD}@9W=G-C@Z*Eaz|~C0_OcpLi}L0roa#~LcnLBW6NTRNc3E+eBWH-=g>S?i zpo}kXjeAz^FKSNZ(!+EDNb=saFh>ilL^1RaKMJR%dO-;9q}g7 zhDPGPfTfFN$l9{h&wM}J?{sbx36A#6!egT%*RJH_qjOEJg3@j@U+3i2PSeCD7%tq4 zH0`m|8)Z~>R#0r0qx93n!d0QPhdNK@@;;pfy92C#$q#HpIpaqJ;%j_N@qYxA?_jpL zsL`qUwHOx0lOGy=&mDwGN3Zfdrtz*s_Ou|2z>6*KkCP8UENUQR<0Rg+GXLFFVa#Bh&^W|rq7kk|9r9i}g zEW8CDvp2jaT2Ru+*uYF@0J3{@%3@l@`b^8R_u*CZcsT8=QSSn+I(!>i^yl-3hlt?> z=P$1o9NJdC4mSxl8S6>?`lSSung8QgN<_2eGKhm(t#p)YHn^{(2 zb`{Yy*S1UmqlnID3(i&SX?|jDoYS7)qr`Df-)2C!**s=U=dm>UIh^v^M)6PLoT=xF zcwoY3Y1J60a7lM^#xJ1ub_cRzF->u}v|nng*t_i_N(;~cK{m@Xtu@9AiQq+}Qrd0R zOy?M5T=HG2oA-w~kVOk4W?nY1$JA`gW439ujVIdkh@7$WGEj5ZG)~U6k6SX@IT$)J zYCjX_GZ`(li&-D$jf3fevjcw>CzmgoUog*xRi3!J+N+ipzRH%Pi4$S}7Niw3!Wkd% zMcJImx8+b_rN|_s)vWTuqqT;6YU2Zvtci?AwI;9MdhXzT&BF)OXy1Qw<~(?v=V^P$%}Zm+ z@jm0J0H4I8C*NG?yt0CAyl?qX-N#EYABTGmDBA#`keFO_Z0rjV*a7KIvX}7;)13FK<5% zGndjlos5RiEVnW8SbDCf=(+Wx?{h8)Vt6)1!nYOjWka7UjrAU_57|<0$2Vly^Lx4+ zkS)^{8b7lB^(muy1!8i-y6?A|wR9RO71My9b-=sqr6>$106o})yjqI4=1c#A0-la1-cwPyovB_IwJg=q(&!VUD)D!5T6$v7V<}r%oSNqye^^fG z&Iy*YGy3G6k4P6nR0O??Ok|yuN+c0iX%Rarsu3m2vpF1198Mtk9-qjn_A$0}IEH8` zl6agw;X#w%ff&vLt7`J-kmrMhMci?7-A-U868~DQtozgTH___rt2oI1*g!x z%U4{`R)V<9I!ud1SEco;TglER)0)(L zgVMXNVpnb7v~5Wvzo{}<&@Sli@?2Jl$@^5zV5MS7l=WcCSpqrUR4JGySUfA}tmg-B zg_zFgcQQ)V)}Fcen+i{T!sh!q)qHmaZoa$T*GTWZ-LrDE(cenRbXVC5cILMj_+`2N z{;n~r=1ul&R!43N3#W0N@*{%mSP;5`>w@G9CNYe)d(5%p7T2$IWS4#OFc0!wFP6ly z>4q2OwtSVC==-0P1?O>kfk&9IZCPFvtC)erm;FZih7>T%MWzVYl2 zi`9PndBF+hO&}`DN&WUBm~1QW`<~OSi#|#VLO*EY5~fSl6Jkc<)B&{Eu_!_)(Jj_N zfzcp@?T87qQLQbjMC;bszd>8*T?O%uuZ^cjj^^j3XYQacYgTwwyYcmh%#1}8qjlMP z$)6;&(s`EyWJhEamp5__Pbkbt$^^o4WOD0A!{W9<44FemNY zOvmO|xSZSclOV1~gVKYCdxQ2`CZexN5Ka>t$awVkGD8-ttHw=CWHUkOh z;ncxt@fMZRUM?HjM$r$r*mi_J>9t^7GRtp&%|B5mZwgqV%=odmDn8CuC;9m^w>1Bo zdUv&S!Vi-$Uqn@v6zG^)t2$glhyH5|3ppi;I#gYFXwMp?s{b0*={Cb z132!HF0jsAlth^xuT%DkR37fn+P&PE(Y`0SC=p}`@fCi$R==*L$COQ`E$YZKEj%pteydtd`1^H&K zY<7yGImZ*W(~e(S&~nE}!Zl~UpKH)+*>UFGiaa$CQz9A0bo)9acbm*`{Ugcm4jFxQ z>Fw#GNXEH zqpTeg85}DyLIEo=x_blRt&B-uKMLC~6rs9!e^FX$QNls{)IP7LM|$n(nbB;Z`)_$K zju_G0h?5sQlvlKQ8m`IJ$tqx4JC2DDFArLIk<69*7qoKvAD>qgT>43!99JcrZe518 z)+%j{blg5qjOve)LB|XWkNwH!(tdbFBSyln+i$C$=S$d?IJtBNkm~Q~x{@C^5y;YeaI|-LM@de45*(B4W;&w_G9F3IQ!I>|79XF8V%nOH*TU=eyY8Ydx3b7(?OOQ`TqE)CA6 z!27%j%Tjx2UI*S0xM+S|uA8aNZV)q5^JfkLgv9DO*HVwSTM&E9vGMVG{6Wr&jy62J zJ%!llz2$OCoy;BQq&L=G<9l%qViAQ?a?%Wst%i~xG;l$Pcu@q0jXKi3YspCu3?+Cr zA-tYcuqimmn2CGA2#>s3)~Nj_?l-V*6-1^t1sBs!;JAaeSScz20lyy50lf5^_>6X(Ld=jZV55NRfZ?;&^^z>14no-e; zNT78j;&h+}TZ> zp!94ifz-%})>%?tag8rY%ME9KrrpA4np>eyjy{6=2&U$^;al)++BLdAvEf^uKnztf z$YlDB)SUJA(QxR(w9(5Kqpo&*{1%mzdf<46p>#6Slki&&L*CA=oN8J0usUXP1k(T4 z5P?54%^8JH*eYH-LL^ju(;pT_!W4|!f1 z;WCO554}YiPxSrge*qYNVAfye2)U5ana(42|Ek{!3kpRuabrt0V+@|8(NW*9SaL;4 zeyMQNT;YEdOm!|<;ilU9g5+tdGg4Iw3kNF3%rYAQmo$KuL%MN@uxm8@DDhN4w@(`Q z-V}~cCP*M+i+ym|S~Ex>mff|OVU&tTYu%!Kkus6))>P5;D>?C6IV2J|daY@Mf&K%; zJY7c*hQ7PTwS$Lmt7 z9tXuzC%COKgh=>fw@CM5lb-y=rLEUhK}K`mSGjxRg2Mb|G6+@c=+ezk2GU#*4Q?6Z z1~^`UR92&oa8SI{f2&8|D z=74}D=T+8beGQ3jO}ER_Gg9h* zlvzhx2s6UvWP1k}u)_~JDx>Lv4Pb~7<3KfiE&FpAg|R-F6^4Cm*(5->dxS|?EzqOn zKiJu~+_)rm*s6>6s*0$m_xM%u+Xvs@tt@|D&8+Fn^xbq-SFbv+V-2dtzK-0GJ}Cp| zPelA?5cBWbvi?A)cwU`?ge{Dwf#079@Zt&#<~s)eAJWb;s;zI``bCNqXwU-1ix-FD z609Xqq@@BBcPLhzpvB$2Xwg#KJ-B;t_u>}Z@8+I!#_9ild&m1VBgxKQS$plh<}>H- z;njb76hTb(UF^FB;8?LwULu$D-5{pO)b_OSym>sPC?#zwyo(}Ds_ZYK^WFWYZMP)F zF9J6lK~;uai;`I#1u6y)20tJCBEFRR zO;gb=Ct_6`fpwd+$4o#!f5B8=> zDy_Ru6r8@-&E*_Wn8|T_x}j}jd@kH}hCN$n->2n*M#^~5Pgy3ce9`}l9$qe0OQ>SF z`iCtFytv?RP+se&keY9&%|)Sgr`~zHDv~3My1?*l5VK<~>*5A(wkgt6#X&jA%*@`k zz;e@W9{uw1T3lTA`Zm(N-|A~47yW!VLSy4eWIX7a+)&YSe9P5|>`)wvfKDNkzd#A0 zF1JT>F^acq)maili)~CGVxbV&0d=km(4U6Nn{ABXL0IHJL@RxTOBcUh$> z=z1%aw%Vi5@F}u=s7rRo3Qa$57+uKfL)JNm>SwuaJb7ZMJjHZd=VuzUyBHtuo^K^C z%*?cnD|*!l5UH9MJz8(#MH@Kx z1T4w?YNGA!2Jw21@-d&>loYuL{}m=Z+kdN+I4=L@eJR)UT7F(WeZ0%!PP5rph{XAE z>D^>i$q4y@&Dv>d7|~jyEsI=ri)|ELaSs#w_fd zl0zr@K@I|s_cqU=V8jiimqP8V)H(HYLJmXPSeC4!a2DU$|nw5m(%h6H|q7fOcDYD_s6IXpkVAhKqlJ>YQ6P$hlnF1Kd{01}&@@pJ(B+x08KDRzZA}Yzc}t2&x9nsy4K>!N#@|oh z4B1oKc@F;rW&pT3L{Q-aC@xNV*&@6geH)lE?l$3!UvCMzMS21^qWK>kHlw<{oAT)R zkl66WC-)cBCMNB`J0uESsTC2!Z|&q1oC}5qMKrq*-{u>M{9NDCq`X6b@1J>`2X zymm8oRrm%)D(;sBlzOKTz0(lH^XFP!Z7jY{D$K>$%WjzsKYDImQ(;0rE|Z!rDX_XH zts(KT&1PIq%0E<`4ue3cGi#q%_Zin>M5NL!uEo)mNeDXr>&oHgMeaiN%zmtzu`%y+ z6NJa%xbb=cwIQoPIO6>N2=nNKJ&R~6?lH^rJ7dhttvBwz-pzdz)7pI;uipDHmdvdt`!`?2K8IBN zpBQ+X)KUUXiA4CzRHcL4H6d_Q=g>)M==Zb*9|&U9zHvUz8wlkod3=pbYo%p(G4yxi zlK1!#AzE^?z}+5~7O+XY6}EQpjGX5wNC*$0u9$PmGwV4hl`A99O}drtzgF5OM1NZ(x~4Y%bX`HSA9 zCtfs8V1&?yll~=zp}U)3V@Z4U-(M;IS(>`-KB6VkGKQK9o6<$_hBtuK3j2R~5t;{i zr$-lO*0;PKPf1U^?zAc{?kJBg*-U$1A-3O_{sZ^$6y&M@N3^8J7YTU=uo{$9^VM!C z6M(D^fuhN;G?R9IUg($SfBT!tgpk$3n&C?_QSy{MWo32T>bL)mqEnkuf9n@%kLCT* z_w{e{4}f$+ov`~Gx&#E`KL9WCec+A9H2cdY^#A?Nh?Pl`Sf+>%4h%kYd`zzboHTSu%t~4mz5m4RT}#k1a;ng#6`VE%_O z^)Jvtm@a@B;Oj6uexntxJ5cibZ;n+2CN2=NA3(L|#{2JhK8zUfp^#?EwotQAJbyW} zmQzesC8(SA9>HX>1{ot_7HI{0qKRz8pCP3b}oNnU`orVbXUE@ z9w6)ep|5kuQi99*8TQ5X_~D@I{n;dJzBz#RGIF200+G zd*2Ndt?@O39m-5c84|%B?iK24D>Ff#%nVLyFqNz9Fa^ss-);@(A5W zxV*p~4qSA5XM+oAg5e`3?d=Uho=XHc4)$>g>%j06!Q_QzDP? z*BHR{L&3U@bRMy)fEMB!RKdOXQd);w%CYCurqsJ3&xn?Z1%!#3r31$(-Rhijy6 zC5UAWpby|=wx(=q2cSSL;Eat>v_eb%EyibSw0G}3S^i!%ofL`?gjKZc6 z8q=0L8CY8Z66vybY);%xhdR1tk#pa}9xOV~Z-EM|f+cq-x|*13n!Kga1Nce)$1CQZ z%?D4z?n0wF?DlNd0B_KQ)bkXuekE^C$WXc2J<3xc)Lra(Kz;!%k8p>+qJLi}2KcDT z4Zi6I35I1x`LxAc)2L_aMJDV$DLkD}-Dtrr{KUbj6YXgLPF;lZJ2^b(i(Rm%98jQViLaS|p z9vf}V)t0XfrD+!*gE#;eQ6?%#2{0LD0~1rI#M%wOT`9vfpr#{bz7Oaun+Y{zQ=t`L zaMT}ARU6ngv(;8Ah<7ko=qb}fK3-@^IiB0F*GDu1@SDU_xnD?(0-&hRfKQNKzcU1{ zyASZ{a6Gu242EDHMdCN5`dxCchD$;$w&B|(ZfH>dg{|K!lVNedSX6wp*2H4zLuyExrZpV9Eg2!vcj}{>nZ|K1odB z2{AkCMygBE&CTf0@5Od0rRG%7h5hER#F23&5{G3X+%U_{|7^KT;L&|?)7=;QNAoP7 zet)bfbgREawcY5C-Q}A0CBj480ThN$lL7p$7vBJnlI|3(T$7~7W<6j(cn96`b~0fB zg5}cl8-4t@kz)QTY2$GKl&f(7H2?t+*XTmY_@6--QWrF=YqNmMJgN&?ew1|uQv-Mj zCG7BA08mDEQvaOy>o4;Yva7F^U&mWd$Gqmi+od`!$b01vq>NETVFjB%{dq?{!D7LR z1R{LxX@|$CJ+Z>^oyK~-K^V8;u;#T$nE1Ss&{o#3x^&VR@V}NuPuien_s_BYdL9Xl zS~fxmIi?8VTzOcvr+i&C?A7m&;>8_8O1r&aQp0}+m0&mn+l;R!#Q=L7)Vl~e@IEW^n9Me zodifDw4m!mq8E58TgT3O(co&egHrb;dh!9)xLQREHu|2jSgK7ThLFRYklGLyU?=fp_eN=%93~& z(3O6GZmzI&*U{e2bc8es*qOA@o|6BPUL9A_9`|%T*&KlvYl8>k8))#?Fm=(qDMg)` zn2%FW{VAOxl5{J{Q_Yb!HU;!ivz5#3Vqr$YUV_%t@1&E$H6hcW^%U8z<9|2Hz8x zpJ|FQUh6-tfHLYxRW4407TW5?sx>$y`hn6h8`_h35H3%C6vT~mwEVz60aDA%(4Fb> zTztb~vTjToY`+J1$Tj^{i(6}ZAi+Ao6u}MPrMs@2)FRlH6?`2`gmW&~@H5ThuJ)B^ zWu9|5P1xqEXqIjsmPhjKo5i?#hZ1M$4KXq)hj#ih?)SHjwab2Fs33kfbOdJU8#KE# zaqaf&`E*?r3o3{x--Cy0^148|0U)W(pk-kxO7eQ${`*lu;44N}Li2uh(e;`=qu^W> z5;A4XHwdBW!6%Ub03R2X%6U;?)wCGhPCfJ}ocKv=0C5gb9ab*Z5b%Pi#+X1uwaQn_W%Bks= zNOg?t7w{&H@i%Jy>DX)R38$Zj>@>m)z2V?1)d%Mz{i2xm8QO|M>#VOaCWr_6fup-C zeAVM4$8&FIAVs|-`(mxTbW&}Q|1F%?Yy#V5mT9Lc`#w+Q*$^S>b7bd@6vwY>5i{xi zYv>HvW8FXoop@ePoB|TgSopc{%l2!(^bZUc=$9>Ra_a5?JY`1Fsp|-NBJYFDP$KKn z`!0fnEh$1#v5zUkWC0*`u>=FIJPE1<@!>xe$8%8IPxpssBb`n(i3OZJdR~%boXqu0 ztymS>8usI#lxyV-A2RV7dzjs=sCn;xO4gk}(-ct^=bPCAa4?@!j)goGU2b29x}=U6 z&fvZ1rZk6e?_rM^yv23vMz0cx;CBw>PrbneoO@WBKj#2u$a_pyEd|b__42ajwO@MK zjcx?tZ22q&bP?!NCDl%IHP$wO30e*~q%Yf>NSB&C^xM!tHi4B&cfjuUiDSgVj$lOr z8ly0;>M3FL*;+21y^!F|Oh}!S(TK;@6D{NaY_CYsJqfL7Otu#I4m%F=?Xg|*|I;6D$?PldF6CXqIx({@=CE;3YkVvrwhf4k((Q}(LaKv zfjRZvKIYztgjj-KOT@Zt94<{ADjGc+Eiwu|3OisVXH0!#@FEN}=f8_DaONoyb<*;^ zizE?aEZolNGYu0ismPlLG|-8=f%E+%&{}LuYD>)eTeOYG$HXQ-x;-SyJk4=UdN*pE zE4R1jpY)P5b6nE+cX`(&AA6EWt*z><+VzZYpe)?A2+ercpjO#^r&=iIHJj`PmaFtE z4J#vr$GTj6)TjJO+0qQ(j0uBb?*qGmEJgllfXdJtQ)b$V*W*)%C5#sPfW_&?|0v^ zBka>s5jR{=+~%J60YPUuP1pN1O1J(={$Hc*x!{udKim1mamHzGpH+Bn&gi#w|GnX$ zUZsxE1DQ`g%g%CINVaS;gE)QMX|fkKSm-URyC?b)4`zRJl|r2M%MY7;H?JPpyN<8+ zC^MoEOp=n^>a?c{H-Hz)Tg8+iY;+N#Y1l7!S?Di3U)l_>P^L09u(Ar@0iN(5xr|72 z#t|o3BrlEtSk*TE%X3$1n3DG-p4HR7E2r>0EKgTz7jypfel;Fr%voY;d?&DMI6oUC zgl2ztSwE48gV_Z{Dsc4tgEpE92By|;A5l!*vfUlBm=rVYl?eQX$)$-(b(K^16}EMn z)Ww{eoP|$mVyfUOg{A4%<1QN(b-_aj!TGJk0n4AYp>)OBQZ_Hq5>|R(iM;$lWK*t? zX;k<_&YgyIQqwVogW-e6WTB?!U8*2s0ZU7-xvdrTlGb|joyz{G7-|4MC76LuLyCCH z*^_;^6Ia77u?PLihdA2sri;==es(QVhH|9vafR=sl-x!;+aSX`o#)*zTza>Joiu5d#(b-CidXX5&2~{J-$Pz#Ig6x-IK2@( zVY7QIxZrl6IiG?2^ThDdYUN7E24}iXWAycLrd7#v&!=Vd=z!Spp{nTJfev*z;4;?e zYw^ZG$^{McPltRKq2maBfD%JqsCrzhe@G@o$A&TLmYd~tD~)%bIe2#`)W zbw3{Z333V~fzSkFSDC-nVNT747*Ml;e5^wyaPX&H()fEemV1y;cs$wAy1Nl{Ya}aj zwcX9sL(LeZKgF>3#}eH#+x1zbBidhp?2S16Sxb_G3dZo1QD_XQ4TrPcor;_;a;>8~ zg#gCZ0Fiv&BrX_6?;qrF8i;vQAWioazlASG9eJJl4(X4_){|0kf?;60#g z7>)QCq*M;-hE(29cj8a~bbSNz2G><cA#>@@&Kw zzUr5u#UfFZ=+B6?K+zf{>(MaOX_(6pOr%V=^K09g_s%6``o2{Q5-(Z%ZffzKO-d;hM zav)w`Aqeh?PI=i~p0b*)x}Z9+bK8@@KJxzUtL-a``t`w|Sg8D~{p+*sf*!Hbw#Qwy zWZ9ikALfmNSXJw#R+7(&l=i%}+2Gt(7vGAGQG-Q}Zvn`%apo)cuQOs)WkxB>cN79d z5WfIiIDo2B1$O%EWPIik54$Np95ROWzIZV8>AypX%R5Q@R&fo`W4|H6=KR@uN2>>0 zDgLT7|6cpw(^%f8xwEi-y#+6rj?dUZt<0f5hT}QoIz^slBeZ(=+ro!dKel> zVlqaqFjE+>uJ28)T`;T186D{wWa}-MJ-!}=QWic}y!Xl}S%Pcpy!PWEWl>lx- zu~#;wC!~$iJqJL(Ga5EXn;bbg`(fAfaX>->2{Rx)5Hc%E|NdO~5+Hm>kqg>D9x_4z zDvvg3&Zduz8z5Hq02wSxLDdo+&#_`{Z6I>R2xxe)&ZGOo?T0u9po0)j*V*ScnJ@n1 z?-xR)&dPq$E<5dI9o1%?({s#cN->3u%iud*R=?V4JUt&eG#3TVI`%l|k5Tear(e7; z7kl4G1LLoH$tGp_i=EaoZ@nRZdad+*@OX6Z(E_r@-F4Uz?5bomyHJ%ievBznU2d(9 zxjN>dJZnnH;japx_h(|Q`onf}+hnkg`D0qRgSel5Gdu}gEpYlE6|7QcJeB8Ew zvzkf(}7(y%Sdg? zf3HMfC7j0XP7;1&Y4%>(Mi|7p+|HQW^YUDW7|G`CRE>Uh)N4`e`=ylJNmf?I&W+f4 zOE!-^*u|Zg`|OSN^gN^gb%33Prkpd5I#YGkbW?cjio>qrq3oGQ{f_W_o1U2Y)DLRk zu;566YPX+imB{e+SWF46GWXnm?Q+fEaRI0Ad3MjW3_dVv0coXmfZZ5@go4%$Fi#&~ zq>}ayc3<#w?x(M?{&?t$DMKK@>+7b)w(XX(#})$ zQm<2Jy2Py-+Zv>N$xlEhAfp{#Ww`Dw%aZkedUcJe+9S3a9Tf%DvFD0{yKnDjN|{rz z$jQ27?MarNa;*tFsh~yZo4sX>J;ifBq*nL3?X#9TerneF5-E4h64^MbIr^b6jbGen zLXF8c0%qQ?*wd6?-)1qL2#{xmc%&e)=yAQ5ZiP;|`6|Wiz?u`SQESsvXTMXi@W3{- zwK`buG8bf0E6Z}$W%*pd1V|D^W4~iB0&Dso#r6ChB^Ou55WhVv7MrNeAndM8)0^Kq zON#4F`%oubYznS6wr}?Ngy;$7rT5p*zMNwzny3>k$LU>*B%Xtf8h5hB&Scy6)!GNs zz5J7UX1LBTqtRrCn!twQ#!U$N6e&(=3gu#7)+)5S*fp`LW2SoCK?RTdc?<#B5)S11 zY7xCG-fBMKH1+?HqyJZx?l#8|*UCTp@#4O4mR*IOM#LIJ_ENAmZA?@lp&bkt<<-;0 zcRbf-Sn!rQPqQT~><83#gP~bAd<$Dh^7dw`q3yyL;j*+{lLW~K?NGR-eZfEG3;=e< zec7Gvdm^FJQ%0m*^$~MGqyr#%uf+l^)lnIPzVW|T!;sMLk(0EOVu9z#bmeoe-^SM$ zKH8yEKcv9vkvALpXF-LV51IFO`&-L%^AB%ufWpi1GxkV@q-sc8U6=8gP@=})YYPc- zp7i(0w7aWoY{A<1jUWF+-1Pu*vd&!wy>jvA)06mqxxWj$Kv{VOJHS2cYhaW{#M}dQ4nrWO$t*XTDFXs<#S=)Tb#_}(0Vj(9dl#m>qU!BAU&0SJKbI3djc%{kMDv|iBJMmUr?J!t3G}&o}K@T5DD*{cqEMV9K5JD zcskwZHoITnS6{!eyD{%_a4dg2kjT>{$TK8@?7Bjb#-x3J)yx9vB(mK{Zd}H(GOx51o>SI3{xMnM+;BBsOnK_M?s3!Q!Mb=k=;+4)-OkI6yT4t$r*IzX@VK4yuy8x> z)4oFl=rXMOc}~xs83D~e^lT1XH-u-^FJVWaRu!BF&>@%Jw_1oOn`R{R7!;653AuGU z9&IPZ0S>`ZJnSUf5U5+QXkWqOJIO30m#4D%fzJ^H@r*PB-4bjsFS@{DG8G3v0`t}t zeDV_%JHSVFX^;Rs8vNRYUGJ}_dp~{IbbouW0m;2y%&qYi}!XVg;g`ZlL112VWa(|S#k1huL?~0 z3@B)oxw^skfX*VU=X177)6P4E*J{buouumnR_?Y?cy#JmEA-`lSsyzK4^^rf`|VNo z05w*4gO46rLDp#t$i1ZDqJx9IJK2tufFiuq>1s4hi@j#aYhu~sC|J9q;(ojM?)SRc zVH40$rSCZH{-zA*cT#1xc=LyUwzBCmsfpRFp!7|XD{NCFWEiM)O>I{Jj~lUo;$g+4 zn@qdX45DDgs8+ljFLtlIxg7GaTz8upO`VX`c3SpbX9Qo$awY5mKh6*c4zmKWZ|8dd zVa`n;2=BPq40ILA#G|ktb35d1BSk{>v6xt~$JDV}%m&}|G6tH{Sh*NfWF;NZs7`_rTO-KiL%=f+|ky`cur@9=M;^uLE+ErA4iQ8>h`} z)0ZX@L^thvkCMBMnXRV|tOboMW~aCbhO?%9vqHR+=*n_(+UK@cT#PyDQZacI&L;ss z2Lj#`)obNItey|qK2Y@qzAvvV55c2ZSF@-be-3jW%{3M~PAt?|yk0o}a<}{4UCN`fUgdKWBDR?I{)p=S zeZ$O;r{?7Ye4(3~Hb?ZfQ@CyhDfSbl_Kko{p7hkJwBy-2z?Np~ro!I*?fx@_(@>@2 zT#lu1Y(`OciqWle?Yg?@)*^}SW$@jl7jZ-SG`UZTVvD>wjYzABlEqedqu2V$)~NdB zDLnz%6;?3N$S6NHaM1O7c99)^GnOkhe)!JFLu=l`>n;DOrrS0U?zE~&`__tCsO{63 z9S#S&$*lHuiJg^B403)(t9dAl7y{asg`vUQB^~4fg+amVWTPQNcc4DWUw`B1aqg(V zKV3#GvKz?V@&-D6tJ1jr>9(#<%KhY>b}qrpncIPj_Vs`k-tF$7@WCzt-(Du$2@(l9 z(&QW9vD=-c+TC%!xR@nh%zhjJ{J2Wp!G#t)=Fx+(56^q6C$(I7UGhYy9Ub;Mg^H4) ze-aJ5C>+htRFYbyU3Lz37xJ|(f1jyS1vr;W0bOz|^OLId!YlIc{7Le2Po$*8jfhAX>(95%Upe2M5l5tng0wT_*$3IFylY0s~_$DivxUK zJG8fGtEo9gTf*~fSpUrRT@2|Yb;)q29f=Va>}hAu;s+|v%N5VtlU(q80|$1+;M*41 zp)bcPlzst^6MZT+@YP8853X-%0X&crrj)0WB;eL0Cj>KNb<3L#`j9x!?|9C)JKBrb z{+`OtLXX#t4RDte3Xc*tQH9=S9Rn9?B@X$JQzuU^_@)h9;Yl!}OFgMx9JSviB;I#Y zdQaji{aEeY=Qmy>*a$d|@QmY|&PP3)6t0IY10-VW%u1#yan=)DU_@kig={l$A$}&& z?(ACf*Ekw92ktEE3Rcb^Jot*+$QmX!Y|qRl(T>4;6{9os2_#e4?eAqMpZa-6^4c1v zW0yhFJQdU`vn8H?o>3K8`%uq}E)ft(EZyhWIrzQZ$%Ft!RRtS$W>h@KP3+JUR53df zTx5d2*P-9D`6HHPG`;yT%?j@ujoA2M0)bYamg_Y}@N_qh72opD#1&(QLJ6?2wtfqo;3vt9w zp3(foDv^P0PPFk*vGZ)V`&iPZp|NJTvlP@ww*5fj-;yi~6p8R9Qqd^I`D{l7gODoI zab$tCe8dVG;Js{sG_@urQ=#v4fj4Yog`zY@A;|3+;_!YVJ5J;G7Jac<1jt~@r}>cr zF*e;yug>jRbyFiak+++60t55rl;^_NL>RhzbR+sD1s4^SRLhS13>ZrV+9xOIk)E|J z;i)ee0GfxKL`7iLoBqnjZsL#d*eNsfiyD6G=3t{;S_*D911-oF_IFU=`jf7AL-#jI zJH*?s&*%LUTL|$=zLo>GCVqrTYrwZYbt>wT7+4h{I*s#z%i`=zK;e52w?4XOoym{k zA1R%^8p;9#gpj>!N^xk+gJ3jl_B_nkvjyQhJXbL=cVLO+6rVfb$*c7%wx6 z9I2~Sra70FLvjM6F#UQ`E=E~^s_n*(9HF@rT>d$plv|1k*_DF!vcx9KHAyt#UXK`T zoz2SBErh+!Vc~M4l8>P4cF6s#+yb8!tcUD5MeV$&_gzW{#4CYs43wv}*bP9_Mf z^GR$X>ti2&q~ zrkiE}O)0<1-@6OTck0iu=Z5G3w2oBo$1mN=DynQ=@LV%Q_lj;pNEL+|HsfW|^9eS#RAt4_c)alA?CNDR3RY?Gb$8&^( z*3EgjDFdDfVUOydzSK9t&fp64Ofl0R2OW)8wJ-I0{~Fx(!F^7+MLbu*>?I4X=drmY zF=v(VAF&M`9FeldQ^@j4O*lL>7`8%|4ioH{!N zhZ3!^`3XZns!(n2EuOe^FhuDRTL#d2Me}z%-S=Pj z4q$;l*0nCMpQJnsPbDSvu-@b7RTSPIGsn`T>29!-#J!?UX)t7X7Vvhn2s0EWFP}bz z;}JlvC(8)^<<|O9JIiHeF{{tux!5<}z|2VoRdezX`GU>(x*q|@xMunukT_YKgA7;@ z_6X756ZG}h5cw~9-1w(n%s;zyqoHjrHs-1nGq#FuhTWVQ*|wjn>s3;#em=Q*pxOMQ*ec!&h% zjeLfr;|Yp0hlNu>SJjLCnVEKfc2f`gPYhR5Q#6y9E*)Ycq-T$dP9!u?Csbd9mfjd= zB(J=TcNiJUEZY86Z1!bI2CNh8`0b6<0hWSsPDBxQ8jL_qvW%XJM@DW@jo zrJcm9B8&=7`CFLgZ=#wB`7vZr)iXew!FKP!XtGlp6ZQ8Zo z^#V{N`kX@umed~?Rx;_h20|eg`n@OFlb3;tftz3C{%8uV@%mRJ=C~jT!X(KO?9nbs zTm!kENcl*l*Qd($N%ul$4=l~3X9{9g9NQ|oPr)dGNMKa1z(M>;5~1>r3oLK4^VJWD zBry?ee2W=SFKkz?a%nWD0}}F#8*JqFMyZ>D82UsB&o!2Ub~)JtC|Bf>KL~JN7#)h( zyq{!}x-lZDt4a99>CR(3VD6pRGMK*iCBV5;JkuS4AhsudI{n<}gcc{QH%Qu-IvRrj zv&?JB@9ClH(XaN#fG#HPIBZIMinTb(*R@;tqCP}=Moa0rEAY4LjIC;v{`OhrWAL}U z4YW-cK0hU1CzIu${|eSuY0_yS=qAIcxh9a$^*0Fl)#i3lZc*v)apGuAb zaWccUW5DlKppxgVyCj1Ix@Hts`Y$%T48RBnA%_V>45@Gs>zRG~9(;iKz_*5jN8389 zLK!WZ0HAt0_p~+{B!ftvxWI{(zM{%f?fQ9>!MkREFEFH@fMg|dBA!HxY3W?EN`VXr z?Ug+!zo1IGM9CP5!n-7|yipilj1al0X=XOA6F?suweEPgixS{vzE-cn?ytQXTz18olcN@f6 z|DYuF+seeHE)mJgV%Jj7vCQ zg90qj;UEkra?m7gMtYXus+;dc_k|y(NpAy<4lB7V@_FRG(#cg8zG`RC%CX)TcxjB~ zw-R#VwH$TvUa8Ce7or8@L~9dbFRf(=o|t0>Zh*G*%&}*BGkA`}xYI+b21$6}EwAuO zg)N&hN!89-7fG3_7j>`srdU>gD5;m@6veTUyvvZ#L!7^Op>GFQ&!KGu@ zcJY+a+nU7St#Uv)l9vP>wm?}&Gbd9>i5h1jm;M``BCdbVXY_OnobJ zq5C3vozdc;ULyK1h{lB$XApxdJkWFO`&zgRCX(Kq`REc?kc5-!LeUXPLz8n$D;bz1 zYe2+&f|5ZMB#pC(P~s%CTuEBR4#n}YvE!&CB^FAl zC()j0NeboeVO#QRIJ)hqz;oJ)+Th|Qhby6($7t*>fw!i@DEJg5R$LXe#eP=y%d-QNs|fNNDyn$pzz;v4pzMcxX@w{Dbd1>1=VKZuj?{2t{&ayeBkO)wuH8M-_Zn{zg5|!^vZjxiY2|GZ4#AnJl+esnN|qA&gVFTr#Kj^f`D`6-yr5+W6x1+Rde5HA1XStrPwP1& z@oTy2gIl<&-h-YxfAH8DCK|9ye3?plDZ<*E9(|(7OfJ0Od98Z z_qt2LJV1=h`MIebc+1Z^oy9w$EOthY^mII?R_42V!U@i*#mR9*uXlluQk;Chb66yc z4M}+p#`8WNYzPKQZv;Li86}C2(L3C94C1tT#ijbE3e)^9e~f(jgSl)5)xA`o-i6n{&Y5DT>J+13R&rUSjRgW z12jRu4-4?^9FE4rXBUbjO}Z~&64XYpLtS#xXX~7}+$%;m-OsX1%#BEWH(ErsaDxab zDuYRji8B4ovxicB3u&cy@o-{%{;Rh*4Br-l>s9_eekS$?}H1A4ooiO{X+{gm0mX%w^0kTafLSuby%`+4$~r+d0PoQql= z7TOaPi-AIgESVc40j-*Ks4tsC^i`IT;VKZaZq2rZb#uOval6_qYdaH;F^tp9i3DvW zEe-P#FpL_Uizv@d6n)T$c3+P5qf+@1;o z1~q)k;e)fi3(7jGGzL$Mlpc|kYICdf@n?viUENfTk(~uHM#VV^B6g?rGu()gRL-NN z#p4`svC_LWmV!ccI}#)=ruv)|JTjrV{Cidkf;E-JWGz|O0J|7jbsESFZYM^tOvL;(fLmr&p;{T>f~P1stH`i)`3Mz)H-n7)lM> z+e0?N$rO%@_Kwqk)MhKL^1Klg(Dq2fRRw-a2cMLjv@1UKOvU!EWQ`@n{6082X1idJ zkY<@Cn&}4ah<~7a9;hAPX-wUvwERTwiycPZ$4N)>v#_$gvvNlN$TX#VoZg>Jvi_TC zGyrhzx$r@S?M`{Oh~|0TkUN4P&72O}$5zG4F}LG85Z%K?ZMhly>LyXzEFOK|CB5O| zJ8?Bk{Ct|oaD&wNdJ7++cpobxWIkXyv{1NLFLMz}tm$d3GBv%w#=ftX7@_U-aoSA7 zQ$Tm5_^@iIC% zpTAHg)ZyOd^v&Rri-_q7?ND2Mi|g+eQl84I!#7}gdWpvEyo3AHH0_wmrv{C3Auh2@ zPX~TK@6$SE0yXkGS(og4pZnjYCJDf|^gd&n7dxl<(Jv-JuI5YN!LN1@&|>`hpIh}) zq5Qk&I_=xC2c{d52AqwNt)X^q34=KGdcv zZDMw`;fM&fwf7f%u50=@g5$jf#)A)3U*85Y^+_LV46mmo-7q{i&N>tcGTJzPNt;cz z<#2Fpu7-@oVQ{YHLv^!8y0y0>D@-|YksC^<$5CJaGW8u$mO6|6Y&0LF2wQbVYYRD@B{)2z6DsQ^s7o~uW4!GO z5{|q+VLBVQJb6`@8-r7z*nhaTm;NjM+u4uK(NUv{Gq2VWE->e*T_fXCBe~wY>T%P< z8(mWzVI%|2d+6=nRMGk5YDa(#r}>Q8EAHhBGNsJ6Zmri!zlBGr!_aIbeHgPTR(U9o zVxUW6Uz`#)KU^t`Xo`Kgh}?%haW6^R8ok;w2+qB%UopKGut;}t%R1N=`AkK|yf0le zx~gk|tHh#2$0qafwKj>8NG{E$l(&|W(v&cWzkLLnnng)yVH|$t^m=+lL31^RK$L1NUBk7G>sR zKAA9q-`&k?PE_8Z`a#%YVRDE^eyf(`{>w00EAn6an+z%QctT;!G=uwD!)SS?r!{0N z<^m|?v{GnPOA)!N8vT#RQwaRO^1KUEwrLO3S4b9?8KJP*Q~e>K`>P>o;Ybt_+e9mn zVL8}0Ca`UUYmXg#jn=&{Mf#lJ*}{*NDH4y~qJ2ozsxCR(Petq9C;>Ff&a<2(tC!Vx z`xAaLb29P~GF@{5Rh>t))|6?hpGb_GI2x8(%TBo8@_Pd|?jb4kFf8ji(vajKOwf5s z$ChqJJmm@8VN(d(VFr4pWo&QvF^Kl_(?TnE#P^Wt7|(7lI?4(cgFm!}@tTgNUDz~r z7{MZ>O(y8Zq6^m>E`4Fbds8!tg1Tr_GGj zH$x^hBbv^rRw#G2Y3Fke*to4Z2R*amc_6ZGV=?e;2~_1ivEdG)IPISr z$K_^Uu7!(w9<4J`faquzR$8XCUdmi0EYl{;U@2NBhlx;>qM~=gM^xN}%g_}WFk!;a zWl6VUb{O({hwhq#MMt_~2QGh_KNXG4(ce2E+dH6nG3C2Zr^HynQ!mx!Yi-eZI%(GR zPBy7%6lEPxE%`Dw}ttZhg$^8;rz$NXiWSeGJGl5@E>%}9K#sCmepbhW$&dPl7`QB-F(a9tOofj$oucFG7xG0`i4Al*eK0z28F_Y5dKiX?BJP*xp6;yQZ^(~L zf|WIHuhffCD?9~3{U{|bTn%S^Y^VZC<83jEbqqf`zZZD})j?pN_uI!#-6 zZoW9`hUVva9@{&1QHo|(TRX%I{&waPqVJwht#6p4o}fpIoLRn`Ya>0W?un8;ZSMRI zc~5F+%%iNDlHy5&jj@{FMx)`Q((#?eocyZ)t;ujs`t@GDcJOibC~OSP;>`ykZ>3ac zujnXY^Q3cm#xXS)RHR&ACz|R7U(=6Sn@Fvxza`? z;yh%N5hRprMeVE~6YboWS9OfuOjrS;gC1v4iN@Xbv2QeN81bNpoeZ5xQC2Xo1{WZZ zTYa0SeU`!&hunpu%o-%VJ!90u>@DBP+gzMKtG|0x)Cd(D`_zbwQEr+LQS5eVHt_3m z5JT8w{E7D-j8*YcYkB{dhbM931Ym{M#RDK=i&;TP_;?LQpS znqt}@_jpQ;N*0b6q6AcN+wEArwC2r$NTZ_dy)CmsHHy2ea&>{&&q+?AaZ!>QBDKA} z1u#6_^_RpCb^Ldm!2ux`?};y1$gIndZgYic0y1xO-}v|kdZIGSyzCuHI&T5FmH2vR z*NvhWyJ&f|o$mFk)% zVtr~mRw_-?)*1WOFjU2Pat3g}W5SG0VWKKy^y*+m1URsOwVZ;+?^D3(HTc_MYTOv@ zNhudEazpTi^?)$N)EIkhClda|^<>tFl!3FEcinE{rtSBld!bc4)P&ZrE$y!zOD-7H zuV1QzvRL=`Z%34RJ(X_b9l)jrL1wLT-ga*Mv`%;IYMYTgd&-<7E*AI5UfPA)@7a8t zPoG@1mgN4mT8U`yvD)V4=J0O!@H?zY;K?{LlpN~=s>c1qhs=ljabP6yum(=*=PVxa zvPQOx>nl;ZWpgvYRT@N!H%!4svv5ndJrMj_RdE{LtFD&0m;T0A?kwAjNQG4nTGKo~ zRLiPF(efXFZ!QG;IdC0R4yb0XWa+M(uzkBK5~Zt#b9u2{ zfxIkoiylH2h?|ll?=^a;0u|Z)B9X&n2d%8hT`$EVN%AqLxq%#7oiq{6qPt9=`a3Ac zHIhCL(Iocw1Q%a(M+4qEF1^9g**`_SveIc~eVEZyy1u_*;;P?pp47Zcpx@q*_QeUXW4L?7Y8*r*oY{+K84~gzMPZ{^!|VyrWinFeKAJ@ zc`Y<`X<)h(<>BY@qo9s7sWu$)lM(SNpPAfB{)D&e4Onf5*(HHEO~i_XVHwjdlqb1C zGudyr`4DS4fkLE1St z_OL6+JAPBYZDzAjAR*6lVG)u1CYZAj`5d^L8FIvhdQt`DZ?bVe^ z`BP|cGL+TGKP9uuy^{wdr3mdk8ol;VN%TyMY%Igg#0Yna9f6+ADPJ*vtV z3)RmWBN;)3c%u_*M#Q}PT;`NS_(>go6wR@Vm0@V*kHkTTMWzv0QrGH-jEZhnRI^#@ zg^!$x3Zb1)o&;R|g(~r{5~Kma45|ZD<$r-fL_xDys-#rU$~{s_D@A)ceX8$Bn`mfs zRrryb&aU$foU{t->kVxfnMf zBU=X#s(C|YP_b`jn4y>9P##S8ruTuOgvU_Iv^85)HC>6oe%y3_6W#$S#epMQ3qmJ_ z+CG_oEtW&$4+)B^9YQS@468M0jHceAVP@9*p}Wq!BQ0xgsD{6FD&nvI!=z&PR0ckD z;U4BNjTp>ckDiFrwz@YaEfE^qswg)!BMR~`H-g8xJSM8H5%3cB%1c~_!mZTSxrQ|c zLA-3ITDdf*v!KUKrBu_h+P&+cWrlox_`THiPE%Bb>{nZr%e6dfOc)K>+oRoQsCsL8 z^BsG9B1PNV#?mLw3N_BBk^bHTGra`Jg4n{5Jm=8Em$r~XfiPUsqn1#xLqzfyB}>=d z;H>(oZ}d)$S=XFYpVHQtc^*7)8DaBR?e6Rb!?=AJHoh!8Si9?S&yc!;U#r0&-@F2E znG_@L;FfUbY%yGnIoVcSf9yi9#Hr^>adXsN(irqLd}T;;o(^N-%U~X5`FW zp&|29Xwoe<5JEitd<`LYut3UY3bYZ;irPKXw(JsU;cE80B94$hFl577Mg9DVNJ|=x zA%1|7m#bYRE{+i@+qu6>H0C3p9LLwEm2uS07tAI?3Ch~H{MeZKW{6d`wI1BcbrbQc zHV?-sf&(LrkN48mrK_#`^j(^tD72HCpO_si7HnOJyvO@#fU6ziT&$@^22;G0%#M6T zd`X9feWEG5EU>h=EvdNAK78Ks4l@dq*M{`#+lI@OP3y@k1$`V z+E?2eQaZixIiHXZRJKuO(Gqllyg*4uu+dmJ>U@76r;I4-m zoi9s4(Oa?~MRZqm*u1?4X`#AZKjzVm->^cNmzADF3jW)vd(UAqJZ>Dl@BB!W#GRG0 zk=vg`toANe&-x83%T84A-Yv}0{3-Eyz!RThW6du1+)g*|5*rWuyBQ$XNvKYAeThE^8MfwgEbuf*a5@?UK%)rNT>85g|ygRxD_e9@P|LwR_YW*UWwT z!j3txYa^mzp|mofAw!asfZHs2=&*DjpBQf5`tpu^s_ur0 z@`$rYUzwNRwV3l<}!o#jlr79q*I9V6sBOA(X|ma&qD>4%P*A+oL6_{gi@W%;{wI|bQ1 zKE*tK_gu7*toUkYvubD|6g5((B`=zlG!Ky_)7s{wld+fblxaHZYsZ`If8nXgzlwoneCnl$d-U&Y9 zGN+b_)_s+B(NBBjxi)!;-`-QCOJeeU=c~?UtWrs-4kV0KXDvSn%|+CXr1S~V5O%qD zhQB&(_&LyZA1_M69PkSTCs*A^N#8E5QobMtyIXR}?tgwrR|oTH>avhil#Ob%w5-_o zoq4-ADG&BHsNV^Pj~b+UD-o?@rOZ$Ih~EF*FD=0tC!?IetZ;NGpnUl|bNfuj!IG1)s>l*Q?<%5+ZW4NTpJzx}~I zfE0R)&DY+4aQNh16h5={Ew{Kqw(JDl;e}oSeMYw2WL=(1c29L;sRyss0F!ggcT;mM z%P83ra!k2CGUWD39BJh7op&$MeoUq;F448v$z3jI&>0vvDeQAm5ivbhl%;(|X@A(x z<%z{rx##I(BQ0V0r{dYY3~??8db02mzX(%!#2TgZYJ8&=vwWJn8e_n9>+?vl_@mPO zP;yKRo}`l0PwV-#j&UVrdXjFp^)#80)N97vmNy&=)4%lZ(KMZ;W@NM+a^L#os9eS` z%}Mk6n99G3ffX!hCG+_CtGNc#>>B0qzz&@0%J#_3@N07lQk3LGU*C>tb;}kQVs>d= zwg+~;f)WhLs7Y=uG^@Ws?*XN=T%1d9v+BY`lOZONAAQNf$8(L5m~IFo7pU@<8Z*1j zW7OqZe~i8)%x7#4hB`&EJ8wEdLrnGKRXfJ!`L$J>vn z^#PcesQFb$DT5KrdV@ApgF!01ZG-j;$6NHc9G4aPW)WkL3$n_^pHJ*{aWNGS_$J{I zsOeH^^4?FU&<}p!U-PNF$J(%MHyllw8koVpUM#i-ijB3U6|iwRhnFJ67uwmfjlh z%24Z-zcqyvxI;@X8ERf5^Q%ycgjh#RksYzV-^s1b82VAp3oM!al8D%-g<&rHVW}Z_ zv6owknIo6D>GYS6wnet#=?9tDhvTezQL<$(CuKm-PPhWOD2bS6BnWR2Q5(Ipf6oqf z|8*5Uz!Iv&sz`KJ>ZSd(-NNb%Z6}?nOFs$2)YmwvuE8o-lkE`ctcV+KmGHDl=LYRP zqH=Qg2pE}BY07N<48FvGdvQovUas$`Qluyc)TWH5^qQAuwn@x!``X!TY5F<;TxO$o z0*iW3pZ)ffI#IYpOqnc9#Zk`x+-?%9JH6ab1ozgL!#{p!RkFpt7HK`Cf&ZnRJc*QT z4pg1wssu|65R>c{#r^bqZg4|~EvIL5Q0VaH9mVQJ4oH2o(+}t>YM@`O_(drdrcjs8 z;d96^vA61nWHo6=`dsfbT(w)>MRgDYu6Uifz1r~!oH1P8!F#U>{LYI$d~lqVfchfN(AnpY>v;P zpXfR0(?D{US_+J*+IYnY9ZX9oMvS>E-Al;bD@~J|=9nk3LkpOV8Cbu;f0|fdeDcUO z*PCm#`c4(mY*`(~*<{%5j<{&Xh$Sc$gvCA zF)1@sGh5%*0}Wrb%!Y(vXV^5@Lf68XwX7>=Lw5LKYg?vZp&8{Z@@1Nxl}{T345^!? z)iLL)Zw4FKW+Oha9wEN>Mp=p(MYPQ33@aJ=FK#%nqTh<+t>&<=&TE~qST!{Ddmp$_ zrqw8CLjQg5s(Cj-XkYhgg2=OTFp3 zI~`jWG3R%CNdyf533&v}{8S)yYx#B19|(Gw&m35EMkUtz)y`}yaJTy1^w+ml{%)9A zF8ofeRYvf;8k-l?9%@}EN88zJ0-44VTR%%brIZb1OrA~qPH%PIJDvof$y;%`pjNLo z?}fO72Ijc5(1^*zwdW7Tk**VadOmAp4__KhQY??-Wd@F{9O7>JC_s@UwjtgOz{9U! zgr@iIe*@$DTfa5zEuq)kfO}Wn!H`RBmmi*^cBLWI>yK+;~*jq6yNllTv=hq)YBFTl1lJp6r)lnqz+cL zd_R9Z$WTlOomdk0}_R%l0uJbP-!yfkU|Q-MN(5p(};gS?Kd9;BX1cPlqR zlssqYDrfwncfZSI(fBG!?q*!9&I~Ja@QJ47Iym)(HC)dl-S-10BAdl$Ry7FZ|Mq5( zQD%P$jA>%a*_^%WRJoUd)mwG0gwX;89eh#DqJy2g{)S z5f?%fF>v@4#es|ELz+*1{`Y$f!sP* z*ZCTk63bT<#g(UO;@m%)X+#C$@6_>+`6MFB1*S_-)#QvP`Og$SP^O~^t6lrt+~ybz z7&}}fZuoF}2*xT5(TwigKUKHtsEhR-vwjl~>m0kI%Jsv0)Iz-DNjz7WC!AYRlp&wF zXRzFw>Ql?H=NYrdK4CGRJgeu@&0tsKZiUd)goA2`nsbGBa=Ns>lRW%acaf3{$X!vOR;9%@e4UONc$_^|p z-A#g|N0wRUD{l?dE+j&_mz|;A5+&ctL?V+tk9*K)9&d>XE)|lvIRDtF7lJ*p(i>y3 zi|)#id(^Dci?^tek+jboo4wygTZj8TB5#k?NoaCcC*?fEhFDC;(tXwut&Ygb)W_BW z$$y#i#h=>qW=ie7ckafjcq~S7>5=u#NiL{(dy(sW4oLZdI%gqY_|8FWLIWb_kV(~K zK`ne2n$0@Grr%WCqU;zUgi-e(xhfWM$N3WTDiDu$m5pTGX;#fmEkm_%DrFUX7X4ZQ zvv+c=)c0A&Vk~!wb!w!a4pO#ZShA;g|5cc^ZKlVnI;|ZcrylkVrAF7)@gA?_n%UTj z*BOwdV20gHP9!f?XJHmZJ;dxbGAD$9e&wx{9#1UIz5b7&z=v^xQEV#{QV)mA6nB+h zzz)Jv*vMG)mlUotd^3(Zz=;+A+>X&m&VjOA2$|RAYXf6HmJp$TM+PcC-0jPPup}ta z!_wChoxZcl%zujI@>*p!w)#pt!5Fsd{_WHKEMjIm=0PDvw}nV1J5Q;ZZ_4s)*^{xM zp35Q%tq0;`&Blz2^dPLn`e2(J@0Hl<*J&Ct)jmT>dv95JH|wldu#*|5NrZOyrOJtg zJE=f&lI7tgk_4LgCx5Lob?_hI@~p3TxpP=^^hxc}h`4d6(yDLrDP6(gXGzAV1B0k* z7XCsX{Gm6DG+g{soMbA>Q$8YYkD=_k4)||7qM-+;%IFoEYU*ml!d-Qq4BT*d%Gka| z7nU{=TDF?T0M6Snvqp z4oYG{g?3!k)`pe_X_lreXo_<)co=Fn_W4eo0(vXSm4~X>Ife2NJD!TGLUkHzctx*Q zB<4YRBxP)hl=GKBHK(bW7kA84>}&k|RV;Y!;q>!mqr`5G+xj#e~HTyMuf? zCaUPnBEllWhBDl6N~d7aXn;ZMXzSOY6qZAaJrh#z%Y)FAn^l!d>s2Z4=*8+wKV|Rz z2}PU=Rbzci8D}!)}#ugg8bZ!3j;hf2!6|<%`R=9!P$| zfWE|h1X~#Z@!>%oG{#8dMOTDy^PrRb`nw|S6)YiY`bm?Yu!j@9gkknL^fqHh{nK98 zdg6MiowX*hGV1R>0n}zzf*^GI&~hpFM7Ha@RsGhnQ^lwWVR>9n`X~JkJ?fhj;9oge4j!CpZl%N=dle<^hCbb$3BoyNUAeDS>B-aWiNWd} zW9PPp#miKN&b9udD2xaxjf!IX=kbNgC|IZOmM>!dAd!N41;40`182>D=ZBgg9v~EE z?jp3IoZwq-C%B_lC zeLq}Jjq&LXK8F0HXc$q&Bw_3^S&*nsVo!vA(8O!Q$0ihfM~c^%^ut+wraoSN6O6F& z$LC4yJnAW!QRw)-UAOyumyBkIQ*Oq`J5>b!>)lF!b+10y)UCqXDMPdNeOmRzs}Z`1 z8LL%_US9Xc)Ay(@eE^O(@BwI1CgpX#nu0veN7uj`v-?IMg*xWGBcCsOlr_F4` zk&By;o>sq)8Ql1B zdU&t#Y|H7ykr?cZcrHNJ`S6Ivz4u121tkEb`>?^)aJ!IMtcmZp z9sg|$lY*NEk`mN&t*+eP4rdTt5P#wLW#}4PnTSX$<2iXxYIH-;jCATk!@6t&Ybf@- zZs`g~DX1T;j_%%5~*q|k;v^KVJx}^I!X`$@)k$! zgoN*u=BFa0cXKI3_ePjU3HUs+0~(WP4NW{^iOY!CWGB3mw7X7brJE97KG!zalBoQn z{3wGtXfWtlv{SQJup(p8H|x&bh{s$P+?^l5ysYyoW=EDTt!1-~XH~wu>R4-P+AWC) zp?Lx5KDh&%-j6CFK7bjeS|5+~$l_vKq54DejHpeYqV^VR*Jo+ z>vQxy#(Ut^l}U=k#J{{2dv?)G2d%8SZd>i7r{MNNVFxj{OoeW+b_nZhYP9VVk1~St z9Xk``BP5jQQS06kZP+2m?{T}^3~T!h*qr3ONG`U4G9-2XeA$%SC7x-ml%D!TJ~H7N zLT+vN)5xLgv6`){9!CDkj=aUIsr zx1*txvIQI-bE;`Ex$_xS?~|X#Au2uJ6lXiLw?5D$|{U=DfE%fb2Tl}qhhc+4W~ z%6A#4^ffjG=lDNDli9=MPS%|D*5+szxiF@Y%lp{=MeR)82ET!{Y6Uh2B|5Ut>VLQi zD^O6;cWtYO`GTi*dXDStD$E!D*l))WyF=n>a*5|kZJI56Lu{R{CMqu4NRBse;k)L9 zo*KaRGEZWCsSftHZ{GDCw@IryeRzr4LKtEn9x|1M1b^Bzj_Yw;Y=yUX<+5XGKqKmd z)B$E{?Lui>fy&Ya`MZ7+*I`cs35gCd@M@h}5Y3XF>?B8WubzZQoEqb@JC8a=FB|pk zOW()W5uI<(Yd7ylvFDRiE_se|>V40D!LCh`bf~Ke{WV7P0Rx%GbZ)27y$-)s?>Zjv ziPn(6cN{M%G>t;n)d``ylx7CRzr^Pq?>=So9$6kx$cBw*^ZGUnD?t31^p0l4rjqq1 z(x5LKzN{^0Fny$@_%b9MggJ||spuAW@~zD~|7Z>A4~&qte^R&5=t7-})%J@w;AUK_ znGh}ZyG#fK2YpkKn6JOEooC3N)*7jdAdT`rjW}&XmI!?rO4MGVa;YoH!X!aj)gLwu z99i8wzxSIb0txAI!@D`)0!^hTz?rQM>8ewPCDnTuWIBUyQH2*?a9v z0gQ^uW#*-SytQ;ErZP6=kHs;cSV}&IbTXqD^}iwkBVVfHq5N2W|IIIU-p`7mK&#b~ z@`Gg!%X~8@&Rk3@UY9&7~(4@W5DAn9{Tz5gHUp%Q< zwqGdzUZJe6gXw1KLPDh~(>F`1EH;`B!8a zPa+sV5^#A&|I}@L6Svw)jJ<~YjYfd8AIpL!zUhmLAWkCxtRA2SpKCDyg) zi=BQYb~3LT8x0-#M}@jl{~;uKPElflhH{|VNycXcjz%=cxvF_?*Cs87HoK&Cj0Lp2PpsAbt(~Rs@wclzL>bSmqC`EX5l(s zzOsifPc`>>2h$WL-GH>DCD8mQQvRI=(N+1CXX+{-AlT##`vDz%h#ob4iu&>kTcvy( zBzyc=@{%|$xdG8M30cY@^g;aU`u?}*N((r*`+i1ry+_GZK_H&X*(P8AfTEXD%>j?~ zSIAzb+iBYhW2@HhnEZbso3f<>XFQ>gLUtC6=f7^)pa1yt9)X7C|Ml<$s$=|CkVUL4 zPv87r}t1uKX5k*nJm3oYaQz!i<6oQrNMeU=}OIsmHIqg55NRE0m{l_fag57 zdw>pV5Ulu#VO8PD+Ut>aiwzI~Kw2*VTxV(=e8YMPpiqlk9tB+*CVQmcS^(e-xVugx zMOHN{>s8#Pd$coJkiVx&=S`?8%|-X){F`{A?jvbjWTLU-pjmFf z$2=V%P!~}3)owjuq#8(TI$v69z6Nq`n%zyUNlr0@enB?64viEr!!zVFDmc=VSs}cBzf+!d<~OUk3?o1ZX&^ zzw5x43U%O)(kP(XG)~Ckq|=aT=jIAPED=Ip?B{oxI2G$tNG`^gck-i><<93oOIce1 zcH;oPjm;1mda|Yy->(i^aR&nAio zJ6EFLcs9LNeOE?r%cb!voeF-)DNuuY>?F)NFXQj4I{H2VB<2D*wcLZR@ei!@$ ztP=R2XrEnROO%=Y2Es@KprFSD(V~0j6WNy0!BSa)r)x{CH-w#=a$G?^JumcoMc``> zm(s0cZ|lH1pL7Dv49tZakLSyQ?$c`98kbrG(rfXyum5>Ta#@n%GiJ|iC!5|F&3FxJ zch{ffi8mA$RSswtpU%v9w+11BDF7SDelGx^v!}!?a%9IrcGdSRHvvFU%&nP#y>1&^ z)u^k-Ls8SrsStzCCV)&g z_ZhQ_*v(w?49jv2=KPWb5$$L@0on8QbAf=`O z=Yp#*t(ki95H?bQctHFy4#K-|k96bthjfR(4NLR0uvhSG%;(5upgqAXCgV3aXD^qC z64?17Hwa(@8hj7j*1Si}4P^nWMKTD!suc6fc%HACn!P^t2xdT~XjT9G!N5k~ZU8{G z&vNk(-aX%)ut0x4&b3omjkLm zoWQ9*ScvjBmSC#fMNx6~G?BcwjO?2PlgwAwozp+i^3o$vGQ<$XWUo0ushl9@cD@ zb%2-hn1F3 zZRXLeX^*T>)5gy{=Sf}rTkb%%yMTQg#q7%OR8*(Q^R6u};fAE|M4nKKVXhZn|l~x>i;}!?y72b zikfk<2GhkNtONHl`L`#nd|V>E#qU&7Wh-Ev=KIiaioIZ<2|(nSN-i#_i#|_}%*TW* zxy#Mhyvn*vEjhrCyp8U>dUo!;f<11W*{z@Ks&ne?kp`R?(XPk;xD?VkM4<%IHKpv; z(-DoFSqUW_cq`;W6pgY80w{tIKY5$7`J8HON>Q}Pd)MTjzQP`3u+^e|DwjE1vBW~u zAo(C()}$`v=-V6d5q!)3^5KyM*d}B0=3VGNe*~Pk^a_IDDyp+msIGZw-^{ixkE@30APrS*0`;s3}N6Hq2X10Xfjg6Zs@{3-W32(9HUPPNso|V3)^NDp#e3W zpCHVRrwgIMD?N2!CT&48KsIz~*tD1P=dfaLe^jXBC<0>@F(d6S(e9E47JkW(^1b+E zDpi<8exv?kmhmW6Y%)_yt?%K{M?=x z7IsUbc_NJ9DrxRj`a=H(9#uVC17$mPbp$qe9UTEN#sL8ZUPFWz3V1<4Ku(WFK!aZi z;6=U!>A$^*_a(^x*6aTg%Ie4|E5olk*6y~p&K_@FJoC1xkl~`{9CQsl4b;`dtX-VA zENxt@Y`J`$T>r8lNcf7ut4_9_mbAW3U}q08UrG9ZD8%6Pzs=nAwEvKJI!e+TsB6;7 zxwzZX3UUc>@z6_Q($dmOxZBu?Y0E48haG+=N&nW<(^ZU{+sDU;%ZH!K#oeBpS5#D# zn}?5^kB<{h!Rg`W>}l!C>FmMqKSKU1N8Z-M+TFp`)4|1=_ODz^D;F?wQ#(z{A{;x^_e*XWf^8fVwhf{+4ujl_K z68{tCe_G*jmco?a{`Z+lVfJ&fCLiQxcnxdH$&fW$u;-MqHBv4SI1=*ZP zPH1kn9523WZhrTrx%tt@$kFM`ELU^O+~?+&H^#>LR)Wd7o!9i@C@;wn39h1_&kM%s za>hGR(D7(bMpJ$Vbe|+%OY8>j`b`F`cu3V6p-h*cvmnKX(*M0IlOUcr8}on7{paqV zMlzJl!)Av6r2rzD@7~S&z7l)>c>*Vx8pJ=Xnb_YD)PxZuzAE9wdnlTc}8uC6BL(@{20$pF@6;b}IUNI6vrYM~GzzLJ4c{-&QQ zBM-4witShy|LxT0;a=aO@Os-WzBOlFl+_ID4T=~)Qfv172xRO%l;gu#1zgoGdKlAa zk^J*(`Ca<^#1CM`dyk)nO$*SJ(Da~P7ltfNd=XOgO7druk{VmK|9VRm3-3h>|N4sc zzb(5+h;YuZ@7Y!Vjfp2Ml6uXePYKW{6K-*V9Li@{eB$Rne>_;iNWQ{}j^f}l0zQ24 zV@EPY2|G3G1bsF)ud1rT58~G&br2xd!GDX0q;kcHp!$&wX&FDvthAKR@_{TaA&G>B zruf+4)z=Jq*A<2mn~)VBMAWswS0#Xo`;s06a%tzxfyqLVEw5xoyPRM@ zfB)O^Iwn+X&+9J{6oBZT>#@IR`QDr3WxzUTT%g8>_EyF-9&(Lqs6ujfMYFq(-fWL? ze9@3TTqi#r+&QMmi{KpDPc+K)?1G8T@QC3zQ z?nc@cgMZXmc$&yy17N=B*+LTbo0^o-ZYThu}W` zTi(T-tn5)zmrgV4=jm1yYrA}>S=i9i%GelN8w~0)l3W-NX&f#_e{*}QZ1sRZF2G{e z=NfOoIZ`?rs-mx_H@2Qhv{>(0tGgMi)ovQs*f>=rxqbJAJLBvuo~f~_8v{F!ChqM? znA{FZ<6VGrS8CbIa3tF+NEM&|7W!hBM{BiNyH>oa>N3=%u#vXn@lJm1v{cU3$cUy2 zaa4e~qmG!JlhahMW|`JnF)1~TAzGPyNk^vx_C*MR+>iBrh(qZOeD6%TGT)g0spcJ%bPB0w%wy`r$2&DVk$2VBc$N+wRubv|Vi6Hn{epHlUO3Zf(!DYUqeV-?`4n)z{2A!dyOcy->SSx0v35 zEb$^PB~DJZVIH<~bTjg4+Er{e&ZT?qeoi*32|!th#s&``y~pQ>bQkNR#+S0@_1iK8MnoZujEWqesdd$}@aFzrjNWMxHQN%$jtMwa2V}Co3yU5Y-?0o`26xWkI8VbGlv236 z^6Ee?Vc4E)fKnqG$y}eTa+iv%h)PMVB!<}+AGD2!1^La_nAWl3rWe(r3locE7xd*8 zW@Rxx-d*iV6_84PGCMir!NWY`;3pO3H6hAyJ#@>QZ-1Y_{ma z1M-fMUq-rpv}CHuy7jQ9oN?8AT~j5s+^d8wetnoh{6#3WYzQuDjX;(tGD%^VB>V9AnXZnO9+RvE!s>=-{9iZAa3+26Ht_AP3FJ z_XCs;hEg*Dn83qNYCv|>SFak}&V?|B5?4%5(fo8^zhW8!5EV7%FbE`btTyU*XhVX%Z#d+x3TM&}Y#~NLicMYb2%!#;?~Ws#QhXjtC}_3CZJ7y&fb`#c=jpXEEa5 zPilFsMn)S{U+^*`$WE^%EpF`1xGu~5k6$Ok_nNY@Vc?aH#Z<|qG?cL}amzjwxgzg1 zH%9Y-!XJL$=YJnSBDURoHu-yGQ?doJ*G+IXMPoLHQ^vD7jV;^~1Yrh6pZM}>ssd`R z$>+0Ib-z~}x9yCswnzyTK00mwTD_90lt znQ82 z3ecmSgL2138<`i{Gp@(dZP{hnq<-+%ttU?Dtbp91U7}x#!~?|8#})0=@;RRUPa^I> z)nVLc7g!gBP>y`eMcU)7AT%3#Gljkkh`+TT-3xwvN*4ZY%IM~+QsuacwALdvK$}w* zs)s%Km!7Gz-Udp)!@**Y-495-&W9{I^VrpvA*a)y(qk6=-@2AebeT$jdH2~CGLoi1 zXZcAI-LaQ`L}s{+mf)P6tVYz-a;-;XvGutmvqTbE(n5VO?inr+62E`c=5{e|@k$B+S4}mS?pj7g;jO%orH!WWJLS%av{$wBBpGe{5k3I^sB2WFE7u_o2u0Mm}sIV zH%{;jUdhb5J){0A)dbAU)Kn&%dD>B34#|vjV3ppy*qiSEnKgft?{`LhFmeZ*Y~zAn z=Q19cqS($-Iaf2lJ`qRVx*nDX4#bkp7|9wM3T?N7dc5llDA_@$*8bVDh9Xz6)=v9_ zVb7)&o;tM^X&nox>ej)Gqn@iZFF>}#J?g>Bj)kJD?@j?vJ3e#cR>Kzq(O&D|?8g+p zQ(-r|ycUxU6~DZqu*(t?NR9&w;kmtqzm=*4E4Qtc|IVVWMIj6Bxt&RctuarX`Dw~+ zoym}G*T6KO!WIZ()Sroa-xFr;BAs%1<GSD1aq8Tzzkbymoh; zA{aSJ4nEf_U$)tDkMm3W+2Xt!r@#Uh?z7iPCYSu-i4fjO-T0P(RkA6cQVH0ri9Kh* zB#MLrq*|U7DMUp+1VjinfTRktglD4+e7G=p>eksmvr)zoBpE_(ct6UptfYHO$l(CR zI8X=68=UT)d(9onW)c~V@*jD zv=r_%@_kC| zr6?=+L0=1PQD)k=y5AMql4L1ILoHVRPj)2_$N+4f$EXJQxTIdH33+}3@Lw@jY%*`# ztQicR?LzXbyZB#x<;(P%t2V575paF%#GwLD1XNIQ>}2L_H*k`Ltab$r5%ip3g9D6E zZt@?)`RZ3VBKQ`&-qE+;roarC)K7HH@^EXDD21UGkl=cs6SD}`tU48rydjx9<}1)+ zVj^~_JD3>tO1LGhz2{(Yagj+1#>k~>;9F>$e&Mif5v=F8_Bj1fv3(qB;>f$G-ybN7 zzH%UJ=Ft|BDv5qjFOMs1fC#-x?#cT@)S@z7y=Yb{3|T=r(hUxyla~AOm4yRuAkF{q z8(qN8JZWJa3e_y&rJPlHj`Zi-_Jzs9&Ad8*Y7LVYEq*^Ni zHVCNa#yww40UIkc9C&Dj8L`AqyjfWCTfqEfE;63VQ(yEou!#hr~s0gxoGi3mX{t6#zK|nwNaZz;#DdS`U7v6Slt*P~GxD4&CO)OGp zVXx4LxRz)UaJ&PNq!Xjr6O0YS5x%IZ14Er-aA}+YTIu))Lx%XamSQXRMspeS6sRT} z2oPKBF~{?=j)Ho(sZTFu5|wKFPQ~;1yyDYR&Ipj_o>A^*Em@9X7%%`O&Kij@8%p&T z&SLAbtVy$T*cItXe#aVlHo+kA{HvK;;Y3CIK3O^a(%LQoeg? zCTB~HS9x{jDqB&9MtNGu`R=6fv}h4O^*!pjtvDev8}Xqvwr))d}FwQjy ziE*55^1Pv!oJppT`YFI;pKBs+XEyuRqN@r#o=rttWk8Ncn$SRCPgqqa-#X$gE2s{E(;LW&}>1 z;yA%p=D~^BCkr(tmg$KxMP#1dbY$dc$_bd5h(Pchp~NF)5G6b1ckuQ~&(fpMF2pJS zT2|K7)h1!%^_t5gl#v%^NGIxsw{4;PVJCD{Ha|jqG-YcjEEbvVo@JB?e!Nc=LStLm zv>1JQ-M6zg9;loP{0Toia4Sdi+dZXKDk}j(uB=D;`IPb0h6Od>E-^uWb{_@H_~Jw=KyZ zK)gC;W6D1o8yW*QRPtDuVKAat)R{EEf^!Qv9e1Yf3Tv$Q>f`JH8@_kNWD_>Xc1%8< z$>Adsx7Y#keYGbnK<)Ju{@aj3Vro6#GYbf^sHJi z+}8-TXGgWXk4?Rr`>TG@%X0;#Pd}^i(B^h3wSZWlm7k*T9hFgctiYbAyLSvM)BO!E*%x zVO&df^v2K35XQnkcXIWABG8IeG7ch^Kly;XZ3+a&#cEK`Ac%7xKNLg=1B8Vv2*WU= zUJQhep>hh<*z(~}U8|g0c$~g%Xb|f6I31Oedr4dGD#O@YZlR!<4%+vyuwk)Lyu-Fb1J?EDEB&v&&Ek>h@*zZh>BLto0dEO`|U zixbYFVEM$%;ahmCb47H4Hdp90c$Wqdm173fz&Bey&dqc2Z5xT~@v}-z?OTRLe(9l7 zu+XXajPZ6@x9G>#Z2A@JN6sCHw8#Lj9p4LLNj7O+O!#sc%O^}6OP)nqVI5*$Y~9J3 ztu}t5r)Ljd@mc~xmb|F zCSt+UZC|8lxd=A0!n=1e-4Js%&_R90-)h^5Q~s9q6TW)~btwaXSP0n=mC z*w2M6ILIDKMo}2JLxOq~39bjAk_P~i{1kG$j-bSsR<(;7B)dl*OcLP3@4^`+q+++O$57rb5>*`aX1Wk1U`tm4tGmKjPZLK( z*h0F6L+3g;3zMRrZlB*}$a)#+!lgFaYulp6BHBK0EA=frazU8zETBIJx56TR? zt*?QG?N_1WGjq}Lh8SsAJ_r%#k}l=(Ep+vsWX|i<8Q_JCP0D~-kR#e3SYr4qm_E#e z;24L`MDYieD(<@KkAxFWnWacDc?_uVzuR_64A{(e`gHK2L3CO^Ou2w?&2o}(9kRNYb_3*sq~HpQ|Ew#G zuywUUqV7;*5guMc*yE0<2l7~-$pb}@3EvqGlK=VB({qUE#24hf)EzQJd$m{@k!$hU}<$)NJ))fKfof#Nn0H0VGBO57&v6MjONk4Mfo5rJM-V zCSOyIy6Ux-XrcUyD}mheCa;8$p%5E6G%kkgC+&0cL$6p}iign1r9Tv`q#(eMF7I26 zj5{&7T^H!;J>Hw`wWIa}dbn3TZ~&5jFR+}J{grflGjRy#2AHZ$&6+FfUbjj1^Sz9r z=t2gHvQm=(kIUF-eQ1P_D)M<*wLlg>Hvn2^pg5*ss0=c(vj7s1D@WNt{L}H`$R`>> zY&D{kGr@)Xh5K=W&my@Vw~^wdzb$*j>YPFNY?dIEv7UxJ=A~FS-@=BvW#3|yiJq(G zYznBSvXb}Cn0M3BzKPKJ=CAV#^?dB$CLBCPRqYmYP46Y{g&Y>-bDtLj88`KXv6Scb z)E&+dPKrCj-euZ-Vt8^J{S9rpvWbhhl~uU?HA zbB>9~n$1L)Fj$l>;8)j<&3BzW zS^;(8&+Po}m@(_JF334l0Gz`VG=kSL6FgCx4(sbLB^I66!`sNv4Qlr_B}!s#ED9-d zrv|o_0UNSbJIemYuhF;2T-Vm{KVG;}ZhUwS3ipCZl%AIjrps$!_20NLyvVBnO4?-- zqy1>HLwXJ+%HwxrJELHAtW%A^NyYX|$Cg>b4b7`1NKS47mDHElRZfY_xv5XL5m&lm z)4bPxK5cyH*W9Trd7%=U;}mc;?{Z#7KSLNH!%=W9LMD>rP;m%7r9rcWLiQfyUana{ zWowDDQ?))S=JlpTXgq&=vg=td8FLFqZh36P(FW1s*aQL&E%qd=ErZ~B*RSbCdnOS! zw~76WxtvQZhF)ttG-KB19)pRti)Spj7&*uc81a+b=NsR-F;XLZh<`2Um_hO*IM1yu zByhfiUch%~n@6SJFc?h1n*i576q3IUqn;S4W{yj2RK|ILzs4X4ur^|+bH#;a<5dV= z*1G$KHwF&5F<}!hV&p&_K^QH_S-R;jwo68r*@#F8Yv}mvo(7^+unq?A2cow5xQ*=h zu!np}Xo9tS>!9w{gv7|uz}rGS#49W|M~?X`tbDX_S7+6+UI|u0SXl>0vDcQ@-yK;7 zlE>ygZ&A?cNUJ4hyvYyk?+5inqz zuQs&410IZAU}sKRy$fkH*XGI|_onO0%yElD+!SvuRv!)?lj^(CRTGiodokX4J1rb~ zX)Wnn>^{Y}$=Z~A*ftxt`%U_J!*L{|zxqX#7CEj(ro8gplfGq>bFRNY$FaR zbdgq`;O1??teKJ6RPHd<9I8H>j8j_jL36=!o}8w|3xjUA1aN4Mkp1^09f8bPI|rNx ziGdhgo)j0`2#v59_&H@#zdD-KH;UJMGFNDtg>c(|p(5iKU2frilj6p@*`+nla66Vs zE0=ssA&)aQI{Cq)4%3)8O)%3*$8#xU*hp&9d{^&V*y+p;BrwO?XWFc>K1#_SCBhmya`H3;KQkjMJr^Ls{Z!7r&lu(yG^aYFoS)>b&c0EWH zOdG`N#Rk!qlK9D;n>P{0!ZIRA(M-gRH)WK_p8Z= zYK{(48XA`kzQVz*ZotOL_iJuHIW;(S8uc48lc@^Ru6-Jzauw+LFj5_MYhViO&h zyux3x?ikHr&KKjuA;xH7s4wl{$oW<`!OBjBZK!djKbIi_%W!kLZ=~U)8eeA~wjHwf zZ=o8m54$WS=#_Flx1reS4S+s|cjqyWmp1fv4SQy$OyZq0Dd-}^);n5Ie~3=}tp-4}2rqewC&?Rh0G*{Ole8OPul#7!Nz~i~ydT%IwUeVCgvuU@lyT@7`t@u1e7Xjmm zq%4lKrX``?ZAUjgyCEn5rAp=?eXB_XF9YTxRQeTw`r|$2q2#`#26FUPJ*7k!h0hkXcW;$@uGF0s&vm8F%ictwD4&a|FAqRB{+B(tM`!L;7~84>J|B=_tH{&k_do7@HM%?ZiOkHEi2N28$>Sx8k9 z$vvZvc!bI(hX|cYoxmn>G#`$E1|;yaE0vQW5On0Vvgx7V-grGR@%`B#*(1=*&J1XA zMC*9a+hM@1;m6Cl>0Y;8+}P1^Y@|434_w)V{Ce`P!KzIQz1anVSJYGk4bXz^OMbW1 zkhoA57Y{^?#d@rHJ*Dv|r$c6T>}RyO=9Y(=uaz$XgTasBMd+<~YpLU-#3j~{hqLwW z1Is7JWwO-`C32SHtkH_*hVu-0r3M?5isoywocY^fN9eUu!67mv=ehtnWT|DtCjnR5 zFxsPgEvO}eovXUm!&qBkXuH^B&1Gz`#hui*c$-f;fo`*HXR}}qOZY&w3ifV6HW&dZ z5bYqbkja7vss@*J)a^<&lQ9x0?yG9 z<9-xC5sMmM@7@b3W~-lB?bahRpVT+<2IIZ%vYyek41!kUFGWt=lv&B_%B7PXY;d57 z(>8~!Slh#N^^G;U_(Q|Skuu}vCClB2lP*j0ii|2>G^wG?t~+fvUYkg>niFTf=i~U2 z8aqjZAZqtJd8Urul_Ugxt9*M6lr1abXeVXD)^KEbU}Fyw3b7*k%|JG$wf?5=5;-}Ov zqD#y}Yd3>Y+LD$(ivMBihd`dH+6 z5Bwss1a^8pJY;b{-1#nIzf#SdMa1lXc(^e?e*IHNV>ZO+>4)#1{lM1z5Md7%Z#zie zINO-9i80Qd_gZUQ6}oEzaYXyQ5cRm7A{(Gh9Ic{sy|RM~cg2IVU$m?JB+)z)$1$Q2 z2r@GQyr%>&f|Q}|#uQ-x{AA46sxa?~{+;bRW&RMxIsS93GrBL!_bLTc)E7_@p08gs z9G?dE%0?sZKTsXnWQpy5dw_lucr+xGSSnUQJK47OiS`K09<_asSsijVAf4WMd2HYD z(m=*!lRpXYYR!bMhc`Y0`TZ>UclPN<=Rj|V#Wk9^Zgv96gLT$|d7Be#6=R$R&e3rl z6OH*oTjxIkI{W$G=YL7$tLz?|Ioi7B9v-B&$_{XekLL1{-aK6Awg_ zF2#9Z2LwwUN1rBiq>00z;RvHAf#}O^MZ1bvMYx^12bKQ?TKUSzjIJQ__#46jVE=%Q z_PG9{l5(>=%-l) zzXdhp8A=4b(21tlnGE2x~7K{S+MdKWLqAawVgr>)%vM=HU@3IiQ{T znc({UHK(pN+=RDWNVlTH%L=n!eVi?)6aGQ>{Nt`HT=9KV`P^U3-9LeJO$bfz{C^;K zZ`Vq8P4gL)8HQC!TGr)p$(&c3m#aQwqID+z4Ej?tUq$gM>OXkAy+d>X+5iTyx*&A4 zLf?h?w`;;?`xiR%Bfm4}ZRm=hZdXl|jJj9S&CQJ`XXzWGzqS#gP^1OP3%s5_JBl(4 z^2#Tgs>lo7vfhYpcF2C8NAD^{tEpqm?)GTCby|wNE9cdm47YzRT!x|H8g@xaV=kTZ zws7}Ts|!P#c#11q9Q~9s{wZJ)4Fx4OrxkiA?m$>&5HHhCzt$_OVeYe7Q|Y)9FN4NV zqTz{#TrNXv*_@zK846l6zb~#4m!`5D8EyKm@oQHB73To z{*9r&Y|o3u14!2wfuga0k$?hdh{86t{P#NupZ*maR`b_p{m&Yv>I#T8D-mEbSV z$G?Z=qZJNm{}P5KCHbDf6X*)KKbBOWyStq zT%t3+tK8LhyAu-D&i}Pt>#yykU*qEc;t>C1yE9y#4_1O19PIZ$Sjz9Xe<6dPIGdUN z>s_cZ3Pna|j@Vw?D;4g4O)(`xTvc$NZ}}Qw27i{idT-)IzbIbDD!3k?zA8aP@=7T# zF0S>xzY+LWF8z<_x14ZyR=#lBUugmf2mvGd$KNML^xwRhH4+spq<1x=T|2-^zrM=W ze}-irE`^+SR5t2q!@c}P6Q*FZnoW5BRL^@hg>0UMSg%E}!Y=<17<9q2a^<4KS(-pE z)2~m=Vd=+wzTJC&lDR5z{{bkP&eZI5tLd!=e)p~n5Q4oTWng;ouA^^(V%pKsv5e%_ zUpDOQootcdY;LZ_#^zDb@mF?1!C4#=Xcz}T`bzr2ja~_esOI<{$>E(k<^0^wc-$qj zQi{jIbJZB)FU_~_tD&%v=1SG(-b(%pavuu;`umpfeB1@$lWtYORZ4oL6#gZyyYco( z4WGX{{@x#SO}gY`QU^d!OxKm|?GWvHz_Y&=F7EYg6@i9c-fsS03EPG5P1&jiu%d8a&WP=GPg# z8J{)-LV=>G9Qk^jSq0UQYM-wCqo}kKlh)aH(S{Ff>@ZiS+G`f*0}Resuq(LRaH9gcAnN8`L>Jl zZBAm?ZLULdlngE7kYh-UL^?V;LEoPd$ORof+t^d*7(wU901%f_EO+ycBWC6Pu~^yi ziPi6MvfEJ&HmqgBF&6AK)EUNeJ^SiRoO(x0Jzv#tRZ{-6YgKQyQ zIZ8is>G)u(`r|joHK}sNwK=9PAsnLjIx-zV)%Vgm^jhL0zo3!%3HTeW0?HnZ-f_cL z56DQ2mJ8_QeBh>PbC}`d4`kx*_y%pcbiI9A00Z;uw4M^`vtwy9Byv)Qm>ug4?ZA)4 zO-!I9Ao~ngE^`$c?kMv~C!?Pzv(2%s{&-oQhwCH#sFHeG_=5zCoV}UzKH;qS@!j0D zRtn!@gL78L{oQ!F+eIC#u)j=)iQ6U>pW6_yZST(kBmZS(AkL3kBt!4=9&a|zXaXHA zJEDR2zusXa>@j$dKy$XJjJGV#aCBohm%WdEY6nuJJ+={5l=nHpq0me8Q?l|iX*X}O zGe=*g1nDT42x^Qa#Bb{wWtY{gaxFc#j=FELWUQz<1jevw}DB-J=upF;z8O)ACNo%4xh|mPh&AvR+p@X;o z5;e}lsnwDqUF0izv*xi9g$WX4X~6xBva?=v$IZoLYL}i0rbC|(3wKlaPm5@m4F<_= zdnp_=M}5P@&`XC&qT%^TbAp%@6jA*#=){37tlNy(+*~@FBO)d#^9dFK0yX7q6+`fA zzc>3BhaWI!VZvNS>9RH}GU}irOHn<(dOp~_M{bXwBIw;6DY;>E-6b z(ljoDSbBh}^Of7}bWp3Zp>jP+iE|b)aRt?M=8)B(P*eq0UjF>+)@ zl~az_ILh<_e8ZUP#HS^{MMI0n1w5%wKA3uUU&m#VlJrX)u}jt2Ec76Lr_BOYRU; zzNTAb81m@@KJbUm-9qNv^F@4>Ak#yAQVOiTPA^{mc6 z1($d7!?}DSgjX>Hx^au=tf_VE%9?(A7VPD1Yj-H19|$9lw;Ru6r@WiC zr?I>|KKJvhpMJx?Gpo41t&Bv@>Ed-u&nf~i?tQD0b8y4|cGX14m=#Lhli@@({i)dD z6A}xK;qZ^LZ>O0=Bn9CMw5Dqip$%<${L@Mn?pJxx!VCaz4D2fS;EV$s=eLNx81tIAoCob0P}(b43ZC* zxK&qfbF38%tnJqzzg0J4uye$zeaRY^w6yii?ChKc%0B@Ym`^&cNEhoJUW~LsL`o!2 zH_gMQG)A+V$01@qwXx{dLrk(aJs2MQ8$MeTIYs<|ew)jyp&0iRcv`#)t43|R;?ikr zMO|lyo6;Fh2l^weG|yizBDK#l?z>r50}maPL*XzNf^p{OGXQ$2no#wF+K#Dz+u>5P zanaqAuYp{_T>BT+}js zAgCzv`_|Yuhek$kivqC0W!~*ku3_=(-7Cty$Z@=C52ac&yZtaJwBK%)_B&1K1wGgy z=YMR|{r7qtHc+P(<*%~Xo!bb#+5t`*oQ#8mOeyYo+lnsTbTm)9G}`gD*w76sO!Y_; zuY(-h72^4LY4kkU9;u9xar>dp?fiI1Dk1r>%!cIFm(^hTjg06~EDcGvrMa&HD(}A{ zXQk(NpOu&dEp4CmsQZi*IZk4>-jtBYPF1RkX2DM9^H$G>O zu9$(oNSV@|bU(#E9Nu<+pnj8H?4wc>{vf>Kg**0a^3wk*GNn)6@47f7>-y=oVch){ zCs8r2-_34!&PdneR;??JY5O6=h!@i*ZFaAjV)(h>Yg24R!fZQyxp0+YSW6-&rrM)G z8wR_sK`;o!EhDP~)+fZY-vS#2_1`|drVuqde!5EG)k#R@betpGt@`{&WuH_WAyP=S z>+BSFqWa+HXvZD>-7xb=KdtGrOh}m_qV}@0-C2apgY_E14Wmf$<7FPRtU<9HMrGEA z4}s*FnV^X1GsG0**Cm=5?B5V=7y4>_d*VBCciO9deuoFrM5p z^6+=PI$ttXmRx|3GD0SNrJw@2NJFl?d#hj57m0?@q0i_`nO<-m6OTdF9h3YdJABI2 z-gnAmbKMEsmC7mvb5UN%?#aEmg>+pI#r(tdE4OT8rayz9G($?)kt56*)PbYFhK zRpPR1^%Pp_4rr}5zl!5+oC%-yJVnaO8ecgwwq3C`PmSZ>rO;k&w2jCtkxC}_YAXd2 z+y>ApTan_)6H0S{a(NpI71VR-s}ym@ph*Xb+vVnJA>@({?@cfWwjp;%^G{%+s=5&= z_hI2D4W`K=d!o9|_7){Gy==U5+0&R@9_yBi;~D*;o9_D;w>fVzDPDyI^{w;$F`?wH zs^*ArN9327Tdm5<%E^+WV^2@$a{b8BvBRXSAxVFb!=3d}^XeOiZ2#*wZ&0fxqBM6_ zGL#NQ$fWF$c$N^qXuSt7)r}dX`d`PT=V5VWF+K;G##7ud`KpR!=KEeyTD4f88?h*V z;xvQ)Xx?e3Y;aK*I?9ONk$B>r>Q8&mF<<%5VQ5~yr=D8_%_nk_ zngunQTyNJ}Y94D9^QwQDUx5jxYKDRhJ71dJ_v;KbsUqzyvST-Pw{MT-ccY{mA}CfD zURo0=enu8=9ZBd8#;(}al_RsjfpE-h@6<|y7=|=@A?LykH%a>C`toJJy;oaHViS9- zPiznL4m~sWOvLb=|i{|tx1S3m2*^eiZZIsEb3`6Rv z#5A1=KOD+0`HI&2>)S2XN^(HA)pqMI9Zwtl z=?x*ZyswrXB)9ty1RPHmz9|UR7y-hI*v#RuW7)v{5l5`dyk)x_l=(p4$;m9YPFN-qf{hB`n6fc-i>Zf3Vn%e74o{Bnz~r2QMpF{m;Pil)Eg}O z0iu3qWz%5fl2HeEE+eca2EqQU+z(Y7qtVaRMx>zK!as_DCekUI45nFXbl6(VO8T&> zUv{V)mN#pEA_*#W({kNo=RLRwh0g)RdffCxY@t!Pw^Tic*WKw@=Wcj&k&l*AF9z-o znNPZs4ey19g(Ml$dM+sJ?7sOiH*Bx+An$J5S#z8X(^&@FMc=b_%(i|t<|M3`OeX^u zHX5Tp@+7By)3DNm7`AkWl(5O@*i3YFuPe zH>pJiQHC~Y?4R3Ps)xaC=zIgJ8&g`z9+;u52e0`4kk1i#oZ`bdFD*Ee$$Ot7MI|ZSl=&N)IOV4Tgyw$*vfMZq`!H|gaW$i|1X{RSf z_f_0r+SPzF!`ojzG=i8jKSfFNyj*vb5BmBXT@cgs#?ThNEAWd45d^Ye-HmY>@O>-i zWU7(opjYMzQ0%R8Z9+jzxPHEkPs_MzG>xd^biSVRYB;klSVchH1SgkP@V&<<4x4{E zW%uw#X8s5!?7u)kIOPptOE4M#)Ms&n?bIXHBX}ceEL^f$ke*Q6g>e-0U3x~}k|G;` z+2iGy9sKr3`(RTR=`!SyZdnxb=bvzd-$P0HTS>wzp?P0)$1I&cC`=w;3$OTQK=SyP zOnf|BJV4bXU*L%j&X=LcjPq;kGQT^~hE<2II)5-!)T%w5d8OBLF6Lg^OM1rW_Bgl* z0&lKDI#gb|_GlKDVrXaPF`$w!#%{)yK{+so@x^*Clm0lW!5nyA%wQh^MPytIo7JL| zsCPBlQ<=DeQ~SS%rgbRRYLZj(V<`hSL1-Sy#-OPYsdd+uh+@Qolf5au)EwsF8Erh` zv;u(Fm{u$!QaV!MET+cv`NJWSas>ZXb1`=>1cq7NtTA}#*ElNRcJbDlLdyS?2RN&p zWG|M3@Ef%SKa0+Yyt2S`iK-tv_58w*blehH-RO1Xo+ssjSk2&8m)v1ru<~BFPjfYc znJ=kL5?CfdOQ%J@p1RO(-ZaoN&5KX{l(XuaVB)&}W+-R9u={y%ZNkeDAjY4ZFEott zGIwG9P+>cE>IxexBP=3@obXW#@@HQ_*}0^72tdF&KXN}bpmI0OVCNIoIiIfKazhR7 znH!N1aW66(a=nP+-|dfxe@OblyV<8|OUA)-p*#}yCgLb3R7~C?FHB`=J*d4@Q|K_? z9@@Q6_G$X9m|6R(SFVeEk>8G?VZ)a7{(Nl;Hl>i-Uqo@gCmfKzh_;)@T;%&Qo7hzrx{MwZKV(;U3dzF%WF)`jfQ$u??lwn2a=H!y!h6hWGB$re+K=yyutRl|g!~ zAqj>wXZOHg0^T=ooyJO?O%zMahJMS-CWwl>=)YNgW6K>naHCWU(ZI(O&0j!c`>$xGAXb?2Yk z6SX$I*!gk>Pc~DKJ0y2s+!=5GF+S!SgZ%U(9eYsa?i0IsH~Ptaa|?f!$Ax3jqJRnd z+b}idtTlqg;jzR%z$I16M@TL7eD!A=mdstNbsnwZRbOonn{Mm!J?9e9r$&HlST4fd zZJ!vD%XM(Q+fW<~;i}Oi{q60RClaDmRPN%5>?0L540`=L*k@)omzz@>Dvt2G;?gy3 zpKm6m4_|MeKW{BGfUhjC{7N}7iB zV-ww-DtD*+ao4>wM#PYvd669FS0Hs=ZReGb-@(_`GBT?Bs!zeFin-4IV+ILRGb8ab zyNaO$MQx7VyxX`1yd=zfmwhDd9Jhxs>Cq0F;E8HeuE*^JyJE?CJJG)DKDVW`el%HL z1GXq~W+TZFu_;l>mOPgl5JZUA+LlC;mNcMwb(~!j}8ZSz*WZ zGvEKm)K^Bu*(}{+L4reqyGw9)cXxLS?iOHhcXtaOEVx^6C%C%?2<|X*hxeTKeD~k1 zSu;=9(^9>wde?5Cg#GYg93A_WP&zH^oEjzWltFM(m|2$()MAR33>5zqy$OmCOc@{V zGtabFZUcE@YzVQ4wzy-)i`EE67290B>~0}#F5m+ja|iX)y=czfu*+mtCm)V|w?O~5 zafSzei2P79OUv^E3E!P?3z8s^s3mVR8Bw4epNR~7!l)}&z%5hCn@@ALOXjrIa@tW` zXB$8f6uKuBFMKFn6vEr-4dzQjJ0)_N_NP)wGAVb5YDWuk93zLonYNLB_J61ryl; z@dmai)zq}^@vg@dnY%a;-ZvAmr5`z%4R>x1{Zb!Bm+|_>#p1(aX%8@1N;zI!t;eKj z7@wMFvEMs(Swm8n92N=Bbe3SH2kvDyn&9+zgcmEd+@}BD=SQXxydzdRQ+h$e815&9 zeT3P8D&Q}ArgVX|hu*+T18uB&LUYHDo}FJKsP11o33x(W52TcG9pqm_Ddu?@Oh0(A z&d_a7>@ILa&=PDu?3Eco7!+f{z3$_=f&xPN!njvZ?k)J@zU>MZpcgrR+~zne-I+4jOkDuAcrt5N3-p@qwRqDO0OOCl7=(>CH zbR51`imMA@&y=BeXxD+@GdR)QU;06@u6n)h>YrdeS2}jLWaLPM<9X=K2}Rti(_lG@ z%%#uN^Ek*9L%W4xE%)&V10pr%V@BCA(#Y2NXKid=d{sGz*s62N^utPcH+CrYlfTc? z)RxGA^~rRZ>2krtPtA?7e3bTcioQp#5iIa*jTxgwG5uivP4HoH7c9|hH9En*3T(9T z{S^}04L5E?wL@x7+hVwG0~WvaB7eB+=UQ^)r2roMxL!P$FC!s7Ej*R`#61tuf0?v; z@j+49-R3C6g>xph&@#iBUi?RUBm^~}d=x|O*Ao3%l#6~OAxu4}vT@3#^)LKhR%_23 z+y}_xBqUt%kUF_gxG%j=32@6!_?gI6Lb+?rJE5YfIBc;^MGpK9BtZ(w4>|}+kLzcm zY&VCCe0iwGt1Tk@;bMlj0f9Kl!A=?Sg?*e{YjsGhX26oWrHD43R#;ZWEFM&!ogPwh z;k02jwTV^F^#T#?9%5aqF78<5_&=1tr}dp82Zs1^Xxb#=URL|bUkW?t?1uysnayD~ z)tKR>G`Y|kHH7e5s6C_$h)&b|?4GwMoy2qCR_Kh|47>Nc+H+B=>3)ogJLm$>8q=uc zTwRkXvJ(?c7PE)qiNPhvw4;Q-*NQ28zi}CYYs|wQ+tRko;hGT)1RwrxL|X)u?K3fT z(@YD2I*Ut7n-Opo3!wfA7KB<1#As;AmfCjlSuM`Q zHjZ9q;R)5rG`l~u{Aw@ik;K`8eH}`dbCUE)iMaBV%7n0gHa>v5&e;bSdN>_36L%T( zhOqc2TxPcXiaD9s{gE_7%#A<(fCSsVVy>Z-v0j+zzy3 z^l5v?qHXwNZ7gnus;OB`G6orjPESXIaG&Zgt$(Ow5i+9{y5rK%y!!A#mCKOKaB0D4 zBftt3ue8?(rrQBF$G8r4}a+}^Rv!A z+HfzGm28>Fvr^yH390P#ng)#qH9St&Y1Bi-h4E)lV?|9>tn0fT22&r6rlpBDYNIfmfv6zZWmW;9geVBOQzLQx7 z;E($$xJemFSL*hDn@BMJb?@m~Z?UFV=4%7Xqtb8yFoYCg3TqFRYOG+Ve-L$=2G!F! zfiujCzaJFliXZPvi6Hp;$Yv=dpfF4^zhg#?6Wu$Dy4hFtt%oOdlC6YScXmIva8U&o z)u|#uG-E()``rJW@L@#W%a+OeMS;8LAj`Rb?AN^I1`fr<(&i?p$Y@L?1K_#G;D)a? zv-5k&4o-fvTjDTSKUjFZQcx${{k-35AYpIrx7drt+}P(!Kt65D)`R^SW`4ZDW#%j~ zR;<^VN?EG$pj7n4Km;7xh6kvHlKv*6n2^_a-)(*hh${E-Q)0Z6mpq32ahs8H* z6W*qVa(f(ZgAa4Fzz~vrZyy)*B&if1mwm#dZF;sv4l=zE2h-LLdb{U3h|o0w%~RRQ z55R8NvDV>e^r-xWWo^FI)tEBvGsbphuK~2D|BKWR?uGUkq^mYYy0+KHoiQaWv$X?5 z^6kSW`h#}9zo{zAH^Ay)fLTNuN?QMLUzNkBcGa_*az4kt#?PNx*|_Ws4%*BD!Y@CK zf8bV*(4@%g`)NA*GSr8jMW2!mwNht-JY|)EdS(-oqy~)`9B(xKNh10_Q*a~UEz`!e z^oyQHer|^C-`D*w`Ghq*fms-HDZM@|B@GrBOmC3K=Spe-CW1X2e0*D?KdNque@~Bq zz)kR`#MrsKPCr0W_w7!1rQV_;q=;PKeIXSi6f^0VtJn4W!}%Kh&EaJ7%&iv!x%RM55I9JZ#=zt$JW1|28&9@U}ZA zG$`z!&l3VZ$mC|4D&nOwIR;HW+G}biD zK;+~Ny+yg5xMa&|&dD1OyW-uOt)6aUcm}{FS|_z27Ioj}9rKQOn3AmBdS@rp(@b_7 zf=#iWCA8C%YSUPXA3eQ(+qL{J#rp2MVeG#X=YK@5!NV6f0{U)(%sT^udH{&uQ5KDgZ z2cJAP)EF5nGKN&(`<(lO<}P*##r}+L zFS(^sud443c1`$!NnT90;N0cMPH6M1Z2#b*9b0$zyI(GfKVGlWh0>UWZv6<_*$7c$ z_ekR7>n=NAUyShN`X#iC&U*LQqrJ7y&aU!Y~c@ z&v*{2Uhn7yMf5J6Xog{&52%SVFh9%qNTwi`80NK~MZ(hEn+!=uyv z&~<@}3f!3);?Z+-Fizd_y;0zC*^ijWU=a&LCZv1Y+m$E6=W_VOi0ri3W53496M9p9 zG?i;R@fylJ+)zcx=k#h~S*q_k$#H$Z=C~(9pGsfb`U{BytTe$EJU(BcHi;$D&Apo> z$CT>1>8@yK?}IY-hWZB2q|Vcox2!9( za)jwr^R(GDT0Qywe9nXuGm%@_QqZYmK#_p-^C0Drx{~N-E-!O>-+(*90dF#EU|e;+z%4{f~R)}G|*>L(Fzj0z_P9RtclbTGSBlt(M4v-3tu zup7U`boWH)Xc;)r$(Sck_mupQb^465Hg}9dh)k`Ni3~2x@o_Lun08J4%JK07;doyORpP1NPd5 z71_FhSk=UgcsmksJ9AV#dDXE)5&LvGLu@+jVT#)vUl+Svx{j`bl6ejj@3@=YbV=9t zwi*5tAM%3zk+g-MvnQ6Orz4@=v#4X--KOBC8g~A|x5%%1ZXh=!K7 zom!&Z+-%(FNFuqO`0Ye`tkawV7cSIRn`?UK{X>r&#uPbK-_hYAu0?RLsUW|oYh7GY zct49|ES^OQMAqxgwW8>z+8tTk@Hji!PpUyRU>qS|7~uya_HMX|c#8uvzl%+D*00J? z@ICPq$O^1avpGs2=(Wc?+&|b7+rGjO4cndM*7c8TO0u7*KHT_UawPav_{N=kQr{E9 zLgj!iYwrU0RWvkI)B8?3xx<{cj{-Sg$NR9Dt?FH5jHFeEMd{fuAf*1_R2 zu8wxPBM!)o7_3GywP!ttLY!hIg~IVwY15SkF6M-Lq)c^vDc$+xW(PiwRhx;vV}_|t zBtVVq%P(w0_ux9JFK@kX_rG9ermm&PIlk8&ZR?~th3&&c>i*94+@xjdy2Y^Uz%+lp zwI)Kyah54p;=d|h_uTTcw6wbq$}1!|TC*TokuU0Cb_OR(=G`n4V1Q(m; z1AK}aXl)gik;Z{>^=tHfy&9(B0;dMa$$n8HGHL$|#H&O`ITE#TKPGyN*Mz_4wCT;M zuIu(t*ed<4B(dEo0b#~DOpsUbFrD440|yN^9v1B4vp2`S!_L@Ma)p01v#3SDcJ5WB z6HmZ}OW{H|@vz*}?U>HP*}_BR+k#{YI|jC-JUk!D6?*<+2G7hW9HlB z@@poM`{a@?@>e9F%oul-3CZ9t1;5hV4DqmT&UFz(6k>5Y^>!Tcil0)v|wR zWOS58d|Kzq?_tAM{si5K2PWcq5F2D?xjqrTw(FH3$-Nmg*pPa{pPs^w6aIV~ zwt7W!=b9&1w-l7d?4*F9@HbaYk z?3Hwp!rUlI6Bq02{{k)~N4FZ=vP=84#P9}l)KbgN_NHlGA+rhz$><3_U)*rHcpzen z3$$zL0qra~Jk?Z3JF+*Fxr#`^nK4543EoVwiN-&!Idl;E9Q&hSzR_?yavK&&2W@nC z*783eE5+*Mk~kB-{;8~G-DQnGgc*wNf97AB2R**tEdRkUwOqqC>YU-8`dyV5-@+vD zcI!BSl&|-CyzROZiv6q2A+buk?pMa^iNEDS++kFA`=S?+iG0RaqmASp&LZN!ds76j&}P9ef| z-F&Cyt54E~<2+p`@br3YY0+sVw_VH~``$o2t-js4+9?IuO>d8zby2XyNY6f5`Ej%s zMiHfD3y}S^*5fb4zp90!kjm6u$VL>ILg6zl@DLzPe^4g$RzW&*TH7;~VCyt7)6DyE`DG*(`%fM`df* zVchb$z}x?k3!$MLM!E%jnemFl+38w#Lwq8aA;MZR~`rQ+i%;Ep@n*R_Xofg3y%9n;IHMal$?mf4x7y@Q$ zE_wS=+Yo-1`1b){p<(cNj+>@!QCEdkCq=BztF!T= zqJ0fN3X>9`rsP4X;6+87r+X()V!o3)8H_^qVjB(!r6!vDM-J2x-hRNlalz0zu@*bx z6IYCS@)8JS&2ALTKED&d*wDU$DK^$awbeh;&T^75z>0`#t_Rtq`I0IyQ+zOw--~lF z+f)O|zpwEg@2J_wmLHbEyCJ{ZqL&*xtuv5s_!Q26dshpA;CMoaP6zagR5;>}X|ZLG zll2CW21vnDi)hLzI^^x|V~@J}12leInNt3TPMpI49xk9=LzP^7BE9#P$kWCR)9BL0 zQ&O>2E;Cd0mCfGLQ&VDNRxFv*6nc4eHM$VZfT5t_^{!m4-1TPgVd~hCKJ(}I-f!oJ zI+wZ+2tyGW9ep_fIs$)=LzjG3hT+A4?7masURe4MtpuNBrpDZHzdzt0mP7hKHvDIR z9MHbHm7^T)9Z6y^X#6`EU~cG#cZzLkzlVq}o5sJBVha7;Y66+>ga0F|g1NsW+hJ!5 zF387M^UaXR>z4K;pIJLdH0al$T-}O=ww;em+A*t5yfyMjT(NV7UQuJ|`wrE83}~~&*rnT>r`#5ga4iH=8zn2aJ#GtN z?%)$KhvzIFN|99!R7j~==Sm#qV{Vn?Q(8m3uI?VhZ1($H64S<@f*IBlspY_vwvp+b z)a(JgXV7#7YnOsq<8Ns2$ngoag0Gmk8QkJgh={_!a8(Uetv^SOdlcDx`%_``W35E@oS#yRlyo*Fee zK0Y2y(y+|ryugs*`Ke{-bv?XGj57}~{yG%}@$Ok{hL!;D;=2Y|kn%|~r9w~Vam2@42TR2y(V@9<5S)mL`iz2)meb{Gx^QKCooVUmQ$@t2#^UpUMX!%QCRyzZ z=}01GaFL%3lnJ)9gbwb+)sD*wa*K?jX8SD?BA4y`X2xB3NQi8Hxi$hkk~l8;zPKcB z#1`U1<;0l)j2tY-BN3Yu$>KdU1uL)SwX4!dCw*heqd3KgE6>g~@Dyoq*yGA0mECLDKj`KIdt&Zup4U^HkH z2oB8UHUa0;7!Y&aJw54PjIO&oU)PMJEx*zEB77SofACW=1a3^1{4D6DfoQ)@IqK%abvAG6oVK()P-h8a8 zZFldK{J`SiCx-dh&RYj1d7?1b)O|ljKZt{#xr4L%-mjk0Q#shcMFK+^m}X8((#U#89`shT->5=F;AljK}*OY*GkR*MlI>(}iIBs*S|f7NkX;gI~j z{BxJ>YEI`ZGfVBF1^wMW78!-~yVt^JtncnU%W+S9-Sm}3FNYWqmy*#3X;;~+@Gbsl z-C`P5cU_Ct$2nybxJ#|b^#4Iz31?pDN)Up~reJH!8;KMAATLEnn>RD_Y<&V%- z6}c+-ZPnOkxCbwz6PP6ix96sFv6yFi`$|tM;|5j({LFS^`t!giDv+N8fBewpKAlY2 z^edtSE2<)Roj1rk8quH*Qr0^!sA{9*;cD4)K3Bju(!SfQ;3_0$NpVZmC$K+c%amM3 z1nSoJTamzsflEP^^Oh*_Jw5dtp9b7AwCyH+@M?GS8N=P4UyJJXjuhq?Ligt$xjRPx z`X}FGKcEV@Pwu)r$?&kwv;}zOVs*RqdcEhM@}1fi(z|VDnclja@jUPZc_D6Iwe-g1 zGnFYGW)z04ogTYcJ#0Mns}qOV3X&W}UAt6Wl}T?Me{Flps~r#3GIEs8!SWH}hR)+n zWRTE~ul-Ob^f(f4K6ds&()sb#{{oLP1n!w&GyhAOiy{fgl@}&I{q4ZFx)Hx=b_IT+ z`{%qdqI`S-NhUuaMxH=Ow_b67WS3G7x$fq8yse>3#!H(fq&(HXr)vaW43n_QtVlpC z`XThFnn|f5h*eof$D-4|LPIUlnBd!?i=+MLbZDnoymq1c?r(d%Vw5Q#8$tqFb3Q+s z2*j>7|GHcAoVcM+q5E{B`WUtuikzlV$Rn0t+(ls(0xl zL`S52(H{>s+Z>C>mg}oX=Ke(eJHdyv+fk=QUv}HC$8}D3Z!jNxY#&2UfQZ%gL^p)% zZ}*#zX7KV7v?S-BP>gOC^DDt)zkua8_CY>toOju^%YJ{$7PBUF91m^y{oc=96?fYg z6z=r5i(J?qK8_JO{T5;HZtO`i`I`VXiD9^ZL|Ijru4MV}mk{sIlpQq+^)cZ~q|YRW zh(}XNkk0FWcEQ8+c`OdBUv2wfNb|u2EPN~-`FBa{y09?ZtWthDuP-SniE5aw!F>N` zAqEx^-3O;%V720=eQ<{2gYjw#0_zuA^rsKf4vKkpE6s8!1d4%J#Ky3YYCnNxU@xBg z36d$nyUq9VqDLgNmIG$Y-Gg_F0dVtq&u&;oLO~(5O^Gu9@HuH$w5q6wNl;tqC}Q~K zHp*%>7k}wjp+kw-lazj#gyZXiuM0l$(xs=JHs#tPBiE9bIimu^>9GzfB1GBCx6YRAuZN#J&$3#EAmaKYt*ynt*8OV1`jY6{^h8~G<<+8`2C2gX@ z643=k<=3GZ?^l^I>3_0#x)1DdOfCob$)l)$T6ElG?D76eF}v~BrA5Bw}&@16<#KQbcQ_Oxn8!irRboz#TA00>WNYlzUh>9kyMiT z#D+Xfx%5JPqbRMS(p^ma&TI5D{Q~K>4yHZ3eR_{w_9h-{QVF0%iCFA@vgBS7Ese_e z_G4mWGYI)U3L*?Nr#0a>j|}J6rge0C1`&s*?6DdfutdbgtN|~qh$BrP=%78 z8p`_nm~@}-VpD{=UX?Og+`q>Va>&pi4EK9_x;EIEmfa z=)B{q3<{mF;B!tO1?S!xXjW-YdJMmzq(Kose#l8OJ!?C% zGYTe~NrZR%K{Sl(=Rh=Z>h73=$N#keyuo-oU@YFr+8UO9p2Mv(A~71n>P> zX#^F)6_;}`Xu)EswOK;^dJp%>9`z8NstG zr<{CKn0t0IlV`Fjda}njtj!2%xVuyB$*H-5ZM5*CrIFo+<6?x5*}0)kO$DKX5|{M} z(&IX!e{x&;=9iwAz44(;M?L}|D{Fm>MxXdB=zrk@W6V1q7_KtWnb5|I%m_4};PA{^ znXIHab+~+v(!k&LGa@lrj(W&80>Yzs>m;#rZx*k0)9W;!Et7NU`-DE(KMZkdB&QTv zCnaT3T|w{e%eCl}V>4wiuw3px3>&V`Cb<;uqT}o@x15fsgZ=@g|3qO_@8L3`7J8C% z=!(Zt!2#Zx%mHz5W^A^jmp}W&?E&}5EIakDw=Pt{RH3&6;$776 z^a6-*F=BELctUq3Z#45&4g%EB}2&md5CYP57GmQMCDy>G7vLUtLiSi42W&hzoEo=lQ?9E*b5f zGTG7*ra>FBTPu55sHC3pIkgJr0seO*1-LJgaGwXDgF3?_(_}8SdDtF}9qtPFfq zz^t@fFbN?YnE{mQ>;_9atl0-SSkRF-6E-aV#~vX&vpndBznd3%7dbZq4|x~o50pha z#uo)|)vj4Ruz{^n!t4WrHH-^Y<0Qst%<$q8+bn;KXE>FD2n@`q6c*ddm1|;B7&P=T zcqnIoWZ81rNk-W*$?WVfejZOx^!0X;VYYEP+wfF=SAEr>OG~DxsriGNKyQWeWOjfz zn%S@R@$(2Z3JzHL8u9vQkprRyHeHCQPd76e{FDW8u}=g(HjZGCO~%6i%0tYPw$Jk|o_sd`EXp z9UZB=yMY+n7fiXlz9gHTz5`4o%DO#E%B+Q(y$R@hy$;M#qZz_mkDgYrzGm<^rl6q$ zc4%A}B&fQ`-e{Bls?b8@A5X5?}d&|z4kgMA(Rg^W=m2QeD(nhl${&#;kRxXb?E3|8h z(;B+rWZsqzDk(KSqRJ!EFKLHD|CxGs%OPEMgwmA)qwBqJpyW%YixYx4(nKf=LC zH;o+>i){Q+?bJ2_>n$Y;X;y0)K}?xsiI6aoqs4M{%lyDh?_qA>3=R<4#)U$#!-!~U zi4XbJSED*ukMS{^kMQ)dT(hmy0|~yhwxq7*e3?N(*N?1-T3)TJvzZ}V_OJfR-CYQ6 zL&@6_m~J1pVKLjd!s-JwjTAoXW>SBkxc$;{p0)e|W;#{ zd;Ri0Ufua-<$8iXSU5+7B+A^8d!GAdBO??C+!vsrfY{h-{T^pAMx!=52}JCKt0LHO zqcu<(lJ%T)HkLxlBdE(N{}W?|(zD-!M(I=*Q-{YzXtBQ-ohA)~Rl1K1?D9=EdpT{3 z!xof?=Lbkf8bp9jspeyZRD5Ax3p&0Yens>QxOfD}EI(z?O2g%_i8XDPNr}!5&yLVt?g}~Vd#bj3sd$G>+h0EANc4=3{1%cO z#gH(+Z0TAk1>txAldY!4ee4BOS^D9m0ba-0^hSF9cVBBk_CR*-0}JI{AWls0KCY|z zdw5nU~$_tSyqo8aZ+4ztTvcjPIy>)>cb z%@~w{5c22k*Pxv8pcK5=#bXO^y1-hhVtIMfr0*hX-f-?0Q@XVRemTMP(B;R!Wk z+Z=L%mo5@iZ8St?=Fdx~HqU9L`)8t%{kK>6*Bp;KpYHDGh*w(7vrFfz*@Kbr)NLV8 z2*+}UR)o7Zg|@d{slzT25=C;fL6K<;CcLsGJ1i@(55j>ljOvXJ0hSO>!odcD(4c-x z-`AOY{z55fK%P^c&66C;F=1Bk8$7Y>YA~KGeP2wlkl;y8!bpY|x;%_EYWh{Kt$whC zj7W#gA|;^rr5-<$QL}0#9 z3~YdaH<^Z>(4L+xaa{lO2~Tvgtuv=?=d=J22xIYHtFgR7_pHZmu^jf+&5Tw#3aNQ;@!*}rL<%YKV z-r!u8j`FY;E-x<=1}^_vG&2ZMENyo>j3)o`SyIf5|31=su)qK19)Xb3tQ6@Fr2NHt z%E;J35DWpuwcGJkpBHLS&`--X9H>QkvwsE_JW_#H@Zw@;`F@YhMl5HPo{UPgMx_~w z5>0+fhFhZ=pNdH^nH~0pN3;1lxG5XU$1>0VymW6A8wakh3NbS)1GPJFacd849w(Lr zcxBsE2lt1n`65CU;DQ{Ps1SUN3vOr*3uuuOUzN=3_MO5 z7>({^>J0O{yr#tDa?8a9?i|lviFx-p6XNm!+9&clKAo`NjJ9<3K(KhGmy9n-uN}tT z5%{28X~0+`cP93dph3eA z(AAa6t_yg(@F)E#0d|`T{E&I7RG0t!{PTA*2jxLLQkv?52Qk9(`nv0^|l7@;DE5Ouv6O~^TcFa9Il3Zz34SG@O%h6v{0 zCk9qyhPmz!*ZV1Fc{)xG8Qn(BHo*YE<)yQq|M?KZ)AcE7Jq}D>v&YoQ*;#RZX#bl3 zLA%fanEf;UINETheP5NCavUyzJFQEdJ%KQ7l?|K|)p=c`^P7Yr$ajDHD8pYvQ`i&F zK_mds_4vy}0{}qI@Gi>+`nC-Wt~^Cv1muCWtN-I6z~e#=Kk&m`DF{H!`4{eIrB6YM z`P7+?`NJl(Y@-zi?QBu3$;v#DEdvXGb0SABF z&1Yp9rWi>kj)x^9bD2in%KM4qSdukGA0fU5kCL*u*c?sfify$U3se_4_nM411>ib` zPD8xG1_cK@uP^^)=*_eO`?&Xkh(Lh(PI^zF!ZMff{rWofbT?FrYX98fcVjok8g#;l zPR?#}^W@-$K&ED%ZjW^(;~0vn3#asE*|dIwexALz&Xvd@ zlyxrpRv-;xEbl&?@fDYbCQ!Lke7J@H2UmM-Sqn%-YoUU-m<6c4;SBiw&X;Mo`23Qv zfw&dAb&Z1Dhpbf~5_!btM0P}ehphh@Z3-1gqj52&?ihv9BFK^j6Bg;7lZz<>=9^F| z{76Yc@IUDSoDbPwi5~obkRB+QGKN1}KxcB7MUD0BaXBy5?*ORP81|4xVo;^T7-OuH zNX%h^uqdq4x{5<4ah`52HljG zJq-9n)+`9dtVs-VRHG;exmV9ibwoZPH;E?PUY9f&XZ%5B{wNBi{bcwp4q#=bFGi`y zxOqoWIB#<4wd3yMsOxR6!yy}k15B$E=nPQodV@z zUR|r+CX9lGeZhD{<6;-4PHWql&6LQOx1Unnlwfs&c()Jvg`JbouY8QUj&cVT8F4|O zdbS-fG~wR3)GrthmkCd`Tlh&S8M6OLdn0NRVe@g=vv$tAoJISmS0?X)it3S<+eh<7 zfyEX}R-&o#V+f?$$sZisES1jrH!h5)?#xe$I2fQ~XZ>(%Fe(xy>Bf9v=Kh=1R;$+$ z#yh5}b{u+~aZtI~N8yQN(j2Gs;9%zmXK4st6C{_cro^#$AY!C-N7oXPBu!Lr2YuOV z74B>4;f%Y#Z82`oI_7ZvPWL3Mi5y4fcxQQN1{gpE|MQS%A?cdUR#J}yh!J@0NN%Q7 znEbpO*%|UlLR{F>tRlW4=B|-FiN9us_+U`v2pb5`yl#;oZ|elAEr=Zdg3V|k5~7qa zU#_b1L#u*h3k4|BY+5^fW>~C_CigJQ1zm|tnMVr7P{E~x0H0KePT73_)+DEOl5Y?A zrR#zv7yg~-ueGj&AHIMfp>GmJNCZjHLYx+B@;wvuc^c&6ySGM%7}hk%wU~_zGr3%Y ze6LmLby`f4x%r#-Zwi{HzMnU!#S%7o%uiLU5?k_0o`?MUodyNh^v%xvaU~weGf{bU ztP+w1o#_*7w$e{Lf|AtW=qG~ zd@hlj2F$i-Y0(-M7d-X}QG4d;Gh^$g5f3S(x#M$@KWT}a{61i#RcU0#iIA$dKXN#^ zUx47g)`bV*cX_d8h-`D9fsve9`=1Tls=G-~4sqb5nT;en`~V45N-^zkHj@Q@@xRJ) z!)ZuUt?I=OFMwj%{m6vlALezxq}NlCce-0FrUSQ1>?Lu{fu& zTdx?_la#?W*s0o-E$6eAbbO*sKs9NKq%d1$Pa*_DzNm!QZU&Xu(BTNU1!ST8S*Rc_ zL6_v;JCexCFVZP$cY+lpSM$$;JBMJ02SZO&gg}}B!L&t_QjY%s{8$K$Yodr&lMdgz z9~k#!p@<{b5)FBAgKjHbFUU|k&52-PCAp62b~#RmPDMGU$28b zW8i1*h&ka}eyAsUHwhJN67Y%G>x^My!tNh52}Fj(N4q?A`dPweS10=6;84Ml$NMnH zccwdfbvu)}QoaIXB<=)8a=S1GGEva|@QaPMm(6vhoK}rB3aU@YQLaRqoDFQf`2@^M z$HGcFI*)WFvQH=HjIZR$dbca~iePOd`IuhA0H>>6u|l%!Vhu=bwp5XlVbhl4e6@LI zBF_kwAWAJ3?`o@C?&fJtf<%@-V@GI48?TJcVeeAf~wZxazEf+`dIc4j4;ZfBC+7}~5vekU-8FcZ9)33DJ4jWnM`f7hx|D^1BiVo06Uz>_X> z@ayFdk&Uk*TnLP}Wu7EK(}NdJJX>JkXx214{O_O!LQwuqE}I%ta>bHMk0TzpQ{jSoboAgmUQ>JPDC69-%Aj)bpy5XqHy#KJL*erZ;BqdWmNK`bDvLTH?oc0VH zIV6UR%Mm_2-#&^G5E7JFHp#l6V(=pf-hADk$kQl|!7r7UKn{8M5FE*UaX{JhLnG~5 zQ{`wCPjrHD^qT}D=^N*bUxp=GgaXWWG@ocOSv$tj3H$#_oGJ)c^r1#tk(|V{`8R# z+gglOupP>L^hD*}(wIlet4y3-f?t!~Vgp}SQ{Fv$=U5czx$!t7=qp_pbFPtnQ zK=DAT))$i_#!QtQlKGEiyNa+~%i?`tyCRO@!I=D8L62@(5a$D!NBtw{nyT|}MHqFV zLWJAu>$vteDf43rqT79|NLwrQ4s%XLfMTjZ(}%_jxS}LWyyY$H_#^LIMCo+K*jJc% zKO@k~Z24gLk14B22VRgfBh{k`OO-h@vhqglV6FVvrdM~z?&3k0By3k}lT!oUKPd3h z5nkCq(R4h#N65yypsls_*oeri!Loy*(8|n&u5=O*;*TO-YIa_hAGm$vD-^P%GfWEC z_a%Et`4poYW2{WXHvbVn{C1kePgN5})d(mE6Z|yUh-v8l@2T3)=6XUWV zFDR)Hpdp7=hHVdy=^@^sly26)xG?Y-?dRfe<`6+%w~4yesDR%bgA0yB|II+YKgJOr zyw*2I?i}HD*kb+%Zg%0JR3hH|f1oK6JYh_*{twe%a=r)1TZiVBcK=1l@8+8l1%8d_ zQrRDVT{5<0{}+(x>G)=G2wFJ~wE`UHNaM1Acqt zcbLt>v=UwbFHu9VQv7YRvftm+u1XcnicXF&0)0OSQ)c*W?$C{|nlC zNiJ>R2Y45dM1Uujv8U2oM7aa8PsJk3xIJT19gCC2qj33hG#P=J$zgJ<>a^y6zYCd< zI?~6?gKV~HPb~fomLVgDEQwvq`Nb!Xb9m}y=fau>1-x0?Jl@j2Jh_Jw@Ve-e37@%& zt~!EDT+8nyWU=VKtTb4N1R`if|JZ7mKWvM8^%}h@LtxIkZDReTH0`&;e-1+36MSg9 z)4y5z->W*K1!||B$uNWAGet>ww&=}KvjMC1v;NiHT`JGqknG|F?p=cgVrf)#a!G;s z3+E(7nU&_rtWQ7+y@7srpd^XekXyMiI8-|mpb%2bFY<>2Y$*I$s-e?s1~GsgT~cRT zz1+sbnw;RP{Zcc}#D@xTJB$&9PfglCZ&5QjZ414O39C09%ke&74RJdiXDoj=)fjTC zD49!tQ#pNZZ3OfrHd=2A*ISHIv|MZ^Gx(n;iIDrp1RGWBB#CVI1VSLkb2c}--FdI3 zD20Pi<(7vmi%OG99CU=GK7=f3#~}Yznm#xjE)n0(GW5(@%9wSF9e&S3m^rvo^v18e zOc|skbdu|^S|p(AaVD1D*^Y0Dg@@i1Y1O=_VH5?;|1q+xfS8&*3(A^35%G1koTj=d zjsIO6RzhPvUaMNySS*t)%Ig8rOrlhvt0=s8T&e1tn^fU8a(L?AO8++1@Fh8$%aalJ)}9HM*}O4_-%s+wthPQzDxO#-c(sYdmpx39jje`MG_Eh+dOu^- zA$k5-P1Wr^MCV0!GjfO_2hsJ%Inu=3x)2GGdEl4L= zuhUg6yG`h%{4pB5ZG5)~GFf-CUMwwgv)R(s^1Qy~v3?eze(IF+Z-bWM!_EjGNO&EE z)LX0ne0ufNa*ej$v{XXx0?ic^RxOyy^n&u6vAG$cSh}!NwNh8YugCkvp}h8> zlb0Hmm&z^S+svn*jQ7pCJ|!}zN){WX_NzTu1c)2!Ndgvhw*}tsIlN6?DDz)B38m6I zP)V#BvR9~`Oq?F~suiL-nXG_Tc_Kb}ggrAfB?cFen*>%53DKNCPK4k03oD+7r|B<8 zLjF??EZkI}wUrf7Zkb>X)tcQd4%K`b+udDde(a5J9;XySad=;7`PVHb!x-v)Kqm0S z66WXEPf3jSHXgq1izBDs=RvPojnr`_EQwhMT?ZNW@r{fIk8|DAL(%-xbMceWp0I&m zDPC~&`2Yb9e@9JKu3OLCrzT6@*^J-! zJS6ds84werso__fU3{!2ePB|6P*3MG7P`3zE+FYLQnpQCx5oI2jTv2884W(F7<+qp zpd{JuZlzSJ@!5biyAtemCE5|(4)H;13hWc<+YR=a8lAH$Igm0MS+HH|b93q}F891Z zQYfyL#CiGvBRvxg*G>iX8AYg6l`=|pubtyyKGSu!hH(s)}7Gum|O~?zqALCQ|?1vDX$lscZr{9`AIS;)OUf~Kw z1tlEs&Y2nJ3Zt^DThI?S&h-4-ze1u^irGaumsPYv(vZumY!>qI#3XiFKb(Fp|NK>$ zhA8Gml^!}QHUFEaTb7D zhyWkN6A=5yclBa!yOy34BU|EgInzO2eC^q)C(g}^94s^8&N9tIU0tinKYOkz9HeHE zT*wO~0(PYC7G%Hq#>EZ#xti9h|2#$rFb@ov5hJd}tkql6eAn;&7_?3^87g^!NhDOx z?x||ZEiHd%kj1&BB>|w=`6^8W%PwYTH+dTHJq>9)MbzUVD!~-=^6~6XB|&YM$B}Bh zfE_4r5$)uNr9rvbdIKw&Y~GSrBv28G(QVq?5#z|^#d8oOZKb>f8cs-s&}4Z2?DQDC z^0j|N)4^Dx0c)`mJ+c3$8lz!*mFP!wgV%A^(PaAY^$(akD3)d~VHP%j_q|?F2~)vy zel#R;OYRRlt#fSr{L$C#ncqoRL30Vfu411pk$Ue_kM%9pG2&>e(Vr~IDPACU??(L| zZ#mW0j$r4yWJJUC;)~uLw3I2?2PWQ;aAu<*>Y10ziG1wQfs=;5OPFmx_NblZr$3Wb zwE^$L`+4e{n8DWu@Fa+j#Vr-@8n?1fjizYHrAZtly%M11YZ{vqywl$M{_elCX01u` zot$%?y+3;&Bk3E!Wcai;US$o3myxoTWD-~oaJ9M`G3owbBEukVv;C0it(S!A=XWe+ zQzuAzf0C0vicv+GhQa3&#~9hfRbKR0h{i$UKLvnzo2y=59O{rM*R9jWsoXZK^)RIQ z_t3}WP+_0fWLdcNHGzM2X0!+dujiH=M136{RJDGON~gO$jfGqq7&<>w3PW-NXds@T zvc9O)V6L+q2FCMHYt&Q!n%7j=vFt0qhcyHA4h7b?=bQp`KP1t7qq><1p6GWM{N=AI z8S#%`X$9 zPO3^7uSI%_6M1^Aq49<(gz>FOq9{`ovyg^#Sa(t$}KuuvtNfVEZ8_zc}Q%_&>+^NT6(UlH|t4#_^rv|5ylJbsz zk8@Ef+)=Alhu_WgA}KPvNny|`5#)fj@2k^h@P9#0y}5Feg-IHMlgb)R~YWm0pG5Y~g|;zoy`v*HH7NxYrBpwHzu zf$E3FPe$<9YU*C7jHy4;>Vm6gYGXkq-b0%M*F&T;lUh#7z-3yV zNsVFDOoCf_!ZVPF39jhg&bOp;I74y)UEV$BvEA;X*Lt~LWY`hepAabuZG~FvP4=9I_Y`*A-}5uj~{?jOE9I%5HccH!-TxsR_XVh8g2)hR{WMJxP~W z%B69oY}`B~OwNJ07>|zY;lQ?lN6VILG;rHXYsaF^tl@U;gb=tjk3_k69~--^X74QZ z@QEx{Z_=}ijs1<$0cRu>Xf-7SkS%zEAj-3-<6&U2s_kpU+4I!tI5L20xFn<0J`*}&d)L%W9J>W}!>AT30 z{%}G1SK8N!jrqz&vO0gn^r~^TD=jw4x4!X588%fdFg?tTLEr#nwbwp)^L?ij+js+a z9`_v}T5yTYO19;$Ec;V`1Dv?uQP(<-OWm-Aklu7R#^e3k=!oQKDQqv`uo{cpbIm9o?g}<`o=h`W@VY`VDye z8BiiU;HOHAGWh=fc=%R&uK~ICIusk9dRgw1BL=^8gkxg(1TpVHI|vs?C>Uken8e3f zH4sNzlHK>G(@ z%l?XuOv2mgJup5)iC(v(002(VW~egv>Hs;nPx8%x&_Vy&QyQx4&4yZYOhdvDRWN|U zbzeYn=dR;=t@`ablkholTeE^5bf3`q=~87ORKQRkzxn(8p(`RiqwCN?0Plr=Ie;hC z26RRgVI&XopY?k9q8O{%X8DOOS-5Szk*Gfi7-Vw^+;nbiF`X&ocQ{-Qc%me1}?BL|&p1vXiW8qcd&BU$}_(B1pz=*!Vno!+?8Cs6-^O^?DzLz=V+kE(pehi4K)9HaEzGg)+&0j?n(DXpvW{hgZ#XP8R)^0Hr>P@)uhF`2a;~ zLA4R2`J&UZK|+EBOO|*N(hj~s^w@?=^)B;<2DSQXh|tZ#*@+ThR?p3<_S4$ zC=l#pY&d=(A0{bgwEO~fw7nA)=wc<~0L;AU^h z|J8`Pxgg-xJw6&2+RGLS+`qD|P99%%8DFzgS)M70o$jDzWB{~N{fJ&S$jB1MgM#L9 zXkD9bKFkZHu`0N%nz{o61W^)Zs0Lc@eDC7ysM)Bc6hoF zVOubX-&Rk%GOr-O9FF6Y_ZPDoJUgay;Cc){Xt&o|t=L)+uBJ`NnIngJ*xx>Gijd-u z_hwYqTvvH<*Y;P>gH)y`EvUUV^N-myB?v%RIYH$$Bm^V}f40Q;55Vm^V%|IsnT@Je z3}rs{P_rjV@Jjqy-sjyupq69pUo-Yei9$p&lFTMp+Cy&83GS^a4|3?kaSK87e7LFT zNW396RtO1UH+8*Ro%% zI8J|`Y$%-e_Yqd^XeNZ=FOH}x-BjRmYQl8bn9ogCbu}FlTd{^oKds1>{YFwHY&a(0 znlg<1Bw%eGBzT$M#$eubG_#%`6+Xkg2BLWdSpAW90>FNJ=oIt*NAmj;*Kd|wq(}>W z<7@k=9kWbPm@1laeekB5+sV6V3+Tawg}I5_+WJ+OX+~A#NMZYf*lz9YOsA>v;M}|8 zV@K3oZqBS)8xW1)$=hg!iR#JCWVL^ZLt3Vox8&YeI=>-T@R_6=0^(Qw4^8IBPcvKw z|M^oelA{GDRyjJ3n-6;$O@@)c$c!Kz=hcpYTUeCEkTBodYDsixC-=vcHay7|sNUW9=tc6>y$H&V{`s*y?e3c$e^%Qx% z^8NYlE(&?3HgN&T=}cr8lpLDv0xT@7UCl%O4(WIrO`6~4KB9}lUS)~lvkuhBLQ})L z-@&6fK2L-M=ZCVeFW**DrAsw9$#;m)CbR^=9ntvIWQB}*@xR<`OL&2K%}m_8rXYmI zWAYzg_xCR#3XyWt2Q`^l5IDJX%3Wet7DuJG%s8k0&6YHT!T@sAbid`8Z$O{bZ{4+q zY<;^iSJ?n_Wuhf%+$8XH-8*Nw^i*6pVC1x(RU}SjR!BZhXSRTPu-OwfcvWlP%;;wH z2~Jo5KVDecQI&sn!@e52cpt4buD*d-Qeb}C;-mQCZsE{m6SX1u_{uu6 zg+@fdVFC28g4=?EFm572p;u<|rDEa%aF_ zr@^*|Qucs0Wik!J*BV^t@+!D`tzJi<$uf0X&vVSX?uTl(HJJdyWGU0{;nO_}I*2Bb zU_Wp3pC=x;EES72IvN`VuAr%!f#73Sp5d!)~c3B>;A(wSLhavI__=iH9atgg>kc%=6cVTHw0*&iq*v9CD z804QOu!oA0d3gDw9?$u2fZtnO-;Sosc(QO@2tY=JKN2%8LA-Fe{TjQ>2^QVKk%Lo* z-(jZfLR`Hl;?XQq6edmT?T-89)6?tzpgV<$n}h&T8fbTE@IJ2}__$8~*p9>%(3rqV zgwEoF+PE(m8qGy__ZxXxOXgSwkuT^Vl+h~haX+9LTc>f#M1ZHplhp9#rWpq z31X2@PIJw;5s)Yg{e9e1ZBr(KzZs=2RtOZ!Uz0bQJqn?n9t?I~c)MApZTg~0b4$;U z58xt^e3sGWc4C~yUxHSPSEg3Xp>ojU3=I?^UTeC{k0#tuVh{e7+vg?2UaC|T)(66Z zsRFn3=bxLtCPG2+QBukuvFMamA%{?IR!U$V11+pNzV4U4HrNtfz+I{JPIyvM*e2AwTr);TJQy_0z>L z9+z-6L221SZi=d9igZSPF9p%LJdmf(DY*v1Dl`(NE(cF8Hg5r#*HfuFGkJZK-nR$w z5ypu^&xLK~6Um0XAEjmNVBp$H`w=aa(9IOEC?wDezvMV!xSjJHx7uqDrVJEff}B@7z) zn1#j=E=tLMVnO<}`r9DR!x7}pTp4`{n(@LBafd}As*!)<7VHuUcXWxM;GIqv5`@z? zTMOLz7M;aokowxG>UwyU{3SLP6t4@AJ_=yBeRWUK>|r)avelq3A8D|!*yG_3Ru4*+ zAMeyZ#&CttOJg)tcNIGqGCjjOzVc?lyBUh(b9!6n6d~NCM1}&FeQ^*zr>9ZMQ2{T# zEhC|$5HsEAuoCWoBkMnsNe1K0gcVip`}170(FI`05pty1mkOXMibRcHHb$d;Ua2(v zr0OX##VOi*2Fz*xD8x~}7M(BIA1!wc0aDz*CS($UI^RmY1Z)y1+3f~cRLf)t==3VX zum%KgZdrgPy#aih($?w4ib<-Yg&ihOe`MI*9>k;{Xh71%x+=~(90JRV(`ITOfF-4% zh`OS5zk-N-zWc~x;|u8I@qYN_Ucm_BxJ@u1IyqjjUi9hdZU~$GDKg3xk+|7rp*X0M zF2#I2dw`u&5iS^?5#oXF^&v&Aj#XYuT7J&^ZL1Z7e|H1tDEPLANq`sqe?+7!IFuk|eAD1`g&1yhUs%`z3UmShN~A&b z?QfAoNym&8$%>!gZ}OWE;o~4(uTV4yxGt($vq`JLkQSHB@E{tM*JA|5MLwK zF<)uv8S#=E5`jp#4FL(!ASJ#eM2v<@tJiv-$RcF(!&k+&`E7*fN1*0MWhAciWdzhk9u2Yj(O9u5AVm7KI z;*fWCoO{$0@hIv9if8PfUYfi#g;omHGzXC`zPw59`9IcXlJN991y%QYr?pfrw+Pc-yVA;&YPhVe*uOllz0O5UNg>^LECYd{LG1QPVC@;C^|g>pT89y^1yQvbSw%9uFDCP|Hj~>PBx=aVu-wOn^ocq*uW>W zV>^Ewmtv}C`!S%IBK?Zi$?kZgBnF%D;E)-bq(lqquuqjstDW3=7o@^dH-<$eO6&L$ zkW~3;assGuDywR5{ocP4`q`-AD<&?6ug1B>Ph>M#E}xFXpFd}|u(=}%__oC@gox$N zGgC@L&re)QVwtq^zZS(zjbNYf@*FQ#8_MW%RG~6|5DwZV%6(mh9@^PybRKtocAHd$ z4_QzU&(^DRAwm35lw4bNw4};I=nx~}ViK<9Z**RB>LPg|tZ+oJPuYV~iOm)2 zJ3E}d{f-~V@+4I*2Eodkc|87_K6h`16kx9AlOcA~C``i^J=;u19Wa@QsK<-%;>U{x zOt^6^+G#Ays74G6dmJG40JsX6e{ugnLZR@@lkUTQeJaYn-gl-mD6DssCT7y4&GWLH zfh)_b0ZA?@Zv{z3O!3@h#gVcT==6LTuQCe&*)l$6+yliQLUJlR9L0^((XdeY+<;RpQbP=9!* zG>eT5b0`Q`&OMGRw?u|Th1vi5aw6tX>i)mFVoh%s1&tJ^LT(R)DtzCWJg&pr{|ZuMV*%kGW+~K2Q!f-_B-sNxyG3 zs`kD#V&cY7xI(u2R^Et*h?|V|L6edG@cVFJ$q)>T+UR!C^G9+W%D61+_+hafEUwyP z3!TAkQZApthGPk(^hB$k=@Y|6P^u^3S5NYJFkRttcy+ zIZyBTf=i*_0}L-uf@wImwM+t^$WsOIP57K?k^k-6PBTWVghm`J@ql z(1OR4`k^W{Oq>3~7C4mYlH~`v5NNN*G4iTa^}ipNn*IrG%f;ndrm?H7R$vd`67zV* z{bk+b1);J6_)??*4(L_tyjvn+<6hQ+l#~U|7)>%=cKHNE6+#zdeXG_mFzoaq6O1E) zfeo*0euHjT59r;$^q)Tqowc~w+x7@;^{eY$uOuEwN{tD7N0kN|A>-DnLdKOqkma!! z_F?$sZg-4=mG?jlw5Yu~))Ck0;}CG~9pK$s{BmM2aY9tF3~Q~A>-UPE1Bzk(Dm2ha zXQ!_4?oZ`r&9-|;8ngmo?Xwg?J?Idl-0NFIV!Jq@F=DqXtvOvwCUMlQU%ci9Y5*&t z1tBxcpQ+#TZC`}j9JR)u$-B=WAucKG>x$Gvnw5^9W@~=C=jL=h-oQ}$=Eu?ul$Hrs z06?BDi1KnP5t@Q-DvK8N^MlNQJl}ay(7X4GHPMNdrk#r1_(;w?F8>X*JAe%0e)BYX zRJIJm{8ioV25L<^>LNi40nlMeAtx9eDW%Oi8?CqQJ7uu>23PAhiRQqJhfNJ;o8$g@ z+wSoUyiWz(gFF;ywzr{O8uc#<@L%WDmGw*@$Fmft%;BgN-(>tT{}gk-jV7p)Dly}6 znta#qbd`AV<5!=_c}xs83ZxT`B}M}D4&N{kl4#ZartEqoR4(f?!kJQ@{U~XV+zdg^ zLrVR(Q&0j|DHyU$7`l9zFazpS3MB1nRe8vuY9iHhq<$5@^hyCxY!_q`hsmMUsBpeb zPbdp=RW7xfHK~!$KwQov9R3aA&AB!;exi?8>b}(O2G(tNQjHeqvc%(jrA7PWLC`i} zVwG39Lf?-x@iOr9x z=iJc8(;E$f`VNPkJ2f#pC@mbTYrnmYzW3QaQgFigmcZ3ccri0ZTfS56qdE`}b%g|8 zS>+U3F{@TxSb18}YK1u3lfS|pTI}>uFp<2lVo7!OOkiSWg-h^e*1l~cb{8zJ|1&ej zgXC7=A&1AKRX+S~>e@@{3|v9<(y3RgOc3_z^$7?&G1ib&z?oh17zunNp&b%IK2Q>7 ziO(a8T7cRyz66VUtv6>;KB`D!9s)Mv&{nyr74=J61}VBCroJLjTa*;bCF!n=`)Y%o zop4yNFKjbL^IHw5$|TW$Dok^%^3SoeJOzsswA?AeOMK+iV-?gFju`Pq^f;${UzB|q zZn=B^%1Q{&7W;T!*6LXaBxrfk&VbW|8!nI)AW^CE)>LRk)Rb z>BFF`SdXngBkZGp^$$n%Nx@wS#&kdks$G$6R;z;$Id?7z_QPv{|Fu2>26p1hnLNf< zTb$=_8+ZEN0frc^cKdwRT5+{HD_+?19_}%2H!dKTik8I9&=%#%*f*eOcDX2zju{tV zyXZ0|a5Vt>M;y7nB!|jGUDy(YDpc9;6B#hQ$~-HW=yW}gVASsvqw+XiG#8#3tewHe z!Xn>BA*P7Y4In?|b_!h>^WgPFk{};`<%7hyRA1BT&SHg*59QzPd~RK|H$ZVsiA zqtdhEAD#}^x71?M5ct-t{}-9AL%aoosxa|k#OTK>6h|J?P>ZL$hR@GM^1q`X0Qs&Vxus@c=PjOSx+-}+j5zzvMOqdv|w#;9Vp zXyNzqu|W-)B_Q|C(;!E#{Pi;$nUB&FV$RAx7~!AZw&Q^&4K{e6{iI#$^SQ$f*6@r? z;+{?LTu0tbZtxULY_dZKmmJH+xID!rj_W^7uXChHVVuc3IPWhJ1w>Y9*C z5ca`IK|QpgLdZ(d3S%0U^uk_Ve-}Y}?YATE_fN(fS69pvy3e%g&AE_l40LM;4V=>^ zcqEdYu0BT1Y3LH5Tdr{(R@&Ur)Z?H&iQ3F6^ebc0@RtGNf|V1?6GcG1A$V(btJSXv zQS~WkdP?S&0-$BUr&>1)!>4zb?+?^BNTDm~68vq|l8JdW;ilZXgUcUgN%2y|!0-!| zWlN&G8Na3@CQ-U6#;ZXEDzixcs%Xd-m{Tq%Dcrf4GLul2Seq>Zs69vs&@wmDtRRk%0CsVpZyeqqY}{taW{I`w4@s6%>ye84VZ@nrigeJK%k;_7yI_kdtFq zUod^z^5&sxRE9M(uYvw-$R5UPkLG2CPFg?$$i2%k2()M1Kzm&kdE3nWjBn~xJ=AcW zqDmo35|($wsHp&*foF@*?9OcXKfyit;}d91dvl|H{qWFt+xiFsQ^4^95#RsS>IE(D$d5Y<47wClkgOR=9At;w+k|+~U zbz4^EYq81Sv@i16e`8~H@Od4SQ+IxkrTxsCMo?+Y<)Ez%PXnlOir?|+!Yc}1ucJxM_>EgCN#akzGhBrazQEbiplaL**TjMWxf7|3&<&`yn z43dAjqrj}`O1yD!Xh0Z2Q#8J?#Hot8e4b)}5r-Q7N*z^bM2w^>y)Qf}V&uc7YFlp0 z=hgHKb}|gz2GXU%l1a7^>Kfz_!kj|RGZoZOi8ke9v9o@p4XK#NTBG}w>b>e;*KjQ? z+Q6KBTaxIX4LX>iw`|j2X>E;J4?9wr}R7>Zqn_-t1$7vCX|w+_`z3aZ7RZ^uuz zk0vbc)>=GDKrwC6QAUGq+Su8o&ylTWGkq5G>16$Ll#ISK0n*p*2`ZGnt_}YIzk@vy zz&BdcT{9D#b82GLX|;5305fz_Dt!1qsm!;gNT@!213FD=$zUYMT+G#`2t?q!Jx_S( zYX5ius?(`T=Mv`Lz#Uc!KbN=MJ>R&k-Y89cU&<;r>9J-1^LuOW^pquo-Mqg8p0dk| zD}&v!T^6`tHJ#L0@_q7-L`s_C3Ds}8pvHVUTV+Hkldi+Pk+#}?0 zqV6O6d-;+KKOfIfShK6!tndpvX6{SPwi5zOLAc-bn$#Zb<$Lb7pHt4@D<|AA4}^;S zgJcZJ{&sN2++Ra7klZ*DLNzirJ2}OUd#Azh9W+KqmH9z*M7~_Q$4(|vg;u@@=FiM# zP-Hy~wvyoD!ZCf^wu$n+Zg%?XHCjy~jc>raJff&}nzOBSAK}nJcLh8RMb@vtPR3+y zxeC)tcix@Q=-j3ypCIc^3|BOn18M~msWoiz1WhWFE9RAb?ucj=<8ORDj;;4@KJvDk zc;5o0{#I%%K7H_~AIXE9P|_pxd`d?|Cr=^iI|^=}@w^^q6;NE6DQ^(vGFPC3Qo`Q9 zsB_6;;;zH>Wy)Iy&U8kS+zHXQkEj`nVc9;pw|;Fln;n7v33I;QQDOBc)chm3X#S;V z1OrCnqoT=bK>A5~mN`$xaP<%odyM886z&#;MH#ZnFLpn^OPG4{F5o+p=%?9v{W+N} z0RX8L!ReBu_@*Qaa;&%i`Wk{iK-n#xZ9%vI(#Y6cFt(mMeQd$lF*H75>8l)Hl=wPd z&(w)eld{7@aL+a~rWu@|GLd_e`jWhqs+C%XU)w#8`SYOrrl(UVqcH70)|VJ2dqbtn zcjqlYHJ|xYL*mh(!U>D=qey8G+$r6&zVs7#0Wz)sSW7>E?%5@w&kNX)%p81}ZFQRk z$7Ay;WXJO92c20=af6|DzUR61v8eTAjAYEiQm87pe#cnNoWVmcILu{yJd@fB}C{ya$j52h& zle1zLI_I5X%7}JTdbxW$?cHgC7R{NtYiF6X#sOdAU)Z1y5zJ}5&(Ak@2@}kl_ha4L z!HZPFq*ep42jl|NcReXj9N~U;!SV|$fIm1-nFVg-=4X1U8U?{gKgUYvR9!!;<}G3g zAjQSU&lw{FeZTq2i-_bCw~ajZU2sZXY@gTWxrf>euCpJG+56kP<3k-x9Q*$&M2j*1 zAUuzIC~k9MB_v3gVR2M$a%uWiFhW9b@NmwLy|i(Pwx^{}>fm8BU$Ur7U?d!Pu!7=m z0Wpyk+U4uoqoZABHl`@D&Jim>Va}sorLf4KN+eCI}Vj7TvjTaZJAX=#db<9fi{HdT)W0t1*GR4!5AJ4hTO?p z9_rc8)BnXHwhaDdtvuN%X=MsJLHU_p44=whftin0*+D}{=QalDd6{Gews@;;41>0c z(x!JzU?crF51(5(DpbdR?bC2Zw17ggdm*pOl;r_quJW{jHAX2zgMh7;u>G`uD?*%tx% z#nQWE3MU$Aj2Y}>uuPf#dMk2VA1!Ksp>7XA1U+j)CX%0@zXts*>g=X#$@gR2ihhA+ zzrKWOoevrWAIij=l7w^$8`XQ1Qk<)TqESx8bQ59w3bm4;Fe5#T0ZrVHo7}9{v*fOC zOqN0yjO|+>{2D?Jhm{LndEhjdkU#3K;EG-`l`(3u(okmqo6K0ojoWqs#onQZ#Ch&o zykD_$u)tpMJrs2KSjnUomR!|nBdE~)zenYr95K*>>y!{{wntBUts0{slj3kc(q}%B zEwSeY$PE)jN;?=6TSBb7F|Sukqc@aREtMzRL&g#g*NBa007ukl{}xkhMT-q-K5*!_&1#xn#@ z7Z%1SDI;PUL^o~W*2Zf`)ppk-kpIw)qLbf2ySi%8YwdLm;^_@^%V)LplV9&g&r>1D z-OF0!#me~k+!Ig}JC0j`$zxJ|{}l2tNS+`ilDXXh4Du5;g>Yt!jxUU@C(}JkAPl-5 z_wOU9SdYxj;g>!{D9N4izh6gwHW`#>CCe(H^Zw*SE^ORnh6^7YukitMx;yDkmw9N=hj&%mLvuNIb zb&#$s%ZCBk7P%N5sF~c!puK7$JG@VZKzenFAw`YOZ}`bZwbGglRD;7~>Zmssq3}B% ze_?O);v!XDD9BtvtdC&Wm&yqoFakQ0H3Os@(z@cNTU0UW4&C<$4Nf{j5!?}#=9+%X!21-2z!2+{>^TB#<6(x zH3eT|;`l!1-6eHnBzr$1kU>Jl*-Tjdao1VjTndT7q^2${#Z5elPXlwF2_JK$$KNXrvRX7_)RdKEJBhDojE}jsTC+) zhv$YP=dU05%uaOx(rL_ZTXXt_1#T>|RzR~xh=}>@4L7^70Lk#qr`f)pHh1p^*|Xz; z$|%-cuVl5HnFGw`7Bw?sY}u3+i3Q_O$mWzLw)=yX?YDfWMa~99xIF``oH4ND-fxx9 zYmwt@7~(NGENL{`m6iUxrpfjCeG(@tl@K296xYGL{n-2L2Q0i84$A(ssr2Dpwixqb z8Z(kMtDAy&7<_IP&O6(sLwc7cpnB^tw!{_Brc#q^IZR_P zfI1cnOi9=;lZ_&WE-`-J*vp`5dMIlR%|xy|T4Nv37tMpL1OS+IBb*D3#~;584Z7x; z<2L5U!|KWIBry1L@kVP~R`rDi{N47P;K0TEx6tA(aYOs%{^IdUu=V}@uoz%j9I7e| z4O_4kd}y3SI;(~D738KL#TR!a3@gchuK1+Sw8U%VeL3M>Liv^w@ED?GpOvf&!b0ad zLxc1*9%*LvmAhW4yq-S0z(j3>JOPnzk+bVKohE(JgH`ElJP2v2w%?w9BF(#4)=Q6e zaezGOMVs5UV1EG_3>*uA)le$#A0oD;N5L38fM)+fU7CHdQEq;ol?D^MZ|JRQcQ&dr zKT7X+!P51ov4ZD}>)8T557bA}>hvS;NXjiCd~TX}$s*dF2Lx2>EHzq=wP^KJ9m(Aj zVsv3&=#|Yx2?D8jsIgg8!S;ogRr6j^sw>|>1WP_L9i&x{lr#Z?157UsJHI(cuC}1N z{H^(a)rCtK2$h)n_J&dQqR>^|;zj}ghY-cJLK#6Dnw;x&|tnPyNSWb#DYJ+(@urTgG&s(O@u z*@o<#@4mrnWZ$P*@Y(0PNX@LP%B8owRtjSK6pbB+ly1x$dY~R4ZZu!j&mVx zI1e!sFN4vww>~0WM*MaN$~hP}3-s@Jm3M^YcwVg2a1sPkl6XGL&g{I*4xk2q^=C)= z|1swNu1FEAaiL%hj@V$kf{83!>&j)Dwo;~dM6@SRP*!7bh?3u4EkKvlGyY2i26Yz9 z+-^xkgkY2zO)$Y~TNmHYhB(1kHH=MYy4;hq@1zg>T<`QV(xnMVl{c7WGZkq)M9U(_ zxfrjR$bmA%$>r55E!hX-IR;59%5Oq=NMC{i^B?wC*U8x=9MEhZ4U72bE4l<^mK)qI z$vhwkVE_VL>px-dFX?{fh2iiS`|Asr__rd}Uw*WFt+4<2o%r@m)u&DIoRWy%PQ^p9 zcIrI}JLnC>T2VD6v}KX`9nXs~g|wP1d_6dtDTzSk=AjOF2XQD!vp#Ziq)`U?i+G6` zJq#RZ+_l^DaIgWOoKG6mKy)Iq;`g6Fu#+pyU>QCqz{rEy`2f)ZtA{}7ah6EGxxX(a zZ6+J25HA)~QbMAm@p>}!dMS8*erEBR@VXOLFkWbr37Se9Hj@d;_9|h&G}OBG*W_M7Q3j$n65{uFZU=v3yo0d#zjf zY%i`Q%VHwBgHNahs>H;04B0XawJOwr{t5DmPpkHKb(HI$bg8~j&`%~qbg=gjwBBu! zW6O6$G2El}^8$hTXXSEPlRlUFbtAh7X6bZ-o%2oB^R1}|ySqec&+;XwG7>Kw9$_oG zS=q!?)DTA`_$_Yz`R~9l?~)f677WXz69b02;xDr)Ll}ZP9>?GBPJ{_>amDJ4LHg=H zaSw$YO~LWC^osAXV3n{aO}y~uU5k7BWGZKoPxJQe{lWn&(hoDURz6!u+J;C9j<}dA z_iz|eCOBT;wg$gIu?Jz)?2_C8TAo?-N{<8Ebrs@#i(99LQtVKM;GD zFez<)Mx>29&*Dj7*x!;QZT0J@3!~iSee+2G2io>$qi6$*$|>6q4=RC>wixX~c2fd% zKT0^((#g~9CcqZJh+tU^J1ga3fNuKCp)--MQ+OR<zU$PZ9o-5#P_=so_ic>n1bxRlt80#eQCu*t%f3e*!Y zMy}&}TLpKz5gEdTI5_QaBu?yXvJfqnyZg6U=IW?7)HEnMcF776vD!Cel&jPEw^POo zuOUt{<^#Vdv-UrTYb>b)~|FY)==W&-zmDihD7z3ZSq!R*_YaRxW5=rjZirk}Muokvw**ZbR6A;|u&jHA_o z&l&gkRd@cau9M!&w?#9edBl7heBYADy!cda?Ky7zRy`;LHoM;FZW3fT_OE`dZ7pDB)xY}FB(~=>~OSR=(S)xLob8iD|87- zmcNT{giXXGAq1EA<;Nrnr+VL|4}L=EBN-`thtP1>Kjd-GEWuLzbtFK`NlM zq$FSqDDC(az>H}vk?^s%A9p0|t?;)$0|VOa7wDr66@GJ=z&vI%P_HHlNhOMV zcC&Pf9+0I0VnECGYjoOV_#R)dTbOJqj3k|6uuTooBuDI@Zdvd+Oa%&q{Ys?LW{a1A zF)8rI5wpm8t>acXzH+mUL~KfqOe6+ifq#PmC#6)l3CZQDGKCwZ-fA82Ar~1`n281k zNw;wO)5jUeG99Y65);cron!BikqM;L$w;p+a)K-nD!jfAB+0q{ed5gtWsJF>Ws;KF zex((ek?XwtmhJG_hZ%Gxuu-2ICh1L+tXZefvvK*y3WY%VOf%jsJuEm*wBvsI;P3eK z?(r)k2B`H-9`>)2R!c52o;goHq>*8Q2o)m{v0;CGJ)@=NL_03(Is9?q zZ}F5qF3prh&b|+LR_*Y^!{e*3Bc>=QrQZEP{ANzPtWx)t)0*gl!erHw6GuJ>b;zQa z_~R+`DD&2gH9a|DtY`P+%IAMjMoILvQEOQZ%ID_^E0SbhasJHJ<3aOgg%w{B!Z6~)NrOfvhg)fA^{{Jq3JW2@vDs@H% zKmz6g!25^F$6kaV+^9Gok>*fmcENhtdakOl#=R)nfGRE?ALf)oP$szvJK`q>K*$`d z2&Yk%E>*{O_TQjf zOYK#Ik{RXh@-zStCta8;8MODRTyh9i-m}YMn8qwV?-}AGBK8*TpE35s)4&pms-cuA zcQk_%=_0_G%Q*{XU(9jZyX5r{o;k?|()rNx z7k;CJEqjywHtSlxluLr~pyC%>$6bCf$-+(~_*qNa!9Gbj$-5FnnTK;vK~E+hXHzjd zsB`|n0vkYc2_2(PDwX9(Ap{@AQ4pg(lz$9X4S8Z z$F;+d0S4QgR5n{FoA=dcy|QdJ7ZtY6IHPF?$L85koDM-xuxGd86RwaY#&?C1E!DAHo_LvX*a7f|9qSY7kmoJRUdqkwFu6!GlVYq{Vb63LlA&fW303_0ixB6nEJkqcQNQ zO5N#l>Xduq(^wKfi{%Y`_D`|JtG43^+{b}XnLkXeAm;D5-p8pvvIX@)A%e>mUvqK` zML9V;$79iJi?CZw@_3)6gI~SP&tIlb>PKEMB@l3u1?SMZSITqU&#r=zNCaizo#Ajp zhDPHiE7Yu-WzEsij4N;7@?}4SA8H|cM^2IZ%xN^?8lrk0x3aKX9;ekrIPli` z2!cb;gsmQ`=k7Llw%{TxhUKPNz-p0WmD&PXAnN4OsxsdC9y3o8?4Sx``pc%ykkQn4 z8LZ3U^EXx9sFRw-&>7j4xN1YKUi4q;Dr(e1x%{O}7%4N^o?8?KeC~e!UeA0z5xB(T zaDw^czi=#95~EeK`Q`O8W5SUDFdz3Ga#xV@SvNQ1)E*tP-%aCVuH)E^8y^~@c2|q) z|1tH=0eQWR-`U)Ut zg0$wsE1x-q7nVQeOPb}nbc|@Ud8J@KVtOOBSsxGe2xKJ~w-ClM52_Z8MpWcLn%Qq< zM> z5?df%AlUmsZ=Lp8wWqsq!Z1jSW=5lX1JlkT?|ftkay?OpQV5Vmr%?)LYG`}Dq#1ShzufI)PL9H>-x*D-TZa$O) zvK;h>CK6%fXBw|1`7FVyy))kt@tN%?;Vy{!n1$j_GpPS;gbvQmJVPx^*Z=eTgF4AY z*~fi779#*IU=@>aW>9M|;A1E2Fbv)m|9J5=|H4zKXA?cgRmBq?A}?2|**E>MN)aBJtuing%iBBl ze=9ZqyHa8s7^c{T9&{S=a#cnucgKBUh%g1G@tqI)B{xp^$C~mbyc45Jw?+LNg?A_} z2B8I*@^N1RrK#V-4~u2ebe6~p3*XzHS&jR33dX%wS<|_0EAB1-bF6+6BsOFhifID% zIuu_L6)e2fq=!=&ZO}_`9kKjzCsKrgwpFZgdSzO}AYQzeqFho%gel3Bi_`quBClr>Ey*MXGVZ{>}BT z@)guC&2cM##0wXUgy{7S&wb^Srhkgvl^a!!)2cVhScbFki=J8dCZ9f@|;s|?@IB^3>meZbHQWQpZ zWRWsa1hcZNIUwj|MvQQr+iP2g!zij*^E#pB;%bzk3#Ei_`Da$^8n zo{lDD^LxNE#iM->(&Df8FbsxtJrCDY66mX4m0}O~_1N&vu_V=F&ipWOpT{a~*vT=Q z*d%~I9_&Kj?n6b=!^49 zfV=Q)DUBfKdEe1n=hVlGVN(;Q#X&3mC&GA_y-Zu+voTHMH8td+`>JHe;iL?UM{;nc zfGrKJ5VY{~1odi%1%BMebZ;fTb$xAV+#4c5 zCuo!k+lxE0*g(Z)d}%$_VD^ZbtorPav+0iDouM@G71pAJZ_g;Xb3gN~#C>(wxC)&v zd(b|A2$y9wwTG(?>#NnfyixM^@tQhW@)ma39mP{@y@G>+R!ba)FBki8qg^6wBOOoe zpY8wx)nxK|>nau6p}WS6P&XYC=~IM7gsya?8j(VIw)e)F+#= z3$EiG73r>Qh}oQ3(|E~`)wX$lKZV0zVGxbRm56vV^u*+$(^F4W>{NNrqAEv*LWUN+ z*!dV2*1mY*59j$dy3;!#3`}nG1NKU4VWFV%=5D-)wvu&WgnOIn*^(l50*AVIVLd&~ zo{B;Tx~FulJ;CH&(7i6?@Mn|9C!3?OyV>B-B=S+SC~siziHDOpLZ> z-KGk?9aVoVl^FHCCBL+*#ftflIM_@>|>sPgLvk6O`39a>zZqe2Cd zTp=Xy3->-eBJoJ@0To&eR!(YeDF>4Y`u1PWA^7%jytaQlnonO|RQmW^ai~F)9z!O< zyPXw^6Ic+?e&+_`G20-LY5|DE1`grwe4#aPf9laULSQ68(;PSE#{RO{Qs7(7*j&3! zV`91T@X_~K%%YcT0;|l+aYCh_qt+;TpdL5f6aV2iZkTCgXyEBk0@&wEDpkdo7wc%) z0d7ukh?`bt8$@J*7$DXv`PVMMzruZJ%g5pMy?z14vFP#}UD?Cr^95`BN`(fvP*?5< zkcin`S6xLvotS+)>{7)@D7KIRzZZq<9#T9GM2|Dnk+a#V%5Wk*ebejf?li>&oZ}`R z&@|G1gc{xDiv!*!ImkHCh1_U19U(m24-}>UG-YxuPkln3{GX=YN1)YP7Jy5N)zC=s$1BFaM(4(p)XW7O&;M6Ox^#TAa`frUgqS?VYoB~1acJ`hc zWZInO)IgYv64L9#&`g->lBI*tC*d@XlNKdvXth_shpdeC(xRif!q(1#srj0V9>xhl zoV2KnP8DeQiJ%cfgK|Al-QX zPwSf~^LvxI7L%w&vGC>b?0Qlontj0A|NSl^0f)KUDy;FOfH3Y(ziD;aY^e&(RKXvUqW? z%`@eL3cssN59h+Um#83l>QGlVUn{B75%&UHw{_xrW48|>;CA{9h)a==P4s!(kNBbB zsD|Kpe~OO2JK|O#Q0h}^bsYqxu)uW^Z&_D(%$k#Xfi5Qg)eMnli<2!(F_CbE7(`P_RSbzgRRsj$3FmR2y&G~gHaEG zzbB#2Nc{%!!Tt7E1h9fAr~{UQMop z?QR4faKdtXR_XidWwDue9f3&P+8vj!3u5~dQW~`azbHZ;sdm{2EJhN!ZaXfUMMl}= zYJW?X+Ng)zYNudKyO#o7gkQY_@mi@-6l4ZcvneByN8bV+ zoFtLDacoxbR+)N}@$5e01`AGiQ(0kRsE^+kVkxW`V(cN#u%yGf{> za&>;0H78%SSMNj~%M>!}H{E%y3%)2Bo`N(AL(r1GB~kX!%oWjb&ug*-JNPVweE8VzA{F1QeW* zejTGFJ;&TREOf@38!S{Fv*%hEajQ5NNi@=&28!4oQOlH4m(aBYFqKUR|<@yf=-pXjrd-S#YGdldubm2M0W z7-z4si}619rklq*L(B;j926Vr0=BjEPnLyOV=0Wf7#+_f8mA6Y-7hVF>c>pEHI+|y+LUXUUk{GqcsPLn}*Kdj&n(gZ#jB=C1+)#W?MLXxJkO-LW0uw>NnAlXx`Owug zv%H7I>|7+o0YM3+yBoD20fSMDPmHsj8-6Cde)llwwC>b_r@WJp_xLRLGhbSQxCs4W z;uG4q*pNojEs_$?jH)tNr^_lLogR_FqqaNL=AJ0hSMbLVK8_u`F6TnjX*C~;m*)w0 z5Feg=33xOpWpj_|Q9Ai>xh6~#`SY(^dP0+fsWi0I ztDIPpRN2xTm8ADhkVbycjJX_BBG1;gNq7x-l-Uc!yWq%rrd}^M+Aw<8E`_oPU}!WO z_-%77jo9{My>?1$oT7!55p}LUKAHVz%07jbQ!%=0kD}pj_<1t}HH3$^qT*4;$mHEed{0dKt_oswPd~j3PeTI=P+qaqS?6@X5%*r&N%SJ$bI?1To%+5S}%SjCKQgwt}t z1Wx9Ij$w?U8~w99oTFgiL#k2+c-!HI>wTxId<){2;|p_mX2Ox$IKhEFu6 z`}e08C&sF*@PGdhCX8WXt7To)Jq4V_|3irUBsP?R6s!7o70ROh-2cKA0N?}v7j447 zx*hxXOMm(Q`%dz}TLxQWLj2!7zyiVn|Mv;+pT#mRg$fZX_WzC@{(|=|i|F<*HQ1r~6ggK_*RPBEuuK)b&JAgYeKl`5Q+;jn0l>a+7L^m}VJ`PR^ z!op|l2y7;e*$~Xbt=yciB~W=igub2CCX-wT(EfrW+Z~qmpPA@kZt}iE1_$~X3WFg+ z38=b#NI77?h7jR@Yw1Z|*QK_SzV|ke9c~pz{d>f)2jZx_-ig2&LrXz}gGY}0s@E7l z)e9qYG+WC2lZ)+yS*5|z!9kQpo33wzR-K+s?D<||Q9JaWy#YJEi$(;Bq2n9u-7y6a z$1d`?Wm_1+e=^O7))rEAZsLFs>a5~YQEFT#!#qGpp`&xq%AFhRN;3D6#S0@zNg_h0lL)vWuY8W zH+7_g1MjLxFr3)_RUzTGb^4=5=;8vfExM8i?Z`O zE^RiDHq;GTNyg)N7`3sfvw4J|f$jDiT|wWk(&Saw;;^Qk=;Gqcocq>t5&bvmq&IDUBmC1L(U^e^EeuzL-k*0QQme^LhzY2ax-+`xDTdCCM05 z=tD^IIG*JJ77}@@Q{DHxO|O!047yA`)&g0wlxp1s0BNB;OhZ)&xiK#^ni4(gt1~#2 zfbSWR-~w*fE-%3u3-E_#L{i_rT0!uXILPno_z&A@h7#(wMuKN+3N|@n9-~r zFC|k`dwc1!j)A5KCQi=A4^i9ZHUqtkYHrvI8~mIW)hCfx9BEt}mT$xe9ufZf1CB3> zpurwI7s0g`)v7>b^5?chdL6tk z2?McHsE6}%K$Q zm`E^OuijRUzL90g?dQ(T6Le^Bd- zwh<%9gd0HjO?B(`TzD-mXblCVC6F^^c_Y9_rc}n~=KF*^E)r>-?N9&m#I~2-qnG^t zBhmh2QFzKc@*~lb9qs&1JVuYi_X^FOhHk zsp^ZpbSASuQ0XHF&nqPNAcsY-X=Bj3nZl&|;BXu zMF{W_+7L!uaU_A!;_mnA?5IcgI({HT8IX17mqg+7Xhduisx~;d7lNG6r*&CdjAI@5 znPhSSs;@sYc)CJbz&_9*9w#|Tho>%L|Mqyga38G2?J~YVK0_h|`KVlnP#!WX;<=D6 z9!xC&kXFh5k=Zpm@LP-LRh8LrmIsEK)SVOBL9WD5uoQS+E6>b~_cq;8?{`Iosb6EG zn6&4c={sm4qtIn;aWqj5Kq&rv8XT3i1XpMS z;I-Mj5MI7OzKjIGnNlBfS*HlSYn7h80(CO942f#3^b7l$P;W+UTvpbV6xzvOKG=>)W&uX3qD)*Q zXSkQ$J`(z7a#>nTP)CQ$s$_Dd2IAfi-6yA0av<6^ZUD}~^X8xy4?p;UV6k4Rj$E3; zdTF-q;(o|Yx6%k9^@@Xu828~DSE*uq{A~Gi%WU;)T7^cvNRb$r6e=Cj*!!}}sW_N^ ztU(n%v${oZ!G0!pxY2l~;@(6$6pSS@rI37;*afaYMW;&W<0^_;2yNG3_nPjTMvcap z`@_aLNNIp?t^$4JhvyyL@kVLjvhgsJVF3_hc6Gdi%L)!5o-6ujG+XjrmO3BZYLlhKU>@YIi}<=(pz2KzSVX?vzk90}BybTW1|k6v12K>^zTf*N>sA?| z18s3=5zVsG3m}g%TOU@P~0qy z@?dYMyb+|$k$%1pIv>9jN4aXO&9{z3jG+XYB02NB6mwrdJSCZLc6{S|xoe1;D+*6? z!cD||XNgDr`(%+7vi#);E~_`R^lV>srz>}x61&%O{7(i4L^dtNEa$P0UN(S|S8_mZ*2^W48;wMHE9E0C&pb){5kkm>c(n!xf_^1AfapVPoz ztT1Y`E7$c<=_02eiJ8@0V=*ct&Ev6m9L-9|bhV8Ma*5L5z##e!?9$cD6+&G*CjcE@ z{zJ*CJv}jA@_JM+dkuJ7*EwGN1P$G;NFa^9{kZ?rZ zcEn!qz3(bDq?q-3MWgZgXy0vCFI6M7(vAF&|Oc-(SRHFbfK@WVWyjf%cRF6biw*%S3$VcvD+B96~36pw2zH0EIubnj-(g@bHjmX0hjrqC% zVVaT~EwfVk?0m&K5KW*aRXJ->zv(-3h1|`$`P&&u=xc)rt-uSJFPJ zelazzui4iMN)_m@k8r4TwnaD0q`QDJ`{m`l_eEWdWo#J@>;omU`1qB{m)&#-pIVSq z67Cm4uZvAZv1b{!X8eV$t!}Aw_F4x4a>uXqVsH25QG`=^r@umBldEcbpsz&?tjozk zF|z7pID!-}guhUwW?&i;K?=ntsuTnjT211QCy%TV1a@NY1}^+&kJmdY@dDovq8zF9 z8lgsm;Qs^`^Uf0k^9eWzuPXCsI_13PpdLK|-+rjw4~P(sJ8ZLgnviLT_01NWOk4+oN)T!A21L2YmC?{A z9LB>Ep^ae^uh4eo*FD&M-@fP871;48M36AcHi8nld|3si>A0T40PEc*>skgzNfy6P zgzJg?72yiC*wa1KdQDt?7(=tyW7*ZSwz$H)9z7|jb$jf-t&M0;+whc+;-@Gm?6JFf zu`v+b#{aVvr7XA~7p<)wjFX+27c=iY4*BvOfx-Y)pIGci6MBnR9qcCnaW1r}A9sLK zqSE$&ic&~rE@~i1hFH^3BYA>mH(-?)#No1i6m=jZ%lFAD8`nmz_h{eyVLXBYHQJp* z{Nd^gPKA0SJ3Sb;D@9N&ZG+aJR_ZiS&*uS_$(sQzFjCM^oinQuRDst@2><}5v)^Bs z{y=4b3~zO@s7x*@Nw~1bWejW&{jR?#`_T;Iaf8^ig+0{YJskZ&nMBWqLl@?{aLmPbiaN~V$wJdgHsAq5;sE$S>c@VJXtozTWd)9hzq%kad zqa$wUc#z>YLYP&XfzML=E)0F}J`P%JuIGGEkv%AIMBx-rb5gz!!+bu^KX#~9)MQkv z$k=@K%&uOyzkO}Sv6kn`vv`neGDsxhwNgL(TL0$G;N@qFHn){RqpD+mq0^|*_E`}# zb)d_5ymF#TU#Y+fIqhYcc(_nA*9-UdD;HT|rJFp?ymS=YcLrX(mAC#<{55 z2Isl^))DO&575-Ts46-ZUGym!KKzYz%+Q1(?&Ty9QVR$B+e!p)6Z_k^d3NTiHNhCe zoX$@=m?W%b5AsJa1&=30pA6f*Y)HE(XbRS_j+zlKpow^q>T$i4C4z(k(LJ3{k@YlK zj6LrCbK-c!v?g5>9y?zv#_>^ejVC&Kal)QTg9s7ucV#gZ@jrA!bPuW*+ck-`YyC0m z-byM8ye*d45DEZ;-opDsOH+B@q9?7sB*f*#u{joWzCW$F4e0pGLQgd z3h3%>Lex%dVCV2&N4q1*iAQe4lXckR>zDzQYVyWpUy`des5r`0em`AH*X?(q6X>DL zwF~ngpH3r0&5~26Hq)dLHmFZ~e0ab}w^f3Ab4fI>-=}M3k38>+$j@4LidDQQw_W%t zHyqI2H{ec0q<$&r6et?20Z3>xLy%99{Ymg4c3y(OgXME)n|p>QA#dI^9#ZlK?5!4C zw9Qw@LT^+78CdoHD2n4dQ0OB)2pdV#y$OdKhTRhPs2i-hfdg@+)lG1Yu)BM3$#%d; zXHZgBlCulcQ4P)$YLS)HP_1A=1F656N)X_lE8`$uS|#M#4*|6(#hb;bfh4N9qp1Oi z&}<#^e=MMxU1-hRM9TFik$Z%H|M-Ra>EbfcYxz-DI+CRAC*0gfjIeYSZ%)@_mGegp zJ4^U&6oARgfolEbdaj)*zw`Jy@2ed2YIe#)M8bM13c5@5DsAWLhm^kN^S-lcK&vpI z7vCOpgQfIX!$I=U-)>b#fD?*7L3QMvA6ObfNbRZiTvvD|^R=THj^DNXW5PP}q^B}~ z_(}M#$GeNlAC(?|O!6d#%jCDAKmMPE2^Rad*}dzX;i50cZBvr3?9hT`WJksufrWtt z%ah;&Xi-W8Jz+G6m?<+jJ!4hvp#^HajT$cdxlg;W1HBZCI&+5Xf|*XX%om~Cc@zcT zM*4?WyA4_OJ)fLP{kKOPqa1JF0k{nlyo2jZcp;nPUcRp{s?gF41&dKP7Y8M9+J`_$ z&MSA5)f`3rK|#A69DHSdU60ddHP=kV@5&Ba^mz?3R9^`N(&#eL_6_L30R42ubR>G* zhHA^OLa$fWta5)IOuba@&OsT3j-dl3YM=)u397jbxOFwSjlag{)tX1Re z%Avp@c+7jXp~vo12y_w`y`21^RgYax9pHYr-G6;dc?o{-CovnoweYUO2;k6Ay?Br& zQG?q_jLvaAnZ+&DsA6|O1gUmiDqGO3HyrCuH%x#yQYJ3e3g{dr!?Kw>+ApToRmx!@ z(2c&323;cDw9KsPua$&3z1?~CEdal2w1mr?32pAzgL>thB-Hkd^vQIa7O7D!9wYl` zb!N8&oKM(raK$wS8?>YE8A%elxZQpQ9KM&V4gBBJpB^KL(4rV=Qz#h{)X`9HN4pSN zuzp91VydH|%~)x&(QS5bM^Hi4V{b+82Dd*U_cmOJ(+i7LAW%b5*wO@{Kyeuow7#-I zD6)G}JiT7?oLk#c6P0+mI|yfRrd_xS8{P*Ir^DmbZHuNbD#8g8VAH5{6)Q)|-Y4V7zZ!T0no*c>~OXaw`zW;i2z0pmEnK`ijrXkj8JfKSf!bwJE z{hB^%&b#lS^RzJW<32VtCYzervuWIVCq|quC$}jfFE)p%8|oZBd)c=0dm^Od*Wq`c z^q_;4_^?5aO4WwqLhvu?(cbYW`12!J3@T`EcYI31U$`%*tBAc}a=WnN1%vb0V;$%c z8LGxFo4w>v490w!SD^30tO|ucf7E7sct7toy{}7~c-d8Zl_R?-CF%OjK-hU!m_@@F z&9pSFpeuMSttaWcmSzCLzp(y?W^oF(Lj= zz5A7 z3vI}Tp1E}caYS+#DrRa`-C)o`rB zs#c*{%}9Ev{+?Hg@YM*#KRTmnBqUnuWHm^%{%bpRQXs+I_z2wGMuO z*C5LYKD9~P)n;UVwNo9ZxeHsq^U2$6?j({2{_fL;4StPN|9x>yfdXkB<}8uF<2O4Z z21*r`lT$*#w5f=5&KAs2Nr*I?|HpY_QcF^q_U~(d?Jm?{SPL~(BHU_lfI-P%lo{nSLNq%n2(02gNXS^_=g33 z+$?PJv)iyI+wW3;u8v}0*IT_M%~9SosAL)P$>(mXH7+uyCtKd3lJ8?h@VUtMe`2wK zx73-2Ad=M6XL4kP(yR+n4cL#Sb8DqHoHGgvauWl%4BuvKW@qL@5ejO2 z(Gd~Ju{k}eLy``R@;UIB1a2)FvaWluwBSMs1|gAvAd_FZ)V_8Q%oh2VUaV$snifCN z$l*|zc70gCa#D`46kBKMrmz!s9Ypo-|L~PyD>l6u0pqM9n=h|2J>ep4i2 zPJeXwny)Q2?6xaquPOEENz(fsb3O&dSp+h2_FIHKtd~c?^lDPGwK-VA1^6 zi^+5n>BSILn-Qn3H`bIu2@T09q@lHpmpyPNX=Xk z;8*n<(M`cY{CKqtR z&iu8JJNnfN&I1{z)XNtG#67Ofo!!llNleVlDM(^TlB;B~ub~sGKa5=tu@a{cz$K^- zUsHW@G3DvLs^A2u0H|P=PK4Za(ki85a2v4R+p(mX+Y%q6{KJ33O`gaLrSLws>j}sy zkX_h*-YCRpEXwB+UmQpr9=XbA=&ox&N7(Awd4SQk;82O-(fjfL->LG9a`ec7ecv&kk`=hP4qap;3a>0fm? z-~%&*LBW~ssxleUntDmTVeZFzI5YbBnm1HtDCtYm2)h}wf)e9+zN{ZGoj{8rg;%;V zQI2OO*Lj-J^MfyvNl;RI=TR2#t?x7}9+x@(+OC>|Vr!E=iiT!&l(oOS^ZFm?$nJcTgIZoOU%ucrSyFeXDmm=*<3BP-UZ-B=Yd+Ub# zc7q*)X=XP>ggD>__MUIYwR0p9gzlLnJ<-DyyC(-8D#ELw0{iL3e_4ds$&5;SI<8QU6ejFb8nWwzM@niR$$p`<(tmmWw_q|sSYBm4gjT(IXyUC18 z!;dy{+W)SDmQMEn ztrN|*&#~tR8dk1e9vwOGzDK(Kjb}($uF>z#u{JP8j8f8=zCui_*-o90m{nCYRyK?*8V5*(?kMY1~)Hv*nD(` z!}T6v-_3f!Z%?5cXdM1D&qo>KJ`Q)*4brXX+8+QqOC;b!>*B&`vHX0H8YL+1fNeh# z`;|z@{gZmNyR?*))Cr>Ix#2~6$Uj%+GopXY@JY6iFDd*-%2+_9A(eDhh>;MEJ%OO* zlgjUnM;og0_!dUgPqm~P>20p|>o{1g=e!yiv<~{FMaIXa zjeh5>#WT6ONXK)d<2)m_+;8LpHS01aW7_m*L@B%8@WQz@Wf-)v+O4jSN)Uko1*V;G z4yz-uZUc4V6v;I+M*rMNB(%WQdJXj)N(fi1|N4V9xWGvcMfu8+`2}R}mpqnCC#g1+ z!It6l(3q_gkSy=aNlmTMlNGx8D@@gkuC}%z?%{G<+RiR)$;Q;OxN!=K2oEo!SC<+v z=kWbD66nHkuWD~6WIuR_=aiEfN+o4r*U`WRspd%YLERvI^zcd@KRvNZ>JmwTG{=`C z6!TDXx%T|t4fO%>P>a?#_MozDa-849Q0k1*hKYwTF6 z+fuRQdv)U;HG`#Ll*I*Q?V%G8&;xj4?vy7TeHeoPRpvWeRJIz7O-=vHi61t8lRc4sm&`ZV z(LSvCZiIQI&5r~Ytr8ZXBKu3;!=is4Ct_IOv0MB7L2wc@iTi+o6F9`w?mgl2Xg&Ao z^0t$%GGi{OkpWMJ77k+Fnpig^;nu#mx5!?QX@zA>EZg=khfWUfF5WEQb zrUCq~ByA!SLxZKthkbJCg7-<{?!g3^rfu#x32}8ys znbJ!QHWe#~PBd;h9hCL^jOe!68A{B)y5>tY$>Ev-xO=1N;t&uJM(?@XtnN?5TRS^& zMYc?YOPonceF!3uO-ZsT0><52ok_SI+bIkMR)=|jL{M7eBdh+XX2C?4;+FfWw)ut8 zB<4tx3iifUVfl2er?34_*Vt;l#zYgqb#YQ7OdhJ>aC}E-KAqF35~vVMEdtT9z0bs! zdz;PYswk60PaA#Y&Di1kLP0_;EQ7^B&Smo)!H{}c%67(j*|bjKI({?C4Q$IB=TLdzT$2=;LRu>0pmi;6z_)7Y&BF$*L$2fkGpEXk#S_x(-ZVJEWe(NX#QT zaE%7l@}zu*;{M2apb{}GGOt-+j!5B-_M+nnXn$&Ejgol5tp@m1!o5V1J^6QWc=+P( z-}{VjN;a#_4E~e-okx>e9X9I_+sx+6{GCqWJ<}z(T+HvL30i;XX#Inh%~hMvyv6=1 z49sVW;#H5ideh@%cz>FI(nSM}iyTiNQ_7uY2)(`4k2bA*i&P|L4uo6)L{*8tO6t zdfACK>SwQng@y@#rjReZrE-;~1%!BUSH=-^Z1%*=2QKSgOff_Paq*z^SxNvW%=4f$ zWP%_iVZ-5D4ypWAXq?bGna38v6>x6BX~Wd{l;$a4A{Ktqc>BClj>lMrpTJl6JX=MPJJxy=HDL_kD)8`wJA@nyWpW3Y3)5(_8msgSE?yH2m*tFiLrbM_Hn2bBr5oHxe$P&ZVfZz8sccH& zpP&EJMputkothySaW9Fl<88Wdp5)PH(rg!D<1XaZLb~6nR8@#CXNc>3Qp{!R9Tt^X z`pQVUcDRT9r3CUO^q*If7c*3z*`57i@E1LfMKR{Q-cp|WDchCjWt8F2Om2J81_tq~ zox1l8d*;Sa=5o_`UckTRRH|I&T|RHCn$F%ZIa|fZYPn@r0ml3oa~L{-xalWo&H7B9 zlY)%Ly<9}-3FHZUqcyAf5-vKqHzI5E`E>p6HgH0V@RJw`925-f(1Ucn@m(S@1xvso z!1`CZP=47U*K@U?m;<#M^;j+HH@e!}tF7Zg&BI1ranhrQJZAD8pbCYTJd5sj#?CPu z$#TH)xH}k!f*BQ7VER_;#U}X3fNd1OcJOk4@t|iUJ1f=z}k0t_u+0djiZ7~ z2Q0XGODRZNZyO!!w7Ek>YABqo{9do;rRS;}5;S_PsUOnSDi-M_Nx7McME~pLdbs#I z$|GJLlwsp+&}OZFxp-EZ>9oJTlXc`sWd*w1bb~;p3(B?7KqmUi#Jb-1Y(c7|g-Ydj zFfd9h>CYl6o2FIo;>d;Wzk2syA1)31sNosLhPx-X0__A-(t=518IbU9=okq6{QQKK zg+PAbpdza~B$%K{&_Qvoj1*eoqoc1E)GNgoGlN6|F5K7l=a{Z9#HFCFVt;>hL!Ov0 zk2`&Y3pQ$mR2K(LX2-Va!QzQmWlB*pz>cT-CZ(3BRaFSvU!MMj|Ux51jD z1-)P}w=93lK-=s)!$~j`RlVJ1T5tJ)SR6@2K(dM!wk=>V9R=f%_91F2_I6QI=2GT=gk?>e_k2sFW zGaAnW!e2s|V^4?wSTaXus}A4@z5+%&T8Ta!utvJ9`9hOkm+6{6>S!c3`sV8lKzDe_ z^iAm_Y(OJZ5KQBTJ_H$*)X7M)K}OfR3XA0=OTmclQsM_Qo$8_m5}9_MmusupsF)w@ z(cSN7-nhIT0kh_FipfCZl~IpmOtxY=V@zSJNC!23?&`cO)4FoeSd7 zVYmt6r~TsUe*?3DXpA$YlI=?$Q}J7|77|))Hg^5oWvGF3P>(no4KYG_8T<8zJl zI9b*vYXGe3)u)r%2;0h3Ur>&8v-#@J4lM?jW?QaD?_rI!hZzMPz!8*Z51TqYHr3|s zy6AZjmifyK-aou?`Z@#U3fBOS*K1ktsNn&@Kd9sA!76RlfPyskQ2-sxIZ zWg4)~*WuuqsZg4~Ol3@$lkFpor(Cp$fDgw z%%C@1YhZ9_=f0yV)IvUGhC%OjzLta-hB_Ed?ZJs4sm?D@NHBWJA3x;rolqzeY#5E6skdR%v-(6OUg?K*G$1R;dj|EN;}EuY4s z_cw=`qbyk++>+fpp;ppHXI~KXAb*7uAd(beob(*PpTRxV`L=ulO{hTJq5FztBk@MspFUXhX-PR0aC)b{hEvYSEX z?+aooQEGQe%;7}8`{_!J z2qpKV$<|WpL(~mj<9e+~FQ5ayqTl07065@c%tCVM)Bq|lnvQGwAf)c>?;t+cCx0FPhr@?nv7EzW`@Yz<(UPSI<{bC zx9kk*9dw%DEwF1ixDZob=H#I5o}13L#+8H$dP~$OgOSV_L-8d126V*(j@bH3AB%M`o8zPOLDR&Pe@FveMKQd8L&He? z_8-~e{>tiMU`ik#)lx8t)^mWOkK~UH_|x$U`Xdd)Q0?{a#3*Mj#f=09#I+`bGaOC9 z^m?G+Nq?`Z*TTQEtKXGtpM2`t5A03@a3O;xGP!TS)c&Ca&aEXwtw@~cGsf6kV7V?x z%4nGRlO&OL;`%IqWe&%5+r*$J6@8!~+YTnHOHpU^%HYXh*n?y?H#EG(lSHH!3ft-> zF^4WE+0qLSjxk!^`S~Xgfy?^O5io&MVQD-9MR(5R34PqSIHrPMSqj|WeNE?@4QLNj$u1;+9G^BF?10a?`y*y2=%K~ zp0p3B>6XwM&D#$WdV0mx6HyOn=`#5$>HA+C05!0y+}7>SKHbS?Z`78xC-@BO=!~r= zZJLRb_6~yM9H^#vTqjnKazZWcRR@vyF;(4c0F(*ygKG}T4b>aRTl|su=R1vLM=K0> zzf%6L-@DVyKL|ri%wNyF-emPJdZV^1DP2c+bN$jjyrui&@AJ$5f;DK^Xf<6Y+dVZ~ zXE-<*144jk?jN`WknfX3$Y~$nz9-8T5B$%|e*m;OFZzy(a!=vc3CjEnX{nmalv^z?+sQulhKX+8u5L-18qoGTa zy3a9Sv<5`Ok$%u)Z!`1zsdaPFg9kyyDX^FA8&lJ-&1Hq zhiA)dXIj@fu7@{|>9FH(`edXOzE{*eloh+>nRtAcaPC8V+*5aVAAxK_+6rmC9*E5} zDL?;KE5gx&p*lQ5i4naRBja3Jrj8Z*A=ntVZ0Sc`1eGSf7rMwXt=TRVgDv*Q%4eHH zkz+QV?>jZx!XtP;b)*qh*Y5I7yr3#AbAm^03NSOtGVtdAe1@km-R9le{(ntw0?G1gtA7B1uuF08m zotd23d+oK?+Ms6UsPZD4QY;ZH3Muy-ymbhX;Hm;M4{rawHUmL>OM06r6ba*Db5+9_ z6Hu+rMwFcqdc`Ua1r%1z6@AYY^oB`By>|#OV#51cCj9@jekkODHB*eQ69~GNvSx|4 zvd0&itCr>mOOj zVd>=#%NM;`^&U{&AVP!KY&S4PXz)We|GsD5Ce6TV`0}Rutu#mVCMzL^9-`h8gF+G+ z(VYr4SAIcAc?e?#I0+)B*mk|EA^-750WAYq^QoUA7CEf)(l$T~M;A#iHO?Y+?vxmF zuTrdzy7R)C5TYSx)NtFF3$QBYEKF`wK%HQs_uCXa)w3+qR?LARvW}eM?*UH)S56I&EJ)V=y?BMNDS^4U)x{=Wwh?ay!!Ct=uj!12`W3-4>3QzT8B62Q zJO~u7Iv4=(%n{04#7NIdvTTbnOgK<0;!Jq$qQz5{RY$maCiW(bqB+K04dcXd*$)lE zC8yek(HD_JGDuTap`CCj_p=RMI>>-2HBpK|?82R&_3EC2h*UsU8p}g^LLplrdLQl} zW~bTU%&hl8EkC{?fvlm;d)gDWdiMV4oKYVOCY=(Pmym6h-hyF_M%=@)Sdz-(e zmfmjC+F<)zjmht~L8Ie5TaI>2L4kpue0p)*vSiF1a81z$=`_I%0TQ+00ax%XYg0 zSe7`aJh$H4YO8J9k;`jX39u9Eu3e>gnW;kvg=YJSK6pt{H=Yq&U4hF#d|l7 zDLIqL7Vi{&6L9Gz&#ZBW?F^fC0Pi(g{ZW|w4mQ;lbdQL-h4JmK+2$!_3d?uc%o*5B zJF;eKITGW>2HCJ#*O4TCxITh%bu$0igfci-PeEt6ykIwF_3F(gbk-Af`7D`yVBaiu z!@En;w;C48an%8TEMF}_`-#Za*0=J^#v>G51R{o6{ro(SKq|~-a5s$lGogA_m4Xqw zZf{tzZzNis_Y*f6pLNN_;jAkC=nc5l&5n+TyeaW=cy1^J>z)oBx?X|`$$aw7bDiO4 zn}8zd&51=4VPI`BzjMu;NhD%GeK=2Am@Kt5zP)TZoosg@FI#B{s(iGK&ic@#?JT~o zm|EX_b8|a{6{J%+RkBu#6{syo^cAWKIST|JnruU` z(0i(u@Uvk7=cKstJwLYn=L{N+kL@R!I4e%VvYW5JCWJydQ(#yrL$w4;#cEGN35eHUh9?7V3#o_grlU`(ZAyrZp1z)T=e<-_z$rU*XPQ z!Vl38Jbq;q!X1W~h0E|gCVHvK+_ZpJeda#2U;q~m#`{b7{RVi$jdiMkF{l>L4&dXF z^vBTgh~e)bz2O$L9sZfFpK^XI zZcZq9K1*bTMbMhv@4`NpP1#7|(0qlTGL$5d^TFj#eg4GsU4%c6C9rxV%!RUS*NKqz zdPnJrtxolH+e@9|Q+6r;WCaNOo@dBj3ucV}reDEuj)&@o9tySCCQWvKixBV2XlYu8_LkT}M z?@PS`(WO#}kJI^;v95EObc!q>FX>glGaO(?y7-KP`orztD>yb*OrlrXj8R>DqdU3Z zdGH@H_NsyUrwoB}1Eeb5qEGeh!IWDIp5+V8-w(X*8A!V>owuZxl`Yd~ zqNBkvH(i|0-5|WodRYsre^s)q_yi~ErucMP#RMJvx3`0)v`C&8Hyj0-#xlx@F+3^C z#mb2a*7WZ2%k=F0CT zu{uwIYoADcQ6CwivsTBzz`!%EeD6m=it?=l78EZ_85O=;BzKg^WUf$KU*F>;ph-zR zggNK8B*EpCm4Be?icBy)oxzhn)$tk35$dhd6^EQS(UaZZQu9-0=X&64Q~ zL&J>eXNg(6WwuV0Ot_lqR(#Z;ulx3aXG`FSgJaK03!9++EX!oAJr`pdU#d_bLhZLl z3*8Qzpc|^CY}X7`He$nZnOW@tJpUOWV=>6C=cLoCw0 z;fFZWx>&lfCriU$LQaof)ibU%FNGrC(MUU+o#prPNa$X>Ge3mW{OorpZEFnpIW#2j z1sU{kCR(lk%QNqqZVFFq_d(I@WrMRjCabkc%8c*1PR~1Wl~-h#l9?HO-9du|u!`Q- z+)@3TjB-)r9AcxfB4E{`JlCMS?n;*zOnUoG>rhFyS1*nh+7&qm2tb7MM~`>zBYJ44 ziLz*_R){{hW^JgSc*r_O5pcz^77zr1D0qq48H)|pgo1-4q!_NJ_PUCY!{^-;Okz2T zQZH&sgu=v9kjP+ZbjPQO2xaO&8LOsPUA={(R)%I(FlcP-lq7;h4nIY|Wn6*(d#@Tf z;fZ$ww>AFIGcj=!KfALq%1its{6eJf#9a*4dZO2xel>Iw6r`*eDA65!lSwvhLo??! ztKNI1XK@UFfPFvesr88uJ9^G#`1(MIVK4*^HDyMRM2wO23cJ(76m{I~S?tEdBeA z5K3aN?+mRn&dj#~H%GX4Yn@SVmfU%Pkb-TEyb~ z%Q6hE<*SDY5}{L65j7>D5< z?rimu+{=ET59r0Lj~$x)S<3F!WW5OV-2GX;d=tEnhv`K+iuu~=6zSG{qxlolaZ|0& zTu#tr`x<-N1ec`7Q8o853j55IbC=tGOm)NVvLf-wO&`B>$4zAfp(I|~X`+SPf7Btg z=`Q$X2>q$VzYs!VLK%wGiKPwFX88mnjy%@d7|G%4sC)Bz8Tom`$Tu^i>B6R1j2?E2 z9q27kcocG=3uanHAiQdEI_I52E7u9oR$2!Fqp&UEelJOwFopl#rTrZr)c1SZAZmf_ zGnY-3q~J;3Db&N;9#nG)POe($X!@1m8`?A6_6iC;31~NYlb@bKox~vX&|_uU^1R-m zbZgCTK+Lf0au*d5X|+@tz-_N1+dp_R8m{yOtACuiMnsN}ZRe4XRjP8sIgB->`}>j5auHuJ2u3i~|TKKo*FC{>M`; zWkh!)I{oC2;H>aVamfi25$n$?y#-$4M|Cd0RAQuP9z_ej+s51=NUo8R6|35o_Fip4MGin>`l?)bG@!EEb@y(670euWgdy zp65fUYYK0T{JB+KjORoolS&-12%aC&W6chN$;#6(Ap|ZY)p1u0N6srtvX@Vuu(Jf5 znPI5dc~jV}@8;}Ee+~6hX)%>z3HBb}3i6PNh#`XD)AugzA`@Tv@W;@K;2%grc>&di zo!~dd?8Xjcd~KbdJEpJd3rR;=(?<5E%h~KR`G6n*_S!2-x^uDjhKrhKKG)T>emTpN z&s5Brypema(Y=*BMHJZIC7P&)2X7;`o8h(95wiMYG8|t1O*LND zr>}L?8C~47feG$$C+hu>g#ENOzL#c#LXLynLy8_1?u66(Iw{1Jj*E#j;dlL)C=yIa z+o27%w1HnQ4pL3%E6{`_5n)7CAFCT$L75GF@j(P|$u^&nY{>7+1=$=AE4lBJ4T+Li zpY|6mzf*{OQ+mLgZ!{Y-v-6~KGlBA`XgM60Z=*|@cm1wqIcI^y}D%ENgQJmD7Ry@ouH%EMdXl)_*HAlh)w z&saAk3+E3w&l4X_3jG&0x+5Ca18rla6>XWYdnzHnE$d?YMM@Imn4_MsQ- zK)~vf{dAw04_FgmUZH-t4Q-z$K&twu0_F!_X-8O^?bUI)c-$?mSo>F-XuF;F%``fy zKq9{**i1fzB3jReH|eW#Hm3m6M9$EgX>(kulQ%p%%$2s+=s|5bC5Jn%iq;20??h(* za>OCPEe@vs{T{v?k zqCmZQJV(#Hyq?uCOpzqa@u@*j6hn(nGw@4KUaS813WwO15~Uoa_Ce3Cr=S5~@3j(+ zgj{}du+MMw@%}w0dgWCG3_6>_g#pizotq&p=krCNK$Z6=G>-BM`akvtJP>Ag zn$g~-bQ#UruUjjQAG+^eR+j0TCB-|X>J~rqoQc5O8cZX5=ofj?X)u_CF;QiEVG}yC z_zY*uPuXp z%|nfK&!RI0w%1fjwln%bEk|CAuRHH-W3^Uk$h7t9N4F#4w_&l_aZ@_!K%_{3es%_! z4tRcx#)0aWNBjWle58&dF@ftzsNp@~xZ%|JZ>cYhh~@~rV#g2$I#BZ=gPv3cJD#r1 z*iumI=%llPY$LXuv5_!Rne_DeMLC5Z?|Z0D7}&*Tt97IN$Aa)A4#M^guZvzdT|_UW zur%C5`(D&KFpbG5;_xcG(VTv+yJkV=)vXGSogu3v`o+i2viQXRo>7fIq;}1qx-4}# zltqDf6xY6er1^eybEg@C*hN)CzB{NbDvF-w{z`t5-Y57QFzdop8A4@lXVRx1T zrC&_R)k+3G9{GfI$puCAnEf_pE~I__Q;rC_nJ1trf0Vshb%#7F*8QrsZ^$ujN^H}S z1|le@Maz55+RY;kOMQPmeuCHz<1~VXxkKW;!~a??d98ql*Z%cbL_9?G~KYW z4n_FDY-O(x#P-;33lnVgtFDIa?fP&Q`QbVR;XbYBo7RYUl=)nAtKNDR+37dws24kf zm%`!U3g~HR(obiW#7vJ>T=7?>{ALmQwiAw4Ds%WN_QfM)Ss3kSmIjP9G1{01TxR86 z{E$zqP>NfQa2n+4o|l4=2+E6`N`7BokxNN-v{58URdPm*B`2j!x1Z>ZpYtx1hXqv; z0CGc2D^b>KeSlewG$&X^c>f}!C>M#sJ&Ge=g1C>4;hM^u> z-!HlpO{20Kl)Us8jM{Yo%;2;JU5np8J@*?c!z@NE!d!PfT`V;gE*tIO1RQVWfWXY( zeA=(l`07SwPrUqY!H#Li$IQ{Go2{|-nT;C;m1uIY#nzj$xG7oTMmu-zRkfWFMZ&B# zDLKRlRTOzJ|ENVYi^slUZN=OG8WST?NZ?lU!+Uh2MYu3W+2sv9d`sRZb?l^iPQ1)) z#G=_%-q9P@1K{=aKSU5A;}X6x(k>3^IT$Ae@M22J)a22iu(onEhm?GKhk~CG^WQt| zY7Z!$je&0Z_qZG64`!@)W&+q}$-!UZ%JM`x%650{Tu)%OAxS7-S;N$yw!Gy-MwkLdKjx zi)@-6@b+zur3v5QE8Ni!rgvWzrmg}&3%#LA7WNq@_FZulc)(PAdNcY>3~WqI3)j1) zT}w2)zMJ&oah#~c456t`&a|k$DO*LA#_aj*%$5Wd??HfT~xRlnrX^rK03JTYrESOJjE$$7>=b23QsTbEo^vnMpe~D;!G~Kutc>22 zKflynMDS4BmSc6Ys177n5#z(fd z35PU0g?3Lo2OoR?sRKhRm0enDg&m^KdSW-wmYIan9#uWR>BPqg+_*+&0@!irWKR`z zsFApQKWjWiGz+K;Gr{x1x4bXZ;I**Ns)xrKHfATS?Na`~VuY8;Tl;)a{!&8yFUGNz zqOVWnmq9g@`EVQw2=DyxFdfMHI6%wiTREamFavmy#RCM7Keg@1qZA5SZTv^wSmf)W9Cl_$7ny5&)S+SAtZZ^NDZ$9X+D7Vr8cSO$+%xX(jF zNld5+RpS9+|Cx~vjRM@Ca4fh#_;Mk{bXq%XIkFLKI95HE%;Wj7C z-5gVyh}Xvo0A}VuNrYD_&WL1(Piw0ShedeOA3fOGCbT}YJS7F7pD^AL`&`{2s8I*P zWNAq3Cx48+4Wyt;L&$h^lBQ<6x*MbY(34f)m`lW)8J(27dbCPtOlY@>@pW^ky};=> zLIZWvTSILaLJx_SB`|N#E&W0kw#2VHwrSxb(*b$29;x@n`c3n4ZC7Q()CO!5Zvgs$>eq5x*T*(tR`-Po_ZETN`eqjJungN)b zB$V_zu=1{L%4n+`whqFDUwMEGqJhdW+K>SS@vBja%g)&p&&Vy{sDK*=%psdItwk{Z z0@)dJ)^Upy*UqBi(fFLg^-5$_MkPuf)q=B|odwg$y<*e6QLXek0$&D-|42~Y0u#mP zHFqp++FB16!UsQpACq|=45LB_Xa~O4w0u8*s}6wkjcxM<({oRo^w2ke{s<^<5fA|f zX$Ygf16{X*05Jq4pnq2thFQ)+r(@7+GJV}?AOeaAJ))(thuq}lWp%9XTd+6+l0IEm z_M_G*H4fsxpUjH}DAljonXW#usv-Ux57y z+6(#LrKPCitgjxTK<1~X82`@V0GMS6yI*mnP^SKy9xz0#WWmMq|39I!7uH`Bt@yvm zicuh?f4Jx0hNr;)4up&OTYx?#p921U&BvHHkbEz?P4zN`w@8Q?KEdFAntwA4rau8S XSqWN3&&@_4z(4ZRDpFMvCV~G45VaSG diff --git a/images/adb/instance-principal-4.png b/images/adb/instance-principal-4.png new file mode 100644 index 0000000000000000000000000000000000000000..44bd49a96a5432148204c4e9831e0ce71f49953d GIT binary patch literal 20197 zcmdqJg;!Nw+c&C$0)m7zNOw2V-5t_hB9aoqCPhF{K)PXrQX(ZSjes<4Y3T-O*yJX^ zx$oQOdEfD!Kj4gW90OTukdab;2Tg-~03a$GdkWq!_Tc$q4-$yV}q{b8WvJ5UCZI9XT~VkWhtMH@X)_ z&Zll^-!V{m`HDC82Bov8_r5#>7FHCCRINKi`EJDAUE4^4yA=0rVRtJ=X?7k_zkVDV zfLg*x-$UBda@Q7<-k$#6pr!v^+iiN9_l0y7pHAQ2sf%fRYw{%~=509U-O2d7DmrW` zQq>-T(stUC*y4!Q{2n!2yar1KV@^XMHI6NROo>o2_E*^CM-sC^`k+bu#ad&X@ zbZ~W{M&8%T+SSWboR$_@(SQH_+fF+lhySU`#p7?cKnJ;y-*EA8a&!In-k_)$@~nuK zgO8n)p`3#=7#>iE1it{k*gx0*Kfd{&8viY+=V9kA>*@>&dP@Ay`2H>YzrXyy3;t83 z(f?G*!!Pi^tNdTz{4FWQh3xtN8i{|0`Jc03oFyKLasBt1Njw~SLx^|hj?^7xIq7FU zcX#LRr<%%7JXm~)8on5s^TUh?1D9PWa%cJ5QnwkgHwJslclN!8?_6ZFSw97&WZYEW z;@?0bOP5B}a4-hc9;}ZY9eXbLscHo1d9`?G=xKUiw9w=$1xr!mqNCmezhd!7Sls?S z{O1f69=i1vUgSB!f6nDmccr5K_xk(6)Ljor4tf5T#znV|68`V;Gw`wG4IUNBe=Y|_ z|9dR;e;55@it~2A<+5x(_dC^pD!VGfBY~{OZTC@8+L*mqmj2sHMLFt-3c8QJ$A9a} z;k+ju<@M=3+TV&O@IbRZ3T+K<{Owl*Ix5dZ7$fCBUBpHAltX<>BD(xVG5c?GwrFuX z4T;lz{xL@s3j_P-^*L`vcQzuka2D%LWSWdn&nV@0~~mF|VWOXrK5%l%O$ zy6D6!t-O?NNp~_4&lh`8iKi*=1S~wfd>(AG4z-T$ag9%!XV(#!@74s-7=RIsB0)9I zB4Jt6lt9ooUu+d`h7p@9fe*G9=PNOz1iET*H^+sN`Slx6QFZZ?u_ESrJQUgOYGd@R zX~%MPn~%cJjjh56D_{M3=7<&AO&P#`;+JcY4$hCTR}JVqb=6(#7k8GpJ|5j%3a1(p z;;Nfz{8P3G`4!!iqRmG`qMO%e3z}6h3BT#rGtQ%5i_0#G zBP=`H1f5ECRxJWf^wM8XJB-pNsKrOW_5H51li^%z)Bw6V>e9HI{Cdu7a;Zy};%wGK zbJ{(UW7IxGLN!5C0y6EpT{?QPT`oE6^~eOrL~?S{A}DJ};&O+-i^L{xT;GHvnr}!* zxc#7u+%*rep21aGGhw{>CpShTH(Jnm8xx-c%$7E)2f^!6oBAgt3x4~})xC@vdG^@> zf~C!;)0?oM8xQ@7ra~OaD{k?FHk8fpD<4%acbYZH0*;4|j(j^0{U{%)m}LjC{E(X0 zR-`ydshkK1yg_UfN{67?S+8+uaQ{GUb#%?NdbJcrT)=78kY|@{#E73eMAvhB*?Oyr z$x*F^-934q{@CucR}+lp#IP!QtD3I~+eR1_G~>o>Ih4ct{Y55O47M8$7p4?7lQj_2lxO9GaDSh9 z)u5MlT8et>Z-cpOs@KyU3tZ+JuIXbg^ChzLUs6cUB6u{NGs;vETN^rMQR;-naJ8H1 zXQH)GgSBBI&0mqji)N%=G*@&Vg+9M3Z!q@O-6fv-ig=gLHGXx7T-PEG?v=_!B~H$` zwq^~9?lCC99$6!)NW!>CslZKU#qVOerufu1mCU#2WY*B1Zh^r&mhPcLM zkZpddI2Hd3tqbs^5(v39!_NG`WXV@<)JRV@f4<4z$DEt@BLuCmpR7ZTGv8q~JY@eX6Wh#3O(s2yt5-b&caBJpxR5N?c?v2>#ZyQoYxP|}W zPhqF#9@gOk%Z2F0 zXZiccD!Y~HDM7i-FA4*RLjvyrC@{t1bqn|TTDB!f5f z@K=9in$aWNn@=W8Mu|+E;sk$Nh8+Juqq3D}8z{H}izivunZWXpB(Y+l9xQ!(nWgtV zs%b7YFQ~ch#!!Gi| zpsN+CG~2Ly_Fy*{t8tw4yuCh~{CW5JjB}k?u2azMyY}UK+>c`xhcGnnQ*D3WysaIQwe#p6Vou!wUHeJ zz4Sj{3g4)2J8-j}X!#^e>%{KQ{$t4JHy?qxRJ^9xp&X^?QD@{1`u^b4g#lYNY`@I{X%AeU(Swb9GVWhRBx{~BuC>_In(y18f^)2dgT zCR??08aBw+g_1^hAGuXw-W91ce|d9>a-#C6Bf$KQ-B06kbyQZa52-1^LVZ6To-wO` zGVH^Pn+rIbOQWZlmrPyR`den!vcJ91a08K)|BGP`PR-n+$Qai);9 zq-b+j9@h0tk0CYo)0Nn+g^3%JA+%hunG0tdii6cg-VwhwQb2r#7hC&!$}Z(eAAbP~ z8uow>W(+wo^yz-46E<$#{W!;%PgW&(I9z(xp2rU8*+=jg(bMm06RTEkZ8I$9p`|-4 zg_-fP?Gan4HXoIlvV3&PP?Y0;8Y-1R{M@YDe%G~})S8T{KV3*Xo%h>U8xr07Zo_B{ zVW8hh=l*O)Z1(yMKA+bM%3&tcRv|B}THP;yX91jLoKQs!XPQZ;~uo6FpI2`MY0f z&FX#Q2E;xlV)T7UDb(!YvH6GHRFYYc^|X44T^F#27*WD8E3}WrCYO_%ZZ5WkuOg0X zv`0o%xw0_YKhcKL^3Obe@2-E=a=ABSn3YhBPNE#w4J{S_X>BT$*#0}##~zDtX=p`5 zh;tzX?`pAkRQ8^R&%HAH?@=;$B)cz7Td3!s2ktnL}uwW-rPlT7aH@4 zgFqqcy_A(GG7L1XG?p|rsPD((p4{_@H_8qjc9dP|FP2dc=KeHuS@>*eam%vE|~GNw#?zz{|uQBGM*8w>CI#2Ma02#nOXSW$HQe@dNfI5 zWRA@w=yhwRAr39@AEHv|=w5%kWj%zV-g55N!{|O;F&kklP4{QM{8}^;$<>tFa<$d( zzd)78?=qItofZxm0@oKYGX0NMc@4i3~2?q5N zeQTC3vy~KH;PdDwXjv(r&G}ilJC_@zJKM@@ zMl|e*IO}>Y9F$Iu+0Of|rDKjYE{TUq`= zKw3}>;wUf=VGE@(@j^(HLJ9&W;qYHe?T*f6&A#rP*I*8v2hja`*{LM`P+Qw`yiBf6`(V>uOl{kmB3!O~CK_@26)DYqFTcs>FxaeT z4%6gJ4)dy1p)X>M9HpU&SdUbDBl zxj9qx@lr3AP>rwRs^}^&gc=tY9oNO$pBU}hlL!yky5ee2#e`e#XYkm5e}T!PGR-#c?A+WM>gas)9SOH3LEKk}YPSryJRM|G zr=TukwAFnq-?z>3Vf=dem5||*rCcd*O1O2^ey482*R`W)#v6kBFS1AG?PwZb>Yq@5 zSS}(c|F$ZvfW3vjq0@!Q`Q37U@^QSJ*y-HP`&v)3DaA=It{7IfRHA2Px)?>>cRxz=MykZ!^NwOyE4`aS1e;?m$k(jV{P(yB? zQq)?wF9@ykGIxlD$Gc_9ugMe~GIG7$y(knPT$ya6tVW_=FKv@Sm=^W(9bII zJj}RAS910}oAW;ERhmzO5xD4aB5L||k6rh6s)TEd)O!l`5B0NSB4F)dJ$N)2bO!iF zSy$CDf<|@OzH_D0BwL;`dVZ(ryIIMo9#)&ngGrYi@G|qGunKq|TetCxkwW$EQbFzjUXLy3z&xd=A&aWV9C;*OA4TKU3jMz&uc3D7z%+0c)X z<>@4O`$&k2g*xp-KD5&BZYwhN^Fz0pN32soUj})Zqpoe^pDGpieiy2iZ&_aRb%~0u z;c>t=xaA%CZR+;pzA%~AMX%P%PdYWZE60dBR(t<{rhwqR(q*2vQ8h!3c~MV-&&*^C zX(`iu)U?4_|1)Ytw#;(+7xK3BaILwpa(0JP94gSls`NO1&@p8Lx zbGWkT9$S>VJ&vmxia-0gzxODdUPJ7#v%2%K>gxVPJyXR&0AbEd-rPvw<$g-*XoP=g zua6)By-vao1r1Z7p(DPXvUIYqneIn|LR@C+TzWhXS5%{9u89Qc<{HWmmrA6cbZZG3 z#PXXi;&5HPhvkXYo5@-PKU0=ls9aE6=s6_J1qFUubWCWvIGd}JylE7e+)JpUw<9HO zDR3LpR{jsojpkXsXW4{Wi+8s69~)r|BGjJP5vUVQgQc5pgh? z_JmKIk&>D*NT$&reST>R(Rl8h*d%Oog#oq!>|m+ltBnlfRc6Xn*{Sn(@vv3QCG6V7 zgH%5Wy07ufjC~nsjB3h|X+_pmRx0a6MWeo!^lH^tw9D=X1!9G5H7uTmPc-GAkK0FU zD+vlN$5)|nnIEg^ZtWPOiMkqTF$lk|m8CFRT6Y?(v)$rx-D>EzqHtDGrXTlcxBKJT z9&PAmZaMKEn-j`3M`bf}XQ2R<*M9X|9JdgKxp5Jd=6v=2)nPQ)*OTHIKEsKRG$o5h z9CYR13KM>|&tTstH8}bG^j%T2x$}2}Eo!Q?@xqS%&bVji8{bp8!%}DQE{Y#y7 zGhNthBvM>y4MP1h;9H~~Q86uaU#?zT;za3l`hq>8d6jB{x5jI%kr7s2S_d`VQ{~`r zY?i>W7r&Z@d>DNlezB zZGDR*68^6Qa~cLzjZ3naSjK;K4@*=!+)kmlXw>OA-~k|SWdU@P+VVqmhkrf-ip~v$ z7CN6hpAmm={r~4%{Hhn2rUHrznO&lGddWrOev2bK@Z@=+M%IWD-F4g4;97a@sA(N)8QW_LifyYs;qs%IE~b`9)R80M(#r- zXTu9)umg?)0+-@@{3lM|$S%oqNO&WU^%cLsJ*C5bu2!|D^FB5Wfb(PwmS+J0fo+hd z`^hFC#LizE^cxk204jev$OjYKY&sh1cW*e`se%;%wS*B7`dN+{)Jv6{2{N zw+rsi*9@4h@kNj?nZO6S#*;k);;%NSzHaXWczm_`GM>Y*6deZq?^%C=aocvt92Q>4a^}7r7ag*4sCDl zDpR1>u8JRa;(O@!L~Qr~nPXkVCq_*7lluJT$)rU#k&$g2|0ot;{?zjf4&?Ta@d}%a zpSg~2yQFGe1X6u{c?NTD2I7pvc4?gkXuHR$FAPXHYQEzwNwD0HA}?t-5F6Rq$`!L* z`vzuSdd5x_KS!eYhEktr2b@;l-dt|@%a_r`i1BjNuf`(~_lZqoE!$TB!w?gYX8_SolXcFEOi$tH}vTMI~`3Q+at;~u&gen&R~$g=p~!GWL}Cy}&*`yPYh z^!pqHr zpfPL$1wpgC1^$u|1vNO;_iQh#Ak)3i6OhKjLZ~n6BT6wLKj@b5=4t|OMJCwdHftu$ zRqGZ4{Tlv!jvPsMEK5U@tes(621+W}DNxb+QA3}^O#$Ho!M8kIuFRN5&jA=|S|?@T z`BG+g0PkUX3<&-FH|HybfP6@Bv|hoDZn~+0#PF+H%mI}uvWGz(MwFSx;ntG^Dzv`x_x*~V_Cf#s0?>8!qMv;hNZULxY{ef0S_QoOqKjUMi zkfVpaFyR4g-Yl1f%@I&%iUeqLr(X@uW|qO$;z#hXxa?wHbv$~i zh7`xPZuM`+>RcDptN~efW^|ftszWLwDMdHhIFK++(0Z6!U;B?e@68bgB-P(Mf=Oyd zm|r&O`F)}DB4atT`AKLn(Afm0lx@f2K?k;`IMJUt9-x)_xtlAr0e+bF2xXpUA3qrG z`xLO%<0Ukz=Qvd))Ov_zkAKzcwgZ}91)V)r(l3yjqwiUkcU4{JaRDF>*-Ftuc{1kv zt+#`ebmVbCgh49_%{tfyTW%)Cs6@U9*h7kS zN_x-s!*6h3Pkc=A4NbH9aoKzY??(y{$*+G~g|oM6eHDbvT@WJmDiGBc>u8V+stEH$ znA;|MXo-kM-Oo1+6(ZSG4`@6cKDsM*Z#0-9X}GajTJ+0!@8s@z3;Ktl#6{0SR=dhy zQ1j8q`z4*CK!UKB2tMPAGy@9*-tt*64lBkDpnTY6<|-{$Iqs)vOu3is^^JQCj?=Fj zokx*sQ#MU$)U){j`CP%xQB1C;*faKkQk2aJvOD71C{v`@?v#EQ!rUY?Rr zXhwCCw8`xGL!N55oo*7k{MZBJ&7W?tk*^A4N8`bjvIu``dmCAJzXTUnw}2G^kQ zedT6~uDU;r0t^EBpklU0e&HbDu+(N$jlprNoiKNDZc1(b^v20@a zUeBY9Q(>faI3;Q;_J-~m&pmbu4Gs$n3xo_{qw}ES3Zvxpe>ZcVKga*q6*`!~{WB(| zBl%ri;AAXbYH;N_SKKFDOWmTeNOp!#^5)M3Xg(J7Jx!`J9i$ICTy2NXPFxM!=N-X$=m)e`;ngjKSjdiyEcI*d0*>e9wvRGLGTLV9 z2F_B31Qi^z#jjJ?l}*knDW11r*5HYF@6<6ef3V6T8U3tW&F^9}7w(~ES27Tn9A92^ z8(U6sl+(;mIdOBP=a;~^B^Bv%I$>S9pp~;sk+>Ui9>Vp-;01M!L@3y4b&<9Y_yy(F9! zosS(37JqDlO;uA@U0_^uE3x8h;!DDp&GulE_XSp6vre`|z=+_C)9Cq18(V3$4&$k7 z>$PCO)nPXxJffS@Z)eqO8H1^S67yJTe1U^K4}l*{&iNHpn@ty@zn9`DcLX3|ek50m zkPQZtwl^1aDjMlL1=im{0%)Kh(lmki_fR4?V}&R8FZeh)-ewJ0%R`a^)8@_rRs0e<8j;0R@T}O@Gmfkp|Zx$#w z`<-bZeV=7JBt~)gai3TXCVpnhxzX(vheIp%4H<4uY4P?AZK9J1>6Z^c7;7AHuA56n zA`c?q%T%{NWV3Zwpr5qx4%qTjTwotXLPNnh&rvm()j}sQxwcB$s`JfvlR{T#lEi#XR#J>Rsry)bCF4Za+0IvUIfLQmzTzf5;5N#ktZ8R3VC9ofcQ zl-A9wV6uOcG7dNyC&xGjqA}79)n%+k+CP&;(QIYkX#;?(<#2s636uNmO29U|cmX`H zJYa+IfnjlpKw89UdE zSI(9UPHi>a}g(7ysRJ|Ahy!3qH87IkhxtZ#x1N&|Su0qFqkjv_H>bR6&? zxdypAu$@@~m z)G|=A`?t%HT#S9GlfM8iFTrKLPl+GK(9{5GdoGjrW)D!OOii;0#fXEqn4C($PPApj zK7-tssW5SBkv9g=zsj9X;yzRLd;TO{VulVIg7zv!WtP*eake_-PFKiMkV?y@b@FhS z*60k<6w!4(EmzUFZw613k%uR8jp2c$DCHTIaodp54w zdY(7GWB{1QCw#)kggnnIY$01kg(F~!r^$K)V(u!ko@z7VHEEVYSm(xVheb?T_>E5n zmvaJb70BMeSWG6Z;KL#zMep|r-Af-?0Jw`kt@{~zz_df5I+z{yjA%9vQM-tAoTDK= z*P+YB5S;bH5y_?3Ru=B0d`i3V@knq0 z7Mt_>7>(2wc&KU5NOAXgB=VS`KD3X>#F)v^*Ew@To=jkvGAKk(p3b;tKL;j`zeRn) zx+N!W5C9T6Kk;6?im6cQA=@CN?K)csk{pzn*_p~+uF)zfvBX@Uq_dN`tKtC@@D5~@ zbAZKKSE9W)rJqcss>EN;4EQ)TN@5cgmz}iH?tm^Y3{6_)b9c)|3-gbqe!OJl<@!4z zK2+T3po#e(5BqaZI76Hx(#}Uh=dwuw_>vVmM8`!*gV|_nI9JgHM>#GsyuvEPfSJY! zN;tAAt+Yix5gI$rb~p>s7Blv0|B-rNx~oTPA1W)I5}-;MeXt5t5@=FgfY|S&-;jxe zP|{_3Ixz~NWg``ARvPv_}ZtsE(tWEF2Axd%*`pZ)C@MTIR+ zaLNgdATE?NcYqKVq#Cua2^B2i{rhJ%%r!#APEbPqmiLCaZnV$*qLq=WO00GCP7K`SxWBoml)IKH+nCf zl6Y8R^DicbxpJ$8fjl-LafO+e;(n(CM`~FNq^?JkSNs-DxN9I+Bo68CH#<51Acq+ACyV@} z*%;#lXLsit=Ws+$w-?gyAUs_U*mov5YR|19ak>Qr#B1$Gv7ww6LDwfGo)Gl47JCA7 z0d0C2b%N1RP4O&T=iXfBhwo(|t4oURLeJ1Y78=_=9$Fjz-D*2Ud`}N;o9YurkT5?z zfqq0Tt86nY@jGtr0kLAI-D2`66Z)V<>hC{CBa!DRTGd}mY z%KUOI@Gx<0hAy9s~?5FVuKa-B(C%sTb)?f*B%<2+#Z3)i3(fCE64OE&J^$-uJoK3r9N}cDF(ZT z*bK8TCG-;{>`2Ld9D5dN>K{RNLRU4*582is z{jrNDG{Z+j;#~a<&mJ*(>FaK)LYN%i`$teF3KwQs^XvC_Ba`u7h@%FMiIx6(;Aqq{ z^y!hlc;)ilLxP9g&awv~n0fVml_iL|?gyjiPy5O7CuZ6c4D2=T zLK#!t-l{>db*Rny63$q~x(iO)r4apNV(w^(J~msfDsbs?bE zEc6TEv4r*ii216M2n%FgW5w9lo^LYzWVMUQLAzg6m@dMm)K{l+Q@_Z4LnFHHwGu5j zROlW?N#*!1IM9Rp!U^BwJCVx;E8ptb&0g zYjQl&M%N9@8`APaB&Cd!6_>|}YnrzAfhIRCdxcz?AZb?i~KvoT~eB=&rf{V#pKZ-xbJ20L(m>+BzR`g2TgG9C)T~> z>XF9!p!TJ-db65IhKP2>Ol9lra|z^qdF!b!nd44HvbY2V zzATPb+5s%ef#Gj@xSYDGq;YLe$M3;uU#N_>BnRq#X&TS z@V$MiiHd%)F$W9NYC6996Q$i+c+KIfL6Z?lyBXv-|I|L0ZT!I-sm#$I6n97+nE|Pm>;ULVwJs&IGL| zWmt&8B_8>eH@SyfJ-I{cm{QT0XP06j82*RU6N_!7APQn1`KF{R@W*{3HJgtD&ndIr zM~o=2$)b7INg(VLTP&4rsZFb#tA^0cD&_~mJGeb5T{A^uZc>TGq zHQ9$og8(jw7oU*9!2ThJMbw-vK8MC%Q&t(&|$36vcC0I@BCz|biTZ_{Gy~rVltsh*eIQK1!|CF**j}0esW>^NrWNt z1>R?^uK~sj(lwI*usdUHpgbgZz=<8dzBM~}$f80S{H*yll2G9tcGw#PT_&p-EaWt+ z4{i}4LV!CwC6$we_hoBqx}e>rM4rA3B^WP4YWxkWbtsKqY{Z#V&KfS#(Aqsv?|)Av zaQ_|!+TBlk)a@Vn=7&d5-%Aw-n&k$(7S2OHmVXqoEnsXJQf2r&{#Qc!br+f6`zD0a z>_7RvF~A86a9URWtEZsJQDZN7wn@iuqTO>2lPVlCoB3Mwq7a4l;JR+VR;4_I#N__O z7o>zuhWFweN8-^fo`k@a=8rCuLmJ$3b#yF;bAx1#tA&oQa8w@s;RqOVWx~;^f6m=P zTP&LF`zI*`UJId$%#9z2R&_(;N9-l9R?Qsm;CB8d`=XEqu0Ei<_Xuju_>;Peis!v% zE7{%F#!(EOTI&xTPfO8Vo>BbO!Y*ZV)FXy((K&9Jj?d7vm7e_VU>xoXJ?rDvKTUnE zS4rEaSZ*f$J(j!U}YBZ)IzW{C5%N%`G+dX*0%xD_BhXae#<} z1dIn~af+16s+={p>BvMBF4Xh;a#KWOmwB!MX@zcUts)@4&<3Ph12UZL5o#GW1*f)~q{HQY^AadM_WmfKT5DL%8 zl+|Wt*IYXgTJz@wMh5~EtGW<)?QFc^x}Dd0b5f2_G)%E@L0oO0*!{ZhScQLOO4)jY#L+`#ClNyWG(1(e`WHtu+zkw*Y9|6G){`AKL;=Wb|^sRZ^|5japfWyDJqcX8PBw8> z&W(}q-JARz$(gPpd3}t`Zp;U6mL#CH&YQsfQbP)GThld;P&MGdnb&r4kVMXxr!M%c zME~O%0r^V3Fwy(y)etyxa{%` z3QiAdjM7yfGFd(wX|jq1kNW{RX~Uw%0;I>eQ|PMk$Qy%{+)+7v^~9uKK=jHA8lQ_H zl04h-fM|C(A(KpIs8mk#JL=MP??b3mUE1-W+t`RJ|Q>sdJed`E3)E<@ofF0e|>yO8HSwmhPl-V*#*q z`Pf`+bIdVcBW)fKg-&~=rQ=*PUv>352ObSR-5k}FOk!xI^FO*rDYGL+13Bs6$7%Op zYzsyhrEr>6`sZW27tILN=L5e_v|n^BQFq<@d$9rSN?RTB5#O**B6Dq@Bbe=1bOL*2 zENYy0PMm>sSkH;T6s&K5Cuu8m{7Y81+`Ktnu!ak#qz6^k%O6lKL<_`>NDv|xG7O$w z%0{rC_v|ahhSOGzYTyOWgWXI~GVfW@BE`nS5CANF}esU`)+N^W!SDm~<0>+t5G=Ry%#DY$OQzfc zf;#UDzJ#mUQKs3uWdIK3A$&btOKci--*mP0s7!Pe`0z=SAtw$)wR$fPlRf`{uJ*w- zfkRG8@ZruEg65Rfa6_XXZ&vfg&lf||o?sEQSE3ts1F6v3*u9sbN|V@V*M*j}bC&ICRJwPe$||2Yo5w~R>gK#v zVUoB0H%ZW`dddgC6Thj^xEJRJNAiWA_L6hW{z4ao6^ zgmF`1JJKciD0CIUzj{OiVdpqNrV!xwf+UI*WiO<0wNegCgdx*0c&u!d)~Rf4K>9XJ zVGKxsX@oYdlb89|mpj>Z{=lIh0m3&liHM>5d_?l$v$uFVT)-dq1SYfXTGOxsxt;%* zy##k^)4<(xr^;T&=m#Hp@Cv9(m$ot2@8v%H_%lZo$dGU`-lurb%V{yNI7e=;7jErB z9tXFNhzSNoWt}WZkTSQtm5hRLG--RWbf?nHv}1B*cB{8q<~EbsXY#}ZLtL*&za=u& zmy9BnzIt_f`8e(G(^6m9%D!1oI~@&HVy(m0GhCss^jlx$-C=T}zPnPozg2in8n5&i z8|emc+F-Hl1MOSzEQ5DkQ-QACo|~KCSXJi~Z0&s_KO44zRbGuC%CN_Y;!℞fkRO z7Mxss>Ea4B?kPv)bLOM;#QK~LeD}xqCSBo^s$IX0$EhG^U;`~V9(+~lYj3Ec^z*0^ z8%SgyOVD~kIv1^6ex+Qz7(hy?rV~xUcV!sr4s#=;SazPy+pc&5fif>f@ zl))KSXiwavplIK6iet+WrH>v6$2Tm)*+xk1p>Z1zctbJGx?(rJ#~R)KAq&uFqSHWt zlxL=GLK{J#*{|JKdD@LL`u4h59`D)}Kceg{k|GF-|W zjKiVEecM=d`ViYYr&E5}B`;j}{rsR#1jY9YI5DR?`IH`@^$_G^ylB@M*pm6v)7Htj zJz?UU#UxYC;Je28#QlKAs7U+wD);Xui;3C`L*)|Z)xpZ~DfwfkuPhd5a@jtJ z;V@^NZ_~DjxKYElGK&t0VxBDP%HC1P1~i4ZN&YUT&BO4!t~f8)6$^lC!kNHKDo@NT zE|lum6>24)Mosr&{zi~D_X-cibfCgM+fNY2VWUvQ8zIMR_)}>zlco1b;8FBbK0E@P z^qpyIEp2Kn`o6bfU5UI};Udjq;m!3`$agy+p}$OZJ(Yf6*|aG|7)p|aCN6CimqQ8T zcn@q$+Y}`_TU_?N_@)jd3}>d_5swU)xkoV}PAo>a9*oH1h$ohpO83RP>rrZdiIFZ` zz(VQDuZ_;GM{L?>KR2xj4jqX3+*!g>$;!`<9D zPC952E#tEdy3-Bm)1zlwk~u-p7?6zOSw=hzmno6=h>I5>9w2vD+2RYtqBE~YZ$nA} z6}_ZeSK1l@#|ej{PaAr!)=u^tVJ%Zi-^vzQS|M$}IpyA5eX!U288w8=`0}_Vc@gz; zt=c}ISn~$KdY}bQa1+$4F z9v+Kiv*%MK;lg}-qaDXdUa-+{^0{}Jt>hPGl%kF)LkYO-SR^siPoIAd%Ka$#a-Mb( ztGDA<6G)3Vnzf|n9r*K5L-hxv!SBAHK-*#>r=E7fVUDqQW)6y4p*JtyYC%PumZ-HO zhoyW9#O`z6Mg0OdmP-uZFkeGBRShLe#sC^TRcXaRDivzULxb@l<1sfL{Td5>LOj+3 zRaO+Ot?cR<9DO>{YVnaCx-~JP5HT?VyMWqX><^ub>#En%Ys#=y-u~pceQFVuFf+#G z048i-WG1oRDs@$FH(VaRCH=LZv(9OcO5XKeno`MAUL8)`WoIoP#%~fjMCgh=@2x9+ zkS{2)$GoTOU@R!-9?xQdFArM>U%0aBZzy5e8y zJf4ki3`~(AlPp^N!-o}{B+~O@h&3`NDzb87OvJJLDB)Fb1$r}mY8V;X+_KoNEq@!W@T?)F2<4vi?O0_tFN^kfX3L}unC+{LXD zd7&>$3l}~vcMQ3EVC}Jeoy>hjr{eZ6&uwy`kkd4O2VC4F>Bl;}2z4FGcZQNV#qVXx zQ&cWbpu`X?%QSjMLdVGXIsdA`BRTz++On-0Uj@odv?%r`^He)3l>6xD8sCDG@?v`A zE78^1z0#uCxvV&^Ya;q=Lt^~nD;{e|$8U{cci4g#s*Xa#=+>-hjQOEP&+s3)$63FIwjXB`|OOQc^#xV$_un%FH2VljIw7&G)(@O}{f+rGU zF+W%`pZ|DCE{IEH*RM75F^uc5#D_Hk>-qW%XKSIvSBE7kFIEK^A3ODaSvctZ)cFpg z10_{F@zGy3P@pYbtrz!&-pjNyfUjdEN$%-JO(#CQ$qqV%tYoGrQv94LC4i|!>|iNx zC(y{!@L^Yr&mE82-Ls53c!rCYUo7#yB+(y{DS5RY6UeigH`0A8e$ksD65F!sq0G%C z@}s$;8xgG5ck~tmPOMX_k4s}S(qv5npEo;thZ}#1m2QnyoLj-{9GiB+VxUm%<*Y)W5=rLfb&&E<84p(-DATa}o8py{Q7YQrv;|P_$X=DzI+n2Dz)57USw6JQmMwR5*h7x) z6gr@b9!@X>oElBZaN0?HXBx4lOs^w|78`#C*WVw%jGASk?PS;1nE2#d{66?C?yjau z20?kH%*SsDK9OSB!^Gy^x~L|lRsIu9UU7{BbMT#%c~zrTt;bx2wUjF#{x~_1P??am zuggHz35~+WSRj;{!d7KCuqM{s6mMR`ZCY+OIy$<6X|@JgbHhfM?AzAS0xUl5`!r`z z%##6IbD60NC-Xj-7A}Tm&L4x#x|;4-mdOrYrq!^l3kHpcT$vKztE@6m>m%odvbE!z z?4pzXI!2P47_w_z=FmAzMkfTDZA?i#2J>|kJs`#_M#Z?f$|@BHBUoP@%?T}i+!%9X zKZQO2kvWPBkMEwxVr%#4OO}aK!;T!KCS%qYDX*T%lT>pe>Qx}W#mx}33gzxB;lS#Y z+BEt#O^busZCQ29{xN>cteub%^YQRU40i1&Wsy3cc6uCWA_|f3?M|`QD1{F0xT^HU z%JT}dh^)~=F}5?KP%dzk=Ex)3KEE?iW<(>k3^@;TF>Dqich)%{zpqhbY=|Ry>{e^e z!F@?e#1=1$erm@2NRF+^jHp`B>UoC=U`03YuOWm(Z_7vxj+6`W}38a_+~$M7#?;qZ58vd`v1aUTdbiBlg{Izy2wk zDynh?!CU4_Ra?cx)o1IcGcK_6^ukm8;+eZ!0ytJCs}tSv?5|gPOe`a`?{tkcO?f7$ z^O1hY`OU{a*(=498qzX&=g8FNMWPOs1TR8u=-YBk7zQAh`#$9?)=K; zl;-{*&SoD{kc|AK!sq0zuFyfM+V|J!1 z;&tt%(c5;z>AG+#OGWGYMOlEbNUW(D_V`3&FUgyP=3;(`Zld{{m03qS+-Y*xZ{e=6={!c<8mJx zul6V@cX0G7pV`h_6Rxz5_m4q84wnNxN%0?+iY?!~1%rpQ3s_z2PhHH6iQV`O26}5% z=$?N66_EZi%CeP#1@@Vz2ojRPpE;EKo8=y?2{%$JZG&7ev`)>bt#3u59Ov5>kF=T) zhLyIA$AGY6t>+G#P@C-3m+rTgKgqSw!O~aZV4(z?P zF7abJwl%rHgt^bM#?b!qfbz=1Tomt?C%7pcOLWWNg zKWjxR7qy~KE+_8bA13>75{c6CRy+~ID<~N2{dsY$c6qAAY|%%lO^ULMKFaMhAFweu zYwu-pd{AK-_kXH5|9GbJIF1)xNm}F*iZV%SSJM2dGn1d4IkGKF#fYwKcEpXNZF0&Y z`MI@2P1$v8g=rbhZ|VFh)EaXPXU){a{HPl{>ApJmxX1nZ`8*z<_xtny`}uml%i`%0 zYr}++_12x?ZboCZejX zo}7(Hsb50t%fCyv^jNp^a!PdlLf`wqPt~*Lj3qv*%X$=Qek<|UPl}s=)sc05gVDZ+ zJgcTZ1RqqiUd!v-pIGS$>Thl)8)`lJx%XKpzp1S_bW!T~7)@4e*3Oc)KXI~cHU5Fl zt=Lv42)lLuRAa8AOk6&_i-$6yxSyUyo^=+4kF?I`Z=mYINON=uapFiKB_}hRs$^dX zdmLm#N1(unO|G?9U*6d{7tmh42R}J}0sj86oyd42fjW6yq(=Ma{)N9j*yh!wbW}u0 ztVv3akO#7e>B-!91>gO`aygSRR|7`KXt`^U#n(-wU4|0@$tKNL4M6gE{dB3Oq%re5 zRYf1@rnt<%ck9BgIa*!o(4qtNfK_pi&e5k$%?Bqih>q}&Ck~A&bQw&H;23y9ljBIZ)Ct*+6N7nZ0F>^8_F;;UiZtD7|+^L zh=;CrKa{u632Zn2%DUn}x57&5{3`zU{V6u^pxM0QBVSmWyTHSSU{5NsSNo_@>b0+5 z)P(9aadKjlUYCjAj$rM{J02VXvO$EYao&P9FNuqGKUWyxs#S&AYsmFOn@E#-3iDUz z_?12c&(!{Qn>A2F<$hEQ%-i==VxcWOE~hFpeCVs6|EMpMshPR z3d^fr$4ycuxsRlr7nMQf28SkcHt)TLgMZr*%%c+NRLs}|Vo`}K;A_`_I@0VhrB&}4 zITaA*c$yGaEeBUO23HTjpm?>YQyqF$2h!D^Nqw7G&u`kyk2wAiwBcvLLNf8A;Dg=-i6Fr8k_Bo34v^D% zoi1Og3p?3axEkE<*bW-~#7+8_!u$L{#MT8LVQla)L51Z}96Ayo4pe-y68f|7!zkPq zb}Wxe22cr;)20RICp7cD9h7^y!ox~9l>a6*#t@}0*n%s<+d@e<)On4k(ZfeQel)en zsG80jf+qSDB$1foy^Fm)EG!QRZ>{5EmT^oyPI-XpmhP5N2cApEDxjxlF%Pj=uDcNB z9m!4#o@Ja9s8|nD&!PRYn9dDZmmc3IVGxo|Oef<)d~lblM&tXwL^wXn#OxanAs&)k zx$W_dTz&t{ON-#;+p5dA1KLzuQAHw8Fg3(eJqpAF;Se zJPI`vYNzbhcg=9$_PjJrb$_Y6v8~@jA`4tLguh&4+OS_VJ$TgYZuQF^#44>(+ojcO zZEFC)x!m&rS;tQOq1cHGBcJe^T5%2#PJj!TJiR(=>az>2ucNmExM-na<^MWosg}nQ zvS&68tKR&VQ{M%Yc36*;t_0BE+ylmq7w2YEKM$v`W0ww)*!+s8T`OI(GJvzU*}|KI j6*QX!a!9#b=udwGt;;3z literal 0 HcmV?d00001 diff --git a/images/adb/instance-principal-5.png b/images/adb/instance-principal-5.png new file mode 100644 index 0000000000000000000000000000000000000000..602d209a804a8df994162a4b3236bd3ace6257ca GIT binary patch literal 76392 zcmZU)1yodB+doVWDcuY`ba$5^jihv^$k5&0-5{-iph$;EcMSuGlt{Pa(9JjA@AKU6 zyZ+y-wb$8a@2k(QbDiJBX}(r`fk}ahfPnBqSxH_S0RizIUh2^S@D^Jong#fcoP(U4 zrm~zIt){z+odehw0f8mW+QNcBnS*7-($d0WWP+U))7?iqHa1b)B4n%wN;}xo-_uK* zmTO|NMuxYBtlNk1QLDGr5j&Cgu7PW$y3tmsoBj(a@30APrS*0`;s3}N6Hq2X10Xfjg6Zs@{3-W32(9HUPPNso|V3)^NDp#e3W zpCHVRrwgIMD?N2!CT&48KsIz~*tD1P=dfaLe^jXBC<0>@F(d6S(e9E47JkW(^1b+E zDpi<8exv?kmhmW6Y%)_yt?%K{M?=x z7IsUbc_NJ9DrxRj`a=H(9#uVC17$mPbp$qe9UTEN#sL8ZUPFWz3V1<4Ku(WFK!aZi z;6=U!>A$^*_a(^x*6aTg%Ie4|E5olk*6y~p&K_@FJoC1xkl~`{9CQsl4b;`dtX-VA zENxt@Y`J`$T>r8lNcf7ut4_9_mbAW3U}q08UrG9ZD8%6Pzs=nAwEvKJI!e+TsB6;7 zxwzZX3UUc>@z6_Q($dmOxZBu?Y0E48haG+=N&nW<(^ZU{+sDU;%ZH!K#oeBpS5#D# zn}?5^kB<{h!Rg`W>}l!C>FmMqKSKU1N8Z-M+TFp`)4|1=_ODz^D;F?wQ#(z{A{;x^_e*XWf^8fVwhf{+4ujl_K z68{tCe_G*jmco?a{`Z+lVfJ&fCLiQxcnxdH$&fW$u;-MqHBv4SI1=*ZP zPH1kn9523WZhrTrx%tt@$kFM`ELU^O+~?+&H^#>LR)Wd7o!9i@C@;wn39h1_&kM%s za>hGR(D7(bMpJ$Vbe|+%OY8>j`b`F`cu3V6p-h*cvmnKX(*M0IlOUcr8}on7{paqV zMlzJl!)Av6r2rzD@7~S&z7l)>c>*Vx8pJ=Xnb_YD)PxZuzAE9wdnlTc}8uC6BL(@{20$pF@6;b}IUNI6vrYM~GzzLJ4c{-&QQ zBM-4witShy|LxT0;a=aO@Os-WzBOlFl+_ID4T=~)Qfv172xRO%l;gu#1zgoGdKlAa zk^J*(`Ca<^#1CM`dyk)nO$*SJ(Da~P7ltfNd=XOgO7druk{VmK|9VRm3-3h>|N4sc zzb(5+h;YuZ@7Y!Vjfp2Ml6uXePYKW{6K-*V9Li@{eB$Rne>_;iNWQ{}j^f}l0zQ24 zV@EPY2|G3G1bsF)ud1rT58~G&br2xd!GDX0q;kcHp!$&wX&FDvthAKR@_{TaA&G>B zruf+4)z=Jq*A<2mn~)VBMAWswS0#Xo`;s06a%tzxfyqLVEw5xoyPRM@ zfB)O^Iwn+X&+9J{6oBZT>#@IR`QDr3WxzUTT%g8>_EyF-9&(Lqs6ujfMYFq(-fWL? ze9@3TTqi#r+&QMmi{KpDPc+K)?1G8T@QC3zQ z?nc@cgMZXmc$&yy17N=B*+LTbo0^o-ZYThu}W` zTi(T-tn5)zmrgV4=jm1yYrA}>S=i9i%GelN8w~0)l3W-NX&f#_e{*}QZ1sRZF2G{e z=NfOoIZ`?rs-mx_H@2Qhv{>(0tGgMi)ovQs*f>=rxqbJAJLBvuo~f~_8v{F!ChqM? znA{FZ<6VGrS8CbIa3tF+NEM&|7W!hBM{BiNyH>oa>N3=%u#vXn@lJm1v{cU3$cUy2 zaa4e~qmG!JlhahMW|`JnF)1~TAzGPyNk^vx_C*MR+>iBrh(qZOeD6%TGT)g0spcJ%bPB0w%wy`r$2&DVk$2VBc$N+wRubv|Vi6Hn{epHlUO3Zf(!DYUqeV-?`4n)z{2A!dyOcy->SSx0v35 zEb$^PB~DJZVIH<~bTjg4+Er{e&ZT?qeoi*32|!th#s&``y~pQ>bQkNR#+S0@_1iK8MnoZujEWqesdd$}@aFzrjNWMxHQN%$jtMwa2V}Co3yU5Y-?0o`26xWkI8VbGlv236 z^6Ee?Vc4E)fKnqG$y}eTa+iv%h)PMVB!<}+AGD2!1^La_nAWl3rWe(r3locE7xd*8 zW@Rxx-d*iV6_84PGCMir!NWY`;3pO3H6hAyJ#@>QZ-1Y_{ma z1M-fMUq-rpv}CHuy7jQ9oN?8AT~j5s+^d8wetnoh{6#3WYzQuDjX;(tGD%^VB>V9AnXZnO9+RvE!s>=-{9iZAa3+26Ht_AP3FJ z_XCs;hEg*Dn83qNYCv|>SFak}&V?|B5?4%5(fo8^zhW8!5EV7%FbE`btTyU*XhVX%Z#d+x3TM&}Y#~NLicMYb2%!#;?~Ws#QhXjtC}_3CZJ7y&fb`#c=jpXEEa5 zPilFsMn)S{U+^*`$WE^%EpF`1xGu~5k6$Ok_nNY@Vc?aH#Z<|qG?cL}amzjwxgzg1 zH%9Y-!XJL$=YJnSBDURoHu-yGQ?doJ*G+IXMPoLHQ^vD7jV;^~1Yrh6pZM}>ssd`R z$>+0Ib-z~}x9yCswnzyTK00mwTD_90lt znQ82 z3ecmSgL2138<`i{Gp@(dZP{hnq<-+%ttU?Dtbp91U7}x#!~?|8#})0=@;RRUPa^I> z)nVLc7g!gBP>y`eMcU)7AT%3#Gljkkh`+TT-3xwvN*4ZY%IM~+QsuacwALdvK$}w* zs)s%Km!7Gz-Udp)!@**Y-495-&W9{I^VrpvA*a)y(qk6=-@2AebeT$jdH2~CGLoi1 zXZcAI-LaQ`L}s{+mf)P6tVYz-a;-;XvGutmvqTbE(n5VO?inr+62E`c=5{e|@k$B+S4}mS?pj7g;jO%orH!WWJLS%av{$wBBpGe{5k3I^sB2WFE7u_o2u0Mm}sIV zH%{;jUdhb5J){0A)dbAU)Kn&%dD>B34#|vjV3ppy*qiSEnKgft?{`LhFmeZ*Y~zAn z=Q19cqS($-Iaf2lJ`qRVx*nDX4#bkp7|9wM3T?N7dc5llDA_@$*8bVDh9Xz6)=v9_ zVb7)&o;tM^X&nox>ej)Gqn@iZFF>}#J?g>Bj)kJD?@j?vJ3e#cR>Kzq(O&D|?8g+p zQ(-r|ycUxU6~DZqu*(t?NR9&w;kmtqzm=*4E4Qtc|IVVWMIj6Bxt&RctuarX`Dw~+ zoym}G*T6KO!WIZ()Sroa-xFr;BAs%1<GSD1aq8Tzzkbymoh; zA{aSJ4nEf_U$)tDkMm3W+2Xt!r@#Uh?z7iPCYSu-i4fjO-T0P(RkA6cQVH0ri9Kh* zB#MLrq*|U7DMUp+1VjinfTRktglD4+e7G=p>eksmvr)zoBpE_(ct6UptfYHO$l(CR zI8X=68=UT)d(9onW)c~V@*jD zv=r_%@_kC| zr6?=+L0=1PQD)k=y5AMql4L1ILoHVRPj)2_$N+4f$EXJQxTIdH33+}3@Lw@jY%*`# ztQicR?LzXbyZB#x<;(P%t2V575paF%#GwLD1XNIQ>}2L_H*k`Ltab$r5%ip3g9D6E zZt@?)`RZ3VBKQ`&-qE+;roarC)K7HH@^EXDD21UGkl=cs6SD}`tU48rydjx9<}1)+ zVj^~_JD3>tO1LGhz2{(Yagj+1#>k~>;9F>$e&Mif5v=F8_Bj1fv3(qB;>f$G-ybN7 zzH%UJ=Ft|BDv5qjFOMs1fC#-x?#cT@)S@z7y=Yb{3|T=r(hUxyla~AOm4yRuAkF{q z8(qN8JZWJa3e_y&rJPlHj`Zi-_Jzs9&Ad8*Y7LVYEq*^Ni zHVCNa#yww40UIkc9C&Dj8L`AqyjfWCTfqEfE;63VQ(yEou!#hr~s0gxoGi3mX{t6#zK|nwNaZz;#DdS`U7v6Slt*P~GxD4&CO)OGp zVXx4LxRz)UaJ&PNq!Xjr6O0YS5x%IZ14Er-aA}+YTIu))Lx%XamSQXRMspeS6sRT} z2oPKBF~{?=j)Ho(sZTFu5|wKFPQ~;1yyDYR&Ipj_o>A^*Em@9X7%%`O&Kij@8%p&T z&SLAbtVy$T*cItXe#aVlHo+kA{HvK;;Y3CIK3O^a(%LQoeg? zCTB~HS9x{jDqB&9MtNGu`R=6fv}h4O^*!pjtvDev8}Xqvwr))d}FwQjy ziE*55^1Pv!oJppT`YFI;pKBs+XEyuRqN@r#o=rttWk8Ncn$SRCPgqqa-#X$gE2s{E(;LW&}>1 z;yA%p=D~^BCkr(tmg$KxMP#1dbY$dc$_bd5h(Pchp~NF)5G6b1ckuQ~&(fpMF2pJS zT2|K7)h1!%^_t5gl#v%^NGIxsw{4;PVJCD{Ha|jqG-YcjEEbvVo@JB?e!Nc=LStLm zv>1JQ-M6zg9;loP{0Toia4Sdi+dZXKDk}j(uB=D;`IPb0h6Od>E-^uWb{_@H_~Jw=KyZ zK)gC;W6D1o8yW*QRPtDuVKAat)R{EEf^!Qv9e1Yf3Tv$Q>f`JH8@_kNWD_>Xc1%8< z$>Adsx7Y#keYGbnK<)Ju{@aj3Vro6#GYbf^sHJi z+}8-TXGgWXk4?Rr`>TG@%X0;#Pd}^i(B^h3wSZWlm7k*T9hFgctiYbAyLSvM)BO!E*%x zVO&df^v2K35XQnkcXIWABG8IeG7ch^Kly;XZ3+a&#cEK`Ac%7xKNLg=1B8Vv2*WU= zUJQhep>hh<*z(~}U8|g0c$~g%Xb|f6I31Oedr4dGD#O@YZlR!<4%+vyuwk)Lyu-Fb1J?EDEB&v&&Ek>h@*zZh>BLto0dEO`|U zixbYFVEM$%;ahmCb47H4Hdp90c$Wqdm173fz&Bey&dqc2Z5xT~@v}-z?OTRLe(9l7 zu+XXajPZ6@x9G>#Z2A@JN6sCHw8#Lj9p4LLNj7O+O!#sc%O^}6OP)nqVI5*$Y~9J3 ztu}t5r)Ljd@mc~xmb|F zCSt+UZC|8lxd=A0!n=1e-4Js%&_R90-)h^5Q~s9q6TW)~btwaXSP0n=mC z*w2M6ILIDKMo}2JLxOq~39bjAk_P~i{1kG$j-bSsR<(;7B)dl*OcLP3@4^`+q+++O$57rb5>*`aX1Wk1U`tm4tGmKjPZLK( z*h0F6L+3g;3zMRrZlB*}$a)#+!lgFaYulp6BHBK0EA=frazU8zETBIJx56TR? zt*?QG?N_1WGjq}Lh8SsAJ_r%#k}l=(Ep+vsWX|i<8Q_JCP0D~-kR#e3SYr4qm_E#e z;24L`MDYieD(<@KkAxFWnWacDc?_uVzuR_64A{(e`gHK2L3CO^Ou2w?&2o}(9kRNYb_3*sq~HpQ|Ew#G zuywUUqV7;*5guMc*yE0<2l7~-$pb}@3EvqGlK=VB({qUE#24hf)EzQJd$m{@k!$hU}<$)NJ))fKfof#Nn0H0VGBO57&v6MjONk4Mfo5rJM-V zCSOyIy6Ux-XrcUyD}mheCa;8$p%5E6G%kkgC+&0cL$6p}iign1r9Tv`q#(eMF7I26 zj5{&7T^H!;J>Hw`wWIa}dbn3TZ~&5jFR+}J{grflGjRy#2AHZ$&6+FfUbjj1^Sz9r z=t2gHvQm=(kIUF-eQ1P_D)M<*wLlg>Hvn2^pg5*ss0=c(vj7s1D@WNt{L}H`$R`>> zY&D{kGr@)Xh5K=W&my@Vw~^wdzb$*j>YPFNY?dIEv7UxJ=A~FS-@=BvW#3|yiJq(G zYznBSvXb}Cn0M3BzKPKJ=CAV#^?dB$CLBCPRqYmYP46Y{g&Y>-bDtLj88`KXv6Scb z)E&+dPKrCj-euZ-Vt8^J{S9rpvWbhhl~uU?HA zbB>9~n$1L)Fj$l>;8)j<&3BzW zS^;(8&+Po}m@(_JF334l0Gz`VG=kSL6FgCx4(sbLB^I66!`sNv4Qlr_B}!s#ED9-d zrv|o_0UNSbJIemYuhF;2T-Vm{KVG;}ZhUwS3ipCZl%AIjrps$!_20NLyvVBnO4?-- zqy1>HLwXJ+%HwxrJELHAtW%A^NyYX|$Cg>b4b7`1NKS47mDHElRZfY_xv5XL5m&lm z)4bPxK5cyH*W9Trd7%=U;}mc;?{Z#7KSLNH!%=W9LMD>rP;m%7r9rcWLiQfyUana{ zWowDDQ?))S=JlpTXgq&=vg=td8FLFqZh36P(FW1s*aQL&E%qd=ErZ~B*RSbCdnOS! zw~76WxtvQZhF)ttG-KB19)pRti)Spj7&*uc81a+b=NsR-F;XLZh<`2Um_hO*IM1yu zByhfiUch%~n@6SJFc?h1n*i576q3IUqn;S4W{yj2RK|ILzs4X4ur^|+bH#;a<5dV= z*1G$KHwF&5F<}!hV&p&_K^QH_S-R;jwo68r*@#F8Yv}mvo(7^+unq?A2cow5xQ*=h zu!np}Xo9tS>!9w{gv7|uz}rGS#49W|M~?X`tbDX_S7+6+UI|u0SXl>0vDcQ@-yK;7 zlE>ygZ&A?cNUJ4hyvYyk?+5inqz zuQs&410IZAU}sKRy$fkH*XGI|_onO0%yElD+!SvuRv!)?lj^(CRTGiodokX4J1rb~ zX)Wnn>^{Y}$=Z~A*ftxt`%U_J!*L{|zxqX#7CEj(ro8gplfGq>bFRNY$FaR zbdgq`;O1??teKJ6RPHd<9I8H>j8j_jL36=!o}8w|3xjUA1aN4Mkp1^09f8bPI|rNx ziGdhgo)j0`2#v59_&H@#zdD-KH;UJMGFNDtg>c(|p(5iKU2frilj6p@*`+nla66Vs zE0=ssA&)aQI{Cq)4%3)8O)%3*$8#xU*hp&9d{^&V*y+p;BrwO?XWFc>K1#_SCBhmya`H3;KQkjMJr^Ls{Z!7r&lu(yG^aYFoS)>b&c0EWH zOdG`N#Rk!qlK9D;n>P{0!ZIRA(M-gRH)WK_p8Z= zYK{(48XA`kzQVz*ZotOL_iJuHIW;(S8uc48lc@^Ru6-Jzauw+LFj5_MYhViO&h zyux3x?ikHr&KKjuA;xH7s4wl{$oW<`!OBjBZK!djKbIi_%W!kLZ=~U)8eeA~wjHwf zZ=o8m54$WS=#_Flx1reS4S+s|cjqyWmp1fv4SQy$OyZq0Dd-}^);n5Ie~3=}tp-4}2rqewC&?Rh0G*{Ole8OPul#7!Nz~i~ydT%IwUeVCgvuU@lyT@7`t@u1e7Xjmm zq%4lKrX``?ZAUjgyCEn5rAp=?eXB_XF9YTxRQeTw`r|$2q2#`#26FUPJ*7k!h0hkXcW;$@uGF0s&vm8F%ictwD4&a|FAqRB{+B(tM`!L;7~84>J|B=_tH{&k_do7@HM%?ZiOkHEi2N28$>Sx8k9 z$vvZvc!bI(hX|cYoxmn>G#`$E1|;yaE0vQW5On0Vvgx7V-grGR@%`B#*(1=*&J1XA zMC*9a+hM@1;m6Cl>0Y;8+}P1^Y@|434_w)V{Ce`P!KzIQz1anVSJYGk4bXz^OMbW1 zkhoA57Y{^?#d@rHJ*Dv|r$c6T>}RyO=9Y(=uaz$XgTasBMd+<~YpLU-#3j~{hqLwW z1Is7JWwO-`C32SHtkH_*hVu-0r3M?5isoywocY^fN9eUu!67mv=ehtnWT|DtCjnR5 zFxsPgEvO}eovXUm!&qBkXuH^B&1Gz`#hui*c$-f;fo`*HXR}}qOZY&w3ifV6HW&dZ z5bYqbkja7vss@*J)a^<&lQ9x0?yG9 z<9-xC5sMmM@7@b3W~-lB?bahRpVT+<2IIZ%vYyek41!kUFGWt=lv&B_%B7PXY;d57 z(>8~!Slh#N^^G;U_(Q|Skuu}vCClB2lP*j0ii|2>G^wG?t~+fvUYkg>niFTf=i~U2 z8aqjZAZqtJd8Urul_Ugxt9*M6lr1abXeVXD)^KEbU}Fyw3b7*k%|JG$wf?5=5;-}Ov zqD#y}Yd3>Y+LD$(ivMBihd`dH+6 z5Bwss1a^8pJY;b{-1#nIzf#SdMa1lXc(^e?e*IHNV>ZO+>4)#1{lM1z5Md7%Z#zie zINO-9i80Qd_gZUQ6}oEzaYXyQ5cRm7A{(Gh9Ic{sy|RM~cg2IVU$m?JB+)z)$1$Q2 z2r@GQyr%>&f|Q}|#uQ-x{AA46sxa?~{+;bRW&RMxIsS93GrBL!_bLTc)E7_@p08gs z9G?dE%0?sZKTsXnWQpy5dw_lucr+xGSSnUQJK47OiS`K09<_asSsijVAf4WMd2HYD z(m=*!lRpXYYR!bMhc`Y0`TZ>UclPN<=Rj|V#Wk9^Zgv96gLT$|d7Be#6=R$R&e3rl z6OH*oTjxIkI{W$G=YL7$tLz?|Ioi7B9v-B&$_{XekLL1{-aK6Awg_ zF2#9Z2LwwUN1rBiq>00z;RvHAf#}O^MZ1bvMYx^12bKQ?TKUSzjIJQ__#46jVE=%Q z_PG9{l5(>=%-l) zzXdhp8A=4b(21tlnGE2x~7K{S+MdKWLqAawVgr>)%vM=HU@3IiQ{T znc({UHK(pN+=RDWNVlTH%L=n!eVi?)6aGQ>{Nt`HT=9KV`P^U3-9LeJO$bfz{C^;K zZ`Vq8P4gL)8HQC!TGr)p$(&c3m#aQwqID+z4Ej?tUq$gM>OXkAy+d>X+5iTyx*&A4 zLf?h?w`;;?`xiR%Bfm4}ZRm=hZdXl|jJj9S&CQJ`XXzWGzqS#gP^1OP3%s5_JBl(4 z^2#Tgs>lo7vfhYpcF2C8NAD^{tEpqm?)GTCby|wNE9cdm47YzRT!x|H8g@xaV=kTZ zws7}Ts|!P#c#11q9Q~9s{wZJ)4Fx4OrxkiA?m$>&5HHhCzt$_OVeYe7Q|Y)9FN4NV zqTz{#TrNXv*_@zK846l6zb~#4m!`5D8EyKm@oQHB73To z{*9r&Y|o3u14!2wfuga0k$?hdh{86t{P#NupZ*maR`b_p{m&Yv>I#T8D-mEbSV z$G?Z=qZJNm{}P5KCHbDf6X*)KKbBOWyStq zT%t3+tK8LhyAu-D&i}Pt>#yykU*qEc;t>C1yE9y#4_1O19PIZ$Sjz9Xe<6dPIGdUN z>s_cZ3Pna|j@Vw?D;4g4O)(`xTvc$NZ}}Qw27i{idT-)IzbIbDD!3k?zA8aP@=7T# zF0S>xzY+LWF8z<_x14ZyR=#lBUugmf2mvGd$KNML^xwRhH4+spq<1x=T|2-^zrM=W ze}-irE`^+SR5t2q!@c}P6Q*FZnoW5BRL^@hg>0UMSg%E}!Y=<17<9q2a^<4KS(-pE z)2~m=Vd=+wzTJC&lDR5z{{bkP&eZI5tLd!=e)p~n5Q4oTWng;ouA^^(V%pKsv5e%_ zUpDOQootcdY;LZ_#^zDb@mF?1!C4#=Xcz}T`bzr2ja~_esOI<{$>E(k<^0^wc-$qj zQi{jIbJZB)FU_~_tD&%v=1SG(-b(%pavuu;`umpfeB1@$lWtYORZ4oL6#gZyyYco( z4WGX{{@x#SO}gY`QU^d!OxKm|?GWvHz_Y&=F7EYg6@i9c-fsS03EPG5P1&jiu%d8a&WP=GPg# z8J{)-LV=>G9Qk^jSq0UQYM-wCqo}kKlh)aH(S{Ff>@ZiS+G`f*0}Resuq(LRaH9gcAnN8`L>Jl zZBAm?ZLULdlngE7kYh-UL^?V;LEoPd$ORof+t^d*7(wU901%f_EO+ycBWC6Pu~^yi ziPi6MvfEJ&HmqgBF&6AK)EUNeJ^SiRoO(x0Jzv#tRZ{-6YgKQyQ zIZ8is>G)u(`r|joHK}sNwK=9PAsnLjIx-zV)%Vgm^jhL0zo3!%3HTeW0?HnZ-f_cL z56DQ2mJ8_QeBh>PbC}`d4`kx*_y%pcbiI9A00Z;uw4M^`vtwy9Byv)Qm>ug4?ZA)4 zO-!I9Ao~ngE^`$c?kMv~C!?Pzv(2%s{&-oQhwCH#sFHeG_=5zCoV}UzKH;qS@!j0D zRtn!@gL78L{oQ!F+eIC#u)j=)iQ6U>pW6_yZST(kBmZS(AkL3kBt!4=9&a|zXaXHA zJEDR2zusXa>@j$dKy$XJjJGV#aCBohm%WdEY6nuJJ+={5l=nHpq0me8Q?l|iX*X}O zGe=*g1nDT42x^Qa#Bb{wWtY{gaxFc#j=FELWUQz<1jevw}DB-J=upF;z8O)ACNo%4xh|mPh&AvR+p@X;o z5;e}lsnwDqUF0izv*xi9g$WX4X~6xBva?=v$IZoLYL}i0rbC|(3wKlaPm5@m4F<_= zdnp_=M}5P@&`XC&qT%^TbAp%@6jA*#=){37tlNy(+*~@FBO)d#^9dFK0yX7q6+`fA zzc>3BhaWI!VZvNS>9RH}GU}irOHn<(dOp~_M{bXwBIw;6DY;>E-6b z(ljoDSbBh}^Of7}bWp3Zp>jP+iE|b)aRt?M=8)B(P*eq0UjF>+)@ zl~az_ILh<_e8ZUP#HS^{MMI0n1w5%wKA3uUU&m#VlJrX)u}jt2Ec76Lr_BOYRU; zzNTAb81m@@KJbUm-9qNv^F@4>Ak#yAQVOiTPA^{mc6 z1($d7!?}DSgjX>Hx^au=tf_VE%9?(A7VPD1Yj-H19|$9lw;Ru6r@WiC zr?I>|KKJvhpMJx?Gpo41t&Bv@>Ed-u&nf~i?tQD0b8y4|cGX14m=#Lhli@@({i)dD z6A}xK;qZ^LZ>O0=Bn9CMw5Dqip$%<${L@Mn?pJxx!VCaz4D2fS;EV$s=eLNx81tIAoCob0P}(b43ZC* zxK&qfbF38%tnJqzzg0J4uye$zeaRY^w6yii?ChKc%0B@Ym`^&cNEhoJUW~LsL`o!2 zH_gMQG)A+V$01@qwXx{dLrk(aJs2MQ8$MeTIYs<|ew)jyp&0iRcv`#)t43|R;?ikr zMO|lyo6;Fh2l^weG|yizBDK#l?z>r50}maPL*XzNf^p{OGXQ$2no#wF+K#Dz+u>5P zanaqAuYp{_T>BT+}js zAgCzv`_|Yuhek$kivqC0W!~*ku3_=(-7Cty$Z@=C52ac&yZtaJwBK%)_B&1K1wGgy z=YMR|{r7qtHc+P(<*%~Xo!bb#+5t`*oQ#8mOeyYo+lnsTbTm)9G}`gD*w76sO!Y_; zuY(-h72^4LY4kkU9;u9xar>dp?fiI1Dk1r>%!cIFm(^hTjg06~EDcGvrMa&HD(}A{ zXQk(NpOu&dEp4CmsQZi*IZk4>-jtBYPF1RkX2DM9^H$G>O zu9$(oNSV@|bU(#E9Nu<+pnj8H?4wc>{vf>Kg**0a^3wk*GNn)6@47f7>-y=oVch){ zCs8r2-_34!&PdneR;??JY5O6=h!@i*ZFaAjV)(h>Yg24R!fZQyxp0+YSW6-&rrM)G z8wR_sK`;o!EhDP~)+fZY-vS#2_1`|drVuqde!5EG)k#R@betpGt@`{&WuH_WAyP=S z>+BSFqWa+HXvZD>-7xb=KdtGrOh}m_qV}@0-C2apgY_E14Wmf$<7FPRtU<9HMrGEA z4}s*FnV^X1GsG0**Cm=5?B5V=7y4>_d*VBCciO9deuoFrM5p z^6+=PI$ttXmRx|3GD0SNrJw@2NJFl?d#hj57m0?@q0i_`nO<-m6OTdF9h3YdJABI2 z-gnAmbKMEsmC7mvb5UN%?#aEmg>+pI#r(tdE4OT8rayz9G($?)kt56*)PbYFhK zRpPR1^%Pp_4rr}5zl!5+oC%-yJVnaO8ecgwwq3C`PmSZ>rO;k&w2jCtkxC}_YAXd2 z+y>ApTan_)6H0S{a(NpI71VR-s}ym@ph*Xb+vVnJA>@({?@cfWwjp;%^G{%+s=5&= z_hI2D4W`K=d!o9|_7){Gy==U5+0&R@9_yBi;~D*;o9_D;w>fVzDPDyI^{w;$F`?wH zs^*ArN9327Tdm5<%E^+WV^2@$a{b8BvBRXSAxVFb!=3d}^XeOiZ2#*wZ&0fxqBM6_ zGL#NQ$fWF$c$N^qXuSt7)r}dX`d`PT=V5VWF+K;G##7ud`KpR!=KEeyTD4f88?h*V z;xvQ)Xx?e3Y;aK*I?9ONk$B>r>Q8&mF<<%5VQ5~yr=D8_%_nk_ zngunQTyNJ}Y94D9^QwQDUx5jxYKDRhJ71dJ_v;KbsUqzyvST-Pw{MT-ccY{mA}CfD zURo0=enu8=9ZBd8#;(}al_RsjfpE-h@6<|y7=|=@A?LykH%a>C`toJJy;oaHViS9- zPiznL4m~sWOvLb=|i{|tx1S3m2*^eiZZIsEb3`6Rv z#5A1=KOD+0`HI&2>)S2XN^(HA)pqMI9Zwtl z=?x*ZyswrXB)9ty1RPHmz9|UR7y-hI*v#RuW7)v{5l5`dyk)x_l=(p4$;m9YPFN-qf{hB`n6fc-i>Zf3Vn%e74o{Bnz~r2QMpF{m;Pil)Eg}O z0iu3qWz%5fl2HeEE+eca2EqQU+z(Y7qtVaRMx>zK!as_DCekUI45nFXbl6(VO8T&> zUv{V)mN#pEA_*#W({kNo=RLRwh0g)RdffCxY@t!Pw^Tic*WKw@=Wcj&k&l*AF9z-o znNPZs4ey19g(Ml$dM+sJ?7sOiH*Bx+An$J5S#z8X(^&@FMc=b_%(i|t<|M3`OeX^u zHX5Tp@+7By)3DNm7`AkWl(5O@*i3YFuPe zH>pJiQHC~Y?4R3Ps)xaC=zIgJ8&g`z9+;u52e0`4kk1i#oZ`bdFD*Ee$$Ot7MI|ZSl=&N)IOV4Tgyw$*vfMZq`!H|gaW$i|1X{RSf z_f_0r+SPzF!`ojzG=i8jKSfFNyj*vb5BmBXT@cgs#?ThNEAWd45d^Ye-HmY>@O>-i zWU7(opjYMzQ0%R8Z9+jzxPHEkPs_MzG>xd^biSVRYB;klSVchH1SgkP@V&<<4x4{E zW%uw#X8s5!?7u)kIOPptOE4M#)Ms&n?bIXHBX}ceEL^f$ke*Q6g>e-0U3x~}k|G;` z+2iGy9sKr3`(RTR=`!SyZdnxb=bvzd-$P0HTS>wzp?P0)$1I&cC`=w;3$OTQK=SyP zOnf|BJV4bXU*L%j&X=LcjPq;kGQT^~hE<2II)5-!)T%w5d8OBLF6Lg^OM1rW_Bgl* z0&lKDI#gb|_GlKDVrXaPF`$w!#%{)yK{+so@x^*Clm0lW!5nyA%wQh^MPytIo7JL| zsCPBlQ<=DeQ~SS%rgbRRYLZj(V<`hSL1-Sy#-OPYsdd+uh+@Qolf5au)EwsF8Erh` zv;u(Fm{u$!QaV!MET+cv`NJWSas>ZXb1`=>1cq7NtTA}#*ElNRcJbDlLdyS?2RN&p zWG|M3@Ef%SKa0+Yyt2S`iK-tv_58w*blehH-RO1Xo+ssjSk2&8m)v1ru<~BFPjfYc znJ=kL5?CfdOQ%J@p1RO(-ZaoN&5KX{l(XuaVB)&}W+-R9u={y%ZNkeDAjY4ZFEott zGIwG9P+>cE>IxexBP=3@obXW#@@HQ_*}0^72tdF&KXN}bpmI0OVCNIoIiIfKazhR7 znH!N1aW66(a=nP+-|dfxe@OblyV<8|OUA)-p*#}yCgLb3R7~C?FHB`=J*d4@Q|K_? z9@@Q6_G$X9m|6R(SFVeEk>8G?VZ)a7{(Nl;Hl>i-Uqo@gCmfKzh_;)@T;%&Qo7hzrx{MwZKV(;U3dzF%WF)`jfQ$u??lwn2a=H!y!h6hWGB$re+K=yyutRl|g!~ zAqj>wXZOHg0^T=ooyJO?O%zMahJMS-CWwl>=)YNgW6K>naHCWU(ZI(O&0j!c`>$xGAXb?2Yk z6SX$I*!gk>Pc~DKJ0y2s+!=5GF+S!SgZ%U(9eYsa?i0IsH~Ptaa|?f!$Ax3jqJRnd z+b}idtTlqg;jzR%z$I16M@TL7eD!A=mdstNbsnwZRbOonn{Mm!J?9e9r$&HlST4fd zZJ!vD%XM(Q+fW<~;i}Oi{q60RClaDmRPN%5>?0L540`=L*k@)omzz@>Dvt2G;?gy3 zpKm6m4_|MeKW{BGfUhjC{7N}7iB zV-ww-DtD*+ao4>wM#PYvd669FS0Hs=ZReGb-@(_`GBT?Bs!zeFin-4IV+ILRGb8ab zyNaO$MQx7VyxX`1yd=zfmwhDd9Jhxs>Cq0F;E8HeuE*^JyJE?CJJG)DKDVW`el%HL z1GXq~W+TZFu_;l>mOPgl5JZUA+LlC;mNcMwb(~!j}8ZSz*WZ zGvEKm)K^Bu*(}{+L4reqyGw9)cXxLS?iOHhcXtaOEVx^6C%C%?2<|X*hxeTKeD~k1 zSu;=9(^9>wde?5Cg#GYg93A_WP&zH^oEjzWltFM(m|2$()MAR33>5zqy$OmCOc@{V zGtabFZUcE@YzVQ4wzy-)i`EE67290B>~0}#F5m+ja|iX)y=czfu*+mtCm)V|w?O~5 zafSzei2P79OUv^E3E!P?3z8s^s3mVR8Bw4epNR~7!l)}&z%5hCn@@ALOXjrIa@tW` zXB$8f6uKuBFMKFn6vEr-4dzQjJ0)_N_NP)wGAVb5YDWuk93zLonYNLB_J61ryl; z@dmai)zq}^@vg@dnY%a;-ZvAmr5`z%4R>x1{Zb!Bm+|_>#p1(aX%8@1N;zI!t;eKj z7@wMFvEMs(Swm8n92N=Bbe3SH2kvDyn&9+zgcmEd+@}BD=SQXxydzdRQ+h$e815&9 zeT3P8D&Q}ArgVX|hu*+T18uB&LUYHDo}FJKsP11o33x(W52TcG9pqm_Ddu?@Oh0(A z&d_a7>@ILa&=PDu?3Eco7!+f{z3$_=f&xPN!njvZ?k)J@zU>MZpcgrR+~zne-I+4jOkDuAcrt5N3-p@qwRqDO0OOCl7=(>CH zbR51`imMA@&y=BeXxD+@GdR)QU;06@u6n)h>YrdeS2}jLWaLPM<9X=K2}Rti(_lG@ z%%#uN^Ek*9L%W4xE%)&V10pr%V@BCA(#Y2NXKid=d{sGz*s62N^utPcH+CrYlfTc? z)RxGA^~rRZ>2krtPtA?7e3bTcioQp#5iIa*jTxgwG5uivP4HoH7c9|hH9En*3T(9T z{S^}04L5E?wL@x7+hVwG0~WvaB7eB+=UQ^)r2roMxL!P$FC!s7Ej*R`#61tuf0?v; z@j+49-R3C6g>xph&@#iBUi?RUBm^~}d=x|O*Ao3%l#6~OAxu4}vT@3#^)LKhR%_23 z+y}_xBqUt%kUF_gxG%j=32@6!_?gI6Lb+?rJE5YfIBc;^MGpK9BtZ(w4>|}+kLzcm zY&VCCe0iwGt1Tk@;bMlj0f9Kl!A=?Sg?*e{YjsGhX26oWrHD43R#;ZWEFM&!ogPwh z;k02jwTV^F^#T#?9%5aqF78<5_&=1tr}dp82Zs1^Xxb#=URL|bUkW?t?1uysnayD~ z)tKR>G`Y|kHH7e5s6C_$h)&b|?4GwMoy2qCR_Kh|47>Nc+H+B=>3)ogJLm$>8q=uc zTwRkXvJ(?c7PE)qiNPhvw4;Q-*NQ28zi}CYYs|wQ+tRko;hGT)1RwrxL|X)u?K3fT z(@YD2I*Ut7n-Opo3!wfA7KB<1#As;AmfCjlSuM`Q zHjZ9q;R)5rG`l~u{Aw@ik;K`8eH}`dbCUE)iMaBV%7n0gHa>v5&e;bSdN>_36L%T( zhOqc2TxPcXiaD9s{gE_7%#A<(fCSsVVy>Z-v0j+zzy3 z^l5v?qHXwNZ7gnus;OB`G6orjPESXIaG&Zgt$(Ow5i+9{y5rK%y!!A#mCKOKaB0D4 zBftt3ue8?(rrQBF$G8r4}a+}^Rv!A z+HfzGm28>Fvr^yH390P#ng)#qH9St&Y1Bi-h4E)lV?|9>tn0fT22&r6rlpBDYNIfmfv6zZWmW;9geVBOQzLQx7 z;E($$xJemFSL*hDn@BMJb?@m~Z?UFV=4%7Xqtb8yFoYCg3TqFRYOG+Ve-L$=2G!F! zfiujCzaJFliXZPvi6Hp;$Yv=dpfF4^zhg#?6Wu$Dy4hFtt%oOdlC6YScXmIva8U&o z)u|#uG-E()``rJW@L@#W%a+OeMS;8LAj`Rb?AN^I1`fr<(&i?p$Y@L?1K_#G;D)a? zv-5k&4o-fvTjDTSKUjFZQcx${{k-35AYpIrx7drt+}P(!Kt65D)`R^SW`4ZDW#%j~ zR;<^VN?EG$pj7n4Km;7xh6kvHlKv*6n2^_a-)(*hh${E-Q)0Z6mpq32ahs8H* z6W*qVa(f(ZgAa4Fzz~vrZyy)*B&if1mwm#dZF;sv4l=zE2h-LLdb{U3h|o0w%~RRQ z55R8NvDV>e^r-xWWo^FI)tEBvGsbphuK~2D|BKWR?uGUkq^mYYy0+KHoiQaWv$X?5 z^6kSW`h#}9zo{zAH^Ay)fLTNuN?QMLUzNkBcGa_*az4kt#?PNx*|_Ws4%*BD!Y@CK zf8bV*(4@%g`)NA*GSr8jMW2!mwNht-JY|)EdS(-oqy~)`9B(xKNh10_Q*a~UEz`!e z^oyQHer|^C-`D*w`Ghq*fms-HDZM@|B@GrBOmC3K=Spe-CW1X2e0*D?KdNque@~Bq zz)kR`#MrsKPCr0W_w7!1rQV_;q=;PKeIXSi6f^0VtJn4W!}%Kh&EaJ7%&iv!x%RM55I9JZ#=zt$JW1|28&9@U}ZA zG$`z!&l3VZ$mC|4D&nOwIR;HW+G}biD zK;+~Ny+yg5xMa&|&dD1OyW-uOt)6aUcm}{FS|_z27Ioj}9rKQOn3AmBdS@rp(@b_7 zf=#iWCA8C%YSUPXA3eQ(+qL{J#rp2MVeG#X=YK@5!NV6f0{U)(%sT^udH{&uQ5KDgZ z2cJAP)EF5nGKN&(`<(lO<}P*##r}+L zFS(^sud443c1`$!NnT90;N0cMPH6M1Z2#b*9b0$zyI(GfKVGlWh0>UWZv6<_*$7c$ z_ekR7>n=NAUyShN`X#iC&U*LQqrJ7y&aU!Y~c@ z&v*{2Uhn7yMf5J6Xog{&52%SVFh9%qNTwi`80NK~MZ(hEn+!=uyv z&~<@}3f!3);?Z+-Fizd_y;0zC*^ijWU=a&LCZv1Y+m$E6=W_VOi0ri3W53496M9p9 zG?i;R@fylJ+)zcx=k#h~S*q_k$#H$Z=C~(9pGsfb`U{BytTe$EJU(BcHi;$D&Apo> z$CT>1>8@yK?}IY-hWZB2q|Vcox2!9( za)jwr^R(GDT0Qywe9nXuGm%@_QqZYmK#_p-^C0Drx{~N-E-!O>-+(*90dF#EU|e;+z%4{f~R)}G|*>L(Fzj0z_P9RtclbTGSBlt(M4v-3tu zup7U`boWH)Xc;)r$(Sck_mupQb^465Hg}9dh)k`Ni3~2x@o_Lun08J4%JK07;doyORpP1NPd5 z71_FhSk=UgcsmksJ9AV#dDXE)5&LvGLu@+jVT#)vUl+Svx{j`bl6ejj@3@=YbV=9t zwi*5tAM%3zk+g-MvnQ6Orz4@=v#4X--KOBC8g~A|x5%%1ZXh=!K7 zom!&Z+-%(FNFuqO`0Ye`tkawV7cSIRn`?UK{X>r&#uPbK-_hYAu0?RLsUW|oYh7GY zct49|ES^OQMAqxgwW8>z+8tTk@Hji!PpUyRU>qS|7~uya_HMX|c#8uvzl%+D*00J? z@ICPq$O^1avpGs2=(Wc?+&|b7+rGjO4cndM*7c8TO0u7*KHT_UawPav_{N=kQr{E9 zLgj!iYwrU0RWvkI)B8?3xx<{cj{-Sg$NR9Dt?FH5jHFeEMd{fuAf*1_R2 zu8wxPBM!)o7_3GywP!ttLY!hIg~IVwY15SkF6M-Lq)c^vDc$+xW(PiwRhx;vV}_|t zBtVVq%P(w0_ux9JFK@kX_rG9ermm&PIlk8&ZR?~th3&&c>i*94+@xjdy2Y^Uz%+lp zwI)Kyah54p;=d|h_uTTcw6wbq$}1!|TC*TokuU0Cb_OR(=G`n4V1Q(m; z1AK}aXl)gik;Z{>^=tHfy&9(B0;dMa$$n8HGHL$|#H&O`ITE#TKPGyN*Mz_4wCT;M zuIu(t*ed<4B(dEo0b#~DOpsUbFrD440|yN^9v1B4vp2`S!_L@Ma)p01v#3SDcJ5WB z6HmZ}OW{H|@vz*}?U>HP*}_BR+k#{YI|jC-JUk!D6?*<+2G7hW9HlB z@@poM`{a@?@>e9F%oul-3CZ9t1;5hV4DqmT&UFz(6k>5Y^>!Tcil0)v|wR zWOS58d|Kzq?_tAM{si5K2PWcq5F2D?xjqrTw(FH3$-Nmg*pPa{pPs^w6aIV~ zwt7W!=b9&1w-l7d?4*F9@HbaYk z?3Hwp!rUlI6Bq02{{k)~N4FZ=vP=84#P9}l)KbgN_NHlGA+rhz$><3_U)*rHcpzen z3$$zL0qra~Jk?Z3JF+*Fxr#`^nK4543EoVwiN-&!Idl;E9Q&hSzR_?yavK&&2W@nC z*783eE5+*Mk~kB-{;8~G-DQnGgc*wNf97AB2R**tEdRkUwOqqC>YU-8`dyV5-@+vD zcI!BSl&|-CyzROZiv6q2A+buk?pMa^iNEDS++kFA`=S?+iG0RaqmASp&LZN!ds76j&}P9ef| z-F&Cyt54E~<2+p`@br3YY0+sVw_VH~``$o2t-js4+9?IuO>d8zby2XyNY6f5`Ej%s zMiHfD3y}S^*5fb4zp90!kjm6u$VL>ILg6zl@DLzPe^4g$RzW&*TH7;~VCyt7)6DyE`DG*(`%fM`df* zVchb$z}x?k3!$MLM!E%jnemFl+38w#Lwq8aA;MZR~`rQ+i%;Ep@n*R_Xofg3y%9n;IHMal$?mf4x7y@Q$ zE_wS=+Yo-1`1b){p<(cNj+>@!QCEdkCq=BztF!T= zqJ0fN3X>9`rsP4X;6+87r+X()V!o3)8H_^qVjB(!r6!vDM-J2x-hRNlalz0zu@*bx z6IYCS@)8JS&2ALTKED&d*wDU$DK^$awbeh;&T^75z>0`#t_Rtq`I0IyQ+zOw--~lF z+f)O|zpwEg@2J_wmLHbEyCJ{ZqL&*xtuv5s_!Q26dshpA;CMoaP6zagR5;>}X|ZLG zll2CW21vnDi)hLzI^^x|V~@J}12leInNt3TPMpI49xk9=LzP^7BE9#P$kWCR)9BL0 zQ&O>2E;Cd0mCfGLQ&VDNRxFv*6nc4eHM$VZfT5t_^{!m4-1TPgVd~hCKJ(}I-f!oJ zI+wZ+2tyGW9ep_fIs$)=LzjG3hT+A4?7masURe4MtpuNBrpDZHzdzt0mP7hKHvDIR z9MHbHm7^T)9Z6y^X#6`EU~cG#cZzLkzlVq}o5sJBVha7;Y66+>ga0F|g1NsW+hJ!5 zF387M^UaXR>z4K;pIJLdH0al$T-}O=ww;em+A*t5yfyMjT(NV7UQuJ|`wrE83}~~&*rnT>r`#5ga4iH=8zn2aJ#GtN z?%)$KhvzIFN|99!R7j~==Sm#qV{Vn?Q(8m3uI?VhZ1($H64S<@f*IBlspY_vwvp+b z)a(JgXV7#7YnOsq<8Ns2$ngoag0Gmk8QkJgh={_!a8(Uetv^SOdlcDx`%_``W35E@oS#yRlyo*Fee zK0Y2y(y+|ryugs*`Ke{-bv?XGj57}~{yG%}@$Ok{hL!;D;=2Y|kn%|~r9w~Vam2@42TR2y(V@9<5S)mL`iz2)meb{Gx^QKCooVUmQ$@t2#^UpUMX!%QCRyzZ z=}01GaFL%3lnJ)9gbwb+)sD*wa*K?jX8SD?BA4y`X2xB3NQi8Hxi$hkk~l8;zPKcB z#1`U1<;0l)j2tY-BN3Yu$>KdU1uL)SwX4!dCw*heqd3KgE6>g~@Dyoq*yGA0mECLDKj`KIdt&Zup4U^HkH z2oB8UHUa0;7!Y&aJw54PjIO&oU)PMJEx*zEB77SofACW=1a3^1{4D6DfoQ)@IqK%abvAG6oVK()P-h8a8 zZFldK{J`SiCx-dh&RYj1d7?1b)O|ljKZt{#xr4L%-mjk0Q#shcMFK+^m}X8((#U#89`shT->5=F;AljK}*OY*GkR*MlI>(}iIBs*S|f7NkX;gI~j z{BxJ>YEI`ZGfVBF1^wMW78!-~yVt^JtncnU%W+S9-Sm}3FNYWqmy*#3X;;~+@Gbsl z-C`P5cU_Ct$2nybxJ#|b^#4Iz31?pDN)Up~reJH!8;KMAATLEnn>RD_Y<&V%- z6}c+-ZPnOkxCbwz6PP6ix96sFv6yFi`$|tM;|5j({LFS^`t!giDv+N8fBewpKAlY2 z^edtSE2<)Roj1rk8quH*Qr0^!sA{9*;cD4)K3Bju(!SfQ;3_0$NpVZmC$K+c%amM3 z1nSoJTamzsflEP^^Oh*_Jw5dtp9b7AwCyH+@M?GS8N=P4UyJJXjuhq?Ligt$xjRPx z`X}FGKcEV@Pwu)r$?&kwv;}zOVs*RqdcEhM@}1fi(z|VDnclja@jUPZc_D6Iwe-g1 zGnFYGW)z04ogTYcJ#0Mns}qOV3X&W}UAt6Wl}T?Me{Flps~r#3GIEs8!SWH}hR)+n zWRTE~ul-Ob^f(f4K6ds&()sb#{{oLP1n!w&GyhAOiy{fgl@}&I{q4ZFx)Hx=b_IT+ z`{%qdqI`S-NhUuaMxH=Ow_b67WS3G7x$fq8yse>3#!H(fq&(HXr)vaW43n_QtVlpC z`XThFnn|f5h*eof$D-4|LPIUlnBd!?i=+MLbZDnoymq1c?r(d%Vw5Q#8$tqFb3Q+s z2*j>7|GHcAoVcM+q5E{B`WUtuikzlV$Rn0t+(ls(0xl zL`S52(H{>s+Z>C>mg}oX=Ke(eJHdyv+fk=QUv}HC$8}D3Z!jNxY#&2UfQZ%gL^p)% zZ}*#zX7KV7v?S-BP>gOC^DDt)zkua8_CY>toOju^%YJ{$7PBUF91m^y{oc=96?fYg z6z=r5i(J?qK8_JO{T5;HZtO`i`I`VXiD9^ZL|Ijru4MV}mk{sIlpQq+^)cZ~q|YRW zh(}XNkk0FWcEQ8+c`OdBUv2wfNb|u2EPN~-`FBa{y09?ZtWthDuP-SniE5aw!F>N` zAqEx^-3O;%V720=eQ<{2gYjw#0_zuA^rsKf4vKkpE6s8!1d4%J#Ky3YYCnNxU@xBg z36d$nyUq9VqDLgNmIG$Y-Gg_F0dVtq&u&;oLO~(5O^Gu9@HuH$w5q6wNl;tqC}Q~K zHp*%>7k}wjp+kw-lazj#gyZXiuM0l$(xs=JHs#tPBiE9bIimu^>9GzfB1GBCx6YRAuZN#J&$3#EAmaKYt*ynt*8OV1`jY6{^h8~G<<+8`2C2gX@ z643=k<=3GZ?^l^I>3_0#x)1DdOfCob$)l)$T6ElG?D76eF}v~BrA5Bw}&@16<#KQbcQ_Oxn8!irRboz#TA00>WNYlzUh>9kyMiT z#D+Xfx%5JPqbRMS(p^ma&TI5D{Q~K>4yHZ3eR_{w_9h-{QVF0%iCFA@vgBS7Ese_e z_G4mWGYI)U3L*?Nr#0a>j|}J6rge0C1`&s*?6DdfutdbgtN|~qh$BrP=%78 z8p`_nm~@}-VpD{=UX?Og+`q>Va>&pi4EK9_x;EIEmfa z=)B{q3<{mF;B!tO1?S!xXjW-YdJMmzq(Kose#l8OJ!?C% zGYTe~NrZR%K{Sl(=Rh=Z>h73=$N#keyuo-oU@YFr+8UO9p2Mv(A~71n>P> zX#^F)6_;}`Xu)EswOK;^dJp%>9`z8NstG zr<{CKn0t0IlV`Fjda}njtj!2%xVuyB$*H-5ZM5*CrIFo+<6?x5*}0)kO$DKX5|{M} z(&IX!e{x&;=9iwAz44(;M?L}|D{Fm>MxXdB=zrk@W6V1q7_KtWnb5|I%m_4};PA{^ znXIHab+~+v(!k&LGa@lrj(W&80>Yzs>m;#rZx*k0)9W;!Et7NU`-DE(KMZkdB&QTv zCnaT3T|w{e%eCl}V>4wiuw3px3>&V`Cb<;uqT}o@x15fsgZ=@g|3qO_@8L3`7J8C% z=!(Zt!2#Zx%mHz5W^A^jmp}W&?E&}5EIakDw=Pt{RH3&6;$776 z^a6-*F=BELctUq3Z#45&4g%EB}2&md5CYP57GmQMCDy>G7vLUtLiSi42W&hzoEo=lQ?9E*b5f zGTG7*ra>FBTPu55sHC3pIkgJr0seO*1-LJgaGwXDgF3?_(_}8SdDtF}9qtPFfq zz^t@fFbN?YnE{mQ>;_9atl0-SSkRF-6E-aV#~vX&vpndBznd3%7dbZq4|x~o50pha z#uo)|)vj4Ruz{^n!t4WrHH-^Y<0Qst%<$q8+bn;KXE>FD2n@`q6c*ddm1|;B7&P=T zcqnIoWZ81rNk-W*$?WVfejZOx^!0X;VYYEP+wfF=SAEr>OG~DxsriGNKyQWeWOjfz zn%S@R@$(2Z3JzHL8u9vQkprRyHeHCQPd76e{FDW8u}=g(HjZGCO~%6i%0tYPw$Jk|o_sd`EXp z9UZB=yMY+n7fiXlz9gHTz5`4o%DO#E%B+Q(y$R@hy$;M#qZz_mkDgYrzGm<^rl6q$ zc4%A}B&fQ`-e{Bls?b8@A5X5?}d&|z4kgMA(Rg^W=m2QeD(nhl${&#;kRxXb?E3|8h z(;B+rWZsqzDk(KSqRJ!EFKLHD|CxGs%OPEMgwmA)qwBqJpyW%YixYx4(nKf=LC zH;o+>i){Q+?bJ2_>n$Y;X;y0)K}?xsiI6aoqs4M{%lyDh?_qA>3=R<4#)U$#!-!~U zi4XbJSED*ukMS{^kMQ)dT(hmy0|~yhwxq7*e3?N(*N?1-T3)TJvzZ}V_OJfR-CYQ6 zL&@6_m~J1pVKLjd!s-JwjTAoXW>SBkxc$;{p0)e|W;#{ zd;Ri0Ufua-<$8iXSU5+7B+A^8d!GAdBO??C+!vsrfY{h-{T^pAMx!=52}JCKt0LHO zqcu<(lJ%T)HkLxlBdE(N{}W?|(zD-!M(I=*Q-{YzXtBQ-ohA)~Rl1K1?D9=EdpT{3 z!xof?=Lbkf8bp9jspeyZRD5Ax3p&0Yens>QxOfD}EI(z?O2g%_i8XDPNr}!5&yLVt?g}~Vd#bj3sd$G>+h0EANc4=3{1%cO z#gH(+Z0TAk1>txAldY!4ee4BOS^D9m0ba-0^hSF9cVBBk_CR*-0}JI{AWls0KCY|z zdw5nU~$_tSyqo8aZ+4ztTvcjPIy>)>cb z%@~w{5c22k*Pxv8pcK5=#bXO^y1-hhVtIMfr0*hX-f-?0Q@XVRemTMP(B;R!Wk z+Z=L%mo5@iZ8St?=Fdx~HqU9L`)8t%{kK>6*Bp;KpYHDGh*w(7vrFfz*@Kbr)NLV8 z2*+}UR)o7Zg|@d{slzT25=C;fL6K<;CcLsGJ1i@(55j>ljOvXJ0hSO>!odcD(4c-x z-`AOY{z55fK%P^c&66C;F=1Bk8$7Y>YA~KGeP2wlkl;y8!bpY|x;%_EYWh{Kt$whC zj7W#gA|;^rr5-<$QL}0#9 z3~YdaH<^Z>(4L+xaa{lO2~Tvgtuv=?=d=J22xIYHtFgR7_pHZmu^jf+&5Tw#3aNQ;@!*}rL<%YKV z-r!u8j`FY;E-x<=1}^_vG&2ZMENyo>j3)o`SyIf5|31=su)qK19)Xb3tQ6@Fr2NHt z%E;J35DWpuwcGJkpBHLS&`--X9H>QkvwsE_JW_#H@Zw@;`F@YhMl5HPo{UPgMx_~w z5>0+fhFhZ=pNdH^nH~0pN3;1lxG5XU$1>0VymW6A8wakh3NbS)1GPJFacd849w(Lr zcxBsE2lt1n`65CU;DQ{Ps1SUN3vOr*3uuuOUzN=3_MO5 z7>({^>J0O{yr#tDa?8a9?i|lviFx-p6XNm!+9&clKAo`NjJ9<3K(KhGmy9n-uN}tT z5%{28X~0+`cP93dph3eA z(AAa6t_yg(@F)E#0d|`T{E&I7RG0t!{PTA*2jxLLQkv?52Qk9(`nv0^|l7@;DE5Ouv6O~^TcFa9Il3Zz34SG@O%h6v{0 zCk9qyhPmz!*ZV1Fc{)xG8Qn(BHo*YE<)yQq|M?KZ)AcE7Jq}D>v&YoQ*;#RZX#bl3 zLA%fanEf;UINETheP5NCavUyzJFQEdJ%KQ7l?|K|)p=c`^P7Yr$ajDHD8pYvQ`i&F zK_mds_4vy}0{}qI@Gi>+`nC-Wt~^Cv1muCWtN-I6z~e#=Kk&m`DF{H!`4{eIrB6YM z`P7+?`NJl(Y@-zi?QBu3$;v#DEdvXGb0SABF z&1Yp9rWi>kj)x^9bD2in%KM4qSdukGA0fU5kCL*u*c?sfify$U3se_4_nM411>ib` zPD8xG1_cK@uP^^)=*_eO`?&Xkh(Lh(PI^zF!ZMff{rWofbT?FrYX98fcVjok8g#;l zPR?#}^W@-$K&ED%ZjW^(;~0vn3#asE*|dIwexALz&Xvd@ zlyxrpRv-;xEbl&?@fDYbCQ!Lke7J@H2UmM-Sqn%-YoUU-m<6c4;SBiw&X;Mo`23Qv zfw&dAb&Z1Dhpbf~5_!btM0P}ehphh@Z3-1gqj52&?ihv9BFK^j6Bg;7lZz<>=9^F| z{76Yc@IUDSoDbPwi5~obkRB+QGKN1}KxcB7MUD0BaXBy5?*ORP81|4xVo;^T7-OuH zNX%h^uqdq4x{5<4ah`52HljG zJq-9n)+`9dtVs-VRHG;exmV9ibwoZPH;E?PUY9f&XZ%5B{wNBi{bcwp4q#=bFGi`y zxOqoWIB#<4wd3yMsOxR6!yy}k15B$E=nPQodV@z zUR|r+CX9lGeZhD{<6;-4PHWql&6LQOx1Unnlwfs&c()Jvg`JbouY8QUj&cVT8F4|O zdbS-fG~wR3)GrthmkCd`Tlh&S8M6OLdn0NRVe@g=vv$tAoJISmS0?X)it3S<+eh<7 zfyEX}R-&o#V+f?$$sZisES1jrH!h5)?#xe$I2fQ~XZ>(%Fe(xy>Bf9v=Kh=1R;$+$ z#yh5}b{u+~aZtI~N8yQN(j2Gs;9%zmXK4st6C{_cro^#$AY!C-N7oXPBu!Lr2YuOV z74B>4;f%Y#Z82`oI_7ZvPWL3Mi5y4fcxQQN1{gpE|MQS%A?cdUR#J}yh!J@0NN%Q7 znEbpO*%|UlLR{F>tRlW4=B|-FiN9us_+U`v2pb5`yl#;oZ|elAEr=Zdg3V|k5~7qa zU#_b1L#u*h3k4|BY+5^fW>~C_CigJQ1zm|tnMVr7P{E~x0H0KePT73_)+DEOl5Y?A zrR#zv7yg~-ueGj&AHIMfp>GmJNCZjHLYx+B@;wvuc^c&6ySGM%7}hk%wU~_zGr3%Y ze6LmLby`f4x%r#-Zwi{HzMnU!#S%7o%uiLU5?k_0o`?MUodyNh^v%xvaU~weGf{bU ztP+w1o#_*7w$e{Lf|AtW=qG~ zd@hlj2F$i-Y0(-M7d-X}QG4d;Gh^$g5f3S(x#M$@KWT}a{61i#RcU0#iIA$dKXN#^ zUx47g)`bV*cX_d8h-`D9fsve9`=1Tls=G-~4sqb5nT;en`~V45N-^zkHj@Q@@xRJ) z!)ZuUt?I=OFMwj%{m6vlALezxq}NlCce-0FrUSQ1>?Lu{fu& zTdx?_la#?W*s0o-E$6eAbbO*sKs9NKq%d1$Pa*_DzNm!QZU&Xu(BTNU1!ST8S*Rc_ zL6_v;JCexCFVZP$cY+lpSM$$;JBMJ02SZO&gg}}B!L&t_QjY%s{8$K$Yodr&lMdgz z9~k#!p@<{b5)FBAgKjHbFUU|k&52-PCAp62b~#RmPDMGU$28b zW8i1*h&ka}eyAsUHwhJN67Y%G>x^My!tNh52}Fj(N4q?A`dPweS10=6;84Ml$NMnH zccwdfbvu)}QoaIXB<=)8a=S1GGEva|@QaPMm(6vhoK}rB3aU@YQLaRqoDFQf`2@^M z$HGcFI*)WFvQH=HjIZR$dbca~iePOd`IuhA0H>>6u|l%!Vhu=bwp5XlVbhl4e6@LI zBF_kwAWAJ3?`o@C?&fJtf<%@-V@GI48?TJcVeeAf~wZxazEf+`dIc4j4;ZfBC+7}~5vekU-8FcZ9)33DJ4jWnM`f7hx|D^1BiVo06Uz>_X> z@ayFdk&Uk*TnLP}Wu7EK(}NdJJX>JkXx214{O_O!LQwuqE}I%ta>bHMk0TzpQ{jSoboAgmUQ>JPDC69-%Aj)bpy5XqHy#KJL*erZ;BqdWmNK`bDvLTH?oc0VH zIV6UR%Mm_2-#&^G5E7JFHp#l6V(=pf-hADk$kQl|!7r7UKn{8M5FE*UaX{JhLnG~5 zQ{`wCPjrHD^qT}D=^N*bUxp=GgaXWWG@ocOSv$tj3H$#_oGJ)c^r1#tk(|V{`8R# z+gglOupP>L^hD*}(wIlet4y3-f?t!~Vgp}SQ{Fv$=U5czx$!t7=qp_pbFPtnQ zK=DAT))$i_#!QtQlKGEiyNa+~%i?`tyCRO@!I=D8L62@(5a$D!NBtw{nyT|}MHqFV zLWJAu>$vteDf43rqT79|NLwrQ4s%XLfMTjZ(}%_jxS}LWyyY$H_#^LIMCo+K*jJc% zKO@k~Z24gLk14B22VRgfBh{k`OO-h@vhqglV6FVvrdM~z?&3k0By3k}lT!oUKPd3h z5nkCq(R4h#N65yypsls_*oeri!Loy*(8|n&u5=O*;*TO-YIa_hAGm$vD-^P%GfWEC z_a%Et`4poYW2{WXHvbVn{C1kePgN5})d(mE6Z|yUh-v8l@2T3)=6XUWV zFDR)Hpdp7=hHVdy=^@^sly26)xG?Y-?dRfe<`6+%w~4yesDR%bgA0yB|II+YKgJOr zyw*2I?i}HD*kb+%Zg%0JR3hH|f1oK6JYh_*{twe%a=r)1TZiVBcK=1l@8+8l1%8d_ zQrRDVT{5<0{}+(x>G)=G2wFJ~wE`UHNaM1Acqt zcbLt>v=UwbFHu9VQv7YRvftm+u1XcnicXF&0)0OSQ)c*W?$C{|nlC zNiJ>R2Y45dM1Uujv8U2oM7aa8PsJk3xIJT19gCC2qj33hG#P=J$zgJ<>a^y6zYCd< zI?~6?gKV~HPb~fomLVgDEQwvq`Nb!Xb9m}y=fau>1-x0?Jl@j2Jh_Jw@Ve-e37@%& zt~!EDT+8nyWU=VKtTb4N1R`if|JZ7mKWvM8^%}h@LtxIkZDReTH0`&;e-1+36MSg9 z)4y5z->W*K1!||B$uNWAGet>ww&=}KvjMC1v;NiHT`JGqknG|F?p=cgVrf)#a!G;s z3+E(7nU&_rtWQ7+y@7srpd^XekXyMiI8-|mpb%2bFY<>2Y$*I$s-e?s1~GsgT~cRT zz1+sbnw;RP{Zcc}#D@xTJB$&9PfglCZ&5QjZ414O39C09%ke&74RJdiXDoj=)fjTC zD49!tQ#pNZZ3OfrHd=2A*ISHIv|MZ^Gx(n;iIDrp1RGWBB#CVI1VSLkb2c}--FdI3 zD20Pi<(7vmi%OG99CU=GK7=f3#~}Yznm#xjE)n0(GW5(@%9wSF9e&S3m^rvo^v18e zOc|skbdu|^S|p(AaVD1D*^Y0Dg@@i1Y1O=_VH5?;|1q+xfS8&*3(A^35%G1koTj=d zjsIO6RzhPvUaMNySS*t)%Ig8rOrlhvt0=s8T&e1tn^fU8a(L?AO8++1@Fh8$%aalJ)}9HM*}O4_-%s+wthPQzDxO#-c(sYdmpx39jje`MG_Eh+dOu^- zA$k5-P1Wr^MCV0!GjfO_2hsJ%Inu=3x)2GGdEl4L= zuhUg6yG`h%{4pB5ZG5)~GFf-CUMwwgv)R(s^1Qy~v3?eze(IF+Z-bWM!_EjGNO&EE z)LX0ne0ufNa*ej$v{XXx0?ic^RxOyy^n&u6vAG$cSh}!NwNh8YugCkvp}h8> zlb0Hmm&z^S+svn*jQ7pCJ|!}zN){WX_NzTu1c)2!Ndgvhw*}tsIlN6?DDz)B38m6I zP)V#BvR9~`Oq?F~suiL-nXG_Tc_Kb}ggrAfB?cFen*>%53DKNCPK4k03oD+7r|B<8 zLjF??EZkI}wUrf7Zkb>X)tcQd4%K`b+udDde(a5J9;XySad=;7`PVHb!x-v)Kqm0S z66WXEPf3jSHXgq1izBDs=RvPojnr`_EQwhMT?ZNW@r{fIk8|DAL(%-xbMceWp0I&m zDPC~&`2Yb9e@9JKu3OLCrzT6@*^J-! zJS6ds84werso__fU3{!2ePB|6P*3MG7P`3zE+FYLQnpQCx5oI2jTv2884W(F7<+qp zpd{JuZlzSJ@!5biyAtemCE5|(4)H;13hWc<+YR=a8lAH$Igm0MS+HH|b93q}F891Z zQYfyL#CiGvBRvxg*G>iX8AYg6l`=|pubtyyKGSu!hH(s)}7Gum|O~?zqALCQ|?1vDX$lscZr{9`AIS;)OUf~Kw z1tlEs&Y2nJ3Zt^DThI?S&h-4-ze1u^irGaumsPYv(vZumY!>qI#3XiFKb(Fp|NK>$ zhA8Gml^!}QHUFEaTb7D zhyWkN6A=5yclBa!yOy34BU|EgInzO2eC^q)C(g}^94s^8&N9tIU0tinKYOkz9HeHE zT*wO~0(PYC7G%Hq#>EZ#xti9h|2#$rFb@ov5hJd}tkql6eAn;&7_?3^87g^!NhDOx z?x||ZEiHd%kj1&BB>|w=`6^8W%PwYTH+dTHJq>9)MbzUVD!~-=^6~6XB|&YM$B}Bh zfE_4r5$)uNr9rvbdIKw&Y~GSrBv28G(QVq?5#z|^#d8oOZKb>f8cs-s&}4Z2?DQDC z^0j|N)4^Dx0c)`mJ+c3$8lz!*mFP!wgV%A^(PaAY^$(akD3)d~VHP%j_q|?F2~)vy zel#R;OYRRlt#fSr{L$C#ncqoRL30Vfu411pk$Ue_kM%9pG2&>e(Vr~IDPACU??(L| zZ#mW0j$r4yWJJUC;)~uLw3I2?2PWQ;aAu<*>Y10ziG1wQfs=;5OPFmx_NblZr$3Wb zwE^$L`+4e{n8DWu@Fa+j#Vr-@8n?1fjizYHrAZtly%M11YZ{vqywl$M{_elCX01u` zot$%?y+3;&Bk3E!Wcai;US$o3myxoTWD-~oaJ9M`G3owbBEukVv;C0it(S!A=XWe+ zQzuAzf0C0vicv+GhQa3&#~9hfRbKR0h{i$UKLvnzo2y=59O{rM*R9jWsoXZK^)RIQ z_t3}WP+_0fWLdcNHGzM2X0!+dujiH=M136{RJDGON~gO$jfGqq7&<>w3PW-NXds@T zvc9O)V6L+q2FCMHYt&Q!n%7j=vFt0qhcyHA4h7b?=bQp`KP1t7qq><1p6GWM{N=AI z8S#%`X$9 zPO3^7uSI%_6M1^Aq49<(gz>FOq9{`ovyg^#Sa(t$}KuuvtNfVEZ8_zc}Q%_&>+^NT6(UlH|t4#_^rv|5ylJbsz zk8@Ef+)=Alhu_WgA}KPvNny|`5#)fj@2k^h@P9#0y}5Feg-IHMlgb)R~YWm0pG5Y~g|;zoy`v*HH7NxYrBpwHzu zf$E3FPe$<9YU*C7jHy4;>Vm6gYGXkq-b0%M*F&T;lUh#7z-3yV zNsVFDOoCf_!ZVPF39jhg&bOp;I74y)UEV$BvEA;X*Lt~LWY`hepAabuZG~FvP4=9I_Y`*A-}5uj~{?jOE9I%5HccH!-TxsR_XVh8g2)hR{WMJxP~W z%B69oY}`B~OwNJ07>|zY;lQ?lN6VILG;rHXYsaF^tl@U;gb=tjk3_k69~--^X74QZ z@QEx{Z_=}ijs1<$0cRu>Xf-7SkS%zEAj-3-<6&U2s_kpU+4I!tI5L20xFn<0J`*}&d)L%W9J>W}!>AT30 z{%}G1SK8N!jrqz&vO0gn^r~^TD=jw4x4!X588%fdFg?tTLEr#nwbwp)^L?ij+js+a z9`_v}T5yTYO19;$Ec;V`1Dv?uQP(<-OWm-Aklu7R#^e3k=!oQKDQqv`uo{cpbIm9o?g}<`o=h`W@VY`VDye z8BiiU;HOHAGWh=fc=%R&uK~ICIusk9dRgw1BL=^8gkxg(1TpVHI|vs?C>Uken8e3f zH4sNzlHK>G(@ z%l?XuOv2mgJup5)iC(v(002(VW~egv>Hs;nPx8%x&_Vy&QyQx4&4yZYOhdvDRWN|U zbzeYn=dR;=t@`ablkholTeE^5bf3`q=~87ORKQRkzxn(8p(`RiqwCN?0Plr=Ie;hC z26RRgVI&XopY?k9q8O{%X8DOOS-5Szk*Gfi7-Vw^+;nbiF`X&ocQ{-Qc%me1}?BL|&p1vXiW8qcd&BU$}_(B1pz=*!Vno!+?8Cs6-^O^?DzLz=V+kE(pehi4K)9HaEzGg)+&0j?n(DXpvW{hgZ#XP8R)^0Hr>P@)uhF`2a;~ zLA4R2`J&UZK|+EBOO|*N(hj~s^w@?=^)B;<2DSQXh|tZ#*@+ThR?p3<_S4$ zC=l#pY&d=(A0{bgwEO~fw7nA)=wc<~0L;AU^h z|J8`Pxgg-xJw6&2+RGLS+`qD|P99%%8DFzgS)M70o$jDzWB{~N{fJ&S$jB1MgM#L9 zXkD9bKFkZHu`0N%nz{o61W^)Zs0Lc@eDC7ysM)Bc6hoF zVOubX-&Rk%GOr-O9FF6Y_ZPDoJUgay;Cc){Xt&o|t=L)+uBJ`NnIngJ*xx>Gijd-u z_hwYqTvvH<*Y;P>gH)y`EvUUV^N-myB?v%RIYH$$Bm^V}f40Q;55Vm^V%|IsnT@Je z3}rs{P_rjV@Jjqy-sjyupq69pUo-Yei9$p&lFTMp+Cy&83GS^a4|3?kaSK87e7LFT zNW396RtO1UH+8*Ro%% zI8J|`Y$%-e_Yqd^XeNZ=FOH}x-BjRmYQl8bn9ogCbu}FlTd{^oKds1>{YFwHY&a(0 znlg<1Bw%eGBzT$M#$eubG_#%`6+Xkg2BLWdSpAW90>FNJ=oIt*NAmj;*Kd|wq(}>W z<7@k=9kWbPm@1laeekB5+sV6V3+Tawg}I5_+WJ+OX+~A#NMZYf*lz9YOsA>v;M}|8 zV@K3oZqBS)8xW1)$=hg!iR#JCWVL^ZLt3Vox8&YeI=>-T@R_6=0^(Qw4^8IBPcvKw z|M^oelA{GDRyjJ3n-6;$O@@)c$c!Kz=hcpYTUeCEkTBodYDsixC-=vcHay7|sNUW9=tc6>y$H&V{`s*y?e3c$e^%Qx% z^8NYlE(&?3HgN&T=}cr8lpLDv0xT@7UCl%O4(WIrO`6~4KB9}lUS)~lvkuhBLQ})L z-@&6fK2L-M=ZCVeFW**DrAsw9$#;m)CbR^=9ntvIWQB}*@xR<`OL&2K%}m_8rXYmI zWAYzg_xCR#3XyWt2Q`^l5IDJX%3Wet7DuJG%s8k0&6YHT!T@sAbid`8Z$O{bZ{4+q zY<;^iSJ?n_Wuhf%+$8XH-8*Nw^i*6pVC1x(RU}SjR!BZhXSRTPu-OwfcvWlP%;;wH z2~Jo5KVDecQI&sn!@e52cpt4buD*d-Qeb}C;-mQCZsE{m6SX1u_{uu6 zg+@fdVFC28g4=?EFm572p;u<|rDEa%aF_ zr@^*|Qucs0Wik!J*BV^t@+!D`tzJi<$uf0X&vVSX?uTl(HJJdyWGU0{;nO_}I*2Bb zU_Wp3pC=x;EES72IvN`VuAr%!f#73Sp5d!)~c3B>;A(wSLhavI__=iH9atgg>kc%=6cVTHw0*&iq*v9CD z804QOu!oA0d3gDw9?$u2fZtnO-;Sosc(QO@2tY=JKN2%8LA-Fe{TjQ>2^QVKk%Lo* z-(jZfLR`Hl;?XQq6edmT?T-89)6?tzpgV<$n}h&T8fbTE@IJ2}__$8~*p9>%(3rqV zgwEoF+PE(m8qGy__ZxXxOXgSwkuT^Vl+h~haX+9LTc>f#M1ZHplhp9#rWpq z31X2@PIJw;5s)Yg{e9e1ZBr(KzZs=2RtOZ!Uz0bQJqn?n9t?I~c)MApZTg~0b4$;U z58xt^e3sGWc4C~yUxHSPSEg3Xp>ojU3=I?^UTeC{k0#tuVh{e7+vg?2UaC|T)(66Z zsRFn3=bxLtCPG2+QBukuvFMamA%{?IR!U$V11+pNzV4U4HrNtfz+I{JPIyvM*e2AwTr);TJQy_0z>L z9+z-6L221SZi=d9igZSPF9p%LJdmf(DY*v1Dl`(NE(cF8Hg5r#*HfuFGkJZK-nR$w z5ypu^&xLK~6Um0XAEjmNVBp$H`w=aa(9IOEC?wDezvMV!xSjJHx7uqDrVJEff}B@7z) zn1#j=E=tLMVnO<}`r9DR!x7}pTp4`{n(@LBafd}As*!)<7VHuUcXWxM;GIqv5`@z? zTMOLz7M;aokowxG>UwyU{3SLP6t4@AJ_=yBeRWUK>|r)avelq3A8D|!*yG_3Ru4*+ zAMeyZ#&CttOJg)tcNIGqGCjjOzVc?lyBUh(b9!6n6d~NCM1}&FeQ^*zr>9ZMQ2{T# zEhC|$5HsEAuoCWoBkMnsNe1K0gcVip`}170(FI`05pty1mkOXMibRcHHb$d;Ua2(v zr0OX##VOi*2Fz*xD8x~}7M(BIA1!wc0aDz*CS($UI^RmY1Z)y1+3f~cRLf)t==3VX zum%KgZdrgPy#aih($?w4ib<-Yg&ihOe`MI*9>k;{Xh71%x+=~(90JRV(`ITOfF-4% zh`OS5zk-N-zWc~x;|u8I@qYN_Ucm_BxJ@u1IyqjjUi9hdZU~$GDKg3xk+|7rp*X0M zF2#I2dw`u&5iS^?5#oXF^&v&Aj#XYuT7J&^ZL1Z7e|H1tDEPLANq`sqe?+7!IFuk|eAD1`g&1yhUs%`z3UmShN~A&b z?QfAoNym&8$%>!gZ}OWE;o~4(uTV4yxGt($vq`JLkQSHB@E{tM*JA|5MLwK zF<)uv8S#=E5`jp#4FL(!ASJ#eM2v<@tJiv-$RcF(!&k+&`E7*fN1*0MWhAciWdzhk9u2Yj(O9u5AVm7KI z;*fWCoO{$0@hIv9if8PfUYfi#g;omHGzXC`zPw59`9IcXlJN991y%QYr?pfrw+Pc-yVA;&YPhVe*uOllz0O5UNg>^LECYd{LG1QPVC@;C^|g>pT89y^1yQvbSw%9uFDCP|Hj~>PBx=aVu-wOn^ocq*uW>W zV>^Ewmtv}C`!S%IBK?Zi$?kZgBnF%D;E)-bq(lqquuqjstDW3=7o@^dH-<$eO6&L$ zkW~3;assGuDywR5{ocP4`q`-AD<&?6ug1B>Ph>M#E}xFXpFd}|u(=}%__oC@gox$N zGgC@L&re)QVwtq^zZS(zjbNYf@*FQ#8_MW%RG~6|5DwZV%6(mh9@^PybRKtocAHd$ z4_QzU&(^DRAwm35lw4bNw4};I=nx~}ViK<9Z**RB>LPg|tZ+oJPuYV~iOm)2 zJ3E}d{f-~V@+4I*2Eodkc|87_K6h`16kx9AlOcA~C``i^J=;u19Wa@QsK<-%;>U{x zOt^6^+G#Ays74G6dmJG40JsX6e{ugnLZR@@lkUTQeJaYn-gl-mD6DssCT7y4&GWLH zfh)_b0ZA?@Zv{z3O!3@h#gVcT==6LTuQCe&*)l$6+yliQLUJlR9L0^((XdeY+<;RpQbP=9!* zG>eT5b0`Q`&OMGRw?u|Th1vi5aw6tX>i)mFVoh%s1&tJ^LT(R)DtzCWJg&pr{|ZuMV*%kGW+~K2Q!f-_B-sNxyG3 zs`kD#V&cY7xI(u2R^Et*h?|V|L6edG@cVFJ$q)>T+UR!C^G9+W%D61+_+hafEUwyP z3!TAkQZApthGPk(^hB$k=@Y|6P^u^3S5NYJFkRttcy+ zIZyBTf=i*_0}L-uf@wImwM+t^$WsOIP57K?k^k-6PBTWVghm`J@ql z(1OR4`k^W{Oq>3~7C4mYlH~`v5NNN*G4iTa^}ipNn*IrG%f;ndrm?H7R$vd`67zV* z{bk+b1);J6_)??*4(L_tyjvn+<6hQ+l#~U|7)>%=cKHNE6+#zdeXG_mFzoaq6O1E) zfeo*0euHjT59r;$^q)Tqowc~w+x7@;^{eY$uOuEwN{tD7N0kN|A>-DnLdKOqkma!! z_F?$sZg-4=mG?jlw5Yu~))Ck0;}CG~9pK$s{BmM2aY9tF3~Q~A>-UPE1Bzk(Dm2ha zXQ!_4?oZ`r&9-|;8ngmo?Xwg?J?Idl-0NFIV!Jq@F=DqXtvOvwCUMlQU%ci9Y5*&t z1tBxcpQ+#TZC`}j9JR)u$-B=WAucKG>x$Gvnw5^9W@~=C=jL=h-oQ}$=Eu?ul$Hrs z06?BDi1KnP5t@Q-DvK8N^MlNQJl}ay(7X4GHPMNdrk#r1_(;w?F8>X*JAe%0e)BYX zRJIJm{8ioV25L<^>LNi40nlMeAtx9eDW%Oi8?CqQJ7uu>23PAhiRQqJhfNJ;o8$g@ z+wSoUyiWz(gFF;ywzr{O8uc#<@L%WDmGw*@$Fmft%;BgN-(>tT{}gk-jV7p)Dly}6 znta#qbd`AV<5!=_c}xs83ZxT`B}M}D4&N{kl4#ZartEqoR4(f?!kJQ@{U~XV+zdg^ zLrVR(Q&0j|DHyU$7`l9zFazpS3MB1nRe8vuY9iHhq<$5@^hyCxY!_q`hsmMUsBpeb zPbdp=RW7xfHK~!$KwQov9R3aA&AB!;exi?8>b}(O2G(tNQjHeqvc%(jrA7PWLC`i} zVwG39Lf?-x@iOr9x z=iJc8(;E$f`VNPkJ2f#pC@mbTYrnmYzW3QaQgFigmcZ3ccri0ZTfS56qdE`}b%g|8 zS>+U3F{@TxSb18}YK1u3lfS|pTI}>uFp<2lVo7!OOkiSWg-h^e*1l~cb{8zJ|1&ej zgXC7=A&1AKRX+S~>e@@{3|v9<(y3RgOc3_z^$7?&G1ib&z?oh17zunNp&b%IK2Q>7 ziO(a8T7cRyz66VUtv6>;KB`D!9s)Mv&{nyr74=J61}VBCroJLjTa*;bCF!n=`)Y%o zop4yNFKjbL^IHw5$|TW$Dok^%^3SoeJOzsswA?AeOMK+iV-?gFju`Pq^f;${UzB|q zZn=B^%1Q{&7W;T!*6LXaBxrfk&VbW|8!nI)AW^CE)>LRk)Rb z>BFF`SdXngBkZGp^$$n%Nx@wS#&kdks$G$6R;z;$Id?7z_QPv{|Fu2>26p1hnLNf< zTb$=_8+ZEN0frc^cKdwRT5+{HD_+?19_}%2H!dKTik8I9&=%#%*f*eOcDX2zju{tV zyXZ0|a5Vt>M;y7nB!|jGUDy(YDpc9;6B#hQ$~-HW=yW}gVASsvqw+XiG#8#3tewHe z!Xn>BA*P7Y4In?|b_!h>^WgPFk{};`<%7hyRA1BT&SHg*59QzPd~RK|H$ZVsiA zqtdhEAD#}^x71?M5ct-t{}-9AL%aoosxa|k#OTK>6h|J?P>ZL$hR@GM^1q`X0Qs&Vxus@c=PjOSx+-}+j5zzvMOqdv|w#;9Vp zXyNzqu|W-)B_Q|C(;!E#{Pi;$nUB&FV$RAx7~!AZw&Q^&4K{e6{iI#$^SQ$f*6@r? z;+{?LTu0tbZtxULY_dZKmmJH+xID!rj_W^7uXChHVVuc3IPWhJ1w>Y9*C z5ca`IK|QpgLdZ(d3S%0U^uk_Ve-}Y}?YATE_fN(fS69pvy3e%g&AE_l40LM;4V=>^ zcqEdYu0BT1Y3LH5Tdr{(R@&Ur)Z?H&iQ3F6^ebc0@RtGNf|V1?6GcG1A$V(btJSXv zQS~WkdP?S&0-$BUr&>1)!>4zb?+?^BNTDm~68vq|l8JdW;ilZXgUcUgN%2y|!0-!| zWlN&G8Na3@CQ-U6#;ZXEDzixcs%Xd-m{Tq%Dcrf4GLul2Seq>Zs69vs&@wmDtRRk%0CsVpZyeqqY}{taW{I`w4@s6%>ye84VZ@nrigeJK%k;_7yI_kdtFq zUod^z^5&sxRE9M(uYvw-$R5UPkLG2CPFg?$$i2%k2()M1Kzm&kdE3nWjBn~xJ=AcW zqDmo35|($wsHp&*foF@*?9OcXKfyit;}d91dvl|H{qWFt+xiFsQ^4^95#RsS>IE(D$d5Y<47wClkgOR=9At;w+k|+~U zbz4^EYq81Sv@i16e`8~H@Od4SQ+IxkrTxsCMo?+Y<)Ez%PXnlOir?|+!Yc}1ucJxM_>EgCN#akzGhBrazQEbiplaL**TjMWxf7|3&<&`yn z43dAjqrj}`O1yD!Xh0Z2Q#8J?#Hot8e4b)}5r-Q7N*z^bM2w^>y)Qf}V&uc7YFlp0 z=hgHKb}|gz2GXU%l1a7^>Kfz_!kj|RGZoZOi8ke9v9o@p4XK#NTBG}w>b>e;*KjQ? z+Q6KBTaxIX4LX>iw`|j2X>E;J4?9wr}R7>Zqn_-t1$7vCX|w+_`z3aZ7RZ^uuz zk0vbc)>=GDKrwC6QAUGq+Su8o&ylTWGkq5G>16$Ll#ISK0n*p*2`ZGnt_}YIzk@vy zz&BdcT{9D#b82GLX|;5305fz_Dt!1qsm!;gNT@!213FD=$zUYMT+G#`2t?q!Jx_S( zYX5ius?(`T=Mv`Lz#Uc!KbN=MJ>R&k-Y89cU&<;r>9J-1^LuOW^pquo-Mqg8p0dk| zD}&v!T^6`tHJ#L0@_q7-L`s_C3Ds}8pvHVUTV+Hkldi+Pk+#}?0 zqV6O6d-;+KKOfIfShK6!tndpvX6{SPwi5zOLAc-bn$#Zb<$Lb7pHt4@D<|AA4}^;S zgJcZJ{&sN2++Ra7klZ*DLNzirJ2}OUd#Azh9W+KqmH9z*M7~_Q$4(|vg;u@@=FiM# zP-Hy~wvyoD!ZCf^wu$n+Zg%?XHCjy~jc>raJff&}nzOBSAK}nJcLh8RMb@vtPR3+y zxeC)tcix@Q=-j3ypCIc^3|BOn18M~msWoiz1WhWFE9RAb?ucj=<8ORDj;;4@KJvDk zc;5o0{#I%%K7H_~AIXE9P|_pxd`d?|Cr=^iI|^=}@w^^q6;NE6DQ^(vGFPC3Qo`Q9 zsB_6;;;zH>Wy)Iy&U8kS+zHXQkEj`nVc9;pw|;Fln;n7v33I;QQDOBc)chm3X#S;V z1OrCnqoT=bK>A5~mN`$xaP<%odyM886z&#;MH#ZnFLpn^OPG4{F5o+p=%?9v{W+N} z0RX8L!ReBu_@*Qaa;&%i`Wk{iK-n#xZ9%vI(#Y6cFt(mMeQd$lF*H75>8l)Hl=wPd z&(w)eld{7@aL+a~rWu@|GLd_e`jWhqs+C%XU)w#8`SYOrrl(UVqcH70)|VJ2dqbtn zcjqlYHJ|xYL*mh(!U>D=qey8G+$r6&zVs7#0Wz)sSW7>E?%5@w&kNX)%p81}ZFQRk z$7Ay;WXJO92c20=af6|DzUR61v8eTAjAYEiQm87pe#cnNoWVmcILu{yJd@fB}C{ya$j52h& zle1zLI_I5X%7}JTdbxW$?cHgC7R{NtYiF6X#sOdAU)Z1y5zJ}5&(Ak@2@}kl_ha4L z!HZPFq*ep42jl|NcReXj9N~U;!SV|$fIm1-nFVg-=4X1U8U?{gKgUYvR9!!;<}G3g zAjQSU&lw{FeZTq2i-_bCw~ajZU2sZXY@gTWxrf>euCpJG+56kP<3k-x9Q*$&M2j*1 zAUuzIC~k9MB_v3gVR2M$a%uWiFhW9b@NmwLy|i(Pwx^{}>fm8BU$Ur7U?d!Pu!7=m z0Wpyk+U4uoqoZABHl`@D&Jim>Va}sorLf4KN+eCI}Vj7TvjTaZJAX=#db<9fi{HdT)W0t1*GR4!5AJ4hTO?p z9_rc8)BnXHwhaDdtvuN%X=MsJLHU_p44=whftin0*+D}{=QalDd6{Gews@;;41>0c z(x!JzU?crF51(5(DpbdR?bC2Zw17ggdm*pOl;r_quJW{jHAX2zgMh7;u>G`uD?*%tx% z#nQWE3MU$Aj2Y}>uuPf#dMk2VA1!Ksp>7XA1U+j)CX%0@zXts*>g=X#$@gR2ihhA+ zzrKWOoevrWAIij=l7w^$8`XQ1Qk<)TqESx8bQ59w3bm4;Fe5#T0ZrVHo7}9{v*fOC zOqN0yjO|+>{2D?Jhm{LndEhjdkU#3K;EG-`l`(3u(okmqo6K0ojoWqs#onQZ#Ch&o zykD_$u)tpMJrs2KSjnUomR!|nBdE~)zenYr95K*>>y!{{wntBUts0{slj3kc(q}%B zEwSeY$PE)jN;?=6TSBb7F|Sukqc@aREtMzRL&g#g*NBa007ukl{}xkhMT-q-K5*!_&1#xn#@ z7Z%1SDI;PUL^o~W*2Zf`)ppk-kpIw)qLbf2ySi%8YwdLm;^_@^%V)LplV9&g&r>1D z-OF0!#me~k+!Ig}JC0j`$zxJ|{}l2tNS+`ilDXXh4Du5;g>Yt!jxUU@C(}JkAPl-5 z_wOU9SdYxj;g>!{D9N4izh6gwHW`#>CCe(H^Zw*SE^ORnh6^7YukitMx;yDkmw9N=hj&%mLvuNIb zb&#$s%ZCBk7P%N5sF~c!puK7$JG@VZKzenFAw`YOZ}`bZwbGglRD;7~>Zmssq3}B% ze_?O);v!XDD9BtvtdC&Wm&yqoFakQ0H3Os@(z@cNTU0UW4&C<$4Nf{j5!?}#=9+%X!21-2z!2+{>^TB#<6(x zH3eT|;`l!1-6eHnBzr$1kU>Jl*-Tjdao1VjTndT7q^2${#Z5elPXlwF2_JK$$KNXrvRX7_)RdKEJBhDojE}jsTC+) zhv$YP=dU05%uaOx(rL_ZTXXt_1#T>|RzR~xh=}>@4L7^70Lk#qr`f)pHh1p^*|Xz; z$|%-cuVl5HnFGw`7Bw?sY}u3+i3Q_O$mWzLw)=yX?YDfWMa~99xIF``oH4ND-fxx9 zYmwt@7~(NGENL{`m6iUxrpfjCeG(@tl@K296xYGL{n-2L2Q0i84$A(ssr2Dpwixqb z8Z(kMtDAy&7<_IP&O6(sLwc7cpnB^tw!{_Brc#q^IZR_P zfI1cnOi9=;lZ_&WE-`-J*vp`5dMIlR%|xy|T4Nv37tMpL1OS+IBb*D3#~;584Z7x; z<2L5U!|KWIBry1L@kVP~R`rDi{N47P;K0TEx6tA(aYOs%{^IdUu=V}@uoz%j9I7e| z4O_4kd}y3SI;(~D738KL#TR!a3@gchuK1+Sw8U%VeL3M>Liv^w@ED?GpOvf&!b0ad zLxc1*9%*LvmAhW4yq-S0z(j3>JOPnzk+bVKohE(JgH`ElJP2v2w%?w9BF(#4)=Q6e zaezGOMVs5UV1EG_3>*uA)le$#A0oD;N5L38fM)+fU7CHdQEq;ol?D^MZ|JRQcQ&dr zKT7X+!P51ov4ZD}>)8T557bA}>hvS;NXjiCd~TX}$s*dF2Lx2>EHzq=wP^KJ9m(Aj zVsv3&=#|Yx2?D8jsIgg8!S;ogRr6j^sw>|>1WP_L9i&x{lr#Z?157UsJHI(cuC}1N z{H^(a)rCtK2$h)n_J&dQqR>^|;zj}ghY-cJLK#6Dnw;x&|tnPyNSWb#DYJ+(@urTgG&s(O@u z*@o<#@4mrnWZ$P*@Y(0PNX@LP%B8owRtjSK6pbB+ly1x$dY~R4ZZu!j&mVx zI1e!sFN4vww>~0WM*MaN$~hP}3-s@Jm3M^YcwVg2a1sPkl6XGL&g{I*4xk2q^=C)= z|1swNu1FEAaiL%hj@V$kf{83!>&j)Dwo;~dM6@SRP*!7bh?3u4EkKvlGyY2i26Yz9 z+-^xkgkY2zO)$Y~TNmHYhB(1kHH=MYy4;hq@1zg>T<`QV(xnMVl{c7WGZkq)M9U(_ zxfrjR$bmA%$>r55E!hX-IR;59%5Oq=NMC{i^B?wC*U8x=9MEhZ4U72bE4l<^mK)qI z$vhwkVE_VL>px-dFX?{fh2iiS`|Asr__rd}Uw*WFt+4<2o%r@m)u&DIoRWy%PQ^p9 zcIrI}JLnC>T2VD6v}KX`9nXs~g|wP1d_6dtDTzSk=AjOF2XQD!vp#Ziq)`U?i+G6` zJq#RZ+_l^DaIgWOoKG6mKy)Iq;`g6Fu#+pyU>QCqz{rEy`2f)ZtA{}7ah6EGxxX(a zZ6+J25HA)~QbMAm@p>}!dMS8*erEBR@VXOLFkWbr37Se9Hj@d;_9|h&G}OBG*W_M7Q3j$n65{uFZU=v3yo0d#zjf zY%i`Q%VHwBgHNahs>H;04B0XawJOwr{t5DmPpkHKb(HI$bg8~j&`%~qbg=gjwBBu! zW6O6$G2El}^8$hTXXSEPlRlUFbtAh7X6bZ-o%2oB^R1}|ySqec&+;XwG7>Kw9$_oG zS=q!?)DTA`_$_Yz`R~9l?~)f677WXz69b02;xDr)Ll}ZP9>?GBPJ{_>amDJ4LHg=H zaSw$YO~LWC^osAXV3n{aO}y~uU5k7BWGZKoPxJQe{lWn&(hoDURz6!u+J;C9j<}dA z_iz|eCOBT;wg$gIu?Jz)?2_C8TAo?-N{<8Ebrs@#i(99LQtVKM;GD zFez<)Mx>29&*Dj7*x!;QZT0J@3!~iSee+2G2io>$qi6$*$|>6q4=RC>wixX~c2fd% zKT0^((#g~9CcqZJh+tU^J1ga3fNuKCp)--MQ+OR<zU$PZ9o-5#P_=so_ic>n1bxRlt80#eQCu*t%f3e*!Y zMy}&}TLpKz5gEdTI5_QaBu?yXvJfqnyZg6U=IW?7)HEnMcF776vD!Cel&jPEw^POo zuOUt{<^#Vdv-UrTYb>b)~|FY)==W&-zmDihD7z3ZSq!R*_YaRxW5=rjZirk}Muokvw**ZbR6A;|u&jHA_o z&l&gkRd@cau9M!&w?#9edBl7heBYADy!cda?Ky7zRy`;LHoM;FZW3fT_OE`dZ7pDB)xY}FB(~=>~OSR=(S)xLob8iD|87- zmcNT{giXXGAq1EA<;Nrnr+VL|4}L=EBN-`thtP1>Kjd-GEWuLzbtFK`NlM zq$FSqDDC(az>H}vk?^s%A9p0|t?;)$0|VOa7wDr66@GJ=z&vI%P_HHlNhOMV zcC&Pf9+0I0VnECGYjoOV_#R)dTbOJqj3k|6uuTooBuDI@Zdvd+Oa%&q{Ys?LW{a1A zF)8rI5wpm8t>acXzH+mUL~KfqOe6+ifq#PmC#6)l3CZQDGKCwZ-fA82Ar~1`n281k zNw;wO)5jUeG99Y65);cron!BikqM;L$w;p+a)K-nD!jfAB+0q{ed5gtWsJF>Ws;KF zex((ek?XwtmhJG_hZ%Gxuu-2ICh1L+tXZefvvK*y3WY%VOf%jsJuEm*wBvsI;P3eK z?(r)k2B`H-9`>)2R!c52o;goHq>*8Q2o)m{v0;CGJ)@=NL_03(Is9?q zZ}F5qF3prh&b|+LR_*Y^!{e*3Bc>=QrQZEP{ANzPtWx)t)0*gl!erHw6GuJ>b;zQa z_~R+`DD&2gH9a|DtY`P+%IAMjMoILvQEOQZ%ID_^E0SbhasJHJ<3aOgg%w{B!Z6~)NrOfvhg)fA^{{Jq3JW2@vDs@H% zKmz6g!25^F$6kaV+^9Gok>*fmcENhtdakOl#=R)nfGRE?ALf)oP$szvJK`q>K*$`d z2&Yk%E>*{O_TQjf zOYK#Ik{RXh@-zStCta8;8MODRTyh9i-m}YMn8qwV?-}AGBK8*TpE35s)4&pms-cuA zcQk_%=_0_G%Q*{XU(9jZyX5r{o;k?|()rNx z7k;CJEqjywHtSlxluLr~pyC%>$6bCf$-+(~_*qNa!9Gbj$-5FnnTK;vK~E+hXHzjd zsB`|n0vkYc2_2(PDwX9(Ap{@AQ4pg(lz$9X4S8Z z$F;+d0S4QgR5n{FoA=dcy|QdJ7ZtY6IHPF?$L85koDM-xuxGd86RwaY#&?C1E!DAHo_LvX*a7f|9qSY7kmoJRUdqkwFu6!GlVYq{Vb63LlA&fW303_0ixB6nEJkqcQNQ zO5N#l>Xduq(^wKfi{%Y`_D`|JtG43^+{b}XnLkXeAm;D5-p8pvvIX@)A%e>mUvqK` zML9V;$79iJi?CZw@_3)6gI~SP&tIlb>PKEMB@l3u1?SMZSITqU&#r=zNCaizo#Ajp zhDPHiE7Yu-WzEsij4N;7@?}4SA8H|cM^2IZ%xN^?8lrk0x3aKX9;ekrIPli` z2!cb;gsmQ`=k7Llw%{TxhUKPNz-p0WmD&PXAnN4OsxsdC9y3o8?4Sx``pc%ykkQn4 z8LZ3U^EXx9sFRw-&>7j4xN1YKUi4q;Dr(e1x%{O}7%4N^o?8?KeC~e!UeA0z5xB(T zaDw^czi=#95~EeK`Q`O8W5SUDFdz3Ga#xV@SvNQ1)E*tP-%aCVuH)E^8y^~@c2|q) z|1tH=0eQWR-`U)Ut zg0$wsE1x-q7nVQeOPb}nbc|@Ud8J@KVtOOBSsxGe2xKJ~w-ClM52_Z8MpWcLn%Qq< zM> z5?df%AlUmsZ=Lp8wWqsq!Z1jSW=5lX1JlkT?|ftkay?OpQV5Vmr%?)LYG`}Dq#1ShzufI)PL9H>-x*D-TZa$O) zvK;h>CK6%fXBw|1`7FVyy))kt@tN%?;Vy{!n1$j_GpPS;gbvQmJVPx^*Z=eTgF4AY z*~fi779#*IU=@>aW>9M|;A1E2Fbv)m|9J5=|H4zKXA?cgRmBq?A}?2|**E>MN)aBJtuing%iBBl ze=9ZqyHa8s7^c{T9&{S=a#cnucgKBUh%g1G@tqI)B{xp^$C~mbyc45Jw?+LNg?A_} z2B8I*@^N1RrK#V-4~u2ebe6~p3*XzHS&jR33dX%wS<|_0EAB1-bF6+6BsOFhifID% zIuu_L6)e2fq=!=&ZO}_`9kKjzCsKrgwpFZgdSzO}AYQzeqFho%gel3Bi_`quBClr>Ey*MXGVZ{>}BT z@)guC&2cM##0wXUgy{7S&wb^Srhkgvl^a!!)2cVhScbFki=J8dCZ9f@|;s|?@IB^3>meZbHQWQpZ zWRWsa1hcZNIUwj|MvQQr+iP2g!zij*^E#pB;%bzk3#Ei_`Da$^8n zo{lDD^LxNE#iM->(&Df8FbsxtJrCDY66mX4m0}O~_1N&vu_V=F&ipWOpT{a~*vT=Q z*d%~I9_&Kj?n6b=!^49 zfV=Q)DUBfKdEe1n=hVlGVN(;Q#X&3mC&GA_y-Zu+voTHMH8td+`>JHe;iL?UM{;nc zfGrKJ5VY{~1odi%1%BMebZ;fTb$xAV+#4c5 zCuo!k+lxE0*g(Z)d}%$_VD^ZbtorPav+0iDouM@G71pAJZ_g;Xb3gN~#C>(wxC)&v zd(b|A2$y9wwTG(?>#NnfyixM^@tQhW@)ma39mP{@y@G>+R!ba)FBki8qg^6wBOOoe zpY8wx)nxK|>nau6p}WS6P&XYC=~IM7gsya?8j(VIw)e)F+#= z3$EiG73r>Qh}oQ3(|E~`)wX$lKZV0zVGxbRm56vV^u*+$(^F4W>{NNrqAEv*LWUN+ z*!dV2*1mY*59j$dy3;!#3`}nG1NKU4VWFV%=5D-)wvu&WgnOIn*^(l50*AVIVLd&~ zo{B;Tx~FulJ;CH&(7i6?@Mn|9C!3?OyV>B-B=S+SC~siziHDOpLZ> z-KGk?9aVoVl^FHCCBL+*#ftflIM_@>|>sPgLvk6O`39a>zZqe2Cd zTp=Xy3->-eBJoJ@0To&eR!(YeDF>4Y`u1PWA^7%jytaQlnonO|RQmW^ai~F)9z!O< zyPXw^6Ic+?e&+_`G20-LY5|DE1`grwe4#aPf9laULSQ68(;PSE#{RO{Qs7(7*j&3! zV`91T@X_~K%%YcT0;|l+aYCh_qt+;TpdL5f6aV2iZkTCgXyEBk0@&wEDpkdo7wc%) z0d7ukh?`bt8$@J*7$DXv`PVMMzruZJ%g5pMy?z14vFP#}UD?Cr^95`BN`(fvP*?5< zkcin`S6xLvotS+)>{7)@D7KIRzZZq<9#T9GM2|Dnk+a#V%5Wk*ebejf?li>&oZ}`R z&@|G1gc{xDiv!*!ImkHCh1_U19U(m24-}>UG-YxuPkln3{GX=YN1)YP7Jy5N)zC=s$1BFaM(4(p)XW7O&;M6Ox^#TAa`frUgqS?VYoB~1acJ`hc zWZInO)IgYv64L9#&`g->lBI*tC*d@XlNKdvXth_shpdeC(xRif!q(1#srj0V9>xhl zoV2KnP8DeQiJ%cfgK|Al-QX zPwSf~^LvxI7L%w&vGC>b?0Qlontj0A|NSl^0f)KUDy;FOfH3Y(ziD;aY^e&(RKXvUqW? z%`@eL3cssN59h+Um#83l>QGlVUn{B75%&UHw{_xrW48|>;CA{9h)a==P4s!(kNBbB zsD|Kpe~OO2JK|O#Q0h}^bsYqxu)uW^Z&_D(%$k#Xfi5Qg)eMnli<2!(F_CbE7(`P_RSbzgRRsj$3FmR2y&G~gHaEG zzbB#2Nc{%!!Tt7E1h9fAr~{UQMop z?QR4faKdtXR_XidWwDue9f3&P+8vj!3u5~dQW~`azbHZ;sdm{2EJhN!ZaXfUMMl}= zYJW?X+Ng)zYNudKyO#o7gkQY_@mi@-6l4ZcvneByN8bV+ zoFtLDacoxbR+)N}@$5e01`AGiQ(0kRsE^+kVkxW`V(cN#u%yGf{> za&>;0H78%SSMNj~%M>!}H{E%y3%)2Bo`N(AL(r1GB~kX!%oWjb&ug*-JNPVweE8VzA{F1QeW* zejTGFJ;&TREOf@38!S{Fv*%hEajQ5NNi@=&28!4oQOlH4m(aBYFqKUR|<@yf=-pXjrd-S#YGdldubm2M0W z7-z4si}619rklq*L(B;j926Vr0=BjEPnLyOV=0Wf7#+_f8mA6Y-7hVF>c>pEHI+|y+LUXUUk{GqcsPLn}*Kdj&n(gZ#jB=C1+)#W?MLXxJkO-LW0uw>NnAlXx`Owug zv%H7I>|7+o0YM3+yBoD20fSMDPmHsj8-6Cde)llwwC>b_r@WJp_xLRLGhbSQxCs4W z;uG4q*pNojEs_$?jH)tNr^_lLogR_FqqaNL=AJ0hSMbLVK8_u`F6TnjX*C~;m*)w0 z5Feg=33xOpWpj_|Q9Ai>xh6~#`SY(^dP0+fsWi0I ztDIPpRN2xTm8ADhkVbycjJX_BBG1;gNq7x-l-Uc!yWq%rrd}^M+Aw<8E`_oPU}!WO z_-%77jo9{My>?1$oT7!55p}LUKAHVz%07jbQ!%=0kD}pj_<1t}HH3$^qT*4;$mHEed{0dKt_oswPd~j3PeTI=P+qaqS?6@X5%*r&N%SJ$bI?1To%+5S}%SjCKQgwt}t z1Wx9Ij$w?U8~w99oTFgiL#k2+c-!HI>wTxId<){2;|p_mX2Ox$IKhEFu6 z`}e08C&sF*@PGdhCX8WXt7To)Jq4V_|3irUBsP?R6s!7o70ROh-2cKA0N?}v7j447 zx*hxXOMm(Q`%dz}TLxQWLj2!7zyiVn|Mv;+pT#mRg$fZX_WzC@{(|=|i|F<*HQ1r~6ggK_*RPBEuuK)b&JAgYeKl`5Q+;jn0l>a+7L^m}VJ`PR^ z!op|l2y7;e*$~Xbt=yciB~W=igub2CCX-wT(EfrW+Z~qmpPA@kZt}iE1_$~X3WFg+ z38=b#NI77?h7jR@Yw1Z|*QK_SzV|ke9c~pz{d>f)2jZx_-ig2&LrXz}gGY}0s@E7l z)e9qYG+WC2lZ)+yS*5|z!9kQpo33wzR-K+s?D<||Q9JaWy#YJEi$(;Bq2n9u-7y6a z$1d`?Wm_1+e=^O7))rEAZsLFs>a5~YQEFT#!#qGpp`&xq%AFhRN;3D6#S0@zNg_h0lL)vWuY8W zH+7_g1MjLxFr3)_RUzTGb^4=5=;8vfExM8i?Z`O zE^RiDHq;GTNyg)N7`3sfvw4J|f$jDiT|wWk(&Saw;;^Qk=;Gqcocq>t5&bvmq&IDUBmC1L(U^e^EeuzL-k*0QQme^LhzY2ax-+`xDTdCCM05 z=tD^IIG*JJ77}@@Q{DHxO|O!047yA`)&g0wlxp1s0BNB;OhZ)&xiK#^ni4(gt1~#2 zfbSWR-~w*fE-%3u3-E_#L{i_rT0!uXILPno_z&A@h7#(wMuKN+3N|@n9-~r zFC|k`dwc1!j)A5KCQi=A4^i9ZHUqtkYHrvI8~mIW)hCfx9BEt}mT$xe9ufZf1CB3> zpurwI7s0g`)v7>b^5?chdL6tk z2?McHsE6}%K$Q zm`E^OuijRUzL90g?dQ(T6Le^Bd- zwh<%9gd0HjO?B(`TzD-mXblCVC6F^^c_Y9_rc}n~=KF*^E)r>-?N9&m#I~2-qnG^t zBhmh2QFzKc@*~lb9qs&1JVuYi_X^FOhHk zsp^ZpbSASuQ0XHF&nqPNAcsY-X=Bj3nZl&|;BXu zMF{W_+7L!uaU_A!;_mnA?5IcgI({HT8IX17mqg+7Xhduisx~;d7lNG6r*&CdjAI@5 znPhSSs;@sYc)CJbz&_9*9w#|Tho>%L|Mqyga38G2?J~YVK0_h|`KVlnP#!WX;<=D6 z9!xC&kXFh5k=Zpm@LP-LRh8LrmIsEK)SVOBL9WD5uoQS+E6>b~_cq;8?{`Iosb6EG zn6&4c={sm4qtIn;aWqj5Kq&rv8XT3i1XpMS z;I-Mj5MI7OzKjIGnNlBfS*HlSYn7h80(CO942f#3^b7l$P;W+UTvpbV6xzvOKG=>)W&uX3qD)*Q zXSkQ$J`(z7a#>nTP)CQ$s$_Dd2IAfi-6yA0av<6^ZUD}~^X8xy4?p;UV6k4Rj$E3; zdTF-q;(o|Yx6%k9^@@Xu828~DSE*uq{A~Gi%WU;)T7^cvNRb$r6e=Cj*!!}}sW_N^ ztU(n%v${oZ!G0!pxY2l~;@(6$6pSS@rI37;*afaYMW;&W<0^_;2yNG3_nPjTMvcap z`@_aLNNIp?t^$4JhvyyL@kVLjvhgsJVF3_hc6Gdi%L)!5o-6ujG+XjrmO3BZYLlhKU>@YIi}<=(pz2KzSVX?vzk90}BybTW1|k6v12K>^zTf*N>sA?| z18s3=5zVsG3m}g%TOU@P~0qy z@?dYMyb+|$k$%1pIv>9jN4aXO&9{z3jG+XYB02NB6mwrdJSCZLc6{S|xoe1;D+*6? z!cD||XNgDr`(%+7vi#);E~_`R^lV>srz>}x61&%O{7(i4L^dtNEa$P0UN(S|S8_mZ*2^W48;wMHE9E0C&pb){5kkm>c(n!xf_^1AfapVPoz ztT1Y`E7$c<=_02eiJ8@0V=*ct&Ev6m9L-9|bhV8Ma*5L5z##e!?9$cD6+&G*CjcE@ z{zJ*CJv}jA@_JM+dkuJ7*EwGN1P$G;NFa^9{kZ?rZ zcEn!qz3(bDq?q-3MWgZgXy0vCFI6M7(vAF&|Oc-(SRHFbfK@WVWyjf%cRF6biw*%S3$VcvD+B96~36pw2zH0EIubnj-(g@bHjmX0hjrqC% zVVaT~EwfVk?0m&K5KW*aRXJ->zv(-3h1|`$`P&&u=xc)rt-uSJFPJ zelazzui4iMN)_m@k8r4TwnaD0q`QDJ`{m`l_eEWdWo#J@>;omU`1qB{m)&#-pIVSq z67Cm4uZvAZv1b{!X8eV$t!}Aw_F4x4a>uXqVsH25QG`=^r@umBldEcbpsz&?tjozk zF|z7pID!-}guhUwW?&i;K?=ntsuTnjT211QCy%TV1a@NY1}^+&kJmdY@dDovq8zF9 z8lgsm;Qs^`^Uf0k^9eWzuPXCsI_13PpdLK|-+rjw4~P(sJ8ZLgnviLT_01NWOk4+oN)T!A21L2YmC?{A z9LB>Ep^ae^uh4eo*FD&M-@fP871;48M36AcHi8nld|3si>A0T40PEc*>skgzNfy6P zgzJg?72yiC*wa1KdQDt?7(=tyW7*ZSwz$H)9z7|jb$jf-t&M0;+whc+;-@Gm?6JFf zu`v+b#{aVvr7XA~7p<)wjFX+27c=iY4*BvOfx-Y)pIGci6MBnR9qcCnaW1r}A9sLK zqSE$&ic&~rE@~i1hFH^3BYA>mH(-?)#No1i6m=jZ%lFAD8`nmz_h{eyVLXBYHQJp* z{Nd^gPKA0SJ3Sb;D@9N&ZG+aJR_ZiS&*uS_$(sQzFjCM^oinQuRDst@2><}5v)^Bs z{y=4b3~zO@s7x*@Nw~1bWejW&{jR?#`_T;Iaf8^ig+0{YJskZ&nMBWqLl@?{aLmPbiaN~V$wJdgHsAq5;sE$S>c@VJXtozTWd)9hzq%kad zqa$wUc#z>YLYP&XfzML=E)0F}J`P%JuIGGEkv%AIMBx-rb5gz!!+bu^KX#~9)MQkv z$k=@K%&uOyzkO}Sv6kn`vv`neGDsxhwNgL(TL0$G;N@qFHn){RqpD+mq0^|*_E`}# zb)d_5ymF#TU#Y+fIqhYcc(_nA*9-UdD;HT|rJFp?ymS=YcLrX(mAC#<{55 z2Isl^))DO&575-Ts46-ZUGym!KKzYz%+Q1(?&Ty9QVR$B+e!p)6Z_k^d3NTiHNhCe zoX$@=m?W%b5AsJa1&=30pA6f*Y)HE(XbRS_j+zlKpow^q>T$i4C4z(k(LJ3{k@YlK zj6LrCbK-c!v?g5>9y?zv#_>^ejVC&Kal)QTg9s7ucV#gZ@jrA!bPuW*+ck-`YyC0m z-byM8ye*d45DEZ;-opDsOH+B@q9?7sB*f*#u{joWzCW$F4e0pGLQgd z3h3%>Lex%dVCV2&N4q1*iAQe4lXckR>zDzQYVyWpUy`des5r`0em`AH*X?(q6X>DL zwF~ngpH3r0&5~26Hq)dLHmFZ~e0ab}w^f3Ab4fI>-=}M3k38>+$j@4LidDQQw_W%t zHyqI2H{ec0q<$&r6et?20Z3>xLy%99{Ymg4c3y(OgXME)n|p>QA#dI^9#ZlK?5!4C zw9Qw@LT^+78CdoHD2n4dQ0OB)2pdV#y$OdKhTRhPs2i-hfdg@+)lG1Yu)BM3$#%d; zXHZgBlCulcQ4P)$YLS)HP_1A=1F656N)X_lE8`$uS|#M#4*|6(#hb;bfh4N9qp1Oi z&}<#^e=MMxU1-hRM9TFik$Z%H|M-Ra>EbfcYxz-DI+CRAC*0gfjIeYSZ%)@_mGegp zJ4^U&6oARgfolEbdaj)*zw`Jy@2ed2YIe#)M8bM13c5@5DsAWLhm^kN^S-lcK&vpI z7vCOpgQfIX!$I=U-)>b#fD?*7L3QMvA6ObfNbRZiTvvD|^R=THj^DNXW5PP}q^B}~ z_(}M#$GeNlAC(?|O!6d#%jCDAKmMPE2^Rad*}dzX;i50cZBvr3?9hT`WJksufrWtt z%ah;&Xi-W8Jz+G6m?<+jJ!4hvp#^HajT$cdxlg;W1HBZCI&+5Xf|*XX%om~Cc@zcT zM*4?WyA4_OJ)fLP{kKOPqa1JF0k{nlyo2jZcp;nPUcRp{s?gF41&dKP7Y8M9+J`_$ z&MSA5)f`3rK|#A69DHSdU60ddHP=kV@5&Ba^mz?3R9^`N(&#eL_6_L30R42ubR>G* zhHA^OLa$fWta5)IOuba@&OsT3j-dl3YM=)u397jbxOFwSjlag{)tX1Re z%Avp@c+7jXp~vo12y_w`y`21^RgYax9pHYr-G6;dc?o{-CovnoweYUO2;k6Ay?Br& zQG?q_jLvaAnZ+&DsA6|O1gUmiDqGO3HyrCuH%x#yQYJ3e3g{dr!?Kw>+ApToRmx!@ z(2c&323;cDw9KsPua$&3z1?~CEdal2w1mr?32pAzgL>thB-Hkd^vQIa7O7D!9wYl` zb!N8&oKM(raK$wS8?>YE8A%elxZQpQ9KM&V4gBBJpB^KL(4rV=Qz#h{)X`9HN4pSN zuzp91VydH|%~)x&(QS5bM^Hi4V{b+82Dd*U_cmOJ(+i7LAW%b5*wO@{Kyeuow7#-I zD6)G}JiT7?oLk#c6P0+mI|yfRrd_xS8{P*Ir^DmbZHuNbD#8g8VAH5{6)Q)|-Y4V7zZ!T0no*c>~OXaw`zW;i2z0pmEnK`ijrXkj8JfKSf!bwJE z{hB^%&b#lS^RzJW<32VtCYzervuWIVCq|quC$}jfFE)p%8|oZBd)c=0dm^Od*Wq`c z^q_;4_^?5aO4WwqLhvu?(cbYW`12!J3@T`EcYI31U$`%*tBAc}a=WnN1%vb0V;$%c z8LGxFo4w>v490w!SD^30tO|ucf7E7sct7toy{}7~c-d8Zl_R?-CF%OjK-hU!m_@@F z&9pSFpeuMSttaWcmSzCLzp(y?W^oF(Lj= zz5A7 z3vI}Tp1E}caYS+#DrRa`-C)o`rB zs#c*{%}9Ev{+?Hg@YM*#KRTmnBqUnuWHm^%{%bpRQXs+I_z2wGMuO z*C5LYKD9~P)n;UVwNo9ZxeHsq^U2$6?j({2{_fL;4StPN|9x>yfdXkB<}8uF<2O4Z z21*r`lT$*#w5f=5&KAs2Nr*I?|HpY_QcF^q_U~(d?Jm?{SPL~(BHU_lfI-P%lo{nSLNq%n2(02gNXS^_=g33 z+$?PJv)iyI+wW3;u8v}0*IT_M%~9SosAL)P$>(mXH7+uyCtKd3lJ8?h@VUtMe`2wK zx73-2Ad=M6XL4kP(yR+n4cL#Sb8DqHoHGgvauWl%4BuvKW@qL@5ejO2 z(Gd~Ju{k}eLy``R@;UIB1a2)FvaWluwBSMs1|gAvAd_FZ)V_8Q%oh2VUaV$snifCN z$l*|zc70gCa#D`46kBKMrmz!s9Ypo-|L~PyD>l6u0pqM9n=h|2J>ep4i2 zPJeXwny)Q2?6xaquPOEENz(fsb3O&dSp+h2_FIHKtd~c?^lDPGwK-VA1^6 zi^+5n>BSILn-Qn3H`bIu2@T09q@lHpmpyPNX=Xk z;8*n<(M`cY{CKqtR z&iu8JJNnfN&I1{z)XNtG#67Ofo!!llNleVlDM(^TlB;B~ub~sGKa5=tu@a{cz$K^- zUsHW@G3DvLs^A2u0H|P=PK4Za(ki85a2v4R+p(mX+Y%q6{KJ33O`gaLrSLws>j}sy zkX_h*-YCRpEXwB+UmQpr9=XbA=&ox&N7(Awd4SQk;82O-(fjfL->LG9a`ec7ecv&kk`=hP4qap;3a>0fm? z-~%&*LBW~ssxleUntDmTVeZFzI5YbBnm1HtDCtYm2)h}wf)e9+zN{ZGoj{8rg;%;V zQI2OO*Lj-J^MfyvNl;RI=TR2#t?x7}9+x@(+OC>|Vr!E=iiT!&l(oOS^ZFm?$nJcTgIZoOU%ucrSyFeXDmm=*<3BP-UZ-B=Yd+Ub# zc7q*)X=XP>ggD>__MUIYwR0p9gzlLnJ<-DyyC(-8D#ELw0{iL3e_4ds$&5;SI<8QU6ejFb8nWwzM@niR$$p`<(tmmWw_q|sSYBm4gjT(IXyUC18 z!;dy{+W)SDmQMEn ztrN|*&#~tR8dk1e9vwOGzDK(Kjb}($uF>z#u{JP8j8f8=zCui_*-o90m{nCYRyK?*8V5*(?kMY1~)Hv*nD(` z!}T6v-_3f!Z%?5cXdM1D&qo>KJ`Q)*4brXX+8+QqOC;b!>*B&`vHX0H8YL+1fNeh# z`;|z@{gZmNyR?*))Cr>Ix#2~6$Uj%+GopXY@JY6iFDd*-%2+_9A(eDhh>;MEJ%OO* zlgjUnM;og0_!dUgPqm~P>20p|>o{1g=e!yiv<~{FMaIXa zjeh5>#WT6ONXK)d<2)m_+;8LpHS01aW7_m*L@B%8@WQz@Wf-)v+O4jSN)Uko1*V;G z4yz-uZUc4V6v;I+M*rMNB(%WQdJXj)N(fi1|N4V9xWGvcMfu8+`2}R}mpqnCC#g1+ z!It6l(3q_gkSy=aNlmTMlNGx8D@@gkuC}%z?%{G<+RiR)$;Q;OxN!=K2oEo!SC<+v z=kWbD66nHkuWD~6WIuR_=aiEfN+o4r*U`WRspd%YLERvI^zcd@KRvNZ>JmwTG{=`C z6!TDXx%T|t4fO%>P>a?#_MozDa-849Q0k1*hKYwTF6 z+fuRQdv)U;HG`#Ll*I*Q?V%G8&;xj4?vy7TeHeoPRpvWeRJIz7O-=vHi61t8lRc4sm&`ZV z(LSvCZiIQI&5r~Ytr8ZXBKu3;!=is4Ct_IOv0MB7L2wc@iTi+o6F9`w?mgl2Xg&Ao z^0t$%GGi{OkpWMJ77k+Fnpig^;nu#mx5!?QX@zA>EZg=khfWUfF5WEQb zrUCq~ByA!SLxZKthkbJCg7-<{?!g3^rfu#x32}8ys znbJ!QHWe#~PBd;h9hCL^jOe!68A{B)y5>tY$>Ev-xO=1N;t&uJM(?@XtnN?5TRS^& zMYc?YOPonceF!3uO-ZsT0><52ok_SI+bIkMR)=|jL{M7eBdh+XX2C?4;+FfWw)ut8 zB<4tx3iifUVfl2er?34_*Vt;l#zYgqb#YQ7OdhJ>aC}E-KAqF35~vVMEdtT9z0bs! zdz;PYswk60PaA#Y&Di1kLP0_;EQ7^B&Smo)!H{}c%67(j*|bjKI({?C4Q$IB=TLdzT$2=;LRu>0pmi;6z_)7Y&BF$*L$2fkGpEXk#S_x(-ZVJEWe(NX#QT zaE%7l@}zu*;{M2apb{}GGOt-+j!5B-_M+nnXn$&Ejgol5tp@m1!o5V1J^6QWc=+P( z-}{VjN;a#_4E~e-okx>e9X9I_+sx+6{GCqWJ<}z(T+HvL30i;XX#Inh%~hMvyv6=1 z49sVW;#H5ideh@%cz>FI(nSM}iyTiNQ_7uY2)(`4k2bA*i&P|L4uo6)L{*8tO6t zdfACK>SwQng@y@#rjReZrE-;~1%!BUSH=-^Z1%*=2QKSgOff_Paq*z^SxNvW%=4f$ zWP%_iVZ-5D4ypWAXq?bGna38v6>x6BX~Wd{l;$a4A{Ktqc>BClj>lMrpTJl6JX=MPJJxy=HDL_kD)8`wJA@nyWpW3Y3)5(_8msgSE?yH2m*tFiLrbM_Hn2bBr5oHxe$P&ZVfZz8sccH& zpP&EJMputkothySaW9Fl<88Wdp5)PH(rg!D<1XaZLb~6nR8@#CXNc>3Qp{!R9Tt^X z`pQVUcDRT9r3CUO^q*If7c*3z*`57i@E1LfMKR{Q-cp|WDchCjWt8F2Om2J81_tq~ zox1l8d*;Sa=5o_`UckTRRH|I&T|RHCn$F%ZIa|fZYPn@r0ml3oa~L{-xalWo&H7B9 zlY)%Ly<9}-3FHZUqcyAf5-vKqHzI5E`E>p6HgH0V@RJw`925-f(1Ucn@m(S@1xvso z!1`CZP=47U*K@U?m;<#M^;j+HH@e!}tF7Zg&BI1ranhrQJZAD8pbCYTJd5sj#?CPu z$#TH)xH}k!f*BQ7VER_;#U}X3fNd1OcJOk4@t|iUJ1f=z}k0t_u+0djiZ7~ z2Q0XGODRZNZyO!!w7Ek>YABqo{9do;rRS;}5;S_PsUOnSDi-M_Nx7McME~pLdbs#I z$|GJLlwsp+&}OZFxp-EZ>9oJTlXc`sWd*w1bb~;p3(B?7KqmUi#Jb-1Y(c7|g-Ydj zFfd9h>CYl6o2FIo;>d;Wzk2syA1)31sNosLhPx-X0__A-(t=518IbU9=okq6{QQKK zg+PAbpdza~B$%K{&_Qvoj1*eoqoc1E)GNgoGlN6|F5K7l=a{Z9#HFCFVt;>hL!Ov0 zk2`&Y3pQ$mR2K(LX2-Va!QzQmWlB*pz>cT-CZ(3BRaFSvU!MMj|Ux51jD z1-)P}w=93lK-=s)!$~j`RlVJ1T5tJ)SR6@2K(dM!wk=>V9R=f%_91F2_I6QI=2GT=gk?>e_k2sFW zGaAnW!e2s|V^4?wSTaXus}A4@z5+%&T8Ta!utvJ9`9hOkm+6{6>S!c3`sV8lKzDe_ z^iAm_Y(OJZ5KQBTJ_H$*)X7M)K}OfR3XA0=OTmclQsM_Qo$8_m5}9_MmusupsF)w@ z(cSN7-nhIT0kh_FipfCZl~IpmOtxY=V@zSJNC!23?&`cO)4FoeSd7 zVYmt6r~TsUe*?3DXpA$YlI=?$Q}J7|77|))Hg^5oWvGF3P>(no4KYG_8T<8zJl zI9b*vYXGe3)u)r%2;0h3Ur>&8v-#@J4lM?jW?QaD?_rI!hZzMPz!8*Z51TqYHr3|s zy6AZjmifyK-aou?`Z@#U3fBOS*K1ktsNn&@Kd9sA!76RlfPyskQ2-sxIZ zWg4)~*WuuqsZg4~Ol3@$lkFpor(Cp$fDgw z%%C@1YhZ9_=f0yV)IvUGhC%OjzLta-hB_Ed?ZJs4sm?D@NHBWJA3x;rolqzeY#5E6skdR%v-(6OUg?K*G$1R;dj|EN;}EuY4s z_cw=`qbyk++>+fpp;ppHXI~KXAb*7uAd(beob(*PpTRxV`L=ulO{hTJq5FztBk@MspFUXhX-PR0aC)b{hEvYSEX z?+aooQEGQe%;7}8`{_!J z2qpKV$<|WpL(~mj<9e+~FQ5ayqTl07065@c%tCVM)Bq|lnvQGwAf)c>?;t+cCx0FPhr@?nv7EzW`@Yz<(UPSI<{bC zx9kk*9dw%DEwF1ixDZob=H#I5o}13L#+8H$dP~$OgOSV_L-8d126V*(j@bH3AB%M`o8zPOLDR&Pe@FveMKQd8L&He? z_8-~e{>tiMU`ik#)lx8t)^mWOkK~UH_|x$U`Xdd)Q0?{a#3*Mj#f=09#I+`bGaOC9 z^m?G+Nq?`Z*TTQEtKXGtpM2`t5A03@a3O;xGP!TS)c&Ca&aEXwtw@~cGsf6kV7V?x z%4nGRlO&OL;`%IqWe&%5+r*$J6@8!~+YTnHOHpU^%HYXh*n?y?H#EG(lSHH!3ft-> zF^4WE+0qLSjxk!^`S~Xgfy?^O5io&MVQD-9MR(5R34PqSIHrPMSqj|WeNE?@4QLNj$u1;+9G^BF?10a?`y*y2=%K~ zp0p3B>6XwM&D#$WdV0mx6HyOn=`#5$>HA+C05!0y+}7>SKHbS?Z`78xC-@BO=!~r= zZJLRb_6~yM9H^#vTqjnKazZWcRR@vyF;(4c0F(*ygKG}T4b>aRTl|su=R1vLM=K0> zzf%6L-@DVyKL|ri%wNyF-emPJdZV^1DP2c+bN$jjyrui&@AJ$5f;DK^Xf<6Y+dVZ~ zXE-<*144jk?jN`WknfX3$Y~$nz9-8T5B$%|e*m;OFZzy(a!=vc3CjEnX{nmalv^z?+sQulhKX+8u5L-18qoGTa zy3a9Sv<5`Ok$%u)Z!`1zsdaPFg9kyyDX^FA8&lJ-&1Hq zhiA)dXIj@fu7@{|>9FH(`edXOzE{*eloh+>nRtAcaPC8V+*5aVAAxK_+6rmC9*E5} zDL?;KE5gx&p*lQ5i4naRBja3Jrj8Z*A=ntVZ0Sc`1eGSf7rMwXt=TRVgDv*Q%4eHH zkz+QV?>jZx!XtP;b)*qh*Y5I7yr3#AbAm^03NSOtGVtdAe1@km-R9le{(ntw0?G1gtA7B1uuF08m zotd23d+oK?+Ms6UsPZD4QY;ZH3Muy-ymbhX;Hm;M4{rawHUmL>OM06r6ba*Db5+9_ z6Hu+rMwFcqdc`Ua1r%1z6@AYY^oB`By>|#OV#51cCj9@jekkODHB*eQ69~GNvSx|4 zvd0&itCr>mOOj zVd>=#%NM;`^&U{&AVP!KY&S4PXz)We|GsD5Ce6TV`0}Rutu#mVCMzL^9-`h8gF+G+ z(VYr4SAIcAc?e?#I0+)B*mk|EA^-750WAYq^QoUA7CEf)(l$T~M;A#iHO?Y+?vxmF zuTrdzy7R)C5TYSx)NtFF3$QBYEKF`wK%HQs_uCXa)w3+qR?LARvW}eM?*UH)S56I&EJ)V=y?BMNDS^4U)x{=Wwh?ay!!Ct=uj!12`W3-4>3QzT8B62Q zJO~u7Iv4=(%n{04#7NIdvTTbnOgK<0;!Jq$qQz5{RY$maCiW(bqB+K04dcXd*$)lE zC8yek(HD_JGDuTap`CCj_p=RMI>>-2HBpK|?82R&_3EC2h*UsU8p}g^LLplrdLQl} zW~bTU%&hl8EkC{?fvlm;d)gDWdiMV4oKYVOCY=(Pmym6h-hyF_M%=@)Sdz-(e zmfmjC+F<)zjmht~L8Ie5TaI>2L4kpue0p)*vSiF1a81z$=`_I%0TQ+00ax%XYg0 zSe7`aJh$H4YO8J9k;`jX39u9Eu3e>gnW;kvg=YJSK6pt{H=Yq&U4hF#d|l7 zDLIqL7Vi{&6L9Gz&#ZBW?F^fC0Pi(g{ZW|w4mQ;lbdQL-h4JmK+2$!_3d?uc%o*5B zJF;eKITGW>2HCJ#*O4TCxITh%bu$0igfci-PeEt6ykIwF_3F(gbk-Af`7D`yVBaiu z!@En;w;C48an%8TEMF}_`-#Za*0=J^#v>G51R{o6{ro(SKq|~-a5s$lGogA_m4Xqw zZf{tzZzNis_Y*f6pLNN_;jAkC=nc5l&5n+TyeaW=cy1^J>z)oBx?X|`$$aw7bDiO4 zn}8zd&51=4VPI`BzjMu;NhD%GeK=2Am@Kt5zP)TZoosg@FI#B{s(iGK&ic@#?JT~o zm|EX_b8|a{6{J%+RkBu#6{syo^cAWKIST|JnruU` z(0i(u@Uvk7=cKstJwLYn=L{N+kL@R!I4e%VvYW5JCWJydQ(#yrL$w4;#cEGN35eHUh9?7V3#o_grlU`(ZAyrZp1z)T=e<-_z$rU*XPQ z!Vl38Jbq;q!X1W~h0E|gCVHvK+_ZpJeda#2U;q~m#`{b7{RVi$jdiMkF{l>L4&dXF z^vBTgh~e)bz2O$L9sZfFpK^XI zZcZq9K1*bTMbMhv@4`NpP1#7|(0qlTGL$5d^TFj#eg4GsU4%c6C9rxV%!RUS*NKqz zdPnJrtxolH+e@9|Q+6r;WCaNOo@dBj3ucV}reDEuj)&@o9tySCCQWvKixBV2XlYu8_LkT}M z?@PS`(WO#}kJI^;v95EObc!q>FX>glGaO(?y7-KP`orztD>yb*OrlrXj8R>DqdU3Z zdGH@H_NsyUrwoB}1Eeb5qEGeh!IWDIp5+V8-w(X*8A!V>owuZxl`Yd~ zqNBkvH(i|0-5|WodRYsre^s)q_yi~ErucMP#RMJvx3`0)v`C&8Hyj0-#xlx@F+3^C z#mb2a*7WZ2%k=F0CT zu{uwIYoADcQ6CwivsTBzz`!%EeD6m=it?=l78EZ_85O=;BzKg^WUf$KU*F>;ph-zR zggNK8B*EpCm4Be?icBy)oxzhn)$tk35$dhd6^EQS(UaZZQu9-0=X&64Q~ zL&J>eXNg(6WwuV0Ot_lqR(#Z;ulx3aXG`FSgJaK03!9++EX!oAJr`pdU#d_bLhZLl z3*8Qzpc|^CY}X7`He$nZnOW@tJpUOWV=>6C=cLoCw0 z;fFZWx>&lfCriU$LQaof)ibU%FNGrC(MUU+o#prPNa$X>Ge3mW{OorpZEFnpIW#2j z1sU{kCR(lk%QNqqZVFFq_d(I@WrMRjCabkc%8c*1PR~1Wl~-h#l9?HO-9du|u!`Q- z+)@3TjB-)r9AcxfB4E{`JlCMS?n;*zOnUoG>rhFyS1*nh+7&qm2tb7MM~`>zBYJ44 ziLz*_R){{hW^JgSc*r_O5pcz^77zr1D0qq48H)|pgo1-4q!_NJ_PUCY!{^-;Okz2T zQZH&sgu=v9kjP+ZbjPQO2xaO&8LOsPUA={(R)%I(FlcP-lq7;h4nIY|Wn6*(d#@Tf z;fZ$ww>AFIGcj=!KfALq%1its{6eJf#9a*4dZO2xel>Iw6r`*eDA65!lSwvhLo??! ztKNI1XK@UFfPFvesr88uJ9^G#`1(MIVK4*^HDyMRM2wO23cJ(76m{I~S?tEdBeA z5K3aN?+mRn&dj#~H%GX4Yn@SVmfU%Pkb-TEyb~ z%Q6hE<*SDY5}{L65j7>D5< z?rimu+{=ET59r0Lj~$x)S<3F!WW5OV-2GX;d=tEnhv`K+iuu~=6zSG{qxlolaZ|0& zTu#tr`x<-N1ec`7Q8o853j55IbC=tGOm)NVvLf-wO&`B>$4zAfp(I|~X`+SPf7Btg z=`Q$X2>q$VzYs!VLK%wGiKPwFX88mnjy%@d7|G%4sC)Bz8Tom`$Tu^i>B6R1j2?E2 z9q27kcocG=3uanHAiQdEI_I52E7u9oR$2!Fqp&UEelJOwFopl#rTrZr)c1SZAZmf_ zGnY-3q~J;3Db&N;9#nG)POe($X!@1m8`?A6_6iC;31~NYlb@bKox~vX&|_uU^1R-m zbZgCTK+Lf0au*d5X|+@tz-_N1+dp_R8m{yOtACuiMnsN}ZRe4XRjP8sIgB->`}>j5auHuJ2u3i~|TKKo*FC{>M`; zWkh!)I{oC2;H>aVamfi25$n$?y#-$4M|Cd0RAQuP9z_ej+s51=NUo8R6|35o_Fip4MGin>`l?)bG@!EEb@y(670euWgdy zp65fUYYK0T{JB+KjORoolS&-12%aC&W6chN$;#6(Ap|ZY)p1u0N6srtvX@Vuu(Jf5 znPI5dc~jV}@8;}Ee+~6hX)%>z3HBb}3i6PNh#`XD)AugzA`@Tv@W;@K;2%grc>&di zo!~dd?8Xjcd~KbdJEpJd3rR;=(?<5E%h~KR`G6n*_S!2-x^uDjhKrhKKG)T>emTpN z&s5Brypema(Y=*BMHJZIC7P&)2X7;`o8h(95wiMYG8|t1O*LND zr>}L?8C~47feG$$C+hu>g#ENOzL#c#LXLynLy8_1?u66(Iw{1Jj*E#j;dlL)C=yIa z+o27%w1HnQ4pL3%E6{`_5n)7CAFCT$L75GF@j(P|$u^&nY{>7+1=$=AE4lBJ4T+Li zpY|6mzf*{OQ+mLgZ!{Y-v-6~KGlBA`XgM60Z=*|@cm1wqIcI^y}D%ENgQJmD7Ry@ouH%EMdXl)_*HAlh)w z&saAk3+E3w&l4X_3jG&0x+5Ca18rla6>XWYdnzHnE$d?YMM@Imn4_MsQ- zK)~vf{dAw04_FgmUZH-t4Q-z$K&twu0_F!_X-8O^?bUI)c-$?mSo>F-XuF;F%``fy zKq9{**i1fzB3jReH|eW#Hm3m6M9$EgX>(kulQ%p%%$2s+=s|5bC5Jn%iq;20??h(* za>OCPEe@vs{T{v?k zqCmZQJV(#Hyq?uCOpzqa@u@*j6hn(nGw@4KUaS813WwO15~Uoa_Ce3Cr=S5~@3j(+ zgj{}du+MMw@%}w0dgWCG3_6>_g#pizotq&p=krCNK$Z6=G>-BM`akvtJP>Ag zn$g~-bQ#UruUjjQAG+^eR+j0TCB-|X>J~rqoQc5O8cZX5=ofj?X)u_CF;QiEVG}yC z_zY*uPuXp z%|nfK&!RI0w%1fjwln%bEk|CAuRHH-W3^Uk$h7t9N4F#4w_&l_aZ@_!K%_{3es%_! z4tRcx#)0aWNBjWle58&dF@ftzsNp@~xZ%|JZ>cYhh~@~rV#g2$I#BZ=gPv3cJD#r1 z*iumI=%llPY$LXuv5_!Rne_DeMLC5Z?|Z0D7}&*Tt97IN$Aa)A4#M^guZvzdT|_UW zur%C5`(D&KFpbG5;_xcG(VTv+yJkV=)vXGSogu3v`o+i2viQXRo>7fIq;}1qx-4}# zltqDf6xY6er1^eybEg@C*hN)CzB{NbDvF-w{z`t5-Y57QFzdop8A4@lXVRx1T zrC&_R)k+3G9{GfI$puCAnEf_pE~I__Q;rC_nJ1trf0Vshb%#7F*8QrsZ^$ujN^H}S z1|le@Maz55+RY;kOMQPmeuCHz<1~VXxkKW;!~a??d98ql*Z%cbL_9?G~KYW z4n_FDY-O(x#P-;33lnVgtFDIa?fP&Q`QbVR;XbYBo7RYUl=)nAtKNDR+37dws24kf zm%`!U3g~HR(obiW#7vJ>T=7?>{ALmQwiAw4Ds%WN_QfM)Ss3kSmIjP9G1{01TxR86 z{E$zqP>NfQa2n+4o|l4=2+E6`N`7BokxNN-v{58URdPm*B`2j!x1Z>ZpYtx1hXqv; z0CGc2D^b>KeSlewG$&X^c>f}!C>M#sJ&Ge=g1C>4;hM^u> z-!HlpO{20Kl)Us8jM{Yo%;2;JU5np8J@*?c!z@NE!d!PfT`V;gE*tIO1RQVWfWXY( zeA=(l`07SwPrUqY!H#Li$IQ{Go2{|-nT;C;m1uIY#nzj$xG7oTMmu-zRkfWFMZ&B# zDLKRlRTOzJ|ENVYi^slUZN=OG8WST?NZ?lU!+Uh2MYu3W+2sv9d`sRZb?l^iPQ1)) z#G=_%-q9P@1K{=aKSU5A;}X6x(k>3^IT$Ae@M22J)a22iu(onEhm?GKhk~CG^WQt| zY7Z!$je&0Z_qZG64`!@)W&+q}$-!UZ%JM`x%650{Tu)%OAxS7-S;N$yw!Gy-MwkLdKjx zi)@-6@b+zur3v5QE8Ni!rgvWzrmg}&3%#LA7WNq@_FZulc)(PAdNcY>3~WqI3)j1) zT}w2)zMJ&oah#~c456t`&a|k$DO*LA#_aj*%$5Wd??HfT~xRlnrX^rK03JTYrESOJjE$$7>=b23QsTbEo^vnMpe~D;!G~Kutc>22 zKflynMDS4BmSc6Ys177n5#z(fd z35PU0g?3Lo2OoR?sRKhRm0enDg&m^KdSW-wmYIan9#uWR>BPqg+_*+&0@!irWKR`z zsFApQKWjWiGz+K;Gr{x1x4bXZ;I**Ns)xrKHfATS?Na`~VuY8;Tl;)a{!&8yFUGNz zqOVWnmq9g@`EVQw2=DyxFdfMHI6%wiTREamFavmy#RCM7Keg@1qZA5SZTv^wSmf)W9Cl_$7ny5&)S+SAtZZ^NDZ$9X+D7Vr8cSO$+%xX(jF zNld5+RpS9+|Cx~vjRM@Ca4fh#_;Mk{bXq%XIkFLKI95HE%;Wj7C z-5gVyh}Xvo0A}VuNrYD_&WL1(Piw0ShedeOA3fOGCbT}YJS7F7pD^AL`&`{2s8I*P zWNAq3Cx48+4Wyt;L&$h^lBQ<6x*MbY(34f)m`lW)8J(27dbCPtOlY@>@pW^ky};=> zLIZWvTSILaLJx_SB`|N#E&W0kw#2VHwrSxb(*b$29;x@n`c3n4ZC7Q()CO!5Zvgs$>eq5x*T*(tR`-Po_ZETN`eqjJungN)b zB$V_zu=1{L%4n+`whqFDUwMEGqJhdW+K>SS@vBja%g)&p&&Vy{sDK*=%psdItwk{Z z0@)dJ)^Upy*UqBi(fFLg^-5$_MkPuf)q=B|odwg$y<*e6QLXek0$&D-|42~Y0u#mP zHFqp++FB16!UsQpACq|=45LB_Xa~O4w0u8*s}6wkjcxM<({oRo^w2ke{s<^<5fA|f zX$Ygf16{X*05Jq4pnq2thFQ)+r(@7+GJV}?AOeaAJ))(thuq}lWp%9XTd+6+l0IEm z_M_G*H4fsxpUjH}DAljonXW#usv-Ux57y z+6(#LrKPCitgjxTK<1~X82`@V0GMS6yI*mnP^SKy9xz0#WWmMq|39I!7uH`Bt@yvm zicuh?f4Jx0hNr;)4up&OTYx?#p921U&BvHHkbEz?P4zN`w@8Q?KEdFAntwA4rau8S XSqWN3&&@_4z(4ZRDpFMvCV~G45VaSG literal 0 HcmV?d00001 From c53d8cc44e1dc9cd3afc88f2c5ac14c4c5eb37b2 Mon Sep 17 00:00:00 2001 From: Gajanan Bhat Date: Mon, 14 Feb 2022 16:06:03 +0530 Subject: [PATCH 162/628] Added duplicate PDB check --- apis/database/v1alpha1/cdb_webhook.go | 2 +- controllers/database/pdb_controller.go | 52 ++++++++++++++++++++++++++ main.go | 14 +++++++ 3 files changed, 67 insertions(+), 1 deletion(-) diff --git a/apis/database/v1alpha1/cdb_webhook.go b/apis/database/v1alpha1/cdb_webhook.go index 49922a93..756b5b07 100644 --- a/apis/database/v1alpha1/cdb_webhook.go +++ b/apis/database/v1alpha1/cdb_webhook.go @@ -179,7 +179,7 @@ func (r *CDB) ValidateUpdate(old runtime.Object) error { } if !strings.EqualFold(oldCDB.Spec.ServiceName, r.Spec.ServiceName) { allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec").Child("replics"), "cannot be changed")) + field.Forbidden(field.NewPath("spec").Child("replicas"), "cannot be changed")) } if len(allErrs) == 0 { diff --git a/controllers/database/pdb_controller.go b/controllers/database/pdb_controller.go index b96b3f4a..11535681 100644 --- a/controllers/database/pdb_controller.go +++ b/controllers/database/pdb_controller.go @@ -173,6 +173,14 @@ func (r *PDBReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.R return requeueY, nil } + // Check for Duplicate PDB + if !pdb.Status.Status { + err = r.checkDuplicatePDB(ctx, req, pdb) + if err != nil { + return requeueN, nil + } + } + action := strings.ToUpper(pdb.Spec.Action) if pdb.Status.Phase == pdbPhaseReady { @@ -258,6 +266,49 @@ func (r *PDBReconciler) validatePhase(ctx context.Context, req ctrl.Request, pdb log.Info("Validation complete") } +/**************************************************************** + * Check for Duplicate PDB. Same PDB name on the same CDB resource. + /***************************************************************/ +func (r *PDBReconciler) checkDuplicatePDB(ctx context.Context, req ctrl.Request, pdb *dbapi.PDB) error { + + log := r.Log.WithValues("checkDuplicatePDB", req.NamespacedName) + + // Name of the CDB CR that holds the ORDS container + cdbResName := pdb.Spec.CDBResName + //cdbame := pdb.Spec.CDBName + + // Name of the PDB resource + pdbResName := pdb.Spec.PDBName + + pdbList := &dbapi.PDBList{} + + listOpts := []client.ListOption{client.InNamespace(req.Namespace), client.MatchingFields{"spec.pdbName": pdbResName}} + + // List retrieves list of objects for a given namespace and list options. + err := r.List(ctx, pdbList, listOpts...) + if err != nil { + log.Info("Failed to list pdbs", "Namespace", req.Namespace, "Error", err) + return err + } + + if len(pdbList.Items) == 0 { + log.Info("No pdbs found for PDBName: "+pdbResName, "CDBResName", cdbResName) + return nil + } + + for _, p := range pdbList.Items { + log.Info("Found PDB: " + p.Name) + if (p.Name != pdb.Name) && (p.Spec.CDBResName == cdbResName) { + log.Info("Duplicate PDB found") + pdb.Status.Msg = "PDB Resource already exists" + pdb.Status.Status = false + pdb.Status.Phase = pdbPhaseFail + return errors.New("Duplicate PDB found") + } + } + return nil +} + /**************************************************************** * Get the Custom Resource for the CDB mentioned in the PDB Spec /***************************************************************/ @@ -854,6 +905,7 @@ func (r *PDBReconciler) managePDBDeletion(ctx context.Context, req ctrl.Request, isPDBMarkedToBeDeleted := pdb.GetDeletionTimestamp() != nil if isPDBMarkedToBeDeleted { log.Info("Marked to be deleted") + pdb.Status.Phase = pdbPhaseDelete if controllerutil.ContainsFinalizer(pdb, PDBFinalizer) { // Remove PDBFinalizer. Once all finalizers have been // removed, the object will be deleted. diff --git a/main.go b/main.go index dfdeef12..f1715965 100644 --- a/main.go +++ b/main.go @@ -39,6 +39,7 @@ package main import ( + "context" "flag" "os" "strconv" @@ -49,6 +50,7 @@ import ( clientgoscheme "k8s.io/client-go/kubernetes/scheme" _ "k8s.io/client-go/plugin/pkg/client/auth/gcp" ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/controller-runtime/pkg/log/zap" databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" @@ -91,6 +93,9 @@ func main() { os.Exit(1) } + // Get Cache + cache := mgr.GetCache() + if err = (&databasecontroller.AutonomousDatabaseReconciler{ KubeClient: mgr.GetClient(), Log: ctrl.Log.WithName("controllers").WithName("database").WithName("AutonomousDatabase"), @@ -185,6 +190,15 @@ func main() { // +kubebuilder:scaffold:builder + // Add index for PDB CR to enable mgr to cache PDBs + indexFunc := func(obj client.Object) []string { + return []string{obj.(*databasev1alpha1.PDB).Spec.PDBName} + } + if err = cache.IndexField(context.TODO(), &databasev1alpha1.PDB{}, "spec.pdbName", indexFunc); err != nil { + setupLog.Error(err, "unable to create index function for ", "controller", "PDB") + os.Exit(1) + } + setupLog.Info("starting manager") if err := mgr.Start(ctrl.SetupSignalHandler()); err != nil { setupLog.Error(err, "problem running manager") From 11848060c42a13d4eb9cf9dcb6897f68ab26b962 Mon Sep 17 00:00:00 2001 From: Ting-Lan Wang <61528376+harayuanwang@users.noreply.github.com> Date: Tue, 15 Feb 2022 07:21:31 -0800 Subject: [PATCH 163/628] Update ADB_PREREQUISITES.md Fixed typos --- docs/adb/ADB_PREREQUISITES.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/adb/ADB_PREREQUISITES.md b/docs/adb/ADB_PREREQUISITES.md index 56c47445..43c8ad6c 100644 --- a/docs/adb/ADB_PREREQUISITES.md +++ b/docs/adb/ADB_PREREQUISITES.md @@ -59,7 +59,7 @@ Instance principal authorization enables the operator to make API calls from an 2. In the **Matching Rules** section, write rules the to include the OKE nodes in the dynamic group. - Example 1 : enables **all** the resources, including OKE nodes in the compartment, to be members of the dynamic group. + Example 1 : enables **all** the instances, including OKE nodes in the compartment, to be members of the dynamic group. ```sh All {instance.compartment.id = ''} @@ -67,10 +67,10 @@ Instance principal authorization enables the operator to make API calls from an ![instance-principal-2](/images/adb/instance-principal-2.png) - Example 2 : enables the OKE nodes in the compartment, to be members of the dynamic group. + Example 2 : enables the specific OKE nodes in the compartment, to be members of the dynamic group. ```sh - Any {instance.compartment.id = '', instance.compartment.id = '', instance.compartment.id = ''} + Any {instance.id = '', instance.id = '', instance.id = ''} ``` ![instance-principal-3](/images/adb/instance-principal-3.png) From b4f582903f893dd373f0ecfb69ab68acda99a975 Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Wed, 16 Feb 2022 09:53:05 +0530 Subject: [PATCH 164/628] Added support for zero replicas --- apis/database/v1alpha1/singleinstancedatabase_types.go | 1 - apis/database/v1alpha1/singleinstancedatabase_webhook.go | 2 +- apis/database/v1alpha1/zz_generated.deepcopy.go | 1 + .../bases/database.oracle.com_singleinstancedatabases.yaml | 1 - controllers/database/singleinstancedatabase_controller.go | 4 ++-- oracle-database-operator.yaml | 1 - 6 files changed, 4 insertions(+), 6 deletions(-) diff --git a/apis/database/v1alpha1/singleinstancedatabase_types.go b/apis/database/v1alpha1/singleinstancedatabase_types.go index 5415646d..6262a9d9 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_types.go +++ b/apis/database/v1alpha1/singleinstancedatabase_types.go @@ -70,7 +70,6 @@ type SingleInstanceDatabaseSpec struct { ServiceAccountName string `json:"serviceAccountName,omitempty"` // +k8s:openapi-gen=true - // +kubebuilder:validation:Minimum=1 Replicas int `json:"replicas,omitempty"` NodeSelector map[string]string `json:"nodeSelector,omitempty"` diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index 66d4767c..ceecfea9 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -76,7 +76,7 @@ func (r *SingleInstanceDatabase) Default() { } // Pre-built db should have 1 replica only - if r.Spec.Persistence.AccessMode == "" { + if r.Spec.Persistence.AccessMode == "" && r.Spec.Replicas > 1 { r.Spec.Replicas = 1 } diff --git a/apis/database/v1alpha1/zz_generated.deepcopy.go b/apis/database/v1alpha1/zz_generated.deepcopy.go index 3e6716ac..42bfd573 100644 --- a/apis/database/v1alpha1/zz_generated.deepcopy.go +++ b/apis/database/v1alpha1/zz_generated.deepcopy.go @@ -1,3 +1,4 @@ +//go:build !ignore_autogenerated // +build !ignore_autogenerated /* diff --git a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml index f1ee3a4e..86aaded3 100644 --- a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml +++ b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml @@ -142,7 +142,6 @@ spec: readinessCheckPeriod: type: integer replicas: - minimum: 1 type: integer serviceAccountName: type: string diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 68849a35..323de3eb 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -320,11 +320,11 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab } // If Express Edition , Ensure Replicas=1 - if m.Spec.Edition == "express" && m.Spec.Replicas != 1 { + if m.Spec.Edition == "express" && m.Spec.Replicas > 1 { eventMsgs = append(eventMsgs, "XE supports only one replica") } // If Block Volume , Ensure Replicas=1 - if m.Spec.Persistence.AccessMode == "ReadWriteOnce" && m.Spec.Replicas != 1 { + if m.Spec.Persistence.AccessMode == "ReadWriteOnce" && m.Spec.Replicas > 1 { eventMsgs = append(eventMsgs, "accessMode ReadWriteOnce supports only one replica") } if m.Status.Sid != "" && !strings.EqualFold(m.Spec.Sid, m.Status.Sid) { diff --git a/oracle-database-operator.yaml b/oracle-database-operator.yaml index 48117e04..0dbb3ab8 100644 --- a/oracle-database-operator.yaml +++ b/oracle-database-operator.yaml @@ -890,7 +890,6 @@ spec: readinessCheckPeriod: type: integer replicas: - minimum: 1 type: integer serviceAccountName: type: string From 63035697f67dd6d6ecb46ceb88787acd9ef3001c Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Wed, 16 Feb 2022 10:24:53 +0530 Subject: [PATCH 165/628] updated go version --- controllers/database/singleinstancedatabase_controller.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 323de3eb..391d282f 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1356,7 +1356,7 @@ func (r *SingleInstanceDatabaseReconciler) deletePods(ctx context.Context, req c noDeleted := 0 for _, availablePod := range available { - if readyPod.Name == availablePod.Name { + if readyPod.Name == availablePod.Name && m.Spec.Replicas != 0 { continue } if replicasRequired == (len(available) - noDeleted) { From f6b6b3cca35ebbc867b22ca0d7a3f9f39b75d155 Mon Sep 17 00:00:00 2001 From: "omar.salazar@oracle.com" Date: Sat, 19 Feb 2022 01:38:46 +0000 Subject: [PATCH 166/628] Add network access tests --- ...autonomousdatabase_controller_bind_test.go | 10 ++ test/e2e/behavior/shared_behaviors.go | 125 +++++++++++++++++- test/e2e/resource/test_config.yaml | 6 +- test/e2e/suite_test.go | 6 + test/e2e/util/util.go | 2 + 5 files changed, 145 insertions(+), 4 deletions(-) diff --git a/test/e2e/autonomousdatabase_controller_bind_test.go b/test/e2e/autonomousdatabase_controller_bind_test.go index 3868399b..56705060 100644 --- a/test/e2e/autonomousdatabase_controller_bind_test.go +++ b/test/e2e/autonomousdatabase_controller_bind_test.go @@ -138,6 +138,16 @@ var _ = Describe("test ADB binding with hardLink=true", func() { It("Should restart ADB", e2ebehavior.UpdateAndAssertState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateAvailable)) + It("Should change to RESTRICTED network access", e2ebehavior.TestNetworkAccessRestricted(&k8sClient, &dbClient, &adbLookupKey, false)) + + It("Should change isMTLSConnectionRequired to false", e2ebehavior.TestNetworkAccessRestricted(&k8sClient, &dbClient, &adbLookupKey, false)) + + It("Should should change to PRIVATE network access", e2ebehavior.TestNetworkAccessPrivate(&k8sClient, &dbClient, &adbLookupKey, false, &SharedSubnetOCID, &SharedNsgOCID)) + + It("Should change isMTLSConnectionRequired to true when network access is PRIVATE", e2ebehavior.TestNetworkAccessPrivate(&k8sClient, &dbClient, &adbLookupKey, true, &SharedSubnetOCID, &SharedNsgOCID)) + + It("Should return to PUBLIC access type", e2ebehavior.TestNetworkAccessPublic(&k8sClient, &dbClient, &adbLookupKey)) + It("Should delete the resource in cluster but not terminate the database in OCI", e2ebehavior.AssertSoftLinkDelete(&k8sClient, &adbLookupKey)) }) diff --git a/test/e2e/behavior/shared_behaviors.go b/test/e2e/behavior/shared_behaviors.go index 652602f9..69b826d2 100644 --- a/test/e2e/behavior/shared_behaviors.go +++ b/test/e2e/behavior/shared_behaviors.go @@ -288,6 +288,7 @@ func AssertADBDetails(k8sClient *client.Client, dbClient *database.DatabaseClien derefDBClient := *dbClient + expectedADBDetails := expectedADB.Spec.Details Eventually(func() (bool, error) { // Fetch the ADB from OCI when it's in AVAILABLE state, and retry if its attributes doesn't match the new ADB's attributes retryPolicy := e2eutil.NewLifecycleStateRetryPolicy(database.AutonomousDatabaseLifecycleStateAvailable) @@ -296,7 +297,60 @@ func AssertADBDetails(k8sClient *client.Client, dbClient *database.DatabaseClien return false, err } - expectedADBDetails := expectedADB.Spec.Details + debug := false + if debug { + if !compareString(expectedADBDetails.AutonomousDatabaseOCID, resp.AutonomousDatabase.Id) { + fmt.Fprintf(GinkgoWriter, "Expected OCID: %v\nGot: %v\n", expectedADBDetails.AutonomousDatabaseOCID, resp.AutonomousDatabase.Id) + } + if !compareString(expectedADBDetails.CompartmentOCID, resp.AutonomousDatabase.CompartmentId) { + fmt.Fprintf(GinkgoWriter, "Expected CompartmentOCID: %v\nGot: %v\n", expectedADBDetails.CompartmentOCID, resp.CompartmentId) + } + if !compareString(expectedADBDetails.DisplayName, resp.AutonomousDatabase.DisplayName) { + fmt.Fprintf(GinkgoWriter, "Expected DisplayName: %v\nGot: %v\n", expectedADBDetails.DisplayName, resp.AutonomousDatabase.DisplayName) + } + if !compareString(expectedADBDetails.DbName, resp.AutonomousDatabase.DbName) { + fmt.Fprintf(GinkgoWriter, "Expected DbName: %v\nGot:%v\n", expectedADBDetails.DbName, resp.AutonomousDatabase.DbName) + } + if expectedADBDetails.DbWorkload != resp.AutonomousDatabase.DbWorkload { + fmt.Fprintf(GinkgoWriter, "Expected DbWorkload: %v\nGot: %v\n", expectedADBDetails.DbWorkload, resp.AutonomousDatabase.DbWorkload) + } + if !compareBool(expectedADBDetails.IsDedicated, resp.AutonomousDatabase.IsDedicated) { + fmt.Fprintf(GinkgoWriter, "Expected IsDedicated: %v\nGot: %v\n", expectedADBDetails.IsDedicated, resp.AutonomousDatabase.IsDedicated) + } + if !compareString(expectedADBDetails.DbVersion, resp.AutonomousDatabase.DbVersion) { + fmt.Fprintf(GinkgoWriter, "Expected DbVersion: %v\nGot: %v\n", expectedADBDetails.DbVersion, resp.AutonomousDatabase.DbVersion) + } + if !compareInt(expectedADBDetails.DataStorageSizeInTBs, resp.AutonomousDatabase.DataStorageSizeInTBs) { + fmt.Fprintf(GinkgoWriter, "Expected DataStorageSize: %v\nGot: %v\n", expectedADBDetails.DataStorageSizeInTBs, resp.AutonomousDatabase.DataStorageSizeInTBs) + } + if !compareInt(expectedADBDetails.CPUCoreCount, resp.AutonomousDatabase.CpuCoreCount) { + fmt.Fprintf(GinkgoWriter, "Expected CPUCoreCount: %v\nGot: %v\n", expectedADBDetails.CPUCoreCount, resp.AutonomousDatabase.CpuCoreCount) + } + if !compareBool(expectedADBDetails.IsAutoScalingEnabled, resp.AutonomousDatabase.IsAutoScalingEnabled) { + fmt.Fprintf(GinkgoWriter, "Expected IsAutoScalingEnabled: %v\nGot: %v\n", expectedADBDetails.IsAutoScalingEnabled, resp.AutonomousDatabase.IsAutoScalingEnabled) + } + if !compareStringMap(expectedADBDetails.FreeformTags, resp.AutonomousDatabase.FreeformTags) { + fmt.Fprintf(GinkgoWriter, "Expected FreeformTags: %v\nGot: %v\n", expectedADBDetails.FreeformTags, resp.AutonomousDatabase.FreeformTags) + } + if !compareBool(expectedADBDetails.NetworkAccess.IsAccessControlEnabled, resp.AutonomousDatabase.IsAccessControlEnabled) { + fmt.Fprintf(GinkgoWriter, "Expected IsAccessControlEnabled: %v\nGot: %v\n", expectedADBDetails.NetworkAccess.IsAccessControlEnabled, resp.AutonomousDatabase.IsAccessControlEnabled) + } + if !reflect.DeepEqual(expectedADBDetails.NetworkAccess.AccessControlList, resp.AutonomousDatabase.WhitelistedIps) { + fmt.Fprintf(GinkgoWriter, "Expected AccessControlList: %v\nGot: %v\n", expectedADBDetails.NetworkAccess.AccessControlList, resp.AutonomousDatabase.WhitelistedIps) + } + if !compareBool(expectedADBDetails.NetworkAccess.IsMTLSConnectionRequired, resp.AutonomousDatabase.IsMtlsConnectionRequired) { + fmt.Fprintf(GinkgoWriter, "Expected IsMTLSConnectionRequired: %v\nGot: %v\n", expectedADBDetails.NetworkAccess.IsMTLSConnectionRequired, resp.AutonomousDatabase.IsMtlsConnectionRequired) + } + if !compareString(expectedADBDetails.NetworkAccess.PrivateEndpoint.SubnetOCID, resp.AutonomousDatabase.SubnetId) { + fmt.Fprintf(GinkgoWriter, "Expected SubnetOCID: %v\nGot: %v\n", expectedADBDetails.NetworkAccess.PrivateEndpoint.SubnetOCID, resp.AutonomousDatabase.SubnetId) + } + if !reflect.DeepEqual(expectedADBDetails.NetworkAccess.PrivateEndpoint.NsgOCIDs, resp.AutonomousDatabase.NsgIds) { + fmt.Fprintf(GinkgoWriter, "Expected NsgOCIDs: %v\nGot: %v\n", expectedADBDetails.NetworkAccess.PrivateEndpoint.NsgOCIDs, resp.AutonomousDatabase.NsgIds) + } + if !compareString(expectedADBDetails.NetworkAccess.PrivateEndpoint.HostnamePrefix, resp.AutonomousDatabase.PrivateEndpointLabel) { + fmt.Fprintf(GinkgoWriter, "Expected HostnamePrefix: %v\nGot: %v\n", expectedADBDetails.NetworkAccess.PrivateEndpoint.HostnamePrefix, resp.AutonomousDatabase.PrivateEndpointLabel) + } + } // Compare the elements one by one rather than doing reflect.DeelEqual(adb1, adb2), since some parameters // (e.g. adminPassword, wallet) are missing from e2eutil.GetAutonomousDatabase(). @@ -329,6 +383,71 @@ func AssertADBDetails(k8sClient *client.Client, dbClient *database.DatabaseClien } } +func TestNetworkAccessRestricted(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName, isMTLSConnectionRequired bool) func() { + return func() { + networkRestrictedSpec := dbv1alpha1.NetworkAccessSpec{ + AccessType: dbv1alpha1.NetworkAccessTypeRestricted, + IsMTLSConnectionRequired: common.Bool(isMTLSConnectionRequired), + AccessControlList: []string{"192.168.0.1"}, + } + + TestNetworkAccess(k8sClient, dbClient, adbLookupKey, networkRestrictedSpec)() + } +} + +func TestNetworkAccessPrivate(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName, isMTLSConnectionRequired bool, subnetOCID *string, nsgOCIDs *string) func() { + return func() { + Expect(*subnetOCID).ToNot(Equal("")) + Expect(*nsgOCIDs).ToNot(Equal("")) + + adb := &dbv1alpha1.AutonomousDatabase{} + derefK8sClient := *k8sClient + Expect(derefK8sClient.Get(context.TODO(), *adbLookupKey, adb)).Should(Succeed()) + + networkPrivateSpec := dbv1alpha1.NetworkAccessSpec{ + AccessType: dbv1alpha1.NetworkAccessTypePrivate, + AccessControlList: []string{}, + IsMTLSConnectionRequired: common.Bool(isMTLSConnectionRequired), + PrivateEndpoint: dbv1alpha1.PrivateEndpointSpec{ + HostnamePrefix: adb.Spec.Details.DisplayName, + NsgOCIDs: []string{*nsgOCIDs}, + SubnetOCID: common.String(*subnetOCID), + }, + } + + TestNetworkAccess(k8sClient, dbClient, adbLookupKey, networkPrivateSpec)() + } +} + +func TestNetworkAccessPublic(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName) func() { + return func() { + networkPublicSpec := dbv1alpha1.NetworkAccessSpec{ + AccessType: dbv1alpha1.NetworkAccessTypePublic, + IsMTLSConnectionRequired: common.Bool(true), + } + + TestNetworkAccess(k8sClient, dbClient, adbLookupKey, networkPublicSpec)() + } +} + +func TestNetworkAccess(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName, networkSpec dbv1alpha1.NetworkAccessSpec) func() { + return func() { + Expect(k8sClient).NotTo(BeNil()) + Expect(dbClient).NotTo(BeNil()) + Expect(adbLookupKey).NotTo(BeNil()) + + derefK8sClient := *k8sClient + + adb := &dbv1alpha1.AutonomousDatabase{} + AssertState(k8sClient, dbClient, adbLookupKey, database.AutonomousDatabaseLifecycleStateAvailable)() + Expect(derefK8sClient.Get(context.TODO(), *adbLookupKey, adb)).To(Succeed()) + + adb.Spec.Details.NetworkAccess = networkSpec + Expect(derefK8sClient.Update(context.TODO(), adb)).To(Succeed()) + AssertADBDetails(k8sClient, dbClient, adbLookupKey, adb)() + } +} + // UpdateAndAssertDetails changes the displayName from "foo" to "foo_new", and scale the cpuCoreCount to 2 func UpdateAndAssertDetails(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName) func() { return func() { @@ -441,7 +560,7 @@ func AssertRemoteState(k8sClient *client.Client, dbClient *database.DatabaseClie adb := &dbv1alpha1.AutonomousDatabase{} Expect(derefK8sClient.Get(context.TODO(), *adbLookupKey, adb)).To(Succeed()) - + By("Checking if the lifecycleState of remote resource is " + string(state)) AssertRemoteStateOCID(k8sClient, dbClient, adb.Spec.Details.AutonomousDatabaseOCID, state)() } } @@ -456,7 +575,7 @@ func AssertRemoteStateOCID(k8sClient *client.Client, dbClient *database.Database Expect(dbClient).NotTo(BeNil()) Expect(adbID).NotTo(BeNil()) - fmt.Fprintf(GinkgoWriter, "ADB ID is %s", *adbID) + fmt.Fprintf(GinkgoWriter, "ADB ID is %s\n", *adbID) derefK8sClient := *k8sClient derefDBClient := *dbClient diff --git a/test/e2e/resource/test_config.yaml b/test/e2e/resource/test_config.yaml index 4a5f8027..1a4c6faf 100644 --- a/test/e2e/resource/test_config.yaml +++ b/test/e2e/resource/test_config.yaml @@ -13,4 +13,8 @@ compartmentOCID: ocid1.compartment.. # The OCID of the OCI Vault Secret that holds the password of the ADMIN account (should start with ocid1.vaultsecret...) adminPasswordOCID: ocid1.vaultsecret... # The OCID of the OCI Vault Secret that holds the password of the wallet (should start with ocid1.vaultsecret...) -instanceWalletPasswordOCID: ocid1.vaultsecret... \ No newline at end of file +instanceWalletPasswordOCID: ocid1.vaultsecret... +# The OCID of the subnet used to test the network access settings +subnetOCID: ocid1.subnet... +# The OCID of the network security group used to test the network access settings +nsgOCID: ocid1.networksecuritygroup... \ No newline at end of file diff --git a/test/e2e/suite_test.go b/test/e2e/suite_test.go index 4c49c990..baddfc2e 100644 --- a/test/e2e/suite_test.go +++ b/test/e2e/suite_test.go @@ -110,6 +110,8 @@ var SharedCompartmentOCID string var SharedKeyOCID string var SharedAdminPasswordOCID string var SharedInstanceWalletPasswordOCID string +var SharedSubnetOCID string +var SharedNsgOCID string const SharedAdminPassSecretName string = "adb-admin-password" const SharedWalletPassSecretName = "adb-wallet-password" @@ -183,12 +185,16 @@ var _ = BeforeSuite(func(done ginkgo.Done) { SharedCompartmentOCID = testConfig.CompartmentOCID SharedAdminPasswordOCID = testConfig.AdminPasswordOCID SharedInstanceWalletPasswordOCID = testConfig.InstanceWalletPasswordOCID + SharedSubnetOCID = testConfig.SubnetOCID + SharedNsgOCID = testConfig.NsgOCID By("checking if the required parameters exist") Expect(testConfig.OCIConfigFile).ToNot(Equal("")) Expect(testConfig.CompartmentOCID).ToNot(Equal("")) Expect(testConfig.AdminPasswordOCID).ToNot(Equal("")) Expect(testConfig.InstanceWalletPasswordOCID).ToNot(Equal("")) + Expect(testConfig.SubnetOCID).ToNot(Equal("")) + Expect(testConfig.NsgOCID).ToNot(Equal("")) By("getting OCI provider") ociConfigUtil, err := e2eutil.GetOCIConfigUtil(testConfig.OCIConfigFile, testConfig.Profile) diff --git a/test/e2e/util/util.go b/test/e2e/util/util.go index 0e6c97a7..7e5097ce 100644 --- a/test/e2e/util/util.go +++ b/test/e2e/util/util.go @@ -119,6 +119,8 @@ type testConfiguration struct { CompartmentOCID string `yaml:"compartmentOCID"` AdminPasswordOCID string `yaml:"adminPasswordOCID"` InstanceWalletPasswordOCID string `yaml:"instanceWalletPasswordOCID"` + SubnetOCID string `yaml:"subnetOCID"` + NsgOCID string `yaml:"nsgOCID"` } func GetTestConfig(filename string) (*testConfiguration, error) { From 4e7dde3b1f465be501949b6e1e6e200039e061c2 Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Wed, 23 Feb 2022 10:58:36 +0530 Subject: [PATCH 167/628] Added support to have seperate pvc for ords --- .../v1alpha1/oraclerestdataservice_types.go | 7 +- .../v1alpha1/zz_generated.deepcopy.go | 2 +- commons/database/constants.go | 4 +- ...ase.oracle.com_oraclerestdataservices.yaml | 14 ++ .../oraclerestdataservice_controller.go | 153 ++++++++++++++++-- .../singleinstancedatabase_controller.go | 3 +- oracle-database-operator.yaml | 13 ++ 7 files changed, 175 insertions(+), 21 deletions(-) diff --git a/apis/database/v1alpha1/oraclerestdataservice_types.go b/apis/database/v1alpha1/oraclerestdataservice_types.go index bc26d46c..19583579 100644 --- a/apis/database/v1alpha1/oraclerestdataservice_types.go +++ b/apis/database/v1alpha1/oraclerestdataservice_types.go @@ -61,6 +61,7 @@ type OracleRestDataServiceSpec struct { RestEnableSchemas []OracleRestDataServiceRestEnableSchemas `json:"restEnableSchemas,omitempty"` OracleService string `json:"oracleService,omitempty"` ServiceAccountName string `json:"serviceAccountName,omitempty"` + Persistence OracleRestDataServicePersistence `json:"persistence,omitempty"` // +k8s:openapi-gen=true // +kubebuilder:validation:Minimum=1 @@ -69,11 +70,11 @@ type OracleRestDataServiceSpec struct { // OracleRestDataServicePersistence defines the storage releated params type OracleRestDataServicePersistence struct { - Size string `json:"size"` - StorageClass string `json:"storageClass"` + Size string `json:"size,omitempty"` + StorageClass string `json:"storageClass,omitempty"` // +kubebuilder:validation:Enum=ReadWriteOnce;ReadWriteMany - AccessMode string `json:"accessMode"` + AccessMode string `json:"accessMode,omitempty"` } // OracleRestDataServiceImage defines the Image source and pullSecrets for POD diff --git a/apis/database/v1alpha1/zz_generated.deepcopy.go b/apis/database/v1alpha1/zz_generated.deepcopy.go index 42bfd573..c2eddf2e 100644 --- a/apis/database/v1alpha1/zz_generated.deepcopy.go +++ b/apis/database/v1alpha1/zz_generated.deepcopy.go @@ -1,4 +1,3 @@ -//go:build !ignore_autogenerated // +build !ignore_autogenerated /* @@ -757,6 +756,7 @@ func (in *OracleRestDataServiceSpec) DeepCopyInto(out *OracleRestDataServiceSpec *out = make([]OracleRestDataServiceRestEnableSchemas, len(*in)) copy(*out, *in) } + out.Persistence = in.Persistence } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new OracleRestDataServiceSpec. diff --git a/commons/database/constants.go b/commons/database/constants.go index 2d84d23a..74121a5d 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -358,7 +358,9 @@ const AlterProcessesCMD string = "echo -e \"alter system set processes=%d scope const GetInitParamsSQL string = "echo -e \"select name,display_value from v\\$parameter where name in ('sga_target','pga_aggregate_target','cpu_count','processes') order by name asc;\" | %s" -const UnzipApex string = "if [ -f /opt/oracle/oradata/apex-latest.zip ]; then unzip -o /opt/oracle/oradata/apex-latest.zip -d /opt/oracle/oradata/${ORACLE_SID^^}; else echo \"apex-latest.zip not found\"; fi;" +const UnzipApexOnSIDBPod string = "if [ -f /opt/oracle/oradata/apex-latest.zip ]; then unzip -o /opt/oracle/oradata/apex-latest.zip -d /opt/oracle/oradata/${ORACLE_SID^^}; else echo \"apex-latest.zip not found\"; fi;" + +const UnzipApexOnORDSPod string = "if [ -f /opt/oracle/ords/config/ords/apex-latest.zip ]; then unzip -o /opt/oracle/ords/config/ords/apex-latest.zip -d /opt/oracle/ords/config/ords; else echo \"apex-latest.zip not found\"; fi;" const ChownApex string = " chown oracle:oinstall /opt/oracle/oradata/${ORACLE_SID^^}/apex;" diff --git a/config/crd/bases/database.oracle.com_oraclerestdataservices.yaml b/config/crd/bases/database.oracle.com_oraclerestdataservices.yaml index f1a49357..13e4edd8 100644 --- a/config/crd/bases/database.oracle.com_oraclerestdataservices.yaml +++ b/config/crd/bases/database.oracle.com_oraclerestdataservices.yaml @@ -117,6 +117,20 @@ spec: type: object ordsUser: type: string + persistence: + description: OracleRestDataServicePersistence defines the storage + releated params + properties: + accessMode: + enum: + - ReadWriteOnce + - ReadWriteMany + type: string + size: + type: string + storageClass: + type: string + type: object replicas: minimum: 1 type: integer diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 310e29f0..f16b38e2 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -48,6 +48,7 @@ import ( corev1 "k8s.io/api/core/v1" apierrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/api/resource" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/runtime" "k8s.io/apimachinery/pkg/types" @@ -142,15 +143,15 @@ func (r *OracleRestDataServiceReconciler) Reconcile(ctx context.Context, req ctr return result, nil } - // Validate if Primary Database Reference is ready - result, sidbReadyPod := r.validateSidbReadiness(oracleRestDataService, singleInstanceDatabase, ctx, req) + // PVC Creation + result, err = r.createPVC(ctx, req, oracleRestDataService) if result.Requeue { r.Log.Info("Reconcile queued") return result, nil } - // Configure Apex - result = r.configureApex(oracleRestDataService, singleInstanceDatabase, sidbReadyPod, ctx, req) + // Validate if Primary Database Reference is ready + result, sidbReadyPod := r.validateSidbReadiness(oracleRestDataService, singleInstanceDatabase, ctx, req) if result.Requeue { r.Log.Info("Reconcile queued") return result, nil @@ -163,6 +164,13 @@ func (r *OracleRestDataServiceReconciler) Reconcile(ctx context.Context, req ctr return result, nil } + // Configure Apex + result = r.configureApex(oracleRestDataService, singleInstanceDatabase, sidbReadyPod, ctx, req) + if result.Requeue { + r.Log.Info("Reconcile queued") + return result, nil + } + result = r.setupORDS(oracleRestDataService, singleInstanceDatabase, sidbReadyPod, ctx, req) if result.Requeue { r.Log.Info("Reconcile queued") @@ -390,8 +398,13 @@ func (r *OracleRestDataServiceReconciler) instantiatePodSpec(m *dbapi.OracleRest Name: "datamount", VolumeSource: corev1.VolumeSource{ PersistentVolumeClaim: &corev1.PersistentVolumeClaimVolumeSource{ - ClaimName: n.Name, - ReadOnly: false, + ClaimName: func() string { + if m.Spec.Persistence.AccessMode != "" { + return m.Name + } + return n.Name + }(), + ReadOnly: false, }, }, }}, @@ -540,6 +553,42 @@ func (r *OracleRestDataServiceReconciler) instantiatePodSpec(m *dbapi.OracleRest return pod } +//############################################################################# +// Instantiate Persistent Volume Claim spec from SingleInstanceDatabase spec +//############################################################################# +func (r *OracleRestDataServiceReconciler) instantiatePVCSpec(m *dbapi.OracleRestDataService) *corev1.PersistentVolumeClaim { + + pvc := &corev1.PersistentVolumeClaim{ + TypeMeta: metav1.TypeMeta{ + Kind: "PersistentVolumeClaim", + }, + ObjectMeta: metav1.ObjectMeta{ + Name: m.Name, + Namespace: m.Namespace, + Labels: map[string]string{ + "app": m.Name, + }, + }, + Spec: corev1.PersistentVolumeClaimSpec{ + AccessModes: func() []corev1.PersistentVolumeAccessMode { + var accessMode []corev1.PersistentVolumeAccessMode + accessMode = append(accessMode, corev1.PersistentVolumeAccessMode(m.Spec.Persistence.AccessMode)) + return accessMode + }(), + Resources: corev1.ResourceRequirements{ + Requests: map[corev1.ResourceName]resource.Quantity{ + // Requests describes the minimum amount of compute resources required + "storage": resource.MustParse(m.Spec.Persistence.Size), + }, + }, + StorageClassName: &m.Spec.Persistence.StorageClass, + }, + } + // Set SingleInstanceDatabase instance as the owner and controller + ctrl.SetControllerReference(m, pvc, r.Scheme) + return pvc +} + //############################################################################# // Create a Service for OracleRestDataService //############################################################################# @@ -592,6 +641,40 @@ func (r *OracleRestDataServiceReconciler) createSVC(ctx context.Context, req ctr return requeueN } +//############################################################################# +// Stake a claim for Persistent Volume +//############################################################################# +func (r *OracleRestDataServiceReconciler) createPVC(ctx context.Context, req ctrl.Request, + m *dbapi.OracleRestDataService) (ctrl.Result, error) { + + // PV is shared for ORDS and SIDB + if m.Spec.Persistence.AccessMode == "" { + return requeueN, nil + } + log := r.Log.WithValues("createPVC", req.NamespacedName) + + pvc := &corev1.PersistentVolumeClaim{} + err := r.Get(ctx, types.NamespacedName{Name: m.Name, Namespace: m.Namespace}, pvc) + if err != nil && apierrors.IsNotFound(err) { + // Define a new PVC + pvc = r.instantiatePVCSpec(m) + log.Info("Creating a new PVC", "PVC.Namespace", pvc.Namespace, "PVC.Name", pvc.Name) + err = r.Create(ctx, pvc) + if err != nil { + log.Error(err, "Failed to create new PVC", "PVC.Namespace", pvc.Namespace, "PVC.Name", pvc.Name) + return requeueY, err + } + return requeueN, nil + } else if err != nil { + log.Error(err, "Failed to get PVC") + return requeueY, err + } else { + log.Info("PVC already exists") + } + + return requeueN, nil +} + //############################################################################# // Create the requested POD replicas //############################################################################# @@ -884,6 +967,55 @@ func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataS return requeueY } + // If Seperate PV for ORDS, we need to unzip apex-latest.zip + if m.Spec.Persistence.AccessMode != "" { + ordsReadyPod, _, _, _, err := dbcommons.FindPods(r, m.Spec.Image.Version, + m.Spec.Image.PullFrom, m.Name, m.Namespace, ctx, req) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + if ordsReadyPod.Name == "" { + eventReason := "Waiting" + eventMsg := "waiting for " + m.Name + " to be Ready" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + return requeueY + } + + unzipApex := dbcommons.UnzipApexOnORDSPod + if n.Spec.Edition == "express" { + unzipApex += dbcommons.ChownApex + } + + // Unzip /opt/oracle/ords/config/ords/apex-latest.zip to /opt/oracle/ords/config/ords + out, err := dbcommons.ExecCommand(r, r.Config, ordsReadyPod.Name, ordsReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + unzipApex) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + log.Info(" UnzipApex Output : \n" + out) + + if strings.Contains(out, "apex-latest.zip not found") { + eventReason := "Waiting" + eventMsg := "apex-latest.zip doesn't exist in the location /opt/oracle/ords/config/ords" + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) + return requeueY + } + + } else { + + // Copy APEX Images to ORACLE_SID only if PV for ORDS and SIDB is shared + out, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf(dbcommons.CopyApexImages)) + if err != nil { + log.Info(err.Error()) + return requeueY + } + log.Info(" CopyApexImages Output : \n" + out) + + } + apexPasswordSecret := &corev1.Secret{} err := r.Get(ctx, types.NamespacedName{Name: m.Spec.ApexPassword.SecretName, Namespace: m.Namespace}, apexPasswordSecret) if err != nil { @@ -953,15 +1085,6 @@ func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataS m.Status.ApexConfigured = true r.Status().Update(ctx, m) - // Copy APEX Images - out, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf(dbcommons.CopyApexImages)) - if err != nil { - log.Info(err.Error()) - return requeueY - } - log.Info(" CopyApexImages Output : \n" + out) - if m.Status.OrdsInstalled { readyPod, _, available, _, err := dbcommons.FindPods(r, m.Spec.Image.Version, diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 391d282f..89fbbd57 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1848,10 +1848,11 @@ func (r *SingleInstanceDatabaseReconciler) installApex(m *dbapi.SingleInstanceDa eventMsg := "Waiting for Apex Installation to complete" r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) - unzipApex := dbcommons.UnzipApex + unzipApex := dbcommons.UnzipApexOnSIDBPod if m.Spec.Edition == "express" { unzipApex += dbcommons.ChownApex } + // Unzip /opt/oracle/oradata/apex-latest.zip to /opt/oracle/oradata/${ORACLE_SID^^} out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", unzipApex) diff --git a/oracle-database-operator.yaml b/oracle-database-operator.yaml index 0dbb3ab8..2ddbdd4a 100644 --- a/oracle-database-operator.yaml +++ b/oracle-database-operator.yaml @@ -288,6 +288,19 @@ spec: type: object ordsUser: type: string + persistence: + description: OracleRestDataServicePersistence defines the storage releated params + properties: + accessMode: + enum: + - ReadWriteOnce + - ReadWriteMany + type: string + size: + type: string + storageClass: + type: string + type: object replicas: minimum: 1 type: integer From ffb0f1172a7b1a79f20757a79fc4bdfee32cf77f Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Wed, 23 Feb 2022 11:02:07 +0530 Subject: [PATCH 168/628] Added support to have seperate pvc for ords --- config/samples/sidb/oraclerestdataservice.yaml | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/config/samples/sidb/oraclerestdataservice.yaml b/config/samples/sidb/oraclerestdataservice.yaml index 19da6a11..6652fdae 100644 --- a/config/samples/sidb/oraclerestdataservice.yaml +++ b/config/samples/sidb/oraclerestdataservice.yaml @@ -38,6 +38,18 @@ spec: pullFrom: pullSecrets: + ## Uncomment only if you want to use seperate pvc for ords. else same pvc for sidb is used here + ## apex-latest.zip file should be present in the location '/opt/oracle/ords/config/ords' + ## Download using `wget https://download.oracle.com/otn_software/apex/apex-latest.zip` + ## use `kubectl cp :/opt/oracle/ords/config/ords + ## size : Minimum size of pvc | class : PVC storage Class + ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany + ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. + # persistence: + # size: 100Gi + # storageClass: "oci" + # accessMode: "ReadWriteOnce" + ## Type of service Applicable on cloud enviroments only. ## if loadBalService: false, service type = "NodePort". else "LoadBalancer" loadBalancer: false From f2176d991b87545b49a716b251a8fb192926fe69 Mon Sep 17 00:00:00 2001 From: "omar.salazar@oracle.com" Date: Wed, 23 Feb 2022 14:37:36 +0000 Subject: [PATCH 169/628] Add network tests to provisioning case --- Makefile | 2 +- test/e2e/autonomousdatabase_controller_create_test.go | 10 ++++++++++ test/e2e/behavior/shared_behaviors.go | 6 +++--- 3 files changed, 14 insertions(+), 4 deletions(-) diff --git a/Makefile b/Makefile index a0bd6870..43700420 100644 --- a/Makefile +++ b/Makefile @@ -59,7 +59,7 @@ test: manifests generate fmt vet envtest ## Run unit tests. E2ETEST ?= ./test/e2e/ e2e: manifests generate fmt vet envtest ## Run e2e tests. - KUBEBUILDER_ASSETS="$(shell $(ENVTEST) use $(ENVTEST_K8S_VERSION) -p path)" go test $(E2ETEST) -v -timeout 40m -ginkgo.v -ginkgo.failFast + KUBEBUILDER_ASSETS="$(shell $(ENVTEST) use $(ENVTEST_K8S_VERSION) -p path)" go test $(E2ETEST) -v -timeout 50m -ginkgo.v -ginkgo.failFast ##@ Build diff --git a/test/e2e/autonomousdatabase_controller_create_test.go b/test/e2e/autonomousdatabase_controller_create_test.go index b0b9c91a..74760dde 100644 --- a/test/e2e/autonomousdatabase_controller_create_test.go +++ b/test/e2e/autonomousdatabase_controller_create_test.go @@ -220,6 +220,16 @@ var _ = Describe("test ADB provisioning", func() { It("Should restart ADB", e2ebehavior.UpdateAndAssertState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateAvailable)) + It("Should change to RESTRICTED network access", e2ebehavior.TestNetworkAccessRestricted(&k8sClient, &dbClient, &adbLookupKey, false)) + + It("Should change isMTLSConnectionRequired to false", e2ebehavior.TestNetworkAccessRestricted(&k8sClient, &dbClient, &adbLookupKey, false)) + + It("Should should change to PRIVATE network access", e2ebehavior.TestNetworkAccessPrivate(&k8sClient, &dbClient, &adbLookupKey, false, &SharedSubnetOCID, &SharedNsgOCID)) + + It("Should change isMTLSConnectionRequired to true when network access is PRIVATE", e2ebehavior.TestNetworkAccessPrivate(&k8sClient, &dbClient, &adbLookupKey, true, &SharedSubnetOCID, &SharedNsgOCID)) + + It("Should return to PUBLIC access type", e2ebehavior.TestNetworkAccessPublic(&k8sClient, &dbClient, &adbLookupKey)) + It("Should delete the resource in cluster and terminate the database in OCI", e2ebehavior.AssertHardLinkDelete(&k8sClient, &dbClient, &adbLookupKey)) }) }) diff --git a/test/e2e/behavior/shared_behaviors.go b/test/e2e/behavior/shared_behaviors.go index 69b826d2..4ce034f9 100644 --- a/test/e2e/behavior/shared_behaviors.go +++ b/test/e2e/behavior/shared_behaviors.go @@ -409,9 +409,9 @@ func TestNetworkAccessPrivate(k8sClient *client.Client, dbClient *database.Datab AccessControlList: []string{}, IsMTLSConnectionRequired: common.Bool(isMTLSConnectionRequired), PrivateEndpoint: dbv1alpha1.PrivateEndpointSpec{ - HostnamePrefix: adb.Spec.Details.DisplayName, - NsgOCIDs: []string{*nsgOCIDs}, - SubnetOCID: common.String(*subnetOCID), + HostnamePrefix: adb.Spec.Details.DbName, + NsgOCIDs: []string{*nsgOCIDs}, + SubnetOCID: common.String(*subnetOCID), }, } From 018bb87a4de06d7527dfc0c81ac8ddb8cef8bc5c Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Wed, 23 Feb 2022 12:50:08 -0500 Subject: [PATCH 170/628] update oracle-database-operator.yaml --- config/crd/kustomization.yaml | 2 + oracle-database-operator.yaml | 371 ++++++++++++++++++++++++++++++++-- 2 files changed, 357 insertions(+), 16 deletions(-) diff --git a/config/crd/kustomization.yaml b/config/crd/kustomization.yaml index c290b81d..da376536 100644 --- a/config/crd/kustomization.yaml +++ b/config/crd/kustomization.yaml @@ -7,6 +7,8 @@ # It should be run by config/default resources: - bases/database.oracle.com_autonomousdatabases.yaml +- bases/database.oracle.com_autonomousdatabasebackups.yaml +- bases/database.oracle.com_autonomousdatabaserestores.yaml - bases/database.oracle.com_singleinstancedatabases.yaml - bases/database.oracle.com_shardingdatabases.yaml - bases/database.oracle.com_oraclerestdataservices.yaml diff --git a/oracle-database-operator.yaml b/oracle-database-operator.yaml index 0dbb3ab8..4d4c54e1 100644 --- a/oracle-database-operator.yaml +++ b/oracle-database-operator.yaml @@ -7,6 +7,216 @@ metadata: --- apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.6.1 + creationTimestamp: null + name: autonomousdatabasebackups.database.oracle.com +spec: + group: database.oracle.com + names: + kind: AutonomousDatabaseBackup + listKind: AutonomousDatabaseBackupList + plural: autonomousdatabasebackups + shortNames: + - adbbu + - adbbus + singular: autonomousdatabasebackup + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .status.lifecycleState + name: State + type: string + - jsonPath: .status.dbDisplayName + name: DB DisplayName + type: string + - jsonPath: .status.type + name: Type + type: string + - jsonPath: .status.timeStarted + name: Started + type: string + - jsonPath: .status.timeEnded + name: Ended + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + description: AutonomousDatabaseBackup is the Schema for the autonomousdatabasebackups API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: AutonomousDatabaseBackupSpec defines the desired state of AutonomousDatabaseBackup + properties: + autonomousDatabaseBackupOCID: + type: string + autonomousDatabaseOCID: + description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster Important: Run "make" to regenerate code after modifying this file' + type: string + displayName: + type: string + ociConfig: + properties: + configMapName: + type: string + secretName: + type: string + type: object + type: object + status: + description: AutonomousDatabaseBackupStatus defines the observed state of AutonomousDatabaseBackup + properties: + autonomousDatabaseBackupOCID: + type: string + autonomousDatabaseOCID: + type: string + compartmentOCID: + type: string + dbDisplayName: + type: string + dbName: + type: string + displayName: + type: string + isAutomatic: + type: boolean + lifecycleState: + description: 'INSERT ADDITIONAL STATUS FIELD - define observed state of cluster Important: Run "make" to regenerate code after modifying this file' + type: string + timeEnded: + type: string + timeStarted: + type: string + type: + description: 'AutonomousDatabaseBackupTypeEnum Enum with underlying type: string' + type: string + required: + - autonomousDatabaseBackupOCID + - autonomousDatabaseOCID + - compartmentOCID + - dbDisplayName + - dbName + - displayName + - isAutomatic + - lifecycleState + - type + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.6.1 + creationTimestamp: null + name: autonomousdatabaserestores.database.oracle.com +spec: + group: database.oracle.com + names: + kind: AutonomousDatabaseRestore + listKind: AutonomousDatabaseRestoreList + plural: autonomousdatabaserestores + shortNames: + - adbr + - adbrs + singular: autonomousdatabaserestore + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .status.lifecycleState + name: State + type: string + - jsonPath: .status.displayName + name: DisplayName + type: string + - jsonPath: .status.dbName + name: DbName + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + description: AutonomousDatabaseRestore is the Schema for the autonomousdatabaserestores API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: AutonomousDatabaseRestoreSpec defines the desired state of AutonomousDatabaseRestore + properties: + backupName: + description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster Important: Run "make" to regenerate code after modifying this file' + type: string + ociConfig: + properties: + configMapName: + type: string + secretName: + type: string + type: object + pointInTime: + properties: + autonomousDatabaseOCID: + type: string + timeStamp: + type: string + type: object + type: object + status: + description: AutonomousDatabaseRestoreStatus defines the observed state of AutonomousDatabaseRestore + properties: + autonomousDatabaseOCID: + type: string + dbName: + type: string + displayName: + description: 'INSERT ADDITIONAL STATUS FIELD - define observed state of cluster Important: Run "make" to regenerate code after modifying this file' + type: string + lifecycleState: + description: 'WorkRequestStatusEnum Enum with underlying type: string' + type: string + required: + - autonomousDatabaseOCID + - dbName + - displayName + - lifecycleState + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition metadata: annotations: controller-gen.kubebuilder.io/version: v0.6.1 @@ -72,6 +282,8 @@ spec: ociSecretOCID: type: string type: object + autonomousContainerDatabaseOCID: + type: string autonomousDatabaseOCID: type: string compartmentOCID: @@ -98,25 +310,42 @@ spec: additionalProperties: type: string type: object - isAccessControlEnabled: - type: boolean isAutoScalingEnabled: type: boolean isDedicated: type: boolean - isMTLSConnectionRequired: - type: boolean lifecycleState: description: 'AutonomousDatabaseLifecycleStateEnum Enum with underlying type: string' type: string - nsgOCIDs: - items: - type: string - type: array - privateEndpointLabel: - type: string - subnetOCID: - type: string + networkAccess: + properties: + accessControlList: + items: + type: string + type: array + accessType: + enum: + - "" + - PUBLIC + - RESTRICTED + - PRIVATE + type: string + isAccessControlEnabled: + type: boolean + isMTLSConnectionRequired: + type: boolean + privateEndpoint: + properties: + hostnamePrefix: + type: string + nsgOCIDs: + items: + type: string + type: array + subnetOCID: + type: string + type: object + type: object wallet: properties: name: @@ -129,10 +358,6 @@ spec: type: string type: object type: object - whitelistedIPs: - items: - type: string - type: array type: object hardLink: default: false @@ -150,6 +375,19 @@ spec: status: description: AutonomousDatabaseStatus defines the observed state of AutonomousDatabase properties: + allConnectionStrings: + items: + properties: + connectionStrings: + additionalProperties: + type: string + type: object + tlsAuthentication: + type: string + required: + - connectionStrings + type: object + type: array cpuCoreCount: type: integer dataStorageSizeInTBs: @@ -1510,6 +1748,46 @@ metadata: cert-manager.io/inject-ca-from: oracle-database-operator-system/oracle-database-operator-serving-cert name: oracle-database-operator-mutating-webhook-configuration webhooks: +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /mutate-database-oracle-com-v1alpha1-autonomousdatabase + failurePolicy: Fail + name: mautonomousdatabase.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + resources: + - autonomousdatabase + sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /mutate-database-oracle-com-v1alpha1-autonomousdatabasebackup + failurePolicy: Fail + name: mautonomousdatabasebackup.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + resources: + - autonomousdatabasebackups + sideEffects: None - admissionReviewVersions: - v1 - v1beta1 @@ -1560,6 +1838,67 @@ metadata: cert-manager.io/inject-ca-from: oracle-database-operator-system/oracle-database-operator-serving-cert name: oracle-database-operator-validating-webhook-configuration webhooks: +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /validate-database-oracle-com-v1alpha1-autonomousdatabase + failurePolicy: Fail + name: vautonomousdatabase.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + - DELETE + resources: + - autonomousdatabases + sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /validate-database-oracle-com-v1alpha1-autonomousdatabasebackup + failurePolicy: Fail + name: vautonomousdatabasebackup.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + resources: + - autonomousdatabasebackups + sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /validate-database-oracle-com-v1alpha1-autonomousdatabaserestore + failurePolicy: Fail + name: vautonomousdatabaserestore.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + resources: + - autonomousdatabaserestores + sideEffects: None - admissionReviewVersions: - v1 - v1beta1 From ca7f016c7c9a7a77f62ee7707c60474cf357a640 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Wed, 23 Feb 2022 14:49:04 -0500 Subject: [PATCH 171/628] fixed adb mutating webhook not working issue --- apis/database/v1alpha1/autonomousdatabase_webhook.go | 4 ++-- config/webhook/manifests.yaml | 2 +- main.go | 1 + 3 files changed, 4 insertions(+), 3 deletions(-) diff --git a/apis/database/v1alpha1/autonomousdatabase_webhook.go b/apis/database/v1alpha1/autonomousdatabase_webhook.go index df6d2f59..09e10807 100644 --- a/apis/database/v1alpha1/autonomousdatabase_webhook.go +++ b/apis/database/v1alpha1/autonomousdatabase_webhook.go @@ -61,7 +61,7 @@ func (r *AutonomousDatabase) SetupWebhookWithManager(mgr ctrl.Manager) error { // EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! -//+kubebuilder:webhook:verbs=create;update,path=/mutate-database-oracle-com-v1alpha1-autonomousdatabase,mutating=true,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=autonomousdatabase,versions=v1alpha1,name=mautonomousdatabase.kb.io,admissionReviewVersions={v1} +//+kubebuilder:webhook:verbs=create;update,path=/mutate-database-oracle-com-v1alpha1-autonomousdatabase,mutating=true,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=autonomousdatabases,versions=v1alpha1,name=mautonomousdatabase.kb.io,admissionReviewVersions=v1 var _ webhook.Defaulter = &AutonomousDatabase{} @@ -167,7 +167,7 @@ func validateNetworkAccess(adb *AutonomousDatabase, allErrs field.ErrorList) fie field.Forbidden(field.NewPath("spec").Child("details").Child("networkAccess").Child("accessControlList"), fmt.Sprintf("accessControlList cannot be empty when the network access type is %s", NetworkAccessTypeRestricted))) } - } else { // the accessType is PRIVATE + } else if adb.Spec.Details.NetworkAccess.AccessType == NetworkAccessTypePrivate { // the accessType is PRIVATE if adb.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID == nil { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("details").Child("networkAccess").Child("privateEndpoint").Child("subnetOCID"), diff --git a/config/webhook/manifests.yaml b/config/webhook/manifests.yaml index 4107f440..6d45c5ff 100644 --- a/config/webhook/manifests.yaml +++ b/config/webhook/manifests.yaml @@ -24,7 +24,7 @@ webhooks: - CREATE - UPDATE resources: - - autonomousdatabase + - autonomousdatabases sideEffects: None - admissionReviewVersions: - v1 diff --git a/main.go b/main.go index e0b9710f..01dd4ac9 100644 --- a/main.go +++ b/main.go @@ -176,6 +176,7 @@ func main() { setupLog.Error(err, "unable to create webhook", "webhook", "AutonomousDatabaseRestore") os.Exit(1) } + // +kubebuilder:scaffold:builder setupLog.Info("starting manager") From 56d7f5ee6c32c4858f0aaefc401f18bb9eac7c71 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Thu, 24 Feb 2022 22:45:46 -0500 Subject: [PATCH 172/628] remove excessive test cases --- test/e2e/autonomousdatabase_controller_bind_test.go | 6 ------ .../e2e/autonomousdatabase_controller_create_test.go | 12 ------------ 2 files changed, 18 deletions(-) diff --git a/test/e2e/autonomousdatabase_controller_bind_test.go b/test/e2e/autonomousdatabase_controller_bind_test.go index 56705060..02d04810 100644 --- a/test/e2e/autonomousdatabase_controller_bind_test.go +++ b/test/e2e/autonomousdatabase_controller_bind_test.go @@ -189,12 +189,6 @@ var _ = Describe("test ADB binding with hardLink=true", func() { It("Should download an instance wallet using the password from OCI Secret OCID "+SharedInstanceWalletPasswordOCID, e2ebehavior.AssertWallet(&k8sClient, &adbLookupKey)) - It("should update ADB", e2ebehavior.UpdateAndAssertDetails(&k8sClient, &dbClient, &adbLookupKey)) - - It("Should stop ADB", e2ebehavior.UpdateAndAssertState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateStopped)) - - It("Should restart ADB", e2ebehavior.UpdateAndAssertState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateAvailable)) - It("Should delete the resource in cluster and terminate the database in OCI", e2ebehavior.AssertHardLinkDelete(&k8sClient, &dbClient, &adbLookupKey)) }) diff --git a/test/e2e/autonomousdatabase_controller_create_test.go b/test/e2e/autonomousdatabase_controller_create_test.go index b0b9c91a..afbb4912 100644 --- a/test/e2e/autonomousdatabase_controller_create_test.go +++ b/test/e2e/autonomousdatabase_controller_create_test.go @@ -153,12 +153,6 @@ var _ = Describe("test ADB provisioning", func() { It("Should download an instance wallet using the password from K8s Secret "+SharedWalletPassSecretName, e2ebehavior.AssertWallet(&k8sClient, &adbLookupKey)) - It("should update ADB", e2ebehavior.UpdateAndAssertDetails(&k8sClient, &dbClient, &adbLookupKey)) - - It("Should stop ADB", e2ebehavior.UpdateAndAssertState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateStopped)) - - It("Should restart ADB", e2ebehavior.UpdateAndAssertState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateAvailable)) - It("Should delete the resource in cluster and terminate the database in OCI", e2ebehavior.AssertHardLinkDelete(&k8sClient, &dbClient, &adbLookupKey)) }) @@ -214,12 +208,6 @@ var _ = Describe("test ADB provisioning", func() { It("Should download an instance wallet using the password from OCI Secret OCID "+SharedInstanceWalletPasswordOCID, e2ebehavior.AssertWallet(&k8sClient, &adbLookupKey)) - It("should update ADB", e2ebehavior.UpdateAndAssertDetails(&k8sClient, &dbClient, &adbLookupKey)) - - It("Should stop ADB", e2ebehavior.UpdateAndAssertState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateStopped)) - - It("Should restart ADB", e2ebehavior.UpdateAndAssertState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateAvailable)) - It("Should delete the resource in cluster and terminate the database in OCI", e2ebehavior.AssertHardLinkDelete(&k8sClient, &dbClient, &adbLookupKey)) }) }) From 2b451df2551eb61e1c917ef3694a9f8b989c46d5 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Thu, 24 Feb 2022 22:46:15 -0500 Subject: [PATCH 173/628] add freeform tag to the test case --- test/e2e/behavior/shared_behaviors.go | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/test/e2e/behavior/shared_behaviors.go b/test/e2e/behavior/shared_behaviors.go index 69b826d2..f0f5f2a2 100644 --- a/test/e2e/behavior/shared_behaviors.go +++ b/test/e2e/behavior/shared_behaviors.go @@ -263,10 +263,15 @@ func UpdateDetails(k8sClient *client.Client, dbClient *database.DatabaseClient, newCPUCoreCount = 1 } - By(fmt.Sprintf("Updating the ADB with newDisplayName = %s and newCPUCoreCount = %d\n", newDisplayName, newCPUCoreCount)) + var newKey = "testKey" + var newVal = "testVal" + + By(fmt.Sprintf("Updating the ADB with newDisplayName = %s, newCPUCoreCount = %d and newFreeformTag = %s:%s\n", + newDisplayName, newCPUCoreCount, newKey, newVal)) expectedADB.Spec.Details.DisplayName = common.String(newDisplayName) expectedADB.Spec.Details.CPUCoreCount = common.Int(newCPUCoreCount) + expectedADB.Spec.Details.FreeformTags = map[string]string{newKey: newVal} Expect(derefK8sClient.Update(context.TODO(), expectedADB)).To(Succeed()) @@ -409,9 +414,9 @@ func TestNetworkAccessPrivate(k8sClient *client.Client, dbClient *database.Datab AccessControlList: []string{}, IsMTLSConnectionRequired: common.Bool(isMTLSConnectionRequired), PrivateEndpoint: dbv1alpha1.PrivateEndpointSpec{ - HostnamePrefix: adb.Spec.Details.DisplayName, - NsgOCIDs: []string{*nsgOCIDs}, - SubnetOCID: common.String(*subnetOCID), + HostnamePrefix: adb.Spec.Details.DbName, + NsgOCIDs: []string{*nsgOCIDs}, + SubnetOCID: common.String(*subnetOCID), }, } From 6cef8a9c31d92d7fc8275bf34264bc482f00377b Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Wed, 2 Mar 2022 12:46:12 +0530 Subject: [PATCH 174/628] change unzip to jar for extracting .xip --- commons/database/constants.go | 2 +- controllers/database/oraclerestdataservice_controller.go | 5 +---- 2 files changed, 2 insertions(+), 5 deletions(-) diff --git a/commons/database/constants.go b/commons/database/constants.go index 74121a5d..82128aec 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -360,7 +360,7 @@ const GetInitParamsSQL string = "echo -e \"select name,display_value from v\\$p const UnzipApexOnSIDBPod string = "if [ -f /opt/oracle/oradata/apex-latest.zip ]; then unzip -o /opt/oracle/oradata/apex-latest.zip -d /opt/oracle/oradata/${ORACLE_SID^^}; else echo \"apex-latest.zip not found\"; fi;" -const UnzipApexOnORDSPod string = "if [ -f /opt/oracle/ords/config/ords/apex-latest.zip ]; then unzip -o /opt/oracle/ords/config/ords/apex-latest.zip -d /opt/oracle/ords/config/ords; else echo \"apex-latest.zip not found\"; fi;" +const UnzipApexOnORDSPod string = "if [ -f /opt/oracle/ords/config/ords/apex-latest.zip ]; then cd /opt/oracle/ords/config/ords && jar -xf /opt/oracle/ords/config/ords/apex-latest.zip; else echo \"apex-latest.zip not found\"; fi;" const ChownApex string = " chown oracle:oinstall /opt/oracle/oradata/${ORACLE_SID^^}/apex;" diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index f16b38e2..5e73e4a7 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -201,10 +201,7 @@ func (r *OracleRestDataServiceReconciler) validate(m *dbapi.OracleRestDataServic var err error eventReason := "Spec Error" var eventMsgs []string - // If Block Volume , Ensure Replicas=1 - if n.Spec.Persistence.AccessMode == "ReadWriteOnce" { - eventMsgs = append(eventMsgs, "ords can be installed only on ReadWriteMany Access Mode of : "+m.Spec.DatabaseRef) - } + if m.Status.DatabaseRef != "" && m.Status.DatabaseRef != m.Spec.DatabaseRef { eventMsgs = append(eventMsgs, "databaseRef cannot be updated") } From 3d3e5799f0259abeddfa7a0865f7de9e8fe03581 Mon Sep 17 00:00:00 2001 From: Mahi0911 Date: Wed, 2 Mar 2022 13:21:05 +0530 Subject: [PATCH 175/628] Updated persistence validation on ords --- controllers/database/oraclerestdataservice_controller.go | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 5e73e4a7..a6aa198b 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -201,7 +201,10 @@ func (r *OracleRestDataServiceReconciler) validate(m *dbapi.OracleRestDataServic var err error eventReason := "Spec Error" var eventMsgs []string - + // If using same pvc for ords as sidb, ensure sidb has ReadWriteMany Accessmode + if n.Spec.Persistence.AccessMode == "ReadWriteOnce" && m.Spec.Persistence.AccessMode == "" { + eventMsgs = append(eventMsgs, "ords can be installed only on ReadWriteMany Access Mode of : "+m.Spec.DatabaseRef) + } if m.Status.DatabaseRef != "" && m.Status.DatabaseRef != m.Spec.DatabaseRef { eventMsgs = append(eventMsgs, "databaseRef cannot be updated") } From 67a9f76a715acf739913566d24f14091ad4cc0ae Mon Sep 17 00:00:00 2001 From: Susmita Samanta Date: Thu, 3 Mar 2022 09:01:56 +0530 Subject: [PATCH 176/628] fixed serviceAccountName key error --- config/samples/sidb/oraclerestdataservice.yaml | 2 +- config/samples/sidb/singleinstancedatabase.yaml | 2 +- config/samples/sidb/singleinstancedatabase_clone.yaml | 2 +- config/samples/sidb/singleinstancedatabase_patch.yaml | 2 +- config/samples/sidb/singleinstancedatabase_prov.yaml | 2 +- .../samples/sidb/singleinstancedatabase_prov_prebuilt_db.yaml | 2 +- 6 files changed, 6 insertions(+), 6 deletions(-) diff --git a/config/samples/sidb/oraclerestdataservice.yaml b/config/samples/sidb/oraclerestdataservice.yaml index 6652fdae..83a684ea 100644 --- a/config/samples/sidb/oraclerestdataservice.yaml +++ b/config/samples/sidb/oraclerestdataservice.yaml @@ -63,4 +63,4 @@ spec: pdb: ## If deploying on OpenShift, change service account name to 'sidb-sa' after you run `$ oc apply -f openshift_rbac.yaml` - ServiceAccountname: default \ No newline at end of file + serviceAccountName: default \ No newline at end of file diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index 8383c6f4..b2bc5b42 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -84,7 +84,7 @@ spec: # pool: sidb ## If deploying on OpenShift, change service account name to 'sidb-sa' after you run `$ oc apply -f openshift_rbac.yaml` - ServiceAccountname: default + serviceAccountName: default ## Count of Database Pods. Applicable only for "ReadWriteMany" AccessMode replicas: 1 diff --git a/config/samples/sidb/singleinstancedatabase_clone.yaml b/config/samples/sidb/singleinstancedatabase_clone.yaml index ad02f147..a29dce78 100644 --- a/config/samples/sidb/singleinstancedatabase_clone.yaml +++ b/config/samples/sidb/singleinstancedatabase_clone.yaml @@ -40,7 +40,7 @@ spec: accessMode: "ReadWriteOnce" ## If deploying on OpenShift, change service account name to 'sidb-sa' after you run `$ oc apply -f openshift_rbac.yaml` - ServiceAccountname: default + serviceAccountName: default ## Count of Database Pods. Applicable only for "ReadWriteMany" AccessMode replicas: 1 diff --git a/config/samples/sidb/singleinstancedatabase_patch.yaml b/config/samples/sidb/singleinstancedatabase_patch.yaml index 13a6df96..173e25b2 100644 --- a/config/samples/sidb/singleinstancedatabase_patch.yaml +++ b/config/samples/sidb/singleinstancedatabase_patch.yaml @@ -34,7 +34,7 @@ spec: accessMode: "ReadWriteOnce" ## If deploying on OpenShift, change service account name to 'sidb-sa' after you run `$ oc apply -f openshift_rbac.yaml` - ServiceAccountname: default + serviceAccountName: default ## Count of Database Pods. Applicable only for "ReadWriteMany" AccessMode replicas: 1 diff --git a/config/samples/sidb/singleinstancedatabase_prov.yaml b/config/samples/sidb/singleinstancedatabase_prov.yaml index b68a462a..616d2528 100644 --- a/config/samples/sidb/singleinstancedatabase_prov.yaml +++ b/config/samples/sidb/singleinstancedatabase_prov.yaml @@ -33,7 +33,7 @@ spec: accessMode: "ReadWriteOnce" ## If deploying on OpenShift, change service account name to 'sidb-sa' after you run `$ oc apply -f openshift_rbac.yaml` - ServiceAccountname: default + serviceAccountName: default ## Count of Database Pods. Applicable only for "ReadWriteMany" AccessMode replicas: 1 diff --git a/config/samples/sidb/singleinstancedatabase_prov_prebuilt_db.yaml b/config/samples/sidb/singleinstancedatabase_prov_prebuilt_db.yaml index f8eaa587..a24f51af 100644 --- a/config/samples/sidb/singleinstancedatabase_prov_prebuilt_db.yaml +++ b/config/samples/sidb/singleinstancedatabase_prov_prebuilt_db.yaml @@ -15,7 +15,7 @@ spec: pullSecrets: ## If deploying on OpenShift, change service account name to 'sidb-sa' after you run `$ oc apply -f openshift_rbac.yaml` - ServiceAccountname: default + serviceAccountName: default ## Type of service . Applicable on cloud enviroments only ## if loadBalService : false, service type = "NodePort". else "LoadBalancer" From 8ea2ec03de4754f414f054b4f5ec348e7d9b8e81 Mon Sep 17 00:00:00 2001 From: Gajanan Bhat Date: Tue, 8 Mar 2022 15:42:13 +0530 Subject: [PATCH 177/628] Added Map feature --- apis/database/v1alpha1/pdb_types.go | 8 ++- .../crd/bases/database.oracle.com_pdbs.yaml | 9 ++- controllers/database/pdb_controller.go | 58 ++++++++++++++++++- oracle-database-operator.yaml | 9 ++- 4 files changed, 76 insertions(+), 8 deletions(-) diff --git a/apis/database/v1alpha1/pdb_types.go b/apis/database/v1alpha1/pdb_types.go index bfa9897d..03f63f01 100644 --- a/apis/database/v1alpha1/pdb_types.go +++ b/apis/database/v1alpha1/pdb_types.go @@ -95,8 +95,8 @@ type PDBSpec struct { TDESecret TDESecret `json:"tdeSecret,omitempty"` // Whether you need the script only or execute the script GetScript *bool `json:"getScript,omitempty"` - // Action to be taken: Create or Clone or Plug or Unplug - // +kubebuilder:validation:Enum=Create;Clone;Plug;Unplug;Delete;Modify;Status + // Action to be taken: Create/Clone/Plug/Unplug/Delete/Modify/Status/Map. Map is used to map a Databse PDB to a Kubernetes PDB CR. + // +kubebuilder:validation:Enum=Create;Clone;Plug;Unplug;Delete;Modify;Status;Map Action string `json:"action"` // Extra options for opening and closing a PDB // +kubebuilder:validation:Enum=IMMEDIATE;NORMAL;READ ONLY;READ WRITE;RESTRICTED @@ -143,6 +143,8 @@ type PDBStatus struct { Phase string `json:"phase"` // PDB Resource Status Status bool `json:"status"` + // Total size of the PDB + TotalSize string `json:"totalSize,omitempty"` // Open mode of the PDB OpenMode string `json:"openMode,omitempty"` // Modify Option of the PDB @@ -159,7 +161,7 @@ type PDBStatus struct { // +kubebuilder:printcolumn:JSONPath=".spec.cdbName",name="CDB Name",type="string",description="Name of the CDB" // +kubebuilder:printcolumn:JSONPath=".spec.pdbName",name="PDB Name",type="string",description="Name of the PDB" // +kubebuilder:printcolumn:JSONPath=".status.openMode",name="PDB State",type="string",description="PDB Open Mode" -// +kubebuilder:printcolumn:JSONPath=".spec.totalSize",name="PDB Size",type="string",description="Total Size of the PDB" +// +kubebuilder:printcolumn:JSONPath=".status.totalSize",name="PDB Size",type="string",description="Total Size of the PDB" // +kubebuilder:printcolumn:JSONPath=".status.phase",name="Status",type="string",description="Status of the PDB Resource" // +kubebuilder:printcolumn:JSONPath=".status.msg",name="Message",type="string",description="Error message, if any" // PDB is the Schema for the pdbs API diff --git a/config/crd/bases/database.oracle.com_pdbs.yaml b/config/crd/bases/database.oracle.com_pdbs.yaml index d80a4fcc..113b2b4a 100644 --- a/config/crd/bases/database.oracle.com_pdbs.yaml +++ b/config/crd/bases/database.oracle.com_pdbs.yaml @@ -34,7 +34,7 @@ spec: name: PDB State type: string - description: Total Size of the PDB - jsonPath: .spec.totalSize + jsonPath: .status.totalSize name: PDB Size type: string - description: Status of the PDB Resource @@ -66,7 +66,8 @@ spec: description: PDBSpec defines the desired state of PDB properties: action: - description: 'Action to be taken: Create or Clone or Plug or Unplug' + description: 'Action to be taken: Create/Clone/Plug/Unplug/Delete/Modify/Status/Map. + Map is used to map a Databse PDB to a Kubernetes PDB CR.' enum: - Create - Clone @@ -75,6 +76,7 @@ spec: - Delete - Modify - Status + - Map type: string adminName: description: The administrator username for the new PDB. This property @@ -270,6 +272,9 @@ spec: status: description: PDB Resource Status type: boolean + totalSize: + description: Total size of the PDB + type: string required: - phase - status diff --git a/controllers/database/pdb_controller.go b/controllers/database/pdb_controller.go index 11535681..931ec556 100644 --- a/controllers/database/pdb_controller.go +++ b/controllers/database/pdb_controller.go @@ -109,6 +109,7 @@ var ( pdbPhaseReady = "Ready" pdbPhaseDelete = "Deleting" pdbPhaseModify = "Modifying" + pdbPhaseMap = "Mapping" pdbPhaseStatus = "CheckingState" pdbPhaseFail = "Failed" ) @@ -184,6 +185,7 @@ func (r *PDBReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.R action := strings.ToUpper(pdb.Spec.Action) if pdb.Status.Phase == pdbPhaseReady { + //log.Info("PDB:", "Name", pdb.Name, "Phase", pdb.Status.Phase, "Status", strconv.FormatBool(pdb.Status.Status)) if (pdb.Status.Action != "") && (action == "MODIFY" || action == "STATUS" || pdb.Status.Action != action) { pdb.Status.Status = false } else { @@ -218,6 +220,8 @@ func (r *PDBReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.R err = r.deletePDB(ctx, req, pdb) case pdbPhaseStatus: err = r.getPDBState(ctx, req, pdb) + case pdbPhaseMap: + err = r.mapPDB(ctx, req, pdb) default: log.Info("DEFAULT:", "Name", pdb.Name, "Phase", phase, "Status", strconv.FormatBool(pdb.Status.Status)) return requeueN, nil @@ -261,6 +265,8 @@ func (r *PDBReconciler) validatePhase(ctx context.Context, req ctrl.Request, pdb pdb.Status.Phase = pdbPhaseDelete case "STATUS": pdb.Status.Phase = pdbPhaseStatus + case "MAP": + pdb.Status.Phase = pdbPhaseMap } log.Info("Validation complete") @@ -559,6 +565,7 @@ func (r *PDBReconciler) createPDB(ctx context.Context, req ctrl.Request, pdb *db url := "http://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdb.Status.TotalSize = pdb.Spec.TotalSize pdb.Status.Phase = pdbPhaseCreate pdb.Status.Msg = "Waiting for PDB to be created" if err := r.Status().Update(ctx, pdb); err != nil { @@ -680,6 +687,7 @@ func (r *PDBReconciler) plugPDB(ctx context.Context, req ctrl.Request, pdb *dbap url := "http://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdb.Status.TotalSize = pdb.Spec.TotalSize pdb.Status.Phase = pdbPhasePlug pdb.Status.Msg = "Waiting for PDB to be plugged" if err := r.Status().Update(ctx, pdb); err != nil { @@ -859,6 +867,52 @@ func (r *PDBReconciler) getPDBState(ctx context.Context, req ctrl.Request, pdb * return nil } +/************************************************* + * Map Database PDB to Kubernetes PDB CR + /************************************************/ +func (r *PDBReconciler) mapPDB(ctx context.Context, req ctrl.Request, pdb *dbapi.PDB) error { + + log := r.Log.WithValues("mapPDB", req.NamespacedName) + + var err error + + cdb, err := r.getCDBResource(ctx, req, pdb) + if err != nil { + return err + } + + pdbName := pdb.Spec.PDBName + url := "http://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdbName + "/" + + pdb.Status.Msg = "Mapping PDB" + if err := r.Status().Update(ctx, pdb); err != nil { + log.Error(err, "Failed to update status for :"+pdb.Name, "err", err.Error()) + } + + respData, err := r.callAPI(ctx, req, pdb, url, nil, "GET") + + if err != nil { + pdb.Status.OpenMode = "UNKNOWN" + return err + } + + var objmap map[string]interface{} + if err := json.Unmarshal([]byte(respData), &objmap); err != nil { + log.Error(err, "Failed to get state of PDB :"+pdbName, "err", err.Error()) + } + + //fmt.Printf("%+v\n", objmap) + totSizeInBytes := objmap["total_size"].(float64) + totSizeInGB := totSizeInBytes / 1024 / 1024 / 1024 + + pdb.Status.OpenMode = objmap["open_mode"].(string) + pdb.Status.TotalSize = fmt.Sprintf("%.2f", totSizeInGB) + "G" + pdb.Status.ConnString = cdb.Spec.SCANName + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName + + log.Info("Successfully mapped PDB to Kubernetes resource", "PDB Name", pdb.Spec.PDBName) + return nil +} + /************************************************* * Delete a PDB /************************************************/ @@ -906,6 +960,9 @@ func (r *PDBReconciler) managePDBDeletion(ctx context.Context, req ctrl.Request, if isPDBMarkedToBeDeleted { log.Info("Marked to be deleted") pdb.Status.Phase = pdbPhaseDelete + pdb.Status.Status = true + r.Status().Update(ctx, pdb) + if controllerutil.ContainsFinalizer(pdb, PDBFinalizer) { // Remove PDBFinalizer. Once all finalizers have been // removed, the object will be deleted. @@ -916,7 +973,6 @@ func (r *PDBReconciler) managePDBDeletion(ctx context.Context, req ctrl.Request, log.Info("Could not remove finalizer", "err", err.Error()) return err } - pdb.Status.Status = true log.Info("Successfully removed PDB resource") return nil } diff --git a/oracle-database-operator.yaml b/oracle-database-operator.yaml index 72d50548..1cebb4bf 100644 --- a/oracle-database-operator.yaml +++ b/oracle-database-operator.yaml @@ -679,7 +679,7 @@ spec: name: PDB State type: string - description: Total Size of the PDB - jsonPath: .spec.totalSize + jsonPath: .status.totalSize name: PDB Size type: string - description: Status of the PDB Resource @@ -711,7 +711,8 @@ spec: description: PDBSpec defines the desired state of PDB properties: action: - description: 'Action to be taken: Create or Clone or Plug or Unplug' + description: 'Action to be taken: Create/Clone/Plug/Unplug/Delete/Modify/Status/Map. + Map is used to map a Databse PDB to a Kubernetes PDB CR.' enum: - Create - Clone @@ -720,6 +721,7 @@ spec: - Delete - Modify - Status + - Map type: string adminName: description: The administrator username for the new PDB. This property @@ -915,6 +917,9 @@ spec: status: description: PDB Resource Status type: boolean + totalSize: + description: Total size of the PDB + type: string required: - phase - status From 5aeb5a989c7d1a4893fb0b5fdbc510784b77ce51 Mon Sep 17 00:00:00 2001 From: Gajanan Bhat Date: Tue, 8 Mar 2022 21:42:26 +0530 Subject: [PATCH 178/628] Updated README --- config/samples/onpremdb/pdb_map.yaml | 16 ++++++++++++++++ config/samples/onpremdb/pdb_modify.yaml | 22 ++++++++++++++++++++++ docs/onpremdb/README.md | 12 +++++++++++- 3 files changed, 49 insertions(+), 1 deletion(-) create mode 100644 config/samples/onpremdb/pdb_map.yaml create mode 100644 config/samples/onpremdb/pdb_modify.yaml diff --git a/config/samples/onpremdb/pdb_map.yaml b/config/samples/onpremdb/pdb_map.yaml new file mode 100644 index 00000000..be543f75 --- /dev/null +++ b/config/samples/onpremdb/pdb_map.yaml @@ -0,0 +1,16 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb3 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbName: "democdb" + pdbName: "demotest" + action: "Map" \ No newline at end of file diff --git a/config/samples/onpremdb/pdb_modify.yaml b/config/samples/onpremdb/pdb_modify.yaml new file mode 100644 index 00000000..7fc298ef --- /dev/null +++ b/config/samples/onpremdb/pdb_modify.yaml @@ -0,0 +1,22 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbName: "democdb" + pdbName: "demotest" + action: "Modify" + pdbState: "CLOSE" + modifyOption: "IMMEDIATE" + + # To Open an existing PDB, uncomment the below lines and comment the two lines above + #pdbState: "OPEN" + #modifyOption: "READ WRITE" \ No newline at end of file diff --git a/docs/onpremdb/README.md b/docs/onpremdb/README.md index 35d59e02..7427dbe0 100644 --- a/docs/onpremdb/README.md +++ b/docs/onpremdb/README.md @@ -13,7 +13,7 @@ The On-Premise Database Controller enables provisioning of Oracle Databases (PDB + ### Prepare CDB for PDB Lifecycme Management (PDB-LM) - Pluggable Database management is performed in the Container Database (CDB) and includes create, clone, plug, unplug and delete operations. + Pluggable Database management is performed in the Container Database (CDB) and includes create, clone, plug, unplug, delete, modify and map operations. You cannot have an ORDS enabled schema in the container database. To perform the PDB lifecycle management operations, the default CDB administrator credentials must be defined. To define the default CDB administrator credentials, perform the following steps on the target CDB(s) where PDB-LM operations are to be performed: @@ -131,6 +131,16 @@ The On-Premise Database Controller enables provisioning of Oracle Databases (PDB A sample .yaml file is available here: [config/samples/onpremdb/pdb_delete.yaml](../../config/samples/onpremdb/pdb_delete.yaml) ++ ### Modify PDB + + This is used to open/close a target PDB. + A sample .yaml file is available here: [config/samples/onpremdb/pdb_modify.yaml](../../config/samples/onpremdb/pdb_modify.yaml) + ++ ### Map PDB + + This is used to map an existing PDB in the CDB as a Kubernetes Custom Resource + A sample .yaml file is available here: [config/samples/onpremdb/pdb_map.yaml](../../config/samples/onpremdb/pdb_map.yaml) + ## Validation and Errors You can check Kubernetes events for any errors or status updates as shown below: From eefe0441d136659d3ecdeb999bcb5fecc07498c8 Mon Sep 17 00:00:00 2001 From: Gajanan Bhat Date: Wed, 9 Mar 2022 12:36:19 +0530 Subject: [PATCH 179/628] Updated README --- docs/onpremdb/README.md | 26 ++++++++++++++++++++------ 1 file changed, 20 insertions(+), 6 deletions(-) diff --git a/docs/onpremdb/README.md b/docs/onpremdb/README.md index 7427dbe0..1b27ad22 100644 --- a/docs/onpremdb/README.md +++ b/docs/onpremdb/README.md @@ -71,7 +71,7 @@ The On-Premise Database Controller enables provisioning of Oracle Databases (PDB ## Kubernetes CRD for CDB - The Oracle Database Operator creates the CDB kind as a custom resource that models a target CDB as a native Kubernetes object. This is only used to create Pods to connect to the target CDB to perform PDB-LM operations. Each CDB resource follows the CDB CRD as defined here: [config/crd/bases/database.oracle.com_cdbs.yaml](../../config/crd/bases/database.oracle.com_cdbs.yaml) + The Oracle Database Operator creates the CDB kind as a custom resource that models a target CDB as a native Kubernetes object. This is only used to create Pods to connect to the target CDB to perform PDB-LM operations. These CDB resources can be scaled up and down based on the expected load using replicas. Each CDB resource follows the CDB CRD as defined here: [config/crd/bases/database.oracle.com_cdbs.yaml](../../config/crd/bases/database.oracle.com_cdbs.yaml) + ### CDB Sample YAML @@ -83,13 +83,17 @@ The On-Premise Database Controller enables provisioning of Oracle Databases (PDB ```sh $ kubectl get cdbs -A - NAMESPACE NAME CDB NAME DB SERVER DB PORT SCAN NAME STATUS MESSAGE - oracle-database-operator-system cdb-dev devdb 172.17.0.4 1521 devdb Ready Success + NAMESPACE NAME CDB NAME DB SERVER DB PORT SCAN NAME REPLICAS STATUS MESSAGE + oracle-database-operator-system cdb-dev devdb 172.17.0.4 1521 devdb 1 Ready Success + ``` + + ### Scale the CDB resource + ```sh + $ kubectl patch --type=merge cdb cdb-dev -p '{"spec":{"replicas":3}}' -n oracle-database-operator-system ``` ## Kubernetes CRD for PDB - The Oracle Database Operator creates the PDB kind as a custom resource that models a PDB as a native Kubernetes object. This PDB resource can be used to perform PDB-LM operations by specifying the action attribute in the PDB specs. Each PDB resource follows the PDB CRD as defined here: [config/crd/bases/database.oracle.com_pdbs.yaml](../../config/crd/bases/database.oracle.com_pdbs.yaml) + The Oracle Database Operator creates the PDB kind as a custom resource that models a PDB as a native Kubernetes object. There is a one-to-one mapping between the actual PDB and the Kubernetes PDB Custom Resource. You cannot have more than one Kubernetes resource for a target PDB. This PDB resource can be used to perform PDB-LM operations by specifying the action attribute in the PDB specs. Each PDB resource follows the PDB CRD as defined here: [config/crd/bases/database.oracle.com_pdbs.yaml](../../config/crd/bases/database.oracle.com_pdbs.yaml) + ### PDB Sample YAML @@ -131,14 +135,24 @@ The On-Premise Database Controller enables provisioning of Oracle Databases (PDB A sample .yaml file is available here: [config/samples/onpremdb/pdb_delete.yaml](../../config/samples/onpremdb/pdb_delete.yaml) + You can also use the following cmd to delete an existing PDB: + ```sh + $ kubectl patch --type=merge pdb pdb1 -p '{"spec":{"action":"Delete","dropAction":"INCLUDING"}}' -n oracle-database-operator-system + ``` + + ### Modify PDB This is used to open/close a target PDB. - A sample .yaml file is available here: [config/samples/onpremdb/pdb_modify.yaml](../../config/samples/onpremdb/pdb_modify.yaml) + A sample .yaml file is available here: [config/samples/onpremdb/pdb_modify.yaml](../../config/samples/onpremdb/pdb_modify.yaml) + + You can also use the following cmd to modify an existing PDB: + ```sh + $ kubectl patch --type=merge pdb pdb1 -p '{"spec":{"action":"Modify","modifyOption":"IMMEDIATE","pdbState":"CLOSE"}}' -n oracle-database-operator-system + ``` + ### Map PDB - This is used to map an existing PDB in the CDB as a Kubernetes Custom Resource + This is used to map an existing PDB in the CDB as a Kubernetes Custom Resource. A sample .yaml file is available here: [config/samples/onpremdb/pdb_map.yaml](../../config/samples/onpremdb/pdb_map.yaml) ## Validation and Errors From 0eff032699767b2e38f07195de9c718de1d7fbb7 Mon Sep 17 00:00:00 2001 From: Gajanan Bhat Date: Wed, 9 Mar 2022 12:39:37 +0530 Subject: [PATCH 180/628] Updated README --- oracle-database-operator.yaml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/oracle-database-operator.yaml b/oracle-database-operator.yaml index 15cbda72..e81f82f7 100644 --- a/oracle-database-operator.yaml +++ b/oracle-database-operator.yaml @@ -2235,7 +2235,7 @@ metadata: name: oracle-database-operator-controller-manager namespace: oracle-database-operator-system spec: - replicas: 1 + replicas: 3 selector: matchLabels: control-plane: controller-manager @@ -2249,8 +2249,8 @@ spec: - --enable-leader-election command: - /manager - image: phx.ocir.io/intsanjaysingh/gdb-repo/oracle/dboper-ords-go:master - imagePullPolicy: Never + image: container-registry.oracle.com/database/operator:0.1.0 + imagePullPolicy: Always name: manager ports: - containerPort: 9443 From dcc4ef193f28c7135baf958be0b820a39a72055a Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Wed, 9 Mar 2022 11:08:31 -0500 Subject: [PATCH 181/628] refactor ADB controllers, add EventRecorder --- .../v1alpha1/adbfamily_common_utils.go | 173 ++++ .../v1alpha1/autonomousdatabase_types.go | 168 +-- .../v1alpha1/autonomousdatabase_webhook.go | 77 +- .../autonomousdatabasebackup_types.go | 61 +- .../autonomousdatabasebackup_webhook.go | 34 +- .../autonomousdatabaserestore_types.go | 43 +- .../autonomousdatabaserestore_webhook.go | 13 +- .../v1alpha1/zz_generated.deepcopy.go | 63 +- commons/annotations/annotations.go | 1 + .../adb_backup_reconciler_util.go | 133 --- .../autonomousdatabase/adb_reconciler_util.go | 493 --------- .../adb_restore_reconciler_util.go | 141 --- commons/k8s/create.go | 104 ++ .../common_util.go => k8s/fetch.go} | 73 +- commons/{finalizer => k8s}/finalizer.go | 4 +- commons/k8s/utils.go | 61 ++ commons/oci/database.go | 708 ++++--------- commons/oci/ociutil/time.go | 23 - commons/oci/provider.go | 33 +- commons/oci/vault.go | 29 +- commons/oci/wallet.go | 113 +- commons/oci/workrequest.go | 131 +++ ....oracle.com_autonomousdatabasebackups.yaml | 6 - ...oracle.com_autonomousdatabaserestores.yaml | 9 +- ...tabase.oracle.com_autonomousdatabases.yaml | 116 ++- .../database/autonomousdatabase_controller.go | 976 +++++++++++++----- .../autonomousdatabasebackup_controller.go | 272 +++-- .../autonomousdatabaserestore_controller.go | 194 +++- main.go | 1 + ...tonomousdatabase_controller_create_test.go | 3 +- test/e2e/behavior/shared_behaviors.go | 4 +- test/e2e/suite_test.go | 1 + 32 files changed, 2183 insertions(+), 2078 deletions(-) create mode 100644 apis/database/v1alpha1/adbfamily_common_utils.go delete mode 100644 commons/autonomousdatabase/adb_backup_reconciler_util.go delete mode 100644 commons/autonomousdatabase/adb_reconciler_util.go delete mode 100644 commons/autonomousdatabase/adb_restore_reconciler_util.go create mode 100644 commons/k8s/create.go rename commons/{autonomousdatabase/common_util.go => k8s/fetch.go} (72%) rename commons/{finalizer => k8s}/finalizer.go (98%) create mode 100644 commons/k8s/utils.go delete mode 100644 commons/oci/ociutil/time.go create mode 100644 commons/oci/workrequest.go diff --git a/apis/database/v1alpha1/adbfamily_common_utils.go b/apis/database/v1alpha1/adbfamily_common_utils.go new file mode 100644 index 00000000..cbb97316 --- /dev/null +++ b/apis/database/v1alpha1/adbfamily_common_utils.go @@ -0,0 +1,173 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v1alpha1 + +import ( + "errors" + "reflect" + "time" + + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "sigs.k8s.io/controller-runtime/pkg/client" + + "github.com/oracle/oci-go-sdk/v54/common" +) + +// File the meta condition and return the meta view +func CreateMetaCondition(obj client.Object, err error, lifecycleState string, stateMsg string) metav1.Condition { + + return metav1.Condition{ + Type: lifecycleState, + LastTransitionTime: metav1.Now(), + ObservedGeneration: obj.GetGeneration(), + Reason: stateMsg, + Message: err.Error(), + Status: metav1.ConditionTrue, + } +} + +// LastSuccessfulSpec is an annotation key which maps to the value of last successful spec +const LastSuccessfulSpec string = "lastSuccessfulSpec" + +type OCIConfigSpec struct { + ConfigMapName *string `json:"configMapName,omitempty"` + SecretName *string `json:"secretName,omitempty"` +} + +// removeUnchangedFields removes the unchanged fields in the struct and returns if the struct is changed. +// lastSpec should be a derefereced struct that is the last successful spec, e.g. AutonomousDatabaseSpec. +// curSpec should be a pointer pointing to the struct that is being proccessed, e.g., *AutonomousDatabaseSpec. +func removeUnchangedFields(lastSpec interface{}, curSpec interface{}) (bool, error) { + if reflect.ValueOf(lastSpec).Kind() != reflect.Struct { + return false, errors.New("lastSpec should be a struct") + } + + if reflect.ValueOf(curSpec).Kind() != reflect.Ptr || reflect.ValueOf(curSpec).Elem().Kind() != reflect.Struct { + return false, errors.New("curSpec should be a struct pointer") + } + + if reflect.ValueOf(lastSpec).Type() != reflect.ValueOf(curSpec).Elem().Type() { + return false, errors.New("the referenced type of curSpec should be the same as the type of lastSpec") + } + + return traverse(lastSpec, curSpec), nil +} + +// Traverse and compare each fields in the lastSpec and the the curSpec. +// If unchanged, set the field in curSpec to a zero value. +// lastSpec should be a derefereced struct that is the last successful spec, e.g. AutonomousDatabaseSpec. +// curSpec should be a pointer pointing to the struct that is being proccessed, e.g., *AutonomousDatabaseSpec. +func traverse(lastSpec interface{}, curSpec interface{}) bool { + var changed bool = false + + fields := reflect.VisibleFields(reflect.TypeOf(lastSpec)) + + lastSpecValue := reflect.ValueOf(lastSpec) + curSpecValue := reflect.ValueOf(curSpec).Elem() // deref the struct + + for _, field := range fields { + lastField := lastSpecValue.FieldByName(field.Name) + curField := curSpecValue.FieldByName(field.Name) + + // call traverse() if the current field is a struct + if field.Type.Kind() == reflect.Struct { + childrenChanged := traverse(lastField.Interface(), curField.Addr().Interface()) + if childrenChanged && !changed { + changed = true + } + } else { + fieldChanged := hasChanged(lastField, curField) + + if fieldChanged && !changed { + changed = true + } + + // Set the field to zero value if unchanged + if !fieldChanged { + curField.Set(reflect.Zero(curField.Type())) + } + } + } + + return changed +} + +// 1. If the current field is with a zero value, then the field is unchanged. +// 2. If the current field is NOT with a zero value, then we want to comapre it with the last field. +// In this case if the last field is with a zero value, then the field is changed +func hasChanged(lastField reflect.Value, curField reflect.Value) bool { + zero := reflect.Zero(lastField.Type()).Interface() + lastFieldIsZero := reflect.DeepEqual(lastField.Interface(), zero) + curFieldIsZero := reflect.DeepEqual(curField.Interface(), zero) + + if curFieldIsZero { + return false + } else if !lastFieldIsZero { + var lastIntrf interface{} + var curIntrf interface{} + + if curField.Kind() == reflect.Ptr { + lastIntrf = lastField.Elem().Interface() + curIntrf = curField.Elem().Interface() + } else { + lastIntrf = lastField.Interface() + curIntrf = curField.Interface() + } + + return !reflect.DeepEqual(lastIntrf, curIntrf) + } + + return true +} + +// Follow the format of the display time +const displayFormat = "2006-01-02 15:04:05 MST" + +func formatSDKTime(dateTime time.Time) string { + return dateTime.Format(displayFormat) +} + +func parseDisplayTime(val string) (*common.SDKTime, error) { + parsedTime, err := time.Parse(displayFormat, val) + if err != nil { + return nil, err + } + sdkTime := common.SDKTime{Time: parsedTime} + return &sdkTime, nil +} diff --git a/apis/database/v1alpha1/autonomousdatabase_types.go b/apis/database/v1alpha1/autonomousdatabase_types.go index 9ce6c010..1dfd8384 100644 --- a/apis/database/v1alpha1/autonomousdatabase_types.go +++ b/apis/database/v1alpha1/autonomousdatabase_types.go @@ -40,13 +40,11 @@ package v1alpha1 import ( "encoding/json" - "strconv" + "errors" "github.com/oracle/oci-go-sdk/v54/database" + "k8s.io/apimachinery/pkg/api/meta" metaV1 "k8s.io/apimachinery/pkg/apis/meta/v1" - "sigs.k8s.io/controller-runtime/pkg/client" - - "github.com/oracle/oracle-database-operator/commons/annotations" ) // EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! @@ -61,11 +59,6 @@ type AutonomousDatabaseSpec struct { HardLink *bool `json:"hardLink,omitempty"` } -type OCIConfigSpec struct { - ConfigMapName *string `json:"configMapName,omitempty"` - SecretName *string `json:"secretName,omitempty"` -} - // AutonomousDatabaseDetails defines the detail information of AutonomousDatabase, corresponding to oci-go-sdk/database/AutonomousDatabase type AutonomousDatabaseDetails struct { AutonomousDatabaseOCID *string `json:"autonomousDatabaseOCID,omitempty"` @@ -74,13 +67,15 @@ type AutonomousDatabaseDetails struct { DisplayName *string `json:"displayName,omitempty"` DbName *string `json:"dbName,omitempty"` // +kubebuilder:validation:Enum:="OLTP";"DW";"AJD";"APEX" - DbWorkload database.AutonomousDatabaseDbWorkloadEnum `json:"dbWorkload,omitempty"` - IsDedicated *bool `json:"isDedicated,omitempty"` + DbWorkload database.AutonomousDatabaseDbWorkloadEnum `json:"dbWorkload,omitempty"` + // +kubebuilder:validation:Enum:="LICENSE_INCLUDED";"BRING_YOUR_OWN_LICENSE" + LicenseModel database.AutonomousDatabaseLicenseModelEnum `json:"licenseModel,omitempty"` DbVersion *string `json:"dbVersion,omitempty"` DataStorageSizeInTBs *int `json:"dataStorageSizeInTBs,omitempty"` CPUCoreCount *int `json:"cpuCoreCount,omitempty"` AdminPassword PasswordSpec `json:"adminPassword,omitempty"` IsAutoScalingEnabled *bool `json:"isAutoScalingEnabled,omitempty"` + IsDedicated *bool `json:"isDedicated,omitempty"` LifecycleState database.AutonomousDatabaseLifecycleStateEnum `json:"lifecycleState,omitempty"` NetworkAccess NetworkAccessSpec `json:"networkAccess,omitempty"` @@ -127,38 +122,44 @@ type PrivateEndpointSpec struct { type AutonomousDatabaseStatus struct { // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster // Important: Run "make" to regenerate code after modifying this file - DisplayName string `json:"displayName,omitempty"` LifecycleState database.AutonomousDatabaseLifecycleStateEnum `json:"lifecycleState,omitempty"` - IsDedicated string `json:"isDedicated,omitempty"` - CPUCoreCount int `json:"cpuCoreCount,omitempty"` - DataStorageSizeInTBs int `json:"dataStorageSizeInTBs,omitempty"` - DbWorkload database.AutonomousDatabaseDbWorkloadEnum `json:"dbWorkload,omitempty"` TimeCreated string `json:"timeCreated,omitempty"` - AllConnectionStrings []ConnectionStringsSet `json:"allConnectionStrings,omitempty"` + AllConnectionStrings []ConnectionStringProfile `json:"allConnectionStrings,omitempty"` + + // +patchMergeKey=type + // +patchStrategy=merge + // +listType=map + // +listMapKey=type + Conditions []metaV1.Condition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type"` } type TLSAuthenticationEnum string const ( - TLSAuthenticationTLS TLSAuthenticationEnum = "TLS" - TLSAuthenticationmTLS TLSAuthenticationEnum = "Mutual TLS" + tlsAuthenticationTLS TLSAuthenticationEnum = "TLS" + tlsAuthenticationMTLS TLSAuthenticationEnum = "Mutual TLS" ) -type ConnectionStringsSet struct { - TLSAuthentication TLSAuthenticationEnum `json:"tlsAuthentication,omitempty"` - ConnectionStrings map[string]string `json:"connectionStrings"` +type ConnectionStringProfile struct { + TLSAuthentication TLSAuthenticationEnum `json:"tlsAuthentication,omitempty"` + ConnectionStrings []ConnectionStringSpec `json:"connectionStrings"` +} + +type ConnectionStringSpec struct { + TNSName string `json:"tnsName,omitempty"` + ConnectionString string `json:"connectionString,omitempty"` } // AutonomousDatabase is the Schema for the autonomousdatabases API // +kubebuilder:object:root=true // +kubebuilder:resource:shortName="adb";"adbs" // +kubebuilder:subresource:status -// +kubebuilder:printcolumn:JSONPath=".status.displayName",name="Display Name",type=string +// +kubebuilder:printcolumn:JSONPath=".spec.details.displayName",name="Display Name",type=string // +kubebuilder:printcolumn:JSONPath=".status.lifecycleState",name="State",type=string -// +kubebuilder:printcolumn:JSONPath=".status.isDedicated",name="Dedicated",type=string -// +kubebuilder:printcolumn:JSONPath=".status.cpuCoreCount",name="OCPUs",type=integer -// +kubebuilder:printcolumn:JSONPath=".status.dataStorageSizeInTBs",name="Storage (TB)",type=integer -// +kubebuilder:printcolumn:JSONPath=".status.dbWorkload",name="Workload Type",type=string +// +kubebuilder:printcolumn:JSONPath=".spec.details.isDedicated",name="Dedicated",type=string +// +kubebuilder:printcolumn:JSONPath=".spec.details.cpuCoreCount",name="OCPUs",type=integer +// +kubebuilder:printcolumn:JSONPath=".spec.details.dataStorageSizeInTBs",name="Storage (TB)",type=integer +// +kubebuilder:printcolumn:JSONPath=".spec.details.dbWorkload",name="Workload Type",type=string // +kubebuilder:printcolumn:JSONPath=".status.timeCreated",name="Created",type=string type AutonomousDatabase struct { metaV1.TypeMeta `json:",inline"` @@ -168,8 +169,18 @@ type AutonomousDatabase struct { Status AutonomousDatabaseStatus `json:"status,omitempty"` } -// LastSuccessfulSpec is an annotation key which maps to the value of last successful spec -const LastSuccessfulSpec string = "lastSuccessfulSpec" +// +kubebuilder:object:root=true + +// AutonomousDatabaseList contains a list of AutonomousDatabase +type AutonomousDatabaseList struct { + metaV1.TypeMeta `json:",inline"` + metaV1.ListMeta `json:"metadata,omitempty"` + Items []AutonomousDatabase `json:"items"` +} + +func init() { + SchemeBuilder.Register(&AutonomousDatabase{}, &AutonomousDatabaseList{}) +} // GetLastSuccessfulSpec returns spec from the lass successful reconciliation. // Returns nil, nil if there is no lastSuccessfulSpec. @@ -190,22 +201,18 @@ func (adb *AutonomousDatabase) GetLastSuccessfulSpec() (*AutonomousDatabaseSpec, return &sucSpec, nil } -// UpdateLastSuccessfulSpec updates lastSuccessfulSpec with the current spec. -func (adb *AutonomousDatabase) UpdateLastSuccessfulSpec(kubeClient client.Client) error { - specBytes, err := json.Marshal(adb.Spec) - if err != nil { - return err - } +func (adb *AutonomousDatabase) UpdateConditions(lifecycleState database.AutonomousDatabaseLifecycleStateEnum, err error) { + var metaCondition metaV1.Condition - anns := map[string]string{ - LastSuccessfulSpec: string(specBytes), + if err != nil { + metaCondition = CreateMetaCondition(adb, err, string(CrdReconcileErrorState), string(CrdReconcileErrorReason)) } - return annotations.SetAnnotations(kubeClient, adb, anns) + meta.SetStatusCondition(&adb.Status.Conditions, metaCondition) } // UpdateAttrFromOCIAutonomousDatabase updates the attributes from database.AutonomousDatabase object and returns the resource -func (adb *AutonomousDatabase) UpdateAttrFromOCIAutonomousDatabase(ociObj database.AutonomousDatabase) *AutonomousDatabase { +func (adb *AutonomousDatabase) UpdateFromOCIADB(ociObj database.AutonomousDatabase) { /*********************************** * update the spec ***********************************/ @@ -215,14 +222,16 @@ func (adb *AutonomousDatabase) UpdateAttrFromOCIAutonomousDatabase(ociObj databa adb.Spec.Details.DisplayName = ociObj.DisplayName adb.Spec.Details.DbName = ociObj.DbName adb.Spec.Details.DbWorkload = ociObj.DbWorkload - adb.Spec.Details.IsDedicated = ociObj.IsDedicated + adb.Spec.Details.LicenseModel = ociObj.LicenseModel adb.Spec.Details.DbVersion = ociObj.DbVersion adb.Spec.Details.DataStorageSizeInTBs = ociObj.DataStorageSizeInTBs adb.Spec.Details.CPUCoreCount = ociObj.CpuCoreCount adb.Spec.Details.IsAutoScalingEnabled = ociObj.IsAutoScalingEnabled + adb.Spec.Details.IsDedicated = ociObj.IsDedicated adb.Spec.Details.LifecycleState = ociObj.LifecycleState adb.Spec.Details.FreeformTags = ociObj.FreeformTags + // Determine network.accessType if *ociObj.IsDedicated { adb.Spec.Details.NetworkAccess.AccessType = NetworkAccessTypePrivate } else { @@ -245,65 +254,68 @@ func (adb *AutonomousDatabase) UpdateAttrFromOCIAutonomousDatabase(ociObj databa /*********************************** * update the status subresource ***********************************/ - adb.Status.DisplayName = *ociObj.DisplayName + adb.Status.LifecycleState = ociObj.LifecycleState - adb.Status.IsDedicated = strconv.FormatBool(*ociObj.IsDedicated) - adb.Status.CPUCoreCount = *ociObj.CpuCoreCount - adb.Status.DataStorageSizeInTBs = *ociObj.DataStorageSizeInTBs - adb.Status.DbWorkload = ociObj.DbWorkload adb.Status.TimeCreated = ociObj.TimeCreated.String() - var curAlllConns []ConnectionStringsSet if *ociObj.IsDedicated { - connSet := ConnectionStringsSet{ConnectionStrings: ociObj.ConnectionStrings.AllConnectionStrings} - curAlllConns = append(curAlllConns, connSet) + conns := make([]ConnectionStringSpec, len(ociObj.ConnectionStrings.AllConnectionStrings)) + for key, val := range ociObj.ConnectionStrings.AllConnectionStrings { + conns = append(conns, ConnectionStringSpec{TNSName: key, ConnectionString: val}) + } + adb.Status.AllConnectionStrings = []ConnectionStringProfile{ + ConnectionStringProfile{ConnectionStrings: conns}, + } } else { - mTLSStrings := make(map[string]string) - tlsStrings := make(map[string]string) + var mTLSConns []ConnectionStringSpec + var tlsConns []ConnectionStringSpec + + var conns []ConnectionStringProfile for _, profile := range ociObj.ConnectionStrings.Profiles { if profile.TlsAuthentication == database.DatabaseConnectionStringProfileTlsAuthenticationMutual { - mTLSStrings[*profile.DisplayName] = *profile.Value + mTLSConns = append(mTLSConns, ConnectionStringSpec{TNSName: *profile.DisplayName, ConnectionString: *profile.Value}) } else { - tlsStrings[*profile.DisplayName] = *profile.Value + tlsConns = append(tlsConns, ConnectionStringSpec{TNSName: *profile.DisplayName, ConnectionString: *profile.Value}) } } - if len(mTLSStrings) > 0 { - mTLSConnSet := ConnectionStringsSet{ - TLSAuthentication: TLSAuthenticationmTLS, - ConnectionStrings: mTLSStrings, - } - - curAlllConns = append(curAlllConns, mTLSConnSet) + if len(mTLSConns) > 0 { + conns = append(conns, ConnectionStringProfile{ + TLSAuthentication: tlsAuthenticationMTLS, + ConnectionStrings: mTLSConns, + }) } - if len(tlsStrings) > 0 { - tlsConnSet := ConnectionStringsSet{ - TLSAuthentication: TLSAuthenticationTLS, - ConnectionStrings: tlsStrings, - } - - curAlllConns = append(curAlllConns, tlsConnSet) + if len(tlsConns) > 0 { + conns = append(conns, ConnectionStringProfile{ + TLSAuthentication: tlsAuthenticationTLS, + ConnectionStrings: tlsConns, + }) } - } - adb.Status.AllConnectionStrings = curAlllConns - return adb + adb.Status.AllConnectionStrings = conns + } } -// +kubebuilder:object:root=true +// RemoveUnchangedDetails removes the unchanged fields in spec.details, and returns if the details has been changed. +// details.autonomousDatabaseOCID won't be affected because we need it to send requests. +// A `false` is returned if the lastSucSpec is nil. +func (adb *AutonomousDatabase) RemoveUnchangedDetails() (bool, error) { + lastSucSpec, err := adb.GetLastSuccessfulSpec() + if lastSucSpec == nil { + return false, errors.New("lastSucSpec is nil") + } -// AutonomousDatabaseList contains a list of AutonomousDatabase -type AutonomousDatabaseList struct { - metaV1.TypeMeta `json:",inline"` - metaV1.ListMeta `json:"metadata,omitempty"` - Items []AutonomousDatabase `json:"items"` -} + changed, err := removeUnchangedFields(lastSucSpec.Details, &adb.Spec.Details) + if err != nil { + return changed, err + } -func init() { - SchemeBuilder.Register(&AutonomousDatabase{}, &AutonomousDatabaseList{}) + adb.Spec.Details.AutonomousDatabaseOCID = lastSucSpec.Details.AutonomousDatabaseOCID + + return changed, nil } // A helper function which is useful for debugging. The function prints out a structural JSON format. diff --git a/apis/database/v1alpha1/autonomousdatabase_webhook.go b/apis/database/v1alpha1/autonomousdatabase_webhook.go index 09e10807..434bee74 100644 --- a/apis/database/v1alpha1/autonomousdatabase_webhook.go +++ b/apis/database/v1alpha1/autonomousdatabase_webhook.go @@ -40,6 +40,7 @@ package v1alpha1 import ( "fmt" + "reflect" apierrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/runtime" @@ -74,11 +75,6 @@ func (r *AutonomousDatabase) Default() { if r.Spec.Details.NetworkAccess.AccessType == "" { r.Spec.Details.NetworkAccess.AccessType = NetworkAccessTypePublic } - - // IsAccessControlEnabled is not applicable only to a shared database - if r.Spec.Details.NetworkAccess.IsAccessControlEnabled != nil { - r.Spec.Details.NetworkAccess.IsAccessControlEnabled = nil - } } else { // Dedicated database // AccessType can only be PRIVATE for a dedicated database r.Spec.Details.NetworkAccess.AccessType = NetworkAccessTypePrivate @@ -100,7 +96,14 @@ func (r *AutonomousDatabase) ValidateCreate() error { autonomousdatabaselog.Info("validate create", "name", r.Name) if r.Spec.Details.AutonomousDatabaseOCID == nil { // provisioning operation + allErrs = validateCommon(r, allErrs) allErrs = validateNetworkAccess(r, allErrs) + + if r.Spec.Details.LifecycleState != "" { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("details").Child("LifecycleState"), + "cannot apply lifecycleState to a provision operation")) + } } else { // binding operation } @@ -114,7 +117,7 @@ func (r *AutonomousDatabase) ValidateCreate() error { // ValidateUpdate implements webhook.Validator so a webhook will be registered for the type func (r *AutonomousDatabase) ValidateUpdate(old runtime.Object) error { - // Do ValidateCreate instead if there is no last successful spec, i.e. the database isn't provisioned or bound yet + // Validate creation instead of update if there is no last successful spec, i.e. the database isn't provisioned or bound yet lastSucSpec, err := r.GetLastSuccessfulSpec() if err != nil { return err @@ -124,18 +127,32 @@ func (r *AutonomousDatabase) ValidateUpdate(old runtime.Object) error { } var allErrs field.ErrorList - autonomousdatabaselog.Info("validate update", "name", r.Name) + var oldADB *AutonomousDatabase = old.(*AutonomousDatabase) - allErrs = validateNetworkAccess(r, allErrs) + autonomousdatabaselog.Info("validate update", "name", r.Name) + // cannot modify autonomousDatabaseOCID if r.Spec.Details.AutonomousDatabaseOCID != nil && - old.(*AutonomousDatabase).Spec.Details.AutonomousDatabaseOCID != nil && - *r.Spec.Details.AutonomousDatabaseOCID != *old.(*AutonomousDatabase).Spec.Details.AutonomousDatabaseOCID { + oldADB.Spec.Details.AutonomousDatabaseOCID != nil && + *r.Spec.Details.AutonomousDatabaseOCID != *oldADB.Spec.Details.AutonomousDatabaseOCID { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("details").Child("autonomousDatabaseOCID"), "autonomousDatabaseOCID cannot be modified")) } + // cannot apply lifecycleState with other fields together + copyDetails := r.Spec.Details.DeepCopy() + copyDetails.LifecycleState = oldADB.Spec.Details.LifecycleState + onlyLifecycleStateChanged := reflect.DeepEqual(oldADB.Spec.Details, copyDetails) + if onlyLifecycleStateChanged { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("details").Child("LifecycleState"), + "cannot apply lifecycleState with other spec.details attributes at the same time")) + } + + allErrs = validateCommon(r, allErrs) + allErrs = validateNetworkAccess(r, allErrs) + if len(allErrs) == 0 { return nil } @@ -144,24 +161,27 @@ func (r *AutonomousDatabase) ValidateUpdate(old runtime.Object) error { r.Name, allErrs) } +func validateCommon(adb *AutonomousDatabase, allErrs field.ErrorList) field.ErrorList { + // password + if adb.Spec.Details.AdminPassword.K8sSecretName != nil && adb.Spec.Details.AdminPassword.OCISecretOCID != nil { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("details").Child("Wallet").Child("Password"), + "cannot apply k8sSecretName and ociSecretOCID at the same time")) + } + + if adb.Spec.Details.Wallet.Password.K8sSecretName != nil && adb.Spec.Details.Wallet.Password.OCISecretOCID != nil { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("details").Child("AdminPassword"), + "cannot apply k8sSecretName and ociSecretOCID at the same time")) + } + + return allErrs +} + func validateNetworkAccess(adb *AutonomousDatabase, allErrs field.ErrorList) field.ErrorList { if !isDedicated(adb) { // Shared database - if adb.Spec.Details.NetworkAccess.AccessType == NetworkAccessTypePublic { - if adb.Spec.Details.NetworkAccess.AccessControlList != nil || - adb.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID != nil || - adb.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs != nil { - allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec").Child("details").Child("networkAccess"), - fmt.Sprintf("accessControlList, subnetOCID, nsgOCIDs cannot be provided when the network access type is %s", NetworkAccessTypePublic))) - } - - if adb.Spec.Details.NetworkAccess.IsMTLSConnectionRequired != nil && !*adb.Spec.Details.NetworkAccess.IsMTLSConnectionRequired { - allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec").Child("details").Child("networkAccess").Child("isMTLSConnectionRequired"), - fmt.Sprintf("isMTLSConnectionRequired cannot be false when the network access type is %s", NetworkAccessTypePublic))) - } - } else if adb.Spec.Details.NetworkAccess.AccessType == NetworkAccessTypeRestricted { + if adb.Spec.Details.NetworkAccess.AccessType == NetworkAccessTypeRestricted { if adb.Spec.Details.NetworkAccess.AccessControlList == nil { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("details").Child("networkAccess").Child("accessControlList"), @@ -180,6 +200,13 @@ func validateNetworkAccess(adb *AutonomousDatabase, allErrs field.ErrorList) fie fmt.Sprintf("nsgOCIDs cannot be empty when the network access type is %s", NetworkAccessTypePrivate))) } } + + // IsAccessControlEnabled is not applicable to a shared database + if adb.Spec.Details.NetworkAccess.IsAccessControlEnabled != nil { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("details").Child("networkAccess").Child("IsAccessControlEnabled"), + fmt.Sprintf("isAccessControlEnabled is not applicable on a shared Autonomous Database"))) + } } else { // Dedicated database diff --git a/apis/database/v1alpha1/autonomousdatabasebackup_types.go b/apis/database/v1alpha1/autonomousdatabasebackup_types.go index bb5ec00a..c3b8c8d0 100644 --- a/apis/database/v1alpha1/autonomousdatabasebackup_types.go +++ b/apis/database/v1alpha1/autonomousdatabasebackup_types.go @@ -41,8 +41,8 @@ package v1alpha1 import ( metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "github.com/oracle/oci-go-sdk/v54/common" "github.com/oracle/oci-go-sdk/v54/database" - "github.com/oracle/oracle-database-operator/commons/oci/ociutil" ) // EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! @@ -52,8 +52,8 @@ import ( type AutonomousDatabaseBackupSpec struct { // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster // Important: Run "make" to regenerate code after modifying this file - AutonomousDatabaseOCID string `json:"autonomousDatabaseOCID,omitempty"` - DisplayName string `json:"displayName,omitempty"` + AutonomousDatabaseOCID string `json:"autonomousDatabaseOCID"` + DisplayName string `json:"displayName"` AutonomousDatabaseBackupOCID string `json:"autonomousDatabaseBackupOCID,omitempty"` OCIConfig OCIConfigSpec `json:"ociConfig,omitempty"` @@ -63,17 +63,14 @@ type AutonomousDatabaseBackupSpec struct { type AutonomousDatabaseBackupStatus struct { // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster // Important: Run "make" to regenerate code after modifying this file - LifecycleState database.AutonomousDatabaseBackupLifecycleStateEnum `json:"lifecycleState"` - AutonomousDatabaseBackupOCID string `json:"autonomousDatabaseBackupOCID"` - DisplayName string `json:"displayName"` - Type database.AutonomousDatabaseBackupTypeEnum `json:"type"` - IsAutomatic bool `json:"isAutomatic"` - TimeStarted string `json:"timeStarted,omitempty"` - TimeEnded string `json:"timeEnded,omitempty"` - CompartmentOCID string `json:"compartmentOCID"` - AutonomousDatabaseOCID string `json:"autonomousDatabaseOCID"` - DBName string `json:"dbName"` - DBDisplayName string `json:"dbDisplayName"` + LifecycleState database.AutonomousDatabaseBackupLifecycleStateEnum `json:"lifecycleState"` + Type database.AutonomousDatabaseBackupTypeEnum `json:"type"` + IsAutomatic bool `json:"isAutomatic"` + TimeStarted string `json:"timeStarted,omitempty"` + TimeEnded string `json:"timeEnded,omitempty"` + CompartmentOCID string `json:"compartmentOCID"` + DBName string `json:"dbName"` + DBDisplayName string `json:"dbDisplayName"` } //+kubebuilder:object:root=true @@ -94,35 +91,37 @@ type AutonomousDatabaseBackup struct { Status AutonomousDatabaseBackupStatus `json:"status,omitempty"` } -func (backup *AutonomousDatabaseBackup) UpdateStatusFromAutonomousDatabaseBackupResponse(ociBackup database.AutonomousDatabaseBackup, ociADB database.AutonomousDatabase) { - backup.Status.AutonomousDatabaseBackupOCID = *ociBackup.Id +//+kubebuilder:object:root=true + +// AutonomousDatabaseBackupList contains a list of AutonomousDatabaseBackup +type AutonomousDatabaseBackupList struct { + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata,omitempty"` + Items []AutonomousDatabaseBackup `json:"items"` +} + +func init() { + SchemeBuilder.Register(&AutonomousDatabaseBackup{}, &AutonomousDatabaseBackupList{}) +} + +func (backup *AutonomousDatabaseBackup) UpdateStatusFromOCIBackup(ociBackup database.AutonomousDatabaseBackup, ociADB database.AutonomousDatabase) { backup.Status.CompartmentOCID = *ociBackup.CompartmentId - backup.Status.DisplayName = *ociBackup.DisplayName backup.Status.Type = ociBackup.Type backup.Status.IsAutomatic = *ociBackup.IsAutomatic backup.Status.LifecycleState = ociBackup.LifecycleState if ociBackup.TimeStarted != nil { - backup.Status.TimeStarted = ociutil.FormatSDKTime(ociBackup.TimeStarted.Time) + backup.Status.TimeStarted = formatSDKTime(ociBackup.TimeStarted.Time) } if ociBackup.TimeEnded != nil { - backup.Status.TimeEnded = ociutil.FormatSDKTime(ociBackup.TimeEnded.Time) + backup.Status.TimeEnded = formatSDKTime(ociBackup.TimeEnded.Time) } - backup.Status.AutonomousDatabaseOCID = *ociBackup.AutonomousDatabaseId backup.Status.DBDisplayName = *ociADB.DisplayName backup.Status.DBName = *ociADB.DbName } -//+kubebuilder:object:root=true - -// AutonomousDatabaseBackupList contains a list of AutonomousDatabaseBackup -type AutonomousDatabaseBackupList struct { - metav1.TypeMeta `json:",inline"` - metav1.ListMeta `json:"metadata,omitempty"` - Items []AutonomousDatabaseBackup `json:"items"` -} - -func init() { - SchemeBuilder.Register(&AutonomousDatabaseBackup{}, &AutonomousDatabaseBackupList{}) +// GetTimeEnded returns the status.timeEnded in SDKTime format +func (backup *AutonomousDatabaseBackup) GetTimeEnded() (*common.SDKTime, error) { + return parseDisplayTime(backup.Status.TimeEnded) } diff --git a/apis/database/v1alpha1/autonomousdatabasebackup_webhook.go b/apis/database/v1alpha1/autonomousdatabasebackup_webhook.go index 6b2c07df..40757eb9 100644 --- a/apis/database/v1alpha1/autonomousdatabasebackup_webhook.go +++ b/apis/database/v1alpha1/autonomousdatabasebackup_webhook.go @@ -78,27 +78,7 @@ var _ webhook.Validator = &AutonomousDatabaseBackup{} // ValidateCreate implements webhook.Validator so a webhook will be registered for the type func (r *AutonomousDatabaseBackup) ValidateCreate() error { autonomousdatabasebackuplog.Info("validate create", "name", r.Name) - - var allErrs field.ErrorList - - if r.Spec.AutonomousDatabaseBackupOCID != "" && r.Spec.AutonomousDatabaseOCID != "" { - allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec").Child("autonomousDatabaseBackupOCID"), - "cannot apply autonomousDatabaseBackupOCID and autonomousDatabaseOCID to the backup at the same time")) - } - - if r.Spec.DisplayName != "" && r.Spec.AutonomousDatabaseOCID == "" { - allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec").Child("displayName"), - "autonomousDatabaseOCID cannot be empty")) - } - - if len(allErrs) == 0 { - return nil - } - return apierrors.NewInvalid( - schema.GroupKind{Group: "database.oracle.com", Kind: "AutonomousDatabaseBackup"}, - r.Name, allErrs) + return nil } // ValidateUpdate implements webhook.Validator so a webhook will be registered for the type @@ -106,24 +86,22 @@ func (r *AutonomousDatabaseBackup) ValidateUpdate(old runtime.Object) error { autonomousdatabasebackuplog.Info("validate update", "name", r.Name) var allErrs field.ErrorList + oldBackup := old.(*AutonomousDatabaseBackup) - if r.Spec.AutonomousDatabaseBackupOCID != "" && - old.(*AutonomousDatabaseBackup).Status.AutonomousDatabaseBackupOCID != "" && - r.Spec.AutonomousDatabaseBackupOCID != old.(*AutonomousDatabaseBackup).Status.AutonomousDatabaseBackupOCID { + if oldBackup.Spec.AutonomousDatabaseBackupOCID != "" && + oldBackup.Spec.AutonomousDatabaseBackupOCID != r.Spec.AutonomousDatabaseBackupOCID { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("autonomousDatabaseBackupOCID"), "cannot assign a new autonomousDatabaseBackupOCID to this backup")) } if r.Spec.AutonomousDatabaseOCID != "" && - old.(*AutonomousDatabaseBackup).Status.AutonomousDatabaseOCID != "" && - r.Spec.AutonomousDatabaseOCID != old.(*AutonomousDatabaseBackup).Status.AutonomousDatabaseOCID { + oldBackup.Spec.AutonomousDatabaseOCID != r.Spec.AutonomousDatabaseOCID { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("autonomousDatabaseOCID"), "cannot assign a new autonomousDatabaseOCID to this backup")) } if r.Spec.DisplayName != "" && - old.(*AutonomousDatabaseBackup).Status.DisplayName != "" && - r.Spec.DisplayName != old.(*AutonomousDatabaseBackup).Status.DisplayName { + oldBackup.Spec.DisplayName != r.Spec.DisplayName { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("displayName"), "cannot assign a new displayName to this backup")) } diff --git a/apis/database/v1alpha1/autonomousdatabaserestore_types.go b/apis/database/v1alpha1/autonomousdatabaserestore_types.go index 6ea2b852..2fb17a8e 100644 --- a/apis/database/v1alpha1/autonomousdatabaserestore_types.go +++ b/apis/database/v1alpha1/autonomousdatabaserestore_types.go @@ -39,8 +39,10 @@ package v1alpha1 import ( - "github.com/oracle/oci-go-sdk/v54/workrequests" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + + "github.com/oracle/oci-go-sdk/v54/common" + "github.com/oracle/oci-go-sdk/v54/workrequests" ) // EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! @@ -60,20 +62,28 @@ type PITSource struct { TimeStamp string `json:"timeStamp,omitempty"` } +type restoreStatusEnum string + +const ( + RestoreStatusInProgress restoreStatusEnum = "IN_PROGRESS" + RestoreStatusFailed restoreStatusEnum = "FAILED" + RestoreStatusSucceeded restoreStatusEnum = "SUCCEEDED" +) + // AutonomousDatabaseRestoreStatus defines the observed state of AutonomousDatabaseRestore type AutonomousDatabaseRestoreStatus struct { // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster // Important: Run "make" to regenerate code after modifying this file - DisplayName string `json:"displayName"` - DbName string `json:"dbName"` - AutonomousDatabaseOCID string `json:"autonomousDatabaseOCID"` - LifecycleState workrequests.WorkRequestStatusEnum `json:"lifecycleState"` + DisplayName string `json:"displayName"` + DbName string `json:"dbName"` + AutonomousDatabaseOCID string `json:"autonomousDatabaseOCID"` + Status restoreStatusEnum `json:"status"` } //+kubebuilder:object:root=true //+kubebuilder:subresource:status //+kubebuilder:resource:shortName="adbr";"adbrs" -// +kubebuilder:printcolumn:JSONPath=".status.lifecycleState",name="State",type=string +// +kubebuilder:printcolumn:JSONPath=".status.status",name="Status",type=string // +kubebuilder:printcolumn:JSONPath=".status.displayName",name="DisplayName",type=string // +kubebuilder:printcolumn:JSONPath=".status.dbName",name="DbName",type=string @@ -98,3 +108,24 @@ type AutonomousDatabaseRestoreList struct { func init() { SchemeBuilder.Register(&AutonomousDatabaseRestore{}, &AutonomousDatabaseRestoreList{}) } + +// GetPIT returns the spec.pointInTime.timeStamp in SDKTime format +func (r *AutonomousDatabaseRestore) GetPIT() (*common.SDKTime, error) { + return parseDisplayTime(r.Spec.PointInTime.TimeStamp) +} + +func (r *AutonomousDatabaseRestore) ConvertWorkRequestStatus(s workrequests.WorkRequestStatusEnum) restoreStatusEnum { + switch s { + case workrequests.WorkRequestStatusAccepted: + case workrequests.WorkRequestStatusInProgress: + return RestoreStatusInProgress + + case workrequests.WorkRequestStatusSucceeded: + return RestoreStatusSucceeded + + case workrequests.WorkRequestStatusFailed: + return RestoreStatusFailed + } + + return RestoreStatusFailed +} diff --git a/apis/database/v1alpha1/autonomousdatabaserestore_webhook.go b/apis/database/v1alpha1/autonomousdatabaserestore_webhook.go index 709a8d93..74195918 100644 --- a/apis/database/v1alpha1/autonomousdatabaserestore_webhook.go +++ b/apis/database/v1alpha1/autonomousdatabaserestore_webhook.go @@ -39,9 +39,6 @@ package v1alpha1 import ( - "reflect" - - "github.com/oracle/oracle-database-operator/commons/oci/ociutil" apierrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/runtime" "k8s.io/apimachinery/pkg/runtime/schema" @@ -94,7 +91,7 @@ func (r *AutonomousDatabaseRestore) ValidateCreate() error { // Verify the timestamp format if it's PITR if r.Spec.PointInTime.TimeStamp != "" { - _, err := ociutil.ParseDisplayTime(r.Spec.PointInTime.TimeStamp) + _, err := parseDisplayTime(r.Spec.PointInTime.TimeStamp) if err != nil { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec"), "invalid timestamp format")) @@ -115,12 +112,8 @@ func (r *AutonomousDatabaseRestore) ValidateUpdate(old runtime.Object) error { var allErrs field.ErrorList - if old.(*AutonomousDatabaseRestore).Status.LifecycleState != "" && - !reflect.DeepEqual(r.Spec, old.(*AutonomousDatabaseRestore).Spec) { - - allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec"), "the AutonomousDatabaseRestore resource cannot be modified")) - } + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec"), "update AutonomousDatabaseRestore is diabled")) if len(allErrs) == 0 { return nil diff --git a/apis/database/v1alpha1/zz_generated.deepcopy.go b/apis/database/v1alpha1/zz_generated.deepcopy.go index ad0c77ff..d3f109d2 100644 --- a/apis/database/v1alpha1/zz_generated.deepcopy.go +++ b/apis/database/v1alpha1/zz_generated.deepcopy.go @@ -194,11 +194,6 @@ func (in *AutonomousDatabaseDetails) DeepCopyInto(out *AutonomousDatabaseDetails *out = new(string) **out = **in } - if in.IsDedicated != nil { - in, out := &in.IsDedicated, &out.IsDedicated - *out = new(bool) - **out = **in - } if in.DbVersion != nil { in, out := &in.DbVersion, &out.DbVersion *out = new(string) @@ -220,6 +215,11 @@ func (in *AutonomousDatabaseDetails) DeepCopyInto(out *AutonomousDatabaseDetails *out = new(bool) **out = **in } + if in.IsDedicated != nil { + in, out := &in.IsDedicated, &out.IsDedicated + *out = new(bool) + **out = **in + } in.NetworkAccess.DeepCopyInto(&out.NetworkAccess) if in.FreeformTags != nil { in, out := &in.FreeformTags, &out.FreeformTags @@ -391,7 +391,14 @@ func (in *AutonomousDatabaseStatus) DeepCopyInto(out *AutonomousDatabaseStatus) *out = *in if in.AllConnectionStrings != nil { in, out := &in.AllConnectionStrings, &out.AllConnectionStrings - *out = make([]ConnectionStringsSet, len(*in)) + *out = make([]ConnectionStringProfile, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } + if in.Conditions != nil { + in, out := &in.Conditions, &out.Conditions + *out = make([]v1.Condition, len(*in)) for i := range *in { (*in)[i].DeepCopyInto(&(*out)[i]) } @@ -460,23 +467,36 @@ func (in *CatalogSpec) DeepCopy() *CatalogSpec { } // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *ConnectionStringsSet) DeepCopyInto(out *ConnectionStringsSet) { +func (in *ConnectionStringProfile) DeepCopyInto(out *ConnectionStringProfile) { *out = *in if in.ConnectionStrings != nil { in, out := &in.ConnectionStrings, &out.ConnectionStrings - *out = make(map[string]string, len(*in)) - for key, val := range *in { - (*out)[key] = val - } + *out = make([]ConnectionStringSpec, len(*in)) + copy(*out, *in) } } -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ConnectionStringsSet. -func (in *ConnectionStringsSet) DeepCopy() *ConnectionStringsSet { +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ConnectionStringProfile. +func (in *ConnectionStringProfile) DeepCopy() *ConnectionStringProfile { if in == nil { return nil } - out := new(ConnectionStringsSet) + out := new(ConnectionStringProfile) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *ConnectionStringSpec) DeepCopyInto(out *ConnectionStringSpec) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ConnectionStringSpec. +func (in *ConnectionStringSpec) DeepCopy() *ConnectionStringSpec { + if in == nil { + return nil + } + out := new(ConnectionStringSpec) in.DeepCopyInto(out) return out } @@ -806,6 +826,21 @@ func (in *OracleRestDataServiceStatus) DeepCopy() *OracleRestDataServiceStatus { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PITSource) DeepCopyInto(out *PITSource) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PITSource. +func (in *PITSource) DeepCopy() *PITSource { + if in == nil { + return nil + } + out := new(PITSource) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *PasswordSpec) DeepCopyInto(out *PasswordSpec) { *out = *in diff --git a/commons/annotations/annotations.go b/commons/annotations/annotations.go index aa4c4e15..8472c627 100644 --- a/commons/annotations/annotations.go +++ b/commons/annotations/annotations.go @@ -54,6 +54,7 @@ type PatchValue struct { } // SetAnnotations attaches the given metadata to the target object +// The obj will be updated with the content returned by the cluster func SetAnnotations(kubeClient client.Client, obj client.Object, anns map[string]string) error { payload := []PatchValue{} diff --git a/commons/autonomousdatabase/adb_backup_reconciler_util.go b/commons/autonomousdatabase/adb_backup_reconciler_util.go deleted file mode 100644 index d819ce71..00000000 --- a/commons/autonomousdatabase/adb_backup_reconciler_util.go +++ /dev/null @@ -1,133 +0,0 @@ -/* -** Copyright (c) 2021 Oracle and/or its affiliates. -** -** The Universal Permissive License (UPL), Version 1.0 -** -** Subject to the condition set forth below, permission is hereby granted to any -** person obtaining a copy of this software, associated documentation and/or data -** (collectively the "Software"), free of charge and under any and all copyright -** rights in the Software, and any and all patent rights owned or freely -** licensable by each licensor hereunder covering either (i) the unmodified -** Software as contributed to or provided by such licensor, or (ii) the Larger -** Works (as defined below), to deal in both -** -** (a) the Software, and -** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if -** one is included with the Software (each a "Larger Work" to which the Software -** is contributed by such licensors), -** -** without restriction, including without limitation the rights to copy, create -** derivative works of, display, perform, and distribute the Software and make, -** use, sell, offer for sale, import, export, have made, and have sold the -** Software and the Larger Work(s), and to sublicense the foregoing rights on -** either these or other terms. -** -** This license is subject to the following condition: -** The above copyright notice and either this complete permission notice or at -** a minimum a reference to the UPL must be included in all copies or -** substantial portions of the Software. -** -** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -** SOFTWARE. - */ - -package autonomousdatabase - -import ( - "context" - "fmt" - - "k8s.io/apimachinery/pkg/types" - "k8s.io/client-go/util/retry" - "sigs.k8s.io/controller-runtime/pkg/client" - - "github.com/go-logr/logr" - dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" -) - -// Returns the first AutonomousDatabase resource that matches the AutonomousDatabaseOCID of the backup -// If the AutonomousDatabase doesn't exist, returns a nil -func getOwnerAutonomousDatabase(kubeClient client.Client, namespace string, adbOCID string) (*dbv1alpha1.AutonomousDatabase, error) { - adbList, err := fetchAutonomousDatabases(kubeClient, namespace) - if err != nil { - return nil, err - } - - for _, adb := range adbList.Items { - if adb.Spec.Details.AutonomousDatabaseOCID != nil && *adb.Spec.Details.AutonomousDatabaseOCID == adbOCID { - return &adb, nil - } - } - - return nil, nil -} - -// SetOwnerAutonomousDatabase sets the owner of the AutonomousDatabaseBackup if the AutonomousDatabase resource with the same database OCID is found -func SetOwnerAutonomousDatabase(logger logr.Logger, kubeClient client.Client, backup *dbv1alpha1.AutonomousDatabaseBackup) error { - adb, err := getOwnerAutonomousDatabase(kubeClient, backup.Namespace, backup.Status.AutonomousDatabaseOCID) - if err != nil { - return err - } - - if adb != nil { - backup.SetOwnerReferences(newOwnerReference(adb)) - updateAutonomousDatabaseBackupResource(logger, kubeClient, backup) - logger.Info(fmt.Sprintf("Set the owner of %s to %s", backup.Name, adb.Name)) - } - - return nil -} - -// update the spec and the objectMeta -func updateAutonomousDatabaseBackupResource(logger logr.Logger, kubeClient client.Client, backup *dbv1alpha1.AutonomousDatabaseBackup) error { - if err := retry.RetryOnConflict(retry.DefaultRetry, func() error { - curBackup := &dbv1alpha1.AutonomousDatabaseBackup{} - - namespacedName := types.NamespacedName{ - Namespace: backup.GetNamespace(), - Name: backup.GetName(), - } - - if err := kubeClient.Get(context.TODO(), namespacedName, curBackup); err != nil { - return err - } - - curBackup.Spec = *backup.Spec.DeepCopy() - curBackup.ObjectMeta = *backup.ObjectMeta.DeepCopy() - return kubeClient.Update(context.TODO(), curBackup) - }); err != nil { - return err - } - - // Update status - if statusErr := UpdateAutonomousDatabaseBackupStatus(kubeClient, backup); statusErr != nil { - return statusErr - } - logger.Info("Update local resource AutonomousDatabase successfully") - - return nil -} - -// UpdateAutonomousDatabaseBackupStatus updates the status subresource of AutonomousDatabaseBackup -func UpdateAutonomousDatabaseBackupStatus(kubeClient client.Client, adbBackup *dbv1alpha1.AutonomousDatabaseBackup) error { - return retry.RetryOnConflict(retry.DefaultRetry, func() error { - curBackup := &dbv1alpha1.AutonomousDatabaseBackup{} - - namespacedName := types.NamespacedName{ - Namespace: adbBackup.GetNamespace(), - Name: adbBackup.GetName(), - } - - if err := kubeClient.Get(context.TODO(), namespacedName, curBackup); err != nil { - return err - } - - curBackup.Status = adbBackup.Status - return kubeClient.Status().Update(context.TODO(), curBackup) - }) -} diff --git a/commons/autonomousdatabase/adb_reconciler_util.go b/commons/autonomousdatabase/adb_reconciler_util.go deleted file mode 100644 index eb4ecc59..00000000 --- a/commons/autonomousdatabase/adb_reconciler_util.go +++ /dev/null @@ -1,493 +0,0 @@ -/* -** Copyright (c) 2021 Oracle and/or its affiliates. -** -** The Universal Permissive License (UPL), Version 1.0 -** -** Subject to the condition set forth below, permission is hereby granted to any -** person obtaining a copy of this software, associated documentation and/or data -** (collectively the "Software"), free of charge and under any and all copyright -** rights in the Software, and any and all patent rights owned or freely -** licensable by each licensor hereunder covering either (i) the unmodified -** Software as contributed to or provided by such licensor, or (ii) the Larger -** Works (as defined below), to deal in both -** -** (a) the Software, and -** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if -** one is included with the Software (each a "Larger Work" to which the Software -** is contributed by such licensors), -** -** without restriction, including without limitation the rights to copy, create -** derivative works of, display, perform, and distribute the Software and make, -** use, sell, offer for sale, import, export, have made, and have sold the -** Software and the Larger Work(s), and to sublicense the foregoing rights on -** either these or other terms. -** -** This license is subject to the following condition: -** The above copyright notice and either this complete permission notice or at -** a minimum a reference to the UPL must be included in all copies or -** substantial portions of the Software. -** -** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -** SOFTWARE. - */ - -package autonomousdatabase - -import ( - "context" - "fmt" - "regexp" - "strings" - - corev1 "k8s.io/api/core/v1" - metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" - "k8s.io/apimachinery/pkg/types" - "k8s.io/client-go/util/retry" - "sigs.k8s.io/controller-runtime/pkg/client" - - "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v54/common" - "github.com/oracle/oci-go-sdk/v54/database" - "github.com/oracle/oci-go-sdk/v54/secrets" - "github.com/oracle/oci-go-sdk/v54/workrequests" - - dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" - "github.com/oracle/oracle-database-operator/commons/oci" -) - -func UpdateAutonomousDatabaseDetails(logger logr.Logger, kubeClient client.Client, adb *dbv1alpha1.AutonomousDatabase) error { - if err := retry.RetryOnConflict(retry.DefaultRetry, func() error { - curADB := &dbv1alpha1.AutonomousDatabase{} - - namespacedName := types.NamespacedName{ - Namespace: adb.GetNamespace(), - Name: adb.GetName(), - } - - if err := kubeClient.Get(context.TODO(), namespacedName, curADB); err != nil { - return err - } - - curADB.Spec.Details = adb.Spec.Details - return kubeClient.Update(context.TODO(), curADB) - }); err != nil { - return err - } - - // Update status - if statusErr := UpdateAutonomousDatabaseStatus(kubeClient, adb); statusErr != nil { - return statusErr - } - logger.Info("Update local resource AutonomousDatabase successfully") - return nil -} - -func UpdateGeneralAndPasswordAttributesAndWait(logger logr.Logger, - kubeClient client.Client, - dbClient database.DatabaseClient, - secretClient secrets.SecretsClient, - workClient workrequests.WorkRequestClient, - adb *dbv1alpha1.AutonomousDatabase) error { - - updateGenPassResp, err := oci.UpdateGeneralAndPasswordAttributes(logger, kubeClient, dbClient, secretClient, adb) - if err != nil { - logger.Error(err, "Fail to update Autonomous Database") - - // Change the status to UNAVAILABLE - adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := UpdateAutonomousDatabaseStatus(kubeClient, adb); statusErr != nil { - return statusErr - } - // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue - return nil - } - - // Wait for the work finish if a request is sent. Note that some of the requests (e.g. update displayName) won't return a work request ID. - if updateGenPassResp.OpcWorkRequestId != nil { - if err := UpdateAutonomousDatabaseStatusAndWait(logger, kubeClient, workClient, adb, updateGenPassResp.AutonomousDatabase.LifecycleState, updateGenPassResp.OpcWorkRequestId); err != nil { - logger.Error(err, "Fail to update Autonomous Database. opcWorkRequestID = "+*updateGenPassResp.OpcWorkRequestId) - } - - logger.Info("Update AutonomousDatabase " + *adb.Spec.Details.DbName + " succesfully") - } - - return nil -} - -func UpdateScaleAttributesAndWait(logger logr.Logger, kubeClient client.Client, - dbClient database.DatabaseClient, - workClient workrequests.WorkRequestClient, - adb *dbv1alpha1.AutonomousDatabase) error { - - scaleResp, err := oci.UpdateScaleAttributes(logger, kubeClient, dbClient, adb) - if err != nil { - // Change the status to UNAVAILABLE - adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := UpdateAutonomousDatabaseStatus(kubeClient, adb); statusErr != nil { - return statusErr - } - // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue - return err - } - - if scaleResp.OpcWorkRequestId != nil { - if err := UpdateAutonomousDatabaseStatusAndWait(logger, kubeClient, workClient, adb, scaleResp.AutonomousDatabase.LifecycleState, scaleResp.OpcWorkRequestId); err != nil { - logger.Error(err, "Fail to scale the Autonomous Database. opcWorkRequestID = "+*scaleResp.OpcWorkRequestId) - } - - logger.Info("Scale AutonomousDatabase " + *adb.Spec.Details.DbName + " succesfully") - } - - return nil -} - -// The logic of updating the network access configurations is as follows: -// 1. Shared databases: -// If the network access type changes -// a. to PUBLIC: -// was RESTRICTED: re-enable IsMTLSConnectionRequired if it's not. Then set WhitelistedIps to an array with a single empty string entry. -// was PRIVATE: re-enable IsMTLSConnectionRequired if it's not. Then set PrivateEndpointLabel to an emtpy string. -// b. to RESTRICTED: -// was PUBLIC: set WhitelistedIps to desired IPs/CIDR blocks/VCN OCID. Configure the IsMTLSConnectionRequired settings if it is set to disabled. -// was PRIVATE: re-enable IsMTLSConnectionRequired if it's not. Set the type to PUBLIC first, and then configure the WhitelistedIps. Finally resume the IsMTLSConnectionRequired settings if it was, or is configured as disabled. -// c. to PRIVATE: -// was PUBLIC: set subnetOCID and nsgOCIDs. Configure the IsMTLSConnectionRequired settings if it is set. -// was RESTRICTED: set subnetOCID and nsgOCIDs. Configure the IsMTLSConnectionRequired settings if it is set. -// -// Otherwise, if the network access type remains the same, apply the network configuration, and then set the IsMTLSConnectionRequired. -// -// 2. Dedicated databases: -// Apply the configs directly -func UpdateNetworkAttributes(logger logr.Logger, - kubeClient client.Client, - dbClient database.DatabaseClient, - workClient workrequests.WorkRequestClient, - curADB *dbv1alpha1.AutonomousDatabase) error { - - lastSucSpec, err := curADB.GetLastSuccessfulSpec() - if err != nil { - return err - } - - if !*curADB.Spec.Details.IsDedicated { - var lastAccessType = lastSucSpec.Details.NetworkAccess.AccessType - var curAccessType = curADB.Spec.Details.NetworkAccess.AccessType - - if oci.IsAttrChanged(lastAccessType, curAccessType) { - switch curAccessType { - case dbv1alpha1.NetworkAccessTypePublic: - // OCI validation requires IsMTLSConnectionRequired to be enabled before changing the network access type to PUBLIC - if !*lastSucSpec.Details.NetworkAccess.IsMTLSConnectionRequired { - curADB.Spec.Details.NetworkAccess.IsMTLSConnectionRequired = common.Bool(true) - if err := updateMTLSAndWait(logger, kubeClient, dbClient, workClient, curADB); err != nil { - return err - } - } - - if err := setNetworkAccessPublicAndWait(logger, kubeClient, dbClient, workClient, curADB); err != nil { - return err - } - case dbv1alpha1.NetworkAccessTypeRestricted: - // If the access type was PRIVATE, then OCI validation requires IsMTLSConnectionRequired - // to be enabled before setting ACL. Also we can only change the network access type from - // PRIVATE to PUBLIC. - if lastAccessType == dbv1alpha1.NetworkAccessTypePrivate { - if !*lastSucSpec.Details.NetworkAccess.IsMTLSConnectionRequired { - var oldMTLS bool = *curADB.Spec.Details.NetworkAccess.IsMTLSConnectionRequired - curADB.Spec.Details.NetworkAccess.IsMTLSConnectionRequired = common.Bool(true) - if err := updateMTLSAndWait(logger, kubeClient, dbClient, workClient, curADB); err != nil { - return err - } - // restore IsMTLSConnectionRequired - curADB.Spec.Details.NetworkAccess.IsMTLSConnectionRequired = &oldMTLS - } - - if err := setNetworkAccessPublicAndWait(logger, kubeClient, dbClient, workClient, curADB); err != nil { - return err - } - } - - if err := updateNetworkAccessAttributesAndWait(logger, kubeClient, dbClient, workClient, curADB); err != nil { - return err - } - - if err := updateMTLSAndWait(logger, kubeClient, dbClient, workClient, curADB); err != nil { - return err - } - case dbv1alpha1.NetworkAccessTypePrivate: - if err := updateNetworkAccessAttributesAndWait(logger, kubeClient, dbClient, workClient, curADB); err != nil { - return err - } - - if err := updateMTLSAndWait(logger, kubeClient, dbClient, workClient, curADB); err != nil { - return err - } - } - } else { - // Access type doesn't change - if err := updateNetworkAccessAttributesAndWait(logger, kubeClient, dbClient, workClient, curADB); err != nil { - return err - } - - if err := updateMTLSAndWait(logger, kubeClient, dbClient, workClient, curADB); err != nil { - return err - } - } - } else { - // Dedicated database - if err := updateNetworkAccessAttributesAndWait(logger, kubeClient, dbClient, workClient, curADB); err != nil { - return err - } - } - - return nil -} - -func updateMTLSAndWait(logger logr.Logger, - kubeClient client.Client, - dbClient database.DatabaseClient, - workClient workrequests.WorkRequestClient, - curADB *dbv1alpha1.AutonomousDatabase) error { - - resp, err := oci.UpdateMTLSConnectionRequired(logger, dbClient, curADB) - if err != nil { - return err - } - - if resp.OpcWorkRequestId != nil { - if err := UpdateAutonomousDatabaseStatusAndWait(logger, kubeClient, workClient, curADB, resp.AutonomousDatabase.LifecycleState, resp.OpcWorkRequestId); err != nil { - logger.Error(err, "Fail to update Autonomous Database MTLS. opcWorkRequestID = "+*resp.OpcWorkRequestId) - } - } - - return nil -} - -func setNetworkAccessPublicAndWait(logger logr.Logger, - kubeClient client.Client, - dbClient database.DatabaseClient, - workClient workrequests.WorkRequestClient, - curADB *dbv1alpha1.AutonomousDatabase) error { - - resp, err := oci.SetNetworkAccessPublic(logger, dbClient, curADB) - if err != nil { - return err - } - - if resp.OpcWorkRequestId != nil { - if err := UpdateAutonomousDatabaseStatusAndWait(logger, kubeClient, workClient, curADB, resp.AutonomousDatabase.LifecycleState, resp.OpcWorkRequestId); err != nil { - logger.Error(err, "Fail to update the network access options to public. opcWorkRequestID = "+*resp.OpcWorkRequestId) - } - } - - return nil -} - -func updateNetworkAccessAttributesAndWait(logger logr.Logger, - kubeClient client.Client, - dbClient database.DatabaseClient, - workClient workrequests.WorkRequestClient, - curADB *dbv1alpha1.AutonomousDatabase) error { - - resp, err := oci.UpdateNetworkAccessAttributes(logger, dbClient, curADB) - if err != nil { - return err - } - - if resp.OpcWorkRequestId != nil { - if err := UpdateAutonomousDatabaseStatusAndWait(logger, kubeClient, workClient, curADB, resp.AutonomousDatabase.LifecycleState, resp.OpcWorkRequestId); err != nil { - logger.Error(err, "Fail to update the network access of Autonomous Database. opcWorkRequestID = "+*resp.OpcWorkRequestId) - } - } - - return nil -} - -func UpdateAutonomousDatabaseStatusAndWait(logger logr.Logger, kubeClient client.Client, - workClient workrequests.WorkRequestClient, - adb *dbv1alpha1.AutonomousDatabase, - desiredLifecycleState database.AutonomousDatabaseLifecycleStateEnum, - opcWorkRequestID *string) error { - - // Update status.state - adb.Status.LifecycleState = desiredLifecycleState - if statusErr := UpdateAutonomousDatabaseStatus(kubeClient, adb); statusErr != nil { - return statusErr - } - - if _, err := oci.GetWorkStatusAndWait(logger, workClient, opcWorkRequestID); err != nil { - return err - } - - return nil -} - -// UpdateAutonomousDatabaseStatus sets the status subresource of AutonomousDatabase -func UpdateAutonomousDatabaseStatus(kubeClient client.Client, adb *dbv1alpha1.AutonomousDatabase) error { - return retry.RetryOnConflict(retry.DefaultRetry, func() error { - curADB := &dbv1alpha1.AutonomousDatabase{} - - namespacedName := types.NamespacedName{ - Namespace: adb.GetNamespace(), - Name: adb.GetName(), - } - - if err := kubeClient.Get(context.TODO(), namespacedName, curADB); err != nil { - return err - } - - curADB.Status = adb.Status - return kubeClient.Status().Update(context.TODO(), curADB) - }) -} - -func createWalletSecret(kubeClient client.Client, namespacedName types.NamespacedName, data map[string][]byte) error { - // Create the secret with the wallet data - stringData := map[string]string{} - for key, val := range data { - stringData[key] = string(val) - } - - walletSecret := &corev1.Secret{ - ObjectMeta: metav1.ObjectMeta{ - Namespace: namespacedName.Namespace, - Name: namespacedName.Name, - }, - StringData: stringData, - } - - if err := kubeClient.Create(context.TODO(), walletSecret); err != nil { - return err - } - return nil -} - -func CreateWalletSecret(logger logr.Logger, kubeClient client.Client, dbClient database.DatabaseClient, secretClient secrets.SecretsClient, adb *dbv1alpha1.AutonomousDatabase) error { - // Kube Secret which contains Instance Wallet - walletName := adb.Spec.Details.Wallet.Name - if walletName == nil { - walletName = common.String(adb.GetName() + "-instance-wallet") - } - - // No-op if Wallet is already downloaded - walletNamespacedName := types.NamespacedName{ - Namespace: adb.GetNamespace(), - Name: *walletName, - } - walletSecret := &corev1.Secret{} - if err := kubeClient.Get(context.TODO(), walletNamespacedName, walletSecret); err == nil { - return nil - } - - data, err := oci.GetWallet(logger, kubeClient, dbClient, secretClient, adb) - if err != nil { - return err - } - - if err := createWalletSecret(kubeClient, walletNamespacedName, data); err != nil { - return err - } - logger.Info(fmt.Sprintf("Wallet is stored in the Secret %s", *walletName)) - return nil -} - -func getValidName(name string, usedNames map[string]bool) string { - returnedName := name - var i = 1 - - _, ok := usedNames[returnedName] - for ok { - returnedName = fmt.Sprintf("%s-%d", name, i) - _, ok = usedNames[returnedName] - i++ - } - - return returnedName -} - -// SyncBackupResources get the list of AutonomousDatabasBackups and -// create a backup object if it's not found in the same namespace -func SyncBackupResources(logger logr.Logger, kubeClient client.Client, dbClient database.DatabaseClient, adb *dbv1alpha1.AutonomousDatabase) error { - // Get the list of AutonomousDatabaseBackupOCID in the same namespace - backupList, err := fetchAutonomousDatabaseBackups(kubeClient, adb.Namespace) - if err != nil { - return err - } - - curBackupNames := make(map[string]bool) - curBackupOCIDs := make(map[string]bool) - - for _, backup := range backupList.Items { - curBackupNames[backup.Name] = true - - if backup.Status.AutonomousDatabaseBackupOCID != "" { - curBackupOCIDs[backup.Status.AutonomousDatabaseBackupOCID] = true - } - } - - resp, err := oci.ListAutonomousDatabaseBackups(dbClient, adb) - if err != nil { - return err - } - - for _, backupSummary := range resp.Items { - // Create the resource if the AutonomousDatabaseBackupOCID doesn't exist - _, ok := curBackupOCIDs[*backupSummary.Id] - if !ok { - // Convert the string to lowercase, and replace spaces, commas, and colons with hyphens - backupName := *backupSummary.DisplayName - backupName = strings.ToLower(backupName) - - re, err := regexp.Compile(`[^-a-zA-Z0-9]`) - if err != nil { - return err - } - backupName = re.ReplaceAllString(backupName, "-") - backupName = getValidName(backupName, curBackupNames) - - if err := createBackupResource(kubeClient, backupName, backupSummary, adb); err != nil { - return err - } - - // Add the used names and ocids - curBackupNames[backupName] = true - curBackupOCIDs[*backupSummary.AutonomousDatabaseId] = true - - logger.Info("Create AutonomousDatabaseBackup " + backupName) - } - } - - return nil -} - -func createBackupResource(kubeClient client.Client, - backupName string, - backupSummary database.AutonomousDatabaseBackupSummary, - adb *dbv1alpha1.AutonomousDatabase) error { - - backup := &dbv1alpha1.AutonomousDatabaseBackup{ - ObjectMeta: metav1.ObjectMeta{ - Namespace: adb.GetNamespace(), - Name: backupName, - OwnerReferences: newOwnerReference(adb), - }, - Spec: dbv1alpha1.AutonomousDatabaseBackupSpec{ - AutonomousDatabaseBackupOCID: *backupSummary.Id, - OCIConfig: dbv1alpha1.OCIConfigSpec{ - ConfigMapName: adb.Spec.OCIConfig.ConfigMapName, - SecretName: adb.Spec.OCIConfig.SecretName, - }, - }, - } - - if err := kubeClient.Create(context.TODO(), backup); err != nil { - return err - } - - return nil -} diff --git a/commons/autonomousdatabase/adb_restore_reconciler_util.go b/commons/autonomousdatabase/adb_restore_reconciler_util.go deleted file mode 100644 index 447d8289..00000000 --- a/commons/autonomousdatabase/adb_restore_reconciler_util.go +++ /dev/null @@ -1,141 +0,0 @@ -/* -** Copyright (c) 2021 Oracle and/or its affiliates. -** -** The Universal Permissive License (UPL), Version 1.0 -** -** Subject to the condition set forth below, permission is hereby granted to any -** person obtaining a copy of this software, associated documentation and/or data -** (collectively the "Software"), free of charge and under any and all copyright -** rights in the Software, and any and all patent rights owned or freely -** licensable by each licensor hereunder covering either (i) the unmodified -** Software as contributed to or provided by such licensor, or (ii) the Larger -** Works (as defined below), to deal in both -** -** (a) the Software, and -** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if -** one is included with the Software (each a "Larger Work" to which the Software -** is contributed by such licensors), -** -** without restriction, including without limitation the rights to copy, create -** derivative works of, display, perform, and distribute the Software and make, -** use, sell, offer for sale, import, export, have made, and have sold the -** Software and the Larger Work(s), and to sublicense the foregoing rights on -** either these or other terms. -** -** This license is subject to the following condition: -** The above copyright notice and either this complete permission notice or at -** a minimum a reference to the UPL must be included in all copies or -** substantial portions of the Software. -** -** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -** SOFTWARE. - */ - -package autonomousdatabase - -import ( - "context" - "errors" - - "k8s.io/apimachinery/pkg/types" - "k8s.io/client-go/util/retry" - "sigs.k8s.io/controller-runtime/pkg/client" - - "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v54/common" - "github.com/oracle/oci-go-sdk/v54/database" - "github.com/oracle/oci-go-sdk/v54/workrequests" - dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" - "github.com/oracle/oracle-database-operator/commons/oci" - "github.com/oracle/oracle-database-operator/commons/oci/ociutil" -) - -func RestoreAutonomousDatabaseAndWait(logger logr.Logger, - kubeClient client.Client, - dbClient database.DatabaseClient, - workClient workrequests.WorkRequestClient, - restore *dbv1alpha1.AutonomousDatabaseRestore) (lifecycleState workrequests.WorkRequestStatusEnum, err error) { - - var restoreTime *common.SDKTime - var adbOCID string - - if restore.Spec.BackupName != "" { - backup := &dbv1alpha1.AutonomousDatabaseBackup{} - namespacedName := types.NamespacedName{Namespace: restore.Namespace, Name: restore.Spec.BackupName} - if err := kubeClient.Get(context.TODO(), namespacedName, backup); err != nil { - return "", err - } - - if backup.Status.TimeEnded == "" { - return "", errors.New("broken backup: ended time is missing in the AutonomousDatabaseBackup " + backup.GetName()) - } - restoreTime, err = ociutil.ParseDisplayTime(backup.Status.TimeEnded) - if err != nil { - return "", err - } - - adbOCID = backup.Status.AutonomousDatabaseOCID - - } else if restore.Spec.PointInTime.TimeStamp != "" { - // The validation of the pitr.timestamp has been handled by the webhook, so the error return is ignored - restoreTime, _ = ociutil.ParseDisplayTime(restore.Spec.PointInTime.TimeStamp) - adbOCID = restore.Spec.PointInTime.AutonomousDatabaseOCID - } - - resp, err := oci.RestoreAutonomousDatabase(dbClient, adbOCID, restoreTime) - if err != nil { - logger.Error(err, "Fail to restore Autonomous Database") - return "", nil - } - - // Update status and wait for the work finish if a request is sent. Note that some of the requests (e.g. update displayName) won't return a work request ID. - // It's important to update the status by reference otherwise Reconcile() won't be able to get the latest values - status := &restore.Status - status.DisplayName = *resp.AutonomousDatabase.DisplayName - status.DbName = *resp.AutonomousDatabase.DbName - status.AutonomousDatabaseOCID = *resp.AutonomousDatabase.Id - - workResp, err := oci.GetWorkRequest(workClient, resp.OpcWorkRequestId, nil) - if err != nil { - logger.Error(err, "Fail to get the work status. opcWorkRequestID = "+*resp.OpcWorkRequestId) - return workResp.Status, nil - } - status.LifecycleState = workResp.Status - - UpdateAutonomousDatabaseRestoreStatus(kubeClient, restore) - - // Wait until the work is done - lifecycleState, err = oci.GetWorkStatusAndWait(logger, workClient, resp.OpcWorkRequestId) - if err != nil { - logger.Error(err, "Fail to restore Autonomous Database. opcWorkRequestID = "+*resp.OpcWorkRequestId) - return lifecycleState, nil - } - - logger.Info("Restoration of Autonomous Database" + *resp.DisplayName + " finished") - - return lifecycleState, nil -} - -// UpdateAutonomousDatabaseBackupStatus updates the status subresource of AutonomousDatabaseBackup -func UpdateAutonomousDatabaseRestoreStatus(kubeClient client.Client, adbRestore *dbv1alpha1.AutonomousDatabaseRestore) error { - return retry.RetryOnConflict(retry.DefaultRetry, func() error { - curBackup := &dbv1alpha1.AutonomousDatabaseRestore{} - - namespacedName := types.NamespacedName{ - Namespace: adbRestore.GetNamespace(), - Name: adbRestore.GetName(), - } - - if err := kubeClient.Get(context.TODO(), namespacedName, curBackup); err != nil { - return err - } - - curBackup.Status = adbRestore.Status - return kubeClient.Status().Update(context.TODO(), curBackup) - }) -} diff --git a/commons/k8s/create.go b/commons/k8s/create.go new file mode 100644 index 00000000..ca158f0d --- /dev/null +++ b/commons/k8s/create.go @@ -0,0 +1,104 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package k8s + +import ( + "context" + + dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + + "github.com/oracle/oci-go-sdk/v54/database" + corev1 "k8s.io/api/core/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "sigs.k8s.io/controller-runtime/pkg/client" +) + +func CreateSecret(kubeClient client.Client, namespace string, name string, data map[string][]byte, owner client.Object, label map[string]string) error { + ownerReference := NewOwnerReference(owner) + + // Create the secret with the wallet data + stringData := map[string]string{} + for key, val := range data { + stringData[key] = string(val) + } + + walletSecret := &corev1.Secret{ + ObjectMeta: metav1.ObjectMeta{ + Namespace: namespace, + Name: name, + OwnerReferences: ownerReference, + Labels: label, + }, + StringData: stringData, + } + + if err := kubeClient.Create(context.TODO(), walletSecret); err != nil { + return err + } + return nil +} + +func CreateAutonomousBackup(kubeClient client.Client, + backupName string, + backupSummary database.AutonomousDatabaseBackupSummary, + ownerADB *dbv1alpha1.AutonomousDatabase) error { + + backup := &dbv1alpha1.AutonomousDatabaseBackup{ + ObjectMeta: metav1.ObjectMeta{ + Namespace: ownerADB.GetNamespace(), + Name: backupName, + OwnerReferences: NewOwnerReference(ownerADB), + }, + Spec: dbv1alpha1.AutonomousDatabaseBackupSpec{ + AutonomousDatabaseOCID: *backupSummary.AutonomousDatabaseId, + DisplayName: *backupSummary.DisplayName, + AutonomousDatabaseBackupOCID: *backupSummary.Id, + OCIConfig: dbv1alpha1.OCIConfigSpec{ + ConfigMapName: ownerADB.Spec.OCIConfig.ConfigMapName, + SecretName: ownerADB.Spec.OCIConfig.SecretName, + }, + }, + } + + if err := kubeClient.Create(context.TODO(), backup); err != nil { + return err + } + + return nil +} diff --git a/commons/autonomousdatabase/common_util.go b/commons/k8s/fetch.go similarity index 72% rename from commons/autonomousdatabase/common_util.go rename to commons/k8s/fetch.go index a344d3f6..21cd1d65 100644 --- a/commons/autonomousdatabase/common_util.go +++ b/commons/k8s/fetch.go @@ -36,43 +36,21 @@ ** SOFTWARE. */ -package autonomousdatabase +package k8s import ( "context" + "errors" + corev1 "k8s.io/api/core/v1" apiErrors "k8s.io/apimachinery/pkg/api/errors" - metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/types" "sigs.k8s.io/controller-runtime/pkg/client" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" ) -func newOwnerReference(owner client.Object) []metav1.OwnerReference { - ownerRef := []metav1.OwnerReference{ - { - Kind: owner.GetObjectKind().GroupVersionKind().Kind, - APIVersion: owner.GetObjectKind().GroupVersionKind().GroupVersion().String(), - Name: owner.GetName(), - UID: owner.GetUID(), - }, - } - return ownerRef -} - -// func newOwnerReference(adb *dbv1alpha1.AutonomousDatabase) []metav1.OwnerReference { -// ownerRef := []metav1.OwnerReference{ -// { -// Kind: adb.GroupVersionKind().Kind, -// APIVersion: adb.APIVersion, -// Name: adb.Name, -// UID: types.UID(adb.UID), -// }, -// } -// return ownerRef -// } - -func fetchAutonomousDatabases(kubeClient client.Client, namespace string) (*dbv1alpha1.AutonomousDatabaseList, error) { +func FetchAutonomousDatabases(kubeClient client.Client, namespace string) (*dbv1alpha1.AutonomousDatabaseList, error) { // Get the list of AutonomousDatabaseBackupOCID in the same namespace adbList := &dbv1alpha1.AutonomousDatabaseList{} @@ -87,7 +65,7 @@ func fetchAutonomousDatabases(kubeClient client.Client, namespace string) (*dbv1 return adbList, nil } -func fetchAutonomousDatabaseBackups(kubeClient client.Client, namespace string) (*dbv1alpha1.AutonomousDatabaseBackupList, error) { +func FetchAutonomousDatabaseBackups(kubeClient client.Client, namespace string) (*dbv1alpha1.AutonomousDatabaseBackupList, error) { // Get the list of AutonomousDatabaseBackupOCID in the same namespace backupList := &dbv1alpha1.AutonomousDatabaseBackupList{} @@ -101,3 +79,42 @@ func fetchAutonomousDatabaseBackups(kubeClient client.Client, namespace string) return backupList, nil } + +func FetchConfigMap(kubeClient client.Client, namespace string, name string) (*corev1.ConfigMap, error) { + namespacedName := types.NamespacedName{ + Namespace: namespace, + Name: name, + } + configMap := &corev1.ConfigMap{} + if err := kubeClient.Get(context.TODO(), namespacedName, configMap); err != nil { + return nil, err + } + + return configMap, nil +} + +func FetchSecret(kubeClient client.Client, namespace string, name string) (*corev1.Secret, error) { + namespacedName := types.NamespacedName{ + Namespace: namespace, + Name: name, + } + secret := &corev1.Secret{} + if err := kubeClient.Get(context.TODO(), namespacedName, secret); err != nil { + return nil, err + } + + return secret, nil +} + +func GetSecretValue(kubeClient client.Client, namespace string, name string, key string) (string, error) { + secret, err := FetchSecret(kubeClient, namespace, name) + if err != nil { + return "", err + } + + val, ok := secret.Data[key] + if !ok { + return "", errors.New("Secret key not found: " + key) + } + return string(val), nil +} diff --git a/commons/finalizer/finalizer.go b/commons/k8s/finalizer.go similarity index 98% rename from commons/finalizer/finalizer.go rename to commons/k8s/finalizer.go index bafe110e..34928a32 100644 --- a/commons/finalizer/finalizer.go +++ b/commons/k8s/finalizer.go @@ -36,7 +36,7 @@ ** SOFTWARE. */ -package finalizer +package k8s import ( "context" @@ -47,7 +47,7 @@ import ( ) // name of our custom finalizer -var finalizerName = "database.oracle.com/dbcsfinalizer" +var finalizerName = "database.oracle.com/oraoperator-finalizer" // HasFinalizer returns true if the finalizer exists in the object metadata func HasFinalizer(obj client.Object) bool { diff --git a/commons/k8s/utils.go b/commons/k8s/utils.go new file mode 100644 index 00000000..0a9298ef --- /dev/null +++ b/commons/k8s/utils.go @@ -0,0 +1,61 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package k8s + +import ( + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + utilErrors "k8s.io/apimachinery/pkg/util/errors" + "sigs.k8s.io/controller-runtime/pkg/client" +) + +func NewOwnerReference(owner client.Object) []metav1.OwnerReference { + ownerRef := []metav1.OwnerReference{ + { + Kind: owner.GetObjectKind().GroupVersionKind().Kind, + APIVersion: owner.GetObjectKind().GroupVersionKind().GroupVersion().String(), + Name: owner.GetName(), + UID: owner.GetUID(), + }, + } + return ownerRef +} + +func CombineErrors(errs ...error) error { + return utilErrors.NewAggregate(errs) +} \ No newline at end of file diff --git a/commons/oci/database.go b/commons/oci/database.go index 1fc35e94..caf9faa8 100644 --- a/commons/oci/database.go +++ b/commons/oci/database.go @@ -40,45 +40,120 @@ package oci import ( "context" - "errors" "fmt" - "math" - "time" "github.com/go-logr/logr" "github.com/oracle/oci-go-sdk/v54/common" "github.com/oracle/oci-go-sdk/v54/database" - "github.com/oracle/oci-go-sdk/v54/secrets" - "github.com/oracle/oci-go-sdk/v54/workrequests" "sigs.k8s.io/controller-runtime/pkg/client" - corev1 "k8s.io/api/core/v1" - "k8s.io/apimachinery/pkg/types" - dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + "github.com/oracle/oracle-database-operator/commons/k8s" ) -// CreateAutonomousDatabase sends a request to OCI to provision a database and returns the AutonomousDatabase OCID. -func CreateAutonomousDatabase(logger logr.Logger, kubeClient client.Client, dbClient database.DatabaseClient, secretClient secrets.SecretsClient, adb *dbv1alpha1.AutonomousDatabase) (*database.CreateAutonomousDatabaseResponse, error) { - adminPassword, err := getAdminPassword(logger, kubeClient, secretClient, adb) +type DatabaseService interface { + CreateAutonomousDatabase(adb *dbv1alpha1.AutonomousDatabase) (database.CreateAutonomousDatabaseResponse, error) + GetAutonomousDatabase(adbOCID string) (database.GetAutonomousDatabaseResponse, error) + UpdateAutonomousDatabaseGeneralFields(adb *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) + UpdateAutonomousDatabaseDBWorkload(adb *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) + UpdateAutonomousDatabaseLicenseModel(adb *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) + UpdateAutonomousDatabaseAdminPassword(adb *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) + UpdateAutonomousDatabaseScalingFields(adb *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) + UpdateNetworkAccessMTLSRequired(adbOCID string) (resp database.UpdateAutonomousDatabaseResponse, err error) + UpdateNetworkAccessMTLS(adb *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) + UpdateNetworkAccessPublic(lastSucSpec *dbv1alpha1.AutonomousDatabaseSpec, adbOCID string) (resp database.UpdateAutonomousDatabaseResponse, err error) + UpdateNetworkAccess(adb *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) + StartAutonomousDatabase(adbOCID string) (database.StartAutonomousDatabaseResponse, error) + StopAutonomousDatabase(adbOCID string) (database.StopAutonomousDatabaseResponse, error) + DeleteAutonomousDatabase(adbOCID string) (database.DeleteAutonomousDatabaseResponse, error) + DownloadWallet(adb *dbv1alpha1.AutonomousDatabase) (database.GenerateAutonomousDatabaseWalletResponse, error) + RestoreAutonomousDatabase(adbOCID string, sdkTime common.SDKTime) (database.RestoreAutonomousDatabaseResponse, error) + ListAutonomousDatabaseBackups(adbOCID string) (database.ListAutonomousDatabaseBackupsResponse, error) + CreateAutonomousDatabaseBackup(adbBackup *dbv1alpha1.AutonomousDatabaseBackup) (database.CreateAutonomousDatabaseBackupResponse, error) + GetAutonomousDatabaseBackup(backupOCID string) (database.GetAutonomousDatabaseBackupResponse, error) +} + +type databaseService struct { + logger logr.Logger + kubeClient client.Client + dbClient database.DatabaseClient + vaultService VaultService +} + +func NewDatabaseService( + logger logr.Logger, + kubeClient client.Client, + provider common.ConfigurationProvider) (DatabaseService, error) { + + dbClient, err := database.NewDatabaseClientWithConfigurationProvider(provider) + if err != nil { + return nil, err + } + + vaultService, err := NewVaultService(logger, provider) if err != nil { return nil, err } + return &databaseService{ + logger: logger, + kubeClient: kubeClient, + dbClient: dbClient, + vaultService: vaultService, + }, nil +} + +// ReadPassword reads the password from passwordSpec, and returns the pointer to the read password string. +// The function returns a nil if nothing is read +func (d *databaseService) readPassword(namespace string, passwordSpec dbv1alpha1.PasswordSpec) (*string, error) { + logger := d.logger.WithName("read-password") + + if passwordSpec.K8sSecretName != nil { + logger.Info(fmt.Sprintf("Getting password from Secret %s", *passwordSpec.K8sSecretName)) + + key := *passwordSpec.K8sSecretName + password, err := k8s.GetSecretValue(d.kubeClient, namespace, *passwordSpec.K8sSecretName, key) + if err != nil { + return nil, err + } + + return common.String(password), nil + } + + if passwordSpec.OCISecretOCID != nil { + logger.Info(fmt.Sprintf("Getting password from OCI Vault Secret OCID %s", *passwordSpec.OCISecretOCID)) + + password, err := d.vaultService.GetSecretValue(*passwordSpec.OCISecretOCID) + if err != nil { + return nil, err + } + return common.String(password), nil + } + + return nil, nil +} + +// CreateAutonomousDatabase sends a request to OCI to provision a database and returns the AutonomousDatabase OCID. +func (d *databaseService) CreateAutonomousDatabase(adb *dbv1alpha1.AutonomousDatabase) (resp database.CreateAutonomousDatabaseResponse, err error) { + adminPassword, err := d.readPassword(adb.Namespace, adb.Spec.Details.AdminPassword) + if err != nil { + return resp, err + } + createAutonomousDatabaseDetails := database.CreateAutonomousDatabaseDetails{ CompartmentId: adb.Spec.Details.CompartmentOCID, AutonomousContainerDatabaseId: adb.Spec.Details.AutonomousContainerDatabaseOCID, DbName: adb.Spec.Details.DbName, CpuCoreCount: adb.Spec.Details.CPUCoreCount, DataStorageSizeInTBs: adb.Spec.Details.DataStorageSizeInTBs, - AdminPassword: common.String(adminPassword), + AdminPassword: adminPassword, DisplayName: adb.Spec.Details.DisplayName, IsAutoScalingEnabled: adb.Spec.Details.IsAutoScalingEnabled, IsDedicated: adb.Spec.Details.IsDedicated, DbVersion: adb.Spec.Details.DbVersion, DbWorkload: database.CreateAutonomousDatabaseBaseDbWorkloadEnum( adb.Spec.Details.DbWorkload), - + LicenseModel: database.CreateAutonomousDatabaseBaseLicenseModelEnum(adb.Spec.Details.LicenseModel), IsAccessControlEnabled: adb.Spec.Details.NetworkAccess.IsAccessControlEnabled, WhitelistedIps: adb.Spec.Details.NetworkAccess.AccessControlList, IsMtlsConnectionRequired: adb.Spec.Details.NetworkAccess.IsMTLSConnectionRequired, @@ -93,329 +168,104 @@ func CreateAutonomousDatabase(logger logr.Logger, kubeClient client.Client, dbCl CreateAutonomousDatabaseDetails: createAutonomousDatabaseDetails, } - resp, err := dbClient.CreateAutonomousDatabase(context.TODO(), createAutonomousDatabaseRequest) + resp, err = d.dbClient.CreateAutonomousDatabase(context.TODO(), createAutonomousDatabaseRequest) if err != nil { - return nil, err + return resp, err } - return &resp, nil -} - -// Get the desired admin password from either Kubernetes Secret or OCI Vault Secret. -func getAdminPassword(logger logr.Logger, kubeClient client.Client, secretClient secrets.SecretsClient, adb *dbv1alpha1.AutonomousDatabase) (string, error) { - if adb.Spec.Details.AdminPassword.K8sSecretName != nil { - logger.Info(fmt.Sprintf("Getting admin password from Secret %s", *adb.Spec.Details.AdminPassword.K8sSecretName)) - - namespacedName := types.NamespacedName{ - Namespace: adb.GetNamespace(), - Name: *adb.Spec.Details.AdminPassword.K8sSecretName, - } - - key := *adb.Spec.Details.AdminPassword.K8sSecretName - adminPassword, err := getValueFromKubeSecret(kubeClient, namespacedName, key) - if err != nil { - return "", err - } - return adminPassword, nil - - } else if adb.Spec.Details.AdminPassword.OCISecretOCID != nil { - logger.Info(fmt.Sprintf("Getting admin password from OCI Vault Secret OCID %s", *adb.Spec.Details.AdminPassword.OCISecretOCID)) - - adminPassword, err := getValueFromVaultSecret(secretClient, *adb.Spec.Details.AdminPassword.OCISecretOCID) - if err != nil { - return "", err - } - return adminPassword, nil - } - return "", errors.New("should provide either AdminPasswordSecret or AdminPasswordOCID") + return resp, nil } -func getValueFromKubeSecret(kubeClient client.Client, namespacedName types.NamespacedName, key string) (string, error) { - secret := &corev1.Secret{} - if err := kubeClient.Get(context.TODO(), namespacedName, secret); err != nil { - return "", err +func (d *databaseService) GetAutonomousDatabase(adbOCID string) (database.GetAutonomousDatabaseResponse, error) { + getAutonomousDatabaseRequest := database.GetAutonomousDatabaseRequest{ + AutonomousDatabaseId: common.String(adbOCID), } - val, ok := secret.Data[key] - if !ok { - return "", errors.New("Secret key not found: " + key) - } - return string(val), nil + return d.dbClient.GetAutonomousDatabase(context.TODO(), getAutonomousDatabaseRequest) } -// GetAutonomousDatabaseResource gets Autonomous Database information from a remote instance -// and return an AutonomousDatabase object -func GetAutonomousDatabaseResource(logger logr.Logger, dbClient database.DatabaseClient, adb *dbv1alpha1.AutonomousDatabase) (*dbv1alpha1.AutonomousDatabase, error) { - response, err := GetAutonomousDatabase(dbClient, adb.Spec.Details.AutonomousDatabaseOCID) - if err != nil { - return nil, err +func (d *databaseService) UpdateAutonomousDatabaseGeneralFields(adb *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { + updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ + AutonomousDatabaseId: adb.Spec.Details.AutonomousDatabaseOCID, + UpdateAutonomousDatabaseDetails: database.UpdateAutonomousDatabaseDetails{ + DisplayName: adb.Spec.Details.DisplayName, + DbName: adb.Spec.Details.DbName, + DbVersion: adb.Spec.Details.DbVersion, + FreeformTags: adb.Spec.Details.FreeformTags, + }, } - - returnedADB := adb.UpdateAttrFromOCIAutonomousDatabase(response.AutonomousDatabase) - - logger.Info("Get information from remote AutonomousDatabase successfully") - return returnedADB, nil + return d.dbClient.UpdateAutonomousDatabase(context.TODO(), updateAutonomousDatabaseRequest) } -func GetAutonomousDatabase(dbClient database.DatabaseClient, adbOCID *string) (database.GetAutonomousDatabaseResponse, error) { - getAutonomousDatabaseRequest := database.GetAutonomousDatabaseRequest{ - AutonomousDatabaseId: adbOCID, +func (d *databaseService) UpdateAutonomousDatabaseDBWorkload(adb *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { + updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ + AutonomousDatabaseId: adb.Spec.Details.AutonomousDatabaseOCID, + UpdateAutonomousDatabaseDetails: database.UpdateAutonomousDatabaseDetails{ + DbWorkload: database.UpdateAutonomousDatabaseDetailsDbWorkloadEnum(adb.Spec.Details.DbWorkload), + }, } - - return dbClient.GetAutonomousDatabase(context.TODO(), getAutonomousDatabaseRequest) + return d.dbClient.UpdateAutonomousDatabase(context.TODO(), updateAutonomousDatabaseRequest) } -// IsAttrChanged checks if the values of last successful object and current object are different. -// The function returns false if the types are mismatch or unknown. -// The function returns false if the current object has zero value (not applicable for boolean type). -func IsAttrChanged(lastSucObj interface{}, curObj interface{}) bool { - switch curObj.(type) { - case dbv1alpha1.NetworkAccessTypeEnum: - lastSucAccessType, ok := lastSucObj.(dbv1alpha1.NetworkAccessTypeEnum) - if !ok { - return false - } - curSucAccessType, ok := curObj.(dbv1alpha1.NetworkAccessTypeEnum) - if !ok { - return false - } - - return lastSucAccessType != curSucAccessType - case string: - // type check - lastSucString, ok := lastSucObj.(string) - if !ok { - return false - } - curString := curObj.(string) - - if curString != "" && (lastSucString != curString) { - return true - } - case *int: - // type check - lastSucIntPtr, ok := lastSucObj.(*int) - if !ok { - return false - } - curIntPtr := curObj.(*int) - - if (lastSucIntPtr == nil && curIntPtr != nil) || (lastSucIntPtr != nil && curIntPtr != nil && *lastSucIntPtr != *curIntPtr) { - return true - } - case *string: - // type check - lastSucStringPtr, ok := lastSucObj.(*string) - if !ok { - return false - } - curStringPtr := curObj.(*string) - - if (lastSucStringPtr == nil && curStringPtr != nil) || (lastSucStringPtr != nil && curStringPtr != nil && *lastSucStringPtr != *curStringPtr) { - return true - } - case *bool: - // type check - lastSucBoolPtr, ok := lastSucObj.(*bool) - if !ok { - return false - } - curBoolPtr := curObj.(*bool) - - // For boolean type, we don't have to check zero value - if (lastSucBoolPtr == nil && curBoolPtr != nil) || (lastSucBoolPtr != nil && curBoolPtr != nil && *lastSucBoolPtr != *curBoolPtr) { - return true - } - case []string: - // type check - lastSucSlice, ok := lastSucObj.([]string) - if !ok { - return false - } - - curSlice := curObj.([]string) - if curSlice == nil { - return false - } else if len(lastSucSlice) != len(curSlice) { - return true - } - - for i, v := range lastSucSlice { - if v != curSlice[i] { - return true - } - } - case map[string]string: - // type check - lastSucMap, ok := lastSucObj.(map[string]string) - if !ok { - return false - } - - curMap := curObj.(map[string]string) - if curMap == nil { - return false - } else if len(lastSucMap) != len(curMap) { - return true - } - - for k, v := range lastSucMap { - if w, ok := curMap[k]; !ok || v != w { - return true - } - } +func (d *databaseService) UpdateAutonomousDatabaseLicenseModel(adb *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { + updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ + AutonomousDatabaseId: adb.Spec.Details.AutonomousDatabaseOCID, + UpdateAutonomousDatabaseDetails: database.UpdateAutonomousDatabaseDetails{ + LicenseModel: database.UpdateAutonomousDatabaseDetailsLicenseModelEnum(adb.Spec.Details.LicenseModel), + }, } - return false + return d.dbClient.UpdateAutonomousDatabase(context.TODO(), updateAutonomousDatabaseRequest) } -// UpdateGeneralAndPasswordAttributes updates the general and password attributes of the Autonomous Database. -// Based on the responses from OCI calls, we can split the attributes into the following five categories. -// AutonomousDatabaseOCID, CompartmentOCID, IsDedicated, and LifecycleState are excluded since they not applicable in updateAutonomousDatabaseRequest. -// Except for category 1, category 2 and 3 cannot be updated at the same time, i.e.,we can at most update category 1 plus another category 2,or 3. -// 1. General attribute: including DisplayName, DbName, DbWorkload, DbVersion, freeformTags, subnetOCID, nsgOCIDs, and whitelistedIPs. The general attributes can be updated together with one of the other categories in the same request. -// 2. Scale attribute: includining IsAutoScalingEnabled, CpuCoreCount and DataStorageSizeInTBs. -// 3. Password attribute: including AdminPasswordSecret and AdminPasswordOCID -// From the above rules, we group general and password attributes and send the update together in the same request, and then send the scale update in another request. -func UpdateGeneralAndPasswordAttributes(logger logr.Logger, kubeClient client.Client, dbClient database.DatabaseClient, - secretClient secrets.SecretsClient, curADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { - var shouldSendRequest = false - - lastSucSpec, err := curADB.GetLastSuccessfulSpec() +func (d *databaseService) UpdateAutonomousDatabaseAdminPassword(adb *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { + adminPassword, err := d.readPassword(adb.Namespace, adb.Spec.Details.AdminPassword) if err != nil { return resp, err } - // Prepare the update request - updateAutonomousDatabaseDetails := database.UpdateAutonomousDatabaseDetails{} - - if IsAttrChanged(lastSucSpec.Details.DisplayName, curADB.Spec.Details.DisplayName) { - updateAutonomousDatabaseDetails.DisplayName = curADB.Spec.Details.DisplayName - shouldSendRequest = true - } - if IsAttrChanged(lastSucSpec.Details.DbName, curADB.Spec.Details.DbName) { - updateAutonomousDatabaseDetails.DbName = curADB.Spec.Details.DbName - shouldSendRequest = true - } - if IsAttrChanged(lastSucSpec.Details.DbWorkload, curADB.Spec.Details.DbWorkload) { - updateAutonomousDatabaseDetails.DbWorkload = database.UpdateAutonomousDatabaseDetailsDbWorkloadEnum(curADB.Spec.Details.DbWorkload) - shouldSendRequest = true - } - if IsAttrChanged(lastSucSpec.Details.DbVersion, curADB.Spec.Details.DbVersion) { - updateAutonomousDatabaseDetails.DbVersion = curADB.Spec.Details.DbVersion - shouldSendRequest = true - } - - if IsAttrChanged(lastSucSpec.Details.FreeformTags, curADB.Spec.Details.FreeformTags) { - updateAutonomousDatabaseDetails.FreeformTags = curADB.Spec.Details.FreeformTags - shouldSendRequest = true - } - - if IsAttrChanged(lastSucSpec.Details.AdminPassword.K8sSecretName, curADB.Spec.Details.AdminPassword.K8sSecretName) || - IsAttrChanged(lastSucSpec.Details.AdminPassword.OCISecretOCID, curADB.Spec.Details.AdminPassword.OCISecretOCID) { - // Get the adminPassword - var adminPassword string - - adminPassword, err = getAdminPassword(logger, kubeClient, secretClient, curADB) - if err != nil { - return - } - updateAutonomousDatabaseDetails.AdminPassword = common.String(adminPassword) - - shouldSendRequest = true - } - - // Send the request only when something changes - if shouldSendRequest { - - logger.Info("Sending general attributes and ADMIN password update request") - - updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ - AutonomousDatabaseId: curADB.Spec.Details.AutonomousDatabaseOCID, - UpdateAutonomousDatabaseDetails: updateAutonomousDatabaseDetails, - } - - resp, err = dbClient.UpdateAutonomousDatabase(context.TODO(), updateAutonomousDatabaseRequest) + updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ + AutonomousDatabaseId: adb.Spec.Details.AutonomousDatabaseOCID, + UpdateAutonomousDatabaseDetails: database.UpdateAutonomousDatabaseDetails{ + AdminPassword: adminPassword, + }, } - - return + return d.dbClient.UpdateAutonomousDatabase(context.TODO(), updateAutonomousDatabaseRequest) } -// UpdateScaleAttributes updates the scale attributes of the Autonomous Database -// Refer to UpdateGeneralAndPasswordAttributes for more details about how and why we separate the attributes in different calls. -func UpdateScaleAttributes(logger logr.Logger, kubeClient client.Client, dbClient database.DatabaseClient, - curADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { - var shouldSendRequest = false - - lastSucSpec, err := curADB.GetLastSuccessfulSpec() - if err != nil { - return resp, err - } - - // Prepare the update request - updateAutonomousDatabaseDetails := database.UpdateAutonomousDatabaseDetails{} - - if IsAttrChanged(lastSucSpec.Details.DataStorageSizeInTBs, curADB.Spec.Details.DataStorageSizeInTBs) { - updateAutonomousDatabaseDetails.DataStorageSizeInTBs = curADB.Spec.Details.DataStorageSizeInTBs - shouldSendRequest = true - } - if IsAttrChanged(lastSucSpec.Details.CPUCoreCount, curADB.Spec.Details.CPUCoreCount) { - updateAutonomousDatabaseDetails.CpuCoreCount = curADB.Spec.Details.CPUCoreCount - shouldSendRequest = true - } - if IsAttrChanged(lastSucSpec.Details.IsAutoScalingEnabled, curADB.Spec.Details.IsAutoScalingEnabled) { - updateAutonomousDatabaseDetails.IsAutoScalingEnabled = curADB.Spec.Details.IsAutoScalingEnabled - shouldSendRequest = true - } - - // Don't send the request if nothing is changed - if shouldSendRequest { - - logger.Info("Sending scale attributes update request") - - updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ - AutonomousDatabaseId: curADB.Spec.Details.AutonomousDatabaseOCID, - UpdateAutonomousDatabaseDetails: updateAutonomousDatabaseDetails, - } - - resp, err = dbClient.UpdateAutonomousDatabase(context.TODO(), updateAutonomousDatabaseRequest) +func (d *databaseService) UpdateAutonomousDatabaseScalingFields(adb *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { + updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ + AutonomousDatabaseId: adb.Spec.Details.AutonomousDatabaseOCID, + UpdateAutonomousDatabaseDetails: database.UpdateAutonomousDatabaseDetails{ + DataStorageSizeInTBs: adb.Spec.Details.DataStorageSizeInTBs, + CpuCoreCount: adb.Spec.Details.CPUCoreCount, + IsAutoScalingEnabled: adb.Spec.Details.IsAutoScalingEnabled, + }, } - - return + return d.dbClient.UpdateAutonomousDatabase(context.TODO(), updateAutonomousDatabaseRequest) } -func UpdateMTLSConnectionRequired(logger logr.Logger, - dbClient database.DatabaseClient, - curADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { - - lastSucSpec, err := curADB.GetLastSuccessfulSpec() - if err != nil { - return resp, err - } - - if IsAttrChanged(lastSucSpec.Details.NetworkAccess.IsMTLSConnectionRequired, curADB.Spec.Details.NetworkAccess.IsMTLSConnectionRequired) { - logger.Info(fmt.Sprintf("Sending request to set mTLSRequired to %t", *curADB.Spec.Details.NetworkAccess.IsMTLSConnectionRequired)) - - updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ - AutonomousDatabaseId: curADB.Spec.Details.AutonomousDatabaseOCID, - UpdateAutonomousDatabaseDetails: database.UpdateAutonomousDatabaseDetails{ - IsMtlsConnectionRequired: curADB.Spec.Details.NetworkAccess.IsMTLSConnectionRequired, - }, - } - resp, err = dbClient.UpdateAutonomousDatabase(context.TODO(), updateAutonomousDatabaseRequest) +func (d *databaseService) UpdateNetworkAccessMTLSRequired(adbOCID string) (resp database.UpdateAutonomousDatabaseResponse, err error) { + updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ + AutonomousDatabaseId: common.String(adbOCID), + UpdateAutonomousDatabaseDetails: database.UpdateAutonomousDatabaseDetails{ + IsMtlsConnectionRequired: common.Bool(true), + }, } - - return + return d.dbClient.UpdateAutonomousDatabase(context.TODO(), updateAutonomousDatabaseRequest) } -func SetNetworkAccessPublic(logger logr.Logger, dbClient database.DatabaseClient, - curADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { - lastSucSpec, err := curADB.GetLastSuccessfulSpec() - if err != nil { - return resp, err +func (d *databaseService) UpdateNetworkAccessMTLS(adb *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { + updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ + AutonomousDatabaseId: adb.Spec.Details.AutonomousDatabaseOCID, + UpdateAutonomousDatabaseDetails: database.UpdateAutonomousDatabaseDetails{ + IsMtlsConnectionRequired: adb.Spec.Details.NetworkAccess.IsMTLSConnectionRequired, + }, } + return d.dbClient.UpdateAutonomousDatabase(context.TODO(), updateAutonomousDatabaseRequest) +} - logger.Info("Sending request to configure Network Access type to PUBLIC") - +func (d *databaseService) UpdateNetworkAccessPublic(lastSucSpec *dbv1alpha1.AutonomousDatabaseSpec, + adbOCID string) (resp database.UpdateAutonomousDatabaseResponse, err error) { updateAutonomousDatabaseDetails := database.UpdateAutonomousDatabaseDetails{} if lastSucSpec.Details.NetworkAccess.AccessType == dbv1alpha1.NetworkAccessTypeRestricted { @@ -425,173 +275,95 @@ func SetNetworkAccessPublic(logger logr.Logger, dbClient database.DatabaseClient } updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ - AutonomousDatabaseId: curADB.Spec.Details.AutonomousDatabaseOCID, + AutonomousDatabaseId: common.String(adbOCID), UpdateAutonomousDatabaseDetails: updateAutonomousDatabaseDetails, } - return dbClient.UpdateAutonomousDatabase(context.TODO(), updateAutonomousDatabaseRequest) + return d.dbClient.UpdateAutonomousDatabase(context.TODO(), updateAutonomousDatabaseRequest) } -// UpdateNetworkAccessAttributes determines if any of the network access attributes changes and send the update request -func UpdateNetworkAccessAttributes(logger logr.Logger, dbClient database.DatabaseClient, - curADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { - - lastSucSpec, err := curADB.GetLastSuccessfulSpec() - if err != nil { - return resp, err - } - - var lastNetworkAccess = lastSucSpec.Details.NetworkAccess - var curNetworkAccess = curADB.Spec.Details.NetworkAccess - - if IsAttrChanged(lastNetworkAccess.IsAccessControlEnabled, curNetworkAccess.IsAccessControlEnabled) || - IsAttrChanged(lastNetworkAccess.AccessControlList, curNetworkAccess.AccessControlList) || - IsAttrChanged(lastNetworkAccess.PrivateEndpoint.SubnetOCID, curNetworkAccess.PrivateEndpoint.SubnetOCID) || - IsAttrChanged(lastNetworkAccess.PrivateEndpoint.NsgOCIDs, curNetworkAccess.PrivateEndpoint.NsgOCIDs) || - IsAttrChanged(lastNetworkAccess.PrivateEndpoint.HostnamePrefix, curNetworkAccess.PrivateEndpoint.HostnamePrefix) { - - logger.Info("Sending request to configure Network Access") - - updateAutonomousDatabaseDetails := database.UpdateAutonomousDatabaseDetails{} - - if IsAttrChanged(lastNetworkAccess.IsAccessControlEnabled, curNetworkAccess.IsAccessControlEnabled) { - updateAutonomousDatabaseDetails.IsAccessControlEnabled = curNetworkAccess.IsAccessControlEnabled - } - - if IsAttrChanged(lastNetworkAccess.AccessControlList, curNetworkAccess.AccessControlList) { - updateAutonomousDatabaseDetails.WhitelistedIps = curNetworkAccess.AccessControlList - } - - if IsAttrChanged(lastNetworkAccess.PrivateEndpoint.SubnetOCID, curNetworkAccess.PrivateEndpoint.SubnetOCID) { - updateAutonomousDatabaseDetails.SubnetId = curNetworkAccess.PrivateEndpoint.SubnetOCID - } - - if IsAttrChanged(lastNetworkAccess.PrivateEndpoint.NsgOCIDs, curNetworkAccess.PrivateEndpoint.NsgOCIDs) { - updateAutonomousDatabaseDetails.NsgIds = curNetworkAccess.PrivateEndpoint.NsgOCIDs - } - - if IsAttrChanged(lastNetworkAccess.PrivateEndpoint.HostnamePrefix, curNetworkAccess.PrivateEndpoint.HostnamePrefix) { - updateAutonomousDatabaseDetails.PrivateEndpointLabel = curNetworkAccess.PrivateEndpoint.HostnamePrefix - } - - updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ - AutonomousDatabaseId: curADB.Spec.Details.AutonomousDatabaseOCID, - UpdateAutonomousDatabaseDetails: updateAutonomousDatabaseDetails, - } - - resp, err = dbClient.UpdateAutonomousDatabase(context.TODO(), updateAutonomousDatabaseRequest) +func (d *databaseService) UpdateNetworkAccess(adb *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { + updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ + AutonomousDatabaseId: adb.Spec.Details.AutonomousDatabaseOCID, + UpdateAutonomousDatabaseDetails: database.UpdateAutonomousDatabaseDetails{ + IsAccessControlEnabled: adb.Spec.Details.NetworkAccess.IsAccessControlEnabled, + WhitelistedIps: adb.Spec.Details.NetworkAccess.AccessControlList, + SubnetId: adb.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID, + NsgIds: adb.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs, + PrivateEndpointLabel: adb.Spec.Details.NetworkAccess.PrivateEndpoint.HostnamePrefix, + }, } - return + return d.dbClient.UpdateAutonomousDatabase(context.TODO(), updateAutonomousDatabaseRequest) } -// SetAutonomousDatabaseLifecycleState starts or stops AutonomousDatabase in OCI based on the LifeCycleState attribute -func SetAutonomousDatabaseLifecycleState(logger logr.Logger, dbClient database.DatabaseClient, adb *dbv1alpha1.AutonomousDatabase) (resp interface{}, err error) { - lastSucSpec, err := adb.GetLastSuccessfulSpec() - if err != nil { - return resp, err - } - - // Return if the desired lifecycle state is the same as the current lifecycle state - if adb.Spec.Details.LifecycleState == lastSucSpec.Details.LifecycleState { - return nil, nil +func (d *databaseService) StartAutonomousDatabase(adbOCID string) (database.StartAutonomousDatabaseResponse, error) { + startRequest := database.StartAutonomousDatabaseRequest{ + AutonomousDatabaseId: common.String(adbOCID), } - switch string(adb.Spec.Details.LifecycleState) { - case string(database.AutonomousDatabaseLifecycleStateAvailable): - logger.Info("Sending start request to the Autonomous Database " + *adb.Spec.Details.DbName) - - resp, err = startAutonomousDatabase(dbClient, *adb.Spec.Details.AutonomousDatabaseOCID) - if err != nil { - return - } - - case string(database.AutonomousDatabaseLifecycleStateStopped): - logger.Info("Sending stop request to the Autonomous Database " + *adb.Spec.Details.DbName) - - resp, err = stopAutonomousDatabase(dbClient, *adb.Spec.Details.AutonomousDatabaseOCID) - if err != nil { - return - } - - case string(database.AutonomousDatabaseLifecycleStateTerminated): - // Special case. - if adb.Spec.Details.LifecycleState == database.AutonomousDatabaseLifecycleStateTerminating { - break - } - logger.Info("Sending teminate request to the Autonomous Database " + *adb.Spec.Details.DbName) - - resp, err = DeleteAutonomousDatabase(dbClient, *adb.Spec.Details.AutonomousDatabaseOCID) - if err != nil { - return - } + return d.dbClient.StartAutonomousDatabase(context.TODO(), startRequest) +} - default: - err = fmt.Errorf("invalid lifecycleState value: currently the operator only accept %s, %s and %s as the value of the lifecycleState parameter", - database.AutonomousDatabaseLifecycleStateAvailable, - database.AutonomousDatabaseLifecycleStateStopped, - database.AutonomousDatabaseLifecycleStateTerminated) +func (d *databaseService) StopAutonomousDatabase(adbOCID string) (database.StopAutonomousDatabaseResponse, error) { + stopRequest := database.StopAutonomousDatabaseRequest{ + AutonomousDatabaseId: common.String(adbOCID), } - return + return d.dbClient.StopAutonomousDatabase(context.TODO(), stopRequest) } -// startAutonomousDatabase starts an Autonomous Database in OCI -func startAutonomousDatabase(dbClient database.DatabaseClient, adbOCID string) (resp database.StartAutonomousDatabaseResponse, err error) { - startRequest := database.StartAutonomousDatabaseRequest{ +func (d *databaseService) DeleteAutonomousDatabase(adbOCID string) (database.DeleteAutonomousDatabaseResponse, error) { + deleteRequest := database.DeleteAutonomousDatabaseRequest{ AutonomousDatabaseId: common.String(adbOCID), } - resp, err = dbClient.StartAutonomousDatabase(context.TODO(), startRequest) - return + return d.dbClient.DeleteAutonomousDatabase(context.TODO(), deleteRequest) } -// stopAutonomousDatabase stops an Autonomous Database in OCI -func stopAutonomousDatabase(dbClient database.DatabaseClient, adbOCID string) (resp database.StopAutonomousDatabaseResponse, err error) { - stopRequest := database.StopAutonomousDatabaseRequest{ - AutonomousDatabaseId: common.String(adbOCID), +func (d *databaseService) DownloadWallet(adb *dbv1alpha1.AutonomousDatabase) (resp database.GenerateAutonomousDatabaseWalletResponse, err error) { + // Prepare wallet password + walletPassword, err := d.readPassword(adb.Namespace, adb.Spec.Details.Wallet.Password) + if err != nil { + return resp, err } - resp, err = dbClient.StopAutonomousDatabase(context.TODO(), stopRequest) - return -} + // Download a Wallet + req := database.GenerateAutonomousDatabaseWalletRequest{ + AutonomousDatabaseId: adb.Spec.Details.AutonomousDatabaseOCID, + GenerateAutonomousDatabaseWalletDetails: database.GenerateAutonomousDatabaseWalletDetails{ + Password: walletPassword, + }, + } -// DeleteAutonomousDatabase terminates an Autonomous Database in OCI -func DeleteAutonomousDatabase(dbClient database.DatabaseClient, adbOCID string) (resp database.DeleteAutonomousDatabaseResponse, err error) { - deleteRequest := database.DeleteAutonomousDatabaseRequest{ - AutonomousDatabaseId: common.String(adbOCID), + // Send the request using the service client + resp, err = d.dbClient.GenerateAutonomousDatabaseWallet(context.TODO(), req) + if err != nil { + return resp, err } - return dbClient.DeleteAutonomousDatabase(context.TODO(), deleteRequest) + return resp, nil } -func RestoreAutonomousDatabase(dbClient database.DatabaseClient, adbOCID string, sdkTime *common.SDKTime) (resp database.RestoreAutonomousDatabaseResponse, err error) { +func (d *databaseService) RestoreAutonomousDatabase(adbOCID string, sdkTime common.SDKTime) (database.RestoreAutonomousDatabaseResponse, error) { request := database.RestoreAutonomousDatabaseRequest{ AutonomousDatabaseId: common.String(adbOCID), RestoreAutonomousDatabaseDetails: database.RestoreAutonomousDatabaseDetails{ - Timestamp: sdkTime, + Timestamp: &sdkTime, }, } - return dbClient.RestoreAutonomousDatabase(context.TODO(), request) + return d.dbClient.RestoreAutonomousDatabase(context.TODO(), request) } -// ListAutonomousDatabaseBackups returns a list of Autonomous Database backups -func ListAutonomousDatabaseBackups(dbClient database.DatabaseClient, adb *dbv1alpha1.AutonomousDatabase) (resp database.ListAutonomousDatabaseBackupsResponse, err error) { - if adb.Spec.Details.AutonomousDatabaseOCID == nil { - return resp, nil - } - +func (d *databaseService) ListAutonomousDatabaseBackups(adbOCID string) (database.ListAutonomousDatabaseBackupsResponse, error) { listBackupRequest := database.ListAutonomousDatabaseBackupsRequest{ - AutonomousDatabaseId: adb.Spec.Details.AutonomousDatabaseOCID, + AutonomousDatabaseId: common.String(adbOCID), } - return dbClient.ListAutonomousDatabaseBackups(context.TODO(), listBackupRequest) + return d.dbClient.ListAutonomousDatabaseBackups(context.TODO(), listBackupRequest) } -// CreateAutonomousDatabaseBackup creates an backup of Autonomous Database -func CreateAutonomousDatabaseBackup(logger logr.Logger, dbClient database.DatabaseClient, adbBackup *dbv1alpha1.AutonomousDatabaseBackup) (resp database.CreateAutonomousDatabaseBackupResponse, err error) { - logger.Info("Creating Autonomous Database backup " + adbBackup.GetName()) - +func (d *databaseService) CreateAutonomousDatabaseBackup(adbBackup *dbv1alpha1.AutonomousDatabaseBackup) (database.CreateAutonomousDatabaseBackupResponse, error) { createBackupRequest := database.CreateAutonomousDatabaseBackupRequest{ CreateAutonomousDatabaseBackupDetails: database.CreateAutonomousDatabaseBackupDetails{ AutonomousDatabaseId: &adbBackup.Spec.AutonomousDatabaseOCID, @@ -606,83 +378,13 @@ func CreateAutonomousDatabaseBackup(logger logr.Logger, dbClient database.Databa createBackupRequest.DisplayName = common.String(adbBackup.GetName()) } - return dbClient.CreateAutonomousDatabaseBackup(context.TODO(), createBackupRequest) + return d.dbClient.CreateAutonomousDatabaseBackup(context.TODO(), createBackupRequest) } -// GetAutonomousDatabaseBackup returns the response of GetAutonomousDatabaseBackupRequest -func GetAutonomousDatabaseBackup(dbClient database.DatabaseClient, backupOCID string) (resp database.GetAutonomousDatabaseBackupResponse, err error) { +func (d *databaseService) GetAutonomousDatabaseBackup(backupOCID string) (database.GetAutonomousDatabaseBackupResponse, error) { getBackupRequest := database.GetAutonomousDatabaseBackupRequest{ AutonomousDatabaseBackupId: common.String(backupOCID), } - return dbClient.GetAutonomousDatabaseBackup(context.TODO(), getBackupRequest) -} - -func GetWorkStatusAndWait(logger logr.Logger, workClient workrequests.WorkRequestClient, opcWorkRequestID *string) (workrequests.WorkRequestStatusEnum, error) { - logger.Info("Waiting for the work request to finish. opcWorkRequestID = " + *opcWorkRequestID) - - // retries until the work status is SUCCEEDED, FAILED or CANCELED - retryPolicy := getCompleteWorkRetryPolicy() - - resp, err := GetWorkRequest(workClient, opcWorkRequestID, &retryPolicy) - if err != nil { - return resp.Status, err - } - - return resp.Status, nil -} - -func GetWorkRequest(workClient workrequests.WorkRequestClient, - opcWorkRequestID *string, - retryPolicy *common.RetryPolicy) (response workrequests.GetWorkRequestResponse, err error) { - - workRequest := workrequests.GetWorkRequestRequest{ - WorkRequestId: opcWorkRequestID, - } - - if retryPolicy != nil { - workRequest.RequestMetadata = common.RequestMetadata{ - RetryPolicy: retryPolicy, - } - } - - return workClient.GetWorkRequest(context.TODO(), workRequest) -} - -func getCompleteWorkRetryPolicy() common.RetryPolicy { - shouldRetry := func(r common.OCIOperationResponse) bool { - if _, isServiceError := common.IsServiceError(r.Error); isServiceError { - // Don't retry if it's service error. Sometimes it could be network error or other errors which prevents - // request send to server; we do the retry in these cases. - return false - } - - if converted, ok := r.Response.(workrequests.GetWorkRequestResponse); ok { - // do the retry until WorkReqeut Status is Succeeded - ignore case (BMI-2652) - return converted.Status != workrequests.WorkRequestStatusSucceeded && - converted.Status != workrequests.WorkRequestStatusFailed && - converted.Status != workrequests.WorkRequestStatusCanceled - } - - return true - } - - return getRetryPolicy(shouldRetry) -} - -func getRetryPolicy(retryOperation func(common.OCIOperationResponse) bool) common.RetryPolicy { - // maximum times of retry (~30mins) - attempts := uint(63) - - nextDuration := func(r common.OCIOperationResponse) time.Duration { - // Wait longer for next retry when your previous one failed - // this function will return the duration as: - // 1s, 2s, 4s, 8s, 16s, 30s, 30s etc... - if r.AttemptNumber <= 5 { - return time.Duration(math.Pow(float64(2), float64(r.AttemptNumber-1))) * time.Second - } - return time.Duration(30) * time.Second - } - - return common.NewRetryPolicy(attempts, retryOperation, nextDuration) + return d.dbClient.GetAutonomousDatabaseBackup(context.TODO(), getBackupRequest) } diff --git a/commons/oci/ociutil/time.go b/commons/oci/ociutil/time.go deleted file mode 100644 index 921dfe3d..00000000 --- a/commons/oci/ociutil/time.go +++ /dev/null @@ -1,23 +0,0 @@ -package ociutil - -import ( - "time" - - "github.com/oracle/oci-go-sdk/v54/common" -) - -// Follow the format of the Display Name -const displayFormat = "2006-01-02 15:04:05 MST" - -func FormatSDKTime(dateTime time.Time) string { - return dateTime.Format(displayFormat) -} - -func ParseDisplayTime(val string) (*common.SDKTime, error) { - parsedTime, err := time.Parse(displayFormat, val) - if err != nil { - return nil, err - } - sdkTime := common.SDKTime{Time: parsedTime} - return &sdkTime, nil -} diff --git a/commons/oci/provider.go b/commons/oci/provider.go index 02fb09bb..c7592002 100644 --- a/commons/oci/provider.go +++ b/commons/oci/provider.go @@ -39,7 +39,6 @@ package oci import ( - "context" "errors" "github.com/oracle/oci-go-sdk/v54/common" @@ -47,8 +46,7 @@ import ( "sigs.k8s.io/controller-runtime/pkg/client" - corev1 "k8s.io/api/core/v1" - "k8s.io/apimachinery/pkg/types" + "github.com/oracle/oracle-database-operator/commons/k8s" ) const ( @@ -85,13 +83,9 @@ func GetOCIProvider(kubeClient client.Client, authData APIKeyAuth) (common.Confi func getProviderWithAPIKey(kubeClient client.Client, authData APIKeyAuth) (common.ConfigurationProvider, error) { var region, fingerprint, user, tenancy, passphrase, privatekeyValue string - // Read ConfigMap - configMapNamespacedName := types.NamespacedName{ - Namespace: authData.Namespace, - Name: *authData.ConfigMapName, - } - ociConfigMap := &corev1.ConfigMap{} - if err := kubeClient.Get(context.TODO(), configMapNamespacedName, ociConfigMap); err != nil { + // Prepare ConfigMap + ociConfigMap, err := k8s.FetchConfigMap(kubeClient, authData.Namespace, *authData.ConfigMapName) + if err != nil { return nil, err } @@ -111,24 +105,11 @@ func getProviderWithAPIKey(kubeClient client.Client, authData APIKeyAuth) (commo } } - // Read Secret - secretNamespacedName := types.NamespacedName{ - Namespace: authData.Namespace, - Name: *authData.SecretName, - } - - privatekeySecret := &corev1.Secret{} - if err := kubeClient.Get(context.TODO(), secretNamespacedName, privatekeySecret); err != nil { + // Prepare privatekey value + privatekeyValue, err = k8s.GetSecretValue(kubeClient, authData.Namespace, *authData.SecretName, privatekeyKey) + if err != nil { return nil, err } - for key, val := range privatekeySecret.Data { - if key == privatekeyKey { - privatekeyValue = string(val) - } else { - return nil, errors.New("Unable to identify the key: " + key) - } - } - return common.NewRawConfigurationProvider(tenancy, user, region, fingerprint, privatekeyValue, &passphrase), nil } diff --git a/commons/oci/vault.go b/commons/oci/vault.go index be48d176..71b162ba 100644 --- a/commons/oci/vault.go +++ b/commons/oci/vault.go @@ -42,16 +42,41 @@ import ( "context" "encoding/base64" + "github.com/go-logr/logr" "github.com/oracle/oci-go-sdk/v54/common" "github.com/oracle/oci-go-sdk/v54/secrets" ) -func getValueFromVaultSecret(secretClient secrets.SecretsClient, vaultSecretOCID string) (string, error) { +type VaultService interface { + GetSecretValue(vaultSecretOCID string) (string, error) +} + +type vaultService struct { + logger logr.Logger + secretClient secrets.SecretsClient +} + +func NewVaultService( + logger logr.Logger, + provider common.ConfigurationProvider) (VaultService, error) { + + secretClient, err := secrets.NewSecretsClientWithConfigurationProvider(provider) + if err != nil { + return nil, err + } + + return &vaultService{ + logger: logger, + secretClient: secretClient, + }, nil +} + +func (v *vaultService) GetSecretValue(vaultSecretOCID string) (string, error) { request := secrets.GetSecretBundleRequest{ SecretId: common.String(vaultSecretOCID), } - response, err := secretClient.GetSecretBundle(context.TODO(), request) + response, err := v.secretClient.GetSecretBundle(context.TODO(), request) if err != nil { return "", err } diff --git a/commons/oci/wallet.go b/commons/oci/wallet.go index 289b5bbf..818ce7c0 100644 --- a/commons/oci/wallet.go +++ b/commons/oci/wallet.go @@ -40,124 +40,45 @@ package oci import ( "archive/zip" - "context" - "errors" - "fmt" "io" "io/ioutil" - "math" - "time" - - "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v54/common" - "github.com/oracle/oci-go-sdk/v54/database" - "github.com/oracle/oci-go-sdk/v54/secrets" - "sigs.k8s.io/controller-runtime/pkg/client" - - "k8s.io/apimachinery/pkg/types" - - dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" ) -// GetWallet downloads the wallet using the given information in the AutonomousDatabase object. -// The function then unzips the wallet and returns a map object which holds the byte values of the unzipped files. -func GetWallet(logger logr.Logger, kubeClient client.Client, dbClient database.DatabaseClient, secretClient secrets.SecretsClient, adb *dbv1alpha1.AutonomousDatabase) (map[string][]byte, error) { - // Get the wallet password from Secret then Vault Secret - walletPassword, err := getWalletPassword(logger, kubeClient, secretClient, adb) +// ExtractWallet extracts the wallet and returns a map object which holds the byte values of the unzipped files. +func ExtractWallet(content io.ReadCloser) (map[string][]byte, error) { + path, err := saveWalletZip(content) if err != nil { return nil, err } - // Request to download a wallet with the given password - resp, err := generateAutonomousDatabaseWallet(dbClient, *adb.Spec.Details.AutonomousDatabaseOCID, walletPassword) - if err != nil { - return nil, err - } - - // Unzip the file - outZip, err := ioutil.TempFile("", "wallet*.zip") - if err != nil { - return nil, err - } - defer outZip.Close() - - if _, err := io.Copy(outZip, resp.Content); err != nil { - return nil, err - } - - data, err := unzipWallet(outZip.Name()) + data, err := unzipWallet(path) if err != nil { return nil, err } return data, nil } -func getWalletPassword(logger logr.Logger, kubeClient client.Client, secretClient secrets.SecretsClient, adb *dbv1alpha1.AutonomousDatabase) (string, error) { - if adb.Spec.Details.Wallet.Password.K8sSecretName != nil { - logger.Info(fmt.Sprintf("Getting wallet password from Secret %s", *adb.Spec.Details.Wallet.Password.K8sSecretName)) - - namespacedName := types.NamespacedName{ - Namespace: adb.GetNamespace(), - Name: *adb.Spec.Details.Wallet.Password.K8sSecretName, - } - - key := *adb.Spec.Details.Wallet.Password.K8sSecretName - walletPassword, err := getValueFromKubeSecret(kubeClient, namespacedName, key) - if err != nil { - return "", err - } - return walletPassword, nil - - } else if adb.Spec.Details.Wallet.Password.OCISecretOCID != nil { - logger.Info(fmt.Sprintf("Getting wallet password from OCI Vault Secret OCID %s", *adb.Spec.Details.Wallet.Password.OCISecretOCID)) - - walletPassword, err := getValueFromVaultSecret(secretClient, *adb.Spec.Details.Wallet.Password.OCISecretOCID) - if err != nil { - return "", err - } - return walletPassword, nil - } - return "", errors.New("should provide either InstancewalletPasswordSecret or a InstancewalletPasswordId") -} - -func generateAutonomousDatabaseWallet(dbClient database.DatabaseClient, adbOCID string, walletPassword string) (database.GenerateAutonomousDatabaseWalletResponse, error) { - - // maximum times of retry - attempts := uint(10) - - // retry for all non-200 status code - retryOnAllNon200ResponseCodes := func(r common.OCIOperationResponse) bool { - return !(r.Error == nil && 199 < r.Response.HTTPResponse().StatusCode && r.Response.HTTPResponse().StatusCode < 300) - } - - nextDuration := func(r common.OCIOperationResponse) time.Duration { - // Wait longer for next retry when your previous one failed - // this function will return the duration as: - // 1s, 2s, 4s, 8s, 16s, 32s, 64s etc... - return time.Duration(math.Pow(float64(2), float64(r.AttemptNumber-1))) * time.Second +func saveWalletZip(content io.ReadCloser) (string, error) { + // Create a temp file wallet*.zip + const walletFileName = "wallet*.zip" + outZip, err := ioutil.TempFile("", walletFileName) + if err != nil { + return "", err } + defer outZip.Close() - walletRetryPolicy := common.NewRetryPolicy(attempts, retryOnAllNon200ResponseCodes, nextDuration) - - // Download a Wallet - req := database.GenerateAutonomousDatabaseWalletRequest{ - AutonomousDatabaseId: common.String(adbOCID), - GenerateAutonomousDatabaseWalletDetails: database.GenerateAutonomousDatabaseWalletDetails{ - Password: common.String(walletPassword), - }, - RequestMetadata: common.RequestMetadata{ - RetryPolicy: &walletRetryPolicy, - }, + // Save the wallet in wallet*.zip + if _, err := io.Copy(outZip, content); err != nil { + return "", err } - // Send the request using the service client - return dbClient.GenerateAutonomousDatabaseWallet(context.TODO(), req) + return outZip.Name(), nil } -func unzipWallet(filename string) (map[string][]byte, error) { +func unzipWallet(path string) (map[string][]byte, error) { data := map[string][]byte{} - reader, err := zip.OpenReader(filename) + reader, err := zip.OpenReader(path) if err != nil { return data, err } diff --git a/commons/oci/workrequest.go b/commons/oci/workrequest.go new file mode 100644 index 00000000..4946bc86 --- /dev/null +++ b/commons/oci/workrequest.go @@ -0,0 +1,131 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package oci + +import ( + "context" + "math" + "time" + + "github.com/go-logr/logr" + "sigs.k8s.io/controller-runtime/pkg/client" + + "github.com/oracle/oci-go-sdk/v54/common" + "github.com/oracle/oci-go-sdk/v54/workrequests" +) + +type WorkRequestService interface { + Wait(opcWorkRequestID string) (workrequests.WorkRequestStatusEnum, error) +} + +type workRequestService struct { + logger logr.Logger + workClient workrequests.WorkRequestClient +} + +func NewWorkRequestService( + logger logr.Logger, + kubeClient client.Client, + provider common.ConfigurationProvider) (WorkRequestService, error) { + + workClient, err := workrequests.NewWorkRequestClientWithConfigurationProvider(provider) + if err != nil { + return nil, err + } + + return &workRequestService{ + logger: logger, + workClient: workClient, + }, nil +} + +func (w workRequestService) getRetryPolicy() common.RetryPolicy { + shouldRetry := func(r common.OCIOperationResponse) bool { + if _, isServiceError := common.IsServiceError(r.Error); isServiceError { + // Don't retry if it's service error. Sometimes it could be network error or other errors which prevents + // request send to server; we do the retry in these cases. + return false + } + + if converted, ok := r.Response.(workrequests.GetWorkRequestResponse); ok { + // do the retry until WorkReqeut Status is Succeeded - ignore case (BMI-2652) + return converted.Status != workrequests.WorkRequestStatusSucceeded && + converted.Status != workrequests.WorkRequestStatusFailed && + converted.Status != workrequests.WorkRequestStatusCanceled + } + + return true + } + + // maximum times of retry (~30mins) + attempts := uint(63) + + nextDuration := func(r common.OCIOperationResponse) time.Duration { + // Wait longer for next retry when your previous one failed + // this function will return the duration as: + // 1s, 2s, 4s, 8s, 16s, 30s, 30s etc... + if r.AttemptNumber <= 5 { + return time.Duration(math.Pow(float64(2), float64(r.AttemptNumber-1))) * time.Second + } + return time.Duration(30) * time.Second + } + + return common.NewRetryPolicy(attempts, shouldRetry, nextDuration) +} + +func (w *workRequestService) Wait(opcWorkRequestID string) (workrequests.WorkRequestStatusEnum, error) { + w.logger.Info("Waiting for the work request to finish. opcWorkRequestID = " + opcWorkRequestID) + + // retries until the work status is SUCCEEDED, FAILED or CANCELED + retryPolicy := w.getRetryPolicy() + + workRequest := workrequests.GetWorkRequestRequest{ + WorkRequestId: common.String(opcWorkRequestID), + RequestMetadata: common.RequestMetadata{ + RetryPolicy: &retryPolicy, + }, + } + + resp, err := w.workClient.GetWorkRequest(context.TODO(), workRequest) + if err != nil { + return resp.Status, err + } + + return resp.Status, nil +} diff --git a/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml b/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml index 200c8ccb..ccaf8317 100644 --- a/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml +++ b/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml @@ -77,8 +77,6 @@ spec: description: AutonomousDatabaseBackupStatus defines the observed state of AutonomousDatabaseBackup properties: - autonomousDatabaseBackupOCID: - type: string autonomousDatabaseOCID: type: string compartmentOCID: @@ -87,8 +85,6 @@ spec: type: string dbName: type: string - displayName: - type: string isAutomatic: type: boolean lifecycleState: @@ -105,12 +101,10 @@ spec: type: string' type: string required: - - autonomousDatabaseBackupOCID - autonomousDatabaseOCID - compartmentOCID - dbDisplayName - dbName - - displayName - isAutomatic - lifecycleState - type diff --git a/config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml b/config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml index 5b298640..b7508fc8 100644 --- a/config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml +++ b/config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml @@ -20,8 +20,8 @@ spec: scope: Namespaced versions: - additionalPrinterColumns: - - jsonPath: .status.lifecycleState - name: State + - jsonPath: .status.status + name: Status type: string - jsonPath: .status.displayName name: DisplayName @@ -83,14 +83,13 @@ spec: of cluster Important: Run "make" to regenerate code after modifying this file' type: string - lifecycleState: - description: 'WorkRequestStatusEnum Enum with underlying type: string' + status: type: string required: - autonomousDatabaseOCID - dbName - displayName - - lifecycleState + - status type: object type: object served: true diff --git a/config/crd/bases/database.oracle.com_autonomousdatabases.yaml b/config/crd/bases/database.oracle.com_autonomousdatabases.yaml index 922b4897..8e13d12b 100644 --- a/config/crd/bases/database.oracle.com_autonomousdatabases.yaml +++ b/config/crd/bases/database.oracle.com_autonomousdatabases.yaml @@ -20,22 +20,22 @@ spec: scope: Namespaced versions: - additionalPrinterColumns: - - jsonPath: .status.displayName + - jsonPath: .spec.details.displayName name: Display Name type: string - jsonPath: .status.lifecycleState name: State type: string - - jsonPath: .status.isDedicated + - jsonPath: .spec.details.isDedicated name: Dedicated type: string - - jsonPath: .status.cpuCoreCount + - jsonPath: .spec.details.cpuCoreCount name: OCPUs type: integer - - jsonPath: .status.dataStorageSizeInTBs + - jsonPath: .spec.details.dataStorageSizeInTBs name: Storage (TB) type: integer - - jsonPath: .status.dbWorkload + - jsonPath: .spec.details.dbWorkload name: Workload Type type: string - jsonPath: .status.timeCreated @@ -107,6 +107,13 @@ spec: type: boolean isDedicated: type: boolean + licenseModel: + description: 'AutonomousDatabaseLicenseModelEnum Enum with underlying + type: string' + enum: + - LICENSE_INCLUDED + - BRING_YOUR_OWN_LICENSE + type: string lifecycleState: description: 'AutonomousDatabaseLifecycleStateEnum Enum with underlying type: string' @@ -173,34 +180,97 @@ spec: items: properties: connectionStrings: - additionalProperties: - type: string - type: object + items: + properties: + connectionString: + type: string + tnsName: + type: string + type: object + type: array tlsAuthentication: type: string required: - connectionStrings type: object type: array - cpuCoreCount: - type: integer - dataStorageSizeInTBs: - type: integer - dbWorkload: - description: 'AutonomousDatabaseDbWorkloadEnum Enum with underlying - type: string' - type: string - displayName: + conditions: + items: + description: "Condition contains details for one aspect of the current + state of this API Resource. --- This struct is intended for direct + use as an array at the field path .status.conditions. For example, + type FooStatus struct{ // Represents the observations of a + foo's current state. // Known .status.conditions.type are: + \"Available\", \"Progressing\", and \"Degraded\" // +patchMergeKey=type + \ // +patchStrategy=merge // +listType=map // +listMapKey=type + \ Conditions []metav1.Condition `json:\"conditions,omitempty\" + patchStrategy:\"merge\" patchMergeKey:\"type\" protobuf:\"bytes,1,rep,name=conditions\"` + \n // other fields }" + properties: + lastTransitionTime: + description: lastTransitionTime is the last time the condition + transitioned from one status to another. This should be when + the underlying condition changed. If that is not known, then + using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: message is a human readable message indicating + details about the transition. This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: observedGeneration represents the .metadata.generation + that the condition was set based upon. For instance, if .metadata.generation + is currently 12, but the .status.conditions[x].observedGeneration + is 9, the condition is out of date with respect to the current + state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: reason contains a programmatic identifier indicating + the reason for the condition's last transition. Producers + of specific condition types may define expected values and + meanings for this field, and whether the values are considered + a guaranteed API. The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + --- Many .condition.type values are consistent across resources + like Available, but because arbitrary conditions can be useful + (see .node.status.conditions), the ability to deconflict is + important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + lifecycleState: description: 'INSERT ADDITIONAL STATUS FIELD - define observed state of cluster Important: Run "make" to regenerate code after modifying this file' type: string - isDedicated: - type: string - lifecycleState: - description: 'AutonomousDatabaseLifecycleStateEnum Enum with underlying - type: string' - type: string timeCreated: type: string type: object diff --git a/controllers/database/autonomousdatabase_controller.go b/controllers/database/autonomousdatabase_controller.go index 7d552d4b..843d57f7 100644 --- a/controllers/database/autonomousdatabase_controller.go +++ b/controllers/database/autonomousdatabase_controller.go @@ -40,16 +40,22 @@ package controllers import ( "context" + "encoding/json" + "errors" "fmt" - "reflect" + "regexp" + "strings" + "time" "github.com/go-logr/logr" "github.com/oracle/oci-go-sdk/v54/database" - "github.com/oracle/oci-go-sdk/v54/secrets" - "github.com/oracle/oci-go-sdk/v54/workrequests" + corev1 "k8s.io/api/core/v1" apiErrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/types" + "k8s.io/client-go/tools/record" + "k8s.io/client-go/util/retry" ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/controller-runtime/pkg/controller" @@ -57,23 +63,33 @@ import ( "sigs.k8s.io/controller-runtime/pkg/predicate" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" - adbutil "github.com/oracle/oracle-database-operator/commons/autonomousdatabase" - "github.com/oracle/oracle-database-operator/commons/finalizer" + "github.com/oracle/oracle-database-operator/commons/annotations" + "github.com/oracle/oracle-database-operator/commons/k8s" "github.com/oracle/oracle-database-operator/commons/oci" ) -// AutonomousDatabaseReconciler reconciles a AutonomousDatabase object +// *AutonomousDatabaseReconciler reconciles a AutonomousDatabase object type AutonomousDatabaseReconciler struct { KubeClient client.Client Log logr.Logger Scheme *runtime.Scheme + Recorder record.EventRecorder + + adbService oci.DatabaseService + workService oci.WorkRequestService + lastSucSpec *dbv1alpha1.AutonomousDatabaseSpec } +// To requeue after 60 secs allowing graceful state changes +var requeueResult ctrl.Result = ctrl.Result{Requeue: true, RequeueAfter: 60 * time.Second} +var emptyResult ctrl.Result = ctrl.Result{} + // SetupWithManager function func (r *AutonomousDatabaseReconciler) SetupWithManager(mgr ctrl.Manager) error { return ctrl.NewControllerManagedBy(mgr). For(&dbv1alpha1.AutonomousDatabase{}). - WithEventFilter(r.eventFilterPredicate()). + WithEventFilter(predicate.And(predicate.GenerationChangedPredicate{}, r.eventFilterPredicate())). + // WithEventFilter(r.eventFilterPredicate()). WithOptions(controller.Options{MaxConcurrentReconciles: 50}). // ReconcileHandler is never invoked concurrently with the same object. Complete(r) } @@ -81,20 +97,16 @@ func (r *AutonomousDatabaseReconciler) SetupWithManager(mgr ctrl.Manager) error func (r *AutonomousDatabaseReconciler) eventFilterPredicate() predicate.Predicate { pred := predicate.Funcs{ UpdateFunc: func(e event.UpdateEvent) bool { - oldADB := e.ObjectOld.(*dbv1alpha1.AutonomousDatabase) newADB := e.ObjectNew.(*dbv1alpha1.AutonomousDatabase) - // Reconciliation should NOT happen if the lastSuccessfulSpec annotation or status.state changes. - oldSucSpec := oldADB.GetAnnotations()[dbv1alpha1.LastSuccessfulSpec] - newSucSpec := newADB.GetAnnotations()[dbv1alpha1.LastSuccessfulSpec] - - lastSucSpecChanged := oldSucSpec != newSucSpec - stateChanged := oldADB.Status.LifecycleState != newADB.Status.LifecycleState - if lastSucSpecChanged || stateChanged { - // Don't enqueue request + // Don't enqueue request + if newADB.Status.LifecycleState == database.AutonomousDatabaseLifecycleStateBackupInProgress || + newADB.Status.LifecycleState == database.AutonomousDatabaseLifecycleStateUpdating || + newADB.Status.LifecycleState == database.AutonomousDatabaseLifecycleStateStarting || + newADB.Status.LifecycleState == database.AutonomousDatabaseLifecycleStateStopping || + newADB.Status.LifecycleState == database.AutonomousDatabaseLifecycleStateTerminating { return false } - // Enqueue request return true }, DeleteFunc: func(e event.DeleteEvent) bool { @@ -116,7 +128,9 @@ func (r *AutonomousDatabaseReconciler) eventFilterPredicate() predicate.Predicat // It go to the beggining of the reconcile if an error is returned. We won't return a error if it is related // to OCI, because the issues cannot be solved by re-run the reconcile. func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { - currentLogger := r.Log.WithValues("Namespaced/Name", req.NamespacedName) + logger := r.Log.WithValues("Namespaced/Name", req.NamespacedName) + + var err error // Get the autonomousdatabase instance from the cluster adb := &dbv1alpha1.AutonomousDatabase{} @@ -124,315 +138,803 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R // Ignore not-found errors, since they can't be fixed by an immediate requeue. // No need to change the since we don't know if we obtain the object. if apiErrors.IsNotFound(err) { - return ctrl.Result{}, nil + return emptyResult, nil } - return ctrl.Result{}, err + // Failed to get ADB, so we don't need to update the status + return emptyResult, err + } + + r.lastSucSpec, err = adb.GetLastSuccessfulSpec() + if err != nil { + return r.manageError(adb, err) } /****************************************************************** * Get OCI database client and work request client ******************************************************************/ + if err := r.setupOCIClients(adb); err != nil { + logger.Error(err, "Fail to setup OCI clients") + + return r.manageError(adb, err) + } + + logger.Info("OCI clients configured succesfully") + + /****************************************************************** + * Register/unregister finalizer + * Deletion timestamp will be added to a object before it is deleted. + * Kubernetes server calls the clean up function if a finalizer exitsts, and won't delete the real object until + * all the finalizers are removed from the object metadata. + * Refer to this page for more details of using finalizers: https://kubernetes.io/blog/2021/05/14/using-finalizers-to-control-deletion/ + ******************************************************************/ + isADBDeleteTrue, err := r.validateFinalizer(adb) + if err != nil { + return r.manageError(adb, err) + } + if isADBDeleteTrue { + return emptyResult, nil + } + + /****************************************************************** + * Determine which Database operations need to be executed by checking the changes to spec.details. + * There are three scenario: + * 1. provision operation. The AutonomousDatabaseOCID is missing, and the LastSucSpec annotation is missing. + * 2. bind operation. The AutonomousDatabaseOCID is provided, but the LastSucSpec annotation is missing. + * 3. update operation. Every changes other than the above two cases goes here. + * Afterwards, update the resource from the remote database in OCI. This step will be executed right after + * the above three cases during every reconcile. + /******************************************************************/ + // difADB is nil when the action is PROVISION or BIND. + // Use difADB to identify which fields are updated when it's UPDATE operation. + action, difADB, err := r.determineAction(adb) + if err != nil { + return r.manageError(adb, err) + } + + switch action { + case adbActionProvision: + if err := r.createADB(adb); err != nil { + return r.manageError(adb, err) + } + + if err := r.downloadWallet(adb); err != nil { + return r.manageError(adb, err) + } + + err = r.syncResource(adb) + if err != nil { + return r.manageError(adb, err) + } + case adbActionBind: + fallthrough + case adbActionSync: + err = r.syncResource(adb) + if err != nil { + return r.manageError(adb, err) + } + + if err := r.downloadWallet(adb); err != nil { + return r.manageError(adb, err) + } + case adbActionUpdate: + if err := r.updateADB(adb, difADB); err != nil { + return r.manageError(adb, err) + } + + err = r.syncResource(adb) + if err != nil { + return r.manageError(adb, err) + } + } + + /***************************************************** + * Sync AutonomousDatabase Backups from OCI + *****************************************************/ + if err := r.syncBackupResources(adb); err != nil { + return r.manageError(adb, err) + } + + logger.Info("AutonomousDatabase reconciles successfully") + + return requeueResult, nil +} + +func (r *AutonomousDatabaseReconciler) setupOCIClients(adb *dbv1alpha1.AutonomousDatabase) error { + var err error + authData := oci.APIKeyAuth{ ConfigMapName: adb.Spec.OCIConfig.ConfigMapName, SecretName: adb.Spec.OCIConfig.SecretName, Namespace: adb.GetNamespace(), } + provider, err := oci.GetOCIProvider(r.KubeClient, authData) if err != nil { - currentLogger.Error(err, "Fail to get OCI provider") - - // Change the status to UNAVAILABLE - adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr - } - return ctrl.Result{}, nil + return err } - dbClient, err := database.NewDatabaseClientWithConfigurationProvider(provider) + r.adbService, err = oci.NewDatabaseService(r.Log, r.KubeClient, provider) if err != nil { - currentLogger.Error(err, "Fail to get OCI database client") - - // Change the status to UNAVAILABLE - adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr - } - return ctrl.Result{}, nil + return err } - secretClient, err := secrets.NewSecretsClientWithConfigurationProvider(provider) + r.workService, err = oci.NewWorkRequestService(r.Log, r.KubeClient, provider) if err != nil { - currentLogger.Error(err, "Fail to get OCI secret client") - - // Change the status to UNAVAILABLE - adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr - } - return ctrl.Result{}, nil + return err } - workClient, err := workrequests.NewWorkRequestClientWithConfigurationProvider(provider) - if err != nil { - currentLogger.Error(err, "Fail to get OCI work request client") + return nil +} + +func (r *AutonomousDatabaseReconciler) manageError(adb *dbv1alpha1.AutonomousDatabase, issue error) (ctrl.Result, error) { + // Rollback if lastSucSpec exists + if r.lastSucSpec != nil { + // Send event + r.Recorder.Event(adb, corev1.EventTypeWarning, "SpecRollback", issue.Error()) + + adb.Spec = *r.lastSucSpec - // Change the status to UNAVAILABLE - adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr + if err := r.updateResource(adb); err != nil { + // returns the error because we didn't trigger another Reconcile loop + return emptyResult, k8s.CombineErrors(issue, err) } - return ctrl.Result{}, nil + + r.Log.Error(issue, "Reconcile failed") + + // r.updateResource has already triggered another Reconcile loop, so we return a nil instead of an error + return emptyResult, nil + } else { + // Send event + r.Recorder.Event(adb, corev1.EventTypeWarning, "CreateFailed", issue.Error()) + + return emptyResult, issue } +} - currentLogger.Info("OCI provider configured succesfully") +func (r *AutonomousDatabaseReconciler) validateFinalizer(adb *dbv1alpha1.AutonomousDatabase) (isADBDeleteTrue bool, err error) { + logger := r.Log.WithName("finalizer") - /****************************************************************** - * Register/unregister finalizer - * Deletion timestamp will be added to a object before it is deleted. - * Kubernetes server calls the clean up function if a finalizer exitsts, and won't delete the real object until - * all the finalizers are removed from the object metadata. - * Refer to this page for more details of using finalizers: https://kubernetes.io/blog/2021/05/14/using-finalizers-to-control-deletion/ - ******************************************************************/ if adb.ObjectMeta.DeletionTimestamp.IsZero() { // The object is not being deleted - if *adb.Spec.HardLink && !finalizer.HasFinalizer(adb) { - finalizer.Register(r.KubeClient, adb) - currentLogger.Info("Finalizer registered successfully.") + if *adb.Spec.HardLink && !k8s.HasFinalizer(adb) { + if err := k8s.Register(r.KubeClient, adb); err != nil { + return false, err + } + logger.Info("Finalizer registered successfully.") - } else if !*adb.Spec.HardLink && finalizer.HasFinalizer(adb) { - finalizer.Unregister(r.KubeClient, adb) - currentLogger.Info("Finalizer unregistered successfully.") + } else if !*adb.Spec.HardLink && k8s.HasFinalizer(adb) { + if err := k8s.Unregister(r.KubeClient, adb); err != nil { + return false, err + } + logger.Info("Finalizer unregistered successfully.") } + return false, nil } else { // The object is being deleted if adb.Spec.Details.AutonomousDatabaseOCID == nil { - currentLogger.Info("Autonomous Database OCID is missing. Remove the resource only.") + logger.Info("Autonomous Database OCID is missing. Remove the resource only.") } else if adb.Status.LifecycleState != database.AutonomousDatabaseLifecycleStateTerminating && adb.Status.LifecycleState != database.AutonomousDatabaseLifecycleStateTerminated { // Don't send terminate request if the database is terminating or already terminated - currentLogger.Info("Terminate Autonomous Database: " + *adb.Spec.Details.DbName) - if _, err := oci.DeleteAutonomousDatabase(dbClient, *adb.Spec.Details.AutonomousDatabaseOCID); err != nil { - currentLogger.Error(err, "Fail to terminate Autonomous Database") - - // Change the status to UNAVAILABLE - adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr - } + logger.Info("Terminate Autonomous Database: " + *adb.Spec.Details.DbName) + + if err := r.deleteAutonomousDatabase(adb); err != nil { + return true, err } } - finalizer.Unregister(r.KubeClient, adb) - currentLogger.Info("Finalizer unregistered successfully.") + k8s.Unregister(r.KubeClient, adb) + logger.Info("Finalizer unregistered successfully.") // Stop reconciliation as the item is being deleted - return ctrl.Result{}, nil + return true, nil } +} - /****************************************************************** - * Determine which Database operations need to be executed by checking the changes to spec.details. - * There are three scenario: - * 1. provision operation. The AutonomousDatabaseOCID is missing, and the LastSucSpec annotation is missing. - * 2. bind operation. The AutonomousDatabaseOCID is provided, but the LastSucSpec annotation is missing. - * 3. update operation. Every changes other than the above two cases goes here. - * Afterwards, update the resource from the remote database in OCI. This step will be executed right after - * the above three cases during every reconcile. - /******************************************************************/ - lastSucSpec, err := adb.GetLastSuccessfulSpec() +func (r *AutonomousDatabaseReconciler) updateLastSuccessfulSpec(adb *dbv1alpha1.AutonomousDatabase) error { + specBytes, err := json.Marshal(adb.Spec) if err != nil { - return ctrl.Result{}, err + return err } - if lastSucSpec == nil || !reflect.DeepEqual(lastSucSpec.Details, adb.Spec.Details) { - // spec.details changes - if adb.Spec.Details.AutonomousDatabaseOCID == nil && lastSucSpec == nil { - // If no AutonomousDatabaseOCID specified, create a database - // Update from yaml file might not have an AutonomousDatabaseOCID. Don't create a database if it already has last successful spec. - currentLogger.Info("AutonomousDatabase provisioning") + anns := map[string]string{ + dbv1alpha1.LastSuccessfulSpec: string(specBytes), + } - resp, err := oci.CreateAutonomousDatabase(currentLogger, r.KubeClient, dbClient, secretClient, adb) + return annotations.SetAnnotations(r.KubeClient, adb, anns) +} - if err != nil { - currentLogger.Error(err, "Fail to provision and get Autonomous Database OCID") +// updateResource updates the specification, the status of AutonomousDatabase resource, and the lastSucSpec +func (r *AutonomousDatabaseReconciler) updateResource(adb *dbv1alpha1.AutonomousDatabase) error { + // Update the status first to prevent unwanted Reconcile() + if err := r.updateResourceStatus(adb); err != nil { + return err + } - // Change the status to UNAVAILABLE - adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr - } - // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue - return ctrl.Result{}, nil - } + // Update the spec + if err := retry.RetryOnConflict(retry.DefaultRetry, func() error { + curADB := &dbv1alpha1.AutonomousDatabase{} - adb.Spec.Details.AutonomousDatabaseOCID = resp.AutonomousDatabase.Id + namespacedName := types.NamespacedName{ + Namespace: adb.GetNamespace(), + Name: adb.GetName(), + } - if err := adbutil.UpdateAutonomousDatabaseStatusAndWait(currentLogger, r.KubeClient, workClient, adb, - resp.AutonomousDatabase.LifecycleState, resp.OpcWorkRequestId); err != nil { - currentLogger.Error(err, "Work request faied. opcWorkRequestID = "+*resp.OpcWorkRequestId) - } + if err := r.KubeClient.Get(context.TODO(), namespacedName, curADB); err != nil { + return err + } - currentLogger.Info("AutonomousDatabase " + *adb.Spec.Details.DbName + " provisioned succesfully") + curADB.Spec.Details = adb.Spec.Details + return r.KubeClient.Update(context.TODO(), curADB) + }); err != nil { + return err + } - } else if adb.Spec.Details.AutonomousDatabaseOCID != nil && lastSucSpec == nil { - // Binding operation. We have the database ID but hasn't gotten complete infromation from OCI. - // The next step is to get AutonomousDatabse details from a remote instance. + if err := r.updateLastSuccessfulSpec(adb); err != nil { + return err + } - adb, err = oci.GetAutonomousDatabaseResource(currentLogger, dbClient, adb) - if err != nil { - currentLogger.Error(err, "Fail to get Autonomous Database") + return nil +} - // Change the status to UNAVAILABLE - adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr - } - return ctrl.Result{}, nil - } +func (r *AutonomousDatabaseReconciler) updateResourceStatus(adb *dbv1alpha1.AutonomousDatabase) error { + return retry.RetryOnConflict(retry.DefaultRetry, func() error { + curADB := &dbv1alpha1.AutonomousDatabase{} + + namespacedName := types.NamespacedName{ + Namespace: adb.GetNamespace(), + Name: adb.GetName(), + } + + if err := r.KubeClient.Get(context.TODO(), namespacedName, curADB); err != nil { + return err + } + + curADB.Status = adb.Status + return r.KubeClient.Status().Update(context.TODO(), curADB) + }) +} + +func (r *AutonomousDatabaseReconciler) syncResource(adb *dbv1alpha1.AutonomousDatabase) error { + // Get the information from OCI + resp, err := r.adbService.GetAutonomousDatabase(*adb.Spec.Details.AutonomousDatabaseOCID) + if err != nil { + return err + } + + adb.UpdateFromOCIADB(resp.AutonomousDatabase) + + if err := r.updateResource(adb); err != nil { + return err + } + + return nil +} + +type ADBActionEnum string + +const ( + adbActionProvision = "PROVISION" + adbActionBind = "BIND" + adbActionUpdate = "UPDATE" + adbActionSync = "SYNC" +) + +func (r *AutonomousDatabaseReconciler) determineAction(adb *dbv1alpha1.AutonomousDatabase) (ADBActionEnum, *dbv1alpha1.AutonomousDatabase, error) { + if r.lastSucSpec == nil { + if adb.Spec.Details.AutonomousDatabaseOCID == nil { + return adbActionProvision, nil, nil } else { - // The object has successfully synced with the remote database at least once. - // Update the Autonomous Database in OCI. - // Change to the lifecycle state has the highest priority. - - // Get the Autonomous Database OCID from the last successful spec if not presented. - // This happens when a database reference is already created in the cluster. User updates the target CR (specifying metadata.name) but doesn't provide database OCID. - if adb.Spec.Details.AutonomousDatabaseOCID == nil { - adb.Spec.Details.AutonomousDatabaseOCID = lastSucSpec.Details.AutonomousDatabaseOCID - } + return adbActionBind, nil, nil + } + } else { + // Pre-process step for the UPDATE. Remove the unchanged fields from spec.details, + difADB := adb.DeepCopy() + detailsChanged, err := difADB.RemoveUnchangedDetails() + if err != nil { + return "", nil, err + } - // Start/Stop/Terminate - setStateResp, err := oci.SetAutonomousDatabaseLifecycleState(currentLogger, dbClient, adb) - if err != nil { - currentLogger.Error(err, "Fail to set the Autonomous Database lifecycle state") + if detailsChanged { + return adbActionUpdate, difADB, nil + } - // Change the status to UNAVAILABLE - adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr - } - return ctrl.Result{}, nil - } + return adbActionSync, nil, nil + } +} + +func (r *AutonomousDatabaseReconciler) createADB(adb *dbv1alpha1.AutonomousDatabase) error { + resp, err := r.adbService.CreateAutonomousDatabase(adb) + if err != nil { + return err + } + + adb.UpdateFromOCIADB(resp.AutonomousDatabase) - if setStateResp != nil { - var lifecycleState database.AutonomousDatabaseLifecycleStateEnum - var opcWorkRequestID *string + return r.updateResourceAndWait(adb, *resp.OpcWorkRequestId) +} + +// helper function to update the resource, and then wait until the work completes after an update/start/stop/delete AutonomousDatabase request is sent +func (r *AutonomousDatabaseReconciler) updateResourceAndWait(adb *dbv1alpha1.AutonomousDatabase, opcWorkRequestId string) error { + // updateResource updates the lastSucSpec as well + if err := r.updateResource(adb); err != nil { + return err + } + + // Wait until the work is finished + if _, err := r.workService.Wait(opcWorkRequestId); err != nil { + return err + } + + return nil +} + +func (r *AutonomousDatabaseReconciler) updateGeneralFields(adb *dbv1alpha1.AutonomousDatabase, difADB *dbv1alpha1.AutonomousDatabase) error { + if difADB.Spec.Details.DisplayName == nil && + difADB.Spec.Details.DbName == nil && + difADB.Spec.Details.DbVersion == nil && + difADB.Spec.Details.FreeformTags == nil { + return nil + } + + resp, err := r.adbService.UpdateAutonomousDatabaseGeneralFields(difADB) + if err != nil { + return err + } + + // If the OpcWorkRequestId is nil (such as when the displayName is changed), + // no need to update the resource and wail until the work is done + if resp.OpcWorkRequestId == nil { + return nil + } + + return r.updateResourceAndWait(adb, *resp.OpcWorkRequestId) +} - if startResp, isStartResponse := setStateResp.(database.StartAutonomousDatabaseResponse); isStartResponse { - lifecycleState = startResp.AutonomousDatabase.LifecycleState - opcWorkRequestID = startResp.OpcWorkRequestId +func (r *AutonomousDatabaseReconciler) updateAdminPassword(adb *dbv1alpha1.AutonomousDatabase, difADB *dbv1alpha1.AutonomousDatabase) error { + if difADB.Spec.Details.AdminPassword.K8sSecretName == nil && + difADB.Spec.Details.AdminPassword.OCISecretOCID == nil { + return nil + } - } else if stopResp, isStopResponse := setStateResp.(database.StopAutonomousDatabaseResponse); isStopResponse { - lifecycleState = stopResp.AutonomousDatabase.LifecycleState - opcWorkRequestID = stopResp.OpcWorkRequestId + resp, err := r.adbService.UpdateAutonomousDatabaseAdminPassword(difADB) + if err != nil { + return err + } - } else if deleteResp, isDeleteResponse := setStateResp.(database.DeleteAutonomousDatabaseResponse); isDeleteResponse { - // Special case. Delete response doen't contain lifecycle State - lifecycleState = database.AutonomousDatabaseLifecycleStateTerminating - opcWorkRequestID = deleteResp.OpcWorkRequestId + adb.UpdateFromOCIADB(resp.AutonomousDatabase) - } else { - currentLogger.Error(err, "Unknown response type") + return r.updateResourceAndWait(adb, *resp.OpcWorkRequestId) +} - // Change the status to UNAVAILABLE - adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr +func (r *AutonomousDatabaseReconciler) updateDbWorkload(adb *dbv1alpha1.AutonomousDatabase, difADB *dbv1alpha1.AutonomousDatabase) error { + if difADB.Spec.Details.DbWorkload == "" { + return nil + } + + resp, err := r.adbService.UpdateAutonomousDatabaseDBWorkload(difADB) + if err != nil { + return err + } + + adb.UpdateFromOCIADB(resp.AutonomousDatabase) + + return r.updateResourceAndWait(adb, *resp.OpcWorkRequestId) +} + +func (r *AutonomousDatabaseReconciler) updateLicenseModel(adb *dbv1alpha1.AutonomousDatabase, difADB *dbv1alpha1.AutonomousDatabase) error { + if difADB.Spec.Details.LicenseModel == "" { + return nil + } + + resp, err := r.adbService.UpdateAutonomousDatabaseLicenseModel(difADB) + if err != nil { + return err + } + + adb.UpdateFromOCIADB(resp.AutonomousDatabase) + + return r.updateResourceAndWait(adb, *resp.OpcWorkRequestId) +} + +func (r *AutonomousDatabaseReconciler) updateScalingFields(adb *dbv1alpha1.AutonomousDatabase, difADB *dbv1alpha1.AutonomousDatabase) error { + if difADB.Spec.Details.DataStorageSizeInTBs == nil && + difADB.Spec.Details.CPUCoreCount == nil && + difADB.Spec.Details.IsAutoScalingEnabled == nil { + return nil + } + + resp, err := r.adbService.UpdateAutonomousDatabaseScalingFields(difADB) + if err != nil { + return err + } + + adb.UpdateFromOCIADB(resp.AutonomousDatabase) + + return r.updateResourceAndWait(adb, *resp.OpcWorkRequestId) +} + +func (r *AutonomousDatabaseReconciler) deleteAutonomousDatabase(adb *dbv1alpha1.AutonomousDatabase) error { + + resp, err := r.adbService.DeleteAutonomousDatabase(*adb.Spec.Details.AutonomousDatabaseOCID) + if err != nil { + return err + } + + adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateTerminating + opcWorkRequestId := *resp.OpcWorkRequestId + + if err := r.updateResourceStatus(adb); err != nil { + return err + } + + // Wait until the work is finished + if _, err := r.workService.Wait(opcWorkRequestId); err != nil { + return err + } + + return nil +} + +func (r *AutonomousDatabaseReconciler) updateLifecycleState(adb *dbv1alpha1.AutonomousDatabase, difADB *dbv1alpha1.AutonomousDatabase) error { + if difADB.Spec.Details.LifecycleState == "" { + return nil + } + + + var opcWorkRequestId string + + switch difADB.Spec.Details.LifecycleState { + case database.AutonomousDatabaseLifecycleStateAvailable: + resp, err := r.adbService.StartAutonomousDatabase(*difADB.Spec.Details.AutonomousDatabaseOCID) + if err != nil { + return err + } + + adb.Status.LifecycleState = resp.LifecycleState + opcWorkRequestId = *resp.OpcWorkRequestId + case database.AutonomousDatabaseLifecycleStateStopped: + resp, err := r.adbService.StopAutonomousDatabase(*difADB.Spec.Details.AutonomousDatabaseOCID) + if err != nil { + return err + } + + adb.Status.LifecycleState = resp.LifecycleState + opcWorkRequestId = *resp.OpcWorkRequestId + case database.AutonomousDatabaseLifecycleStateTerminated: + resp, err := r.adbService.DeleteAutonomousDatabase(*difADB.Spec.Details.AutonomousDatabaseOCID) + if err != nil { + return err + } + + adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateTerminating + opcWorkRequestId = *resp.OpcWorkRequestId + default: + return errors.New("Unknown state") + } + + if err := r.updateResourceStatus(adb); err != nil { + return err + } + + // Wait until the work is finished + if _, err := r.workService.Wait(opcWorkRequestId); err != nil { + return err + } + + return nil +} + +func (r *AutonomousDatabaseReconciler) setMTLSRequired(adb *dbv1alpha1.AutonomousDatabase) error { + resp, err := r.adbService.UpdateNetworkAccessMTLSRequired(*adb.Spec.Details.AutonomousDatabaseOCID) + if err != nil { + return err + } + + adb.UpdateFromOCIADB(resp.AutonomousDatabase) + + return r.updateResourceAndWait(adb, *resp.OpcWorkRequestId) +} + +func (r *AutonomousDatabaseReconciler) updateMTLS(adb *dbv1alpha1.AutonomousDatabase, difADB *dbv1alpha1.AutonomousDatabase) error { + if difADB.Spec.Details.NetworkAccess.IsMTLSConnectionRequired == nil { + return nil + } + + resp, err := r.adbService.UpdateNetworkAccessMTLS(difADB) + if err != nil { + return err + } + + adb.UpdateFromOCIADB(resp.AutonomousDatabase) + + return r.updateResourceAndWait(adb, *resp.OpcWorkRequestId) +} + +func (r *AutonomousDatabaseReconciler) setNetworkAccessPublic(adb *dbv1alpha1.AutonomousDatabase) error { + resp, err := r.adbService.UpdateNetworkAccessPublic(r.lastSucSpec, *adb.Spec.Details.AutonomousDatabaseOCID) + if err != nil { + return err + } + + adb.UpdateFromOCIADB(resp.AutonomousDatabase) + + return r.updateResourceAndWait(adb, *resp.OpcWorkRequestId) +} + +func (r *AutonomousDatabaseReconciler) updateNetworkAccess(adb *dbv1alpha1.AutonomousDatabase, difADB *dbv1alpha1.AutonomousDatabase) error { + if difADB.Spec.Details.NetworkAccess.AccessType == "" && + difADB.Spec.Details.NetworkAccess.IsAccessControlEnabled == nil && + difADB.Spec.Details.NetworkAccess.AccessControlList == nil && + difADB.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID == nil && + difADB.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs == nil && + difADB.Spec.Details.NetworkAccess.PrivateEndpoint.HostnamePrefix == nil { + return nil + } + resp, err := r.adbService.UpdateNetworkAccess(difADB) + if err != nil { + return err + } + + adb.UpdateFromOCIADB(resp.AutonomousDatabase) + + return r.updateResourceAndWait(adb, *resp.OpcWorkRequestId) +} + +// The logic of updating the network access configurations is as follows: +// 1. Shared databases: +// If the network access type changes +// a. to PUBLIC: +// was RESTRICTED: re-enable IsMTLSConnectionRequired if its not. Then set WhitelistedIps to an array with a single empty string entry. +// was PRIVATE: re-enable IsMTLSConnectionRequired if its not. Then set PrivateEndpointLabel to an emtpy string. +// b. to RESTRICTED: +// was PUBLIC: set WhitelistedIps to desired IPs/CIDR blocks/VCN OCID. Configure the IsMTLSConnectionRequired settings if it is set to disabled. +// was PRIVATE: re-enable IsMTLSConnectionRequired if its not. Set the type to PUBLIC first, and then configure the WhitelistedIps. Finally resume the IsMTLSConnectionRequired settings if it was, or is configured as disabled. +// c. to PRIVATE: +// was PUBLIC: set subnetOCID and nsgOCIDs. Configure the IsMTLSConnectionRequired settings if it is set. +// was RESTRICTED: set subnetOCID and nsgOCIDs. Configure the IsMTLSConnectionRequired settings if it is set. +// +// Otherwise, if the network access type remains the same, apply the network configuration, and then set the IsMTLSConnectionRequired. +// +// 2. Dedicated databases: +// Apply the configs directly +func (r *AutonomousDatabaseReconciler) determineNetworkAccessUpdate(adb *dbv1alpha1.AutonomousDatabase, difADB *dbv1alpha1.AutonomousDatabase) error { + if difADB.Spec.Details.NetworkAccess.AccessType == "" && + difADB.Spec.Details.NetworkAccess.IsAccessControlEnabled == nil && + difADB.Spec.Details.NetworkAccess.AccessControlList == nil && + difADB.Spec.Details.NetworkAccess.IsMTLSConnectionRequired == nil && + difADB.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID == nil && + difADB.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs == nil && + difADB.Spec.Details.NetworkAccess.PrivateEndpoint.HostnamePrefix == nil { + return nil + } + + if !*adb.Spec.Details.IsDedicated { + var lastAccessType = r.lastSucSpec.Details.NetworkAccess.AccessType + var difAccessType = difADB.Spec.Details.NetworkAccess.AccessType + + if difAccessType != "" { + switch difAccessType { + case dbv1alpha1.NetworkAccessTypePublic: + // OCI validation requires IsMTLSConnectionRequired to be enabled before changing the network access type to PUBLIC + if !*r.lastSucSpec.Details.NetworkAccess.IsMTLSConnectionRequired { + if err := r.setMTLSRequired(adb); err != nil { + return err } - return ctrl.Result{}, nil } - if err := adbutil.UpdateAutonomousDatabaseStatusAndWait(currentLogger, r.KubeClient, workClient, adb, lifecycleState, opcWorkRequestID); err != nil { - currentLogger.Error(err, "Fail to update the status of Autonomous Database. opcWorkRequestID = "+*opcWorkRequestID) + if err := r.setNetworkAccessPublic(adb); err != nil { + return err } + case dbv1alpha1.NetworkAccessTypeRestricted: + // If the access type was PRIVATE, then OCI validation requires IsMTLSConnectionRequired + // to be enabled before setting ACL. Also we can only change the network access type from + // PRIVATE to PUBLIC. + if lastAccessType == dbv1alpha1.NetworkAccessTypePrivate { + if !*r.lastSucSpec.Details.NetworkAccess.IsMTLSConnectionRequired { + var oldMTLS bool = *adb.Spec.Details.NetworkAccess.IsMTLSConnectionRequired + if err := r.setMTLSRequired(adb); err != nil { + return err + } + // restore IsMTLSConnectionRequired + adb.Spec.Details.NetworkAccess.IsMTLSConnectionRequired = &oldMTLS + } - currentLogger.Info(fmt.Sprintf("Change AutonomousDatabase %s lifecycle state to %s successfully\n", - *adb.Spec.Details.DbName, - adb.Spec.Details.LifecycleState)) - } + if err := r.setNetworkAccessPublic(adb); err != nil { + return err + } + } - // Update the database in OCI from the local resource. - // The local resource will be synchronized again later. - if err := adbutil.UpdateGeneralAndPasswordAttributesAndWait(currentLogger, r.KubeClient, dbClient, secretClient, workClient, adb); err != nil { - currentLogger.Error(err, "Fail to update Autonomous Database general and password attributes") - } + if err := r.updateNetworkAccess(adb, difADB); err != nil { + return err + } + + if err := r.updateMTLS(adb, difADB); err != nil { + return err + } + case dbv1alpha1.NetworkAccessTypePrivate: + if err := r.updateNetworkAccess(adb, difADB); err != nil { + return err + } - if err := adbutil.UpdateScaleAttributesAndWait(currentLogger, r.KubeClient, dbClient, workClient, adb); err != nil { - currentLogger.Error(err, "Fail to scale Autonomous Database") + if err := r.updateMTLS(adb, difADB); err != nil { + return err + } + } + } else { + // Access type doesn't change + if err := r.updateNetworkAccess(adb, difADB); err != nil { + return err } - if err := adbutil.UpdateNetworkAttributes(currentLogger, r.KubeClient, dbClient, workClient, adb); err != nil { - currentLogger.Error(err, "Fail to update Autonomous Database network access attributes") + if err := r.updateMTLS(adb, difADB); err != nil { + return err } } + } else { + // Dedicated database + if err := r.updateNetworkAccess(adb, difADB); err != nil { + return err + } } - // Get the information from OCI - updatedADB, err := oci.GetAutonomousDatabaseResource(currentLogger, dbClient, adb) - if err != nil { - currentLogger.Error(err, "Fail to get Autonomous Database") + return nil +} - // Change the status to UNAVAILABLE - adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr - } - return ctrl.Result{}, nil +func (r *AutonomousDatabaseReconciler) downloadWallet(adb *dbv1alpha1.AutonomousDatabase) error { + if adb.Spec.Details.Wallet.Name == nil && + adb.Spec.Details.Wallet.Password.K8sSecretName == nil && + adb.Spec.Details.Wallet.Password.OCISecretOCID == nil { + return nil } - adb = updatedADB + logger := r.Log.WithName("download-wallet") - // Update local object and the status - if err := adbutil.UpdateAutonomousDatabaseDetails(currentLogger, r.KubeClient, adb); err != nil { - // Change the status to UNAVAILABLE - adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr - } - return ctrl.Result{}, err + // lastSucSpec may be nil if this is the first time entering the reconciliation loop + var walletName string + + if adb.Spec.Details.Wallet.Name == nil { + walletName = adb.GetName() + "-instance-wallet" + } else { + walletName = *adb.Spec.Details.Wallet.Name } - /***************************************************** - * Instance Wallet - *****************************************************/ - passwordSecretUpdate := (lastSucSpec == nil && adb.Spec.Details.Wallet.Password.K8sSecretName != nil) || - (lastSucSpec != nil && lastSucSpec.Details.Wallet.Password.K8sSecretName != adb.Spec.Details.Wallet.Password.K8sSecretName) - passwordOCIDUpdate := (lastSucSpec == nil && adb.Spec.Details.Wallet.Password.OCISecretOCID != nil) || - (lastSucSpec != nil && lastSucSpec.Details.Wallet.Password.OCISecretOCID != adb.Spec.Details.Wallet.Password.OCISecretOCID) - - if (passwordSecretUpdate || passwordOCIDUpdate) && adb.Status.LifecycleState == database.AutonomousDatabaseLifecycleStateAvailable { - if err := adbutil.CreateWalletSecret(currentLogger, r.KubeClient, dbClient, secretClient, adb); err != nil { - currentLogger.Error(err, "Fail to download Instance Wallet") - // Change the status to UNAVAILABLE - adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr - } - return ctrl.Result{}, nil + secret, err := k8s.FetchSecret(r.KubeClient, adb.GetNamespace(), walletName) + if err == nil { + val, ok := secret.Labels["app"] + if !ok || val != adb.Name { + // Overwrite if the fetched secret has a different label + logger.Info("wallet existed but has a different label; skip the download") } + // No-op if Wallet is already downloaded + return nil + } else if !apiErrors.IsNotFound(err) { + return err } - /***************************************************** - * Sync AutonomousDatabase Backups - *****************************************************/ - if err := adbutil.SyncBackupResources(currentLogger, r.KubeClient, dbClient, adb); err != nil { - currentLogger.Error(err, "Fail to sync Autonomous Database backups") + resp, err := r.adbService.DownloadWallet(adb) + if err != nil { + return err + } + + data, err := oci.ExtractWallet(resp.Content) + if err != nil { + return err + } + + label := map[string]string{"app": adb.GetName()} + + if err := k8s.CreateSecret(r.KubeClient, adb.Namespace, walletName, data, adb, label); err != nil { + return err + } + + logger.Info(fmt.Sprintf("Wallet is stored in the Secret %s", walletName)) + + return nil +} + +func (r *AutonomousDatabaseReconciler) getValidBackupName(name string, usedNames map[string]bool) string { + returnedName := name + var i = 1 - // Change the status to UNAVAILABLE - adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUnavailable - if statusErr := adbutil.UpdateAutonomousDatabaseStatus(r.KubeClient, adb); statusErr != nil { - return ctrl.Result{}, statusErr + _, ok := usedNames[returnedName] + for ok { + returnedName = fmt.Sprintf("%s-%d", name, i) + _, ok = usedNames[returnedName] + i++ + } + + return returnedName +} + +func (r *AutonomousDatabaseReconciler) updateADB(adb *dbv1alpha1.AutonomousDatabase, difADB *dbv1alpha1.AutonomousDatabase) error { + if err := r.updateGeneralFields(adb, difADB); err != nil { + return err + } + + if err := r.updateAdminPassword(adb, difADB); err != nil { + return err + } + + if err := r.updateDbWorkload(adb, difADB); err != nil { + return err + } + + if err := r.updateLicenseModel(adb, difADB); err != nil { + return err + } + + if err := r.updateScalingFields(adb, difADB); err != nil { + return err + } + + if err := r.determineNetworkAccessUpdate(adb, difADB); err != nil { + return err + } + + if err := r.updateLifecycleState(adb, difADB); err != nil { + return err + } + + if err := r.downloadWallet(adb); err != nil { + return err + } + + return nil +} + +// updateBackupResources get the list of AutonomousDatabasBackups and +// create a backup object if it's not found in the same namespace +func (r *AutonomousDatabaseReconciler) syncBackupResources(adb *dbv1alpha1.AutonomousDatabase) error { + logger := r.Log.WithName("update-backups") + + // Get the list of AutonomousDatabaseBackupOCID in the same namespace + backupList, err := k8s.FetchAutonomousDatabaseBackups(r.KubeClient, adb.Namespace) + if err != nil { + return err + } + + curBackupNames := make(map[string]bool) + curBackupOCIDs := make(map[string]bool) + + for _, backup := range backupList.Items { + curBackupNames[backup.Name] = true + + if backup.Spec.AutonomousDatabaseBackupOCID != "" { + curBackupOCIDs[backup.Spec.AutonomousDatabaseBackupOCID] = true } - // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue - return ctrl.Result{}, nil } - /***************************************************** - * Update last succesful spec - *****************************************************/ - if err := adb.UpdateLastSuccessfulSpec(r.KubeClient); err != nil { - return ctrl.Result{}, err + resp, err := r.adbService.ListAutonomousDatabaseBackups(*adb.Spec.Details.AutonomousDatabaseOCID) + if err != nil { + return err } - currentLogger.Info("AutonomousDatabase reconciles successfully") + for _, backupSummary := range resp.Items { + // Create the resource if the AutonomousDatabaseBackupOCID doesn't exist + _, ok := curBackupOCIDs[*backupSummary.Id] + if !ok { + // Convert the string to lowercase, and replace spaces, commas, and colons with hyphens + resourceName := *backupSummary.DisplayName + resourceName = strings.ToLower(resourceName) + + re, err := regexp.Compile(`[^-a-zA-Z0-9]`) + if err != nil { + return err + } + resourceName = re.ReplaceAllString(resourceName, "-") + resourceName = r.getValidBackupName(resourceName, curBackupNames) + + if err := k8s.CreateAutonomousBackup(r.KubeClient, resourceName, backupSummary, adb); err != nil { + return err + } + + // Add the used names and ocids + curBackupNames[resourceName] = true + curBackupOCIDs[*backupSummary.AutonomousDatabaseId] = true + + logger.Info("Create AutonomousDatabaseBackup " + resourceName) + } + } - return ctrl.Result{}, nil + return nil } diff --git a/controllers/database/autonomousdatabasebackup_controller.go b/controllers/database/autonomousdatabasebackup_controller.go index ce977b67..90fc2100 100644 --- a/controllers/database/autonomousdatabasebackup_controller.go +++ b/controllers/database/autonomousdatabasebackup_controller.go @@ -40,193 +40,249 @@ package controllers import ( "context" - "reflect" + "fmt" "github.com/go-logr/logr" apiErrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/types" + "k8s.io/client-go/util/retry" ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/controller-runtime/pkg/controller" - "sigs.k8s.io/controller-runtime/pkg/event" "sigs.k8s.io/controller-runtime/pkg/predicate" "github.com/oracle/oci-go-sdk/v54/database" - "github.com/oracle/oci-go-sdk/v54/workrequests" databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" - backupUtil "github.com/oracle/oracle-database-operator/commons/autonomousdatabase" + "github.com/oracle/oracle-database-operator/commons/k8s" "github.com/oracle/oracle-database-operator/commons/oci" ) -// AutonomousDatabaseBackupReconciler reconciles a AutonomousDatabaseBackup object +// *AutonomousDatabaseBackupReconciler reconciles a AutonomousDatabaseBackup object type AutonomousDatabaseBackupReconciler struct { KubeClient client.Client Log logr.Logger Scheme *runtime.Scheme + + adbService oci.DatabaseService + workService oci.WorkRequestService } // SetupWithManager sets up the controller with the Manager. func (r *AutonomousDatabaseBackupReconciler) SetupWithManager(mgr ctrl.Manager) error { return ctrl.NewControllerManagedBy(mgr). For(&databasev1alpha1.AutonomousDatabaseBackup{}). - WithEventFilter(r.eventFilterPredicate()). + WithEventFilter(predicate.GenerationChangedPredicate{}). WithOptions(controller.Options{MaxConcurrentReconciles: 100}). // ReconcileHandler is never invoked concurrently with the same object. Complete(r) } -func (r *AutonomousDatabaseBackupReconciler) eventFilterPredicate() predicate.Predicate { - pred := predicate.Funcs{ - CreateFunc: func(e event.CreateEvent) bool { - return true - }, - UpdateFunc: func(e event.UpdateEvent) bool { - oldBackup := e.ObjectOld.DeepCopyObject().(*dbv1alpha1.AutonomousDatabaseBackup) - newBackup := e.ObjectNew.DeepCopyObject().(*dbv1alpha1.AutonomousDatabaseBackup) - - specChanged := !reflect.DeepEqual(oldBackup.Spec, newBackup.Spec) - if specChanged { - // Enqueue request - return true - } - // Don't enqueue request - return false - }, - DeleteFunc: func(e event.DeleteEvent) bool { - // Do not trigger reconciliation when the real object is deleted from the cluster. - return false - }, - } - - return pred -} - //+kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabasebackups,verbs=get;list;watch;create;delete //+kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabasebackups/status,verbs=get;update;patch func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { - currentLogger := r.Log.WithValues("Namespaced/Name", req.NamespacedName) + logger := r.Log.WithValues("Namespaced/Name", req.NamespacedName) - adbBackup := &dbv1alpha1.AutonomousDatabaseBackup{} - if err := r.KubeClient.Get(context.TODO(), req.NamespacedName, adbBackup); err != nil { + backup := &dbv1alpha1.AutonomousDatabaseBackup{} + if err := r.KubeClient.Get(context.TODO(), req.NamespacedName, backup); err != nil { // Ignore not-found errors, since they can't be fixed by an immediate requeue. // No need to change the since we don't know if we obtain the object. if apiErrors.IsNotFound(err) { - return ctrl.Result{}, nil + return emptyResult, nil } - return ctrl.Result{}, err + // Failed to get ADBBackup, so we don't need to update the status + return emptyResult, err } /****************************************************************** - * Get OCI database client and work request client - ******************************************************************/ - authData := oci.APIKeyAuth{ - ConfigMapName: adbBackup.Spec.OCIConfig.ConfigMapName, - SecretName: adbBackup.Spec.OCIConfig.SecretName, - Namespace: adbBackup.GetNamespace(), - } - provider, err := oci.GetOCIProvider(r.KubeClient, authData) - if err != nil { - currentLogger.Error(err, "Fail to get OCI provider") - - // Change the status to UNAVAILABLE - adbBackup.Status.LifecycleState = database.AutonomousDatabaseBackupLifecycleStateFailed - if statusErr := backupUtil.UpdateAutonomousDatabaseBackupStatus(r.KubeClient, adbBackup); statusErr != nil { - return ctrl.Result{}, statusErr + * Look up the owner AutonomousDatabase and set the ownerReference + * if the owner hasn't been set yet. + ******************************************************************/ + if len(backup.GetOwnerReferences()) == 0 && backup.Spec.AutonomousDatabaseOCID != "" { + adb, err := r.getOwnerAutonomousDatabase(backup.Namespace, backup.Spec.AutonomousDatabaseOCID) + if err != nil { + return r.manageError(backup, err) } - return ctrl.Result{}, nil - } - - dbClient, err := database.NewDatabaseClientWithConfigurationProvider(provider) - if err != nil { - currentLogger.Error(err, "Fail to get OCI database client") - - // Change the status to UNAVAILABLE - adbBackup.Status.LifecycleState = database.AutonomousDatabaseBackupLifecycleStateFailed - if statusErr := backupUtil.UpdateAutonomousDatabaseBackupStatus(r.KubeClient, adbBackup); statusErr != nil { - return ctrl.Result{}, statusErr + if adb != nil { + if err := r.setOwnerAutonomousDatabase(backup, adb); err != nil { + return r.manageError(backup, err) + } } - return ctrl.Result{}, nil } - workClient, err := workrequests.NewWorkRequestClientWithConfigurationProvider(provider) - if err != nil { - currentLogger.Error(err, "Fail to get OCI work request client") - - // Change the status to UNAVAILABLE - adbBackup.Status.LifecycleState = database.AutonomousDatabaseBackupLifecycleStateFailed - if statusErr := backupUtil.UpdateAutonomousDatabaseBackupStatus(r.KubeClient, adbBackup); statusErr != nil { - return ctrl.Result{}, statusErr - } - return ctrl.Result{}, nil + /****************************************************************** + * Get OCI database client and work request client + ******************************************************************/ + if err := r.setupOCIClients(backup); err != nil { + return r.manageError(backup, err) } + logger.Info("OCI clients configured succesfully") + /****************************************************************** * If the Spec.AutonomousDatabaseBackupOCID is empty and the LifecycleState is never assigned , create a backup. * LifecycleState is checked to avoid sending a duplicated backup request when the backup is creating. * Otherwise, bind to an exisiting backup if the Spec.AutonomousDatabaseBackupOCID isn't empty. ******************************************************************/ - if adbBackup.Spec.AutonomousDatabaseBackupOCID == "" && adbBackup.Status.LifecycleState == "" { + if backup.Spec.AutonomousDatabaseBackupOCID == "" && backup.Status.LifecycleState == "" { // Create a new backup - backupResp, err := oci.CreateAutonomousDatabaseBackup(currentLogger, dbClient, adbBackup) + backupResp, err := r.adbService.CreateAutonomousDatabaseBackup(backup) if err != nil { - currentLogger.Error(err, "Fail to create AutonomousDatabase Backup") - return ctrl.Result{}, nil + return r.manageError(backup, err) } - adbResp, err := oci.GetAutonomousDatabase(dbClient, backupResp.AutonomousDatabaseId) + adbResp, err := r.adbService.GetAutonomousDatabase(*backupResp.AutonomousDatabaseId) if err != nil { - currentLogger.Error(err, "Fail to get AutonomousDatabase. The AutonomousDatabase OCID = "+*backupResp.AutonomousDatabaseId) - return ctrl.Result{}, nil + return r.manageError(backup, err) } // update the status - adbBackup.UpdateStatusFromAutonomousDatabaseBackupResponse(backupResp.AutonomousDatabaseBackup, adbResp.AutonomousDatabase) - backupUtil.UpdateAutonomousDatabaseBackupStatus(r.KubeClient, adbBackup) + backup.UpdateStatusFromOCIBackup(backupResp.AutonomousDatabaseBackup, adbResp.AutonomousDatabase) + r.updateResourceStatus(backup) // Wait until the work is done - if _, err := oci.GetWorkStatusAndWait(currentLogger, workClient, backupResp.OpcWorkRequestId); err != nil { - currentLogger.Error(err, "Work request faied. opcWorkRequestID = "+*backupResp.OpcWorkRequestId) - return ctrl.Result{}, nil + if _, err := r.workService.Wait(*backupResp.OpcWorkRequestId); err != nil { + return r.manageError(backup, err) } - currentLogger.Info("AutonomousDatabaseBackup " + *backupResp.DisplayName + " created successfully") + logger.Info("AutonomousDatabaseBackup " + *backupResp.DisplayName + " created successfully") - } else if adbBackup.Spec.AutonomousDatabaseBackupOCID != "" { - // Bind to an existing backup - adbBackup.Status.AutonomousDatabaseBackupOCID = adbBackup.Spec.AutonomousDatabaseBackupOCID } /****************************************************************** * Update the status of the resource if the - * Status.AutonomousDatabaseOCID isn't empty. + * Spec.AutonomousDatabaseOCID isn't empty. ******************************************************************/ - if adbBackup.Status.AutonomousDatabaseBackupOCID != "" { - backupResp, err := oci.GetAutonomousDatabaseBackup(dbClient, adbBackup.Status.AutonomousDatabaseBackupOCID) + if backup.Spec.AutonomousDatabaseBackupOCID != "" { + backupResp, err := r.adbService.GetAutonomousDatabaseBackup(backup.Spec.AutonomousDatabaseBackupOCID) if err != nil { - currentLogger.Error(err, "Fail to get AutonomousDatabase Backup. The AutonomousDatabase Backup OCID = "+adbBackup.Status.AutonomousDatabaseBackupOCID) - return ctrl.Result{}, nil + return r.manageError(backup, err) } - adbResp, err := oci.GetAutonomousDatabase(dbClient, backupResp.AutonomousDatabaseId) + adbResp, err := r.adbService.GetAutonomousDatabase(*backupResp.AutonomousDatabaseId) if err != nil { - currentLogger.Error(err, "Fail to get AutonomousDatabase. The AutonomousDatabase OCID = "+*backupResp.AutonomousDatabaseId) - return ctrl.Result{}, nil + return r.manageError(backup, err) } - adbBackup.UpdateStatusFromAutonomousDatabaseBackupResponse(backupResp.AutonomousDatabaseBackup, adbResp.AutonomousDatabase) - backupUtil.UpdateAutonomousDatabaseBackupStatus(r.KubeClient, adbBackup) + backup.UpdateStatusFromOCIBackup(backupResp.AutonomousDatabaseBackup, adbResp.AutonomousDatabase) + if err := r.updateResourceStatus(backup); err != nil { + return r.manageError(backup, err) + } } - /****************************************************************** - * Look up the owner AutonomousDatabase and set the ownerReference - * if the owner hasn't been set yet. - ******************************************************************/ - if len(adbBackup.GetOwnerReferences()) == 0 && adbBackup.Status.AutonomousDatabaseOCID != "" { - if err := backupUtil.SetOwnerAutonomousDatabase(currentLogger, r.KubeClient, adbBackup); err != nil { - return ctrl.Result{}, err + return ctrl.Result{}, nil +} + +func (r *AutonomousDatabaseBackupReconciler) setupOCIClients(backup *dbv1alpha1.AutonomousDatabaseBackup) error { + var err error + + authData := oci.APIKeyAuth{ + ConfigMapName: backup.Spec.OCIConfig.ConfigMapName, + SecretName: backup.Spec.OCIConfig.SecretName, + Namespace: backup.GetNamespace(), + } + + provider, err := oci.GetOCIProvider(r.KubeClient, authData) + if err != nil { + return err + } + + r.adbService, err = oci.NewDatabaseService(r.Log, r.KubeClient, provider) + if err != nil { + return err + } + + r.workService, err = oci.NewWorkRequestService(r.Log, r.KubeClient, provider) + if err != nil { + return err + } + + return nil +} + +// updateResource updates the specification and the status of AutonomousDatabase resource without trigger a reconcile loop +func (r *AutonomousDatabaseBackupReconciler) updateResource(backup *dbv1alpha1.AutonomousDatabaseBackup) error { + // Update the spec + if err := retry.RetryOnConflict(retry.DefaultRetry, func() error { + curBackup := &dbv1alpha1.AutonomousDatabaseBackup{} + + namespacedName := types.NamespacedName{ + Namespace: backup.GetNamespace(), + Name: backup.GetName(), + } + + if err := r.KubeClient.Get(context.TODO(), namespacedName, curBackup); err != nil { + return err } + + curBackup.Spec = *backup.Spec.DeepCopy() + curBackup.ObjectMeta = *backup.ObjectMeta.DeepCopy() // ownerReference + return r.KubeClient.Update(context.TODO(), curBackup) + }); err != nil { + return err } - return ctrl.Result{}, nil + // Update the status + if err := r.updateResourceStatus(backup); err != nil { + return err + } + + return nil +} + +func (r *AutonomousDatabaseBackupReconciler) updateResourceStatus(backup *dbv1alpha1.AutonomousDatabaseBackup) error { + return retry.RetryOnConflict(retry.DefaultRetry, func() error { + curBackup := &dbv1alpha1.AutonomousDatabaseBackup{} + + namespacedName := types.NamespacedName{ + Namespace: backup.GetNamespace(), + Name: backup.GetName(), + } + + if err := r.KubeClient.Get(context.TODO(), namespacedName, curBackup); err != nil { + return err + } + + curBackup.Status = backup.Status + return r.KubeClient.Status().Update(context.TODO(), curBackup) + }) +} + +// Returns the first AutonomousDatabase resource that matches the AutonomousDatabaseOCID of the backup +// If the AutonomousDatabase doesn't exist, returns a nil +func (r *AutonomousDatabaseBackupReconciler) getOwnerAutonomousDatabase(namespace string, adbOCID string) (*dbv1alpha1.AutonomousDatabase, error) { + adbList, err := k8s.FetchAutonomousDatabases(r.KubeClient, namespace) + if err != nil { + return nil, err + } + + for _, adb := range adbList.Items { + if adb.Spec.Details.AutonomousDatabaseOCID != nil && *adb.Spec.Details.AutonomousDatabaseOCID == adbOCID { + return &adb, nil + } + } + + return nil, nil +} + +// setOwnerAutonomousDatabase sets the owner of the AutonomousDatabaseBackup if the AutonomousDatabase resource with the same database OCID is found +func (r *AutonomousDatabaseBackupReconciler) setOwnerAutonomousDatabase(backup *dbv1alpha1.AutonomousDatabaseBackup, adb *dbv1alpha1.AutonomousDatabase) error { + logger := r.Log.WithName("set-owner") + + backup.SetOwnerReferences(k8s.NewOwnerReference(adb)) + r.updateResource(backup) + logger.Info(fmt.Sprintf("Set the owner of %s to %s", backup.Name, adb.Name)) + + return nil +} + +func (r *AutonomousDatabaseBackupReconciler) manageError(backup *dbv1alpha1.AutonomousDatabaseBackup, issue error) (ctrl.Result, error) { + // Change the status to FAILED + backup.Status.LifecycleState = database.AutonomousDatabaseBackupLifecycleStateFailed + if statusErr := r.updateResourceStatus(backup); statusErr != nil { + return emptyResult, k8s.CombineErrors(issue, statusErr) + } + + return emptyResult, issue } diff --git a/controllers/database/autonomousdatabaserestore_controller.go b/controllers/database/autonomousdatabaserestore_controller.go index 0b318155..a6c2a550 100644 --- a/controllers/database/autonomousdatabaserestore_controller.go +++ b/controllers/database/autonomousdatabaserestore_controller.go @@ -40,18 +40,22 @@ package controllers import ( "context" + "errors" "github.com/go-logr/logr" apiErrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/types" + "k8s.io/client-go/util/retry" ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/predicate" - "github.com/oracle/oci-go-sdk/v54/database" + "github.com/oracle/oci-go-sdk/v54/common" "github.com/oracle/oci-go-sdk/v54/workrequests" databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" - restoreUtil "github.com/oracle/oracle-database-operator/commons/autonomousdatabase" + "github.com/oracle/oracle-database-operator/commons/k8s" "github.com/oracle/oracle-database-operator/commons/oci" ) @@ -60,6 +64,17 @@ type AutonomousDatabaseRestoreReconciler struct { KubeClient client.Client Log logr.Logger Scheme *runtime.Scheme + + adbService oci.DatabaseService + workService oci.WorkRequestService +} + +// SetupWithManager sets up the controller with the Manager. +func (r *AutonomousDatabaseRestoreReconciler) SetupWithManager(mgr ctrl.Manager) error { + return ctrl.NewControllerManagedBy(mgr). + For(&databasev1alpha1.AutonomousDatabaseRestore{}). + WithEventFilter(predicate.GenerationChangedPredicate{}). + Complete(r) } //+kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabaserestores,verbs=get;list;watch;create;delete @@ -75,89 +90,156 @@ type AutonomousDatabaseRestoreReconciler struct { // For more details, check Reconcile and its Result here: // - https://pkg.go.dev/sigs.k8s.io/controller-runtime@v0.6.4/pkg/reconcile func (r *AutonomousDatabaseRestoreReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { - currentLogger := r.Log.WithValues("Namespaced/Name", req.NamespacedName) + logger := r.Log.WithValues("Namespaced/Name", req.NamespacedName) restore := &dbv1alpha1.AutonomousDatabaseRestore{} if err := r.KubeClient.Get(context.TODO(), req.NamespacedName, restore); err != nil { // Ignore not-found errors, since they can't be fixed by an immediate requeue. // No need to change since we don't know if we obtain the object. if apiErrors.IsNotFound(err) { - return ctrl.Result{}, nil + return emptyResult, nil } - return ctrl.Result{}, err + // Failed to get ADBRestore, so we don't need to update the status + return emptyResult, err + } + + /****************************************************************** + * Get OCI database client and work request client + ******************************************************************/ + if err := r.setupOCIClients(restore); err != nil { + return r.manageError(restore, err) } + logger.Info("OCI clients configured succesfully") + /****************************************************************** - * Get OCI database client and work request client + * Start restore ******************************************************************/ - authData := oci.APIKeyAuth{ - ConfigMapName: restore.Spec.OCIConfig.ConfigMapName, - SecretName: restore.Spec.OCIConfig.SecretName, - Namespace: restore.GetNamespace(), + if err := r.restoreAutonomousDatabase(restore); err != nil { + return r.manageError(restore, err) } - provider, err := oci.GetOCIProvider(r.KubeClient, authData) - if err != nil { - currentLogger.Error(err, "Fail to get OCI provider") - // Change the status to UNAVAILABLE - restore.Status.LifecycleState = workrequests.WorkRequestStatusFailed - if statusErr := restoreUtil.UpdateAutonomousDatabaseRestoreStatus(r.KubeClient, restore); statusErr != nil { - return ctrl.Result{}, statusErr + logger.Info("AutonomousDatabaseRestore reconciles successfully") + + return ctrl.Result{}, nil +} + +func (r *AutonomousDatabaseRestoreReconciler) restoreAutonomousDatabase(restore *dbv1alpha1.AutonomousDatabaseRestore) error { + var restoreTime *common.SDKTime + var adbOCID string + var err error + + if restore.Spec.BackupName != "" { // restore using backupName + backup := &dbv1alpha1.AutonomousDatabaseBackup{} + namespacedName := types.NamespacedName{Namespace: restore.Namespace, Name: restore.Spec.BackupName} + if err := r.KubeClient.Get(context.TODO(), namespacedName, backup); err != nil { + return err + } + + if backup.Status.TimeEnded == "" { + return errors.New("broken backup: ended time is missing in the AutonomousDatabaseBackup " + backup.GetName()) + } + restoreTime, err = backup.GetTimeEnded() + if err != nil { + return err } - return ctrl.Result{}, nil + + adbOCID = backup.Spec.AutonomousDatabaseOCID + + } else if restore.Spec.PointInTime.TimeStamp != "" { // PIT restore + // The validation of the pitr.timestamp has been handled by the webhook, so the error return is ignored + restoreTime, _ = restore.GetPIT() + adbOCID = restore.Spec.PointInTime.AutonomousDatabaseOCID } - dbClient, err := database.NewDatabaseClientWithConfigurationProvider(provider) + resp, err := r.adbService.RestoreAutonomousDatabase(adbOCID, *restoreTime) if err != nil { - currentLogger.Error(err, "Fail to get OCI database client") - - // Change the status to UNAVAILABLE - restore.Status.LifecycleState = workrequests.WorkRequestStatusFailed - if statusErr := restoreUtil.UpdateAutonomousDatabaseRestoreStatus(r.KubeClient, restore); statusErr != nil { - return ctrl.Result{}, statusErr - } - return ctrl.Result{}, nil + return err } - workClient, err := workrequests.NewWorkRequestClientWithConfigurationProvider(provider) + // Update status and wait for the work finish if a request is sent. Note that some of the requests (e.g. update displayName) won't return a work request ID. + // It's important to update the status by reference otherwise Reconcile() won't be able to get the latest values + restore.Status.DisplayName = *resp.AutonomousDatabase.DisplayName + restore.Status.DbName = *resp.AutonomousDatabase.DbName + restore.Status.AutonomousDatabaseOCID = *resp.AutonomousDatabase.Id + restore.Status.Status = restore.ConvertWorkRequestStatus(workrequests.WorkRequestStatusEnum(resp.LifecycleState)) + + r.updateResourceStatus(restore) + + workStatus, err := r.workService.Wait(*resp.OpcWorkRequestId) if err != nil { - currentLogger.Error(err, "Fail to get OCI work request client") + return err + } + + // Update status when the work is finished + restore.Status.Status = restore.ConvertWorkRequestStatus(workStatus) + r.updateResourceStatus(restore) - // Change the status to UNAVAILABLE - restore.Status.LifecycleState = workrequests.WorkRequestStatusFailed - if statusErr := restoreUtil.UpdateAutonomousDatabaseRestoreStatus(r.KubeClient, restore); statusErr != nil { - return ctrl.Result{}, statusErr + return nil +} + +func (r *AutonomousDatabaseRestoreReconciler) updateResourceStatus(restore *dbv1alpha1.AutonomousDatabaseRestore) error { + return retry.RetryOnConflict(retry.DefaultRetry, func() error { + curBackup := &dbv1alpha1.AutonomousDatabaseRestore{} + + namespacedName := types.NamespacedName{ + Namespace: restore.GetNamespace(), + Name: restore.GetName(), } - return ctrl.Result{}, nil - } - /****************************************************************** - * Restore - ******************************************************************/ - if restore.Status.LifecycleState == "" { - lifecycleState, err := restoreUtil.RestoreAutonomousDatabaseAndWait(currentLogger, r.KubeClient, dbClient, workClient, restore) - if err != nil { - currentLogger.Error(err, "Fail to restore database") - return ctrl.Result{}, nil + if err := r.KubeClient.Get(context.TODO(), namespacedName, curBackup); err != nil { + return err } - restore.Status.LifecycleState = lifecycleState + curBackup.Status = restore.Status + return r.KubeClient.Status().Update(context.TODO(), curBackup) + }) +} - if statusErr := restoreUtil.UpdateAutonomousDatabaseRestoreStatus(r.KubeClient, restore); statusErr != nil { - // No need to return the error since it will re-execute the Reconcile() - currentLogger.Error(err, "Fail to update the status of AutonomousDatabaseRestore") - return ctrl.Result{}, nil - } +func (r *AutonomousDatabaseRestoreReconciler) setupOCIClients(restore *dbv1alpha1.AutonomousDatabaseRestore) error { + var err error + + authData := oci.APIKeyAuth{ + ConfigMapName: restore.Spec.OCIConfig.ConfigMapName, + SecretName: restore.Spec.OCIConfig.SecretName, + Namespace: restore.GetNamespace(), } - currentLogger.Info("AutonomousDatabaseRestore reconciles successfully") + provider, err := oci.GetOCIProvider(r.KubeClient, authData) + if err != nil { + return err + } - return ctrl.Result{}, nil + r.adbService, err = oci.NewDatabaseService(r.Log, r.KubeClient, provider) + if err != nil { + return err + } + + r.workService, err = oci.NewWorkRequestService(r.Log, r.KubeClient, provider) + if err != nil { + return err + } + + return nil } -// SetupWithManager sets up the controller with the Manager. -func (r *AutonomousDatabaseRestoreReconciler) SetupWithManager(mgr ctrl.Manager) error { - return ctrl.NewControllerManagedBy(mgr). - For(&databasev1alpha1.AutonomousDatabaseRestore{}). - Complete(r) +// manageError doesn't return the error so that the request won't be requeued +func (r *AutonomousDatabaseRestoreReconciler) manageError(restore *dbv1alpha1.AutonomousDatabaseRestore, issue error) (ctrl.Result, error) { + nsn := types.NamespacedName{ + Namespace: restore.Namespace, + Name: restore.Name, + } + logger := r.Log.WithValues("Namespaced/Name", nsn) + + // Change the status to FAILED + var combinedErr error = issue + + restore.Status.Status = dbv1alpha1.RestoreStatusFailed + if statusErr := r.updateResourceStatus(restore); statusErr != nil { + combinedErr = k8s.CombineErrors(issue, statusErr) + } + + logger.Error(combinedErr, "Fail to restore Autonomous Database") + + return emptyResult, nil } diff --git a/main.go b/main.go index 01dd4ac9..2d6fde29 100644 --- a/main.go +++ b/main.go @@ -94,6 +94,7 @@ func main() { KubeClient: mgr.GetClient(), Log: ctrl.Log.WithName("controllers").WithName("database").WithName("AutonomousDatabase"), Scheme: mgr.GetScheme(), + Recorder: mgr.GetEventRecorderFor("AutonomousDatabase"), }).SetupWithManager(mgr); err != nil { setupLog.Error(err, "unable to create controller", "controller", "AutonomousDatabase") os.Exit(1) diff --git a/test/e2e/autonomousdatabase_controller_create_test.go b/test/e2e/autonomousdatabase_controller_create_test.go index afbb4912..17714d89 100644 --- a/test/e2e/autonomousdatabase_controller_create_test.go +++ b/test/e2e/autonomousdatabase_controller_create_test.go @@ -43,7 +43,6 @@ import ( "time" "github.com/oracle/oci-go-sdk/v54/common" - "github.com/oracle/oci-go-sdk/v54/database" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/types" @@ -149,7 +148,7 @@ var _ = Describe("test ADB provisioning", func() { Expect(k8sClient.Create(context.TODO(), duplicateAdb)).To(Succeed()) }) - It("Should check for local resource state UNAVAILABLE", e2ebehavior.AssertLocalState(&k8sClient, &dupAdbLookupKey, database.AutonomousDatabaseLifecycleStateUnavailable)) + It("Should check for local resource state \"\"", e2ebehavior.AssertLocalState(&k8sClient, &dupAdbLookupKey, "")) It("Should download an instance wallet using the password from K8s Secret "+SharedWalletPassSecretName, e2ebehavior.AssertWallet(&k8sClient, &adbLookupKey)) diff --git a/test/e2e/behavior/shared_behaviors.go b/test/e2e/behavior/shared_behaviors.go index f0f5f2a2..76a70522 100644 --- a/test/e2e/behavior/shared_behaviors.go +++ b/test/e2e/behavior/shared_behaviors.go @@ -496,14 +496,14 @@ func AssertHardLinkDelete(k8sClient *client.Client, dbClient *database.DatabaseC Expect(derefK8sClient.Get(context.TODO(), *adbLookupKey, adb)).To(Succeed()) Expect(derefK8sClient.Delete(context.TODO(), adb)).To(Succeed()) - AssertSoftLinkDelete(k8sClient, adbLookupKey)() - By("Checking if the ADB in OCI is in TERMINATING state") // Check every 10 secs for total 60 secs Eventually(func() (database.AutonomousDatabaseLifecycleStateEnum, error) { retryPolicy := e2eutil.NewLifecycleStateRetryPolicy(database.AutonomousDatabaseLifecycleStateTerminating) return returnRemoteState(derefK8sClient, derefDBClient, adb.Spec.Details.AutonomousDatabaseOCID, &retryPolicy) }, changeStateTimeout).Should(Equal(database.AutonomousDatabaseLifecycleStateTerminating)) + + AssertSoftLinkDelete(k8sClient, adbLookupKey)() } } diff --git a/test/e2e/suite_test.go b/test/e2e/suite_test.go index baddfc2e..103c8612 100644 --- a/test/e2e/suite_test.go +++ b/test/e2e/suite_test.go @@ -158,6 +158,7 @@ var _ = BeforeSuite(func(done ginkgo.Done) { KubeClient: k8sManager.GetClient(), Log: ctrl.Log.WithName("controllers").WithName("AutonomousDatabase_test"), Scheme: k8sManager.GetScheme(), + Recorder: k8sManager.GetEventRecorderFor("AutonomousDatabase_test"), }).SetupWithManager(k8sManager) Expect(err).ToNot(HaveOccurred()) From dcdec955f126d0d926cb533b6f541d97b4b881cc Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Thu, 10 Mar 2022 09:02:44 -0500 Subject: [PATCH 182/628] update ADB status when restore/backup changes --- .../autonomousdatabasebackup_types.go | 22 +++-- .../autonomousdatabaserestore_types.go | 3 +- commons/k8s/create.go | 4 +- commons/k8s/fetch.go | 19 +++- commons/k8s/utils.go | 25 ++++- ....oracle.com_autonomousdatabasebackups.yaml | 6 +- .../database/autonomousdatabase_controller.go | 95 +++++++++--------- .../autonomousdatabasebackup_controller.go | 92 ++++++++++++----- .../autonomousdatabaserestore_controller.go | 99 +++++++++++++++++-- 9 files changed, 264 insertions(+), 101 deletions(-) diff --git a/apis/database/v1alpha1/autonomousdatabasebackup_types.go b/apis/database/v1alpha1/autonomousdatabasebackup_types.go index c3b8c8d0..8409f4ce 100644 --- a/apis/database/v1alpha1/autonomousdatabasebackup_types.go +++ b/apis/database/v1alpha1/autonomousdatabasebackup_types.go @@ -63,14 +63,14 @@ type AutonomousDatabaseBackupSpec struct { type AutonomousDatabaseBackupStatus struct { // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster // Important: Run "make" to regenerate code after modifying this file - LifecycleState database.AutonomousDatabaseBackupLifecycleStateEnum `json:"lifecycleState"` - Type database.AutonomousDatabaseBackupTypeEnum `json:"type"` - IsAutomatic bool `json:"isAutomatic"` - TimeStarted string `json:"timeStarted,omitempty"` - TimeEnded string `json:"timeEnded,omitempty"` - CompartmentOCID string `json:"compartmentOCID"` - DBName string `json:"dbName"` - DBDisplayName string `json:"dbDisplayName"` + LifecycleState database.AutonomousDatabaseBackupLifecycleStateEnum `json:"lifecycleState"` + Type database.AutonomousDatabaseBackupTypeEnum `json:"type"` + IsAutomatic bool `json:"isAutomatic"` + TimeStarted string `json:"timeStarted,omitempty"` + TimeEnded string `json:"timeEnded,omitempty"` + CompartmentOCID string `json:"compartmentOCID"` + DBName string `json:"dbName"` + DBDisplayName string `json:"dbDisplayName"` } //+kubebuilder:object:root=true @@ -104,7 +104,11 @@ func init() { SchemeBuilder.Register(&AutonomousDatabaseBackup{}, &AutonomousDatabaseBackupList{}) } -func (backup *AutonomousDatabaseBackup) UpdateStatusFromOCIBackup(ociBackup database.AutonomousDatabaseBackup, ociADB database.AutonomousDatabase) { +func (backup *AutonomousDatabaseBackup) UpdateFromOCIBackup(ociBackup database.AutonomousDatabaseBackup, ociADB database.AutonomousDatabase) { + backup.Spec.AutonomousDatabaseBackupOCID = *ociBackup.Id + backup.Spec.AutonomousDatabaseOCID = *ociBackup.AutonomousDatabaseId + backup.Spec.DisplayName = *ociBackup.DisplayName + backup.Status.CompartmentOCID = *ociBackup.CompartmentId backup.Status.Type = ociBackup.Type backup.Status.IsAutomatic = *ociBackup.IsAutomatic diff --git a/apis/database/v1alpha1/autonomousdatabaserestore_types.go b/apis/database/v1alpha1/autonomousdatabaserestore_types.go index 2fb17a8e..16cc09d2 100644 --- a/apis/database/v1alpha1/autonomousdatabaserestore_types.go +++ b/apis/database/v1alpha1/autonomousdatabaserestore_types.go @@ -117,6 +117,7 @@ func (r *AutonomousDatabaseRestore) GetPIT() (*common.SDKTime, error) { func (r *AutonomousDatabaseRestore) ConvertWorkRequestStatus(s workrequests.WorkRequestStatusEnum) restoreStatusEnum { switch s { case workrequests.WorkRequestStatusAccepted: + fallthrough case workrequests.WorkRequestStatusInProgress: return RestoreStatusInProgress @@ -127,5 +128,5 @@ func (r *AutonomousDatabaseRestore) ConvertWorkRequestStatus(s workrequests.Work return RestoreStatusFailed } - return RestoreStatusFailed + return "UNKNOWN" } diff --git a/commons/k8s/create.go b/commons/k8s/create.go index ca158f0d..9dfdd24d 100644 --- a/commons/k8s/create.go +++ b/commons/k8s/create.go @@ -86,8 +86,8 @@ func CreateAutonomousBackup(kubeClient client.Client, OwnerReferences: NewOwnerReference(ownerADB), }, Spec: dbv1alpha1.AutonomousDatabaseBackupSpec{ - AutonomousDatabaseOCID: *backupSummary.AutonomousDatabaseId, - DisplayName: *backupSummary.DisplayName, + AutonomousDatabaseOCID: *backupSummary.AutonomousDatabaseId, + DisplayName: *backupSummary.DisplayName, AutonomousDatabaseBackupOCID: *backupSummary.Id, OCIConfig: dbv1alpha1.OCIConfigSpec{ ConfigMapName: ownerADB.Spec.OCIConfig.ConfigMapName, diff --git a/commons/k8s/fetch.go b/commons/k8s/fetch.go index 21cd1d65..c9afb1e0 100644 --- a/commons/k8s/fetch.go +++ b/commons/k8s/fetch.go @@ -50,7 +50,24 @@ import ( dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" ) -func FetchAutonomousDatabases(kubeClient client.Client, namespace string) (*dbv1alpha1.AutonomousDatabaseList, error) { +// Returns the first AutonomousDatabase resource that matches the AutonomousDatabaseOCID of the backup +// If the AutonomousDatabase doesn't exist, returns a nil +func FetchAutonomousDatabase(kubeClient client.Client, namespace string, ocid string) (*dbv1alpha1.AutonomousDatabase, error) { + adbList, err := fetchAutonomousDatabases(kubeClient, namespace) + if err != nil { + return nil, err + } + + for _, adb := range adbList.Items { + if adb.Spec.Details.AutonomousDatabaseOCID != nil && *adb.Spec.Details.AutonomousDatabaseOCID == ocid { + return &adb, nil + } + } + + return nil, nil +} + +func fetchAutonomousDatabases(kubeClient client.Client, namespace string) (*dbv1alpha1.AutonomousDatabaseList, error) { // Get the list of AutonomousDatabaseBackupOCID in the same namespace adbList := &dbv1alpha1.AutonomousDatabaseList{} diff --git a/commons/k8s/utils.go b/commons/k8s/utils.go index 0a9298ef..0e5a7aa6 100644 --- a/commons/k8s/utils.go +++ b/commons/k8s/utils.go @@ -39,8 +39,13 @@ package k8s import ( + "context" + + dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/types" utilErrors "k8s.io/apimachinery/pkg/util/errors" + "k8s.io/client-go/util/retry" "sigs.k8s.io/controller-runtime/pkg/client" ) @@ -58,4 +63,22 @@ func NewOwnerReference(owner client.Object) []metav1.OwnerReference { func CombineErrors(errs ...error) error { return utilErrors.NewAggregate(errs) -} \ No newline at end of file +} + +func UpdateADBStatus(kubeClient client.Client, adb *dbv1alpha1.AutonomousDatabase) error { + return retry.RetryOnConflict(retry.DefaultRetry, func() error { + curADB := &dbv1alpha1.AutonomousDatabase{} + + namespacedName := types.NamespacedName{ + Namespace: adb.GetNamespace(), + Name: adb.GetName(), + } + + if err := kubeClient.Get(context.TODO(), namespacedName, curADB); err != nil { + return err + } + + curADB.Status = adb.Status + return kubeClient.Status().Update(context.TODO(), curADB) + }) +} diff --git a/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml b/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml index ccaf8317..998ed12f 100644 --- a/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml +++ b/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml @@ -72,13 +72,14 @@ spec: secretName: type: string type: object + required: + - autonomousDatabaseOCID + - displayName type: object status: description: AutonomousDatabaseBackupStatus defines the observed state of AutonomousDatabaseBackup properties: - autonomousDatabaseOCID: - type: string compartmentOCID: type: string dbDisplayName: @@ -101,7 +102,6 @@ spec: type: string' type: string required: - - autonomousDatabaseOCID - compartmentOCID - dbDisplayName - dbName diff --git a/controllers/database/autonomousdatabase_controller.go b/controllers/database/autonomousdatabase_controller.go index 843d57f7..6716e2a2 100644 --- a/controllers/database/autonomousdatabase_controller.go +++ b/controllers/database/autonomousdatabase_controller.go @@ -45,7 +45,6 @@ import ( "fmt" "regexp" "strings" - "time" "github.com/go-logr/logr" "github.com/oracle/oci-go-sdk/v54/database" @@ -80,8 +79,6 @@ type AutonomousDatabaseReconciler struct { lastSucSpec *dbv1alpha1.AutonomousDatabaseSpec } -// To requeue after 60 secs allowing graceful state changes -var requeueResult ctrl.Result = ctrl.Result{Requeue: true, RequeueAfter: 60 * time.Second} var emptyResult ctrl.Result = ctrl.Result{} // SetupWithManager function @@ -97,16 +94,32 @@ func (r *AutonomousDatabaseReconciler) SetupWithManager(mgr ctrl.Manager) error func (r *AutonomousDatabaseReconciler) eventFilterPredicate() predicate.Predicate { pred := predicate.Funcs{ UpdateFunc: func(e event.UpdateEvent) bool { - newADB := e.ObjectNew.(*dbv1alpha1.AutonomousDatabase) - - // Don't enqueue request - if newADB.Status.LifecycleState == database.AutonomousDatabaseLifecycleStateBackupInProgress || - newADB.Status.LifecycleState == database.AutonomousDatabaseLifecycleStateUpdating || - newADB.Status.LifecycleState == database.AutonomousDatabaseLifecycleStateStarting || - newADB.Status.LifecycleState == database.AutonomousDatabaseLifecycleStateStopping || - newADB.Status.LifecycleState == database.AutonomousDatabaseLifecycleStateTerminating { + oldStatus := e.ObjectOld.(*dbv1alpha1.AutonomousDatabase).Status.LifecycleState + desiredStatus := e.ObjectNew.(*dbv1alpha1.AutonomousDatabase).Spec.Details.LifecycleState + + if oldStatus == database.AutonomousDatabaseLifecycleStateProvisioning || + oldStatus == database.AutonomousDatabaseLifecycleStateUpdating || + oldStatus == database.AutonomousDatabaseLifecycleStateStarting || + oldStatus == database.AutonomousDatabaseLifecycleStateStopping || + oldStatus == database.AutonomousDatabaseLifecycleStateTerminating || + oldStatus == database.AutonomousDatabaseLifecycleStateRestoreInProgress || + oldStatus == database.AutonomousDatabaseLifecycleStateBackupInProgress || + oldStatus == database.AutonomousDatabaseLifecycleStateMaintenanceInProgress || + oldStatus == database.AutonomousDatabaseLifecycleStateRestarting || + oldStatus == database.AutonomousDatabaseLifecycleStateRecreating || + oldStatus == database.AutonomousDatabaseLifecycleStateRoleChangeInProgress || + oldStatus == database.AutonomousDatabaseLifecycleStateUpgrading { + + // Except for the case that the ADB is already terminating, we should let the terminate requests to be enqueued + if oldStatus != database.AutonomousDatabaseLifecycleStateTerminating && + desiredStatus == database.AutonomousDatabaseLifecycleStateTerminated { + return true + } + + // All the requests other than the terminate request, should be discarded during the intermediate states return false } + return true }, DeleteFunc: func(e event.DeleteEvent) bool { @@ -236,7 +249,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R logger.Info("AutonomousDatabase reconciles successfully") - return requeueResult, nil + return emptyResult, nil } func (r *AutonomousDatabaseReconciler) setupOCIClients(adb *dbv1alpha1.AutonomousDatabase) error { @@ -343,6 +356,21 @@ func (r *AutonomousDatabaseReconciler) updateLastSuccessfulSpec(adb *dbv1alpha1. return annotations.SetAnnotations(r.KubeClient, adb, anns) } +// helper function to update the resource, and then wait until the work completes after an update/start/stop/delete AutonomousDatabase request is sent +func (r *AutonomousDatabaseReconciler) updateResourceAndWait(adb *dbv1alpha1.AutonomousDatabase, opcWorkRequestId string) error { + // updateResource updates the lastSucSpec as well + if err := r.updateResource(adb); err != nil { + return err + } + + // Wait until the work is finished + if _, err := r.workService.Wait(opcWorkRequestId); err != nil { + return err + } + + return nil +} + // updateResource updates the specification, the status of AutonomousDatabase resource, and the lastSucSpec func (r *AutonomousDatabaseReconciler) updateResource(adb *dbv1alpha1.AutonomousDatabase) error { // Update the status first to prevent unwanted Reconcile() @@ -376,22 +404,12 @@ func (r *AutonomousDatabaseReconciler) updateResource(adb *dbv1alpha1.Autonomous return nil } +// updateResourceStatus updates only the status of the resource, not including the lastSucSpec. +// This function should not be called by the functions associated with the OCI update requests. +// The OCI update requests should use updateResource() to ensure all the spec, resource and the +// lastSucSpec are updated. func (r *AutonomousDatabaseReconciler) updateResourceStatus(adb *dbv1alpha1.AutonomousDatabase) error { - return retry.RetryOnConflict(retry.DefaultRetry, func() error { - curADB := &dbv1alpha1.AutonomousDatabase{} - - namespacedName := types.NamespacedName{ - Namespace: adb.GetNamespace(), - Name: adb.GetName(), - } - - if err := r.KubeClient.Get(context.TODO(), namespacedName, curADB); err != nil { - return err - } - - curADB.Status = adb.Status - return r.KubeClient.Status().Update(context.TODO(), curADB) - }) + return k8s.UpdateADBStatus(r.KubeClient, adb) } func (r *AutonomousDatabaseReconciler) syncResource(adb *dbv1alpha1.AutonomousDatabase) error { @@ -453,21 +471,6 @@ func (r *AutonomousDatabaseReconciler) createADB(adb *dbv1alpha1.AutonomousDatab return r.updateResourceAndWait(adb, *resp.OpcWorkRequestId) } -// helper function to update the resource, and then wait until the work completes after an update/start/stop/delete AutonomousDatabase request is sent -func (r *AutonomousDatabaseReconciler) updateResourceAndWait(adb *dbv1alpha1.AutonomousDatabase, opcWorkRequestId string) error { - // updateResource updates the lastSucSpec as well - if err := r.updateResource(adb); err != nil { - return err - } - - // Wait until the work is finished - if _, err := r.workService.Wait(opcWorkRequestId); err != nil { - return err - } - - return nil -} - func (r *AutonomousDatabaseReconciler) updateGeneralFields(adb *dbv1alpha1.AutonomousDatabase, difADB *dbv1alpha1.AutonomousDatabase) error { if difADB.Spec.Details.DisplayName == nil && difADB.Spec.Details.DbName == nil && @@ -580,7 +583,6 @@ func (r *AutonomousDatabaseReconciler) updateLifecycleState(adb *dbv1alpha1.Auto return nil } - var opcWorkRequestId string switch difADB.Spec.Details.LifecycleState { @@ -612,12 +614,7 @@ func (r *AutonomousDatabaseReconciler) updateLifecycleState(adb *dbv1alpha1.Auto return errors.New("Unknown state") } - if err := r.updateResourceStatus(adb); err != nil { - return err - } - - // Wait until the work is finished - if _, err := r.workService.Wait(opcWorkRequestId); err != nil { + if err := r.updateResourceAndWait(adb, opcWorkRequestId); err != nil { return err } diff --git a/controllers/database/autonomousdatabasebackup_controller.go b/controllers/database/autonomousdatabasebackup_controller.go index 90fc2100..2ffdc9be 100644 --- a/controllers/database/autonomousdatabasebackup_controller.go +++ b/controllers/database/autonomousdatabasebackup_controller.go @@ -50,6 +50,7 @@ import ( ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/controller-runtime/pkg/controller" + "sigs.k8s.io/controller-runtime/pkg/event" "sigs.k8s.io/controller-runtime/pkg/predicate" "github.com/oracle/oci-go-sdk/v54/database" @@ -67,17 +68,36 @@ type AutonomousDatabaseBackupReconciler struct { adbService oci.DatabaseService workService oci.WorkRequestService + ownerADB *dbv1alpha1.AutonomousDatabase } // SetupWithManager sets up the controller with the Manager. func (r *AutonomousDatabaseBackupReconciler) SetupWithManager(mgr ctrl.Manager) error { return ctrl.NewControllerManagedBy(mgr). For(&databasev1alpha1.AutonomousDatabaseBackup{}). - WithEventFilter(predicate.GenerationChangedPredicate{}). + WithEventFilter(predicate.And(predicate.GenerationChangedPredicate{}, r.eventFilterPredicate())). WithOptions(controller.Options{MaxConcurrentReconciles: 100}). // ReconcileHandler is never invoked concurrently with the same object. Complete(r) } +func (r *AutonomousDatabaseBackupReconciler) eventFilterPredicate() predicate.Predicate { + pred := predicate.Funcs{ + UpdateFunc: func(e event.UpdateEvent) bool { + oldStatus := e.ObjectOld.(*dbv1alpha1.AutonomousDatabaseBackup).Status.LifecycleState + + if oldStatus == database.AutonomousDatabaseBackupLifecycleStateCreating || + oldStatus == database.AutonomousDatabaseBackupLifecycleStateDeleting { + // All the requests other than the terminate request, should be discarded during the intermediate states + return false + } + + return true + }, + } + + return pred +} + //+kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabasebackups,verbs=get;list;watch;create;delete //+kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabasebackups/status,verbs=get;update;patch @@ -99,16 +119,8 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req * Look up the owner AutonomousDatabase and set the ownerReference * if the owner hasn't been set yet. ******************************************************************/ - if len(backup.GetOwnerReferences()) == 0 && backup.Spec.AutonomousDatabaseOCID != "" { - adb, err := r.getOwnerAutonomousDatabase(backup.Namespace, backup.Spec.AutonomousDatabaseOCID) - if err != nil { - return r.manageError(backup, err) - } - if adb != nil { - if err := r.setOwnerAutonomousDatabase(backup, adb); err != nil { - return r.manageError(backup, err) - } - } + if err := r.verifyOwnerADB(backup); err != nil { + return r.manageError(backup, err) } /****************************************************************** @@ -137,9 +149,11 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req return r.manageError(backup, err) } - // update the status - backup.UpdateStatusFromOCIBackup(backupResp.AutonomousDatabaseBackup, adbResp.AutonomousDatabase) - r.updateResourceStatus(backup) + // update the Backup status + backup.UpdateFromOCIBackup(backupResp.AutonomousDatabaseBackup, adbResp.AutonomousDatabase) + if err := r.updateResourceStatus(backup); err != nil { + return r.manageError(backup, err) + } // Wait until the work is done if _, err := r.workService.Wait(*backupResp.OpcWorkRequestId); err != nil { @@ -148,6 +162,7 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req logger.Info("AutonomousDatabaseBackup " + *backupResp.DisplayName + " created successfully") + return ctrl.Result{Requeue: true}, nil } /****************************************************************** @@ -165,13 +180,31 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req return r.manageError(backup, err) } - backup.UpdateStatusFromOCIBackup(backupResp.AutonomousDatabaseBackup, adbResp.AutonomousDatabase) + backup.UpdateFromOCIBackup(backupResp.AutonomousDatabaseBackup, adbResp.AutonomousDatabase) if err := r.updateResourceStatus(backup); err != nil { return r.manageError(backup, err) } + + return emptyResult, nil } - return ctrl.Result{}, nil + return emptyResult, nil +} + +func (r *AutonomousDatabaseBackupReconciler) verifyOwnerADB(backup *dbv1alpha1.AutonomousDatabaseBackup) error { + if len(backup.GetOwnerReferences()) == 0 && backup.Spec.AutonomousDatabaseOCID != "" { + var err error + r.ownerADB, err = k8s.FetchAutonomousDatabase(r.KubeClient, backup.Namespace, backup.Spec.AutonomousDatabaseOCID) + if err != nil { + return err + } + if r.ownerADB == nil { + if err := r.setOwnerAutonomousDatabase(backup, r.ownerADB); err != nil { + return err + } + } + } + return nil } func (r *AutonomousDatabaseBackupReconciler) setupOCIClients(backup *dbv1alpha1.AutonomousDatabaseBackup) error { @@ -232,6 +265,11 @@ func (r *AutonomousDatabaseBackupReconciler) updateResource(backup *dbv1alpha1.A } func (r *AutonomousDatabaseBackupReconciler) updateResourceStatus(backup *dbv1alpha1.AutonomousDatabaseBackup) error { + // sync the ADB status every time when the Backup status is updated + if err := r.syncADBStatus(); err != nil { + return err + } + return retry.RetryOnConflict(retry.DefaultRetry, func() error { curBackup := &dbv1alpha1.AutonomousDatabaseBackup{} @@ -249,21 +287,23 @@ func (r *AutonomousDatabaseBackupReconciler) updateResourceStatus(backup *dbv1al }) } -// Returns the first AutonomousDatabase resource that matches the AutonomousDatabaseOCID of the backup -// If the AutonomousDatabase doesn't exist, returns a nil -func (r *AutonomousDatabaseBackupReconciler) getOwnerAutonomousDatabase(namespace string, adbOCID string) (*dbv1alpha1.AutonomousDatabase, error) { - adbList, err := k8s.FetchAutonomousDatabases(r.KubeClient, namespace) +// No-op if the ownerADB is not assigned +func (r *AutonomousDatabaseBackupReconciler) syncADBStatus() error { + if r.ownerADB == nil { + return nil + } + resp, err := r.adbService.GetAutonomousDatabase(*r.ownerADB.Spec.Details.AutonomousDatabaseOCID) if err != nil { - return nil, err + return err } - for _, adb := range adbList.Items { - if adb.Spec.Details.AutonomousDatabaseOCID != nil && *adb.Spec.Details.AutonomousDatabaseOCID == adbOCID { - return &adb, nil - } + r.ownerADB.Status.LifecycleState = resp.LifecycleState + + if err := k8s.UpdateADBStatus(r.KubeClient, r.ownerADB); err != nil { + return err } - return nil, nil + return nil } // setOwnerAutonomousDatabase sets the owner of the AutonomousDatabaseBackup if the AutonomousDatabase resource with the same database OCID is found diff --git a/controllers/database/autonomousdatabaserestore_controller.go b/controllers/database/autonomousdatabaserestore_controller.go index a6c2a550..7e74126b 100644 --- a/controllers/database/autonomousdatabaserestore_controller.go +++ b/controllers/database/autonomousdatabaserestore_controller.go @@ -41,6 +41,7 @@ package controllers import ( "context" "errors" + "fmt" "github.com/go-logr/logr" apiErrors "k8s.io/apimachinery/pkg/api/errors" @@ -67,6 +68,7 @@ type AutonomousDatabaseRestoreReconciler struct { adbService oci.DatabaseService workService oci.WorkRequestService + ownerADB *dbv1alpha1.AutonomousDatabase } // SetupWithManager sets up the controller with the Manager. @@ -103,6 +105,22 @@ func (r *AutonomousDatabaseRestoreReconciler) Reconcile(ctx context.Context, req return emptyResult, err } + /****************************************************************** + * Extract the restoreTime and the adbOCID from the spec + ******************************************************************/ + restoreTime, adbOCID, err := r.fetchData(restore) + if err != nil { + return r.manageError(restore, err) + } + + /****************************************************************** + * Look up the owner AutonomousDatabase and set the ownerReference + * if the owner hasn't been set yet. + ******************************************************************/ + if err := r.verifyOwnerADB(restore, adbOCID); err != nil { + return r.manageError(restore, err) + } + /****************************************************************** * Get OCI database client and work request client ******************************************************************/ @@ -115,7 +133,7 @@ func (r *AutonomousDatabaseRestoreReconciler) Reconcile(ctx context.Context, req /****************************************************************** * Start restore ******************************************************************/ - if err := r.restoreAutonomousDatabase(restore); err != nil { + if err := r.restoreAutonomousDatabase(restore, restoreTime, adbOCID); err != nil { return r.manageError(restore, err) } @@ -124,24 +142,20 @@ func (r *AutonomousDatabaseRestoreReconciler) Reconcile(ctx context.Context, req return ctrl.Result{}, nil } -func (r *AutonomousDatabaseRestoreReconciler) restoreAutonomousDatabase(restore *dbv1alpha1.AutonomousDatabaseRestore) error { - var restoreTime *common.SDKTime - var adbOCID string - var err error - +func (r *AutonomousDatabaseRestoreReconciler) fetchData(restore *dbv1alpha1.AutonomousDatabaseRestore) (restoreTime *common.SDKTime, adbOCID string, err error) { if restore.Spec.BackupName != "" { // restore using backupName backup := &dbv1alpha1.AutonomousDatabaseBackup{} namespacedName := types.NamespacedName{Namespace: restore.Namespace, Name: restore.Spec.BackupName} if err := r.KubeClient.Get(context.TODO(), namespacedName, backup); err != nil { - return err + return restoreTime, adbOCID, err } if backup.Status.TimeEnded == "" { - return errors.New("broken backup: ended time is missing in the AutonomousDatabaseBackup " + backup.GetName()) + return restoreTime, adbOCID, errors.New("broken backup: ended time is missing in the AutonomousDatabaseBackup " + backup.GetName()) } restoreTime, err = backup.GetTimeEnded() if err != nil { - return err + return restoreTime, adbOCID, err } adbOCID = backup.Spec.AutonomousDatabaseOCID @@ -152,6 +166,48 @@ func (r *AutonomousDatabaseRestoreReconciler) restoreAutonomousDatabase(restore adbOCID = restore.Spec.PointInTime.AutonomousDatabaseOCID } + return restoreTime, adbOCID, nil +} + +// setOwnerAutonomousDatabase sets the owner of the AutonomousDatabaseBackup if the AutonomousDatabase resource with the same database OCID is found +func (r *AutonomousDatabaseRestoreReconciler) setOwnerAutonomousDatabase(restore *dbv1alpha1.AutonomousDatabaseRestore, adb *dbv1alpha1.AutonomousDatabase) error { + logger := r.Log.WithName("set-owner") + + restore.SetOwnerReferences(k8s.NewOwnerReference(adb)) + + if err := r.KubeClient.Update(context.TODO(), restore); err != nil { + return err + } + + logger.Info(fmt.Sprintf("Set the owner of %s to %s", restore.Name, adb.Name)) + + return nil +} + +func (r *AutonomousDatabaseRestoreReconciler) verifyOwnerADB(restore *dbv1alpha1.AutonomousDatabaseRestore, adbOCID string) error { + if len(restore.GetOwnerReferences()) == 0 { + var err error + + r.ownerADB, err = k8s.FetchAutonomousDatabase(r.KubeClient, restore.Namespace, adbOCID) + if err != nil { + return err + } + + if r.ownerADB == nil { + if err := r.setOwnerAutonomousDatabase(restore, r.ownerADB); err != nil { + return err + } + } + } + + return nil +} + +func (r *AutonomousDatabaseRestoreReconciler) restoreAutonomousDatabase( + restore *dbv1alpha1.AutonomousDatabaseRestore, + restoreTime *common.SDKTime, + adbOCID string) error { + resp, err := r.adbService.RestoreAutonomousDatabase(adbOCID, *restoreTime) if err != nil { return err @@ -179,6 +235,11 @@ func (r *AutonomousDatabaseRestoreReconciler) restoreAutonomousDatabase(restore } func (r *AutonomousDatabaseRestoreReconciler) updateResourceStatus(restore *dbv1alpha1.AutonomousDatabaseRestore) error { + // sync the ADB status every time when the Backup status is updated + if err := r.syncADBStatus(); err != nil { + return err + } + return retry.RetryOnConflict(retry.DefaultRetry, func() error { curBackup := &dbv1alpha1.AutonomousDatabaseRestore{} @@ -196,6 +257,26 @@ func (r *AutonomousDatabaseRestoreReconciler) updateResourceStatus(restore *dbv1 }) } +// No-op if the ownerADB is not assigned +func (r *AutonomousDatabaseRestoreReconciler) syncADBStatus() error { + if r.ownerADB == nil { + return nil + } + + resp, err := r.adbService.GetAutonomousDatabase(*r.ownerADB.Spec.Details.AutonomousDatabaseOCID) + if err != nil { + return err + } + + r.ownerADB.Status.LifecycleState = resp.LifecycleState + + if err := k8s.UpdateADBStatus(r.KubeClient, r.ownerADB); err != nil { + return err + } + + return nil +} + func (r *AutonomousDatabaseRestoreReconciler) setupOCIClients(restore *dbv1alpha1.AutonomousDatabaseRestore) error { var err error From edb06936114c108762a28ecb71661865abb25f95 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Thu, 10 Mar 2022 09:18:48 -0500 Subject: [PATCH 183/628] fix minor errors --- config/samples/kustomization.yaml | 15 ++++++++------- controllers/database/suite_test.go | 3 --- 2 files changed, 8 insertions(+), 10 deletions(-) diff --git a/config/samples/kustomization.yaml b/config/samples/kustomization.yaml index 0157dec9..214006e2 100644 --- a/config/samples/kustomization.yaml +++ b/config/samples/kustomization.yaml @@ -5,13 +5,14 @@ ## Append samples you want in your CSV to this file as resources ## resources: - - sharding_v1alpha1_provshard.yaml - - autonomousdatabase.yaml - - singleinstancedatabase.yaml - - shardingdatabase.yaml - - database_v1alpha1_pdb.yaml - - database_v1alpha1_cdb.yaml - - adb/autonomousdatabase.yaml + - onpremdb/pdb.yaml + - onpremdb/cdb.yaml + - adb/autonomousdatabase_create.yaml + - adb/autonomousdatabase_bind.yaml + - adb/autonomousdatabase_backup.yaml + - adb/autonomousdatabase_restore.yaml + - adb/autonomouscontainerdatabase.yaml - sidb/singleinstancedatabase.yaml - sharding/shardingdatabase.yaml + - sharding/sharding_v1alpha1_provshard.yaml # +kubebuilder:scaffold:manifestskustomizesamples diff --git a/controllers/database/suite_test.go b/controllers/database/suite_test.go index fdebedce..9b62324a 100644 --- a/controllers/database/suite_test.go +++ b/controllers/database/suite_test.go @@ -87,9 +87,6 @@ var _ = BeforeSuite(func(done Done) { err = databasev1alpha1.AddToScheme(scheme.Scheme) Expect(err).NotTo(HaveOccurred()) - err = databasev1alpha1.AddToScheme(scheme.Scheme) - Expect(err).NotTo(HaveOccurred()) - // +kubebuilder:scaffold:scheme k8sClient, err = client.New(cfg, client.Options{Scheme: scheme.Scheme}) From 53cbd270cb08acf7d0d824951c74d549ca02f96b Mon Sep 17 00:00:00 2001 From: Susmita Samanta Date: Mon, 14 Mar 2022 17:43:04 +0530 Subject: [PATCH 184/628] added fix for pdbname for XE image --- controllers/database/singleinstancedatabase_controller.go | 3 +++ 1 file changed, 3 insertions(+) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 89fbbd57..9fce2c0a 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1013,6 +1013,9 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex pdbName := "ORCLPDB1" sid := m.Spec.Sid + if strings.ToUpper(sid) == "XE" { + pdbName = "XEPDB1" + } if m.Spec.Persistence.AccessMode == "" { sid, pdbName, m.Status.Edition = dbcommons.GetSidPdbEdition(r, r.Config, ctx, req) if sid == "" || pdbName == "" || m.Status.Edition == "" { From acdc259cd969c394fa429adc78034483c0ddfa27 Mon Sep 17 00:00:00 2001 From: Susmita Samanta Date: Mon, 14 Mar 2022 19:17:08 +0530 Subject: [PATCH 185/628] adding support for xe image --- apis/database/v1alpha1/singleinstancedatabase_types.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/apis/database/v1alpha1/singleinstancedatabase_types.go b/apis/database/v1alpha1/singleinstancedatabase_types.go index 6262a9d9..1b20472d 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_types.go +++ b/apis/database/v1alpha1/singleinstancedatabase_types.go @@ -50,7 +50,7 @@ type SingleInstanceDatabaseSpec struct { // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster // Important: Run "make" to regenerate code after modifying this file - // +kubebuilder:validation:Enum=standard;enterprise + // +kubebuilder:validation:Enum=standard;enterprise;express Edition string `json:"edition,omitempty"` // SID can only have a-z , A-Z, 0-9 . It cant have any special characters From f4908c830a174463e32c4662b13926d04a8cd286 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Mon, 14 Mar 2022 10:01:18 -0400 Subject: [PATCH 186/628] adding owner refereces for Restore and Backup --- .../v1alpha1/adbfamily_common_utils.go | 16 +- .../v1alpha1/autonomousdatabase_types.go | 100 ++++----- .../autonomousdatabasebackup_types.go | 35 ++-- .../autonomousdatabasebackup_webhook.go | 12 +- .../autonomousdatabaserestore_types.go | 18 +- .../autonomousdatabaserestore_webhook.go | 25 ++- .../v1alpha1/zz_generated.deepcopy.go | 189 +++++++++++++++--- commons/k8s/create.go | 4 +- commons/k8s/fetch.go | 38 +++- commons/k8s/finalizer.go | 30 ++- commons/oci/database.go | 29 ++- ....oracle.com_autonomousdatabasebackups.yaml | 21 +- ...oracle.com_autonomousdatabaserestores.yaml | 28 ++- ...tabase.oracle.com_autonomousdatabases.yaml | 72 ------- .../database/autonomousdatabase_controller.go | 149 ++++++++------ .../autonomousdatabasebackup_controller.go | 27 ++- .../autonomousdatabaserestore_controller.go | 133 +++++++----- 17 files changed, 561 insertions(+), 365 deletions(-) diff --git a/apis/database/v1alpha1/adbfamily_common_utils.go b/apis/database/v1alpha1/adbfamily_common_utils.go index cbb97316..2abf9d93 100644 --- a/apis/database/v1alpha1/adbfamily_common_utils.go +++ b/apis/database/v1alpha1/adbfamily_common_utils.go @@ -70,6 +70,13 @@ type OCIConfigSpec struct { SecretName *string `json:"secretName,omitempty"` } +// TargetSpec defines the spec of the target for backup/restore runs. +// The name could be the name of an AutonomousDatabase or an AutonomousDatabaseBackup +type TargetSpec struct { + Name string `json:"name"` + OCID string `json:"ocid"` +} + // removeUnchangedFields removes the unchanged fields in the struct and returns if the struct is changed. // lastSpec should be a derefereced struct that is the last successful spec, e.g. AutonomousDatabaseSpec. // curSpec should be a pointer pointing to the struct that is being proccessed, e.g., *AutonomousDatabaseSpec. @@ -159,8 +166,13 @@ func hasChanged(lastField reflect.Value, curField reflect.Value) bool { // Follow the format of the display time const displayFormat = "2006-01-02 15:04:05 MST" -func formatSDKTime(dateTime time.Time) string { - return dateTime.Format(displayFormat) +func formatSDKTime(sdkTime *common.SDKTime) string { + if sdkTime == nil { + return "" + } + + time := sdkTime.Time + return time.Format(displayFormat) } func parseDisplayTime(val string) (*common.SDKTime, error) { diff --git a/apis/database/v1alpha1/autonomousdatabase_types.go b/apis/database/v1alpha1/autonomousdatabase_types.go index 1dfd8384..501bf546 100644 --- a/apis/database/v1alpha1/autonomousdatabase_types.go +++ b/apis/database/v1alpha1/autonomousdatabase_types.go @@ -43,7 +43,6 @@ import ( "errors" "github.com/oracle/oci-go-sdk/v54/database" - "k8s.io/apimachinery/pkg/api/meta" metaV1 "k8s.io/apimachinery/pkg/apis/meta/v1" ) @@ -125,12 +124,6 @@ type AutonomousDatabaseStatus struct { LifecycleState database.AutonomousDatabaseLifecycleStateEnum `json:"lifecycleState,omitempty"` TimeCreated string `json:"timeCreated,omitempty"` AllConnectionStrings []ConnectionStringProfile `json:"allConnectionStrings,omitempty"` - - // +patchMergeKey=type - // +patchStrategy=merge - // +listType=map - // +listMapKey=type - Conditions []metaV1.Condition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type"` } type TLSAuthenticationEnum string @@ -201,17 +194,53 @@ func (adb *AutonomousDatabase) GetLastSuccessfulSpec() (*AutonomousDatabaseSpec, return &sucSpec, nil } -func (adb *AutonomousDatabase) UpdateConditions(lifecycleState database.AutonomousDatabaseLifecycleStateEnum, err error) { - var metaCondition metaV1.Condition +// UpdateStatusFromOCIADB updates only the status from database.AutonomousDatabase object +func (adb *AutonomousDatabase) UpdateStatusFromOCIADB(ociObj database.AutonomousDatabase) { + adb.Status.LifecycleState = ociObj.LifecycleState + adb.Status.TimeCreated = formatSDKTime(ociObj.TimeCreated) - if err != nil { - metaCondition = CreateMetaCondition(adb, err, string(CrdReconcileErrorState), string(CrdReconcileErrorReason)) - } + if *ociObj.IsDedicated { + conns := make([]ConnectionStringSpec, len(ociObj.ConnectionStrings.AllConnectionStrings)) + for key, val := range ociObj.ConnectionStrings.AllConnectionStrings { + conns = append(conns, ConnectionStringSpec{TNSName: key, ConnectionString: val}) + } - meta.SetStatusCondition(&adb.Status.Conditions, metaCondition) + adb.Status.AllConnectionStrings = []ConnectionStringProfile{ + {ConnectionStrings: conns}, + } + } else { + var mTLSConns []ConnectionStringSpec + var tlsConns []ConnectionStringSpec + + var conns []ConnectionStringProfile + + for _, profile := range ociObj.ConnectionStrings.Profiles { + if profile.TlsAuthentication == database.DatabaseConnectionStringProfileTlsAuthenticationMutual { + mTLSConns = append(mTLSConns, ConnectionStringSpec{TNSName: *profile.DisplayName, ConnectionString: *profile.Value}) + } else { + tlsConns = append(tlsConns, ConnectionStringSpec{TNSName: *profile.DisplayName, ConnectionString: *profile.Value}) + } + } + + if len(mTLSConns) > 0 { + conns = append(conns, ConnectionStringProfile{ + TLSAuthentication: tlsAuthenticationMTLS, + ConnectionStrings: mTLSConns, + }) + } + + if len(tlsConns) > 0 { + conns = append(conns, ConnectionStringProfile{ + TLSAuthentication: tlsAuthenticationTLS, + ConnectionStrings: tlsConns, + }) + } + + adb.Status.AllConnectionStrings = conns + } } -// UpdateAttrFromOCIAutonomousDatabase updates the attributes from database.AutonomousDatabase object and returns the resource +// UpdateFromOCIADB updates the attributes from database.AutonomousDatabase object func (adb *AutonomousDatabase) UpdateFromOCIADB(ociObj database.AutonomousDatabase) { /*********************************** * update the spec @@ -255,48 +284,7 @@ func (adb *AutonomousDatabase) UpdateFromOCIADB(ociObj database.AutonomousDataba * update the status subresource ***********************************/ - adb.Status.LifecycleState = ociObj.LifecycleState - adb.Status.TimeCreated = ociObj.TimeCreated.String() - - if *ociObj.IsDedicated { - conns := make([]ConnectionStringSpec, len(ociObj.ConnectionStrings.AllConnectionStrings)) - for key, val := range ociObj.ConnectionStrings.AllConnectionStrings { - conns = append(conns, ConnectionStringSpec{TNSName: key, ConnectionString: val}) - } - - adb.Status.AllConnectionStrings = []ConnectionStringProfile{ - ConnectionStringProfile{ConnectionStrings: conns}, - } - } else { - var mTLSConns []ConnectionStringSpec - var tlsConns []ConnectionStringSpec - - var conns []ConnectionStringProfile - - for _, profile := range ociObj.ConnectionStrings.Profiles { - if profile.TlsAuthentication == database.DatabaseConnectionStringProfileTlsAuthenticationMutual { - mTLSConns = append(mTLSConns, ConnectionStringSpec{TNSName: *profile.DisplayName, ConnectionString: *profile.Value}) - } else { - tlsConns = append(tlsConns, ConnectionStringSpec{TNSName: *profile.DisplayName, ConnectionString: *profile.Value}) - } - } - - if len(mTLSConns) > 0 { - conns = append(conns, ConnectionStringProfile{ - TLSAuthentication: tlsAuthenticationMTLS, - ConnectionStrings: mTLSConns, - }) - } - - if len(tlsConns) > 0 { - conns = append(conns, ConnectionStringProfile{ - TLSAuthentication: tlsAuthenticationTLS, - ConnectionStrings: tlsConns, - }) - } - - adb.Status.AllConnectionStrings = conns - } + adb.UpdateStatusFromOCIADB(ociObj) } // RemoveUnchangedDetails removes the unchanged fields in spec.details, and returns if the details has been changed. diff --git a/apis/database/v1alpha1/autonomousdatabasebackup_types.go b/apis/database/v1alpha1/autonomousdatabasebackup_types.go index 8409f4ce..3c27d3bb 100644 --- a/apis/database/v1alpha1/autonomousdatabasebackup_types.go +++ b/apis/database/v1alpha1/autonomousdatabasebackup_types.go @@ -52,9 +52,9 @@ import ( type AutonomousDatabaseBackupSpec struct { // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster // Important: Run "make" to regenerate code after modifying this file - AutonomousDatabaseOCID string `json:"autonomousDatabaseOCID"` - DisplayName string `json:"displayName"` - AutonomousDatabaseBackupOCID string `json:"autonomousDatabaseBackupOCID,omitempty"` + TargetADB TargetSpec `json:"targetADB"` + DisplayName string `json:"displayName"` + AutonomousDatabaseBackupOCID string `json:"autonomousDatabaseBackupOCID,omitempty"` OCIConfig OCIConfigSpec `json:"ociConfig,omitempty"` } @@ -63,14 +63,15 @@ type AutonomousDatabaseBackupSpec struct { type AutonomousDatabaseBackupStatus struct { // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster // Important: Run "make" to regenerate code after modifying this file - LifecycleState database.AutonomousDatabaseBackupLifecycleStateEnum `json:"lifecycleState"` - Type database.AutonomousDatabaseBackupTypeEnum `json:"type"` - IsAutomatic bool `json:"isAutomatic"` - TimeStarted string `json:"timeStarted,omitempty"` - TimeEnded string `json:"timeEnded,omitempty"` - CompartmentOCID string `json:"compartmentOCID"` - DBName string `json:"dbName"` - DBDisplayName string `json:"dbDisplayName"` + LifecycleState database.AutonomousDatabaseBackupLifecycleStateEnum `json:"lifecycleState"` + Type database.AutonomousDatabaseBackupTypeEnum `json:"type"` + IsAutomatic bool `json:"isAutomatic"` + TimeStarted string `json:"timeStarted,omitempty"` + TimeEnded string `json:"timeEnded,omitempty"` + AutonomousDatabaseOCID string `json:"autonomousDatabaseOCID"` + CompartmentOCID string `json:"compartmentOCID"` + DBName string `json:"dbName"` + DBDisplayName string `json:"dbDisplayName"` } //+kubebuilder:object:root=true @@ -106,20 +107,16 @@ func init() { func (backup *AutonomousDatabaseBackup) UpdateFromOCIBackup(ociBackup database.AutonomousDatabaseBackup, ociADB database.AutonomousDatabase) { backup.Spec.AutonomousDatabaseBackupOCID = *ociBackup.Id - backup.Spec.AutonomousDatabaseOCID = *ociBackup.AutonomousDatabaseId backup.Spec.DisplayName = *ociBackup.DisplayName - + + backup.Status.AutonomousDatabaseOCID = *ociBackup.AutonomousDatabaseId backup.Status.CompartmentOCID = *ociBackup.CompartmentId backup.Status.Type = ociBackup.Type backup.Status.IsAutomatic = *ociBackup.IsAutomatic backup.Status.LifecycleState = ociBackup.LifecycleState - if ociBackup.TimeStarted != nil { - backup.Status.TimeStarted = formatSDKTime(ociBackup.TimeStarted.Time) - } - if ociBackup.TimeEnded != nil { - backup.Status.TimeEnded = formatSDKTime(ociBackup.TimeEnded.Time) - } + backup.Status.TimeStarted = formatSDKTime(ociBackup.TimeStarted) + backup.Status.TimeEnded = formatSDKTime(ociBackup.TimeEnded) backup.Status.DBDisplayName = *ociADB.DisplayName backup.Status.DBName = *ociADB.DbName diff --git a/apis/database/v1alpha1/autonomousdatabasebackup_webhook.go b/apis/database/v1alpha1/autonomousdatabasebackup_webhook.go index 40757eb9..a3dc9299 100644 --- a/apis/database/v1alpha1/autonomousdatabasebackup_webhook.go +++ b/apis/database/v1alpha1/autonomousdatabasebackup_webhook.go @@ -94,10 +94,16 @@ func (r *AutonomousDatabaseBackup) ValidateUpdate(old runtime.Object) error { field.Forbidden(field.NewPath("spec").Child("autonomousDatabaseBackupOCID"), "cannot assign a new autonomousDatabaseBackupOCID to this backup")) } - if r.Spec.AutonomousDatabaseOCID != "" && - oldBackup.Spec.AutonomousDatabaseOCID != r.Spec.AutonomousDatabaseOCID { + if r.Spec.TargetADB.Name != "" && + oldBackup.Spec.TargetADB.Name != r.Spec.TargetADB.Name { allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec").Child("autonomousDatabaseOCID"), "cannot assign a new autonomousDatabaseOCID to this backup")) + field.Forbidden(field.NewPath("spec").Child("targetADB").Child("name"), "cannot change target.name")) + } + + if r.Spec.TargetADB.OCID != "" && + oldBackup.Spec.TargetADB.OCID != r.Spec.TargetADB.OCID { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("targetADB").Child("ocid"), "cannot change target.ocid")) } if r.Spec.DisplayName != "" && diff --git a/apis/database/v1alpha1/autonomousdatabaserestore_types.go b/apis/database/v1alpha1/autonomousdatabaserestore_types.go index 16cc09d2..1378f854 100644 --- a/apis/database/v1alpha1/autonomousdatabaserestore_types.go +++ b/apis/database/v1alpha1/autonomousdatabaserestore_types.go @@ -52,14 +52,18 @@ import ( type AutonomousDatabaseRestoreSpec struct { // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster // Important: Run "make" to regenerate code after modifying this file - BackupName string `json:"backupName,omitempty"` - PointInTime PITSource `json:"pointInTime,omitempty"` - OCIConfig OCIConfigSpec `json:"ociConfig,omitempty"` + TargetADB TargetSpec `json:"targetADB"` + Source SourceSpec `json:"sourceSpec"` + OCIConfig OCIConfigSpec `json:"ociConfig,omitempty"` } -type PITSource struct { - AutonomousDatabaseOCID string `json:"autonomousDatabaseOCID,omitempty"` - TimeStamp string `json:"timeStamp,omitempty"` +type SourceSpec struct { + AutonomousDatabaseBackup BackupSourceSpec `json:"autonomousDatabaseBackup,omitempty"` + PointInTime string `json:"pointInTime,omitempty"` +} + +type BackupSourceSpec struct { + Name string `json:"name,omitempty"` } type restoreStatusEnum string @@ -111,7 +115,7 @@ func init() { // GetPIT returns the spec.pointInTime.timeStamp in SDKTime format func (r *AutonomousDatabaseRestore) GetPIT() (*common.SDKTime, error) { - return parseDisplayTime(r.Spec.PointInTime.TimeStamp) + return parseDisplayTime(r.Spec.Source.PointInTime) } func (r *AutonomousDatabaseRestore) ConvertWorkRequestStatus(s workrequests.WorkRequestStatusEnum) restoreStatusEnum { diff --git a/apis/database/v1alpha1/autonomousdatabaserestore_webhook.go b/apis/database/v1alpha1/autonomousdatabaserestore_webhook.go index 74195918..407403e9 100644 --- a/apis/database/v1alpha1/autonomousdatabaserestore_webhook.go +++ b/apis/database/v1alpha1/autonomousdatabaserestore_webhook.go @@ -71,30 +71,29 @@ func (r *AutonomousDatabaseRestore) ValidateCreate() error { var allErrs field.ErrorList // Validate the restore source - if r.Spec.BackupName == "" && - r.Spec.PointInTime.AutonomousDatabaseOCID == "" && - r.Spec.PointInTime.TimeStamp == "" { + if r.Spec.Source.AutonomousDatabaseBackup.Name == "" && + r.Spec.Source.PointInTime == "" { allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec"), "no retore source is chosen")) + field.Forbidden(field.NewPath("spec").Child("source"), "no retore source is chosen")) } - if r.Spec.BackupName != "" && - (r.Spec.PointInTime.AutonomousDatabaseOCID != "" || r.Spec.PointInTime.TimeStamp != "") { + if r.Spec.Source.AutonomousDatabaseBackup.Name != "" && + r.Spec.Source.PointInTime != "" { allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec"), "cannot apply backupName and the PITR parameters at the same time")) + field.Forbidden(field.NewPath("spec").Child("source"), "cannot apply backupName and the PITR parameters at the same time")) } - if (r.Spec.PointInTime.AutonomousDatabaseOCID == "" && r.Spec.PointInTime.TimeStamp != "") || - (r.Spec.PointInTime.AutonomousDatabaseOCID != "" && r.Spec.PointInTime.TimeStamp == "") { - field.Forbidden(field.NewPath("spec").Child("pointInTime"), "autonomousDatabaseOCID or timeStamp cannot be empty") + if (r.Spec.TargetADB.OCID == "" && r.Spec.Source.PointInTime != "") || + (r.Spec.TargetADB.OCID != "" && r.Spec.Source.PointInTime == "") { + field.Forbidden(field.NewPath("spec").Child("source").Child("pointInTime"), "targetADB.OCID or source.pointInTime cannot be empty") } // Verify the timestamp format if it's PITR - if r.Spec.PointInTime.TimeStamp != "" { - _, err := parseDisplayTime(r.Spec.PointInTime.TimeStamp) + if r.Spec.Source.PointInTime != "" { + _, err := parseDisplayTime(r.Spec.Source.PointInTime) if err != nil { allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec"), "invalid timestamp format")) + field.Forbidden(field.NewPath("spec").Child("source").Child("pointInTime"), "invalid timestamp format")) } } diff --git a/apis/database/v1alpha1/zz_generated.deepcopy.go b/apis/database/v1alpha1/zz_generated.deepcopy.go index 82c24b70..9507b955 100644 --- a/apis/database/v1alpha1/zz_generated.deepcopy.go +++ b/apis/database/v1alpha1/zz_generated.deepcopy.go @@ -49,6 +49,123 @@ import ( "k8s.io/apimachinery/pkg/runtime" ) +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousContainerDatabase) DeepCopyInto(out *AutonomousContainerDatabase) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + in.Spec.DeepCopyInto(&out.Spec) + out.Status = in.Status +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousContainerDatabase. +func (in *AutonomousContainerDatabase) DeepCopy() *AutonomousContainerDatabase { + if in == nil { + return nil + } + out := new(AutonomousContainerDatabase) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *AutonomousContainerDatabase) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousContainerDatabaseList) DeepCopyInto(out *AutonomousContainerDatabaseList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]AutonomousContainerDatabase, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousContainerDatabaseList. +func (in *AutonomousContainerDatabaseList) DeepCopy() *AutonomousContainerDatabaseList { + if in == nil { + return nil + } + out := new(AutonomousContainerDatabaseList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *AutonomousContainerDatabaseList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousContainerDatabaseSpec) DeepCopyInto(out *AutonomousContainerDatabaseSpec) { + *out = *in + if in.AutonomousContainerDatabaseOCID != nil { + in, out := &in.AutonomousContainerDatabaseOCID, &out.AutonomousContainerDatabaseOCID + *out = new(string) + **out = **in + } + if in.CompartmentOCID != nil { + in, out := &in.CompartmentOCID, &out.CompartmentOCID + *out = new(string) + **out = **in + } + if in.DisplayName != nil { + in, out := &in.DisplayName, &out.DisplayName + *out = new(string) + **out = **in + } + if in.AutonomousExadataVMClusterOCID != nil { + in, out := &in.AutonomousExadataVMClusterOCID, &out.AutonomousExadataVMClusterOCID + *out = new(string) + **out = **in + } + if in.FreeformTags != nil { + in, out := &in.FreeformTags, &out.FreeformTags + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + in.OCIConfig.DeepCopyInto(&out.OCIConfig) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousContainerDatabaseSpec. +func (in *AutonomousContainerDatabaseSpec) DeepCopy() *AutonomousContainerDatabaseSpec { + if in == nil { + return nil + } + out := new(AutonomousContainerDatabaseSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousContainerDatabaseStatus) DeepCopyInto(out *AutonomousContainerDatabaseStatus) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousContainerDatabaseStatus. +func (in *AutonomousContainerDatabaseStatus) DeepCopy() *AutonomousContainerDatabaseStatus { + if in == nil { + return nil + } + out := new(AutonomousContainerDatabaseStatus) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *AutonomousDatabase) DeepCopyInto(out *AutonomousDatabase) { *out = *in @@ -138,6 +255,7 @@ func (in *AutonomousDatabaseBackupList) DeepCopyObject() runtime.Object { // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *AutonomousDatabaseBackupSpec) DeepCopyInto(out *AutonomousDatabaseBackupSpec) { *out = *in + out.TargetADB = in.TargetADB in.OCIConfig.DeepCopyInto(&out.OCIConfig) } @@ -335,7 +453,8 @@ func (in *AutonomousDatabaseRestoreList) DeepCopyObject() runtime.Object { // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *AutonomousDatabaseRestoreSpec) DeepCopyInto(out *AutonomousDatabaseRestoreSpec) { *out = *in - out.PointInTime = in.PointInTime + out.TargetADB = in.TargetADB + out.Source = in.Source in.OCIConfig.DeepCopyInto(&out.OCIConfig) } @@ -396,13 +515,6 @@ func (in *AutonomousDatabaseStatus) DeepCopyInto(out *AutonomousDatabaseStatus) (*in)[i].DeepCopyInto(&(*out)[i]) } } - if in.Conditions != nil { - in, out := &in.Conditions, &out.Conditions - *out = make([]v1.Condition, len(*in)) - for i := range *in { - (*in)[i].DeepCopyInto(&(*out)[i]) - } - } } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseStatus. @@ -415,6 +527,21 @@ func (in *AutonomousDatabaseStatus) DeepCopy() *AutonomousDatabaseStatus { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *BackupSourceSpec) DeepCopyInto(out *BackupSourceSpec) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new BackupSourceSpec. +func (in *BackupSourceSpec) DeepCopy() *BackupSourceSpec { + if in == nil { + return nil + } + out := new(BackupSourceSpec) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *CDB) DeepCopyInto(out *CDB) { *out = *in @@ -1178,21 +1305,6 @@ func (in *PDBStatus) DeepCopy() *PDBStatus { return out } -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *PITSource) DeepCopyInto(out *PITSource) { - *out = *in -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PITSource. -func (in *PITSource) DeepCopy() *PITSource { - if in == nil { - return nil - } - out := new(PITSource) - in.DeepCopyInto(out) - return out -} - // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *PasswordSpec) DeepCopyInto(out *PasswordSpec) { *out = *in @@ -1632,6 +1744,22 @@ func (in *SingleInstanceDatabaseStatus) DeepCopy() *SingleInstanceDatabaseStatus return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *SourceSpec) DeepCopyInto(out *SourceSpec) { + *out = *in + out.AutonomousDatabaseBackup = in.AutonomousDatabaseBackup +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SourceSpec. +func (in *SourceSpec) DeepCopy() *SourceSpec { + if in == nil { + return nil + } + out := new(SourceSpec) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *TDEPwd) DeepCopyInto(out *TDEPwd) { *out = *in @@ -1664,6 +1792,21 @@ func (in *TDESecret) DeepCopy() *TDESecret { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *TargetSpec) DeepCopyInto(out *TargetSpec) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new TargetSpec. +func (in *TargetSpec) DeepCopy() *TargetSpec { + if in == nil { + return nil + } + out := new(TargetSpec) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *WalletSpec) DeepCopyInto(out *WalletSpec) { *out = *in diff --git a/commons/k8s/create.go b/commons/k8s/create.go index 9dfdd24d..b3e7bd8f 100644 --- a/commons/k8s/create.go +++ b/commons/k8s/create.go @@ -86,7 +86,9 @@ func CreateAutonomousBackup(kubeClient client.Client, OwnerReferences: NewOwnerReference(ownerADB), }, Spec: dbv1alpha1.AutonomousDatabaseBackupSpec{ - AutonomousDatabaseOCID: *backupSummary.AutonomousDatabaseId, + TargetADB: dbv1alpha1.TargetSpec{ + Name: ownerADB.Name, + }, DisplayName: *backupSummary.DisplayName, AutonomousDatabaseBackupOCID: *backupSummary.Id, OCIConfig: dbv1alpha1.OCIConfigSpec{ diff --git a/commons/k8s/fetch.go b/commons/k8s/fetch.go index c9afb1e0..17e0125b 100644 --- a/commons/k8s/fetch.go +++ b/commons/k8s/fetch.go @@ -50,9 +50,31 @@ import ( dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" ) +func FetchResource(kubeClient client.Client, namespace string, name string, object client.Object) error { + namespacedName := types.NamespacedName{ + Namespace: namespace, + Name: name, + } + if err := kubeClient.Get(context.TODO(), namespacedName, object); err != nil { + return err + } + + return nil +} + +func FetchAutonomousDatabase(kubeClient client.Client, namespace string, name string) (*dbv1alpha1.AutonomousDatabase, error) { + adb := &dbv1alpha1.AutonomousDatabase{} + + if err := FetchResource(kubeClient, namespace, name, adb); err != nil { + return nil, err + } + + return adb, nil +} + // Returns the first AutonomousDatabase resource that matches the AutonomousDatabaseOCID of the backup // If the AutonomousDatabase doesn't exist, returns a nil -func FetchAutonomousDatabase(kubeClient client.Client, namespace string, ocid string) (*dbv1alpha1.AutonomousDatabase, error) { +func FetchAutonomousDatabaseWithOCID(kubeClient client.Client, namespace string, ocid string) (*dbv1alpha1.AutonomousDatabase, error) { adbList, err := fetchAutonomousDatabases(kubeClient, namespace) if err != nil { return nil, err @@ -98,12 +120,9 @@ func FetchAutonomousDatabaseBackups(kubeClient client.Client, namespace string) } func FetchConfigMap(kubeClient client.Client, namespace string, name string) (*corev1.ConfigMap, error) { - namespacedName := types.NamespacedName{ - Namespace: namespace, - Name: name, - } configMap := &corev1.ConfigMap{} - if err := kubeClient.Get(context.TODO(), namespacedName, configMap); err != nil { + + if err := FetchResource(kubeClient, namespace, name, configMap); err != nil { return nil, err } @@ -111,12 +130,9 @@ func FetchConfigMap(kubeClient client.Client, namespace string, name string) (*c } func FetchSecret(kubeClient client.Client, namespace string, name string) (*corev1.Secret, error) { - namespacedName := types.NamespacedName{ - Namespace: namespace, - Name: name, - } secret := &corev1.Secret{} - if err := kubeClient.Get(context.TODO(), namespacedName, secret); err != nil { + + if err := FetchResource(kubeClient, namespace, name, secret); err != nil { return nil, err } diff --git a/commons/k8s/finalizer.go b/commons/k8s/finalizer.go index 34928a32..f73947be 100644 --- a/commons/k8s/finalizer.go +++ b/commons/k8s/finalizer.go @@ -46,27 +46,25 @@ import ( "sigs.k8s.io/controller-runtime/pkg/client" ) -// name of our custom finalizer -var finalizerName = "database.oracle.com/oraoperator-finalizer" // HasFinalizer returns true if the finalizer exists in the object metadata -func HasFinalizer(obj client.Object) bool { - finalizer := obj.GetFinalizers() - return containsString(finalizer, finalizerName) +func HasFinalizer(obj client.Object, finalizer string) bool { + finalizers := obj.GetFinalizers() + return containsString(finalizers, finalizer) } // Register adds the finalizer and patch the object -func Register(kubeClient client.Client, obj client.Object) error { - finalizer := obj.GetFinalizers() - finalizer = append(finalizer, finalizerName) - return setFinalizer(kubeClient, obj, finalizer) +func Register(kubeClient client.Client, obj client.Object, finalizer string) error { + finalizers := obj.GetFinalizers() + finalizers = append(finalizers, finalizer) + return setFinalizer(kubeClient, obj, finalizers) } // Unregister removes the finalizer and patch the object -func Unregister(kubeClient client.Client, obj client.Object) error { - finalizer := obj.GetFinalizers() - finalizer = removeString(finalizer, finalizerName) - return setFinalizer(kubeClient, obj, finalizer) +func Unregister(kubeClient client.Client, obj client.Object, finalizer string) error { + finalizers := obj.GetFinalizers() + finalizers = removeString(finalizers, finalizer) + return setFinalizer(kubeClient, obj, finalizers) } // Helper functions to check and remove string from a slice of strings. @@ -95,10 +93,10 @@ type patchValue struct { Value interface{} `json:"value"` } -func setFinalizer(kubeClient client.Client, dbcs client.Object, finalizer []string) error { +func setFinalizer(kubeClient client.Client, obj client.Object, finalizer []string) error { payload := []patchValue{} - if dbcs.GetFinalizers() == nil { + if obj.GetFinalizers() == nil { payload = append(payload, patchValue{ Op: "replace", Path: "/metadata/finalizers", @@ -118,5 +116,5 @@ func setFinalizer(kubeClient client.Client, dbcs client.Object, finalizer []stri } patch := client.RawPatch(types.JSONPatchType, payloadBytes) - return kubeClient.Patch(context.TODO(), dbcs, patch) + return kubeClient.Patch(context.TODO(), obj, patch) } diff --git a/commons/oci/database.go b/commons/oci/database.go index caf9faa8..4a092197 100644 --- a/commons/oci/database.go +++ b/commons/oci/database.go @@ -61,7 +61,7 @@ type DatabaseService interface { UpdateAutonomousDatabaseScalingFields(adb *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) UpdateNetworkAccessMTLSRequired(adbOCID string) (resp database.UpdateAutonomousDatabaseResponse, err error) UpdateNetworkAccessMTLS(adb *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) - UpdateNetworkAccessPublic(lastSucSpec *dbv1alpha1.AutonomousDatabaseSpec, adbOCID string) (resp database.UpdateAutonomousDatabaseResponse, err error) + UpdateNetworkAccessPublic(lastSucSpec dbv1alpha1.NetworkAccessTypeEnum, adbOCID string) (resp database.UpdateAutonomousDatabaseResponse, err error) UpdateNetworkAccess(adb *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) StartAutonomousDatabase(adbOCID string) (database.StartAutonomousDatabaseResponse, error) StopAutonomousDatabase(adbOCID string) (database.StopAutonomousDatabaseResponse, error) @@ -69,8 +69,11 @@ type DatabaseService interface { DownloadWallet(adb *dbv1alpha1.AutonomousDatabase) (database.GenerateAutonomousDatabaseWalletResponse, error) RestoreAutonomousDatabase(adbOCID string, sdkTime common.SDKTime) (database.RestoreAutonomousDatabaseResponse, error) ListAutonomousDatabaseBackups(adbOCID string) (database.ListAutonomousDatabaseBackupsResponse, error) - CreateAutonomousDatabaseBackup(adbBackup *dbv1alpha1.AutonomousDatabaseBackup) (database.CreateAutonomousDatabaseBackupResponse, error) + CreateAutonomousDatabaseBackup(adbBackup *dbv1alpha1.AutonomousDatabaseBackup, adbOCID string) (database.CreateAutonomousDatabaseBackupResponse, error) GetAutonomousDatabaseBackup(backupOCID string) (database.GetAutonomousDatabaseBackupResponse, error) + CreateAutonomousContainerDatabase(acb *dbv1alpha1.AutonomousContainerDatabase) (database.CreateAutonomousContainerDatabaseResponse, error) + GetAutonomousContainerDatabase(acdOCID string) (database.GetAutonomousContainerDatabaseResponse, error) + UpdateAutonomousContainerDatabase(acd *dbv1alpha1.AutonomousContainerDatabase) (database.UpdateAutonomousContainerDatabaseResponse, error) } type databaseService struct { @@ -103,6 +106,10 @@ func NewDatabaseService( }, nil } +/******************************** + * Autonomous Database + *******************************/ + // ReadPassword reads the password from passwordSpec, and returns the pointer to the read password string. // The function returns a nil if nothing is read func (d *databaseService) readPassword(namespace string, passwordSpec dbv1alpha1.PasswordSpec) (*string, error) { @@ -264,13 +271,13 @@ func (d *databaseService) UpdateNetworkAccessMTLS(adb *dbv1alpha1.AutonomousData return d.dbClient.UpdateAutonomousDatabase(context.TODO(), updateAutonomousDatabaseRequest) } -func (d *databaseService) UpdateNetworkAccessPublic(lastSucSpec *dbv1alpha1.AutonomousDatabaseSpec, +func (d *databaseService) UpdateNetworkAccessPublic(lastAccessType dbv1alpha1.NetworkAccessTypeEnum, adbOCID string) (resp database.UpdateAutonomousDatabaseResponse, err error) { updateAutonomousDatabaseDetails := database.UpdateAutonomousDatabaseDetails{} - if lastSucSpec.Details.NetworkAccess.AccessType == dbv1alpha1.NetworkAccessTypeRestricted { + if lastAccessType == dbv1alpha1.NetworkAccessTypeRestricted { updateAutonomousDatabaseDetails.WhitelistedIps = []string{""} - } else if lastSucSpec.Details.NetworkAccess.AccessType == dbv1alpha1.NetworkAccessTypePrivate { + } else if lastAccessType == dbv1alpha1.NetworkAccessTypePrivate { updateAutonomousDatabaseDetails.PrivateEndpointLabel = common.String("") } @@ -345,6 +352,10 @@ func (d *databaseService) DownloadWallet(adb *dbv1alpha1.AutonomousDatabase) (re return resp, nil } +/******************************** + * Autonomous Database Restore + *******************************/ + func (d *databaseService) RestoreAutonomousDatabase(adbOCID string, sdkTime common.SDKTime) (database.RestoreAutonomousDatabaseResponse, error) { request := database.RestoreAutonomousDatabaseRequest{ AutonomousDatabaseId: common.String(adbOCID), @@ -355,6 +366,10 @@ func (d *databaseService) RestoreAutonomousDatabase(adbOCID string, sdkTime comm return d.dbClient.RestoreAutonomousDatabase(context.TODO(), request) } +/******************************** + * Autonomous Database Backup + *******************************/ + func (d *databaseService) ListAutonomousDatabaseBackups(adbOCID string) (database.ListAutonomousDatabaseBackupsResponse, error) { listBackupRequest := database.ListAutonomousDatabaseBackupsRequest{ AutonomousDatabaseId: common.String(adbOCID), @@ -363,10 +378,10 @@ func (d *databaseService) ListAutonomousDatabaseBackups(adbOCID string) (databas return d.dbClient.ListAutonomousDatabaseBackups(context.TODO(), listBackupRequest) } -func (d *databaseService) CreateAutonomousDatabaseBackup(adbBackup *dbv1alpha1.AutonomousDatabaseBackup) (database.CreateAutonomousDatabaseBackupResponse, error) { +func (d *databaseService) CreateAutonomousDatabaseBackup(adbBackup *dbv1alpha1.AutonomousDatabaseBackup, adbOCID string) (database.CreateAutonomousDatabaseBackupResponse, error) { createBackupRequest := database.CreateAutonomousDatabaseBackupRequest{ CreateAutonomousDatabaseBackupDetails: database.CreateAutonomousDatabaseBackupDetails{ - AutonomousDatabaseId: &adbBackup.Spec.AutonomousDatabaseOCID, + AutonomousDatabaseId: common.String(adbOCID), }, } diff --git a/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml b/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml index 998ed12f..c714fe46 100644 --- a/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml +++ b/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml @@ -59,10 +59,6 @@ spec: properties: autonomousDatabaseBackupOCID: type: string - autonomousDatabaseOCID: - description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster - Important: Run "make" to regenerate code after modifying this file' - type: string displayName: type: string ociConfig: @@ -72,14 +68,28 @@ spec: secretName: type: string type: object + targetADB: + description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster + Important: Run "make" to regenerate code after modifying this file' + properties: + name: + type: string + ocid: + type: string + required: + - name + - ocid + type: object required: - - autonomousDatabaseOCID - displayName + - targetADB type: object status: description: AutonomousDatabaseBackupStatus defines the observed state of AutonomousDatabaseBackup properties: + autonomousDatabaseOCID: + type: string compartmentOCID: type: string dbDisplayName: @@ -102,6 +112,7 @@ spec: type: string' type: string required: + - autonomousDatabaseOCID - compartmentOCID - dbDisplayName - dbName diff --git a/config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml b/config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml index b7508fc8..055bc148 100644 --- a/config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml +++ b/config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml @@ -51,10 +51,6 @@ spec: description: AutonomousDatabaseRestoreSpec defines the desired state of AutonomousDatabaseRestore properties: - backupName: - description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster - Important: Run "make" to regenerate code after modifying this file' - type: string ociConfig: properties: configMapName: @@ -62,13 +58,31 @@ spec: secretName: type: string type: object - pointInTime: + sourceSpec: properties: - autonomousDatabaseOCID: + autonomousDatabaseBackup: + properties: + name: + type: string + type: object + pointInTime: type: string - timeStamp: + type: object + targetADB: + description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster + Important: Run "make" to regenerate code after modifying this file' + properties: + name: type: string + ocid: + type: string + required: + - name + - ocid type: object + required: + - sourceSpec + - targetADB type: object status: description: AutonomousDatabaseRestoreStatus defines the observed state diff --git a/config/crd/bases/database.oracle.com_autonomousdatabases.yaml b/config/crd/bases/database.oracle.com_autonomousdatabases.yaml index 8e13d12b..168acd67 100644 --- a/config/crd/bases/database.oracle.com_autonomousdatabases.yaml +++ b/config/crd/bases/database.oracle.com_autonomousdatabases.yaml @@ -194,78 +194,6 @@ spec: - connectionStrings type: object type: array - conditions: - items: - description: "Condition contains details for one aspect of the current - state of this API Resource. --- This struct is intended for direct - use as an array at the field path .status.conditions. For example, - type FooStatus struct{ // Represents the observations of a - foo's current state. // Known .status.conditions.type are: - \"Available\", \"Progressing\", and \"Degraded\" // +patchMergeKey=type - \ // +patchStrategy=merge // +listType=map // +listMapKey=type - \ Conditions []metav1.Condition `json:\"conditions,omitempty\" - patchStrategy:\"merge\" patchMergeKey:\"type\" protobuf:\"bytes,1,rep,name=conditions\"` - \n // other fields }" - properties: - lastTransitionTime: - description: lastTransitionTime is the last time the condition - transitioned from one status to another. This should be when - the underlying condition changed. If that is not known, then - using the time when the API field changed is acceptable. - format: date-time - type: string - message: - description: message is a human readable message indicating - details about the transition. This may be an empty string. - maxLength: 32768 - type: string - observedGeneration: - description: observedGeneration represents the .metadata.generation - that the condition was set based upon. For instance, if .metadata.generation - is currently 12, but the .status.conditions[x].observedGeneration - is 9, the condition is out of date with respect to the current - state of the instance. - format: int64 - minimum: 0 - type: integer - reason: - description: reason contains a programmatic identifier indicating - the reason for the condition's last transition. Producers - of specific condition types may define expected values and - meanings for this field, and whether the values are considered - a guaranteed API. The value should be a CamelCase string. - This field may not be empty. - maxLength: 1024 - minLength: 1 - pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ - type: string - status: - description: status of the condition, one of True, False, Unknown. - enum: - - "True" - - "False" - - Unknown - type: string - type: - description: type of condition in CamelCase or in foo.example.com/CamelCase. - --- Many .condition.type values are consistent across resources - like Available, but because arbitrary conditions can be useful - (see .node.status.conditions), the ability to deconflict is - important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) - maxLength: 316 - pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ - type: string - required: - - lastTransitionTime - - message - - reason - - status - - type - type: object - type: array - x-kubernetes-list-map-keys: - - type - x-kubernetes-list-type: map lifecycleState: description: 'INSERT ADDITIONAL STATUS FIELD - define observed state of cluster Important: Run "make" to regenerate code after modifying diff --git a/controllers/database/autonomousdatabase_controller.go b/controllers/database/autonomousdatabase_controller.go index 6716e2a2..e1013068 100644 --- a/controllers/database/autonomousdatabase_controller.go +++ b/controllers/database/autonomousdatabase_controller.go @@ -67,6 +67,10 @@ import ( "github.com/oracle/oracle-database-operator/commons/oci" ) +// name of our custom finalizer +const finalizer = "database.oracle.com/oraoperator-finalizer" +var emptyResult ctrl.Result = ctrl.Result{} + // *AutonomousDatabaseReconciler reconciles a AutonomousDatabase object type AutonomousDatabaseReconciler struct { KubeClient client.Client @@ -79,8 +83,6 @@ type AutonomousDatabaseReconciler struct { lastSucSpec *dbv1alpha1.AutonomousDatabaseSpec } -var emptyResult ctrl.Result = ctrl.Result{} - // SetupWithManager function func (r *AutonomousDatabaseReconciler) SetupWithManager(mgr ctrl.Manager) error { return ctrl.NewControllerManagedBy(mgr). @@ -213,11 +215,6 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R if err := r.downloadWallet(adb); err != nil { return r.manageError(adb, err) } - - err = r.syncResource(adb) - if err != nil { - return r.manageError(adb, err) - } case adbActionBind: fallthrough case adbActionSync: @@ -247,6 +244,13 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R return r.manageError(adb, err) } + /***************************************************** + * Update the lastSucSpec + *****************************************************/ + if err := r.updateLastSuccessfulSpec(adb); err != nil { + return r.manageError(adb, err) + } + logger.Info("AutonomousDatabase reconciles successfully") return emptyResult, nil @@ -283,19 +287,20 @@ func (r *AutonomousDatabaseReconciler) manageError(adb *dbv1alpha1.AutonomousDat // Rollback if lastSucSpec exists if r.lastSucSpec != nil { // Send event - r.Recorder.Event(adb, corev1.EventTypeWarning, "SpecRollback", issue.Error()) + r.Recorder.Event(adb, corev1.EventTypeWarning, "ReconcileIncompleted", issue.Error()) - adb.Spec = *r.lastSucSpec + var finalIssue = issue - if err := r.updateResource(adb); err != nil { - // returns the error because we didn't trigger another Reconcile loop - return emptyResult, k8s.CombineErrors(issue, err) + if err := r.syncResource(adb); err != nil { + finalIssue = k8s.CombineErrors(finalIssue, err) } - r.Log.Error(issue, "Reconcile failed") + if err := r.updateLastSuccessfulSpec(adb); err != nil { + finalIssue = k8s.CombineErrors(finalIssue, err) + } - // r.updateResource has already triggered another Reconcile loop, so we return a nil instead of an error - return emptyResult, nil + // r.updateResource has already triggered another Reconcile loop, so we simply log the error and return a nil + return emptyResult, finalIssue } else { // Send event r.Recorder.Event(adb, corev1.EventTypeWarning, "CreateFailed", issue.Error()) @@ -309,14 +314,14 @@ func (r *AutonomousDatabaseReconciler) validateFinalizer(adb *dbv1alpha1.Autonom if adb.ObjectMeta.DeletionTimestamp.IsZero() { // The object is not being deleted - if *adb.Spec.HardLink && !k8s.HasFinalizer(adb) { - if err := k8s.Register(r.KubeClient, adb); err != nil { + if *adb.Spec.HardLink && !k8s.HasFinalizer(adb, finalizer) { + if err := k8s.Register(r.KubeClient, adb, finalizer); err != nil { return false, err } logger.Info("Finalizer registered successfully.") - } else if !*adb.Spec.HardLink && k8s.HasFinalizer(adb) { - if err := k8s.Unregister(r.KubeClient, adb); err != nil { + } else if !*adb.Spec.HardLink && !k8s.HasFinalizer(adb, finalizer) { + if err := k8s.Unregister(r.KubeClient, adb, finalizer); err != nil { return false, err } logger.Info("Finalizer unregistered successfully.") @@ -336,7 +341,9 @@ func (r *AutonomousDatabaseReconciler) validateFinalizer(adb *dbv1alpha1.Autonom } } - k8s.Unregister(r.KubeClient, adb) + if err := k8s.Unregister(r.KubeClient, adb, finalizer); err != nil { + return true, err + } logger.Info("Finalizer unregistered successfully.") // Stop reconciliation as the item is being deleted return true, nil @@ -356,15 +363,15 @@ func (r *AutonomousDatabaseReconciler) updateLastSuccessfulSpec(adb *dbv1alpha1. return annotations.SetAnnotations(r.KubeClient, adb, anns) } -// helper function to update the resource, and then wait until the work completes after an update/start/stop/delete AutonomousDatabase request is sent -func (r *AutonomousDatabaseReconciler) updateResourceAndWait(adb *dbv1alpha1.AutonomousDatabase, opcWorkRequestId string) error { - // updateResource updates the lastSucSpec as well - if err := r.updateResource(adb); err != nil { +// A helper function to wait until the work completes after an update/start/stop/delete AutonomousDatabase request is sent +// , and then sync the resource to make sure everything is updated when the work is finished +func (r *AutonomousDatabaseReconciler) wait(adb *dbv1alpha1.AutonomousDatabase, opcWorkRequestId string) error { + // Wait until the work is finished + if _, err := r.workService.Wait(opcWorkRequestId); err != nil { return err } - // Wait until the work is finished - if _, err := r.workService.Wait(opcWorkRequestId); err != nil { + if err := r.syncResource(adb); err != nil { return err } @@ -397,10 +404,6 @@ func (r *AutonomousDatabaseReconciler) updateResource(adb *dbv1alpha1.Autonomous return err } - if err := r.updateLastSuccessfulSpec(adb); err != nil { - return err - } - return nil } @@ -466,9 +469,14 @@ func (r *AutonomousDatabaseReconciler) createADB(adb *dbv1alpha1.AutonomousDatab return err } - adb.UpdateFromOCIADB(resp.AutonomousDatabase) + // Update the ADB OCID and the status + adb.Spec.Details.AutonomousDatabaseOCID = resp.AutonomousDatabase.Id + adb.UpdateStatusFromOCIADB(resp.AutonomousDatabase) + if err := r.updateResource(adb); err != nil { + return err + } - return r.updateResourceAndWait(adb, *resp.OpcWorkRequestId) + return r.wait(adb, *resp.OpcWorkRequestId) } func (r *AutonomousDatabaseReconciler) updateGeneralFields(adb *dbv1alpha1.AutonomousDatabase, difADB *dbv1alpha1.AutonomousDatabase) error { @@ -490,7 +498,12 @@ func (r *AutonomousDatabaseReconciler) updateGeneralFields(adb *dbv1alpha1.Auton return nil } - return r.updateResourceAndWait(adb, *resp.OpcWorkRequestId) + adb.UpdateStatusFromOCIADB(resp.AutonomousDatabase) + if err := r.updateResourceStatus(adb); err != nil { + return err + } + + return r.wait(adb, *resp.OpcWorkRequestId) } func (r *AutonomousDatabaseReconciler) updateAdminPassword(adb *dbv1alpha1.AutonomousDatabase, difADB *dbv1alpha1.AutonomousDatabase) error { @@ -499,14 +512,13 @@ func (r *AutonomousDatabaseReconciler) updateAdminPassword(adb *dbv1alpha1.Auton return nil } - resp, err := r.adbService.UpdateAutonomousDatabaseAdminPassword(difADB) + _, err := r.adbService.UpdateAutonomousDatabaseAdminPassword(difADB) if err != nil { return err } - adb.UpdateFromOCIADB(resp.AutonomousDatabase) - - return r.updateResourceAndWait(adb, *resp.OpcWorkRequestId) + // UpdateAdminPassword request doesn't return the workrequest ID + return nil } func (r *AutonomousDatabaseReconciler) updateDbWorkload(adb *dbv1alpha1.AutonomousDatabase, difADB *dbv1alpha1.AutonomousDatabase) error { @@ -519,9 +531,12 @@ func (r *AutonomousDatabaseReconciler) updateDbWorkload(adb *dbv1alpha1.Autonomo return err } - adb.UpdateFromOCIADB(resp.AutonomousDatabase) + adb.UpdateStatusFromOCIADB(resp.AutonomousDatabase) + if err := r.updateResourceStatus(adb); err != nil { + return err + } - return r.updateResourceAndWait(adb, *resp.OpcWorkRequestId) + return r.wait(adb, *resp.OpcWorkRequestId) } func (r *AutonomousDatabaseReconciler) updateLicenseModel(adb *dbv1alpha1.AutonomousDatabase, difADB *dbv1alpha1.AutonomousDatabase) error { @@ -534,9 +549,12 @@ func (r *AutonomousDatabaseReconciler) updateLicenseModel(adb *dbv1alpha1.Autono return err } - adb.UpdateFromOCIADB(resp.AutonomousDatabase) + adb.UpdateStatusFromOCIADB(resp.AutonomousDatabase) + if err := r.updateResourceStatus(adb); err != nil { + return err + } - return r.updateResourceAndWait(adb, *resp.OpcWorkRequestId) + return r.wait(adb, *resp.OpcWorkRequestId) } func (r *AutonomousDatabaseReconciler) updateScalingFields(adb *dbv1alpha1.AutonomousDatabase, difADB *dbv1alpha1.AutonomousDatabase) error { @@ -551,9 +569,12 @@ func (r *AutonomousDatabaseReconciler) updateScalingFields(adb *dbv1alpha1.Auton return err } - adb.UpdateFromOCIADB(resp.AutonomousDatabase) + adb.UpdateStatusFromOCIADB(resp.AutonomousDatabase) + if err := r.updateResourceStatus(adb); err != nil { + return err + } - return r.updateResourceAndWait(adb, *resp.OpcWorkRequestId) + return r.wait(adb, *resp.OpcWorkRequestId) } func (r *AutonomousDatabaseReconciler) deleteAutonomousDatabase(adb *dbv1alpha1.AutonomousDatabase) error { @@ -564,18 +585,12 @@ func (r *AutonomousDatabaseReconciler) deleteAutonomousDatabase(adb *dbv1alpha1. } adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateTerminating - opcWorkRequestId := *resp.OpcWorkRequestId if err := r.updateResourceStatus(adb); err != nil { return err } - // Wait until the work is finished - if _, err := r.workService.Wait(opcWorkRequestId); err != nil { - return err - } - - return nil + return r.wait(adb, *resp.OpcWorkRequestId) } func (r *AutonomousDatabaseReconciler) updateLifecycleState(adb *dbv1alpha1.AutonomousDatabase, difADB *dbv1alpha1.AutonomousDatabase) error { @@ -614,11 +629,11 @@ func (r *AutonomousDatabaseReconciler) updateLifecycleState(adb *dbv1alpha1.Auto return errors.New("Unknown state") } - if err := r.updateResourceAndWait(adb, opcWorkRequestId); err != nil { + if err := r.updateResourceStatus(adb); err != nil { return err } - return nil + return r.wait(adb, opcWorkRequestId) } func (r *AutonomousDatabaseReconciler) setMTLSRequired(adb *dbv1alpha1.AutonomousDatabase) error { @@ -627,9 +642,12 @@ func (r *AutonomousDatabaseReconciler) setMTLSRequired(adb *dbv1alpha1.Autonomou return err } - adb.UpdateFromOCIADB(resp.AutonomousDatabase) + adb.UpdateStatusFromOCIADB(resp.AutonomousDatabase) + if err := r.updateResourceStatus(adb); err != nil { + return err + } - return r.updateResourceAndWait(adb, *resp.OpcWorkRequestId) + return r.wait(adb, *resp.OpcWorkRequestId) } func (r *AutonomousDatabaseReconciler) updateMTLS(adb *dbv1alpha1.AutonomousDatabase, difADB *dbv1alpha1.AutonomousDatabase) error { @@ -642,20 +660,26 @@ func (r *AutonomousDatabaseReconciler) updateMTLS(adb *dbv1alpha1.AutonomousData return err } - adb.UpdateFromOCIADB(resp.AutonomousDatabase) + adb.UpdateStatusFromOCIADB(resp.AutonomousDatabase) + if err := r.updateResourceStatus(adb); err != nil { + return err + } - return r.updateResourceAndWait(adb, *resp.OpcWorkRequestId) + return r.wait(adb, *resp.OpcWorkRequestId) } func (r *AutonomousDatabaseReconciler) setNetworkAccessPublic(adb *dbv1alpha1.AutonomousDatabase) error { - resp, err := r.adbService.UpdateNetworkAccessPublic(r.lastSucSpec, *adb.Spec.Details.AutonomousDatabaseOCID) + resp, err := r.adbService.UpdateNetworkAccessPublic(r.lastSucSpec.Details.NetworkAccess.AccessType, *adb.Spec.Details.AutonomousDatabaseOCID) if err != nil { return err } - adb.UpdateFromOCIADB(resp.AutonomousDatabase) + adb.UpdateStatusFromOCIADB(resp.AutonomousDatabase) + if err := r.updateResourceStatus(adb); err != nil { + return err + } - return r.updateResourceAndWait(adb, *resp.OpcWorkRequestId) + return r.wait(adb, *resp.OpcWorkRequestId) } func (r *AutonomousDatabaseReconciler) updateNetworkAccess(adb *dbv1alpha1.AutonomousDatabase, difADB *dbv1alpha1.AutonomousDatabase) error { @@ -672,9 +696,12 @@ func (r *AutonomousDatabaseReconciler) updateNetworkAccess(adb *dbv1alpha1.Auton return err } - adb.UpdateFromOCIADB(resp.AutonomousDatabase) + adb.UpdateStatusFromOCIADB(resp.AutonomousDatabase) + if err := r.updateResourceStatus(adb); err != nil { + return err + } - return r.updateResourceAndWait(adb, *resp.OpcWorkRequestId) + return r.wait(adb, *resp.OpcWorkRequestId) } // The logic of updating the network access configurations is as follows: diff --git a/controllers/database/autonomousdatabasebackup_controller.go b/controllers/database/autonomousdatabasebackup_controller.go index 2ffdc9be..44bb8831 100644 --- a/controllers/database/autonomousdatabasebackup_controller.go +++ b/controllers/database/autonomousdatabasebackup_controller.go @@ -100,6 +100,7 @@ func (r *AutonomousDatabaseBackupReconciler) eventFilterPredicate() predicate.Pr //+kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabasebackups,verbs=get;list;watch;create;delete //+kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabasebackups/status,verbs=get;update;patch +//+kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabases,verbs=get;list func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { logger := r.Log.WithValues("Namespaced/Name", req.NamespacedName) @@ -139,7 +140,7 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req ******************************************************************/ if backup.Spec.AutonomousDatabaseBackupOCID == "" && backup.Status.LifecycleState == "" { // Create a new backup - backupResp, err := r.adbService.CreateAutonomousDatabaseBackup(backup) + backupResp, err := r.adbService.CreateAutonomousDatabaseBackup(backup, *r.ownerADB.Spec.Details.AutonomousDatabaseOCID) if err != nil { return r.manageError(backup, err) } @@ -192,17 +193,24 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req } func (r *AutonomousDatabaseBackupReconciler) verifyOwnerADB(backup *dbv1alpha1.AutonomousDatabaseBackup) error { - if len(backup.GetOwnerReferences()) == 0 && backup.Spec.AutonomousDatabaseOCID != "" { + if len(backup.GetOwnerReferences()) == 0 { var err error - r.ownerADB, err = k8s.FetchAutonomousDatabase(r.KubeClient, backup.Namespace, backup.Spec.AutonomousDatabaseOCID) - if err != nil { - return err - } - if r.ownerADB == nil { - if err := r.setOwnerAutonomousDatabase(backup, r.ownerADB); err != nil { + + if backup.Spec.TargetADB.Name != "" { + r.ownerADB, err = k8s.FetchAutonomousDatabase(r.KubeClient, backup.Namespace, backup.Spec.TargetADB.Name) + if err != nil { + return err + } + } else { + r.ownerADB, err = k8s.FetchAutonomousDatabaseWithOCID(r.KubeClient, backup.Namespace, backup.Spec.TargetADB.OCID) + if err != nil { return err } } + + if err := r.setOwnerAutonomousDatabase(backup, r.ownerADB); err != nil { + return err + } } return nil } @@ -289,9 +297,6 @@ func (r *AutonomousDatabaseBackupReconciler) updateResourceStatus(backup *dbv1al // No-op if the ownerADB is not assigned func (r *AutonomousDatabaseBackupReconciler) syncADBStatus() error { - if r.ownerADB == nil { - return nil - } resp, err := r.adbService.GetAutonomousDatabase(*r.ownerADB.Spec.Details.AutonomousDatabaseOCID) if err != nil { return err diff --git a/controllers/database/autonomousdatabaserestore_controller.go b/controllers/database/autonomousdatabaserestore_controller.go index 7e74126b..25d0bd8d 100644 --- a/controllers/database/autonomousdatabaserestore_controller.go +++ b/controllers/database/autonomousdatabaserestore_controller.go @@ -81,6 +81,7 @@ func (r *AutonomousDatabaseRestoreReconciler) SetupWithManager(mgr ctrl.Manager) //+kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabaserestores,verbs=get;list;watch;create;delete //+kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabaserestores/status,verbs=get;update;patch +//+kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabases,verbs=get;list // Reconcile is part of the main kubernetes reconciliation loop which aims to // move the current state of the cluster closer to the desired state. @@ -105,19 +106,33 @@ func (r *AutonomousDatabaseRestoreReconciler) Reconcile(ctx context.Context, req return emptyResult, err } + if restore.Status.Status != "" { + return emptyResult, nil + } + + // ===================== Run Restore for the Target ADB ============================ + + /****************************************************************** + * Look up the owner AutonomousDatabase and set the ownerReference + * if the owner hasn't been set yet. + ******************************************************************/ + if err := r.verifyOwnerADB(restore); err != nil { + return r.manageError(restore, err) + } + /****************************************************************** - * Extract the restoreTime and the adbOCID from the spec + * Extract the restoreTime from the spec ******************************************************************/ - restoreTime, adbOCID, err := r.fetchData(restore) + restoreTime, err := r.getRestoreSDKTime(restore) if err != nil { return r.manageError(restore, err) } /****************************************************************** - * Look up the owner AutonomousDatabase and set the ownerReference - * if the owner hasn't been set yet. + * Get the target ADB OCID ******************************************************************/ - if err := r.verifyOwnerADB(restore, adbOCID); err != nil { + adbOCID, err := r.getADBOCID(restore) + if err != nil { return r.manageError(restore, err) } @@ -142,31 +157,28 @@ func (r *AutonomousDatabaseRestoreReconciler) Reconcile(ctx context.Context, req return ctrl.Result{}, nil } -func (r *AutonomousDatabaseRestoreReconciler) fetchData(restore *dbv1alpha1.AutonomousDatabaseRestore) (restoreTime *common.SDKTime, adbOCID string, err error) { - if restore.Spec.BackupName != "" { // restore using backupName +func (r *AutonomousDatabaseRestoreReconciler) getRestoreSDKTime(restore *dbv1alpha1.AutonomousDatabaseRestore) (*common.SDKTime, error) { + if restore.Spec.Source.AutonomousDatabaseBackup.Name != "" { // restore using backupName backup := &dbv1alpha1.AutonomousDatabaseBackup{} - namespacedName := types.NamespacedName{Namespace: restore.Namespace, Name: restore.Spec.BackupName} - if err := r.KubeClient.Get(context.TODO(), namespacedName, backup); err != nil { - return restoreTime, adbOCID, err + if err := k8s.FetchResource(r.KubeClient, restore.Namespace, restore.Spec.Source.AutonomousDatabaseBackup.Name, backup); err != nil { + return nil, err } if backup.Status.TimeEnded == "" { - return restoreTime, adbOCID, errors.New("broken backup: ended time is missing in the AutonomousDatabaseBackup " + backup.GetName()) + return nil, errors.New("broken backup: ended time is missing from the AutonomousDatabaseBackup " + backup.GetName()) } - restoreTime, err = backup.GetTimeEnded() + restoreTime, err := backup.GetTimeEnded() if err != nil { - return restoreTime, adbOCID, err + return nil, err } - adbOCID = backup.Spec.AutonomousDatabaseOCID + return restoreTime, nil - } else if restore.Spec.PointInTime.TimeStamp != "" { // PIT restore + } else { // PIT restore // The validation of the pitr.timestamp has been handled by the webhook, so the error return is ignored - restoreTime, _ = restore.GetPIT() - adbOCID = restore.Spec.PointInTime.AutonomousDatabaseOCID + restoreTime, _ := restore.GetPIT() + return restoreTime, nil } - - return restoreTime, adbOCID, nil } // setOwnerAutonomousDatabase sets the owner of the AutonomousDatabaseBackup if the AutonomousDatabase resource with the same database OCID is found @@ -184,13 +196,20 @@ func (r *AutonomousDatabaseRestoreReconciler) setOwnerAutonomousDatabase(restore return nil } -func (r *AutonomousDatabaseRestoreReconciler) verifyOwnerADB(restore *dbv1alpha1.AutonomousDatabaseRestore, adbOCID string) error { +func (r *AutonomousDatabaseRestoreReconciler) verifyOwnerADB(restore *dbv1alpha1.AutonomousDatabaseRestore) error { if len(restore.GetOwnerReferences()) == 0 { var err error - r.ownerADB, err = k8s.FetchAutonomousDatabase(r.KubeClient, restore.Namespace, adbOCID) - if err != nil { - return err + if restore.Spec.TargetADB.Name != "" { + r.ownerADB, err = k8s.FetchAutonomousDatabase(r.KubeClient, restore.Namespace, restore.Spec.TargetADB.Name) + if err != nil { + return err + } + } else { + r.ownerADB, err = k8s.FetchAutonomousDatabaseWithOCID(r.KubeClient, restore.Namespace, restore.Spec.TargetADB.OCID) + if err != nil { + return err + } } if r.ownerADB == nil { @@ -203,35 +222,16 @@ func (r *AutonomousDatabaseRestoreReconciler) verifyOwnerADB(restore *dbv1alpha1 return nil } -func (r *AutonomousDatabaseRestoreReconciler) restoreAutonomousDatabase( - restore *dbv1alpha1.AutonomousDatabaseRestore, - restoreTime *common.SDKTime, - adbOCID string) error { - - resp, err := r.adbService.RestoreAutonomousDatabase(adbOCID, *restoreTime) - if err != nil { - return err - } - - // Update status and wait for the work finish if a request is sent. Note that some of the requests (e.g. update displayName) won't return a work request ID. - // It's important to update the status by reference otherwise Reconcile() won't be able to get the latest values - restore.Status.DisplayName = *resp.AutonomousDatabase.DisplayName - restore.Status.DbName = *resp.AutonomousDatabase.DbName - restore.Status.AutonomousDatabaseOCID = *resp.AutonomousDatabase.Id - restore.Status.Status = restore.ConvertWorkRequestStatus(workrequests.WorkRequestStatusEnum(resp.LifecycleState)) - - r.updateResourceStatus(restore) - - workStatus, err := r.workService.Wait(*resp.OpcWorkRequestId) - if err != nil { - return err +func (r *AutonomousDatabaseRestoreReconciler) getADBOCID(restore *dbv1alpha1.AutonomousDatabaseRestore) (string, error) { + if r.ownerADB != nil { + return *r.ownerADB.Spec.Details.AutonomousDatabaseOCID, nil + } else { + backup := &dbv1alpha1.AutonomousDatabaseBackup{} + if err := k8s.FetchResource(r.KubeClient, restore.Namespace, restore.Spec.Source.AutonomousDatabaseBackup.Name, backup); err != nil { + return "", err + } + return backup.Status.AutonomousDatabaseOCID, nil } - - // Update status when the work is finished - restore.Status.Status = restore.ConvertWorkRequestStatus(workStatus) - r.updateResourceStatus(restore) - - return nil } func (r *AutonomousDatabaseRestoreReconciler) updateResourceStatus(restore *dbv1alpha1.AutonomousDatabaseRestore) error { @@ -324,3 +324,34 @@ func (r *AutonomousDatabaseRestoreReconciler) manageError(restore *dbv1alpha1.Au return emptyResult, nil } + +func (r *AutonomousDatabaseRestoreReconciler) restoreAutonomousDatabase( + restore *dbv1alpha1.AutonomousDatabaseRestore, + restoreTime *common.SDKTime, + adbOCID string) error { + + resp, err := r.adbService.RestoreAutonomousDatabase(adbOCID, *restoreTime) + if err != nil { + return err + } + + // Update status and wait for the work finish if a request is sent. Note that some of the requests (e.g. update displayName) won't return a work request ID. + // It's important to update the status by reference otherwise Reconcile() won't be able to get the latest values + restore.Status.DisplayName = *resp.AutonomousDatabase.DisplayName + restore.Status.DbName = *resp.AutonomousDatabase.DbName + restore.Status.AutonomousDatabaseOCID = *resp.AutonomousDatabase.Id + restore.Status.Status = restore.ConvertWorkRequestStatus(workrequests.WorkRequestStatusEnum(resp.LifecycleState)) + + r.updateResourceStatus(restore) + + workStatus, err := r.workService.Wait(*resp.OpcWorkRequestId) + if err != nil { + return err + } + + // Update status when the work is finished + restore.Status.Status = restore.ConvertWorkRequestStatus(workStatus) + r.updateResourceStatus(restore) + + return nil +} \ No newline at end of file From da2052339b078adab2901ff633ddea59636bcd03 Mon Sep 17 00:00:00 2001 From: Susmita Samanta Date: Tue, 15 Mar 2022 10:01:00 +0530 Subject: [PATCH 187/628] adding fix for express image support --- .../crd/bases/database.oracle.com_singleinstancedatabases.yaml | 1 + 1 file changed, 1 insertion(+) diff --git a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml index 86aaded3..02666bfc 100644 --- a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml +++ b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml @@ -85,6 +85,7 @@ spec: enum: - standard - enterprise + - express type: string flashBack: type: boolean From e6f24a450d6d81481b1c6e0699a76e31c1928620 Mon Sep 17 00:00:00 2001 From: Susmita Samanta Date: Tue, 15 Mar 2022 12:54:57 +0530 Subject: [PATCH 188/628] adding support for XE image --- .../singleinstancedatabase_controller.go | 33 +++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 9fce2c0a..3aef1f9e 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -675,7 +675,40 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns Name: "datamount", }}, Env: func() []corev1.EnvVar { + // adding XE support if m.Spec.CloneFrom == "" { + if m.Spec.Edition == "express" { + return []corev1.EnvVar{ + { + Name: "SVC_HOST", + Value: m.Name, + }, + { + Name: "SVC_PORT", + Value: "1521", + }, + { + Name: "ORACLE_CHARACTERSET", + Value: m.Spec.Charset, + }, + { + Name: "ORACLE_EDITION", + Value: m.Spec.Edition, + }, + { + Name: "ORACLE_PWD", + ValueFrom: &corev1.EnvVarSource{ + SecretKeyRef: &corev1.SecretKeySelector{ + LocalObjectReference: corev1.LocalObjectReference{ + Name: m.Spec.AdminPassword.SecretName, + }, + Key: m.Spec.AdminPassword.SecretKey, + }, + }, + }, + } + } + return []corev1.EnvVar{ { Name: "SVC_HOST", From dd6ba9532fda96119598758c6cefe625e59ae051 Mon Sep 17 00:00:00 2001 From: Susmita Samanta Date: Tue, 15 Mar 2022 18:14:49 +0530 Subject: [PATCH 189/628] fix RunAsUser for XE DB --- .../singleinstancedatabase_controller.go | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 3aef1f9e..3d8bbb3f 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -517,17 +517,17 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }(), SecurityContext: &corev1.PodSecurityContext{ RunAsUser: func() *int64 { - i := int64(0) - if m.Spec.Edition != "express" { - i = int64(dbcommons.ORACLE_UID) - } + // i := int64(0) + // if m.Spec.Edition != "express" { + i := int64(dbcommons.ORACLE_UID) + // } return &i }(), RunAsGroup: func() *int64 { - i := int64(0) - if m.Spec.Edition != "express" { - i = int64(dbcommons.ORACLE_GUID) - } + // i := int64(0) + // if m.Spec.Edition != "express" { + i := int64(dbcommons.ORACLE_GUID) + // } return &i }(), }, From 45f1ddc6244c64742cedff454f56fde5c5a52a15 Mon Sep 17 00:00:00 2001 From: Susmita Samanta Date: Tue, 15 Mar 2022 18:39:39 +0530 Subject: [PATCH 190/628] fix Runasuser in XE db --- .../singleinstancedatabase_controller.go | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 3d8bbb3f..2c035ea0 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -831,17 +831,17 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns SecurityContext: &corev1.PodSecurityContext{ RunAsUser: func() *int64 { - i := int64(0) - if m.Spec.Edition != "express" { - i = int64(dbcommons.ORACLE_UID) - } + // i := int64(0) + // if m.Spec.Edition != "express" { + i := int64(dbcommons.ORACLE_UID) + // } return &i }(), RunAsGroup: func() *int64 { - i := int64(0) - if m.Spec.Edition != "express" { - i = int64(dbcommons.ORACLE_GUID) - } + // i := int64(0) + // if m.Spec.Edition != "express" { + i := int64(dbcommons.ORACLE_GUID) + // } return &i }(), }, From 25bca32ed1be3a3f0168b9c824ad07f997b355ce Mon Sep 17 00:00:00 2001 From: Susmita Samanta Date: Tue, 15 Mar 2022 20:31:55 +0530 Subject: [PATCH 191/628] Remove oracle user condition for XE db --- commons/database/utils.go | 3 --- controllers/database/oraclerestdataservice_controller.go | 4 ---- controllers/database/singleinstancedatabase_controller.go | 2 +- 3 files changed, 1 insertion(+), 8 deletions(-) diff --git a/commons/database/utils.go b/commons/database/utils.go index 211a92d0..1373fdbb 100644 --- a/commons/database/utils.go +++ b/commons/database/utils.go @@ -638,9 +638,6 @@ func GetSqlpatchStatus(r client.Reader, config *rest.Config, readyPod corev1.Pod } func GetSqlClient(edition string) string { - if edition == "express" { - return "su -p oracle -c \"sqlplus -s / as sysdba\"" - } return "sqlplus -s / as sysdba" } diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index a6aa198b..428f2032 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -1037,10 +1037,6 @@ func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataS r.Status().Update(ctx, m) configureApexRestSqlClient := "sqlplus -s / as sysdba @apex_rest_config.sql" - if n.Spec.Edition == "express" { - configureApexRestSqlClient = "su -p oracle -c \"sqlplus -s / as sysdba @apex_rest_config.sql;\"" - } - // Configure APEX out, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", fmt.Sprintf(dbcommons.ConfigureApexRest, apexPassword, configureApexRestSqlClient)) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 2c035ea0..5cac157a 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1494,7 +1494,7 @@ func (r *SingleInstanceDatabaseReconciler) validateDBReadiness(m *dbapi.SingleIn if m.Spec.Edition == "express" { //Configure OEM Express Listener out, err = dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, - "bash", "-c", fmt.Sprintf("echo -e \"%s\" | su -p oracle -c \"sqlplus -s / as sysdba\" ", dbcommons.ConfigureOEMSQL)) + "bash", "-c", fmt.Sprintf("echo -e \"%s\" | sqlplus -s / as sysdba ", dbcommons.ConfigureOEMSQL)) if err != nil { r.Log.Error(err, err.Error()) return requeueY, readyPod, err From e0dccacd6444dba355ec95df53b22f268bd10197 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Mon, 21 Mar 2022 11:19:42 -0400 Subject: [PATCH 192/628] add acd and improve reconciliation --- PROJECT | 9 + .../v1alpha1/adbfamily_common_utils.go | 33 +- .../autonomouscontainerdatabase_types.go | 169 +++++ .../v1alpha1/autonomousdatabase_types.go | 90 ++- .../v1alpha1/autonomousdatabase_webhook.go | 14 +- .../autonomousdatabasebackup_types.go | 32 +- .../autonomousdatabasebackup_webhook.go | 29 +- .../autonomousdatabaserestore_types.go | 47 +- .../autonomousdatabaserestore_webhook.go | 29 +- .../v1alpha1/shardingdatabase_types.go | 2 +- .../v1alpha1/zz_generated.deepcopy.go | 225 +++++- commons/adb_family/utils.go | 79 +++ commons/annotations/annotations.go | 4 +- commons/k8s/create.go | 9 +- commons/k8s/fetch.go | 12 +- commons/k8s/finalizer.go | 72 +- commons/k8s/utils.go | 34 +- commons/oci/containerdatabase.go | 93 +++ commons/oci/database.go | 39 +- commons/oci/workrequest.go | 18 +- ...acle.com_autonomouscontainerdatabases.yaml | 111 +++ ....oracle.com_autonomousdatabasebackups.yaml | 36 +- ...oracle.com_autonomousdatabaserestores.yaml | 39 +- ...tabase.oracle.com_autonomousdatabases.yaml | 51 +- config/crd/kustomization.yaml | 3 + ...ction_in_autonomouscontainerdatabases.yaml | 8 + ...bhook_in_autonomouscontainerdatabases.yaml | 17 + ...tonomouscontainerdatabase_editor_role.yaml | 24 + ...tonomouscontainerdatabase_viewer_role.yaml | 20 + config/rbac/role.yaml | 16 +- .../autonomouscontainerdatabase_create.yaml | 20 + config/samples/adb/autonomousdatabase.yaml | 11 - .../adb/autonomousdatabase_backup.yaml | 8 +- .../adb/autonomousdatabase_restore.yaml | 20 +- config/samples/kustomization.yaml | 2 +- .../autonomouscontainerdatabase_controller.go | 535 ++++++++++++++ .../database/autonomousdatabase_controller.go | 379 ++++++---- .../autonomousdatabasebackup_controller.go | 166 ++--- .../autonomousdatabaserestore_controller.go | 112 ++- main.go | 15 +- oracle-database-operator.yaml | 658 +++++++++--------- ...nomouscontainerdatabase_controller_test.go | 125 ++++ ...autonomousdatabase_controller_bind_test.go | 12 +- ...tonomousdatabase_controller_create_test.go | 34 +- test/e2e/suite_test.go | 27 + 45 files changed, 2581 insertions(+), 907 deletions(-) create mode 100644 apis/database/v1alpha1/autonomouscontainerdatabase_types.go create mode 100644 commons/adb_family/utils.go create mode 100644 commons/oci/containerdatabase.go create mode 100644 config/crd/bases/database.oracle.com_autonomouscontainerdatabases.yaml create mode 100644 config/crd/patches/cainjection_in_autonomouscontainerdatabases.yaml create mode 100644 config/crd/patches/webhook_in_autonomouscontainerdatabases.yaml create mode 100644 config/rbac/autonomouscontainerdatabase_editor_role.yaml create mode 100644 config/rbac/autonomouscontainerdatabase_viewer_role.yaml create mode 100644 config/samples/acd/autonomouscontainerdatabase_create.yaml delete mode 100644 config/samples/adb/autonomousdatabase.yaml create mode 100644 controllers/database/autonomouscontainerdatabase_controller.go create mode 100644 test/e2e/autonomouscontainerdatabase_controller_test.go diff --git a/PROJECT b/PROJECT index 3f15ac0f..3866b50c 100644 --- a/PROJECT +++ b/PROJECT @@ -102,4 +102,13 @@ resources: kind: OracleRestDataService path: github.com/oracle/oracle-database-operator/apis/database/v1alpha1 version: v1alpha1 +- api: + crdVersion: v1beta1 + namespaced: true + controller: true + domain: oracle.com + group: database + kind: AutonomousContainerDatabase + path: github.com/oracle/oracle-database-operator/apis/database/v1alpha1 + version: v1alpha1 version: "3" diff --git a/apis/database/v1alpha1/adbfamily_common_utils.go b/apis/database/v1alpha1/adbfamily_common_utils.go index 2abf9d93..65f7f64f 100644 --- a/apis/database/v1alpha1/adbfamily_common_utils.go +++ b/apis/database/v1alpha1/adbfamily_common_utils.go @@ -49,6 +49,9 @@ import ( "github.com/oracle/oci-go-sdk/v54/common" ) +// LastSuccessfulSpec is an annotation key which maps to the value of last successful spec +const LastSuccessfulSpec string = "lastSuccessfulSpec" + // File the meta condition and return the meta view func CreateMetaCondition(obj client.Object, err error, lifecycleState string, stateMsg string) metav1.Condition { @@ -62,21 +65,35 @@ func CreateMetaCondition(obj client.Object, err error, lifecycleState string, st } } -// LastSuccessfulSpec is an annotation key which maps to the value of last successful spec -const LastSuccessfulSpec string = "lastSuccessfulSpec" - +/************************ +* OCI config +************************/ type OCIConfigSpec struct { ConfigMapName *string `json:"configMapName,omitempty"` SecretName *string `json:"secretName,omitempty"` } +/************************ +* ADB spec +************************/ +type K8sADBSpec struct { + Name *string `json:"name,omitempty"` +} + +type OCIADBSpec struct { + OCID *string `json:"ocid,omitempty"` +} + // TargetSpec defines the spec of the target for backup/restore runs. -// The name could be the name of an AutonomousDatabase or an AutonomousDatabaseBackup type TargetSpec struct { - Name string `json:"name"` - OCID string `json:"ocid"` + K8sADB K8sADBSpec `json:"k8sADB,omitempty"` + OCIADB OCIADBSpec `json:"ociADB,omitempty"` } +/************************** +* Remove Unchanged Fields +**************************/ + // removeUnchangedFields removes the unchanged fields in the struct and returns if the struct is changed. // lastSpec should be a derefereced struct that is the last successful spec, e.g. AutonomousDatabaseSpec. // curSpec should be a pointer pointing to the struct that is being proccessed, e.g., *AutonomousDatabaseSpec. @@ -163,6 +180,10 @@ func hasChanged(lastField reflect.Value, curField reflect.Value) bool { return true } +/************************ +* SDKTime format +************************/ + // Follow the format of the display time const displayFormat = "2006-01-02 15:04:05 MST" diff --git a/apis/database/v1alpha1/autonomouscontainerdatabase_types.go b/apis/database/v1alpha1/autonomouscontainerdatabase_types.go new file mode 100644 index 00000000..0d9213b9 --- /dev/null +++ b/apis/database/v1alpha1/autonomouscontainerdatabase_types.go @@ -0,0 +1,169 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v1alpha1 + +import ( + "encoding/json" + "errors" + + "github.com/oracle/oci-go-sdk/v54/database" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! +// NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized. + +// name of our custom finalizer +const ACDFinalizer = "database.oracle.com/acd-finalizer" + +// AutonomousContainerDatabaseSpec defines the desired state of AutonomousContainerDatabase +type AutonomousContainerDatabaseSpec struct { + // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster + // Important: Run "make" to regenerate code after modifying this file + AutonomousContainerDatabaseOCID *string `json:"autonomousContainerDatabaseOCID,omitempty"` + CompartmentOCID *string `json:"compartmentOCID,omitempty"` + DisplayName *string `json:"displayName,omitempty"` + AutonomousExadataVMClusterOCID *string `json:"autonomousExadataVMClusterOCID,omitempty"` + // +kubebuilder:validation:Enum:="RELEASE_UPDATES";"RELEASE_UPDATE_REVISIONS" + PatchModel database.AutonomousContainerDatabasePatchModelEnum `json:"patchModel,omitempty"` + FreeformTags map[string]string `json:"freeformTags,omitempty"` + + OCIConfig OCIConfigSpec `json:"ociConfig,omitempty"` + // +kubebuilder:default:=false + HardLink *bool `json:"hardLink,omitempty"` +} + +// AutonomousContainerDatabaseStatus defines the observed state of AutonomousContainerDatabase +type AutonomousContainerDatabaseStatus struct { + // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster + // Important: Run "make" to regenerate code after modifying this file + LifecycleState database.AutonomousContainerDatabaseLifecycleStateEnum `json:"lifecycleState"` + TimeCreated string `json:"timeCreated,omitempty"` +} + +//+kubebuilder:object:root=true +//+kubebuilder:subresource:status +// +kubebuilder:resource:shortName="acd";"acds" +// +kubebuilder:printcolumn:JSONPath=".spec.displayName",name="DisplayName",type=string +// +kubebuilder:printcolumn:JSONPath=".status.lifecycleState",name="State",type=string +// +kubebuilder:printcolumn:JSONPath=".status.timeCreated",name="Created",type=string + +// AutonomousContainerDatabase is the Schema for the autonomouscontainerdatabases API +type AutonomousContainerDatabase struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + Spec AutonomousContainerDatabaseSpec `json:"spec,omitempty"` + Status AutonomousContainerDatabaseStatus `json:"status,omitempty"` +} + +//+kubebuilder:object:root=true + +// AutonomousContainerDatabaseList contains a list of AutonomousContainerDatabase +type AutonomousContainerDatabaseList struct { + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata,omitempty"` + Items []AutonomousContainerDatabase `json:"items"` +} + +func init() { + SchemeBuilder.Register(&AutonomousContainerDatabase{}, &AutonomousContainerDatabaseList{}) +} + +// GetLastSuccessfulSpec returns spec from the lass successful reconciliation. +// Returns nil, nil if there is no lastSuccessfulSpec. +func (acd *AutonomousContainerDatabase) GetLastSuccessfulSpec() (*AutonomousContainerDatabaseSpec, error) { + val, ok := acd.GetAnnotations()[LastSuccessfulSpec] + if !ok { + return nil, nil + } + + specBytes := []byte(val) + sucSpec := AutonomousContainerDatabaseSpec{} + + err := json.Unmarshal(specBytes, &sucSpec) + if err != nil { + return nil, err + } + + return &sucSpec, nil +} + +// UpdateStatusFromOCIACD updates only the status from database.AutonomousDatabase object +func (acd *AutonomousContainerDatabase) UpdateStatusFromOCIACD(ociObj database.AutonomousContainerDatabase) { + acd.Status.LifecycleState = ociObj.LifecycleState + acd.Status.TimeCreated = formatSDKTime(ociObj.TimeCreated) +} + +func (acd *AutonomousContainerDatabase) UpdateFromOCIACD(ociObj database.AutonomousContainerDatabase) { + // Spec + acd.Spec.AutonomousContainerDatabaseOCID = ociObj.Id + acd.Spec.CompartmentOCID = ociObj.CompartmentId + acd.Spec.DisplayName = ociObj.DisplayName + acd.Spec.AutonomousExadataVMClusterOCID = ociObj.CloudAutonomousVmClusterId + acd.Spec.PatchModel = ociObj.PatchModel + acd.Spec.FreeformTags = ociObj.FreeformTags + + // Status + acd.UpdateStatusFromOCIACD(ociObj) +} + +// RemoveUnchangedSpec removes the unchanged fields in spec, and returns if the spec has been changed. +// The function only takes the fields associated to ACD into account. That is, the ociConfig won't be impacted. +// Always restore the autonomousContainerDatabaseOCID from the lastSucSpec because we need it to send requests. +// A `false` is returned if the lastSucSpec is nil. +func (acd *AutonomousContainerDatabase) RemoveUnchangedSpec() (bool, error) { + lastSucSpec, err := acd.GetLastSuccessfulSpec() + if lastSucSpec == nil { + return false, errors.New("lastSucSpec is nil") + } + + oldConifg := acd.Spec.OCIConfig.DeepCopy() + + changed, err := removeUnchangedFields(*lastSucSpec, &acd.Spec) + if err != nil { + return changed, err + } + + acd.Spec.AutonomousContainerDatabaseOCID = lastSucSpec.AutonomousContainerDatabaseOCID + + oldConifg.DeepCopyInto(&acd.Spec.OCIConfig) + + return changed, nil +} diff --git a/apis/database/v1alpha1/autonomousdatabase_types.go b/apis/database/v1alpha1/autonomousdatabase_types.go index 501bf546..37926c36 100644 --- a/apis/database/v1alpha1/autonomousdatabase_types.go +++ b/apis/database/v1alpha1/autonomousdatabase_types.go @@ -49,6 +49,9 @@ import ( // EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! // NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized. +// name of our custom finalizer +const ADBFinalizer = "database.oracle.com/adb-finalizer" + // AutonomousDatabaseSpec defines the desired state of AutonomousDatabase // Important: Run "make" to regenerate code after modifying this file type AutonomousDatabaseSpec struct { @@ -58,30 +61,38 @@ type AutonomousDatabaseSpec struct { HardLink *bool `json:"hardLink,omitempty"` } -// AutonomousDatabaseDetails defines the detail information of AutonomousDatabase, corresponding to oci-go-sdk/database/AutonomousDatabase -type AutonomousDatabaseDetails struct { - AutonomousDatabaseOCID *string `json:"autonomousDatabaseOCID,omitempty"` - CompartmentOCID *string `json:"compartmentOCID,omitempty"` - AutonomousContainerDatabaseOCID *string `json:"autonomousContainerDatabaseOCID,omitempty"` - DisplayName *string `json:"displayName,omitempty"` - DbName *string `json:"dbName,omitempty"` - // +kubebuilder:validation:Enum:="OLTP";"DW";"AJD";"APEX" - DbWorkload database.AutonomousDatabaseDbWorkloadEnum `json:"dbWorkload,omitempty"` - // +kubebuilder:validation:Enum:="LICENSE_INCLUDED";"BRING_YOUR_OWN_LICENSE" - LicenseModel database.AutonomousDatabaseLicenseModelEnum `json:"licenseModel,omitempty"` - DbVersion *string `json:"dbVersion,omitempty"` - DataStorageSizeInTBs *int `json:"dataStorageSizeInTBs,omitempty"` - CPUCoreCount *int `json:"cpuCoreCount,omitempty"` - AdminPassword PasswordSpec `json:"adminPassword,omitempty"` - IsAutoScalingEnabled *bool `json:"isAutoScalingEnabled,omitempty"` - IsDedicated *bool `json:"isDedicated,omitempty"` - LifecycleState database.AutonomousDatabaseLifecycleStateEnum `json:"lifecycleState,omitempty"` +/************************ +* ACD specs +************************/ +type K8sACDSpec struct { + Name *string `json:"name,omitempty"` +} - NetworkAccess NetworkAccessSpec `json:"networkAccess,omitempty"` +type OCIACDSpec struct { + OCID *string `json:"ocid,omitempty"` +} - FreeformTags map[string]string `json:"freeformTags,omitempty"` +// ACDSpec defines the spec of the target for backup/restore runs. +// The name could be the name of an AutonomousDatabase or an AutonomousDatabaseBackup +type ACDSpec struct { + K8sACD K8sACDSpec `json:"k8sACD,omitempty"` + OCIACD OCIACDSpec `json:"ociACD,omitempty"` +} - Wallet WalletSpec `json:"wallet,omitempty"` +/************************ +* Secret specs +************************/ +type K8sSecretSpec struct { + Name *string `json:"name,omitempty"` +} + +type OCISecretSpec struct { + OCID *string `json:"ocid,omitempty"` +} + +type PasswordSpec struct { + K8sSecret K8sSecretSpec `json:"k8sSecret,omitempty"` + OCISecret OCISecretSpec `json:"ociSecret,omitempty"` } type WalletSpec struct { @@ -89,10 +100,9 @@ type WalletSpec struct { Password PasswordSpec `json:"password,omitempty"` } -type PasswordSpec struct { - K8sSecretName *string `json:"k8sSecretName,omitempty"` - OCISecretOCID *string `json:"ociSecretOCID,omitempty"` -} +/************************ +* Network Access specs +************************/ type NetworkAccessTypeEnum string @@ -117,6 +127,32 @@ type PrivateEndpointSpec struct { HostnamePrefix *string `json:"hostnamePrefix,omitempty"` } +// AutonomousDatabaseDetails defines the detail information of AutonomousDatabase, corresponding to oci-go-sdk/database/AutonomousDatabase +type AutonomousDatabaseDetails struct { + AutonomousDatabaseOCID *string `json:"autonomousDatabaseOCID,omitempty"` + CompartmentOCID *string `json:"compartmentOCID,omitempty"` + AutonomousContainerDatabase ACDSpec `json:"autonomousContainerDatabase,omitempty"` + DisplayName *string `json:"displayName,omitempty"` + DbName *string `json:"dbName,omitempty"` + // +kubebuilder:validation:Enum:="OLTP";"DW";"AJD";"APEX" + DbWorkload database.AutonomousDatabaseDbWorkloadEnum `json:"dbWorkload,omitempty"` + // +kubebuilder:validation:Enum:="LICENSE_INCLUDED";"BRING_YOUR_OWN_LICENSE" + LicenseModel database.AutonomousDatabaseLicenseModelEnum `json:"licenseModel,omitempty"` + DbVersion *string `json:"dbVersion,omitempty"` + DataStorageSizeInTBs *int `json:"dataStorageSizeInTBs,omitempty"` + CPUCoreCount *int `json:"cpuCoreCount,omitempty"` + AdminPassword PasswordSpec `json:"adminPassword,omitempty"` + IsAutoScalingEnabled *bool `json:"isAutoScalingEnabled,omitempty"` + IsDedicated *bool `json:"isDedicated,omitempty"` + LifecycleState database.AutonomousDatabaseLifecycleStateEnum `json:"lifecycleState,omitempty"` + + NetworkAccess NetworkAccessSpec `json:"networkAccess,omitempty"` + + FreeformTags map[string]string `json:"freeformTags,omitempty"` + + Wallet WalletSpec `json:"wallet,omitempty"` +} + // AutonomousDatabaseStatus defines the observed state of AutonomousDatabase type AutonomousDatabaseStatus struct { // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster @@ -247,7 +283,7 @@ func (adb *AutonomousDatabase) UpdateFromOCIADB(ociObj database.AutonomousDataba ***********************************/ adb.Spec.Details.AutonomousDatabaseOCID = ociObj.Id adb.Spec.Details.CompartmentOCID = ociObj.CompartmentId - adb.Spec.Details.AutonomousContainerDatabaseOCID = ociObj.AutonomousContainerDatabaseId + adb.Spec.Details.AutonomousContainerDatabase.OCIACD.OCID = ociObj.AutonomousContainerDatabaseId adb.Spec.Details.DisplayName = ociObj.DisplayName adb.Spec.Details.DbName = ociObj.DbName adb.Spec.Details.DbWorkload = ociObj.DbWorkload @@ -288,7 +324,7 @@ func (adb *AutonomousDatabase) UpdateFromOCIADB(ociObj database.AutonomousDataba } // RemoveUnchangedDetails removes the unchanged fields in spec.details, and returns if the details has been changed. -// details.autonomousDatabaseOCID won't be affected because we need it to send requests. +// Always restore the details.autonomousDatabaseOCID from the lastSucSpec because we need it to send requests. // A `false` is returned if the lastSucSpec is nil. func (adb *AutonomousDatabase) RemoveUnchangedDetails() (bool, error) { lastSucSpec, err := adb.GetLastSuccessfulSpec() diff --git a/apis/database/v1alpha1/autonomousdatabase_webhook.go b/apis/database/v1alpha1/autonomousdatabase_webhook.go index 434bee74..521b7988 100644 --- a/apis/database/v1alpha1/autonomousdatabase_webhook.go +++ b/apis/database/v1alpha1/autonomousdatabase_webhook.go @@ -163,16 +163,16 @@ func (r *AutonomousDatabase) ValidateUpdate(old runtime.Object) error { func validateCommon(adb *AutonomousDatabase, allErrs field.ErrorList) field.ErrorList { // password - if adb.Spec.Details.AdminPassword.K8sSecretName != nil && adb.Spec.Details.AdminPassword.OCISecretOCID != nil { + if adb.Spec.Details.AdminPassword.K8sSecret.Name != nil && adb.Spec.Details.AdminPassword.OCISecret.OCID != nil { allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec").Child("details").Child("Wallet").Child("Password"), - "cannot apply k8sSecretName and ociSecretOCID at the same time")) + field.Forbidden(field.NewPath("spec").Child("details").Child("adminPassword"), + "cannot apply k8sSecret.name and ociSecret.ocid at the same time")) } - if adb.Spec.Details.Wallet.Password.K8sSecretName != nil && adb.Spec.Details.Wallet.Password.OCISecretOCID != nil { + if adb.Spec.Details.Wallet.Password.K8sSecret.Name != nil && adb.Spec.Details.Wallet.Password.OCISecret.OCID != nil { allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec").Child("details").Child("AdminPassword"), - "cannot apply k8sSecretName and ociSecretOCID at the same time")) + field.Forbidden(field.NewPath("spec").Child("details").Child("wallet").Child("password"), + "cannot apply k8sSecret.name and ociSecret.ocid at the same time")) } return allErrs @@ -238,5 +238,5 @@ func (r *AutonomousDatabase) ValidateDelete() error { // Returns true if AutonomousContainerDatabaseOCID has value. // We don't use Details.IsDedicated because the parameter might be null when it's a provision operation. func isDedicated(adb *AutonomousDatabase) bool { - return adb.Spec.Details.AutonomousContainerDatabaseOCID != nil + return adb.Spec.Details.AutonomousContainerDatabase.OCIACD.OCID != nil } diff --git a/apis/database/v1alpha1/autonomousdatabasebackup_types.go b/apis/database/v1alpha1/autonomousdatabasebackup_types.go index 3c27d3bb..3d6defd8 100644 --- a/apis/database/v1alpha1/autonomousdatabasebackup_types.go +++ b/apis/database/v1alpha1/autonomousdatabasebackup_types.go @@ -52,9 +52,9 @@ import ( type AutonomousDatabaseBackupSpec struct { // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster // Important: Run "make" to regenerate code after modifying this file - TargetADB TargetSpec `json:"targetADB"` + Target TargetSpec `json:"target"` DisplayName string `json:"displayName"` - AutonomousDatabaseBackupOCID string `json:"autonomousDatabaseBackupOCID,omitempty"` + AutonomousDatabaseBackupOCID *string `json:"autonomousDatabaseBackupOCID,omitempty"` OCIConfig OCIConfigSpec `json:"ociConfig,omitempty"` } @@ -63,15 +63,17 @@ type AutonomousDatabaseBackupSpec struct { type AutonomousDatabaseBackupStatus struct { // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster // Important: Run "make" to regenerate code after modifying this file - LifecycleState database.AutonomousDatabaseBackupLifecycleStateEnum `json:"lifecycleState"` - Type database.AutonomousDatabaseBackupTypeEnum `json:"type"` - IsAutomatic bool `json:"isAutomatic"` - TimeStarted string `json:"timeStarted,omitempty"` - TimeEnded string `json:"timeEnded,omitempty"` - AutonomousDatabaseOCID string `json:"autonomousDatabaseOCID"` - CompartmentOCID string `json:"compartmentOCID"` - DBName string `json:"dbName"` - DBDisplayName string `json:"dbDisplayName"` + AutonomousDatabaseBackupOCID string `json:"autonomousDatabaseBackupOCID,omitempty"` + DisplayName string `json:"displayName"` + LifecycleState database.AutonomousDatabaseBackupLifecycleStateEnum `json:"lifecycleState"` + Type database.AutonomousDatabaseBackupTypeEnum `json:"type"` + IsAutomatic bool `json:"isAutomatic"` + TimeStarted string `json:"timeStarted,omitempty"` + TimeEnded string `json:"timeEnded,omitempty"` + AutonomousDatabaseOCID string `json:"autonomousDatabaseOCID"` + CompartmentOCID string `json:"compartmentOCID"` + DBName string `json:"dbName"` + DBDisplayName string `json:"dbDisplayName"` } //+kubebuilder:object:root=true @@ -105,10 +107,10 @@ func init() { SchemeBuilder.Register(&AutonomousDatabaseBackup{}, &AutonomousDatabaseBackupList{}) } -func (backup *AutonomousDatabaseBackup) UpdateFromOCIBackup(ociBackup database.AutonomousDatabaseBackup, ociADB database.AutonomousDatabase) { - backup.Spec.AutonomousDatabaseBackupOCID = *ociBackup.Id - backup.Spec.DisplayName = *ociBackup.DisplayName - +func (backup *AutonomousDatabaseBackup) UpdateStatusFromOCIBackup(ociBackup database.AutonomousDatabaseBackup, ociADB database.AutonomousDatabase) { + backup.Status.AutonomousDatabaseBackupOCID = *ociBackup.Id + backup.Status.DisplayName = *ociBackup.DisplayName + backup.Status.AutonomousDatabaseOCID = *ociBackup.AutonomousDatabaseId backup.Status.CompartmentOCID = *ociBackup.CompartmentId backup.Status.Type = ociBackup.Type diff --git a/apis/database/v1alpha1/autonomousdatabasebackup_webhook.go b/apis/database/v1alpha1/autonomousdatabasebackup_webhook.go index a3dc9299..880c5beb 100644 --- a/apis/database/v1alpha1/autonomousdatabasebackup_webhook.go +++ b/apis/database/v1alpha1/autonomousdatabasebackup_webhook.go @@ -78,7 +78,20 @@ var _ webhook.Validator = &AutonomousDatabaseBackup{} // ValidateCreate implements webhook.Validator so a webhook will be registered for the type func (r *AutonomousDatabaseBackup) ValidateCreate() error { autonomousdatabasebackuplog.Info("validate create", "name", r.Name) - return nil + + var allErrs field.ErrorList + + if r.Spec.Target.K8sADB.Name != nil && r.Spec.Target.OCIADB.OCID != nil { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("target"), "specify either k8sADB or ociADB, but not both")) + } + + if len(allErrs) == 0 { + return nil + } + return apierrors.NewInvalid( + schema.GroupKind{Group: "database.oracle.com", Kind: "AutonomousDatabaseBackup"}, + r.Name, allErrs) } // ValidateUpdate implements webhook.Validator so a webhook will be registered for the type @@ -88,22 +101,22 @@ func (r *AutonomousDatabaseBackup) ValidateUpdate(old runtime.Object) error { var allErrs field.ErrorList oldBackup := old.(*AutonomousDatabaseBackup) - if oldBackup.Spec.AutonomousDatabaseBackupOCID != "" && + if oldBackup.Spec.AutonomousDatabaseBackupOCID != nil && oldBackup.Spec.AutonomousDatabaseBackupOCID != r.Spec.AutonomousDatabaseBackupOCID { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("autonomousDatabaseBackupOCID"), "cannot assign a new autonomousDatabaseBackupOCID to this backup")) } - if r.Spec.TargetADB.Name != "" && - oldBackup.Spec.TargetADB.Name != r.Spec.TargetADB.Name { + if r.Spec.Target.K8sADB.Name != nil && + oldBackup.Spec.Target.K8sADB.Name != r.Spec.Target.K8sADB.Name { allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec").Child("targetADB").Child("name"), "cannot change target.name")) + field.Forbidden(field.NewPath("spec").Child("target").Child("k8sADB").Child("name"), "cannot change target.name")) } - if r.Spec.TargetADB.OCID != "" && - oldBackup.Spec.TargetADB.OCID != r.Spec.TargetADB.OCID { + if r.Spec.Target.OCIADB.OCID != nil && + oldBackup.Spec.Target.OCIADB.OCID != r.Spec.Target.OCIADB.OCID { allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec").Child("targetADB").Child("ocid"), "cannot change target.ocid")) + field.Forbidden(field.NewPath("spec").Child("target").Child("ociADB").Child("ocid"), "cannot change target.ocid")) } if r.Spec.DisplayName != "" && diff --git a/apis/database/v1alpha1/autonomousdatabaserestore_types.go b/apis/database/v1alpha1/autonomousdatabaserestore_types.go index 1378f854..1e893a97 100644 --- a/apis/database/v1alpha1/autonomousdatabaserestore_types.go +++ b/apis/database/v1alpha1/autonomousdatabaserestore_types.go @@ -39,6 +39,8 @@ package v1alpha1 import ( + "errors" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "github.com/oracle/oci-go-sdk/v54/common" @@ -47,23 +49,27 @@ import ( // EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! // NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized. +type K8sADBBackupSpec struct { + Name *string `json:"name,omitempty"` +} -// AutonomousDatabaseRestoreSpec defines the desired state of AutonomousDatabaseRestore -type AutonomousDatabaseRestoreSpec struct { - // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster - // Important: Run "make" to regenerate code after modifying this file - TargetADB TargetSpec `json:"targetADB"` - Source SourceSpec `json:"sourceSpec"` - OCIConfig OCIConfigSpec `json:"ociConfig,omitempty"` +type PITSpec struct { + // The timestamp must follow this format: YYYY-MM-DD HH:MM:SS GMT + Timestamp *string `json:"timestamp,omitempty"` } type SourceSpec struct { - AutonomousDatabaseBackup BackupSourceSpec `json:"autonomousDatabaseBackup,omitempty"` - PointInTime string `json:"pointInTime,omitempty"` + K8sADBBackup K8sADBBackupSpec `json:"k8sADBBackup,omitempty"` + PointInTime PITSpec `json:"pointInTime,omitempty"` } -type BackupSourceSpec struct { - Name string `json:"name,omitempty"` +// AutonomousDatabaseRestoreSpec defines the desired state of AutonomousDatabaseRestore +type AutonomousDatabaseRestoreSpec struct { + // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster + // Important: Run "make" to regenerate code after modifying this file + Target TargetSpec `json:"target"` + Source SourceSpec `json:"source"` + OCIConfig OCIConfigSpec `json:"ociConfig,omitempty"` } type restoreStatusEnum string @@ -115,22 +121,29 @@ func init() { // GetPIT returns the spec.pointInTime.timeStamp in SDKTime format func (r *AutonomousDatabaseRestore) GetPIT() (*common.SDKTime, error) { - return parseDisplayTime(r.Spec.Source.PointInTime) + if r.Spec.Source.PointInTime.Timestamp == nil { + return nil, errors.New("The timestamp is empty") + } + return parseDisplayTime(*r.Spec.Source.PointInTime.Timestamp) } -func (r *AutonomousDatabaseRestore) ConvertWorkRequestStatus(s workrequests.WorkRequestStatusEnum) restoreStatusEnum { +func (r *AutonomousDatabaseRestore) ConvertWorkRequestStatus(s workrequests.WorkRequestStatusEnum) (restoreStatusEnum, error) { switch s { case workrequests.WorkRequestStatusAccepted: fallthrough case workrequests.WorkRequestStatusInProgress: - return RestoreStatusInProgress + return RestoreStatusInProgress, nil case workrequests.WorkRequestStatusSucceeded: - return RestoreStatusSucceeded + return RestoreStatusSucceeded, nil + case workrequests.WorkRequestStatusCanceling: + fallthrough + case workrequests.WorkRequestStatusCanceled: + fallthrough case workrequests.WorkRequestStatusFailed: - return RestoreStatusFailed + return RestoreStatusFailed, nil } - return "UNKNOWN" + return "", errors.New("unable to convert the status: " + string(s)) } diff --git a/apis/database/v1alpha1/autonomousdatabaserestore_webhook.go b/apis/database/v1alpha1/autonomousdatabaserestore_webhook.go index 407403e9..edc4db7a 100644 --- a/apis/database/v1alpha1/autonomousdatabaserestore_webhook.go +++ b/apis/database/v1alpha1/autonomousdatabaserestore_webhook.go @@ -70,30 +70,36 @@ func (r *AutonomousDatabaseRestore) ValidateCreate() error { var allErrs field.ErrorList + // Validate the target ADB + if r.Spec.Target.K8sADB.Name != nil && r.Spec.Target.OCIADB.OCID != nil { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("target"), "specify either k8sADB.name or ociADB.ocid, but not both")) + } + // Validate the restore source - if r.Spec.Source.AutonomousDatabaseBackup.Name == "" && - r.Spec.Source.PointInTime == "" { + if r.Spec.Source.K8sADBBackup.Name == nil && + r.Spec.Source.PointInTime.Timestamp == nil { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("source"), "no retore source is chosen")) } - if r.Spec.Source.AutonomousDatabaseBackup.Name != "" && - r.Spec.Source.PointInTime != "" { + if r.Spec.Source.K8sADBBackup.Name != nil && + r.Spec.Source.PointInTime.Timestamp != nil { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("source"), "cannot apply backupName and the PITR parameters at the same time")) } - if (r.Spec.TargetADB.OCID == "" && r.Spec.Source.PointInTime != "") || - (r.Spec.TargetADB.OCID != "" && r.Spec.Source.PointInTime == "") { - field.Forbidden(field.NewPath("spec").Child("source").Child("pointInTime"), "targetADB.OCID or source.pointInTime cannot be empty") + if (r.Spec.Target.OCIADB.OCID == nil && r.Spec.Source.PointInTime.Timestamp != nil) || + (r.Spec.Target.OCIADB.OCID != nil && r.Spec.Source.PointInTime.Timestamp == nil) { + field.Forbidden(field.NewPath("spec").Child("source").Child("pointInTime").Child("timestamp"), "target.ociADB.ocid or source.pointInTime.timestamp cannot be empty") } // Verify the timestamp format if it's PITR - if r.Spec.Source.PointInTime != "" { - _, err := parseDisplayTime(r.Spec.Source.PointInTime) + if r.Spec.Source.PointInTime.Timestamp != nil { + _, err := parseDisplayTime(*r.Spec.Source.PointInTime.Timestamp) if err != nil { allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec").Child("source").Child("pointInTime"), "invalid timestamp format")) + field.Forbidden(field.NewPath("spec").Child("source").Child("pointInTime").Child("timestamp"), "invalid timestamp format")) } } @@ -111,9 +117,6 @@ func (r *AutonomousDatabaseRestore) ValidateUpdate(old runtime.Object) error { var allErrs field.ErrorList - allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec"), "update AutonomousDatabaseRestore is diabled")) - if len(allErrs) == 0 { return nil } diff --git a/apis/database/v1alpha1/shardingdatabase_types.go b/apis/database/v1alpha1/shardingdatabase_types.go index c372b991..39f74eb1 100644 --- a/apis/database/v1alpha1/shardingdatabase_types.go +++ b/apis/database/v1alpha1/shardingdatabase_types.go @@ -298,7 +298,7 @@ func (shardingv1 *ShardingDatabase) UpdateLastSuccessfulSpec(kubeClient client.C lastSuccessfulSpec: string(specBytes), } - return annsv1.SetAnnotations(kubeClient, shardingv1, anns) + return annsv1.PatchAnnotations(kubeClient, shardingv1, anns) } func init() { diff --git a/apis/database/v1alpha1/zz_generated.deepcopy.go b/apis/database/v1alpha1/zz_generated.deepcopy.go index 9507b955..f3c1b7b2 100644 --- a/apis/database/v1alpha1/zz_generated.deepcopy.go +++ b/apis/database/v1alpha1/zz_generated.deepcopy.go @@ -49,6 +49,23 @@ import ( "k8s.io/apimachinery/pkg/runtime" ) +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *ACDSpec) DeepCopyInto(out *ACDSpec) { + *out = *in + in.K8sACD.DeepCopyInto(&out.K8sACD) + in.OCIACD.DeepCopyInto(&out.OCIACD) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ACDSpec. +func (in *ACDSpec) DeepCopy() *ACDSpec { + if in == nil { + return nil + } + out := new(ACDSpec) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *AutonomousContainerDatabase) DeepCopyInto(out *AutonomousContainerDatabase) { *out = *in @@ -139,6 +156,11 @@ func (in *AutonomousContainerDatabaseSpec) DeepCopyInto(out *AutonomousContainer } } in.OCIConfig.DeepCopyInto(&out.OCIConfig) + if in.HardLink != nil { + in, out := &in.HardLink, &out.HardLink + *out = new(bool) + **out = **in + } } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousContainerDatabaseSpec. @@ -255,7 +277,12 @@ func (in *AutonomousDatabaseBackupList) DeepCopyObject() runtime.Object { // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *AutonomousDatabaseBackupSpec) DeepCopyInto(out *AutonomousDatabaseBackupSpec) { *out = *in - out.TargetADB = in.TargetADB + in.Target.DeepCopyInto(&out.Target) + if in.AutonomousDatabaseBackupOCID != nil { + in, out := &in.AutonomousDatabaseBackupOCID, &out.AutonomousDatabaseBackupOCID + *out = new(string) + **out = **in + } in.OCIConfig.DeepCopyInto(&out.OCIConfig) } @@ -297,11 +324,7 @@ func (in *AutonomousDatabaseDetails) DeepCopyInto(out *AutonomousDatabaseDetails *out = new(string) **out = **in } - if in.AutonomousContainerDatabaseOCID != nil { - in, out := &in.AutonomousContainerDatabaseOCID, &out.AutonomousContainerDatabaseOCID - *out = new(string) - **out = **in - } + in.AutonomousContainerDatabase.DeepCopyInto(&out.AutonomousContainerDatabase) if in.DisplayName != nil { in, out := &in.DisplayName, &out.DisplayName *out = new(string) @@ -453,8 +476,8 @@ func (in *AutonomousDatabaseRestoreList) DeepCopyObject() runtime.Object { // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *AutonomousDatabaseRestoreSpec) DeepCopyInto(out *AutonomousDatabaseRestoreSpec) { *out = *in - out.TargetADB = in.TargetADB - out.Source = in.Source + in.Target.DeepCopyInto(&out.Target) + in.Source.DeepCopyInto(&out.Source) in.OCIConfig.DeepCopyInto(&out.OCIConfig) } @@ -527,21 +550,6 @@ func (in *AutonomousDatabaseStatus) DeepCopy() *AutonomousDatabaseStatus { return out } -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *BackupSourceSpec) DeepCopyInto(out *BackupSourceSpec) { - *out = *in -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new BackupSourceSpec. -func (in *BackupSourceSpec) DeepCopy() *BackupSourceSpec { - if in == nil { - return nil - } - out := new(BackupSourceSpec) - in.DeepCopyInto(out) - return out -} - // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *CDB) DeepCopyInto(out *CDB) { *out = *in @@ -896,6 +904,86 @@ func (in *GsmStatusDetails) DeepCopy() *GsmStatusDetails { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *K8sACDSpec) DeepCopyInto(out *K8sACDSpec) { + *out = *in + if in.Name != nil { + in, out := &in.Name, &out.Name + *out = new(string) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new K8sACDSpec. +func (in *K8sACDSpec) DeepCopy() *K8sACDSpec { + if in == nil { + return nil + } + out := new(K8sACDSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *K8sADBBackupSpec) DeepCopyInto(out *K8sADBBackupSpec) { + *out = *in + if in.Name != nil { + in, out := &in.Name, &out.Name + *out = new(string) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new K8sADBBackupSpec. +func (in *K8sADBBackupSpec) DeepCopy() *K8sADBBackupSpec { + if in == nil { + return nil + } + out := new(K8sADBBackupSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *K8sADBSpec) DeepCopyInto(out *K8sADBSpec) { + *out = *in + if in.Name != nil { + in, out := &in.Name, &out.Name + *out = new(string) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new K8sADBSpec. +func (in *K8sADBSpec) DeepCopy() *K8sADBSpec { + if in == nil { + return nil + } + out := new(K8sADBSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *K8sSecretSpec) DeepCopyInto(out *K8sSecretSpec) { + *out = *in + if in.Name != nil { + in, out := &in.Name, &out.Name + *out = new(string) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new K8sSecretSpec. +func (in *K8sSecretSpec) DeepCopy() *K8sSecretSpec { + if in == nil { + return nil + } + out := new(K8sSecretSpec) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *NetworkAccessSpec) DeepCopyInto(out *NetworkAccessSpec) { *out = *in @@ -927,6 +1015,46 @@ func (in *NetworkAccessSpec) DeepCopy() *NetworkAccessSpec { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *OCIACDSpec) DeepCopyInto(out *OCIACDSpec) { + *out = *in + if in.OCID != nil { + in, out := &in.OCID, &out.OCID + *out = new(string) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new OCIACDSpec. +func (in *OCIACDSpec) DeepCopy() *OCIACDSpec { + if in == nil { + return nil + } + out := new(OCIACDSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *OCIADBSpec) DeepCopyInto(out *OCIADBSpec) { + *out = *in + if in.OCID != nil { + in, out := &in.OCID, &out.OCID + *out = new(string) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new OCIADBSpec. +func (in *OCIADBSpec) DeepCopy() *OCIADBSpec { + if in == nil { + return nil + } + out := new(OCIADBSpec) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *OCIConfigSpec) DeepCopyInto(out *OCIConfigSpec) { *out = *in @@ -952,6 +1080,26 @@ func (in *OCIConfigSpec) DeepCopy() *OCIConfigSpec { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *OCISecretSpec) DeepCopyInto(out *OCISecretSpec) { + *out = *in + if in.OCID != nil { + in, out := &in.OCID, &out.OCID + *out = new(string) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new OCISecretSpec. +func (in *OCISecretSpec) DeepCopy() *OCISecretSpec { + if in == nil { + return nil + } + out := new(OCISecretSpec) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *ORDSPassword) DeepCopyInto(out *ORDSPassword) { *out = *in @@ -1306,18 +1454,30 @@ func (in *PDBStatus) DeepCopy() *PDBStatus { } // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *PasswordSpec) DeepCopyInto(out *PasswordSpec) { +func (in *PITSpec) DeepCopyInto(out *PITSpec) { *out = *in - if in.K8sSecretName != nil { - in, out := &in.K8sSecretName, &out.K8sSecretName + if in.Timestamp != nil { + in, out := &in.Timestamp, &out.Timestamp *out = new(string) **out = **in } - if in.OCISecretOCID != nil { - in, out := &in.OCISecretOCID, &out.OCISecretOCID - *out = new(string) - **out = **in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PITSpec. +func (in *PITSpec) DeepCopy() *PITSpec { + if in == nil { + return nil } + out := new(PITSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PasswordSpec) DeepCopyInto(out *PasswordSpec) { + *out = *in + in.K8sSecret.DeepCopyInto(&out.K8sSecret) + in.OCISecret.DeepCopyInto(&out.OCISecret) } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PasswordSpec. @@ -1747,7 +1907,8 @@ func (in *SingleInstanceDatabaseStatus) DeepCopy() *SingleInstanceDatabaseStatus // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *SourceSpec) DeepCopyInto(out *SourceSpec) { *out = *in - out.AutonomousDatabaseBackup = in.AutonomousDatabaseBackup + in.K8sADBBackup.DeepCopyInto(&out.K8sADBBackup) + in.PointInTime.DeepCopyInto(&out.PointInTime) } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SourceSpec. @@ -1795,6 +1956,8 @@ func (in *TDESecret) DeepCopy() *TDESecret { // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *TargetSpec) DeepCopyInto(out *TargetSpec) { *out = *in + in.K8sADB.DeepCopyInto(&out.K8sADB) + in.OCIADB.DeepCopyInto(&out.OCIADB) } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new TargetSpec. diff --git a/commons/adb_family/utils.go b/commons/adb_family/utils.go new file mode 100644 index 00000000..1650dd22 --- /dev/null +++ b/commons/adb_family/utils.go @@ -0,0 +1,79 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package adbfamily + +import ( + "github.com/oracle/oci-go-sdk/v54/common" + dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + "github.com/oracle/oracle-database-operator/commons/k8s" + "sigs.k8s.io/controller-runtime/pkg/client" +) + +// VerifyTargetADB searches if the target ADB is in the cluster, and set the owner reference to the ADB if it exists. +// The function returns two values in the following order: +// ocid: the OCID of the target ADB. An empty string is returned if the ocid is nil. +// ownerADB: the resource of the targetADB if it's found in the cluster +func VerifyTargetADB(kubeClient client.Client, target dbv1alpha1.TargetSpec, namespace string) (string, *dbv1alpha1.AutonomousDatabase, error) { + var err error + var ocid *string + var ownerADB *dbv1alpha1.AutonomousDatabase + + // Get the target ADB OCID + if target.K8sADB.Name != nil { + ownerADB = &dbv1alpha1.AutonomousDatabase{} + if err := k8s.FetchResource(kubeClient, namespace, *target.K8sADB.Name, ownerADB); err != nil { + return "", nil, err + } + + ocid = ownerADB.Spec.Details.AutonomousDatabaseOCID + } else { + ownerADB, err = k8s.FetchAutonomousDatabaseWithOCID(kubeClient, namespace, *target.OCIADB.OCID) + if err != nil { + return "", nil, err + } + + ocid = target.OCIADB.OCID + } + + if ocid == nil { + ocid = common.String("") + } + + return *ocid, ownerADB, nil +} diff --git a/commons/annotations/annotations.go b/commons/annotations/annotations.go index 8472c627..9c354c0c 100644 --- a/commons/annotations/annotations.go +++ b/commons/annotations/annotations.go @@ -53,9 +53,9 @@ type PatchValue struct { Value interface{} `json:"value"` } -// SetAnnotations attaches the given metadata to the target object +// PatchAnnotations attaches the given metadata to the target object // The obj will be updated with the content returned by the cluster -func SetAnnotations(kubeClient client.Client, obj client.Object, anns map[string]string) error { +func PatchAnnotations(kubeClient client.Client, obj client.Object, anns map[string]string) error { payload := []PatchValue{} if obj.GetAnnotations() == nil { diff --git a/commons/k8s/create.go b/commons/k8s/create.go index b3e7bd8f..74afcaa7 100644 --- a/commons/k8s/create.go +++ b/commons/k8s/create.go @@ -43,6 +43,7 @@ import ( dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + "github.com/oracle/oci-go-sdk/v54/common" "github.com/oracle/oci-go-sdk/v54/database" corev1 "k8s.io/api/core/v1" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" @@ -86,11 +87,13 @@ func CreateAutonomousBackup(kubeClient client.Client, OwnerReferences: NewOwnerReference(ownerADB), }, Spec: dbv1alpha1.AutonomousDatabaseBackupSpec{ - TargetADB: dbv1alpha1.TargetSpec{ - Name: ownerADB.Name, + Target: dbv1alpha1.TargetSpec{ + K8sADB: dbv1alpha1.K8sADBSpec{ + Name: common.String(ownerADB.Name), + }, }, DisplayName: *backupSummary.DisplayName, - AutonomousDatabaseBackupOCID: *backupSummary.Id, + AutonomousDatabaseBackupOCID: backupSummary.Id, OCIConfig: dbv1alpha1.OCIConfigSpec{ ConfigMapName: ownerADB.Spec.OCIConfig.ConfigMapName, SecretName: ownerADB.Spec.OCIConfig.SecretName, diff --git a/commons/k8s/fetch.go b/commons/k8s/fetch.go index 17e0125b..6a41b4b1 100644 --- a/commons/k8s/fetch.go +++ b/commons/k8s/fetch.go @@ -62,16 +62,6 @@ func FetchResource(kubeClient client.Client, namespace string, name string, obje return nil } -func FetchAutonomousDatabase(kubeClient client.Client, namespace string, name string) (*dbv1alpha1.AutonomousDatabase, error) { - adb := &dbv1alpha1.AutonomousDatabase{} - - if err := FetchResource(kubeClient, namespace, name, adb); err != nil { - return nil, err - } - - return adb, nil -} - // Returns the first AutonomousDatabase resource that matches the AutonomousDatabaseOCID of the backup // If the AutonomousDatabase doesn't exist, returns a nil func FetchAutonomousDatabaseWithOCID(kubeClient client.Client, namespace string, ocid string) (*dbv1alpha1.AutonomousDatabase, error) { @@ -131,7 +121,7 @@ func FetchConfigMap(kubeClient client.Client, namespace string, name string) (*c func FetchSecret(kubeClient client.Client, namespace string, name string) (*corev1.Secret, error) { secret := &corev1.Secret{} - + if err := FetchResource(kubeClient, namespace, name, secret); err != nil { return nil, err } diff --git a/commons/k8s/finalizer.go b/commons/k8s/finalizer.go index f73947be..256597fd 100644 --- a/commons/k8s/finalizer.go +++ b/commons/k8s/finalizer.go @@ -44,56 +44,12 @@ import ( "k8s.io/apimachinery/pkg/types" "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" ) +func patchFinalizer(kubeClient client.Client, obj client.Object) error { + finalizer := obj.GetFinalizers() -// HasFinalizer returns true if the finalizer exists in the object metadata -func HasFinalizer(obj client.Object, finalizer string) bool { - finalizers := obj.GetFinalizers() - return containsString(finalizers, finalizer) -} - -// Register adds the finalizer and patch the object -func Register(kubeClient client.Client, obj client.Object, finalizer string) error { - finalizers := obj.GetFinalizers() - finalizers = append(finalizers, finalizer) - return setFinalizer(kubeClient, obj, finalizers) -} - -// Unregister removes the finalizer and patch the object -func Unregister(kubeClient client.Client, obj client.Object, finalizer string) error { - finalizers := obj.GetFinalizers() - finalizers = removeString(finalizers, finalizer) - return setFinalizer(kubeClient, obj, finalizers) -} - -// Helper functions to check and remove string from a slice of strings. -func containsString(slice []string, s string) bool { - for _, item := range slice { - if item == s { - return true - } - } - return false -} - -func removeString(slice []string, s string) (result []string) { - for _, item := range slice { - if item == s { - continue - } - result = append(result, item) - } - return -} - -type patchValue struct { - Op string `json:"op"` - Path string `json:"path"` - Value interface{} `json:"value"` -} - -func setFinalizer(kubeClient client.Client, obj client.Object, finalizer []string) error { payload := []patchValue{} if obj.GetFinalizers() == nil { @@ -118,3 +74,25 @@ func setFinalizer(kubeClient client.Client, obj client.Object, finalizer []strin patch := client.RawPatch(types.JSONPatchType, payloadBytes) return kubeClient.Patch(context.TODO(), obj, patch) } + +// No-op if the obj already has the finalizer +func AddFinalizerAndPatch(kubeClient client.Client, obj client.Object, finalizer string) error { + if !controllerutil.ContainsFinalizer(obj, finalizer) { + controllerutil.AddFinalizer(obj, finalizer) + if err := patchFinalizer(kubeClient, obj); err != nil { + return err + } + } + return nil +} + +// No-op if the obj doesn't have the finalizer +func RemoveFinalizerAndPatch(kubeClient client.Client, obj client.Object, finalizer string) error { + if controllerutil.ContainsFinalizer(obj, finalizer) { + controllerutil.RemoveFinalizer(obj, finalizer) + if err := patchFinalizer(kubeClient, obj); err != nil { + return err + } + } + return nil +} diff --git a/commons/k8s/utils.go b/commons/k8s/utils.go index 0e5a7aa6..092357a4 100644 --- a/commons/k8s/utils.go +++ b/commons/k8s/utils.go @@ -40,12 +40,11 @@ package k8s import ( "context" + "encoding/json" - dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/types" utilErrors "k8s.io/apimachinery/pkg/util/errors" - "k8s.io/client-go/util/retry" "sigs.k8s.io/controller-runtime/pkg/client" ) @@ -65,20 +64,23 @@ func CombineErrors(errs ...error) error { return utilErrors.NewAggregate(errs) } -func UpdateADBStatus(kubeClient client.Client, adb *dbv1alpha1.AutonomousDatabase) error { - return retry.RetryOnConflict(retry.DefaultRetry, func() error { - curADB := &dbv1alpha1.AutonomousDatabase{} +/********************** + Patch resource +**********************/ - namespacedName := types.NamespacedName{ - Namespace: adb.GetNamespace(), - Name: adb.GetName(), - } - - if err := kubeClient.Get(context.TODO(), namespacedName, curADB); err != nil { - return err - } +type patchValue struct { + Op string `json:"op"` + Path string `json:"path"` + Value interface{} `json:"value"` +} - curADB.Status = adb.Status - return kubeClient.Status().Update(context.TODO(), curADB) - }) +func Patch(kubeClient client.Client, obj client.Object, path string, value interface{}) error { + payload := []patchValue{{ + Op: "replace", + Path: path, + Value: value, + }} + payloadBytes, _ := json.Marshal(payload) + patch := client.RawPatch(types.JSONPatchType, payloadBytes) + return kubeClient.Patch(context.TODO(), obj, patch) } diff --git a/commons/oci/containerdatabase.go b/commons/oci/containerdatabase.go new file mode 100644 index 00000000..30e57919 --- /dev/null +++ b/commons/oci/containerdatabase.go @@ -0,0 +1,93 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package oci + +import ( + "context" + + "github.com/oracle/oci-go-sdk/v54/common" + "github.com/oracle/oci-go-sdk/v54/database" + + dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" +) + +/******************************** + * Autonomous Container Database + *******************************/ +func (d *databaseService) CreateAutonomousContainerDatabase(acd *dbv1alpha1.AutonomousContainerDatabase) (database.CreateAutonomousContainerDatabaseResponse, error) { + createAutonomousContainerDatabaseRequest := database.CreateAutonomousContainerDatabaseRequest{ + CreateAutonomousContainerDatabaseDetails: database.CreateAutonomousContainerDatabaseDetails{ + CompartmentId: acd.Spec.CompartmentOCID, + DisplayName: acd.Spec.DisplayName, + CloudAutonomousVmClusterId: acd.Spec.AutonomousExadataVMClusterOCID, + PatchModel: database.CreateAutonomousContainerDatabaseDetailsPatchModelUpdates, + }, + } + + return d.dbClient.CreateAutonomousContainerDatabase(context.TODO(), createAutonomousContainerDatabaseRequest) +} + +func (d *databaseService) GetAutonomousContainerDatabase(acdOCID string) (database.GetAutonomousContainerDatabaseResponse, error) { + getAutonomousContainerDatabaseRequest := database.GetAutonomousContainerDatabaseRequest{ + AutonomousContainerDatabaseId: common.String(acdOCID), + } + + return d.dbClient.GetAutonomousContainerDatabase(context.TODO(), getAutonomousContainerDatabaseRequest) +} + +func (d *databaseService) UpdateAutonomousContainerDatabase(acd *dbv1alpha1.AutonomousContainerDatabase) (database.UpdateAutonomousContainerDatabaseResponse, error) { + updateAutonomousContainerDatabaseRequest := database.UpdateAutonomousContainerDatabaseRequest{ + AutonomousContainerDatabaseId: acd.Spec.AutonomousContainerDatabaseOCID, + UpdateAutonomousContainerDatabaseDetails: database.UpdateAutonomousContainerDatabaseDetails{ + DisplayName: acd.Spec.DisplayName, + PatchModel: database.UpdateAutonomousContainerDatabaseDetailsPatchModelEnum(acd.Spec.PatchModel), + FreeformTags: acd.Spec.FreeformTags, + }, + } + + return d.dbClient.UpdateAutonomousContainerDatabase(context.TODO(), updateAutonomousContainerDatabaseRequest) +} + +func (d *databaseService) TerminateAutonomousContainerDatabase(acdOCID string) (database.TerminateAutonomousContainerDatabaseResponse, error) { + terminateRequest := database.TerminateAutonomousContainerDatabaseRequest{ + AutonomousContainerDatabaseId: common.String(acdOCID), + } + + return d.dbClient.TerminateAutonomousContainerDatabase(context.TODO(), terminateRequest) +} diff --git a/commons/oci/database.go b/commons/oci/database.go index 4a092197..c4477984 100644 --- a/commons/oci/database.go +++ b/commons/oci/database.go @@ -74,6 +74,7 @@ type DatabaseService interface { CreateAutonomousContainerDatabase(acb *dbv1alpha1.AutonomousContainerDatabase) (database.CreateAutonomousContainerDatabaseResponse, error) GetAutonomousContainerDatabase(acdOCID string) (database.GetAutonomousContainerDatabaseResponse, error) UpdateAutonomousContainerDatabase(acd *dbv1alpha1.AutonomousContainerDatabase) (database.UpdateAutonomousContainerDatabaseResponse, error) + TerminateAutonomousContainerDatabase(acdOCID string) (database.TerminateAutonomousContainerDatabaseResponse, error) } type databaseService struct { @@ -115,11 +116,11 @@ func NewDatabaseService( func (d *databaseService) readPassword(namespace string, passwordSpec dbv1alpha1.PasswordSpec) (*string, error) { logger := d.logger.WithName("read-password") - if passwordSpec.K8sSecretName != nil { - logger.Info(fmt.Sprintf("Getting password from Secret %s", *passwordSpec.K8sSecretName)) + if passwordSpec.K8sSecret.Name != nil { + logger.Info(fmt.Sprintf("Getting password from Secret %s", *passwordSpec.K8sSecret.Name)) - key := *passwordSpec.K8sSecretName - password, err := k8s.GetSecretValue(d.kubeClient, namespace, *passwordSpec.K8sSecretName, key) + key := *passwordSpec.K8sSecret.Name + password, err := k8s.GetSecretValue(d.kubeClient, namespace, *passwordSpec.K8sSecret.Name, key) if err != nil { return nil, err } @@ -127,10 +128,10 @@ func (d *databaseService) readPassword(namespace string, passwordSpec dbv1alpha1 return common.String(password), nil } - if passwordSpec.OCISecretOCID != nil { - logger.Info(fmt.Sprintf("Getting password from OCI Vault Secret OCID %s", *passwordSpec.OCISecretOCID)) + if passwordSpec.OCISecret.OCID != nil { + logger.Info(fmt.Sprintf("Getting password from OCI Vault Secret OCID %s", *passwordSpec.OCISecret.OCID)) - password, err := d.vaultService.GetSecretValue(*passwordSpec.OCISecretOCID) + password, err := d.vaultService.GetSecretValue(*passwordSpec.OCISecret.OCID) if err != nil { return nil, err } @@ -140,6 +141,23 @@ func (d *databaseService) readPassword(namespace string, passwordSpec dbv1alpha1 return nil, nil } +func (d *databaseService) readACD_OCID(acd *dbv1alpha1.ACDSpec, namespace string) (*string, error) { + if acd.OCIACD.OCID != nil { + return acd.OCIACD.OCID, nil + } + + if acd.K8sACD.Name != nil { + fetchedACD := &dbv1alpha1.AutonomousContainerDatabase{} + if err := k8s.FetchResource(d.kubeClient, namespace, *acd.K8sACD.Name, fetchedACD); err != nil { + return nil, err + } + + return fetchedACD.Spec.AutonomousContainerDatabaseOCID, nil + } + + return nil, nil +} + // CreateAutonomousDatabase sends a request to OCI to provision a database and returns the AutonomousDatabase OCID. func (d *databaseService) CreateAutonomousDatabase(adb *dbv1alpha1.AutonomousDatabase) (resp database.CreateAutonomousDatabaseResponse, err error) { adminPassword, err := d.readPassword(adb.Namespace, adb.Spec.Details.AdminPassword) @@ -147,9 +165,13 @@ func (d *databaseService) CreateAutonomousDatabase(adb *dbv1alpha1.AutonomousDat return resp, err } + acdOCID, err := d.readACD_OCID(&adb.Spec.Details.AutonomousContainerDatabase, adb.Namespace) + if err != nil { + return resp, err + } + createAutonomousDatabaseDetails := database.CreateAutonomousDatabaseDetails{ CompartmentId: adb.Spec.Details.CompartmentOCID, - AutonomousContainerDatabaseId: adb.Spec.Details.AutonomousContainerDatabaseOCID, DbName: adb.Spec.Details.DbName, CpuCoreCount: adb.Spec.Details.CPUCoreCount, DataStorageSizeInTBs: adb.Spec.Details.DataStorageSizeInTBs, @@ -157,6 +179,7 @@ func (d *databaseService) CreateAutonomousDatabase(adb *dbv1alpha1.AutonomousDat DisplayName: adb.Spec.Details.DisplayName, IsAutoScalingEnabled: adb.Spec.Details.IsAutoScalingEnabled, IsDedicated: adb.Spec.Details.IsDedicated, + AutonomousContainerDatabaseId: acdOCID, DbVersion: adb.Spec.Details.DbVersion, DbWorkload: database.CreateAutonomousDatabaseBaseDbWorkloadEnum( adb.Spec.Details.DbWorkload), diff --git a/commons/oci/workrequest.go b/commons/oci/workrequest.go index 4946bc86..3edf5352 100644 --- a/commons/oci/workrequest.go +++ b/commons/oci/workrequest.go @@ -51,6 +51,7 @@ import ( ) type WorkRequestService interface { + Get(opcWorkRequestID string) (workrequests.WorkRequestStatusEnum, error) Wait(opcWorkRequestID string) (workrequests.WorkRequestStatusEnum, error) } @@ -93,8 +94,8 @@ func (w workRequestService) getRetryPolicy() common.RetryPolicy { return true } - // maximum times of retry (~30mins) - attempts := uint(63) + // maximum times of retry (~60mins) + attempts := uint(124) nextDuration := func(r common.OCIOperationResponse) time.Duration { // Wait longer for next retry when your previous one failed @@ -129,3 +130,16 @@ func (w *workRequestService) Wait(opcWorkRequestID string) (workrequests.WorkReq return resp.Status, nil } + +func (w *workRequestService) Get(opcWorkRequestID string) (workrequests.WorkRequestStatusEnum, error) { + workRequest := workrequests.GetWorkRequestRequest{ + WorkRequestId: common.String(opcWorkRequestID), + } + + resp, err := w.workClient.GetWorkRequest(context.TODO(), workRequest) + if err != nil { + return resp.Status, err + } + + return resp.Status, nil +} diff --git a/config/crd/bases/database.oracle.com_autonomouscontainerdatabases.yaml b/config/crd/bases/database.oracle.com_autonomouscontainerdatabases.yaml new file mode 100644 index 00000000..7ea17e10 --- /dev/null +++ b/config/crd/bases/database.oracle.com_autonomouscontainerdatabases.yaml @@ -0,0 +1,111 @@ + +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.6.1 + creationTimestamp: null + name: autonomouscontainerdatabases.database.oracle.com +spec: + group: database.oracle.com + names: + kind: AutonomousContainerDatabase + listKind: AutonomousContainerDatabaseList + plural: autonomouscontainerdatabases + shortNames: + - acd + - acds + singular: autonomouscontainerdatabase + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .spec.displayName + name: DisplayName + type: string + - jsonPath: .status.lifecycleState + name: State + type: string + - jsonPath: .status.timeCreated + name: Created + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + description: AutonomousContainerDatabase is the Schema for the autonomouscontainerdatabases + API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: AutonomousContainerDatabaseSpec defines the desired state + of AutonomousContainerDatabase + properties: + autonomousContainerDatabaseOCID: + description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster + Important: Run "make" to regenerate code after modifying this file' + type: string + autonomousExadataVMClusterOCID: + type: string + compartmentOCID: + type: string + displayName: + type: string + freeformTags: + additionalProperties: + type: string + type: object + hardLink: + default: false + type: boolean + ociConfig: + description: "*********************** *\tOCI config ***********************" + properties: + configMapName: + type: string + secretName: + type: string + type: object + patchModel: + description: 'AutonomousContainerDatabasePatchModelEnum Enum with + underlying type: string' + enum: + - RELEASE_UPDATES + - RELEASE_UPDATE_REVISIONS + type: string + type: object + status: + description: AutonomousContainerDatabaseStatus defines the observed state + of AutonomousContainerDatabase + properties: + lifecycleState: + description: 'INSERT ADDITIONAL STATUS FIELD - define observed state + of cluster Important: Run "make" to regenerate code after modifying + this file' + type: string + timeCreated: + type: string + required: + - lifecycleState + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] diff --git a/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml b/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml index c714fe46..1c34f7fa 100644 --- a/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml +++ b/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml @@ -62,32 +62,42 @@ spec: displayName: type: string ociConfig: + description: "*********************** *\tOCI config ***********************" properties: configMapName: type: string secretName: type: string type: object - targetADB: + target: description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster Important: Run "make" to regenerate code after modifying this file' properties: - name: - type: string - ocid: - type: string - required: - - name - - ocid + k8sADB: + description: "*********************** *\tADB spec ***********************" + properties: + name: + type: string + type: object + ociADB: + properties: + ocid: + type: string + type: object type: object required: - displayName - - targetADB + - target type: object status: description: AutonomousDatabaseBackupStatus defines the observed state of AutonomousDatabaseBackup properties: + autonomousDatabaseBackupOCID: + description: 'INSERT ADDITIONAL STATUS FIELD - define observed state + of cluster Important: Run "make" to regenerate code after modifying + this file' + type: string autonomousDatabaseOCID: type: string compartmentOCID: @@ -96,12 +106,13 @@ spec: type: string dbName: type: string + displayName: + type: string isAutomatic: type: boolean lifecycleState: - description: 'INSERT ADDITIONAL STATUS FIELD - define observed state - of cluster Important: Run "make" to regenerate code after modifying - this file' + description: 'AutonomousDatabaseBackupLifecycleStateEnum Enum with + underlying type: string' type: string timeEnded: type: string @@ -116,6 +127,7 @@ spec: - compartmentOCID - dbDisplayName - dbName + - displayName - isAutomatic - lifecycleState - type diff --git a/config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml b/config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml index 055bc148..a6ecf6d1 100644 --- a/config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml +++ b/config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml @@ -52,37 +52,50 @@ spec: AutonomousDatabaseRestore properties: ociConfig: + description: "*********************** *\tOCI config ***********************" properties: configMapName: type: string secretName: type: string type: object - sourceSpec: + source: properties: - autonomousDatabaseBackup: + k8sADBBackup: + description: 'EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO + OWN! NOTE: json tags are required. Any new fields you add must + have json tags for the fields to be serialized.' properties: name: type: string type: object pointInTime: - type: string + properties: + timestamp: + description: 'The timestamp must follow this format: YYYY-MM-DD + HH:MM:SS GMT' + type: string + type: object type: object - targetADB: + target: description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster Important: Run "make" to regenerate code after modifying this file' properties: - name: - type: string - ocid: - type: string - required: - - name - - ocid + k8sADB: + description: "*********************** *\tADB spec ***********************" + properties: + name: + type: string + type: object + ociADB: + properties: + ocid: + type: string + type: object type: object required: - - sourceSpec - - targetADB + - source + - target type: object status: description: AutonomousDatabaseRestoreStatus defines the observed state diff --git a/config/crd/bases/database.oracle.com_autonomousdatabases.yaml b/config/crd/bases/database.oracle.com_autonomousdatabases.yaml index 168acd67..1701111c 100644 --- a/config/crd/bases/database.oracle.com_autonomousdatabases.yaml +++ b/config/crd/bases/database.oracle.com_autonomousdatabases.yaml @@ -69,13 +69,35 @@ spec: properties: adminPassword: properties: - k8sSecretName: - type: string - ociSecretOCID: - type: string + k8sSecret: + description: "*********************** *\tSecret specs ***********************" + properties: + name: + type: string + type: object + ociSecret: + properties: + ocid: + type: string + type: object + type: object + autonomousContainerDatabase: + description: ACDSpec defines the spec of the target for backup/restore + runs. The name could be the name of an AutonomousDatabase or + an AutonomousDatabaseBackup + properties: + k8sACD: + description: "*********************** *\tACD specs ***********************" + properties: + name: + type: string + type: object + ociACD: + properties: + ocid: + type: string + type: object type: object - autonomousContainerDatabaseOCID: - type: string autonomousDatabaseOCID: type: string compartmentOCID: @@ -153,10 +175,18 @@ spec: type: string password: properties: - k8sSecretName: - type: string - ociSecretOCID: - type: string + k8sSecret: + description: "*********************** *\tSecret specs + ***********************" + properties: + name: + type: string + type: object + ociSecret: + properties: + ocid: + type: string + type: object type: object type: object type: object @@ -164,6 +194,7 @@ spec: default: false type: boolean ociConfig: + description: "*********************** *\tOCI config ***********************" properties: configMapName: type: string diff --git a/config/crd/kustomization.yaml b/config/crd/kustomization.yaml index edd4806d..fa930212 100644 --- a/config/crd/kustomization.yaml +++ b/config/crd/kustomization.yaml @@ -14,6 +14,7 @@ resources: - bases/database.oracle.com_pdbs.yaml - bases/database.oracle.com_cdbs.yaml - bases/database.oracle.com_oraclerestdataservices.yaml +- bases/database.oracle.com_autonomouscontainerdatabases.yaml # +kubebuilder:scaffold:crdkustomizeresource patchesStrategicMerge: @@ -26,6 +27,7 @@ patchesStrategicMerge: #- patches/webhook_in_pdbs.yaml #- patches/webhook_in_cdbs.yaml #- patches/webhook_in_oraclerestdataservices.yaml +#- patches/webhook_in_autonomouscontainerdatabases.yaml # +kubebuilder:scaffold:crdkustomizewebhookpatch # [CERTMANAGER] To enable webhook, uncomment all the sections with [CERTMANAGER] prefix. @@ -37,6 +39,7 @@ patchesStrategicMerge: - patches/cainjection_in_pdbs.yaml - patches/cainjection_in_cdbs.yaml #- patches/cainjection_in_oraclerestdataservices.yaml +#- patches/cainjection_in_autonomouscontainerdatabases.yaml # +kubebuilder:scaffold:crdkustomizecainjectionpatch # the following config is for teaching kustomize how to do kustomization for CRDs. diff --git a/config/crd/patches/cainjection_in_autonomouscontainerdatabases.yaml b/config/crd/patches/cainjection_in_autonomouscontainerdatabases.yaml new file mode 100644 index 00000000..3985a5ae --- /dev/null +++ b/config/crd/patches/cainjection_in_autonomouscontainerdatabases.yaml @@ -0,0 +1,8 @@ +# The following patch adds a directive for certmanager to inject CA into the CRD +# CRD conversion requires k8s 1.13 or later. +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + annotations: + cert-manager.io/inject-ca-from: $(CERTIFICATE_NAMESPACE)/$(CERTIFICATE_NAME) + name: autonomouscontainerdatabases.database.oracle.com diff --git a/config/crd/patches/webhook_in_autonomouscontainerdatabases.yaml b/config/crd/patches/webhook_in_autonomouscontainerdatabases.yaml new file mode 100644 index 00000000..03a73384 --- /dev/null +++ b/config/crd/patches/webhook_in_autonomouscontainerdatabases.yaml @@ -0,0 +1,17 @@ +# The following patch enables conversion webhook for CRD +# CRD conversion requires k8s 1.13 or later. +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: autonomouscontainerdatabases.database.oracle.com +spec: + conversion: + strategy: Webhook + webhookClientConfig: + # this is "\n" used as a placeholder, otherwise it will be rejected by the apiserver for being blank, + # but we're going to set it later using the cert-manager (or potentially a patch if not using cert-manager) + caBundle: Cg== + service: + namespace: system + name: webhook-service + path: /convert diff --git a/config/rbac/autonomouscontainerdatabase_editor_role.yaml b/config/rbac/autonomouscontainerdatabase_editor_role.yaml new file mode 100644 index 00000000..8dccd4a5 --- /dev/null +++ b/config/rbac/autonomouscontainerdatabase_editor_role.yaml @@ -0,0 +1,24 @@ +# permissions for end users to edit autonomouscontainerdatabases. +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: autonomouscontainerdatabase-editor-role +rules: +- apiGroups: + - database.oracle.com + resources: + - autonomouscontainerdatabases + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - database.oracle.com + resources: + - autonomouscontainerdatabases/status + verbs: + - get diff --git a/config/rbac/autonomouscontainerdatabase_viewer_role.yaml b/config/rbac/autonomouscontainerdatabase_viewer_role.yaml new file mode 100644 index 00000000..e9bcec50 --- /dev/null +++ b/config/rbac/autonomouscontainerdatabase_viewer_role.yaml @@ -0,0 +1,20 @@ +# permissions for end users to view autonomouscontainerdatabases. +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: autonomouscontainerdatabase-viewer-role +rules: +- apiGroups: + - database.oracle.com + resources: + - autonomouscontainerdatabases + verbs: + - get + - list + - watch +- apiGroups: + - database.oracle.com + resources: + - autonomouscontainerdatabases/status + verbs: + - get diff --git a/config/rbac/role.yaml b/config/rbac/role.yaml index 4b08b9e7..897e0e77 100644 --- a/config/rbac/role.yaml +++ b/config/rbac/role.yaml @@ -146,12 +146,22 @@ rules: - apiGroups: - database.oracle.com resources: - - autonomousdatabaseBackups + - autonomouscontainerdatabases verbs: - create - delete - get - list + - patch + - update + - watch +- apiGroups: + - database.oracle.com + resources: + - autonomouscontainerdatabases/status + verbs: + - get + - patch - update - apiGroups: - database.oracle.com @@ -162,6 +172,7 @@ rules: - delete - get - list + - update - watch - apiGroups: - database.oracle.com @@ -180,6 +191,7 @@ rules: - delete - get - list + - update - watch - apiGroups: - database.oracle.com @@ -345,4 +357,4 @@ rules: verbs: - get - patch - - update \ No newline at end of file + - update diff --git a/config/samples/acd/autonomouscontainerdatabase_create.yaml b/config/samples/acd/autonomouscontainerdatabase_create.yaml new file mode 100644 index 00000000..8ee34a04 --- /dev/null +++ b/config/samples/acd/autonomouscontainerdatabase_create.yaml @@ -0,0 +1,20 @@ +apiVersion: database.oracle.com/v1alpha1 +kind: AutonomousContainerDatabase +metadata: + name: autonomouscontainerdatabase-sample +spec: + # Update compartmentOCID with your compartment OCID. + compartmentOCID: ocid1.compartment... OR ocid1.tenancy... + displayName: newACD + autonomousExadataVMClusterOCID: ocid1.autonomousexainfrastructure... + patchModel: RELEASE_UPDATES # optional + + freeformTags: + key1: val1 + + hardLink: false + + # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. + ociConfig: + configMapName: oci-cred-dedicated + secretName: oci-privatekey-dedicated diff --git a/config/samples/adb/autonomousdatabase.yaml b/config/samples/adb/autonomousdatabase.yaml deleted file mode 100644 index c1dfc078..00000000 --- a/config/samples/adb/autonomousdatabase.yaml +++ /dev/null @@ -1,11 +0,0 @@ -# -# Copyright (c) 2021, Oracle and/or its affiliates. -# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. -# -apiVersion: database.oracle.com/v1alpha1 -kind: AutonomousDatabase -metadata: - name: autonomousdatabase-sample -spec: - # Add fields here - foo: bar diff --git a/config/samples/adb/autonomousdatabase_backup.yaml b/config/samples/adb/autonomousdatabase_backup.yaml index bd671910..f478faaf 100644 --- a/config/samples/adb/autonomousdatabase_backup.yaml +++ b/config/samples/adb/autonomousdatabase_backup.yaml @@ -5,7 +5,13 @@ metadata: spec: # Before you can create manual backups, you must have an Object Storage bucket and your database must be configured to connect to it. This is a one-time operation. # See https://docs.oracle.com/en-us/iaas/Content/Database/Tasks/adbbackingup.htm#creatingbucket - autonomousDatabaseOCID: ocid1.autonomousdatabase... + target: + k8sADB: + name: autonomousdatabase-sample + # # Uncomment the below block if you use ADB OCID as the input of the target + # ociADB: + # ocid: ocid1.autonomousdatabase... + displayName: autonomousdatabasebackup-sample # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. ociConfig: diff --git a/config/samples/adb/autonomousdatabase_restore.yaml b/config/samples/adb/autonomousdatabase_restore.yaml index 401b03a2..96e5bcc7 100644 --- a/config/samples/adb/autonomousdatabase_restore.yaml +++ b/config/samples/adb/autonomousdatabase_restore.yaml @@ -5,13 +5,19 @@ metadata: spec: # Restore the database either from a backup or using point-in-time restore # The name of your AutonomousDatabaseBackup resource - backupName: autonomousdatabasebackup-sample - - # Uncomment the following block to perform point-in-time restore - # pointInTime: - # autonomousDatabaseOCID: ocid1.autonomousdatabase... - # # The timestamp must follow this format: YYYY-MM-DD HH:MM:SS GMT - # timeStamp: 2021-10-23 11:03:13 UTC + target: + k8sADB: + name: autonomousdatabase-sample + # # Uncomment the below block if you use ADB OCID as the input of the target + # ociADB: + # ocid: ocid1.autonomousdatabase... + source: + k8sADBBackup: + name: autonomousdatabasebackup-sample + # # Uncomment the following field to perform point-in-time restore + # pointInTime: + # # The timestamp must follow this format: YYYY-MM-DD HH:MM:SS GMT + # timestamp: 2021-12-23 11:03:13 UTC # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. ociConfig: diff --git a/config/samples/kustomization.yaml b/config/samples/kustomization.yaml index 214006e2..9e0f5d41 100644 --- a/config/samples/kustomization.yaml +++ b/config/samples/kustomization.yaml @@ -11,7 +11,7 @@ resources: - adb/autonomousdatabase_bind.yaml - adb/autonomousdatabase_backup.yaml - adb/autonomousdatabase_restore.yaml - - adb/autonomouscontainerdatabase.yaml + - acd/autonomouscontainerdatabase_create.yaml - sidb/singleinstancedatabase.yaml - sharding/shardingdatabase.yaml - sharding/sharding_v1alpha1_provshard.yaml diff --git a/controllers/database/autonomouscontainerdatabase_controller.go b/controllers/database/autonomouscontainerdatabase_controller.go new file mode 100644 index 00000000..f905100c --- /dev/null +++ b/controllers/database/autonomouscontainerdatabase_controller.go @@ -0,0 +1,535 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package controllers + +import ( + "context" + "encoding/json" + "reflect" + + "github.com/go-logr/logr" + "github.com/oracle/oci-go-sdk/v54/database" + + corev1 "k8s.io/api/core/v1" + apiErrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/types" + "k8s.io/client-go/tools/record" + "k8s.io/client-go/util/retry" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/controller" + "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" + "sigs.k8s.io/controller-runtime/pkg/event" + "sigs.k8s.io/controller-runtime/pkg/predicate" + + dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + "github.com/oracle/oracle-database-operator/commons/annotations" + "github.com/oracle/oracle-database-operator/commons/k8s" + "github.com/oracle/oracle-database-operator/commons/oci" +) + +// AutonomousContainerDatabaseReconciler reconciles a AutonomousContainerDatabase object +type AutonomousContainerDatabaseReconciler struct { + KubeClient client.Client + Log logr.Logger + Scheme *runtime.Scheme + Recorder record.EventRecorder + + adbService oci.DatabaseService + workService oci.WorkRequestService + lastSucSpec *dbv1alpha1.AutonomousContainerDatabaseSpec +} + +// SetupWithManager sets up the controller with the Manager. +func (r *AutonomousContainerDatabaseReconciler) SetupWithManager(mgr ctrl.Manager) error { + return ctrl.NewControllerManagedBy(mgr). + For(&dbv1alpha1.AutonomousContainerDatabase{}). + WithEventFilter(r.eventFilterPredicate()). + WithOptions(controller.Options{MaxConcurrentReconciles: 5}). + Complete(r) +} + +func (r *AutonomousContainerDatabaseReconciler) isIntermediateState(state database.AutonomousContainerDatabaseLifecycleStateEnum) bool { + if state == database.AutonomousContainerDatabaseLifecycleStateProvisioning || + state == database.AutonomousContainerDatabaseLifecycleStateUpdating || + state == database.AutonomousContainerDatabaseLifecycleStateTerminating || + state == database.AutonomousContainerDatabaseLifecycleStateBackupInProgress || + state == database.AutonomousContainerDatabaseLifecycleStateRestoring || + state == database.AutonomousContainerDatabaseLifecycleStateRestarting || + state == database.AutonomousContainerDatabaseLifecycleStateMaintenanceInProgress || + state == database.AutonomousContainerDatabaseLifecycleStateRoleChangeInProgress { + return true + } + return false +} + +func (r *AutonomousContainerDatabaseReconciler) eventFilterPredicate() predicate.Predicate { + pred := predicate.Funcs{ + UpdateFunc: func(e event.UpdateEvent) bool { + desiredACD, acdOk := e.ObjectNew.(*dbv1alpha1.AutonomousContainerDatabase) + if acdOk { + oldACD := e.ObjectOld.(*dbv1alpha1.AutonomousContainerDatabase) + + oldSucSpec, oldSucSpecOk := oldACD.GetAnnotations()[dbv1alpha1.LastSuccessfulSpec] + newSucSpec, newSucSpecOk := desiredACD.GetAnnotations()[dbv1alpha1.LastSuccessfulSpec] + sucSpecChanged := oldSucSpecOk != newSucSpecOk || oldSucSpec != newSucSpec + + if !reflect.DeepEqual(oldACD.Status, desiredACD.Status) || sucSpecChanged { + // Don't enqueue if the status or the lastSucSpec changes + return false + } + + return true + } + + return true + }, + DeleteFunc: func(e event.DeleteEvent) bool { + // Do not trigger reconciliation when the object is deleted from the cluster. + _, acdOk := e.Object.(*dbv1alpha1.AutonomousContainerDatabase) + if acdOk { + return false + } + + return true + }, + } + + return pred +} + +//+kubebuilder:rbac:groups=database.oracle.com,resources=autonomouscontainerdatabases,verbs=get;list;watch;create;update;patch;delete +//+kubebuilder:rbac:groups=database.oracle.com,resources=autonomouscontainerdatabases/status,verbs=get;update;patch +//+kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabases,verbs=get;list;watch;create;update;delete + +// Reconcile is part of the main kubernetes reconciliation loop which aims to +// move the current state of the cluster closer to the desired state. +// +// For more details, check Reconcile and its Result here: +// - https://pkg.go.dev/sigs.k8s.io/controller-runtime@v0.6.4/pkg/reconcile +func (r *AutonomousContainerDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + logger := r.Log.WithValues("Namespaced/Name", req.NamespacedName) + + var err error + + // Get the autonomousdatabase instance from the cluster + acd := &dbv1alpha1.AutonomousContainerDatabase{} + if err := r.KubeClient.Get(context.TODO(), req.NamespacedName, acd); err != nil { + // Ignore not-found errors, since they can't be fixed by an immediate requeue. + // No need to change the since we don't know if we obtain the object. + if apiErrors.IsNotFound(err) { + return emptyResult, nil + } + // Failed to get ADB, so we don't need to update the status + return emptyResult, err + } + + r.lastSucSpec, err = acd.GetLastSuccessfulSpec() + if err != nil { + return r.manageError(acd, err) + } + + /****************************************************************** + * Get OCI database client and work request client + ******************************************************************/ + if err := r.setupOCIClients(acd); err != nil { + logger.Error(err, "Fail to setup OCI clients") + + return r.manageError(acd, err) + } + + logger.Info("OCI clients configured succesfully") + + /****************************************************************** + * Register/unregister finalizer + * Deletion timestamp will be added to a object before it is deleted. + * Kubernetes server calls the clean up function if a finalizer exitsts, and won't delete the real object until + * all the finalizers are removed from the object metadata. + * Refer to this page for more details of using finalizers: https://kubernetes.io/blog/2021/05/14/using-finalizers-to-control-deletion/ + ******************************************************************/ + isACDDeleteTrue, err := r.validateFinalizer(acd) + if err != nil { + return r.manageError(acd, err) + } + if isACDDeleteTrue { + return emptyResult, nil + } + + /****************************************************************** + * Determine which Database operations need to be executed by checking the changes to spec.details. + * There are three scenario: + * 1. provision operation. The AutonomousDatabaseOCID is missing, and the LastSucSpec annotation is missing. + * 2. bind operation. The AutonomousDatabaseOCID is provided, but the LastSucSpec annotation is missing. + * 3. update operation. Every changes other than the above two cases goes here. + * Afterwards, update the resource from the remote database in OCI. This step will be executed right after + * the above three cases during every reconcile. + /******************************************************************/ + // difACD is nil when the action is PROVISION or BIND. + // Use difACD to identify which fields are updated when it's UPDATE operation. + action, difACD, err := r.determineAction(acd) + if err != nil { + return r.manageError(acd, err) + } + + switch action { + case acdActionProvision: + if err := r.createACD(acd); err != nil { + return r.manageError(acd, err) + } + case acdActionBind: + break + case acdActionUpdate: + // updateADB contains downloadWallet + if err := r.updateACD(acd, difACD); err != nil { + return r.manageError(acd, err) + } + case acdActionSync: + break + } + + /***************************************************** + * Sync resource and update the lastSucSpec + *****************************************************/ + err = r.syncResource(acd) + if err != nil { + return r.manageError(acd, err) + } + + logger.Info("AutonomousContainerDatabase reconciles successfully") + + return emptyResult, nil +} + +// updateResourceStatus updates only the status of the resource, not including the lastSucSpec. +// This function should not be called by the functions associated with the OCI update requests. +// The OCI update requests should use updateResource() to ensure all the spec, resource and the +// lastSucSpec are updated. +func (r *AutonomousContainerDatabaseReconciler) updateResourceStatus(acd *dbv1alpha1.AutonomousContainerDatabase) error { + return retry.RetryOnConflict(retry.DefaultRetry, func() error { + curACD := &dbv1alpha1.AutonomousContainerDatabase{} + + namespacedName := types.NamespacedName{ + Namespace: acd.GetNamespace(), + Name: acd.GetName(), + } + + if err := r.KubeClient.Get(context.TODO(), namespacedName, curACD); err != nil { + return err + } + + curACD.Status = acd.Status + return r.KubeClient.Status().Update(context.TODO(), curACD) + }) +} + +// updateResource updates the specification, the status of AutonomousContainerDatabase resource, and the lastSucSpec +func (r *AutonomousContainerDatabaseReconciler) updateResource(adb *dbv1alpha1.AutonomousContainerDatabase) error { + // Update the status first to prevent unwanted Reconcile() + if err := r.updateResourceStatus(adb); err != nil { + return err + } + + // Update the spec + if err := retry.RetryOnConflict(retry.DefaultRetry, func() error { + curACD := &dbv1alpha1.AutonomousContainerDatabase{} + + namespacedName := types.NamespacedName{ + Namespace: adb.GetNamespace(), + Name: adb.GetName(), + } + + if err := r.KubeClient.Get(context.TODO(), namespacedName, curACD); err != nil { + return err + } + + curACD.Spec = adb.Spec + return r.KubeClient.Update(context.TODO(), curACD) + }); err != nil { + return err + } + + return nil +} + +// syncResource pulled information from OCI, update the LastSucSpec, and lastly, update the local resource. +// It's important to update the LastSucSpec prior than we update the local resource, +// because updating the local resource triggers the Reconcile, and the Reconcile determines the action by +// looking if the lastSucSpec is present. +func (r *AutonomousContainerDatabaseReconciler) syncResource(acd *dbv1alpha1.AutonomousContainerDatabase) error { + // Get the information from OCI + resp, err := r.adbService.GetAutonomousContainerDatabase(*acd.Spec.AutonomousContainerDatabaseOCID) + if err != nil { + return err + } + + acd.UpdateFromOCIACD(resp.AutonomousContainerDatabase) + + if err := r.updateResource(acd); err != nil { + return err + } + + // Update the lastSucSpec + if err := r.updateLastSuccessfulSpec(acd); err != nil { + return err + } + + if err := r.updateResource(acd); err != nil { + return err + } + + return nil +} + +func (r *AutonomousContainerDatabaseReconciler) updateLastSuccessfulSpec(acd *dbv1alpha1.AutonomousContainerDatabase) error { + specBytes, err := json.Marshal(acd.Spec) + if err != nil { + return err + } + + anns := map[string]string{ + dbv1alpha1.LastSuccessfulSpec: string(specBytes), + } + + return annotations.PatchAnnotations(r.KubeClient, acd, anns) +} + +func (r *AutonomousContainerDatabaseReconciler) manageError(acd *dbv1alpha1.AutonomousContainerDatabase, issue error) (ctrl.Result, error) { + // Rollback if lastSucSpec exists + if r.lastSucSpec != nil { + // Send event + r.Recorder.Event(acd, corev1.EventTypeWarning, "ReconcileIncompleted", issue.Error()) + + var finalIssue = issue + + if err := r.syncResource(acd); err != nil { + finalIssue = k8s.CombineErrors(finalIssue, err) + } + + if err := r.updateLastSuccessfulSpec(acd); err != nil { + finalIssue = k8s.CombineErrors(finalIssue, err) + } + + // r.updateResource has already triggered another Reconcile loop, so we simply log the error and return a nil + return emptyResult, finalIssue + } else { + // Send event + r.Recorder.Event(acd, corev1.EventTypeWarning, "CreateFailed", issue.Error()) + + return emptyResult, issue + } +} + +func (r *AutonomousContainerDatabaseReconciler) setupOCIClients(acd *dbv1alpha1.AutonomousContainerDatabase) error { + var err error + + authData := oci.APIKeyAuth{ + ConfigMapName: acd.Spec.OCIConfig.ConfigMapName, + SecretName: acd.Spec.OCIConfig.SecretName, + Namespace: acd.GetNamespace(), + } + + provider, err := oci.GetOCIProvider(r.KubeClient, authData) + if err != nil { + return err + } + + r.adbService, err = oci.NewDatabaseService(r.Log, r.KubeClient, provider) + if err != nil { + return err + } + + r.workService, err = oci.NewWorkRequestService(r.Log, r.KubeClient, provider) + if err != nil { + return err + } + + return nil +} + +func (r *AutonomousContainerDatabaseReconciler) validateFinalizer(acd *dbv1alpha1.AutonomousContainerDatabase) (isACDDeleteTrue bool, err error) { + logger := r.Log.WithName("finalizer") + + isACDToBeDeleted := acd.GetDeletionTimestamp() != nil + if isACDToBeDeleted { + if controllerutil.ContainsFinalizer(acd, dbv1alpha1.ACDFinalizer) { + if (r.lastSucSpec != nil && r.lastSucSpec.AutonomousContainerDatabaseOCID != nil) && // lastSucSpec exists and the ACD_OCID isn't nil + (acd.Status.LifecycleState != database.AutonomousContainerDatabaseLifecycleStateTerminating && acd.Status.LifecycleState != database.AutonomousContainerDatabaseLifecycleStateTerminated) { + // Run finalization logic for finalizer. If the finalization logic fails, don't remove the finalizer so + // that we can retry during the next reconciliation. + logger.Info("Terminating Autonomous Container Database: " + *acd.Spec.DisplayName) + if err := r.terminateACD(acd); err != nil { + return false, err + } + } else { + logger.Info("Missing AutonomousContaineratabaseOCID to terminate Autonomous Container Database", "Name", acd.Name, "Namespace", acd.Namespace) + } + + // Remove finalizer. Once all finalizers have been + // removed, the object will be deleted. + if err := k8s.RemoveFinalizerAndPatch(r.KubeClient, acd, dbv1alpha1.ACDFinalizer); err != nil { + return false, nil + } + } + // Send true because delete is in progress and it is a custom delete message + // We don't need to print custom err stack as we are deleting the topology + return true, nil + } + + // Delete is not schduled. Update the finalizer for this CR if hardLink is present + if acd.Spec.HardLink != nil { + if *acd.Spec.HardLink { + if err := k8s.AddFinalizerAndPatch(r.KubeClient, acd, dbv1alpha1.ACDFinalizer); err != nil { + return false, nil + } + } else { + if err := k8s.RemoveFinalizerAndPatch(r.KubeClient, acd, dbv1alpha1.ACDFinalizer); err != nil { + return false, nil + } + } + } + + return false, nil +} + +type ACDActionEnum string + +const ( + acdActionProvision = "PROVISION" + acdActionBind = "BIND" + acdActionUpdate = "UPDATE" + acdActionSync = "SYNC" +) + +func (r *AutonomousContainerDatabaseReconciler) determineAction(adb *dbv1alpha1.AutonomousContainerDatabase) (ACDActionEnum, *dbv1alpha1.AutonomousContainerDatabase, error) { + if r.lastSucSpec == nil { + if adb.Spec.AutonomousContainerDatabaseOCID == nil { + return acdActionProvision, nil, nil + } else { + return acdActionBind, nil, nil + } + } else { + // Pre-process step for the UPDATE. Remove the unchanged fields from spec.details, + difACD := adb.DeepCopy() + specChanged, err := difACD.RemoveUnchangedSpec() + if err != nil { + return "", nil, err + } + + if specChanged { + return acdActionUpdate, difACD, nil + } + + return acdActionSync, nil, nil + } +} + +func (r *AutonomousContainerDatabaseReconciler) createACD(acd *dbv1alpha1.AutonomousContainerDatabase) error { + resp, err := r.adbService.CreateAutonomousContainerDatabase(acd) + if err != nil { + return err + } + + // Update the ADB OCID and the status + // The trick is to update the status first to prevent unwanted reconcile + acd.Spec.AutonomousContainerDatabaseOCID = resp.AutonomousContainerDatabase.Id + acd.UpdateStatusFromOCIACD(resp.AutonomousContainerDatabase) + + if err := r.updateResourceStatus(acd); err != nil { + return err + } + + // Patching is faster + if err := k8s.Patch(r.KubeClient, acd, "/spec/autonomousContainerDatabaseOCID", acd.Spec.AutonomousContainerDatabaseOCID); err != nil { + return err + } + + if _, err := r.workService.Wait(*resp.OpcWorkRequestId); err != nil { + return err + } + return nil +} + +func (r *AutonomousContainerDatabaseReconciler) updateACD(acd *dbv1alpha1.AutonomousContainerDatabase, difACD *dbv1alpha1.AutonomousContainerDatabase) error { + if difACD.Spec.DisplayName == nil && + difACD.Spec.PatchModel == "" && + difACD.Spec.FreeformTags == nil { + return nil + } + + resp, err := r.adbService.UpdateAutonomousContainerDatabase(difACD) + if err != nil { + return err + } + + // If the OpcWorkRequestId is nil (such as when the displayName is changed), + // no need to update the resource and wail until the work is done + if resp.OpcWorkRequestId == nil { + return nil + } + + acd.UpdateStatusFromOCIACD(resp.AutonomousContainerDatabase) + if err := r.updateResourceStatus(acd); err != nil { + return err + } + + if _, err := r.workService.Wait(*resp.OpcWorkRequestId); err != nil { + return err + } + return nil +} + +func (r *AutonomousContainerDatabaseReconciler) terminateACD(acd *dbv1alpha1.AutonomousContainerDatabase) error { + + resp, err := r.adbService.TerminateAutonomousContainerDatabase(*acd.Spec.AutonomousContainerDatabaseOCID) + if err != nil { + return err + } + + acd.Status.LifecycleState = database.AutonomousContainerDatabaseLifecycleStateTerminating + + if err := r.updateResourceStatus(acd); err != nil { + return err + } + + if _, err := r.workService.Wait(*resp.OpcWorkRequestId); err != nil { + return err + } + return nil +} diff --git a/controllers/database/autonomousdatabase_controller.go b/controllers/database/autonomousdatabase_controller.go index e1013068..ce9a1b27 100644 --- a/controllers/database/autonomousdatabase_controller.go +++ b/controllers/database/autonomousdatabase_controller.go @@ -43,6 +43,7 @@ import ( "encoding/json" "errors" "fmt" + "reflect" "regexp" "strings" @@ -58,8 +59,12 @@ import ( ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/controller-runtime/pkg/controller" + "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" "sigs.k8s.io/controller-runtime/pkg/event" + "sigs.k8s.io/controller-runtime/pkg/handler" "sigs.k8s.io/controller-runtime/pkg/predicate" + "sigs.k8s.io/controller-runtime/pkg/reconcile" + "sigs.k8s.io/controller-runtime/pkg/source" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" "github.com/oracle/oracle-database-operator/commons/annotations" @@ -67,8 +72,6 @@ import ( "github.com/oracle/oracle-database-operator/commons/oci" ) -// name of our custom finalizer -const finalizer = "database.oracle.com/oraoperator-finalizer" var emptyResult ctrl.Result = ctrl.Result{} // *AutonomousDatabaseReconciler reconciles a AutonomousDatabase object @@ -87,55 +90,116 @@ type AutonomousDatabaseReconciler struct { func (r *AutonomousDatabaseReconciler) SetupWithManager(mgr ctrl.Manager) error { return ctrl.NewControllerManagedBy(mgr). For(&dbv1alpha1.AutonomousDatabase{}). - WithEventFilter(predicate.And(predicate.GenerationChangedPredicate{}, r.eventFilterPredicate())). - // WithEventFilter(r.eventFilterPredicate()). + Watches( + &source.Kind{Type: &dbv1alpha1.AutonomousDatabaseBackup{}}, + handler.EnqueueRequestsFromMapFunc(r.enqueueMapFn()), + ). + Watches( + &source.Kind{Type: &dbv1alpha1.AutonomousDatabaseRestore{}}, + handler.EnqueueRequestsFromMapFunc(r.enqueueMapFn()), + ). + WithEventFilter(predicate.And(r.eventFilterPredicate(), r.watchPredicate())). WithOptions(controller.Options{MaxConcurrentReconciles: 50}). // ReconcileHandler is never invoked concurrently with the same object. Complete(r) } +func (r *AutonomousDatabaseReconciler) enqueueMapFn() handler.MapFunc { + return func(o client.Object) []reconcile.Request { + reqs := make([]reconcile.Request, len(o.GetOwnerReferences())) + + for _, owner := range o.GetOwnerReferences() { + reqs = append(reqs, reconcile.Request{ + NamespacedName: types.NamespacedName{ + Name: owner.Name, + Namespace: o.GetNamespace(), + }, + }) + } + + return reqs + } +} + +func (r *AutonomousDatabaseReconciler) watchPredicate() predicate.Predicate { + return predicate.Funcs{ + CreateFunc: func(e event.CreateEvent) bool { + _, backupOk := e.Object.(*dbv1alpha1.AutonomousDatabaseBackup) + _, restoreOk := e.Object.(*dbv1alpha1.AutonomousDatabaseRestore) + // Don't enqueue if it's a create Backup event or a create Restore event + return !(backupOk || restoreOk) + }, + } +} + +func (r *AutonomousDatabaseReconciler) isIntermediateState(state database.AutonomousDatabaseLifecycleStateEnum) bool { + if state == database.AutonomousDatabaseLifecycleStateProvisioning || + state == database.AutonomousDatabaseLifecycleStateUpdating || + state == database.AutonomousDatabaseLifecycleStateStarting || + state == database.AutonomousDatabaseLifecycleStateStopping || + state == database.AutonomousDatabaseLifecycleStateTerminating || + state == database.AutonomousDatabaseLifecycleStateRestoreInProgress || + state == database.AutonomousDatabaseLifecycleStateBackupInProgress || + state == database.AutonomousDatabaseLifecycleStateMaintenanceInProgress || + state == database.AutonomousDatabaseLifecycleStateRestarting || + state == database.AutonomousDatabaseLifecycleStateRecreating || + state == database.AutonomousDatabaseLifecycleStateRoleChangeInProgress || + state == database.AutonomousDatabaseLifecycleStateUpgrading { + return true + } + return false +} + func (r *AutonomousDatabaseReconciler) eventFilterPredicate() predicate.Predicate { - pred := predicate.Funcs{ + return predicate.Funcs{ UpdateFunc: func(e event.UpdateEvent) bool { - oldStatus := e.ObjectOld.(*dbv1alpha1.AutonomousDatabase).Status.LifecycleState - desiredStatus := e.ObjectNew.(*dbv1alpha1.AutonomousDatabase).Spec.Details.LifecycleState - - if oldStatus == database.AutonomousDatabaseLifecycleStateProvisioning || - oldStatus == database.AutonomousDatabaseLifecycleStateUpdating || - oldStatus == database.AutonomousDatabaseLifecycleStateStarting || - oldStatus == database.AutonomousDatabaseLifecycleStateStopping || - oldStatus == database.AutonomousDatabaseLifecycleStateTerminating || - oldStatus == database.AutonomousDatabaseLifecycleStateRestoreInProgress || - oldStatus == database.AutonomousDatabaseLifecycleStateBackupInProgress || - oldStatus == database.AutonomousDatabaseLifecycleStateMaintenanceInProgress || - oldStatus == database.AutonomousDatabaseLifecycleStateRestarting || - oldStatus == database.AutonomousDatabaseLifecycleStateRecreating || - oldStatus == database.AutonomousDatabaseLifecycleStateRoleChangeInProgress || - oldStatus == database.AutonomousDatabaseLifecycleStateUpgrading { - - // Except for the case that the ADB is already terminating, we should let the terminate requests to be enqueued - if oldStatus != database.AutonomousDatabaseLifecycleStateTerminating && - desiredStatus == database.AutonomousDatabaseLifecycleStateTerminated { - return true + // source object can be AutonomousDatabase, AutonomousDatabaseBackup, or AutonomousDatabaseRestore + desiredADB, adbOk := e.ObjectNew.(*dbv1alpha1.AutonomousDatabase) + if adbOk { + oldADB := e.ObjectOld.(*dbv1alpha1.AutonomousDatabase) + + oldSucSpec, oldSucSpecOk := oldADB.GetAnnotations()[dbv1alpha1.LastSuccessfulSpec] + newSucSpec, newSucSpecOk := desiredADB.GetAnnotations()[dbv1alpha1.LastSuccessfulSpec] + sucSpecChanged := oldSucSpecOk != newSucSpecOk || oldSucSpec != newSucSpec + + if !reflect.DeepEqual(oldADB.Status, desiredADB.Status) || sucSpecChanged { + // Don't enqueue if the status or the lastSucSpec changes + return false } - // All the requests other than the terminate request, should be discarded during the intermediate states - return false - } + oldState := oldADB.Status.LifecycleState + desiredState := desiredADB.Spec.Details.LifecycleState + if r.isIntermediateState(oldState) { + // Except for the case that the ADB is already terminating, we should let the terminate requests to be enqueued + if oldState != database.AutonomousDatabaseLifecycleStateTerminating && + desiredState == database.AutonomousDatabaseLifecycleStateTerminated { + return true + } + + // All the requests other than the terminate request, should be discarded during the intermediate states + return false + } + return true + } return true }, DeleteFunc: func(e event.DeleteEvent) bool { - // Do not trigger reconciliation when the real object is deleted from the cluster. - return false + // Do not trigger reconciliation when the object is deleted from the cluster. + _, adbOk := e.Object.(*dbv1alpha1.AutonomousDatabase) + if adbOk { + return false + } + + return true }, } - - return pred } // +kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabases,verbs=get;list;watch;create;update;patch;delete // +kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabases/status,verbs=update;patch -// +kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabaseBackups,verbs=get;list;create;update;delete +// +kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabasebackups,verbs=get;list;watch;create;update;delete +// +kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabaserestores,verbs=get;list;watch;create;update;delete +// +kubebuilder:rbac:groups=database.oracle.com,resources=autonomouscontainerdatabases,verbs=get;list // +kubebuilder:rbac:groups=coordination.k8s.io,resources=leases,verbs=create;get;list;update // +kubebuilder:rbac:groups="",resources=configmaps;secrets,verbs=get;list;watch;create;update;patch;delete @@ -216,23 +280,17 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R return r.manageError(adb, err) } case adbActionBind: - fallthrough - case adbActionSync: - err = r.syncResource(adb) - if err != nil { - return r.manageError(adb, err) - } - if err := r.downloadWallet(adb); err != nil { return r.manageError(adb, err) } case adbActionUpdate: + // updateADB contains downloadWallet if err := r.updateADB(adb, difADB); err != nil { return r.manageError(adb, err) } - - err = r.syncResource(adb) - if err != nil { + case adbActionSync: + // SYNC action needs to make sure the wallet is present + if err := r.downloadWallet(adb); err != nil { return r.manageError(adb, err) } } @@ -245,9 +303,10 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R } /***************************************************** - * Update the lastSucSpec + * Sync resource and update the lastSucSpec *****************************************************/ - if err := r.updateLastSuccessfulSpec(adb); err != nil { + err = r.syncResource(adb) + if err != nil { return r.manageError(adb, err) } @@ -287,7 +346,7 @@ func (r *AutonomousDatabaseReconciler) manageError(adb *dbv1alpha1.AutonomousDat // Rollback if lastSucSpec exists if r.lastSucSpec != nil { // Send event - r.Recorder.Event(adb, corev1.EventTypeWarning, "ReconcileIncompleted", issue.Error()) + r.Recorder.Event(adb, corev1.EventTypeWarning, "ReconcileFailed", issue.Error()) var finalIssue = issue @@ -312,44 +371,54 @@ func (r *AutonomousDatabaseReconciler) manageError(adb *dbv1alpha1.AutonomousDat func (r *AutonomousDatabaseReconciler) validateFinalizer(adb *dbv1alpha1.AutonomousDatabase) (isADBDeleteTrue bool, err error) { logger := r.Log.WithName("finalizer") - if adb.ObjectMeta.DeletionTimestamp.IsZero() { - // The object is not being deleted - if *adb.Spec.HardLink && !k8s.HasFinalizer(adb, finalizer) { - if err := k8s.Register(r.KubeClient, adb, finalizer); err != nil { - return false, err + isADBToBeDeleted := adb.GetDeletionTimestamp() != nil + + if isADBToBeDeleted { + if controllerutil.ContainsFinalizer(adb, dbv1alpha1.ADBFinalizer) { + if (r.lastSucSpec != nil && r.lastSucSpec.Details.AutonomousDatabaseOCID != nil) && // lastSucSpec exists and the ADB_OCID isn't nil + (adb.Status.LifecycleState != database.AutonomousDatabaseLifecycleStateTerminating && adb.Status.LifecycleState != database.AutonomousDatabaseLifecycleStateTerminated) { + // Run finalization logic for finalizer. If the finalization logic fails, don't remove the finalizer so + // that we can retry during the next reconciliation. + logger.Info("Terminating Autonomous Database: " + *adb.Spec.Details.DbName) + if err := r.deleteAutonomousDatabase(adb); err != nil { + return false, err + } + } else { + logger.Info("Missing AutonomousDatabaseOCID to terminate Autonomous Database", "Name", adb.Name, "Namespace", adb.Namespace) } - logger.Info("Finalizer registered successfully.") - } else if !*adb.Spec.HardLink && !k8s.HasFinalizer(adb, finalizer) { - if err := k8s.Unregister(r.KubeClient, adb, finalizer); err != nil { - return false, err - } - logger.Info("Finalizer unregistered successfully.") - } - return false, nil - } else { - // The object is being deleted - if adb.Spec.Details.AutonomousDatabaseOCID == nil { - logger.Info("Autonomous Database OCID is missing. Remove the resource only.") - } else if adb.Status.LifecycleState != database.AutonomousDatabaseLifecycleStateTerminating && - adb.Status.LifecycleState != database.AutonomousDatabaseLifecycleStateTerminated { - // Don't send terminate request if the database is terminating or already terminated - logger.Info("Terminate Autonomous Database: " + *adb.Spec.Details.DbName) - - if err := r.deleteAutonomousDatabase(adb); err != nil { - return true, err + // Remove finalizer. Once all finalizers have been + // removed, the object will be deleted. + if err := k8s.RemoveFinalizerAndPatch(r.KubeClient, adb, dbv1alpha1.ADBFinalizer); err != nil { + return false, nil } } + // Send true because delete is in progress and it is a custom delete message + // We don't need to print custom err stack as we are deleting the topology + return true, nil + } - if err := k8s.Unregister(r.KubeClient, adb, finalizer); err != nil { - return true, err + // Delete is not schduled. Update the finalizer for this CR if hardLink is present + if adb.Spec.HardLink != nil { + if *adb.Spec.HardLink { + if err := k8s.AddFinalizerAndPatch(r.KubeClient, adb, dbv1alpha1.ADBFinalizer); err != nil { + return false, nil + } + } else { + if err := k8s.RemoveFinalizerAndPatch(r.KubeClient, adb, dbv1alpha1.ADBFinalizer); err != nil { + return false, nil + } } - logger.Info("Finalizer unregistered successfully.") - // Stop reconciliation as the item is being deleted - return true, nil } + + return false, nil } +// updateLastSuccessfulSpec updates the lasSucSpec annotation, and returns the ORIGINAL object +// The object will NOT be updated with the returned content from the cluster since we want to +// update only the lasSucSpec. For example: After we get the ADB information from OCI, we want +// to update the lasSucSpec before updating the local resource. In this case, we want to keep +// the content returned from the OCI, not the one from the cluster. func (r *AutonomousDatabaseReconciler) updateLastSuccessfulSpec(adb *dbv1alpha1.AutonomousDatabase) error { specBytes, err := json.Marshal(adb.Spec) if err != nil { @@ -360,18 +429,53 @@ func (r *AutonomousDatabaseReconciler) updateLastSuccessfulSpec(adb *dbv1alpha1. dbv1alpha1.LastSuccessfulSpec: string(specBytes), } - return annotations.SetAnnotations(r.KubeClient, adb, anns) + copyADB := adb.DeepCopy() + + return annotations.PatchAnnotations(r.KubeClient, copyADB, anns) } -// A helper function to wait until the work completes after an update/start/stop/delete AutonomousDatabase request is sent -// , and then sync the resource to make sure everything is updated when the work is finished -func (r *AutonomousDatabaseReconciler) wait(adb *dbv1alpha1.AutonomousDatabase, opcWorkRequestId string) error { - // Wait until the work is finished - if _, err := r.workService.Wait(opcWorkRequestId); err != nil { +// updateResourceStatus updates only the status of the resource, not including the lastSucSpec. +// This function should not be called by the functions associated with the OCI update requests. +// The OCI update requests should use updateResource() to ensure all the spec, resource and the +// lastSucSpec are updated. +func (r *AutonomousDatabaseReconciler) updateResourceStatus(adb *dbv1alpha1.AutonomousDatabase) error { + return retry.RetryOnConflict(retry.DefaultRetry, func() error { + curADB := &dbv1alpha1.AutonomousDatabase{} + + namespacedName := types.NamespacedName{ + Namespace: adb.GetNamespace(), + Name: adb.GetName(), + } + + if err := r.KubeClient.Get(context.TODO(), namespacedName, curADB); err != nil { + return err + } + + curADB.Status = adb.Status + + return r.KubeClient.Status().Update(context.TODO(), curADB) + }) +} + +// syncResource pulled information from OCI, update the LastSucSpec, and lastly, update the local resource. +// It's important to update the LastSucSpec prior than we update the local resource, +// because updating the local resource triggers the Reconcile, and the Reconcile determines the action by +// looking if the lastSucSpec is present. +func (r *AutonomousDatabaseReconciler) syncResource(adb *dbv1alpha1.AutonomousDatabase) error { + // Get the information from OCI + resp, err := r.adbService.GetAutonomousDatabase(*adb.Spec.Details.AutonomousDatabaseOCID) + if err != nil { return err } - if err := r.syncResource(adb); err != nil { + adb.UpdateFromOCIADB(resp.AutonomousDatabase) + + // Update the lastSucSpec + if err := r.updateLastSuccessfulSpec(adb); err != nil { + return err + } + + if err := r.updateResource(adb); err != nil { return err } @@ -407,30 +511,6 @@ func (r *AutonomousDatabaseReconciler) updateResource(adb *dbv1alpha1.Autonomous return nil } -// updateResourceStatus updates only the status of the resource, not including the lastSucSpec. -// This function should not be called by the functions associated with the OCI update requests. -// The OCI update requests should use updateResource() to ensure all the spec, resource and the -// lastSucSpec are updated. -func (r *AutonomousDatabaseReconciler) updateResourceStatus(adb *dbv1alpha1.AutonomousDatabase) error { - return k8s.UpdateADBStatus(r.KubeClient, adb) -} - -func (r *AutonomousDatabaseReconciler) syncResource(adb *dbv1alpha1.AutonomousDatabase) error { - // Get the information from OCI - resp, err := r.adbService.GetAutonomousDatabase(*adb.Spec.Details.AutonomousDatabaseOCID) - if err != nil { - return err - } - - adb.UpdateFromOCIADB(resp.AutonomousDatabase) - - if err := r.updateResource(adb); err != nil { - return err - } - - return nil -} - type ADBActionEnum string const ( @@ -440,15 +520,15 @@ const ( adbActionSync = "SYNC" ) -func (r *AutonomousDatabaseReconciler) determineAction(adb *dbv1alpha1.AutonomousDatabase) (ADBActionEnum, *dbv1alpha1.AutonomousDatabase, error) { +func (r *AutonomousDatabaseReconciler) determineAction(adb *dbv1alpha1.AutonomousDatabase) (ACDActionEnum, *dbv1alpha1.AutonomousDatabase, error) { if r.lastSucSpec == nil { if adb.Spec.Details.AutonomousDatabaseOCID == nil { - return adbActionProvision, nil, nil + return acdActionProvision, nil, nil } else { - return adbActionBind, nil, nil + return acdActionBind, nil, nil } } else { - // Pre-process step for the UPDATE. Remove the unchanged fields from spec.details, + // Pre-process step for the UPDATE. Remove the unchanged fields in spec.details, difADB := adb.DeepCopy() detailsChanged, err := difADB.RemoveUnchangedDetails() if err != nil { @@ -456,10 +536,10 @@ func (r *AutonomousDatabaseReconciler) determineAction(adb *dbv1alpha1.Autonomou } if detailsChanged { - return adbActionUpdate, difADB, nil + return acdActionUpdate, difADB, nil } - return adbActionSync, nil, nil + return acdActionSync, nil, nil } } @@ -470,13 +550,23 @@ func (r *AutonomousDatabaseReconciler) createADB(adb *dbv1alpha1.AutonomousDatab } // Update the ADB OCID and the status + // The trick is to update the status first to prevent unwanted reconcile adb.Spec.Details.AutonomousDatabaseOCID = resp.AutonomousDatabase.Id adb.UpdateStatusFromOCIADB(resp.AutonomousDatabase) - if err := r.updateResource(adb); err != nil { + + if err := r.updateResourceStatus(adb); err != nil { return err } - return r.wait(adb, *resp.OpcWorkRequestId) + // Patching is faster + if err := k8s.Patch(r.KubeClient, adb, "/spec/details/autonomousDatabaseOCID", adb.Spec.Details.AutonomousDatabaseOCID); err != nil { + return err + } + + if _, err := r.workService.Wait(*resp.OpcWorkRequestId); err != nil { + return err + } + return nil } func (r *AutonomousDatabaseReconciler) updateGeneralFields(adb *dbv1alpha1.AutonomousDatabase, difADB *dbv1alpha1.AutonomousDatabase) error { @@ -503,12 +593,15 @@ func (r *AutonomousDatabaseReconciler) updateGeneralFields(adb *dbv1alpha1.Auton return err } - return r.wait(adb, *resp.OpcWorkRequestId) + if _, err := r.workService.Wait(*resp.OpcWorkRequestId); err != nil { + return err + } + return nil } func (r *AutonomousDatabaseReconciler) updateAdminPassword(adb *dbv1alpha1.AutonomousDatabase, difADB *dbv1alpha1.AutonomousDatabase) error { - if difADB.Spec.Details.AdminPassword.K8sSecretName == nil && - difADB.Spec.Details.AdminPassword.OCISecretOCID == nil { + if difADB.Spec.Details.AdminPassword.K8sSecret.Name == nil && + difADB.Spec.Details.AdminPassword.OCISecret.OCID == nil { return nil } @@ -536,7 +629,10 @@ func (r *AutonomousDatabaseReconciler) updateDbWorkload(adb *dbv1alpha1.Autonomo return err } - return r.wait(adb, *resp.OpcWorkRequestId) + if _, err := r.workService.Wait(*resp.OpcWorkRequestId); err != nil { + return err + } + return nil } func (r *AutonomousDatabaseReconciler) updateLicenseModel(adb *dbv1alpha1.AutonomousDatabase, difADB *dbv1alpha1.AutonomousDatabase) error { @@ -554,7 +650,10 @@ func (r *AutonomousDatabaseReconciler) updateLicenseModel(adb *dbv1alpha1.Autono return err } - return r.wait(adb, *resp.OpcWorkRequestId) + if _, err := r.workService.Wait(*resp.OpcWorkRequestId); err != nil { + return err + } + return nil } func (r *AutonomousDatabaseReconciler) updateScalingFields(adb *dbv1alpha1.AutonomousDatabase, difADB *dbv1alpha1.AutonomousDatabase) error { @@ -574,7 +673,10 @@ func (r *AutonomousDatabaseReconciler) updateScalingFields(adb *dbv1alpha1.Auton return err } - return r.wait(adb, *resp.OpcWorkRequestId) + if _, err := r.workService.Wait(*resp.OpcWorkRequestId); err != nil { + return err + } + return nil } func (r *AutonomousDatabaseReconciler) deleteAutonomousDatabase(adb *dbv1alpha1.AutonomousDatabase) error { @@ -590,7 +692,9 @@ func (r *AutonomousDatabaseReconciler) deleteAutonomousDatabase(adb *dbv1alpha1. return err } - return r.wait(adb, *resp.OpcWorkRequestId) + _, err = r.workService.Wait(*resp.OpcWorkRequestId) + + return err } func (r *AutonomousDatabaseReconciler) updateLifecycleState(adb *dbv1alpha1.AutonomousDatabase, difADB *dbv1alpha1.AutonomousDatabase) error { @@ -633,7 +737,10 @@ func (r *AutonomousDatabaseReconciler) updateLifecycleState(adb *dbv1alpha1.Auto return err } - return r.wait(adb, opcWorkRequestId) + if _, err := r.workService.Wait(opcWorkRequestId); err != nil { + return err + } + return nil } func (r *AutonomousDatabaseReconciler) setMTLSRequired(adb *dbv1alpha1.AutonomousDatabase) error { @@ -647,7 +754,10 @@ func (r *AutonomousDatabaseReconciler) setMTLSRequired(adb *dbv1alpha1.Autonomou return err } - return r.wait(adb, *resp.OpcWorkRequestId) + if _, err := r.workService.Wait(*resp.OpcWorkRequestId); err != nil { + return err + } + return nil } func (r *AutonomousDatabaseReconciler) updateMTLS(adb *dbv1alpha1.AutonomousDatabase, difADB *dbv1alpha1.AutonomousDatabase) error { @@ -665,7 +775,10 @@ func (r *AutonomousDatabaseReconciler) updateMTLS(adb *dbv1alpha1.AutonomousData return err } - return r.wait(adb, *resp.OpcWorkRequestId) + if _, err := r.workService.Wait(*resp.OpcWorkRequestId); err != nil { + return err + } + return nil } func (r *AutonomousDatabaseReconciler) setNetworkAccessPublic(adb *dbv1alpha1.AutonomousDatabase) error { @@ -679,7 +792,10 @@ func (r *AutonomousDatabaseReconciler) setNetworkAccessPublic(adb *dbv1alpha1.Au return err } - return r.wait(adb, *resp.OpcWorkRequestId) + if _, err := r.workService.Wait(*resp.OpcWorkRequestId); err != nil { + return err + } + return nil } func (r *AutonomousDatabaseReconciler) updateNetworkAccess(adb *dbv1alpha1.AutonomousDatabase, difADB *dbv1alpha1.AutonomousDatabase) error { @@ -701,7 +817,10 @@ func (r *AutonomousDatabaseReconciler) updateNetworkAccess(adb *dbv1alpha1.Auton return err } - return r.wait(adb, *resp.OpcWorkRequestId) + if _, err := r.workService.Wait(*resp.OpcWorkRequestId); err != nil { + return err + } + return nil } // The logic of updating the network access configurations is as follows: @@ -806,8 +925,8 @@ func (r *AutonomousDatabaseReconciler) determineNetworkAccessUpdate(adb *dbv1alp func (r *AutonomousDatabaseReconciler) downloadWallet(adb *dbv1alpha1.AutonomousDatabase) error { if adb.Spec.Details.Wallet.Name == nil && - adb.Spec.Details.Wallet.Password.K8sSecretName == nil && - adb.Spec.Details.Wallet.Password.OCISecretOCID == nil { + adb.Spec.Details.Wallet.Password.K8sSecret.Name == nil && + adb.Spec.Details.Wallet.Password.OCISecret.OCID == nil { return nil } @@ -923,8 +1042,8 @@ func (r *AutonomousDatabaseReconciler) syncBackupResources(adb *dbv1alpha1.Auton for _, backup := range backupList.Items { curBackupNames[backup.Name] = true - if backup.Spec.AutonomousDatabaseBackupOCID != "" { - curBackupOCIDs[backup.Spec.AutonomousDatabaseBackupOCID] = true + if backup.Spec.AutonomousDatabaseBackupOCID != nil { + curBackupOCIDs[*backup.Spec.AutonomousDatabaseBackupOCID] = true } } diff --git a/controllers/database/autonomousdatabasebackup_controller.go b/controllers/database/autonomousdatabasebackup_controller.go index 44bb8831..5cee5e5a 100644 --- a/controllers/database/autonomousdatabasebackup_controller.go +++ b/controllers/database/autonomousdatabasebackup_controller.go @@ -40,13 +40,14 @@ package controllers import ( "context" + "errors" "fmt" "github.com/go-logr/logr" + corev1 "k8s.io/api/core/v1" apiErrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/runtime" - "k8s.io/apimachinery/pkg/types" - "k8s.io/client-go/util/retry" + "k8s.io/client-go/tools/record" ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/controller-runtime/pkg/controller" @@ -56,6 +57,7 @@ import ( "github.com/oracle/oci-go-sdk/v54/database" databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + "github.com/oracle/oracle-database-operator/commons/adb_family" "github.com/oracle/oracle-database-operator/commons/k8s" "github.com/oracle/oracle-database-operator/commons/oci" ) @@ -65,23 +67,26 @@ type AutonomousDatabaseBackupReconciler struct { KubeClient client.Client Log logr.Logger Scheme *runtime.Scheme + Recorder record.EventRecorder adbService oci.DatabaseService workService oci.WorkRequestService - ownerADB *dbv1alpha1.AutonomousDatabase } // SetupWithManager sets up the controller with the Manager. func (r *AutonomousDatabaseBackupReconciler) SetupWithManager(mgr ctrl.Manager) error { return ctrl.NewControllerManagedBy(mgr). For(&databasev1alpha1.AutonomousDatabaseBackup{}). - WithEventFilter(predicate.And(predicate.GenerationChangedPredicate{}, r.eventFilterPredicate())). + WithEventFilter(predicate.And(r.eventFilterPredicate(), predicate.GenerationChangedPredicate{})). WithOptions(controller.Options{MaxConcurrentReconciles: 100}). // ReconcileHandler is never invoked concurrently with the same object. Complete(r) } func (r *AutonomousDatabaseBackupReconciler) eventFilterPredicate() predicate.Predicate { pred := predicate.Funcs{ + CreateFunc: func(e event.CreateEvent) bool { + return true + }, UpdateFunc: func(e event.UpdateEvent) bool { oldStatus := e.ObjectOld.(*dbv1alpha1.AutonomousDatabaseBackup).Status.LifecycleState @@ -120,7 +125,8 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req * Look up the owner AutonomousDatabase and set the ownerReference * if the owner hasn't been set yet. ******************************************************************/ - if err := r.verifyOwnerADB(backup); err != nil { + adbOCID, err := r.verifyTargetADB(backup) + if err != nil { return r.manageError(backup, err) } @@ -138,9 +144,9 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req * LifecycleState is checked to avoid sending a duplicated backup request when the backup is creating. * Otherwise, bind to an exisiting backup if the Spec.AutonomousDatabaseBackupOCID isn't empty. ******************************************************************/ - if backup.Spec.AutonomousDatabaseBackupOCID == "" && backup.Status.LifecycleState == "" { + if backup.Spec.AutonomousDatabaseBackupOCID == nil && backup.Status.LifecycleState == "" { // Create a new backup - backupResp, err := r.adbService.CreateAutonomousDatabaseBackup(backup, *r.ownerADB.Spec.Details.AutonomousDatabaseOCID) + backupResp, err := r.adbService.CreateAutonomousDatabaseBackup(backup, adbOCID) if err != nil { return r.manageError(backup, err) } @@ -151,8 +157,8 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req } // update the Backup status - backup.UpdateFromOCIBackup(backupResp.AutonomousDatabaseBackup, adbResp.AutonomousDatabase) - if err := r.updateResourceStatus(backup); err != nil { + backup.UpdateStatusFromOCIBackup(backupResp.AutonomousDatabaseBackup, adbResp.AutonomousDatabase) + if err := r.KubeClient.Status().Update(context.TODO(), backup); err != nil { return r.manageError(backup, err) } @@ -162,57 +168,61 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req } logger.Info("AutonomousDatabaseBackup " + *backupResp.DisplayName + " created successfully") - - return ctrl.Result{Requeue: true}, nil } /****************************************************************** - * Update the status of the resource if the - * Spec.AutonomousDatabaseOCID isn't empty. - ******************************************************************/ - if backup.Spec.AutonomousDatabaseBackupOCID != "" { - backupResp, err := r.adbService.GetAutonomousDatabaseBackup(backup.Spec.AutonomousDatabaseBackupOCID) - if err != nil { - return r.manageError(backup, err) - } + * Sync the resource status + *******************************************************************/ + // get the backup ID + var backupID string + if backup.Spec.AutonomousDatabaseBackupOCID != nil { + backupID = *backup.Spec.AutonomousDatabaseBackupOCID + } else if backup.Status.AutonomousDatabaseBackupOCID != "" { + backupID = backup.Status.AutonomousDatabaseBackupOCID + } else { + // Send the event and exit the Reconcile + err := errors.New("the backup is incomplete and missing the OCID; the resource should be removed") + logger.Error(err, "Reconcile stopped") + r.Recorder.Event(backup, corev1.EventTypeWarning, "ReconcileFailed", err.Error()) + return emptyResult, nil + } - adbResp, err := r.adbService.GetAutonomousDatabase(*backupResp.AutonomousDatabaseId) - if err != nil { - return r.manageError(backup, err) - } + backupResp, err := r.adbService.GetAutonomousDatabaseBackup(backupID) + if err != nil { + return r.manageError(backup, err) + } - backup.UpdateFromOCIBackup(backupResp.AutonomousDatabaseBackup, adbResp.AutonomousDatabase) - if err := r.updateResourceStatus(backup); err != nil { - return r.manageError(backup, err) - } + adbResp, err := r.adbService.GetAutonomousDatabase(*backupResp.AutonomousDatabaseId) + if err != nil { + return r.manageError(backup, err) + } - return emptyResult, nil + backup.UpdateStatusFromOCIBackup(backupResp.AutonomousDatabaseBackup, adbResp.AutonomousDatabase) + if err := r.KubeClient.Status().Update(context.TODO(), backup); err != nil { + return r.manageError(backup, err) } return emptyResult, nil } -func (r *AutonomousDatabaseBackupReconciler) verifyOwnerADB(backup *dbv1alpha1.AutonomousDatabaseBackup) error { - if len(backup.GetOwnerReferences()) == 0 { - var err error +// verifyTargetADB searches if the target ADB is in the cluster, and set the owner reference to the ADB if it exists. +// The function returns the OCID of the target ADB. +func (r *AutonomousDatabaseBackupReconciler) verifyTargetADB(backup *dbv1alpha1.AutonomousDatabaseBackup) (string, error) { + // Get the target ADB OCID and the ADB resource + ocid, ownerADB, err := adbfamily.VerifyTargetADB(r.KubeClient, backup.Spec.Target, backup.Namespace) - if backup.Spec.TargetADB.Name != "" { - r.ownerADB, err = k8s.FetchAutonomousDatabase(r.KubeClient, backup.Namespace, backup.Spec.TargetADB.Name) - if err != nil { - return err - } - } else { - r.ownerADB, err = k8s.FetchAutonomousDatabaseWithOCID(r.KubeClient, backup.Namespace, backup.Spec.TargetADB.OCID) - if err != nil { - return err - } - } + if err != nil { + return "", err + } - if err := r.setOwnerAutonomousDatabase(backup, r.ownerADB); err != nil { - return err + // Set the owner reference if needed + if len(backup.GetOwnerReferences()) == 0 && ownerADB != nil { + if err := r.setOwnerAutonomousDatabase(backup, ownerADB); err != nil { + return "", err } } - return nil + + return ocid, nil } func (r *AutonomousDatabaseBackupReconciler) setupOCIClients(backup *dbv1alpha1.AutonomousDatabaseBackup) error { @@ -244,67 +254,12 @@ func (r *AutonomousDatabaseBackupReconciler) setupOCIClients(backup *dbv1alpha1. // updateResource updates the specification and the status of AutonomousDatabase resource without trigger a reconcile loop func (r *AutonomousDatabaseBackupReconciler) updateResource(backup *dbv1alpha1.AutonomousDatabaseBackup) error { - // Update the spec - if err := retry.RetryOnConflict(retry.DefaultRetry, func() error { - curBackup := &dbv1alpha1.AutonomousDatabaseBackup{} - - namespacedName := types.NamespacedName{ - Namespace: backup.GetNamespace(), - Name: backup.GetName(), - } - - if err := r.KubeClient.Get(context.TODO(), namespacedName, curBackup); err != nil { - return err - } - - curBackup.Spec = *backup.Spec.DeepCopy() - curBackup.ObjectMeta = *backup.ObjectMeta.DeepCopy() // ownerReference - return r.KubeClient.Update(context.TODO(), curBackup) - }); err != nil { + if err := r.KubeClient.Update(context.TODO(), backup); err != nil { return err } // Update the status - if err := r.updateResourceStatus(backup); err != nil { - return err - } - - return nil -} - -func (r *AutonomousDatabaseBackupReconciler) updateResourceStatus(backup *dbv1alpha1.AutonomousDatabaseBackup) error { - // sync the ADB status every time when the Backup status is updated - if err := r.syncADBStatus(); err != nil { - return err - } - - return retry.RetryOnConflict(retry.DefaultRetry, func() error { - curBackup := &dbv1alpha1.AutonomousDatabaseBackup{} - - namespacedName := types.NamespacedName{ - Namespace: backup.GetNamespace(), - Name: backup.GetName(), - } - - if err := r.KubeClient.Get(context.TODO(), namespacedName, curBackup); err != nil { - return err - } - - curBackup.Status = backup.Status - return r.KubeClient.Status().Update(context.TODO(), curBackup) - }) -} - -// No-op if the ownerADB is not assigned -func (r *AutonomousDatabaseBackupReconciler) syncADBStatus() error { - resp, err := r.adbService.GetAutonomousDatabase(*r.ownerADB.Spec.Details.AutonomousDatabaseOCID) - if err != nil { - return err - } - - r.ownerADB.Status.LifecycleState = resp.LifecycleState - - if err := k8s.UpdateADBStatus(r.KubeClient, r.ownerADB); err != nil { + if err := r.KubeClient.Status().Update(context.TODO(), backup); err != nil { return err } @@ -313,7 +268,7 @@ func (r *AutonomousDatabaseBackupReconciler) syncADBStatus() error { // setOwnerAutonomousDatabase sets the owner of the AutonomousDatabaseBackup if the AutonomousDatabase resource with the same database OCID is found func (r *AutonomousDatabaseBackupReconciler) setOwnerAutonomousDatabase(backup *dbv1alpha1.AutonomousDatabaseBackup, adb *dbv1alpha1.AutonomousDatabase) error { - logger := r.Log.WithName("set-owner") + logger := r.Log.WithName("set-owner-reference") backup.SetOwnerReferences(k8s.NewOwnerReference(adb)) r.updateResource(backup) @@ -323,9 +278,12 @@ func (r *AutonomousDatabaseBackupReconciler) setOwnerAutonomousDatabase(backup * } func (r *AutonomousDatabaseBackupReconciler) manageError(backup *dbv1alpha1.AutonomousDatabaseBackup, issue error) (ctrl.Result, error) { + // Send event + r.Recorder.Event(backup, corev1.EventTypeWarning, "ReconcileFailed", issue.Error()) + // Change the status to FAILED backup.Status.LifecycleState = database.AutonomousDatabaseBackupLifecycleStateFailed - if statusErr := r.updateResourceStatus(backup); statusErr != nil { + if statusErr := r.KubeClient.Status().Update(context.TODO(), backup); statusErr != nil { return emptyResult, k8s.CombineErrors(issue, statusErr) } diff --git a/controllers/database/autonomousdatabaserestore_controller.go b/controllers/database/autonomousdatabaserestore_controller.go index 25d0bd8d..1e3e16ac 100644 --- a/controllers/database/autonomousdatabaserestore_controller.go +++ b/controllers/database/autonomousdatabaserestore_controller.go @@ -44,18 +44,20 @@ import ( "fmt" "github.com/go-logr/logr" + corev1 "k8s.io/api/core/v1" apiErrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/runtime" "k8s.io/apimachinery/pkg/types" + "k8s.io/client-go/tools/record" "k8s.io/client-go/util/retry" ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/controller-runtime/pkg/predicate" "github.com/oracle/oci-go-sdk/v54/common" - "github.com/oracle/oci-go-sdk/v54/workrequests" databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + "github.com/oracle/oracle-database-operator/commons/adb_family" "github.com/oracle/oracle-database-operator/commons/k8s" "github.com/oracle/oracle-database-operator/commons/oci" ) @@ -65,10 +67,10 @@ type AutonomousDatabaseRestoreReconciler struct { KubeClient client.Client Log logr.Logger Scheme *runtime.Scheme + Recorder record.EventRecorder adbService oci.DatabaseService workService oci.WorkRequestService - ownerADB *dbv1alpha1.AutonomousDatabase } // SetupWithManager sets up the controller with the Manager. @@ -116,7 +118,8 @@ func (r *AutonomousDatabaseRestoreReconciler) Reconcile(ctx context.Context, req * Look up the owner AutonomousDatabase and set the ownerReference * if the owner hasn't been set yet. ******************************************************************/ - if err := r.verifyOwnerADB(restore); err != nil { + adbOCID, err := r.verifyTargetADB(restore) + if err != nil { return r.manageError(restore, err) } @@ -128,14 +131,6 @@ func (r *AutonomousDatabaseRestoreReconciler) Reconcile(ctx context.Context, req return r.manageError(restore, err) } - /****************************************************************** - * Get the target ADB OCID - ******************************************************************/ - adbOCID, err := r.getADBOCID(restore) - if err != nil { - return r.manageError(restore, err) - } - /****************************************************************** * Get OCI database client and work request client ******************************************************************/ @@ -154,13 +149,13 @@ func (r *AutonomousDatabaseRestoreReconciler) Reconcile(ctx context.Context, req logger.Info("AutonomousDatabaseRestore reconciles successfully") - return ctrl.Result{}, nil + return emptyResult, nil } func (r *AutonomousDatabaseRestoreReconciler) getRestoreSDKTime(restore *dbv1alpha1.AutonomousDatabaseRestore) (*common.SDKTime, error) { - if restore.Spec.Source.AutonomousDatabaseBackup.Name != "" { // restore using backupName + if restore.Spec.Source.K8sADBBackup.Name != nil { // restore using backupName backup := &dbv1alpha1.AutonomousDatabaseBackup{} - if err := k8s.FetchResource(r.KubeClient, restore.Namespace, restore.Spec.Source.AutonomousDatabaseBackup.Name, backup); err != nil { + if err := k8s.FetchResource(r.KubeClient, restore.Namespace, *restore.Spec.Source.K8sADBBackup.Name, backup); err != nil { return nil, err } @@ -196,50 +191,27 @@ func (r *AutonomousDatabaseRestoreReconciler) setOwnerAutonomousDatabase(restore return nil } -func (r *AutonomousDatabaseRestoreReconciler) verifyOwnerADB(restore *dbv1alpha1.AutonomousDatabaseRestore) error { - if len(restore.GetOwnerReferences()) == 0 { - var err error - - if restore.Spec.TargetADB.Name != "" { - r.ownerADB, err = k8s.FetchAutonomousDatabase(r.KubeClient, restore.Namespace, restore.Spec.TargetADB.Name) - if err != nil { - return err - } - } else { - r.ownerADB, err = k8s.FetchAutonomousDatabaseWithOCID(r.KubeClient, restore.Namespace, restore.Spec.TargetADB.OCID) - if err != nil { - return err - } - } +// verifyTargetADB searches if the target ADB is in the cluster, and set the owner reference to the ADB if it exists. +// The function returns the OCID of the target ADB. +func (r *AutonomousDatabaseRestoreReconciler) verifyTargetADB(restore *dbv1alpha1.AutonomousDatabaseRestore) (string, error) { + // Get the target ADB OCID and the ADB resource + ocid, ownerADB, err := adbfamily.VerifyTargetADB(r.KubeClient, restore.Spec.Target, restore.Namespace) - if r.ownerADB == nil { - if err := r.setOwnerAutonomousDatabase(restore, r.ownerADB); err != nil { - return err - } - } + if err != nil { + return "", err } - return nil -} - -func (r *AutonomousDatabaseRestoreReconciler) getADBOCID(restore *dbv1alpha1.AutonomousDatabaseRestore) (string, error) { - if r.ownerADB != nil { - return *r.ownerADB.Spec.Details.AutonomousDatabaseOCID, nil - } else { - backup := &dbv1alpha1.AutonomousDatabaseBackup{} - if err := k8s.FetchResource(r.KubeClient, restore.Namespace, restore.Spec.Source.AutonomousDatabaseBackup.Name, backup); err != nil { + // Set the owner reference if needed + if len(restore.GetOwnerReferences()) == 0 && ownerADB != nil { + if err := r.setOwnerAutonomousDatabase(restore, ownerADB); err != nil { return "", err } - return backup.Status.AutonomousDatabaseOCID, nil } + + return ocid, nil } func (r *AutonomousDatabaseRestoreReconciler) updateResourceStatus(restore *dbv1alpha1.AutonomousDatabaseRestore) error { - // sync the ADB status every time when the Backup status is updated - if err := r.syncADBStatus(); err != nil { - return err - } - return retry.RetryOnConflict(retry.DefaultRetry, func() error { curBackup := &dbv1alpha1.AutonomousDatabaseRestore{} @@ -257,26 +229,6 @@ func (r *AutonomousDatabaseRestoreReconciler) updateResourceStatus(restore *dbv1 }) } -// No-op if the ownerADB is not assigned -func (r *AutonomousDatabaseRestoreReconciler) syncADBStatus() error { - if r.ownerADB == nil { - return nil - } - - resp, err := r.adbService.GetAutonomousDatabase(*r.ownerADB.Spec.Details.AutonomousDatabaseOCID) - if err != nil { - return err - } - - r.ownerADB.Status.LifecycleState = resp.LifecycleState - - if err := k8s.UpdateADBStatus(r.KubeClient, r.ownerADB); err != nil { - return err - } - - return nil -} - func (r *AutonomousDatabaseRestoreReconciler) setupOCIClients(restore *dbv1alpha1.AutonomousDatabaseRestore) error { var err error @@ -312,6 +264,9 @@ func (r *AutonomousDatabaseRestoreReconciler) manageError(restore *dbv1alpha1.Au } logger := r.Log.WithValues("Namespaced/Name", nsn) + // Send event + r.Recorder.Event(restore, corev1.EventTypeWarning, "ReconcileFailed", issue.Error()) + // Change the status to FAILED var combinedErr error = issue @@ -329,6 +284,7 @@ func (r *AutonomousDatabaseRestoreReconciler) restoreAutonomousDatabase( restore *dbv1alpha1.AutonomousDatabaseRestore, restoreTime *common.SDKTime, adbOCID string) error { + var err error resp, err := r.adbService.RestoreAutonomousDatabase(adbOCID, *restoreTime) if err != nil { @@ -340,17 +296,29 @@ func (r *AutonomousDatabaseRestoreReconciler) restoreAutonomousDatabase( restore.Status.DisplayName = *resp.AutonomousDatabase.DisplayName restore.Status.DbName = *resp.AutonomousDatabase.DbName restore.Status.AutonomousDatabaseOCID = *resp.AutonomousDatabase.Id - restore.Status.Status = restore.ConvertWorkRequestStatus(workrequests.WorkRequestStatusEnum(resp.LifecycleState)) + + workStatusStart, err := r.workService.Get(*resp.OpcWorkRequestId) + if err != nil { + return err + } + restore.Status.Status, err = restore.ConvertWorkRequestStatus(workStatusStart) + if err != nil { + return err + } r.updateResourceStatus(restore) - workStatus, err := r.workService.Wait(*resp.OpcWorkRequestId) + workStatusEnd, err := r.workService.Wait(*resp.OpcWorkRequestId) if err != nil { return err } // Update status when the work is finished - restore.Status.Status = restore.ConvertWorkRequestStatus(workStatus) + restore.Status.Status, err = restore.ConvertWorkRequestStatus(workStatusEnd) + if err != nil { + return err + } + r.updateResourceStatus(restore) return nil diff --git a/main.go b/main.go index 14dfa0f9..ffe2d91b 100644 --- a/main.go +++ b/main.go @@ -96,6 +96,7 @@ func main() { // Get Cache cache := mgr.GetCache() + // ADB family controllers if err = (&databasecontroller.AutonomousDatabaseReconciler{ KubeClient: mgr.GetClient(), Log: ctrl.Log.WithName("controllers").WithName("database").WithName("AutonomousDatabase"), @@ -109,6 +110,7 @@ func main() { KubeClient: mgr.GetClient(), Log: ctrl.Log.WithName("controllers").WithName("AutonomousDatabaseBackup"), Scheme: mgr.GetScheme(), + Recorder: mgr.GetEventRecorderFor("AutonomousDatabaseBackup"), }).SetupWithManager(mgr); err != nil { setupLog.Error(err, "unable to create controller", "controller", "AutonomousDatabaseBackup") os.Exit(1) @@ -117,10 +119,21 @@ func main() { KubeClient: mgr.GetClient(), Log: ctrl.Log.WithName("controllers").WithName("AutonomousDatabaseRestore"), Scheme: mgr.GetScheme(), + Recorder: mgr.GetEventRecorderFor("AutonomousDatabaseRestore"), }).SetupWithManager(mgr); err != nil { setupLog.Error(err, "unable to create controller", "controller", "AutonomousDatabaseRestore") os.Exit(1) } + if err = (&databasecontroller.AutonomousContainerDatabaseReconciler{ + KubeClient: mgr.GetClient(), + Log: ctrl.Log.WithName("controllers").WithName("AutonomousContainerDatabase"), + Scheme: mgr.GetScheme(), + Recorder: mgr.GetEventRecorderFor("AutonomousContainerDatabase"), + }).SetupWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create controller", "controller", "AutonomousContainerDatabase") + os.Exit(1) + } + if err = (&databasecontroller.SingleInstanceDatabaseReconciler{ Client: mgr.GetClient(), Log: ctrl.Log.WithName("controllers").WithName("database").WithName("SingleInstanceDatabase"), @@ -233,4 +246,4 @@ func main() { setupLog.Error(err, "problem running manager") os.Exit(1) } -} \ No newline at end of file +} diff --git a/oracle-database-operator.yaml b/oracle-database-operator.yaml index 99918bfe..fd538e47 100644 --- a/oracle-database-operator.yaml +++ b/oracle-database-operator.yaml @@ -7,6 +7,105 @@ metadata: --- apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.6.1 + creationTimestamp: null + name: autonomouscontainerdatabases.database.oracle.com +spec: + group: database.oracle.com + names: + kind: AutonomousContainerDatabase + listKind: AutonomousContainerDatabaseList + plural: autonomouscontainerdatabases + shortNames: + - acd + - acds + singular: autonomouscontainerdatabase + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .spec.displayName + name: DisplayName + type: string + - jsonPath: .status.lifecycleState + name: State + type: string + - jsonPath: .status.timeCreated + name: Created + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + description: AutonomousContainerDatabase is the Schema for the autonomouscontainerdatabases API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: AutonomousContainerDatabaseSpec defines the desired state of AutonomousContainerDatabase + properties: + autonomousContainerDatabaseOCID: + description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster Important: Run "make" to regenerate code after modifying this file' + type: string + autonomousExadataVMClusterOCID: + type: string + compartmentOCID: + type: string + displayName: + type: string + freeformTags: + additionalProperties: + type: string + type: object + hardLink: + default: false + type: boolean + ociConfig: + description: "*********************** *\tOCI config ***********************" + properties: + configMapName: + type: string + secretName: + type: string + type: object + patchModel: + description: 'AutonomousContainerDatabasePatchModelEnum Enum with underlying type: string' + enum: + - RELEASE_UPDATES + - RELEASE_UPDATE_REVISIONS + type: string + type: object + status: + description: AutonomousContainerDatabaseStatus defines the observed state of AutonomousContainerDatabase + properties: + lifecycleState: + description: 'INSERT ADDITIONAL STATUS FIELD - define observed state of cluster Important: Run "make" to regenerate code after modifying this file' + type: string + timeCreated: + type: string + required: + - lifecycleState + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition metadata: annotations: controller-gen.kubebuilder.io/version: v0.6.1 @@ -58,23 +157,40 @@ spec: properties: autonomousDatabaseBackupOCID: type: string - autonomousDatabaseOCID: - description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster Important: Run "make" to regenerate code after modifying this file' - type: string displayName: type: string ociConfig: + description: "*********************** *\tOCI config ***********************" properties: configMapName: type: string secretName: type: string type: object + target: + description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster Important: Run "make" to regenerate code after modifying this file' + properties: + k8sADB: + description: "*********************** *\tADB spec ***********************" + properties: + name: + type: string + type: object + ociADB: + properties: + ocid: + type: string + type: object + type: object + required: + - displayName + - target type: object status: description: AutonomousDatabaseBackupStatus defines the observed state of AutonomousDatabaseBackup properties: autonomousDatabaseBackupOCID: + description: 'INSERT ADDITIONAL STATUS FIELD - define observed state of cluster Important: Run "make" to regenerate code after modifying this file' type: string autonomousDatabaseOCID: type: string @@ -89,7 +205,7 @@ spec: isAutomatic: type: boolean lifecycleState: - description: 'INSERT ADDITIONAL STATUS FIELD - define observed state of cluster Important: Run "make" to regenerate code after modifying this file' + description: 'AutonomousDatabaseBackupLifecycleStateEnum Enum with underlying type: string' type: string timeEnded: type: string @@ -99,7 +215,6 @@ spec: description: 'AutonomousDatabaseBackupTypeEnum Enum with underlying type: string' type: string required: - - autonomousDatabaseBackupOCID - autonomousDatabaseOCID - compartmentOCID - dbDisplayName @@ -141,8 +256,8 @@ spec: scope: Namespaced versions: - additionalPrinterColumns: - - jsonPath: .status.lifecycleState - name: State + - jsonPath: .status.status + name: Status type: string - jsonPath: .status.displayName name: DisplayName @@ -166,23 +281,47 @@ spec: spec: description: AutonomousDatabaseRestoreSpec defines the desired state of AutonomousDatabaseRestore properties: - backupName: - description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster Important: Run "make" to regenerate code after modifying this file' - type: string ociConfig: + description: "*********************** *\tOCI config ***********************" properties: configMapName: type: string secretName: type: string type: object - pointInTime: + source: properties: - autonomousDatabaseOCID: - type: string - timeStamp: - type: string + k8sADBBackup: + description: 'EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized.' + properties: + name: + type: string + type: object + pointInTime: + properties: + timestamp: + description: 'The timestamp must follow this format: YYYY-MM-DD HH:MM:SS GMT' + type: string + type: object type: object + target: + description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster Important: Run "make" to regenerate code after modifying this file' + properties: + k8sADB: + description: "*********************** *\tADB spec ***********************" + properties: + name: + type: string + type: object + ociADB: + properties: + ocid: + type: string + type: object + type: object + required: + - source + - target type: object status: description: AutonomousDatabaseRestoreStatus defines the observed state of AutonomousDatabaseRestore @@ -194,14 +333,13 @@ spec: displayName: description: 'INSERT ADDITIONAL STATUS FIELD - define observed state of cluster Important: Run "make" to regenerate code after modifying this file' type: string - lifecycleState: - description: 'WorkRequestStatusEnum Enum with underlying type: string' + status: type: string required: - autonomousDatabaseOCID - dbName - displayName - - lifecycleState + - status type: object type: object served: true @@ -235,22 +373,22 @@ spec: scope: Namespaced versions: - additionalPrinterColumns: - - jsonPath: .status.displayName + - jsonPath: .spec.details.displayName name: Display Name type: string - jsonPath: .status.lifecycleState name: State type: string - - jsonPath: .status.isDedicated + - jsonPath: .spec.details.isDedicated name: Dedicated type: string - - jsonPath: .status.cpuCoreCount + - jsonPath: .spec.details.cpuCoreCount name: OCPUs type: integer - - jsonPath: .status.dataStorageSizeInTBs + - jsonPath: .spec.details.dataStorageSizeInTBs name: Storage (TB) type: integer - - jsonPath: .status.dbWorkload + - jsonPath: .spec.details.dbWorkload name: Workload Type type: string - jsonPath: .status.timeCreated @@ -259,38 +397,51 @@ spec: name: v1alpha1 schema: openAPIV3Schema: - description: AutonomousDatabase is the Schema for the autonomousdatabases - API + description: AutonomousDatabase is the Schema for the autonomousdatabases API properties: apiVersion: - description: 'APIVersion defines the versioned schema of this representation - of an object. Servers should convert recognized schemas to the latest - internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: - description: 'Kind is a string value representing the REST resource this - object represents. Servers may infer this from the endpoint the client - submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: - description: 'AutonomousDatabaseSpec defines the desired state of AutonomousDatabase - Important: Run "make" to regenerate code after modifying this file' + description: 'AutonomousDatabaseSpec defines the desired state of AutonomousDatabase Important: Run "make" to regenerate code after modifying this file' properties: details: - description: AutonomousDatabaseDetails defines the detail information - of AutonomousDatabase, corresponding to oci-go-sdk/database/AutonomousDatabase + description: AutonomousDatabaseDetails defines the detail information of AutonomousDatabase, corresponding to oci-go-sdk/database/AutonomousDatabase properties: adminPassword: properties: - k8sSecretName: - type: string - ociSecretOCID: - type: string + k8sSecret: + description: "*********************** *\tSecret specs ***********************" + properties: + name: + type: string + type: object + ociSecret: + properties: + ocid: + type: string + type: object + type: object + autonomousContainerDatabase: + description: ACDSpec defines the spec of the target for backup/restore runs. The name could be the name of an AutonomousDatabase or an AutonomousDatabaseBackup + properties: + k8sACD: + description: "*********************** *\tACD specs ***********************" + properties: + name: + type: string + type: object + ociACD: + properties: + ocid: + type: string + type: object type: object - autonomousContainerDatabaseOCID: - type: string autonomousDatabaseOCID: type: string compartmentOCID: @@ -304,8 +455,7 @@ spec: dbVersion: type: string dbWorkload: - description: 'AutonomousDatabaseDbWorkloadEnum Enum with underlying - type: string' + description: 'AutonomousDatabaseDbWorkloadEnum Enum with underlying type: string' enum: - OLTP - DW @@ -322,6 +472,12 @@ spec: type: boolean isDedicated: type: boolean + licenseModel: + description: 'AutonomousDatabaseLicenseModelEnum Enum with underlying type: string' + enum: + - LICENSE_INCLUDED + - BRING_YOUR_OWN_LICENSE + type: string lifecycleState: description: 'AutonomousDatabaseLifecycleStateEnum Enum with underlying type: string' type: string @@ -360,10 +516,17 @@ spec: type: string password: properties: - k8sSecretName: - type: string - ociSecretOCID: - type: string + k8sSecret: + description: "*********************** *\tSecret specs ***********************" + properties: + name: + type: string + type: object + ociSecret: + properties: + ocid: + type: string + type: object type: object type: object type: object @@ -371,6 +534,7 @@ spec: default: false type: boolean ociConfig: + description: "*********************** *\tOCI config ***********************" properties: configMapName: type: string @@ -387,33 +551,22 @@ spec: items: properties: connectionStrings: - additionalProperties: - type: string - type: object + items: + properties: + connectionString: + type: string + tnsName: + type: string + type: object + type: array tlsAuthentication: type: string required: - connectionStrings type: object type: array - cpuCoreCount: - type: integer - dataStorageSizeInTBs: - type: integer - dbWorkload: - description: 'AutonomousDatabaseDbWorkloadEnum Enum with underlying - type: string' - type: string - displayName: - description: 'INSERT ADDITIONAL STATUS FIELD - define observed state - of cluster Important: Run "make" to regenerate code after modifying - this file' - type: string - isDedicated: - type: string lifecycleState: - description: 'AutonomousDatabaseLifecycleStateEnum Enum with underlying - type: string' + description: 'INSERT ADDITIONAL STATUS FIELD - define observed state of cluster Important: Run "make" to regenerate code after modifying this file' type: string timeCreated: type: string @@ -481,14 +634,10 @@ spec: description: CDB is the Schema for the cdbs API properties: apiVersion: - description: 'APIVersion defines the versioned schema of this representation - of an object. Servers should convert recognized schemas to the latest - internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: - description: 'Kind is a string value representing the REST resource this - object represents. Servers may infer this from the endpoint the client - submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object @@ -513,8 +662,7 @@ spec: - secret type: object cdbAdminUser: - description: User in the root container with sysdba priviledges to - manage PDB lifecycle + description: User in the root container with sysdba priviledges to manage PDB lifecycle properties: secret: description: CDBSecret defines the secretName @@ -554,12 +702,10 @@ spec: - Never type: string ordsImagePullSecret: - description: The name of the image pull secret in case of a private - docker repository. + description: The name of the image pull secret in case of a private docker repository. type: string ordsPort: - description: ORDS server port. For now, keep it as 8888. TO BE USED - IN FUTURE RELEASE. + description: ORDS server port. For now, keep it as 8888. TO BE USED IN FUTURE RELEASE. type: integer ordsPwd: description: Password for user ORDS_PUBLIC_USER @@ -622,8 +768,7 @@ spec: - secret type: object webServerUser: - description: Web Server User with SQL Administrator role to allow - us to authenticate to the PDB Lifecycle Management REST endpoints + description: Web Server User with SQL Administrator role to allow us to authenticate to the PDB Lifecycle Management REST endpoints properties: secret: description: CDBSecret defines the secretName @@ -700,18 +845,13 @@ spec: name: v1alpha1 schema: openAPIV3Schema: - description: OracleRestDataService is the Schema for the oraclerestdataservices - API + description: OracleRestDataService is the Schema for the oraclerestdataservices API properties: apiVersion: - description: 'APIVersion defines the versioned schema of this representation - of an object. Servers should convert recognized schemas to the latest - internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: - description: 'Kind is a string value representing the REST resource this - object represents. Servers may infer this from the endpoint the client - submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object @@ -719,8 +859,7 @@ spec: description: OracleRestDataServiceSpec defines the desired state of OracleRestDataService properties: adminPassword: - description: OracleRestDataServicePassword defines the secret containing - Password mapped to secretKey + description: OracleRestDataServicePassword defines the secret containing Password mapped to secretKey properties: keepSecret: type: boolean @@ -733,8 +872,7 @@ spec: - secretKey type: object apexPassword: - description: OracleRestDataServicePassword defines the secret containing - Password mapped to secretKey + description: OracleRestDataServicePassword defines the secret containing Password mapped to secretKey properties: keepSecret: type: boolean @@ -749,8 +887,7 @@ spec: databaseRef: type: string image: - description: OracleRestDataServiceImage defines the Image source and - pullSecrets for POD + description: OracleRestDataServiceImage defines the Image source and pullSecrets for POD properties: pullFrom: type: string @@ -770,8 +907,7 @@ spec: oracleService: type: string ordsPassword: - description: OracleRestDataServicePassword defines the secret containing - Password mapped to secretKey + description: OracleRestDataServicePassword defines the secret containing Password mapped to secretKey properties: keepSecret: type: boolean @@ -786,8 +922,7 @@ spec: ordsUser: type: string persistence: - description: OracleRestDataServicePersistence defines the storage - releated params + description: OracleRestDataServicePersistence defines the storage releated params properties: accessMode: enum: @@ -804,8 +939,7 @@ spec: type: integer restEnableSchemas: items: - description: OracleRestDataServicePDBSchemas defines the PDB Schemas - to be ORDS Enabled + description: OracleRestDataServicePDBSchemas defines the PDB Schemas to be ORDS Enabled properties: enable: type: boolean @@ -829,8 +963,7 @@ spec: - ordsPassword type: object status: - description: OracleRestDataServiceStatus defines the observed state of - OracleRestDataService + description: OracleRestDataServiceStatus defines the observed state of OracleRestDataService properties: apexConfigured: type: boolean @@ -845,8 +978,7 @@ spec: databaseRef: type: string image: - description: OracleRestDataServiceImage defines the Image source and - pullSecrets for POD + description: OracleRestDataServiceImage defines the Image source and pullSecrets for POD properties: pullFrom: type: string @@ -868,9 +1000,7 @@ spec: serviceIP: type: string status: - description: 'INSERT ADDITIONAL STATUS FIELD - define observed state - of cluster Important: Run "make" to regenerate code after modifying - this file' + description: 'INSERT ADDITIONAL STATUS FIELD - define observed state of cluster Important: Run "make" to regenerate code after modifying this file' type: string type: object type: object @@ -936,14 +1066,10 @@ spec: description: PDB is the Schema for the pdbs API properties: apiVersion: - description: 'APIVersion defines the versioned schema of this representation - of an object. Servers should convert recognized schemas to the latest - internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: - description: 'Kind is a string value representing the REST resource this - object represents. Servers may infer this from the endpoint the client - submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object @@ -951,8 +1077,7 @@ spec: description: PDBSpec defines the desired state of PDB properties: action: - description: 'Action to be taken: Create/Clone/Plug/Unplug/Delete/Modify/Status/Map. - Map is used to map a Databse PDB to a Kubernetes PDB CR.' + description: 'Action to be taken: Create/Clone/Plug/Unplug/Delete/Modify/Status/Map. Map is used to map a Databse PDB to a Kubernetes PDB CR.' enum: - Create - Clone @@ -964,8 +1089,7 @@ spec: - Map type: string adminName: - description: The administrator username for the new PDB. This property - is required when the Action property is Create. + description: The administrator username for the new PDB. This property is required when the Action property is Create. properties: secret: description: PDBSecret defines the secretName @@ -982,8 +1106,7 @@ spec: - secret type: object adminPwd: - description: The administrator password for the new PDB. This property - is required when the Action property is Create. + description: The administrator password for the new PDB. This property is required when the Action property is Create. properties: secret: description: PDBSecret defines the secretName @@ -1000,9 +1123,7 @@ spec: - secret type: object asClone: - description: Indicate if 'AS CLONE' option should be used in the command - to plug in a PDB. This property is applicable when the Action property - is PLUG but not required. + description: Indicate if 'AS CLONE' option should be used in the command to plug in a PDB. This property is applicable when the Action property is PLUG but not required. type: boolean cdbName: description: Name of the CDB @@ -1018,16 +1139,13 @@ spec: - MOVE type: string dropAction: - description: Specify if datafiles should be removed or not. The value - can be INCLUDING or KEEP (default). + description: Specify if datafiles should be removed or not. The value can be INCLUDING or KEEP (default). enum: - INCLUDING - KEEP type: string fileNameConversions: - description: Relevant for Create and Plug operations. As defined in - the Oracle Multitenant Database documentation. Values can be a - filename convert pattern or NONE. + description: Relevant for Create and Plug operations. As defined in the Oracle Multitenant Database documentation. Values can be a filename convert pattern or NONE. type: string getScript: description: Whether you need the script only or execute the script @@ -1042,8 +1160,7 @@ spec: - RESTRICTED type: string pdbName: - description: The name of the new PDB. Relevant for both Create and - Plug Actions. + description: The name of the new PDB. Relevant for both Create and Plug Actions. type: string pdbState: description: The target state of the PDB @@ -1055,9 +1172,7 @@ spec: description: Whether to reuse temp file type: boolean sourceFileNameConversions: - description: This property is required when the Action property is - Plug. As defined in the Oracle Multitenant Database documentation. - Values can be a source filename convert pattern or NONE. + description: This property is required when the Action property is Plug. As defined in the Oracle Multitenant Database documentation. Values can be a source filename convert pattern or NONE. type: string sparseClonePath: description: A Path specified for sparse clone snapshot copy. (Optional) @@ -1072,12 +1187,10 @@ spec: description: TDE import for plug operations type: boolean tdeKeystorePath: - description: TDE keystore path is required if the tdeImport or tdeExport - flag is set to true. Can be used in plug or unplug operations. + description: TDE keystore path is required if the tdeImport or tdeExport flag is set to true. Can be used in plug or unplug operations. type: string tdePassword: - description: TDE password if the tdeImport or tdeExport flag is set - to true. Can be used in create, plug or unplug operations + description: TDE password if the tdeImport or tdeExport flag is set to true. Can be used in create, plug or unplug operations properties: secret: description: PDBSecret defines the secretName @@ -1094,8 +1207,7 @@ spec: - secret type: object tdeSecret: - description: TDE secret is required if the tdeImport or tdeExport - flag is set to true. Can be used in plug or unplug operations. + description: TDE secret is required if the tdeImport or tdeExport flag is set to true. Can be used in plug or unplug operations. properties: secret: description: PDBSecret defines the secretName @@ -1112,20 +1224,13 @@ spec: - secret type: object tempSize: - description: Relevant for Create and Clone operations. Total size - for temporary tablespace as defined in the Oracle Multitenant Database - documentation. See size_clause description in Database SQL Language - Reference documentation. + description: Relevant for Create and Clone operations. Total size for temporary tablespace as defined in the Oracle Multitenant Database documentation. See size_clause description in Database SQL Language Reference documentation. type: string totalSize: - description: Relevant for create and plug operations. Total size as - defined in the Oracle Multitenant Database documentation. See size_clause - description in Database SQL Language Reference documentation. + description: Relevant for create and plug operations. Total size as defined in the Oracle Multitenant Database documentation. See size_clause description in Database SQL Language Reference documentation. type: string unlimitedStorage: - description: Relevant for Create and Plug operations. True for unlimited - storage. Even when set to true, totalSize and tempSize MUST be specified - in the request if Action is Create. + description: Relevant for Create and Plug operations. True for unlimited storage. Even when set to true, totalSize and tempSize MUST be specified in the request if Action is Create. type: boolean xmlFileName: description: XML metadata filename to be used for Plug or Unplug operations @@ -1198,14 +1303,10 @@ spec: description: ShardingDatabase is the Schema for the shardingdatabases API properties: apiVersion: - description: 'APIVersion defines the versioned schema of this representation - of an object. Servers should convert recognized schemas to the latest - internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: - description: 'Kind is a string value representing the REST resource this - object represents. Servers may infer this from the endpoint the client - submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object @@ -1218,8 +1319,7 @@ spec: properties: envVars: items: - description: EnvironmentVariable represents a named variable - accessible for containers. + description: EnvironmentVariable represents a named variable accessible for containers. properties: name: type: string @@ -1231,8 +1331,7 @@ spec: type: object type: array imagePullPolicy: - description: PullPolicy describes a policy for if/when to pull - a container image + description: PullPolicy describes a policy for if/when to pull a container image type: string isDelete: type: boolean @@ -1255,8 +1354,7 @@ spec: pvcName: type: string resources: - description: ResourceRequirements describes the compute resource - requirements. + description: ResourceRequirements describes the compute resource requirements. properties: limits: additionalProperties: @@ -1265,8 +1363,7 @@ spec: - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true - description: 'Limits describes the maximum amount of compute - resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' type: object requests: additionalProperties: @@ -1275,11 +1372,7 @@ spec: - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true - description: 'Requests describes the minimum amount of compute - resources required. If Requests is omitted for a container, - it defaults to Limits if that is explicitly specified, - otherwise to an implementation-defined value. More info: - https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' type: object type: object storageSizeInGb: @@ -1299,8 +1392,7 @@ spec: properties: envVars: items: - description: EnvironmentVariable represents a named variable - accessible for containers. + description: EnvironmentVariable represents a named variable accessible for containers. properties: name: type: string @@ -1312,8 +1404,7 @@ spec: type: object type: array imagePullPolicy: - description: PullPolicy describes a policy for if/when to pull - a container image + description: PullPolicy describes a policy for if/when to pull a container image type: string isDelete: type: boolean @@ -1335,8 +1426,7 @@ spec: format: int32 type: integer resources: - description: ResourceRequirements describes the compute resource - requirements. + description: ResourceRequirements describes the compute resource requirements. properties: limits: additionalProperties: @@ -1345,8 +1435,7 @@ spec: - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true - description: 'Limits describes the maximum amount of compute - resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' type: object requests: additionalProperties: @@ -1355,11 +1444,7 @@ spec: - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true - description: 'Requests describes the minimum amount of compute - resources required. If Requests is omitted for a container, - it defaults to Limits if that is explicitly specified, - otherwise to an implementation-defined value. More info: - https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' type: object type: object storageSizeInGb: @@ -1391,8 +1476,7 @@ spec: type: string portMappings: items: - description: PortMapping is a specification of port mapping for - an application deployment. + description: PortMapping is a specification of port mapping for an application deployment. properties: port: format: int32 @@ -1414,16 +1498,13 @@ spec: secret: type: string shard: - description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster - Important: Run "make" to regenerate code after modifying this file' + description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster Important: Run "make" to regenerate code after modifying this file' items: - description: ShardSpec is a specification of Shards for an application - deployment. + description: ShardSpec is a specification of Shards for an application deployment. properties: envVars: items: - description: EnvironmentVariable represents a named variable - accessible for containers. + description: EnvironmentVariable represents a named variable accessible for containers. properties: name: type: string @@ -1435,8 +1516,7 @@ spec: type: object type: array imagePullPolicy: - description: PullPolicy describes a policy for if/when to pull - a container image + description: PullPolicy describes a policy for if/when to pull a container image type: string isDelete: type: boolean @@ -1459,8 +1539,7 @@ spec: pvcName: type: string resources: - description: ResourceRequirements describes the compute resource - requirements. + description: ResourceRequirements describes the compute resource requirements. properties: limits: additionalProperties: @@ -1469,8 +1548,7 @@ spec: - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true - description: 'Limits describes the maximum amount of compute - resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' type: object requests: additionalProperties: @@ -1479,11 +1557,7 @@ spec: - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true - description: 'Requests describes the minimum amount of compute - resources required. If Requests is omitted for a container, - it defaults to Limits if that is explicitly specified, - otherwise to an implementation-defined value. More info: - https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' type: object type: object storageSizeInGb: @@ -1506,8 +1580,7 @@ spec: - shard type: object status: - description: To understand Metav1.Condition, please refer the link https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1 - ShardingDatabaseStatus defines the observed state of ShardingDatabase + description: To understand Metav1.Condition, please refer the link https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1 ShardingDatabaseStatus defines the observed state of ShardingDatabase properties: catalogs: additionalProperties: @@ -1515,45 +1588,23 @@ spec: type: object conditions: items: - description: "Condition contains details for one aspect of the current - state of this API Resource. --- This struct is intended for direct - use as an array at the field path .status.conditions. For example, - type FooStatus struct{ // Represents the observations of a - foo's current state. // Known .status.conditions.type are: - \"Available\", \"Progressing\", and \"Degraded\" // +patchMergeKey=type - \ // +patchStrategy=merge // +listType=map // +listMapKey=type - \ Conditions []metav1.Condition `json:\"conditions,omitempty\" - patchStrategy:\"merge\" patchMergeKey:\"type\" protobuf:\"bytes,1,rep,name=conditions\"` - \n // other fields }" + description: "Condition contains details for one aspect of the current state of this API Resource. --- This struct is intended for direct use as an array at the field path .status.conditions. For example, type FooStatus struct{ // Represents the observations of a foo's current state. // Known .status.conditions.type are: \"Available\", \"Progressing\", and \"Degraded\" // +patchMergeKey=type // +patchStrategy=merge // +listType=map // +listMapKey=type Conditions []metav1.Condition `json:\"conditions,omitempty\" patchStrategy:\"merge\" patchMergeKey:\"type\" protobuf:\"bytes,1,rep,name=conditions\"` \n // other fields }" properties: lastTransitionTime: - description: lastTransitionTime is the last time the condition - transitioned from one status to another. This should be when - the underlying condition changed. If that is not known, then - using the time when the API field changed is acceptable. + description: lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. format: date-time type: string message: - description: message is a human readable message indicating - details about the transition. This may be an empty string. + description: message is a human readable message indicating details about the transition. This may be an empty string. maxLength: 32768 type: string observedGeneration: - description: observedGeneration represents the .metadata.generation - that the condition was set based upon. For instance, if .metadata.generation - is currently 12, but the .status.conditions[x].observedGeneration - is 9, the condition is out of date with respect to the current - state of the instance. + description: observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance. format: int64 minimum: 0 type: integer reason: - description: reason contains a programmatic identifier indicating - the reason for the condition's last transition. Producers - of specific condition types may define expected values and - meanings for this field, and whether the values are considered - a guaranteed API. The value should be a CamelCase string. - This field may not be empty. + description: reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty. maxLength: 1024 minLength: 1 pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ @@ -1566,11 +1617,7 @@ spec: - Unknown type: string type: - description: type of condition in CamelCase or in foo.example.com/CamelCase. - --- Many .condition.type values are consistent across resources - like Available, but because arbitrary conditions can be useful - (see .node.status.conditions), the ability to deconflict is - important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) + description: type of condition in CamelCase or in foo.example.com/CamelCase. --- Many .condition.type values are consistent across resources like Available, but because arbitrary conditions can be useful (see .node.status.conditions), the ability to deconflict is important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) maxLength: 316 pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ type: string @@ -1664,18 +1711,13 @@ spec: name: v1alpha1 schema: openAPIV3Schema: - description: SingleInstanceDatabase is the Schema for the singleinstancedatabases - API + description: SingleInstanceDatabase is the Schema for the singleinstancedatabases API properties: apiVersion: - description: 'APIVersion defines the versioned schema of this representation - of an object. Servers should convert recognized schemas to the latest - internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: - description: 'Kind is a string value representing the REST resource this - object represents. Servers may infer this from the endpoint the client - submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object @@ -1683,8 +1725,7 @@ spec: description: SingleInstanceDatabaseSpec defines the desired state of SingleInstanceDatabase properties: adminPassword: - description: SingleInsatnceAdminPassword defines the secret containing - Admin Password mapped to secretKey for Database + description: SingleInsatnceAdminPassword defines the secret containing Admin Password mapped to secretKey for Database properties: keepSecret: type: boolean @@ -1712,8 +1753,7 @@ spec: forceLog: type: boolean image: - description: SingleInstanceDatabaseImage defines the Image source - and pullSecrets for POD + description: SingleInstanceDatabaseImage defines the Image source and pullSecrets for POD properties: pullFrom: type: string @@ -1747,8 +1787,7 @@ spec: pdbName: type: string persistence: - description: SingleInstanceDatabasePersistence defines the storage - size and class for PVC + description: SingleInstanceDatabasePersistence defines the storage size and class for PVC properties: accessMode: type: string @@ -1767,16 +1806,14 @@ spec: serviceAccountName: type: string sid: - description: SID can only have a-z , A-Z, 0-9 . It cant have any special - characters + description: SID can only have a-z , A-Z, 0-9 . It cant have any special characters pattern: ^[a-zA-Z0-9]+$ type: string required: - image type: object status: - description: SingleInstanceDatabaseStatus defines the observed state of - SingleInstanceDatabase + description: SingleInstanceDatabaseStatus defines the observed state of SingleInstanceDatabase properties: apexInstalled: type: boolean @@ -1790,45 +1827,23 @@ spec: type: string conditions: items: - description: "Condition contains details for one aspect of the current - state of this API Resource. --- This struct is intended for direct - use as an array at the field path .status.conditions. For example, - type FooStatus struct{ // Represents the observations of a - foo's current state. // Known .status.conditions.type are: - \"Available\", \"Progressing\", and \"Degraded\" // +patchMergeKey=type - \ // +patchStrategy=merge // +listType=map // +listMapKey=type - \ Conditions []metav1.Condition `json:\"conditions,omitempty\" - patchStrategy:\"merge\" patchMergeKey:\"type\" protobuf:\"bytes,1,rep,name=conditions\"` - \n // other fields }" + description: "Condition contains details for one aspect of the current state of this API Resource. --- This struct is intended for direct use as an array at the field path .status.conditions. For example, type FooStatus struct{ // Represents the observations of a foo's current state. // Known .status.conditions.type are: \"Available\", \"Progressing\", and \"Degraded\" // +patchMergeKey=type // +patchStrategy=merge // +listType=map // +listMapKey=type Conditions []metav1.Condition `json:\"conditions,omitempty\" patchStrategy:\"merge\" patchMergeKey:\"type\" protobuf:\"bytes,1,rep,name=conditions\"` \n // other fields }" properties: lastTransitionTime: - description: lastTransitionTime is the last time the condition - transitioned from one status to another. This should be when - the underlying condition changed. If that is not known, then - using the time when the API field changed is acceptable. + description: lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. format: date-time type: string message: - description: message is a human readable message indicating - details about the transition. This may be an empty string. + description: message is a human readable message indicating details about the transition. This may be an empty string. maxLength: 32768 type: string observedGeneration: - description: observedGeneration represents the .metadata.generation - that the condition was set based upon. For instance, if .metadata.generation - is currently 12, but the .status.conditions[x].observedGeneration - is 9, the condition is out of date with respect to the current - state of the instance. + description: observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance. format: int64 minimum: 0 type: integer reason: - description: reason contains a programmatic identifier indicating - the reason for the condition's last transition. Producers - of specific condition types may define expected values and - meanings for this field, and whether the values are considered - a guaranteed API. The value should be a CamelCase string. - This field may not be empty. + description: reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty. maxLength: 1024 minLength: 1 pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ @@ -1841,11 +1856,7 @@ spec: - Unknown type: string type: - description: type of condition in CamelCase or in foo.example.com/CamelCase. - --- Many .condition.type values are consistent across resources - like Available, but because arbitrary conditions can be useful - (see .node.status.conditions), the ability to deconflict is - important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) + description: type of condition in CamelCase or in foo.example.com/CamelCase. --- Many .condition.type values are consistent across resources like Available, but because arbitrary conditions can be useful (see .node.status.conditions), the ability to deconflict is important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) maxLength: 316 pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ type: string @@ -1901,8 +1912,7 @@ spec: pdbName: type: string persistence: - description: SingleInstanceDatabasePersistence defines the storage - size and class for PVC + description: SingleInstanceDatabasePersistence defines the storage size and class for PVC properties: accessMode: type: string @@ -2139,12 +2149,22 @@ rules: - apiGroups: - database.oracle.com resources: - - autonomousdatabaseBackups + - autonomouscontainerdatabases verbs: - create - delete - get - list + - patch + - update + - watch +- apiGroups: + - database.oracle.com + resources: + - autonomouscontainerdatabases/status + verbs: + - get + - patch - update - apiGroups: - database.oracle.com @@ -2155,6 +2175,7 @@ rules: - delete - get - list + - update - watch - apiGroups: - database.oracle.com @@ -2173,6 +2194,7 @@ rules: - delete - get - list + - update - watch - apiGroups: - database.oracle.com @@ -2230,7 +2252,7 @@ rules: - apiGroups: - database.oracle.com resources: - - pdbs + - oraclerestdataservices verbs: - create - delete @@ -2242,17 +2264,13 @@ rules: - apiGroups: - database.oracle.com resources: - - pdbs/finalizers + - oraclerestdataservices/finalizers verbs: - - create - - delete - - get - - patch - update - apiGroups: - database.oracle.com resources: - - pdbs/status + - oraclerestdataservices/status verbs: - get - patch @@ -2260,7 +2278,7 @@ rules: - apiGroups: - database.oracle.com resources: - - shardingdatabases + - pdbs verbs: - create - delete @@ -2272,7 +2290,7 @@ rules: - apiGroups: - database.oracle.com resources: - - shardingdatabases/finalizers + - pdbs/finalizers verbs: - create - delete @@ -2282,7 +2300,7 @@ rules: - apiGroups: - database.oracle.com resources: - - shardingdatabases/status + - pdbs/status verbs: - get - patch @@ -2290,7 +2308,7 @@ rules: - apiGroups: - database.oracle.com resources: - - singleinstancedatabases + - shardingdatabases verbs: - create - delete @@ -2302,13 +2320,17 @@ rules: - apiGroups: - database.oracle.com resources: - - singleinstancedatabases/finalizers + - shardingdatabases/finalizers verbs: + - create + - delete + - get + - patch - update - apiGroups: - database.oracle.com resources: - - singleinstancedatabases/status + - shardingdatabases/status verbs: - get - patch @@ -2316,7 +2338,7 @@ rules: - apiGroups: - database.oracle.com resources: - - oraclerestdataservices + - singleinstancedatabases verbs: - create - delete @@ -2328,13 +2350,13 @@ rules: - apiGroups: - database.oracle.com resources: - - oraclerestdataservices/finalizers + - singleinstancedatabases/finalizers verbs: - update - apiGroups: - database.oracle.com resources: - - oraclerestdataservices/status + - singleinstancedatabases/status verbs: - get - patch @@ -2458,8 +2480,8 @@ spec: - --enable-leader-election command: - /manager - image: container-registry.oracle.com/database/operator:0.1.0 - imagePullPolicy: Always + image: controller:latest + imagePullPolicy: Never name: manager ports: - containerPort: 9443 @@ -2532,7 +2554,7 @@ webhooks: - CREATE - UPDATE resources: - - autonomousdatabase + - autonomousdatabases sideEffects: None - admissionReviewVersions: - v1 @@ -2561,9 +2583,9 @@ webhooks: service: name: oracle-database-operator-webhook-service namespace: oracle-database-operator-system - path: /mutate-database-oracle-com-v1alpha1-oraclerestdataservice + path: /mutate-database-oracle-com-v1alpha1-cdb failurePolicy: Fail - name: moraclerestdataservice.kb.io + name: mcdb.kb.io rules: - apiGroups: - database.oracle.com @@ -2573,7 +2595,7 @@ webhooks: - CREATE - UPDATE resources: - - oraclerestdataservices + - cdbs sideEffects: None - admissionReviewVersions: - v1 @@ -2582,9 +2604,9 @@ webhooks: service: name: oracle-database-operator-webhook-service namespace: oracle-database-operator-system - path: /mutate-database-oracle-com-v1alpha1-cdb + path: /mutate-database-oracle-com-v1alpha1-oraclerestdataservice failurePolicy: Fail - name: mcdb.kb.io + name: moraclerestdataservice.kb.io rules: - apiGroups: - database.oracle.com @@ -2594,7 +2616,7 @@ webhooks: - CREATE - UPDATE resources: - - cdbs + - oraclerestdataservices sideEffects: None - admissionReviewVersions: - v1 @@ -2714,9 +2736,9 @@ webhooks: service: name: oracle-database-operator-webhook-service namespace: oracle-database-operator-system - path: /validate-database-oracle-com-v1alpha1-oraclerestdataservice + path: /validate-database-oracle-com-v1alpha1-cdb failurePolicy: Fail - name: voraclerestdataservice.kb.io + name: vcdb.kb.io rules: - apiGroups: - database.oracle.com @@ -2726,7 +2748,7 @@ webhooks: - CREATE - UPDATE resources: - - oraclerestdataservices + - cdbs sideEffects: None - admissionReviewVersions: - v1 @@ -2735,9 +2757,9 @@ webhooks: service: name: oracle-database-operator-webhook-service namespace: oracle-database-operator-system - path: /validate-database-oracle-com-v1alpha1-cdb + path: /validate-database-oracle-com-v1alpha1-oraclerestdataservice failurePolicy: Fail - name: vcdb.kb.io + name: voraclerestdataservice.kb.io rules: - apiGroups: - database.oracle.com @@ -2747,7 +2769,7 @@ webhooks: - CREATE - UPDATE resources: - - cdbs + - oraclerestdataservices sideEffects: None - admissionReviewVersions: - v1 diff --git a/test/e2e/autonomouscontainerdatabase_controller_test.go b/test/e2e/autonomouscontainerdatabase_controller_test.go new file mode 100644 index 00000000..cc704ab5 --- /dev/null +++ b/test/e2e/autonomouscontainerdatabase_controller_test.go @@ -0,0 +1,125 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package e2etest + +import ( + "context" + "fmt" + "time" + + "github.com/oracle/oci-go-sdk/v54/common" + + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/types" + + dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + // +kubebuilder:scaffold:imports +) + +// These tests use Ginkgo (BDD-style Go testing framework). Refer to +// http://onsi.github.io/ginkgo/ to learn more about Ginkgo. + +var _ = Describe("test ACD binding with hardLink=true", func() { + // var adbLookupKey types.NamespacedName + const downloadedWallet = "instance-wallet-secret-1" + var acdID *string + + AfterEach(func() { + // IMPORTANT: The operator might have to call reconcile multiple times to finish an operation. + // If we do the update immediately, the previous reconciliation will overwrite the changes. + By("Sleeping 20 seconds to wait for reconciliation to finish") + time.Sleep(time.Second * 20) + }) + + It("should init the test", func() { + By("Save the database ID for later use") + acdID = common.String("ocid1.autonomouscontainerdatabase.oc1.phx.anyhqljsfj4qgxaah6iftchkd3i6co6bqudtzvrfuumkbpxcq6gcw5tjodea") + }) + + Describe("ACD binding with HardLink = false", func() { + It("Should create a AutonomousContainerDatabase resource with HardLink = false", func() { + acd := &dbv1alpha1.AutonomousContainerDatabase{ + TypeMeta: metav1.TypeMeta{ + APIVersion: "database.oracle.com/v1alpha1", + Kind: "AutonomousContainerDatabase", + }, + ObjectMeta: metav1.ObjectMeta{ + Name: "bindacd", + Namespace: ADBNamespace, + }, + Spec: dbv1alpha1.AutonomousContainerDatabaseSpec{ + AutonomousContainerDatabaseOCID: acdID, + HardLink: common.Bool(false), + OCIConfig: dbv1alpha1.OCIConfigSpec{ + ConfigMapName: common.String(SharedOCIConfigMapName), + SecretName: common.String(SharedOCISecretName), + }, + }, + } + + // adbLookupKey = types.NamespacedName{Name: adb.Name, Namespace: adb.Namespace} + + Expect(k8sClient.Create(context.TODO(), acd)).Should(Succeed()) + }) + + It("Should update the resource", func() { + acd := &dbv1alpha1.AutonomousContainerDatabase{} + + acdLookupKey := types.NamespacedName{Name: "bindacd", Namespace: ADBNamespace} + + Expect(k8sClient.Get(context.TODO(), acdLookupKey, acd)).Should(Succeed()) + + newDisplayName := "tinglwanACD" + acd.Spec.DisplayName = common.String(newDisplayName) + Expect(k8sClient.Update(context.TODO(), acd)).Should(Succeed()) + + Eventually(func() (string, error) { + createdACD := &dbv1alpha1.AutonomousContainerDatabase{} + err := k8sClient.Get(context.TODO(), acdLookupKey, createdACD) + if err != nil { + return "", err + } + + fmt.Println("============ test: displayName = " + string(*createdACD.Spec.DisplayName)) + + return *createdACD.Spec.DisplayName, nil + }, time.Second*10, time.Second*5).Should(Equal(newDisplayName)) + }) + }) +}) diff --git a/test/e2e/autonomousdatabase_controller_bind_test.go b/test/e2e/autonomousdatabase_controller_bind_test.go index e4ac2327..a028f711 100644 --- a/test/e2e/autonomousdatabase_controller_bind_test.go +++ b/test/e2e/autonomousdatabase_controller_bind_test.go @@ -88,10 +88,6 @@ var _ = Describe("test ADB binding with hardLink=true", func() { err = e2eutil.WaitUntilWorkCompleted(workClient, createResp.OpcWorkRequestId) Expect(err).ShouldNot(HaveOccurred()) - - // listResp, err := e2eutil.ListAutonomousDatabases(dbClient, &SharedCompartmentOCID, &dbName) - // Expect(err).ShouldNot(HaveOccurred()) - // fmt.Printf("List request DB %s is in %s state \n", *listResp.Items[0].DisplayName, listResp.Items[0].LifecycleState) }) Describe("ADB binding with HardLink = false using Wallet Password Secret", func() { @@ -111,7 +107,9 @@ var _ = Describe("test ADB binding with hardLink=true", func() { Wallet: dbv1alpha1.WalletSpec{ Name: common.String(downloadedWallet), Password: dbv1alpha1.PasswordSpec{ - K8sSecretName: common.String(SharedWalletPassSecretName), + K8sSecret: dbv1alpha1.K8sSecretSpec{ + Name: common.String(SharedWalletPassSecretName), + }, }, }, }, @@ -168,7 +166,9 @@ var _ = Describe("test ADB binding with hardLink=true", func() { Wallet: dbv1alpha1.WalletSpec{ Name: common.String(downloadedWallet), Password: dbv1alpha1.PasswordSpec{ - OCISecretOCID: common.String(SharedInstanceWalletPasswordOCID), + OCISecret: dbv1alpha1.OCISecretSpec{ + OCID: common.String(SharedInstanceWalletPasswordOCID), + }, }, }, }, diff --git a/test/e2e/autonomousdatabase_controller_create_test.go b/test/e2e/autonomousdatabase_controller_create_test.go index 17714d89..0bc0573c 100644 --- a/test/e2e/autonomousdatabase_controller_create_test.go +++ b/test/e2e/autonomousdatabase_controller_create_test.go @@ -91,14 +91,18 @@ var _ = Describe("test ADB provisioning", func() { DisplayName: common.String(dbName), CPUCoreCount: common.Int(1), AdminPassword: dbv1alpha1.PasswordSpec{ - K8sSecretName: common.String(SharedAdminPassSecretName), + K8sSecret: dbv1alpha1.K8sSecretSpec{ + Name: common.String(SharedAdminPassSecretName), + }, }, DataStorageSizeInTBs: common.Int(1), IsAutoScalingEnabled: common.Bool(true), Wallet: dbv1alpha1.WalletSpec{ Name: common.String(downloadedWallet), Password: dbv1alpha1.PasswordSpec{ - K8sSecretName: common.String(SharedWalletPassSecretName), + K8sSecret: dbv1alpha1.K8sSecretSpec{ + Name: common.String(SharedWalletPassSecretName), + }, }, }, }, @@ -132,7 +136,9 @@ var _ = Describe("test ADB provisioning", func() { DisplayName: common.String(dbName), CPUCoreCount: common.Int(1), AdminPassword: dbv1alpha1.PasswordSpec{ - K8sSecretName: common.String(SharedAdminPassSecretName), + K8sSecret: dbv1alpha1.K8sSecretSpec{ + Name: common.String(SharedAdminPassSecretName), + }, }, DataStorageSizeInTBs: common.Int(1), IsAutoScalingEnabled: common.Bool(true), @@ -150,6 +156,20 @@ var _ = Describe("test ADB provisioning", func() { It("Should check for local resource state \"\"", e2ebehavior.AssertLocalState(&k8sClient, &dupAdbLookupKey, "")) + It("Should cleanup the resource with duplicated db name", func() { + duplicateAdb := &dbv1alpha1.AutonomousDatabase{ + TypeMeta: metav1.TypeMeta{ + APIVersion: "database.oracle.com/v1alpha1", + Kind: "AutonomousDatabase", + }, + ObjectMeta: metav1.ObjectMeta{ + Name: duplicateAdbResourceName, + Namespace: ADBNamespace, + }, + } + Expect(k8sClient.Delete(context.TODO(), duplicateAdb)).To(Succeed()) + }) + It("Should download an instance wallet using the password from K8s Secret "+SharedWalletPassSecretName, e2ebehavior.AssertWallet(&k8sClient, &adbLookupKey)) It("Should delete the resource in cluster and terminate the database in OCI", e2ebehavior.AssertHardLinkDelete(&k8sClient, &dbClient, &adbLookupKey)) @@ -180,7 +200,9 @@ var _ = Describe("test ADB provisioning", func() { DisplayName: common.String(dbName), CPUCoreCount: common.Int(1), AdminPassword: dbv1alpha1.PasswordSpec{ - OCISecretOCID: common.String(SharedAdminPasswordOCID), + OCISecret: dbv1alpha1.OCISecretSpec{ + OCID: common.String(SharedAdminPasswordOCID), + }, }, DataStorageSizeInTBs: common.Int(1), IsAutoScalingEnabled: common.Bool(true), @@ -188,7 +210,9 @@ var _ = Describe("test ADB provisioning", func() { Wallet: dbv1alpha1.WalletSpec{ Name: common.String(downloadedWallet), Password: dbv1alpha1.PasswordSpec{ - OCISecretOCID: common.String(SharedInstanceWalletPasswordOCID), + OCISecret: dbv1alpha1.OCISecretSpec{ + OCID: common.String(SharedInstanceWalletPasswordOCID), + }, }, }, }, diff --git a/test/e2e/suite_test.go b/test/e2e/suite_test.go index 103c8612..3726049f 100644 --- a/test/e2e/suite_test.go +++ b/test/e2e/suite_test.go @@ -82,9 +82,12 @@ var ( BeforeSuite = ginkgo.BeforeSuite AfterSuite = ginkgo.AfterSuite Describe = ginkgo.Describe + PDescribe = ginkgo.PDescribe AfterEach = ginkgo.AfterEach By = ginkgo.By It = ginkgo.It + FIt = ginkgo.FIt + PIt = ginkgo.PIt Expect = gomega.Expect Succeed = gomega.Succeed HaveOccurred = gomega.HaveOccurred @@ -162,6 +165,30 @@ var _ = BeforeSuite(func(done ginkgo.Done) { }).SetupWithManager(k8sManager) Expect(err).ToNot(HaveOccurred()) + err = (&controllers.AutonomousDatabaseBackupReconciler{ + KubeClient: k8sManager.GetClient(), + Log: ctrl.Log.WithName("controllers").WithName("AutonomousDatabaseBakcup_test"), + Scheme: k8sManager.GetScheme(), + Recorder: k8sManager.GetEventRecorderFor("AutonomousDatabaseBakcup_test"), + }).SetupWithManager(k8sManager) + Expect(err).ToNot(HaveOccurred()) + + err = (&controllers.AutonomousDatabaseRestoreReconciler{ + KubeClient: k8sManager.GetClient(), + Log: ctrl.Log.WithName("controllers").WithName("AutonomousDatabaseRestore_test"), + Scheme: k8sManager.GetScheme(), + Recorder: k8sManager.GetEventRecorderFor("AutonomousDatabaseRestore_test"), + }).SetupWithManager(k8sManager) + Expect(err).ToNot(HaveOccurred()) + + err = (&controllers.AutonomousContainerDatabaseReconciler{ + KubeClient: k8sManager.GetClient(), + Log: ctrl.Log.WithName("controllers").WithName("AutonomousContainerDatabase_test"), + Scheme: k8sManager.GetScheme(), + Recorder: k8sManager.GetEventRecorderFor("AutonomousContainerDatabase_test"), + }).SetupWithManager(k8sManager) + Expect(err).ToNot(HaveOccurred()) + go func() { defer ginkgo.GinkgoRecover() err = k8sManager.Start(ctrl.SetupSignalHandler()) From d7143be1294b99a53e9f5b4787f27708c3d71266 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Mon, 21 Mar 2022 11:23:17 -0400 Subject: [PATCH 193/628] remove autonomouscontainerdatabase_controller_test --- ...nomouscontainerdatabase_controller_test.go | 125 ------------------ 1 file changed, 125 deletions(-) delete mode 100644 test/e2e/autonomouscontainerdatabase_controller_test.go diff --git a/test/e2e/autonomouscontainerdatabase_controller_test.go b/test/e2e/autonomouscontainerdatabase_controller_test.go deleted file mode 100644 index cc704ab5..00000000 --- a/test/e2e/autonomouscontainerdatabase_controller_test.go +++ /dev/null @@ -1,125 +0,0 @@ -/* -** Copyright (c) 2021 Oracle and/or its affiliates. -** -** The Universal Permissive License (UPL), Version 1.0 -** -** Subject to the condition set forth below, permission is hereby granted to any -** person obtaining a copy of this software, associated documentation and/or data -** (collectively the "Software"), free of charge and under any and all copyright -** rights in the Software, and any and all patent rights owned or freely -** licensable by each licensor hereunder covering either (i) the unmodified -** Software as contributed to or provided by such licensor, or (ii) the Larger -** Works (as defined below), to deal in both -** -** (a) the Software, and -** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if -** one is included with the Software (each a "Larger Work" to which the Software -** is contributed by such licensors), -** -** without restriction, including without limitation the rights to copy, create -** derivative works of, display, perform, and distribute the Software and make, -** use, sell, offer for sale, import, export, have made, and have sold the -** Software and the Larger Work(s), and to sublicense the foregoing rights on -** either these or other terms. -** -** This license is subject to the following condition: -** The above copyright notice and either this complete permission notice or at -** a minimum a reference to the UPL must be included in all copies or -** substantial portions of the Software. -** -** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -** SOFTWARE. - */ - -package e2etest - -import ( - "context" - "fmt" - "time" - - "github.com/oracle/oci-go-sdk/v54/common" - - metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" - "k8s.io/apimachinery/pkg/types" - - dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" - // +kubebuilder:scaffold:imports -) - -// These tests use Ginkgo (BDD-style Go testing framework). Refer to -// http://onsi.github.io/ginkgo/ to learn more about Ginkgo. - -var _ = Describe("test ACD binding with hardLink=true", func() { - // var adbLookupKey types.NamespacedName - const downloadedWallet = "instance-wallet-secret-1" - var acdID *string - - AfterEach(func() { - // IMPORTANT: The operator might have to call reconcile multiple times to finish an operation. - // If we do the update immediately, the previous reconciliation will overwrite the changes. - By("Sleeping 20 seconds to wait for reconciliation to finish") - time.Sleep(time.Second * 20) - }) - - It("should init the test", func() { - By("Save the database ID for later use") - acdID = common.String("ocid1.autonomouscontainerdatabase.oc1.phx.anyhqljsfj4qgxaah6iftchkd3i6co6bqudtzvrfuumkbpxcq6gcw5tjodea") - }) - - Describe("ACD binding with HardLink = false", func() { - It("Should create a AutonomousContainerDatabase resource with HardLink = false", func() { - acd := &dbv1alpha1.AutonomousContainerDatabase{ - TypeMeta: metav1.TypeMeta{ - APIVersion: "database.oracle.com/v1alpha1", - Kind: "AutonomousContainerDatabase", - }, - ObjectMeta: metav1.ObjectMeta{ - Name: "bindacd", - Namespace: ADBNamespace, - }, - Spec: dbv1alpha1.AutonomousContainerDatabaseSpec{ - AutonomousContainerDatabaseOCID: acdID, - HardLink: common.Bool(false), - OCIConfig: dbv1alpha1.OCIConfigSpec{ - ConfigMapName: common.String(SharedOCIConfigMapName), - SecretName: common.String(SharedOCISecretName), - }, - }, - } - - // adbLookupKey = types.NamespacedName{Name: adb.Name, Namespace: adb.Namespace} - - Expect(k8sClient.Create(context.TODO(), acd)).Should(Succeed()) - }) - - It("Should update the resource", func() { - acd := &dbv1alpha1.AutonomousContainerDatabase{} - - acdLookupKey := types.NamespacedName{Name: "bindacd", Namespace: ADBNamespace} - - Expect(k8sClient.Get(context.TODO(), acdLookupKey, acd)).Should(Succeed()) - - newDisplayName := "tinglwanACD" - acd.Spec.DisplayName = common.String(newDisplayName) - Expect(k8sClient.Update(context.TODO(), acd)).Should(Succeed()) - - Eventually(func() (string, error) { - createdACD := &dbv1alpha1.AutonomousContainerDatabase{} - err := k8sClient.Get(context.TODO(), acdLookupKey, createdACD) - if err != nil { - return "", err - } - - fmt.Println("============ test: displayName = " + string(*createdACD.Spec.DisplayName)) - - return *createdACD.Spec.DisplayName, nil - }, time.Second*10, time.Second*5).Should(Equal(newDisplayName)) - }) - }) -}) From a4edede1d362f688c17238ca58355fd8efdb1a49 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Mon, 21 Mar 2022 12:30:01 -0400 Subject: [PATCH 194/628] retore the oracle-database-operator.yaml to v0.1.0 --- oracle-database-operator.yaml | 1730 +++------------------------------ 1 file changed, 111 insertions(+), 1619 deletions(-) diff --git a/oracle-database-operator.yaml b/oracle-database-operator.yaml index fd538e47..ae464a57 100644 --- a/oracle-database-operator.yaml +++ b/oracle-database-operator.yaml @@ -7,354 +7,6 @@ metadata: --- apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition -metadata: - annotations: - controller-gen.kubebuilder.io/version: v0.6.1 - creationTimestamp: null - name: autonomouscontainerdatabases.database.oracle.com -spec: - group: database.oracle.com - names: - kind: AutonomousContainerDatabase - listKind: AutonomousContainerDatabaseList - plural: autonomouscontainerdatabases - shortNames: - - acd - - acds - singular: autonomouscontainerdatabase - scope: Namespaced - versions: - - additionalPrinterColumns: - - jsonPath: .spec.displayName - name: DisplayName - type: string - - jsonPath: .status.lifecycleState - name: State - type: string - - jsonPath: .status.timeCreated - name: Created - type: string - name: v1alpha1 - schema: - openAPIV3Schema: - description: AutonomousContainerDatabase is the Schema for the autonomouscontainerdatabases API - properties: - apiVersion: - description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' - type: string - kind: - description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' - type: string - metadata: - type: object - spec: - description: AutonomousContainerDatabaseSpec defines the desired state of AutonomousContainerDatabase - properties: - autonomousContainerDatabaseOCID: - description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster Important: Run "make" to regenerate code after modifying this file' - type: string - autonomousExadataVMClusterOCID: - type: string - compartmentOCID: - type: string - displayName: - type: string - freeformTags: - additionalProperties: - type: string - type: object - hardLink: - default: false - type: boolean - ociConfig: - description: "*********************** *\tOCI config ***********************" - properties: - configMapName: - type: string - secretName: - type: string - type: object - patchModel: - description: 'AutonomousContainerDatabasePatchModelEnum Enum with underlying type: string' - enum: - - RELEASE_UPDATES - - RELEASE_UPDATE_REVISIONS - type: string - type: object - status: - description: AutonomousContainerDatabaseStatus defines the observed state of AutonomousContainerDatabase - properties: - lifecycleState: - description: 'INSERT ADDITIONAL STATUS FIELD - define observed state of cluster Important: Run "make" to regenerate code after modifying this file' - type: string - timeCreated: - type: string - required: - - lifecycleState - type: object - type: object - served: true - storage: true - subresources: - status: {} -status: - acceptedNames: - kind: "" - plural: "" - conditions: [] - storedVersions: [] ---- -apiVersion: apiextensions.k8s.io/v1 -kind: CustomResourceDefinition -metadata: - annotations: - controller-gen.kubebuilder.io/version: v0.6.1 - creationTimestamp: null - name: autonomousdatabasebackups.database.oracle.com -spec: - group: database.oracle.com - names: - kind: AutonomousDatabaseBackup - listKind: AutonomousDatabaseBackupList - plural: autonomousdatabasebackups - shortNames: - - adbbu - - adbbus - singular: autonomousdatabasebackup - scope: Namespaced - versions: - - additionalPrinterColumns: - - jsonPath: .status.lifecycleState - name: State - type: string - - jsonPath: .status.dbDisplayName - name: DB DisplayName - type: string - - jsonPath: .status.type - name: Type - type: string - - jsonPath: .status.timeStarted - name: Started - type: string - - jsonPath: .status.timeEnded - name: Ended - type: string - name: v1alpha1 - schema: - openAPIV3Schema: - description: AutonomousDatabaseBackup is the Schema for the autonomousdatabasebackups API - properties: - apiVersion: - description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' - type: string - kind: - description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' - type: string - metadata: - type: object - spec: - description: AutonomousDatabaseBackupSpec defines the desired state of AutonomousDatabaseBackup - properties: - autonomousDatabaseBackupOCID: - type: string - displayName: - type: string - ociConfig: - description: "*********************** *\tOCI config ***********************" - properties: - configMapName: - type: string - secretName: - type: string - type: object - target: - description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster Important: Run "make" to regenerate code after modifying this file' - properties: - k8sADB: - description: "*********************** *\tADB spec ***********************" - properties: - name: - type: string - type: object - ociADB: - properties: - ocid: - type: string - type: object - type: object - required: - - displayName - - target - type: object - status: - description: AutonomousDatabaseBackupStatus defines the observed state of AutonomousDatabaseBackup - properties: - autonomousDatabaseBackupOCID: - description: 'INSERT ADDITIONAL STATUS FIELD - define observed state of cluster Important: Run "make" to regenerate code after modifying this file' - type: string - autonomousDatabaseOCID: - type: string - compartmentOCID: - type: string - dbDisplayName: - type: string - dbName: - type: string - displayName: - type: string - isAutomatic: - type: boolean - lifecycleState: - description: 'AutonomousDatabaseBackupLifecycleStateEnum Enum with underlying type: string' - type: string - timeEnded: - type: string - timeStarted: - type: string - type: - description: 'AutonomousDatabaseBackupTypeEnum Enum with underlying type: string' - type: string - required: - - autonomousDatabaseOCID - - compartmentOCID - - dbDisplayName - - dbName - - displayName - - isAutomatic - - lifecycleState - - type - type: object - type: object - served: true - storage: true - subresources: - status: {} -status: - acceptedNames: - kind: "" - plural: "" - conditions: [] - storedVersions: [] ---- -apiVersion: apiextensions.k8s.io/v1 -kind: CustomResourceDefinition -metadata: - annotations: - controller-gen.kubebuilder.io/version: v0.6.1 - creationTimestamp: null - name: autonomousdatabaserestores.database.oracle.com -spec: - group: database.oracle.com - names: - kind: AutonomousDatabaseRestore - listKind: AutonomousDatabaseRestoreList - plural: autonomousdatabaserestores - shortNames: - - adbr - - adbrs - singular: autonomousdatabaserestore - scope: Namespaced - versions: - - additionalPrinterColumns: - - jsonPath: .status.status - name: Status - type: string - - jsonPath: .status.displayName - name: DisplayName - type: string - - jsonPath: .status.dbName - name: DbName - type: string - name: v1alpha1 - schema: - openAPIV3Schema: - description: AutonomousDatabaseRestore is the Schema for the autonomousdatabaserestores API - properties: - apiVersion: - description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' - type: string - kind: - description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' - type: string - metadata: - type: object - spec: - description: AutonomousDatabaseRestoreSpec defines the desired state of AutonomousDatabaseRestore - properties: - ociConfig: - description: "*********************** *\tOCI config ***********************" - properties: - configMapName: - type: string - secretName: - type: string - type: object - source: - properties: - k8sADBBackup: - description: 'EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized.' - properties: - name: - type: string - type: object - pointInTime: - properties: - timestamp: - description: 'The timestamp must follow this format: YYYY-MM-DD HH:MM:SS GMT' - type: string - type: object - type: object - target: - description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster Important: Run "make" to regenerate code after modifying this file' - properties: - k8sADB: - description: "*********************** *\tADB spec ***********************" - properties: - name: - type: string - type: object - ociADB: - properties: - ocid: - type: string - type: object - type: object - required: - - source - - target - type: object - status: - description: AutonomousDatabaseRestoreStatus defines the observed state of AutonomousDatabaseRestore - properties: - autonomousDatabaseOCID: - type: string - dbName: - type: string - displayName: - description: 'INSERT ADDITIONAL STATUS FIELD - define observed state of cluster Important: Run "make" to regenerate code after modifying this file' - type: string - status: - type: string - required: - - autonomousDatabaseOCID - - dbName - - displayName - - status - type: object - type: object - served: true - storage: true - subresources: - status: {} -status: - acceptedNames: - kind: "" - plural: "" - conditions: [] - storedVersions: [] ---- -apiVersion: apiextensions.k8s.io/v1 -kind: CustomResourceDefinition metadata: annotations: controller-gen.kubebuilder.io/version: v0.6.1 @@ -373,22 +25,22 @@ spec: scope: Namespaced versions: - additionalPrinterColumns: - - jsonPath: .spec.details.displayName + - jsonPath: .status.displayName name: Display Name type: string - jsonPath: .status.lifecycleState name: State type: string - - jsonPath: .spec.details.isDedicated + - jsonPath: .status.isDedicated name: Dedicated type: string - - jsonPath: .spec.details.cpuCoreCount + - jsonPath: .status.cpuCoreCount name: OCPUs type: integer - - jsonPath: .spec.details.dataStorageSizeInTBs + - jsonPath: .status.dataStorageSizeInTBs name: Storage (TB) type: integer - - jsonPath: .spec.details.dbWorkload + - jsonPath: .status.dbWorkload name: Workload Type type: string - jsonPath: .status.timeCreated @@ -415,32 +67,10 @@ spec: properties: adminPassword: properties: - k8sSecret: - description: "*********************** *\tSecret specs ***********************" - properties: - name: - type: string - type: object - ociSecret: - properties: - ocid: - type: string - type: object - type: object - autonomousContainerDatabase: - description: ACDSpec defines the spec of the target for backup/restore runs. The name could be the name of an AutonomousDatabase or an AutonomousDatabaseBackup - properties: - k8sACD: - description: "*********************** *\tACD specs ***********************" - properties: - name: - type: string - type: object - ociACD: - properties: - ocid: - type: string - type: object + k8sSecretName: + type: string + ociSecretOCID: + type: string type: object autonomousDatabaseOCID: type: string @@ -472,802 +102,67 @@ spec: type: boolean isDedicated: type: boolean - licenseModel: - description: 'AutonomousDatabaseLicenseModelEnum Enum with underlying type: string' - enum: - - LICENSE_INCLUDED - - BRING_YOUR_OWN_LICENSE - type: string lifecycleState: description: 'AutonomousDatabaseLifecycleStateEnum Enum with underlying type: string' type: string - networkAccess: - properties: - accessControlList: - items: - type: string - type: array - accessType: - enum: - - "" - - PUBLIC - - RESTRICTED - - PRIVATE - type: string - isAccessControlEnabled: - type: boolean - isMTLSConnectionRequired: - type: boolean - privateEndpoint: - properties: - hostnamePrefix: - type: string - nsgOCIDs: - items: - type: string - type: array - subnetOCID: - type: string - type: object - type: object - wallet: - properties: - name: - type: string - password: - properties: - k8sSecret: - description: "*********************** *\tSecret specs ***********************" - properties: - name: - type: string - type: object - ociSecret: - properties: - ocid: - type: string - type: object - type: object - type: object - type: object - hardLink: - default: false - type: boolean - ociConfig: - description: "*********************** *\tOCI config ***********************" - properties: - configMapName: - type: string - secretName: - type: string - type: object - required: - - details - type: object - status: - description: AutonomousDatabaseStatus defines the observed state of AutonomousDatabase - properties: - allConnectionStrings: - items: - properties: - connectionStrings: - items: - properties: - connectionString: - type: string - tnsName: - type: string - type: object - type: array - tlsAuthentication: + nsgOCIDs: + items: type: string - required: - - connectionStrings - type: object - type: array - lifecycleState: - description: 'INSERT ADDITIONAL STATUS FIELD - define observed state of cluster Important: Run "make" to regenerate code after modifying this file' - type: string - timeCreated: - type: string - type: object - type: object - served: true - storage: true - subresources: - status: {} -status: - acceptedNames: - kind: "" - plural: "" - conditions: [] - storedVersions: [] ---- -apiVersion: apiextensions.k8s.io/v1 -kind: CustomResourceDefinition -metadata: - annotations: - cert-manager.io/inject-ca-from: oracle-database-operator-system/oracle-database-operator-serving-cert - controller-gen.kubebuilder.io/version: v0.6.1 - name: cdbs.database.oracle.com -spec: - group: database.oracle.com - names: - kind: CDB - listKind: CDBList - plural: cdbs - singular: cdb - scope: Namespaced - versions: - - additionalPrinterColumns: - - description: Name of the CDB - jsonPath: .spec.cdbName - name: CDB Name - type: string - - description: ' Name of the DB Server' - jsonPath: .spec.dbServer - name: DB Server - type: string - - description: DB server port - jsonPath: .spec.dbPort - name: DB Port - type: integer - - description: SCAN Name - jsonPath: .spec.scanName - name: SCAN Name - type: string - - description: Replicas - jsonPath: .spec.replicas - name: Replicas - type: integer - - description: Status of the CDB Resource - jsonPath: .status.phase - name: Status - type: string - - description: Error message, if any - jsonPath: .status.msg - name: Message - type: string - name: v1alpha1 - schema: - openAPIV3Schema: - description: CDB is the Schema for the cdbs API - properties: - apiVersion: - description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' - type: string - kind: - description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' - type: string - metadata: - type: object - spec: - description: CDBSpec defines the desired state of CDB - properties: - cdbAdminPwd: - description: Password for the CDB Administrator to manage PDB lifecycle - properties: - secret: - description: CDBSecret defines the secretName - properties: - key: - type: string - secretName: - type: string - required: - - key - - secretName - type: object - required: - - secret - type: object - cdbAdminUser: - description: User in the root container with sysdba priviledges to manage PDB lifecycle - properties: - secret: - description: CDBSecret defines the secretName - properties: - key: - type: string - secretName: - type: string - required: - - key - - secretName - type: object - required: - - secret - type: object - cdbName: - description: Name of the CDB - type: string - dbPort: - description: DB server port - type: integer - dbServer: - description: Name of the DB server - type: string - nodeSelector: - additionalProperties: - type: string - description: Node Selector for running the Pod - type: object - ordsImage: - description: ORDS Image Name - type: string - ordsImagePullPolicy: - description: ORDS Image Pull Policy - enum: - - Always - - Never - type: string - ordsImagePullSecret: - description: The name of the image pull secret in case of a private docker repository. - type: string - ordsPort: - description: ORDS server port. For now, keep it as 8888. TO BE USED IN FUTURE RELEASE. - type: integer - ordsPwd: - description: Password for user ORDS_PUBLIC_USER - properties: - secret: - description: CDBSecret defines the secretName - properties: - key: - type: string - secretName: - type: string - required: - - key - - secretName - type: object - required: - - secret - type: object - replicas: - description: Number of ORDS Containers to create - type: integer - scanName: - description: SCAN Name - type: string - serviceName: - description: Name of the CDB Service - type: string - sysAdminPwd: - description: Password for the CDB System Administrator - properties: - secret: - description: CDBSecret defines the secretName - properties: - key: - type: string - secretName: - type: string - required: - - key - - secretName - type: object - required: - - secret - type: object - webServerPwd: - description: Password for the Web Server User - properties: - secret: - description: CDBSecret defines the secretName - properties: - key: - type: string - secretName: - type: string - required: - - key - - secretName - type: object - required: - - secret - type: object - webServerUser: - description: Web Server User with SQL Administrator role to allow us to authenticate to the PDB Lifecycle Management REST endpoints - properties: - secret: - description: CDBSecret defines the secretName - properties: - key: - type: string - secretName: - type: string - required: - - key - - secretName - type: object - required: - - secret - type: object - type: object - status: - description: CDBStatus defines the observed state of CDB - properties: - msg: - description: Message - type: string - phase: - description: Phase of the CDB Resource - type: string - status: - description: CDB Resource Status - type: boolean - required: - - phase - - status - type: object - type: object - served: true - storage: true - subresources: - status: {} -status: - acceptedNames: - kind: "" - plural: "" - conditions: [] - storedVersions: [] ---- -apiVersion: apiextensions.k8s.io/v1 -kind: CustomResourceDefinition -metadata: - annotations: - controller-gen.kubebuilder.io/version: v0.6.1 - creationTimestamp: null - name: oraclerestdataservices.database.oracle.com -spec: - group: database.oracle.com - names: - kind: OracleRestDataService - listKind: OracleRestDataServiceList - plural: oraclerestdataservices - singular: oraclerestdataservice - scope: Namespaced - versions: - - additionalPrinterColumns: - - jsonPath: .status.status - name: Status - type: string - - jsonPath: .spec.databaseRef - name: Database - type: string - - jsonPath: .status.databaseApiUrl - name: 'Database Api & Apex Url ' - type: string - - jsonPath: .status.databaseActionsUrl - name: Database Actions Url - type: string - name: v1alpha1 - schema: - openAPIV3Schema: - description: OracleRestDataService is the Schema for the oraclerestdataservices API - properties: - apiVersion: - description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' - type: string - kind: - description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' - type: string - metadata: - type: object - spec: - description: OracleRestDataServiceSpec defines the desired state of OracleRestDataService - properties: - adminPassword: - description: OracleRestDataServicePassword defines the secret containing Password mapped to secretKey - properties: - keepSecret: - type: boolean - secretKey: - type: string - secretName: - type: string - required: - - keepSecret - - secretKey - type: object - apexPassword: - description: OracleRestDataServicePassword defines the secret containing Password mapped to secretKey - properties: - keepSecret: - type: boolean - secretKey: - type: string - secretName: - type: string - required: - - keepSecret - - secretKey - type: object - databaseRef: - type: string - image: - description: OracleRestDataServiceImage defines the Image source and pullSecrets for POD - properties: - pullFrom: - type: string - pullSecrets: - type: string - version: - type: string - required: - - pullFrom - type: object - loadBalancer: - type: boolean - nodeSelector: - additionalProperties: - type: string - type: object - oracleService: - type: string - ordsPassword: - description: OracleRestDataServicePassword defines the secret containing Password mapped to secretKey - properties: - keepSecret: - type: boolean - secretKey: - type: string - secretName: - type: string - required: - - keepSecret - - secretKey - type: object - ordsUser: - type: string - persistence: - description: OracleRestDataServicePersistence defines the storage releated params - properties: - accessMode: - enum: - - ReadWriteOnce - - ReadWriteMany + type: array + privateEndpoint: type: string - size: - type: string - storageClass: - type: string - type: object - replicas: - minimum: 1 - type: integer - restEnableSchemas: - items: - description: OracleRestDataServicePDBSchemas defines the PDB Schemas to be ORDS Enabled - properties: - enable: - type: boolean - pdb: - type: string - schema: - type: string - urlMapping: - type: string - required: - - enable - - pdb - - schema - type: object - type: array - serviceAccountName: - type: string - required: - - adminPassword - - databaseRef - - ordsPassword - type: object - status: - description: OracleRestDataServiceStatus defines the observed state of OracleRestDataService - properties: - apexConfigured: - type: boolean - clusterDbApiUrl: - type: string - commonUsersCreated: - type: boolean - databaseActionsUrl: - type: string - databaseApiUrl: - type: string - databaseRef: - type: string - image: - description: OracleRestDataServiceImage defines the Image source and pullSecrets for POD - properties: - pullFrom: + privateEndpointIP: type: string - pullSecrets: + privateEndpointLabel: type: string - version: + subnetOCID: type: string - required: - - pullFrom - type: object - loadBalancer: - type: string - ordsInstalled: - type: boolean - ordsSetupCompleted: - type: boolean - replicas: - type: integer - serviceIP: - type: string - status: - description: 'INSERT ADDITIONAL STATUS FIELD - define observed state of cluster Important: Run "make" to regenerate code after modifying this file' - type: string - type: object - type: object - served: true - storage: true - subresources: - status: {} -status: - acceptedNames: - kind: "" - plural: "" - conditions: [] - storedVersions: [] ---- -apiVersion: apiextensions.k8s.io/v1 -kind: CustomResourceDefinition -metadata: - annotations: - cert-manager.io/inject-ca-from: oracle-database-operator-system/oracle-database-operator-serving-cert - controller-gen.kubebuilder.io/version: v0.6.1 - name: pdbs.database.oracle.com -spec: - group: database.oracle.com - names: - kind: PDB - listKind: PDBList - plural: pdbs - singular: pdb - scope: Namespaced - versions: - - additionalPrinterColumns: - - description: The connect string to be used - jsonPath: .status.connString - name: Connect String - type: string - - description: Name of the CDB - jsonPath: .spec.cdbName - name: CDB Name - type: string - - description: Name of the PDB - jsonPath: .spec.pdbName - name: PDB Name - type: string - - description: PDB Open Mode - jsonPath: .status.openMode - name: PDB State - type: string - - description: Total Size of the PDB - jsonPath: .status.totalSize - name: PDB Size - type: string - - description: Status of the PDB Resource - jsonPath: .status.phase - name: Status - type: string - - description: Error message, if any - jsonPath: .status.msg - name: Message - type: string - name: v1alpha1 - schema: - openAPIV3Schema: - description: PDB is the Schema for the pdbs API - properties: - apiVersion: - description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' - type: string - kind: - description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' - type: string - metadata: - type: object - spec: - description: PDBSpec defines the desired state of PDB - properties: - action: - description: 'Action to be taken: Create/Clone/Plug/Unplug/Delete/Modify/Status/Map. Map is used to map a Databse PDB to a Kubernetes PDB CR.' - enum: - - Create - - Clone - - Plug - - Unplug - - Delete - - Modify - - Status - - Map - type: string - adminName: - description: The administrator username for the new PDB. This property is required when the Action property is Create. - properties: - secret: - description: PDBSecret defines the secretName - properties: - key: - type: string - secretName: - type: string - required: - - key - - secretName - type: object - required: - - secret - type: object - adminPwd: - description: The administrator password for the new PDB. This property is required when the Action property is Create. - properties: - secret: - description: PDBSecret defines the secretName - properties: - key: - type: string - secretName: - type: string - required: - - key - - secretName - type: object - required: - - secret - type: object - asClone: - description: Indicate if 'AS CLONE' option should be used in the command to plug in a PDB. This property is applicable when the Action property is PLUG but not required. - type: boolean - cdbName: - description: Name of the CDB - type: string - cdbResName: - description: Name of the CDB Custom Resource that runs the ORDS container - type: string - copyAction: - description: To copy files or not while cloning a PDB - enum: - - COPY - - NOCOPY - - MOVE - type: string - dropAction: - description: Specify if datafiles should be removed or not. The value can be INCLUDING or KEEP (default). - enum: - - INCLUDING - - KEEP - type: string - fileNameConversions: - description: Relevant for Create and Plug operations. As defined in the Oracle Multitenant Database documentation. Values can be a filename convert pattern or NONE. - type: string - getScript: - description: Whether you need the script only or execute the script - type: boolean - modifyOption: - description: Extra options for opening and closing a PDB - enum: - - IMMEDIATE - - NORMAL - - READ ONLY - - READ WRITE - - RESTRICTED - type: string - pdbName: - description: The name of the new PDB. Relevant for both Create and Plug Actions. - type: string - pdbState: - description: The target state of the PDB - enum: - - OPEN - - CLOSE - type: string - reuseTempFile: - description: Whether to reuse temp file - type: boolean - sourceFileNameConversions: - description: This property is required when the Action property is Plug. As defined in the Oracle Multitenant Database documentation. Values can be a source filename convert pattern or NONE. - type: string - sparseClonePath: - description: A Path specified for sparse clone snapshot copy. (Optional) - type: string - srcPdbName: - description: Name of the Source PDB from which to clone - type: string - tdeExport: - description: TDE export for unplug operations - type: boolean - tdeImport: - description: TDE import for plug operations - type: boolean - tdeKeystorePath: - description: TDE keystore path is required if the tdeImport or tdeExport flag is set to true. Can be used in plug or unplug operations. - type: string - tdePassword: - description: TDE password if the tdeImport or tdeExport flag is set to true. Can be used in create, plug or unplug operations - properties: - secret: - description: PDBSecret defines the secretName + wallet: properties: - key: - type: string - secretName: + name: type: string - required: - - key - - secretName + password: + properties: + k8sSecretName: + type: string + ociSecretOCID: + type: string + type: object type: object - required: - - secret type: object - tdeSecret: - description: TDE secret is required if the tdeImport or tdeExport flag is set to true. Can be used in plug or unplug operations. + hardLink: + default: false + type: boolean + ociConfig: properties: - secret: - description: PDBSecret defines the secretName - properties: - key: - type: string - secretName: - type: string - required: - - key - - secretName - type: object - required: - - secret + configMapName: + type: string + secretName: + type: string type: object - tempSize: - description: Relevant for Create and Clone operations. Total size for temporary tablespace as defined in the Oracle Multitenant Database documentation. See size_clause description in Database SQL Language Reference documentation. - type: string - totalSize: - description: Relevant for create and plug operations. Total size as defined in the Oracle Multitenant Database documentation. See size_clause description in Database SQL Language Reference documentation. - type: string - unlimitedStorage: - description: Relevant for Create and Plug operations. True for unlimited storage. Even when set to true, totalSize and tempSize MUST be specified in the request if Action is Create. - type: boolean - xmlFileName: - description: XML metadata filename to be used for Plug or Unplug operations - type: string required: - - action + - details type: object status: - description: PDBStatus defines the observed state of PDB + description: AutonomousDatabaseStatus defines the observed state of AutonomousDatabase properties: - action: - description: Last Completed Action - type: string - connString: - description: PDB Connect String - type: string - modifyOption: - description: Modify Option of the PDB + cpuCoreCount: + type: integer + dataStorageSizeInTBs: + type: integer + dbWorkload: + description: 'AutonomousDatabaseDbWorkloadEnum Enum with underlying type: string' type: string - msg: - description: Message + displayName: + description: 'INSERT ADDITIONAL STATUS FIELD - define observed state of cluster Important: Run "make" to regenerate code after modifying this file' type: string - openMode: - description: Open mode of the PDB + isDedicated: type: string - phase: - description: Phase of the PDB Resource + lifecycleState: + description: 'AutonomousDatabaseLifecycleStateEnum Enum with underlying type: string' type: string - status: - description: PDB Resource Status - type: boolean - totalSize: - description: Total size of the PDB + timeCreated: type: string - required: - - phase - - status type: object type: object served: true @@ -1790,27 +685,33 @@ spec: description: SingleInstanceDatabasePersistence defines the storage size and class for PVC properties: accessMode: + enum: + - ReadWriteOnce + - ReadWriteMany type: string size: type: string storageClass: type: string required: + - accessMode - size - storageClass type: object readinessCheckPeriod: type: integer replicas: + minimum: 1 type: integer - serviceAccountName: - type: string sid: description: SID can only have a-z , A-Z, 0-9 . It cant have any special characters pattern: ^[a-zA-Z0-9]+$ type: string required: + - adminPassword - image + - persistence + - replicas type: object status: description: SingleInstanceDatabaseStatus defines the observed state of SingleInstanceDatabase @@ -1915,12 +816,16 @@ spec: description: SingleInstanceDatabasePersistence defines the storage size and class for PVC properties: accessMode: + enum: + - ReadWriteOnce + - ReadWriteMany type: string size: type: string storageClass: type: string required: + - accessMode - size - storageClass type: object @@ -2010,223 +915,10 @@ metadata: name: oracle-database-operator-manager-role rules: - apiGroups: - - "" - resources: - - configmaps - - events - - pods - - pods/exec - - pods/log - - replicasets - - services - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - "" - resources: - - configmaps - - secrets - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - "" - resources: - - events - - nodes - - persistentvolumeclaims - - pods - - pods/exec - - pods/log - - services - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - '''''' - resources: - - statefulsets/finalizers - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - apps - resources: - - replicasets - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - apps - resources: - - statefulsets - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - coordination.k8s.io - resources: - - leases - verbs: - - create - - get - - list - - update -- apiGroups: - - "" - resources: - - configmaps - - events - - namespaces - - nodes - - persistentvolumeclaims - - pods - - pods/exec - - pods/log - - secrets - - services - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - "" - resources: - - configmaps - - namespaces - - pods - - secrets - - services - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - "" - resources: - - pods/exec - verbs: - - create -- apiGroups: - - database.oracle.com - resources: - - autonomouscontainerdatabases - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - database.oracle.com - resources: - - autonomouscontainerdatabases/status - verbs: - - get - - patch - - update -- apiGroups: - - database.oracle.com - resources: - - autonomousdatabasebackups - verbs: - - create - - delete - - get - - list - - update - - watch -- apiGroups: - - database.oracle.com - resources: - - autonomousdatabasebackups/status - verbs: - - get - - patch - - update -- apiGroups: - - database.oracle.com - resources: - - autonomousdatabaserestores - verbs: - - create - - delete - - get - - list - - update - - watch -- apiGroups: - - database.oracle.com - resources: - - autonomousdatabaserestores/status - verbs: - - get - - patch - - update -- apiGroups: - - database.oracle.com - resources: - - autonomousdatabases - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - database.oracle.com - resources: - - autonomousdatabases/status - verbs: - - patch - - update -- apiGroups: - - database.oracle.com + - "" resources: - - cdbs + - configmaps + - secrets verbs: - create - delete @@ -2236,23 +928,27 @@ rules: - update - watch - apiGroups: - - database.oracle.com - resources: - - cdbs/finalizers - verbs: - - update -- apiGroups: - - database.oracle.com + - "" resources: - - cdbs/status + - events + - nodes + - persistentvolumeclaims + - pods + - pods/exec + - pods/log + - services verbs: + - create + - delete - get + - list - patch - update + - watch - apiGroups: - - database.oracle.com + - '''''' resources: - - oraclerestdataservices + - statefulsets/finalizers verbs: - create - delete @@ -2262,23 +958,39 @@ rules: - update - watch - apiGroups: - - database.oracle.com + - apps resources: - - oraclerestdataservices/finalizers + - statefulsets verbs: + - create + - delete + - get + - list + - patch - update + - watch - apiGroups: - - database.oracle.com + - coordination.k8s.io resources: - - oraclerestdataservices/status + - leases verbs: + - create - get - - patch + - list - update - apiGroups: - - database.oracle.com + - "" resources: - - pdbs + - configmaps + - events + - namespaces + - nodes + - persistentvolumeclaims + - pods + - pods/exec + - pods/log + - secrets + - services verbs: - create - delete @@ -2287,22 +999,29 @@ rules: - patch - update - watch +- apiGroups: + - "" + resources: + - pods/exec + verbs: + - create - apiGroups: - database.oracle.com resources: - - pdbs/finalizers + - autonomousdatabases verbs: - create - delete - get + - list - patch - update + - watch - apiGroups: - database.oracle.com resources: - - pdbs/status + - autonomousdatabases/status verbs: - - get - patch - update - apiGroups: @@ -2480,8 +1199,8 @@ spec: - --enable-leader-election command: - /manager - image: controller:latest - imagePullPolicy: Never + image: container-registry.oracle.com/database/operator:0.1.0 + imagePullPolicy: Always name: manager ports: - containerPort: 9443 @@ -2536,109 +1255,6 @@ metadata: cert-manager.io/inject-ca-from: oracle-database-operator-system/oracle-database-operator-serving-cert name: oracle-database-operator-mutating-webhook-configuration webhooks: -- admissionReviewVersions: - - v1 - clientConfig: - service: - name: oracle-database-operator-webhook-service - namespace: oracle-database-operator-system - path: /mutate-database-oracle-com-v1alpha1-autonomousdatabase - failurePolicy: Fail - name: mautonomousdatabase.kb.io - rules: - - apiGroups: - - database.oracle.com - apiVersions: - - v1alpha1 - operations: - - CREATE - - UPDATE - resources: - - autonomousdatabases - sideEffects: None -- admissionReviewVersions: - - v1 - clientConfig: - service: - name: oracle-database-operator-webhook-service - namespace: oracle-database-operator-system - path: /mutate-database-oracle-com-v1alpha1-autonomousdatabasebackup - failurePolicy: Fail - name: mautonomousdatabasebackup.kb.io - rules: - - apiGroups: - - database.oracle.com - apiVersions: - - v1alpha1 - operations: - - CREATE - - UPDATE - resources: - - autonomousdatabasebackups - sideEffects: None -- admissionReviewVersions: - - v1 - - v1beta1 - clientConfig: - service: - name: oracle-database-operator-webhook-service - namespace: oracle-database-operator-system - path: /mutate-database-oracle-com-v1alpha1-cdb - failurePolicy: Fail - name: mcdb.kb.io - rules: - - apiGroups: - - database.oracle.com - apiVersions: - - v1alpha1 - operations: - - CREATE - - UPDATE - resources: - - cdbs - sideEffects: None -- admissionReviewVersions: - - v1 - - v1beta1 - clientConfig: - service: - name: oracle-database-operator-webhook-service - namespace: oracle-database-operator-system - path: /mutate-database-oracle-com-v1alpha1-oraclerestdataservice - failurePolicy: Fail - name: moraclerestdataservice.kb.io - rules: - - apiGroups: - - database.oracle.com - apiVersions: - - v1alpha1 - operations: - - CREATE - - UPDATE - resources: - - oraclerestdataservices - sideEffects: None -- admissionReviewVersions: - - v1 - - v1beta1 - clientConfig: - service: - name: oracle-database-operator-webhook-service - namespace: oracle-database-operator-system - path: /mutate-database-oracle-com-v1alpha1-pdb - failurePolicy: Fail - name: mpdb.kb.io - rules: - - apiGroups: - - database.oracle.com - apiVersions: - - v1alpha1 - operations: - - CREATE - - UPDATE - resources: - - pdbs - sideEffects: None - admissionReviewVersions: - v1 - v1beta1 @@ -2668,130 +1284,6 @@ metadata: cert-manager.io/inject-ca-from: oracle-database-operator-system/oracle-database-operator-serving-cert name: oracle-database-operator-validating-webhook-configuration webhooks: -- admissionReviewVersions: - - v1 - clientConfig: - service: - name: oracle-database-operator-webhook-service - namespace: oracle-database-operator-system - path: /validate-database-oracle-com-v1alpha1-autonomousdatabase - failurePolicy: Fail - name: vautonomousdatabase.kb.io - rules: - - apiGroups: - - database.oracle.com - apiVersions: - - v1alpha1 - operations: - - CREATE - - UPDATE - - DELETE - resources: - - autonomousdatabases - sideEffects: None -- admissionReviewVersions: - - v1 - clientConfig: - service: - name: oracle-database-operator-webhook-service - namespace: oracle-database-operator-system - path: /validate-database-oracle-com-v1alpha1-autonomousdatabasebackup - failurePolicy: Fail - name: vautonomousdatabasebackup.kb.io - rules: - - apiGroups: - - database.oracle.com - apiVersions: - - v1alpha1 - operations: - - CREATE - - UPDATE - resources: - - autonomousdatabasebackups - sideEffects: None -- admissionReviewVersions: - - v1 - clientConfig: - service: - name: oracle-database-operator-webhook-service - namespace: oracle-database-operator-system - path: /validate-database-oracle-com-v1alpha1-autonomousdatabaserestore - failurePolicy: Fail - name: vautonomousdatabaserestore.kb.io - rules: - - apiGroups: - - database.oracle.com - apiVersions: - - v1alpha1 - operations: - - CREATE - - UPDATE - resources: - - autonomousdatabaserestores - sideEffects: None -- admissionReviewVersions: - - v1 - - v1beta1 - clientConfig: - service: - name: oracle-database-operator-webhook-service - namespace: oracle-database-operator-system - path: /validate-database-oracle-com-v1alpha1-cdb - failurePolicy: Fail - name: vcdb.kb.io - rules: - - apiGroups: - - database.oracle.com - apiVersions: - - v1alpha1 - operations: - - CREATE - - UPDATE - resources: - - cdbs - sideEffects: None -- admissionReviewVersions: - - v1 - - v1beta1 - clientConfig: - service: - name: oracle-database-operator-webhook-service - namespace: oracle-database-operator-system - path: /validate-database-oracle-com-v1alpha1-oraclerestdataservice - failurePolicy: Fail - name: voraclerestdataservice.kb.io - rules: - - apiGroups: - - database.oracle.com - apiVersions: - - v1alpha1 - operations: - - CREATE - - UPDATE - resources: - - oraclerestdataservices - sideEffects: None -- admissionReviewVersions: - - v1 - - v1beta1 - clientConfig: - service: - name: oracle-database-operator-webhook-service - namespace: oracle-database-operator-system - path: /validate-database-oracle-com-v1alpha1-pdb - failurePolicy: Fail - name: vpdb.kb.io - rules: - - apiGroups: - - database.oracle.com - apiVersions: - - v1alpha1 - operations: - - CREATE - - UPDATE - resources: - - pdbs - sideEffects: None - admissionReviewVersions: - v1 - v1beta1 From 5917bc3c880a3fc0ea6eec19964e2ce88c5e3427 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Tue, 22 Mar 2022 17:03:46 -0400 Subject: [PATCH 195/628] fix isDedicated() --- apis/database/v1alpha1/autonomousdatabase_webhook.go | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/apis/database/v1alpha1/autonomousdatabase_webhook.go b/apis/database/v1alpha1/autonomousdatabase_webhook.go index 521b7988..3b5a39f8 100644 --- a/apis/database/v1alpha1/autonomousdatabase_webhook.go +++ b/apis/database/v1alpha1/autonomousdatabase_webhook.go @@ -238,5 +238,6 @@ func (r *AutonomousDatabase) ValidateDelete() error { // Returns true if AutonomousContainerDatabaseOCID has value. // We don't use Details.IsDedicated because the parameter might be null when it's a provision operation. func isDedicated(adb *AutonomousDatabase) bool { - return adb.Spec.Details.AutonomousContainerDatabase.OCIACD.OCID != nil + return adb.Spec.Details.AutonomousContainerDatabase.K8sACD.Name != nil || + adb.Spec.Details.AutonomousContainerDatabase.OCIACD.OCID != nil } From 05f593dc1cf71dc72f014e4161fe4dd8755773cc Mon Sep 17 00:00:00 2001 From: Susmita Samanta Date: Wed, 23 Mar 2022 13:06:40 +0530 Subject: [PATCH 196/628] debugging --- controllers/database/singleinstancedatabase_controller.go | 2 ++ 1 file changed, 2 insertions(+) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 5cac157a..7b86b518 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -314,10 +314,12 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab eventReason := "Spec Error" var eventMsgs []string + r.Log.Info("Check Edition=", m.Spec.Edition) // Pre-built db if m.Spec.Persistence.AccessMode == "" { return requeueN, nil } + r.Log.Info("PBDB not identified") // If Express Edition , Ensure Replicas=1 if m.Spec.Edition == "express" && m.Spec.Replicas > 1 { From 69c9abf6876230eeca358dd89c01b4aa21bbf94b Mon Sep 17 00:00:00 2001 From: Susmita Samanta Date: Wed, 23 Mar 2022 13:25:29 +0530 Subject: [PATCH 197/628] adding XE image support code --- oracle-database-operator.yaml | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/oracle-database-operator.yaml b/oracle-database-operator.yaml index 99918bfe..42f38296 100644 --- a/oracle-database-operator.yaml +++ b/oracle-database-operator.yaml @@ -1706,6 +1706,7 @@ spec: enum: - standard - enterprise + - express type: string flashBack: type: boolean @@ -2444,7 +2445,7 @@ metadata: name: oracle-database-operator-controller-manager namespace: oracle-database-operator-system spec: - replicas: 3 + replicas: 2 selector: matchLabels: control-plane: controller-manager @@ -2458,7 +2459,7 @@ spec: - --enable-leader-election command: - /manager - image: container-registry.oracle.com/database/operator:0.1.0 + image: us-phoenix-1.ocir.io/oracassandra/oraoperator:0.2.4 imagePullPolicy: Always name: manager ports: From 2b5781c9c34b323a70a3d6713cf64f40ac77413d Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Thu, 24 Mar 2022 13:43:45 +0530 Subject: [PATCH 198/628] Revert changes to oracle-database-operator.yaml --- oracle-database-operator.yaml | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/oracle-database-operator.yaml b/oracle-database-operator.yaml index 42f38296..99918bfe 100644 --- a/oracle-database-operator.yaml +++ b/oracle-database-operator.yaml @@ -1706,7 +1706,6 @@ spec: enum: - standard - enterprise - - express type: string flashBack: type: boolean @@ -2445,7 +2444,7 @@ metadata: name: oracle-database-operator-controller-manager namespace: oracle-database-operator-system spec: - replicas: 2 + replicas: 3 selector: matchLabels: control-plane: controller-manager @@ -2459,7 +2458,7 @@ spec: - --enable-leader-election command: - /manager - image: us-phoenix-1.ocir.io/oracassandra/oraoperator:0.2.4 + image: container-registry.oracle.com/database/operator:0.1.0 imagePullPolicy: Always name: manager ports: From 8f44749065c7720774f3551b57556373dce3062a Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Thu, 24 Mar 2022 21:51:54 +0530 Subject: [PATCH 199/628] Use single operator replica pod for testing --- .gitlab-ci.yml | 1 + config/manager/manager.yaml | 2 -- 2 files changed, 1 insertion(+), 2 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index e4d80bb9..b7e26f24 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -10,6 +10,7 @@ build-operator: - echo $newimage - docker rmi "$IMAGE" && docker system prune -f - make operator-yaml IMG=$newimage + - if [ "$CI_COMMIT_BRANCH" != "master" ]; then sed -i "s/replicas: 3/replicas: 1/g" ./$OP_YAML; fi - curl -s -n $ARTIFACTORY_REPO/$CI_COMMIT_BRANCH/$OP_YAML -T ./$OP_YAML only: variables: diff --git a/config/manager/manager.yaml b/config/manager/manager.yaml index 362aac61..18412546 100644 --- a/config/manager/manager.yaml +++ b/config/manager/manager.yaml @@ -26,8 +26,6 @@ spec: labels: control-plane: controller-manager spec: - imagePullSecrets: - - name: container-registry-secret containers: - command: - /manager From b8934761747c39b7184ecbd18a1fb4f113b52d49 Mon Sep 17 00:00:00 2001 From: Mohammed Yunus Date: Thu, 24 Mar 2022 16:24:19 +0000 Subject: [PATCH 200/628] Update .gitlab-ci.yml --- .gitlab-ci.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index b7e26f24..7f87dda7 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -10,7 +10,7 @@ build-operator: - echo $newimage - docker rmi "$IMAGE" && docker system prune -f - make operator-yaml IMG=$newimage - - if [ "$CI_COMMIT_BRANCH" != "master" ]; then sed -i "s/replicas: 3/replicas: 1/g" ./$OP_YAML; fi + - if [ "$CI_COMMIT_BRANCH" != "master" ]; then sed -i 's/replicas: 3/replicas: 1/g' ./$OP_YAML; fi - curl -s -n $ARTIFACTORY_REPO/$CI_COMMIT_BRANCH/$OP_YAML -T ./$OP_YAML only: variables: From 6c9a7dab253b304ecb0bf92ca75be02341d5fe41 Mon Sep 17 00:00:00 2001 From: Mohammed Yunus Date: Thu, 24 Mar 2022 16:26:32 +0000 Subject: [PATCH 201/628] Update .gitlab-ci.yml --- .gitlab-ci.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 7f87dda7..b779c338 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -10,7 +10,7 @@ build-operator: - echo $newimage - docker rmi "$IMAGE" && docker system prune -f - make operator-yaml IMG=$newimage - - if [ "$CI_COMMIT_BRANCH" != "master" ]; then sed -i 's/replicas: 3/replicas: 1/g' ./$OP_YAML; fi + - if [ "$CI_COMMIT_BRANCH" != "master" ]; then sed -i 's/replicas\: 3/replicas\: 1/g' ./$OP_YAML; fi - curl -s -n $ARTIFACTORY_REPO/$CI_COMMIT_BRANCH/$OP_YAML -T ./$OP_YAML only: variables: From d1186273e324fa90036003811f6f18a8238e0755 Mon Sep 17 00:00:00 2001 From: Mohammed Yunus Date: Thu, 24 Mar 2022 16:31:55 +0000 Subject: [PATCH 202/628] Update .gitlab-ci.yml --- .gitlab-ci.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index b779c338..1c5aaf93 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -10,7 +10,7 @@ build-operator: - echo $newimage - docker rmi "$IMAGE" && docker system prune -f - make operator-yaml IMG=$newimage - - if [ "$CI_COMMIT_BRANCH" != "master" ]; then sed -i 's/replicas\: 3/replicas\: 1/g' ./$OP_YAML; fi + - if [ "$CI_COMMIT_BRANCH" != "master" ]; then sed -i "s/\(replicas.\) 3/\1 1/g" ./$OP_YAML; fi - curl -s -n $ARTIFACTORY_REPO/$CI_COMMIT_BRANCH/$OP_YAML -T ./$OP_YAML only: variables: From e73c68408b349a5d19ca1153dd689a7b5df21135 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Thu, 24 Mar 2022 18:26:06 -0400 Subject: [PATCH 203/628] update acd/adb controllers --- .../v1alpha1/adbfamily_common_utils.go | 2 +- .../autonomouscontainerdatabase_types.go | 18 ++- .../v1alpha1/autonomousdatabase_types.go | 2 +- .../v1alpha1/autonomousdatabase_webhook.go | 2 +- .../autonomousdatabasebackup_types.go | 97 +++++++---- .../autonomousdatabasebackup_webhook.go | 20 +-- .../autonomousdatabaserestore_types.go | 18 ++- .../v1alpha1/zz_generated.deepcopy.go | 5 + commons/k8s/create.go | 2 +- commons/oci/containerdatabase.go | 8 + commons/oci/database.go | 5 +- commons/oci/workrequest.go | 16 +- ...acle.com_autonomouscontainerdatabases.yaml | 6 + ....oracle.com_autonomousdatabasebackups.yaml | 5 - ...oracle.com_autonomousdatabaserestores.yaml | 8 +- .../acd/autonomouscontainerdatabase_bind.yaml | 11 ++ .../autonomouscontainerdatabase_create.yaml | 3 +- ...uscontainerdatabase_restart_terminate.yaml | 13 ++ .../acd/autonomouscontainerdatabase_sync.yaml | 12 ++ .../adb/autonomousdatabase_create.yaml | 11 +- ...onomousdatabase_update_admin_password.yaml | 11 +- .../adb/autonomousdatabase_wallet.yaml | 11 +- .../autonomouscontainerdatabase_controller.go | 120 ++++++++++---- .../database/autonomousdatabase_controller.go | 153 ++++++++++-------- .../autonomousdatabasebackup_controller.go | 80 +++++---- .../autonomousdatabaserestore_controller.go | 41 ++--- 26 files changed, 438 insertions(+), 242 deletions(-) create mode 100644 config/samples/acd/autonomouscontainerdatabase_bind.yaml create mode 100644 config/samples/acd/autonomouscontainerdatabase_restart_terminate.yaml create mode 100644 config/samples/acd/autonomouscontainerdatabase_sync.yaml diff --git a/apis/database/v1alpha1/adbfamily_common_utils.go b/apis/database/v1alpha1/adbfamily_common_utils.go index 65f7f64f..c3d505c7 100644 --- a/apis/database/v1alpha1/adbfamily_common_utils.go +++ b/apis/database/v1alpha1/adbfamily_common_utils.go @@ -187,7 +187,7 @@ func hasChanged(lastField reflect.Value, curField reflect.Value) bool { // Follow the format of the display time const displayFormat = "2006-01-02 15:04:05 MST" -func formatSDKTime(sdkTime *common.SDKTime) string { +func FormatSDKTime(sdkTime *common.SDKTime) string { if sdkTime == nil { return "" } diff --git a/apis/database/v1alpha1/autonomouscontainerdatabase_types.go b/apis/database/v1alpha1/autonomouscontainerdatabase_types.go index 0d9213b9..9d4e456d 100644 --- a/apis/database/v1alpha1/autonomouscontainerdatabase_types.go +++ b/apis/database/v1alpha1/autonomouscontainerdatabase_types.go @@ -52,6 +52,15 @@ import ( // name of our custom finalizer const ACDFinalizer = "database.oracle.com/acd-finalizer" +type acdActionEnum string + +const ( + AcdActionBlank acdActionEnum = "" + AcdActionSync acdActionEnum = "SYNC" + AcdActionRestart acdActionEnum = "RESTART" + AcdActionTerminate acdActionEnum = "TERMINATE" +) + // AutonomousContainerDatabaseSpec defines the desired state of AutonomousContainerDatabase type AutonomousContainerDatabaseSpec struct { // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster @@ -61,8 +70,10 @@ type AutonomousContainerDatabaseSpec struct { DisplayName *string `json:"displayName,omitempty"` AutonomousExadataVMClusterOCID *string `json:"autonomousExadataVMClusterOCID,omitempty"` // +kubebuilder:validation:Enum:="RELEASE_UPDATES";"RELEASE_UPDATE_REVISIONS" - PatchModel database.AutonomousContainerDatabasePatchModelEnum `json:"patchModel,omitempty"` - FreeformTags map[string]string `json:"freeformTags,omitempty"` + PatchModel database.AutonomousContainerDatabasePatchModelEnum `json:"patchModel,omitempty"` + // +kubebuilder:validation:Enum:="SYNC";"RESTART";"TERMINATE" + Action acdActionEnum `json:"action,omitempty"` + FreeformTags map[string]string `json:"freeformTags,omitempty"` OCIConfig OCIConfigSpec `json:"ociConfig,omitempty"` // +kubebuilder:default:=false @@ -128,11 +139,12 @@ func (acd *AutonomousContainerDatabase) GetLastSuccessfulSpec() (*AutonomousCont // UpdateStatusFromOCIACD updates only the status from database.AutonomousDatabase object func (acd *AutonomousContainerDatabase) UpdateStatusFromOCIACD(ociObj database.AutonomousContainerDatabase) { acd.Status.LifecycleState = ociObj.LifecycleState - acd.Status.TimeCreated = formatSDKTime(ociObj.TimeCreated) + acd.Status.TimeCreated = FormatSDKTime(ociObj.TimeCreated) } func (acd *AutonomousContainerDatabase) UpdateFromOCIACD(ociObj database.AutonomousContainerDatabase) { // Spec + acd.Spec.Action = AcdActionBlank acd.Spec.AutonomousContainerDatabaseOCID = ociObj.Id acd.Spec.CompartmentOCID = ociObj.CompartmentId acd.Spec.DisplayName = ociObj.DisplayName diff --git a/apis/database/v1alpha1/autonomousdatabase_types.go b/apis/database/v1alpha1/autonomousdatabase_types.go index 37926c36..d2c9609c 100644 --- a/apis/database/v1alpha1/autonomousdatabase_types.go +++ b/apis/database/v1alpha1/autonomousdatabase_types.go @@ -233,7 +233,7 @@ func (adb *AutonomousDatabase) GetLastSuccessfulSpec() (*AutonomousDatabaseSpec, // UpdateStatusFromOCIADB updates only the status from database.AutonomousDatabase object func (adb *AutonomousDatabase) UpdateStatusFromOCIADB(ociObj database.AutonomousDatabase) { adb.Status.LifecycleState = ociObj.LifecycleState - adb.Status.TimeCreated = formatSDKTime(ociObj.TimeCreated) + adb.Status.TimeCreated = FormatSDKTime(ociObj.TimeCreated) if *ociObj.IsDedicated { conns := make([]ConnectionStringSpec, len(ociObj.ConnectionStrings.AllConnectionStrings)) diff --git a/apis/database/v1alpha1/autonomousdatabase_webhook.go b/apis/database/v1alpha1/autonomousdatabase_webhook.go index 3b5a39f8..e7a57ea1 100644 --- a/apis/database/v1alpha1/autonomousdatabase_webhook.go +++ b/apis/database/v1alpha1/autonomousdatabase_webhook.go @@ -239,5 +239,5 @@ func (r *AutonomousDatabase) ValidateDelete() error { // We don't use Details.IsDedicated because the parameter might be null when it's a provision operation. func isDedicated(adb *AutonomousDatabase) bool { return adb.Spec.Details.AutonomousContainerDatabase.K8sACD.Name != nil || - adb.Spec.Details.AutonomousContainerDatabase.OCIACD.OCID != nil + adb.Spec.Details.AutonomousContainerDatabase.OCIACD.OCID != nil } diff --git a/apis/database/v1alpha1/autonomousdatabasebackup_types.go b/apis/database/v1alpha1/autonomousdatabasebackup_types.go index 3d6defd8..3ade554d 100644 --- a/apis/database/v1alpha1/autonomousdatabasebackup_types.go +++ b/apis/database/v1alpha1/autonomousdatabasebackup_types.go @@ -52,38 +52,49 @@ import ( type AutonomousDatabaseBackupSpec struct { // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster // Important: Run "make" to regenerate code after modifying this file - Target TargetSpec `json:"target"` - DisplayName string `json:"displayName"` + Target TargetSpec `json:"target,omitempty"` + DisplayName *string `json:"displayName,omitempty"` AutonomousDatabaseBackupOCID *string `json:"autonomousDatabaseBackupOCID,omitempty"` OCIConfig OCIConfigSpec `json:"ociConfig,omitempty"` } +type BackupStateEnum string + +const ( + BackupStateError BackupStateEnum = "ERROR" + BackupStateCreating BackupStateEnum = "CREATING" + BackupStateActive BackupStateEnum = "ACTIVE" + BackupStateDeleting BackupStateEnum = "DELETING" + BackupStateDeleted BackupStateEnum = "DELETED" + BackupStateFailed BackupStateEnum = "FAILED" +) + // AutonomousDatabaseBackupStatus defines the observed state of AutonomousDatabaseBackup type AutonomousDatabaseBackupStatus struct { // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster // Important: Run "make" to regenerate code after modifying this file - AutonomousDatabaseBackupOCID string `json:"autonomousDatabaseBackupOCID,omitempty"` - DisplayName string `json:"displayName"` - LifecycleState database.AutonomousDatabaseBackupLifecycleStateEnum `json:"lifecycleState"` - Type database.AutonomousDatabaseBackupTypeEnum `json:"type"` - IsAutomatic bool `json:"isAutomatic"` - TimeStarted string `json:"timeStarted,omitempty"` - TimeEnded string `json:"timeEnded,omitempty"` - AutonomousDatabaseOCID string `json:"autonomousDatabaseOCID"` - CompartmentOCID string `json:"compartmentOCID"` - DBName string `json:"dbName"` - DBDisplayName string `json:"dbDisplayName"` + AutonomousDatabaseBackupOCID string `json:"autonomousDatabaseBackupOCID,omitempty"` + DisplayName string `json:"displayName"` + LifecycleState BackupStateEnum `json:"lifecycleState"` + Type database.AutonomousDatabaseBackupTypeEnum `json:"type"` + IsAutomatic bool `json:"isAutomatic"` + TimeStarted string `json:"timeStarted,omitempty"` + TimeEnded string `json:"timeEnded,omitempty"` + AutonomousDatabaseOCID string `json:"autonomousDatabaseOCID"` + CompartmentOCID string `json:"compartmentOCID"` + DBName string `json:"dbName"` + DBDisplayName string `json:"dbDisplayName"` } //+kubebuilder:object:root=true //+kubebuilder:subresource:status -// +kubebuilder:resource:shortName="adbbu";"adbbus" -// +kubebuilder:printcolumn:JSONPath=".status.lifecycleState",name="State",type=string -// +kubebuilder:printcolumn:JSONPath=".status.dbDisplayName",name="DB DisplayName",type=string -// +kubebuilder:printcolumn:JSONPath=".status.type",name="Type",type=string -// +kubebuilder:printcolumn:JSONPath=".status.timeStarted",name="Started",type=string -// +kubebuilder:printcolumn:JSONPath=".status.timeEnded",name="Ended",type=string +//+kubebuilder:resource:shortName="adbbu";"adbbus" +//+kubebuilder:printcolumn:JSONPath=".status.lifecycleState",name="State",type=string +//+kubebuilder:printcolumn:JSONPath=".status.dbDisplayName",name="DB DisplayName",type=string +//+kubebuilder:printcolumn:JSONPath=".status.type",name="Type",type=string +//+kubebuilder:printcolumn:JSONPath=".status.timeStarted",name="Started",type=string +//+kubebuilder:printcolumn:JSONPath=".status.timeEnded",name="Ended",type=string // AutonomousDatabaseBackup is the Schema for the autonomousdatabasebackups API type AutonomousDatabaseBackup struct { @@ -107,24 +118,44 @@ func init() { SchemeBuilder.Register(&AutonomousDatabaseBackup{}, &AutonomousDatabaseBackupList{}) } -func (backup *AutonomousDatabaseBackup) UpdateStatusFromOCIBackup(ociBackup database.AutonomousDatabaseBackup, ociADB database.AutonomousDatabase) { - backup.Status.AutonomousDatabaseBackupOCID = *ociBackup.Id - backup.Status.DisplayName = *ociBackup.DisplayName +func (b *AutonomousDatabaseBackup) UpdateStatusFromOCIBackup(ociBackup database.AutonomousDatabaseBackup, ociADB database.AutonomousDatabase) { + b.Status.AutonomousDatabaseBackupOCID = *ociBackup.Id + b.Status.DisplayName = *ociBackup.DisplayName - backup.Status.AutonomousDatabaseOCID = *ociBackup.AutonomousDatabaseId - backup.Status.CompartmentOCID = *ociBackup.CompartmentId - backup.Status.Type = ociBackup.Type - backup.Status.IsAutomatic = *ociBackup.IsAutomatic - backup.Status.LifecycleState = ociBackup.LifecycleState + b.Status.AutonomousDatabaseOCID = *ociBackup.AutonomousDatabaseId + b.Status.CompartmentOCID = *ociBackup.CompartmentId + b.Status.Type = ociBackup.Type + b.Status.IsAutomatic = *ociBackup.IsAutomatic - backup.Status.TimeStarted = formatSDKTime(ociBackup.TimeStarted) - backup.Status.TimeEnded = formatSDKTime(ociBackup.TimeEnded) + b.Status.LifecycleState = b.ConvertBackupStatus(ociBackup.LifecycleState) - backup.Status.DBDisplayName = *ociADB.DisplayName - backup.Status.DBName = *ociADB.DbName + b.Status.TimeStarted = FormatSDKTime(ociBackup.TimeStarted) + b.Status.TimeEnded = FormatSDKTime(ociBackup.TimeEnded) + + b.Status.DBDisplayName = *ociADB.DisplayName + b.Status.DBName = *ociADB.DbName } // GetTimeEnded returns the status.timeEnded in SDKTime format -func (backup *AutonomousDatabaseBackup) GetTimeEnded() (*common.SDKTime, error) { - return parseDisplayTime(backup.Status.TimeEnded) +func (b *AutonomousDatabaseBackup) GetTimeEnded() (*common.SDKTime, error) { + return parseDisplayTime(b.Status.TimeEnded) +} + +func (b *AutonomousDatabaseBackup) ConvertBackupStatus(state database.AutonomousDatabaseBackupLifecycleStateEnum) BackupStateEnum { + switch state { + case database.AutonomousDatabaseBackupLifecycleStateCreating: + return BackupStateCreating + case database.AutonomousDatabaseBackupLifecycleStateActive: + return BackupStateActive + + case database.AutonomousDatabaseBackupLifecycleStateDeleting: + return BackupStateDeleting + + case database.AutonomousDatabaseBackupLifecycleStateDeleted: + return BackupStateDeleted + case database.AutonomousDatabaseBackupLifecycleStateFailed: + return BackupStateFailed + default: + return "UNKNOWN" + } } diff --git a/apis/database/v1alpha1/autonomousdatabasebackup_webhook.go b/apis/database/v1alpha1/autonomousdatabasebackup_webhook.go index 880c5beb..2407ea27 100644 --- a/apis/database/v1alpha1/autonomousdatabasebackup_webhook.go +++ b/apis/database/v1alpha1/autonomousdatabasebackup_webhook.go @@ -101,26 +101,26 @@ func (r *AutonomousDatabaseBackup) ValidateUpdate(old runtime.Object) error { var allErrs field.ErrorList oldBackup := old.(*AutonomousDatabaseBackup) - if oldBackup.Spec.AutonomousDatabaseBackupOCID != nil && - oldBackup.Spec.AutonomousDatabaseBackupOCID != r.Spec.AutonomousDatabaseBackupOCID { + if oldBackup.Spec.AutonomousDatabaseBackupOCID != nil && r.Spec.AutonomousDatabaseBackupOCID != nil && + *oldBackup.Spec.AutonomousDatabaseBackupOCID != *r.Spec.AutonomousDatabaseBackupOCID { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("autonomousDatabaseBackupOCID"), "cannot assign a new autonomousDatabaseBackupOCID to this backup")) } - if r.Spec.Target.K8sADB.Name != nil && - oldBackup.Spec.Target.K8sADB.Name != r.Spec.Target.K8sADB.Name { + if oldBackup.Spec.Target.K8sADB.Name != nil && r.Spec.Target.K8sADB.Name != nil && + *oldBackup.Spec.Target.K8sADB.Name != *r.Spec.Target.K8sADB.Name { allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec").Child("target").Child("k8sADB").Child("name"), "cannot change target.name")) + field.Forbidden(field.NewPath("spec").Child("target").Child("k8sADB").Child("name"), "cannot assign a new name to the target")) } - if r.Spec.Target.OCIADB.OCID != nil && - oldBackup.Spec.Target.OCIADB.OCID != r.Spec.Target.OCIADB.OCID { + if oldBackup.Spec.Target.OCIADB.OCID != nil && r.Spec.Target.OCIADB.OCID != nil && + *oldBackup.Spec.Target.OCIADB.OCID != *r.Spec.Target.OCIADB.OCID { allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec").Child("target").Child("ociADB").Child("ocid"), "cannot change target.ocid")) + field.Forbidden(field.NewPath("spec").Child("target").Child("ociADB").Child("ocid"), "cannot assign a new ocid to the target")) } - if r.Spec.DisplayName != "" && - oldBackup.Spec.DisplayName != r.Spec.DisplayName { + if oldBackup.Spec.DisplayName != nil && r.Spec.DisplayName != nil && + *oldBackup.Spec.DisplayName != *r.Spec.DisplayName { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("displayName"), "cannot assign a new displayName to this backup")) } diff --git a/apis/database/v1alpha1/autonomousdatabaserestore_types.go b/apis/database/v1alpha1/autonomousdatabaserestore_types.go index 1e893a97..cd9637f6 100644 --- a/apis/database/v1alpha1/autonomousdatabaserestore_types.go +++ b/apis/database/v1alpha1/autonomousdatabaserestore_types.go @@ -75,6 +75,7 @@ type AutonomousDatabaseRestoreSpec struct { type restoreStatusEnum string const ( + RestoreStatusError restoreStatusEnum = "ERROR" RestoreStatusInProgress restoreStatusEnum = "IN_PROGRESS" RestoreStatusFailed restoreStatusEnum = "FAILED" RestoreStatusSucceeded restoreStatusEnum = "SUCCEEDED" @@ -85,6 +86,9 @@ type AutonomousDatabaseRestoreStatus struct { // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster // Important: Run "make" to regenerate code after modifying this file DisplayName string `json:"displayName"` + TimeAccepted string `json:"timeAccepted,omitempty"` + TimeStarted string `json:"timeStarted,omitempty"` + TimeEnded string `json:"timeEnded,omitempty"` DbName string `json:"dbName"` AutonomousDatabaseOCID string `json:"autonomousDatabaseOCID"` Status restoreStatusEnum `json:"status"` @@ -94,7 +98,7 @@ type AutonomousDatabaseRestoreStatus struct { //+kubebuilder:subresource:status //+kubebuilder:resource:shortName="adbr";"adbrs" // +kubebuilder:printcolumn:JSONPath=".status.status",name="Status",type=string -// +kubebuilder:printcolumn:JSONPath=".status.displayName",name="DisplayName",type=string +// +kubebuilder:printcolumn:JSONPath=".status.displayName",name="DbDisplayName",type=string // +kubebuilder:printcolumn:JSONPath=".status.dbName",name="DbName",type=string // AutonomousDatabaseRestore is the Schema for the autonomousdatabaserestores API @@ -127,23 +131,23 @@ func (r *AutonomousDatabaseRestore) GetPIT() (*common.SDKTime, error) { return parseDisplayTime(*r.Spec.Source.PointInTime.Timestamp) } -func (r *AutonomousDatabaseRestore) ConvertWorkRequestStatus(s workrequests.WorkRequestStatusEnum) (restoreStatusEnum, error) { +func (r *AutonomousDatabaseRestore) ConvertWorkRequestStatus(s workrequests.WorkRequestStatusEnum) restoreStatusEnum { switch s { case workrequests.WorkRequestStatusAccepted: fallthrough case workrequests.WorkRequestStatusInProgress: - return RestoreStatusInProgress, nil + return RestoreStatusInProgress case workrequests.WorkRequestStatusSucceeded: - return RestoreStatusSucceeded, nil + return RestoreStatusSucceeded case workrequests.WorkRequestStatusCanceling: fallthrough case workrequests.WorkRequestStatusCanceled: fallthrough case workrequests.WorkRequestStatusFailed: - return RestoreStatusFailed, nil + return RestoreStatusFailed + default: + return "UNKNOWN" } - - return "", errors.New("unable to convert the status: " + string(s)) } diff --git a/apis/database/v1alpha1/zz_generated.deepcopy.go b/apis/database/v1alpha1/zz_generated.deepcopy.go index f3c1b7b2..7be52b26 100644 --- a/apis/database/v1alpha1/zz_generated.deepcopy.go +++ b/apis/database/v1alpha1/zz_generated.deepcopy.go @@ -278,6 +278,11 @@ func (in *AutonomousDatabaseBackupList) DeepCopyObject() runtime.Object { func (in *AutonomousDatabaseBackupSpec) DeepCopyInto(out *AutonomousDatabaseBackupSpec) { *out = *in in.Target.DeepCopyInto(&out.Target) + if in.DisplayName != nil { + in, out := &in.DisplayName, &out.DisplayName + *out = new(string) + **out = **in + } if in.AutonomousDatabaseBackupOCID != nil { in, out := &in.AutonomousDatabaseBackupOCID, &out.AutonomousDatabaseBackupOCID *out = new(string) diff --git a/commons/k8s/create.go b/commons/k8s/create.go index 74afcaa7..fa53704c 100644 --- a/commons/k8s/create.go +++ b/commons/k8s/create.go @@ -92,7 +92,7 @@ func CreateAutonomousBackup(kubeClient client.Client, Name: common.String(ownerADB.Name), }, }, - DisplayName: *backupSummary.DisplayName, + DisplayName: backupSummary.DisplayName, AutonomousDatabaseBackupOCID: backupSummary.Id, OCIConfig: dbv1alpha1.OCIConfigSpec{ ConfigMapName: ownerADB.Spec.OCIConfig.ConfigMapName, diff --git a/commons/oci/containerdatabase.go b/commons/oci/containerdatabase.go index 30e57919..359f8a4b 100644 --- a/commons/oci/containerdatabase.go +++ b/commons/oci/containerdatabase.go @@ -84,6 +84,14 @@ func (d *databaseService) UpdateAutonomousContainerDatabase(acd *dbv1alpha1.Auto return d.dbClient.UpdateAutonomousContainerDatabase(context.TODO(), updateAutonomousContainerDatabaseRequest) } +func (d *databaseService) RestartAutonomousContainerDatabase(acdOCID string) (database.RestartAutonomousContainerDatabaseResponse, error) { + restartRequest := database.RestartAutonomousContainerDatabaseRequest{ + AutonomousContainerDatabaseId: common.String(acdOCID), + } + + return d.dbClient.RestartAutonomousContainerDatabase(context.TODO(), restartRequest) +} + func (d *databaseService) TerminateAutonomousContainerDatabase(acdOCID string) (database.TerminateAutonomousContainerDatabaseResponse, error) { terminateRequest := database.TerminateAutonomousContainerDatabaseRequest{ AutonomousContainerDatabaseId: common.String(acdOCID), diff --git a/commons/oci/database.go b/commons/oci/database.go index c4477984..142e634f 100644 --- a/commons/oci/database.go +++ b/commons/oci/database.go @@ -74,6 +74,7 @@ type DatabaseService interface { CreateAutonomousContainerDatabase(acb *dbv1alpha1.AutonomousContainerDatabase) (database.CreateAutonomousContainerDatabaseResponse, error) GetAutonomousContainerDatabase(acdOCID string) (database.GetAutonomousContainerDatabaseResponse, error) UpdateAutonomousContainerDatabase(acd *dbv1alpha1.AutonomousContainerDatabase) (database.UpdateAutonomousContainerDatabaseResponse, error) + RestartAutonomousContainerDatabase(acdOCID string) (database.RestartAutonomousContainerDatabaseResponse, error) TerminateAutonomousContainerDatabase(acdOCID string) (database.TerminateAutonomousContainerDatabaseResponse, error) } @@ -410,8 +411,8 @@ func (d *databaseService) CreateAutonomousDatabaseBackup(adbBackup *dbv1alpha1.A // Use the spec.displayName as the displayName of the backup if is provided, // otherwise use the resource name as the displayName. - if adbBackup.Spec.DisplayName != "" { - createBackupRequest.DisplayName = common.String(adbBackup.Spec.DisplayName) + if adbBackup.Spec.DisplayName != nil { + createBackupRequest.DisplayName = adbBackup.Spec.DisplayName } else { createBackupRequest.DisplayName = common.String(adbBackup.GetName()) } diff --git a/commons/oci/workrequest.go b/commons/oci/workrequest.go index 3edf5352..cf05e506 100644 --- a/commons/oci/workrequest.go +++ b/commons/oci/workrequest.go @@ -51,8 +51,8 @@ import ( ) type WorkRequestService interface { - Get(opcWorkRequestID string) (workrequests.WorkRequestStatusEnum, error) - Wait(opcWorkRequestID string) (workrequests.WorkRequestStatusEnum, error) + Get(opcWorkRequestID string) (workrequests.GetWorkRequestResponse, error) + Wait(opcWorkRequestID string) (workrequests.GetWorkRequestResponse, error) } type workRequestService struct { @@ -110,7 +110,7 @@ func (w workRequestService) getRetryPolicy() common.RetryPolicy { return common.NewRetryPolicy(attempts, shouldRetry, nextDuration) } -func (w *workRequestService) Wait(opcWorkRequestID string) (workrequests.WorkRequestStatusEnum, error) { +func (w *workRequestService) Wait(opcWorkRequestID string) (workrequests.GetWorkRequestResponse, error) { w.logger.Info("Waiting for the work request to finish. opcWorkRequestID = " + opcWorkRequestID) // retries until the work status is SUCCEEDED, FAILED or CANCELED @@ -125,21 +125,21 @@ func (w *workRequestService) Wait(opcWorkRequestID string) (workrequests.WorkReq resp, err := w.workClient.GetWorkRequest(context.TODO(), workRequest) if err != nil { - return resp.Status, err + return resp, err } - return resp.Status, nil + return resp, nil } -func (w *workRequestService) Get(opcWorkRequestID string) (workrequests.WorkRequestStatusEnum, error) { +func (w *workRequestService) Get(opcWorkRequestID string) (workrequests.GetWorkRequestResponse, error) { workRequest := workrequests.GetWorkRequestRequest{ WorkRequestId: common.String(opcWorkRequestID), } resp, err := w.workClient.GetWorkRequest(context.TODO(), workRequest) if err != nil { - return resp.Status, err + return resp, err } - return resp.Status, nil + return resp, nil } diff --git a/config/crd/bases/database.oracle.com_autonomouscontainerdatabases.yaml b/config/crd/bases/database.oracle.com_autonomouscontainerdatabases.yaml index 7ea17e10..bac3a28c 100644 --- a/config/crd/bases/database.oracle.com_autonomouscontainerdatabases.yaml +++ b/config/crd/bases/database.oracle.com_autonomouscontainerdatabases.yaml @@ -51,6 +51,12 @@ spec: description: AutonomousContainerDatabaseSpec defines the desired state of AutonomousContainerDatabase properties: + action: + enum: + - SYNC + - RESTART + - TERMINATE + type: string autonomousContainerDatabaseOCID: description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster Important: Run "make" to regenerate code after modifying this file' diff --git a/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml b/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml index 1c34f7fa..cd9e3729 100644 --- a/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml +++ b/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml @@ -85,9 +85,6 @@ spec: type: string type: object type: object - required: - - displayName - - target type: object status: description: AutonomousDatabaseBackupStatus defines the observed state @@ -111,8 +108,6 @@ spec: isAutomatic: type: boolean lifecycleState: - description: 'AutonomousDatabaseBackupLifecycleStateEnum Enum with - underlying type: string' type: string timeEnded: type: string diff --git a/config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml b/config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml index a6ecf6d1..bf1b2d51 100644 --- a/config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml +++ b/config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml @@ -24,7 +24,7 @@ spec: name: Status type: string - jsonPath: .status.displayName - name: DisplayName + name: DbDisplayName type: string - jsonPath: .status.dbName name: DbName @@ -112,6 +112,12 @@ spec: type: string status: type: string + timeAccepted: + type: string + timeEnded: + type: string + timeStarted: + type: string required: - autonomousDatabaseOCID - dbName diff --git a/config/samples/acd/autonomouscontainerdatabase_bind.yaml b/config/samples/acd/autonomouscontainerdatabase_bind.yaml new file mode 100644 index 00000000..75c88089 --- /dev/null +++ b/config/samples/acd/autonomouscontainerdatabase_bind.yaml @@ -0,0 +1,11 @@ +apiVersion: database.oracle.com/v1alpha1 +kind: AutonomousContainerDatabase +metadata: + name: autonomouscontainerdatabase-sample +spec: + autonomousContainerDatabaseOCID: ocid1.autonomouscontainerdatabase... + + # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. + ociConfig: + configMapName: oci-cred-dedicated + secretName: oci-privatekey-dedicated diff --git a/config/samples/acd/autonomouscontainerdatabase_create.yaml b/config/samples/acd/autonomouscontainerdatabase_create.yaml index 8ee34a04..93b8edcd 100644 --- a/config/samples/acd/autonomouscontainerdatabase_create.yaml +++ b/config/samples/acd/autonomouscontainerdatabase_create.yaml @@ -7,7 +7,8 @@ spec: compartmentOCID: ocid1.compartment... OR ocid1.tenancy... displayName: newACD autonomousExadataVMClusterOCID: ocid1.autonomousexainfrastructure... - patchModel: RELEASE_UPDATES # optional + # # An optional field for Database Patch model preference. Should be either RELEASE_UPDATES or RELEASE_UPDATE_REVISIONS + # patchModel: RELEASE_UPDATES freeformTags: key1: val1 diff --git a/config/samples/acd/autonomouscontainerdatabase_restart_terminate.yaml b/config/samples/acd/autonomouscontainerdatabase_restart_terminate.yaml new file mode 100644 index 00000000..73e241a9 --- /dev/null +++ b/config/samples/acd/autonomouscontainerdatabase_restart_terminate.yaml @@ -0,0 +1,13 @@ +apiVersion: database.oracle.com/v1alpha1 +kind: AutonomousContainerDatabase +metadata: + name: autonomouscontainerdatabase-sample +spec: + autonomousContainerDatabaseOCID: ocid1.autonomouscontainerdatabase... + # Change the action to "TERMINATE" to terminate the database + action: RESTART + + # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. + ociConfig: + configMapName: oci-cred-dedicated + secretName: oci-privatekey-dedicated diff --git a/config/samples/acd/autonomouscontainerdatabase_sync.yaml b/config/samples/acd/autonomouscontainerdatabase_sync.yaml new file mode 100644 index 00000000..290da77b --- /dev/null +++ b/config/samples/acd/autonomouscontainerdatabase_sync.yaml @@ -0,0 +1,12 @@ +apiVersion: database.oracle.com/v1alpha1 +kind: AutonomousContainerDatabase +metadata: + name: autonomouscontainerdatabase-sample +spec: + autonomousContainerDatabaseOCID: ocid1.autonomouscontainerdatabase... + action: SYNC + + # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. + ociConfig: + configMapName: oci-cred-dedicated + secretName: oci-privatekey-dedicated diff --git a/config/samples/adb/autonomousdatabase_create.yaml b/config/samples/adb/autonomousdatabase_create.yaml index 2a3ffe82..9db481ad 100644 --- a/config/samples/adb/autonomousdatabase_create.yaml +++ b/config/samples/adb/autonomousdatabase_create.yaml @@ -15,10 +15,13 @@ spec: displayName: NewADB cpuCoreCount: 1 adminPassword: - # The Name of the K8s secret where you want to hold the password of the ADMIN account. Comment out k8sSecretName and uncomment ociSecretOCID if you pass the admin password using OCI Secret. - k8sSecretName: admin-password - # The OCID of the OCI Secret that holds the password of the ADMIN account. It should start with ocid1.vaultsecret... . - # ociSecretOCID: ocid1.vaultsecret... + # Comment out k8sSecret and uncomment ociSecret if you pass the admin password using OCI Secret. + k8sSecret: + # The Name of the K8s secret where you want to hold the password of the ADMIN account. + name: admin-password + # ociSecret: + # # The OCID of the OCI Secret that holds the password of the ADMIN account. It should start with ocid1.vaultsecret... . + # ocid: ocid1.vaultsecret... dataStorageSizeInTBs: 1 # networkAccess: diff --git a/config/samples/adb/autonomousdatabase_update_admin_password.yaml b/config/samples/adb/autonomousdatabase_update_admin_password.yaml index 47ad1d65..70b952b5 100644 --- a/config/samples/adb/autonomousdatabase_update_admin_password.yaml +++ b/config/samples/adb/autonomousdatabase_update_admin_password.yaml @@ -10,10 +10,13 @@ spec: details: autonomousDatabaseOCID: ocid1.autonomousdatabase... adminPassword: - # The Name of the secret where you want to hold the password of the ADMIN account. Comment out k8sSecretName and uncomment ociSecretOCID if you pass the admin password using OCI Secret. - k8sSecretName: new-admin-password - # The OCID of the OCI Secret that holds the password of the ADMIN account. It should start with ocid1.vaultsecret... . - # ociSecretOCID: ocid1.vaultsecret... + # Comment out k8sSecret and uncomment ociSecret if you pass the admin password using OCI Secret. + k8sSecret: + # The Name of the K8s secret where you want to hold the password of the ADMIN account. + name: new-admin-password + # ociSecret: + # # The OCID of the OCI Secret that holds the password of the ADMIN account. It should start with ocid1.vaultsecret... . + # ocid: ocid1.vaultsecret... # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. ociConfig: configMapName: oci-cred diff --git a/config/samples/adb/autonomousdatabase_wallet.yaml b/config/samples/adb/autonomousdatabase_wallet.yaml index 34953403..64925483 100644 --- a/config/samples/adb/autonomousdatabase_wallet.yaml +++ b/config/samples/adb/autonomousdatabase_wallet.yaml @@ -13,10 +13,13 @@ spec: # Insert a name of the secret where you want the wallet to be stored. The default name is -instance-wallet. name: instance-wallet password: - # The Name of the secret where you want to hold the wallet password. Comment out k8sSecretName and uncomment ociSecretOCID if you pass the wallet password using OCI Secret. - k8sSecretName: instance-wallet-password - # The OCID of the OCI Secret that holds the password of the ADMIN account. It should start with ocid1.vaultsecret... . - # ociSecretOCID: ocid1.vaultsecret... + # Comment out k8sSecret and uncomment ociSecret if you pass the admin password using OCI Secret. + k8sSecret: + # The Name of the K8s secret where you want to hold the password of the ADMIN account. + name: instance-wallet-password + # ociSecret: + # # The OCID of the OCI Secret that holds the password of the ADMIN account. It should start with ocid1.vaultsecret... . + # ocid: ocid1.vaultsecret... # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. ociConfig: configMapName: oci-cred diff --git a/controllers/database/autonomouscontainerdatabase_controller.go b/controllers/database/autonomouscontainerdatabase_controller.go index f905100c..06050a3f 100644 --- a/controllers/database/autonomouscontainerdatabase_controller.go +++ b/controllers/database/autonomouscontainerdatabase_controller.go @@ -41,6 +41,7 @@ package controllers import ( "context" "encoding/json" + "errors" "reflect" "github.com/go-logr/logr" @@ -72,7 +73,7 @@ type AutonomousContainerDatabaseReconciler struct { Scheme *runtime.Scheme Recorder record.EventRecorder - adbService oci.DatabaseService + dbService oci.DatabaseService workService oci.WorkRequestService lastSucSpec *dbv1alpha1.AutonomousContainerDatabaseSpec } @@ -157,7 +158,7 @@ func (r *AutonomousContainerDatabaseReconciler) Reconcile(ctx context.Context, r if apiErrors.IsNotFound(err) { return emptyResult, nil } - // Failed to get ADB, so we don't need to update the status + // Failed to get ACD, so we don't need to update the status return emptyResult, err } @@ -209,18 +210,18 @@ func (r *AutonomousContainerDatabaseReconciler) Reconcile(ctx context.Context, r } switch action { - case acdActionProvision: + case acdRecActionProvision: if err := r.createACD(acd); err != nil { return r.manageError(acd, err) } - case acdActionBind: + case acdRecActionBind: break - case acdActionUpdate: - // updateADB contains downloadWallet + case acdRecActionUpdate: + // updateACD contains downloadWallet if err := r.updateACD(acd, difACD); err != nil { return r.manageError(acd, err) } - case acdActionSync: + case acdRecActionSync: break } @@ -259,10 +260,10 @@ func (r *AutonomousContainerDatabaseReconciler) updateResourceStatus(acd *dbv1al }) } -// updateResource updates the specification, the status of AutonomousContainerDatabase resource, and the lastSucSpec -func (r *AutonomousContainerDatabaseReconciler) updateResource(adb *dbv1alpha1.AutonomousContainerDatabase) error { +// updateResource updates the specification, the status of AutonomousContainerDatabase resource +func (r *AutonomousContainerDatabaseReconciler) updateResource(acd *dbv1alpha1.AutonomousContainerDatabase) error { // Update the status first to prevent unwanted Reconcile() - if err := r.updateResourceStatus(adb); err != nil { + if err := r.updateResourceStatus(acd); err != nil { return err } @@ -271,15 +272,15 @@ func (r *AutonomousContainerDatabaseReconciler) updateResource(adb *dbv1alpha1.A curACD := &dbv1alpha1.AutonomousContainerDatabase{} namespacedName := types.NamespacedName{ - Namespace: adb.GetNamespace(), - Name: adb.GetName(), + Namespace: acd.GetNamespace(), + Name: acd.GetName(), } if err := r.KubeClient.Get(context.TODO(), namespacedName, curACD); err != nil { return err } - curACD.Spec = adb.Spec + curACD.Spec = acd.Spec return r.KubeClient.Update(context.TODO(), curACD) }); err != nil { return err @@ -294,7 +295,7 @@ func (r *AutonomousContainerDatabaseReconciler) updateResource(adb *dbv1alpha1.A // looking if the lastSucSpec is present. func (r *AutonomousContainerDatabaseReconciler) syncResource(acd *dbv1alpha1.AutonomousContainerDatabase) error { // Get the information from OCI - resp, err := r.adbService.GetAutonomousContainerDatabase(*acd.Spec.AutonomousContainerDatabaseOCID) + resp, err := r.dbService.GetAutonomousContainerDatabase(*acd.Spec.AutonomousContainerDatabaseOCID) if err != nil { return err } @@ -370,7 +371,7 @@ func (r *AutonomousContainerDatabaseReconciler) setupOCIClients(acd *dbv1alpha1. return err } - r.adbService, err = oci.NewDatabaseService(r.Log, r.KubeClient, provider) + r.dbService, err = oci.NewDatabaseService(r.Log, r.KubeClient, provider) if err != nil { return err } @@ -428,45 +429,49 @@ func (r *AutonomousContainerDatabaseReconciler) validateFinalizer(acd *dbv1alpha return false, nil } -type ACDActionEnum string +type acdRecActionEnum string const ( - acdActionProvision = "PROVISION" - acdActionBind = "BIND" - acdActionUpdate = "UPDATE" - acdActionSync = "SYNC" + acdRecActionProvision acdRecActionEnum = "PROVISION" + acdRecActionBind acdRecActionEnum = "BIND" + acdRecActionUpdate acdRecActionEnum = "UPDATE" + acdRecActionSync acdRecActionEnum = "SYNC" ) -func (r *AutonomousContainerDatabaseReconciler) determineAction(adb *dbv1alpha1.AutonomousContainerDatabase) (ACDActionEnum, *dbv1alpha1.AutonomousContainerDatabase, error) { +func (r *AutonomousContainerDatabaseReconciler) determineAction(acd *dbv1alpha1.AutonomousContainerDatabase) (acdRecActionEnum, *dbv1alpha1.AutonomousContainerDatabase, error) { if r.lastSucSpec == nil { - if adb.Spec.AutonomousContainerDatabaseOCID == nil { - return acdActionProvision, nil, nil + if acd.Spec.AutonomousContainerDatabaseOCID == nil { + return acdRecActionProvision, nil, nil } else { - return acdActionBind, nil, nil + return acdRecActionBind, nil, nil } } else { // Pre-process step for the UPDATE. Remove the unchanged fields from spec.details, - difACD := adb.DeepCopy() + difACD := acd.DeepCopy() specChanged, err := difACD.RemoveUnchangedSpec() if err != nil { return "", nil, err } if specChanged { - return acdActionUpdate, difACD, nil + // Return SYNC if the spec.action is SYNC + if difACD.Spec.Action == dbv1alpha1.AcdActionSync { + return acdRecActionSync, nil, nil + } + return acdRecActionUpdate, difACD, nil } - return acdActionSync, nil, nil + return acdRecActionSync, nil, nil } } func (r *AutonomousContainerDatabaseReconciler) createACD(acd *dbv1alpha1.AutonomousContainerDatabase) error { - resp, err := r.adbService.CreateAutonomousContainerDatabase(acd) + resp, err := r.dbService.CreateAutonomousContainerDatabase(acd) if err != nil { return err } - // Update the ADB OCID and the status + // Update the ACD OCID and the status // The trick is to update the status first to prevent unwanted reconcile acd.Spec.AutonomousContainerDatabaseOCID = resp.AutonomousContainerDatabase.Id acd.UpdateStatusFromOCIACD(resp.AutonomousContainerDatabase) @@ -486,14 +491,14 @@ func (r *AutonomousContainerDatabaseReconciler) createACD(acd *dbv1alpha1.Autono return nil } -func (r *AutonomousContainerDatabaseReconciler) updateACD(acd *dbv1alpha1.AutonomousContainerDatabase, difACD *dbv1alpha1.AutonomousContainerDatabase) error { +func (r *AutonomousContainerDatabaseReconciler) updateGeneralFields(acd *dbv1alpha1.AutonomousContainerDatabase, difACD *dbv1alpha1.AutonomousContainerDatabase) error { if difACD.Spec.DisplayName == nil && difACD.Spec.PatchModel == "" && difACD.Spec.FreeformTags == nil { return nil } - resp, err := r.adbService.UpdateAutonomousContainerDatabase(difACD) + resp, err := r.dbService.UpdateAutonomousContainerDatabase(difACD) if err != nil { return err } @@ -517,7 +522,7 @@ func (r *AutonomousContainerDatabaseReconciler) updateACD(acd *dbv1alpha1.Autono func (r *AutonomousContainerDatabaseReconciler) terminateACD(acd *dbv1alpha1.AutonomousContainerDatabase) error { - resp, err := r.adbService.TerminateAutonomousContainerDatabase(*acd.Spec.AutonomousContainerDatabaseOCID) + resp, err := r.dbService.TerminateAutonomousContainerDatabase(*acd.Spec.AutonomousContainerDatabaseOCID) if err != nil { return err } @@ -533,3 +538,54 @@ func (r *AutonomousContainerDatabaseReconciler) terminateACD(acd *dbv1alpha1.Aut } return nil } + +func (r *AutonomousContainerDatabaseReconciler) updateLifecycleState(acd *dbv1alpha1.AutonomousContainerDatabase, difACD *dbv1alpha1.AutonomousContainerDatabase) error { + if difACD.Spec.Action == dbv1alpha1.AcdActionBlank { + return nil + } + + var opcWorkRequestId string + + switch difACD.Spec.Action { + case dbv1alpha1.AcdActionRestart: + resp, err := r.dbService.RestartAutonomousContainerDatabase(*acd.Spec.AutonomousContainerDatabaseOCID) + if err != nil { + return err + } + + acd.Status.LifecycleState = resp.LifecycleState + opcWorkRequestId = *resp.OpcWorkRequestId + case dbv1alpha1.AcdActionTerminate: + resp, err := r.dbService.TerminateAutonomousContainerDatabase(*acd.Spec.AutonomousContainerDatabaseOCID) + if err != nil { + return err + } + + acd.Status.LifecycleState = database.AutonomousContainerDatabaseLifecycleStateTerminating + opcWorkRequestId = *resp.OpcWorkRequestId + default: + return errors.New("Unknown action") + } + + // Update the status and then erase the Action field + if err := r.updateResource(acd); err != nil { + return err + } + + if _, err := r.workService.Wait(opcWorkRequestId); err != nil { + return err + } + return nil +} + +func (r *AutonomousContainerDatabaseReconciler) updateACD(acd *dbv1alpha1.AutonomousContainerDatabase, difACD *dbv1alpha1.AutonomousContainerDatabase) error { + if err := r.updateGeneralFields(acd, difACD); err != nil { + return err + } + + if err := r.updateLifecycleState(acd, difACD); err != nil { + return err + } + + return nil +} diff --git a/controllers/database/autonomousdatabase_controller.go b/controllers/database/autonomousdatabase_controller.go index ce9a1b27..13f586a7 100644 --- a/controllers/database/autonomousdatabase_controller.go +++ b/controllers/database/autonomousdatabase_controller.go @@ -81,7 +81,7 @@ type AutonomousDatabaseReconciler struct { Scheme *runtime.Scheme Recorder record.EventRecorder - adbService oci.DatabaseService + dbService oci.DatabaseService workService oci.WorkRequestService lastSucSpec *dbv1alpha1.AutonomousDatabaseSpec } @@ -271,7 +271,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R } switch action { - case adbActionProvision: + case adbRecActionProvision: if err := r.createADB(adb); err != nil { return r.manageError(adb, err) } @@ -279,16 +279,16 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R if err := r.downloadWallet(adb); err != nil { return r.manageError(adb, err) } - case adbActionBind: + case adbRecActionBind: if err := r.downloadWallet(adb); err != nil { return r.manageError(adb, err) } - case adbActionUpdate: + case adbRecActionUpdate: // updateADB contains downloadWallet if err := r.updateADB(adb, difADB); err != nil { return r.manageError(adb, err) } - case adbActionSync: + case adbRecActionSync: // SYNC action needs to make sure the wallet is present if err := r.downloadWallet(adb); err != nil { return r.manageError(adb, err) @@ -329,7 +329,7 @@ func (r *AutonomousDatabaseReconciler) setupOCIClients(adb *dbv1alpha1.Autonomou return err } - r.adbService, err = oci.NewDatabaseService(r.Log, r.KubeClient, provider) + r.dbService, err = oci.NewDatabaseService(r.Log, r.KubeClient, provider) if err != nil { return err } @@ -463,7 +463,7 @@ func (r *AutonomousDatabaseReconciler) updateResourceStatus(adb *dbv1alpha1.Auto // looking if the lastSucSpec is present. func (r *AutonomousDatabaseReconciler) syncResource(adb *dbv1alpha1.AutonomousDatabase) error { // Get the information from OCI - resp, err := r.adbService.GetAutonomousDatabase(*adb.Spec.Details.AutonomousDatabaseOCID) + resp, err := r.dbService.GetAutonomousDatabase(*adb.Spec.Details.AutonomousDatabaseOCID) if err != nil { return err } @@ -511,21 +511,21 @@ func (r *AutonomousDatabaseReconciler) updateResource(adb *dbv1alpha1.Autonomous return nil } -type ADBActionEnum string +type adbRecActionEnum string const ( - adbActionProvision = "PROVISION" - adbActionBind = "BIND" - adbActionUpdate = "UPDATE" - adbActionSync = "SYNC" + adbRecActionProvision adbRecActionEnum = "PROVISION" + adbRecActionBind adbRecActionEnum = "BIND" + adbRecActionUpdate adbRecActionEnum = "UPDATE" + adbRecActionSync adbRecActionEnum = "SYNC" ) -func (r *AutonomousDatabaseReconciler) determineAction(adb *dbv1alpha1.AutonomousDatabase) (ACDActionEnum, *dbv1alpha1.AutonomousDatabase, error) { +func (r *AutonomousDatabaseReconciler) determineAction(adb *dbv1alpha1.AutonomousDatabase) (adbRecActionEnum, *dbv1alpha1.AutonomousDatabase, error) { if r.lastSucSpec == nil { if adb.Spec.Details.AutonomousDatabaseOCID == nil { - return acdActionProvision, nil, nil + return adbRecActionProvision, nil, nil } else { - return acdActionBind, nil, nil + return adbRecActionBind, nil, nil } } else { // Pre-process step for the UPDATE. Remove the unchanged fields in spec.details, @@ -536,15 +536,15 @@ func (r *AutonomousDatabaseReconciler) determineAction(adb *dbv1alpha1.Autonomou } if detailsChanged { - return acdActionUpdate, difADB, nil + return adbRecActionUpdate, difADB, nil } - return acdActionSync, nil, nil + return adbRecActionSync, nil, nil } } func (r *AutonomousDatabaseReconciler) createADB(adb *dbv1alpha1.AutonomousDatabase) error { - resp, err := r.adbService.CreateAutonomousDatabase(adb) + resp, err := r.dbService.CreateAutonomousDatabase(adb) if err != nil { return err } @@ -577,7 +577,7 @@ func (r *AutonomousDatabaseReconciler) updateGeneralFields(adb *dbv1alpha1.Auton return nil } - resp, err := r.adbService.UpdateAutonomousDatabaseGeneralFields(difADB) + resp, err := r.dbService.UpdateAutonomousDatabaseGeneralFields(difADB) if err != nil { return err } @@ -605,7 +605,7 @@ func (r *AutonomousDatabaseReconciler) updateAdminPassword(adb *dbv1alpha1.Auton return nil } - _, err := r.adbService.UpdateAutonomousDatabaseAdminPassword(difADB) + _, err := r.dbService.UpdateAutonomousDatabaseAdminPassword(difADB) if err != nil { return err } @@ -619,7 +619,7 @@ func (r *AutonomousDatabaseReconciler) updateDbWorkload(adb *dbv1alpha1.Autonomo return nil } - resp, err := r.adbService.UpdateAutonomousDatabaseDBWorkload(difADB) + resp, err := r.dbService.UpdateAutonomousDatabaseDBWorkload(difADB) if err != nil { return err } @@ -640,7 +640,7 @@ func (r *AutonomousDatabaseReconciler) updateLicenseModel(adb *dbv1alpha1.Autono return nil } - resp, err := r.adbService.UpdateAutonomousDatabaseLicenseModel(difADB) + resp, err := r.dbService.UpdateAutonomousDatabaseLicenseModel(difADB) if err != nil { return err } @@ -663,7 +663,7 @@ func (r *AutonomousDatabaseReconciler) updateScalingFields(adb *dbv1alpha1.Auton return nil } - resp, err := r.adbService.UpdateAutonomousDatabaseScalingFields(difADB) + resp, err := r.dbService.UpdateAutonomousDatabaseScalingFields(difADB) if err != nil { return err } @@ -681,7 +681,7 @@ func (r *AutonomousDatabaseReconciler) updateScalingFields(adb *dbv1alpha1.Auton func (r *AutonomousDatabaseReconciler) deleteAutonomousDatabase(adb *dbv1alpha1.AutonomousDatabase) error { - resp, err := r.adbService.DeleteAutonomousDatabase(*adb.Spec.Details.AutonomousDatabaseOCID) + resp, err := r.dbService.DeleteAutonomousDatabase(*adb.Spec.Details.AutonomousDatabaseOCID) if err != nil { return err } @@ -706,7 +706,7 @@ func (r *AutonomousDatabaseReconciler) updateLifecycleState(adb *dbv1alpha1.Auto switch difADB.Spec.Details.LifecycleState { case database.AutonomousDatabaseLifecycleStateAvailable: - resp, err := r.adbService.StartAutonomousDatabase(*difADB.Spec.Details.AutonomousDatabaseOCID) + resp, err := r.dbService.StartAutonomousDatabase(*difADB.Spec.Details.AutonomousDatabaseOCID) if err != nil { return err } @@ -714,7 +714,7 @@ func (r *AutonomousDatabaseReconciler) updateLifecycleState(adb *dbv1alpha1.Auto adb.Status.LifecycleState = resp.LifecycleState opcWorkRequestId = *resp.OpcWorkRequestId case database.AutonomousDatabaseLifecycleStateStopped: - resp, err := r.adbService.StopAutonomousDatabase(*difADB.Spec.Details.AutonomousDatabaseOCID) + resp, err := r.dbService.StopAutonomousDatabase(*difADB.Spec.Details.AutonomousDatabaseOCID) if err != nil { return err } @@ -722,7 +722,7 @@ func (r *AutonomousDatabaseReconciler) updateLifecycleState(adb *dbv1alpha1.Auto adb.Status.LifecycleState = resp.LifecycleState opcWorkRequestId = *resp.OpcWorkRequestId case database.AutonomousDatabaseLifecycleStateTerminated: - resp, err := r.adbService.DeleteAutonomousDatabase(*difADB.Spec.Details.AutonomousDatabaseOCID) + resp, err := r.dbService.DeleteAutonomousDatabase(*difADB.Spec.Details.AutonomousDatabaseOCID) if err != nil { return err } @@ -744,7 +744,7 @@ func (r *AutonomousDatabaseReconciler) updateLifecycleState(adb *dbv1alpha1.Auto } func (r *AutonomousDatabaseReconciler) setMTLSRequired(adb *dbv1alpha1.AutonomousDatabase) error { - resp, err := r.adbService.UpdateNetworkAccessMTLSRequired(*adb.Spec.Details.AutonomousDatabaseOCID) + resp, err := r.dbService.UpdateNetworkAccessMTLSRequired(*adb.Spec.Details.AutonomousDatabaseOCID) if err != nil { return err } @@ -765,7 +765,7 @@ func (r *AutonomousDatabaseReconciler) updateMTLS(adb *dbv1alpha1.AutonomousData return nil } - resp, err := r.adbService.UpdateNetworkAccessMTLS(difADB) + resp, err := r.dbService.UpdateNetworkAccessMTLS(difADB) if err != nil { return err } @@ -782,7 +782,7 @@ func (r *AutonomousDatabaseReconciler) updateMTLS(adb *dbv1alpha1.AutonomousData } func (r *AutonomousDatabaseReconciler) setNetworkAccessPublic(adb *dbv1alpha1.AutonomousDatabase) error { - resp, err := r.adbService.UpdateNetworkAccessPublic(r.lastSucSpec.Details.NetworkAccess.AccessType, *adb.Spec.Details.AutonomousDatabaseOCID) + resp, err := r.dbService.UpdateNetworkAccessPublic(r.lastSucSpec.Details.NetworkAccess.AccessType, *adb.Spec.Details.AutonomousDatabaseOCID) if err != nil { return err } @@ -807,7 +807,7 @@ func (r *AutonomousDatabaseReconciler) updateNetworkAccess(adb *dbv1alpha1.Auton difADB.Spec.Details.NetworkAccess.PrivateEndpoint.HostnamePrefix == nil { return nil } - resp, err := r.adbService.UpdateNetworkAccess(difADB) + resp, err := r.dbService.UpdateNetworkAccess(difADB) if err != nil { return err } @@ -954,7 +954,7 @@ func (r *AutonomousDatabaseReconciler) downloadWallet(adb *dbv1alpha1.Autonomous return err } - resp, err := r.adbService.DownloadWallet(adb) + resp, err := r.dbService.DownloadWallet(adb) if err != nil { return err } @@ -975,20 +975,6 @@ func (r *AutonomousDatabaseReconciler) downloadWallet(adb *dbv1alpha1.Autonomous return nil } -func (r *AutonomousDatabaseReconciler) getValidBackupName(name string, usedNames map[string]bool) string { - returnedName := name - var i = 1 - - _, ok := usedNames[returnedName] - for ok { - returnedName = fmt.Sprintf("%s-%d", name, i) - _, ok = usedNames[returnedName] - i++ - } - - return returnedName -} - func (r *AutonomousDatabaseReconciler) updateADB(adb *dbv1alpha1.AutonomousDatabase, difADB *dbv1alpha1.AutonomousDatabase) error { if err := r.updateGeneralFields(adb, difADB); err != nil { return err @@ -1025,6 +1011,52 @@ func (r *AutonomousDatabaseReconciler) updateADB(adb *dbv1alpha1.AutonomousDatab return nil } +func (r *AutonomousDatabaseReconciler) getValidBackupName(displayName string, usedNames map[string]bool) (string, error) { + // Convert the displayName to lowercase, and replace spaces, commas, and colons with hyphens + baseName := strings.ToLower(displayName) + + re, err := regexp.Compile(`[^-a-zA-Z0-9]`) + if err != nil { + return "", err + } + + baseName = re.ReplaceAllString(baseName, "-") + + finalName := baseName + var i = 1 + _, ok := usedNames[finalName] + for ok { + finalName = fmt.Sprintf("%s-%d", baseName, i) + _, ok = usedNames[finalName] + i++ + } + + return finalName, nil +} + +func (r *AutonomousDatabaseReconciler) ifBackupExists(backupSummary database.AutonomousDatabaseBackupSummary, curBackupOCIDs map[string]bool, backupList *dbv1alpha1.AutonomousDatabaseBackupList) bool { + _, ok := curBackupOCIDs[*backupSummary.Id] + if ok { + return true + } + + // Special case: when a Backup is creating and hasn't updated the OCID, a duplicated Backup might be created by mistake. + // To handle this case, skip creating the backup if the current backupSummary is with CREATING state, and there is + // another AutonomousBackup with the same displayName in the cluster is also at CREATING state. + if backupSummary.LifecycleState == database.AutonomousDatabaseBackupSummaryLifecycleStateCreating { + for _, backup := range backupList.Items { + if (backup.Spec.DisplayName != nil && *backup.Spec.DisplayName == *backupSummary.DisplayName) && + (backup.Status.LifecycleState == "" || + backup.Status.LifecycleState == dbv1alpha1.BackupStateError || + backup.Status.LifecycleState == dbv1alpha1.BackupStateCreating) { + return true + } + } + } + + return false +} + // updateBackupResources get the list of AutonomousDatabasBackups and // create a backup object if it's not found in the same namespace func (r *AutonomousDatabaseReconciler) syncBackupResources(adb *dbv1alpha1.AutonomousDatabase) error { @@ -1040,42 +1072,37 @@ func (r *AutonomousDatabaseReconciler) syncBackupResources(adb *dbv1alpha1.Auton curBackupOCIDs := make(map[string]bool) for _, backup := range backupList.Items { + // mark the backup name that exists curBackupNames[backup.Name] = true - if backup.Spec.AutonomousDatabaseBackupOCID != nil { - curBackupOCIDs[*backup.Spec.AutonomousDatabaseBackupOCID] = true + // mark the backup ocid that exists + if backup.Status.AutonomousDatabaseBackupOCID != "" { + curBackupOCIDs[backup.Status.AutonomousDatabaseBackupOCID] = true } } - resp, err := r.adbService.ListAutonomousDatabaseBackups(*adb.Spec.Details.AutonomousDatabaseOCID) + resp, err := r.dbService.ListAutonomousDatabaseBackups(*adb.Spec.Details.AutonomousDatabaseOCID) if err != nil { return err } for _, backupSummary := range resp.Items { - // Create the resource if the AutonomousDatabaseBackupOCID doesn't exist - _, ok := curBackupOCIDs[*backupSummary.Id] - if !ok { - // Convert the string to lowercase, and replace spaces, commas, and colons with hyphens - resourceName := *backupSummary.DisplayName - resourceName = strings.ToLower(resourceName) - - re, err := regexp.Compile(`[^-a-zA-Z0-9]`) + // Create the resource if the backup doesn't exist + if !r.ifBackupExists(backupSummary, curBackupOCIDs, backupList) { + validBackupName, err := r.getValidBackupName(*backupSummary.DisplayName, curBackupNames) if err != nil { return err } - resourceName = re.ReplaceAllString(resourceName, "-") - resourceName = r.getValidBackupName(resourceName, curBackupNames) - if err := k8s.CreateAutonomousBackup(r.KubeClient, resourceName, backupSummary, adb); err != nil { + if err := k8s.CreateAutonomousBackup(r.KubeClient, validBackupName, backupSummary, adb); err != nil { return err } - // Add the used names and ocids - curBackupNames[resourceName] = true + // Add the used name and ocid + curBackupNames[validBackupName] = true curBackupOCIDs[*backupSummary.AutonomousDatabaseId] = true - logger.Info("Create AutonomousDatabaseBackup " + resourceName) + logger.Info("Create AutonomousDatabaseBackup " + validBackupName) } } diff --git a/controllers/database/autonomousdatabasebackup_controller.go b/controllers/database/autonomousdatabasebackup_controller.go index 5cee5e5a..9ee1576f 100644 --- a/controllers/database/autonomousdatabasebackup_controller.go +++ b/controllers/database/autonomousdatabasebackup_controller.go @@ -51,10 +51,10 @@ import ( ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/controller-runtime/pkg/controller" + "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" "sigs.k8s.io/controller-runtime/pkg/event" "sigs.k8s.io/controller-runtime/pkg/predicate" - "github.com/oracle/oci-go-sdk/v54/database" databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" "github.com/oracle/oracle-database-operator/commons/adb_family" @@ -62,7 +62,7 @@ import ( "github.com/oracle/oracle-database-operator/commons/oci" ) -// *AutonomousDatabaseBackupReconciler reconciles a AutonomousDatabaseBackup object +// AutonomousDatabaseBackupReconciler reconciles a AutonomousDatabaseBackup object type AutonomousDatabaseBackupReconciler struct { KubeClient client.Client Log logr.Logger @@ -73,15 +73,6 @@ type AutonomousDatabaseBackupReconciler struct { workService oci.WorkRequestService } -// SetupWithManager sets up the controller with the Manager. -func (r *AutonomousDatabaseBackupReconciler) SetupWithManager(mgr ctrl.Manager) error { - return ctrl.NewControllerManagedBy(mgr). - For(&databasev1alpha1.AutonomousDatabaseBackup{}). - WithEventFilter(predicate.And(r.eventFilterPredicate(), predicate.GenerationChangedPredicate{})). - WithOptions(controller.Options{MaxConcurrentReconciles: 100}). // ReconcileHandler is never invoked concurrently with the same object. - Complete(r) -} - func (r *AutonomousDatabaseBackupReconciler) eventFilterPredicate() predicate.Predicate { pred := predicate.Funcs{ CreateFunc: func(e event.CreateEvent) bool { @@ -90,9 +81,8 @@ func (r *AutonomousDatabaseBackupReconciler) eventFilterPredicate() predicate.Pr UpdateFunc: func(e event.UpdateEvent) bool { oldStatus := e.ObjectOld.(*dbv1alpha1.AutonomousDatabaseBackup).Status.LifecycleState - if oldStatus == database.AutonomousDatabaseBackupLifecycleStateCreating || - oldStatus == database.AutonomousDatabaseBackupLifecycleStateDeleting { - // All the requests other than the terminate request, should be discarded during the intermediate states + if oldStatus == dbv1alpha1.BackupStateCreating || + oldStatus == dbv1alpha1.BackupStateDeleting { return false } @@ -103,6 +93,24 @@ func (r *AutonomousDatabaseBackupReconciler) eventFilterPredicate() predicate.Pr return pred } +// SetupWithManager sets up the controller with the Manager. +func (r *AutonomousDatabaseBackupReconciler) SetupWithManager(mgr ctrl.Manager) error { + return ctrl.NewControllerManagedBy(mgr). + For(&databasev1alpha1.AutonomousDatabaseBackup{}). + WithEventFilter(r.eventFilterPredicate()). + WithOptions(controller.Options{MaxConcurrentReconciles: 100}). // ReconcileHandler is never invoked concurrently with the same object. + Complete(r) +} + +func (r *AutonomousDatabaseBackupReconciler) backupStarted(backup *dbv1alpha1.AutonomousDatabaseBackup) bool { + return backup.Spec.AutonomousDatabaseBackupOCID != nil || + (backup.Status.LifecycleState == dbv1alpha1.BackupStateCreating || + backup.Status.LifecycleState == dbv1alpha1.BackupStateActive || + backup.Status.LifecycleState == dbv1alpha1.BackupStateDeleting || + backup.Status.LifecycleState == dbv1alpha1.BackupStateDeleted || + backup.Status.LifecycleState == dbv1alpha1.BackupStateFailed) +} + //+kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabasebackups,verbs=get;list;watch;create;delete //+kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabasebackups/status,verbs=get;update;patch //+kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabases,verbs=get;list @@ -144,7 +152,7 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req * LifecycleState is checked to avoid sending a duplicated backup request when the backup is creating. * Otherwise, bind to an exisiting backup if the Spec.AutonomousDatabaseBackupOCID isn't empty. ******************************************************************/ - if backup.Spec.AutonomousDatabaseBackupOCID == nil && backup.Status.LifecycleState == "" { + if !r.backupStarted(backup) { // Create a new backup backupResp, err := r.adbService.CreateAutonomousDatabaseBackup(backup, adbOCID) if err != nil { @@ -205,6 +213,19 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req return emptyResult, nil } +// setOwnerAutonomousDatabase sets the owner of the AutonomousDatabaseBackup if the AutonomousDatabase resource with the same database OCID is found +func (r *AutonomousDatabaseBackupReconciler) setOwnerAutonomousDatabase(backup *dbv1alpha1.AutonomousDatabaseBackup, adb *dbv1alpha1.AutonomousDatabase) error { + logger := r.Log.WithName("set-owner-reference") + + controllerutil.SetOwnerReference(adb, backup, r.Scheme) + if err := r.KubeClient.Update(context.TODO(), backup); err != nil { + return err + } + logger.Info(fmt.Sprintf("Set the owner of AutonomousDatabaseBackup %s to AutonomousDatabase %s", backup.Name, adb.Name)) + + return nil +} + // verifyTargetADB searches if the target ADB is in the cluster, and set the owner reference to the ADB if it exists. // The function returns the OCID of the target ADB. func (r *AutonomousDatabaseBackupReconciler) verifyTargetADB(backup *dbv1alpha1.AutonomousDatabaseBackup) (string, error) { @@ -252,37 +273,12 @@ func (r *AutonomousDatabaseBackupReconciler) setupOCIClients(backup *dbv1alpha1. return nil } -// updateResource updates the specification and the status of AutonomousDatabase resource without trigger a reconcile loop -func (r *AutonomousDatabaseBackupReconciler) updateResource(backup *dbv1alpha1.AutonomousDatabaseBackup) error { - if err := r.KubeClient.Update(context.TODO(), backup); err != nil { - return err - } - - // Update the status - if err := r.KubeClient.Status().Update(context.TODO(), backup); err != nil { - return err - } - - return nil -} - -// setOwnerAutonomousDatabase sets the owner of the AutonomousDatabaseBackup if the AutonomousDatabase resource with the same database OCID is found -func (r *AutonomousDatabaseBackupReconciler) setOwnerAutonomousDatabase(backup *dbv1alpha1.AutonomousDatabaseBackup, adb *dbv1alpha1.AutonomousDatabase) error { - logger := r.Log.WithName("set-owner-reference") - - backup.SetOwnerReferences(k8s.NewOwnerReference(adb)) - r.updateResource(backup) - logger.Info(fmt.Sprintf("Set the owner of %s to %s", backup.Name, adb.Name)) - - return nil -} - func (r *AutonomousDatabaseBackupReconciler) manageError(backup *dbv1alpha1.AutonomousDatabaseBackup, issue error) (ctrl.Result, error) { // Send event r.Recorder.Event(backup, corev1.EventTypeWarning, "ReconcileFailed", issue.Error()) - // Change the status to FAILED - backup.Status.LifecycleState = database.AutonomousDatabaseBackupLifecycleStateFailed + // Change the status to ERROR + backup.Status.LifecycleState = dbv1alpha1.BackupStateError if statusErr := r.KubeClient.Status().Update(context.TODO(), backup); statusErr != nil { return emptyResult, k8s.CombineErrors(issue, statusErr) } diff --git a/controllers/database/autonomousdatabaserestore_controller.go b/controllers/database/autonomousdatabaserestore_controller.go index 1e3e16ac..96d8ea52 100644 --- a/controllers/database/autonomousdatabaserestore_controller.go +++ b/controllers/database/autonomousdatabaserestore_controller.go @@ -52,6 +52,7 @@ import ( "k8s.io/client-go/util/retry" ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" "sigs.k8s.io/controller-runtime/pkg/predicate" "github.com/oracle/oci-go-sdk/v54/common" @@ -81,6 +82,13 @@ func (r *AutonomousDatabaseRestoreReconciler) SetupWithManager(mgr ctrl.Manager) Complete(r) } +func (r *AutonomousDatabaseRestoreReconciler) restoreStarted(restore *dbv1alpha1.AutonomousDatabaseRestore) bool { + return restore.Status.TimeAccepted != "" || + (restore.Status.Status == dbv1alpha1.RestoreStatusInProgress || + restore.Status.Status == dbv1alpha1.RestoreStatusFailed || + restore.Status.Status == dbv1alpha1.RestoreStatusSucceeded) +} + //+kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabaserestores,verbs=get;list;watch;create;delete //+kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabaserestores/status,verbs=get;update;patch //+kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabases,verbs=get;list @@ -108,7 +116,7 @@ func (r *AutonomousDatabaseRestoreReconciler) Reconcile(ctx context.Context, req return emptyResult, err } - if restore.Status.Status != "" { + if r.restoreStarted(restore) { return emptyResult, nil } @@ -178,15 +186,13 @@ func (r *AutonomousDatabaseRestoreReconciler) getRestoreSDKTime(restore *dbv1alp // setOwnerAutonomousDatabase sets the owner of the AutonomousDatabaseBackup if the AutonomousDatabase resource with the same database OCID is found func (r *AutonomousDatabaseRestoreReconciler) setOwnerAutonomousDatabase(restore *dbv1alpha1.AutonomousDatabaseRestore, adb *dbv1alpha1.AutonomousDatabase) error { - logger := r.Log.WithName("set-owner") - - restore.SetOwnerReferences(k8s.NewOwnerReference(adb)) + logger := r.Log.WithName("set-owner-reference") + controllerutil.SetOwnerReference(adb, restore, r.Scheme) if err := r.KubeClient.Update(context.TODO(), restore); err != nil { return err } - - logger.Info(fmt.Sprintf("Set the owner of %s to %s", restore.Name, adb.Name)) + logger.Info(fmt.Sprintf("Set the owner of AutonomousDatabaseRestore %s to AutonomousDatabase %s", restore.Name, adb.Name)) return nil } @@ -267,10 +273,10 @@ func (r *AutonomousDatabaseRestoreReconciler) manageError(restore *dbv1alpha1.Au // Send event r.Recorder.Event(restore, corev1.EventTypeWarning, "ReconcileFailed", issue.Error()) - // Change the status to FAILED + // Change the status to ERROR var combinedErr error = issue - restore.Status.Status = dbv1alpha1.RestoreStatusFailed + restore.Status.Status = dbv1alpha1.RestoreStatusError if statusErr := r.updateResourceStatus(restore); statusErr != nil { combinedErr = k8s.CombineErrors(issue, statusErr) } @@ -297,29 +303,26 @@ func (r *AutonomousDatabaseRestoreReconciler) restoreAutonomousDatabase( restore.Status.DbName = *resp.AutonomousDatabase.DbName restore.Status.AutonomousDatabaseOCID = *resp.AutonomousDatabase.Id - workStatusStart, err := r.workService.Get(*resp.OpcWorkRequestId) - if err != nil { - return err - } - restore.Status.Status, err = restore.ConvertWorkRequestStatus(workStatusStart) + workStart, err := r.workService.Get(*resp.OpcWorkRequestId) if err != nil { return err } + restore.Status.Status = restore.ConvertWorkRequestStatus(workStart.Status) + restore.Status.TimeAccepted = dbv1alpha1.FormatSDKTime(workStart.TimeAccepted) r.updateResourceStatus(restore) - workStatusEnd, err := r.workService.Wait(*resp.OpcWorkRequestId) + workEnd, err := r.workService.Wait(*resp.OpcWorkRequestId) if err != nil { return err } // Update status when the work is finished - restore.Status.Status, err = restore.ConvertWorkRequestStatus(workStatusEnd) - if err != nil { - return err - } + restore.Status.Status = restore.ConvertWorkRequestStatus(workEnd.Status) + restore.Status.TimeStarted = dbv1alpha1.FormatSDKTime(workEnd.TimeStarted) + restore.Status.TimeEnded = dbv1alpha1.FormatSDKTime(workEnd.TimeFinished) r.updateResourceStatus(restore) return nil -} \ No newline at end of file +} From 8f5cdd0ad3df5a9e35d5deadc274f80859d44f61 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Thu, 24 Mar 2022 18:49:39 -0400 Subject: [PATCH 204/628] update oci-go-sdk to v63 --- apis/database/v1alpha1/adbfamily_common_utils.go | 2 +- apis/database/v1alpha1/autonomouscontainerdatabase_types.go | 2 +- apis/database/v1alpha1/autonomousdatabase_types.go | 2 +- apis/database/v1alpha1/autonomousdatabasebackup_types.go | 4 ++-- apis/database/v1alpha1/autonomousdatabaserestore_types.go | 4 ++-- commons/adb_family/utils.go | 2 +- commons/k8s/create.go | 4 ++-- commons/oci/containerdatabase.go | 4 ++-- commons/oci/database.go | 4 ++-- commons/oci/provider.go | 4 ++-- commons/oci/vault.go | 4 ++-- commons/oci/workrequest.go | 4 ++-- commons/sharding/scommon.go | 4 ++-- .../database/autonomouscontainerdatabase_controller.go | 2 +- controllers/database/autonomousdatabase_controller.go | 2 +- .../database/autonomousdatabaserestore_controller.go | 2 +- controllers/database/shardingdatabase_controller.go | 4 ++-- test/e2e/autonomousdatabase_controller_bind_test.go | 6 +++--- test/e2e/autonomousdatabase_controller_create_test.go | 2 +- test/e2e/behavior/shared_behaviors.go | 4 ++-- test/e2e/suite_test.go | 4 ++-- test/e2e/util/oci_config_util.go | 2 +- 22 files changed, 36 insertions(+), 36 deletions(-) diff --git a/apis/database/v1alpha1/adbfamily_common_utils.go b/apis/database/v1alpha1/adbfamily_common_utils.go index c3d505c7..b0c28f49 100644 --- a/apis/database/v1alpha1/adbfamily_common_utils.go +++ b/apis/database/v1alpha1/adbfamily_common_utils.go @@ -46,7 +46,7 @@ import ( metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "sigs.k8s.io/controller-runtime/pkg/client" - "github.com/oracle/oci-go-sdk/v54/common" + "github.com/oracle/oci-go-sdk/v63/common" ) // LastSuccessfulSpec is an annotation key which maps to the value of last successful spec diff --git a/apis/database/v1alpha1/autonomouscontainerdatabase_types.go b/apis/database/v1alpha1/autonomouscontainerdatabase_types.go index 9d4e456d..c5a928f8 100644 --- a/apis/database/v1alpha1/autonomouscontainerdatabase_types.go +++ b/apis/database/v1alpha1/autonomouscontainerdatabase_types.go @@ -42,7 +42,7 @@ import ( "encoding/json" "errors" - "github.com/oracle/oci-go-sdk/v54/database" + "github.com/oracle/oci-go-sdk/v63/database" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" ) diff --git a/apis/database/v1alpha1/autonomousdatabase_types.go b/apis/database/v1alpha1/autonomousdatabase_types.go index d2c9609c..c078aa73 100644 --- a/apis/database/v1alpha1/autonomousdatabase_types.go +++ b/apis/database/v1alpha1/autonomousdatabase_types.go @@ -42,7 +42,7 @@ import ( "encoding/json" "errors" - "github.com/oracle/oci-go-sdk/v54/database" + "github.com/oracle/oci-go-sdk/v63/database" metaV1 "k8s.io/apimachinery/pkg/apis/meta/v1" ) diff --git a/apis/database/v1alpha1/autonomousdatabasebackup_types.go b/apis/database/v1alpha1/autonomousdatabasebackup_types.go index 3ade554d..52113f4d 100644 --- a/apis/database/v1alpha1/autonomousdatabasebackup_types.go +++ b/apis/database/v1alpha1/autonomousdatabasebackup_types.go @@ -41,8 +41,8 @@ package v1alpha1 import ( metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" - "github.com/oracle/oci-go-sdk/v54/common" - "github.com/oracle/oci-go-sdk/v54/database" + "github.com/oracle/oci-go-sdk/v63/common" + "github.com/oracle/oci-go-sdk/v63/database" ) // EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! diff --git a/apis/database/v1alpha1/autonomousdatabaserestore_types.go b/apis/database/v1alpha1/autonomousdatabaserestore_types.go index cd9637f6..6f65ee9a 100644 --- a/apis/database/v1alpha1/autonomousdatabaserestore_types.go +++ b/apis/database/v1alpha1/autonomousdatabaserestore_types.go @@ -43,8 +43,8 @@ import ( metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" - "github.com/oracle/oci-go-sdk/v54/common" - "github.com/oracle/oci-go-sdk/v54/workrequests" + "github.com/oracle/oci-go-sdk/v63/common" + "github.com/oracle/oci-go-sdk/v63/workrequests" ) // EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! diff --git a/commons/adb_family/utils.go b/commons/adb_family/utils.go index 1650dd22..ed0e1efc 100644 --- a/commons/adb_family/utils.go +++ b/commons/adb_family/utils.go @@ -39,7 +39,7 @@ package adbfamily import ( - "github.com/oracle/oci-go-sdk/v54/common" + "github.com/oracle/oci-go-sdk/v63/common" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" "github.com/oracle/oracle-database-operator/commons/k8s" "sigs.k8s.io/controller-runtime/pkg/client" diff --git a/commons/k8s/create.go b/commons/k8s/create.go index fa53704c..ad522504 100644 --- a/commons/k8s/create.go +++ b/commons/k8s/create.go @@ -43,8 +43,8 @@ import ( dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" - "github.com/oracle/oci-go-sdk/v54/common" - "github.com/oracle/oci-go-sdk/v54/database" + "github.com/oracle/oci-go-sdk/v63/common" + "github.com/oracle/oci-go-sdk/v63/database" corev1 "k8s.io/api/core/v1" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "sigs.k8s.io/controller-runtime/pkg/client" diff --git a/commons/oci/containerdatabase.go b/commons/oci/containerdatabase.go index 359f8a4b..bd2c7429 100644 --- a/commons/oci/containerdatabase.go +++ b/commons/oci/containerdatabase.go @@ -41,8 +41,8 @@ package oci import ( "context" - "github.com/oracle/oci-go-sdk/v54/common" - "github.com/oracle/oci-go-sdk/v54/database" + "github.com/oracle/oci-go-sdk/v63/common" + "github.com/oracle/oci-go-sdk/v63/database" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" ) diff --git a/commons/oci/database.go b/commons/oci/database.go index 142e634f..bf4b9443 100644 --- a/commons/oci/database.go +++ b/commons/oci/database.go @@ -43,8 +43,8 @@ import ( "fmt" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v54/common" - "github.com/oracle/oci-go-sdk/v54/database" + "github.com/oracle/oci-go-sdk/v63/common" + "github.com/oracle/oci-go-sdk/v63/database" "sigs.k8s.io/controller-runtime/pkg/client" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" diff --git a/commons/oci/provider.go b/commons/oci/provider.go index c7592002..83dee0a0 100644 --- a/commons/oci/provider.go +++ b/commons/oci/provider.go @@ -41,8 +41,8 @@ package oci import ( "errors" - "github.com/oracle/oci-go-sdk/v54/common" - "github.com/oracle/oci-go-sdk/v54/common/auth" + "github.com/oracle/oci-go-sdk/v63/common" + "github.com/oracle/oci-go-sdk/v63/common/auth" "sigs.k8s.io/controller-runtime/pkg/client" diff --git a/commons/oci/vault.go b/commons/oci/vault.go index 71b162ba..bc72a9d3 100644 --- a/commons/oci/vault.go +++ b/commons/oci/vault.go @@ -43,8 +43,8 @@ import ( "encoding/base64" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v54/common" - "github.com/oracle/oci-go-sdk/v54/secrets" + "github.com/oracle/oci-go-sdk/v63/common" + "github.com/oracle/oci-go-sdk/v63/secrets" ) type VaultService interface { diff --git a/commons/oci/workrequest.go b/commons/oci/workrequest.go index cf05e506..0ac02911 100644 --- a/commons/oci/workrequest.go +++ b/commons/oci/workrequest.go @@ -46,8 +46,8 @@ import ( "github.com/go-logr/logr" "sigs.k8s.io/controller-runtime/pkg/client" - "github.com/oracle/oci-go-sdk/v54/common" - "github.com/oracle/oci-go-sdk/v54/workrequests" + "github.com/oracle/oci-go-sdk/v63/common" + "github.com/oracle/oci-go-sdk/v63/workrequests" ) type WorkRequestService interface { diff --git a/commons/sharding/scommon.go b/commons/sharding/scommon.go index bc9030ec..60e661da 100644 --- a/commons/sharding/scommon.go +++ b/commons/sharding/scommon.go @@ -48,8 +48,8 @@ import ( "strings" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v54/common" - "github.com/oracle/oci-go-sdk/v54/ons" + "github.com/oracle/oci-go-sdk/v63/common" + "github.com/oracle/oci-go-sdk/v63/ons" appsv1 "k8s.io/api/apps/v1" corev1 "k8s.io/api/core/v1" "k8s.io/apimachinery/pkg/api/errors" diff --git a/controllers/database/autonomouscontainerdatabase_controller.go b/controllers/database/autonomouscontainerdatabase_controller.go index 06050a3f..badd0b03 100644 --- a/controllers/database/autonomouscontainerdatabase_controller.go +++ b/controllers/database/autonomouscontainerdatabase_controller.go @@ -45,7 +45,7 @@ import ( "reflect" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v54/database" + "github.com/oracle/oci-go-sdk/v63/database" corev1 "k8s.io/api/core/v1" apiErrors "k8s.io/apimachinery/pkg/api/errors" diff --git a/controllers/database/autonomousdatabase_controller.go b/controllers/database/autonomousdatabase_controller.go index 13f586a7..7cc37315 100644 --- a/controllers/database/autonomousdatabase_controller.go +++ b/controllers/database/autonomousdatabase_controller.go @@ -48,7 +48,7 @@ import ( "strings" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v54/database" + "github.com/oracle/oci-go-sdk/v63/database" corev1 "k8s.io/api/core/v1" apiErrors "k8s.io/apimachinery/pkg/api/errors" diff --git a/controllers/database/autonomousdatabaserestore_controller.go b/controllers/database/autonomousdatabaserestore_controller.go index 96d8ea52..7ad1f591 100644 --- a/controllers/database/autonomousdatabaserestore_controller.go +++ b/controllers/database/autonomousdatabaserestore_controller.go @@ -55,7 +55,7 @@ import ( "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" "sigs.k8s.io/controller-runtime/pkg/predicate" - "github.com/oracle/oci-go-sdk/v54/common" + "github.com/oracle/oci-go-sdk/v63/common" databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" "github.com/oracle/oracle-database-operator/commons/adb_family" diff --git a/controllers/database/shardingdatabase_controller.go b/controllers/database/shardingdatabase_controller.go index aa37750d..ead408a7 100644 --- a/controllers/database/shardingdatabase_controller.go +++ b/controllers/database/shardingdatabase_controller.go @@ -47,8 +47,8 @@ import ( "time" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v54/common" - "github.com/oracle/oci-go-sdk/v54/ons" + "github.com/oracle/oci-go-sdk/v63/common" + "github.com/oracle/oci-go-sdk/v63/ons" appsv1 "k8s.io/api/apps/v1" corev1 "k8s.io/api/core/v1" "k8s.io/apimachinery/pkg/api/errors" diff --git a/test/e2e/autonomousdatabase_controller_bind_test.go b/test/e2e/autonomousdatabase_controller_bind_test.go index a028f711..5608fc91 100644 --- a/test/e2e/autonomousdatabase_controller_bind_test.go +++ b/test/e2e/autonomousdatabase_controller_bind_test.go @@ -42,9 +42,9 @@ import ( "context" "time" - "github.com/oracle/oci-go-sdk/v54/common" - "github.com/oracle/oci-go-sdk/v54/database" - "github.com/oracle/oci-go-sdk/v54/workrequests" + "github.com/oracle/oci-go-sdk/v63/common" + "github.com/oracle/oci-go-sdk/v63/database" + "github.com/oracle/oci-go-sdk/v63/workrequests" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/types" diff --git a/test/e2e/autonomousdatabase_controller_create_test.go b/test/e2e/autonomousdatabase_controller_create_test.go index 0bc0573c..feba24d3 100644 --- a/test/e2e/autonomousdatabase_controller_create_test.go +++ b/test/e2e/autonomousdatabase_controller_create_test.go @@ -42,7 +42,7 @@ import ( "context" "time" - "github.com/oracle/oci-go-sdk/v54/common" + "github.com/oracle/oci-go-sdk/v63/common" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/types" diff --git a/test/e2e/behavior/shared_behaviors.go b/test/e2e/behavior/shared_behaviors.go index 76a70522..b5497f52 100644 --- a/test/e2e/behavior/shared_behaviors.go +++ b/test/e2e/behavior/shared_behaviors.go @@ -47,8 +47,8 @@ import ( "github.com/onsi/ginkgo" "github.com/onsi/gomega" - "github.com/oracle/oci-go-sdk/v54/common" - "github.com/oracle/oci-go-sdk/v54/database" + "github.com/oracle/oci-go-sdk/v63/common" + "github.com/oracle/oci-go-sdk/v63/database" corev1 "k8s.io/api/core/v1" k8sErrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/types" diff --git a/test/e2e/suite_test.go b/test/e2e/suite_test.go index 3726049f..3d478e8b 100644 --- a/test/e2e/suite_test.go +++ b/test/e2e/suite_test.go @@ -48,8 +48,8 @@ import ( "github.com/onsi/ginkgo" "github.com/onsi/gomega" "github.com/onsi/gomega/gexec" - "github.com/oracle/oci-go-sdk/v54/common" - "github.com/oracle/oci-go-sdk/v54/database" + "github.com/oracle/oci-go-sdk/v63/common" + "github.com/oracle/oci-go-sdk/v63/database" corev1 "k8s.io/api/core/v1" "k8s.io/client-go/kubernetes/scheme" "k8s.io/client-go/rest" diff --git a/test/e2e/util/oci_config_util.go b/test/e2e/util/oci_config_util.go index f6f2abd9..84770a96 100644 --- a/test/e2e/util/oci_config_util.go +++ b/test/e2e/util/oci_config_util.go @@ -48,7 +48,7 @@ import ( "regexp" "strings" - "github.com/oracle/oci-go-sdk/v54/common" + "github.com/oracle/oci-go-sdk/v63/common" corev1 "k8s.io/api/core/v1" ) From ab2d86929ead47a16d8547968389d125bf26acf8 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Thu, 24 Mar 2022 22:24:00 -0400 Subject: [PATCH 205/628] update oci-go-sdk to v63 --- apis/database/v1alpha1/adbfamily_common_utils.go | 2 +- .../v1alpha1/autonomouscontainerdatabase_types.go | 2 +- apis/database/v1alpha1/autonomousdatabase_types.go | 2 +- .../v1alpha1/autonomousdatabasebackup_types.go | 4 ++-- .../v1alpha1/autonomousdatabaserestore_types.go | 4 ++-- commons/adb_family/utils.go | 2 +- commons/k8s/create.go | 4 ++-- commons/oci/containerdatabase.go | 4 ++-- commons/oci/database.go | 4 ++-- commons/oci/provider.go | 4 ++-- commons/oci/vault.go | 4 ++-- commons/oci/workrequest.go | 4 ++-- commons/sharding/scommon.go | 4 ++-- .../autonomouscontainerdatabase_controller.go | 2 +- controllers/database/autonomousdatabase_controller.go | 2 +- .../database/autonomousdatabaserestore_controller.go | 2 +- controllers/database/shardingdatabase_controller.go | 4 ++-- go.mod | 5 +++-- go.sum | 11 ++++++----- test/e2e/autonomousdatabase_controller_bind_test.go | 6 +++--- test/e2e/autonomousdatabase_controller_create_test.go | 2 +- test/e2e/behavior/shared_behaviors.go | 4 ++-- test/e2e/suite_test.go | 4 ++-- test/e2e/util/oci_config_util.go | 2 +- 24 files changed, 45 insertions(+), 43 deletions(-) diff --git a/apis/database/v1alpha1/adbfamily_common_utils.go b/apis/database/v1alpha1/adbfamily_common_utils.go index c3d505c7..b0c28f49 100644 --- a/apis/database/v1alpha1/adbfamily_common_utils.go +++ b/apis/database/v1alpha1/adbfamily_common_utils.go @@ -46,7 +46,7 @@ import ( metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "sigs.k8s.io/controller-runtime/pkg/client" - "github.com/oracle/oci-go-sdk/v54/common" + "github.com/oracle/oci-go-sdk/v63/common" ) // LastSuccessfulSpec is an annotation key which maps to the value of last successful spec diff --git a/apis/database/v1alpha1/autonomouscontainerdatabase_types.go b/apis/database/v1alpha1/autonomouscontainerdatabase_types.go index 9d4e456d..c5a928f8 100644 --- a/apis/database/v1alpha1/autonomouscontainerdatabase_types.go +++ b/apis/database/v1alpha1/autonomouscontainerdatabase_types.go @@ -42,7 +42,7 @@ import ( "encoding/json" "errors" - "github.com/oracle/oci-go-sdk/v54/database" + "github.com/oracle/oci-go-sdk/v63/database" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" ) diff --git a/apis/database/v1alpha1/autonomousdatabase_types.go b/apis/database/v1alpha1/autonomousdatabase_types.go index d2c9609c..c078aa73 100644 --- a/apis/database/v1alpha1/autonomousdatabase_types.go +++ b/apis/database/v1alpha1/autonomousdatabase_types.go @@ -42,7 +42,7 @@ import ( "encoding/json" "errors" - "github.com/oracle/oci-go-sdk/v54/database" + "github.com/oracle/oci-go-sdk/v63/database" metaV1 "k8s.io/apimachinery/pkg/apis/meta/v1" ) diff --git a/apis/database/v1alpha1/autonomousdatabasebackup_types.go b/apis/database/v1alpha1/autonomousdatabasebackup_types.go index 3ade554d..52113f4d 100644 --- a/apis/database/v1alpha1/autonomousdatabasebackup_types.go +++ b/apis/database/v1alpha1/autonomousdatabasebackup_types.go @@ -41,8 +41,8 @@ package v1alpha1 import ( metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" - "github.com/oracle/oci-go-sdk/v54/common" - "github.com/oracle/oci-go-sdk/v54/database" + "github.com/oracle/oci-go-sdk/v63/common" + "github.com/oracle/oci-go-sdk/v63/database" ) // EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! diff --git a/apis/database/v1alpha1/autonomousdatabaserestore_types.go b/apis/database/v1alpha1/autonomousdatabaserestore_types.go index cd9637f6..6f65ee9a 100644 --- a/apis/database/v1alpha1/autonomousdatabaserestore_types.go +++ b/apis/database/v1alpha1/autonomousdatabaserestore_types.go @@ -43,8 +43,8 @@ import ( metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" - "github.com/oracle/oci-go-sdk/v54/common" - "github.com/oracle/oci-go-sdk/v54/workrequests" + "github.com/oracle/oci-go-sdk/v63/common" + "github.com/oracle/oci-go-sdk/v63/workrequests" ) // EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! diff --git a/commons/adb_family/utils.go b/commons/adb_family/utils.go index 1650dd22..ed0e1efc 100644 --- a/commons/adb_family/utils.go +++ b/commons/adb_family/utils.go @@ -39,7 +39,7 @@ package adbfamily import ( - "github.com/oracle/oci-go-sdk/v54/common" + "github.com/oracle/oci-go-sdk/v63/common" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" "github.com/oracle/oracle-database-operator/commons/k8s" "sigs.k8s.io/controller-runtime/pkg/client" diff --git a/commons/k8s/create.go b/commons/k8s/create.go index fa53704c..ad522504 100644 --- a/commons/k8s/create.go +++ b/commons/k8s/create.go @@ -43,8 +43,8 @@ import ( dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" - "github.com/oracle/oci-go-sdk/v54/common" - "github.com/oracle/oci-go-sdk/v54/database" + "github.com/oracle/oci-go-sdk/v63/common" + "github.com/oracle/oci-go-sdk/v63/database" corev1 "k8s.io/api/core/v1" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "sigs.k8s.io/controller-runtime/pkg/client" diff --git a/commons/oci/containerdatabase.go b/commons/oci/containerdatabase.go index 359f8a4b..bd2c7429 100644 --- a/commons/oci/containerdatabase.go +++ b/commons/oci/containerdatabase.go @@ -41,8 +41,8 @@ package oci import ( "context" - "github.com/oracle/oci-go-sdk/v54/common" - "github.com/oracle/oci-go-sdk/v54/database" + "github.com/oracle/oci-go-sdk/v63/common" + "github.com/oracle/oci-go-sdk/v63/database" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" ) diff --git a/commons/oci/database.go b/commons/oci/database.go index 142e634f..bf4b9443 100644 --- a/commons/oci/database.go +++ b/commons/oci/database.go @@ -43,8 +43,8 @@ import ( "fmt" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v54/common" - "github.com/oracle/oci-go-sdk/v54/database" + "github.com/oracle/oci-go-sdk/v63/common" + "github.com/oracle/oci-go-sdk/v63/database" "sigs.k8s.io/controller-runtime/pkg/client" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" diff --git a/commons/oci/provider.go b/commons/oci/provider.go index c7592002..83dee0a0 100644 --- a/commons/oci/provider.go +++ b/commons/oci/provider.go @@ -41,8 +41,8 @@ package oci import ( "errors" - "github.com/oracle/oci-go-sdk/v54/common" - "github.com/oracle/oci-go-sdk/v54/common/auth" + "github.com/oracle/oci-go-sdk/v63/common" + "github.com/oracle/oci-go-sdk/v63/common/auth" "sigs.k8s.io/controller-runtime/pkg/client" diff --git a/commons/oci/vault.go b/commons/oci/vault.go index 71b162ba..bc72a9d3 100644 --- a/commons/oci/vault.go +++ b/commons/oci/vault.go @@ -43,8 +43,8 @@ import ( "encoding/base64" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v54/common" - "github.com/oracle/oci-go-sdk/v54/secrets" + "github.com/oracle/oci-go-sdk/v63/common" + "github.com/oracle/oci-go-sdk/v63/secrets" ) type VaultService interface { diff --git a/commons/oci/workrequest.go b/commons/oci/workrequest.go index cf05e506..0ac02911 100644 --- a/commons/oci/workrequest.go +++ b/commons/oci/workrequest.go @@ -46,8 +46,8 @@ import ( "github.com/go-logr/logr" "sigs.k8s.io/controller-runtime/pkg/client" - "github.com/oracle/oci-go-sdk/v54/common" - "github.com/oracle/oci-go-sdk/v54/workrequests" + "github.com/oracle/oci-go-sdk/v63/common" + "github.com/oracle/oci-go-sdk/v63/workrequests" ) type WorkRequestService interface { diff --git a/commons/sharding/scommon.go b/commons/sharding/scommon.go index bc9030ec..60e661da 100644 --- a/commons/sharding/scommon.go +++ b/commons/sharding/scommon.go @@ -48,8 +48,8 @@ import ( "strings" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v54/common" - "github.com/oracle/oci-go-sdk/v54/ons" + "github.com/oracle/oci-go-sdk/v63/common" + "github.com/oracle/oci-go-sdk/v63/ons" appsv1 "k8s.io/api/apps/v1" corev1 "k8s.io/api/core/v1" "k8s.io/apimachinery/pkg/api/errors" diff --git a/controllers/database/autonomouscontainerdatabase_controller.go b/controllers/database/autonomouscontainerdatabase_controller.go index 06050a3f..badd0b03 100644 --- a/controllers/database/autonomouscontainerdatabase_controller.go +++ b/controllers/database/autonomouscontainerdatabase_controller.go @@ -45,7 +45,7 @@ import ( "reflect" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v54/database" + "github.com/oracle/oci-go-sdk/v63/database" corev1 "k8s.io/api/core/v1" apiErrors "k8s.io/apimachinery/pkg/api/errors" diff --git a/controllers/database/autonomousdatabase_controller.go b/controllers/database/autonomousdatabase_controller.go index 13f586a7..7cc37315 100644 --- a/controllers/database/autonomousdatabase_controller.go +++ b/controllers/database/autonomousdatabase_controller.go @@ -48,7 +48,7 @@ import ( "strings" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v54/database" + "github.com/oracle/oci-go-sdk/v63/database" corev1 "k8s.io/api/core/v1" apiErrors "k8s.io/apimachinery/pkg/api/errors" diff --git a/controllers/database/autonomousdatabaserestore_controller.go b/controllers/database/autonomousdatabaserestore_controller.go index 96d8ea52..7ad1f591 100644 --- a/controllers/database/autonomousdatabaserestore_controller.go +++ b/controllers/database/autonomousdatabaserestore_controller.go @@ -55,7 +55,7 @@ import ( "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" "sigs.k8s.io/controller-runtime/pkg/predicate" - "github.com/oracle/oci-go-sdk/v54/common" + "github.com/oracle/oci-go-sdk/v63/common" databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" "github.com/oracle/oracle-database-operator/commons/adb_family" diff --git a/controllers/database/shardingdatabase_controller.go b/controllers/database/shardingdatabase_controller.go index aa37750d..ead408a7 100644 --- a/controllers/database/shardingdatabase_controller.go +++ b/controllers/database/shardingdatabase_controller.go @@ -47,8 +47,8 @@ import ( "time" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v54/common" - "github.com/oracle/oci-go-sdk/v54/ons" + "github.com/oracle/oci-go-sdk/v63/common" + "github.com/oracle/oci-go-sdk/v63/ons" appsv1 "k8s.io/api/apps/v1" corev1 "k8s.io/api/core/v1" "k8s.io/apimachinery/pkg/api/errors" diff --git a/go.mod b/go.mod index ac3012cd..df92e9e5 100644 --- a/go.mod +++ b/go.mod @@ -6,7 +6,7 @@ require ( github.com/go-logr/logr v1.2.2 github.com/onsi/ginkgo v1.16.5 github.com/onsi/gomega v1.18.1 - github.com/oracle/oci-go-sdk/v54 v54.0.0 + github.com/oracle/oci-go-sdk/v63 v63.0.0 gopkg.in/yaml.v2 v2.4.0 k8s.io/api v0.23.3 k8s.io/apimachinery v0.23.3 @@ -23,6 +23,7 @@ require ( github.com/evanphx/json-patch v5.6.0+incompatible // indirect github.com/fsnotify/fsnotify v1.5.1 // indirect github.com/go-logr/zapr v1.2.2 // indirect + github.com/gofrs/flock v0.8.1 // indirect github.com/gogo/protobuf v1.3.2 // indirect github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect github.com/golang/protobuf v1.5.2 // indirect @@ -49,7 +50,7 @@ require ( go.uber.org/zap v1.21.0 // indirect golang.org/x/net v0.0.0-20220127200216-cd36cc0744dd // indirect golang.org/x/oauth2 v0.0.0-20211104180415-d3ed0bb246c8 // indirect - golang.org/x/sys v0.0.0-20220207234003-57398862261d // indirect + golang.org/x/sys v0.0.0-20220209214540-3681064d5158 // indirect golang.org/x/term v0.0.0-20210927222741-03fcf44c2211 // indirect golang.org/x/text v0.3.7 // indirect golang.org/x/time v0.0.0-20211116232009-f0f3c7e86c11 // indirect diff --git a/go.sum b/go.sum index 0046074c..e2b44fb1 100644 --- a/go.sum +++ b/go.sum @@ -177,6 +177,8 @@ github.com/go-openapi/swag v0.19.14/go.mod h1:QYRuS/SOXUCsnplDa677K7+DxSOj6IPNl/ github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY= github.com/go-task/slim-sprig v0.0.0-20210107165309-348f09dbbbc0/go.mod h1:fyg7847qk6SyHyPtNmDHnmrv/HOrqktSC+C9fM+CJOE= github.com/godbus/dbus/v5 v5.0.4/go.mod h1:xhWf0FNVPg57R7Z0UbKHbJfkEywrmjJnf7w5xrFpKfA= +github.com/gofrs/flock v0.8.1 h1:+gYjHKf32LDeiEEFhQaotPbLuUXjY5ZqxKgXy7n59aw= +github.com/gofrs/flock v0.8.1/go.mod h1:F1TvTiK9OcQqauNUHlbJvyl9Qa1QvF/gOUDKA14jxHU= github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ= github.com/gogo/protobuf v1.2.1/go.mod h1:hp+jE20tsWTFYpLwKvXlhS1hjn+gTNwPg2I6zVXpSg4= github.com/gogo/protobuf v1.3.1/go.mod h1:SlYgWuQ5SjCEi6WLHjHCa1yvBfUnHcTbrrZtXPKa29o= @@ -393,8 +395,8 @@ github.com/onsi/gomega v1.17.0/go.mod h1:HnhC7FXeEQY45zxNK3PPoIUhzk/80Xly9PcubAl github.com/onsi/gomega v1.18.1 h1:M1GfJqGRrBrrGGsbxzV5dqM2U2ApXefZCQpkukxYRLE= github.com/onsi/gomega v1.18.1/go.mod h1:0q+aL8jAiMXy9hbwj2mr5GziHiwhAIQpFmmtT5hitRs= github.com/opentracing/opentracing-go v1.1.0/go.mod h1:UkNAQd3GIcIGf0SeVgPpRdFStlNbqXla1AfSYxPUl2o= -github.com/oracle/oci-go-sdk/v54 v54.0.0 h1:CDLjeSejv2aDpElAJrhKpi6zvT/zhZCZuXchUUZ+LS4= -github.com/oracle/oci-go-sdk/v54 v54.0.0/go.mod h1:+t+yvcFGVp+3ZnztnyxqXfQDsMlq8U25faBLa+mqCMc= +github.com/oracle/oci-go-sdk/v63 v63.0.0 h1:OOGCUmaDzrd5zTG8pljcnkR1ZHxg/991uEiQJi95/4E= +github.com/oracle/oci-go-sdk/v63 v63.0.0/go.mod h1:n6V9PcyRW5wtHaNd2TltbV3sWvbNy3PNqLmLrcT23Fg= github.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc= github.com/pelletier/go-toml v1.2.0/go.mod h1:5z9KED0ma1S8pY6P1sdut58dfprrGBbd/94hg7ilaic= github.com/pelletier/go-toml v1.9.3/go.mod h1:u1nR/EPcESfeI/szUZKdtJ0xRNbUoANCkoOuaOx1Y+c= @@ -452,7 +454,6 @@ github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1 github.com/smartystreets/goconvey v1.6.4/go.mod h1:syvi0/a8iFYH4r/RixwvyeAJjdLS9QV7WQ/tjFTllLA= github.com/soheilhy/cmux v0.1.4/go.mod h1:IM3LyeVVIOuxMH7sFAkER9+bJ4dT7Ms6E4xg4kGIyLM= github.com/soheilhy/cmux v0.1.5/go.mod h1:T7TcVDs9LWfQgPlPsdngu6I6QIoyIFZDDC6sNE1GqG0= -github.com/sony/gobreaker v0.4.2-0.20210216022020-dd874f9dd33b/go.mod h1:ZKptC7FHNvhBz7dN2LGjPVBz2sZJmc0/PkyDJOjmxWY= github.com/sony/gobreaker v0.5.0 h1:dRCvqm0P490vZPmy7ppEk2qCnCieBooFJ+YoXGYB+yg= github.com/sony/gobreaker v0.5.0/go.mod h1:ZKptC7FHNvhBz7dN2LGjPVBz2sZJmc0/PkyDJOjmxWY= github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA= @@ -737,8 +738,8 @@ golang.org/x/sys v0.0.0-20211124211545-fe61309f8881/go.mod h1:oPkhp1MJrh7nUepCBc golang.org/x/sys v0.0.0-20211210111614-af8b64212486/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20220114195835-da31bd327af9/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20220207234003-57398862261d h1:Bm7BNOQt2Qv7ZqysjeLjgCBanX+88Z/OtdvsrEv1Djc= -golang.org/x/sys v0.0.0-20220207234003-57398862261d/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20220209214540-3681064d5158 h1:rm+CHSpPEEW2IsXUib1ThaHIjuBVZjxNgSKmBLFfD4c= +golang.org/x/sys v0.0.0-20220209214540-3681064d5158/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8= golang.org/x/term v0.0.0-20210927222741-03fcf44c2211 h1:JGgROgKl9N8DuW20oFS5gxc+lE67/N3FcwmBPMe7ArY= diff --git a/test/e2e/autonomousdatabase_controller_bind_test.go b/test/e2e/autonomousdatabase_controller_bind_test.go index a028f711..5608fc91 100644 --- a/test/e2e/autonomousdatabase_controller_bind_test.go +++ b/test/e2e/autonomousdatabase_controller_bind_test.go @@ -42,9 +42,9 @@ import ( "context" "time" - "github.com/oracle/oci-go-sdk/v54/common" - "github.com/oracle/oci-go-sdk/v54/database" - "github.com/oracle/oci-go-sdk/v54/workrequests" + "github.com/oracle/oci-go-sdk/v63/common" + "github.com/oracle/oci-go-sdk/v63/database" + "github.com/oracle/oci-go-sdk/v63/workrequests" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/types" diff --git a/test/e2e/autonomousdatabase_controller_create_test.go b/test/e2e/autonomousdatabase_controller_create_test.go index 0bc0573c..feba24d3 100644 --- a/test/e2e/autonomousdatabase_controller_create_test.go +++ b/test/e2e/autonomousdatabase_controller_create_test.go @@ -42,7 +42,7 @@ import ( "context" "time" - "github.com/oracle/oci-go-sdk/v54/common" + "github.com/oracle/oci-go-sdk/v63/common" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/types" diff --git a/test/e2e/behavior/shared_behaviors.go b/test/e2e/behavior/shared_behaviors.go index 76a70522..b5497f52 100644 --- a/test/e2e/behavior/shared_behaviors.go +++ b/test/e2e/behavior/shared_behaviors.go @@ -47,8 +47,8 @@ import ( "github.com/onsi/ginkgo" "github.com/onsi/gomega" - "github.com/oracle/oci-go-sdk/v54/common" - "github.com/oracle/oci-go-sdk/v54/database" + "github.com/oracle/oci-go-sdk/v63/common" + "github.com/oracle/oci-go-sdk/v63/database" corev1 "k8s.io/api/core/v1" k8sErrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/types" diff --git a/test/e2e/suite_test.go b/test/e2e/suite_test.go index 3726049f..3d478e8b 100644 --- a/test/e2e/suite_test.go +++ b/test/e2e/suite_test.go @@ -48,8 +48,8 @@ import ( "github.com/onsi/ginkgo" "github.com/onsi/gomega" "github.com/onsi/gomega/gexec" - "github.com/oracle/oci-go-sdk/v54/common" - "github.com/oracle/oci-go-sdk/v54/database" + "github.com/oracle/oci-go-sdk/v63/common" + "github.com/oracle/oci-go-sdk/v63/database" corev1 "k8s.io/api/core/v1" "k8s.io/client-go/kubernetes/scheme" "k8s.io/client-go/rest" diff --git a/test/e2e/util/oci_config_util.go b/test/e2e/util/oci_config_util.go index f6f2abd9..84770a96 100644 --- a/test/e2e/util/oci_config_util.go +++ b/test/e2e/util/oci_config_util.go @@ -48,7 +48,7 @@ import ( "regexp" "strings" - "github.com/oracle/oci-go-sdk/v54/common" + "github.com/oracle/oci-go-sdk/v63/common" corev1 "k8s.io/api/core/v1" ) From 76874abb2c7433937c56ad2f7846bf71c72bafee Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Thu, 24 Mar 2022 22:27:47 -0400 Subject: [PATCH 206/628] update oci-go-sdk to v63 --- test/e2e/util/oci_db_request.go | 4 ++-- test/e2e/util/oci_work_request.go | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/test/e2e/util/oci_db_request.go b/test/e2e/util/oci_db_request.go index f02c2759..3d9b09cf 100644 --- a/test/e2e/util/oci_db_request.go +++ b/test/e2e/util/oci_db_request.go @@ -41,8 +41,8 @@ package e2eutil import ( "context" - "github.com/oracle/oci-go-sdk/v54/common" - "github.com/oracle/oci-go-sdk/v54/database" + "github.com/oracle/oci-go-sdk/v63/common" + "github.com/oracle/oci-go-sdk/v63/database" "time" ) diff --git a/test/e2e/util/oci_work_request.go b/test/e2e/util/oci_work_request.go index d15a1402..cc5ca82d 100644 --- a/test/e2e/util/oci_work_request.go +++ b/test/e2e/util/oci_work_request.go @@ -41,8 +41,8 @@ package e2eutil import ( "context" - "github.com/oracle/oci-go-sdk/v54/common" - "github.com/oracle/oci-go-sdk/v54/workrequests" + "github.com/oracle/oci-go-sdk/v63/common" + "github.com/oracle/oci-go-sdk/v63/workrequests" "time" ) From 5ed0f811bc4a22a13de0323c199692260e1c053b Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Fri, 25 Mar 2022 11:54:21 +0530 Subject: [PATCH 207/628] Fix logging --- controllers/database/singleinstancedatabase_controller.go | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 7b86b518..9b57708a 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -314,12 +314,11 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab eventReason := "Spec Error" var eventMsgs []string - r.Log.Info("Check Edition=", m.Spec.Edition) + r.Log.Info("Got", "edition", m.Spec.Edition) // Pre-built db if m.Spec.Persistence.AccessMode == "" { return requeueN, nil } - r.Log.Info("PBDB not identified") // If Express Edition , Ensure Replicas=1 if m.Spec.Edition == "express" && m.Spec.Replicas > 1 { From 1b8b5693ca1bcc913558833654e0a99c1418b002 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Fri, 25 Mar 2022 19:30:09 +0530 Subject: [PATCH 208/628] Defaults for XE --- apis/database/v1alpha1/oraclerestdataservice_types.go | 2 +- .../v1alpha1/singleinstancedatabase_webhook.go | 10 ++++++++++ .../database/singleinstancedatabase_controller.go | 8 +------- 3 files changed, 12 insertions(+), 8 deletions(-) diff --git a/apis/database/v1alpha1/oraclerestdataservice_types.go b/apis/database/v1alpha1/oraclerestdataservice_types.go index 19583579..79647f57 100644 --- a/apis/database/v1alpha1/oraclerestdataservice_types.go +++ b/apis/database/v1alpha1/oraclerestdataservice_types.go @@ -108,7 +108,7 @@ type OracleRestDataServiceStatus struct { ClusterDbApiUrl string `json:"clusterDbApiUrl,omitempty"` LoadBalancer string `json:"loadBalancer,omitempty"` DatabaseRef string `json:"databaseRef,omitempty"` - ServiceIP string `json:"serviceIP,omitempty,omitempty"` + ServiceIP string `json:"serviceIP,omitempty"` DatabaseActionsUrl string `json:"databaseActionsUrl,omitempty"` OrdsInstalled bool `json:"ordsInstalled,omitempty"` ApexConfigured bool `json:"apexConfigured,omitempty"` diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index ceecfea9..e4c56d3a 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -73,8 +73,18 @@ func (r *SingleInstanceDatabase) Default() { if r.Spec.Edition == "express" { r.Spec.Replicas = 1 + if r.Spec.Sid == "" { + r.Spec.Sid = "XE" + } } + if r.Spec.Pdbname == "" { + if r.Spec.Edition == "express" { + r.Spec.Pdbname = "XEPDB1" + } else { + r.Spec.Pdbname = "ORCLPDB1" + } + } // Pre-built db should have 1 replica only if r.Spec.Persistence.AccessMode == "" && r.Spec.Replicas > 1 { r.Spec.Replicas = 1 diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 9b57708a..587d327b 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1045,11 +1045,8 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex m.Status.PdbConnectString = dbcommons.ValueUnavailable m.Status.OemExpressUrl = dbcommons.ValueUnavailable - pdbName := "ORCLPDB1" + pdbName := strings.ToUpper(m.Spec.Pdbname) sid := m.Spec.Sid - if strings.ToUpper(sid) == "XE" { - pdbName = "XEPDB1" - } if m.Spec.Persistence.AccessMode == "" { sid, pdbName, m.Status.Edition = dbcommons.GetSidPdbEdition(r, r.Config, ctx, req) if sid == "" || pdbName == "" || m.Status.Edition == "" { @@ -1057,9 +1054,6 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex } } - if m.Spec.Pdbname != "" { - pdbName = strings.ToUpper(m.Spec.Pdbname) - } if m.Spec.LoadBalancer { m.Status.ClusterConnectString = svc.Name + "." + svc.Namespace + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/" + strings.ToUpper(sid) if len(svc.Status.LoadBalancer.Ingress) > 0 { From c6e2288f33830a76c3116d34be130180e9fb5652 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Mon, 28 Mar 2022 20:09:03 +0530 Subject: [PATCH 209/628] Fix prebuilt db validations --- .../v1alpha1/singleinstancedatabase_webhook.go | 11 +++-------- 1 file changed, 3 insertions(+), 8 deletions(-) diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index e4c56d3a..a2bbd4d2 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -105,22 +105,17 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { // Pre-built db if r.Spec.Persistence.AccessMode == "" { - if r.Spec.AdminPassword.SecretName != "" { - allErrs = append(allErrs, - field.Invalid(field.NewPath("spec").Child("adminPassword"), r.Spec.AdminPassword, - "cannot change password for prebuilt db")) - } - if r.Spec.Sid != "" { + if r.Spec.Sid != "" && r.Spec.Edition != "express" { allErrs = append(allErrs, field.Invalid(field.NewPath("spec").Child("sid"), r.Spec.Sid, "cannot change sid for prebuilt db")) } - if r.Spec.Pdbname != "" { + if r.Spec.Pdbname != "" && r.Spec.Edition != "express" { allErrs = append(allErrs, field.Invalid(field.NewPath("spec").Child("pdbName"), r.Spec.Pdbname, "cannot change pdbName for prebuilt db")) } - if r.Spec.CloneFrom != "" { + if r.Spec.CloneFrom != "" && r.Spec.Edition != "express" { allErrs = append(allErrs, field.Invalid(field.NewPath("spec").Child("cloneFrom"), r.Spec.CloneFrom, "cannot clone to create a prebuilt db")) From a92841a32ecf6cb36a38cf2a4aa433c038ced2db Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Tue, 29 Mar 2022 11:55:40 -0400 Subject: [PATCH 210/628] add documetation --- .../adb/autonomousdatabase_backup.yaml | 2 +- .../adb/autonomousdatabase_restore.yaml | 2 +- docs/adb/ADB_MANUAL_BACKUP.md | 51 +++++++++++++++++++ docs/adb/ADB_RESTORE.md | 51 +++++++++++++++++++ docs/adb/README.md | 9 ++-- 5 files changed, 109 insertions(+), 6 deletions(-) create mode 100644 docs/adb/ADB_MANUAL_BACKUP.md create mode 100644 docs/adb/ADB_RESTORE.md diff --git a/config/samples/adb/autonomousdatabase_backup.yaml b/config/samples/adb/autonomousdatabase_backup.yaml index f478faaf..1462d557 100644 --- a/config/samples/adb/autonomousdatabase_backup.yaml +++ b/config/samples/adb/autonomousdatabase_backup.yaml @@ -8,7 +8,7 @@ spec: target: k8sADB: name: autonomousdatabase-sample - # # Uncomment the below block if you use ADB OCID as the input of the target + # # Uncomment the below block if you use ADB OCID as the input of the target ADB # ociADB: # ocid: ocid1.autonomousdatabase... displayName: autonomousdatabasebackup-sample diff --git a/config/samples/adb/autonomousdatabase_restore.yaml b/config/samples/adb/autonomousdatabase_restore.yaml index 96e5bcc7..708a79ef 100644 --- a/config/samples/adb/autonomousdatabase_restore.yaml +++ b/config/samples/adb/autonomousdatabase_restore.yaml @@ -8,7 +8,7 @@ spec: target: k8sADB: name: autonomousdatabase-sample - # # Uncomment the below block if you use ADB OCID as the input of the target + # # Uncomment the below block if you use ADB OCID as the input of the target ADB # ociADB: # ocid: ocid1.autonomousdatabase... source: diff --git a/docs/adb/ADB_MANUAL_BACKUP.md b/docs/adb/ADB_MANUAL_BACKUP.md new file mode 100644 index 00000000..7b803e45 --- /dev/null +++ b/docs/adb/ADB_MANUAL_BACKUP.md @@ -0,0 +1,51 @@ +# Backing up an Autonomous Database Manually + +This document describes how to create manual backups of Autonomous Databases. + +Oracle Cloud Infrastructure automatically backs up your Autonomous Databases and retains these backups for 60 days. You can restore and recover your database to any point-in-time in this retention period. Automatic backups are full backups taken every 60 days and daily incremental backups. You can also create manual backups to supplement your automatic backups. Manual backups are stored in an Object Storage bucket that you create, and are retained for 60 days. For more information, please visit [this page](https://docs.oracle.com/en-us/iaas/Content/Database/Tasks/adbbackingup.htm). + +The operator gets the list of the `AutonomousBackupOCIDs` from OCI in every reconciliation loop, and creates the `AutonomousDatabaseBackup` resource automatically if the resource with the same `AutonomousBackupOCID` doesn't exist in the cluster. + +## Prerequisites + +You must create an Oracle Cloud Infrastructure Object Storage bucket to hold your Autonomous Database manual backups and configure your database to connect to it. Please follow the steps [in this page](https://docs.oracle.com/en-us/iaas/Content/Database/Tasks/adbbackingup.htm#creatingbucket) to finish the setup. This is a one-time operation. + +## Create Manual Backup + +Follow the steps to back up an Autonomous Database. + +1. Add the following fields to the AutonomousDatabaseBackup resource definition. An example `.yaml` file is available here: [`config/samples/adb/autonomousdatabase_backup.yaml`](./../../config/samples/adb/autonomousdatabase_backup.yaml) + | Attribute | Type | Description | Required? | + |----|----|----|----| + | `spec.target.k8sADB.name` | string | The name of custom resource of the target AutonomousDatabase. Choose either the `spec.target.k8sADB.name` or the `spec.target.ociADB.ocid`, but not both. | Conditional | + | `spec.target.ociADB.ocid` | string | The [OCID](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm) of the target AutonomousDatabase. Choose either the `spec.target.k8sADB.name` or the `spec.target.ociADB.ocid`, but not both. | Conditional | + | `spec.displayName` | string | The user-friendly name for the backup. The name does not have to be unique. | Yes | + | `spec.ociConfig` | dictionary | Not required when the Operator is authorized with [Instance Principal](./ADB_PREREQUISITES.md#authorized-with-instance-principal). Otherwise, you will need the values from the [Authorized with API Key Authentication](./ADB_PREREQUISITES.md#authorized-with-api-key-authentication) section. | Conditional | + | `spec.ociConfig.configMapName` | string | Name of the ConfigMap that holds the local OCI configuration | Conditional | + | `spec.ociConfig.secretName`| string | Name of the K8s Secret that holds the private key value | Conditional | + + ```yaml + --- + apiVersion: database.oracle.com/v1alpha1 + kind: AutonomousDatabaseBackup + metadata: + name: autonomousdatabasebackup-sample + spec: + target: + k8sADB: + name: autonomousdatabase-sample + # # Uncomment the below block if you use ADB OCID as the input of the target ADB + # ociADB: + # ocid: ocid1.autonomousdatabase... + displayName: autonomousdatabasebackup-sample + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey + ``` + +2. Apply the yaml: + + ```sh + kubectl apply -f config/samples/adb/autonomousdatabase_backup.yaml + autonomousdatabasebackup.database.oracle.com/autonomousdatabasebackup-sample created + ``` diff --git a/docs/adb/ADB_RESTORE.md b/docs/adb/ADB_RESTORE.md new file mode 100644 index 00000000..933287a8 --- /dev/null +++ b/docs/adb/ADB_RESTORE.md @@ -0,0 +1,51 @@ +# Restoring an Autonomous Database Manually + +This document describes how to restore an Autonomous Database from a backup. + +You can use any existing manual or automatic backup to restore your database, or you can restore and recover your database to any point in time in the 60-day retention period of your automatic backups. For point-in-time restores, you specify a timestamp, and your Autonomous Database decides which backup to use for the fastest restore. + +## Restore an Autonomous Database + +Follow the steps to restore an Autonomous Database from a backup or using point-in-time restore. + +1. Add the following fields to the AutonomousDatabaseBackup resource definition. An example `.yaml` file is available here: [`config/samples/adb/autonomousdatabase_restore.yaml`](./../../config/samples/adb/autonomousdatabase_restore.yaml) + | Attribute | Type | Description | Required? | + |----|----|----|----| + | `spec.target.k8sADB.name` | string | The name of custom resource of the target AutonomousDatabase. Choose either the `spec.target.k8sADB.name` or the `spec.target.ociADB.ocid`, but not both. | Conditional | + | `spec.target.ociADB.ocid` | string | The [OCID](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm) of the target AutonomousDatabase. Choose either the `spec.target.k8sADB.name` or the `spec.target.ociADB.ocid`, but not both. | Conditional | + | `spec.source.k8sADBBackup.name` | string | The name of custom resource of the AutonomousDatabaseBackup that you want to restore from. Choose either the `spec.source.k8sADBBackup.name` or the `spec.source.pointInTime.timestamp`, but not both. | Conditional | + | `spec.source.pointInTime.timestamp` | string | The timestamp to specify which time to restore the database to. Your Autonomous Database decides which backup to use for the fastest restore. The timestamp must follow this format: YYYY-MM-DD HH:MM:SS GMT. Choose either the `spec.source.k8sADBBackup.name` or the `spec.source.pointInTime.timestamp`, but not both. | Conditional | + | `spec.ociConfig` | dictionary | Not required when the Operator is authorized with [Instance Principal](./ADB_PREREQUISITES.md#authorized-with-instance-principal). Otherwise, you will need the values from the [Authorized with API Key Authentication](./ADB_PREREQUISITES.md#authorized-with-api-key-authentication) section. | Conditional | + | `spec.ociConfig.configMapName` | string | Name of the ConfigMap that holds the local OCI configuration | Conditional | + | `spec.ociConfig.secretName`| string | Name of the K8s Secret that holds the private key value | Conditional | + + ```yaml + --- + apiVersion: database.oracle.com/v1alpha1 + kind: AutonomousDatabaseRestore + metadata: + name: autonomousdatabaserestore-sample + spec: + target: + k8sADB: + name: autonomousdatabase-sample + # # Uncomment the below block if you use ADB OCID as the input of the target ADB + # ociADB: + # ocid: ocid1.autonomousdatabase... + source: + k8sADBBackup: + name: autonomousdatabasebackup-sample + # # Uncomment the following field to perform point-in-time restore + # pointInTime: + # timestamp: 2021-12-23 11:03:13 UTC + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey + ``` + +2. Apply the yaml: + + ```sh + kubectl apply -f config/samples/adb/autonomousdatabase_restore.yaml + autonomousdatabaserestore.database.oracle.com/autonomousdatabaserestore-sample created + ``` diff --git a/docs/adb/README.md b/docs/adb/README.md index 55778566..d304414e 100644 --- a/docs/adb/README.md +++ b/docs/adb/README.md @@ -28,7 +28,7 @@ To debug the Oracle Autonomous Databases with Oracle Database Operator, see [Deb ## Provision an Autonomous Database -Follow the steps to provision an Autonomous Database that will bind objects in your cluster. +Follow the steps to provision an Autonomous Database that will map objects in your cluster. 1. Get the `Compartment OCID`. @@ -103,6 +103,8 @@ Follow the steps to provision an Autonomous Database that will bind objects in y Other than provisioning a database, you can bind to an existing database in your cluster. +The operator also generates the `AutonomousBackup` custom resources if a database has backups. The operator syncs the `AutonomousBackups` in every reconciliation loop by getting the list of OCIDs of the AutonomousBackups from OCI, and then creates the `AutonomousDatabaseBackup` object automatically if it cannot find a resource that has the same `AutonomousBackupOCID` in the cluster. + 1. Clean up the resource you created in the earlier provision operation: ```sh @@ -386,7 +388,6 @@ Follow the steps to delete the resource and terminate the Autonomous Database. Now, you can verify that the database is in TERMINATING state on the Cloud Console. - ## Debugging and troubleshooting ### Show the details of the resource @@ -397,9 +398,9 @@ If you edit and re-apply the `.yaml` file, the Autonomous Database controller wi kubectl describe adb/autonomousdatabase-sample ``` -### The resource is in UNAVAILABLE state +### Check the logs of the pod where the operator deploys -If an error occurs during the operation, the `lifecycleState` of the resoursce changes to UNAVAILABLE. Follow the steps to check the logs. +Follow the steps to check the logs. 1. List the pod replicas From d3be19a8b3dd150d628906312d98d6b798cb1a1e Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Wed, 30 Mar 2022 16:09:13 -0400 Subject: [PATCH 211/628] update documentation --- .../acd/autonomouscontainerdatabase_bind.yaml | 9 +- ...scontainerdatabase_change_displayname.yaml | 16 ++ .../autonomouscontainerdatabase_create.yaml | 10 +- ...mouscontainerdatabase_delete_resource.yaml | 16 ++ ...uscontainerdatabase_restart_terminate.yaml | 8 +- .../acd/autonomouscontainerdatabase_sync.yaml | 8 +- .../adb/autonomousdatabase_backup.yaml | 4 + .../adb/autonomousdatabase_restore.yaml | 4 + docs/acd/README.md | 266 ++++++++++++++++++ docs/adb/README.md | 27 +- images/adb/acd-id-1.png | Bin 0 -> 90967 bytes images/adb/acd-id-2.png | Bin 0 -> 46632 bytes images/adb/aei-id-1.png | Bin 0 -> 63773 bytes images/adb/aei-id-2.png | Bin 0 -> 71307 bytes 14 files changed, 348 insertions(+), 20 deletions(-) create mode 100644 config/samples/acd/autonomouscontainerdatabase_change_displayname.yaml create mode 100644 config/samples/acd/autonomouscontainerdatabase_delete_resource.yaml create mode 100644 docs/acd/README.md create mode 100644 images/adb/acd-id-1.png create mode 100644 images/adb/acd-id-2.png create mode 100644 images/adb/aei-id-1.png create mode 100644 images/adb/aei-id-2.png diff --git a/config/samples/acd/autonomouscontainerdatabase_bind.yaml b/config/samples/acd/autonomouscontainerdatabase_bind.yaml index 75c88089..6a270f9b 100644 --- a/config/samples/acd/autonomouscontainerdatabase_bind.yaml +++ b/config/samples/acd/autonomouscontainerdatabase_bind.yaml @@ -1,11 +1,14 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# apiVersion: database.oracle.com/v1alpha1 kind: AutonomousContainerDatabase metadata: name: autonomouscontainerdatabase-sample spec: autonomousContainerDatabaseOCID: ocid1.autonomouscontainerdatabase... - # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. ociConfig: - configMapName: oci-cred-dedicated - secretName: oci-privatekey-dedicated + configMapName: oci-cred + secretName: oci-privatekey diff --git a/config/samples/acd/autonomouscontainerdatabase_change_displayname.yaml b/config/samples/acd/autonomouscontainerdatabase_change_displayname.yaml new file mode 100644 index 00000000..5e844ede --- /dev/null +++ b/config/samples/acd/autonomouscontainerdatabase_change_displayname.yaml @@ -0,0 +1,16 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: AutonomousContainerDatabase +metadata: + name: autonomouscontainerdatabase-sample +spec: + # Update compartmentOCID with your compartment OCID. + compartmentOCID: ocid1.compartment... OR ocid1.tenancy... + displayName: newACD + # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey diff --git a/config/samples/acd/autonomouscontainerdatabase_create.yaml b/config/samples/acd/autonomouscontainerdatabase_create.yaml index 93b8edcd..2889dd00 100644 --- a/config/samples/acd/autonomouscontainerdatabase_create.yaml +++ b/config/samples/acd/autonomouscontainerdatabase_create.yaml @@ -1,3 +1,7 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# apiVersion: database.oracle.com/v1alpha1 kind: AutonomousContainerDatabase metadata: @@ -5,8 +9,8 @@ metadata: spec: # Update compartmentOCID with your compartment OCID. compartmentOCID: ocid1.compartment... OR ocid1.tenancy... - displayName: newACD autonomousExadataVMClusterOCID: ocid1.autonomousexainfrastructure... + displayName: newACD # # An optional field for Database Patch model preference. Should be either RELEASE_UPDATES or RELEASE_UPDATE_REVISIONS # patchModel: RELEASE_UPDATES @@ -17,5 +21,5 @@ spec: # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. ociConfig: - configMapName: oci-cred-dedicated - secretName: oci-privatekey-dedicated + configMapName: oci-cred + secretName: oci-privatekey diff --git a/config/samples/acd/autonomouscontainerdatabase_delete_resource.yaml b/config/samples/acd/autonomouscontainerdatabase_delete_resource.yaml new file mode 100644 index 00000000..499ee21b --- /dev/null +++ b/config/samples/acd/autonomouscontainerdatabase_delete_resource.yaml @@ -0,0 +1,16 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: AutonomousContainerDatabase +metadata: + name: autonomouscontainerdatabase-sample +spec: + autonomousContainerDatabaseOCID: ocid1.autonomouscontainerdatabase... + # Delete this resource to terminate database after the changes applied + hardLink: true + # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey \ No newline at end of file diff --git a/config/samples/acd/autonomouscontainerdatabase_restart_terminate.yaml b/config/samples/acd/autonomouscontainerdatabase_restart_terminate.yaml index 73e241a9..436a6185 100644 --- a/config/samples/acd/autonomouscontainerdatabase_restart_terminate.yaml +++ b/config/samples/acd/autonomouscontainerdatabase_restart_terminate.yaml @@ -1,3 +1,7 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# apiVersion: database.oracle.com/v1alpha1 kind: AutonomousContainerDatabase metadata: @@ -9,5 +13,5 @@ spec: # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. ociConfig: - configMapName: oci-cred-dedicated - secretName: oci-privatekey-dedicated + configMapName: oci-cred + secretName: oci-privatekey diff --git a/config/samples/acd/autonomouscontainerdatabase_sync.yaml b/config/samples/acd/autonomouscontainerdatabase_sync.yaml index 290da77b..b758fe9d 100644 --- a/config/samples/acd/autonomouscontainerdatabase_sync.yaml +++ b/config/samples/acd/autonomouscontainerdatabase_sync.yaml @@ -1,3 +1,7 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# apiVersion: database.oracle.com/v1alpha1 kind: AutonomousContainerDatabase metadata: @@ -8,5 +12,5 @@ spec: # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. ociConfig: - configMapName: oci-cred-dedicated - secretName: oci-privatekey-dedicated + configMapName: oci-cred + secretName: oci-privatekey diff --git a/config/samples/adb/autonomousdatabase_backup.yaml b/config/samples/adb/autonomousdatabase_backup.yaml index 1462d557..750fa763 100644 --- a/config/samples/adb/autonomousdatabase_backup.yaml +++ b/config/samples/adb/autonomousdatabase_backup.yaml @@ -1,3 +1,7 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# apiVersion: database.oracle.com/v1alpha1 kind: AutonomousDatabaseBackup metadata: diff --git a/config/samples/adb/autonomousdatabase_restore.yaml b/config/samples/adb/autonomousdatabase_restore.yaml index 708a79ef..74c36061 100644 --- a/config/samples/adb/autonomousdatabase_restore.yaml +++ b/config/samples/adb/autonomousdatabase_restore.yaml @@ -1,3 +1,7 @@ +# +# Copyright (c) 2021, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# apiVersion: database.oracle.com/v1alpha1 kind: AutonomousDatabaseRestore metadata: diff --git a/docs/acd/README.md b/docs/acd/README.md new file mode 100644 index 00000000..5a7b9265 --- /dev/null +++ b/docs/acd/README.md @@ -0,0 +1,266 @@ +# Managing Oracle Autonomous Container Databases on Dedicated Exadata Infrastructure + +Before you use the Oracle Database Operator for Kubernetes (the operator), ensure your system meets all of the Oracle Autonomous Database (ADB) Prerequisites [ADB_PREREQUISITES](./../adb/ADB_PREREQUISITES.md). + +To interact with OCI services, either the cluster has to be authorized using Principal Instance, or using the API Key Authentication by specifying the configMap and the secret under the `ociConfig` field. + +## Required Permissions + +The opeartor must be given the required type of access in a policy written by an administrator to manage the Autonomous Container Databases. See [Create an Autonomous Container Database](https://docs.oracle.com/en-us/iaas/autonomous-database/doc/create-acd.html) for the required policies. + +The permission to view the workrequests is also required, so that the operator will update the resources when the work is done. See [Viewing Work Requests](https://docs.oracle.com/en-us/iaas/Content/ContEng/Tasks/contengviewingworkrequests.htm#contengviewingworkrequests) for sample work request policies. + +## Supported Features + +After the operator is deployed, choose either one of the following operations to create an `AutonomousContainerDatabase` custom resource for Oracle Autonomous Container Database in your cluster. + +* [Provision](#provision-an-autonomous-container-database) an Autonomous Container Database +* [Bind](#bind-to-an-existing-autonomous-container-database) to an existing Autonomous Container Database + +After you create the resource, you can use the operator to perform the following tasks: + +* [Change the display name](#change-the-display-name) of an Autonomous Container Database +* [Restart/Terminate](#restartterminate) an Autonomous Container Database +* [Sync](#sync-the-resource-manually) of an Autonomous Container Database manually +* [Delete the resource](#delete-the-resource) from the cluster + +## Provision an Autonomous Container Database + +Follow the steps to provision an Autonomous Database that will map objects in your cluster. + +1. Get the `Compartment OCID`. + + Login cloud console and click `Compartment`. + + ![compartment-1](/images/adb/compartment-1.png) + + Click on the compartment name where you want to create your database, and **copy** the `OCID` of the compartment. + + ![compartment-2](/images/adb/compartment-2.png) + +2. Get the `AutonomousExadataVMCluster OCID`. + + Login cloud console. Go to `Autonomous Database`, and click the `Autonomous Exadata VM Cluster` under the Dedicated Infrastructure. + + ![aei-1](/images/adb/adb-id-1.png) + + ![aei-2](/images/adb/aei-id-1.png) + + Copy the `OCID` of the Autonomous Exadata VM Cluster. + + ![aei-3](/images/adb/aei-id-2.png) + +3. Add the following fields to the AutonomousContainerDatabase resource definition. An example `.yaml` file is available here: [`config/samples/acd/autonomouscontainerdatabase_create.yaml`](./../../config/samples/acd/autonomouscontainerdatabase_create.yaml) + | Attribute | Type | Description | Required? | + |----|----|----|----| + | `spec.compartmentOCID` | string | The [OCID](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm) of the compartment of the Autonomous Container Database. | Yes | + | `spec.autonomousExadataVMClusterOCID` | string | The [OCID](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm) of the Autonomous Exadata Infrastructure. | Yes | + | `spec.displayName` | string | The user-friendly name for the Autonomous Container Database. The name does not have to be unique. | Yes | + | `spec.patchModel` | string | The Database Patch model preference. The following values are valid: RELEASE_UPDATES and RELEASE_UPDATE_REVISIONS. Currently, the Release Update Revision (RUR) maintenance type is not a selectable option. | No | + | `spec.freeformTags` | dictionary | Free-form tags for this resource. Each tag is a simple key-value pair with no predefined name, type, or namespace. For more information, see [Resource Tag](https://docs.cloud.oracle.com/Content/General/Concepts/resourcetags.htm).

Example:
`freeformTags:`
    `key1: value1`
    `key2: value2`| No | + | `spec.ociConfig` | dictionary | Not required when the Operator is authorized with [Instance Principal](./../adb/ADB_PREREQUISITES.md#authorized-with-instance-principal). Otherwise, you will need the values from the [Authorized with API Key Authentication](./../adb/ADB_PREREQUISITES.md#authorized-with-api-key-authentication) section. | Conditional | + | `spec.ociConfig.configMapName` | string | Name of the ConfigMap that holds the local OCI configuration | Conditional | + | `spec.ociConfig.secretName`| string | Name of the K8s Secret that holds the private key value | Conditional | + + ```yaml + --- + apiVersion: database.oracle.com/v1alpha1 + kind: AutonomousContainerDatabase + metadata: + name: autonomouscontainerdatabase-sample + spec: + compartmentOCID: ocid1.compartment... OR ocid1.tenancy... + autonomousExadataVMClusterOCID: ocid1.autonomousexainfrastructure... + displayName: newACD + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey + ``` + +4. Apply the yaml: + + ```sh + kubectl apply -f config/samples/acd/autonomouscontainerdatabase_create.yaml + autonomouscontainerdatabase.database.oracle.com/autonomouscontainerdatabase-sample created + ``` + +## Bind to an existing Autonomous Container Database + +Other than provisioning a container database, you can bind to an existing Autonomous Container Database in your cluster. + +1. Clean up the resource you created in the earlier provision operation: + + ```sh + kubectl delete adb/autonomouscontainerdatabase-sample + autonomouscontainerdatabase.database.oracle.com/autonomouscontainerdatabase-sample deleted + ``` + +2. Copy the `Autonomous Container Database OCID` from Cloud Console. + + ![acd-id-1](/images/adb/adb-id-1.png) + + ![acd-id-2](/images/adb/acd-id-1.png) + + ![acd-id-3](/images/adb/acd-id-2.png) + +3. Add the following fields to the AutonomousContainerDatabase resource definition. An example `.yaml` file is available here: [`config/samples/acd/autonomouscontainerdatabase_bind.yaml`](./../../config/samples/acd/autonomouscontainerdatabase_bind.yaml) + | Attribute | Type | Description | Required? | + |----|----|----|----| + | `spec.autonomousContainerDatabaseOCID` | string | The [OCID](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm) of the Autonomous Container Database you want to bind (create a reference) in your cluster. | Yes | + | `spec.ociConfig` | dictionary | Not required when the Operator is authorized with [Instance Principal](./ADB_PREREQUISITES.md#authorized-with-instance-principal). Otherwise, you will need the values from the [Authorized with API Key Authentication](./ADB_PREREQUISITES.md#authorized-with-api-key-authentication) section. | Conditional | + | `spec.ociConfig.configMapName` | string | Name of the ConfigMap that holds the local OCI configuration | Conditional | + | `spec.ociConfig.secretName`| string | Name of the K8s Secret that holds the private key value | Conditional | + + ```yaml + --- + apiVersion: database.oracle.com/v1alpha1 + kind: AutonomousContainerDatabase + metadata: + name: autonomouscontainerdatabase-sample + spec: + autonomousContainerDatabaseOCID: ocid1.autonomouscontainerdatabase... + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey + ``` + +4. Apply the yaml. + + ```sh + kubectl apply -f config/samples/acd/autonomouscontainerdatabase_bind.yaml + autonomouscontainerdatabase.database.oracle.com/autonomouscontainerdatabase-sample created + ``` + +## Change the display name + +> Note: this operation requires an `AutonomousContainerDatabase` object to be in your cluster. This example assumes the provision operation or the bind operation has been completed, and the operator is authorized with API Key Authentication. + +You can change the display name of the database by modifying the value of the `displayName`, as follows: + +1. An example YAML file is available here: [config/samples/acd/autonomouscontainerdatabase_change_displayname.yaml](./../../config/samples/acd/autonomouscontainerdatabase_change_displayname.yaml) + + ```yaml + --- + apiVersion: database.oracle.com/v1alpha1 + kind: AutonomousContainerDatabase + metadata: + name: autonomouscontainerdatabase-sample + spec: + compartmentOCID: ocid1.compartment... OR ocid1.tenancy... + displayName: newACD + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey + ``` + + * `displayNameName`: User-friendly name of the Autonomous Container Database. The name does not have to be unique. + +2. Apply the change using `kubectl`. + + ```sh + kubectl apply -f config/samples/acd/autonomouscontainerdatabase_change_displayname.yaml + autonomouscontainerdatabase.database.oracle.com/autonomouscontainerdatabase-sample configured + ``` + +## Restart/Terminate + +> Note: this operation requires an `AutonomousContainerDatabase` object to be in your cluster. This example assumes the provision operation or the bind operation has been done by the users and the operator is authorized with API Key Authentication. + +Users can restart/terminate a database using the `action` attribute. The value will be erased after the change is applied. +Here's a list of the values you can set for `action`: + +* `RESTART`: to restart the database +* `TERMINATE`: to terminate the database + +1. A sample .yaml file is available here: [config/samples/acd/autonomouscontainerdatabase_restart_terminate.yaml](./../../config/samples/acd/autonomouscontainerdatabase_restart_terminate.yaml) + + ```yaml + --- + apiVersion: database.oracle.com/v1alpha1 + kind: AutonomousContainerDatabase + metadata: + name: autonomouscontainerdatabase-sample + spec: + autonomousContainerDatabaseOCID: ocid1.autonomouscontainerdatabase... + # Change the action to "TERMINATE" to terminate the database + action: RESTART + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey + ``` + +2. Apply the change to stop the database. + + ```sh + kubectl apply -f config/samples/acd/autonomouscontainerdatabase_restart_terminate.yaml + autonomouscontainerdatabase.database.oracle.com/autonomouscontainerdatabase-sample configured + ``` + +## Sync the resource manually + +> Note: this operation requires an `AutonomousContainerDatabase` object to be in your cluster. This example assumes the provision operation or the bind operation has been done by the users and the operator is authorized with API Key Authentication. + +Users can sync the resource manually by setting the value of the `action` attribute to SYNC. The Operator may not response immediately if it is still waiting for a work to finish. + +1. A sample .yaml file is available here: [config/samples/acd/autonomouscontainerdatabase_sync.yaml](./../../config/samples/acd/autonomouscontainerdatabase_sync.yaml) + + ```yaml + --- + apiVersion: database.oracle.com/v1alpha1 + kind: AutonomousContainerDatabase + metadata: + name: autonomouscontainerdatabase-sample + spec: + autonomousContainerDatabaseOCID: ocid1.autonomouscontainerdatabase... + action: SYNC + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey + ``` + +2. Apply the change to stop the database. + + ```sh + kubectl apply -f config/samples/acd/autonomouscontainerdatabase_sync.yaml + autonomouscontainerdatabase.database.oracle.com/autonomouscontainerdatabase-sample configured + ``` + +## Delete the resource + +> Note: this operation requires an `AutonomousContainerDatabase` object to be in your cluster. This example assumes the provision operation or the bind operation has been done by the users and the operator is authorized with API Key Authentication. + +The `hardLink` defines the behavior when the resource is deleted from the cluster. If the `hardLink` is set to true, the Operator terminates the Autonomous Container Database in OCI when the resource is removed; otherwise, the Autonomous Container Database remains unchanged. By default the value is `false` if it is not explicitly specified. + +Follow the steps to delete the resource and terminate the Autonomous Container Database. + +1. Use the example [autonomouscontainerdatabase_delete_resource.yaml](./../../config/samples/acd/autonomouscontainerdatabase_delete_resource.yaml) which sets the attribute `hardLink` to true. + + ```yaml + --- + apiVersion: database.oracle.com/v1alpha1 + kind: AutonomousContainerDatabase + metadata: + name: autonomouscontainerdatabase-sample + spec: + autonomousContainerDatabaseOCID: ocid1.autonomouscontainerdatabase... + hardLink: true + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey + ``` + +2. Apply the yaml + + ```sh + kubectl apply -f config/samples/acd/autonomouscontainerdatabase_delete_resource.yaml + autonomouscontainerdatabase.database.oracle.com/autonomouscontainerdatabase-sample configured + ``` + +3. Delete the resource in your cluster + + ```sh + kubectl delete acd/autonomouscontainerdatabase-sample + autonomouscontainerdatabase.database.oracle.com/autonomouscontainerdatabase-sample deleted + ``` + +Now, you can verify that the Autonomous Container Database is in TERMINATING state. diff --git a/docs/adb/README.md b/docs/adb/README.md index d304414e..dd86015f 100644 --- a/docs/adb/README.md +++ b/docs/adb/README.md @@ -2,6 +2,8 @@ Before you use the Oracle Database Operator for Kubernetes (the operator), ensure your system meets all of the Oracle Autonomous Database (ADB) Prerequisites [ADB_PREREQUISITES](./ADB_PREREQUISITES.md). +To interact with OCI services, either the cluster has to be authorized using Principal Instance, or using the API Key Authentication by specifying the configMap and the secret under the `ociConfig` field. + ## Required Permissions The opeartor must be given the required type of access in a policy written by an administrator to manage the Autonomous Databases. See [Let database and fleet admins manage Autonomous Databases](https://docs.oracle.com/en-us/iaas/Content/Identity/Concepts/commonpolicies.htm#db-admins-manage-adb) for sample Autonomous Database policies. @@ -55,9 +57,9 @@ Follow the steps to provision an Autonomous Database that will map objects in yo | `spec.details.dbName` | string | The database name. The name must begin with an alphabetic character and can contain a maximum of 14 alphanumeric characters. Special characters are not permitted. The database name must be unique in the tenancy. | Yes | | `spec.details.displayName` | string | The user-friendly name for the Autonomous Database. The name does not have to be unique. | Yes | | `spec.details.cpuCoreCount` | int | The number of OCPU cores to be made available to the database. | Yes | - | `spec.details.adminPassword` | dictionary | The password for the ADMIN user. The password must be between 12 and 30 characters long, and must contain at least 1 uppercase, 1 lowercase, and 1 numeric character. It cannot contain the double quote symbol (") or the username "admin", regardless of casing.

Either `k8sSecretName` or `ociSecretOCID` must be provided. If both `k8sSecretName` and `ociSecretOCID` appear, the Operator reads the password from the K8s secret that `k8sSecretName` refers to. | Yes | - | `spec.details.adminPassword.k8sSecretName` | string | The **name** of the K8s Secret where you want to hold the password for the ADMIN user. | Conditional | - |`spec.details.adminPassword.ociSecretOCID` | string | The **[OCID](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm)** of the [OCI Secret](https://docs.oracle.com/en-us/iaas/Content/KeyManagement/Tasks/managingsecrets.htm) where you want to hold the password for the ADMIN user. | Conditional | + | `spec.details.adminPassword` | dictionary | The password for the ADMIN user. The password must be between 12 and 30 characters long, and must contain at least 1 uppercase, 1 lowercase, and 1 numeric character. It cannot contain the double quote symbol (") or the username "admin", regardless of casing.

Either `k8sSecret.name` or `ociSecret.ocid` must be provided. If both `k8sSecret.name` and `ociSecret.ocid` appear, the Operator reads the password from the K8s secret that `k8sSecret.name` refers to. | Yes | + | `spec.details.adminPassword.k8sSecret.name` | string | The **name** of the K8s Secret where you want to hold the password for the ADMIN user. | Conditional | + |`spec.details.adminPassword.ociSecret.ocid` | string | The **[OCID](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm)** of the [OCI Secret](https://docs.oracle.com/en-us/iaas/Content/KeyManagement/Tasks/managingsecrets.htm) where you want to hold the password for the ADMIN user. | Conditional | | `spec.details.dataStorageSizeInTBs` | int | The size, in terabytes, of the data volume that will be created and attached to the database. This storage can later be scaled up if needed. | Yes | | `spec.details.isAutoScalingEnabled` | boolean | Indicates if auto scaling is enabled for the Autonomous Database OCPU core count. The default value is `FALSE` | No | | `spec.details.isDedicated` | boolean | True if the database is on dedicated [Exadata infrastructure](https://docs.cloud.oracle.com/Content/Database/Concepts/adbddoverview.htm) | No | @@ -81,7 +83,8 @@ Follow the steps to provision an Autonomous Database that will map objects in yo displayName: NewADB cpuCoreCount: 1 adminPassword: - k8sSecretName: admin-password # use the name of the secret from step 2 + k8sSecret: + name: admin-password # use the name of the secret from step 2 dataStorageSizeInTBs: 1 ociConfig: configMapName: oci-cred @@ -101,9 +104,9 @@ Follow the steps to provision an Autonomous Database that will map objects in yo ## Bind to an existing Autonomous Database -Other than provisioning a database, you can bind to an existing database in your cluster. +Other than provisioning a database, you can create the custom resource using an existing Autonomous Database. -The operator also generates the `AutonomousBackup` custom resources if a database has backups. The operator syncs the `AutonomousBackups` in every reconciliation loop by getting the list of OCIDs of the AutonomousBackups from OCI, and then creates the `AutonomousDatabaseBackup` object automatically if it cannot find a resource that has the same `AutonomousBackupOCID` in the cluster. +The operator also generates the `AutonomousBackup` custom resources if a database already has backups. The operator syncs the `AutonomousBackups` in every reconciliation loop by getting the list of OCIDs of the AutonomousBackups from OCI, and then creates the `AutonomousDatabaseBackup` object automatically if it cannot find a resource that has the same `AutonomousBackupOCID` in the cluster. 1. Clean up the resource you created in the earlier provision operation: @@ -239,13 +242,14 @@ You can rename the database by changing the values of the `dbName` and `displayN details: autonomousDatabaseOCID: ocid1.autonomousdatabase... adminPassword: - k8sSecretName: new-admin-password + k8sSecret: + name: new-admin-password ociConfig: configMapName: oci-cred secretName: oci-privatekey ``` - * `adminPassword.k8sSecretName`: the **name** of the secret that you created in **step1**. + * `adminPassword.k8sSecret.name`: the **name** of the secret that you created in **step1**. 3. Apply the YAML. @@ -284,14 +288,15 @@ A client Wallet is required to connect to a shared Oracle Autonomous Database. U wallet: name: instance-wallet password: - k8sSecretName: instance-wallet-password + k8sSecret: + name: instance-wallet-password ociConfig: configMapName: oci-cred secretName: oci-privatekey ``` * `wallet.name`: the name of the new Secret where you want the downloaded Wallet to be stored. - * `wallet.password.k8sSecretName`: the **name** of the secret you created in **step1**. + * `wallet.password.k8sSecret.name`: the **name** of the secret you created in **step1**. 3. Apply the YAML @@ -398,6 +403,8 @@ If you edit and re-apply the `.yaml` file, the Autonomous Database controller wi kubectl describe adb/autonomousdatabase-sample ``` +If any error occurs during the reconciliation loop, the Operator reports the error using the resource's event stream, which shows up in kubectl describe output. + ### Check the logs of the pod where the operator deploys Follow the steps to check the logs. diff --git a/images/adb/acd-id-1.png b/images/adb/acd-id-1.png new file mode 100644 index 0000000000000000000000000000000000000000..c8e422d5d79cbe5d48e1397ed7da857c7112aeb2 GIT binary patch literal 90967 zcmZ^~1yo$kwy28+hXe@j?gV$&MnmK7?hZi{EV#Qn!KHBzPUG(G?hd)+-}~%y?s@lh zk2O~HteT~BR@GYdb?8?GDHKFJLHQ*y|L`smw#HSS+bX=4fjAsw2S45y(ygyp}Vq=K$R0tFPc&yz+Wp$x8yNrZEtxSug+rz2}d3()F=Jm~^|hUlLN|Y1uESaULSiin+DK z=5^tZet5^%f{zgX1tMb~>0mr#@FBaxVOTu{A$mX3G-hQmE7#!Eekb6C_iPEPn`u+& z!?jvlDvtj+5Tt_!k=nlxu7jAc3wBcbPJ*;F=?k-{N|zuFIlHuKveplkPs7s4 zQ#^ytH>C%k2ZLNyE?|Xr^k-{i0fr=$#*E4p`^fuCw6EYueCKz47>u9srOZ2T{ zrH;+7@e^a%x?T2Drcsa%mV<@J$C8fABefRe{JPUCRqFbsOq7MIMPI5iLiy1iI+Zal z7_RKp*fRC1nme+CL2;6pS&bp-B9B8Si&^34*L}EU+Ku~1dzfZ5id~AkQmRG$N|{b{;jN&R6Hs?GdEN${n$c#nBEnOzleHlTS1p~yD!VsE0Q&yHh% zoAcAa_=%uH_H{LFbm)De@xK~}2HsA?n%v!D-9!#TMw9M;mySX5qkonJz?sQHMdyfL z^_)N*STmKEb`YBq(1cD&C%n38^1i0^el>z;`GF(p(kp)fv4R%;W1K`|)q(SD`utSg z=eMya{U+olU2dd<^N#}Qw@l`IVSEsAQ7HHJ@=r%Ti)jqB{?}Yr3taOpsxH`j|7(W} zVeUHW1SbyPG9&ZXbZ$7Z2daHMfiQa5LgYLcC4vs(U#A79JO&bpllIs7cO1qfWh>va z&iD6y}YJ9AyVhyRg&^>|dI-G?nY!hbeQALCD z#y{vHG)3;zoa=&=-1*SvPYe&0Ye4Z65`hFZBE%D`=avC3zT4#s9A*Hp8nYbAx?8du z_5`A}TktC!>UZM}3T`;2A7vYyIzp!#JO@zrB4|HIC<3Ftd^GNbs*%X}#1$9`Ly{|v zi5^lwawv^OCXq1cV!{LrW00C7OOK`7QQskPLgonok#WU7OT%3uKK?YKLE#V{{mSQp zGc3ttL6PPkB~~{9ups3{(~U^VPds1(VF?6hiB;v<9yo*Wp;NH>dl^`;BO)DZa2P&= zdv_M(%3XExYdFg>p8YW#KaBm>-N1|L!f=EXM92Q*eFgG3jk4n>sd_^9idVnXCkYl zJ|pEs*A7JsU8O)DO4*USqV9`%DDVcdSfW^BAJQD6|H{0{Dwk*=X^rg}{AEJh{8LAA z4In7mA?_=BmU{rSq3uY(35#J(tPQM9oDyypwk*E+dRua*;%5cLkCKsN`enUgy>36D;o|h9b(_ND zIOA&LQsbO{5Qp}H&Jt-5`PFR~fEB#xk2U$rJxdjf z5GpHvFD?Zh$l>c~j%>ghE(#|iD>`l>&M8Za)y-szDS=gnaob4K*h3|P-IfXKl1c5* zhj+oxKFTEP9k%@Rgk(yh|98!)vSDi+-8PidFk-R|6m+^$cLPO_Hp7q3r{PRmZu zxhDvr2&s5fcy$~pcxwrr3Fi6qTtqYZGkX<2D`00RnKPS*UVq!4BMl@K6e|#;3(;}U zZFbN>)G5_HLo-E<#M5F2*&T!%a`pg281db#7O>S7=XK^QDjP$N(TI84EIfKH)a?}P zpc)Yy>Cas5zuaq{-JUTEWC(~8;q%*hu<}cJj@{c_y`1cxQr?$d9$hqCD%|Vs`()a7 zMVT@iW#5rLz&wv!@$6i8&x5(v-LucveVKfDeED9pqN<5W@oHRq#r){~I{ujbX@F1@ z{?=9G-yjSY_BAlw2p~BoF`Q_~p(c=0;55hHm#DU`=6?{r;HzYI|Io++gM@%ai(rW` zizbh(i~o&zgpPr}oVT2Qh0R;b!x)B?jMYf9*EI`Qv8!y$HS>$JoS~f{|6w$#y4z zrkKW4rH964W>AS(4h4K0A0mhl(ep{|qk)zwY|OHcm@jx$pqg+v@MTSi7+W;JO^ zfYarqwC2<@%d*dMzJD<~o%L+QBmS~^8%=Q{wJWbeZ z?f_c`J02wRJMlMaY^O!@0ieil?`ou2^G%b!mbdr&7OP`%AiXBNXDVTAJIk}Kg520$ z#@foxih$=%+f7>l(8n?gxlFMRD5G*6BHvxc{ArYtC%6tlv^Gvp zm>@)ZjtzsUg4vdx!S4A7vz>N|cE6ZcUYGk8V5qUlJbhvL!%}{lv3NYy*YAUQ|6c0g zNJOf9UAg&a`m7+I_gxgv3=fa<8;ujAmfnq--PKC4r`fgHxYvX7gW_!8Qs33q@%8JT zy^cR~41&Cfz1Bm)g}sGSh4V?U zOv^Po_MV$pA@q_JCAvdenzdJ>r3htX37!eII`X;|I;(0#s(PLMW}{Oa*>hp@dKK`M z3w2{ItG`{(=aA<0mp+1W&GHxfry_~D0Gz7}1Rc!{QT6q!)t4&6TgBYw=bL71)_PWyTd+@5&wTEz zpQ`5XB+jgt#@F<((@*V>QNN%j%#hV9Telwdy4v2--d4a#5!qN>SaV-IPpmf**Yc}4 zE?iv1k#bxu9Uui@St~VA5noy-i&n zcaA5kk%bU-n z_v1o8{T!ZiH#V=O!lbg7_63TBn7oWG3yyv*Pm2*65@KgZ`pmuQ+_e?gJutU-&b2#u zTrRJR_+_8tP0zjo-)iq?^XB`j8appNre_G3ea>yymb2U`&m-5*AG+Qgj*PDcs0HGM zVg(cZ++J2LD6hboYi(`_a26afQ3YdfPN1_zJDPg@gQLT2}3~0K;r)=41}cl zcN!=Nh!9H%n183ydcXhu#Jpec+W)>o#|1;cz5hagzXEfh{wp+uJf3 zn%EneGI`iK{LKe~--Gu(Xlv?XNaA5@W9Q84Awc#|3f}kdUokTo$v;V4tOdw4Bq1T;cQP^K1&T}lTmAi)0GWl0ivuq+v%9-HlRF!ey^}dJ3l9$u z^A}cTR#wLM6pYTEb}ohXfaIv(vBl(-Jp^?3-ivStf--7=0 z`ByznJuLsbBs=GSH|xED%ztB;S(v^s|G(5+EY1FZ)c(f&OYNU_{i`_sznSr>I-5F) z+1uKh+PMh+caQV`Q_}wx`QP>Ym*Q7T4^tZram#m2=l3=Vvao*P{I}Zwj{1M9YW}w> z7Z3OU)chYY|3mX{8+a9+EZ-|?__q@US@@a%KeGRw9%$-pZ{zwmxvHI|iy-U2RsNUw z-&*|4f5+v2jMKl2;2-I`6$m2oGylh;1QE@qd`90N)gb`lA}St`C+%?l1bsCRx>Ku% z3C`l8yp%EUWbhQMWML-AIC5X87RUD%&lmeQ7vHkj#&DL;?Tzu%N#a6`k z#6bXZ7!sVhcVCax>jaitx#}97ml73>@LkxZb#--Bb#-TFSL>!tp6^b770IU$pDfmy zj-)tFwYXX@b{MyM*D7O}3aOHPN)gxcP=Ie4wKD*pH+BO;5QAz1FG4}aPh ze7SsozP&^XLse-P_o6$ifSz`Txlm9mi@4sj>(9JgZ5V(|%qjN)jfg_t?PMX=YN{Z# zT)$IKtx|{f^XJcTGCbR;*N^B99sY4$^Ki$F$J5(OH{e$<;-r^?3qdCL^X-ev=SS3} zGVs0Os}pg5r~9pw;dYkK!_%3pM7T8($4hX@t>aSR0ge|sfS?Ujl7LR73*TmI>SV0s z^B$-3t5uHD4F37b0B7~#7q2W`r^mCj^z*FM+3x1|YE=2Awz{hmGu^LsJ5ciY{wUt4 z3fLT4Z7oAf_BwZHd7)XUj410lJmT?q&+Osem*;U3pqpwjL3O%RU*{)3lbXusJSwhE zzn?_;HAz?#$`m1ivU*>dq;L#`?&N64H*;(|?I-S>8-MG`0wkOos;YgER^T=0Z7FLOR5QH(qOTy-0=%b+_ZV^^G$sf3aIvDzUv&$1Zifs*pqz=cc7Eg5HxWa^UUR;%}2&8ohb^*D_p5;@ID>@Y0>xl@1PN7 z+DwM={Jx<+4!Tc}+bN6FxbRkm)a=~*<_pwf>*eAguhHmJa=7EPFDsv1 zl(U$ebKL6r6`3kyyn!%p7FrR^65X#9JLkx_U^yC z`J@KDT3G&<{SmkHC%Z)DfA#!7abBS?qHy!QM$7GiWvuxBQOWi|2t*x&YI6E`AFBSd z7GfBkMmC4xd%FZwZa^Idsa>6e{f3?WmZ|%llx9?`^3!eu_&o*5FBtyx&%E5)d2(ST zEIrBjlUp|mno)hR#<;S7%RIqDSQxs<`8Cl!%hR9STHFc!hRDc!g=WU z>UaS*whiL<&TT;yiC!6PVWi321bxHIl&6kO@q5_e^AsLi77;zMFoFmSr-^uiVZ}|H z+gJGx?16XO{7R371+XiC3+nH-)jF(oPF|Dv{Cq$@ihARaHB-bPdlv-A&*{CvTQ(N$ zK=^u=Ng(2zU??}(XX(MKeMKs=HrhkpKl^l8+`(lg>!MV+CKG>k`fCPCg*sxQg&Zj1 zN2em$m{T_An;!GM()KaAwg=PJ0rbp#3&a5WsTimtAyU7i3<6N8U#M zRanq+$|{QAmIvJjT7Y9RdPhRjwhM`6x+AM%7&U@7fz8wlaB3~ZVv*3B%9c$*4{+Jr zy}#!bk2>DHE4nLK{Og8pUsPNW8^1&3w-w2$67u5<&QbwPVeFGf-qO%$_>cQ*Z`PL= z9e$;8z-GP!CVdqJh`stx{8SuF$HxVudrMMVtf^?gB_@o%B5~j5Pw@RCO(8-2mIC7W zwGw<4>U%%B{;em$TZuJ+w@yd2vH8x{_U2m8`#ADRq(;Z|*LtT$_v=V{ay}F!=6#gd z`R=BeC=0_UF?y9yPSOV0+>Vjl1T0A@aa%Am2_Y~#Rx3Rg=_|peHI`Y1lIDvnDuJ-e zqVwQ~xn9UELq*Z!ldGchN)zIeO76l^u_?=pHWlfTp_WkyGbXd5zk9<>Xmqr|qAP#Y zpk1=0Q8=2VPX3Xyg!DabxKQvx^qB@xQAJNMcA&fUB{P>&zqQ%)@di8@r%ipvL+eeD zWwawTiJq8TRq?dpUKD-Qb2^<^746+5dX~QwIZO|uPHPY*4<5Tw5`5$56@xSHtUdj_ zQ%0=Wz~)N+sox|iFY_Pcz?mCf^rV zLq5O@ugy2=pk3DNf960q*{(Xx+yLtdd!=tb5z(c94?kjxzT;t>6@VzdD?)`^#QokM zUO~Ed4o4R{XJ8D)XD|u^PU*?4NMMRklme%+Fx`O<(!pT~sR>C17l1*zFg^rtbPWq8 z4t{wLPI?yylqW_RG$oNM1w`T?b_HR_rtOV%`Og~I-f-`Qxe~J1w6%h52 zwYHWRTzzRRqkA&N&j1GDv}N2&;ggcU6~~Lh1bKq@di8d5QSnGl#wDC_KNL7cZ%h2FFLD?Z+~wWnn+)nWb6&Pq;f`+cDj66JIuw$>_G?U-wpxP`&3mXoAvp^23D zht3i?hi#fN4y3(v0Ku2Cj2lImvo@Nz-vcB8^Zv^a5TEAfLvAl^FsLNdZ{ET-N%ikg zUq3k>lq6$)hKd_RQm07W&v!Qm$M}%=C9rptM}jlJi6JRgFOSEH>-yP6eUC&;gP`x^3OQB}1v zFq?Aj4oB>0+rLX4LyW=Z(&Oyl%_PP0?8by98lmyU(VA>~U6U%_tSVk@G;MeDLP z85CP_wDZ*l8#3310`D#pa-6_{gTJm5!Ie-Cl5k((^JFb8e3NdiV2wFOV zPL84c=uV|27IefTy7bP1dR0w{Oqu1#C$m~lXVbj4IMCcy{=YZZnmp50$};F~i9HB> zD@W5*2g`$w4`fcO=4w@U#fY0)G+FaT37SRVsyCdz<(EWYS3;7yVLyj_lTR~vX#n$` zw~_l;JxM2z>4KOWgIoy&4&a|areG}VsuoQ)_=y(W9t0cV`0yww_U1u z&!#P=JJPH;?fv!85%s@rVI0L+SP%wCP;-NgY1V!-gS* zX12EL9Rjjh%^*I0RO;LAfzF)hM0i_NPYz|=(R40FPTM7l=hMdQyGu3o)k;s#>8BSs z*5YhAQM2(ZMkH(|_AL5a&-JzTLi6#gT3y@FSz-9Y9gCF$c24CdI)1zw8aZH&LFL|R9R{bP<`ygD z{s_Y$l!Q{c;U9hxitxXtSi#fZ5|1XNXUc%kh=$f&cJcDsPyqpuC>QEhx<}!&QinKS zHb}cs$5@4&hPSt*dJCbHvGOVT4Iv~B9-=9{O)1_Gg{ISlT0$ftpW2$=9|6n7+QRbnYLT;!XFE^R6>faXC7enDMH%ACE3jClDpW6Jv0UVnA`hx==byOm|#8~Jm6!ulVt zEM)uzwYBs*wH8;4%3&o^XKecfPgzcx=&_2O(?j~5n)&3x;7INAv0;vM)4MH{tIMfK zR=Xp_#cIoRm-DZ-w|f?u5@0SFY#+PI;9A=M_tnP7SI>IsNBd8r?&+T`628kBFqvg&V;bNXZ&ey{tCzU>8fK zQ>_w1rS{X=ih2gkdVpX`quw_+J44pAmrcCI%3zDODX(RG`U3rDnZtGEO*K2zvBtm&u^6 z>2^9Hnm>HXx*4E4)o$z@zn@>l5T~9vr#35J-`0FpK|sGexFsLNw_B~?G!j$kWU%h$ zwDtXT*Y7Noks8&m>_#0dYtI;4ysndHJ0}x4#r>E+dzFlhB^xwWj&?AuWyNK${v?>B zUR*J8;1zN*-CQ`L3(~W46|Gfy-s@{j2JE|yf{061&y1XM{s`(+D;$<3%`ujlM*WUf z!z&)qV$@yYig&9hlkYI5VvgSjb$l3{(lw1Hhl7*+KzyUZnXsvj>X9El+Nv#Czr-PWYN?8{s6z07gxBnR(8a_ zTyI^BYFyY>a{4jR<#cK`g6q>feN~$gyW7Eph*rH-p^LZ07*ac>X!B{q@=jbhb1Z2Ha%>bK-!N$i0old8*fG>|Hym*ZhM%h1ukE8! z(6)(%_4_0A)tt6HE=v2iTh^9r?<>G!gKPNB;q<@`yCmTRUpzG!RaN>SU%+7s+j(y! zmh$59sB90J*X_uHeN(4D3YXq`t~@sD8Ab`Sx=gJi?$>nDP{;EfkOSFld3r6KN(xl~ zh{|*G@o>H>8yS~XTyoVhgurD&WU9edJ6uNJ@6}ba#;EXSp*ZC7_GVg9yK}lxJbo~^ z<2%*U$#%3*Lx*PjxB3a8)8$4korBQ`TsG5{rW&kW*P{i1R)dx0g+fPHgXOg3twTuL z>)i6yUN4vY{sg{>fo2C@P7?cD20)PA=Q-it-3Hg|a_tXogGwUXq51nVWq*l=T!xRt1Tut-=bj@ljG z%_wGE&c?zj2NC$DMAvJ0JohPQPDim2S> zgJ~ge{NB~D%t3M6F(a`5!NjhB3YLK}Rv(VU+063i**ewaRwp;VJJBgn_w~6m z{NwTKbr5WAu3OoQpSvGL$c;U{&*wX|{Qb1=-@IQ2Y7I0iGV)$84y(A?b^1F-9 zBuR9?rmw$9s+A}oueFD)cx*@F<+d>gVD5E1pT|?mCJ$bjf_L}Q?K!>05ybVmQ7!sH zW9P~>>5Vz!x_Q>FqP0yny1wrj5P4rfvaNADz2#1eGqul>8u~ZdtQp-hH}9ac-Cex& zVkpu?2AG_+HH-R0KtyFqmlP>R@@D?R0&JRe&q@*^9lkxE&tFceL~(Dyds^J|)H(%4 zNxABAqY!}YeXp(a(e^#SjE#X~ue&V9CI_FyOoz58S$L>140-Rer7l#bqy)nkb$;njh!`l*#hE!;O@*-hH2!I3A&7v-4!d zlVp<_6LZBvhQZ98CQoddgaYgt4qFIVlpO+xwz_NcEQJ_tPD`Wc@H0a{;R$uBFI7nl zE$y3BQRM4NHz0TCYm5N4Sz~i$E{nD8V}tXh+lx%#dHZS}p~Gc87ap2los{273PKu#v_@A@JJXLXy{+-LVMVvBj6g%bRXu2qV+3N4l{ zSKGVprL z+oRk@$d3FcLZ3X_SgYm;*qh!sY%6d#lJG=9lLv4?>f%T2=`vE08Np}sBB-myK?J-x z0TF)g%@GS#kr4!IVhTh{x=h@l_K5kP6*6chFYAtkk&Q@FH!9!K23IKYO-^oHuWqlO zhr3^wz>Ul&^onh*l1UXy3?cU=Nl4MhxlxMwbgt~w7UCQQesIkO9_G#(7;E~p186qn z<^=W85!~hJI-KS4ZS0n*lA?)II)u@(t6GO0IlQNpyqhWQ8p)0~Z+`We3byHoPTJMX zSY4&50QH`Yn9{^fE!Tn&I=|%Xyk`=+Y?)o-!=uruDk*E+xz->8cL!?MT2-gVU+p^^ zZ>Nk_D@D9D0BTlkke@xyq^1HtsBCB}kDPZHr?5tUobzh*`l)2Cgyky_lV;DFyv#RH zMZzNSv7AOe=y%Bw^COYh3kzFyw9s1)g!Ed=Te67HY(YhL4ID0!^tKGF&BnYPRI^*(fD1J~o7 zU3nCDKctsuSfBzTq;Yg^65A!-KI!^MdK!&K(8rShaX6sPbP7Yp;TBflyJZiz`1$ZN z4^{BNz#07_jKQHJBEO!uh9obDiQqAYGz2I6LY9!sgzR^N?aZTvr+H+N-rRxJ=}~?+RdT(o&3xNsH@w?OHlm*^Y9;wXr=V$*i=1vrZD~JY-^cT_7-6j-3tmvLWvea@ zPxnKOh4oU#t(w$8+-MrtaqY~rEY~t{>)mvR%PCvfi)FRo&X<8&zXG(H4O>T_gi<<1 z2vY_>Q@b`wpTw-S+!QGVkNslIOKo?Fm&!Hf~<^HUn;Hf)(%5+OVU zB_4g5m$i4bQcm!_>jr671Pr?Tkw3poWf&LX2ra2eXDV!&(amo{Nnaqb&jbJL&O++% zv<;V8U0!CgP2;NXq5T?c!(9L^NXC|&X4%ROjjoW^&PCQ_*AlT0RBzYUZ`DnX1p+gJ z3yEH~nyPqOC8LOD^s+LWbnHBxDjDWXXQ+5dYXsGC9-y*jbPoNqXpyYm7r&s9j7GIH z;?@PS1wF!mLcNUxxIA5g-IO8l?qg%p;?K93vT>c6>IZ0k)K3Hig54x7AXP%qXSdVUe3vOoE;61BRLRW1 z=?$V49H;#lhaVF#sM=dHQpy$DjZ<1Ti0p7K%E^V2$-IZPWS>5Hi{gjUgKD^LgRDCY zWK)>iZ}Rx>SDYm=K4j0RMY2UDJx%-;I(pw)i?O<&=|tLT;iDQ1nb)e9@o%1_7ZVCb zb@TX{KoL?TZLfK3qc}S-yt3p{@+>wg_nf0%kmw(g5~DtTfn2CHt;_Ja;~9QM)t<90 ztgY@qmWn>g-9jc{|F*DlF`g|Xq`@?Ikg{u*H#3&W7sOs`1v9$wE!k8>gk4TBpuSqZv?JF$+zVySC!KFVu<`fs z@8)3gs`~~_4y!Wr%9Tro1jk5HcnR}Y-nTF{VoPKNg2`V}E{Vl!+QqS8pwe<_-5KN6 zAEGDeG2ZG-7aMik59kZ}5n0SqC}$sH9*1+xqgkaKWwa9Z%d*Jvd_zHvAfzDP%& zVvipdN99)O{87jG~54 zJIBuLJ=W4{w5q=6QwI$7km*M0l(GAjKYRQ6K8WE?)TiDY&r14UOnBpZdQkHmb&b{9 zDOVRuW~$;hE_Y`?wn%-Owx-{4E|9jgv0&4_>ES@Bu6R zbC^$y3Yb>R^^erE?e0?rtA29Lxu@^<$j$Oo(2#~FcL1Fh z98GaI{iRy`GXp);!1s99mgN4xfI#CO1Ppn@Caz;Avg(uB+tcNFgD)iozm7^mu^jXm zLQ#nYb(orWfX=)O@mn|!RNU7+x4k<)_p9czjVFnl31(w26L~Ddn)M_i^q}*6ka2G; zY-kxDw`lqVAR&a|XUx<6x&5;W!{OFMgUzDVYmHrdR+>aJ$GfxZ5x&2NEon=EAaGxC z)Ux;IEjC?JXOs)Ru*3M|ETT5*;ZL;`qW(YqaD;3l&0>FLHe4KriU&~4bG3d z$w=VI3+bKq=qVPXUU=<>pz-Wa_Dp zK7u?}!6e7&=uAU5uGM{n47b=z@`M^?YPSWfcwTY9J~f3)5rR7R>-s6-%yJX?!;KXs z^I-=T96i;D4)s3K?5x=(_?DgQS*{zH!c<||t`#{kCeYBD38hoc=6BZ+jCy3_=Tlho`Xj*i;J9d-jBDS1BRmjs$Yyis5XF0UE$CBu*_}nf>Ys&j%(W1= z;roojp201iL`Cbh8MHqOsF)Zsq~6Xfsc&gX{ph;&kczqS)uH8P^gA4p#jAh(E<1Tt zY5+4gIRT%~RtDs)!^L?iTXxyx((^Z*4?NkfbBlAtH!P?TKf}FHK_q-+0vSjlmKu`2 zfcrs*g`07#Q{n?j>b;RvTWXV7QgE-ht+||Ip$o6OMRV;e`J&mZly#=Aq_jxaw=W3Zxz+3tZ@}(kba&v=G-yHM z*`FoW6HNgQ0W1$dA3luoGhT6E&2W-Ctc8gh#>=S#AX*nQYl3Im>c1`36yiCf@@@DN zXh7JI+=WD%Cu0Wj0Pcq8tU=$um)ogXQcoQSFR@ub+-4`4Qze(c-(UrmfcdQ9eLVq-b>55!#t5tG)9Bl{6_S2-*1C-i{UI~)@#x6Pm0AmpPO830?11{ zL_B0@55Zg+c=1RUWW;#QEe$c)4wA;EiJ2;}MEo~g>I3*rkZX5X^*$CR>sZ>eEv{uB zv05fzsfd_FmbC<`r+}vD;B;s=^=S2E)s9k_Fa~TFNWwvGZcH$401P^HT+f4HQ%+p+ zH<8Le=9IIrU@=+;fi^kYe(^|}+i6akGKCpv4-2>+aC`VztpPqwISrnL*bv*J7~g4a zgp446+Te{9^QINmRo7+Qmy-w`^>tQ8>aGEjTKAKgm~FY0I>it_MW3v08GR9?aRyx{ za~taqQIOtDBJtgH-Q1|h0U4-2P-B!Nb-iYU`s;3OyU=2pJTC~jN!lmP*ARN)TfGkg zh;N{DZj^<69JD^0W87utEmT(Lu1v|+N+akxy@*eqdW^IK(L97MBc}L5{9}6-V1FQa zG>YUVF6dJja=*`L2!V5>^22GWa38$87E1m1!~(Jj%Cq+^3YbC@q5A_SOEKW6kUKEQ zh|{1JNQ`dNhc=Mb$^wO`=o^SluPpLzp{2%l!rK>`V7Oo5>_v#-Y8~3Ne05Am-9a#d{A$>tA3Rw@OL55aIm@&VlE6DAz2XpREc^W0g%Zv=sH4OBtZ zVMso%Pg@UA+1B`)7di@&B2WvF26_8DO)fC|Pw>kdXUDmz-#;Q>w?suw+L!R_C|&HN z`r+J5DFF{2Xlnm;fw@RQ_A9N-)dy|V8YT8O#%3GA3*e4!(Vn?oUvP41r=AD0|K4>!B5?iY^^ zdu93~tWvTpj(b~l3FDcCg%fU0hvDDWG_~>^IQQB_O_Noah`bLrPUuVYx|tGXb0f}x zd3Ykmrp*-Rfxuw~+V{o*n&iXyXuZ+3;(jMth9+qk$H}q>wNi8a(&2xY=;pZ5c4!UR zb}IW1US^80lPr9&VN>VfF5&QK$&ma%fWR1yzl<5B3^PbJ;+IY?!}tF%G?EnW1kED) z?iyzIseIdsLJ%$3(Osr<#qBR!Mv>@mUMC7?E}H=O#9{W}#IBXGMhmXtc49E5{fn;Y zl0&`*7s>vGZ1|vX`BC-XtXVd;9eQ^MloDVBQ8_K&)+CW$dWT5D<|(QHj#BmNBW=Jb zq7XhyUzSHkOm%`t$J_WW>hFc^z#gef%y}~%U(BfB38D~cy3PcUKOe#yMKwy2teIMt z=cKmZ81)0=u$*B+xtkXe6dh+MO5ftdx5AG`#_wbL;)Cv(^bESZ|JEFhXG%QlIl0_c zoyl&QaAWq9uE81FSR8ZEZzfLf zfWTSdrF@*LM-~tOL5BSHQoKGs9UJz1|Iy^(vDqUpOBRKA)=WnasI_%rvdi^<(`RNw zjC-WCSwA+85A(SR93KS_3loh0;yylth?}$B?>(x3>8+C0L+lzE5DmU}e4oF)`B%J} z-p8_-=K9l=MiTS&WD1yO{6jVfh?)13f8y+yQ>a04iDp*Z=RmG^>T8OIAMg%_!JYQ) zQs+{la?-8e(3c8dyKykGn6T6Eh20t5Dq)H|2K!MkDoGn%UEM0FBm$?MY{ov@$D;IopAXQvGy*8elK?mo}qab-V!V;z-Ey zj@y&DIQ&Xz*my;B&U$KX94zkmD1-Pt8pF=#0jXP;ILat7IWDQ7DgHOM><2IWED9$% zsMf}zUxL0ewCR7v^8?FkKs}Xa6Xb92vs@69P{Xn|j5;y#AX%sbTWuMd^VQYgWYoDO z2~F`6lM4p-Cd~ z?cXF3Djofs!N8>QJ6Gp`O)nJIZ?viKgF3r|9Sd*!fwr)tw;tfjvJ;nZX9U6>Q2y&i z6r;?Pib_rKSE-nSrSc}|iP3asXdv^Zh^xre+RiQL5!)gUtE^Sw)L7PR7ZyU zMINTGfD+*hcrQh3Nvrh!cV&X(q(>Z@Pmr+p0Kp|hNp$EG%to}%)?M_TDZ8P&-M9q* zryUKMm1a;;;-IsVeYm~zRA?>V%|5=Q(RS?7_9PXMk6ibDNy_5gL zOcSF2412d=da8F!S(+8F5igkla1gWkYvBM#>L7bL5IfZpe!NZ&C4vCJAbyzOf#TN; zh@$)5afoCQ^sW8@kyqp2TYd-9@xElA%s!~yC;SFDU4R_!&&-*-p14I%$L!DxBod}| z^cux<(U#GQd*^cXKL(iQw4qDWT^C$<;@j^874C4*o>5!?4Dda%XJqwaJ#F9rNx*Kq zq(avdeCaGGs!1>;Gk>4>od60J;H5nDLcqs?5Xgv}8~Lt0H1{x2wdMo*77%Y)b>#mc;s}q9z>sd(?i-df@BOUZ9eHAsfD2DBg5sa@f3b;7Q z)792Mex(Z~Pn2A$e-PZ2+pc-#jBWy0oWy-A@kEJT9*_1p<&5tUP!Yr6nG&yL%nqht zl42Z>ub{jUjGbt)uIyWUR64KJ} zEQ`OT&ku2Yb#wh0J?M{?MAI`510Ug@?i?`r6nCueUV$bE@cd^B7!OK>QA>>CF8_Gh zge!}FBN4(_$dA~k8&i!p*xaKls`UkPY%rq5IN|RZ7OVF{?2o!0h{7qbW)Q7J@OXlP zK`1E+{ne)LkJ`O@+(tz|r5)37AaCxMcA2_T*Qe|UP!ur}H@TDuhY;!cV~aVajv z-CJCX7k3X@+$ru}+})wY-JM_sf;$8PU!He=d;dv}Op?hQGk0dL`&#E((WHvqU&ZK-MCGrrUU#*;CMgzU`7%{kr5V(*Q z>sjjdyHx#ys>Z2?45+?iw*^b6Bu^}vmm77(+Y6VKwdGVc)1g0P{u)Nf5cix3K>oZ& z&U1Ul4%pl4b#{aoYd_d66B`a7AsoQ@kyi~OlAafA)+`x)47yyk6{?8O!5|l2W&zH* zxE-kT7mhVt`rJw{KHT(4@89pp0!H(pNMnH4OXUCP9P^vjqyRO?na2Lo4%h>%GjsM9qGia@%Ct$-1NfZ(4dW7+X-&bu~nG-{tSyQa$>E zc_Eo6sfk?3?a0JkA4bDyS82~!dh`m2R7y53YRt{^kQ(UqT<$T*Vtdz!VHjr;UUNsk zv^ZqMz6;DziE4-piz0wb6Hiic0f3eS;uHvH>N&M@TWY0T!JdjE%oKWNk8%@Wl-?Wq z>VK9%X9X$$nMNvu*(Qmo=IP|L8W&S>+;kqBB!L34x2Nfjg?Xd@SUrKW1h+Ws%SW67HIKv zSDI45W?jqhK6V5-2Kc+8DhZQ}cSJSdT649_qA~^`W)vszFP(*@+da|+ruP0v9z9(* z8>jBbVx~!A3D6QhbQ}$3#ubg$qWCxMWSeR>6A{>~i|eK|G94>mZH@N0SAOBs!DaZS z&DYoi_j8uAfUJ+o(<$%*Hg+*@)d+s9-`iViXM!6lL&i&lB5P*iaB=U;2Bk$?rtY}i z+VEk9ht8rLT_v$#HMP3ClcjMyRZ*%u$U<2$47}axuv;OCBdyUJj!-M8G3?aJwolw}=+?aAf*8dtX_{`+<`TXv60&`utMBb301pBM+qRgT3`S2F!4RqpE{mup zn{|4QFHyfG!{fO89yyLz>djJz|D%ABDZRd^2wzDFygu$(^amrlq_7!6k7UKT1`m-q z>KyWF9JhORa*-Q~jni25E#04CDu9_aj3jM&k$QqDU*F;df8*FQ7I{pfUj02y~tlD%`%=`iy z71P-0)9%mK!|eqb+MM@f>|IO8Obd>!r*lOtVDOBBZ>1B)XPddNqH_^H#{aQ$sTF-P zzHru(>YhQYru8a_2U%=&dH-|jbU!I|0oR%}Ny>%5`3BgL9}#E6fIStR^Q3^6J;1+> z1d+rhTcWkqfaiF2pt?gZ)PqFGqe_5Z4NkI`-KcBDy7Xe$5qL^JpCn9Vk&u(-Ye4^yJgs zjWEn&)~fp8R_2+vHD|Bb=6m+i;sigkfQWV_4 z^$y^3RIt!+FkXfA6sBstLc>IJe>&ggKE9N%_Es@h-16gD+*?h%Ql~;w+c{*LZaMXn zJ4#-E^W+*LayeFwFc*f`coRGjz}_S%=+N`@ZvjRvCd~z9YrvPi{VStaN>70NWd|nC z1zFhB#KR<_V)7hw!L{}PSOZwI0%3!6vgzQ+!`omHRD9mHhvkq`0D7--DfHxWv5e`3 zdi(c+=*K)Zu$u)E^G~TggxMIrTLO5|{XofGxRB$yHEITiJ=P!gLeFaNtCACbVcB&C z{L8SaI&EGyYP5n3KqT{Wy>?NVv#!rO%b#uzS4F(AMIS+zuf}hXw8g!6wvbX4frDRl z7&@znxUNUxE=hOH^q8WR=X@bcVgu z^uGF3qWjZAxX?ZHIYfWh+G+|i$(JXp&K~xu89$7In_|gdZ{J>j>9sl-T4OIH5^q2s zw$nPEF8G$qU8hAJbe_R1gIuxrgT8jLiXC>O|AaDJyJ192^;@>R*UQx~=J_P5qu#e1 z^VOT$0O*Mt01D}SIbi>mmtBJ(DB^v+zfeC+ps1)X^7%|4T@Ak~- zqRQb@81rpEFt>IJH(7G$6xx+f+CAJ-&3N9+y12()@Bj^y6pklW#-L)16Vt$ZZcX#> zc*tc8H|=p!eDmGNkt`Ry%5zXtxrZj&PoqwcSdkl9D+8v@nHH;@{xY|))I`A4CuKaB zd^Qf+yscQW5TI#bJC~L4(L7yrTKcT31!y8=o&lNxq=nwq1~m6a1yL8w6B)XxD_B)9 zIol=2UvI=}l?}aieil%)84UyY zIsPe;gA9vU#^*tdzwn?Sx3xP~GuelB4aqPJ)ExtTnCW0;ph-DogjEnrUR4a*)O@46 z7|yrtH&0J_(fbBeDDoZey&Bripg>+4h4zJDDH-~VmXymosy5V5A%M}1AA|ehC4H4h z>)Bdewc6*!0ez?Xd2JMf}Q#rabm z%NGs0R_OcG#?)FJHfYm1jBPU+ryEHP_*E~}pVY4|OF_U^@>{)8bQaFBLFDQWo;{QJ zaM*Xt#NzB3g47Kdt*QiB`XBqR#*tHR`3uKSR1TOqrRPr5+M7lm2fOKoLv(X@lay06KdE zwYhl5NlmAnfhqG0YixrQX7&*{%)t3C0n%WeDom?@A#cZR*(_OZqZRGfV>Vw5yFa+2 z6vlCeTIgbs?~6c=%;>FBPiF$$Ep7O3x#XBwH3Q1H(C$%nakGAV?8BxSE%09!=<0u! z;WWP7hi$JBo3!(MsyB!H&i&r~a+8*2@7%Q%^sVXFEZ?Q-TRqFO*xQX5m-U-?p^6;r zvIwXg`WD+A_^iJTX^C5(Uz8!wrZl5w)o=RxD3+cBryWlz662WDwUR`on6lCdtBXi_ zfE^o=v2dw8?qK{*;7!(c(}9)mjM&j&-s>$xV|P}oUCUXSi4uHbA{AF=dD*A>!y|I9 z(IV^ueFk<*+k`j!)-B5;$Idg4qBLal%4D?0>9@P>H#$;^%p6I2Y~ja11DE5HFnA$_ zLUzNk?5}d3znB@1GIt8tygD$-f@!GE_?*Pt)h<>RGrAQ=EYklba~J}?BJLuPg&j1_tE4gH`Qd(TiHqfYdpQ)V zbu(a0D|gbRK_)^%L*5bOs0Q%PfF7E)jkyn8!rl-6Zp0GWnEmy=P9XTeC*$3FH_Q!gjX?Iu7 z`QXb*&$Cn-;lcH^zY1F$=)C|850|#;`A6>jhZHkP2FGIzoLO1R+0ISXhqHUh8F*R` zKGk;{vJmFxmyHLm92M2DSsi^ew7{dd$KfFN))e{ceL~pD0c(4LM0DH;B*SQIN`1?w zj=uNFA5lFQk4$Umjcz=may3T1R5{K4(Wx)K`z(=U380T_#xS%#1DLUvU7^;@tlKS z%l*Ws7c25eseWu>Q7pqzY%Yu@Vu|_vHr#=uFza<3>6dgf{*g%WYLWy2Nn)=_fmY1o zKI@!Lr1FT7hngh)4AJiehRl-i^Sf8w{-j`P{c%q|!pjdArG7yClI&9of?a63yWX~* z8!Ycgizs}rHX5f_rHK6;x=J}$|K8y>jM3g%zS1DBv9BO_6)EHLhSg6Zpp?k~aEL!5 z-;ay>&RSufZ+bGUFCLFrq#0UHz4v-eT;ZePY%yGGDqnJ-m>~Q+2}gSh5R9CsQJ*UI zeqr8YJqjG!*tY`+_8D<_;LC#DqM3tGAR%mtgtonJ5We2jydyYrzEPva}8RVX#sjGRt z)(-FHMf+;^{=YG}K^(pXHOUhoc%=1@7G|k%O&2}gRooFO9RIo+-y33NYOTo=dlYJ} zxktfI+Ps;&&CG((Fr!l&SCyNZEwVltQs_c95qOo4SsuT)41F2WZK%J%#UV!%FT_qD zifnNn`z6V63);v@bL1O!jU1VUd3#0|}rf0Q(Kf40@q2Z$rP z2THCU6RnwQyvMRMq2=viZv*+Qlb2yl&){Z(M`#HfZV_Rf{CCqUlFNxyILMK@ukisd zcu!ZVKx(m<9c<||R^>9swjHDlZqK;UsdwN0U6t3^<;=Qk|5_jsuv|Vu_s0-T(2u_ZQ zVK5G1PnlqQCHa;F!8_9)HF^+4kY94X?i{XK#wB zB4&HWV;WP^SJD2jls$R2-A;p56YDO0LC7OGvXoR}izs^;R(XlBWV~y{PIV;v{S+P{ zs7#6cm9jxEO_;s*nP0uBWevx;hh^-sQWJdc(T4E$Vj&|z3+1j1ol}{g*K63eSCNrb z3yK`G0B4MBxK72TKGLNz7LXHIl}5Cw$SpKiZOi9xd_f!H+7^uboOOpRs*Cy_88!1s zRs=FAZBXfUt>^=KeV*hk@T9o?i0?scEdEY{`W==1Vk9NgzUm!zR}bSLox=Qw3kH=c zOZc~s__&JSv+D+CcCOU*QkRghN$na1_)uK3oZp8y3r5mONC1)G-A#gdH*6&EGwFy- zIB^Ziqq(Y=4&*Qd;v$3vv0p&&(@5V&Y7B!jsD|~b@xDs1y?Exlt2)=$6>yvB!L;t4 zg9BO2%(oP6UrCd28{qE(&$?ix;>7?etzZrb_hlTfHiQ1&=%gE2SpLuJA$e}|eA|^F zzA-`XxpkL0c?L4l&4_u=h+w#}IgGMpj{F%c5tb)P?|qJBn>RRUb=)=sY&zUjBL7(O zmg@sy%9@26zZ+vM8 z6YCE+3b>6<_Up)_8m90x;CDzb{_vO68T5;Cmea+Z0pjA$svkp=gL(t+jsj?R=Fg}4 zG^-k9guF*u#cu&}eNM0n_)JaKDvRxQL{pa5AM)CXfF*KkNk0m6`SG{1hOaN8=gkWb zU74MgdSjk2;tnZr%z6?x);3M2e?NuwU8be5r);D+CAXI3FJS)wGk8kxAvxcGM0tvq zkutbVt1c%!-G@9oCOjGy(Q9{9!HP_+5s6r9$lZyRVjIgcq)Bk)H}m7m{(QnHHbg=f zO;c|o!zz}mvZB)N;1pImEPTsYy_OUi9R~+fVl|+~QJ&9erjwn3#Ul{gKLw1bI=l&F z{kyKrL=2j|VjyRKdk%T~{G>i4b_>sd?D*N=R^oJjG4i7+?WE37|LjSrhUw-}R?~MG zBBT&Rh9I~sLHi1F?>~uzbYEyGrg|pLzo9Ih;RvItD4dwrEhNG!2on@P4mo7sCghZu zv_6}laKUqh&C*XR`8EGKWZJ$-i3>FLPIMAz>Fy}ap7jrxH^$x01#9xEks4+KWQtPD zpE6}hxH;$elf#MMj*FZKzCbPjeT@oyY|!(lEZtZJr#>6m(0+ zL*2i$W#o>kJ_m)GlGlb^@#Ki$!tH{dT49D^o6U-n$;=>EKWA0Iu9Q38<~>gYZ)Igx z7M_`^`S_Aa``!oW;!o)P_Vm)osu^_L_?7yt{F!b+WIsXDK4BM>A0EJB%K2pSGfdO8G&UOMk$|V3R>D)FJ)`zubM(KVAaflL6No1B$ij5fWHu$0My^O;J+49~hC)-UWH-3=UH7Pd5zIF(zQ294$oiodIP|2a#mu?kJ)6%>6%q^Zn#V!Z;(*u9{ z?*A=b%Ogj)j1!0@^hbEQn5VS@%LEG@ z^~;)>HBp94o@4*ej+DsXQ;cjSqkfZqPXUul63|MbU3?#uE{^#vX))?-G|O~_eEzJ) zAmRXU7(#-?d9-uhKT6Qe$~aH(!{IjKJ2~eFS=Dq03?@xyfj_RY0C=neOh?}t>C8D% zOhvD=e&waTmk=v@c7$Q_EiWiF@s|GP{8`7)bWSt7(Ro&EO=JUNMy&L@FqKV8k&dj5 zQX99aAc{gPk$x-tM!u0PCcooZx8seH#!(z##w;$u98a zm^-!*hLB(?=4n^-fTUz%NlW6+tk)x~$}zi3RrSGQZq2YR>~*!Hkrj^6P}qV}+NQkJ z3N5I@JtL8Eo7EHIV}X_()`$iPkJu;MZkwW~=NCx5c`pp-fWa9_fWqf(zg5`YXrV3XCR=?uj@suG;?hQswh%P^C?t%aYr{le})K@wD1+q0&n-iPkrG%7YkbAQa&`xLwQ z*CPqCupq}yAsf(hOm7QM@;*K%&@r`H{`^wS8V^Us;5v$&xGAA;GDr+72sUj?DR6c5 z62R*h7q(-mqb;+g)QdiHJu8P}{xB;FRqi#j?J316$W8x+cynmAv$u2c_So5P5|G%w zYTs>n+h1W0MWrA&Ok5<=HGzN}yZ)tmPj~Uie}_++)WL0hhasYk`$Q}d(?!|bj1r&m zE_fe6q4GVjQA>IEVU_>34~ZvqY8F!)OL^wWXMZE65zZ%IsWc5kHk)69(htL-*8tq&6>zkUwAL{!}y>i!OeQ~kB4lsHHwiB%5-In;Gi0FcrSAKbGTf!olH7b*fQc_T`H)jG7X*{Uxg=gr z>7yKb&tC5P>`BNg$5SWHS# zDO=_!-9;VkF(27KM!Dxs37(URr?a1bF7$>vmft2+z~}Lghe?8}nw5g!MVIP&idgu` z(>vTfbR8|7I(6fv`6}AS{zy<=?_IScDg&A`V;`4HWX*-9Zlc#LG0CjslubfB3$bKZ z`cwfwZZdG_kHMyPiE<2|6}%HrNsV7XQA+ZS+u`;b%1IvJ1%yoKtck4u81VKGPlcW8KEz%`lXXed^Lz z)O^4>CCZrSi%*d4WkIr~Y**6*{!}?tjA9%pJR|+`C&1`Aucc{Rie$5*V(wi@yWk$< zpE%QRK*hDbscz)q=m?0a&^P0VD&?#1`J_g^9_&mN!8-asj;hFYwhr2!YdX(zqO^_u zKbM3F`w4hYf)|;#ayCWeyzsT$kJUK~t*LITufBX>Xa6grcDPC;wq~hq^&P`p#2ca&A?~g( z=SOq>S#xx1wu;!H#a;oZq+ZJi?cn?tqN`&w3Ue(P<4xD;O(rilf6VRuL2mcO(Xd;1 z%}cRcW>h~W(`c#a6G@{OPxcCM@xZ)^>hRz^5SN}mPMPzApggLaLq2an&hiXX-B(ow&kUupBGt< zufB|UM-@el6+-*TUek)$=F+=xMC=_jj;{qY2K~ccF8+JQ-nB{$yG({MpI>}-XF%uB z!|wYSm8!UcXw8+fP#ltVNe!E7xEPjr2Jgk>9PsO z2eO4jsauDOP&@_O>*m99E=*HsQ&H1d898}Rb}O1zCMHR>(m{R5+zBTBM8uXJwA6&H zer030uWZ>|76<3?lhpH#px+%*K$c;SXIJ-vLFt61W6Q3)T@4*vrv<7^Q~DEdrE@(A zMO*vw6pHudIw!3HJVOt4aw-)W5z{;+SD-^{(-)DokL|i7Fsy}5dzgk}!QJf|{_!cv30O1AM7il-C!!G26`w!NQs4gAP<6btnWp z`6uD1;>FI0He|TL+_d&u*pyrFh{*nbcIfWgbW3UDp|dI!LK(S%S&Z=!Aw3qnO86=P zv+=)hR@=WJr{=6$egBN;bsJl9VRNC(O|hzqP1??h2EAoX6k|yBW{p7b2U+UB-#{k< zR!tM!cc8ym?q~`A#?EQ7z55ogGeoDiqYd5(hmu5Ts2O+6(BjfotX{qf=$GyNubC?vg@li^a`fiWST=O<^ack6D0Q)cos8~ z#oshF&3M9_Ry8-pYLu}`sqk9R^9gzq1?+EnNPR$VL4qUc0XOcb@uCbq=H4kS$QZ;X zP%|?v8-rSA`uW>^+gc-$vtp{3q$5c!%^6GxAVV!Iroi;PtG(sx_$Q@{E&rx&`cmv$ zbV$Dz#s%O(OE7Z7%RiJAMT;n+O-DTcA_0{b?zL_?o0LI7G#@XUWzcHOty94rUU-2< z!qU5~)G_uL&j+D;B+#?;nA?1@iUe7W3j*?sIRtSroX14vyujnl!s8=}Q~sLw?&mb& z_2i}!aqr)`L{K2Z@F6p=*r2tQysA%2UeB#WCr7Jg41rTHtPc6`Uav1p7q@_Nba)uy zomY}(o1mL@Mo|XWIYrgr-$thLmSM_*qQMWo0pyr$<02u$JHbk+ZPwA|7fK}uGGrjj zxuhWMdmhuwNtd*XwU5T!v7&rmg08&ZGq~v+_9=(?6h#81rs8C;$v00wOGF#XJTZut zSXe_#2EK^imMidT@znIfImxU|(`=7!-qnZDY%9wJ(%5T;mu%YEo0+)cxo4|8Thxf$ z(k2IUM$FScSxoJin}l>qI3LbUK6}Zyk(%O*jBrH(DohM=CcKfv__Mo4X!=`|)Aks! z2)#@gr~gXoI*iEz>Y9|VorALWtsBB`VR^7a4oG>5MJEAYPErw}sz5d)SxE;`FTd4G zjEfq8LHnwpoqETH!!5RfiAs?>UB`9;(Ob5#f&TvLj+o=fJIVEekikddgd!6nBf~O* zffHNOM+wtLr}D2wyw2yTI$A2RT%PxyO+JNQ=;lB=@(gpn;| z%(Zf^B|pL;F-3aDe@7$_T>GfaT#_6ON*8UI93g^FokhG{`|=<-*^DUYHndEpZRwqx z3*m6smGa+~zf^MJcNsV6S&wx#t5@5&M2Ry1UN@JYG*~Ld`fo-z^ESV47*{SJCrJ0j zwHN5{2a>aJfK)=36B!PkaFa=Tdjs#*<0?4_e5!P6kMsde@bl+*7h%NJZ=f9< zs{TeGKIFtY!~2vtG|4-J5Z&2mQ+wh`4@Z0>flxHz%2EDN=aN8N$A-ZqmT<$tIg?F_ zno6numRU+?TDivBHLs|GIYz$_!w}$_^@qtO1qQ~W{P=(L^|c~e9`g-t2y3gu`OVZ^ zQnaJCXj;MRcEO6UN7HNbAIY(mfkO?SVBBs+HZTHD4VLTp z$hhI`E$=+s&+r$32nZL7oBltGV0o$oL?Nkdyox!-3Tt~#42izDE8oXH#laxdM0`(j zKPC&ja3LmE=gV06IQ1sl5*V{vPF`d!fB2*viWS#9|EE_vqbxCr;Aw$qPLAr*d7v6Q zfU50eP`&CJ7Txt9toeP^#BnfLDo)wuXX*g=uc@e;@A|~fA@iQ zt*t0EqM`}XxI32S+JJxi2mkSvsuJoj5VWAEzW;pGg=yZ1+Iombz5*H3ZvEX;v`d@+MLGszAkK9fuKrt<_5{211(b z|37j&US{ma~=-SC*7-Ot_6fuzeE0qyQnnWb6D0wNh7&(yhYr6PS-ChE=X50DgS4cFVtabncKZIeoiqSDvqH|e)V3Etzq3e-|7|5k#-qIq z$%MrEHBYkzZwd|vr77R&6+4mfvVVq%)H_3sv^j{Bi1vYv0nur(SleJhGb6%pC3e(obp)H zIu6`j{rq124M$8;572qU{CHWPOeyZ}P9ln!Df8YWszcaD32>t_KG@eg6DIdaIDe+6 zRFGF>awhs@i@cm}?aU)>C?YsW)P2^<82wnthtwDIZ8F?i zKC1Ig8Pb!$&aoyz3ZyRW{l=!Wx8YJSJ-2+a!DDE_WJ{@pjeH>x88a95y~1auO}=pdXi`U$o+F z3=i`2^r%=osxFV0JJ3oU{FI0*FvPKP5|ZK49^=AmQ+l1+xT?hHm$$5Aoc-iQCV8Ju zsfcO)F8gI1S`s1m^(j5BYKEU%QumYhy)EHu9~<>(lV*Yey7fFH60k03OG$gd<8Y=5 zEQs1p!oX033+2&BDE2uux}DW(yqdPp(k5;QQy0i)up(WYrg~n-{HL6-MC&D z3bkK~I*e_&viaeQrX7M0c0Hc3YCdc%z-={OY-`Fy4IExGdb+nWv~w+sdbdC9Cl;EF zdR_{J2x0XOo~`YGac|prU2R;SL|GoNj29OBT_*p^<_;dWDvFLyk1v!UiIp7yk*B{z zUzUmby*i>2u93_1L&HoY!9;5+MM3jZoVhRyC#IqB&ECwpr#ap%`0J-qTz^9T`IlY_ zQcV%&tWl*JN!MO@o8%6bqXB>fY5kc>EPCx0<43Z>VU=8%B`5Av(uXgiwo}5ojUL@! zk88!&u>(t`2}-!x;wu$v*bOqqJvr3>A*b_uX>aeu2H)L~Q+nQwle6U!rXAJ*MbZw9 zT~B2C4+r*_rVoC)Fr;m8y$KM-R~}w_rV?KAlb?f~h7JAO+I+RsT+&>gu1^Gs|NEZk zgF3$fP>W*G7MR{<7T%&U->Ln$w}fRZS<^I$ZG3Zb36aNt-hJ=&SP0+Sw^gca``M4+ zK9-+87AQ54?jAh#IgS`HhOC!#TGs37`#bNx=SH72*{qk+yqyK#AtyGV2_~gUH}Vj) zw8+@(DjyICI*&)42lSRrrJy><(bzNE)zn_eIk9=+cb^L z^TfenjD(Ucok{<=C-xw@Lr>^M^8>A^-%E`csFUq+%Et;fBfO8G98JjH;MjQ;GNs}W zF?5A3@gNk#Ha!|&Ilea*FLpEjbSpINhnH~C3n$x~{C?F&q$GSFEYytx^qfyuCxt&8 zUq(uR$D;rXWjH%NHkglI$Y~hAF=4u!a@&)_oyQN5VXX^I0et@#IX*-5>PeYxqSeol$j^|j#V&y&z zLdx9PE$Z`oHNW|%46#YO-$528(Vj~0IEqjO+lz?zx zqw18vA^9^oTHfZBSZ^pC5@8XTfylx{%|N%#CpnUmLinjK(cszSMhsJ0Xo~+}*SU%8 zJ#_Ep^l|Yb@i;B-jS75`F+LNsxcmf^;l(+Ca{LOgeVoHKgmgK}%YSHDG=W=BZfU9xR}amc zX8f*?z?HRFy%7oh0KcH0*wnq%00t4`laH+%>@` zBwiYM!!kvnv~A^C2*5I%QAtl)R&UTr37u@|9nA8vRa9-HC%jVZCo^z3gBJ6w&CV*| z?kU42AR}t^6=5%U!p>L?8G}aBEu?@%@D@eDs8?mWZ^O7({C=?(xY#2$#8=D6fVR_V z?uVIa^?LQ!qUUI<;v4g`C85lc6F8-o_6FJn!<$uL2p&FnpP%cvG#Y3y;xd-EryP+b z>;rt`f3)Ljm2RVX2Qay^8R!_p%aIO?Y+G~;t4?OpEM4&&(QC1`!NJ^y?xIgX#K#h3 zdI*rjh;6}_pY3|YWtz|cDFi9Ju6ZK9alq#j|GglOzQ5Yp?ZY3h7oV!LT35Jvy1Aby zp0~)i-9oI$k%s&sw*6s53`Xk>cFZ*Drl z;&_muSiYTCq5xMxypzhzWb2d{{ajc}UM08aPAJ|fIv92+i~Wg|#qe#pcJ$qE7LKoT zc-I27p6^kibDz6#N~(40H7NR}X0f>ywPRmzY1(;BN&WgvGK!3$mUF?}yv<*sc@@TI zxtwM`zr|wcK=i7acFd?FX~J$YVqfjNj`_-7`2|lI;}AuDF5la zj#5dkac|BDYl%h#CORI0(#fr4N<|jl%5vediNt42W{Nw@f&(sSoSx3XX@582^-;P) zoR|6i*tY+>1w`$I9_;lF_`LsQ`}wTR4|amzJ%<$)Tm3u;qBUn5FJT}Lw}aj5JPM_P zM?*vHX%Sh&pwY7DUkA+7f6L)Tm(Xqh})5x_kD)dK~3j5mn0-D>y>&ip40(g*}b2nIGJHG@#=ojyZ=j5 zl@s|1dJ(W4!=VDV_t|cRqx~W8?U%ko_aw&Bkt}5@(}fDBRladJUy)4c`!iV5QR~jj zeBP^h9IEaMc)2%8bCL4~XR0#=D~3P?1R%|+ck>*)a7ciIN0yuR_X$()lf)Kbgt0CS zuqe$x*8N9UoD)ItNDAmF^VO}J^z5H{{i<%YV(o$s1~avdyh}#kuW^p}h2FsiFCG-F zeKrJsj@|GCS*~*&mOyt#Qv`x;UI49J{J_U$6Xm;Zug>FQ6QJvYuTDqWbIsj)-Y|T2 zKiw>@K0f62Dn6jjI#hIsGcjHGa%W~>JdvU;XVrHe48RGW?xEDcuBs z;%|^c_6P%y9RyuB`Fb3ZlVPZ$=*x%SK=*aGD2qi{*z_trAxt}Q*^A{HTVT?(4-Qh2 zjp3JPV7(YSYu#qs^l?l64~TV{2||`XRLMa+9F#SjRi31aeI3y2aCq9 z*JGUU?u4KcovOCKVwD_UHe;8<`M1`a9cNYtzdbme9!VdvdX7u8_F+uiuU5Bf=jrZ;mHqdm`Sq28su!Q(C-!gj2C7>3N79=Nt86CNerf(k4txvWu1qAUMr!N!$Askip4T7n_fY!7go5>|JjP`9 z5WGnWSQe|xS@W`9r$HoEM2`xhIh)t5^Pw3*bX1VS$OA*sE9K1`OV{EOx4r zis!daJ#gBPYNgF4d*+ZrodC(^a%x}+%k|d8Zt(1k$VnmdE8`B~k|clvU|0@mkTQNY z<5ZFRiPkkNH*y>YmSZ}{u4nAN_-$$G^Rj`N3`zTcxkTij&vyGbGx@2q7rpaDr$Lbm z5rNJ|#71kqZ?DhxxWPNUuaFb_?IWVz$M(gaTB(ZvDF=Gc(-M<1c$pSf!v?>02dLnQ zEID=JwstRs;H->&{69!Y9ot`a`}Hzh>{rNQcN*+c3mli<8$8j7wX}LN~7sC@aQUX(7!E;lB(Go+w*HJNxx~yR) zk99jOzd~oNTH=Z76`Zeak|;2JOy_5Bx+o|IJow`dg0W zuLz8iYrzk5b~ojCsi&2uRq$Ol^vofVpk~LFiASJJwU<8~?_6#!lR3sap>by6UabuMcC;5S2e@!M#%8vH_tMhNEu+v(R z>k%4KXw@=GeYF5(>a2=3vP7W>gKuskj(VaZ|J2#K=h1$CCF|$7Mc`?b{qn8+y!EV4 zpKoW&+5S%=qXg}yJSV-G5nPL2pg(H`yj87Hca{(kZ-!@Gpf|b}iPhI1LJ2hkwqiutRThJ-CzY`Ra`{SZw{6ILHDiVs%T9 zdaGlxk-sDKPH<9bwUHt^(yo(zcx&{eeCI(R+rcJZ#33(msE&X`@PRr61xbm+#OYl~ z|4Qe{-5T(I@T9M?e<-@!F>g|kEF2NJ4*?!2*)=s3LE>Ym%ha5;%$zk3chc6|V!QuE zyY?Y06vpyPx{b~G@qf3eGTbbysO(S$+(*oK-j0T%;%zD*5qb(Pu_xE8TT~ZL^iR`9 zCU_^g;nk1L53VR|o(|K^8&66mA*kwoWHZQu-HEe;X=8O})KhsT>8mt#A#GoAlNvu0 zdv0o9nwQ6yWH06SN(zcB*$N1vL19BtwEqq&TR&_(IniqAn0z-oQ>x%&s!sgm9Y`aM z2S@b*{v>N?8`mDpV8#%gApf4NKksFqp=Kc?UHDVlm3Fh~J`JF~PfblnA|`nz z{tcS^=79)df4gS)7>H!x4-j%cKDE_V@<2IdJD{={#9^C;f$)_z#A;uvicEJ@a{YcG zzP{cbin;S`7i5|kx=wz64VQou4c!7)rWphOR_ox4b`|Cg#ZjD)Nd}rhQ+UP(w)02w zMP{bkB;*UL)Y2J$E3$Xa z*C*8evmYrLZF0%Eb?5r!+_Sb-QzZjT#w=4-g#tEaC#ry!r7u17%QV3_d^^SRc$?rq(orAjKZZ&O)p4wv1XBO~JmP~* zNI-~N^M$1z=E@zZ(S}epKy;YDhp9a761`1sYTBVhvc$=`eSFa&8tjn7*?BLSzr48I zo+R!?>G|T^gNpb!>igSlJa9kNA@P?A3ges)Q-6O5()yvp69d!2o?R5edER~@D;^e$ z=0?yB(S&CSwpf=v(~kU`eOiJgG`zXsKk-=4iqxK6_$_$p^tBfKUw(pf{|}l@ZA`~H zUfqjettyo&i&vcs^D@>NDSD)vCb$rJWN75y=s6MjfgDr}xHO6BZm6)Ccy22^9eggjd3n9K z&C2D@TK;wB1;XD=KUlp0p|v?*cRkxxWb+{6G2^}zia~bTm5?rw?G$yqyxYue2;H8g zCteG-718m%+DsDWr)b9cc?q+B%^J4rjOsjx-rBxT-|+P~X0kmWJo&p0T;rb^3L@MI zq!d9A69So2+Z-D4$?6++FEZq|<$qZE_D!Ew$5)O+x|3Fy&(v~a2t8C-qDZTj(5`aJ zvfJmu#TMOvRlE%+4ux*XdZi?+uP>_K_Le@*$Wjohct6ByA3?-5_k)StAa0@AWaoYBgc{aN6d9(`yvJn$aim#^i|YDFSwLVo z77M_`7D2cOlRTnE_g}j zEZWqt3TMu8TPf_dd*TAnFmg(i9O*DhV`AH&Yf-!yGy9Das(SCLo5Ac_M?zr;gScN~w0bN)k ze^j>$%v@weiY1YSbm~jT_h<8k%_4cPl|FExspLd^Zxa6RNIrzYqg0q&kIw?jGQDBbx*A#!8Fq3_bPoUCOjGf(00`qpdg$*uSL zI{Sl)k+XfELCbrQWj~R#`y|OF)GC_fu+luJnkiz2P26!f0GSfP zhP+mGKiwRWUsPjr_zi=I`tpVQv$ULeCXlPlte|- z&koRNxr}-Qq$wyLj93x~Jw=8Z=F}fRs1C;P4r;0G_k8$kTYn&H7Hf$f>VkMpCYcYW z@&gFD5(p-WeTRdI>>`m4b7(=+$oke0F;Z66zr(2@MN#snxtqI9-*?lEM7#|9T7Yh{ zl-%&rVb%slJ3kXRv2Gdhunq1|P$}vdS4y^_G%g@%u0he_94)BiE7c{F&LbW#i+S^B z$-g~+(F*8fgieHJ{#ZuwY~wLT%v10N%@CQMaaGE_XT7@CJyQvEzy9vGyMR3e$1i9d zg6$9B4KQEK>Ojqs@=!!pvt)12EmL8~^7@giWaf^r;@8&Qz03NibbVc?TOL!LRC>;e z3YpkABF3gCVe6>*!vRv^Qs_2d!>PEwPOOb=kn9?*-x!7E5u5gL>Hp*Dtizgo-?%MZ z5|TvNqa z5{6#?eqsI>DDx(q$Wlde@5R%JPuJxoIg;eSvqn2s__w>+{L>j5k?kI>QOWykey-`1 z1h@5(ZzxS)J?;2-G0_sw+&9vCH{7QVc2WmBPD|84JH7Jr*55FrGDLB93dh1f3^Pjh z;;hcfvMe*?VWfn~iTSiVkD_QxGRh)neg(j^$zs+@n!Uz!?3- zJL99d@+z>ZMEk0bqUp9?jt)bKp3b*As#Qw@q%KR{Drsil&W}Q4bcOqZ6$vm19QKT0 z&naM#T-TlQk|>GJ@f!Y&f1v$5?Im{F9oJvx%8q+Yw%3{N$lki;OTc8f>D4$uwe!Ju z(NFsDzu0fS&#=k_8>N(IU%(e^o~8^>>uhoN2r}O8)$$d^+9}j%&Ch6ge-sG4&hxE~|6tgSQ*p^R_vF>Vp-zMS0iBPJfp<*S`fM@c%FX&m z5<%I;EZXT!0P?v{(KQ*ywuI_=h(rVz5Z7zrkm*qY&M&<4*uf7#fb1}&)v>Zui1rui5?2@_oTmeP-B zK|vSo!DV`)N!R5$oIW zT`SKb!AsdB629(;leSaD#7+TG>E}#~>!WQyo6Q{b-uLm?OUaD#xAqh{aqo<+#}^8l za#Cxp{u1*}#Da6ChRdkh(2ZXp@+S-+!kj98~Gf;d%~H8}E+v zpOT|!v0zE)UakWvUyXZ(HtO=45YFKB&Hgn=X)lQt8vIK+_3kU|Auq$zUYeK3Q$yf1 zgW>nYLfIQ?tAC?o>+9C0q~bD}TbW3HGjVB}&sgm7-K;jEQ;``pw*P)^@RoSqT)eq^f|ix$r8nUcwLZ9Cy@?qk^i2P~{hb#ziq{i! zTa?BYqq4m}ZVmv2f#2}%UDhFpXm;b?TJ(m7L;-I8*>jymxSpMF3Zzq*85XSFp1f&3 zX}{#<(6&PQho2&G)HbmaQl$6kM!1m_ghZxdp1T|h_!f%ek1oS=qNf*^aS(o&Wd-76+oVtsWcLdAd;8YJ+DM%B;{ZHV*K*mFj!-)IY3(*al(k8bW zLHIOlocG{uJ9zI_Ut&ASJt2z3wl{S14Ia;Ua=)m0395fh(j+#}%CPhdGskaz#_&Cs z1nFpi_f24lBo6x(2IXdRin*nIJ0S^#s>)vCU9)eh@a2Ros#z*bR8o4fUlp~YZ~NZf z`34AQ0fBM3Z~cBE_0p9(c1mB>yW@$LH0v1yb?JIt%M*kR89sJr#^{-PgOH$^{3fJq zC7U3eeuFePf~FprN^bkoLy_ASmA`oB!`FzLG(A&mD$u{17aY9_eNM=VfK}VB0!$5Y9~(nD-QmrU}eOl01~^XMMTQi~_U1i{{>Qo1KA zo{kYpG4*Uvx>7u>_T%Eyo404!Y#a%xVVF`gVM%-F!Q&r+=b-%pbE$`=E~J>?v+XO7 zr?+9mg@doYV1G`1$2wK4-}UQK*ga7azD-iKe)9uG`KqFR;=-Hy@Y8FpwRkb%*c4sS zXu6LRN(c$d*`c()$l#|-GqFl+{}H!9Bw2tu-IMlJ>sr^weUJyH?WgIc|M&tyaTMuT z4M8SD=`=pa?g<&{yR!#YZwLgQmL`IB1q=B!h^e^;ZnKeNPvvwm@4850hxoV z6$>_g?^)P^cUZAM|6)FD9HC80JKqUDqmu@PXVal5=95h0j{9h`NO%zpsN=QqehsoR zwEaBkWpb{lEKEiQQgn_R^lc&Xxskz(?DeZcF%fVt{w_8GCqiC+Y(;$7DGnSzMt_Bi zaSViU5xJ3jcju<~2S`t*V!&xvnO?2DejnMp*OwMiI$4_?5q@_8qL5LMS|q)=ieSOe zV~iZ?=>TSZ^QMz>i+RDDW<7BG<6n(C=htNVo4*PM3@>U)2i0Gn+6rHA*CS;+%aJOL zQWw!~0}eSLUgp!(un+F^`!{E7SaWJ=Au>G|OJwspHBF)%F#+hiNTA3;Rmohr9%QeC zkLdfXQi!U8@?@b?%|EwfcNP|u9@&7`*ZPG(KdM+F=5QkPp~kh=d6sB$DRob^k&(@P z{)AME%bs|V)%}>j&5~%I54W$`qh!r0)$26q(#JBqS(wAp{cFDK+6VrYcWy^AD>|gc zEBc8cMO%ZN#eGS)Y+OS=xt`-uCM~j@4=7k`gW+a5Uva3h^!uta12 zD}*6oAtB&=*X~04-$%%T%A9SS)QRm6esWRL_iUn&4&GRatRMAj za`2T}G_&~5zRr%Dh`rfM@>pepSzmlCc-G`UbBjrxKVEXRlX(^RIiptOi-M(2-0!n@ zN`a}*|7@kUL`7fc2%_@nFN1P;m7j|yk=-AauBfZK$XRXu9Ag>VwEnOo@2B6Mf|RWIX%TVv|UD9117ZZ0&F73{B2XDk1Kp4>O#{s zf_HRr@*pOwN+{pjO~eF9BfD(0D?2W91qE=eiw*0cSp zfu2e#|AnTEf*+*lfQoB$hMRD!0ceW~ZU%ii#m1bT?cHa<(Y*6r!jd|0KCbbbc?i1H z@*8w48A6Izdv{;$3%?K@Z~XgoR%krJEV?+^JM>wvLZAJsM@?HMktr~J59&7=g8Bbg z0CxlwqCtU|V^#n3x4Ii6h&K~YTQ*Q1e|BaJinQwfWU|!C@QV6ISvB=Tqt;xrztdQ&Ob(_aAc}?|Qfux*?Hz4(WJLe2xlAZ@+J$EKO=& zg~q-xJxdvX|E*b1a$l-}QF&6JLQx`K>g`$kOB%6%WWu;%IOwPnv|`G&3f6PvJB0;h z&_o^*`V_p1LS~ipW*z)Mh5a^H;BjjA9#CdwQpK!0m>_+(;qkllS9AFz-)6yjv?GIx zjm&Fl1-Dlkt2tZWWg>jySP8OdIUSR`BXDD}!SCSiBf)`>JmSV%r%0l}FQfXdn*rYC zm8JER?C%6OiMA*?C6^dceB`|);nhMBN`&6$kuTa`=M|oNxMev^z^AH) zn|NDJP7{mNd-m3<_k(RKM)Bah&~Yu@lXs>-T-|@yog+G@Bg?3=#S4X!{;#$E)TX#^ z^JsZvFU-#{@z}dHtHMWQi?KFNfBSyYwo;v9=*9wk5^Mk;TsAHCx#}z>JssxMB>*O3 zkB;}Qx#!U^rMpSews57yZWH$A7mMbgfh@&LQ7FNsYO`x_KEIvi3k#*^#?$ zxz`1f(8R{BKk%Td-R-&F(dW!8swO)lv#+4y;+@~pFW)Nh6%I1T;4-Kae6*)&jl!Xy zr?pJzE^MjLX6UA!6I=Lf5RWYl7@cxKooC5k`*>E!^>>dD%n{}iWvtDH7O}%(kc4h? z+1#V?mNn0w1irJ!OuodPbq7r(EFV2OLsv>$hYjg= zKb+NmRKaN`tDUY{*Qom}$iXRcPbPPNcU1lNP;zCOlYILuis(W6vgoE89NE;a-04Kpt*Mfmk*b@ z6hj0RFRg~%RB0k&QMrW`Z9KqZpPq9z?F#PEkjs)LQ+Y`le0|VNvp1 z<-z!P^5=x{JoSylvzAeS8G4tRj5UZ;V5`skf`K?BXTZp5w8=lfQZwyw`!;?aJVV=) zX4E)+5(G%=U|QtKs(86?;xIbz4E7rZFFouSvIVBq$I@xhBOvdCaP=6!rm-{ z_4BJ2z@XJtt0Uplnl5!C+qAhx-Q||>mCI$0><+hBe3H-hO04A&Q(MB$hQT~wV6pdN zJrwhrvmTz-h`8(JpyGHOEm+C?Kdd;-8T^X?stS%14j2d)h3afga5Eq7=4q&5Z}BM=M#t{VC7}BzZl^a%8%^dIz`owxXtc<=>_v;kanv+IMHWubTqtZRbit(H!A|dY#BWMrqKm=UF+Ug5SuH;}Pf^2kU=vQv1Lb zyqS{kc4Us+v+=aw{942SdK<{(kD<;me}3k%jwTzRq;P4nz*!P!-e1kSPY=8~CcHnU z{up6yExA)S%H#b}fr{wIKgg?p`tI0L#wPkG6#<1IOFu)Hdu_0mE%iS&ly2)^oNJ0Ak(Oh%kW+<`EkDP7vHT<>AxkUMhtF~S%YA2ve;;ot^suYH> ze)huO>BIUGx`7@vo)bY}{8e9Rm?MJe2B2I}Cy!|=-1Ozkmu8)Cx6c2VGx4cHX1DU0 zGrWY-S2gNiev=$w(RSmZi{Dmv3__q=td)q1M0ADVI$eRC*UwGx->gkLn$2)JK zEfzLd_p`I1h)b*tqwsnAPOlT9fb|`3UDM?rsep0p`$N}VNXZ6B_`HA?(b1N%m@z(0 z;XW$9q#E}8caH$k^V}7D-4wWc^$~I9NLA@MgaEA8q8&d!dPkc8Jc7l&J`vIx10mZ9 ztQpXR))l~xL7#DQv6ZXl{he3Ji+ybiuyosCr~edofYY5Jw7If}N7DOy;As;E;HF0F z{${oZ0qx>#EDUU2tc`*ydoVm#XP?X?KY05Dh5iHtFgcmv#wR zxnFjmTJ2ht^1AxqqeEZuzuQrg^odV-Y6sJ1_TT4M4cHEw5sI0gjK*c(&{{urlO?9) z8H-O_o1x)Tz;MNe}?2>!iQ zT;YbLKcPN_VzQ1d66%`EFnl9S>FB5*=hR%tg(9}uj(3$pQ7Y7p{*cNj-5qf!6VO8V z@Pgto@-1Wk0Ri8Ja?pBZUysq_z~z?vK6XR@%oaR3|AZ07VI5W=bAs|#GPPLhNybRF z)$C1pSY*@W$p=iD=l@jq;>^1XxR$g4i6~b zrVa2{163~lrV*(ZfeLA1T7KJyYRHZ_@4BuvVbong}^U7kx2S6o}wf<_11 z{peJDkOy@%ep%bmy6HcVg+s?X0jnAT<`Dn3{fMV|^?8P)PzfW+l9#bp9};NvEeOYc zv{}sQ-Y-X$Se3*rs*+>oFka>Z?8&vOZ1#Al^rq&d7_Hk{l-JH7wWGpc7K~0^a$)N| z%)&umL^#vZUc@J_7)&uUK-1_ zB$?_AnH3EkR_RXQ$>sqLqq5UCV-PFN{LGWd2q7Nnj!zh*%OfzpV)t#^76>13SQ3<1 z%ECVnm2vcZu@O$6j*1=5NKE`@0}n^y-;=2?C@&-M0E}f?S>gwEr}AGA}RU9 z_VByrM9OvRBD-~4;R!U=pm;bL4DqD9q8a##rtqZ1ebc*~|C4V4nr|uQ9jY{G;(o$l z{LsMA{`>ED`3QM_!9Q!a1l)(!le85#CUs((aiKhi@H>oDT{os=eGP2=ECWSTsYt{L zCiPNr__AxGLpQCs=l6$yu6htTkLr!Ap1K(NH%9?hx4%EKZ)#vmTF*sJHhzjNA--Xw z-hA=)L51l79%ujP0!|~xx&e5Mc_;A(XON4&;nOT2NM#Rg=b?fA{j_Smf?n=&aiw)D zqZ276CqT^p?M%n8j@|Q*abmdKbGFJKnEFIj8iq(WOX3lkWh?oRxPrG=PxYU(!%b(z z-;_IOJ7^BDHhKiBQ0j1wSpI3$P#dAai9wde7hvVT9V2!!nFr`MZ&45;+1)tiwx+V< znuky~R6;Z?!O|U&g!j#D6@3;hj&S12{r0;Y+yJAPfcY>Eqnlo_HszS+ayyI1dWV2; zuC8&!K^a=Ry5WF7r(3%y2|;T9e2Gc3ADg!9n9=cdSyNOqo}(ClI_SkI60UOrPk?r`}^i1)s!=T&|#q4eJ_Qix8mY4 z`3aqtJx2$#oqbv2G!_35Q-XnjE>WRnsU6=t1!==j4*$g_c^LU(<~7av6YkmUn^Io( z6V>=ekt+#NRN5ON)Pj${PR*v11r4!iiz_c0&$$E#z7}a#APlD0wtl2n+)g=| zgKJl?Z&?M>-RXZpL?vF(Z|WZqJA+z}F;+_uewn!LN65=-MBrBx9YEmNWDMn|sq^r7Je#tJ%O7@KN?K z?zCyMmO|Yi|CanuzW$04@}E?fuGdfpM?3YVm6?rF5Wr@<4P<>lwU&)eQfUV_RFz{* z-%!AvaZ$tOVZ(4>q-Po*bwWtP5`L_1S_tGNm5Oe%xnM5aN6DTi1}X{hjOSgkkTkek zu?*B3CD5$fimZv@97|FHS!t&f(BAQfj@GsVj;|USmeZdZBfsc$%p_22S&$jpQq7jm z#vC0jUx+@PtpZB8es(!%qn|V>g-NTz&Q1JE$XH!= z>T4|9Uf#kq^r;MhJ9R-%rhk$38uZ1$k~Vfa-|Zkz#}H0WkcW^tP1PbO&=C|z zVMp?h7#0K6UND8+oG;bGTW-|kgg>7w4B0r>i8Sv;W-)Y+GDwRJo{m zwF6k4v}l9>ooiCUy%;WJZc=jj_t7JqcD^ILa{3505!c0TZ5)tiXB7|QHuy1N=lK$L z2r0{j!+v1|R#vZE+;C(IgQpW{w0STV7dZOSgpbS_s>t&^6Jf3de%a_`=nfn~1|vGc;7^_U&2#WqO-DKn4K!$U5J{%7sGW_Jov z(}CK}kaMqF+ogzUIda^1g!*K;lyOQkqA==%fw3nS>oM_3O-G&{WlUo^Nw8Z%q@ZwG z2=63*CnXG>yN{V8iz*`V4ktm+{V-NS?9V4Ds6V@(spfCJz$+BkR|?9NY0Y#7_y)g) zd!Dcc>#xAvRQeOe>ChEq(I$HSu;7){@9!1X`w6LJf_0VDjHa(6S5KzZ^&&Lc#hHDXN!LOGM4y zpcowq^qAp^Bb6j&pagjNN!qc1{QX~=6>xSJ>1#SgG$)+hNlevKiwxV5h#I4C;4x>L za%%Y73Gfx-qtNCQK8ePg#=Aucx_$6zc!y~!r@Vv{rwGSEdBC}O%RyMMHpz~KY`(Pn zn-N4mx%@+r6f!QeVH7r>{-+*vMy=?qT(r|+XWJ=u31XefWyOD!QM`WR6~@PivvxH3 z4^x=rbLF}9bf61m%%_JDTqXM#+bR6^5PFQy~cGU}*3M z0wxKpQv*&yCc{+$7N#%DKY*{Q%zFleGk;H0Gvvu(qPH6rz7w+$6eRgPEnR1BA%_>k z!XEBrNMJU7fH6nEu5W2E<;3MDaKToA?z<1QrK~TrUuvV4YGS*Pai_ptE9&2+7f-VL z%K($?;Ev$MnfMqxy_(fP8rCL9l@?MVWN#0W8etO+Cgds9WGG`tHLO3MtdYJk%0h^8 zt8NfH+|wIx0ofZmRL^ULkd54stcLCRx!FMN20KE)pRq?B?XEGrr#9l*Mb^PL zvsuk}pmLDCeWg^)RHm+5ZG%iUP0y)F7|S1L)rZJU9@5U}UyprOU=}k&5}!gEH~OiI zk3}6xO`ZAozmou+`CpC76D?u{D%bgkyu0$>&&vn=|KJ9&;60Ic&c=7bHR=tQZv9=X zEG6f7S7egq9}1wRDN>lA!QEW?eZkOglf${{hA>Mi%m93-%ztk`JMpCVhco&Qov&1P zR1OLgJXm_rSFSJ18Q4)xMMdfZD{-EK2Jk2F3Ok4vW!%BSKu6@S)Qq=-*FcnY>tx>S z-gKZAIA1a^Emc;B62*v`k>p1yO|OR1UxYGSO+aU^q|{@J0&>YwJHxt>s43*>`bOne zvUy(OD{0VB; zEngYgtB`8*+{EJIFcKx59kb6v=cLjgK*xRSMuD53h$sQ<_N@@t{O=D70a&yYE(`0p z)PhF!j-ZaYLROFV|0r9%J`%qsE!b)!6@zTJhYNM@Y`7@Jw|hb`Q*Q;ww284QAo~-$35O?y({_`Y{r0WLx5COuky* zP1+gR{TB@~{D#b`XXK%Qh`_0JoJesPPz%!8AdRE0WoX%)9pd^{Jp4Tl5`>znIHIDA zch)MLoE!QXnJgYnT@=@4dsuy4l$zc>LoAAj>@4w5nbvA9_7(|rZwV?5ei}~8US}tc z{LEFtWR|HZNJ0Mc-HpEB`nYML`U~ukGDqQ02onOkaqgN3CP>N|6G-m0Om-bZ-6^V7 z$k;LoT6)qamDH5SY(fzYCBNG<3e;@XkGniL3d8Xibqvjn+pdn7o$nT3eu+q23k+2+ zvNUs7*m^C3=R)E+&*k z=hf)=r<4~d(=yQ|?Y@vUC6xm9pvZJ|%1!#Qm?0=BuLkxBP;~PKA+)M5o^q(FJo<;q z#|nL$E@L}n(JP20U2TNQp2aCE@xO5u5DLm&!VEV2yj7|U{*0t8=)%v z$~_NK>cSeLRh~zzrgY0Le8FyrbkshJ+@^>T=#Fw^@FqirQg@;QD^*o~m)1HuPEQ&y zN1j#GHZkpBEORTWMBw&6?9e0bWI#&j%pf6@d~G@6Zx8mW>#ODFFQv#=@Z!+dm?lXz z+O1G(2n?|$YB0JfQ17cI>k=D8be!_<*l_G+vRIi*hT4GDzB6a`cnQeXVKKg7Uct(| z8Z$zZa_#F?oyozBUca?02~JXpIX*Fdf;{n9-V z3{qZ$7f%0e3S4!V6|(W{%Ikj<-Mn_&9W(@JDRPPu zFs+vAVf?6R0yVl9#G_$&&~Ko`GZ!xVe!r9$!kC}KTtzM=Tm%t&j9?vmjJ;~V;*XYu zIy`RB!2ivil+^2mDnF-w{pod(ZJ9eV3I)E&JAHW7yG5z|O+lojH0BgT!9RpK4^N$v zVLkq&h@XKQnaADITi?1xwrCG*pmDh|NcOj2`ff5rGyoS~P*k}$Cy`gjHk1TRv#Vg= z8gDVI6WDJ^r7}vIHq}0aFm?}}Mq}kLNC)BHmPAf2w}!aB^chCunN`XmI@!{jmR;$` z*^pX~nRzoo=w9WX)3Wn-=s;h0dMZD0|2>1yf7NJ{)WY4*K}HqvXwKBWdZI}WXOW5b zJfO)*`XAc!CtqB}#U>=g?>HLnxHv+zTOQ`ASZ75An7>fXX+%ju;}``$MAymqdY{_=Aat5_Qct!_ml zX5jamU-#+ABT;4D)T7uE1T%0&Qd_iKqi{tB8^};YG?#T&8QrDaBH3_&zo_7isrQ9JJX2ojnMRO7w!tRk`IR3;Jik@BE(-H&?=e2%72o@Xq2htyz z)ad)-0$T^*YFap7@}CL5Bo+Z*+3w|8$f zT%^REIijv#meyT{nni>+Z20NF*P`gPSNR&Y=Y`bPUbOfXetw{giXPg6?}jIqvJUhp z*ukWV5{w~AqxdK+#tk_YDvz7jg}mr|Tt2K&$+;V-bO*=7$$WH_>D&asA+h?v8< z!R!BK73n@kcJGCA3qI1tlS5Jxv6P&-(VH}umEc0@^N8#z)$mV5N{gQ9PVU4MjWEo5!A&FZI`0KAtz)*iSO%W%nP#kU{^fzM~g2 z$ghr&o_O(@*^}K0UJx!=boCYT8%Ncp4U=Gd!)JJPh!Z%`NS`II^P2kugoBu#Oz3c z-UnuRW)E^JixDfp;u?jgmZ_SFK zP*x%COz%cnl{shXVFfjko|H62H2~q?9v)lBW7>#rWX$bl9s9)deNQ%n^M2l>BvMbm zs^bKlH0g=VLVvo@I4c??kLjDa&KY(c{nVmAzv%aahd9qOtY6@y?#H9^UK@__Z1EB= z1{BrPYu(7SxM8q*JEwZ&wzTv#;ovAITrCrs+q$e+mzN}=#RtuQA~5N`z;bpsKKE^> zLCF)7HD;?H@s}Lgqdbe?+}T8VT`LmFi6-~5vpVc_QwT_hyS6ZA1iB%ZSZ8;e%==`i zxEJ%o&s!yYy|`Mb!HR;*>xPVQfDCJDZ@XCP8= z_rR|?e-K@h$-GL7p~88Gv4EXw3;N@Px?!qsh%40~YJB~NO+Bpdlo6)HX0KkrkRXoJ zRNH_I=~62cM2l@5#7_}jAYZ5^+ikMn2(tV8M?_A)d_6(u3tJGGD)}>ej={sf)@U6{ zoTWM9IxHXVI*Kljq{oYTOPI)Az+J3mMG86lqO!?b_DH#*ML7)*GuDGIzXW|MaV+)5 zRwD}+3$Z$-#|u*nc~X^S06b>sFQ7;&~Lq{W_^ z(HsH0ZtE?Nm!5?NTMtYU;3J$A#3O8n-dZ9Ivz#j@2W;Rc6A1KLeSb%pWT409?VW`} zxPkOWw#kw?A<*Mzpu2~9fGo-PpfDe&uuwCZ9ddE_ltRO+J9`rNlvIV#w1%k=ha}8w zXwk?_yr1@=_akEDS!^-raw=4~%?5ng@VR{^kENXa)mg|ne?>)j;NH(XWh}i{Luxx7N4L5=%RU z)dJvYyTl9RGHbGoO&c6fPx!-!`TZn$)mY{11|Ih;`b9{>QRwD8zM#-Y|8kold-HWN zV%*4yC~+M1!obx37l3-p^*vjL)|N++|GJH-a=z?Fy_(vx(Yx(98~hLKx+1owtP*of zsFzp^f&!raIB9w!LW;jLjVSbSSH!{KH_A|+=rD~;meJzUX^6^jdj)?RhtJaNdB<^o zUqBWP0gAMxWq2B%jSZX!r{07|P?PV6YE}0Kug#5)isq7(T=BAu%rpAMU8S&cW$-MM zv~%wM5?5&8>iHvV|J_cL&P|h2l8kyj56!U-&Y2Gr7rVG8lm|>X!nAEhwk)so1^w^xv)0v2WPQE4aQTCmL2E}Fsh@avBI-}( zUI!h_!cN^Q?&)0&uG%;J3i3WMyK3h(X94Su0ygwt;rWO~%Bcs(^Jdcp$AQb>=X?xZ zU9QgO8GR#eE6>Oj{=kyB0-uz#LUYr!ofx=s_Ak6=U}K}JeX@y!cTFR&!)f|7r-ODf zWJ%jPp9}_KhQW5g;6-TpTg2THA=KI=P1@wP@sJRLzGib! zU2I;ECuOU}_d7UK@)jb0e;zn`{MXj__TH;j%Y^BXP_Nhn>X+C z2Q=ZS(>k^FhjTRErs3o3a;>Hxo~CiXm4C*%=4=G~iK?y9dK}l6HJB(Gi%ERNYg^Va z3W9@Y_KISaQ0JN2QPcTjg-jg=IPX&as;y2c>iSbK(~!drX$9ly&xu#JmaP2}Sw*(R z1-K}xYVRvDZix?_E1Y8YqInIj`Z>P)Qqt6ts*F?Z2;T0C{`&PZmtr6^MbnNXE6mkJ znsq}-*)dN0c3t-FY4V4T>j%Z6&2y+|4q=He>iIBPWI;Yd&u#e3y&)D(h4%9)us$#A zUfH*z*}CxYvbFp}x#0*7{I1Qwv9qYh4+i+NS;o#%y>fUPU#fU|ls;R2uNPKnM@mmuHBmDW#uAQ~Xc zQw;i?&en+2hiV?c%nBYXW(Es7g|@I1>*#}2vAv%qSL8E$zVW}`D{h6(GFnA)t+`lY zWf+`R<-$_%z)SNmIHGMLt|@`|44yUhy#Hj~)cR-j3E&qwH=IQ+USbRvUIaha_0yUe zICe<|hp1ArGXJTu8_;dr2CKlj#!qdxzNHz72c~^BKjlG;9KiVQZj zvAsIA3Vyg!l*e*aSW7d2D(p0nM^?_R(bBKOh1w)^y z8%CV-wtNHY1@-ai`dUP~hr3i9$f&7N|AdRAA1CsrFFJnw(^cb;t&>T)o* zf&5Ez@9yOguwsA6eh*04-#ogbr36ydX`4nfT|CTPk_ExcZuBi*5i2$nh~&ubs+zBQ zp4F#Xa0CTjoVfs=&e1@nh#S${i`>y;5>_}k8BO^(fbNY0#d6E+Nry*sJf-nTvHLDG z4X|UT3nAE$2(+LR_Xg4(Rh=W&RUD5YL&A#`rCwn5Tw&3bE|Hb4*;R#RJgJ;9CPd#PTBJXr@Gkx;0xARf%N^+PLo#sDIv|=#GTivmGmvt<6fHLWup$2 zgY%}(u70k~{(+s2%-QB%{PO-KEbOMqp0;Ax$vZe!zkLhFGu#rT3LfA2zef^-EK zw%Q0)yIq3L*E_P__fK^k_ELKB^oqGTtp}w76G4?(?zd6A0gyULMmlZpDgW$g&j1Cl~D(3kjyc zwWhlCWx(};NBygS{oxbggH{n(^qZsOvt*MM{vKQbgKuOG3nSjpoogpoZBTQ0S&`ycMx8D zg&8JdwWXZrI3n%#iP2q*{S)vj-`yx=sgRLo7+{aKhoLmeZaas{;dIqJ- z((L_*e5!N2SSt&QK3CV=uP86~G=vISn9h7FB9P0z8cW( zBW20?ZSv|pw8$i+B-3U$Ff}bSyAB(Nqi10Jld)JcWfxG5j*4x=q_JbXlUD_{nkF7N zDA{ZlE%^YNs^a^G&(>(hz39T*S^M{-)9H6E7xT`Q)0%bmpTi0y)^9t8Qg3Q(gYKc% zj*8pykAffTz2fuu$TW`x`Y)SBl0ifg(TPKF&@nv~1iYx0C&2fd+B0dd`jc?d=0WRS4(B_C6{8 z_Qks^wx(=a4auU7>8^@8nmYL|A1ghmvB@5f#L@g}Q#8+V`15Tq3o^jgx7xhHrzZ*? z={;f`i`0?LIA6AO$Lt(!+ufsmoh39GbG?V;~T%5l{u zU~|dg{_@zth?n!M-crT6hU;ttZStaFo)@Omc>Gp~*%+LlMap!D1;|9=bkeE;TP3P1} zyLnQsI4FNByP#H=bNhOj*r>g5yz`=4%!o76cSlq5c%)6Q85F#rr|^fcRlm%5N;wwK@*5i32*T(;X$&JK%9{sn)K$j0V zNl{vTZ~aeN_MDJrXI%pW`|BrowW(PR8^5m%;jfwl;ca*Pi-;Q}{l}h-Jyk1$^-p4Z zh=`hYH_62b(h>-5ZG;@0wM(|;j*G1N_CX29N=P=X+0{+seZ7%@5Wj3dS$wFl?}qpG zG%e$|hme`0Nl93NWCQ1!z~qR~!@#7kV0VPq3EGLqFIi12qgJm>b>r_*R~jtFXjL~t zw252z)>6%SgAUs2N2|AoyTjJSjhPD+Iz|Hp9y6M(h{;~V0Czxqu7;+?GjP=n%H90O z*>GaYTUlN8SA*L1$nqHAXvPti{D#WvT}9Jc&eGa=&h*+zZd1?692dN@+R` zdN%yw;3;><2&R`<-4yl9wdQ`?-w@oi{B zXgX0yZT=#|2Xz*jm(%)FzZ}Q~@YZUVzcEOWDU6$C!8Fpcm1X*;NzEpc>f?hWOQpY! z`yF0o2&$hSpv)iVsXqbHtWe5r*y(4*pV)zlOt$TYJ}1+#j=qdEs}$nMOMKE|Yx)vn zkQtqWUN$nOr@C{Up161t_rH74csiRYwK5Wb(DsXwpbDBNnIWv z#QH#jY1c-jf)0rS5M#>sD|heQxek8XII7UFYAwvJ2MvUMFYN? zrlr|CDT3t#W0E2$u3Q;EeknnvBsdY$qz~C^(_$?wG;ISS@q2i@?R?6Lk1mTfJ!#Yj zjPh}AoLG_HbtrnPOuVd^ez)uzzGZ98Z4`G#%5H>repA1b)HDK>fVcFI@1rML|MpOA)-OfLTR2*qc zpay|SU2`(dSZQY6w=o7}{i)OMC)1JsN<*Y2xia|qX_ZB9ecglhpokT#wR`R3{DBKj zTS$3lr&M`$Pxdaa2;F+p8~4ATu_ttgG4hR=ac))77O;#Ff9eJSw@*N+c^m&5|K+ z%dw-%zl3yg50~5y#84IBvIWgK+BkWSdS*@d(Wcau>+8x$LuhDYlfaJDW7ED_O*`c2 z=t2{b@v;1PQya5&6uv_yer%?bX!O}|P38c8IRCe*B{)seUsd{kE8|<+g$?(~QCGh0 zp4udVG&>h`#D)MzujR{9%&^)tKYNc3gza6xlg&LV`x7zyB65GKH*eVSxJQ!RASE~~ zbzk^#zRM_?#?#l#sHS41lO4RwT1rs&%!&cFcqE`=Ir<*Yag1(N=eB~E27vTC1ngzW z!a^x@TYhw|m-HZdkMR*(L&8fty|?SO3eZR1sz&vE$=O}5MT?{I;-}waCe0b*N+_f% zpxh#$mBP1rxay@DXryRO6_VNV@fEW0Z`HnRYbY4FSd^wY(p+3c*h!@hyY9S;psxk!K6wAxg6!e--R=B1e#;XVh54Ob&G88 z_GL}WR`GxNHmIrZ>bey1YOB(brZt_~rCPlK`k8 z*jPuuigbZDgvi-3o{hAnw;?|&uMPIQn57hhuEuk3+?xSXZp%w?(#G^!lcn6MvI7C( zGOD+U$A8V&Jzr(wI5054WB#hDoBl(sp=kh2$f((^Fv6w-e#opB`#Pj%rLuMJPlDg& z?whLV)eiqQw;%!9s+J{h3x*zWGn~N+Qk^l zHsk)rYR{c*Lc}2NOsL6EqjIo+E==eM4KstK;YBR-BG}bzug}PHO-mB5!U*g#-{NVj zel!h=y>#pQ7O%#7@DebHt^6u>5i6LL6o_TiV3)jw+1cLy&#%EB?hs%};JoIr(&iW_ zhmwJmFbCYBG0V4!;SlQGIV{y#H#tLvi^q=UtLqH}Y~Q5hPETtM;jR6?55eavBOA8N z1!UyK$7gd^nyuM=#-#O48?ha|M)T0?;J(%}!;xJ1g~nuWWk`r~^ek>!PqWeyPV@d( zHvW(DnE1EPck5!=OnAQnrf9R9A{vVTh~ahz=coO*dRDIN$uxj1OGhgqyA^pTJW;wK zs-le>@}RR%3v%-~Zw4ESuVFqb*$Ap-6El z4#isBwYa;xI|K_3#frPTyOyFsgHzny-QD5jJu~MIoNviwl6jI_azA^owJtu7%=I6C zmX9lK>>}pF;jP(**CuM3a;27t^W7@$lKIYlR6EuecVy>u;iWn}kc3{N+7ynWzr2rT z+Jjc(2CdO-6IT;9TD{YmzTh>FeBm|z*@`x)&@*W1P{%=JdzLxj(1^5sRF3`pI1rs{ zBXG)NABA7xMni+w;r6!GIL0~rw7#YvtQdZw`Js>SDN<8$ti17R`{XB**L=4ixyPPJ z!z}N)b)9vA?cd$It-Uk*rDq*)Kj-LSN(K+Ta^JUn4)?NW`{T#;`}yf=toM~wFa5tY z1N*g0Jj&Y3-dbuQ+d=p-Y(+NZ&n(`**~iK6lSSH?#4zxnwAHWo0@G})5-Mp zE3d$n2vZs5_g_4HJN{P2bNsP6GkIO3WH>G1nQPKP_6aaMq_^Rk*uVM1rK72B&ef%1 z9+rGskEv4#y5IWCDNW>FX8WJwlO-qB%BO>g!9RIEh}N38;FJITCcqn@U7dVd=9YtT zO3nhlz3aU97>y*Vl6+Zn&!8b}Vf24Elxg=jj-InmLD-HXb+21pQyg11+Q?S6#x@IC zkVX#^j(QsXcIF`9ZI>=TuR}u4EZI1tUah|Q3u}2J4BsNpiPLf2?qS`#R^R#Q3)W=A zPnBSD5TE-j=d#di&$!@=-N{;eGlUK!z%!VdvG(pl+_9uU@<{L6*n4X9o5B3W>+60}KK>!2FL% zKQdm6Ry^Ggy(ro&;4c!dp?m|{^WGqILWaPZ4GZadiX=t8Gj_LI(wCPFGFJlVHJ_U?POn)jB7+~R$>XYu0_LhQIVO~n(T$cl ze#`q(!+y7gWXT*_wvf2GU&r&zRnu5Rd>R*Q^#1?SesJb}e9G)H>Rj-+>c@K_Jb)0| z?Ze3YS{MXySdi}^o)b#4n9u96}PB8|OV?bX);r2%WyULBQo+FZoZ4rm=t9vY(S= z4-`7rpCR$9E0Fgea4VZc)fUo{+3ObX8hGxR%R9teUvWY_(^vPdM4WbI7L{JZYRK0? zSer=Lk=br{oQ^#ve6N#1>|$e9Q%Ch!-+0|X9xnNO1a&1h)>^~S~+Ouw%oh$%IMyC|l8C8Qm zH(=3L@~@n{MVs|_q{6Yu_5RIqKlF=fYQUH#)Kx@Mu0nPPHr!>BWQUD>^Ga-hUFZ57 zEjiTREUY0O-4yunB^yRy4bCij210p9&tqaNK;>u!I3i8ZZ5eyNGFs(FKMUzalry7(R9CZj8uhxVnQ6XS*vWc(ywe z+*cKUfVUazBtus0(SW6rNa|i ze+e2k%uqk$2xb1Iu$(E%+3Rw(0WmUG|3*Fo}06=(dpDfm&)c0NBf2cx8UN3OD(QT zkLcDNlh?o9>m@Rr?K&-Oa0JQP((g8{k4jGPJo}q|ulR90Vg2j#JmS6!H9wMm+t8PS zfCSM`7(|@^F6uLs)LLe2%#D+h-8f)b^9u@kx0YT3i^gHCb6ncXzqxCxxuH+!96*8@hXA%wO|0d@@p+p77}HP15gbywLihEt#UjsTv5-W* zYcK?3-8PJ6NdeVy1tkte>N%~Hp|vkZl?Q`zGhCY6$WYTl5$yr&cWQ@(XWLEY#kLn8 z?|jdG5ec`7qk>BS!xUPP(jjaa5B~6NuX_HuJJ8VO*F*Pw8k5*HQSBAydP?L6Mm!nI%%6=C7^QV;!o+M8=_^W&dFN z;(8Kc`s-L_u4cC0J?z%zq9u@rHw1xPs5RG}@0)x-hy)bNp(5IBX0?m1K1W*S0{A#| zgZ^(vKs}@tzA^C8d`91bfs^qY`I@eYysC;3!F!9avaUSP23$A0!dq>8vzkJ5cto7o ztG;b=CMolF#d;hHl2pzngsEV;jC-x$c1BON4o_p@CJ?p;Mt~WbsoVoCt~Uo?vBJ znG?$A9yHWnc!7#W0GI%E*8MFdUb|DmS0kh*{+V zalQE5AXGKL7FUrx6S29L}i_AUXiq&UpfRBh32@VnZ* z5=lopsBhVn$u7>yO#d`7_c6HE-(g8*&sKMH(04^M;x3I|r?v>anqdDM_4L*Qv;Bs+ zGpYaU6;(Lb-x}ca=rJFgdC%Ew%tQuBeg00|vvf2G>DgvQ-!;G5JTnvuHrJ`UwzIm8 zB;V7Fv#&#H@sOG6W_O^1w`scq$&}C}#)LQyj#!2%T=3_}gd`WDylG8)>FIK8OhPP7 zJw){5#c%XI0^Fa!ufmWSC7sH^XiY=YcRQxsB0JGhk0dG(2C@bNNur<{JP`5q5OO z%V+5ZmkH0DeHEC5wxin-6@ZiF1=5pYG$Mnsd|*f51EZm5D}9tRRXpjinG|`5*$aV) zneM@1oCt-Y$yZkTBN+X}Kif?HoPUVFAN_UQ4TZ4>F$#&3kVo+iHlfBGxY za`ViEaJ&AP$^*^hOZhy)`+XWD^Tobkxt3mNu*&dQ4GQYqyu*fWxfAlYqGFSk39P|f zeEkxa*zEMP2oGJ^89`(`V-A;|vuSYW7{xfu=kHD|@r8q=p`$71%G@jFC3+!Mm2RboUMb;{U$DJ7Zi;_3=p)I@59WLFBbq813WC8Wa)+@z5+r%8>E*fgf7b;tYp{{yFp!A66> z)HxsZ1o4cjkLt{)E)PGg+HtjU%0s&#Kq1AuK@AuGSDln7?pCadsL_q2kpL5-`g*mg zeMG?LGYVolxvtdx+6~o<2sCbJ2PGWp6WLQQvus)|!0sR4v;d_6LgX5`Dym??Ov=xf zefiN!EYy?Q5eW&IT#nan5);4QU^PjkkH+yjmdIBv@rQKC0>@3q9D9Sq-IXdjivCOm zb?Z>oD6Gazif3v~oUV6-HRG*c9*E#A<3TH*Tbl;t^Ap)?Nd|pr1y>oiv1)o&8C2On zlW%Sl&PR8P2jFrN#N#<7#!Qdb z{kMeU!cVC|Va&U0kQ)e;E&M{S`?pdU^PPvIs9>qG($-5PIzstx!6_1$)MgN2mC|GI zw;P%-5$zmXDE;z<$n>gojE|Xooya{MTx*B8F}z^Z-MRRpIAy2FS$zGFX$n^@eJ$Cm zI&r?cMI!*Dh~4(dY=MkOS|Qylf`Q9O+e37TgvoZdx$CW(#P@c}p(8E~A`=S%;@V1ot~oz?rU>DerkYxueD~OJYu1N^w+p^6=dZ%#qfbE#)%~fj z&(;;szii!mI1)U*0Zv(AkuzxNKE_U;ceWd>+uYo)C$h+xUC}k$S=m*P`$_lD6tY#8 z%*rFjvZ%Aeld;J1&r0%#1eJw7V&`+4uJnkd>j9DbN7y0SxtKWkoB=148tuja+Aj?$ zP5|dHFEQRnb!MsyaoHiMo>F5Mh`zOTLBbjepFc3gX(IWv;(;K-yfu>RZcuD7*RrJa zV+dcgpOAG=X}P>G@x(q3lK$(|5FPh#Igb{w#YYj+h=Rffk1DGx<*L@Gh*iOEU3Xmr zr^D**2pUQh`@Gkp*(YLaJDu)cQ&Z@9lBcexF~`%EofsDsI1!(tQs$cm!~i2@Uo8z( zZgk$mugwO)YNI61-w>m}RixctT~1m=&1kD`k`al714H0Cmx_`4s{4w)!9fU~{vnZk z-TU*y$px}q+MyL`h*o_f9vV4(7Jl9sDQHNf0`%_BWR_+nu!L? zg>XcOdkhd*7U_Y|kFinQ1rovlX4Bm@3yF$?OUXGG7Wz%I9>Tpu&vsJqLJMFeq>C=qXcOQd;8Mu^scQVkm3#`Re9G>S zu!K<$x<77O^i_wMh*{U^#xvlO$pr8!RcUcn*#=@q!V%`<>1_rY9O~pfjLz4tH^7VEE*F?D zXb>&Dm6DpgQZ`ixGM$g06)RHO#KcBgjoh@PrM{a=wZqr02ZVG1x-+Qe@%brZ8Ah-| zetM-jDUm%iCi4R)EbxH?KjCsF$K6im84M0Uf$-}BW&+7$%&%MXA-R`%jdG=tw$+%n zF<$iTX2T{IL0jO$1)A=y9kY+nwU-L^H=+)jJUmxC#(FTulr8j?c!ZiQEX_KoHe+7G zjtF7h4V_AWmH>`+3wz#AC!!$_wEd`eDWm;W3W5J9jC&1J{T!h%^f6C7o)aCg@4x zFJtr~MbJfbt-9Ke8tcI)1xnw7pUYAFgjqD-x9gxV`ZvkUCeOKjx{=6~EH+Vr`bQDp zYYJ5IFfX?G0}Y@xYsIdreCALlx{>6DWDB5{F;}(u-A^aKQq#f_V5}?lzKEa*O7xW; z3bcr;I8`~)+>qa5ygH)13#Cz9|%^kO!laE zD~H7Js^H9y+9$!E%k0`i`f#_&JAenVwkNv==5(=FH$?}iCL?^Q?CF=4GNi;S3;qwuNX1` zhGX@c&_XI}<+@(EaYwOoIu5)Kf;zwHf`R88Bhh|h1`awc6V+UP7^r(l$W^FNJ`btA6N05wx0p<-0;?;!4@dY!4x0c z_P^slBaQ9Kjf8C8QBz27xleNY(3La)9?L3xX7z0SVFXdY@0VV5=uhcm`D6(F;7*In zk85b7?eVA3Q?|i@{4`4G|I`?x4IzkiHglpNa+Vyj+OG`yL;1WpxI@vUA=f#$pF~Yo zIL4qzkIK4Of3UJKjvkG%YbkgUn$ieljB_`sInc166NYb zIpU#nBbrNr*`Q-`Ui7&F3Lt&g&Q#m3>)W|(lb=b5lT=ZF7)X3`IB6>hnlZUDpwghN z`dadz&6k=-KCK)Ld) z)vA?!5%Vw7@yb%eqDFsCY#PiGKVf#-)t$DV5+D8pT_G4zZ{FUhDA0tz@0|{*FVYrAjylI7qFVMMCve-V>4t$AL z#;jnWjm0nD6#sIpT5K(#)f`tf4s5D=O5+(9#@>dLvRFj_(tTp)8OMro*P6uGj!RFelf#q~HM-?4~HBb)tsW2>>Xs7mf z@LJhajWoyViJT~`swm=9Hv{Pe^4Id8!NOjWG`1Iv*Mq7A^DA~dfjUvc&Sk#5cXVpU3*Ag<9?C1qf|?7c#HwhQge4z|UaRX7 zg8;d;v2W9wl-$0jK8%%0Li=LdvE=NDK^~L%F^kNJ3;=z;kBbK38mt0BdO5;ape$`? zu#T-ymowPe@jS@+ZL651J)t~Mrm^y&e^?e=lhQWO%0x4lkEo>DH6&nVBIJ6`!{r9h zR62ZfOM~sk5eeL9FDotG#Evw_`VjKofHIO)Z+a_f9;0cO=Irngl@dkkfGd(w{;9lt zK$}-UhO4H;7!VLq5D@SS2U7&o`Wr{EaDcEBbkTzWSLT?*vc=5%f#(E>&2oRd!`rbz z_G&`feLv}d=^(51AX}A|h9QD8KhVqImse2T#e`()C+w~--I>A4Tm7Td;&@$7g=URS z(beNu;&%AFyndd+ptlCpBTAG0>+OQu<89XO2836g>zFXH^PXoJfeWygQT1+6A2M20 zBi-&EtG+%-SdRsWiYa7AKr0hgUQBeftj_cnfo_Sntu<^JU*IpH{e=5u*{v_LapoFy z7r!onl?z}r;C7_8$^xPed0ck?qo1rtb)Z;LDbcjb*}1C4jieUfN-Rq(Vjpenf>O@m z_55>s%RhWcyne|}ZpmZwYhVB1SRZE@yj-Ta!E*xEZB(*%$&TAYBRF*BHmdCv`sc9q55h==E);lA+;QDnkXXlu!&*cr2to4&Q@lN@H zGo8)un);?@D&HOA?_IJIN_2@-;O+t}_8D(G)R|Ksa_%ybubA7Z^3G^MQ&LKMWT)BKiSWq79>?7FO_{`ZK4*T>-s+ z&g<3QR>NC@uL^ikfN3n2&7W|6`t;n?r>mIuBDCI{Re)d|FSW?_cioZ6Lui~b-Gq

P5H^;$?VTdozzQz9*RE1MZZq(k%xFAQ3~Y_=$NfWVwU4-M(+OjPk?@y~v9UsO>tT9TVdYIdioS-UqJ3Ly#|)jw#m z#yr*rLi|Fm%pU^wE^oMNgXMaHa{6B*H~!AlYUKT_pXST@x1I{Thxq5@|#v z7C&xx2XrfhknEhpy{$I0wsDjKMhh@-gQOoE^pTAkG1?#lqfM=JKdTbPaD@-9hCgRh`@z#!%9+!1kosvq^;{jqir&x5*=Zo+HAH6Y zHyzP-HHv5_d1$Ga+7|ElkNW7X&x{c4^kYp-Uo;km{xjZB+NGZX!7O;+({T?IOaike zVNYI@rLbtg%(-6E(s4x+=$U=iwx{X8$B;@79SW^}=&2-yB;MlYYP5*7_8rspl>(=M zy6PoAq`^ch?4V+}hskbhs9j@B)7ug)Ml7hlKTtzQa?b4su#{6y?zJe6M^Bq*rXGc3 z?p#@Gf3^~?AWy-3`c16vOCU_fSpV*>Xuc~>IDT%RqTnb9ge)goF_wDrtEX$utTllzQ+((KaOnpaOW(9)1^P9kjD}Z15pIe{ce8gYyK;=WYbhT zjDC9+$qh9=*^r1=wXWIm*tuoLh4ba^hV}rVogE?xKjj?5zhs19GTEhZ=P*}hq1`*?&W=y(l9lOI$h?2lqZoP}H`X)j1!LItH6Weev%zj~ z$^d-7VfcJ^QhN0l5qk!e4AY{ufaQ!ACh)k(Ji8v;lZyV)4+dD8lekkb=1+P@yoWTD zIr`ilCo?YDB{T>nF3}-JI!e8x4Y$|n4UqPNq=-}=LU*$@_XSJxL`RF2Ko*%y86S8o zD7LGbjE3Ze6442mloCUbrknXt9612e6CY|YADs;lc-l?8Ubd~J{}CPVAxY}D_zD~A zZJu>q6j)#iV|6rt`9p4J|K>->BAwu6H_V9Xc3=32=~|l`U^5V5q)7V0YVmO?yZ`4p zu0lIoHfutQQVjr|*?KPrR}$C+%+0+(5DcQC?4=w^-tSRqvB2jdq}lhMDiEjJfZ6rv zddXgWDHW3bcz@d)rp;AxFdd9ZwykNE7)@~NTW>VXDXOiiN`jO-q*GW7YhH&I2IJyW zY5f9Z3RPaiQHw)<=^JHM zK+_V`eT6&{x`+cBrJc@G8fQfazAb-b1eG^4^aeBUCcBnot~15KdKwG+==TE1siYl@ z*W;ylw}TR-`(HrX%rVfKJH8>Y3;%O^-PG;*1SA8c00QXCl?GosZkW}tbzC@ZMqPN4T!eYwre=NrObp2e6FKqfF zz5pgg5Tr{LvL{b$Y0S;D!CWs8*rO)V?D<;TFa&e>t#Y9%Uj1+{nwUq%8|gH?o~usC z3)9tSJ0dc2&L0AU8v2i~R;}}Q+9|O_!IBUpJy{BmAVcQV4Ye%%r@p#s_3PR4m(km! z8B9nkJ7O6*+Kg+66#YG`Yo*h-ecEABi8_oJPsu*&qqS>mOjirT`@{09{iY3%L zxL++RC~T7D4N)j4BUGBNgZ!m3Iqi29zXNs#qQ&2cf4y)&&G;hP zI;{;zMAm)~j^XX}EH+x4XR93L`3_4Ar` zyzL9(@(ck+M~>RyY*Xq3wulnL-l=cz-8j-|6fe@-%y!uVOG#T@r&v*3t)oDpi~O$S z_P-Hg)0m+;2H@IguZ(@48XAs7qdrRcg8-<ZeoskjJiLaNy|yX$QHT}l6VZcExaL0K!)i^(f=EU;BDSLJcOuG z!0+e&b_e))dl=1h?8s+@tV-N_QJhVE6Y4Y-|9aP0hguLl0C0_RMZaA1Gk)RI?c$*W z=t{!#s5tdMDy@j)xx2QZGI?UDT+01-EuX8+Na(O&%keCFm;HY|P~$LW%Z=6zpIF3P z9lPF|=6TT8@H&A|sQdgp=q$23hD5bT%aN*vZ~DEAA*8V$TDZS&-bkq`xfJ zy|U=49Y8x@q`B5H-J<+8^oa#S(TdoZw=G_?Eu=xlB4I1x4G~i*+m?v0bP%-CbHkS484PtP_7v-AYoIlLGgV1k3L#1 zlYp7jr*5}x@usaz=Y3vbN#5~^tFcab<#^gZm8A0ToaTV8L&YTf2|2wq%>)oSu9{`6nMF8GdKjWn^M_Wca9 zp%34ik0Q(38~}6F?hi>4CdFQSmhL(|*;@!=LgH;NHAG-8cjM8;^R&Ke7_w>Wa{w%i zk};WE7lu4Ld?|IGE?@KnBq|i=yGkQtCIT%~k z(#HqXuibWK0m@B@Zr+l|yUrgCohU{*lB2m8cqyER;9t(9N%kuFJlVQ5Vp2XD0k3gx z`N;fJzO*!_2Wql~>lFPLDYB~U7404llF8}gvnVkY3z3CJvDy@AG`*o8X`GJddLd4T z2`=A=xa?biSU68Do$AgF-3GqC;G-R@X%t8}2gt?_iY0oea1Al`g3Q?6 zb?=Licz-$O&PNjtcP5=i;dq`Bc9pC!UxW_x``aTe1)r%rQ-WnZhOlU!)v^42z6+3Q zlkr5uhaG)SnCI{xJ!Exyb|>W^eY!~QPD~=?78-p>)s%iurM;^618K15Mf62x(^L&Q z`g`)Mm6}j|ihW-4OaI=?z?m;D9mA9sZCXQ^_~6r*bI(jR%O*XD#^*XrXtfZm5^3y% z9`<8LX-+5?DC*6V<+w$R;;qMS_8({4=V{b1bgDeLG}lk3^CtVEEdFG^xDuT#$9aH& zw{PeP#8CU{wYbBJ3{E~}stfcb=%6Q5)h!dw9*PBLyYBiUcp+qdw_5^vy7bQ+Cf72B z^#Jg<8HLzE`^}-iscrG)(Oo@ak^|x;nB7YoZ32U=t=mzQt9E>y?~Hkq?iDpZW4!py&=aJh5}^+iV(v%Ul!6#mI$h)0 z1>e7$1#SX0Pz^pF>>|Vytkk$s9~TQ_hnqX^m`IR!zt24#+^#18Ye>|BYBt~ zd*P;$1~zGh-owW$ox`8%)7rPP@D7%_)rPlZ(aGFJC&I-vJ>K4#TlXv$;u>7~xHI$sYqKBma6I_vBPF7L_SA0_u}J0#FvQ}qbRRi9<|b;-l1OrllE z)6FOY!`!jgQ9X9hoT7;tYK6SJe4e8GcrVovD}5JzsV|LFKdp2UR()OlEsiTcKum8> z<~xF>3;@zv1)S*8zTX0$2RUy8EGEDzm&Rg~Rxj1x{W4zNdJ#{%_8?bUWyZb9DEqNrk-<1W3MLI@%c%fS593=r{an|l-(pNu`qDkrlZL| zvdFD+o?A9WAY={SB#W*z%dcV82h1|L=*L1#2=T#=l%PZYT{Iy1vjfp{(^x248Df(6 zbQP7Ae@lngN$>KncJ#i63AdJoh*=hu#_wcN%-9b_i6tjZL1gd6)c%Rp>D4y{AzHK} zP{=(ACKH#8wkhD{Yvl6f7HRnVlWM~!^b;YYlY?aMEiFHGtyKi;m_2d7EDv+-dPZ%H zD%}*y`XW<;CWjeFevKBZJ;c)4R*4iycj|&?+HXKd?OV%ZM!&3Rg4<&;J9& z11wNg7n_4w*QOKN2&&f-k08tAG)EqYGj(<9+CEL=_U9PJnK*^r&q*44*2%=psHA zQ3UP6aY}FS@=1o07aYNdR9XLF6K*6r z#b4U6gG61L7Q#7VKB_9{@vf)6y;SvExzqVA69&f=@{`rjfZ2x7XvOW9BC&xEyf~k!J^?Le(z*0(o+El;h@^tGyBBmdL z0$1b3Uo_&9+#|JPKgIp=mD_TBj4F!y4O%{C=V06Znm90aJT`p?Y!tRr6Pya?@YBxB z>nmv6# z%$gBQ&WaduCH#^ngl%dTAxv#~NwMaW6HM)fA`N2z;<%=nO?XTrmB@0BRr{@S)|{P^ zf{Ujt(W%Ezp|+|?dm$%mfvN=`Kg@a;(my(C@M-%lwa7!DNi95I&QLx{!m_>!sddT+87^O3$KJi&VP>%8X z)!33RL2^r(uGVe=?qS9{?|3`6xW|d5D_w%Eq>zOnCE(KxpqTc`dQ0zim|ZA~TG5(9 zWoL+C7<`aRHIj@A-bafnL#AHReO;fH%;2(ndjVkiO?01ipgXS*mr~?paHv`3TbL&$ z#0_NHl=PRz+0K$$Fa+sm{L=%W^jjkS!<>o(x%_6Lqn982g})1R?@ilkaNls0z|^TQ zRRw$rT6kO(AQF%U7{4AMdCuf0G5g^15D1j9lZG6c0Hz+S&0-_nSqA_>B6MudVyjsvt;sFvVTr30wAoBA-xlR%ne!Wrh)GV*bsS&rBVI`ROU|Aob>Q#xN z^wSdTBO`!JpFM#>>kIv{51aMyVaeGYeQZP9Y!uxuS5ce?gA2uxvFgvzu2)wCKG5c`Ky!Tr|F{qrrwr#@k#06nyk^bqXMEss zN_4$v%m&v0V85-0QJ1sGfsu#4s>X>tVxgZQ-oUCA-!m?^L|z&3l;^7>-{oH`+=h+# z??8PIHdbv-F%bi;^)|XZ=9s5au71sx>>UbdV#HoJj227KlQ*OMMx9`wqM+eji6gS3 z;+F{`e%e0p4;&!JYF#RhbcI5ms$lN}VlIX$*s0o0f$dz z;JUf=auLFdh6@4B@C3-*RTgpivNA7v*kOIf!m5snhRP9_&MSVb?BR}G+ zaUPjc+l(LT`cC8WV63x0nU!8$ziyrW) zP=JpgRr)D~;v+WBd5~`P?*B zQ9WKIiNBFnDBq=z9r$LdfuQ$W_e|KfG(IAYU*AX?or(?w!e1v}bV7xOg zqwP-tun$OikE>ts85S6{m z0KmW?S{PVu_(WaRJR8g;M0{r>r?YwFh)mfXeL>}64(^buh`QbdOZkoEeI>GR*I_4c zp0DFlFAO4;?@oOBu>dR3ydb2-u;J|!E~nQe_S|eu(IxmNGs{zr%Nq7AJ_>*;I4Xbb~mQ z)sBXbBS*HqoRYQrL%&l?;ok@P2z0<*16$mvon6?L^?X&mn%dlm$9XQLZDq5p=a~b2 z|L90-;qvGc?X42QY%P)%QDF{U_pQEir}#jZA-N^VlK66?4PoDn_~4v2_O8n@2>I!` zFmi9H@;XqBZ4Q4g-z@_4TnPu|hW><+t*L38 z8IJ;o$;t zXc=4-ljjs-qkXuEcwm)5SMph%E3-YlPmV_(!XB!kYC3DFXw1`?MtqbldHzum7_!L za)0raB(gOfx0LEDM$yqE=`|)>i>7Yt{cXuI(vYaD$j)qh>@Xe`E0xK2qHDxu))n$L~<8)^$kY4%rvdy_MNgmON znzxDz>v6PwH~S*dGMk16#`qe?e?BS~HoDHfpb2$%QJKf{{L04*oUlN!)8N>HZYpev_X&sO8a6BsKjqD@8M* z!@^{Ya}H_@TYX&S<+*qyRhe!I;eWm8Zzw9zaCpkiZ!o!&Xg!`l**!6B8<}ye?I+J? z+HLnYWW?|@kKFW+-Y*Zv|5NlJ|f7g4Q(_9#a z)7Kqh!b6KY&(<0JgRLL#5Xu5yZZ9l`Ysb&XB~Le`-#H!9|LJvV zJG20Q^FL2x5pY>1+J#P?Xim%3yovhLi-X5PQ^xiwcQl51!W!c%1-U*A)Q6*V5n}E)T{OD6o}%nmTx~p1AYI7*94R9y4EGw!hxY8oPepdn@^vy0qspFi%nYn)!L-BWETk@0ZLf(-;G1yxM3wNC5K7&5_pK2IpM$LK{T|4_2{$-wpCLCQ#%a*F=dw9CuKW>m+x zeWQ>6i%#q5Mtz7{uhs@YMCd5wQI{;2C33+rwOD;+Z&yV8y87lBJn5Kd_-2|fXP{20 zuhhKY+tiGDHlu*%Q6(ZWw`e+whslf5t~Y-%2XrZGI=VJ*FdEm}@<4i;uAA_Mr54`# z72C@B&4yCm>ux)()%%kcnTB?t4nAKm_O>PtlppuB5cX)}Cq>LOFYOjr@k>r|_KqVE zJ8y|Bp`bh--oc3poe=Z%^3(M6^mf!ZP>1`UC600!OJdO8QmQSTP)yX5_r8?T19SE^ ze2in2Hp*Z0r-euB)9+!h-fm^Ruy}*HUw!w!ahFm?S89ckVLP>WCQ#A&Z`Iec6 zrB6a^4tkr)r8 ztVqV2pB+iWpC?pl(qZd`}uX8o59dVWHGnV1^Z-{57+wFyL~8DLV*fe14C-{%w}17FX?|D66{0R znzKWeVR+z{EyiB^6}O?Qtim-*sg$ZEs=+slPJTaYMV|M(@$0#biKG=H*P6HKub#fc zxIDN56+~MXSF@r|_ZulgizgDpERv~1h8wjhj(%;;<45WaE7TYUoLWr+GAKDZPtK0pvhLaPo_XHa#AHC+gm;xI+lJ6;k#)V9yV5vT-ol{LZd< z-euFmSi4BW!nDfhk-ecSfo@gWVUGT1aUu?|s9VrOGzoJ+-ixoU;V8!9(o8H=Cso5; znaeNW*Xnb>POakSk*??A#l^GV^mIMM^-@Nji4o+x(f?LV>7~Ec#VgH>uTS35_G_Y7 zhrrp-_GyH^W4Cu>EbhhlwDY51qyZmX-|Sn%bzG>$tIsAr{+K6DR!D=LB;vn;_gR9(AQ_1ULWOXeN+q*u}F@6p%xjdV(Rzk6(J-)qGm z4i|vkRO<%z1Zo@VxF{~tlyRr&i59)rXj*4iX~vxd{_r1d?EKb;s5_!do@{Ht<+bgo z`E$r^&b4pO0By8CmZPh?B$Pk+E@&zOU(w zpg7iIS+8grWPudWh{I@PCwKB}2b?>aYqWm&BS;~I z(h5a;W>p$6N?w>&qSG4yjl7Sdo1ZXYuHF(kZOzd_j|W&588x$4;lPnhdy6nvYvt#% z=0_FsX|>i=z8F?aVd!UHM~$?KG+lVSfCxuxf2BgvdXaAMM=}t6N8y{3DA5m#!12x5 z)8aWk9vx9Uu3@*Q)2QH7hw~zQuZ3>9hr>mg_xDIBzzKtG@eALV>m@L6p2xCYn9P#< zRGAx;`2UIqnOcjObk5yvKC4D6~U2*s>RQ=n)&)^IE|I(uCC_y zT5*Ra`rDC9Fu$|Zt=ZL4&o)GYn}yvtX`^69Bda~MNegO+4bdcYXn@w8Ef zAfjPc7BS4|0$5P%b5`HaS|G;Ox~r8F+NHYw%lLe%GoAmkR|RCX*=}{Hd}CgOfl7Bm z`N$+(a^oTr+23>WnlEYG%F;JYfQ>iO@;?BmKv%!|EO+4mT;UOn&7sre1sPaIke#tk z1$>B+zAnE1LsTKPJo3>c8>Z0_U0qv+`D%3JI5C}>ovg$GG+9B!o3pgEIKvde-}o3` z@-TY48(&M~J&89=GTsjxhWA`@lQ{Cog-_~fSn|mxKPe-DC0S!wxF+bnga(fD96Ar@ z6W8VUl}W)`vfUk|k-Ryr0u4Ckl%UDm^ES$57)rYFy=wW!h8k%HL$;`r4Zlb=Ru>9} zAgORI=gKg=w-t993qb~!ui1j3ruMP|y%9F3?O+@@=VVj$T&2lYnsYAQFjO~Z*=FTi zQH8G}ufobj;5VT0#?k}reQcu)5N}5hX*(KPw^*f-&hf2Xu}$ZJ%1TT*--I zl##t#%lR0toWFddtbkD}K{-{UvN&(ar;c-vu{v$;=1w{1S%vca!@rkFe+)nN?a@|# zdjCvWgF3wD|Axye@2-)%rft+=1oDtAm>+{-oV&P6PWaXgnexX2@WN0eKbyqwh6Rgy zs$4m6_X)^XFgXEz7I>LZXQ+?P=9)ceBYbW)Rw`Z)Aldqj6 zum5^~=?UXDpi4`+?9+W^$?8oqcf}U%P1CyR5M@vfe92J9o97MVTx#9dSmD$KCm4EQ z?o;r7)%eu&rTHV5$Zq|_R9&?D((9!J@XYf|D7BCNS$4iT4V8dj5vs-gIv^ogp9s5J zLmEMTy6^XtQ+zBQ{aUCcZ8aREG4TTt0{`WM}P!R}c!^%~6>#J+z zlE+udoo9BIub(?W1uVy+fInXNj=YX}r{VqE$z^|7A>X=ZI(kD$MvJ=*280U_q(vh+ z<##jX#cvIhCok(GCqJ-6ZkWGOuA5jX_np@jb1<#twN=}sa|stRKqd{r>}}esyW-Ry za_}!_V-d?Pc?ECuSDsLUd9U{J{=5w`fBjB2ZRE6vm&i|`$1UhhUD~^a&a<8Jmqqe}5yjHK3r<=e z)KUiZ(1;hC;x4!z;?=?Cs)c>5AUFh zZN{&vmH)*sB%fj~1~TJFTGr85o?0be$Gl}N_(~C0bzU(*+O)vd2N=hRD{AE>7`K1D zyG{i1S)W|YBU^XLcmFh7&OZV@>R~PA?b#^T1si4OjIFZqw4ItisoW{IJ+o9_e17rV zI+^g}AU-9i~CImlF-|wuTkf&(sb~^sDWrO z>H8S_CBVAaUe?(~dFNvKm>?0q0BFHsDAbbh)^^rTL{Xhp|TTiqx=vmho;!Od2d0rHawl%x6nq08cf9;TuZFD8;W73 zcd?>yJ?3w?@L~;ylKORRtt%I2EUre6Xp0;&q*U5tXp6>a;mS?Y8%E@S0r==5WK5n{ zDQ#g02KOq~g(Y*b=wlS-jEd0v;85d?MOD%PLt+P_m&8?)q+h#Xn;bf%M2807TTmq< zU}V{wd3ScDVh40D()p_?2v?(*Fk&EvKxw$3_geG@Ct<-1^Fo~}(SxCW>(Cp0YsPw= z0~-bWK&%vuwk&XYFjmgNip+#QOVJbUfChggbmEX8%Y(*?D=WDYa%dk6AtFDu7;c+9 zcY}1pB97i&+MqYG8N+@x7^-Zqy}GF=4=y(!Ik;42EUc0RE79}oSFHQg%3$=HVr-Jb zkSwb?ORLcXWjVFgEg18FJqUVoh352y8)PQRn!}guU7}cJ5_F+K?}L{Jw@}7yi(rp> z+M;TgH|X6Gy=9h72b2Z3Z*W!X8`EIeP*_?gsL%a+7iq6}8g%5!#zXh(AbiPre_oaB zj|DK?Ti6P^mB8RG4$5Xa>ZKR!84HQ{@-Y`ZUT%#Tg!#S}z_RB)a$t$}?k3Hu)E+B! z?}0q<5udYkqf7&jE-1T05YJVw91UgO@J6H0Zq1u-$|igSwDFiDr1-I?rD=zb_*gGi zw&|E|@Iyczv;Hp9($eArR;*}lXTh&Ss&4q1$j@$&roZ`(wEX6`WH);HP7W2^QAj;pScCO6+Ct**XCc5|BvUEWndppJ!+lIBdW+poQ6>N+(OojQz*WH<01k5!KV|}Su+3t zKmbWZK~!<9E3OU$$5x3AHy0-8$>He5*nTmx=JIs9T^RSaeN6h~QDtB`4A%tncXaFl z>C*im%sAUH?Dn2vpBo-dof|^~!VfA!J?xuu^u~ko-Ci*c^Fb~$?>a#nN06O}-$^{7fn!;)-E{lIERtp7ej2D7h>P?kPu7j; zCQCcwHx{MX&ra+<)15$a9SJNB%fO%ITiaz}} zL=`d}38ECAJc@Nx4p|pIp6(QKoQ!vpZ=_S4HWxmQ^!1ClW%}Y&@lYn(kIB#lnP|jq zM|Z%${a>bmv*#pxUVZLSwdI4)rFU9dhVF6E2*s%i!N)1%jMLK6;&M^dtxH6m3+v>@ zr5m0c*le)9<{FOhSI@4oRb(V)Z+w3B(oe%iLxi5J;GEptCZ` z+vi0d%!lvDo6eO7jiO8U^F}{$@p6jdeErCaVZJyRFUF}uJj<4jbD3(uG~_ORU0HoH zAHH+sd?`xgLv>{N25;9M5%o{TGmc-$i03IPXOEnPqjjp92dA3E(-C5#%UUccHNZUO ziw&Vx5V&)4BB1BZQo=|t#>tN(7~)eJ#X4WaD@q}Klxm@Px*;r;cjqFGkuOfmko(0r zpYe+D#>?H)`ayB5<8N^h-eO!SDau5u4o-5YO9+U|*HFMJl4O`jn#J&Ll3l8$MAXh_ z@ew{-OcGBVU%L1bJ;dH~eMDkS0%S;W_(eT+4vX=!1c zw%RVF-k@P-OfFtVZ-&mocK2y5E_o02b9E6*4%1?>*>y_d8BTK+?WZSc({xXZBmZQX zC1q;8rmK=D-z23A`+E9_Z2F|eC!Q~#5B*{BTj2x%s?D^}WywdFp7i8{_k=iEMufY0 z!8fKyEaWlp4(6E6W*Bk{O>p^u;hTyv|>a>fJQK$%>B&}}#nOL7D zN;PqeaR?t5rADz4Z4hS>5k$3(Xd~Vh=iiN-an2v%J(fO6=uehQm;eul)>)T)>Em5T zc~mpUHH|kVbLz<{!V-_POT+u}h*esT2WcV>L_03Jh65>a^FrBiG%4s@EU%6}O_%ud z5Hy#p1Vl*J?F8Ha$gC2m?hvFo8T8zpFfGZvSL5lbev1& zHI9p7{06~LY^1oLi9K5)^;kKqX6Dg!EwHGIa}|Cbt$}j%Y!#d|cQ?uIZXu^H8(sMD z$0Oi)93ExRX*})`o<9GNWKk+<;vqx3Si`E`f%1{wq>j@Y-le!}$UmU>(?a21T8Ix5 zV-!Kaq2jtQE~1X>62(W{;&`Q{9{6k3q8awQHbq@xK{@VRh{bq)fpHK|mg7xbc+wH~ zsH$;PU0j;!;_FMlX{EG~!Fw~x#X1Srr6Kf=2;Q3qIA=iBn#AI%Cvi%nnKdaI$f$qD z#u4!t)7?iApQH)r;PkhhOeN1eT^gy|k3a^<9YqH>$C3d$k202yq*yoz_;J3UtYTpX z7zh4T#2POXXIQ-l*nD&e@s^)}QE+oIRWRh5FpUi^S@4AVyaN`Y?mjD%<>M_br1Qkl z8`1+vEF#&#~(vZ2P@S?WW zm5r7iq6ksm3Uxd*GVlh}0TMSl62V#N7_SKH(%scDGOWdg?7=LeXKIwWIdGAW)rghjK+fhahJXi+y7=k%)t~GGwAIV0T(h$FD`ATV5QiA=r?rc`Y z1}{7;=UrIj#TOvmTESc#C6%`FW3pQ+UME)9|2COmO${=C|~GbqVn%FNd%4?^Bbi5Vbj%kSvW(NK%S)=4?j$`)e%b%FZ2h0=GS|WtT3jlC&C(kqh)&N zPw+FW@i!bhFwGA;nDRG#1~1}?^Y5NN4X@%!6nF6-OFXTlH>F&+_l*5uz5Hj= zS{+I=dg{nL=&EEZb_ihDc+qfD77ZBX>B;+uYssD=Xf+;w9?t##!fK6UpF(7s>-~ z-zqyKFRciBOk3fP2DPSUt33ADqcCW-^1uUsl6iAyqa697)C}91nxP(XiWBN*^%oU=;)yxY*Hl~?w1HAMWLj?XOfum|TC zPn^}Bg?86s|1Szc9rl=Hirz!MAj`#z8PAozXHwItK1^BZ8bq-TsY^rhj&e{xzCv{+ zo=%?`e4&6&-*DB}u$O#wr9RwtQiDB*G*mQb-Ow{ad#w!-_g?a|Mf_?7_iwVXVy|W~ z_DOcY2{6uBxDEYfZJWutM|DwyLL3`>o&dwIKJZ934l8nV{u1mzWE$nrp7EQ1Trjj5 zr`7P>(OzY((UYiE8t#u|Z|#kl>t!|0lD+7}9%@iHNA$#mRWcF#5|25c1NMk+SA(;r za$GxB2zewX&j-F)|&I|32-Z%>>kM;>*w@?5rTxs-M8f@i0x)NI`%RaG0L zM~`k&US1*d=gyH*?EmaPaFCwJf7V%NLx-07X)SW(tHT=f>Xt5Eq&>TVg9hL^*(Gh- z7E5I%UYO?3*Z7`2aO@c#Aoc`#M>b^H$~Cfl*%IkLV4#$imgs>Ge1_@jDi?<$n8<8s zKi)A84P5j}=vd`-CBHyH!}D#43%J{z*j$c%`C+ME)fpQmvDBNdAt`vH%eWVUC9#k9 z{TYmd2p{A7)9`hrCuw}V?@!_^oDXJCu%z*M`+nn>yyt^4nnJLo@$tUj_$BW?#`h<2 z4A+&Or19}+NHBgQyOu7N#=|;Gqoz1zC3vnwSm;SI$`URFPb{1h83ckbRbwxt635m( zJ#nq{LgUWO^YhSL^`caQ*?q!UGSQH{B}l$E5V5}8)PF484bldbF1VtFo-KL>^KfR zzaiaP%PMTvpFF=>H{w&TDL8%RoFlr(Z16a4c!?TG8WtW}aP0o2@-G~EJ+l7}dG_r! z+WX|^oW{@CAcx}%6eIez(_YwY=)jY7rlLo7=3!+xkYKA!244=b9xX5&DB1#7lq7;U$V8PP$F zHxK{QvJQ(n^k^9|EsUWjcorCGwlIdtqza!$$H%`a*t-Or0y)7gG!C0hO5U0Hw*2(^>*bC+ z?vMrZ=E~sxhRIKVe4SkT!yn1uf&Jx$SI41;wNVZ^Y_wec?QhFD=bkH%{`FBg=QHQZ zVWSV1d+)tZt^=O)Z>?Ijln|6Y3a z>M7&LkC*RXbFB;+GDNm+Mcv@jH8W<QVpH1IJK7^c+PJYl8*O>URkMz?=XBzum$uz2qzW2^4>c1@|wz-Fls!b z^`V!S|7t7OkHnZJOZ=9@rDgXMrT_OYPW&^K_fFNBV5wDsV($SCkbJ@t?i!>C>m7 zH&!Yo?Qs$kjLnNLy&wk+A122hcf7nmb&47|_Utac_SrQ>MzH zFnU`y!qZ$RJLx}ED!wYemU*5Q{+gT6!heiPs!cCzf*4c<*m-Ru?R8>ijkS|gas*@oV+B{ zLIbB%d_Bn!lw^@R+_+m;Y5v`-q*2FC*lfHDZy8`{8SwAM#x&N7^VAFD0raBXLy1IP zAQ3PztntjaCjb_-yWuUI)Fqi73xgOeAIGlW-4j*nmwY9)M66-|xhx z;={-Uo-&dysSDGy)~V0W_k#w(P%L?9I1!l$5*BkwU99YqbjCX@7jQ{p$k_Ptj8!WPyLG}bar`hak6mkqMv=X@ zAw7!p_%<4vK{)D;j~w}Q#1Q4DVFWnu#vUBMJi(#QgZj4BkMi=fNJD$z7&;UZX@>S} zgW0(IcT!7NC ztV|w$=s|fMyqAOL;GskC^0i$KI_Mzu-f);al<3p9ue1hTHHJRhW8ujL3|+3TTqmzj zm>^@vz9OyL;EdQ|`{RSgcqwY$B3#&Fxz|kS#bVsu^2a1giLfpi7sEiWhN0xyvIFo5r8W2hNCk#B%V6l(D;bU6SXT_Q&R}m5#*kl|(B25- zEyi5i_!(95)zf;a>{%Gjq@#TH7U|l-c+N%qU>NpV7`Ztswqkg+M2Ayh@6_14hl7>O z@(#)zK^kDjMlLfHpaC(T-J=+zHwhMwkS-<}HXJ!w8lMWTjw~rIm0?{ElxJo>AbpC5 z;{(doT3#HAENb3P7FJD@!~0*Xt19)ZTQK0f-QvZ|WbT|}V#x_{A+S zV6V$5Cy&8u$RfG#-h1VUBaYC;J2zbaQ#s{RpOUjaeWtwl@;I42Yo;{CVh|cHYQ?#< z?X|V?i<@tjGtM|ePB{KpdFttBWo`Ld$ZH|*PMIPrSFV&Hn3G$DdsS7H6k%R&V|A4j zw`(Wu+qIW-&OS$bkTYh@kwuFZV^t>R_V_ysoaLQCSB4g2cSEwI;-wtN8wVKwh@>tb zg@6q?<#H%^J&Im&9ylNmNiD3pEhR;L*~8~ zidSwXJoZE*C18dd4N6}O-%VY(QHSq1@3ln@z%Drn-_aX~xwn3uTVlb*R(vb40}cpp zE>FI(Mqa{dMh-3VtMYBJNP`PWieN0+o8lk06*R@$K6_xzf?m-`9K^jAUqgQxwCwG0 znDRgz<~+$(0&>WBX zvTf*6a5U$wIXGY&hnp{2TO&seZLh}oFnrgL3skzJhd2kHGUCwZ7R=4m?sP8~o$%?Y zjxfAaaF{r37rvB*tq2$BDt2fl* z0CDz$m={*HYZ^5pxVX~CRXr|bX?ABUYa=z)5)B;Ov*NSh;&?Dc9=Xzx``yku^0RW^ z2S1VGt@tit%TC(kVb88&3l?qcSS)8BdA`o>6CSo~2jE74 zgfB})CQvS@vT?CmvwH|egvIGy38FaS_=}-&GN6V-28a%5f?EvFG@A`p5`<&&7-0&JE(j~22$SN!wV}!fuu2BV)i;oY~)SK$+^C8e2^XZ3!c*0Yn zjVj|_7aq^I&aZ*68ebD=^s^hJNqE>Y8!HH-Jn-27qsEnu)!5V4w@a&FZyUZEgx=b+ za_l?9lKGx6D*OTi>>*aDVZ|lz{KOK6H_^e;p3w%ZR4m0XrBxNbz`-ytM_4nuOzj(nI!>ShLyJrB;Wt;LXG2dr%Di!VUXP_lo?MnMQFwO`JcCOLGjFzdL)9{O&(D%Dd$_1RRuHD05J!v*fDN ze~2&H9|A)YEaZa(9%jFG^-5`5Tr928Q`3XwVcppqD_^@>x^(T1xE)eiu}(VS^GsaA z&+zQov(%^##0tvQt5#!Q8@}v;GH8!|Y?bRPv`5C?;Ph$J@UqlQ`u6Luy|xY=Ismr= zJa);V#fzm!_ioa&S5Isc*e(?nYcUVl2@6!3%gW`e(6d~u3sHLX?4e&r@d_mHMx_$g zk*37UC8CSt8P76H`I-P$-|&|~{GK?!j`oB;Uz$SYPg|G2A$iv&JDFZzy0D=|<4seR`ZWQ>(-3i}?`?dli3W!IkT{3MMxSjn zzc|miE;i=1cpKI)qJxe-7B2ZG?M__y)8-!=f6h~}5imzuJ`eUGUGnB$GByP?W*iD6 z9V2wH3eveZ&4vd#q{#U^4v&(i6^w=UmcWyXLdbtOK7;h?2NiM}R({Tdfm;uw$et-F zc;y~9%L6|Z#eC9{FCBZFjN`B+b*GVIIWUhzx?xh6XJ(Cac+5L#$bsoxlv0YV2{ec- zSH~IIq9%wfW#v5(At?b={z-BU+kTwyB@{*K8?7mEjV#|X#XVZ&!esmYfAh>4Uio)Okz zXigNb>Nz~aCYOAS7G8ZmS^xaV?QzHh4Hf72EKNP+Xu1Y8EAjWn}KjkQ1#q-e+1T{VKD0|>oQz<9d5ON(C}Z^l{4m;~O_TqMX*IGu>p z6kKR9+O;W>!-tIy2dB_!!V5^SvJzP1P8#A#Od~^{bbO(xLC=@QjQ(xgpeg+NJR!UV zJuZC)K<9RLA&-2{fN@`LA!oj{xQ{H8P7gIO&QjF) zq_f~dObnmDx}0M8WCU31mWwV|I_2v84JM}f=Eg>4Z|aWx_J+Q$ynj^buDE%H3vO2a zVMS;XdS+an_6c@;mbh|~!&vNfeiTk0?chPDUrRLfP@|xX6&M7~Z)t?V<*E(8A$Hi1 z34*>r^ZVRL?WiJJ_x$gGMdKC-8X+}=Fka4p8Q#g!GCqhu+Ok*90KBJ`O7V}$Py0YPjOMvu}0 zI%Soczd`Hm-WWp{Y=Cw3PZq9hNLg2$;LN~LUn}4&^cJ0~K{iabdm*&hux5f-fbO=dF4x4tD&Aj5D;+08IaK?B2~8hu_ty*~}KdpN|AqV{@ zCCTy_)7L8+rdu#;b#e9X^o}sb)#>MSH+e42<-&54skuvW9G&D)zIwutiXhMI&am#D)Qxn8 za6J4_R7bupaanP^abqlhKW5M<*VR#kkJnigAMy7~#&n5N<36Ledz4d)vAd(Qkh@3d zh^rye@xE?G6Q{7U@Sgxf+C4ISq(ymIg7$lp{KCZg^{|2CveReZSSf#ac9BjpYwMg4 zqc~{DCBr9nPp-ayw!Dg^-%O)y(i0n~<0irp z;35E~IbO!kq5e8{nQpM9vEjtI&ZVeKho6CszgvPi5G%7#KgBuSxIp3drxxmb3-jLu zW7pic`_KzZWg(7xy87Okdh8SHjN|&ePoBMAuDO2}7CQuciy&Whz|yFB%c_w!M}F~- zMRM(fbL6dQhfK`Xbal zEKGk~mnhB0nBUihv`$Z#8kgu&le)NAyBb#uNB8V05hl_lip%B8txG1Q;gcy*iCT_b z84{n5-sDFteqjaS^X2Y%=Zki_P^Ut4CptSmo)Q{3X1HKiC<*Yc5p0Ze=eEhq(>KV! zrd7#`wOi1rvbO<1>{o!d4D=qjV(Zptmj?|nCrW{ZvYd=&!|4nfCq~^WFrJt|KeI$5 zNK6gp0NCRptr`-@A%>IId^2J%gMN&lXgqZ>eq5bJ90XiJZgu=L?_R|n~sKz zI#C~P(44Vsi)=>OAeSnSE6e!EkPjCF{QkM+dU`y|RP&8HG2}s`O~)QyY1`&9c@fU_ zTh61AD66qdnzdePu1*bIDW#IAaH zp`3qInf&nlesaYrJ>=3ym&n|uRa$r-U^B1&y!=9hDl8{Xdb5tyXhIMTDa-Tre=XNM zyLv+11Izre&gc-to*iy<#b^K%8^SdJ+r4G!YA|>J3(Pa$-<`928Uw#o$c3|pDmNxB zfmnx+bcrM4q5O4mbb0Ecuq@b=Xp73X?i}8~#&eYn88RRKKDHt7R*vNDWF@slI3YL@ zOd)VlS@~t9V1j!Fa?wU<&KC|*I>VB$dVv-Zx14H)iNw<~1r6xYgNx-%EIWRA@>;q0 z)V^wjo_uo!ju`5K!&6(Jhl5RoSk1Kp%b@>>^YjkJ@jI8F-dh$dua-w}R1gL{5oi(`7p%JR*68vEyu z>#njM9k)V`8ivn7K<-00sl5^gryut7d}&M%X@fmLTn0_Y6+J_Gw84S6JQ*NzW2dZ< za_kHG{0Y5e>YVlR!IDjK@yWg9mC38+*{PLkypJ1#ayqiB8neHST_JPJx5>b=mb&q` zHDqkqxJ~YOW{Ff`byP`<#&Y>-y<`&h485{=v$T70h5X>N{bW`77P)uqDy`=IyS9?^ zkMFLl?>MI49D8j(cW6g>Yi6a47+9(X_O)4VV=eZP$T!aID2pI--G&s~BRT|Ph!7dNMtVHF!E#{d563OV=aZhE@- zOOsYhH=K;oyK9mB>G`Fys(J^GAZjdMJEND3ow`o`xnzqBeqp(MHRu zyVFOum!n2>mKWYxt%hv^&QSaYj+^Sr<>y%1MT1yxt{z#%bs}T;yK^GoXOR0zId70d zuA_3jCg+cHP76iSjKc0|XrFg3oaG=lCN3d=-j1$td|Y0ZkgAN!Zude=l$H_WEjwNv z_)+TTMbo2_B@c5B?_Vt?JD$zBqcCjx-MFT6%B0DT&(6Vg$*K=YJ(mc2DRFOBey;T6 z;2>nLIN514$z0a}@j4{5H>)T!d8fv>VZ~iC5ohQfJ-j1M!R{mz7xGgcJPEszOq^34 zPC_rnS#lfDs2+t)oF{c|A?F^^St_fy$qyb|EEgQzMZR}#A9?cAiiCpvOA{d4p*f3j#278-)|FiwEv9(tIGJcI* zb#`AF-M>iwg+py?pwAB;TO?y}cr0nT0rfup$eD0{uvI+aO?m@5ge^2Tz-~UV>`R7~Z+GAXa5r998TO*g8*i8=)Cjb0; z1+H#Bs;g@ZqMxh!&}$X^=TTve@Z!-3z0|t?@b*+E#&{HhB~3%(8VZw7C*p(!?%Cmadm}LX*9s5&klt-^^iieETvROwpkW=oe<#iAC1bkF zqvKaeU-U9MqsOOnLMT0+r^n&G^I>preP$_)(*i7mZU^H~6zoB9CB-4N(LGzs30Nx4 z&swlC=)hhM4MKO6TNyUyE`wp_-(mfVn5sjloZmVadx)NWbB!E6e22_m zy;a6wKhxL`D*b0kpEQmT+9)B*h;>+jmlRB+>d-0E_(1Y7U45y7pC@6kCwVwtcwU6D z|II(<%kbVsa=^e6#j?D)k@x-iRkEm}78`~uRBknT*8C(4UnH(RwW|#3*-jU;fTa_n zo{#&5rGY|0k4=5}w$`q-6|$RWVxn#sN5|*I{2Gyn^V8|4^YOXT{WM1zmzPXWhDqbc zCH+kQSnI;^aePf;H_BceUeq|15|w2N0=ZSp$ri5ip2yxllUy? zNKQcKc;!O7Ix?9Vc;yr2Z?`Rg^C~VEcTEd;Wbh7z`t+L5Z1{vO+^ERv@E;e4b*eBqU74f{U+=YA5}0CLJH7yN>TDui?0%fjB9>1}$N$ z;7B0O0qxq|Opp0-D!30CRIDF?s&=OQ`($TB${ zpMzoVk}Fv0c;)x8IKMaMfcEk#PMYVpzm7WqU+0BmuX;0%KI+@82!?X4JU)Jf4DQ}q z%5Z!VkBefN(9u|N+3^tc^xi=aZy=0~tH(z2%p0pPdV@5}ZMJ0fuXJVF~hwYQykvb2$+w+B2>nPRZa&8S`-lVXJ@_4QP-x z<=r3`@0pJA&lVp8r3YAVje4Rk#{OHjKJfr5hP??g^c;&?e?;m*z?Hp2SX=_npt4CQ z$9i)=NRJ?a@lrCpDlQv7MHey%qZ4#{tBWd^;2A0$3JqhzVk3zb4lwxov~=#Jm`0fY z){W+kz2-5Z{>1yTmDDFbV?hR(KWhWL8f*aLMuB|3;Mf4tuvBl}CVzgbLhid_h?E8M zQ6u`dms5W+O-?*wSigLFsa$tnW0{OE8?!MV07Lp` zeB$HqpsT?{WZ6rh<82v+*)Hi_44qfX-^VVMgV1~V^|Q<5%g1+>Zs}$5+?L zBi}keeutsUzl~jj_+nX)Pob1x@y9HzguG>Hm0U7ryNnrGDz`qhRK9z5Z<&fYwqL)q zLH_S-ky$uc_5=)VvNtyl-`Of_gB}s&vCQAb@Z|`cFE|%HpR#r>bV1J}=soX`-sAxT z&_hC5=@t>Z5*)H$2l>=>@5y)eD?)F`86z5!W%!6CmnRR$;l0QFaE>gzZ=_c1!b*1< zg=SP^cFAXs?IxGqGY2mhOJSVb>!((pc)v;>|Mvc}bk#=mj&TA$dadj6k}~eyHPQ_q ztNi-7b@I+-edWy$%GJnUer8`idvPV|fhRJo-LzBQLfUX_5N?egA%`US%`<)eCgw!%OKivzSftFiFIg;W*I_uG zJE=pLZ{@N+4eR?uMhTCP@%;&y2J1>s(ljJK$vYX|TU~tm{CTrIeRIu{%fQ2MUTMv z^v9cdXPhqH0mE$U4ZJnI9G?o=D(4;BUH4c08wabh_jCk?CPxk`l|?Hz>LIR&p^@eS zgEug2$>V`a(Wt*QeVx3E-ccufXz^1RS}Vaa4p#_TE4QTGGUV6z9>4f$jqD>v4l2enR47*(dr@ZZEd z<#9L>ok!)cgL%xz&M2ofG8HQkhxaPd-qk@k2$!o4|2BT5p5ZsDe=*8;qnvhF7hUQ< z9;+Wei{Z}2t2W`olk4Olq?h5WKYr7T7(Ni{P!-I}OUN%zcpnj*Hn9??zRz2^Nsb*+ zrt_a1HXVZ@QPPdWHitupl;DJgX6UIpgU%u7atulGyI!j>yxI}No~Inr84Eu!WX!Tf z@9_u>`z~BjEl;5i2jK&n9gACFkx2_3f_@rXN9L`rkzNQ>zjj!J^zQ5mnT170To}Wz zPd|;rd^aG^#}6;X%SQ*9IAa}XTk7z1>f(|@pWQ<7K@k>u<4=#PRyicS_@$RIM2sIK zdJp-#hE%jr-i6}x$Ysbf^yEQANTGPgmQ;<=ldHuStk;*~84q7iUAkTOjJr{Pa_)uZ zvrv2?C6O<0HfP(hj-ae#2kF=WFYuv(;~PJQNy3+pLZlO$hXypIY~VSRwG%&Tlf`&8 zj8Gtm3^gutp)BOW#Suqhs9aIon@m%l#b^SIA&vto%a18xGPl z*SH)KtvC5_sXxa4LQ<{@WTT|In(|2HQhOcYRIDqA-T0xPv#2PSzqrnhDY4FN$C6Iw zfn`HlE&z!;EZTtHTsrT&UBGe}F)EKpK8G!-fewE;1p`tim;8Je)&nY&`6sRJt%FVs zlbzde&otJh+ejX~BWau~AIjyQ(}j0Gf3Cim59;lleu)?gkbmg3e%w$qu`GZ0f5qJfpY}g6VXmVkQm>nKK6qYlaO*e%$^^EmOT9AIP3&&0m;3YyII)I%lFM(bHLnRO-PwHs8I3Wnb z8L85_s4y~6H^e0IVY>150ReB*#ZL-ksfuCQAu`M}z^XHe=w%vzKYEL$ImF&yL*99^ z3h|zv*+}bxB-2CQ*%&6{lMIsSUN{#tEv-IvVcQXnhp@7P2f@>?{=JZ43ln_i)@V`* z{cZS(@JH!<=rS>k^PP|P{f6;%@coXBii%{MrJD{eAqvIh1x%8> zOm-|j3s(6B>`iuzE)pI3jK4+1b&BU9iH&F?oXKUH@7FT-3pF6ttrhTVhck%tNeBL01xz$A78-bmS1 zPF$6$R8pz3ldAl2*_EoKiflhp`H?@NLitCkVkb_BlVEI&gHkL8%q9a0AR&R+gfv=a zB+aVP?Btx=eY#KIcW!szH?K!es9QC!?>*J2M0T-hL1RHFgnnrSfdt+g2o+bQCK8C;ZDzEB4$)c5TuCq z<&HWKMKT2|Ts$Bno-R!JY&P7kXK~s1NV3vb!s4IZBnn|W?0N)LxXEX8M_NK>)Lt({ z16*)nRR=k(iC|7t?5G9al^_?lFFsZqmx|=3C@%h3L0ocSYtc`a78kcK-laY6U9#kZ z_rbb!C!;58KKz9B!MjfnI#I~2hw!l`yJ%{u+=`r3yrKn7id7L3$I|hNH8h!Bg}cI# zA;iL4)udp>shE;&tXdl0_rjFTW!J6H71dO{<)kY9S=N}+cG&e884xAvfyv#(?O~c| zfHU6`(oSUD;zIAU`;=7Zi%+&ruNBFi9=Xz6q!wRpUwmb6E*VSWBbTUf3RELIpS{MT+0rHZpztB#r;H#u)bN2I0#br>N)0{q5ChWEV$dDZIdlv|gD|A@0Ko~|8!;0AdYgTTA#chriOZFOl&7QDF;pshM#6s z;@9#J-f91(f=cc2l&SaLFEO;-wJx=tE=-2V!!#$vNjR7Sn4d9C;ut61mX~~qGKEwf zcpCHMbc7TBS!)YMzk#94XT};{XEI~?OfuzNIUM%}a+msh9aU##Jv-N#i0Ct6V@GtY4kKHpaM!T(4vKX@<8`>L96 zy}^5b;YU_q?tLouM^=b0zOn`mYAoeJdJ;h3%Gmy4(~q+G9H9>p@%}M}7dxH+laH4A zo%HoxeR?mYEVlp8eR8E}#wPvK0nr4JeUp#*Wex8GlM#qkSQF7UxHv zUe2S`)=K*nX4Z2OlZQ|1TQV#zcYJ=|qsmSYew!z0$2*e4+?ekv#9^CtA6TWGK&&VK zjIuhjC#^%(zu(!uDplFxB~VVctoE&4E>%D@|Dwv6x|Lyb*X(u80a;042M1++IroZQ z-z&j?v}!`W=CJ#uno^A1Vz?B-vO2t&IIWk`i)B9v^4}=&3EChrNLX|I@p&1DVwyu! z>_A>a^I>5Fgpg1)wMz1*hR{IDmS;U2nui;_B1i3W2J(8f5Vrzb)Ad*HN$=&xF^==8 zmSSTf@A5~+8!|co#|jdRLy(?7*kWF0FtekvIkyYEr$6!qiLJ;OfyBSUKfL<$_1kr~ zPC@@@>`mv>iZVz`T#{XFUaeuNMHc{;Ip*xFBo5N*$ZvABH#du^l_zEcYW|wBE8ZN_ zx6T^;k4EUf%w2$g^||FziGX^u6f;Xbq$T>pmZ1F_9C%*xr?k#E<&+aNj>|u{E9C}Y z2kEV1`o}LcFMd&fXZga?@D}VR!WM3|H{hBa$SHWAP^=;jmycJkK_~g2pJpZ^1}zre zc`xJsLI1E!l(s1(71hn{%K1<~t{1{+BM_czutVE~YI*zkg}kp_ulpLOhc2Cyz8fTh z41|!kt>rIPj-$x#txjsId{-Ch(Dfs$a^e1J6~8z!Gn}IOS6FH5*tf-xa(tz8JkxS+ ziN({uES}f7`&e;+kIT85w{6WoPO?p5$3K6}%!-c5$hT|C2`vm|jH$OXER$HJ*HC}_ z2B_dJw)s&T%Kg-K*hLluz_#ov;{-B%3R%*BZ?tCsf#`O2aNqY|$M;90UxnrtUnV5* z90|N*BJX|}r=}&|$Go939t_Wd0So6ZdpI97van<9W_9BgCzauXbiN)g4cl^xIeJ{k zx4ZJZuG*24xB>OAM;TIM4Yjwzj}L5B>1QYdDg08(Nd(&S$WPiC7;ouST_H+Xpi+b# zVB}PDzK*-EG?FeMMdD!E99=M@7qo-eigw#chbd5*9}LfQ#by(oCrKylyMh_IGlfiS)=@%gVvTzaLNlsvXxhAq^_R3`Wj*^xuTXoP-)ts$KsZqp>Ghu2VS zNk7$w`*@o_1(>XXmm|I}!#+4Y`X_@`sR3o;G@>uVCMop{UZd2YoC>1el)~`;AG1ymHtYCi_i#F78r!7VXNJ|o&F4j%3$+$M10t*~4uwrWN7z7*-}JrW6!a3g#9GeZ zxm;uoWmE_0C{L)_G7a63RC`|1}IG4)W1N3Z+dU)^Q* zaN7?4{;v7PnE28yA~%s}!?Fi|e(s1a^M z=Vz^JDyPUwYgf{2I2e6Y1P>lrn7ucQ6sOiLGod{5bEUZ&Rw=+T_1zj;AoeFmUD9OH z9aUNk(3wNg>AGx&o1K1t=9Hx1;cY##%vN*mcNN&~A9wCBNGr+9YO4*m-|k8iRrm|g zPL-Ncnl?DnNZo95Yg}jLY7G1}zfr9%i_}%ezxacj2l#E);7ZSKdSiLzZq2i=OLg?Y)u082sSwI6nzSij=4nG7$-t(eF3Az{! zY?rR`(6R`(SU+tT*rgA~N4Dg-RM$so$_95`<%K{#v@%iP`vhXxO`FSHWv z23&FZL2+rx3I}7{J7`s|>2R6r15FU@I4$CK9B#AJeQkpUq`8!v%--AbnKOg91DtMp zJE2b!b;L@IDCwjw{=v8_i1}?%u`b>4s>|JCyAIcnG^S1Z5HUX z!wa|4Z(O1$NDZL^NXkpt_HS|623DZXPGF#t4haR!p*#6aB#9TVho3?R>a6x>B|G-8;B}P*c3`)FZI@a6b3w-8gOKvFH~? z(jp--_3fU<%Ce>@lK%*iSTBd3Y70$=t%E-UP0TcWH&DR!dhM3xd0M+Q2Q{dSj|<%fNx zU;nZDk75QJg?X52slC$EV|U?XduLW1j}B9~2ZPbU{1n!{-k5Ao>)Vtx#)`!uax{OtbJKzgT)dC#YiE&92w1{DD+zh_RG}W=|*b1A6 z_Td+=%&32AF21gySx$`VH&YTmuoV<&V*;aoKlj`F!?WMr{8{A5;#ql*CM5G&7QOcB z_=Kr#{Q;%bs?xi|e%C%nt=W@8r6<4#NmOd0WF)?+lE*v-FOouJ`t{)G*l&-zde$P- z6aZgCFb^c2gMy|Fs`qID6KTcV)LhF0)yfbSH(V~!P{_M2Gc`GtFoX1-Ej2IWabmA+ z6B@6@I$Gq_#!02WRe1ZD*?8Sgk(P`wA?B6`%cf8Edbe!GL6z<(>$G+f6`RX1 zG9n*nC0FkesB`w{FpC9M-pOLfN}E@sx)ds807%bB4rK?su+c7!;mFu5%Z=-W0q>~q zn*#1D>H)Z#T>9NJZsOE1L8+Ge0L*sUfhvyXiLfsQSvQXle^Y=}R42!5>C|<`u+F|( zkfMD%4_y?ile-+V#a#rs;GhYR#+aZ3yUE$rko4AI;%DrY zL1*x`%8yWWU+0vJP<}w1^*6`uYO3MWP>533M>)Zd`a1oVxal{pd5(k~ye-hrPt@un zefBOe(oSl5_-=o~k7+*O49)qQREhd|ua{LL;Z=^VSaFcvyse+{LrFx-)b0&cdG5vzdtfL0j5 zsWZR9k+liCW5EEmgpPgA+j8u6Y205&kx0Ry)Zl<|WjEbFPs4l7No;&-T+VF2a7#J% z3e+`rJ{YLMh`kbVCPO{VMOd3wj$aJ0cnA*1%!87yFWX&&`lPNf6fh$4 z1<8GhS(^%D+F>?8=o>cl=jH$`+q)$WZy&!&>Nhd4PGxdhHGXzLkG&USe}sw{de@n2 zCJZrbiu4*Xier0{8dR|uFmjy#xFS>9%knM{k4+Z2GtArJ_Svc{rN75M-_`ZatLX7N zoom+dFPQV4+nQJt;Wk)?xx~)MSZmOyDbk2eOuU_sQ31anbW_#@U8Bd#Py%TEXDfTS zG#e5lD$|Qbcs*vSyL4S^7xH=n579g*e$}6(s8X%)qn6E}*iy|=M`QYD8wyR*Q*oiN z_Qjf^=2$cs5l?sEGkVF(i*03KsHAhgo1HJxG>SW#s`v!7Z^qiSuvqM6|I_f;d4ioK zJtRCshID=hz28_13ciFq@xNxU$98Q{;Aa+yj=sa< z$&1`?=dr5kl~whRGEkCFofoR^dcJfH3h2>i1FA1H)-sT}oPXI}$zvP;BoH`|Qsd_% zKE%AnQ-B_(OQ;iBHj{Alr?0H5cnm1`xnvM67Q?m!q=AyYH2jLBaWr!>%TAs!Hk33Y zA#al`-wSRf567;75K>~|(D;N39s>9~QsR2{E7^YY3n$->0B_Hn~4{*2_A zn$ryu}|Kzn{u+UChx^8oTCZni6=i#_{k{?v-w4JJ?Dy5&y_7?P44nAsxy$sh$= zqU(`eu7-!imebXxN`l48oWcVU-}c#I-_XrTm%h%16mXauHTnj>Mq1?DHKspKEsQHB zo)_@A*w)vpt**F&3*e6;-$2r{yVqFfXp zq2WC37yfR(eQXG7dJrl)CSSiqE@~9k<;a+LZ)5g7@dA(fK&X3^G+ZR5mq8pIZG8|T zd!;%ZYaR7Ah*);y^l7Q?{(}X9<){%w-Y0-B8wP;(0Pf#{o-Xx2%6hwSR1{=te;`21Hi>5BnT*FSyPv z{d!Ez>JSklV6^7By)0o48Q*c|k1)?a61-jtALOu&h*Z0IbWSsEcL9*kD^Y(odrZzf z!oujq6&Dg(El^H`jgV~64+mX70N$o)0JF~1YMC2ztxc1lxM7z4`117> zd67fH!>a#w3S~@Cuil?J1DeY|w70A}{*gIm9l2jFK=c1U( zExRMqe8%70ipha0(!{Hn82)x~`r%XeYN2JNg+YAAmEVE4hj#Fb_fh-=!Lp!D!S!;T z!$I1%h+*gWAoun&HaNG}9A^2@NB0=J_61orI>oBY?5Mq}$`gORo!d&K0`8d#N<9ZX zar>4ocB*N-xhku%NT+U1(0)l`hZ`KT zNM2%F)d;V7%jc#dEh=_y$r_Pg-8!RpsJj}9yc)*+nvBe_8bOpwX0(rz$0Q)=9k-cJ zG^P%}{~Tb-?6JCxT&c6Lnuz-Sp36(8?d1a*G-{Zdrn9T@$T}fo)^Tz9pM&Z&0GGS< z65`5}IwH}ERncS4d1^;=Z14GisJFlbj=Mgp)2g>FS!O-qs+v!{(cyAOVi}?io`r3r ze2N)K*=uQ%zE+e^dvEI-bi8%nfOxBWxqixZj$t1SP0{vHDq~MgdC|z8N}KSw=!|-# z+Qsb~J}z|A7+O>@h*J0xxWfC~qY!z|M=M5tu^r8{m)oB>$7Gtv$i7Q&7*Es$em8|% z50Y&`ElZ#**J9f2+|p0mMW&b^B|b-@Qp{fZK_?9AblPx=*~YM<_?bK)XJw60!WCux zXodsp)Wa4DZt8|*&uaOD{GkSu2TdKQ+2GR}cSaLxIpuN^B&cA*Ha_yi^}Wj7*Ywc> zlX50KTKcm^YN>v0ZyliDzb0fUT>V=3G!-JIR_&(RV<{DwH!s}5RK#63dONNnc9+LV zFk7GP5ArAL){x3()bM~x17{=LT|2j1v6-$807SwGVs=gQHW*Xe(Nxp0T^nh5_?_jM z(3cm)H7n{~-^tIiQh9{w87t4fUf|r6ehSe!g+j@HFAye+5hRQ0C~kfv{1D9XgvQ9g z$dhi9MA&bR$HA&whFD>BlmOG-;#?F$0eHmOfFp_pr`fV=jIQGb%nW)Nr&FUO_^Gyi zfvHT%ihosY;1NgTx}zb#L0-Afv&)ryR9u_54P#Mvc=P)5`@DppSWPL^=t+GIttG`< zd&8(MrvAMF;DNq(k-{QBBW*I1DO%r~@9W&8K^~)|jzRiGT0VDK>#E_~Cn88HtASqw zA5RdUEx6oLCe$N0#Ur|7HMMuUP%Jc1ShLbj8}OxP$1m7lU*s=p1ZDa^>a@{2 zHe}MT4vlA7p8J~q)hLq#1&Nug$hts`G*B9MnsUtleyu@xhw4rvdi7m_I88t2c;`Fw zziuKx`hY4|Eu3G?r5U_2s)wnlm5@uz7e;nzu%0O}De6x*C>I;NCmrtKd0Smn^^99hyA&9 zr|(s);v4@yGJW1#+~Zw@&#w0_yhB=ahPHv`BJnu*g(+8kT&&NQjr5;xvAC0vGeT5d z#zh0OLF8Zzq0acfI@F?jWtWl&&?zXWGLRld1zj#iIq?6Amw(Oq>7Ktb9d61jgjZRMZl#DB8i41hFh{VzZL-;xvr$m=tW;P6*-xs zIAN1hEu*l~ov(lxXo1|ch|ciN^b`cVr0v&r6fespI2j3Io;sQE#_^i933(8(+}c8o zpiVMZ?7;U!(7)27mpWB*R zx4Bk?p>w;NgRCG zm|WXhN963zX>BPY!N7Ia$lI-61L(axeNl5ER|`oyaTrLITx_}LCyYuqrMTCdv&!wP z$n-(`G0BS~T)_Rv!Q{V)wbh>+em6Ka`r$o#^j=+O(SmD#^iJZ<^rX+GV^RM9lI|}r zEp0deSV+W}{k%kKv;1F{{^cEklilb~dJ)loh3bEQdPbELM(g}Pu3C}VeIa@kOJHSk zE9tQIFq3TbS54wMo~`OZ8e!J+KBr4zLqOal63{|GMQOKYI(oS^hTFFzp->O`r7K9F zRw(WPDl-3o%OOHJpK$=#xiZGd3NfN!)v+{ANai(Xc%*3w35tp)xG5M^olE+uk+nm& z$sOSqXn{zKABA7e`=V}FMKV;#TfZ{dp&edjEE^A zNoWa37gB_{?GFhv318Q{;npiIX!-puL1n*8!?W=(G=qcXV+wRr2ZQHgrZQGo-ZQHhO+qnJhz0W=8-v9o!o~nw~z;9;;}fPjGD#Y6?=fq;OUzv&(*i0^AhOaTHA5S+f5fPk!+fB=E4y^XP% zr4bO2XlPx`-X}-By6#ru= zP!ka-b?|7c4rtms$X?lx0Csi88*D|1B0&^*n%)Yw3*t{Aa0jk}H zH6H~sY0)4_aG*Ri(YRbf3jr3{{eF>BC%Q6zde&AQk;({#7aP!2>bM}tvI_&V)Z1#d z$WAJm8GKr0s-){&X3Z>G=@8i?$TjlqrxzQr7Uf?DzaH0qk`D?#3=yFO29z2MsSC~u zO0_R@sd?LG7+WS&u`1_9#|yRWP_&>LA`9`g8&h!;;nPSrcyWp+;iu<3BFju2Mn@>U z`J?-ib%PW7f+tiP`-T!=i>d?806Vl!G8o&S?86{L+qtp#(V~}UvArgoWMDpgNWcT# z&D)*YKgoP=r$FT&=D-afAJFb2hk>IBkNiYqV0|dxgvB6@B|xHc1aJG!L5?kG%1b-( zjc~|9XGIe}oYepyY5lVLQ1pK=gdO{(u7TDOqyJ11sBYM@T+UrxsCe}n@Kf#p?@;`W zv~~ECFZ!8Do5zb?k6#$dwzn2y$6-2$j5zr5*U1#qWS68H{K@yu_L`Tijx@oZ*}F{N z61)H|7fcSP6Tjpl|AJjtC~?N-F7J`qfS_#MFYD^)$UEMI(2}@o zsUJ=Etwv#){v6I3stlCsoJ=`7a|of#RTrM?R72pSZs6f19`r)gJ0-d*RUuI<(4xU2 z_>5Eve;8eAM8%5kQgB7F<(EXp)NXf`Z^%GAOb~)zOG|4DaL#LCv5rDjs@E1wfZJnP z`7k1okGqY{)!(Yx;c%dH9+0M&iYkWSx@)a-;O>(Jkb-tz>OLhz5KrvmZd@a{F15vO zVBvj;c3*rbkiWXWLV%$Oz$1d)(E1*zAme)+=^#=3<*QN3K`eTNtHIBK+IqNUArbrx zwtulf()=mgX3^xi*k(TlvEf7fL+~pgitdL&KS+&G#!uFONHBuGq9{ng`2;7TXhcE@ z!;XeD@?lgWi$v+M6#FXs1om+3{`Ewxv2UV~w=gdu`eg9Tyc4n}4f&CpNCPLN76@3YE<8VTBB`-V#l$y-7+g*U~x`8x%@`7i$- z%UhCnCSZicFeKIn)F#gIw(y!2-OD}{KPvi|gD|H9g4Lxf{$}ND{_R|xovod{ox!U3 zJ(E_URN0_%R=?;~fym^Qh8!s&kw^Z?WjXFM_X-=Qa7aXsi*gL(nGCEN@BA#k5 zT`U?L!!L!;jLm7}Hz{02Y$|xgb!K`Ocgp%?eWHEry&_^_^riI}L18#xYG9ILTtEnd zdVpr}HS(48So@>-(+GG8Aol(0BM(9i3Wi@x!I+C0F*syMrhB5VqUS+i!0yNVg;j4W zJCP&dzljOY0>glWnTT;g-)eq8lVXHpo?*~F-aPqS32nV=$gpZyI|31pOp@sj5Ag<% z21kS0ZgWkoFh4)SH0hXjf;M+oxM6%~ykpEz$80<_0hmZVFsaI0JZn>hrVqj#GXsTq`FmzhUypU#yw_^S~}nb*C$(6^CyjMrP& zXxpFQj6iR?F^3dKM4H6}{Ya?VqMGxW_nM=U)&-)89t;*5ksR6#dI3=iP7B+ecASEW zvK&xOxz6aR?rH!=NW`G8*6);sDbroH>lAs9$8KQg@Qa#(+umy5pR9&>NUf%~_3I~^?RsVjkdB<*UPkMlHm$d&X!YBPlM)j{9 z=UjnYh8l7)QgZ4NiW4n+(c9boM*69oo5Tvp;51bJRsssop*iCWD8m!{ZUDQg!7f6X^@w z9G;I+?DOpG4xeQ9)au&z#@4s%L2kx(##0{83ePeN1FHkKyJvSFhc=qNv@uksYR8?n z6TP&*d*kW~DhrGXveBT?GC$m&rw(|YXeKq+Yx^xmf(rTzW($^*z-iWMG;Q2=Zi6X> zD~h#7)YWQlCrY8qCKKEetTd&xDl|8g@szZ?28}0XnX?zemb5FNDwpdf9XEQNt`=dJ zbXI@V|258A8Jvy8XBA`FSjOpWX`HRBwS={k=t&yVh;Ph#4g8Q;V2QUXSmU-ywzgUO z*_78Z+E!{_bTavS!L*^a)j(ynGQ3T|d1(Z!Vy4ZDqNlBa|(xZj9bHH z@=Wu>ZR!s4^ZI>{H(W*H0{ja00o{v+r9R4(%bxstwq}elSw?$Di`|9!)%>iX--5NW zNXw>;u>I!V_;Mn-3C8+iJK$aGgF>GwiH=14#3b<~qy{*Cl$%6uAA9QqZoq2fh z^l?x)Bl`}DowgO0{@B^jjc*F0vC-ne_EedFnzqq@?*yknV?zV8YK_C`k5 z4yHDa1_-h_-wDvRqG}F6Kxn`JIf2FGe_aCsfjF2csyeDlNdgRPtZ4KMZS;+3T&--s zrGbDrT>;;um64+!fvc6JwFAJFi|AhxfN%PrWLhGEe~CC+a1p6W$r1?I*c%Zr(=gG{ z5ply15D;+M8yW-T1%>|u|9;{kGIeyc1<=yExVX@`Fw)rAo6yp;v$NCEG0-wFP=8BM zJGfap>bX)|I}ranlmE&`(8$5S-ptm~%*LAFpM3T7ZJZprh=~4C^uLdPm($4A?7uZx zJN&0w-xZ|&heJzGLr44nf;pNQ|Np@L;rtu+uX_Dk9p^up0hAn!>;-JBtc^A5S`IcfhtY5%vpype;ArPDv+O4eqM+}~>c1^K^{|AXSB{ij*}M>GAq z5Bw|jyAyE3aMJ$o4#f>4w>l{b1jGv@CdjAg3Vf~wo`XBEz-_0Z4~B#krz>_%hNe#7 z8k;i^*5)yTZ#E4A9@gY45!)ylyZd9ES~Mmm25k58>8q2`<8gAM$))PNj>#p?#l`t> zIU|+J#IcDFLht1^Vp}BNPm!P6i~y7y)oCfNmeP^}tJpMW)107!dc0>ys!td`pU*!? zUHijo8mOef$ckJltvYuP(x}SLsc1n#9yhSQPZY^{Lk_*pNX zX(2HbjRE?mPc)@rS7?+c02TCIuV@IGFj8&pGohneY<(jgIzEU8 z1FJ1@^8rQ=Av1a{mEIXM60CN*QMr}(gZj&ZU~|1w99maa@n7kIVk_LjgJiTW>BNp( zdYh0L4ga9Y={7sf@&%KMMgJIOIQ%Z0qr<)AQiNLxe;tahv_CQ$L9YSXjqnNnItmZ+ zFdJV50J4Jh0I4?|WtGgOA^@3EaDdLw{i2ABShzn?+s&Viobgiiv7i``*LuFo|P@YrH~y~NF^WtymFl>fZ2TtD+AUZ=>-Qmty39#wdz;R?FU-0|a> zN`^n-N~*tW{+&y)scztm@73Nq7LJz&$6$o>IPyNvp2ofvIupXIG zfBozD(PT89P-m|ZiCLW*29g<7Oh?u7EOR895`?&CVF+_;up#nM@qynZ)Lz;xsKyy{ zyV#j&ore}81Ebl9Jkpo)x_j3WQk0EwFUN}_@2iV?imXl(JQOm2-|C%IL%i8v%$!u0 zo(~e(fg#r&QIE){RSd-MwKzlfEQrX&Shy4r@ zt1UlXSWCp55bs2!ptjjaBoJW818I2yBMb83UI(G-JLd;L0ml;?h086ySEw4HcSFtZ zG^82INEDtQ(aCD5e{Rl-WGV?;Vv4&)8Qe&aNk#6UD+D3!kHe-F0-4#8%c^@8_|fjU zI?ngVxp))p^lk$Oo5hM6DyF2L?Y-IYgC{1bx^PF~PaVX4O%C1caSO)*eLu3k|60#) zhw>&WzG*1iabL>4=6a^XB1nqF|3QJpql0<-Q@1sGZ>Wzy;YSJdyJu}O0U96j2nwOg z%9I{~$;|uRNL)0~oEHuc-ZhQujorH|7=1R>&+5`9V0|KQ5FKJwCT~1-vm2?X<+7={ z=(lb$kY_gU_3LiP@vKFA7+syGz0&v>H{@EiO2*gZmBkRtt_gyXQpN`%O@wOlP=IRWLTeN`0B za^zyQzkiYis0_CZ><*Ct$dPJGWrCq4k!sBYebQQogVPlY*)npsk$u1CAR9b!Ao1bS zM1bfRk91K|Qnd5mqv7lDj>Elp|7@UymfX4|Hz1~ZjnR7p;b>ER&}KYz>fe1Ci41njYj=-9C7!1Xw!Va={g`NoMNp-PD767cF{t}pm81(tVwpiFQEMnS{Bk>;zMq^N zu-`rKD-)+<4}4gaH5y$!t2J`vh&mhiMZC1Mj?!A=JXxv1RJ~bmm4}c-9bLj9P#k8? zo@ilZmLm_@P;hd(gM4z{#@sw^zoXJoHnDo}levzQI5M3Ll+EB34)bfzRjt`JmP)Nw zRlVMla3Y;K*?OaOgrclG7v=bvLb|e)p(I`?9DSrbwXOjAex=gu4TDh~HCou^@qDG$ z^+w-prM95ndILhoUPl|nTFHC`Wwu~1KQEK%gfx}Wq>{;IMpOB!vz5tw9g4g1a7|e` z?e(Vx#`$hoCrGpttIa0ohs8{8;3D58-s>x&k5E_`gt8+1G0iB%CXG0?=sqLpf)*R| zAG*Q0BB^+f%NCI+u1nPw+dg=s<7vX$0W^)_!gNatPzbmfgD7_4&(Ei>u_>Ofa$yH{ z5un5$=DrZ|gLiW>+=+bttcl??t)fZPYDiH~{JgaM8?3!=E2gR<(RkElm93?mcjJt$ znTt-AoZCowhwI*aUXgfSf#)qpWNJ1Y4~7$)7Nat@&Iza=X^9loQHPV6Wj^zW{+~qB zUYwuLd$pghpbzZ2o!AEzhT`fa>Vi^ejkDJF4X|FWosz){@G;#^@o6wD_HU~ml@7yLd-ISw8pm%vdy)ekRTVKv+5jQ07|R} zGqTL#ARixCw}uL*ttoIh9D&9lp3{05QiEnTJgz!ORW$7a z+uv`;YPY=KMu?-ii7f6l1#YEPdPhqw3L6gQroLj`R4L;h3bh&)faczhxbZ7z2&(m; zMRl>JkYaqpZyi3|cG)l^8b^javfHC=s^PI(2wfc{9q6;KxubdxG=Z1f-C(75v70_} zPC>zOrNHhLU4NgFi5wVliv-1!O7#}+Z;4;NVpk?xHEkfM**50%IdS95jF~d6V93E8 zoWLUYLm#4~1kNxaK>J~hmi8)S2ORkX&Qj`men$PegDw^<-A5+DlC=JMq5p)icxZx} zyL{45`m;;p&k0_SpD-e>XU`zvh85uhzui|`>niT2 zSd8)B8kB!?O`48NvS6NH`LblLXAz>N{=EWwb2lUqqo=h_gvOLR*uDbAsE_{m8G>k? zCmdNr!YEQ1Fqi9q-SQ#Yvh#{Hay3oC5Uf|Wq_8bAexb85i;PtR?O$Q`<;U%s{|NJ z&+zF2oScF6<#h!33H4AA%)ybRnD0;|0em8j9ebQrh{x!-uKg^69d$&lsx~jLT{0hu z*GI&Vf)%>&aJrx6!t=qgXk9P<>B@EmRg&4=VnQt4I`B%d-83f%!6;vznLD#xl5WNQ zMblQ2x7VAVdu>-GY#hYd5T;s@MjSM&Du#h(iGx+rFVPpwA4YD)w1sj!38^NUSS)sv zsjqod^O$#a=%o}(&EmFIH;H4QC=19ytNVaP1i zleU{NrU|twH=%?cN&zsp=dlj{UF*8t5U%*iy2GT>(V!XiVTl~VbrMk|%!aQ2!$I?2 zlmev7$nWG~8=p4~HIIvi4aMKK<7{)X)av~cXmy5<87r-Jxms<`v!zOufisX;*Jx-o z>ctQ_#`9@C!(26d{o-2tJ7oMRUn+CGaqNN2W4s+6hGyjX3DXCWh9sK|4aUNW=I zdkus|w}2FWfLwvP8$&7Lu@>DO_L<2YXw_b8cXJrX;Id#jHEF#)2r6i_T7zKVFd0Tz zx<8piO}YqtafBGnzWDva&z@Dd;~L;`X_?An(>Xo>k!C!RE|$(>70_(i7bMalYZE>_ ziAJ4P#ks=WQWESoj|jSFc^1jrM$fVn01VI#_Pq#TTmK=bCC3sUZ>3#y&cz-olVxec zZsoD6SkO}+Rot~$piKTKFVLXuwGBYTAAHrkxk zmj&oEPKQrckY)!@m-6oEJA*5(MB$eNV19bNo)opp@*F2SeJSADlX$#E5DlIGp=ZV!9V;wA~GCKQAXT zGEFEzn769wHJS0(LHMY}Bs(z}-%W7S{fzqMw5-ag?5D< zqhNe*vZG9wtn>Og=~lPHHM_n2O^>U15quBD#w~A;m#(iiu&4%dxJCVG4~;C(TWGhp z%qF984Fki^^OxEMWexGKPta;R1DO@~vkIsAmWYJ}QdypnBti&GF{T_uuKO5GmklTT z3Ff8Y!RdhwmoKHykE@}BlcMaTPOrCI|C*@tt}pM}H|sVt6JO#o*1>jS)=kg&`rfE< z=0#Nj7iW9-)Jd!G3D)>NL>~SJJr1huz)L*8%Vs{~acl!agYvFQe|Yk?%i6m0_Boo_ zQYXp^+D2T!`?t9uDb2Yf_%dPL)w6mra{KYLHRkoSQDjlSX4RCb>AYfeyBkhBqGpm! zo~&h`YFT%|a(kQ?Ju2|!Dw^eUuOO6YsL+rlRfjfO;}tQ)gK}B3%2u1dMgQol}-Qb*JIsaFwOE8m)DnmQo(9 z*=>zDE$I|Z-aK+xCx3B2;Q&Hy_@=PO+&`8C*0|+^5Vxoru5{HD0_z8xa1I0UBJE8p zE!il25r}EPr=URG6ELa%m|RaQu5HB)NQeSJ^AdV00i`5&%LdVASnfsao^frA$^2&|At0FZyDps9j2|~#vahr3 zIp5A5!iH|?+n;yC(|mef&<(ktxBbU+di({Rznc~RKs_LJ_8|4`s*6WbC(mSR{L0$} z$`C*%Nqj+?<;ofSXi?Qs_~LQZ&_qdkoMV!mr&1=98ovqe_u9sUn^Vp;nbNHd2iupD{NemK)3x^5k^~mEPQ(XfU+xyNPf>Q zE9|XN=FgUrW21q2SgTU-qwScOLWVuKGA>Wr_GkWQTQB-Y*aHcF>|1}i$9?*iQcdVC z*H8x>+K%_y+4Jwga5zGmH7uFMDHM5}FBU(4jDW)?dq=Mhb+Wz8XFP=KG7hKd^C5Xu zxxv81hVOMRXM9Sl!1%q+O?p0UJf+r6ackUExBT|krUxuIVmTs=G5%vaGhgYl30MY< zNM-_*^g6#T+s2tybSm7KMDcyz?S_%DjhH^3_oBwm!3739BW$xj& zySyz*&Y6@`^c4)oCVN&)G7DaaMXa-X=F?6iUH|&8aoF09sRn0hcT9${vq_Gz`$Mu{ zwj624jZkUtJGx&qRC=C+F`2DW2?(&c-R{E``Y>V$nOwHtNhkUnJ?}cGM9=piZhghL zT`Z_DPN>%p>teCr-BMPWuJ%JC3#fgeVBBk4l*hVl`#~}wlYStOH? zW*hQZ-U&83oXSSu?RZ#BAN^;PO9jbp&#hJzMk3Ss6tk)SE9p^~1 z?SM(yY_^wZucYd7q=wq-DJUHJz2OftS?}U>mytn&a(J9;X1X1whV`AP=y(-8OaOB5 z=vyZob*z{tOt1}g-KzZ={9`pC=BjL;I~c2Y@%M(JRgSD4Y`>>Qa=gN=1}`;}n0Ga_ zPc6@zfmEB$7hyL!>2V0dZxghd?u(QLEK?qZ3Mgbwv` z^{4I>_Z!Z@2EJ8#CNq{_#?iC|?M|JVmYEwsdtnmtw)RdV(3<`oG)48DEzND#9Ih*c zVL!GcjQf;Q(<$L6Q+0p-6ok)wkDfmmmWO0t4iVKxr4Ypso;m>2!<1AqB`2Z}GQ5{c zt8mbIHh<)_*i`YPdfiF>uqfMmHh4V(6#tM*BvL+&%{u4)xw;nVrJ9lRWnUqU2aFg| zQp_9+7ZMl82-8n4eWX_s`@oCdx%|-Oe3|fWr6`*M&xh@m2(OTcaZlF~!*L#V1@n0P zdm-@*+%fehhK0cHv8xYpN{7SMW=g5)L(xnLltTA2m?*~C=W;xj#So9M{ljnwXnyhQ z39AyA+hB@dI?fTRD)ny@o=X*3MXQFD&r30%+>XfY)oqoYD|Ud0Y7c2*85IMa(slAX zX^g4d+PU^nTb4xW<$OpeQ{AwvFoUS_dw8VO(sjw6Qdc$`ZQo zm2$CcErPi9Fb|rnYJqhL6?2>FA#4UR3emF6o(7slD~8);#{S|y?(2;waPM`VX#dU6 zLjHTb>d+mPipKpB?E_ct0U?P$YOJc0JE>$nZM01i@3f>Qve)&x0M+pwFK}S+;h-?% z`}x6~x%P%B>W>}lsvSS1t5anSOh!`*+m)e9HPwr=4M^M@qX3|^XhTyN-kLz?B&H;hwGD) zPGJyQyR5wKH^8O$NtDhsY`e(u0h^uO4&6mU^7eO*1B%T!*n7J%EPq~SdvC6Hwd5vK z63|pfDaA-3i}JbJOV0Jk+u74lx6+2ap-s3N!|HVpgPQZ~4rmTO6?ie653`4D#RP!= z%0)ad2zJx!pK`J@+)16)4Z>2eGheUk1_nZEZYxt-TV${trc?W)B3n{3Xbrg6cVZOD zgIMN91eH9x9wE&pgC3m8QxQlkIMGMRghNnaPR&;ne@iq&1P<#p{Hb6L8YcieYV6WMHU<@!W`Y$(gVH z0Ef5yQ~ALUp|^Uc5B2if_cLfc;ARb3NP7M~0193ln=N6_ap*C4%KOCQ3$2IpI`=Vg z!671voM8H1=e8Ad+63IV=K!+nB_jw^IOpd3YiiYpB(#4o^zJK6 z&lBiUxD~CMO-kqfYV~ycOhf&+-p7;;vJ$Z_!ggL(4z)O71aJn;;A0DOe;aT;_pRlt zR!ReX2){aN_Xcwgi0$fO2i@ueIRY{5w|q{Z|F&$(@F$^38_9gTl__Ahkbg&DNwo!G z3JHr3g!hVEC0beLB9rN$IiVj0#+C*SCnut1XYjA;<-2|g?fExyVF+(5pD1C+9`tL% zFOSvcvne+CufRFk224S(yk{78{Ih$;Dao6#md?DLwKn+xtO%^9%lii+s>9~-@_}&B zf5LK*{v#hDDmd*WW|mDl^;{Scahsad`97x@7+ZHCtsm)$5P2gmO5Voe4YXVJ9g&LxK!zh1+c2%~q8m2c(t?63H53UA(D z*wKUh^>j#qbWVy=CgR`wle;^)B@ot2rrv+w~@DN(oezWeaNhyg{`i?nrkzRLDS=dszUvj zOF=_VpR{I{%kJYbogsCs1OwA1xHx0WxN4(s%?>7!v{I(`E9PlVGWFhlivRo|xF*2- zb{~^{jm4S?VBk)M^P6qbyxVotj0$cIdI&$*>Uhcmfu(hrUIahSb5YAhE0YE{w4}h= z)yGhOy7u>yo@~D+gPuO#`m|~=#q2A=&(7Tm#Bx#|*%QgtYd@5 zQPVwYXwxZePSD zGQwE?fL5uc0;SGB{+?rh;A9)BduA)Q*y%vt}T_PvbdN{mb6@E zN?4L8#VH|}nJ%fKG2MHAcFFApwFWt?7$uiFa6HT_ETq_*&lP=KxNMC`;yh}>UolF; zQFN&g(WVX5!Auqg)P&U4I+BRg)K>s?yHEr?DT$hF;pMqlR?EpFlS?a&h)l|DlHuqe8CxrzU&Hb3t{@fdmwKRn42G=;~Oa1wp1fdR3b7~MA z50F~Zy&kWdGDXZtm&d@t7ZE_dR*l+B)NatCeD_XCk`*8nu5{>G7 z-X_~_j0jW9iSVkQdl{(UAV9V;;5knFor+fAH2V3Sal_4rS5ZpAdDj-)ith`-v%n4| z01=3e?`_$8@dBkYAGb(a-b*ckEuF zS`{#TYEbCVG(A=V?F>|wpgmZr{!rdZu?Q)3U~p{q1bc&CWl@o@dMz)($vtM=QGa#k zd6ZinF9W3HGMOg6pw9?mdZxydkd#o&i6G1E_sj??!e)0pausuhOo3h>(>nodJb$5( zg#52Rf;>focct*rcfI1^L~xt}I?etF`=x`JURy5dDy%y&1%vkgp_iOi+f`r6%{K0~^ez~b3nhI(xC!jxwb#!HT`Q8FEuw+1geL+hF zs>2~fDxyV4!VuOMD~AD+Q_;8% z)ArSRifxkRekwL${pV*zV!V+%u7zguZlWdJ_b!Bd=Bd;*^1Q1$xa%ighw(~q!1 z-b>P=&h~rp`dmL$HPFsX2jb@DMIGwjjtEf-lIuA=758-}6R}>8nkAEs zXz;A_@R%$@`uGnljchKuj_1$KwPD&{wjU$pEcqoRYX0+}Rhp-p3s#-m5(O1wxG8E8U5yD%KiD3AY8w+J(B`UfqOs$2qCizkjY&-|^Ca zJ+D$Z3t~J2Xe!HsEUJJjjwP~D*`DdWUx_a0`W%Qv>oRNfqWiW%NXH}b;FM_qC~KL+ z2C!@qQD(cTtzs~coelmhI2^`pV0j9ili`!&AX`=|29er;yzV8Jn?wyZiJ~aD2K>_y zTn@C@g9GVZVgwfvq36;1J6blKO}8W54{st?WxFA|qMGd9nLw6`wNbM6&Pb<|Fkh{> zm=*9MpQ^LGk@*l@JGxA@H^d*5$zc$mkWAf4)>FLK%)lRbIN1r7?hU93h{#170KGi$ zOt2~W9ohPf@15b-CiX=ZTOd>u_rjeHHyg68g_=#rrwyOPq-FXbfqFKYZbmcfjc|<& z)-6w1xAW}iosN(;*iGBNY?6DhCT1uYu9a|AN-!SN=f53TfA!V=w!RB1A!-LXG_h1r z!C+H>*sWpp#F(_3S=o6IZ_SHx>-v1CIv{2{DRHf#?fc%vs+?2Hu+Nhl)PQHo)Ho0u z`os7s@er{Xh_0QJ4Zh1k@FG$qeH7Z)3*)_{bJ^R*`Q?8ygZQ8o;A!$}Gc33Ehws{{ z*XkhgW7y1n{q{LZG{6+{4)!3V3|MYHyQg5dPa*;$fWqj|WDMw=~_1Nn^_wIC)d*vwwqND|1 zKBY=j`0{NmUD7R&PHe3J-mz|xu0pJKEJutUmg9oj;7OU?>6x0MG3w=PeXv1NulZuG zn}VyoeR<3F?MD;ond_c?i3oP)k#3L2`j);QHcT^8RIXgH*Wt4pHGNh^nJ%+7wlh6; zW$c+qk-w`r$NHrZH7r}$odM2recSGJwt5&=GsEM9#*lc2hb@$sLG@GhwjPhudIC54 z2Y+kLpOhXhUgrvB;(G*wj_a{>7}`G`IZSC&O&W&dZTyScU13AvRvl#@Gm@};PTTxO z1+R@~4(?d=UZ~4uc8MM>N6~(w2Tft<|RcTn)6nIxf98AvIE12OJHCZmm<5JtPPe^t; zGL$2C{z!Wmb1#@9=C1O5`^=GIQ97Ea3UwJUi6z-9C(t<^f6aBsBi@z+b`A;=X=OUk zmm;3(+g`+zq`BRHp_g`281e*WD0(l)Wz3#Tabm@I*4WPBYMOD-8ZV+ZyZx2W5j(n7 zrb5dFI-CmP%+h=);Uy(9l*2x$d@DM}zI+m<(jiv0+2N!(R3!&DMCh7`@J66^bT(Mc%b_#)h|tAE~3^O1`15r9LmfUqsVSo5fxt< zixz+)y+B77X-h$-`{0>2gBSrUA{H^3!7i|M-{Vx~{OlPiHIVX?;78HYv8?61pNrVv z_7-qsZBEBhn2_j0U?r=ty>>!VIQE<7KrV*^yyl9@I4x%;5q1Si!^1 zZ=zsFOws5Ug>6kyUqCLEaQdye8VVLqA#vMNB^5p{`wiy%ZA;%=T4_{xX$S;|u9Xq9 zal6U}FrSFky5r7Hdb{H*72=eDy?UC{q%>3}zoN^&mC-M&0_G24c_j-SlEFsA!-#@d@}mGScc;2iu^*EKyT%{s1-rVtaR) zIL8#RbF1MkOh8W09VotR6a&V1^9eGh+b7%*o~gBJbvLfNvnofvUTPs-n|iC(T><1v zLORKx!GIto&{;MsakXMztk^h^>5#FX;Yu)Ll3Z-@l7H0lOkT-DMVvs}a*az~pS(F? zcSiqkE{5t{oi6`rR4M4>d~Xdgf6Hyh*hodp0c9i=O;tr3l^QxJYJ1#nFz5pMZh}|0 z5Dg9cdM%Q<@=JqKtNXnM5dETmnTHtn2Zf>NNn94_nE?%jcx{+cov z2z}g-(S+ZA`X_1!!?Flbw5icoLtZK3oNAyCVm59@J3wHWle$5Dk=xtr>#c(JFW>(t zg;!*f?ZX;=+g_yGf_W+bCk$<200v`EGQ|nPKXjC#aIyVQ9oM{QF@3L)czTi#^z(0D zG;&WaU}Wu3f7sLMhQt?=>8FgR4=$*17uCvM>Fng;d%BoDGB#NzH~c17}5|NOm;4jR_d zQA$WtL817WQ2+cw-j<;clk`hK{k_=Ju6=hl*VP#x5kB7kdiE4eh8CVCrzH>I~03|ltkVSZ^gr6xWMbY>zOzCYiR7mA{5WtO0;*fSku4a z?X-$P{O2$`Z@c|Hg9R^a-Wk<$ZsI+p73MupA^8Va$yJ5BSIK8j}uf3L=T}z&8ZucjJTYqZahlUfadewknc-H^u zpGrRJBk2kF_CO9f-UO8AKEI*l86lVj|HKFQwEw!s(3J=qMIQUv4S_!)qX8La!@q9; zQV6ZK(hf$zLpe(G?Gu@iUmWkKVRVDBqj&YMdpqr>yL%;wy3@iO*d|YDdM;vwmwK^6uosXTg_5@aE0| z;r#Z~>^eBx@ldKC($PlrZx8Jc{8AFCPxx241QJBmZ6$$B>Rp{nkSMUo_&_b`V;>bk zoWZ}%@qgmEi;dyC$xcY(bUF2nCQ89s-iIlC7}Cx*8^Sue#Wmi}z6KRwdy zB3m>{J&SYkDh*_E7xmY`1r%F+F33U5KI@Q+u)h}58qumCY|y0&DMJpq^pjZUkM|Ch zl}~dz5x%>YHYA^p+0?$8Ef9vzP$w%}ZL7GXN|2XLwXuwJup=vNc(nihkZ zB$ZWv|CQyf&HV|XK$dMgFlx>e-ceYfq47L$Oh)AZXDcZWr7aoo%&fM^bi0tRIdEL4 z`uQ@Ng4TtB2^}BpQ}g_QS*eP9NGVYB{2jcGq_r(c0rEQ(o3;_-6g4XgACmbk^<7*-QC?HT_Wt|^X_l| z{C;uF46&G5d0*#suFiOjh@1}vSfb|TlMgEE{lmm5`2BJ%)z;5%+c8dbUkY35+Qi+p zOVO+^8KrPy#Gh@6H*cVB3f~~elUH~-5FK#`bPSn|30cr&yXt!#;S{at8DFOTtv>)J zeuV$68$i|fw@6|GSVKDFEP;I&!XUFrfjI+w4$nw8FlG`sP6(hP0ybMk-%!;HGRbpC z`<+CsdbE&-Ls>okhmJA-H8W=QPyy23{hxTx0=h+8e|%XDnf!tM=)=HZ)qj5{lP7BX z_wIvpVi{Hp@QtjJp)CXV=H|jon4_3d2Y#4WS+De>mnt z9C?1(w4oUIy@VL=UZHP3toV^uXDLCC`_tsgH~JE8x` zc>=j!C*L%bozGpNjsnv@a&d+tIO*p1|1H1p$ZRLSzdR6DHepi49QM*@<`DGP`B2b} z%uaGY6qawN1gF5*-Qak&R3n=u@7ls52jd{A*J_HR*OcK+H(l8jS6yDCXb%6^WqGR$ zs$tdWpo=f7a&-#f0UZ!kz_1NcRDZms^iAi9wiY+oElFs{aMCb%zSOdJo;<*zv+cv3 zXtm@)CG4V{-bQr%d$;wrr!{VeIh5`j%c`;%gfc7K(HzMMF9+#Odwj5GwMxif9Ovwx zz|L)EV=H&=Z!j=Sus{v;w~A^-AvydO)E$ip6g3Vg{EG z32(0A-QNP$Yn}DC$JxDjPoUqSF-xKAFT(?}k>u=Fcses0IJG*oePdhv4r|f?f5QNz zstHAMDYApHWJP{|?oN=B*>srDg$K1@AJY+8HPMw#b_W!*@NAA}%o+5?qf$~pUUhko zTt6gK7+8A49CaX%uu*Weg55J`>4`+m5kYy~Zj!FeBc$(#n`uRZ*n_6)nXmr}{AS(j zwe$}PMW1Jw;Yf~D^7dVEe&Fe-$^sF;3MS_HUgCx+MUMXP?K&*>zCQR)O<|aDxqDsP zWPm=<$rab_?CJcX+Oa>HGTieqs_#5{b4 zSHE@OlHdE6tTx1ECp)Bw7=$Y@8usV5 zaO_~b-|(q!1CW8Mzm1!t$4I@VXf}2^!g3$t%`liIStsQ5b z3zp1oE8af!(EV0AF~en)I^kkY#g-UjQu-Gj6@YAIGH7v|=DK7)?t7;fmi_CISft?N zFiVUxs?1}ur3+!~Mv$$<%_slIeS$}T6)cnp!My=9RW&m8ca?)k{&_r#Mm z=<)3^zXn*3t5|fa;$yW$^WmA)+NrC!5u2_y%$q-q(>LWD!susFfuLmBh`W0*pN4Ye zK~HZwS_jcn69d8vh~Pe6{!p zPv*QFa`coY>+CKPFbZRjKU|2zKMC%Ul2d)#r=kjH^6Oo6p_*K@B;9qjz3Jx|9JLL5<5zuJ`ld~8$gy_075HRF*X7(UW5acFW(p(bnw1aj`N+hyg=bB z40bFBpG{tn3VjgIi|x|T?UdlRh#)F^SK{Pq>!m;mQWEp0XCnUQs@8L`;67AsuVK|8 zi8@w5b^nJii3lTGI9pJ~+G4LyMg#AogkWl1LiP0ht(b<>NXpgI{QC_Srx2gm+@3i?m3ncE5Q*{@ASZfo&G4ioU3^DR);=Q+64NNAcjLU zU@%P03Fb|<+-UPo7xh`NW4+ctfp4crNDdg@p(Un2&lIeC8#iJ9$fbz*`R$=pLUv*a z9uwrh6WnRB1mZxEVW*M8HDxxYOX1dqqjzv4Fs>7L~h*lFE6Moh!fOR6u$iM_;|=zYw>%7>mY{TEW^Gr zL5{s${$MAkB3INYBfrbzxDMt z^RgY!S8~nQu|k|KD;h6t$ z&c6+A=W9Aox4%lLKgKR(Bc6v4x|4lA%DWn2u4eG<^#7}_v^x-!*nYQxIvX+TUg*{J z9#O2bWE9|USb+nJhHk}56#FA2EB*Pr)p7sCwko-@aYJ{&@48aox!=mb_Fwa;p8RK| ziltj7DgleKff;iXPWit+Y$QVDtXNbsYmg3Kmz1xBGahW4uRpyJI3ke(0g>-CNd&PG zO?w|jqQ@}TbT|CV!2QE+TnZwEFIGT6ue1XN8Rsz;&IPFen|_UQNS0OpxF2mxI+wV05|zgGI)00LMz9L~%g@no+McTd2emfaaR_ zg;(yO3oQOQZ&d;aLk=QgubQ(TgXC9RAk3kN++io0VphEd8^+9$dM8*?MyVnId@G;> z`b}Em^miA%n5R#BibB7F6V4~NEXD{8dV%(nBCjD$j{j2`8o-|F_U2VZV)VR0B!$r# z{9Yv$OIlXrmL4biRJ!KAXi&RYs#3e3tT&7ybWRxud1+EC$_`f?PK_^*_(XiZsltdQ(uh)FLyY|v3K>i}9=}#bqS)IQ zNbK8w$PJ#?1BYT+DdlyJ@5I7Z9Fk6+%$xI?r&WrMwn>bjb=ALLi^x=JMXD`~O#;&;OD>1@lWmls*+| zS^L4*D=G`0O2ysjIW+c3Kq9i$!gI12B z%!(}(0AmSKHT?{S5?dk1rMo@YVH9~+O9*89AhFgJ1cDtGrg*(}!SNLUTcI7qCX{M{ z77U&!p3^C0!Y~BkWGN)~>Z@G5b~7*#{5R1kvK9L!{dKmcg2pEi znu56Lhsh@Nr;sz21{GuLTk6M1C1v7!pkdQfBeeTxT8g5~)fOi@(8@}%NQye(%z5z0 z0wsgrokTYCvdB)@u~#e|U$?;u`Sbb442h|Bg&wY}@p}05MbjH<5O@^St4KcvfPJOT z$3-bak|g#t_nV#&;Or;$lgN978w#rI)N?f( zFtPjON;e75WL5t+y1|+j`>G-{6p0nlIUbdhfvqh`b5Km+;dN)+#+trlzH|z6BSuL@Zm z*kNodiaTH&$Cb7E-7iZWJ2pHX)Gk}}x~oZ(R4hB|#+}+unvDEv+svUqt*DFjy)Dw4 z-Caeon~P}FV|saK_c&SH{KHx~c9m>=H7h1F-Z>Yk{98js__L`o7%o zngSk<8u^T7RA4BkOucE9?&(FuvO%x`8}?eI|~km6N!s?B%P z$cVJf*dZ8qwWI8_w_@ZU1rAb8p>T0%sT=m-IV*NUSTagNOa~hiFVis+*+g4!_Kt`A z8StO8I*rAaU5%g6YObUY`z8MyQ8E~UiqaEl0#wnocyAu2cadY8^@2byv96<1EKFkJ zZv?*9)H$sYEw_3)Hj}NsHkZfcnGR~)5OyU;s=A-#1<95nF!`_YrrC8`770aoxR zH^^oUhiijhVL!GgcoOOo%P~GJI0vsrl?bKZy09Ey2qx% zHeZ-p`%!*Wp;omaAt&oJH44-^jLc*1nYp?RC#)e(dD)w;QXy9}bXW7O28V^Qi#{NQ7_A zWS9p`+p%NU(-$E-zF8=-25jNhEiuZ7XoTUW2w~l2?Ua+*R~0~RiA8$PgworX!C}=p zrTzcCTcuR!AO1=~6~00uzMl9&vv*7l#|m~g%)U;4RA|=>DZnFkAvTH?_Hr{`s7RYZ zKGn(+@=U;A@-lI}pkzVkCPSVqaF(Kq;aF@*olwEpgy90UV#vgWW*^IO*le5Fbg-5C zh23opg65*!J>QX$@n<^KH*tC>o&#rx0#^PRGudB`CnvI6x^$}Y>ycaW)Piy8<>$~j z*7}F9ZPr3$(!k>QRojfxZe3o~lej+-D6InMf5Bgg!1dNkI z(-~tu0|ev(j)<0XwdTyZR~Jw6B3#oL!cBLQSwGrlbHmEK+%;$E6n=ul!Uk@8`;V}c zbAo5dGN%xw?B~!gOG&c6eGt7J%1WOj$|7;+wmc_2>VW5Ks+ZDpzH;?p<)6FA6*f6# z`=IBrE~C1w58>IG^q8|>9V>~G%_y?aR@%;xw<0*DblNhJ<6m}l@^w^4hu2a(Z1ctE z8RA*sslfLQv$6&d_0I$(e6Ijgf!23OoG5gURrn?ugIb>2b*ue)Lg+J}{Zfs|)fo3A zQi&Mrlr>IhZe|tt(^yGeO+vE&0pjzSA%Cc{W0wgBVkQ5B4uTFbd@a{xF*>@@tXR%N za-v!z`8+Jsv}BpQX=y&mMf6ExflZ{XWRKJF-V#HTCO-&cz8aU-a4U>w z`NtGEBz6YQLzlr*vw>$$(@x(OGzaglI!iU`$oMeN)3|e_M%RFBSO)S4x0N91mpLSV zJ+b-Ie^pWKRFQIF0Hx)p7p*) zyki{+tdFNi`n}1W81SBwEXW*(OmV^wW|T*ZD_`iDg>>J!iKU~soQpKGTkjF7BvugV zzyP`um&)7kmR4%Ur0gd*>A5 zAQSdm!Ip#?#tK0C$uex@Zvb&yg^83h-yKX1t_VHpgsmlj)D!#?M47X(^g7Tnkz zXRky)aiwVT4x;S>SC2n1Rupx630JM=3HT}e@dw_^XLIS4C<)TqA?_D&Ec-x`nKL%_ z=^nc{FJYE|`7bA7vkdK@ZLEnwR?bpwXg(x~NGiNL9!x0jGcu7Vm|5_wvNGmWNLhi-e((VUy`RdICHc#BtYKa^j%@pv}V} zj7se7UlJ3vOso+$$WD5@_~XhtxvBWm>?K|pMqk#K%Af{py)b=QC#kPZnLBZmC)$3@ zR6cA!%{r&Pk#)mcwrGmSs7LB65Z92q>z1e+xP33vc9$Kv60SoV5a=O0w;Ed#ix^M~%8-wC-q)QDw4rPu z(4SWL-x*1~A*f}T=E$xIq$6B2=Ahb4{qkg7a}tDReG+=b4b?;JQz#Fre}Lwt#;Xt1 zbHQw~x~XAtz3VQFhf9PW&axn|1#&94GNombNHt|ni|MXEe$-Xj)m&G%ZlVtpxRrp4Ypdt&@?uzFm!jM3n2?rGdsW>9Z$eH(EnqE-f&C06TAOdl2_8FlE; zNGep*xoAu4OGs}|R5kbWlLQAWS3s9amUoz4dQq_GsIP{$D}o$Sl|1Lg;!$%`5&R-c zIhJVTBpT?t+N}fKOFpRD3zhT-PFWd+siE2p>U5>;OURxc_SNhO2dlr|4ED&*gjlO3 zuK=QD93xE(KHujYb;2GlffY8x5yzNqDL7dswRnPVh7LTL`lvjdk|0&9fo+1`MdO*B zL`BXkx^9~Nw9K}rq~I)fH?%0otL(u}K_j}ylCD{|aDzu!w+cR}WY@Y#1U`eL8 z@{X}=cO11Iia(%WVkv5SG#iro7j57VLV&=>^4=LN9pza6*H$#$`Pol6Y7tNL_^`mf z4b@<@i(dBXek(tw=RC^9q9z1xGUZ=5gx0@9-%Y#!c4yLM9l{ks|HrIF~Ef~8Pi~{y5-DhHT`^p5x z@*u8D?wn3$1;q|&T?!xAO#FyF)S)2i0;Px$l!t2-r0byi_;=@{1BY`=3#<5|f1xZ( zK9KvNkZ5rXcan(YMs;gV58o5!7>o*$!XJay4<$*Rb9dkHH)hqgwjSIhK!%cv!W6w? z*8bJ}6jpYhsfzww_2Gzpq9T60U=cxvu)gA4B=&Krm{yxpBh1b045IAQ;+KQ+|1k;P zd%Tfc>Wl;G@xAGq626yIH;&#`$1|xv;(7vW1gou*BND%>|GRQkfD-O8@gsb)ifHkG%vR zLU)Z&FYirucZz$PgypXH?{>%7Lpy;uuN<^#Cg1WT{I_Wvk0}UvLc|xGmN|vw`2-89 zKZC7SLeU&@nX2P&z)w+#7|5@X8$Rq!wO$I&$qvuo-}nRkqTpISXrUB<0g$B7==c0* zxk5Dn);2?=$t%=t=)2D8o9d7YBAkSTf7#o0(>_7VKVktZdp{+}v}xY` z*8F0(e+GEv;g!pOi{0&}Kj-VdBCPZXvu4Dl(?KeCMEi%Wr*{vRUbPx(|CZ`4muNq5 zY`vWNe-Dd*%vn{}cjn6a!8u1sbq#n`M!-KCcrW}?3!8v1iB8XLE>7QZH%t1xM7@Ov zOMFUimq!puY18kr$7X}yl0g)f<+>GS)Km>Qg#p{2Dw-yKv3z~JSU81bPleTo68iC| z58S8SesoaQMizLALH&yvIh>uagc&R7|Bo(^Y?{#+R#{vVAF`);kL|Zr_o*~!w!ZMI z+4k%pifA*uc44&XmGHekfSxe_KRsa&lCFmU zd-vy>mKs|m?N2nU{%<|1(7HD!Df!o~b>PAB(hQ%G67l|N3;(Azgkrf90*m)u_&YPL zeb(DP_Bu54{1BV}yoPT9e_`bJp!^F72X~5y=!M!0d2v)@u7i4UIeGVa0&qspkp`9g z3;*Obi93&q7-ekFu3TkkwM;H{xV2lY&_U|^To?+Kpy``W0kBW~Uk|(BYr$#C_(E=( zAob^%q{<2xyvbsw!V*wocVN#GG7gWC#<>>(j|4do*sQC@8iCd0*}e?7J_d z(=PE2hV4(!vruYDFOsf=k9>O@=Cf*j&s+WQUB?~0`LLX+Mat*Rt|+29�sojP^L|+WxEZ22GPOV= z(|XcCU-ZU5gB0XN!0=$wh_bw5Mh57D(voQ%mnx1n5eh>*9sw45p+-BH>ipE>Z&%C|P{H3e2!27tIc8Y>MBtZJKttm7h~MU8wv?`uvW2bf~lRnprbeb=wY3E1!TpNddk?AAL>}#HQ7OVpxUn&clE# zlg+KXyJBO7r&l20McDgcTWW8NdjiS`Q9_1a0Vc)!XLXYaCcU92N0INQ9;S}gd^Y`D zb|UCQYobYAZ#Z4nE(hgaUrw!`gnqZNt~#_<-kh!~8T#*=TW@xV9A&ZIziHGM)#SqT zCE{CmQ~OVLw&{1D#M-!~ngw4XXZ>exXS=ZljW}l4byB2cAfOtw?V=Gou;_JYWRYNl zF;B)?yS@0;2K7Hxuuwm3|KzkOLkV*t!hGx4iUJZ<{`F$f|BWMXc(&3oh9`(D4rOtaqb7H*H;~vP4cfeZ==bsihkaBuYNCKn&?yd$b(@yN|5PzYBFhAR`-fFrI{ILh=c2r8MIFQiV`4ZrT`3 ztyq$j_3dW=ss!mVPnXz^`?=6ot*K8XZHv&4`M8+hI$9zno|Ym*9&uhQJK)88eml)F zyR3i=f zwCODad2}6*{d6VJd|ngaT2<6ex1{B(Tu9>S2K?^1?;_f3v1p-iphJEu4CMEHa~;5kTM$l2N z@w@0o459A$=es$7IW>AaiPmahjh)WdErf&}D|*7v{QB}|KQG`}f=)h_5~$()D5h}g z`f~MXnNwz(D&mPzJmg)9lT6S8vk8pg+Xo;9JKCI~m0Pc;xFAB30+aGi3tT$_ z{CMP{l>nlk=>b}Qz~Qe(4L;sBr*|tz`TH>qZ!U5j1o!^{(vefEwAntJ8>&rR+sdMX z9UyFy9xYHyim$ewEsnhIKg=nYFwxN=@3eD09Mkabcsw9U<9EzHnEf&SKI{2rmc|_k zFOJ03A|FkSLzn0B;Q+HC#agYT%(uCJEAh(?5yz^~i?eKoi*TsQg+(uaqTk&z;_P0z zu|ZP=kK=|s5Wc)aBzFoFz8a!lAt&xBQ?XFwt|RXx{Zpja=|VGprN&M=?}9hVVLpOK z6!o-M8t>g_@b%VQ@pvKwLLg8@-7gFlOA)0wp71`9KWKT!u%4OYdn_o1%x`^(E4`S( zQhv_^OZ(51n=0u!ACjStC?nj1xcR2pfxg83yR&F8LI}bq0MNJs_Lxj1s1vPX4eJ(q zye?~J(^UpSz)TgiUsjSmH{A`a>9qDWig%5je-*Yd1^2LsViV$SUwnXYMln+D8&547 z4-Z;!M5qNJ>-5yjkxR@zU}#EiG}!|}XPHN&3FG#czxx|L=LsYnMIMyJn3KZ0Xzql9 zLLvnpt~Xk-m-mJf%>k)QptB(pko@PZ`TzO#FN5A2=iiLSRRVtm8e}3_`Om-ywhuTn z7R%IQ^^N==&B;VaJ&sIT{6EUQpxre59ZIM@2cHf<|d1?VsZAg>Mr z^Lcc^l@T0Y$piL)X1P!dB08WF69T3h+0BkPOT=DzCvXxPLoza%=r6e*Ktqt}*#XX@ zb@H++8Ew)dnC#LdY_<=MeY=WIhJBi-mZr8XJ)A-oiyde}Vi$XNJKp5TfqL15M#v#1 z`f_AyjUXkHInK8#FJx-xV>ih%SgKMm_d_K`Kgxps9IalkLBW$w1iGLurxBuPfLr$w z)kHS3Z(GYm6{r8E%YD3a4yW(9MxA&bJx4RlJ;rGK$=GuGy9`<8sYpKbpD}rq9?K+X z3X2-&ZeaP^{0K9wrv`5zf?7!Ny?eLrMDF__AVYpb{1?z%XoI5RCUY(o5QZDF}VJ-#PTwQ}sUitt&F zT1El1|6EG@H4>O1u?V&zz;X1{$tD0g%GRo0Qmxl$T{)ZMhQ@6%p+4AiI|?K{Vn(NB zxR2I5O0^a_yb{Be@57Lft~titkM!B3PP2sE7aGI_8YJ5J#6-;;7b=Jr2wH83ny@3) zR?6LUKPpPlf6cHsN)@1$JCvlC!Z7lCEK&@ih&i5UNM*_io<0OJ_JRo?fH#6tG3=VY zK9@v)@S92F^aakmz)$c^f)(sxOkuU!%uq`!izfsX_xURziGgC@A<-{=-Or_Um_Tw2w zYH+GJMo&x4h_vhE2*sPH4b2&2;%)3lXAXo_!ggz7zfdFpwC`$GYY&&e@P0^e>8K|w zS-%8Xnpr<7#}^F4yu)X^ppNbaupM(pvw@sbM`{>~dd|dA8OlpOMgYoSf~04v&Le%^ z`P;u4Z^CxFC*%rWGMy*fO@B?G7kif6CswuO2TTT)^yzrxMo;VvU+v#J>Urd8>}hXw z3Q;AA2(FSsCot-tud zBfALrRjcTsb+2#*b_fe^O|+{nA<>IG9`c%q>ugii?q)o_ug)E#B4+QwQqar=bHqH^ z8m`@uTHn{+doXYFA-Ru@!OUZ2-30ibBGzPJ8-5;H%8FEQ_R;g#%10@Gw-&(R);S^T zgyKpmS(IsbFn~X^PkaqH3Uf1@ymFE@%h3}13AotCC+~NIS>ZPZ&D*Lj6+%n(J${#s zeEY7SIAX-+k278h;|!LM`hCp+@XIfv{&`+h&*INAkg_)3P^BK70S7oKxs(jUpAR~j z-@Uz6n;PCr47wNV)LAk@!PsxR>{$Luk z9vQz}@L!X{?|TynLFh>Q7E?n`X1z6%)x1!G#r}suG3s;i!rWqFIz9Yb_+7JZQh0CY zC%fe<^X+;flcVr;^(C(D^jW~-m`^QqwMS9Xhk!rU+NvVk6n)5_Hxnkhxwh_- zbiR4ak;Bu!z7<8yO}q4>EtU&Z3Rut3{GxourcRzNsO~}vb?kG@El~ZoUz%aQOW`ij zEZC#tDN^Ek5?MQ?a&x|}cZ}`W?!K6)W#s)<82=<4Un=4+$IiZFN`ljo(Wf~c7;adF zt(swpQPU(bSTwuIgc=kuit7)>VT1Ne7c`Ku;BHr&v(pFL%JyXP;Ui$LWCQ zDk1QpjrNY3MG466;YeWP5AT~P*J)urP!x(kiMx4GjTT| zt5q(3-A7tc|E|3FP!?QGw^t9AW|q`_;SKq{Y>q$u)U2~>480J#TPw9FE6Bf2rih!*kT+egu8CvP=7A*Q_!c5tsUJV4KsB-xS!LG~@?)>eqVy zbI+B&k0j~Vd2d%IqZa+$O1b0jxn9M+NReAhuQAt@z*VkG13~5RGR09K2?g=?B&;dA z*d*o}B@QtYG^_RWxl6c;_HqRKR6TQD59P1bXXDgk4#ISk!cpu|QoA0c7`Lu$YFNcZ z)#Bo*;biTAmXi_Y{eLFU)!$79Royp4B|c@;U{~^phnra?^GT6=D8SvyoOc{L>|Z3k-GDnkC-)sP9tWEx0#oxkWZ6urK#Hiz2ckVh**z&M;p$pBX5C`7|6Qsn?d~0 zuA$jw9g)@qduI2M%$9TX!@;aZVWpv1%*EykQ=FNIn7x)5MJLI+ZMUZGpf4_Ers842 zwSGZl$!>G{LI=9^Sx_=<{w&te%r&yHtSmHWhWDNsM}=M#ZYcE)JBR(~G>Oy&iG6)u zgL_YZDBh$YOM8oA#p%@?@9W9b>#I1?cL~h?siAaR*Z=47e>>@!Bv#UG$T|6eb1ifn zbL{})u-16g{~ApGtFlD3?(cmhLU*W-aHqCz$$*~d_y=%K2YYxP$|yPuY_<4gA@Rxh%j~E)R$cfA z6JgRkvZy~|EfAt(qfU8wo(3|!-C*#&CV%3CZ+|Z8=PEt28`ltfZ~gUoXNS zL%^k#8yJd^{Dpeo>LQA4d{_M)f2wbI47WPYe%`L>)3fYbttB9SQUvD2MmA{7xB z&K|SL#jP(Z&yBpWcLKvx{TrW@HQE`RT}^i=_+*=*RPK=N1WVxCZZR$S>(6~{^z$D?80Ad z&2puY4Ul`^1R-#u((l5W^s%93Pf_=(*4kr%5%BANndbF0bEuylz|)b?)XE&fz-85( z;XM~*2@xCW#G|3$<9(B(?R}|@t;`QE5l9$ige8coZ2Hg-rDB$R{J{e+9V8bqyTgp` zp2+3ZKmH3a+QA*YYJ{P*Sqi4`V@kztdh``Lm{lGS)(}dAG4MJZ}>^fd9 zCurCY$iLq;S<~J9#l+tJpt`k1Tx#O8b4l;E_)ZF>|7siTo&(%MKyFji?}vkXQk8`B zNX)>+B>$_h>HV->NUS2!Yp0S;_b95?7+XVbd+BP->Bper}IJa-WK`7SuS! zwgT#(z^WuHNG8S6uXOLg z1&o}+G%TwAT>ytM3GruDyrJ$eyTlQQ)v5J5{;b$O#vd~wBGm{3qLw>Jdokg zqEf=|O_@V&gP>?C8fZ#>rdoy>%_+d=^>7ad+qMRDc2hA7c}p=MQ~p=Ro@h$!bFncuETEDgx_G`DYSJx*hrp-mf2b z5A%}nH}}HZ|3wOznKXgG{2Uz+bsK-Hvc8l7Mu|fCtWCTE#d@+g>HO3vgaMPFD$@RG z>DNQZYr=11s>JhZ-zd_g2HrN)h2u?iGe_qZA^0X9>;VZK#6 z+&gH5=Z*}$td#H1G|6D<5C9fi1QXlEGSU%4@PD#?UKRO#(|2UgedczC0gAGwljn5a zbDfc)AMW}B%KD*Yg_kPbgq1_0wqWS_%W2R+F51H5YmoiQm_o)?;P29QNo4+H4rPq< zZ+k?-;FF&Vx_U-b1Fp(vm)xW={IwOaAp!yG!bVObO-(hhaKXvok~j>Y68_JODV5KHsNxaBrJ z2y+@;FnD*!VXn0O=X#C9QR&fG6Lu*8NWo2i-ja?vrv{=OKfRY!HxMgwb-7T&YA;D7f+nQgm4@EXEu!swr)^nyvo zGwf~q?=DML_~*^jLDUt^0^qKnG{mSGGb+?_a(!ZyeaoYv#$et-#_l^Oo?uI2DGHv{ zXKw0z1iVtxFW8}TPUSKACP9);#<8OO8BXYOb+En$+(C=aXY~~4!Z6^wvjVT}B|(rQZ#Ny$oRHM^nc0+g0)7_0RPx?rLqAWl z94ZW{J^vKd?1PkmXNPMZSXS}PH3MRtB8MZaq5ZpryGm14HkxvGHUR6}NY>B64s(a%8 zbN-1T>*zL*@V#N6Drb29j$^L&&B`)wW-xLm5xCuq1{6aR<@shFPj$;R%5vZi)Pzl% zbi~Xxd7b~E2l#AfFsP?l)@H?~2xq5Gp5W2G5#v&(ZM)o%)%NsqI2EAy^z&G$coB>C z@={9^yIBxv>pmkiL^iQddcjY`IPXkhmoj=R%TFK_nvd7Z6`PuDg^&_L)#4|7l5hci zXQW<_W77-5y7v;!Kpet?AcRntu%Wmztt?{m>+kzyTEUm`5jK z6<`v$I){YN<72PqeiAx9(!)A)+2R6&$0o$R-%NNKV{|BGTZ^MYcrqza%JGD_wb-sc z5mzVDx7$43{G#^xO12fgB=enRjX=B+MSbLGMr(CxnQ8gy5jqH(=kZP@`-*VPvDwJ) zbjHPqWc*v&*)K`_UFP=B!Scyiuni5^S;!XOfK2-e&s!`x|B_#66{Y+RTN{}Zi%^$o%t$bMjxCXq7ZTqztL`hoa~p{#{MgPgD+~z`RMT>Jd|tbsao^ti zpem;M1MRO)uGJ@V)kaFP7J53Jf!kq+j+j=)29$wqAG^K$2i=70zgV;qU1>OAG3NO8 zE3#5z{Rs#CYdM36Ui0M!2BWL#<)$cl4yOuz@qLy=k3!SLa>OGU5(j>qA#p#}_@wuK zzl_*Ce_;s=w<6NpLLS-j_0x%bo8YZ%1&6*&`$mBqot&8z-7UX3_%u567dd6(4NQep z!EF}l$fGvb_JuQl=gvYjpY%@J2%*msetqttpG`z)AjGSIM~tKNE2@Kzwq~=)yZ$=% zHXhdUYPVojWlb>PMx|*CY#oo!>2{W(`yN79lJy>6hqBH#LB=MHQZCAv(w#`nN`N2} z{|D($e*!NIy3dav@_SXAax>w*(vNqV>UoTY-w2`^neVzmu=4wm0?iwl>*0Ht<|FN} z8n`n#u8HFW1eW72NFF2YGUf00yJ79F>=ErOl0|tnMVG1NlBBQcT)ZhHH~ZY4Ba9M} zW(5kab;A)Q#0V%DT#N!IKfvuIewTWGhev(c-GqL3k*ttov4Xa=v}c&f#A9<+(6bQi zC0Q?uKc%ZlagIKyL1*(A>0s&9QZef^V*#s!x`)bA7jF86qb|2yF7Z{K^XNC7(o@CFj%nOkylZwI>KAMXYdvEdcf{GeVyIh4Ejmw$G;- zaTh;fSpE!?h99HZ1rsm&7!r-hXlXa9*b-YO$=(|`uQ941YGL$k6vq8^y>=;Ga8c%}G=z8Hy>KS=%L@|_@HPz+R)WsA4R~uNQ&-m`v;L;DHA(9a5@kzigpScBiSkk&RiaYJ0N!tUTtOKx6)qBV4 zpA)xyIAp|jc`6AzWuJG=2W{T2cs6+O=Z{|ArbqQRCefZc6`H+;EG3^&5$R$vH6YSZ;F_~=>`9x$H8ol zBze0qNurZfyB6sRvlmlN?J_eVnC_#2>HDNGLy2&j9?b69!ukhU`Ws^E#x8D%Y+&h) zvLrH2uNksj)H@9fJG_|i8*L5Ti%K1#!ZzdwsnVnhvXAe=%z}q^ib7U6QVBzU|3LnV z?-bXq)|9@TSTnjA!+fo4BzMV@^m{}LqarGW=3Mx#v8B^98tVUP>a3%p3cI#XkF){; zLw9!$T`CPqcc*kXfDAd((l7(kAfSS@lzlVuVjPl);C{D_h&Dn^V zHrHSlEKZisXwoVok1Ww#g_Tlhti!PiYrp73a)o?)DR}zU=EfdH&L55EKZNIhS9wd@*8F49S0u=VniVxcQkUeGvT#wOIJ#so_)Un{I>)$Z&>F;p=H zc3h?MSk~W?LGMx6OaJy_Hz0n6^9z3#J*NQ!8#g%k5xQoIi-Il3W}aipsP3JBp5w&} ztSz!)n~fd2>LCm-a;tdUGFvZ*v4yQM19xW+SEAa-{(-8o*N2>HN9nkP^O{Y;zvC%( z$%3|48&90xcVw%1n&CO0n41uiYn!>4Q8%7^ywM9^o2U?P<5q`vb8~g7`Fakv$Fmpf z<0hy}@wTa#O*C@DsexyNZm-ohoSh$qzI!GNxqGkrlfXJ4wUA8IMiq#C>6eZ=?n|5c z?ojJ1m9;bIUDu;TmANDT6(443a$N?H`|1C#g=$o_pIt{K?r(PGyWW^#{?}4_7})t~ zpXK>RBW3ffg3;@JFf;#@t!aIs=UrXfT4x^69puVOmF|)3-Z2l;qHCVmaw7@oXhamE zV7b{O!V^#?4aQ;?k#`vSs;@(k5dsak26SV%pSiJXn$P=D{0!=?pYkXgm(WgPF$U$= zmFY&(!vdZ}r=qF2c9L4O+x*dAw6@=SoF=Aaa#MU^MR%mCw9|C$p545Yi#Ivr=M?Z5 z8RESAWw4Bc?iJ2+&sVsBXnxyaXN5ep_4+{B%`q3=8AXwULz-Y5z{PF%9ePoe^syrl z57qAByHS<*cuF2SKUNL_GQp>QuGXxnd9|EDF<e3BZ)dVJ3oz4miinG64U2VV?Tw>#N)khRvSp6pNPwp&WEm_0`btLQN$E+7EtE~QD%$9&u>;#1(;RzO$8i-~9W7sv+40Q4Q1fTeoAyme%@#3Qc_V~q&wnbVES5SOu^ zGybeyak_!f{mq4NS%rh!fV3eJP@aqeluokdKT0#?d2Kk5UBeNL3;|Sm!{XaQpSHyb z1@TPIN^QY2@*!dB^SQo%2qeWkSXaPWGZ`lO;v^f-cVO^swkvG!@ICN)133($54LLd zJ8%$FS_FK(VK-BZ#xi)kUwOO&s?i+J9=@P+xf2yhwSb+RfBhZnQ<1%VD8KY6&4E&V z|5NzG^e?n*6nU&p&Og%sD@IpLgyYVct|AV3<UfRN6D=?Q4H+6u_lK5%CO{qb(le5nK6eff zxA_DTf>WsLkE+bhBB?if5-;sAf?)SjrLaZVnIF_ic*Y`PT-l;8abq z{sVE?kNMod{bXt(Unlp51?cF(kH69LArEKtVWd*Jj^lH)s?I=U=bR!Y?Z^??!fVrKIVk97C-@MQ4w-Gw-ky`wKU!QkO+v7CO|~< zuPOqMWJJW@?CRy=(C8fYG4sZUVfsC$+^waoXNb3wpqN(l?VE!`Wvsgn=82ejNdSZ2ifZo+H#G`42*fQF%d?&wN zr3Szk##l7r7@6L-k{wQ=0cR4e^CAWrofFzc>_p`ca`QTC0CA}ci z@A}U{hb37LOb|+jb@#>>4E@`9*(6NYJ0;c}W5!qW;Ra<6CacFjQ zC(#(OR$AS(033xx)~xaKRH3*JkAg3yRsw*xV2?OXn4JSfp0cT!O7#J;4?PYNr#IdK zf1Ev`lE9Qx!~M^;&$uez$Z*cB=)>m5=8DJRe3NHYeJ8Wvs*6>WqH_!)!(i)qo-vZJXwYlkK}j{d=56ZGjII+ z0P-%)g??$b2weE4=)&pys&yk4s&wP)8EEDChS|>gaiD5eO|$MOfax>a-2~8Z#le$@ zu35vaAAJ5LrJQER-Kh8Wc=pnUTmqb_K^E-?mx{KY3wu|;EAAxj@0No5VUj)Q2%PW9 zVY=+!-ag&1O_XyV8_xu?GXSLF`LCMns2=yoLm54U{K!qW@n*@7MLQ_CB~GDOjB*%% zXx63DEqdJ@ha?XuNynm ze-bjmojR0VxtVcsFq^e#)ZFpO2*C?{EzeP`N1VDA;yDU=gLE#|0j_`^LosVG!7O(M zYcRsBPyxgcVs=V0o61fNVX2+eg4<=L){RB`3)U{?u?{Mjx$jjg5E~=X2p;@P-;(oM zg#W280j#i4?FlK*(2k@HO5eIjAt0M#40wb-a{!RbILxoV4`Bre9`%I2qu!Ubm+L_A zqvBpV@LiVIoDM)`sIWDpUa|At3uttt?6_3^aG1KA`cUKBBI>y9d(*dO4>pu7gI2XYR4#o>$M+Y) z6c(OgJeb}wkD0~8OmLY82lH`tg$wN5Kc z@US@4{)}fpsx?q@g_LQ6-(?=pW2ODoN+G{g3e*U&#oN8pFEKolEg8HqziaL|b+8Hw zUVZ=#@ro}o4fbesF1P>LUWC8Gtt_et{xja|FqR;tHgkWP?K6~Dab3jonFLLk)~5&T z_=vZhAOoO~4P|?F;MUMqaUw>*(3t?ov#x3O_tO-Y05q<;-J^M@q-H-Bm2_<1^iqt^ zI^-yDi8L#oP6kiC&n1SpqH)!`cE=$LKC+90sa9aFXJj!`Jy2^N1i#q@tp|LGT}$MeDU>vCF-8@P0=)X-aig0{QFu5?~2sbP`GZ{FgAIGgaQJL#zN(=UG~M^E+n~ zz-tOC(*(Tx$?c4^ChVpt=}=L*FHh1rADb9RBbQT4t@GtZInt(@*hifD?G_Ug`SQH- zRxSy|s3xh^@upwTEX^Z7woR0nS}L<)UX>j_>8166QUP+1u&P3zL;wXwEb5r+Rgx*? zQE@$)_h2_7aZ4j?mE|^+vGg$YUiNYxV~X-169~ev^sk<@QSKS5)Rq%ISa2r!0WgP$ zER-&UO0#cB&t}V}0xa(K;)b`!e~^^^dUE>=NMve5rNp)G5x*##z^y@fC0cbwKiPJY zP1z@+{V>#*+^cE^u+^-UJLP!5{STyxeq(zAxql)x{yr&id#vCE ztst#{l;WvvKH*ldjN$5@%~4bsCNojA*n*wz%=dyrBh`OA#w>_9wBsYHd7xbZ^}4Lm z!M$mNyw(EAS4eSg8}+b}%|HBwyOPP%OYU_dqf+F6Jdx=lC8aO9L}ssC$LOS??-}0!KpI6ho1Ge^nr@m#Z#C=e}?Rl<_-d#WHdxc z%{C!{a6+-irCmXN6?=Z7--#CWeVHFSOgI3M@q!s{_%w5m?HS|->a89yz-DwoKgTys zDetOWqX1FxL7YH_W}JCM!B45uu{z?a%wSriDU?9Vs?{nmpKo1ZE>#xua^T>c+IgwG zh+tmH;iW%=x{TCvDCY3-{R@>$A?>|vG{dvrSE7h=$0(x=06%}+DS3qGN}MbzK6`f5 za=N;_eR5|3iI)#vnbXS1!MPB9w~9mU&R}TZU|FN&ZY5#S8D?|ZN-MQ;3YCL94%az_T=ssOyuiUZ^o%qx~=f`zrNVmqi zikBG8F=3?7>T(I;E*gR^0uRA4tZ!)HT+#8R?>U9JT@?+-&Y5S@#@y24kQH~kG(gfd zirIDn_%S~+=>J~zQudjnMV<4)?~OZ+srSH`#5)4V0>CGj+tbO@y1o8wci1t2Z{Pi; zGA$1Awd`A3oK>KlMTsb(`^U3y0TOI)OOjseIFY04Y%1s-y4IBDiN zvIvg7{@rG-aPQOo`lONFq8m2YL5xnM8jE1Xg0OH>$uY45=Fnr~V$Xzm3KeO*@74`W zH%;y=ABCD{ti7P2XInEwrMS#pR?9q*wDeN6xVdKVs%L&H&nc*A1_XwOhlM_r=;j8l!NXPdj zyw7>C^k@UCD=uu*QP8qW)jiz1b><1#;~@x^QWM;+j-N0QBg-3UE`UO<=kzV=%nAau z%PVP%qH@>$+8^OOMrwcl6w3G)#wV14Lu3NQvf}fVw?+*3I6-5pDb&_ncyyI~89`&@ z9r$AN016ri({=W)3+Z~IOgTbnfl4_v^=OSTenkO^OC*ref|3z{=zQ}b;v?bP*?*6E z|2n$2oZwUo4=PDXqxU3^Xa(@s>T8orvOpLA0(WjgqNkZau9}v^t{KkWOj&8uL*(9g z(@1xXx9$_o;;)8N6jPQ1ec-={3@`WZ3($v9+a2+@*=B9iXp**L{Ku=((sR2=Q~ zmB$I&0RZVbE?XVJm;AOrEDU-B>R`0xCmt#Ik&Sn*?fkTcvyqwjx|c5VAnL+Ow^J^c z%PwdcExcBTZ*6TkC1AS%G~8#KkZf-~+Zgiq!PE1*%FHWwbEwQ^yH@BgX|CQj;g_qd81Axqd_pU%C#5H` zQ)<>+R z)nhWff(m`y!g<4YBhRC?8FBbD=>FGl9FSJmIN+LaQ?z;FkY;wVMn|1kQ+Yl7A*C($ z2aRKZ*G@uM@0L1Fp?~^I`_}t}x$3Py`I&dSEFqgC;D25$PW4|9<6Q~JuA}1_AN{VT zvTA8(!Yv???wZ(%7iwI#U{+$6e%qdJ&m(?S08ny)xW7A?URW(VUxWA20RSye2G8w| zB@(?mow)W2_4xz5Lw>tg0VwO0=zc9*;i4%bP6a*%d#~NCt8>uX@{iWM#xGTn=1bS{ zs0T&nKxe^&U8Y%|J+z;GUtUi1#~}j3AvrO{Wa|aI*Husj<&b0!=n6qRI>^IrDhg9e zD7yLXba2>*G*?vF#!9IPiK-qC)4|~biCNEoCoy>JIbd|$B-dMfA_HAgx)_b=Bq`{r ze}Qi_o;cjl&s1v&3J6?_B4^l%AVjPw+|OT*k)m46q6>o}bq z;B&moSAeft`bb)vdOhujUwU7ugW&JT^ZFMbdHOt0_g~VcOK)EfB+D7mPdhlT_T_|-iWvhYZPA)R);UE^}d1U+_MeO}Xp<((v4Z9&Mu!SrW1 zY$7L6uGc2-hnLbQ!JNF4Sn^)Qq}f%c;s$ohhpOf=@U6EiGY`F2d1DG2NZqW$-pc&f+i=}d2a$Ezt?XLO zQNqAjy7Erx=&p82mA3@V&jvS+6Et(|BAY!|n4E2QHE@-4fDWO-sdR3j+#TIPjoW$$ z2|Z}JFcIowHdElBWOa!@V}Rw%^~uMBibbk;^pJ^FCN+o6dn&Zci?2gt3)}6coPHv$ zcTIH<7-^<02N65cdhe8zVWp!Ya!Jp~FtNR@)lgho5{AfN34gq~*%SRl6H9A$DD@{g z61xnZK~)hevqyuM6N-CqWR#=nkVBSxWHB;o=z1$J@zu2L>0?1h_mFlvjLts5V|7zX zl)x;PKq7Io9_`x|eLw8V!|ojuN1RXri1)WMEYj^e0=D}glQs}4Uw;g*prO!4v;G+{ zebN*IEt~uHnD&Vy7QsC6 z$Kl4`-R(l_4^NTx8CtUuj3L@I>8R5K1`w8e(>eK>#6Vm^WR8TJgms6{w95hW$Hvm0 z(`2G0lTkl{b!D*8V#{v9Q^}W-;*oCz=VssTx!EPgT&|xn(mmR=*oS8P60RajpcbEM zxtfSuwaR=k%R?e&%Z{F_!7S|>W7zpw68B$P;|LW6GhuDRmZ8RBoM8ay4`m;XZ zKo)*aaE)Q*BA^!L=~n(R5GMu#}cUXkUqXds1ix9Pz zJIdY<@f8CIJZB-b&C?ygsJ2c9CjRDov`FX?cs1r4Q!E+@#LVebn2oGHaX9TclgVX1 zgW~7o=PXcrXrR3JzhE5w9gDP3ZlvrjnF;XvtA$C;-D&f_3*1=Qp>;_}W4(lRE395s z&Uv(QhNln0mAoZkxde%AXxu!Z`LIk7(R`vwB?mcqCgFs-SB1(GP^>KX0N!iv!7qP- zaj#eVx=nx-wj(ajT)5UV0oU%rv+ux*0xyD)Paj_gQP(dC_+S!U0UkxhO%@ve6=sTF zs~e1)M-%okE1kYm@JcU+HD&fZI~;3pA|`=!XJn96U{xsX zuAp*an$jcN9y6xeCnE%6;qg6YLOAR;QP$wMp9o&|Eaww+${pT=9I7mF-HabPHIm-M z+V`jCmP<*mclP4U`Z+~`CoqgF)*gP{9*uTPEcI+onRGDZgJi2!#o;?9(=S4$eI~rM8S;O#YsYSistJQ&vD&n5=W67c8q?7$A&Zrvq5bt{wNSOwgyr-nHBPjADC zA6$a}6r~d8>I({nojI`xo3alHz1w#Xq`7 zs}CG+48p{I0>4yvfeHlW#;3){5oT9ruX1)@p8PKL^N`l9<-F*xf2JI>A*^R5>-UFB zGMzB+B5hojI(^lrr(5gdprO=Sk|?x_^5Ipy~L0oPB%@M}BUY3+=oa)>taq@8O5uG5kB zzS?cRSkiqRa2lGJHY^xaab9VT=#u8^+{V)pU;o#CsE49+)9l9;q=-^j%??ELdNY%d zqcR88bB7rLRygmT$Q<+(8@G5V5rvPfWC!%@Y%)QzhO4wYSlLF>(r^w$-1m*ZUoXR5 z6_h`E&iAgCrA@~O$KUcwS$!_Ubdb&#)#zHGJ?w0WZDrpbE>@rQvi0S=W|b(-@Am$( zr;}XZ(r`GvG;cP5&*Li7UUOPIs?lEd!J^mOS-F}PIlaF#Nwa@d)3h4PJ(M2OI8h$a zDb6J4wATi9L&4Qg37+D-x>LS(hTkvC8(Pea)L%(&VC_o)ereZQiCoZJ021Y`msd+6 zO+%>M@EBfzJ*~uw{a)}B9tJI=l=YMn5qYDB`fPcTzm>Vx`2^AY;N8MerNHo~GBb%s zv&PW@o7NJOxxPKpluQSN1OYn9BWJ+mFYQkBw+X412RDIqeR}IN=;^5%R~^#4+OU_^ zU{wo+%82pr3dQ`o81yzXOo||Frkue2ox9Y}VMEVYXgLqJ54L@;0%o<1^<}h#l7nd> z=>Z7T6RE9?vTtiRj{PtqJlu`EHaW*j_mP?yR}AUK!)Rr#GL8@ET?KQV^g6+?T^ zS|sEHUv`&Ls8WiVNN(4Z3~X;%;EoKmrKm4hk&a{mZ|OuIjJ05bRcx0nl5lcjju*SP zYFKGONR_{V_$&ru<_^Zi?0@`yvK}=vKgI{ovgJ_)+EWzX7o_;5X zdxqQB|wa;RRBt#_AiNu>N|F{+xCgjVGVb;hS9yFPM2_ z^W#3-Hk=9XJ@8!X;os^X;}^xp`vNPm#-Iljv<3CmI?oZU(=T&kCl=IXp8l%05 zK8GKqqOwp0LSFI3kzwiPRK(mIRGi!_Bx8{)JS{I)I~gpcSZP_Eat+*9Nnfn^;taqF z2+gBAmpfMwqcNNbRoEfBT1LfgZY$$#xMWE^2!5ZI%!~g)(^dYKsp5w5N&O(w@R6~c zsw)=Ei!GNhbLZ^21Sfqm`lg-Z6&k`SKrd(X_4Up56^s<$BOHZzKIZ{kwO5O zfm2{jlUNcg@kbir8{37HdM)$iZS`O3%E3s>xE^!xyuS{k8IAa&$#LpHCJqtQd9JH% zu3p&7v%IR@*CcFmO#5A&8UZdzSR{tNPGiA4j2PVYi3G;b8Jh+k2$bwlgCQ+Ihx$nZ zV+AB-q9!fn{>9k3Yvdbv?MI``6fbl=tRj{x)|Wd;GwwfE$dY-&EYaw3s1XLCdaEyv z2U&F3{BdzlXY$pzK59wkB{L!NTTAlYUro+3^9H}hHW0w55(mjjHr5gmG3OsApWr!U z8x*>?=D2DO>%}Ehy}+oY*?zy)gClB_lRV6mWX$jrgXY;rL*c&kh};NGY%;4@2KE<8 zJa>VGZC;bO;51<$_jrRap#TF{{Ho{Pv#xMRqZtaG^&uV5Y1)zZ{riMkd6ybA2hWiz zVLAAqRO{(CRk*2a!>;YSetLvK&`A9A>Rid+sea7qid0e`QC}kocZc&N^^_uCk9~@i z*;6QbmV+j_5NIO$5obPK&YDLUR-B_@ncpJ zSyVRooz<6DSuDa0x5-S3!?G2zN*|?8ReYSTT@2hyxr*HOKuSQ(nvlkuDPPAi}`aD(LD2tVJ?&`#N z@Z~TdpUa)!#CKG((baSJ?t9?O%#m7|GQF!8V1{s7pkC1sZ!nQgaG@%(dN+1@RMmIVU-R=>7`4#AHdH04_(4k>mb@?| z+bXJZ)!#!XCcsEkp5eQB0&lyTp>u?J!tg}l7km4$!c8K3M7qK!V-xj4>Fr@S>mxux zDUM3elyQ~$rpB%0((Bs+gjLw6k0^!dcgq?6z_Kuu$=cf)>e6c{^M|U$#ppu^dlRCA-(G38cbp?a8%zzB zbLtc@?wkns_qzPn<8fgB(&KtauxgIab{aNT5a@7|4bGSC}A|2lv+>BjlskNO7|;$W|nLK z*-?QPgNS4;h!c}bE2+A~F#O-)#bD^0>+cYEX###8-srUnA~kTvWTz6?s!fr2nu9!VE?zUB|BT*UuH!3QK#sB;( zCk)z`7T1Eqz$);;1n45PA>#berI&mfNu^n($6kir{P_GhMmycdLl*Tkw%I(~9T??4 zxLk%<^=?MzU5R8oga67#zP>41Q0lEHx)eOlD@9*lj5lv8!BXFLDu!|}KX>Fa)!k$U zpBt@4=XaCF^zCIp?x<#`7H5e&`l8+I9J-d?H#K|0JuZPubj!AtEbc)}^!Ygsx0;Hg zEFGI>ugbuAyWcR7<=FDZMA_Yh8i1+AzZ^K{fY%-Lb$fVG8^koZ_=#J@$o~JsSS8s} zUiMfm5XkM!CIhtS_2abKp36+|YuDxXu(B%oS?EiTJG~cL+%T@qc=$ZkQ0R>5NPsZ% zji;94nvrlSORc;^*bqi0wh(@rE&AJ)JCTt?l!OanBWb|u|I|rl%1l_Z3QKI~6!JpXT-FG{&zq=RN$Ed(x2ufCg1-FSa;cjxd;N{Yxs%ima%8TiYMdo^v-8Gl&bv=S?`zau)I2z;U(9?gzA`K# zFjbrKc6-WBaSruu2=uI;Y~VO#+(BPue0-t`_%|0`@s09WCBLL3mgM8t7LY_>{UFm2 zkn^V>Uf8uHf+dlAO)jTeklaBs8e8&PtS@=P@({`y^B)ihv6IO$D<4(3wu}j8aFY-M zQ+pcdPTLSrHIqndp;vs7jneo}h;>1*8rQW)xp$guW!y8uBl~AY8E>*M!a1Rq|yQ>{_>*o#DZ}HhmmOsq;cpAO&)K%qDRl;9oAxB~IMC2Jn zP9@}Z-dzBI<@>eRP3v#;}lIW2CjClg67hf9borDo?- z;skei@|4_v6>S@gJIR*@zBSCHf47UQJN16xss!pC&A)od8&ChUfGot&py$6uZin?h zMQ&9%t#Ac6DsG|XYGASz!Tckx^hD?Coeyvc50!}rlYq@K7aCU)6O7SH(&9g^?z>|H%ytZWzNSthndu;n@ng~57$GiCU9;&37|Oa zx%}AO-Y=cgL<+vGQS9Uz6flL06(B}vnL5N`F)LSbtJ7O`Xz9>Z4!H^r|w(yR_hC!^;d z*=O`97Ptw-D{zsq@eYby{o?4v`&B_}LIwY+^+jdi7YAH~E9z8RDd|H=_oWKNu0;m& zkxX>H@STcjgHa^2zfEoh)p?IL&mV7`X%+jIiIP>)WRA?~)~px48~oIAPrYa6nVy&l z?!HPa>zC8%PXjaATD-i?zpSx$Fpkq&)8R@kaQvUH%OQ*jf4%DOlwBinua)2G7eP@T zn>5z{ZORbS`Fcv7^` zP+Na-H}5q)+PK|cDzgc~)!Wb~$#qu>H5;duPlMAIVkak{LtF2J&Xc`vLBSFCAjt6z}CWse@ItmgjdW* zrrS0fR1!T!+PK7){#sluEl!!TIp54Pc<{Y){#Gg8_Dl9F*^X-?%xcSRx}JCO`HtpV z7mAl2&%xyXEJ+J?ixS)QJ08Tpw)MXy8v(+%BXSSm=+=n(Zw0p_=ctk2T#nU9`QOEC zKeY-QLh%w8XYBl^;r(~!bMihKvt-qpcmH#i5gpI*-<+?wyJ&bh6KKGX>T|7U6^d5j F{|_bD;P?Ol literal 0 HcmV?d00001 diff --git a/images/adb/aei-id-1.png b/images/adb/aei-id-1.png new file mode 100644 index 0000000000000000000000000000000000000000..d407a624c801b01cbf5bf9669078fe2a351ec000 GIT binary patch literal 63773 zcmZ^~1z23ovM!7}OoF?+LvRKOP6!^{8C(a4;4VRfg95x~jTYcePYkzZ0pVB#Vhgj0OV(gDEd3r3M27+xSXvpdh{eHUobMz`$Ub zT1!f*$V*C6syI7ZSlgMyz{o`=ry^^s58?;xrF_EHrG!(HaLkp%q@)YMe=lB4$xMYx zn?#5c6h&t634BZ)g=c`L>gtAQ#uhPefa`6Ls#ST`Wv|(;uo2fJy7y8?J$wa0qkO45UO~ob*Fj&|prIgPq_CAfe1grqPI4)g9t zb4hBdbP24)T^U*HZKAZVoaAPWx)V%aeGG#vj^Glgc?>}=nLeiaAH^((nn*|IiQ6EK zhRJ?4+zbhMRsNE!n^#Nz>WoTqVIL7;^6=%p=2F>8@~Le1$QHYCmPl~%Hyg>8N2}jC z!l$u)4MQ$D23e<_R|>?mu_{keaA>loKy!KZvpAF@RQ8aUnAUIZ91)r{X?JL^m)d9B#^l>+pgDaMbI$372tl zCx;1LmOw@X|M%Fiy&a9~ZH8|d1J1|c)vl%y&91NTFJlH^Jx(SsrBMku4P9 z;(kb-cOSv++p?Dyw^5jrGDc3xB|W+82t9r8Q87i~3L=zo?NNlnECb?##woQ|zw_R2jWe@ z1sEzWP{WYq%8+D7r~IR9ACSUP#q@gnb<7eraI z9GG>wx968=No7aVKG%b9{7|Jn&vk_1j#2{8dc>$1m)Vb1;%SU2a-b*qRMUHPmk56> z=l3nHB27J6KGe3(HvEu5kz#;AaZt;i^F(UlQxmO1#`tDOMLl)dOE(j>O;^?#e2-!)u@hN3b1#e!~?&#M!zv z0B}Ae`yFKF80|W<9k4RnNRR*u6u2B?+6Y)wO2p`JFZ}K+R^-G^S59QyV6{rzQaIaA znM%YXn3hga6=bYH&^oOEGJ8yNO17am>CgiQCga8K7DS65J?6vqJVOf$Dx%T@mwIuN0@cVjLxd@_ToT>;} z@rHZ0=RcMruyiw55tqU~Pp4M#mEt@E;5Z?Tbs4M^$9CX2!HS}0M{;z{Z3;tmAw=!b zYyB-y4g{gjr1g(bIw%SVMJHr1#8iUFgk%F|17TV5IHW;}Izhgc;dA%OH;EB?B^q|> za)uKsK5V^6K;#N7_TabekLL`%@wfRtYFyTs)&zeT|6u>hyv!<xdR5>%0~vj&&CC7V8#MYmN3*T?4BJBR& zX^ukZLZnB;KzNKK1@8@?^}hc7uTFL+F;}zA<>WVWQkx7AWTbKIwj9-A(~Nu3 zta=bB5r;lA7!&CM6CZ<}2;vB3Q=gd`xNjcXoUCxHwD>G+vARGIQMEv+zNTPF+HZ4f8zPfmwaa!3rVAi*$7CxFv%3C+ za^BzH?K7=grYDKJU#^g;0q~=1Ve% z>wDxhf%Vb!iw#Zy=2$Vrx;(WG`(I4>y4AwjNZf7a3AB{w^ykXT8^R9(6oL>d&u*xe zgOUSW16l*iiR%sLjn2u{35Q6Ah!i;q(7}@%DC;$LV|V^|v~x^%Q+##+tv^$`G2HXb zwC{*D=P=E_rn*IV7&#Z*zUZ7A=3n#3K3(%;_Y?FJe#(liq@W_Ma_f=wXYp@)v3RM6 z(G>sO@iU-ad|2Gi*nB;h@{rPGqW%X1sjL#8CBdF_rEMkfRvao^&f$U7z=eR03SdID zM*R&?#4sTF%rU~u%2Fy+%Ch{exEVy4QNHZy_$7`((s3@vLbJpgfV*kzFa48K0HW_U;KsEX-)|bZp(c zba)>dj97v=9{Rp3xeF_y-KuF{{-^)$`ADJ8vkptOP*30{w^LyViw=ud8d*Xs*Mou5 z$Fb{-)#dGF5wC5?B_vqQ*E+V=EBCI42}jR-tlCs(iFQrCgx+nCW@iaE!ZagSbPa}l zb)1DPNsQ@~00H+CE+jjH$Lj^Rm1&A;uTV(QK;TN=M0H{Kou*h*YB`$zTU6NRI#zH-P|)R>(V0!x@Y2HJd^ya^;=*Fw`&Ru{`FHPP@A>B8 z#nY~%egH>2tCh}v8+f#fgRbjqO+k5qc|kTlDt_jZ*X{U@*bV!b{&IDX?O<3zPr+2d zTnZxlQkA}=*T#7`i%eOO!Jw{A_4#NqYROoVSCYNHqCuJdiYB>+VSAs&=oD}EY}A}# z8A|zl&6w*-m)q$q`kc|?o7xwx;^2@@hMDCxy zh2MOM_619#j;RigOK%(Unuc16Z3_R4(fzint8NBqEtY?2k#wJ%Yn-uL?Os-ICOFbK z@x8Xaub8`*KCxXKUp2Z&KXyFC;>1dtp{`T6Z8_+1v%g}xDnphfx3huT3P2ww)*2|P zfuEe_q0p~1Gc*^mm*i}~#T7=|BVdZ#k;!z(&!zr;()KA}bJLF0cH}few-UKnub|sy z?aLoYOP_I4;IOEk$Jn9%vDf$o@-y^tOZl@|+JNS8wVdS}4ldj-KUt~v{&~av;`->KoUXOB z@j!JmE*9Yre#lwhxR8yK&0gFS`6rSn(XiXxD2H`AI@BbtQry-jhzak8CT|}rNKZJv0y|HVUU25D*eT$KE8;mYw8au zj|`e?=nqi@FuawjWOGe1Fh`-k4Snfgi}OB9yb)hnK`#Eu$i1>MQ*QlS8q#|h|@KWO<`{AOdtII$s$-O)gd#7Ar9X2Q|*u?>+K0PMIq zZiiv~6>%hNGVA!d=}0lxmbXw;gkgTAQD6{Zv0>m}DcIL93@kAW;y*MD%m-MK|Dx4k z8UHN<2Llss4TJD+8Qs_W-%tGO^{V~vJN(x$806O{?AI&g2i$+8VH^$t8)S_sVl$1bcGYcU#DVcw(zdngjTe-S|g*Z4oJUrMv-myD6TXJv- z3JP*?a&vHVv%N~Nxp+Cans~A~xX}D7lmFx+W$prUwg$UeJ33JQ&DX@#(alwan)+`+ z|MmITdzyP%|943aF8{99YXv#}ayYoyIXV7cYOdB6|37MfIsa1or(XXm4)`}SAq^LE zXGuqUdvga@(f@97;6Ek(U(El0&wnYZSbLh=X-iqZYP!7ENtBCQko(_i|C{yyRMq)! zRX)D|OY?tl{)gt@ItVE{Tfe@j$=^m4A4wuf0GN4ao6dJxUaf5L|K!10xP2FZKSDC+txs@(+FK-~Gy; zPX`qM@^TUq;o{<-E2a?eLY2i4h6{^u@x&@wv{kBL_=@EkR1p4XP>Q3(BdE!pwzgd! z-MnPB)OfG8wQ#g}ozig(W}4lsdhfpO%`Gj)1YImn+GL366x5jmKf(P%P(#hfp6{fc z@Ibl7?OdbPmiX_P&D3isR12fg&Vu*Rq z-S;r`EvD(;1sn}XdwF%5nf$Ka3;>U(7^Zk;1)NCu;Tb1{(=O8z_9m`xQswdSFx$yt zm!p^!gER}%+4MrBG|~jss4cFLS|CN`#0{v8aJ(PZQ;X%!{D#j}VjQsb~8f*H*5 zHqSA#Zp&t?C6-Zm?}Xp-ztzrMW7Va0r<%w{59ATgmS}>*E!7=HbFx;&PZ>tz-#}B6 z!}?+#vIIutSWXojm#9)nHo^&fD^pi$6TqBQEK0lC7vQuCS|2@2vh^++9mdB-Nkg3L z^1|QCf#ID=%p-N^!g~Z$ln2UqV$B7RoTK++k2@^~RCuYot(&9b2Gxv*8uzo4X=jGB z2HJo~J*f=*V_`uJ>HT=lmBJI{^w|PjZ<4UZHH;6I`V_xMl=1+=Kk^MW7L+u~x&QI_ zPy5@gVOuRXR*^J5l9Ot3DgMhO|Fk)Bg7|93F(u}Q^n`yk_+RtHzozwqvVYnA$2645 z9q#ObkU#~=q$Mw{pGSrN80^(R`fy(GspadN&P>OmZfGA8r>~=uuN}Mho5Cgln1SJ8 zlOzRBlj7US{9w0E6@v*zMITJ~k{lsK%p3}0I6xg49cG=s?Na%1+2ia{xan4XJrFl9 zeTLvFH2ss7Z>J4Qh0rF88RnX;&j`FA*lVbL2SR8_>T%a#A0(0s;@y@>p0<>4ls{Gs;x!}6hiE+QL^J|z zmQw53DE8?5jIdVnok$An3T~E2R`T6bX!`TduMk;e>wCc<`(dI+X0Ze{_6x3{9el;J zh6IG6KQ+$a2340Sn;L9JzCRu^u)pUTS625Olzok2?G!pjNg9vkQJ5;aO-Ol=hFX?4_@Rs^-UsuS~OOs1*Kd#EhR ztvC6k@?!rjY%SY#eR+H3F)B5))Y%>!(CH|d-k|jqlNKd_ap|Xq1pu^ z@W_*d{tA^T4c^8Pn<{q`XQNY%MNo-}AwE>D*#BA8b}4)%UcI*9B4p;!jr;t(kW#r0 zvQ=M+o>mb%cZRi(WwlzjPzSB>W4SQEB$g|sibj5>K}FPF?XZtcFvV5#{M!q?P_4*$ zr?L;`lP1@+dM5s?t~>)}?mTiAYOU2@W@HkQjhuAhQ0>&j92OfV=j=_MkVun>L?_86 z>=U@_of2<@)11`_^+q3+gH!Z`-ckfNsh(5!#%NJF6ArJdh1EK3(Ql16ghINwHv}gN zSfA8ccl46^b9=!hT{${7Y4=@aL#m_7J5ZH(1wIbG6Cs}kT}j{^45@`@4Fsz-hSR={ ze&?;>0`<)U{GjlM%S|=DH!W**St8zxVONJVpzx5P92R~0i>U$y`>TWBvjqwS{H{d6 z6Scjy*`s9VMNMX-tM@K0Pm~3bEL8LO&w8*e;!`+UW;lhdc%QbquLmM8evaXNCf8+(tY_7!?wKt$ zur=VV|ITA=cWa@?Q&0Qud4h54l0ob)ax_D5*zfrwTc^Qcg%&5qzpP%4@EvLW&T#5d z-JD?wB*wiPv%Qi@{pb8m+l$EF%Xx*7LpYhcc3r(nN`09+iQ{cAhu!7=Wc|44!~9OF zkwcko-T8AANk0U)nrxSJ1}cq+!lw$;YnAlw?QmyV;JA&>)q9V3Ji(>$T8~@7&BVc6 z;u#KHX-Jo>>~3dgpGrSwI4~L=>0&ux$Cv_XH~2njr*Wg;97Zx^RU(snc05DF#pVYR zXjm*Qe{rJ-OY-}&3L6rw_TjbbtRQ#m{cEZTXnX?EOi$8bE9h@ zBwnw|{XKyAeZ9UsVbWkC!&0$E$)g~9xr9%aWG;E%ESbl$N$Yk9n_m6LamCzyN%1K> zh-W4jHW%v(^T!s|zGAn-Sw8T(_+;x!Uo6RWEeU|JsX*|I5SLiYg<|OcEmBUH72-DDT{SFhJDm6mp@&b6xDbrF@5{FJVE2vgygH z{S-sO&)6akj|;?4Sxd`Gz%%@yW=d4vo@e+yRV(PDq=EjKKya>Nv3E)5hjs6gvuZqM z3E0Zd>B1i625!GKM4eQH7~^o1fs?wmR;mS#-|~GJW!rnz{bGM{)VQ!-T7QFXq&ZJm zE$06k9~QSv>?1TO2~fsV+%`~WFXi0a!r>qm-E@UFs%6PyRF2x>(}+M9CG>alF@}Df z1HFpPsNo7Q zL$J^mB;%bGaRBlVbVIZ&<{rxr{ZCVLucQyLDt#=;BHjVzcp_RT#FHWs z`E>hzQJZU~19k14ma#Qj&V8$C^<{8S^q!B~MusD4Wh*?ydli1Tx- z*t2Iprjaa#TW!VnmmB|=IrkN}Ip&f^mFr|JL-$MHo0Yl6`j~r$q&OoN;&0gpQRwet zds@vlyLuJA@!5ahXljJ9QdUGnNA1nx*Gu|<6M_=StdmZM>*o1JDH^f8OoAn=iKtmr z1tc7|;j}%Q1TwQ$XREbjKj1Jcbw@JGl0;=mmHLUj8)g7D3aV&;LAw#w zY{5i2QSK`YNJc>1H8b_u$!dz!+9XxS`H#+#z0N3moN4%pb za$GwN+;$$}>YvY-PqZ5sxw&jJ#el|fVrTK2;Nj+u(#S;e1tIYPyJbeRw2F#ao$z zlYns>2EvfuFZc_&yj!IOls&8EDl_s(D z<+sB@teMV+7qrn~1!=?KT(od`D7hq~!Z;#$86}RUMLbX93HFIolaU15&0#94!l>Qv zI&Iviue^0vw`JQURr}^-#djoFR%h|^xQKs^y4by{(J`#7U;gyCfB}%BUdzy{7c&FC z=KLq?W99k7bwj(odZCh&?o7IZjvzDxUDe^Fu(>M=l#!B=YWVpV+ibuMstf>{4NFXe z;e-Syu-;X=g$~d^4{`v&hjVX-0=S@HWpXCqyQ1z^mTgvmhL!vTEQ6;KFdP@0r!+}9 zwj0XrB%K}HDVgGc_uc8v;?Elj3!l#8V@2X;BmXD2 zmi^o>`oB~xPgh%yhgd3VQE}*{hCW7NH^%|bBQkxj7p^zLDelRsJOzsN9h?8q^Jh&+ z=75$}XggZ09QREE z39Sd>bClpK9z$@H2M!TpK$u|1@sf(RgD9s|19zu86Uz3hGRLOG4j}@-`3Pg$pRX!T@;kga8%vY80{ovZjLZgvy~H@#<9B%H92*9Pfe9_k!*<%k{1O@29hUSf^42i5L0N1wx(kTBUAK1hf3^ z|FEm|dCI|3wB9aXj4u_6T`}=Pw%=XLh4@?VDOa?w_M1)rbcVT0Jh&uxnz*u(h>oCT-vQ01b&5w%iJa_y60nt&rO z?0x4G59IJ3cUg}t!F}_Cl>Nqu9FE-$`LnXZVUIHkO_|Op5<}mp6h-++vhF!2PI^Eq?Ba_Br$q|K3M)+G5n+_Pu1)P zsShVD;}`i-+-fpuEhi9`i&t;&w~5+J+5?4W#~vDSAW3uBe#& zwOyzY8Fx|Pvr`?dIhMw|;MJ|4_jvl+?MG}YT2m6!%{{vXO4;KaZ+qO~ z5I#B5sUal*?n*zxvx5bhhy)R1D_ZAC+xHa@ydTf}_l$8Elw9YD3bV7k?*A0Niw?ES z^5S#3UZ8OC)b|+Bao3ifD_8ZPS<#hk5Gk^Z+gA@qNUSeHzbabNs~)h z`><;^|HfVuXF^hCEEO3f?u({z-?5E`6?Z)u?|vdSIp7O{SezL`%$#)aGm7v1%Bi2$ zam-?AvK!t)=pComxmn>uH^uXWd8ALRuG^yjEr^GYJ}Nw(@ZCU!yA{>WO>H8=D5Zy> zW16zhCQe!?I)R0f$koh9zu5CxwC3?DMGi&q>QE?S$#)SA9EDsC$J}3h+zid;qOTo# zC9mgQ7kvDAqBI%Hu<*#uepjDBO3Ul|fBkCdCC`<1XcxMB2P|@9DsWrj72 zScIbbUad!Imf22@u@e|oCS7oz8mFxfS*ZOMg*8F=D7a#Nv9%k#sKzpsl9d{66V2xAQ49*HOlQzcyQRzG~j?CiYf z!h*w@uKf5g4IL(hU%<^fL#R_fnd6*+N9NmR;8}hmf(Py?$BBeCeURqpbpExI+w)6C zc6yd&mfKIa!^P3Js3eB2la|NES6h;cN&S)xPG4i=87Y*mpU$<~FTIxZpJas&it>Dd zv$!M|ZO2=Ec(0({fR?$HW<^`0_i|}z(Pi>!0^~->6h86cz1mdd?)xV0@mbcmpcE$w zxhyVO(-G{mglb;F9``O(27wHYZSeUZ!_ucw-d&Cyh4VO}bH=iu)2lO7a{C3v%P?DZ zoD@^HK7mr~MPyaf$tj%Cbp8vk5k459gNF8}GT-Z(Y+fR_3E=xwLAE@{>?!7;O4|u0 zSO)#Hn*3AWmj`y)>j_B&)D=$eF}s$_uL_DXzaO5u&^dUBaE0ZcL}ZV3>+PE6&W4Si z)hZL8C}k3VD%R%u&Hs5;E9B9k3Y}^74i&kaSo6(t2?*tT{NOA1Sw4YNv!5!)$38#v z=Fde}o{>(@AXPQVCkmFF_=#3^dgZG2<$D4nyqfPj!Ov7x%?uF5@1pk22426D>1BVQ zVQ%;3kYK6$8kB5jESnDh>_gMbbGg6#1Nu7)J>_rHrAnC$DGo`dF5jo>DTe*NXHJS- zPGh0U8MX!uTa@`MAGNV{ne`=9{zZJDS?s*9a(!h z1MquVW_X1uV?#~i1vq6*!cWDP$Ak`9h4s@@<3#U^`tx#fXS0TI{^5h^wgy zd-f~;&o}e(WBiE{Sw^1{?@t|VY(u;73$xwB1{FN?y#h~YfwvREhWyU&cY7Lj4ianc zYecS>yOjiYaI=#JEV_D7$&t({^8!%#*GYkS;Pmh|Tz0hTNWyu+b|<)R8PNc~-ioGc z;%+XqC!h5|;)m}&0MW~MVRRE}0vDpI%q#hIGfKg`djVC*}*oG+YFPUnbz%H4ZgJ`wXZ0M5x3MHgQR! z5a|?k8{9K5;v`c0k17caeL5>bcEPVL7fjLY_XCJUh89^#AIbG`s(7e^QOtkVhdQ(> zLXGZH#>;c-6S>dhe!m!BS%;59P#j#@SK0wej$Yk-w%j+qH5Q|aC3-Davw8+Pxsvn# zK@Aj7P}y=ZU3m}Al#40+GMTf&qMsA&+tK6!Vz>GRO+x$x`dN$Nk7FKfTcLipc@zs8 zYmlM6+1I)1h!Y>?vUkBU8?>L+=+r$Z8=2Vk*mb++)Svk?xt7K67R8OGf4J>)+SW8Z zHq|DeFD`YmwI{oK&L>oc%TL5x>Y;4HlJ$L@RJvzaxYFm9K*S+}HfX5Ia)kh*z{LJ= zhY=nEIjvCNTaYvh>v$MGYQH#(FrdVC=Rim>T0wR~i%__|eaEA1TqbjLyEeY-mzgUK zq`FRBeJFkO;OWl}WmmqLaePpqDGK${6YBcv?YEWRvFhY5&MdsZgI53;FsbbP@D)~Aqt^@?~EZ9#%o}x`*3@O0UQ48 zb@{?8trN=w)`r}8^EFs|3%K=AitS~%JPsVkcTPT|+GAMnJ4pKesB+%*#YGZ%8VL_< z&CDjH1TBPdOV=$-! z6t_0H{@#qa;$3H;Xe|Y7BHzI6+e*h8`49Ex!@JMy)L2%Wv#A{qWc7p18yr`g8#r+87f2?vQZ@6);l ztf=LqRorn#w(UR+&Z5O*G};n_;8dcsDg7)JVTyeWJLBZ!lA{I7tcw^+v@LsTF|$yD z=zb+5m@V=wxmp%RE}*ybpRDEDH=A?^|=3C4GY#{Fi!UesC?Nc z)p_i!cI`-vA75&M8~TgWL{aRWO@${3K+Km9pclUpy|IG_kM;)xJSI{fXy@|??V1aK zNPq`SsI9@7ha4%G?#vIXvO|oexz5BucOQ<6YfhMQ36k4j$3nl2W>V`w&HFCcmtl5< zs1Mz|0j4HG7v%@uF_{R+MYk6-ff2KpWtY!mI>Q-oc@rY4xCb2L{Q9%JPjo_?9KhWC z%tv=#p=a?_jb4J=)*5AiW-kVsZRzS5PgEX>eL%)KD|L%vy1?Dyj0uJC1IQr&4>@4Pl&Sn>$h$%V<-lVjl;HL~F~%|v(H~g~-;svq0n45K!0+GHMEP^X-V(-yM!v?!7Xt#}^?x}` zyz2u^{4fxy4S*@;(X0)6|Nc&6@QYI-MbzXE)GNGAc;uAua3%)R<^n9WkS+e_k4>IK zsRTUf((A?7mpsGHD{e=Him7usUVUZw$&|%elhPlxV@I ze|~ysao+U1J`yy<70S5p)=Q`2c;;VhXu7;eEsoQB_|RC0WV(*ryuODkq3IqNp84m4 z{}hLS@8e>xvj4rh7Bk~3x^vPyPXO--_DRNDHRL8}I<(Yh(Z6uoV~b7HVN4kK?LOdsr2= z=fL1cZpzkDHl*k!qtpA6%!@OZ<-02k->QNtc`Qe@AaaWPZ@(G(V${Fa(g+AAfyCSr z>IJ$46mmm>@i<$-H{z(N@05kClEvgLQRcDU8wwSLK#Sv19!idgYF@mDmx$AjXP_}0f|Lvv-R$*S};>Qx%%`RHL0oC37uCa7c-=9KFO&-`K6j<(+up`Rb z#_KvVl46FjmR$w;yWF}10f7<&fs*vyVaMg)&PhaJw1-?CQzu}mjg39kC@OQ!dV4mo zg)#;M0&U7X?C~wS2|PQ5%<+>a`KzfO<#WpzHR!qA-kh8DvRvEsp5wJ~$detRSATC2 zH^e!n8hNjcxFxF{0k}CTL#mi$iLjMNo`Gs$IbfRAf6$DCRNK&Sal~sC#%B8XxX!j{ z0`nlKY|1JxF`Dg(U~;)j7E~|nK}!9REXH48R#*@@0w6URHPY9rj~DcX3qY1cZu0V$ z2J3tP$uqkDyPEV3xoG$|1;WkO!1Nj(FHdBIcq;DVmnup`X+097@O{+Ctrsaa-HEZj zgpZu-{Ll9?bAsfZZ^@A>tDm9QF@C#v?w1i-PXlJ1yCv-MA_pKPEXYuew6N zK-Sn312Oy9E^p?mCKtktKh2AZ_b>ZQkJhVrx6UYJ$1AssokMBf1}`|0;!I8UlO78U zB=*dAfyuxf;61A7aIm>I)&R3TjE#4_7C z_VP<_xs53p?4iVN6n&HYx-K(Doz12IjQNe|AfyE3%Bkaoph?f^G@M5I*u1Vd_Lt>rKiZm*8qv%`Rcq{sANO?Uu9 ziw_BA>%9O6M&Ik+eL-DWgeeq6F4~LUg+hgX&;yT-!x53eOymRoV3nGvxvUPriBHrU zAn5K2pe>E6rip%yE{yLf@l~Y;@JFb@Q*cD^RaKljGIbHXv-A{Uq%*xC6!aAD16Wl= zC)yG6XCyHKn^r}nAe`H>f*1^L6Xc?V_k8D`ObQUv?;T9zf#1fsEG|TbKo5A314{XaM9q^X3wAgqB|bpa9&iq?f^yidQ#rW#eeH>ZZlU z{1P5bY)TYRNaREx)CJGAyhSM3)@ShY{PZNk3jrw+a<{3NM)#i{_E21$A&cF%$s+!E zoAH&iA}}<>^l*mhGb|}#EXkk!yjbVp`cF}5@k=1Iw@!@c6^{|!ao`Mb4n|pIi*`6x zDR~LQZw!G97-N|2kWLXdtVGWCc@|5DH>QUCK zIzO3d`f_7F5tHMafg#(@S%Hi}xX!bl9?^ApJFdE`1cj;Ws zw{bbudv8ZIaFYj7kM_wE6Q-2^o;1$F+W0yd;p$cba{Oe#@IZP_R5Wf;JE&9GVEv## zWn(2hq3hCvrz3&$(E#1eU52H0`N2%Bk%&l6<*}7=UkthqpVt0#d=gFH(w)Nsd)p#7 z!646NX)zwPT9Yt!=0RuZeSAfF3Q3fFg771VLBf#$w5zI`|)=Vxi?) zRvcPH(b0Y#{L^g)dUFO7VRJ5HqQVK~NxY0{T^>zE;rHwOmwJ$_eD7U`)3$_u@6WW` zZ#;iG4ubr?ngw8cZD!6+D?M283Z31MEY;g@-Sv}s*qyBKtkTf3O+ASyKUm{u@=8cB8-`cin&GlfG z&-vKlMKg@t98Vj*RsrdkT|rB>(9|eD_cMvkl?^=XFrB42RIaMw@{ssNIn@2_J~Kc3fMrHo%C; zi?n`$KQv0La58@bE(LOSe3-=yu?P)eDWINMcgvCl$_m`m47yRm?Zt^k3s~t_GbNFD zTPAg>!Sp;qd$3ti#iN7iU4d9TZ(egyQD^YJn-B$~pf`HZrO_m!h<$Dt>!zcmqqXYv zUq{EyvSxYgUxHtuB#Di2(ZwPU2&T9-af^)ZxT@!VV+!hAU?ikKEwvX`A-g~ly!O)O zeVazeX&eyV6|dt%I$_-Z8!f|#LZ{L=NAp3%w>=pMv^8wVF+$O(vaxU*vo_2nyL`u! zW_gr$z6wC)$o$RELUoXyo6TxUr&JptsT-0W$vhF~^NI#3KCh|$Fu~OGKKPNg{NjU` zw4@mV@O#&w{yp$gWc3B1Y<(lVN-jV2x2D3Vh-cVIbDb^l@dKj@higgO{x{LvB#;}N zn2A0tvdpKzc#k=DgR5dyo{Dd&8MmP9u*nsSvjmvy)C1qW<(s91aH}>=`KYCCN6QBH-kgS^VkI8_fBzWxfspT#Hko$-yEcoGk4iH=p#e#uylO! z<;EnU31RDicxbbTGEh2bJO2lEey^UD0?!~dBK9rd3Vd~MA_B#M&6Ft$N227gO0ZHN z*h0e;odC$J9sX{kf1)yRzG-^7Z3DoG`FcUcCIQ=g$gQocgPWrc;p*X=%A?!J=}6(p4tVyP96%F*sXW_S%te5GEkKrh?Bj&S+qi@GZc!Ycsw1GG z`b8xw76}eTA87UqRFvyjG-zi?`cQ8Ze0Wl;4_Nu>XYgIY>w+DUWI2u!M#uuU&@6s1 zp02P8ig~&f4lh*SfkOa$+&+KLynAG1SmC%dR z7wX@;ubEfZRNoW9a3&+d=(C+++kTP=rqUQ7sO-YMFMDHn8?E0;ZvM$O)!2!gdKE0O z!Sf0{Rk8HMBFxF|dUr=_)$g?BMz;Ys91{$eR`V^;j)@1(p(t-&LnFeF$8XE0?@ZD+ z^iHrbKJvL&JzmyCjGnNuI0!3|TB&qjM7kvQFt^BB20lT2(GoMCMvd<;%CpE86#+h# zKOFJ8j5Ro!xS{zZ`@S&sTIXq4pp5D-5Z9jhFa8s6mG3yB_WVB>-XcBJZA|@@DaBs} z|3QTQTapDs)j_>}YDoRx;NDlP=<#1L0OGXhio64ti@Ge3(asn zRc+afs4WV_!4a6mtOp&k*8Al;oTFpZxXJtS9j0HywgrHkS684~45JR&#I{0tnt0>M z$ezQ)kStz}aA3QfBH7RePSk@)N|a>hmVUAL*;mG%ukB6zZ>(vm*#{HL#Q#E!j?Jdp zIG$)R+d30_PO`s3V%>33LsCJ9`69fE8A88)WJTg`wG#p(GylSh&A)WoD)9e$978H8 zv->hwLfYJGWv3|cXrVsV@q3$x_6hirGTu-f)lM0^L+2v^+wTHy8nV7qOnv@b zp4c3(P*Tnz{NTA12erFQSbTobx@3DUuVl#v!k9ADxU=+kU|vj!^nJSR;U$CfW1&?K z;|@rs8tw4Diuh@GP9=iGpFNBO+?9;B+1MeUo-^VrgaMu)Q3tIqc00&%e(S(R5Tyf_&iM-5K=srDyQ?6m1T-P!4wJz<*T*o7NEJOL#AfusI5pd%aa>1h5oST@5-iaytN-vs%p(Suj;?j@_#-fX zyYXwSm<76wpz2TXpKwOKwP#67${r_?xr^d0iwkFZMm@d+hYmyrS*GH5deNV)wPsQR zaLRFmjhL2My`Ou6FESBj%6^5h1PmbSGt)|&Lp(jBFI42_4hxfp?u_uq3^TD1){~RU z7?!Xlc2`=s71SXAZBcJrWHlW`gM$+ ze>;5F>5_}WAFGfpZC@5ea>ef-nPnyA{ZuCWg!ZEm%|~^?JteBj5Th-Vx8i-xuxBYi zaE@T5k@GP4RN!fGb*>8IgOD_lUsY2*q;Tieq%NofMGiAYrbPsZb)zJy-0`nCX&7vj zLdFZ0Mq*ZB|6Jo1Ww#?DyPNa*NRp0&a(4Avuz5fM)89c8M*BKwN;+X?>Lljr;fnM! z>lMGpYc4`*#lNLb_~!L#yp-#8Z}IFfCU=Z6{2gDI9bBsnPTQ!(NqugB$A;+W~i`yA>U;3HjdK3lB(0NwN|_|)134E$v|Vl zK(VNkO}ss)1B;f%Vh^BDw;$`+jh=p_=iZFDAoDMxoC}o`aRCJIN8xP`V$sYc1X{Bb zsH4a??o8PKGYe>wd|m&CcH_eTYF5K*lAgeZWIuc4HCw0!DA}Jiw{=dL{pZgBsDFXa zqaSy%t#(qg>8J|jgliEFOo8?caMZXGUA7;!oR;JZ3+zrNuu(2JE!VqPJ4vU@S zOA=|do34^2^FA%BogF03Ia|-Y9$XK;hukY{HFN*!>M4$+pFtf}+fiXQipM~jR6^?jila1JNqJ4aGj9$&SGgXvINP&LVa zpKEp$%xWi3qM-i~7?bO%rP^C!GplTLu z&Pxiua?@McWEu#4Y7lBrUh(NRBJ1{}&6_@VK1C*KJ$K9`{D!Ko*?OJ7b%ITp$IPyT z4k@PIcow~A{^Ye|LFH_7apCbu9EkzzY}g~QXx8_1>QU=PgI!w+c8cE?Gl*xj-}>5@ zuq12#@|1s^`BeWCBB6T~svxrSdc0OL?-I#FV#_Y|HWnK*BriEwFS8R8$TGr`V9kNw z&z6wgb#QRmL3CFywdic+pGuR`-!FcnNvWbwBombc84xh^^v+Z}|eK>O5 z&7?OVAV(I(c>o@chuQ+|vWg{;1FIxbi48Br zZwdetzj~s0@=ivkqPt2osbzJdGX2UDC=2N4jnx~FB;|-Dfd<&ep6r5K{S-r5!(HqD z4*(}Y*uIW!o9Xvrqu+)tth+v*^9HV1E9#@xlx8Y{9b5t#g(VxwpuO?_A{%l22YNY2 zzkf-JYl8@qey0-Hu_eGskJTAF@8Ax0)0g(J_N|-derLVD8=g0CZf5Jdx|Dt@ft^DF ztlUU1j$Wu3WyVh4rKK%jC!+_DuR=qYK z+`wBYZEGmFabAPGxMGc}!^%=psRVWw2?V{-Jo^)sOV)0(CF`>WZf6lf+Rju0k^ly76Mx34 zMb_e#C6zFf$|{sY&KtOL>6C+CU%0tulyK7!cqoC*dPPZPP@Z^8l2@{47@%ybllWu7 zO&Vp4XiXO6@M2$^Qw8z-_6`xg}B#jD&`#jRNeRdJ&~$6JM1 z?4c@d>2zZxs@=wER?Dmbu(@*a{P=wm$<5`*%fxZx_i>!0`||p1%lZL+Hfq58`)D4e z2a=GA%{Bjfm@)ifcrldtZQj7ep{3jx@X1kVR-B11y2fzBgfF@}jp3HUM#(tk(5(SC zywn@&jb82zPt~uRYcCtFq(r$@Cs#X24ehE%xE$PSg#{>cASRL6=CInzAS;G=JEGo> zL!wAsYlU#Ajt>f1Xq^=AYRQ{+kfOY( zRS_hTX($t5T=jP(tcB>UQTdP`FUh$ga5ktflJWyB@>(}hpNW^yZ%Cmvv;}LWD}1h% z;OQIPS_=_xp@)_J*UTdRp~ENS(NFL|zpm3mE7d_hbV5sYlbFCDsfefSkc=J*z5_1c zV)s#zcy5eHwvHGix-EnkK36MH6_7V@WfqK+VIDPW2B}T9VMEP7R}>wYR%aJ!XPj)?aXtwNq;4pcyciV)kFi3rY5x)SW^&0IFehQ?sk7yi}AF5wY~NOxurD7$OY@A zbwmqt?Hb#G9h%xoX%v(h2{0lJwH1mt4o7z_Raul{z$2eec-Cz$1-h@zU2i=*HnECk zja|tW;zs2gNuh4?6QRvCZc*Dc(N3x}2?jHHIDc=wI)bj!mnynO22*D9a~2>o)hiTEc#B#YWp*H0Ue%4!GcBlr{*brTQWy za7b}_WWTw1gY|6FSn|g?!7ycPFiy>(-or#jlTZEU-W&|2|~0sByvZ~5|#HndAq zwJoX_+O3qgSnZrTUyaNTpuydmSqIU5bEPbuaQBtPT!-F*k%9#OB=Y!R9HEXe4SAuB zb#C9ndR4TRvW}Vr_d)Ud(NKwN z-3FJID|YI`I0UJM)leJ;KjA&1$mxh^2VCevA|z;r=R6LuY&juvx-EK)`15;x0gu-W zEohLhKlx>AG+IBHy4Wf!890rseVgVsY(NKVCL^^WdrB^%8_ANgYjMexnl)|gbOIl6 z(_FB{%h&3&&Ia^q8~x;y>Li>+imw(!TdMLWA1~Y9KsVqy0lugxB9{Zz$xVIHtrO-F zh-^)MsjI})T;mAE_KOp`**=3h+Ir28r$;TaU%t8AMs#f|&p+~@(Zk#aMaeU(4wN=9 z#vNm!Qb?_&?JAl#wpALhtAs-xdi2C_#Cs1zf-7`e$jJWr?4I`2m}U0#+_lzA1*8vs z_^>+BP<1-8ZPu%e_!5ofqR&BrP(Ky0^2Fmu0WG+ z8zI9#iTqBIV{^ff)%w7jXww#L%Q_6)mAe1=vEA&=sVnWdIg$LAtNhQ8?P{ZERNCQ# z+u6v;EA8$nmA02EAG3Oso!hUa9X_;!{pHm~wm?>hK5Q-8Z&hxz!#XyzUwyu>wQbYP z<}X@pKYn_iy}4wAT`|0aZBoO3|G_HTPlj`a+6ym~T`NH|PUvfr%T_#hmoY@YEJyHa zTj)b@Xe08Vzv%1GCf(*zbe|PUZscXNB@Ds z*Ka4;ozKk9awoV1*0d>tiD);K)SBq|#djCkt78{?9)H%LS<|MzDq>|MIeBU(Jf5RZ z?P}MnFi65Z!fYR+%?`-$l&ouFC$(ezQJ#zzZLfS*hTjC#HD@@rNBpX>TA^4 zrq5Yr54|$a^F1#sGXxQExiWwv8N}KeWDswk+67)Nl{#2*1xmcBFS;dGUp87@@*5Lx z;>;;lSF9W;D9^VSueX0)*xyb&thdcsP-!byt+&fRG1zW8t-Fn!zusm@W4y6&y^UJ7 z!Ik02IqPiX>~;2*JbKDwd`4>J^#*X^FVacP71BdL_(Xr3FQvLn3XH=oq1;$0!j>AF zLh;1v@dJ7Frw(gp=YMop`}m<1wrJ@F_x48#{;u#*XiG)o1C9B2X05eve6)`pF|dt| zReglcLj7+n(z9r*N%n^LeM>j5&y0A*05wzI8#3n3t9T}4h5~wXxo}nI$XV;`O_ihG zH+0XcR?YMJX|I=4JEv&n(pEgZrE+uBM(Uia>nP!1oI)M13wJ52O$i&tJyZDy4{B#8 z9@5(yYi_n|+0WepoJ9g0qJ94hz@riwG?UIewS&dmE+Mm<&XHOew9cA#A ztDj|`ZK`B*TK5)q+y4%;E*+ZNloS6l74 z%iW83YvMB7N9FgHXWK*?euiF_eR<4cYb%%Wh<&^TQX9To zC!0QZm2ekYD-Eh+_U~$4v?^V%!Q+Zdmt*?0va^otL2r9>vfcOGO#9qN2inxx zE9|8)iyZf)1G`ByjqTx)3uUMpTmPPItgC4K=k+-@ZNVDbbC))1Z-q5iUNa5+SH~`} zi8EK(5qov9{*n#vQN1%^v5lOt%zAWcZb$Fe&DyqZX75i~X8(PEiFMcDJ#yb})=nPU zGC3H}j#?lOGh@dHuXBeMT19V^SmGl}dN8 z?qtXQ{5_4ywPp)9+I1JMvHb>ju%r64u!9G*u^#Q4$vJFnKX_q*-Fsnwjf1B4#2X9j zckivR{S{if{ZoCcZ}-+VV{WBg_}Co#!O`7qkAWTSy3=~wr~f|F&g|aYe)+M!?&v)r zXX>w0SKDxHM6tTVz+ml87~ak9eR{g>*}t8gG@z9|p;g?~2X?YmQeMA#XPKSZx0UVH zueB{)vBBp_=l0F*)#wRgiPogKtL^sjtL!towXp#eE$#a+E%KGnzC$|N;$>^? zmS^VLR4L!qEuwZdlR};}bGe;<#2B-~nrVrr&gs>}{_l+Lt~0*<=sY`b-*$HJkoNYK z$LHE*d$%{{vh098@#aGN^_b3J9$KR8`P(*U3+SU{qQM(|4{${KmbWZK~!aUQ~Sl+DwVnI*}3-m!gV%?O+hV} zpL%nFoq2dKYul-*EtCN};q1=#p^nwRH*c*?n!Qp6xtV=xueNs2zj^5(n*OeWdeTb!+k};N(-}QI z;xu;hGQ0AL`S#@lJNPR7kU{P3{aLH*x;K{C{_UFjYP$NQh)sg(nN=Z$b`E*2LvDv~OFx`hjUSPfP5%`h4Z~N!G8Tm52SV zzH_3l;GTYCp8e^`SvFXM=bNE1uaOMIdi$q3bKa7*_SHKl**?2=uy&f9e|}v2X#B_R1R=-DHS67g(9T2m z?&9SZE?Z+ierT%gJ+Qr1bZla;3D;e5*o}FPc7Ob}8_wQzZdUl>o znz77{^c62HwnfX<*{;3X*w^p+P|iW6l*KZ;>-jmF)ZJ{VCeeRJ&a)ZwEA8v|Otm9r z+?TG}VE=k$wkw949-eC5JGHPrGVnL*9)pcT5MRhSGh6x<6m%)&%fT&$UW3OeGPA1j zofiyuvRB4Z>)>G>-7)%b##%dWU`so6cpJOvjm37-4HIlc?-q8|CkENUdvfr|x3M_Gn?(oZ4ICT^{<&^KAd2o$YtWR@fYkWA=1WX8UUVee>f3Y{AmC zK7KAcqrV;Bt+~yS;bKo_q*fXi@7_l9v6)@?o5{9PWzHJj+3cM)cG3Y|?Z`blYUv{- zT+zZlc6d))u5xSFZPaIzy(r^$_eHzf&|TWun{r0akaGUrX+5l?mJ(NdaCqx{#Lx5b95gYCH`?NZ@*}ey`vSJ8^}y^PPSCF|^ahjp=!9@a~GaiMjPe!23@U2Lw5H{sshWz=p}n3HJJ z7j{>hq%(KhrGvfv-eMcrwux_J%$UE*PCGzIZo~V^LB4r{-SYAhYomHsDK>oCo-*7ARG7(-cG803^h0~t z?iv^0QX6JT&XZ@avWqpw(AAZ~$H2X_Xr0}8b|2eAi;gE`jE@;!Vc$Qxiw%`C^WFa$ zYCWW17fYA2n$3^)Y7=3r0`dl~OpXZiHE!I*K9mRX-rO}dMWMD8@=6{Yh0##>MM6V3 zBn$*`=>9!g+r9(a*&#B7o!hmryI+}a-}u-7+k4kecKO)@?X^iOY>o`sS$ZksNA(DxO+ zj~`Ou{EVKk#0D$we&*qQ?C`xS>@!+wzo-=&2COBU3`%%s?Zt0Pt=e`~?D~W0D{PnU zt?ayG`>3O;?(w;pgdZ6__V6?oHGp|L{|XJ*c3Nd&&?m_=>(-&A9U@QgBWiC)?bosD zJW4C9Ra&JYKRiDM&b9lsw~s1pLfgI|@B7i$=Gm@NT7)1(+9wE#qLp~;-bx;*!VY-& zrz@;Ae(F*iJ!_47_=H49&Rk>fPSSpW!k<+_mEO`V{EEa>TH$hVOQYDpv63fFb!ZTb zuF4Eg>;Y}0leLPM?eM*--s)%T(cJfZ$UFGP4{WmL*zWzc@>#y#@mgrCe(i*wcGY9E z?O{1*>?z@JR&;1)-+6L@4B8s|{NHEW|LoV%dUR>!A@yZa!mRE;d0=OS**92w)vePg zcB%|nD>-2o9oNZe93qb$epoHBjlk&wlrT|+yQL(jYi0F}A_m854)*Qd#?JloM7#F( zS$5kK({04=U3}#>Tgv#7zs<07Z<#8Ga;;sgRpG&+d%vO`*W5bGj{nm{>)NTMUAb?2 zo2U8x*JtP071vF(nR4#l((32h|CwSFrY+a}Zf55yZ2In`CH95iPP32y`a^d}e|KCr zkIc9xZOI;Z#`nSJOFl$0J~E`8J*pMgAGAv9-=T@`#W9fB)?ljxqlE70Ec)Fw=kI=F zx&7&p$@ZHs?J0e;+CFo~bki1<{r-`uHc7^^i{qC1Jf(jK8NT@LV%LqgKEK39PhVw!{I~Em=TRK|Y!$In_P*Gw z`%q!aX^J2nyjur5VYk)>jqi?JVx6_ZYfh9$4*5Vi@>)W=Q-SegAD_NqK&iWY?^I9H z8D<@&*c35z9dc+Ft>m;{=wnZH)3-Q5tQ;AOJY)P<$UqJ5(?;WCo!#{B8TRPNWr{4V zu+iFP8T&z{9kO>Y_o7_sN8y zEQ#w%X%`I0|9xtZ^;GEL5qUuOK0DLHni2hop;p3W;ITqNXj8N#@2<&$xBTEsb8Ldb zjGZ)T`gLmNlWDaEe`E~JUen&=lW!49++@4=Y#FspR8UGGb(ANK$2UzL)Ppb2wfRz1 ztXeq$(^Vnf>n|AWdxwA1>h9^0b0x1$)=e{UrDEqOk}n=vVLiLG61m9u2D<5W>q?-? zbgJ?Y^$YU>nNqa z0E^nGiS5>}y)9K3|6>XZuF<^d*0H7Sb!b~xfa^48ZI0%(l%F4dQaR=}7gnc+LCvQ-##5eWU|$D9~4&Xx&|o@p(t} zvJpc%*kaYsVX!umDY|1OXY+SyuG5Vhe1+3f{Cxbd?h5^N@jk#I#h7-|DweYV)P2yd zZS50B^s+%cT4|qctvle}2xW6<28;T-j94$t_wx_ws_l}MX4BVc^{M@ReSWF%=sM|> zKb_m#`{u9jF7xq#og(3|ytiPTJ0kn_Z(~cxuW<)u^n}&+owIkb?;qLK^%Q*vJwF(? zV57~E6{K$on;-PYiHaVruw%6!__fn_QB>#n5yvbL$A(vJv&JY z9qZoIhG?}(TM&0cNRftzlSqG;#FHO(&_J2DxYC{&yWFllZ?IjgFvCSB^|vcEpzeEq zw)UNx*nbteT)t|ZjnXRW?(r+#J7iyIjF$Wuhy4`OzyJAJK1m*Xb&gMr-d$VSg!$Tw z$TEasTd%=8VrVD3<>@&Z=xY^TTxO4`-XXiRcW;6LM`CjOfY&`<+yYIeJ$1_QAKF9s zaMp4S&^7j_=jV9{l91uc?=RED-5@V(p$}kIiZ?wn%?=#a**<$xKYL)D_*aMl&zM9l zA847*J|kho7v7#{*F8DcK5=w!Z{re8>XSzF@Xhpp>a^4P?8uuUC5E?i?Q;th7U*U} zrBFuCT@Kqc;huETes3H4{Gofwdpg*^|4+G{oQd+Z>RIf2P>?D=%RE@D@=?T`_i^_ zlb>+JFthJ|_&$o=T1p+n3KAU|+putgm?g^4Wg2 z?|`II)+@e3aKosjaz>gdl-b7~eO+5Tf?>aK zzSk?9{I){D7o0M{-u-a7O_Jk-Y$F4jH7XL_qtd0P9p2ln{d5oC9{J-X{iJi+*{>B{ zV{53T68MMcT4(%0)W5SZ*P9T|GDXo>IA?ECw~{IOUy*gv3TdKNA0OGhv$blp$^NcbczX@fV|D6^&DL&OdA&GhiFMW9zzGL-w=t8K zyE5kp{>3Nv^VQMn)iO*nUYs;sA*J75he`Gr*vZ;7+Ne&SZ|`fB^rho_S-&1_wg0u+ z`s&pLUQxgpb=G9*qvy&s8+`KZxobxorcL&TTBDL)tg-Pz@)T9#g1funy6js<6r3GGKERJ3l~^XOXY&q< zlpVFukNqCKe|JAb@|?mOqr~&By;{3R%wA=sCdj=n&an@5py#4vdTRfrgH@=mdqwwU zdGsT6oEW8qmz5qesjF#{OsBC{BnuZU(Oy#<9RT1IgQl-*>c|=5o>R0>C~VVL?=WH1 z!sKVVbAoZYRtg`gkB{23lbxWkdBh%_G$)qYd3Pw>r#-k1QbY zTV=m|ZIKPqYGGF8COdCf?Em?Ly4oB$ ze{(c$ACPxHu(`sf+A~|CaN0`>lbtB1?E>MjmwoEp)9e80g!Vewci&iTAtqg=q%Mr z>D!OYw*z}Ox3kzM+NG^M`ro;B%gCkHQqJk2;^Vx1JJ~etPqRn&d$sK}?Hhhojy8JW z!RKb#lhY)B8Hkyh6GPO#J^Qt_y>yrcrO%uq=rcg`f1cX+{rl!v(@pENozcZ^eQCb! zAt#ei@Omlr^;+F(s@Vdj z)ZLEPsY#62A0+GN-(6`(>HrY?tlxfik)5K0IiEbFhn$ZMI^(d$9)EX<^-z819NAm1 zK6LT0##D(290Gq44!>T#Ni;Cb@D%Y8>Z6q?xrpxjz(vkP!Qd|p#GfhfvUl}OggPN?> z*q%}v9OoV-hlsShl-M}o4Aa4c_hl4%%YkI9%#ji7s1tGhv?_R4-hH#?T4~A9yrt8c ztP}=HiB8smi`_lcr133gCMZIpAA0u}f7v&IpJ}RdiR#`(Wv0mp4i(K0#GgVFwx8;L zTXnMMu)kJqquAt;k=|3MImhTVfg!pwVdiH?|jDj!hHK2h$F8`)ZTK z!#;ZH{vFAUw72f3D=fOSGGlw_rTDR`3%LwaTi;ik_E7#<_)@)tb>b0F&xlHX|I%h$3VdLFlIqj+p$6D5DrUh3!fM7xvv zxUa^_yK38BYU5<_y^8)+`Gt}{<80q9%{&5tBhyjzbokfh#e0o~iS~7pjQ^`wL*DxK zF#Xtamq=HM&s?Hd?6D}k7+$VG)tfCha9KKZfNxV~nTy&N!4Nk_j3qdU2_M?8TkkyJ zC~GDtDZ+`A1JB?)X??ko6E2wCQ!ZYARoaw>ULKd{d^w&X8W;xJ5O70Cl6X9I#8bS! z&@S2->Xu#Ais(&-tL7c+SzFoN&B+99Q>V zFwS^-U#Aja9F%R~WuezBbwy-`=-@^Urhc+ZM^JZs*?LVrd%>`g^4GGp9C%9TFGlrc&Yy1!%P z20<`hbVh+P#3|E82;lO*U1#xx0!GONzM%m<_}0aZ*kUo#PUU!pFWS$$WkPWU5C70W z*#qOApAlQ8a_~djR;V0&sF+pitD`cg8+;7qLgC6D_$rx~k!hPytF$+oU6CY2Hizr) z2?`l}`h}Oj=@W2P>YnLShW?|jAW!By?PH&dwg=hL3h)9QEp!v7ChA(_>K1kyz>h> z@P<)>2kItq&q_@H^{%7#7nKhWtZzW!=g7beJa{2}fm77LfgF(!=yiH!so|;cH!TBv zb64-`7=!56z%h9YI23(IvD+M6i>ur{c*XD|g9bum6l=nb*At9G)pB0i2PcCjB(KAn z&esw0yac$}9*ma@BpB?h3_}WVp24TBaXtc^D|yZl~Am8+@Uj4SIkVbELLR=Uy9lda4K`)|2LqhSBz(c`sRkY0&CUBig@mCHQca$SP zjtdS79vI3EMH^2zM}WvDhbwi4XY#0zx=9!clsWk*I$kvAUXHxuL|9Rq1oi|y;E-T( zqXnne0gSS=pTvFm&Bk!=8Tz7aD1B0Re=qRCkJJMr8%orHqDF}anME~3#**r2M}!^K z%M)z@4nKhwxU`L*$0ET)sFxK~oId4g8wMF3kPr9Jh?m2#1b&6?JUelHQ7`p#TqmkA zx`uH;f8n5n6lBYDz~?@Ug%0wVphsSz5rTJk3Hg+RFL zDLh9{(RT!+Wm`Yx44mHm%lC9Se86`NWlL&SNl00y%E67dqnvW(;D$PCkU>E4YdN%w z;Kq$+j&>2;Tz=tAsr(j}3(raA%jF@^+R?7aCYyp2!X%MylH*cU3B*N)@+wR~0u)?- z)>j-BlnZ4&U)LNiz-&U{lS0`nIClx~P&VLFjJ!CnJO_No%RU5HF-XvJL&8-U>rPf*7x@ud#R6ZLbUVd(iy97Z1%HsG* zTDiWw2g*t)wzzMIElYVyB~a%QU?Ip~%V18h-BM$UPbW}oT6vX#n>TQAb8!sPl$cw|v!G%U)`9QbuS`*#+z%Aq})K6XU_VQdRZlDn_A7HhiTN4lQy1*-z z3fmCy3gLm@B)SQ>fY^jpthc*Y^Xn6Lz#Ge zg?WJn@K9eU7q0O#;l2Z}o#7gA;@3Dn@M5@ycmbDd;GcU^z~w&h z5$?IhQxa|=y(IiVH;!A#d;B@n6YxV`fKx{@yeIM`9>vBsxcvHQZx<9Os1KF48&dEq%+1MTn}u5lUxr|?=6ZpaJQ zc-a68*N|8E3_J-|3|Tq=oi}iWX1D(8Oev=l*l8pXUU_kk*T3>jcLT_y;EmW! z*Sk8=X)~#afQz>9{46Mo%j(e8Og5h<*E$8bY|=M!y7_#^4Ua(^W$8vdM48S~7TMAC zMYR#DRp}Pu^4bxp3T~{fC^!2Jocg(_fb$!N#gSw1EOA)Q3MY>k(Td=~(clCh{1>Kp zKR_s)^h=~G!cGHzqsBS8D!75i7;d2JWwZH#4<2cJ)TcZ|JOtcA**b27Rx)nrhe%iy z&dNKwjlP``!BugtAF_yrJcRZ}ynBH>X=gl7(sA{;wT&AbbTcPfEY0(B{Y zND)-0n1}1m|2rhoJ^?)r@_VV^EX+|7?z3ph)?M^Jq87)Ea%1r7xCQ8t%JUha=gP-% zbN7xDA;w`*q1bme(~V1no|8eYTtLqOB1C`8!2(jYcorTUjRZ6=kf)Ky$iwn9xy!?K zmqpLRvbfdY5g-fBm5Wo(_i1(9T)8-T2bPW-$rkIAr;+Tzes@2m1#YN>#iH*mG16Va;+8iBL)y5$mR2>s9yc-V3Y)?AXd2DrFV1YHc$%F`AeF$@kwDJC5hvY|byB^H6NR_43vefu!P4c+b3g2z4>Q&Kh)XLd zMFN_WP4olh{sZKt;HNnGA8g-FWx>I&wryJbDd(JlV-scKXGV zG&ML0tlzLff70kf+hge92FF`NWoXQ}3AX#tLHd#R=qITgDxanBm0Q7W@#?LVT2h=; z0z0e(v`XOb?zZ&+?eCxNy|4CTnVzHqmQ z@e&ImFaO)18g0l816N;TxPUYhr^uPgbDx)8V$ItG znCGtM>bL(jXwB_Qb5jY_vjie_P|ua5d5k$PfKi80y|>=HiUV; zRe1=1d%Dz0OU2vl=qW%bhi-l0!cy{}4!;+fXSy67@}>N)7DWpv=`oq#>UhbJyn(A7 zorro}&`9R9VM6P)BkQn<4WSy!)R{;+kkLR>tI5C*XL@(_F=aZPSG*Ge|H*Tu=NPR(liSz=n z9NbW*@LJg3ke>t>ED0Qn6uSna6s|eElKGG3LuoT0auRcOa1-S3H}OjI_qo>u(x2+8 zBD$I%kYC}5NP^W&W&6%eYHt#3`)*|k+|%fqQ&5@#D?FnZ!aaFB2NAkj))*^P9sxgG zt4{K$2J)hEDD12rpsp}z6X;Sh(Dbr7Mxs(ld<0nF6&e8+r^o#krNBeHa1tz@pKu>Z zHI@;lk-ZP^e63rz&RVo+A>`3vviD4Wf^6QT#R{ zRXo2ga^f=B0(zmHp=fxH+FfGe2Xye9>vu@qG`OjMgc)+1nmqP)6)gEK~RLd&{*pf2P3NlEbJAc6<>(-^S+7bo$ z3V9Fhd+V*Y?B_rKxsT?KeOS( zhg+*wEyWRKqwx%zt5&VDU;XM={@ap$`}Xy+u#GeFlb`&AThF`hy6bGjfg|i6|NN)5 zY}wK}ckXQY5M>PwhCrBD@hfmon<*R6lCELUsDh;ItqUu{LypH1xC!zp0d?D=l+$PB z;BMPiZJpYy4Am+RQT>uv7&99;Zm`QPyUgBw_gxz{Y?!TBvBC~N{4l%kzWW?^cNw<+ z{rmfUBN?~K)s?nx{ra5o@Lx%hGC-+p(9cdq3L;|$CiUN4>ylZ}@8pYuDuI zxg4BCzV5h#4vp$5hFh4KZI6Qs=@#N{<~cYKW8QnujyU27JOBLi?Ju|9YFAx#m3{1E zXL;%S@4w%^@r|oo{J@W2CYxqft;1RTQ}S;uwVcB*Y>~z4|qxP`0Y=I5N*a#`IyZ*UyRRayhLVa_uY55 z{{8#g;>C;YC)fQ%<9ChI!MHL0kROTbx4!i)!;v6CgVdx+6E}(&y(Mx=uD||zt6a0% zhREq_-MY2op)TcgWXPp%( zO;&H`o_nsHa>^;b3Tvz1u|yHvb=O_K!kag5u3dWR=k0_OPH@kamBjcDKCp=sC)p2w z^h2N2tSCBmjDA4)&O7h4haY~}Lu-c}c9_$~xU z27+miUeZVcA+!L2kh}*;2<@LxLf#Juc|afugkBOLkkFfHp_pbc?p^LpvMo!tx@`Tw zZ_b%>_a2>lb+0YCfY#PMdv<4LXJ_~9?9A>td&nV&Xc=F8@rB%d&)stSA8wZ}UAic< zufFn%S{;Rjg>uK8cgWROT`lLFbB-)uzFcm<{dOHpIDq{0r$3bg4>(Y6x%C$5)~%Z? zShzsetX?A*UU;Fr_10VR#v53DXPtGX zJpAw@LTit*p+cnPefQmW%h<7F<(g}*QHy3#$s+mR-~X=;#Ft%mnH+P>F*+9Txr>v7 zQ;WDL`HJE<4rp6%wvVvYp`@fl{tC|oEhyIeCzo6zd2sFi_<{=rPC?ld^0@#02h@@} z?zrRRkAM85ELpM?<71|rd+xb%^wCGl;*!OZlbtPH(XSYCb8}?n%9V1%4cDu(ql|y? zi(kl)Aw%TQ2@`}CC$`zgX(2JyG&01=IL_40SCz1wQHpSii7RQ z*)L&ky74Ah8du7tmtKm2WQn@iIPfsM{PN3i)pSthX3yLD_3PCox6eL%%gw*LS3&HsLUxWKijS z_q*SPe3q)q@-7Un7hoVi_0&^!Qo(kcIdi65am5v~YSn7lZ`^*`4s4HOk3Cjs;W3!( zTa-z-e<~#4aBj=xYzIrFaC{z0MQhYY$Ga0+*Sh)wTdnztkWwh9+V-_x4 zDC}#8AMp)2^UO1~?_YiGRhn)zc>VO#Ph-|}WF7oOHV7Xgnv3+8KJ)A|^6!8DTiqiz zYoytkolUF7b{cuR#K>$^gEHMdmoH)?Xr9o3#VQJF= zpMMN-d~1sTgKW#=ng`)Kz)YunR-waKxyVoig8}(ah?T_*(h2Ai7ytQChv^!BE-_1U zuWo#+)xQ}JE|srCJT(Di`iG)YBBt!P_P});hLL-Wl$&n435x7z@{fP~LkC+bv1&|~ zegzlVh!GMFY(8Iea^Y(FiW33jo^nC z;`s3spsbG2Bu-xGYU|UdkKF&j18U(788TFHIMF)$M?VrO#J%_4M=rsPeZT$oQ$A@) zE?%-&rovJzg*C)yVEBmP!r3+5tsHc$?^xM8_8L868($ocF1qL?Q_$B798t?Pjp#2L75w|H@h#wTOICk)-v^XARdG#iBVrnP^f ztGG?2BqGH&q!+Im6Q0K(e;oayQ9k+P6RqQuPd=$-U3=}dP+XJbv(G-$`+fG=NAqdn za-v4rjvF^lc*;ZXKD`20B7rHN3FwcVVU6AYxBF%O{Q2rNAndH!v*fHZ&yr!ohRM0- zoU4efhtv(C%t#Ss+Fra^bEH-8b^j$cXW%#_hc^tr*mX(!_ znZvts#&^%q%2Dg{q0C0v7LxHoc;m(3?E#?jV>#AA-SWLUXDSM3adgXcfWjot4CH}< za4w$ZlXuE2Gb1J!%A0sxui@f#-@bjde50G?%h&3z1>lFiYj!|Or@2nqTOi%stX#DW zXdxI~DtO|!!P0mouT~GXnOhIy@y}%A(rAx+66k>$Ze3liPQGY4ak6;Isi&y?IeDZ! zIZ@;~5G@0`z!;x3Yqs>mb3x0BVUxec!(D#tH&kvMFzD`?fpsJ*%o}dFLHin&6lc&^ z{Lkf5hzEY}z4ulc)=s>di?hDYThrh(KqX5BeDlpWOV=*NP@2!_^ThVIA%cCGWib$k z3;JC1;2?DMHCM~A|8=a)nLS5psxkTX`@b$7D8^3y@hbuGA$VgFUBhe>4$5?g_3YV8 zUVr^{S-*ar_HCPxJp1gkfg5+tnl*CEEw^Z8YSDHKlm#ax45yuT8suIiTxX&MxH|FfFOCC9Lr0N069}-bqhCa= ztWy%14;2SH{;{*?=H#l6|3J7*Cc)Gsj5jDYse5HA%yL>}S$KFk05Xr32(>b`J|A>G z7&cgrMn~wFm#+^l)0Sdq8BI-^rA8Z>TJ{PEiw`yje^V5Mr%PvH$wIyI0p!B?%o#IP z+tX@^;|0Q=^K((&LXWlU!er9E?~8t~Kyspbwd{-~-RyR?J_BE6Xw^ zwHhvqjpdL7;8>R_pG{F%YY?W_VSu4>WSv&SVxz1%J72PRiMp}qlH$X|4nTNXb#!lW zU?Lssc;sXA<|@5cy1?NigljPC*R5B&oTV?tfLR7B)9BI#Wo^xBv%LsMy5xhj2+NMe zw&6s8ymDqrOOE-3Z4b>Q!2VeVx%~*%Pfu6@oK#HyaI)NX%Wc}1C`Yy{``{5r93ij2 z@w&F}!3Q4%1yrQ{k_-8_+;XcNb<|Pn`r<71Rp5i?tcv(7q8 z-g);ObtQ65!ukpGIC*;U`R5r8T=VQdgj=v+fx0BQiQxVB-_u3@SKwhG4g=eN@#4iA zZ)|8(FAUvOPdxF2x|Cjg;YIoJg+Eqr0sANWQ=lCcYe)aEZ^rjeuod*#6Hho%t|^-zpZ~1x**ovJ6FY?O z4HP})M7QDzC!C-P@UX)VS9nf-7Qs5EhwGVV{;A_5`x-qF>{E<0P(HN2O0b5>wYEF& zxlP4k~hjk8NWAk@s6 z7#vSN`D9f5S)r?h0($ACmz35XBlnQCrE66nT(suE|K^)-!39_bvv&*z-8H)A#k^TF zXUQwCyetP{20_;eT?hY!Fb578s2yzDm(ygzp%Y{jTn-e-^LV&9yP%MkEn5m#3Lb9U zAAkJup0Mx)bnqWeHryF-1HFUU2L~c{SayCYwZg(8%(f4QGX4y+zSq>}&RsoJ;*`O9 z2%A2B4nPd%lcu6zy{QbJ!?sqYlP+iCtlRM6!*syng8nhb9;2=-x;#2}?yO3U_?&@r z#zfh2pj(L!^C4U#XPj|{ltLlB2szTN!g@}hJ_DN*_R|5D+tsP0*%8_KsHnIQY}Ofh z;$X>55iesiLOzr~72L#$M*^k}v#|fDMZosv#DZ|-i=E6Aw^nv{|KT%kW=p6pd=9AC zd6;Hz-SANur$CrbS+H$rEmJm3WB(sKc(Aq!NzTH=CI|hRGgLnCoYZl4d)Q%zYx~kV zqjIFHhl+)LiBTJXGIZEbOkh6O$=2R`@1yHwe1@n0bURTVQ>RYFMwEpz3gwyAXtD^2P;ZlnI0o(mr3C9^S(%(@&Rw`z_Sth( zjGhxSIIf+OcP=l`iX-nFZ#dE8E=zi-XpM1V%XXlw$}w>rI&_GxZE}LjNjuwzZOx4~ zoE&n!m1~+-j#KnbibhBgN%kHz`J90RPe+d&hRLiqabD@`ez*yptO<+M_ltnt&fDdDQJWy(R6S4@LB+ATQL1cDr=c zjdf+1GjCB-d3lA#p+NL%KcQPo)+^h?4Pjj7u`eHV&_PY?*KUY{b@Lf#|Hp{Z1lL>8 zUWR8^!ntwYw-j7>H=YnLUL*;@T06sdqYS6boQsMkF2P{K5O|d$oU~2%iRf<=v9>^R zs!3icFHB&xq&VIkIA*dBGv9@A684&+^ z_wJ=v3|czlHghJ(bp?RydIV>($6$8)7B-nMaQ25m1G~89VDbAnO{=Y~(^|1p_TJ(%_7gRtRUZ<#@@H?*P%=!pj(qOWtPPm2WN}qxgP5+7?^Z16++gT z2rVgEJT%GaM&!(w@8QszeGb+nXCbsEX>D;GhUKx&q{&c+*%4h>qego(QNC|N3)2hA zsNxjl7w9(Gp1pc1ZQ^qVMt?PT9x-rc$$DF0emHpReT`~a(h6tb8VfB=wrxcPT)x=y zOBr*<#F=gZEDA0>E4}6D%VBL-NXLDzwv zRL)T8&0xQ0e_%f0C|_>C(!ObUL_#{@_)5=7C!M4&Wj=qzVY;=mhtG07SeG_=;k^Zq z9`Ol}XTi$DozW#MT7^qkNaHCYD1x%0~)<67$!sR(UP#(H*%Nhw+UC;LA$@&pS z;aEQ(*JNYzw0qVaBfGW-?1@YW)1YXF30lZd7d$8nc+)_VD{)_1Lcm@G_Y zytlx$HTDflGx-@klQ-eRuttM8{4-u$8l-Fafr2v=)d=tkd<{(&@ov2|5a9sJfs)FJ zYix95eTFqAC{9$~^Sv{VuD9hF1icD)t{oELJ2HiEZ82oxq4Wmb40T-MV(u-In_B0f#PF2Daw5U{@VeJtdubVXstLAh2G%jwtgXmntQDe( zqEDdx6NR810N}JQi;?q^bFQF3$BT$*m&=)16%#;Xf!aThjFa2cUW z5IGq5kjHEr%%LAuab)FW!S`RHGfnzCIaqVeU<+oKoDosEZGZ*ELB9rXn0v~`+fm58 z8sDN}S;XOtl`|)Py@5<}@Z{{376&(wP=Qur@IDng(D-pv(x3)6D@O&ZOgHIbukwO? zmP=Ur$Qh>+;mm-6J1e;s$g*m|BLhj;Gwwmz*ch7ydlK;mhX9zp*hjFsvL5eZp`X?x z*9IvUK1|#s!g9D4M!9i4jtZU|IrwFOi!Z)d<)I9bC;kJ1H6HIRP8l%JQl(`{<;KlC z44iB#Yk(;0%=k)Gf?i_p-FmFl9Z>_gZXR=uC;KHm-1qVA`JcQlrM^k z93+Ul(?gpL8WDuq2Q1l*M+u>Ye3QSnwF~RU!*o%$;W5kliGmqlh7-QG^p>v4$`9j) zVUv~V!X|@b87aalnU-MP{4oFF`__O3Q_ZTE4qf9Z9UK?7S>Z-7;@T2yI&@vwh@#@! zG#%I|T1fN!3d~du7Y)V1l-qqdh|%JqWyLN1ix)4}O$c<`aqy!8EtqX#D*y!qG;OLuu^3M!NcvQ4+@N7w zbNRjm-9dDlaiHhcR1U&~CH;r+{XWv<;w5*lad0+XxTzvGC_u7t!leWdpbY#N?=%p{ z@HFnE+rYw2I^5jCK;=fI#`3yg8#Wh4r%dtQ8KHuwyOj#h6bu!K;z15p4(~ak;06x{ zx=IsEsZ&;z1?ymw0$O$ChqUc`id?uQF5OP5kWn@R-K?L& zjmvhX#Y_3|-9x^x$$NEUwe&D3rQQ5TJA>6B?#4@WW#GmQ#l1GmkEh2zWc{f5i0BfE zj~kxh2K_dkd^}P*cglB)#99;9m6sb&hxaJhbZG4guvQ13my}vUVN-%M>Zx7Rq1zIy zUmq!3GH3fy8y3g$ox2A17rgETaa4Ff9ZQ+}xST9XC{>40~Zm4hLes zi^83Q9Arze!*b%piK+kx4<4*L!X9|wL98|XK)(I$Z|e)QWALJ75e9J%+Q%Mqth#Lu zJ@ioBUP_Cuf4~0vl4`%c{iL9i_YR8=3amTovB{;T&v`IoVU}Y@*JqC5Sd4%G!L$`&Ddzc z@A1)X#{r*f1+-XbP08K;uus}Q=K8oYG^9V!&ok!e)0bS?68O`J4z@77%gV=FfMIM*}y z9Xwi7lr?!TfZVx|PwUC#MC*0{>c>qgM*Oy?~uk-ZbVCa}v^U?(v~xy_h^*91ny zzb3Gg*xBx4V+sxxOBSeH!hth<26iL{y0bE~IPk_ipL`LNioiOYONTq|xLJcQWHX(+ z3aK0zIFMS{!g8De*3!u`X%Lop{4+dLs>T~}m`24wUZ^~{Q_zAcP*_O3hG$yj%@i!l zupMQHC=Jr%pTUu)8)u&J#XOIO2TvK2C;nMb9#A*Bl}LwmAP(2cxc10EnoMWD1&>F5 z$=u_SvQ1Qfqf4AHe@r7kEYq!rmP;8!AzEOYbN$pU$MU|)SCbjvgX@L&;JWwK_jqUncL;k(a@Hv|E;lL9`@=7!Nic#k5Fsj^k3O9NfU7sJPs~ zdv9>{Ha)NqgRugQ4l(KSQ#TmkXvHWz;tU4IxDEcyC%ko1gEz&+d&YSsjt$=E^ak3pp{*=&*VwJEpt! z(7I@wp)S<3uDM;q%8%qz8EnY!uf<-&oNabLlc65gjYrMW==C>>78ozJ77Yq8eU`JXp>2CM11`c6aH(ddKW? zhu5Nop(&CdPJ-!u;JUv4`Dcwq0`GE(>C`wK*yxG|6fnThBU6Ldtg5&igN=mr;&7}= zHv}}(MQjClwmmo3sh0{jeA`?rmR3E|slitjHl2E<({Sm~{VKzzBeS-EO(I)9M07hZ zn7cqWl$PmwD|hgwKNPeDAV_3gl2&2&#Ao-8C0TQf9+wQDl~Cm|IWf)kE`ZNa#@ z+F`{-1s9h2$;t@BM$xmhFrDzd!SQN&;j0T{af3;8Z|S^-VUy( zQuyeQELObjVJftqM^dMO`DsY4j_(IH< z+3Tcf^JB7gnV0re`|Yr2AK5x@k&b#wmHrC8fvLA z;=!^`px_dIHYP@=J!TSMn^2f;h)Q$wQ;DZTBURW`;Zotxg^8jYmFD8?%y=vF7LdbE zRffzne<_9MoUenwSIuw-@ltpGZ@_sAtN;!hG@9@|)T*1V!-q&$<33y`_a1Tn_sP?x z*=+rm;LKM*{}BoXS`XjP^)fLiDL65FTve&gIx@rZ|!q~lm9pMKRiAoDHQ;eiTBURW$aOt#J6l@e-rbWS~Sp7Byi{iI6 z3dEDjDtEy9gE^x{w+6*siUy8{kSbq^2aGwg)F^&}BtH_T-26V#43-2#17BSKap@{7 zz}5QkIRV`OJq%0IKA4ZHcgB~S=Vds+hz9vB&?T0_`UnADkikeSB!e|YV>FV%8buqo zO(__z<;N<5ltl7shUCE#H&wbpIf@gM?nPYsNqInNeqT|(tux+%{{8c3Z)=RPL^t&j-)pP2*4m40PXd)MXmGVK_vk zN2R%VE#(t76&RPMixY{9=`r1qP596N-w7d15TYs;WJW~dkR$Tlq?Taad^hgWuy{-P z2AA%2SEB3llG2c;uqi#b#YwV{G2QE9I1TFO@zT!a+kM%S(Z z8vQ>E{_aEn&P&@Jf16cs2Uo)FFfA3m4X~+KFLg+=nI3dY72B5l?c*S}cQ)LE)1jLV z?6%ag9jEmr(B5X-E>pv7a>A*IV#qN@jP9@t4wyk894!>=|!*)GPZNX6M()Y16 zd3&`|B-Kh_^!Rb$WZ=e#@H6ENs$OC4S8bSX3;kX$>BQ>b)f5n+Dtko%*U3;k{y4Dw zG?T|ya~f7&v#VE)X34unf>U>wUr+wtN6alAU%bMC-)d& zqH+?okKrf6i&5T=SOR%PQ6xXcljXa-F>WyKmy;~b#W7f>xipf|=T}(Rw_W&{f@7zR zs0@J@Em?-I%UA2CPr?NlQ|;Gw3GB)eU_a)m7kzv8z_COKEf15DD<`c`g0Cp-aQ?qtZ0be~rb3izzr4%ZU?sK;Pc_5C+c_<=Im3pBZh}RA5t~ zNjkdq<>DMD4M=bfu2ehc%HrF)D zn)NkO+&M?O7v*XRU{`ADw@7Yw2OK_2{=p6|q)}DZsOK?eW_FOmd>%gQ(IbGtGytw+ zj=u^4+-lIO$49+M*XmUS+&X;4I~(6D%>pe8(bbMlH5|P=PL}s)zAI&$OQn0Ko^tq}N67fS4wX!Nc-7!ybioZ3B816fx$-;at4w*% zVErq&hroGSisdn%cy`JNKYq^R!U*r-#3n_A<6+45k#@6wn4X)Hi`Nx-Ffl&2=$ETy zrz9+8NBdPe??GFg?5f}NMSbf0?|yoWcoR2%E{k>JbH_UHI5i&W#iPqi7EI$G>5+FH z>t!F}=b4y1uFGI~P%~i~^#qy@dPJIiT8wmgWEzkEG9Fq~a0JW+V{FX$;7HJ-dbJxD z+DWshU0pDZM`T1Kd1w(CniP?4HFnF30#i5wh3cCSHUvl1RAC86;AFBT2tz>ABJPy{ z4j+~kLm~h|kP(YINgj;-IvEzVtAPYpC&QwA(6wHcV8G#fq(>xq&=DCCNk%!i8jM|; z;q$|X$R5l9K2gLGX&eaYhJz35qIgAS$R;SSJN~suDr&Y$w@z8IY<->l;02$abAIB0=L1)OUl8tiJ(~ITs zp`GxdXM8dY1)XD8kskKU zLzHj0;RZd(`kZsl3Ftib*kkhcTW`zW`*??1Q|aGy(@nwysqee*KDqJ68?_dc%d4-x zsuKYohJEwRH_JWu+@ng3XN2AcIo7M5S*p-5rmwu00mJJs}0QRk`_P-0{pJ88fgz zzI{Z0O_{c!TrPQdp}cY3I4N1NNp5?2h1~y>(bBy*SL$I+{r1s$^6E#c|LfAYyEI0SlgP|YPvmgwAY z(4fIONK;YJl3BfawGQrGx^z)q%FD~u)lNhn99@P(sR!baX_jYo3y8K##A`SnmdrBN zu3f8^Yq##6HOBi3F8HyWc;bn{QDC&(GNG8d%UOX26^<+N946vUz0<|AEJjchZR1{IsW+LW!SLc^6azE%9&@LDc4?m zttwMmdpzRHEaSi@LDmxnG#DE#{Xaa8&G4eq64Bm~uxXUVZWpdsRGPv>(Tz$2%}5#v zV8i7nEQ`FRLnESYQDrHw+e>!`;`_42rBy=G=LIF`EFEPnUt1-sD;fgj#=@y^9=mk3 zvhwODYvqdL`>JwlfO5;hfx(xZGFV=Q^;O^4Bt;$Z&BqYWY|lL3^_Z8Pq4d7JPqBOe zOOWC7xf^73zkIp$q=E9vXJwj34<~6;A=1TJUE2tErROrEyOh>k%fV_Lg0*CqKvWv@ z6Q#Sj7LS6BN=rmH2`sZXaBkV!C~r=CT_zOmsmiUku?F&KfZMiK>bBI%p+$Sin_s>z z4M-!rwktFopMN{q{`%{$m$%=38>ffY$%>UL<&Hb<@M-|KKizegl$DmsGtc}}Zu`R@ zlG|^;UGt|-ohr0E=$g#U%T+^y7S2;oJ&i&BF<3+Ygk_eYZpgb~sXT-GE3dp# zzWnk_EuV%5aaXNcr8HlMb@NA9FclS*nyZ|3Qciz!-Dl;lBT6EPoZig;ZhIjw-ADKIMu1uahSsr}wA(=Y$OZjrzm-5@+ z{x-lXEG*PGkLVdPWQc66*r@WRcY?~hqM|~TJn5S9x(pMGS+iyd<$uBnC&+;Z9;h@d zpErR87wWPK*#4710#U+>N=tyAFu65oBqZL-8M_m&ta@wm+v4C516Z@ZR)+P2iyxs5 z3gut#uaY$zYNcmUj{Mg_z2UapBK^8|)O=Gio$|9VsCAHv&GnKC zRR8W6=r+`8+PkyL<;-vNmBD>G%YxFyQns;HdKBkjEvZTFeQmi6=~*W0H#f@2KKXLq z34>HL5k)Ai$e-|geo?Y@(@5Oz?HWlKi7mmp_}c^PSGcMc6OhUklAj5y7Rrv3OcMw# zxxCB*xOHK90j{{9Sh?l_#Tk+*pMCb33>`L9x_9rcRv9g^Z+zn$vT)%-9h5o1*TM4Y z(X*#??b=O#fAjBk&_DCcv*fbNE|YqEyqW_xl_iw}t-Xm8C+gpV1q;+QchX5GN#&+W z`PR3-CC40djFc=cksEKgQU3bZzY3KC%Xsj?2i2k(KYqMC^2j6d#1oIprI%jX+=^3q z`9gE$7}XX@WJ`-{{P+oS#u?vLC4c^T=g9#F9H8rJcinZT%$Yk!HdStt{DOS>tvX4 z{cNiabX0`2w5aH4#U&eyXqhprSXV29dUaIyY7^la=}J9_Q2j) zuM_#7lLyG2SZk|+1(yXF2Ai4djsFvVgyMK$W;!7bxdp_wf=y5?cT0Bd*h3a>oh_X* z3MDx32z*fSG{9w9+_5Vx<~*gqs&{}g;^6z{n{Q!2t(8hxOp6vRl4`i_sNkrG=FFL+ zZn}dHK3Lt8ojP@r2OfAp2i{)T#mfmwW)|FcphZiJ12YvCuYdW=U(iRgrC0A>uqGR! z9J|Qgd+)8~4~E;4Yi<>=aEgnIp%B)otMI+|-jlc9dP^<30sRLktn1p-gl&**xF3)X z-LD58cz}}Z3FSQ=%8ORyR;*?I?C1Y2r=NbBDqfaP*DN0g7j)`w-szlD86G_Lh^wBWOha@#b&L1Jq zd|ZZ2B{-lJPkYBaZ%)f=cD7euJ*+^p2$cc+gdd@1bu_@;d;vvU21bdrp-1 zE2l|*RwpR^Y*@Nk`d5Ja_bR8#H^v+(dD-3$N1F}In>SAf?^|!V4FmFzE8%Vgh;_$-{7p|#YbkW6f;YAneIvsaDW@94-0~ICtpt9mR9+lK3ms}#J zoO-HmKA~bQ-%zd!lM77)jGQFy6bfPi%NhY+C>=>ghD3% ziGt>iM=EdzT98wwOo5f!0~Tq69x}>wT3b{|40Ks)U|qc!6gk%5Fd<;5gC#p=%$`_N z8-@)k`>1l$3=~dxBk33|u3g&0b@$zO%OQsxA~#%rgFOHI3-ZaQpDOO}u=%84zkW(0 zX2G!syR~uS3Yz369oQsf+Ou|`1k!1^9ay*sLf$O3+Zat3tEk%Eq~p+F1%s{YtG=eH57dF+A5 z%Qf?_l;2OiRgNk?Sh720!*Z*a7fL6|HRFCI|Al-DwCWhBz>YuuI4Qt3cnh>#UU=aJ zwIt{=qw9@U3Y7q@lfOUlgesi{aA8jUa;mqrAA>#vmBs*U5Ls7Nrq&48_vo^lzhJ)f z?A1&D3agImbsxO{0XA}6pyjSzvsUP)|A7^f-`KkX7|mq2m};F-yQ8>#xm7o_^s(MO zw#eq%E%N!o3SCdzcSs>-rr6+scX^88!t4N7;hbfgrDSc5^eD`hal?vqYkvdWlPj@a zSH9VEaZ+K@>g$7@k}I*cM^`4_ksE+Dyzam&-_#&$E9+$-lpzBb=(AzrtuC+Aqud&S zUs_QgC}$2hqxu)f2H|RBzN=}49qz2 z6x-%^%fL!IGZI}-%jh80byf1p$1ls9xEq~KSc_13xf7!o!Pcj zK->XHi-O}P2VV}-+z>$*B6mLW)dDIY1}cNaix6MZIf0Vc`*6YdZzMYn!$ffrfn*-_MdhM z>~IMzfr6U}<<+$yOL}z13#bT-R&7@6jt?|{W>aWKq>68p{(XUK_%oTM8{ z3i1jxjW9M?GS0wu>jkTdfq7IyN&^-ftv1GKwOBr{REGSsK*i(+!=r1>+-pp8gG-Y< z^uX481`hCy+kKc{mqyz0ki3Y`H8b)?mm&kpW9ZVQSpW3K|KUziZpPvI7z33X;i-g7 z5q0aBJNqd5oxxVzmFXOQ2l>2_+CU-rECt z#J)X+f4fb1Hh|tq68V zgLF|c=whP5@}07cAJS(q9<)5*DM^PX1!{wzv(G*|P~={HDIZS(8ILTMd<`0eb>N2{ zQpM!ud39sl!^PwPo(1Y2lO=IYo|J)i1eKFU06!Rx$&Ob`BS=ia8Ahtthz3fC`c;KZ zr(Ow#6OoY)-LEoiglKkXQi`(G;g7HgisIw(LGvDDWbgr22oJ~(re<2W7@w3vbFI&X zWnv=uFs#whGH~OJl}f~=>(Pu24gs&qh5baQTCQHh%*ACkzex$)2TBLFc}WqxI|RK! z(HnHRCd5@#cc2Ul-IR=54&fCCbmMRV2NBQvHt7D9c3(M!glZlt_!tz}|hA_(t-+~Mj(f@JfVd>kh z;9~VKTFp7MoG4h!3tx99*mQUe*E89DI&^nc*l=&P`liFL;q0tyI=uR@dU5c)2RT#$ zCi|OHdVN`#?kiFAraj~=~iY-*X_k`iN2ekPFZaT+pIgb zMp+IGZ?zH?-=6k%<@U9RM+qw`EnQ)xLnBq#RN>l^*K}#O1$=ADfe%ok;Q6=`*dq+-Nh_}2wETDd9WH`s*?0YrTi#)pty8dUI0q~1f>f&WW~WImMyW!oAH1vks0*r z@ol#mR(JA35Eqv6w)RCH^ZDzwSrS;2mj@0EuYCjhF5Li@40ybe2Zi+*_IU6LPqK@W zjQc<=H(AgEeAt%IQeB`-R$hyFDe~PSod^JX0J88MIR?Imq;*cS%v9uHJB1_INIQw_ z@l#Q3Q@$|FKfO;t$vQ6jlh@IC4F$`JOKzzH!~yT7H%`Sfp#b#uiD$%n9eX#eQxdkV zx*7P&1cU8*3)VCU!1TCZ|IW{|To1?_-Zc&R2BLks`lMprz6HwlNO^u1q$JUF(qw&J4526H!?ZP_p(yV}7 z5K#mV#pru^M%PkZVZ$diM2GHndct2pVB4!=afmwg$xG+u*%bk5+|Q!G6ZEW{0&$z{?(=DVa1n`EY0$E?JJI$0fQBN21q78Ta?GI zE%2Lr))2ck^Qm*^cc>1ne}4)ktd${$g(s2uQ@crI-`&nOet-|XN~TCX^EcBOeIj% zv4dQ6(qR465zq6=CN6OJWq8u&JDJ|qQ$h{+;sif(yS%hYHsh$2(Sr&hFCOWR0TaMm zfk%0itlT8;PG2wQG+L)CN~;0iAfpBqqn*4j&ntguCteq#9Y3D6UVeDo z5O9LFLm3`hp4%WML!Vhdc}g8^=gXBcCI94?X_P+WyF>8$Lzc`gSt^ya_!4|4oVbm) zNyNTMAyK|tXM<&07@m2lq`CAMH$1y0$~Rcc56VKrY~^ssM7W0iW?yM zl&s)1VA+>hStZ3dLjt-j!ut3|CLvwFOs_ES*$w)nA-g;{@b>H}ZWz|RM_kd2j^0>_ zxNZZikw{9HMfiX!FWhR~-0}!x0qH(?;AV{D&4JS>>eFI75YXiTI4XuDJP;tMY+XK3 zAx_Fi3>ze!@tx4b7Mvj^x$+_JfUiBzEh(3EHCtu!yh=G@To37lFO~BfoX zz5vSLoiEqxr-z32?SvyCI_Mx=w!TJ+@S)f|d_A3my;=rfbRDduE`_-YZ-e2w@>(c} z4pP)9M+cO8d~B9<)^DuC>9Se)$SjT@*;u3B=<8gNqm|_ssVg?u2^COxe5jU+ko-_^ z{Jas}b^K^;M;;=Ha@Ve}2F*;Aou^g~g=-4qW*l>J_#TCF@}YfIQ9tp@@vXNla`IvQ@k!n$DaVm5JOGpBlK2<%H_EcL)iPoL zk9Wz`PeiQ)e>}paupk%T?%M*HHmY(hMVs&>X|Ll_NzuZs(RL%h)}hHO+e+n%PDn+% z8E3fEfEMLNMUDU7LzkrA?MhjhQ2rx(cEiiB#oAZ2VXVS>HEz*nk?G90yfCcYC%YQ1 zU0v94n%&#gg$>6E!(by1qL|p&gio{lA40g$7I$GRuCOjb1dTXY+_>Ow+~5*jqhO=b zBJq;|GK(RoClYKeEtcrP@HY{uaC-PYMxo^x$0(K^@YDi|Tb3ISqb17=!v^<2vURs0 zt)+-x5HActJ(4icEX!!%Y6eV9!37k(0#f+7WhTJD!;428nVlm}j3EGu!g7SGB$a?c_;_MpDdCrQE%(2?L{^m7gMJ6Ou!BZ1-lFXaC1N=P9f&t zZ)aWY!tA!Q)-Hjsl>{&h`+?dojldZLXW(V!H9F|gm9_tfBAGO8gHoX-wjN(Lw=i#I zwd~Qqlbp0)SNXxg-DK?GBDwS5OJqE(p)0>PLcVusPr2(Q9RC6L*?&ql%ORtS<@&St zP^)X%nriuU_67{_b@H3DM#`1n87gF=AkmYyj@>URWWQojg#k`N2qh&n{D5 zp1fLNr{QRm9~{+BZu!v&x&OoUYFV9mSWo%RxMJCFL>F}jUUY0f`ET$&plgmy0pI-S z?qVov4)kx#+a%u@+f{BlXOxsdN$?BxZ+>11>oZfXK)sI`+eQBR@-nqbfA!%8IekJe zIqo1FmZlz(|Y;xnP&(xYVYP$&wo`sb%P_XNPLpYPqY4C}!Yv1M2fCK}c&JH|uHbUj`@JeD0v zm(7H%fUYkK4>pFYylb8Vt92p_`3}LlG&RqyTL=~;lk;G*!!-gnvR*ze-q=^ zp&P+(GP<$)vMyRdy?Rlj>(w)kheWUxfy;y5C!))APiC?5JOVy>pO}(*3%H`BfM{&tp3&4 z`2Jx=rCbSR!9kNpc`$Gthyy;?nfNvHoJ_8jZIx2ExDMT$Uy|O6bE~@QY<@*qjZ7Tc zNe1-tK6pBIU_Oo-X_UE8T;Du|UzW~*ODH!a6F1t8&!fLKcG!AGtCFmI_06 zSRHk+^ycGB>!(4H(5=RB+QHpr;mRuMk0V8lzihZE0V;jFb(Bxf8w_h@v2=&B{1y}i zd2WDZ#6UOT`c2+WGBy^ff`cTt%1oS?OgG=5Lkhq*PpNH^;}7Z~&wsd9`r;^)Z;mTM zSy{3bUc8}DU{sF$O1$^wb{>jZC~tmICZ8>=l!Z9kYmff;A~~pZf@MoZci518(DlA7 zzVDy{D2E1F2{|n&tCw4zSqMeeBze+L!Im?2%@>f?3*2C87+B4U($gK@6U zRbnzh_h#Q7`SQ>kE7YafE6-a8qbreeVc26(XISar4`+?dSyLlJdgZCrc>exH(x*Gy zsY#AKpocvD{u((2u3Nf&?}fEpjWtHDQ_?N_(z!$AHcV)WQKvJH?1KqGy{v#W{@ll< zQV0I&#^qWeY3(xBn^%2`AmyMas6Y%g7BeDVN-RAp77xq2#8~mLQPg5-@vy1UHJqe8 z_(cRI`HdLdI}%ZP$nf$E!mcr%tlU^m@|YD7K*jM8qeb>)6D3eieAEq12K#|@} zEGZT9fj(bmtgM0ilr%GBArx{K9FamLq3eA}|g!Rf=x#8KR@}pz>%MXw459=*cl@eu2xij4MbcuAs zd1L2;zr#lrsymnyn9r9~>jo69FOF~-2uG$KmkN^`CVuy=fpW#;i{uQrK$$jacBTC8`$N)AC|u6kjmjN1cl%0c<^1+2GCuy{ysM2{RO%#OO* zhO*vNAG2y|QAaBOlMd-E*FU*Pl_3?%;aD&10ohf6h8LWt1ko~V0{ZidOK~`6o-9~f zEAT?f`B-zTfYryfx3|A2m5S;u^6JVuxuB#H_30=-e`dMthjr7vz~7f8n;@eMc^mxo zhf8?afX?zxX}$dIm1WYcbC$e3ZKM3*oDn)*>_D%K>g#U>brbo!zqr1Il+&r~3{A10l$+6p#NEu3zl zAmMw%ONTGRN%fkj%rO4;@{kUGTfv6wllkAS&YphupY)> z4Ug6lU3FAKh1fC4chl$&?SY+uJl&YP^ZH^y=Nj8=;L&OtGo%PR0pU;tpUc4~x5JOX zU`^%Nw_6^ROryG8DU7{vvNCDwIvkV}l?~S&SL-CGAy||88UezG|P!IjUhTooAS11!&s+`17Rs(zI z2hF{T?oMp`C(lIT4$PrwvpGvDr4X+baO2Mxb2q5cCx2_O%anDag5Ir<0aE)NShga?sC^Js~sJn|)^i?X4cwk z?M*C8+JAx?+&z8(>G!*z+*9RF>9LW@s%E^}g@#Cd(A0x@z5T<*yUU87BXnh0&O|Q` z@Ns8wS-0H|}eME|Ez_Ao#A4ftM2aC%@>QXX) zVF4l#{oM(aI_b2(Vi44Fjrq0hOQ-E>r`?wlm?g&?+)B7X5bca!fU??#xEk$mFteea zYo1M*(?aTyknUkqLvdd|!uQ)Q2RU2x-3j4?7T(Z~@ZRd*UzV2~pTL9SL@u9_VR>%d zR?!L6ow$QYayvhr0aQkNO9o{P4TX>^$Tib&VA0>-hN(rj*W*_3j?7_ZBJ)k5*%Ur| z3v;XqFl1h(1y-#jNQ!IppjMhJRzkxc2+UGZmzT+efF9XZKL0BT+jNSe(q?J``kr20 zShKrw1T^gxPzQ=)?Pk(7A#npC_Q(RQ&TvA`(~?nJ1+@-Nhl-ZEzn_zp4HosUHa9yg zYI_=FPovy~a`VN1q4+!dmgjFfpVHno_?H_l6Iv)6izp^p!iIfAN)2*ZKN@lo{dN`! z51JGSd7?$d=kSwK>9n1LFM016w0Pl$r%t_%e~uZVDbxzHhMG}bZr6jiFw6INrK+L| zQ-SHSU&JtJ`on;(ZqLJFT`k@hVE3Q0tfM6bc473PF)n?blWDjre(+2ztHOue!+z8A z)o4e5Lb-=P9xZx8#P(^fy__V6tWBSLA~RmRZx z4JQsy4sj3e``wysOSkS-Y*P}6*6aJg(*EUzL988`xz@nKW3H`tdbJ*VQhTPmE<(`r z@NfS4aIfvcF57V2=$4OATQAsj4`x_==hKO@O8=V!Q5ES}&$SkBzK#B`PAC1j)O8zH zFR4By5sFckS>Qls4Pw<-i7@zOh95K0EObohCjavT)Oumk4`nvXyj$V}wUOhFDZB4o zNBw{o{g(ae-$_R9cJIcE_P%q(YIGy)1W(c~q4@)%3YJ8nBVy5Q<97yJ!J3O>t&SSg z)&k~MY2#y1FE(2V5Kq(f2g@!_I>){zOTA!!vQC5)>&X?QV!vfkyo4lJP{QKXtJJDpk6?t-otuZnkd(C>ZBvAOs5dU#W| z(ru&};$_B0748DqIc*te2|N668XgM#2tREVvyh8us)27Jgmx%*i6PFt#X~mdOPtRGoB~Jg;`nCObHwmZ3V!deM&j44dzX z2HD|gFatF4Fg{JjN_9E82o_87QZ7p^tQiO5&nVLizkgM4L(7iZ`j1tLE=?g~YU7gz zOK^XfZ6Ax-yb%P8I)+-#FUaGw@u%Yl1Kzz28U&445Cp;M`+6PYppumZ z*x6=n`V{LNNo?aMUjVaHc_{U;|IfL?Xbk8MLT z^{;W-eABbdQcU$&{{2DYkTi|gQ)>Bez$`uACPq>K;v30P>3%8svg z8GS0r2%WZr(S#bUcu57DywczCSSMqYd47i%eE(W4yf-5IcXM`bXWSJ&D9e}F-fp2{ z;1(H3{nj44!>qk&BtXEw#K1U}0&nKcVCkR^1}G#{Q7aI%PmxY;luG)N71>&P*&S8Q z`Sge~##tk-f`9D`Vo}J;4CjS#$MqV2s6{?~kK!xGqFJD~xRenp6M}M=MaU2CC z(mP_}uIqwF0i;*nw2ON-aSz#E8m+5IAF#w-(#YIJ%|N0bGlkv4g!o=$2DupXYGqL? z+#0a2tHk2Fvig&2O-iTb6g&yRC75LT4^iH4{%mKAYV1L!F*$g3S{COM1w!3+H<};n zMq^Esq*k1hqxFIcS5~cce@l_mW4YK(4dTT1#4KMZ>trJ`;)J>XT8jf z$cVPygS52^tHvd1NE8!*8)yL1+~Os=2B(N6G@-NY|Mp6qHp>BOjZlH7bPD1dyM^{` z=bS6>Maq7-+jq3Hmh?=V%?rcnPnSDSo={K?i1c{?1LMF3>>b!Qh*ujmo_YXZ!h`2R zm6J-7?-|}ym=&Y#yKSI>6u6{2I+3d9{N;<7PcTXfK+F)a_p7x5n&y5wwvi}8Lo#m` zha?C<*)Fc>VX{(;EU~%Hm`f!|Z+Y0!M&iZRQ{-`_K!zuBao>Q6%vLkZ! zA)(#B14q`5nrXYQ!#_hY4|%IrrcOxBMpiGsx1qG`gsjyIjXYYjwG{C!U(;WdXEkRu zlFNU{wejSwT)b-_({yFWD3=aC-!Gx;H9}fr54kh$*r0i~$G5%waNK<9%u%;YM!{I9 zO&YhRY!3r*{#pBh00uq(qmy46OWC<(0aQokB5lJ`qp{%CWt_k6*|s?$Z+k+CfA2-0 z^vVyGq1dMPn)PpkH7RgKQZD2%lp1hV4SWgW6UYJzu&(P?B7YVY;1`!J>EOS6@TH4< zoK>l~sqK>-h;AkHIR5R#x*V3~yb%ju**221)#0=AJyz|$IQutNonC0}X$$IMQ!~Mh zEf>PIQ0oS_+WV`swpgs!*e_)?2CdF-Qxj8iG*Z#Q7`Dl1`$aYH>@o75ModR%>FBWh z1^>jKOXO9IHZ-oBwatiLPPDS~jmL!pY#;!y=)I8N9}MyyGM}5xE(IK$Cxvr~O7V zd$&XuWYxdc%(dC`FJ_=Mta{8G@D{>3B2 zYWB$rIT$^zfD#t2ysO-7Nh?LFuDJ<#US-_IgJ*l_#y6+ZxZI0=i#}1(%c&?GXE+2wG*r7iV#y++{klmaERjMkYQ{Pth&Mqt0NJ#km{J!RRva@Qsr*g32HL*U(8ER{P;$fox<=_QsunOt5KWBB(FbR9Px#fCkJ9hu0Nv{E%TgZH!Im34EQ?-Hg1nDDKEtM#T~$`)Q>bfn`*D+Iz@@Xppd)< zuoU^Eu?F+jzdfY_szT9XVyG64< zsne*^i27tKJN)%pjL%1yN5i?Zl0b4;gJeQ=MAcDDy1@FX{#D^Rbvy8^kzovD$$JRa z2H6iekpYmmi;m=h@N1#vy9jz5Y7F8<(;ItUlp7$XgPobKNm^8f7Fb$Ge`7Wi?Oe2EiNQL%r?e{(7LK?tj1D2_% zSfgnV&TlD~z=3CrWU!8^8z1RGt|EEuh}v@L6c~RCRoWX6v}O6LWBh;rRM+^3e|Lg2 zZc*jW7k~_5O8zW~@hzm+fW%-&kJX__E z*L8htPm07Jd$grjI%UAANvtK8p`yoSc1e|GqSGra^0NgE*1xhru_Ft8TvWoJRkWf? zRqwq3xDN#iTLJXagoM3{Ji%ai_Ny+^umi%TJDoqtoDnJnDayV%-+Zr-9=A+~9(%#U z#iCO>#hFp?Phk8eoLu_RX8fZ?C)gmr`aM6hNuI6-ZLhuoJ1sg9*ze0Y;mc3w6&B1{ zu{5oj^}7jkFtXoiU~io#>qH85P>)P}S3p6Ik1h4_w;WoF4oi9L6WGH#7wKkQ%FU(T zhq;-HX4_HVh-b>aS|YVXg*%e#&5Lj`NHf0c%B`ZSp<2c`r%PWY_qQyWX>Jd{6=6^2 zIvxJ~=||z;B4>it*wN|D`(EzQHmX(Zn#;0PBf^TbpzGms# z4Quc_+GvIyk&iF1G-QT#pnaD1u~De#x_NO2z~V~OqO3Hu64^NeTU2TTeL|l1CimUf z^NHk_1`K~2&_4Zh%fpIjuzVnXwTjp0Pv~lpiU*kfU}LUIk$pifkt{)yzW@IF=~y&< z0DD=_Il!s;c|UwuL%Aejq!KM(D=L3NpV8v=d;CoZQci27Lj22~NzLdpxi+<>Qt z#w?Yr@7nEJo}aJr^gxyR$1ujyW%K$E9_Z2n2m1?_@#ncG?j@Ibb&feNu}+J(T8BA_n*?J+~}?FrfL z$KYEZdZjM_(pV!&uDwRm4V=$e3dF8?IU0#hYT7iq3EoO*>Ox@C?B>iCAfrx- zxP)$5w|>O!Uz(hLUpLsnO?Ggdln}|~7_Q}BAI;?7g}W~#oe^tFr-^bc@<(KpW8$|2mucLOypF63RgtP#U@;8P<#wp|21}ZnMaF1N-)0R z@#~jk#E+IGFHxq2_;%xrg69TPO{>}6n0nQ|;$g=F zR2o8A-iv;AhLNJ4n)#d~

`b=sM}o-UFrMq!kajvBtmVb9ciwLk(I==z+~$QlOCp zKlLGBBe*_g$#Tz3~KNqhr;O;nY5PU!yIeJ-&OgtQt?@;AAew zWR_&l7WVaNTTwIJ^EX4GGFM;WFn{6l-HKTXpCo4A!wMp0~g_Jl*4$MN`(5pMCsaBCM z=9G?cJ5ni;$+L&MTO|U-jAV{i|AYzV`5r+5s+r3+^vps-Q>6zT$dT$7G7K zx@@ifP2%y#lKaU8jZX%m92s-Ck%~^Q7clHL^aKhW2CL<27$hTmKMzxOV4+pTSobo3 z-k7}#)k`;FJaKQtE5+hM_l+W`ip{uLkEphMDc3LM^W1AgwKyQy?PEi^NDHzX0LZcc z9Q907q|T?`A~RuW)@AiIcP}w~1s0Ixe6pRYOlmEwzGX{Ot)$;$B!$hQS6SMF8QNj! zw3h5Xorf_P?NCpbFDR$t_tmMq5a9CF=QkV@vu6nH&)Y{Y?w5aJtnk^2{d$&UM>e@+ z7dIR7g>Kz4J;RI(9WFNy6`|l{vIp|&(fs&{j91{(UDlTdXcDi^4@c+JNs_amt6BT@ z(<|qWz+1R}!pZ~m)C7D!Xmo#gnaPGWGoLO9`#@j1p~t)#;AlL5g6vz_U$fgS0s6va z11Fv5byaY2@%;?9z<~$5CZuM(rI&*{=`iX~Aw9 zXm~1XAI!56T7G;Q$%w_JJ58qt=;)tUbWi)6+~4^9+nlnsX>x(Kc5bw6-aP;NXY%@Q z+VQn60q_T$O9{(lC;SOJ&%j;pRK0i_Pa^@RgO^-_j0Hr$R(nT7Vvu2L#3~S)3n)ci zA}Wd3l9}=)vEZlf&SvA9VJbvk*^raPd27A=yn=M7p=JBQh@VHO2^0h#r9VVG3^}_B zesNFedGwHD2?iyAdd1u*qFl-(U+=?W;U$U3YSv18{^^6oFTo}yhx7C5)djI$xDXSg zRw*aCilNdF)P?ZbHIWa*wJaH|i{Hj1r@uqqEvj#Z&rD0GD4I9QNlN!LedJ4ns*HK9 z5kb;NgxX(Q8KG~y9|O++DO&ACv)G-JB*vArH=K9bqcA$tQzyqPX+ww{D|JmWX}5-> zib@=GUv6vn8yYGMmBnj^nD!gu7VPOh<%!a`3~G_c;_XQS8vxRoj`~LI#2ny^%S)k; ze)0F?I$KL@*-KiA%%un!U!h~b>85emBh5-zh17D0%Lqe~=-F{{{_{&QfjJ9|buHtV znF|HmS>t%n*Id58JPWUYy-X#cPo>}2phu{)S61Zq!pePN5slPN(x9Fh`dH%yTkoM- z3BRqij-_jzCx%F$vg{}SXNKCDg9$nk6m0QW6npaX#H=@&_Xu4%Mp6v>Zby{j*;R5R z);O?OAwDlBQ09=tg0VArcGVFaH~Zu|-u7%Yk-UNck*C2w_oa3jq70db2~1R0!nzLG z<2Gp#X()vyHPG=?$frktIUP6nx0@Lh>1S854G=KR2;+J1yvCicob$$rH(9x$N$(Ffpx0F_*Q%m=6Vcv=!K8N`kxtYrYaGjRhn$ zLu!$QVPHWbh*kEd6DhIk3W^LH+?&b{7vVak7fSJ$_tm659v*3iVR520l)wfMn<|H{ zW{HbBN*aZddc2o6dK3@Myojq>sOrl+m6OS!~%S!+9L@iOD zT9kD(jfhwj@t5v(XMfl?z8an7NIiN5sNDzim%<*++9N~6aa`*A?Qwxb!uiRr@lJcf zCeK~5fp;D>qJe6a&OPexGM@D-+sQxr{%Vw}KA+Tum? z4?1tXU^%DEP&KrECTG(zdfY$T^`~YQQHPD1)+Y(s%*#I*uhdMDCnZPOEQB3tzjqyX z@C_iM{b`1fg4EX()@+H-bi6&-oe2`iZC})5eU)Cpv|e6?X##N;WN}W1%I2%172K-@ zt3@|_!+askkk8y?)|}vQeHB2(qwuSnfX)MfyXfl-$?9asjkAb9X|S`eMo(dXAF~jG zWB#?C0#zHu<6r{%OHqJtR_23SQwYz1^C}YM!c^0#%GTj-U?H`fL0{jG$dyTM9jAkC zgji=rSx8sH-@g*GDhnK&tUp6Z_+7<(wQEyz#y8mrai8lZsw$G+p1x@5-tP!)3w=5i zGDR|UHx9p^yqRHfQ3(x@Bmur>jQ9N1$1hMp;c0kwwhZ`#=+Hb>thPP$n#$^(TCkN; zpQKJ$J)uJ3?zR#6iWsVDgsU1wjh;t>_Ki?Xs7nG5H?|T8k@Hp~yY?iUM)3(&rxX#? z>62Ag*;~HWNTB57g&ZMN06{dv=(2J}XT{glPT!&-)~>#R1#b;+=KWWxs|{9?G;jso zV0LUyrsN0WVxpedB7bGgR^SxHU>an+GOM6|iycz&9*sua1z zj6~9>F2x!7(h?(&4yMbv?!L?CDD#sUx$zp=bIHrMi&~^b|Hl<^_b03T?x%y*24>}e zJhd^90i4?h=mVH?_I46GUQ6UWc$5c|mjgW7bkMI<~*@%eP)ueee+^hdaELU6Rw|FV*9&tmTaqEcb^E_`eSC%XT9uohV{haQe4c<479 z`s-3)bN4CrzVxb%I+^0 z!$N3jstsw|U&;MH50x;URHTlPh+D^t$BkiiTD(0C@c>ZHUJQr*m}~wR#j75Y_5;)5 z+j~Fnju%K`h~X93FnYF}X?a4==T6COg*=^?EuK9VlYH0@-cA{Ral%$uAx%AN7vwR9 z?6x7L?z1;ARWNRwl||BDGGUyVH~7nE-T?NAx8!a<{6cc?k;|RI*h1tIU(khjbc(!I z$wHYjVk_r~5_^Q=->1}-eeX?Nmy9)lXf^vd-Qq#RAMb!;!0yk`mnnSmLRU_%<6G4< zFKTt+cb@8@P_u`!DrzlN!!GgLN$m&6JKD8_5~*NK)yQlrdU`IOfl!HmtQXFOq}4wa4WW;j~^@QiTg%U8p{|2yGssBmS|d5w;4 zQaIvQ)&vSN2)%=zeE>+fMNfF%0*{KWMizcde>E00k-09)4>W9G>cJk+cTz-YOD(rz z-sML!x>+?NU2P@qUH=@lIXvO0FScNsmu+b;Kl;;svWmNOgscX7|7g0Y<@he1$*6Or z>Kr?f#2}eXfS4#dx`?Ai8Rg0vLy>Cj0^C($A-X~HkMvvsL^SDqqFA@)syZ6+LB+i& z_MWcCHjav)jlT=%kq-;`1b?KVz13Fn%4e$ z2dX0-O8u!(QHE{;zc|grCV$@mo<$%URiUYTVGU-nA_0jp#_YLba93=cF#hHUsc6oR zwdutVnMQT}KZ~3=t&~awQ*cbs@Lh`{Erf*h6SLrMsW^5BLZZ`mcXBqEOde z-d*T)4(;cn=b#^SpeG|bYt$8XPUXNbz$+ax?&-TaN_ZRj`z`Xm+Iqe`7(7P2+0#W$ z4y9&Ae}xs+3a|3st|jCXvi41be#&A@g?U*`Ax|NDK#;*ET%U2%OSpqVHIp~ZMPihw zM=X35;>|r>o9dZqv7d&>fhgvhEd0ov(X&BTS(QCqcDm##8k8#YFdRfBzxoq}ldED$DI;t+Ym$B*!p!0=KE<~Nf9nTr=o6fobjUeb$v4*+LD34gVIo#4 zp4wD$o>w26icBScPa3}ooE`t~JO~fpK9sgEO6tDxuv5MX9Y+bJO$vM@Rnd?4Q%+WPfMBNc-Lp(t7qSz zzIoaQh||vut3q3OuxzkRi_Y#ac8{awCr`i(%{Jc6I0+D z2T6*YYC#Ma9@D``Ru&G0wE=h8omfN%8CUDx-kxQ%INX65wO#^xQ|bQ41wy(QATN%} zDs}i0F1aUVsQ(`0{od^$DwxYm^4)!V^8bxH`2WXUOW-}~xgA7Xxr~e(fHxwm;>1`~ z&F2#J1A@%<)%o@Y1{N)YF&)})Icy)M(6iDCGo4@Q%4FKj7?^G360vzK%bDY?aR47` zpLj4_443&mNtHG65jnO;_pmwsM!CE*#A|p(1(um`g>2*%AkPwPIb$XG>Z}qh6O6O7 z1@ShX3j0!5Y7i)6KB7eONjs(GhWMVZ2aRS1PCJ{v56(-^-dLdU1QLoO4Iv+q>gd^;6>5 zhJX&|RptE8p5&SJHkS^<8tPe`*bBox;EI=z_Gm)GX6;G0Ga`@$@(#1{v9UI2QT){0 z;Oy#Wrqguoa^i%OjwY_P!KXY~>vBN}!DHEHMLKJJ$4a3n#JGs&#CCY&$9?-I-ijxTVG*GA}cLD8X`CG`!Fh$mW0hV zto|}Jw9hQ&pSGv9@dQ4_PEjC6_r7i}@<#=i08s`1b#)0HACpTTigI+nNt9i^a^=Sp ze2u>_J3gikujRRo6g1f1mlUq-Lh!GS^p6#zcv}I{s*qEBO3V@4Hswk%_Fq`n0`P1c+EO|3TeFH~LQ^rq))$dB*zBm$|i!c@_KPR=i z3Y`qtCJ|SIR?~JvcxZ@N*N%YVC@m_>{X%j4u4hgDuRpYJt=)Qzayao+k9G`EWn3qa zTVbeYroZXYow6d2txVH7+!DYfV1H<6u`~g4BoM>P@#u~O=JFb5jKbH@sj2m z)eFDp;`xk2c`NnW^#G$r0^$kDe;uuyW7A3IDAPMT>JO{Pk?(vkEtd*J0CNL0?fVc0 zz)=Oy3cyn>FR)qh@m-U?U?lWi=;_lIJO%J@^VaU1QWAkOV~wLGWzxX`I+{S862OR7 z0!PKps~WV}&BHL!PSx?A7Bf5JPbS{yr^%Yav~L{oS;jbPIZ_ojwHO&yZ^6iK!l>KL zGKF64BcJc5Qn+wlbC=YZmT1~UZj?HoeWvZ&t-qI%-Np|FxopfbF|IyYjcPLPZ)EU7 zRNpQ8nOW$`=NUG)tMuGw?{S4%*Os#<8fh3 zU5!Lqljn!e53{&;5Sz;_R+GPV0rn^j0S3AL_w_^%36Z$tClNGumD!w}=1{LuD{R>D znvUY+GvLjejoH;r#`~hd*Jnyr=-?7>Rnl8!{t$IN{8F6Pmu?5krxxFb>8+-*5i3IX zxd``m%zkGx<)kzIP0suts|tVEA|Y%&DUg45HU{y%>=%K)`azp?n=PLfO%B02g_r9K zZrIR^@$m_1^4dGTS`%TD+GSnFAOU}UZ_0}_RVR}lzj#=bZ--%k`G9{1_JEsq&DjM( zw$?M-q2yn?pzykG5FsW2{4WK`@kP6 zs3?nNPU6NZ!@q2cuzU)4;R9g_TU^+{G^6R#cBU+fsfXvRpYGiR}N#f*~Wo(s7!v7S%E0V$C zAePsH)t({IZhX0*sT=W^5$X3B27F?le7*0Z;?$ff6b~qllM)pk% zGOABqrkmN;qZ`vO$Q}SYfU_pfY~cwbQ2f?vjeS4d?+SpV7NCP8#oI6Y;&oW=wQELU z>WAFK+~zFbrHq}(lShqDQe{n!5(8tyI?->Q%k;}Mv;Oau^#R20i*F={IA5(neO_fH zn9=puMRMt;9d6=IKL{*38VKhcMn9$bOs6%&u2M$T#cTcdWzkDY*-S*P?&x8Mr>yg- z`pVKFXHKMYhChIBkkbDAfo71T3AMHwhPML1MMb zYV6cqrFx6z9SIKV-h~mf5^H=NJ5f>C+}Sd1n^rQy4{z8E@hl!^_C~Fy9lE3MOJh!o z-^}H0{G**!(`19M>VKfQYh=)tONzw1Jn|B(#}8@8 zKDxR$V#pdMh-}c*@bX1*Ao@6^gh!uQscZAao5_cZ`S4*YFZQQ zeU{bhDA!6ii#u0S>^&P&0n3^kr%9~NaCl0KShkROhS{0kW&j25Q;wUNnFFkqX{9~C{=1R`>2{8zw>)MPPFksJ21wn2d4 z5`qEMHRxrlEfeM^;l8bs#a>>=sUBvx%fU5xTzC_M7(?4deE+?2BiTh_gS`yxmqDRt z{>nzdy5~!!LIn%w&`84jmG4dH*;WS@dlNtB#k6C+l|Fzj25C{&hr!sNo4h{qlx%;D%F@Yr5Z`$k( zVfb;q!)jlbGpCLZPc!xR4^Q?v4ksCv#{P)duR}=_%(z2dTQWOwn1YcU#SiR|MqF1{ zqO$uo#geYDB@rhYvt6eq5u`#*h0V%F}w3OUN|{~|Ocy^nE4 z;efpt{EWy!m^j%~+`~FHit=-{As;&1DHxZ7TiO_#ts(Py1MCeFZpzFS;0rHLY0+Gn zdTc8%n{C5j!5XPl21TM>tD{@naR56Z<#lrYIB=sHT^nL?>@-qVi@kxy?HCwyq*+-S zd+rb*VntpqiIb57$D#iP-`@dR`y+gO#Q2as#|}9(R5WZsda6z9vY6LffedRM0KJOH zwsJrmVvLT=S0{5CY!7S7c-@Tx##0^=G{zLAZUVO=G}#;_8E01ooj6l<$|`|$wunxO`-`{lC7+^pt%(mZQ?mpl(4^;hvVjURwW+hB@Y>fU zbE0rCl%m{QI`L`9$^d?Bx4WHZ6Td?Tms!t89iOBOCk(0*EiyzZ`kL}Qy>}WP-j=wX zIRbs=+bKuZnES%GvF+ZtfUNF()dp%@Xoc5fTSHcVp^9Z3I11}05%CZrrz{eTNw7w- zIaqtQepG=~j5VV)p6m!R)5qpRhp6{V4227$Pv$g#L)q99hv}V0@%}RVZvcRq=YT0z zKEKRDrp3K1S6>P4Rw3WOh>TZTyIQ7@H4r<~QI*2*jyndpaq1JuO7&0Gh=!jz)pS~~_&McRt(+s?x29|?jW%$QZm&x>uJ1Pd2kQTe zFB`~U1l~D=-LvmhuKyPxp+qd#W6e~eF{}Sbg#QieEI5_GQMs> zT-#rvv!6Cq=(`;*a5g7Aw`OUg%qS%v_yOPJ!aIC!E16N`#m!-gYrgm$!=x(J zFq-H;UVS3Vpv3iQW6|pT*bZ^nG3@bpntR1FfE-%K)Fn=FsNuYl{l-ql_t4G^l%w>yBga=@5i4$OZcXa5hHUf9%8FRIFH ze0cB#{JUwCxAy8Fn2!vm&e(Q99r8=owhoKnMIL+P?63W?Km3wuB{FO|G5JXAskd!oOtc` zQ@!aJ1w#9kYUwJ`3!jChcH(1}TFiJhs6RfZ!DnroDLyK+Irkjql7n7o*w_YgF+4Sz uoIp004rag@}lPl!yqi zf`hH8g_Q{aAQ_gF0;8@xg6Vsh`~yvs7+gu%Hct|fm?98UNU)5Uh6I*85gXk0wL6Y76L2)qkmo=Z{<|RPhPhfkA)!Xui zUPPBd0SdskKxh($2GTti53D-^lG&Xf(2w%1IXe@mT#H@jhtK`Ry)C?czT?LL&d<%& z;slhTAZ=to+Th_>Jz&}<*g@5g7=Cre8*)X3CQ%Y>n#mWR809u77M&P*WGtg(XC;>4 zR~Nz!((qZNt(ECppZB1z7r%g|?}Gh3hC`?*$=JU9n}G7|Z)x4CIv*q+bqhmx(M%fe z)IK~ebTSp0fOYDL&}hYd+#Am@g)CQqLYX>FT*;z@EEF8WIFY`v3&xEIDM^MfQo@0_ zgvB0CT0A46af{|Obv#Jd-}DSTj+9l`$T16Ks4XL4Hrk3mv5ec*VXK?uUC&GvmtX56 zLbr3j=A-;gPBK^q3XqQ}8J|aDDZ;L>*Dq1#%vdhW#L=cFQ5C8DYzvV_A0G@;er{xu zc2mO{)kP;iLj+W%OTNlu)6NFUg(@7vtWj@0KHEaJs*>-MKdhlr4~pIok$w#dEHfI? z6rB~7?O5j1@V3h|wMwDmP|c4?5NqA0X~i%`7Zd6*rQ;_hVvuX};{BdXl=15UU4H5y zCQ{`!0L%AVHx!94RAP;(Zx}J2q$c+%_cqX39(xOdYZ#n#D=+RYM)Kk`uGftB8>EjA z8rVR0%T|{z>Nnq;DF~(eIVj_Yd(6A2VXzpILqEw_cputVaVZ#6Y4Dg|qBnhK;767W z6=hvSCivgNW+fBfTr{}f()$$*zcBe@i#zqpUIEsTWBjLx)i>>T|IB;T`u`GR;2L`7koyKABLJmz!g$b)aW&gM8~JCxl}kG{8dSAv}NREZ93 z-sOg7ZyB60q)(KGxP0NXP=yG2kc#+SM5X5i=UnrIi->%8pe_xQL|>e5MBAvN-mu_wbNbg8#<5x7mQ@k+S=9%_UomnL{GUo&Fc?b zp!-94#V|6!$HUh4GPk;JI0A4c0N(suS-=~5M4#GR?CNM$x zt+Ch*CcX#V;Y;)dJXfDQ6bzOaDl)_!v+te`CZWfP5$0=vQq9*2aLXR?8mKcsdk?)<=#Fco3Jo+|Lg~uu7$QVegs4hs@(X_&l z!#2p#MpE}=Zm0%gp9(-qOcsb1SjXRv(Mq%KvMa=zh}+}(hD(j9TSK+QH>LQ6yF|T( zFLIBRtf;#ZvBP7Tlj;KNl4b>41ucs26z)qNe)#+ZXUpIY(Uhyq&HlBS+qF15TQ_?% zgIh^4lU}J()u?vbu;^8Z%<7en9wjZEPi+xlv1Sp~kkPQ@#Jv$sC@NogIEOV?el&Fa zcy!r_vntZ)zAAjKv5K+UxSGkQ{S>hy4`J$*c1c(i)#;QZXi;!Nv= ze4@K_zG!rWxD+unHm6n4tb7@{sq7WsmE~R1rQnnOj`_Cxf{cURm)>LY1=|rv3x^8( z99k5@6CzuvNvO2PCIB;lLBvZ0xsSY$Iv6831aU1DdoFs!=zuwe@sX*TNdSo%uOEjT zx4}+f;+J&5CJrJy95WhD681S$+t0h1R1^H4nMNJsEt5}Gur@o!%&W$ABhU%xlvx3Y z(65M?2n;wKwpaAZ^YbIDlTPW!m~*#98>R=Q+osI*Y^FmK+!JXBLI*1cv=cF@uIh|* z;B@4ewDi%Myhb>6oE6%tT;*vCCKf+&3`!f;tyT47SgS=EIMb~ zM%u;(drcE3oHBl9Xf=J+YN}jzX*{Q1b9H%e8F9HiKRL@@#ap>OgFi1nzvP@I04Jd2 z`oXPjPtILO;E2D(qw6G`#hcYHharcRsb~f?3%m6=TqFr3;TI_op$XA;&26>QhSM(7 zxj;5Sio(@oZLm3tFktUf3Zchy`MHdxCcmV;R9V#=a*9mE)nV?|cco?{X9M00*Gzli z^vL+AadCeErkx5vWE9b@QSAj=nCD;1PKp|Obu%RyMQcCVpU1tGlx*0)|kTM%;^gCzc$ak0TI7i(Pav2#rlG8KuJ6P`pd@KIu{-f`k z)BLa^t@uV*Q&h6}(tt0v?s#}=d+kGQf}szQEIt8Go8jGV%eMX8uG|324pskUq)!G) zW)1naOP)v`b1k(L6*YY+&9RPyY-vw;o9q`5fGg7i+w0(AIJH`|U8>7YEX6(7LA)#C0;g>d47VK4fmL@2&H> z?{(aD1Sok~L^rtSJ@-?iYne>e8EUMN|B))EbRHqwU;7$rn3>1_2SB(vMN5zf6A%D+_-M**W^+sWfjNXtTkCwBhl{G}-zJ zvx$raejd<6G}k;Am*e|42YOB2J5!sR^#kg@M(9o1N3!w*y;kUtlbq zxyDhK-9#^tqBp+2u&U6cFb5MBGwaR$X=-2KkzrDMy{_MKB)G7@aJFzM8H!=8R@>Hn z`zC}|ys|`RL{p>gW}*zXd@|8J(OO$pr&4=El~6_Z_n_&-EL+ZE_>ykrm#XFZNvDlo z=gUR-CB0RYhFsJ9mBHC4A`U6`jb;3<)~4C2IxBd4>7L{vt%RoRm!LQ41@;8%!Zm)| z6dT($)aLxw(e|>R#mAEr3+9b=ZANOVRT1qXE=x--^H!UE>&k6dXDSz750)?0OAlff zma9{ndbb(pwx>vpNQv{LjmnnoC;iUW_tf{5FcO4TKd&q~uU@DBG!xbF{;*%Zx{4>8 zC%cWlBc$hD-S}pC#+&SXW-u36v^F$^|9h6Vt#w~?FKmvgPmWMvfXn%=;p14u3^aw$ zJI1f&I(e#n?ml%3^M3VrS1>|N`W)&K{vOMVfxRKxoX>&!YPNQaBt>3#TZhY)?d9ib zW4|RwRk4n3J4wg&o$19yN;90z{d%@nh+Xks)x}1gXVD|ghtpq2`3$wS?N^e=DS=QQ zyVu;U?OTa(iJa9#z9InzPs8hilhU<05ds4OtehyX#dqz8j^g?!VCV0}PCK{jwLd~W zIhVL|3-3zrb&m^qOM}(Tzpvfq<_XrkF0D7$vRx@IqyAhzb-&x47~Kp}@g)ev@hADX z{9V7IxEagZ?AX-B(VM*Ffzz+aOG^WEtReyUpa9UkA__xgmmn`F#XrigU;a|5{(*lD z#{#g`C=x8SMgh)(7IeKRz{>JHgi!=HHekw%zAf4iV)ka@cw&wm5Hm4=@2 z7%bX;{^>|IQI|56l?Bjz(q8~jU}yk{PYUew27uuLp#DVz0N=sz{+m_;`}Q9hZ~!30 z0s#3R8O_i9pDXq=eWL%9L&OIIU_PJFK2zW?@c)tqYxxE7Uo_y)Cl4T`EFvZKnJXJP zn3&i&n%g?XARpFz7Qos`YB&M_m=ymcFexSSD*yo8(c*`?le(-7w~?(igMqQFp$UVV zwcS5@0K9J8pQN>klL4`twUvz{w;Lbnza+Rn>3@oWq{ROcakAtiRhLyD7O{0OA!cJ> zWnd)bha)B?=5;VO?#79c{PeuQI{kxqeZWjMj zla1ql%=$DC_zwrj#J~vrzrma=O#gpi|8V{d`>Ho$2pLYHms9@n{Vx=x>@riW&w27aSk@-KU|C957MQZ#Hl9iGDe2LJRSKNBzT|5NsV$}5>T+FCjPBd%g&;l%&hz<)vhkK+HJc!B>6%l{gt zf6sw`m3~eFemGv>e@`fWxM0lI?*M=xKuT2Tha1?L4$LnCv4!lt!Vr{r1!3aYX(6ye zEcng<;$KdR;iStx-ND2U3^c#}m8>KsP(_1siX@Y&?gFQ^N=W!xKb|MY)A)FJ>i0J@ zI$%2dT+I#=FEhE?dFoks+Hn|7PUaWk&?!?%M1oU;;I&MBbNdD8`x-IJ{z61E(HkQD zYQq2zk!Ek^RaB`3}IgI&}tbcBCVW~j5DV{m}Cf<12;o#(@%2o z%HplU-7y8#b*QvAf7mK5{tSCR;WTtuBz?`Th}(&_xJL?}SaUEmvU)Ef9LgEwk4s+Q z6Szu4lu!l#$ZqD=A~Upc3Bw^gz)L%H$V6IU2gJYU<%bvrZ}BO7+};8U8;U7^YOLLl zg*I~=?balV*<;vRHz6t5D7DiYYcjOjTw=FL;bi~>vr?YN!op(3Uedt)W2mhU1CB-n zjAq|bXT}ovrs$x7`Aw+CYTjXJb)hml+I9x(!SCw4R0ygLIUQd*wJBtQ3k-F-B9_gM z`;95f+p%klIDyeDR-f-n3IUluQ#b?#-}@h$*Y{|1z}_h?75p1AT7ebe6X_2_heZ0! zFFXv6*bqy1zJLB6a8z!%{@I{o^Jqx>tq*2_rdZLs0|{A_AR^7Qfy#@}%>#U=Y-1A?r*J>x1?tYqLsebJ8&KXwoa!l3Lx%8?Z`w8kQ#ye;z~)n7^yMPfp=xH%sg zQ-p?kI)<~jp^+WD>w%zvTF0(#Xo-=XZWuIw{8xWC&B<_zLjq$_R9poT0?!N6C(-C; z*t+q)m;@}zNea_g|B5eM_R^CyihA8H{+rp+K)3!YUUn0vuz}d&M(53iN7({$r&g( zfq&6t31Bx2zYkwy-A8W9_QG@8F(ma0?)jp=hO!s!yaoTaehAk(sJ^89eKMYofK*CP zG{GT$*HrBp9ODUsXt|>w9N~Lp{v^+IKhcr9SYoL-Zx^9XkQg?E!b2Gfn<|Q~TSXXb z>yEQEhJJo=@el$omjf(|9q|IOY#56grvHy~9R|hHLV)ir|2V1W4~7?2b^jU>-Do69 z5=q5{o?V*5D*uvtIwA%J9%0qRU$Koe46Yc{xk6M5ZL%nKVS$yr4r=x=sz~U!Bp2{O z-w)vHnS~gVUgAf>-Ql;7HnA-KNWc2u;(d<~9kQBa5t62jkfww$q3epS@&eOa8N=Y~ zgOg&5s8;w68kxpvo5%7;@2`j*5ClQQ>~p$ebgc?qZgtGrj}TZknZ6C#s_rwZtso9v zaV}zQdw$^bj=;^RR9X$z7sfw*7xWY9%}XO7v+N$E*!2`rMW?4db&pK()1*F^E*hdZ zhqoi(IW6wTVn3NMhT6|@gzCt|_;GUQ>OSz`H4<4*rZllC`t@Be%aznPMM!Q^VM&z&V zDCKC+&W3iytVy+FZiThZe14Ue)@uL#dHk6^)(%7oUEHE4FVBvj~s^H+-n%Pm?r zyrww$Mar#5>qTv;j3b{`M(47UGRoeQJ8Gme(}K+-T)r<&BS<)fwG)qFrC7EvZK4fH zA2$S=VO6Wzm+lqaIHfBiye_=yUEn4u*>NtGg=1_8@_^F%(Y6~QhMoZ7-b>xgUrYPV z`DXrcO_Q!C4qEkqB)CjEfQE_q*PZb5Dkq7!IY^A!as+UTZ6?CN zIlQ*S%c9xq{Dg#Xwg`o+qId^qc5MOPYlgE8f0eepxxCm*s1+Umx*88V7kF+M>|vk- zs`}InVZ$-W(*w}u0D(ozkx;JHi(5b+qQO4bO><9(^kwaIn%tvH6shg3rGTU|Q9995 z0)ZQ(kA>{_ud|i;tB>|=_7&}Zev%6VBwVNj<`MH2*9=KrM3>D)Ar37s!l$qCKra%) z&SQ}s(|it{ljaQ~zs+bsFw0?|;HJxWujIAJ<+UodW+1&qRhn2X!A7c_-VMcpp1`>* zrXRz)m6O9{*GOKPrJP>4wBC&vOlkE?Awz^;gLI+U@ZO-Ic#+}DEd zTz|fVrcT0R zMKg1zd&A!jJ*21*W)H2u2u?G3efP=tCK1xpSBc$RIJRv$5p907R6&vx`yjO`Q}0yKlO8GRM%JV$ZRvB6H2`SZh$ln(2LO7g7a#1%-ZB zWvbH69rt6|=F8=0F@nNYAbG@rw>m;?c@u`5Twp4&CIh;XYBcu|VJ)FiZGy5O#pN<{ z`T;vQ;mOieD;*y(h9E?5gx@l{j65So?CrLT=fu<|P4rp@;r_&s>0b|%~grRi8#ZpFB-x|`@R+arMTa@y~s zSS0og$VMU^C4G}N-|!Y4q0umk6Ic>@M_{RWhl`yO)lHa!QS-sf6<^|i$~SvSZ3p?m z?N2|%QgWBJ_!7;9%|Oy7i}26Go-*;tMi}@K^?}hZ6n%|e4&R2AKV)jfM&yo|P5I&J z(TCN*X38NRU=<53wJ|mXj+Gx~NEb6=i5q$Bp@T52avY>CyD##_oDrx~Oifsa*H!S> zB0MWze#VQ;Us#4;9Dd$mArTawt*n|v^UMuH8x)sEd)sQL{G3@=UceZt`w3zW+J*Z0 zRSBEg1CE&@Cy%4&7#jL)z|N5Mx-ycAS;ZYr@sqewJkMx~}C$iYxA{4(#Vuz6RD@D0v}pV;=O z=tqN7ZT)TTdDpWIeua&G!$KX%j@yL{c_~>U>ASIrc)e8uEWNDC>P3rlQdJWCUDKPx z<>r$%fu7DqRPue(+ori=>4X?|tk@Tw)q<{G5N+dcdo4WLL?RW$;<0_riIzx936}N| z$dias>>64Lxi9oZ^YC4C4he_|!(V zW{C_v#=Zun(O^^w=nEJ+NY@$jcw4L});mI!T0?>aX$ZA%vpaS9yfW!B%!v*u5i6AF z?7B>&h4?Zv+jMRiHS0yp2$)~Im2RlH=Z(7h_lz34L6v2FrE{Rch!rS$aV(hb06%7& zVQ%l)lvvb02zqfCel#}{B$ge4Mloe}WhW-JAwi5_eW~ntS0{Fy(O-lwXrll0@^l&zh8C38MsN1jtm^Iokca+OjyK1(A;wUnBCcb z+>o`!hvZSW4KG|5!im(7%a8DQm@Ccy9H0oICsqN)IS&3EN=|@J#6`(IZS$qj5(V#t zzwd1uhBc*I;@-HS2%ekAdMfim|C6y(0HW>-JH9Idma0q2#HKV#>^b2XT3f-uaN{-; zL~?fpo+q?55%i&QJz+@a%Bf>z(clj|pGb$ImIo{%>Xq5`2iu@K^^!Y~=q#~8?>X!2 zjoJkk-v?5@3>&ddNZ??ThG-l4$B=`xB0he0V#^UAMB(=Vej~{hclBGD%Ul*yAoW+- zxeu%%J7|(XBAgO3(xFJ=M58Z}k_hKAG>O4FQ)2}}_$Nnf&-=~(IgK-NDT|02v>E=80>$zwL>xml{K5q&EPS-P zhpUv$cL7I*^VENGI@~y)L>x(1ZTvil;GScZth}YDj=iEdkc&))8Cxpb>$ISj%ns3@ zTN3PX^3%@>Z=@E7PoFM1kuj^;krp@T_?2Fegd5}c4T`B!a>fvWQv7z5Nna`SxZgZ* z?g&>>U)>e=Y3J?o*Yq-$jTH>EW!Ye_U|ECcF|XX0y{!c1!pbHTO{{%f8m_g{a92G5 zg*-K+)45vrcU2?q8FqScy$1P+Zmbm-Esw1RWV8;OCsOqwJIpVnI8*su{bGDR`XfD>UAP%M6|qF$SR1PN}F z)a$-ZNf_$FJ@Jo{Ffy7XH`0=uR3vk#IVgYdu&)F*2>hg3R2GGyK<26ypP4mxTxWR* zF{m4t2sXs67VGU;8_4fx3jt05+~VxK)LJelR_K!REkXI+h5)}9V{nnv6K1Pyd#os0te)oUnjTrx&&MN@DTDUd)EMyd$o9%c@2}6W2&J( zWTW9)y2g?))3DL%$JqJ6=N;SysSNV2EPY9L009tVZyPBq|LZJgwQPT!&w7zTP zHh{YaHm1=g|7nH%vLCB<4iQ1$IdT^ed+(v02t+G(xsBf4L2&gayZ(w$`skAJbG6t9 zS&qDx=8RVOjP8g95*VJkbIbaBZoAd;{T||8;+Ft;66(=!U0RWO;8k*@uQm?EHR5*!2YLjj&RETc z`|J-)Qa$srhpw&TV`FuNlcPeI3#Y?!@G@XJ#C&t) z`jPhmuak~`fbg86B!aV2#g{jh-@MuNNo8M zKogJQ5GZ)eD=|JnWGAM-*b_W|g8a9Z!lVftv$jONNJxPkYpFd!e)A;a%H|n?D0&E- z@K_96LJU~8zM&H+jNhj~sP_1eLIKdX*ZoXG_5@rG1vtEJcHXh@UJ=_v<|h2%=+2Hk zwsTmD)s{#Oe**yz!Y1>_j12v7Fa$*WMX_36C|*ukgC0h&MA^+!LN7XKl`99(|8n;# zt{{O%8pH5IKeb%3iB-=b(UWmHgJwMBvXKMy&AoX{dMr*Nmf`LLH{}TLgF?e@k&1Ht zMe`$dCgUHW>9}t29GCTy{6I3vRpL$#ErQn^)~l?y)jn@y+CFdh2k);JEb;=c6*$6K z8>6(j#OE#V*EFw(mEw;4 z=xunH>146Z?WH>kO3XG~mvH9tfot^|z#+GV!Wr~*Mp?`A>s-#e@kPfiQvH7NB|Fb$ z2fEJM7b~pzdSx6QSA0jzG;u&9?is5QC-^EP6%D-JC_-p1yO)%KgqLE|Ny`sP<)BNM zTJ;&mAaH)o4Tfu*U^CMYK zvuYES&6}j={J#&gJzM^kvrIBtR&)h(UEB_31eE3z^17k?xqlU!v{dkeD2xo^orkxQ z_%_{GM2$uoX~(>^SXSPJvXp4Ev?e$Loss zyaejJ`&^6pM%%u*C+TR`lkj$KpD>4~3!wcW-Ev}HqBz~PW0$}*KZ<=P1700pQhKmO zsP+BjbiB0qy_)NJ57o3d%cWt`T)>&j34^y_oTyXhboKNZs@{sKyy{U0;Z>6kV^3u>$H9?^gavg>%O!DRWC@e-z8F z=`^rlI71ddoQa(Z{w)cf(bx`RkLxhg2#ISCMf{Ip#KgpQ2$2E%niUHR=;=$FDrWi$ zi&;j7ZRbkn?W^N>lA5jslE4`wyYo?Q$N|qWyB9U9^zPTxXq1MKQcYwN^5FsFG>x8s z$U!Ur{5wV-UHbuy33Pd0Ar@{Qw}AjUEYKlOwljkSi&+7Mc+}|LNg^bC-6)pR7B(-| zxSsyI!vF?v%X#f9;X$fAejh}&m#TSEK19tH&A9mWf`^~&vnjPuZ7I}Ot>)@DAg(-! zq{a`zb9}wkae~W-b12jOAnhPgS#BFl4*4k*G1)cFuHKjXDYL&h^^a=a-^{>B{ZJ5H z%yx)}Qp^aPDGoBTKgo?r>mb~HL6wPc1v8;B6vRIB3bSr#fCpV6W8fwh2=(_iJQ@i7 zBNgRu;x#diby+;TerL8mF)}DlZ4=ty0a|(;M+ijiVV?_u{MxWS=XDTje%OzDQ^;r>^#kHp?a}%qM?%U01kSuq&2` zX~075URdBH_{aV|?7B0tI_Ztn&_IHPI;FR(l$?l1!RFXWGb@eOWujid#vIq9j8l!ecqM(v_s(wrvnQ6MQ4uO0_{qA>=`#uMc}m4!@T_V_awi^ETIGMx3>; z-NiC=&dkh-lz@XUoN|PCWsk<~EQk~jd_jqs`L54PI}d-V%cykJ1k4R-A+s_*BaFeO;YcG@g_4G!Dg{@(;OG5 z2V3bo29WdaPCMt4<#5>Y^Dl>Omf2lPnW5DqZi4FQA;684c<(FVKtHXbgXOT=B)=8^ z1s7-rgW+#u{S|U|2&RD=q>--GPD;vlUXLKj^xLlD5ctbY(JYjGYlUzpT2`cght;&= zd|0tZFu16#D-kAG2m&UJ&`a51;R$O9oCDvW9JQLan^&dZuY6PZ}uDE$|F2-JpZH9Hxx9I!Xq0?M&meeJ`eXO` zr|aE`pXHS{s+tyg=L-D-OWfUr^a`Rlr>?~D5#G_0bMz@?v-=4+R+FV|p#6+a6K?g8zhzTFapDvZ;#TN- zKbzbxHw?AR={PT_EK?drMF}s@Gg~d&)z6NiPpn#2PVkU1p&0bRyGZjt&nK8(WVRk) z^a^_Xt)&`=_&jhPb3iYB``k7i^PKfGAUwXDvK(>Bu*uI*CSvKj^+akKCZ^SZ6O1LP zGLO-><0HScrUK5-NTTFxd(2eOcx`BLQ- z*ZzYAH9a1E1ScfO?KG~dlS5kzt%P|34PGqe`) zVc8QJno8UExd#|=CR7C^ed|gUyK%l6{(6f&FsdtoK7LYOJxEIrUpir>CWk&UbM&K( z`|?EpOH*Hb+0?0_(^yI(kzI*a*VCZN*DZLZahpif!rR|E(b9D>rKia5DgDEq*(KB> ztac40WO?sx3whC{VpLu))!{@(N{8}GBIROO?v@{Kk0RJw4G9{s-wl8=(diY;2(Y~*y|JE0pl)sLGE?@lgKlde@A^c9}r;|LFxmO=r;Y=Wfk0g1dil#Vm*5{3CMA;PJTN z513i&h5UU!)a&lr0`a{Z#jspj>hZLSm7{@p?avfUD^9B{NyOXemENaA_~&0Py*^A@ zpTDwj9XLVEHsRxyM)m^fD!kC|{Y zWiKw604=NGJ)q5i5bc6t-=%ulH7ZETz?i9`q(yEOitp$)NdDNrW@&qWygnZAh_wlx z>2NB;COfp<=uDW-+fU2bTGS5_dNP8!PPn*_;Px0|n7Y2sxHuL-AaPDxbzBd!(&C$& zPcyTorcQwlSa@0TqlY5%CeT~jB{YAItJn9VkxkGyuUpA+9~-9^b{I$VTyWOIO+4v( zp0nK^3h0^Iygpi+Qmw{&tywpxA#$w8npxI_Q7na+FFwq~5L#5_ZrCp02OkFP9@{*J z4dpPV=suWfD)D7}*mT~jw$nzNvbE60?Uxr?6g_gR)=EOKyw)L^rXGe_LoI3OS=Z;$X`i158V8;Hf=kfOx3v|a} z(-2|%+%@6Y(C)EWHoKN>XJZdxpLP-Cmj+UfX|_XEpnsuSnwRc*nqXuGM(U<4dP@5lB(Y56f+>GoIkFX@&F z=0_znL}<Sf-h>^fSwKgNL7(USl#KSvwl#eBy*N6@RiGS;>mSGF=TSAe zUn^sx4-oA>L~d7L^-H>**FPcICBTs1jgnsrfnH%8;gJMe<1)b3!^?vB8R zd%*MOs>GR4FHWEO>}u250x90tj=NcbG@f@as^95?9Py^;tFRF`ZF(L{RA~$mV7c4t z@pV_^@-snt97~@EZEBzA8v0At3B#6uv1M-yEB2ifeW*il6V;)JY2&7vXv zAz;qmf#G%-0u2$kdaTz^awnMmY~dLoGA(SG({)~0P(P4K=OUpmXjfQAC33pzsVq#^ z40k8`+YO`nxakR;3f_XZTKD!}hO zOCOhzBTzr>4nQ(~uWCPE&67*F-bG5~#x0tYn_XPBtX{~Qthf&m$ee52jposmpFsd5 zx%*(&+`0>v0S?{o(wzMJ?`PV&!zHS@%Qpv`&5-nW*w8)V%l-K#kcp% zMlmMuuD@S(_!k`z*B$D!bYk#pA;+ef+aKK*E7Cf`Yc63POg54ztJd>`5}EAtwmVkM zE<3eoD{8=Rk$JBK`CW4*IB2Y3>{o0&JsGmycEX%CCbu>6D}McoYW;2e2jvb!SMJy{ z9SGyW<=KDh04HHQ>P8yCOId8urF|d`6+IOCly*yDD02GZAUe5AzC+p;Uk}2OE0EP( zO2IP>WU7wXUBe2?9U6AyoUx$De(yV;XjZdPpU)>#mrOqx>(V!I_$}B3dub|#%MMDV z=cat_p2{t$XPFJM3ETb`V^hGyQ8&{hzN@o(oNJ zYx)tU79Z9foZ^Y1cM4Ik*F?A`hod(}?8!@e>YGZ3~fKQ{h^$>+NC9J^6JT*(Wsz z${+zUn-4J`=c^&2UL2#5MDJkq%-xyU$V~pTJB4{)FygLJfDv1&sWvMhRbWv5#MyYF+dHNjvb?)d`r8JQZYDFxpSM3&?nT(j!T;@TZ&K}1wI zIJfoTsRl64y=|G?{cB$Q7 zk+WZZQOd0{Z=q#7rMjupTVU%vaj#ff`MM`3=a~n06M!WPu@?&L-GsOv)a-@oct|db z`XV7(5NRF=nfV3c`_z#)?jrG^9he@-;TBwUz1k$l`F%+e5pXKeh+z2HCel2X@)f+T z)ua`}f&~n1!e*GkWfO4==&kKALX_scY!H&6LRsBwQcLSy@#)DAussZLkD*@-$3U8x zHvuR!*}r6|bKb_yCfZJ`#t4v-qlh8?Fdn_{zW7|}$7B^9N!R2;N#~20!=f{*PYMfL z4d_vBNg(C#Wh<(DnT!wRPc~$F8XsWBqL1007dorq<&0zBBR25OZ)HxYjXW}v{)%9J zm2;8KMF{MAisyPn_R-g%LzU*O$@PO=fu5Lm6CCIcce>#6*)lQ1hDq*H=YOb;xfvu0 zqcSW*hmNUiS`C-#4q^sDPTrs=n{l%j>f80-5Fuwfk-!1@qxPWMcA@F2Hdv8ehhY~7W&d4^1<$NXlp6tP6%ST}6{&=ClaNto~wVSF_gSCmL}!9-Cd;a`+u3gU`}MF1<=w?uhuY za0wNczEZOtvc_7|tNY>Vn>Ddp4a1W$&5^B@AugY>hWAWF@4~G3Gf?mfw zNvB-<4P{F-JkMaPo6YfA6{o#ewsS5^tWaFgk%Wtzp&Es2foNZJy_QXDn3!&Xj?1Sf zD(pNeB|#I=Hz@Kau#j|P2+_baHN5$2oFL17a2T|eFB0E_%)?s?@zqDN+r#VNbe-Bj zjnE}x)A5q;05aXVN)lrGGwkOUYhdsg;La8Qv^$J}14B%!v!)l45Tw}5^c6la zf6fI)8Cqwao<2gh$Wqa4wpMH?tYL&Q!vP<2y4?T`dAcxiWq>Au>J|Zu2cH6#TUVf` zJJ8o-?ZQ+H)W^c$z7%Oc;+$E$wd=(~&a2jcp*q-DjHhU=wZD=uM1>SR({>uKlVYy1 zFrLklF|~haQR9=H&7mtdb$Qj1nh=QF#3e4;ctw+EvWf<1Tc;`(s+oG&BQR#+<~yR1 zZ!FYNgY>+M&*$rxZb%cM($;TL)^=GjnK=H$rTqw-b!$ z;qr?lVgI;F1O@cP&^;7*Bf0x5muhT`19!8$cm0LF*x>kJtf4@eD%YrQ5Ze3KrGMRM zS@|kh43ANeu(MTDnk)+>s0pE3RX^>)P*XF{j9C{YgYC#2et@KUsHiDv^)1RB&oisv^%Rz02spaV)pi&=f}Nk*F{Oo&0w{LKC)G z^jlAN0}cJFTpD+h?u6U2Rr@)+aYk}f7NhC9de`K1>8Ydj%~1SV$aGot%MeB1>2I9FSE5}SQf(K>_`kMU^!Ni~U2aY0Y$s4t$3OIHz_cp_PJ~U4& zCt*#C!X$SEOxcWfy`^`1R&%lq?^l95xWR;uxPjhPy=Xp+N)@IfL%|>9qTG%u#;FTq z0ukLQTP=@2Aw1_&Y(kpW+-f02*!vMGPg@P$+6k4dBop9Kfu&Wgi|Ge`uS8oLPHvba zuKNg5ozR=jn&F!t;e}kR0dRAnsAQ>uD}>hJTRvU@X#+#$J$smleL}i1eKkU~X7wr^ z)VddsTL-@ zfjY8320X=zgB7UtqN`Ol{L<~@aZQ^Zc#ePVhdwtKTqom{Gh1!Zxgj5aWy~N}H`3(q zYIOw;N(c9X@)`>oPRR#cRf8G9kA)A63gg7&jeF)yNPQ+sk{a!yoX7#3wyXY{FW{9Z3PBxaxO`1o?fUjB)VxNMlHX+Ge~2aKdU zoej>_|JJ(!giPgG;5}=8hWn1v3BCym+9!5=dGSM!MuuxmrpeoCXvh&wgVJ9fJCxjD z1!gQ+T{V%%z)5?yr;^vg1hbGJ@Zc!MCsVHrW1|kgrB)7ufJpB*XWAA56<0m4pWfF& zhxyNlwa_kiSk9q7ARmVsP4_3J)5cIYhn2+8RugaM_}g4gwy;)vb1@ii29yZV z+Hf57Oq3A6;y4uNlzBU4WlE-BBI4JL@A{L_h9?O>N@~+1Q4cb@`Suf-5A`5S{X)I;SDz13{{GN}9vfi(Mu*p?bc zy5WAPV+GIB=ze?f!psg7ycc>g9MU@`>*M$Xf(KHZOiFQEZ>h7Sb|UUomuz+N6vWI1 zhjl)H1dwf(>HOHYpOI$AHC$SifH}+o-q0-%$EGhv@J6!?igsLGcBrzXK(0Xi=vYHD z(5|$=dnyJ zaXL$1pHof-D z98E{3*wDb=JI_N19)5TuNDS^0t&Ygm=BOlskO=Pv7C-ngBXzQ{^=tngr4Wxujq}!~ z9h~01^iDS5d+sOs92QDk5HAP(MIQf^({oD>T7}Kg6iydi+npiYJeTO&4hDP>)3I-8 zyYH*ekwJi~U5N;qx694~oUMdLhvnq@QCkgfc2>UVGlhaFX-C*im!vX%Q}nDUh?LJD zaI5hO4!1sWTft=^`hRFT$MCq?HeAOxCst!jY}3DxU3yiS7)>QV>m3~1k_hzCJ=;J3oN$n8)qS(MX_G(EB@ime0{k8K(o z^^2#R5_@=}WZ|1p71k|1>%75Sy}EI{WsJ9h=R^V;qxh%U#J~wRA2~y9=rU}Rh*!1Z zFY;X#hlH7sTCkb+HqqhidHev!=M>f)@=TvX>&021ZL0O6t;UvH{Za-nTiv?Lf|qX? z>mW~2HJI%7BLADXfyshDb{h=y&@D3(({~0I>4`Gw8Ij0PV98ZoKiZqY85J4d1n3if zd;jvU&rk$1QQ*l$JuK;`0|FfNmrm>O0hSqbBitSxcC?kiz>2`g452_pd{#I(P&(9= zOmV31buj#-ZDsxtd*RDM#F;CvUbur@Qz4&&ARZ9W2Iz!&rX=Z-Y`kmCC<&(3R05*7NRk{ zZxT0FQ6n_B)py+l{=?x)fZuMWH_DdlYq#y)C{xBUSzmfNt8bTCq9aWTOSqV8N6`w~D^0XdByydP z(dKZut){&l;9C9IJ^n?FOdg|b6fGR(NJ2?%4*Bla@6$q$g5Byt)OxSgYRWjWs>ma- z2GCDS^hWMt?I$V7lRVuwN&z}J%NS;n05h%~1&;*fVt3pQ)f+AHYo8)%jg7WR^7~Te zL&m0`E8D!$9&!cH>TdM>k}<_vF0Su`b-wj8oZadtXwPz@w8BXbKKdJlk{mK{r@a7)9 zwBXxTn_x#@Z14R)l)G*bp+&{OLu$WDCk+YXHdbAl@4ZAilB1wKo_ExK(HdK-89Rx&!i(>WE`AhjxL}@>yq7*w!(ctsDahBgIY8 zmf%v_9wZ7dSVaD&=NwxhNeSfg2!F4FKrxH2IYDv{6x`=f-!94ClR|^@@wBJs*`QRfktIe_4C^uu*}*+1F*yNvOIPZ%_JZ}*%3zVxKAMbdCfX> zyW7C{8xyT6j8T*}8rsHpyDeKg%?yUG^NPoI&VsbZ#VIha0DiwYJ^qvn1H~dM`QsyR zJ9qPR&>hr;JbWeZ7flwrCI(M3(8yXXIHEKOuB|xMwC({xsrQW*v+*ZYg=z_-3Cn8g9$u`a)1bUQJu$7(Z#40lRP&Z2Tm47 z-UV4KJYh$cg||wO2EVlYUQA`Q@}!Ep{9w=v*BB&5y0&wdeL^8k2X9!WI;GyY&Cmq~ z6&%YvB>3txQfkM6sXzO?+8-`$><|x9##p9zM) z;h{Ob6Zd$uEUly2~zb4yyf5x{tk71q;}Kp6fIF^mvcnAMhM&%>G#+;PNK7=9l|xvi{SZ zsaqnFTBuXn<)L<(`#zeEbjQV-g$U_ll_f`vg?lkLGkehCF;4|fHoD*Ob7~kb$}1xZ zGJMO`qr=eGiu!0}3L-~o<>40jZX(oFKz66Xbe#D&Yv%jmH0R8F9|&Bi;^;0TiR8bO zGbZSTfaGgRhUZ=4c_??sDo~jgV`kti55|MN!l(Sh`%bMe@;+8yf86);o;pCT#ZtYI zgLdIa{VQ9~E-pT3A7>a!VdP2sSkJm}Ft8;2syHCR=xi7snzhH!`?C2hQVe}O&SrY* z9!f}?cdED-SN$8> z*t|I}JxZ|x>HK07LmPu|ojN;xdrR%O(jmBzH4Mhz%c(9P@#lWi{jMZQP)HyhB>64! zoL&dC`V&dZ#k}K$GYy+`Js4ZX>7$s>p8j!~l$jk7NNGLEwOmxIRy{DTnpDW)&CR?i z*!y|L)AIE->}NcSwg=P6!i_NQ){Ph~Yk@FIC(56%k-g~N<0l^FjwhDmUaunEX! zVfUPW$o;YZc!$@jnFL8#^WFW+K8mI-G!Y3FaT}1m4dxf;+pDe$n9*)$8!n0uEP-R| zox`_U2T1N9Z@quJMT~(472)qk&e0aR-MG1AIZpaG!gZ&HKN4b6889Uc-m*C{Tw7+8 zXG(Xn?L=uTXr}W9-3q*XQC9(Os*Y9n>NE zkT*jXTK31@u6${NYDVP%+@SM7fqUK=r;*bm-!+5~YS*T1VR#QCw>-F<7ReM9j8dOz@Bv)6Z8LD(J{8883kG4K*cw*E3y+N!9PH~;GmDknq zenyH}jzxEKce*e!@{v=ev>fw|0{4ouaVi-$gv=x2%`$GI5pLZFli3U_G9(OQbfkxb z(Q~ZDzAool@0G?UDt3C!RM+svP$WZVJE|6aq!8z|_mKk)U1AC-pfGv_nx4MG83ahw(RB09??1(o& z7RS|2@Mci&SyuoEwyYgc${j%XbU$@&_yP~UWQ^y8%BvoR^i!g#%^sreVGWC9leCy3 za(a#>A-}_Qmfg9vf&XgDrK^(ouY%(lbk|N!F(Z^AwWnB%ieh*Thu@5Do34Y5qtacE znD9)q?8i#5B)9ht)L}UR3h*z7X10VRmCz6+UvGq5P;uih$}sK2VAmU}!ANXErWj;a z)7t=h6ozS-+xPttW7)!tIY<;&&$~Z`Hps-4pZ#CF9C9HY!$%BRLJ~iN21f}XNE!!y z?<2!DJusiXPcjWvXUB46JO;%M)W?OWcOI<9dOoyag7`n*9~xMYfx80Vh>bVreZ;4G-u{px~3R_jZf`M7^4$DC}lTE$~#k=Pio-*YF@JY$Y&&!FM1_KcNb&1|tl1vzb}WTLG|H zVD+U+_0bjVLRR}{xG^mIu^EGMu^{$-4n;@D7{t+X#@|RkmOJI14M^1`{l>V3#`Ogt>vfuz{E$2|)>p9q|5*UOH9uf=i_L3DW$E8#*@7{XnP{n9 zUQ&REs1Q27u@5KQr*2%TG?-`M{}kEj=#6`-W{}IEnRHOu`uq%j%A)@3^HVF8Mf3#! zp;AWtXYfZ`5QZ(7;$qF0$0exPRnzyTiN3DnVmU#A4qz7cPgPi}b_Q86u`~=|DpVu2 z03ifs3+3{#q_I&hmA~UJDJatDQ3Dqs`jUTH$IG<&gW9&iY6x z!-M1BH}9mXo33J|=;mv$TZ;6_qf)utXEP0ca^02hmE)l10l&?wCkXRJ%> z2)tRFc9m#imCe8QLn!57Rs7LW&!R%ra%%Y(^oi?VQ(5*u0l8li>7o;?g0)LZ zq!`DQK+(alAFzvA5rjL?Ud2kV;zbym(EEBIXg)(+lF@J-RzvbA3qka2c4K)LC=zr7 zReFq08{6Kh#an7Pn9rP?8XFC;L2B>TFO9dRD`D*M1%DrQ5lZZf{h=O6VPFt`RH~`w zlz|V7cvPulz#E`-6aMfClg!2Y2`on6GXNhjBwaDdvLvmy5@wM6FuRUYolP~}?i%QK z?AclYDK#DqyBNiu>~mk%2}4^0D7K3WG91UBo#hQSQ^k!3p;5#qA9Me-&Fk?|jo*&} zHizwF6s*+)342Ogbm)TZ7Bu7MWoO%%ZuQqEU8@*(~ zfT$W06pD1m@onRVswx4xr3|cC?TBue&}qv4!EEuEztt0exEfFqHkVh_W!2{vw`FW% z9|-4(Of*CDfGbUDeiLi0gjn*-xwZ{vr4}arf z<^LzOwr~|ojGKFYS3^{P3h0OQh?AH+hm$tuy*k*q%m6-&EWaEo(P%eZ(o)$RU0mmGVC;jzMWVA{mp?s=EBf}zg44inB0!WGG%;qm5)IuP#`>h~2hnR1i`CV? z%SkhCL)PU?5Iyg*3Y}!prw^|9oXW{Qrmo;|wQjMuh1$S{(om-`DOoG+)+4Nrz{B9+ zAM-pX7<0CKX#bL5jsrh|&y!sqyFSck+{|`w+-T;^>0Jkpxnr3ud5v6XMY;r0(;}t= zCq7WL@M5=w%92uNC$W!e5K`5l(0EHnt~#)AI3{d z6)Tj$wx<=&+EU}A-z_@u3geN7Spl-=89*&wGaa|5ALGSWriRzW^OC>;u|FFA@p|FZ z0rn(@c>beYwQ_o=Kg~#s@j*9eVai(1lrhBxGv`Na?*QhI%E61s;1dm+CrV4(A11Ge z)o_XyoQxzl zU)+QxmC=e5K8fU*A-E1CnS*4VQRxZal%1fYKDk+;DCPyKQ) zZdZR&rBboqW&7>mZ@GC0&BIs#^1m>a|BjEZrv#la13Q0+k!qhL!fMlPfn_Jt1f||U zC|*xR!B*u+FoUR7CPqO*=*3{?Y5$I;i$YsLbSHIiW){^sUQSKL)ij?8 zd?x_p!XSi~9&65yq~)$ccCh9?6$)(xQK>CxSS4IiGPO$_hp#0F?C(p&c})qdAh`ax7<%eK zaBU@(6iA!ZWkb0(=V!|?DR&72`Vzodn7y@gI#N|oy|w$Il(Hx}hzeIwfkwv<%#L;A z;Fv5Gra~^rl2Qc7cB5SLp+%ok?7Pw=3YJ3S79{v)Wk~m&!5e{knNk&Q z!dL)W50a#A^_uXs;bI=z!{}HC>J5D|1@dc|X^JQ!3#opJ+AiZhNth_fDu-;uO4!6o z$o1-i#mI2?VR&YmH$C`yUaXQbtUoTI1rN;R_6MGd%2}hP&Yr|V4CJ++Ml|HWs%nta z2)@uYR@X>{+-nJsoN4*seiRmj$1PNjZYeRiw`51Sj0Kkn+20wAu!%2Y`z<}+ID6%u z7hoh_OaA4&l(;q-6Kc%oX5{YJ7&jHL@HE#R{`YciaK_KHO~P20L`2C&oUCk%MH<{K z!is>|vZsb|r8R8((wUr{`fEKU_g@ zPL(JE?P^MF?UxD~ulilUmpM6yS?;E)dGjx9XBohd<=i&nbLI6gZjYGRN%MtVw#AlL zeNeM=ep-iXwLpEXXUDEWCE!|y%%0v}q{C2~dxIv`=_|P)VjR2nu>3)`KS^^)T-G!j z-f!Skv`udNp@Lyv77*);$-oS?G#zba2K$+(Z zne}Wh3i!f%>NYV9!+fS7&MjYa3m&{c6xc}^#$^H3Dh0?NQh;qUvgI|vnzmW38{3l8-WO*%by>a>1L7@aeO# zfVtBX`5NCJ#06(D=_1l>c}}k-iK#5dc_s1___S26axq^Be5peE7r>QEjeKu0(A}&$ zS9I+`%%GIabiTJK2t2<>&M*n{>&rCopvL!0qBV6>PEkix{tH69ZqqNZAN+;?VLomb z%np}%IuKMY#b$(}bif6ao(H7`1KmbejZ8G3l2oXfOUEh6z<}(re$7N?j}&S$BXn(z zgK{#%T4qwDA1)dDn%7t+Kd%|2HtvAPUuEm3ZJIRc5eDpJh__hZX4@dZ>EDOFU z$7;ZZv%9DoffBzzDC^oh04eUM+(lAbj3I?N<`+~y=8q=~`xAw+G-q%5q6DBm4=#AO z`qG)OBGT9eQ&MPc38^Cpg=atL407$IQDpA!a-MTWeB*j3c8{CqxDr$G^|nt4 zhzFItROMlyyH}%T+>{Y#+mbot1t;+eX1=Jj^sqGCb!njJ!xZ9>!-(lsocU_|ZDO4<&8lMP)zDzS?ZisNt0?cq(@SzS^YNr|asCOZg*4 zBVxo;QHrPLowmiut_ikCQ|?*1(hDq^iMfd)mCFJBU%ZzE>+1JJi%LVq#FZ8uUR| zdTQx;h$3m_4CsjA#jHh`-J(nxrJCvS6~p{7FY;A0o=K@u{NI_$aJrXeX|<^0C_KqNHz87#2Np=_Qs$1c&B!P4~*Q!8}PLYq2*w!O_Jh2 zfbIerlYDP*MLe~0W(MNNO??IBm5k$;UCFOQFbZn@+u66d>U^CUx zQ`yiex^U-D0WpFoc?G8&L=kLzp&c}@D^fBug70XY zN!8jchg7-jUv=pbU1X;ct;xk0GfyWTE^raCmYt|(%ub+HGfm~fvLB7Pn~wQJqMyOs&#!M6%&^V)7ulXnc)HS)M|D{k2g0a3DfTR!11 zV$PEXTq~c!pG0N@<%fy4VGNJAi_-xia+Cq>gyJ}7Oo}4K>SLLzEHDdiIMUx!e|FR^ z+Zh^8jcRK)0lyGxGN$z=*o1Y3QJNZ7!`v4!s8jn&Qr(wJ3YcRmBc%6X@{I~bQQ7dD zp8UX9!qv?R=uBZO17Q*afnl;n;h1V_2^Jd|k(sa448Tf)h+At^} zdL&IXH?k-5-4gr%c5sdR|8_7p#hI4Hpg&6-%+S^@uHk{BwTOsIFE23von#w_bXJ4O zc?K~m?|?$&x||$u?3MrjemnTu)|>B9IlgfEH{+- za6E?)H2iH+L=C_e1D&pQvb4LxJT@F5AXK@a&B-N~`;yK5MiIbe}xnaMyllsF!ePevBsJL?vhc>JsQF?=~K^>eF<6UM^E-7pFd5;}qG zMlb&iB`MR~b~4gkKg^R{3icRTOvAM(o5#Dx_a)o;=3=F$ADqCCZ)~9L@hm9KFP z-*5`CO&(XIB6YUU9fjpKh9DY6TxtqlYoaMPBg7q<7TOAlXVatcd1YAG>Tmn`6x5zh z*DbF5qLxm!VyRKb6)+zGtV$lWw4j= zwmM-CDjh?Im0yH{;Qd1^740KC%Z?zS0|#4^>X0#B`&@gOu>81LE_j}Omh`N%fdMzM zM}aCri^QG_YF5HluBzS`Z4a);m(b3J^lx@uEch_c;@GRATspYKqg;*jN(a>73#sRX z+^9PU&U``X&mbJTzCmEiSswg6$|+I%>y0UrhJGIMl@@=KHSsh?F;1r}J)B2iN$Ss5 zvF!R6<1nK%SQ=~X*OGBQ3Z1IGVGnlw8{$%yIOn&jLP_hW1PtgfkGO0*DMSWp#86nc z-~Wktpo{N`w>vH8P950_z&{Hc-=>9*z!WY=B!p0BopYH1O9csa8G|M@*<5s7yVt-NSPy?w42@PB?pZl>3%u0kk z_65SyoA1GJUK=U02Wim=WG|5{75T~C(Udt1A=V_=E@aM$O&{rVO;i$|suZcBlJRGG z4s%0ic28ksnQb^W88C$yhLs$VHDE<*M3ou7n*wtzejtKtw*EABnvfoee1CG#5KJrJ z;X&*|>LsF5%vf%9CYZ z(-r|={@8geWR>d(j;D@9=lRc#Y`gA5o@jMd$c^XTh`QYH-uAZwM>|*%FlCMfL9c^_ zHj+tEXLg`+;4HYxoaTEhuxr16N;X*X;T;YuzEal#pom2i{f=R$-@KYe;V6v#~5uz z^BuR||2}42wrp>qlzK6jQH!ILc+W9Cy`0zZ%vUIJVACpFlCQSgRlDP+;)g3jKZ_&gS`#7Xdka}*aypZ zoM7M6yag;u;7n8eRsEOf;x_eg{J)zl(D?bxW16g6#p{sL*b@Nf6kN8U!a(^KXu2rTv6*LGPcznsZk z$V~49TV4HmC=UKBeFqjaa>2sW8t74A`>^HW&*Rn+Sid6refPTkDQvD(0qMIkV`>Ff zH3Ve^C7@Ry)*bBw{&Tcy{V(m0Dn;I_R>R>sx@X9lZC9>MkG?08XkEW-g^y(GbWRpZKs>p-*0&e{b{c7g=&T@_yfU%5o$lf zC}#-!l^0IqXK>T4l)V@CsuHof!hz$)KsPv2Apjy<&^G0g+;$Zinh1_}LSEb0AGHzu z#@$Xt@Qq5~3pP5#>kNb!*gCo)VZyg`S{>CBJOJol+9-_p^UVu{r$6Mf+U2gELO2qJbL1av2zkN2xM|G8+|hK=$- zuqm#NA`IAQSusN{IQ;W=%P2@Bd9kGu`j)C9JQ#WkR%E9-G}t7h6u~#}GwVMyyMAZA z6>LI-8&ivibuhkzG0A`~aKU;9ZYAy}aBjr<;^9b0bCZ6%UgUcPF8z1C?`2Hbq=NVv zKDND#-cZoe@I<1juEqE}!92z#htQa9PmYp;qpXl%dr2DPt{%CC1vHEm2+Vp@hqdne z4-0JdDn#oFzg|a*Ox$-|wdHb`G8GLgd4aYH`R$k(yTA?%N3KHy;yAJY1RE!YgRQoq zrF9nZCNP!e) zMeWCbzjeO)v_y&#o+A29fwdvyUA+xuflJBI77_~-Mv=OMMBH(Oj(b|jCsdo#pn(RdM*e}D zBnhPPp``r?>04J}#ZVb+r9k`a0P( zOlq!#5^xe$KMQ#%^7i4)nd*}zunT@G(F(t|Zlkr{oUc&88coDlKeAw#;?MOP^4k_3 zw=sEfqjZi6c)2kH&=wY$Km+{~r6eg|sj-V|NjFr$8wE)g5Vmz!@ObOC55|Lrr%`Gv ztbGD2v&k~>lDSuQ@{dT)yF;yw*jo`lz=qjMQR_NF>*SHUp>6rp%rSNaCDo^AStYbIueuvsD&^;zlY@&=*yOKI)B-f6}3SI}0 ze$G1h-JekV)oJv~ zf+IsgvYt($u#$!+^?^VqHZxxbAaBUYbEbV!>)<3Ncv;qGF?x7-oS+vkF%ce&+lcSKj*%{1f5+HY?!wbxsk3 zhzo3wdIq3{TLcTH}lwzFsaq^ zAAqGc;$Q1H;ajxr7K}e# ziH{~oXe|-Bqt16j*FSnofkaSn$&+=BR-|DU>m5f5v*P9^}2Pa=J;FZNPnUS4=LO0eF z)>yx!YMl(6yZmlPk94~E`@xakkp@k`83m@*;E%%pH(~MTg4zgNQ})0R@hdHu`d1~D zDFF^~B?P$Q6t=q=RrKGLxlezF^1Vg%`yvZ*$1yct%pE-+J5-`aU4b4uIuA(>q0xDh zyzv9hbGO-lf4-t^xd9czD)ZcsMBM#?;o>q+VeP*7UZEW7HCj<{a3pt-5>JmMHBfg# z-V^qn|IbJh3Vn@wrkW&Xn|}ifYU6Ik#n!gqSc5pd0yy-tc@=^D%yU+Rok1HpcW|8_ z_-CRF_6a9bo+=Y?n)!Tv)+ffP@&57HbzI0}hNv+|tREqjBQ}j-d61EPijtFWdN&+# z?>G3LDkmUt)%vd7xUVd53rF0kUAGThKR!Zzt$|@*1-LT~V*@ zhxZ=2pVci4LL4QJ)f-FxM^(W;W-vsOoPfXw-Ho%cm^RKo7v&U`PoIh*2a`bou zi&q`)CQ({eKA(m~xWPk2f^jjt^^N*jvv6FRnY)%_TbW(Wxy{u)m%dEj%r9G$eX*;+ znOB3Avrnfr(O@y0>5gmCHFEtDXl^*xi5gY_l=pr-5BHVda>qm*LG2AkH6l2!onx9m z6@COfA{=+-vtbl|=>}{RY&rKI;eB}$ z^8PSr9JKm+?Ib-RX$+Q@TmH~H=B;|nPfn=%Sw(sy)K^)U+CyF?fK`QSH8PPgV($xA zL9h|CEl9|_Y-I1)xVZH7P89X+*ft{q;Lvm<-H8d`A;K1vST2TEh)E=3>Kpoagz?qE zX4Q2#e*!Ppxfhte{{Ua8);}3qfD83oE`@qonRS4)?s^%OsTag?$h-6hGuMf9pJ)EY zW-j9cz#W~DKG-Il_wa4SeJerXL;~Kxr#Jl9*0SY=F5KxSY!+)4y+(n-Oo`M|L!}n(;>hn~H zoGvb0c~OgyPoLKXOH8v<9kC$BY&bIXemUc)5`*Mf8acdmC|nzypWUWI89-8BSrHVl z^hQ<}VRkKIA!jUw-6G>#dPPi%fAvVQ$X&!xn)4bai~V&qloIo)7^Wn+jMa%?NeNob z`NdCYeNV-c6>Os@6z2Zjm!n5)?oz@4gDlUReMZhf(#S(MJ79q9s3XP%%3zJ;tk}uZ z;jMoksw-%`9y=#d$^<9~M+r}ymi&#Nay;t{zGc8!>Y)okX(8FYBxIN-F@Gza9UpbWIii*VKG^%7QPyu9v+-p@V5f2jMj29ihwAVamWmpa5&z3 zbGN-1xmio$djAP_wK_p%QlPuxy!&keph}KUlryZP!T)Odm|0~ zgmSibLs1PDY$&!)=MjZX>LAG;D3Hm_0`^x|ib(nE<={RkZpbCBUW?(@ufyL*Z; zR&%kC!e82W{k`|XBq{H_?D+IH(9fG<(zHZa<|w)=r!fQh6ha-u+%^cO;$-eTx#q0H z%k65vqxq&ZLTF3==|A}r!YzUmV#2HsB1x>aM?-;8xlKIbkdnuUL}G$BV*`hohE~Ed zWSBkAVW>)b22r8>DJHU;b6JO)cU?BZVTTyj1OW}=2YnXbzazT>Vn(-3YTOeFE%Zy zhEV_1Fzrz?e1>B1ZxcV?@O|_=P0`_a4_*96c;d?jtiU-1>>^LS=sxB+8D)pkj%wUE z$51;G z2E@&#)F7Iv~Y3v?+m!-$YAskkGLU=S7nEi2LXj{4aH6KmP zp71x;nZr~z7i0iIe}n1|$%$wd4q^PG_X5%Q*KRi7`RbD!@)NBCHdw6i3cA{>1qH(l zhpflN#3Nq_K5jHxty*T3>U_H}htHxP7-V3DOKW^OGe>3Qcl~m4DrxbJiD~-&g8y&XOn*+ z);$W)$bfw2dRZC5+bPl0%}-LXPfH1&w8suC*Gu?sjq03w(@vZ2?9N5<%X@R3le}KU zN^N2%QpQMIkJQFUQ@DuGzDP=U$`{Jf#w=3hbg-zI-QPu(C|TRnf7JaTWO_+csNcyM znz3X?IQ$ZXt$2!*aH7F7SWPa@6tfPog@p8QYodtsVD*S5*^N zeo38a8plCYYE%QHqx9Umf(Gk{a7oUh3sW+8>Y`2*Sp1?l0z4?7klu3uh$o8OM7&yn zWSWCFEdNpW2{_zxpj4MMIMY!R__X(xTp6)=Ad{GJ0sFnZ?L@&yXL@whhC3;OyKBAg z#hG54K2>#=|JnXTQQ1I-9-N~+4apZMo>QZ<8wE_eiw{~#cf*A{I2$NOqCJY8@te&& zkkeQ!&@{+tkvm9UuBWd@a_@x{3QsMuXWFr0n*8~731G%Cfa3=<4-U+iOpr6;r6Bi8 zuC&nbKz-J}m-;QDmGOkteJL!>`bA~+2fwnuu>}`ut9atdNWJ(URbg6U_9~W_5se0V zeoX^bTQ*4ad)5g=vjKoC?nPJzbtZ4hLa4HhhKo4AXjmo;3PBl0KjNTj;xnb2C!6Hf zA*}cL-T(-*t{o5z47bD1OLMrjW7Zuf5wMNPl;8aZsbxd?bHQy;7bHaco*NwT)Fr!a4(J0@7ak*nF5>=Gl@_^q-XXZ-8c+FSL-%5_gE|(uPn7)ioh-`6 zk5W^`4G+8o0YS+`8_IJeldHN7{&4s%4sXns&qKq*%kE+=20(ndt0`2Y0{eErDs0Nh##p^!hIwDAP9mjt z*o9N+ZI-iA(HV!rG*u2YkP-r|Qg4Yxxx>$Li>p_XJaj=)NR?T89oAk4crW_9_G{MV z-hB&jEdWsIw6Go@+c#z{W) zAq>#&pTA=RE}c5+maJLq8F?+lY|`~-Icf5+t`D9pB(f%5zCiv__CFo)lK(yb;905^ zGFt_#;$ABEp6_a^s=54~V!&fIPcLoEzqEN@4lIy~E%kTkFTin1MU)DOiXHHhQ1x{W zNG8rTx~2Ae=ejTV5RD^@l3}I&!Pri$^Q&UXF|`p_Xvvf9#~)h{Y|MY<38=}*+;WD; zsOt;$z$vf4Nk{$;&CC$p^iqEfMOX0G(b z)c#=Hww-fq)_rnXOL7KGFctF&RUL7*)$_?Wq}ZoENM#n2aA}G!^V1uSkn;sh;g3gu zR$Qxl)T{kg`eJ+_O8)RH2mmWA@4=F_L$X!(wtLVHjo-Q1EZsRub;+ zwQ?>41t~mOac|m!mW)%V2QW$o$HZGib!3N1eS3rIArm0zf!|wv6pd7$pquWQ%@JTM z%r>Q5?Gz?Vg`_~_kv42>VbH-vie*3PSi+Kve#>(ArO6=B_GV}j<8$0bJ{js?BnsXf z&?eN08mUH+P9np_Ato76ECZ`^fKD)PPO20VobkxQ3J8+3&))VDEfBmg*d?%zs+2Hu z7IS+zA=4Oz3hwWemA1E{F;7@^siPvX%uWpQFA}zK9IRa0#1)Z0jN zCW{b9*cU8coSL8x)}67rLa{Ooath5b4M|yyxRDye)4bFAUe3jd48=LSZeiQ<^Tc0M zVR-J5Y=C^o;l-;L^LrEkTkSqQ79%C^1SHUrUgHZNnIOnJUOqotJwd|qc`L%z@|@s5x|v0@izYlnN9$W%}+mWCpZ z*zU~6{=V1HB;|r5JN(8g@#6=H5G9eDqs25%J*FULco9b6&NKLF>Pybx2aT4XHiPbf z`wm}OOAP2!Tpj8`+#Dmrd2tJ04Vxl)IqChHg_T=FQ|SeSXwE3OO(iQsX@9!YUPR-; z3mE}*CyXUsznJLgugVul2}=1Y7{LFKRiY7xNeW2?@>o#uF`C&Bs-sD zp-}av`L!2ps>ik-!tZjbA{G1HT+p4}$pA}-ECKC9>6zgYWPmK&uH_zuWQSBh0V6o+e$)^UDB7m9kw!T`srWz_&DQhf*NrZv$xYk43GfDXCC&-{B znBij!e=INHgU(zCfwmcTvENU#yWm%%R#RxqenhBW*!JMh?_Vj-e*BRCRHd2MK<6Io zK@7wVC|h7=e>q?L?Gr#~Gd`C7l>QLrcYIe16X4$x9VK$-y!b4t*XKmpaQ(@1T`gy; zpXwk-FTjP(jBoT@z`_mqJzFNLLmPvN6z|*H7jE@;qb&q1WZCD~%B0PlZDE%tg@ z@YXSXZJBtxpI5O7@>I%Qa)kK?E|Je+Hal&uaoVk5h`>%a(Fp8KWOEEN>9THfI<8GL zIn1i_^KC2NPUi9~PZW0da5^olR4@85H`>fna(sJmm@g88mhg9&X^M1T#IvK;t9=}EMpvQ&IJUs;OA$)%LTnJk8A|D|~L z<3ALKQ;hgdw*g-&Xeo1kxlN7Ek+y;FDc?o+7Kd`*tPht6o z7Nf)`(#m?pj{If36HP?Jj`|3}aU)`CVwx_^(p{V(c@5%h006rOXF z;M&*cJ73YCZ=3wY8>}OT*O^zO_5#5d@yst0=t2(sUKezM;0HsbqlIzkx!OoNfbF;b zI?lH!X0rqzEjX2IttGB)f1cUu-&=ka{;v;5!vrE`qW_1cvkr>#``a**(%s$N2up_| z4FXC?H^|c6Al=>FN_R`c(kb1rOE-eF@ALi5y#F%H!t9*g=bZDo?(5Dq!vYxx%nIAD zgN|^Pg=cjAu#nxh<0QGB|iESZ|wdUI=ar`9;RI}~k!<8uT{XXw%G9@zAP{5hXr z0|Vrb#^Qut&3VKmmhauE@P10N+Ev{e`V=tJ*SmRrx`=M3$dGsdpgeMZ%b0B!<1Y}g z_UEYa$^3uMR-_zTzBekr3vVZUE`C{gWeS3o8fv-$=0oFPIOahxYJ88y@HG+x{thVM zWVG9DKgf%TJ-y3mJtPZomG}QS-^eatP|DMGl06A=vyG}mbhoLQ7vPUAmJ5hx zseHPz-tGbzmPx?L{S8a<*%2ITVvDY;_>}1pr6jh?l51jL$h)PW7YUkVRuT0|`Peft zj7qihbcNw!i?w?6&MD8Wnn15gTxaqeTv^`K`!X|LL`OV+$zIG!2 z^$+trMx{@b(`~7Rbbq|0s?b4>OPug-1U>i{(tS)O|70+>n6>+|2@1dI;M>NdvTW1R z5Wb*jX8p@~LAOEZaX8UM=SBDaJ${@}kN8#V{;$7V{3lh7adolOlA9EO6sPV)5NHX} zgOAR5NG&$K8hz6wp#`X=|NR06r*bOOdE%3rYT|al7qvsx^Vbo@PY-f2BsPFN0gWFA zpQ#3?-4up+1qfRC5Ar;^AO0lAPlS=%TWb~Mo!k|VHRLt;$C8PE1$W#RMNi%e={tZD z^4~s{t7LOMK%6aCH(IJlDsM++T>1UKuyf043wJf!wdQmGZ>_C;=ew-pG<9};z!DdX zxOWg$h3G)CfApz!@iEQxJRY;(-03-Fs|X(1bhuqu+IQ`U=-SBBOCLb0e_2?|W6tW1 zq@jv<3sTIj^tOr5)zKnHcHD*zBSVJ!bm>}k+c+U{k^EPHeVFK|!fa93hmzmom|&h{ zO{S(YdP>i7Z8;1oNbS>#-0QGW(F{cr{kmcJtv3kO>aC>7niHx>TXS9MupO)Y-tWFK z55Pbvr038WYxDg~&`DX6V!k&nzfu@WV6Zq>BqO05-YQrbWp8j1rfJyOoKIxp38cT#le(n!W!6h{i|MJ^-9@f)|g;%nbNd7%fM8o z*J#6@AjP!($Wl*0KqG)ApbMgeTPUTI9v9mU=11FYg`yKFirE!oP@b?(6Y6rGR(u}0 zL$IOC1yvERqv_W<2I+#RK`PlJi)2Sl;cmhBVYhX3tKOB>45u`a+May@!=1M@GcWXx(4h!Dfh$WD>s3V|LUyaF;Xj; zF1N>;ozQ>0kCK=CW}q&t)9u8b*%%jY+^*;4jz2h#(!8d7|C`Hvtk9?+h?$XXK%+tP ztr;@5{{)5e2^;HTemK`e4yr9;sm-zpsc zh>ryf-ib+gAv9PKg#t)kwzEXBB+z4C{39Y zvXQ(&m>MOpDLJh2;<^JsNc5WmX-Fo*og^IL+Q=PJ&J$D63X**0S;z_K`8%^y*$+U$ z4MT04^Pb>pp!glmla75jr8i597q$Sv57j&X@8qQDa+&Dt?(-P=u~0jp}-z&j+_mny*>MEMit{FZ?FN zFr5Exc+{>C-kLoAv1uUN9kjR)r(V4-5dT5U5pWLgs39$ha@4%m#EU}dn~)jCDIrqL z&Ot1Kbtl$nn(-(+TS$RozuKf8Po=>`ka4EA)v8lvBLLO84r}|Lm zA6>!|+qMS|?L*Q7y(H=5CZUNZn|=rtlYZBu%+jY0k`Y2SLSxs5()Du&gwtmp z{~YBG^fLMad};B_6zIhjwfH8TrzOQ%Cl|&;BcE~i=^wDI$`#!XMBteAe}Zv!o;Iul zPmDBCU>_Rl{@1``2c%fw{9m(Bi2Du_F|s<1-5MkUzlrKDp z2_jaw**GO38g4hS-j9Ut44TI!Yx(E`fO1~;fW<0>$Ig^$p4W&qJk_YiV9mXKFJ4lj zbvBF~t=ei@1q!f_5GW5l@xRC1rFaX6wjJcc=5MqA5s-fsHt)_ippPj!N{may^J7&w2-?p#KS|3*C-qXu}uikE&rAVW|0W#Ges+_?@EkmPitdnrbZ1@+>%MFdw z7v01L%c{iP04x+maW;TRAAYmEjwO1UAxdd6>D>VLFv`bx2Ou{Ld{qv939Uk#p2;B_ z@rf+-*P8+V0zzU&gF_xlRLhd@E*-{FSla=+J6Cx1>3V-C`X~P{*Wr|ytnaI^Tqs37 z;1^%Mm!74oVf;V}sNx2B5J=?UMo(c&+)MQ<LW4+fR?D|<+iwGcNVSHw?YpJA+Uc$jJ<&1#!)g6u;JquSc zZjvilI>?HI+a&jk%SW;2fgMYL(n%v+1~MSu41KR;brZ431m5aty4xnLrPb_lq{WFG zWeM8St+|qLn6IJcmaViq7`aSQzXq;=4T7gWPpKOYYwgw6SL{u^m_f-6{=Ig}>E-1V zuCedq?f4#^X7iinAqawh&$te=CECMJ*${DljrbGG^B-kY7X?~=MmVyfy6Nwg1<|(y zYhnU8&v5hGuY3!p(2kwngC9uKyW>?8yF#>l3O7aiOW2Z!9Gzbo5nR6^KxD;^?^IKv zmpif6?dKu5=FpRSC>I`q{w_aD&2h&#-N3yscK)Nxfjh%ZNyp?kL!d}brH6;RR z9;cHnIc4|JTxm2o{i+{cyGMV=g{*kZqyYzx$YFGD!*c-hiimc)tZnG&#OUAZpOdBO zrjIXsi(8Z)>$CxM+rIxX;?8~VwcoFIz}RB%98*_pqYZUT&gs`8GTKHlKkm-gb6$mA zh7U+e{Q{FNwB|rzTeYv9E8xb-x}g7gSv?>uC8Aqg}I3hKby+uYl&L7b-KB zDCjuOn65UxfQdc@XUbGb+yEc<9Y&1EFcNu_gd6d0!b50?)? z+s4RTR`U4u7WarP^E@tLHHG~HcBJlN>lMWVY?@f}!Yo=^T4OX5Z{mV^T185Wk*N33 zb1gF#emV9PiNz=HIJ;m8UNxXj)3>LFf5`vZuj~*X4@sdT{RWi4P|5e;CH9B z%a_O9-)y_2Dw3^UET`b9jloB;?s*lm*7fpuMu$M5MemGNz9z4;VY1d(IpX-Ka1Zl6 z)}3n?lbR$7C~x$NA_^ZakDX7A8c1EVX9N;fT%NSlu1<5xT&;2Rr>(g2!8zK-wb}`- zTGNwm|Dj0A73LOqOgJE_(LXdvlcIX*X)9gES_eQ43%o&R=p7A8V|8l-KIx`8jFH%0 z^15icJwQU648>uj;q_8Eq89OTxMcO`witGrODpVd4o}qDRLDMNiMQzbAQp<|dS5IT zo5ZiY&kJbpE`V6rcZEU!t5U+;XU9&5y;vV(ACI3K*YCYONuk%{TtkEA^(E1|gKvUG zfGYyb!mB%mHSF<78yM7{;E$;r>sqcdw%vKSbnaT>dP5KLS}O!oWXd=C%waVqHFw<} zI;mAPQi_2j1`Ebl=xF(&tur*n5M*Pbr-R`K)L&$-06Qf{NgM6#=Cuq_`yTW&hfv_j z`WoyE=wrR`o41n`KZmhaG|Hy-n^>~NpLoo^+bsP8F~XY4GLJ5KXY3pJIfc_+M89V| z0|^4t?qNT8d}o;VIZANQC7MEy`k!yQdB$`)j^$7pdF)L^IJLjLh=buTh+ray?C#22V6VWPWJ=?xjqPH|K|2TCp6;OfTaW6m=bW9Vu#P!ns~=vy+HVg%=e4w1CgPXh zuvFNaa-;Ax>!IN0MjW0kTY&Unq2aMUt9>HFY?sSPQRjB63!tRYH)(N4JJmykUH9;k zZuQ`@yT&UhcI__{BXJmG*3s1B*@?`ZU7|vSGl> zyLCm1Xz~OflXA1~KTqdm2Q#@f>(~wy)2uvzd{we^UShuQb7W@J=nha)VtU?n;Lh)G zr|+dwsB+Rg>DZ;wuZ`E%|GR;x_b~Y3r)%EQ9B}Cn@y7-fU9vYhx&qpRRiwd2L`Fmg z9~Z0jw0Cv}@3=t@{y^DZXuRO!h>aN=$}sGUq|l>+JCQcEH@y@2hxkS zJQbhtUuN=oC`~tKNA`JcQw(VDdOy4{SKU@L*>?t%8zip?IqcW>?X5%HaqLC5%x)JxY5mqVBhO+|axRERz#jjTX_c3@5q&e=19vo_CAexT`-0Zh&+}eu+!qp$ zpx>PnAvK?Dikad9DXY}t{duI>j1jB*q#+lFo=f<;0EnmX9F{QQO z!8D-9{{`pcOB@qdySp^fH7a9^5ivRt$db6cRh^Wbj{c7#c|?oJUtgBA{n8mSl+wYTwu4{#_EyK zC*3UIYv?x|3P)PC>GM~-5v6ABCR+k|SAS>4Dld_fYwXH5jvd(@9=z=;wr4zGD)%BH zq9KAM)k3R?6U^LA8;ptnuMPGAf)(DVC}Q926gF{#V1Lr}Zxcm^IX-%TJWJFRXPqkd z9m+0*sGjv!ivZ13{iE~fMhOvjLsy_V zvmA_mQz0@90j@vv4@JQ}#bOr5!ST65H}P>DWbeQnwP=SlAF^sbEI(X@Q5mRprS;x` z_TOj&_U)Bw%SrTo-rO33mCX5iGjl@C^5CB1y|E|fuIwT5z-vA6ZgPqzOO&- zf<49fOU*KG^lXP|dqmxiS@il!gzV=&*Xi1?KSFy%)GIBh5>yWk!hio_;Xbr_<#(_~ zKkK@w>70=1l5slzSso#_)<=JTjhJV9-f+X^{Bh)McK$b>`c-?c&XMm(f9M9BB$@8 zvwy~V=_yS-@USD5OCKV-Zf~f`cXC~*A4;#Z)0+cd&u@n_s)LCWnU8kH+Fvua;C~a- z@Ej5;Q`Ax!B~oiiNEpbfBlx3GX1m*;ny=94K7|sFd92znlWx~^9V@zw6o!u|6?UjR zoOimzT8D`AB8af8pZ0FQ)4tE+>sr5p;;Uuh()T3r!|~P4UQ*aw5YA%Os0Xkv^`4ld z-BWeH1jRMp*BEq;Ss>V_o`>J$Y?A!&Y}VJ7+8d5fCY(u zd{Watl;BUIf7F&#us4L#oO$%M!L@!z3F?cI5NBmwEVo9;*_z|!6hzE>&&i2G(blt4 zF|B3!_PWIdavKsr#}z%}YUo+GH$>LK2;wd!4~~{!eMolb54n;t`o4sI*v58mzx!ce zp%AN|&$F&n&a3|mUeD-0aBU4a?|X7?xs@=C*%zAXx|!lR^3=8T#lBo+u1$R$V82A` z4YF9Q%D<%AQMp>I#>!h~^Sde~hU%GFW20=&ZpEaf9-DzgEyAas$-*3tc7%c=f^9?; z>O!8Gndx0Q$VEhPq5?w#dY3{bGp1@M`1XO&T;h^hTX3I;xb9uor@@8?N znOmj4`|=%?pmt1jhCE4AHZxKq@#awRUQ`?c#~JK{`O{eo@#E7-V|7uqw(m>eS%SNB zQXqMHKR@&*t+TY!NtiMg12Xlejv_X12khEZaOm9-r)FO9jqUXPHHiD2lN+~%3cbNW z_y#D=Nux{2RhIzg~)yVF9*om4k)v|OJ&(Na01P`B`9 z8{<3wH~sANOi8O@*Djq)7Ln%_$;tkzwnngQ{rqlKX0D{@Ng}=62+?jmgpxg%ZEJk@ zI0&-__LUPRNM{u@$qRB!?F~Njx;cmx&%rbb^?-s-ECy`D_`kdG;#_h>QX;leMa~lo zZQS#nP<39Ia7m+FBirX+`F=ytN1P1`UvpbCa|^eT+E&k-u3=fCEvyd zvlDmEp(r_WUw1-T+0Tra^5#9=7gC&=U7V6{6sa*B681U?%Pb$X%wm`ht&Ou;aORx< z>0;%2PHwVRQSVw117VHYiT!m>?k;4sn{xpnmQH^s1Oc```bx}^NX9CnoM5RgwJ?WRY|cz6HBsK#0JRYRw?O|04` z1=q5G9k{}jMW$t%@Chn+_2F3Sn?IL++$)-tLP(zNJ>ZPSL%;DOk64H9mdD}PqOvo~A-3BU!;MAD=G%#QXnqz3wz^*t0s0d;-e1MJ5 zV6}E_e}E|9ouC@!>`O^!pDga!>2}VET@B(h8hLU}w3xOE+dVYNks zVCzr|vu?LDX=efjl%(AsPq=K-*A=^n@wxX)!Ij?=w^p6|Q6b}Ao^oLE1`3!`R-#dT zu_EXg%3!SoR&iF!Pd8;!$>Kp+sFBQITBKC*(GicFacv%6vE z!6w38^e$~F9*)1OwYCrhi!3Wxqx9QR3L@G}{CN|MsdejYSwUH0ddW)pRupb}R4N)P zC}X{)kP0B@6(HoJPERp5>ORYfRTJuQt#Ig9?JuKB2=BwYIdjEPHrOxyk_OX#Z`iS1 z+d9NG8Aq@Zax;%n5*A`_C`8{w9ht(?Ok-uqSf>$!!WB(KS9 z@nQ#lXjVli#*7!M_Vu1u?|ZN&Y6jQ6ZRU52D}FsZPN%LZE{FAx@(+Acsnw&_Egz{> zT}3*MNORI*KKaRcV=T6jd5pFo9E`F}5ol`L%%YIq>>DtH%(qdybgkpfXtkDQ)UVpAqG=+Eq0_Ys@BNN{d*rL7S#XH+9=f3+t@t za{oyuV-aF1v-dnftHoYi>k^;+>tF7EwgH7G;pvU~6>U(5etn(xk@ODApT?OoPB-7Hp<<_Zw-@;|{ zUoNwNKZW0Kb-wKVyG+rDL5$0YGXcCfkZCOx5mg@jAFmAMGY7T`f~eQfjBE13-1Zw} z2RS}fn+Yp(aESxs;lx!JSEsn;Ol4Io~wjSTGlJ6eJi^yg+=Mp>Izr?xf*U!{X zRrgv@M=ALC4K%V&75~T=r~#+R-KOs>^E#>4(TH!eZ?Q4@?xK*-*ZUtv68rDMbP#MVF5Rj z$>t00-=nOi+HBVI@HhK`;#|3%jp}6tjkFp^P>^K=5=sffT_kPsZ?hFOV80u8Re_gnLNy z#u7<2V2AlUF0(RX+P6%zp3VrwYNb~ulF^|w>^tQ66MiNN;be_ZoQ@ZZJ(cm$U_OF3 zU=l1EzwTt?(4vyNR(6)ftW(RpI#{XVB4#md#m>$o-*t-*zH-xP;6vf|7-al~h4tWe zZDxVX`cJnNoZH))JCaH5fMFljsMOgpw0*L}Q>>FxA8d3ZR5rVJ{Q=isg5Xl~!~9ss zRL~YAqQBHJf?W)4dygmYUrvaQ?(O@jKz8*+XBng?C^Vw!) zFtEh~O;3on4VGh_d#m8jm|$|z5NMJ?DHCHL^FjmS-GV}x%>-8siLm?;w7iVTvA=I2 z7A$lUsHr()1PvGsGIBOfg~9o#tQ0l9=l4dw)9Y!==R{d{a2}6AptMp7O+UN?aXQ<2L34tSWFlR%MG+JoVdkOYS}lad^Ib0M?hniS5vt7&vbD* z6T$Y|hSRE_L7W4{1{AKlER#{^9U|>V;VA`-pF9+fsw+SJsH!t0s z{$2ei^YK<~1jt^mOJEb=^LjnxToTankdzXy;7S8g=Thw$hAwm1L=qAW465sbBQv9& zLW)`r`3KVir5n1|VwnwP_JVxFoY7e6G$PK)lT)Wtm6B19l=nQG%p?(?;~3*^#B7!q zno8gDh3sYGqWBpi-5e3jwCS{^puiXUvJx|!DGiQs=>~o zF}-_eO4deNymNT~MjK0#_#o@p&B^;-(c6p)fFrEFKkpd${Og9u{}7{x zVCR<IQaEYG}>n^%Da zJ@RyEzfqTnaC|W;h$v9rTaYsEQXxlv-P&aq8yo-oGC>g1DZu;1j5u?B>213bm%yEoHo7p8YZvkT!lX&Yu%f=;{|{Ztzu0LVNfx zvq@mh?G!O;ShC~y3D!Xz)rfs5>n6Wha@=P^Q$`VA+qBlIbp!FTrEhr)Y&!4edYE7r z-&#tsx*aqddLS)KLObuxn3jjDEH4!fBvZ>|A)YXkoRgFqwc(%nYOn2}7?0fHPQI9> zy34yGd*u`Nvh!sRr~WwNfdHo-s8p=8lx;53?4z#Th)mnJff6*C@aJLHjLYD19@1d) zfY7#*bFy--g{rwk$=267qE`<`2EwN7o|=CL@(ag{+~d>&G4<40QD)Qc2s=-si1FJQ zWGo{&zkPDGJevg*tq2a5X8M#cj|=swV-yUFDVkLxDl&{req=`W4>2E=Q+XgUK-#6m zPafdr;sDlO@{?1R6l%{qju*|Hx%Ssq)8y6O3VB#oN{_{wMvIg;R)=Gaq(McOPM^h_ z6GnW&j4!1zU$v|(in;~<@-H`xeMu{n(s^%Tfc)|6oh94J>Rjr#w{(q%wDENxGx<UY*~AyoRvJVq5=&!o%o!dfnIUGO6*IC6R$Y|Ev7tboikn|as-64GqA{;7-? zfamvRwMg(`dL#|6Ju4k;MU~6|PhNglk-$H#VF-e~vzW#inwE+c3NHMNXMp=6z$$(j zvuFrEO1;hCiX}P0{=n5rE=^-7&urr=)-<3R(Pegi7^kfZ(DLO~rZeceL3Aa%@?sJc zde?0QX27W3hzUY$`8~YHJt*AQ@}8z;nxd%vQ|Ye3CY&!t?iYuw?xr>!Tru5;&rFe$ zk*e*V^drhd6ve-23A5#`&lPm#*fNlD0quEBB(s5R#xpm2rQN0ogP-SM}{!7Bv@U3UMeYrNgv$=GUBzAWbQ={wtO&+ zSv@f3;Gi}gTlW+%A@T*yXsEcQdcV7V~RXp_{J7Zs*awtTBK4kE$ z6Xup|W;|xr0Xz~Zo=PMg9UOKz3KS;nBQB~?#xbwIVS)N_m37aQfiyMreE0J>5gg7y zs@sb$_!F>K;*);V^BJ6E_<;;kTOtbAZ&?~rXk`2;b8HunI8?|$-1zaMd2=aAuZD)5 z^e!Sb^|8wMGuaw^a>UO5&l5Gp_Yvo-cv0N4=!E^#7SZk)_F3OdmE0^g#43MsHuVkB zxVxP>JJH276r||pqqZE9=KPXnRWz7PiONAQX{u6_8B+k|kI=i*$Ob~N#xv%8#B{L21+eXcx#HUw%M*A9H~6%e@DQ3wn^~f z_uU84&^K^z^q@=ScBYNx{DH}>OwqL@G!k2>eDz1k&mKAtmAj@DdQ_^uM3YEjSxyX= zLr!kQfp2;TBIXHT4iu^bERZz#7xi9lTcOx7Zueb;)KPw2;p^!n46nrxMs~(HUBe(; zIO;-Q1!XCkB0)RU-gMcDdd|d_tZi$5wy>~q#fX6_5SGms?<*fiv}psABkzL&SpT zXmgRFEXSx(e~l5g^Q_wusQ>Eo&J*$iQ}{E#*VW0qypq8xh2xhm>{6T)?hiuk*rsotu;i9O zReC!36N~`X3>d7FuxfyTNbaEu7Vf7kgdsU;5?86D@V(*LLE=XDvL~~9A(u^3^5ycl zg}GSbU8Dc=Tr9`Uuf-f(j?Bs3dQD?Fh_nTCBym4`b}J*Z(K5?&GCR}WpfeM=d7A(N zB?LV=U35LGffy*QD~(p!xbz=0Q1_TgYUV_0mS(|qJIG}~Ji!wNsaZ|qGGNOq`wsG$?uUvns}b=H=Oy-ZcE{#tDTL?{?K&|I0A%?XDK^!ni@jc#5|uA$$h(~oR7 zyV`9wI+|x1N%~T`<^{g}X5qJg1~oqH>t&p8bP(>_5fx-z7My}`P1@D61Qf>Xr@Dm8 zynqPE_@jaH!L6uSAkbL$Tjuqv;JEvl+oH6HUSS-M-IC`_OZ6hL8z<%dW;9dy}D>>>kboo)jIQ;>)u?I$lL&zmdMO-87qRJ z6E?=%QB+jpnt?*;T9CCe;pd_&|i226)y2@dGe$s*{pW-^c9 zrrsnzTmkTE_8SD`gz+vQL&4WuUG+6G6(w{*M_#nyL-Rgrg>NkFfNPY)`%;NWJ!1A# zwwqyK989Z*$(t+UmZUILmDxDq_jkhzOXASD@xTy4c_1uyqCYfxB30YK0`Nd?WdrIT zD#2~oH>U?z*t@UfcKyZRr74PUne<34->B*Xi_F&qoT~9>iv;%FLj1nQTTKA)Ew=x< zFHrPV=JOZR9RB*;=OWX~x~T0Ly|vXF2K+xs>q82;ajl!8BWoX`1`p|GXK=ds}x-u`%8 zIVoQ?2TEv$pv9c)Rdu^B+sN@Bl&@GestCOcr}EFR=~UIb0W58qfDTp$xt!S06Od#6 zm1rP4Ty54sZX1rxj_~yZhYA8wE!U%l?iJ%;Ol2UzG5P#(Gx@fPW-H*r@8Q4bOFJ;E zQ!f4(Yw=E=lu0x^h;HlZt^V&b9=w1opBU#C3}Sp5GZcD{&bVOQf1iC%m+RjiOw7N} z6X0C>I&R+wU#*fYn1=l;dl`i)PW_ySR2%F{IP1QBGcD=x3czZL-HM13WCGxV&F@xS zU(l5zYUA`quH1RF7?Gx?5tx$>Y;Jjt2S48eJ~1@))jRN69y5i{_u*hoATDjT}(89=S2~0ICqjA?-RG`G5RoN zf3jH3>^1hRk|!L5g`u^&9JDNWVgr;hj(SL5-GLbSuK|#Ydf4jSmrcR z3cd^Ja;$dx`v@7b?*yG*)LX4WZtWUU+6SC!xTV$IhN}X$Z#WF1Q>|;M&dc97BRm*I z!q5a}0P0kTY8`MVbGiIhms1Gui>A{LJ!`#q-5*LfEU4&|W7+?OHqWC9}5saZEp8 z0MqjC##1)qwr-&G`Rl)Dh`!5N^FCP>mdA<(@qfbFF&eReK40iZ16PenoS62!A7v39BLP>^ongMRGg zflP|Pl7G)P^h975_hdgGZ`mgVNq#@l#pd)n2AcVE4cu{{w3Kd>O4@fV&$k%pHENAF z^L%o2jBRP-P55^?@VRRFz~p-F!`!6v?gtY=t*dAv&S60ly;;b@@pmNj#4usY+~&2c zDC9fkq~z*MozUjKg4H^c_oVs)PReKo`h)e5Ibq%`KDUbY=Z0|$(Il(l9DN+C+Fy5k zIXu6B1e9elmAEoONi!Gn226e}<)`L^FVt*2jZ?X^V}5OUTOTlo4_uaq!RQuBtG64 zMir)kBj=4ae6L^{bCfa8uy-n4oTZzIDhEC!+h8kZJwa&|+6N6@?;3cIj? zWIs`kinq4swFAS^DDHmlnN+V-|JDG|v&}s;xa_jgaDnSbp5E?9!wNAe23J_cUw+T; zZ@3%)7g%0fOUuYXzbbQtt`xYBK8>VBUnlEASnpJ>s;AJ-O5wnzE@B7UV zzM9x9#=2Dj>FuPo%Z2#g!7#C>)5av^JX>eFy8p)MChxZdwM|aqz5ew_bB{4dqfu%2 zc^un@{o+?fer-&+h$i9|{JY^(D-27~RDN<_Cd-uozJaF$(wwvStpv5cFO9$rN;mz; zBPgMMlOjraMjh|MhGnilD_}kI>F^l@L`uxjHBl#L`jRhHs=oU_qUts2ur^!I_CKF{ z^h5(x)O*R>-pB4z z1$`*oVpqGP^$;rdTgsDsrfW`J^n_Fo{S9 z!fYl$k$8ix;!5{^a7_8iD+IYnZMvBFNd`3b1xqy{y`_Zplk6;i?7N738XTybD`SK( z#lpFnJ_c{GAehz~PZe+DT}jJ}bS@3@QKIv{aE@;!CAkVg+}Gw<&{agtf;rbF5oU3Yv|ObDsnS%f3@Y;po2Pu9uN=WnjZ<=?zaK8vUoI2+K;;FdgOm~ipnu#MXHkk)0p zFFa#l#>OG=CKsK%Yi1&cS5u^k0-3mGcuM?2pWhL{MDxo`Xga9bH(u`6)R0f(D3|!R zXljMleED5~*h0od&%aIZFF+QF9dI8^D&(G%bK@6pxAVpai^sOr-sXlx4&oa^uTWY1u;?E0a6~5%F z)~n&=l|Zwn2y(IKEFuZnXuOcMrGm!)HI;;vLOiJ$lJ< zqi18YTXBMrUdz?7v%}kZJ2C4S8h-GysB>r2o0jkG)f16haNr>q`1vjim&SQjQIk{; z9IYmY5nT%HUW$eY1>aS1pgwuN(t~;>3HBjT$mwXkf(7yA68xn>tofV>=NBBz4`Zxd zV93daD#pm@*;+vgmsjfYOn0H>_s6V?;6lEv)(Nyyqy|Qp*F}Q&XFplv%E}4ivb7de z$X>|Yd6tRic9swicLYn59S9sVMp?K%%(sOwx8im>%xw6EDxtVI!HmdR){1MY+6d)a z??i0PEH@~u*zPnC$gkk5st!FDdzmKA=xWeA9g>s`%9PdWxyHC^Av8=VRv|+_0L~HeuRWj)G0jvaX5!hJ!R)*3cC|KW*_}u*&BZ3B4`$ zZs@=$gWRD_{d6ZJgji>p_M-W39!X#ih8B{E>V3i#k;c%XE=%t_XR~gjKi@hX z6fMjozbo zghK6Wyg!}Gy_UHKZmfPFakE&nYfTM$+*6YnR=l|M^}&m_#(y0Z%>6oQg<6iv+NVWu z2Hd{Vy$Rp8EO;Nszwm!Von=s5UAUzoSO^v%Sa5eIxVyW1a1z{|5FCQLy9IY^pmFyE z2u|Y;jr%>{otc_nU0v03_NlY&-D^Fo#Paf~9EL3FVTy@+Mxa>^X)!%wLvpGSMMX%qCd`vb>V z>0a;ucZ{pID(~!A&1Ae$m_>W0o^$6IW+lI%vHnV^-3M&I!YTpBtqiYWVcqQGo4^`) zwbp@vf@dmqS-R=0P=VZ1PsCn;hh3h1kI(D8C2=F0&PDx0$;Jg51mk7>!iBPLmbFzl zerCmE*MGVYsMBt9iQj_g9o8oniidNLC5YrpeBth!YLI?H^e}~==caC<<_LCE8sH|n z;RuvXjYZF4nIH7a4eNC_F#OsB?22|k%I>maKV%#gu?(&X1N;p)O*^VMXu}L)X{$iG z^%MVTQlXjQlPKAvMfsr?N^0=-3>~m*j}WD^4s!e{+*XvmL#eAc<)!4M%OK<;q_;tE;RZp#n%3YUxB1 z*?byveQjq;Lsplvxq}_Lc+Je66{=`n%T=!L|6VV^%P3VMhh-XR{90})p}8$*&TPpP zw3V@6QpZMJuC@J5ue<MgH{*)2ID;!R>#ld;*rkU9?kv2PFR2m{+8$ zraSHM(;{BSr@$JJTj%wLAyNb@O?9EKs?&ADa|Fuzd6C{5HXr!T#1gE;T?8m&iL76EHl5c)l}IR5(F4h%0GhW8z@D_~~Cf|HmjiuGbUEZUDY6;COzfKZB2u#^$eK%02M_Lu@kblb;>C#%S2(&bss zKOYYFqTp#|%oPoO5RBl83^r zNCt^-_X29tZ~Gm07&BLnzMv=%m%-7LHNf2VAFN|!EM<30i|ROcx6#8@e5lvc9<@PS zMjt?__|h|?-$Fa!+?2(H99JBO_2lMD~kJ?mty zC_E20MNR24cm9!F1evQ=Fg_A9N?60TqV~E>k-`M7!Qkj>bV5n z1wD5865xMML#-uYZ97G$-M4!ZMg3J|p#}TYIYg?9Ubr$LH zF&BulI(W_+sWRFKoYxbc|5Ig)GGD2~dUd`6*zjLavwhK4J;_+;vxCuH5giMr=Ofco%^Nmqf~l*_?G8RsoA?ZSh$f6nW%u&x7Xlx!oHLLJ%O zdl}YFG`DSr8fE?f!k%Nw43WS=W-JufP;~Rio@{O+HlYnU##mr29|Z)y@L}Yp_#F9eg0Ltt`#XBX-*i;@G@gtN&01vSAfat+6@lxKh4a zu5`=<=6o$g0*?j|S1;GaBcJ0Z=o~tdR}GD~nAv*=f8h&oaAJ>C^Casr-9%70BYNt_ z4_Mz!Pp677JUU)*1Bp}qj_A#5tgX-OB8c0X1}EVyUY;(bZmkwD7*-K%rH11HB)-+sTFTKg({Xi|Ytu?YPpm ztX=Qg@k?I$8>1%T)f<-dX-`OJqDVT9Lm~{dmwC)+N703^CA(c}Y>_cHV9%B>(42p8 zRx_hsn!AySsMeA~<)o2>HgfFTSynZA(2{ji;TPRz6w;dbw+wO-wC=X=(Z@R7vc~hp zJKu!9e^V%KDrU|^_kSFkL*Kv^IT`B+z+h}-PDa_1^wDqz^v)*O27-EoO+$d*d48!6 zktDf}K(6Q+RntQRo!)Y|Uu%`GZFCWH!=^VH2}AJy2Rp{^_+C-hdcMQrB*^jNuPuTB zLdPy!2e5!s>#9pMh^_`pFs=;}HST0xaUkU>7QO&F+e2 zY|V=U$B)ENvzcDX>0JGWg*rh5j(8bktQ~93xgJb$K_@fpQhNCsWxEWjSYk8Vc@NTd zZn+1XyscLE!~K(wISXkhD8uOd3(v>5fmuLqjUkJW(~o`0VB(WlffG9Hpium5tZ@TX z6p^;f{G5{kJ~?kJI38Mr9R9EIL#8*x-CqGRi28cy&H_D>NUlu1Ynl1ygH*^1YmH6h z;N#Wzw))wU*z~m}+4Hy% zDn+!BmT;sP*D->K<|0Zh?0Djm2p&Z?p$1&S>QlcSrbss|1+FFYM(gOlq z#{m)mlL^`|Z2Z2XU(OHf?CfGEU)S;K$B~_%=Rc1av7b)5Fs4{r6K;!Ao!uXwuUJQP zR@rm>2n>BbFM4EU(kW&B1Z#*9>^{O(RWOxUd=Yj~K>lFp!Uw9!K9c{kgfp^<@vqQF z-1$R=pj=DaB?Gn(18JV^BLE|mu$S9L`99|rjbS(ZcK}8!w2Ifx2N>4Se^{<8dzJ4 zmvNk!dQ)cxBTQ@a2d6O#tpBvwA4X4MRZz+16>}(deu_3h`{KB6`0w%SHZ_KtEihqu zX#P zb>}dX;NUF@$$ETG;ot%16N{)x#qkRCLKLtpqER^ZNfW@JTpN;SVR zX!Bu%$)V@x)Hv0I30^pV#>UC@UuPUSy#-8nCPy>pl{g-$70TF_r&#wA*cL~FEt-lB zQlRVz2gll7p7a6PxOUx;Cm->R8e44l7b#d=SPIG8|x zqV{yukgoe~e^br@IRqo?I$Vl#o=+rAj%_(+rBKpjSe^HL|7_nHJlc!+31kmb#Z_Zo zbYa_zE*kpt-RHxDiw960-Y1y<%oi%gGA9y-f)bID-YS_kx#RW!$U7S9MQ(Y2#qxZr z=JMx15n~kyF30$_{lB_*vlbANbAg{4rhk;E7#T)^vvBM<31H!%B}I^!WnsP@&V^Cd zCt8A=wX=4Kp35DrPnG%7Ro=H>BKkHsAJj&dnUjHTY{bxs|2k;E^wA7%@08DB zySxaMP{4X9yZuNCCq_rmgPW+y=eiD04}Y3B;25rFfNwM*JT?u1cjOWR`}Rk!tN6K^lcNb14+ouJRJDGNV*Lc)3`Y(TYC3$2vxCaQ@CKG$ zR>Q4JAJ}h3EaZXjopstX?x#ns_y7pIkpLxKey^e=*8Zy!uL@5!@gEeY zlJkSy{W$Dbm&|_TFi3`wMG2oaG=*U<*JvnX_W9_f_C+)^lQ$@0^totq`IVP_>-LD{ z#iN%8YI>1+{S(7>7iKteXlFFZ!hN{mtY^ic;Ab<5vQViaOj59XFpeKIC8tJN2q)ba##-P|m(uuI<4 z(j0I$)bv@?>DxXsCDWnTvph{v556vKyHfsYQ*D~NMxJW8T1=Qcc8dJ!8dY8+yTfX$ zfEgj-bSb)E>d$8q!TH7EL#)NASM#VWt+edHr#02lMt)BxjhGX|HVQ}!sZwdiiR#j* zY#DCGxN9_no15F`Vf18BcTz~DAZ4#Ofrr_cBRlUBKlBgUhv}_@E%3n@yZ8u*5)V=9aVdx5I;NppO0>=MMN#^6%qyfe}fxH$rBR>vdYXs_jgplCQa4M?dH5_b+hnY#9!Vr}lR?LRhUBeonW zNL-~8WR_$>W?78Zo7Nqxg{%t!9YqhiY>>ho27NYGZQnvMtU1w@&d!gI+jjwJjfy4T zB=FOI1^WUpq1`OK+l|1$`FAD{gGL_%1W+HS9wy#7bLqm}Run`Oq*s$su%0ILi|OPX zW4NX%p)KczK#n|L>{uEOZj12xP1CRprzjeGSLz92pN65lyW9!GjAGj8Yp=>5L{8uwLAY2aG4| z@2(@5-pd{C!PrjrT3g@==+p!(v11TGKWR63gbrSRX!W0^)Rs^oL3;h^MsEY#2_F(C zKZ>PO9P{}qa8pN_leNTtU*EIl5eCrbCd1+3L`ewZhFkhK>rR@8vexA^C zVDM|A-SSWj3st9A4VdJ9mqxF%vn!MK{UFe!Wlb(HYv!--o_I49EW@GjkG-Th6-|!} z{90vdXkLXLa51)s=i9m9A=zXpahrctw8nPv02N-zd1k~fhuT0Xet*FiX|C_Zf?A!IJ(=I6JLrBD5~l0m(9^IHYR z92wou!cV;OnxE#ZDCXEK2DB5>2jqj)> zu3L{em`haix_{yB0C+tl2&Tn`Vy*3{pf{GNG8vg7S-;xB7{3Q9t74*6IJ~Jo);FSs zvB~b1mJm0dsHU|dh(|=&_j*{P*CwXpwE2ZB8h=}pYOxx+c_1iNV3ZhFdz>F;VOASa zV>ZA_rin+vDGm8XN2(=OnIEQ0uN_uY%ycDhZXjF8SXp(&tIhkUNQQR-|BgR%3l&{k zGE5Vj5yM3LC)5m~n+cyjjDx{LI2G_unu`Gi__kRoe4u%$g7mD1;}2h%lR+Ujp>=+_ zqP%}oVFGE?Gnu6hk;%e#8b=WF6B`%@GNmgUPid?U{olC9a|!zr6cI74$1GOlvol1^ zKL4@u{`t|K7WbgOD4E)}17|ok1eYmQ%M@?0s9nr#(>-l9IdZ7U4laoX$h!$>7Ru-! zQq|#`+P~}HHGKft>gvrU(bRTCIbnrhjeO96W=M69qzW<=nWZ5xTGf=6jiiF+fGe#) zrA8J<-r+K>m(X@-laiBODUD)M>92GC`;i{)vvV;poa7AE%KP()RQbJfq6vf~4Am}m zXOn7oX9_nispnIQkp&r&9U+d%#YN{xP3?NRFAT)h>T$-v%)C2CWyy4O!G35INMQf1 zlJ}=5z&}xXxQ<_Ebzg(T|M4EQZHI&(`R4~XtM`B&ITJm`HIF>|V+@CFO9&*1 ze}*f&m={9(Z?NsCgUY|etz@8fK&NB({@Y6B`2bB)4omS=_?**$ty+|bzn@7e7E8=l zRg2Si&TvoaQC1W+1d53BIY@Gvpj$VrzKNaP>-_%s@DBFw+@Cza03LR4n0c&Y_m&Yl zoFXT$L7%OD`A3X|-z0pmHg4-m@6f5+hA2-smayzq(g6Gs@GM|fCfBH6)(CaSsyC>8 zE;LB?h|mBHl+{v2)_*do@Pb%v(a+?O#f!#DDGZkVOO)c)hF75nnj3`!L77b!xmJYe zbO=i{uvGu|X*0_$XLp(hg)R7PwLA8ATn}sXG@b+Oeno@_YR9TV4(2W|pQo*{)^-y@ zq?a=8Q9|s=l8EL)JWr0UDmTffO+-H``of4rCR61}vf{}+G(tlEe{<51Llw4?qiC(L z;^uA$=FaEjFH@qX&7vz=C-bCqGC@P3PU2jeiRAzDF);Iv)ll~mb$yLR|WL!!kzaJT&Ee^UT<2{}}9a2aB>Xx$M^9Z*3L>6jSNAthMN1Q)Y!$DB8j?l zmZtQPVqr%RO&ylWa;>0@QdZ^~I|GU!m)%5$3}D`wx2v&TJ!+z6SJJ)Q^ykWzVpOQt z`$hg`jXvCk(YQm?xn~=gjd5jW74jPrtO-^dJ|9+A)?~+Rhv^`cM%TVV6#u3_H~#OH zFyU8HDZviHsteCkm?Sa-0(I7R<*;H-wm%FxdN*m%3J`ykNG4$!$UA5ayrH&G{Z{-% z$GNo&31QG~$CsnA>UH)OZ=w2{rr>##oS1TsJIOg{g=0GKN3dw$mCrD+BEuDym2Iu= zc)AirT>2(uAYV#9 zZMtD&nI2eTh^2y+(5G*1xZWlGgV?U?-=Ufxkt~#;nww($mdMo>Lrp@Qnr1 zz=WUNH8jm-@cyifzPRp_v@DGzW2gpUOY&TShm>C=!%IX}TU4zIsmx)JHAN+9f`tNB zPRGZziuIDO@OH{nC0lG$DjM(U(29s&)p>#61^gx!!E^taPZ)FzTJCwHNJXwtp)YqW zE;Ai|k`u?DCvAAD6k*w8qR4?ECp)abwEGt+4M4W~gD7qPPyIu93kT^c$mu1yjFCqv z>U+s4t*tF@1omI@7oY>7Klr1HC}waOe3}PT`6X=l=i~QVZypBXwtt=#eE-iE8FUx| zNB765|0RY6^9DINYxX=T_Nxk$_G2WmqRDKFyZkQt6b5a_A8_v0Wf@rzgFC(MugtI);zFe>>fY^X{NTH~g=U-x-); zt(6+6{|1O89y-tOJo>LU)dAB4wq;i~<9{-FHQ-W-QW;7CUPPO2VLAl3QnN_5!L;bU z45$PqHX9fJRU@}k#1@42^=}f5N_3HktML6V;V{zMhonC09l`N_I^Yeu0*(f7pMVW& zraF39mafSZM%scMA5ghEAKiZPXJaRWdo>gg?Cvw2!1w*IO|AKk4NIw>`3C{nAWgYU z9_%_6)pv!P-7A7}{70q&UiJ<|=$DUZSw%k{-P+}o3u&>kW!s(PUh!pw9u=|P&s36Q zi8_Cdes{z>gQZp@9D_?6Ws4*7vk#|bnUa{v zsg`wWj<9Thw=g$N*Hgz~a|jlu)KOy`z?6Xp)&8#({)0A^zNG4u;&cPFq?j3*K_LTw zaNnMudh@Wr$TDi+J+kd@W?*SjKV%l?80<1&s~#tuLheHoAN@2-C5=>D%$%x|aN(K} zFDp>s4y*M$WG$BO;itRvW@`_gvhkMKbm_*MRc3Y#Yw!lIFz8an9_^LbV3uOr-%fyEYVx0 z*q`(*7JcMN(U=Q~F-H&!^|M~9WVJ7stUjCwV*kiyH8MXHwC*&UnSq#u^)ePWa|(e|-Jxv|25-V5?6f4YysryA#x89=_dswU6b7V+jRP7_(lI4?HdaPcpDdu*0wXwF#Ult!)t{d*J<9CHp z79g~+oeyl$bKHM4s1nOKN}+J_=#@hx)1{51g?bIeGTjG!eC4i)Pe796=OTTeyuf9W zj^vj&Fsb@&L6p!i8kxN3{HN@J(tW5I4^`hdm8b+`A$)9M(A;Ir*=I-y@_U z)(-RMfLK{@KWUNW`a`>h2PIzu~+3eqoPj~AbEqVnSZ(Di_HkUi$}7Y;N+?oVi;J?ci-CI2k=M!p4a z{ZOY2msiCwS*RpNytIR34*I~yefx-WOE{0ay|4UxrY)1p!H(e~i57Z1soyKEs{9&O zSSf++dnB{5iTH!z^`K(0{!bjll1y5knp7u<`J2cU;NbUG=(=GzDHip9+5h0z-<=vw zx*sMLe)r@!;ZtgIESFoRYF&Pon1+ozChu=b91T@-$`&joHkI+-cW-Zr6rm0@GP=G( zQkT-v^yE7CoA{lwnKY_J&TB81NIvvoQ-dNIU8K5Od93dAZ*`p`oh`^f`R8#8!h`sO zqJ)j35h|B^QHN@%i3}bbx%uhNPQqzd-KwVN9H&W71}TnB#SoKW9oWfivDj7v)10uR zOJ63j?)Cd&s8*5BXgM0{Hx@V0%E?^!CfFT&UQ!<*(X5j| zWkhLzDrZuf(B8&lwZ8Unra*{+?Q^WYwvv(FK)D%Tccml3>k_gSmrcAB(@0T_-_f6n zz2-&ZFEMPqr_KAurQTP#hs3zErtgtn9J>kZjs17E{u5GI%|fKCfFtfN03yBfA3(LX z#BEPx?Ef3jnwct>M3aGM;-_|fcQ)UdZ~k@vx)>_Ba_XICTjS2E4NQN01b+jzaOc2ow=s8;FmR^ZQOZ& z(29&8R%3_1zf=BwtE^Vc;xe1MLsH_L~{CayZf72O=Z*b14LWOY4sL<9= zYELG7Z%=F&3~J#J0~B@11p-!iIe}Avj3y7go;kbj!95OVw31-&{@WH*lXps4-EXhH zK+J{&U|-0}01!j+LURr2JdU{!K#<2|lkLcUD)U6DfrIUG%jACJUhqZ#`UA>d7C?5M zaywa?y*^#x$xXk!gWBx7ZGtW!R}m|N6PaH$#_F7thBzKJsj|EWA0>yo3pC1Nty1)POqa`d)R70&rt-Ax2hVmb$xa{Yv`qK4xO3CT|Nadg=eFZV%3kk8QE}E3YFcQ zjbCtolLY!XIsA9xlt1M8Qlt<&37uH;tZh5CXuYt<*xzSgbt^RP z8QN->;ZFgw%P5)U9SLstn2md4rsYL)5>9Q_W){uiq4H6CwOOuSSmajh4S z!Z;9j0wbbbWOfHUJ|tx*F>*HJA6xyrM}!$wN&zC5%#JdEOIZb|`YFDTht@9t0sUtX zF-U%OTU*-QHH-*1Tk>5|%BL;Gcuk6Y=W!FbWJtR%MqL9G&2*29jGcR>iWy8?dwjbG zHSK%fBHwHmRIs#jv=bLuaZg(cnP##OG?nxmbMa9NGB1s;l%Tm(*=%pG1q z-t8$`OBkY|gQCgDovtSJB}!S%yOF)Dt*6{8QEz~(68rr!#H{mmR~B!r&3W6!_q@@w z#(UnXT|=3Vda+P%o(M5#6%ceYUp|YLEzJjXxw{^>oW~P>`POv7_Hx9A zzvIyTV6~OQZJ!yRCVdG!S=CEx&EM}dl$akA8hOADMpsx~A&+_Hu`U zJ}*{g$!C3*z3Q>zkoJAH(9&Ts;KcR&qo3a;c0iY4MolOSCn0HFF{ zEOK%Fr`6E`;KhzM+pnEltS^C}rUt37COOV93H?k;P2rD`u;ZN>AsL*m^>N^1a_*h~ zadz{0$5Za%`l=1|X>IZ?SE$OiR;ae|cC^l>`PL+<U;iQOdlpUZb6`K5y?{2+E{clg1XBtb<)0=kLu|ef7tan6zui z(8r^jI=87Fd!No5d*qSsaF9QKxN3$x!gqTJpQsm|2i$eekpsyR;Rk?3cX;4+gQMX5 zrR!Yo)zjl20`ZeOrhMQDj1>tjr83di87UZAQN3bu!BAS3<6bAg(ePozd0RP+E6|M{ zNjr3kQxj={^&dgrN;%Yd90wp9m7AHPvm(y^%({H zv!pO7$L1!&<7OFQlkz5F)X;yCcs0f?NMVv3wTRPDYwATLeE$Z-k^cwP=t%nW z920JRqIl**F24MvHz|(OicJf5u?-&+8YKp8?~Q+s3IOI1LT8BZL4L4gYMvQnYp24k zEuTrhiBA9etn=RBQ&;qVdNCrkZvWeVGIPZ$?Hb1l1Z;=ucf&;ficNQSx6&)jApvhB zT`~I+FEXYO5}dErDjc(V*17a9fT8FLbHcX)+~Bu^Av0zsmX2rQ#olqcutG65Cg(ac z6-=>qQ5TrF2oRDFd&ww0&I|87Nx#`Q>@W?a>3KJYb`cbJjZJ$9qsJdhl; zboU?AISSMNRZp;9MC9$kCM88DVeqO5Lj=pee(gVjX|aknH`Y(2S5)i58azX%AaGaf zfspNYT~i7NZv1txZKW;mtx!DqHD~o-xn2kWtfGM6p(nDeXm zzQVeJz8$e7<$^Wi)$dcTF2mcq-_bd z)$|esBnHlWkNi(${XiG{H8~HCcvM@Cy4wuKc)wmO*Q(I-5&l*5twSzgCxJduNpJ2^ z&mq@dXCwHwHp~%JtkN7|NqE71sgYvP6lEt_#$=LgzQN zH|5;VVQT(04TIZB!|q+(rEJZ!mF>D3D{#7Vcy}<{zvh0ocD6N|v^K9If~K*H7+Z>) zISLG^Of(&Gfo*FyXY(=>8V6ox(wRcOsp2e*aP3qVVt?uE$jA#V4qoc(@HUK3%9#V zRgnY`%b`thY+P8;Wjf`C+gdGStAJN8Srn__1~g@M?$!<9U1faPJxzz{J5F1+!fO8W z5AtC7(x@)FHRz$b;<>wmVUgPYnl2B}CGh*9(P%LFoyALFxGz~oD{?J@-d>R$%5pp; zIXC^`6or_w0HyZvVr^c!*GL@HkFd0PSZ$bZdcqk<5OEILzd^H$D znC>*B_mID+OZvqoxqoOP$8`$6V%WClwYhUEj4=P0gyK8UW7och6SL6NUv5vk5HByU zj@RakD0QsaH;mGpf#ERR8fR3tVKIA?N=@h18g%SzR)*Bwt`!JX(z+MAzUV*Or+p>zzh0RODDIi=erjVkALh+A z9Rzt3Jgxz@!tn0CMO>4J!&_jm_W;f~JsYh>;ILm?>gD0Om>x~l1cxL!UgWs8NN0$H zH;3dc15=%?pf(E@Z`o{HhMseBwC%v+ke0jF1vS6s&o1CcV!81ru#bvSKS{>&{GBe7 zv7rkel)Pm+n3Cy6pJP%dP<&NXc%nNh85*IWW^1>vJuC3!{4M*$+ zIU;-hSj}L*L7uA0KS%~FEobQhP3CbzsT9&hGI6+9IG*LJ6DUYbEc?^Sa^ZudXm!W z;UVFGn=d*ttJJrLYX;CdHpwjTwU~b3y@cIV9~gQuww<-H#MbwORWO}b5%}tghN?u; z<#YS%VoJus>tQ!dXhP_2m9otveAs6mar@@v5$P^due-`^cM_@OV8IrJ*{x}X<& z+;g^&{~|M3uWQ9#9_?E6+#r#(()jVStO1zkC-nvKKG+cg8b?B>DxUp-YxPsd>W#7c zUo9-1mj)$vSSh2$Q-(5{xgBML71Sg#NmoX9&(V^!G20M+>qj7}g}kkHzF4k4BmEvV znOQhz2zrS8k%{s)bgMsHAu!R%!v#W#{)7eR`MBc>>^$U_j|7^#N6~YkM!u(CX0sxR ze9Ss=++-auk)mIa{egXqxWU)lz<_QrFcxhmst7a0on2gZD-Yqp+CUo92Q^ib9t(<6MQExoG2(Bk1kob~%WrKq$!hM8GF>jJy{s zaQ106oWw`g$wX{pFib_6?mwMUx;`JvlE}YTRGtkNS0ILfa7&VSe>SArzsT1hEBi>d zH3ta;ci9ew317{imAgJS9`T90`eQbw$J&j+tUFtD_wDlWyEr8}F$!MeQEWo_(Vp#C zxx|Hs;M3{4i|SC|2R))`l(XqNmns&7fB8jSkW83u_ zt&q{(hrv4NzXMCA`6A2oCmrwgpl)+)FTdFH@1k%!0(l2Q#3kDW$M_S_(QSVJ=?cYA1<%&B3#yS@PbfSoyea+!D zZqJk2uGilqeb}Eo-G}2yn6HOo2=`5;T?H}cU?F_Z)riwGtMLa|aB?0&WjQ`kD-K!q zYg4>`Z;w0M1s;+$ERZH1FGB@AqfiC6+|TCt)<>wh%#dJsnW>D^i3MuT_TJTED0i;@ zyJ`}29a6m8;KjUhV-j28ur1ew+>Xcb6qYbx*h#pj%u7Sn(d}!Wi)@68OT#0hL_w<0sm_wVzhuCchu;0VB#)VuYx9asw zsxB^ZCMby#ES&rq?Sju`D=0dzflPrb(#qIx)tHDJdKpokE8;Jn(-gh*kD3;i(Mzqe z^97Owl_LZn#6=G7P_Njg@J;tp$_@sPVfAouhIb#wv9Cy#6R157h7&N zxih|Qx(Fl2r+yR*-b?r~bW_20YddCx=zVwUS*$6T7$W)3l%IeW@sR7}I>KD%958$x z)w!k(#-i-Pr$^!26do)U6cI`E+*CU%tQQN=M9Mi{P4c8kg=4GHrq{0b zc`emP+|ekRp>jFT4)lw<67wJp66%Jc^kG0(e@PYHtJ%hr(=d! zEeI-)6ps7v#3BQ$+2N$iXIFODJXCiIIq_vTf-*KfGRt0`&kZYzQQ7pl{y5`xl-G8K zse%w8+jPxXBD@{$PDE8PaNrF}T5qe2$Erfb4rCy(+{tYs)(E|7H1;%5Vq|`p7J2hy zKb_T-yMnr;1O=U)U=_y%Rg5@0sHak?lHQ_(6{AogIAzJa{nUsy_MicHW~O?l;~%3(cAhNu>X96$zUj5NK)<%*`OdfzHkTc- zSSxFgglC<$tVz*&m!DkGLEm&}M3%0P`!`iR`%W7j8YbnzhJvRT_a3|}w#l?>i(NAw zO#aFbe^B06AF9Fp_T$Y!Mm0eiC}?4C>e6kMH$jt4Y9Ax~^kBh! z*d}|Gg6R5Xnlek)n4p)5I9MBJvM|U`td*<5RF19o*RFiodzR|ey^@((G&3lxPmwvy z$Tr=((7Il@rHN-=+U0Ki0Tpw|9#!;<#~dbu;#tpIfHOg9=fz%@Bf008Y|6Pv8!wwv zQ0LqFlco6m>)kej=!e8ST9Uv%X1`-*@AH1-sB>Q4_9Nx^^I)&SdGO$mAZ&8aEE7l< z_697Qq5F+idc?5Y>Z=ph-MZj5BVtdVZV{_#!A1lPsrXs zY0~5ZXa-|kioP6=<%+WenbJ13BI$Otg$+^M59qlTudt(a_E ziB0hLI$;*#JcPY#?vsIGU`#x%n-|y0frCkPcJ?K$d0N~a2zZ_j*xNJ>(pZZtwtv4)a&AP>_L1{I^YLza`|+x& zP|Tgtouh8x>p#GMe5h??spSbtuRqfQQ3fMp(hhEz!dNu4U3{Zh9Ze zC%{~WD3Yv_*~X)Ntf@T#ik|3Je)XzOLFR8rsJwtTr4`&w!uXn3Y%~=C_TkjMoLy z0<+$s1#WL&@UJiivf`hlA0bpLwv)ew2;gkj#pb}hvn|e9S$8Lf?2*8lz{ta|vMMSP;ioHP%gk$|>#n_^Kdg`aD<=;w z_Injsw>^B}XC!&~T>hmcxx-ICUl~}(^>{x_5eqsyU2Y*v2li4su+(ww0%c2si;e!A zifQ)t>K1*w#Ajaiug-n5a^W|NZNibiTJaq$|Aez14abSB7@;E}@z;X|(5Glm>&DM% z$|U$#W+VraKCVc4Ohh=7YdpxQWX55_ugQ%m^V4raXiWAZ#EQE$Gf^#Uoj&Ei^4rpy zzKV6rka#DjR{?}%sye3R*Fr%jR(a@54X?-uS($PhN|_>vRi-R@#rYfGL4HD8qg!4& zf^BS#@pF)J23GqL%jXj+syT|_hS|t12VMps2I0`mfl5SRl&u>}S=vWj&Bsq2saWi4 zZFZRGG&k!hTTvgBHZ=bk&*_$(U!ASI2*sNX6yBMGDp=1Mp-iRyptjbC;gqno7Y;(_ z-yxfMTXhctPHya5z7R9wzgLi<%0={-r94AvCf`QAEuzk2=^eV%;+6N8P`;j9&o3x@ zQZbFP*tBBY8$?*@RolY>$=e|5Ic=c%hkF_3)2%d4PX(c;g1J^^g<_xLF}o!PMDanx zX-al^-7kr+ks3S)HU@PVs04!Wo+rk5L=4d-ZmO9)cX~{qCt2ZPNJylph-RC+!S0Qw6e#R_QPRAO{Z$+_X4P*p~nWSv|UfJpRXJg107)jEzZl3=n(Q5qx11Hi^e zFiJp`+&KboiuXo3#R++MnwWxT;lEgnd@ShH%bO(Xx?TsMDH$Q9uin2e_>s>KrN8dC z5>GG`_j?i^dl>1IV~Kjt{E^mxpZsP3fgcqO;2OMy=d+oaqI($+w#9`#0t~x;9A5f7WW_0r8Oo-V!9c>3_PV_Apno*p81+p0LyZD3 zLxZr7A%mnH=q7Qr+GpweQ@*&^JJ#ZPY}v^KZuQZ$Qe5<(mu{(*jVH`a4P3uk5m)|9 zrWMsf2zfKaHO+0ZMfAH+!4MaL+a<=Jg+043oAy$#}xJ z1-2l$_di|^v}l(#uO`jseQHcU_{3!ahoGLldiYmi9ERuJ^9KyLjb4r!11<3}U8pN9 zE*6WEUpQ*n$JV{i-#gs5sz>=12L(2&P0{yDoDpM*9qN8RD&`ZyPTUvh#U%OX33m+! zr`S`RM~Mb+hkjkNP{TDoz!p4hTz~Uk2^`ytZ@>zVCY%g+f=?Qs7VNFe_8XRYK)#zD zCgt;ygN(%;gh!+Buom~zK8?ZrVl<=_-FOvcDQ`5H%6kKWl6Iuiw?2Ww_rgNWbJIHM z=bLED(jHw61`9-Jmu^;{>q-}w?5b#{Scu7m84A-ki0&5 z!eI&LI5&8ClDP)|EQ&5R|EG=R$O*E!^L7gOn2Ucfv~r|y3E{_B4;t9^uT9&8MdEaT zzq^0GM>ZOMsO7}oF8hf1C;Tq2bc^G5y$w~tu&otk;(RMK?2hWf^hp;qU-y*uq`-zV zw#N3Zmw@JTq_3q9vi>r+tw>jSm2vXXp-kMDbf;Zm{A|Av+H=mi?+LIdc{}y!j-bm9 z!=o}CdP`r}bw|`Hf%BIfKlv&F9|Pzd>v^Bn@!+-Ch*YgSqR%usAvf2hn}Q(AR>ts; zyVSn#d%p-0aWNzp=r8L!?{a;Mww^HJtfq5`(`fN3M0B3Z%>9l+?uHHyc~R=F3xoAm ze4(>N@$P*)C~;uVXKMd-ge!A+XoD~+@977@E-r#{nXt=?_cX*l`@JWpH)Ss&Y;nSm zbdM0@i}mb&i{{wGTgcMflS3Shak8!xJc*UeC8FSQRXUys9SpWGC@3$=v__^!Sl>;~ z)OekQ%S)B(D$x2v&C!+LFKVBoZXo;gq8OxWu_ptA&#TNARv(lcedgF3osipIg{NHj z4z3bO_=8h`8xY%makroA=h6Xo0yU5}#+~vyi8isGF(=aW@8WwF2MN_>xbv%dsJO5qW45qFdX*oF^;f%9peNx$0_g3>1x%sT*iG?VZM97t$jG6#q1&8| zOjd}#%}n@C{PF1XWuC0gtky_-Qr{4bTSDgFjjc%WZ@Ry;p!G3Z#?~^C{1?!9T1e;s z>ct>i&Fe3j@e10>Ujm9_7MJ*6zG;R$i9*hlgdIT-(r$v?()kTe9A>8Bdys!!zF8v? zoLXECs|2r~-&t-B2!HM4!#vAEo|yUCzccq3@N`CqM%UuJttDOpBf6+lW0BkItaje{e1vb3OT3BO>N3;SLG2)aaw#EAXr6irqG}5 z+riE4koK?LKaNkesZNfj2R+osZX=(RW^E&%1t!)y`QMa8KIZha?Rk=l?DfEa0n zXh?_5oBru}Yc0g2ydB1g4i!&B;Sf=;qkmMZH5MuYQipX`=WhBsFG5%4)3CWf=DXrN z$g1IX*v8uX%}D79MKLSfeSc$xA^g+aMZ)-AH^Kg5c%Q11>{*AJjV{o!pboq4`43*T z_%COkcv3kFXH0^z)jK>ODUjO&M#B&UUDD1@q^LegK`1!z87&U5udu#$Ohs9 zDR#_3A>=c0R)ZYn9(JH$)~USTp2j(dOUY{#FXfds+SP(MZ|=}bu-JIZbi zTa~MLdaGn8n`{?>oYTm!8!P{Vb}w)QoZ(%|S?LrznI)C|>2)FcCs+foSwAVy+uH+` zKS&$H-O-M+HOLx#JxQ_w#h8mS8T}Ixf>5g!8zcLyc|KS|qIS(G3_I-9Ksy!nUb$&v zr%l)zooQnZ2L}7EN9nRkWmvY2KhlmuXvF*y2C9Sa^nARte2NUm(#}uIS+ygSzbv_R zw9eALA(URZFBr>QSDwmoGTUOO=~&zJ74s#~eCw9j5B!r04leW}G!r%@UeXaB9L8TC z8h*E=v!G{fzcaXvnLp=9hqrcJFNXc%27VH#GWDVz@w{7xTau0V&e6NuRuq{{3VMBT66VjnU6~=J-oN}hxF!qA-2uN~|dCwpFd_ur<%2ZWIzKB4tShX`) zn3{7Rsi_O~gO#)Ys?4YVDeB6jl0dt@E2Buc<%a1fZmFP_WXfRUh8rS+3!sjvp@n9q zqmDS`B<@>^0`BGl7?>McSy{Q}u7#S_G%XkvZG)DjRx=;H-+R9A`SU*a{+{1CcRA1G4uxHA?-ebL#}Y zCYbR~>g@$fk6%hUw{3r1QA@en=WJ;I8tW_n#;|nXt)LF$viGFh z9y4xDosT8V8I)(b<*Dh4=i*o6qY04`VqM9M%B-Kh72uH3s%D^j6X}P`eS9X|w_*EY zJD?$~dFNx2a-MzW896KyyS{B6x?%dWp>Lsd=j5J3M9Pbx2ZMdju9F+Tig#3Re7OH( zUabW?MXavSSN7Z|p}xRzw=PB0ABjxRDRjgCJ6Oilq{tup)A9YagRkzi>;(=WymCPsK)V>uC{v^Is?lpVjgPKTU3BoKv}b z>0QX}-pIJ4w^rAukTd4>c9qubD@cy5E5`tTu;!uAcRXtJaBmg#l=Dbe-_C)kr&ZTx z6`MQ}2u|e)6R)H2JZ-O}OcnF4g;kOf_S1cjD%sEkkXm$lJ;(>Q(}Fjqouf}-w5?R0 z8Th!@&_yOoC5SokD-@L@7w>g;3|mb7t039{?qYfa&;e{7U)8WhEm)p|!-Hzd);hS`^ctulp2&s> zc{zP>qHH3&TVgU|YyA=N+|M~l+vBdezQ~f#VRpQ%IerJ|<(0RL`Qy=rzQ?e@G6nOV z;xwAPJA1X4`N#&1aOU}s*TR(y?I9;) zK8WU9mOx-epFvu^yP2o%Cj*4DWsSdsi=sofT?YJ*K-ajTvo=KLAwC*aVVjd#m?nKv!XqKOU-akT!2!-b4m1~Z9C6c2X2$IDr ze~l$+@Rw@N?wlnl(l@ZR14F{o?6KCJ3j(I!Yt@@B7d!9LH+omt$VLVs`*e1adHz|e zD7B^r?WzPolQ$)g=Hx$lkB0-J1ZNAg33{KOYDmYXr#C*qS>Z)_!}z5fFEpmTSa5-c zXL9qYi&^C9qxeF+gF*3C9as~Bs>G&|@pYEdRUKd!$I*V25BtH$P9MZe@19Ueb&KZ3 z`Kd72o-RvqPD&f~rWf23gq8lVe{CFV!dc(Z-8oF}v7Jx_Oi)Wz0+Hnf;4CT8g?hyZ zl|#WS=;f!aC$|3XnRI`fSU)v-LcDtXsV$DAU3WHQr1?6xWW@l(;M7YGpbuW|g*m49 zYuYiW6Nv@*&8U-SSytaJlPQA{!>*t~_*fCIv{(lU#X{_{f{W4fME^&uY&4Ijyeu3h zwB?91k^JL*!5Y})t%znPAji-*@7`<1ZN9og#Sm<^YGQPGlKQ~SK6!)DB`&$+&ik{+ z<5pZIBgeY)OBN+x8gmW$L<~5od|{Yi+9swSe9|`$eMGodpu@dZe+j*p>6PcdPO!vP z>oXg-!CldQ8!2)4v%!LYT#QU`zttl#3948w?w3O)FAe^!Hd{`*@FaejrwMM|i*B#~ z4}2DdJUy1=g{6Kx{eTcEzu2o~9nleI8f*GDh->hG=?^O<&M_dJ)vETBDjgub>eJRh z+w&EVmxTue&~oAI9W0!bu%2)YYM>BLUhVplC0P#s8GRkpoPR}UdghA5{JENEByJ!o zmD!5(W|K=x2rl+!nzSNf+<&w;n33$<5JV{9p_f z<2^OUcC3vj>t)PTZVHxrcqra00mI_rZ*;d=JPY}l@~)oc14gCW*er7BBLk$YX_NEa zV8x-3bcJ-!nGXqJO-Og`X@(w+JrjHCO>z8&wIAD?1={xQE+XKlesr5?@4>u`)DJc{ zdBO5?MFwB>+bqo#_fMVFfVF6u8=5~f2_GJc6ciD@bD6U;yWXgw8lqWVzfoV7`gW#; znZpyYaw_TXaHTV@(XMnru~Si41=;wr4E?HA2-ZCJxcB9b$dLE)YLK{Bj2vEU^?$Wh ziGbVMbH#h^TH7p|{_-16v!`w2nn9QbW8hZ!-Xp>;Iri=POCnZlyH>b6w7EQ@Hft^J z6?-~%o6#EctY4JA3fTKhEo^)II%lBWtwz7VNCssv*4^{h3b9^SnWJQ6LM-O4Zz@)}YtZ5J-z&*S{ixXOq%z zS{e1o$pNzcyxcosCM4>-U{+bQ2riOM8?xRk zLrL}S9R_fP&@bd0S?jX#sjRZ2+M^I*x4=C$-YvV&U#vN|@mUF6#=grVumGghmT$pv zyby#F?!Yh8om2IeWX^zAaO9aVe-Y6Dq1t`pvPP9PbD_$e{)Uc90iWSRcR&&bJ>QwM z<=<1eTaHn1Sch;vAFkyhkiOw~Zb8H2Wa*jg#^P>X=kS7Q*CVIe5Ocq?C>?DGW-Hw) zm&I&tPin_btQ<1{=gp7y6b90x7-4t&>jZn&ZETVYpeUq2lK{s5lcQ(+R20^^Clk?n zC#L>v2^U*&0z%2;5-HC>@x)v(Iy;opU!tVdvwprIsk4VpcK2 z_hMe2=ugQHaXyo9^GrczO#L8rD_mDZGA@(##$%~mPjF1V19b~}%y+C5EqapY9e?X6 z)z>Vbx@x9vu{LWnS?0ajDLkex%!$;Ee;w(qn7Dh!CsM{LGkeN}TJZ zW>hAyV-cQZTJC^pM4sl<6?DWE$l%OC6e7ZDA+^AptST{w&s3Jwi4L*vLP^M6T9G(; z%^22=`KsO@Gh0i#+G`Tx-+z(q3ga5_OdQD!}SoGX%d7iZW=im?i5UP^rkb{xXrh1`$J#;EoCaiJQ^|(pK zZ!e@mNX=)>pU$2!w=_HeEOGtY1{`{I(+a8nV=;w6N8vtN*M=4+P*Y3zIK^Sy8D zC}^I~CVg-qAVRhtbD8h8InI@muobV(#Br)YvE$2g(Dy6b#w$KrB>o6~GZAa1xSl;Q5gnHKnbrb8Aki487n z=~~=;I59e7Q`7z}@1(HT<>gI%hqrQJ7l#h05%wGblR^MzXoQnZph!OP<|7}iUK1m8 z<$V_R1YL-xReMZlM)s)5npJ)>`2EiqUE2&uqBTiY!3Zl?Y+_nju^=UQE?GxAIE-io z(Y<`hbYB@QtujEd_yJoUn6_RaBa_xR5Zh2QuD0nh=y8Kx`t(=Wk@*4Y=2Y9(vn8S0 z>>{ILe8<=g7HA5h;7ZY6?uXYgv#HY2v9qWTDZl_Q>sCj3F>{h^|fS;bkH8l^U^pKOgPbStOM%pQ`8;?)_ycllnAmX0#wb)?#~iwgRv=MGJJOg-xE3gj$ai%*Ua8W0 zOSQ~>4H-s=v{UaMPAo7!J!#pPJ8iCs4A5Ji)LSh+HX`e_W~Q_he)lW+Z4ZEfLr+hh zF_+)HBws#9HjeTq`G-MUV7=?0+q&bjma8}&Na0RZ1GVA{BsK__d>2%xIiCL(cSh!> za^Eo70v$EG3XCj^I_?aZ$O6M!C9dChOxU-s1xF56nES?DQ-aC!v=?b#1}hkJ3neGo zLx|xa6Q=}sSyX|mC)&RhiEE9p%Ic0|X+Re>(c0lET4fazXdM57O&C?R6Dr%~XWMpi za#(^2c>9!S4v&(2B5K1Mpr*Ld&J$ zuA;7cRagHHAV{%GxjP62JNy5DIGN}q>cl11+;^-0nCFAH{;*HJl9v{T&X$uMKUAPs Jw?{PXe*nts@h$)W literal 0 HcmV?d00001 From fa01f08073cc67c7bd7b9218bb505e9d34f2fbc4 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Wed, 30 Mar 2022 16:21:01 -0400 Subject: [PATCH 212/628] update the README.md --- docs/acd/README.md | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/docs/acd/README.md b/docs/acd/README.md index 5a7b9265..b0cc05d3 100644 --- a/docs/acd/README.md +++ b/docs/acd/README.md @@ -44,9 +44,9 @@ Follow the steps to provision an Autonomous Database that will map objects in yo ![aei-1](/images/adb/adb-id-1.png) - ![aei-2](/images/adb/aei-id-1.png) + Click on the name of the Autonomous Exadata VM Cluster, and copy the `OCID`. - Copy the `OCID` of the Autonomous Exadata VM Cluster. + ![aei-2](/images/adb/aei-id-1.png) ![aei-3](/images/adb/aei-id-2.png) @@ -56,7 +56,7 @@ Follow the steps to provision an Autonomous Database that will map objects in yo | `spec.compartmentOCID` | string | The [OCID](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm) of the compartment of the Autonomous Container Database. | Yes | | `spec.autonomousExadataVMClusterOCID` | string | The [OCID](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm) of the Autonomous Exadata Infrastructure. | Yes | | `spec.displayName` | string | The user-friendly name for the Autonomous Container Database. The name does not have to be unique. | Yes | - | `spec.patchModel` | string | The Database Patch model preference. The following values are valid: RELEASE_UPDATES and RELEASE_UPDATE_REVISIONS. Currently, the Release Update Revision (RUR) maintenance type is not a selectable option. | No | + | `spec.patchModel` | string | The Database Patch model preference. The following values are valid: RELEASE_UPDATES and RELEASE_UPDATE_REVISIONS. Currently, the Release Update Revision maintenance type is not a selectable option. | No | | `spec.freeformTags` | dictionary | Free-form tags for this resource. Each tag is a simple key-value pair with no predefined name, type, or namespace. For more information, see [Resource Tag](https://docs.cloud.oracle.com/Content/General/Concepts/resourcetags.htm).

Example:
`freeformTags:`
    `key1: value1`
    `key2: value2`| No | | `spec.ociConfig` | dictionary | Not required when the Operator is authorized with [Instance Principal](./../adb/ADB_PREREQUISITES.md#authorized-with-instance-principal). Otherwise, you will need the values from the [Authorized with API Key Authentication](./../adb/ADB_PREREQUISITES.md#authorized-with-api-key-authentication) section. | Conditional | | `spec.ociConfig.configMapName` | string | Name of the ConfigMap that holds the local OCI configuration | Conditional | @@ -147,7 +147,7 @@ You can change the display name of the database by modifying the value of the `d name: autonomouscontainerdatabase-sample spec: compartmentOCID: ocid1.compartment... OR ocid1.tenancy... - displayName: newACD + displayName: RenamedADB ociConfig: configMapName: oci-cred secretName: oci-privatekey @@ -171,6 +171,7 @@ Here's a list of the values you can set for `action`: * `RESTART`: to restart the database * `TERMINATE`: to terminate the database +* `SYNC`: to sync the database, will describe in the next section 1. A sample .yaml file is available here: [config/samples/acd/autonomouscontainerdatabase_restart_terminate.yaml](./../../config/samples/acd/autonomouscontainerdatabase_restart_terminate.yaml) @@ -189,7 +190,7 @@ Here's a list of the values you can set for `action`: secretName: oci-privatekey ``` -2. Apply the change to stop the database. +2. Apply the change to restart the database. ```sh kubectl apply -f config/samples/acd/autonomouscontainerdatabase_restart_terminate.yaml @@ -218,7 +219,7 @@ Users can sync the resource manually by setting the value of the `action` attrib secretName: oci-privatekey ``` -2. Apply the change to stop the database. +2. Apply the change to sync the database. ```sh kubectl apply -f config/samples/acd/autonomouscontainerdatabase_sync.yaml From a8698cbac8896e417c6f96b62e455ce2056ca509 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Fri, 1 Apr 2022 17:45:50 +0530 Subject: [PATCH 213/628] Remove EM express listener workaround --- commons/database/constants.go | 3 --- .../database/singleinstancedatabase_controller.go | 11 ----------- 2 files changed, 14 deletions(-) diff --git a/commons/database/constants.go b/commons/database/constants.go index 82128aec..fb42bce6 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -58,9 +58,6 @@ const RemoveChkFileCMD string = "rm -f \"${ORACLE_BASE}/oradata/.${ORACLE_SID}.n const CreateDBRecoveryDestCMD string = "mkdir -p ${ORACLE_BASE}/oradata/fast_recovery_area" -const ConfigureOEMSQL string = "exec DBMS_XDB_CONFIG.SETHTTPSPORT(5500);" + - "\nalter system register;" - const SetDBRecoveryDestSQL string = "SHOW PARAMETER db_recovery_file_dest;" + "\nALTER SYSTEM SET db_recovery_file_dest_size=50G scope=both sid='*';" + "\nALTER SYSTEM SET db_recovery_file_dest='${ORACLE_BASE}/oradata/fast_recovery_area' scope=both sid='*';" + diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 587d327b..b3284d31 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1486,17 +1486,6 @@ func (r *SingleInstanceDatabaseReconciler) validateDBReadiness(m *dbapi.SingleIn } } - if m.Spec.Edition == "express" { - //Configure OEM Express Listener - out, err = dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, - "bash", "-c", fmt.Sprintf("echo -e \"%s\" | sqlplus -s / as sysdba ", dbcommons.ConfigureOEMSQL)) - if err != nil { - r.Log.Error(err, err.Error()) - return requeueY, readyPod, err - } - r.Log.Info("ConfigureOEMSQL output") - r.Log.Info(out) - } return requeueN, readyPod, nil From c983b54b4d620b1dfa253967d7285fdba7753d4b Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Fri, 1 Apr 2022 19:37:09 +0530 Subject: [PATCH 214/628] Remove commented code --- commons/database/constants.go | 4 +- .../oraclerestdataservice_controller.go | 4 +- .../singleinstancedatabase_controller.go | 69 ++++++++----------- 3 files changed, 33 insertions(+), 44 deletions(-) diff --git a/commons/database/constants.go b/commons/database/constants.go index fb42bce6..db061ad3 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** @@ -44,6 +44,8 @@ const ORACLE_GUID int64 = 54321 const DBA_GUID int64 = 54322 +const SQLPlusCLI string = "sqlplus -s / as sysdba" + const NoCloneRef string = "Unavailable" const GetVersionSQL string = "SELECT VERSION_FULL FROM V\\$INSTANCE;" diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 428f2032..0fb7ffe9 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -144,7 +144,7 @@ func (r *OracleRestDataServiceReconciler) Reconcile(ctx context.Context, req ctr } // PVC Creation - result, err = r.createPVC(ctx, req, oracleRestDataService) + result, _ = r.createPVC(ctx, req, oracleRestDataService) if result.Requeue { r.Log.Info("Reconcile queued") return result, nil @@ -907,7 +907,7 @@ func (r *OracleRestDataServiceReconciler) cleanupOracleRestDataService(req ctrl. uninstallORDS := fmt.Sprintf(dbcommons.UninstallORDSCMD, adminPassword) out, err = dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, true, "bash", "-c", - fmt.Sprintf(uninstallORDS)) + uninstallORDS) log.Info("UninstallORDSCMD Output : " + out) if strings.Contains(strings.ToUpper(out), "ERROR") { return errors.New(out) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index b3284d31..0e857c74 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -518,17 +518,11 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }(), SecurityContext: &corev1.PodSecurityContext{ RunAsUser: func() *int64 { - // i := int64(0) - // if m.Spec.Edition != "express" { i := int64(dbcommons.ORACLE_UID) - // } return &i }(), RunAsGroup: func() *int64 { - // i := int64(0) - // if m.Spec.Edition != "express" { i := int64(dbcommons.ORACLE_GUID) - // } return &i }(), }, @@ -677,39 +671,38 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }}, Env: func() []corev1.EnvVar { // adding XE support - if m.Spec.CloneFrom == "" { - if m.Spec.Edition == "express" { - return []corev1.EnvVar{ - { - Name: "SVC_HOST", - Value: m.Name, - }, - { - Name: "SVC_PORT", - Value: "1521", - }, - { - Name: "ORACLE_CHARACTERSET", - Value: m.Spec.Charset, - }, - { - Name: "ORACLE_EDITION", - Value: m.Spec.Edition, - }, - { - Name: "ORACLE_PWD", - ValueFrom: &corev1.EnvVarSource{ - SecretKeyRef: &corev1.SecretKeySelector{ - LocalObjectReference: corev1.LocalObjectReference{ - Name: m.Spec.AdminPassword.SecretName, - }, - Key: m.Spec.AdminPassword.SecretKey, + if m.Spec.Edition == "express" { + return []corev1.EnvVar{ + { + Name: "SVC_HOST", + Value: m.Name, + }, + { + Name: "SVC_PORT", + Value: "1521", + }, + { + Name: "ORACLE_CHARACTERSET", + Value: m.Spec.Charset, + }, + { + Name: "ORACLE_EDITION", + Value: m.Spec.Edition, + }, + { + Name: "ORACLE_PWD", + ValueFrom: &corev1.EnvVarSource{ + SecretKeyRef: &corev1.SecretKeySelector{ + LocalObjectReference: corev1.LocalObjectReference{ + Name: m.Spec.AdminPassword.SecretName, }, + Key: m.Spec.AdminPassword.SecretKey, }, }, - } + }, } - + } + if m.Spec.CloneFrom == "" { return []corev1.EnvVar{ { Name: "SVC_HOST", @@ -832,17 +825,11 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns SecurityContext: &corev1.PodSecurityContext{ RunAsUser: func() *int64 { - // i := int64(0) - // if m.Spec.Edition != "express" { i := int64(dbcommons.ORACLE_UID) - // } return &i }(), RunAsGroup: func() *int64 { - // i := int64(0) - // if m.Spec.Edition != "express" { i := int64(dbcommons.ORACLE_GUID) - // } return &i }(), }, From a20d01a8253078e1c13c6240eae4ed008fd5bb61 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Fri, 1 Apr 2022 20:48:24 +0530 Subject: [PATCH 215/628] Fix pre-built DB deployment --- .../database/singleinstancedatabase_controller.go | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 0e857c74..c70ea90c 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** @@ -495,6 +495,18 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns Name: "SKIP_DATAPATCH", Value: "true", }, + { + // This is for the pre-built DB useful for dev/test/CI-CD + Name: "ORACLE_PWD", + ValueFrom: &corev1.EnvVarSource{ + SecretKeyRef: &corev1.SecretKeySelector{ + LocalObjectReference: corev1.LocalObjectReference{ + Name: m.Spec.AdminPassword.SecretName, + }, + Key: m.Spec.AdminPassword.SecretKey, + }, + }, + }, } }(), }}, @@ -690,6 +702,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns Value: m.Spec.Edition, }, { + // This is for the express edition DB useful for dev/test/CI-CD Name: "ORACLE_PWD", ValueFrom: &corev1.EnvVarSource{ SecretKeyRef: &corev1.SecretKeySelector{ From 2acb956314670c71328c47f8a5c1ae55ce9c981e Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Mon, 4 Apr 2022 13:24:34 +0530 Subject: [PATCH 216/628] Use secrets as volume mounts --- commons/database/utils.go | 2 +- .../singleinstancedatabase_controller.go | 53 ++++++++++--------- 2 files changed, 28 insertions(+), 27 deletions(-) diff --git a/commons/database/utils.go b/commons/database/utils.go index 1373fdbb..0a372e40 100644 --- a/commons/database/utils.go +++ b/commons/database/utils.go @@ -583,7 +583,7 @@ func GetSidPdbEdition(r client.Reader, config *rest.Config, ctx context.Context, } if readyPod.Name != "" { out, err := ExecCommand(r, config, readyPod.Name, readyPod.Namespace, "", - ctx, req, false, "bash", "-c", fmt.Sprintf(GetSidPdbEditionCMD)) + ctx, req, false, "bash", "-c", GetSidPdbEditionCMD) if err != nil { log.Error(err, err.Error()) return "", "", "" diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index c70ea90c..b1f00363 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -438,7 +438,7 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab // Instantiate POD spec from SingleInstanceDatabase spec //############################################################################# func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleInstanceDatabase, n *dbapi.SingleInstanceDatabase) *corev1.Pod { - // Pre-built db + // Pre-built db, useful for dev/test/CI-CD if m.Spec.Persistence.AccessMode == "" { pod := &corev1.Pod{ TypeMeta: metav1.TypeMeta{ @@ -453,6 +453,15 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, }, Spec: corev1.PodSpec{ + Volumes: []corev1.Volume{{ + Name: "oracle_pwd", + VolumeSource: corev1.VolumeSource{ + Secret: &corev1.SecretVolumeSource { + SecretName: m.Spec.AdminPassword.SecretName, + Optional: func() *bool { i := true; return &i }(), + }, + }, + }}, Containers: []corev1.Container{{ Name: m.Name, Image: m.Spec.Image.PullFrom, @@ -481,6 +490,10 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns return 30 }(), }, + VolumeMounts: []corev1.VolumeMount { { + MountPath: "/run/secrets", + Name: "oracle_pwd", + }}, Env: func() []corev1.EnvVar { return []corev1.EnvVar{ { @@ -495,18 +508,6 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns Name: "SKIP_DATAPATCH", Value: "true", }, - { - // This is for the pre-built DB useful for dev/test/CI-CD - Name: "ORACLE_PWD", - ValueFrom: &corev1.EnvVarSource{ - SecretKeyRef: &corev1.SecretKeySelector{ - LocalObjectReference: corev1.LocalObjectReference{ - Name: m.Spec.AdminPassword.SecretName, - }, - Key: m.Spec.AdminPassword.SecretKey, - }, - }, - }, } }(), }}, @@ -572,6 +573,14 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns ReadOnly: false, }, }, + }, { + Name: "oracle_pwd", + VolumeSource: corev1.VolumeSource{ + Secret: &corev1.SecretVolumeSource { + SecretName: m.Spec.AdminPassword.SecretName, + Optional: func() *bool { i := (m.Spec.Edition != "express"); return &i }(), + }, + }, }}, InitContainers: func() []corev1.Container { if m.Spec.Edition != "express" { @@ -680,9 +689,13 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns VolumeMounts: []corev1.VolumeMount{{ MountPath: "/opt/oracle/oradata", Name: "datamount", + }, { + // This is for the express edition DB + MountPath: "/run/secrets", + Name: "oracle_pwd", }}, Env: func() []corev1.EnvVar { - // adding XE support + // adding XE support, useful for dev/test/CI-CD if m.Spec.Edition == "express" { return []corev1.EnvVar{ { @@ -701,18 +714,6 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns Name: "ORACLE_EDITION", Value: m.Spec.Edition, }, - { - // This is for the express edition DB useful for dev/test/CI-CD - Name: "ORACLE_PWD", - ValueFrom: &corev1.EnvVarSource{ - SecretKeyRef: &corev1.SecretKeySelector{ - LocalObjectReference: corev1.LocalObjectReference{ - Name: m.Spec.AdminPassword.SecretName, - }, - Key: m.Spec.AdminPassword.SecretKey, - }, - }, - }, } } if m.Spec.CloneFrom == "" { From 63f57e54528f6fecd4be54bbfa714d4bd32221af Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Mon, 4 Apr 2022 13:38:34 +0530 Subject: [PATCH 217/628] Relace unserscore with hyphen in volume names --- .../database/singleinstancedatabase_controller.go | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index b1f00363..4a4b01c4 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -454,7 +454,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, Spec: corev1.PodSpec{ Volumes: []corev1.Volume{{ - Name: "oracle_pwd", + Name: "oracle-pwd", VolumeSource: corev1.VolumeSource{ Secret: &corev1.SecretVolumeSource { SecretName: m.Spec.AdminPassword.SecretName, @@ -492,7 +492,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, VolumeMounts: []corev1.VolumeMount { { MountPath: "/run/secrets", - Name: "oracle_pwd", + Name: "oracle-pwd", }}, Env: func() []corev1.EnvVar { return []corev1.EnvVar{ @@ -574,7 +574,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, }, }, { - Name: "oracle_pwd", + Name: "oracle-pwd", VolumeSource: corev1.VolumeSource{ Secret: &corev1.SecretVolumeSource { SecretName: m.Spec.AdminPassword.SecretName, @@ -690,9 +690,9 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns MountPath: "/opt/oracle/oradata", Name: "datamount", }, { - // This is for the express edition DB + // This is for express edition DB MountPath: "/run/secrets", - Name: "oracle_pwd", + Name: "oracle-pwd", }}, Env: func() []corev1.EnvVar { // adding XE support, useful for dev/test/CI-CD From 8a03048d9cb3af994be97c22e26db5fd703fe041 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Mon, 4 Apr 2022 16:35:07 +0530 Subject: [PATCH 218/628] Fix pwd mounts --- .../singleinstancedatabase_controller.go | 24 ++++++++++++++----- 1 file changed, 18 insertions(+), 6 deletions(-) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 4a4b01c4..ba087f75 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -454,11 +454,16 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, Spec: corev1.PodSpec{ Volumes: []corev1.Volume{{ - Name: "oracle-pwd", + Name: "oracle-pwd-vol", VolumeSource: corev1.VolumeSource{ Secret: &corev1.SecretVolumeSource { SecretName: m.Spec.AdminPassword.SecretName, Optional: func() *bool { i := true; return &i }(), + Items: []corev1.KeyToPath {{ + Key: m.Spec.AdminPassword.SecretKey, + Path: "oracle_pwd", + }, + }, }, }, }}, @@ -491,8 +496,9 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }(), }, VolumeMounts: []corev1.VolumeMount { { - MountPath: "/run/secrets", - Name: "oracle-pwd", + MountPath: "/run/secrets/oracle_pwd", + Name: "oracle-pwd-vol", + SubPath: "oracle_pwd", }}, Env: func() []corev1.EnvVar { return []corev1.EnvVar{ @@ -574,11 +580,15 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, }, }, { - Name: "oracle-pwd", + Name: "oracle-pwd-vol", VolumeSource: corev1.VolumeSource{ Secret: &corev1.SecretVolumeSource { SecretName: m.Spec.AdminPassword.SecretName, Optional: func() *bool { i := (m.Spec.Edition != "express"); return &i }(), + Items: []corev1.KeyToPath {{ + Key: m.Spec.AdminPassword.SecretKey, + Path: "oracle_pwd", + }}, }, }, }}, @@ -691,8 +701,10 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns Name: "datamount", }, { // This is for express edition DB - MountPath: "/run/secrets", - Name: "oracle-pwd", + MountPath: "/run/secrets/oracle_pwd", + Name: "oracle-pwd-vol", + SubPath: "oracle_pwd", + }}, Env: func() []corev1.EnvVar { // adding XE support, useful for dev/test/CI-CD From 3a584764861b8972cf2d58ae3bf73361f1eb94f7 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Mon, 4 Apr 2022 16:36:09 +0530 Subject: [PATCH 219/628] Remove new line --- controllers/database/singleinstancedatabase_controller.go | 1 - 1 file changed, 1 deletion(-) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index ba087f75..7ffd66b9 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -704,7 +704,6 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns MountPath: "/run/secrets/oracle_pwd", Name: "oracle-pwd-vol", SubPath: "oracle_pwd", - }}, Env: func() []corev1.EnvVar { // adding XE support, useful for dev/test/CI-CD From 26e10a26c4d0c4ec7738c938962add37d5d90797 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Mon, 4 Apr 2022 17:46:51 +0530 Subject: [PATCH 220/628] Set ReadOnly Flag --- controllers/database/singleinstancedatabase_controller.go | 2 ++ 1 file changed, 2 insertions(+) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 7ffd66b9..9f684414 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -497,6 +497,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, VolumeMounts: []corev1.VolumeMount { { MountPath: "/run/secrets/oracle_pwd", + ReadOnly: true, Name: "oracle-pwd-vol", SubPath: "oracle_pwd", }}, @@ -702,6 +703,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, { // This is for express edition DB MountPath: "/run/secrets/oracle_pwd", + ReadOnly: true, Name: "oracle-pwd-vol", SubPath: "oracle_pwd", }}, From 0a6504018585c1c75fad5859de8dded978e64965 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Tue, 5 Apr 2022 17:52:53 +0530 Subject: [PATCH 221/628] Code refactor --- commons/database/utils.go | 10 ++----- .../oraclerestdataservice_controller.go | 20 ++++++------- .../singleinstancedatabase_controller.go | 28 +++++++++---------- 3 files changed, 27 insertions(+), 31 deletions(-) diff --git a/commons/database/utils.go b/commons/database/utils.go index 0a372e40..04699044 100644 --- a/commons/database/utils.go +++ b/commons/database/utils.go @@ -364,7 +364,7 @@ func CheckDBConfig(readyPod corev1.Pod, r client.Reader, config *rest.Config, } else { out, err := ExecCommand(r, config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf("echo -e \"%s\" | %s", CheckModesSQL, GetSqlClient(edition))) + fmt.Sprintf("echo -e \"%s\" | %s", CheckModesSQL, SQLPlusCLI)) if err != nil { log.Error(err, "Error in ExecCommand()") return false, false, false, requeueY @@ -462,7 +462,7 @@ func GetDatabaseVersion(readyPod corev1.Pod, r client.Reader, // ## FIND DATABASES PRESENT IN DG CONFIGURATION out, err := ExecCommand(r, config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf("echo -e \"%s\" | %s", GetVersionSQL, GetSqlClient(edition))) + fmt.Sprintf("echo -e \"%s\" | %s", GetVersionSQL, SQLPlusCLI)) if err != nil { return "", "", err } @@ -488,7 +488,7 @@ func GetDatabaseRole(readyPod corev1.Pod, r client.Reader, log := ctrllog.FromContext(ctx).WithValues("GetDatabaseRole", req.NamespacedName) out, err := ExecCommand(r, config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf("echo -e \"%s\" | %s", GetDatabaseRoleCMD, GetSqlClient(edition))) + fmt.Sprintf("echo -e \"%s\" | %s", GetDatabaseRoleCMD, SQLPlusCLI)) if err != nil { return "", err } @@ -637,10 +637,6 @@ func GetSqlpatchStatus(r client.Reader, config *rest.Config, readyPod corev1.Pod return sqlpatchStatuses[0], splitstr[0], splitstr[1], nil } -func GetSqlClient(edition string) string { - return "sqlplus -s / as sysdba" -} - // Is Source Database On same Cluster func IsSourceDatabaseOnCluster(cloneFrom string) bool { if strings.Contains(cloneFrom, ":") && strings.Contains(cloneFrom, "/") { diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 0fb7ffe9..856f0a19 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -276,7 +276,7 @@ func (r *OracleRestDataServiceReconciler) validateSidbReadiness(m *dbapi.OracleR adminPassword := string(adminPasswordSecret.Data[m.Spec.AdminPassword.SecretKey]) out, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", - fmt.Sprintf("echo -e \"%s\" | %s", fmt.Sprintf(dbcommons.ValidateAdminPassword, adminPassword), dbcommons.GetSqlClient(n.Spec.Edition))) + fmt.Sprintf("echo -e \"%s\" | %s", fmt.Sprintf(dbcommons.ValidateAdminPassword, adminPassword), dbcommons.SQLPlusCLI)) if err != nil { log.Error(err, err.Error()) return requeueY, sidbReadyPod @@ -841,7 +841,7 @@ func (r *OracleRestDataServiceReconciler) cleanupOracleRestDataService(req ctrl. // Get Session id , serial# for ORDS_PUBLIC_USER to kill the sessions out, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf("echo -e \"%s\" | %s ", dbcommons.GetSessionInfoSQL, dbcommons.GetSqlClient(n.Spec.Edition))) + fmt.Sprintf("echo -e \"%s\" | %s ", dbcommons.GetSessionInfoSQL, dbcommons.SQLPlusCLI)) if err != nil { log.Error(err, err.Error()) return err @@ -860,7 +860,7 @@ func (r *OracleRestDataServiceReconciler) cleanupOracleRestDataService(req ctrl. //kill all the sessions with given sid,serial# out, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf("echo -e \"%s\" | %s ", killSessions, dbcommons.GetSqlClient(n.Spec.Edition))) + fmt.Sprintf("echo -e \"%s\" | %s ", killSessions, dbcommons.SQLPlusCLI)) if err != nil { log.Error(err, err.Error()) @@ -931,7 +931,7 @@ func (r *OracleRestDataServiceReconciler) cleanupOracleRestDataService(req ctrl. // Drop Admin Users out, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf("echo -e \"%s\" | %s ", dbcommons.DropAdminUsersSQL, dbcommons.GetSqlClient(n.Spec.Edition))) + fmt.Sprintf("echo -e \"%s\" | %s ", dbcommons.DropAdminUsersSQL, dbcommons.SQLPlusCLI)) if err != nil { log.Error(err, err.Error()) return err @@ -1057,7 +1057,7 @@ func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataS // Alter APEX_LISTENER,APEX_PUBLIC_USER,APEX_REST_PUBLIC_USER out, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, - "bash", "-c", fmt.Sprintf(dbcommons.AlterApexUsers, apexPassword, dbcommons.GetSqlClient(n.Spec.Edition))) + "bash", "-c", fmt.Sprintf(dbcommons.AlterApexUsers, apexPassword, dbcommons.SQLPlusCLI)) if err != nil { log.Info(err.Error()) return requeueY @@ -1071,7 +1071,7 @@ func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataS // Change APEX Admin Password out, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, - "bash", "-c", fmt.Sprintf("echo -e \"%s\" | %s", fmt.Sprintf(dbcommons.ApexAdmin, apexPassword, pdbName), dbcommons.GetSqlClient(n.Spec.Edition))) + "bash", "-c", fmt.Sprintf("echo -e \"%s\" | %s", fmt.Sprintf(dbcommons.ApexAdmin, apexPassword, pdbName), dbcommons.SQLPlusCLI)) if err != nil { log.Info(err.Error()) return requeueY @@ -1249,7 +1249,7 @@ func (r *OracleRestDataServiceReconciler) setupORDS(m *dbapi.OracleRestDataServi cdbAdminPassword := dbcommons.GenerateRandomString(8) // Create PDB , CDB Admin users and grant permissions . ORDS installation on CDB level out, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", - fmt.Sprintf("echo -e \"%s\" | %s", fmt.Sprintf(dbcommons.SetAdminUsersSQL, cdbAdminPassword), dbcommons.GetSqlClient(n.Spec.Edition))) + fmt.Sprintf("echo -e \"%s\" | %s", fmt.Sprintf(dbcommons.SetAdminUsersSQL, cdbAdminPassword), dbcommons.SQLPlusCLI)) if err != nil { log.Error(err, err.Error()) return requeueY @@ -1326,7 +1326,7 @@ func (r *OracleRestDataServiceReconciler) restEnableSchemas(m *dbapi.OracleRestD // Get Pdbs Available availablePDBS, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", - ctx, req, false, "bash", "-c", fmt.Sprintf("echo -e \"%s\" | %s", dbcommons.GetPdbsSQL, dbcommons.GetSqlClient(n.Spec.Edition))) + ctx, req, false, "bash", "-c", fmt.Sprintf("echo -e \"%s\" | %s", dbcommons.GetPdbsSQL, dbcommons.SQLPlusCLI)) if err != nil { log.Error(err, err.Error()) return requeueY @@ -1350,7 +1350,7 @@ func (r *OracleRestDataServiceReconciler) restEnableSchemas(m *dbapi.OracleRestD // Get ORDS Schema status for PDB out, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", - fmt.Sprintf("echo -e \"%s\" | %s", getOrdsSchemaStatus, dbcommons.GetSqlClient(n.Spec.Edition))) + fmt.Sprintf("echo -e \"%s\" | %s", getOrdsSchemaStatus, dbcommons.SQLPlusCLI)) if err != nil { log.Error(err, err.Error()) return requeueY @@ -1396,7 +1396,7 @@ func (r *OracleRestDataServiceReconciler) restEnableSchemas(m *dbapi.OracleRestD // Create users,schemas and grant enableORDS for PDB out, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", - fmt.Sprintf("echo -e \"%s\" | %s", enableORDSSchema, dbcommons.GetSqlClient(n.Spec.Edition))) + fmt.Sprintf("echo -e \"%s\" | %s", enableORDSSchema, dbcommons.SQLPlusCLI)) if err != nil { log.Error(err, err.Error()) return requeueY diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 9f684414..8dbe8366 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1630,7 +1630,7 @@ func (r *SingleInstanceDatabaseReconciler) updateInitParameters(m *dbapi.SingleI out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", fmt.Sprintf(dbcommons.AlterSgaPgaCpuCMD, m.Spec.InitParams.SgaTarget, - m.Spec.InitParams.PgaAggregateTarget, m.Spec.InitParams.CpuCount, dbcommons.GetSqlClient(m.Spec.Edition))) + m.Spec.InitParams.PgaAggregateTarget, m.Spec.InitParams.CpuCount, dbcommons.SQLPlusCLI)) if err != nil { log.Error(err, err.Error()) return requeueY, err @@ -1640,8 +1640,8 @@ func (r *SingleInstanceDatabaseReconciler) updateInitParameters(m *dbapi.SingleI if m.Status.InitParams.Processes != m.Spec.InitParams.Processes { // Altering 'Processes' needs database to be restarted out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", - ctx, req, false, "bash", "-c", fmt.Sprintf(dbcommons.AlterProcessesCMD, m.Spec.InitParams.Processes, dbcommons.GetSqlClient(m.Spec.Edition), - dbcommons.GetSqlClient(m.Spec.Edition))) + ctx, req, false, "bash", "-c", fmt.Sprintf(dbcommons.AlterProcessesCMD, m.Spec.InitParams.Processes, dbcommons.SQLPlusCLI, + dbcommons.SQLPlusCLI)) if err != nil { log.Error(err, err.Error()) return requeueY, err @@ -1650,7 +1650,7 @@ func (r *SingleInstanceDatabaseReconciler) updateInitParameters(m *dbapi.SingleI } out, err = dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", - ctx, req, false, "bash", "-c", fmt.Sprintf(dbcommons.GetInitParamsSQL, dbcommons.GetSqlClient(m.Spec.Edition))) + ctx, req, false, "bash", "-c", fmt.Sprintf(dbcommons.GetInitParamsSQL, dbcommons.SQLPlusCLI)) if err != nil { log.Error(err, err.Error()) return requeueY, err @@ -1713,7 +1713,7 @@ func (r *SingleInstanceDatabaseReconciler) updateDBConfig(m *dbapi.SingleInstanc log.Info(out) out, err = dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf("echo -e \"%s\" | %s", dbcommons.SetDBRecoveryDestSQL, dbcommons.GetSqlClient(m.Spec.Edition))) + fmt.Sprintf("echo -e \"%s\" | %s", dbcommons.SetDBRecoveryDestSQL, dbcommons.SQLPlusCLI)) if err != nil { log.Error(err, err.Error()) return requeueY, err @@ -1722,7 +1722,7 @@ func (r *SingleInstanceDatabaseReconciler) updateDBConfig(m *dbapi.SingleInstanc log.Info(out) out, err = dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf(dbcommons.ArchiveLogTrueCMD, dbcommons.GetSqlClient(m.Spec.Edition))) + fmt.Sprintf(dbcommons.ArchiveLogTrueCMD, dbcommons.SQLPlusCLI)) if err != nil { log.Error(err, err.Error()) return requeueY, err @@ -1734,7 +1734,7 @@ func (r *SingleInstanceDatabaseReconciler) updateDBConfig(m *dbapi.SingleInstanc if m.Spec.ForceLogging && !forceLoggingStatus { out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf("echo -e \"%s\" | %s", dbcommons.ForceLoggingTrueSQL, dbcommons.GetSqlClient(m.Spec.Edition))) + fmt.Sprintf("echo -e \"%s\" | %s", dbcommons.ForceLoggingTrueSQL, dbcommons.SQLPlusCLI)) if err != nil { log.Error(err, err.Error()) return requeueY, err @@ -1750,7 +1750,7 @@ func (r *SingleInstanceDatabaseReconciler) updateDBConfig(m *dbapi.SingleInstanc } if archiveLogStatus { out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf("echo -e \"%s\" | %s", dbcommons.FlashBackTrueSQL, dbcommons.GetSqlClient(m.Spec.Edition))) + fmt.Sprintf("echo -e \"%s\" | %s", dbcommons.FlashBackTrueSQL, dbcommons.SQLPlusCLI)) if err != nil { log.Error(err, err.Error()) return requeueY, err @@ -1775,7 +1775,7 @@ func (r *SingleInstanceDatabaseReconciler) updateDBConfig(m *dbapi.SingleInstanc if !m.Spec.FlashBack && flashBackStatus { out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf("echo -e \"%s\" | %s", dbcommons.FlashBackFalseSQL, dbcommons.GetSqlClient(m.Spec.Edition))) + fmt.Sprintf("echo -e \"%s\" | %s", dbcommons.FlashBackFalseSQL, dbcommons.SQLPlusCLI)) if err != nil { log.Error(err, err.Error()) return requeueY, err @@ -1791,7 +1791,7 @@ func (r *SingleInstanceDatabaseReconciler) updateDBConfig(m *dbapi.SingleInstanc if !flashBackStatus { out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf(dbcommons.ArchiveLogFalseCMD, dbcommons.GetSqlClient(m.Spec.Edition))) + fmt.Sprintf(dbcommons.ArchiveLogFalseCMD, dbcommons.SQLPlusCLI)) if err != nil { log.Error(err, err.Error()) return requeueY, err @@ -1811,7 +1811,7 @@ func (r *SingleInstanceDatabaseReconciler) updateDBConfig(m *dbapi.SingleInstanc } if !m.Spec.ForceLogging && forceLoggingStatus { out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf("echo -e \"%s\" | %s", dbcommons.ForceLoggingFalseSQL, dbcommons.GetSqlClient(m.Spec.Edition))) + fmt.Sprintf("echo -e \"%s\" | %s", dbcommons.ForceLoggingFalseSQL, dbcommons.SQLPlusCLI)) if err != nil { log.Error(err, err.Error()) return requeueY, err @@ -1905,7 +1905,7 @@ func (r *SingleInstanceDatabaseReconciler) installApex(m *dbapi.SingleInstanceDa // Install APEX out, err = dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf(dbcommons.InstallApex, dbcommons.GetSqlClient(m.Spec.Edition))) + fmt.Sprintf(dbcommons.InstallApex, dbcommons.SQLPlusCLI)) if err != nil { log.Info(err.Error()) } @@ -1919,7 +1919,7 @@ func (r *SingleInstanceDatabaseReconciler) installApex(m *dbapi.SingleInstanceDa } out, err = dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf("echo -e \"%s\" | %s", dbcommons.IsApexInstalled, dbcommons.GetSqlClient(m.Spec.Edition))) + fmt.Sprintf("echo -e \"%s\" | %s", dbcommons.IsApexInstalled, dbcommons.SQLPlusCLI)) if err != nil { log.Error(err, err.Error()) return requeueY @@ -1970,7 +1970,7 @@ func (r *SingleInstanceDatabaseReconciler) uninstallApex(m *dbapi.SingleInstance // Uninstall APEX out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf(dbcommons.UninstallApex, dbcommons.GetSqlClient(m.Spec.Edition))) + fmt.Sprintf(dbcommons.UninstallApex, dbcommons.SQLPlusCLI)) if err != nil { log.Info(err.Error()) if !strings.Contains(err.Error(), "catcon.pl: completed") { From 5eab66d5680d4bd38fb678ce556e9ef6f161d1bd Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Wed, 6 Apr 2022 12:40:37 +0530 Subject: [PATCH 222/628] Revert "Merge branch 'master' of orahub.oci.oraclecorp.com:rac-docker-dev/oracle-database-operator" This reverts commit 4cab613fbb9eb6cf9eb2c1b0ac7b8c814c58f6be, reversing changes made to 4eec4066ab86d147c04ebd515cc87bad5ca2cc78. --- commons/autonomousdatabase/reconciler_util.go | 127 ------------------ commons/finalizer/finalizer.go | 122 ----------------- config/samples/adb/autonomousdatabase.yaml | 11 -- ...onomousdatabase_change_admin_password.yaml | 20 --- 4 files changed, 280 deletions(-) delete mode 100644 commons/autonomousdatabase/reconciler_util.go delete mode 100644 commons/finalizer/finalizer.go delete mode 100644 config/samples/adb/autonomousdatabase.yaml delete mode 100644 config/samples/adb/autonomousdatabase_change_admin_password.yaml diff --git a/commons/autonomousdatabase/reconciler_util.go b/commons/autonomousdatabase/reconciler_util.go deleted file mode 100644 index 2f820410..00000000 --- a/commons/autonomousdatabase/reconciler_util.go +++ /dev/null @@ -1,127 +0,0 @@ -/* -** Copyright (c) 2021 Oracle and/or its affiliates. -** -** The Universal Permissive License (UPL), Version 1.0 -** -** Subject to the condition set forth below, permission is hereby granted to any -** person obtaining a copy of this software, associated documentation and/or data -** (collectively the "Software"), free of charge and under any and all copyright -** rights in the Software, and any and all patent rights owned or freely -** licensable by each licensor hereunder covering either (i) the unmodified -** Software as contributed to or provided by such licensor, or (ii) the Larger -** Works (as defined below), to deal in both -** -** (a) the Software, and -** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if -** one is included with the Software (each a "Larger Work" to which the Software -** is contributed by such licensors), -** -** without restriction, including without limitation the rights to copy, create -** derivative works of, display, perform, and distribute the Software and make, -** use, sell, offer for sale, import, export, have made, and have sold the -** Software and the Larger Work(s), and to sublicense the foregoing rights on -** either these or other terms. -** -** This license is subject to the following condition: -** The above copyright notice and either this complete permission notice or at -** a minimum a reference to the UPL must be included in all copies or -** substantial portions of the Software. -** -** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -** SOFTWARE. - */ - -package autonomousdatabase - -import ( - "context" - "fmt" - - corev1 "k8s.io/api/core/v1" - metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" - "k8s.io/apimachinery/pkg/types" - "k8s.io/client-go/util/retry" - "sigs.k8s.io/controller-runtime/pkg/client" - - "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v51/common" - "github.com/oracle/oci-go-sdk/v51/database" - "github.com/oracle/oci-go-sdk/v51/secrets" - - dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" - "github.com/oracle/oracle-database-operator/commons/oci" -) - -// SetStatus sets the status subresource. -func SetStatus(kubeClient client.Client, adb *dbv1alpha1.AutonomousDatabase) error { - return retry.RetryOnConflict(retry.DefaultRetry, func() error { - curADB := &dbv1alpha1.AutonomousDatabase{} - - namespacedName := types.NamespacedName{ - Namespace: adb.GetNamespace(), - Name: adb.GetName(), - } - - if err := kubeClient.Get(context.TODO(), namespacedName, curADB); err != nil { - return err - } - - curADB.Status = adb.Status - return kubeClient.Status().Update(context.TODO(), curADB) - }) -} - -func createWalletSecret(kubeClient client.Client, namespacedName types.NamespacedName, data map[string][]byte) error { - // Create the secret with the wallet data - stringData := map[string]string{} - for key, val := range data { - stringData[key] = string(val) - } - - walletSecret := &corev1.Secret{ - ObjectMeta: metav1.ObjectMeta{ - Namespace: namespacedName.Namespace, - Name: namespacedName.Name, - }, - StringData: stringData, - } - - if err := kubeClient.Create(context.TODO(), walletSecret); err != nil { - return err - } - return nil -} - -func CreateWalletSecret(logger logr.Logger, kubeClient client.Client, dbClient database.DatabaseClient, secretClient secrets.SecretsClient, adb *dbv1alpha1.AutonomousDatabase) error { - // Kube Secret which contains Instance Wallet - walletName := adb.Spec.Details.Wallet.Name - if walletName == nil { - walletName = common.String(adb.GetName() + "-instance-wallet") - } - - // No-op if Wallet is already downloaded - walletNamespacedName := types.NamespacedName{ - Namespace: adb.GetNamespace(), - Name: *walletName, - } - walletSecret := &corev1.Secret{} - if err := kubeClient.Get(context.TODO(), walletNamespacedName, walletSecret); err == nil { - return nil - } - - data, err := oci.GetWallet(logger, kubeClient, dbClient, secretClient, adb) - if err != nil { - return err - } - - if err := createWalletSecret(kubeClient, walletNamespacedName, data); err != nil { - return err - } - logger.Info(fmt.Sprintf("Wallet is stored in the Secret %s", *walletName)) - return nil -} diff --git a/commons/finalizer/finalizer.go b/commons/finalizer/finalizer.go deleted file mode 100644 index bafe110e..00000000 --- a/commons/finalizer/finalizer.go +++ /dev/null @@ -1,122 +0,0 @@ -/* -** Copyright (c) 2021 Oracle and/or its affiliates. -** -** The Universal Permissive License (UPL), Version 1.0 -** -** Subject to the condition set forth below, permission is hereby granted to any -** person obtaining a copy of this software, associated documentation and/or data -** (collectively the "Software"), free of charge and under any and all copyright -** rights in the Software, and any and all patent rights owned or freely -** licensable by each licensor hereunder covering either (i) the unmodified -** Software as contributed to or provided by such licensor, or (ii) the Larger -** Works (as defined below), to deal in both -** -** (a) the Software, and -** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if -** one is included with the Software (each a "Larger Work" to which the Software -** is contributed by such licensors), -** -** without restriction, including without limitation the rights to copy, create -** derivative works of, display, perform, and distribute the Software and make, -** use, sell, offer for sale, import, export, have made, and have sold the -** Software and the Larger Work(s), and to sublicense the foregoing rights on -** either these or other terms. -** -** This license is subject to the following condition: -** The above copyright notice and either this complete permission notice or at -** a minimum a reference to the UPL must be included in all copies or -** substantial portions of the Software. -** -** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -** SOFTWARE. - */ - -package finalizer - -import ( - "context" - "encoding/json" - - "k8s.io/apimachinery/pkg/types" - "sigs.k8s.io/controller-runtime/pkg/client" -) - -// name of our custom finalizer -var finalizerName = "database.oracle.com/dbcsfinalizer" - -// HasFinalizer returns true if the finalizer exists in the object metadata -func HasFinalizer(obj client.Object) bool { - finalizer := obj.GetFinalizers() - return containsString(finalizer, finalizerName) -} - -// Register adds the finalizer and patch the object -func Register(kubeClient client.Client, obj client.Object) error { - finalizer := obj.GetFinalizers() - finalizer = append(finalizer, finalizerName) - return setFinalizer(kubeClient, obj, finalizer) -} - -// Unregister removes the finalizer and patch the object -func Unregister(kubeClient client.Client, obj client.Object) error { - finalizer := obj.GetFinalizers() - finalizer = removeString(finalizer, finalizerName) - return setFinalizer(kubeClient, obj, finalizer) -} - -// Helper functions to check and remove string from a slice of strings. -func containsString(slice []string, s string) bool { - for _, item := range slice { - if item == s { - return true - } - } - return false -} - -func removeString(slice []string, s string) (result []string) { - for _, item := range slice { - if item == s { - continue - } - result = append(result, item) - } - return -} - -type patchValue struct { - Op string `json:"op"` - Path string `json:"path"` - Value interface{} `json:"value"` -} - -func setFinalizer(kubeClient client.Client, dbcs client.Object, finalizer []string) error { - payload := []patchValue{} - - if dbcs.GetFinalizers() == nil { - payload = append(payload, patchValue{ - Op: "replace", - Path: "/metadata/finalizers", - Value: []string{}, - }) - } - - payload = append(payload, patchValue{ - Op: "replace", - Path: "/metadata/finalizers", - Value: finalizer, - }) - - payloadBytes, err := json.Marshal(payload) - if err != nil { - return err - } - - patch := client.RawPatch(types.JSONPatchType, payloadBytes) - return kubeClient.Patch(context.TODO(), dbcs, patch) -} diff --git a/config/samples/adb/autonomousdatabase.yaml b/config/samples/adb/autonomousdatabase.yaml deleted file mode 100644 index c1dfc078..00000000 --- a/config/samples/adb/autonomousdatabase.yaml +++ /dev/null @@ -1,11 +0,0 @@ -# -# Copyright (c) 2021, Oracle and/or its affiliates. -# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. -# -apiVersion: database.oracle.com/v1alpha1 -kind: AutonomousDatabase -metadata: - name: autonomousdatabase-sample -spec: - # Add fields here - foo: bar diff --git a/config/samples/adb/autonomousdatabase_change_admin_password.yaml b/config/samples/adb/autonomousdatabase_change_admin_password.yaml deleted file mode 100644 index 47ad1d65..00000000 --- a/config/samples/adb/autonomousdatabase_change_admin_password.yaml +++ /dev/null @@ -1,20 +0,0 @@ -# -# Copyright (c) 2021, Oracle and/or its affiliates. -# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. -# -apiVersion: database.oracle.com/v1alpha1 -kind: AutonomousDatabase -metadata: - name: autonomousdatabase-sample -spec: - details: - autonomousDatabaseOCID: ocid1.autonomousdatabase... - adminPassword: - # The Name of the secret where you want to hold the password of the ADMIN account. Comment out k8sSecretName and uncomment ociSecretOCID if you pass the admin password using OCI Secret. - k8sSecretName: new-admin-password - # The OCID of the OCI Secret that holds the password of the ADMIN account. It should start with ocid1.vaultsecret... . - # ociSecretOCID: ocid1.vaultsecret... - # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. - ociConfig: - configMapName: oci-cred - secretName: oci-privatekey \ No newline at end of file From 4d2097bf51527983f339fdcd39b0aa71b75d416e Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Wed, 6 Apr 2022 17:52:11 +0530 Subject: [PATCH 223/628] Add prebuilt db support --- .../v1alpha1/singleinstancedatabase_types.go | 1 + .../singleinstancedatabase_webhook.go | 30 +++++++++++-------- .../samples/sidb/singleinstancedatabase.yaml | 1 + .../singleinstancedatabase_controller.go | 24 +++++++-------- 4 files changed, 31 insertions(+), 25 deletions(-) diff --git a/apis/database/v1alpha1/singleinstancedatabase_types.go b/apis/database/v1alpha1/singleinstancedatabase_types.go index 1b20472d..490d7b4a 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_types.go +++ b/apis/database/v1alpha1/singleinstancedatabase_types.go @@ -99,6 +99,7 @@ type SingleInstanceDatabaseImage struct { Version string `json:"version,omitempty"` PullFrom string `json:"pullFrom"` PullSecrets string `json:"pullSecrets,omitempty"` + PrebuiltDB bool `json:"prebuiltDB,omitempty"` } // SingleInsatnceAdminPassword defines the secret containing Admin Password mapped to secretKey for Database diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index a2bbd4d2..e268be55 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** @@ -85,12 +85,7 @@ func (r *SingleInstanceDatabase) Default() { } } - // Pre-built db should have 1 replica only - if r.Spec.Persistence.AccessMode == "" && r.Spec.Replicas > 1 { - r.Spec.Replicas = 1 - } - // TODO(user): fill in your defaulting logic. } // TODO(user): change verbs to "verbs=create;update;delete" if you want to enable deletion validation. @@ -103,13 +98,22 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { singleinstancedatabaselog.Info("validate create", "name", r.Name) var allErrs field.ErrorList - // Pre-built db - if r.Spec.Persistence.AccessMode == "" { - if r.Spec.Sid != "" && r.Spec.Edition != "express" { + if r.Spec.Replicas > 1 { + valMsg := "" + if r.Spec.Edition == "express" { + valMsg = "should be 1 for express edition" + } + if r.Spec.Image.PrebuiltDB { + valMsg = "should be 1 for prebuiltDB" + } + if valMsg != "" { allErrs = append(allErrs, - field.Invalid(field.NewPath("spec").Child("sid"), r.Spec.Sid, - "cannot change sid for prebuilt db")) + field.Invalid(field.NewPath("spec").Child("replicas"), r.Spec.Replicas, valMsg)) } + } + + // Pre-built db + if r.Spec.Image.PrebuiltDB { if r.Spec.Pdbname != "" && r.Spec.Edition != "express" { allErrs = append(allErrs, field.Invalid(field.NewPath("spec").Child("pdbName"), r.Spec.Pdbname, @@ -128,7 +132,7 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { r.Name, allErrs) } - if r.Spec.Persistence.AccessMode != "" && + if !r.Spec.Image.PrebuiltDB && r.Spec.Persistence.AccessMode != "ReadWriteMany" && r.Spec.Persistence.AccessMode != "ReadWriteOnce" { allErrs = append(allErrs, field.Invalid(field.NewPath("spec").Child("persistence"), r.Spec.Persistence.AccessMode, @@ -190,7 +194,7 @@ func (r *SingleInstanceDatabase) ValidateUpdate(oldRuntimeObject runtime.Object) } // Pre-built db - if r.Spec.Persistence.AccessMode == "" { + if r.Spec.Image.PrebuiltDB { return nil } diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index b2bc5b42..8faaf147 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -63,6 +63,7 @@ spec: image: pullFrom: pullSecrets: + prebuiltDB: false ## size : Minimum size of pvc | class : PVC storage Class ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 8dbe8366..c2f04955 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -316,7 +316,7 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab r.Log.Info("Got", "edition", m.Spec.Edition) // Pre-built db - if m.Spec.Persistence.AccessMode == "" { + if m.Spec.Image.PrebuiltDB { return requeueN, nil } @@ -363,7 +363,7 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab } // Validating the secret. Pre-built db doesnt need secret - if m.Spec.Persistence.AccessMode != "" && m.Status.DatafilesCreated != "true" { + if !m.Spec.Image.PrebuiltDB && m.Status.DatafilesCreated != "true" { secret := &corev1.Secret{} err = r.Get(ctx, types.NamespacedName{Name: m.Spec.AdminPassword.SecretName, Namespace: m.Namespace}, secret) if err != nil { @@ -439,7 +439,7 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab //############################################################################# func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleInstanceDatabase, n *dbapi.SingleInstanceDatabase) *corev1.Pod { // Pre-built db, useful for dev/test/CI-CD - if m.Spec.Persistence.AccessMode == "" { + if m.Spec.Image.PrebuiltDB { pod := &corev1.Pod{ TypeMeta: metav1.TypeMeta{ Kind: "Pod", @@ -961,7 +961,7 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplacePVC(ctx context.Contex m *dbapi.SingleInstanceDatabase) (ctrl.Result, error) { // No PVC for Pre-built db - if m.Spec.Persistence.AccessMode == "" { + if m.Spec.Image.PrebuiltDB { return requeueN, nil } log := r.Log.WithValues("createPVC", req.NamespacedName) @@ -1061,7 +1061,7 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex pdbName := strings.ToUpper(m.Spec.Pdbname) sid := m.Spec.Sid - if m.Spec.Persistence.AccessMode == "" { + if m.Spec.Image.PrebuiltDB { sid, pdbName, m.Status.Edition = dbcommons.GetSidPdbEdition(r, r.Config, ctx, req) if sid == "" || pdbName == "" || m.Status.Edition == "" { return requeueN, nil @@ -1205,7 +1205,7 @@ func (r *SingleInstanceDatabaseReconciler) createWallet(m *dbapi.SingleInstanceD } // No Wallet for Pre-built db - if m.Spec.Persistence.AccessMode == "" { + if m.Spec.Image.PrebuiltDB { return requeueN, nil } @@ -1516,7 +1516,7 @@ func (r *SingleInstanceDatabaseReconciler) deleteWallet(m *dbapi.SingleInstanceD } // No Wallet for Pre-built db - if m.Spec.Persistence.AccessMode == "" { + if m.Spec.Image.PrebuiltDB { return requeueN, nil } @@ -1565,7 +1565,7 @@ func (r *SingleInstanceDatabaseReconciler) runDatapatch(m *dbapi.SingleInstanceD } // No Patching for Pre-built db - if m.Spec.Persistence.AccessMode == "" { + if m.Spec.Image.PrebuiltDB { return requeueN, nil } @@ -1620,7 +1620,7 @@ func (r *SingleInstanceDatabaseReconciler) updateInitParameters(m *dbapi.SingleI log := r.Log.WithValues("updateInitParameters", req.NamespacedName) // No InitParameters Updation for Pre-built db - if m.Spec.Persistence.AccessMode == "" { + if m.Spec.Image.PrebuiltDB { return requeueN, nil } @@ -1670,7 +1670,7 @@ func (r *SingleInstanceDatabaseReconciler) updateDBConfig(m *dbapi.SingleInstanc log := r.Log.WithValues("updateDBConfig", req.NamespacedName) // No updateDBConfig for Pre-built db - if m.Spec.Persistence.AccessMode == "" { + if m.Spec.Image.PrebuiltDB { return requeueN, nil } @@ -1868,7 +1868,7 @@ func (r *SingleInstanceDatabaseReconciler) installApex(m *dbapi.SingleInstanceDa log := r.Log.WithValues("installApex", req.NamespacedName) // No APEX for Pre-built db - if m.Spec.Persistence.AccessMode == "" { + if m.Spec.Image.PrebuiltDB { return requeueN } @@ -1954,7 +1954,7 @@ func (r *SingleInstanceDatabaseReconciler) uninstallApex(m *dbapi.SingleInstance log := r.Log.WithValues("uninstallApex", req.NamespacedName) // No APEX for Pre-built db - if m.Spec.Persistence.AccessMode == "" { + if m.Spec.Image.PrebuiltDB { return requeueN } From fa5c4b579c3ded258cc60ee2376ca74f18da30e6 Mon Sep 17 00:00:00 2001 From: "paramdeep.saini@oracle.com" Date: Wed, 6 Apr 2022 15:01:21 -0700 Subject: [PATCH 224/628] Added DBCS Controller --- PROJECT | 9 + README.md | 1 + apis/database/v1alpha1/dbcssystem_types.go | 229 ++ .../v1alpha1/shardingdatabase_types.go | 2 +- .../v1alpha1/zz_generated.deepcopy.go | 288 +++ commons/dbcssystem/dbcs_reconciler.go | 764 ++++++ commons/dbcssystem/dcommon.go | 438 ++++ commons/finalizer/finalizer.go | 122 + .../database.oracle.com_dbcssystems.yaml | 240 ++ config/crd/kustomization.yaml | 3 + .../patches/cainjection_in_dbcssystems.yaml | 8 + .../crd/patches/webhook_in_dbcssystems.yaml | 17 + config/manager/kustomization.yaml | 4 +- config/rbac/dbcssystem_editor_role.yaml | 24 + config/rbac/dbcssystem_viewer_role.yaml | 20 + config/rbac/role.yaml | 44 + .../samples/database_v1alpha1_dbcssystem.yaml | 7 + config/samples/kustomization.yaml | 1 + controllers/database/dbcssystem_controller.go | 302 +++ .../oraclerestdataservice_controller.go | 2 +- .../singleinstancedatabase_controller.go | 23 +- controllers/database/suite_test.go | 3 + docs/dbcs/README.md | 183 ++ .../bind_to_existing_dbcs_system.md | 32 + .../bind_to_existing_dbcs_system.yaml | 8 + ..._to_existing_dbcs_system_sample_output.log | 177 ++ docs/dbcs/provisioning/database_connection.md | 0 .../dbcs_controller_parameters.md | 33 + .../dbcs_service_with_2_node_rac.md | 59 + .../dbcs_service_with_2_node_rac.yaml | 38 + ..._service_with_2_node_rac_sample_output.log | 352 +++ .../dbcs_service_with_all_parameters_asm.md | 58 + .../dbcs_service_with_all_parameters_asm.yaml | 36 + ..._with_all_parameters_asm_sample_output.log | 313 +++ .../dbcs_service_with_all_parameters_lvm.md | 55 + .../dbcs_service_with_all_parameters_lvm.yaml | 34 + ..._with_all_parameters_lvm_sample_output.log | 266 +++ .../dbcs_service_with_minimal_parameters.md | 43 + .../dbcs_service_with_minimal_parameters.yaml | 19 + ..._with_minimal_parameters_sample_output.log | 257 ++ docs/dbcs/provisioning/known_issues.md | 5 + .../scale_down_dbcs_system_shape.md | 45 + .../scale_down_dbcs_system_shape.yaml | 17 + ...e_down_dbcs_system_shape_sample_output.log | 378 +++ .../scale_up_dbcs_system_shape.md | 45 + .../scale_up_dbcs_system_shape.yaml | 17 + ...ale_up_dbcs_system_shape_sample_output.log | 351 +++ docs/dbcs/provisioning/scale_up_storage.md | 45 + docs/dbcs/provisioning/scale_up_storage.yaml | 18 + .../scale_up_storage_sample_output.log | 389 +++ .../provisioning/terminate_dbcs_system.md | 46 + .../provisioning/terminate_dbcs_system.yaml | 9 + .../terminate_dbcs_system_sample_output.log | 556 +++++ docs/dbcs/provisioning/update_license.md | 46 + docs/dbcs/provisioning/update_license.yaml | 18 + .../update_license_sample_output.log | 388 +++ go.sum | 3 + main.go | 10 +- oracle-database-operator.yaml | 2106 +++++++++++++++-- ...autonomousdatabase_controller_bind_test.go | 2 +- 60 files changed, 8834 insertions(+), 174 deletions(-) create mode 100644 apis/database/v1alpha1/dbcssystem_types.go create mode 100644 commons/dbcssystem/dbcs_reconciler.go create mode 100644 commons/dbcssystem/dcommon.go create mode 100644 commons/finalizer/finalizer.go create mode 100644 config/crd/bases/database.oracle.com_dbcssystems.yaml create mode 100644 config/crd/patches/cainjection_in_dbcssystems.yaml create mode 100644 config/crd/patches/webhook_in_dbcssystems.yaml create mode 100644 config/rbac/dbcssystem_editor_role.yaml create mode 100644 config/rbac/dbcssystem_viewer_role.yaml create mode 100644 config/samples/database_v1alpha1_dbcssystem.yaml create mode 100644 controllers/database/dbcssystem_controller.go create mode 100644 docs/dbcs/README.md create mode 100644 docs/dbcs/provisioning/bind_to_existing_dbcs_system.md create mode 100644 docs/dbcs/provisioning/bind_to_existing_dbcs_system.yaml create mode 100644 docs/dbcs/provisioning/bind_to_existing_dbcs_system_sample_output.log create mode 100644 docs/dbcs/provisioning/database_connection.md create mode 100644 docs/dbcs/provisioning/dbcs_controller_parameters.md create mode 100644 docs/dbcs/provisioning/dbcs_service_with_2_node_rac.md create mode 100644 docs/dbcs/provisioning/dbcs_service_with_2_node_rac.yaml create mode 100644 docs/dbcs/provisioning/dbcs_service_with_2_node_rac_sample_output.log create mode 100644 docs/dbcs/provisioning/dbcs_service_with_all_parameters_asm.md create mode 100644 docs/dbcs/provisioning/dbcs_service_with_all_parameters_asm.yaml create mode 100644 docs/dbcs/provisioning/dbcs_service_with_all_parameters_asm_sample_output.log create mode 100644 docs/dbcs/provisioning/dbcs_service_with_all_parameters_lvm.md create mode 100644 docs/dbcs/provisioning/dbcs_service_with_all_parameters_lvm.yaml create mode 100644 docs/dbcs/provisioning/dbcs_service_with_all_parameters_lvm_sample_output.log create mode 100644 docs/dbcs/provisioning/dbcs_service_with_minimal_parameters.md create mode 100644 docs/dbcs/provisioning/dbcs_service_with_minimal_parameters.yaml create mode 100644 docs/dbcs/provisioning/dbcs_service_with_minimal_parameters_sample_output.log create mode 100644 docs/dbcs/provisioning/known_issues.md create mode 100644 docs/dbcs/provisioning/scale_down_dbcs_system_shape.md create mode 100644 docs/dbcs/provisioning/scale_down_dbcs_system_shape.yaml create mode 100644 docs/dbcs/provisioning/scale_down_dbcs_system_shape_sample_output.log create mode 100644 docs/dbcs/provisioning/scale_up_dbcs_system_shape.md create mode 100644 docs/dbcs/provisioning/scale_up_dbcs_system_shape.yaml create mode 100644 docs/dbcs/provisioning/scale_up_dbcs_system_shape_sample_output.log create mode 100644 docs/dbcs/provisioning/scale_up_storage.md create mode 100644 docs/dbcs/provisioning/scale_up_storage.yaml create mode 100644 docs/dbcs/provisioning/scale_up_storage_sample_output.log create mode 100644 docs/dbcs/provisioning/terminate_dbcs_system.md create mode 100644 docs/dbcs/provisioning/terminate_dbcs_system.yaml create mode 100644 docs/dbcs/provisioning/terminate_dbcs_system_sample_output.log create mode 100644 docs/dbcs/provisioning/update_license.md create mode 100644 docs/dbcs/provisioning/update_license.yaml create mode 100644 docs/dbcs/provisioning/update_license_sample_output.log diff --git a/PROJECT b/PROJECT index 3866b50c..9150c705 100644 --- a/PROJECT +++ b/PROJECT @@ -111,4 +111,13 @@ resources: kind: AutonomousContainerDatabase path: github.com/oracle/oracle-database-operator/apis/database/v1alpha1 version: v1alpha1 +- api: + crdVersion: v1beta1 + namespaced: true + controller: true + domain: oracle.com + group: database + kind: DbcsSystem + path: github.com/oracle/oracle-database-operator/apis/database/v1alpha1 + version: v1alpha1 version: "3" diff --git a/README.md b/README.md index 6afd01dc..eb4ef3fa 100644 --- a/README.md +++ b/README.md @@ -85,6 +85,7 @@ The quickstarts are designed for specific database configurations, including: * [Oracle Autonomous Database](./docs/adb/README.md) * [Oracle Database Single Instance configuration](./docs/sidb/README.md) * [Oracle Database configured with Oracle Sharding](./docs/sharding/README.md) +* [Oracle Database Cloud Services](.docs/dbcs/README.md) YAML file templates are available under [`/config/samples`](./config/samples/). You can copy and edit these template files to configure them for your use cases. diff --git a/apis/database/v1alpha1/dbcssystem_types.go b/apis/database/v1alpha1/dbcssystem_types.go new file mode 100644 index 00000000..dd03b28c --- /dev/null +++ b/apis/database/v1alpha1/dbcssystem_types.go @@ -0,0 +1,229 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ +package v1alpha1 + +import ( + "encoding/json" + + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + + dbcsv1 "github.com/oracle/oracle-database-operator/commons/annotations" + + "sigs.k8s.io/controller-runtime/pkg/client" +) + +// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! +// NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized. + +// DbcsSystemSpec defines the desired state of DbcsSystem +type DbcsSystemSpec struct { + DbSystem DbSystemDetails `json:"dbSystem,omitempty"` + Id *string `json:"id,omitempty"` + OCIConfigMap string `json:"ociConfigMap"` + OCISecret string `json:"ociSecret,omitempty"` + HardLink bool `json:"hardLink,omitempty"` +} + +// DbSystemDetails Spec + +type DbSystemDetails struct { + CompartmentId string `json:"compartmentId"` + AvailabilityDomain string `json:"availabilityDomain"` + SubnetId string `json:"subnetId"` + Shape string `json:"shape"` + SshPublicKeys []string `json:"sshPublicKeys"` + HostName string `json:"hostName"` + CpuCoreCount int `json:"cpuCoreCount,omitempty"` + FaultDomains []string `json:"faultDomains,omitempty"` + DisplayName string `json:"displayName,omitempty"` + BackupSubnetId string `json:"backupSubnetId,omitempty"` + TimeZone string `json:"timeZone,omitempty"` + NodeCount *int `json:"nodeCount,omitempty"` + PrivateIp string `json:"privateIp,omitempty"` + Domain string `json:"domain,omitempty"` + InitialDataStorageSizeInGB int `json:"initialDataStorageSizeInGB,omitempty"` + ClusterName string `json:"clusterName,omitempty"` + KmsKeyId string `json:"kmsKeyId,omitempty"` + KmsKeyVersionId string `json:"kmsKeyVersionId,omitempty"` + DbAdminPaswordSecret string `json:"dbAdminPaswordSecret"` + DbName string `json:"dbName,omitempty"` + PdbName string `json:"pdbName,omitempty"` + DbDomain string `json:"dbDomain,omitempty"` + DbUniqueName string `json:"dbUniqueName,omitempty"` + StorageManagement string `json:"storageManagement,omitempty"` + DbVersion string `json:"dbVersion,omitempty"` + DbEdition string `json:"dbEdition,omitempty"` + DiskRedundancy string `json:"diskRedundancy,omitempty"` + DbWorkload string `json:"dbWorkload,omitempty"` + LicenseModel string `json:"licenseModel,omitempty"` + TdeWalletPasswordSecret string `json:"tdeWalletPasswordSecret,omitempty"` + Tags map[string]string `json:"tags,omitempty"` + DbBackupConfig Backupconfig `json:"dbBackupConfig,omitempty"` +} + +// DB Backup COnfig Network Struct +type Backupconfig struct { + AutoBackupEnabled *bool `json:"autoBackupEnabled,omitempty"` + RecoveryWindowsInDays *int `json:"recoveryWindowsInDays,omitempty"` + AutoBackupWindow *string `json:"autoBackupWindow,omitempty"` + BackupDestinationDetails *string `json:"backupDestinationDetails,omitempty"` +} + +// DbcsSystemStatus defines the observed state of DbcsSystem +type DbcsSystemStatus struct { + Id *string `json:"id,omitempty"` + DisplayName string `json:"displayName,omitempty"` + AvailabilityDomain string `json:"availabilityDomain,omitempty"` + SubnetId string `json:"subnetId,omitempty"` + StorageManagement string `json:"storageManagement,omitempty"` + NodeCount int `json:"nodeCount,omitempty"` + CpuCoreCount int `json:"cpuCoreCount,omitempty"` + + DbEdition string `json:"dbEdition,omitempty"` + TimeZone string `json:"timeZone,omitempty"` + DataStoragePercentage *int `json:"dataStoragePercentage,omitempty"` + LicenseModel string `json:"licenseModel,omitempty"` + DataStorageSizeInGBs *int `json:"dataStorageSizeInGBs,omitempty"` + RecoStorageSizeInGB *int `json:"recoStorageSizeInGB,omitempty"` + + Shape *string `json:"shape,omitempty"` + State LifecycleState `json:"state"` + DbInfo []DbStatus `json:"dbInfo,omitempty"` + Network VmNetworkDetails `json:"network,omitempty"` + WorkRequests []DbWorkrequests `json:"workRequests,omitempty"` +} + +// DbcsSystemStatus defines the observed state of DbcsSystem +type DbStatus struct { + Id *string `json:"id,omitempty"` + DbName string `json:"dbName,omitempty"` + DbUniqueName string `json:"dbUniqueName,omitempty"` + DbWorkload string `json:"dbWorkload,omitempty"` + DbHomeId string `json:"dbHomeId,omitempty"` +} + +type DbWorkrequests struct { + OperationType *string `json:"operationType,omitmpty"` + OperationId *string `json:"operationId,omitemty"` + PercentComplete string `json:"percentComplete,omitempty"` + TimeAccepted string `json:"timeAccepted,omitempty"` + TimeStarted string `json:"timeStarted,omitempty"` + TimeFinished string `json:"timeFinished,omitempty"` +} + +type VmNetworkDetails struct { + VcnName *string `json:"vcnName,omitempty"` + SubnetName *string `json:"clientSubnet,omitempty"` + ScanDnsName *string `json:"scanDnsName,omitempty"` + HostName string `json:"hostName,omitempty"` + DomainName string `json:"domainName,omitempty"` + ListenerPort *int `json:"listenerPort,omitempty"` + NetworkSG string `json:"networkSG,omitempty"` +} + +//+kubebuilder:object:root=true +//+kubebuilder:subresource:status + +// DbcsSystem is the Schema for the dbcssystems API +type DbcsSystem struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + Spec DbcsSystemSpec `json:"spec,omitempty"` + Status DbcsSystemStatus `json:"status,omitempty"` +} + +//+kubebuilder:object:root=true + +// DbcsSystemList contains a list of DbcsSystem +type DbcsSystemList struct { + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata,omitempty"` + Items []DbcsSystem `json:"items"` +} + +type LifecycleState string + +const ( + Available LifecycleState = "AVAILABLE" + Failed LifecycleState = "FAILED" + Update LifecycleState = "UPDATING" + Provision LifecycleState = "PROVISIONING" + Terminate LifecycleState = "TERMINATED" +) + +const lastSuccessfulSpec = "lastSuccessfulSpec" + +// GetLastSuccessfulSpec returns spec from the lass successful reconciliation. +// Returns nil, nil if there is no lastSuccessfulSpec. +func (dbcs *DbcsSystem) GetLastSuccessfulSpec() (*DbcsSystemSpec, error) { + val, ok := dbcs.GetAnnotations()[lastSuccessfulSpec] + if !ok { + return nil, nil + } + + specBytes := []byte(val) + sucSpec := DbcsSystemSpec{} + + err := json.Unmarshal(specBytes, &sucSpec) + if err != nil { + return nil, err + } + + return &sucSpec, nil +} + +// UpdateLastSuccessfulSpec updates lastSuccessfulSpec with the current spec. +func (dbcs *DbcsSystem) UpdateLastSuccessfulSpec(kubeClient client.Client) error { + specBytes, err := json.Marshal(dbcs.Spec) + if err != nil { + return err + } + + anns := map[string]string{ + lastSuccessfulSpec: string(specBytes), + } + + // return dbcsv1.SetAnnotations(kubeClient, dbcs, anns) + return dbcsv1.PatchAnnotations(kubeClient, dbcs, anns) + +} + +func init() { + SchemeBuilder.Register(&DbcsSystem{}, &DbcsSystemList{}) +} diff --git a/apis/database/v1alpha1/shardingdatabase_types.go b/apis/database/v1alpha1/shardingdatabase_types.go index 39f74eb1..50058e48 100644 --- a/apis/database/v1alpha1/shardingdatabase_types.go +++ b/apis/database/v1alpha1/shardingdatabase_types.go @@ -266,7 +266,7 @@ const ( // var var KubeConfigOnce sync.Once -const lastSuccessfulSpec = "lastSuccessfulSpec" +// #const lastSuccessfulSpec = "lastSuccessfulSpec" // GetLastSuccessfulSpec returns spec from the lass successful reconciliation. // Returns nil, nil if there is no lastSuccessfulSpec. diff --git a/apis/database/v1alpha1/zz_generated.deepcopy.go b/apis/database/v1alpha1/zz_generated.deepcopy.go index 7be52b26..04d45f54 100644 --- a/apis/database/v1alpha1/zz_generated.deepcopy.go +++ b/apis/database/v1alpha1/zz_generated.deepcopy.go @@ -555,6 +555,41 @@ func (in *AutonomousDatabaseStatus) DeepCopy() *AutonomousDatabaseStatus { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *Backupconfig) DeepCopyInto(out *Backupconfig) { + *out = *in + if in.AutoBackupEnabled != nil { + in, out := &in.AutoBackupEnabled, &out.AutoBackupEnabled + *out = new(bool) + **out = **in + } + if in.RecoveryWindowsInDays != nil { + in, out := &in.RecoveryWindowsInDays, &out.RecoveryWindowsInDays + *out = new(int) + **out = **in + } + if in.AutoBackupWindow != nil { + in, out := &in.AutoBackupWindow, &out.AutoBackupWindow + *out = new(string) + **out = **in + } + if in.BackupDestinationDetails != nil { + in, out := &in.BackupDestinationDetails, &out.BackupDestinationDetails + *out = new(string) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Backupconfig. +func (in *Backupconfig) DeepCopy() *Backupconfig { + if in == nil { + return nil + } + out := new(Backupconfig) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *CDB) DeepCopyInto(out *CDB) { *out = *in @@ -806,6 +841,224 @@ func (in *ConnectionStringSpec) DeepCopy() *ConnectionStringSpec { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DbStatus) DeepCopyInto(out *DbStatus) { + *out = *in + if in.Id != nil { + in, out := &in.Id, &out.Id + *out = new(string) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DbStatus. +func (in *DbStatus) DeepCopy() *DbStatus { + if in == nil { + return nil + } + out := new(DbStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DbSystemDetails) DeepCopyInto(out *DbSystemDetails) { + *out = *in + if in.SshPublicKeys != nil { + in, out := &in.SshPublicKeys, &out.SshPublicKeys + *out = make([]string, len(*in)) + copy(*out, *in) + } + if in.FaultDomains != nil { + in, out := &in.FaultDomains, &out.FaultDomains + *out = make([]string, len(*in)) + copy(*out, *in) + } + if in.NodeCount != nil { + in, out := &in.NodeCount, &out.NodeCount + *out = new(int) + **out = **in + } + if in.Tags != nil { + in, out := &in.Tags, &out.Tags + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + in.DbBackupConfig.DeepCopyInto(&out.DbBackupConfig) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DbSystemDetails. +func (in *DbSystemDetails) DeepCopy() *DbSystemDetails { + if in == nil { + return nil + } + out := new(DbSystemDetails) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DbWorkrequests) DeepCopyInto(out *DbWorkrequests) { + *out = *in + if in.OperationType != nil { + in, out := &in.OperationType, &out.OperationType + *out = new(string) + **out = **in + } + if in.OperationId != nil { + in, out := &in.OperationId, &out.OperationId + *out = new(string) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DbWorkrequests. +func (in *DbWorkrequests) DeepCopy() *DbWorkrequests { + if in == nil { + return nil + } + out := new(DbWorkrequests) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DbcsSystem) DeepCopyInto(out *DbcsSystem) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + in.Spec.DeepCopyInto(&out.Spec) + in.Status.DeepCopyInto(&out.Status) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DbcsSystem. +func (in *DbcsSystem) DeepCopy() *DbcsSystem { + if in == nil { + return nil + } + out := new(DbcsSystem) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *DbcsSystem) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DbcsSystemList) DeepCopyInto(out *DbcsSystemList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]DbcsSystem, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DbcsSystemList. +func (in *DbcsSystemList) DeepCopy() *DbcsSystemList { + if in == nil { + return nil + } + out := new(DbcsSystemList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *DbcsSystemList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DbcsSystemSpec) DeepCopyInto(out *DbcsSystemSpec) { + *out = *in + in.DbSystem.DeepCopyInto(&out.DbSystem) + if in.Id != nil { + in, out := &in.Id, &out.Id + *out = new(string) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DbcsSystemSpec. +func (in *DbcsSystemSpec) DeepCopy() *DbcsSystemSpec { + if in == nil { + return nil + } + out := new(DbcsSystemSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DbcsSystemStatus) DeepCopyInto(out *DbcsSystemStatus) { + *out = *in + if in.Id != nil { + in, out := &in.Id, &out.Id + *out = new(string) + **out = **in + } + if in.DataStoragePercentage != nil { + in, out := &in.DataStoragePercentage, &out.DataStoragePercentage + *out = new(int) + **out = **in + } + if in.DataStorageSizeInGBs != nil { + in, out := &in.DataStorageSizeInGBs, &out.DataStorageSizeInGBs + *out = new(int) + **out = **in + } + if in.RecoStorageSizeInGB != nil { + in, out := &in.RecoStorageSizeInGB, &out.RecoStorageSizeInGB + *out = new(int) + **out = **in + } + if in.Shape != nil { + in, out := &in.Shape, &out.Shape + *out = new(string) + **out = **in + } + if in.DbInfo != nil { + in, out := &in.DbInfo, &out.DbInfo + *out = make([]DbStatus, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } + in.Network.DeepCopyInto(&out.Network) + if in.WorkRequests != nil { + in, out := &in.WorkRequests, &out.WorkRequests + *out = make([]DbWorkrequests, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DbcsSystemStatus. +func (in *DbcsSystemStatus) DeepCopy() *DbcsSystemStatus { + if in == nil { + return nil + } + out := new(DbcsSystemStatus) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *EnvironmentVariable) DeepCopyInto(out *EnvironmentVariable) { *out = *in @@ -1975,6 +2228,41 @@ func (in *TargetSpec) DeepCopy() *TargetSpec { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *VmNetworkDetails) DeepCopyInto(out *VmNetworkDetails) { + *out = *in + if in.VcnName != nil { + in, out := &in.VcnName, &out.VcnName + *out = new(string) + **out = **in + } + if in.SubnetName != nil { + in, out := &in.SubnetName, &out.SubnetName + *out = new(string) + **out = **in + } + if in.ScanDnsName != nil { + in, out := &in.ScanDnsName, &out.ScanDnsName + *out = new(string) + **out = **in + } + if in.ListenerPort != nil { + in, out := &in.ListenerPort, &out.ListenerPort + *out = new(int) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new VmNetworkDetails. +func (in *VmNetworkDetails) DeepCopy() *VmNetworkDetails { + if in == nil { + return nil + } + out := new(VmNetworkDetails) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *WalletSpec) DeepCopyInto(out *WalletSpec) { *out = *in diff --git a/commons/dbcssystem/dbcs_reconciler.go b/commons/dbcssystem/dbcs_reconciler.go new file mode 100644 index 00000000..753a2667 --- /dev/null +++ b/commons/dbcssystem/dbcs_reconciler.go @@ -0,0 +1,764 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package common + +import ( + "context" + "encoding/json" + "errors" + "fmt" + "time" + + "github.com/go-logr/logr" + "github.com/oracle/oci-go-sdk/v63/common" + "github.com/oracle/oci-go-sdk/v63/core" + "github.com/oracle/oci-go-sdk/v63/database" + "github.com/oracle/oci-go-sdk/v63/workrequests" + + "k8s.io/client-go/tools/record" + "k8s.io/client-go/util/retry" + "sigs.k8s.io/controller-runtime/pkg/client" + + databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + "github.com/oracle/oracle-database-operator/commons/annotations" + + corev1 "k8s.io/api/core/v1" + "k8s.io/apimachinery/pkg/types" +) + +func CreateAndGetDbcsId(logger logr.Logger, kubeClient client.Client, dbClient database.DatabaseClient, dbcs *databasev1alpha1.DbcsSystem, nwClient core.VirtualNetworkClient, wrClient workrequests.WorkRequestClient) (string, error) { + + //var provisionedDbcsSystemId string + ctx := context.TODO() + // Get DB System Details + dbcsDetails := database.LaunchDbSystemDetails{} + // Get the admin password from OCI key + sshPublicKeys, err := getPublicSSHKey(kubeClient, dbcs) + if err != nil { + return "", err + } + // Get Db SystemOption + dbSystemReq := GetDBSystemopts(dbcs) + licenceModel := getLicenceModel(dbcs) + + if dbcs.Spec.DbSystem.ClusterName != "" { + dbcsDetails.ClusterName = &dbcs.Spec.DbSystem.ClusterName + } + + if dbcs.Spec.DbSystem.TimeZone != "" { + dbcsDetails.TimeZone = &dbcs.Spec.DbSystem.TimeZone + } + // Get DB Home Details + dbHomeReq, err := GetDbHomeDetails(kubeClient, dbClient, dbcs) + if err != nil { + return "", err + } + //tenancyOcid, _ := provider.TenancyOCID() + dbcsDetails.AvailabilityDomain = common.String(dbcs.Spec.DbSystem.AvailabilityDomain) + dbcsDetails.CompartmentId = common.String(dbcs.Spec.DbSystem.CompartmentId) + dbcsDetails.SubnetId = common.String(dbcs.Spec.DbSystem.SubnetId) + dbcsDetails.Shape = common.String(dbcs.Spec.DbSystem.Shape) + if dbcs.Spec.DbSystem.DisplayName != "" { + dbcsDetails.DisplayName = common.String(dbcs.Spec.DbSystem.DisplayName) + } + dbcsDetails.SshPublicKeys = []string{sshPublicKeys} + dbcsDetails.Hostname = common.String(dbcs.Spec.DbSystem.HostName) + dbcsDetails.CpuCoreCount = common.Int(dbcs.Spec.DbSystem.CpuCoreCount) + //dbcsDetails.SourceDbSystemId = common.String(r.tenancyOcid) + dbcsDetails.NodeCount = common.Int(GetNodeCount(dbcs)) + dbcsDetails.InitialDataStorageSizeInGB = common.Int(GetInitialStorage(dbcs)) + dbcsDetails.DbSystemOptions = &dbSystemReq + dbcsDetails.DbHome = &dbHomeReq + dbcsDetails.DatabaseEdition = GetDBEdition(dbcs) + dbcsDetails.DiskRedundancy = GetDBbDiskRedundancy(dbcs) + dbcsDetails.LicenseModel = database.LaunchDbSystemDetailsLicenseModelEnum(licenceModel) + if len(dbcs.Spec.DbSystem.Tags) != 0 { + dbcsDetails.FreeformTags = dbcs.Spec.DbSystem.Tags + } + + req := database.LaunchDbSystemRequest{LaunchDbSystemDetails: dbcsDetails} + + // Send the request using the service client + resp, err := dbClient.LaunchDbSystem(ctx, req) + if err != nil { + return " ", err + } + + dbcs.Spec.Id = resp.DbSystem.Id + // Change the phase to "Provisioning" + if statusErr := SetLifecycleState(kubeClient, dbClient, dbcs, databasev1alpha1.Provision, nwClient, wrClient); statusErr != nil { + return "", statusErr + } + // Check the State + _, err = CheckResourceState(logger, dbClient, *resp.DbSystem.Id, string(databasev1alpha1.Provision), string(databasev1alpha1.Available)) + if err != nil { + return "", err + } + + return *resp.DbSystem.Id, nil +} + +// Sync the DbcsSystem Database details + +// Get admin password from Secret then OCI valut secret +func GetAdminPassword(kubeClient client.Client, dbcs *databasev1alpha1.DbcsSystem) (string, error) { + if dbcs.Spec.DbSystem.DbAdminPaswordSecret != "" { + // Get the Admin Secret + adminSecret := &corev1.Secret{} + err := kubeClient.Get(context.TODO(), types.NamespacedName{ + Namespace: dbcs.GetNamespace(), + Name: dbcs.Spec.DbSystem.DbAdminPaswordSecret, + }, adminSecret) + + if err != nil { + return "", err + } + + // Get the admin password + key := "admin-password" + if val, ok := adminSecret.Data[key]; ok { + return string(val), nil + } else { + msg := "secret item not found: admin-password" + return "", errors.New(msg) + } + } + return "", errors.New("should provide either a Secret name or a Valut Secret ID") +} + +// Get admin password from Secret then OCI valut secret +func GetTdePassword(kubeClient client.Client, dbcs *databasev1alpha1.DbcsSystem) (string, error) { + if dbcs.Spec.DbSystem.TdeWalletPasswordSecret != "" { + // Get the Admin Secret + tdeSecret := &corev1.Secret{} + err := kubeClient.Get(context.TODO(), types.NamespacedName{ + Namespace: dbcs.GetNamespace(), + Name: dbcs.Spec.DbSystem.TdeWalletPasswordSecret, + }, tdeSecret) + + if err != nil { + return "", err + } + + // Get the admin password + key := "tde-password" + if val, ok := tdeSecret.Data[key]; ok { + return string(val), nil + } else { + msg := "secret item not found: tde-password" + return "", errors.New(msg) + } + } + return "", errors.New("should provide either a Secret name or a Valut Secret ID") +} + +// Get admin password from Secret then OCI valut secret +func getPublicSSHKey(kubeClient client.Client, dbcs *databasev1alpha1.DbcsSystem) (string, error) { + if dbcs.Spec.DbSystem.SshPublicKeys[0] != "" { + // Get the Admin Secret + sshkeysecret := &corev1.Secret{} + err := kubeClient.Get(context.TODO(), types.NamespacedName{ + Namespace: dbcs.GetNamespace(), + Name: dbcs.Spec.DbSystem.SshPublicKeys[0], + }, sshkeysecret) + + if err != nil { + return "", err + } + + // Get the admin password` + key := "publickey" + if val, ok := sshkeysecret.Data[key]; ok { + return string(val), nil + } else { + msg := "secret item not found: " + return "", errors.New(msg) + } + } + return "", errors.New("should provide either a Secret name or a Valut Secret ID") +} + +// Delete DbcsSystem System +func DeleteDbcsSystemSystem(dbClient database.DatabaseClient, Id string) error { + + dbcsId := Id + + dbcsReq := database.TerminateDbSystemRequest{ + DbSystemId: &dbcsId, + } + + _, err := dbClient.TerminateDbSystem(context.TODO(), dbcsReq) + if err != nil { + return err + } + + return nil +} + +// SetLifecycleState set status.state of the reosurce. +func SetLifecycleState(kubeClient client.Client, dbClient database.DatabaseClient, dbcs *databasev1alpha1.DbcsSystem, state databasev1alpha1.LifecycleState, nwClient core.VirtualNetworkClient, wrClient workrequests.WorkRequestClient) error { + return retry.RetryOnConflict(retry.DefaultRetry, func() error { + dbcs.Status.State = state + // Set the status + if statusErr := SetDBCSStatus(dbClient, dbcs, nwClient, wrClient); statusErr != nil { + return statusErr + } + if err := kubeClient.Status().Update(context.TODO(), dbcs); err != nil { + return err + } + + return nil + }) +} + +// SetDBCSSystem LifeCycle state when state is provisioning + +func SetDBCSDatabaseLifecycleState(logger logr.Logger, kubeClient client.Client, dbClient database.DatabaseClient, dbcs *databasev1alpha1.DbcsSystem, nwClient core.VirtualNetworkClient, wrClient workrequests.WorkRequestClient) error { + + dbcsId := *dbcs.Spec.Id + + dbcsReq := database.GetDbSystemRequest{ + DbSystemId: &dbcsId, + } + + resp, err := dbClient.GetDbSystem(context.TODO(), dbcsReq) + if err != nil { + return err + } + + // Return if the desired lifecycle state is the same as the current lifecycle state + if string(dbcs.Status.State) == string(resp.LifecycleState) { + return nil + } else if string(resp.LifecycleState) == string(databasev1alpha1.Available) { + // Change the phase to "Available" + if statusErr := SetLifecycleState(kubeClient, dbClient, dbcs, databasev1alpha1.Available, nwClient, wrClient); statusErr != nil { + return statusErr + } + } else if string(resp.LifecycleState) == string(databasev1alpha1.Provision) { + // Change the phase to "Provisioning" + if statusErr := SetLifecycleState(kubeClient, dbClient, dbcs, databasev1alpha1.Provision, nwClient, wrClient); statusErr != nil { + return statusErr + } + // Check the State + _, err = CheckResourceState(logger, dbClient, *resp.DbSystem.Id, string(databasev1alpha1.Provision), string(databasev1alpha1.Available)) + if err != nil { + return err + } + } else if string(resp.LifecycleState) == string(databasev1alpha1.Update) { + // Change the phase to "Updating" + if statusErr := SetLifecycleState(kubeClient, dbClient, dbcs, databasev1alpha1.Update, nwClient, wrClient); statusErr != nil { + return statusErr + } + // Check the State + _, err = CheckResourceState(logger, dbClient, *resp.DbSystem.Id, string(databasev1alpha1.Update), string(databasev1alpha1.Available)) + if err != nil { + return err + } + } else if string(resp.LifecycleState) == string(databasev1alpha1.Failed) { + // Change the phase to "Updating" + if statusErr := SetLifecycleState(kubeClient, dbClient, dbcs, databasev1alpha1.Failed, nwClient, wrClient); statusErr != nil { + return statusErr + } + return fmt.Errorf("DbSystem is in Failed State") + } else if string(resp.LifecycleState) == string(databasev1alpha1.Terminated) { + // Change the phase to "Terminated" + if statusErr := SetLifecycleState(kubeClient, dbClient, dbcs, databasev1alpha1.Terminate, nwClient, wrClient); statusErr != nil { + return statusErr + } + } + return nil +} + +func GetDbSystemId(logger logr.Logger, dbClient database.DatabaseClient, dbcs *databasev1alpha1.DbcsSystem) error { + dbcsId := *dbcs.Spec.Id + + dbcsReq := database.GetDbSystemRequest{ + DbSystemId: &dbcsId, + } + + response, err := dbClient.GetDbSystem(context.TODO(), dbcsReq) + if err != nil { + return err + } + + dbcs.Spec.DbSystem.CompartmentId = *response.CompartmentId + if response.DisplayName != nil { + dbcs.Spec.DbSystem.DisplayName = *response.DisplayName + } + + if response.Hostname != nil { + dbcs.Spec.DbSystem.HostName = *response.Hostname + } + if response.CpuCoreCount != nil { + dbcs.Spec.DbSystem.CpuCoreCount = *response.CpuCoreCount + } + dbcs.Spec.DbSystem.NodeCount = response.NodeCount + if response.ClusterName != nil { + dbcs.Spec.DbSystem.ClusterName = *response.ClusterName + } + //dbcs.Spec.DbSystem.DbUniqueName = *response.DbUniqueName + if string(response.DbSystem.DatabaseEdition) != "" { + dbcs.Spec.DbSystem.DbEdition = string(response.DatabaseEdition) + } + if string(response.DiskRedundancy) != "" { + dbcs.Spec.DbSystem.DiskRedundancy = string(response.DiskRedundancy) + } + + //dbcs.Spec.DbSystem.DbVersion = *response. + + if response.BackupSubnetId != nil { + dbcs.Spec.DbSystem.BackupSubnetId = *response.BackupSubnetId + } + dbcs.Spec.DbSystem.Shape = *response.Shape + dbcs.Spec.DbSystem.SshPublicKeys = []string(response.SshPublicKeys) + if response.FaultDomains != nil { + dbcs.Spec.DbSystem.FaultDomains = []string(response.FaultDomains) + } + dbcs.Spec.DbSystem.SubnetId = *response.SubnetId + dbcs.Spec.DbSystem.AvailabilityDomain = *response.AvailabilityDomain + + err = PopulateDBDetails(logger, dbClient, dbcs) + if err != nil { + logger.Info("Error Occurred while collecting the DB details") + return err + } + return nil +} + +func PopulateDBDetails(logger logr.Logger, dbClient database.DatabaseClient, dbcs *databasev1alpha1.DbcsSystem) error { + + listDbHomeRsp, err := GetListDbHomeRsp(logger, dbClient, dbcs) + if err != nil { + logger.Info("Error Occurred while getting List of DBHomes") + return err + } + dbHomeId := listDbHomeRsp.Items[0].Id + listDBRsp, err := GetListDatabaseRsp(logger, dbClient, dbcs, *dbHomeId) + if err != nil { + logger.Info("Error Occurred while getting List of Databases") + return err + } + + dbcs.Spec.DbSystem.DbName = *listDBRsp.Items[0].DbName + dbcs.Spec.DbSystem.DbUniqueName = *listDBRsp.Items[0].DbUniqueName + dbcs.Spec.DbSystem.DbVersion = *listDbHomeRsp.Items[0].DbVersion + + return nil +} + +func GetListDbHomeRsp(logger logr.Logger, dbClient database.DatabaseClient, dbcs *databasev1alpha1.DbcsSystem) (database.ListDbHomesResponse, error) { + + dbcsId := *dbcs.Spec.Id + CompartmentId := dbcs.Spec.DbSystem.CompartmentId + + dbHomeReq := database.ListDbHomesRequest{ + DbSystemId: &dbcsId, + CompartmentId: &CompartmentId, + } + + response, err := dbClient.ListDbHomes(context.TODO(), dbHomeReq) + if err != nil { + return database.ListDbHomesResponse{}, err + } + + return response, nil +} + +func GetListDatabaseRsp(logger logr.Logger, dbClient database.DatabaseClient, dbcs *databasev1alpha1.DbcsSystem, dbHomeId string) (database.ListDatabasesResponse, error) { + + CompartmentId := dbcs.Spec.DbSystem.CompartmentId + + dbReq := database.ListDatabasesRequest{ + DbHomeId: &dbHomeId, + CompartmentId: &CompartmentId, + } + + response, err := dbClient.ListDatabases(context.TODO(), dbReq) + if err != nil { + return database.ListDatabasesResponse{}, err + } + + return response, nil +} + +func UpdateDbcsSystemIdInst(log logr.Logger, dbClient database.DatabaseClient, dbcs *databasev1alpha1.DbcsSystem, kubeClient client.Client, nwClient core.VirtualNetworkClient, wrClient workrequests.WorkRequestClient) error { + //logger := log.WithName("UpdateDbcsSystemInstance") + + updateFlag := false + updateDbcsDetails := database.UpdateDbSystemDetails{} + oldSpec, err := dbcs.GetLastSuccessfulSpec() + if err != nil { + return err + } + + if dbcs.Spec.DbSystem.CpuCoreCount > 0 && dbcs.Spec.DbSystem.CpuCoreCount != oldSpec.DbSystem.CpuCoreCount { + updateDbcsDetails.CpuCoreCount = common.Int(dbcs.Spec.DbSystem.CpuCoreCount) + updateFlag = true + } + if dbcs.Spec.DbSystem.Shape != "" && dbcs.Spec.DbSystem.Shape != oldSpec.DbSystem.Shape { + updateDbcsDetails.Shape = common.String(dbcs.Spec.DbSystem.Shape) + updateFlag = true + } + + if dbcs.Spec.DbSystem.LicenseModel != "" && dbcs.Spec.DbSystem.LicenseModel != oldSpec.DbSystem.LicenseModel { + licenceModel := getLicenceModel(dbcs) + updateDbcsDetails.LicenseModel = database.UpdateDbSystemDetailsLicenseModelEnum(licenceModel) + updateFlag = true + } + + if dbcs.Spec.DbSystem.InitialDataStorageSizeInGB != 0 && dbcs.Spec.DbSystem.InitialDataStorageSizeInGB != oldSpec.DbSystem.InitialDataStorageSizeInGB { + updateDbcsDetails.DataStorageSizeInGBs = &dbcs.Spec.DbSystem.InitialDataStorageSizeInGB + updateFlag = true + } + + if updateFlag { + updateDbcsRequest := database.UpdateDbSystemRequest{ + DbSystemId: common.String(*dbcs.Spec.Id), + UpdateDbSystemDetails: updateDbcsDetails, + } + + if _, err := dbClient.UpdateDbSystem(context.TODO(), updateDbcsRequest); err != nil { + return err + } + + // Change the phase to "Provisioning" + if statusErr := SetLifecycleState(kubeClient, dbClient, dbcs, databasev1alpha1.Update, nwClient, wrClient); statusErr != nil { + return statusErr + } + // Check the State + _, err = CheckResourceState(log, dbClient, *dbcs.Spec.Id, "UPDATING", "AVAILABLE") + if err != nil { + return err + } + + } + + return nil +} + +func UpdateDbcsSystemId(kubeClient client.Client, dbcs *databasev1alpha1.DbcsSystem) error { + payload := []annotations.PatchValue{{ + Op: "replace", + Path: "/spec/details", + Value: dbcs.Spec.DbSystem, + }} + payloadBytes, err := json.Marshal(payload) + if err != nil { + return err + } + + patch := client.RawPatch(types.JSONPatchType, payloadBytes) + return kubeClient.Patch(context.TODO(), dbcs, patch) +} + +func CheckResourceState(logger logr.Logger, dbClient database.DatabaseClient, Id string, currentState string, expectedState string) (string, error) { + // The database OCID is not available when the provisioning is onging. + // Retry until the new DbcsSystem is ready. + // Retry up to 18 times every 10 seconds. + + var state string + var err error + for { + state, err = GetResourceState(logger, dbClient, Id) + if err != nil { + logger.Info("Error occurred while collecting the resource life cycle state") + return "", err + } + if string(state) == expectedState { + break + } else if string(state) == currentState { + logger.Info("DB System current state is still:" + string(state) + ". Sleeping for 60 seconds.") + time.Sleep(60 * time.Second) + continue + } else { + msg := "DB System current state " + string(state) + " is not matching " + expectedState + logger.Info(msg) + return "", errors.New(msg) + } + } + + return "", nil +} + +func GetResourceState(logger logr.Logger, dbClient database.DatabaseClient, Id string) (string, error) { + + dbcsId := Id + dbcsReq := database.GetDbSystemRequest{ + DbSystemId: &dbcsId, + } + + response, err := dbClient.GetDbSystem(context.TODO(), dbcsReq) + if err != nil { + return "", err + } + + state := string(response.LifecycleState) + + return state, nil +} + +func SetDBCSStatus(dbClient database.DatabaseClient, dbcs *databasev1alpha1.DbcsSystem, nwClient core.VirtualNetworkClient, wrClient workrequests.WorkRequestClient) error { + + dbcsId := *dbcs.Spec.Id + + dbcsReq := database.GetDbSystemRequest{ + DbSystemId: &dbcsId, + } + + resp, err := dbClient.GetDbSystem(context.TODO(), dbcsReq) + if err != nil { + return err + } + + dbcs.Status.AvailabilityDomain = *resp.AvailabilityDomain + dbcs.Status.CpuCoreCount = *resp.CpuCoreCount + dbcs.Status.DataStoragePercentage = resp.DataStoragePercentage + dbcs.Status.DataStorageSizeInGBs = resp.DataStorageSizeInGBs + dbcs.Status.DbEdition = string(resp.DatabaseEdition) + dbcs.Status.DisplayName = *resp.DisplayName + dbcs.Status.LicenseModel = string(resp.LicenseModel) + dbcs.Status.RecoStorageSizeInGB = resp.RecoStorageSizeInGB + dbcs.Status.NodeCount = *resp.NodeCount + dbcs.Status.StorageManagement = string(resp.DbSystemOptions.StorageManagement) + dbcs.Status.Shape = resp.Shape + dbcs.Status.Id = resp.Id + dbcs.Status.SubnetId = *resp.SubnetId + dbcs.Status.TimeZone = *resp.TimeZone + dbcs.Status.LicenseModel = string(resp.LicenseModel) + dbcs.Status.Network.ScanDnsName = resp.ScanDnsName + dbcs.Status.Network.ListenerPort = resp.ListenerPort + dbcs.Status.Network.HostName = *resp.Hostname + dbcs.Status.Network.DomainName = *resp.Domain + + sname, vcnId, err := getSubnetName(*resp.SubnetId, nwClient) + + if err == nil { + dbcs.Status.Network.SubnetName = sname + vcnName, err := getVcnName(vcnId, nwClient) + + if err == nil { + dbcs.Status.Network.VcnName = vcnName + } + + } + + // Work Request Ststaus + dbWorkRequest := databasev1alpha1.DbWorkrequests{} + + dbWorks, err := getWorkRequest(*resp.OpcRequestId, wrClient, dbcs) + if err == nil { + for _, dbWork := range dbWorks { + //status := checkValue(dbcs, dbWork.Id) + // if status != 0 { + dbWorkRequest.OperationId = dbWork.Id + dbWorkRequest.OperationType = dbWork.OperationType + dbWorkRequest.PercentComplete = fmt.Sprint(*dbWork.PercentComplete) //strconv.FormatFloat(dbWork.PercentComplete, 'E', -1, 32) + if dbWork.TimeAccepted != nil { + dbWorkRequest.TimeAccepted = dbWork.TimeAccepted.String() + } + if dbWork.TimeFinished != nil { + dbWorkRequest.TimeFinished = dbWork.TimeFinished.String() + } + if dbWork.TimeStarted != nil { + dbWorkRequest.TimeStarted = dbWork.TimeStarted.String() + } + + if dbWorkRequest != (databasev1alpha1.DbWorkrequests{}) { + status := checkValue(dbcs, dbWork.Id) + if status == 0 { + dbcs.Status.WorkRequests = append(dbcs.Status.WorkRequests, dbWorkRequest) + dbWorkRequest = databasev1alpha1.DbWorkrequests{} + } else { + setValue(dbcs, dbWorkRequest) + } + } + //} + } + } + + // DB Home Status + dbcs.Status.DbInfo = dbcs.Status.DbInfo[:0] + dbStatus := databasev1alpha1.DbStatus{} + + dbHomes, err := getDbHomeList(dbClient, dbcs) + + if err == nil { + for _, dbHome := range dbHomes { + dbDetails, err := getDList(dbClient, dbcs, dbHome.Id) + for _, dbDetail := range dbDetails { + if err == nil { + dbStatus.Id = dbDetail.Id + dbStatus.DbHomeId = *dbDetail.DbHomeId + dbStatus.DbName = *dbDetail.DbName + dbStatus.DbUniqueName = *dbDetail.DbUniqueName + dbStatus.DbWorkload = *dbDetail.DbWorkload + } + dbcs.Status.DbInfo = append(dbcs.Status.DbInfo, dbStatus) + dbStatus = databasev1alpha1.DbStatus{} + } + } + } + return nil +} + +func getDbHomeList(dbClient database.DatabaseClient, dbcs *databasev1alpha1.DbcsSystem) ([]database.DbHomeSummary, error) { + + var items []database.DbHomeSummary + dbcsId := *dbcs.Spec.Id + + dbcsReq := database.ListDbHomesRequest{ + DbSystemId: &dbcsId, + CompartmentId: &dbcs.Spec.DbSystem.CompartmentId, + } + + resp, err := dbClient.ListDbHomes(context.TODO(), dbcsReq) + if err != nil { + return items, err + } + + return resp.Items, nil +} + +func getDList(dbClient database.DatabaseClient, dbcs *databasev1alpha1.DbcsSystem, dbHomeId *string) ([]database.DatabaseSummary, error) { + + dbcsId := *dbcs.Spec.Id + var items []database.DatabaseSummary + dbcsReq := database.ListDatabasesRequest{ + SystemId: &dbcsId, + CompartmentId: &dbcs.Spec.DbSystem.CompartmentId, + DbHomeId: dbHomeId, + } + + resp, err := dbClient.ListDatabases(context.TODO(), dbcsReq) + if err != nil { + return items, err + } + + return resp.Items, nil +} + +func getSubnetName(subnetId string, nwClient core.VirtualNetworkClient) (*string, *string, error) { + + req := core.GetSubnetRequest{SubnetId: common.String(subnetId)} + + // Send the request using the service client + resp, err := nwClient.GetSubnet(context.Background(), req) + + if err != nil { + return nil, nil, err + } + // Retrieve value from the response. + + return resp.DisplayName, resp.VcnId, nil +} + +func getVcnName(vcnId *string, nwClient core.VirtualNetworkClient) (*string, error) { + + req := core.GetVcnRequest{VcnId: common.String(*vcnId)} + + // Send the request using the service client + resp, err := nwClient.GetVcn(context.Background(), req) + + if err != nil { + return nil, err + } + // Retrieve value from the response. + + return resp.DisplayName, nil +} + +// =========== validate Specs ============ +func ValidateSpex(logger logr.Logger, kubeClient client.Client, dbClient database.DatabaseClient, dbcs *databasev1alpha1.DbcsSystem, nwClient core.VirtualNetworkClient, eRecord record.EventRecorder) error { + + //var str1 string + var eventMsg string + var eventErr string = "Spec Error" + lastSuccSpec, err := dbcs.GetLastSuccessfulSpec() + if err != nil { + return err + } + // Check if last Successful update nil or not + if lastSuccSpec == nil { + if dbcs.Spec.DbSystem.DbVersion != "" { + _, err = GetDbLatestVersion(dbClient, dbcs, "") + if err != nil { + eventMsg = "DBCS CRD resource " + GetFmtStr(dbcs.Name) + " DbVersion " + GetFmtStr(dbcs.Spec.DbSystem.DbVersion) + " is not matching available DB releases." + eRecord.Eventf(dbcs, corev1.EventTypeWarning, eventErr, eventMsg) + return err + } + } else { + eventMsg = "DBCS CRD resource " + "DbVersion " + GetFmtStr(dbcs.Name) + GetFmtStr("dbcs.Spec.DbSystem.DbVersion") + " cannot be a empty string." + eRecord.Eventf(dbcs, corev1.EventTypeWarning, eventErr, eventMsg) + return err + } + if dbcs.Spec.DbSystem.DbWorkload != "" { + _, err = getDbWorkLoadType(dbcs) + if err != nil { + eventMsg = "DBCS CRD resource " + GetFmtStr(dbcs.Name) + " DbWorkload " + GetFmtStr(dbcs.Spec.DbSystem.DbWorkload) + " is not matching the DBworkload type OLTP|DSS." + eRecord.Eventf(dbcs, corev1.EventTypeWarning, eventErr, eventMsg) + return err + } + } else { + eventMsg = "DBCS CRD resource " + "DbWorkload " + GetFmtStr(dbcs.Name) + GetFmtStr("dbcs.Spec.DbSystem.DbWorkload") + " cannot be a empty string." + eRecord.Eventf(dbcs, corev1.EventTypeWarning, eventErr, eventMsg) + return err + } + + if dbcs.Spec.DbSystem.NodeCount != nil { + if *dbcs.Spec.DbSystem.NodeCount == 1 { + } else if *dbcs.Spec.DbSystem.NodeCount == 2 { + } else { + eventMsg = "DBCS CRD resource " + "NodeCount " + GetFmtStr(dbcs.Name) + GetFmtStr("dbcs.Spec.DbSystem.NodeCount") + " can be either 1 or 2." + eRecord.Eventf(dbcs, corev1.EventTypeWarning, eventErr, eventMsg) + return err + } + } + + } else { + if lastSuccSpec.DbSystem.DbVersion != dbcs.Spec.DbSystem.DbVersion { + eventMsg = "DBCS CRD resource " + "DbVersion " + GetFmtStr(dbcs.Name) + GetFmtStr("dbcs.Spec.DbSystem.DbVersion") + " cannot be a empty string." + eRecord.Eventf(dbcs, corev1.EventTypeWarning, eventErr, eventMsg) + return err + } + + } + + return nil + +} diff --git a/commons/dbcssystem/dcommon.go b/commons/dbcssystem/dcommon.go new file mode 100644 index 00000000..51dedb00 --- /dev/null +++ b/commons/dbcssystem/dcommon.go @@ -0,0 +1,438 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package common + +import ( + "context" + "fmt" + "strconv" + "strings" + + "github.com/oracle/oci-go-sdk/v63/common" + "github.com/oracle/oci-go-sdk/v63/database" + "github.com/oracle/oci-go-sdk/v63/workrequests" + "sigs.k8s.io/controller-runtime/pkg/client" + + databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" +) + +func GetDbHomeDetails(kubeClient client.Client, dbClient database.DatabaseClient, dbcs *databasev1alpha1.DbcsSystem) (database.CreateDbHomeDetails, error) { + + dbHomeDetails := database.CreateDbHomeDetails{} + + dbHomeReq, err := GetDbLatestVersion(dbClient, dbcs, "") + if err != nil { + return database.CreateDbHomeDetails{}, err + } + dbHomeDetails.DbVersion = &dbHomeReq + + dbDetailsReq, err := GetDBDetails(kubeClient, dbcs) + if err != nil { + return database.CreateDbHomeDetails{}, err + } + + dbHomeDetails.Database = &dbDetailsReq + + return dbHomeDetails, nil +} + +func GetDbLatestVersion(dbClient database.DatabaseClient, dbcs *databasev1alpha1.DbcsSystem, dbSystemId string) (string, error) { + + //var provisionedDbcsSystemId string + ctx := context.TODO() + var version database.DbVersionSummary + var sFlag int = 0 + var val int + + dbVersionReq := database.ListDbVersionsRequest{} + if dbSystemId != "" { + dbVersionReq.DbSystemId = common.String(dbSystemId) + } + + dbVersionReq.IsDatabaseSoftwareImageSupported = common.Bool(true) + dbVersionReq.IsUpgradeSupported = common.Bool(false) + dbVersionReq.CompartmentId = common.String(dbcs.Spec.DbSystem.CompartmentId) + dbVersionReq.DbSystemShape = common.String(dbcs.Spec.DbSystem.Shape) + // Send the request using the service client + req := database.ListDbVersionsRequest(dbVersionReq) + + resp, err := dbClient.ListDbVersions(ctx, req) + + if err != nil { + return "", err + } + + if dbcs.Spec.DbSystem.DbVersion != "" { + for i := len(resp.Items) - 1; i >= 0; i-- { + version = resp.Items[i] + s1 := getStr(*version.Version, 2) + s2 := getStr(dbcs.Spec.DbSystem.DbVersion, 2) + if strings.EqualFold(s1, s2) { + val, _ = strconv.Atoi(s1) + if val >= 18 { + s3 := s1 + "c" + if strings.EqualFold(s3, dbcs.Spec.DbSystem.DbVersion) { + sFlag = 1 + break + } + } + } else if val < 18 && val >= 11 { + s4 := getStr(*version.Version, 4) + if strings.EqualFold(s4, dbcs.Spec.DbSystem.DbVersion) { + sFlag = 1 + break + } + } + + } + } + + if sFlag == 1 { + return *version.Version, nil + } + return *version.Version, fmt.Errorf("no database version matched") +} + +func getStr(str1 string, num int) string { + return str1[0:num] +} + +func GetDBDetails(kubeClient client.Client, dbcs *databasev1alpha1.DbcsSystem) (database.CreateDatabaseDetails, error) { + dbDetails := database.CreateDatabaseDetails{} + var val database.CreateDatabaseDetailsDbWorkloadEnum + + if dbcs.Spec.DbSystem.TdeWalletPasswordSecret != "" { + tdePasswd, err := GetTdePassword(kubeClient, dbcs) + if err != nil { + return database.CreateDatabaseDetails{}, err + } + tdePassword := strings.Trim(strings.TrimSuffix(tdePasswd, "\n"), "\"") + dbDetails.TdeWalletPassword = &tdePassword + //fmt.Print(tdePassword) + + } + + adminPasswd, err := GetAdminPassword(kubeClient, dbcs) + if err != nil { + return database.CreateDatabaseDetails{}, err + } + + adminPassword := strings.Trim(strings.TrimSuffix(adminPasswd, "\n"), "\"") + dbDetails.AdminPassword = &adminPassword + //fmt.Print(adminPassword) + if dbcs.Spec.DbSystem.DbName != "" { + dbDetails.DbName = common.String(dbcs.Spec.DbSystem.DbName) + } + + if dbcs.Spec.DbSystem.DbWorkload != "" { + val, err = getDbWorkLoadType(dbcs) + if err != nil { + return dbDetails, err + } else { + dbDetails.DbWorkload = database.CreateDatabaseDetailsDbWorkloadEnum(val) + } + } + dbDetails.DbName = common.String(dbcs.Spec.DbSystem.DbName) + if dbcs.Spec.DbSystem.PdbName != "" { + dbDetails.PdbName = &dbcs.Spec.DbSystem.PdbName + } + + //backup configuration + if dbcs.Spec.DbSystem.DbBackupConfig.AutoBackupEnabled != nil { + if *dbcs.Spec.DbSystem.DbBackupConfig.AutoBackupEnabled { + backupConfig, err := getBackupConfig(kubeClient, dbcs) + if err != nil { + return dbDetails, err + } else { + dbDetails.DbBackupConfig = &backupConfig + } + } + } + + return dbDetails, nil +} + +func getBackupConfig(kubeClient client.Client, dbcs *databasev1alpha1.DbcsSystem) (database.DbBackupConfig, error) { + backupConfig := database.DbBackupConfig{} + + if dbcs.Spec.DbSystem.DbBackupConfig.AutoBackupEnabled != nil { + if *dbcs.Spec.DbSystem.DbBackupConfig.AutoBackupEnabled { + backupConfig.AutoBackupEnabled = dbcs.Spec.DbSystem.DbBackupConfig.AutoBackupEnabled + val1, err := getBackupWindowEnum(dbcs) + if err != nil { + return backupConfig, err + } else { + backupConfig.AutoBackupWindow = database.DbBackupConfigAutoBackupWindowEnum(val1) + } + } + + if dbcs.Spec.DbSystem.DbBackupConfig.RecoveryWindowsInDays != nil { + val1, err := getRecoveryWindowsInDays(dbcs) + if err != nil { + return backupConfig, err + } else { + backupConfig.RecoveryWindowInDays = common.Int(val1) + } + + } + } + + return backupConfig, nil +} + +func getBackupWindowEnum(dbcs *databasev1alpha1.DbcsSystem) (database.DbBackupConfigAutoBackupWindowEnum, error) { + + if strings.ToUpper(*dbcs.Spec.DbSystem.DbBackupConfig.AutoBackupWindow) == "SLOT_ONE" { + return database.DbBackupConfigAutoBackupWindowOne, nil + } else if strings.ToUpper(*dbcs.Spec.DbSystem.DbBackupConfig.AutoBackupWindow) == "SLOT_TWO" { + return database.DbBackupConfigAutoBackupWindowTwo, nil + } else if strings.ToUpper(*dbcs.Spec.DbSystem.DbBackupConfig.AutoBackupWindow) == "SLOT_THREE" { + return database.DbBackupConfigAutoBackupWindowThree, nil + } else if strings.ToUpper(*dbcs.Spec.DbSystem.DbBackupConfig.AutoBackupWindow) == "SLOT_FOUR" { + return database.DbBackupConfigAutoBackupWindowFour, nil + } else if strings.ToUpper(*dbcs.Spec.DbSystem.DbBackupConfig.AutoBackupWindow) == "SLOT_FOUR" { + return database.DbBackupConfigAutoBackupWindowFour, nil + } else if strings.ToUpper(*dbcs.Spec.DbSystem.DbBackupConfig.AutoBackupWindow) == "SLOT_FIVE" { + return database.DbBackupConfigAutoBackupWindowFive, nil + } else if strings.ToUpper(*dbcs.Spec.DbSystem.DbBackupConfig.AutoBackupWindow) == "SLOT_SIX" { + return database.DbBackupConfigAutoBackupWindowSix, nil + } else if strings.ToUpper(*dbcs.Spec.DbSystem.DbBackupConfig.AutoBackupWindow) == "SLOT_SEVEN" { + return database.DbBackupConfigAutoBackupWindowSeven, nil + } else if strings.ToUpper(*dbcs.Spec.DbSystem.DbBackupConfig.AutoBackupWindow) == "SLOT_EIGHT" { + return database.DbBackupConfigAutoBackupWindowEight, nil + } else if strings.ToUpper(*dbcs.Spec.DbSystem.DbBackupConfig.AutoBackupWindow) == "SLOT_NINE" { + return database.DbBackupConfigAutoBackupWindowNine, nil + } else if strings.ToUpper(*dbcs.Spec.DbSystem.DbBackupConfig.AutoBackupWindow) == "SLOT_TEN" { + return database.DbBackupConfigAutoBackupWindowTen, nil + } else if strings.ToUpper(*dbcs.Spec.DbSystem.DbBackupConfig.AutoBackupWindow) == "SLOT_ELEVEN" { + return database.DbBackupConfigAutoBackupWindowEleven, nil + } else if strings.ToUpper(*dbcs.Spec.DbSystem.DbBackupConfig.AutoBackupWindow) == "SLOT_TWELVE" { + return database.DbBackupConfigAutoBackupWindowTwelve, nil + } else { + return database.DbBackupConfigAutoBackupWindowOne, nil + } + + //return database.DbBackupConfigAutoBackupWindowEight, fmt.Errorf("AutoBackupWindow values can be SLOT_ONE|SLOT_TWO|SLOT_THREE|SLOT_FOUR|SLOT_FIVE|SLOT_SIX|SLOT_SEVEN|SLOT_EIGHT|SLOT_NINE|SLOT_TEN|SLOT_ELEVEN|SLOT_TWELEVE. The current value set to " + *dbcs.Spec.DbSystem.DbBackupConfig.AutoBackupWindow) +} + +func getRecoveryWindowsInDays(dbcs *databasev1alpha1.DbcsSystem) (int, error) { + + var days int + + if *dbcs.Spec.DbSystem.DbBackupConfig.RecoveryWindowsInDays == 7 { + return *dbcs.Spec.DbSystem.DbBackupConfig.RecoveryWindowsInDays, nil + } else if *dbcs.Spec.DbSystem.DbBackupConfig.RecoveryWindowsInDays == 15 { + return *dbcs.Spec.DbSystem.DbBackupConfig.RecoveryWindowsInDays, nil + } else if *dbcs.Spec.DbSystem.DbBackupConfig.RecoveryWindowsInDays == 30 { + return *dbcs.Spec.DbSystem.DbBackupConfig.RecoveryWindowsInDays, nil + } else if *dbcs.Spec.DbSystem.DbBackupConfig.RecoveryWindowsInDays == 45 { + return *dbcs.Spec.DbSystem.DbBackupConfig.RecoveryWindowsInDays, nil + } else if *dbcs.Spec.DbSystem.DbBackupConfig.RecoveryWindowsInDays == 60 { + return *dbcs.Spec.DbSystem.DbBackupConfig.RecoveryWindowsInDays, nil + } else { + days = 30 + return days, nil + } + //return days, fmt.Errorf("RecoveryWindowsInDays values can be 7|15|30|45|60 Days.") +} + +func GetDBSystemopts( + dbcs *databasev1alpha1.DbcsSystem) database.DbSystemOptions { + + dbSystemOpt := database.DbSystemOptions{} + + if dbcs.Spec.DbSystem.StorageManagement != "" { + if dbcs.Spec.DbSystem.StorageManagement == "LVM" { + dbSystemOpt.StorageManagement = database.DbSystemOptionsStorageManagementLvm + } else if dbcs.Spec.DbSystem.StorageManagement == "ASM" { + dbSystemOpt.StorageManagement = database.DbSystemOptionsStorageManagementAsm + } else { + dbSystemOpt.StorageManagement = database.DbSystemOptionsStorageManagementAsm + } + } else { + dbSystemOpt.StorageManagement = database.DbSystemOptionsStorageManagementAsm + } + + return dbSystemOpt +} + +func getLicenceModel(dbcs *databasev1alpha1.DbcsSystem) database.DbSystemLicenseModelEnum { + if dbcs.Spec.DbSystem.LicenseModel == "BRING_YOUR_OWN_LICENSE" { + return database.DbSystemLicenseModelBringYourOwnLicense + + } + return database.DbSystemLicenseModelLicenseIncluded +} + +func getDbWorkLoadType(dbcs *databasev1alpha1.DbcsSystem) (database.CreateDatabaseDetailsDbWorkloadEnum, error) { + + if strings.ToUpper(dbcs.Spec.DbSystem.DbWorkload) == "OLTP" { + + return database.CreateDatabaseDetailsDbWorkloadOltp, nil + } + if strings.ToUpper(dbcs.Spec.DbSystem.DbWorkload) == "DSS" { + return database.CreateDatabaseDetailsDbWorkloadDss, nil + + } + + return database.CreateDatabaseDetailsDbWorkloadDss, fmt.Errorf("DbWorkload values can be OLTP|DSS. The current value set to " + dbcs.Spec.DbSystem.DbWorkload) +} + +func GetNodeCount( + dbcs *databasev1alpha1.DbcsSystem) int { + + if dbcs.Spec.DbSystem.NodeCount != nil { + return *dbcs.Spec.DbSystem.NodeCount + } else { + return 1 + } +} + +func GetInitialStorage( + dbcs *databasev1alpha1.DbcsSystem) int { + + if dbcs.Spec.DbSystem.InitialDataStorageSizeInGB > 0 { + return dbcs.Spec.DbSystem.InitialDataStorageSizeInGB + } + return 256 +} + +func GetDBEdition(dbcs *databasev1alpha1.DbcsSystem) database.LaunchDbSystemDetailsDatabaseEditionEnum { + + if dbcs.Spec.DbSystem.ClusterName != "" { + return database.LaunchDbSystemDetailsDatabaseEditionEnterpriseEditionExtremePerformance + } + + if dbcs.Spec.DbSystem.DbEdition != "" { + if dbcs.Spec.DbSystem.DbEdition == "STANDARD_EDITION" { + return database.LaunchDbSystemDetailsDatabaseEditionStandardEdition + } else if dbcs.Spec.DbSystem.DbEdition == "ENTERPRISE_EDITION" { + return database.LaunchDbSystemDetailsDatabaseEditionEnterpriseEdition + } else if dbcs.Spec.DbSystem.DbEdition == "ENTERPRISE_EDITION_HIGH_PERFORMANCE" { + return database.LaunchDbSystemDetailsDatabaseEditionEnterpriseEditionHighPerformance + } else if dbcs.Spec.DbSystem.DbEdition == "ENTERPRISE_EDITION_EXTREME_PERFORMANCE" { + return database.LaunchDbSystemDetailsDatabaseEditionEnterpriseEditionExtremePerformance + } else { + return database.LaunchDbSystemDetailsDatabaseEditionEnterpriseEdition + } + } + + return database.LaunchDbSystemDetailsDatabaseEditionEnterpriseEdition +} + +func GetDBbDiskRedundancy( + dbcs *databasev1alpha1.DbcsSystem) database.LaunchDbSystemDetailsDiskRedundancyEnum { + + if dbcs.Spec.DbSystem.ClusterName != "" { + return database.LaunchDbSystemDetailsDiskRedundancyHigh + } + + if dbcs.Spec.DbSystem.DiskRedundancy == "HIGH" { + return database.LaunchDbSystemDetailsDiskRedundancyHigh + } else if dbcs.Spec.DbSystem.DiskRedundancy == "NORMAL" { + return database.LaunchDbSystemDetailsDiskRedundancyNormal + } + + return database.LaunchDbSystemDetailsDiskRedundancyNormal +} + +func getWorkRequest(workId string, wrClient workrequests.WorkRequestClient, dbcs *databasev1alpha1.DbcsSystem) ([]workrequests.WorkRequestSummary, error) { + var workReq []workrequests.WorkRequestSummary + + req := workrequests.ListWorkRequestsRequest{CompartmentId: &dbcs.Spec.DbSystem.CompartmentId, OpcRequestId: &workId, ResourceId: dbcs.Spec.Id} + resp, err := wrClient.ListWorkRequests(context.Background(), req) + if err != nil { + return workReq, err + } + + return resp.Items, nil +} + +func GetKeyValue(str1 string) string { + list1 := strings.Split(str1, " ") + for _, value := range list1 { + val1 := strings.Split(value, "=") + if val1[0] == "version" { + return val1[1] + } + } + + return "noversion" +} + +func GetFmtStr(pstr string) string { + + return "[" + pstr + "]" +} + +func checkValue(dbcs *databasev1alpha1.DbcsSystem, workId *string) int { + + var status int = 0 + //dbWorkRequest := databasev1alpha1.DbWorkrequests{} + + if len(dbcs.Status.WorkRequests) > 0 { + for _, v := range dbcs.Status.WorkRequests { + if *v.OperationId == *workId { + status = 1 + } + } + } + + return status +} +func setValue(dbcs *databasev1alpha1.DbcsSystem, dbWorkRequest databasev1alpha1.DbWorkrequests) { + + //var status int = 1 + //dbWorkRequest := databasev1alpha1.DbWorkrequests{} + var counter int = 0 + if len(dbcs.Status.WorkRequests) > 0 { + for _, v := range dbcs.Status.WorkRequests { + if *v.OperationId == *dbWorkRequest.OperationId { + dbcs.Status.WorkRequests[counter].OperationId = dbWorkRequest.OperationId + dbcs.Status.WorkRequests[counter].OperationType = dbWorkRequest.OperationType + dbcs.Status.WorkRequests[counter].PercentComplete = dbWorkRequest.PercentComplete + dbcs.Status.WorkRequests[counter].TimeAccepted = dbWorkRequest.TimeAccepted + dbcs.Status.WorkRequests[counter].TimeFinished = dbWorkRequest.TimeFinished + dbcs.Status.WorkRequests[counter].TimeStarted = dbWorkRequest.TimeStarted + } + counter = counter + 1 + } + } + +} diff --git a/commons/finalizer/finalizer.go b/commons/finalizer/finalizer.go new file mode 100644 index 00000000..bafe110e --- /dev/null +++ b/commons/finalizer/finalizer.go @@ -0,0 +1,122 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package finalizer + +import ( + "context" + "encoding/json" + + "k8s.io/apimachinery/pkg/types" + "sigs.k8s.io/controller-runtime/pkg/client" +) + +// name of our custom finalizer +var finalizerName = "database.oracle.com/dbcsfinalizer" + +// HasFinalizer returns true if the finalizer exists in the object metadata +func HasFinalizer(obj client.Object) bool { + finalizer := obj.GetFinalizers() + return containsString(finalizer, finalizerName) +} + +// Register adds the finalizer and patch the object +func Register(kubeClient client.Client, obj client.Object) error { + finalizer := obj.GetFinalizers() + finalizer = append(finalizer, finalizerName) + return setFinalizer(kubeClient, obj, finalizer) +} + +// Unregister removes the finalizer and patch the object +func Unregister(kubeClient client.Client, obj client.Object) error { + finalizer := obj.GetFinalizers() + finalizer = removeString(finalizer, finalizerName) + return setFinalizer(kubeClient, obj, finalizer) +} + +// Helper functions to check and remove string from a slice of strings. +func containsString(slice []string, s string) bool { + for _, item := range slice { + if item == s { + return true + } + } + return false +} + +func removeString(slice []string, s string) (result []string) { + for _, item := range slice { + if item == s { + continue + } + result = append(result, item) + } + return +} + +type patchValue struct { + Op string `json:"op"` + Path string `json:"path"` + Value interface{} `json:"value"` +} + +func setFinalizer(kubeClient client.Client, dbcs client.Object, finalizer []string) error { + payload := []patchValue{} + + if dbcs.GetFinalizers() == nil { + payload = append(payload, patchValue{ + Op: "replace", + Path: "/metadata/finalizers", + Value: []string{}, + }) + } + + payload = append(payload, patchValue{ + Op: "replace", + Path: "/metadata/finalizers", + Value: finalizer, + }) + + payloadBytes, err := json.Marshal(payload) + if err != nil { + return err + } + + patch := client.RawPatch(types.JSONPatchType, payloadBytes) + return kubeClient.Patch(context.TODO(), dbcs, patch) +} diff --git a/config/crd/bases/database.oracle.com_dbcssystems.yaml b/config/crd/bases/database.oracle.com_dbcssystems.yaml new file mode 100644 index 00000000..3f4b1c46 --- /dev/null +++ b/config/crd/bases/database.oracle.com_dbcssystems.yaml @@ -0,0 +1,240 @@ + +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.6.1 + creationTimestamp: null + name: dbcssystems.database.oracle.com +spec: + group: database.oracle.com + names: + kind: DbcsSystem + listKind: DbcsSystemList + plural: dbcssystems + singular: dbcssystem + scope: Namespaced + versions: + - name: v1alpha1 + schema: + openAPIV3Schema: + description: DbcsSystem is the Schema for the dbcssystems API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: DbcsSystemSpec defines the desired state of DbcsSystem + properties: + dbSystem: + properties: + availabilityDomain: + type: string + backupSubnetId: + type: string + clusterName: + type: string + compartmentId: + type: string + cpuCoreCount: + type: integer + dbAdminPaswordSecret: + type: string + dbBackupConfig: + description: DB Backup COnfig Network Struct + properties: + autoBackupEnabled: + type: boolean + autoBackupWindow: + type: string + backupDestinationDetails: + type: string + recoveryWindowsInDays: + type: integer + type: object + dbDomain: + type: string + dbEdition: + type: string + dbName: + type: string + dbUniqueName: + type: string + dbVersion: + type: string + dbWorkload: + type: string + diskRedundancy: + type: string + displayName: + type: string + domain: + type: string + faultDomains: + items: + type: string + type: array + hostName: + type: string + initialDataStorageSizeInGB: + type: integer + kmsKeyId: + type: string + kmsKeyVersionId: + type: string + licenseModel: + type: string + nodeCount: + type: integer + pdbName: + type: string + privateIp: + type: string + shape: + type: string + sshPublicKeys: + items: + type: string + type: array + storageManagement: + type: string + subnetId: + type: string + tags: + additionalProperties: + type: string + type: object + tdeWalletPasswordSecret: + type: string + timeZone: + type: string + required: + - availabilityDomain + - compartmentId + - dbAdminPaswordSecret + - hostName + - shape + - sshPublicKeys + - subnetId + type: object + hardLink: + type: boolean + id: + type: string + ociConfigMap: + type: string + ociSecret: + type: string + required: + - ociConfigMap + type: object + status: + description: DbcsSystemStatus defines the observed state of DbcsSystem + properties: + availabilityDomain: + type: string + cpuCoreCount: + type: integer + dataStoragePercentage: + type: integer + dataStorageSizeInGBs: + type: integer + dbEdition: + type: string + dbInfo: + items: + description: DbcsSystemStatus defines the observed state of DbcsSystem + properties: + dbHomeId: + type: string + dbName: + type: string + dbUniqueName: + type: string + dbWorkload: + type: string + id: + type: string + type: object + type: array + displayName: + type: string + id: + type: string + licenseModel: + type: string + network: + properties: + clientSubnet: + type: string + domainName: + type: string + hostName: + type: string + listenerPort: + type: integer + networkSG: + type: string + scanDnsName: + type: string + vcnName: + type: string + type: object + nodeCount: + type: integer + recoStorageSizeInGB: + type: integer + shape: + type: string + state: + type: string + storageManagement: + type: string + subnetId: + type: string + timeZone: + type: string + workRequests: + items: + properties: + operationId: + type: string + operationType: + type: string + percentComplete: + type: string + timeAccepted: + type: string + timeFinished: + type: string + timeStarted: + type: string + required: + - operationId + - operationType + type: object + type: array + required: + - state + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] diff --git a/config/crd/kustomization.yaml b/config/crd/kustomization.yaml index fa930212..19983b29 100644 --- a/config/crd/kustomization.yaml +++ b/config/crd/kustomization.yaml @@ -15,6 +15,7 @@ resources: - bases/database.oracle.com_cdbs.yaml - bases/database.oracle.com_oraclerestdataservices.yaml - bases/database.oracle.com_autonomouscontainerdatabases.yaml +- bases/database.oracle.com_dbcssystems.yaml # +kubebuilder:scaffold:crdkustomizeresource patchesStrategicMerge: @@ -28,6 +29,7 @@ patchesStrategicMerge: #- patches/webhook_in_cdbs.yaml #- patches/webhook_in_oraclerestdataservices.yaml #- patches/webhook_in_autonomouscontainerdatabases.yaml +#- patches/webhook_in_dbcssystems.yaml # +kubebuilder:scaffold:crdkustomizewebhookpatch # [CERTMANAGER] To enable webhook, uncomment all the sections with [CERTMANAGER] prefix. @@ -40,6 +42,7 @@ patchesStrategicMerge: - patches/cainjection_in_cdbs.yaml #- patches/cainjection_in_oraclerestdataservices.yaml #- patches/cainjection_in_autonomouscontainerdatabases.yaml +#- patches/cainjection_in_dbcssystems.yaml # +kubebuilder:scaffold:crdkustomizecainjectionpatch # the following config is for teaching kustomize how to do kustomization for CRDs. diff --git a/config/crd/patches/cainjection_in_dbcssystems.yaml b/config/crd/patches/cainjection_in_dbcssystems.yaml new file mode 100644 index 00000000..9d8521ac --- /dev/null +++ b/config/crd/patches/cainjection_in_dbcssystems.yaml @@ -0,0 +1,8 @@ +# The following patch adds a directive for certmanager to inject CA into the CRD +# CRD conversion requires k8s 1.13 or later. +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + annotations: + cert-manager.io/inject-ca-from: $(CERTIFICATE_NAMESPACE)/$(CERTIFICATE_NAME) + name: dbcssystems.database.oracle.com diff --git a/config/crd/patches/webhook_in_dbcssystems.yaml b/config/crd/patches/webhook_in_dbcssystems.yaml new file mode 100644 index 00000000..69e578a3 --- /dev/null +++ b/config/crd/patches/webhook_in_dbcssystems.yaml @@ -0,0 +1,17 @@ +# The following patch enables conversion webhook for CRD +# CRD conversion requires k8s 1.13 or later. +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: dbcssystems.database.oracle.com +spec: + conversion: + strategy: Webhook + webhookClientConfig: + # this is "\n" used as a placeholder, otherwise it will be rejected by the apiserver for being blank, + # but we're going to set it later using the cert-manager (or potentially a patch if not using cert-manager) + caBundle: Cg== + service: + namespace: system + name: webhook-service + path: /convert diff --git a/config/manager/kustomization.yaml b/config/manager/kustomization.yaml index c42eb919..6de6f53f 100644 --- a/config/manager/kustomization.yaml +++ b/config/manager/kustomization.yaml @@ -8,5 +8,5 @@ apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization images: - name: controller - newName: container-registry.oracle.com/database/operator - newTag: 0.1.0 \ No newline at end of file + newName: phx.ocir.io/intsanjaysingh/oracle-database-operator + newTag: 0.0.1 diff --git a/config/rbac/dbcssystem_editor_role.yaml b/config/rbac/dbcssystem_editor_role.yaml new file mode 100644 index 00000000..934eea97 --- /dev/null +++ b/config/rbac/dbcssystem_editor_role.yaml @@ -0,0 +1,24 @@ +# permissions for end users to edit dbcssystems. +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: dbcssystem-editor-role +rules: +- apiGroups: + - database.oracle.com + resources: + - dbcssystems + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - database.oracle.com + resources: + - dbcssystems/status + verbs: + - get diff --git a/config/rbac/dbcssystem_viewer_role.yaml b/config/rbac/dbcssystem_viewer_role.yaml new file mode 100644 index 00000000..8153d112 --- /dev/null +++ b/config/rbac/dbcssystem_viewer_role.yaml @@ -0,0 +1,20 @@ +# permissions for end users to view dbcssystems. +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: dbcssystem-viewer-role +rules: +- apiGroups: + - database.oracle.com + resources: + - dbcssystems + verbs: + - get + - list + - watch +- apiGroups: + - database.oracle.com + resources: + - dbcssystems/status + verbs: + - get diff --git a/config/rbac/role.yaml b/config/rbac/role.yaml index 897e0e77..e6d35b96 100644 --- a/config/rbac/role.yaml +++ b/config/rbac/role.yaml @@ -24,6 +24,20 @@ rules: - patch - update - watch +- apiGroups: + - "" + resources: + - configmaps + - namespaces + - secrets + verbs: + - create + - delete + - get + - list + - patch + - update + - watch - apiGroups: - "" resources: @@ -246,6 +260,36 @@ rules: - get - patch - update +- apiGroups: + - database.oracle.com + resources: + - dbcssystems + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - database.oracle.com + resources: + - dbcssystems/finalizers + verbs: + - create + - delete + - get + - patch + - update +- apiGroups: + - database.oracle.com + resources: + - dbcssystems/status + verbs: + - get + - patch + - update - apiGroups: - database.oracle.com resources: diff --git a/config/samples/database_v1alpha1_dbcssystem.yaml b/config/samples/database_v1alpha1_dbcssystem.yaml new file mode 100644 index 00000000..0ca38ddf --- /dev/null +++ b/config/samples/database_v1alpha1_dbcssystem.yaml @@ -0,0 +1,7 @@ +apiVersion: database.oracle.com/v1alpha1 +kind: DbcsSystem +metadata: + name: dbcssystem-sample +spec: + # Add fields here + foo: bar diff --git a/config/samples/kustomization.yaml b/config/samples/kustomization.yaml index 9e0f5d41..a726efa1 100644 --- a/config/samples/kustomization.yaml +++ b/config/samples/kustomization.yaml @@ -15,4 +15,5 @@ resources: - sidb/singleinstancedatabase.yaml - sharding/shardingdatabase.yaml - sharding/sharding_v1alpha1_provshard.yaml +- database_v1alpha1_dbcssystem.yaml # +kubebuilder:scaffold:manifestskustomizesamples diff --git a/controllers/database/dbcssystem_controller.go b/controllers/database/dbcssystem_controller.go new file mode 100644 index 00000000..9f731061 --- /dev/null +++ b/controllers/database/dbcssystem_controller.go @@ -0,0 +1,302 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package controllers + +import ( + "context" + "reflect" + + databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + dbcsv1 "github.com/oracle/oracle-database-operator/commons/dbcssystem" + "github.com/oracle/oracle-database-operator/commons/finalizer" + "github.com/oracle/oracle-database-operator/commons/oci" + + "github.com/go-logr/logr" + "github.com/oracle/oci-go-sdk/v63/core" + "github.com/oracle/oci-go-sdk/v63/database" + "github.com/oracle/oci-go-sdk/v63/workrequests" + "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/client-go/tools/record" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/controller" + "sigs.k8s.io/controller-runtime/pkg/event" + "sigs.k8s.io/controller-runtime/pkg/log" + "sigs.k8s.io/controller-runtime/pkg/predicate" +) + +// DbcsSystemReconciler reconciles a DbcsSystem object +type DbcsSystemReconciler struct { + KubeClient client.Client + Scheme *runtime.Scheme + Logv1 logr.Logger + Logger logr.Logger + dbClient database.DatabaseClient + nwClient core.VirtualNetworkClient + wrClient workrequests.WorkRequestClient + Recorder record.EventRecorder +} + +//+kubebuilder:rbac:groups=database.oracle.com,resources=dbcssystems,verbs=get;list;watch;create;update;patch;delete +// +kubebuilder:rbac:groups=database.oracle.com,resources=dbcssystems/status,verbs=get;update;patch +// +kubebuilder:rbac:groups=database.oracle.com,resources=dbcssystems/finalizers,verbs=get;create;update;patch;delete +// +kubebuilder:rbac:groups="",resources=configmaps;secrets;namespaces,verbs=get;list;watch;create;update;patch;delete + +// Reconcile is part of the main kubernetes reconciliation loop which aims to +// move the current state of the cluster closer to the desired state. +// TODO(user): Modify the Reconcile function to compare the state specified by +// the DbcsSystem object against the actual cluster state, and then +// perform operations to make the cluster state reflect the state specified by +// the user. +// +// For more details, check Reconcile and its Result here: +// - https://pkg.go.dev/sigs.k8s.io/controller-runtime@v0.8.3/pkg/reconcile +func (r *DbcsSystemReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + r.Logger = log.FromContext(ctx) + + // your logic here + + //r.Logger = r.Logv1.WithValues("Instance.Namespace", req.NamespacedName) + var err error + // Get the dbcs instance from the cluster + dbcsInst := &databasev1alpha1.DbcsSystem{} + + if err := r.KubeClient.Get(context.TODO(), req.NamespacedName, dbcsInst); err != nil { + if !errors.IsNotFound(err) { + return ctrl.Result{}, err + } + } + + // Create oci-go-sdk client + authData := oci.APIKeyAuth{ + ConfigMapName: &dbcsInst.Spec.OCIConfigMap, + SecretName: &dbcsInst.Spec.OCISecret, + Namespace: dbcsInst.GetNamespace(), + } + provider, err := oci.GetOCIProvider(r.KubeClient, authData) + if err != nil { + return ctrl.Result{}, err + } + + r.dbClient, err = database.NewDatabaseClientWithConfigurationProvider(provider) + + if err != nil { + return ctrl.Result{}, err + } + + r.nwClient, err = core.NewVirtualNetworkClientWithConfigurationProvider(provider) + if err != nil { + return ctrl.Result{}, err + } + + r.wrClient, err = workrequests.NewWorkRequestClientWithConfigurationProvider(provider) + r.Logger.Info("OCI provider configured succesfully") + + /* + Using Finalizer for object deletion + */ + + if dbcsInst.ObjectMeta.DeletionTimestamp.IsZero() { + // The object is not being deleted + if dbcsInst.Spec.HardLink && !finalizer.HasFinalizer(dbcsInst) { + finalizer.Register(r.KubeClient, dbcsInst) + r.Logger.Info("Finalizer registered successfully.") + } else if !dbcsInst.Spec.HardLink && finalizer.HasFinalizer(dbcsInst) { + finalizer.Unregister(r.KubeClient, dbcsInst) + r.Logger.Info("Finalizer unregistered successfully.") + } + } else { + // The object is being deleted + r.Logger.Info("Terminate DbcsSystem Database: " + dbcsInst.Spec.DbSystem.DisplayName) + if err := dbcsv1.DeleteDbcsSystemSystem(r.dbClient, *dbcsInst.Spec.Id); err != nil { + r.Logger.Error(err, "Fail to terminate DbcsSystem Instance") + // Change the status to Failed + if statusErr := dbcsv1.SetLifecycleState(r.KubeClient, r.dbClient, dbcsInst, databasev1alpha1.Terminate, r.nwClient, r.wrClient); statusErr != nil { + return ctrl.Result{}, statusErr + } + // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue + return ctrl.Result{}, nil + } + finalizer.Unregister(r.KubeClient, dbcsInst) + r.Logger.Info("Finalizer unregistered successfully.") + // Stop reconciliation as the item is being deleted + return ctrl.Result{}, nil + } + + /* + Determine whether it's a provision or bind operation + */ + lastSucSpec, err := dbcsInst.GetLastSuccessfulSpec() + if err != nil { + return ctrl.Result{}, err + } + + if dbcsInst.Spec.Id == nil && lastSucSpec == nil { + // If no DbcsSystem ID specified, create a DB System + // ======================== Validate Specs ============== + err = dbcsv1.ValidateSpex(r.Logger, r.KubeClient, r.dbClient, dbcsInst, r.nwClient, r.Recorder) + if err != nil { + return ctrl.Result{}, err + } + r.Logger.Info("DbcsSystem DBSystem provisioning") + dbcsID, err := dbcsv1.CreateAndGetDbcsId(r.Logger, r.KubeClient, r.dbClient, dbcsInst, r.nwClient, r.wrClient) + if err != nil { + r.Logger.Error(err, "Fail to provision and get DbcsSystem System ID") + + // Change the status to Failed + if statusErr := dbcsv1.SetLifecycleState(r.KubeClient, r.dbClient, dbcsInst, databasev1alpha1.Failed, r.nwClient, r.wrClient); statusErr != nil { + return ctrl.Result{}, statusErr + } + // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue + return ctrl.Result{}, nil + } + + dbcsInst.Spec.Id = &dbcsID + if err := dbcsv1.UpdateDbcsSystemId(r.KubeClient, dbcsInst); err != nil { + // Change the status to Failed + if statusErr := dbcsv1.SetLifecycleState(r.KubeClient, r.dbClient, dbcsInst, databasev1alpha1.Failed, r.nwClient, r.wrClient); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, err + } + + r.Logger.Info("DbcsSystem system provisioned succesfully") + + if err := dbcsInst.UpdateLastSuccessfulSpec(r.KubeClient); err != nil { + return ctrl.Result{}, err + } + } else { + if lastSucSpec == nil { + if err := dbcsv1.GetDbSystemId(r.Logger, r.dbClient, dbcsInst); err != nil { + // Change the status to Failed + if statusErr := dbcsv1.SetLifecycleState(r.KubeClient, r.dbClient, dbcsInst, databasev1alpha1.Failed, r.nwClient, r.wrClient); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, err + } + if err := dbcsv1.SetDBCSDatabaseLifecycleState(r.Logger, r.KubeClient, r.dbClient, dbcsInst, r.nwClient, r.wrClient); err != nil { + // Change the status to required state + return ctrl.Result{}, err + } + + if err := dbcsv1.UpdateDbcsSystemId(r.KubeClient, dbcsInst); err != nil { + // Change the status to Failed + if statusErr := dbcsv1.SetLifecycleState(r.KubeClient, r.dbClient, dbcsInst, databasev1alpha1.Failed, r.nwClient, r.wrClient); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, err + } + + r.Logger.Info("Sync information from remote DbcsSystem System successfully") + + if err := dbcsInst.UpdateLastSuccessfulSpec(r.KubeClient); err != nil { + return ctrl.Result{}, err + } + } else { + if dbcsInst.Spec.Id == nil { + dbcsInst.Spec.Id = lastSucSpec.Id + } + + if err := dbcsv1.UpdateDbcsSystemIdInst(r.Logger, r.dbClient, dbcsInst, r.KubeClient, r.nwClient, r.wrClient); err != nil { + r.Logger.Error(err, "Fail to update DbcsSystem Id") + + // Change the status to Failed + if statusErr := dbcsv1.SetLifecycleState(r.KubeClient, r.dbClient, dbcsInst, databasev1alpha1.Failed, r.nwClient, r.wrClient); statusErr != nil { + return ctrl.Result{}, statusErr + } + // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue + return ctrl.Result{}, nil + } + if err := dbcsv1.SetDBCSDatabaseLifecycleState(r.Logger, r.KubeClient, r.dbClient, dbcsInst, r.nwClient, r.wrClient); err != nil { + // Change the status to required state + return ctrl.Result{}, err + } + } + } + + // Update the Wallet Secret when the secret name is given + //r.updateWalletSecret(dbcs) + + // Update the last succesful spec + if err := dbcsInst.UpdateLastSuccessfulSpec(r.KubeClient); err != nil { + return ctrl.Result{}, err + } + + // Change the phase to "Available" + if statusErr := dbcsv1.SetLifecycleState(r.KubeClient, r.dbClient, dbcsInst, databasev1alpha1.Available, r.nwClient, r.wrClient); statusErr != nil { + return ctrl.Result{}, statusErr + } + + return ctrl.Result{}, nil +} + +func (r *DbcsSystemReconciler) eventFilterPredicate() predicate.Predicate { + return predicate.Funcs{ + CreateFunc: func(e event.CreateEvent) bool { + return true + }, + UpdateFunc: func(e event.UpdateEvent) bool { + // Get the dbName as old dbName when an update event happens + oldObject := e.ObjectOld.DeepCopyObject().(*databasev1alpha1.DbcsSystem) + newObject := e.ObjectNew.DeepCopyObject().(*databasev1alpha1.DbcsSystem) + specObject := !reflect.DeepEqual(oldObject.Spec, newObject.Spec) + + deletionTimeStamp := !reflect.DeepEqual(oldObject.GetDeletionTimestamp(), newObject.GetDeletionTimestamp()) + + if specObject || deletionTimeStamp { + return true + } + + return false + }, + DeleteFunc: func(e event.DeleteEvent) bool { + return false + }, + } +} + +// SetupWithManager sets up the controller with the Manager. +func (r *DbcsSystemReconciler) SetupWithManager(mgr ctrl.Manager) error { + return ctrl.NewControllerManagedBy(mgr). + For(&databasev1alpha1.DbcsSystem{}). + WithEventFilter(r.eventFilterPredicate()). + WithOptions(controller.Options{MaxConcurrentReconciles: 50}). + Complete(r) +} diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 856f0a19..356db917 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -907,7 +907,7 @@ func (r *OracleRestDataServiceReconciler) cleanupOracleRestDataService(req ctrl. uninstallORDS := fmt.Sprintf(dbcommons.UninstallORDSCMD, adminPassword) out, err = dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, true, "bash", "-c", - uninstallORDS) + uninstallORDS) log.Info("UninstallORDSCMD Output : " + out) if strings.Contains(strings.ToUpper(out), "ERROR") { return errors.New(out) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 8dbe8366..ebf75c6e 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -456,13 +456,13 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns Volumes: []corev1.Volume{{ Name: "oracle-pwd-vol", VolumeSource: corev1.VolumeSource{ - Secret: &corev1.SecretVolumeSource { + Secret: &corev1.SecretVolumeSource{ SecretName: m.Spec.AdminPassword.SecretName, - Optional: func() *bool { i := true; return &i }(), - Items: []corev1.KeyToPath {{ - Key: m.Spec.AdminPassword.SecretKey, - Path: "oracle_pwd", - }, + Optional: func() *bool { i := true; return &i }(), + Items: []corev1.KeyToPath{{ + Key: m.Spec.AdminPassword.SecretKey, + Path: "oracle_pwd", + }, }, }, }, @@ -495,7 +495,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns return 30 }(), }, - VolumeMounts: []corev1.VolumeMount { { + VolumeMounts: []corev1.VolumeMount{{ MountPath: "/run/secrets/oracle_pwd", ReadOnly: true, Name: "oracle-pwd-vol", @@ -583,11 +583,11 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, { Name: "oracle-pwd-vol", VolumeSource: corev1.VolumeSource{ - Secret: &corev1.SecretVolumeSource { + Secret: &corev1.SecretVolumeSource{ SecretName: m.Spec.AdminPassword.SecretName, - Optional: func() *bool { i := (m.Spec.Edition != "express"); return &i }(), - Items: []corev1.KeyToPath {{ - Key: m.Spec.AdminPassword.SecretKey, + Optional: func() *bool { i := (m.Spec.Edition != "express"); return &i }(), + Items: []corev1.KeyToPath{{ + Key: m.Spec.AdminPassword.SecretKey, Path: "oracle_pwd", }}, }, @@ -1500,7 +1500,6 @@ func (r *SingleInstanceDatabaseReconciler) validateDBReadiness(m *dbapi.SingleIn } } - return requeueN, readyPod, nil } diff --git a/controllers/database/suite_test.go b/controllers/database/suite_test.go index 9b62324a..fdebedce 100644 --- a/controllers/database/suite_test.go +++ b/controllers/database/suite_test.go @@ -87,6 +87,9 @@ var _ = BeforeSuite(func(done Done) { err = databasev1alpha1.AddToScheme(scheme.Scheme) Expect(err).NotTo(HaveOccurred()) + err = databasev1alpha1.AddToScheme(scheme.Scheme) + Expect(err).NotTo(HaveOccurred()) + // +kubebuilder:scaffold:scheme k8sClient, err = client.New(cfg, client.Options{Scheme: scheme.Scheme}) diff --git a/docs/dbcs/README.md b/docs/dbcs/README.md new file mode 100644 index 00000000..2d40a52a --- /dev/null +++ b/docs/dbcs/README.md @@ -0,0 +1,183 @@ +# Using the DB Operator DBCS Controller + +The Oracle Cloud Infrastructure's Database Service furnishes [co-managed Oracle Database cloud solutions](https://docs.oracle.com/en-us/iaas/Content/Database/Concepts/overview.htm). A single-node DB systems on either bare metal or virtual machines, and 2-node RAC DB systems on virtual machines. To manage the life cycle of an OCI DBCS system, we can use the OCI Console, the REST API or the Oracle Cloud Infrastructure CLI and at the granular level, the Database CLI (DBCLI), Enterprise Manager, or SQL Developer. + +The Oracle DB Operator DBCS Controller is a feature of the Oracle DB Operator for Kubernetes (a.k.a. OraOperator) which supports the life cycle management of the Database Systems deployed using OCI's DBCS Service. + +# Supported Database Editions and Versions + +All single-node OCI Oracle RAC DB systems support the following Oracle Database editions: + +- Standard Edition +- Enterprise Edition +- Enterprise Edition - High Performance +- Enterprise Edition - Extreme Performance + + +Two-node Oracle RAC DB systems require Oracle Enterprise Edition - Extreme Performance. + +For standard provisioning of DB systems (using Oracle Automatic Storage Management (ASM) as your storage management software), the supported database versions are: + +- Oracle Database 21c +- Oracle Database 19c +- Oracle Database 18c (18.0) +- Oracle Database 12c Release 2 (12.2) +- Oracle Database 12c Release 1 (12.1) +- Oracle Database 11g Release 2 (11.2) + + +For fast provisioning of single-node virtual machine database systems (using Logical Volume Manager as your storage management software), the supported database versions are: + +- Oracle Database 21c +- Oracle Database 19c +- Oracle Database 18c +- Oracle Database 12c Release 2 (12.2) + + +# Oracle DB Operator DBCS Controller Deployment + +The step by step procedure to deploy the OraOperator is documented [here](https://github.com/oracle/oracle-database-operator/blob/main/README.md). + +Once the Oracle DB Operator has been deployed, we can see the DB operator pods running in the Kubernetes Cluster. The DBCS Controller will deployed as part of the Oracle DB Operator Deployment as a CRD (Custom Resource Definition). Below is an example of such a deployment: +``` +[root@test-server oracle-database-operator]# kubectl get ns +NAME STATUS AGE +cert-manager Active 2m5s +default Active 125d +kube-node-lease Active 125d +kube-public Active 125d +kube-system Active 125d +oracle-database-operator-system Active 17s <<<< namespace to deploy the Oracle Database Operator + + +[root@test-server oracle-database-operator]# kubectl get all -n oracle-database-operator-system +NAME READY STATUS RESTARTS AGE +pod/oracle-database-operator-controller-manager-665874bd57-dlhls 1/1 Running 0 28s +pod/oracle-database-operator-controller-manager-665874bd57-g2cgw 1/1 Running 0 28s +pod/oracle-database-operator-controller-manager-665874bd57-q42f8 1/1 Running 0 28s + +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +service/oracle-database-operator-controller-manager-metrics-service ClusterIP 10.96.130.124 8443/TCP 29s +service/oracle-database-operator-webhook-service ClusterIP 10.96.4.104 443/TCP 29s + +NAME READY UP-TO-DATE AVAILABLE AGE +deployment.apps/oracle-database-operator-controller-manager 3/3 3 3 29s + +NAME DESIRED CURRENT READY AGE +replicaset.apps/oracle-database-operator-controller-manager-665874bd57 3 3 3 29s +[root@docker-test-server oracle-database-operator]# + + +[root@test-server oracle-database-operator]# kubectl get crd +NAME CREATED AT +autonomousdatabasebackups.database.oracle.com 2022-02-08T18:28:55Z +autonomousdatabaserestores.database.oracle.com 2022-02-08T18:28:55Z +autonomousdatabases.database.oracle.com 2022-02-22T23:23:25Z +certificaterequests.cert-manager.io 2022-02-22T23:21:35Z +certificates.cert-manager.io 2022-02-22T23:21:36Z +challenges.acme.cert-manager.io 2022-02-22T23:21:36Z +clusterissuers.cert-manager.io 2022-02-22T23:21:36Z +dbcssystems.database.oracle.com 2022-02-22T23:23:25Z <<<< CRD for DBCS Controller +issuers.cert-manager.io 2022-02-22T23:21:36Z +orders.acme.cert-manager.io 2022-02-22T23:21:37Z +shardingdatabases.database.oracle.com 2022-02-22T23:23:25Z +singleinstancedatabases.database.oracle.com 2022-02-22T23:23:25Z +``` + + +# Prerequsites to deploy a DBCS system using Oracle DB Operator DBCS Controller + +Complete the following steps before deploying a DBCS system in OCI using the Oracle DB Operator DBCS Controller: + +**IMPORTANT :** You must make the changes specified in this section before you proceed to the next section. + +## 1. Create a Kubernetes Configmap. For example: We are creating a Kubernetes Configmap named "oci-cred" using the OCI account we are using as below: + +``` +kubectl create configmap oci-cred \ +--from-literal=tenancy=ocid1.tenancy.oc1..................67iypsmea \ +--from-literal=user=ocid1.user.oc1..aaaaaaaaxw3i...............ce6qzdrnmq \ +--from-literal=fingerprint=b2:7c:a8:d5:44:f5.....................:9a:55 \ +--from-literal=region=us-phoenix-1 +``` + + +## 2. Create a Kubernetes secret "oci-privatekey" using the OCI Pem key taken from OCI console for the account you are using: + +``` +-- assuming the OCI Pem key to be "/root/.oci/oci_api_key.pem" + +kubectl create secret generic oci-privatekey --from-file=privatekey=/root/.oci/oci_api_key.pem +``` + + +## 3. Create a Kubernetes secret named "admin-password". This passward needs to satisfy the minimum passward requirements for the OCI DBCS Service. For example: + +``` +-- assuming the passward has been added to a text file named "admin-password": + +kubectl create secret generic admin-password --from-file=./admin-password -n default +``` + + +## 4. Create a Kubernetes secret named "tde-password". This passward needs to satisfy the minimum passward requirements for the OCI DBCS Service. For example: + +``` +-- assuming the passward has been added to a text file named "tde-password": + +kubectl create secret generic tde-password --from-file=./tde-password -n default +``` + + +## 5. Create an ssh key pair and use its public key to create a Kubernetes secret named "oci-publickey". The private key for this public key can be used later to access the DBCS system's host machine using ssh: + +``` +[root@test-server DBCS]# ssh-keygen -N "" -C "DBCS_System"-`date +%Y%m` -P "" +Generating public/private rsa key pair. +Enter file in which to save the key (/root/.ssh/id_rsa): +Your identification has been saved in /root/.ssh/id_rsa. +Your public key has been saved in /root/.ssh/id_rsa.pub. +The key fingerprint is: +SHA256:+SuiES/3m9+iuIVyG/QBQL1x7CfRsxtvswBsaBuW5iE DBCS_System-202203 +The key's randomart image is: ++---[RSA 2048]----+ +| .o. . . | +| .o + o | +| .O . o | +| E X.*.+ | +| .*.=S+ + | +| +oo oo + | +| + * o .o o | +| *.*...o. | +| ..+o==o.. | ++----[SHA256]-----+ + + +[root@test-server DBCS]# kubectl create secret generic oci-publickey --from-file=publickey=/root/DBCS/id_rsa.pub +``` + + + + +# Use Cases to manage life cycle of a OCI DBCS System using Oracle DB Operator DBCS Controller + +There are multiple use cases to deploy and manage the OCI DBCS Service based database using the Oracle DB Operator DBCS Controller. + +[1. Deploy a DB System using OCI DBCS Service with minimal parameters](./provisioning/dbcs_service_with_minimal_parameters.md) +[2. Binding to an existing DBCS System already deployed in OCI DBCS Service](./provisioning/bind_to_existing_dbcs_system.md) +[3. Scale UP the shape of an existing DBCS System](./provisioning/scale_up_dbcs_system_shape.md) +[4. Scale DOWN the shape of an existing DBCS System](./provisioning/scale_down_dbcs_system_shape.md) +[5. Scale UP the storage of an existing DBCS System](./provisioning/scale_up_storage.md) +[6. Update License type of an existing DBCS System](./provisioning/update_license.md) +[7. Terminate an existing DBCS System](./provisioning/terminate_dbcs_system.md) +[8. Create DBCS with All Parameters with Storage Management as LVM](./provisioning/dbcs_service_with_all_parameters_lvm.md) +[9. Create DBCS with All Parameters with Storage Management as ASM](./provisioning/dbcs_service_with_all_parameters_asm.md) +[10. Deploy a 2 Node RAC DB System using OCI DBCS Service](./provisioning/dbcs_service_with_2_node_rac.md) + +## Connecting to OCI DBCS database deployed using Oracle DB Operator DBCS Controller + +After the OCI DBCS database has been deployed using Oracle DB Operator DBCS Controller, you can follow the steps in this document to connect to this Database: [Database Connectivity](./provisioning/database_connection.md) + +## Known Issues + +Please refer to the list of [Known Issues](./provisioning/known_issues.md) for an OCI DBCS System deployed using Oracle DB Operator DBCS Controller. diff --git a/docs/dbcs/provisioning/bind_to_existing_dbcs_system.md b/docs/dbcs/provisioning/bind_to_existing_dbcs_system.md new file mode 100644 index 00000000..6fcff5de --- /dev/null +++ b/docs/dbcs/provisioning/bind_to_existing_dbcs_system.md @@ -0,0 +1,32 @@ +# Binding to an existing DBCS System already deployed in OCI DBCS Service + +In this use case, we bind the Oracle DB Operator DBCS Controller to an existing OCI DBCS System which has already been deployed earlier. This will help to manage the life cycle of that DBCS System using the Oracle DB Operator DBCS Controller. + +**NOTE** We are assuming that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-deploy-a-dbcs-system-using-oracle-db-operator-dbcs-controller) steps to create the configmap and the secrets required during the deployment. + +This example uses `bind_to_existing_dbcs_system.yaml` to bind to an existing DBCS VMDB using Oracle DB Operator DBCS Controller with: + +- OCI Configmap as `oci-cred` +- OCI Secret as `oci-privatekey` +- OCID of the existing DBCS System as `ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa` + + +Use the file: [bind_to_existing_dbcs_system.yaml](./bind_to_existing_dbcs_system.yaml) for this use case as below: + +1. Deploy the .yaml file: +```sh +[root@docker-test-server DBCS]# kubectl apply -f bind_dbcs.yaml +dbcssystem.database.oracle.com/dbcssystem-existing created +``` + +2. Monitor the Oracle DB Operator Pod `pod/oracle-database-operator-controller-manager-665874bd57-g2cgw` for the progress of the DBCS VMDB deployment. + +NOTE: Check the DB Operator Pod name in your environment. + +``` +[root@docker-test-server DBCS]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-g2cgw -n oracle-database-operator-system +``` + +## Sample Output + +[Here](./bind_to_existing_dbcs_system_sample_output.log) is the sample output for binding to an existing DBCS System already deployed in OCI using Oracle DB Operator DBCS Controller. diff --git a/docs/dbcs/provisioning/bind_to_existing_dbcs_system.yaml b/docs/dbcs/provisioning/bind_to_existing_dbcs_system.yaml new file mode 100644 index 00000000..49647229 --- /dev/null +++ b/docs/dbcs/provisioning/bind_to_existing_dbcs_system.yaml @@ -0,0 +1,8 @@ +apiVersion: database.oracle.com/v1alpha1 +kind: DbcsSystem +metadata: + name: dbcssystem-existing +spec: + id: "ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa" + ociConfigMap: "oci-cred" + ociSecret: "oci-privatekey" diff --git a/docs/dbcs/provisioning/bind_to_existing_dbcs_system_sample_output.log b/docs/dbcs/provisioning/bind_to_existing_dbcs_system_sample_output.log new file mode 100644 index 00000000..454a4452 --- /dev/null +++ b/docs/dbcs/provisioning/bind_to_existing_dbcs_system_sample_output.log @@ -0,0 +1,177 @@ +[root@docker-test-server test]# cat bind_to_existing_dbcs_system.yaml +apiVersion: database.oracle.com/v1alpha1 +kind: DbcsSystem +metadata: + name: dbcssystem-existing +spec: + id: "ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa" + ociConfigMap: "oci-cred" + ociSecret: "oci-privatekey" +[root@docker-test-server test]# +[root@docker-test-server test]# +[root@docker-test-server test]# kubectl apply -f bind_to_existing_dbcs_system.yaml +dbcssystem.database.oracle.com/dbcssystem-existing created +[root@docker-test-server test]# + + +[root@docker-test-server test]# kubectl get ns + +kubectl get allNAME STATUS AGE +cert-manager Active 13d +default Active 139d +kube-node-lease Active 139d +kube-public Active 139d +kube-system Active 139d +oracle-database-operator-system Active 13d +shns Active 88d +[root@docker-test-server test]# +[root@docker-test-server test]# kubectl get all -n oracle-database-operator-system +NAME READY STATUS RESTARTS AGE +pod/oracle-database-operator-controller-manager-665874bd57-dlhls 1/1 Running 3 13d +pod/oracle-database-operator-controller-manager-665874bd57-g2cgw 1/1 Running 3 13d +pod/oracle-database-operator-controller-manager-665874bd57-q42f8 1/1 Running 4 13d + +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +service/oracle-database-operator-controller-manager-metrics-service ClusterIP 10.96.130.124 8443/TCP 13d +service/oracle-database-operator-webhook-service ClusterIP 10.96.4.104 443/TCP 13d + +NAME READY UP-TO-DATE AVAILABLE AGE +deployment.apps/oracle-database-operator-controller-manager 3/3 3 3 13d + +NAME DESIRED CURRENT READY AGE +replicaset.apps/oracle-database-operator-controller-manager-665874bd57 3 3 3 13d +[root@docker-test-server test]# +[root@docker-test-server test]# +[root@docker-test-server test]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-dlhls -n oracle-database-operator-system +. +. +2022-03-08T23:27:48.625Z INFO controller-runtime.manager.controller.dbcssystem OCI provider configured succesfully {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-08T23:27:52.513Z INFO controller-runtime.manager.controller.dbcssystem Sync information from remote DbcsSystem System successfully {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} + +[root@docker-test-server test]# kubectl describe dbcssystems.database.oracle.com dbcssystem-existing +Name: dbcssystem-existing +Namespace: default +Labels: +Annotations: lastSuccessfulSpec: + {"dbSystem":{"compartmentId":"ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya","availabilityDomain":"O... +API Version: database.oracle.com/v1alpha1 +Kind: DbcsSystem +Metadata: + Creation Timestamp: 2022-03-08T23:27:48Z + Generation: 1 + Managed Fields: + API Version: database.oracle.com/v1alpha1 + Fields Type: FieldsV1 + fieldsV1: + f:metadata: + f:annotations: + .: + f:kubectl.kubernetes.io/last-applied-configuration: + f:spec: + .: + f:id: + f:ociConfigMap: + f:ociSecret: + Manager: kubectl-client-side-apply + Operation: Update + Time: 2022-03-08T23:27:48Z + API Version: database.oracle.com/v1alpha1 + Fields Type: FieldsV1 + fieldsV1: + f:metadata: + f:annotations: + f:lastSuccessfulSpec: + f:spec: + f:dbSystem: + .: + f:availabilityDomain: + f:compartmentId: + f:cpuCoreCount: + f:dbAdminPaswordSecret: + f:dbBackupConfig: + f:dbEdition: + f:dbName: + f:dbUniqueName: + f:dbVersion: + f:diskRedundancy: + f:displayName: + f:faultDomains: + f:hostName: + f:nodeCount: + f:shape: + f:sshPublicKeys: + f:subnetId: + f:status: + .: + f:availabilityDomain: + f:cpuCoreCount: + f:dataStoragePercentage: + f:dataStorageSizeInGBs: + f:dbEdition: + f:dbInfo: + f:displayName: + f:id: + f:licenseModel: + f:network: + .: + f:clientSubnet: + f:domainName: + f:hostName: + f:listenerPort: + f:scanDnsName: + f:vcnName: + f:nodeCount: + f:recoStorageSizeInGB: + f:shape: + f:state: + f:storageManagement: + f:subnetId: + f:timeZone: + f:workRequests: + Manager: manager + Operation: Update + Time: 2022-03-08T23:27:52Z + Resource Version: 55191827 + UID: 96d7bc49-33e9-42cc-8dd0-ada9a5a4c7e5 +Spec: + Id: ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa + Oci Config Map: oci-cred + Oci Secret: oci-privatekey +Status: + Availability Domain: OLou:PHX-AD-1 + Cpu Core Count: 1 + Data Storage Percentage: 80 + Data Storage Size In G Bs: 256 + Db Edition: ENTERPRISE_EDITION + Db Info: + Db Home Id: ocid1.dbhome.oc1.phx.anyhqljr5gy3jhqat52milqwt3gq6lwohhacwg5yi4mtzq7c7hag53lrkugq + Db Name: db0130 + Db Unique Name: db0130_phx1zn + Db Workload: OLTP + Id: ocid1.database.oc1.phx.anyhqljrabf7htyackgmsaqjfexoqgrzuuk33ju2q25z2al43tnd5mhhvkra + Display Name: dbsystem20220308221302 + Id: ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa + License Model: LICENSE_INCLUDED + Network: + Client Subnet: k8test-pubvcn + Domain Name: k8testpubvcn.k8test.oraclevcn.com + Host Name: host0130 + Listener Port: 1521 + Scan Dns Name: host0130-scan.k8testpubvcn.k8test.oraclevcn.com + Vcn Name: k8test + Node Count: 1 + Reco Storage Size In GB: 256 + Shape: VM.Standard2.1 + State: AVAILABLE + Storage Management: ASM + Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a + Time Zone: UTC + Work Requests: + Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrxivzvgzel47zuoyke5yk36o7mrgjl27vscd5z3bqptmyh3rxwbqq + Operation Type: Create DB System + Percent Complete: 100 + Time Accepted: 2022-03-08 22:13:02.999 +0000 UTC + Time Finished: 2022-03-08 23:11:50.46 +0000 UTC + Time Started: 2022-03-08 22:13:16.995 +0000 UTC +Events: +[root@docker-test-server test]# \ No newline at end of file diff --git a/docs/dbcs/provisioning/database_connection.md b/docs/dbcs/provisioning/database_connection.md new file mode 100644 index 00000000..e69de29b diff --git a/docs/dbcs/provisioning/dbcs_controller_parameters.md b/docs/dbcs/provisioning/dbcs_controller_parameters.md new file mode 100644 index 00000000..82fc3dc2 --- /dev/null +++ b/docs/dbcs/provisioning/dbcs_controller_parameters.md @@ -0,0 +1,33 @@ +# Oracle DB Operator DBCS Controller - Parameters to use in .yaml file + +This page has the details of the parameters to define the specs related to an operation to be performed for an OCI DBCS System to be managed using Oracle DB Operator DBCS Controller. + +| Parameter Name | Description | Mandatory Parameter? (Y/N) | Parameter Value type | Default Value (If Any) | Allowed Values (If Any) | +| -------------- | ---------- | ------- | ------- | ------- | ------- | +| ociConfigMap | Kubernetes Configmap created for OCI account in the prerequisites steps. | Y | String | | | +| ociSecret | Kubernetes Secret created using PEM Key for OCI account in the prerequisites steps. | Y | String | | | +| availabilityDomain | Availability Domain of the OCI region where you want to provision the DBCS System. | Y | String | | Please refer to this link: https://docs.oracle.com/en-us/iaas/Content/General/Concepts/regions.htm | +| compartmentId | OCID of the OCI Compartment. | Y | String | | | +| dbAdminPaswordSecret | Kubernetes Secret created for DB Admin Account in prerequisites steps. | Y | String | | A strong password for SYS, SYSTEM, and PDB Admin. The password must be at least nine characters and contain at least two uppercase, two lowercase, two numbers, and two special characters. The special characters must be _, #, or -.| +| autoBackupEnabled | Whether to enable automatic backup or not. | N | Boolean | | True or False | +| autoBackupWindow | Time window selected for initiating automatic backup for the database system. There are twelve available two-hour time windows. | N | String | | Please refer to this link: https://docs.oracle.com/en-us/iaas/api/#/en/database/20160918/datatypes/DbBackupConfig | +| recoveryWindowsInDays | Number of days between the current and the earliest point of recoverability covered by automatic backups. | N | Integer | | Minimum: 1 and Maximum: 60 | +| dbEdition | Oracle Database Software Edition. | N | String | | STANDARD_EDITION or ENTERPRISE_EDITION or ENTERPRISE_EDITION_HIGH_PERFORMANCE or ENTERPRISE_EDITION_EXTREME_PERFORMANCE | +| dbName | The database name. | Y | String | | The database name cannot be longer than 8 characters. It can only contain alphanumeric characters. | +| dbVersion | The Oracle Database software version. | Y | String | | Min lenght: 1 and Max length: 255 | +| dbWorkload | The database workload type. | Y | String | | OLTP or DSS | +| diskRedundancy | The type of redundancy configured for the DB system. NORMAL is 2-way redundancy. HIGH is 3-way redundancy. | N | String | | HIGH or NORMAL | +| displayName | The user-friendly name for the DB system. The name does not have to be unique. | N | String | | Min length: 1 and Max length: 255 | +| hostName | The hostname for the DB system. | Y | String | | Hostname can contain only alphanumeric and hyphen (-) characters. | +| initialDataStorageSizeInGB | Size (in GB) of the initial data volume that will be created and attached to a virtual machine DB system. | N | Integer | | Min Value in GB: 2 | +| licenseModel | The Oracle license model that applies to all the databases on the DB system. | N | String | LICENSE_INCLUDED | LICENSE_INCLUDED or BRING_YOUR_OWN_LICENSE | +| nodeCount | The number of nodes in the DB system. For RAC DB systems, the value is greater than 1. | N | Integer | | Minimum: 1 | +| pdbName | The name of the pluggable database. The name must begin with an alphabetic character and can contain a maximum of thirty alphanumeric characters. Special characters are not permitted. | N | String | | The PDB name can contain only alphanumeric and underscore (_) characters. | +| privateIp | A private IP address of your choice. Must be an available IP address within the subnet's CIDR. If you don't specify a value, Oracle automatically assigns a private IP address from the subnet. | N | String | | Min length: 1 and Max length: 46 | +| shape | The shape of the DB system. The shape determines resources to allocate to the DB system. | Y | String | | Please refer to this link for the available shapes: https://docs.oracle.com/en-us/iaas/Content/Database/Concepts/overview.htm | +| sshPublicKeys | Kubernetes secret created with the Public Key portion of the key pair created to access the DB System. | Y | String | | | +| storageManagement | The storage option used in DB system. ASM - Automatic storage management LVM - Logical Volume management. | N | String | | ASM or LVM | +| subnetId | The OCID of the subnet the DB system is associated with. | Y | String | | | +| tags | Tags for the DB System resource. Each tag is a simple key-value pair with no predefined name, type, or namespace. | N | String | | | +| tdeWalletPasswordSecret | The Kubernetes secret for the TDE Wallet password. | N | String | | | +| timeZone | The time zone of the DB system. | N | String | | Please refer to this link: https://docs.oracle.com/en-us/iaas/Content/Database/References/timezones.htm#Time_Zone_Options | diff --git a/docs/dbcs/provisioning/dbcs_service_with_2_node_rac.md b/docs/dbcs/provisioning/dbcs_service_with_2_node_rac.md new file mode 100644 index 00000000..8401a0e7 --- /dev/null +++ b/docs/dbcs/provisioning/dbcs_service_with_2_node_rac.md @@ -0,0 +1,59 @@ +# Deploy a 2 Node RAC DB System using OCI DBCS Service + +In this use case, a 2 Node RAC OCI DBCS system is deployed using Oracle DB Operator DBCS controller using all the available parameters in the .yaml file being used during the deployment. The type of the Storage Management in this case is ASM. + +**NOTE** We are assuming that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-deploy-a-dbcs-system-using-oracle-db-operator-dbcs-controller) steps to create the configmap and the secrets required during the deployment. + +This example uses `dbcs_service_with_2_node_rac.yaml` to deploy a 2 Node RAC VMDB using Oracle DB Operator DBCS Controller with: + +- OCI Configmap as `oci-cred` +- OCI Secret as `oci-privatekey` +- Availability Domain for the DBCS VMDB as `OLou:PHX-AD-1` +- Cluster Name as `maa-cluster` +- Compartment OCID as `ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya` +- Database Admin Credential as `admin-password` +- Enable flag for Automatic Backup for DBCS Database as `True` +- Auto Backup Window for DBCS Database as `SLOT_FOUR` +- Recovery Windows for Backup retention in days as `15` +- Oracle Database Edition as `STANDARD_EDITION` +- Database Name as `db0130` +- Oracle Database Software Image Version as `21c` +- Database Workload Type as Transaction Processing i.e. `OLTP` +- Redundancy of the ASM Disks as `EXTERNAL` +- Display Name for the DBCS System as `dbsystem0130` +- Database Hostname Prefix as `host0130` +- Initial Size of the DATA Storage in GB as `256` +- License Model as `BRING_YOUR_OWN_LICENSE` +- Node count as `2` +- Name of the PDB to be created as `PDB0130` +- Private IP explicitly assigned to be `10.0.1.99` +- Oracle VMDB Shape as `VM.Standard2.2` +- SSH Public key for the DBCS system being deployed as `oci-publickey` +- Storage Management type as `ASM` +- OCID of the Subnet as `ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a` +- Tag the DBCS system with two key value pairs as `"TEST": "test_case_provision"` and `"CreatedBy": "MAA_TEAM"` +- TDE Wallet Secret as `tde-password` +- Time Zone for the DBCS System as `Europe/Berlin` + + +**NOTE:** For the details of the parameters to be used in the .yaml file, please refer [here](./dbcs_controller_parameters.md). + +Use the file: [dbcs_service_with_all_parameters_asm.yaml](./dbcs_service_with_2_node_rac.yaml) for this use case as below: + +1. Deploy the .yaml file: +```sh +[root@docker-test-server test]# kubectl apply -f dbcs_service_with_2_node_rac.yaml +dbcssystem.database.oracle.com/dbcssystem-create configured +``` + +2. Monitor the Oracle DB Operator Pod `pod/oracle-database-operator-controller-manager-665874bd57-g2cgw` for the progress of the DBCS VMDB deployment. + +NOTE: Check the DB Operator Pod name in your environment. + +``` +[root@docker-test-server DBCS]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-g2cgw -n oracle-database-operator-system +``` + +## Sample Output + +[Here](./dbcs_service_with_2_node_rac_sample_output.log) is the sample output for a 2 Node RAC DBCS System deployed in OCI using Oracle DB Operator DBCS Controller with all parameters and with Storage Management as ASM. diff --git a/docs/dbcs/provisioning/dbcs_service_with_2_node_rac.yaml b/docs/dbcs/provisioning/dbcs_service_with_2_node_rac.yaml new file mode 100644 index 00000000..183246f8 --- /dev/null +++ b/docs/dbcs/provisioning/dbcs_service_with_2_node_rac.yaml @@ -0,0 +1,38 @@ +apiVersion: database.oracle.com/v1alpha1 +kind: DbcsSystem +metadata: + name: dbcssystem-create +spec: + ociConfigMap: "oci-cred" + ociSecret: "oci-privatekey" + dbSystem: + availabilityDomain: "OLou:PHX-AD-1" + clusterName: "maa-cluster" + compartmentId: "ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya" + dbAdminPaswordSecret: "admin-password" + dbBackupConfig: + autoBackupEnabled: True + autoBackupWindow: "SLOT_FOUR" + recoveryWindowsInDays: 15 + dbEdition: "STANDARD_EDITION" + dbName: "db0130" + dbVersion: "21c" + dbWorkload: "OLTP" + diskRedundancy: "EXTERNAL" + displayName: "dbsystem0130" + hostName: "host0130" + initialDataStorageSizeInGB: 256 + licenseModel: "BRING_YOUR_OWN_LICENSE" + nodeCount: 2 + pdbName: "PDB0130" + privateIp: "10.0.1.99" + shape: "VM.Standard2.2" + sshPublicKeys: + - "oci-publickey" + storageManagement: "ASM" + subnetId: "ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a" + tags: + "TEST": "test_case_provision" + "CreatedBy": "MAA_TEAM" + tdeWalletPasswordSecret: "tde-password" + timeZone: "Europe/Berlin" diff --git a/docs/dbcs/provisioning/dbcs_service_with_2_node_rac_sample_output.log b/docs/dbcs/provisioning/dbcs_service_with_2_node_rac_sample_output.log new file mode 100644 index 00000000..c4c5efcd --- /dev/null +++ b/docs/dbcs/provisioning/dbcs_service_with_2_node_rac_sample_output.log @@ -0,0 +1,352 @@ +[root@docker-test-server test]# cat dbcs_service_with_2_node_rac.yaml +apiVersion: database.oracle.com/v1alpha1 +kind: DbcsSystem +metadata: + name: dbcssystem-create +spec: + ociConfigMap: "oci-cred" + ociSecret: "oci-privatekey" + dbSystem: + availabilityDomain: "OLou:PHX-AD-1" + clusterName: "maa-cluster" + compartmentId: "ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya" + dbAdminPaswordSecret: "admin-password" + dbBackupConfig: + autoBackupEnabled: True + autoBackupWindow: "SLOT_FOUR" + recoveryWindowsInDays: 15 + dbEdition: "STANDARD_EDITION" + dbName: "db0130" + dbVersion: "21c" + dbWorkload: "OLTP" + diskRedundancy: "EXTERNAL" + displayName: "dbsystem0130" + hostName: "host0130" + initialDataStorageSizeInGB: 256 + licenseModel: "BRING_YOUR_OWN_LICENSE" + nodeCount: 2 + pdbName: "PDB0130" + privateIp: "10.0.1.99" + shape: "VM.Standard2.2" + sshPublicKeys: + - "oci-publickey" + storageManagement: "ASM" + subnetId: "ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a" + tags: + "TEST": "test_case_provision" + "CreatedBy": "MAA_TEAM" + tdeWalletPasswordSecret: "tde-password" + timeZone: "Europe/Berlin" +[root@docker-test-server test]# +[root@docker-test-server test]# kubectl apply -f dbcs_service_with_2_node_rac.yaml +dbcssystem.database.oracle.com/dbcssystem-create configured + + + + +[root@docker-test-server test]# kubectl get ns + + + +NAME STATUS AGE +cert-manager Active 14d +default Active 139d +kube-node-lease Active 139d +kube-public Active 139d +kube-system Active 139d +oracle-database-operator-system Active 14d +shns Active 88d +[root@docker-test-server test]# +[root@docker-test-server test]# +[root@docker-test-server test]# +[root@docker-test-server test]# +[root@docker-test-server test]# +[root@docker-test-server test]# kubectl get all -n oracle-database-operator-system +NAME READY STATUS RESTARTS AGE +pod/oracle-database-operator-controller-manager-665874bd57-dlhls 1/1 Running 3 14d +pod/oracle-database-operator-controller-manager-665874bd57-g2cgw 1/1 Running 6 14d +pod/oracle-database-operator-controller-manager-665874bd57-q42f8 1/1 Running 7 14d + +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +service/oracle-database-operator-controller-manager-metrics-service ClusterIP 10.96.130.124 8443/TCP 14d +service/oracle-database-operator-webhook-service ClusterIP 10.96.4.104 443/TCP 14d + +NAME READY UP-TO-DATE AVAILABLE AGE +deployment.apps/oracle-database-operator-controller-manager 3/3 3 3 14d + +NAME DESIRED CURRENT READY AGE +replicaset.apps/oracle-database-operator-controller-manager-665874bd57 3 3 3 14d +[root@docker-test-server test]# +[root@docker-test-server test]# +[root@docker-test-server test]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-dlhls -n oracle-database-operator-system +. +. +2022-03-09T04:56:51.425Z INFO controller-runtime.manager.controller.dbcssystem OCI provider configured succesfully {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T04:56:51.912Z INFO controller-runtime.manager.controller.dbcssystem DbcsSystem DBSystem provisioning {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T04:56:58.650Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T04:57:58.865Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T04:58:59.218Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T04:59:59.440Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:01:00.337Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:02:00.893Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:03:02.191Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:04:02.716Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:05:03.081Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:06:03.311Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:07:03.748Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:08:04.219Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:09:04.561Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:10:05.402Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:11:05.798Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:12:06.382Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:13:06.739Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:14:07.309Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:15:08.005Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:16:08.293Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:17:09.084Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:18:09.600Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:19:09.996Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:20:10.354Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:21:11.059Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:22:11.365Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:23:11.665Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:24:12.008Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:25:12.551Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:26:12.988Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:27:13.371Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:28:13.745Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:29:14.034Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:30:14.407Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:31:14.713Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:32:15.202Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:33:15.451Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:34:15.791Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:35:16.216Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:36:16.526Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:37:17.150Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:38:17.447Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:39:17.790Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:40:18.475Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:41:19.115Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:42:19.717Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:43:20.357Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:44:20.661Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:45:20.888Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:46:21.140Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:47:21.431Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:48:21.902Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:49:22.473Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:50:23.330Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:51:23.947Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:52:24.471Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:53:24.961Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:54:25.256Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:55:25.720Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:56:26.148Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:57:26.807Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:58:27.458Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T05:59:28.274Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T06:00:28.616Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T06:01:28.966Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T06:02:29.594Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T06:03:29.902Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T06:04:30.357Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T06:05:30.791Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T06:06:31.781Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T06:07:32.253Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T06:08:32.581Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T06:09:32.969Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T06:10:33.868Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T06:11:34.492Z INFO controller-runtime.manager.controller.dbcssystem DbcsSystem system provisioned succesfully {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} + + + + +[root@docker-test-server test]# kubectl describe dbcssystems.database.oracle.com dbcssystem-create +Name: dbcssystem-create +Namespace: default +Labels: +Annotations: lastSuccessfulSpec: + {"dbSystem":{"compartmentId":"ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya","availabilityDomain":"O... +API Version: database.oracle.com/v1alpha1 +Kind: DbcsSystem +Metadata: + Creation Timestamp: 2022-03-09T04:54:51Z + Generation: 2 + Managed Fields: + API Version: database.oracle.com/v1alpha1 + Fields Type: FieldsV1 + fieldsV1: + f:metadata: + f:annotations: + .: + f:kubectl.kubernetes.io/last-applied-configuration: + f:spec: + .: + f:dbSystem: + .: + f:availabilityDomain: + f:clusterName: + f:compartmentId: + f:dbAdminPaswordSecret: + f:dbBackupConfig: + .: + f:autoBackupEnabled: + f:autoBackupWindow: + f:recoveryWindowsInDays: + f:dbEdition: + f:dbName: + f:dbVersion: + f:dbWorkload: + f:diskRedundancy: + f:displayName: + f:hostName: + f:initialDataStorageSizeInGB: + f:licenseModel: + f:nodeCount: + f:pdbName: + f:privateIp: + f:shape: + f:sshPublicKeys: + f:storageManagement: + f:subnetId: + f:tags: + .: + f:CreatedBy: + f:TEST: + f:tdeWalletPasswordSecret: + f:timeZone: + f:ociConfigMap: + f:ociSecret: + Manager: kubectl-client-side-apply + Operation: Update + Time: 2022-03-09T04:54:51Z + API Version: database.oracle.com/v1alpha1 + Fields Type: FieldsV1 + fieldsV1: + f:metadata: + f:annotations: + f:lastSuccessfulSpec: + f:spec: + f:id: + f:status: + .: + f:availabilityDomain: + f:cpuCoreCount: + f:dataStoragePercentage: + f:dataStorageSizeInGBs: + f:dbEdition: + f:dbInfo: + f:displayName: + f:id: + f:licenseModel: + f:network: + .: + f:clientSubnet: + f:domainName: + f:hostName: + f:listenerPort: + f:scanDnsName: + f:vcnName: + f:nodeCount: + f:recoStorageSizeInGB: + f:shape: + f:state: + f:storageManagement: + f:subnetId: + f:timeZone: + f:workRequests: + Manager: manager + Operation: Update + Time: 2022-03-09T06:11:37Z + Resource Version: 55318179 + UID: 69389564-7574-4150-b44c-3705ea358800 +Spec: + Db System: + Availability Domain: OLou:PHX-AD-1 + Cluster Name: maa-cluster + Compartment Id: ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya + Db Admin Pasword Secret: admin-password + Db Backup Config: + Auto Backup Enabled: true + Auto Backup Window: SLOT_FOUR + Recovery Windows In Days: 15 + Db Edition: STANDARD_EDITION + Db Name: db0130 + Db Version: 21c + Db Workload: OLTP + Disk Redundancy: EXTERNAL + Display Name: dbsystem0130 + Host Name: host0130 + Initial Data Storage Size In GB: 256 + License Model: BRING_YOUR_OWN_LICENSE + Node Count: 2 + Pdb Name: PDB0130 + Private Ip: 10.0.1.99 + Shape: VM.Standard2.2 + Ssh Public Keys: + oci-publickey + Storage Management: ASM + Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a + Tags: + Created By: MAA_TEAM + TEST: test_case_provision + Tde Wallet Password Secret: tde-password + Time Zone: Europe/Berlin + Oci Config Map: oci-cred + Oci Secret: oci-privatekey +Status: + Availability Domain: OLou:PHX-AD-1 + Cpu Core Count: 4 + Data Storage Percentage: 80 + Data Storage Size In G Bs: 256 + Db Edition: ENTERPRISE_EDITION_EXTREME_PERFORMANCE + Db Info: + Db Home Id: ocid1.dbhome.oc1.phx.anyhqljr5gy3jhqanf2nzf4535im4dgfaliqtyeqa24gu3j5cg2u7676wo2q + Db Name: db0130 + Db Unique Name: db0130_phx1td + Db Workload: OLTP + Id: ocid1.database.oc1.phx.anyhqljrabf7htyalxkrhk636ibrjzbji7d4fnfm6xbhpizxybllfqzykaca + Display Name: dbsystem0130 + Id: ocid1.dbsystem.oc1.phx.anyhqljrabf7htyahapx2mkbpvilwvfhvisk3umch23s3nnz4spx3zthw55a + License Model: BRING_YOUR_OWN_LICENSE + Network: + Client Subnet: k8test-pubvcn + Domain Name: k8testpubvcn.k8test.oraclevcn.com + Host Name: host0130 + Listener Port: 1521 + Scan Dns Name: host0130-scan.k8testpubvcn.k8test.oraclevcn.com + Vcn Name: k8test + Node Count: 2 + Reco Storage Size In GB: 256 + Shape: VM.Standard2.2 + State: AVAILABLE + Storage Management: ASM + Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a + Time Zone: Europe/Berlin + Work Requests: + Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrapznsaptxjhk3o2ao5nzq7axpinpekj7lf36qmd6veh4ntg45hxa + Operation Type: Create DB System + Percent Complete: 100 + Time Accepted: 2022-03-09 04:56:56.007 +0000 UTC + Time Finished: 2022-03-09 06:08:59.539 +0000 UTC + Time Started: 2022-03-09 04:57:18.983 +0000 UTC +Events: + + + +[root@docker-test-server DBCS]# ssh -i id_rsa opc@129.146.35.79 +The authenticity of host '129.146.35.79 (129.146.35.79)' can't be established. +ECDSA key fingerprint is SHA256:KeuW7n18XXH8mFWnSvcMIeER7NpKyfG4njRpN9Xq/Mk. +ECDSA key fingerprint is MD5:64:e9:52:4f:18:14:fb:eb:ed:48:34:9d:15:80:04:5c. +Are you sure you want to continue connecting (yes/no)? yes +Warning: Permanently added '129.146.35.79' (ECDSA) to the list of known hosts. +[opc@host01301 ~]$ +[opc@host01301 ~]$ +[opc@host01301 ~]$ +[opc@host01301 ~]$ sudo su - grid +Last login: Wed Mar 9 18:23:10 CET 2022 + +[grid@host01301 ~]$ +[grid@host01301 ~]$ cemutlo -n +dbSys3zthw55a \ No newline at end of file diff --git a/docs/dbcs/provisioning/dbcs_service_with_all_parameters_asm.md b/docs/dbcs/provisioning/dbcs_service_with_all_parameters_asm.md new file mode 100644 index 00000000..5ef94c30 --- /dev/null +++ b/docs/dbcs/provisioning/dbcs_service_with_all_parameters_asm.md @@ -0,0 +1,58 @@ +# Create DBCS with All Parameters with Storage Management as ASM + +In this use case, the an OCI DBCS system is deployed using Oracle DB Operator DBCS controller using all the available parameters in the .yaml file being used during the deployment. The type of the Storage Management in this case is ASM. + +**NOTE** We are assuming that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-deploy-a-dbcs-system-using-oracle-db-operator-dbcs-controller) steps to create the configmap and the secrets required during the deployment. + +This example uses `dbcs_service_with_all_parameters_asm.yaml` to deploy a Single Instance DBCS VMDB using Oracle DB Operator DBCS Controller with: + +- OCI Configmap as `oci-cred` +- OCI Secret as `oci-privatekey` +- Availability Domain for the DBCS VMDB as `OLou:PHX-AD-1` +- Cluster Name as `maa-cluster` +- Compartment OCID as `ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya` +- Database Admin Credential as `admin-password` +- Enable flag for Automatic Backup for DBCS Database as `True` +- Auto Backup Window for DBCS Database as `SLOT_FOUR` +- Recovery Windows for Backup retention in days as `15` +- Oracle Database Edition as `STANDARD_EDITION` +- Database Name as `db0130` +- Oracle Database Software Image Version as `21c` +- Database Workload Type as Transaction Processing i.e. `OLTP` +- Redundancy of the ASM Disks as `EXTERNAL` +- Display Name for the DBCS System as `dbsystem0130` +- Database Hostname Prefix as `host0130` +- Initial Size of the DATA Storage in GB as `256` +- License Model as `BRING_YOUR_OWN_LICENSE` +- Name of the PDB to be created as `PDB0130` +- Private IP explicitly assigned to be `10.0.1.99` +- Oracle VMDB Shape as `VM.Standard2.1` +- SSH Public key for the DBCS system being deployed as `oci-publickey` +- Storage Management type as `ASM` +- OCID of the Subnet as `ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a` +- Tag the DBCS system with two key value pairs as `"TEST": "test_case_provision"` and `"CreatedBy": "MAA_TEAM"` +- TDE Wallet Secret as `tde-password` +- Time Zone for the DBCS System as `Europe/Berlin` + + +**NOTE:** For the details of the parameters to be used in the .yaml file, please refer [here](./dbcs_controller_parameters.md). + +Use the file: [dbcs_service_with_all_parameters_asm.yaml](./dbcs_service_with_all_parameters_asm.yaml) for this use case as below: + +1. Deploy the .yaml file: +```sh +[root@docker-test-server DBCS]# kubectl apply -f dbcs_service_with_all_parameters_asm.yaml +dbcssystem.database.oracle.com/dbcssystem-create created +``` + +2. Monitor the Oracle DB Operator Pod `pod/oracle-database-operator-controller-manager-665874bd57-g2cgw` for the progress of the DBCS VMDB deployment. + +NOTE: Check the DB Operator Pod name in your environment. + +``` +[root@docker-test-server DBCS]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-g2cgw -n oracle-database-operator-system +``` + +## Sample Output + +[Here](./dbcs_service_with_all_parameters_asm_sample_output.log) is the sample output for a DBCS System deployed in OCI using Oracle DB Operator DBCS Controller with all parameters and with Storage Management as ASM. diff --git a/docs/dbcs/provisioning/dbcs_service_with_all_parameters_asm.yaml b/docs/dbcs/provisioning/dbcs_service_with_all_parameters_asm.yaml new file mode 100644 index 00000000..1dcec54c --- /dev/null +++ b/docs/dbcs/provisioning/dbcs_service_with_all_parameters_asm.yaml @@ -0,0 +1,36 @@ +apiVersion: database.oracle.com/v1alpha1 +kind: DbcsSystem +metadata: + name: dbcssystem-create +spec: + ociConfigMap: "oci-cred" + ociSecret: "oci-privatekey" + dbSystem: + availabilityDomain: "OLou:PHX-AD-1" + compartmentId: "ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya" + dbAdminPaswordSecret: "admin-password" + dbBackupConfig: + autoBackupEnabled: True + autoBackupWindow: "SLOT_FOUR" + recoveryWindowsInDays: 15 + dbEdition: "STANDARD_EDITION" + dbName: "db0130" + dbVersion: "21c" + dbWorkload: "OLTP" + diskRedundancy: "EXTERNAL" + displayName: "dbsystem0130" + hostName: "host0130" + initialDataStorageSizeInGB: 256 + licenseModel: "BRING_YOUR_OWN_LICENSE" + pdbName: "PDB0130" + privateIp: "10.0.1.99" + shape: "VM.Standard2.1" + sshPublicKeys: + - "oci-publickey" + storageManagement: "ASM" + subnetId: "ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a" + tags: + "TEST": "test_case_provision" + "CreatedBy": "MAA_TEAM" + tdeWalletPasswordSecret: "tde-password" + timeZone: "Europe/Berlin" diff --git a/docs/dbcs/provisioning/dbcs_service_with_all_parameters_asm_sample_output.log b/docs/dbcs/provisioning/dbcs_service_with_all_parameters_asm_sample_output.log new file mode 100644 index 00000000..a2fc6690 --- /dev/null +++ b/docs/dbcs/provisioning/dbcs_service_with_all_parameters_asm_sample_output.log @@ -0,0 +1,313 @@ +[root@docker-test-server test]# cat dbcs_service_with_all_parameters_asm.yaml +apiVersion: database.oracle.com/v1alpha1 +kind: DbcsSystem +metadata: + name: dbcssystem-create +spec: + ociConfigMap: "oci-cred" + ociSecret: "oci-privatekey" + dbSystem: + availabilityDomain: "OLou:PHX-AD-1" + compartmentId: "ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya" + dbAdminPaswordSecret: "admin-password" + dbBackupConfig: + autoBackupEnabled: True + autoBackupWindow: "SLOT_FOUR" + recoveryWindowsInDays: 15 + dbEdition: "STANDARD_EDITION" + dbName: "db0130" + dbVersion: "21c" + dbWorkload: "OLTP" + diskRedundancy: "EXTERNAL" + displayName: "dbsystem0130" + hostName: "host0130" + initialDataStorageSizeInGB: 256 + licenseModel: "BRING_YOUR_OWN_LICENSE" + pdbName: "PDB0130" + privateIp: "10.0.1.99" + shape: "VM.Standard2.1" + sshPublicKeys: + - "oci-publickey" + storageManagement: "ASM" + subnetId: "ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a" + tags: + "TEST": "test_case_provision" + "CreatedBy": "MAA_TEAM" + tdeWalletPasswordSecret: "tde-password" + timeZone: "Europe/Berlin" +[root@docker-test-server test]# +[root@docker-test-server test]# +[root@docker-test-server test]# kubectl apply -f dbcs_service_with_all_parameters_asm.yaml +dbcssystem.database.oracle.com/dbcssystem-create created + + + + + +[root@docker-test-server test]# kubectl get ns + +kubectl get allNAME STATUS AGE +cert-manager Active 13d +default Active 139d +kube-node-lease Active 139d +kube-public Active 139d +kube-system Active 139d +oracle-database-operator-system Active 13d +shns Active 88d +[root@docker-test-server test]# +[root@docker-test-server test]# kubectl get all -n oracle-database-operator-system +NAME READY STATUS RESTARTS AGE +pod/oracle-database-operator-controller-manager-665874bd57-dlhls 1/1 Running 3 13d +pod/oracle-database-operator-controller-manager-665874bd57-g2cgw 1/1 Running 3 13d +pod/oracle-database-operator-controller-manager-665874bd57-q42f8 1/1 Running 4 13d + +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +service/oracle-database-operator-controller-manager-metrics-service ClusterIP 10.96.130.124 8443/TCP 13d +service/oracle-database-operator-webhook-service ClusterIP 10.96.4.104 443/TCP 13d + +NAME READY UP-TO-DATE AVAILABLE AGE +deployment.apps/oracle-database-operator-controller-manager 3/3 3 3 13d + +NAME DESIRED CURRENT READY AGE +replicaset.apps/oracle-database-operator-controller-manager-665874bd57 3 3 3 13d +[root@docker-test-server test]# + + + + +[root@docker-test-server test]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-dlhls -n oracle-database-operator-system +. +. +2022-03-09T02:59:43.691Z INFO controller-runtime.manager.controller.dbcssystem OCI provider configured succesfully {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T02:59:44.410Z INFO controller-runtime.manager.controller.dbcssystem DbcsSystem DBSystem provisioning {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T02:59:52.341Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:00:52.845Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:01:53.382Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:02:53.737Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:03:54.188Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:04:54.545Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:05:55.030Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:06:55.429Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:07:55.789Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:08:56.188Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:09:56.905Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:10:57.308Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:11:58.068Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:12:58.444Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:13:58.840Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:14:59.194Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:15:59.772Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:17:00.249Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:18:00.599Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:19:00.881Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:20:01.121Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:21:01.488Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:22:01.874Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:23:02.726Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:24:03.634Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:25:03.978Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:26:04.450Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:27:04.763Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:28:05.246Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:29:05.825Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:30:06.398Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:31:07.256Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:32:07.551Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:33:08.057Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:34:08.452Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:35:08.772Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:36:09.216Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:37:09.584Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:38:09.881Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:39:10.602Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:40:10.869Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:41:11.301Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:42:12.468Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:43:12.732Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:44:13.243Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:45:13.582Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:46:13.873Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:47:14.440Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:48:14.941Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:49:15.381Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:50:16.038Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:51:16.335Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:52:16.785Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:53:17.374Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:54:17.675Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:55:18.054Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:56:18.623Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:57:19.033Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:58:19.611Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T03:59:20.320Z INFO controller-runtime.manager.controller.dbcssystem DbcsSystem system provisioned succesfully {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} + + + + +[root@docker-test-server test]# kubectl describe dbcssystems.database.oracle.com dbcssystem-create +Name: dbcssystem-create +Namespace: default +Labels: +Annotations: lastSuccessfulSpec: + {"dbSystem":{"compartmentId":"ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya","availabilityDomain":"O... +API Version: database.oracle.com/v1alpha1 +Kind: DbcsSystem +Metadata: + Creation Timestamp: 2022-03-09T02:59:43Z + Generation: 1 + Managed Fields: + API Version: database.oracle.com/v1alpha1 + Fields Type: FieldsV1 + fieldsV1: + f:metadata: + f:annotations: + .: + f:kubectl.kubernetes.io/last-applied-configuration: + f:spec: + .: + f:dbSystem: + .: + f:availabilityDomain: + f:compartmentId: + f:dbAdminPaswordSecret: + f:dbBackupConfig: + .: + f:autoBackupEnabled: + f:autoBackupWindow: + f:recoveryWindowsInDays: + f:dbEdition: + f:dbName: + f:dbVersion: + f:dbWorkload: + f:diskRedundancy: + f:displayName: + f:hostName: + f:initialDataStorageSizeInGB: + f:licenseModel: + f:pdbName: + f:privateIp: + f:shape: + f:sshPublicKeys: + f:storageManagement: + f:subnetId: + f:tags: + .: + f:CreatedBy: + f:TEST: + f:tdeWalletPasswordSecret: + f:timeZone: + f:ociConfigMap: + f:ociSecret: + Manager: kubectl-client-side-apply + Operation: Update + Time: 2022-03-09T02:59:43Z + API Version: database.oracle.com/v1alpha1 + Fields Type: FieldsV1 + fieldsV1: + f:metadata: + f:annotations: + f:lastSuccessfulSpec: + f:spec: + f:id: + f:status: + .: + f:availabilityDomain: + f:cpuCoreCount: + f:dataStoragePercentage: + f:dataStorageSizeInGBs: + f:dbEdition: + f:dbInfo: + f:displayName: + f:id: + f:licenseModel: + f:network: + .: + f:clientSubnet: + f:domainName: + f:hostName: + f:listenerPort: + f:scanDnsName: + f:vcnName: + f:nodeCount: + f:recoStorageSizeInGB: + f:shape: + f:state: + f:storageManagement: + f:subnetId: + f:timeZone: + f:workRequests: + Manager: manager + Operation: Update + Time: 2022-03-09T03:59:22Z + Resource Version: 55276756 + UID: e7d874e7-3cd7-4b8b-8cd1-32d68795a38c +Spec: + Db System: + Availability Domain: OLou:PHX-AD-1 + Compartment Id: ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya + Db Admin Pasword Secret: admin-password + Db Backup Config: + Auto Backup Enabled: true + Auto Backup Window: SLOT_FOUR + Recovery Windows In Days: 15 + Db Edition: STANDARD_EDITION + Db Name: db0130 + Db Version: 21c + Db Workload: OLTP + Disk Redundancy: EXTERNAL + Display Name: dbsystem0130 + Host Name: host0130 + Initial Data Storage Size In GB: 256 + License Model: BRING_YOUR_OWN_LICENSE + Pdb Name: PDB0130 + Private Ip: 10.0.1.99 + Shape: VM.Standard2.1 + Ssh Public Keys: + oci-publickey + Storage Management: ASM + Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a + Tags: + Created By: MAA_TEAM + TEST: test_case_provision + Tde Wallet Password Secret: tde-password + Time Zone: Europe/Berlin + Oci Config Map: oci-cred + Oci Secret: oci-privatekey +Status: + Availability Domain: OLou:PHX-AD-1 + Cpu Core Count: 1 + Data Storage Percentage: 80 + Data Storage Size In G Bs: 256 + Db Edition: STANDARD_EDITION + Db Info: + Db Home Id: ocid1.dbhome.oc1.phx.anyhqljr5gy3jhqaubltt77vlwmsx7w5d5dvq6be7isglwbpqijfi5gflh5a + Db Name: db0130 + Db Unique Name: db0130_phx1sw + Db Workload: OLTP + Id: ocid1.database.oc1.phx.anyhqljrabf7htya5bzvoxrrc2qu6yjw6c27hcsx32bp7c76vzy35kesa2nq + Display Name: dbsystem0130 + Id: ocid1.dbsystem.oc1.phx.anyhqljrabf7htyaz42sxinatef6xieeppxmwg3bwlw5chpefc52s4joraxq + License Model: BRING_YOUR_OWN_LICENSE + Network: + Client Subnet: k8test-pubvcn + Domain Name: k8testpubvcn.k8test.oraclevcn.com + Host Name: host0130 + Listener Port: 1521 + Scan Dns Name: host0130-scan.k8testpubvcn.k8test.oraclevcn.com + Vcn Name: k8test + Node Count: 1 + Reco Storage Size In GB: 256 + Shape: VM.Standard2.1 + State: AVAILABLE + Storage Management: ASM + Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a + Time Zone: Europe/Berlin + Work Requests: + Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrlpxe723pq3z5fkeyfgbu4ewsysjcdrxiyxigponwosy44uhcpcsq + Operation Type: Create DB System + Percent Complete: 100 + Time Accepted: 2022-03-09 02:59:48.969 +0000 UTC + Time Finished: 2022-03-09 03:56:52.77 +0000 UTC + Time Started: 2022-03-09 02:59:56.287 +0000 UTC +Events: +[root@docker-test-server test]# \ No newline at end of file diff --git a/docs/dbcs/provisioning/dbcs_service_with_all_parameters_lvm.md b/docs/dbcs/provisioning/dbcs_service_with_all_parameters_lvm.md new file mode 100644 index 00000000..d6beeb16 --- /dev/null +++ b/docs/dbcs/provisioning/dbcs_service_with_all_parameters_lvm.md @@ -0,0 +1,55 @@ +# Create DBCS with All Parameters with Storage Management as LVM + +In this use case, the an OCI DBCS system is deployed using Oracle DB Operator DBCS controller using all the available parameters in the .yaml file being used during the deployment. + +**NOTE** We are assuming that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-deploy-a-dbcs-system-using-oracle-db-operator-dbcs-controller) steps to create the configmap and the secrets required during the deployment. + +This example uses `dbcs_service_with_all_parameters_lvm.yaml` to deploy a Single Instance DBCS VMDB using Oracle DB Operator DBCS Controller with: + +- OCI Configmap as `oci-cred` +- OCI Secret as `oci-privatekey` +- Availability Domain for the DBCS VMDB as `OLou:PHX-AD-1` +- Compartment OCID as `ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya` +- Database Admin Credential as `admin-password` +- Enable flag for Automatic Backup for DBCS Database as `True` +- Auto Backup Window for DBCS Database as `SLOT_FOUR` +- Recovery Windows for Backup retention in days as `15` +- Oracle Database Edition as `STANDARD_EDITION` +- Database Name as `db0130` +- Oracle Database Software Image Version as `21c` +- Database Workload Type as Transaction Processing i.e. `OLTP` +- Display Name for the DBCS System as `dbsystem0130` +- Database Hostname Prefix as `host0130` +- Initial Size of the DATA Storage in GB as `256` +- License Model as `BRING_YOUR_OWN_LICENSE` +- Name of the PDB to be created as `PDB0130` +- Oracle VMDB Shape as `VM.Standard2.1` +- SSH Public key for the DBCS system being deployed as `oci-publickey` +- Storage Management type as `LVM` +- OCID of the Subnet as `ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a` +- Tag the DBCS system with two key value pairs as `"TEST": "test_case_provision"` and `"CreatedBy": "MAA_TEAM"` +- TDE Wallet Secret as `tde-password` +- Time Zone for the DBCS System as `Europe/Berlin` + + +**NOTE:** For the details of the parameters to be used in the .yaml file, please refer [here](./dbcs_controller_parameters.md). + +Use the file: [dbcs_service_with_all_parameters_lvm.yaml](./dbcs_service_with_all_parameters_lvm.yaml) for this use case as below: + +1. Deploy the .yaml file: +```sh +[root@docker-test-server DBCS]# kubectl apply -f dbcs_service_with_all_parameters_lvm.yaml +dbcssystem.database.oracle.com/dbcssystem-create created +``` + +2. Monitor the Oracle DB Operator Pod `pod/oracle-database-operator-controller-manager-665874bd57-g2cgw` for the progress of the DBCS VMDB deployment. + +NOTE: Check the DB Operator Pod name in your environment. + +``` +[root@docker-test-server DBCS]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-g2cgw -n oracle-database-operator-system +``` + +## Sample Output + +[Here](./dbcs_service_with_all_parameters_lvm_sample_output.log) is the sample output for a DBCS System deployed in OCI using Oracle DB Operator DBCS Controller with all parameters and with Storage Management as LVM. diff --git a/docs/dbcs/provisioning/dbcs_service_with_all_parameters_lvm.yaml b/docs/dbcs/provisioning/dbcs_service_with_all_parameters_lvm.yaml new file mode 100644 index 00000000..73208317 --- /dev/null +++ b/docs/dbcs/provisioning/dbcs_service_with_all_parameters_lvm.yaml @@ -0,0 +1,34 @@ +apiVersion: database.oracle.com/v1alpha1 +kind: DbcsSystem +metadata: + name: dbcssystem-create +spec: + ociConfigMap: "oci-cred" + ociSecret: "oci-privatekey" + dbSystem: + availabilityDomain: "OLou:PHX-AD-1" + compartmentId: "ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya" + dbAdminPaswordSecret: "admin-password" + dbBackupConfig: + autoBackupEnabled: True + autoBackupWindow: "SLOT_FOUR" + recoveryWindowsInDays: 15 + dbEdition: "STANDARD_EDITION" + dbName: "db0130" + dbVersion: "21c" + dbWorkload: "OLTP" + displayName: "dbsystem0130" + hostName: "host0130" + initialDataStorageSizeInGB: 256 + licenseModel: "BRING_YOUR_OWN_LICENSE" + pdbName: "PDB0130" + shape: "VM.Standard2.1" + sshPublicKeys: + - "oci-publickey" + storageManagement: "LVM" + subnetId: "ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a" + tags: + "TEST": "test_case_provision" + "CreatedBy": "MAA_TEAM" + tdeWalletPasswordSecret: "tde-password" + timeZone: "Europe/Berlin" diff --git a/docs/dbcs/provisioning/dbcs_service_with_all_parameters_lvm_sample_output.log b/docs/dbcs/provisioning/dbcs_service_with_all_parameters_lvm_sample_output.log new file mode 100644 index 00000000..f6946eff --- /dev/null +++ b/docs/dbcs/provisioning/dbcs_service_with_all_parameters_lvm_sample_output.log @@ -0,0 +1,266 @@ +[root@docker-test-server test]# cat dbcs_service_with_all_parameters_lvm.yaml +apiVersion: database.oracle.com/v1alpha1 +kind: DbcsSystem +metadata: + name: dbcssystem-create +spec: + ociConfigMap: "oci-cred" + ociSecret: "oci-privatekey" + dbSystem: + availabilityDomain: "OLou:PHX-AD-1" + compartmentId: "ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya" + dbAdminPaswordSecret: "admin-password" + dbBackupConfig: + autoBackupEnabled: True + autoBackupWindow: "SLOT_FOUR" + recoveryWindowsInDays: 15 + dbEdition: "STANDARD_EDITION" + dbName: "db0130" + dbVersion: "21c" + dbWorkload: "OLTP" + displayName: "dbsystem0130" + hostName: "host0130" + initialDataStorageSizeInGB: 256 + licenseModel: "BRING_YOUR_OWN_LICENSE" + pdbName: "PDB0130" + shape: "VM.Standard2.1" + sshPublicKeys: + - "oci-publickey" + storageManagement: "LVM" + subnetId: "ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a" + tags: + "TEST": "test_case_provision" + "CreatedBy": "MAA_TEAM" + tdeWalletPasswordSecret: "tde-password" + timeZone: "Europe/Berlin" +[root@docker-test-server test]# +[root@docker-test-server test]# kubectl apply -f dbcs_service_with_all_parameters_lvm.yaml +dbcssystem.database.oracle.com/dbcssystem-create created +[root@docker-test-server test]# + + +[root@docker-test-server test]# kubectl get ns + +kubectl get allNAME STATUS AGE +cert-manager Active 13d +default Active 139d +kube-node-lease Active 139d +kube-public Active 139d +kube-system Active 139d +oracle-database-operator-system Active 13d +shns Active 88d +[root@docker-test-server test]# +[root@docker-test-server test]# kubectl get all -n oracle-database-operator-system +NAME READY STATUS RESTARTS AGE +pod/oracle-database-operator-controller-manager-665874bd57-dlhls 1/1 Running 3 13d +pod/oracle-database-operator-controller-manager-665874bd57-g2cgw 1/1 Running 3 13d +pod/oracle-database-operator-controller-manager-665874bd57-q42f8 1/1 Running 4 13d + +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +service/oracle-database-operator-controller-manager-metrics-service ClusterIP 10.96.130.124 8443/TCP 13d +service/oracle-database-operator-webhook-service ClusterIP 10.96.4.104 443/TCP 13d + +NAME READY UP-TO-DATE AVAILABLE AGE +deployment.apps/oracle-database-operator-controller-manager 3/3 3 3 13d + +NAME DESIRED CURRENT READY AGE +replicaset.apps/oracle-database-operator-controller-manager-665874bd57 3 3 3 13d +[root@docker-test-server test]# + + + + +[root@docker-test-server test]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-dlhls -n oracle-database-operator-system +. +. +2022-03-09T01:28:57.125Z INFO controller-runtime.manager.controller.dbcssystem DbcsSystem DBSystem provisioning {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T01:29:04.321Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T01:30:04.972Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T01:31:05.417Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T01:32:05.728Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T01:33:06.284Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T01:34:06.763Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T01:35:07.237Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T01:36:07.594Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T01:37:08.416Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T01:38:08.724Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T01:39:08.998Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T01:40:09.408Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T01:41:10.348Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T01:42:10.845Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T01:43:11.152Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T01:44:11.560Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T01:45:11.927Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T01:46:12.217Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T01:47:12.442Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-09T01:48:12.826Z INFO controller-runtime.manager.controller.dbcssystem DbcsSystem system provisioned succesfully {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +[root@docker-test-server test]# + + + + + + +[root@docker-test-server test]# kubectl describe dbcssystems.database.oracle.com dbcssystem-create +Name: dbcssystem-create +Namespace: default +Labels: +Annotations: lastSuccessfulSpec: + {"dbSystem":{"compartmentId":"ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya","availabilityDomain":"O... +API Version: database.oracle.com/v1alpha1 +Kind: DbcsSystem +Metadata: + Creation Timestamp: 2022-03-09T01:28:56Z + Generation: 1 + Managed Fields: + API Version: database.oracle.com/v1alpha1 + Fields Type: FieldsV1 + fieldsV1: + f:metadata: + f:annotations: + .: + f:kubectl.kubernetes.io/last-applied-configuration: + f:spec: + .: + f:dbSystem: + .: + f:availabilityDomain: + f:compartmentId: + f:dbAdminPaswordSecret: + f:dbBackupConfig: + .: + f:autoBackupEnabled: + f:autoBackupWindow: + f:recoveryWindowsInDays: + f:dbEdition: + f:dbName: + f:dbVersion: + f:dbWorkload: + f:displayName: + f:hostName: + f:initialDataStorageSizeInGB: + f:licenseModel: + f:pdbName: + f:shape: + f:sshPublicKeys: + f:storageManagement: + f:subnetId: + f:tags: + .: + f:CreatedBy: + f:TEST: + f:tdeWalletPasswordSecret: + f:timeZone: + f:ociConfigMap: + f:ociSecret: + Manager: kubectl-client-side-apply + Operation: Update + Time: 2022-03-09T01:28:56Z + API Version: database.oracle.com/v1alpha1 + Fields Type: FieldsV1 + fieldsV1: + f:metadata: + f:annotations: + f:lastSuccessfulSpec: + f:spec: + f:id: + f:status: + .: + f:availabilityDomain: + f:cpuCoreCount: + f:dataStoragePercentage: + f:dataStorageSizeInGBs: + f:dbEdition: + f:dbInfo: + f:displayName: + f:id: + f:licenseModel: + f:network: + .: + f:clientSubnet: + f:domainName: + f:hostName: + f:listenerPort: + f:vcnName: + f:nodeCount: + f:recoStorageSizeInGB: + f:shape: + f:state: + f:storageManagement: + f:subnetId: + f:timeZone: + f:workRequests: + Manager: manager + Operation: Update + Time: 2022-03-09T01:48:12Z + Resource Version: 55235730 + UID: 53f67e5d-7725-4c8d-a3c2-53ac82f6ef11 +Spec: + Db System: + Availability Domain: OLou:PHX-AD-1 + Compartment Id: ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya + Db Admin Pasword Secret: admin-password + Db Backup Config: + Auto Backup Enabled: true + Auto Backup Window: SLOT_FOUR + Recovery Windows In Days: 15 + Db Edition: STANDARD_EDITION + Db Name: db0130 + Db Version: 21c + Db Workload: OLTP + Display Name: dbsystem0130 + Host Name: host0130 + Initial Data Storage Size In GB: 256 + License Model: BRING_YOUR_OWN_LICENSE + Pdb Name: PDB0130 + Shape: VM.Standard2.1 + Ssh Public Keys: + oci-publickey + Storage Management: LVM + Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a + Tags: + Created By: MAA_TEAM + TEST: test_case_provision + Tde Wallet Password Secret: tde-password + Time Zone: Europe/Berlin + Oci Config Map: oci-cred + Oci Secret: oci-privatekey +Status: + Availability Domain: OLou:PHX-AD-1 + Cpu Core Count: 1 + Data Storage Percentage: 80 + Data Storage Size In G Bs: 256 + Db Edition: STANDARD_EDITION + Db Info: + Db Home Id: ocid1.dbhome.oc1.phx.anyhqljr5gy3jhqahugk47wa6hp36fwamqh24lv7bavbqleyerdjgpoublgq + Db Name: db0130 + Db Unique Name: db0130_phx1t6 + Db Workload: OLTP + Id: ocid1.database.oc1.phx.anyhqljrabf7htya4wwpjsm6bc4jlipqrxl7lpgm5dt7rjpfcwnuynslifra + Display Name: dbsystem0130 + Id: ocid1.dbsystem.oc1.phx.anyhqljrabf7htya3y2uepxpcpy4t2gv5ctnw3r2jkfaevxloydy5uilgpna + License Model: BRING_YOUR_OWN_LICENSE + Network: + Client Subnet: k8test-pubvcn + Domain Name: k8testpubvcn.k8test.oraclevcn.com + Host Name: host0130 + Listener Port: 1521 + Vcn Name: k8test + Node Count: 1 + Reco Storage Size In GB: 256 + Shape: VM.Standard2.1 + State: AVAILABLE + Storage Management: LVM + Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a + Time Zone: Europe/Berlin + Work Requests: + Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrmfvsuabnnapzaxlpzxyipcfbqlquxd4yg7cfw57ectybunbjw4tq + Operation Type: Create DB System + Percent Complete: 100 + Time Accepted: 2022-03-09 01:29:01.315 +0000 UTC + Time Finished: 2022-03-09 01:46:27.292 +0000 UTC + Time Started: 2022-03-09 01:29:13.294 +0000 UTC +Events: +[root@docker-test-server test]# + + diff --git a/docs/dbcs/provisioning/dbcs_service_with_minimal_parameters.md b/docs/dbcs/provisioning/dbcs_service_with_minimal_parameters.md new file mode 100644 index 00000000..f91370ca --- /dev/null +++ b/docs/dbcs/provisioning/dbcs_service_with_minimal_parameters.md @@ -0,0 +1,43 @@ +# Deploy a DBCS DB System using OCI DBCS Service with minimal parameters + +In this use case, an OCI DBCS system is deployed using Oracle DB Operator DBCS controller using minimal required parameters in the .yaml file being used during the deployment. + +**NOTE** It is assumed that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-deploy-a-dbcs-system-using-oracle-db-operator-dbcs-controller) steps to create the configmap and the secrets required during the deployment. + +This example uses `dbcs_service_with_minimal_parameters.yaml` to deploy a Single Instance DBCS VMDB using Oracle DB Operator DBCS Controller with: + +- OCI Configmap as `oci-cred` +- OCI Secret as `oci-privatekey` +- Availability Domain for the DBCS VMDB as `OLou:PHX-AD-1` +- Compartment OCID as `ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya` +- Database Admin Credential as `admin-password` +- Database Name as `db0130` +- Oracle Database Software Image Version as `21c` +- Database Workload Type as Transaction Processing i.e. `OLTP` +- Database Hostname Prefix as `host0130` +- Oracle VMDB Shape as `VM.Standard2.1` +- SSH Public key for the DBCS system being deployed as `oci-publickey` +- OCID of the Subnet as `ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a` + + +**NOTE:** For the details of the parameters to be used in the .yaml file, please refer [here](./dbcs_controller_parameters.md). + +Use the file: [dbcs_service_with_minimal_parameters.yaml](./dbcs_service_with_minimal_parameters.yaml) for this use case as below: + +1. Deploy the .yaml file: +```sh +[root@docker-test-server DBCS]# kubectl apply -f create_required.yaml +dbcssystem.database.oracle.com/dbcssystem-create created +``` + +2. Monitor the Oracle DB Operator Pod `pod/oracle-database-operator-controller-manager-665874bd57-g2cgw` for the progress of the DBCS VMDB deployment. + +NOTE: Check the DB Operator Pod name in your environment. + +``` +[root@docker-test-server DBCS]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-g2cgw -n oracle-database-operator-system +``` + +## Sample Output + +[Here](./dbcs_service_with_minimal_parameters_sample_output.log) is the sample output for a DBCS System deployed in OCI using Oracle DB Operator DBCS Controller with minimal parameters. diff --git a/docs/dbcs/provisioning/dbcs_service_with_minimal_parameters.yaml b/docs/dbcs/provisioning/dbcs_service_with_minimal_parameters.yaml new file mode 100644 index 00000000..9e4a5f3d --- /dev/null +++ b/docs/dbcs/provisioning/dbcs_service_with_minimal_parameters.yaml @@ -0,0 +1,19 @@ +apiVersion: database.oracle.com/v1alpha1 +kind: DbcsSystem +metadata: + name: dbcssystem-create +spec: + ociConfigMap: "oci-cred" + ociSecret: "oci-privatekey" + dbSystem: + availabilityDomain: "OLou:PHX-AD-1" + compartmentId: "ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya" + dbAdminPaswordSecret: "admin-password" + dbName: "db0130" + dbVersion: "21c" + dbWorkload: "OLTP" + hostName: "host0130" + shape: "VM.Standard2.1" + sshPublicKeys: + - "oci-publickey" + subnetId: "ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a" diff --git a/docs/dbcs/provisioning/dbcs_service_with_minimal_parameters_sample_output.log b/docs/dbcs/provisioning/dbcs_service_with_minimal_parameters_sample_output.log new file mode 100644 index 00000000..ddf1d81a --- /dev/null +++ b/docs/dbcs/provisioning/dbcs_service_with_minimal_parameters_sample_output.log @@ -0,0 +1,257 @@ +[root@docker-test-server test]# cat dbcs_service_with_minimal_parameters.yaml +apiVersion: database.oracle.com/v1alpha1 +kind: DbcsSystem +metadata: + name: dbcssystem-create +spec: + ociConfigMap: "oci-cred" + ociSecret: "oci-privatekey" + dbSystem: + availabilityDomain: "OLou:PHX-AD-1" + compartmentId: "ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya" + dbAdminPaswordSecret: "admin-password" + dbName: "db0130" + dbVersion: "21c" + dbWorkload: "OLTP" + hostName: "host0130" + shape: "VM.Standard2.1" + sshPublicKeys: + - "oci-publickey" + subnetId: "ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a" +[root@docker-test-server test]# +[root@docker-test-server test]# kubectl apply -f dbcs_service_with_minimal_parameters.yaml +dbcssystem.database.oracle.com/dbcssystem-create created + + +[root@docker-test-server test]# kubectl get ns + +kubectl get allNAME STATUS AGE +cert-manager Active 13d +default Active 139d +kube-node-lease Active 139d +kube-public Active 139d +kube-system Active 139d +oracle-database-operator-system Active 13d +shns Active 88d +[root@docker-test-server test]# +[root@docker-test-server test]# kubectl get all -n oracle-database-operator-system +NAME READY STATUS RESTARTS AGE +pod/oracle-database-operator-controller-manager-665874bd57-dlhls 1/1 Running 3 13d +pod/oracle-database-operator-controller-manager-665874bd57-g2cgw 1/1 Running 3 13d +pod/oracle-database-operator-controller-manager-665874bd57-q42f8 1/1 Running 4 13d + +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +service/oracle-database-operator-controller-manager-metrics-service ClusterIP 10.96.130.124 8443/TCP 13d +service/oracle-database-operator-webhook-service ClusterIP 10.96.4.104 443/TCP 13d + +NAME READY UP-TO-DATE AVAILABLE AGE +deployment.apps/oracle-database-operator-controller-manager 3/3 3 3 13d + +NAME DESIRED CURRENT READY AGE +replicaset.apps/oracle-database-operator-controller-manager-665874bd57 3 3 3 13d +[root@docker-test-server test]# +[root@docker-test-server test]# +[root@docker-test-server test]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-dlhls -n oracle-database-operator-system +. +. +2022-03-08T22:12:57.414Z INFO controller-runtime.manager.controller.dbcssystem OCI provider configured succesfully {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:12:58.013Z INFO controller-runtime.manager.controller.dbcssystem DbcsSystem DBSystem provisioning {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:13:05.772Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:14:06.499Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:15:07.256Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:16:07.877Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:17:08.237Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:18:08.852Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:19:09.184Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:20:10.253Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:21:10.576Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:22:10.948Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:23:11.443Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:24:11.872Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:25:12.206Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:26:12.543Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:27:13.053Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:28:13.582Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:29:13.927Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:30:14.415Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:31:14.712Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:32:15.236Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:33:16.113Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:34:16.397Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:35:16.723Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:36:17.178Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:37:17.454Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:38:17.849Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:39:18.305Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:40:18.724Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:41:19.050Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:42:19.570Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:43:19.836Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:44:20.258Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:45:20.592Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:46:20.917Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:47:21.225Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:48:21.587Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:49:21.841Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:50:22.214Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:51:22.519Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:52:22.875Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:53:23.324Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:54:23.755Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:55:24.273Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:56:24.672Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:57:25.518Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:58:26.113Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T22:59:26.373Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T23:00:26.650Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T23:01:26.895Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T23:02:27.437Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T23:03:27.835Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T23:04:28.231Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T23:05:28.653Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T23:06:29.286Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T23:07:29.637Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T23:08:30.061Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T23:09:30.327Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T23:10:30.838Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T23:11:31.312Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T23:12:31.755Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T23:13:32.075Z INFO controller-runtime.manager.controller.dbcssystem DbcsSystem system provisioned succesfully {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-08T23:13:34.641Z INFO controller-runtime.manager.controller.dbcssystem OCI provider configured succesfully {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} +2022-03-08T23:13:45.117Z INFO controller-runtime.manager.controller.dbcssystem OCI provider configured succesfully {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} + + +[root@docker-test-server test]# kubectl describe dbcssystems.database.oracle.com dbcssystem-create +Name: dbcssystem-create +Namespace: default +Labels: +Annotations: lastSuccessfulSpec: + {"dbSystem":{"compartmentId":"ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya","availabilityDomain":"O... +API Version: database.oracle.com/v1alpha1 +Kind: DbcsSystem +Metadata: + Creation Timestamp: 2022-03-08T22:12:57Z + Generation: 1 + Managed Fields: + API Version: database.oracle.com/v1alpha1 + Fields Type: FieldsV1 + fieldsV1: + f:metadata: + f:annotations: + .: + f:kubectl.kubernetes.io/last-applied-configuration: + f:spec: + .: + f:dbSystem: + .: + f:availabilityDomain: + f:compartmentId: + f:dbAdminPaswordSecret: + f:dbName: + f:dbVersion: + f:dbWorkload: + f:hostName: + f:shape: + f:sshPublicKeys: + f:subnetId: + f:ociConfigMap: + f:ociSecret: + Manager: kubectl-client-side-apply + Operation: Update + Time: 2022-03-08T22:12:57Z + API Version: database.oracle.com/v1alpha1 + Fields Type: FieldsV1 + fieldsV1: + f:metadata: + f:annotations: + f:lastSuccessfulSpec: + f:spec: + f:dbSystem: + f:dbBackupConfig: + f:id: + f:status: + .: + f:availabilityDomain: + f:cpuCoreCount: + f:dataStoragePercentage: + f:dataStorageSizeInGBs: + f:dbEdition: + f:dbInfo: + f:displayName: + f:id: + f:licenseModel: + f:network: + .: + f:clientSubnet: + f:domainName: + f:hostName: + f:listenerPort: + f:scanDnsName: + f:vcnName: + f:nodeCount: + f:recoStorageSizeInGB: + f:shape: + f:state: + f:storageManagement: + f:subnetId: + f:timeZone: + f:workRequests: + Manager: manager + Operation: Update + Time: 2022-03-08T23:13:34Z + Resource Version: 55187354 + UID: 0cdbebc0-3aeb-43b1-ae7f-eb36e6c56000 +Spec: + Db System: + Availability Domain: OLou:PHX-AD-1 + Compartment Id: ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya + Db Admin Pasword Secret: admin-password + Db Name: db0130 + Db Version: 21c +apiVersion: database.oracle.com/v1alpha1 + Db Workload: OLTP + Host Name: host0130 + Shape: VM.Standard2.1 + Ssh Public Keys: + oci-publickey + Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a + Oci Config Map: oci-cred + Oci Secret: oci-privatekey +Status: + Availability Domain: OLou:PHX-AD-1 + Cpu Core Count: 1 + Data Storage Percentage: 80 + Data Storage Size In G Bs: 256 + Db Edition: ENTERPRISE_EDITION + Db Info: + Db Home Id: ocid1.dbhome.oc1.phx.anyhqljr5gy3jhqat52milqwt3gq6lwohhacwg5yi4mtzq7c7hag53lrkugq + Db Name: db0130 + Db Unique Name: db0130_phx1zn + Db Workload: OLTP + Id: ocid1.database.oc1.phx.anyhqljrabf7htyackgmsaqjfexoqgrzuuk33ju2q25z2al43tnd5mhhvkra + Display Name: dbsystem20220308221302 + Id: ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa + License Model: LICENSE_INCLUDED + Network: + Client Subnet: k8test-pubvcn + Domain Name: k8testpubvcn.k8test.oraclevcn.com + Host Name: host0130 + Listener Port: 1521 + Scan Dns Name: host0130-scan.k8testpubvcn.k8test.oraclevcn.com + Vcn Name: k8test + Node Count: 1 + Reco Storage Size In GB: 256 + Shape: VM.Standard2.1 + State: AVAILABLE + Storage Management: ASM + Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a + Time Zone: UTC + Work Requests: + Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrxivzvgzel47zuoyke5yk36o7mrgjl27vscd5z3bqptmyh3rxwbqq + Operation Type: Create DB System + Percent Complete: 100 + Time Accepted: 2022-03-08 22:13:02.999 +0000 UTC + Time Finished: 2022-03-08 23:11:50.46 +0000 UTC + Time Started: 2022-03-08 22:13:16.995 +0000 UTC +Events: +[root@docker-test-server test]# diff --git a/docs/dbcs/provisioning/known_issues.md b/docs/dbcs/provisioning/known_issues.md new file mode 100644 index 00000000..2bba6bf7 --- /dev/null +++ b/docs/dbcs/provisioning/known_issues.md @@ -0,0 +1,5 @@ +# Known Issues - Oracle DB Operator DBCS Controller + +Below are the known issues using the Oracle DB Operator DBCS Controller: + +1. There is a known issue related to the DB Version 19c, 12c and 11g when used with the Oracle DB Operator DBCS Controller. DB Version 21c and 18c work with the controller. diff --git a/docs/dbcs/provisioning/scale_down_dbcs_system_shape.md b/docs/dbcs/provisioning/scale_down_dbcs_system_shape.md new file mode 100644 index 00000000..abe98eea --- /dev/null +++ b/docs/dbcs/provisioning/scale_down_dbcs_system_shape.md @@ -0,0 +1,45 @@ +# Scale Down the shape of an existing DBCS System + +In this use case, an existing OCI DBCS system deployed earlier is scaled down for its shape using Oracle DB Operator DBCS controller. Its a 2 Step operation. + +In order to scale down an existing DBCS system, the steps will be: + +1. Bind the existing DBCS System to DBCS Controller. +2. Apply the change to scale down its shape. + +**NOTE** We are assuming that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-deploy-a-dbcs-system-using-oracle-db-operator-dbcs-controller) steps to create the configmap and the secrets required during the deployment. + +This example uses `scale_down_dbcs_system_shape.yaml` to scale down a Single Instance DBCS VMDB using Oracle DB Operator DBCS Controller with: + +- OCID of existing VMDB as `ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa` +- OCI Configmap as `oci-cred` +- OCI Secret as `oci-privatekey` +- Availability Domain for the DBCS VMDB as `OLou:PHX-AD-1` +- Compartment OCID as `ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya` +- Database Admin Credential as `admin-password` +- Database Hostname Prefix as `host0130` +- Oracle VMDB target Shape as `VM.Standard2.1` +- SSH Public key for the DBCS system being deployed as `oci-publickey` +- OCID of the Subnet as `ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a` + +**NOTE:** For the details of the parameters to be used in the .yaml file, please refer [here](./dbcs_controller_parameters.md). + +Use the file: [scale_down_dbcs_system_shape.yaml](./scale_down_dbcs_system_shape.yaml) for this use case as below: + +1. Deploy the .yaml file: +```sh +[root@docker-test-server DBCS]# kubectl apply -f scale_down_dbcs_system_shape.yaml +dbcssystem.database.oracle.com/dbcssystem-existing configured +``` + +2. Monitor the Oracle DB Operator Pod `pod/oracle-database-operator-controller-manager-665874bd57-g2cgw` for the progress of the DBCS VMDB Scale down. + +NOTE: Check the DB Operator Pod name in your environment. + +``` +[root@docker-test-server DBCS]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-g2cgw -n oracle-database-operator-system +``` + +## Sample Output + +[Here](./scale_down_dbcs_system_shape_sample_output.log) is the sample output for scaling down the shape of an existing DBCS System deployed in OCI using Oracle DB Operator DBCS Controller. diff --git a/docs/dbcs/provisioning/scale_down_dbcs_system_shape.yaml b/docs/dbcs/provisioning/scale_down_dbcs_system_shape.yaml new file mode 100644 index 00000000..5e2cfb3f --- /dev/null +++ b/docs/dbcs/provisioning/scale_down_dbcs_system_shape.yaml @@ -0,0 +1,17 @@ +apiVersion: database.oracle.com/v1alpha1 +kind: DbcsSystem +metadata: + name: dbcssystem-existing +spec: + id: "ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa" + ociConfigMap: "oci-cred" + ociSecret: "oci-privatekey" + dbSystem: + availabilityDomain: "OLou:PHX-AD-1" + compartmentId: "ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya" + dbAdminPaswordSecret: "admin-password" + hostName: "host0130" + shape: "VM.Standard2.1" + sshPublicKeys: + - "oci-publickey" + subnetId: "ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a" diff --git a/docs/dbcs/provisioning/scale_down_dbcs_system_shape_sample_output.log b/docs/dbcs/provisioning/scale_down_dbcs_system_shape_sample_output.log new file mode 100644 index 00000000..acc45208 --- /dev/null +++ b/docs/dbcs/provisioning/scale_down_dbcs_system_shape_sample_output.log @@ -0,0 +1,378 @@ +[root@docker-test-server test]# kubectl describe dbcssystems.database.oracle.com dbcssystem-existing +Name: dbcssystem-existing +Namespace: default +Labels: +Annotations: lastSuccessfulSpec: + {"dbSystem":{"compartmentId":"ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya","availabilityDomain":"O... +API Version: database.oracle.com/v1alpha1 +Kind: DbcsSystem +Metadata: + Creation Timestamp: 2022-03-08T23:27:48Z + Generation: 2 + Managed Fields: + API Version: database.oracle.com/v1alpha1 + Fields Type: FieldsV1 + fieldsV1: + f:metadata: + f:annotations: + .: + f:kubectl.kubernetes.io/last-applied-configuration: + f:spec: + .: + f:dbSystem: + .: + f:availabilityDomain: + f:compartmentId: + f:dbAdminPaswordSecret: + f:hostName: + f:shape: + f:sshPublicKeys: + f:subnetId: + f:id: + f:ociConfigMap: + f:ociSecret: + Manager: kubectl-client-side-apply + Operation: Update + Time: 2022-03-08T23:32:50Z + API Version: database.oracle.com/v1alpha1 + Fields Type: FieldsV1 + fieldsV1: + f:metadata: + f:annotations: + f:lastSuccessfulSpec: + f:spec: + f:dbSystem: + f:cpuCoreCount: + f:dbBackupConfig: + f:dbEdition: + f:dbName: + f:dbUniqueName: + f:dbVersion: + f:diskRedundancy: + f:displayName: + f:faultDomains: + f:nodeCount: + f:status: + .: + f:availabilityDomain: + f:cpuCoreCount: + f:dataStoragePercentage: + f:dataStorageSizeInGBs: + f:dbEdition: + f:dbInfo: + f:displayName: + f:id: + f:licenseModel: + f:network: + .: + f:clientSubnet: + f:domainName: + f:hostName: + f:listenerPort: + f:scanDnsName: + f:vcnName: + f:nodeCount: + f:recoStorageSizeInGB: + f:shape: + f:state: + f:storageManagement: + f:subnetId: + f:timeZone: + f:workRequests: + Manager: manager + Operation: Update + Time: 2022-03-08T23:32:55Z + Resource Version: 55197836 + UID: 96d7bc49-33e9-42cc-8dd0-ada9a5a4c7e5 +Spec: + Db System: + Availability Domain: OLou:PHX-AD-1 + Compartment Id: ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya + Db Admin Pasword Secret: admin-password + Host Name: host0130 + Shape: VM.Standard2.2 + Ssh Public Keys: + oci-publickey + Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a + Id: ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa + Oci Config Map: oci-cred + Oci Secret: oci-privatekey +Status: + Availability Domain: OLou:PHX-AD-1 + Cpu Core Count: 2 + Data Storage Percentage: 80 + Data Storage Size In G Bs: 256 + Db Edition: ENTERPRISE_EDITION + Db Info: + Db Home Id: ocid1.dbhome.oc1.phx.anyhqljr5gy3jhqat52milqwt3gq6lwohhacwg5yi4mtzq7c7hag53lrkugq + Db Name: db0130 + Db Unique Name: db0130_phx1zn + Db Workload: OLTP + Id: ocid1.database.oc1.phx.anyhqljrabf7htyackgmsaqjfexoqgrzuuk33ju2q25z2al43tnd5mhhvkra + Display Name: dbsystem20220308221302 + Id: ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa + License Model: LICENSE_INCLUDED + Network: + Client Subnet: k8test-pubvcn + Domain Name: k8testpubvcn.k8test.oraclevcn.com + Host Name: host0130 + Listener Port: 1521 + Scan Dns Name: host0130-scan.k8testpubvcn.k8test.oraclevcn.com + Vcn Name: k8test + Node Count: 1 + Reco Storage Size In GB: 256 + Shape: VM.Standard2.2 + State: AVAILABLE + Storage Management: ASM + Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a + Time Zone: UTC + Work Requests: + Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrxivzvgzel47zuoyke5yk36o7mrgjl27vscd5z3bqptmyh3rxwbqq + Operation Type: Create DB System + Percent Complete: 100 + Time Accepted: 2022-03-08 22:13:02.999 +0000 UTC + Time Finished: 2022-03-08 23:11:50.46 +0000 UTC + Time Started: 2022-03-08 22:13:16.995 +0000 UTC + Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrc3fx6kwq4yxerk3ngztdbbngm7w4dnlddcdhxqxjn6e4kcyux5ca + Operation Type: Update Shape + Percent Complete: 100 + Time Accepted: 2022-03-08 23:33:42.807 +0000 UTC + Time Finished: 2022-03-08 23:46:21.126 +0000 UTC + Time Started: 2022-03-08 23:33:52.109 +0000 UTC +Events: +[root@docker-test-server test]# + + + + +[root@docker-test-server test]# cat scale_down_dbcs_system_shape.yaml +apiVersion: database.oracle.com/v1alpha1 +kind: DbcsSystem +metadata: + name: dbcssystem-existing +spec: + id: "ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa" + ociConfigMap: "oci-cred" + ociSecret: "oci-privatekey" + dbSystem: + availabilityDomain: "OLou:PHX-AD-1" + compartmentId: "ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya" + dbAdminPaswordSecret: "admin-password" + hostName: "host0130" + shape: "VM.Standard2.1" + sshPublicKeys: + - "oci-publickey" + subnetId: "ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a" +[root@docker-test-server test]# +[root@docker-test-server test]# +[root@docker-test-server test]# kubectl apply -f scale_down_dbcs_system_shape.yaml +dbcssystem.database.oracle.com/dbcssystem-existing configured + + + + +[root@docker-test-server test]# kubectl get ns + +kubectl get allNAME STATUS AGE +cert-manager Active 13d +default Active 139d +kube-node-lease Active 139d +kube-public Active 139d +kube-system Active 139d +oracle-database-operator-system Active 13d +shns Active 88d +[root@docker-test-server test]# +[root@docker-test-server test]# kubectl get all -n oracle-database-operator-system +NAME READY STATUS RESTARTS AGE +pod/oracle-database-operator-controller-manager-665874bd57-dlhls 1/1 Running 3 13d +pod/oracle-database-operator-controller-manager-665874bd57-g2cgw 1/1 Running 3 13d +pod/oracle-database-operator-controller-manager-665874bd57-q42f8 1/1 Running 4 13d + +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +service/oracle-database-operator-controller-manager-metrics-service ClusterIP 10.96.130.124 8443/TCP 13d +service/oracle-database-operator-webhook-service ClusterIP 10.96.4.104 443/TCP 13d + +NAME READY UP-TO-DATE AVAILABLE AGE +deployment.apps/oracle-database-operator-controller-manager 3/3 3 3 13d + +NAME DESIRED CURRENT READY AGE +replicaset.apps/oracle-database-operator-controller-manager-665874bd57 3 3 3 13d +[root@docker-test-server test]# + + +[root@docker-test-server test]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-dlhls -n oracle-database-operator-system +. +. +2022-03-09T00:24:08.850Z INFO controller-runtime.manager.controller.dbcssystem OCI provider configured succesfully {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-09T00:24:12.990Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-09T00:25:13.409Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-09T00:26:13.878Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-09T00:27:14.206Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-09T00:28:14.465Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-09T00:29:14.735Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-09T00:30:15.027Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-09T00:31:15.331Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-09T00:32:15.768Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-09T00:33:16.188Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-09T00:34:16.476Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-09T00:35:17.125Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-09T00:36:17.598Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-09T00:37:18.000Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-09T00:38:18.344Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} + + + + + + + + + +[root@docker-test-server test]# kubectl describe dbcssystems.database.oracle.com dbcssystem-existing +Name: dbcssystem-existing +Namespace: default +Labels: +Annotations: lastSuccessfulSpec: + {"dbSystem":{"compartmentId":"ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya","availabilityDomain":"O... +API Version: database.oracle.com/v1alpha1 +Kind: DbcsSystem +Metadata: + Creation Timestamp: 2022-03-08T23:27:48Z + Generation: 3 + Managed Fields: + API Version: database.oracle.com/v1alpha1 + Fields Type: FieldsV1 + fieldsV1: + f:metadata: + f:annotations: + .: + f:kubectl.kubernetes.io/last-applied-configuration: + f:spec: + .: + f:dbSystem: + .: + f:availabilityDomain: + f:compartmentId: + f:dbAdminPaswordSecret: + f:hostName: + f:shape: + f:sshPublicKeys: + f:subnetId: + f:id: + f:ociConfigMap: + f:ociSecret: + Manager: kubectl-client-side-apply + Operation: Update + Time: 2022-03-08T23:32:50Z + API Version: database.oracle.com/v1alpha1 + Fields Type: FieldsV1 + fieldsV1: + f:metadata: + f:annotations: + f:lastSuccessfulSpec: + f:spec: + f:dbSystem: + f:cpuCoreCount: + f:dbBackupConfig: + f:dbEdition: + f:dbName: + f:dbUniqueName: + f:dbVersion: + f:diskRedundancy: + f:displayName: + f:faultDomains: + f:nodeCount: + f:status: + .: + f:availabilityDomain: + f:cpuCoreCount: + f:dataStoragePercentage: + f:dataStorageSizeInGBs: + f:dbEdition: + f:dbInfo: + f:displayName: + f:id: + f:licenseModel: + f:network: + .: + f:clientSubnet: + f:domainName: + f:hostName: + f:listenerPort: + f:scanDnsName: + f:vcnName: + f:nodeCount: + f:recoStorageSizeInGB: + f:shape: + f:state: + f:storageManagement: + f:subnetId: + f:timeZone: + f:workRequests: + Manager: manager + Operation: Update + Time: 2022-03-08T23:32:55Z + Resource Version: 55214174 + UID: 96d7bc49-33e9-42cc-8dd0-ada9a5a4c7e5 +Spec: + Db System: + Availability Domain: OLou:PHX-AD-1 + Compartment Id: ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya + Db Admin Pasword Secret: admin-password + Host Name: host0130 + Shape: VM.Standard2.1 + Ssh Public Keys: + oci-publickey + Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a + Id: ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa + Oci Config Map: oci-cred + Oci Secret: oci-privatekey +Status: + Availability Domain: OLou:PHX-AD-1 + Cpu Core Count: 1 + Data Storage Percentage: 80 + Data Storage Size In G Bs: 256 + Db Edition: ENTERPRISE_EDITION + Db Info: + Db Home Id: ocid1.dbhome.oc1.phx.anyhqljr5gy3jhqat52milqwt3gq6lwohhacwg5yi4mtzq7c7hag53lrkugq + Db Name: db0130 + Db Unique Name: db0130_phx1zn + Db Workload: OLTP + Id: ocid1.database.oc1.phx.anyhqljrabf7htyackgmsaqjfexoqgrzuuk33ju2q25z2al43tnd5mhhvkra + Display Name: dbsystem20220308221302 + Id: ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa + License Model: LICENSE_INCLUDED + Network: + Client Subnet: k8test-pubvcn + Domain Name: k8testpubvcn.k8test.oraclevcn.com + Host Name: host0130 + Listener Port: 1521 + Scan Dns Name: host0130-scan.k8testpubvcn.k8test.oraclevcn.com + Vcn Name: k8test + Node Count: 1 + Reco Storage Size In GB: 256 + Shape: VM.Standard2.1 + State: AVAILABLE + Storage Management: ASM + Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a + Time Zone: UTC + Work Requests: + Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrxivzvgzel47zuoyke5yk36o7mrgjl27vscd5z3bqptmyh3rxwbqq + Operation Type: Create DB System + Percent Complete: 100 + Time Accepted: 2022-03-08 22:13:02.999 +0000 UTC + Time Finished: 2022-03-08 23:11:50.46 +0000 UTC + Time Started: 2022-03-08 22:13:16.995 +0000 UTC + Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrc3fx6kwq4yxerk3ngztdbbngm7w4dnlddcdhxqxjn6e4kcyux5ca + Operation Type: Update Shape + Percent Complete: 100 + Time Accepted: 2022-03-08 23:33:42.807 +0000 UTC + Time Finished: 2022-03-08 23:46:21.126 +0000 UTC + Time Started: 2022-03-08 23:33:52.109 +0000 UTC + Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljr5sveun3f6k3zuz23py7mm7jncmpq5vwyajbo5ezhc765347defwq + Operation Type: Update Shape + Percent Complete: 100 + Time Accepted: 2022-03-09 00:25:03.644 +0000 UTC + Time Finished: 2022-03-09 00:38:59.526 +0000 UTC + Time Started: 2022-03-09 00:25:15.578 +0000 UTC +Events: \ No newline at end of file diff --git a/docs/dbcs/provisioning/scale_up_dbcs_system_shape.md b/docs/dbcs/provisioning/scale_up_dbcs_system_shape.md new file mode 100644 index 00000000..8efccb5f --- /dev/null +++ b/docs/dbcs/provisioning/scale_up_dbcs_system_shape.md @@ -0,0 +1,45 @@ +# Scale UP the shape of an existing DBCS System + +In this use case, an existing OCI DBCS system deployed earlier is scaled up for its shape using Oracle DB Operator DBCS controller. Its a 2 Step operation. + +In order to scale up an existing DBCS system, the steps will be: + +1. Bind the existing DBCS System to DBCS Controller. +2. Apply the change to scale up its shape. + +**NOTE:** We are assuming that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-deploy-a-dbcs-system-using-oracle-db-operator-dbcs-controller) steps to create the configmap and the secrets required during the deployment. + +This example uses `scale_up_dbcs_system_shape.yaml` to scale up a Single Instance DBCS VMDB using Oracle DB Operator DBCS Controller with: + +- OCID of existing VMDB as `ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa` +- OCI Configmap as `oci-cred` +- OCI Secret as `oci-privatekey` +- Availability Domain for the DBCS VMDB as `OLou:PHX-AD-1` +- Compartment OCID as `ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya` +- Database Admin Credential as `admin-password` +- Database Hostname Prefix as `host0130` +- Oracle VMDB Shape as `VM.Standard2.2` +- SSH Public key for the DBCS system being deployed as `oci-publickey` +- OCID of the Subnet as `ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a` + +**NOTE:** For the details of the parameters to be used in the .yaml file, please refer [here](./dbcs_controller_parameters.md). + +Use the file: [scale_up_dbcs_system_shape.yaml](./scale_up_dbcs_system_shape.yaml) for this use case as below: + +1. Deploy the .yaml file: +```sh +[root@docker-test-server DBCS]# kubectl apply -f scale_up_dbcs_system_shape.yaml +dbcssystem.database.oracle.com/dbcssystem-existing configured +``` + +2. Monitor the Oracle DB Operator Pod `pod/oracle-database-operator-controller-manager-665874bd57-g2cgw` for the progress of the DBCS VMDB Scale up. + +NOTE: Check the DB Operator Pod name in your environment. + +``` +[root@docker-test-server DBCS]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-g2cgw -n oracle-database-operator-system +``` + +## Sample Output + +[Here](./scale_up_dbcs_system_shape_sample_output.log) is the sample output for scaling up the shape of an existing DBCS System deployed in OCI using Oracle DB Operator DBCS Controller. diff --git a/docs/dbcs/provisioning/scale_up_dbcs_system_shape.yaml b/docs/dbcs/provisioning/scale_up_dbcs_system_shape.yaml new file mode 100644 index 00000000..d1c2b95d --- /dev/null +++ b/docs/dbcs/provisioning/scale_up_dbcs_system_shape.yaml @@ -0,0 +1,17 @@ +apiVersion: database.oracle.com/v1alpha1 +kind: DbcsSystem +metadata: + name: dbcssystem-existing +spec: + id: "ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa" + ociConfigMap: "oci-cred" + ociSecret: "oci-privatekey" + dbSystem: + availabilityDomain: "OLou:PHX-AD-1" + compartmentId: "ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya" + dbAdminPaswordSecret: "admin-password" + hostName: "host0130" + shape: "VM.Standard2.2" + sshPublicKeys: + - "oci-publickey" + subnetId: "ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a" diff --git a/docs/dbcs/provisioning/scale_up_dbcs_system_shape_sample_output.log b/docs/dbcs/provisioning/scale_up_dbcs_system_shape_sample_output.log new file mode 100644 index 00000000..96b52924 --- /dev/null +++ b/docs/dbcs/provisioning/scale_up_dbcs_system_shape_sample_output.log @@ -0,0 +1,351 @@ +[root@docker-test-server test]# kubectl describe dbcssystems.database.oracle.com dbcssystem-existing +Name: dbcssystem-existing +Namespace: default +Labels: +Annotations: lastSuccessfulSpec: + {"dbSystem":{"compartmentId":"ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya","availabilityDomain":"O... +API Version: database.oracle.com/v1alpha1 +Kind: DbcsSystem +Metadata: + Creation Timestamp: 2022-03-08T23:27:48Z + Generation: 1 + Managed Fields: + API Version: database.oracle.com/v1alpha1 + Fields Type: FieldsV1 + fieldsV1: + f:metadata: + f:annotations: + .: + f:kubectl.kubernetes.io/last-applied-configuration: + f:spec: + .: + f:id: + f:ociConfigMap: + f:ociSecret: + Manager: kubectl-client-side-apply + Operation: Update + Time: 2022-03-08T23:27:48Z + API Version: database.oracle.com/v1alpha1 + Fields Type: FieldsV1 + fieldsV1: + f:metadata: + f:annotations: + f:lastSuccessfulSpec: + f:spec: + f:dbSystem: + .: + f:availabilityDomain: + f:compartmentId: + f:cpuCoreCount: + f:dbAdminPaswordSecret: + f:dbBackupConfig: + f:dbEdition: + f:dbName: + f:dbUniqueName: + f:dbVersion: + f:diskRedundancy: + f:displayName: + f:faultDomains: + f:hostName: + f:nodeCount: + f:shape: + f:sshPublicKeys: + f:subnetId: + f:status: + .: + f:availabilityDomain: + f:cpuCoreCount: + f:dataStoragePercentage: + f:dataStorageSizeInGBs: + f:dbEdition: + f:dbInfo: + f:displayName: + f:id: + f:licenseModel: + f:network: + .: + f:clientSubnet: + f:domainName: + f:hostName: + f:listenerPort: + f:scanDnsName: + f:vcnName: + f:nodeCount: + f:recoStorageSizeInGB: + f:shape: + f:state: + f:storageManagement: + f:subnetId: + f:timeZone: + f:workRequests: + Manager: manager +apiVersion: database.oracle.com/v1alpha1 + Operation: Update + Time: 2022-03-08T23:27:52Z + Resource Version: 55191827 + UID: 96d7bc49-33e9-42cc-8dd0-ada9a5a4c7e5 +Spec: + Id: ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa + Oci Config Map: oci-cred + Oci Secret: oci-privatekey +Status: + Availability Domain: OLou:PHX-AD-1 + Cpu Core Count: 1 + Data Storage Percentage: 80 + Data Storage Size In G Bs: 256 + Db Edition: ENTERPRISE_EDITION + Db Info: + Db Home Id: ocid1.dbhome.oc1.phx.anyhqljr5gy3jhqat52milqwt3gq6lwohhacwg5yi4mtzq7c7hag53lrkugq + Db Name: db0130 + Db Unique Name: db0130_phx1zn + Db Workload: OLTP + Id: ocid1.database.oc1.phx.anyhqljrabf7htyackgmsaqjfexoqgrzuuk33ju2q25z2al43tnd5mhhvkra + Display Name: dbsystem20220308221302 + Id: ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa + License Model: LICENSE_INCLUDED + Network: + Client Subnet: k8test-pubvcn + Domain Name: k8testpubvcn.k8test.oraclevcn.com + Host Name: host0130 + Listener Port: 1521 + Scan Dns Name: host0130-scan.k8testpubvcn.k8test.oraclevcn.com + Vcn Name: k8test + Node Count: 1 + Reco Storage Size In GB: 256 + Shape: VM.Standard2.1 + State: AVAILABLE + Storage Management: ASM + Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a + Time Zone: UTC + Work Requests: + Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrxivzvgzel47zuoyke5yk36o7mrgjl27vscd5z3bqptmyh3rxwbqq + Operation Type: Create DB System + Percent Complete: 100 + Time Accepted: 2022-03-08 22:13:02.999 +0000 UTC + Time Finished: 2022-03-08 23:11:50.46 +0000 UTC + Time Started: 2022-03-08 22:13:16.995 +0000 UTC +Events: +[root@docker-test-server test]# + + + +[root@docker-test-server test]# cat scale_up_dbcs_system_shape.yaml +apiVersion: database.oracle.com/v1alpha1 +kind: DbcsSystem +metadata: + name: dbcssystem-existing +spec: + id: "ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa" + ociConfigMap: "oci-cred" + ociSecret: "oci-privatekey" + dbSystem: + availabilityDomain: "OLou:PHX-AD-1" + compartmentId: "ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya" + dbAdminPaswordSecret: "admin-password" + hostName: "host0130" + shape: "VM.Standard2.2" + sshPublicKeys: + - "oci-publickey" + subnetId: "ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a" +[root@docker-test-server test]# + + +[root@docker-test-server test]# kubectl apply -f scale_up_dbcs_system_shape.yaml +dbcssystem.database.oracle.com/dbcssystem-existing configured +[root@docker-test-server test]# + + +[root@docker-test-server test]# kubectl get ns + +kubectl get allNAME STATUS AGE +cert-manager Active 13d +default Active 139d +kube-node-lease Active 139d +kube-public Active 139d +kube-system Active 139d +oracle-database-operator-system Active 13d +shns Active 88d +[root@docker-test-server test]# +[root@docker-test-server test]# kubectl get all -n oracle-database-operator-system +NAME READY STATUS RESTARTS AGE +pod/oracle-database-operator-controller-manager-665874bd57-dlhls 1/1 Running 3 13d +pod/oracle-database-operator-controller-manager-665874bd57-g2cgw 1/1 Running 3 13d +pod/oracle-database-operator-controller-manager-665874bd57-q42f8 1/1 Running 4 13d + +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +service/oracle-database-operator-controller-manager-metrics-service ClusterIP 10.96.130.124 8443/TCP 13d +service/oracle-database-operator-webhook-service ClusterIP 10.96.4.104 443/TCP 13d + +NAME READY UP-TO-DATE AVAILABLE AGE +deployment.apps/oracle-database-operator-controller-manager 3/3 3 3 13d + +NAME DESIRED CURRENT READY AGE +replicaset.apps/oracle-database-operator-controller-manager-665874bd57 3 3 3 13d +[root@docker-test-server test]# + +[root@docker-test-server test]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-dlhls -n oracle-database-operator-system +. +. +2022-03-08T23:32:12.728Z INFO controller-runtime.manager.controller.dbcssystem OCI provider configured succesfully {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-08T23:32:50.935Z INFO controller-runtime.manager.controller.dbcssystem OCI provider configured succesfully {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-08T23:32:55.703Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-08T23:33:55.990Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-08T23:34:56.830Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-08T23:35:57.120Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-08T23:36:57.675Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-08T23:37:58.011Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-08T23:38:58.566Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-08T23:39:58.929Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-08T23:40:59.368Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-08T23:41:59.837Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-08T23:43:00.298Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-08T23:44:00.581Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-08T23:45:00.942Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-08T23:46:01.332Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} + + + +[root@docker-test-server test]# kubectl describe dbcssystems.database.oracle.com dbcssystem-existing +Name: dbcssystem-existing +Namespace: default +Labels: +Annotations: lastSuccessfulSpec: + {"dbSystem":{"compartmentId":"ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya","availabilityDomain":"O... +API Version: database.oracle.com/v1alpha1 +Kind: DbcsSystem +Metadata: + Creation Timestamp: 2022-03-08T23:27:48Z + Generation: 2 + Managed Fields: + API Version: database.oracle.com/v1alpha1 + Fields Type: FieldsV1 + fieldsV1: + f:metadata: + f:annotations: + .: + f:kubectl.kubernetes.io/last-applied-configuration: + f:spec: + .: + f:dbSystem: + .: + f:availabilityDomain: + f:compartmentId: + f:dbAdminPaswordSecret: + f:hostName: + f:shape: + f:sshPublicKeys: + f:subnetId: + f:id: + f:ociConfigMap: + f:ociSecret: + Manager: kubectl-client-side-apply + Operation: Update + Time: 2022-03-08T23:32:50Z + API Version: database.oracle.com/v1alpha1 + Fields Type: FieldsV1 + fieldsV1: + f:metadata: + f:annotations: + f:lastSuccessfulSpec: + f:spec: + f:dbSystem: + f:cpuCoreCount: + f:dbBackupConfig: + f:dbEdition: + f:dbName: + f:dbUniqueName: + f:dbVersion: + f:diskRedundancy: + f:displayName: + f:faultDomains: + f:nodeCount: + f:status: + .: + f:availabilityDomain: + f:cpuCoreCount: + f:dataStoragePercentage: + f:dataStorageSizeInGBs: + f:dbEdition: + f:dbInfo: + f:displayName: + f:id: + f:licenseModel: + f:network: + .: + f:clientSubnet: + f:domainName: + f:hostName: + f:listenerPort: + f:scanDnsName: + f:vcnName: + f:nodeCount: + f:recoStorageSizeInGB: + f:shape: + f:state: + f:storageManagement: + f:subnetId: + f:timeZone: + f:workRequests: + Manager: manager + Operation: Update + Time: 2022-03-08T23:32:55Z + Resource Version: 55197836 + UID: 96d7bc49-33e9-42cc-8dd0-ada9a5a4c7e5 +Spec: + Db System: + Availability Domain: OLou:PHX-AD-1 + Compartment Id: ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya + Db Admin Pasword Secret: admin-password + Host Name: host0130 + Shape: VM.Standard2.2 + Ssh Public Keys: + oci-publickey + Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a + Id: ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa + Oci Config Map: oci-cred + Oci Secret: oci-privatekey +Status: + Availability Domain: OLou:PHX-AD-1 + Cpu Core Count: 2 + Data Storage Percentage: 80 + Data Storage Size In G Bs: 256 + Db Edition: ENTERPRISE_EDITION + Db Info: + Db Home Id: ocid1.dbhome.oc1.phx.anyhqljr5gy3jhqat52milqwt3gq6lwohhacwg5yi4mtzq7c7hag53lrkugq + Db Name: db0130 + Db Unique Name: db0130_phx1zn + Db Workload: OLTP + Id: ocid1.database.oc1.phx.anyhqljrabf7htyackgmsaqjfexoqgrzuuk33ju2q25z2al43tnd5mhhvkra + Display Name: dbsystem20220308221302 + Id: ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa + License Model: LICENSE_INCLUDED + Network: + Client Subnet: k8test-pubvcn + Domain Name: k8testpubvcn.k8test.oraclevcn.com + Host Name: host0130 + Listener Port: 1521 + Scan Dns Name: host0130-scan.k8testpubvcn.k8test.oraclevcn.com + Vcn Name: k8test + Node Count: 1 + Reco Storage Size In GB: 256 + Shape: VM.Standard2.2 + State: AVAILABLE + Storage Management: ASM + Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a + Time Zone: UTC + Work Requests: + Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrxivzvgzel47zuoyke5yk36o7mrgjl27vscd5z3bqptmyh3rxwbqq + Operation Type: Create DB System + Percent Complete: 100 + Time Accepted: 2022-03-08 22:13:02.999 +0000 UTC + Time Finished: 2022-03-08 23:11:50.46 +0000 UTC + Time Started: 2022-03-08 22:13:16.995 +0000 UTC + Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrc3fx6kwq4yxerk3ngztdbbngm7w4dnlddcdhxqxjn6e4kcyux5ca + Operation Type: Update Shape + Percent Complete: 100 + Time Accepted: 2022-03-08 23:33:42.807 +0000 UTC + Time Finished: 2022-03-08 23:46:21.126 +0000 UTC + Time Started: 2022-03-08 23:33:52.109 +0000 UTC +Events: +[root@docker-test-server test]# + diff --git a/docs/dbcs/provisioning/scale_up_storage.md b/docs/dbcs/provisioning/scale_up_storage.md new file mode 100644 index 00000000..64514025 --- /dev/null +++ b/docs/dbcs/provisioning/scale_up_storage.md @@ -0,0 +1,45 @@ +# Scale UP the storage of an existing DBCS System + +In this use case, an existing OCI DBCS system deployed earlier is scaled up for its storage using Oracle DB Operator DBCS controller. Its a 2 Step operation. + +In order to scale up storage of an existing DBCS system, the steps will be: + +1. Bind the existing DBCS System to DBCS Controller. +2. Apply the change to scale up its storage. + +**NOTE** We are assuming that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-deploy-a-dbcs-system-using-oracle-db-operator-dbcs-controller) steps to create the configmap and the secrets required during the deployment. + +This example uses `scale_up_storage.yaml` to scale up storage of an existing Single Instance DBCS VMDB using Oracle DB Operator DBCS Controller with: + +- OCID of existing VMDB as `ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa` +- OCI Configmap as `oci-cred` +- OCI Secret as `oci-privatekey` +- Availability Domain for the DBCS VMDB as `OLou:PHX-AD-1` +- Compartment OCID as `ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya` +- Database Admin Credential as `admin-password` +- Database Hostname Prefix as `host0130` +- Target Data Storage Size in GBs as `512` +- Oracle VMDB Shape as `VM.Standard2.1` +- SSH Public key for the DBCS system being deployed as `oci-publickey` +- OCID of the Subnet as `ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a` + + +Use the file: [scale_up_storage.yaml](./scale_up_storage.yaml) for this use case as below: + +1. Deploy the .yaml file: +```sh +[root@test-server DBCS]# kubectl apply -f scale_storage.yaml +dbcssystem.database.oracle.com/dbcssystem-existing configured +``` + +2. Monitor the Oracle DB Operator Pod `pod/oracle-database-operator-controller-manager-665874bd57-g2cgw` for the progress of the DBCS VMDB Scale up. + +NOTE: Check the DB Operator Pod name in your environment. + +``` +[root@docker-test-server DBCS]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-g2cgw -n oracle-database-operator-system +``` + +## Sample Output + +[Here](./scale_up_storage_sample_output.log) is the sample output for scaling up the storage of an existing DBCS System deployed in OCI using Oracle DB Operator DBCS Controller with minimal parameters. diff --git a/docs/dbcs/provisioning/scale_up_storage.yaml b/docs/dbcs/provisioning/scale_up_storage.yaml new file mode 100644 index 00000000..34e64b5e --- /dev/null +++ b/docs/dbcs/provisioning/scale_up_storage.yaml @@ -0,0 +1,18 @@ +apiVersion: database.oracle.com/v1alpha1 +kind: DbcsSystem +metadata: + name: dbcssystem-existing +spec: + id: "ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa" + ociConfigMap: "oci-cred" + ociSecret: "oci-privatekey" + dbSystem: + availabilityDomain: "OLou:PHX-AD-1" + compartmentId: "ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya" + dbAdminPaswordSecret: "admin-password" + hostName: "host0130" + initialDataStorageSizeInGB: 512 + shape: "VM.Standard2.1" + sshPublicKeys: + - "oci-publickey" + subnetId: "ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a" diff --git a/docs/dbcs/provisioning/scale_up_storage_sample_output.log b/docs/dbcs/provisioning/scale_up_storage_sample_output.log new file mode 100644 index 00000000..667667b8 --- /dev/null +++ b/docs/dbcs/provisioning/scale_up_storage_sample_output.log @@ -0,0 +1,389 @@ +[root@docker-test-server test]# kubectl describe dbcssystems.database.oracle.com dbcssystem-existing +Name: dbcssystem-existing +Namespace: default +Labels: +Annotations: lastSuccessfulSpec: + {"dbSystem":{"compartmentId":"ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya","availabilityDomain":"O... +API Version: database.oracle.com/v1alpha1 +Kind: DbcsSystem +Metadata: + Creation Timestamp: 2022-03-08T23:27:48Z + Generation: 3 + Managed Fields: + API Version: database.oracle.com/v1alpha1 + Fields Type: FieldsV1 + fieldsV1: + f:metadata: + f:annotations: + .: + f:kubectl.kubernetes.io/last-applied-configuration: + f:spec: + .: + f:dbSystem: + .: + f:availabilityDomain: + f:compartmentId: + f:dbAdminPaswordSecret: + f:hostName: + f:shape: + f:sshPublicKeys: + f:subnetId: + f:id: + f:ociConfigMap: + f:ociSecret: + Manager: kubectl-client-side-apply + Operation: Update + Time: 2022-03-08T23:32:50Z + API Version: database.oracle.com/v1alpha1 + Fields Type: FieldsV1 + fieldsV1: + f:metadata: + f:annotations: + f:lastSuccessfulSpec: + f:spec: + f:dbSystem: + f:cpuCoreCount: + f:dbBackupConfig: + f:dbEdition: + f:dbName: + f:dbUniqueName: + f:dbVersion: + f:diskRedundancy: + f:displayName: + f:faultDomains: + f:nodeCount: + f:status: + .: + f:availabilityDomain: + f:cpuCoreCount: + f:dataStoragePercentage: + f:dataStorageSizeInGBs: + f:dbEdition: + f:dbInfo: + f:displayName: + f:id: + f:licenseModel: + f:network: + .: + f:clientSubnet: + f:domainName: + f:hostName: + f:listenerPort: + f:scanDnsName: + f:vcnName: + f:nodeCount: + f:recoStorageSizeInGB: + f:shape: + f:state: + f:storageManagement: + f:subnetId: + f:timeZone: + f:workRequests: + Manager: manager + Operation: Update + Time: 2022-03-08T23:32:55Z + Resource Version: 55214174 + UID: 96d7bc49-33e9-42cc-8dd0-ada9a5a4c7e5 +Spec: + Db System: + Availability Domain: OLou:PHX-AD-1 + Compartment Id: ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya + Db Admin Pasword Secret: admin-password + Host Name: host0130 + Shape: VM.Standard2.1 + Ssh Public Keys: + oci-publickey + Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a + Id: ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa + Oci Config Map: oci-cred + Oci Secret: oci-privatekey +Status: + Availability Domain: OLou:PHX-AD-1 + Cpu Core Count: 1 + Data Storage Percentage: 80 + Data Storage Size In G Bs: 256 + Db Edition: ENTERPRISE_EDITION + Db Info: + Db Home Id: ocid1.dbhome.oc1.phx.anyhqljr5gy3jhqat52milqwt3gq6lwohhacwg5yi4mtzq7c7hag53lrkugq + Db Name: db0130 + Db Unique Name: db0130_phx1zn + Db Workload: OLTP + Id: ocid1.database.oc1.phx.anyhqljrabf7htyackgmsaqjfexoqgrzuuk33ju2q25z2al43tnd5mhhvkra + Display Name: dbsystem20220308221302 + Id: ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa + License Model: LICENSE_INCLUDED + Network: + Client Subnet: k8test-pubvcn + Domain Name: k8testpubvcn.k8test.oraclevcn.com + Host Name: host0130 + Listener Port: 1521 + Scan Dns Name: host0130-scan.k8testpubvcn.k8test.oraclevcn.com + Vcn Name: k8test + Node Count: 1 + Reco Storage Size In GB: 256 + Shape: VM.Standard2.1 + State: AVAILABLE + Storage Management: ASM + Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a + Time Zone: UTC + Work Requests: + Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrxivzvgzel47zuoyke5yk36o7mrgjl27vscd5z3bqptmyh3rxwbqq + Operation Type: Create DB System + Percent Complete: 100 + Time Accepted: 2022-03-08 22:13:02.999 +0000 UTC + Time Finished: 2022-03-08 23:11:50.46 +0000 UTC + Time Started: 2022-03-08 22:13:16.995 +0000 UTC + Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrc3fx6kwq4yxerk3ngztdbbngm7w4dnlddcdhxqxjn6e4kcyux5ca + Operation Type: Update Shape + Percent Complete: 100 + Time Accepted: 2022-03-08 23:33:42.807 +0000 UTC + Time Finished: 2022-03-08 23:46:21.126 +0000 UTC + Time Started: 2022-03-08 23:33:52.109 +0000 UTC + Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljr5sveun3f6k3zuz23py7mm7jncmpq5vwyajbo5ezhc765347defwq + Operation Type: Update Shape + Percent Complete: 100 + Time Accepted: 2022-03-09 00:25:03.644 +0000 UTC + Time Finished: 2022-03-09 00:38:59.526 +0000 UTC + Time Started: 2022-03-09 00:25:15.578 +0000 UTC +Events: +[root@docker-test-server test]# + + + +[root@docker-test-server test]# cat scale_up_storage.yaml +apiVersion: database.oracle.com/v1alpha1 +kind: DbcsSystem +metadata: + name: dbcssystem-existing +spec: + id: "ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa" + ociConfigMap: "oci-cred" + ociSecret: "oci-privatekey" + dbSystem: + availabilityDomain: "OLou:PHX-AD-1" + compartmentId: "ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya" + dbAdminPaswordSecret: "admin-password" + hostName: "host0130" + initialDataStorageSizeInGB: 512 + shape: "VM.Standard2.1" + sshPublicKeys: + - "oci-publickey" + subnetId: "ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a" +[root@docker-test-server test]# +[root@docker-test-server test]# kubectl apply -f scale_up_storage.yaml +dbcssystem.database.oracle.com/dbcssystem-existing configured +[root@docker-test-server test]# + + + +[root@docker-test-server test]# kubectl get ns + +kubectl get allNAME STATUS AGE +cert-manager Active 13d +default Active 139d +kube-node-lease Active 139d +kube-public Active 139d +kube-system Active 139d +oracle-database-operator-system Active 13d +shns Active 88d +[root@docker-test-server test]# +[root@docker-test-server test]# kubectl get all -n oracle-database-operator-system +NAME READY STATUS RESTARTS AGE +pod/oracle-database-operator-controller-manager-665874bd57-dlhls 1/1 Running 3 13d +pod/oracle-database-operator-controller-manager-665874bd57-g2cgw 1/1 Running 3 13d +pod/oracle-database-operator-controller-manager-665874bd57-q42f8 1/1 Running 4 13d + +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +service/oracle-database-operator-controller-manager-metrics-service ClusterIP 10.96.130.124 8443/TCP 13d +service/oracle-database-operator-webhook-service ClusterIP 10.96.4.104 443/TCP 13d + +NAME READY UP-TO-DATE AVAILABLE AGE +deployment.apps/oracle-database-operator-controller-manager 3/3 3 3 13d + +NAME DESIRED CURRENT READY AGE +replicaset.apps/oracle-database-operator-controller-manager-665874bd57 3 3 3 13d +[root@docker-test-server test]# + + +[root@docker-test-server test]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-dlhls -n oracle-database-operator-system +. +. +2022-03-09T00:48:11.373Z INFO controller-runtime.manager.controller.dbcssystem OCI provider configured succesfully {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-09T00:48:15.961Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-09T00:49:16.273Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-09T00:50:16.557Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-09T00:51:16.910Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-09T00:52:17.277Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-09T00:53:17.600Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-09T00:54:18.189Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-09T00:55:18.506Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-09T00:56:18.862Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-09T00:57:19.180Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-09T00:58:19.544Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-09T00:59:19.870Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-09T01:00:20.230Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-09T01:01:20.663Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-09T01:02:21.303Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-09T01:03:21.690Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} + + + + + +[root@docker-test-server test]# kubectl describe dbcssystems.database.oracle.com dbcssystem-existing +Name: dbcssystem-existing +Namespace: default +Labels: +Annotations: lastSuccessfulSpec: + {"dbSystem":{"compartmentId":"ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya","availabilityDomain":"O... +API Version: database.oracle.com/v1alpha1 +Kind: DbcsSystem +Metadata: + Creation Timestamp: 2022-03-08T23:27:48Z + Generation: 4 + Managed Fields: + API Version: database.oracle.com/v1alpha1 + Fields Type: FieldsV1 + fieldsV1: + f:metadata: + f:annotations: + f:lastSuccessfulSpec: + f:spec: + f:dbSystem: + f:cpuCoreCount: + f:dbBackupConfig: + f:dbEdition: + f:dbName: + f:dbUniqueName: + f:dbVersion: + f:diskRedundancy: + f:displayName: + f:faultDomains: + f:nodeCount: + f:status: + .: + f:availabilityDomain: + f:cpuCoreCount: + f:dataStoragePercentage: + f:dataStorageSizeInGBs: + f:dbEdition: + f:dbInfo: + f:displayName: + f:id: + f:licenseModel: + f:network: + .: + f:clientSubnet: + f:domainName: + f:hostName: + f:listenerPort: + f:scanDnsName: + f:vcnName: + f:nodeCount: + f:recoStorageSizeInGB: + f:shape: + f:state: + f:storageManagement: + f:subnetId: + f:timeZone: + f:workRequests: + Manager: manager + Operation: Update + Time: 2022-03-08T23:32:55Z + API Version: database.oracle.com/v1alpha1 + Fields Type: FieldsV1 + fieldsV1: + f:metadata: + f:annotations: + .: + f:kubectl.kubernetes.io/last-applied-configuration: + f:spec: + .: + f:dbSystem: + .: + f:availabilityDomain: + f:compartmentId: + f:dbAdminPaswordSecret: + f:hostName: + f:initialDataStorageSizeInGB: + f:shape: + f:sshPublicKeys: + f:subnetId: + f:id: + f:ociConfigMap: + f:ociSecret: + Manager: kubectl-client-side-apply + Operation: Update + Time: 2022-03-09T00:48:11Z + Resource Version: 55222013 + UID: 96d7bc49-33e9-42cc-8dd0-ada9a5a4c7e5 +Spec: + Db System: + Availability Domain: OLou:PHX-AD-1 + Compartment Id: ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya + Db Admin Pasword Secret: admin-password + Host Name: host0130 + Initial Data Storage Size In GB: 512 + Shape: VM.Standard2.1 + Ssh Public Keys: + oci-publickey + Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a + Id: ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa + Oci Config Map: oci-cred + Oci Secret: oci-privatekey +Status: + Availability Domain: OLou:PHX-AD-1 + Cpu Core Count: 1 + Data Storage Percentage: 80 + Data Storage Size In G Bs: 512 + Db Edition: ENTERPRISE_EDITION + Db Info: + Db Home Id: ocid1.dbhome.oc1.phx.anyhqljr5gy3jhqat52milqwt3gq6lwohhacwg5yi4mtzq7c7hag53lrkugq + Db Name: db0130 + Db Unique Name: db0130_phx1zn + Db Workload: OLTP + Id: ocid1.database.oc1.phx.anyhqljrabf7htyackgmsaqjfexoqgrzuuk33ju2q25z2al43tnd5mhhvkra + Display Name: dbsystem20220308221302 + Id: ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa + License Model: LICENSE_INCLUDED + Network: + Client Subnet: k8test-pubvcn + Domain Name: k8testpubvcn.k8test.oraclevcn.com + Host Name: host0130 + Listener Port: 1521 + Scan Dns Name: host0130-scan.k8testpubvcn.k8test.oraclevcn.com + Vcn Name: k8test + Node Count: 1 + Reco Storage Size In GB: 256 + Shape: VM.Standard2.1 + State: AVAILABLE + Storage Management: ASM + Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a + Time Zone: UTC + Work Requests: + Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrxivzvgzel47zuoyke5yk36o7mrgjl27vscd5z3bqptmyh3rxwbqq + Operation Type: Create DB System + Percent Complete: 100 + Time Accepted: 2022-03-08 22:13:02.999 +0000 UTC + Time Finished: 2022-03-08 23:11:50.46 +0000 UTC + Time Started: 2022-03-08 22:13:16.995 +0000 UTC + Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrc3fx6kwq4yxerk3ngztdbbngm7w4dnlddcdhxqxjn6e4kcyux5ca + Operation Type: Update Shape + Percent Complete: 100 + Time Accepted: 2022-03-08 23:33:42.807 +0000 UTC + Time Finished: 2022-03-08 23:46:21.126 +0000 UTC + Time Started: 2022-03-08 23:33:52.109 +0000 UTC + Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljr5sveun3f6k3zuz23py7mm7jncmpq5vwyajbo5ezhc765347defwq + Operation Type: Update Shape + Percent Complete: 100 + Time Accepted: 2022-03-09 00:25:03.644 +0000 UTC + Time Finished: 2022-03-09 00:38:59.526 +0000 UTC + Time Started: 2022-03-09 00:25:15.578 +0000 UTC + Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrbaqah6qktukvdlnx66fp2hlevegryfuppsshkqemfcdjtwfwaq3q + Operation Type: Scale Storage + Percent Complete: 100 + Time Accepted: 2022-03-09 00:48:54.849 +0000 UTC + Time Finished: 2022-03-09 01:03:10.885 +0000 UTC + Time Started: 2022-03-09 00:49:05.911 +0000 UTC +Events: +[root@docker-test-server test]# \ No newline at end of file diff --git a/docs/dbcs/provisioning/terminate_dbcs_system.md b/docs/dbcs/provisioning/terminate_dbcs_system.md new file mode 100644 index 00000000..071cda30 --- /dev/null +++ b/docs/dbcs/provisioning/terminate_dbcs_system.md @@ -0,0 +1,46 @@ +# Terminate an existing DBCS System + +In this use case, an existing OCI DBCS system deployed earlier is terminated using Oracle DB Operator DBCS controller. Its a 2 Step operation. + +In order to terminate an existing DBCS system, the steps will be: + +1. Bind the existing DBCS System to DBCS Controller. +2. Apply the change to terminate this DBCS System. + +**NOTE** We are assuming that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-deploy-a-dbcs-system-using-oracle-db-operator-dbcs-controller) steps to create the configmap and the secrets required during the deployment. + +This example uses `terminate_dbcs_system.yaml` to terminated a Single Instance DBCS VMDB using Oracle DB Operator DBCS Controller with: + +- OCID of existing VMDB as `ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa` +- OCI Configmap as `oci-cred` +- OCI Secret as `oci-privatekey` + +**NOTE:** For the details of the parameters to be used in the .yaml file, please refer [here](./dbcs_controller_parameters.md). + +Use the file: [terminate_dbcs_system.yaml](./terminate_dbcs_system.yaml) for this use case as below: + +1. Deploy the .yaml file: +```sh +[root@test-server DBCS]# kubectl apply -f terminate_dbcs_system.yaml +dbcssystem.database.oracle.com/dbcssystem-terminate created + + +[root@test-server DBCS]# kubectl delete -f terminate_dbcs_system.yaml +dbcssystem.database.oracle.com "dbcssystem-terminate" deleted +``` + +2. Check the logs of Oracle DB Operator Pod `pod/oracle-database-operator-controller-manager-665874bd57-g2cgw` for an update on the terminate operation been accepted. + +``` +[root@test-server DBCS]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-g2cgw -n oracle-database-operator-system +``` + +3. Check and confirm if the existing OCI DBCS system is NO longer available after sometime because of termination: + +``` +[root@test-server DBCS]# kubectl describe dbcssystems.database.oracle.com dbcssystem-terminate +``` + +## Sample Output + +[Here](./terminate_dbcs_system_sample_output.log) is the sample output for terminating an existing DBCS System deployed in OCI using Oracle DB Operator DBCS Controller with minimal parameters. diff --git a/docs/dbcs/provisioning/terminate_dbcs_system.yaml b/docs/dbcs/provisioning/terminate_dbcs_system.yaml new file mode 100644 index 00000000..075ce54e --- /dev/null +++ b/docs/dbcs/provisioning/terminate_dbcs_system.yaml @@ -0,0 +1,9 @@ +apiVersion: database.oracle.com/v1alpha1 +kind: DbcsSystem +metadata: + name: dbcssystem-terminate +spec: + hardLink: True + id: "ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa" + ociConfigMap: "oci-cred" + ociSecret: "oci-privatekey" diff --git a/docs/dbcs/provisioning/terminate_dbcs_system_sample_output.log b/docs/dbcs/provisioning/terminate_dbcs_system_sample_output.log new file mode 100644 index 00000000..8f0d5a36 --- /dev/null +++ b/docs/dbcs/provisioning/terminate_dbcs_system_sample_output.log @@ -0,0 +1,556 @@ +[root@docker-test-server test]# kubectl describe dbcssystems.database.oracle.com dbcssystem-existing +Name: dbcssystem-existing +Namespace: default +Labels: +Annotations: lastSuccessfulSpec: + {"dbSystem":{"compartmentId":"ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya","availabilityDomain":"O... +API Version: database.oracle.com/v1alpha1 +Kind: DbcsSystem +Metadata: + Creation Timestamp: 2022-03-08T23:27:48Z + Generation: 5 + Managed Fields: + API Version: database.oracle.com/v1alpha1 + Fields Type: FieldsV1 + fieldsV1: + f:metadata: + f:annotations: + f:lastSuccessfulSpec: + f:spec: + f:dbSystem: + f:cpuCoreCount: + f:dbBackupConfig: + f:dbEdition: + f:dbName: + f:dbUniqueName: + f:dbVersion: + f:diskRedundancy: + f:displayName: + f:faultDomains: + f:nodeCount: + f:status: + .: + f:availabilityDomain: + f:cpuCoreCount: + f:dataStoragePercentage: + f:dataStorageSizeInGBs: + f:dbEdition: + f:dbInfo: + f:displayName: + f:id: + f:licenseModel: + f:network: + .: + f:clientSubnet: + f:domainName: + f:hostName: + f:listenerPort: + f:scanDnsName: + f:vcnName: + f:nodeCount: + f:recoStorageSizeInGB: + f:shape: + f:state: + f:storageManagement: + f:subnetId: + f:timeZone: + f:workRequests: + Manager: manager + Operation: Update + Time: 2022-03-08T23:32:55Z + API Version: database.oracle.com/v1alpha1 + Fields Type: FieldsV1 + fieldsV1: + f:metadata: + f:annotations: + .: + f:kubectl.kubernetes.io/last-applied-configuration: + f:spec: + .: + f:dbSystem: + .: + f:availabilityDomain: + f:compartmentId: + f:dbAdminPaswordSecret: + f:hostName: + f:licenseModel: + f:shape: + f:sshPublicKeys: + f:subnetId: + f:id: + f:ociConfigMap: + f:ociSecret: + Manager: kubectl-client-side-apply + Operation: Update + Time: 2022-03-09T01:15:19Z + Resource Version: 55226409 + UID: 96d7bc49-33e9-42cc-8dd0-ada9a5a4c7e5 +Spec: + Db System: + Availability Domain: OLou:PHX-AD-1 + Compartment Id: ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya + Db Admin Pasword Secret: admin-password + Host Name: host0130 + License Model: BRING_YOUR_OWN_LICENSE + Shape: VM.Standard2.1 + Ssh Public Keys: + oci-publickey + Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a + Id: ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa + Oci Config Map: oci-cred + Oci Secret: oci-privatekey +Status: + Availability Domain: OLou:PHX-AD-1 + Cpu Core Count: 1 + Data Storage Percentage: 80 + Data Storage Size In G Bs: 512 + Db Edition: ENTERPRISE_EDITION + Db Info: + Db Home Id: ocid1.dbhome.oc1.phx.anyhqljr5gy3jhqat52milqwt3gq6lwohhacwg5yi4mtzq7c7hag53lrkugq + Db Name: db0130 + Db Unique Name: db0130_phx1zn + Db Workload: OLTP + Id: ocid1.database.oc1.phx.anyhqljrabf7htyackgmsaqjfexoqgrzuuk33ju2q25z2al43tnd5mhhvkra + Display Name: dbsystem20220308221302 + Id: ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa + License Model: BRING_YOUR_OWN_LICENSE + Network: + Client Subnet: k8test-pubvcn + Domain Name: k8testpubvcn.k8test.oraclevcn.com + Host Name: host0130 + Listener Port: 1521 + Scan Dns Name: host0130-scan.k8testpubvcn.k8test.oraclevcn.com + Vcn Name: k8test + Node Count: 1 + Reco Storage Size In GB: 256 + Shape: VM.Standard2.1 + State: AVAILABLE + Storage Management: ASM + Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a + Time Zone: UTC + Work Requests: + Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrxivzvgzel47zuoyke5yk36o7mrgjl27vscd5z3bqptmyh3rxwbqq + Operation Type: Create DB System + Percent Complete: 100 + Time Accepted: 2022-03-08 22:13:02.999 +0000 UTC + Time Finished: 2022-03-08 23:11:50.46 +0000 UTC + Time Started: 2022-03-08 22:13:16.995 +0000 UTC + Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrc3fx6kwq4yxerk3ngztdbbngm7w4dnlddcdhxqxjn6e4kcyux5ca + Operation Type: Update Shape + Percent Complete: 100 + Time Accepted: 2022-03-08 23:33:42.807 +0000 UTC + Time Finished: 2022-03-08 23:46:21.126 +0000 UTC + Time Started: 2022-03-08 23:33:52.109 +0000 UTC + Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljr5sveun3f6k3zuz23py7mm7jncmpq5vwyajbo5ezhc765347defwq + Operation Type: Update Shape + Percent Complete: 100 + Time Accepted: 2022-03-09 00:25:03.644 +0000 UTC + Time Finished: 2022-03-09 00:38:59.526 +0000 UTC + Time Started: 2022-03-09 00:25:15.578 +0000 UTC + Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrbaqah6qktukvdlnx66fp2hlevegryfuppsshkqemfcdjtwfwaq3q + Operation Type: Scale Storage + Percent Complete: 100 + Time Accepted: 2022-03-09 00:48:54.849 +0000 UTC + Time Finished: 2022-03-09 01:03:10.885 +0000 UTC + Time Started: 2022-03-09 00:49:05.911 +0000 UTC + Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrvhcpme5ijlsxup22kuumjuzn367vdxwhblv2nxpwshfwnig5au7a + Operation Type: Update DB System License Type + Percent Complete: 100 + Time Accepted: 2022-03-09 01:16:16.991 +0000 UTC + Time Finished: 2022-03-09 01:17:05.025 +0000 UTC + Time Started: 2022-03-09 01:16:24.716 +0000 UTC +Events: +[root@docker-test-server test]# + + +[root@docker-test-server test]# cat terminate_dbcs_system.yaml +apiVersion: database.oracle.com/v1alpha1 +kind: DbcsSystem +metadata: + name: dbcssystem-terminate +spec: + hardLink: True + id: "ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa" + ociConfigMap: "oci-cred" + ociSecret: "oci-privatekey" +[root@docker-test-server test]# +[root@docker-test-server test]# kubectl apply -f terminate_dbcs_system.yaml +dbcssystem.database.oracle.com/dbcssystem-terminate created +[root@docker-test-server test]# + + +[root@docker-test-server test]# kubectl get ns + +kubectl get allNAME STATUS AGE +cert-manager Active 13d +default Active 139d +kube-node-lease Active 139d +kube-public Active 139d +kube-system Active 139d +oracle-database-operator-system Active 13d +shns Active 88d +[root@docker-test-server test]# +[root@docker-test-server test]# kubectl get all -n oracle-database-operator-system +NAME READY STATUS RESTARTS AGE +pod/oracle-database-operator-controller-manager-665874bd57-dlhls 1/1 Running 3 13d +pod/oracle-database-operator-controller-manager-665874bd57-g2cgw 1/1 Running 3 13d +pod/oracle-database-operator-controller-manager-665874bd57-q42f8 1/1 Running 4 13d + +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +service/oracle-database-operator-controller-manager-metrics-service ClusterIP 10.96.130.124 8443/TCP 13d +service/oracle-database-operator-webhook-service ClusterIP 10.96.4.104 443/TCP 13d + +NAME READY UP-TO-DATE AVAILABLE AGE +deployment.apps/oracle-database-operator-controller-manager 3/3 3 3 13d + +NAME DESIRED CURRENT READY AGE +replicaset.apps/oracle-database-operator-controller-manager-665874bd57 3 3 3 13d +[root@docker-test-server test]# + + +[root@docker-test-server test]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-dlhls -n oracle-database-operator-system +. +. +2022-03-09T01:24:18.773Z INFO controller-runtime.manager.controller.dbcssystem OCI provider configured succesfully {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-terminate", "namespace": "default"} +2022-03-09T01:24:18.793Z INFO controller-runtime.manager.controller.dbcssystem Finalizer registered successfully. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-terminate", "namespace": "default"} +2022-03-09T01:24:22.461Z INFO controller-runtime.manager.controller.dbcssystem Sync information from remote DbcsSystem System successfully {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-terminate", "namespace": "default"} + + + + + + +[root@docker-test-server test]# kubectl delete -f terminate_dbcs_system.yaml +dbcssystem.database.oracle.com "dbcssystem-terminate" deleted +[root@docker-test-server test]# + + + +[root@docker-test-server test]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-dlhls -n oracle-database-operator-system +. +. +2022-03-09T01:25:05.199Z INFO controller-runtime.manager.controller.dbcssystem OCI provider configured succesfully {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-terminate", "namespace": "default"} +2022-03-09T01:25:05.199Z INFO controller-runtime.manager.controller.dbcssystem Terminate DbcsSystem Database: {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-terminate", "namespace": "default"} +2022-03-09T01:25:06.920Z INFO controller-runtime.manager.controller.dbcssystem Finalizer unregistered successfully. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-terminate", "namespace": "default"} + + + + +[root@docker-test-server test]# kubectl describe dbcssystems.database.oracle.com dbcssystem-terminate +Error from server (NotFound): dbcssystems.database.oracle.com "dbcssystem-terminate" not found +[root@docker-test-server test]# + + + +[root@docker-test-server test]# kubectl describe dbcssystems.database.oracle.com dbcssystem-existing +Name: dbcssystem-existing +Namespace: default +Labels: +Annotations: lastSuccessfulSpec: + {"dbSystem":{"compartmentId":"ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya","availabilityDomain":"O... +API Version: database.oracle.com/v1alpha1 +Kind: DbcsSystem +Metadata: + Creation Timestamp: 2022-03-08T23:27:48Z + Generation: 5 + Managed Fields: + API Version: database.oracle.com/v1alpha1 + Fields Type: FieldsV1 + fieldsV1: + f:metadata: + f:annotations: + f:lastSuccessfulSpec: + f:spec: + f:dbSystem: + f:cpuCoreCount: + f:dbBackupConfig: + f:dbEdition: + f:dbName: + f:dbUniqueName: + f:dbVersion: + f:diskRedundancy: + f:displayName: + f:faultDomains: + f:nodeCount: + f:status: + .: + f:availabilityDomain: + f:cpuCoreCount: + f:dataStoragePercentage: + f:dataStorageSizeInGBs: + f:dbEdition: + f:dbInfo: + f:displayName: + f:id: + f:licenseModel: + f:network: + .: + f:clientSubnet: + f:domainName: + f:hostName: + f:listenerPort: + f:scanDnsName: + f:vcnName: + f:nodeCount: + f:recoStorageSizeInGB: + f:shape: + f:state: + f:storageManagement: + f:subnetId: + f:timeZone: + f:workRequests: + Manager: manager + Operation: Update + Time: 2022-03-08T23:32:55Z + API Version: database.oracle.com/v1alpha1 + Fields Type: FieldsV1 + fieldsV1: + f:metadata: + f:annotations: + .: + f:kubectl.kubernetes.io/last-applied-configuration: + f:spec: + .: + f:dbSystem: + .: + f:availabilityDomain: + f:compartmentId: + f:dbAdminPaswordSecret: + f:hostName: + f:licenseModel: + f:shape: + f:sshPublicKeys: + f:subnetId: + f:id: + f:ociConfigMap: + f:ociSecret: + Manager: kubectl-client-side-apply + Operation: Update + Time: 2022-03-09T01:15:19Z + Resource Version: 55226409 + UID: 96d7bc49-33e9-42cc-8dd0-ada9a5a4c7e5 +Spec: + Db System: + Availability Domain: OLou:PHX-AD-1 + Compartment Id: ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya + Db Admin Pasword Secret: admin-password + Host Name: host0130 + License Model: BRING_YOUR_OWN_LICENSE + Shape: VM.Standard2.1 + Ssh Public Keys: + oci-publickey + Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a + Id: ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa + Oci Config Map: oci-cred + Oci Secret: oci-privatekey +Status: + Availability Domain: OLou:PHX-AD-1 + Cpu Core Count: 1 + Data Storage Percentage: 80 + Data Storage Size In G Bs: 512 + Db Edition: ENTERPRISE_EDITION + Db Info: + Db Home Id: ocid1.dbhome.oc1.phx.anyhqljr5gy3jhqat52milqwt3gq6lwohhacwg5yi4mtzq7c7hag53lrkugq + Db Name: db0130 + Db Unique Name: db0130_phx1zn + Db Workload: OLTP + Id: ocid1.database.oc1.phx.anyhqljrabf7htyackgmsaqjfexoqgrzuuk33ju2q25z2al43tnd5mhhvkra + Display Name: dbsystem20220308221302 + Id: ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa + License Model: BRING_YOUR_OWN_LICENSE + Network: + Client Subnet: k8test-pubvcn + Domain Name: k8testpubvcn.k8test.oraclevcn.com + Host Name: host0130 + Listener Port: 1521 + Scan Dns Name: host0130-scan.k8testpubvcn.k8test.oraclevcn.com + Vcn Name: k8test + Node Count: 1 + Reco Storage Size In GB: 256 + Shape: VM.Standard2.1 + State: AVAILABLE + Storage Management: ASM + Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a + Time Zone: UTC + Work Requests: + Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrxivzvgzel47zuoyke5yk36o7mrgjl27vscd5z3bqptmyh3rxwbqq + Operation Type: Create DB System + Percent Complete: 100 + Time Accepted: 2022-03-08 22:13:02.999 +0000 UTC + Time Finished: 2022-03-08 23:11:50.46 +0000 UTC + Time Started: 2022-03-08 22:13:16.995 +0000 UTC + Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrc3fx6kwq4yxerk3ngztdbbngm7w4dnlddcdhxqxjn6e4kcyux5ca + Operation Type: Update Shape + Percent Complete: 100 + Time Accepted: 2022-03-08 23:33:42.807 +0000 UTC + Time Finished: 2022-03-08 23:46:21.126 +0000 UTC + Time Started: 2022-03-08 23:33:52.109 +0000 UTC + Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljr5sveun3f6k3zuz23py7mm7jncmpq5vwyajbo5ezhc765347defwq + Operation Type: Update Shape + Percent Complete: 100 + Time Accepted: 2022-03-09 00:25:03.644 +0000 UTC + Time Finished: 2022-03-09 00:38:59.526 +0000 UTC + Time Started: 2022-03-09 00:25:15.578 +0000 UTC + Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrbaqah6qktukvdlnx66fp2hlevegryfuppsshkqemfcdjtwfwaq3q + Operation Type: Scale Storage + Percent Complete: 100 + Time Accepted: 2022-03-09 00:48:54.849 +0000 UTC + Time Finished: 2022-03-09 01:03:10.885 +0000 UTC + Time Started: 2022-03-09 00:49:05.911 +0000 UTC + Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrvhcpme5ijlsxup22kuumjuzn367vdxwhblv2nxpwshfwnig5au7a + Operation Type: Update DB System License Type + Percent Complete: 100 + Time Accepted: 2022-03-09 01:16:16.991 +0000 UTC + Time Finished: 2022-03-09 01:17:05.025 +0000 UTC + Time Started: 2022-03-09 01:16:24.716 +0000 UTC +Events: +[root@docker-test-server test]# +[root@docker-test-server test]# +[root@docker-test-server test]# +[root@docker-test-server test]# kubectl describe dbcssystems.database.oracle.com dbcssystem-create +Name: dbcssystem-create +Namespace: default +Labels: +Annotations: lastSuccessfulSpec: + {"dbSystem":{"compartmentId":"ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya","availabilityDomain":"O... +API Version: database.oracle.com/v1alpha1 +Kind: DbcsSystem +Metadata: + Creation Timestamp: 2022-03-08T22:12:57Z + Generation: 1 + Managed Fields: + API Version: database.oracle.com/v1alpha1 + Fields Type: FieldsV1 + fieldsV1: + f:metadata: + f:annotations: + .: + f:kubectl.kubernetes.io/last-applied-configuration: + f:spec: + .: + f:dbSystem: + .: + f:availabilityDomain: + f:compartmentId: + f:dbAdminPaswordSecret: + f:dbName: + f:dbVersion: + f:dbWorkload: + f:hostName: + f:shape: + f:sshPublicKeys: + f:subnetId: + f:ociConfigMap: + f:ociSecret: + Manager: kubectl-client-side-apply + Operation: Update + Time: 2022-03-08T22:12:57Z + API Version: database.oracle.com/v1alpha1 + Fields Type: FieldsV1 + fieldsV1: + f:metadata: + f:annotations: + f:lastSuccessfulSpec: + f:spec: + f:dbSystem: + f:dbBackupConfig: + f:id: + f:status: + .: + f:availabilityDomain: + f:cpuCoreCount: + f:dataStoragePercentage: + f:dataStorageSizeInGBs: + f:dbEdition: + f:dbInfo: + f:displayName: + f:id: + f:licenseModel: + f:network: + .: + f:clientSubnet: + f:domainName: + f:hostName: + f:listenerPort: + f:scanDnsName: + f:vcnName: + f:nodeCount: + f:recoStorageSizeInGB: + f:shape: + f:state: + f:storageManagement: + f:subnetId: + f:timeZone: + f:workRequests: + Manager: manager + Operation: Update + Time: 2022-03-08T23:13:34Z + Resource Version: 55187354 + UID: 0cdbebc0-3aeb-43b1-ae7f-eb36e6c56000 +Spec: + Db System: + Availability Domain: OLou:PHX-AD-1 + Compartment Id: ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya + Db Admin Pasword Secret: admin-password + Db Name: db0130 + Db Version: 21c + Db Workload: OLTP + Host Name: host0130 + Shape: VM.Standard2.1 + Ssh Public Keys: + oci-publickey + Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a + Oci Config Map: oci-cred + Oci Secret: oci-privatekey +Status: + Availability Domain: OLou:PHX-AD-1 + Cpu Core Count: 1 + Data Storage Percentage: 80 + Data Storage Size In G Bs: 256 + Db Edition: ENTERPRISE_EDITION + Db Info: + Db Home Id: ocid1.dbhome.oc1.phx.anyhqljr5gy3jhqat52milqwt3gq6lwohhacwg5yi4mtzq7c7hag53lrkugq + Db Name: db0130 + Db Unique Name: db0130_phx1zn + Db Workload: OLTP + Id: ocid1.database.oc1.phx.anyhqljrabf7htyackgmsaqjfexoqgrzuuk33ju2q25z2al43tnd5mhhvkra + Display Name: dbsystem20220308221302 + Id: ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa + License Model: LICENSE_INCLUDED + Network: + Client Subnet: k8test-pubvcn + Domain Name: k8testpubvcn.k8test.oraclevcn.com + Host Name: host0130 + Listener Port: 1521 + Scan Dns Name: host0130-scan.k8testpubvcn.k8test.oraclevcn.com + Vcn Name: k8test + Node Count: 1 + Reco Storage Size In GB: 256 + Shape: VM.Standard2.1 + State: AVAILABLE + Storage Management: ASM + Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a + Time Zone: UTC + Work Requests: + Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrxivzvgzel47zuoyke5yk36o7mrgjl27vscd5z3bqptmyh3rxwbqq + Operation Type: Create DB System + Percent Complete: 100 + Time Accepted: 2022-03-08 22:13:02.999 +0000 UTC + Time Finished: 2022-03-08 23:11:50.46 +0000 UTC + Time Started: 2022-03-08 22:13:16.995 +0000 UTC +Events: +[root@docker-test-server test]# + + +[root@docker-test-server test]# kubectl delete dbcssystems.database.oracle.com dbcssystem-existing +dbcssystem.database.oracle.com "dbcssystem-existing" deleted +[root@docker-test-server test]# +[root@docker-test-server test]# kubectl describe dbcssystems.database.oracle.com dbcssystem-existing +Error from server (NotFound): dbcssystems.database.oracle.com "dbcssystem-existing" not found +[root@docker-test-server test]# +[root@docker-test-server test]# kubectl delete dbcssystems.database.oracle.com dbcssystem-create +dbcssystem.database.oracle.com "dbcssystem-create" deleted +[root@docker-test-server test]# +[root@docker-test-server test]# kubectl delete dbcssystems.database.oracle.com dbcssystem-create +Error from server (NotFound): dbcssystems.database.oracle.com "dbcssystem-create" not found +[root@docker-test-server test]# \ No newline at end of file diff --git a/docs/dbcs/provisioning/update_license.md b/docs/dbcs/provisioning/update_license.md new file mode 100644 index 00000000..7c7c43b7 --- /dev/null +++ b/docs/dbcs/provisioning/update_license.md @@ -0,0 +1,46 @@ +# Update License type of an existing DBCS System + +In this use case, the license type of an existing OCI DBCS system deployed earlier is changed from `License Included` to `Bring your own license` using Oracle DB Operator DBCS controller. Its a 2 Step operation. + +In order to update the license type an existing DBCS system, the steps will be: + +1. Bind the existing DBCS System to DBCS Controller. +2. Apply the change to change its license type. + +**NOTE** We are assuming that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-deploy-a-dbcs-system-using-oracle-db-operator-dbcs-controller) steps to create the configmap and the secrets required during the deployment. + +This example uses `update_license.yaml` to change the license type of a Single Instance DBCS VMDB using Oracle DB Operator DBCS Controller with: + +- OCID of existing VMDB as `ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa` +- OCI Configmap as `oci-cred` +- OCI Secret as `oci-privatekey` +- Availability Domain for the DBCS VMDB as `OLou:PHX-AD-1` +- Compartment OCID as `ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya` +- Database Admin Credential as `admin-password` +- Database Hostname Prefix as `host0130` +- Target license model as `BRING_YOUR_OWN_LICENSE` +- Oracle VMDB Shape as `VM.Standard2.1` +- SSH Public key for the DBCS system being deployed as `oci-publickey` +- OCID of the Subnet as `ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a` + +**NOTE:** For the details of the parameters to be used in the .yaml file, please refer [here](./dbcs_controller_parameters.md). + +Use the file: [update_license.yaml](./update_license.yaml) for this use case as below: + +1. Deploy the .yaml file: +```sh +[root@test-server DBCS]# kubectl apply -f update_license.yaml +dbcssystem.database.oracle.com/dbcssystem-existing configured +``` + +2. Monitor the Oracle DB Operator Pod `pod/oracle-database-operator-controller-manager-665874bd57-g2cgw` for the progress of the DBCS VMDB Scale up. + +NOTE: Check the DB Operator Pod name in your environment. + +``` +[root@docker-test-server DBCS]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-g2cgw -n oracle-database-operator-system +``` + +## Sample Output + +[Here](./update_license_sample_output.log) is the sample output for updating the license type an existing DBCS System deployed in OCI using Oracle DB Operator DBCS Controller. diff --git a/docs/dbcs/provisioning/update_license.yaml b/docs/dbcs/provisioning/update_license.yaml new file mode 100644 index 00000000..1fb54a64 --- /dev/null +++ b/docs/dbcs/provisioning/update_license.yaml @@ -0,0 +1,18 @@ +apiVersion: database.oracle.com/v1alpha1 +kind: DbcsSystem +metadata: + name: dbcssystem-existing +spec: + id: "ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa" + dbSystem: + availabilityDomain: "OLou:PHX-AD-1" + compartmentId: "ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya" + dbAdminPaswordSecret: "admin-password" + hostName: "host0130" + licenseModel: "BRING_YOUR_OWN_LICENSE" + shape: "VM.Standard2.1" + sshPublicKeys: + - "oci-publickey" + subnetId: "ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a" + ociConfigMap: "oci-cred" + ociSecret: "oci-privatekey" diff --git a/docs/dbcs/provisioning/update_license_sample_output.log b/docs/dbcs/provisioning/update_license_sample_output.log new file mode 100644 index 00000000..7bed4383 --- /dev/null +++ b/docs/dbcs/provisioning/update_license_sample_output.log @@ -0,0 +1,388 @@ +[root@docker-test-server test]# kubectl describe dbcssystems.database.oracle.com dbcssystem-existing +Name: dbcssystem-existing +Namespace: default +Labels: +Annotations: lastSuccessfulSpec: + {"dbSystem":{"compartmentId":"ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya","availabilityDomain":"O... +API Version: database.oracle.com/v1alpha1 +Kind: DbcsSystem +Metadata: + Creation Timestamp: 2022-03-08T23:27:48Z + Generation: 4 + Managed Fields: + API Version: database.oracle.com/v1alpha1 + Fields Type: FieldsV1 + fieldsV1: + f:metadata: + f:annotations: + f:lastSuccessfulSpec: + f:spec: + f:dbSystem: + f:cpuCoreCount: + f:dbBackupConfig: + f:dbEdition: + f:dbName: + f:dbUniqueName: + f:dbVersion: + f:diskRedundancy: + f:displayName: + f:faultDomains: + f:nodeCount: + f:status: + .: + f:availabilityDomain: + f:cpuCoreCount: + f:dataStoragePercentage: + f:dataStorageSizeInGBs: + f:dbEdition: + f:dbInfo: + f:displayName: + f:id: + f:licenseModel: + f:network: + .: + f:clientSubnet: + f:domainName: + f:hostName: + f:listenerPort: + f:scanDnsName: + f:vcnName: + f:nodeCount: + f:recoStorageSizeInGB: + f:shape: + f:state: + f:storageManagement: + f:subnetId: + f:timeZone: + f:workRequests: + Manager: manager + Operation: Update + Time: 2022-03-08T23:32:55Z + API Version: database.oracle.com/v1alpha1 + Fields Type: FieldsV1 + fieldsV1: + f:metadata: + f:annotations: + .: + f:kubectl.kubernetes.io/last-applied-configuration: + f:spec: + .: + f:dbSystem: + .: + f:availabilityDomain: + f:compartmentId: + f:dbAdminPaswordSecret: + f:hostName: + f:initialDataStorageSizeInGB: + f:shape: + f:sshPublicKeys: + f:subnetId: + f:id: + f:ociConfigMap: + f:ociSecret: + Manager: kubectl-client-side-apply + Operation: Update + Time: 2022-03-09T00:48:11Z + Resource Version: 55222013 + UID: 96d7bc49-33e9-42cc-8dd0-ada9a5a4c7e5 +Spec: + Db System: + Availability Domain: OLou:PHX-AD-1 + Compartment Id: ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya + Db Admin Pasword Secret: admin-password + Host Name: host0130 + Initial Data Storage Size In GB: 512 + Shape: VM.Standard2.1 + Ssh Public Keys: + oci-publickey + Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a + Id: ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa + Oci Config Map: oci-cred + Oci Secret: oci-privatekey +Status: + Availability Domain: OLou:PHX-AD-1 + Cpu Core Count: 1 + Data Storage Percentage: 80 + Data Storage Size In G Bs: 512 + Db Edition: ENTERPRISE_EDITION + Db Info: + Db Home Id: ocid1.dbhome.oc1.phx.anyhqljr5gy3jhqat52milqwt3gq6lwohhacwg5yi4mtzq7c7hag53lrkugq + Db Name: db0130 + Db Unique Name: db0130_phx1zn + Db Workload: OLTP + Id: ocid1.database.oc1.phx.anyhqljrabf7htyackgmsaqjfexoqgrzuuk33ju2q25z2al43tnd5mhhvkra + Display Name: dbsystem20220308221302 + Id: ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa + License Model: LICENSE_INCLUDED + Network: + Client Subnet: k8test-pubvcn + Domain Name: k8testpubvcn.k8test.oraclevcn.com + Host Name: host0130 + Listener Port: 1521 + Scan Dns Name: host0130-scan.k8testpubvcn.k8test.oraclevcn.com + Vcn Name: k8test + Node Count: 1 + Reco Storage Size In GB: 256 + Shape: VM.Standard2.1 + State: AVAILABLE + Storage Management: ASM + Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a + Time Zone: UTC + Work Requests: + Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrxivzvgzel47zuoyke5yk36o7mrgjl27vscd5z3bqptmyh3rxwbqq + Operation Type: Create DB System + Percent Complete: 100 + Time Accepted: 2022-03-08 22:13:02.999 +0000 UTC + Time Finished: 2022-03-08 23:11:50.46 +0000 UTC + Time Started: 2022-03-08 22:13:16.995 +0000 UTC + Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrc3fx6kwq4yxerk3ngztdbbngm7w4dnlddcdhxqxjn6e4kcyux5ca + Operation Type: Update Shape + Percent Complete: 100 + Time Accepted: 2022-03-08 23:33:42.807 +0000 UTC + Time Finished: 2022-03-08 23:46:21.126 +0000 UTC + Time Started: 2022-03-08 23:33:52.109 +0000 UTC + Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljr5sveun3f6k3zuz23py7mm7jncmpq5vwyajbo5ezhc765347defwq + Operation Type: Update Shape + Percent Complete: 100 + Time Accepted: 2022-03-09 00:25:03.644 +0000 UTC + Time Finished: 2022-03-09 00:38:59.526 +0000 UTC + Time Started: 2022-03-09 00:25:15.578 +0000 UTC + Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrbaqah6qktukvdlnx66fp2hlevegryfuppsshkqemfcdjtwfwaq3q + Operation Type: Scale Storage + Percent Complete: 100 + Time Accepted: 2022-03-09 00:48:54.849 +0000 UTC + Time Finished: 2022-03-09 01:03:10.885 +0000 UTC + Time Started: 2022-03-09 00:49:05.911 +0000 UTC +Events: +[root@docker-test-server test]# + + + +[root@docker-test-server test]# cat update_license.yaml +apiVersion: database.oracle.com/v1alpha1 +kind: DbcsSystem +metadata: + name: dbcssystem-existing +spec: + id: "ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa" + dbSystem: + availabilityDomain: "OLou:PHX-AD-1" + compartmentId: "ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya" + dbAdminPaswordSecret: "admin-password" + hostName: "host0130" + licenseModel: "BRING_YOUR_OWN_LICENSE" + shape: "VM.Standard2.1" + sshPublicKeys: + - "oci-publickey" + subnetId: "ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a" + ociConfigMap: "oci-cred" + ociSecret: "oci-privatekey" +[root@docker-test-server test]# +[root@docker-test-server test]# kubectl apply -f update_license.yaml +dbcssystem.database.oracle.com/dbcssystem-existing configured +[root@docker-test-server test]# + + +[root@docker-test-server test]# kubectl get ns + +kubectl get allNAME STATUS AGE +cert-manager Active 13d +default Active 139d +kube-node-lease Active 139d +kube-public Active 139d +kube-system Active 139d +oracle-database-operator-system Active 13d +shns Active 88d +[root@docker-test-server test]# +[root@docker-test-server test]# kubectl get all -n oracle-database-operator-system +NAME READY STATUS RESTARTS AGE +pod/oracle-database-operator-controller-manager-665874bd57-dlhls 1/1 Running 3 13d +pod/oracle-database-operator-controller-manager-665874bd57-g2cgw 1/1 Running 3 13d +pod/oracle-database-operator-controller-manager-665874bd57-q42f8 1/1 Running 4 13d + +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +service/oracle-database-operator-controller-manager-metrics-service ClusterIP 10.96.130.124 8443/TCP 13d +service/oracle-database-operator-webhook-service ClusterIP 10.96.4.104 443/TCP 13d + +NAME READY UP-TO-DATE AVAILABLE AGE +deployment.apps/oracle-database-operator-controller-manager 3/3 3 3 13d + +NAME DESIRED CURRENT READY AGE +replicaset.apps/oracle-database-operator-controller-manager-665874bd57 3 3 3 13d +[root@docker-test-server test]# + + +[root@docker-test-server test]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-dlhls -n oracle-database-operator-system +. +. +2022-03-09T01:15:19.090Z INFO controller-runtime.manager.controller.dbcssystem OCI provider configured succesfully {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-09T01:15:23.534Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-09T01:16:23.931Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} +2022-03-09T01:17:24.701Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} + + + + +[root@docker-test-server test]# kubectl describe dbcssystems.database.oracle.com dbcssystem-existing +Name: dbcssystem-existing +Namespace: default +Labels: +Annotations: lastSuccessfulSpec: + {"dbSystem":{"compartmentId":"ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya","availabilityDomain":"O... +API Version: database.oracle.com/v1alpha1 +Kind: DbcsSystem +Metadata: + Creation Timestamp: 2022-03-08T23:27:48Z + Generation: 5 + Managed Fields: + API Version: database.oracle.com/v1alpha1 + Fields Type: FieldsV1 + fieldsV1: + f:metadata: + f:annotations: + f:lastSuccessfulSpec: + f:spec: + f:dbSystem: + f:cpuCoreCount: + f:dbBackupConfig: + f:dbEdition: + f:dbName: + f:dbUniqueName: + f:dbVersion: + f:diskRedundancy: + f:displayName: + f:faultDomains: + f:nodeCount: + f:status: + .: + f:availabilityDomain: + f:cpuCoreCount: + f:dataStoragePercentage: + f:dataStorageSizeInGBs: + f:dbEdition: + f:dbInfo: + f:displayName: + f:id: + f:licenseModel: + f:network: + .: + f:clientSubnet: + f:domainName: + f:hostName: + f:listenerPort: + f:scanDnsName: + f:vcnName: + f:nodeCount: + f:recoStorageSizeInGB: + f:shape: + f:state: + f:storageManagement: + f:subnetId: + f:timeZone: + f:workRequests: + Manager: manager + Operation: Update + Time: 2022-03-08T23:32:55Z + API Version: database.oracle.com/v1alpha1 + Fields Type: FieldsV1 + fieldsV1: + f:metadata: + f:annotations: + .: + f:kubectl.kubernetes.io/last-applied-configuration: + f:spec: + .: + f:dbSystem: + .: + f:availabilityDomain: + f:compartmentId: + f:dbAdminPaswordSecret: + f:hostName: + f:licenseModel: + f:shape: + f:sshPublicKeys: + f:subnetId: + f:id: + f:ociConfigMap: + f:ociSecret: + Manager: kubectl-client-side-apply + Operation: Update + Time: 2022-03-09T01:15:19Z + Resource Version: 55226409 + UID: 96d7bc49-33e9-42cc-8dd0-ada9a5a4c7e5 +Spec: + Db System: + Availability Domain: OLou:PHX-AD-1 + Compartment Id: ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya + Db Admin Pasword Secret: admin-password + Host Name: host0130 + License Model: BRING_YOUR_OWN_LICENSE + Shape: VM.Standard2.1 + Ssh Public Keys: + oci-publickey + Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a + Id: ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa + Oci Config Map: oci-cred + Oci Secret: oci-privatekey +Status: + Availability Domain: OLou:PHX-AD-1 + Cpu Core Count: 1 + Data Storage Percentage: 80 + Data Storage Size In G Bs: 512 + Db Edition: ENTERPRISE_EDITION + Db Info: + Db Home Id: ocid1.dbhome.oc1.phx.anyhqljr5gy3jhqat52milqwt3gq6lwohhacwg5yi4mtzq7c7hag53lrkugq + Db Name: db0130 + Db Unique Name: db0130_phx1zn + Db Workload: OLTP + Id: ocid1.database.oc1.phx.anyhqljrabf7htyackgmsaqjfexoqgrzuuk33ju2q25z2al43tnd5mhhvkra + Display Name: dbsystem20220308221302 + Id: ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa + License Model: BRING_YOUR_OWN_LICENSE + Network: + Client Subnet: k8test-pubvcn + Domain Name: k8testpubvcn.k8test.oraclevcn.com + Host Name: host0130 + Listener Port: 1521 + Scan Dns Name: host0130-scan.k8testpubvcn.k8test.oraclevcn.com + Vcn Name: k8test + Node Count: 1 + Reco Storage Size In GB: 256 + Shape: VM.Standard2.1 + State: AVAILABLE + Storage Management: ASM + Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a + Time Zone: UTC + Work Requests: + Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrxivzvgzel47zuoyke5yk36o7mrgjl27vscd5z3bqptmyh3rxwbqq + Operation Type: Create DB System + Percent Complete: 100 + Time Accepted: 2022-03-08 22:13:02.999 +0000 UTC + Time Finished: 2022-03-08 23:11:50.46 +0000 UTC + Time Started: 2022-03-08 22:13:16.995 +0000 UTC + Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrc3fx6kwq4yxerk3ngztdbbngm7w4dnlddcdhxqxjn6e4kcyux5ca + Operation Type: Update Shape + Percent Complete: 100 + Time Accepted: 2022-03-08 23:33:42.807 +0000 UTC + Time Finished: 2022-03-08 23:46:21.126 +0000 UTC + Time Started: 2022-03-08 23:33:52.109 +0000 UTC + Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljr5sveun3f6k3zuz23py7mm7jncmpq5vwyajbo5ezhc765347defwq + Operation Type: Update Shape + Percent Complete: 100 + Time Accepted: 2022-03-09 00:25:03.644 +0000 UTC + Time Finished: 2022-03-09 00:38:59.526 +0000 UTC + Time Started: 2022-03-09 00:25:15.578 +0000 UTC + Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrbaqah6qktukvdlnx66fp2hlevegryfuppsshkqemfcdjtwfwaq3q + Operation Type: Scale Storage + Percent Complete: 100 + Time Accepted: 2022-03-09 00:48:54.849 +0000 UTC + Time Finished: 2022-03-09 01:03:10.885 +0000 UTC + Time Started: 2022-03-09 00:49:05.911 +0000 UTC + Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrvhcpme5ijlsxup22kuumjuzn367vdxwhblv2nxpwshfwnig5au7a + Operation Type: Update DB System License Type + Percent Complete: 100 + Time Accepted: 2022-03-09 01:16:16.991 +0000 UTC + Time Finished: 2022-03-09 01:17:05.025 +0000 UTC + Time Started: 2022-03-09 01:16:24.716 +0000 UTC +Events: +[root@docker-test-server test]# \ No newline at end of file diff --git a/go.sum b/go.sum index e2b44fb1..bf657b15 100644 --- a/go.sum +++ b/go.sum @@ -395,6 +395,8 @@ github.com/onsi/gomega v1.17.0/go.mod h1:HnhC7FXeEQY45zxNK3PPoIUhzk/80Xly9PcubAl github.com/onsi/gomega v1.18.1 h1:M1GfJqGRrBrrGGsbxzV5dqM2U2ApXefZCQpkukxYRLE= github.com/onsi/gomega v1.18.1/go.mod h1:0q+aL8jAiMXy9hbwj2mr5GziHiwhAIQpFmmtT5hitRs= github.com/opentracing/opentracing-go v1.1.0/go.mod h1:UkNAQd3GIcIGf0SeVgPpRdFStlNbqXla1AfSYxPUl2o= +github.com/oracle/oci-go-sdk/v51 v51.0.0 h1:eDUVMsAzvf+jfq4xbtpQrxznYNjccwKKyIhmRvZLgwI= +github.com/oracle/oci-go-sdk/v51 v51.0.0/go.mod h1:d9KSNXwE64drofxoor+y/JWofJqLqRF9D1/AtfYIE10= github.com/oracle/oci-go-sdk/v63 v63.0.0 h1:OOGCUmaDzrd5zTG8pljcnkR1ZHxg/991uEiQJi95/4E= github.com/oracle/oci-go-sdk/v63 v63.0.0/go.mod h1:n6V9PcyRW5wtHaNd2TltbV3sWvbNy3PNqLmLrcT23Fg= github.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc= @@ -454,6 +456,7 @@ github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1 github.com/smartystreets/goconvey v1.6.4/go.mod h1:syvi0/a8iFYH4r/RixwvyeAJjdLS9QV7WQ/tjFTllLA= github.com/soheilhy/cmux v0.1.4/go.mod h1:IM3LyeVVIOuxMH7sFAkER9+bJ4dT7Ms6E4xg4kGIyLM= github.com/soheilhy/cmux v0.1.5/go.mod h1:T7TcVDs9LWfQgPlPsdngu6I6QIoyIFZDDC6sNE1GqG0= +github.com/sony/gobreaker v0.4.2-0.20210216022020-dd874f9dd33b/go.mod h1:ZKptC7FHNvhBz7dN2LGjPVBz2sZJmc0/PkyDJOjmxWY= github.com/sony/gobreaker v0.5.0 h1:dRCvqm0P490vZPmy7ppEk2qCnCieBooFJ+YoXGYB+yg= github.com/sony/gobreaker v0.5.0/go.mod h1:ZKptC7FHNvhBz7dN2LGjPVBz2sZJmc0/PkyDJOjmxWY= github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA= diff --git a/main.go b/main.go index ffe2d91b..4f79b45f 100644 --- a/main.go +++ b/main.go @@ -229,7 +229,15 @@ func main() { setupLog.Error(err, "unable to create webhook", "webhook", "AutonomousDatabaseRestore") os.Exit(1) } - + if err = (&databasecontroller.DbcsSystemReconciler{ + KubeClient: mgr.GetClient(), + Logger: ctrl.Log.WithName("controllers").WithName("database").WithName("DbcsSystem"), + Scheme: mgr.GetScheme(), + Recorder: mgr.GetEventRecorderFor("DbcsSystem"), + }).SetupWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create controller", "controller", "DbcsSystem") + os.Exit(1) + } // +kubebuilder:scaffold:builder // Add index for PDB CR to enable mgr to cache PDBs diff --git a/oracle-database-operator.yaml b/oracle-database-operator.yaml index ae464a57..b071a47f 100644 --- a/oracle-database-operator.yaml +++ b/oracle-database-operator.yaml @@ -11,45 +11,33 @@ metadata: annotations: controller-gen.kubebuilder.io/version: v0.6.1 creationTimestamp: null - name: autonomousdatabases.database.oracle.com + name: autonomouscontainerdatabases.database.oracle.com spec: group: database.oracle.com names: - kind: AutonomousDatabase - listKind: AutonomousDatabaseList - plural: autonomousdatabases + kind: AutonomousContainerDatabase + listKind: AutonomousContainerDatabaseList + plural: autonomouscontainerdatabases shortNames: - - adb - - adbs - singular: autonomousdatabase + - acd + - acds + singular: autonomouscontainerdatabase scope: Namespaced versions: - additionalPrinterColumns: - - jsonPath: .status.displayName - name: Display Name + - jsonPath: .spec.displayName + name: DisplayName type: string - jsonPath: .status.lifecycleState name: State type: string - - jsonPath: .status.isDedicated - name: Dedicated - type: string - - jsonPath: .status.cpuCoreCount - name: OCPUs - type: integer - - jsonPath: .status.dataStorageSizeInTBs - name: Storage (TB) - type: integer - - jsonPath: .status.dbWorkload - name: Workload Type - type: string - jsonPath: .status.timeCreated name: Created type: string name: v1alpha1 schema: openAPIV3Schema: - description: AutonomousDatabase is the Schema for the autonomousdatabases API + description: AutonomousContainerDatabase is the Schema for the autonomouscontainerdatabases API properties: apiVersion: description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' @@ -60,109 +48,1469 @@ spec: metadata: type: object spec: - description: 'AutonomousDatabaseSpec defines the desired state of AutonomousDatabase Important: Run "make" to regenerate code after modifying this file' + description: AutonomousContainerDatabaseSpec defines the desired state of AutonomousContainerDatabase properties: - details: - description: AutonomousDatabaseDetails defines the detail information of AutonomousDatabase, corresponding to oci-go-sdk/database/AutonomousDatabase + action: + enum: + - SYNC + - RESTART + - TERMINATE + type: string + autonomousContainerDatabaseOCID: + description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster Important: Run "make" to regenerate code after modifying this file' + type: string + autonomousExadataVMClusterOCID: + type: string + compartmentOCID: + type: string + displayName: + type: string + freeformTags: + additionalProperties: + type: string + type: object + hardLink: + default: false + type: boolean + ociConfig: + description: "*********************** *\tOCI config ***********************" properties: - adminPassword: - properties: - k8sSecretName: - type: string - ociSecretOCID: - type: string - type: object - autonomousDatabaseOCID: - type: string - compartmentOCID: - type: string - cpuCoreCount: - type: integer - dataStorageSizeInTBs: - type: integer - dbName: - type: string - dbVersion: - type: string - dbWorkload: - description: 'AutonomousDatabaseDbWorkloadEnum Enum with underlying type: string' - enum: - - OLTP - - DW - - AJD - - APEX - type: string - displayName: - type: string - freeformTags: - additionalProperties: - type: string - type: object - isAutoScalingEnabled: - type: boolean - isDedicated: - type: boolean - lifecycleState: - description: 'AutonomousDatabaseLifecycleStateEnum Enum with underlying type: string' - type: string - nsgOCIDs: - items: - type: string - type: array - privateEndpoint: + configMapName: type: string - privateEndpointIP: + secretName: type: string - privateEndpointLabel: + type: object + patchModel: + description: 'AutonomousContainerDatabasePatchModelEnum Enum with underlying type: string' + enum: + - RELEASE_UPDATES + - RELEASE_UPDATE_REVISIONS + type: string + type: object + status: + description: AutonomousContainerDatabaseStatus defines the observed state of AutonomousContainerDatabase + properties: + lifecycleState: + description: 'INSERT ADDITIONAL STATUS FIELD - define observed state of cluster Important: Run "make" to regenerate code after modifying this file' + type: string + timeCreated: + type: string + required: + - lifecycleState + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.6.1 + creationTimestamp: null + name: autonomousdatabasebackups.database.oracle.com +spec: + group: database.oracle.com + names: + kind: AutonomousDatabaseBackup + listKind: AutonomousDatabaseBackupList + plural: autonomousdatabasebackups + shortNames: + - adbbu + - adbbus + singular: autonomousdatabasebackup + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .status.lifecycleState + name: State + type: string + - jsonPath: .status.dbDisplayName + name: DB DisplayName + type: string + - jsonPath: .status.type + name: Type + type: string + - jsonPath: .status.timeStarted + name: Started + type: string + - jsonPath: .status.timeEnded + name: Ended + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + description: AutonomousDatabaseBackup is the Schema for the autonomousdatabasebackups API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: AutonomousDatabaseBackupSpec defines the desired state of AutonomousDatabaseBackup + properties: + autonomousDatabaseBackupOCID: + type: string + displayName: + type: string + ociConfig: + description: "*********************** *\tOCI config ***********************" + properties: + configMapName: type: string - subnetOCID: + secretName: type: string - wallet: + type: object + target: + description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster Important: Run "make" to regenerate code after modifying this file' + properties: + k8sADB: + description: "*********************** *\tADB spec ***********************" properties: name: type: string - password: - properties: - k8sSecretName: - type: string - ociSecretOCID: - type: string - type: object + type: object + ociADB: + properties: + ocid: + type: string type: object type: object - hardLink: - default: false + type: object + status: + description: AutonomousDatabaseBackupStatus defines the observed state of AutonomousDatabaseBackup + properties: + autonomousDatabaseBackupOCID: + description: 'INSERT ADDITIONAL STATUS FIELD - define observed state of cluster Important: Run "make" to regenerate code after modifying this file' + type: string + autonomousDatabaseOCID: + type: string + compartmentOCID: + type: string + dbDisplayName: + type: string + dbName: + type: string + displayName: + type: string + isAutomatic: type: boolean + lifecycleState: + type: string + timeEnded: + type: string + timeStarted: + type: string + type: + description: 'AutonomousDatabaseBackupTypeEnum Enum with underlying type: string' + type: string + required: + - autonomousDatabaseOCID + - compartmentOCID + - dbDisplayName + - dbName + - displayName + - isAutomatic + - lifecycleState + - type + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.6.1 + creationTimestamp: null + name: autonomousdatabaserestores.database.oracle.com +spec: + group: database.oracle.com + names: + kind: AutonomousDatabaseRestore + listKind: AutonomousDatabaseRestoreList + plural: autonomousdatabaserestores + shortNames: + - adbr + - adbrs + singular: autonomousdatabaserestore + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .status.status + name: Status + type: string + - jsonPath: .status.displayName + name: DbDisplayName + type: string + - jsonPath: .status.dbName + name: DbName + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + description: AutonomousDatabaseRestore is the Schema for the autonomousdatabaserestores API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: AutonomousDatabaseRestoreSpec defines the desired state of AutonomousDatabaseRestore + properties: ociConfig: + description: "*********************** *\tOCI config ***********************" properties: configMapName: type: string secretName: type: string type: object + source: + properties: + k8sADBBackup: + description: 'EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized.' + properties: + name: + type: string + type: object + pointInTime: + properties: + timestamp: + description: 'The timestamp must follow this format: YYYY-MM-DD HH:MM:SS GMT' + type: string + type: object + type: object + target: + description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster Important: Run "make" to regenerate code after modifying this file' + properties: + k8sADB: + description: "*********************** *\tADB spec ***********************" + properties: + name: + type: string + type: object + ociADB: + properties: + ocid: + type: string + type: object + type: object required: - - details + - source + - target type: object status: - description: AutonomousDatabaseStatus defines the observed state of AutonomousDatabase + description: AutonomousDatabaseRestoreStatus defines the observed state of AutonomousDatabaseRestore properties: - cpuCoreCount: - type: integer - dataStorageSizeInTBs: - type: integer - dbWorkload: - description: 'AutonomousDatabaseDbWorkloadEnum Enum with underlying type: string' + autonomousDatabaseOCID: + type: string + dbName: type: string displayName: description: 'INSERT ADDITIONAL STATUS FIELD - define observed state of cluster Important: Run "make" to regenerate code after modifying this file' type: string - isDedicated: + status: + type: string + timeAccepted: + type: string + timeEnded: + type: string + timeStarted: + type: string + required: + - autonomousDatabaseOCID + - dbName + - displayName + - status + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.6.1 + creationTimestamp: null + name: autonomousdatabases.database.oracle.com +spec: + group: database.oracle.com + names: + kind: AutonomousDatabase + listKind: AutonomousDatabaseList + plural: autonomousdatabases + shortNames: + - adb + - adbs + singular: autonomousdatabase + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .spec.details.displayName + name: Display Name + type: string + - jsonPath: .status.lifecycleState + name: State + type: string + - jsonPath: .spec.details.isDedicated + name: Dedicated + type: string + - jsonPath: .spec.details.cpuCoreCount + name: OCPUs + type: integer + - jsonPath: .spec.details.dataStorageSizeInTBs + name: Storage (TB) + type: integer + - jsonPath: .spec.details.dbWorkload + name: Workload Type + type: string + - jsonPath: .status.timeCreated + name: Created + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + description: AutonomousDatabase is the Schema for the autonomousdatabases API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: 'AutonomousDatabaseSpec defines the desired state of AutonomousDatabase Important: Run "make" to regenerate code after modifying this file' + properties: + details: + description: AutonomousDatabaseDetails defines the detail information of AutonomousDatabase, corresponding to oci-go-sdk/database/AutonomousDatabase + properties: + adminPassword: + properties: + k8sSecret: + description: "*********************** *\tSecret specs ***********************" + properties: + name: + type: string + type: object + ociSecret: + properties: + ocid: + type: string + type: object + type: object + autonomousContainerDatabase: + description: ACDSpec defines the spec of the target for backup/restore runs. The name could be the name of an AutonomousDatabase or an AutonomousDatabaseBackup + properties: + k8sACD: + description: "*********************** *\tACD specs ***********************" + properties: + name: + type: string + type: object + ociACD: + properties: + ocid: + type: string + type: object + type: object + autonomousDatabaseOCID: + type: string + compartmentOCID: + type: string + cpuCoreCount: + type: integer + dataStorageSizeInTBs: + type: integer + dbName: + type: string + dbVersion: + type: string + dbWorkload: + description: 'AutonomousDatabaseDbWorkloadEnum Enum with underlying type: string' + enum: + - OLTP + - DW + - AJD + - APEX + type: string + displayName: + type: string + freeformTags: + additionalProperties: + type: string + type: object + isAutoScalingEnabled: + type: boolean + isDedicated: + type: boolean + licenseModel: + description: 'AutonomousDatabaseLicenseModelEnum Enum with underlying type: string' + enum: + - LICENSE_INCLUDED + - BRING_YOUR_OWN_LICENSE + type: string + lifecycleState: + description: 'AutonomousDatabaseLifecycleStateEnum Enum with underlying type: string' + type: string + networkAccess: + properties: + accessControlList: + items: + type: string + type: array + accessType: + enum: + - "" + - PUBLIC + - RESTRICTED + - PRIVATE + type: string + isAccessControlEnabled: + type: boolean + isMTLSConnectionRequired: + type: boolean + privateEndpoint: + properties: + hostnamePrefix: + type: string + nsgOCIDs: + items: + type: string + type: array + subnetOCID: + type: string + type: object + type: object + wallet: + properties: + name: + type: string + password: + properties: + k8sSecret: + description: "*********************** *\tSecret specs ***********************" + properties: + name: + type: string + type: object + ociSecret: + properties: + ocid: + type: string + type: object + type: object + type: object + type: object + hardLink: + default: false + type: boolean + ociConfig: + description: "*********************** *\tOCI config ***********************" + properties: + configMapName: + type: string + secretName: + type: string + type: object + required: + - details + type: object + status: + description: AutonomousDatabaseStatus defines the observed state of AutonomousDatabase + properties: + allConnectionStrings: + items: + properties: + connectionStrings: + items: + properties: + connectionString: + type: string + tnsName: + type: string + type: object + type: array + tlsAuthentication: + type: string + required: + - connectionStrings + type: object + type: array + lifecycleState: + description: 'INSERT ADDITIONAL STATUS FIELD - define observed state of cluster Important: Run "make" to regenerate code after modifying this file' + type: string + timeCreated: + type: string + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + cert-manager.io/inject-ca-from: oracle-database-operator-system/oracle-database-operator-serving-cert + controller-gen.kubebuilder.io/version: v0.6.1 + name: cdbs.database.oracle.com +spec: + group: database.oracle.com + names: + kind: CDB + listKind: CDBList + plural: cdbs + singular: cdb + scope: Namespaced + versions: + - additionalPrinterColumns: + - description: Name of the CDB + jsonPath: .spec.cdbName + name: CDB Name + type: string + - description: ' Name of the DB Server' + jsonPath: .spec.dbServer + name: DB Server + type: string + - description: DB server port + jsonPath: .spec.dbPort + name: DB Port + type: integer + - description: SCAN Name + jsonPath: .spec.scanName + name: SCAN Name + type: string + - description: Replicas + jsonPath: .spec.replicas + name: Replicas + type: integer + - description: Status of the CDB Resource + jsonPath: .status.phase + name: Status + type: string + - description: Error message, if any + jsonPath: .status.msg + name: Message + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + description: CDB is the Schema for the cdbs API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: CDBSpec defines the desired state of CDB + properties: + cdbAdminPwd: + description: Password for the CDB Administrator to manage PDB lifecycle + properties: + secret: + description: CDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + cdbAdminUser: + description: User in the root container with sysdba priviledges to manage PDB lifecycle + properties: + secret: + description: CDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + cdbName: + description: Name of the CDB + type: string + dbPort: + description: DB server port + type: integer + dbServer: + description: Name of the DB server + type: string + nodeSelector: + additionalProperties: + type: string + description: Node Selector for running the Pod + type: object + ordsImage: + description: ORDS Image Name + type: string + ordsImagePullPolicy: + description: ORDS Image Pull Policy + enum: + - Always + - Never + type: string + ordsImagePullSecret: + description: The name of the image pull secret in case of a private docker repository. + type: string + ordsPort: + description: ORDS server port. For now, keep it as 8888. TO BE USED IN FUTURE RELEASE. + type: integer + ordsPwd: + description: Password for user ORDS_PUBLIC_USER + properties: + secret: + description: CDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + replicas: + description: Number of ORDS Containers to create + type: integer + scanName: + description: SCAN Name + type: string + serviceName: + description: Name of the CDB Service + type: string + sysAdminPwd: + description: Password for the CDB System Administrator + properties: + secret: + description: CDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + webServerPwd: + description: Password for the Web Server User + properties: + secret: + description: CDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + webServerUser: + description: Web Server User with SQL Administrator role to allow us to authenticate to the PDB Lifecycle Management REST endpoints + properties: + secret: + description: CDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + type: object + status: + description: CDBStatus defines the observed state of CDB + properties: + msg: + description: Message + type: string + phase: + description: Phase of the CDB Resource + type: string + status: + description: CDB Resource Status + type: boolean + required: + - phase + - status + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.6.1 + creationTimestamp: null + name: dbcssystems.database.oracle.com +spec: + group: database.oracle.com + names: + kind: DbcsSystem + listKind: DbcsSystemList + plural: dbcssystems + singular: dbcssystem + scope: Namespaced + versions: + - name: v1alpha1 + schema: + openAPIV3Schema: + description: DbcsSystem is the Schema for the dbcssystems API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: DbcsSystemSpec defines the desired state of DbcsSystem + properties: + dbSystem: + properties: + availabilityDomain: + type: string + backupSubnetId: + type: string + clusterName: + type: string + compartmentId: + type: string + cpuCoreCount: + type: integer + dbAdminPaswordSecret: + type: string + dbBackupConfig: + description: DB Backup COnfig Network Struct + properties: + autoBackupEnabled: + type: boolean + autoBackupWindow: + type: string + backupDestinationDetails: + type: string + recoveryWindowsInDays: + type: integer + type: object + dbDomain: + type: string + dbEdition: + type: string + dbName: + type: string + dbUniqueName: + type: string + dbVersion: + type: string + dbWorkload: + type: string + diskRedundancy: + type: string + displayName: + type: string + domain: + type: string + faultDomains: + items: + type: string + type: array + hostName: + type: string + initialDataStorageSizeInGB: + type: integer + kmsKeyId: + type: string + kmsKeyVersionId: + type: string + licenseModel: + type: string + nodeCount: + type: integer + pdbName: + type: string + privateIp: + type: string + shape: + type: string + sshPublicKeys: + items: + type: string + type: array + storageManagement: + type: string + subnetId: + type: string + tags: + additionalProperties: + type: string + type: object + tdeWalletPasswordSecret: + type: string + timeZone: + type: string + required: + - availabilityDomain + - compartmentId + - dbAdminPaswordSecret + - hostName + - shape + - sshPublicKeys + - subnetId + type: object + hardLink: + type: boolean + id: + type: string + ociConfigMap: + type: string + ociSecret: + type: string + required: + - ociConfigMap + type: object + status: + description: DbcsSystemStatus defines the observed state of DbcsSystem + properties: + availabilityDomain: + type: string + cpuCoreCount: + type: integer + dataStoragePercentage: + type: integer + dataStorageSizeInGBs: + type: integer + dbEdition: + type: string + dbInfo: + items: + description: DbcsSystemStatus defines the observed state of DbcsSystem + properties: + dbHomeId: + type: string + dbName: + type: string + dbUniqueName: + type: string + dbWorkload: + type: string + id: + type: string + type: object + type: array + displayName: + type: string + id: + type: string + licenseModel: + type: string + network: + properties: + clientSubnet: + type: string + domainName: + type: string + hostName: + type: string + listenerPort: + type: integer + networkSG: + type: string + scanDnsName: + type: string + vcnName: + type: string + type: object + nodeCount: + type: integer + recoStorageSizeInGB: + type: integer + shape: + type: string + state: + type: string + storageManagement: + type: string + subnetId: + type: string + timeZone: + type: string + workRequests: + items: + properties: + operationId: + type: string + operationType: + type: string + percentComplete: + type: string + timeAccepted: + type: string + timeFinished: + type: string + timeStarted: + type: string + required: + - operationId + - operationType + type: object + type: array + required: + - state + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.6.1 + creationTimestamp: null + name: oraclerestdataservices.database.oracle.com +spec: + group: database.oracle.com + names: + kind: OracleRestDataService + listKind: OracleRestDataServiceList + plural: oraclerestdataservices + singular: oraclerestdataservice + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .status.status + name: Status + type: string + - jsonPath: .spec.databaseRef + name: Database + type: string + - jsonPath: .status.databaseApiUrl + name: 'Database Api & Apex Url ' + type: string + - jsonPath: .status.databaseActionsUrl + name: Database Actions Url + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + description: OracleRestDataService is the Schema for the oraclerestdataservices API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: OracleRestDataServiceSpec defines the desired state of OracleRestDataService + properties: + adminPassword: + description: OracleRestDataServicePassword defines the secret containing Password mapped to secretKey + properties: + keepSecret: + type: boolean + secretKey: + type: string + secretName: + type: string + required: + - keepSecret + - secretKey + type: object + apexPassword: + description: OracleRestDataServicePassword defines the secret containing Password mapped to secretKey + properties: + keepSecret: + type: boolean + secretKey: + type: string + secretName: + type: string + required: + - keepSecret + - secretKey + type: object + databaseRef: + type: string + image: + description: OracleRestDataServiceImage defines the Image source and pullSecrets for POD + properties: + pullFrom: + type: string + pullSecrets: + type: string + version: + type: string + required: + - pullFrom + type: object + loadBalancer: + type: boolean + nodeSelector: + additionalProperties: + type: string + type: object + oracleService: + type: string + ordsPassword: + description: OracleRestDataServicePassword defines the secret containing Password mapped to secretKey + properties: + keepSecret: + type: boolean + secretKey: + type: string + secretName: + type: string + required: + - keepSecret + - secretKey + type: object + ordsUser: + type: string + persistence: + description: OracleRestDataServicePersistence defines the storage releated params + properties: + accessMode: + enum: + - ReadWriteOnce + - ReadWriteMany + type: string + size: + type: string + storageClass: + type: string + type: object + replicas: + minimum: 1 + type: integer + restEnableSchemas: + items: + description: OracleRestDataServicePDBSchemas defines the PDB Schemas to be ORDS Enabled + properties: + enable: + type: boolean + pdb: + type: string + schema: + type: string + urlMapping: + type: string + required: + - enable + - pdb + - schema + type: object + type: array + serviceAccountName: + type: string + required: + - adminPassword + - databaseRef + - ordsPassword + type: object + status: + description: OracleRestDataServiceStatus defines the observed state of OracleRestDataService + properties: + apexConfigured: + type: boolean + clusterDbApiUrl: + type: string + commonUsersCreated: + type: boolean + databaseActionsUrl: + type: string + databaseApiUrl: + type: string + databaseRef: + type: string + image: + description: OracleRestDataServiceImage defines the Image source and pullSecrets for POD + properties: + pullFrom: + type: string + pullSecrets: + type: string + version: + type: string + required: + - pullFrom + type: object + loadBalancer: + type: string + ordsInstalled: + type: boolean + ordsSetupCompleted: + type: boolean + replicas: + type: integer + serviceIP: + type: string + status: + description: 'INSERT ADDITIONAL STATUS FIELD - define observed state of cluster Important: Run "make" to regenerate code after modifying this file' + type: string + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + cert-manager.io/inject-ca-from: oracle-database-operator-system/oracle-database-operator-serving-cert + controller-gen.kubebuilder.io/version: v0.6.1 + name: pdbs.database.oracle.com +spec: + group: database.oracle.com + names: + kind: PDB + listKind: PDBList + plural: pdbs + singular: pdb + scope: Namespaced + versions: + - additionalPrinterColumns: + - description: The connect string to be used + jsonPath: .status.connString + name: Connect String + type: string + - description: Name of the CDB + jsonPath: .spec.cdbName + name: CDB Name + type: string + - description: Name of the PDB + jsonPath: .spec.pdbName + name: PDB Name + type: string + - description: PDB Open Mode + jsonPath: .status.openMode + name: PDB State + type: string + - description: Total Size of the PDB + jsonPath: .status.totalSize + name: PDB Size + type: string + - description: Status of the PDB Resource + jsonPath: .status.phase + name: Status + type: string + - description: Error message, if any + jsonPath: .status.msg + name: Message + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + description: PDB is the Schema for the pdbs API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: PDBSpec defines the desired state of PDB + properties: + action: + description: 'Action to be taken: Create/Clone/Plug/Unplug/Delete/Modify/Status/Map. Map is used to map a Databse PDB to a Kubernetes PDB CR.' + enum: + - Create + - Clone + - Plug + - Unplug + - Delete + - Modify + - Status + - Map + type: string + adminName: + description: The administrator username for the new PDB. This property is required when the Action property is Create. + properties: + secret: + description: PDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + adminPwd: + description: The administrator password for the new PDB. This property is required when the Action property is Create. + properties: + secret: + description: PDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + asClone: + description: Indicate if 'AS CLONE' option should be used in the command to plug in a PDB. This property is applicable when the Action property is PLUG but not required. + type: boolean + cdbName: + description: Name of the CDB + type: string + cdbResName: + description: Name of the CDB Custom Resource that runs the ORDS container + type: string + copyAction: + description: To copy files or not while cloning a PDB + enum: + - COPY + - NOCOPY + - MOVE + type: string + dropAction: + description: Specify if datafiles should be removed or not. The value can be INCLUDING or KEEP (default). + enum: + - INCLUDING + - KEEP + type: string + fileNameConversions: + description: Relevant for Create and Plug operations. As defined in the Oracle Multitenant Database documentation. Values can be a filename convert pattern or NONE. + type: string + getScript: + description: Whether you need the script only or execute the script + type: boolean + modifyOption: + description: Extra options for opening and closing a PDB + enum: + - IMMEDIATE + - NORMAL + - READ ONLY + - READ WRITE + - RESTRICTED + type: string + pdbName: + description: The name of the new PDB. Relevant for both Create and Plug Actions. + type: string + pdbState: + description: The target state of the PDB + enum: + - OPEN + - CLOSE + type: string + reuseTempFile: + description: Whether to reuse temp file + type: boolean + sourceFileNameConversions: + description: This property is required when the Action property is Plug. As defined in the Oracle Multitenant Database documentation. Values can be a source filename convert pattern or NONE. + type: string + sparseClonePath: + description: A Path specified for sparse clone snapshot copy. (Optional) + type: string + srcPdbName: + description: Name of the Source PDB from which to clone + type: string + tdeExport: + description: TDE export for unplug operations + type: boolean + tdeImport: + description: TDE import for plug operations + type: boolean + tdeKeystorePath: + description: TDE keystore path is required if the tdeImport or tdeExport flag is set to true. Can be used in plug or unplug operations. + type: string + tdePassword: + description: TDE password if the tdeImport or tdeExport flag is set to true. Can be used in create, plug or unplug operations + properties: + secret: + description: PDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + tdeSecret: + description: TDE secret is required if the tdeImport or tdeExport flag is set to true. Can be used in plug or unplug operations. + properties: + secret: + description: PDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + tempSize: + description: Relevant for Create and Clone operations. Total size for temporary tablespace as defined in the Oracle Multitenant Database documentation. See size_clause description in Database SQL Language Reference documentation. + type: string + totalSize: + description: Relevant for create and plug operations. Total size as defined in the Oracle Multitenant Database documentation. See size_clause description in Database SQL Language Reference documentation. + type: string + unlimitedStorage: + description: Relevant for Create and Plug operations. True for unlimited storage. Even when set to true, totalSize and tempSize MUST be specified in the request if Action is Create. + type: boolean + xmlFileName: + description: XML metadata filename to be used for Plug or Unplug operations + type: string + required: + - action + type: object + status: + description: PDBStatus defines the observed state of PDB + properties: + action: + description: Last Completed Action + type: string + connString: + description: PDB Connect String + type: string + modifyOption: + description: Modify Option of the PDB + type: string + msg: + description: Message + type: string + openMode: + description: Open mode of the PDB type: string - lifecycleState: - description: 'AutonomousDatabaseLifecycleStateEnum Enum with underlying type: string' + phase: + description: Phase of the PDB Resource type: string - timeCreated: + status: + description: PDB Resource Status + type: boolean + totalSize: + description: Total size of the PDB type: string + required: + - phase + - status type: object type: object served: true @@ -642,6 +1990,7 @@ spec: enum: - standard - enterprise + - express type: string flashBack: type: boolean @@ -685,33 +2034,27 @@ spec: description: SingleInstanceDatabasePersistence defines the storage size and class for PVC properties: accessMode: - enum: - - ReadWriteOnce - - ReadWriteMany type: string size: type: string storageClass: type: string required: - - accessMode - size - storageClass type: object readinessCheckPeriod: type: integer replicas: - minimum: 1 type: integer + serviceAccountName: + type: string sid: description: SID can only have a-z , A-Z, 0-9 . It cant have any special characters pattern: ^[a-zA-Z0-9]+$ type: string required: - - adminPassword - image - - persistence - - replicas type: object status: description: SingleInstanceDatabaseStatus defines the observed state of SingleInstanceDatabase @@ -816,16 +2159,12 @@ spec: description: SingleInstanceDatabasePersistence defines the storage size and class for PVC properties: accessMode: - enum: - - ReadWriteOnce - - ReadWriteMany type: string size: type: string storageClass: type: string required: - - accessMode - size - storageClass type: object @@ -915,10 +2254,263 @@ metadata: name: oracle-database-operator-manager-role rules: - apiGroups: - - "" + - "" + resources: + - configmaps + - events + - pods + - pods/exec + - pods/log + - replicasets + - services + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - "" + resources: + - configmaps + - namespaces + - secrets + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - "" + resources: + - configmaps + - secrets + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - "" + resources: + - events + - nodes + - persistentvolumeclaims + - pods + - pods/exec + - pods/log + - services + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - '''''' + resources: + - statefulsets/finalizers + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - apps + resources: + - replicasets + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - apps + resources: + - statefulsets + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - coordination.k8s.io + resources: + - leases + verbs: + - create + - get + - list + - update +- apiGroups: + - "" + resources: + - configmaps + - events + - namespaces + - nodes + - persistentvolumeclaims + - pods + - pods/exec + - pods/log + - secrets + - services + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - "" + resources: + - configmaps + - namespaces + - pods + - secrets + - services + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - "" + resources: + - pods/exec + verbs: + - create +- apiGroups: + - database.oracle.com + resources: + - autonomouscontainerdatabases + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - database.oracle.com + resources: + - autonomouscontainerdatabases/status + verbs: + - get + - patch + - update +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabasebackups + verbs: + - create + - delete + - get + - list + - update + - watch +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabasebackups/status + verbs: + - get + - patch + - update +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabaserestores + verbs: + - create + - delete + - get + - list + - update + - watch +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabaserestores/status + verbs: + - get + - patch + - update +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabases + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - database.oracle.com + resources: + - autonomousdatabases/status + verbs: + - patch + - update +- apiGroups: + - database.oracle.com + resources: + - cdbs + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - database.oracle.com + resources: + - cdbs/finalizers + verbs: + - update +- apiGroups: + - database.oracle.com + resources: + - cdbs/status + verbs: + - get + - patch + - update +- apiGroups: + - database.oracle.com resources: - - configmaps - - secrets + - dbcssystems verbs: - create - delete @@ -928,39 +2520,27 @@ rules: - update - watch - apiGroups: - - "" + - database.oracle.com resources: - - events - - nodes - - persistentvolumeclaims - - pods - - pods/exec - - pods/log - - services + - dbcssystems/finalizers verbs: - create - delete - get - - list - patch - update - - watch - apiGroups: - - '''''' + - database.oracle.com resources: - - statefulsets/finalizers + - dbcssystems/status verbs: - - create - - delete - get - - list - patch - update - - watch - apiGroups: - - apps + - database.oracle.com resources: - - statefulsets + - oraclerestdataservices verbs: - create - delete @@ -970,27 +2550,23 @@ rules: - update - watch - apiGroups: - - coordination.k8s.io + - database.oracle.com resources: - - leases + - oraclerestdataservices/finalizers + verbs: + - update +- apiGroups: + - database.oracle.com + resources: + - oraclerestdataservices/status verbs: - - create - get - - list + - patch - update - apiGroups: - - "" + - database.oracle.com resources: - - configmaps - - events - - namespaces - - nodes - - persistentvolumeclaims - - pods - - pods/exec - - pods/log - - secrets - - services + - pdbs verbs: - create - delete @@ -999,29 +2575,22 @@ rules: - patch - update - watch -- apiGroups: - - "" - resources: - - pods/exec - verbs: - - create - apiGroups: - database.oracle.com resources: - - autonomousdatabases + - pdbs/finalizers verbs: - create - delete - get - - list - patch - update - - watch - apiGroups: - database.oracle.com resources: - - autonomousdatabases/status + - pdbs/status verbs: + - get - patch - update - apiGroups: @@ -1199,7 +2768,7 @@ spec: - --enable-leader-election command: - /manager - image: container-registry.oracle.com/database/operator:0.1.0 + image: phx.ocir.io/intsanjaysingh/oracle-database-operator:0.0.1 imagePullPolicy: Always name: manager ports: @@ -1217,8 +2786,6 @@ spec: - mountPath: /tmp/k8s-webhook-server/serving-certs name: cert readOnly: true - imagePullSecrets: - - name: container-registry-secret terminationGracePeriodSeconds: 10 volumes: - name: cert @@ -1255,6 +2822,109 @@ metadata: cert-manager.io/inject-ca-from: oracle-database-operator-system/oracle-database-operator-serving-cert name: oracle-database-operator-mutating-webhook-configuration webhooks: +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /mutate-database-oracle-com-v1alpha1-autonomousdatabase + failurePolicy: Fail + name: mautonomousdatabase.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + resources: + - autonomousdatabases + sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /mutate-database-oracle-com-v1alpha1-autonomousdatabasebackup + failurePolicy: Fail + name: mautonomousdatabasebackup.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + resources: + - autonomousdatabasebackups + sideEffects: None +- admissionReviewVersions: + - v1 + - v1beta1 + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /mutate-database-oracle-com-v1alpha1-cdb + failurePolicy: Fail + name: mcdb.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + resources: + - cdbs + sideEffects: None +- admissionReviewVersions: + - v1 + - v1beta1 + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /mutate-database-oracle-com-v1alpha1-oraclerestdataservice + failurePolicy: Fail + name: moraclerestdataservice.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + resources: + - oraclerestdataservices + sideEffects: None +- admissionReviewVersions: + - v1 + - v1beta1 + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /mutate-database-oracle-com-v1alpha1-pdb + failurePolicy: Fail + name: mpdb.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + resources: + - pdbs + sideEffects: None - admissionReviewVersions: - v1 - v1beta1 @@ -1284,6 +2954,130 @@ metadata: cert-manager.io/inject-ca-from: oracle-database-operator-system/oracle-database-operator-serving-cert name: oracle-database-operator-validating-webhook-configuration webhooks: +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /validate-database-oracle-com-v1alpha1-autonomousdatabase + failurePolicy: Fail + name: vautonomousdatabase.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + - DELETE + resources: + - autonomousdatabases + sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /validate-database-oracle-com-v1alpha1-autonomousdatabasebackup + failurePolicy: Fail + name: vautonomousdatabasebackup.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + resources: + - autonomousdatabasebackups + sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /validate-database-oracle-com-v1alpha1-autonomousdatabaserestore + failurePolicy: Fail + name: vautonomousdatabaserestore.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + resources: + - autonomousdatabaserestores + sideEffects: None +- admissionReviewVersions: + - v1 + - v1beta1 + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /validate-database-oracle-com-v1alpha1-cdb + failurePolicy: Fail + name: vcdb.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + resources: + - cdbs + sideEffects: None +- admissionReviewVersions: + - v1 + - v1beta1 + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /validate-database-oracle-com-v1alpha1-oraclerestdataservice + failurePolicy: Fail + name: voraclerestdataservice.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + resources: + - oraclerestdataservices + sideEffects: None +- admissionReviewVersions: + - v1 + - v1beta1 + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /validate-database-oracle-com-v1alpha1-pdb + failurePolicy: Fail + name: vpdb.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + resources: + - pdbs + sideEffects: None - admissionReviewVersions: - v1 - v1beta1 diff --git a/test/e2e/autonomousdatabase_controller_bind_test.go b/test/e2e/autonomousdatabase_controller_bind_test.go index 5608fc91..6be0c4e5 100644 --- a/test/e2e/autonomousdatabase_controller_bind_test.go +++ b/test/e2e/autonomousdatabase_controller_bind_test.go @@ -229,4 +229,4 @@ var _ = Describe("test ADB binding with hardLink=true", func() { It("Should delete local resource", e2ebehavior.AssertSoftLinkDelete(&k8sClient, &adbLookupKey)) }) -}) \ No newline at end of file +}) From c2a98d55f721c0f21df9ac514e57648a2d999f8d Mon Sep 17 00:00:00 2001 From: "paramdeep.saini@oracle.com" Date: Wed, 6 Apr 2022 15:04:10 -0700 Subject: [PATCH 225/628] Added DBCS Controller --- config/manager/kustomization.yaml | 4 ++-- oracle-database-operator.yaml | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/config/manager/kustomization.yaml b/config/manager/kustomization.yaml index 6de6f53f..30ed1f75 100644 --- a/config/manager/kustomization.yaml +++ b/config/manager/kustomization.yaml @@ -8,5 +8,5 @@ apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization images: - name: controller - newName: phx.ocir.io/intsanjaysingh/oracle-database-operator - newTag: 0.0.1 + newName: container-registry.oracle.com/database/operator + newTag: 0.1.0 diff --git a/oracle-database-operator.yaml b/oracle-database-operator.yaml index b071a47f..267df8d3 100644 --- a/oracle-database-operator.yaml +++ b/oracle-database-operator.yaml @@ -2768,7 +2768,7 @@ spec: - --enable-leader-election command: - /manager - image: phx.ocir.io/intsanjaysingh/oracle-database-operator:0.0.1 + image: container-registry.oracle.com/database/operator:0.1.0 imagePullPolicy: Always name: manager ports: From 5fd8a196ac1936daac66d1e5b2672f4ceddf1729 Mon Sep 17 00:00:00 2001 From: "paramdeep.saini@oracle.com" Date: Wed, 6 Apr 2022 15:05:33 -0700 Subject: [PATCH 226/628] Added DBCS Controller --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index eb4ef3fa..0b6433f4 100644 --- a/README.md +++ b/README.md @@ -85,7 +85,7 @@ The quickstarts are designed for specific database configurations, including: * [Oracle Autonomous Database](./docs/adb/README.md) * [Oracle Database Single Instance configuration](./docs/sidb/README.md) * [Oracle Database configured with Oracle Sharding](./docs/sharding/README.md) -* [Oracle Database Cloud Services](.docs/dbcs/README.md) +* [Oracle Database Cloud Services](./docs/dbcs/README.md) YAML file templates are available under [`/config/samples`](./config/samples/). You can copy and edit these template files to configure them for your use cases. From 8fd0117a52afc8de1a143a75a862520676bc0f5e Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Mon, 11 Apr 2022 10:57:41 +0530 Subject: [PATCH 227/628] Installing apex from ORDS pod, using the image which has apex in it Signed-off-by: abhisbyk --- commons/database/constants.go | 7 +- .../oraclerestdataservice_controller.go | 157 +++++++++++++++--- .../singleinstancedatabase_controller.go | 24 +-- 3 files changed, 156 insertions(+), 32 deletions(-) diff --git a/commons/database/constants.go b/commons/database/constants.go index db061ad3..102a9046 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -46,6 +46,8 @@ const DBA_GUID int64 = 54322 const SQLPlusCLI string = "sqlplus -s / as sysdba" +const SQLPlusRemoteCLI string = "sqlplus -s sys/%[1]s@%[2]s as sysdba" + const NoCloneRef string = "Unavailable" const GetVersionSQL string = "SELECT VERSION_FULL FROM V\\$INSTANCE;" @@ -366,12 +368,15 @@ const ChownApex string = " chown oracle:oinstall /opt/oracle/oradata/${ORACLE_SI const InstallApex string = "if [ -f /opt/oracle/oradata/${ORACLE_SID^^}/apex/apexins.sql ]; then ( while true; do sleep 60; echo \"Installing Apex...\" ; done ) & " + " cd /opt/oracle/oradata/${ORACLE_SID^^}/apex && echo -e \"@apexins.sql SYSAUX SYSAUX TEMP /i/\" | %[1]s && kill -9 $!; else echo \"Apex Folder doesn't exist\" ; fi ;" +const InstallApexRemote string = "if [ -e ${ORDS_HOME}/config/apex/apexins.sql ]; ( while true; do sleep 60; echo \"Installing Apex...\" ; done ) & " + + " cd ${ORDS_HOME}/config/apex/ && echo -e \"@apexins.sql SYSAUX SYSAUX TEMP /i/\" | %[1]s && kill -9 $!; else echo \"Apex Folder doesn't exist\" ; fi ;" + const IsApexInstalled string = "select 'APEXVERSION:'||version as version FROM DBA_REGISTRY WHERE COMP_ID='APEX';" const UninstallApex string = "if [ -f /opt/oracle/oradata/${ORACLE_SID^^}/apex/apxremov.sql ]; then ( while true; do sleep 60; echo \"Uninstalling Apex...\" ; done ) & " + " cd /opt/oracle/oradata/${ORACLE_SID^^}/apex && echo -e \"@apxremov.sql\" | %[1]s && kill -9 $!; else echo \"Apex Folder doesn't exist\" ; fi ;" -const ConfigureApexRest string = "if [ -f /opt/oracle/oradata/${ORACLE_SID^^}/apex/apex_rest_config.sql ]; then cd /opt/oracle/oradata/${ORACLE_SID^^}/apex && " + +const ConfigureApexRest string = "if [ -f ${ORDS_HOME}/config/apex/apex_rest_config.sql ]; then cd ${ORDS_HOME}/config/apex && " + "echo -e \"%[1]s\n%[1]s\" | %[2]s ; else echo \"Apex Folder doesn't exist\" ; fi ;" const AlterApexUsers string = "echo -e \" ALTER USER APEX_PUBLIC_USER IDENTIFIED BY \\\"%[1]s\\\" ACCOUNT UNLOCK; \n ALTER USER APEX_REST_PUBLIC_USER IDENTIFIED BY \\\"%[1]s\\\" ACCOUNT UNLOCK;" + diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 856f0a19..9e678be0 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -115,7 +115,7 @@ func (r *OracleRestDataServiceReconciler) Reconcile(ctx context.Context, req ctr return requeueN, err } - // Manage SingleInstanceDatabase Deletion + // Manage OracleRestDataService Deletion result := r.manageOracleRestDataServiceDeletion(req, ctx, oracleRestDataService, singleInstanceDatabase) if result.Requeue { r.Log.Info("Reconcile queued") @@ -164,8 +164,15 @@ func (r *OracleRestDataServiceReconciler) Reconcile(ctx context.Context, req ctr return result, nil } + // Validate ORDS pod readiness + result, ordsReadyPod := r.validateORDSReadiness(oracleRestDataService, ctx, req) + if result.Requeue { + r.Log.Info("Reconcile queued") + return result, nil + } + // Configure Apex - result = r.configureApex(oracleRestDataService, singleInstanceDatabase, sidbReadyPod, ctx, req) + result = r.configureApex(oracleRestDataService, singleInstanceDatabase, ordsReadyPod, ctx, req) if result.Requeue { r.Log.Info("Reconcile queued") return result, nil @@ -211,9 +218,10 @@ func (r *OracleRestDataServiceReconciler) validate(m *dbapi.OracleRestDataServic if m.Status.LoadBalancer != "" && m.Status.LoadBalancer != strconv.FormatBool(m.Spec.LoadBalancer) { eventMsgs = append(eventMsgs, "service patching is not avaiable currently") } - if !n.Status.ApexInstalled && m.Spec.ApexPassword.SecretName != "" { + //Not needed as apex will be installed from ORDS now + /* if !n.Status.ApexInstalled && m.Spec.ApexPassword.SecretName != "" { eventMsgs = append(eventMsgs, "apex is not installed yet") - } + } */ if m.Status.Image.PullFrom != "" && m.Status.Image != m.Spec.Image { eventMsgs = append(eventMsgs, "image patching is not avaiable currently") } @@ -296,6 +304,33 @@ func (r *OracleRestDataServiceReconciler) validateSidbReadiness(m *dbapi.OracleR return requeueN, sidbReadyPod } +//##################################################################################################### +// Validate Readiness of the ORDS pods +//##################################################################################################### +func (r *OracleRestDataServiceReconciler) validateORDSReadiness(m *dbapi.OracleRestDataService, ctx context.Context, req ctrl.Request) (ctrl.Result, corev1.Pod) { + log := r.Log.WithValues("validateORDSReadiness", req.NamespacedName) + + // ## FETCH THE ORDS REPLICAS . + ordsReadyPod, _, _, _, err := dbcommons.FindPods(r, m.Spec.Image.Version, + m.Spec.Image.PullFrom, m.Name, m.Namespace, ctx, req) + if err != nil { + log.Error(err, err.Error()) + return requeueY, ordsReadyPod + } + + if ordsReadyPod.Name == "" || m.Status.Status != dbcommons.StatusReady { + eventReason := "Waiting" + eventMsg := "waiting for " + m.Name + " to be Ready" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + // Logging the event messages + log.Info(eventMsg) + return requeueY, ordsReadyPod + } + + return requeueN, ordsReadyPod + +} + //##################################################################################################### // Check ORDS Health Status //##################################################################################################### @@ -907,7 +942,7 @@ func (r *OracleRestDataServiceReconciler) cleanupOracleRestDataService(req ctrl. uninstallORDS := fmt.Sprintf(dbcommons.UninstallORDSCMD, adminPassword) out, err = dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, true, "bash", "-c", - uninstallORDS) + uninstallORDS) log.Info("UninstallORDSCMD Output : " + out) if strings.Contains(strings.ToUpper(out), "ERROR") { return errors.New(out) @@ -949,7 +984,7 @@ func (r *OracleRestDataServiceReconciler) cleanupOracleRestDataService(req ctrl. // Configure APEX //############################################################################# func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase, - sidbReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) ctrl.Result { + ordsReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) ctrl.Result { log := r.Log.WithValues("configureApex", req.NamespacedName) if m.Spec.ApexPassword.SecretName == "" { @@ -959,15 +994,34 @@ func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataS if m.Status.ApexConfigured { return requeueN } - if !n.Status.ApexInstalled { - m.Status.Status = dbcommons.StatusError - eventReason := "Failed" - eventMsg := "apex is not installed yet on " + m.Spec.DatabaseRef - r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) + + // Obtain admin password of the referred database + adminPasswordSecret := &corev1.Secret{} + err := r.Get(ctx, types.NamespacedName{Name: m.Spec.AdminPassword.SecretName, Namespace: m.Namespace}, adminPasswordSecret) + if err != nil { + if apierrors.IsNotFound(err) { + m.Status.Status = dbcommons.StatusError + eventReason := "Waiting" + eventMsg := "waiting for secret : " + m.Spec.AdminPassword.SecretName + " to get created" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + r.Log.Info("Secret " + m.Spec.AdminPassword.SecretName + " Not Found") + return requeueY + } + log.Error(err, err.Error()) return requeueY } + sidbPassword := string(adminPasswordSecret.Data[m.Spec.AdminPassword.SecretKey]) + log.Info("SIDB Password: " + sidbPassword) - // If Seperate PV for ORDS, we need to unzip apex-latest.zip + if !n.Status.ApexInstalled { + result := r.installApexSIDB(m, n, ordsReadyPod, sidbPassword, ctx, req) + if result.Requeue { + log.Info("Reconcile requeued because apex installation failed") + return result + } + } + + /* // If Seperate PV for ORDS, we need to unzip apex-latest.zip if m.Spec.Persistence.AccessMode != "" { ordsReadyPod, _, _, _, err := dbcommons.FindPods(r, m.Spec.Image.Version, m.Spec.Image.PullFrom, m.Name, m.Namespace, ctx, req) @@ -1014,10 +1068,10 @@ func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataS } log.Info(" CopyApexImages Output : \n" + out) - } + } */ apexPasswordSecret := &corev1.Secret{} - err := r.Get(ctx, types.NamespacedName{Name: m.Spec.ApexPassword.SecretName, Namespace: m.Namespace}, apexPasswordSecret) + err = r.Get(ctx, types.NamespacedName{Name: m.Spec.ApexPassword.SecretName, Namespace: m.Namespace}, apexPasswordSecret) if err != nil { if apierrors.IsNotFound(err) { m.Status.Status = dbcommons.StatusError @@ -1037,8 +1091,9 @@ func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataS r.Status().Update(ctx, m) configureApexRestSqlClient := "sqlplus -s / as sysdba @apex_rest_config.sql" + configureApexRestSqlClient = fmt.Sprintf(dbcommons.SQLPlusRemoteCLI, sidbPassword, n.Status.ConnectString) + " @apex_rest_config.sql" // Configure APEX - out, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", + out, err := dbcommons.ExecCommand(r, r.Config, ordsReadyPod.Name, ordsReadyPod.Namespace, "", ctx, req, true, "bash", "-c", fmt.Sprintf(dbcommons.ConfigureApexRest, apexPassword, configureApexRestSqlClient)) if err != nil { log.Info(err.Error()) @@ -1056,8 +1111,8 @@ func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataS } // Alter APEX_LISTENER,APEX_PUBLIC_USER,APEX_REST_PUBLIC_USER - out, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, - "bash", "-c", fmt.Sprintf(dbcommons.AlterApexUsers, apexPassword, dbcommons.SQLPlusCLI)) + out, err = dbcommons.ExecCommand(r, r.Config, ordsReadyPod.Name, ordsReadyPod.Namespace, "", ctx, req, true, + "bash", "-c", fmt.Sprintf(dbcommons.AlterApexUsers, apexPassword, fmt.Sprintf(dbcommons.SQLPlusRemoteCLI, sidbPassword, n.Status.ConnectString))) if err != nil { log.Info(err.Error()) return requeueY @@ -1070,8 +1125,8 @@ func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataS } // Change APEX Admin Password - out, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, - "bash", "-c", fmt.Sprintf("echo -e \"%s\" | %s", fmt.Sprintf(dbcommons.ApexAdmin, apexPassword, pdbName), dbcommons.SQLPlusCLI)) + out, err = dbcommons.ExecCommand(r, r.Config, ordsReadyPod.Name, ordsReadyPod.Namespace, "", ctx, req, true, + "bash", "-c", fmt.Sprintf("echo -e \"%s\" | %s", fmt.Sprintf(dbcommons.ApexAdmin, apexPassword, pdbName), fmt.Sprintf(dbcommons.SQLPlusRemoteCLI, sidbPassword, n.Status.ConnectString))) if err != nil { log.Info(err.Error()) return requeueY @@ -1121,6 +1176,70 @@ func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataS return requeueY } + log.Info("ConfigureApex Successful !") + return requeueN +} + +//############################################################################# +// Install APEX in SIDB +//############################################################################# +func (r *OracleRestDataServiceReconciler) installApexSIDB(m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase, + ordsReadyPod corev1.Pod, sidbPassword string, ctx context.Context, req ctrl.Request) ctrl.Result { + log := r.Log.WithValues("installApex", req.NamespacedName) + + // Initial Validation + if n.Status.ApexInstalled { + return requeueN + } + + // Status Updation + m.Status.Status = dbcommons.StatusUpdating + r.Status().Update(ctx, m) + eventReason := "Installing Apex" + eventMsg := "Waiting for Apex Installation to complete in SIDB pod" + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) + + //Install Apex in SIDB ready pod + out, err := dbcommons.ExecCommand(r, r.Config, ordsReadyPod.Name, ordsReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf(dbcommons.InstallApexRemote, fmt.Sprintf(dbcommons.SQLPlusRemoteCLI, sidbPassword, n.Status.ConnectString))) + if err != nil { + log.Info(err.Error()) + } + log.Info(" InstallApex Output : \n" + out) + + if strings.Contains(out, "Apex Folder doesn't exist") { + eventReason := "Waiting" + eventMsg := "apex Folder doesn't exist in the location /opt/oracle/ords/config" + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) + return requeueY + } + + // Checking if Apex is installed successfully or not + out, err = dbcommons.ExecCommand(r, r.Config, ordsReadyPod.Name, ordsReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | %s", dbcommons.IsApexInstalled, fmt.Sprintf(dbcommons.SQLPlusRemoteCLI, sidbPassword, n.Status.ConnectString))) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + log.Info("IsApexInstalled Output: \n" + out) + + apexInstalled := "APEXVERSION:" + if !strings.Contains(out, apexInstalled) { + return requeueY + } + + n.Status.ApexInstalled = true + // Make sure m.Status.ApexInstalled is set to true . + for i := 0; i < 10; i++ { + err = r.Status().Update(ctx, n) + if err != nil { + log.Info(err.Error() + "\n updating m.Status.ApexInstalled = true") + time.Sleep(5 * time.Second) + continue + } + break + } + return requeueN } diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 8dbe8366..2ee0f00d 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -456,13 +456,13 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns Volumes: []corev1.Volume{{ Name: "oracle-pwd-vol", VolumeSource: corev1.VolumeSource{ - Secret: &corev1.SecretVolumeSource { + Secret: &corev1.SecretVolumeSource{ SecretName: m.Spec.AdminPassword.SecretName, - Optional: func() *bool { i := true; return &i }(), - Items: []corev1.KeyToPath {{ - Key: m.Spec.AdminPassword.SecretKey, - Path: "oracle_pwd", - }, + Optional: func() *bool { i := true; return &i }(), + Items: []corev1.KeyToPath{{ + Key: m.Spec.AdminPassword.SecretKey, + Path: "oracle_pwd", + }, }, }, }, @@ -495,7 +495,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns return 30 }(), }, - VolumeMounts: []corev1.VolumeMount { { + VolumeMounts: []corev1.VolumeMount{{ MountPath: "/run/secrets/oracle_pwd", ReadOnly: true, Name: "oracle-pwd-vol", @@ -583,11 +583,11 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, { Name: "oracle-pwd-vol", VolumeSource: corev1.VolumeSource{ - Secret: &corev1.SecretVolumeSource { + Secret: &corev1.SecretVolumeSource{ SecretName: m.Spec.AdminPassword.SecretName, - Optional: func() *bool { i := (m.Spec.Edition != "express"); return &i }(), - Items: []corev1.KeyToPath {{ - Key: m.Spec.AdminPassword.SecretKey, + Optional: func() *bool { i := (m.Spec.Edition != "express"); return &i }(), + Items: []corev1.KeyToPath{{ + Key: m.Spec.AdminPassword.SecretKey, Path: "oracle_pwd", }}, }, @@ -1500,7 +1500,6 @@ func (r *SingleInstanceDatabaseReconciler) validateDBReadiness(m *dbapi.SingleIn } } - return requeueN, readyPod, nil } @@ -1952,6 +1951,7 @@ func (r *SingleInstanceDatabaseReconciler) installApex(m *dbapi.SingleInstanceDa func (r *SingleInstanceDatabaseReconciler) uninstallApex(m *dbapi.SingleInstanceDatabase, readyPod corev1.Pod, ctx context.Context, req ctrl.Request) ctrl.Result { log := r.Log.WithValues("uninstallApex", req.NamespacedName) + return requeueN // No APEX for Pre-built db if m.Spec.Persistence.AccessMode == "" { From 97533f38f3d157cafb32e08b5c1dee934f4a2a9d Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Mon, 11 Apr 2022 11:01:36 +0530 Subject: [PATCH 228/628] Comment uninstall apex from sidb controller Signed-off-by: abhisbyk --- controllers/database/singleinstancedatabase_controller.go | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 2ee0f00d..b068d52e 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -222,12 +222,12 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct return result, nil } - // Uninstall Apex + /* // Uninstall Apex result = r.uninstallApex(singleInstanceDatabase, readyPod, ctx, req) if result.Requeue { r.Log.Info("Reconcile queued") return result, nil - } + } */ // If LoadBalancer = true , ensure Connect String is updated if singleInstanceDatabase.Status.ConnectString == dbcommons.ValueUnavailable { @@ -1945,7 +1945,7 @@ func (r *SingleInstanceDatabaseReconciler) installApex(m *dbapi.SingleInstanceDa return requeueN } -//############################################################################# +/* //############################################################################# // Uninstall APEX from CDB //############################################################################# func (r *SingleInstanceDatabaseReconciler) uninstallApex(m *dbapi.SingleInstanceDatabase, @@ -1999,7 +1999,7 @@ func (r *SingleInstanceDatabaseReconciler) uninstallApex(m *dbapi.SingleInstance } r.updateORDSStatus(m, true, ctx, req) return requeueN -} +} */ //############################################################################# // Update ORDS Status From 9cf9c4d4230a7d96dc15a0c5310ccb13f1d8a3e0 Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Mon, 11 Apr 2022 17:56:34 +0530 Subject: [PATCH 229/628] Removing status check from validateORDSReadiness Signed-off-by: abhisbyk --- controllers/database/oraclerestdataservice_controller.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 9e678be0..78297e87 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -318,7 +318,7 @@ func (r *OracleRestDataServiceReconciler) validateORDSReadiness(m *dbapi.OracleR return requeueY, ordsReadyPod } - if ordsReadyPod.Name == "" || m.Status.Status != dbcommons.StatusReady { + if ordsReadyPod.Name == "" { eventReason := "Waiting" eventMsg := "waiting for " + m.Name + " to be Ready" r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) From 7d3b854b8cd7dbcf50eddfe7c7b2e9bf2475c0b8 Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Mon, 11 Apr 2022 18:24:22 +0530 Subject: [PATCH 230/628] Correcting InstallApexRemote command Signed-off-by: abhisbyk --- commons/database/constants.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/commons/database/constants.go b/commons/database/constants.go index 102a9046..87961dd4 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -368,7 +368,7 @@ const ChownApex string = " chown oracle:oinstall /opt/oracle/oradata/${ORACLE_SI const InstallApex string = "if [ -f /opt/oracle/oradata/${ORACLE_SID^^}/apex/apexins.sql ]; then ( while true; do sleep 60; echo \"Installing Apex...\" ; done ) & " + " cd /opt/oracle/oradata/${ORACLE_SID^^}/apex && echo -e \"@apexins.sql SYSAUX SYSAUX TEMP /i/\" | %[1]s && kill -9 $!; else echo \"Apex Folder doesn't exist\" ; fi ;" -const InstallApexRemote string = "if [ -e ${ORDS_HOME}/config/apex/apexins.sql ]; ( while true; do sleep 60; echo \"Installing Apex...\" ; done ) & " + +const InstallApexRemote string = "if [ -e ${ORDS_HOME}/config/apex/apexins.sql ]; then ( while true; do sleep 60; echo \"Installing Apex...\" ; done ) & " + " cd ${ORDS_HOME}/config/apex/ && echo -e \"@apexins.sql SYSAUX SYSAUX TEMP /i/\" | %[1]s && kill -9 $!; else echo \"Apex Folder doesn't exist\" ; fi ;" const IsApexInstalled string = "select 'APEXVERSION:'||version as version FROM DBA_REGISTRY WHERE COMP_ID='APEX';" From 8a445d1e40da070110b0da37949ad4a4522d1045 Mon Sep 17 00:00:00 2001 From: Gajanan Bhat Date: Tue, 12 Apr 2022 22:11:25 +0530 Subject: [PATCH 231/628] Added CDB secret changes --- controllers/database/cdb_controller.go | 36 ++++++++++++++++ oracle-database-operator.yaml | 58 +++++++++++++------------- 2 files changed, 65 insertions(+), 29 deletions(-) diff --git a/controllers/database/cdb_controller.go b/controllers/database/cdb_controller.go index 11bc68ca..c2a3d967 100644 --- a/controllers/database/cdb_controller.go +++ b/controllers/database/cdb_controller.go @@ -327,6 +327,12 @@ func (r *CDBReconciler) createPodSpec(cdb *dbapi.CDB) corev1.PodSpec { LocalObjectReference: corev1.LocalObjectReference{ Name: cdb.Spec.SysAdminPwd.Secret.SecretName, }, + Items: []corev1.KeyToPath{ + { + Key: cdb.Spec.SysAdminPwd.Secret.Key, + Path: cdb.Spec.SysAdminPwd.Secret.Key, + }, + }, }, }, { @@ -334,6 +340,12 @@ func (r *CDBReconciler) createPodSpec(cdb *dbapi.CDB) corev1.PodSpec { LocalObjectReference: corev1.LocalObjectReference{ Name: cdb.Spec.CDBAdminUser.Secret.SecretName, }, + Items: []corev1.KeyToPath{ + { + Key: cdb.Spec.CDBAdminUser.Secret.Key, + Path: cdb.Spec.CDBAdminUser.Secret.Key, + }, + }, }, }, { @@ -341,6 +353,12 @@ func (r *CDBReconciler) createPodSpec(cdb *dbapi.CDB) corev1.PodSpec { LocalObjectReference: corev1.LocalObjectReference{ Name: cdb.Spec.CDBAdminPwd.Secret.SecretName, }, + Items: []corev1.KeyToPath{ + { + Key: cdb.Spec.CDBAdminPwd.Secret.Key, + Path: cdb.Spec.CDBAdminPwd.Secret.Key, + }, + }, }, }, { @@ -348,6 +366,12 @@ func (r *CDBReconciler) createPodSpec(cdb *dbapi.CDB) corev1.PodSpec { LocalObjectReference: corev1.LocalObjectReference{ Name: cdb.Spec.ORDSPwd.Secret.SecretName, }, + Items: []corev1.KeyToPath{ + { + Key: cdb.Spec.ORDSPwd.Secret.Key, + Path: cdb.Spec.ORDSPwd.Secret.Key, + }, + }, }, }, { @@ -355,6 +379,12 @@ func (r *CDBReconciler) createPodSpec(cdb *dbapi.CDB) corev1.PodSpec { LocalObjectReference: corev1.LocalObjectReference{ Name: cdb.Spec.WebServerUser.Secret.SecretName, }, + Items: []corev1.KeyToPath{ + { + Key: cdb.Spec.WebServerUser.Secret.Key, + Path: cdb.Spec.WebServerUser.Secret.Key, + }, + }, }, }, { @@ -362,6 +392,12 @@ func (r *CDBReconciler) createPodSpec(cdb *dbapi.CDB) corev1.PodSpec { LocalObjectReference: corev1.LocalObjectReference{ Name: cdb.Spec.WebServerPwd.Secret.SecretName, }, + Items: []corev1.KeyToPath{ + { + Key: cdb.Spec.WebServerPwd.Secret.Key, + Path: cdb.Spec.WebServerPwd.Secret.Key, + }, + }, }, }, }, diff --git a/oracle-database-operator.yaml b/oracle-database-operator.yaml index aefc93f3..46ed26ef 100644 --- a/oracle-database-operator.yaml +++ b/oracle-database-operator.yaml @@ -2021,7 +2021,7 @@ rules: - apiGroups: - database.oracle.com resources: - - pdbs + - oraclerestdataservices verbs: - create - delete @@ -2033,17 +2033,13 @@ rules: - apiGroups: - database.oracle.com resources: - - pdbs/finalizers + - oraclerestdataservices/finalizers verbs: - - create - - delete - - get - - patch - update - apiGroups: - database.oracle.com resources: - - pdbs/status + - oraclerestdataservices/status verbs: - get - patch @@ -2051,7 +2047,7 @@ rules: - apiGroups: - database.oracle.com resources: - - shardingdatabases + - pdbs verbs: - create - delete @@ -2063,7 +2059,7 @@ rules: - apiGroups: - database.oracle.com resources: - - shardingdatabases/finalizers + - pdbs/finalizers verbs: - create - delete @@ -2073,7 +2069,7 @@ rules: - apiGroups: - database.oracle.com resources: - - shardingdatabases/status + - pdbs/status verbs: - get - patch @@ -2081,7 +2077,7 @@ rules: - apiGroups: - database.oracle.com resources: - - singleinstancedatabases + - shardingdatabases verbs: - create - delete @@ -2093,13 +2089,17 @@ rules: - apiGroups: - database.oracle.com resources: - - singleinstancedatabases/finalizers + - shardingdatabases/finalizers verbs: + - create + - delete + - get + - patch - update - apiGroups: - database.oracle.com resources: - - singleinstancedatabases/status + - shardingdatabases/status verbs: - get - patch @@ -2107,7 +2107,7 @@ rules: - apiGroups: - database.oracle.com resources: - - oraclerestdataservices + - singleinstancedatabases verbs: - create - delete @@ -2119,13 +2119,13 @@ rules: - apiGroups: - database.oracle.com resources: - - oraclerestdataservices/finalizers + - singleinstancedatabases/finalizers verbs: - update - apiGroups: - database.oracle.com resources: - - oraclerestdataservices/status + - singleinstancedatabases/status verbs: - get - patch @@ -2352,9 +2352,9 @@ webhooks: service: name: oracle-database-operator-webhook-service namespace: oracle-database-operator-system - path: /mutate-database-oracle-com-v1alpha1-oraclerestdataservice + path: /mutate-database-oracle-com-v1alpha1-cdb failurePolicy: Fail - name: moraclerestdataservice.kb.io + name: mcdb.kb.io rules: - apiGroups: - database.oracle.com @@ -2364,7 +2364,7 @@ webhooks: - CREATE - UPDATE resources: - - oraclerestdataservices + - cdbs sideEffects: None - admissionReviewVersions: - v1 @@ -2373,9 +2373,9 @@ webhooks: service: name: oracle-database-operator-webhook-service namespace: oracle-database-operator-system - path: /mutate-database-oracle-com-v1alpha1-cdb + path: /mutate-database-oracle-com-v1alpha1-oraclerestdataservice failurePolicy: Fail - name: mcdb.kb.io + name: moraclerestdataservice.kb.io rules: - apiGroups: - database.oracle.com @@ -2385,7 +2385,7 @@ webhooks: - CREATE - UPDATE resources: - - cdbs + - oraclerestdataservices sideEffects: None - admissionReviewVersions: - v1 @@ -2505,9 +2505,9 @@ webhooks: service: name: oracle-database-operator-webhook-service namespace: oracle-database-operator-system - path: /validate-database-oracle-com-v1alpha1-oraclerestdataservice + path: /validate-database-oracle-com-v1alpha1-cdb failurePolicy: Fail - name: voraclerestdataservice.kb.io + name: vcdb.kb.io rules: - apiGroups: - database.oracle.com @@ -2517,7 +2517,7 @@ webhooks: - CREATE - UPDATE resources: - - oraclerestdataservices + - cdbs sideEffects: None - admissionReviewVersions: - v1 @@ -2526,9 +2526,9 @@ webhooks: service: name: oracle-database-operator-webhook-service namespace: oracle-database-operator-system - path: /validate-database-oracle-com-v1alpha1-cdb + path: /validate-database-oracle-com-v1alpha1-oraclerestdataservice failurePolicy: Fail - name: vcdb.kb.io + name: voraclerestdataservice.kb.io rules: - apiGroups: - database.oracle.com @@ -2538,7 +2538,7 @@ webhooks: - CREATE - UPDATE resources: - - cdbs + - oraclerestdataservices sideEffects: None - admissionReviewVersions: - v1 @@ -2582,4 +2582,4 @@ webhooks: - DELETE resources: - singleinstancedatabases - sideEffects: None \ No newline at end of file + sideEffects: None From d61bbb71108ce168f9175f311b9952780bba9d5d Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Sun, 17 Apr 2022 21:14:33 +0530 Subject: [PATCH 232/628] Using silent apex install command for installation automation Signed-off-by: abhisbyk --- commons/database/constants.go | 5 +++ .../oraclerestdataservice_controller.go | 45 ++++++++++--------- 2 files changed, 28 insertions(+), 22 deletions(-) diff --git a/commons/database/constants.go b/commons/database/constants.go index 87961dd4..b315091a 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -371,6 +371,11 @@ const InstallApex string = "if [ -f /opt/oracle/oradata/${ORACLE_SID^^}/apex/ape const InstallApexRemote string = "if [ -e ${ORDS_HOME}/config/apex/apexins.sql ]; then ( while true; do sleep 60; echo \"Installing Apex...\" ; done ) & " + " cd ${ORDS_HOME}/config/apex/ && echo -e \"@apexins.sql SYSAUX SYSAUX TEMP /i/\" | %[1]s && kill -9 $!; else echo \"Apex Folder doesn't exist\" ; fi ;" +const InstallApexRemoteA string = "if [ -e ${ORDS_HOME}/config/apex/apxsilentins.sql ]; then ( while true; do sleep 60; echo \"Installing Apex...\" ; done ) & " + + " cd ${ORDS_HOME}/config/apex/ && echo -e \"@apxsilentins.sql SYSAUX SYSAUX TEMP /i/ %[2]s %[2]s %[2]s %[2]s\" | %[1]s && kill -9 $!; else echo \"Apex Folder doesn't exist\" ; fi ;" + +const InstallApexRemoteB string = "if [ -e ${ORDS_HOME}/config/apex/apxsilentins.sql ]; then cd ${ORDS_HOME}/config/apex/ && echo -e \"@apxsilentins.sql SYSAUX SYSAUX TEMP /i/ %[2]s %[2]s %[2]s %[2]s\" | %[1]s; else echo \"Apex Folder doesn't exist\" ; fi ;" + const IsApexInstalled string = "select 'APEXVERSION:'||version as version FROM DBA_REGISTRY WHERE COMP_ID='APEX';" const UninstallApex string = "if [ -f /opt/oracle/oradata/${ORACLE_SID^^}/apex/apxremov.sql ]; then ( while true; do sleep 60; echo \"Uninstalling Apex...\" ; done ) & " + diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 78297e87..2cf50410 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -1013,8 +1013,26 @@ func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataS sidbPassword := string(adminPasswordSecret.Data[m.Spec.AdminPassword.SecretKey]) log.Info("SIDB Password: " + sidbPassword) + apexPasswordSecret := &corev1.Secret{} + err = r.Get(ctx, types.NamespacedName{Name: m.Spec.ApexPassword.SecretName, Namespace: m.Namespace}, apexPasswordSecret) + if err != nil { + if apierrors.IsNotFound(err) { + m.Status.Status = dbcommons.StatusError + eventReason := "Waiting" + eventMsg := "waiting for secret : " + m.Spec.ApexPassword.SecretName + " to get created" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + r.Log.Info("Secret " + m.Spec.ApexPassword.SecretName + " Not Found") + return requeueY + } + log.Error(err, err.Error()) + return requeueY + } + + // APEX_LISTENER , APEX_REST_PUBLIC_USER , APEX_PUBLIC_USER passwords + apexPassword := string(apexPasswordSecret.Data[m.Spec.ApexPassword.SecretKey]) + if !n.Status.ApexInstalled { - result := r.installApexSIDB(m, n, ordsReadyPod, sidbPassword, ctx, req) + result := r.installApexSIDB(m, n, ordsReadyPod, sidbPassword, apexPassword, ctx, req) if result.Requeue { log.Info("Reconcile requeued because apex installation failed") return result @@ -1070,23 +1088,6 @@ func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataS } */ - apexPasswordSecret := &corev1.Secret{} - err = r.Get(ctx, types.NamespacedName{Name: m.Spec.ApexPassword.SecretName, Namespace: m.Namespace}, apexPasswordSecret) - if err != nil { - if apierrors.IsNotFound(err) { - m.Status.Status = dbcommons.StatusError - eventReason := "Waiting" - eventMsg := "waiting for secret : " + m.Spec.ApexPassword.SecretName + " to get created" - r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) - r.Log.Info("Secret " + m.Spec.ApexPassword.SecretName + " Not Found") - return requeueY - } - log.Error(err, err.Error()) - return requeueY - } - // APEX_LISTENER , APEX_REST_PUBLIC_USER , APEX_PUBLIC_USER passwords - apexPassword := string(apexPasswordSecret.Data[m.Spec.ApexPassword.SecretKey]) - m.Status.Status = dbcommons.StatusUpdating r.Status().Update(ctx, m) @@ -1111,7 +1112,7 @@ func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataS } // Alter APEX_LISTENER,APEX_PUBLIC_USER,APEX_REST_PUBLIC_USER - out, err = dbcommons.ExecCommand(r, r.Config, ordsReadyPod.Name, ordsReadyPod.Namespace, "", ctx, req, true, + /* out, err = dbcommons.ExecCommand(r, r.Config, ordsReadyPod.Name, ordsReadyPod.Namespace, "", ctx, req, true, "bash", "-c", fmt.Sprintf(dbcommons.AlterApexUsers, apexPassword, fmt.Sprintf(dbcommons.SQLPlusRemoteCLI, sidbPassword, n.Status.ConnectString))) if err != nil { log.Info(err.Error()) @@ -1131,7 +1132,7 @@ func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataS log.Info(err.Error()) return requeueY } - log.Info("Change ApexAdmin Password Output : \n" + out) + log.Info("Change ApexAdmin Password Output : \n" + out) */ m.Status.ApexConfigured = true r.Status().Update(ctx, m) @@ -1184,7 +1185,7 @@ func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataS // Install APEX in SIDB //############################################################################# func (r *OracleRestDataServiceReconciler) installApexSIDB(m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase, - ordsReadyPod corev1.Pod, sidbPassword string, ctx context.Context, req ctrl.Request) ctrl.Result { + ordsReadyPod corev1.Pod, sidbPassword string, apexPassword string, ctx context.Context, req ctrl.Request) ctrl.Result { log := r.Log.WithValues("installApex", req.NamespacedName) // Initial Validation @@ -1201,7 +1202,7 @@ func (r *OracleRestDataServiceReconciler) installApexSIDB(m *dbapi.OracleRestDat //Install Apex in SIDB ready pod out, err := dbcommons.ExecCommand(r, r.Config, ordsReadyPod.Name, ordsReadyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf(dbcommons.InstallApexRemote, fmt.Sprintf(dbcommons.SQLPlusRemoteCLI, sidbPassword, n.Status.ConnectString))) + fmt.Sprintf(dbcommons.InstallApexRemoteB, fmt.Sprintf(dbcommons.SQLPlusRemoteCLI, sidbPassword, n.Status.ConnectString), apexPassword)) if err != nil { log.Info(err.Error()) } From fb103f1863466910b56bdd97218b32b7682a3b6b Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Tue, 19 Apr 2022 10:04:05 +0530 Subject: [PATCH 233/628] Installing APEX to pdb Signed-off-by: abhisbyk --- .../database/oraclerestdataservice_controller.go | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 2cf50410..4c7f4919 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -1092,9 +1092,9 @@ func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataS r.Status().Update(ctx, m) configureApexRestSqlClient := "sqlplus -s / as sysdba @apex_rest_config.sql" - configureApexRestSqlClient = fmt.Sprintf(dbcommons.SQLPlusRemoteCLI, sidbPassword, n.Status.ConnectString) + " @apex_rest_config.sql" + configureApexRestSqlClient = fmt.Sprintf(dbcommons.SQLPlusRemoteCLI, sidbPassword, n.Status.PdbConnectString) + " @apex_rest_config.sql" // Configure APEX - out, err := dbcommons.ExecCommand(r, r.Config, ordsReadyPod.Name, ordsReadyPod.Namespace, "", ctx, req, true, "bash", "-c", + out, err := dbcommons.ExecCommand(r, r.Config, ordsReadyPod.Name, ordsReadyPod.Namespace, "", ctx, req, false, "bash", "-c", fmt.Sprintf(dbcommons.ConfigureApexRest, apexPassword, configureApexRestSqlClient)) if err != nil { log.Info(err.Error()) @@ -1151,7 +1151,7 @@ func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataS } // Set Apex users in apex_rt,apex_al,apex files - out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, true, "bash", "-c", + out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", fmt.Sprintf(dbcommons.SetApexUsers, apexPassword)) log.Info("SetApexUsers Output: \n" + out) if strings.Contains(strings.ToUpper(out), "ERROR") { @@ -1202,7 +1202,7 @@ func (r *OracleRestDataServiceReconciler) installApexSIDB(m *dbapi.OracleRestDat //Install Apex in SIDB ready pod out, err := dbcommons.ExecCommand(r, r.Config, ordsReadyPod.Name, ordsReadyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf(dbcommons.InstallApexRemoteB, fmt.Sprintf(dbcommons.SQLPlusRemoteCLI, sidbPassword, n.Status.ConnectString), apexPassword)) + fmt.Sprintf(dbcommons.InstallApexRemoteB, fmt.Sprintf(dbcommons.SQLPlusRemoteCLI, sidbPassword, n.Status.PdbConnectString), apexPassword)) if err != nil { log.Info(err.Error()) } @@ -1217,7 +1217,7 @@ func (r *OracleRestDataServiceReconciler) installApexSIDB(m *dbapi.OracleRestDat // Checking if Apex is installed successfully or not out, err = dbcommons.ExecCommand(r, r.Config, ordsReadyPod.Name, ordsReadyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf("echo -e \"%s\" | %s", dbcommons.IsApexInstalled, fmt.Sprintf(dbcommons.SQLPlusRemoteCLI, sidbPassword, n.Status.ConnectString))) + fmt.Sprintf("echo -e \"%s\" | %s", dbcommons.IsApexInstalled, fmt.Sprintf(dbcommons.SQLPlusRemoteCLI, sidbPassword, n.Status.PdbConnectString))) if err != nil { log.Error(err, err.Error()) return requeueY From e74b6c613bd1004fe5be3a05257c5428cd904c0e Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Tue, 19 Apr 2022 12:22:26 +0530 Subject: [PATCH 234/628] Prebuilt DB support --- .../singleinstancedatabase_controller.go | 29 ++++++++++++------- 1 file changed, 18 insertions(+), 11 deletions(-) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index c5dde64e..44e0ee1e 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -438,7 +438,7 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab // Instantiate POD spec from SingleInstanceDatabase spec //############################################################################# func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleInstanceDatabase, n *dbapi.SingleInstanceDatabase) *corev1.Pod { - // Pre-built db, useful for dev/test/CI-CD + // prebuiltDB, useful for dev/test/CI-CD if m.Spec.Image.PrebuiltDB { pod := &corev1.Pod{ TypeMeta: metav1.TypeMeta{ @@ -559,6 +559,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns return pod } + // If not prebuiltDB pod := &corev1.Pod{ TypeMeta: metav1.TypeMeta{ Kind: "Pod", @@ -697,16 +698,22 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }(), }, - VolumeMounts: []corev1.VolumeMount{{ - MountPath: "/opt/oracle/oradata", - Name: "datamount", - }, { - // This is for express edition DB - MountPath: "/run/secrets/oracle_pwd", - ReadOnly: true, - Name: "oracle-pwd-vol", - SubPath: "oracle_pwd", - }}, + VolumeMounts: func() []corev1.VolumeMount { + mounts := []corev1.VolumeMount{ { + MountPath: "/opt/oracle/oradata", + Name: "datamount", + }} + if m.Spec.Edition == "express" { + mounts = append(mounts, corev1.VolumeMount { + // This is for express edition DB + MountPath: "/run/secrets/oracle_pwd", + ReadOnly: true, + Name: "oracle-pwd-vol", + SubPath: "oracle_pwd", + }) + } + return mounts + }(), Env: func() []corev1.EnvVar { // adding XE support, useful for dev/test/CI-CD if m.Spec.Edition == "express" { From e98174f0341002542e73ae9163e7af7f3293b823 Mon Sep 17 00:00:00 2001 From: "paramdeep.saini@oracle.com" Date: Tue, 19 Apr 2022 14:45:59 -0700 Subject: [PATCH 235/628] Added the code fix --- commons/sharding/gsm.go | 6 +- commons/sharding/provstatus.go | 9 +- commons/sharding/scommon.go | 111 +++++++++--------- commons/sharding/shard.go | 17 ++- .../database/shardingdatabase_controller.go | 2 +- 5 files changed, 71 insertions(+), 74 deletions(-) diff --git a/commons/sharding/gsm.go b/commons/sharding/gsm.go index 5ec9be10..ae1f94e2 100644 --- a/commons/sharding/gsm.go +++ b/commons/sharding/gsm.go @@ -41,10 +41,11 @@ package commons import ( "context" "fmt" - databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" "reflect" "strconv" + databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + "github.com/go-logr/logr" appsv1 "k8s.io/api/apps/v1" corev1 "k8s.io/api/core/v1" @@ -429,8 +430,7 @@ func UpdateProvForGsm(instance *databasev1alpha1.ShardingDatabase, ) (ctrl.Result, error) { var msg string - var size int32 - size = 1 + var size int32 = 1 var isUpdate bool = false var err error var i int diff --git a/commons/sharding/provstatus.go b/commons/sharding/provstatus.go index ac7e8c02..4245fc07 100644 --- a/commons/sharding/provstatus.go +++ b/commons/sharding/provstatus.go @@ -335,12 +335,11 @@ func GetMetaCondition(instance *databasealphav1.ShardingDatabase, result *ctrl.R func CheckGsmStatus(gname string, instance *databasealphav1.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, ) error { var err error - var msg string + var msg string = "Inside the checkGsmStatus. Checking GSM director in " + GetFmtStr(gname) + " pod." - msg = "Inside the checkGsmStatus. Checking GSM director in " + GetFmtStr(gname) + " pod." LogMessages("DEBUG", msg, nil, instance, logger) - err, _, _ = ExecCommand(gname, getGsmvalidateCmd(), kubeClient, kubeconfig, instance, logger) + _, _, err = ExecCommand(gname, getGsmvalidateCmd(), kubeClient, kubeconfig, instance, logger) if err != nil { return err } @@ -353,9 +352,9 @@ func CheckGsmStatus(gname string, instance *databasealphav1.ShardingDatabase, ku func ValidateDbSetup(podName string, instance *databasealphav1.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, ) error { - err, _, _ := ExecCommand(podName, shardValidationCmd(), kubeClient, kubeconfig, instance, logger) + _, _, err := ExecCommand(podName, shardValidationCmd(), kubeClient, kubeconfig, instance, logger) if err != nil { - return fmt.Errorf("Error ocurred while validating the DB Setup") + return fmt.Errorf("error ocurred while validating the DB Setup") } return nil } diff --git a/commons/sharding/scommon.go b/commons/sharding/scommon.go index 60e661da..04cd5079 100644 --- a/commons/sharding/scommon.go +++ b/commons/sharding/scommon.go @@ -41,6 +41,7 @@ package commons import ( "context" "fmt" + databasealphav1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" "regexp" @@ -486,15 +487,21 @@ func PodListValidation(podList *corev1.PodList, sfName string, instance *databas func GetPodList(sfsetName string, resType string, instance *databasealphav1.ShardingDatabase, kClient client.Client, ) (*corev1.PodList, error) { podList := &corev1.PodList{} - labelSelector := labels.SelectorFromSet(getlabelsForGsm(instance)) - if resType == "GSM" { + //labelSelector := labels.SelectorFromSet(getlabelsForGsm(instance)) + //labelSelector := map[string]labels.Selector{} + var labelSelector labels.Selector + + //labels.SelectorFromSet() + + switch resType { + case "GSM": labelSelector = labels.SelectorFromSet(getlabelsForGsm(instance)) - } else if resType == "SHARD" { + case "SHARD": labelSelector = labels.SelectorFromSet(getlabelsForShard(instance)) - } else if resType == "CATALOG" { + case "CATALOG": labelSelector = labels.SelectorFromSet(getlabelsForCatalog(instance)) - } else { - err1 := fmt.Errorf("Wrong resources type passed. Supported values are SHARD,GSM and CATALOG") + default: + err1 := fmt.Errorf("wrong resources type passed. Supported values are SHARD,GSM and CATALOG") return nil, err1 } @@ -612,8 +619,7 @@ func getOwnerRef(instance *databasealphav1.ShardingDatabase, } func buildCatalogParams(instance *databasealphav1.ShardingDatabase) string { - var variables []databasealphav1.EnvironmentVariable - variables = instance.Spec.Catalog[0].EnvVars + var variables []databasealphav1.EnvironmentVariable = instance.Spec.Catalog[0].EnvVars var result string var varinfo string var sidFlag bool = false @@ -725,13 +731,14 @@ func buildDirectorParams(instance *databasealphav1.ShardingDatabase, oraGsmSpex result = result + varinfo } - if idx == 0 { + switch idx { + case 0: varinfo = "director_region=primary;" result = result + varinfo - } else if idx == 1 { + case 1: varinfo = "director_region=standby;" result = result + varinfo - } else { + default: // Do nothing } @@ -744,8 +751,7 @@ func buildDirectorParams(instance *databasealphav1.ShardingDatabase, oraGsmSpex } func BuildShardParams(sfSet *appsv1.StatefulSet) string { - var variables []corev1.EnvVar - variables = sfSet.Spec.Template.Spec.Containers[0].Env + var variables []corev1.EnvVar = sfSet.Spec.Template.Spec.Containers[0].Env var result string var varinfo string var isShardPort bool = false @@ -836,20 +842,17 @@ func GetShardInviteNodeCmd(shardName string) []string { } func getCancelChunksCmd(sparamStr string) []string { - var cancelChunkCmd []string - cancelChunkCmd = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", "--cancelchunks=" + strconv.Quote(sparamStr), "--optype=gsm"} + var cancelChunkCmd []string = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", "--cancelchunks=" + strconv.Quote(sparamStr), "--optype=gsm"} return cancelChunkCmd } func getMoveChunksCmd(sparamStr string) []string { - var moveChunkCmd []string - moveChunkCmd = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", "--movechunks=" + strconv.Quote(sparamStr), "--optype=gsm"} + var moveChunkCmd []string = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", "--movechunks=" + strconv.Quote(sparamStr), "--optype=gsm"} return moveChunkCmd } func getNoChunksCmd(sparamStr string) []string { - var noChunkCmd []string - noChunkCmd = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", "--validatenochunks=" + strconv.Quote(sparamStr), "--optype=gsm"} + var noChunkCmd []string = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", "--validatenochunks=" + strconv.Quote(sparamStr), "--optype=gsm"} return noChunkCmd } @@ -860,8 +863,7 @@ func shardValidationCmd() []string { } func getShardCheckCmd(sparamStr string) []string { - var checkShardCmd []string - checkShardCmd = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", "--checkgsmshard=" + strconv.Quote(sparamStr), "--optype=gsm"} + var checkShardCmd []string = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", "--checkgsmshard=" + strconv.Quote(sparamStr), "--optype=gsm"} return checkShardCmd } @@ -897,32 +899,27 @@ func getLivenessCmd(resType string) []string { } func getGsmShardValidateCmd(shardName string) []string { - var validateCmd []string - validateCmd = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", "--validateshard=" + strconv.Quote(shardName), "--optype=gsm"} + var validateCmd []string = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", "--validateshard=" + strconv.Quote(shardName), "--optype=gsm"} return validateCmd } func getOnlineShardCmd(sparamStr string) []string { - var onlineCmd []string - onlineCmd = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", "--checkonlineshard=" + strconv.Quote(sparamStr), "--optype=gsm"} + var onlineCmd []string = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", "--checkonlineshard=" + strconv.Quote(sparamStr), "--optype=gsm"} return onlineCmd } func getGsmAddShardGroupCmd(sparamStr string) []string { - var addSgroupCmd []string - addSgroupCmd = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", sparamStr, "--optype=gsm"} + var addSgroupCmd []string = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", sparamStr, "--optype=gsm"} return addSgroupCmd } func getdeployShardCmd() []string { - var depCmd []string - depCmd = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", "--deployshard=true", "--optype=gsm"} + var depCmd []string = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", "--deployshard=true", "--optype=gsm"} return depCmd } func getGsmvalidateCmd() []string { - var depCmd []string - depCmd = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", "--checkliveness=true", "--optype=gsm"} + var depCmd []string = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", "--checkliveness=true", "--optype=gsm"} return depCmd } @@ -957,8 +954,9 @@ func ReadConfigMap(cmName string, instance *databasealphav1.ShardingDatabase, kC ) (string, string, string, string, string, string) { var region, fingerprint, user, tenancy, passphrase, str1, topicid, k, value string - cm := &corev1.ConfigMap{} var err error + cm := &corev1.ConfigMap{} + //var err error // Reding a config map err = kClient.Get(context.TODO(), types.NamespacedName{ @@ -986,19 +984,20 @@ func ReadConfigMap(cmName string, instance *databasealphav1.ShardingDatabase, kC value = line[s+1:] LogMessages("DEBUG", "Key : "+GetFmtStr(k)+" Value : "+GetFmtStr(value), nil, instance, logger) - if k == "region" { + switch k { + case "region": region = value - } else if k == "fingerprint" { + case "fingerprint": fingerprint = value - } else if k == "user" { + case "user": user = value - } else if k == "tenancy" { + case "tenancy": tenancy = value - } else if k == "passpharase" { + case "passpharase": passphrase = value - } else if k == "topicid" { + case "topicid": topicid = value - } else { + default: LogMessages("DEBUG", GetFmtStr(k)+" is not matching with any required value for ONS.", nil, instance, logger) } } @@ -1010,10 +1009,10 @@ func ReadSecret(secName string, instance *databasealphav1.ShardingDatabase, kCli var value string sc := &corev1.Secret{} - var err error + //var err error // Reading a Secret - err = kClient.Get(context.TODO(), types.NamespacedName{ + var err error = kClient.Get(context.TODO(), types.NamespacedName{ Name: secName, Namespace: instance.Spec.Namespace, }, sc) @@ -1067,7 +1066,7 @@ func Contains(list []string, s string) bool { func CheckShardInGsm(gsmPodName string, sparams string, instance *databasealphav1.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, ) error { - err, _, _ := ExecCommand(gsmPodName, getShardCheckCmd(sparams), kubeClient, kubeconfig, instance, logger) + _, _, err := ExecCommand(gsmPodName, getShardCheckCmd(sparams), kubeClient, kubeconfig, instance, logger) if err != nil { msg := "Did not find the shard " + GetFmtStr(sparams) + " in GSM." LogMessages("INFO", msg, nil, instance, logger) @@ -1080,7 +1079,7 @@ func CheckShardInGsm(gsmPodName string, sparams string, instance *databasealphav func CheckOnlineShardInGsm(gsmPodName string, sparams string, instance *databasealphav1.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, ) error { - err, _, _ := ExecCommand(gsmPodName, getOnlineShardCmd(sparams), kubeClient, kubeconfig, instance, logger) + _, _, err := ExecCommand(gsmPodName, getOnlineShardCmd(sparams), kubeClient, kubeconfig, instance, logger) if err != nil { msg := "Shard: " + GetFmtStr(sparams) + " is not onine in GSM." LogMessages("INFO", msg, nil, instance, logger) @@ -1093,7 +1092,7 @@ func CheckOnlineShardInGsm(gsmPodName string, sparams string, instance *database func MoveChunks(gsmPodName string, sparams string, instance *databasealphav1.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, ) error { - err, _, _ := ExecCommand(gsmPodName, getMoveChunksCmd(sparams), kubeClient, kubeconfig, instance, logger) + _, _, err := ExecCommand(gsmPodName, getMoveChunksCmd(sparams), kubeClient, kubeconfig, instance, logger) if err != nil { msg := "Error occurred in during Chunk movement command submission for shard: " + GetFmtStr(sparams) + " in GSM." LogMessages("INFO", msg, nil, instance, logger) @@ -1105,7 +1104,7 @@ func MoveChunks(gsmPodName string, sparams string, instance *databasealphav1.Sha // Function to verify the chunks func VerifyChunks(gsmPodName string, sparams string, instance *databasealphav1.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, ) error { - err, _, _ := ExecCommand(gsmPodName, getNoChunksCmd(sparams), kubeClient, kubeconfig, instance, logger) + _, _, err := ExecCommand(gsmPodName, getNoChunksCmd(sparams), kubeClient, kubeconfig, instance, logger) if err != nil { msg := "Chunks are not moved completely from the shard: " + GetFmtStr(sparams) + " in GSM." LogMessages("INFO", msg, nil, instance, logger) @@ -1117,7 +1116,7 @@ func VerifyChunks(gsmPodName string, sparams string, instance *databasealphav1.S // Function to verify the chunks func AddShardInGsm(gsmPodName string, sparams string, instance *databasealphav1.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, ) error { - err, _, _ := ExecCommand(gsmPodName, getShardAddCmd(sparams), kubeClient, kubeconfig, instance, logger) + _, _, err := ExecCommand(gsmPodName, getShardAddCmd(sparams), kubeClient, kubeconfig, instance, logger) if err != nil { msg := "Error occurred while adding a shard " + GetFmtStr(sparams) + " in GSM." LogMessages("INFO", msg, nil, instance, logger) @@ -1129,7 +1128,7 @@ func AddShardInGsm(gsmPodName string, sparams string, instance *databasealphav1. // Function to deploy the Shards func DeployShardInGsm(gsmPodName string, sparams string, instance *databasealphav1.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, ) error { - err, _, _ := ExecCommand(gsmPodName, getdeployShardCmd(), kubeClient, kubeconfig, instance, logger) + _, _, err := ExecCommand(gsmPodName, getdeployShardCmd(), kubeClient, kubeconfig, instance, logger) if err != nil { msg := "Error occurred while deploying the shard in GSM." LogMessages("INFO", msg, nil, instance, logger) @@ -1141,7 +1140,7 @@ func DeployShardInGsm(gsmPodName string, sparams string, instance *databasealpha // Function to verify the chunks func CancelChunksInGsm(gsmPodName string, sparams string, instance *databasealphav1.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, ) error { - err, _, _ := ExecCommand(gsmPodName, getCancelChunksCmd(sparams), kubeClient, kubeconfig, instance, logger) + _, _, err := ExecCommand(gsmPodName, getCancelChunksCmd(sparams), kubeClient, kubeconfig, instance, logger) if err != nil { msg := "Error occurred while cancelling the chunks: " + GetFmtStr(sparams) + " in GSM." LogMessages("INFO", msg, nil, instance, logger) @@ -1153,7 +1152,7 @@ func CancelChunksInGsm(gsmPodName string, sparams string, instance *databasealph // Function to delete the shard func RemoveShardFromGsm(gsmPodName string, sparams string, instance *databasealphav1.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, ) error { - err, _, _ := ExecCommand(gsmPodName, getShardDelCmd(sparams), kubeClient, kubeconfig, instance, logger) + _, _, err := ExecCommand(gsmPodName, getShardDelCmd(sparams), kubeClient, kubeconfig, instance, logger) if err != nil { msg := "Error occurred while cancelling the chunks: " + GetFmtStr(sparams) + " in GSM." LogMessages("INFO", msg, nil, instance, logger) @@ -1163,19 +1162,19 @@ func RemoveShardFromGsm(gsmPodName string, sparams string, instance *databasealp } func GetSvcIp(PodName string, sparams string, instance *databasealphav1.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, -) (error, string, string) { - err, stdoutput, stderror := ExecCommand(PodName, GetIpCmd(sparams), kubeClient, kubeconfig, instance, logger) +) (string, string, error) { + stdoutput, stderror, err := ExecCommand(PodName, GetIpCmd(sparams), kubeClient, kubeconfig, instance, logger) if err != nil { msg := "Error occurred while getting the IP for k8s service " + GetFmtStr(sparams) LogMessages("INFO", msg, nil, instance, logger) - return err, strings.Replace(stdoutput, "\r\n", "", -1), strings.Replace(stderror, "/r/n", "", -1) + return strings.Replace(stdoutput, "\r\n", "", -1), strings.Replace(stderror, "/r/n", "", -1), err } - return nil, strings.Replace(stdoutput, "\r\n", "", -1), strings.Replace(stderror, "/r/n", "", -1) + return strings.Replace(stdoutput, "\r\n", "", -1), strings.Replace(stderror, "/r/n", "", -1), nil } func GetGsmServices(PodName string, instance *databasealphav1.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, ) string { - err, stdoutput, _ := ExecCommand(PodName, getGsmSvcCmd(), kubeClient, kubeconfig, instance, logger) + stdoutput, _, err := ExecCommand(PodName, getGsmSvcCmd(), kubeClient, kubeconfig, instance, logger) if err != nil { msg := "Error occurred while getting the services from the GSM " LogMessages("DEBUG", msg, err, instance, logger) @@ -1186,7 +1185,7 @@ func GetGsmServices(PodName string, instance *databasealphav1.ShardingDatabase, func GetDbRole(PodName string, instance *databasealphav1.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, ) string { - err, stdoutput, _ := ExecCommand(PodName, getDbRoleCmd(), kubeClient, kubeconfig, instance, logger) + stdoutput, _, err := ExecCommand(PodName, getDbRoleCmd(), kubeClient, kubeconfig, instance, logger) if err != nil { msg := "Error occurred while getting the DB role from the database" LogMessages("DEBUG", msg, err, instance, logger) @@ -1197,7 +1196,7 @@ func GetDbRole(PodName string, instance *databasealphav1.ShardingDatabase, kubeC func GetDbOpenMode(PodName string, instance *databasealphav1.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, ) string { - err, stdoutput, _ := ExecCommand(PodName, getDbModeCmd(), kubeClient, kubeconfig, instance, logger) + stdoutput, _, err := ExecCommand(PodName, getDbModeCmd(), kubeClient, kubeconfig, instance, logger) if err != nil { msg := "Error occurred while getting the DB mode from the database" LogMessages("DEBUG", msg, err, instance, logger) diff --git a/commons/sharding/shard.go b/commons/sharding/shard.go index e1063ba5..6a000a14 100644 --- a/commons/sharding/shard.go +++ b/commons/sharding/shard.go @@ -40,10 +40,11 @@ package commons import ( "context" - databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" "reflect" "strconv" + databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + "github.com/go-logr/logr" appsv1 "k8s.io/api/apps/v1" corev1 "k8s.io/api/core/v1" @@ -105,8 +106,7 @@ func builObjectMetaForShard(instance *databasev1alpha1.ShardingDatabase, OraShar // Function to build Stateful Specs func buildStatefulSpecForShard(instance *databasev1alpha1.ShardingDatabase, OraShardSpex databasev1alpha1.ShardSpec) *appsv1.StatefulSetSpec { // building Stateful set Specs - var size int32 - size = 1 + var size int32 = 1 sfsetspec := &appsv1.StatefulSetSpec{ ServiceName: OraShardSpex.Name, Selector: &metav1.LabelSelector{ @@ -355,8 +355,8 @@ func volumeClaimTemplatesForShard(instance *databasev1alpha1.ShardingDatabase, O } func BuildServiceDefForShard(instance *databasev1alpha1.ShardingDatabase, replicaCount int32, OraShardSpex databasev1alpha1.ShardSpec, svctype string) *corev1.Service { - service := &corev1.Service{} - service = &corev1.Service{ + //service := &corev1.Service{} + service := &corev1.Service{ ObjectMeta: buildSvcObjectMetaForShard(instance, replicaCount, OraShardSpex, svctype), Spec: corev1.ServiceSpec{}, } @@ -402,8 +402,7 @@ func buildSvcObjectMetaForShard(instance *databasev1alpha1.ShardingDatabase, rep func getSvcLabelsForShard(replicaCount int32, OraShardSpex databasev1alpha1.ShardSpec) map[string]string { - var labelStr map[string]string - labelStr = make(map[string]string) + var labelStr map[string]string = make(map[string]string) if replicaCount == -1 { labelStr["statefulset.kubernetes.io/pod-name"] = OraShardSpex.Name + "-0" } else { @@ -418,8 +417,8 @@ func getSvcLabelsForShard(replicaCount int32, OraShardSpex databasev1alpha1.Shar func UpdateProvForShard(instance *databasev1alpha1.ShardingDatabase, OraShardSpex databasev1alpha1.ShardSpec, kClient client.Client, sfSet *appsv1.StatefulSet, shardPod *corev1.Pod, logger logr.Logger, ) (ctrl.Result, error) { var msg string - var size int32 - size = 1 + var size int32 = 1 + //size = 1 var isUpdate bool = false var err error var i int diff --git a/controllers/database/shardingdatabase_controller.go b/controllers/database/shardingdatabase_controller.go index ead408a7..91964906 100644 --- a/controllers/database/shardingdatabase_controller.go +++ b/controllers/database/shardingdatabase_controller.go @@ -1672,7 +1672,7 @@ func (r *ShardingDatabaseReconciler) gsmInvitedNodeOp(instance *databasev1alpha1 count = count + 1 continue } - err, _, _ = shardingv1.ExecCommand(gsmPodName.Name, shardingv1.GetShardInviteNodeCmd(objName), r.kubeClient, r.kubeConfig, instance, r.Log) + _, _, err = shardingv1.ExecCommand(gsmPodName.Name, shardingv1.GetShardInviteNodeCmd(objName), r.kubeClient, r.kubeConfig, instance, r.Log) if err != nil { msg = "Invite delete and add node failed " + shardingv1.GetFmtStr(objName) + " details in GSM." shardingv1.LogMessages("DEBUG", msg, err, instance, r.Log) From ee91846afa763bcdf391e70c3c43e6a9522a61dd Mon Sep 17 00:00:00 2001 From: "paramdeep.saini@oracle.com" Date: Tue, 19 Apr 2022 14:46:13 -0700 Subject: [PATCH 236/628] Added the code fix for static check --- .vscode/launch.json | 7 +++++++ 1 file changed, 7 insertions(+) create mode 100644 .vscode/launch.json diff --git a/.vscode/launch.json b/.vscode/launch.json new file mode 100644 index 00000000..5c7247b4 --- /dev/null +++ b/.vscode/launch.json @@ -0,0 +1,7 @@ +{ + // Use IntelliSense to learn about possible attributes. + // Hover to view descriptions of existing attributes. + // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387 + "version": "0.2.0", + "configurations": [] +} \ No newline at end of file From f1b90eaf0f02bf5d9e04cb757704de31411f0f6f Mon Sep 17 00:00:00 2001 From: "paramdeep.saini@oracle.com" Date: Tue, 19 Apr 2022 14:47:33 -0700 Subject: [PATCH 237/628] Added fix for static check analaysis --- .vscode/launch.json | 7 ------- 1 file changed, 7 deletions(-) delete mode 100644 .vscode/launch.json diff --git a/.vscode/launch.json b/.vscode/launch.json deleted file mode 100644 index 5c7247b4..00000000 --- a/.vscode/launch.json +++ /dev/null @@ -1,7 +0,0 @@ -{ - // Use IntelliSense to learn about possible attributes. - // Hover to view descriptions of existing attributes. - // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387 - "version": "0.2.0", - "configurations": [] -} \ No newline at end of file From 793e0e36cfc18ef4e29c8f39a830fe5fbe4cca04 Mon Sep 17 00:00:00 2001 From: "paramdeep.saini@oracle.com" Date: Tue, 19 Apr 2022 15:25:03 -0700 Subject: [PATCH 238/628] Added fix for static check analaysis --- commons/sharding/exec.go | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/commons/sharding/exec.go b/commons/sharding/exec.go index e368bf74..24e912b6 100644 --- a/commons/sharding/exec.go +++ b/commons/sharding/exec.go @@ -40,9 +40,10 @@ package commons import ( "bytes" - databasealphav1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" "net/http" + databasealphav1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + "github.com/go-logr/logr" corev1 "k8s.io/api/core/v1" "k8s.io/client-go/kubernetes" @@ -52,7 +53,7 @@ import ( ) // ExecCMDInContainer execute command in first container of a pod -func ExecCommand(podName string, cmd []string, kubeClient kubernetes.Interface, kubeConfig clientcmd.ClientConfig, instance *databasealphav1.ShardingDatabase, logger logr.Logger) (error, string, string) { +func ExecCommand(podName string, cmd []string, kubeClient kubernetes.Interface, kubeConfig clientcmd.ClientConfig, instance *databasealphav1.ShardingDatabase, logger logr.Logger) (string, string, error) { var msg string var ( @@ -75,7 +76,7 @@ func ExecCommand(podName string, cmd []string, kubeClient kubernetes.Interface, config, err := kubeConfig.ClientConfig() if err != nil { - return err, "Error Occurred", "Error Occurred" + return "Error Occurred", "Error Occurred", err } // Connect to url (constructed from req) using SPDY (HTTP/2) protocol which allows bidirectional streams. @@ -83,7 +84,7 @@ func ExecCommand(podName string, cmd []string, kubeClient kubernetes.Interface, if err != nil { msg = "Error after executing remotecommand.NewSPDYExecutor" LogMessages("Error", msg, err, instance, logger) - return err, "Error Occurred", "Error Occurred" + return "Error Occurred", "Error Occurred", err } err = exec.Stream(remotecommand.StreamOptions{ @@ -100,8 +101,8 @@ func ExecCommand(podName string, cmd []string, kubeClient kubernetes.Interface, if len(execErr.String()) > 0 { LogMessages("INFO", execErr.String(), nil, instance, logger) } - return err, execOut.String(), execErr.String() + return execOut.String(), execErr.String(), err } - return nil, execOut.String(), execErr.String() + return execOut.String(), execErr.String(), nil } From 2897ef7a317566129ea653c39d3b64a6cc5b2f74 Mon Sep 17 00:00:00 2001 From: "paramdeep.saini@oracle.com" Date: Tue, 19 Apr 2022 15:49:27 -0700 Subject: [PATCH 239/628] Added Static Check fix --- controllers/database/shardingdatabase_controller.go | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/controllers/database/shardingdatabase_controller.go b/controllers/database/shardingdatabase_controller.go index 91964906..6f2a54cd 100644 --- a/controllers/database/shardingdatabase_controller.go +++ b/controllers/database/shardingdatabase_controller.go @@ -122,6 +122,7 @@ func (r *ShardingDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.Req var isShardTopologyDeleteTrue bool = false //var msg string var err error + var idx int var stateType string resultNq := ctrl.Result{Requeue: false} resultQ := ctrl.Result{Requeue: true, RequeueAfter: 30 * time.Second} @@ -152,7 +153,7 @@ func (r *ShardingDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.Req return ctrl.Result{}, err } - idx, instFlag := r.checkProvInstance(instance) + _, instFlag := r.checkProvInstance(instance) // assinging osh instance if !instFlag { // Sharding Topolgy Struct Assignment @@ -564,7 +565,7 @@ func (r *ShardingDatabaseReconciler) finalizerShardingDatabaseInstance(instance } // Send true because delete is in progress and it is a custom delete message // We don't need to print custom err stack as we are deleting the topology - return fmt.Errorf("Delete of the sharding topology is in progress"), true + return fmt.Errorf("delete of the sharding topology is in progress"), true } // Add finalizer for this CR @@ -874,15 +875,15 @@ func (r *ShardingDatabaseReconciler) validateSpex(instance *databasev1alpha1.Sha // Compare Env variables for shard begins here if !r.comapreShardEnvVariables(instance, lastSuccSpec) { - return fmt.Errorf("Change of Shard env variables are not") + return fmt.Errorf("change of Shard env variables are not") } // Compare Env variables for catalog begins here if !r.comapreCatalogEnvVariables(instance, lastSuccSpec) { - return fmt.Errorf("Change of Catalog env variables are not") + return fmt.Errorf("change of Catalog env variables are not") } // Compare env variable for Catalog ends here if !r.comapreGsmEnvVariables(instance, lastSuccSpec) { - return fmt.Errorf("Change of GSM env variables are not") + return fmt.Errorf("change of GSM env variables are not") } } @@ -1072,7 +1073,7 @@ func (r *ShardingDatabaseReconciler) validateInvidualGsm(instance *databasev1alp msg = "Unable to validate GSM " + shardingv1.GetFmtStr(gsmPod.Name) + " pod. GSM pod doesn't seems to be ready to accept the commands." shardingv1.LogMessages("INFO", msg, nil, instance, r.Log) r.updateGsmStatus(instance, int(i), string(databasev1alpha1.PodNotReadyState)) - return gsmSfSet, gsmPod, fmt.Errorf("Pod doesn't exist") + return gsmSfSet, gsmPod, fmt.Errorf("pod doesn't exist") } err = shardingv1.CheckGsmStatus(gsmPod.Name, instance, r.kubeClient, r.kubeConfig, r.Log) if err != nil { From ef804bbbce7b1ea04698210cd727014912ce8575 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Wed, 20 Apr 2022 16:48:40 +0530 Subject: [PATCH 240/628] Prebuilt DB code refactor --- .../v1alpha1/singleinstancedatabase_types.go | 4 +- .../singleinstancedatabase_webhook.go | 7 + commons/database/constants.go | 4 + .../singleinstancedatabase_controller.go | 202 ++++-------------- 4 files changed, 60 insertions(+), 157 deletions(-) diff --git a/apis/database/v1alpha1/singleinstancedatabase_types.go b/apis/database/v1alpha1/singleinstancedatabase_types.go index 490d7b4a..fde6059e 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_types.go +++ b/apis/database/v1alpha1/singleinstancedatabase_types.go @@ -81,8 +81,8 @@ type SingleInstanceDatabaseSpec struct { // SingleInstanceDatabasePersistence defines the storage size and class for PVC type SingleInstanceDatabasePersistence struct { - Size string `json:"size"` - StorageClass string `json:"storageClass"` + Size string `json:"size,omitempty"` + StorageClass string `json:"storageClass,omitempty"` AccessMode string `json:"accessMode,omitempty"` } diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index e268be55..b22e31c3 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -112,6 +112,13 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { } } + // Persistence spec validation + if r.Spec.Persistence.Size == "" && (r.Spec.Persistence.AccessMode != "" || r.Spec.Persistence.StorageClass != "" ) || + (r.Spec.Persistence.Size != "" && r.Spec.Persistence.AccessMode == "" ) { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("persistence"), r.Spec.Replicas, "invalid persistence spec")) + } + // Pre-built db if r.Spec.Image.PrebuiltDB { if r.Spec.Pdbname != "" && r.Spec.Edition != "express" { diff --git a/commons/database/constants.go b/commons/database/constants.go index db061ad3..4e8f8b94 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -349,6 +349,10 @@ const WalletEntriesCMD string = "umask 177\ncat > wallet.passwd < /dev/null; do sleep 0.5; done; fi " +const InitPrebuiltDbCMD string = "if [ ! -d /mnt/oradata/${ORACLE_SID} ]; then" + + " cp -vr $ORACLE_BASE/oradata/${ORACLE_SID} /mnt/oradata && cp -vr $ORACLE_BASE/oradata/dbconfig /mnt/oradata; fi " + + const AlterSgaPgaCpuCMD string = "echo -e \"alter system set sga_target=%dM scope=both; \n alter system set pga_aggregate_target=%dM scope=both; \n alter system set cpu_count=%d; \" | %s " const AlterProcessesCMD string = "echo -e \"alter system set processes=%d scope=spfile; \" | %s && " + CreateChkFileCMD + " && " + diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 44e0ee1e..8718d226 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -438,128 +438,8 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab // Instantiate POD spec from SingleInstanceDatabase spec //############################################################################# func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleInstanceDatabase, n *dbapi.SingleInstanceDatabase) *corev1.Pod { - // prebuiltDB, useful for dev/test/CI-CD - if m.Spec.Image.PrebuiltDB { - pod := &corev1.Pod{ - TypeMeta: metav1.TypeMeta{ - Kind: "Pod", - }, - ObjectMeta: metav1.ObjectMeta{ - Name: m.Name + "-" + dbcommons.GenerateRandomString(5), - Namespace: m.Namespace, - Labels: map[string]string{ - "app": m.Name, - "version": m.Spec.Image.Version, - }, - }, - Spec: corev1.PodSpec{ - Volumes: []corev1.Volume{{ - Name: "oracle-pwd-vol", - VolumeSource: corev1.VolumeSource{ - Secret: &corev1.SecretVolumeSource{ - SecretName: m.Spec.AdminPassword.SecretName, - Optional: func() *bool { i := true; return &i }(), - Items: []corev1.KeyToPath{{ - Key: m.Spec.AdminPassword.SecretKey, - Path: "oracle_pwd", - }, - }, - }, - }, - }}, - Containers: []corev1.Container{{ - Name: m.Name, - Image: m.Spec.Image.PullFrom, - Lifecycle: &corev1.Lifecycle{ - PreStop: &corev1.LifecycleHandler{ - Exec: &corev1.ExecAction{ - Command: []string{"/bin/sh", "-c", "/bin/echo -en 'shutdown abort;\n' | env ORACLE_SID=${ORACLE_SID^^} sqlplus -S / as sysdba"}, - }, - }, - }, - ImagePullPolicy: corev1.PullAlways, - Ports: []corev1.ContainerPort{{ContainerPort: 1521}, {ContainerPort: 5500}}, - - ReadinessProbe: &corev1.Probe{ - ProbeHandler: corev1.ProbeHandler{ - Exec: &corev1.ExecAction{ - Command: []string{"/bin/sh", "-c", "if [ -f $ORACLE_BASE/checkDBLockStatus.sh ]; then $ORACLE_BASE/checkDBLockStatus.sh ; else $ORACLE_BASE/checkDBStatus.sh; fi "}, - }, - }, - InitialDelaySeconds: 20, - TimeoutSeconds: 20, - PeriodSeconds: func() int32 { - if m.Spec.ReadinessCheckPeriod > 0 { - return int32(m.Spec.ReadinessCheckPeriod) - } - return 30 - }(), - }, - VolumeMounts: []corev1.VolumeMount{{ - MountPath: "/run/secrets/oracle_pwd", - ReadOnly: true, - Name: "oracle-pwd-vol", - SubPath: "oracle_pwd", - }}, - Env: func() []corev1.EnvVar { - return []corev1.EnvVar{ - { - Name: "SVC_HOST", - Value: m.Name, - }, - { - Name: "SVC_PORT", - Value: "1521", - }, - { - Name: "SKIP_DATAPATCH", - Value: "true", - }, - } - }(), - }}, - - TerminationGracePeriodSeconds: func() *int64 { i := int64(30); return &i }(), - - NodeSelector: func() map[string]string { - ns := make(map[string]string) - if len(m.Spec.NodeSelector) != 0 { - for key, value := range m.Spec.NodeSelector { - ns[key] = value - } - } - return ns - }(), - ServiceAccountName: func() string { - if m.Spec.ServiceAccountName != "" { - return m.Spec.ServiceAccountName - } - return "default" - }(), - SecurityContext: &corev1.PodSecurityContext{ - RunAsUser: func() *int64 { - i := int64(dbcommons.ORACLE_UID) - return &i - }(), - RunAsGroup: func() *int64 { - i := int64(dbcommons.ORACLE_GUID) - return &i - }(), - }, - ImagePullSecrets: []corev1.LocalObjectReference{ - { - Name: m.Spec.Image.PullSecrets, - }, - }, - }, - } - - // Set SingleInstanceDatabase instance as the owner and controller - ctrl.SetControllerReference(m, pod, r.Scheme) - return pod - - } - // If not prebuiltDB + + // POD spec pod := &corev1.Pod{ TypeMeta: metav1.TypeMeta{ Kind: "Pod", @@ -575,12 +455,18 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns Spec: corev1.PodSpec{ Volumes: []corev1.Volume{{ Name: "datamount", - VolumeSource: corev1.VolumeSource{ - PersistentVolumeClaim: &corev1.PersistentVolumeClaimVolumeSource{ - ClaimName: m.Name, - ReadOnly: false, - }, - }, + VolumeSource: func() corev1.VolumeSource { + if m.Spec.Persistence.Size == "" { + return corev1.VolumeSource{EmptyDir: &corev1.EmptyDirVolumeSource{}} + } + /* Persistence is specified */ + return corev1.VolumeSource{ + PersistentVolumeClaim: &corev1.PersistentVolumeClaimVolumeSource{ + ClaimName: m.Name, + ReadOnly: false, + }, + } + }(), }, { Name: "oracle-pwd-vol", VolumeSource: corev1.VolumeSource{ @@ -595,20 +481,37 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, }}, InitContainers: func() []corev1.Container { - if m.Spec.Edition != "express" { - return []corev1.Container{{ - Name: "init-permissions", + initContainers := []corev1.Container {{ + Name: "init-permissions", + Image: m.Spec.Image.PullFrom, + Command: []string{"/bin/sh", "-c", fmt.Sprintf("chown %d:%d /opt/oracle/oradata", int(dbcommons.ORACLE_UID), int(dbcommons.ORACLE_GUID))}, + SecurityContext: &corev1.SecurityContext{ + // User ID 0 means, root user + RunAsUser: func() *int64 { i := int64(0); return &i }(), + }, + VolumeMounts: []corev1.VolumeMount{{ + MountPath: "/opt/oracle/oradata", + Name: "datamount", + }}, + }} + if m.Spec.Image.PrebuiltDB && m.Spec.Persistence.Size != "" { + initContainers = append(initContainers, corev1.Container{ + Name: "init-prebuiltdb", Image: m.Spec.Image.PullFrom, - Command: []string{"/bin/sh", "-c", fmt.Sprintf("chown %d:%d /opt/oracle/oradata", int(dbcommons.ORACLE_UID), int(dbcommons.ORACLE_GUID))}, - SecurityContext: &corev1.SecurityContext{ - // User ID 0 means, root user - RunAsUser: func() *int64 { i := int64(0); return &i }(), + Command: []string{"/bin/sh", "-c", dbcommons.InitPrebuiltDbCMD}, + SecurityContext: &corev1.SecurityContext { + RunAsUser: func() *int64 { i := int64(dbcommons.ORACLE_UID); return &i }(), + RunAsGroup: func() *int64 { i := int64(dbcommons.ORACLE_GUID); return &i }(), }, VolumeMounts: []corev1.VolumeMount{{ - MountPath: "/opt/oracle/oradata", + MountPath: "/mnt/oradata", Name: "datamount", }}, - }, { + }) + } + /* Wallet only for non-express edition, non-prebuiltDB */ + if m.Spec.Edition != "express" && !m.Spec.Image.PrebuiltDB { + initContainers = append(initContainers, corev1.Container{ Name: "init-wallet", Image: m.Spec.Image.PullFrom, Env: []corev1.EnvVar{ @@ -653,21 +556,9 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns MountPath: "/opt/oracle/oradata", Name: "datamount", }}, - }} + }) } - return []corev1.Container{{ - Name: "init-permissions", - Image: m.Spec.Image.PullFrom, - Command: []string{"/bin/sh", "-c", fmt.Sprintf("chown %d:%d /opt/oracle/oradata", int(dbcommons.ORACLE_UID), int(dbcommons.ORACLE_GUID))}, - SecurityContext: &corev1.SecurityContext{ - // User ID 0 means, root user - RunAsUser: func() *int64 { i := int64(0); return &i }(), - }, - VolumeMounts: []corev1.VolumeMount{{ - MountPath: "/opt/oracle/oradata", - Name: "datamount", - }}, - }} + return initContainers }(), Containers: []corev1.Container{{ Name: m.Name, @@ -703,9 +594,10 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns MountPath: "/opt/oracle/oradata", Name: "datamount", }} - if m.Spec.Edition == "express" { + if m.Spec.Edition == "express" || m.Spec.Image.PrebuiltDB { + // mounts pwd as secrets for express edition + // or prebuilt db mounts = append(mounts, corev1.VolumeMount { - // This is for express edition DB MountPath: "/run/secrets/oracle_pwd", ReadOnly: true, Name: "oracle-pwd-vol", @@ -967,8 +859,8 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePVCSpec(m *dbapi.SingleIns func (r *SingleInstanceDatabaseReconciler) createOrReplacePVC(ctx context.Context, req ctrl.Request, m *dbapi.SingleInstanceDatabase) (ctrl.Result, error) { - // No PVC for Pre-built db - if m.Spec.Image.PrebuiltDB { + // Don't create PVC if persistence is not chosen + if m.Spec.Persistence.Size == "" { return requeueN, nil } log := r.Log.WithValues("createPVC", req.NamespacedName) From 79d87b8276e70835197c67dd2454a9e4cc87b1c6 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Wed, 20 Apr 2022 17:57:09 +0530 Subject: [PATCH 241/628] Update webhook validations for prebuiltdb --- .../v1alpha1/singleinstancedatabase_webhook.go | 9 +++------ .../database/singleinstancedatabase_controller.go | 10 +--------- 2 files changed, 4 insertions(+), 15 deletions(-) diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index b22e31c3..4fbc4745 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -116,16 +116,12 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { if r.Spec.Persistence.Size == "" && (r.Spec.Persistence.AccessMode != "" || r.Spec.Persistence.StorageClass != "" ) || (r.Spec.Persistence.Size != "" && r.Spec.Persistence.AccessMode == "" ) { allErrs = append(allErrs, - field.Invalid(field.NewPath("spec").Child("persistence"), r.Spec.Replicas, "invalid persistence spec")) + field.Invalid(field.NewPath("spec").Child("persistence"), r.Spec.Replicas, + "invalid specification, size and/or accessMode missing")) } // Pre-built db if r.Spec.Image.PrebuiltDB { - if r.Spec.Pdbname != "" && r.Spec.Edition != "express" { - allErrs = append(allErrs, - field.Invalid(field.NewPath("spec").Child("pdbName"), r.Spec.Pdbname, - "cannot change pdbName for prebuilt db")) - } if r.Spec.CloneFrom != "" && r.Spec.Edition != "express" { allErrs = append(allErrs, field.Invalid(field.NewPath("spec").Child("cloneFrom"), r.Spec.CloneFrom, @@ -176,6 +172,7 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { if len(allErrs) == 0 { return nil } + return apierrors.NewInvalid( schema.GroupKind{Group: "database.oracle.com", Kind: "SingleInstanceDatabase"}, r.Name, allErrs) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 8718d226..00832fab 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -352,7 +352,7 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab if m.Spec.Edition == "express" && m.Spec.CloneFrom != "" { eventMsgs = append(eventMsgs, "cloning not supported for express edition") } - if m.Status.OrdsReference != "" && m.Status.Persistence.AccessMode != "" && m.Status.Persistence != m.Spec.Persistence { + if m.Status.OrdsReference != "" && m.Status.Persistence.Size != "" && m.Status.Persistence != m.Spec.Persistence { eventMsgs = append(eventMsgs, "uninstall ORDS to change Peristence") } if len(eventMsgs) > 0 { @@ -1517,10 +1517,6 @@ func (r *SingleInstanceDatabaseReconciler) updateInitParameters(m *dbapi.SingleI readyPod corev1.Pod, ctx context.Context, req ctrl.Request) (ctrl.Result, error) { log := r.Log.WithValues("updateInitParameters", req.NamespacedName) - // No InitParameters Updation for Pre-built db - if m.Spec.Image.PrebuiltDB { - return requeueN, nil - } if m.Status.InitParams == m.Spec.InitParams { return requeueN, nil @@ -1567,10 +1563,6 @@ func (r *SingleInstanceDatabaseReconciler) updateDBConfig(m *dbapi.SingleInstanc log := r.Log.WithValues("updateDBConfig", req.NamespacedName) - // No updateDBConfig for Pre-built db - if m.Spec.Image.PrebuiltDB { - return requeueN, nil - } m.Status.Status = dbcommons.StatusUpdating r.Status().Update(ctx, m) From 53976b5a95a17378af35cbdf107907631ae957dd Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Wed, 20 Apr 2022 19:06:08 +0530 Subject: [PATCH 242/628] Update config samples --- .../sidb/singleinstancedatabase_clone.yaml | 25 +++++++--- ...aml => singleinstancedatabase_create.yaml} | 21 ++++++-- .../sidb/singleinstancedatabase_minikube.yaml | 41 +++++---------- .../singleinstancedatabase_prebuiltdb.yaml | 50 +++++++++++++++++++ ...ngleinstancedatabase_prov_prebuilt_db.yaml | 22 -------- 5 files changed, 97 insertions(+), 62 deletions(-) rename config/samples/sidb/{singleinstancedatabase_prov.yaml => singleinstancedatabase_create.yaml} (75%) create mode 100644 config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml delete mode 100644 config/samples/sidb/singleinstancedatabase_prov_prebuilt_db.yaml diff --git a/config/samples/sidb/singleinstancedatabase_clone.yaml b/config/samples/sidb/singleinstancedatabase_clone.yaml index a29dce78..65a3b4d4 100644 --- a/config/samples/sidb/singleinstancedatabase_clone.yaml +++ b/config/samples/sidb/singleinstancedatabase_clone.yaml @@ -1,7 +1,18 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # + +apiVersion: v1 +kind: Secret +metadata: + name: admin-secret +type: Opaque +stringData: + oracle_pwd: "ChangeOnInstall_1" + +--- + apiVersion: database.oracle.com/v1alpha1 kind: SingleInstanceDatabase metadata: @@ -10,26 +21,26 @@ metadata: spec: ## Use only alphanumeric characters for sid - sid: ORCL1 + sid: ORCL2 ## A source database ref to clone from. ## If cloning from on prem source database, mention connect string `:/`. - cloneFrom: "" + cloneFrom: sidb-sample ## Should refer to SourceDB secret ## Secret containing SIDB password mapped to secretKey ## This secret will be deleted after creation of the database unless keepSecret is set to true adminPassword: - secretName: - secretKey: + secretName: admin-secret + secretKey: oracle_pwd keepSecret: true ## Database image details ## Database can be patched out of place by updating the RU version/image ## Major version changes are not supported image: - pullFrom: - pullSecrets: + pullFrom: container-registry.oracle.com/database/enterprise:latest + pullSecrets: oracle-container-registry-secret ## size : Minimum size of pvc | class : PVC storage Class ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany diff --git a/config/samples/sidb/singleinstancedatabase_prov.yaml b/config/samples/sidb/singleinstancedatabase_create.yaml similarity index 75% rename from config/samples/sidb/singleinstancedatabase_prov.yaml rename to config/samples/sidb/singleinstancedatabase_create.yaml index 616d2528..071404db 100644 --- a/config/samples/sidb/singleinstancedatabase_prov.yaml +++ b/config/samples/sidb/singleinstancedatabase_create.yaml @@ -1,7 +1,18 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # + +apiVersion: v1 +kind: Secret +metadata: + name: admin-secret +type: Opaque +stringData: + oracle_pwd: "ChangeOnInstall_1" + +--- + apiVersion: database.oracle.com/v1alpha1 kind: SingleInstanceDatabase metadata: @@ -15,14 +26,14 @@ spec: ## Secret containing SIDB password mapped to secretKey ## This secret will be deleted after creation of the database unless keepSecret is set to true adminPassword: - secretName: - secretKey: + secretName: admin-secret + secretKey: oracle_pwd keepSecret: true ## Database Image image: - pullFrom: - pullSecrets: + pullFrom: container-registry.oracle.com/database/enterprise:latest + pullSecrets: oracle-container-registry-secret ## size : Minimum size of pvc | class : PVC storage Class ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany diff --git a/config/samples/sidb/singleinstancedatabase_minikube.yaml b/config/samples/sidb/singleinstancedatabase_minikube.yaml index a3a4824c..14d7d71e 100644 --- a/config/samples/sidb/singleinstancedatabase_minikube.yaml +++ b/config/samples/sidb/singleinstancedatabase_minikube.yaml @@ -1,28 +1,28 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: v1 kind: Secret metadata: - name: db-secret + name: admin-secret type: Opaque stringData: - oracle_pwd: "Change_On_Install_1" + oracle_pwd: "ChangeOnInstall_1" --- apiVersion: v1 kind: PersistentVolume metadata: - name: sidb-pv + name: xe-pv spec: accessModes: - ReadWriteMany capacity: storage: 5Gi - storageClassName: sidb + storageClassName: xe-sc hostPath: path: /data/oradata @@ -31,33 +31,27 @@ spec: apiVersion: database.oracle.com/v1alpha1 kind: SingleInstanceDatabase metadata: - name: sidb-sample + name: xe-sample namespace: default spec: ## Use only alphanumeric characters for sid - sid: ORCL1 + sid: XE - ## A source database ref to clone from, leave empty to create a fresh database - cloneFrom: "" - ## NA if cloning from a SourceDB (cloneFrom is set) - edition: enterprise + edition: express ## Should refer to SourceDB secret if cloning from a SourceDB (cloneFrom is set) ## Secret containing SIDB password mapped to secretKey ## This secret will be deleted after creation of the database unless keepSecret is set to true adminPassword: - secretName: db-secret + secretName: admin-secret secretKey: oracle_pwd keepSecret: true ## NA if cloning from a SourceDB (cloneFrom is set) charset: AL32UTF8 - ## NA if cloning from a SourceDB (cloneFrom is set) - pdbName: orclpdb1 - ## Enable/Disable Flashback flashBack: false @@ -67,28 +61,19 @@ spec: ## Enable/Disable ForceLogging forceLog: false - ## NA if cloning from a SourceDB (cloneFrom is set) - ## Specify both sgaSize and pgaSize (in MB) or dont specify both - ## Specify Non-Zero value to use - initParams: - cpuCount: 0 - processes: 0 - sgaTarget: 0 - pgaAggregateTarget: 0 - ## Database image details ## Database can be patched by updating the RU version/image ## Major version changes are not supported image: - pullFrom: container-registry.oracle.com/database/enterprise:latest - pullSecrets: oracle-container-registry-secret + pullFrom: container-registry.oracle.com/database/express:latest + prebuiltDB: true ## size : Minimum size of pvc | class : PVC storage Class ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany persistence: size: 5Gi - storageClass: "sidb" - accessMode: "ReadWriteMany" + storageClass: "xe-sc" + accessMode: "ReadWriteOnce" ## Count of Database Pods. Applicable only for "ReadWriteMany" AccessMode replicas: 1 diff --git a/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml b/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml new file mode 100644 index 00000000..dd81606e --- /dev/null +++ b/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml @@ -0,0 +1,50 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +apiVersion: v1 +kind: Secret +metadata: + name: admin-secret +type: Opaque +stringData: + oracle_pwd: "ChangeOnInstall_1" + +--- + +apiVersion: database.oracle.com/v1alpha1 +kind: SingleInstanceDatabase +metadata: + name: xe-prebuilt-sample + namespace: default +spec: + + ## Use only alphanumeric characters for sid + sid: XE + + ## Secret containing SIDB password mapped to secretKey + ## This secret will be deleted after creation of the database unless keepSecret is set to true + adminPassword: + secretName: admin-secret + secretKey: oracle_pwd + keepSecret: true + + ## Database Image + image: + pullFrom: container-registry.oracle.com/database/express:latest + prebuiltDB: true + + ## size : Minimum size of pvc | class : PVC storage Class + ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany + ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. + persistence: + size: 100Gi + storageClass: "oci" + accessMode: "ReadWriteOnce" + + ## If deploying on OpenShift, change service account name to 'sidb-sa' after you run `$ oc apply -f openshift_rbac.yaml` + serviceAccountName: default + + ## Count of Database Pods. Applicable only for "ReadWriteMany" AccessMode + replicas: 1 diff --git a/config/samples/sidb/singleinstancedatabase_prov_prebuilt_db.yaml b/config/samples/sidb/singleinstancedatabase_prov_prebuilt_db.yaml deleted file mode 100644 index a24f51af..00000000 --- a/config/samples/sidb/singleinstancedatabase_prov_prebuilt_db.yaml +++ /dev/null @@ -1,22 +0,0 @@ -# -# Copyright (c) 2021, Oracle and/or its affiliates. -# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. -# -apiVersion: database.oracle.com/v1alpha1 -kind: SingleInstanceDatabase -metadata: - name: sidb-sample-prebuiltdb - namespace: default -spec: - - ## Pre-built Database Image - image: - pullFrom: - pullSecrets: - - ## If deploying on OpenShift, change service account name to 'sidb-sa' after you run `$ oc apply -f openshift_rbac.yaml` - serviceAccountName: default - - ## Type of service . Applicable on cloud enviroments only - ## if loadBalService : false, service type = "NodePort". else "LoadBalancer" - loadBalancer: false \ No newline at end of file From bfb09140eb162c4a867838b572a63a3b17695898 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Thu, 21 Apr 2022 22:28:11 +0530 Subject: [PATCH 243/628] Update samples --- .../singleinstancedatabase_webhook.go | 2 +- .../sidb/singleinstancedatabase_clone.yaml | 6 +-- .../sidb/singleinstancedatabase_create.yaml | 4 +- .../sidb/singleinstancedatabase_minikube.yaml | 6 +-- .../sidb/singleinstancedatabase_patch.yaml | 40 ------------------- .../singleinstancedatabase_prebuiltdb.yaml | 6 +-- 6 files changed, 12 insertions(+), 52 deletions(-) delete mode 100644 config/samples/sidb/singleinstancedatabase_patch.yaml diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index 4fbc4745..6a8aced7 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -116,7 +116,7 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { if r.Spec.Persistence.Size == "" && (r.Spec.Persistence.AccessMode != "" || r.Spec.Persistence.StorageClass != "" ) || (r.Spec.Persistence.Size != "" && r.Spec.Persistence.AccessMode == "" ) { allErrs = append(allErrs, - field.Invalid(field.NewPath("spec").Child("persistence"), r.Spec.Replicas, + field.Invalid(field.NewPath("spec").Child("persistence"), r.Spec.Persistence, "invalid specification, size and/or accessMode missing")) } diff --git a/config/samples/sidb/singleinstancedatabase_clone.yaml b/config/samples/sidb/singleinstancedatabase_clone.yaml index 65a3b4d4..4845c89e 100644 --- a/config/samples/sidb/singleinstancedatabase_clone.yaml +++ b/config/samples/sidb/singleinstancedatabase_clone.yaml @@ -6,10 +6,10 @@ apiVersion: v1 kind: Secret metadata: - name: admin-secret + name: admin-secret-2 type: Opaque stringData: - oracle_pwd: "ChangeOnInstall_1" + oracle_pwd: "ChangeOnInstall_2" --- @@ -31,7 +31,7 @@ spec: ## Secret containing SIDB password mapped to secretKey ## This secret will be deleted after creation of the database unless keepSecret is set to true adminPassword: - secretName: admin-secret + secretName: admin-secret-2 secretKey: oracle_pwd keepSecret: true diff --git a/config/samples/sidb/singleinstancedatabase_create.yaml b/config/samples/sidb/singleinstancedatabase_create.yaml index 071404db..69680826 100644 --- a/config/samples/sidb/singleinstancedatabase_create.yaml +++ b/config/samples/sidb/singleinstancedatabase_create.yaml @@ -6,7 +6,7 @@ apiVersion: v1 kind: Secret metadata: - name: admin-secret + name: admin-secret-1 type: Opaque stringData: oracle_pwd: "ChangeOnInstall_1" @@ -26,7 +26,7 @@ spec: ## Secret containing SIDB password mapped to secretKey ## This secret will be deleted after creation of the database unless keepSecret is set to true adminPassword: - secretName: admin-secret + secretName: admin-secret-1 secretKey: oracle_pwd keepSecret: true diff --git a/config/samples/sidb/singleinstancedatabase_minikube.yaml b/config/samples/sidb/singleinstancedatabase_minikube.yaml index 14d7d71e..14b24f93 100644 --- a/config/samples/sidb/singleinstancedatabase_minikube.yaml +++ b/config/samples/sidb/singleinstancedatabase_minikube.yaml @@ -6,10 +6,10 @@ apiVersion: v1 kind: Secret metadata: - name: admin-secret + name: admin-secret-3 type: Opaque stringData: - oracle_pwd: "ChangeOnInstall_1" + oracle_pwd: "ChangeOnInstall_3" --- @@ -45,7 +45,7 @@ spec: ## Secret containing SIDB password mapped to secretKey ## This secret will be deleted after creation of the database unless keepSecret is set to true adminPassword: - secretName: admin-secret + secretName: admin-secret-3 secretKey: oracle_pwd keepSecret: true diff --git a/config/samples/sidb/singleinstancedatabase_patch.yaml b/config/samples/sidb/singleinstancedatabase_patch.yaml deleted file mode 100644 index 173e25b2..00000000 --- a/config/samples/sidb/singleinstancedatabase_patch.yaml +++ /dev/null @@ -1,40 +0,0 @@ -# -# Copyright (c) 2021, Oracle and/or its affiliates. -# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. -# -apiVersion: database.oracle.com/v1alpha1 -kind: SingleInstanceDatabase -metadata: - name: sidb-sample-patch - namespace: default -spec: - - ## Use only alphanumeric characters for sid - sid: ORCL1 - - ## Secret containing SIDB password mapped to secretKey - ## This secret will be deleted after creation of the database unless keepSecret is set to true - adminPassword: - secretName: - secretKey: - keepSecret: true - - ## Patch the database by updating the RU version/image - ## Major version changes are not supported - image: - pullFrom: - pullSecrets: - - ## size : Minimum size of pvc | class : PVC storage Class - ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany - ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. - persistence: - size: 100Gi - storageClass: "oci" - accessMode: "ReadWriteOnce" - - ## If deploying on OpenShift, change service account name to 'sidb-sa' after you run `$ oc apply -f openshift_rbac.yaml` - serviceAccountName: default - - ## Count of Database Pods. Applicable only for "ReadWriteMany" AccessMode - replicas: 1 diff --git a/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml b/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml index dd81606e..a51b683f 100644 --- a/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml +++ b/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml @@ -6,10 +6,10 @@ apiVersion: v1 kind: Secret metadata: - name: admin-secret + name: admin-secret-4 type: Opaque stringData: - oracle_pwd: "ChangeOnInstall_1" + oracle_pwd: "ChangeOnInstall_4" --- @@ -26,7 +26,7 @@ spec: ## Secret containing SIDB password mapped to secretKey ## This secret will be deleted after creation of the database unless keepSecret is set to true adminPassword: - secretName: admin-secret + secretName: admin-secret-4 secretKey: oracle_pwd keepSecret: true From 82cb3d12c97da026a34bffa3bee0d6482d9f3d91 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Thu, 21 Apr 2022 23:05:55 +0530 Subject: [PATCH 244/628] Update POD Spec --- .../singleinstancedatabase_controller.go | 56 ++++++++++--------- 1 file changed, 31 insertions(+), 25 deletions(-) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 00832fab..356f8337 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -481,33 +481,36 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, }}, InitContainers: func() []corev1.Container { - initContainers := []corev1.Container {{ - Name: "init-permissions", - Image: m.Spec.Image.PullFrom, - Command: []string{"/bin/sh", "-c", fmt.Sprintf("chown %d:%d /opt/oracle/oradata", int(dbcommons.ORACLE_UID), int(dbcommons.ORACLE_GUID))}, - SecurityContext: &corev1.SecurityContext{ - // User ID 0 means, root user - RunAsUser: func() *int64 { i := int64(0); return &i }(), - }, - VolumeMounts: []corev1.VolumeMount{{ - MountPath: "/opt/oracle/oradata", - Name: "datamount", - }}, - }} - if m.Spec.Image.PrebuiltDB && m.Spec.Persistence.Size != "" { - initContainers = append(initContainers, corev1.Container{ - Name: "init-prebuiltdb", + initContainers := []corev1.Container {} + if m.Spec.Persistence.Size != "" { + initContainers = append(initContainers, corev1.Container { + Name: "init-permissions", Image: m.Spec.Image.PullFrom, - Command: []string{"/bin/sh", "-c", dbcommons.InitPrebuiltDbCMD}, - SecurityContext: &corev1.SecurityContext { - RunAsUser: func() *int64 { i := int64(dbcommons.ORACLE_UID); return &i }(), - RunAsGroup: func() *int64 { i := int64(dbcommons.ORACLE_GUID); return &i }(), + Command: []string{"/bin/sh", "-c", fmt.Sprintf("chown %d:%d /opt/oracle/oradata", int(dbcommons.ORACLE_UID), int(dbcommons.ORACLE_GUID))}, + SecurityContext: &corev1.SecurityContext{ + // User ID 0 means, root user + RunAsUser: func() *int64 { i := int64(0); return &i }(), }, VolumeMounts: []corev1.VolumeMount{{ - MountPath: "/mnt/oradata", + MountPath: "/opt/oracle/oradata", Name: "datamount", }}, }) + if m.Spec.Image.PrebuiltDB { + initContainers = append(initContainers, corev1.Container { + Name: "init-prebuiltdb", + Image: m.Spec.Image.PullFrom, + Command: []string{"/bin/sh", "-c", dbcommons.InitPrebuiltDbCMD}, + SecurityContext: &corev1.SecurityContext { + RunAsUser: func() *int64 { i := int64(dbcommons.ORACLE_UID); return &i }(), + RunAsGroup: func() *int64 { i := int64(dbcommons.ORACLE_GUID); return &i }(), + }, + VolumeMounts: []corev1.VolumeMount{{ + MountPath: "/mnt/oradata", + Name: "datamount", + }}, + }) + } } /* Wallet only for non-express edition, non-prebuiltDB */ if m.Spec.Edition != "express" && !m.Spec.Image.PrebuiltDB { @@ -590,10 +593,13 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, VolumeMounts: func() []corev1.VolumeMount { - mounts := []corev1.VolumeMount{ { - MountPath: "/opt/oracle/oradata", - Name: "datamount", - }} + mounts := []corev1.VolumeMount{} + if m.Spec.Persistence.Size != "" { + mounts = append(mounts, corev1.VolumeMount { + MountPath: "/opt/oracle/oradata", + Name: "datamount", + }) + } if m.Spec.Edition == "express" || m.Spec.Image.PrebuiltDB { // mounts pwd as secrets for express edition // or prebuilt db From a36e65a154ed576460b697719ee35ba88f8ea99e Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Thu, 21 Apr 2022 23:27:14 +0530 Subject: [PATCH 245/628] Fix prebuilt DB validation --- config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml | 2 ++ controllers/database/singleinstancedatabase_controller.go | 6 +++--- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml b/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml index a51b683f..437506f5 100644 --- a/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml +++ b/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml @@ -23,6 +23,8 @@ spec: ## Use only alphanumeric characters for sid sid: XE + edition: express + ## Secret containing SIDB password mapped to secretKey ## This secret will be deleted after creation of the database unless keepSecret is set to true adminPassword: diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 356f8337..3c8cbf78 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -316,9 +316,9 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab r.Log.Info("Got", "edition", m.Spec.Edition) // Pre-built db - if m.Spec.Image.PrebuiltDB { - return requeueN, nil - } + //if m.Spec.Image.PrebuiltDB { + // return requeueN, nil + //} // If Express Edition , Ensure Replicas=1 if m.Spec.Edition == "express" && m.Spec.Replicas > 1 { From 33a717fa09ec96eb39eefb29ad1f247221e21515 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Fri, 22 Apr 2022 18:35:25 +0530 Subject: [PATCH 246/628] Fix edition fetch --- commons/database/constants.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/commons/database/constants.go b/commons/database/constants.go index 4e8f8b94..7fce42be 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -427,4 +427,4 @@ const SetApexUsers string = "\numask 177" + "\numask 022" // Get Sid, Pdbname, Edition for prebuilt db -const GetSidPdbEditionCMD string = "if [ -f ${ORACLE_BASE}/oradata/dbconfig/$ORACLE_SID/.docker_standard ]; then echo \"$ORACLE_SID,$ORACLE_PDB,Standard,Edition\"; else echo \"$ORACLE_SID,$ORACLE_PDB,Enterprise,Edition\"; fi;" +const GetSidPdbEditionCMD string = "echo $ORACLE_SID,$ORACLE_PDB,$ORACLE_EDITION,Edition;" From 01c3f2751d49b5c5347d845b3557de5b3ea1005d Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Fri, 22 Apr 2022 19:20:10 +0530 Subject: [PATCH 247/628] Update init prebuilt DB cmd --- commons/database/constants.go | 2 +- .../database/singleinstancedatabase_controller.go | 10 +++++----- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/commons/database/constants.go b/commons/database/constants.go index 7fce42be..91ca4c06 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -349,7 +349,7 @@ const WalletEntriesCMD string = "umask 177\ncat > wallet.passwd < /dev/null; do sleep 0.5; done; fi " -const InitPrebuiltDbCMD string = "if [ ! -d /mnt/oradata/${ORACLE_SID} ]; then" + +const InitPrebuiltDbCMD string = "if [ ! -d /mnt/oradata/${ORACLE_SID} ]; then cp -v $ORACLE_BASE/oradata/.${ORACLE_SID}$CHECKPOINT_FILE_EXTN /mnt/oradata && " + " cp -vr $ORACLE_BASE/oradata/${ORACLE_SID} /mnt/oradata && cp -vr $ORACLE_BASE/oradata/dbconfig /mnt/oradata; fi " diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 3c8cbf78..c241f1c4 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -314,11 +314,7 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab eventReason := "Spec Error" var eventMsgs []string - r.Log.Info("Got", "edition", m.Spec.Edition) - // Pre-built db - //if m.Spec.Image.PrebuiltDB { - // return requeueN, nil - //} + r.Log.Info("Entering reconcile validation") // If Express Edition , Ensure Replicas=1 if m.Spec.Edition == "express" && m.Spec.Replicas > 1 { @@ -431,6 +427,9 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab m.Status.Edition = n.Status.Edition } + + r.Log.Info("Completed reconcile validation") + return requeueN, nil } @@ -971,6 +970,7 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex if sid == "" || pdbName == "" || m.Status.Edition == "" { return requeueN, nil } + m.Status.Edition = strings.Title(m.Status.Edition) } if m.Spec.LoadBalancer { From f4f7e56a15dfb5b9ac2cf06e5be02b02bca0a627 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Sat, 23 Apr 2022 12:26:38 +0530 Subject: [PATCH 248/628] Sample file changes --- .../singleinstancedatabase_webhook.go | 4 ++-- .../samples/sidb/singleinstancedatabase.yaml | 18 ++++++++------ .../sidb/singleinstancedatabase_clone.yaml | 17 +++---------- .../sidb/singleinstancedatabase_create.yaml | 9 +++++++ .../sidb/singleinstancedatabase_minikube.yaml | 24 +++++-------------- .../singleinstancedatabase_prebuiltdb.yaml | 13 +++++----- 6 files changed, 37 insertions(+), 48 deletions(-) diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index 6a8aced7..aff5c3c8 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -138,8 +138,8 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { if !r.Spec.Image.PrebuiltDB && r.Spec.Persistence.AccessMode != "ReadWriteMany" && r.Spec.Persistence.AccessMode != "ReadWriteOnce" { allErrs = append(allErrs, - field.Invalid(field.NewPath("spec").Child("persistence"), r.Spec.Persistence.AccessMode, - "should be either \"ReadWriteOnce\" or \"ReadWriteMany\"")) + field.Invalid(field.NewPath("spec").Child("persistence").Child("accessMode"), + r.Spec.Persistence.AccessMode, "should be either \"ReadWriteOnce\" or \"ReadWriteMany\"")) } if r.Spec.Persistence.AccessMode == "ReadWriteOnce" && r.Spec.Replicas != 1 { diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index 8faaf147..8b38b501 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -13,10 +13,10 @@ spec: sid: ORCL1 ## A source database ref to clone from, leave empty to create a fresh database - ## If cloning from on prem source database, mention connect string `:/`. + ## If cloning from on-prem source database, mention connect string `:/`. cloneFrom: "" - ## NA if cloning from a SourceDB (cloneFrom is set) + ## DB edition. N/A if cloning from a SourceDB (cloneFrom is set) edition: enterprise ## Should refer to SourceDB secret if cloning from a SourceDB (cloneFrom is set) @@ -27,16 +27,16 @@ spec: secretKey: keepSecret: true - ## NA if cloning from a SourceDB (cloneFrom is set) + ## DB character set. N/A if cloning from a SourceDB (cloneFrom is set) charset: AL32UTF8 - ## NA if cloning from a SourceDB (cloneFrom is set) + ## PDB name. N/A if cloning from a SourceDB (cloneFrom is set) pdbName: orclpdb1 ## Enable/Disable Flashback flashBack: false - ## Enable/Disable ArchiveLog + ## Enable/Disable ArchiveLog. Should be true to allow DB cloning archiveLog: false ## Enable/Disable ForceLogging @@ -58,8 +58,12 @@ spec: pgaAggregateTarget: 0 ## Database image details - ## Database can be patched by updating the RU version/image - ## Major version changes are not supported + ## Base DB images are available at container-registry.oracle.com or build from https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance + ## Build patched DB images from https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/patching + ## If cloning from on-prem DB, build on-prem DB image following the instructions + ## https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance#containerizing-an-on-premise-database-supported-from-version-1930-release + ## Prebuilt DB support (https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/prebuiltdb) + ## Specify prebuiltDB as true if the image includes a prebuilt DB image: pullFrom: pullSecrets: diff --git a/config/samples/sidb/singleinstancedatabase_clone.yaml b/config/samples/sidb/singleinstancedatabase_clone.yaml index 4845c89e..749c1063 100644 --- a/config/samples/sidb/singleinstancedatabase_clone.yaml +++ b/config/samples/sidb/singleinstancedatabase_clone.yaml @@ -3,16 +3,6 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # -apiVersion: v1 -kind: Secret -metadata: - name: admin-secret-2 -type: Opaque -stringData: - oracle_pwd: "ChangeOnInstall_2" - ---- - apiVersion: database.oracle.com/v1alpha1 kind: SingleInstanceDatabase metadata: @@ -24,20 +14,19 @@ spec: sid: ORCL2 ## A source database ref to clone from. - ## If cloning from on prem source database, mention connect string `:/`. + ## Make sure the source database has been created by applying singeinstancedatabase_create.yaml cloneFrom: sidb-sample ## Should refer to SourceDB secret ## Secret containing SIDB password mapped to secretKey ## This secret will be deleted after creation of the database unless keepSecret is set to true adminPassword: - secretName: admin-secret-2 + secretName: admin-secret-1 secretKey: oracle_pwd keepSecret: true ## Database image details - ## Database can be patched out of place by updating the RU version/image - ## Major version changes are not supported + ## This image should be the same as the source DB image being cloned image: pullFrom: container-registry.oracle.com/database/enterprise:latest pullSecrets: oracle-container-registry-secret diff --git a/config/samples/sidb/singleinstancedatabase_create.yaml b/config/samples/sidb/singleinstancedatabase_create.yaml index 69680826..3bed87b1 100644 --- a/config/samples/sidb/singleinstancedatabase_create.yaml +++ b/config/samples/sidb/singleinstancedatabase_create.yaml @@ -30,6 +30,15 @@ spec: secretKey: oracle_pwd keepSecret: true + ## DB character set + charset: AL32UTF8 + + ## PDB name + pdbName: orclpdb1 + + ## Enable/Disable ArchiveLog. Should be true to allow DB cloning + archiveLog: true + ## Database Image image: pullFrom: container-registry.oracle.com/database/enterprise:latest diff --git a/config/samples/sidb/singleinstancedatabase_minikube.yaml b/config/samples/sidb/singleinstancedatabase_minikube.yaml index 14b24f93..ab11faeb 100644 --- a/config/samples/sidb/singleinstancedatabase_minikube.yaml +++ b/config/samples/sidb/singleinstancedatabase_minikube.yaml @@ -16,13 +16,13 @@ stringData: apiVersion: v1 kind: PersistentVolume metadata: - name: xe-pv + name: minikube-pv spec: accessModes: - - ReadWriteMany + - ReadWriteOnce capacity: storage: 5Gi - storageClassName: xe-sc + storageClassName: minikube-sc hostPath: path: /data/oradata @@ -31,14 +31,14 @@ spec: apiVersion: database.oracle.com/v1alpha1 kind: SingleInstanceDatabase metadata: - name: xe-sample + name: xedb-minikube-sample namespace: default spec: ## Use only alphanumeric characters for sid sid: XE - ## NA if cloning from a SourceDB (cloneFrom is set) + ## DB edition. NA if cloning from a SourceDB (cloneFrom is set) edition: express ## Should refer to SourceDB secret if cloning from a SourceDB (cloneFrom is set) @@ -49,18 +49,6 @@ spec: secretKey: oracle_pwd keepSecret: true - ## NA if cloning from a SourceDB (cloneFrom is set) - charset: AL32UTF8 - - ## Enable/Disable Flashback - flashBack: false - - ## Enable/Disable ArchiveLog - archiveLog: false - - ## Enable/Disable ForceLogging - forceLog: false - ## Database image details ## Database can be patched by updating the RU version/image ## Major version changes are not supported @@ -72,7 +60,7 @@ spec: ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany persistence: size: 5Gi - storageClass: "xe-sc" + storageClass: "minikube-sc" accessMode: "ReadWriteOnce" ## Count of Database Pods. Applicable only for "ReadWriteMany" AccessMode diff --git a/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml b/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml index 437506f5..314bb353 100644 --- a/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml +++ b/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml @@ -37,13 +37,12 @@ spec: pullFrom: container-registry.oracle.com/database/express:latest prebuiltDB: true - ## size : Minimum size of pvc | class : PVC storage Class - ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany - ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. - persistence: - size: 100Gi - storageClass: "oci" - accessMode: "ReadWriteOnce" + ## Persistence is optional for prebuilt DB image + ## if specified, the prebuilt DB datafiles are copied over to the persistant volume before DB startup + #persistence: + # size: 100Gi + # storageClass: "oci" + # accessMode: "ReadWriteOnce" ## If deploying on OpenShift, change service account name to 'sidb-sa' after you run `$ oc apply -f openshift_rbac.yaml` serviceAccountName: default From 5107d493351f745591ce32e25e05c673daf63837 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Sat, 23 Apr 2022 14:10:21 +0530 Subject: [PATCH 249/628] Fix validations --- .../v1alpha1/singleinstancedatabase_types.go | 1 + .../singleinstancedatabase_webhook.go | 24 +++++++------------ 2 files changed, 10 insertions(+), 15 deletions(-) diff --git a/apis/database/v1alpha1/singleinstancedatabase_types.go b/apis/database/v1alpha1/singleinstancedatabase_types.go index fde6059e..0448c44b 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_types.go +++ b/apis/database/v1alpha1/singleinstancedatabase_types.go @@ -138,6 +138,7 @@ type SingleInstanceDatabaseStatus struct { OrdsReference string `json:"ordsReference,omitempty"` PdbConnectString string `json:"pdbConnectString,omitempty"` ApexInstalled bool `json:"apexInstalled,omitempty"` + PrebuiltDB bool `json:"prebuiltDB,omitempty"` // +patchMergeKey=type // +patchStrategy=merge diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index aff5c3c8..19ed1ae2 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -135,8 +135,8 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { r.Name, allErrs) } - if !r.Spec.Image.PrebuiltDB && - r.Spec.Persistence.AccessMode != "ReadWriteMany" && r.Spec.Persistence.AccessMode != "ReadWriteOnce" { + if r.Spec.Persistence.Size != "" && + r.Spec.Persistence.AccessMode != "ReadWriteMany" && r.Spec.Persistence.AccessMode != "ReadWriteOnce" { allErrs = append(allErrs, field.Invalid(field.NewPath("spec").Child("persistence").Child("accessMode"), r.Spec.Persistence.AccessMode, "should be either \"ReadWriteOnce\" or \"ReadWriteMany\"")) @@ -197,34 +197,28 @@ func (r *SingleInstanceDatabase) ValidateUpdate(oldRuntimeObject runtime.Object) } } - // Pre-built db - if r.Spec.Image.PrebuiltDB { - return nil - } - // Now check for updation errors old, ok := oldRuntimeObject.(*SingleInstanceDatabase) if !ok { return nil } - - edition := r.Spec.Edition - if r.Spec.Edition == "" { - edition = "Enterprise" + if old.Status.PrebuiltDB != r.Spec.Image.PrebuiltDB { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("image").Child("prebuiltDB"), "cannot be changed")) } - if r.Spec.CloneFrom == "" && old.Status.Edition != "" && !strings.EqualFold(old.Status.Edition, edition) { + if r.Spec.CloneFrom == "" && old.Status.Edition != "" && r.Spec.Edition != "" && !strings.EqualFold(old.Status.Edition, r.Spec.Edition) { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("edition"), "cannot be changed")) } - if old.Status.Charset != "" && !strings.EqualFold(old.Status.Charset, r.Spec.Charset) { + if old.Status.Charset != "" && r.Spec.Charset != "" && !strings.EqualFold(old.Status.Charset, r.Spec.Charset) { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("charset"), "cannot be changed")) } - if old.Status.Sid != "" && !strings.EqualFold(r.Spec.Sid, old.Status.Sid) { + if old.Status.Sid != "" && r.Spec.Sid != "" && !strings.EqualFold(r.Spec.Sid, old.Status.Sid) { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("sid"), "cannot be changed")) } - if old.Status.Pdbname != "" && !strings.EqualFold(old.Status.Pdbname, r.Spec.Pdbname) { + if old.Status.Pdbname != "" && r.Spec.Pdbname != "" && !strings.EqualFold(old.Status.Pdbname, r.Spec.Pdbname) { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("pdbname"), "cannot be changed")) } From eda48cbbf85fee27e182ae273de24364d9d8dbd3 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Sat, 23 Apr 2022 14:25:43 +0530 Subject: [PATCH 250/628] Fix validation webhook --- apis/database/v1alpha1/singleinstancedatabase_webhook.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index 19ed1ae2..64ac4c9f 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -202,7 +202,7 @@ func (r *SingleInstanceDatabase) ValidateUpdate(oldRuntimeObject runtime.Object) if !ok { return nil } - if old.Status.PrebuiltDB != r.Spec.Image.PrebuiltDB { + if old.Status.DatafilesCreated == "true" && (old.Status.PrebuiltDB != r.Spec.Image.PrebuiltDB) { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("image").Child("prebuiltDB"), "cannot be changed")) } From c97048bb1fd1902bb7cc185b57f4c9bcc4f669a9 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Sat, 23 Apr 2022 14:44:33 +0530 Subject: [PATCH 251/628] Webhook validation fixes --- .../singleinstancedatabase_webhook.go | 32 +++++++------------ 1 file changed, 12 insertions(+), 20 deletions(-) diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index 64ac4c9f..0814100a 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -120,21 +120,6 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { "invalid specification, size and/or accessMode missing")) } - // Pre-built db - if r.Spec.Image.PrebuiltDB { - if r.Spec.CloneFrom != "" && r.Spec.Edition != "express" { - allErrs = append(allErrs, - field.Invalid(field.NewPath("spec").Child("cloneFrom"), r.Spec.CloneFrom, - "cannot clone to create a prebuilt db")) - } - if len(allErrs) == 0 { - return nil - } - return apierrors.NewInvalid( - schema.GroupKind{Group: "database.oracle.com", Kind: "SingleInstanceDatabase"}, - r.Name, allErrs) - } - if r.Spec.Persistence.Size != "" && r.Spec.Persistence.AccessMode != "ReadWriteMany" && r.Spec.Persistence.AccessMode != "ReadWriteOnce" { allErrs = append(allErrs, @@ -162,11 +147,18 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { field.Invalid(field.NewPath("spec").Child("pdbName"), r.Spec.Pdbname, "Express edition PDB must be XEPDB1")) } - //Edition must be passed when cloning from a source database other than same k8s cluster - if strings.Contains(r.Spec.CloneFrom, ":") && strings.Contains(r.Spec.CloneFrom, "/") && r.Spec.Edition == "" { - allErrs = append(allErrs, - field.Invalid(field.NewPath("spec").Child("edition"), r.Spec.CloneFrom, - "Edition must be passed when cloning from a source database other than same k8s cluster")) + + if r.Spec.CloneFrom != "" { + if r.Spec.Image.PrebuiltDB { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("cloneFrom"), r.Spec.CloneFrom, + "cannot clone to create a prebuilt db")) + } else if strings.Contains(r.Spec.CloneFrom, ":") && strings.Contains(r.Spec.CloneFrom, "/") && r.Spec.Edition == "" { + //Edition must be passed when cloning from a source database other than same k8s cluster + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("edition"), r.Spec.CloneFrom, + "Edition must be passed when cloning from a source database other than same k8s cluster")) + } } if len(allErrs) == 0 { From 0b836ed19c9535734878356640c4253d1f034168 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Sat, 23 Apr 2022 14:55:15 +0530 Subject: [PATCH 252/628] Missing status update --- controllers/database/singleinstancedatabase_controller.go | 1 + 1 file changed, 1 insertion(+) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index c241f1c4..285ea8fc 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -382,6 +382,7 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab m.Status.Charset = m.Spec.Charset m.Status.Pdbname = m.Spec.Pdbname m.Status.Persistence = m.Spec.Persistence + m.Status.PrebuiltDB = m.Spec.Image.PrebuiltDB if m.Spec.CloneFrom == "" { m.Status.CloneFrom = dbcommons.NoCloneRef } else { From 2d8a7f4849b261853a64508b09c60d0f1574203d Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Sun, 24 Apr 2022 09:59:34 +0530 Subject: [PATCH 253/628] Config sample fixes --- .../v1alpha1/singleinstancedatabase_types.go | 2 +- .../singleinstancedatabase_webhook.go | 14 ++++-- .../samples/sidb/singleinstancedatabase.yaml | 2 +- .../sidb/singleinstancedatabase_clone.yaml | 8 +--- .../sidb/singleinstancedatabase_create.yaml | 10 ++-- .../sidb/singleinstancedatabase_minikube.yaml | 6 --- .../sidb/singleinstancedatabase_patch.yaml | 47 +++++++++++++++++++ .../singleinstancedatabase_prebuiltdb.yaml | 6 --- 8 files changed, 63 insertions(+), 32 deletions(-) create mode 100644 config/samples/sidb/singleinstancedatabase_patch.yaml diff --git a/apis/database/v1alpha1/singleinstancedatabase_types.go b/apis/database/v1alpha1/singleinstancedatabase_types.go index 0448c44b..f9dd6da9 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_types.go +++ b/apis/database/v1alpha1/singleinstancedatabase_types.go @@ -117,7 +117,7 @@ type SingleInstanceDatabaseStatus struct { Nodes []string `json:"nodes,omitempty"` Role string `json:"role,omitempty"` Status string `json:"status,omitempty"` - Replicas int `json:"replicas"` + Replicas int `json:"replicas,omitempty"` ReleaseUpdate string `json:"releaseUpdate,omitempty"` DatafilesPatched string `json:"datafilesPatched,omitempty"` ConnectString string `json:"connectString,omitempty"` diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index 0814100a..bac7571d 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -71,12 +71,16 @@ var _ webhook.Defaulter = &SingleInstanceDatabase{} func (r *SingleInstanceDatabase) Default() { singleinstancedatabaselog.Info("default", "name", r.Name) - if r.Spec.Edition == "express" { + if r.Spec.Replicas == 0 { r.Spec.Replicas = 1 + } + + if r.Spec.Edition == "express" { if r.Spec.Sid == "" { r.Spec.Sid = "XE" } } + if r.Spec.Pdbname == "" { if r.Spec.Edition == "express" { r.Spec.Pdbname = "XEPDB1" @@ -198,19 +202,19 @@ func (r *SingleInstanceDatabase) ValidateUpdate(oldRuntimeObject runtime.Object) allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("image").Child("prebuiltDB"), "cannot be changed")) } - if r.Spec.CloneFrom == "" && old.Status.Edition != "" && r.Spec.Edition != "" && !strings.EqualFold(old.Status.Edition, r.Spec.Edition) { + if r.Spec.CloneFrom == "" && old.Status.Edition != "" && !strings.EqualFold(old.Status.Edition, r.Spec.Edition) { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("edition"), "cannot be changed")) } - if old.Status.Charset != "" && r.Spec.Charset != "" && !strings.EqualFold(old.Status.Charset, r.Spec.Charset) { + if old.Status.Charset != "" && !strings.EqualFold(old.Status.Charset, r.Spec.Charset) { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("charset"), "cannot be changed")) } - if old.Status.Sid != "" && r.Spec.Sid != "" && !strings.EqualFold(r.Spec.Sid, old.Status.Sid) { + if old.Status.Sid != "" && !strings.EqualFold(r.Spec.Sid, old.Status.Sid) { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("sid"), "cannot be changed")) } - if old.Status.Pdbname != "" && r.Spec.Pdbname != "" && !strings.EqualFold(old.Status.Pdbname, r.Spec.Pdbname) { + if old.Status.Pdbname != "" && !strings.EqualFold(old.Status.Pdbname, r.Spec.Pdbname) { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("pdbname"), "cannot be changed")) } diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index 8b38b501..bc998370 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -60,7 +60,7 @@ spec: ## Database image details ## Base DB images are available at container-registry.oracle.com or build from https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance ## Build patched DB images from https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/patching - ## If cloning from on-prem DB, build on-prem DB image following the instructions + ## If cloning from on-prem DB, build on-prem DB image following the instructions at ## https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance#containerizing-an-on-premise-database-supported-from-version-1930-release ## Prebuilt DB support (https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/prebuiltdb) ## Specify prebuiltDB as true if the image includes a prebuilt DB diff --git a/config/samples/sidb/singleinstancedatabase_clone.yaml b/config/samples/sidb/singleinstancedatabase_clone.yaml index 749c1063..9fa611d3 100644 --- a/config/samples/sidb/singleinstancedatabase_clone.yaml +++ b/config/samples/sidb/singleinstancedatabase_clone.yaml @@ -27,6 +27,8 @@ spec: ## Database image details ## This image should be the same as the source DB image being cloned + ## or a patched DB image built from the souce DB image following instructions at + ## https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/patching image: pullFrom: container-registry.oracle.com/database/enterprise:latest pullSecrets: oracle-container-registry-secret @@ -38,9 +40,3 @@ spec: size: 100Gi storageClass: "oci" accessMode: "ReadWriteOnce" - - ## If deploying on OpenShift, change service account name to 'sidb-sa' after you run `$ oc apply -f openshift_rbac.yaml` - serviceAccountName: default - - ## Count of Database Pods. Applicable only for "ReadWriteMany" AccessMode - replicas: 1 diff --git a/config/samples/sidb/singleinstancedatabase_create.yaml b/config/samples/sidb/singleinstancedatabase_create.yaml index 3bed87b1..09d44c40 100644 --- a/config/samples/sidb/singleinstancedatabase_create.yaml +++ b/config/samples/sidb/singleinstancedatabase_create.yaml @@ -16,6 +16,8 @@ stringData: apiVersion: database.oracle.com/v1alpha1 kind: SingleInstanceDatabase metadata: + # Creates base sidb-sample. Use singleinstancedatabase_clone.yaml for cloning + # and singleinstancedatabase_patch.yaml for patching name: sidb-sample namespace: default spec: @@ -39,7 +41,7 @@ spec: ## Enable/Disable ArchiveLog. Should be true to allow DB cloning archiveLog: true - ## Database Image + ## Database image details image: pullFrom: container-registry.oracle.com/database/enterprise:latest pullSecrets: oracle-container-registry-secret @@ -51,9 +53,3 @@ spec: size: 100Gi storageClass: "oci" accessMode: "ReadWriteOnce" - - ## If deploying on OpenShift, change service account name to 'sidb-sa' after you run `$ oc apply -f openshift_rbac.yaml` - serviceAccountName: default - - ## Count of Database Pods. Applicable only for "ReadWriteMany" AccessMode - replicas: 1 diff --git a/config/samples/sidb/singleinstancedatabase_minikube.yaml b/config/samples/sidb/singleinstancedatabase_minikube.yaml index ab11faeb..0d81ec3a 100644 --- a/config/samples/sidb/singleinstancedatabase_minikube.yaml +++ b/config/samples/sidb/singleinstancedatabase_minikube.yaml @@ -41,7 +41,6 @@ spec: ## DB edition. NA if cloning from a SourceDB (cloneFrom is set) edition: express - ## Should refer to SourceDB secret if cloning from a SourceDB (cloneFrom is set) ## Secret containing SIDB password mapped to secretKey ## This secret will be deleted after creation of the database unless keepSecret is set to true adminPassword: @@ -50,8 +49,6 @@ spec: keepSecret: true ## Database image details - ## Database can be patched by updating the RU version/image - ## Major version changes are not supported image: pullFrom: container-registry.oracle.com/database/express:latest prebuiltDB: true @@ -62,6 +59,3 @@ spec: size: 5Gi storageClass: "minikube-sc" accessMode: "ReadWriteOnce" - - ## Count of Database Pods. Applicable only for "ReadWriteMany" AccessMode - replicas: 1 diff --git a/config/samples/sidb/singleinstancedatabase_patch.yaml b/config/samples/sidb/singleinstancedatabase_patch.yaml new file mode 100644 index 00000000..406b0e2e --- /dev/null +++ b/config/samples/sidb/singleinstancedatabase_patch.yaml @@ -0,0 +1,47 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +apiVersion: database.oracle.com/v1alpha1 +kind: SingleInstanceDatabase +metadata: + # sidb-sample should have already been created using singleinstancedatabase_create.yaml + name: sidb-sample + namespace: default +spec: + + ## Use only alphanumeric characters for sid + sid: ORCL1 + + ## Secret containing SIDB password mapped to secretKey + ## This secret will be deleted after creation of the database unless keepSecret is set to true + adminPassword: + secretName: admin-secret-1 + secretKey: oracle_pwd + keepSecret: true + + ## DB character set + charset: AL32UTF8 + + ## PDB name + pdbName: orclpdb1 + + ## Enable/Disable ArchiveLog. Should be true to allow DB cloning + archiveLog: true + + ## Patched Database image + ## Using the source base image container-registry.oracle.com/database/enterprise:latest + ## build patched DB images following instructions at + ## https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/patching + image: + pullFrom: + pullSecrets: + + ## size : Minimum size of pvc | class : PVC storage Class + ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany + ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. + persistence: + size: 100Gi + storageClass: "oci" + accessMode: "ReadWriteOnce" diff --git a/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml b/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml index 314bb353..dadfcddb 100644 --- a/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml +++ b/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml @@ -43,9 +43,3 @@ spec: # size: 100Gi # storageClass: "oci" # accessMode: "ReadWriteOnce" - - ## If deploying on OpenShift, change service account name to 'sidb-sa' after you run `$ oc apply -f openshift_rbac.yaml` - serviceAccountName: default - - ## Count of Database Pods. Applicable only for "ReadWriteMany" AccessMode - replicas: 1 From 6e0d5e6e55f111e6abfcd4c733901e030abb7f00 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Sun, 24 Apr 2022 10:17:49 +0530 Subject: [PATCH 254/628] Add missing edition --- config/samples/sidb/singleinstancedatabase_create.yaml | 3 +++ config/samples/sidb/singleinstancedatabase_patch.yaml | 3 +++ 2 files changed, 6 insertions(+) diff --git a/config/samples/sidb/singleinstancedatabase_create.yaml b/config/samples/sidb/singleinstancedatabase_create.yaml index 09d44c40..672d3b61 100644 --- a/config/samples/sidb/singleinstancedatabase_create.yaml +++ b/config/samples/sidb/singleinstancedatabase_create.yaml @@ -25,6 +25,9 @@ spec: ## Use only alphanumeric characters for sid sid: ORCL1 + ## DB edition. + edition: enterprise + ## Secret containing SIDB password mapped to secretKey ## This secret will be deleted after creation of the database unless keepSecret is set to true adminPassword: diff --git a/config/samples/sidb/singleinstancedatabase_patch.yaml b/config/samples/sidb/singleinstancedatabase_patch.yaml index 406b0e2e..f843568f 100644 --- a/config/samples/sidb/singleinstancedatabase_patch.yaml +++ b/config/samples/sidb/singleinstancedatabase_patch.yaml @@ -14,6 +14,9 @@ spec: ## Use only alphanumeric characters for sid sid: ORCL1 + ## DB edition. + edition: enterprise + ## Secret containing SIDB password mapped to secretKey ## This secret will be deleted after creation of the database unless keepSecret is set to true adminPassword: From d825d956ec8b130e7726a90e5b2b6568f978d4f2 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Sun, 24 Apr 2022 11:32:44 +0530 Subject: [PATCH 255/628] Remove commented code --- controllers/database/singleinstancedatabase_controller.go | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 16c8ec25..864d5a63 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1843,7 +1843,7 @@ func (r *SingleInstanceDatabaseReconciler) installApex(m *dbapi.SingleInstanceDa return requeueN } -/* //############################################################################# +//############################################################################# // Uninstall APEX from CDB //############################################################################# func (r *SingleInstanceDatabaseReconciler) uninstallApex(m *dbapi.SingleInstanceDatabase, @@ -1897,7 +1897,7 @@ func (r *SingleInstanceDatabaseReconciler) uninstallApex(m *dbapi.SingleInstance } r.updateORDSStatus(m, true, ctx, req) return requeueN -} */ +} //############################################################################# // Update ORDS Status From efc23c063a8d4867af65d0cb6cf027a323afc7bc Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Sun, 24 Apr 2022 11:36:57 +0530 Subject: [PATCH 256/628] Remove Apex installation from SIDB --- controllers/database/singleinstancedatabase_controller.go | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 864d5a63..02f14e9c 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -215,6 +215,7 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct } } + /* // Install Apex result = r.installApex(singleInstanceDatabase, readyPod, ctx, req) if result.Requeue { @@ -222,12 +223,13 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct return result, nil } - /* // Uninstall Apex + // Uninstall Apex result = r.uninstallApex(singleInstanceDatabase, readyPod, ctx, req) if result.Requeue { r.Log.Info("Reconcile queued") return result, nil - } */ + } + */ // If LoadBalancer = true , ensure Connect String is updated if singleInstanceDatabase.Status.ConnectString == dbcommons.ValueUnavailable { @@ -1849,7 +1851,6 @@ func (r *SingleInstanceDatabaseReconciler) installApex(m *dbapi.SingleInstanceDa func (r *SingleInstanceDatabaseReconciler) uninstallApex(m *dbapi.SingleInstanceDatabase, readyPod corev1.Pod, ctx context.Context, req ctrl.Request) ctrl.Result { log := r.Log.WithValues("uninstallApex", req.NamespacedName) - return requeueN // No APEX for Pre-built db if m.Spec.Image.PrebuiltDB { From ffbd4f4c7c92b477b1bf24bcb446a01e1c6f3c51 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Sun, 24 Apr 2022 14:34:04 +0530 Subject: [PATCH 257/628] Add ORDS create sample --- .../sidb/oraclerestdataservice_create.yaml | 62 +++++++++++++++++++ 1 file changed, 62 insertions(+) create mode 100644 config/samples/sidb/oraclerestdataservice_create.yaml diff --git a/config/samples/sidb/oraclerestdataservice_create.yaml b/config/samples/sidb/oraclerestdataservice_create.yaml new file mode 100644 index 00000000..c225212c --- /dev/null +++ b/config/samples/sidb/oraclerestdataservice_create.yaml @@ -0,0 +1,62 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +apiVersion: v1 +kind: Secret +metadata: + name: ords-secret-1 +type: Opaque +stringData: + oracle_pwd: "ChangeOnInstall_1" + +--- + +apiVersion: database.oracle.com/v1alpha1 +kind: OracleRestDataService +metadata: + name: ords-sample +spec: + + ## Database ref. This can be of kind SingleInstanceDatabase. + databaseRef: "sidb-sample" + + ## Secret containing databaseRef password mapped to secretKey. + ## This secret will be deleted after ORDS Installation unless keepSecret set to true. + adminPassword: + secretName: admin-secret-1 + secretKey: oracle_pwd + keepSecret: true + + ## Secret containing ORDS_PUBLIC_USER password mapped to secretKey. + ## This secret will be deleted after ORDS Installation unless keepSecret set to true. + ordsPassword: + secretName: ords-secret-1 + secretKey: oracle_pwd + keepSecret: true + + + ## ORDS image details + image: + pullFrom: + pullSecrets: + + ## size : Minimum size of pvc | class : PVC storage Class + ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany + ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. + persistence: + size: 100Gi + storageClass: "oci" + accessMode: "ReadWriteOnce" + + + ## PDB Schemas to be ORDS Enabled. + ## Schema will be created (if not exists) with username as schema and password as .spec.ordsPassword. + restEnableSchemas: + - schema: schema1 + enable: true + urlMapping: + pdb: + + \ No newline at end of file From 5cdc9f1fed07cfd99b882fe359dc13fdf6a527b0 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Mon, 25 Apr 2022 11:00:56 +0530 Subject: [PATCH 258/628] Remove image pull policy --- commons/database/constants.go | 28 +++++++++++++++++++ .../oraclerestdataservice_controller.go | 1 - 2 files changed, 28 insertions(+), 1 deletion(-) diff --git a/commons/database/constants.go b/commons/database/constants.go index 86534994..8c363ee0 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -282,6 +282,34 @@ const SetupORDSCMD string = "$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-pr "\nsed -i 's,jetty.port=8888,jetty.secure.port=8443\\nssl.cert=\\nssl.cert.key=\\nssl.host=%[3]s,g' /opt/oracle/ords/config/ords/standalone/standalone.properties " + "\nsed -i 's,standalone.static.path=/opt/oracle/ords/doc_root/i,standalone.static.path=/opt/oracle/ords/config/ords/apex/images,g' /opt/oracle/ords/config/ords/standalone/standalone.properties" +const InitORDSCMD string = "$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property database.api.enabled true" + + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property jdbc.auth.enabled true" + + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property database.api.management.services.disabled false" + + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property database.api.admin.enabled true" + + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property dbc.auth.enabled true" + + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property restEnabledSql.active true" + + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property db.serviceNameSuffix \"\" " + // Mandatory when ORDS Installing at CDB Level -> Maps PDB's + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property jdbc.InitialLimit 5" + + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property jdbc.MaxLimit 20" + + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property jdbc.InactivityTimeout 300" + + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property feature.sdw true" + + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property security.verifySSL false" + + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property jdbc.maxRows 1000" + + "\numask 177" + + "\necho db.cdb.adminUser=C##DBAPI_CDB_ADMIN AS SYSDBA > cdbAdmin.properties" + + "\necho db.cdb.adminUser.password=\"%[4]s\" >> cdbAdmin.properties" + + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-properties --conf apex_pu cdbAdmin.properties" + + "\nrm -f cdbAdmin.properties" + + "\necho db.adminUser=C##_DBAPI_PDB_ADMIN > pdbAdmin.properties" + + "\necho db.adminUser.password=\"%[4]s\">> pdbAdmin.properties" + + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-properties --conf apex_pu pdbAdmin.properties" + + "\nrm -f pdbAdmin.properties" + + "\necho -e \"%[1]s\n%[1]s\" > sqladmin.passwd" + + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war user ${ORDS_USER} \"SQL Administrator , System Administrator , SQL Developer , oracle.dbtools.autorest.any.schema \" < sqladmin.passwd" + + "\nrm -f sqladmin.passwd" + + "\numask 022" + + "\nsed -i 's,jetty.port=8888,jetty.secure.port=8443\\nssl.cert=\\nssl.cert.key=\\nssl.host=%[3]s,g' /opt/oracle/ords/config/ords/standalone/standalone.properties " + const GetSessionInfoSQL string = "select s.sid || ',' || s.serial# as Info FROM v\\$session s, v\\$process p WHERE s.username = 'ORDS_PUBLIC_USER' AND p.addr(+) = s.paddr;" const KillSessionSQL string = "alter system kill session '%[1]s';" diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 4c7f4919..a10c36b0 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -460,7 +460,6 @@ func (r *OracleRestDataServiceReconciler) instantiatePodSpec(m *dbapi.OracleRest Containers: []corev1.Container{{ Name: m.Name, Image: m.Spec.Image.PullFrom, - ImagePullPolicy: corev1.PullAlways, Ports: []corev1.ContainerPort{{ContainerPort: 8443}}, VolumeMounts: []corev1.VolumeMount{{ MountPath: "/opt/oracle/ords/config/ords/", From 269664356e118e451ee63c62c45d792eb4f57e6d Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Mon, 25 Apr 2022 11:18:24 +0530 Subject: [PATCH 259/628] Fix properties --- commons/database/constants.go | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/commons/database/constants.go b/commons/database/constants.go index 8c363ee0..7fa1639d 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -254,6 +254,7 @@ const SetupORDSCMD string = "$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-pr "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property feature.sdw true" + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property security.verifySSL false" + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property jdbc.maxRows 1000" + + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property pdb.open.asneeded true" + "\numask 177" + "\necho db.cdb.adminUser=C##DBAPI_CDB_ADMIN AS SYSDBA > cdbAdmin.properties" + "\necho db.cdb.adminUser.password=\"%[4]s\" >> cdbAdmin.properties" + @@ -280,7 +281,7 @@ const SetupORDSCMD string = "$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-pr "\nrm -f sqladmin.passwd" + "\numask 022" + "\nsed -i 's,jetty.port=8888,jetty.secure.port=8443\\nssl.cert=\\nssl.cert.key=\\nssl.host=%[3]s,g' /opt/oracle/ords/config/ords/standalone/standalone.properties " + - "\nsed -i 's,standalone.static.path=/opt/oracle/ords/doc_root/i,standalone.static.path=/opt/oracle/ords/config/ords/apex/images,g' /opt/oracle/ords/config/ords/standalone/standalone.properties" + "\nsed -i 's,standalone.static.path=/opt/oracle/ords/doc_root/i,standalone.static.path=/opt/oracle/ords/config/apex/images,g' /opt/oracle/ords/config/ords/standalone/standalone.properties" const InitORDSCMD string = "$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property database.api.enabled true" + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property jdbc.auth.enabled true" + From 65bb6127d83359df73b35e298d561af26713c7ed Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Mon, 25 Apr 2022 12:46:20 +0530 Subject: [PATCH 260/628] Fix ORDS install --- commons/database/constants.go | 4 +- .../oraclerestdataservice_controller.go | 56 +++++++++++++------ 2 files changed, 43 insertions(+), 17 deletions(-) diff --git a/commons/database/constants.go b/commons/database/constants.go index 7fa1639d..cb1520a3 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -224,7 +224,9 @@ const SetAdminUsersSQL string = "CREATE USER C##DBAPI_CDB_ADMIN IDENTIFIED BY \\ "\nGRANT PDB_DBA TO C##DBAPI_CDB_ADMIN CONTAINER = ALL;" + "\nCREATE USER C##_DBAPI_PDB_ADMIN IDENTIFIED BY \\\"%[1]s\\\" CONTAINER=ALL ACCOUNT UNLOCK;" + "\nalter user C##_DBAPI_PDB_ADMIN identified by \\\"%[1]s\\\" account unlock;" + - "\nGRANT DBA TO C##_DBAPI_PDB_ADMIN CONTAINER = ALL;" + "\nGRANT DBA TO C##_DBAPI_PDB_ADMIN CONTAINER = ALL;" + + "\nalter pluggable database pdb$seed close;" + + "\nalter pluggable database pdb$seed open read write force;" const GetUserOrdsSchemaStatusSQL string = "alter session set container=%[2]s;" + "\nselect 'STATUS:'||status as status from ords_metadata.ords_schemas where upper(parsing_schema) = upper('%[1]s');" diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index a10c36b0..7bb9c3a6 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -157,6 +157,14 @@ func (r *OracleRestDataServiceReconciler) Reconcile(ctx context.Context, req ctr return result, nil } + // setup DB for ORDS + var cdbAdminPassword string + result, cdbAdminPassword = r.setupDBPrequisites(oracleRestDataService, singleInstanceDatabase, sidbReadyPod, ctx, req) + if result.Requeue { + r.Log.Info("Reconcile queued") + return result, nil + } + // Create ORDS Pods result = r.createPods(oracleRestDataService, singleInstanceDatabase, sidbReadyPod, ctx, req) if result.Requeue { @@ -178,7 +186,7 @@ func (r *OracleRestDataServiceReconciler) Reconcile(ctx context.Context, req ctr return result, nil } - result = r.setupORDS(oracleRestDataService, singleInstanceDatabase, sidbReadyPod, ctx, req) + result = r.setupORDS(oracleRestDataService, singleInstanceDatabase, sidbReadyPod, cdbAdminPassword, ctx, req) if result.Requeue { r.Log.Info("Reconcile queued") return result, nil @@ -304,6 +312,36 @@ func (r *OracleRestDataServiceReconciler) validateSidbReadiness(m *dbapi.OracleR return requeueN, sidbReadyPod } +//##################################################################################################### +// Set up DB prereqs +//##################################################################################################### +func (r *OracleRestDataServiceReconciler) setupDBPrequisites(m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase, + sidbReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) (ctrl.Result, string) { + log := r.Log.WithValues("setupDBPrequisites", req.NamespacedName) + + if !m.Status.OrdsSetupCompleted { + + cdbAdminPassword := dbcommons.GenerateRandomString(8) + // Create PDB , CDB Admin users and grant permissions . ORDS installation on CDB level + out, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | %s", fmt.Sprintf(dbcommons.SetAdminUsersSQL, cdbAdminPassword), dbcommons.SQLPlusCLI)) + if err != nil { + log.Error(err, err.Error()) + return requeueY, "" + } + log.Info("SetAdminUsers Output :\n" + out) + + if !strings.Contains(out, "ERROR") || !strings.Contains(out, "ORA-") || + strings.Contains(out, "ERROR") && strings.Contains(out, "ORA-01920") { + m.Status.CommonUsersCreated = true + } + + return requeueN, cdbAdminPassword + } + + return requeueN, "" +} + //##################################################################################################### // Validate Readiness of the ORDS pods //##################################################################################################### @@ -1294,7 +1332,7 @@ func (r *OracleRestDataServiceReconciler) deleteSecrets(m *dbapi.OracleRestDataS // Setup ORDS in CDB , enable ORDS for PDBs Specified //############################################################################# func (r *OracleRestDataServiceReconciler) setupORDS(m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase, - sidbReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) ctrl.Result { + sidbReadyPod corev1.Pod, cdbAdminPassword string, ctx context.Context, req ctrl.Request) ctrl.Result { log := r.Log.WithValues("setupORDS", req.NamespacedName) @@ -1365,20 +1403,6 @@ func (r *OracleRestDataServiceReconciler) setupORDS(m *dbapi.OracleRestDataServi if strings.Contains(out, "HTTP/1.1 200 OK") || strings.Contains(strings.ToUpper(err.Error()), "HTTP/1.1 200 OK") { if !m.Status.OrdsSetupCompleted { - cdbAdminPassword := dbcommons.GenerateRandomString(8) - // Create PDB , CDB Admin users and grant permissions . ORDS installation on CDB level - out, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", - fmt.Sprintf("echo -e \"%s\" | %s", fmt.Sprintf(dbcommons.SetAdminUsersSQL, cdbAdminPassword), dbcommons.SQLPlusCLI)) - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - log.Info("SetAdminUsers Output :\n" + out) - - if !strings.Contains(out, "ERROR") || !strings.Contains(out, "ORA-") || - strings.Contains(out, "ERROR") && strings.Contains(out, "ORA-01920") { - m.Status.CommonUsersCreated = true - } // Setup ORDS out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, true, "bash", "-c", From 2d84bd6a06d4b905b0504e6a8755df039a585d70 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Mon, 25 Apr 2022 12:56:50 +0530 Subject: [PATCH 261/628] Fix escape sequence --- commons/database/constants.go | 4 ++-- controllers/database/oraclerestdataservice_controller.go | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/commons/database/constants.go b/commons/database/constants.go index cb1520a3..c83a6329 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -225,8 +225,8 @@ const SetAdminUsersSQL string = "CREATE USER C##DBAPI_CDB_ADMIN IDENTIFIED BY \\ "\nCREATE USER C##_DBAPI_PDB_ADMIN IDENTIFIED BY \\\"%[1]s\\\" CONTAINER=ALL ACCOUNT UNLOCK;" + "\nalter user C##_DBAPI_PDB_ADMIN identified by \\\"%[1]s\\\" account unlock;" + "\nGRANT DBA TO C##_DBAPI_PDB_ADMIN CONTAINER = ALL;" + - "\nalter pluggable database pdb$seed close;" + - "\nalter pluggable database pdb$seed open read write force;" + "\nalter pluggable database pdb\\$seed close;" + + "\nalter pluggable database pdb\\$seed open read write force;" const GetUserOrdsSchemaStatusSQL string = "alter session set container=%[2]s;" + "\nselect 'STATUS:'||status as status from ords_metadata.ords_schemas where upper(parsing_schema) = upper('%[1]s');" diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 7bb9c3a6..753cd357 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -319,7 +319,7 @@ func (r *OracleRestDataServiceReconciler) setupDBPrequisites(m *dbapi.OracleRest sidbReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) (ctrl.Result, string) { log := r.Log.WithValues("setupDBPrequisites", req.NamespacedName) - if !m.Status.OrdsSetupCompleted { + if !m.Status.CommonUsersCreated { cdbAdminPassword := dbcommons.GenerateRandomString(8) // Create PDB , CDB Admin users and grant permissions . ORDS installation on CDB level From 538b814f0c5a39f94f21dcc32e49966d929d72f3 Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Tue, 26 Apr 2022 13:00:56 +0530 Subject: [PATCH 262/628] Changes in the readme to address security team comments Signed-off-by: abhisbyk --- .DS_Store | Bin 0 -> 8196 bytes docs/sidb/README.md | 75 +++++++++++------- images/.DS_Store | Bin 0 -> 6148 bytes .../sidb/application-express-admin-home.png | Bin 143392 -> 123404 bytes images/sidb/database-actions-home.png | Bin 231151 -> 211501 bytes 5 files changed, 45 insertions(+), 30 deletions(-) create mode 100644 .DS_Store create mode 100644 images/.DS_Store diff --git a/.DS_Store b/.DS_Store new file mode 100644 index 0000000000000000000000000000000000000000..32e70b56023cc58b7918ddca0681d4be3ea805f8 GIT binary patch literal 8196 zcmeHM&2G~`5T0#K;t&BrqM`>XB5^IDm1wENB|nWIKq)i04{21#Tg46mRYkc1 z@4zc?Vu)bpAbfS4 zF9!IDFMV|q>dKfWxUvX#C?eQ+h`fT6DEd;jngPwgGy}YMH^`!0>XRGWzn5^X@KY6~ zI7=;{oEYj$40Rr;5joT$o4Qn>b&}Mk9rTCv8u}QG&L9?IGo*o+G{8?{o|skQQw*I_ zXh+T6i_QEahCN3W+NAr`pe?FVnI556!|^THIiOBdo7n%)A^)?%l{|PN;RpLSjN3=d zM_B9hF&PN!7+{ZCOY%L8OHmIvwq3@p7wmfy*2U@!racvBL!KJ6jhMiy#c=Q~f=gjN z2SE0~MBG-a>e1NN*xe1j6n$;&+I`nsTKXntW{p#+w3#-u=8O8a9M?U|8@Xj`_>7Ml zQrfM9zSVl%?~a?f3l-^jR^RCkl|sMM1?2gYzSEK8vK%=bSLxRM6K2NDG;f?$`rIegJ(aB6!}=8UOgO{fJ(~{WtZYV@s#U zN~l@7htR4}72#D!TOg0#k{2bEgE>M{TG5Fl10G8oLt{xJXp=TTKBrbfy5>_D-Qc&>|)=w+_O}Yb(3N@luamYuk%BbsaT7wmSijvkIJ+$vnmuT8FT;he@M5 zCy9Ys-YrGiFn y$05~r9P;oFL-g~2vQ9CO6JKHk<*$DTi2Oxaf&t+tm45#J?tMZ(|5e!j^UW`52N&W1 literal 0 HcmV?d00001 diff --git a/docs/sidb/README.md b/docs/sidb/README.md index e932a1c2..78b7f529 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -2,13 +2,15 @@ Oracle Database Operator for Kubernetes (the operator) includes the Single Instance Database Controller that enables provisioning, cloning, and patching of Oracle Single Instance Databases on Kubernetes. The following sections explain the setup and functionality of the operator -* [Prerequisites](#prerequisites) -* [Kind SingleInstanceDatabase Resource](#kind-singleinstancedatabase-resource) -* [Provision New Database](#provision-new-database) -* [Clone Existing Database](#clone-existing-database) -* [Patch/Rollback Database](#patchrollback-database) -* [Kind OracleRestDataService](#kind-oraclerestdataservice) -* [REST Enable Database](#rest-enable-database) +- [Managing Oracle Single Instance Databases with Oracle Database Operator for Kubernetes](#managing-oracle-single-instance-databases-with-oracle-database-operator-for-kubernetes) + - [Prerequisites](#prerequisites) + - [Kind SingleInstanceDatabase Resource](#kind-singleinstancedatabase-resource) + - [Provision New Database](#provision-new-database) + - [Clone Existing Database](#clone-existing-database) + - [Patch/Rollback Database](#patchrollback-database) + - [Kind OracleRestDataService](#kind-oraclerestdataservice) + - [REST Enable Database](#rest-enable-database) + - [Performing manual operations](#performing-manual-operations) ## Prerequisites @@ -44,7 +46,7 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI $ kubectl get singleinstancedatabase sidb-sample NAME EDITION STATUS ROLE VERSION CLUSTER CONNECT STR CONNECT STR OEM EXPRESS URL - sidb-sample Enterprise Healthy PRIMARY 19.3.0.0.0 (29517242) sidb-sample.default:1521/ORCL1 144.25.10.119:1521/ORCL https://144.25.10.119:5500/em + sidb-sample Enterprise Healthy PRIMARY 19.3.0.0.0 (29517242) sidb-sample.default:1521/ORCL1 10.0.25.54:1521/ORCL https://10.0.25.54:5500/em ``` * ### Detailed Status @@ -81,13 +83,13 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI Reason: LastReconcileCycleCompleted Status: True Type: ReconcileComplete - Connect String: 144.25.10.119:1521/ORCL1C + Connect String: 10.0.25.54:1521/ORCL1 Datafiles Created: true Datafiles Patched: true Edition: Enterprise Flash Back: true Force Log: false - Oem Express URL: https://144.25.10.119:5500/em + Oem Express URL: https://10.0.25.54:5500/em Pdb Name: orclpdb1 Release Update: 19.11.0.0.0 (32545013) Replicas: 2 @@ -175,7 +177,7 @@ $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.status}" ```sh $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.connectString}" - 144.25.10.119:1521/ORCL + 10.0.25.54:1521/ORCL ``` The Oracle Database inside the container also has Oracle Enterprise Manager Express configured. To access OEM Express, start the browser and follow the URL: @@ -183,7 +185,7 @@ $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.status}" ```sh $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.oemExpressUrl}" - https://144.25.10.119:5500/em + https://10.0.25.54:5500/em ``` * ### Update Database Config @@ -320,7 +322,10 @@ The Oracle Database Operator creates the OracleRestDataService (ORDS) kind as a For the use cases detailed below a sample .yaml file is available at [config/samples/sidb/oraclerestdataservice.yaml](config/samples/sidb/oraclerestdataservice.yaml) - **Note:** The `adminPassword` , `ordsPassword` fields of the above `oraclerestdataservice.yaml` yaml contains secrets for authenticating Single Instance Database and for ORDS user with roles `SQL Administrator , System Administrator , SQL Developer , oracle.dbtools.autorest.any.schema` respectively . These secrets gets delete after the first deployed ORDS pod REST enables the database successfully for security reasons. + **Note:** + - The `adminPassword` , `ordsPassword` fields of the above `oraclerestdataservice.yaml` yaml contains secrets for authenticating Single Instance Database and for ORDS user with roles `SQL Administrator , System Administrator , SQL Developer , oracle.dbtools.autorest.any.schema` respectively . These secrets gets delete after the first deployed ORDS pod REST enables the database successfully for security reasons. + - To build the ORDS image, please follow the these [instructions](https://github.com/oracle/docker-images/tree/main/OracleRestDataServices#building-oracle-rest-data-services-install-images). + - By default, the ORDS uses self-signed certificates. To use certificates from the Certificate Authority, the ORDS image needs to be rebuilt after specifying the values of `ssl.cert` and `ssl.cert.key` in the [standalone.properties](https://github.com/oracle/docker-images/blob/main/OracleRestDataServices/dockerfiles/standalone.properties.tmpl) file. This newly built ORDS image should be used in the [config/samples/sidb/oraclerestdataservice.yaml](config/samples/sidb/oraclerestdataservice.yaml) file. * ### List OracleRestDataServices @@ -337,7 +342,7 @@ The Oracle Database Operator creates the OracleRestDataService (ORDS) kind as a $ kubectl get oraclerestdataservice ords-sample NAME STATUS DATABASE DATABASE API URL DATABASE ACTIONS URL - ords-sample Healthy sidb-sample https://144.25.121.118:8443/ords/ https://144.25.121.118:8443/ords/sql-developer + ords-sample Healthy sidb-sample https://10.0.25.54:8443/ords/ https://10.0.25.54:8443/ords/sql-developer ``` * ### Detailed Status @@ -355,8 +360,8 @@ The Oracle Database Operator creates the OracleRestDataService (ORDS) kind as a Spec: ... Status: Cluster Db API URL: https://ords21c-1.default:8443/ords/ - Database Actions URL: https://144.25.121.118:8443/ords/sql-developer - Database API URL: https://144.25.121.118:8443/ords/ + Database Actions URL: https://10.0.25.54:8443/ords/sql-developer + Database API URL: https://10.0.25.54:8443/ords/ Database Ref: sidb21c-1 Image: Pull From: ... @@ -367,7 +372,7 @@ The Oracle Database Operator creates the OracleRestDataService (ORDS) kind as a Access Mode: ReadWriteMany Size: 100Gi Storage Class: - Service IP: 144.25.121.118 + Service IP: 10.0.25.54 Status: Healthy ``` @@ -399,7 +404,7 @@ The Oracle Database Operator creates the OracleRestDataService (ORDS) kind as a ```sh $ kubectl get oraclerestdataservice/ords-sample --template={{.status.databaseApiUrl}} - https://144.25.121.118:8443/ords/ + https://10.0.25.54:8443/ords/ ``` All the REST Endpoints can be found at @@ -442,7 +447,7 @@ The Oracle Database Operator creates the OracleRestDataService (ORDS) kind as a * List PDB's ```sh - `$ curl -s -k -X GET -u 'ORDS_PUBLIC_USER:<.spec.ordsPassword>' https://144.25.121.118:8443/ords/_/db-api/stable/database/pdbs/ | python -m json.tool` + `$ curl -s -k -X GET -u 'ORDS_PUBLIC_USER:<.spec.ordsPassword>' https://10.0.25.54:8443/ords/_/db-api/stable/database/pdbs/ | python -m json.tool` ``` * Create PDB @@ -465,7 +470,7 @@ The Oracle Database Operator creates the OracleRestDataService (ORDS) kind as a Execute the follwing API to run the above json script to create pdb. ```sh - $ curl -k --request POST --url https://144.25.121.118:8443/ords/_/db-api/latest/database/pdbs/ \ + $ curl -k --request POST --url https://10.0.25.54:8443/ords/_/db-api/latest/database/pdbs/ \ --user 'ORDS_PUBLIC_USER:<.spec.ordsPassword>' \ --header 'content-type: application/json' \ --data @pdbsample.json @@ -474,7 +479,7 @@ The Oracle Database Operator creates the OracleRestDataService (ORDS) kind as a * Open/Close PDB ```sh - $ curl -k --request POST --url https://144.25.121.118:8443/ords/_/db-api/latest/database/pdbs/pdbsample/status \ + $ curl -k --request POST --url https://10.0.25.54:8443/ords/_/db-api/latest/database/pdbs/pdbsample/status \ --user 'ORDS_PUBLIC_USER:<.spec.ordsPassword>'\ --header 'content-type: application/json' \ --data ' { @@ -502,7 +507,7 @@ The Oracle Database Operator creates the OracleRestDataService (ORDS) kind as a Execute the follwing API to run the above json script to clone pdb from pdbsample. ```sh - $ curl -k --request POST --url https://144.25.121.118:8443/ords/_/db-api/latest/database/pdbs/pdbsample/ \ + $ curl -k --request POST --url https://10.0.25.54:8443/ords/_/db-api/latest/database/pdbs/pdbsample/ \ --user 'ORDS_PUBLIC_USER:<.spec.ordsPassword>' \ --header 'content-type: application/json' \ --data @pdbclone.json @@ -535,7 +540,7 @@ The Oracle Database Operator creates the OracleRestDataService (ORDS) kind as a Execute the follwing API to run the above script. ```sh - curl -s -k -X "POST" "https://144.25.121.118:8443/ords/<.spec.restEnableSchemas[].pdb>/<.spec.restEnableSchemas[].urlMapping>/_/sql" \ + curl -s -k -X "POST" "https://10.0.25.54:8443/ords/<.spec.restEnableSchemas[].pdb>/<.spec.restEnableSchemas[].urlMapping>/_/sql" \ -H "Content-Type: application/sql" \ -u '<.spec.restEnableSchemas[].schema>:<.spec.ordsPassword>' \ -d @/tmp/table.sql @@ -546,7 +551,7 @@ The Oracle Database Operator creates the OracleRestDataService (ORDS) kind as a Fetch all entries from 'DEPT' table by calling the following API ```sh - curl -s -k -X "POST" "https://144.25.121.118:8443/ords/<.spec.restEnableSchemas[].pdb>/<.spec.restEnableSchemas[].urlMapping>/_/sql" \ + curl -s -k -X "POST" "https://10.0.25.54:8443/ords/<.spec.restEnableSchemas[].pdb>/<.spec.restEnableSchemas[].urlMapping>/_/sql" \ -H "Content-Type: application/sql" \ -u '<.spec.restEnableSchemas[].schema>:<.spec.ordsPassword>' \ -d $'select * from dept;' | python -m json.tool @@ -574,7 +579,7 @@ The Oracle Database Operator creates the OracleRestDataService (ORDS) kind as a ```sh $ kubectl get oraclerestdataservice/ords-sample --template={{.status.databaseActionsUrl}} - https://144.25.121.118:8443/ords/sql-developer + https://10.0.25.54:8443/ords/sql-developer ``` Sign in to Database Actions using @@ -631,7 +636,7 @@ The Oracle Database Operator creates the OracleRestDataService (ORDS) kind as a ```sh $ kubectl get oraclerestdataservice/ords-sample --template={{.status.databaseApiUrl}} - https://144.25.121.118:8888/ords/ + https://10.0.25.54:8888/ords/ ``` Sign in to Administration servies using \ @@ -642,10 +647,8 @@ The Oracle Database Operator creates the OracleRestDataService (ORDS) kind as a ![application-express-admin-home](/images/sidb/application-express-admin-home.png) **NOTE** - * Apex Administrator for pdbs other than `.spec.databaseRef.pdbName` has to be created Manually - - More Info on creating Apex Administrator can be found at [APEX_UTIL.CREATE_USER] - + * Apex Administrator for pdbs other than `.spec.databaseRef.pdbName` has to be created manually. More Info on creating Apex Administrator can be found at [APEX_UTIL.CREATE_USER] + * By default, the full development runtime environment is initialized in APEX. It can be changed manually to the runtime environment. For this, `apxdevrm.sql` script should be run after connecting to the primary database from the ORDS pod as the sys user with sysdba privilage. Please click the [link](https://docs.oracle.com/en/database/oracle/application-express/21.2/htmig/converting-between-runtime-and-full-development-environments.html#GUID-B0621B40-3441-44ED-9D86-29B058E26BE9) for detailed instructions. **Uninstall APEX:** * Set `.spec.installApex` to false in [config/samples/sidb/singleinstancedatabase.yaml](config/samples/sidb/singleinstancedatabase.yaml) * If you install APEX again, re-configure APEX with ORDS again. @@ -661,3 +664,15 @@ The Oracle Database Operator creates the OracleRestDataService (ORDS) kind as a The following attributes cannot be patched post SingleInstanceDatabase instance Creation : databaseRef, loadBalancer, image, ordsPassword, adminPassword, apexPassword. * A schema can be rest enabled or disabled by setting the `.spec.restEnableSchemas[].enable` to true or false respectively in ords sample .yaml file and apply using the kubectl apply command or edit/patch commands. This requires `.spec.ordsPassword` secret. + + +## Performing manual operations +If some manual operations are required to be performed, the procedure is as follows: +- Exec into the pod from where you want to perform the manual operation using the similar command to the following command: + + kubectl exec -it /bin/bash + +- The important locations like ORACLE_HOME, ORDS_HOME etc. can be seen in the environment, by using the `env` command. +- Login to `sqlplus` to perform manual operations using the following command: + + sqlplus / as sysdba diff --git a/images/.DS_Store b/images/.DS_Store new file mode 100644 index 0000000000000000000000000000000000000000..b07c0cc2d2a53d29089c95afca8976eb4a9ac6fb GIT binary patch literal 6148 zcmeHK!AiqG5Z!I7-BN@c6g>vK7Hz@Sf|pq9!K)EHsMN$14aTfAsX3HFuKF_`{S$vj zXLh%R()OrOnSt3iJ3F(Oh(E;cf%r$}sgzHp5oytv#!AW13ynGUk3mH literal 0 HcmV?d00001 diff --git a/images/sidb/application-express-admin-home.png b/images/sidb/application-express-admin-home.png index 6c08bbbcc936e77b417a7520bd944105f627f4a1..0d581d4068a0f285c28db82442df4510c10c5aa2 100644 GIT binary patch literal 123404 zcma&O2_Vz)`#3HoBI$BOOs8^9LQ=#W zIfprN&djw9GyAX4_xt^QKcD*iKfaFb{eHim*YO;$=XpKP>v_GOnA|WB6_ys}+$2~x9(t?K7arG<#X+wub*SzT3HQC?H=B0R>zlqv$h zP7($Hg!3Koh8T+Rw;q#y^U?gx1Lw*0hfN=TEI;2XU~uQeC&hq?W7O#rQU|0CoK|~K zcksj7lBlP}>8JPQzuf9SAoaZT!9y#d!d!)&J5d>D)P#~RJkEC7ig#%I>?vO)UwC5A z9N$+DA81j1;ycVDP%5}Tm3vVbXCHHuEqBgerLKo zXoA6`>s<$ntByuSe2TnlAwK>sd-=BVuD0-g`66BTw*BLpkMB6|o5zY*5qyHY@4dW# zlT`lyDHRM!-TFV*TRVRP>YGBYUgdq8+Iu-TxO+Q$_?*94@5;x=ujgWB;bUQRUEAIR zscLu60oqc>fwSho?e}7f~GpZh5PQX*= z&Yc6Qod%vht-^y)@eXwNu?tXf_dfDpjr>nL7ahFqy<9wfTs++6f46IQ*TdHbbolV^ zj{e`ze~r^2z~z5?a`*mkw0H;tf0qDHsj310T{RyU$N!&dzf1nB+CT96ukLhzH>M5s zcJP9DAdwF4KH&c)uI@j2`j^809p`^lG;s-VaJzlcg;&#?hZFdenuf-ISNpT*-&VEw zr>f^po&D#k|5Wr(RevL)ZS3X38>!uIDuPex0{?pU-|)J?-z5GglK&S!|M8T^P_VEr z@c#)1SlC;IqQS>^p6}{KeY1cqvx9;UevT(`7wB_8=8D2?w(0BN+^yti=9YRF*V}o$ z{nw&#W%J;rPF4TPqZ6v}Rzc^CMgyPVS*lM3M3~zjAHK8YV|-pQmp=I}&@s}^zt37? zC653YOkygL=o-bv&A}m*$&-PrrF|`zA$(i5?LPk>Kc-&tPx~}oJ3dR)s`ZEoUYkN#U%H_~G(NFlPDSrr-s8wd7aE!~b*Vze|_zggOFO zkFC&ZjpeNk{v&$7k-t{+D}Fe>XUl)W=^vf8)e8uPpjP%>VeR?PMB|k{w6)o~8KEAW z7n8UXO_5vw)$Y3)AI_iWKL-8|6ere5Y{Iege4%p6ZEh{wh@h3#bn2#qcn~;eINwa& zluWy)zVm}Sia((ix3u|hHry-8pC7%trK4l*h{hwLT3X69Ul$$+8mcTkD(V0jp3I)j zh+Ckos9u0ivn%qm+mqe8@c`WAdG84*C%Gls~gbn16c+Srm&@dHJ)_}WZP zO$kkrVv$FlJ>+hx`UR@uSqT$pyaSWn)g0dJ*c|Zf-NLwUc7_Ng{1eFR#;QZ*oSF3c zDc55%JAUe2-hWVWGjZ~X|Gbvw;To2fB>oF~p}ae6MC09Y__!@%JqQmIRQd^h52wfA z7~~x9Ek$v8tP)sE&cliyokPN0tfGo&*B6cKH_#yH{aB1I4CS*n=GWq@<>TVbUF}Ti z>*}%cAbSwe@!j+v*MwWkuQbrrjJsLJ@x4OCf!6v}tZH;0zB^@Lbwkl&n8fIeYGK-_ zHqd|Q7b|ZzYB!tExNElI1S;Jy6gRQn^-0keItZW8cklPeYSl8w#VHLitZr^8NYG5j z+JqBE{M@VaL`7sJrXoC!4V5=FVO5bX^Dho{<1l(kN;y*pX z^xjdY;$oR572M~5T)|~~4)Y#mF4%b4!_B#@Qv2A0i2ta1c+>RBZW9f6hwl+-9!Y`( z>}Z!-3qJUrja=^6)bfh4zQ^rb?MOo6g6_)0Q7=cn#wiVvCaO5^jL~`=_Tzfz4ij|v^=!w7zTf`FN7H&t0k*p?cWsUYdi#Pq2b7A^q}ufl65i&jLGp@3CjAV>AMT(9JDceN`HW*<5c@agk4xk>TlMWUVV1rzZ@}&|=5`W1+vDBNb2rKY7VfecxR5=o>v|INJrY-9aMhp_g8{?+9t= zqt((c0-jcuwy_|3!#be1)>9-i*yKc)Rc$?}Rc|P4JoFN9JIy;yVZ1Z3duej}CbJH3 zE_82JW;E1|g`Cl$cvxua zXcd%wk}!911*r#5^f}G`ICG;Ia{NZgpg(zV1|)>T=Bf+SDJ1)NS#a4DcC=9<(^JS5 zkic}Vz|E6cge4W!a=XDSh08JU`||Sjtc_S8WTdyL>y5!x8XvtbE(!(bMHr zb?gQ7&U&(*0cqt7_*B;{TQT^%MFpp+$`Kxi*6yKiE8oO4t~?eBo~CcK-Z)3w$(}0* z?l^Q09tnlR5dK5=t3lex1TEo4@0{yek|*T+$dE9T&pbTfW>sJ z#w+@FnTcAB?7(e^l!KOXfZtmMbJfm{vk)s&Zzw|C@w?v{`ygVkO1-F+IHPmSv+P!X zf@B%7DY9M_2o&jQzU83lK9E1M_FSPDqN}A|2G(UFa0H1sR8M4wc_NEhM9 zG2(}yNGMzEzF-SthQhWft;p&Id^;{k?6=_&CNqwD_4^J4btr$ijrC6RPO+g6ty&R< zK$BCHg)y0LG{{|fdV)zq?IV!@J5&rsA(urVuH4R{0t&cWz&2}N*6k79d8MoL8hdJY zeyK~0?t7vwtc#wS0@}-MgoI3IZU8EM8r`W~*{$QE#IYyUXSKa032e zd0^KyxGx=4HXlzcdS&(YU^y(HuPD9_Rouhf01!slCkyw*=yk{8EZn)Qo*cKIS+aFu z9&09j35oveLn!iVW3Zk0xVVhFd>ZRS8m(*?)IExGb(IyH;JjkVzF++!?nysf7S26j5Zh>POV)8Y|K}$ zbKul#*(vYtx1O8qrFzu3pK)q+E0xJi@~cu|6fs;vj$eHR*Ny8nh-%!}5CrtvcV@!T z^>GL2;JL(RQGwB=rJDhHQBvSKb>q)bT0{3reW{wd7IS?c_}O#SDYTl-8-5&ymW6$1 zgww95182IKWPjW0deUR~$P(yiDXo|U*7W*yqIx<*X32UMdeiRHr;~81!m$R;cCHC_ z+0JmTno9_PMXItHG(^hAveRrC*}b^ZqlMO{t^5RYj_Gz83ONbd7{wI zYZBq{BPe}k*i;s`g*PQ6F##lSST7Z;xf}c<2dPL>J zzwP`{vs7$Rljr2gF#>d7x8~_mxJze?5de+QFPiyj7X2n=N zBgt>zB;X)rl-=$HuZ~?48+>sHQYUjoyL2RQo7cfg4s%1p72)rj&W>rkHIx${d6c^D z7IVv#OI-d0yv+^aXjW~+*~rr%yQ~66F2RG!;YiHFYCA9|dBc)Qk7ch_W@mld1!#ww zwTOGp*Y?>ywI19~eTOU`^waz~Y5X?8x%r)axa#ZRknJGb*o{WBSMa8-Hh7BS`ZL8!2hO@*pV!zjRFU~UzuQ;Wzd~o!37a>eHRrMsWd?Lb4zx6XI`1qq* zL}FsyeB1rRoVm*8iIw#RobkDS@XCl--JlCM7t1|mE#iP-Q*CcsTU)8wWGsa-8GAWeZW zSEEbhXFQ_Z%ASq5YVA~JQZQ>X&4AcI*^X2z5sChtFlZmUhO`Q|VB`?4_xvCwwvEoW z2s$kkLbox6Lq{knhPM zYtwDEMgbx1l0;`z6u?7FZ{71lv|XdDzYrAlWag*D)DMH{(z0wTooG2EeTBUx54DZi zYcGacerd2S={6US^YDwe_A9i(D^@uPCI%`M*VHY&fb3e}Lp}#B&7K(%D+8m~9_^62 z93}1Gu?kR##Vg{U7C(PF*s^ZrHrFC1)kcaCm7VYosfnk=CX-(tF_`f1n2F`NJ1WE- zNp5F)4dkci?2t^d2WE-*9*IQe(dZc9w8p?o^YtNEZx6E@XyxuLYl>n|p8DHzrB-jd zp<|C7^rZPx(Q(;G>P0pwAa2=%!c7J(w}sjHQ&t)=Og0z*KBGLq4P#L}Y&rcs$U6Of zZfna1B7m7PYTqjA3~zBG)NZ|~zmc+mHSowBkxL1*TJQnYFu?FmcVn%W;DJ=!X zDOYcei$g@CqEbm1?YP#V>^oR=4+{ zzHgm3kb5W5t=Ag1oa8suw|{;5Ml#`7`Io=~-UUy94l-rqmi3JuY@&RH`K;qd2 z6T?hXb1L92ag-xNs~xc2Se#U|deB70;ph4bWCfaRSL(w3Vpv81AL`ciea70~vFHH$ z+S>L+4Ynw1sy4DNk@@CiV+8FD%Hs9H$Zqvt0k6r|=X=tJn1n?xL3*gbDX7hRZQ7|< zp6h9IIV#G?ud_v~zb;&)9!zGJb>7g4yjB}27j%NV-Y~B5W$~qf+_Vk7_x-uzu|YU( zF@437T|<;eUVJht_Ezms&Gzox+f<@6ps>dlGZ+@PD4N_ucCYS=jDPE9qfL9c>APy6P!FEYv{;d5=IxDfYU%waR)vVG!IeYpBc zpQEs$eWz!mG}EFBSIS@zLX0d7D(e0& zJ7bnJe^X$&SgF2xJ?4#=1bVk7p1Z*&^K1hRV`3K0s(4PsVXZ?}9lD9!2bz>4br&{_ z>P@JV_K~uFLi8Q>{s&~3>8yu?!OQ#tL><(LTuo|J-v-!9qO5Z8F>M`Rc(jJ4yVACG z#8)zAys$BoTq`K+mhf=6>qFoL^^)a7Azq}hki(s*4V#c3Z%(GYvS>7i!<~q%X(8r9 z`|14`aZ9W(FUc__dnu&DfD7CR8FwMbR5>ew)`ak?N%>ezG;T>{-Qv6~_N?Wv-iV<3 zev?5)-gxia90$}&3V;r%`?M}J$|6_;{un8_%#{TdG@P{bD?Pf1%O)xGW{?VkP6E~^q?zk{uCir>)hteIQ+{Q;CAwFD{@`3#mkJ^{y z%JatE7gHAeDP&J)*C=cnLqgeVxR$=LA@;gyeV#S$Fg^ludCN6Q?TN~$sHlw$12<%j zoCu~@GiP#^(D+3RZl01S8BTd^4AMBVe4Bp8Y3&7Q4SoSKOD*l~KlR-%@@D$FFPmME zOi1#qdon_Qs>hjLTt_3Vr>_zX$lBZI{tC8+CLc;AF)q{3_Ny|kr&dg#DGe8yh|E0> zW$tlv^s1-sqr3+5eDI5|cnSmvy%LghY@#(0OP_@px;zW9f8G zW@((UpjP1|`Q$^Ov&e^#T67}Sb%F4d-OJ zX8m<`Z(NsPY+fm*X5}uM0{}K-&>T9ku=Z|$c|@rBGjCA}syx1!lvG?>1ZPJ}+6J}X z+OS*F7X-YaH`zr3CcN3IB7p9%Iz4&TPR1>7?O<%dQWm@K=#jEPO|#CK`z+(>WnIX3r_tfL!)C%%+fI2fr(lX{CZz#;z;z_I=%O@dnXRpOeEwW z!!ZH47cl!Cz!YiHLQKw)l%3 zL0?AKjBv<3keAoQ0)Dy%=TBYJBh+QkTmi>6DAP{wN4f$}iWLD0CT#;+{m4j`2@S*x zJRRya{f17*c!KNozS)$yW#bv$@R2YtBz=fISEyx!X^GZelad*&Ew7u0d*HhuOYyk` zRUO?6BO&l=FsntYK62yraoK67_L3Tvsj4~C&z~G#wSk}FO0S9CSboOhu!f4+^rhn0 z>ohaSJ8UVlCLa!?+RfM}lN{?M@U2}{YfGPchR->r>}$Ofw6Qul>n9enoB-2d{4|A3 zv45HxhdqHLX8eg|bCrZ~}-K!mXWMk)SMSb+4 zJ5P`BeF;&WS6hfCdz2$^`0nmpn|(tH%N9+Th-MDQkS+tnuiO=SA-MCPv%$gp0T$5u zfSL)r4bdK3`j$SGL+_tsJWuX7f4oM3d2TRpm?clnAw!hs`Uk;$Utb22VDJUp(9ByB zwHd!Zu-AP~+x7HW5($4O zeHq^M!GA4FO8cF@!qhD00W2s8LDe9~_Iw;N*6{!CDyGEACQb#iD6CON~1qfct1?17Vq1$hBYSXEUVk77NLYRtwfNL7ywH%t`mrt`Qh(Zz z<4^xF=@`)h=GHB33@tfJ-LUfBp%uWFM}gy3>8W<)T}!FVl7jci{z#M)SC*NjGxKSz`w4!7;3 z`B4FNl_YR+jKHyQS@RdGPnJlfUo+e^Q%O*9e6b#JP=Jt8UxKtog8WJu|a8AeJ#XmQ1U1G zkU3}$f_rCeZ54=#O?k$$;qnEWCMY_hF2F1AItUq=ji-yf(I9;UyM+wbx?OVDjp*}s z>=Oyu{{6gB2hGm;HLO#jm7CL4!mVCxGHF5O!1!}$g_Ck4x4Lwef^X6HZ0OHl!?~H7 zhDRk#2wIJXMwa1b(j-AXHyK0cxp#skx$?Hqk{Bb5ve_pP4e$Ul)m;~;R)5)xvY!w< z7^$_=5x=1t9FTjhM?ntrVUU(pbXCtpYcRkxYq)E-ut*Jm{t+acTVN&Lwl>eEl~b6X zr;fqL6n4%|#7-c*oL)QI7(n-s-Y$H^j{t7uSQ-K@Q&o)1eP4aH(G>dHZ`JRT|QpR6;9#pvo^Y6gu%8uYX|HxsAzI(fcOqX}*b)=8Q^GM6*rC0Lzv zS=&S{_qbDOBU+uGGvH`EkX1|r6(H%L!?|YjG^C#0)sf=M;wA+dZM0(;8`LTwAc!P0 zlSaLT!I_ams%b|03)UeQUMj}nmgM0`URXgiel4Sbc3f{ucH8;-tMtkGMa7aH_H)ZH zL3ana;GX5|h1VA+5|l;AT$}rsq)rXXag!?Q!fp#ZbVcGFs}~d8oabxp)|YFsGt!tF z9n88VV6K*yfgQ%OI{&1!PX|N@<4;ym{t%dCtUI!@yO(NV6{J_>X4QA3zX@WPj|4YqxR;O8?J7*$xDef7i)V)mTg$QUmyxr^weUQ=oL zI%Q|t@%Zuyf_OfOMX8m}wx^?+Ov+Bj@}XewfV`R2?k7sZ7!h)kO?yZ=kx&vf>h}Wn zit51^bu{gDhR=syRo>9q%k~DsSUJ&X>VEk4R2e*qI`b$ZO?S6d8 z5rO{ojhO(B?S*j3=-m0cQl&Oytit1BPl}h;BdT+2dtmc2#030l3zf0HmemW+#{EN@{n zX%%hpsxgi?W=DRds|NH$gZ-`vomN-Qh}h=rp@ZFhi!mN)D54m)Zj^N-4+nVNU%S!Y z0dvznjsBc}-U9k~GRfaY%8Loc2_=n-2Q|TJ6MD$&MD8%<; zgIFFk^sQ(xBbqu&9)p{4Bl2>g-}eJDF`{=sK|Y zHdo?|_Xuja2`a6qy)U9LI`OCHYD1n!3Ss2qK2ec_iHfOE{ajZkv`e+tD%;qNA<9^A zm72K?Y0oNU{PtyT(e1FB$+Nsqi7wRQkICfblrJyn=XGFrS8{|YUSd-QF+=STiq?gS zzE)TGNZN$w;kGm|{sI4xbtBCYhzh_pF?xMQU-u6TJULXgG4Wmt;;{Nazb_?l5VnFs zSfNPY-F^(=$9TM zq~|GcW|D{X!KWsc@Ea+UHN*o; z#{~$!l|NLW{chqLfyMm8mcWC`tb+!~&jKkTwj&W3rJJEojH}Cj7)U;^*N6|2MTBE) zK62IV=TqyWg`Q~z?Jp-&v;qS?n`8dTU|> zTXD|NiK#LD?fYtjf}CH#Mo>)=4t8#Ecu62XXV4RfruAIYiZtF z$^`rBTQKSy3}=ZUYx~%(#C3!ahPM@SkWIH;&n<0n?_!%B$ZY%ddF!s7fL_5f)={zv2NJu7Wu^c}oi; zh8I39JkLyCRxF>@jY=Tleo{^dEA+JDdYfW{a8@Y95d4rjh(6#=>m7?05jUVAGwX7w zUjw>uiZlC@vtok)_NE$Ev9Pm^-9Sm^40F+;?~ocG@YlQ zM$hYpLO(prKV~QqiA7txUa^ikn;J;!>JN$$5}P1Q-;)flkruWUPh0pR){>JG*xR*# zSL39)_~$~GEB3NvY*)^xb)F5@6$T+2gT}5`&lWYW$9(m*+y780f5lLWVBy3Fn0%->1o7q?UUmiD{Fa^0rz z1`1i(udG!FH{fh`RzS7%)7w-k3yY=LceDWC@duZP4i z?OGWAVIj6(D9sJc5#&ny4qu~7h0VRzZY+>rISt*5i1KXf>oyIE@ncq4$o;j^|LRD= z-SMX;D;XqjvPg&%DFNv}7-*1~hKqJK0J>I>mCa7&B#sG*uepA#y2Vw#QkX_qKG62f zI7D$)b*%Dox|nfZjI$4>$>6?WpunQ$X%q|=j$d=T-bhiQg{O!N=h$AIb>qaMOu3~{|}Dj_vz$~78oa^(r* z6EgLhV+HM1Tt4m{&=Zjc*(gBS=m>HJh%3IQDK(696>C|s+?OYEI5+z1lu+B;w${cbOKTeo1B>;k-G zDom{=bHZq#Rq1sT_H&^er;2+5o<>- z=iyn8XZKPuwuInILS4d2;u^*kLy6FKnfhTd-JRXLI|Wz1YeRx6D*|3ersE)8}cykSnOAvd%qt63^r9W6h zBoV^fTO^5{(diJA+@1$FPE1|z^v1@I1z=*NhP$SKs{^gB3hzB{y(OY}>+dDjlM%2_ z0JBm|tZWHcBe+<)!M?7bzg{_PsF1vqReZfOyGSjmb~j*jGIfXGD*dx?XK?lX8)J1W zLbu=2qnJ0TPyJg|IM2oKLzYV5<0|PyiaC4tRl%Ut{F$*yowGl@7nrCfdi!XIx6EIY zF(~?T{l)=;a6>;rYw%Z!$2hpr1zFji?$4Z7_ZK#;DrE-vQ!>@ejTDx}7TdRWNo*|m zGds>Xk#mEBN>_ZuF07b&u z*LUS$X}7m%%YuxaX_q$m)O*z+Gnv_xPOOkgN~K?^FT8=SWOOV}HmdrQl4(-{ZHA#2 z=@wX!nC}9Reo<184eayYDoh_-@kPt$-GR~i5&8Xh&W@v5gOu&N@or+BbEQ zgUCxxKLR?UmxN6PXbEKW@2LHeqf<;7bZIX2IiXi3`oMIw z{w41K*_ouGKKF%l0mk6*{W`n|;nvWH!(@(ARbh55V6dRnFD0&HcZc`Hj~(3IZ-XY@ zF&p(Z4*jqPExdT*$IHKXB$}PTiG#cE>LK?>%l-T&h34e8bSq{%ftQ{~&D&M6hMa-} z=;x@j-gSs8xFetxX_bp4C*D?duS=~KCn|Mcn`GTDKngWhStiZuN?;fA9_V#u1=&vN0R2XniuXL zV|-xJVCv%jHpuYPgru4`J(I#rQV4kY<_-#b9GEwtb9<7CAcq@iX^=eNv^S7Op!L_fNg&Sj& zZ{Eqiu_3N8o7SF#ir+=F4Ur5wIQLf9*r*TfN-wLVG%>u8)^4nuM#*A15pSo)ezK(J zvQJD;_uz6lVb%E1FI`@`1Nusv8)HHexW2w7f2rKB|Gol3Wj^Xz!*IE0;sT-K`iQ1k zcyPL4A}DYAdic;rnn&)}nd@9ABfXYSy~NoK`;_xD6O4=r3@3F~Hp$V(a%?$RJjlal zBh0y_FViG|w=r+M7W#yZ@X!%XVKZs_Y-CJHZ>6zOAuKkf0b0MF+UH7$3zS+CTN7^q zU=;_&!9C9JsTPeLnZ|Sz@ft^3LR~{WW3gz_Brkisegjm{uJ*p6%T^|LV* zQDV|!HAcsPJ+A2LLln|alypAXF;jiP4%#VE7KL}zVJDb5+`n{b^*F%T4uD$o?V~E^ zXduizI?|@_S7L|4iMBqk9w>SuL~i)?e@xn+m&?+M8g1{V>BSxhtwx@{u|l-C~Orpjp_Xy8aCUEVKtcR6(i~1i4ZZ+kC6F#K%!S3ZzB^~EISJ5 z7%%HD76~s9K89E?J|^0dn^9m6t&%peh>1nbCo&P)o=WiRSpkLZ@tyH?*Tp-t?^X;D zU2A4wDO&kW$Rk6Ud5G26kh@$x&2l*t_sLGS%e}gI8&}LSB+BO;Q#{m#(pT zG{{BfR159G`x3O9{L^Org!OS?WZF%rK#o`nbRlOXh0b_72(v}t`<=W2_Yk%8i6I3t zrH211IuALN?ns4S7|!Bt0{%{#RM(}m!56RE0`_8le!DqxG?u%XqUZiZ)h!Oae>*YJ zLC|e~sQdo57rh3n>ieaSuPrtM$-_s(-3P=IFZ`X6uX+3YksQu63DzqD)b-G-TyD&Z zAm-8TmGiP_Q#hkAzdk^S2QQThiFoEiMq?Cn3VdxoRd*Bj4=BFRV~##FQ@zG>pc;L% z8?~-WiSs!b955TsX@p*~8XqZ~5Y2Bv!dWaPO;Ojpb7R>8UQ4HN>$U2&tVP9 z%$BzB%G?YN27UycMkd6Zug<|wn+~}smPrF6ehT4WwhS!*G9b;EajvO#{w2fgdV0o9 zs(a;lh9@^;8!=3G#fX>7Hp{ll8Jss_IOcG^_o;yGE0HtIXt_eC^^> zpzS>%rQ5~fn4d-8S}eg5zGX=n&+iOS+9J9`o7GaDTMld3MfM9=88Hi3>|JhjD-#Q? zEVYah1Er35O}aIJ-SX_YS#w(1xY8Ibt8uzwqBV(%`K(Ufol+c}B{n$p3$jQUtuGir zk?_`7#T!qpf!y>gT*^7tOBre1lb@H<;R5PzC|{$}!C&=woK@)uczMg$e=|+yB(iuL zKA9;&{ov^%A8zp0VkwO5EgG_paHAtBN%z+dFmc?CBh~F&DZhP5_%lB;Co^EgCfg~j zm4+FmY-B|($p`aX)RO1%$Ba?G>|{yu@1?i zj&}x4Bg8|dy!mX(SD#SOWM^u0zugQk)Z1~6Q)^mo}aha$tZ$N zZPmkF(lM6@`pl=(4`t|7cYKY5Inadw?v(-GCJW_yqZx{W-AijxNZ+v-#N)}ie8r&^ zo7;y^@;{4L7WTe#{FB~{0o+?iAvl@c_SOesX;2pmpSd{@Y2*k@fQ=;U^q!+syv8psXBAlR@Yl6`|wGAiR98XR=4KbqGJ?Z{%;$ zdzyf6269u>1V$G>HP6m^lSIXc<92i~dlvC~TdM-(_d~VpSN*Zt+y2j&utT&mzyZ0= za{mcD62l?eE|tf#mxFkjGtdf<^ZahHca7oHrAeZ_$ONf8$$zL%9h4Q3D|f~*w*`f= z2^q|{YCq(Un$L-LjvoB9E98O_w9T7pp`XPIkA9!V`*D-LDb6KHZgmbM*J5&iP zDahI0Lcg_XQ=07tL(tqB8cWArxFmWmxFLn>##&u|i|EN~(JQOOWeu5+v0N=CgmItq z7&_|4^!|zCh8l=(lUA0A@Ea%dCsw*VTMNy#qPaO_%ktKxMr~t1UpNBMCoC4yr5^8j zP}7p-U!tL|`KS%n47^=*HrJdMThowN0}E$NlN&dtQ;6>{&MgTNq}#=cxYAmL|M_TAVCxzdvlmsCtJ#3xkh^gHed={hA)=&gH5uQk#^F1>%IrOk15 z9|{I3ZiymoIQ9xAUi@1f|MT3I|IY7Gv%oVvje0w6(d^~YT1Z51RVf15{^3%_9dCp> zbVYF)`mwLMzAxj|bJ&LCR!h-FtLd2i5yS@cau@zaZ{`bzgG>#fe8^j$s1)pSJG{C} z?k%<-cGJ#6b=EY^ljZQbKaM#t+*O$a!<8xFDlSxu<@K?<`}}_W`d-G%P+?P)ipi^) z7(zvHqOdI@BWQ8S0f=G^h9~vIUXv(^*AO>G-xz^ii()@jE9A$*SZ*#yBGVr* z&2c}FF5x}!xpC>gBEx^2Ra_3<(olc^J%^spQ8Tx#%6GMe(iQpKon4zqteh7;^j#{) zpGadZ?QjX$NMED+%Q+9vMiBbrE;tI*TVD1SD%bKXD!bkL)O`YAIS0`o^0JxyX74wW z+w92wAb*OiD$mU@>x}PTo{lNoMr@E#BqWg+k921A?WasM#9NI*uJ`mk9apN;C00g4 zhGlX76Q>`I?qgC=oOJ3B2@8q8ocezjvEKrCV(*GI_J=u2J}2!h;+pB9m9E2f7LH;F zgvKLp{j<_ro9pgcTlYLK5)Krf!ggrw0_4%MI8L<6?0d^Ck3p57yjZqk5!={c94M9G)RmZA>33tDAJ5%h&X5OI0ylAY02i>TGtF z@I`Q_C2lTn(OvOvR4nmSi3&?ytLwe+(qi{`ovlF4r?d>1_i|7#m0}OYO;%E&3$7b? z1R6`_~HJM-uF!$R}1UZ!$C<4V1Pk)mBzRw}s#6?Nrl z*CAw_lo)_JO_<6%Jevm` zJ_3RVF|ZnNqQdf~`0YNQYRDgp$G#maH}uMV^!F(L&_GA8@CLna`Ci+9J~#85j;Tug z+lbaT?PWJbPhOt~xg+TGlaB0Q`2bXR z>|XbK5*52S&GW6Qx*Q`?C;z2nF0xD(T9{9#fAKv6ZT-}Ub=9thgY&#(4XgA ze&!3-{Li}p{+1=5s}F|D4c-1PVrKAQSnC`~0JQlG)mHg@keGDbUSM-EyUl;hyzU_J zd(gEh(M{28yB+HIkkb09Pdq_w7-m-&%+HybZ7^y;5KEw|q4rB^|ZDzbULdkG4Ze z9rt{n4E>=o{@tyQIZqIZ?<&XqJ-$D{k6uabV_%q0OjUT4|3}fkxlIzklQr4;^#8)L z^IH@{_N{zuJGLomsb_fFDXIPL?M)%&Lx1bB;9FCh8^g8(Jnhu#T)63pb@>(uNufTmPHb z&mTk8RyDc1jVRl5_v-(JsJxd!>c-TbO;tvlU8tk&tKsr`E9J+V6V^EOOZCP1#G1))xy@}!(q8!$ z+I~rqe{;g@53+pU$Fn*^-WOv2;LhK-p}R!X8a*PjEp~4zgjX`oKZ)f5pH7hu3f~kp zAG1{6dMVvZIdW6bqc?wRu9e(2g-scex{rS+V_=7}#HCFaK(9`IHX(fvwwV&&oM-wx zH9sf6|Jmj|xcXZWeFUFb7TJ_uQ>o{RHQ-fY%|%jwj({etuBmbRH9?w_Q`y(l+_Zk$ zK1x*74*%-aUc=b9ggDbqT9MB?4RuGrZ+FPi7Dq@MY6`Ya$~y>n);zsXkYv`>vC;MR z`^tuNeUM!`GPqNlU|_iF;DY^mi{O4c0kgNfwqfuOp#WV!&z%{!tpv?pO|Sl!vh%OD z`CPt_w#I9yJLpas!6A@z2i;kt*bLoN4a6e3@Ds=x`7Qi&mGMZ8L&xfe73BzBwk@7LJOc^i=aWicF@rOgLG6l^zhX)nsFO zY8C9#C5uRpe_qm%u6O7K7Hxff`4W$H2F; zeT1S*ZB=j~4Qa`~*`DRuCw1IPun|jSrl7c{(|QpW2F$zh=PR314R)9;kkgY@wg1(; zxLI!z$iu#Xwn@WB)JRo&v_rVKCr0`O5l+!61FfL^Yr_^#a(0_1wy58$w-IIMx>BAl zKHWX!3LXxgqOv5-Hx%c26oyM_H=<{5QC$It)R_;ol&P^q26(``A2Q+d*06VSgE3>P zh!GjD_+!Q5h+UFDtPg#LZCLRuXr`60s)c4X6HmQ%Pv@;TIgq zK;vnZKg4Z!uOk1``%t@_HZzr7X$SLBFR1U_oR7)X%>?aq6yL2>5VbU`=!slRg&HmK zH_i-5?N)jq8ndb?{wty~6rIwae8DEXw8`2b)KK|b8bUs3YI5%C9j(V!0Ju-Yd6O;AsaK-!Ns&Z3iFH|O6tlpPzYI- zB6aiI;KBt%mSWn+mZ#Zx<9^sgw|i83JHyP%O7Z#YmbQy~@9RSWruADc(-_@l?joLZ zK0lj^@m55ggumE_i}sZ{6oHXalF9!~hy|A=m2tV^j#s*>?5G`TfNvfDJ_UQB^1x#a zA|4mt_z=VkWM-8lQ@8lpTh3-1Z^DxJH=rUGdI{08e7^j1q#({9pTRC=Y}7bXB5ZLBlL@+N)hQ(d}Zy;qagw=uQkEl}vbMslZX->JRsKF_izS|Mi}d>Pbi&88D&WwFCzWtf)7$0Ak8@yE;rw?&J6xXH3e(6@UZC)Z zgs90ExE4@>ypT^y-i8Ma5Eh8Kfb$j=TF+F-QrboH<-079-?h+KhV^QXA zv|8mA+|yxkHIzARYz7s0X04Ks+zzVaiGsRb#GfMq5!|xkC1? zN!0ca<_^9yvzrlkbB%=u2S2RHj;QJ=>@m?903Gr(&IOpA1z7TAEO(*gj+>kmLgnUW zK3ODhxh!i9%5{R_C~3}24F#ku3<6z7i=X(J|8zKopjhE?FEEM<{T@2YtN|MhX+O+> z2Hm*j@97foh~DMSqls`QSN7TIXJVTC_X3l5lIn};L(?y^b{nx*hhO3}N~>(EoErmfF;x~RipNFm%ifm0re_5H+&57_1BSNmC+Q` zCvuvpSuQxW(^TX3mVAY;@3=M&BiMZ7IX2X49O(hteDNarY?g1$1!2AUjWmlwZ{7p- zf3bN-FY4P3TtOVgiQjy-OU^{!T!?Vva4TZR-^$;o@Emb3(1I50RX3hTCM(_*v7k{naPuvKmjfsrCi|}oEd|rg~>}X zqekY$7lEAH;PWqtU%2>Qs{|A+iv}RACr#$bwNp5Zg|1HS(>Yk=6RM$3OAbFSx)=)xernE0^&r8>L_XUAoCRdoSrF;0(K*&>ykq*(`b&u?~_ zuER1t$Mm;aT|PBZPgaxe9}1lCtH|D{ght1T5N<-(jD@ZQ6hneH6iq0%J!A&j`4k|Q z3lf9P8pCeR!v?S5)U*)i+5Yl`lM*dY;u5@e>lD#&^><7-mAF0EwR^oxN)B3SY^YE) z^~<=oxRXl@P*bEentl9v-GVf%%$(aa+JUMwzF?|1Dvk{e-KtY*#|fcf4!&0Wm9AW2 zy)MWZYW^1g6z`zd=ggfb+5WA>*UN!H7&}7a*mbO@Sr?XKW&GBPkf70$lh1&OTY4Of zD)Zq0sYO9JxtIUah|js0OxEhFT)-}&P&Z&d9-5q#GzkmgZ-ok)tdoP>yuSZ-JL$GI ztT%sjY85vD0|`+;*ua6h<+f=`)s40ZiF--dff|i^@@+TU`qR&dm3D_Bd~ev)Y<(2JKqpK30I9I<7cAURABgv|ggm*cnQOzQH}A8E|IA8MQB&s! zSkKfN1}dJ$!+GeP_O*i72~KC8y}npJ1gXv%$VnNvx3LG^gsUIrSVjKsdvlu&oBge$ zdyh92Vb9rzBShC)l3A5XVG|*4cB$--dUjba zh4@WnKk+vtx?lJc?KhuWrs+6d$=+>yCKQhaN3{n~HM9fyi>8RuRM(N~1hAr+$E!F) zRZAZzIFBboenbn}<{mgK7#uu#K(T0Hv6%96>?{hN-%P7hAWSJ-BX$Z%mtEp3Fq}2w zEU9?&BH3fDODR$Ny+X*-=Jj-l)R{Ge>KEdNy|F*0P(xv`gG{+*@7^QV2U)p7aYo`o z&$^j;Plq=Y1EUa7$0z+lFYd3+`cw=>Hu;r78jb<7=QlHV^oXCHh{;*g?57!^{>p! zd5po{t+LCqh#q9UvSAE79>a{u#wKB_&q#)~)w-ID7rrns;Z$D7;1t!KV+UGCi|rcF zMui36s`7pJugjuy*(522gya`g0MttX>4_3VhuDCHJbz}H7#IuF%AkG-98T#uBE+ob zrU;|1Yz|Vh&6hHmMyx(;(2|>PJOEQi=U%CKZPIt8d=-#XKX4YVts*KTnc8xlU@Ws& zsXchlWSBl?@$NA$7Ys#tsc?N$X2@aNI1hfuX1HJw`k6;VEclxsEL7bcGm_ZK|DJj zwOzaRobgR-H+_zO_!#HZOmJd&?(I#V- z{p*9GldLXHk{vXFDwTHb&*F!RnqRDvC&w4KwAW!=KD6Yisa1DGk11Jv)jgGwA6*xp zd(#{u;X%xXCK}Jel3JjS(u z>XA<~+i0?`_+8^XB(wUC9dx6@%sAa3AdNeM+ON?GISh4?z2X0;?f$mKxFwb+{uJ_( zOhs$y`#;tZU~Xc&p1j7#%$q7ibP}32j)tn_A_Dx~M4rqdU66xr zfIY{pd!7z0oXFb^%Q!{fJY|V@$9doM?FrW2*(1@Yv~bq0lT*#Q&&N|R%jn#avm!J{ zNyoAu#b0~hhbbtaE;JFFCy-L;Y)h3}9*#^t=c(cT-C!%pd%Ir0H z);t-lypeg6$B)7$r=`~po9WR1-o21N5tH9)4_9}yg#9$ufqL`>_Cl{2N2bt~wmS4t?z8uJ26nRx z&6ZTh1<_6mNzapRl#jN}!svN@nM%GnhQF;LqH>{Q<4xcAbx6tA#p;i>a4LV*iCt&I zjLbggbpGPRyszIAZ!7JRt%wWUmpO68Elb+t(5ZF=49U#``)bHTfkVxWdzzU|0}BrS z>CgD#`tkB2&~-;?C3=#>+62rosL%H>rJ$7zls0n_i-wHv(MxIQUlg8|a) z7*#a*`X#yMHUBg$Fr^qpHzaT4Wz%m zM~|^p$OYT{WeWKYcU{nCc7+~$^)Ysr=4R58$KNnrj66h9bZ!=A*`)D$^c%F)t?jZG z79A!yC|9KyMqRP@{nG6(A-9B^@k)IFWCX;mV{Opf^}G+wQDc;goGG7P*`hw0KLGX! zz1nG*raL)(ctrGz%oi>^7paEP_5xGbr8exbDteRm7=rOv_AoOyfVr}9^N$I;R#tB! z@bYY-4DR@{YrmrvR9qCd_}x{IWNwUJs=Gdl6gK=0&36Uxl)zgg_V=0XhzFayIQY?x zD?kv=*%!r@8bSP<{M|fOH;=b@3wLkP}Of6n&KclS2B4r zA?A9oaVMfWT?OA&cW)u{pTxHupC8);C-$)(!ij&M(xzwI z*;OiMWw~A+&vF;~nOH2=aZNEu?O&oZc!Krtt=#Pk}+L zD*_71TqXRvFr=$(9S|sEB1i7O=I{B1f%wAN_BY1Az|HhSHV1NdlU(YN?8m>nXaySY zK)n?c&-C2zyFxh&W$l%l20wvtMoero`7M6bl*VXUjN1&rYW^*<;uLMI5K*=ezaTWV z8owoe2!hK_A`8N?;OR>Gfvt1xYlDCh-q;teK=;}t0onqtmg^(`14;JR{Pj0q8}JoR zEOsK`9ag)9jp@T*&=pm%@2X!AD>8k`xm*2+USaj64s3!aJUh8OC&EfxO>n57+L9m9 z!5#K@-asWr9kE4Bwf8kSB`G?m!qF2<_HPX}`K6&iW}qY>ltvti6k(1E_;EA6a;x4% zr99Lj@TQw4_ndMq5h|hO)(q1AS|_N|k26hNKR=Pwpu@n8HQtb1G*-hg-I1?A1$WsF z75Eb)*kxJ1l)pKgLMAxC;pG)g41ynGcj-nJc2spZ6k>c(s4iq45LHQ&co@iE%=Z>f zGUI?-6n(5cH-y|eUg=1980{dT{F%Ncrn?%hAXJLtrzM9}Hj_jky=f5Dkrr!nj}&VK zn8todykh@*HdF~RtFAYwO>)fm9`dUt^|8)bFfrj?LR0c`;7j9@3?zd(cx~4OVHy=? zw!=;&#!@1Q`Aqce*-FO^G*sTQ>U^ z&QCmEo0AD-ShwUm+|$$IJJ8N+F9x~b{I36# zc49`uj%8h>?Pg8f?nP|tcM8(XlB&6MFg@tLXn&JpShz8Z%gLz$SSbHKP^Eo48YW@G zQwS#!pRJ#K;v`%Jy#DS9ZMCd-KM%?j4=-)1%KhBPCQQSt$+yV!#OE5AHC}BYF5lip(AteD+dC-aP9ii zFwuSuo2bGxf)bMimyDEErY?HkJryRE2!oz6?H#nX!rTo0%#n*+E3o}Ip;)d?gvE%Y zZARdNFM4`Sf^ElO4nit&)~2KO1;V8WQ=Wpk6s4cM5#`WOej`}6D=|84&D4w{^tv(n z!(Z6E%kWR;k9g6C(-rzOHM=gg%k8d^?9nRB^*GwA64n*s_F zp=@lsT?ZESqf$9BqjTaN2j?$X?E&$BaGcyV{ue;Jeg|~heYq6KGdVP4>vpo|hSCDq z5>*SgoJDp{0KkdfxnM&GCFsD_>gwkEMGEP@gFXXH-ES9F#%$y*;c0yD^kDf3fkC|@ z$kmadu~I1vL(M-y%|_@}2+~a--a1(hM*i4B%iuWI-c3CGpgcVLcg75(`^GLo-N(!~zZ_z$553`|(NE?tOB+Q$zF1S0 zFX!LyH#jhwk99R%U)om*&J$n61k$wq2!_Z3Xa)Jsb$B?sFC{(#WsuKhkEMR9#J~2DOZSYo1T`<zjl-tepdN(AgZqVoQnQ1T9ZAV6`%*vvs_o5hFBx}{bF5Rr~3LZ^7ARwG}R#N zke{^%*srfswg8^|xYxAG*Mc+)sbd7w2YnF`Ux2lKzk#vg9+iFg`xff~&h06B;y2;l zk%B;tde##0D24B^!grWbHLYSoWP?CPg#c=@H3?VmX**EogATdo5HUAUYkjHR;v9yKq2>IA5vm za+ZhYnnQ>z#0Q}r5rWIp*#P^2QvTAwUIBtPVahz#g8Kp>)&g`fBTYEOH^PR8Ms%7> zt}CqOp`@4s*cJw}liv+kFf0!8F&N-h!4s<$RC7@xZEYAP{TyRQUNSHnWSA|nwnxPr z_uJA}R#7QOAIjcq*ni(+VZ{b6!He~FeWETbP%H|icSh#p=kOqY5F%`S0bLh1`pR_~Qu@MQr4R4rny0SPG<17v2;?ZpGHq~$X@+d3qZsoOU-=gXh z+jLiwMc{mFo=m;vcMFLE^oUBa?D<79VpLUH&h56dBq{-4YOc}ekrDRSiIZ8`Lsw|0 z>~?m25`i_VkIXTV@^oeCqt8YdaxV*ay_p|jb7?Tx@vz9jvliB@%^l3rsU){HICq=P?OSIEo% zisnaEO5!c5Ugp1-v5T++ra7T?5#x~qK^Z7sNr4Bb@X`*iiS0c>j&ODB6QYahgwDfT zyHiiLn^*Yr;x*5x*pU7Y6r2HgmF3TtYV1g`tuUk z-0Lm&0ramcD7a^FfZcDIbaMFk)*;SI`#vRxc370&o01uwkE|M5bFYc2$jS>`%jf7( zCM&`#kV>7AT>_~nhs(Q8l?mj2`?73>jsDqC^S?J#w8`LRfkG({ObTVUsL;joHr>WLJRIZ%8At@gaXRQQZvG1+ zXShc04~lSqw%CWW7$p`|e5V#XMLBHKw_bFXZOUAR-!|2zxJB534!Y$%h{=D0rv^uu z1JEbn6=P*hpD=+QEc6!Euer%J#A7AWU>0X&{qiCQ(1xK+dJgXw7LKI#B zjTcy~sAj5OKnI{8exVFsEAR-f`T}vYi z?Y1y_fGdC!=D>kB0>^c=c@cwMJN&&yy~lUkC>C~J4wm4G5A_SVp85TnyFMI~6}W6t z8xVBUfoF}hW9e$oQ)H1t;7+6Yce55V7nK9&uv`CpKmRF;rZ`HQtGL#aj32-QYR#^= zH>C^c`we!VZl5i2KpzAR10E14)M+w;xr5@wU;XF43RG6uAzn!D%ScXU&ay~<%SbqJ zZ7qMmPx@l{{DN@nd_u8ZdYNj;YvUI=b^n@a}8;`ydO*5QGTzcWs?^Zesl+ zP^;6ewpFf9R@EW6-5zyr!V$(0+GKdc07hU47jL$xC=DswAF9A9WMN5|6Q)}UzpF%_YPc&(J6w`-%A?!t0W zI3sMVw#H}f)E-vf`e<=-S+YzZGXDXp9?_&L-34ke6hehlLgV}9@#;k!R-gD~)h#PM z-|C;??ziB!C4&f&HK^BjqGBq2gBz?fPagS9#e=Et5L8yVsAo{MlRK466eHJd_9_oD zh%~ras}r|ka8qQvpWMBND|2rb_eA|XIXU}x$}TPjb7oxj+f5>Vmt?(oZE_{+PN7Ux zDW#W%n;pb$E_=}jPA|8Cnt}ABMrQ8k+SpAV&6avVYEo7#4A>& zpv76bo(x8S%7!XUifv7w-r6d!<9JJNutGDNHlgBDpu`u_Z>;)y9{hbRp0huohHz#| z`SI^MX6BNbEVHYaCm3I&ZB4e;&UR}|UK^k-RU~7qW5?Ktn#CvTsriA0DOO~_sK6Z- ztI=yhtE^2$Kp@W=ajiD_>xew#eJ{S1K)ZzcMO)QLxO4(n-o*|%!@aX7iTJcP>*pZV z-@2!6#qElT|K?x;ZGz~ed14~;wk1%c3SVl<2D1iYQOM-T!0i0!HP#x%c7t00IlNkX ze(-G@WHvC}u}NMt__n1Gdd#z!!_xv=r`S3Akzk-69ATU8)a0%qzcxS=NRhIBB$cG_ zH#m7*f7JdhX;WX>XYT4Q`FV4IT}t2Lv-6RMxG`kZV`;>D4sVP=P0jk8*KHH-ei~b? z;^~nz0&fdIhL}4b&j>HN7l~0Xu2`kwk%?FH$;BKg(p_X(lrSK_n0ZmhdbKy)w+p!(yN!A~^jDv)koehKJKd+U%NQs_5Fj(!uK%?54b*{S!eEri6DU(}7K_ zNc_2e?H0G&Hi%?ATnWwW!GgP!B>%g;RI7A%+v zh{;dWeS0)2CN|G6f6uvyz7b)c<8WdCJwmT1o@U#-AMmgtb&;y3tA5bW`7d~7yYr(6 z;?`19bno}nfVK^?h1xfU!5zaJe@WbTBx`buMCyVBR#?1cH&48G0e|Xwem`Ntu+5F% z-hU-0?EUte9S+`^vqv}Dccwp_-@SFa@X`O3^N0{Bb?9_Ri1&@ZD2KgYzI>gvd_(&B zblUmdseF6C$E2G>B-F>0va`&1|1|}b>owC%M*0-}>ukBh-wXbryxCWi#ee_ct}Cma zGsb!N|GR~XzhT0lmC09=Isc2-KP|cl@YGe*jM=3~_6iD9tTv6q1+bgcpWc>xVc^~` z>)Wt%ZVRLuwnyyts{Z~z^-z8$xCM4f{kv3i+z+n@N8G@jzP#&J{SWnPzcnM+g7ld? zF!5nB^TtgJrJ8~S^7)!r3-PheeL zN*C#rX!+M72M#UKdj>NTX!T4l-hZuPUCyolqXrh?-VgrX_}z!CAI`LBmf^aJ(f=+5 zhT%q?n=7^6M*sTIoIelE(N#LIJ7;#UwEPh*y+03K$nd`VuMe&N^U!+T>(T#G)?G=9B@Wd1wZp6k7OjIetytM9kd|BFf&J`N^n(3seIcDFC__uADPj$F@))*3^P zWq@BF->rH6Uc_?lzQU}P@rRRd_KL_Je}<{G4tD)Qk7fG2KKZW?{lJvN5WhP&PXC)6 zUJhU?P41B!(=T_W6aNoCAew#f@J73n*!#wRD|?HG6jNuu{r_JJO;V3xzP_p0`)B`) z*Y{*4W%w@C@VF8q&kDn0K>bScpOlplXyfc~qj(_v(&>PYX0aCNDO z_wSF3Tfo1}4PeAfl0vRv9NJ8JV4V+-w8;B3A3CMrZS?Kwr3;!lA)Cwc0W)u2Cb?T; ztzTf}mF;dG+aq0nn^jR$_A~{MjQ?mb*akqb^ooL8p%vw%Yt9Q81+Qb)^Yq`w?YxJ0 z^k0>y)hv|NXrYHLrdVB~l5w$SE!7od&h7hOI|d`a*%z!|R_N zDHajK^PdR6Q0{|ly}V6FYD`1|e;s_-z2i{fHcF>A zfb^=x;p@i12$#5jsHiuOBI1B;@2X7j|G{1QHXK?>G8cJ-u5q3YOE?zn+C|Q+zqa#T zlwMNp`Sz178BPd{US?%Ng(6S170g1z`%zpXr zoT61E-CuhMQ3GQ6B`+y}`aGEp*JdSB_)bYMhyUdL>=X42y{s)Dt9pTVq7{aG6>!PQ zpH7W-8}l#Qzp*CL6`H3ongUCCp}CM59hY9vN!%eoR?dj_3#0-h5r( z+Xr`UCa5t2cARZ09bkCx_6oueW(NrhqX;v<-(~2+$$qMA7eIbzNET7uuH77~!vKs zb4=%kAZ%5Q41Z}poe^yP?Hc&bmLCOkd%enb_-2XwrBBMMW6uIrUna#(C>!Cunol`Z zo5bTwOYEAs0v*vHLg9JYx{cD~vewEZRddiI?UmE8)rCUA z1o~xDCL`>wbo4T32&X%(-zqx;s10}5lgv0Bi4FJ9TTCt(4zeVfl0ALcFv+0qvo%VgQyz(c=fBh19#?~p!)%yes> z_iKp6Ah8T+$Xa(x>0E-^(6+pTR@;>pzxOpezR35|iTDL1+9umjFG(zGTMhEi4x{v{ z^O>lebJye*0(IhgG{EB>^GetUT`w&B9Mp0A71uv3>hXO{FvBr+!RO_F{ZZY6J5K1T;^B=7ihoyM zYyaIh;`;&CPc3n;ug4TwRo`O|)gu_ym1toZbo&j- zMz*pVKZ?Lf^xN)F=g|(mZSSQe5dgF|q*(DL>GD9eZ~8V=`mK|z=luFWSz@wq_1#xT zd*FdSE(6e{YXm(ZV3}jQlvTA+-S)!EM}Vgm;VUyJ%I72*;3r2NbClbM?d@JY`6tbX z(89Np&GII^&g6NHfAv6d;4sOtO))1DWvptnI)yRzV3P1D>hOaPa<-A5=U=&bxx2*b zks$9puoY6hkStY5TAYxQ?%LezOlUt=&6rQkavh5(ckR8u^$2lMl3^x;ZKvskrm3p* zE#rV{-Q#C9kcW(<4Xfjxk}|q)hKytcmMTi+Rif3ayeC&{(Oz49GhgnPx-{M|F`4Og zdX#ui@a|GZEzNv$yA!g0&j`yLlyfe!j-llk*v)a``*V@d_XZcWUpQ9a$|rdQ72j~I zoOqtE;6gN#FxCn_Wyc)!Rq8U4-=!RU>f@7^*Fi&Mds?|m?s?Bt;D7Z3m>p3c^rL(( z{IfAUg?@K^#SO||ZdXm}ZH`AjT&5_EwsvIRs)c%~A$G>^#|u?~KHggqFnA2C>@N>} z2Dij$-4ESD{A{zW9viAwI>1}^GmmGD7eD_aK8?{o`YvQFr{Tl?&?lM4d~Tv9N_t?X z;FOpU+UhBJm!9YKo2w`aMA#ft-G=~n#xru)TMdBP9WQ7ZB%jtFc7(Z%8$R|u-(Ce- z(N8NFMxyJ?vi*RHZpTAzH;{_VO5{BbU#aQ58a(wB|A>c5!X=S#pa;0~(AG7|V3UDC zXSN!!j!@pmY`+*=U4wGNhSJ$qTnPxF%F%*HAAV&$T>Ev;3n5~CY3KTlqTYsp4x+D{ z75_DlJ~LjJzIC#UwUpXyS#Zy{gLbkIWWRZcB-N8DPMITIYL}tD|bos6;Z4XVuZb08k!lJWOwQL{HhNI z0Jf5E4A#1o!7a+PlBF!>vfOX^eG9!$^vQv2@fw&S+o3cJMgzIv3nH{GPfFVNp+~<9 zp7|&DvS`X}rs*u+E+mqa!Q0IseX`y9FEz1`HBifD`}0G)hMPz!i&8UMw$}?KlIkdA z0YZS<3OtA2jT+$J}1zZ$(Mk~2Y-W|or`R>uP#Dt}yb zQvU8#XZi_M?flN|8;7NWe!bdBBmf}_AG0_h^VM6zH_E`HF9R6|6EEq!hanH`zv5oU z5ttP`-t!WrE;l?&U`cm-BScAMaTV+WMhk=4r) zdy~Wqq724~S5EiVWx@=#fdTXqcZe?O>V<~V=j2?244Lx+zuTG{L1>HC3#S&IXH0A@K3ks*$`(}3kL_{s03EQa&p;JxA)4kS_XqogX03b_b@GOFFl?<+a zEQ7JP`6j={`t9Ngo|mav!_yzAxgqp;B~URsXsqf~9SewmCu1ZXiAi3Pa`##7_GWh3 zC2(L71aMf4tRfp7$q`1|g#hKYQ6^gRKH_IQhG1vTTA1?PV=> zDxLB0&_Am2-&~83Fy>9U^AI2;`47`oFD3`>upCWw4lj1J8DiyOT!RaIs}OmHF~EjG z2s`;o9@+7bLT~qwk)d*vhl*}_=4DgzL310Gw}CJx2rqWMA=;Zcj(pHLIkuwbbOB;X zo|{ID0X;v1)~T6I1`Mrz%BwielT4TX=@hal)eTn66;>> ziIv@7iL_1~j)z8Q})X*!jtji)H+YdJ+hAKJ2 z-9|EZY9(Fw^YR0` z{1)prlJT~rA-&GU@b@*a@ITW9ep`+)d5MhkrThQe-HmW&Jtp>=?-2^`#@LsW_kg<7 zf6JlO{zJj64#sXJXmxHpzWRN0l7ul={9t{64scGk(=}E;Ou`DB#Syf9+acY9j9uvn z(_*#)#JC;|;|*psse`7TpXiIh2hDtpmEk>IgAUnHB|aV016}DeFuc+tDew{sb&$d`~+JqRiP_q(WM}BD&BGSDzU;|vecti6gyHryq zSlcv(O19R3g8k+A7^V$iPAi|yH!EZ7r7+fde(!hub~11&>V!fU-TIql@c5SVymc&a~zN6A@Yh0D4^DlE%?q^i`RE&1gn7eALUp&_V)Sa!?$KFiMQ;n0>djBd*_ zp>yM!_BfJ4gu9XRAt3DS(TJ#ou-?f&p|FLM@^!HENx@)Is>1wkIHch`yhO3F}G zf>T$16l4j4k6oD_z<95#?8MwqdwWVq(e33IR4BdrrOTs#I+EtB_>4#a?2jIMmH(J$ zmWaoruMfoyK2QSI7lN^6tSf4p8|P0gHJylXfk5c0!oxD94*mXhPcbQcRmxpoJ5B8A zX~Mm^nNOXA%e5spU!pQr4qSlq6WfV2Majw1+bIu|T9epf`)`g0ubrP*}BO~mYYw0_wuj5`bq@;A+ndSH(c#tsO2t3$trUQz1nvJJVcDIP|u|wR0lkX7L zhBF3TMysdhW%1{5G?osyGX3{4?_pm;CAMninM)<&UZql}s#@Q*aqGabhy&*=YAsxs z$~sA6?iLmAkKOcrXWOZI#w+9mZ{-6Cqese|4!55#+R@KB<+-g=2S9C6*q?{NR*o2R zKx}*Z2{#>Z?+Gc(FSC!cy~MpJw6Yc3mQaG)LY&#;=5)45stNWs#2rGAlxx4A4+`%q zIuFRUA~m} z@o}CA|56HJ^2|D5S_9<8ccwg4z6_9A$4!%F|WJO_Szt_)VnE0pIS6rbB{frQeD2F z4d_TlZ{gaDjqf?xFzmnztQM;(_$6;{cUW~oJZSp_z5=FD> zOVt>jYR>_SjgpxP`=+TWgSi?9z@=MWLBF2Ak}q4;TSgChaO;2v70;jS#Q@g3o(dJ? z^V*Uf*oXJ?2_H>Wb0@ujxv$~YLaj7aNb#wQ1*X-i#y9bRf@iApshg{<>c-sFnV(pi zqqqfL|E07CqA`5ES|3O5kNMFW18YlYhF0jakaPa%hi0n_4}VfJ@6tOEg&r1q(f!e z(&e`{qBgj5zDs%ME1tE=a5~P!O%a^CpSPn&wAK{mK{vDxTDr)S+L0)h6+%p(W9{Fm zW%j9l2snGmKX3A;^f(#T@4dI8(yU>vzt(a9?t)_>ITRQcQOkP8`jb+9Jq3?gFZ$NGt0R2+88x!Q0YZlnnEq2N}c&V^6 zpd^8EG~*{ip)`p~#n1i)LEMhY-xh_X>{}*7c)`Bj{UK zs}FvUpN4Z(%L-kWD?hyt34@9oXx8Y2I7t5U5x!lA~b;i3A z*UPc}UNr0iWy^JQncr^5-6o7-V2WM%+NQP|gr3eZowKe89%X#wPZzOP^cXs+`s$xu z*WO)bhKMiNl}h}_tb0ZygnsK{%T>P;L#)H zI_!KbHsw3#qk#}vXJLc~+-01ZZpHG!yhOBDgHQFOZ?IozEzdk4rp0XKAXIND){?gw5RQdqm_@YLSu@tSIsYpDhVr}6VHhBZP zBOGAfn7%%p9pH5Cf}!~kJ}~u+bUv0YoY7fY6BkHjq;K!A~0uYTsSu)L?Qm{@Od&^T@le( ze4;xmaBU!!PS}z}T9&k`kzPb&7T=YxW6A{{pLP4%9RD_Av^`zkpBKRM=|9$;^MaHO z@|)u|?1O+S693J)GD{d!>#tGJgi)eg<3^SeC)u1MehNu*7*$y+!4dtCmjyQxY_GhFM>QUKLJ3Y4A7RllB zE$;F$*S?}NsjlKa>vQ;IkH;|;23EN)Vj%pNBWVDv^>CMGdBplin{yyJ8ygaMpZj9E zhl_E6ZpY#Z$)mschtMI22`sA{>^X5iUZadjGPe;W&Wgob8LHm`MTPXiH`(-m?a_hO8WFwWhr<$o_@byIGO>3?&@P1|2U&+C9i zV6JT&77ADAPo*<{Xr4kqFhR&V8gxIlI^a_gbv>Mg?GHn!(QC)EUAXQDU&g0{e+&>U zhJk`4NLeWT!p`TeVk=}xmJ0sPNYCXhKnNb(C0FjIdw^-5SHL^vy}BtZg7vp*4IIP` zi!5`RDzI5RV#}o(8}jC-)cQNkgKsxmT-B6JFQ>Y?03{i%NwB-Hu3@s!OVf5-RpL+-qXQLLaf&0rKbi>NI8lQ^FO@Q9I zLK4o&YJ)0V?Zl0b zNhf`!MURx$vpiL@0@CXA4~B$pBjgH83kT$A!Sp z{x5u!TZO`EZ?C%$%JTS8`!Lv_+p}UNs`sQ};ihxuNe{*dFcC^GITo*meB5|n22&np z@v%|B25KhtsmYUK(Qs|}RgstlCPTH;EpzQ_Ls{dCd!LIFP?#ZR-H3`4Wkc#B2K)KA z_?$>Pe$kvxi=nz_4BJU}FKo9^WSV?7ZqC(=zaSTJXJ=5B;MT|lHQIwd_lxAJrB`bd z75f?eLJNfnj!WQY(d4!VkP2?rt+Hg|LM=UCd3~-}-v-Y}A8R{up4Cbwpv_{Zof-Ke zG3HKPB@Mn^uBY$3BDxZHG6iMT7ZBJNzS5{Zhk`JwTnp2g8AU<1s4Mq}3lKiUzQSA=eo0d<9+96BjjbXAB{VZ> zzw;8L=f8-=0idG8I}0B&9>E|PT!K3|+BnL@y%#?bxLs8|P?>%icqL#Sy%0A#J)eJ> zE@=W!Jz^G%E(u(jy8p9?AO~V0l-j<4nYQpTTC~Ni%z8P)7MGl^JY%}k&qu??59k+# z*vC#U_SKHJ**hrzx!>8kq~R8!r`ZC1SzjOv9jTi>-?EuRL6Y=gd%D^5=l)GDC1lXu z;v|0)Jl02ZF-XSr>{8$A2Y0c8&N4xT*Y8agk``?r7oA-oa-+YyWxuIY;qaPfoOKdeWaGDzVTNTdDGznk|UUYfCVNC#_AU@FgMZHZQ$drt9iN zxYGkQ%U1ACovGGh)OXvt1Ve3;WYM3noP!3qVEZL|Zs9?Yn44sny@ z)TGsrGnFhI<~|w@Gm%Q*l)pq$4x3+ioN%S!m?_foEDy=4{4VbTRxyOGY1CFJfC&Uu zfPdeOp;82`nVervoQO~$Pc)&-i!4-X^utE2K|5J0-b5kSh2D7+0FC0{FVIVu;8XU* zO7(znHJU}(+508*IFn=_17F9z{fj)>KsCZc! z0G>UzD9{%swROG*b_~=_kxlg`lMj|rxhcIyum4z3$4EDY71zn1yWgb&_iCQI||Vq4IE?pa2Cr`parv8({Q2j{mB z^zx}~h+gXy+F)|BY!SdPkrb2Z6kzBQw3?Yiy4vXr^Db*1VlSzHtZ}vOhJvC77m1riCf=44wEkq#Y z7X-I=2HnHf1-K!)xX|J)K1yd^tqPNJ8jO4 z7qJb7kr7IhC7K@#=6Q^izn_n$`6v!36Rg7^Ic?#K)KS{{pu^9Z?ovk`J6Y{0*#!|j zJT*uzOQ7%bn|Lra?6N($UhO56-!%{>E#;kvVTtU zeZ?7f=ITwy3Ke3@6w@!!{F%KyVSgenCiShV?NDk-ggG$~;IW?N1~A9=m!StwypR@guX=XFeQnM>0tZm9i{G!xcmJTC zmnF)r>Pz|u>L55`cFnHglvCEJ)G3i!;-}~^BSp9V!zVl4ej}7dIgLsvDr;eS_g2pe zeffM|C1@?qnOYgxduekSyTgM_TMS4l78v(&Tzz zz3+eS7#WN`Hp$w1uRYgVbItiY&kVLGY}tO~K9I64cR4iOX`{gJZ6kLTFX@xz?anV?)|9SLF{oe?0q8b{)TXY`jFekD*d50cXh=iE;kV3+3!g(g9RM2qS*+#9tuC z{WX4$-tXZ2C*v(uo1gAty~HO6L;I)tzgm~VH^BN8zGH-(H`_=V zF=%(D%>1jlg2&l6p;?E3l#zz`*tKsJB3W(z-H6xD2HN#=m4m`sNrKj^hNj;f43wNL&u2XkAuOOU(8)OK zk*b1}e8O=%$7u_CV&0c2-%PSra&J2e$dX58jkEt z_mS5|3Dmyn)QZNomU3qRRN|oU;=7*-Zo~$6WRxC$Rho|K{flO=`DRQkvq3`#fze^{ z@LKRghoOb{mZz7urraZZ2}j3lb&FXH0#O47^(47^M#%YKPovzh7=&*FrU7%h-Hkqi zuf#P=E2bwHx%s^(&Qk2^#@dvUjJF+&i*Hr&vBI2y3`vKr?FM(*6GQ*)_;*uIQ$Rh7 zm%?M|l}O=Li{43mR|u&2k6O^_P|>xTS+5YEpAkxFE2|8G2!)E7rW49b3E|w9*3A?i z;=^+k&Ln=~t%OOV-;_kiX>AnoQJe7WD-g^%Z*Jj;G zzlayLPO}^lZq0`?+{3Q+yF)I+>}GYnEIy(>(s1+uTd;fBwf^Ygc?gJ{4&+4O@Zv0U zDM$}LC~e$1vaWcchSP0X`eSRf)~otXBL~_!4hBn=u+=rvwF}oihib#mk4oI;^Eaiv zbVJzRvHhef{Ua}Wa4DA6TVD~QYG(J#2<5gNZ#C2hobLb2s=6wkoU(`!jA;!KntmUS{&f%83i9Y#vX>z(B<><&HD;1?t z&)__c8sd0DxxNgPr9r^QXyu^{0Nc3wLQNOIk$7sC4gxiCUls-JB|?sWR>G#ebn;wp zycu1uaGHmIKg$OemX5bBz;}1DKpr$(S>f6!IoH5WkJqy$yzHz2OcO5%0$8vL-coMAcb zv9EuTcD4Q-kSUcq(k}&94L&0XGx{ppH{#+p6=oSFgf|W}v<%qi6x7_$P+&Y^Nu$d9 zyZjbWYY+SgJ^`0?$20xtZ(bhpOY$=cY)FG!4of%#1Ea$^%QGZy_gl1b!*xRQ=}Dzx z+DU+R`gEaYX&f0%kHTOYJR9H5MoQ_7+xQeSQHrXzu|tUXehqZ2P&|Hf_RLVeqN#C9 zI-7hjypQYAbbNa@^sJKg(WOcE)d0L~)8@6w%QC;g8oT@ex8>$S`7fxr)N3N_YG3H3 zbCcl(hES|VAB{HI40Psj@V!(|W1Vv*kI9qaM9KaLgsp!;u&A5Z%T18ZO0=>>Tww`Z zlS8BEhPG*^iRedyG-vxqT5_c(9X$=mzGqo=Eb$2XWX<5S$l?Vv)83V!-L5vO-h#aJ z+c^rRRWY!bQb4ND8QMM7IXt!Y#$&z`#zcM>X;>InTqQZ(JG06RU4FGbT>~{rw>h(` zUF@iBz-|Xn$y-@j&4>(}`DQ>=(uYTHjney6b(@SPRVYA=i$YE+gu z)eaLyA4nOXl&9zO+>FdVMG8#m0zd*8%d2-M4J)gOKABn+_)>F|9?w;|z!q0&x2crr zn606*p(mQ9>ArqQAgA(_*=ND8rD{7e7l=k^`WbmyLYb`TPj?kS$87C4B~G@#I4u%i zm@zDpcYg=BxmpO;%w1dd6m&fCSf9;Pws3ZXf?bB&(IEbZt$qMzR7)g%=$^_ndJ(*i zye?QYeaqgOrXq0~R2q!?ok!-i=-*%}sjG4*FKwNsySNq9(w}U!8^HIJRC1xFvwlzqB3=N^3FJ$CT@U;xNJQK5~5f>jkKtJf~3)Q-l? z3+Xcbu5JJjnaG$nDaw3*n@u}HS?K)A^!56+zYJ@ePLvo_0KB00Nb38a!V^}N0ExT9 zfdq+JMsbVu01_|<3Q;Zd-X*`Y(7f9(t3FHliJ~uqw4l_NfD%d@79`wx8IhtuQ3D%FC%M?){SEc{8TL9#t zt{x~odQkW)?eYHljpFlPws%uuynRL53XR*ZEt5@RZKq+q8NDPu6cdJ=dV=<08vRN& z{LR$-x0pZ?HZ<<&N&0eTa*mX-%HYP*;xX0g(q8R$-*f5MV@!pU2Jc&2cHai8Z49YW zs76@4T_;+Ht9nars$wr_ExnijEnMdqeexrK>W#gB zfjr?DuqN}BFrX4_oT^}Dg$+`5eYP-O;mJ+Te7iN1;iz_}jsiB>i%%ZuL8}gb`fVb6 z4&41)vS-@u11mimg)4E%_ED(P1|B=kr7`hVI-*VD_W-W9HGn_oQ12R{cCySxmHRz^M|`eGBt+r{ux$*9HEGk`n-&+?o$fN1 z)!piXZcUY=#tgZb3ajsiY5F46m%_#NA~uIz<`&ya9xC+ust%$bStINhEAMn{-K1#T zi+aE1O|wND?L3GEu~>Y>ZC5A^CN^iILwzF6+^x2~fHd!}R}UK7r`wx8(ooan!#m#0 z_<50A57q?~Q_EQ|F^m(oiqfgu_Q$Ri*UW!9R;+g2oMaZWONOxt?CLKA>9_qm0!8y% z;)yfOFvW`At(y63UA{&W%QVkOq-Xnzp$+%G)B7gndP-J!;qr0LPH?+^rqu;0i8Mk5~6$BjWb9t zB8X0+%4yB|`B!WKCrDtjRyJTobj&l63n39wOf~oBx&pVp*6}&+qV+e$T*J;5tv%Z9 z`vpP!80}^-WP)Q@x3Xkp0oBR{vsX$Mj0J$_mMjs{)rRYb4A+x+5JJ1HJon^dhN{~7 zvvZg$iIZd@$4X%(JDKet<)iP{H^p)Dwl*FxFB$+svs11hD(6klSodoPNhc?M<`^SI1}4vDDkL21man~Npxn|w zn?#gW5KP%W50T5+WAKsgw5y&}} zvE^>GSsw{hbMMr}e?CnKb=4IftBbh&d&w-%CmHgw98VX7Kq;5^ z&;Oc1ejeRYCi_NyV&)pWL_AVa&$Dp{n96IxR4`tY>|c8!sathKv0){!KdU zx75S`gGWdFlzebbO`-U0Z~PxvV?V##{gaKRuZ;Ltoz}mzrGEZ+xk8GpAkA4PrT?Ee zK<9|$XdmB_j%t+A{5$g-&@^tk=CN~VJd$ZzIJEPLMd&_%%DA7h#Ls%bG~P)N|`ondEPe_J${7q;cmJ>7!B^ir?0_-+Z9oa2MzN8Xu?tfNNFpwRY+@ ziGRLhoof$bVtdB^i4F(6uP$$(bFp*6#{WfamO$bndqbk;%g(RI?0+628>AnXG?e>HiO_0g_ol zK+Zo%!WX+*U|jltT=~x@11PW0i7A1VGUlV#Z!h%kQ2syvAqBpEYup(u#<@xT&o@B~ zaG=f4hkW_-$Nz4%|9cUsxu31hiGeo%xtL5u20%)q4$S;Feb0ZpN`w~i3tWoU;FaTF zOqu_y$L>CGkOIDZi)Q?vd#jN>BC|o`fjyeR`=1-W6_F3P{?Bvr|8=bWduJ>^0oWkQ zfQc=9FpmG{+jvgQHSQewoKubJuQz$?flB6^?uCngD?fc}nyA4I-M3%yTfro$g?Q_~ zX0I;TIY~v+P8-1l` zG+gf`F1C z0GN%|6!LFl;a}}uiU;tG`%N7L|Eoq@>?@>oSpIpy_x@bi=>MA~sQdck49+{-$%?Oj zqp|(#P5wDVfVJ(TU3iN$b$WccXA8))3UgGj{f;A=>N2=-UXX~ezh-h&p}S^ zt3|v1^HK%%llquF)QK@%7`XqR2h?x&<9C11K6MvWm{nIgFLP8EAZ zus!l0;~(5+5@k*Xg>#qQsAM@*$au9RGm`B{4UlVePDRR|ZCixpy9EMdLtR1JDK78* zO^J))w=ftdFd-~oyF9bXc0v<|nY=BE3;$KBG6H&0ejNjm-W^W}*58fZI31UtrKsFIU#!>rOFUFA%oJzruA-<+Nh>li+XxtAnh&$B`)s<;`v zYTA4HUiD76Vo!%>fy(DbY!9|GPWF8zobhs6@|#&wmET_v+>MJ|htG#c~EaX~hquneaC);cF+e`ER$(R1BW5n|3 zIX@2~hV2Lf>6@vTf=7@k{zP$Wn04xKhViqygUpcc-1HAil~4#CqeEaP`!fzo`!M}- zBKe*~L5;pAP9st4liD46)OM-Fy?fGjjfeLwN2)$Rozmz}zlVT!`aB5co*z{w-|LCyu->NUpJ0=29}O z+g&m3gl*|q-AgHd^-eJZMq;+P)twUarl_r%Q16NYLJed&*Y}mp9)}QhOMrA{#KxB{ z-F33tZG#LI3H!A^j&Hp!T$ly>QdQyDMlvx1BruaaMX{~73vSM=7aV1#5MKS+k-$KI z@SZ3xJxqO&c&E>~?|-e&!Gin;w0&!;POaxUH)jx84q;6{l2tm}zPU*?OGb%ZKf6p} zUdVc%i~ew{Mis8>x;~m0PEM&?c5uXKKZM}zT^?1I7;e%JQ2=OdEAQ`wIrMJHDa8_F zS9`V2jtw8A?f}8Xd(rn_M#;x=$}QbjT8=k_M(&t&%C)_t$c~4-<$hHWh@ClFOP`85 zaMo#XQwqeG2JnpDa;5hK^4ZT&)$)0%ZuMiE~5bu(5gjO zoyE`h?iWwb2xg`RXG(vR?~UW0J&}NTDTIT?R}PKazQrjopSWyIY3N$AgiGT*wr0+b zu%K(5)b>wmsc`lhL?R^e%RYNwef7O0e$4aI6+t+FO286n;J8~ zx)qn`)gV$EQjb2BJM9ab_u9d=PKV6jF=_4m(VmKXf9=j2AbRK2k!apyyh0UyAt`YI zFWlU#Wq3NsZGQH~Z4-DN34o@>287EMMC0qzJe}9Np=Y8`zo#Y-$+%>AhciFKTxg&g z6vMi}gI!sGj+{&2HEulNUt&AsAZi1$i7~$BxcIiho9Hw;QTI)kHESR>dON(GoVMF> z3W`J6PS1FT;jbXj#i`Oo-ahdq&1Xn+x&9=9Szmo<8%2f2aT~LQPc5VJp6)*7SJl}m z;TjB41E52z!K)`z3X`X4d$$E|Fa6aD9itbW~LF)6gJT!ggd@a)|4@YLnDi^rD5rB6}eype3 zpT8g=@R$%SZjS?yq&g_g;uT&cYy0*zZUExN{gbBbF__kucJ@I(yqN!v+}c3uEVe!E z9_@p(O*J?(%qFrA&3)zy;u!zk9J#BfZwk$8wz#1oh>NxLBXwhx=7&_of}Oz0XE zV0wMF3L>YAtQ?f+%*lYA@swc-l_d$!G2P=OFUKiau)iw zUO;7Gma6_Iys*zIqt)a%ToqMVTYb9i9d&xu-tR47!9S>`IQ27dPYJE{PAeRCb&tHb zXtTLqv2sst78iXwowUZjjx`Av)>OMKn4sgpK5wF^WPy76)FDNHW$jjh2eTcqpT--$;Gt!_a8v#+q}R;5VVLfq=R<~Z@{ z$%Uq1Ce1lwGp-kn1_5%i0u>A9HAD9^Q%R2_r+weoR4BzGvyV-WDG4DA*(GU(_}e6Z0gO5;IiMRnJY{P zyQB4y4eq7W6Pm?(bsGDCh6MkpHM{h&Ak5Zo$GoWg};wz;}~8%du+ z#{f5bUCAk0DT21)jk34xIq{S+WDJevafc{GiAG+g(?~bmz=+4klX39p9OkeXbRoNT zE}T(d+x9%4XtS2SV@?At!m?!m{3c%)=KwkXeQd~VP~zAI&l*HLv3WXQVOn|f^ZB;k z(i1SiuOI7Mp10s!1v&v)sNSzp`_csota=f^u2ZOQ&}Oc`|2$D~qoOjqve|b0gAE*&Bi6tCq%A z(Y@GnO~=sa8d z_pis*?LiLO>6JUVZ(PNO&NUUTfsvPc1m6|{3;{Bly z?4nzMxAKI%>Hy!Zh(f|wIC9QR&bTW*hFRSkO?z@6EN|SD4&!SU`}QXcCFk6mn!yAT z94_rk4u0i65^i1N{cfsP72Lg9$AY?ff7hqhq2*jb4oH5IJOGOmwg)NexZnOIb?JA) zp3$s3{9Oo|rW;1~zq8P8^xJ9)_zI$#n6lg7+rII%KG}?Wio+^|Sz$@cW|CK^- zJucZm#r#w52N+G9$%q;4xOB)7=#en>VD+*D! zF1;6QLVg8uid^F<^H|S`uU|*>H~@ijyV^l^wuX%GYN=|()6?BHD#;gF_e1qjD%R|8 z+I{NyqU-;<`5gyy*rugCbV%u^h< zPWnHl1t} zri09dE;2E^y}934X%|%kqIL1Y#V(?KE8bS7(pmSLxBF+(X4D|wBmS88Kd;GUWbeAE zeP1zQAs2~#b1K$wHhM%-g9xC#1w&K4fb6L+Jy zQK|#K@1m^3i9R*QPrTLSG^a())G{fYb?)~P%`W1fq*uOlJ6|P7eZMDLE?VEV;eP3m zb+<0~CBn$WLl_V&;Bja&p7rtsxX&}?K@;WPGeire8$P0gAH)KdW|t6TGimI!38MM| zSo=vnUCcdBow~TuxT~Hg<(&XN*|Vvs@ijh+Scd!DI%xlO&@9%Kf5^JML>ctM0;Kzi zL0CSWynnXQ5Qj{SMctq2F1|}(X`tf%VSRoDNSOA-bgRwWBp;bp5NZo<-Y8|h0V5nA zw1rm(*-yC#S6>&G8Pp3;W2^V(NUJXHy2*9U`UwJH(xKAUPbuyKbI3%vfX|F$XM>KU z70R2l5|@(&+nSQsBY=}1-P`h*y)rN92G@gb(Y`gPHTvvJI_B=Y<=y4KOIua_HXm+d zinq~h652XmN-k}RGK4A}rwvW;&(!yiHpraJ8z}AFBnSI6Q@Nm@mJ72B-0$;_Dpx`5m;%T5)+4|6AYj!PLN$wU~$ph~vhRKJ8xv{Vo`=F1O+;t;b z@hjot3`Nzp+GKL2+9(cAm@-0l%>TOE6>z;qat)-@ZDM0`LmM^P_mN;>FXQ%dV1(9! z&#cd(81YDNU{$ZBN-jD#X#@X&J=5}b@x3ZS7Yekv)+#wDnSwkl*OaFPR=`w1xqM{} zlmN(7aLPM)e9!*ySt>SFPl3+Hua$wpW~%5s8MNDGzOjBks~Y}XrO=Jr+_=PaL~oA= z!9N+dJB{{R=gS3lF((Wg4I(;gk0J@%8)L&piNp*5ui4K9l2nGEF# zQw8gVba3p1i6b6ScZ5>)Cjn_{9tUfXs|P+>oT?}%kvO0jNQq5AbdL;0&B2L;X)d~h zYW4cl*99){*W3@cPF^O92<#~)@Y#ovpTrF#b~G!THZjfVJ5k@h-wPSRe6BiIUDLnm zqY0>_Yx>hs?|&KhZ8jd=&5edZBs$=FAr)IoD$|B!h1Cm?rKhr3wKD0j#5pF{fc~SBZp&po|%bhY;FS%JwD_w67hqtr&;l!EY;fE?`xA< zBVB&3;^{E4FSn9&Q47YNK_B>?8sxG$vci<7%X;01+rd!T@Z*CLd?~RNKypo3!6=ar zc>zSH)BO$CC{emz{?vvu(-v0f$AiYK zgTeCaXIpP9`v`-4D0%-D`vV>GzjDuAl0>bBQM-eVj=utPP1jzGFKk?*cyyousH+&9 zy|b-ai6C6EY1ie&jW$0Ism`o=UF-*Fiipr2;Cy+jJmXl0{2RgLnjD%%<1akebIvzd zwEMp*4!66*&7XkB_9dcaZOGr~qPP@-Y}!%j<9c)c8nh{&=vwwSI*wBcrcnTTC0O=j_AQ*-CkYH@AV#6rBFhYDZN z$#e(BD#BZ)WaKXndS?!uSMIJCv!{)YWP4#7W4;SE836Ry9o^pPvRMH0Pk_Z1-E2r* zQWHm%Y&G$)x9Z*2nIV!Php=0REt(iSD-j2n!lGV9!^z^>1GO!tVIx?Ojt;6^~QrH zEVuSdLsS{Wcvf^xr(f&9?M_~v_Ipiyj^~(8#+T2vL5}yG6KTCH4V&YLJmN@hHJ=_? zm&$)zGgBJ1@mqg=^fV;4uZYv)tX-3qOM5}0?M!Z&o~B9hz=M%cQfVIN+H8cc(%PJY zU0*~{mFG-QffNKLtvdm#zgQYA<^En~$J%CzV#CD|Dx?QnG?!@D^kBAMsxtmCQvC`SA(J#6S6GzNxqUK>SJbF_&SsU!>*y+vUn{jhd zlc7H(_uJ|Kxrt6EPH^vY)R}vj1>ivj@#&(9c$`d2g}3El2%aRilaz7{;AVLl&N6josiBn z7~nC zqG^fZLY$lpMNC8&JxEI5VBq`u&i%MF(_lj1Lz!CZ>6x0(h=ra$$LXdQ9lP57(P2t= zIJNTGwvHYtPqjV^PB>XPhVS$;pS5SJbJ{&K13r7Z2jMdK^2)SgcG&C6V4NhuhHMzeO9txm>7p1U0J=SN+c=)j9>+i+sM5^h8D>wLu9a4GY^C3Vp} zLnFO#j%7ztr*92fotJTCe!;rl6rj!PzG`ZtinH4XWM8o-&0I7} z#8B2tX{|0&qK4Us+l9hGH#p4HWMQnkWzKJ);&Ym;7v}SZ6J9B!!f&;e*nowa{{*zk zyi3_umiFyiN7A8hZiY{jUSs%Sxa+k#*LcX8IAxFk$hs_E*cS-6I~UhZ$~b9CulIaf z$)AQIcFGFxEOzU7KMQV2j01F>J$1Lb!}&P1yD%yyOp-NSckf`OW{}iXwuh7 zDR1&(ZcFlh(@DUM;E(DoUy|*RGk#gM^qn;~mjvCB3BB&IIki3Zm;#ffbZ{7ccA`w} z%ka6VI7s*G7>BX{NX!^sOS*UGYem+}hPqzFmZ=@GVv+e-D7&|v>1w#R(2%)rhCoML zO?a#U9$rMn=CfD{2rQa`%$cXFEY;8ghJRT(a4@5H@xjTQ9-+J8_W3_<{m-YC^zzua zPwDoXZzb@0j;r?zZUun?cTCr*B#WY{kJ=w!)CdgsdGgE$tuk1|N`XuI;&}yBFuC5> zzy1w~lUdOS)hEtmnT(;x49m=6z5Iq7Ukl0WQ2xkWnqH+B%Z>~qwOM4+-uufh?)2Ze z{KO4S%JPG5&h%;V@dIH>yWS9pPT}rz57%f4rEW4Y-cid_qJDp88Zw7iRU!~CdaU+! zn#R8y0E?K_fT&ihvNuBiRprGA|wv45PTHyNDFkCk1q-mc4-Ko};5&}Qo z8{TewSOJ7`cSaoyFS-O=_oXgQPmTqim;b|Jof2dlY&;`iapN8- zzvAQjB!Q4g_H=Vte3Og1c^b(^yu+3Q!td}P?IWA>YG2&vws(mtb=HWYhhX%2@hCC# z%(T0W(Ctst5AfW{HC?p*YwP?VJemv2HyDock$pc?d+<0>5K_OhtN8NeBkX0ozi&N# z$llyD15TCs+FsHm(s^i&`h9?D3CLd#hh8gB8mw0L`tpEk*kO$lXT&{?Q7Hi^tP?Nj za9-aggJpj#wgsz;y&UqKnQEqt5q)B~yt?|%AOPOC$cSUaP~zYj$AlW!H0Q6L)fm&*Q~0W z_jYccV(J%0=hYr+WeH0`#kcHYw|D4?r!mx0J2%1IKqo{}FbcisXAm%5$awOYt8iSD zhBi#VY4tlF#}jF%PlTbsjqI?w1*k(K7ju7tDGJvL{hwbJt!NR#5H ztH6x?Uk^{|4d|#{OP-XytI3}Rh&LSDBkyiQg|DOwX&%g$c-9ts9?n#mJ`;bPb^cT{ zVY}tRBf<{-?$o?t?Ecmi>S$cU&6+W?@J8~v2u8-_m=q%0eB6AmM;|iw4iFUOgB)e8 zKIy2RzR#x?c5AF^6nVf#d z(Aqo|_4tu7P_u9LO!aN<_SN53suuC5Z@*&1M47kHV%7zvlvnS3#G zujzWwRnk`%GC?-zM$bp4yKk`4ZkRcE|HG3CLx$-Kg5N;kw&dLl;uwG&hF!z7Ir^{@ zc4S$;7tE`Za+r5rXjFanfs$VA+k4`xSB$pn;piUD4Kc^jj?fW9@wZk##8(O?FZ*>* zjpQhdPuG-OOWUNIxzye0bV$~mNB5l6Q=8C%#H{GfgQSvY*I~$pI-n=`58Z{87S$_U7ag1BD3qpaK|}>HJ3w&ray5DGA1e zuS~`b!P!>?v@VAHLBmg^Q)^v6SXVOVQD!Zo#adm5X$GljIPihHfzAua576({{L#Ri z37*bdDM2x2cUgW=+5U{4wVw4m%AFCwNoRPzy_qu)IZpjF6w2qu^SNKR{^6a;cQZ`KJl$he1P!#_2qw06t`(TJ+^s?{eR}OJR-M2nXeD%Ib9LAL%TJ5?swFIjWB1k@6^RPW;Dq(t<8LB?jPqG0H17^CzG^add4|j> z_@f9Ii7usT`}+7Pzy7QA^{tMK|5;4} z#-r9%Qk%s~hh?=tnUl{<%udkDxROo+6c4Cod}4y$!B-cHRV8oft|x?wm}-)wV*Bo_?#>cC=L<}9I2(?+JVor|Un z#o$Eq@wQQgmwk>)os51C!+Gg)18L&ujZdWJnI9|SHJ(#0a|cmhm3aKBJu;f1L}W%B zraHLBu9lauogj$+0{@{<+Gw+JG=Fx=;+?;F^g*tJuX3U2os>r8u%va zf7ZIp)8hRgFx7uXoN+Rm zH>(j3N9?n`TUWq;@H@JvE;_SD>K!T^mv0;#saC)J)L0r(LS6Vhse~h-De!$<5%_cD zp21KLQRB6BIBEXZn`C1`=!(y)=f$M4SZrHGjjN!=kQDrf2_bEH-y^@%;iqF5Cj7yx zr$6gb=Ob-n0~wELJ<}SinT6A(CFZfaRIkT;CdLG zxiRYCmcb?m>#^e3jE?XdEXEQ-cS=UxpV9|Bc;aV^Bhfh& zZOIpt!baB)~S}IUsfSSvO z6Eiy-Y$N__0*|?KZuly=Jd&aD!>vrsj!zLeuQeaH#n)dDxOl(u={3Pe89zeg)qSqu z+wWe~LT%kd*WcR^C$|2aRzzRS$M3Uz?amDs^bFGDfHmAyxD6L@`9qVx-!~(3rmKeG z$4)5WaM5(qFBiBttq-2gaUW%tI1nzxwH`?nn!qdvRB=gHtb4YyL7Og<6-`9I+fx~C zy@I!PW+mk`=}=8*a!F^*H+)*%2JtPMNHDo%h#LR$JqcVIR8}0hpE?0ZwT^gGX}V`L zImS~6P~H8yZYl5Am}$!IgtS4aSfDyE5z^2U% zw&B9H5T7L!z81ei3(l5LFiQw=P75ku`C0^w*gLCs`My*_HBV>LCmx!jwnxBvwBdj6 zXbID3I+4kC&y9HctNS+RS%{vMs6QY-2ej&vu=MwmWNfVb|&(7CwvX#)Vw{a*3)QdUpEu_EURP zQFKmymH@x&=>1UQI`v4#IGLZ9sfjxG6mN()DP3gcnCTHLLq8(YKNJ4r*A;+g%s9Fx zm2XbgJYH%z$@=Gl|EEj;{~Y^&|GTYt?yJ4hV2)12pZ)Y#hy7m%8M=RDZ0m711ckEx zQ|o_Tuk@nT(9#4aN8FwN@)v#^atk6s^Bc&Af7^TixjFwebN+V5gAJ(PJEv2A_t)$F zkKg<2q`Cspb>BkMnzC2FkH-JMr>LY)m9oScu^xMsGO35hRejtOdVF^T^XP- z^~h-9T!B_;2_PVz+p|HTF|D+_Z<9&??b~vjznPX}BqXJGWAAaJJe- znLM~b&&>93om$IuF*7qWdw;9;YUz{T%v6PN`3kd`HV?S*fnJSW;b|zmo0jqhuhbuZ zP@4itNFg0HoTT`}+pmuHwm1c>#vZsg|J24S)G9r)H`=JO1Q;;n07f#l=*$v+=8?~) zgGBx@ogZm;e}J7DrT53-B;#!O02q!{tVnQs*87&0hJt7ZgC%-6L0MWIWhc zXJf4Rj(yYVyA*Nv0ceWy-|YT>-UAv$bdb*aOnlxDGSG1Ew+^Wtyks*A(;(9aR~D

}SGCQNc6R8MQi3O^$L zE(Ncq5y~WLR@8XF(#)sQTVekT%KVk!cX03q`^mjNfun#K~>!|^j-1kUi)|Z!#_Mv){J*t!y^8&xi3O~ zY-wKd-}yhG4bP6~vQ-2GAssHlj^6{h3}G{kG8S{PEZ&NOwlfc`HA{z6@^NoQvcl!5 zDVlYGOd31fM`wR^GxDAxWQB_2D0%)w22pIl_CfB$NYgV?WQA);5wPzh`UD7lbuF{2 zt$al6AoT3Ja}?rxVxOeVEVjKv`<6iKL!%ggr(y!&ws=(h+g zDqPE~%`bb=cDV9}qIk{;mxz>+31#4>$YkijSx?+EcMk((A6_5BzF7|}0g7gFK0b~5TE#0)>!v!V zghzfttTwwb-Kwg(9Hjf`_|qf|Z2K#5A|Pili9Mm)7B9!$6n4>eo~DcefG2W}!_qp9 z9_5Rt+jqiWWYx;#hKjmxUFY0uadrpNmKApWy2!G{cNU^{Cm!846Pj~?H<43&@Q&`* zS5Zlk!-tm0ik!OLUI2&?l$7j#dUCki=RBBX!Wmy2VJLLQ(RDZjl;1dfy+gOV08q+Z zY(A|QX32(INb>MPHk+s#DLc7_gr5ad#`1U-i-<7un$_6NseZn3WL4!Qim+FiZ`8^s z93k`4tzT|L@AmOk-sKRHIX$gujD0!;cIq(wu{+cqa`H|e+1>XQ{d@py_=?lqyLwo0r?}Qt5zo!Zhf!&1 z#;f#Sk6-Ki?Cr=#K}F_g_4%ke!x+tvn4QVF!dd#qPYIJT(mxgfyGg1(kKI`AR;$>X zw(mcAcAk&Yl@i_E9r^iLaI67eOJlx+vG~=}#U?|KMdA-1KL{>ecRq?%UQHGb*ag9N zD+UNzW}A~(Q5OpkP;_QokZW9JF_`1i6TN}MAErf*drib%raXT2{rmSEv*M&&Dk^5N zTk{~jTvQLS+E~d4ZUHJcndB=41`RNMj;6bYki(ptj~!ZKKDbDH`}XZ(Bg-~)`!nf1 z!TX3$u}>-b8-3d4+r-9_lJY;Vvv@3=|6qCDdJ4OX0Y|17G~C4>T&5SX$rs*VZJvuE zZxZX(zCcPvzmX228=xZ-iy8*;dP^o8YcG9|q2|JHgK5tn8Vw0ZC>m@3*%yq@t@ zhBo%7zE}rq_Q+&a@lGR;ZezaxkC5GUOOA;A+Q99W26s#xr&h^}!8v%g!URuly+~3X z&C|JP4x_x>Jo)0JYW0ittZjQ!SD7~QYl(~t)q~WkJl}nEK|Gp%!?0*PKL>X*I`bVJ zc*tQPT_}Zu0g8SJD3>3Tt!J9)O@wU6=#G(UZz1O^taRx03UIF_RpBjlJ!g|N)$%Nv zeuj}I6K}r+vVa5Z-e~1riHA=4ezWdk20`evmFrPCk0rg`m$` zxqUKTV$jL_DEzA3#(1&$Mg5ynmwxyV*wxD~rM%K|v8iHb%yGR1#q4#O|A0)y5{~zi zo8&caUm|*atFR9a$Z<`6N=_26BI~g6`sfodK7H+utbDoz zMDdkqM|u)2=I;Ib-M48#gZZ~Y9`cbttsjun2GTZOGtQ>VKVC(R7et5kC%uC{@okzj zhkO|EsrR#)+OzBc(kkSc=tKX?ks@h6=rFJg?G-kSujK4pR(>fb7uT(pcQD`i*d`gN zc|U*1Fb`~Smf$=<4mF;j?psW%M_$a_Y0qSBJH2f1tZ01v;q{3J15NzlI7WV66CRoN z?@}gZ`o@q&)1unJ4_d7?kzhLOyBB*|XH$k@OoGK%uypN7k6-V-u$*oxCfmqc*=}t! zD>9CLp1FUIYvl0i4_}puD@>IMS~*%g$*8g6LkpqTfx2p0nVHKU?Z9sj!qfQ<%@rl) znM7f|IU()uOvDX(jQ|`tO;Nly3njByK5Fw}rG$71qhnN5loGwSr1lcS`VGmPCE{cU zRffDS-r4u(D;Ym9pky+m*_5rksn%uU|1da9O2Gf;0rqzOlDKc7R(hGhujO@ttxO3n zuV)T>_>{7#5i?V#s1oEs8cCd1X<4iaNRYfL(HaEHoQM=Kw_@POX$3(3(caLjJ0^I| z1Po3S2J1=dythe*?$^yzPl9i-ev7eG>+hvn?dSUr@UIUqg@f`Y=$>Z@gPLxos^?cQ zM?MI*hE%@!Ft6usR+G4z1UKW;ak2Fo4p__Mv8pl;c(4hZCCu^G zJ#bQx{-vCU9Du%n-xU&!wzeKkgjRu&fmyO(2cy%xA7QnY;JW)w&J*bBM(O zJ~mO`&!zjz8OjoTvWwia&*2vqP02u}@5NIqMCl`i_2>Z-#H=H%CeJA7IH={4-Gee$R8@7suQQsW`2k zfwJ1B!pbG(j*lRg(515ozY+n9RwvKWk?j~vjRNxEddFSPp+mu5d*xEc-Ra7&b37V# z`2-j15KoT%vmG_%eSb`n0aLU1du`e)gU@WPMQ=87ofL{Sst2Jyt{x&?PIvc_R&#(P zt2lpxRg&VnQ8Vmj5hpm8a^qZ`Llk5TeI8ypXZwPNaZYmuh%bA+QM=nzvV!(T+zdlR z1AKF}a5H}#MtziXcLUjY=OxL0`E+C5RET606FYrfy!J6Eh?2eOY>>}DQt`lJYc1}D zh``R0H9E^e@Iw-xQ%#UwM{H&ImA&TOLfWl9BvI0-uWAC5L{HoHx#$pmlH)A+#tCQ z6F6E3%*UO)`Of3YoM)wPe-3EXMkJ_PbNF6TYobW2iMIH5ci)gvV#edHO9*v>hmiBJ zwmYBAd7vBMP&j=i`1Q9P(H}bB4 zYy~2@2(cJ_l>{;&mtwUf4t8*+5&c)cD;|RRAnd2u65}qlOQ9LT;u1Ad zveGOW3}GT6-rBL>sPxU97LYOJ$huGSRgq}T$4slLTt167e`bCn_kn+Z@i=0x%-Obf zROoWkKi+zA(PQ)$zmd6HCW)QHfGUqNnY|!ZPBlznADq*|Fzqi9y2@`v<`gzuHbJ>A zc0QL&m257w+dF08yYLBdTd!5+FsI(c9fCdY?40wyUOiXaNx4BX1%IodEpyoSxr#Lj z`PYflojID79d!2)1lq*`eYLw63-UYT*~!5Cg9n}GxtZW_?3(iOa?EOo66z`I-rMo0 z5#t`>DxxZ9+XSH325P28!Fth9yP-cil%x-jExeU)bXqW;P`Eiioc)|p%ouEw!e_5MboZz;pQwvv|GKrf z@d*dWOkl5zAAyQUVq(A!*Qhdi#h6%m@cR042(J%qrIRAEY3x+1H|IMXq8G?{81Eu3fhGnrD{YkFiks&`Y-nZk8Jw)+c0O2SZN2RQ;?Nh{FQZ%~`Qgn_O>KU5gsuvd#j`z=j(6eTzAl>yzr*ub&ePl^>+<^5C_tDvI>;7;z; zZ?!f$*fx>Dljh39ZhIFSFPH=eD>7Fr$2xDJL-*V&Q834s2^o0w98_N>!)i#E>@qA0t=+@_!irnnA7^_^9`M2Xu5^q zXY*lx7;4*WrYqJ|oqJM`v|BfwrZbVFk!CBXoo%&HBHl8G>N%5W99=$_X{rC|6a52L z@uGFrl&DC}86Fk+{iC%$8Nz+`xf3tzK#Gt`IRd7F#rvcbkWQOoe@QHTR3m;#+XK;0 z1Oqd{7oi7zUa!ZXsEUHarJ0gXNS3%HY{gXkoXHsWOk0Ar`eWT_F?1S6U2EM*9uW?U5 zdfu79M#*qW=%%E^wK%vpC(U_Utzz~V7f?=g2+}R6iYU|Q-hfGcV9dUTPWL2r31aB7 zDwG&h65VnKZ9#t69Sp9iL{# zLw$B#vGqLH6SErED?wk&z{bhhr(dY_1w0lJ!FCUb%+>ogt@ZRbic;=#VTpg_>ia*g zKaBU^tQ_A8?dCObn*Lq@JP_6{2hzy+UoTy#XDt`vdE#^ul529l*+Wa?GIhl?-mXwY zJn<(Ja2ezGzw#^T@8&?qS^-pIyUE}~oNI#EzB z9N^$NrU=!ftq*7<{;YS{VKq3+hzfFE+VNfI^luKvF5+{$upALwW%k>hETz%JTbmyO z_8d@)^j+&JoFVg~RU5i^w?LbeKqD5NM{(+< z%BBT_$$DPRUX|27*>Win*2sr7Xa`>vNXS2!J%H6We=~mavHJ&A)y{h)&%#J~&05C{ z;Z7niKbhV83`_xguCw==8=m()|3xDN@ghP21IfH^y*?q9@W!DWJ zp5MZ~Rmjwev&9C) zJp<{utmK7c9QxIxvBjzuqCVlH5#>&n;p@ZZS=74q4(0{`G!u`B^D_K8pAd8~Hh10^ zItz6I&DMw!<+$I?eN5mt&kG@ElhEbJvHddGmMY*%SOvHhPc{TPdE)7SCY^D+Gga?w zY|65-zY|$c4kk$zo6mS%nGZ9+ee!AZQ@`J9NRbI@uZ52!-^8O;o>WdU^C^?oXDi7s zAdDlhnXEMbpiaZExq1g_2j!|*#^ZShm{I~v#@_bv96jYb~jVs!W#dD zY-Hz+Zy)YgNUi%E^Ycrdd=8)$q>gtKGERl~zP0>8Vz`}0$0SPcs-_TZfBe3}e@KFC z#x82m0n>DkGY}{${)G-55ud>q7M^J|nkCdb4Z}9m64)r>v=RlsEuIibnjE3Y|BQ`htQ!5Abu6H+QGGTpQ zJ}A;F(a$`IuJpeL5u$4Ge(KiVDiR&CM%4HXuN1cte0*llvzt`h#7W+PIyb|TpBX|b z@YZi$_M-}H*e60rYgr)X;Uw?meC~+dbZgoAAQ3JN2&IU(v#Q+2Y1W86(nsK4c z?6*e*q8pwPzj*P2wt{(3Om&h+)inQQ{BlEa z+-sJz_z_w$mt1DGywoKK6Iipc2JWF^vNw4R9hDH?#_fbZ^(MPCp7lMXt!7m90GjU_ z3|Jwo4fbx(G1V}JX3yJ^iN5RIUtW5G_ev&AX+eup4)GTA1BRG~R!9Qa-NjUD<5TUP zDbl>p`IpeAB@{}kw(2$3M(|Q7)>d}Ve&fv`YC|`S&#m2RWj864dQt`Au~ne{5Jo)* z9afYGC!?9OV-gO5ocBDqiw{1Tt~&Zi=6Wzj%3_k$14;h+Q&sk5KH&W9$A5Lk4OmfW zdfQf9L;M}cIo)bpq`N-VoaWQtvtV`)lff4Yz>IPlD24(SLEBrUneZa(<$_roYx1^Z zbYgPyT3f2^Fi&k!B09XaNLM#6){@g&3tEwdZ=}eZkBGyrqQW-;pab!=XX$11P!lMw?;ApirPBLcCPjZt_xI(Q)>yTyIlPq z9sbvvJ)+s1>a+Y1N7+m@=4~O&-1rel^sn9GbNO-P13!MKuv~tP<2W6?B7(!r{mZtF zlk9PUdg(Yh)>STf$QNQ(VsOPnqoH$g);s4*kFxdC0lpNK!TO(fx8`mSQonrp(&<`y z;JsdjI47_(7z#d_SX@dd0~Ormac}q*&R142rithzqX|%_OL8Fk$I3;F8Ly7_u^JEgbVK+l+SEwdw01I zyolDSN4H7HpW<1!%u6OB@cQj<=lb6>jVao5<7Z{8)j=ArWD|b_Ns`c-kB_oI8~cTg z?EH2N2K4H_&&p12<8YaBB$2Y8t^F6{KzQV5VygM_A>&1J7`+k_WJbfBX=3z zCrI+i8h=$ltAK>d{-=OAmf+zQPB*I>PGmQs*&1}KueGV$H%qSz;?;khEOU=(D_Hfu z&l(ygg1mdNg}f@oOBTZxgX95IO3kzEXvLn&hA=!MJ!lw_87*pk`nRXYEmQdXSkKR= zS-}&EM#xC6U_Z%_>T8nPTn6a=q41Z}!hyKm!)N%vuLb|m(Q1&8?e2=1$BxPF4ZZvC z<-?!K?cWZBm7W6i^SrE{y1(|CfF`&|PW5@V1gI3?uhvyM+wMndaQyNzh6ff@9DpkbtTkjrQaFxzrBrr z{Ggh5X(I|+L4YD@u?;2vuYdWENB`$f#a~jN5=Jxt5&UmY0bT~68XRyp5P$pdziHF| zy!~4cn*4d8?ze)lzph^Ye9YgUZovh-sp=%|V#)s+qkl&q{#?{4z5m>5iHBrh|K~0L z9|u=+_k=I=|9ceu{;6#{0j1^qO7d%|q9j$YbySkF)yc zwL1i1vJ_1PACnorp5m3i`x&nG6(30iUBhQ$#C+* zpQldf9cgF& z!QNPPp}r9pm)_-bum{KgHmsX*@1r&V`6n^$tIvOY$n@vgxJ1CeI?~IndtM*9gC4;0 zl-?DA8@9g~CdJI}trS)XbOT5{I?k4Pii^uo z4vA-sQ)0jCySuV2QP`_nVN|dDT2S!1)+xQapNuO%ua4@Bu&5VNo;MxnvJEQ+-Cg{& zWKdL7|41)Xpg~8%Y0>O)f5`vP77*x%)-07oHWrwxw}}hlemhN{k@t)qoLfEsCaWQu z&u4#LYUFo;iaAF#f#)%a^jqmzZ#JWo;JdUnSej>~?L(s%->rk72g|{oNHx<7-HayM z`D2O~a}*w+kJ$kW8ToI(yFF1ks|V>~U!$UMFM4oNOFi!dZ3}U^wlGz-)Fc9ZIxKO# z>aE>o2c0(`*4GOJ<1-%FsMoT7k9*fQ5vx`0VgAZ-N#k0#Cl2yOvp!wFPEDy}4JmXf zPhBWK!`*B9WuxkAp_>zxAJs1Z$pVPF&PcntI@^U=TA6f5=!pI5X@RcDzsM3;c%BUO zvJh)Nv)pI_`u53^yPPl0I7@^7F30XM0H&W1@}@JTxMRaVNBth>gB7~>5s8&1y@27E z`dyQTdeMbmXIPAE3SZAT3T?OdMOw7$zK3&UOo|etC0+)|!y>?3wMpc_wp>|DZGOl7 z@H&smiYqL9h6k%bz9{H4TC*-Il6>8@c2-L(n_ji{J~DBeQl&>-v62$d!2rK^J0|~n zyv6oR2;w2FUTL-_guf^4;!^ho|6{pCp_RDh))n?Qy|K2NlP$LD6W)T~F*o45bOdI` z{WF~At3NAlxf95X>2`=-r3;U(GrZ}GU9JVGt}wC}jEnx;9}L_5?6)!4bA-oW5I!Vke=tmy7*L)xR;VCZ zq}5<(^X}hF$F)6o8YL+rtBX7)diB{4rTW&;ciranin<6MNtCijUItiIP`nTP>VaD$bbF%bFu>Kroy&M z_Tm!$g4K1<%y_sYopiDZg6|GqA15fBpTUw#d>%@+hEs$G@!wcm>j|K6qXb_KaWc`; zN~AjPsU6#_vF@%6G(EQIrw-V@zBo4c*)IJwhg1g@^L9j%0?n5uM-9J4mHKRbkxJ~l zhtK7D1^Nd1!_*+Jqflys?MdsE5S2@yVdPDm*T$tSP$cw1LeU>FN+U50=Rb;Q@8o1% z!E&z$oHQ5W4O2RVXM`m1iGmnR5{tm;mCo?E_X9|mokF+9qRA*E8n@_hpa&~}0x6oE z!y$LeSfRYXkwDMHbIqKF&8KTVOca1=0P|#&bhseBOd>mr9&YWYvp*o97pUb=%H3n) zqhIdw*#TjR)zP@0-c#L-E80%Wrm`xD1wuGi%7M@TR+Iiu#zj0DfVU#-;^N}ybjC7C z!K{IizVJfalraC9*NwygnDmk38>Y7U=ui+OwV~T(rM;-dMPc9m(v97i{h%?!N;}Xt{cq zzIT^isUc*1gHAHKp>6Rud-awB$&yn*5k`80Noy5w|HQBnr?gB16_-r77~=(u#8UB7 zIOB?2yLhIIdatDncET*s5U8OcaJ0}}eomv0~{h_sua;B$mB14;GNW@?YUs7IIC7>a zjjMKi(5|Q3`g`N=t<&F^bpiKJI8M;CzF}e=^t!B7TMzG%Tq^$$46GkDTDorpldM+P;5DTd&V~@F5;MRmRjI>S6vbc&Jdr(UYZu0LG>p`}kw|M*j%w0OmpiuV7JL z$uv+-dY>dX=lSH)BzG|87dBLy{~v6K7iPG9oktUl4+1jqq>t_8aGVy?wHECg51?Cg zu1wp0+@n?Q8d0Xi$K@hzj|?*(tTVk9ka?(M4uVoU?$0x-C^o}?IDZE^dFsw%anX$w z1=uS*-0qm=@jNXcVhe}Y-AjiVEAa>>BfRO_*GbLq>~CP=UBv55{qL?WPq*@^%_PeO zcV|n*!pQmP5%#m+im4XcW{*o`Q>L@ij0RjJ)BzQAV-SB+BJe5AH=*-IpY>$zre^kh zh1jr2%~8wEQ8X-mN`P%;%XtHzY$p`8P|ndY?{sB$bWL76AWcFZ1_4$|^P)klK>b|U zbmDk^mhyo5Q#>VQx-eHYu3--|i_e>`q8m$L<}+DuxQ4YcTeE%1_#7wSMZ2~?^0Va? zv~gJLKAc^l52EeUCu^HvMqvZ;lp0AaI?oy#uCkOv6Mt5F3LW!!+r(Zel&Ah2N_)-& zR0xxKQb32isYW_0L4!QiwQ7DOqcImy)Cvz5wX~fZUlPfsKYrg55Ir`zsg1j-65EG2 zK9~|E=$Az55uTmNa0BRavNk+9jwRr@P=~Raq-!{pZ8}VMZrHo=tZaVT%%kN=J$5rmFX!Wo54tvAUF2U}^|i3Uf%23=q04RS;)^fX zapvO+Hh?#4oul#10)Q~QzVk>Z?7c%3!GO@;tqpm`_I)>i5DtBy!hm^oDU}9&mxa$1 z7awZ&5AiYHDa+LfQ**S_I0ZEp?zgQxBUeJ(c6U4!jfCdSOEOn%oq1mS9Ez%!>iX@d zh#dd&-fesjiIX?QiySFBn|D`9RkV~vP2AtM9S;lfGxZe1tG9Xr{Rwv6OJA5#uul++Z%ITzC#033#2oh>(gUcyv>x?u4)nk9|?7jV`{eMA5 zauc@t#s)cou>C_pGt7wqK2AC)!Z-CmC^mSbT_V>6G1wP)-Oaezkb%2WR{j zd2*KI_z08uxw~CzC?U^VLFHm|r958A!iYi(D+mp6%53JGL_)pG2Y9avs7ImV}yt{IN)81zWG8;WF zVL#^GUGg0-v1@G*LABAY`NTvShjj#)#gO?BGa&NQszY3wqAq=T*EhGti@Y$B~tLH^W zTorG5_QZ#D9*Ezh@F{@nEZ79>Caxtf5EgffT-}@*M7~x3tB_BdOg53Fu8uDVz@$$^{PS93Rq07U~m^$eI*XC zOxi{5X|$EH+{3pha=V=}Vg+jOM9Z>X8j^iUDK9O@>pJzJT-R9-Yn_#yuK15lf2bk!KS9PlTK`(N$xPy~6 z7&#Q#Fw!mNmgAh;&d|xjIs2KROgay@t&NH!QzI>qNu7SGwcQvP7?@7ZNwXUCKD#-t z^v9Xmx!RKi?OJ}Prr0?jnYGy!Cp0d8bRXv|=*sE6n8XUXC{g?KXvgEy;wUA>YW$)* zvs$Zg)EhHDS51^9sQITwtBqy%6C|i96|33^L07ejP}?$YFH~*-y{j1FIj;|HF(rM3 z^Foki#CO@h%e2fNDZqT9WNf0O_Wn`})pON-`>WJA$&E>yxP^T3z?wT{#=c7d0f9Lu zlI3{B8~+2<#p41wV6CNOw$RuH!RsX#aA}8ja#uc*+Xmr#en^MF;g8T5E=6DaZa_Is zkVN^15;^jBF2fEj`n1J-@!YL3bVhZgdq;>lb(+P9(;d2gsBlgb?jAqD=dgLf7UU~pS}~0#CygfJL5Zfe!o%qq=P3=^j`32F0zur;h}-8N#8`-twh5^;wMB& zimhrj_H7oFb?b|Mydqd5WTHf(#X<+oK8Be7lCS#*CW9KCKzAp`Q5pAsJZc`FzP-7+ z&H#n&&PD0XnTD)Irml6U!(0ct9@oJX)h!uuZ+2V`=zBowrm-OVDTl8u4 zj)M*y?-&$u!}I%(wc(mIEE{X;kp$c-!4WE>a9Fpqrr8wM*raON*h-UU(_KUD;*|0Zwa_RA+J*b&6vj{PJ8>Dlrt-xm4x?>EXN6{6tWOYepN{@`E& z#tFA7y%#cmZ?1)F)En&*AX&Dl!Bs#bE>#xwAqg=wyL8A1NeIjS-j z@?fV7v1#{~Y7>`f?kSbgMRSYKYVAg9RKwNj+xK=M2?5%ThU44XaV%Zt-D(<}-j9cv zeW7P21RNDy{HijkhQ)M#uL5}dYIV4}eP4n1cCFx@fXXW~hgwQTM)o3Syp2nMaiG5o z;F)9c`x=F_ym>%S`KrC`^ETB}mnPKA(A+1XnoHB>b58AbT*3`%OXI|Yzvf6cFmhdX z9OB1hToL6gcV54yV>KF2%+gv~WGd|nM+Ntr$rtY|oUT+lG#wzLH@uU)1Qz|ZaI_QN zuL#cJ3;usp80Ay%ai00lPir*tLw+xb&PqHu81@c~jEMf!u3p{j3v#2<&A@nrTI{ek ztrX?MB7b4K9f~UE9xMb=Oz-Ydpkp8{hj@#bRPd&9ysLOVhs-^!%1iW8wa~+ z@)dvJ`X+;Kf`9Wv7Q!W7mDi9ZCUr(|8tgj1L_%<1e&!`=2zD-|Ff!{b%~4^Me7H69 z-dJtaEC1o;l8=~e=++Nc2rvpd=%URZ2~$u*8T*5XtKMU#1=PFmB3J+;%3-3bFxS&& z&MRo_6&C5YT?X_UsKL90xbI|bjOy%hDSVC>ATQvxMAM`;J!zV8opy>QGz+@Jug_Q5 z);KVY!Ui5Nqi)@~tYF%P|FX~&5zljL5bY`)K3&$NWOLRVKfZh0gXiVQYBL~u;J#a_ zas!@gx@)7j&hbx*ic>ld*jW6-42W;_1mS!q1lDHytWxEY!0o%RKnE>W`#=&Cmk+Ujdj2YTaci4)10RvHBsC$jPD8 zkE2ew4Dwwtf(w$laZ0*5qcB0UNbVkHfO*w;5YzSta&~Ue%J!tFBS`Xw@PbiT|aVt z#^36(^`PnCP#!5oRmv%jE+;2pvnlx*X_0$3Rj^w-sh-jxm556mZMo>F~@QyVcN zIYBe;f>}MPxy^Y;-#XSNh7X2@O#kO4_+Uo#QW2 zZUCC(xhug2t@^*(jpAS&cd3&=!ql#&1tr8ePFxRR3PW~*nV-rrn?h|ciPkN1Z6w{ zR|26!(KjY|0?Ev#zmdIC2uyny!{Iv5GE=TCmxSbgo=F3ygUsLEm|j=mfE#$h(|wSS z_CKb?ntW8Z%q7SM^dO^W7q)t_!M?vCGJ1=Fx1h_R$qHp^y0}aV+%q2UjX4@_m(&M* zmq$_S7Xq%Q1r>@*xkW+fsF09@616Ec)~gNzeak^|%n`Tu2jY%p>$h(EDUS4-rig8h zpWjVYwz5;D?BQcEZ2?$6O5!NDhzgNnB_CoZqYtHU>29JEzqJc+>B0Z88JU;*DNeEf zV!IFYFx_*Bi0^IN5_tH`#cuxtRg0 zj|id}+8G0x{ODJ2&PT&<-O~4tEjHl7uc^-MwkDKPT~B$`nsgdmFeYL>ZmxjLWJuNP zPsiA;co}lQPz~X(npxtxYA8$ogK_HgMKFFCsWciQ&ra@1%2VGk9>57IB_p7q16V8* z8<93!LQxF^j0~oBt-uePmW1t))k?n{UQtU84FE8+(A_k&SqLNIn?Nm3*kygVvgZ9b z8(RZ9oiQXt5V9XzsGjHs1!RUOG(?Kyc60&~HorCF{|gzY}J1l~oiTJt5Yg z*C@0lfT7I9GHN97W@co5uP=V0AyUIr>(*g|+v}d}PrpA|ni&+r^HZu#TdX4ass+VA zw6IT5dF@;{MU9LPwY~Mc#_3nK(bh~1gK}b@pJ^K2bE?w;ms2&{C(y3o;2`|~i#pY6 z2dx;_e%&3NPzJ`@0urReA307&Y~f9ZescpoQ|5bsUN$IYR!BZ>pXD>J_uPMWXA+rN zR_v1VXXWcWooH~m>-VmPH=xE;gGWk=O(c=7={2Ec_&cwInH)Rg(AhH05loQrP{P;U!O)>m$4m+ zYP8g8XCMTcg(1oWYlqNIiFIy#2JHK@$d4_DH%36uTk&Eu$yy~B;5bu>Q_v-wPKkZ21v#j4bR zNRWJmJ;I(+jIh@)?KS(7TBPGM1Y&DOWvJL72%_Re6)}*)B{%qKQUlfGAh08D@xY>r zTz3JaWrCc`q*d2_50B!PnZr~G!+N@Rtq&7rY$OK)IgX7x9ZKSY1vV9qub8!WM|NI0 z1M*h_=)3vN2jWDcbWI%UadT}oNCRwmk7cFnZ8*7r;+xl;YYyuBww z->7_KY9?cdQ>R^?*TGARS^Nv+>#gix-8@-#r3KgSfXzWJrcP^B1<}; z^_%3k>3>NhuT#(n^cA_aaa!JExXKm?11w3mR}%U@lM~!8Is(1pgvPf4&=PJ6G6+?uS~;7E2Yfm5%LkV_}oZh`}CMXzxk zES`0L(i$;4pF#4X#87*{dlMbYlN8->?{r5#{bWN%D5=tMvqUFbZg8tJc~`?SWayiH zOw$Wbp=sBfqhL_~kFGFswq%^Jn>_zmf0dhlx>V+i;yL@`g_4$n;>mNLQ_9NbWpRU7 z_lhnBRd#qbRZ6;QBdI)I_juc+c8SypI|tk* z|Jd2^GjDP~sSvCcRM@S>>x{Ixhsp_C>Ig0UDAc@xzTLK=Md`zJ0o3CAY4Occ(YRFP zyMW4N#jN0uOcFxCLs%bUh^5HiO7^W3;1ffQhv9EkDN8@zDUX2yqe5OP zSe?GF1#S3f5l|b4_=h8&){Rblcy}zD{b_#w-HR6Gcb0VUWv)Z&qqQw+Z4QkMD--Frth*F9z zWN3PL2>RPU^s!1#PR>|X_d1*`lwZWB5bjp*j~uP*X|<5b4SMHo?dImK`MU9C_u6iS z(CuNkVL?-#vlmNR#&{+dkS!xD_@&NzadygfdGKzx00azGibdjs>*@>6l#Ti{mGN`t zr4;)@fK)6HdQ)0Rnu|Y`QE@A9hl=e&8Cz zq}UACJ|=S;2GMco*WhWfY&VahGHVKczM1;IYNx6Epj z0s1+6*u8`qIUKaU$J^HMgd>~b#u;ZZV`(DiUXCL7Gm|2fmu^ug4+ijzVhC02?PYud zEeh#7^9vciO1e43ZgblWMru^fOZ;St{XM6F(yKm2`0Fi%wPdDq&cnz|=r^e;6Zwb( z*+NNOxCI6l_N$R9Mp^0(cG3rVNdl9r;A2D1Gn0}tYtm(@hhFVsS4$tvZKfXOx_xuW z#LuO~2+z~@2Mq&(H;+&ld{$0_Pk~gWgyA%9l#9>+3?dU*hB5hx@ec1Y49YMRQ#Q_* zKv%OVCq$7FRWfgQ!t?1<&FDXu(XujIVU|~w1&nn}OI>CDkvuuq z{g%E4*B^aaKsH`3%dWv|6wdd+R8#JXOq#wa!d z#yR5|&QYiE!Qt6vS+V&^Il+r|=glIIFzgPwA(!dE#_qLRgTBz=UPnY|>EXEly*DEb zuBTE-{qLjlC#RTwds6R~0tUt0ijc7-+^1LAvb3&F*SpZxKCiFNf73}Z{>S_Q{OKrk z-`}P2>MFwVPmBpTumW0uhQZdi4G1!I6hP0kA^=aA^HD_oJBy)(eh;L}zH}s%7hGI0 z&k#8%jxga$-}mpS{5=ihPW!Qp8D!q z0yVVb_jCCBwD-^BZ@f5C%1hOT_9oafzk{~Ff7<^(#^wQ7yhyecqW?Z&|MxBOg{T1T zr##ov`Ts^M{!SYTuL7``#9)Zg|NMkXYT>17b$b*NvA=|+|NGti=L1CoWE}|)Tk3!F z_5XRqKb}7x^hfeeo-(7Z-tVU}FGLvl>hv9ZzpiwJ_f?w0@(RGp&+dB>Qv;It*N6y8 zm11xVkO(em#Tq<6gbeLpNaQf>{p^L51#YObS?^CU31^_sOF!9~kyQU)_0`+E{G+N@}W8*KG0@u|~#g4)-J=;uwhN5QXeEPmuDpKgyUsBaMzBzex+ zD0u7rojgO5gGr?SBL#=)+L`|A#IPr<7=i_v5JL;pm2IgnJ6l@%ct7~PleTxo{;-J2 z*f7cQ6hQK<0whh$?o9QJSoJCkEv>R)6Z6c;736jy&9aB1Rmo((KJSuk7GcqG<#Ip)GN%W@>L4IB7_RLq-M{o=(;54To74CLtt-*Ux>_} zmZS1H5x-N3Ce-NzkdEoV;6HTp!3eK-<_x7G$Wbj{nX#OwyKxk7r*4guOLjYdjIni| zJXl(Hd3a&8I>5D1F_3%IRj_0B#eA&W%%9ioJaG2w(Aqsc*5xKii{(M zn>T_$BDb*RA>m&a2wWBPSjqP}!Pfflq*63-8j&rG5pL zP13TmvY#D&D5lFD!+G|Xzb`NPk?x0(u_=SE!xy;26mMU(7cJZ@o&8MLL-BkO(}3mq z5QpE3PA4FYEcBqxafp2C8@1+Msb+09ST;0?{mZxPUOdJIP4x+*dTMxiuNLEh+K|v) z@~>@;*8=oi*u42!2>`#1?$eF+EhJu%@2yV0o`#=_vt9vispJ`n{yzhxZxZq^MwVSlm`)K7t^~d}M z7I7jc9dcP!t9`}Pyf2iRuKl|#l~e)f?69IxEi<;Mu6I75$s0BGWZWvY?#X}W0GaRv(@n3ybS$l6IZbdOeFMSahzPz2n01iDou}4zJ>jS?6X}s$MZ|u_7p9xZF z)j7lfF}#UMp2rZ&UcXjR>38pbHS7a&kMtC70m`ooz|eq5fJ}ZMFLwG=RpEARp@EvG z-l6B~3rMi}u~)+tqVek8k{>#&<(?+)&nn~5?2kZ#6U{-Z>8Bt5@)Ck8OEeNvKnFe< zt(DHu7#zUzCj@p^O7*zhvN`j5fM7q_$&#dqq22Ziu6xUYvOk|JP_~+1d`*=jL-O{m zez*%i78&n*VhKqi37&deZ;w(}K$agH7yJ?y^yJlbr>V!tpXo^hSN~2=nyNF;H}?E? zH3VxkUr29otI|#<<#n5kZUdF$6)2+J0-t5|!>cTF;jMUPVPHDsXs*lyOWTTQ@;p@HE7Zg`r%dY!S zfP4yx&K^kW^_CNi)RPSF0m{(ql7?KigFyUL|D-CgJBa=SY5h<3RJV0#Mk_VYSSJSU zie#I-6_H?G?QRA@$`}-!vmqp!D7HjYGb+a1h9p6=eb?gd{|aPEJSz_iZ<`p<*f9XO zra#C)m$foHn~UDH&o{$WTCUdNm^;;k0xlI5Dj<92&e{GxzVELIq^zwIx*B25*g^SC zr6Q>zq#Peh=G|F68@I%swUzpwBVU}agb%1`pQkn56k5=87p8A)Pzge-tKiccV9D8m zPkGKtbaDxK?sjUwh{qoq)kPu1Y|{nSPE2*2y0;BCr!(tg3)kyhWfr@<##4DQ`oIo- zy3(}o`|`V^@i<5?Ab7tg3W);>(r-02p=xJu-@RR1Y(YA@DKp?*XFA`1HAyJxm6w>b zu6v1Y3Z!1~KQAqtCZiezqAV8Oi}koP1#&MbgzU+r>>MUDxxe!VRGbGs!`o{V(2HUz<- z4;Az{Gm0V#sGTLkKut$M&rIeuLIg}hE5cL9hiDH~CiRO%D9N#PC2nFK9dD^{ptm z+kwcJuuzoG-~K7{S#w3hUV(KmC1 z1e=y$G;fmHZ_ZGQMt>+&WtS3e-}Tp9HOenWhBVP=pL(prU3`bOgx$3tOPn)saw?~$ za=g3SqmSic$Mcs7_}rw$Yrpp#;(q9ND-*H!<@+wd89(M3PfRTS`96zVsvZz9=MX|U z1if&Tyo+yNaHMd*j$BO}3_q|?`yKCHJbhZj1bAp&zZ{$VwM4>ZxxWg3Q&i)Ab(*(m zCE*9vHJ;D#yJ($M_6H>&8kn1DCSQC!Z*D|-S~ws3rAN9d(Ak$cIw9c|jiOYo%UaOl zJ6X3MXYr3a{~e^1-yKKCVKRY@3IW2DzIhLiN%^6xk_`-ffz=qkLjXChRl-faOf2?G zec;J3fYkk3g}1g1xH1_^sQV9lqUeA)ATfH;xJ?46%yWABw?J^AN#k`p!8fB5L33H{ zidq>sCo5o1qrq4S{z%xqZfi7@C!DNzl$4$Iyl?e~Lcc3F3dEs}DOWxu*)JF7PJvBl zIFyjdpY$&M@hEDKo052Km>QYE^vTw`+qI~>IonJxazi!mG(qf#tFYaNRJ=%25^1ot zv8_@4pIMSh_bgP?3EyPoq@@~&n&NN8rGT`tK02@p3^X_^%L?Mgs%o&O@D2CHI=>477c8e?%S$!on$r`v&uYoQxEq-Eqd&fIEGf-pz~``wJgZLxLh6~%>4j)4Z#*W*T2J?zUpjSa2?*^f}? z^%S)wAP?(NK>Os0Z`aGP8;D z51vatkvAo5n@sj9#UUdhJ0YLdL)ws=;P_@sU>^QBDINv5e#Fr7K1%b{3B$QB-wEeO z?Z89$vYy~gFTAQZTY1!p!5K-Z!s9DJ2)$TJak{U)hh&_nl9BIHu&k=x5+dz*XNo4#(8Ijsq#EVqK{&uRx4MBz`v%5YY?}*6Xe7d- z1>G$QUe#2m^M=jM}!vmyfD>zp5+eUKzlv*sUjAvSwocSkP5 zSGQf~jUZ1mKkx4saOl4dJ*=@XdH0rH!*u6n(Uw=00=X{7s%sGLVzb!nVH>_e7sAz% zw7DQUXJ4c>hPZz*y<6E!R{Uy>WkQho!trG?A@LV z{JBIup$Z`EO1az zX)0F*X;K8~SO68JsPqyPPS}=xLigd;^Cy(k&!W$LSARWz1b2{UVlgKqNGH$ zGn!C*w5c8W-hTCokSkXdStnSQUFg!7Ev?J(VVnwN?GDq?ew!9s#dyaXP~lu=hSMThu^lA`nA!e6PS`jos;h5@gPcJmlkcLk=sO4;QEO1Qaz@L|Efea zbMS6hmF~R}p&-(D&YndN5=OfoE4>_U;(M)}0ZF-JL?Q8tsB+h^O~*%@9emshJ99=P zE#Z3jnO8NnvLnqGTld4XwDU4oxFSC3L(8r+IXn?sF(_MGpCK}(I>e?iwfa3jZS~0m{((k61kVSx?8hG{L}GOSvs%g zJExJD*ktVBmt`&)d0A3DOwwey@91fKB#SUqgW({3C(W)oXbR!J$yAwR^g0sj}&p0MZK*~fl_m+cW!nXz#JoIm;8r3 zO>tgJ++4hJtV0jOVyke@lD%l_S!WUQK;tw5!#F09g zC9pCBalryW4zLb;yoy2BpAq%5B+16yj*b=Q!L!17N2&k$W2cW z+WMXQ+ov(W%NJFwz3Uwk=uroAgGJ;Na_81vRImWmiY%zStu*9_LvHSF#A8==r@7 zt;uS9v23xTb|qG7SfNLC&!1%Xy(n_Ga2GBGGVF|ZdhUnWdV}_iiNeJD{+d4d_T4pS zwM)B<4lf9V`SB+)tI6=jcr11+ZsUpND*1GBv`%&)kWD^Q=k8>i4_jT8u##@A`S2+R z_hns_%krofy^j2H8*|M11kYz(!B+Rx@^w;*(5OOB84376Py6xSWfQJd7mZzFp}p?8 zfo}(vE=b7?-MdfDIoG2UW0vJdx2$bN;c75@G3+t*KjO?T)! zvfm4_5~wb6MFbapbj~{mtu=W%FW7TZQh#bX5MrH@HIS_up|L*XC-hcqoUN`hY%+ed4>P{0sEsFV|hy13FDUk2VI=7jwjjE)B8!&UJrQ8a=vFs|Tri>&nJt9mv4+ zdSq%=vW~y%Xzc@KT*In_NS^PFk!6~Inkj)^lOTDm0gpQ5uoQYEh-44r?eO86q^DK1 z=p5^aBAtgA)=djFL{Kz3&R+efq&3P};%s_MdDn^)vy_~6!GL&NousJbm?Qz0jYI|A zL08Omg`CGwPzxb(LuyaVp0ayWXBtddi$M>2=#78(L&F8dRtD^`IfMa@%Ng`254uib9o*#>xDH- z==ooaexZ?;AVA5mvDxqJxr-Bc(Q~3}I8Y^Oaft8*RV3z+C+*qGG!v9|Waz@X8iPQF znn-b6Y}i%N++JHn7TqN=+~(nt_p zW0bS7A7ZL}S0j&SzSEGeZaK8Bv_leN7Gh>?GdoJi?=*ER%EPZAW;(vI6X`9}&+y{Qk5>eXYmFqR7{#qe8YdyJ#UziJe1p{oC z=Oj>jpm}RaPW~4|ZsIc^ea?aw9`ky&ij$$6*05-jNfu_c(Wn(BaqF4hDk#vZ7$aBb zAP3y6gjolr?K|vEui3>!(VaQ)M9bc}vG-$Dp=0qpK%!7LMx+p&kSs zwWGzlS+XtMuCKMJggdw3IdtC$D>3z1@9PPvZtY)nzJ)D12XCCR&s}#anGN4JT=*=Z zNOq+wVWKNUIai=AJ{YgH8zsXT=(@(lWPLOo?{u%~H0x}DET zLr_94?D3efM_x^N;PO?IdhT>7IbZ-$ui$UjP%7sOOn57!&Z!*oXQ2IrNwq)km}w9e zQy(#F7XQW|Go=?5cm=n(vO-D|^4ec4?F`S20Wy*k_ssFps?zIuV#C>oyYJeOr}|%w zU#KUtXQa%YW|^$W+R4Dmk93GmV7e2?z^zw(a?Bl#3zYVIJ|nDcy>@9gHa)Xg1d>Qx z9W7wqKO`!d(=!b1X7b_s?Ba1nPT9owsq+Y_4_dOjydV}Aus095e=tlVu=coc*m|iu zld`kG!v>AdL-~!hhcfCG7hHRa-L-@2{6zGy=>cA=A0=Dd5Xz9M_&x}nyyuB8Pk%nE zeHzgPqgdd)P$Z$-(-1mZppQwBU5}ZZAS?$+4Cq`t$Vk!h^NRbZ=I&I*-Z2_Y7)AAe z3oP_+J!ojqR+#W}$Nz~v3Kxyz%1DCvYStFB!PMD1sQE_keX5;?&NkdS*@}%kF#{oc zyGo~e?#o|2QmtmXt`sr4TT;|zZnipEIaxO618Y}52fLy^&j*&C0(j-eM(z`w^E66YRagLILshF?4()z;qZV*QRtLqFpW+Gsb8geixRvqJiZ zt*7wo((+9JY>mtuR7nc$bxlWVD9P%XCQ2X$kmEItxCsMmBWy;oICK~h{qzU|v-EVZ z73f1f$DRu^o{?^OdMeIB1Nf3Wk)O>W8Fa`_9z>QWW6Fw%DG2jAW!G|bJ@lsU4)QA1< zgF7s)#fHh~sbO8b&=Rbpe(uMY*tXY>9mQ280NpV`jJ!bqB(Z{g=!Fr1U#JV>wcjl_ z=5vg0pWS=4`dQ96TrdI-5p37L>qfd;y3&G139^~R0=M{#xxat^aR_?nmi4*OkejWY zsKxw);+jSd2Uz7Dh*1Aj{ubb~tV9FIZMqmW|4*#Um-tkeo#pyR8=jk~v<}JyxoT%f z#HAQ!bgmfGXS3N^LjobLBhs^Iu(XMiK^e7RLy; zoV}X3w~-V0JVWldpBhEKT%9g~1aJ2=MY&9;bQ*T{e{l`?2i+Bn_cDA$N&*(`!Z;QC z&yVj!fQ-S~)O6@lL8DoiK-}3s4+~?x#=R|ko^nJe31Oc%3VRLglgzbVF$p>tF2 z3XwJHwtMS~<>ou6{dt{G2 z&f#8T%cNBaHnwD5=Cv|*D!0|oYa;*6x4Qr|p@itH-1O;7zz0Ebz{_HWrbm&2^KSdd zHqmvQzz}1O@F{zo$msCP0LZdipWyTY#LJs_L=AnAz-s>#YYs4e(?qVNYL~YCDH_ym zcGhc`&?SEu*hVxYT!*{pWn1?JO%;xHXmYJc-39u~ybnD!-W1m-V#M!dhw3J0qX$a8 zxQ%ZtShy$yO*1Z=uo$Va1%6J4+4OX|JVC8cam?vBt_FdzSbuS=F|M4nCp2Ro~?Y3x1><661(Z2j7jnZg%2#g=ya@au6xvN-l5Y5la#_D0+7V(lZW*n$i z2&6Hq+&Z*A?#q?T3OkGmhs)%giqeNvr5gy5IWP5^k36<8!am{~wSta{2U_T%!g)RA zWk|JHbVb-B+UL~&v>?pugGiUfV)bM2dUpr@+uZBE-z3I%T-Elbi+}Ojdzsd zBMZ48?_!+?;H`NRt*w1w*hJx|zy{d3Di11NU%;=J(7?z0tEm8B(E)M~$lrKkL* z-Tj8n{b$#kn#d`xVRijM&pS#~d(IlpWHYo5w{q2)JW~l6GdUcUi+PQ9?)SJC{KcvH zSW&$bSw6w0%0{2|L!?BDBiD-fUAujX3TC(rB=6W#|Ix6%0co^Els?dMP#gKPrgs!J z?9H`i47^tl_ucdMP&!^u+rwPInf-(@H(aLH$j8pKE3hF>K%+&x>P%z6K*2!a;$3|q z8PWv}r*netygNuu*r}DaE2hf?Vajo5E`D)n|9w znU`4_df1XWm)oLtj00Xw?ozG)#IDISB)T@WCUIJ?6q!(u&t1Nd%myueMbEP+W;E;C z(KFT{+Q7}*a+^gQ@r#)w(ag;mksiw16~^1HSy_GEDu?(3e#(zYoEX8 zM>w2jWn3~`t*&AEC*eDLuqsOk#I0%=vNpu(DmXmsTdq9~$il78o_` zHhh631GZ7RL{6=CT`7L5mdh|gRf;%reDJWDtcg4f>tIT%EUlvEtNWxE-W?xgfq5wD z=eS%E9wk)--TGvVP= zdZt9B(7yO2J6RmiEEAeze;fSh(D?;Jr~ag+r=w$I)-C2t4_p?4mHo;WKf4ZeoEs>~ z{gT-4WEI^BG!?^ec#C>p4zYFLmUKEUx64u^?=Je?8*l3|Uy1Lqzp3a6MHHl3a@ z@Sk3&%@@6BuPYI!ydUd#bco$OhqwHQZV}T`fUz44zFIwyo&T;_O!^X|L=`Tr%X#0N zq6K#8Da-tGWn(;f6_3;5r35y`)eJ}v)o}Y^??@xDFtkh~+ngkQ$zIK`W>t35b~RFSgFk8b7K?BucPDE7gFrb1%$@ovmJ>={ZYae{r91b zS)&~jGKF7z6b0}e)wMF0qYob~(?s%2%lyj=bxC|`TqD6Rr>S@Po#Dy{EJc*kKS9oQ zD3WuKE97lTEB@4V-Iu{&QGVvZ@tI!KbJ7)m&Ahn+7;|OAheyycKXGvj)4FOAroRYp zWC+{o@Q}AKOc_ZG`r3}tC{5s}LHA;>E#(j$GQ;XBF)x?;0{oz_D<9p$P&}(KZNL{* zUKuz8*HO>TziTcdi2{E#LGd>Hkx!ZA0Yi#M`u}GQ(%l z_EvWL*^hkhAllOivyKk2tGiDVB|_nzHm;0js>-7xDH;RfsLO;miW8Mjj!87h7v+pT z#_p-j%z0TmI$YOT!zkdbJvyI(#EqF9AD$hJmMn3Ym^&e)A2wY#9AMuuwIcbX>N@dL5r(dw{-9^j?7>S&Bi)?+Y*Uk zZ9)BkDyd+b{yq|Nh~vZsWvmEBsKG6bWD+v`Apur28vX)?FEzMONQ|z_3Tl9@qC#@t z6^lZ2xmHB(+D&DREI$t$Sq{wsWLTNB}@>`Yvc0eSl4yBo#fWh zayQ){yU#K}bZql$+5)c4a*Z*#9o_Sx%F&v~rv zJZi-aErE5TRX4H47c%nY1WGiVA4ksX{^|81L2;%ypEx$3mk!Hul-8KN?XYi9sF79D zQPa@u(WrJS^SW7gKU$!870yX<>N!Wz&k&$o4_}|FOHb=`y7;wC4D`2VoOV|EbELP; z_bTSe{bLQrHq7IK^u^-rRD`Uvj+{DTi+e7fgCacSt;Bxkkrm@aqIn)-UkbCR>) z@)5Y`J(DT4qg(}9F4P{*RBy9)JT$A)hRMBg`*@}vv?|N7EOQOx>uf+jvuY4ps3t(C zRI=V8h31^!MrzaQ@Im?OnpP~fxk<&DtlxLx!&(A4h-@qmopf&QH8-44{X9^b6&VIp5Puh~Ch#;I(oNf^a~cN3|ExE{6O#&V^Q8E6qFx-% zR@!E{G*H1cq&{d*1O13v&HM-3e%Tx{ zgk=oi?_JT}j+xVe&%5@tOX_A~3LB~q&--SzPuu7?k)VEfYBw>awkW34l8sLJn?&D5 zE9j$gG}5r6mu3EG$A4|oe@(C7^4zc3E0M?zTuB8StbDFiE4oDc$@7KEC~53NfISED zpLF}HBF4QCi-~P8N_x-nZ*KaPMw^0hlnt^W$%~b!QyIVOK>oy?kn(=Hrl#WVyHTuW zA;8(Tetl3MS46Ru=pCTR`lZNWX)MYn@($-jIJW^DN5hXPjyb!VdT;>Hb7XmPqlpGk z-;RZtZxz6|t$gc?Qdc6j3!QPAI%(1TfbeMFKYzQC+YMWd@1U8cLFaRN#hltmVbdl6 z3dZFOd05VxC61Yj)E)l=slPdC(~TF%-@VqcOxx`I4a$C|_I2ehlmh(5r0ZbNYfexW z)vXW(eek4MD~~_MWqwQPo60m98J99X@J#LiU6p^(yh#?4Qc%`u_YZ&NdV*)}e5`SJ z|A;lQ(vLMoh>T@jA)3?eJrV-U`*dXW<_Z69CmsF|ZZug{^&5RtPuZV3c`h1mV-qQ@ z3l^>Le0M0TrInRR+{!{+a*T6#5i(upTdR7btP%MJ3RMI>IfZ>ee=9XLtNeom=9>1} zjISr%A$d=C-$6or!TWWT4cosq{^38{G;W~JyWV4QC&azun-@m1g~PF;kG^Q z^9`+oj=NY9Um`^BCE>i5>lT0`8dLlI$7p4XeY1x0$v+E6$1sWSy?3ST6+bl4HM)Y! ziVHk9S(67`z&&C-{b8ZDugIe2*|CXKlR&|T`PqY#*yxz!>7;yib&oq$%oFX)S;>1P zdQ!Wpo-lFyD%i=mOy_sD058kjfWxaDA29KFnhlh`&Rg&kDh(I!Hno=GgQi<&<3Dno zOlDS<;@$gcJ61Q>=&Fr(Pm5~FR7gR$Q}3PZntzdNl!eCWk78Z_it`xjE=4O^^c4*I zb|Sm%{Tym7Qj&-r5TS+`(NkrSBnkV9y{+&jIP{2&&Mte4>Cj(cKmoEd#0{SwKKjip z)2|pi?W+~k%S)x75+{g=*hA684hO5owE(BJl7W{_F?m*zg~J~g#`Bb0YuHBuh$vzP zR3b~O554jxz2%q>N8}>Li{dcffJ{ug$z{nuU)1DTU?-0+U6mzV(YMVvJ0k0LtMO25 z>#W|sg~qk@dWTOWl(r0~yypd};kTkKv2eUby|Q~bg6@EBRW8I$(gQ1vv7mh^t)*%) zC;Y;%q|*p?-R|w|;1cuPETPyc_~}{+W+HW$rK;!GIV8PDz{0MZS-27 zxrs!#X%SHfy+#*MttLmj#MIYsiOcZA77V-@UPRDw=^PtpWAN`wHHlIQ<-fvQ5T(9W zS~S>jr~kTme_kQcZzEg(7$E}1P_r~$5^?sMCtBSy{xnosH%)2p9Agdg`B++3w7yi! z*J_ax6AN4p{{q_*6CD`gel2ok@Ztm(sCH)Ew9&8sT2K}$eHZ>hVvKA!GZw`$zr#8E zK5Z~y% zR2SD=>vi-?J^AeGEx~INV)Pjm*=x z8#k+(soAS6RxEP2etd|$PQqYZY*%+PA0ffv*rYI{(=7c~S!E>#qeN|NN^>-siD;~at;$K+=tbt?62&a*!|xQK){2+M z^jkRL_=v?Gc$Sz$P4hy%s9K6ff^yO<^N%J}a%k@8Owdm;cNnpcj&?8WV{}6Hr65{?+#qPu;D*%v_~*xrC6d?8ZaqK`6y#iUGB} zMQ($*3NSq6s#k1kQM?#w)pF%jTy$t?Zxf7?`S#_iQ8;!NYED@@`Gu;iB*E*(EOQos zO({CWr-#)Nc}{tBz4A%2sfOyWePmE!5#n9>Ebemuh?tH>E#}mVU3A^ozTS2u_nTtdBVtCvt#FVrndsU&t6?@nfX!y zi+8eJZqr8@Ld%HuO0zZ8pdg53>aNPpas>{GB)RT~@wq<#ovR+CtqSDen<;tdd+M7F z{4SI*9zFAb_V`SC;J~&cXMC>yF5mi&Q;1AjTF;Ujv+Uo0`RzY;J$q12*mi_}D{5MZ z#kIf6q3OQWzh~^591khb*evVp^rUNkAbIO|w*F&(4kv(nQ(pRFfBdq+8^6(`?wA&! zX1o=(yg$N+k(wV(z@NG245FazZXjm6_}zVBztO}783|CcQGw%}-_ZT{Y#J-<<{`?& zKdyVf#X3a*U9`n!_iT8=ZzQ=Z97yDLSeLU4g0Bl@RiAWq-5>OOroI`*nL*kRH_rPP z#~yqOz`uDUK;m8L*ZkL&X)Ai=v^{gP`;Hh_Pu&Nns1UEHFo@jw@!}Th_!nZ){T&T@ zhs8&BKJXpu0b8-%KKl$@X*)E4{8fSdZ1uHoGWc#t8~=egxgCKyxyKoN6K5EA0;V^V zDtQ-t7h=2V*TioZs(?Qd!E&u?4sZ*jofLv zCT>-z_Nx#!{DRm{mD@JDhGSdEO&5@xZiUAda#H~0w*8#>zAfaIqyR^8&*)ngp7^%; z-zu$q+X3^s>e9w-GPXV&Ee@f_z~5_sNiS#qq(!$nMMWFO{3=db-u(0PyhiTamUt|d zG#_fSxdPMjdEk#}e`U?JQM%*G+s|V>Kjx)9Rxr5Chz(WRAerqnJ8mirIvWQ@C7k@h zWqcKlh=-NTQhFh-F++2Y_54P1I*K%7ONNT?#J8~3_7lyaJ zcKXM5C2(JQ7YX>U(e>sZEu2kj5XnRX` zPv6eh05{yXoQTCzHUy^K#71a~XP=(KVM# zW}z)_RLNw11=jK<$URR77`S&dk}|=Vw3vv-UBP-xXZc|p<(Cp2@Z{=bvozWD&aAAd zdCxnKI3#*c+LX;ETg>mRE=?k4T-v||0AE@ufO;m|3WPSQ=j{oaFr%d)i#QwS(}8zq zbCM=rVecloLhIFfp~aeRj_<9~r+9+8ZP0n^C-swR#60<0BetwbuqKuv`-@!GleDp!`Q3E^`wGAJS^svDmGc0$aWUFF+-OTXt{7tl zs?I({wZcNDcqz*h2lsh(3RI`42(6_!G!SgBM)|YHEOMY#R;ohyTD|(0UO14Col0T| zk`dk8biO00E;OkmHab#nxy(xCi9E^hy8VbxVA zMT5<{OJ}6sBS;m4TJFhf2ygJka=PQQhZbX)oz*DrRi|zoKI7?1?pui9TDw=g`bpWh z6S|IrCP!q|U%8pwuW9;+Pu+{$SSxD@{Ix!JoI|%VMKa@s=<6($^VOA-ICt*#1`Dn$ z%J_vqF{hC#Mcs)m{D>Fj(#p)2l*Ax*Pq)wL{p|MkJ@&)bTV9{=gF(DP<4WFk**a@w zD=f@%HLmGM*|waw=xTmt@u{V9qP7Hj&C9F3(xP>*#8rqZ1(N(U$|h>p{v#pY#HIchrwC*e@bJ}64mdc4@8OX^~4chO&S zUy9flc3t!t-G-}KKzOUwZ0{qn9xf2}!aqCj-Fm)vI+i@#A9q?y-m-H?%Zy{EiuVou zS?2CCiv8Fr-_FKvN}-s;3|FRn;B>WnnXFCja-psJ2)?R<+j-y=T}04nPIQr}h8?`c z$Y*R2|F|-{npr{(Ej0bETWexyo+4!%PS6l(7Y=I86f~b@jBf%9W z=k!sR_hCzdL|3W{YVE9j8L`z1vLvL4Vc~ZZUB}>1G34>^+?Enjlz}y0C?8VVp+?GL z>PdjF$rA2`{EZO5z|I6Tm&{0;!=figbzh7oKZUq|T~o?~z}>Rq%Krg2KKM#;lDiZgz0w;E+<&(4G@*%?rDQeDr!|q{;k2@Hons!$Sz2LKIS*MG-QThkvyRQH zhLlNj%tBcYG%1$PKVi2Xz(Si;kaC`0vwfuos%}`c44=p73Kyxc;m)H*$ozndY+3A% ztj~*4XL!r3NZx*{l*LxU0c7xY_(JCDLDFz%78ia{xwB zIztpvl|g4)yJBNgQj)Uz?z=9(D=Zs&LD8Z`vkDkKZ(N&;OM2uNHnPsHUbCOy`mS_OGrud?#f=kn@4uDd7Cqbvj>(ck~+>#l?raM z+d!J8CXb@e)4{m?O~A?6fJqIe!sh*CzU(5z`bt(^|p{rHwQ3MgOt($g>k&TO{wyrnH81)4ed2!>MSFaZc zO6d0u1lMVK4dT4C`!HS~GAnHsOrkWR4wBIJ>-$?7VE0%_Q1=m_bI@wz>)*%gH5(v? z%P*SnLobo-VSfnKCB5VFn2orzoF*g_yU2mGYTPwY=mHpN$P2+<)=Y}0Nt{}vTXxM= zK}g9EGsUQnqqye z>awan*1J+A?BL)8TskuQ$u$j z6}WV_3>EF?aI(y9a+yhj7{v;%PdBwZZ|oF5jIvuv5tWPG*K-?LwNhuVUTkf4awfju zysX|{jy|TE07r<;?)v>~b3UC%dK#rrD3OYH|iLl(o{ZxwVgM$##k?8%8Sy9f}}h9K#a5%EqKE=J0gM zlyf9CU;DaKQ+D_D+79b*hKNQg@6i`(@PeI>%xib5v^DxX^si zvlp!#qRA{l71lj|YDp$ib=M$EFOMGBjyCIwqT5zv|$$@4gqQ1J`nD-o2Ws zE0(m9o+iMh(xnK$yGUrg%jRUtQSE6GOhR7>_#cO>yi>15Z^D@0&m6|C|?)i>r73n%w)-%^feJR5C ziper`0wzWD?`(n7B5rS|9tO4K6r&Z7$hq;dN~E9 zKDctVI}Q?gT&J(EXZ~}T%@wS0SY;$-{JEDEJ1MMbb)J3DBm0b>_0YI7Z5zR5?v=xA zNP6w{14yQD0dEwO3^^#ysE03ozt=pM@AK~LV4_c^{?u@QLCbPG(3oUE&8t}$HL?e? zta3@aCoJeSo({CWvsoaMyv^bkDNjc!6&*JfvNjfNXUb^@+&cr;-9Bpgx&WzVN_SS1 zidutPHofiQUAkVIVntt9SX|uvoV{O`v}sIUWtBM74$8cmkhb*oVMs>dor7| zE=^0Quzi2yYD0d_cQHy zG|4uV zf#y@nt7YYl^uGQ{2#t5GwCzSf*vQI18!{f;ork^1vc%ysJoW^9Z8PK736Z_~oPV{> zjlY~ZL+!GT6-ruy>q87U2VgsZ|J8su{$gxQZDrQm!P++N`d4!?rw3ZLJez~^uR1F~ zz_`Q_K0KDX)lM0gY}9`)+BfbIkR)J|t{R5xgB$fJHE;rL^~sYv{NA8`NvUFYwI zCw%`~g%kz-KbIUuHj3!iWyT5&c?@g&#%EzO;8Lrh5%G-K1-|CcAkcE&;l7RI+gRAI z&-?ldt!E7Q^bX^{Nr}i}-qg>!ltGe=CxNoI=RV*QYgFbK{}-7%Qw=Svh7dq4DD?pV zg`(BaomCpFKbg&DqfE_LC@F_#R70?!^bXAc(PRJ8v9`^B{j(cP3dXm=`kS_`gU(yU z(*VS@jTs#g-_7bb|3FAF@DL5KpF6=Ptxg{U9(m%i2KX^&YA*FWs;F@|ivX(8TMF?4 zq}%E{S6(rIu47Ei2Vz`?&sJg^rc6xAHK(RgF)d@F87B^^BTXhCYsjcopQHm_Mm+dz zRv(ky2?LEjr;i_<(GNzn?FC&m$p@_ZPtoFSjx7Eq3ztH%KD-v6(1 zXB_x*Q{dNNrhe@2)Am0CQZhk9kh%)O34zfIEKvmwj_QYjl#EX_MC!u-5LCW8nt~ya zTW8{Ga1x-3KL{jbMHjr&_&}kfje0U96<90>&KNJ1v9r-qKQ-<*JH7(P&Yl-HnjG1k zJvJQJ{koO-3p6%GLf)s_xD3IQN(BKRb+VG2lDhsYksrS(Xwvq;lxaQFMIu)=p0WHQ z9e^AlxY2jG?{kX30s8GUn#_R5j&DrusVz!qJ@NbNnLq5|hf)!~z~q9KwSpsPwux8)R{f`F zxdHV9{*Sc|t9s)1`KtecwY*U>l{@wSRqlWxBBSAqO#}76#&vaIt~_EG`EdA+3OION zjR#&;*BZ{q1zmPa6~LIxV9fg6AxR-g3dX_ zzXr#l=%Utha7HL;EF!v;jjeNS2DXFF+rAOEX^9`Pv}zO+xzSp@WTJL){)g zE)A=>c0eVa4hb}prlNpV|0!Ap3UGXEIW)gohXDT2 zLLkh?QMuFo-{cNBg-Mr1xqW}1@@E&Jnz+JV+MXLxlAcy)8jCM~$Kl`nLm_D=kY)49 zqNKp-QBQF)*BEg}K`}-X7}hSD~J8|q^_?R#@x-Z6MHE^fpPDyuJ9fY%-uW+uU3I?!Y_ zZ{j$xl(E8t;{p}jzX{INHk+Z6o;5LZzmDZL^Ix zPqJ(l(ND3=@oSr{P91j&bXhyK&DNjYE(#hyEvap`7l@AQpsU_c+id?STGTe%i5dNd z>~=oThgwqGY`g$>@Bdf11KMnsf$2uznK1X?ZMGu;$V~7A;0iUfz8bhyLB~cdm_*y8 zK+k0rG^yY>Su^?lKH1-5!&z#jspOrJ4j%QWZMM^aZ{)x?`tMb82fw!2G}*O1z>|I@ zApg$twrqnKf|d{u6ixtbHa^L>Qi{L8@rEd)U`TDV?exTn^sHO~U3h{jxRH002S7c> zk=kZMGIc^g&GIz0%~o`*{TS$~Not$z{}-(_%5=YLw4q|enZZeG)BP$RRJ%R|<`7m< zCEIf*=wEZ!7d8!IFc{ImdEQ?4gyY9)a(463^OPJ-&$)b5;hF5SzXTZR8OI48YW6RB zWALgMx5<)ZDuo5W7Vq?o z>KN?-?@VBK>&*7Ldc0Bm`9YiPN9-Cn;0=kOnOws#k3V(CVR;Zwk0(5+d1Q*-A z-Ce`T+?AVFoB#SJlroQ3(4NSthVEE3d;xB4JP2FB+$Plm!1roLb3;}lv+gW-=T(*Y z*`n5cuQmuv5g8 zf}gXyO@`I1fXFnTf3~YYA6#rln>7HPaq?7<)Fe$!q5!}??0Kl-dz!(vkz|B|34ATy8vW5+D z@aw+BG~@LLk!glm{h-Sn0v{Yi$Fzp&fgk8y1mZ7S&uk?4F&7qS%{r+|$_uDF|N^FQl9WzoRFRahzpo`iW;7J@{Vv001#5 zF{%LfsRP^7Vx^N#<$Hr#_%4Ra#J<7CP6yCON}X4DgH;_@wAthyKPx!?GZk}-SI_?D zt^OZOAXFL01#ieYUy!#1zm{h&%eZgBevA97;9}cO!$FIGo4f#XU6;G3PG(OMH<|D)LxRC1O=K+3l&*tQLHRiq%Wg9Sc?RZx|tmI2r> zU824N96^eyNn6&oY4CBPMF33m2^aQmwMhVS(KLB+KlHars9)V0(Zc|o#-JF^f~DKr z5Xr!*e?wq`&6>ag?7*Wppf4mVwCu2V>yl z1ztdyS|Pn+`5DYM(Qdv^P1+#aBEk7D+gY5j0vu1^FXn?Q>I;yxc~U9TZb5M!fM*=W zNAKq#u!&oR3Mz&;7nEGqy_yKNK0KO3I9Gk|Nj0q;pr@oqyLZ_o7uwa4u=`7A*4*U_Ic_gi+p zI|h&9n|s>Ze>MA!zgP-33$();;OazBfti;oXw&bpD5U|11Nt_qJ&tjtu7GkTrXMdI zDKyhNWu<@veQatGAlb77g?3O&d!X=#&MZ#q`@Vns4}}K6eyj!%;NH1yI|9=Av*v!X z3rhg=^yl~8+TyAJ`nBR+pm=vw9R@gDq;vcvsPif)40fLViG`o&A|k0cR=PtI6z#SH zz?r#G_dijg{;xK@_;i3&@eWPUP`I6@p9U>R0~#9W+YHNnF~y|#70~T9y8uF{g3xUH z4L3jhDYXQM4gGNSbx?Bw9I+faEVO+asPzHY`bnP_n$gE{-U1&sJtEY65Rh2_de4@g zjrfmkG#5zCaDDK}&98&}ydebRxG&}Hk|XaA5&h2Ufh!RKMdjn=UxGSN{46#9A^*7z z)XWt6cP6@J!uP)c4_vgqx0<#z4Lu-J(Q}Tg1vL$Y_ycLNhTs8^F||kmNyUis;1~gH z(eT)r#MwW#{)>>NH|XfH<#J3c(YX|aSwq{EiH$}z}5S=z(XKv1Gy_% zWJ}KmGO(>ulYeugjWml(88AYcG9qLR1g`91`ML2+>I*KW`K;sdjidqdW zscZue-Hxn4-h`J;CI1Ykn_1<#AzQMQX;=ZtoK?Ta=`E7E8;NdC2e-iJF`9lVM$myr zZ(xfKkIJ7_@qydfgMZH_6>@42Qf8cXR$9(bV#d%7ad2N1!i!^K{4k65V7AbWo#L- z1AxF{vvy(|sH_mh)Eu*nt{pVT1gw;bH(O;AF@W{W0eFZ%@3<8n8d7^COLbd%HV_;C zLmQQ10Q&xoWhr=Yl%PjsE;ac=!e};w$IlwE6aW;Zk0aj%k4Hy@fZo{FsjDH7H*J|x zE5u(WPHklx8PsIXk(3604>j%8;K^>yEigKg!9fMU2+l1kqOa-n#UokZcD9}7-}A`^ zmhr$bhBfikBh7B0umhB9K!Lkuw(|_=iWTqMA|d!cSg`$D1^53drslIyPcQTF3gAV@ z1E9B(wN-De-VxY_DE`BrQ@VH$d_BL&F$3)e9!h}j z8n<$w;J=gHvDDlGzWZ_(ub>H<`2#(Dpn?AXVm@ZvwVFoxYd3Q(i@vRfczDivVxBS= zJg?80;4#{+4e^jqNn6jRehN1|D_bc-%;HXjnB9%TqE6O_Hif3dy1iMnG3wZr*mO6_#z;%wCVvTOc^LRCg)8x;cC9})F34(J$gIjA z8%8`K16H9anCvVT$GBAB-{*Nw721s!ngj^|A2)&20w4XoA!`PFC5Rs&uDLwwihbD$ zf!mw`xD}2g*QeBte>SPj^WhQ{z1P4GGm5!6;E9cyThxbY+OP@Vx%A0vxH?FPFRRqP z+k7~njtyEb5C(B_o$e){XM5M7XBFzbPp+_Zd8|~#vaQB`z!j^V5-DPKwc9Mru(o6# z8wKJ#pkuiXI*D;9GyFR`tj1<>=&4n}Y7-DI|qFT8lB# zO;+=Ido4;lBl2I*n0In|j=!=@T98DaOL3bTytpsTI?6H$bPz=@wLJcYEOxT})kU9| zz?VFYpLTw#cIvwAWBCXdEeQ)F3isL9Pao||4XUHJ4ggF-5E1^?W7k zWl`ZO$(t#1I`&a1d*ukeJPaL5u5u|YLos0Z|vqyMlG%Ed$C>Qyg zbaNUJa0Z5E>Ilr)4ub?4$#S{;G*&KT=9XJilctE|E{f15C@5%#8!9Ry zx#9+hY=3(2Yrc=>&m*3k`*mLTx$kqX>vMhX>s@bD=lptI6eJu;z}~0HMJ4h<`fk0h zPZu3U*N4w0LG#^Y7X#L>_(b7$bI5WrTZvxVL(%uGX;)NDv-9{6Z2rV)=$h)Kty962dnJYnzXi0LCm}}x*=BJf3jNz3gTho2&X^S-?#WtM|~^Rni=K( z9U24eIIWRe7G$4$68C9BbjaDn3k>Ho6fD|C4>c!phK^r`-9nRP)W@~3XR}h_h*uPT zoHTPApvw$xi41W#-Ru{Hp=J~A!Ys}&0vECt z;Y&NtUe#sMUPku?&pc2SBqkpXs6(A5r|GyGC>nU?lN%LVuIsrKDgy3fbwsk*zE5z; z)dU>J@TA5wqr4D;+FFXEc)?wkEOg*}c)POIu<(*u^S4)F+5-VaHq40HnE@Gr_kSdp z@5#w0D^?9u>hC3P-EF|vM#o^@p>OpJljM-sfMv6b$;m_)}MkTYENc0h&S zdV@SD>V8$zW(f2Vj7GC8?e|Vi<<$23!{d?;TTbutZ{Do2rt`(>TUxg=aWq9T;uqao zf8Iv=Yl$EjFe0ipU!L&qkrF4brRaia_A6MzA8A?+dZ8Ukh=mx5sA=#En2pwW6}Z&r<*Gmr>yg{FTZ#$ZAfJ$I=ykjvP=0@fs-V)= zzO{5l$l7Zt6==@~u#Ey4S3%>iE_bej`kM6%LWvams+Q7<&lg zSzO-w*w=l98K#OSG#F>AQVe*!;=Z(=-&%%MN{D*4qTmeS>~G1X`pO*vYacZzXS5>s zs9no7E{JG8hnw$XQdqhrU`KxWMdyZpUI$BJLo4qf%KhU-QqNXZhq-R(d!@~uALvNw zdOtdT@_i%G(nMtd@b~Hh$p`@

JogM1m`9ZTwl5 zkMN0WQvxe4AuS)s2-YsH?~I{WTg1c7?-O`}Jn&7)n%lgI9D&i>UY z<++I0Y2Gg#SM?nF))9`q>L?zhgT#0me4c+Lh&61Ie=v;u=#jZi(5d`)AD8usxhF?A zQj}!M7Rru?n9iTOUN8Y{MwSb_~GcPwF`hF865>j4ND~EoEJ@xSH8HIx0y`CsYbgm!A04*Y!a=W zF(*lusbx|kSG;jBPBhX_FaSsC(}c4w$g89bEiXdyL@s(g?(}+t_|Sy}Tf`oH^8tJl zFJo|r&+-QPiN&2Z!`*C-C4cB~farWjK_F&%o`)zqPZqd!2Hwb)26Yo}79MiYP51mr zOpiwt!bdAoP(Rk5uQ6a?phM8*v9Xf*?9}~PN@$Zq2aV}57K^?|LiM?@sj?Dysbog7 z{k)2o0^1!rKebOE1fPO2l2Hzd@GkFO%SBO-D$*O%(B(l5crxc$R=oZis=y%SlG%8z zsJd$+4vuI-hL1OGK{#HLEs9xW>%-#bh36a&e)*Xj#ukpkv4yY&oTm=dFCWt|dgp!h zNwKv(Gm_|(G}+@!3AV^s>$4$x7AGwXE{CQ)FxzbtlKIn5I&tSto%A5iEx#46hqTN$ z@OH0tW#e!c=ts&5ue6TZ&Q`fM2Oo}SczSt8Hd%nQqU*c*EX)e7U{+fvCtMExEI!YV zW^UOQt>-7j3rArLzyc~TDn~mdnNvdag`SQ|NO)hL?SHvYUNbo}(m0yb;;Y`gWKi%N zQk-P4f!GLQwS0TEo$5tcW?0(RY^n#Tb8Sj5mVo>GV=*&%B_;59{;>bU;eeazXtUYIxc$ubBp& zNvhJOg^%A~q4Ab#;OXu|s>c^!Dx>?mVq6=f6I)Jk(^iwE_$liQ1`W4u{o?oV)cbg; zs;;h7N8oB#kPn{nx z%*zaxE4CkEcJD<8)0y9^>D7yWR|h3OSQAl#S&zn+Z|=SX`Vld1`TfEH^54Ovbkkfg z^qlLA!>hC>5iS_hYtXqR|LZ2c<~86}EhQ3DzFHQR{5<9jHz3i`fOAHrbl)+1>DOM4 zmxh1$gDR`|_N|2cdq(8X#+6M$`NAo@QE8T15{xV}_$+vV{N{gpxPhr?nKp?~XxGh;+3w%wEJm%DC-j$)^VcAMA8dxzwLF z|I{)nSEz(oN{y{}nA1xB6P`W~^1a!%Eb>k-Xh=eiO%+6&35R3pciE*8(dfyUVR+D7 z33ddgT};OB_j4;ktltblwUg>UJm2Yi5$za-V}Ka&6zzUU$GNWj(uSiwFc519ct0r?41Xix z$tA?)fmJf)=|h*m6f&X~eP-p~@782M8ycMWqF3 zC2+Bc@}q)?!rHcIOmUV?a6?9cy58xIZ+CXpB@D!rJ*(v$yCs_EC6}$|mT-gZNJ7iC z0+>YLX{qnk2cWjE{W=Uqo3ahwpXPiC2=I-&wV;zMOCnYh+_?t?w~Q~e#)M&@S^C6&)E~W6^eRH=5?SybXq?9z@RFS&Iieg ztxrS7-AI+Iu{qHy+mF3jo|A?`L5?jH@AGuPXPVVN4elIFp)M8&F9KQqxdz ztVQ6i#4gNYy^Nptdk6Dp7Ly}H1JQzXn<)^F?~Kx{{r3CpDZ68r2JODJe7pOwWDZZI zg?xTD&KVUim&q9M$eq5i}-iBeEUOhD;BQEzG~W7A(K|K#0TKQjj__%JQ7G)~Sf zd6&$;1Y=Dq+c>{lmCMG6RL!y?FsQf=%HRTG;>z~*OD#yZ6^cL@4%0tSMT|Kw4xidK z-Z{4VU`z6pz(MmY?HRU2ys0Fv?;_e0)AD)ea}(UdD=KiLmfwM^1AA{{2)IQg;AY@d zXH)SjJ=I5=W6=cB8PO6--Z&i3#4tc-0R+ly9Dm<2P8b(m{&-fz-B zMCU4hA+j@=)b49eutG!SVW&grfp_Dd*6krGBwmwLu1cRw$16X({h8ZaM-(;X;1WbLbw)U3S# z85r5hM+=H#1#~KOiSITjJ3Jr!QAUb~@oJ;9LynN208#t6t9kgAx)^sqPb&9dZJ+V( z;jWknMcA*KyF|bzbbI+&Y?zDgEMl_#lCam>)HP7Ku;kkL)RUOCbXjWa9kcZ3{sEPt z+w7Oy4>68F^~!g6PepvYM`2!GQm}Xlla#SPyRz{BHo)#61O<73M9%XL{fujS)jN5> z0rC-&FZ$TI9MyRrX%wnGN!+mktAl>XC5+bg)CLq@ee*MwYp+o|-0uS!Stk3f@4`=6 z=+!0fQ*0NA`QdOQh{yE~JPUVajV_en)@`FJua|>>&g2CxC05}k;vb#XH?mznN!wpJ#1F5j=y6e z|Itb0+m?Gf+@=mxU~8rg3UUT>Q`&qCwaXq+f#=B5oR*|U+^>B1w`(42)&y?c4Mm6k1kT|g;8VEK(lIrFf~s2QUFuT1M;(I{-QuGaOR zs#`meFocyG^{!8-66L4Jj7F#@l(t$7B=YWEpVh{d@#eNX(M3wO?mk}Z8M5z=;V6p! zXe%C2H2?r*gtkB>IpRGWM`A3OaQQ;Kw6BqP6`Z={U)7#$aK%{FWAF9&A$`!nfQB2l zw6^~#HeYJ>cPM(m46o8mE`2qZ<&PU_NlWz^n!h(KAZS`aJw%er&e>bpWG{4%7dVbn zBWRxW2xjyV-UIeVnoAWih5I?kRRM79J>|B?@XPiS5orIIHVy+pvKiT9Tp4W68U`j( z*Yaw+S9~eU-Jjgi4hTO5JHEv7CG6z&Mv5ctr931T?294MmcVW&Xcp^*d4Q4+13M^! z+>iEHls2U7Txk$uGSfMDnBd~SnVfsm;+mcE=sagc#1jCD3wQS;X$a)8P$@TVDXPxG z3&92Cg4S&zZue<*-T4b9^xXc~fZ%cehs9_zP|fEv)HBho`+p+@w*#aLxVRoYggm8U zTGer_-}|$u<8%P`LH7l=sTa$xwi- z-Q+TW)G?2!SydO$I?%impUgOw-lgpz3Qaq;c6C==b zr2$aB+DS&I1jhFy8t3mmZ1_|*p;6XAxSAe!Y)c`hy8X`Y`ZYoIK`4yH_}?iyeUKe9 z<=!`N0^OF};6v=M%BsV9!cBWCr*v)%f4ws<6|ji=v7C?ns#lh+Px8K4wMQmxuAi{E zSgc&-^=hRb3@O_8GtyB2wl#Vs4<~K1fb=|D8qK}|hp+ZH=hhz_OB{(q9!?mEgYa+B z*1OaR(vd&gPlll<$C}`Gi%V3~J+8F6By=8)&ek5Ddgz_|tbD{*AIQ5Z)IJVLGE`|0 zDcc%{E^}2w(5t)d3*!idfXwp21w6ZIU`*VPDMS3Gw-QR;X1t2eZy0{0@1?uoc z1Dz&iXflI0C0RR&?U_Y#-82tYFMIRvE=6KI1N`s7>j=K*@AKj&+>cky3~+#Gb5GQJ zLh+UdK?VXd&3#=#jX3HdOXvaal*i{S8b0qMM z>yww$jL3Ukgi)%{(OKM&v!bh59I-KcKugd5#CjZY%1}N_^hl}HQ(i4Anbv$4fsC46 z@n1?}@T#7yl6%o9%cK*#I-;6ne;M)noT|S7lxaBAt~u{F1(EI9^v2P3Mbi!X+xuxm z+{BZ<>72DZ<1(NkvCM8+XgVf1?7Ij?M7iGTQZL0kt0mUfft2)OJF6Msk6ekQQ1{+a z-;xxr5Me{JH7kU_^*_!eh&g$YBSsiPhsWp5r{mw?x8CyE_=e@*lumzo)*_Yf1J^>oqRWc(-CU-S!@n{T4x{{`JD&tIYqjVj#tW5>M=>G zedAd}z{GfsKAkrI#hXgx#wAAxgb$=>iQtXG-ZhDY*DP;3Gf4!!P6fe+qpWD z!19zzLMoOzJKy{(3f_2jLM1$QHd{IP!=#WPY&BBzFe7wkziV}Iv!_|w*Bs@W0TBE8 zjQuXo=2bncZWv|1CIlfd+*Q6sV|b(Loe0=Y_BuOCYBjDU>saCY_DmEKnY{w4Fmht1U3y+1_ei73eV&xmb41XYoun8!_Tp`pEB$`lmrR__*=Yg zex2jlB>e`Mj#_n%;2CpRK&xZQq!N2rVu#(nR@4)$p=;hH(*Rwa+BSNI)`t0$1H%xT z*Z`KM@>-#Kw^zrWK;Z~1x_-xv0b7TfXUF-CWLbYur;(tCOg66U*FJ0sqb*5U@)j*?u~B1CCH{ zXDPtb6$QWEWVB4qLJr7icD%|jK`7rQ9U5FX>Xq(fIpR}lPE*I_m!dt_waV%UB}Z~s zUUWFrK`RT(Uvp_XiQOeI13ek;lW|sVu%xl)FRgkvuOee1qj~x-Ei@VGx(1D)KEL^@ zB(<)4wz9=dQ!^`)Ct7g`S@zA+aEN|+(K2peKN@@a%^fW@w!ofS-|w%b7O7{PLLLnq z%r%vxg6s!bfLOb7*R92W0WLSut8cmfOoQX2q-+dB`Y0OCWrk2x+$*G{W$>q37oU5M zPg8mSR*~MOzTd@+4k&)I&u>AAX+$o$m2Bm91JgNTmVr~Sp$}%w7^0n^_?-NZ{s5#j zj6T%4$I8)?Hwi+El6cGCUZX%h2+L31^VRyUp{|q|h$41OjuIaazE+8|4GG0K2)2_w z{ft`om6tb;#24%fcjy&!a%6F7n}%G>3_8^t@|Tl@Bk|Z8K#`i?+t7~qko~iTfrUP4 zda2cJyf3T)Geb3QqJwb1qe4Wdth<+bES29QT_{OASJyI6fMtVi$zP<|!_A?~L$jyW z=fyDXnht{klTQ1%=xlg68q-(-+Rtq`htpV5a1X;#Y#1{0cV!hsEG}Uo%g!S7nq2=& zdJ&9`bGT1P`WSJ`Y$qKa8T)hL^kcnfztAG`a1vK1r2lHXspcxRwayovB>&m?ZY-ON z1ah9|y^+M%tGeqFPj&nphs}skbu>SY_~81mFIMcUGO|8egXr9IWfnsO`#_4~f)|0m zs(gA~p*@{68bQV^^GTi8pqi3JwR8S2U4yq zFbBNsTArvJ)VE>FK+QXy2m4QF+ zc)F<|4Verdefh)I$`5=>gCed@_e*uVssHZ)zvTiWV0v8r@k=x%Q?mX zx1+4MQgCuep)FLX6JYbLUOS!&G51~OVNa$`BxdDR=CMl!HJ_XG!G@?2%LcQK)%Mvo zlR5u4d73%j3l{pL%tjgm7SVkkP{g9gkj;ZU=XN`5U&aWE^o(6o+QPbbd@hFf`AFTm zBa2%277yM82Qi>-f=KhM;HP+zp|!b&Z06(XBQN||bZHh8dNVw=2!4p#J#aR6Sa;3g z+n7_u%h{>j_B5oAY<=v(I6TB^(Q*7wwF--%DB&`C%h#!&5?}9o3h6j;J4f3IAqUE8 z*YQ#$ycO)rNgQ8XT@xf!d^+|{EIu2aqvqlFhG8Ql{l?Kr7%KOL7cnXL<(d0qQMkF_YwLKTbJ*bK@Zmd|bA2cj>YCMELv& zIpPvQJc2|=xhBsXE2+BD$O#h+6PwjnySQd?_Jh`2c<4xl{wB7A$kHYqyIAXejTP{x z;{Dr8#MIc}w4|W8$Q|j=A)1HjfpFi+nVI1-zkgdbba!k3tY4TSo1f;!HLV7iT?Z0bo0SKJ!%+0uKE+%wz?`wpQ4XlF~ zyb1tKvzMSDGe`*;iN{tTo^tDm@85a9Dy}v3Rq9=(6llVI8h}_LJ$7XlU%h%cw-A7w z%q%hI7RJU~#mZCVt8H5r!5YV~8fC|+b>sQl%#r1jR=LxEi!bcf=pXoVNv&dVmrJWI zaE1i<@eSGu=Pi>@NZMtS+dlP)=Oet0IZ*^2?b!pC_Fd_iSnk=iIIrVhfENrga*jia z5$ji2;XJ(~P1YF)mQaYbPcy@s-y}0$a-78eeq;W4=*usP_ftIJ8lKJj1R?buou8kE z4?FYuQ7yc^_f|Yr>M2d0DG|Nxd7iR68!K4RK0~t{FmW)qRgA{=PbU=aVLvVGDW1lN zO1*Xq>>qakO$lV9fdpDVwbDOH znuaO>Vrh>f!0^!{hclz*KIvT)B+x*uzLDNRCfWivDFeMeCH<#9s)+tSokWYwo-UOa z4i^ljSe_laQzFL($nVr+rZA1BTmGC%3_SWq2ov55LbkH#=C(*YN+BvUqfG|{p*?K9 zWyAb)VQuQ5!$3PRL>V>hi)E4r%$P@^xJ(C2*v`=5ueKaZWSVWdo(dqPhcS&b2{hNhL;QK7n%cDZgJC1?`TZ_<|WNmTxckhRY0Cr@qp1e;G9dv=U z;xms96l|3WuBJJy?_3Yn&#`oyo?WMu?ew_Ax}EcZZAbFhUsntVt435E2;<7yPy zjgnR43md)xsLLH+NVhSuLW>5f^=cG3kaPj#Zni>x>mC(yY~Yua+`o-4`cKPO`F}dZ zvtAGXQhtvW-J#jRj3zqaW~*98dG6sclF#SVs5yR7>ad;ZZv{=1b_DH4d#W9f9vHfi z{`*bNInM*fa(n5I0@vzN${4TUdhz;@&b-}JJ|z2rD&V&(1L@}Is^+l#y&|^rXYd6@ zh3K-+p$){6f#j8@9)4Io(RYFM&%as9nzLGaU8}-5;Zbt3T3eaZ_Z!ZX^_j}BKym+fxIym?7e0PjmeZP32p@o(&Azh)DVRmQeAZP4N`>p><>P`G||mS&M-GY@Vg)MtWgmb93{nQiTiE zYTBIfpY3+A3F2(A4!)D|AWueqhgk1YDno~ep!$WLs*l|`2(=^Y}5gbcr@f*hsuYI6X@5>wIX(otaIx8r(jR^4t zA>@<1XKiNNDN5 zjYLYMOTvCJN*@U^IqD;U1uaGYf<#KCd|@M()!XMa|9WH-(0v|dB6Fx+<5F09NaWrC zjZ|#@CX^|`O(Khw{w5N$c_<X;={U{r?As C=8h`> literal 143392 zcma&O1yG#Jwl++F1P_wnE(y+{!6CT2yF<{y85|NEf=lpV!QGwUKKS77?k?Z#ea^l2 z)c@D1Z0enw=IvfBt7WZzp6&@%l$St5CP0RPfkBg!6jg?SLGXcrfhR?L1AX)OBDn(t z^GeK8L_|?aM1)My5om5{V+I356Jug%h$Y29({E&CXxKkOPmAp6rW_Iyp={_o*fG%2 z-SOM7wMiDTEdVfj}El`;b4yj4<8SomD#V8DEs~b&0do=%z><1c^)4xqSo)LE@EW~H7cs- zFE(bx4B?PM3%FLZz7Bh`DzZu%%ww2dAVZI3uQOHovpn6p|)08rolY^myh7n<21zW*m&gl!#8A-@-0-DvV zrK+a0rkpH~3DB0w$P{R7#sspp|4Rgh55xlv+L}2Vk%4S&?3{Q&{1ktu;DLt!dJUi; z`#XuVH9v)>oFbVB(9w*HgNdDqnL+@WjEs!W(bSwrSycSr;?V#2DZV&6+w%YbZfffS#fWO-QAG-LLqW^vi)wBRIAK-s`O#nGtN&N-}<}-|xsE{h?)nPgUNL36s zNWlX!D_}WQQev-Fy4wGzAK6!9k@1zFFwF{>udOJOm4>f`gfQUY3#Ct5Lncx$V>2Fi zQcV*_s-#@IK4Lt+1s}J%9S<)yo}@3G9&+vwwG;cE$jM4dz9&G6hWTgD=WFulA3tc; zxVuaz;z@flvw_#{_C&P*Oc(}96V}Ja2Me2+QjK06%U7qL@t+Dtf4-)`z{REhARYhn zG_{q4P(lWl>hnK(XpsW8`=!=6KVXr>{PR}&*@Kagkly=LgDE#8rTpL}|H-pz4C}K_ z?T?FF-AxFal|>#)%_sZy+czpvgXd@2)I&7EnGdfFusW;~cYkPxjzbD3cc|z-_9iV= z1F*&a>b&^r51Vd~EwaSDItDg?`?qVCqOWpF%8n#z^rQ_G zyt~+n8H-*;tucLvMdn}f(GP6pb2Xa(XPtLUOOvs%_;;4qqjR!ricwur+Q8gJ33%CN?01#@d{~410i11rJhmN5W?^;BlHpCo|WZKPVX5 z1E<{n0q`D=9%vj~H2nvkY@h?o{tgKq;iB5}E)!PbkK-+x8~%}snUeT{^g-im5>t$y zi*+{;B0ni++<1=V>n#^*Ii`qpj(b|9(_TP{1+uTaCDtN;MO~j-cjitnX`I!BrBM?~ z6@+tM)c$Q;!t{E)r)swwdCu89*iwfBQaiz(T~@<5D;gFAf@EIwd=sG|;l5ZEKr+RJ z)vbxq%Xr@>^`{xJ%J|CHSMY4%8j>LjBzR-DLPxIhQD!_{r(2LgmWrLK&|S5LDoqf8 zFr<;V8<|d3Cj23)35}~!3f4_Fy0fzQ13RB8l|)qq0Y1HwMb4(ZB4cRtwR|X8XFBJ3 zCT)%($2BUcu}YuH;_VTDyWGMwqQ6nn@&jbxTVAu&h^`vnKDbsn(D8#_I7k+jDq0TF zW~G)hH6p%?>QB8(_6m#y4}a@P{n$Cl+4n<9*=vO5h)Hl=- z%l+lsp2ssZ5vYq=naBQ{BnfuRJKZ-?sNMOaL&pvKSSnAI&iuY>d%R!zs98Cki?8z= zK`ftk&Ws}RN2QAkq|eW5qs;m*AUbPqZ3Ys&01O60YuNvYOc6t_>-jfy(1=OY#&YAnye(|_fSnNZSdF$tM50!J8fAN zqx|zo(SUW@8I7UNc+8Yd83fz6=mQ?R6ds0UaB?tGy+YY1C`YeVxPRa|d z3C(|BCp6n7ZCH#BqK799oVK;*ed&IEhMRqI+IBNA%t5$02k0HDPPaP;iLSgbD|uWt zL6iiQPU>BnQ@7&QD2dE;>TTni&{F&7sx64k%6L;bZ4>KP+*fLEt4M+KEm}UQY@8w+ zFyS|XkF%!@S9J?{LUG&Ymrh`7cE^)WyL`8^{(i|l!BsKteC{$fV}tJxy?mJ}`SNDD zcVf0_eHY+ke%8}kA;;6Ptnhtz?bUnKXWc4Q@rcL8$!!hq)+J9@6n*wTuZi{_VT9Zc zXIWs6K5}$%PiYn159Ot3WMu%Zj9<2{#ncv;d{(1nb?UUmM=}6iq(`MnB~BsO-hHE5 zG=fi+x^#&QA0HyqPw~`_QaDVZAx8l%=fz5={4^eK?LIBtKyN$&vCm5tAS^0UB|-^kc%8o?8fQ{85N zjL7!00Jw09vl*LSmwxD3mnu&>ao9e%47^(R_Q>q8AO`;!oElW`yh{Z@^}5Rb2xxl@ zTXfXP4u(N}b&T)66&Lqoyn6a(#X$pNB}|^Zu8NXmie*n6@av!i+Fa*~Wu~w8nNB zu;&TeRmSd33EBIaTCjLz>@_hki4fQ-x9olbCnYKAIGs^%n8o)p@%iFw@YzeoKDG5s zSsc@n#}A(%R8JL3d+J7GPu9rHj|wrEnNhJ~;W(+*>0B{XA^EP}yaF~3>40_uYB&P| zG8ZdIte+nYtwvo*3x{(^iB7wI9S!L$ocH~pMCM!0n~UQI(v+&09PuGrH)ocN&aP#T zK1>RN_4}dn7W3oRJ1hcj+XJn0uDotfj0kT@;$N=3HO&iP4sA%i?~@G~K=L5{DE_+| zA|9_<)mPSt*+lxBP;o{|G&p-d+fRMSb~$7t|&ej>ZuQVrrMNX_8*KE|^%rAAEbRm+26NQpIeK|&pSD-oG2E1SEyYSkTzB^gm`kuaK8*`O6EJC)8uyU zyohf1p|jz%no^wkhyof)ERRl|uZy8epdTvYupT)oo$@MiQALeZJu6?Se##TRvo@VN zRmhXf5J!p5n;ZG^<}O+NmG#mGJJ;mzHjNz-0Ah2O)_j z0~C*TXdn0;FEtCxQBaPX>yx^BdWHaOJ{(C=lYk%mp4ZayS2pG9rA6Fs>)lek>``@a z&dW8nOM~Rk+T*xpToQjaYyR{x9sAStDyRo8)5&d5*AMRj@!SgpNuNe{CKk{UQKLI9 z>9{kapf2WHyLp3swgk-d54a2JNHEF|(oaq0A(05UXF zy5MXk6we2G@m@bU0YvdVMxVXm-;8~6IV1HYU}+3S&Z^+l_f-qrqKaB~zrdBu3?_;G z^_?))$(!_;(xm-Sn`FApuqO;Vp}E^fV3ed*8x38rT3-5pahU>VzPICD4e%fjB*Q04~ zh;E0={#0W76g{qu!%=x}L@b>eos8U;R*@;vK%9;sf_x&5$&RPUyG^o;!p?Ix1wpf3 zt$I7+WPZ=c%a#Wwi6}IJDYsKE_s_n2Q+cUrGdoBkpxq<{K7f#rkg`^w58@O^-o-_r zfi2=_YLD$a``Tn)wa8S|1++F(tkQSnU9TcEe}&I=)9TvwrqJQZ1s1G3JHlD3QhARO z9PBWfW)x|WRFJXO7pZqjk%%}wN95ct=yk_ga;n!13;*Hld= z)N_JnRqvM?#U8s0SGmTPY@j4BZB(IWO|-Z;M+ZE~IhSw6Erx@0djVJC`9N%h0x%ah_gOylyE_ zCdDFKfCgg3)xmUv#+1y>{r>cK&M%Q;igTsQMHv1%b&V`NP~T24dL2(cZEG;zHC{|q zRB~v0WcbX0_a#=Jp9wL}tcIk;kleVF3`-kjMseoZNAV<%!L3R%Wrj<)M`>ZkZn8nTc*gXC8oEMEZn$M6lap~T7_ zf5G(t)N@jIN5;Rtji5^WA zPMV`Q%xl}*S5Vs2F%I>A8sFo#$mL)7W3NVkR)6PL!0Yn}+M8h<@W*;6TR+WO%8uVEQiSMORad@5F_jSgHF_aP?RB;75Jm*t5Z$4RH10D6QtUe;J%u|Y92a|eJG(SHttbOoY z9eX|Vsl_piB2dn5`x_4q;&CE>z(!}Fiouv-35ew>EzhP}Bha`;>Zt9=JFkxMCkkI= zX2WWawG-0LAa)sfE2WC{wpybT$b!8lqy1DAY~{4`+nV1SGsFKMZ05*df(0~;+~#XC z^U8jK1B_OtOWysoZlvx*yz&yNJkJ$#Bt~pjs;^0M0-(lM=d^zU9OeuYjzjdZI_+%c z7C!uXRt&Yd$#FL#HCqm8s}n?QzKL(p6W1W6lOd2}g(g093~j?(=&(!Q4DJ~+ov%`x zX!EhHcCX>fSI;iz2vgP#rvr2wO z5asiFak&|Ixs~poByrq?z1@20@&dqX&@t)oQ%-c5?;~mXh~B~86?u>V$QQVQ^!-89 zM`v@x`}z(yw=(EMnPz47MKsYvu-cbE3o9nMI};^svc3yllm5E~E>JY^4_@qwIHFh02eXEpD&IWeD#L``^lB%k@5d)WxeSVWw)(LXz9aDz|EU~m91C!3kkxDuZQ=w{>7`C-oGe#5%8Qu# z-rEE3Jf4m3+y|uJRo#>>FXintckv%^HR6aaTTWU?rf;5h(byFjGk5ENVmqTxu17fE z3=wuM$KMFv_Jg_w7EWm8AZe@$wxuZ?TLaY3)aB?tT>(Ksvs})n2fv~ajt>|SIlLqw zxn94%v-_T?CYK}^QY`DX=Lh8qlx1ZE zx3CN^LKDo_grfSm{9KDL(Kv9>;B>EAahfPGv!>0?sDS8cEGfOtHu1%dWWmv_FVo9) z=n4{-txw=oSrcF0cBg)S?LPMY+evg5(9&+vMEug*X%|D{ij`p0 zBQLDXggxIbW3h27!5@&!yfT`VPn_U_0KR*2TaJeWqfYa+7^LjH+|vV6-D76Fc?G{2 zK>Jb0Y~0Jj+v1N?J2opH#u<;BA~(LX^~MPZMCw&jsF)!pJU zwI;HRQNV&Wp`@0NhR+c4c?(FE8=Yo$~mnqvq>-FB;UVL#N!s=s&>FWypU6HkBrQ zAZNgf=OL2i&$lu7wX3USCr*#%m&PwwuG2&D@n|oH30lQaSJE3nd6moSyylWvKs{(_ zLGEn^*N1Yi8KSiLi3$!F?$>8i4>km%GiM29 ztabZ|4@2{&gQ<^fi6aC{+ZK@;>TInRrsi{9h#?lMi=2GTlRPK7H&zP(y`sx$SD6;l zKSCv>7+9^SW>iOC3VoO{?A~mhaox_WB#0fTe?d;%gS}X1kT9jOE zmt0+(b7DP8vaIsBlAde#Z5L8(0XYcC8w9wG;%8jMR>Hq|qkManm|fp_l0CApH6o?b zisMUue|@qjl%ig^dW4~s2`@>84Fxq-UUeD&sm1;_a%5y3nNZ+fVN7QYvhe)3^aY~g z?Fe7pVe%jAdyaW}bMx+H$nEaX&5ULX3P$-K%J^lj9X=d#IsUQmDWuy61jSo9VsMXz zu*g>clPMBX^yfNq46I?WeVw}|5zU{-CBGebtar4J9`Z6&0iETX4Vp>tWn8ejEe<{W zSZ_oARQ(PKc)aFWk!UER1?742gKM>*3&N^2RW!AR{BMioe_+uyuwNwz(mZQMDyi5Qf)5PR33 z$yp!33OT`kis0h?XL)%*dxVNzfULgoGTs*ytqvMY2K$jU*Pz>$t)8U+6F!=nr5gipZyP z16V36v54Q%L)o^urx`AvKh@L!h3u8w8NCm|2|g_IKUjkQP27>CVx5A2ZERQkQ!dXB zJ|4!)oA%L>?$6%#zi7qJpFaArCqXI0Gq}yMe_9L-NxOfy{S`*Y_o$?X2CmR3g)xm5 zhrKqB<48B*KO_8eNick@aImnKTL6E5f2`aGcC1tESHl27HQ#sM)8r40eDK*O##B5X z<*~5V0bm%A+Mo3M7rb7M2%TDzlJ-B_zP4KR4N^rDyh3g-j2U>d<$#2Qg^g|7;#K|I zo?G;v#9evZSGtTomAF#l}!THGTv=!Shi^uK8 zr2GlE{IMCGC|b1rb0{KuT-^xakRNVlQfDB_ohyvitjPA4(v@&9>#KVMq3-% zfWV(Qg`;+p@Q!|KZx3uQjwpG=R{eJU768a@VzUyF-^%{M=M zhT@qLXrByd)3wj)jC;bW?7H4)HTw{suMD24SmfCiK2B86xx}`ESjIO8_dgr}RB+1T zt_V4IPn!|>ETBWNEMDi#ib&v{4>*G#0B1mI%JjUfBhpF`P)-NdY0!Lpe9U$D=7AM2 z#xmxxqLL4ePAFC%SZ;k7DUD~)(k)UgqgrZmjMadv+&-#SxJXGCK0>i7 zkMRyaHU|=PiV_&6Z~C3;MRjH*XWrs+|wMkKAzi(7VX(lv`P-;dYlbf>V3We))d%Pan6A8E1tk)V2lkV(Q4fnMgv76Jt z(d$;4#U|U-wDFU|CYJ+`#X*;65?&u&^#kL$uYt%?-w(>&F8^Zk$jQkgH->7Rjvw7e zV(jISu^B*8BD3|r?b|9e3_|`HPlaLOQIOY6o@lrab8eh|m-k|V23upJ$u+P1;xEiUo$1bOUZ z$SGm2d)KERX{wt0UP(VgKmy{q&hlTamo5|RIEzCYt`9Dgl({B=yK7N~wpmJ8h4~#pJXzb57+d!CCqqj3}4c;5)U*^we3l z4>KE=mzv1ycIuP}lmrBTqCEX}A7*$@2G2~Wb1lHlxmt0(zZhG55zz6?ugPIfmf96E zJbz_ste?~rPUbFsKT^iNkWc6NK(A32|BcY9;`UJ0_c{BZRb?9LC9v;g zIrh_k<4`pnMdS^7A~H7JU<0ttH6EPFX`t_OXZBmyZ)+epn^a)afRf8q3di|T1CS`=#Gejv*=@yJxmH^(6CxCRydAqCjK8Q==UhlM z>)S!U262BDh9|k4KL{$+>t~k0*-VN4q)7dD0UPZOi84;f$BLD<^F) z`sxficEav{AGp3+SOsxEC;Ma)@i_OZ{s!?bIW)6ddl=;{R+$Z%KHhA|WflbZkRDz{ zt2SI~v*9j#UC7lhaZU^Hxa~3A%0@lxTWdADmgeY}&ev`pXYdn;Q_5#>(dP6;B(~lY zzl@2XCxhX)wLH8{Y*xG<(sK-&CtE$;EXSDls-L%T4F(CG4jbnWzQt~n5AGa%{h68n zc-YYAb$6cJsFW*-IXOIOQRC1hezz7(O0)uJc-0|-&a;gHLick2d7zkUK9b@HJn@vf zD{FThC{W57-g))xF^Zn5pt+zgELI!!ej?4s*o*yS9UF6l$W z*sK-bXnuc~rmeE>OWnFGd>2cvpifKWrfy6LE-(OQGZ@rrIqr|yVAl5|*OlS-y0RT(Q zOM94bp57s9C+#n7Q>7Z<_geZG-!eVUX|G!Nv-O`pP?4VrXS;(A1)gs=CJr(@H!J%{ zrkz61%fglw_?`oXb(jU!PBt(p#U)46xbrfZL=nLWGS80}3dxsUq8AMGmZ(0=FVe3I z?&-u`+H$G|n&I!Bn@`N3Ct6q!M~uwN?usH?eW8XaV`#86y}e;O44Xaz?|#s_oM0SX z7$3)E(Rpa3_rYzlSB}+XUzIl3DL*mHsO?k8u-YoT!u`&;$U-ok6E+Dr6Xs6yD7% z30oDsH6_}G*(LGgB*wqIjiRyBY2b3Ny+7(5wbY#><+v(;xr~*-PU!%JtG2Bk0BXU0VbV2WD>+!LeSIA{`Eb3L zl#6U)(if?uH9&C0dm}t$BJ5g zt(|X2Oevh@&5ss{T8T)3hiVJ;hi4{t-kDxoLkeyAFu@0;G$Ar=MJ7BD&Yed1av8*A zjHT7ku*M~=KuN+d9AQ1X5jcCzF2$~mE0QY#Q9SdIilOOm?=$g3V$0|QCsH=>f-fEe zqmi<3L$?e-K7Q<$^P**w)}L3cW-GGrU45R2Hd6Xtz#_gdpD zGGDWr4M~n0i^&l~>IHFMjJ7{ZJjn}z^hJ^U5Fh(UeM7K)+zK>a@X-+L0&PK^Gs*xD z!s{lRxvB~A9nx;e0MuT$RO+Yfw=HMZ2ie$~3$C_i2|-i(HD6*xZuN>f?OF7=zMgJr zPATSo!MuvFEm(fIlT9U5T@38w6_rJ7Kdm|`LSZ7{v@#t<*z+oP@J5>}{pwn>z1wkN zZ^q)ZElXLVb8;U|l;xNjw)Mk_Ii%t=wwJ>Yk^H4U$D1RfD+=KzvsW_pdo8F_qexbVxWKsZN!T*R8rE&ANeo(Oa zCDHH&CX`K{jU21lxA=Jmue?1~LRtxpGo9C>;60tX@K=JxRIQ=9V zIPFv%w`%uSvh5N`P1X=$qdoVRoZiv%wYKrK8_&mc2Dv-7+x)K;AFk z&{Z!>KF)l@Cn-6F5p=qaaNZsJ?W`ozBRQNJOGh9al5%%Y{%g@gTUfM9R~SzdG3spp z>|ruWQ0f)-Rf$`6`TlBWnbTf)rOEeQ50PTRh~1z-6jJZgtTXe#u|UOU3^VeomHB7& zD>{G?4ksBr+qOxg^&oyFA$UHhJ<$Hg3{f9f+71ARTU0`9$iVl>Oh82}9Gi_KLj6n6 zVkei;bh!-H^HIZU?R=YvC*`QH;ZRm)XsTVy_*9^V$Jf*oyh{UNL4%pq9xd4+g1yey z8f!O(js+VNW<#_41O$|-j)fY#St7x5$yYOrCGOxeZy0jCjH~pkhu=Yg2&9{Cb)BRH ziFA6>IJ=#20kJ_4>P`}o2zZT6{~!#!oE6eh^LDnEX)$aJpUDH;ZqQ0>&MDsVS+|^)oJruim#@V{fkUy`(ib)a zH87+q3t(>ERDUfh4fg!4IgWsum2QMVJ<00kohgf>yw=#F)2IH^@YRL4jiB2u`rbR{ zsAZo6+v0Ugy&&7u8q2WjT#_zA+Oa{Wqbu%gF8%Jp&Cb{VL z&z>8{yr@rO{}rX_NP<5+NozY~E53IeolBkP@J<-wtbIO9waMmLg_n?!8vWX{sXN8} z-B?>X(QbvpfjUykzX5N~{XQaXa)m2uhk^12Vq@;p*8_~38$ac>3F7lHn5n4!M9x;% zO6tY6^l-D)m6~t2cY1w3Wiz^hBS+{tPc`Cky^J|Amrg#+rr*X6RL3Qgs>OCc!jmor zjlazKY@cpK1*7nBmkN1tx$@cm?wG07LG9Ps~ffAbHPa2 z?eojXp%i*PT2#GUpOVv(W}0`J7b8pO!D@FJ&~`tu=KbZsqdM@%^C26-vTCLvA5Hb3 zzW0?H8tJp&WkYdrBUPuB@7We^`k3Ali^Y4Tnz~(QrxI!Gf0vX~HPP)W&^tvXlix#RlQcOpBp2gZKrO z$}>f3(x4a>Ql)oh5LpmM!OCZSuQRdgC{EC~e2zY)KKEy1|1NcGxtpRNQQ-J$BRgX| zA-?7p{|A)-Rf)(+Y7Tb0hM%7Hwr}ZaxP22wuPz$5xy+X~Z9L(ioReqpja#XC z!S57ZXxJN+u*m-f?IFC^)+D2$@#Z!g-f3Q z5)bDyW*i`Ix%-P#MqWK#! zxi*x3(@I;_?K}R4o=vdYT=Zo;yKdHI$2!t!lQGCBgA7>8n_A@VSgDw&R25udov!&L zPgJ;8y>_-x)Mxunrmv++QL!pSUgmWE;8`dGX5 zy`G_ytG$4{Wv$YnQQB<79bKqd?O>)i&i(QB%$z|p1ny!J-{`Fo!)@wRg^~zm4dvxE zxgO~Ja#A7HCo+@E4hPj{sY#9COS-S?Q|Vf5K1u@_pK}kg6iPVR*ai@#jnZfQwMav7 zO$KGlK>AMnLFK^;q1%d)xu4g6x1*&0vZHgJo=diAr_noZJ%iK7MO=f?FSy*kI_y9? z#uPn!b0^hP$G!!kc74}O9~W%z>1NSvlhMKPcfnLNx|6=W;zN@gzHxdBFR`A*n=!&` z?jJNl+K2VpRDAe7Bz+IcX~pQ&N1V@YnvvcSXJvRF-KCPRaeAlVLKIt)rkSp~9JCNJ z{liZd8)bYi;FCY!#*uPUz6*-_e&Q$X)aB zW<=H}m!7orM4iv*;BHN;G5eo3Tbz(kW^^>W4Q|p_33mmmm+H%<6!zs{!;+UQu2YoR zXtoO#S@K(-RxPx7H!r3}<^%SlAk1yS9FXH5fZo|{TwDG^Cnd`KuP5x(jPOXC*ERYn zTzU$xe-i@R#G01orl*d}vOIa0XQV#9aB+8a0*<{q{BSo(uZl8GyWoSV(IyOM(@dLdk?=yZLE!4Z|lEQMdAXR!G8R^H+yUbOYeblfT zCF15_cVOhUU-i>6gu!gKU8oa}PASv}krqLYPynr8T6nzG5a66LWH7Xi)TSTC2wI5A z6@SZ2Ah+YCUa1CRpeU6>Pq5X6p7{k9Ao(M`A^%j$eLV8mBm6qVJnH$R@_i<)Zhmgi zhq`P)EkcPq84bQ2?$W9C3v%iMW!HYMUb|;LAh+)NvB0Bu>GW;)Rpx#+Ovoa} zU33oay5TaPyIk$%ea|}lkJd!)j>={9`ZZr2&bOZ6$&V}uK%-0jYw9d>xoPTlC$;!*w#C)aeb0XSe? zy!lhZ_(SDmWgLEr;+I}qVco`4?FI&Lx5mq2O7G{=XSXyim$yX24C-^UQ&H5tOY zCuX{9;~MR*;NkIwvH(gNp|~^mmdD}!P45-&@b%Dtzwf9k1wTk7!NriH^myz|W=k!f zdf^twLzzk5AQ`z5p+BU@yBmAkD$om%bi|>ddo6;sm6w?`5Sy@&=Md?nOvNN= z{JccHAm7)&=qo?a&JudQJQX-`+?zTAOiSLo3V-|;MAiBGptF2#8!Mv~jo`LQv)ll5 z-d5JPwq}&UaV=DuNK(T7d3HcBk%^N;0Fh%p*T|a`$_IpFd4Nyvm#%IEq40Au`-7O~ zVU!>tQ{I(`BP}}myJ3~HCehH?kA3AA9r8rHO8YA4`;!oMK`A$1pC|rDj4oIMuJb5ahPeCJe;jO9BPsZBip0t z^j$%36R0!2&a=l#y&nW(v+?1}n)dfv+=sT`Z!HtG+>AD}O7Ndz)HOwTNSP;)2e9!G z2ogUhi~J}m*I&M3=L7^~1y7-Xr|SwnBLJTBGxv5dy2)A52zg|oxtj#6h@s%(zdEM9 zdaTaYoTgKo&bhhH12eg-?L~W3XEv-=Vhw@I7xyN>F(4Y4ZaR zjjhC(Tv)l8iGn*ZjkC!w!^r{D{ytVQq(Lsy_8J)qLZ@qoFWSc99!L`vho=RqQcZ{ts z51Meb*k~sO@LrO{fFa9C%^t^t$_IpZbLnVz)iE-&Os;;0$M?fT;Uw$|ES<9(zz*+y zv&c`p&RJ;NYfxkt@RaE}$|A3jfjPg%7I=bg-EV}phKr&}_9*OJcUNLdESlUQ{Du&@ zCE24$V;o~0dQS>R&{vRBSA-d$S`cm9`QIm%DGF>dt$;of%=U0PbsRgg@#dXmCVJn# zl~LAjWjYE~dC|V1Y}=F>R*fHT=UgdWG^St!K@ zrZ4pL_DyEwdq4hM;vr6vM(r53F7c#+&}Qt2+$A$HO%~FD zp*%G|?_#Y6PsgafTeVfD6S8k)#QnW*^M_K-$X-K(W_3EdLv{xtMn?DEw3OU!AnpqE ztX^JL*ojiqL2gC+sLAd`UU5zPmHNBE*+dZ#lU`E<|D_eLyvU8u(-q|WcQEOXifMh0 zw(da-4mT%5!)TF9F*l)uIMe)-qT zz=Z@uDzErliP0VTL^f+t=v{Yzbl?~hq3TZ_msAF=9~UjxI3S&UEZ<-62`8Q=gYUvv zuFu=%XG&uzD%EcsBwwa%XU_px>$S3mdp!sReU+s5+f?hT+IzMaCd_gfd-M zP>_t4YE{oy)NDv3SCg->w#m%zXQKxY2Yr9oCS>D0FPKHX$TdK0vF6xp8Yv9lfg<*w z-o2S~Uf)L3Z=hti;iaItbURnARU?lF&Q|=!93BV{U~6X4SJ@*9CD z9J!;14`NzL3Y)gtwDCW`1@I{V0=<#Ko$&+({6~r!)<4MnSaUy*7~H+NpXab1rhp$> zjehp_!ui2vZ~prw zzGuhn!ed~LR@u@d&O6f&90X!|r(5pW<-VP_k?6`ZwT26nF0yL0te}v}*@Q3~NFOkbk^eyr(H_T6qJUa`hbjI0&gSjeN~#0S*x4$= z3+Ia0+^#J~oX3d2~)j4^hpZ#s)ehWj>tXg_XNqatL0W2;<>9F+nDw z+IiY?*PdTcr~Yu@;-+vXFT!Rr%&f34o6aLA9!W%Q(ig#DJbBFTxjEhzC7sM_mM4ES z|M8`=6~oa=c(_Sjq{!!fFJHMxm9|i|F;@1;y{JUHR{9M(zHyQ8m&WG4C0Klk>zcB-`9>VZl%ZTwpXjWv<*C!r8@%5R zQ~h8${W)`ww{;qsTNasKbZ%CX)*wFhaj(XT$`qX%32^fVg=4-d=RK@nrCS95jmlt2b zr#A%iM41mf1KiQ{yL_1isC|+6l=W_HA|m>kGm9%LcuMt>=j%~Wev4r32Fh3~0*)6< zA7*Tu$V~H*ux-j;rjraTHLtsRn{muonk3=Avew;32!-etLMho1E9fo#383c{fkfn9 zp-R4$po{6RXA4Bwn-KzGyf!tdY2a7WE$myiud&T zveH)YL3;%9zF&i4gDu!`xD82uuZ~!u>6dEKzyy^+&(|Qw_2D#5pJhf)(?xbFuH5l1 z)u!{y7Xw4it1U)ZjS9}5F7kcKNv6`8^io_nMoKNt>Nnb0$S*1lSD`2|`;N`7)27~w z$O5sTtFE1;Z}I1`-s)gTDVpWU0GpMY39nY+4Ugk3B6bY#AZQqyPJf>yp9z0HshUB4mU)4|K2V_ zPM9ZIR{lGN)A#q1;eB_-S!H>l9@RusEbk1y@|?(N%21VN>;pnC2+Cf3C#VFd$oa|y zriaT#7xcp?x1Rx#Ze-#w?sZ%vp3Md9Hk7#WLnV_MYvq-jgknS|KQPh8Ob zl-;HDJKt*P#j~5GVhPS_tXIXDp9-1BpSKB#jWR2xjh4TPRe{bj%>^#qf7U-2WKgN6 zd_`>$GISjAOjhUY$85HE@!@U|9MCDM+!l4dhX3mt(@G27kL)r4nh{TV^D~cDn{)Ev z^fXmJ1koBwmtnFCf?nY6PIIWxMM4lU{3WP8=O~EiJAVk5_{06Cpiq zvQ1!sr|_lYVVu5@kbnnXZQw3+nc%ll?)EJ8=(`sIH|e}iZBa-A*;t)?8)j(P zG1xXX_;sVUdVO{Wb3CrrLvi!@s1B$O9v$I;L4=c5>J{II(n4ef{gByOs~Pq_48i}$ z*qg_**>&sVtyZhF)d{VksSdQpPN=!nK}${5mPzG#sS^^_4g*L;7!E>0VeH1X#;gr4Xl&YKs;3Y--A z#0d5|DWu0b{rj@-c$1tbPgU=t_Rt=KtXhpT?<+kK0AGCFAaK-6n|c^t?TXJ@E$xbt zez8Y$L5o0he|0^^=jRA*K}pw`-E&DS0Z{!W@f4d@4l~NK4I)osKKm~YJjwkDSrCa4 zo8g}^j(MCbEVlXu^ebD&C@r;nz;E`KN|VlaXZi5cp=vs<0^42(PHXHxS=tp{Umb{8 z77%{?Qo8j~obL90cKV@`M~}00!X$&^R%%BpTw;cMYW?0u$)j~KLA$c%1|+0r4~cJ} zh@eV>q^)l4#e2Pr7)!jv>bz@qyQO$$lZ~H!&crXyz5}(}#Y+9(0@e1z;zK-t^L6k~ zdbb)&8Qpy{ytm|If!g&rh-a48nP0+;m~zJ+;r*<2-2*fT|7J@Xe0;od7aOb5S8rNq zSL@qyRGedVx33z)%g3HpP>OKs>Dv;2OX~6q#P|3c*Pg_1u}`-SUJ;v=_&*U%>!~}! zW9a#WQu7XJYQq|JZ^ff&4^Tk%XC%XSQ*Q^l)*IZshm9%2--qc&lc}T4+Xjqz*f~|X zJC6wO6)t`==~{pkz3I7hq^v3Ae(2}jMdOR{mR|4hO4%0rV@9cS0&7c#cu>$Ld*1d# zXy$-Gulg%3beB484C?8op?G9ElBjGjr%9cjNd}+ZYA`VydXM}^Vqg?F z2JEp#(gDMPVnLq*05(wsEe}QI{QOseZ4x%R{e$ z!fx{Gbq>3RPhFkjmi%0>X-#x}JgaZK)*Drg)ogYZ%xX(@F7q25Xtfit7-d2t6Iab5lZMP!LnC(9BmVAO;?4yx7=&r zO}Dw~c{*NaHK8=^99i9Ow(VAI^zmRF1L|B*`tZQSlWaTU-E6kzWj$@A^)-90c8y6F zbx`5V3^HY?Q12{6uO#&@W?zW+O1xyUty1_nQ!lr0!y|lF8nlNT(M>4}o6nlk+N9&&v3+pP<*rjqM_j zA3YTM)Q#`;?&1wey`XZ3q%wuln#ypB27Ri5F1#r*P?{n~_@F0%L3b=o^}kGHlnK$s z-qtGrJ|9p9p%H2ot}w4?1Q5#|>s{JEK2;2mEDX>+eE4vrrAPEeLfZ0UvYo+S=t^dc z;(AbrKsJA1_Fgg)qn`DCBRy@$6J%oAvKOA?5yLM@6@5C$eTq}1Ar<|qqXg!=x<@l7 zpkE&Q-gOL|1zYp`eP(9&+D%__hnHuOoo|RcW{W^O7XgmmX@NHPW(RfM7 z%4s}>z@sIs(bv}FRbGSJs;9b~kB`P4+WUa6h4~#*@dEMTIi~iRx_7@NJuqsf?m*{Z zIRe5GCt2O-$!kOSH=zEYdpo4Fny*~2L8RPlLlcY!o&8Uf>K~(*9B_J{nbZ%LY>nF) zSl4q3(#DZjBXj3EO!&tWYVNA%=jNmj6kn*6jlFUbcgUEL4?Ly}O4W>?C|UJp{8%4qO}hDQ#I8;?R2~T*Yk{bEChD z!!LdKQ)dm2YV8za-*v`;gM#e~9S&>67%k|X0%a=&KTF8_QVjEFuCxBRLP2b>UzFkd z&BG6HcU0yBc)-}N*KeVpI5pWWN@!)e4<%gBsLRp6eH)~r4y&9vGoJA0mW=Ta0;dX< zn>73SSRXD{lpo1yr2#J<_|FE4L?h4B9)E`^@0*Cpw=v!8dRKq@-2EC2D0u%|nAZ}u zw)4Aue$3!{(A1U$Ap6uxorx<|e!9&^@{eNJ_WuvoyhDd_lwlDmkOXSqU0$X1{3iQ@ z4CnNp*^vID)ghwe`nT=Nc8Mu}gJ|nNdn1+)@4UrVu)KICbt(F9NbEoO@s#%Ne0ijB z8LIx1p7zK0`p*l-&+sj;9~iFkFO08qR{OjEb)C=J=+woF*EhxcOFyL&#Aeq1Py6&Q z%zdFZbFN45@mi)NJ}yQ5?N<}t8N|NaN}G1VBmTYp_;1tl+(cmAUHMpeSmcMy|Ni#h z8lQx3)6xk=O1NX;%EC~Gq`y15P`z&^_Td*c*}KnaO8;%0(0|@Un|8*_C;s~({r`XE zy$?^3_uUEjJKZ|}VMS72?eN=lI7zMlW9Qg^J!Jje^K6-o{8l91r{c_?+jRaPZ{p7o zzBeE4PN*%G)BjAx_FsB;lkbh*1o73iA(4OQ4gROx9;@$+Whg{ghXWn{)YJN(+BgoE z>HdZXQTOlvpcRtx^vxSz9qAjBDw0nV~?%K09Z#d^HMGb*dQ-m(lJh!`OLt% z91GvJ#is}UJ&Mau#^2sHyq8a-`;|0wW#tmWE~ z(@LQcF9`C3f|$B91zAHf29a+w_SpbzkE@34+xO5+{d#rd`CIzi(sV<8({170@zTG! za@OBoH|S9t|J>bm$af-FBEM<5_GOTI9@4K4kT=?R$X|N4Wp0BW5~^x_{m#g7AoS_U zlizQ^nbC9dzsAObj=tKv`|iOtxr2QvNUxqrU6-csfGhnYWLmxKly@KehC~E%5E)vt3As*9|k*YC(Zcxkmhe z`i=QP-`@fqrY8JCA|L7x2cJ8C{_IT(*8TBdP~SIl-So%RSh;nJXJKih0UzxEkxy@n z%*ii2m8S$E<=DDEWZIP!_T~JUO5i`1nBC#sF3@nSxL=u?`te&XKNHlxm?ahzP%2yu%^53gg=c=P9hscWx1iV@ zK`yM}dF+?JYyEmr9Qz~3j-Tq_afFFw(Ueh({qNPHvF;>Y_Fcfe?z(3Ea3va+j@BBS zJ=`XCSXw{+WD<|rs@qQUN7QBC9EHob*RfZAnur(cdt02?Jrf%K=JE|J>~HX=@{Hs% zNZPdcfy;c@_O9L{r!zyfeuYbHAtdgsmSxSu;s?OG;!`I?M9%3NXPno)zwW?`Mk;p4 z;zH(aUYz&T3Ij53&prA5J4y_RIQORZ=fvW|i07h=L4A9~iX5(GBEKEtd(2JW(o_fx z*3a!TrViJh(@`?!+N~C5XouO5&=@nea8S={WM`U6;*(C>yCqp5Oh{5c!g8!Q|84{y z|Cd{Dj@+9imL`f5LCS%{J=U!W=NHvVQaphVy}NUAhr8jZUui1lagyq%e*j+#@7o8Q zu%u~Uyvx?zj+ESDfrjdX6N7c|Ou+cj39)NoN}H-ZroX7S#+*Z%uRKg0L7K?r)ds(p zuc>L@P(ysp7<0Kip2h)Q07+&?>I(lYmp%?ZGhVp*WVXxxcTa0}f;FxyVU$M#)@84$ zT!%Lmk%G95*ib29*;gS$dJf_juOF1sd1j^!61xVS<56#AHI#i;zBRg=aM*b*Dd&;W zOZ%VC-5~8Bwsd+#F*cnU#O5!dxB~A7-i{3ZD@KzeG12mtr5}|7pM7XCO1q1K6VBd= zTK+1jzB_Wcl%ko+L>>&iC#sMNc}WTu68&^}wmL!?dn!g$@nhqQu>?^?NWzfZ>OpZv z+OW?}`QWhbL+-?~%wHMmH+jfvkC&iXbZZBH?GAMf&gCGxJgPl1!s)q{8m07wmTy3r8VG5{ihvS(hKhsoj=ofAgjFlkXv| zzZD%uAYOhu6UJ!=XHpYVK$4EPe)fCI%V`)qxxal|RMF~`asz`$`YxOHAR@#x$o1zLP2w(E9Q3Kq(|`+I|4;W!bhuO9H$U2Cn(zApc~pU%3|WYAxTn`jcVJtduAN$m9A~vcpXvP<4Z%#*?O#x$ZYSIt->s7A#o+q7Sdp5oTYJLER=)& zc)@%iW7F`a%NaFDY?_M8eDApjlSCa$t;rAABir-&8M|qQt zM`0{Q_3+7;>Ia1mV2aAr-*1YlD*jpm4}QH-w^n1@)YNnq))-l_lD{*~<51XBEL~0^ zUQ&J7EW9>g)8FMzwfD&5MRX~T$HJq!d9HsxK)uX+QRvyZwj?j6!_x7?@mt*!&8w}G zKaLztS6QJ_P_@|9=V3aQ&35u;^)FAEd^6ECN-moF83Rc^XKj!ZK+*68Z>(N+?~Okq z23u**<#h@gWRg%Wb!D$L^EXbGJwyNz$sd*M@5q>N+b(_YPB^r}5>BAkoo}-_x)uN6 z-~roJa>kg~dwp_!@aj)(*04Mea-C=oqpcPlkl#ptzD2$M`qNgO=`VusY_I9n>QOm_ zMd=4Hxw>`Bxv) zt>2yL-kg%MLq*biPlgnkSb&%n_IMq@W_df+Pf<^swNkrp-#tOVd$XdSBHcx1y?`I@ zn2iCa&lx6vJe{VSd#_JOzBQwEIA`O;a=q^ytf1_%kZFY2SoNXp5`P@xmDc+B3z^?< z6fs=(;<@;@HKXD>FRb`y0f)xH<^!z5J2@jgaG-6q_GmXe=@a7iTTy@LXu6)lL4lrO z<$;hnJ2{K0RBc9^sL;`}z6~oLNlPQ2kV)qn2Jc@|yf$T{e1&=eZYL79-gb1o)CxjQgkzx)BZ2F_5zTASnuV3F3xt{IIIcGZfpxZuN#2 zqQeT%QLjBkZp|LTFEbT$_)s99uV-6i)PMiRu^i6q$u@bXHjxX?c73uXr>O#483+LX zUQW1CaLp8kc&2}a6%yN1LE>km5Mf%hB#w-L% zDmS|ST$#>q*nO$_h+Tbx@-Q~bJeQn0ict}m`WlqNT?&SvOYf77j8QQ zabz}XHGu6e?Sq(QgbZAl1MYaoqsQS5yDKNsrMp$#N9{a{#UKrD4j-i12ioKNzYh(J z+Y>CDU*=_=b+2K598$8c_zsSwY*z*Xox@I7&Ew-Rb-yc4uGA&OT|z z^|^B{=J!+~6}9OjUaCVMu&fay-!Uj_)Fq`V>g-vGQ!gEko;v3I)=1_1al3CK666{& z_a|3FwsEt+u4stIKfaf|@}3cPz4xPER|Z|Z_ionM=C+B|T}tLymL^e3E4V_FDEG>B zNNvPbhZPKC_5O%CN}N=8F*zk`q8W^Vk0u&xs7pS`-q4sefz+iqgtXC+GFrHa4p4x; zWbhX#uke4=fB#e~U>w0bf-(m*b}3Bj9KHBXLahty3d`s}0I!tm z5|;}IT&W!-r1SVyQ^`=qsOoI%+uob=!&DwKK>ng8bRzfFNd>ST1U=NJk-!MBO?Qsh z?ivDhd3%~CYIboQdRab}cQ*V$p&(6L-G<>Zje@R|*_=byckJB#4<8YuN7lbjTh?G` zoAwyuWfU0O!^JBYvT0+Sy{vw76ZR|43@EI_n-dA^yvSWQ(OG%PVS_@OjkKS_(=Sa8 zt=nS*)YEX)<1fbEA(^oiw{8aoWBixavRC6#EpjQWsx7aoWV|XC*|{sD0@+hBty#%@ zuRomN*Y(uD)x^IUx%I`CWyR8v@POiVG)FT1XWSdQoqn_@MHDQMs? zB%AvVv>htcE37qls&L$d78&NxdcR-a9cM#0Qg}4p+ET*-D z$l0(^3CltG5;W~BkT#SosCTq&{o`^_b)`w#a=jzu(V%C)LwI=?+VA&a{W|wPP2K~IYWvni;IzZf}Y9Z6$txO!I527EjDu!D>$J$F)!*-X7Ei2cpKO#C(u$4m@ z>1x|jto}rk4OxmGUKFAMWX~#EN}K3?x!oIW@vz?k zGqaxS%ZHg)g5H>$7M*&swxB{qlVMXYns{URRgF16NHI)N@;4y^_AgRgQSwv1wyfIO ztZMYUe6?+86>9q0#_z=85mpsSX4E@3es0f}*Zl0vSs~*!L)~}o14?IIX{zQ{;?X22 z1f?7fO0Td6h~hudW!3tZ%BpoAKG9;8z?8Z*JS~vu*__sq1r@~b&%Ei6v z^IZp*@%s`FX8i`}@x~Tl%|COtvP*}&MTBkDF}htdf(y38^2Z{a z_E%H+0M%oqxlK@BL`IF^(5iJUi0giS30uY^U!T4}X=KmgvQrf7A5lC{2E@k3GO?jp zT?h`wFbgc-3h%91p8^ERQ-mipVJuq?_E`=mbt}T&b33&gAMeexh0v{xiyAa1vJKUN za{>nY!7ME%0B^aXo z3ZXG`m>wq5*xFK&1HgbS?G<1pt6}rhsgtm#ec>X&0gv!eONTdbT^=9CM&Gqy(f$%+^ZBxtHr&92?S zOK0WFQA3RHf_oupxM6WK1~t(FwymzUzT#S^xfDf`T^c#-%o=L7>{|%$bPxT43N2~t zU$2$5v~~2YGRhhmyf}#DPLNQ>=a|*j%cYk&7nqSx-5NuA&XycgnYU^861PQ>@_vC0 zLMB*Z7frs>)!QOp@eE?EK81hjKDcFXqf@p98~)A6Pk5mwnRL{}m27q`B#rVS>pY;m zHK>(IJ1%s4L1Dbdt*Z9(;?nx$O)|mx;SemvKD-tc4erlpe-Y;tfwzqIZW8WZ%u_Obsvf2ra^y30>bc~b|LfmI*d#e z&t&8rz*)Dla0q|fl6RPdF0D(QqfoefPUo3@^&;C4P*4~J zh#Mlh;=FmzlN~UbW_ZnYXH__)6E$`NN~^5qM`bT}%-4I8_YTp%MlmtY*ezov0K4J% zW89g);Lxmg-uVct<|gmB+};X6bwV(}b7v_ne=j94Eu6OKn!8!6+<@$4-r2u(sVrc| zDZ>#`R1VXw8G`(xGE!%-^tdqqKf!5E-I13%obV_Fae8Sz#=SPE?Osf) zq>ZWRvT1=Y9Km%SPozxm5`(~`eV?oO}i^s=hFfR|jjmtEE1m;-QZ z-1eqt+4y>Ef!-+iE3s@3DZ$6K=}LE8L*kS+C&Oo?mljf)-~O@mm{mjbRftcbzp z87naFO;)d6ef*HnxaO6Im&jr*Ee9mR+&;?zusJ2cGHZ|Mfb4g2zRj0@fiB`Z(SZr? zy3dY?Mx(7%z@c0auwaXbgi#Q|fUgFQu@_#Mtbl;n%0Zd6lb*`EAyw_F2Y`caZsFfl zM1$No@RK3UdSXCY>{qcCVYJvP@HS`o-E1h3WXR;Jv8isEZDdGX2!F7?dcYI1oP^x0 zg{@d6l__g&>lKpCToS``sMuKtP$NQAu%m>pOx_s)Ew>Zt4q1Wn7{5f1W9QL>5_=A! z4CsnK0u$XF^nr)*85^IWtqvI$sAPwC1Lj#}i$;95isL;x?<8vrZC!w!c&oWwx#qI{ zYZVTyk`Ri0)zmPDbxbz%3hY=&J4aG){x?yMDXT1a#O-egM% zN$OgJ69DV9D_gzhQIk!N&J+8xk=Q%5>0O5WXYf`~C|LO+b|LH=}p|acOqVwn+nU*KXy!g`^mwL1@&_}jG4Ew>1vou z&aE|_s$$Xb&1Q**Rl^FNBHK&vTt^VhE7HV6VYdK|wIg}R?E1+To#1&Z^EqZgCh-PyZ&nulj_q9CK$g4; zFACQ!%vm>s-h3~Y40FAKZn@MdIfYwDY(hgR2kt)Ig-X;JAg#((5$W&lbn#V-3pGf* zx=KpFHR@mSE*>-zf}kB=-pk3XX^gbyj}(^*!!v(FFQ-(_Tx2pE+0l12W!=>{YeSTo znb$Ov{EoEGHy|O)6SUk{?Auq7^xI+%{|#&_mn1 z95Uq?7^*)flhY)GC)46TiSEEN2#fHBgHzRkn*-P<#^s6dQ%JaWgy+a&1N_a$GI6%EsmEoyH14r-&Ulx7#XS zyF2aMv6-JzcHy84l962z3L6XfVq?RQc=0;n%fk^+F`D$HE)g0IR3a zL$65J69Ju$HE+!OFlV%re^v3 z(RfYNb>NRGhieuReXFX?yoIwM{YvstQc-8a$Y)Ya%h>PSQ)ho6i#t33Lylk*l~5?b zsX6p~TD4fc3ZqQ)0LoCONp7v|Ye1aQz!T^&cWTpD@Gj6Faq2fcd%GT_8rmz=FERFp z@3&8SL&IZbaU5#C?=B$)P!!4uK>74bw|g7Kb(Pa*X!B{QYe75bKwj#ko?0`B>4iVb`M=gg9LOOubvF z;P>;(grm+CdSgD4ThmZQ$@qpATUq&ceQsWoUK2J%yr&7QAzBzf1wfoGNv>EfC3fM$ zTNw3LT>8=E`F7$VZWdUiffsYsnQM-IiE7X|n7Iqwb;)*{UK0PQ4R6Ge$%p+9T05O%Fw)mYM@i4|EeNOAcLq)9wo!0C; zHSY(Axk!k!lxlIn#I^SEMveGb zB4B5_FQ{Ul^Nrq=0W5GIPhjPU&D|2&S4R~aF5OBIQ?iD))2Tg0sK!`$by7<-n=G)9T^VpBr2tA0Y>Ecr$ zVUc65n1j)r*>sN!6V7t)lfLgyvq60w-Oa2=rPJEGxyBXQ=9Fx1fvF)FRZUx%GsNj* zLhdWP`o~=cZ<)3owF2M1CEi}a&@j-S>Vq*|nNK*vBV%E+@QY%mZ3EUY@pb?+wLvn z?|C%}F}&?zBP^C{E_(D161I?UDX%Jd_ohyd`~cRcpWKeVRA5)T#2u}-;aW{|dNW%! z=&P=g!vj@Yd7DVU<~L<@z{snH3i%TAd4%4W6v~UsDAZlF z-11oyXW_Pqys>Xh>255u;PS|)+pIAzs`p9YA{hS8+|9RXMRr}xi{NoZr~kpYsZiKh z#q*nVsz>O2v~sK1{BlL)W0ZdbZImCOzzjnJc*%wNUrQVh!HKbf^_GQD{4a17qcj*VRzeib{%ljI>iD&|$UINb-A@9K%v9wu0sjnI2axLR%w$EZ| zKUt$u_b*qdDTuFphw1YBV-aS=54Lku>Iha&ueNkuuH zooR#y4-km(z*1`ORa)&Gef(i=*Q6|V=(aS{W&&~gYW>Ph`hv&FfCEv@6T@)6Yx_}G z2g<>u`U}GkR}k<30*U*!1Nn^CfK|EGQ_~qJU9&a-Vl&s$oGnFoEs5blKi2eZTKDpg zW%4{M1Ns2M*ZWEj+8JM#%t2z=Y1$66t;fcC4sjhE@mn57rL{AO*Bnp-yi8H*LRrN? zr#PNOOIR16B(Y+*>}b_62DaX5WKX*IDfs1hWMy+l59iph3}>j@FMc$qz#t%YIm~9B zmbXfQi&PE}X+R+ERc zqOhK+3u%Bd-oMh^ALf5+s_FFp{oaqi<|I zs&K&YYMr)={qO^ehSG1N-#UWG$t0l`p7 zs6h7N`AZ)?e%D!Lh@H&S4JCLwYj7P-RMIuhfv4_5SAv9LaFRD!B!zK2q-6d4Sju{s zW?hM+=mrg$s9Wrz2T9}RqNXYFMi%#wxKq*b z$Ek{43TTodsH}HQCOzF6?IEOiv$I+vdZvJ2N(is?*ZgUlpUZvSGqh08K{3$OKaclFSWny?cn ztlaa~U`kmia2-n24wO}85H8Kd*N7<=^|Rmw1wW_rjE_&O_kXr8jZxu$F%zTfv$-2^ zY!P}La(=F>Tb)on^`0qtX&Bx!R#ls+qEx^9NEgQ91>c_dqs}G%LC>gLnVF#)y?h%o zoUj758=I$@xoTPO$^$oc&%bOQLob2Ax-YslcdXv=wY%c7Jzi_`5w-SI+L)MZyF;tuCZk^;% z3Vw^?&$O|?0{IbxCeDNWVzs+qYo2OD1)`ciNtOt?Yph0t>IJ6e=3x1tHbP+XwiXk>fy~}>+R@@4T3#{F zg+Q)v4oHCpgn9)7RH&O23K?atGm|CLsnpD4+ztMNji{yEP09BY=vDyq z7d%>~CPtZE!&!-k^-lyaA~B@GlkQHR;QS)*&e!jAW-dZ%+EfE`n}iQw=gOEFA%!sx zr^1&T_YU6{11)t`3ZgD%RR%i%DZzF8dhieZ-QoIvEAX&cJ-}DbhE8mWoCgXu{L$$6 zYR~4|EieN?LmdpK&AH0Fs$w#`l2>AnoymPgfl4AiTP%4XL+@HyV~s-cTVe-zmL{f+U+ zj;o{PYRUJj7P-Mn0iz@@l4{A@UqW6DgUX>3^zdNdXk7NiMr94ZnHXv);v%rStae2B z+W3w!=^I3t9~q(%8dr2|T+zO6F!p%ED~|b6d|j>ISb$(L@rMmgclq0{2Aiu@pIEJ~ zHzqmEgPPcg{LtNdN6Jjs4yQwbcw3JZt)EP5@T5vr2GbVrBx|(@i@n60`J@~+Z^*5P zMDa4J{%p-1l7L14ya#<$kb zrPK&1)qFd8ULgEucmZ8e9TUhgn^#rL7wr`iRao9WR5RHwKY|Zun(M}|@#<-{o@KQy zkh_E>))*`sH{28OtWd*wKPnJx0sj)dHk~qJ<=Ge3-=oi-%|etlw7`VtId^BxNA#ZQ zbYycjU$p4VLv~|#miEj*2hvZ<#qu_r)1#q&Xjm68ebzu1l!X2P{-25R|F_fprmyIO z*AA>rhipVC8}}>*C$$i9+yhYXLtVII}mpp?(hAo^M!Ku5A>B~=+ zqVm{btu-G-#Lnm22M|Y?`QyF(o@QQA@!A7+-%s^n24q@ZB9BV&caBWT4a1MOh4TXW@WTf<-fQ9wpCQRGE==l zVVp&;Mr?hIjT^u-^O@6sd1ubk*=+)3rIq*5Ds7z;z*TbW(YHTNdD7jC@Z zc8zVKgw^qy+kBq;#DS3T4MUKd@Ph!#x0Sth1`!Y|h26Lna5TB;@(Q>}aO0$qDu>jq zDYAzg(~v8U(BvTD0F_;)o;?>Iig4ylU@h9NB09<<@=(Tc&^DVm^7%)Mq4F)yE}f_* zUUSZTkMRAV`Yv?da`0TYG>-<37u78j?&xyc;+x^&;oU`PNbt#)-pqg-^jC0$z!#k= z;xifp<@--u9<=0#4Aq@$sKI3vx2<~^DbfIAd=jV$1tgc`{bQ%kP0+T3P@$XtED$(p z)M*sHlBxa#?D(})KBKZ*%}aUdMFT$C>fH1l&gfe_QJe~J0|8Ow$BNrie!S@Nx=68L z{^S(NvkWCSd3ZpNt!to`?LRQo;6gW2w`TkuRxqE(Av&6eQ18_4b2&ZQ-ESSJUWs)HT}E9D;zWE+v#AtXg|x zV=LXVkK8Tnr6NFEa7Zj%&I-F;@)m@Gh5wQZ7)!IQCAnGx9_Nn-TI$M>$54Ftwj9VP z{mubh!3f@NpDYzH=VVL?wzP@)kUMY$_E&Nw{UMjG3uBrsH7NzZ3tGk)o_rpAx;uu? z<4qXa=w_~J53evv4}d(F`|;|L`Tt@kjD?CSNqE>(M7lP|pVbTq#XVZLoxgEV`?o0z zxy__eXR^89x2R>B@`}Bl)m!xi1K>)zky2*vxW$Cm3lXs|Z_Pe8fku$XIvD4r*-oE+ z7t;tN)l%dsHmWsEt){pJJfbs*Ml4(1j4$M92!?NyLE(MMh3zDB6Yb8od-SaiY4uaU z=2GyyMe_jz+e|lP1$>HhzZ9RH70kYnZn5Nop~!XQv}}CNsh+8y&I;PR1VMsz=!`xp z<4TuLUv_YdM)%1{QPRi2w6~=_Z>0$nYjI>qrFU}&7><8R^<69}d?bFrzZCOVag(BX zbTh7MTumk5*XS7CmJ($GS0j7L&UX8>ye0Rqt%#P3EqyGhyz;Pi;AyMo#h;?;P3W~Z zX*|*hBaQ*6wDfs!R>Q92y9}-?-WaoA&yw&0mh;Y;ZrS#c(+c)(@$+s#eH*sV>#-3o zvZqFl)Cgl12RVyVGHcLMT!`?OsSwskAVoOULalv90RS!~^TG?7q-XQ$AmHSp8Sv?Y zr9DG{+cznhep_WKK@Xk}#XE?Yl9|yhdjgRt=va;WO{8~v?7!p^ckp2qyLp8l<<5|{ zuy&2lS?(jdK;^|dRivKi-^|1-;g)<^CfhVz4OC$n%iB^0RJkEX!rzEcfwroHHu-S- z>6Z!hSmy5`9dvR6C^}b>P!f`V_15z8^JC^~m+kOV@>EX-+Y&Y&~?B_agOLI)o9V zp8huTUTsVSi}$%uRnqtP5AU2BzPrdS3Qi%|U-7*C&crsgzN^6WFIIsq)rX-4VLitR zf}ycBn7pn!=>iOwwEErLKpgf~XYuq?E3l;n~}c zYiYpngTKBk#WVBr)L$l6#V;AM_<8(e16z5Hv1bK;Rc-jM^<=m3LG{yhjtemOBg7m+ z8|PpAFGh;3rzV7z{KK}L+j2Nied-|3rtA*Bp7K1t#yu5 zAFvN6B$YKZ^lVLy!HC-QUwwcfX}HC+gdaHT!eqQz6w*fH)q>p!$aMlK!>+<8Zh}22 zC7D&2Fpg`~cP0+dDHx{M4^%AjZXaZm;_eKnKKQEX(cIM2Nzw7FYhFJ;$n>tA*4;Yx zdXZS>X9?nIBS%{Z8n44XuEvQS;yffoGO58RLmHcs}p8Z5%iwZ3C3Np z2SNH4N1nD)xa^ zFOGD$vm7(Asr z5fW+gL8-@v-F?rPjtf5sk~oi9Ne};<+aP;lU1zecs%-^g(Z;r8z-AedTl?}R*==N$ z@V;X~!YFX?D7U8Iv>(K#&PoU$87y2M{Dq#&(3M!IbB5WJ>zoWAs3C(YBJ z7F~%GI{Cur^dIKoaYW!~aBJ8$XM0IE`rwHLrld+cm|RCFwL!IC7#xWT}K4`@((c4+Eg6Vw;fq{2^O@rRdrp%T6geMWVUl%M5wA(j&Kjq?Wl0| z?Nq@isB(Hgn(O@^L6~U#^EDhxu1g4(P5?PmwMwOhzbI0aa8utR$!IJ*{X4F`SV#rD4Lr0bkzkt==}C@QBVT=uJz!Hr=JcQLs#;X4Y6ZU@P;bB;)umOSRm)&}6g)9p#rdb+Yt9&5C9 zYl4Mifpo*$@{IxFSDt8*MK_ZnMVlc{JO1)~nV#fL<`}mWmFgWJF#uVnZ;H~G5rH+s znMj>u*&)Mbl=#0G_^kgl{_8pS$6JfVtB3fkQIEV!j5o$DW7g5XLwbd=J-=EPUvV*ABbu!sUl)eu-Wz(U&ntT_zB0Q0RI8W`A@*k zzkbb$7u;0M_OmSXw9BO6n#N)W27(i!8d1rUa@|0ncY7o$O|X?{hL~CPaEkhYLLUIK zqfM=Iktd-!2co8S<91O>{ObR(9+n#0~AnX@7S9qwZcmb%GD*5t#YT-bwlI-1&6(g3Z2gC||3_AKv8D z^Xm_G!@cy2){WYS9CiPA?}(EEgU4{ZLHjt1{83>~sq-aoK3GZ5=xfaSXQK;8JemUWvPcl&$b%;d8#9KyRBv9BsP})c{a2UpIWE95nufCmHUpIS~u!$bF}>9 za3it=4s9}T#rpsG%H8`8ZOY6&{wn|Hmtyh5B^x0ug4M2nna_WI&fSFT@{N$lu05E@ zKbaE&#@**{$4>Ij;rs`L;lHYlz^Meuosna|GlZDC|2TlT_d6OP@qNEmaBmKTE6aa8ZT5^tw^k#@|w1o*0;~W0U@QBLB&?UEdM$nYe->CZf9g~<`EkkrHkbeS-C4t&cHpH}NQLn}=pP+`z%$F7r15IZV}?`18>!MP%)OS$KVMtHGrNKC|Btcv4yW?}y6!I{>- z#QK0*FtVlnl<>bAWTgCe=3Z1>An^*v-uvr6PYEb4B-TYWi2RRy`M{$N{szQwj@G|K z2G`#lxX7O$FKGdGDB*AMpG0+8vB@4+rg!G=fS3XR z;!-=K*Q-D8cmVcXH2j8f5X0;5{`${LntsC)9uu$R{)19o@3hu9we1!kO8<9h{bJPV z7dg3oWxlFV_3vW9{T5hLQ^Vi?xdi+(%KdTQ#lHq9X|1&17fEJ<9Dk2GBClzUAwG^u z#ODN8{|7z@cB(IsK!UL<2X*uh5k?9CAa^N+eeLh=ZZh}~09^h}pZ?B+Tl7E>^L@GD z+23u}h%F88%$crt><`ion&txpaQ`Sz^FGnPb2gL~Re;Z`EV~|hkb=%U#hR=p%;ZIT zJ`_qsTiX=MERx!~xr#s_3=tJXD=8Kuv|4YOp?rmcc9bIu`%FB}RSa7&gyNZFZk_U} zP1b42n!N}W=o=l|@$|{y_3oBldo5xHMQYI4{sHZediAWvpaR9CPx}Ei9A(B^Ub<$~h{}-6sYR3~$mAY?+|!GlJhi zlgJM^R$YqV`F8URMZMpnr%refUY?CGHg1Du+X%j)w@z^irEh3oQ)%Q_rG9D+QiieJ zCHdUxFFs4kAriSNRrl_iA27!Aoxgntb*|jBW1rpl4+^HW4Cd1pBPKz5NIy_II#~Hx z5SqzS%=)~xbs|E z>^1OBLCP9d3Js=IUICw_R-tGtf(&LxCZa0znr{ebj~*V|C^o!*Kws0)43khYT`hdf zrYLov+9?LmB_t#Yf5N2o5~X3ep13AuU6!DVCm4HR&i`5x$dCKJdcDJ|4N(F% zm>=|w$x5tY*5GkWZ91qlmIWtO*Flw8hot^5Up;)-j|&|F*HK7!NUo@PWPq^e@!6C{)6Az< zd^{9KOx2oDsV6}Q7qk)S>)lccER{YOQTxmPhU-*mqnYdbVunI5 zdbU!8GFSLJOLe7>S9m}YqR7kuXGJ@uu*S+kJ`x(T5Y~yfx(uc%$Al6WAJzERl zWtuo;Xr9um%bjBIW+KL|!-_fnhmVL!BDELco3|a- z>Ueg)F{g?CPAY;*KFc5A7OQ2El;d(!iy>^>h6YxIDFNXH@8Y!z5$Ovc6%$yCopu>l zZ~>CoOSa&56z9fR6<}HQXo>=i>5EWqQZ}0S1_UiQrbu~A%PPN6+;(|&8hazBFksRJ z@r4l9U`79xp5%_oIIquVaG%ue=30sH%~+6_74ik8=f5>QvX4JNPyoMoev8us+7DSf zy`O$#9D~3YP!LL4JFHW=ABsKSdrMydas3s3jP<+Vd9X3}ct2>E_*@AYJWhWcU(SCX ze26k%Vk`J{)%h^V)Y^;5gIzqtmg3o?ES<2f%U^^D#29IW!OzU7JZFYupiG`(qJ_cX zD(UD3quHq1wgyupf?I<386#N%U(wV*U=Ry9+2W29Ui(KLFDak55Kd8<?>fBso)_(&7}Nbwk!ME^h*~b6KCXL?*LCw=g7;=xBFlWC=x^h z7!u{})+ezKp3n2s*3g%L+JjF9fcVl3AQLYcY|P3{ZeFehdvk-@1#|IQ!-&-xUvhvx z?%I1{5DVn}fOSOH4e`<_0#8lCXs;*?C&X}EI2E*sjA7rfRwzxGTj%4L7VN{D@vU&+ z(mi}eN6>=4jjns=*RH|*Z~tKsH5Y(TQ@5Rsx<4$MxkAI|XOM>2YY9oE^adER(C2WO z_&|-o^o6v&HzW{Lq#A=3}_i*hsjV&LQp#JOJYI?soo9c%7m`}hJN-Bth~NvNd8 z%*B10SjbK9&1?>u6OAU^c+2fgF_8$h1?Dw);o=spX0kl<$^CiH4m3FbJVV}(qg5u>X_bcX4EjnzGz zbG@gQcL+N3fy{#*l>-DboeeY`Gfh!d&A#3dk!h=#+~nsL2kbTS5-X^VL&Mm%nJsOr5My(dJvaF6v%##HJ9HJR zf!%Y^nj8I(cz#TtODXT(`Bi^83j+zIu`5#|Jpe&8%$c9BN>)G{kA7P_BR_-j11q6| z5q_Xqq}l3+Vkz%hNvDr8*a8Bo=+H(w3uNFn1iJ}zH#3GMOT7uJH*jdO-lDcu3e-#X z53KvfStubD!6o(6>l^vW8tXo3D;m2q-mxeF`;Z9TQb1c`$w1@RPuVIW&RWCM6>_E` z^F6jke70sUOJ(p9GhFOSxUGu{_ftCpiL$-AmYT2Lx65q%3GXTO>YKGFN)b#`c)~w4 z0)#Tl|0w~Aa`S>PY?GVA-A1j2C9%WhLjl7VkC+*LA;iP*)dr7M(p`YziDsfE3vVE} zM}DIjI|jRb)|c;37(^(ls02Rx@%dn#!}us|dIdQ2+H2vys#>l^(&#dEx`uUYcR09F z{dc@nw(zQJ4wm|HSTv*)q^tau8;fjaTt|AAZTp}ytf65=UUO45GUlSa#!7ZK`-L$H z^1cq$q*moYfl@o*O0e0mW<6Q_iS{-Jq#!;Nkq?XoZ^Q_MN0O-}Dsht#B4P$AOMu~s2#-d?+9!nr9A|U#jhwA5lerz|J`^DM zM+p-4j-lc{!&gr5j^d2^oTDMVeUrg8>jjXc)=FO^OglzTGFd&x`!y=4$DjhQOjf@6 z1WcU_kh61v5*4s(O8E9ad{yN2&dyGt5cBKw=bHWUP-E2s8n5d8tnbXNZ^y%c<=Dr!KI#eJFyS)9t0>d z_+Uht?y+xKziNRRortLw1B)RSbWqXkw6&GbOCZyYS=yfB^HA~1!fJ2Jn-K3{2J=hZnT%nH72 zA46Y_SOA4if|=jr-x{cpv{>`eBiK^<>JcZG?|Fq*2T~!R=>EXp6R3+WRTb_ZSW^5% z$F?o_5mbiM(gl@SC*}+G2hdlhFX{5o$xyrnkPBevt#H|n@fDQT6l5TJue-uCiaLA(U7;*8v3^x&DHx3R zlw~2`BChr8cR5gtG7g zC>O~@RMw~u7{<1c1=vJ3TF9}_A?B#4qG4V3hERpR=^-Aas#cW=?)PL5}SE$l{;2KOSTa~yddxzJXW%AN4&z!?@G=f!{ZJP z(I|G)_mB{?vps5L$l893Q#8gg3LIiFvFl?b`}7O?I&T_YTL}|1&v>CXV$~vVuw<)N zgk9WDTAd4V|9&E|e-8FYnhEaO{K+Q9%PgYnH_inPv@4TsiU1~N+0?S#6WiQ{E`x&XD+$XIukyAK9IeE5Z8Um`f5yEa@mCyrMJ;{a_j*Se zrW{=RST!9@l;M14_{#uQ?R>`i9*U!V%mg#HUT=p19nSf5MbF?df6;xbq36v0Vx_fz zh&WOdsNyAVM}0oGKWnapo@eq#dcWr~kbt4RWz3jWU40o0M$_$eT)9{E3A0R)MyDTS zJyY9HKjzTrgu+3ZSY>Fd)9RVzpqhjqxNJcXlZT2TYTC4^8zC)n!Vfh5-l_Fc2GC)jD3X zn{q?fF832lec|{af?*v1>V*im@0!)v7MxCa$0mdcUa2*O?8G6Ou=<09*nXS`7S4o9 zAWi!T1#ZDh-DFF6 z@@xgxLj&VBB`5UQQ+~f!X%1537KCGxl!v1S3(S;-ERDc2=E4%D?b7Qzq>0T&(V&{m zc9KklU7J);4Lxi=sd(pUAy06FW!nU)Ye5O!HcN^4)?i8Bz1XQc;%ceQt2`Z!ZTjPY zlS@iz_#xDzCi_$EbW|WX_!loP87jsUHd#54X7f%w5w>IjVf&TupbHwbTixISGe|$- zUv1{jS+T85uh|)>QZBXy!#QFT_NLVk-@aG8?Nc&EMJ+px{(*xlv5s>JbMW5&hmBdY zg5Oy{m1E1xU@H7A5kxuPGgofx@AwS=UEJk55zPudz2(VqFwI+`p?Oq%cY$>DF(&W*CLB#1|`&FA5ob~98%1}b?PKklADhPw1ZO1XcU>2r_DVvdsZipwgRw!Kz zE~BT;Ql83u{2{>vhYb!#JO9XjS7u}7dB~9aV!rE$|euXz3e))232Y13jao~yLvErKP8yv zn|vAwqkkO?qa*3q7N!+;`1sO4>Ef<^X0eThkJ%R`JAa(lO-F<6i^jIADmcorL-3U& z6TD*x3e5WqDY+9@@TJAa)(`++uL1CS)KQMja-T-|+@BJ31OQ|U8=!3y$dj=@ZceJC z7p^^O45{IGh+`uuQn@z|L^ytm%JeG)3n0o#;c+`)t&xhkE30iEg!w@a5{k>=V#+~i z5ttN^SH9=6>oo(aF^BAm+*;*VFbrj$50bg9((PkFTh^6EjPTVntMSpc87O+^A>#d1 z?>>h_QOVieK|x>mdv4?!2!y`BfoNqaVt)*Sifwz>VhwYR2}M?O#?T!Igc3J$`rmf& z8-mq1fCk)BzX^F?y6jKKD4?|(Q;XOIjoI*9Pw}|jwB<+8YAa0G%TrW_Z}=7{8wa7X zI&nqfOmOa79Gex#Rml90+UEGed8Q{;SRziJfMXQ6abpALu*u62q#qnst_l^tAD}(D zRO?apV6rcA9K?!JG~jp0Jw$NSf%x$f6?FBI6ZcU8Al!o_MeClgC&?NE&x%z3Wx~Hn ze{<5Z{r)mJ*sX)fq0u+p8jQzQD3G7erZCgF}tFOB#bk77`f9UPdUT3Yi;ReGH1e@ao_gJXzib=T_)3=J9o;kLwah18V9f% z@VLymMBSLe-Nx}6&}(D_A{s;k34H--7Bt2?Vd4#ZEA6z0xlmDStvv=0sDql%UeLhU zirv_@!0@srvs9x=16%o_iSc}Qg);cFxMSirZRO4pYA2@ zKJa=nFhv{4Ay1vg|P!cW@!lFrmq_ zU$z0&GWp>}^p|H5OnSmGAS18t=r?mMx5HOemx}ls2d>_qn;N(2{{`lRs8MCEue3de z4l|Wkq1nJ*<66L8Gf)(yTz=xQ^oxkuJJ_)qM{UAdzcTs8@iRsfM|yJx^Krdf=MCzM zfwfmaM4Vtb8k|}pA^nxvqPd+dO z#^W??KQ)S%DiRTIQ$1G?$6?fmms-d#hmpXs$Y<t;iL6l#5h$vxYG@-p^`X`6WdSTtB{Vvy>)p$){|{`S;P+!uS% zxQ@yuJ{wmJU3=%8hiCPPk2ixjkctJ!upx@9jmk`7L9mw&H4dsov zuY=(+T9Y_P=pldk-&!s25ZQNczgD|53%mxEk@^=vb>JPyEYgZuB(Zht(E2bVwX!SY zzFuLXEp96|+k8FsMD1jXk8KLN6qE*Z()5fTlh(W^6Jao36lqnjW6ugjO?BIZ=sxOv zB*T+XA`#U!(D)>Df>i`YWkQR2GuYs3i0cl6(BVJ)#2XHvPI=XHuTN?pX@~qjGYNQu zv=CCx@IDxO32@*NI&$!**2l{A8fpc}ylSjKl1Mw8?)un&9)5esC89lnB5pf44s<@M z@K_rLLrewXkv>CWNvE#x$HvZ4f*x$#P-H&4w&)8{m2-e>=&V&%`Lj2~0_CDvRoq~& zjmm)lGsa)|dUdHp(;zK96yneP5t;_fRBT>-Be!2yBl*+LV+GCl(HQmP8Cv`^pfmXn zV0J8UyeZaLatvz&b8~Mq7%rytmQ74tFZSzGeh$%X75ogMj=dC`aH+`FnSxwf!QgF- zCWKrqwylz3<+HrMaV==jdL5KLALcH)fV0}$k*#eyuTRZ`GhU=?t0wgkkCxK0yd z3#Zj-SZC|J*`8_vs?mqxXMhCDY3OK;LG9CfXzXbNp^#1g!z!|uz&DWF`ZOhku2ICj zWx!6TO)4AINe$#c1fSl_;c;t_AScot0wD&F3*m6kjeJ%E6>4sad1&sA>j46QVulM) zILb`rKaj@uwb3#uiLoUKDhjg&Y6n%JIHtxS8HIOh5}B;oGsVqHvg&l#Lk{V(*DgBf zI&W_=J!?D31FPmCbX8S(%2K#(CGMf(K~nk6TA`D`$Th;3S$^B7grD+desJ=fw@Ep8 zT?OA_1-q6E*5ZLJ&)}>yHg1r@wMm^?@ADP==N5ba4F9@+_u+hk`xN#u*I0sCfDUwq z)Xmk_O6r;u=s^{mA%zuz@+nZEN8isK2i0-ekMPvW*|XQzB)iPe=+NYANydfFX}?4^ zcR9&_$51h0xvnmysm>g>4)fPZKN|-YHarKSg>%F^MzC(SDYG|qX1V9I8BoRslp6;C zkM%)?-|p(8g~XMT&%68Uhn4zi>cYtx7f9VH0uMmL)==+Ov%z$ABA6LmQu9>Ypftm{ z_h=EvDWSeFS~6A<44dBNN~GwH*f79dz4+~8Dib)%HqLlHga+C4;yGLd}7Qd>*<9-1=h3q zX?ihrTvLKDep}Ls`g|;JgF#S7OGYEyhn||LSf*1ByL}P{r|&e`SQWD<4BMz`+Lg?2 zkxcEC=1?RR-99MCNoLw@W~^w^<+YRh3ITu}$Zv;%I=B3K0-k_Ifi8Ywg1)ZhwX!78 zp^)@39VH|y7=($Tg$i>kj}$)Z%Pzp>Cg_?Jx82O|nr?iSsM=Rxod|>iK3b$K!XHy93lpM%z25DH5 zZtFrc7Pt)$c81>D(Ck2xhb8(aURUoQ%R0rMmS6zdu`7w~hzX|E3BrNa($P!?kbI*O zWwLsssQn1ao?6$n^0zm|j=k+4;);clR2*e=X zc1tVQ&~dWf1_<^^%qq)tl&!RPSU%y?bE85B)39&sg}#Tcx=-}o-Uk+k{4X))I)P^+ zmkAg2>rutp^wmt7uKNDA^^rt#W+(OvFc<&%$64K3W^8>3Wj*Iv!2!HD0{O#-#6N@K z#xYwpu3@UMIkR{99f}4^Ks#GoFG#qTi59bHY)*P?2UB9BR$qQ#UV5AJ`H=%O0SBse z*4i-oQ0v}2=iEPyl*w?cIJTpII>|mgA8rj|wP7z#zwE3^%_{`W#{b}$6MV&jO8voc z1!lS90$S13q5sNdQe)(MuQx`7^R@wf)uOf0;24@jTQL(N_c&A-%_?&u7zB z3X!B9qP~gEw)o&V<>4D0XuBU;6O*hzrz;vp*4XR1s%k1kpLHQQI{D8K;9R>m4N3Ay zY*~T$XirPRpCuVKt-$NRxKq>Z<^xT6MpiAYgaWn7q@dcjPw-%MCA-`{+jevKS45ro zxrJ#b4>9!dtNFT9ou!XuJ3*&-0*rw-X;O3VUig1!0nC!xY;h#BpjIZ&Q58~`^flO2 z{(`52O7bA@mP&KnuUiKd7OwJJ9HX*Dar?#boPOm*>%%Lf>!K4vUd!qZMli<&Lwa;g zBw>zOr5-65pdw=)REO2mAwYoAgoF-@woy$0ZSp5@tJ2Q#jWpClW3m ztj!*n;XCB-BM1!OXlh;$3Ho1;^6&2stNIPQ4=2IG+4eb?gOTj-*_k|Gc1A(2h=*gJ z0Dwn0NcgeeuYUMD7f3zb4{?9qVKN8ku zQh{Fk+K8b7GNl|Sd_IDBf zBtX%ZJ?$s1Hbp;tf7S4RaR11=L-q51Qg|MLKiGYMRV%sx_jf(br9}S5{WCWJ?jIi4 zaZdY_wEa@;J(8b9{&uMDVT(CPmGu;Am5Hvj*?0>G@$7O|rTn)rY4z@G+q zfkDRjvDndr6CnF({cTfE{GW?E-&5WvIs`)wR)GH? z;qY%Kc6joIbM9Yxs{o8{4=8j9ss7zq)qh`f+Qqr>uNU0|UTPcIYrm%YcS;}rzUa&= z=d%BB(cf>2jtrT) zZvNF~Zv!^#QRazYYVp97V#~l87_+8(%cG~vv3!9_txG>v+ zK_j)-%r{S+zwbx)`UC60@o)tmhk+4)mB6c#Cr{DR(0)!9aCCQf=il2}ovTWYw^()~ zizK$8=Sn}zX?3=r!Qt&!ul%=7vky!G7x&1Nx25?%uBI*9 zV$fRjT3wY&{rJB(ARVOc{G*c(uIg-O=N%tw{AYL8|63{hKff`A(}`#UH*uBz&o@Dx z{Q8k_?sbO>?Y}o&zTxl8YGrwvXKhydNg;esiGYSo#I@6%Vz__asKO=kn)4MY-#PEX z>sA-4&D#vR3rxLgE%Ym1ym^>%r(ZOgv5WuxRn2o}C11XWnuVM=7d?`Cd|ktW_K$sX zb$AuvB~4@wdNcE%yT89chX5f(I^_4NMxU(@!f`VSh2#&heWwNRHM~;l-4+iwH2c>> zs)21~4!sk2lGGDEBPozU(J?Y}Cy3FP{>BG}c zI%=vVGShmp9!t@{m$LPG@Yyzr%gMW;`6%L3w7-u$pR5;7nQ=qN7J3r+sHMg*q zX#aIazl7IHcAqx+Tb_EbVZhO5pwIa)-qnFL&U=N=JWcoZXkNyZ7oY^IYaBECHIGa- zPaivb`(tJ(r-)=U=bE=<0jLtI^OQc(=y8gymBOQ*#kph|T0x)FptH~zIuts?U* zUYgTAzz!Ra+ZOHDFTdrK@NI*XMpqtTiKWh+y`TYNOyYP!i51rUY}kU@ zi>ND)W^%$u6EN1T@O6drD!~J=-8tW3{yf8`ygm%4Bi$$_OE6C3+41j5EJ{Czo6&eWy0$I1lT;c^tnoSm_?Ow>?o?F9LlsRlB#AI5Su&ipQQ! zJbLTsR1UCoNCA(PF>J)z^k3J$qRALP+nFBdG}CvpnM~B-LGutke57m$8Dv^^T9AMx zk)!dzPR|P#9my;SshkG^YuK9KPk7vA)f=-So*MNJ$US{uw=~w8WlN%XyT5DW7K*=R z7RfyQrC)fXZ$37=Euk{Xd&11f@nsxZZ?5u7VClW{ADPE$sn?)qrF+nfT5CSLn+#|7 z2KD-DOhd}gosLo|uh&lll53>X>PKmO0Ac zsgbW^jXhbuA8VVqz5U}=AxE;70|BKAwYJ*)2_Ws-EXk}t)wd<`vc^NXm$Am(8^kY> z{3=JB&#Jq&I^A@R{oeh_Ov8$loora;@a&3a{G}3$j#%c29pQozY#BUc&2x`fY(%Ph zxp*O7$~ikpfpY*K_N6JOdIH>03w+sFG>TID=F$$67 zJ>YOp=vp(M?|029SO;A!j_j8BJ!L)QMMrF#dtOR~uqZUW%4VRjD-N7x4f+-*YIplH z;L(149l2No>pRv}<_PR)J6!y->jJdGRSWF{&qIub#tDerW8#c?7k{f|(X`oYsH%L4 zYG2zLe>(@2&+mKEx9sT{u)Ve?_nFFb*T$v~fgvx}hvaTet&`2*rv~YBdIn}B3Tqv? zZRD>p*!4?~c_VX-gPfnj3vh zmyk5qSzg>`{!L%0H!r-`{nWdDK(Z|ZQqC@h!pxJ`>Jg%SqT^;k>g_B{Un^m~bMMtZ ze=e}lb56+12?m_VgRW0aSz05?PSZWt{57UoxW&KfxN5+sy0iS+?)BRZR>%~@2x@=1 zN#?w+XH2+cQ}y-J=HRWph)sT}%gxfEj_K5c9YFK#K{{3jVWZU+_Kef4v-3V&L#hA3 z+=KI=lOM(03l#M&zKx?Gnba@|gPieo(ua(d!>;VA39iLTqmo1UEzruUopzb-d=F;m zDK_RupiL`Ifs?_<;e#JKe0ZBfB6wmekFrFvPVDlY&} z@mP~8&x7gK!q#2^p~%;2l_d3th9Qv?J0}#FEEjEJnpMZ@iH`*|Y2Q?%cBrc{F+$*6 zvndONU%^R%%6D91PoMcL-g8EoHis__EbpDvldBku_Srrs>yVcJl-}-!|Anze&0vi; zKbr#0ZnhA-=FeWZs+q))(CkCwD!3=y`;WHOGRDRE9?8fZkiV+*k27SnH@yN)(7(H*KRK{do+$q zZ%sgbdpEL!9;E<4*`gs@B@iLESmshw6WYNpk}{HTWc5)>WQG24OmDEd2e0PYQ3)DJ zJ7bynJ64ghixJJrf;H)1J%4`Ni+XrAaQ>uxcI<{Q$p879>b3`v>_nlhA;%Ec{`JJ3 z@+suk(fXCMw=d%aTZKI+zI#!;GVWhHd&z~7-|Z=2VIC&8=6iBQ3gnMn8NcppPlu77Dy&Qu8rU#cRUeJyo8I(2J8VXPN=wZh(X z91Y?WNzrayzRxoHJWXNaYh(9%yN{1qAvs|YMG1U25sDxz?i#~^|E5$SFiJqw8l0}{4Q2L-NjYP{`ujJX@^~d^% zmO+aYug7*(d9XUSt7c(SE<3ABKD>#2{97=LLy@XO$=_{?E#(J!uYgC9ZnRV1pimru6nz(n9HBO%h@p=FE$ z7v(;=X6ME5aDC)OFE?nl|8au-!1?MsM=Fgv=W-)YX=#}e2DbUH|7AITbT)JZU=&E^G>A3UAxWjn z*u9bIvWWZ<$kPHtsnfncq?_%{5|M^UXeg&e%q!Ce7;C&4@tr<@OwHGjkBt{qBR+^o z4EX(cO%G1Ejpq_g>gHtUbeEIpKIbuW(|J$PuE23HlGYltKDbV1mOmE~)ox_>o>5_s zM`tQ@soHb!0uC`YbqF+FlWP}z_!|7f3BKmqjV+k1*Uh9YH8jNTL+X`Xj;B+h%35`g z6YPl z+b2DzL6OBNt}`yxY@*=n0?bI3_>vb-=!*n1Kr|MNU&Qm#!a=Dq=cd<1b;&_3i98e= zU|8bU^6$mWMuF?Km@kTNsO}8ncYtoccF}9`>Vm|N1GiRIdwJEN-=hE|bFjEQ(Zz$F zTy~o!wSIk)k(DIq8yMyfV6*7t_}O>W4(d?Z7xCR@JRRZkuZpDs(FL(K^Cm{jIIUO) zG#?S#`Fe8i4u#8C&F^0dbAIY~qMPXQI^X}_-?I8kIh#vGnzWhg9%{IU*fE_CpkFqt zu8_dc^%6^MO_RCezHBayEhmo}$5x19TfMBGn7e1ei~>J37wFuVp7!0E@>R=z7@u#G z;u`{A_|TZp@2u3uKuIUr*ETNBfIZx~KZLtAJkQOpcz*n{)?N0e67eIy7O(DJTOZt0 zEVCJ}>DZg7_1OeAabJqqTUM)G%b2wb;$l)6ITYISi>>Ssz@*+l!4BF%Zo61MpOr4f zxPa{Xd%2L|00OcOkA<9MwKVkuK|MfB?o-`O(qWSloqEdjxh24AWFOZZ>gJw$(9+mW zcSg+nr_MjDMe~uf#rNcn*M$iUc#9g8cKWDEs}|B^8pan-+ffZvHx;jb#O;BbuH>@^ zGDfeSP*{3U*c$Hg-M;lv!Km{4s0jJ(2ke6vcM6_2hCu)&D5n&+2Zq&B`8*#O#ZO?0 z?`?av3RnN?D>C&N>&|hP*L6ab=vAtCuoX(oe9K8GPR*WNq@ZiAc&*Jj50p4&e4*8C zj;(X`iOO!v*IuT!C)#uMHiduQh_a06)h!KssqiW?vw*wMXRfE*ITu1#9bCRNxJCEBl6?4QNF3aB%9TQ5*zYhxO3r@ z(Jww%Py1|leIVdD){E@mzO9AV#NDgiSt?phMnc}Q_O2uZuV$H4SA0SJf>}#h=su`p zntg()3A?*>Vp5cTX~LaNDfwdU&iz5zp-ZlKxS5JAC3$+CLMP z2%H=UDEs>1s|n=$vuI~~a|RHn9v=u+v`R@PtULhDmW`@Vh4+?yZrV`s9qv!v?ET>b z8uFx|!b_dTTJu(oWK_8xb07oQMj2L349R_)o*54Z(&n@-euz#;1ShhtSB!{-0g{#R znb9L}#zDR3WWLZCc3+ft%A}1Wbja?M)`9N)lx@6MVN}>Z_Cd88B}!eJjd36H(Jp|I z43125l~|S3dae5+6DJ?g*)@z{C8966%Bc)GT-=_BY+6e7-6Q2g&j^{zS!!wexS*!A zUQ5^#8tzODRnD)qP5rQFmrN+PUKJ#ZeP1`K{7EPhZW&`V<9!_ImE5$`r3&mwf3I&b z^b(#+{=)fAsBG~uqE66lHz0_AkJX@F*nJgP-P;E2kM8}R5YEDgo-{vOS=UxH$d21d zph`-85i*RKA1cwsehCqO{iWijqUp;@v!1W^ZIxVM1ep^Z`(dkyu3Js|+sD3! z>#F;*v2+F-a#5oAU!+}ds3`fi3eR*9QEaC(ABjn74Lk`b*X_3!AB+`lCpQMsd&^DXX!-0G~-1dULB)eQ&iFDz-~O zD~+x_zkN>>v;rsJ%i-){6)gn({bqnw@Im>7*jw&6Eqpop@!M>f?veBzWiERf6MqGx7L&d1;v&@oY_*uiFzB@6jSI%E1=nGU( z*C`XQCSa+f84B24GdZg1Z6%ot&hV_;_#XbypYEw|11e76DbHTc;3#;Au#z`ze0&URO>;KtVfI9wq8(i?!5vtGkvsDz^604v#u7p zV`uZ-XK5)VinPcFmA~3X2Y=q~gl9iYnbF!JQr!5UKPI_UPq96Lc zcU)!)F;C6ZW!f`by9?8!u1&*%Vi}9x&bz7Gzx-NhkxntzG|NZSzekEG-3+3{cFSrF z8C%)iyLtV5BV+VKbk(*~9W~u|utnA#E`ybtFexy+8Gik;dW>6A;0MSQ-ed0`Mn5_a9P)$3v$sAbFXm>j36lv)9VfAQz&3+n@4FcGq0Cz zE;mbe&s_be;+BQpA%^a4M(p(PR8FKpgBecIu0Fa%IcvxMa0+H0Ose+idj8lfNzVRG zc$8PNoa0jFv2!y6map7Ib#U``$a%$PM#n%QUBrw=Gfn>Y6z}Oh z9xH-cdGT1j$}%L^n7`Sh^sykw70bwB6v`>gWU%x7?Cngi^;e5=qV_3i9)p~2xsVAi zG3$-U^t%hvZu!bVc;$W(t;|=aY>vO&Ui5v{^1J|q1RD*;Dj=A`PMK>5d0o7l%j5(U zhsw4|_NRF#)TGq%%B?U(?+w1hUH@>y;=TM_B^?YZM1FnpqRuA1L0XJJhHtHW_}x>< zqL+~@aK8P~dv#2=TN{O+eX2CkRv+j*t5#$BuGbiL;`*Lx?V85=`dqf5?J~0(!{sw_UD1e^5YMGJBfMLW>WIABehmB z&E{m52@Atj&Okk*jDfVuBQ3wIegu9|OjX!B5ii+AN{U!}nJjLLumKLOo?rZgvHPBU zTa{R9MakuJ47<85`($mNeARhntM4ZH8v-Q6>)}vr+2vhZ`nrCZr6DIiN196g8&ovut~^S(RBh+@<+Lp^Kd;ob zcj$@F_WG#zglP@$l-4%Zvj3_4<}cHbrNi+i@#bpyLvP7|qim|+S(AX)R3Sbx&ifQu zi}_MIS-5*cP~;Y^2g1p-#n3_NkjTI-6Jo)Ju+F8@tbxmkmD*m_a~tu%%4<{ak8T3y zHXM1c>0z9|mRqD$+mqHlJ(0my2Hwxg0W#3&GSekxrG#rhO+7aBwSWINkVn^rdvqe8 zYIS*C@5zg*J2A6s6=0#BYu4&n zM7r_Ywj|@5hKS4`k8r)7v|}36p25;ZBg54ujoXHOM=PM}Svy9Q>nSHhre-5WOpj2l zK81U^Py}DwCh7TK-ukh(p6R$aQ*d}bION>Ry5ywESS^R#(p0aw5w-@=~Vaf;~J&U)}BTgirfKi1PUPnbFJbjibJ~v`BWH+C!1O2~B@eaMisOJqvQG^l9nwxetdH*-cGP zv%%8sus%D|wXJ4Cce5SZrkeMX4+^XRYysRad^4x>c)C+07ZB&Y;Xg$z<4SZL88U$%%2lw3;e4 z%hOT5Q{z*}^EgThiPtN9Lqsr|yR+!iy~C z&nZYu&J}-1Qk8xdsQ*~id9Z9Mn44~HNrf@LbiM*s_;7nxBBXUa2Tw2l)8puK3w(%J zwh`5BiIn-Rrjv~zw*Bnx_Vi0BU;dFk^-X+e-I2|` zxt69?l&O;Q_&9T8e~Gz4oOkW=-bUj{N)dFArUoXdnC#~MCQ-u1$*}obN;UQWaQ4=5 zRdwCku#y4-N;fFof^-YgNOy-@q{~gG2q@j1N^iQm*_%dk(;d>%4d3#7@AKSxpL3q? ze!t((Kh|2hU2D!U=A2`Wab4HE8AYH?<@eAH&yZ2(LhUolv6)`80|#8B85RYoz!=S>k(+mf#ueB8I4rY( zZm0Qto;?E6zuSerg4D>G?_#|eHMNaIFz?L260Bm=O0eai3a`w|aGM~-R)IUT?AK{@dpK5}2>-Y9TK$6ze(+4f5aRD%(h3myh1M(W3WmcwasClC6s}mQ4d1g`|}i|8d;mSzB*Zr!*1=+tXQi z^bB|>SZn=WH2oq^W8l0|dDWO+)BS9FouNT?z8#BfX^v*O=gATh!4Bbs9QQX}Y(3Q$ zamk$*y{*4Y`2CfC5KVnEE0Fr)v#q?%T}SL%l4LpPlEVrMu}uIDm6%nE`H(t(T4Z>O z5`OJ-I+1X6W#wmZF2G55+#A<2Bt<%Wi!Jiw5%E&V{h^NtkmGw3<8|F+6YrQ%n;~u0 zL1$r}NBtgM z`w}9&%vHij-fMzR=~uc}FP>id4Dj|bv|=2hz04c7*Pm$sg{+d%*Pd?eCB%<^dXsX) z8gkHl?mawVJO6{ls6pN0mFKdUYQ<}`*1AtA`s;{s@AtDa>vvl!CyIEG9CL?`2Bf}(PJHHI$?{k zn&OO!RiMtYwwdAWXnyKx*amvq;v84$Qxt4k5uh2EQB|)iorJYF8{|a+*|+~?r%C{b zFj1)G!i6e#)hUe?bR&^!)M%{W(s1YQq52x;KhE@>b!V~!Oz$Sp+Tvs$Bu){)L;LE> zZi|KL}|Fsn|Ohd;lm z9Bs7OoOgrCbUfM3rF-!E%e5FFgUh=syK!;3nQ<&1LbH6LlQ&vn0;Jfwv+VaLEAxyR zgiQu@^DwOHXm5vo?yQDa_`-XW)QnFzBvmbK7rdn@Mb9DZH=Ax2V-=vDR*`1AQ;$$@ zM3k$XY20nz_YCt5;9B~m)!;ZL*-8RdAWUPKHA@s^+m^)&o4xJ8U~h;SyztsB_qoU! zKKxVyHea)0(F9 zD-$G<%l~@&P(wK|CYGdhM3=XWj<;pf$Dm3yZ`S|w zt?r<1wM2L)f92*(t)|FGgzgV4hZq+c%>_fqyW*mdKZffHTB?vk z2ZOwsBcDP1$`SUe&!fIA7$p<~&(fA3yd9gVSSxh7Cb}M@6m%V(rG9&rV)v`k%XxV& zi%^@(!EcuR`irV+o+7!$OjA?cgX7s)T zPhbSV(n!YY-bZz45*#RHf?_Hk z+oI=j+{u<#BfF0+c=d88oAKySnzFACpE+CX4q>#~gV$o}Cy5aSnHVE4m+!~14gnW6hYY<6sct>9}Y$g-VLN?Y1XA=d*5C;Rt{kZO03ypj~bI+D+(fEdxY0p!2Dnan{w4?RS48gfg-Aja9!>yg7>!g;8 zZRYw9%U=cgmoA0RA3wDk8?U_gUTE?u3Qfe7;6g%JbMrc=wjG;kLM+~KIor#lctn?Y zXq|(6^`_vEBnRpE-ZICt##|n6qM6zrOgCyBZ46S|%+|zUr3vs;I2|k? zCBQRwv{87qVZ-UdYBg30XXFixXyr}iPYceA@6shTXPXi~94vGW`#)GnVsHJ?I^8+) zDE-f|gYye_g5=5~_gfsY6xMcCy>o;2;35&oMdJEvJ|L{SKR;=`!DHNk7h`Ip$c^Oq zutm*)Q%izKd0RAp60#EW+7`_@-QPyD^GvFMR;9v_ef(&gatB)0q$dh3Z?F}um>EXX z@8B<6g!U23#OueYfb+8pKeQ}M^+pM|tCFx8HG+5E-F=;aT97!6l6w%G`#neVskWRP zb*;G_+*@$9$^Wo7Ge1ICbpV_#gf*S2W#p~HreF7jQz$XS<>Z53BY#s>uocQ+L&@E(_G?AUIvbrNo-P?L-|D zrCPH4^=K|Nwj0V9jq?GhRtIactveT68+D%W%*OTg1VhK{dZj{5NQohGDzK+>GbX<4 zg=I{|e53%76f*9Mldv1){*x-J%2#EcQQ7n$;gpwF@gIInjkpL74F4)w-CW`bQ+Ns< z=~+xR1yJ*D*f+Y`p#w;==u+;$GOhYixX!f`$_y8%)mC(PX#y$~mrezUq5!b|hP6aO-`;&)4yT#2c|9-7X3GwUvH zXS+Jy7MdNm+kGy*E8UY86}MPF;96K@LW%iTwKnri5kfbaR4wI%en*ewI=rrNiXxk0 zB=QW1Su_J~`5tx!!KzW^B89>kNH=Y|_w)%*e zu=GAJFxv>-8f}ooXy=nDR8YZiVvz1YJ5osD;eK0V{voD2Lq6JWpJsEeX>;NFCY=?B zcG>*lN{aKoqrXvF6pir$7Rd{$B7{NtG@?@Wf&gPKs6ys7t+TMlhLO^BeG9hwed9qB zYY-MC)`#8%9=oAA-@^qj;^#tjqwwiRl<+9>$$UEqhIixcOtqefY6P0IPMxveK=PI+ zZ>L0et3;h!Z34D4+2CRFIJk3X1A9k@fL)h?vgy7BOfqs%FqXHz?;5iG8J+XxAHDrL z2>`;Xp!9A3UQ4?{Q&2;W)62~dFyZr~P1jqh;ZFoIuW~3S2lUv&uB~K(?p)eDF<88< zl^V~;<)T7E%@s2aA*gZzXEw7v4ArWQndNN$Xd=F)29;9I2M$bS%PX}xO!pfFR&TYd zQEylZR`(TD_vjuvck%_VAA)- zA6w>{%}a&Oeo!ARA!2{3;%u>BsDJi=luEqU01-7`{sUh zf$9JNQswxxdP^mdZ|g)7%4Kiy@J<-cq>?$|+W2uf>YSJ63tcA+Rf@}7VB=c?7D=DYy)tN)@wcD^7d@4;eAeB zgQCXuqO&9eHvN-D8K?b^PC{v-o1*DBjp^;WcUM^CACB9_JzjdkVBzF3o9SLs^9DZ8 zt>R?)w!CJGk0#(of%j54wX1FcU=?S5L=f@4W>Hjj^WfQpdq&hhJ%rW=mVIxkQ?`mg;jD8DU{U8Zq4cl?d zhAr-t8A^)3jorw2v}`rPuytnj7v&$Hn&Srt+Z3(=$IL?7nr*6(4%MwDN*F&M2W#gG zy6kh7^TCS^HhWjtBcR#EYPI2bZ2GUA4;FGX`+bt9sz)r19N~xiQS*uNiqGbmN`ifx%?Yq0<0$hkN{_5<{NIV7Be6%8>;cVNasBNh?^hT z2BKSCI1bS=G820@S~p6n&RC=Ugp|LY)@rh7+qo`zeB3LNX0!{B`lV2Bts^Oc`jcf;knS?sJ0D~B`- zR42N`(s;mysaWO2&-DdOO&w((f_=y#Y^T_e>Yu>)5^CJX(mV(DoG2b8YYIv>tK5t` zQr=<0M!5<)xZj$}poXIEYx&j-6|y`#Qn~gN+@_8jv79;b@>`>zZ0$o+gn(1c`PWr$ zWA9$_EQ3Fa|7I#NgaSV?KLq4Y!HeVHajg;mm??g(&0>W#Pvj0sE&L@IN1qpmT*^|_ zquDpn@tXW?wjQB<(9LcjESlbKl>tzbY(MybW~>EBc>)G5Xjq zLWGEp+j8nyDEBJ3I1LRU{>KJ9eJYdp=IaXY=_!WZ>pv{@AHMGCQ{Z@@=lTP`69Im+ z@qhp2|DU$_a(D}WGnM+Mjh9LLRj!9#yh$_Y)chj>^LsMpZ`ZzB2nj%k1qWaLuZH`( z$8-NGYm84ZTjTz>q4~G}{9m`P#e7TBseQYP{(Hat&rc!+e$`{QcL{5P|EX6B0bgYv zk(n6wPk*4w{IJ1Yu~@j%`5*U{G(ti^NxJY?Q=b1Dn^;ti<_+bRYy8hm{hz$X5a7)1 z#Ni$P(+?A4qOsSRecf%xv`fVE<)3yy6OJ!rGt}_qOUmEu=o=$8X+52aCiJ%{Q-;g0 z(tL=ffdwP_UseU?V&%hrj^7&ag{}=-?TZxb<9}$X3Z0nN@ZV#-|KxjAIlyaDM!VcZ z|7oXx=6uNH{q1q=#6RvoIb2{k=X^Q#{tqAi$6q}G0qJQeVg|@x-BjAnyZtXMfInv? zs=wX(#nJ(*j2!Of?SE?b@57F&@7F5BsjT@Y4(E?g`ppi&bH+CSRV|io==vY`fjU|? zF3@f0v2p+SCvv!T3P3}s|1TOcgY^E8)aGkZ#6R^?)kI(sV0@ff`&Uo@tvmmA>-YYb zF=t-u{O;Dj`RpIt`~P~L{Q~e8H6lK{f84txz<6W+w!)kI`VXqx|HoVWKDdkidXd2Y z#fyGv@GA4jy=Dx81zX&d5fpzAaYz+9Hn!RmNj4CbLEOfEePtZ>GpRSlH zK$mgi4P+eDGfejU!EIIL@FU}|kpE^yDz6Yu=D5`6r#`33vv)uhDdOdBF;xwsDL!s@~-373hi_Ss8S(aNK!GK&!iRasLtE}YP3Rw zImUc75VHc<9YkKbaXj$*@Kw=iw0FJ#b8@uI7b)@*JqeR*xRFo2Dz)ffBI7qM%Q@|{ z-34k=9_!Gij}K@8hzMJ~<0gUU`-K)SXm*hRi&S$8Rqjl1ry8m)%+(ZsA;wo|gZqn;UV_?hkiYoj=BO5&lKTy6yTXl{>{YPH zg@|(UK-|C2HU)ma!oNU6W76OH2JfMUYmZR^2w*e*q6_IXTUM73F&JfioO8I|%aXe? zfT%UrsAzltyhG)@yFT=WCp}3Q3pyiz+tYmhX_)A_G}m^SRtB4afZN50bhg|C)7hH3 z)6HsxXl`=NCrg>CYufQK2n+liV^znJ+nmnonpU-T)1NuI zqH+WS{Eh66;r9-vvFFC|^$}=rZz1tWAO3fkX%E8vejhC8u@Tz!wvTaZ40gMz4-F;G zv8pXmZE{8HjpXomSgZv=r%5N*2dY&vDqB;u+enao2k@^cgP6*e zo#eE0*+c5^v>)u*;2+-&`(hcbj@nMoi02=0Se`I|_1t>6__dAF1vf)MI+;}(D4Dr}vn5$kGOrXR^|J-u-eb4-Kc#&Xaer`hDHt(h(tm5>EJvc-J*7(2)1TcX( zbrz8)1uWM)?Phmo(C5m1c;AKY&eZqoi|bl1XcG&&SMf`epU?ZWlBp+@EqWhg-pqSY z@Y*WoiqDF=qRzW*l1FQ-Du+>?Hr)WMC^K@bD$cek8;;>E6+uRap$RpjI0ydTeDwvx_2M*{Nr6d7b3 za=H_AxSF%*`%vIra{h|T+cr-7W3PkT^Q7ud`$_%Fqi(sp9l`#N0F1YL$1M0iPabv~YP@4!wg+YlDK;}zanHMhe~{g?8I%58M5I;czonnlBR|j{wuBe5 z)=v}*BU$yI6pXzEz!9d!!poCB?5pMR?lcyYMM*kFW$|JvB6l9-*syTfx_XdLWGcUt z+`S*7!f0s0*sl`V#x1#}!gT-Yl)T3tNG<{EW(Ythe`&EM=6oBQE^NocmVda2>}uq7 zf6JE^WA}#Ea3esLl<0X`%6a_pCY$_4gUhBIMT+Qhc{I_V0*H z9~o(h=P22#grkp~fm$*3b4b4njJ#H!@%G{ZzTP?j^Y9!m$w42DFMK`t6xS@)y)8x& z|49%wc|3Ixc1py#KMHodX?KW)Fg=ok5BnDCHlTaZ$6C9v{q2iKje3Apnmag5{A^%r zOtnf|rCtR;Zo0z!C74{)eZ0$IZ$7VfiTax!)~@J#ksp)!DxD4mO-vDolsTU@^z}q< zYT($1%1GAp4H}~F4>*G-7p~d`eC`l(^+QE4Vq?O!CQgAL+tsafA z3YRb_&qBRb-rI?5u`kGkGfx=e4xb6eFyZ{{$i!M>~ zigZaRS|(A?uvHDq-*Z`w=x_5{XU2DIy;-E>_w4HOIGO+6`4CF5a=tKg&C{DLmpqul zLuyMDa^XrL(^OTguvHPQa!tH1058^~CuBF|vMJVRj4!CLY23+94$t{OGD~zW^i}R7 zivV3&{R5mr_BSZAqFGlw`hwOIi4p^Ex<&#Kz(ux1z8zof7QLgj;&{sjL;+%|V;{|) zAoxWvzl&R{w7U`n+x>i>TIYiZg=RM|-n$=*jRzE_(2j>1Qzq0BS>niG#MkDFHbCTJ z!J{fxA9(4rJEg){gx=Y>PhpK0*z=h~^OZq=tqN#~b`RAW+J+KMp?h{Fxc}l93mwD| z57B_UyHg(Oul%+jjL+0}+F(Hy1LdsUF3_*8XxnqhZ)q21LF2(b`Y{FYS!a-}-OUZ2CL%!i?j zJrc$33`CaU5>I1koe0LHdqPexpG>}wlYv$+X5NcgxGTWdPOac6uJgd+6Pey@Tf-*k zT|Q~@rdUPT7V!%k41rc(HtZpKw;Mk!#K;&mhqryE)s_5dt zx#5iB+(EW2O~;%_$wA%Rj8D8Vc^67Ta2#{;RQ7DGE$_iIhMG*8fKR426HfibwfP;5 z$vOvFM55kn`FN~`AxCzfyELLudT1 z&B(;JCyVI=&NSs^b8$N37cKRc~US4%KclYqg{f z6+{cLNA%QJZVr$H530t0(mVG*%}O}Ua>)=;Rppg4?p#7K*erKLUl#u(c(RD`^{)Qf zZ8x*4D}w*8y_i-HzX`GrW0RLZBv6VN#BkB9Jz>sQEI-yVgqE2N`fk|ai4UZ0rl1H6kZW|wSS7bC}{n!3-<`R2PYuAzYr> z0bF(yqc`5U@diPwaTMU42{o&hNz21=l?>jJc{;UE_gv}$WQkrUBSFv=oNiLXG3DYA zy?k-reMHwZ0q~pj+C}&N1oJW`=rMkeU2NUbSB~TXqx3 z0nV8MJk0+0&4+>C1hnxJ)R1Eg=5EX0FF*gb;2=We9&$Ooy5iiEoV2m`31C?oi8YM$_t97*?Wy#r{R8_9Qq;Ra&t(kZc zpA3>u;F!R=KT#`ZQQJF5n@A|zs7iR1fwYrgzgM2yC!t6k(d z&XqbPks{VLxPCuTsNrZcM)%G;$je!S=wP;Sm^aV3d%oG+MdSSh=fKC)FbnM)(f5RS z^ZRb!%cbeRiKPg7gb$<%!q=dQO?ogK7WPd@#RzK{2`^{xu>>kspLvGYyY#^{R3|d@ zpHJP%Z+AhMSlloZ zkV%;Xy7zZ`l*5kK_tJbI68V@g zky&}vY;5r7GWG5>Iqd!76o=VF2k-v;L-`S_s>PYA=AnTC!+urz^{*s~51v85V=B#8r`hQ(Z~~Uk9)?csXV_4|m1>EHjTWnPUZ}Az zwGJshs#3`5F}yCHMt9AZ7-i|OYn zr7dT|ly|ew^Ge#4d(4jG;o!yjAg}Ecw5vtdM6+q|%O{g~+EuOhch@baMOL-82q3P2 z-x5f`uKfG`ERZRg+0hhv`qzM`eh@&*QcI;%@lKO4)h#Sw+}96r{UsO~9rC>AX*Eqd z%v3&8!MB1wF_ukjMqjtuvYH)ZADOoqbZX|51n@6{jkGa3(x^)$903V~M=Tu09v$;^ z*~U%Wk^EooH<#KGq2%U%ACbdMpHLQA!As0aI@XzH(2#jWF7)J#^!2lBbFwZ5NlDFn z@r?|+L^m~bLOj|6vT4goK4!itBM68_7&flg>#kPIQzC|#t@)i=@U=^Oyb#YgQO2mE zm5(QaRNJrHlcSFTk`RTE#o%a_dGQV$5=ytdm3`D}?rVp-JHI#_ylCY)9XX@e#WDx~ z4^>)ghpt0&{35{VZv^`vvxrzq`f|S&W|qm<)#$h#j*z50l8a}u(Dt=bXp%f20Q+Mm zEG8m`B(TKe_iJ64YDq)bo2xqe)O6!O%3NYw7?KU^iVZT$&*?B&Y>VBZUtJSNOBYM} z%jTP7YrX>l-WGW)co9dip7nOw2JR%HcJeFY$BYIIp_c*IMkh(M5UmoBq6`})+%$w6vpdZdQntzx_m?G>=zPmn zxe?RBXcySfyfGyg%38q4B=fHg)a9NiG=$4)pT{%nDbnrS%f+hFCD$16R<-T1ckR@b zJ8>ef-q%}wr+P_Xu8>k<8;ZM9u4}UeFU=U*qpf&1Xt^jw1`*=nx0%MtJCP)r%au4x z5j7HFmhU;mv!?PGpD*Ut@%S-a)cpvS&Pm33!1+I_Q-dS^p)4Qku*4`a`bBV=~_(4tI0Z~0Z4XY!&ZitA3iP8 zWSg^}Dr&INj{mcgc1fz(rua5FWCY05+P7K(22_5Wh0gN=suFQ-{qbzvl;v4HpTGqA z&m@X2iZtCAAepI;bJ}W-0dKIZGD#T!FziBW6hxI&Bh>QZ4v@HT4T{{nc|E*D)^9WlWwBnT|FM`k`a84 zRbI625DFZ0eQhH{1IP-Tec2yuYk1HP%|^dMK1Q>x*+bB(v3JuG6Q-@hZ;kK2n5ef-XAV++S<$Y4+GO)0;nA@63zAAl?9vr$&wm(yya#;0U(d@PZ z`6rWx*d36Q(IjYH{I;D-9K1EqkcEsB4mJ!kdOQt?Y+ZuyS3zNp2@VP&^Ch7V0 zO^-b%R$HF2DkX#&E?(|qqR>y|aCf<5yBeJDdHn!+mGKOkNKpR1QY>v#J#a#0@-7W; z*--m;B`cc)0fS(OH+WI?D#{zw%^4asI`0isV%@K)WdyuiZ1JLP zg>EQ>e%hgAV0c!q{xF>^+6dqBNv$fT#qvzN1b+C0rTRZGTwUtAz>G6G$_~lzpC!F{NfM)T<%HKAzt;e5v zR=Dw8dwTFdN28~nHrk`1U& zC#$EDx%vpY!7)LO@g(nB!E?aSnyhTH$Ns#=bTLrZANM7!_)L;BUhsYD2Wyff4g*#T z!Y-Xr9=qv|E_UoQvi^zZGj&v&>+G?zW6qczF)|=E%sKujg$V@KRh6@XaG>#x3 zSTXb*f%*qZ%X`#7EF#eoBUU=t9rIkVt$B*wIxuZb&bSlms&#kbLwb`Dl>Qd6Xr;JF zdo48H+C|UXI5N1i_Nm@wmbM@1>p4nWet*X1Ncz;%RE^CcazSJtgGT2#NBlnd?qK{k zp)fHp)evnF1#4}S{c(|h7@16RXn6s;EFaXEq@N2iD*!)*hM6p9tT2Bsz`Sa=wAZ6h( zEfc|tQMS^6GL<7YQO8l{V`RJ|=CPm*ixTn3ZT#oAi2DKf@=GvF zh>@Oys-Zzt5DtSumO>TJ!yS5YQQS^*vflyIM+`FhUQ zbZ@3vZt1FuQ4O@1-aJ}xe5z_-CZiW@WPS5WRCkAf#q*gfrFcjALa`M_HjpWAnHr{h zc9&sq6)S9)xvDSFvLrw{S#Go=2xB7#cdEBOyv-5sMn8KC?k?0o zM*>rwEe|idWbtQr?SX}MTrK;)uUuqPfV3)$Gwb;{L*UMP+`Ike1Cdp4l9D1;ApYlV zIn#|{opSAZnMt`EPY%{iX)J1J9$;m*t6Nky>tjkuyfll$JQ>2Sf@)tD{~uRxKHRSi z!DqIdAG(QWH(GEQ!u_r6=~pEQl(efRGCWEC_0*3`hN=?k5kO1rya&=mHu2VwS@wS+ z5V^)~ zGxRv7#E6EnZSma(V@{hZu2r1+Zw*`}g~4AjNhjrP!>G{$cHxWO5Xz=iyz2d^;y75?^iiFWZb zj31~MfU4_Utl&5Fl|+6195`$3qgZOlN^$X^dZHa zKf@&a8Uk$L2r7TbH;TPlEYz?9%erezg%X*xuJPFdl^Z-&MP;a##(B1}m%iC`$7rE_ z6R90Utggmn&Y;0n){HeF*6Jk5v8zkQK-*##X_ZayjI2U({*#%hSUL7IrCriq4!g}n zQm5Fj;>MZ43(8jY_1>j+|7;LM6RNQf)CI!gJdzwiNCa~b%~DRb(0=Cg_5S#N$FNNc zLiS1QCi98UT4bZ=;OTmH+~!Kv9d8a zo2<>*2G?f|9M2P=PQvV`8jRRE%gGa#?Ig8}?J5(1>|;*6uCVED=kpKOXEudG;S+B9 zL$z3Z=s$Y*=Nmgq+6^}@fU0q5$cG|!Wp#g(Y%{lszPM3sgOKy4%Xh0fSq-+KU&3nJ zTxw^%TlCC)J8rpS<0hdA1`Sbix{|D%oGtvgW`4bfZ;1Opx^Y(R6{nryj5;d z-X1p${C0&Mx3eT;>%#~!*ZW8vfX@ac-AF(4N#?S#RMfI;j_1X=KJ9E^+Zd}y3E(c0 zoUbtLM)!;3K|1Vz=DMgIyr@AEG>gN2@P^qg-^iM)&4*ckvA%K+NDjN*H=C%|tAY$# zE&By*s7>+Ba37eOnZ-mKydt%{J?Er=mm09g6{>T3T?Ij3K^yd1#F3{TYK@WH^K;yL znfYk4oAvb_WDp}qf5K8@kaAn_>!f}B&DpFSNol!1w4EiZoZaby(R(BvNWlrt6{d^0$!!#1 z*y0AMGNY@*@Vw4Q0fvJR?mn`7I)C9wIo_kmTbsHfn(1uxIUt3fl-Rpep^7hRc(q9n zK5(mK>JEO-V%RnEdREJ7+O&I><;LFMTBnj&ezxd}RPF^u`{Q7zM?6@b*EVxR6;S^P zxZw=S`Pa|AbAbA^8s!{K2Arto^0>PmB2;Py)Q~yfjUjNh;q$SiKqDyajy$&HV@A%D zGQH8@Sm%{#d=C0_`PY+d+Z2T8IW7zCvvm@!Dx!6rHZCC_SqI76S5ld{5M=9s*ic{7 zN~p(3Ktp`L<&{Du-Ycu2bVh!?|qV;!MVB_5nTh7P& zg6Aq;NFJeU@Q5E`bjKP*bK1_`e6KO}Kj0x%0;DnC@|?6)zr3URoPFpO= zsrPvYUr-+6x7u@Hfyn2=?7o+yw|jn674cAT)`4WI?(Vdsc!B3fpB={rv!3LrD3JpzBg5`+`kb)BXqTBxt_IhD!C%6CB)1vphwqR~M`2s~~`7w!a;U zY%ZeFXQ)i;NM*%{=X<^6Sx7dWw_n&PY$jXn*-{T5gbL^4drUNJAz}-^-*&&P&eBw! zyFFg#jZcyeFT_yS;A}wN&o^OxA3UGT(UVeQ~oX@>S9U)}7g1ahFu9xZRr)S1sXN<2Y6NhA9YpL_N z;ka=aNDg>;?*1tEEsr{Ea*J~i(0D*D;QHMFJXc>MYLwXWf$;G-d)CY{8|5yU6H7(| zxs<{dF1^x4r89mgRMMN> zxOY=#5lN>GQ)MO}@7iBd@VQbMXSjpH>Ya`6{d^Bv?5)%t%fspnLnB@s41V4TWq$&r z3ZfGFbn8XhdvyR3GTj#+@!Y-OXgw!N#<8^xaBopIV^aR*HVMZ^Kmi8ZIrRs4vGd9| z@q3wA&58x872??Mpz0E91my84KY~^*XUdAxzY;}DR1b2V6`4}wYvaE9l1#)A%%brw z0M~QerjYH02|jvl)Wf)z{MT_2%OVQm%a%HVL5&Ce)#)GcwB#o%*82*td&s3slq=u8 zFIu!mgCyrIyJPP zC9zm^D<-`JLI`6p>Ncs%g7UH0^Hf9V5vTX9{UZI8+kee3u+)=T^%WC?idwXyXtk{n z4_d$|m3p?bd&6rYSa4B8ognvwPkP_xY_4X9gqwJAXL*z!cNEFC+VyyMvbk}!<$Zr*lw$2<_+zch-ttxHU&8Qra`}){49edX zfPX`ns#5;~Lx(kY@ce}W6eFQgu9Gog1a`VCmrkHvTd4bX<0c6JRVMdM!t(hZxdB!{(IpNVM%vpgPq^Oo1y{v`rgZh}fQY%BBHE0Bh&cBT{k8uWhKmH? zKpkbEv2{-mUj1g+$SX81y9Is43=vjl%_4={us5a{DO?uM%P)_5fek~9w%Rs(Iy+B> z{#brgwtT|SVH1E&i3GI%E-5mBos!HUPB&4ak_weS?O9hWCX5r4Z=nJ~|#JpdF^OPC2p6s&!>skVG<$3J>@`4fm`F%5bUx zuKiqt-A|WLd>7Ipr*onT(Fiy{XafI)H%&(1c5%Hw!Apn2_2;W!g3_K>BQVF4DGnn6 z?Emh_C2R;6KEc2l1YqExm3XAPjOO$3(!##GJHlp~PwMWn2wAjQ=Y|qmViv*Z+G4GX zD(0!1_nQZc#{-@j4(~odKbpPDDew(O-pb^P!^%3x5%;C}^yy45!|R|WHFa(dhwY7M zKVfZZ<*>+g)9}JI^my8e^)E-eNP9X}?i93E-}Ekp+m6l0b5BV*($o3J_gDPRC;6%T zUNg0A>%IPPK=(yYeR7JEvoJI z`A@waf0tGFdMt=n&Z3l@D0u~n!2RC#6?NCBfNa^ky#9t;} zH2BJ0{O^qx86Om^0BuiNJK!BLS#>K$E71^5VHud1+__qiB@$9YomM-$C8$#(XygznzclWYR=$6Fk1E&QPJ#nS(pi@j{ z=%%H0yl8v2ReWiCjDfzkTJPjfu_@?x?q9rlF=CcHTT?oouOktRUfKiX+%V6jVPP>1 z<={7@=ezUNnm_zD_b<>^L7fe!6>i5iBhG92D$$b-aZR_SKJqFfHtb@bT)eGUR!6HW z)SNc7481gtxOoBXV~n>OPQ%GI5UrwUquN`hp3Av%HR(&otprZzUF|R71!nsbUvOck zB5a)JQ(oj36roA2W++%>4CE*J6IG&_!~0V;%?ccI5SCNtA{|KXbVBL*X~;04mEv|K zr_;vcFTo|$_ct(6xoV7aAx6etYiaErLi-V{7ot%)?M~gp+1E0=2KOjz@YfLqt+<>=^WODJ*2^;e^T z6j3F|;mw8=E9mm)DC)sx*c3wl7`@`wZJJ%*^Amf(YX;|h9l&S1{5p5@F7Ix#e7AxG zP|nTG``i5dg*H!Kvh}5E^LAO(9ZXl6%d4loA5-GUC1le#hCc3Vz1e0KI8#b&F^jO9 ztqFS@MwDfvU6R!i5x?bMlX@&1+p@wEamVL<HtTEimOC4f6IY zxV8MXulUz;CZvr-w0+s$KxvKn*IaS&2rs`3`QeLtw~A?$s9J{@x=~r=H&#h|v!vw% zLIYYR+W|V2RSIU*eB@KDMz;Axvuw?23P}PJ6wiTUBI5o0QDyVk(!v;&v)#PrC-WVs z-Are=sGHby8nQ!wgd8#UGDYUsefrt<3xKx+FO>&`1+pbzSh)?B$$9wV3u8>iyh;{q z8cbR$Pk%H#9(9uVke|DP{;3!TUvXrEUOl@qev)(W5jH9|S!8DrR+2N8)+=URWZhbF zQSZ6bJe$Uv76W1Tv*;_#ZF$&G+5pdI{kmK$ceXs?!jH-tgJEup-QjX^%(5xYHb|yS zk5`6IwqDY(Ju^htuZ@+?)4{)93li!8`2Ezo^*bo6ac zSr$`~KH+DsC*zbEd@AYgR~BEw!_TTYgoO=$VxweHhMJiQnMM{@hQ|l})T*}RZtQK6 zs4Gxu4t#1~{_t6*)2UHQu;fx`!&kGfZ>AZF5idRRL@qxWx`6pV{EV#E8q|37(f7wN zljGhk$_ejnu47K$N0F4guNDa<267q^9zh-&5NFo0jebJ*9GaO5&^>ze#X-4e2#cXVYi#&)beR!8(e6SW z<*E9!Bu?p{?CiB5Rjw<9gCmm9>QmgtB+^VNPxcLTfkX?}nH8oJCm}<69ttGCtFydr z-fjK$(#N6Id62p1K_i-_zu-V&hIdM%*hn{VI2Gj%T?g4nQ{slaH0`j9J>P>E>)DWMmvjULrJig#YluX(ZVGcEgR(}{hq%7RHl4a*SSf$$B!}ZDv9q7PFA!x^}h>$aKKu5^T^LSrASb$ zn(+F0QIidUGOL8LjU9ea`yL`XutLz`m|m}v2{{7HC z-)TY5V8UKdL?rDIr2VSrTxA(A0{q3u5h+AA`2g3qSG*9DT#uT==j{42jhVr%{c4aA zznt>qpds)(Rp>Ve`d($`F`?E>8`V#5l&}w~7-b1vVV)Nyw&uY;^_cJz-(J~Cq`N@w zyb|<4#?gocpZD?&2D4KxK<>+VrkL%_l)iNMjJP@@HJLsr_(A*TgYgOPUr=E2#8MO^NY#n2S4|DE+TWqkLW>Xecf|uEexQqk-hgH&Ht6 z=hMYz@71jRNgi}C>5BEw9kd;irp={OC{{j{)+~Zpt03@{Df z7(ABjsx{2xxgKrdLjWRK55gu*s=`&ll_gc}VPxtnaxI@+CbSXD4suMXe&Q)%u3?3X zpt-W0z#Tmi+8cpd1uU5(ubb8Cyt_QodBSm_yu2{!=A;6BH-%QoRHFUbuI%xE=W6z3 zl(91>RI{vfd&FK*hnVLF4E3k?yf9ROG$^{VPAn2sc9Msn=<;ZPKih)03Jp)3w^~+O zxUAIAijfMyEJF0H5M8)!jE7n$V)nHs6`#}itBb4!U(_#&Gn=9{Ltvic-DQhly<}2&lLrocEsrvv&Ewjv+9QGdH&}WEtP$bn{cmY9 zf1Uo_qA?b-4lbz|pVa-a8KXqX=RfVsl&Q z#I>BZz{RDxddWk1rl(S_8DehhPmk5r7^cnZwP+$4PGZr*R7s&**q@16`NM`6CaBvx zOIhCNV1Vlo6Ed`=aQkjfP(&bgL`$FT3ipClZ~@Lxt2~!pDdGaxkCe;16AfmHXslxq z`fM{J0Kw>5$aN(CRcVz!@X>RMQ9JM_oTVW^FT|{0D}rrFR_udz0E?U=XoePhZ*Oeb zBvgCk4d2Ty-rG9`3S=83u|Wr#Ht(ZwZ^j`)H@nquDSz18Q^uon2s8>OvX1=|F?MK&jsXX4p812p++)mG#rrK78h+Af3l!zS(_FwgWH&|xbyzo}& zGWJ(;)=9zbzZZdiYL5{Z*)sH?Ovq#?kS~Ak=o(rozc+d}qVfJIVQ0l;XsFbw!7&z- z)jJI5D+wl@DrJdqa+*#5v*7V7#)PF&$NiG9^#9y@_F3;XxW?&jc>2tDRsY|kD^LSB zVB8+j(ptd-B!n#b^HWw>LD5Z#(DAYPX)N3z+yOQ>!~I{?5YOJQk-qQgddu^wP;|G} zQk~}B^wc}wYy5}wD8)QOJZvtqrjT5GlAG?T)kK{rR21bv7 zSj1`hiDweWg7)Mq;;?6j#O|*FVR@fPa~tstDIHW1CSQ-M7ClBl%UaiAMIy>B@VTp! zagac9E@`@0Mo1Y}sdhhPl~8b0!>1q}1Z^=0)iju*bmb0Qc~2-{aNzB%)c{E=KUXz) zGvV9QVZ|Uq1KR6!N2r7`?&W$PX64J-TskLva6D1KIhHQmLx$+qyQCHh$9bCtiaUO8 ziz`@ua!72Q>gTELSyDc_orDfa=REKT(^G&iho;QWJgT%^@eB0x?0Lq=(&V~h+cImP zs^@i{%WMioaK7Rx@!-6%A9a4?QBrkARwU^Cq9-mt*yx<`TZR7}kiOmqEcV1oW^}T54BWJ4w3y zhl2VA*N~<**)%e#Guo~G20)-88@xjoH~v|p>574X{muXN z%qd;E-$F)n)|pHK4esam)b7!N32V!(^~%U>wKkt7Rh`8a8tSs~XE#RB9MA^r*n+OL zYH)a^4VqoyPgKPVQM^FR9bBeHaFlDcdgPn-_8oXaC?%}COn;t5?vWEWGWi;pLS;~^MJM^ESA)DGy;wtN8G+J< z?m7%iqVL;znB6D+eeUrdE5?I}2X`ufGs2}NX3IUzV%1X5=JAK?u@cFw>+X)L+fdC~ z0q9Dde$vK9Q0iphV?tiwINoNI*+Dd-ay6p`N|YHG7;s5|qvB4S)swkwJm*uW@CgWp z)%f$5rG}=I&0<2lZ5QVzBMJHh5l_(i_(&T?Zm6b=E}%+cuY)Z!PY7MT07D%%L=HAM zyke;m+JAqvN=0S44^?Z5(9t_Ox!)|9a;P6w30HHv4myJtD9|+zoKkM5gtn=r<#REx zQ*Lh;hV#yBEr!Y8G_-YgE@OT6jBxP$;dOO|82cTaM=c^aYQI9?8}+Zg$>07Br{C^~ z2`;H!3H(9iqS4(^AU32Iz=|RlGz=BH3M_aKgE*@+YGarw)hdRTrb<7VTA`EZ*FgHo zSf!eZLWv6Of;=NK=VE)pOzMIH#Un>41wHx!Jywb$v}svp9su*Jg&FuX`F0w`Di@Rd zc?9WnblvRh$mM;{pNgV$n^S*GDn@plE6tXURcSw)I-QE4AHe&s?l5s-ly1An}m-zwph?>^Yx~?sRMF;VF26uf@ zoc`BCY_d4M5?aU;n!>=hsX%b1^{=LBx0<@i=tWsAASv?^wveLOsNs_LNT!yWcsH;v z0}FCOZ?I_PDbLj=^p*>oe<#d0x>vHQw~IZ?M;)LX5_wFjjIpU$-Xu*ns`nWHGr^B< zxyG|2$~BVkuZ42b-*ot0KPoE5a>&VYtyWNQucXb_-G^uuyTp;tj8BkYZ9%oCapM*r zM!5`Q_d5!hFw9kT1-$NkGt{P=x8(YX6`L@khy3zn|6BO+W9FShs0&0T29E7)kGYS1 z6yUnIdRQe{VAIU@z+eYB=9u&zKZ{Lm9n;3@9rb6?*M8gW4JEDfj?h`qyOB zt5C}!EwL3Ii+SrDC)x@&S$%++H~zM{fAa3CTlCSa9772&vwC|qQXMMktk841F z$k8@6Nh854R}b#?)qu4FeyZ)B?_S~zC6K*q0m9T?N*;}9ahtTJ2R^~oVYJ$}W+P)W z&-kKyuy%noAsP-~r?e*91)p4_stguAl(;*Q{=^rT&LdGzu($eaiusG^%LVVt-9XHsqVpFz#{D@p%%@s&Lc=m~+ zUr$T?0gVrD7!Aq04?#Q_c;#&~Q5E&qgBSd24UovX?&34>Nr^}G!(}bWz+rPFTX22O zh}9na>bEV{4E%~b#Ici(gPXfvd4Bm&V6QMAcyhc*s^()68pW6(Zfz_XQdaSGT!Z#n zm>aI!jH=+kVR0P&9rx@8;|9yyo!Axz#m*hoH;E=7b~lmw^y%IVwgjYnalCT< z(F8?_uFs{|9^noUW@*c=s7NNOa`B-S*9VXl7f0^<<-C*>=lR9bO zfc0ptWUV{VXO0pMmh@5by-C7EaGjQe^hl_1Dfx9Tm(LdvCa^8GZ3Bh>{63ncs1>rK z-4_`4_ubX`|EQ~e2)?{CHdGBwX?MMxIv6jFE;PrJ@BfMNb=XqfOd%A&5RgkfT9?Pp z4-KUx9_5uMN^`&Nym?rRo~0nN(yy9vx>~{wt6Y4#E4W}K$uGK9jX`1Eg^q6Jzl7?z zNKN-2K39h0n7-?o!b~AMo!Q98cJXeeVj43qaP@hJbCR)l?5ZDaqmFBb9G<4meNGE| zQd`bLMqruVVd$5Eg~`B1nPSsn=Hpk+OfZn*#fXT!jtBD-J13%OZ0TpfRtS?| zH;l0VZ1OIk-)p*71B7v*&#>43d9e7;CgbfD>R3QZ=Z)p~A9h7o5*XCn)?E*g4;pQ4 zpbJa=x#GKTabNPQI38@6V{C{}rr`0@Fg7uwC2@kvXu_rLdDJ=xED#=-#QkyG|3gWn z_Cv$nPV5As`Tl!|)_;6we{{hcAZW45XC?(OuuLlKzwWYQ5M&&X8D#vgW{Cgmr{Cwi zi`jmf#549^`#a&i6j|Z!6w#pgF zm;d9F@00l!6wnuXERBv5{HZqtyuTv~Umy7Tg66dXFlTBU+336zzn(G_^8f+6*%x01 zuJuQdaf=#0E5Jqjp4RFy0GJ!RIK4U-5%=HUnQ2WvreYJ3O%-^Z?s@fb{K!>N|0wSD z&m0Q^jb_sb>$!$4*`nAFl`Yv40_*6f@Fl&n%}!t7%?quoDb}6GY*JD1JH>{dJagEc zwSP@7pn38fZvns1o=2q5A@w`SZ43guz%yzT9E>1)%=Kr`Owl z<8wA#5B$a2&q+9n8SDM%7~i z^suED_i&7E8WPsnp6>3SFlkghkL&vM?RBH-YNFN&6xEnJ^#y*EV4~;6?0~3?-7+Qo zGI_bv6_X@PHzDNX{8Z<4Eb!$GvR=&lY%bGy`6+#T_?Q$3Ex&GPU-hnkXWaugo_Dc9 z?YAe8rTbj-y01@2!hgSZ|F;Mw{4*WAA8T(suCB{$d0FHC-uage=-4ce%?q`2tFA7F zSgWosf9EC%dv;?CSf{_v?hTI@$0CZKI|ZM|ro~)~=rX+5yJ{w`0~~)l+WBkIzx__t zC60=E>>FI^voK^&g8BPHP5#-oO2lymb=;)jlnHIeAodSEs>rFUEg`j>HoeiYgxX@r zS&=_%->3bkio^qM`_Y`5uppqL;C}tq5YLu`!%z)P`;)-08J?_A(;*!wX~v149dK zzHGXKuHyLEGPGV#uTZVL&4OxwGHnsOo(zf~z^c}b)~vM}0<6l62MnXs^lCgn!h8mP zQ?7B64XOf<#|8#s-NYKv3AL&*$UfMYpAEz6eX`^T#g0h zc>N6$M6lY@()OFb(%Wr2(d@ZuPf+^}%305R>h?g-ZZ_HvBznjw1PlhDFU~M)886N` zD*`2d;=!!Z&y08XC%I$7PDD<;HBOKaKeinJ8*Ey`%ocVj* zAHQ-2oN-PH7ps-C+#b-4Mt+5k9XJe+jC=!PNyo?ycWjC>K}uSAN%nt}7C}^)zKG>_ z{<$traa+P9{OaUY5?x;Nc4fZ3&$mBFS-sZil04hDy8b-`WS@j_wr^|fo@hM33@%qh zMQ0jB_C=7HZP2ryxb96=-n5o~7(dx+;nXDVt+AY*Tc8_OdcilQFW}oDd~+D}c8!hZ zjY6IvrwsIF%;!0yRYN1Xy1M`24bYgdNEce;Z*Rw;lL+U~{rY`v+zfD4;Ah`Za2a2I z_p?n5{rmuhv+gHhR>9@CA)QK9)DN^Nb@Bx3!Vwe@LG3Z6Y*TFZQ%XUHsbPh>;p{;s z-WR{tX?%8;K#3!DV`#~z6JwH5q|6g}uSNGV< z`+9o@#Urlq%u60qzI{Je4+0fY@;U@O2AcCX9&6aZvr}&TI@I5K$zjaV1zaIyR{qGK z1@A3|7k#dHXv*ntXW`pmG?_d;{8m?u~_~!yxB7^i(s}Gw3g?X!c(FS5Nym22tge1Jm z6SQ!Uiq~#`*%wLv)pFYE?7cDu2`)}?Ascvm{WV=4hwV@GG$2)(T(@_;7pAhwvHIDA zjsmif3gIBPa6lL<5v>{z5iyvClV(DN-Z&sFV}4y9Ps)8ntQ$|a+71p7ypp-(E3kVj zFD{{;g4yQ0(@W}m*UvH&v$s@J?10obV1{(gG(?aspkovn%A8qqGhi{9lZa`h78Ys@ zm`F;{TPDVvZe#xL!SqL}kO%;0%(ntXt1gvz=XL^UJa*;95dA#`LW2&BQ)}1>#YUT` zw0NUTGFOse=7O*m6_n0O(TP2y6=ic_I&FLd@%g zMXxLua=tiiPyJGRyw`Ic{yo=k3ai@GBC$6Jes_pTgrVYH8Wt z(His_$1$o4AiI(-1&n&7fdR5Y?WXVfSTvy#pM?N^&QaQ(RX-^$vCL~dt`XFbpWPQ8 zBN0KGD3(I-)!PQxR+7WrL2RkCDskOzoA0!-fO^68P04=@vIe2n2E@ss(o#q>$f z32Je)x=5t0ra!?K^(WhLD0%2r7*uY1T!~9~an%dyiHG6W*Utwhj3j?|o@;-ZycW2R zrRFr;I5NVZo?g}u4{SeYk2@dGm8<+k;)H_NN)1@*vHHf-u;wCZb;kX`RD+&YJ1c+61i+zAME{<8k!;k` zkMM>+((Hh8`MKARCyWPc*d}BTCg_3cSg;`0^Y?Nh640cobOn734r1Sj45@CvZUn)F z5EM8QKyo7kqmQ~l@lmT7ZkO2aTZ#p!)7sQ-e7plYaP}c$&>&Yu`U7IGW7at< z3|T-3p=zYx*io+G#vVks7-R4*v1oalfTJQvY;JZT#{s0It~AzmxY_ zqq^ZJVVa+sQg*HU%yfxEYCd=)*{V=VY-FaR%%)CE{KFA`A8G4EO1YV?G`4y%NK9)| zCbSS-Yctu~0sx95Zbw;y2OEnO9g~tYkt5#rbshnyZ_b%yR&ezeyoS7HWsw4)n3eHxADBN2Qm^4#4T7QsccZD{jk5BA;1gUl-)5UF))E7!i{q&XSm^T)Q_a zCpfYipiTxnk#j+6yw@834eV0qCZ?vO`khd7TP9Y_rQdB_e0sMO%-SDbB(? zd^E8*W;*w%>SS9$SBJ0CgDn6k)BUiAI9DtjVE+ZyyL4nhmGW2!fG6#hH^m`{)=t=RMjnr@KfW zf;@)_0ThT~8dfp(*UxJc9tghf=D~3G#+-=~eYVpuBSMsI)C^i`GqrzAw3y=tvtxSv z8E~lzYd;gcz}^HAO_xlYKWv_=7Sd(}rD4s$Qed~|v=>^iPsd>deX{*#>CA7No1L+| z+MH+l#{Ttm{Hw!t{Yd%e^tW#)_^Rq3%7^cz%yJMUxwa~ZmSrp&j^l#r#n!_Mpsvkg zqEH#FvH!oZAu=*iD7u)=Wq9Ij&}Nru|?55^u=^BjLBArds=>k zEuhIcOwTm<^Z{iQ*DzZPi-G&Lz1dhyY?2`wx!X^TE@W=LnfdJEqqi16pQ8}9v3T>l z-lt&tJy0Gk)$Z{ZrQEqNwx5}#Xz#Td%hv3%pG8<<7~_7GhmQ1l%PA@>YF1pD4Ltx$ zZkjPJtO;PiY{c%P^Ala#Q!VRY#SDbD&9;fb*_l|NYhIRZJi6W}_u})1<@O}+;~E0c z2tvsy%RiDnA+M9^Qq<~NeHefq9If99*=v)Brg{zOTz~eUUcE@>L@t~HhnqE<5Stl- zXPriAkZK{I)>5vtg4pV*(1TaI?GY5HDN=#LcI^w3#7j*j;t%Q*#D+8iu+kMF>TX2% zGj;)(>#?iWLoC{o%`h8`~T1yQ~)XTX1b z%MAfc4}x+xHw)D@0|w?6E)_62otQ74p}lk?5B@3ip5yc4Px)Unwr;R@K8QGG|I=(f zZ<_)`PvH_HDpZZh=Rm4uD{(z?e|oeE^@vZnufxI+;e~tr0;goi4GSJ)CdEgl9UIuq zpEvXTiMP=N-@9TMW3&pH4y&`HchM1Tm|BbL?G6e0Ky&CtitgIg-cBQ=1MeRF#4K4y zCYifhe9#MLCf{Dap`G|zmT2_qX$e)}+Zj}CMq)(vOo9%t9vqxH?DHE zq?|T=C>iF5Y?L1*9Q7poo~@DWOM?)4VGX(5kcW zid5NV?PyRNt-y=NnQk=CpSrb{4NJhKJX2nAyy^6z?JJzm%FbFy_B4=oC4)~D90>c%1`sw6by_?9E+H8B=LLIGF|eZ*{4ps;xubDOHn*j z*-2c^ui{|`>vxTqiM>Y{Z=T^?rQU0;E(3g#2kb^Dl4pu)nY2zK9^DTEheUU?g$y*L z&+Lp*hDA?}C?!e7dSC@vjkf(3BJ!i?eAZWNjyZuTItbVelwi?lN_|2YBO;BgFL%FJ zotN2o9Xyn^nGnS%_v01~yfpgKL%^0owvbx4Sv|(#ibPv`m!8|`9c+UARfpfD2p!Hn zYM3iT7xl{Ft{OzXmu)?1#5~yL!D6y| zW~hfq&UL5ov1Cs9%joJB(2e)eg5%f#0B+~VI#IAc>F``ks{NhXYh6!cW|Mi3FwFqN zovKJ@VjOy)&*`2dCP@E*H~e)tcC*OK4~L4){v%|Ur&MgtblomaPH`z}I7@Ns#_W_h z6awl+ESC>rFil6&_7YudZ!%efJz6ocBMP24>#WS#EV}izKqOl=R;Eb-Q^4B;#~HsF zJwhplTjyPcw{lG7zAb?RL)VZn=L-!93v>IBilO+$lar`!b(i!>R4Y({sHZzZ*5F_s zL}~k4+$;#`Mrw~ zvX|)D?MDeyWOzXr@hrNEdHuf}&4#S2aXM&-n!7|2Uhgj2omasGByp&|rF&mYfGSH$ znCMJL++jNDeOt#{lP@O7?#(Z~%d8RBY~?q&c9^;-u5Ph8XzCPpL(vM@6?tHpMv$|B zPxjx4rd;xbY&580uw(mT8M1XcmuWT*>Kyx?d21lbJGVVr#cZ;v*+et^?yQX2MfBIt zy%+55;Hmb?dmS{sT7i$fDg?TuXv`+d(d2lm|avOZoX1&JA zEkTU`qt3lG9(=l&y@v?!hPExPnOI!feSHZ`Vx_gVDN)}rFfB3SU$~r|--%kRO$sAC zk4g7*NNOou8Lm{n^N+J}ctyFroRj?Al9yjrhkd?`h z+7kDYXBTW5l@HNHP>@ng2KrTJsQM$`n)>-v}&8r&w>xxBu8Lu6*5vQxIDDJ^#W z`zbzMMpF@~O7(oS3t;nRfu%emj9e%^?W8`Kzcz%kIGusCgtZoj(!OCHQLSPmMSF2PpCLm$*}{HW6L2 zztyihj|6tnqJ406gZlY5i3}y9p4kK_tdn5(l@-_#7I$i9OGK|{Q|I|g(p?=Z-C%U9fb>+c2u_K`@B=kxb%fjQ8y zh(bnU6}HE@C?K1UtB=EWp}%GR)+c-}%Ahr!$6NFObai(4GQk=ke2!^aUp+1<#W0A9 zNk3fwLQthFcAuD75BbQYZW!h5TFpV($dH%GDzaH?kd3!ZvI;WD?34Pb*UW9RU>vYS zDcCGUut$dby@W(9!|!bisKWvPIyUU4)FcJ(Pn@!>>#*q#%-%hm)ePUSr|I0QlV6I_ zB3V%+A8Iw22XPXjGcZ-B*ILUCPd~ylgqMDlp^gR;c+E zv?S387+=n>ZU{1uu*1?2)folK?@;*vxc$7Ic;c9NTaN#z#^_!;ysmpdx^q34P>;P* zGjGuzY@S|n0}LeKJC$Ux>?D#Yoz|H-KfkbY-(vz@V#7t#=M?IR@4dHdV}_?J{JVD2 zE`0AX^6Ih~)?)Z~5$GEn`lFuo+IeP|SjM*f;h1o~yKvO^ll-DHqU@E}73cFk?xi4S z87tNDc*)+VuLDYX{SqSx@ZqYsMNy}MGa)f<8_9~nB>T_bfy8D*zPl*Wtn-}ZcdoG; z*H;Iq)1efry*cNfnMRUZIcXmNZ8ibW8Hy^1Szi}2U&Tv4z)eB0>TnruT4TRx(rPuv zQaLs+OEXtf89u)#E=SAsS$W1U-L*or*~XIE95(<@PDn887SP6e-wo)I4{UBKjrt;R zpHV3Fh15z$QYy6B42FdEGp3}Z(9YL`zW!cfWp!S}70Z_aE1sB|NkPxCV%3E|mUB;s zsOBq2J8Io8uct>l`ZY+nO7uR#lkmuU?(8e8UcqGp_I%sMo@4R zi9x`xni&hfYsCz|*WK~*HbB6s-}PdRhSJw4S6~NvIpsRq=xF*_#oxcF3U-hbeETcGefEdK?EFmr_Wd@?C^V2tS_+9kvW!&WlY!zsk4nN_u z>fExRjv#a1)30)m$+Qx@qKt1uQn&WzFuOY&W_h+|vYQ+0i-Cc(e5@HE#t)*q$%nhu z8zgQ~#DqkHwmMFg2B@GA@NgE~Ad>NQTOQVZ(!zWGH7vR6lzvR$6hUnuT?DR z%xO^!A&I2n<|n!vJ{8^O?938YM3D>I)n4(23%I6mw+M9_tx*Z&;5G4bpSF}})(!?_ zxyJXwbdspr)Y64aS328T!7s!Dtph+OW~{?W%*oD?4J8Nh&z9CC@1BPkGg_F1s>jDhh#M^;N3F;bvWCfKNE#Qy}sQdUJj7OUt2aB_& zTq-VB!&v{yUk2=)2bc7Gbo?R+haF)~ZNAQslx2+477rxpm-jx*(+#+7`YV`anQhmC zZ(Mfis*o_|i)4z%bFP4w6&U~8=5L!H8;|tQ8(y7Hfvy$Zeg7zR?eISX#`-tNy!DL~ zZc)}wfmJ;ULW_z$Jy6>g7N=#oe0p(wl@>I*#cdgLNEK%VduN6W=bf8?sQKqJM;$~! zCp8ybt%EpIKL*F4V1rtmes23>F`ee|Z!;`m_7#m7*QZG2<}anFaznse(YGUg1a)TA zm|*~-Dd?ah!s%F9v=;34R&QItkf-2tiWXuJS^x$)W{OExYk*b@>63zN$?`N)HTu1q zv$aNvE1OJ8GeF4UmHlo4 z$(|);_oKh!iM|xQ1Dc2~#D$j?dD$gSj2k%Cj-LK6sTYM8$S0k*@J zMvdC=yF~YH=Xg)^0m|&9_O2zioQ)vfAhITGj1L{nEYKRzL=?WJKY`d5UraBv3@xRplmFmJjcQYYL^(+S4Pz&XO45tq{ zjN>$H*n;#oMu=n0Phm}x2IvG__IxJ+fT^uu;W9nY1@>Tx>Rl3=d3#|XRGbN5x?{5E z@^{_+BR;M(>XsT(J$C>0rkAE|CNh+WU+;TrWWom|PJq$FgyUi9$H_-5XrA84^f+J*-O>={Uzsa7UxQp>F;yFU&zY`HfmQ zxuD|QPD;RarHErW?aw>AH6FGgNFe@qb~dwj$AQ7Zb>GbPPj*!^k71?4$&Ied8SggI zw0{QjHoGkBr~jT&xrp^VozUF@!mBNw2)eK4tc2U`a&y#hv~1@KBI5>p_R7GM8zZLP zJ)KlF=!FmBSq^jVG_0U>D*Kk(`~)vJv&ev0VClBYGAE2JU|?G3{nU8eRt}2Dan%fq z_}h^H3yg>!{-d3p5Y<|-R}ZHdkt-NfCtXKb{j z_#=vNM?H7SqW*CV1Z`D&SZO_!1%3c-YSoTsxX_-AuH~w?s*vcgLgKKt zTX*ksJNxW@;Mze6+QO z2De5&b6dBz1~NkHrJB%Sy3??1Tr&;LHA(4ys~7S`(ZLj%(V;DY_Fs2@boBjwkh$X6MAZZcrx5 zi_0Zlh&-aFO()fP0+UE@cebsI>EdwD&hzt!FG7CNJ+{f`yV3+rsGDn_o4)ldIq6Tz ztdJDNwFh`O)GTzbF7QVvqR64Rk4nw=%h+d6MZB6i+ljqpNv*w)96uz=`1X1vcDgon zEUDY8e(_e(fpLUv&d^Ti+32jatkqY)zsz(DUno82N^z{{e~Iy_0ufK);sWlq2`Q>b?K)F-P!kEBYPu zoNpjZm5%mlnwKX{eoFon$S4Vz`wV8Qryu*b+goy!mEPdsFj5`&dPLU3>|R`%wz!8p~Z zdqjG1M!1crFKzp)>Ys53Yl9UNnr&gg!t=$o?bhS)%PpgR<-B_A3qww-sM=2&}p#Cbet4JJ);TDANy|mUT^(qp$+9? z{yX@hbM_29=+66z`wHXy6vS|X(VuKgN9NgPX zPqrtuA}NJ{o`33*&06p{26gyhZES3`z<^lsVZ9b6O~~YYUNY`51|kx7I9R#+n3(IT zUW%rL(W<~80T_bJEvjlxasBv&q5SGTKQaE!_FJ;49)J|KiBkqxX7nZ zMaIT?x0u-Pjge%$;snCLcNwuCthGx3{A>tdgVur8yaD-eR=e4~%~Yk1WygxE;fB)_xjPa2ibFd&f~HJmASwnq0X00ivvc=pXWGcj}Mi% z&`uYreRW;(&|=YTdV0k}_>pnT2iVatb-4oYaeGqyT*b@j0+h~~-+Bz2-&(mS7v&Id z)b2pah5&m06=0-YmApR)PUU-Z)VWJM?B#>HLG6{L(5dI1G|@Q=`=`QfciYn;xPxb< znx#d@yVFv#>3qz9vKS4VL_YutZ9q&HXB7~e;ot3reOe7tE>{q1a@l)n?kX!AxjPT` z2}}n}mWBXq&A2Zz)iwL_N@zPA7=O6@ta`#wtXybFg#Js=X1?w#i%v5d`KuW6At20( z3pO;=N(~3f85@c2rAhX;&+}|`s!6tIE7u&%jYT_#or6wl9})ZqjKk#VVkS7+9}S-f~*u%hvOVPx1KLc z#nZ*vU1CjE3E+1=id#@-&A-4V4Ny?(360lgJY1-5U0JLSO&z&W7!$rp5jfh+zIGLQ zu0>U6#pAS@(Fix28d^%aB;7Y#>TiyIHD6~_EwgqB`YiGswzqUA5>rPMpyx*5-^#$4 zL$ti{?8?!BJukWela$ShIXu}Fz^5pV!J8Xc7UY244dv_Ue!KQfuWrA5V7e{-yxyvB zQRkS@R!Jl6Z-5Q>(Jj|Yex_`1Zv_uB{$E?LGi?0LTuQGBtj0$C^l0jlBu+xN!39SD z%`l&pxvKKECp*y_7YAgN?J*U%{TU0bMnytBXTsJedJhadHw#yo45+*hd_{JfAgZ3o zBBt-{%ev_^u0j}a=c(eIt8;YWtBUf8G8~*WPHSk5#=%%pyH&%IS65vGydC>qOiH~E z+567+(Btq>=FQM4b=#?e7{F?$>y%pp4sHE?32Mi>5oNIJUOdw|fxPNMfphL9C~ik< zaf1dD^jMf{BlP}l+gRQwDCm|SbESGa_Qy8mDtjjr50BREkXWxj+${YY)a#<)K#NbA z`lw$viDx3jhQ}x^w0l3~78;3Ljm78nKK{P(V9$#3n21m}24GkQRR}SuD7a;}duIp* zDi*3;qh`@_S%*E|Sr$SL|6;~gU&Ln6trPF+!r z9-#Re8#+Cv3+YHoe$&B~2iDC_dkj)YN|SBmtNiCuF+r7Hx75_zTFdF!M^VBrk+Iu} zR?JO`#-r7y)0Wl3%%!=;g%IZvP0o?itD%J7w@yaalhqqw_ft54=p~{~UI2{iLvwSx zkjU4}ifE^ZzhR~B`5B`ng)zW@=tZeBW1((oQln(PTuST%tc}Zy)nK>}wOG*OZ$S}c zgFqU~@AWTVfBG!n8HxEdK<m6_xQ&Ilhuk-D7^H?N;i@Fm%;^GF(aXO?Ky_XeP_ zov=Ih6K(3fK#O^`k1083kI7Qdjh@lWL(F+h$wCF<4#O_~@?_$Gn{l!RQ_R=RPksQ> zs{XQeg5O#@SwYIrTX9M?>+xKdet3idJqn9#-EIBw%EYgZvKlP0S+r4o-UVyaxN3Sv zL+5Nxv+^CD^>!Wi9B~vL&#$hnng9+=8}w{faN(NRI4UN`@ThtV_X(c1cw_d%EQO(64pfk!OTom`N zU2c(KJ)iU3YZ+WBd?~Qr!WzdcT-l-OroHa~9Lehfm~8n8AU5C>C?kBIrICNQYz4X3 zGW{tl+kqY};0JlM)pxu9@%Y0?zH{VEtG3YLkFIRjkRhrY1CVsx0&I2PR(#nhXL`GP zlY%>gaVX=>k#*7^KGld1^9N+s)Nnk@5*>`#fjq$BJ(P#r6Nab%!%0ashEndLwrKQ7ZA__8;UNZ@8orrR8P=*Xt=iNBO~lYX!~XA=7Ed6JM<(4$g~0vr{` z(NuQBdu4<;ZU2Y8w~mTx{{lw^L=YqlN>U7zR=N>XR8)|b6j4xW=%GU?NxdRnDh(3S z3^0U*bl1QP0|Jsm&(QC1@AY1|xDMa9)?M%Y-dp!S*5Nbz?9cAKKYM?U{&uEYoZras zK$%gnlC8|w8!>lwBKytpzR{tKV!Hopb~sc0NIj_g*X3 zL4fZZG+VysP!)8civ3=XXH4co>(AFF1YCI4SauAsNeicK$)BUHBR*AKs;vZO0~0+~ zD<_OvudyC!@DwG-x4bFgaRN&++uAhGIVc@h$$IKLJCISm5|S!C&7)WTwA^RF&TYvz zwPNGt)%mY^FKno$Eq>kwV0Z$!XY4DdcSMFYNsD^CKMU|}?F(T`Jk#PoPIg()R?Qpe z-$@tztU@W0wKQ;*&zoiZ;`4fKjp&xB-G0rh$4PZaY#$hRpY_K(7O`c2ysPfKh@fSr z24k;+#8SvwnXtoD$ZABjA(Hl*P0I2QOQ#2w`D9&|U8(X$=L7UlzV5_tMtm4NZKQ(> z*j9auVq>8HfvxCaY+zu@VDdE?K3cizOR_=9+Aen$%F?x$Zh$pY^EcLza6y0jy zPkr9SLh3srWTYBx6D^a2L?`ogw;ZqCrg5(rG@yHel%7?$wIRQyCvH6+P|HA_ss}H> zg252N$#9I-Rbl?3g(D@|I09P{u8?x3L^s)CsKL}bg&)Fc=Cvork>i>)H*DR;w{>cZ z9Aw;}iVCl}bt`REly9jqQ1Hiv_OcDR1;;Zo+Uiz12crZx4P!+eIyM6r61a_luZlGZ ztUAt%=^!QMq4z5bbb8jl&!c6>&&mZdJ@XT4gL{ciYrsA(5iI5-tceUJA<@S+Vn|k< zB>7Fk6O0U}P%qy%OMI27IxAqZz8y>DGrKz=`D-pyEn5N@WxCzEak1&WCxe)W9&+*KfX@ z(tMck{U;gUm5~|(Tk-GS!@`)(4rD{hX$sw)_L`(@|EfGpxq0+@Fp`P8FuDhQQW7SU#o89i7H{? zAN&5@zU9*sTtA=D{U3&=?rcp4rKHe@E4OVCS(oA zN(a$#j_Y|5SzNo@kl9I-L$-C=|J?A}37#G((t#+-noX`JRVFWsy{T;L9|w+V8iB8qXW#jl zuh7yX`TF@OdLeUu#QaN3FRvs1a1Cw4mWt?E2$Fmv7Wd%$mb;zT#)hXX!4}hFZhmSR z6q#D-*BfvPt@yQN zg22t;EQOEc6R7hzq+6SG={L77r=6GTrkD&jbuy-F8n*(OuB0>$02Ae7fo?cBQ9-~4 z<5IK9&UEWs^I{Kt+iV@aDO{f zW}sEeP4MH{(ilDGg^Q<;of$hTIB&l);=yk}tViXdal(JayG&4TxyEH{gWvQT6KCP7 zpRQw<2kl@(;(>*N^XUSmvdu5^Gm&iezZuOJ>(Ih9)27lV}QI?WAXFpx-7 zh?7*!{NZn9FI;ZmBRA}0bgHj{l<{S3S!@-D*JaJLU}+orG$!XCvcz#@{^}>o#7QS& zMW)^R0=$TcMSw2FCJPG6V+_;{I4_^j<2N%&`G@;W%yvAsRt>yR@5(6Fu7<{Z(12o= ziLVpSPUR`qUN6!%+Y;(pV%FC=$x5yJo^imb^bOhr$=c1A#c!7B@<`&{+X?Z(0$AP2 z7A<8=AeD>M_K)*Gi=oSNQW^2q@T=AKH`F_S5Sq(sKrjYb`lq? zT+Y8cb%`YB7*Qkyh=8kYtK|aTte@Ltvg>SX%MO%sIO>*QTp`12&uxGg(J3=IeHqRKm(|6iwRT3M$+;N7!MWk|O zm~>C^7MP=pU)Ecouu<+8{=4>y+d3)DeO-LetMHre$@4Bp=HF1n=GCLUEwa?&B`n%T zJ-%~jv=GrBql3-9nwb_dm#X(RI(b5W8$W_dcn6Kv73r~f=nw)r+Rr#NkxM*G#=4FCZIGkFTmK$UtFCGH%1<3;=H4ySa{VjQ zy;0ikI^8C8Ov+r{vZS+OmYbe1_9aIBQhgXBYhnMeOHOR%DBC>?bHO;1bY8n;ECrQU zBYho9>$w?sZv*nDA`@k?{hF6`mo8S%voNkj4H?#vmM#x}_92?5e0{EkFgJyFF5)w( zfzYS2I(aHA@ko+#Ue4@se%H}5m#bKr%+G>PFqW^GYBlQ7c`h5+Zx-#8)3)L+b#tVG z6kn7s4)^(8j30zK7Mdk&R;t4d^b8Qa`?6(HUdSQbSTSz;_e zwp1TOEjlUvQiee{#eVJ1VxeYjd@G6SmqR?tmjl9f8&(tB<<4C-(D&RxvP^7@2S(ky zZU5b`M#|G-d&Ixh=50@e(#kOD_*47*(L&R+MZNsqGA?^l?iS!zj{%-4EZpCJ47k4C(RG^G;8%K9|p{zdn&1>eZ z&CtIEn#F@C7&u`d!o+6Z{JYjae7-Hw+GvT6BH^>ecSI`l`tlryo@;Rzn;O4TaEGan zP<-ypt9iO}feRG6aN^wPwAji==Y1N1;g<&S3i2-G$*HTy(N9;>RKN5`$ehuh8Jjoe2SQbE|Ncdv9>rCNt&0xVCbnnMMb5`QvA6+JG!1?GB2GQ zuc4SC4*kK$XRe-Q`F)_MG>?v|VUflJ4z#AD9KwxjUmYQxl0#!{ca` zik#faognjpmXFJLs%EJTAtHw%#PVdYk4OjkvU7Pca@TJ+H8t^m#JnCQmXK`qb*-Nk zb{V%ZYQGj&U1s!pw@$G=KI2KG|Q8d_SBu15R|uhO96EGg*Rzd>kDG; zede_(a0+|j`@0vFx3`!)l@wm~`qPuTz8Pg)5*4>_>sBo5=@bjuplVt1(xTRn1>*B` zQgY1*)Quaa_RH;-i&?->&OKGT_LJ07xD9%Ow>R;R!{^F%$B!b(UYCR)F_b;+B(&iI zb6xUQX%`>g<_uZuLRrq4)#s3=PY2rQV+|~i-OwXOW2@#1DR1a4pAYz^Tjm%+x#iKq zNYbe2nvA~{D|cOcGwndA0pCiB*)hQbEr(fo@lR%)G|Snl3KA z%VM8&U)3`slFVRARgk6eZ{r&FbhkPvV^8f^#0agR?SLM!TSp-sMX9DrJfAJhrhB*| zkNZ2xHPoy=nLB&o}{1E%rpSzwK4(r`4%`oW*3zv8S|VU2y5u^}lKGe)R% z(?qU&(JYe~qkMLR4wwm{*OOGz^BQw@A11BU;PseZWz!Z^8_)jZI=d4X;p8=#w*}^r zlFY}GTa;-7jvYekJZjU={Fc0~xt8f>COh_xFjbvz@vcFZyZDG)P%@JXpI2!v z_7}1J=)Gk(aZc_j;dgq8blrEmWAFh1_u}TYR0-*cZ>`4mF*Jsijwl623IxnUWE-Be z%VqR(Fw&g-J({3?4jWOe_*&}EFvpt*5856dKD*nuijZ-syxQVR`nH7aL#$K?>XqUY zi5dlMAd^@RuV&HX^5`Dk@_Gq!%$;N}Yxs;MPQtyWw(Gq=(u>1R?~izq;6?A8LKMAy=h^5Ei^ZPv z19R4Jy>>$NoRYSW4?o?)57tQbx)%4b6s90LU12mjo5@s+3v_QYnJ`R%R=99;E_3Et zGShs+6fs|7^(q)0f}+|j7WopPD9JBSl;c9Of49dc$9S$OiK{b?nMS~ zNOxQRbb*Zxrci_V+q43wr?hw%JU`5DxDTBX^!fYThSNjRWg{|J3_3?S9rFZ>-g@cy z@#2Mxk-Q&)^SpPv1%HObHNrKejN(V=ROR$lv-W0(nJruZQATHy#1*I9+>4k^!aEsp z2_iPi%>A29j3XuqBSs>Z=On|0t_j&dO>}4?Q!G_<}-bAPp4b8mL01nT!61m=xi!JUYvUn zA*yOOX)jGYw7hHs%)DQlxFYXz>-O|=fQ zBv`g?F?#D`R_z1MsJEsYe_UqxMo!oEy8#mT^0fHMa;Krkv7?uL9}5<)0~g4-gyCg< ze*{eN#tmcymA_Afz+DYdnk%KA=R!kn>%x){L@rIrtHWK0%tNU(+4~>?*c`??&91u@CZm-Jo0nWgwXn|kRP6eHFk8JOY0 zFspId15Q9$-p93`l#p=tB5R_rUdXA zmhIRCQfes{1tFkBn8^C(9DeQj(YgpYfuM2t3s3J`_1G!)HddV**=uKPH>TgsSwBH* z_yG6F6dT381IDH9bhH1_EljJ#H<+rbl(hq#rG=)BmnL0wr&;+lS{p%l8s)O|Il>LT zz0N^)e8=}*EPrtb)Q%&q*-&pP&aJ1KiIqwS5pWxR#ws)ju zxROm8jje>snV$QZ7*t?Fq$5epvzw!YHZEqbOvJ5Jx`cXL%@|g(je7hDziQx&4q9k~ zt`(vhfxbszMsY&h;h0Bqz#1aSiv_=Gux{`agwZufwtpwCY{NY_m;LlH*SUl@o)_L+ z``{URNq3n|?Oex4xy-r@>Gq>$91#9#v>}cmA}d9U9)*C-s4v;yonveL_@8z-hF#|&Ern-#HOArAGQn&42JJa#N4U3Ze1cjE1n=yp66zq zmuA$c77Jcu$?ok?I?oF9s_Z;3 zaUSbS`pVkhbw#-fhYLn*%)lT;Qq=yi0_6uay&UBmLl)zYj8tS{F5+OX9`msr$L#2SSrub2DAJkd&CpNW*hUtn0B(vyWU7XUD(xC%m{r=qtzM zn80qzOpifC);g*4M;Bp{&C?|a+9$BN{ILwt>&cMRm8oL7k%ITuP)AMN@Udx5c4|&qkf6<)&Oz8V>CV|Sj2lC*+Hh?q1@gdWzF5o^ zr=LxJ@~|Y?;8T4`fyq_IWc5rAquZH3(Af4@3$m1xXQO*kDJib#T9!|DCc5gPs=0+*JSzkR9+}X0&6pK2A+(Pda;UpD95Z*78+#jRJuw( zS}LN;mmFiK*(bEkbXUFsYnGGUi?dL0{G4o3n&l?g>GIgkb+k3zDM+aHO|7_y>*Wa} zsjU}hA6UQU$+DfC!`eZtI#w#mip$mdjmCx@mnN=5g31a;6Qp>y!~MJ#2>4U;m)vn1 z7n{8q1*}xMSAlsgD0$q@%*05Xn8XgZ&PDI{3g&@W6~<1bk$}I5&_RXH&dF$V- z-iOwXHpr?__|x$CF%QsoMsvj#+_{OH8(3mmYORK_gTV8TD zc!bDD_Hl0&|K^HGqCh2Fba^x7$053XPp1>be0Jl3>vH>ilpcy&YS8w%0D)jxY)h13 zRnjvvxI(Akkj}fwl@|Gu6`2P8b$9tuBcY+FjUKH_0!B;l5C)YA-z#^SDR0kKz_@u@ zP~0txZh|um5Zi?bTp!N8_dJEvDJhLb71a!d!p!HK5|0E6CfBV?3gB7^(;t)J!CbF< z*JDrDsk-^xnYjHqt5YMZe0{3+io_`vY4;;|_(zFNtMKz*!xVnh>@=8Fouoax`-~F! zX9YX{q;813!Sw|%8k&s7^`ylM`67&Ti0Gu%f=T8Mn3mFXjf(7?^UgN`Ga>$#_!;v8 zC4-IS;H($k&p+9B+m}s`8d^s2k1iAoTidNIs%q)l4cY0$Lg5_zXZ4+|ha*vl>ja}npY_#=wEp?SnRx!@isk9K&Butvl zwj@xK%`GBt;3kRWad=c3w-_0U+}yn6!>{GEzAU_9%(xB7gwd7+ou}eOjLxj_D;Hhu zESMH7b41Gb>N2K>(XaIP!J>H_f@!Z_L%wsWG{#l6RI$gq4_$aE?c<~L1BiL!0d)eOlIO-EouleyYl{>K#ZH28BH_9d%9*w>hES&(+*@pRdM^QmX!)VX|%Fc(vm@g+titbFRrU2cw#LuhugC& zVmruL=S*82OmluLNy;R0aPpPLi=G9w9O5l2*RTYscb&ia_3DHnHzjB1B4-?=6v+DMY@*4l{k5?tw26Fu3M1LkH6Np8!!TJ1uNlFe@- zn0DTlW`)_nYmtMwS7YGQTPHRgIRi%%pP2W)G#@Qn)}V?rz$<|z3aw3-=s~x6YQ$V9 zhg);F7DTyncn3zMH0!XPF7CYr8CSe*zded1gB3h8>$M(`5-ANa$=_ubTmGw<@-O!t zd$B!}4m>)&li?=)MXj~SG4XFs_nCkDQThVnnkfp*AFK~#E|hQg->V}X+DU;5l``o~ z&tX^B8}2{}CY$4khGu(L@?(KR`CeQ0kmS&{7v6^K--Cm(UPG?W)O~<&$$z#gie1ca z@aBk1*i6m}gJn!R^sYFYzEQBz!G3DAqE*`Yc2yr0{wmPHQKs}Igau>JUrEZ zlUR8a*%Fl(Sb6fN>SeGh`p21eO*v4VQutlxe$sqbGt)^2otK)?KMaD8*D*Mmu5|aY zQhjr!(;CB$zoHLx5&uhp0UpI;2&b;6Pn4_M-uP2{|A2H^`mqlqG~lyaSVq48b-Xh* zp5dCqbEhz^B`ZA$6|<0urU>OB=3*YhDf0&3vf>_}+8#8*v`qCj_U!{Kabc+qUMSB{ zOY9#8;fTj?gi7f~UZ~%)j^v#Fxrbd-u@~*#wQY;|v$F;=T8(n&j_kwdz}hohit0En z(VdrU2ci7^Bz}x=>MA~;tCnOS{s9ns5r}{4p_y`JGt?Q+H|C#6{SD6jh(`}i3eo3S zEh&FiaCpx>Gy`EFe1~#4Nh#>DTyCxj5bg~KY>3UC4RnAYB?D5{?E6xad=#m z5A$S;vuC~MYHyWQfSJ9OugO)qA+bZb+Gib%oO>enD?SKX;_NP_O!}M!u?e zB8H}*-!Q5z%MG5!EAFx$V9^;U1K9AwqX{VCJ%^#rqQ(6 z);J*^?XA%`PddIN^;VZ(a%Sl000JS4D4Bg?@)t?SF3wNuZCzV7nWhm$IX_z)sLl;lPp6gggJXEienv_PYTm86-ft93W>A~wlRKJoR(=3lm3g}A zTXOWNV_#qH-dZ91`7^a0${L$p9VDg}l^F3A;EW}-|5Ql& z+mF>a8@L~RNnoF`lYAEW)@j46&y`jv&m}*%h3m#I4x`E zXM(U#K0w#wl6^4F8;I^D%h6gNzE(-TnL<=~_m(e(!`QiQ&e2DFain3|R(En!26L>5 z>zJX=`Hs8Ug~5G=2jU_@Zdqk%5%q4kYUZ<1j=$KR^kc@OO*kDQ&ZwVuSCG)7E429G z^8HbV`R$;Q**O^IAsJ+Qsa3tEmz1ZJcP1B8u3u4sSH@%)S-dgO8+)onWFk8IS@i3u z(ABF3YW+JKK%TO+mRFaoRx>nPfGT%1+n5z)+ObD5t8=U;k$FSyf{GP+$cWExG< z+I-ROwQi2{&F&6_-rMX;a__eW|`5&KxZN=tJKM>Dh zUq_0*P%2sq5=%EDms<3-XuD2i3)_m+)h>FjztRJAF;!K0F;2%6>YO!NC;x$x+MM5z z@?9AVav~K%#))CqYSZL?D&t=J)$>p~eZux=uE?Jc<5iE2H^rVdFJ7ZTd%x#%VS6wL zJL_mrJ3r6mEg(xL;i!`D3iLS&>LH$O;>H~z>Of2VBeRm7Z6X_3qyN&lkLG7;Z*qF3 zSjVYU)$;MyxZYVw-Cv5*EWY7p=1@KLPgaGVvLB5?o-u3*KN;&V9+AD4{Xu_6cv^@{ zIr)A%lhIdfhgDl8k&k}0x0e=V>zO?Zk`4Dh%TZNW42i(L^+NV zT}jvTals;ZX51>a^@rU((BIhNh|kLfo*1pgFqB(QCZa>{6ngTti1S4Q7vJf(N?-%{ z-mv*_jg)yp5oR()n|>TJT-(t0n>4CeqU{;Ct=^L8fr4Lfm4rHl&zy)ghrl zX8&^Npo393uYO_nN{^P0Swa6LPQ3zgZKq}dbcnJ78E^s^jNR7h>=4F|*h7ZXUF}tc z$Ci~x*VxqbCd))}jnrWeVy+1!*mcM;xZdcq91QAKcQoklc&5sb@46+UC3XKr&-WSI zU&z1^X_u_o$lU3kO&Y+glZN&2oOY((=~AJg-6RvU%`ZL;7Uv<`}U;#0DM|_KxPKMchk5 zjr;LsWjbi602R;P0Omj_B^~c|R0*xlTtF?jA}1aNhHq1C=r|8Kz>6DsGR*b|f923ven%Amfl#goI~Y zwRm56-vigw7JyU#8Zd8ZiOkEJg8cjP>4Wc&CnPksY}(SY7~F@Q$45f707mN2*{hc_ zXn(j(`ju0z3*!ndHp=Yy7We zY&;Wy9&={Uoxcgbcfp_9;6Zjt(cj1z@z}`v5CkbTcmScD_UgV4ZiTY-k4`P{lfda~ z2lk7=0X(vGcPYq|L4JtSCF5vwtB!5y-%EYz$I7Q@1#Ikv%||NHZ91#R{X@CQ;89k3 zMeB2S^c^GwUJopXF6lWq)8?#YTkLcySAVA1(hnpU)L&LZhivIGD$$~%m)~VSUHc+n z4oebh5&K>=-|G`c9jiAlBfKHMQ<~IV`d~lGg2w?Xe4mx<*!4ZnJ;-?G^M&9%4CM&C z^k`hj7CGtG`Uk`DKbnU(*-;JZ(Fx)3}1tT=;R<2FWqQy>Jl4mjLxQ|Z30C$rX zuF8>q%+iVZj2&IB&TYOxy#_0caN7yZMBYSwvkaYi5$DuuT#YiBDYEFV zVJ7X}MECF-;!@`QESs1V?Gn&#>%Pv zg@3d28-&=XYy`*qH1~S<)#Mg?QRgulb?+T01|ky~;Z*s7O+71^_L}%6`cs(C&Srli z8diwv8@(zV2ycSG;Cj>q*{EpzhT(CD0Xo4=vP@zTEdQ4ILU0aoQhdur)BU?msqt{usB!g zY&~BZ4(Z&Nsz2j3nEH<5zF{C{nNVFiev4e*l${ET2c{pKpPCYttyNslsSp(}UF&A6!=H;vqbr!) zE+!aQ_mYJo*__u`Fqw8zhFP*zZOzAg3KY}JJ7$n>QqwWrP7_SdGPHddo;axobqd&N z9mG|!;OS7GIb%r)$6x}3w`<{3YagGsH?q0XuR0HR6{^mD?A*A5giFpWTeMtf zG?@FwZnm>hSSl55S@BT+cGJy8hCaw+PDzYP%tQ&)?c7ZMEUrPy3^q3W0c|}2FJvc9 zp7^U+_`4^$tN?`jrP2O-VpfkT!U3N^Qxa|4-sa{NfzO=5R7iSeoLMw5={?_K(E3>r zHoZVp{nbr`&3UUQXk((>v3D!I7j8d_t-2ZJDn7HZ-Rtv7(&6EXCc8Zom47pl>kl$? zLEYSE-6(p*9n3;`Zgm)Byo5lbwz(o`oNBdx4qPu&Y&CT)7O1NPK z6Rh`ox8wS%KN04{j#YzJ+QgNhrSkRi`PwYE>=G<+0=m6q!eeTV&?w$`?E?Vab{&1O zVm}6xd_)Bg6GwR z*r{?#!*<%eaF~9ZmCoe!|BB|?UYC9MjJ;!vb#t~41>IT9?8cKN z2Pj8e&}=x*+KNJYMA8dEi00~$exXILT>0r^%hNbSPzH6f_)cd&BtYUM0%yKJ=m`^wSbh$+}Rr;W^{Y_J^ zt@>OAHH6fBlL4AaY?jK1AUGu!`}F<59ih-xl+Deh=uSg%t>(MtrO+`kDb>&9Gg9%6 z484*C3!(d~M3?VX1fZ%;acl1lJ?_PB+n+|i)NP_HTUrR(7#4*3apb>O2<6s$*j#Zx z*P@fNyS03~On;;=X4h|3=VLdm-m^ZR*n(+c=E8@}u!t*e8M;i8&u^;+xH|Xnbvu|l zWsI(f+W=kOci7j_Ty7z zcWrZrm-dR=McfNm#K}>3jc+HpxH(i-kOHt{&Z=kb&j#u( zr9!>+7s`v+=9bblE^ar@bQhC~28o%rjW2BHnd!fXK8fhO8pZY_@3#RVJ)5s|&Fbb3 z^FhX24X7>gQoCo<6U$XYolL+t@En1LF}D$}st1W_4^aZoDa_U~*E6 zeP+9wD(<*m=`eG;6Fqz#>w4t?>J`r9A9^hfN()RP?~3yLL|vV!K?r_e^p0zS**)enFOX;FGbJVIh~ zW)xlY-Ykiy{5z^zr&E+DVKdcE7{ri85@IXM58r2Ds#I=R5XYRWT7Gb8Q{#6tmQJ8SnX) z8Absc|2KZ=nDte*DipJutLoz`SM zTGdIsRKP8xy#0eEVSYMw{00fn7kvuztpT4 zUD3<7CG0#br}xJP-U@DIp>5W>F z=<+C}__)L+ms<5Z&cm~i2t)se-I?$hN4J$C=P?NvNYrMXMeitunpcBO`AJ!CpV**B zvxCirDi#mqjUmx32#vW7mqeh=pm_Vs8^4;E zFh>R#gGU8cO@W5ZZW}QQf^}PoLYvB*^O9N-iTz)%1gR+Q4CQT93$f`83Ex-=C&>-` z2w`V(TTc#KX_)%FvjkCXoUwUxjzB2AC5NTJCbrPxdeM*Hg1w3@$1OBF)(}4Cbh^H| zex0UINcWn1;*cwfQAHb`wL8_JS6Gbq%4S}8#m9Gc)14=i)xJ8Jig7nNTAJHC*XF8P z<_fX(Ii#D(mO$n@bK|t0Ej1rora%Ner|2BX>B&pzFF^z`FebIeC{x{|M|_?P55(%t zjgo&VOe0_TZr!Oj_aSXdIYTU9poIMbGzcr348Ir+oj=|^?X)xPpr5?@xq60XT=gcR zg>5cSwV@UG;c@FHcxw?NxuO`|{I|$d7sPx$pF+k+QNyii+|#yJrbl%Wru*{=dad31 zY5GK9%+ZOx1f6}3S<$6@A%nZKzA!Cn9-SV8cA?pKngUV+CgaAitjX|nGu$b? zAL%b4qsv!%2w2}TAxkjSPn#r}2n4-^I;zoYpvt=1alTyd=E zI$@q8GJf#E-ni34@+2*zmPFyox3>(Q`@tP3D+>BabH%Enp`Y5suI%IgftBkIfPf3b z$@q^9=%4H8Z%PBuvJZuJE1rqs(>y{bG-$s&*_%l1R@poPSrthIyikFxn1p}+Z$H96 z_&iFgAaRdGDfd7ByH@u{-X!>Px72_)bY&kS_W88k7qx`}RZ(nWlEGfmA6O;*_|xv8 z`oq4MLyDL#7vsCF_3nt0AA;O(yT~0!zSxW0yw%ZxP);Wi7$U6VETOiRNy5UhyRr?cz`$0iSq>14yk#$OHUu~8hr!Q_IV(;4ynl{NgJv>3Ug}GqWl5AtHS%Fa!&*K_>SRvabHD+2mo{Mp*( z_lo=e^2RPN@FNVEBV)>U;Xinw5Sf!zeXeRtAG9_&~(JV zpG)!8f1*4{@jXv2YsP2JYuRiQDjXE%zn@4qk^%=Ni&xHob374PrM*tY;tI;lYm{wM zBCjrxfQk>$3zp}|Ewb@suk$_db{0~gE}MJ%6U&|*9pwMzN4w=#ZjqfsfDr)ASRc4v z340#Gr;k9j-2VZ57*rOY+j=;j06bxL?(@yRxbmw%&Ts(+zkQMhTuX|H!xy6DFV*ZV z-W?DC=^VhU)g=AZ{&ir%?*#nL8bALZDCrOQ`SJ5w&!@chB%}uf%w78TV_R*dE99Uu z-9W`VS~xi9AAJzQ-%mX50SDB4cM^EPrBxk}ntutDay(PD8jgz(`w{dAljSrZ`LbxC zcTY)P{f+KH>G7}vB>zBY`nQ?*|u%^8hasxjxx@9ug_B?pm>Xuy{}_R(~^|iQplw z`C`<_LA$!=N&IbK3x`9y50tzHudy?MQr7RZB94oD3?H0kLO^WxA1IR3yDd^xO8cvb zf2qaKKdertaBPl;DlPnc@JmTkj0GaGu`%thpsM!t65u`_$#}{BQ}6ePgS)f?;EDbv zP>=9eXfvxIk3?v=4w|GrPa=+YJrO5?M_g1D2Hiji8({Z!TW*489MXos1|lx190KYh z&?NM1Oe+ucg-W2C<9&qS!r0P z)P_*vguj~R_I|LXVN!OEm zW^}a|FMyf~;xiWkUaFK3f*$irn*$Q`rV3*Ihzh(5;U+*??L(nu0^OD4k&i4v^L@nt zAYr+hN)6u5P)u_9KTs@+zS3NmH>rO!)(02Xbf85FpMM$L;i|+_k1FAs`HQQ-Z3P3c0V^vmCD09A zqf!Jw69Qf3wdVDzQ94pO^D1>RtJlemk@)xSQM1$>`sK#*ms z%)twa|J{>qQ^g_o{NI6cJOBu?T7-*bel;=i7yj`_fe1yu{tGQQTABb_8J!%$;Am0q zPWQ;y=NAm0BER1=}`b+pjZ8B%bP4iZk0z>AYLoIzdg zcb>^OgFlD>YFUP{kI|sw^#%@jqJIfgp|pf|@_ZCD=%(CXiuH>>q{s2| zY2&IO<9De4oc#JJoO@$LK=Rgr4UESboCLS~#lQx}j~@ccVYe)XT%%$Hr5NEKWiitF z{pY0z(5v4qi$&SG5`yn@6$rA7;+_B>r#JxN?j^VaY>I9j>V61%`$4*`Y;@X|mF&ywUVn5n?Sf3z!%F}nLs z77ob98<@1jv6T5Jr_dU3%J%G1_AqFQen7&K_*66cpla-S@{|KeSj>zG!D)5{i1+^k z^^{!^7|Mu9Qq$N&z=KI+24%35Q2QgjlXnOxW#H_0 zsVO}LoMN}*sg36rv=qy|K$~-=LwCF9IhWWkgYMIBH^>TCAq72{^w8c7vIKHRz@t(Y zAjl%UTc8L|u-(BphFtbb;AjB_w&D~U(tjLu?-xc7hTjL%8I}$KMsg~pMgu|Or+&!P zPrVYr%f>ts&<1V~rK3|<6}?bOU$`ECZsE%rphkEow2s|`g==&KJYm@#zM4^DX90cT z-QCjJf1vn0G=aQ_GqLI%h+CG9=ie>2ytqQcbHIb{)#SC~z{!i=gPY*x7Ja~dcwN*k z0yUf8fy&dr1?r+=Fd7J+-Yd87*NA`YMKN5Ot2TXJOQXXJyhh>*Y+!9nm$;vT%Ji30K<(1-y+S4UKBIvkt0qyq0u=x6 zz_^FCIo%;;AVQ#$rTw_W3%o%10Ekc~Lhk2)qm=-lbu&q)4IHftiWIxR6b=Cv5AbqO z?eU-tXldMCS`FakqeNw1(3gf>QVho9*bEJ|J)}nX9@xU+(Aa^5B{hOZ=pXX8Ujh4p zIX=U8cRj#qmIlQ8|AC4i2AV7&mr~E{YvDN%7$lMaI&D%%yL|v{|9IoITf6iZCjsB*lilM& z!4-V)rUx1z$g=*HAbda;_R{UL1`uSueD`y9{y--7hhkdo_G0*O`hoXieB4D#bPN0v z9TuY9<}wi<@k4;U1bDex@N$$!6x384{Xz-wk~Q$<--+~oTk`v6<5d6&(nc3{&G<)w zN)RV7&DL@#G#emck%!}dJ**i9Xd!YYA^ZhCf0J2yayOyW!y5tRC2;c)FnKBIb;z|t zHh{!-WF1)ml`-H}Boa7RoD3egbu9s`>!i9n4%A*ecL3fPs_@|mxQqdnzrtk>Y48S@ zyrSLmet|v7Aw5?eyOps-J^Vu`wnEs1GuvYD``IeSNmT z{axa(ybs%A{^d5cQv5@%A^YFj7cXl8?K(hRW#cR*cyk{YHxR~xbwhgWppY?j{i>?3 z@saOeH|gJIxu<_S{NcDmtPz{^C9~^!Ugi_#n;Tn0W(b9d^?W~H;`$5mEb&b*0~%O7 zj$Ft0kUo_p%_1#Rmj7VvLLs~A`65>RR$!F#ROd*HYG_53V1%K~Elbjk*pv=dXl!P%1ie-EX0e8!t2 zh3#yL6QIVKrR~bXG%Ki9xl4EMn#py#E;mq}8PJs;DV*iOI|52hejQ-dq>~x8+cvRaV(n2tqc~Okzi=US zA}Q|r)*cw}m(OyyrFL0Yx5m#M&Q0Ew zSt@YpkpAB+g|ri?O)Xk}vj8hF`~zGli`Jehk4}O2 zXCJ-bszU%y*XKomELJNVLNaOTkwaU?{|`jz7OBmd*Qi@@(lLj~Y4X=B|NX}OfKR>; zoKpX5hQ|glPlu{c{4C)GSH$+IBcQS%{f@gcS1pJKe4B@+@?`&mONS$|T9YXK#hbo`mbSRKND@Z}%o z_<50Z@~)Y91sELR(y56=Pu+3S8t%7DhdDfc!?@fjE5fB`Z>P~$|*uNkilgw_)Z;Gs1T!zRWaZvHz7LmbTy z4X%lIEf0A1PWe0!h))6~T|_}icEm#|#Ld&Zp|lZ0IdIvp9zhVOt^nZ^5Of}%y4?n9a5_YX+3Wuw2T=cBOqVAu_ZINn$GgBi zH?hs#%@X1R!Ph<(wC$Eg|24z2Yn~2OpZ_s8eH70p3OLd4d?}cH82>YX%PW(XBi=PU z&kiv>NxNbG)`=BhOU@j^gy$X)c0+3^ z@X#8FVFB}WxcT4BIuFh2Srh@wbMg?&)4m&@tS?W3&oCan3#6v1|K!Vl(14Y&VF%0v z-ys#A-?@3h*ohFlazXjOrCM!B{1{8zDfp}lnXn_MBRR_r1f7ScZvPd<{*R6A zyIer+Dd3SyjveClEP%p|cbxPPxDN$TGC33_+C|-UDf7XiFWo&_?*o%eRU~E?*qC@I z26v<#R=U+ujpk?Jvk70G);t9L0NP7(Mpy$*2kz9wU@7zD074&Qc+;DbJh z;4?CJnCmxc>{O5%!Wrnc6WosDD(L%?Q9K^G0C_?uW~c42@?O0mVcYI$s?3dAT6=P9~SUrdU2$lasBk!$u6W0RbsfWkh7_SKr;H7qc^cfR##5CcLg?Ep!F!v zLf^q~yveG$TRp6$1y3gkQ^*ku*1SDzz{aaz<#}ybsa1~I_GvNj5wiy1lcGAhuOlMY zc%)kEfe4?c`){RQdoN5OPp-j(o}%9Myv7CDo6 zg7|WqY-8yI$%4)YVyb-F6U|YDO;}VTK@=p%A!4CR`PufS5VEmTa?uR3Gs@iP{Z5FJ z-ZWePk;9nfGu-^iZcfxpJpuyPq2{#bJ2jdWN~0I8WrZ*Z5*a^_8Fg}~yh4BfO0_pR zhjQ`*7aj$zxq|D1Sz{8w#-_!3r=XY-Y`u+iY7Y?80eZ8;G z%edyA6ho&iMH`2LL|i_ACfd+4oP7K>gQ4a6@s0Z5xQ@l3ERWE};-%KkW%|dPV(*YS z_90c||HhA>6B@QRp6)I!ozL9iAvriU`&{1}!l$nd+UVfX%asO@1Vr6(6QBcDGD$N7 zMh1mlj}^q3#8YTI_q2C-mO$X`lZIW!B1uWID&Egzl-#|APt@aqfcIjiM~IVhTA*iGD}PRpCX6?^0O*-r19p6lV8A`k1omM@ zV4qvloV~i^e>oA{a33_cpulD~ZN)3TBLDD&_dnaN;@n!wI_Gi$&YI&EZVWr@&f4^f zFTfCBkdd-JHr;^pp~`id*MCh$3(xNhD|dT4Ayb~FvoRh)QeW6YO87`{@alIH6Wt-_c@wAHf~Bfa3e~PKj^^!mI86ke78<8TwAsy{?qTa-PjYZnm8*Ke_ePc1z~? z#nM~T-kj#3^7riL>8ce`U?h0CG1RNTfK?sKI?zKCZ(wg`(7aP$0;lsr>jOX!7gZ!L-&eSD%pAobnkjbLgB-is1}#`OSl zJlzXZW3@AZR&AUlS&y<7!D0oyo*G)Q`&p{8X;c2=FM9#K4b{UZ$x=%lFmF)Y=ygDTi5wtLiN@ZD!bHN1a7^n28ohS?Q`bvi?3uHc{%v+ss7ODn)O8 z2h_SMpPflk)QpqDH|{2XCP9`X0KoP0Dx~);F4@OL&Ae7H_&t1b&94nAA6?w$jQnL=XbUb_1xC9>aUjTot({aI#KB)Pf)owC>iM`6dBs2?Kc zdXr;k{3n!!@sM=Jee6WE&b6fd=l4xQD#l~jIHzR|;WvF5Qs>}eE=lDu;Z5s_&hSvP zy7wX6ZhJUbb`-xqCAqOt%KadFdnhO0Y7$|IiE5BNQTrQCMM}`TZBX6@H4NQz8~UkP zbTc~Ue&x%SI_F554oHgh%&H}sF|QTuDk%EMxk1`P(fdTYlxnVG=;2P~*$KUwIW}fA zNHZ%Zq0lhqG$iNo=D*|Frgq4=M)W1qyNc4v4>^*3px{dp`ZQS`&xVy9kmBX$D;$`< z^P{W&KGmORyHDJBz#XCsm{l5%D&Y6Qt$ew36L589b;oh~e5^U-`*nFZg4F9dh$u3L zM!NYZ7u zeacj1Bn8AMAYcy0&YdLMw=?~@g_GNtJx?=8P;mrk{_`H2Ga2}mzjd;18&FQ~{!(?< zfSOPX>}17c;Z2Qk35$-%)IjUb8(R_4y>XB-&v5!vC%mlr^a`-wBI#~72`UnhD`vVr zVbKPsw=XIFAy}#Kh! zZHCQ!9g=cAX7qm2&Cx2%mt-1d#J$2((#)JTF>^EHP)J$wj9wI6*ln>bM!Y~QeSvw? zf|)|S2qBIe)2++}4sI(Wf0`+{qy@!DcF4ZkWI1*j{-{+V(KCI( zz{UqtS;?!{X#+q4ytWYO6ior-z^$5~O_T4f>C4T_i{g6(7q>6`si(5%rF1d%eKa@M zP$_+?yk;PC!~>nMQHw;4fC?mI%;L7DMoNw4x2p7mJ^`a2^+L0=Z6G?CEqk-JiLdEy zH`AwDwfDWoy!+;yx^6Z<^|WYwt-oe6j+yga^_ffJ99qf)TI)EWl0cb1Jbk0RE+Ea> z@W5POJnA@KJmTSH5j@#)OirY%A8m_h=xD16AMc+>fmz=iXBxaKLSgv{Yu)rP-Cv*; z$1tWwp-r>N!agVb`I0sR(BZjnj*?Y;KS{2Fk4w6XPg0g?n& zzttF`pdW)4v%(18@ynQAF$By<6Xtb&dvff>z+rsqIoig(tOdJH*&_O`Jb4~Y_w{cc z-cyL5TNgg$HQ_wr-dFeow>-0p5^8sbP2Wl=-TKbMrvd)h&@j%3bItmVNaFo?bOm8Flq*tRfYZyzAe?h31Jkvl@W zCmK|EJ1PzGj!Iwqbg8j73h(U&wLhx!aO^!Cb+^mAhZc#-E;#Y=)= zE5A>Tab~6)mg(#jf(gv2^@)AEHQnNKw=Io92SG+B?Z}{~WEEeCrY2 zgH+L44jwiuZd2D*KI+$qRJ>i-ErgZDOajP(S)h*Z=Av%-ypO)?yQ0Nj~rv zzuO*ir**SkCu?=)?A^4!*uWdsUz@l`?F+ZEGYPLOo)Ui3b)EWjO{V-oeFBk;Uroud z-mnhWl!%@d*Ve|oU`AnG=NM}ZRdcPI(|Wpr#3RH+1qC6LN0XrPK>~$=l2u&Cm!wYE zYRWW20}EEoAoyGU?)GD&dw=M5g{0Ku`c0k?3P87%SE_faTx;0o%fqEr&%2sq7s|c0 zUhJGHfC?OsssOwf`0|Bm7L5*x=N~-tFNA#9T737iX<)een#@YyFLxCVqiYt+0iW~s zmFL^Z%(4)|6Y_+j`VIwXa=8sxD5<>R;+>*aLNwvPYFzHlj6LeP!5u13 z^9ZZG9Yy>~h-cwqxTqbf2t)R;OOHfMl4#i5Lf`FoH`x{asM7F??x7^aep6n4 zM9)gP>Vv;{77iblYR;`jfx8#_8s*3+?6~``*v4AhlhT?oRSu7fvzvrb3__g~j%n30 z6&|tcINEAf1pnzZDvfW){io%LPGtku49S};6u`P_DWd-S>a_;0fk)@i3bbHPT9c$K_vMKa98 za_N2P;}RYpprDHK(< z3XanUj$z_A-2Aga)$o80{j`oJ{rjiaOatpfRPX0mlNITC-Cs1%i<)ni`swK+Guih! zIQ|ATL_T?j(8H7R7u7!z*LoWb0_&yaBcQR`0`q-Zf@p9J_}Zj0=L_B)5hZMYB5@ke{ACznoy^`a-4 zX^NKuyF*9PW_m3Fr1$KM19H$BW@t0~{lN6 zBTT%!^M2i7M|&O>rf)nkHuo31XU4foIndOQ{|S|@DZ-m?vVkKC{_x?mmEHv$#fB!E zwkRkpcX@rtUMFV#IZN=u(cMSi|7ZVQB`TKF`Z?@wOJIG1$Wv!iy1ClDFK|LD^vy$p zT9)2|km|a9VH*BEh}+jLAz19F_3z{!VXVG6woN>j7nY?W*@P05V#d9-{?1x;@lQdFHZmpGM%); zuoE#JUtU#PbyWJd%{D7OZ=qJL;*M=0K<~RVOWIBjAn&ePX%A$s-VCbnO-j%_T+cT% z_x+%_nP;<)3l}mK{OejJq@7V`UP4tstr10)t5e55A*3OUseyI^*V3VgE@@=8cVE-- zU50`5oF{_VxSr}xB@Zn^rkn!Qe)2IeDuGcTqc@vqDMoeZ8o}G6HvLy z=?b??`z$3W=D9byo5^Qmypv}`JE~1*R>rkCU=aHawm$ImC@t3yJk{UQvuXr_6&-nSd^d2gORD55Q* z@VTOC;hrra6RMKjg@P}gdAhE+7a6j#ua@xc*e*#V%%}^B*y*j~w~qwH2`xzA@KC4U zHIW=o7|;hQI?$sM#xz}hO>LoVSX8c3IhY7knW96w78Am-{ccX0a) zV!GOvc`Dqbr2N9VU5g6x=#E(Zs?ke+sdM3gEjaTC{W6}zz&jj>7L8!>UfwTbwf!Fw zVS_y)OjVCvbGz-V27D)0*P?+fMU!>V+8jv==WG z<{u?PQz20O8zM$j^|@4Ce+a`Yha5-sm-&-e)9<#d$;%-}gjGK#{!XIIAd}S}7;Bj9 zc@*?RpdGV2IsL)2XM(@aq--dJtfH>TSW~WtqS(0+xh2?#beRC|G&~l&6Cp%;-r!o1P z0*L$MnR&|P-3^-TC(^-NME$8aH9K(}6RomsmEp@O_B|!U+j;)Vf!ivuDait+Z<=X+ z<%zwWCqg%`NkPZW6-g1_!iXy@<{8V~Wl@BwhD%pTq&-&W%L-BYH*(2ZPpK`x<_*@= zc}%Qv3c;jL#?&hZN9Sbc^N)2S&9kDIh@I>Zs4?)AP3;7jY+C(^hh~;Q&Z%(xI@K#8 z@re4tj{tvp#?8zo+vth(fub8Rc@~99g$$FlD-u)1;?|N**+=zfkQ_lQ_}>O%^U*G# zUjUqoqWNN5<5G=?S23Xq=>0Dd$)+A>Zs-?w^H_~bx6jM~MPmc$xy_8_czCebeJ94$ zqqU^(^jdhQvWuONH`Mdj&DY-4jL{_)E`bEDgA~|sf{#1}MqHTR$~E39D7_K6h!@*n zx@UjxM$H}dP zcPjay)Dk91BqsbA8p~$REKzDJdj2ro)0mpJHlfq`h}ksEIAe2A<tizQ4t1;+RyXzrPNbrhS3DjU@3xRMhO|uOW!*X45}>-rO+-f_ngoJtAUv;4|8J5dnj=oycT8;x3|njc>+r@;A5+^ zS2x@w`5SWk{1xWo*|-t`pKxc!4Po;inayrHqOz~pMJ9D`CQUfWE5G%JF<;OB%SN#A ziV?NH^vkzez}AKLR+z%Lywvu_`odL*utvPV7zgg9yL-$OzyO!!bhhPDWlzfyE#6_r z69^g5D0-#h`+QJ4OeKz>GAFJuYE{F!cK!u!k2dx&CL2Sv%9YtzHpB@-qqpDYU%tX> zam>DlU!S(=VLDhdMRhI@8p#gn8bKkAy&nMqqW^vy-R~SZfE32t@IFdI#Y^25BKG?C}gXRPTCOiJlyoR(7MyA zHlyHNEgK5>w~^l`r3HI4&Ng*^9!1xA-vmfzk8ChwqVvb68m%Pw(dB|@U~o!tL-8BZC@Qi@@+NW>Q#Bhbr+-ZcmmLFzN1Q^ zd+P=Uoh^SVe5~o&<0?`!U(Vxhaz}-GMze8|J~E;QpYM)3oZVSq{Y|sP$tAf*o)DFu zYASA6dt5p3nn@bbN|Ssl^}G1np_+&};`t;+;LV-woQi=l3eR&=jgZPUjHtts37pLE zuKP(U2d$a=DlElj!X?Lv{nizQT0Z87SEY|L#z`)|ZJPdAxj~YsF4GP4cWX8BpY=mj zNqfqYZ!}T{0n|5p++9J#e_@`_5y}c8`es$lV;%z5fNjvc;1+9KbD&RS)rgkigr_N7 z_dB6l(-EEw(eC*jjsGkF#O|-hu}QpaYo?w1T8r4Md+>e`8WZib7;?9b)ktEZo6UBQ znK`7wFrU{vN5m>OOekct2;(x%y@k(=|7ak)8twj>aoHOyS>0HKv|(F0EImNc`QbQU z5c)fX-g}izYUlmX$1YyKSE3Z7X>XOPMSg;!CZ8>EZQsl79anmgCq0w4dKsi&xTSL=V3CETX+J+}byjpGU>jw?1fAw9Q|QtGbjh`aLPT+wsF)ewldsgMG)zF{$=YtFBW}9qFT$ zQ-!FT6gWVHpF{J4>nVOOHS8yRoM>Zy)R1ySM`Q4vs`f2%l6WjcSm<;5Ha?k_zg)na zO-*wcg(d@i8MFil_I=p$1zDq#XN;zLg`NJO>9Zam+!zMZ-`a%&#!lFr5dpPOaRzLa z2fH6@Z*8$l3gn8iXjLQkdssOEhy<6+93f9?u%F0Kc@w+PKEQWgXm2Vs;02AaGk6A%Zj`$_Fxo96AZ zj&{T5D{pq#?5?AD=j&r<-;J3fX&h!8k|O1aMZ3o`UiXD*DZ6KaA1fd0s<_+r`l-k%ac>2qm*Lz=>E|wORQ(E`=K3jN|d*F5Q zXavc)gtV=R+saHsXeu48g-3i03(7lUu?9zu*8HPBG!_{-Xzc{cCE&Ky4`CF33Olq7RBvnPPdK(F zl&m6$87K*p#T`_i`CD~phDs|lAhj*Ciz?k@{y=$vLzmMmdakf*Y%U2e=VXKE!&S4P zyBCI#x!!$aePq(Wy#Lm`)$kCf5!`;Bd&tbVMp&j`pACKSJ^}ta#LPSX5n*Xnz-5YO zJ7L8fJxU3W`5Q5K$3b>su4t_G_+4B`mh`C$1)k37;xsfWI>AaaFV-Gj~lh{NTvYME{J*Ui)qrxEI&UkiO%d=L2h zcV*p`i!&Abi1;_8ai5_4)u$2xTCF)=dpghRgV`bGin4qyaE}upH|+WNAN7;df2-Mh zZdpH5bOSKsOSB4Z<~~K!;({zda}>nzr&+hv<7c7VifYB7=;7L$JA=9}BO-OKFC}!c zusW0wR=-Oywq4G{x%zR|pXKnAIAiq(^iLFNgb+Q`IsMPVn72c2a5Y@}hH)lRG=ooH%*gdzorT6_t&4Q0v! z)aCdkfu*-Ybo8|HEHN92zka{LytZdq{QHo65dkOyFL1OgLzJn@8$N;DyzW2GYE z11CWYXH_14)7}TRlYKyUEiNFrR`Wz*@yc4xE$;|7P2Et#P zhucVW_%mqe<{`;^l6mDuK{8RgPJkqkR6%)1Lj^&mzE2COer^>BDrQ&U&jL&f3#njN zA9giEIo-&|_i8PSX*R2SAkBe}wt6SP)zfj_RopyA4THbPiW0ywPswy2WVuZ?k?Zq~ z)X>#|9UId@Wn}Pr?1ixYhltt{y+^doIq4XufT8kqfvEx8Kp)HC?Ouy&^>(KBF#Uta zw?I+ecR7n(I}2%;R!R6L%`qXvogZyA6*qiuq?Hl-;?D{E$J?6s#)O;(y^ zSq}=pVmw5|9Ikndr43~N+!uTAB#IHw69b{9G?ESl+LA^x+G1gw;5fIn%{^Pz&Kd4e z*-{i0QRF7k^%6DV+M+@Gzq%ZsMAeY5PJf8Y(b&=zT#W7=Ro5|i1*VIhnE7voo}jGM zl1wV7=Mo_?$1Uzx>8~eLvLk`8CX`hInY5teK*@R^2G&&Jb{1}1dJzXcVokgF`f!jX z{ZnE-AYlSS&R!TQv06Z-z)wRsySt>07Nz1@h{wiQ5IV3^wz zSv_yU5*2MJ)F(Y71B0f>3H{_;)npzej1~yWOKc$%501D!U@~c&H}*d%P*JUODf2@R z3vrPKfh2Mzep%tfw=}X!#FK~4ll*v$w3g4VfW~v<c@@|t%ddgpkleY7m z@#~6%{)2{Bs`HJCu}lr*hz}Dq1h`c>+%mY5-GkC9uJJ%hAl^k>K9@=7*0$tU*q4kb zwRVz(WBwCpX+P5T>KPu#u(Ov0HI6$%%0cw~-fD+Amq+)V5UuqmuP zh1AsfP{Et93C`*4=NRoiWH{XH`8{o*#Iwimjh2DVQx_Hdm}G&qH=fn#(x`Q72c8U9 z&4?KfF9DKFc`f7;g9pOZ!yP#&?Btn|WO?mk-Nu)!ZC)K<~nhm=VL{leu-=i$14Xy49YZa?szzC?oM%ypU}q2j+?2kAVz zy~6{+On*%VX)R+&Wi}l04b8LI5aE`B9D2olhwix%U4I1Qt4}^%-36@in+u3@b|_F_ zZZ|)gX{>K1YN)3;IB3{t>gYdNpNabv?pgEzH)CdF%9!V*<20vDeogay$$$;nt8{%x z8TT7}Z#`@~yf=Mk;RVR=7#U|f9uba^v?S`E%_xX?aA1;A2(DkpK8aA>HfdEdbe8!R z-8k2_iS>(GrCwrxLf z=$J|}&Emi!y6XQNdwKQARoTSULxiD?*xsBUiphVyivC{{rS7EpfReVbYs8$;|NDFI zDI|V=x+mwLs@1J6ZPl57JtY7A;G*TduKu@x`7T{Rzto|1d=N14uRQSoZPWRmi%fJs z8G-x9p}?6;DB158u(_i5sQ*pPvLQ=6pf})V%Z#wH_CcCHCYxr? zZ+=*2lao!udqF{4A~!+AAlK|9S{4cwiMTv0^XywD4?vbMk=Y}iaPhZDTI=M96g&yt z5^=@86qm5XD{PSkFI4@0{Uz4&& zT*0QuLvorgoMETo&AC4LRkUMV7cQx5(8@!_O%U(5z&`pvABgla1ocIl)~{b)<&}u7 z*LeQdC1RMO>4*JFy5%i$`rs0CJq+x)arqYXhF&^WRC8?YC+W-jx}9a!@Av-vf1@sj AUjP6A diff --git a/images/sidb/database-actions-home.png b/images/sidb/database-actions-home.png index 6c272f4683e4963f0068b4d7813fe4c20a7897f6..90796d367a84a2eab3f3333077cdc6189f979c1c 100644 GIT binary patch literal 211501 zcma&O2V7HGw+BiSL_ri(1QC!bO0UvUKsqSB2c&oDH34L-NRi$_rAV&!(3CW%Jb~ZNj8vJ*@+uGXLe23lR zyWkV3|N8YieVg#!_73(h?LF-MHVJ8#mJ2s7FOVC4A^G&Aqrr{l9s6mG;P*0LEt6-g zJHk%A^GKj=)4GtX_R*b)d)(mrOzH&E#k%F^AL%J6yRNcZKw9!izHO7#sJ@MeC96S3 zOs-!>Q(l&({-Qu;%s{_PV?R$)b1p%QKGIRRjikmuA^zLw`^m?l4Tw|}1* z8i|VV)u7@ozQ)ZyEe*}9xg#e3Nu}e;4b9>UIl(<4G8Z0EK6*(*lPCP@Gd_%*CFR{y zmh;s~0hE+>0(tC=5BWE#$-m~T)6hIGy8LuwO391#D^|(=!P6%kGoUL?|*04rvEz#Af2pE{UoIO^z- z+$CP0Cm|!HCm|+IzhGncMILPDnKVrc4Ts-rDq=j9=2Ywz{cK`_YU+0P~< zazQf0OAiM>TlOFicTZoLAbE~IYse6#xVj{p?J}$k)L~*~`Pj!P8ISzr>aM z^GW}&!vBu*uZE9Zf*jn@7{mv%L)A?@jpoZ*LwbWm$;w`7vzNghdC53Vbirw*_wBy1>ECEpUE>eYcxEzu;#6lgktu zBInoRQy%ln{DGVIg-Z==ZSyx*UEJE<0V&jq9W1s5LTk5ZeCMR$MO%H>a6(p1T;dd$ zl}Jc``4Rd3Ui^LUr+Y_xZ(7I~`mg-1J}CvBVYeBCs)HG)Rq-TcOC;O6 z33=5%#rgZ{pCeZBr>YO|E+INNX18l!ux1Xi$DG@xRX#k4fPj3I> zB>fQnH1Yp3g5q=HCvX#W(%3N}4&J!m4*7FLA^vnaH|RG^@F&?qNE1m}*1OwDe$ST9 zLU|tmQ0GdL-woisN_#ilq&HPS;M@T+xdq~@UU;~^yj#6gq1K*_I(K&Xf`?iPrTai1uPmA(FbpPdGp=_6xrB_ zyt~4fUVVGFimckMyWWeX8-zcxZGIa5izN(FinN6I^yZ6jIvlIn+3zK+@N^$Qxdu7a z;@-}s@LTYK-HcNmE;?!U1z6b+=H#;OI@3WdV9Z7Zz|!j@s8&*+gh=&n&x@3lUSjIm-ay^5`lFN7obe2v=< zXsUT9LOWAJPtdw$ZCX%o+p@-Im>+%+&+0zeqEqjVxV^*+yguR>U==5Lt3WwZ%3-Xr zN7Q|p7c;9MzBHs8K&S(hjFkx07U?97bVh$~w`F^=5jLa8ZhdyF-`aameo z|7ix}j7g~61m0=Dik(SEI`W=L#rNvmxX`k|@O{58e9PKsI!zKF&3YOCv%3-HkS%A78==1DyMldoXAb(tt|}`l_2_U-&TQ-&pW+ z3`I`J2H%POF}urbTPEL83nEs|pzEDzUSMG2NoMD_E0=n)Q7`*`aH`o4Bm0(*_Y4BC znw1lYH>Et~e2R;5LFRLt=vc$@(eFpNv$*B}jjyZaIxL>`W%E@Uf`B%bk^|SNmh?Ia zvu_=pd~K%nO0ldsGZ{aW3wC3)HdEY7AF!QHKiO7jSWnqA&nEZK5ti-+XPbl=Ep|sU zE;sJ9DXOpb1Gol4oWf&vw&b6wXVkm%rV{Xo?bJI@IMQWo>z2h<8g7DvEF`_AuIEF$ zAxeqixS7gJ=lgof07npeb%Orq0BP5aSg2)z@#lKrP>lF%>F`so$_D@ph=E)4j)Ue^ zuoP{tg8Gml*7x+YD_N0|o76MKKf0G*9sz1Q09K#7We@tK)5W|#bW08-R=rcktWTDw z)^3u|7U^LU_%jui$Cg2;v2`)Y<4hqv!5&w#Oy(h4J`&4Hbm>Tu8ZRUSm9Zvd!IG#^ zf+-p};}Do`u|v0pKXU1!SuZ%stkgoXw(bOm=+_%kGbyYAdr~SVfepZ`H(L-#x3Frf zQSU#kjJ`2=2vlzsRU-sxr^|fZuI;sqk8g06h8$uG9%J?zCC`E;*&)H3h2k#5tKQiU zI2=BbN_zWQE(2G8-$wGdPdzl#%|RP_RhF0 zIjntuvX7RXEpSgyKnFug{Fmbt@l}X*bklsq?XOYmB2O7tM5cNz8p4qm@G@$n_5xNF zo6xD2PP4Pe7+tVkOr`T~L%`STg1kD7ESNpvDL1m^4BB!8Iw(A^KO1^~(yplz4oSJV z4C9Q#qLYMD`x8Gp+|xL4j43dIU#ba#h4f@|mL8uTQz8bJEI(HR6valUWL~|#wD$5S zeOeY-l5a30Z>x745VAE=jvMIh8t-OO*%Zhd83mPz^^k3oO0UmBNXyO& zk;A735UruRm>sB4U)~kASNG?9kwTNc0RzYvhJ3C5DDw)0V;^AV$dWu0IEUf@l$)}5 z_L-ytKqsAG!A7c#7|Y^^W(*{kJW+|QWuE1(KKRAT{iYK;h7^)#E(Uap!8P_3B+oDk zy?lJcX3xohJpxwkdc{|;Zayzb26#K*Q%X#g6^`e- zYuqq7m%W&`#z%U@Rw?ZlFWnqMP}+!}1e8;TyGaIp zJ*&yB9`p}hn8<%H%f2*O7hDSWJr!7+1Rxy8Z8eADe5EbZmU4HfkUKV9I6Oj^b;iZ(CY(RDh=D79;MQEqFyZFiJ#7~x zaNaBB?X})FDe!SWJOdS|4zaOMJd;DS9zo}JdiQXOlSi_W<=4s&w(GiRH+nj(5L+@R zyT!@4EF=9Lu(>1{Ft|kQBsv8Zkdfju(dL)EJx^$j=Nu6t9`Mz6oe3tJlF7>&iRxW6 zOj#9MzzxIVl}zxxt^0LZ?Tt?eVh(R{_V{-_tAWK?%Y&Jtv1Wbd7-18noI|gMoA12d zp!bim;*SM|;G>N>vzO>_K#E|R3wC@DCR~tt6!)S&zRO7}h(ht<{yYvPRZ+cG{JA@4 zr|o^3h@Ix%Q&s!%MTqEUFCppE(Mwnvcf4zXbFfp<`ti6vfV2E@US;Iy+0)Zt|0ysq zYIfq})(qk6^ITVJ14AWs*(4D?Sn#@#9$np(Guu!}5p)=aRLVMl)+h5Urkp&}+=tBMGy1x-s$gkEC%!Gk%EI?&TKZwX<$~$SO)z zs`OG;aqM`?3&9gWp{w&CMyR&3!MH|6BpAGLFW{&dNQHpBJhILdgdc5}Yv88Q#gxfk zWiDYr4-f9A_Xv;EPfWiAip!`=`Z%8krFlq()v8)+-rS{fnj72mnXm*tIb6A`pJ%UX z5Gr3mvp%o3DtK(EkSX1*Jp!~aZ<*Do&vakYQ)IGTHPPXz%&-Y5oKaowmJf9}Tf0(W zpmA9-$9dJGwa`mr?&->Md&k$B(}O8`L{hG9j$As;I^4h!X2CyoF{uuiB*lG2ysB*T zh$G@Q;U>3B?MkX(>uEa8o?`;1h4IZKOK?#AmdL>)B)_5gwBMFs%+0a)l7(|E4HE@c zZe~X-GFY{R$wA)b*#VSQrmNDy=6Ezw&Jz+i9PVY}L)7`i!hI1F0{NB2%Fw>EVf@Ll zhO8rQ>1?KUP`%8w+Jk)z2rcnv)@n)}nKVC{4xVNN#6LQ6OAk)b(~uNvEA#@VCa7N2 zXqBmad0zqkMs+tT{_H4jx^|FDJpA~~(&@*|uWC6W2kOx$TFJ){CvG(~zkdI_B9#{# z`el|9wLp%O`5}BjXsc=7%4Up;+aXfO`TuW#BC&15N1xgDIyiC_Hx>@owMwRu=3pg8-^qMsK711%*Y06l&=NZ zDd{~Wwd1U|54ZmCEaFZBqmJ=vp=>E^<}ZU&skf3Pq3*@{3PR8{C41-0l*gX)ciGQ3 z2F{bl&DLcd41rp=Y8*-@UOVo)j)nrQKfNuM#t#egnVw!V*){6B~h7R`Dj%TQerJ!M^|l9WH5wT^OhSg4vqoRnuPGTd?Is zp+h=fwavZQMe;oBB4aB!hUR$ zy6eaD8LibF=33u~1@vgoD0FuB#l5~0Juy;<7@$U1rj5{U%V*tK3SspQ?1$AUn-dXXgPylmz>i>^h@B!&Y{*i`{*SabHTcb#(=m$oFzEX8 z6D}~Hh@SGli`6IR>pW)lOH7K{!8gJ2Zt&;1#)nU`USUN%=bHpjGMJ?F#IB6x@{T`k zCfB?5AmIi1n+zjISBAxP^Q6lAr%O}~ip|Osw&w6)oy8`Q0K4pIZTf+>_QJimuYi-l zY{9UCFeW~WlE&k%U9gdBw@!|rvf7!+lL@=-#|p#8L$b4>eXrPPgdZ%}jyS-kEnILwHZLCuQ|136SwxpI{UJ< zfcGM@Uuf3@+h?`4(yI?k+7p~D&W2qW@T^dg_Fql4p)+dGxM)#&k6z3BdiG>Usbh?8-&vfrp2+7E`w=XqfY&sjFy}BHQ|++c>&Z0^`@DX ze03G+<+Pjd_vy9BsReW_Tj2CUWi8>$VCtK>hpx<5=~{9rVL~|ktE&Q*?b{*C-@W4m zL1LSWnPqG7UC3%>3Wec_Iw`^s3wru!vtpsrlMu~UeOWeue^_)XqeI?qAS}+aH_c~v z+Pc)pss(aiw(M-=pf~&4DqT1{*6Y195}ij# zCC7D9a9w4U1<9%N_W%hPoiKGRdAG#{JX=GWslFy=obJptsy#d8@cnz9^KxBWZ|XP4 zl<|;Ixc1JJMFO`5f4Pj```rugu&%-5R4#h@)Xp&NapTWQZNhY_V*Xm!?zv-Rfs)Qc zPs9RB;T})PAszm0!($7A29T6{(IyQ&iLyk2SYMSHC%TelkmiS;DV%8Z9Yx-h^s3$T zmz}lF9H1*Je+m&rD{zEo$r1oS9|7(xQD7`~?n^DdKf26}i~T+mAixV@}K#p zv6<1~mkR>evWD%Qx>^&rI#1TiSHte=y2r`jDi|6E$-S^HFa0bJUKt=@7Evmqu0bSTyQO<=MiM3~QR)g?~;PUK~_sUHiJY0+@004jq*hC8?W^X$o zv(RYlI+F40w$IYbq7zsqU9dwe_Ra2%f8!nY>5I-r<1~e3-dXn_MCJXot)>nX=_HR! z5u2bU!jdzyU2x~O7V-Ppg3WxT?j>QycL5V!)?a)S0O7S$JkJYslc(*ZbS`PVmn(Lk zy!!Y>;P6)QKB#h=6)_|H0C`YI?{<196wFTtV~A0vVa30ZM{o&3K$Es*W0<5> z-lm^;IrUp{0Ym%}cz6u_h7(q7nO}v5e+UlSTT~rh06TJQo=mt+jBvibm7&1umwC*y zW(|30mM9^wT9U3mDf7dBwMEsEsk-wmg?%mMHo--V%{{UOQlN6OA^FJ&Zc=PGY*%Yq zs8Tl(2e0YX?MvYMRAvSp9;j~aPTByCQygYbR63bpcUNi(3LdSz9r@siX(ba?E>Q>ex@*ErgVSNaFjQ*;n>qby1>|9&b}{{;wPwi5qO>S-o_=r+?Vpk= z_{6dh(n@-rFAV@{J`8;g%?-3^8PF$2A+?OOW*`$|d;XP)JW@HCSOOA+o2yXo$MJ%;rCzG-oGxvIdx%;&ks zkZ%L7CtIU>mI!!-nN76(J2HUspAi2EB-r>~ANH>XSQimwGt{p8NENFn3=Dl3mZ zY=-n(LQ~k(fLNA-mAOfkbOj$)c7kZ>?7_8kO)9Vhx2C@bDsJtc$8TuTy1EuS3fzS8iTB+khSYQjl=9Nu!{AsAjLaQg*+Zp68Zi0DtV+4-ZT|*aWD@kb;($qZD`0fDC zZfgzx>z*bR2Zzr9_OLOc%Cf<6^vnks+YaGYwpyjmaoBQqt7rEl`5{zwW zTh2g(66a@TDY`Adn=BO_0QrhXAmIL0zNPRz`22Pt>w^h5>NT1V+Nla5TS0#~(YP)6 zbmfcOcATbU`Ubt|*l%@59!pa-C|>JZaI!L7Snxg&HD|}xOt|4XQk^Lo1QX0q2#q}4 zkR4LcP2o@9nrRwPxWuPC*(yYP<|=GvQ5=rsr|r8h-q)yEekvL|t&$1boC{Qs%f^Yn zmJhSZy^Opcu%lKRpe*a;ebLuQbv{tGdB>Bi$;4qZcBeo&HK(4YW?;VQsPvj-Q20r& z1knbr(9V!VSbq@l8rHXy{f;LeliAFkL)`1p)Nvq%WvZrklr3+F2xP*rEY1Ud%B!nt6v_# z32`Mm(;baCY};w82D(^b+si|dm-Z)a#&82;pYOXK-QE8&gkLpn*kb8&8q7(|Sna)n z(yNVR?9bvdZ<2BPA?o9dH14se%c5on>`&^EFUiFD%oYkb5F(>Q*GJ<#(B6sHJuF}4 z$lP9_!`>{P?o;D@8FVYx?UfDdXGA~W5?A04{f@FD%erS1h6Zot*?r-@G3wZgopce* z!A9;Bg*z54euRP7jon*A=(qO`y-R(JwRaV3Y6gaDsw z{M^h*06C`;`+;~}d|Q~&(~SLow%F`D289ESm3`}9n;lkC0aNss%zYjpweh`%5fsG@=)x+zq@a5~eQ3fp}=i6xXP zX|*cC91|=@1Mddp*3)VOd>dnd0IO>=V@;zhJ zxhWOzq|>szRJ}xAd13!7yS{uS4oXO2s$HctLwZDZMT1F)CTOJHF+?5}x@^Pasmx zWkWlT-OHJ*eYz-g!5DeqG2RxfeM4TKsE}6?y}NB8vyW2^=g>#4HybBF%+O&?KNg7+ zt>#(*dxZ?)|&LcP5}rayqZbh2fmg$T+rvz^Bg^jW5Jr?MGuZB;W?6tAgGOKsf6%G z65|BwX-2yzQBn2SJz4k=q5m}!GngZvX$cAIG;*9at3X;^Iiq9^3nzEsiGNLwBiHWjvUeEAyeA57z3p_a24bu-b7z#UC%x7Q7PwT*6!TyPI)h0yqKCq(V6mDm3+-* zQi_jRN&wI^a!-b2*OtsZPsDXFFAvNhSF z$moBI<^7CL2VD~$wLT59>bdZ5#QyI{BGox;e8Pb}gcvTN>qT#V)$N~=@juw}Akem? zh)WM%8~yus{!b8HiR;SHR5678>GJ-%jKAJ3EVmy3R0dkyHWzo zV#0pET7Gu>$g*?2FI!06i1di$RbqEZ__nCEi=htUzJKml4gVx5I}1gaKga)xK+BA~ zF=xs)6n|6f7c6#nAr!N1#r0eA5chLLL}hL+;OY%Z<-ckB?_>Vh907eyCeuW?Bn@<2~?3mf~-jBqD;2sr`)p_dP#()DG|N{v*=m|FCZFE!)-3 zlc?>d#*hAcDCqC4$U;g~ySAm{AiuAA`SmxlwnEj#QRLYFVbhXPoV}p20WtoTeIoDN zZf9wKvglxc`L~mcxbT>eK2QETwozi&JWpgX&6amGWd9xoi(KdHO<)f?ykZ(FE`1iiQS zX8Xds-y^zSlacX|r_m2RS@5?%|9cW2zh=Ae>soq+N^LC=Zm){11a#+^&uPE(Nyw7L>;pXrWc z1Nf|bcMT65$dbP)(ACe5>qJhJTi{Qlf8lQb$-3vQ+1(;*Ix1i170_-@B(P>re;=h@ zq>}#98j7!R#7T`*XbBB>$CY{jpxQshFw8tK1!e&SS{D8Awxm59$P(4OunJqLo(Zlw zEu|f4>kEw=he@ZZc>^j3O7|Z?S@%(N%>^Bs9;}_WrMv8|t7j}r?{Obv(*Wl#{Sv*_ zNF8?kK#}PTBz5zwHDIaGd1!yMN6cX*qjF^|9qeOn&YjU}|!f*w?+{vil z7rPUFXo_#l@pB%1NSxVY{Mn{g@e_g$wdKBPpUYsnoaRWrE+33}JkdL`5TYx=MGk-T z<8XmEhb-N6PmbgWMW*)m8%K-wzmPrby%)IDs(&!1mvbIRFWC;oAk=iFpJz(-!(BgYxHe+mo5zRG#I34bvdu1L(2AM zxH)hK^u+hsy9>la?qmD#CpCw@6ak;(U0XQn+ZzTS)U?EK&z4>1+and)P%IktvB7<+ zM*a!1v*$07emVJ^Zmxm=5t)_bloQ+qQaImdT4~*S_h1#4_y;wECWk>Uzgz|gG+Z3G z*cr{Zy3Oi1FU+=eJK!}x^5n9zy~#f$o&8q+`jv{IjhHU|x@QUypOuoN(rKmbx#rEI zG?Pl}#4cfwq~5pv36qo@C{Ecfvrx zhk|#Rl4YC-D1wg|P^tl*d40#$Cmn9fW4SV=p%Y$`$vLYmM9?Jnpgg8*Yq}OL9Ttz3 zgZOG90|W}Ru*UkIZ4@A$y?|U9U{^}+_D~_@ENs8-Y_B`_0owQans*(SeLw+rjId;y%Zd-Y2J-<@5<1k;-b8R&uz*QAZY3KFk8Pth1EPYobdG1lO}DoYuPC z1jbAIQU+IAs6jvnDaZ2+>oa(P^>J|Par#aLLxw_2a5I}0mgxVl^b2c#ar8ayb}JJG z;6wDm_!(8{^aVo4TizV5&pqj7+EiI3fJ4#Ssas}ZT9A=Wlp8c?y~GCNU^_MLvk4?1{gv+`?K6!kvoxEURN*RKjCef4r6SnEt*oUb-?? zY!J8mnT5zl#9bG;Ca}Aj>D8Mh?VE4c9V_a$HY99I{O6GlzCalM)_u{=Au)sw}3CJ`^8tl<@JBs_q<6mZ8>8kcf$#v~84vNYb33+qqH^b{mQ zFRM%I>inQNJKwtmWb+NHpI5*aHgym`F%kP4C>E`#<@qp(>7x8K-rWz&&wCmLPc(1A ztFvgkK;kNXo=fdIg*6skh$FsBtwxLp(UzkI%t;&&O27Ujr zxj!~5CFE=HcU)p9yQtF)!o3ve*kV1?Nfj&)laypXZpLSNjO@ZL?fZZ!_t%%xeRgu7 zwE1nLXaNKe+QF&Ax8CHl(JzD!CT<_p0UY61Z`Ri2IlM1~YO0jcc*jy+d@0;PLkfnIf?)(}SO)JAHYk7j|?KRpmT;`9s^}VGtHsZ%s`` zA^w%VtC-0-M56S?)F~s{O~cgS(Qb{Dex_EB4Rh9nNYDIqFxw^S^LSUA7K1@j$p!hq zJs$`V8w+b=74xYoRYX2mUwq+yy(+djr1iRJ&^Kque6{#x{7QnEX-hG`!o|vRNPrh8*8`V=xuCQv>EH?++pX{&v^tG#kKd6=LIYYEO z;;?-f^%q?|96t?oxukYvZn!6AXS_7EoOyrT1fWr%nao-;_-ghficTSDL6FrC0NiQy zUg{!egCk<+hzocq*Lv&K%0JX3qLZPqFu$rEh6s=o?#Sx-GH_!n-+uwl_Y;*B=Y%c3&mdEpE z`0`gUz$?Z0R#UUF-kEzLpK_RW7v3u`>7G$PxTV8Qg1ecIhLhbKOL8>*wKQN6D&5#0Oa2dx{q)T!68OfBPO zA7G2E_L_RKW=$a%T(Jjx1|!WDp?sF?Yb^aTtS}n<36g=;i6sH^o$hp6>9}~*slYPk zr4MF4I-_VXN2qk7ED`2*U2DKSy&xp{ZB5uXbEbFsRO|H1aM1@{IvI{^n{h&c{ovFbx*=|>{*Nn^!zNu zx2VjbLHF$JET4F^Ch+R9zPZBra@G-ddi3!C4tAd!0ryXx5b2gORqLcx{j}4aTFnXs z#b*Jdn&J9y5cM93I+>ChBl#LA2t^LBsg{~ssU@W5Z1*eO{t+CIN%vH$Ks#Mjzww!^ zZ0^fZ43TA)5nj-10@CGnW56S?=Iu<`K+ooMM;))pst#)hlRD=od?poHrDv~4^_;u% zfo{@?B}S!hSxsie5t?zIA_I$Rz7d5z>MVL1Q|B_;pX@WJwbT`ZI=elv_aA5;5i05k z%iv}bwU1|&5BZ+2k+_9zXdu}I6slxUzCHpH&x|_O+?!bkr>KT``yXFk=+`>xVDSx4 zJ1<-O)wnl#ZUgp(p*%_#lfZiPG|Zj0XM=PtSvWTZ&I|33j;9}^=EW~ZlWkk5g9gD*ngye6uv!IX%wYVC!Q^gD& zBHN%?JK7=RVtrNd&t<-38;n2WhOPEJ^zhU&243B(xP5fR6>YIq9o2eZ>Rc{mHB?65qYY)l}c3SY1@W-sV_x? z2V$bFVck1xWjYtsHZ({>P0uw{B*|b#UZ>lcFw~Yx^m2oETt@QW$mdWd6WudzilZ%< zxzLYNxU3Jz!lmI%1!Ag03(kEh%z6k|m>ykwm}QW!Zab(I)yb?r0i5anh;@A?1v)#v z-PSIi=L$=69@-{3}D)r<5X+e%_*&SA6o9ZU!iqz$XZZRAvBV9e`+gb(8${%Pnb9OD>u)s7u0+v97xh5l2UMm-# zV;L~M(q}@v*JHT#*?qEvorOvo}4TKf}Gckv% z%ymm>hXhQiEnvy9%6E~np;gd8wp%E(N3g=QqRz17@uQK}?pYSu%rB`S<@?1+oQt zDH6_V#a+j~x9Pcd6+0eosA`!PKP+{jfQuiO?sT6@FA~*6$%tY`Sr^Ij3R&OPcc@SP zn-l1R7mj@|#~vP7GY5s^=#_+w@0nKR$-lZK8!+ix`S`6}E}U zu|z-V=#ZKHtKprVACqx)(o7#R_N2Ezb#%;$-!!&JdAE)Ns+>L3)TDwph=JeL3c4&h zp}yXl-|@SC4x$){e;DuZ!K#X|V0-fHkb)0T-gdRp)PiUTj7?Kw)?JBVMiI1%2T-U-NGPxPY_>3GElVW(YX7zDd!g5{ zXRoXjr?3m4wU*AhHJ$~|dWt#&efB2W+&zv6rLk*6KHoB=OLepa+}MqxK4o-%)Tw=P z)JGI-bO`q!#H51pl*eQ2g_Lh3#3YT&Oam-%K88@5aUtQO4O!gB`*pafOEVy6o-)cX z|8N2S&pJ^Zw`BaPNH_Ql3MYl0N!;vqVP+vJ0QCoBjyUJ~8qn>MNI2O!vWoo?T1uK5 zM1H4eS>N=*(VJr6zVIyz5A*8=Vkw8wPPj$W@-I*AWknQMVcAaD5hRnC<5k$ea4;jd zvOUt>iEaBYrv;J4Xk%R+E@W3c5VHfjP}`NuePAlv-qFdxm84hnr3y75*6K5Em+ho8 zyN{8$EEMh(4`97B#9i`<9<_jGqMkC9S2p^}@yEssllkzS-~!MB{xH~dUW=HPYp?N96UDmwwrgR%dA>IYb%!i zQNRAeVLH`VZkXS?sCfLNYP9=hQz7Tk{W0CqG5~FrY%cWhsYLn4NQISb=y3YMN!L_P z=syjYlJ-c>?yVVcu&)!{_~m=)`)$*HtQHWjf@`K*|^zZ>S>S6j4D~8jQf^ zYtVv3uCA4{k4@mE%5W0)`c_$_eb$Cj*HZ=3(!$ViPee043BVS%zB&rAaL>qk6S6m*?@w!qk)s?#iJJrmU7xzzUTTRUcJlurz)6`7E{A}`jk_hr!JdzX&IN)@%~reW!iVf@*F z5X678o>n`Yh>3VzIhzVNXW7_-YL3_eAEcjrXa%Evxa2QRcz9Qs7h2Pp8y?a_Elb@+V)GW~tVX z1FbLowTV*2Z+`a3G>&azgr_1q_@}MO9?`CR{{BiXF-z2~hLHuNXCMR;lNA z$Gfle-=7~8PQX~f3sGfsCCjCR=9BOu6^mV z8KH$O&{?DRQj;P$?&zto08{xCjFM9%CbpjZfxr@fV|+9GF9VHi<^V^ zc8`No$}Y`_t5@jMX8tp(q{0{}1g}Vi;@jM{7=+U0H5R+$cKp{Iv=~!Gxyk4s^62Hd zHOTs(zJsv?C7(RhIC!{WgTdKl_Rz(h&l%Ym7 zMrWmJ;{bvN9FxA6YCfc%i;VL>-I8v0z{yBN&3>|-HAjPQjzEj7pe~!2LUV;o0)i*{ z)mR$WO~neKgclT%n5AH1ypz__O@5#5%@y_tB|padA%~w~i~gcMS@dr!>}(`K@bj9MO1_NRB~5;6mx2+ZqKB|Blp{}`z-&nQ%NE}3qB)yh{U}R zuuBF}N%t6a6R-9p2Gbyj;Riq6>7Gxo5bbMHJm-%)rwN5M!^n`~4WerRMqnZbz$<+WXXk1IR+P`uWZWWg1oPgj zJ@#jxe`G+`Hguw9vJnk_@^tGRY=v?MJ>g;>pk}R)FDwr-*dGwFUgnHe)jJDU;*REM zK!m`vTl1h%JM7j}&G#D&+Tro5DAQb~iypvv)SVX>gN?mLY_{9ythkH$)?0wxaRJTb zRCH4{W9H98)?4^k1H5zIa%v%UL7z#$^5+#faw8z|(|XVwx6iYjSj!F&anjEjhZ0_( z5Dg9Yvh`y!sLOmeU3tm}j5g=Q8%qF(GE6*8xGS z$WAdi1N_{YTi#`aYkji%Q+BOMDR8+rxlb39N5l>i=pa%~<`ftQz1}6_nLZ^Q@zW~l z&RZIS1VfJ(FUl%b5BSeOe38Pr+r<39vRp?k6QTza3_bm6O*xV`Uw^q6d0_UaG$ZNG zmBT(x*xi77YSlpZ=l_jfD%CMeg0Z|Va1@Y9z6F}Ek*!}}ZgJPCe+Dd5=|xnQfs6JmX@WI`eEW{`pc8! z$Yu;{JXx@>=2w~V0@IF`nzLyC{U2-8O}*FL$9zw04;X9Q28k}YKTNPIslFp0tpbTo zyLjD`T=v%>4PD z9lotDv$Qt@CG|dX9-i=}nWI_dH@%CpXcg4igx7Q&;Hcs>OOkq4UKkGcp%8(~Z>e+K zR=rI#8th1UlraNj=3i==ujt>hBJsLOh2#BdwD8or( z*INgq%K(ozA&pW_>69O{^@j#d1+AKb>`6CMq2}l(79G#eWy4rE@U~r7M5Wv$7^|*=rB7Jp46j${#JxJv^AJ$StqS zuQjL4ftp0^$*|QPNW)saSpQ+rMzn{Mm9`yg-W_x2pGVEuK}W|HtUFpD+9dbWX*bp_ zt9==TQeO&8;~PeyapHGO!MX-!1ehem=O}}ggRwUiL1`dyQ0DcFl4p zMla=!GZ^)I?yYDwPwdQwD_s{Lj8u%0xK1UgT5WEh%-6+B5D3M->u10R92gXnYoR5~HAnRok)aC(CAU zgB+29s)ID?XXCnqR;lHAx8O3a3@ zy-upoH}Qn%czpuS;xH_;-AImR)=96p;a=JeD52rkI**UtDcT52K$->tFl;fSK7t$AQe zOTXCyJyk7A`D5Y)coe)Ky&)byLrlLwbPs({wgRq4QLPWI=edj)q8rU?_K0#szsb)9 z+qprEEvaak2$|KWEPkVIyr72}mX7dOSR!g((V*b+^!uj{WsbZtEYkY=MYZ(x&i z*-22hI~Ze;;`(!`>2zT~kVEKS?v7vciP72^@986cHB~wVKsx>V%e`xNEdgcTAgNJ# z-=d@WnyL&!yI%;+eWtgkYu9$5nbM;V=FPii`%?uDnn0i;NVoMkKwH)aB-%MZYJCns zX36I^)cOn8<}5(n54X?W-ju-ax>-}g;-GfE&f|IfV!75lMKY18onZ!H(fBXrc_QS> zHh`IsqcsV3xR0%-aAl9_e+$^_pqM=2oqNOr6R+FhT8NCsE1$W9A)sJISUpz_v|lW6 zx^0+`JA0t<;%k>`yrY6UGFvE7_8d>Sw%-{sH>!wbYgfH-xH~rxOKA>#$Rvumh#g4MN9~p3FC*R~o(9g!|Y?is&xT*_@44RUL?j`F^FcJ4Wr&XJbxO z*tH^k39sdJ=d;-wpa_s8dK`$O)oPrYsdZVJIl8AyVx3>TE^F1aG52hp>8c?{0t$^0 z$gjdl7|A<768(aM9{DlNh(W7ZNEU|6{;u{i%kX~xH6sqM`DAX`*2JnQfc=^2aFuHV zqG$F>rEZ*29jb2ZdPjeX>*0Xgn0p_nThC9#uo@1P7hOsKHC;mqKnl{Nj+zOyTSF}t zYD=Cfm9cQAhgyQaf#mZ2w1*vmL7qvF_>&I+@HJcy zIw1Z6fieC}kxo&^LK>>~7Hf!W?yvR-gE43i(UGK**-(aIz#+}mZk2|}ph^DA%!&jp zm;ULvi}XF*&Y7A+52VEv1m6Xvl{&!(t^JnPk)%eGNebNj=^OLBJtJRs5O~&vCe^@Y zEl!IGT&696OElx`+w872Molx7B@#g}Q3Lu2SN#^7MWqZ7TNizZ06@fiCjkwp#45hp zo9s=G_s{}?i<92satx8GlxY+31qQGx{h8zcqn5M}iDS%bvzge;*QZi*|06!`om3+L z=yeVGaR`ZKs_R;;qGUC7F`gKpwIg2_f?c+SJ#{fTG2=h(v2`!`aCEQk;p2Qy_a33?H4fcyd(5gN#j zWaLXXYRM$x=`_APFg-opACzU1<6L3Zo9yjXZ9@$s2|Vg3?@iloTh{JQTcRuhq3iO< z5nv7UUtBuBasQE~=%J07ir#w^os{9|gwaVA`55<`Hs{mTkD`olT|RL-9M1kENQrj? ze>}4t<=e4ayW+1tIl6C<7?cN{{&gnRM}5pbwI62s{7;nki!;gGA*|W1;MRs%x${9ABcxlSKnSFVQ!NT-GdBeiHT%aYJDI@G)zH z1h~A&e^7czZx6n8p#4vZ`_E+eFVDP1sD9Y4w~XK81G$bVwQOrGCT6D)Rl?|4yRgdo2l59uXmPiN}-mx4fl4)|eUV)&Hu+@kA>Y z2i5n#Z3h6nHF8Rh41romrTD^MV#WQml95~Ini*R2HX z`3e1asPRcoh2VsyR>CPwBUAr%pO8x-`PcKX+n~KW72`&L@WBU~?Kb5(CxIm2@t5!` z?_PSV!&~d;Fo9z%?+sF9Z^(N~UJ{%PB$Cf@{3uQ~&?Nhoy)mqxN1*@ZIAJXzQ6Tw0!|euI<;eWqqF0IkFn<2cK~@TA58S==|IGLBl4# z)tMao52Nreo8bfrjl-EwDq#L1?qH@ue32UtOfTc;kLH#q1HiBAL%BRREgWzGgNe2q zvBqev&>6`@sZ@y7Q4AnitJc3i0kSOuD*f+IIUUlH&725o<$xq_Bt0g#-KpFLjr$Fm z0@)M92FH_hO;-17w+4r&UzbN`zh|?J*VqS%?t=85!XIHPLFUCMJZjhE?%^Cm^=`{$kJYOF0M7}q6*51(TK}L|hEkpm zagqPsnG)T@?rOBcp>`!8rqlwomXTd7dR1a>_*S|%Nch&6Vh7t{!?h~|(HJ0dttKay zYHg%O1|ZI5F*>IbOCo6qO=DZKh~ZRrxLb+jcX+BArHBTqI^r$7a|UPU(=4v-s7Xxb zwZ9p4+@-BWQ+E~(X{~|g6Dp#2aAFs`lj=XLTdC&{JM_IfVT719En!D@W_7y48h%VO4^r!K> zh&~*j3<7k(&6yR3xhr5i4J&pFH}8GP*Q|(z-}2G6fmVMD`dH2Zz? zKqez*FH(#=I*1o9Kou6OVZ{BdA!w)e6eylB$P@bI#j#r%ST% zmaoF}`(wxToULTk>WoPMzZHpts3ux7liZ(T-q=SUbB3FYN&3AE#7a;31k6VS@B>Qm zy@4_@?+L}D#V&lc*0uTXK!f0^K73+`!1F`zqkcMxk^jFr#eex#$lW{ss4-+@vB7jr z!TF(NCI{f!veawKn9LQo(DydItgbzf;C-+evY4!pLZFtb_mX_iu7N7>Fywx-(D)LG zh}R0x*~FrFlYUQI1eYb!#`HYOOf*HR4a8BKum(qD+*>ZMJ~#}(0Lyw0{n#A=IEmE} z;=kJ(?~jmxW<^Oi!}PUjX=bYjD3#XyE*tzBp07_2xYW6tFWU<1xyshq)4g^8I-I%h zFT})d?I0Ud>h?!3FzI6)=u^adIV?p1P&7xe_qft(kETFAt1^}z`|Gvps9xI`lHdp4 z13+9W5N-n_3Kotn=k)|Iq<;%ceMTGO&ZxwFg4VAUL!{~!t!9_sbm6jx?a>VFW!DqF zAZ*s$8FLk&?*NGs@YrXt51h|dMm@_w3{UXeXBz~guZ_ov5Te}OwQYR#P6s%lYU^D= zqy_`AQ8?_T&}xJ701uwgJMex1MQcYy;7>n=+zefbN%bK5CY9^V_V;MZ%7?sWn*;C> z6Cuxtyft7U{eY2_-AqTg0v>LxMyK}b$EuE(S65N7qj|BmSPXqjp3iXd+Ge4EBi8z( z4e4eVX%Ge2W^YQFkG`z{a`yJRt56AOgfG4A`|-&)+wo)>+S^xAml#r})Jtpir}^{a z-OmsGNw&`%r^;ACAK=4lvfpGTg6|noj^;bv6O% zzmA2KNpH)MXbo8=s9tKamS;#*S6KjHGCqd}$5HK()SSbrH}EpZPMNQt=^;wI5&tk0 zUXBnz$MdU1qH;%4FwrR)!mcoZ3$uu^dXv-H=SJ!UVy}Qi|frIR@=8Gy1` z0>A-i@M&3UbWA`Y{UQpa4Zj)Qa=?F3mA0S$M^y29pl^2|N@rXTA+~g)?QhRT5a9J( zV&~K5`vxfaeB3^ zj_#$cd9U{8Fy^}^OMO@1e-Q!VaxR(2byAL3U5XH7MkA@8 zd|JYA*egBtGUZ?Wj%RGH7hgX(Z&Ay`^JJE#Q z^URtU>{kH-M-pf@G$y(v-{q0Dl#-Q!$Zv%GM?8*)6k?M2&ryQ+68j1e7sLcyc0w?S zFQOwENQp6|Ym;!Pxbwi1}D$H=X)ZAPCU?qC?8k(A=`&+U|?X|`LUb=Y9eVN#C>YT>lRlE98<3NVzGdY5hRPCU^on0lJX2rZ7?oNO4XwTeg zm*1#if0KX4;ErwH0IL=QfysDnmB#Z!Ws!CsQghj4zCZ7UBuNsRVYX(mrzMQX>CfHQ z)bl)Cxpp)#i7S6wNi;1InH7jprgItt{Pm&IDv%! zchM=r{PRQC)5=8F{lZUA#&AM`y$`}qjT5C;DZsE$=9xSr}=Y3)@N|N7A(QP z1>iz95QJxww1q%q(yuKd4KxXnC=glc?0)iTzpryx9gC97Gh{ab2=9wb*p8Ym*FcZH zKn28-w&Yly{0+Yxk*C((t}jG%8#I4>>Al>SuoQhC4!9hp;g`L@o_*S z@IziN;YHS_Jv~TQHF!+njSQ)E;lQg5KSV=__{~)L-60uzzo>FI_>Dwo6sc9`SKP{di)DY3BR=M`jjefu1nG@!*Os*DtG0#sWA^L5=ghLUty-r zL0_(zA-WEv0Gxy=TN|m#Wh-POPg-xW%VhJU_l@Aa11CR^jpt;}ljO1xG0RN+?%?!< zKa!=DaAr3m?iiYf2gGc%qd?EsH0tPvQF5qF@ym-#BIfo~>A2a0~8u{uDbqqG!^w#OPy&g{aJCW!(+FQ&>u9 zYvXm-`do|&+5nA4lO4u|k>p38+q3eQQ(3#CaYBLy8zT4lsh;~SA!KA|XoycA|Lx({ z8RfR`gu3NfxO;TL+N6Pq2o@IdXGoi^OG6=>ex<$~<5yCVV83U+4_VgDYK6oH92TFZ z@}+l|N}$%e!vt}xqtN^1^FL}0X+LW3awwUiL3=Ug7RH*(-<(V= z(=T76wAw}b%>8P9cBtmkOfE-P)sOP?ZCjLHkCkNxkEtL)YlT7dOn^R%>+$4|(CA^) zuHtyl>iOXg?2>9?i%>>s7W6}_#MJ5b6vdq3EC4B~Bpi#gmc+B6khdD!uxqs?5MM&W zGI0f}0rOu@4u~YUghKsh&oo1ESv=iBT3TK=gv5Uel0;S$|OKkRyn^Ac|5KPj=S(8gZLr#7VKK0iXH>Wv zJD2EB;j}#P-P8c&3y7zs&=Q{@L4@xROsF-OGLlc!8V)$jKQXi9sa1-8Gn;FXK&W-r zX!bx`WNrK@S)iNk?fS;?1be7UeLx{o<=f$sC$iaktiSss;GCezgE8f19_{xAFlZPz z5)PzB%my82|%Rj>KDvQAMBzgD|Rkx3cMb;yaX zr?bnT=#MH(VkEI>QlL|7^9jLT#$s0z$)2q#yogRWP(%--z4~5jmt#9+@Z7{jSW@Vz z9ymnK7^jl?kXNN#d3wy!opZ3X-V2NxicfNImtcW7%qsS=II+#y#1e@Jz?O?&l3fkT zT{;cPNRC>7?z^>)En0YF(1BFPt@X4^R#b2Nm|h9@36Eo#ti0inFmxJwl9=B24t5h% zy`~(rYun|l8zb#)?jWb*hZ6ysz3HNv@pI2KWMpJ7TDuWHDFD=@#~}UY^D&?3>yM8R zLn)j{0Og164C?UYq^tvQ|3POPy`96uhy(-##Nga96`pHCu23|}(YBK%9`+mCXk2(L z+gpRtZF3%!5RSq#H$&9h49^FeBUOs8bHV+o1$zBL4|bAC{37nNGFB;=VX-~*zNhRU zUv8%rwb4}+y>|ZPlbEs#prs-UpU<{pb0Ge;8G#GQ9T_2;Tj;$4F&2JF3Ok#nj|KuH zu%o{X@t+R@(B7LuYAy~p9huaf%q=o+?g{6RM#l9rm$jE<^Q4T1@Wyu zz!aM}NGzX2g9d>DaubRxk(=xvj=-l)nWT-8%sf9MSS_8;;%N_=uH03T-(xYAjC+9$#Rh> zwsai!I{dQLH8$UPX%Yc}5>aHj$NcPa&0UgUO4vW4YylFx%5`;aQe?K-8snXrsyQ7c ztx1myI&DZc_2l~Xw1f(m^Lv8Z#n1bpbFz8%MscMJyA53SR-8&hIC&g_m}MGo<}R?_ zq-60|c)n2@PJ^bf8_v-HOZ!8YT;PlkXA}=qg;+RJ^X?Kk8NMU7r1dcj|o% z3W+lH^iw}nBOG83w-sTHMpk{LCX+b&Ei%KM6eEi{|GN@%gx&Xcle(xt?di+A80BYd2M!IR-fOW&mi4U58l(Rd#h zsZF+8Zwzq9Ft5*bBH$(rx5riW#`Wug4EC6VFj>QKM5?}A3D)EuA@ezC9IiCMEX>gL zsIMp&_5e$kYg8j~kJqU8yy3tcPb``)M9W1y3h$jFv|Ss9&v`CJs!P2)^X}>N#>XwU z?x&lM2u0tHr$=hC)DR95uPHZ}{(PuW1r28s8e0mE?D?j&*yKil8$Z%fbCIVoUV9wk zyaX;y%j%XhmrmzEW60nSkGr5)U&HsEmQp{OK4s1@0`J4AuK^kjd1r@;@{cIuHuNJv zroi|#l`V#bk*|3XC5)hq_32R;Ew5^uUa3H}s)y9kq1t%yBMRPQC9727CYQrHdb62( zh~p(wC!U%;lY8%bPZ7Z@zG@U|2&Clv(BdDRZX;UP{`eaHZJYphAQe zgJw%GK*!AgVOf%=n-p!zXCj<(0?dvwW6NmO*@I_V8}4BE@N7CQboJD>HtG?snC%Al zKX&|o%sH$#8TG{K05{ZKj%CNB$PKgS;m2FFv-j;I@8D1=1@Gpig1FLx;%q5sAgew- zXaz$P%#@LW<(;Xfm6j@-`+ZH*j4Ur-NGLg3(IFfYN;WO4$j0odvq>skNUb*% zSLXG@VtpvJ?=NhDl=xBQt7TFtEzE1#!}LZXOc`u%YTricmBBdNY9(txpL@tjOp{`_ zShRSZi#0&q5{X^2E2xV5?O9IF#p#@{C%@Z^T)wXu+stSY>cGBa%IXOWs+`6}VSYj0 z*n8$}I>=g2TGgl4Y-U~}RSw@wZESHIqXwh-(itW zCr9iuT+IEGZw_lu=GP>Fc0WFeE>YP?av9=m>p^jMyg;-Ehqj>O5~G`Mr!5{q_sj4* z;e&Kq*5-)#YdhO_TY} zikDTC&Ec4fkit7`^OF`|XvLm8!-P9XL(SHal&nvS?qKTHlyl zeMlyw)vm4XciE8S7rre0(_N;|6T zbSZo3VfPG8e;_hlvd>bnD}R~H=KH0|sZ1SzEEsE46yz}ER}lCa+9F+J&dS#fj*#Ex z?cK#Tv%|F8^T7-Zy=*C-DznMF8JSExx`})av?YtGi2Y2FrY1RDflDwJb-turL<%q3 zlZ#!)oGn&d4`Z358H)E4k?jg=u1pcTRQSKUbtFNrpsM%T{sktK^QN#bCrwW0u(yj% znbG8SQbGDAi(?I+R9V&zBX~GVb*7v8slFDXN)yh9Q6ioE6san7x)7_fy;=~oXO^I>xs8UbH<>Xrc65}Lr&&qD86kP$RGCcQ zqR|Vb#&vtXWa<}KZI7ih{z91{XFhgaAu%7dr3r@{rIJl+6*ZzJD)&$s==@gPKWLbu zibpE^<$?iTa2^4 zkO~^v28KkkW*cyMgB=XY_ETQz*Y}IRY_xKk+@DBl%ha+*uy>ls2Rq{)Nwg`WdHGFX zwE1n{MjYSe-+oI=L3K8JDfqsM?<0>{=YG_&XBUBpYp_8f3GF!Hvt0F!8#QKh13E}! zFYQVjUclltNVo2Yy~ETue)xUZj2{Ll=wg<)$reoxARUh{!3;WeCF&(BBW>JLtK`yH zpxrfE@0lrX^PS*|eli?e;)ACSvsAz7HYFVb7N!-d=lV7!d1ss=IDG?a-0-^@m#aP! zB&h6;mzDOzTkU8#WAH0XRIaxxq$bHgsP6Xx5@?o_#c6XpT&S}QoEoKgyHINuY#oz_ z8>Hib{lQL$QPD$^#P9oLNZ+zPPOa?V3-*D22buCUT@Q!@PBl-Kc7q+Q?;=lzNoz0& zxF9ql1Q`06Ak%zNM}A;$BG+f;2Y)aVKEDeIifB;SCgaR-ml@_ws7kAIBQ=C1QU{&n z+lQzOSui?Sf|-OBXc4!Fd$nDyNE_cxFfs8E6TXc~`{j}mj7v#L`B-Le3*2mME>WX( z`BY85#f-YzWV!GCD5aWbO&fR^m)88FgtdF4qg?;xmdz`@n)K%amIPh@b3K+jEPPG_ z7?`VLQv&)KpEgHfuM%6}Ruqu99A;kl=C!(LV=jARKYgPv>4A1*k^rEktweMcVEuV- z_>#_Rzz^5xy-teXK|&~lFk@&DZEwn-&L{xX)Q{ci5-pU~zG1?pMBzupzN+-6+GOW_uB zEH!O~U6-W%B%NL;@7~!dfv;{+oe$JAOHBRWWa>{<{OQ-$=xDhrvq=wXhY==NY%q}Q z(Jxw2*N!bQ8Oz=Zd~c0L#B73IRi3miD#WxyrGk1EG-YcOp4)+sN8jiZEaps#Dx1vb z$*fOW)kC+|DdN?f#<$+(D4Uz5xSGdEkrr`02VaNhjB&OTKMZB`?!@=xAmLtr|8dh> zlS&w;D($;WZ`6`{VCn)q*y>#{bRA**K=F7Jlyqd$!3E_zoUD~5mrt9ny!v#>NX#2u zl!zeZoC;G(osBLpMKYEHGa?smYP6AJ`@?8DayH1>o$uaW+fKq0D@|W;2H?;knj9}^ zCTfq~%uHb`EQFsdL0U}Ch*}L!^m&#JKHMABn0SrOxz*iV(SA8k=(NyaF}AyCAa}ky zq(()tPUV-7o|q5+S>1E8*kmf{GWw7k^|;kW7Rv#jG*tSvE{7Qr#6|f|G^ez6tHt=W zR`aQ9R{C)4MGsU$c0W1)I{=lvy55O8{m*t+}%1tZN zXyafE#&rxYEN@P-&?NeeOU$zslCW#f za6tkPk#dj^YaX|ZmDu{x!AsSAVyjrlOtcf?p*iXh$cy5R*QQ-AsU|RcEuuj(gK6S@ z>>Pcw*s^KdptnHs?hNhoQT3APx_6)C@CJdkP8y5LJjisUGw8a*y zGzg6v%~XxoQsH^tWoV(Ti^G#GiDLx8Y-zQ9qgQ*XC?_iGucZKB7fwl$bxPF0i(v^= zaCaa1u#M69JswlLmfNGMSy$HfHs17ng`Mr;;XI_En!@SYP9_+@zIl zPXs@9EiZq%Gk3qfC2lBlu?*E>>mKw&X8SVMzowZIC$k>5*~!kDB)?o|U&24{^c>o^ zOh8|VimeHs6v$-w(q7aTdc|=L2U6|8^^J*fFt)n?Hgvn^=l-_WR?Md%8?G=R69bef zdbcn1#Zr0J?T2eSPWdMk9+GZ>iQ!x1Q*6qS9U0sQY!RtTE}L0kLRd9bD&H@pW;F%D zV-kZvuadxlO-DzTq)fw%mQN@xy~)sZ#Ste0NZApD{23S9yQdBr6+7ph{;bX5{i$#> zrPP|%xrj@?`5$e$1y`Aj8J*6H=M)f6^?Z`av!et8a&s+!kY2%zxsM^lv|V4rggATG zBwXz6B(wc|(F(d$n>XZ9Xoz6GMJq`cjMJ6;^J?L$tIgr6hA4&u%gSJ%&#g!B<7ZSAY`Lc>70SR&U|M{1*L&$I=EkM`x%V*11*ajJ z0@m68kpIL~7E#u)D2~X>++>T8<6V>kCxZJmM6uZ4M9>vSipyxw^3=B|XHCyZ+(?Wd zdT#p_i!&ahwcKqa3_Zfbvy_lw!yK0F*R8u45q)^)_ zNslErt;bxB`WY-+X4GYzCRZ(P&*)nR)n0;KqJf_&(o>!%3GL_KFVZyMcm`w_Oh#pB zQ3^Q(e-E}#NkQ%&727xmW<2yo@?{iW@HHdw)$5TKvV)LzcA3AO>;~$_nP%N|kq3S( zwV~@5nRw6FWJ_vVv+u_$a*f*m0G4B!{sL~Qc4T{8={AhiZgINrw>(rNv$7V>)p)Jg z)F^^NMtNi)1qsw@Mjrc;>%(|1+s{=u&JrJqSsU{CgV3{VqyCSeIfY>5lJNUISU0 z?EHKvV@j6Cb>ENIp!A?&Hlu+g8I&Y9fw{vQk^4z9E&~&1I($|6$DW)jw(xu65?AS* z3eC!WqZ@&3=Vi2ap;aJLPJi>{A?_%48R83likV{SA>3ED{H#&KWbS2W2vJpE_lTAf z*ph6Bgk2nCScVw~rcFOnG{?}cjOIL>1yWVZuEXD{KPO*T-q}f{Kn)$dMVMI|eq*X) z99#YAH)!TP_JycCp{DpXul0J@m__^)(;FWhJy2`r_sG-Im(#xBnZ|2846f zsk1MHT@-TFJBfGi8=P>T_c>uGuTQ!rlJu(eweg;w9u-eiR#EKjX#edK;q912YcH2ot#y+S(RrB^iT$`#p290D`Gl&7P7`} z$y&KZ8!QI(A;p*ibXT*1a=w!i3VT4R|873nOtZ!dcIMUFq^fsIHHi!m#$=g`JtGCgC$($)#tQ@T{b(M?-R`)BF|~=78>li zh|;P(l9?>`2`4ta1EzT6%0WIiu@Xtv%HfdzKKob1tF|`` zauhNQD!^}TjKOVjpc(z}m&_ZbU*xb%ki5N`sARC#Oq+K^UBq#auf3W4+iui&Ius~# znE$Zme|VkKNuo%8k*Kfcw4n1ZA=n5J$195Sk)KLZFDhV89$vjLR+c;LtCPG{1x-g& z<$Rj>V0bQr1{SR$Wop$R-R8FTmTWIVBGres^sSh^M#h_(fZ$D4_v4XXvTdl}b3jCq0!BuaCSX{X^WXz)9YY2w{b7w&IEuW9SO#zTB+cF67LzKT#(Bbp>Y(0Hi%m5zAQIUmD0Zc1jO@91a==K*(=br-bzc0FZLeJ(^ zDeqwF4>Y_JS1yu)E@T9>$~z5Yo?`#}O#h(KS-A%u&h(VVurcD9Vd$>vMfzOYcA_*P zqH;a;v_uo@^ZMV+&fo4UQ88&-i%r5P%Wc#t%W-%U#umgO>HMvw8 zLH~r|+luoJ(@#k@7(&JMaDq9`4i4{DAM{UFPnQk`HLKt1b_KmG9Ynzop%5YJ5PJ=O ziUR%DRsQ+4ItU=S;n79n7yUc5I{9nzOO1?#!cZ=X=CPP<(1j(xBVdgr3V_l{`qIl1 z&+e7^?y_8<$i&k>DE!)4-GyzD33U_|PxT90i{~q===)z_Hg5Sn1?A5Gue3z3cYNhJ z6#v)Nv|xHWZ@{>lgPAfIe?xhRyuYEmH_(1}*Wyr8D5CK_hyYNsJ`~kVv)=je>hh&1 z+=Qe-7B?yDe`W!|<3sa%_-XpVq7ac;?XJ?-{2A8%*9RR$@7DY9_We8g$TA^k{h=r% z&N7X*@8nfhTZh(hTM_$VXV)c+wIv!qO3^5!yeFn{NbGxx>z(g-Iq1^-K|3<4w990? zcnz@he}!w#1XccoYpN5i$A0Bu0k_-9ISBMfYC2-9VIc%Jd8g^;d3NgWj8Rn-L*icRTHzy+d{=)@GND^czHb{BCZX zX96x1Umi9kr=pKb%1RS1*HlWFVg*ZeWm+8_{hg<6JMrS}dis-|7ov@|za~nLRPwSJ z^44N27?cV)etU6aX59_EZ_Pmcy6MjDgiA4W_DuGqj6gJ&j{M!pY`&F&n=0T}MYd)> zez?JNx?E)dGp74b1~?SdhjL$Z(yJ3Vt<(pj#+iN_Ec6e2&x{Hm_avY6a?ET20bqKe zXb*J&010dMqY3byU%3RDJm0`+M3acLqt@GBLDrUP_zfC`7uwv>%ADE!EuK=M|+zE2Y{v}pJd?XUCz36dA>d1!s{su8i!Jxrb?RxOUjfQNky1_f}N z-QPLh?15724V_eV(;3K09aDcnm5#>m&Uh6$YCoR0$`D@_pY!M#fEGnl|7xFFAt5W) z8)pFbM%C-led*1;b!t*311rw1++Vbu#w7#cI!1Z0(=6;bG@s-8J5&87h<#p%c)3(GSo-Bc006wo zs(DBU-uR#doRDum$#5G0wz7V?#%DZVK}fULk>haQ4)kd7aIC^|xMq=xsx_7cuDB5vGQ&ls_%v)&B~ z?eE`BGTDnda*0Ib^h7a3qyf1fs=5+gTj*MI6p5fAI<>NnxXg0s`u*~-CX1E2BZ=NZekWf|947c0CH#(JKP_3EReMi`VK74Ak$h3u zVi_?g;_Rr(Lt<{x7WoY$9Sc|r>GVhP5f*ZqBQY46om^YlSqV9l>|lFFt&$>ZSxWHT z&XEWFAsYe$kar*GgzIV0T5rv`&o%R-(CT~7zB)VoCb8m7LsWs#aPW5C2%r!KuRs(rxuWHK z`)iQmM+8~S_bVn)2&|yV?nq(+k@4o?nH>v!+$6Pm0;FYehI{WOMw*@WsTTR*lPGJP z+;v~1-=0iB&M>Absdc6#jEd0I(e+dh5te9jX?CZ5s7G_~?=?8%AM@+LXEMJ_)rLW_wCa;C+T_4QI_S1e|!Dy9-wF6?Y$ z6z4J{Q)+!iBZ;l~fn=syvd_YpxFA<(>OO?e18Wpv0pI9`e}0o#XnjT5yKk6TTZ4XF<6*;b|==L(`hD=G-&u>jJ#Sv$mWIH8OirMw0 zPAY#a34t>`Q%+Y`%b#L7Eg~^Z)9ubYaos%hMN~H`qh>HApR{FT2jZcxg<2{c)KlOZ z$8;|_Btgn-T`5?YJEvg*jN=)7>of$^3Z4l1qOV@ahreO*n~kiLFSnFov(esYcmLww zisBa=D(h!({*3Lt;(WBLMvwd9C22a)>FI1imc><9$gZ6AyZezCy2t}j3190v&S_2J z+yLb$dhJ*g*Iz>}4=rd0ly3!C951PTSFAq^y2-`TVPr*FYd2{1;N}lL#3FtIQN%F z#V%kcYH-`)5eD5(ZO-4ZQc0^-(}_jrP@rCAW9Eb_Q@2>nnNob4VMt}svRzSO{}~|OMnN=qrRLjmWkafc53LiHsH#4)F*Mvu6-|xe0v-vf4${l7mSnn0 zv_(#dSo$0rc%7d}`PyZ=UwBiVOW&gAmFst9;VIGUp5Url)xaYt+AQ7H8em8~ze7xq zM#S73D<=heA&2TJ)r1P<4`&xIUZqtxdQWS(`sbTaYH`ER+7t6)IGi-5Xa$S0uw}{q z6LPL-I8`+`bx?5^xgVKtx`N+N&ZiL>qQlDsg)6@0d6}I$Y*3f63aCa@#;*@=4?ABU zNHgk7q~j>uktwLARxPqQ`U!Hn>LDbrB=7$5-Fa2$$>`fr=QoK6YP2R{R?07p;>Uea zA4)6m=iyRv(wd&>YEfj9d0SZL6|6)`e7?U?2G1VLk3wk;hOeJ)y%CL}`tYTW4*Ev_ zdq7hPlGe|iR45hCO9~BL;ZG~~&egiDz|^6uclf!7?ckU?6izj=*x@+5CD1m}BrVh4 zU`&E#)XMPd&g5<3CgUiS_wN>aP=?`ujQR+2*`4v*WCu&Q4`*^<+ao~#+sM9NH>xX|@=K?=g)wU%&ifvZ(a zkIi??i?RG{T+^}C5^jAc2gHKR;918tf~IGlQZPYXcE#5^V>LFac%|;M#5(r$;%VI3 zZDbB89=AJYcA3hYP>=?wUXFvl%rb+$iJLx!_lW(3h-2I(g)JQIp8d2fQIV3l&>5oU zyGqBDnzM$|19jtc>igz(vh;@*!INLkk=)K^)oJBWSj`#4M#)Zz_sS?;Aq3bE)j?F{ zcdi%h5|870aUHF?88xCusQ4h4gwE!?mR;!sj2G#yP#hG`O{<(0l+F{s9Q!Yggo{Q| zClPIMU-)*aYqwVhpg7APijjEturtj@6$^JCJsF`!Cts&Yxl2?Sx~5;A zRfN&e)8(*7dDb0ZN(3l*))%&l{}XDX*RSw8W`gCh)kL{mwY;+p#B( z3ja_7TO?h>MqLV!7tP@$RvtFch@jO#N!hBu#fm-S^lW~}aqvosu&GpzGs~gL8Ov)b zNH^aI<6ZUDBPKXqvHE&s&f(M3@}{FiVC!Cz4`-CBOYYaSTt*Idfi$}6TV9Van#)km z=yVCZLqzZ!v#XT}mx4HIk-wT_ppGiI%7>Eda+U|;2^c*=so`>Ei5i41){;0sEtPV* zzKx3S+O9C}PNw?op&xNH7anlabmk4)+$BVFu$499GB0vZsDj4h`UW}@T&xATtX_ww zFG7)&H1>e3BaUOUa#%p8e!k{wLeMOi)vPDf)|0q?W^lc`f-1r6TPAQE|J}GKzw3CB==L zCJ|jvqKp+XX~!@WlGIqr`YBWvAZCw&9!W?J7}r_M`JLQ0_$TU(I7_rSF(pYL8<9;KuHm*9P>LPuGF1iIBS4TDY!&z+Lph0>e8X;+tz8d`PqM8 z(q+_Y70h@ECl*W1=i5^r&#%T6MkA=HHkoVsE*t3=t6&YsO*PXi9=c zq!?h?6WMcTWuV4~P8@bXL0U)SBexQuj->swXElsU!jEj@-Z8E?pU;Js0fnDVQpq8u z0c**0^J^DI>)4Tq5G}%nRthY=t4n)7^h3Rp|Cr@7v45So&h>QI{Gg4E8U4_aLUdnN zN@D9pH2lG5E(-x?gqQ`)_U|nivHjrS=9$9!Hmg+*j??*27oi{cxSCKofl~$S`lPeH z$E`w4btRG57<|!14=Ft6ymS;c<`^b0rN@fz{vTg&8PsOOb&a+ZC=@8}Rve0ZaVc7y zqQTwWy|`;|E$&WmEAA9`cXvs!lly+>``+g{XXfMwGl4&0a&23Ct+m@i)s;}q1uSla z^{1WA$%v@x$LskL6>YpF+ip^KjkdVh-rr{rEO}ZAVNTC*XR$+^cX3x``CZ6}P@^m4 z31Ksjf?M9lb|1f_Z7=oRoR_02vE&=5mvo*vphM@Z|N3dRwm$OqQdrb^j6% zoeJf0xfHnnjY5X)Rjt+_+&xukAVK zfW?J?8x`-f=yW+a&jHWz*ngk zzJ~arjkAJONe(o{GeqwbF1C{zLhiqaHomtIY9y+UmimfnO{x;UKPP!h84y85d&A8d zKi!@^NHsp#uOm~=GkT&5@;^BuC1gVv@4{@KKP7jpxS_#ZOrFPA&Gioc1S7Pwocl3^ zJ8%gifNK5Pk#`UjdfWaxT8HVW>)eE;(}m23^Z5CdUC%jpT<*$9Of6&H9rL?gI;o7s z4kFUv%>@dur-^1xgbfgCyY82}Z}y&?y?DwFxK1Pl99pv2N%Tcgg!6&4$}_Qjz1Jz{ zlFdw4#AF;bR++?u$)9o9waPnm)ZGC5=09?b~oiWP>8 z*Kf-oM~7ya4(?jOCd4@GKR2_q7UKVM!hxG#tm%e=8*D3wd%Xm21v%bbVbE|=8wmA; zYyZU1v;x@KyVFzCT_kXo&ASPn>%NhK<+*6}@#v9u`#4Q_fmloh4Y*@8NrZOUPk zj9rk}>cR?E6-~ZQ33ED2r}Z*mMsWWL3rcEaOn*KaV9(CT!FtU9`VTQ`DIa3^k(fD? z-!Qvx|Ly~M)Q7OY@+DTWSy>;z98B-1$36fnz!7v#t2@y3u4(1L*Pk<)LbQ3Ls3OwU z3QkYQR0U(3w#bGX-KFd%?7E}U_Gph`mwE4rh`K2&|4D~s2pl9WaHSbztYDVXZa_f3 z7LHhX?MhP7*tSkenAB*seyUVKnflB#trRmexX`TWc-rp55aLDZsIcvMZiZEk$%H+CLe!U+Nec)`1)5Se``8JdYj$hg2MZOiw?oG zxtqY0^Bo@a(B3e2$8citq%HB6td6~#4?kbA91@cmGKyn{}A20x53%QxtKl^g&bghLeT!ozs!or0@kWImYQuSeU)5!IdE)K!pTwi zIA{v{y%uGCBzZ#U%!`R(lAdV2+QXq5oMP%wEe$gYA7SOz@42}eG0*2T=E9y@m!&9L z$)NkZtQ6Iav=T>h@~{i$42pf%0tE~ig8_npio%)NADgJ$(K3aD+Y^NTKC-#M&yVBp%J50qwHc2(!Xdv-zZM+I_Ef#A8Y2o&~T$UBzkry4g^n zWM{*#S+1@Z3I@Y0TPr1VN!n+%*M?N0H822&`O{h(gCG2F9xNt&yN48(JG~VoK(xO; zHiVpr$Cg%JH}F$r9H^T<5@IWat`?x{2)a22(`DG%sx70?RO1g7{T|bs51OBMTDR{u z+WlL+5ng*s*n5HvwRW>OyU*j)z5$hy0wH&5R{2C-q%cA!%eiD2i>>Ih;6dtbP%R;i zr`V-k1J!UGN0BGj<_4jzkbC_CR$anx=^17G#Cyg`m&TwxY3c=yA67Zs!?oU<(e_@| zzdc1}Qgi^Gf{rUoq8}#ZUdJm^Y0{=ubVVo4pHI2`IZVtNnQgs4Z=0dNIK@?(mDM~} zuz9@TBUAB%AJP}hCfRH)F_(IXQy2t)vPi@ek?T7{c_EFG;^7S8NpNx44O3D5YHzAA zkm>3aKq}FHZ-zv1{qy#$;+WU1YuxOG8=w2PvfJLt6D?keQ$P`QxlVh5p#wW~mNWty z1Hzljsl#=Nh z6mlLO3fE_`64?V!Q5C(RkK}%+=Zn7x#`$Q;6UU)k(S?W+cYi1+sia_1+6lbISErjS zK$`lD6hXAw`QF5e3#u$h)Su>rZuyb*YQ*noXu%+kDvLG6Fj-!9k@Rok1c}vavY?b5 zg?PUmjEmJEAv)MbG4Q`jmIQn5waKCC3dgI5s-cCK>TAZb>{AT zcTF)G;L1NMha;hrjNLiC_Wxm3kdY^0F3}Nuowu>YR`(G+%e*-29Jf-;7ysDG%yj&! zQnC_ou3%N;aoQck`FE#G?C~DCGW;EoZU{fNGT@{eCIFMdTZA>!!D@dv4 zG)D~9OoVunY1Z#x6Nlmm8)o~127JK3z~ZO(To-9HSw~gnLpft*H5NN_Pu02h7pfX_ zrHRux#?ejr>Lg#_;zP`A1zGbR6J&$4Tdwew!Z%0d_MOvKXMzx0nZiHN+%?=Tv(~iU zLVb$8gd~;ODALLQ9FR&-Yu%s^<74qi7g=jU5^@yfQqcs3sjHR}S!8(WJLxAqgjBDr z)l``)u3VGHBo5?9#nLd@IC4S(Vn31!a1&%_Q<0id!XJ(bJjSWbaymWB$7H*v;(4rX z)iJqj#$9Skp&vkCrz-8+F2_<>Njo?o zsN7>Fqog4uFuC(fTGV``WGn2V_ieJ^kh^a5fJXepTW+_|s3M)h5AnXEi(J;+B}CwS zXjsFB*qIsoT+`4XeTL^eB~n`dtjs)?gEal|djXaJ4u4WZVM9w*GR|r=`h! zNlt6!ORAfkbUH^XpXrU6?g4gSc?BJpAz1p6-y_(JLqTJ4tfU3j?6Z}}OoC|}wMbe@ zW^ztB``>5dneEV7)37FErdzTEV55${WBSlof*8{w74!I{r%@A~Xh?5mba8d%1J-!G z@)Gc{obu-hB8z#rbX@Bu=O2P~{tKt+0StlC#H6v=^hlDQT6+$H>lHnOoE#2RDBxYO zY0K>&R)H5b_}AQ#PJIv=YR3RP2q|(%#2gm~n+p*b=)>XkqA9FM{B-Or~~c zQ9kAKc)ZU4t?VqDbn72K_V%6xUJQuHP7uNU2k{qSBnvRu1FinV?N*m zzo$DYVak{on3s{Eh!gdeCeL-`H&e+wkq*ia$h6(#&e_eNKNBa0UiCFZP^;1X&2 z+apB;kJqfjax?((dmNpch>Ub)2xA5-0SNFKIMHiH>({-S9UcVY4A9m4TkEqDM<=O! zo6#<7VZ+V^6(s(psde`SHdIX#^9VnD!u~rJhp5HGFiL0_8O8ht227mnBT)xf7~K8I42Y#&OxUnwz2j>sJ@!nV6Op zMhOcxB&{z%BnuL8a+D=?Y6~KB?i?zEN*>8+&ncv?cYCgy37=YYwggI(5oH}Rle_n)vpnB-?U zA>S)`i(gKpcMlvjSMB$gavTJ;a4u3vj9SO;VbTDH@?qvis;SGyMqQ1O{0%DnM2}&U z8pOHwGu%KIxZ(FEClh*n@Ml*k`$9Na&;Fb3|BEMwaFLOrx(qHv*N<7bf&NtM^kaUD&x8AgrY@yY#bYkD3dgQGR#NM{kY^ zRmy5fx_11=m4u+|9Z-%>ss$-@2|+lPJwVT;aXZ!GBRwOZmhIJhL++Umw(h>#-bztV z5X8eIhYdb@A#HQUj^HIz>2t~OFmr8*c{{5fCprfi^w&zQ~gaw9si=(tV6 z#jQlA2hYy82bMX5(^O999jg=2jIE%{75U}Nvf)apWcRjpLn)2_v-oTg%9tt%Tu562zoH6q>*T) zZ4PfE-@;_41q6IBhVh(8eNE-_MI$n&=x*b-i;~8txbb=llXbQ-xELN^#UgJfCYQ)# zz4AW?id>1%|B`llO+|XJfT~uHUlTlABF<;ZLRixZy;$3SIvIIG`409k+ZLINU2Ayr z%|=2jB;kd|^1t0JRLI^*NG;f*vu+?jyQyA6k(mP*WdWL;UYDiXK*cr)8iYV?8QJoP-{Tq2`6 z{!sAw`S@q*auouTkWb|^&OoH?m$xN`J-?@_T;q94fNREJeo#xN-r%shokW&IRo6i4 zPda{GatsL&QlVN7bs_ujA6+iy(l*mpoX7OW^~*+FS^dFEoQZVSF)Pli*S+n6q&YXK zhBI}8!;*QrGRL%F*%xdog=JXdkw*^nZgII*x2Y%m=~ob1teLNG_dEtHSGK_5MfW>4 z&}yn|TK%N}2dSAJdnnY3c<}ecsNA64)_d^Uy5wsMn53TQtzfP?X|U6ZQJR_{V{oWU zq%xp1YHCn=G-fay#9Bgy{G$T^cwzm?R(4Zoku_+Zt%1R%X^CRT4e)w=t;6?}dwN3v zKQ~Y{j?^bjDz7e#=`qn@RyIYB`O>({ZFBNdyfkJ~jhn`kMs`oOyTUfw!kWEWxQS@Z z{KL*hc=P>hmrOl4Ov)_Ob_h}geY>H^h;}XJH2(;$)MY3!VZjIrtxV+$g`ZmvC(eAxd%@H}eD_b-#p*5*uaHc8l1dN_4%$UsN@yBd|#tk(t&)9qP&T3?)25agD z^x+^=`ye^2roU=#i|+6Te%b zpm+ymGj-bqzsuk@y%v5mz=(etfuc_+y~C)s9eEZ*l1BW_StcxQO3@8Wnno(@O74le zuGFPMNRr`yhr~_8^f0%U?6wT{M&m)qT5+TX<8Oe^Z2jCxvQ$prx>8RC2WNxeGYHpQ z_N%53cL)v2nO>p|p+_!E6+)7C8tL^wvYpV!U#|msZrX&pQurR|b~M4R`Z|P76Mf|| z6!jG|vUj+Gu}gOE*-vgg8W6~Bu~bR2)Mq%JIlRSVTTI>y6SkRI|ALXgohkPXVlbRt zPAf(o(=y4hzKfZPSpTgct29N-cpAJ~Iav0hJS~TrIMd%gBeAec z&zRfiIHgGE#yAzQV>vp!IplV*1yWc)^+_R*eh+adxSLmM9JD_ZzZA-(Gw67Hc z<@Om%q>h87ZF5e9)sxS%e*}-|r$hK~Zp=Pmo9~ROz1bK8Y5#khq9~@n)??xBKFKF$ z=`FFJ@8-v5L_`kvniov82BMOpj;VCnNNH%mRUj70P?iIQR2HV#k0KDQb)+>Ue~h!R zMr9LUCz@b2>?xO_U+r}K;|Pxk3jd{}cY?q9Co{gl`PXPyb-6|St2@tdFykbAKJXRp^e^$;h!B4}uC+fl!5JV?P zl6G>IG74P6s@?}*nRVTTt=WhC;0y3Zv&(DcD3#xJJR>9AssS_Tj@uEL>i`4=cbbt}5Evnj2JnaS6>v_y@dYNvA{sd7>CLQUer35MUxv?i;ak<{&mk(-a2{s?lY z{8Z{LgF~UwHu&2LLH4zrB|T;}Br*D+q+N5U;=a!BO>s|cG6yDfnBM_{e%L zTh?Zlu_*%FL&@dABz3Xf;WnSj4F{W;!uInU%kSBg#+kcXtf@`LN4JqZi5^?+d{9+R zy5!?ePG_84PsoQPcKKwK+;EY#T(KrxiVU=3>k3e^e#Vc7AIqyVmoB|+&A#C`sJi8J z1d6vY<>Onxodlm`EVkAxYy1rkvl#fzmo(${hb(DKnGp2UhsZx_+-GxLo!ZsQ=29II zpKM*t+fws%TWzKs3@TD{W)9T7`JB6w*DC?sjFu^ov?Cy8NWQG&%x(k1dqs;7qSX$l zB>4&75e{{JickshqOKUX70*6TVqAo9VNmHy)Vd3(;w} zo^GT9Z^&=)kiL|e(A>;fL>C~$ju-mjN~)9V|3Gyw8Z%vB&dweP&#(-m@@}}o(&)&X zQ1-o-9=^#t3(6YsV5^Y!slB{NYDvw|td=UU-3jMoE!2FkGd*eb%UAHylXIdTZCMH8)i4$5m%oC3 zG&;%65J=$h7rvcBEdBhC*Yf7VX>1hrIj^-qR|`!nc2Tx!vk>FqLyjCc<*-%M=6Y)* zo$4%drl_x(=%t@Mn1nv5`xF}7;|U{$|6)V`r&aGp72WKf2a&1W+(nv+R#fPyy;glV53mm z9U6(wYio?&flu7R&aBJ8R@pHtlZKT(`$yx|dXEq?HVZ1NBKz^3yM;-;ctlglQ6G~t z`}ueKDMunn?x3QrzeC3(+7wp&T7-n2K#(?VJNxen~7qy{; z3kQBN4yG2u?$cFB&HRD|nPv&BPK)bV+oS`@18R1M5`k&56rv0gb^LR9UNm%`gSX^gE!Yg03hOc`Ax)zqNdU+|a|JPKiTOnE) zLL}~cd@r*DeM~kVgumEkYTS0Q@l4||Zf@*^$S^~H{pArsgIsE?HROB578Sir<{7@@Yck+w%rxN&S=`+T3Qr+6J{bXHj5A^75=f3{EzawZ`X zomn1LVM;;7f6|ZZ+PA6YSrPN-JH{fZY3ci|MHQ7IzE&bL9$FzP473 zv&EoU=8t@Q&XnxOnnQXHmE;!o58=#dF+zb51PznS38}H06=WP zlP7G?_OipDC!P`Dyt`N^vmf8B017ly96!$hDnAy6V#a*s{Zk=Wy{1Dt@D@*8BDXyyqu6b&o@M{|sCg zvS|D_nTUw}2E5LOS%`n<2@{v9)BnPf>OBGQq$7?ZkCG-#8u~pJNBCj)ALlMvH2pl& zr_B+(rhTj;+i#&I=x2&4vO3(9m>c8)-PKXT?dIrH8L|&==A~L3j0pqbko*jT3O}*> z=Ts5F0QIx3&q8kxL#g~+5}eGJb2#EraQ|8v{HOZh5=b7$_8q6)X9wxR3Re}Ch=^Ee zp`Lw$WC=45);|7v&cXR&zMA}(jJjP=g-7J(Vzx~P@tqLe{Km5N1Aj17?*_e1nmgK_ zC9qab^;2RSZ>vmtmsUv$DuZY?x%vI(@r)K+wKV%M;sKQP-t)XC@<*aUH9Cm^1Jron zKloR8M08KAuS>nln9SG5x(iJ34O$D`@TTcU71b@n{s-~=qJmDFIIMC)r9%IPLT{zW zU=w!NDjPGo|Mh!?BApc4V?Wv^3hwEGx)|DH7Mc{+3gR<(N|{_{n{*8p2=VwS#eSVJ zO8WoRGrhaU0)K!?yErkr4bXw7;7Eg|hN*8})_xk+x+M*C@;!a#tGTEl&oi)1)dWMH zDaNIL4K68HyPS{&7npm8d3cv^Nl>6H!>J}4n|9wTqWjMUQ(0ZGuM1`j0}m&^$`9m& zet7vc{qPO(F!ofC`mo3$T=m~0-2dM@UsxEwJK{hnmYL^X2>3^7QvCU>r$@S1sn%9z zmvUxYfkd!7p_JkuImjG;<29h&k(m5_WG|T^B^2ZMP`)qU6Z-Jpe98tvIUfGfi+56~ z=G(x%ZERDuUa2>$;h(28llxF`b5qvq2{B}WDc*};a;Qw1C_U4b-nJO z(fE?jTHc}Rc6j^ypB?9GpY6NL?#BWPWFxIf#98XRti*qrj_9BT|4{cw0)elOb~L)2 zi#{EQvhR;60BSw#ccyuu=32sce&@}9+(|cMkH_C6#=c!STzAX&IN}Avr8Wf0F|nUh zw1Rp@%@kC%VfnnC*`2?d#-1YOWT?lnSbX~0M5b&u7Qx3|ZFm6L`&Jvvku2?q^EKGG zbgc#os=18o6_%h_8)k6YgBm^lkz+XPHB|t@vaFKvX0D~Rdp6frxSJ1rdhsR->?TcB zw+fQxI;`x*GKuj&Ck(SF&twy(Y@xZJHK5Nm^ow0xol;MDld#S z=0UW#lGp^mGZpZ2e~??&ybepn&wA$b_)1o#W?5BAsZ3-j*wzEJPr7;@rck8@?qHvI z^sME7(G~EMPoG_>u~@P$)X}Lt+GuvzBl^dku41D;6qk!B!zfb=5LD=K3k3&ZO`@z! zf?lsE($Hhk$Zk)P2>$I$pdO*H{yI=?$L%-`nR0SxoOoOH<`q(JV6Y&6MW$ZI6HG?T zM&oj(+w1#9-{}?0b}p5(ir|SEs|@qt3y& z7Jj;-z!;zCN(QL*#u1CwUmVyT_WLAQ#r9mx6QpwWkAYPoyCn7%r->qWySw2Rcb>&>l5y)vu1==n~X;fvifs$9(8Ul8oicn5s4p|LzhlPBS^Z@cIMG{6Pb!p*8 zPyz*+UHG-sZu)ep5HYA_Hs?P*x9AiwjoMwWA#+IfZMt3FF`P(62JoO9D$A(U62)R3 zT_L{+#23EpDFMsA#mrH{LSmCD~t$dt(&QIrMA z=TE9PufyF($Inbd*d+@7QGN?yrbb?pb%nn+s*sLwPTRy{BsgDf{bY9BWj49xe!WRD za0?;*jA5a1QEe*)9EM(qO8BWF7HeZ3n!LdSKo+Bl*1@sWO6Cua8(^g0kHVMgyZZbY ziry%QEuv4n#77NjP^)O_@_Pj|b%wlYViZe#C+HlE-mUKZ1&t!w9&J*bStn|9Eob5= zJ~74E<0%|<=1lQ5E*)&v>LjQc5#ni>;v6LmcrZGJ1rfyF{;7`hGsWT(T|lGvS&^kv z?+6d3!`(Ao!Eh={ZTRDSpMbi^Y8$kuT&L_XMu%e^ttvISsEez`7P&rQE^R=SFf1OE z%B`M8eF~=$=sE(35AQ93uo+9H;Ux5*7c(1m4F{r8B?rSGUtDZ0NZ?$e{FF3k;TJpn z-UPAAfo1=u&c_b)McD$AvY@Fgg2~&#Ju4+mQ8AlVO4c}53)u{DmbA8=DDE=%f|@k> zNTQlemTAfee+`3KyUW^g)XNQ$WmdEP)7`C6RHcGR0{-l!%M72ZkXwNnmgJpSapC|yU9fxE(#L&o zB!E^3$NbnBu&Y3kn}>g-j(|KBjfMP&DRG`+*s ztbyvV?{uQ}<4l>fU zyO4{~lmc`xo|~QCPw>)5meGHdg-5qaE}a%>n6H!Tn!h9+Tnru~MMb}SgQXGv3+o1b znb#UJo@2Q&pP#GIo}c5aBA%lmkWKhapwyq*a%G6Wp-r6CFQuJ4WNDi6Ze%)HCV*(P z6zVQDpkaog+XkfCwNRbtIsLnIG7R`ee6GZ>g`s+4xV&lGdVe$U5}oVxs_Ft(=qVV! z+J*X|>U0%cYj{VZvnN++ujtEYXX9!e2~xyATy?Af(4&2lVGq+npw(%a*3?!sYa6u?%idR>iwwEq+j>74VvOL+)Y0(JFuOO_mMbcu~!KR2IUXe zQeC8GS1NT#YdKSm0&sj4PFF&p8u{(J$l|iz-M+WWi8op2&)8l|bVc>N%p5YdKx{eV zI;sZ>!!F+~ERe3QUVs?W&` z+8FiaZHOf+W0>WRukMzJd*;lIi5&N@^?L^LOSgT7HKg}sUtk3>Qfv2Mj&YryOkrvY z^)L9nB6BH^0bk!~E!!Das?!3IVAP?ol*?g;qaOR@+XCxTI*-hXousfYpB@w|fc}sN zeDbfR`s;j50wuyf7vuiw-RVYkf}bA?e?<}9Xf#7w2Zh>^`5mY>ca`u61_d?VYOTXe zo%Td*p27w6Z=Cg({fX8c+i`vzF~T~nCt09gkbL@ae%uRIKkpE2@qOE#cs6aPmqtMN z=i>(bLPLWQ6-IbQgJJk)aTNU^v&W{svaq_cyu3tVD~emk8`GNQfqNV zyf-U$n@N3JqapS@b^YW(l?zm-XabjgKDOme0JxVO6UPewkdI*m=_3is%Bgg*p=zAsE)iQU~`_`Fv$iUS$K$JfvYaW9`>Qv$OnQ%!+o0{k-A( zZ+SN?v@D8pJ9PbQ^9nTgXTnpTK#A|hlDn;wy!vkne6a)1Y7WHUVHG!22?uB9kjUMZ zPOCEDHox<6{T7cUliEwhH;X0iPW#Sxm0dB>B@V$kcx+0QeDv}DrSUFQLGlOmdwCQ` zfQ9$#{#{5@bM(mQ$bJ~wqnThRI~pLhdm2_%kNStv!rJzsCWgh`-kkNoL&kS?qUKAq7hID5 z@Nkzcym1`s!g~D!6YWb+e+-i0OSfMBp*VhdJji>0vctAR+IiB>@L#|68!C4V6c6Ti zxbZ*`tj6qH<#~p=;bk|)opueUL+aLXK?G)i);^carPdO|p|U`^WBi!Y+L?!*=BRT? zItofe(aasV=Sd(vW-Yw3xIJz>X`4gkN_daIRK>&HNOLAbyW0DC%4J^Ul-CUvF^gD{ zu9_Y)Hz{Z55$tMn+P)@}yC!SW5(N&PP5`-&+rPQ&lx|N~I7`ef>=b2W zT8k0DC~Y<6zj?9TUBn1lzqCHvr9Zpg9t}03)np3iR+Y1Ms-hzUT!{v&1x3!C5vNK$ zT@%M}j}{Fi7xaLz|G`ojh{BZ>@t3R=@6NtTQxy(ERqW|5f~+J zhiJ#^eAH`w!a&t~{QELt6ZKsA<@zsY+`(F0p+#`s!nTGd>eItrt=R%<%BsFT0-6qvKSa-1<|lm(x4noNn-^X%M|g3pZYm zJ5r>w7By*J9W~z@PHfxp7uTG3-mXYA*qmomki+jkUN!HZkKZ1coV%Ya0B!ywdm#7^ z$^2UPXrNt+M;K$4>ZFcmr`!cLEkLE}rOg%lyi?Vv-UNS>`;5-ngR+H=i-j?k-TAfn zX7(5wEsr?EJ6=|1^O}42HsjmA^xxV9qwLg4Tfdf4l&84KsE=7C#JqcSt6sGq_qGFE49Bi6%DZaJ9GpQd^^* zTv;@T;1XStfF`8kIbhR@VV9&7g-sbp=UU&qpautDP+=`PcYE}Q_>tf}( z3{<8u%2I#DIN`v)<`+m#|Mw=Ox+w+%&mPb8Z+?E`hUwTRrDnZl5-?}k}w@9oTVWvF=Bt zQpwI%L4_N_04;YbHc?tZRD`K8+(Wr(ns6wrzFxWYPVwEodq98y54~)E_!yJrHt+u( zA92ZLMaw?el0mI)_!up7!|$|%uXf&5RCd*0zg1!rwugrjIxstc3>f9bh|pX#o^rj{ zquLVb;f!4=5~W`C*$?!jXv$UYoV8qK;&l4GRiM*+e+jZ(jG1|u`hLE<>B2#qO`$K*(j3o0AJaOZ+P)?je+*@A0qinW_^9To9`WpL} zRip9YYmpnRKgx%%JM$FKy;q$=5?pZ5=JCs4&}HwXWgvI1=();erW&cAinC-SVw?T( zzLTokOZ>_q*TEOwnLsBT=xX$LPVoZZvX}z^0>4Jr|bbN}i z%Hvt;liOfw`)ilQ^$Ked#gk~aBg_J+XZ}(%vAHr^U@J~?{)yCw*7@r$FuD#>^aZIh zmHp=j-f?#2J6s}(QAf=hV3kMbs|c+Kb4*7r?)ZIdRWJ$2DfjO5L-Y3q4&TzkKSO6> zd-~N?VBiY&;I@uBi_HxjrNU-G)!pjBvM)9($cMH5u4O&L?G>-#oapR3kKWArk&} zFZP%4G(Mr*li|kyX&L^fe-NJiav8O~2_Fgz1u%A?J{QsFPQQY_k1MCDkJOql%AH2= z(0)S*Pbi+{tZl4SW%Y$dL%5l7))-8!FAvf$=>)=%Y^2+V!L745L*ABzY|cFHkp+vY zNV1BPM|Q4usVI|$P6%DFmFHdpKrGyB*y`?)C^q9uZKKuo%A?zD<)i~J&{w>OGbc0Q-&pVZ`rdc*o9m4S!zlp1 zWvDKc(T)bh7_dz_+v$B$UDLpjg71P_c5lb$Z^l!Y1Hj*XDW2L*>&LA&q^+eN8?SbFmcy*^NAhZ;5_1J(_YNd5FNX*XYO-yjA$>u=TMHa|P><)`qa-6du7% z0Au=r+t=s{%5<-S2ubT7hGd%Z0)9r1$t!FaIfMznUI;ah>r~yK+_9;ciehyitjS^< z-q@)wT+x1Qx}&WoOieeFexY%s?d)))fx#_|mKi4LgO5(e6AW=QxGLBa_uxEv-7wZJn_snh&CThEfU8h`*Aw#9`9zW1%*a2V8pp%)`lJ6sBWU;z!c0De=Ti3G36Dv;x^i}*!I6Di^!tD)tE z(CrJ74v5N{q#RkKe&@bZmb6Voh&)qfm+o%-&O1$+dcFhcH_yDI)H8Fbbz#ENx{^7;#&u18AII-e7LxwGK*^RLqpI8}`Q_@kLx{m3e+g&g}fr;8eWP59n;{DQNEI_v|--?+*AW{jG|J%GM(``W_dk8S5dhh$D8uARx_$Czg-KO z>-jr<%ce_EmhTB=ZV-LoN0{~AZt zYC`QdthRLKOXltA)pwPO4bgrM(@0I%!K^IEmIHH+Kv8R}{q{S4i3-eoU2GqiO?f^hJmp74->;|;6XV=q&w6uf5S)tIOJvXP|PfZeeh zN2Wy*rPKJ6CeDqR6AAkSqr@-&M+iV7pD9kRu-YV=F z#nTofV^CgUQ|D{5{Xy`=8ry-@&2DHOz%XToN4jCZZ-R)eIIMh?lT;m5>17D+j=ecy zuDTR!=Fev)N{-1j2mA%n_M*@#-)52du8TVoZyX11Z<$JA30zt5BzsRSAlXe4G^JI0 zb1P5-QmGWJx_!qRQkPuk+%dNF*t{5T;+l;dZtt&c)z;^vf?B@QA5o-564~}_|J+0D zdV=>e#qV|A=`mrBr42lva6mkAgQV=l>ve)p-Nf!aG=>V6ql%$eZ2Q55b<_#(644B0 z-Wfy1_t>XW+9_B(h+@GTpS9MV*d8uoB=mgw2oS+IFdHGlN}rH&a~=WBJF{yD)xoP0 zZ?At6PMqO8o=_SG#>Ggz>cZxf2~rkEkOp{KCpS6Vmx@maka{Ggi9REL&b68@=F+|( z(&6R_R?it_n-&O4?z{z<4R^qW4*XRf31a>8pul|CUwPxa0c7?u47}SfNI4#(`poGN zDW)%PJ-!lNQOpw*=nMF+_IK(QPhuhR;}|)6XvAj9Ef-N+wDifO;lzC8WhOwVXK@(X z6#lV*7Jx$7+B_qN2R0VB$=o@V*!MS&(_ZYAcCRsq?u0p{5E~P%{~Y!=4=8Ia_cb^y zH$M6)=pcGwNL+ZWbw?QpNSC1akPO$_tPNW_&-TdSEq0Qudq|#In9W5wnE$Q%=&7Lb zPRomLwZ#*CCd7HB1y`5Lhr{=TEp9-&D7)H53xt+385EHy>#z`F0$uNLOE4}+{r)A4 z{pL1t=&J|!xchqcRl>Ub{JZoo*vS48%2u)4z!=IjW8xa zeYJ^ZvB!Y~<16{np@Qx=5UQ4XU4J7luJW%BQekVgF`^eP#3kFzUI~h$S7t&qI(&W@ zqeU^IFI;PXPpe;zmxyI+!#0ezg8uBL^WJK5Sb4}4MG5O*9B4U{R%{DdF}0yu>p=H4 z-*a@Vc9O{=4POt7Sp~Fq?&pvon(k6p8Mtrk$f^WQB2b9@DpgzYO@^tBp4+mryvg%i ztl>n6aYx0&ts~jXN;)CIcE!G>hv}%X6NkyQ8jH2WotMIXs=hF$bTa8wje!>6A*KV;m>i zAg(WDM7^ukHbVz%|D*w9Z+OiCJ+{D@aaTCGO|aE*y@t~f#YhrIMC4YVC$F|_Uncm8 zoxq5=UrtwKzG-c^l4oFA?L*Rh}X?xc=|or*fa$ znXryrc`Q{{`FdXMk_|jq6pwH!X>K6eqpRwQ9H+b%PqZGwzBGtvvdUcit~G0<}7dfb!VskN`6KrmI31^}oHn zFK{q8b)Zm~gK0D-sU41$rg4;lZP7-DPkBXGTWof;(Nkl0f#ec5kLMeC>Z?%B zq`|q2%n6R{Jl9-P>4uW~!MAEN`!~Cr=l(KL~k%DKQ6xfKN<6~7t#0! z{>KHqj-Ry;i^N}w(I6i^&$hH^GbqGU!N%kU$>L+j;H9!V;2m8XvU5Sx*0S7Mq`__o z_@ORMWu@ujgNn~fjOb7_2aW5D27?Z{u{=I1wo~z}szC~9em(>>(?i=;;CXCCfrjI{ z*Vt7@V@Lq8F{dT6c>Tg5)$qZgD|5oT;c<%ha}flP4%!RA@H=p$D>N}!U1o2!S^p;I zlXCfpwXiidZZ?$g)i=nf_Qcv%dKmR5nB9AiPIm1uyk{3@dHT+j`XQxV*Y$uXr^2$q zIh~=1{y+|8|5*BD2QPfNS62|;`Xw=V86G~^TyFC`{T;(?J(4*iYa5zpq&y!yO{rdE z`_#1VWa!rzLxL2IuQGh5s}r&7fH4W%V(?6xt-K2UC8-)wbJctqDCJ`4`I^OCmNt)1PA4Z#oa?R%*tjR3b-0_W zrYey<+7LKuUgxTGlIIr&z|&S)R1Y}Icmo^O*lh{A1UoKJr<1{bD2<*$dd+fQI9VuJ ze_`o~Hth_^La4h!LWx&D=TxtpW9VM1m>GFKx(JjJ3tzL%__2S^BRjV+Ss$e5{$L7u z@jkzrx1(Qy0^k3Of(voK$8qEIUzq(dmXuaD=|L(b^ofJ5@o%*i1M*4A6<4I?z;=y8 z7V{D}jz(7j>JAsvuugoFJCFV8iYA2@A@0fjc=R|-e&b3v3eGT*8CV9ho(0G6?upH9jNFq) zq>FT&>y0rC{Lt z&F2i~w?W&KlLH6wHoB)o1bi{W^E`ot&1G-0{u(6VvBUe3O7_TR26@*Ery+evDaJt0 zD^hIY9vbc}nT&E{+#Cs4b}N{OE}f~n^d#587Bq0GR;znHYtHV`6d^ZeXx0@hndm<5 zlJL{pFecnE06peg8+f4WIm4EE_)Xi}WfUpcYbmt?h@Q3xO)ihP9q^e@Zj$5?$WqxaCdii_)hM8Gw;m3cb;c{cGGm9vv-}^t5&V5 zLbtEe?XW^|h@BD@=}-95U~E5{imZ%{*Pj>q2n

TV{{?yb2l)28xR}HkABlcAoe0g@X0@22;5!41I?ZIZOfO#r#A|Y))W{ zjYDB79%yg*$B9TangIQGTMLILq&VZ8_}7s(%_X*-sL|-J|Mp{0EI$MW=QBp;wU|;f~-L!CMrtg|Oo z_oA|ta7cmPZGYY9`N6*Dl3ATDL04$?weZsuw&yREH*b6>Kk1nP!fsw$B&HhmVj3Ba zu)~4nEoCiLC;qR=U!K)*X(+FO^Tj!^G|(`^aH_cej)5f?wzqfd-gKU} z`zKe!SVkqRQ^ZvHG-mKTZBQfC4AFdj)H9_d2Eo)H9vY>sJz>dQh)%))9~x3HXx$tC;}G75 zo`Rv?6k_s2qk4-?44v#d8Vg-W4{b(g+RgEh69VE!S)3Up>z>2);1hP0agbzF!2?5( z&WHi^oFslfLyje*HZvl{_-@*gTIIt>Hl#v}TE3ftv|kMzMdqF^GhCh!mVyTz_cD`C zSq4_f%bHm1cDK9N31vqfCsV0B1(#&V@h8SG>?30Iu6H<#NnejbcZTjfw9{JqQHv0l zN$DvvbhK`BXg4mM`cJ;fv*5MpZ3iR@32r#r;-wDy;{)9O$5dO?;2V<09fqlh9)N6E zTS_v0I*+K{d>q>!towIP@pn!#?M*UwBj#fSnq=F<7BlW##aYw(m~+FUHtanacip$C zCg&5r_#KaHopT5@$1gW+&?%uX1kOh`Va7^RySF^}kBy0$) zG&zwQ44w-aIw@Gl+49zOxgK(H>J-aa<U64-?opQO&$C+B7J ze*IEJBx}KnrDNbcC~Gulap|F^j+cNlxIpo0JWD+(q~(V5U~2fFEvVh^>S!8uY1dq1G{KIh7Z1)E)ZymgzdNr9tV~Uxhww1D&1lR@ga^<8B5u;xD5+ENL3*R^8Uf zuJ_l(@l`#$u43-WF=M}8Pumdryl9dGxOBUx;*ayO=gITvXslUb3R zB3-rioZyplpxoR2{b%WL)Q-N%g+$vI_W8q(CLZ-SIC=GqotU$Ktm#C!j0I2Fp_uIN z1fJpX3ZCIUz|xZwKM1Z4!hLrh4%*fD%Co*R&2DAgOt8$Fc#wc1KRm^LoOBaf=BRq? zo+UZG3FL+sdjQy9imAhNWuWci2xdzwJ1Icvew??Gu4Q)~)vs*tm6p0ycf+ME0lE}? zCjWPWjPM^X_(>p&J1p@7gt_IgRX zHMaQ2w%krbo#!Iq@ZV{M6FtHCGPkz-bXjN?KwEy zX8B-?J+FMd4WZd(<7N0E2GQ^T%MSi|0e@%^-uUQ7dDq0C_qY9n3jfdFZlm#0CY{8u zza6u{qcdW39$rgAo}Ty!6qG>T=-hs7`cKC4|8K}2>s3am2aW6vhD>8Y|6@P?{T8rL z>n{#z{Uh`LEQt8OD|0*hB^i=$ z!?N2WOqNPo$8?m=59B4pDfsO6^d8rW?J*gLBoNSY_vvTDFv_C}8+@bEXb|ttvEg=X z@a&N*!S9nyXvzX`ped!Jn9C>Q4u`T9yjy-2rEp`dL9icpByt8iae|k(yQvAh98w(L zH;pU_37Wtv4IBh{UfKKN@k0NfWZyiZRpGlVjW?RB^g?&<4l@bvfKB#b$Nd z9e&VxJtBKOkm)+KKe5^nJ(P%>QfMM%RrkFx={KZ1Y-HxJS<{biz}#=dKKf|Vo$WOm zX4Hz;MO2i;AX9j#m~o&S3pfi$T!=}D3i^PJQtDjpl}+V^d){(Btkvr zyZF=+u^>;SYnGh!nLrx|13D#|Y>w8<@!El$Vun%eCI@c;J~o82R#ARoZ2c< zQo+rzeJ-t2`)xsPIdgthCjTclI#1pdfMJ%D=L(WWm2ZH|pdky~ zR+c7O$Fc(J&~*+!!Q4N5!N1$yA12ls-3I#u`ZTUG%IQDt9q^$Nkk3A!u!Zd90Rg4xZr!NVM$|w4jAX9rg3R7S<_tW|vLm#*7yTQ1!zD7? zTCS^JmBz<8j&!ESug@0kJZ4c8_`_DevqZ|^X8(AE9lf{e2)(vJx#u)@_9qQUzq$i` zJU*a|yOoS*T9BjJ$_dJpl!tU}`;)4@sBmTN7XpI()8-@K@r~V)M%CRyFjsly$JCI2 zqj8qR=llmOt+DDf4gkz-ru*@&7+VaEw z^AejiTHo$Bsu$Ofsw3MyX6nm$Q=-=0BNI3BVZ}D)=yqhb?EaPb+n%#W)t-`A!-$wC zt&?z-_r`0MJA*&tg{>%ztXiND#CPeEcsb8aq>BsFz*o4f5T6F7s;7aFZ0io%w5jhW zjneJ*JA2R`cAx)$d2p1WEWm{|EPu5WVMfSj(yw+}x*KT2Y?oeX9JN;MBh)cD*2Z=R z$-Xq&cv*And(^rcUz(7XNl`B|*Pg*grCxFO_FWmesf zi)rIn2Vu({Z!3F__;v_YZVJ4=I!)_Uu&YBd@lQ*2CSeK$K`~z05ieKQ)OH-NO;sFS zY9I2r%Qbk=?Wew@c8^h>?twGp#tP{)>>;84N_!^NtHCrA2Mi(pFTTBu=yV*q`uXLj zmyGl8aN3-99*d}E}8Q_Q(id8xz!9Cd0^xFE? z2qJSnHMt#7jBgxAJv_dnQpZvq9mQx*TjT|2@rXT?ud@Yr$DM!$Au6x890G*RY47d> zG%jKX6CfY=botqT)u?3`hb(EM5;8ET?La0jFckg#saMj_Uf0lCnG?D`lb}@mWL6ayh!BUa@85*UN-q0 zF&NHhIN`dq9@XvqeFxQg(d|)mi%bd&HSV}o9t7>@(S{rJ(i8`!`@|9jMEn6+7dFX@ zo<*{m?&`Rzx;cQZOHvGZtCiyK65D={+0jcc;Ptg8C23J} zl-zloe)(1rq;Zm&)yM@veBp{?bEOX^%XvT^jE;hkeNOqJ!Ql8>YWC<(e#&e8C^)iZ zdOZyx&1IJ>69F*uKPiO%Z@U3ZHY@qKLLWdu`)5=6kqDJrM<4cO!FuK4&-0YD?o#oQTd&;8xTan2gtcHBqe@MC53fA!LknNL-{} zLXYu96U{N}wlz(QMD9L|vN7UeTgXUa=qw#kh-Vxo1>k5Ex2AlTdv|4~8c8UH8_#bR^TR6$- z>NrWZqvMn9>j81#bb6Hk9j-*K+#-?D`NH`SNO^m0NN8)d-leZRJPX*X4>5X%qBe0| z6CG%PFbD4)>lb^l^)g~nHBsz?K4sdv1eQ3aWG+N;Sg4U?Ry2bk-}5H2c0VI2&_XUt91#JxH_~5As0Y!MO;34MCMmoAHZnCf}V>+%BwuW_R%RI@O}7O$jA z`G7sceT7IoftXW;F5dq;9G{=M>+?DKKAUB*X5ES2Y@1HLF3@F$X6PneA&S9zjmbib zl^}t+eyxAPBwyp?<_+|aCogw~Yv;U;BmMhax#L0FQ0iYhl$q2IEvBz8kES7_OlZ4i z11B8rFvCxJP*J3$l!~(knQJNxH|%ET<}=mqG;)Jvwy%z_DKuCLirk6fLP>XM)&1H@ zdXkD?Y7#6pSVY3X^XW5f3TmxpOEXRsM}9z#_qNX3+|p9UI%dLr>u+JQw>QkzHA-rU**!M*8pu%_aU+iFPe?Az6aYtjsA>h0jx z_Pkw#XHY)yJbVanzQ%Fx}y5#O+Q+Od(rowSpu-=$Hvr47K^8(TB zk$3Uc@N1MH&JrD-*NBLSUnkGyk~y7Ii@li*r@JD6N+b-P^SGXRuP+8ujWUH+bCDwB zjjpdRaT!o%_}&h&TQ4aX(>&AZHX*l!_9*zJ-I**LzwV1=61}{!R z5e`egIMvu*qOMAZL3u?(_QnHE@ z=4hSLAUb1iL3i^A+EG5BruK_IL{{>$IBiXH(~CqAH6C}SYfJ5-fdHiate%ksk|Y)A z!|Q~dU)B2BHqSmKMGI#S?*$vCzItOzikBsofm^LL>|JvLi_$PJr*G9PtyOo?hGS zXQup2i1bAo+t}=_+AX8X>Gl--7|>^9;8#3#salH;+tMgZna_`FtY`tV7{mQRYEsRX*46=`nzcoYE-B+#er}o? zRSx8Ecbw(sFGR-LDc&@x>}4A1!sfvI>tby#90@HH!|lf$($zG9Zl-R=Z^B<>pZY+K zfyB_|7Q-+h$qWenQH^@n(c|;wKp1`bMvchAo3h1V4h`}z=Kn8O`)_}_wT1djCg+cz zvOo>5g@Gfqamv>#g(6TBO(CXYOjdYw;T$!cVzWA?P<`+V$zB4BMgXG>M|H>#?Db0J zrRl@#fN&u_y>w61s0E7;gQdZoPDg8od6TvML1_uDkzJdQ4OK-dHRehx7xT`m49l%o zpJXa8m!vZDVd18C=yenv8>`LXyiihuRr7uIFnsccJMOZY#%e_j59h0N+1>m!WHK6J zbSi8bQ6wDw+;CJ&wR)>LgZsA}g)a9ORY1+h%c{gInws_o&SR-8o)aowg-*9PmM&p>e@T~3EJIWbm|SQK~qCp&8daxQduT18GYgy6CRKL;R)HZwhI>c2@Q#) zM0&!juPx>*1#E1}6?_i=?>PInHU39P^zWt_;mP_|k=YA|`YA?{JkDHILl$C#O{q%3JM8HKQAk6!=mL$E^x^PD{oq+();Y0j zr4gk9M_Og&)>(K8SHL&40}e#>87@xFb+BdAWc9BcDFhrl^%lQHI`e)2QWPkX&p4g! z*VM^AZVkQE9zzCiP$`nK_sw93WX^&WO=tzKK5hldkf9hYl{=kDPnw;>jzmq-ho^Cg zFj>4fCrk^2c1G5G_6Y2cBkPM!w4J9P$>O=JC}%+z%ywld9Qu&&aJakEc~8*jus)*D z>t2Q9bxh)ufL+skKR9+E-@5^a%MU_yOsZC(i=@Y0fTjon52`fW?;ZP!#W&d$ zc&fx()|zd}rTKJLSj}lwmJWNnY}(D%moXp=8`%tLKN(q8m+5|KZgG9E3c;I4x-wJ# zv@Q(C7_rD@uOT$upA&>G-08+T^$Z+BQ+2!zvWRcq_V|j0Pw~C19renk?p^x|2|6q^ zukGyQUE|?q;cOn9+}_^mjsH^K?__j%(PSmS(_j~RpaUt_IJdUy_wd-<)|Upaz8A_G zu;jj>99-NE-Cu`jXna0hdyPS>TT)bJdFsXi4U<&c0qEf6EkkDpEiEi8Dja*Q%4_UL zM~;35cReZix@8$pz`Up`r0tMXF`*D*ju}AUen1Kt`ZJyP@7DLv`0X|ZuDof5stPkx zabaZ~I1SJ{*@^Mzs$BKvlD)=SUKlK9t7Vjt;O@M&A7)$ZV2q6)av;lB+`YPftfhZ9 z9gbn?Ex&4fA_;zEj+h<4Z<+ywzsv94_;=`wuPSI9dg$WtQ+@)l#1svt$V}5SgxA#45A}>Ba2u641rtm}W%~pIvve)LOskZ7xTnS_o-!@16-Njaq?vcjY zYAx&TQjTva;;6|*qF=szX>?GMR;vReSz9Hgd2(1((Vo&KpW9Z)he9w< zzikqRj6#haXR$gHYRq7a(zlaF1kJvhOIYIwtx6Z<$zl~`F{kHXkt*`S=h(3u%RRF2 z#P#fL5JbN8odJTghS56o&?XCC{kmPOPvD_FexZ{sz$&T|}|D)@R9bB7bl3Y`w; zVc~Yn;qw#N zBy0npWz*5Ecre*{oeoDx1_%;R88G^nWA;h&{~fz|CN`W4qmus!zuauS z?Z~0t)VRB1Pil&aLT&iwG}%D)`~@#*@5d4WKT(l>Ee2bRi!4oL7~=sVzj7+5er@|F>TPPQfbF+gG*_PgMj$3E;nHD*yhUk;}i;d&eq% z`U|)BcOM78_ldSj_{rt#o~>H}NW{f|l5!SXSP(yH(WdBm#2Eqewy~2p$jhty<&^0g zvrr9spRP8O(8tLuTrAhZ zHQMTeh!jevu5`VpyNaXQ@O3J0^UbhIGKm-5?EKu?bNox>xL`aQ?tzmbl-fx zjX9sAkjh4WcGNdE&S_;WIp`?pjOVLvDmZp@Epov!FlDO1loj}mcY6dKo9--H&}+oq?F?Gr`s z8|U6e3ltPytw!>&LOwpN%B3m7IvGryb2e@~mtg7>&|+iP9RvnbAQ3YVu4+2Jpx&6E ze|!b&#S`JR9@97ZXD;q<&Hr=)Rjoxhzb0v)?#q(gJ8OktE!w$lrb6 zb7_W9W*(;}BKku@F-NFM_6d1u9NB#5DJ$t^H?>3wPrIiV!D=G)`>CZlhQUhi)W?~A zPDHME@!ssqW#uPeWvpMV8KvqMBd>uz96}aaOhR!cRwRsY(TvFkL{806ml8jK9&6Z2 zCPWQFCBz>`efEt9QZeyhW!r$taoe4F5#PqNrYK%7M%?8JjoJTM^Nq8(&=p4 zT%?1*B-N3%*Lz{MsEw<2rM}|tW&Jn}@1tx8unK10Pm!BYuUji|XIJ9?56{`E2}K+6 zJZfd?QpGg9#g;~$Bj!7+6#?4d8>yME-?jXtI{6x31Fe375rR!9rKFr_7oP!DA8R2J zcJ*EuX~C~mpxreomuzp*j-0fCe(=J%V`AqpFD$;>QdcYOfz9=)m2%xio5bwZ36mBN zNO-39(}PaE;}&Gt@nSx}Bekmx-Sh|Dt=4)elsljoo=d!9Q%<|U#Wf&QIjpa@e~K&( zLV^RMKGX)(l$6k&NU_^tQu$^z)wPZH!*RmLkye1z>$O{MT!mDMx#hF9SdB*1+p=FJ zlPj_*Ss0PcS?M3e&Z9k6nY()w&bKkN07K zIJ5lAF)2^0v!-e(xSc0hwK#_+0;u>111kGJ7TG;)56>C+=tKh*oQ7!7f^!*RSF)GH&Pw+RY|{DnU|bq>-+J4e;f1luf@ zLO0;oGOjsss3g05y}40q_PED-d3Oundu{P4ivNMIMa*g{Z6JDp+41z^CbXJnW!-r$q(qwL*9=v<+rMLFUsrfI0NHU$1X>o$qg~e;yE#ZnTwef&W6)J z_*~8|iGF<<0mSPsJ+>$b6i-hSseG+vUG6*eXH#n967Nik)-KiUeztKX3*WWqZ|fc3 z3OiU>QaM=xEhP}{H#czD^$lY3QlylzYe{bwE~ccQU}8)IHlDb8Y%(z+M?PS-$1OTv zz(Y&yLBE#Go$kb#yIOGQ)~UQs9n0fQ<#Y9}ed^i2myBY>37mXAdA5FWF}|d^%Vhu) z`aw}uY+Z++XsNx|1y6Sy zu=;()j^YV)Bv#tjO0M&1htN{;9&3N}u^2EoP{tsXQYr~9N~I8YS?~kH$dMxtt8}jz z>)&R~(3;un9uN;XCx)o6ZEhCRI59_vn2tb_F(%POUQOr83dda?p7DkdBF9X0 z1o|qJIw?Lb8xsb)K%DY4BDsqb8rA*MjZ!1L1np$eSSLgnhF}@FCdr!EnlG{kXK?lp zBtl({Rk!;Kg0M3r4y}H%&6+cxVMlP3O~}?k>|R zYh>DKAF5Ren@9}Df-Rs24Ufr-%wxW$+V@oKb}dtU5AOMcc` z(>{pXIKB2#=1fbo-V=FMD|_vEC%t9rCHaf9$+R0cgGtS(MaX=4NUqjIC$% z+;R|B7Cz*uN?K!^>VMczKr-Uz{aRhEGIir(yC$dvJLP*n;CYEC0_&2CODy%ord!}p zC`9D-(g&Ixx1z=lUY0R$taXJV(Mt89R?!NUY%VG@uC|8-<4qyKTk$Q^uO$*u zmqtOG*LwKbdaST1V=NOmwI2|y2JDXGDi<$Rw9Kypbs{iRadJ3@gdfc=wk54~z+D6C z8x@XIkVffP&16NBVs*&Oj2XIOuh~0c@HEP!@?6yIj6q5b@!R`5rrdH7m1;NRTTDBQ zJNowQr{uVXJ%Zxm3>b2Ff$p1bvmCt5_P+Hxy(@|;KQ;G;Iiv(x;V)xzf49Y`+1K#h z|J@b?x`Au@*FSn%X!S0h1@6w8846!MU9M3$5A`^%M8wdigZ{>2XyZ`oaW(}m-B=0<+f%9Bj z5$@k6Ahbru!BJiMMcBpS#K~w_Q@r8Maa8*aPVCUH6XK!kL&m3QpZX5efA#`c#Yp+y z($1_(qLWDq>~b&CaePQj(aHz-_TGN7tH2h`@y44sK_psx!O(h5U&kYY4TeRtPJT@Z z8d6=TS;*YetEc$OTj>-Z&H*U+VYHo)#oo z!cQV6jkK1tFTb@T2Q?IvZCurVxojf#73WYi$JR)32OjU0TwP6X8OmS$^n20f%Oib+hWvcRm3LD!1!bzB; zRLy>)wG+uv;<9$#C=PDb75-LLTYsS8wi7M~1Tfn}iBVgZ-z)Q+&)eK9#4dTvoT(kfft|ab5##r z7k;5F|9%ecqpnI?v{|}B@WggHIG1~G!Leb(rDb!&_O^EH!*p?1iWpovwi{$DgPG!f z8naclWZhV)S&4Si`cIpVPfQJTa@K^8O$|F8Rbn@j@JRQLCn%$0MvS1Z3vSxA*G*%_ zkHk3B(Vh`G15AHi*1w7A-Z#K;bUTOwBjbO!&)dSNlUW(Ju?q|BRus(-tYY1)XjzcY zQ#RQen5ENTi7s4z|2nViv#ndr)D(b+P)H?87Y|gde_{A+zMVM`oyBTQ?L`8>0^KH+ z4*sI5on4k@0@Om5n@W-#zDxVg5ss4OFPPq&lQIADUH6w*)>XAeS1Bci{82_N*OYq# zj6#B?Cf`HopeURb{`X6a{*`nxspPz~C6{+m0q^c_lT(_>1~xW*@NNbCBjy82Jmv6b z{hGT6K(eZhjR|&}sueLC9%!_#GPP_@q`9q=Q;c=CA;n45(@zM16N#8Ah5!lH45XyeWVJgL`&jggKHhf z#r+K_6QAdhfCi7BV-MD@kfkw3f$#lQ;q;B9JG6AVzUH|5wUxcg47pPN1&$U~DN9<~ z&<|bFbW{@QH~>kB4%rU>v%|6cX+ zw=X=2A&t#gAiZXNi!i&RT0Ga@VaVYl>IT&m<|s4wW-2`J-Ma5f&PLg|tQ^36d3PXz zQJbc3`9($l#(ztHUM&)t+DXf3Bo)bg=NUEkelE*>-1d}|CXJC? zT3z@lbt@W4|pjq<^|OZ$byyDe6=NZI3+QUlVueLgfA{Hj}CX9usS(rA+X(aGOW zk>?x4o^NNqds_Q}b~TAVuMY7J=j-1>ODczaRo2io|0g0TZjy;SHzCsFNfBjp|01ByhfuwnHgm?6sB*^ zd=52yb#aHz{)xDnfC2xJ*bnLH3x*7KkO|U>`O*{dib}cL%QLTfUXtzdQ*ioEV*s7M zzk~ZH;q)SG>qFLb|f>wmj_ z|1t_5T8s7j>-GN?<|+if(tO+x2KSg7s`kdd`J%zCxK6J*icugRRUC<9cu?r4F-35G zwDA;F%zh$bb{6-EazvNt*^VH0*4|UN|JR0LsTUjC6UI9IXv}EcD7sezUU1EJm+tQx zV#ZTr2?tQ#V~#0Yj}3Hu_a-K!OWzWH52=vk zUGE(>->a%Z@t?5J6B;sN9Qv}XO?#RW>U-M`vUofpz6IIy$iH5!#FcJeK;C)i7=UaobwU#(K z%`QICREn7*!5A=6DXiwPZV;rvdz6~BG)!NHzRJ^a|F?^}^+E=UgZXuMTz>$K*IMt; z*1P1sNAEGM9Y*l3G@lSm=^whlA)}$a3B#o6XM!BD-5?oT(d}MBDDz86;aw+pXdm2> z%P-Y+nR>RqTe)yP^U|DT$GLrNQBqPuq|&w$lJ5YBEZgM7^&W4}%4=#wPL^93zm5?$ zgB~zSH0ydE?=BeKF2DMsS9r5nFY~aNPY)5kh#lVE-EHh$V2bZYzF}iy<2MlhAoNBU z4)QSxpUZ)Af2t^e$Mu}h^YK=HTDkhG0*}|T7O4KX+H_)4ictTfv^c&bLAD%!{mLMnLGgT(H2x)n(qCo zSeFHX;>LQW1av;{+i3X|%12vB6r+hzeaI&qHw@Z8FeU+5kQN+Ohpir*f<>mRg}#J0 zL|H_#KJ7mBj{T?$Gj#1W7Zl1m35lU=iS2k30Zyt9_g?-Z+Tl5hy<=E$3>L>Q+7V|w z(<5miOE|Fy12UGUTvLnu9`fBcR=S-LOPV%%gV@@RUzAE!m^bNEV8X&=%gKCh_Bxwp zn@m~5PL>LbFiB#W_iK**->KVoK1|tj^ajcuaQYcfh_>$xn@?Je!--H&DqXR@Rt(jY zw|W0us4YD+bF+AxywZJ>EVg-=MB^PQm)*yykP!2m^IUyEt_PZTRvHibxBJ3&E-c`T zjEsoK(tOZq^)O}F-J8g(uvrtVwp>tAe}j#tQLfnRfuK!`VWXHU9R|_TNaV_dDV@~v zihAa;hOiau|qYA6l$a2wn>`gMKyAB-K!VegoE>rz={yl2mNY8y}62iPGckqs^ zqCrJ1{fgkKDp_0UlKN$dQo-l?H;aplZ3MDFg^tQ}TMz!L4oeGtpSl~ID|aV}r0XX)1Y{){K^d6D}`Vlq~|U&H`?H^CaPGEwVJT>-AhlmucVfET)Z|rhU3|-UI8v_qzNOB zu|=cEv2x2Qs5;?PGY+h8VjaHpLg$Ol8*VjUffSi7Ej84LQuQFoA}2Dv;&u7KFTP?! zZ|2_Etau*$QXqV!GyE~i*zwf|xw))m0`2j1OSvK3anzzO;i?3sS^$5yNw59Eu(7V% zCH3=}*n1$eUDZcXl(-X*PD1*oll#A7u>bGzKl z7yz0#m45>Lqylj*f?wWmnHk|zV+*n#tBD8c-rd0)@TDH{n z%XOlyV>|*iZ>;G>9<474r^M6zJ2`?`5nIW;^C!PbGKCU5=o*3r&e&4=x%ZG@CWs@M zu2KC{GRw<{m!oj7e72q^i@6e}j`K;D`3Z!}Gx22(3J?Nbo=ibQU+gHz&}ldL17fzd zv#30?6;2gt7E3jxsiJ7rhrAJ&!t2KqvFAlj$Ig%x7QFB@zV5FrUhEEjm4B@n_=C53 zjD{+^)H=%Ti_BvXgnVriUCdioqM@gEl$cnoWX@t9?y`A)vpD_) zu-la*N7lke@KI}0<3X&*yon?{;U~S@=oisBlyZxVqS8vHi!jFxZ&z8>qwr^IA;g?L zQA`jpOx9%AvkQLKskxS;m)#8RG^*TKYj=0R3H7Z{GZd$lb~rD)oLs=RYOJw#S<7v7pxegjs1 zgZNf3)2P4(Xqid&e7>j#rNe0WF4+K-GA{T}B2+pF;@_Cinn!-v7vGoI;ZG8kPA0*7 zKtQ8U0XH*Bu%x<${6wcjc9$jA_IBFMz48yP<(3RisYnF@l|uHr;ya3Li73%P@c1q* zBN7r4I{fWqf#LxA+0ryDo|4EpSNJbNAS5kzhWu_!{#w!mk2?LkV%-kupK#F*g) z%Mpog(d|a7q=`|K^G*O>6x>}jdq1siB^ z@XP6ier1TTZN-jy{e)El)@^y19>zv8kxRRN$V!(NNfU>g)JCFoKb5aKNlS$NlP!wj zTw#JDVZ)54NQF0;g-kXT8=za2k8>$5O~do#@Y5~96^+O1MLtbhG>wt%^@@NB*7lg5k^rE=vPr|YGFSfe$~ zA`LP$x}QjMJkR{r6?^0qID)GbMCGOvVJGDKhXaxRq_X0EHFvZ_RTb~(2>#7h8^8LMwP>Qapji4-2_5j zxz=w@HwRro0l8aHq?-<{smc5hi@WE8qbQ#JWQBR^($ehG(o&$$H@)Du;y)KB+%TBVPcN{gGJ~PZBJRJ6upgci} zL$#K?u6A1%zHM<6(FX~wZmoqyJvb98)-Dc|48#Q!W06hgcHTqV>H-l!+=P2kkg$*{IUiMwf5GTh%jd@jywiZ}m^GPhZW5h*F=snFZ2`$s?e z#utp%p#G#}@7J*5lQN^-Ij{V$Py9Nd&wYfzvUxo1B)u}9%iJ8z(#;>}U&7IrRx3#W zXpBiBUntC2tkZ-{12vPcf((7JJ?&@Lx^^7a>}t2yrE4j#djCkkXD7|Av_SOQEyFVJ zIA5)`x)k*Nt=-=x!2zt>WeQnKrPw(%iR-))T;Rd%Ne@~B(Q6GEe_}JW|j03IaFRpYl7f?g5we zpd;c;v(dFo>KSvN<&Eva@^%x%qiIeNygo=3{dr5 zp4P!-&)6f4^ZkSfC_1IHU=IeZ-EX^mNrhud@ z-`TIT6_~X89Nh5S^` z)4CirLro-=kQ$d?*_3o6HR4U=4;8l7$lA``uJo#qpCW`U{ z<|Z!QYy}^jdrU-=JmX~gV>b?lOm8Z}pu|tsys>s4m*>j}E1qRA=&Jc1rznx+< z64@m}N{!mH%%f)MjpitKoqV9M$rID_2}4$>V(GW9Q=oLaasP+P~m%PPLQ-(4bKt=o)k?X}c_rUX-}G|UO2N(`?TcPFji(WUZy zcaJVgSZV5}o(>vgxoj~^l55R!ACr{k!_?58GlQaps^;)!#i|4x#qX5;N`O!RqVmh_ zst1?`Y6UE}Ch)b0(?={68@lznfY|29EJo>hg{9z#(cnpC>#rBeCeCq*7} zvIODwV=LI%Dp+Xq9gov_$eXr^Pf-_w0^zTAF1M$`8EMUONNess`+?~_3sj|A7EfIC zeod-lv3jCNV$grP-21iOns);$P>V-%OeqXHhFPEiogAGG-=qAYMW1L$Q0!DaCce7B{`#L!S=^7^AE zC&e?!6zG-85LTJJo&A=kDqa4A9;3<&AF%^k_ox2CtBN?S zZcUXTg__7Qce-4%#!(UJhr5RTZlOmxbVg4)dAmBmnLh49ed{-6H|qQOW}W_p#OQdj zAv$eBg97s5bE*!8&y^|(=xx$lV=1BrOD5?APTiSgAvt%~hg}^$H1fp3E$SIYn_bGq ztNOfWeTUsXTOQZ@cMZu^d-DTf0Gy0KWw?eH*{ta-^{~L9f4er4rcl!Cc9kf26i*F% za>)rFU?|+Y1PetHtQWpURB15571C=}V{ z;3GE#p{kX8QLGn3B(;1kHp_R3&QHL!_Gog(S;>lz+ATzqDn-GsHC04*f03)dwQ;e0 z!mRosL8X=ZF>-;sE4y%jSXFf4 zL+WOEe@~ZZr&|P`J=O`?L~puj9ABSUM+3B<;kLPa##4KdJ_W!WG9P z2WF{iRPqu#UGg32S=@t*)bZ}6?A%nnN}z>&lPhdi@T>Y#{lH^;eKp;%&NuNMhXfqZ z$-h^8Rg9Fua-O9t@8#AEm|EWJ^BT6WIT>_+sX&yjerrr|`WxTct_we&^%A6oe8XIE zFq^>^5)1R~<&~QnO=-FrJDeG5!t;a(7{8iP5? z@lsP6hH3Q2pZCfi1}F`?{v9&uv%=F*Y) zyqi6)eXlx-KeP1z7<fao{|;2%>~@6z z_|k;#R7bN(w->V3>_G$AAO^r)Vr@PaaqjkdNFakRy6CGlKqphrWieXf_RS{z;VnBr zxL>p)#%hfl9cBa>-@Xg_P2qllgJ)bgBx0s=Q$>dmT2c#OiyI)b-4lecF&}#xTJ{k2 z^>+YS?deyp)p915kXgKYc3e0JK`4G*$;|4-qqteg1BLs30XP1D->_DpnB^2=wt{;^ZC95nvS$j1dlAGkf9_}-9QuzESVX$(At z7T|I@&0!D>sR`|GRGSQX;C+-q>{W+(B2ck#CRMPeox%K%xtpX)cZ=%MA!A_P`;*c@ z5Kq#yK78OLaQMdJ z9;Zr*qw(Fdl6Y3j)v#VrT}9YNC;Ly+fb6mmY@@N)hRdv9xdJ4Do`N9KsF%~D06RR4 zWmkNgoMWt9gkf^xEpsk}lPnRH?T3iKmHZzBbt7^P8lST_psP%zmp}D5&^9#m%nYpO zo;-1%WZ~hr=dA+sb`@WfFy{`oH6FogD}{=XjYLqdr7AjS3W!gi;5v(;+Rgirn_F6b z_4UCE``>XfRM#R3nE~O51ijvR9?wdBD^>~;8+KFpquJhPz+5!7GL9cRO0XFBtab>! z^vJ~7V#py*3*h5*L$uo>ap{s;g@>QTj5~-S&hlb#o6eC2Z~lFPxygurp8|8Clh*0+ zkO0T0>DDL&pKNeTnHh9>1Y5!~&UxELluUtMQz=6QG@=E)*iB` zylIczUGu#DF$hv1kzG(Cc8m77C;DT<8ekf1QXxo?T`*NvK$6mF<;LN{jg{ zhA(xH!SjYyH=Q)S)=!Z;=h(CrzQ`lD-?X&dFx5ynIL=SB*5pmB|2nz(_h0%~==+N< zV}}SXu=c&jqNRngh;^19u=q%lxqWwj$U)yu2T!1OPK?mZ=v$r>CXmL0|^RQ4>|K zyxMVrMw9glT5I!m1(8_p|8s`%?=SE_eCjj)@^u8NhovgeX@N4wJ5+~|eRj^{R&-bc zl3gFtV}8$&knmPY9*TM=sF!X$^1eM$C57DS(rw1CN0o#kuWB93`Le$m=(HMg%6b-B zZ1gP#SJ0oPzH@Q#?RA?JSW(<`U~u0%vN@Twd?L0_+3fGrC{u!bUrVRI;|$N*d`)G| z0W#!q2jq(O0V7_4$Ky&`2O7baH00Utw7CM+5@XEnu>n39*-1CQ8h>W$jjpmhzqYi=F}>;JEx=RZE&kle`b40w%d|26eqg?K1>4Sq83EzCo1 zKOU=?BL(1+EiuZ^oU>k;Gs(G5`?c-Xb!q0+&%QgazZ%?WJr3%LEzI-1lp0A@Oox&Z zNH-2qn)repNb3)rQ)z_XC>uwTC^3*7Dt&S)1fMNz^7wxiVA0N_F=U+WUNk=SSZE-| z1q7Fys1xhc$iF=Z08BYv1mmxiz@Du)zfI-PitqB#dwX_;0$J%;d0e%!`I;A;xr~VR z;$e2Tn8Uo8s%x5F@U_56N%HAgjr8Yg>VK~d|3y$h*FkEhJlK_BeRpEG z1MQ{vGMz^o=H-Z~+2-wSW$Zswn{k-%hWLHc{e5;7i^ALK# zp~W@#>~-_*mQSYAkwq)mHk8--sRq(~uC=cXaU1;j;11CMx4Rn~_*&fy%0@&vK29ir zCNY24@$D=>d|OliX+iLs=;O5yI<$=^aP2`6GYH=KmV3*CYTrvSKWP*xM_I=OctB>~ zw}X{2&Eiq0;xUq}l`Cb8B!2$e7k8dVjgi_F0HJy@^W{m=Iri|gGa*wZ>ANQ!J$F3G zDYrRxe??J`6W;fFgV0YY>naeTqI>t3@J0XmEtu!Wb%56d-daD(((e_hPYTQ3hjYh8Ju7y3xY%jaLJ)e3H88k>J zV6H9hjRGXJ@fh?l7_FjdFK-vFpP5QFWmFd1&{ftD&gN7$TKO)ZBL5I)*J@%eL32{SUeDs zPVBbVO)RRWRD;mnIl}TRi%>Y*8SZuc1+NLUk;AtJ=HjoFtrzW4w6S96m+m!QTbhN+ z2c?peL5@eCj0z2NL<7H$j{Ck1k9bP!PvZaiZq7QfjA&)Mpn18(#2p?&NY(w*gjpKQWaX~AH|I{f(>Eh?#9gW(;l_yY z-qG34Dt2#Xpw!p>Y*`4PKtbE`z2kgCI%InSMn&;gx|TYr3eQ(zGp?mQQDBQ*pTn-%?(DNYPpmQrZM66uZ|Tyd9%!S zP6h=;6J?3(afGQ8-+5XFm)M=m%_F&Xw?%5L>)cY^504a=+_uURRQFT{xldk?aQ2FW z$`)NyvtZLkc$(l_CyxZRX~jF{jcgNsX?ilVyt0|e>YML32T*c&|37`tp1#5{VbQ4P z?BQqJFezfWZIrr?}WM`n&;NuRwW@L)xE?5jY)vZLiTDSFT2e! za&qeJYRz-=L6BBMU4sHD3MK+d3SAsb-*A~blWay}cj{-uJT!t@Rj`4;Wh$c!nto7U z{k@OhVXukPw{dnxOnur9Klja8EJRX!dElqREVtGW0okRHtZr$i)NOgF@q~@!FDF$;bB?O0~(pu-%;4Z)8yiDaiidjVgp z&Qd327@_LEpX|9ZiVLEp8rHgHBOY}2Lfgmp95|^O>FLo*+Z-A&5&K_Y~=MCHJfkPmO9s1NxG9YH5W(01p?#j7q;h9;M_1iL3T8KTl!yG(0{3!MDd`} zz=WIUG7str=LU?W^HKdXV3 z&0fa-)ncgu2P^u@w!{t?_kL3>HL z_V8_2Ht6yu;!HRAVD^_0!xP5=^teW0A*S@_>}<Nf3l`3!jv0+gl6tCvpwudnoG$q{M* zTH-h_k0Sg?_+;Bm|DGm7DYFoS#4rQOXXImyC=on z`9IZqKV0N%3-aZW(n81Zjb2m`O6*pD=!FYIu3*vDi1PN!%tG!;Oy#q}D;c**8;yH6 zYmVr~z^)RWEsm~kL)QW6+}gL7XH-2ub*+n_vv7S0o}QXp@h1**!s*7Wa4#&-rj{7G z@`j&z{m=P#aael3Ct&FYqh`Tvc}5F&tNEzA^-0e@P&Mc&yu8%|C;G%k_);QFSwXfp zC<5*><`s5KmXjaifa%tu=#G-FTnmo)98(FZ8{57+t zNoR};mn(E59_TirkCADKo!{gc_iIq%!LP-I&jQRv5Z~rPZ)czPMNDQGo=~gB<%tZk zoVx6&U&U11X{P?(`pB(0snWcjAl&AU!6n1vOld2(N7A|g=l5=h!GdbNxF=Pf|Hr&m z`OuWsGND&j1#fZ#V?58;Tp6Hb1U)arB3M(Imyit^tg4D^)!QsqmJElCj9y{0I2Br* z2=9R&x!-#p%JC;>C&!PSZyYHtt1TX@4toYJ9rmsbU5=3F>%}h+U8YVD$;3!^Z6gEr zV=Jn!`5BQ0AX?+BZo(XxeY`7_?^AFzExk7cX6d#@_fgmq^+GL;b@L@g7Y8?y+%xdn zMu_TIeB5B5OXj?_zIpM?==85DUp1J-+%Kj=A1mSaeQjd;~Zp!%rtlyp&xVtNtJmaoC zs);P!QmsSd#gX>6K4{D?W})~TEK}d#PW)nUz5skfaP_g)U0-@f1A2iw2+ZaULbq!= ziRKZ0$?K|mV=?iF!{=)1Z>i)Gc?FePK8L@e!yH|YW}?dt3O?j;fe9PahwWNmt#Me! z-(|q38Lc(&{9Ug`W%>%+SX+G`;Ig8hB^pFFED~cB@bE-W^X_*#gzkWY@{uO}o=YPNDExxGYFtT;Fn6T)`3#NVI?Y1mza?%E$gQ_BLg_)=?I+|G2L?4$}A7AD9Kc4=0C)+$F6!CD$BG_vQJ>Dnw6r z#EXEohh9e^|JyeZaG)pN3Xs5*z>oa86^8n%Df5^#Uq<|D!|jA;*CLRwJayp@3MBqQ z#eA?mcGV~kHP0}y9t3#o2yK8Z%bRGYtfs)A;($)e?uoKh#<>E7%lImNej6S-p2Lzq z31)|3yC#}x6H#Qc`6p=-evWAM||uNs^Ne~99p^BoT9Uif%7)c#Hr87nHp$Zq#n@9n|G z%^u-u+L~dH%;y`4KF5i#0!O5Y0l!4qBwNaP-bJOn=-)646z3p>Lm&A2i78*84xE|JR`pGm1@QbVbBOo^K3y?^^xLC5^f{yU&mZu; zl=wy+ttnwHXxmY4;Wy=zl$5Qtqme4+>TUQC0|yJZv#GT zY~i_V&0AFDolA!AZZT$2a>a%s5EbRE1-E&;21YVAb?07k-QBrz)thl8)gge2X!xEIzk3rfR1NmWBrMF_Gy>t=oNAC+ZYZ&to!XS{g8^cZWBL z@xcPQ&1D1bUTUz)z9LZC4afUP5E-J2#cF2FI$O()uk*fC{W>$VfE3!$RF*+y($>k7 z@3nV72W}ISsMYxF3D9Zhy^+3snp=#oxg!z41?gGOoENfQU=Ty9^wqui)CZ) zMXykYJCL9u?_4BYECxv6ngSH7)3_b)(J2-7;4H%({)95~p0P=P<<+ZPbx5dR!magL z7=)-!-2fbVeg6DG3B}@Kdq>yvejY*N-v0#y**)MLryTl{oOprW=X)Ezfbz0b$`0mv z(vAZA>o0J_@G`5(Uw;S^^r4|lw#a40%GpBrm=@=EApkkAJy$tPv&WEo!f&1R0ta$% zA3yxm_44N`cu75L%QK4f@V}d2WYt@ zJIuE}cLotIy$Zk$Th|S=8|jD~W=X_fb3b!@!==_dY%&Ju956?L4V`?uG|gsorme&9 zFfS+~`zf1d#AC&bt_PRxPBY`yek`{(5&7efdS#f^+aLWc!s3_?DCi@ z-CZg2unlISlnz!v-~kYYto~u4WCDZLcnTTRbpV7tRdp0pSqe8gI4ckAtojIcF1yTL zW{Ff~Z4GsAA}z2~_NRVvTl-QEwHKT%pD%M7Q1<)R$a?eEEzAg5Q+=B?*D^0{t>;+w z`>A5?ifV^cMDRT8CllHe?awtwo0O3CTbbX*ig(j)4fNdbSImLS?yd7OgL$Kd_YcWn zf8zb~*5bB)!O^&hYfsz;m35!O%4_hamEU;AyK3Q`Ce0V0ae_qW2pgpu_Rzsk8)1-T zEkCR?3U)9rz!Hp1LGN@uV3_hbk1J=;KH+V-L>zVOOGiZ?p{^CafO`XD4g8Id)&G0W zh`BH{t^_4kux<67{|3uJXi^2Lam z?)zjnAkavM5>eohxDy#sd;5B{V?iz5uB+`eLH8EmL1X*zo8J?iTL)bPUu%2f&A}i&Uq+wGG?nfO zF;{Hz*Rnb2B7bbqC|SM5_4O!Cgw@8S!6%GBcUlauv3OzG22fcK!jA&_?gXetQ;N@V zgF1ufCgRrJ-&x!yRQCzI?;^Rq`q3wpj8YgrvSFVTPq;?$4%Z)niBJ5zBx_MB%6}1H zm~xfXRe0VCYjITI9s2o@4NP`6{h5O08IP#QWocdaN{8A3+=QF+_~!fb>s^_>TE_z4 zRqu)DaEQ@y`$dt3mi3%;(0_`0e?<;!~EP@Hr(6*Tn@F2=o#e}$VUMCJ%Uf;oxYn*^uDdBtkK9WG#Hh@Y@Wff4-{+2Ms1L(}{ zZX0fw+0UbRW7%{rx?6F&@*OHx9dUjM`)aviLx0pkhvG9CZk{VP!e~?vJL{nyH18n) zm^l+3Zt3kcU5R~*ShjWJs5wJKgFc<3XGoBX@lGfAb-rcQ995)Xo*@Z>CgoHp{BkLB z-G2PkiNGvtUu{YMqs5lfi36J4Evw9;vzV`JaCS9{L}puQJY);uI%KP<6#817yDzJ$ z)ZyqRnc$v4@>L+BWv$9v1*(!E-*7tU6ufsXzqLej z-JCZ3iO$Y)mK*Bz5rGSWnqz>6o6dvw6}5&Ax$y-tOyAdIRrQXz2f!)*)tX#S`ap_ltR>Oo;YO@Nek^7?D349fe~NpF;J;YK;jVi9L12kqfh<(9Pu zcyz;TJTb%OYhC6~oA3tc zeHc?)aY+GoT!Y9DBy-!hh*;G5+>8S_GJqQX6Ha*)GA_@0!AStm8UC~@fmJnmqHRjg%L}DjZy-o2V8!7;3UE*dK}K3h1}r z;=kzhm`i)YN~{BcqhG{ftziU%Gvkkw)06F@kW`X`1KW^N>GJP|pl{nac(zX|?|#wW_JXSVG8SdiLPV&|wQQJf(& z{$h;fxX`1{F=HPI3iqzyMqN&WU@CQ?Scx^}dm_qA`Fr&a)0%oUa)?%7V zTb3;h*6kV~uH2L)&3MgkiSa=gY}^bf3NJtD+mmQdYkW(}57;~J?O6{xnL-+ON3&`# zKPyA!d~ka5+U1yIy-PYs$1_#@UfudMY7$R3Lj4Ky27RVrWS}SIMFw{9pV*kbxNADH zCf*OB2;bnby9~Pi%qoE=GYn%vF|h=GW;!a10T!R3pGL)_%WY0#8=hs#n}V2xn<@gd&JNsSF;;|L4UYgOJ$ z5&pSop5d|HVawG(9Nx7i$5%epXrei0nsc?hV*cpcU6vw>o1YPV>=E(t%#QcwFleCh zmZ63|wpCjiFqNGd5-Z(SJ}G8KoYpMTFpg8^AjL4~{X+6zX5~Mhn7=1wG}c#I^6xRB z33ZUu?db%Lk9TL2dO(dKH53!oP0iTI3&tJM=Cp5obr>FK)Kwx|F~@2xC&rZBhjw`@ zDe6V$W@F)Xe3Z($t3Q@ZK1#4q{RL}Yb0P3aiBEw8;UM(%SlB=Rc3zSi9G&>Rcd>VTK=))%;)d5> zc(G%r$kKT`;AC++WpED=*K?0JTXHSc$34E0@?k!AQS46^Fv%68cC%`o@qfhxcv=sf z{LYMnL_y=5ZI11tdp%fC3+1-Mpw!<+ebmm+(jIcb`1EohSk`zlA6-^0xJZ-NqHhU| zQn*hJ?ZnzwEw!-#Y^2(wwUsNhKNneA^?E>mF$>Pyve%p~D#ew>Z{+r>U}n&}v4+P{ z`-GS5vh}o^ddO*B;c|A7nFPxpj*%K*f!T7cwqooTr4W6KvBW<#024ExnG6YU<@T6hh1YP`?5dOXXC)HgD~GlZJ9<>da@~9*r9PgPFR(>P^PJlv zaG3u;da^&udrUH*USr7k0eC#z+~8>=!c*3cZ=4)fQTSt%Hjq38#cXoU+Fd5}p5>Y< zeZ#C^hzh7$s<|6hyeRV^lZ~+lwZFLc4>Chk5ZY_82LCa7)izVb_xs*rwhQR0Zx*Cn ztuIBy?md8W(t@5M3d*Yz9ab|Qlp|z0nq=nNR&EhdZW~k5m;IBoVNGfc%r?ovCX5sD0tt-0+ zo(=}YFMk6V+7R)yf-6EdR|^e&L}2Mx9W3w$5Lu**tkq70?gB&iXz5K{r@or&3t-m# z=~(nTvRc(V3feqCQwmg0Ja`A^Um~PBgFg7kvvelC;s4}`dgG;w501KWs2v>RH}=lz zs1pA<@8vmLj&tiIfjN_+;j|qyaQ5A6tJ**EZ0Vx{7OuA6ek0>iOKp@##metegv9cV zBq&yN8-<4;YxU-9y_?4d^H{`j{hUz`H~4qXzaw2wBN@FN*$*^_wMf22NYAD8pp&+2 z6Pr!-{UD6p!qS5o=M1FD3`_&GW&*AaI@WGh!ZAtz2FI}SMwNRfo z1{UaALxNi5V`tK^jdg3bP16oXZ#PtRpXzS-Ee_ljag%o^vc&gPc?`e*3RI%7<~gWW zvT_^`Crx^#GrYsq$F6(qIDGy9kX61H0u2P)kLTk{@ofSrxLBK~Sgn6|@`*M9^a2__ z?I!)Y4ZP_H9sw5(%RCj~`tr?yLke(a^UkTCt;fZK^!3TGzFnnsZu~(e$sqF4zoHNK z@8K#i^$Q-HpZ~-Lc@PR9QQ|pBFMPm?XK}x)V$Z6ac&hxUR_)`vc6o622Ttx|(2P^# z#nnbqD-1y8fR*a_Dk_~Fm?uxl8Qeq8c`%58Fi;Uz()GxGKv@a0Z>kcBeTz;E?FKba z%2umfeboATr12u#J3>5+L8O6gj5XB+(A_ZHI z+)h$2rG2uwDir#S65`f*M3V%b+8UJo_)^Pd(>IYIL%^~WXP*09Rx7>zt4G?UKLJiY zpMQ8?wG9{HimZ$AToVJP#+f1u)3yuAkW42u-8aEivx?)dUfUfeZ4xFstut9D^`d+C zL}C;y2f)2^Q1C*w8mJFc-0`9pVAzt;^!5jUFe#M_V=Yc&Me1msc}$VLz`)ihmwio7 zsU@;}PqE+&Sq~2?eaV@d8#G~NotaS(?oBXesUzD@M3BwdtP%n&1t�*xVWk z0j{9kB|}P)!~G86S+nYV377|eaC+1|R-WCzv^CQ(?-R&Sh=x}jz*DsAmQ1Yn?B2Ws z14`*@p8U@+3_)*A#MG@W-ni+6Z!54|ZMrPAFBXp^Ju@srvyST3gWeDsu=XDFeo$M> zZ{3eym6x(NrEUA~T!=}c7)55`y~QH-qk*x|+tYb$*p)e{TQnoz8-v_%{YIAb&kv+pD zaCmHl)oYkW*{1l5{&Scu{ld-}M#C0Z08yK2{P5s$w|Y?D3}JnG9Y@Pp1=2w;R<{?% z4>7?UgYrzm`?kHy1&?yo!CiBMs6nm-Q^Q&vvmptjzcpxp<7N%A4v1m25!8@TB*kvo zth1YU5aV;tyE{FPz0{iNSfbRYZF7h5N5`97z&vgEh_X140A;6N-k)*@%|Tq8FJg;z zh`)k$(+wQ1u!4U~1LVgo)_t}Sglq`m9P z=q*O&gwi(DAC6dNiYwdq$!;rdHH}=M1{(~A9G;@mZQ1H)lSlPXmJl)YsI`Xl*2cB3 zU4oZA9GA}qAj%szcsq|vg&VB!L*%59Xjq_9m;2SU zp2n)zat>sE0Q2-DZWO+9v3}kSW zUx>$aUdK%WbMY7^oJNqx57n6+YS4m}Nri4DieAi~_ztcUo%TFhW?LK&hQO0C%hwG# zw^tTZmF0Gd&n}xsl2oGgePo=r?5b_AcVzCFQb)cA1pwM|u#VO_z~}Ob!j{-8<64z7 zO3^4~dm{|~m*Iaqw#8OyJ_%2;N zs#@jG3f6IEJ;2`(&f%`jt;b8)b>SIV`DCrke&ge&kAq1EA_Qj0m!3iMd0Y(7YF|?Q zZlm|a`<=1vE1CcAoelMO-t#mG{hn}hlRr;x;ZGuZq_zW4dw zxUjDT-K8r~a6X`aDLq4$d6(@bO(xj7f4Q~*bb&*+Aoc|qCV0dFy!H&g6>+wrS+ES8X^@orUmrUUg3*n95gq_Xm z&mP|SaaZLlqoy~!A^Wz4i|G>xv9PZZrcRZC$pmmB*N#)jZHG1G+r2WMNtM5aroqoC zKljt6o~G=%#}CpF9MvCv-tliTT^W8TovCG1R8*Q<)L-L$30iFQBTuAml8D*>TTB&h zQ7x0q78kButqR3 z%5ndkSN_o}f5h47I)5CBm$RXA{y9H3ixB}!5o1DItH3%&)!}|sOo3zF0pn+@uG?Ru z7=c?d=YXS_<;lSNDZ>GZmJV1*<_a>zB~lZ7Z6|bh_a(W3+b=FNR0Hh2wfl?M<@?hh zs^Cp;v{|6bGf)mQlCj`Jrc?8Wb~FcrsBmKvd9fG&RYT8!mqlpHk>k&0UG@E?IgS?) z{cn*FF+Ll-P0^2fm}@zcqPv+dm1|8v3L zd%%p*S2*~&K?OURHI2h6=ZT znKiHYnH~{=ERz%GHpWAriL*ZO#BL3yuBSq%Gz;{CmC@3@F%YYq2~N%KE|et#GTFUv z-M9tsqW9+N>vAdE;Q9KW2^uUHPI*o61G}svBHvweLgfE|hl~`GO@-0io`KAC>E}^O zLKy=^^d(RJ{|4>5Mu|Zo`KFE&eNc8AI9tJ|+1~eLCf=7WGLdDRcZ{`#Shq4k(XBOq zuHRY z5sGFvorz4uE}g0*|HYA>Y@$xV_=sLD+)qm(8T{79H})C`0agE`9%nmW-hn-0zD$6Ri>KqkFZv!~V|B@-(WY6Mn2}93~&34LsZU z5Fy`??4E8^&E(@|76NjVO}l=8q=4{wnj(CbLqDi>KUK&4A^0P(hg9u$0){OW>|JSb z9)@F=J3z1fgN7sC{c-$O;j7b2S;Bf8grV8aI}r9+ZO_X6D)+OxaS%;NwnSNl#HL8P2gR0Tp zWqXw3k~^jI)HKxvs}zK{{91qj$sDR6T3sz+ZmC)tV%h^^v0+?-z@l*JA&^eV6|&wZ zz_x2pX?NTdB}nce2o5quf$J)S?5vXe1ZeFjI5HnDMoH^9MXq30oXirxZ>+?BiIP-$Y4Dd$_COMZ#cPx9s;lb@ zCx2~;vxZaLwfbY(a#+RIKs|L+Dxycf?X%y&!!{OBl;x_hNFLNj3Vhj3n~R)yZd z(6h^C(wh(SoSN&ucV`7K%_E0rS(P|qVwSg5mlAHi>hnADJFsskP-Ie7S65GF1XO=a z5TNIeb5CZh1j?_JjEPNaCKuY?iWH8lezr9x@Xwf#$Wlh5VXSF9JVsNNpOv~wLqT{32W~_tJ zjeA?d-GAI5f3FiUCMuV7qVBQQTWVJhdN=j670z&PMuduph##2C;t~|n9;j$&VYb)_ zMSLDLLB8Rob)VO1@vJj$(r+^l?@ng~fZ`>TLRwb-q@#cCUYJJmB|3u@@=!mQE9z0(y_rrhc>J)tbva({lWnV0_-4E}f2#CgvXngvAa1jw8h5)aWE=P7@X-3) zKqKwO_j?wS<#yDvQxbQuL7aG> z)z}!|n!ah=eNf&7Mu&l&pJxXRYcnU+!U^6GO*kAwp?tQ1| z7Qxt4OoIja%V6y7QF4KpT5_=O#=aKA6VF1N3g|n|#LB`}8h5H!X`?M26(~@^nXcGA zneWM6-wloQd;8F%6jtcbd3l zyeO?*hJV1#R`?pmZ#2}OwJs2Q3;7-b9#8!twl8UbsMDGS+m))aUJvl|9^=LVgo~ye zWnxNTt3e&J#m{)2U#Kn<5Ru~#E9bQ+}+K zn~u`<^TD{qh3>ZwYwmHciSzJ6Cbu^%LS4HrSCm@RH6Xc_m8|mp<3)f}uro(9t4Y zZHtk@3eWSpjZg?g|3Y9I@xgEREwX}YVdaW$?wE`h_cyydW~72lilB_K_Y8o z7AZh=hh}SxU;b_bXj1p?*UlUvUrBX;RkB;Yb+*u=|2h_DlW~6zN{)+SGJe2Z zZRyhP@AkJlE2-VUwT0@_dW0*1VzGsc7PTJg&{mQqUF=He2@{0v3BB-Y@Ka!6eU0QT zFqR>g?p1>R{#5X-o*!nW%4+}dCsR$P_#C${trQqzI+5sK8UI?O8FF10NSyAQ0F1+X zo??i594)uqc$vz6ru8fI6(o-xGT zhO9IXq3JOMdzEwIs)SzSfv#sp&qDo=o|?0(AT(2viGdU(EXwZlkA-=Ow<|IWT|q_` z;Hj~V>X?WK0?f1wNx!ZWV;;-Vhsce4Fo74&UJW(>Fn_O)r5}ahG`~r8(&7iQkCD4< z5yTn^dVl-ZPSOwvu>R}vSVVWZEfG@@ft5e6cjdzv@=WmLr{aR90?BbMJ_rtSCgc|Z z5va&G`JoAuM=P%2Kv}+sZPK$L$x<+@{!i~?q#boA&cC2hyBtRHzK`CBfDG{uv%=Pj z&t=Zvz_Yg(F?uEF8oJN&WEFK@TUN(-RA}$j_DX>H- z%So4faltu0;Rn#M6**hJxLySi;3~*+yTQyPfar#%d(pPwAWcaVL~3Lrzd36$GN9Iy zgG!5G^cr@H8ds3nWfqA0P=NV}+GpbQTC>gC6aiUGag7khV|<6H!l}!0f0#`dGlkBqupHT)#1Fjvvurn@B6^YVFU(n2Aa&%ht%o z`9a-Y0g}!4j^2SOesWb42?&;%)5>=!E7UGEVZtigl?~P4CAD9pM{tdD)6)uOwgu}n ztm-PpnzAIs^evnT`vLO#Pt3jzDn<+uzWxZQ^_~7iJX*kO(w#5LNVmYz7W+Vwl0cYd z)bnRD$fo>VdMla_yb1pV6Iug7w%(e!VoKO-4WviJs3wUV|ACrNbwK5jM=iu``JQSg zgj(;k_i6C!@8Oi&MEilFjOHto`iO;Er9>EFj51Ze7E7JSY04X zwKwo|{%6vvb2{0Ibpx29dJAsXJ^ zNcq{LSk^R6jfM2k;E}PXp@i#MzS;AiNV54EY+-lUalng4W{E= zMOzhvOwi`|AO4r{&t69u2>$+cg_o=j;=d%rMfZWw1{S|G;^`r$)devU+#5h?duNyi zTS*_gAzzbI4GClNOp|m%^q>j_O4}=mFo@S7KXxfkrw}Z!nMc%4?OsA&GnkmwhT(3M zrb>eRY;K21O1^s*Xm1Jgt9Wy`f8>2mF7GeRV*T>$bOahorQ0NJw{y2!ep1A|2A*-7V7HB`Mv_P*Tz$ zGjx{>T>}j9z3y}FJ>R`&@3ZfJFbp%#^StX>>$iTZfVruY2iE0p2WeS3mwZB}THevu z)&T6iES)pm=595z63e_)p|{0ax_DA+KtxkcR}QF7Xr)P=bFt9LE^=J8j9&XUO8Z~o z(SN}#Unay8gC5%44{FeyU!X0$KbndiVF)F(!7wodV|-S2Gkw^x?F z;$Flhi{ZLuJBXdH%Eh9-QE-*mkF$T8aWIjycfVJ~wX!lXEyej`so3Z`9S=Gj*@NaO zPcl?IS0NZ}6OojSD4F}QkrIj&F-zR$k9}ut`la`aDVK2+Olg~J*>12G)`>8Bw73Ov zv|oOuIuk$;_6*ux(N&G=TjBL$DZaYog;hfreTYw6-gBar-?kAfza6|ue9fNMIU5AC zVI}JL^GF)trg$qNj$P}XoUb7Pon5Yydw*yEFMj{rCHlRXAiq@YADwsU307lAqcBZg z4h(q>_K}u22Ht>el77s>AN%av6H_|P zb}@2Xabn@bLdGseq;g66=O3hETN}-faD*E5K11vR&%=^^fXBW}xEewoltPIXQAWAv zbCV2&gIYksFP#$9G~bR0Cpn@)k+2HKsYu5v#4QF)e#06d-Q{I3LE5jjr(cmC5IKlW zAWnE^fssbkDsj9%wJZbc0tlO@6*^C;`Wsn8GwL#o9%NOB_rz0-Ft^mV@h9fXgj<52 zk=jq>6zyaE4}$w&zpR!KvS^=?3W@C8?`;e|`hCWj0q%5<{4fP9ez#7Hku~sy@FZg0 zQCVw`%6w&VI-Zh57iZfkh|G|xJa63n_>9X@^@+J?E7%Au?Oc`=1`_LC?4krW5Mn$} zYh~}NCN7>RuP2;}M<4M?6G#nM+7X*?J8k@xQvM?(oK?IZjDQ&Kn!-^YMSOoa~a(E~PXO&(tYdGYj-iL@r4D4RDc zUNksy8jMkSUzIj#4X#~$$CHtNBf#D{E|d#Wxe_g0_=!}KD$cLQtoXU4-%->VtI(^W ze>%mGwnUtj`MK!rySJ66|DEu1EQND10ioHt*9&@0(L-AJWK25xk1PD!!~WNq|Ld2l zYxM7DFc@B4-K{r_(~TW);3(-=w+^C4qNDX~!0FmCPPgPFu2>)@w+i3SCU+v=B5^D67H2aI|FbOe6>oO`vINPfe9^zBu>oZzVT+w;!k(&njVmlPZ6fTB;ts8Ky?m*WMvHM77!_X z{1(PX#=EiIS2x>g2@XxXWFTI~42l_*F(bI74Yqb&WM2Ftl1WgC4mn>kTD8Bbk z3t`M!2?JVFKRvhkY)LH{fMF?M0~Cx8ui2QhfVnb5Kk;==hq-O+jz!tC>XMOVIezLJ zshPJgS*#`oMoRu}Z?gHnT5kZ&UmlnQ3zTy(fFW$5?<#MPn>~mZ)H%8i%1GbS6q%Zq zEPcfSTK9!OKp-I}cRbf}y^=T2Tcv-`CqT$YR%2?CZSU&pa?3>1`0|^OdD=*LHIHa- z-RYN>gMT%N;mv_?Wz_88zVniZd^z!>^AjT61!_tHOy22gEANrKjRegMzv@^Ss~=b* zfiGS}a-xP?vfJ)JN#$vCIO;3utUVULZ@vUOCdf6jpuKpSOn{Fdlq~TiE%|?wKio{m z9LTG*b>OE!_YrtgCG8}P5|ijz7(o)$;AM=u`{f{oQtN$`7()s5JO~%4#0-Myx)r#L zeBli0k9#DF02E%+=QWDy41eewq^?+l?_-oFFvuQpJ2@l=|E@`8J!v*E#gxT7_!?i)`r6<2T%=$YZvc8+`dHG#`!+x{; zzfb1)?D6N3t#O9gimLF=`5K!~XDqa7OrPaG)t<;d0?gq79Z<+2MUwSr;rHJdL6Rxp z8N+_9OfpuR*WAG{;{N3atQldFP(jYLZn#i!$ zNV5D5VHjAj{`70wl&m1xks0!k?P^6HmuY7#D;}zBdE~(Pw65 z#xZDOvLaQ?Onlu~Pul>m<%oGdI#RrJsy@Wt8ohX?2VSjmea)dcT(kaN?!(Y|L;Y2F z{L>`A2^U-aHlk6H%NO%+<{Zpn>XeHhew8LMo?+=MkzuGAGG z?o`iFAN7BboqrtukxuTLB$2^X`W)lq)>Id7CsMX(nKpqC&*deGk6+aeisTbmGlZG^ zPE=w$q#g^KMhI_LC%y$=xJYv90JY=v0UGG*Dwg%bpAFA?p;~$8G{gZ*0whCcY`N zA&Z5r2AG_&^4u9Qw+|I%RxZLOb^dl&ow4k6lgz_O$@}6o40Na%FtPXxyvZ)Zhuf&H zJLSMH&PWX;z$pk^7q(->kw@DJ|DSCA>5pGOcg6oQdRul4vsXzXaVz9IEsyi5m}{a{ za>z+k246U7-~OaHAz%uCZAGN=kDJ!{$5US(%3ZiybPMo;% z1|Ju09gu*z8KllM{CJ!M&Y*P{5+gsTabRz&-(@O-`AsiZtzV-t5)v>@w>Lbf%_YSm zWjy=)d>`**eJ_{k!79C1(+kKkPT5lPUdV4nqO}3OD)qs;Lb)>Rjc=K)GieaEzTCYx zKh+gXtW?vMkn2q|f^*yTJgwt0U33TL{h_;-Ylmt>DuZ3f5A(aw*zj>Cy?uNby;eY% zH)y5Z!$pr!zBYG#u(NY7P@(BF8~O!ExZN}R%Uh3oGCyrGYPkTv0RMY3m9@iru2j}y z`Tu!ipBe{PqbP*6(-SmZQg)(Ghh6rk<-^y=MLE4`B)#(|jlM^Q6icYQ5NCQdGOJd5 z9nK1+Y)>$q-%IZt_x1@o;>qD}gUL!gy}dhtP&7KbIy(xG4H{;xOhGPZK5U=!B?|w# zxDxlX<%zS2sYxtpTsh533(e;#s5u z{It3PEKVQzgvaIn=4<~@`m4I=FO4vE-MW8pv;Nn1|I^pqo*~wvR@eL8w~NvL-Ld`S zb6(c^8$XhZAmNBd+p_;#h4IH5j>oa&QSA*3Rwe!Pvg7`fh5FZ$=`^Ky{OLVj;{Knu z?T>gQ3%s9-ifU!}n}^!p*uj5(%V!;gFn@plL8`d?zg6FV7}hQ?{8)mCkw?CGhhmKX zm(vQ6C;j6eqMbkB{;wPIZ&LG*$0v>soK(H}cy z9Q26Se+5ZI8vvA`+?JoV40XdDQme+$T@5VQk!#_!(dcSA~M3ixlZsiQ<;hZ{SW72HNW z=$d+b={AtJE09t>vK2l%%01sh0Nc@N1A^XiZ)cqE_%~Z%!M{f2FZKPBMUlLIt1oKr zzN2-%2cA4+K=O3!>pm!}fJ;`q1-D89O`ETr?ujx*Ymv|}EEMYLfdm;Ea7=y=p3 zfgNz%Pp9>BB?|?_+nidVF<4tN(&M+=)j*?8!?B-}LN$H~WL^WUjPeO0NpnpGH$v}k z3W#)cGV{p3CtnMc!%!M4?s`Yw#Xu(uWz65Lx&j(53WWhF6=79SeFNcd6v!$K@$pCw zX(#N4hVrYh!TzSEkk>uk9;7sMOtO*pYYTiQVQ-W{uatrzg=OYCO-vH3+EbLh z`b4>1E2Q?HFm^j*1|RaZUer0}-eWB9i;vHCt&Yv|D3F=ap&0Qa>U!&)m3^W z`#`nrmC0fp_R_oJ@wJ-@ZTY?A*?qUD$3!|lH?^H6k)&LR9Cq}#=^R!hgw;zmW}U*O z8(%!+lNPD;QmWPiA;dyMV$($!T1Y|#(n_(O=Tf^m_ad$ef%6|@*<@>DTOEN#9w#Fz z-*7~d#$%5BtzN+AM;RN|YMrY!4h}-rKtc*aYyt*$#5OqJ9j|sH{fw<0HXGCZ&inzI zj&=1$fs2%D-D3G@v17bFnM)V3cs!H$h}Y!QUp?F>VVf>sdx)TYrqOm<-S|gpH?kk| zn~aF%yRg)Qd{meR$D{q-U4zSCRnaRAJV_RHq%0;pvHWLEpIUIhciu99Jm?|}O_@Eq zI-)yu|IZ4O1z#AEbul5&;cnhF$R_r}v3N|&Kkv@d;AB7iiLfu*7vjrc;I>tp_Nq_A zy7LF*6WCDLKCq*iu}&^F$uUEp=%sLOEZ?C`xCG}&_(iwD4)eSQj>_ROUK{J1=G*H} z5f%|W9P>&N;4zCchzFp>W<=#=yYS$J1ggK0nMWwg-=r{M1-mD8 zKK25Dc8Sd3qaqWZ%9q_}eIi;A^X;$|*7dmlrZPWGC5*G9{fo$VE z4eR*iB*`N^=C@z^GEl0%SS1H!fG^*Ot+7g&tdO0*V1fK`5R(RDGF!jRb>UL#K~u0XKEu$e4407{xj0DG&8lp6JC zoA`H1x|nN;=KaZWTo#X_(mRXQ;aXpCgT8%w53~ z8wXgJ|9B$%DD%!Vk&h!94X*+@o`CG1wZ>$Jk?Z>+?d)>zi-LMpa=^BMiNAPdcd=i` z7r%(UrI9PKGj?Np+fI2(#7359ct-pz%pHSJ8}2E;+U%S(|Cxj%-9_6Nat(~}I&-X)?j}@HjOz5q0Yr0lnjalE;{c5uF0X^IY z{B~X*?p;nX{~U4t_w(%PlrlJ}SH0RkTfb8GWja5=hQ0qav0tpG>bU)jXGdzz@?v>- z(KLELA|xS9c6X7Vyy+DI>^&INCQvTbNt3>=`(el@hZ7ueOWJV)@}^z_-BY^R>cB~X zIU_3kkD2q)${Ise=`C%50Y;){p;zchee z7G-)kj6x{|s&pb{Lm0w7)ichuSkab;^ zMjttg>NloOFJo9CohZ+?m0uZ1JrX&mG2>C}yaiDlF17|pOE$f1Y!6W_cw_SB$%HIV zP(rQ^OgdGhp45{6f%=57&2VSlWVYFwI{QXUz-T~aGt@Fli&i^3uzAgg%xM7kWe1pw z+d&Ef2~5swF`rw<)aQGJ+aed>J6>Zo%~qZ*Im7v?*d)=4&*B+l^OA!*goHg^1kraq z8)RBU;YiFDpq3;dsE;q?V2*bm%Y`e}?6q({qxJ(J%A^nJ7>}>IktkJLNk63UixEu`hmQ^s4hctqH8jW`0JzEkobEPh$ z9suKF+9CM*U4is8uK45*ydNx6yUqFe>sHw(@6NH$R9p!24+5Azj4V~-?#Ib`I_GIB z8LXnp)XNb|pGdVL2Jf(;WOsQ-8fTIE!hZ9aU%Lgf9N;$}%HzEXl`4R~wxJfM?Jt*@ z$hrao0#x06Gs%8CY%qqmZLT)&D|O~>cfADu`hp+Hbn&ZOBO_laK7!~YriD)w!KD^E zJg9oTyu*HTd#Y`mgPCRiXw?>Q_C{E_uVand}c~KM7)d3qc7lJg-D$X|N!vK>gIgrQdZ0(zV~K zx|{HKNVpV#gYRx!MN{K=NMwFlfm$SENq%KDec=YsrQ|`PP5mc9ZIiW5l69yxs7)PX zSuaOZcBcIe8)L*qJzUC#9If}8Pe!XX&xqnzE$QFT%u~p;`UC-^Yrm>xnI0!Ssu?>N z%16@>@S-z~-Zi?BOEK_#(NwVAk)M!&@m0O_tPtZXXX_WDuor$U7B6x<&e$*3g zqcnr_J(OvsV?o(@Y}1@y#%IqMNS7)K+l}O_y{)Tqstr!j6U)z0PUjPqqDdvWfwIs5 zNwesI#O}_gKniQg*#({eeWzyzt(7gG^}-zTLzoMk#Ft>v?!<{EEvD|*d)2^=A+{uy%B!)muln%7{_KXB5WgvLBs$?KZK=Q&xeE1S0`y@L zSw??!bEYkmfM_(695n&X80`y!f=c=;&m# ztP*KIe}t8_eyZv3`AYhw-+Xy??OwFdnAj<2o6k->Nv9TP(J2 z5qCJ7--=bH3Tb?eYEP7oQ~Uz36;FSl1H-V9hm$#Y&nBhpjRo$umy^mMaf?ehH#ec_GBj3o;{>Z>aX3zN-Ht{nXB0zWN>RfonUo;OGA8%hb^V#aQed&uQ zD^;o^;(zM!yxQcjCRY(WKWb z8Gm!s5Mh?*xuL$@VVber;q$IGz=u)DZP{9p(`wH0dP7->E#q3R&7DNiEL!FSbLIq+ zT~U?Atv|`r%-+2+gH@E~lb)0*4c-~mEnomYaAQRjN4|tTu8Y;>aRpkShiT@+JCh5` z(Jbvx`jZroMVuBR^QFf%9M;!Ej2}T4`QX>d#tzHPdnT7?4}~+1DT)(Q@fW|6#R`o* zNCp}|L;xdIO`D6~M2rzoHWk~ys&X4BCjdqBMv7VXH=-gf22j-4svS>~g+*&Lic%nz(q zf@Z;xUmNmFWKCuaZ%fn4=6#QT0`)1<-G;+Yk|9$$GyU@+ycL@B>4sUa>U?xp+_Il?&^xiT{5pQj(6|H-HpeiG}U~?ESpH} zr&eYBZxIZBvp@i+^ene~M*jx8=Q69U;$ey7U8#`B{V`7FR%f4ngGJ2O5U~@JUJ-Ek z)u4Vh+e}H!kWm<&&EChASj!V7w8b_znlrOE)t5_t$LJ!?hlY3i>-&-39!~k%#h3Q3 zt}irM2ME-j(e3OnbrANy$qa45x_>Tharf{7)K0SosiY!9$Hj1Dnubz1opGQgb9_*uKDU#&(YFcSr(re_1>6!ZjFg3#0p!i@74AbIlc`^EG zSIC~=(ndE7X#u>kJX|{oR$gnb!NJv!ds>280c7OHFMQ<8Yyj4^05rkB*osO=#e4VT z!gP2eT*!g&sZsP|QE-z}BA%h5h}eezNWM2c(4YyW8&^aV~8S zXJ$p#ji~-jRl01N!yRviHf7qH)`2AUDwH@5w4#$i_O$O2>@=g$mJK;z&^Z&d(MnGm z!512ezr0Jye`RGsszN2ZL?RiS%4qY8Fb2DCgpS(B$;qSFe<3(ssU zTQFe0jj&pR_C@}g?=0~a2GPSTLr4L$ESFGyBedjj4Vh_-xJ*8R)x4q_IkbIx6;Zn9{L*9k8~<^w40BaYQMfPaN?9dizgYgZ~i#8;axy>h-k z5#79IDW1y|*Cx5W(zPmIxbpl(IP5^%QK?mjKutD9>wD|20dnM+2MV!lMDFZ7+nXIm z4f`ZGgT>*v>s@iVcKM94_w%0=W3@B3>Oh^27wH%*J0sb$l5`wJiJD+q`94p~8(aEKCX%)I>GG~iXX&J7KgMTCLW!c)ysL0y=lzHfL zyRn4i*$KDRzSa*aW_f~ol1dY?VJp7trvBQQJ;QbXLjoZ&0|6_|)m0_lBn~CWflJbb zTa&`8ZzdA0Tm`FDwvf~DZwzG~!(T#fCg<6^t*P491hcr!7N2U;G_{n!YwJ%GYILpl z$Tul2}wmF_ZL8I<#!jWz?Q=l`*6ogJ zAiCH?c{i%k@TwxuBwq&ek=KxhmdH(}@tjt=sVS8Af{DW=l-(7kIcn>vFR;~{BYu%& z6=lnjU%tWQ32@!+JI!<^->2P;8=%Q-y=D8xZB&o1@Z-$hL%xoksv>+>**C4NrCI5W z8Sk_V3xgf52OA^H-CZK$jy#IKX$t2pc!UBt{X1N!cUwPr{U+8=lAp0@mZN*RK+Uhw ztA;<;I34*yV^Hmbnux8ydF6@GocgVa$T3aGD>irK(mO`XDzy2?5xrKj5k!2tdwz33 zYe264iUqegPMbY+DvIUVMU}tuonJC?HrhLMw5e9Vox8 zno(g&E!oO|Hr_r)bF-~Q4d%)c4X4{;Q?!5fZAiZ2O4g{&h=a!YZc%E8a*V|)+wwVY zczFc;brfpy()L9Wa?(5baJ|vwpN|SqOK8NK!`N`NdKg2Wed>z}+m^g|g@myl`jd}B zJ*c52A~E@ms<<56`4fJ$Y{qBlhlPbiji@i1Ds{}k1hx}Jbt79JCVui1WvlR7^B-4m zXW%&cy%y9oKMcVTcY&_;}@ol$xfCeE{EVRD9AU3)*<|xcGjaOL{=m_3pg)FsiIz z<7hvCbjKDdV~zDXY{eAsPZ{YfS7+gPWJf0%lo zl0G2MVG(uHvX8GCKpQ5X%jKX>v?)Cv6~Z-z!lU1l87<6TWMoaYa|$> z#JJ)4zL5QpBdpBhALK{Bnj_3T{u}wOt}q|F0qt#Go7g1pFHThYo`I=mTdeHN+Z?XH z2)1|FttRj=KISbdOLV6W?N!^m&ZnFk;l3%%R$p-mPIhT=UyEn0nmmCw=0%&$J)WK+ zo@S2hI_6Gz?mOjtmJcQLo36EJc5005{`gl8+v*agxq-ur2hNfc6K%emq6kf>Tklj0 zfw{J!cn#zi;e?C7m&y&2nm;Ks?or?GTm?9Q6~729q2|M`WZl17?MG8T?VkJ!zH78I z#XfqLR=;#?QNh&g0~Jvp&-oPCmMo!Xrm_k+XS-s?Y zB_mWl2pmxqSxxx~Rtl|Grv6DaL*CT?5vVa#`Jrj2!9(iRST4}L`lXiI5g<^~jT&FO z+D)pI<^dRAgWs=oNeHN26G-d1E3Wc?0DQXpqwfg16+QxY!gw=%?0(^nS1m(k;Dg5= z`2okTk=pXTqe4rTn{jNt^t*r3ik}ScdgM`4#}DV!_2+#^@$KLI3G^+v+VR#*5pHq8 z9(^O;>{M41N%kyS6XrVAW&3-dm$rJ0Ry@qI3)DfQB9k#R;$7#49TisS!eo{xQmh7!N|!Ib1`ZP89`PXzMTs!&@3ILjrfjYVRq3Tf26>bJL-9Fn<~R&Gj_9 zYO^0yt6Y@98L}^8~rwC&hHNS^Zn)$5)x~qhtZaG z+ui_D##37@Z4wllDvmj~s?urUN#@Wt?QdvrYTtV`>@{3=AmH>C+1Eq2HIPZ)FXEb) zna-gHnU;F8e#VW3DXm#wpxkQdXagFdz`tj7I-4h-Awv90B{kFo>*2W-x>a#!be7b& zb=k?SV%J%}IK8hTIJXfV&dp#LTr}%?Tb|tcB=l$Y zuzu{lPR*+^ew3V#57-=?nD7N_+Nf?DB6o^_x}Czz=r;HQP!TDzKZM!&r?&`%o2X}coOkcm|awAo9fY=my`4$ikKTdZGcm=$*B3m4SpXRs#y zGOebx>mPNSXF}%}nGO2pN=O@dx>n``!3}pxIg{1>Ulibr@eH9s*S!QurhP5HLE{x3 zW`S~2kIoDU254S@1GZ?kYQLal2m(TQ^q8L819HN389D2+SVqH*RHtrbxt#^S$h*j; zY{AW-#Bk1lUv)d04?RTSu8gLmo_QQ;MsXkrt6HbUn8>C|eGs1w3k(Ey#*)Kco7t&cZUk} zNGYq3r&aNWw0Z<9w|mcs6ueIOL4G?E#J8Kl?513<)Ao2qvdlnGnSk}|El9^AGJ)Sm zvP@&_u=4^}E55JjL*-i%xtOe%Df+&3wxih`ydc~D({XitP~QJ^ZU31BlPg6;I>GO9 z5ys(&LM_=I)AvqBZ$QX#{QKnZfZk-bgb$m(9(o(dYcump{vTrMg?RJ{XVr8xeeLD~ zYR6w=JQQ!(6O3W!U4AbJo7#G2@1_#E%P9?|Z9jQ>wEIZ+0X8*bC7(Tl!>y~kGN~!V zj`D(JmAs3#&HmY+z-3jLr==ALl>G07umFas6r-8(HFC0lw540f8L)wS)qp09)%zt3 z%bAdR3$=$O!{woq*83{CR=y)+Yuvw#VBEGL|As{S9zZv?27s>dxuwZzPe{NzO<|9@ z(^l_2x^CmJ$$ILbRxp%`6Xt4*eIY{FR{Szq=lTS)d|)3`Gmc2e(j4 zrAw_7dDLB0-N;|NyVXMZW~qGvT>58P@n5fANuhlD9vP)S!1V%11zz2}@?|4ym5*&O z%uxRPm=`SCiN!kQnd^9GnwQON6zzTW?ZKVF(J3qenMM42;XflR!rRmHNw~~N`qB%?0YHoFQY`-02LC)+xte(5(L&TSy;Skm&Vy~d zOP}l>#u}r891R|ydhX`BHF8%QjguI6Z?${`-2GW(-H6#nM#UuDnok3=w=Z9B4&VUZ zZ~YRL$+A~+>F0J?#f=n!h7*8-;;o7q{t(x7@%Hwp%<}lmE1EB@HqWN1TSLT&vLAMY z*lDJ+#s*O@)WfPj;w6`PfZnGq1^Aa1g5i_vh@>Dz?2oKw?HE8%M%KW;bB;#;#~%2)wSzJPD`h;VA3URIZ$1zT zA>z}nGY=`XPhwxSr8hI03A7M$ai^Jgr3n@`LR&wp0`+xh3qG$r8ZT0rc~ue1p>GIe zsz(Ni3L0G8egkjHWSxC!0Sb0J#S4Zrk>oC|xsBdi&0=P&Engp2WB?<i+5CGtJm#c#d2 z85#6OjpJj3#O2`KY^!751vV&#=F@VpwE4q=`B{0nE?@8d*%_=i`9s9prkR*jp{`}w zj@d4P32*fOd~b85-s4{tn9>u)jST*d1$RE&Jr ze*ygjepfM{Tj;W9QU(!}?rt$cGyIWD4J26$mcF(JPiZ;t;~XiHo-DXU)ff&@{!DRf z!H8$5+iwKfYS$rWq;UB+AuA8S4gvw*CDjo`K2omM*@CglnI?fcNl7(%KA<1>wm(&@ zS~OX~wcKn7vqYoqn+toEQxw;NK}V+DUTpSYaxuQh>$!D!{HSobm#9vMJNc+_r z+4ZH2b(oQXTt{&tTn5{|mJw6mnXI-kv({^Q(%b5Tg`?4U)e`eWB;)>Z2j0A}>J-r_ zB|}_;-}qdXB4nE5eQDr`MMul>i$XR{m{*p#A!1K54*4M0Q+-S!r~TNu0$73VFzzff zgtykrtBylVdsxHn%+no4(sG2NT4y23knChNz4skvt1IU}Yp$5U9r2{|p3$E)AV7$I z0Z8$&zR~opNa2!TP)^nf+}PsCxUzhVuZ<>m-TQy2p|}Sa;gbY%9Bw0b8bbRH`#lyW zMSMYkzL2-5Q{Rhr89q)ip3IhXcrt9JB6WEivbMkfiAU&}_wti>VV6BEgkko8^E~nd zjMgVvVS(5{{`^fk9ZvM&f`tMKjc7IDlJ<;eMR_%*`M0IECcE{tsw(7w@r|M0T%IQ8W+VLvFz;eZPm4Bo?peJU6tPYfDxnm@x=8-B>hmz6X ztd>GFDVN%CyacPHQHMfttCm__$pC8>K1{W-2D|l04NLr{MKQdi%r`d!VL-1isV7u? zDrj;O>7D@3Wez9glJL$J6M*$x7LXf*4}e0J@3J21fYg}izH{qD4sm*&T!*^bU?FT* zbyjS@)6WmZPZtI$*4kePhkvfP=xTD$-+3}!Ojx^p@trGwCvSM-7mIEKXQoM%nGJa& zgzNEQyC>`SNg#Td7-NQ+q%BT(dUEXVldQYjlPnK&&|@fpF<6{`zz4)Pr3%k z=$(k~;nvZ0__YXV2>`>&hwhVM?0c-n8@pJZfr#FZX{=b;bGIsDe~*7u%5>Xj9~FMv z_k%7cF&4bNx{7W#B%(y8nL@6!&}=O+fBxiihc;LN-z@`!&nk2)qdo(ohw@9U;G&Da z%IWyR!d zqlpHA4dp|Mb~Q`ekr&;hhqbKWipyRkTrqv5W2;{hVZOJ!0f`l3h9H($?H{UE9wrN} zx`Cz1B#*TfSDHL#;Kx8LOo%OQ)1q+G*>|Y5a%$CT^Tz{ssIyIk*|_(pNS9{?Z37VM z`r%~Md0%rmEp=aHf;CaH^zcT&Lw>-wWFeEUwCWhBEBPBUe{e05V%a z!2T2ETn90PF*a-B;NldLM+IkE$jTq++JL*+AB`zN4a*oeuJ=Ofw$Cya!-^e`oW;!+ ziZ-n9`CHqQ%hbnPp_4&g8azgknrE6wpcb)+^{F(%K~5+yS31w)yvoY#a>D1?d8*rF z8a-s>)He>JBtQ}}qX)?iG&5kGRi|>m0=b+tTD%k=^=MgBh{G-RflvV}> z^?SO4C7RtvbNh)$&w;eT|AOg%ewJNu!e*B{wp*mzIKB_$PzL9NmBe@xuZtH~nx##Z z`qKZX8wvsnfVBJ?6<(Dy4y&Vq*vP&{FyRewsoiVxy3yC7R=T|CUe-y-%|hLw%nAgv zfR}8C-lgdYzF^&Oqh7x_6R1zB|3xL*I6z`L>AqLTU@KWxNi&WOiF|vzJG`glR_XLy--MPGBP^h3)cWC%|l_UhHIy zE2dsJUywKa^6c}7NPH2x-@;gSTd>4yAToRV<#UA+46}k$8v-M*NLsV=C)=?(VW{VRkMoUzSa?P(vbv# zopIKHCXQ*5pGMnFu?Xy|?~2e>|K?l)Ps1ZX3giATxqMVa-%X0HX|uM-it@ePPayIx z`MvOr)=YAKOpRCH%}&dP$E$(8y0S{E(bMOa#}tGy(H}{Ohh$d43+>)>sH{CKMJ2Hb z>TTSR-Rwv%25$g7K58XTGAh*h9$?5qJN;&%o3D($7bG5^%+W%z{Itqg`JR4*@UnbO(0;efqZL@S}WOe{ZHDWR*(DD{IKr zXBuw5Q`D5!rd^u|z9weZeGG4`q+f+#I)!9Kk6P7Bd8aA7%#tFUEtI{-~v!3?o zc66UI*iE!HpQ&>O=Oe%pEheto9?}|;jRvA(VkEDz;Y}4ymp-RgHDaWa^7axLns!31 zi6T`xRl=6(00?GW7hlT@XTwGi=%+(L!{txpgr=EZFV7fY`4en>9OCUw`B6w+- zKwm%|F&FH7uRC-AyVLXCU2mX%m0sZ7m9V7NVMsXrQXVIA*`!>=UK?W^KSsWUr{8#l zP=Ql6*QfoFqknBln~u*_KI^vN_KYP>f5TOG((F_X`RaOO7j%%yYbqtb^l;gD?2UUc z>mMAxH1WQY+xIJjyZwVe|K*(~m=a_=c2}I(Wjj+yLuq2O`A!bcU?b-&?z{N)O7_QN z1b}Z0364iuXdE#3p4`<|IUwqzVfSuN0}dMUneg&EJ*EZ#H-41K8Y{I^x z2(%gS1A3Y|aeFgsIx}ilZbA_62)ax3{eCVPHXd7_k#Hah!iZ>5KfwlpIIcE&r_!xV z?d%-`mJe5Bg|4OeO6hgN78V-!Bxsz4FyZIvs$c;!tV>y6CWqEWg;7Ji$W892<)yd? z_7gN_H0Ff7N+>1tBDg8@pwuHFuFW0;Dts|o+>r5Ur=M8J?KXDl^5H&=^z-({MlXv0 z1KchBqkOb);X{q4d1z_n zF!^Mc-Ep|ZWkEt+9Wufnj1_bN1-?B+F0kFjv}W!ZZq*I-C1apMH*y&e*d6g%2zgeWAz|#-^$sL5(c#$d?tq97z=zG%Hhr%$2jqiw0gYsotC_2)<9k`oCys3ZIp1qYwoEUPTQE452gjpSPv_~o&z?epl=IZ(8GBf)_9~;DM;rt zBUHayMNKpL+h*soAJnJS%5B#Bm8qCF9@c`-es$so3tuLL;o$?HhWB$vtu7T{-Yg|K zXndjbNu3vw{feox#(I9{-TD{P+xX`_D}N-Uz6s!d7PB^O>lf;NlI-I(w)o;#_VDop zvm>OuTKw0ojaki>`9xE!Z+1Jm%#!v&&rD5IVBadP0U2BT*u+3o$y=&*2f}U zMG2_2(~arj!-6zF1Uu`iN5U3xGdn_9t(0G)8_eIlo8Y2;SuiId`uNwl#9!fsu1*z$ zH%ByUKG(Ni8CTm|SoRNNqcGR|CgGDF{2#`hqe2fLBfYNIkcSx}=RY#sXue8~(veMA z_NeQ$+&y6>9B8CZZ8%{_PZ|$cpIcfJe6Z4VTFtq(+p5)Pv+_V~YiS?HqlnSz?t|i~ z3R%?(X7}#}rA*}7qn_W@4pPv-X=#ALp|29hb1=DeG9`>r^+f;5E+4CP)v6!D5houV zcYYIZ_M%80`QF>hU16}ys9Vpo_~$z3ejihU|?UVcy)*7v!lnmeC<0#d2%i$rgDkswwUbo;_)-B zfwcNdWZT8F?(v4xw8*jSy30>l$g*}4$FgzdL)>3oFc=~Um*(|XoHIZa!qbee^kNuF zE{A#Ad~Whyw^(1*m{Pm1czwF56t~AX#4yV1AD}jcTcHhxD#=wEuC61^I_;sJdNg-{ zRTOn%60CP|2=2+T{!|vo`kBpgO?r+K$ls-x@+b&3* zsN{?9kFjvOXF`JH)l$N)2gu47ly&q8@i zhe=621=!7DvH=fD3n3n-lmo6VuR7w>CZ)3<+vH^_=CoC3n~AIQcA|0O^8Cn>z?fGb z3K28RB(PJ(K(*55)eIMXi$ZG`df9DF{QFvOR8UE%oeK()q zb$lc$dBf;}CyAn4HV{{r=miJxkCG!VF&dN|+;5mM`CCY|-?|)~I4vd|%(sru2HmNl zx@(20R0Bm;?o3dsTOq@R;p{3sTgmf2dJNCdOF}Ix|Nx7}m=U zUn_jb49bGv10wx=Vo&4+Gq-2oHN+;GtM%Md9r^lV6OICsPXrVLG zan^?fCB7Yzt|=lY4Cra3`Xr;~(Ht;UHZkv0Y;jU5sXYdVYA<_^pSMEdzS*@hU=2{P z!%A}hyu{4f`mVoLX9Zut@QyEnhUXPe<1qPMCUk+zifDg- z(oYo5c;%#wc4CZ1nn)LoHrt^IEbM19{+1Wn!>T+V6?XY=S){!})VmQy?{3W7Nh_&Qb%Rf6q4_HUMsEVQIx>uUmm)PH;_-q{}P4<+nIw!LK zA7fV?73H?}r9}`BRHQ_a5Gm;{kxoJBlxCQryFme^yF(BV8M+&!rE7*Bx`yr?zSr~J zyVkk))ctSPtar^@d-i_zubz)@W#3aUZP$DQ(S=&+sHU@Ldf$u(Q*hSR!i_sDOGArI zvsl0r)Ioiu#IU2HK=J+%6#cie#CYuPyK#i=LD3AgO%MkR-c*3r*)h3SiyE9A9m2C@MM4!R1iMzo>Htj zefMR~Cr-@REm&(UQu|U|SU{J@EurN@EK_A>-t%qyNYdNYOz!h+p3S@;rkYQ}EtAjf zrIBnB@A0kP;E(E?A*YnypRA64V-|0yigQTjfG2lMeOF}#bakcK#tzNo-wZfNqUnNx z9B(EwWk0X7lTz%KrA|ljt@Q!3#{-=S_q#iXRP@pn23O$Ojd1_+AV9s z*OgW^N!Kz>*FsEX{ELV+l~T9}vIO~!=07(ps0jH;@tQbB1btx%k-s&g%;9KI$iTjg ze4pzxYOnQ1rp&3}zFb5Wad&^bz*FtFeSU3g28433*vw;^o^bCc;;vxwAy|vexlq1h zXSLabf=~JZA{foQY=P{(285Z@`Z3!ccg2vDTx$HT6xEA~J&>YU(J)LO<0T)b1NqL= z0{r=*bgW)s_AQgGw22%fh2T2AF*QLZ3vRXULgU`Y0Alsi^8 z2}nnTi#x#Do?RCF{0vKrx{_iUle)I{0lFy=^_Z7Y4@ef3{y!nIjvzMf_!RQ}uJ)A6 z=XT}CS`J?T62u65^Vz=R&4h}(;zoJnMfG-x64c;Tcht=i(Bv@aw1>O}*n-dpY>Rrv zHE{i^&5Z6?s2s}U+V{u!CSa>pQxo3#X_db;U#QMb7`sz(yTeC2c=MyMf+Yff$1cDH zZ{HzzdB0Q|y&6QI3kVt#Ai~kiYRyEK*a$wuRy00A{!G2f%$r)O_>tZz@3cnv)H{qH z^$j~MY)^>X72}=l?=g<>Zr!ugQl-CLn)29t{fY(OMsEh<^C^{45f?IhMnCA!ij$_- zSAlt&uEyJc&2PW(aD>6jROK!DNmZ{DX-_=9N5@H0hEL+!9i*H3vy6nC?<}PD%URZV zty5(g#eZJ(*|5zYo}uO9qp-krgzY_T9(F69L!|o!T-G1WmAy)>THhk)hnJ}XKF1b> z0`Ph{q`ZM~e*^gd$$UAt%H4=bGVMt90f>2d|8|hzmx8%wp9C;46j|&kG9)=DzmCyV zo8Ack{5M>hu+L$D@1kjg9g~oHrft>qu+*W=-ZdWT3of`RNs+1&`nZ5RyuGGGSvHL- z5}_2Wp?Ky|<)hIiZ80-Tp`N*R_Hp7la&xZEng3N+gKVB2M-$4kn?-Mx^0{g*2Y~9L z&^TnUOHk7Ozr1NJL98koFoiUxq%dz)CD(9Fe~P$%tLmoj9n$bFVV)|HCKOkmN!pyh z^t4tP!PdRr9k{;11hf>jJyB>OViK^S_FSGrP5ZPGpITyX{-aqKZ`B$5eh81#=TJm-H9|JFWhni zf-nmtK)t8#t%vn)HTIWLP_-?3vbM*zG58C?JklvQyFv-I8_89zDIqc0vE2kOs-b?n z)!@pU(9^7K^Ztbk?fRM^x0|;EAW$ZSo_@OY;@VRlE!1Su$CI1ETc2(r>$;L>t*3Q$ z^^=-=$};6_I1iso8E$onJJ$xv}r9TrV|BV}h<}h0QG1YiF=| z4OCE8h+O^!inuE^e`Ii3x-p3LD=G!xXEC%7s$?dACUUHkVq!PQ`SUp&lHQ5?{(Q3X zoho{Lp)U{f!f-8FzB}wOLVf18_cr#Xp_F?`BZK02_#vQN>S;XeyxyY~WS^`TBt(*HO%R-~N`J9CQP0d@=+(Cs3H-)85B_`pRd%nH@WzWy|6 z^MMt2X)+e|CF1%VOJ3}z-sXuppr?Et-y*$1+^Q#c&N}UdhF6s)Da;r|sp_$Osf+$m zDOdL(mip)W0_e)L4ymiB{S}NNe>t?wxu#PCU)~%PmA}sjvtH-gbx$R zqfargP2bR+zb^3rc|f;4SeBqmltQOJHBWa%o!T5j*2{^`Mm+mlC>s}#xULJr(0{(k5(N>VTY+|p~PKIl^&hsavK|W&Q+(`u;*vM)~|h!*HQuldt0Mu70)(CWS0yx~7X{H;7T5VlQz8|Hj7pqLtc%OyR+0P8@P z?sih!q4Cyi8U2i<)%;=a3E-qhYU3m42zZMPqHZld!L_C5>FSh?RQdVBCyc?qX}Hm+ zeH*FKM#bqGbS%FngS%5VWDg@>h60wViE_8f+9rH0hiuEB#}mz77=fo}1ak=k*%F9S z*nCNR0;|duNgtJ<4El+1B)c>O*Rbl2_Z{R~h`BP2+Q3i(X=ujtPfpgo;<&TXPwPC{ z#dIVYanqUKY2)wAV2=jf1OhBiEAe_LD~OxH5${d~2p8gl`&Q%oAss4_=MEk{eq4(^ z)^b8iJjJk%$k;L{T(famk>Q_?!i6@ z?W^((?@H_jZX)E7eWS4h0gg1SV!F0MK(rTekHCnNj&qAhDLxBDy=q!|2ndWk?EdQI-?j&n!Oq8k)2|m%wJ|iC|r=9i`%K zy6VJi>N>f=#iNf9>ya}M3+TRS1iXPajUkIbyIsKlE$D}BT9&mE97g<7(OM&V!ntC% z%SHkjMWNWDQJrz<9c-d%r5OU1ZqHZlfmU+7<N` z7N=;=+drMtWt!l6%3~Y;{tSC8&Ph{V7M2HQVy27cj^tDM`4Y5u{t44?I&Av0`?dZ} zi0b$&OLp581Jze&(Mj;A5SN#Rt=>pm#@WUBvm8Od971+Qx6WsGY-cS%avwUTluz8% zs;^~p5@n+j$*Y7tp0U3-?5TTC3fUBrjp6v9S`qyUVH_sTcia@iQb*-C% z%MopS4!fVWhm4xws-vQcfGb#I^!y;l-_(OG*_h^l`F`ZV2n7MoH5PZA=NmD&W)S3!{9Ien$lwB5_yEXAJHXy=VwYKCP?LM9WSR=o@?#Dt2OUmzxt+vOF2`^ zZck5>y`G3mTrc@eVZY)_^WssI*O^>d!sVbYBU#RDq!hL%0yj2+qZ3b{QgzV;8+I1}u8`KlcgTx|nBr~5!oPa6=< zwgB`<8Btl)7R?_f0s<{tORoj{*xju+@wFmH=${F;2Y`8sbU3DGQ7sQPq$^$MZ{2uQo_ z`54fMB&#u}3iXBNUmpvcLi7XhwP>nxZTI-4wg&-2dbW@&8k)E%Pxgrkzu>R^#IAa# z`WwQEw2HgQD;9{X)??H2If#?+u>G0TS8Ki=Yx(C$xYAis=!U)~EOJ|wn^(>e?;>!v zi?oso&}3eHAP(rvmoWBpOB)pmNrY8#HBmKAVzN6a&LUDC zNI2krwgNW214z1Ft~W6~s}6dDM;7!sF)W9SF@jj$1~t@n6bkKukhtMW2-?_2{T=pA zg_qYp1$r$eDNR=g@snof53Y2i;@Md)kDZ{gRS*%AXJy{PZMxDup|Yi!hk0hR2+;5k z!aD5y$#9FCrd$&pwvBS;q&#)v zX-+k#4`-Jr53Apa{-^~J_Te90)b~zk|ERw_(leW0J79LX zqA{rXZHtl}1(sX9WKXfXZi=W4-(_0%ohtcrOMioopS9Z^IVWDxuhCmNELeqFRjt#L z#yGUShvu|#V)J1I`t;d{Lpv<=>kk@sr@g&T`rv(S+|DYv%O#z|WkZLZgq`2d)uJR< z4RYo5t|nY410Am)3TS`<4VHK#kEqs~Zsuz@EegK(pmD%r7uwH6I;psjYC@kEZ{HFB zhH|;*-&p{;XbXFQnX`*lp7*)h*vRQXKQg|0;)zyEvZ4;4TUtMD(}0bPS#p?`3f|!L z%xl$K1oA7%({)82bOJIn$)*Id0fOk`O=h~f7?~>M(&foG_HkyONW?v9n5_47mDjdv z&&)QRX?Xe4&`u#=jaM4?5ZOrLMX2V~1Kdo~VwG7>g{PAPzpRFMi3UPmVya+0^hQBV?xzTJ4N16Lp}knPYsY}suTZKLC9Yw!)xIe*Q7jr>}Qt-HDL|d z00_!~8QmV@M~bJ@#PgQCdZLCe=hI}mHcVXaX^t!R;kZE=3xNBSA`t~|o;nuqev%gP zmU+ZbH!H$2t5ay}9%ET>$u)QR?mW|`rA7{#YRR6%8Cz5Awy8z>4#ka8tQ$c3p>0Wd$WQ4 z(UCCXJBTCFSU#{YJ{47~-kl={Ktd3~`^g)}O$}zf1i}btct+yZ_?jwg>f}mGm9C|f z{g_3sAGxKpvMmBShomW6Y?qp_f?kT`#imq&xkF5RQ9`a)ly-Cym!?;G)S8|t_b z85c?0RSqKryk=OFli}u<`dypwiren8`PK$K164u{g(7$FybL;KUgy*PT>mDCnO;Y? z>yCTToyF1_BxdP$22*eOsCFl8ZA7+-%Dh%tKwWd~&aNBkY6-ur(*gT%=37*Jwb^gR z-9He0uGQ#3@W+LyhYOm1M_dQm6?KQOxPQHT>7o+j$M-oZpXH-og^j#23aNC(g=KZj zR?2*_VBFbJvSq++KS@m)K*|R%&|d9&_oVPTtPP|ggq%-%-PUwHRhqG+9B;^vX-VG6 zFR&(GO_Pw0-&#gQ_vGi;kt$I7mAw;xFAg7oah+L)MYb2#Yg_!OkwOc?C;035mQN%bi%r zR-jAlh zW>3Z+R=?t3>LCHPVjOOR^4LdCjJW!8FyHWRmY*zG$eIk$S2b8O64+F}hJN$l5&_@= z?tjPQq$@osg0@;=9ly%RM|;E6O@5Jx+SM)l(AtU^>PkOu#I|^L2Q?KzG1pKLfz2<0 zBd|SKN%&&Z`|j&NeVd7(KaPAwB)MclUg;SJzqT zGoI323RqPc86#OtktT|fwemli1{RFVO6_w3dcW${29U*rPT)y3)(sRUrEa9BW;cm=VS`<1| z%Nm}|+V9HNZrbye+2C*)QpPze{bbGSiRFF`cZ}|L6ZiLFE8A#@uJbUwE^|A3UFKWp zywiT~>tOP%J4nBLg~fVDIUoYf55-sKBFMv7( z`_l8cYEDpqzqjEPHnY>#yF+?o!@T%2^L~!Ev1L8V)2zMPo%y5Y;(%EmoM+Y(hp3`* zU#f!^Zc)!i+_xN^CfJy9^qP}5_b<1kcJq)ZcP)%>7hX1RnWCOUIu(b|cU$}1z02zH zwclDWrw_-e6tY37MSaEqO@L=#I!}dbQJiFG^~VmzS3eP|uZppsXztaX46(iOwvCf4 z@ac2A>;H_?*LYxQIqf&{zQqSUVrANu!SpWTUPeM&kR<1oi?7a~gLU^^C9ve)UJya@ zRgE`7OU(iSw5(GKmxc$GRbX=T5x>PM2q4uLDxlL-uVAy+9^;F)yK?0w+)vDJvR51` zH$aSxI8eWtlBqJSrJM%H`!So#v_6)*j$Z0_Q9S1t(v)-OgcbLT6Jig&@?N`*So-Kc~oEFtdgHxLbt&q)yXDO$hse_ zMMbIy7sAV|s^}iokczt38{Etxt~c?7uqW#b;O364(q_Ys)K4;LtmWXa1s~J}2K5-o zPl5kX{shA8aLwj%mwS>lnXEz4e?Pf2?WLWnLxbve;5xVa4vC)wD4N_c?AujIyyXA` zh_)cRr~|5hLN<_w9Uil_r{1ZJkw$LO?C@;Mu-vJ?M4#J3y_s)s@>Dd1jrcy}rO5X& zFtNHuIh%@qGkN>XS4DaHSb-Q zh7tlyncUr?W_Wk`I;~b+J`t}tjWGzpD^}J#YTj#U4TLIx&<>kU%q$&Wo60%NrL9bf zorkf}V|F)i2d6&Od0fuwaB?O2ro@%MzysgI$+vv2!W%E(Um;pVF_>H_H9m*L>>Kf} zY=8dd*)_#5Or4T_{BYA7zjEzKOsBD?u@eY;r+hk~hP4@lm`$AMP^8@Ge%2-Ko2s>| ztVDi-Lu>IePZMjOp0Fhg{Dc}6fU>L%-*M8kS&0x&P|U7*jDe52-9r=o#I7S^@9I}O z7O|)l=7i$No0*2#cx0Dhickxme9R8WHW?n^25nA7fY5`H>RFECQ@NhSP6`2aZJXIxf_YlAqeY0C2Dfe6#luF1Kexgq|Q7n=BKIAOd5?K6nTf63kgSKAlUwqiV&6{B-!v_ITSXsC48Gd`lvFcs_Sk~Q<_i(ue+?Xiz|5N zk+N2AEoU7;8os#(X=)tv{u@yrbDaYBxerDb2(Q~0p*n`>ZJg3~U(Nl<`9ZKA3YG{gV;XaE@O4`K{|A!#O@DGYbeOZ!nr_RO%?~ z$hx0cDVB>2qI>L*P0(hwPeLc2>bjpZxMF+x!6{w(qVMd=r+MBdW1IkK8|;;AYE4Z` zXt&2dXA$od30(kX@R7kp`jby^&9C+QJIA@Lsn5?o2$JX-oE7Ll8!GVfIurJ$gop|g zF)8T+;;uG?=$5zD7h?<9Ryh}g)+k)B%%oU$UL)XeN;Rkd1Eei?{<8Dk!u6haO`AYB zM|mB63E~;z<)wvGr8oZ*nWmR}(0)2~>vKx1K#!Ybzbz)Mb=ZuAf%MvQgl37bnpb+M zN#9<&!TMqL(&h3F1hAlo6NWngdD?#~ZDV`Y@(e81V>^2<=S#)R;c+kNv#rbd9q{_# zD34zDF|dnxBd~3X>2v(U2$$T|8Xo!) z??GlOMF-?-_xlTO#D37qzA19o+nTO>`U6P4mTktICmkR~JE}2@5VgT$tFaEFc;l;m z`2m&XFESXMp1}=q^l<{R1f$qEUgGlAiOLLQpxgWw8623`=i!-xm56r5BkDExM-|L` zKWT72S<6Q`4IaAi#$AM0A>mV?D<|Nz2@T1MXok5pd=5b-|0LX*b$gKTd4$_hBo4$I zeH>UW+gZe`^msDpu1{YWS0q(V>*Pipo&|PETL2EBKc#6aIS}UbOzVO4)Cnl?XEzC3R}v zeXjNgF7)^BkmwoaL8H#Yo@kGT3Xh(5C47#BdSFQYNT$r>3$|_TH&lG;seI4%j-S=k z)JYx3uI-rb?#%?)-MZ6YhfZLNA8cs1$_r^qCHnh;aXf%MG>eOi z+l*E4k^QOi?00p%Ujjpk)3*KkAQGZas+8@KQdjIj-XMbYKGIjh_Ew@+3;TM~f%@+s z!XNd?!S~S5KgUV(Ge)Ee7(YzqE5Tt}Rs4~|$zXk|UhG|k7`{&zy({4 z^F{;z&iZmF`CXlCmHf}QZ{xQwGG<$^Z3M=Pv%V=6@TMNS`zg9ETY^wo;qeLQrq{_` z;(EHS`0?Hn$Ry>NyXea`I;D8Z%H#@(BA2<2cg1*#!e%c;t{XTz2`}#P7V8tRJO7G2 zsaCO5pc?Mlft*JC z^WgRBUVBGP06M#vFI}c_i^MX^9|e)We}qKkm}Se}IQ@brEhmZv>~<#(Bx} zHUOgkWi1e*zozs>Y~P;Vu%OF+nazJWe}u*X5PTQRNnBXh+O7N`o|qEf}VkX z(7w)YxS5X$0np7s+%J*lNnA$SG4(^B^887%;`I+ti3Ib7rzSPf5TOgkE#JRwv07|) z@$iKn8HjYSy%4D})aaylc9rs_-f_wzHoZX?Hli4K>0*=ps{i8A@u&-so0P!slVwhL zMApowORk4^7NJD~SaIw|8EU1;??)<3H*2uI20D{Oy->tzUa}V=!x5jOpDjTj=gj&#P>+bNgC;I5qkLpbF<3Chsm)m%x z+^TjW^r*IO83^;i&R|FbJT`t)iql%N{YdiC(mpQZGfS#h_?^7>gTv9wpw*lK&BOaW zb_0L1ZvO7Mw43-HX_Xnbea3nU*&kcDmSI?ltrR{v7Ei6&fAiJ)!PnS%>#Gl+`n=(q zc9G0(lr?qjKPE|n`bDmf_vZ9mWnc1SIUkSkPcMWJ+p->dgAYZVF8XR*SHA+5Z^6OG zq83OBcL|Kzpi^tfi>+EaJ&pvmTqOprZc%Z0E>Lf(FkAT6?Bk0HqTLXAH>>JpMs_Dn zn`!S1oYTm57OVqx=8s=@80kR#81K72nF|(C{6d5pmE&iYk|?b)Yy;4T-?qGI9&Y`| z{NugV+cm$oZV4D)IJIZ<9HTo3@h1yOmTeTT0_3!~*i5&zG{R==NIc_rp&@{NHODNIUKbD$Iz0mcr` zg(LRErhjeNm@|au8kCunO4aLkoQB)IRlh0{vQQQFcXT{>Y{{vkm%jO~moHB#1Iuoa z;e*+NRc|ztt1w~@Ti~fUf>87tzq&{-&U${{8UciJJ#kqeruX$&P_WJ47+R~Zsmoe% z5j8q_q|6WRb)FE9=PG=QR12jZ!(br9!ilJ)bbh@pDfrrv<#BdP3hn$i#MJ+Xv;FS! z|M1b(0kf)5$It3mr`x@@mKF(r@UgccXhu@ba2FP47L68tH&sMOsXd-!$Zm<@6-!TI zB{2$}cGLxq0PV?6E6F4aJy@id`QTnCD00}JmXLX<)$Kp#NG4Gyz=LPtbj zzPtrAEAs^=z!}r9#nVl$G9|o2?iU1>oaWM_#r18IliYpmH8JSJZ(81HJErokDg5Ey z{EoUL3?49lE#_dBg!?`R+M}3G65mxFLgy9{M64G|@8rD?nwP318<2*htRD z@+uQnoYI8;r`I0q07G-3q{&PdQ@qff4u&n{xsT5m#2Yu!B5%PVQ;)IcSi*WFJfsCW zqh_b)myj$g8O140=Hb+04XSN`!Jz}PX$;oD71y3KFZf)A!LNN{ z3-n1PI+hNph3PQdofDTPc0>D~)=CR3M?p>Ce0E#POFURvrD85`DbyW4JqF6qq^AkhPH|n`^Qz7XUntS}VwPW9Imnl2mliAtnlI+c&=U z#M_p4Y_1z2wQLwS5bk;ZBvb5&2#}AAE;IlAserTU2Z^Z|0_aJF_b?`^0!|w}#zntKa+2 zTOuL>4xZ@gVWgkI-TkLVA-|oZMTc$v5fRZI>>1JD*8Ts!RI#Ifb$3BUM7jL?%m2KH z|N8dUZ{1*4asH>v5mO-g?QsSr+A=iY{7=KdrqS0}MP*1QLhRQ6bUeGbFIfxw0^`Bs z|7rLH`#yW{KOLy@#OyX7HYR3!sNjO$?_c1bJXgO*tn-_TpWp93l*$byqiGn$#3i1e zrTI9&xJYay`WT4Fc%SZ@U0&|cSI;ze@1OQco#=K;ovld$I(sW?g?)PM;Up9exuLD4 z!W1tZu5MpdI?{)4&jl0C72jWE1-&ispG{GJe-kgt4q7Z8F~>mZH{RxVFeyKl-`O$i z)UUJ+1ZPwZm+LS%^tlfLSzJ%*a54Rc5$h*?d&m8gi`IVBsBU1RbyaqI|UWX&}Xe$8WeU964veh+S0N=B0 z6j&d<7EI-LK}+*EY4ekiep9AtbhLiO9Y2un8^`Pf&Is#GXq=~&_oi1MqDJz7E3tf7Ncssx98feI=5jXM9L2`; z11*JuE;(g=A?ipr4r^zlyo`Soz7#dA@vu{XqgD zJM#6@J#~fk?%}0G4iK8v+rkecIiG_>!T>#&-#R@JEnQ||-QF;GCzb8*Z}PtDG$yNF zU1)2Y6hKE2vU?uU*3{HwdZ&@^)!C0@x*o3eKcJUg4c)Kphd+R#?-yJ8xTX zxND$lAChM_E<0g4S-tg(C?;HG*8F^*ab~z$+MILJ4KE=H9#7TOmVuXN+!~5tcUtwl zYBUx6-fVz-Y^BlFcu6>P-j&hTSi97v{C@GM)?)aZevOwaFvSHE0L$#&@$ua|`1qj& z0*2SG&vHaJQ=o!fXbGu1JHf#8X@n`u3|0<0wfv zejP>l(RGuD0kbA$DlYVrDV{Arys@4}Hu`mL3O$;B{IMSe*RC%fDO<8v&+Kdocvkgj zi}!v^-XPB<)IbL=XUktp!LhpF-V%>%z(|yad4#AAsIKN6^~oMpThjgeOLbjAXA`_- zJ*%S1IG)Nb;l4Em$k`t(a;HOwQ`ih3y7BgdYXb(i?fOuPS7-ZCg67U!bM@4~lyMSN zqlk8=-4KoBmKnA!zW0v??!TIg{(p_o4{j;AbE`SB$DRntTP%Davy+Y=&U|wP6Y-hj z`-hSKtGFS`24=za`FWyj$S2G z!{PJbhqjLP_Uo}zpPOOt_3~I%qZluTMBZxAIyjf2Y|nYIaxZKWEZGP)&=O zCOk(awyau@uaZDO6Ig=IN&e%{O7bU0#Jtl@yq3>gTHim{s=Y|z${`g0jH?s6Q?fU~ z>03*KSuDg)Cal)U;o*CI=-x~jA~wl?pW?(Tld0OTTy?j@V`Is` zd-X|fT=C`kkNl-X?h{hoddFXh1^}(p+PVU1%QL#EQq<%+=T&pBvrm(H&P6AWEjpop5*Zvs*J^K*T8qu=&PZ`R-#a zhjP9F=l1z9|LQ{#?j@adQ_ZqB0q+XEHw>iQU({iSsO2a{(3m3H&ef#%;fs@ecD4!p z8uIVpRwIwGRn1b4z;iNeLinDwKeAZ5Z*7)ylX0Yc3XBMN`Rdp|zYQ?>uho=tnl%cL(Jr+h_AXZzZ{e;$D4P}*D=ZXxkf zkI7EA_S?R_xICUzx&+?>x?8I>_g9-KT~nketZtwnzG2hgxSYkf#AMxS-(!;#)M8mSCq&<0;QR!;#Z`xn` zJQ9UFNn(UNe%4iB(I&!_2!)4_E?Anjv0Ng!61i%EFfF9BEzVErZcmII^+#V)aqUiE zO!F3Y?maeGn2HOb52lJ!ZAo%?D7HkPS8CMp3V#jrX2o}^AWt6BAjXhi=oF}X(4K$s>)!+N1bxd{upiJz2b>vDs3B8;@CN4?J zp>fxzAkPCb*|eveodoNF*_SR05Otzq&w<6F#>CEyabKMuktM>n#%dju0Qu7Hc7Ab z=`i?SzI=JnpThYktk!?bitI9Cocu90bX@>XNYluqxRb)^e48~*(DK5)o4ihYvqrvr zliuU-U@``X%!A^FKAvd8{gnp;$XgZFq3I?6!pO_mV;1JNQ)=8dQpviISC%mw`Af|P z)r^5x#TcyVB4*)=6mx`W!VHFAOc2;un2BC6w*EWgnLvB5N;>)D^hvMF)psUfN5 ztV9IJ4Tki}!1@eWK!wQGB0Zr|a{4~_XKv(D#N<-xX-4JhnL^LvmMGBI+T`cw!)U+n zhm_OJOyS!Iqi|(Pd+azX9x*7NokILxAqAZF*_gC{OJ2VnTm?BOWeOo3plk!I+WqHN zZ_DKFnsg`QnTJ!omg~V6v9slc=+>Hq-;Atq@(Ez!V94K9<(9LYxT3`9i(!*Kg+2h3!eTmZdJGATj!82eC$&xSP%jYD4DL_sPl=WC& z5vtFJVIKbYrTI!+=#C>3qSJ-B*R)!@J{X1H(2WsqFty=>%(`M|?dhHmkORnZ2E=>|F!T zj+pelSP5->(rQD7~hm?iAP<#!DU{7{S=l33R?$hB=NRaUr{Lb>iq27^) z&lPHHb?fk;c2wor-CH*LjhKZ3H=#h~VO$(LI9KkbcsnR~HHt?KXqChRhe;+_3)|VU z5#LaJRbuhG?piwJV7>q~dhqyjv^t`3|B! zjI)bya^Nngt)pWc6ANgqIYfTA>crOfc+FDBU>qf6&M05#H@ufTFo@W-3vToFj!>ww z2u?F<7BXYxXL`(Y5+r4+MuaU^2&H~-KI>c zzjnI;3!8GwIf?DpW;1=aovQguLhs8X|#&B0OWIkH^zD!1ET#?3z2Z1$qHjh zdxo~KzZ%;f%}u865NqE;zS+PEI>N@hSX!Uk9^zBbL7F7jLJS9Uv(B?lEn6_6EZ zqhHQ^YFn)-$CPq-$PBD4RVmytPI*Nsl=>d&wOYn=jZP9jX_R?HE0aA5SaiG&h8`nr zYl=yMApZEWDi_%d?mQD71pC@LDY_7OHTNN5B({KSEtey6<3J)}SKn=0xN}(6v<^Y}V zTtmlN+-K9vNNPLRHPPHy2hGzWP~lv&2L(5IRloC<&c?=@<|}861(Y(ci-T3&xq-j?j&fIs^mLM9FYt2 z_8ERRTbiqJ$92l_nZf*|;I_O<}{8Zes+ic8bb0UEtV#S`EdkcR@-y z2xS>e!wO?ZU@TDrUJosX*n&*h3Daw(&{HfG@)vy6+1ePuf~|@_@j*llxtIz_uH7d| z<*kz$+C|#%fbk_GtQQ&P5P@AKjIJ+J4269Y${wX9HxUHq}|3AZh5#}nwdeOhx}qXaYn_)hiwWC+4;-T9{=#}>u68* zC+{}#yH6onZh@t5GGAkv{M<*JRpCk688^q)4CsCPK-Tr9teCrI#$lQHK8ptT=V!MP z*it3ue$dfgPLHP^LcNoZ3zH@Rdh2C|?6pvXMqU*7@x_WikR!O1z1r#wiKf$)uG(&X zaxbZwAYRzup2@(w79Pma#BFp=-|ViK#ed*Af)~*1ayQmR5ZRD|;$h=Y*9*+)3lN zdUEi92}GeB@QGD4t1@x*6XH4#v?t%g&DN>SB)>FVaUjWe9>aA081wMigK>Ve&I zM_KU0mU|)Vf<~p75`GxdJjTOc5ef-us`Q%h&tMkujn}fX3v}WwxB+!bKR>4-&olseD=*3DKED*Um%dzXka&em7ILpnpylzpNc&3 z^otKztMNXg(8w2w)2C&?NOLpuG47c8B=d}{f9j|A~^z8*K%&9XCy3HgnC!FTMfw6kpVEa@3O3d(CQBKv(636I_pM1H z!#8b_bR>;cnCo~6g`w2RUT3=AfK*Pp9N3ryxcA#GT|9tB50-f2zDmjP{9>egLp`J^ z5)-hDXqPyz<~DUT;7UNLhx8d-dOb1FohXC21v7(@SG=d=Xaqn<0phK2cduREQ!_r! zf0%Oj>@n>h;f>x4^n21oJdN?^-jfL7Jh%Zuod70X)? z5l}H`b)A4Vt<(#O;x4+ox;7s!6qqG%o~idN4y75(XO6-zjn-$Qn&7n^QTS$h0X)#$ z$yV}j1UBjJx>*}lEw-A*LdY3MW?>DCs*|0^s`h7xH`o3*Uh0C--pLcJL1FboB$G}W z7oK2?L79G=oAbR|LH_#0&}nnK>LTikcNrgx2itB7OJo?|=!DM?6_nk`L>*J=mG044P%z>vYtXB4(0MJ);OnrT8QY(V@g3QkT?+K=ckLVF}Dj?HZ zk^XI@+d}1@f1bJz31{?3-3bd{B4q;#dX3tv;y&caI)RGIr0Kjqgl9Urwi={Gc&P|B zoC^at*HAK`7$qP~usugSw_<%6u(kvCpoRq^vge~=MpUC-_i8~ZQz9>^<{l}RBf2hI zM;(2#ioJjgFOhh1?Ny@H64Q z@xIRZx8r_NJ3t~_{xksYci{JLALqA;Z5`t!A&I^ z)Nm<<-S3BxvIQn2Xv7W(I<7}<_*|ZBKR@COeHhq&S0Nqo*a%*vH`KBHm)GW3 zt&?eY$tn{`n=k$==V6XfeE?hao1-bM+@_hw4Kqb8-a`q%Ym)6~Wrh1t!)D@zoL8k< z{LJnX#oVnoJ2wIXQ z(ax-(oH-A8YU~J6W-1?hAqvWi5Jf%0!h6`BCg_sx6#P5`UTh#5%c@7tqz(-};NOz@ zD|E5l(+_~Gx~@L8fqAv_Hr8yF<6tK!#p&nz3mD5Rx(A9W57#98ECBVRxwN#jS(M~q zSNDe6JotX1kQ>Lv$#w{>Z1e|@%M@Ol&V{z1HenPEAr}aSk9Ft1Sk~)L)@`fZQ5Y*f zI@y`-yCAZN*ju!c;UlT=a~)TIX1hr6yC4 zFtN{4aIJW33h6T>%I%BY^x8w;U`Qxj1zfN!Adz8(p!hGNEu}Jv?RWc|H^Nik#Xk;% zs-9j1p5|(HSUtaNm6q8|C-y1U4h4i;uN;3(QdV3LCckOo8iz}4o}7D8<_P<0qRs=I>;OJ( z41E3o2&ec^kT9?WXi98dU2I>e&Sm99<*C@|3VGbw9Y@^<#rORPGw=eH zsTmM*vLVD$eEH3eo9=P#QU|+UF(uhyp1@a2t!NLCciB9CV$F0`leLM3oyapA-KQ4uP5Nu z5A-#;1-kVc86)O+KatB)^~0$5upc~_74}t@#MRyX)s6JH6$*vg+s8fk35Oa3$#q|4 z?kOa`L}OIT^-MQ%W&NwY?p`47+wZePl0O@QCXI#YGGS`4IhH4A2xqhSaHRo@eK1?Q zax7-{h4U{r<#37D8j(w+Sa^qW;v*4a%*O!c^mwj0#FVqk_@emw)e+vV;sE+?mU#Vc ziQUSp(Dm~$Ow?i)pun=8~ajI4%%BJ5&38DGc$(m#%+rlv)`0872S{L24O zk?a!yu}Tad>}Hq5_QM`NFc@VYW4qfjSMPw=e6sWWaQ5?H-I4fDE;5DU3A@FGw8fQ3 zVP$bZyQ=KpfTVk#fxqp6A8&a2fE`5+RJmd?#dS|TnrK!jXEP+x>!*M&g3DKT=xeN< zyNh+PJ}*wK;02S2K5g0(z7Kjh_zDr;Z?`b^hU@yfASLaSLKYg2(bgS!{{J!d6;N?) zTed(55<;*95`qQ`!GgQHli*Hphu~I7un^qcgS$J06Wj}TcP*@tf4JSh`~B{J-+N=6 zF(@b~j_kem+-uD_7a6j`Sqp~n9SEk~H_YdLk@2pq+nGO+0a1ov;hA)9n9!KraklwA zZY@?+o*c4?s_MKul{nS!6A7yI=e;D+efCcjRj2-{2icSvomTHEv5UWgrwF?blTrW5 z)v{MF(2fN4K0tZLX|w;$U^+i>>?h@pBvuO{;s7L*tAklz$9>4^>3Z)+-BzAoxO6Y@ z@bENsd0QR#RjMtfU**ZAGOefZf$-ILRPh1nJ|BRHDG}iOy*Pewo`i1vL&@yofJ`=&>RB5^ zapnKlOy>#6>Vyi6Zxmp1%V|+RYN7f|seHzBjFFMT>qBsif7w5Y{jd?>E0JGu{`Q%wihhwBJft zDMR`#pT>dOQfo|2CFfxVWK-b;;3P!^l~QK3sIpF7L;kMn4zPhV;*tBiiR*y=iT zx6pj#KfBH|#8o(KxA3?;sA7TpQ0aLTY&raubWHvsz5`$+PqmgeIL_|OE3UVc92bAL}PeB5b%IXw!JGvC1*=OAgra03k~K%kyeY6Gg< ze{^5>VjOf~^}QI51b=2S?4yRi;iZIW_?`omq0WoQXOR;9QeIQ;h-OzeG zG}RCR-2KC(dh+77ird)C5aMD<-W@S%pbZ7 zI^C`E0MLemalFWQ%!;Elowiu5FZdHoreR;5O8!1_F??!z%sPml?lDD$$&ejTGZV6v zad{fbPPhIyUHp)35iRzW&N6$YJ@mTj)*Nzi38k@$mq->I_?TYag3*$>*}`lz@?O7= zuNMt4FrP0j1*MRAVUasY({E-mtobO_6wja0EHr`8#5c~ru^s|WrpsF#>SQ85OFU%F#9p-B+;%9T- z)n#f9C9w|C(^Xh5A50%j-QO-JCutW8ZsnP2d|z$zd6utMkwtfHo=QxkLaS7!LufLb z;uDu&tI_`BdC8E3E?IJ(v6zLcxnsqT5lBW~G?n;lrBO-U6F=#H?sxx_)KcM%kn|=KYUyJZ5s)GrvgCR7O@f5c6M}3J`Y=BV%EnF#BYn_12timnu49>x$f%yM_Y7S zLQG#D2PQoLWwWjHo934~JN8bN8lijX)hzoDja|P&zbc#iqo|mloINbg=a1#nX&Lki z@}R!;yaW-m+-6zqWXeQvWx>X9n#o5~d38yCJJ@d5@T zkyQ%S3tj_o0qkDd(`-*%dZSP+bd|<|+t%YLvx~0gj`>sCRxU`>=CU|I`v|RQjVnGQ zb+US(X&%r67ssaX>VA7or#r=Cpgn!l833qv&tJ@QieDXyOob;WF+AIm({{fCI(t;s z+Onq15ECp6Qy3+A5!H>xG(-3RrPn}X8GWv#oM4Lvbk>~X}pxzi2z!qvYu6bkdH84IGV?^ zyZ2Wr_{uw`Dox36HSu+{k<#{Pp`nw9RsM*_7<><1??!9j6HTCcXkxB(P2+l_btH1k zSR6+se**2~>H+=D?9FH!z)RqEDn~rA&IWRzCn>G?{!+?a39Z^PpQnx^8G4|P@bBz5 zx-+N;P8ZuMfM%)q9-!vV)S25H`-rNQCQ4={ZZnel3Pg)e=~oMtZq#f8@_|RfpAEP0 zH{9~xstl%@zSkWP4nI2QW!n4`=))BFfD@Nw0Dv>|eFHGo*u5#3_pvzoYX>|Ngcv(U z4qZFl84yOP8`M(-4Cw?D2gq~`gQt`9vM<9eAD*&~4SSA^4iF8GhiodKno^P{TEH7z z_KL@1P%q}OoQTKd!9U=zNpI;Ii_q$DYOlSu%q%r?rQho1uxn@rB@H!$C04<3PprV{s*>a&ov+-ct!EBYRq^UNJT9xK| zc58yIQG6B%NwINf;~zKN%w*$b<6#2V*_hF7(ao0R8PD4(YLmvo}lS>x8$^?OeJizv0 z7ZJM*L#LrAx&)t%LTVlrM||BOr04jFpr6B!tz#p706~J{+(q+eXPpu6X6B=X4O85+JBkUe$+-!Q+(wi6 za9QKVGD8{*w(qbx3o8SM$Ko)n6$n??H_HsU6dW4LW6OYfOlZn5A zEizbYmL0FI!WX@C+VOEdxIUgY%nU!}{hPr3@njE?j$t%ZH+!ylrO?>S zT!5o8x4pak2l#Rukl~?X-PQ2px^Y*+Dz;3#(&BhuX)keoI-j<0CSCxoX~8PU<~=oZ z3FXsOcW%TyQ>lExk*PSh^+=k+$Xkbe1IK)R4pY)dAw)S z`H7=D`o5)Zsz}uU*aVY`rEq-z00c>Ved6W+kwgA(2??Y03ZU(rsW$&o-BPf__sIT6 zr|7xKa4fM!{7}zach%CQnVpV2DQ3#~<7=w&r;UmayX;DKIG<2Fy|#>k;O8I(BvB(%i zmQ1$q*(`qojgZ$;otN4(g#$l(-nm4lavKjO#P%zKQlj3$?6v>MBYd*J zE(Iujm-Bnu`+$2hLb~z=b~r2~r7!o2S|5dM74u0MsuW_=-@b=cGFr zZ>+`L`Ql)90-_6s{vPV<>#IFBV*{Y4&)v;=GNYrVm1Goo`*@D@mx8YpGO1is#mYS9 zlOJExX}k``r4yQ~HnWN!tqJLDto8NvEeW^g8RCK7qT$htWQ&E$v^6Q_$>owmfv&MK zpLx34JpMWNRiGN@iK!fGw@}uOneuJqig4ji-4WGA4>$Bc8HCT@4)_M$n7;i^Bsk2WE;v#0}{r=>HLpfTDqf~ zoG<@oPGyw{lzw=lGq7O)hBho&m0^7^$G&Xy*5+Ikhf?m>!A-DUZ-BD=Pz0R~=`{?t z3b6L%K}QoQ5qz5TlGt_~fKlK|+$VG<)t9IP4m+md=w(?;}$WcvHzUve85`^eS!HQ1|c1$Dqp-NC9P@Rzg2+N%>^Y>#Tc5sw93`&ZAfg8VxEUnCxXV{mB zaqEhzEqZ3w6HE_69#Qk9ZCI|5trIvs11gKuTu@?P8_$O~_mp&^?NM*6dL7BEn-NX# zMA;{f2hznd9yj<$GkYvoe)q02Zf!v) z_G3SXw^jKoO;F+;2iB=Ef8z!PYLrwE?5QTF!;Z~?_(&R6`j^CfcT;jdRsYsOVM)U$ zWcEBmq7m`M zuQ<$;4rRE}iM+}?dpUbOm5)i5ovLQX1p8w z`-ioUH9B6yieDR-p(KhZfv^nN52PhX0PZMXO83MifIOWX!Hn;lbIL@x_$fv}%6OgO zX>T~9iZWfQ)sf`ST<$!;WnaH!vb=WC*E!Q_?LS3$*yOGcu=^;la{xk3&Vs|-OLd^S zp;o5F;Tt6EMkel)m9GPSj)3I{#vN)4;C~qSF?kbHrh->6Zrl~wGGAfZFR{?#*ctK| zYp712%t`cyHJDRe=U}lB9?0Q#p_c{c18xNU%8zfqul0yuEgS3iRaY3N1ucGEiDo0r z@#53s01+gs(Df{NLFTsJj|Ie8dOoh_{!y%ELCC2-hCpd5%#RFlkOydBXRkkn{cj@d zT)-r!O35_AyWb&aV<7Hw*AhMUbe(O%@;~+8cv(}*3dbS6~*h2y+Ag8|DU8CCNvT65)u8J~trcX7ZdIzKW zo-ne@{mpkFf_mPMINLidfY40uruekhfP!XPfQy=+R}o<`$+Xbm4XrvnV7S}U&Z}8r z*l+5>UvF{i9O)`^cHmEgHZCR)<+iN*S2WglEyg#fv7~?a1HUP0{-Za3B(-Cw zHFc;JFu4*9YktnysB40HH!RwsvP|N-=dh)=mGXSFURc#qtpc;N$qSWV{k#^6alf*0 zpdV&|x_Mi|tv@jL<}G(dvbuAPadz!i0@4yo!~uoi*7y6S)fpdcW3T;T(2C;&?XBmc z(vQ5V(UbFGiM6zGg5U$y)Gb|-TEv*%yF1=3*}FHJg`kihk8$E*gj7Ull9(LQX$aUM z2cbjQ*Corhny6YPOr4Zcdtq$a$O$(u_aB~z5if+gFY%fhuT3WBZ;KW3DpN#LWe}0H3j^vPJs=j@o zssfTkIq7&U@fagLpESNsvz6;X2I-r6s?>U z{z0?j`X^i&f-u0`XEmK4&!E!qq$lkC2%DJU`#g&LI#(!mO5}JQK>rc6>kFIoz7Xx} zxwjaTS%^919Y;paq3QQ{OyFHI+|l9l!Uz1T{4{?}0`3S!hfMf>5X-3vJASXhJHiGu zyuje#qCV_a$Mqyb?^}scD2oTtf-km*P*xZl+Gz1DBNnhKvSsB?c}-2-QlGbrt1-Y|tz(-`!|lQxll3b-`c{Ar zy&Ud$n7Qc3$cO7BK9i%e$5T_RwT7uC%z+odNV`>AZLL zYxLZ0t_gyD&AN^(D>MAUSAN(8IxIw8*$cZ+w_B-5CpKr3Gv+Q$%p>}WbhB`}0z2XI z>T#0s?W!P;cq?CN01#j~9nhlmwi}J?$6B}b4rb~#5PH3m_3DqprJB69YZ23{$fmhP zBLT@#^i<*OpbEh5MH2klKV!ZvPvmc$ZZ1tmQ6o>Z^uFWB&oBVDNNs43XpL2%GS&J1 zJ{}QSI?3BUO^AYg=MDoPVPlNRF9Rl7SjWL&0zb!oCQ1yoaZXnk` z!EycoG6L)@BA}KyWKIVw#fRm?s4ZpnG5$``GRFm-Zpp%u(Ro6aeN6OPjo2?FXN|3Z z)`a^X66cTBz7HKSXY>}k52;qi+~^BhQ?r5q?>WUaP5`IAqj@!)&XuJtI}z0vKb-d2 zsDC8=&IatrQ#?(BHylvI-46qZ*k+Z1P&7nWqqHZk{P(`mqp{plS4zOjB=T$$wUa66W0I#Um+;BwFm4$tmR^SM?|PKnpvdH1^ZMhzZYR5sW9#&tQ7~WN@ev`0>%A8$ zi-s(-+P%@DI&5qEN}?aJm#HZwRhvQ$@5}?%`prsn)2_rouyWW&DY=vxwlXZc5ND~q z|3pEKO7$3L^8-=LWn}z$)-Yfk(!D}&bt)lsWsbT9SgZ+6UxrG-?t0q++^UMI(A)8* zZ?D;Pil)oV=Yh1$hWO*O{p5J-jkGOyR<7V4Lia!t{f!ab2H~pVON;U*_xVO+-t{Ld z!QTAYd=iV>fO&G^Y4fC(#qBH6kB{GY)kh$Bc;uru3Y9t8b1Wnp*?@iZ+l5ZVXeiH6 z8^j%K5Aehr7XP#Ik<|Pxw8UrM`Byx9;29o`0m8*?kNS@uE|r~;lJuEf#|}%UD-*(a zQXPpVKqyh{&a2(bALkf+pE)9$TuPOCyF3UT#T#c@QQhFt#^Z9}Cf&?&1Z@B*u^B1p zz@u}@R%eK4^55o%{)?`oliv1ZN!=t1o+Z4xogx5w-=NqlCe`p-B$*qRo8rgTeBXy}m;;QP?=uS;m zsd~;h(cf5Fime9BwU0hiz?2QxNn*3NHFo@&>SQBFycph9=~Qma^CY%gM{=s-#rPhR*(ImUeopma^1>N?F@Zs@c#UBT%gcd*KPG3_(Da_ zfr#OAQph!EWY-XfNpg0@*(Y&T?ZLdrTvt?77|zspaQ=`weurDtpVdxF`pRP8TyW4W zSi%0xJ8X`cZF!-s+{R4&4w?PiWBc0n)ogZI$4T!2_igmDc%#SdN8Zsy(OQRN>vGiN z{4oQC50{dub9CH8MiXn;MEv#g#OkAkUe=op*vnJNal^`H9!ss8Y!qK}Yc@rST^>At zDgQ=)T&q;6Olan7Z!Mz`3B&hIQGa$jyNPLh+$6SSeqQeP*Mfq1^bvaF5fP6c+Qas1=3jwDbzUjr1TagNp0e{YyAs_jn)*}b8ceK#9;XvW4! zE9g;~+aKZoBBnqnYtY=rC^)6Mh*e7V8gc+obh4so;de9?JjK07n=((G$vsOP1~ zX3S^F7rtPF^D-Os3wB#MJl*{?PNnZEqpr1T zUw5%UD|j{Y<@V_h>aO7|B3juK@9Ny>B&(xE4K#i%EUPl4M-xxqP@@@x)T-)C{qYN1 z){1Hsc9&n?c<9S&U|IKuBnvFsj}e5uVNWAn8kd_2INV`F6C5eWFmB-A3+D`0-z96R z%ior+F4|P5R8`|u9hV?pSZWyy3D37m?<@!Gb^SnZmYW=>NG3h++M++5 zRMEDUla-5AR+G9vto3^x=S+wzF|Og<4zBV}7N6@$r7{Le#eX)Ff*c$Qjm?tZcme96 zl3{m%O>e|q?RM*xI?8T{Or*?iG>Rk(CxahvC} z(}iF-=_F5&V-<*Li1-jUM%LS6*FDE$wjrXiWTJ(poIgC@=+<_Hn-pt$4)fX2HkPUw zAgTLtl_Rr`*%y^*7iqzt{ zDT~&F@gn1!8MW@%XtT@RXGdc0^th3?!huf3d7_E|U)EoNBtuAoDR*0kiqy*w8CHhB zlp}UKOInU1PHo@&v3{AZxY<$R4fuk1Y2xk=B0biqOId3mAE#n&Vt5G3Ia1!MQ!@VD zlCWEuXrNwxqkImT?CzaSw?VyRZ!+5@P#oJ^2&)NJPV2<8{2fm05&O8TPUcp?aVp7U zGYD5WU0XDM0M!p&Ak;d(qqqlQq#F_stIv#_wC&Qw<{nyKMtYoXe5*1t!x}zVuamej z-z$fM!9hCGLW@jiL%vi4v4Xw|ncrgIPmq5hk#p%pb-OmYaaPU6Z3zTIN3*prKVb*? zT-y0#^2lI2(l?rHvDMBe_vKwL38pTiJ3-AZZvfJeS`K1oa9&8pV8aWfKb_+~rUGU6 z&iMDdw%F2H{6&qW+MkDP`=;!B%}zH~l>jlpawPRTxBE}*Fs@lpW6O(jG%q2J%hJMzUZx3khKct?rTQusuSkjJe72Ly(>{ol{E_WvBh~*!uJ?rV45T zPR8!Mh2%BOuE?k{{dKvqtu0a{D=558sah}8mpW7?{&j)rP4V|0$!7$?l559U4^qa< z6LD~{j9i#Lt(=*y`lfZiy6UH4;&LJRtOKDrDlHYm8-7F3VkBO3u zNN_EDW`H>a_LKc!`Ub_14q<2g+rDQOD~X;A9B3f{B}|aS63bSg70ePzqY1+u5_1th3QI+iC(OG#Uj@`-`(F;TU^t~el` zp9i+@Q2da9|B<;$pBYAi=>WVGGzp{;Bw@jf;0yc5^$*hxWrn-l-UJa7IMIK3e5SW;{Z_chI^9E9@3J zQVXtaTs$0F{Cdmz>2R?+ij6K+nwa17vz+xAhx6)Jk3-Ey*WKd2YYtQ#MJWIL1h5Fo z^sJKuw@CqICek;-`&EBoXvXWnzwY$E8UceiYOFTchiX+n4?c)wT%$%S#PRs&pPwYM z7^pcnx*hr5pgA&5;n(JJ8r&kQ!=;odQ4~VT>0}8P{H=93#naW}Uh}FDn2)wnO<7Eb zZiVgU(G1-kfujSG!bM)9c(-AEXv1$?dsJ14`nMvB-%ErK4nQxxS5}&i;?!qTmSd4f z)g{@VsB)tR9>yY=8t^{wFJm~kTp7<*dJu`~(=Ez%=2exm(X&1@vGaoNBE+063QGIF zl#_2Vs7C@MQ~4`vtsII8@$qb!GuW*7Bf5&kOaQJNfe<@=8e13G<3TBvM!STMQ#3?` zcnO@1Kg2bDxR1BHQz{$aVC5E`OGktE5q;x~HF*?vWn;GO6|CHBaXY9~DrO)R$wsEl zXCD$~+X5Brr&lf^+SErkre43uT lWjTXge4&uuj0-KSwB>G<#%=^izE-*|5j@s9 zQzkwjU_@4u#XZhFL;M#@^rxRP-*Jln`faXO@}1i#%5|=7xLSW$eS zg#evlv%a^?k4>FZ6^Na`x|J>UWKav1SR4>l+}&Lyox}51x%(pSpnZ?|IWq#AwM|@eq8pG{5qa$foHava!R`a z%3(1$UTzrXbEx}Tc%i=2)oMkn*CLtDcRLPrWr#(psNMmf^<>k{@kJQuMoO*D_S2iB zx-W^2xI;Q1Tgfdco8{(QRfe?pa=|^;)d}BU_SVs-=S(x`z74aQ?NcvDQ1F$rF{1)}PIQG48j-%b>^hb#9$GJ`{GSOU= zS7A~SwBI>jyxwN2EHac-%*{CkJ(yJS>(d>RQ!7q5vQd{fOS>I*nEAqSICId~rn^ z?!S(OLLcDcf0q`wzyi{E&%RFKoDVVT_vTcIUyeN+X(Bg{r+YsAQaxWe9LwQo#R#xW&Ws>N7RliaU9_u2OqAeK(&C>zyz2%bPFL$FGHFjT1S>F&4ruI!w zQwdCp>*{t^WM@ZXzXH45Cfwb%exe5&2_&(4E*I7_E^0<<4kN^n>0;gLKB69oUw<5W z3t5anc6u6$s2mdFlV!DVZ~yf|0p}ATD1ew9yT_)8U-w}|GJ=CMXDG$Dk)=2TG5hPl z?}G8j=xhV;7r~dsxG&-#Q=LjQ#ooscf6deHkF4SmhE} z65YPxH6rf722`)vh~0oR0_Bxu;cWHiIbT+n=)BBp12G3QN#YVG z(Lx|S=e2t639U13H{9a~QCJA&8AXq&qTlEtBMP*I9SoIp!4dpB7nCkdVqX3BHI?Uw zU+El)OtTA>+bF;A$tdYkSd^4ECi23`g;rmiF-Hu-ZiD4GCb)zI52gXUvzn$Hk3(wjFvX?@0DaS$Oky*pb2EVslKMnrqaZ7PDc_TC6$#-u>>nI9H`R z+LY;%PgZY8$#_h$M*OX;_V89dq0-GWlj2-8UB78yJQ?ukg&h=KC-!%?sBwPM%#I}G zKkycCp?t}Hqxf{i5+ciwLfmT-v9#`9R z2wl=JWoqCTM95i--2=JB6@8uyeb7s*nW4E><@a?tTB1^4>cnEz&YfS??6Ad%(av;f zG})M)#;nFr*kB2}trMP7@mQ$If6|4?7iyJJfVLW&-EMxcl_f}XPt!A!FV`8NqZyxn zjmN4|N&>b}WB2p9AT0N#7D-^oq`zt}UASDz9aOLjh)}+k;1Z(W(53VJ+N(Jf1W_`O zQQ_*iFEq$aAUPZ9b`kGlC%f~t+PsG*RsZw=dBh~{G|=W>0$XkE<0N{Vpn`*;fi3*A z&Q~aYst8WU%iwKKG*Vj^5ZtIWCWZ}c(rhD*@_l6S>=8lO= z@OH&iH;_P`^r9hr$4j8-uHhveGbXV6{ny{f+&VOU!eFEZYAM}uMM!)SkWzqdx{{jwSHL=aa zuDq^}$`n3uI}pwa86I;K=j7xhL%Ydo4B*P__JqFw1kfRrVz?a5KdBVIr2rh~(g|R| z$A3#`3&bG)19+{ztizOj_8+L-Cqhg_?-iS(}tdj2{QbNJaRY!aBGF|;nh*9+@R(f75dL(KYsZk z6;Ce<&`uPmn(a1UXt#PKaycK#fFB>Yl}j{Y4G%<~HaRzlNo-b9%qGLt4Oy+Ntqv<~ z@N>T{ zl>a0&rz(mHB>a)ke}S@Ag$^k((Gzq{)Y z*Bq023g@4N1^mDu*kw+g&GLBum+U{?-J&_K+anRNh~9iu@a~UikOvszxxo46Utl88 z@LVG!kS4VOCuLcfZu)35S1qXyhZ&kxX*BRPQ^?=45jO}JVM3HXH|N_$&P|p8*$I=0 zCP)Lgmulp;r)#49$$-@e%NZ}d8Rq{UiWi7#&(yqrKH*tRlyF2yv3fS;Pj06JiGet} z?90P>nwLEqzJMYsCyq`7XdS31S&AjQGK#&_3KF+j2ItjR82Dd*3Km#ee|RRfft~(A zK5;nKEnEHy4>vb`!+sO%*sDo2(0;2)qCtYa}WBIXr z88Qa7b#wj3+b)~cwm7`}fDerS?>rLtNCIqkm>TGDGM>T)2__NhACd91&4Jia6MzO6 z!(=2C{kH3P<|h_e%+~R;*Q0DImtz)inut;gO2t~N!b?n-x$^NzNEdrj6)tvW{e!O6 zmYZGyF0(-doJ>|~4G~Gzy8?j3n60Zj6eoGE)?7;T#kGa-&iNEJ`p#nPi!J-S;P|p} z@62pT%1ZLCpG2YfkP=a4FPglt(1ABqZeZn)!bA8|%BNp1rPI`OT2C}_8o|?6LI5-C z_d6W%g6gy5$;9)?ObeU6sT#633}4%9bhBbSb1!Wu3jsbwAuOtN5`(`iJh)^d&Ku6bLyq!;uCT5mSp{>ylQZ@fU=-g~|JXVm121-yte2F@k z%B@N_iCsigsb{g&>^cb$-t^2|&Ye03kBsE)lIKm~AMh%E@!Y zr<28H=Iw18NLq*ow=E2RAv}um`_0z8!di`|$98F8cdO|Uhye|Jv0QTYih#@Q!{JiZ zvO?g?-HCC`2S{kUaXE#@$W`}9c%)uev&-}pP5Z)I^z%+(ObUK*HtbPz# zmq%>M?EnM`u_tvF)1@(dAh%c%mgk#ux6u9M#;${g_Ivk;_h^I+x=n)Xl(MSP-4j$H zN=YsSiaEbmpMmA%=I3Fscew3l(8Ab!&yu~3`!iNLwdR!DR|Il^FgfQf;~kYR5_DJY z-MH5qW3f=&*duq|-(c>m4_6DEW(z-kUb9NQzO`A$#pgE)Xp}>&W*Q^k3?oRQW!%W1E<;_7qnT0NB<=*23_2`$kf$TQ*i7g1->&!FN4gtL<*ubMRb=MBchTja8B4 zOk_Mls4b?p>>5obid@qB6A%`of#H!oEV{~9<*+*rck08b23>ahq+A^FEz^fbRU@MN z-G@8E9?%l63ENQ_M!xI3T(#j1zWIX*ueriDMr^g~e3()i9I3t(`a)PeZH0O$}N{@d#?C4isHqS!|>nib0akjUImDcjF zIxpAkvrjc00VJImuAE%uyBe?ec$kyhkU6!{Qbp$}_N|eW{OUKNqYHs}#+J7>k`XmX zq5EM5m?)@-NUsPaD5T?9cAu%W_WF|j5$b{$78^Uh8(Lf&xxEOZ+jcmB#0P}ni%0jq zths)8$7mfj+v*Yuz8PvcHmb;a(B^dOV4171U_jrQh_&1NBUX1a-!PJ~IvIv1*!Mw7 zTxWxnG=CZIwmY)D{iW*@jQ@wj&k(`M@Udl%p0vR=|NYZjYGk(ew~o=#?EuEi11Nr9 zOEhX;b}U=YR=%qoh#-B|SL1^|rk+7gbA7%sD2Kj3sSy2VPKY3J63a^OP+oJF!AKcc{bJVG?NkzIv~}DVr#`7>_@S;%%%0=DJk9!2Z8wC}S#eOTR+YBV%=f2jmJb)H#_WMA zl?x=7m_6AWoZ&ir*X=OUDea|kI8dXhZacYK#zw0*R9tN-8hn-x)|fcQTMD@NQjd;L zJS!IVx(^d?f!mHhtE#8k_9lDI=99bZz^D2FAy57^fg+WXm;#g2Lz*OSK(+)EyQZnZ zTs;P+^i(<}$}983jl^SCy|*_><<)E7mlaVXm`pBom7-=SGRKMN6Zpf&^QY460eAkK zZ@&qH%Q_=Qr#D~~aMO2nZ`YTaF!@}Qk&i3l^-$&F%4;9E#|g=9eWTY z`O;b2f0Pk0eapt?E0fmQ34lk=yOz6b%E4S{{oE%4gIkNoDy!uRSJ1e zUeB`_syc1-u^ja#`9UV>CG4*Blhy3r*dP_*4@;ab*9pa+^kus%$&Cx^4i}2TjiV=B zKjNXax}#rAXN`(48KNVj9yPSSnBA;5&AekNcgCAlUL9_MA@OzWVxa}eJePFdnL+Jb z9$VL~L6yy6ca$4+C(>iCdtY~yQECZMq+CVY7_j<8J?xKwfI}&h0kGx2j%A4g?D(uh z%?f={C>5-oP=nngj?^98zK_&XL zQ4Y5UuAK>0jxg(AINWiwYzced57qsHkmqtKoOv3q2{5aMuiaBZF=V~+h4%XBB=V&M zNH3O|Rq&bG7$d08&O~fliq37^pYs5S*pNCSp;{maL!Godh zu%uRpA1b(s@GO-IEyC@RumfUe9O)6*yXBmO;#Z?rO02mTT-$nVvvm513)t!H-bRpk z@<6Ry95(kmCWxV+Oa{ECz0E%!nW^4{oPWBKm&m`F+F@jw_gUnflBTPj1^?T!0wco! zOo1lv?gUL-oK0DrU0i%^6%u7uMoVqHnsAD3Q}w`fbK}c+$wgeJ@k0c4od%~UN#Lp{ zRC98#jQ-+uUA%D^wnn9P&Okf;d!w&h- zgC9G!-7&u2jCY&u4=*YO_}6XyXn5F9;|^pZq`Q}v*}vRf3c8Qu&lMJzFE@iZm^WiT z{?WKjtN1$bY`$Dlz;gnylt{;CF+B+dJ7I@ov*V5rv`G<2>Ih(N&AmJ4SU( zD}pXOAZhzB-4TA)&-{u`RZuvu(AzkPGsr$td8` zKAYz=5tcCBoRyj9yr}+hpZE*TWoRx`X+OMhWBpti?%|;YgU}>&!R7n<=(4t3mDh?% zzt*B6Br6n$CYO{3%kutQRTG~@Uj{C;^);A#tGO1qw!yU1^)$Uu zo~q;M#nc}Dno;izLI8>zRM3lGWTKuGtxr5GS?ZbF4caEutb+Iq6*13dxviQ_yq1<6 zjn}P;dIw@jq)n`aO_|XSYC6@&*43!{P9vMhscJS-XE`pAPTS@wv1egkAs+do+N#%@ zD7}*Ud_?%x2}42m-lyRqIsAFsgvn6edfv#_$^0*pgWvEBtzfOb;sySAlCJGRY&w#j zx2**ux|b!Pu|2!r32IR+7aNsZ$j-M%Y%0Im=~N?Aq?#fkvlVaq%VOVFr7O@CFV_!I z!k>h@Sm$6signcQgZS4_qfJK3$8hp(+nYBAjmxkENQyB>WdsJ;VMVGBxk;O#NFZgh zH(ys~z74%4J<%oaaS!Cc3xgwPdTm*8u&y7F58th_M!Omr&Ple4gCU(c3L8-EkF>|I97QxVRji)(XX0b zVo;9aC7QvRVS<+v+8Fe_?-ny$mOl7c=LW|Mj6I(7Jtvx0VN6EIV%Pa#NxUzL0`h9s zQU@W{`O0gKW_6DFhjeV&gP&1gA52HzEM zz~03LH4FjWF)gHU8ALc|QgqSd-yC;&nPutBGB2Lu9<+IbJr}!@l7gNoU8au^gA63uDnC z&mWoA(5;4o(#gLIdC~B>U+!0_=H_?vf4_BjK;wl*w%>pMVj1*8h%DWO9By0YCcy{ z&0#g)DUL&%HvA7+_0;=JC6YcCjFM+0RbQb zD;!$Ea!g2Yj(OHxz7c((YCzw*X<&5u8U6Qfa%RYl>FXA(TU2YKgh1u*r0gS{yw90W z;#4Lc(;IYn8@9U$+*K#83eR3w^A1W~=hF0RX`dR+bmp5IV}HL}8f>aFJ#%j;;YeyT z?$CtQ^)lZsCQyM$D|PL}DW?cNxwz19pnn}ev#-0ahN1>HsyPp=0>J*H}}`-V25@??o0AIP8uDc z%GA0dZd-5Mj>co4p*C;p7aO5Ds8xdZN?Q*5A3%o^SK!I}pHd+gLG-SgBoGV~cn zBtMFQ64R*a)G3ui=zX|(>c?(rcdIgqxtD2%4%O$lS7ZA#n!~aS0$*7RA&dr<32z7_zc3rPmFlZPl0*_KoYRZ=_wm?+m_Cw*t~*}bG8ij*hsXt`CR z*%>w3GyK6f%NJ5}X+mBT4;-GFb7)@JrP@q37-~Y)_1p^Yh1nPyZW}c-agw$#hIDf{ zH~detjs_#z%BJiJRHT205G)Hoa7;K26(d-m1Z$&H-x@YLkX0JCCLPt$5!J3yIaEX{ ziw$+Zkf&jzFlX)$!20+-4^(c+ZF6Hs_aq5uRx7%KCwW6!$MFZ>^metu(s?TsRkK8j z)z*t^Ff&fd3^)XZeLuh+U{gOQT+4)^u#Q#mUzGBHCR+jTWV@nuYW*6_C*J;oyn2q} z$)7;}EmWCQcd`V33;Qisp*-2_Vzntcrb*?fH^%RK^~oYhlRniVj|Uu_Y$u??jrBvy zRSxG=rNw77h=&dT{u9NQyv_a9UFo%;_uR{)v3@u~fX;nB$45RCvxxiWcI~L1gc6F$ zH`tEoK9!Zb(nC8(AXSBgclWBq3%9>IESYf%uop4>q$r3#)>{teZI5Q7>Si-L(B1}8 zYa9{%=#{L{tu2v{r-vF4ET{Y?zh(*%z77yh9#5kyhu-#?lJ7IwT!xop=CmBJ#Z;|KP4X?~C8 zeYZ)%*8Sp%E)^w6s1|#LNBbQWDGCj~!TmI3?{(txUhFeH$`Bfi{+CYnCMtO;!n!RHacBWoTS*UzI&b`(>HzsTySi%9=k?l0 zJo(c5^Z`0bOFG8={NTOkdun1-6c|%|fc|^TA7>mL_X!ThkHi8_G<#buU(W}hdHUBBNeG*wkqQa*OeX`;(mEraeI=j+^Ow0{r~nO3$FQKd}DW zi^O9beBJE4Fs-w(e)$iU>F>blzdR1+L%&$N#QzI8{df7~|L_cD0)ffU=>PZs)X)Sz z`t-Ilv_$0p^~$U<4h|1=pZ!0%QU3S$g*%8fhlkOXdrOpe^Y?`Ie|nsQS95uGHhEEt z`A_QPKX>+jd;Uo;#sPou+Ft(N|MbHDyoL7|A_Ux@PgwrTxBq|rRV;<_0Y#VT8gCvR z>Pa))K`e*G}iDAt$qej=oJn3&!sH?oZ zCTBD*0~z`D=db)u^Pl&_#(u!JD;Q+LKgd&p;Z7hZ7~@u_0SLon7x36lE!@Lo9GPi& z%;;i(fzFFtb0GqzilC%xUWK0)cObwD_hrtI(@Ohckbg!qJmE;x%k0p({N(hI%V&|U zd^eWz0QX7@By@m#+dyYD3LV=`SM7+p61NIGLp+9(iSE!->Hlr124qV-xXAW@YWWf! zwW`&zl$Kos1}}}Q>bwC<O^zd=~Zuj26_lTJB_TuWq-;KJR5ZB*p59^i5 zTF&2leF6Z*F$S4&R%()deY_pmkFhzL$>t+jwAn%lYf)Ia-vFY#TEeVs zsE=IfNZL5i z*6H)Av#k!%mSKLm$wq@Gt{i>qCB6uGqbHq7VxBweok_X+O0LGA)FnmFmqhy#4Nq@cGB@&el(nWG&(vi$wWbo19;YZ1by( zp9B|+I4pHa49!h{I%PZcBkmxY%L#hK9-!)%Z8JDB^r)QhW=Z-ppXHg?T`48(btg`- zy?`o%?z`8d-BtDE;Ezc7*oSG8()AlZ!ct?yOR;SWq3A~f4-JkbpX9_KaVtbwCy#iX zC!(pe`~Y644@i*K2hV==67%o!cQ63`1s=5&rKo7j}pM< zFrGb4)>f)=W^#6GXjDI#s@O4XZM9uho$6IheWs?ru zk<&UY%J zBAdV_uS5zhduyRcBHJ6$cleW#VRtJiS(6&5e$#5pJDADa5iaO?jTU9J2~AQOSpRT` zzgwy{aL%NHtTw2Ia90{~I4i%(JQDI&1`)j}Qf4^%u=ZZfTCB(Srl2fim04xXk~f{j zzI0IH;vRlBHu#WlCp#6f`=XJ`Y{-9ilry!&Y`UVru@$?q6c8|Vt_rWd{3rml8HGB9 zRe5<&I(liSXSrfMOeiM%nmYJ`>q1SigJM}ve|uzoRx?fSc}aG%I_=k@I6Wpsr8dJh zM5K^jyw~BX{?v8M8}4*SWkHA5jbjct%xvPuFLCGlS>sg1BKswiJNT=nh{w}Cjaj4- z$u`HqDEiBR`GBfrfALZN?8L`a(N2tTta&o~Oly zG~+TO@giuIqR#2OT*#(z8g)muR?FO}GEm>BLatJnjd?wK0;P;Oq2tl~^s6tu_HsMx z3-x;MUGUQyR#^xK;!-TZ>?5OalvVN$KeO{JA>2y7d-R)Np(?53@*~C3fKmiF15>c? zuAE4ecwvri)Y2qNlCekq?%eI__r9!fvued^?N(2}>Qu`|A`p1uD3QYsQQCU`6G65) z8vB*LflXqCC8d1Ms!~C}WL;q}RX~LG2+()%iW(%=tud0nT5?#1v(s-9Mxgvr!=g8O zONl~H?!D_g7N8CB^xE${UtLJ#yefThOw`W4Iw1&UkxWp6Nw_o^KRaei2C&V!CDI<@ z#ViAA&X4MOM{Hp|^kyDoM?jqEMlslR=K+7mZor@7J>Q>Z8`|#{PvOD&&NqwI_u6G!a~XYq*$j%DGOCiW$}gg zIr!;UR;8dggC)A#p~Ga33MPL^!fjY|z7EC2PkKyEhnY22$k4)NXh+Mbaw5Nl0o)Jl zt_s;#ipqBjzv(pg$hgS)Zg(iv2fTE}vR2>EOJFh5bhGjFZM6O6x5dkH?zC!AiX#st zn{0arH}*}4Hu92dp8T;5yExFK*2fP4ATuv-GT9SHQZv5rj>xKA`{*NPO7vMWNzFD| z^*UJTA&i62@)76~lVT?hi}2)WB@1Zp2V8c~<3qkWNs0S6ZYF;huX~RxUVY4F$5)Rl zIG*gmNGrd6yf;MLV?9g$XsF}QRi&79>4~=gSz6Xww6#%g#!rcdqWA$d!TNT4 zwS%9ZE9`iClm5k@D$k-@?&lp=_+^oC>ea-x|!UYo6# z_kntc88I-~GJf-VfhFDorr}#-tOb`5)I0;xf)TfLpI8tsv~ZW@U7RO}w1L?%eb9hr z>hE2S%Lnk1M18E%N%}D9hhA%P-I=o0JC|yU)}HZ!t|0A)Od|wx>c%I)a(XMh8odOD zP?hLvY2 z1DK2K)%9cLb%AQj+*qaaud%*c#&mBL17CrCblq(AR-(?%^96bu)X(KqnC^b?_FAt- z4{=-McQzy`(T3GPUN;`@%%vp=`MK_KjAln96yvCO_OOf{^%9({1Bh~i^6ztg;*phH z)C`xc%nW*8_l4e63k(pnjPLwRA)P9SOfR$Yo%w7u)Ba`{rU}^8F#Abg*Vw3WDRDHg za&}qD>=JL1Aqz*Z+FJ^SSiMzSvIa*iW{$GrHF{wW&-n&uUsmOu8hKnjVqNqYrWPGQ zu0}K}{M6yqai+7g9+Wu0(6}fWU^JX!pv!p(aFy~}texcF+ZZ=omQm?thsb*6UDfVM;=|5P^1Y+uPie~{G4X?k zHX5WWFVN%{{r1SSo~f4gRSF1=-@U#(nLL1L1UA>vI*?$0w|HhW+lYVoS}Swt=2wJX zw47$nLOEMkC$8-8(Fq6cOeh&ANj0WuNB<;WM!uy*C*V`f#lK=lqVS-Gw4eVAn+#*& zepJfqvO{#o1r)vO#jf0F2SkQspAJ)Y?9=^3cw?}~l8%EZ56O*`%Zp5byVpFlYh=K? z$)=1fh>HedP_OhUwt3;`=4Oag(oIkY%UuWLf_1?qDuh8pQrvhC-p41ycBYAE$!3m9-78C-G}p*$cVV?GV2b57TzeAD5Oa>f4Y<{` zMN$Vg#JBc2Il5_XjQ+mlgvaZs6`ALr;zf~E7GB>>$?mIW`eL(gpZi|b5#J$jIvpQt zYh&l7#Vi&+076L>ukQr+tK-05pyij@?M2c)9UUn5le2QQls6(VkzmLCTD($eQ6Ru6j4y4AE}4QywFeIJdAgGtVsXp}t!j zlC1eg*9KjAi}1$A;N#oMH+bm%U)L%y3i5fgGy>xdDljSk?4~VwWn%%)>+<;l6_z%_ z=QQK@hrQhsi7c)}&o_MR76gs%#uEu?iMBE=7f~`D&J(X7Wbq<$by0g0W8@`$YU7UYl)( zoVyZ@#qC2pnDr?$QK>=R1Tp13`%0GPjt^y?QxOb<(q`}Jv1<#n@TDfT!3@!U z%{p<_VB@2K+vq|Y?sVd5-;}+xSStyMG%1dR1{Z~qMh|z}KyDXoWagka9fCED9s_4Qk)#$qRJ2nKJdEhg((q{+y^U z-z3v+L~VVbfz^HWD+>kk^ff7uqSB>mS^sfL`B%)G@RHOBd}rgt5E}dn;m3fig&9rc z#2VNh1ss6z$ytjY1TL2(1#Q0oIJe5BlYXv55@&VLe2Ds z_Okc*=?3@iK7<{!OA*9!h|*kENA-8GwJeOlR)g*IByKW^eyoK;JhGdew1*@cPE|g+ zM|Qz%8G2lVGLLHY(x0X#jJXXQRimeTk2)>)$hYj3%q}k6*2x}8x-Vsy zT|ut(Q8RUV6pr$rN^mOPi5n;RK>-NMlkOsp+ZRO1c&E{E9`5db)wbUOZH4ahr+2xD zX12f~VF-70^ag?n`|d}J!sCWl_2HFia!*EK+8P=D8_hLi{grU4)!7u{`dhVk5K_sF z*X{WJF4{d;2Q&=$u8k>pCITCNlW1*O=y4Kgw+;l(! zDjr=16j}jRu8H+tH(>bd4|=ruZb#$@py~6~B^2kHtki~xfAE{#8F~kC#3t5QZ`qjc zP|#-TT~E6e!5yLsoyY&oLSF~ZN-4t``Fn!#xhsa@ICXpF;4tygj~Mo6bboSfv7fY* z1i4XqIjxZ)t<;}|K8+iH*8d7Zl7_ml6!;)|f0Ph;T<~W5Al1l-M$>{{)Pz~5(&B!H zC5b1N9*Z(h|A^es4_59{@VcS2q&}>)3AZiS#kAz^wz_U8_9%tjGVdk z)?cMl40T(+fW!*WIfh4lT!WBsLZx!8ia}2-i7pCSVG$gFb+k)P9#Y;b@ zKhl)=bmwJ{TskZ+t5T^zs|d-k=}6PJ z)Mxlqh!CH9z*^DiST@0G_EL62#c)N)OJD!$x>RfZt1x**(9fRBxdbC-v0v zEaWZ@)%-sZLQ*>cxIA0`;=&y4$P7$=tTm0*|3j z#ylg@ov(%SgA4rtYRN)v(=H6y)L_b@w=Q+>bg}Ek#Fe@nguPT#YdeSH`yD+4!b?pQ zDc)xAJAGC>v-*@PZwx+RNz@8b{zwdK34iKgB(HT=s4XFe_ZFk59qnnK_*a@SupNCG z_=rVoBwb?Ko|GMIZMo68ob+eM%%}%Jwyh1yKf@}}?M`n+8%}dV&g}|&3~J7AZD*}F zbxB2OD+`@zR}B1P-)o37&l(`iPnJf?aS{q~?%0i+e^*tn5KX+i5En+LlSH@E}njkwn8Xa%zf!}NWwQ8${j5|2MQJWDd?~m z*6KRG-YmdcrFJgf!N^{txAk6)rtGqI>q2tE+#^cyZZ15=spbOFwd6=%8~$J*Tn>u+Rw`RtsD=DGV>E?EaObMXp( zC#!{q%=*SeuHo_)8mkYJ8T?$Mg5o7Hc=K-tJ}rdZHK#w#Gi{6?mT@#SJ-?W|Okh;N z%L&i^XcIvIM1j@#Ab;VGpOSDjTVt?6rdt}wSzU>0a3darqg^kjhqG{(nG~B}H;2uV z1KvTb)yFyr6>WRWig9JQm0L=e_w12j4|A5GXgCV26NC=Ug*JbK!D~OPiJ;cxKRwQZr z>>(m!Ch+!xlbA)gbAboIztxAsnu%J$QO);b9nM6o5fAh$4SnSef3FRaZOY;YC5t;l~z{%wMySac2xHgMq&UjrRcgv|F$;KaIY*-?)+EuIsPP| zmw%8};FGUHbWITQsh?-?zjk$xjvBvW1~?LX5q6~L zGsjOFw>D^An1h2tC}i2Z)Lm$lX$NRr2Is(@mllN~Mau8On|FdLI1H$LK3do+NB2If zB~q!oFewybgh--vL6Lh2d5!>P|MvMdr?evM1WP9(hlmZ0&|xg#>PsMI1eTfJfFsk- zaNqHSB*I&OVJ~~?lX_Y^VYg&*ug?6S14$iv8`_MUyv_0#S5Ssb|Gr<0A1C)7&Y~YD zv_bujwdnH;*@_*6w70S$`>MKQ_le$LJ1sdt4C1E(1L&V^?RfQ2iGUqpq{ZqD8&8YQ zM+cQCa~3x2U!1qSrzfQ^+0nW7ae<<(n4gxC+26HoN!Dz|VzxARDyC z`CjM3$?VGnUwDyx*Ou-2hmfWF_27bU(ij&Pcy@cTeO|WxH}{2^*atHxAb8ze+Xcu% z_OHO6TYj=o67oBwBK(yhrze_H14lO`*BKgPZ_H{b9+fymTsv89qN4Hp0WaVTN+lDq$eGPCIq15)SvZ>33qoYRvJIlbRXPW z?M9+i0OgniPmk)SybtH6Z-#CBV#0^eSu?h_HL8V+GWk}*v2^rr!nLEDPkP=P-_^pB z{phz@;Ey$+4*<Q-$r#?{bUveAAC!!CELe?(pa!jN^V!vfFdUFzr-$gf zb!$)mMnCrbE+h6)dF#XV1ulmw92*vuV?ha*6vQ(^z-riTKG`v2k}{QO6uJL;m>`$k+0cQYQstjm{h{go-y(q?Pm9GiIU$U3b* z`s=EtiyXM#0@<%vCIHUTkpH8=)Q<)20`1leTkPl$or|jSpJM(N08m+yrrwxRHQLkv zz+cTgyyE@MJqvz0c|C?;P26b+r>_t$jvFYj39FE|`B}GKahQ&(`%a_ds5Co^VPEn6 zAYu%^9`GaQwYmC&33-_OM#IlMyW?C9uIsY%<+NIhFQ#08NId_^*DsatOK%qdry=Eu zwZ6#bOoLl0bgEe72L!tGF>AGqU}zKT8kaj4usba014gr<6-7>(^n^{afBkSeTYzo9 zi9`03ba0eC!y!k3;^0;htj*goC0O9cR!y<}u@y~N+>aDardnKv6@uuQ-L*nm0f8_AEtf<=U@1pr8egu;;z@wBybx zL0N}-G^Fm=X0d2?^4T{Zy6bRW@KSsdfrm!r665^i^TxD=8=(DsW8bJc`Ny~_@%yPM z=WaH)&O*N)os7IAmlsT?EfA|&^%YaPBOSPt#;t7)Z-vmJ#~gcQ@E?k%?#o$1-^UNx z57+N4$_(^J^~$nEaJ-H-uN?=9)(b`DD2{lLv4BX*-5;W5)+}UA8-+yXOa7-{Zje4w z*_fM_@lArGOFG+jHmz4U3ANHho(G*f((OqU#d`Zpq(vE#*{lu~vqHtQ}@N-5eQu4zJsL#u~pK&e$xpHRo#j4Em8~A+m z7ice&4yL7dKq(?vUenbFsJXb^tHv2TOXGIn7U=6}GzM|#*_Mj#3{AlFf!WrRLro1^ zW2|tpG2nMKfgV&yF=1U%Z><3NTU+?Tcx%?*u2e!5g^p)KXGGXoAv2-0!zfy8T0+^Y zf8Z6T?Ib@h5T>wZl-tnC3|WOa=yg#+d$HXO>1O-RcHsyd=hETt6R?MhqvwaR?2O*+ z^R8P-oj^jzd;7D5PQ-dW4}qfB9bdmV_-JDthyJcvv>~R6F^e(KF z#oTqr7LVL}jN^8urqcr;q&e0L?d*+R(HR^>Gef$li=I zRj&!si(=01yltj;j{Vb1cw?R!Cg}Z3LEnsixZr#yl z$>GNGRD4Jm^&H6uQFiPCkS@(!G-s{5M9kp)Tdwqgr1y9Qjnx^64xZlSLcK_B=B@Oy zVQNxc8sa}0W|E8F#KkJh54UP8`EVWL>VJ+OLjB8nvj?QiCS?9>-E zKbaADPCqT6PA)pr0$=R_mMY(1TV+nw7S>gy4kbLJrD)cvq(3Y=Qj96(4e#<7fYM+U zoxXEZp_{pj=k5l8zp|ciSq!SS%Em3D7^sroCc`~HY(>0^fjfM(WY*EDDg{&LOVplG z);Gd|Xk9DW=1Slh<(wP^AwA|OdBxcyo|#X~PPkBB@4#AlnDA_^V_$I zRTJ~%CRJ#$vEe9q80Qm7TFB!DC3(D+Zgv%^g?PVmtb`MSC|)Qbrv@3nv2sKicm;;= z)mvM^!tL;W`1+4keXHz1A*lF}SJhBZ0b|?o{`Y!5O%%bnr^d{8*!jD#4ou`&X_(X@ zdhV74S}l!=M{!^GfORLZybT}doDbpI=ohpRn(+i1WYDF%FBxYk)^n?AHYNj!WkL}) z+@1DL@aGIpr|t802Pz87<^iaIOj&Q~D-ZOHK-mlP?cZp5F@c>pzwTzjb~bWWnhSj@ z)H<1^xQvMwG>2h>E((5@{w~(gW_5o3SFiZGlb}3k)z>$R!&7{^U0}4B$bPr5={|f` z3yg+`)tkWGEbHBW>e>Ou(p^V~i;Y6VU;omlsj|C3?Cc7MwuTW~RuC!ha$dcAdI}eI z6=fQ~$_w|6gI8m)$_>~M0W)pQ{?UoJUCUwiE}809#DfKcCf7^;I}uMB!xVsy zjhP-3R9$cDluv}c8}pa zE#mM?I=J0gbWIA`P>liSe>dG3IRt^vsozDOjVW=3!?f0OHVFjN>P;s!vt_tbX`?K? zXObEeMWac_chhex66xSCv%eSh5gYOJDN)bM3DYu!9w}L3-2DtmhF}B8PZ_nDJPM)+@ z;Lx|;Qh1I}Y;gGlz~3ogbmRGvIN8|ndBR_&{mzWf`bTS>Z7)ht)W@I7B3w};1M3I+ zLBXDXaQHh}F?y>YL0wK7mcYK8r>~_;@-C_D3}87*>iTJ}ai_nq{0r@rf96+*@|zYW zt&9gA>tx3yt*Boo4RVeQEjfL_IN3Vo6mg8ocJf{~gID;Dd-F!eucPi&8WRIJ8$ps` zJ|5DHwUQ(9xadab11zNQhGg4D;kdl8HYw%=qmI=cRo+sW2p5J=Ed<%mQQ{_eWIGLQ zdQD<$(ix3l@;CxVt2=JH3Z|>Ls>)GV0O+z|I#8xN<& z8VeJP#1M#Q=CC6WIv^%$?3;X7k zrN)2_X1J5{vs-k6b%6mFhrfuZg(bI{)2TfZ?1!wCCo?mgk=6C^xhdgBcl%E$u(wJ1 zPq1gl&_$nPdOz=OAcysrwGh`C>+2Ua4_=L``{@ZjYdCyfoeYLgwVMp#kOXwaP=C5zpdp3aJQMcWJj)M% zGJAcr?sRohn(EU1KQ|NmuEU_Bh$}hg({>{G zaLUj*5?83F_DMX{|EeV1=lbHITe{1b!8jCj-&>9|WpRO)e#$@EeJ+_yK9_K@5hcrQ zH6)!rL5=t1mI2MnYqGt_s1Th^I!(NQ+NOpid@m<#{TE6p`3d%=YMm~IL9d| z^H&bB@?^>HHvG|v`KJ%vTBF1pcQc_|ZY(zA#zrQm4)48)y*d%NDT;my3x-z0)|l8X zxPP4asNNok%(Cqq2Oj6Rt8AI8W{kY|f^o1@B259GiERCM3fL|f9oxd>TP8W;l(>(u za$-Wir-#%3gF=1qWKIe-AME=2dOqRRpE7(gN*4UTq;%ppMLZP)h}CrlTSE@T&z38` z$8le(-;1ZmKy%;Xk(JusYfg3gkpl)uKO=uFsd^Z-7e}%p3q^;_^P;utC#tx}xj~bd z3j-3Wv%%w~hMRVp#d2v6yuvR`ib)VtkW_uvytI%>b(lc&?(faV+}_iS3;h7c)VTiK z8@dUN{J7m|NTwzEUlxUh9zL6%YlN$X1gk>&+oJGR_%Wf@LA~}N@m^D$b5XQvzWz(i zenAzVvEPOLySJIe8(C~<%C?3HalH{`Yi(JbxhvSwV-4{h19a{$%d{zZ?lgp3!@2Gm zwzOp{MdW3xg*2~U7jw-XvDM?$m0J62Or@(X<(3|XYaDT(lpyoWz?qP23KSo3OM*ZC zV@4YN$pq11vkbw#pt8~E%7%U5o za-m@K#573ky9%?l(pmSE|A`v>c1wjI`@SkIQ^8y@k8}sITklQI_oiLG8hWk1n=$Zw zG(E*9(P}WB=YEE~L%QdlMeOX#YXx*jRVnJ8-gm;)f>LyEen8dRJ>Pumcqs^wV#*f8 zgqNm=LJ)^?O4uI1?~VRutWdC2m6<*2%-I(5s}TB>xz_(xDt?iOhQ#qdyd0heua+xv z1|_)>!*G(){Ebb-K#cZ5tBm6cLLEiRSSCo|BhTazFKnF69J$kUw(e69OMZIs6iDU&O8fbRR-!PMb+Q@G^^G zeU#GH#`{&PK|*pT*llK@(~wo;BBwPWRM-Z;mOFh@d8qL%?gyMgG>WZfm7Vsu9=fi2 z-9G(k%012_?Yy|5(;`>li18(w3-sB6M`%{(#i|F=Z^3CRYwzsTVP8wI(R<{4wC3df zIrey-HSQBUjIn;nANFE&|HULJK^QRlQixYpQY&9g?@w%cBA2tc6y$oG$H2O${)HE( zM5(w+w8+Azx=botlwg{yBLrpODtbtR%&|F)n6xE&8Sze<-Uo$(1M6tN=3{*CH}i<_SAqQ<#JiZW&>7Qi-4*pxwI8~S^=2y($n`eyzv4_ zn40792#+22n8qt_Y(RHQ*^OgvOC@OnT zTFo6YLeX@DbSy@Tjrhywr$xN+7bVt;7V?3jj~vx+9Y<`$+I36rXTj~RQPns~J`it{ z*St>*&d!Tu@*CA28hxMwm9t=5?SJY_+Ln+Q^$Oj2C+dQ9&R!C#;SBp!eElA|jo5rm zvDdjr%bxaCMz(J%XAjfB#v2~HZRDPH+ z=0bw)N~gQW1u)9ieb5%x{|?W66h-r>Kz;M7{w>bC{q7}0P3ts|csMZ7zQ`2ELHF)% z?X-Pc)BE?MvRH<~wYd`TYwYh0nz`F?Bqh@@)$j?pp0UQF)7;bbwjEO)LS8` zgCtl7ha5S(OsJL)(H2MsofORPMyFo)z$b>?R9S#%z_%Kq`tcA zqlO1>?3g$O@i6>coKdh?N*^&e~@)yVCxK!(Z} zCm&@BbTb-fUHqX1H0du3T;*?hLIgF3R8S;>$zk8E>&BkssU_9a&mzpki*E@GfooZ3 zIBMmm_oVIsk_fNB7HaHaa7);uC6reu@srGioY&TZVX>SNQI4RRChvUNbS+9LR=^eJ z?ESh|5RPmAi)h`$wwG~uhrmBM@BR3l^8U!KZXo_Wf3{35Z$V*QTs`G2b^0dTcL`Sx zJpHj@Y2`85h=W%p7o<#qneQZXIIx+KZ+D;`8~h@mGY(b@*0Yf|IkWe`P48hlF)TA_ z?ZZS4O2D!ncNhdt`%-~W)n>Oeh#g7oL=pXc6e2WVr?noHs;^)B6J~ zDr}a*`$g@2uY?_#T8dvkrgb#ap4mGb?ijSfg2~-4+?}Q9?=?hNUuEAf3~lP_F4q~z z45VRR?e0Jgb*y`87j2C*H~cxHV-`H49hA!Q!2z4MS2H2=LP#g?i@|;-K~@TTX22r6 zITQF|r68kpt!xrFXDfL~y$hzD`$CD{xhBk(B(~v|Fc#ZLtXpqH^ufYj{WA-QYYB87 zUUupc(1zG>BxC>NJvv&=@}q|P#efeEs?!NG+WE<`_x8BMzacghBTqI)C6%kFI5Z}E z48G4-*)5mO%hMY{kp6-~P(E9CfSNX>&C7Xwf7Ws57Sn(K^VVJxquC?WxOXNpe(wy{ znL3&15L_CWXM-C_(MKz11wQ2DMCyR(!AxesYro^dNvlq+FSWjf*K=SSb+rKBBiH*i z3jwFoXBDtC*^m?pmET1=kSN~ArW$RS-HffqmLY$8%6m|MTGLE7 zKhb%@HFeJT1kumfSW=}Wc|YU6T~~QCpc=1ZuN|1kxrTC|G#!+5JF;{-`{GT`nJEkU zYLC!2)*^E!Yj43-8i3}MpSj-qDIccm;zB23y5j`Jlbh(74+fMUZ-(1ErQVhryNr}qNv z=cqmeE2HmA-alglAO0|G**P)DB&t8W!ObSdH85fQfJNf@tmfs${9rnp>1UC6YH{wu zRsYo$?C@V{`FV%z*i`4@+0{ZYb<4r4@#pgDHCWf>b_;re5Rv%I;jul*Xw8#d6wX(m zQ43xsSbRAy`Y!oYM9-hYG>H z!i^@|_;d=3S$(jb5Ydh#8+p3$+?z%b>-DCTrP)B12PC(>7!6m}4csf%vpbFc#}etH z{dUD@@a04-8;;B6Y@QB0339Zi^#|FCB?KfccSI6`C(4SlAx z%M$X^XKWd@((e}+^dWDh?IES@I)}JY2g>fZ+1o$b+rwC>Xm3Zh<2Tx2J}YNgmp(4l zCrjJ>Az6N+Bn;C$c{hTR+X5kD)*H4J@uCPYzBtO4I~1B0mvUg57_|eE6aDo=CV)@t zbM*-yJarqS@xMo=tk?yh`casO!A6yqiU4=o?>$zy)MMuf`l+K@U+;1I;4r8JYvM zo?gxW+9KU-Tut&8?QF?WE3(c}lk5frCs}cvRKgFz*KM_3vOjBBzRFqLR(k}gWN->X zXIB#A{LJ&9La*zud1ot9Xrg*L8nf`{iy0OmSNj%v<7@W&&z3hc$p=dx{fsm9PGy4V?&ljEB^K*m3@L){LI4VQ-9ll4hk{nfihu;kGMS*(lS9 zAfMs>tnm_d^_ymxxqKLpKgY3m2q7}ml#I9 z+3PQOkvTeeAkgDMiYc|b%v{sdiYD$-G9uV{>{E>5N1ObKk|#;EHP{P3}1}L!9t8i z93)fJ*df8Q9G9unkhGi_8J+fAUk~Aey5)*p37`G+h}Yfz44wYL6L1?hUrZNN4TVFJ z7-o=N%RVECw;#(FHU&N?^mi3o|Y4{P1nKJ#P*DS3{4Zmp^ zLHNAywBD!xI&|G>K1}BEWn~q(Bs(tf9kOREN7N^@2%mH&%}h+8F2R~)@K%VWFXoo^ z6a9C0dmOsYg{cFW7OZ2&N#xOEy4ixpss7`3rN4BbM7`rCzny<2yJ==``VC88A42fw z7^-dV+H1HC_giDzw$6pc9u`Cm+0w80TWG^Q*lH&yDPGiOI3nt6+LE>$1x$@ZlXPp7 z$Y(B-B=po*;}QUVSwxw$N>c0l2<=S+n;M-J{kkRwi2I`J6{qXrv6YwM%aOe=to>~j zg^q4Dtg>wS{m)wQGQD8&Uc37vZF_kiS}ConNcTD`kj z=?JXW0S)#b1Ci@%jRbYle(=!ajMmp2Bs1bWig<`#y*7Gscj~9ERi>u95Kuq68KmT( zcAqH@B|ZMD zcgLcE#;m$}Mjn(-)zC_X40PPdFcm zq3OSekgq?&{Aeh4@Azwlmri(w{U`RH$*y0`&;j0HivhbGCW|Dvs3p(NSyV+pIocEZ zRM$#&Mfr$4$M zY4;cKVg}$K5v6B5K%4V28uh1w<_@8w&>ufT5EXZMEifeq`C8c->eiWS;AL#Zp)2F1@8mO= z%Un^1!Lx0))HzgqN#Z}=-lutYb-7&htE>%7E9WwPE5esBn~~(}sKtaDU$V&3nSDPt zdszKil?s8VKk1qk1w63WfSi92sd;DjT||RT7pmBDUpLU!IG4m1J!7)!8h!+%&(IjI z;?+9MLCJ>iy>_F$9LEvWr_1UCFJAZ+&4xtR+imloW#yXPHSE5$tcbdzU>{Fy?3les ztzB-rEL)Ua=OyI>8Rad4m(F}Po%8lbUt6$Rik`iR)b)G-vRIlLAdud_qi+N9RHqP} znGc(`Z_5&zOU+Y!rHk{so;y{fYv5F_hJmkz3Zxa3;mprLIAyj^z%8{C|Xyxe&rNYyB zhWnoHrM>QOzq2g)U410<>G<6XR6|=nlL!WqZpWk-u#3c}a?TIAJ{#|Hbw!{~NhfiW zk1Zl-AmCpHOBoo9}nya|VvPGuvgR(Gu#6onigU74~%tY95a`>Xv+oafV-rz1m;FO z2-bk|ibijp!2ZMXZOL`#d1Y-ESJ)-vgQKsP=s}Cgl4n%Rm7Se8h>B`{`*_jHz!6ag zu@M|Up6Z}R^Qb*hM5mb%&1$2KHSDEjP5A7Xz;8BH&cEaO00gsx3#?j@g=A`)9V$oenlvkdYe+ zgWlET>>Fvk(r53CjE08&bqjlh;3CA+ZSV}7jk}W$j0Ho0>SU<3CO1LRz!G>@IM}T& z{^?kfPd#h7aEC0N>ortn118Wf6nbNtBD!&>thg^T%XR>9z5D2aMVTD?%vL9PY@aV3 z>r_7j&vV{C8j0$$e;#)_yjTDT>Bu@CB{R%6df1J3mnm+HGbvIXO@1npOW}Qbf)iL3 zAbhgU)W=bGnfw0XaM*|6gjmNs_u;?2^!`0B)2O)%C5Sm~uLw_-s*#`?8Onr(yZzo@ zM<|^+ySYine}UyVL@^zoUG|?&?{5Ohj-Z_stMBLQECZ_Q5$^MG@BLTtlUR&{l=T~F zYp2>A9QH7n<2&lCsfeM}c(j-ERvDTd$jq z1KW`4?;gJ9MH1L{Zhnu_|5j~fx7(ln^6PUnK?({Ay$Ha4V_+0&6N!ueT)EtI@rFI_-lwig_=YBpXxYa(k!w-OTixXF_x6ByCzZ%0KJYpC?ay3BkP zdA|oMuENex5)s0kEb2^dA-A97+~Xh{y=dEMv$R8mE;nRQjNi$^Tbf+g(b#_K@ar`$ zPH?9?-K3daS!r?&RTcXIs>)%n8>-=qM}h4MacC3!T=P zai7g;vyZTT(E#dp*hi@Dj&7NKLe!P$Q!CE(_`Z1b@C+y^3e^=O_v4}E7@~$dhGr4(EfSW3s0_E%LGZG^s~NRQ z>9pqOw33UdE6g5gLEV~=GPm@hv|rzoO#c^SUmXzTwzW?qQVK|?5+bE^hp2#*fJ%on z4&61RfJldQr*t-F#cs~?vw_uj{{fFLc*Y2vxRKOzJK+GAw^ZS0S?#Yx2 zVUN#2Le@Apn@0E=fn?RAMne`!R-_|!X85k+^84zD?Dpt&4DTA77w!6Dq+?)hzt$=B zJ!N#2sM-8EnHS+Ix_um)-);R{-FE;J+ku^4-g!ZCH7YVX4?_SKCBO8bpla8r5fLt< z*dbG?0&M8un;seE=o7BV6X~_RE-X!HjXo%dc^S!fB*U9CY13eBT;>Gc3tO+D-B|cQ zPzat_D02=Tq_zwaer&SRBq#LI42t2JFX-P|%Wvm~G`STjZ3Qh}>&cwBU|sWwXGu8d zcOu{Dc!diJ+!c)gLN8c0)Zv%B^lt%D8CsIl^re3UQ`FM4A742+6KkEaoY+d{tSo`i zn2XgsdQ)Vl&Aay#zmE}JV>*EjB6VQ+QaX%pHC~1Jya;ug?p1_z!)Dme7KDJH-6kG2 z4NZmpB0HmeT)%sGDM-Zd4c(FJZD#iR(3UpwryHjmL&iw&bmg1*r=ag%FuffgcD=SO1?~W@j%y4f2_#|QE#YquFp(}Z!{Ng zrE!$7L2UGyXHfJl_0yMs&y#E4XT{-J%^$-8wBMiKLJZv&O+FR*(Vu; ziH&;dTKYG1CH6vq-o}-90b(A63zuQZ%z~mD-qI>r&t)YqQm4jLY7B+WFD|m({on6H z?8j1;TP~lxzirX;D{6hD9$p;1Gfk%}e*J?wna?itDtVHC^2%x|uj+(yo`9BtDXQ*6 z_qhrjJT$8z>hi=dM_AjK!{kT)E!T(Ra#Y)x>onEMM)wXl+>o;2>xL!NkXMw zQjhnWHmOp!Ndw4{=;$WneM5*ps3r9!Wyc5L|M$63LVzbVm+bQqMB^Ww}o#A z)VwH7{nqs6)K#iJuuXT&T>Y5h^hGVF(`^!!>^=2mmnzc2Fdq1CHt+ke(7 z*+Fxyvih5;Hx=cP=kM?r9>%e$?oUp2gvcTu^(hkjXg>1;PKWs2Yz*l z-DmHRenx9fOUju5y&4UjCSlddB{?)=?Udd_QTgQA5ZQHcyS{ur714g07@)GP(ePZeZ&%iv-*RlDU>Tc$7LcmpT}`?C5v`PyZ7R(gkWv8)C8=7%VV{-h);pu*->ii9%( zmU6V;-f;&uM6g?+mh8nO|0y;C;=GJ?g?=>vDWB)Xtq0nIeQ=Sv)EIvTv3qpL=Oa=J z3XTvrr}{{JHd>&NxXqr({W@_iG&6is+_mVrHD#)|k} z?yJ9C6(MZ6I!|T077A4(uEAiDC*!(%{iXN;msSsk&q%(Lz%1nayj68@G1bh*CWyB( z&AMTjssBTS1KIa2)6<=a7Ow9`_i5_Au1#}ROHP`nU3LS7#mok`nb!+NSVC!tHZ>-& z3udL!CO(`^i=Q^Hm>{2yo({4ArxK(a-yk^%D@|~4#3N|Pylm%&BP3Y&xiP5qx?j5D z|FNL|G%bJm=$nAsuV0T5wl&V}JL{%;xAARmqcXlTAO}%pGa?~!aCXU|Oj9>o0klj$ z`U&G|E7labf(F-n2wZKe?|duxRA)UKScj2a1N0ysy~5kmPU2O;*6tA5(>$APrigx4 zY*NyC;pLc0O8$b@U>5ufh?Z7u`^ZFY*2o;Byp`lQ+_Q8B7ORg z0=SYh57y6z$R_HX9;oKZhDXhNBS&AjY>UxTGtVl}b{(Wx7pF-1X~&w@3>No5M1Z}c60@p* zIV7G_ipLs|XZUn?wNKPZ2@M;Y*G6@(;JTt_`Fn1k>3ZA%WspZOd>}aidA-;qzDQd) zTOh7TC6=mL0-mb9L*WgB_}Av!CZpM@wxw;|cK0FzVzU0LY5Sll<7`hfVB^M@@*|ye z^?;sv=XO;}K)Xv$jbr2pXh`YYj@%yc0}F|93P>^J37wotkUrA*K^D)Ui`OE)Wh#<_ zLk>&lqqUfswLOr_cqekz@}X!$U?=`QP;5~A=3uILHDvraXo@0)Fb{Uo5h|W>mjC$3 z7;MUELIbU@~1MHxM1%oZL$ue`&X7+z?mVdL9C3!I+yAtGI=Q;7~Vp*T3 zX8O(~g-sd!+DxA6Tc-$@W3s@^dzYX+EK-UkcZ-63Td?t}(p;2YOM3szCNmAC3$RX| z%9P$R6?9)B8I}#ST~nY?z?0*Nz`og^r)&~~Y=@lmtpT8p?_7m7PIH+~2OrL31T1C* zBkYVu(9iHCu2$LoWU8Z_QKz|-$svd$Pk-tuk9Ci5SH6-5z49(KT=KYJ1W|6eIdy4 z*B5&EwNQJnr_OS{%Yef6($onR+=2IJ@%Y3sj8pXZ;ALa=t|`qT1`hSpJ(!949HeNY z=sN3ToVa_*H?5yBNKZ?LR`JBwuE~cl`6~9>3F5<-!H>e@U6l< z^jpFk_ne=$>kqhd#!5tBRMRZD4dJ#xE>M%}tO;FGGTJ0XJ@)`RO~(72S_ z2%G`6k%;W0EO)nn-~s*5i8>K&W|t zdjAfl9yThZTR+Gw>P*u2Gv+a08iQ`*XA$5Rk0pHtUeJ5@79Jcb)!^A@b>i49b#9o$ zTHp9y)!QAJ)=!o$;zu(`yW>!cEN^QL@O#w^^bX+weQSRu$F{dO*XD|O;AB$r2?USj zOd~pt#O3#tY!EeBPdZr;0gb=6k9X{{`S;wIDq>G2`oUMe>Z}<|Fh!hobs-$Rc%S($h6I78_qiPjh^sZ2zZr4Px^Rz#ghaw@Q1d?{ z(f@6*3qqJ)$#wT|Nx7sYB^MmRJ4aaWkzs5vWKgu>p+N&!{2l&gvHy?DbDczMZ;xp3 z+Ut9TI5AX_d@adyP89;7PWq=I{Ix=U#fSbasj0K_P1Ia3{&NNW&u#waIVxBTV3;LF zgz=C5`Tc-_G5V)XtCObAgY4bN2y7IO&#EH=Aap(Q-leNi-}@Nn6k zwNp{)eVt#%SQa386a7v<1i7I2>!&&&kBrKV%=}V6%F*19j68Q>U+)Z z#h%Ih!+bAQOP-+`MYWo<64yl?mh2x32J=7(45z1Z>%A@vQ__$}3Y=!XbqiZZcj8yP zvPX!RmHe_+3k}CFSG#|R`=S@}nKK!kKSiOYkvw%9z}$Y0)|Q0X|LPy^`gs2%CMggF zoXC3x>H1;VDkt9$kXLV0f0SqVV;>^uWISJ#s>80^VmanwyIkl}jou~~-I%3&HaAga z@xTua{XR9xXNL6Ju-r*Ijk}!^z8HWq5Docl-T(z)t~Ob1*Vu%S+%LZf9KhXRB(9Gz ztY;x%#&R)``)5CrbN2GjCleyX8EV_mERyd#Zx*LdP=nG0*U2i!>!|C|t-QXWj zX={tGM*5}VPm3q3Y36pWggy#vp`0mb=FY2NoFg?O#?6OG2UcBY<5DBV<(*T+?NNMD zZ)9T_aK$N(iElKkI^88MzQny+KAo-bo_4R#QwU4zU|#eyUUuH#!8@6Dl*+)+w3)B} z60YwR6sPOXU^5ggXlG7%#RyW@USgCRI`&r{Dg<$nl)a&8S9Lv&hO#>JbI&!X-DZub zOTWoCkzrJb#fTqFB9@xF+~fuMG9(s4=PZl{_rkZ!k&`+E9)`sZ}S`Y!cs z6~4u=S)fDK3sXP7WJr)bAEm*hjoMSIQ32VshhH(&vE_e4+o{Xu>hYhztwT*(l{-f7 z&(mujzGY4*!lp?{Sx94u*)Y3UOL_cEwK1QMkXTTVUN_J3n`$f5nj&?;szZKOB8Fkp zF-^GDR-DoIGkrM}feMV!ro#SyTy1~-o~VAd+K|C(dmMed)ID=r zAY1Vk*DS(lE+=wlBOAg8K9(%x-B^fynz<39AlgVs&VjnfcJ8)dHc?~uU=CSLcSyo% z5e`CWEbr$sYtKp$GL4L#&#i%Ra>Uld{$C@qbH&-o6)oZ6j1f5(Z2=-xELy#qY1*fM>q8m7TpL z9Jz&`u$`ZFnC`nBw~6ZUn>~EeuZ4rZJ=X*t>6a&T)!%>0l%dFyY-97a>IrTB2&Y)1sjdcjEmCKTT;tnXo)##TXRfGnE0!|zo|pMHUKn%YdNMV#zy!Fq6Rok!>4uL^h z31c^Nue7g3lkYb9>UNJHj)NH993q@4LcVV;cr<7aSumF>d}y}$Q2yDm-V?yA=BGgoj4K7BlDjE;u3u)h8Y1E3!e-||-RO>v}Xs5m-c^ToU$r-h?btoBL! zm9Zg|1*AZI)U?>w@uBA$WH{kDCNOn!f9Bg@&zJIxH!ahGE}I&RE$QO1k_g5vn(5k% z3>3l75loz<)Z~^P3BA#oyyx@~TtN)%pc~AHJ%P@Pe+VCZ$6reNHv;ajyWkO0A@cC@ z0T%`q=0n9i_WfuDEvv>;o9<$vfy<`xQk~}+v4%=q5n{@>pl}gEK?eN_ZookbTroDm zdxt3GV)EO7sTnwUs|>&8BTPG(Hi1>iu0-Q$t8?6a@B$3KTYZW{q*11GkZL+nzfFkq zP?-lA-IQBFTQyrAe-hMz>j08NY`fi^bbcAgE@Z26TJ^;GF{!w#BIZiXhufq%N}G^> z2tGQweyQ5ZuBOrMQg9~|_2?oaK81kn`zZwaXJpGUyj`rc!T`^nGV@_EH8Y)vcp+-Xyuv~Xl6H?9Q|?|u@~~gDLI+Dk0R%jLqHO~ zVQ5Y}9O+_{Ej3+g`#Rx{T#k+dqAW0D2MW#>%R9?G>QSo~Bx#qB*2f#NTx zDHBm>2kHyYwi8jQz0<`9Q~vAg`;NgMst>~#CU;ztS_=Hq?5KjJkrL4Vyp!2SOTsDX&CkGMy_a#U#c_ z(LjDdVB5ldrneWzOI-W|VWPtH3%TgUA^#?2nve~G0ThC}c(G+ZTxLYh^{ z;9TPxQ?a1&es@STD)pp06o;6_x2!n5A0UyvY!j=YqMnOQUInZE#ohhKw&6Q2CyAbp zdiX$PWO)(OyQY;H8`85-7QVsNK83KL6s@%aq`~XJ=cBc9wxKIGCm2}X>4wYU<9a?! z8T!cl1OeH9ithkF+bo23ioI z*;&fIjyLKwmi z)&OT+@T}PO`F~8+A8-3x2GHz_CKuh1WfHH~qn*RJI|o4>C$R>iWx6#ba$oGTU5%Gt z;!6BB!?YNH7-;|OC?MT)XS$J~524t#oIyWQ=Dy6|qxMa(8~^gVyPh%CEOzM2o2=aZ z^8+K%xK(K}F{P#c3jc{(XVy}E2>E0I*GfW670;B@UYL=`t&~seEiT2;OFYB$HdOp6 z&(A~;K1w<~NhM}#=i3We6KZfhT?wi0WTSgz2KoN&Z5_p_^~JR4tq^WArhd9iJ!32S zGmF|*`C3H8<}4;cnWk#CYe(aLxxaLAXrI)t_0q}XOMf9hRBJU|4n{ibG&$qZ1q+YL z7-UHBvemB2hz&Jr+bGb&rxjW~OJ12@=?N%MPGOsc)A6Ft$L?JN%=c)rrIrumD6g&;`t}w!*9h?S<{I1-{o0%#KSVv} z3R3+36E?M67;h#D>4+@Yn<*!{G_f!#g7{OZw-`~Q!0p&u<>Q`R9GTpe#VW^o#=GDC zD{Wd(Bt5@D?yaiXdwtWWcnC<%-gR+tVSF9i6M24qzA#k-0@QQc8lASv?UrBRkw0Bf zTw7}S+g-MTK9bq!{q|P{J^Y)GBk4yhVSe(JBx@B|bW}B4UxZrmx;Ca^Y$MQ( ze?#){^mJGZR4x&_?N($A;{}gO%3uR_c z<&i@h_x;9|Sc!^yqj3#Y*fL=oweZ8!v^SXRjFL@ZWTou6!u|R;mvi4KSo(C8K-kGRAj}$yLZJjn0`UlBv4wZmxrt~Dqn-?GpSwsTC3(I`_?C!oAt9Ra{!4*#48z3dE$HBaA;sE?x6AT@Yixxy45G1JG*& zcRbT@iqATjAz4uvN7bofUKpz;H2TxG>Ktk{1foi`0jVuw!k*85_IuBqwUmFnQQ)52)_|;em zVw7@wbZsdKZMKynKpvs=!E)aE{H%@BaYsbwGG72n{F)U}_v8JmG{3I>EtoXZx2E3h zy6ZgOEjB$epe{LWM9DtaYSl8C%M>$zYlz}(qZ^Wr<@p7q&Y21Id1kA)VH&%2hgVPYE(R=It_DQCj59tqN1OHkOAu&7+g z?CxWQjrBkC&nZ6Ru0G$FXJts6rFMyv^m=_3S(8Jibc>U}(t&x|Bxs@`G1KJ4v*K1l zap}eZa32uL)HOPsQxbel&Dq99pr0Py>!=!+TNvgjP2m_8931ba@XW_*SWbUXvorO_ zhY#u?_ z!XhV~KaV?V13)DDIUm^CO&bVscyP@v5G@{OhJdzlz0+1)?;W^QipfrJEUVf>hqeB1 zZwrVBXxuNk)M54&0I7r_Gc-}OVXG>|^ite?#a~7>X^O+7hvg>C&F8U_>;C$ET5;FU zyt@V2>%AR+w=cGZkWN%@6UIm(!;MS$clIk{b>||yZ*Fk08JFJ@za2Ii43{CKTbN7L zUeueK9zmz5TERQhP;k)$SKio6g-+To_S@mSse?}3h!wB;BCdEz^dl0(yQ-keD~m0% z34%DoW`n3)Fdk-sglp*RJ`}z!0J<-74GK z4Ug4fx0T2m>yWr2OKzzFrG~-kXY*R)iu_1RNG_R;7+2x@8ZX@9>mvaWf0gNtX>TG~ zm&xn`zV`rMJ_zMX8K-pfsQyLqTa&Xr2Jtw9okC2W5?Gh8 zUC3jR8DaATd_o-At)#K3_go2KyYu1QpY>7~*~S}I>|Vo#yvR0IfyO9)mt`(7L_Yxl zxh36jjs~Dgfqj$sK+jvVK1y64tG(6zE(QULxmQm;*{{&i`VU46W}jXL?L~^GiWH7} z2+TPA_{ti{*5(1 z8>FeW#AnaU*#u1CA4tKZH24K#Sc|UqZ*&2S zZlW09l9kxXgAX4v7biI#Vi_D=pAb3+L~pX)Cc+NaL=OtH>tjvfrZ(v);RNBb9Gc0L z8q;SmcO3RO0n`P*uD-?fv>l>qNQ7VRTwv?oqfyXUuW$=R9A}O5KWUWil{9&g(#fOsv+E-8B13-p35fm4z zv?A^Cve%?#O}QdzHLWL|clevRQgNNi$Ocn*4k}d~07u14s?$zO!)q znZ+=>j%i$$i0v9$cgEbb8jDHGf#DhqZ>RZ}i6Pm=E>;&b!(esJv~fDvXz4VD^GU-| zjPVnIQ!=#!QqLv&7PGntx>DozIZ-2t)2|;tS8@aWauRw4lI5OgR{KAP%;ELojgFSs zbQk`JK5=wFsPXox*Y;EPB$1AOo9V&XbGNIv{mEjJqR!6Fe(@M461SUFu}?IqZ>cWC zbfOvC_1VT3@^RD_Hk0t)v3(WTTNZLK?)XNB)iy72PrGiX4Mv>LAX8hsR!`u(47W-V zUq&st`c+4?H|;Yl&6HYI_BfF#Q02a{Wo==Y?CMi5>xht0etubP@upHkv#R^w zy-W$>zyKw07;NG`&g|j5s{x_nKj`E>kJ+!5s$9y886lC0Q7__oIvZUj6*f`9(xc?iTZQxfB}7R zqCR&DX~<5PLmJt@J=U~XfA~Tvf1lCpjni(rY|dSEPh@0V~q>eEr$h!I-y=4_0u^@$r{;paYA#NaZPx7%LvH^xJNm$CDlE zrFua@7+-)xR2*_kFEZCmoDct&H?sd6<5wCP1Z#Z;+R}nwjrsD$yd9r#ioCr&Iy#7| zY(~6ku;x)4R7W=eK&DPGD^3Zjux`=W_eo+dV{zfnSlU~48|=B?&JumLoiwNU9}C`N zA=o8&N2!VU5G>%9A{e^DswQ7BOZ_t%ruOQX=V+){S$EvE$oPlW&U=uNDWtCWisx;5xm3p+ z><-a-c9Y_mo)jnD#d4EEkrPV4Di%Vc<>8C{*uxQuDnvA5Gs6jEKoatJvXU#?!r5{2 zIPtmelNwK1Ne;uQZKkN6+SIR-E%MyzIq?X3vzxl74%_t89Yb^Q;rWeercqcqWJ^%@rj!`_a_eI$(hvneN$Zhwi`{ap$~Rd66A40DJpeL9_lQ+CU@p-TV`~F2)nw8`Fd8=`EB?$+pS*E=DR0Z@5i2K)Y*kN-3F`WchI!T zLu<{4sxxcS5ihy6r(D`Dopvt0(_^cgm$3e(lK1%DBUFxeMNA>nj(cGw))xKbrL40* zGX{cwLGWnCE!tK*-CjY%@3t~dn|4gXW=9i5yfD6eEo08j!ZupHqNu+y9?;t`P{2fn4z#hlP6A?tKqW z_2~DyhS#WJZ&7x_Q)m7@5Mh66VQ$lW+a-yYc9nlMpH7!Tooct;Huz>TE=fwr$<=$* z0yu%uE}ZP8B|I+Y*D?*id9XRvaF?VjLk{=(j`raeMzJn0%cyvtf0E+V^HxVW5f~5D zP76MfderwuG`c2;!>WB{$b34R4lDFlx^&}{d^D5DM$jDGQfa479Uf8wFaK)1HCUd) zyqf+SVoRj36LG&v{h@C$ji>9`d-kM@keY~{ZU7qo*@zrpM=rAq3JNV<^%^DYgSkMG znn-Ich$PzAuAjxrEc;*CCuo2Q-gFX^X+n`+TEa6^6v~>tK*CO5`o}0#H%T^VVYTDI zAGQ0)S)3Ft{9@(Qa(txGWON|51dlwNx>Kpt2R1x%tE)@7t}-#I`5y0q4@C1KuJ~uT zXH|5$n4wPivo#^!($KW|#FGTVy~!u7xbjBau|f!+*!x^>vrx>??C2#^aKvHGE*4PaveWk-sQ1npI~0sUe;2(8e7XOBTcY}To; zH~+B1+={<5A+4yZrH#!kGu8gwFV4Z#Z*xni-?Y$5$nzf* z0vx@o*hG!G4?LT#YQ*s^WpaUgru}K$1c}^vJYW`u)G&z~9lxvq+$5+>IE~ELFE-ip z>gms+$_YO;Un>NKmMzag9LThdX+G$h86Lv2e;#DR-el@nG|!j5`zExHim8{=MnOnu zEK7DMf_w;F)%d@%vxS#;R=6}zUg8m#JWDabX&@;WURZOo0j=4%vV7z(lRqRe1<&!% zB4xWxRdKq#XbL;+T@fvx!CK$PRpTx-p(A((1Iz?>s{(CYmnY;{4RwhpqU9_%MjB=4 zaN6{EK*P)mDHu1=3?+#ji+oPQMBi(3^EDHYDfZXf(JwB!=xI->Q$xy%6R)0V zq1g?yuaY@|r@05*B_#hiGHU_T|5b(XJzyh7R}!92d=iE_Aic$V_biWA!`?TCq(ZhDXq(vxOe< z&oM0O&g@!^cf}zPS)lZLVxiS)!NzSiXaG*(x1uD@7;EjM7WKp!F}*d#-z?Q_y`L-p zRdP;8NWmLl)aJrWr znJmTJx1@;JFgTZ0t6ZX~ZuYwybk)tL65*Sp2yt4XHhhT%%CUtT2*W%4?9F^A9X6)i zsr8#2-R!%rTA>eV7%8!BdZs#6YQC)zja|STbkH}4a0+G4DC79tcNnAI%KMF*k79vH zuP>kbwF+a)YaXo+${;t~p_VE*qFC~hGA5nN02V;?CURUqJFvIw(6MXRU~LTNN*M!3J2ItOgf;4o zF)sz)9C^MxzhiKZF>9Q#2$3 z9)qa<5AOT#p?H~|B)WHF?PfBdQmctAp^t^G-+fea8M}(OJqn@P&<@O!Rcay7*}3S`NTLhQ~?nxk$?- ztMXeEVQIuU~TWEo&5$JTs7XiCFX^>b{VE2aX4* zKrC59%1K)258hrtw&GL0$Ec1{vCA5CztSAO-DZe=fV>|G+~|CYYW;QjYCAy*4D z7Q#I@i*ff6uAIa+2d0h`4#*wRz|@F{h_|?5*QzK70iTWl-po>I+g3SNIb_~lulZAs zT+DOyCfF39DjOlAdc$Nzd1fZYr**wI*JAcs5eh#aO2QCnsX)^~KMwYW>1V<0r)*2AE%~FzfU$2hlYExBo>;=ix_mq+~l+qNyHNXA4qE$5AZ=9jORc6X?3 z6&C~5WKKWqe!A~!yBB_98uF*IQann1)73EVGe56+QaZ zB((CU3(E@zuI?$FiALBJRy(RXt)9;+)%1Iwv6z`vZ_QCbPj8O2u=rw_l!NK=xN5XE z?0r94S%jG0_~FYVG-+R5^5kj@k47SH=dVRDAwr&C-=$M2?xGdU8pEm@4~5Kf=>8j zxb%zeQ?`!d1KQ5*iOblEZR%A4Ko%2PO;KfVxo&fxgq%E^N|zTz4d=R`aaMgcBxd!B zgi{w)KP=%3YX>q((lHckRqoX<&~bfe#j2=#f0FH)QNLu0VEuF>S0fdGy6s!Vo5FVi zPFvE#MVew!F$@fViYldAVpu*UuWg0{X~>7%TtxJK4RbSL`mHU`uSK zyc)}sKqymU#v8Nt8!2uA)qb|MD{hkfh+Q7!@pK7jj$Ghe9xt&G14zIE5e42QP{Ybh z%}(_e(F*4%mg6LjM$h1OF|FjxW4kbBE0wJ0_<~FdpZrDmflyYCsGi`(H9y@$`Zl_v z+XRai%K%};NCtU={$xQc%<=nw^a3FIMQlJ%^v)z&wUyycsVmj|lDZYAiJY?5C{-)= zd{@t?7@m#oY`C!eY~A z*V)|R>rV70%!W7Zp-J*vOTIk3<834_fbc4({d)+ znethgdbjH@&u*834#-bUWQ+&ip;&o*@)XRNS9Y*<9e;l=+8puYWbWld*84X*^$mwt z3ULnnNeJ#N#2}x^$7AxRSVzoZuc`>e+jq9qSEaGR-;Ff?N z^noiVcpkTC4fZ1DbT-3xVqfc5=|m?COXGB!M1krAKRRDFKuI4z5R8UBB4+U=_S;xB zhpOD^(RsUAg&tacl9Uw%tcwq`(VOQkz&rdt0h!(3^VMCSa;ZO7wi#4Tc#JdmM4Q6x zE@?-Z1mV|=UFNF!Q*G`B(=*g;rZz833BoJ9CjQdz9_gpj00l)+fF`@VUm~BppPEPZ zhY6Q-2w`jN@XVxDP%!+oWNS|LiGxx))5?s(dgSlRK8DkKG*ocxd)Z-z70T+wkzD~ z%8T(?$POaRSpC!inoFbYNnkQ_4Ki%r+l)E!9x?3?#5D_-_h}UCKHc+`uY6TrDm^{1RLG z*^jADZhx%9H0PVZ$%Po_Y6j4QC|k3yk=gc!hzKQ(z+6(KL7TGS%sWcZq9+L5@$I|} zp<->VwMQQ|3j1oS&1^lE1RdR$=qPU`LJ2l5?yW!zN(0ELr23e&9lq`J=a(yXbN{#h z`{UbYB{VH@P5-uOrWjsz-?}%M&eEPg9qz>q3>g1-BGK$kh64okAav~?FQ(_bzjw+3 z1U{!}U06%%_&LGUh9j1&BvVWDpsebI1B3wpGPft7Lvv>w~=No7?OR- z^oH$AtG13XXB@40nRZp1j`_3un>CxpscLsAxYWW$;n29S$Sp?h$&xs*mw3d2(+w}T zai0Y~O1hAH8^GrS161Od_`Pr7ahlxbF|r%x9Lbgzy=u9y^4e~>bINoML!X|5x_jcN z%sPy<)4zh=7L3nuGyX!-`MLvmSR!Au8xAW`PlW(XJj1QRmxooc#(=S)8sO)EUhQuf zVGm$q7;-zmh3Ohd`cRxst>QkTNJDh8q5>FKej0&N;6D&q&XJc36Of%X`93Lio6#S} zm3mPI*`^kaxqUuts&fLC^2w}hr#AEC&Swr{aQklYXp_P%y!_tCF5`#uQS=VBwvkRV z7i!(IwHuF)Tmhij6n~Pha?U!6`)fJ)`fimzj|o8BVw40AxTjfzgg`AU(`)poCIg@G z39j}i-9;t+V#C9g;H9WQk~fX!f3-*tbrNk!!%DL%t267HuXi{KJc)(2N0dx2S~30< zl7BN-i;0P;QI?7HN0;WRDLe|d5OH#BEsnZkpiU+J&Y|cYGl|@Tr1r4VVf}Q9<&zf7 zQ=E0QzAxs2)2TDMa9aJ5T{^jOyV3pk;p$XJ1=hGi?-Kv4$D@GalR`!LTYmE+ z1|^!g2o%k|_tT*j^O=rg!r!i~J%rLI9iS1C6^COvZ;uU%QzWF}#VzKS8v+A-m)nWdgTd=}C@)G}>62U*j>&zo!Ypu(SW4p964LN>bp!M~u)Gw?^@cA~4< z8Vf!k2DVs>5tLKq^(v#H6|WD%rM<-(Is?i}abu#tVQ{wAcre5YYq>qY(fu}GnT;_i4JSVpw7Y{D z67TEL7z!L!b|tH4(;V5TM19rw$85#(R3 z`QP(Luj4Bx?Xi5#g=M(EBC$xffXyOfMlDiK3>*TfJZ)vCC{+tOlSw7r>vL~RRg{py zW)YMtcbT4QExvYqr0$yq@I9rzf#tnYeQHA1K24dqy{VGZUXSk%6NOPjEi{c>2DzjU zWZ|^6)tE6d*;Wx=+vxwU`83eC^# zyh9UOh;WQ|4Dic2|No%E>ffP4qEQhz2eg(?zW*r@-vXgG3h&2xe7esTKC#ww`{SP5 zbCunv+)>t`drpPMPX*jsvFDKWbh~nSCxN*~6`Xd%!AGkZH5V7> z3ytGUGx2;0eD$}mU2hZjgw-AZAoY%8`$1hF^0EMy0d;>1*Tnp+e2q^lpA(HF8a*S z_d*-CP_{#Re!Z1+-(@N4G3KDFE4{RETv9O67n(o9%s=Jeu1shb^6$YG<4#JkcUV<( z+XuGh;QjOaoTZb^oV{uZCvHG3!oQPKj)@%UR2<7f-~k>uuZhhi@L(^5CZQHn6i9u5 z@8j=@ipUv{^7(QkcE*X9cpSH-JuE>e`Q`#1sO`@J8 zI((aMVv!oDD2m9kr5t~QG&+>5@O8F?)Fh z24`&=ij}taqa^#}o%jsAdh4qh5;edIYo-|EOLJ?}5Nv1k46LZc$I|A!5QU|h;N67w zp+zteWTs=d;@Du=CHgx2$^+hr*$DS*oN>AHyRngU0}a5WB6ORsdm6ow!EEGJ>nOFf zEunszs+qbq&FgQ(%7o?|Hu+-*ld<|-QlXpRG+X(&d-JudVWbc2la`$znqTDT)Z4Ko zN}+ip>I3BSgLA=Rw;XH_#s(NxjLpg`nQtD$u-1%&Z#^R z5}x_=o>yp5B1U(&TDDpSqTQ%Znu7}icnl?zReO?I12$}sC9-u5=Vp#PjEcwliQ9&B zu#WCb#f8nKPZD+MUEczbiwvkcKs&0%qwbd~+;vS9hR_2x@)p6B2G%4oK6l>7p_jRp zQv^1(P7r^ud4DHt&V?ymy4b#W+E&M+PqA{TnUl@=ExSGXPy8!vw=6`D@gpU{w0Jvj z;KJd;{d!zXww&`X{kbRK2ov%Yl?eU~#tJN%Bt?%ic$m8Vj=J3tfWEZD&RTElzz-Z2 z_*j9X7uyD~?fLqPQQP;w_}&lJHFzYL%S4|@wcd(Tkg?N+CpItL5gz9>NDalJp(51s z{Xx_=bE_tQ{P8`WzVf`OU-Z<*W~$6O-wE9)XH*x4lgQYU&&te}wc%TCCuzmK_diJ> zTm0R*yMLbbeHhu#V?J*(h6)`gof3-2_~eD_zVu5D!z%MBI%a z*))Bp&a28fAt=bpB1}ks{QEpo>;So@w$3HBpiBuUnb}Vc)clb35iSHK*z;&hsk$hQ zkc_1!AiEekWPeUM8e@RSv@6*+88I|J8gWl)^n~iD^%?I;c+fZ?N85){Ps;a3`4qGr z`eRe!1oAFv(7M*$NCQ_^2d{R^f}f5(ugwFUCY;YVbunJ=kJZvdZejEuExPT9Cxz*S z+^1^};~ANg9?PN_lJnc3EEaP)fgHH!8^5Pc`+1jcjKes5M+AAS?~dI+83r~hHFAa} zX^7P`T;iXC^r`&x z&uI*8M9i%Jo+`^?d$>=VBH_e-%Gnmhdd~((KlJ1!f5U_U*xx=!J`hb))o_(onXRkQ zKR%M*gsAsW(PCD>rq3egU@}{e!Pr*b59nTF^cHb9p?>LGjq=E$(9mmcBc6QcsO`~* zwu;-IN}$apP$y{~#Z(&NHojnD)h1?MD6_oWdKA$_F{Z4p^(teBzNznhSa6xKhZG9TR@{mHA+Z^7kPvfuyu#eKvRq%A z0iD71H8cSso1=YZ^G8fYBYCo+rmGGaO1^j#9+$$7Rz@6|Vn-pAkJCB^j8BveIMB-*$ce?*wRis3`!d5CvnFhfl=|aZy zn-=ahAxm~&hwDmX0cAZmJfI_x08EY4h3wcy^EvnDL{!TzppWGLv0eSXb2UFl6%lOy zUUW44%cU>HO6E^tKC_Ao#O17dIJ*{K3P?0pzx9PwGMalpNtrIrPNkKDi^&Ho zc8{2ktvNX|vo4>pYL;cl?gyIIrun+Fkr`e9j>E z;1;8}nGCxHZtdx;LgQ%KS(PMQ|Gi~*!8QM{UAe1n_hSQL1f~v|ToG(`{BEQBteL@0 z*@SF(+4tQ!&G?t_z0t zS$wqA6r3B!KCU3gq&hKC8!#paOCI12-c>x6%CCloRws!p9-r&S^Ts@_h8tD-qj;-> z@AJWx%*!V}<$myUuk`G0gTdLu8o=#-6aMg{a}D6BiGXkz4IMMTdI%LY=-(e^IGIl$ zZ7DWf=B#PL{*3tQ_HF7GpI_0fk9VB#Qf@)Rf4M}8YA3D^=*X-T zpp}>05u_jJe~}BVXzAR> zD2^ZYAK-xnP5Eu9tVH<^!1VG}2VS|OU`q4H)#kY6_$n9LB@u&B2q>4D18LgB@7n*0 z<;ORfdXQ{yd>MO&KJu|yBp|v0S%|I}>c#{w7~9%ooxS~J)~xhSoXcuOwx%CORgDw4c{9yr(qYrKn>GbrwgG4Fi;FEUHQC?3yjIlj z2RvW8V036aQ<21ji)5HJ$$jb9^NC%tT-uje`AdZ0^vW zRTVC2T87kke`<^?e&;cT9egK`?=~sWaFeU@GBlU;o>(YbT0A58*W@`zhFD&mWWy1o zX-}gE=ByP8CE{I3l&@>|hQsuO#thQo)62(8n>u1=XTflm<)qqF!%?Gt_Ca`@qaqX2 z0Iu1?2W~NnQ(0WESz309k?Cnf$}he=ULO{*PTJJ+5$(qR=7AwxyZ?Lo)S>Il|ifz@3 zRX*zt_wPfAdF*;5=5j<2i@!LM`l`2VdDO8rj7d#^c(>qLMTV!4Akz*(&_;C9z%HPF z3Oq~*zENr4pTT#gYx=kl;y7Dog{9YuHXHj>Ob=IeYD%9(HO$y;@$ziMQ=jMFUAGE6 z_|`hk&VDY;t5!CDd#VR3zFL8B`+n7J{%AsXT}#58T9do`xxrgU@vn;wV`I79Y-{rt zyQUaQ8dWm!F?@ywqa%!)yLLGv$uS@GYm`wnKH6T*(A^;g0h3q-%*5EdK!?HP`o|aD zX)SjZ8b6=JBs;7c2q0IV>%1;K8@Q~()V{@sx8Zi*%@ycCMwPeX zleZ-$cRUe1p;#XWQ?Y7JZc{}lS8Cg;N>`c{zsF~grm7S@@f@Yh7OJR+cQ%T(TF)F< zQJ!>+j+rZ-EMfsS*;}34=0lYBS5LMVw(|=91$Vkic7%2XeaX?t{8fNddg4MWD@q=v*{Z<%Cd!$4+I_PE}vG zaU)w{U%hj!ak|AQ$}FIiufvKVMtrqwa_uA5#l%3h^t!Pw*aPik)r?MOubv`UCLWtE zJ@p#vw;3gxB~^u9sc>5HTCkY4MyStgy*%vMGMe#OJlY%uVw=oP^8xwv8D;zbkJ?`Gdh&C#{^$bPx?@gnoD!I95Tp?i%Tk=tlL%kIuiO{hR@Uu{=XUj-LD z;I38NcIF`Wyo&N|Wp+`leeZFIV@bW3;T>oHo^?MDO}i66Z?;znQYkm#Je|>-3dErA zN9fNrehxhrzZ(?cnUO>VnP5;JqHz+}6^ac=Xs#7i3y?(eE{_QZ6c0T`IG^jt^tu)} z`=+Z5u{;{5_a z%%rl}S52KyOuvQ+dUjFP-5W$mn=lf@^ zS#fal{Y(qzPi9Q500khv8o;+azwh?7%Sd$*)4N3DN95aGEV%0w`~hmAv^c$owHdt7<3oF;)pb5jVHqq`F0@(3&1t=I}wt$q@0K74r-4gp)&2 zo7K~DT6lBc)vq`rUujBC(5l9-S3|R-mBnJLjvaVVdkaLfUt@*`v7*g2VmAZoebK}@mkEAi;EW@3Cmn|Hwg@aa)dr28d&?#k$)Ifo`xOV;yW*p>gHN+< zD>yUs>|7Z&^g>!%7)*V78r=rVI9i{A>rb$!flXJ+w;k3}tDRW;G_8|lhbJ;{z54yJ z2&j+Qlpo^bT6wavoGNDYn$@--dqGSWd(VO0g(>;%4^KxFEb7M{))vcyObc%qVCT!g zb@o=eO~~xAUD}?GY&1dgcDWsHGS}XH+QZlph4POX`xk~kOeb>BHUq* z*1GsIZe{p9y*wYwBWEnAx|y_y>|-A>LInEgIx=;X*_rY%DJzS5Pulj$e^-+YFy(Gm zK<>!Tt-qM-K~JAl^~BvR>oV?XMfW$V>NU(fNqM`l>!z-NEHP09KbebBz@*{6f3a(4 z6rA?jQdFGY?B-}kP3j_$100AWP|`MHw){9MJQ`*y)U-`>nyu9lHRVA#mYJ^k!Py65 znVN$Moc-`1k0n7;%pSdvd%zwPn*78z2f2RzBY-){sUIXYlhL{4IUMD zYLxdW*m@ODB^eA_sBV4ew6nu=6d%E_uFsCyN9TIn>b=%5LQTB%8!MVmRnMpDr+Iq; z@X!lKJJHPz-?G$_4o+^*7Kt}Y9G>*4#~EBj*&LrQYwrkl^}hSU$11RT1-uTaZ^ao( zO>b_-7n6!VZDyf;PJk- zAuW?r2*7vczv7y;>Q9B}pGFdv0;(ef^y{o%I})Zz-#^!^qQVL=)Be~;BbyeLU%Cxu zJ!9j?;MmE$M2+<*i^Kk#?S-vh_{LO6f!MsHik)yV$+xxTXPiJmq`61I9|YmR8wG3M8Jbgsq6*&IIk z5X0D3lwL2qyKIm!o0mrvXu?1z+t#4LUZ!%3nPd1Dc?K^!>P8t8(2;W7f$rQJsN{AT ztp=5}y+3=d_Ilvf>I=h>>iP#w4*9m`GQk0; zW?N1jpO4Er1}JBfQgfbagqV0ln|9Bllo<3g+?0?G-o1XVsS`9LV$kacW$2S@yeTUe zTzl%0#cdI{3N9ZKte6jYyU32>dn6GvT9bF`bw#{FklY=oi;1UIm-LPLT~UFU;b9`D zQgXz6khi&=KEIDU1~Tidlc?CDr3@F-v+O4J zyWwBf-DGb47#Z~d+~uItHj+!b z<+{z6+JNLH!77|xg+d3;6u+!6@G2b=B-II+SAOx(>Q4}_3jE@}_SpJ0_?_w1I@`@v zBk(4+rgmx){!LkdH*LLhgGZC#WE+IIag)*;N9%3+xVtWyX>;4T92cX z@dLxt^h;h_%8K6N7VN&?cFtZF%Jl#MF2@;W@*7U^{VT2Z^d;Kw33i_Q_vj4&B)3Z$ zNqly2PW-72yG2$Jvd$`h#nibs{B@FHZ>{0?1&QD{1``U^*2ekFymF|rujmEB|H+&C z?^%`1DG6I^E1(1COkyX!rmjnst9Zr?O;(s~q(`mg`d9fxRTACM!Q1|eH|eM9P2JJm zAisgT+YMItUEqXjz5FT_tI}dE=AN}{@MfyA9TNu(6J;tFEo7@ zUOn&|GS-s7a?Ig5sWeEK4XtE1JRU(nU`9HhIVab{oOjP zM{^zIOlbA*c>4FuPhmee3Uw69`I+A@9z5;XV{SmKN_yQAJ8$8Es3@(;EwESl`l@HH zptM&M&<9OSK( z{eA~_r+Nxg+Fu8R*L9All_W7Hy$sn)n=a0#Xg{5{$42@waGkSDsG7G$iBmj?GfvfN z7RFx9^+k;7b4~GqeVbtcTCmj;e2;%;g5kFkhtjAk!SywMY5RvE0dj zx87H2Sz5PNvJ^w~|0e1-h8E|+pxl9%r#~on*h`AE8IWfTzAbUYHl9ZXxjH;BH>P&E zIiaR@Jw6mj-L^+28hoadawo7baATEmj%=afK%M)|ME<7jtpl@CT2Q5ro8^paXTZ&V zPX^Ynl_6jFf;tEh2aYta#U2yGOLG$E&$nOad2D ze33nQM(@0bwkG@3DFPtu;O z5QEqDEZZ0UjIpaW*?YHfk|_4Q*=dQd+L7(18cD zMkWT-Z=w#&0R*e4Zu$;v!mDSb+kc>^Z6bgmKYBW`S&7#*aQ8c5GRX^~GwUlh%R}GG z@H?AlqUB;z+jZ4q_2rFFJPPFFssByB%{I>{fDQBJ{YxUnl@X>HVB-M!*GXRB3K3u1 zxu&q!ZMbaBD03ItNq#$8G;3t9+$W(rEKW(NOkIon0?G~Om28=_`!t0M+xGPOo|;c9XW94wd&1L zugos)7_VYwROW%#y@UH3u)zQWnx5Ogy(t4T<2ZXxZi{Q`(TafgPkWn5J$fe~XEk&D zxT_cNr(Q_QCmm7FOYzMf{5BjQ8VB)bPe*I75rwpcEIYJ6h}~pOlsY~h1n#H$KsNRjstTPeejx)T><4TTe@<-A<-GEzfgR2#mpvDK(Z^A!&?^qkeoWGLHk4p!?Sg% zswY<=wtH(NoH!pmRZ}xxx4biF3m2pYd5Au1p44@4q`6go64s zb6!Ch$*YWc!2_asg$lcUq%Mg3NKe|dj~_q6VZjtusLsbA-Lo(^>9hRP(tj5Dd9y4w zkq{LI^26Rd4v`6Ql!$&vUM}hf7f34#6vl|sybCF5lRij6`)39pR2rWmYf1d-S^GhK zdU>}Voew3eFQBTzVXxrAKa8Y=SR z7Qm2CAY)orl70>yd3iJ6*hA7Se)7A>I=qMBL-wCfuu7002|z{~P}-bfp&~yHm{BBT z`Fx}md6lTM*xw{Sf(qhKCq239t#Xou_Sd-a_d@oY0+nrP=^2_26qL(DqiC!S`-`iM ztPM~>!Zq!wS1J1j5P3N(=`J&qSJ|-#kek%!sFlZJLl(#)(3@DG0=1c{6ZiMNeyQN| zouG_Qpt=zUMoGvR+4=adGeI;%56<}O=AQoLRR2hU`A>oG35_dsAZwDmm|Mct_uThu zGzWRX@DDbF`XqO-C^!XHK&LD+NPo127br)N+sEgNEnF>?DVi6^%1@o2b_2Y{O@2%k zI|wioD=u2{ybp53zRFmXmF%jR`LFMT*yauAUpPh?rKDdfC@U-f2}=E-hy2VGKnh?X z|DlJTv}+Vt&3*s~?l3HH_2w{>6EZVDb1o=luhB(r)aMIXx2dX)ixo9aQc&8V2K5TK zGm|oiiu{Zl&Fc55i$OJ%348btW*9(W7uOE}UO8niy~O-iEN1iJ2 zUq3*7w8bmsG6fvXqLV&D-uOb)RcHXZT+bUvk#&g;a&vKUxq`k>Q0`$655Nk~M^gf2 z4S{zdf8ZNH$cQoyJxNDiWrwPO=$AyryP~DOLEc)l)EO3bdULOkRm8D-5WV}0a=VJzFtH4ECSC!W{kB}c2Zku!CQ_F8?=|5W9s8zoNsu{2wTqQ4NOO32O_g!!Q zUAf?Iz+JF_*m4pAu&DH(Dc;{B{u>&3k1m8Bv}D>x-rXEdSTfES>*ny2l^_2z(2iQb zVe(_HGf<0W{Rw%LHYsMOC$E3zHxTuT;lelUf%-L19b$K)PvCY{+wxg@V%doA0H52Y9I| z%(!Eb^)AGadWtlB8bn^+8bD42qz4ps@gGhq&Gl!TG;f%bA}4+GDCFRbV(&`|IC`BX z?Ff0}Q_tcB^?BEu#a_{%5O=9L><=?CGRi}~Q&8>!NDTX+SBfSDQ09LLoD87s8-S&l zYKmTDp&{>&`%Qr8r$fcGCBW}bR%J^9S+t%(^EXeCHM~RR>Ur-%fHdNfF8|NPx}P%N z&zG`TvOzVxx6T|Sdto|_ilNi@Eh*H2Xm(7qPEb!Fr53ixi~0P?!9Dl&o1;L~={8E$ zE24;v0X}*^E&Ly>{qKj^gPs>@--c0ew@T)>`&~s}-=M(2uTT@9DhJ5Qm>I~L*$4lG zJf;;JMpDo}-DmZ^Qj!Dn6UrzRZGV`FiRn*Js(imIrwpA21@}JY`5#`h-|WIJg&jkH z@H`5t+mAU-p=j9=cAAljO2*`>#)aD&`Bble3R3^$)L#EN5L|dTE}Vh__lXZasGi&x zOGUvP2;x8Lf+*7cBh=)5@1!<^J&-C;*wGJuD#sQ5XZ&<{@eoCR`uT!7tt*f14GMYw zY0$NU+x{AwXy7$&>x*>NxT<0*J50u%M`$Ljj3#t zIql2aRJX`GBo#!F4k$AM=TXA0v{FzSgk$g6CPa&h!Z;$wk8Bo8l8&HYrYyO%?|@W+ z!Y+RBQ=lF4XZ#dshfw6FOZ-`&n!^>RDdhP_AVL(YrD1!e0GI&lji1@Tza7uAg*NTd zrAzzt_yZ}J^M$M&T2G$Wlt4M$K9qfsvTp#El6)ibIt_WifV^i1ZiiuQc|@QK&m@N| zkR3XEYBOl<5qh%l^XS~*8PP10;!%FK@Ci`Fb^{bxgTo1ci2nhR{QtTaEU?!h`XiJAwyMbpVxi#P`^^9&1$fN`BW5+S zRz>5mm#9{Tl)P6yNP(l>5DB4Ju;Kx_Ca#4e1(!}ys4xMCqJf(JR*L`?1?7?$eq^(F zFxO=Y5E;JL5P5rm!Y+Pr(rC?`KjNf7+@j1$y_f-#5q9?og*-n(J%NJK=zy$kuM5ob zltP*Q{LGIoFtsqVDtYBD@edAx>JAT60%Z;9iwyRZeFLzRo;R0IKP0cR7E7SXW=TKu zKA1*>{NaY6y{_VbX$1Ke{{{B{G0N(lZKjo(Ea4oX!V&J5WP24Tj$9y5NJ+} zQvFF+Hj7k&q`(>~nE~?s2jo$`V#Fbi(l)v1q?I!C7(XymFn9o^So*$v`SM37H42|8j)HQ%m=^#}6?UHzD4zsQzr3UD8-NAad_H-ELPOE= zEzo3pOY_}Q4fK+1N|&X&*HyF=yL^$HSGMpocml@&4GU#DX{^Z3x7R4*Vgb7C5{VRQ z1?tkrqvQ9>{!gXm-^4m=fR>B3B}ny=GX*gVfcD%sPR^V>P=%PpgG}4^(S4~LDIMf1-CHXBj0~O9<#RUzAa5 zFnk_(&D$TLlp~OBJW{88SOzfCXie)sIxKV05@_z6kG)6%Sb#82u}Jl!ed1+wS9oyWx~At{(P&1M65Ou7g8 z3I((8?H@<_C{&H#{NE#eKg;p|9_jm8B7f7~|GyvU6Pot5UIZ00daAs>MV|K^Do`&R ziD(C2GpM*%CUM|*{@tGcBXO5BG%YFk(UtjsOT4svv34-YL$`I%3NdnK~ zdh%?M3bNp-2$cS~D3lr00UjucJqt8`6DT!)pMl0f{--%W!6F@^)l;mCcwUI;nxu6r}R@*MzxHHofpZ*~xQ;CQg-iyhaQPmrZ@( zUT&+Euyza54s7hNFrL|O;$T1JgyJ{ibqg=$7hfgJ3h-@rO8N7_uj0CmnpjO-8oeI8 zeZSxB%VUoA{JcyqJh~a|Q}MeyPyZV?Qu12{Zzb*$+U1&91Z%nZNVpE`{M@9pFSDs* zz2HgbdM9qbuW@dG2-=(QD16i@3n_-+m9h>)tpC=yT?f0#1Bs7~syr@Y&AaP0BqH9L zun8;eE_bP0d|hx+u{2}k)3$0M1Y<6BxAFT2h7QH$B8=x$nj=|uG$;(xALnYhyZxdI zQS^oa;#gd1sGD*=dqM#`kKa|U zMg}7YC?A2?7YT9n#H+x=N7$}yKz*kDB%CJRDb#-MRP`&?y@MqlAD3UcEL0hQ?ZLt& zMypV&aHwD3v^56fyt3WU+trCbrm~(cRk7SI?@v%I#!ga6@j$;705-v`fQU z`x5qbK;yK>{1&P?HKRG8H=WpPr#KmpTnJLh>llb6^s8W}QilD~;hW|7mQ9$2T0J~- zbJD|Eya_kEq;tX=zEWI0&6RBCeYdhljkp%OG|uS=p8OqNK#1(e(qPq4N22W{vS_#O z5Qh2yH*-C@DIl?RrP0u^k=LVFVr@|$4BU)a;?ssyBW>StY237}7y2&OyxoTbq-UNX z!j=QP=mNo3!kww;<*pJ-?;*)Tm+hXk)eFB#cRA<9gt{6!fVQ zswQUwJ5n0#^|e`nQ;o2wKW9+tZ?N)iSkK_565Hh2;&-18g*9Ot(Z%tysc+05l(o$Q za|V5N@^axHi z_g~~x?U4%re5I4{4bRbxYm39CS{FHu9F8Rl!;*smKp7spF}~@Sl>Kj^K&Nrn2eC(c|Oano0QPLUD<&97p{-prlQcG=T$d`zLS`eIhe5Tyu zI_{Yscdq&CD<&@S3xc>+Z6_~;=#@hR#x+an+fD_b8`pC92ycwT>ad#$3dkv5{2U*t zX2;CpUfkvRt-+2}sOSiXsq6bYHi-tj)x}WNPNcX=>j287@2HXKjpC2Q4;)QzcV1re zK6;EG_Zm9uUHZE`wPg%+DU~}tc^RGQ(XTL))(r1PnEAjf3tao|nG3Oc+*`*6An@iZ zwaCm7{FEGwzTescKLIm@_N}PheVo!2rRxA3SZeD~-6|8w9=ev^w~|t(+T#V5qW^66 z0=HQT4hmd#w1_OOnh=?Xf{A5^Gya-n0^>dwMBqH2c+-&*g@pI1o#|%rf`w;Fn|79b zCnu*3GsF|>x5e42)wa9zPbMs;KJTaTFUg+b8CO**Oq7DC1n=Td9a6i!5rewB-*&q1 zr2qE7-om(%6!D-% z9WA|N$B`za3i|2C!%9WkINSJ_FLs08Gnk(*w-A!AXk5D^a28xW?Y|uNRZpmPy+Syi zs=u`tO5Din?Cx$r7ST)tc3;>!!52&lOBE8g^KwZI_e0>PjRbSc>zt!yM#FW)8$GM& zXLN}TD6i5;@!++{GEy+Uw*L2XgM&M(EDkmirgc zzyYMNO~hu_{i$Hm8!_n9@uGWu zP{1KQDp787ZgQhly75Slk!$9`b+_Z}OaVL`(E89#<4_&#ZEjr@Ct4z2GxKoF4jnW* z=)!AUX^0|#)P)$OAek1noo;t4i5hZRnhh&v?#_&OM;F?T@qJ7g*HkRQAuSgwl%1aFC|aoZP5JDmE?198xA$$b7= zh7GpvYG7ga-BlNGVL<{W$eXuQT=Od;W%uUQ;0vdqc}nPpily_<-0TGL@j7Chjo^Sf zBy=PwXsZHR%(?{Md|$Dy+B{f3pXR7(7-JU&4|VpI8uJ{!W_Ldw?1+ivOg44D>NURm z@(OXaQ-Sc!tvQN~yS)Njn63P7S7wqz`79oe4))o+Tjkv^e%KUXHX{??#<^6R zxf29PN%P5Es({}cqP0lEB;vQgugQiJSHMEeJ_USg7B=XrarUwkN?8x|sPlI9Dig**b*b#iw-2b{E+l64Ml2LL z^u}Q`t`>2pru4Lq=c#Vyi%_T0=j*oiJDUh~OmFx57}9^&J#AO6poQk(bXP5ZsF>g} z-lULF1V6apfXpO)l^GT?QA0{49<6N@Xtsamc-gL8y5p&E<{d%dfWqqmSZD$)^;NpE zpSuiSDy|3lM(#{%SFcOZvWWmmvZ|;69*F5H?Ea$bPnwr&&Swp4>{?Q&^)@_c_fY!m z2dOW~S3mE`b}7lt7-;KlFG3Y!Yq^3<+f!C2rRH~!VnLdGddRsORI_<8*Fo`fmCu;zTp+rSbe~y=AN}hx&6SEV>1@mBgo8KKQJA|32X9F! ztXR`B?g=vCNq}8{Z>rQldTK_G656K#gMrKPWR3U-i(u$<} ze0qg$*{t6Cv1RyMMFj=U$;!lUTX0pDTHkG|p+gYIOC2|ilar2raTzHv_%d%=>u%$= zJX&l8yfCa{1mg}Wv5tZ4xVp^VcGBJ#?Hkdeb6vb-&|yXs2(-R6)+~CQ*qZb$juTv8 zpjiaJ{(J^GY%+LTk~=f1Ix1YJc~yp4Usy=!r8!Nr=dk+<%^YF24pAs(A9{xF9 zXGh`v+Oz$cMf4j|8s;~_QnB`ZJ%Thrk2!?BQB&i$xR%G>^r&sD6&jUlM)-zWawDG5 zoZ_dZzoHIeeHQYIkB_cVMHl3#ZNO-v>?KrGRKgCOiFf0PPm&wHAaE%0k<(a}D05*q zN~KBMtnQst!i1#8F&cL6M6WWZ$tu^ENmzqw-X7+RD-x#W$!Hrb*Ws#j!jlUQE~@^Qik+4PDN=%HpWojWr5VtIoF#w-0EYMsmbCBAS9u?8dWJbG=g0 zEczH7S)b-DndJKVME@qsw)0dC3(6q9^;%Hardu5HUABNCj5fyFG(%V~J$&{{b}j4# z@X&hs-)8E&_Q4uGU4hpex?0^5{;(356DWT5J(nR}NqCL? zJ-_8pKklaIL$RC{@;OstO?E1G`-PE5V3LanmhM{dx4f_uUQ%Bs`W*`zL=ips)5(>U zXG;Y}2hiL)x1=RrXM|6V>(RiEm|xv)up7u^4!WuoEtO$?@yL9N+rEV0G*uIh?N8J; zbi?q6s!y&EacgrauF1!Lt+KzrJtRxL5m{Ssi!bIt8Pf&PeQR>Vssgx~tRgnZ_!Fl| z(EEq)-qqk;lV)4a)$g0E>XQt8e}IX%HzvBryN zEDmNlO$u0=*SZVdJd!7M^snzNkbNiO<4f4fp8F-SMX~8lm`)_!Ph8%DbiVUlo3&a> z$Um*w&}vW*yxzNR1YfrTIBOg562O;w8g{-#X7#f3S7p+zw&g6qxA;ACZ?T<_YHO&H z>{m;ljNuDI^}Ud@hHfOLrluMXB}sogepI+&1j3G(nBHZOj?1hE&^!pxy=pHC(Y2+MQGhq2IK&DJB$*3EEz<(k8# zX?!xre)N0QJk-I-mOOWF*F5RVM*HPx+(0AMT878f`qOiYFU@KKYAtT>vy+~)5V#{1 zECE^O=53g1Azm@NR$ybbxinI->7G84tZ$XuuXb(qp}13?*_6_!#GW2ux_56wIbciK zm%?TP*sM{IbS-o5mZ6@?3-v5sSL6EbEWGkGudu!L+>LZ9L-C}9n49xNL)LZ1iEzgk zO&64|o`s9LdJpfv89gk5e0@{}9h6(@*e?2%&mYxM0(tMj(DDy`>3&!z`Q2}IC^9t)-O+*r6__*C8W zV$H?DzT2&Iy5W}BAj8b>>98|-a;l7Zk{x4?iip?#0eAaxdsfX@#v8}7CjEHlo`y;e zI+h&(-?NALRJm7koT6Zp#fNUlWK@*7BcTPiJb|}So_KIVL*U|v!je+{=U+N&>w*o7S`P+s}I`Zx1 zE9s>k3DTeJ`qQ7P#tw#scQeq*BUtvaG_$mgcQDa^QnA@+AWG*<<}{*OlgLJ1*_dh2A88`y6T6vuVC6 zUIhCr7P>T?r{A$fSKM$uev(hw_k8+P0~JUdMf+SvLI}vNfhvPN10l-0Oa&{e%F^ow z#$>oX#t>WOhZGDws~}Y#rI#D>Tv}orUvY4Je-d*kBfzCDm*rWlmOI}-vfdlSFY);w z-CLfg%j!Dh#T{DObMc~Dnw~KXxB=~P3ar?<_1aWgI0qeXBh6vu&yE_I+pRfTu@MX$ zVj4T+<;)TLSi)8STt(jQ7<(IqPkZ?_{X-h(RLMK~q1Uh$#&F9R%P8DK2Bx+B)KYW1 z+n6}R3Do>JlbheLLh3`sK-jW*EPN$JK-?NRPw2AXJH^c z*`fKB869fmn-jiMa$W$k^~#+@!q)gDZ?lyMq-)2|74TV03R|b z-2;yj>;r#^%T4H{j{BkwsR6A+Ke7rPU$w<JIN`TeNrEq*b<4n?ehZq7I@Kp#GAAT#pd}ha_eOLHCHCaNqYe?7GfeXAESZQf=v0c&lZ+Io$CRqpJ;WmNsK zR=42ZB@ioak=0Un3lL{Vr1(@$k9hjM!dm3HyXB|s2+w~Nmj931)#(m-O}UqHyezKi zWY*`@kN;fzN~VA9dR|_hb;FIDvCiY=f~|$8ZP_ZSs^;hC^Y^a! zJL2)r4zr?L9x8-PdOM_&SK!}2*hr?jn;gcWxakgl@xbZtZ^>V0P{T5%%%AZORs$^e zSRhC-_G2!7+9f|@{2f4^mPc0Tzlia}-BtJqT{t=cB-Ap8`p~bPG?zf==>*J5~p=yCd5$w?>{O&g4 z?fH@AcBBimxo5z{^z1i3XmcQOeY*!Kj-vnehRCzF-9sXEtIlAm2N2%^Xc0lgt?d`V z*9rAs&`eECogCA z>@YpmJyYFP-Ss@x8~ROF3;_-s4gdfkNQeu|0|3xI001N*%m>gFXk~$V002zbOi1XP zgpd&7H#-|+GfN`?fHKxVR~J>{3+0fWp04iDI1Lq?owIyMNQAtu_efVaVNcgE;h1h* zil*i!Ch{hPVh4|5k&OdSy_O?-F6lc;9~!cHb}aFC zu``O!S?WiE2E+A!W(V?YM|nn zP<@TRb^vM|<6`<|Vi&);3-}E7&kc$Zbq%df{r2~x2ADMB9?raHmt!v_P}96}7iFDI}Ny;(=_@P5CL z^^ceMZNPe;`3(j1zZ0*5MQo^s%|t{JTArW5!cU=OX-rqYL-{0Sz zu^=FBXmi$W034D!TG($+p&+#?8>vbd%g6wzLFX_4uwXL)B(SjneHnwFB~BuA-A2OF{iw+=zoiY{^KDwb#Sodq^EawcBXS?qO-9x zp=aRW;GqA?NYBVf3%Y~W-qqSc&xO|7p5(tO`JZ}(jqDBV%xoRZY^(|Y)T^g&G)tPTYoApWX*Sfd5l#Ho{#c%x7T%5n+)daN^%k!fqGsB$Tv)vlng7 zsYbP@)~r(CupI}ktX%d7^BZI5`>D*Zw&Fi)xkl_Vza8 z#ux_)Dk0cEJfa~51qJg(!uR87He^sSQ86jQG3W~(PEatAVgRuJ=-}Tc1Bh+(hbM=~ zwEICI5%7Mwy1GhrZpgs;XY!zCd{n?;VPWlVhxA}q8*Q>l$;sWy$NWk8!T;frnhb#4 z^nAXT()IAj0BP1_J=Mz>aY;#398d3= z1K}l0R#*gt#I7zvSU9*iAD^zfM68(Fhu~CDO05*!kM-LsLpq6HD*fd0udfy~`Ab-aKcxxX{ zfB)SoHbu#NNK={lU_g3#oDP!#NmQ`;R#i=%@IXa@l=QcegC*(%UX^VM0TDVO=U^!c zzL3bTt2^b0m+c?_*S_HwMQtikErRj~FAxzASw%(tvS;P=-N;1n=L;4F{m1R?T^D`% zw0&*DU*ffP0`ir@K8j-my35vlsb9SD#FIo?QTO_qnH?!YV~M8|{YV=uefrRAJ~j{y z#*tb&*BGuOQ3{yM%xBwMrJt2daX9x|p%VI9h!awVfM2rojpzPVITZ|Gol0Za)IRIU zw3;pGLo~ii%oyhdRV9fA^pb;zrl}s=brI1~u?VoRytKt`TYw`iHu09(QF6T{mNq+= zDl-*Ya&1n{ki=qdkx}PTRHL0~!|bPUn$MI6*ie$vBua@SJ@uW)Gjy!vvic{3L;Xev zqz5&mup9}ju}{~VPQFjqyp5oztO57+0k~b51m5&<#QjaH4`PHxwp+xNq9P&%#Vp+3 z1kpp}Jd&T1d{&!91Qo>)zX^Pzzx#j~5)wnbm5YjtM}wmT=!iMhstY`@*$`ztnq!u? ze6$PcYMno(UnFn9)srvRje+bN945cpjBGjdY`J{lSs@nJSKmh)F6HV*j#s+%z0A9b zKi!J^wn#1k%Ya$}ZmUcE+KfZXttn!|NBi|Fzq|l($Ly883oD85JGbxRV#+%=1o&S) zYtt2saFOguGW6FT-rgG(yCPK@>T}MM)TQ0-@-MU$OJoYb{-xi~3r>aH;SaI;SoxzCf5QGD&d z1IMp?h~!5easGn&mVw9eO)jv*nG}`Nt#a8P=fcpPC=mRpj4N z4BIvIXwk!~u}!~TJ~S3Gk6i>&1QWq|Cn;PCIOU#7T=`eAfi|*t<&<+hJaJV}P!z4Z zEYv2=^UzS7{+@MG$1Aktqg?^%;%Cdc^V;c9yC=am(i0Ck0=_S3yzl&o7wQK<=8*(i z+F(TdIQH?ujP=hy@i*Gtfk~lg-Z9}AueRB;7qHE@DxKO|u!{6|=rqbLxWk;SD93v5 zn-IWJ44tUagos+iVGFjI?nwkD|MvF@jRos(QNn+76)|KmvlSY0Zg(9jo4F^J$bMJm zYlY<=?`w@U#&bFo!LQ5rG+4&28ZkTv^pMJ<5epRxV5}_kRmB!_SC`rhonh6Ld{8p@*8-dI7 zLlP11U8CWghdLdIo9G`bH?(CyQNTZqdOf5dq?oTfVfHm3690McpvL-4hjH&fqUN{~ zoe{X8L#o4jH!mwTn#B_g@A5=7oXMq<=9c;XTv?;{YJQ{MX`MLsY}LU1*k`~Cyt-ep z5wF%4Q7GSNyvVzLFnEeYCs=NE;NHVxSK2cE`;xSt4F0$VVU-%yLTd%Bb{2~ zhw%^AE*Y@=FP`~woiu$q2b{=3-+$T7nNRe4Sv`{ho>@?Fu)x6t1nkZQGcwwdX<*}I zoU%fa&%}1fm`OFQN6@I6Nk~Z5?HLK)58BKZ%+P%J9$oThE3~AvJMi@8gf-aO|lR8J0OF;Z~Fz)dy$qrlOT_?WEU&448xImes5>|7~CT-Q6Q?16OBjWLeNv+*H(Bp0{xzKX92<4czZPf))?!~$-z;w}C zwL-JmaCtTxa=nF}_w+HhaP>~k|HGZpmY-`Oew#3SC-5W2ZeM7;=nLL`&nTnO z2=mcWZE)Tw@9|yBX|zaGM;HdZag^Q*zzA49aaSuRmdYsS_yc*HMp$SJ&%X3~UW=JO zU)FNs7d1_YV6$vu_fs zWSjQa#l$-cxD3ldrNyF&4g5C@tBL`_%EG0$*|r*q>FH(JL`5lSN6!U_Kb~?)2Re>S znMXhJ@?-AjTE80_?ZpnoAkL!nw7|1@&hHHO&Bb4j4+oDij8ZD+85|W(zT!nwkdsU3 z>gKq;1dS{eK}yVKbcV>{;Uts3iw+KNdH>u!Or;Esl$$d8)uC)*K3B3NvVhe&RueY% z{Pqe>23i>&&zBqA*P3bXtJj;41e1u4a9y;~Xtg@TeQ2CbUeNJYSogTWE7oj|<9?a) zFFX%`)ftt6f*Y#RqaT)w|Am=*->{-yc*rLfh0AEZ3xm!D97dDlrG0;UuJm~9Km6p_ zb|#&|J`K}qG@29(4s|Wn>E*e5FsxuSm17dQaDC}~`lz_Y0ROxjKe=m_+1S`pq*9^D zu~h3G4exr-5gHyHKPl$@a}}bk!Tn-wD=sC4(s^x8gC`MVq;?XK(a$VT6k#uP@M$DJ z+l%#Pq;mUZbcTK z&2f&kd4s9SPw7`0bvG8J$r6w)q0eC zm=3dTyjJkhZvRr^uEMdt_Sj6bMp#1ZYC$JD`J&@_{q*7+irvWUk^W5EgzNRb(#0y8 ze7!k3Nyldzb;S?xxTGI3=#rM~y%Nt>8gMSO#Kc;rn|_egMs*7XNyddOO%~a5*$*hj zw|*8hGHmpRJnJFAk`{FST*4yHILM|4KuMmA?Gxnxp z(8(xRzx3l#p<2-OReU$*YNwOM9`kk#V7A^?tOuif*b3wUj)JBFqpLVFyNh<9c!TBE zb_?%;=3wLc@(@fna2@D#b*kOw5?c0RpIX=P%Bt4nQk@D0XHZ+wCKYklr-LYH{|qfpZ@Wwh0UG|6q}*b&?40Wt}qU(V6YZK-ceZf4W3_q2m@q z)DL$4*NA%c5>hhZOR9b(kmoi8eqpF$!l^TfLqj5&bz;zC>0S!JTcKUHlQ-9U-}U`S znXnSFM5UM?KFDm*#N|zpP!w-gW?VJ{?a>-NV6pahfp%Mf5z-81$Nkqn7zBh#9hkm9 zZb)rPEjDBH>p4}ZSw699z`!t~3Ik@LcEjaM;6pfPkvlDdUIa;Pu%w5kUj`Qe?GP+( zb_`)RwcQ}aGU;*K{s+V*D%kFKNnOFJE|5egktH6hsqKzm z!~km&41_CO3PQk%ynOCYnLf_-s69f1p2%|a?%5hYDL;=T5yP9|@OlYAi6*kVSgs#6 zXkF0u?hQzJJ8!+Bo+??mJTNdkZK^C#EZ&xHvL#Y$J7g4ch-w=_w#z)*62oU~)$zJ& z&U@&1K4$^{swz^<)-XpdAXKdJvnH=59idcfmCVKs=IoNVLb3R#5FqG)9^N9*XyFn- z(`cD!HdC1XbvD}E^o+IziGC|@Fe;T(?G*_h<{yOg{K~|DAB~RbT(h}04XLaRS|Zz} zTG2Y-Uq7tn{M>W@{=H`3wr%yNp6O^QG!;CDH_AO$R%b^@XJPEyk_)q%>=)@6P-Kv= z9co&f!qIH42Z8FWxdzg;<}Gn7j@%tIj|2+U4gtT`a9Kwb^$-+dAJGbXext}u2e z$jKxH^=o!V(TEP{mm7ryA;`GJfjfvposVLpkC*$=Pxtvx5 zj?RDI*;VL^9nj&(q%*S;GxaGirZSo27Y3OG1>a?@wUTM5rl7PCZTFOv{5oIvd?v+4 zQY=zCVd6@D+tkd-tkshlV#r8qzdn8+PPb{NEmyCPQ_*pi(DuADLV&hH`jx4^GUl|( zamuuB>lA!>^I^l6C2qfw)HTnE!JZ zahML)3Sw8hoH8=V3WKNIvDrO1hh3_ddBbrb^BNQvrM6WMEpR>6M(;HRIcm2!6t^17 zi{C5_3=WPS&HUPn;#wA;CKi$Afr%SBc;3gUl!_$|gz}V0f2u`6&E!(Q9wZph0M@JM zFy=-u-0RRJht>j4H2G}SR|i@#UG4au9TJ$oci-+r^RXzE{lbaD=P|!ta5`OFZ!U(L zg#BIUXhBj~sLZC*;qkljBF`8<+w-X!rMS^@Y5T#a8OLDaH1#^EWmPC5i@`n!iwhK* z?vFofNhZ^govm!>UEf7vvj~1$m^-%U&~bqxxn&MeX?x|Ty{J$`kkMxs5x;|S1QLi zZg8;0dRsD&Yiuy4Aa&p3FKGB%b>~Vq@(u5I&+8O7OcIRqZ{tgFlSVcMpx|QRx-G2c zY_*)jH$2}EoG?yB>l=pP3ozZHY(8R_iSI>#pY;IRc?S;$Pl$Eo_dPe>GqVL&5PUDXn zg_cJ&`@VjpdtAWA4d#~{2yzOh5xOEV*Y{u0`b?usR*j8zA)WKPo;rYpkbyV^|u zG2Tos4p%q+*T#!ri6SM*$~`W6)gw>5IMOLmk6W z>O0LhL<23v){JEr7!u89c`^aIjg-a3*^=~~$D0GDpXKOji_PB+4P`?^LzRiTDO5N~s=dXqCR-D+MVcxtQWP59cgQDos&vf10-S{_D?p7ZD_%A>mAQGkKxE&Vw zllfsa?Oy=>)h0;ug>s7M+oOXSa4|E|rSf#PN5uuLf|th|sc&07BHEF^H9P9+P+*XN z^a<8Jq*!{aI$L(A8BE!Rqge{X#I+Jj9%P2jw}@)kRWcL(??kPJt%lJsv0YTG5-(hL z$Ws%wDhF~L#o*+EaL?wKiXvRjPFLF^cbqajNDNQnNa54S+gSs>xS7$erQyweZpdBj z^}Pq(cpnq``xfo*w9h!pGde3?n<^&n*!MG)!=vSpGlZ6nz^ki|?D6S=l=8)r=y!m3 zv84Ac-qjX+E$hFIj2CAZTrM}^C;~om^f_N9cJ{uro~=i&ry02l$rq796ciLeB;Cdk z1^U|sh{a?yUG*HRD)~2{iXYqwTu*Q7cb%GjbF&j3)ekDjTi@4bs6Pt{4j$aaO)zP& z#YviTbI45q0$E1JZugq<=Gn}VXCji8?bm17%s+nVD z_3M1`-?mBOOwIW6m$C6*3tvNG@J~Lk>{!0Oef!X?tVF)jLO$7B()QHsD1U9(ft;Ug z1}R5CK)|a`AB1W8jtPQf$Lz@2k^U-sOU@6ri1brHfR#_Z8WuK&-(lUc0}_mk9!vZFKFFQ)v$*{%$G%jp%H~0_t@3 z0rw%nf3)$B4nhkQ7ha@oL|24od0k`;n3o zciPvG-Uyp)F^v@e)zKJ2frAa)F!Eap2%vs4jNjbUtv)(YSN*2z6NN~|Dj_SWXhkc58eh}k5_V&KW42e8KnA1PhDqZ>hCR(DC7l{i`d5nB7bnPP zjfh@t8!he@oxU2wKrsO!Zfff5%}7Y#vZa8iZ4mD#9QSEFNFOzB^3dKha_Z0T@gz zXtG5mz@X4uswq}Y=Y31{J-+&gWFQ>GxBaygGA8~-jO4% zK`X!wQ7QL&wfP1##I3N!gbm~8t(L_z-5f!&&xm}PDT9L7;U&>hjZx{BjLvUK^v^;f zvhJ^Skq;ZiR-xTnTT7%MNXqVepk6@G1oa1`O`p%S8Wh=_j?|@NsW%S9fGalYV?PC0 zwtu1-MlRowkW;VlE@W?^*&EoSp`c6e>bdpcz zC-P}x!Yg5$ul?Q#*^gtROeNvXbn6i@u*G2ebgELq(;FyH$$+fR)azSY`M=_xq?@t) zJ_ASNStKTDy#ges_9^?tufPp&yzPs_Nxo!irBWB%(KTP%^hWW{4o@oZR{tIO4=C); z=c{Rxk3aX7l!43A)}Wv-p1;qx(1-_o=uzs5?Ie@UVgR`A9$i1|{8zL8@-N2f@_I1m zivL8S6cpU{@RCR~aC>{F@^FoB(+>pbMl>Ay?`Hy3{W%s*&})eh9o*D~g_}GsyMqfy zJ3OBYOcr>7b4_Mwuv3Arj|XtC0Mdumb8~9_wMbrcTZauq2d1UYCra?H}x%I5t*4hF~sK;sFlA52VLH2{BtFV<4-D; z+B+D~raV&~f<~FrZ5`IC%Aes%P@$$tH`nAsji;;xSE=Vq}a#A95%=rQw7}Ew73C4d>rHyz)_$x>vJlNymfPd z?|@vR(F3!3gC!B)3%kVA{mBlpHv=X6>ddpXJ-j2~o73s+R>^@sy?<#nD;}pr`h8|a zVfe{1E3rEwm10u}B$Uxhb9=?daZ-PkGS%-TO65*6#%LgWpDR^KmQ18BIB_{_)NXf^ zSFOa+p{1l0hxt#ckXm)jT5_jcfJ zZfP04jZB45?Fs@(fGu-;j%;rnFHGbE`_C*vPm(ivV z*b&D9^Ig4Nzg_W0PA<7oLwFa~5iGnO5EQO3-lQ{l7D{I^s@-33>E25vQ4_7TI5GXK zRHBgi_(K6p&WV)#>D^EuGTnkId9~2Q(nhs$ew?xcorf1hT0=sPq(t+@Jv^|B<)vR^ z#JCf)C?gWo)!O3TR(;QvN54rJeQS5;o<*ioZp(K#_WBiOMHI#94m^$1DU?kL z2(Z{WI>UV6w1mg!7SD3KhK-+X7adnu83Xu@mvp`{dw_zvz_!PI4X?Rs?D~#Zr^nlw zXgnf>?Ym|7IypYIDqY`V@E<0RH3iC!gU>CFCj|-zbWxrQ2y!Pb+o6MdOGN&6X*GP@ zsFe$kXwiIBoo`R278p^lT3n4MXl%Jro|c-km@j)HDh>&+7q7s4`vp2wk0U88SCkfq zGa~nPO$(Vc=Gc)$`tT_R1_nmYC-`1`JcBVrJWg>Ckc78`%jczOJe~%MyIY;T+~%r3 zbY8nWM@tlP?2;MMy zeY&F>shpd*OUb4Y^PW#zUnt|V*Twmdu4sr79OU?wO;XdPfs5;nHkCR;0yn<*22x~g zwlSRAKP%9m9(|z+9Jto!X*G;C);(2lg|MQOD)fQsP8B{KUT;Sf+1Fqjjw*6$?Z%O( z>wAw`rS=;O2G48G8{frh!`uiu?OPC$?%ARDHS>k&HbFis<#Ee%+j-;Hisl)9*U?la z*VO`?D+u)Y)%+0z3Bq{npK=U~%juT6cXjz1dsTMcdz07IlD+*zUV#C&na>?FS9U8v zNNHX>tZHyT76-|sah2$a38@$Fu@C*>ev{YCv&r%f! zaRUnwpCUnd8w>`brJMBvgscmt8~kr?&j}GY&w;%+*jb5H>%dhC5T5ARmgRCyrEvZ< zxWc(v&2hU##Q0+y>6k-DM=yyIYfN_5u4dq-PgQ-P6S%QPm95hkBg7I-sG0kCB#;&Rdv__C^w!wl8`_!V|hN81;ID#~W#XSFGVh zvC6QFd1P@n8;vATF}mFzh~fq{+WLP$_}z}k*~e+WKbA|pPbc=iV2~#qx}CPkD*-Zn zC!V0m7zR+1961v2gU{9deS%86VzGRUNwoKSo7P!A)AI{=Ieo2WZPxzae_ppXse>cn z%qBxXBc@?AR1SDdd-?f+M?!oWFq>&8UzwU@XLUSQY-?tb8$>p$`kpsnz0&l#<7(jK zce{ql!nSlh(l5dT%9_qMPAPns%{PZsl8h-^y5GIL?;a6H18#EBywkyJ5D(Puqn6v1 z^|(@y6XJ54vYA|^l<``dk6hjzYc#*HH{8S)4?wo9+N$!Jm2%km%Oc*GmX!O&j?vCC z&vJd0!)x+nwYgTlMVg~nwYkJ{qh-l}&uAnmeqdZf;+)%6u`eQSoZ6I5r)6>W;&~Q; zj$2!*nk|)u&8%2h!hUefk-@ehBl@sf^TR)b$90ivK`MyEX71c(=7C_w#l}K z4Q|)n-7V-Y6R#&Ow+hGk-e2C_wIio&Bn*k_a z(sJH{Wh9m)nB0Spfpsx=*mbLvPr&GQ_GVNDb*_C7hWwE}!=c{it^N#gO zgA4(4wR_voHxLm{n$BvK1cy$Y!i~H$5INH%4E(rUi=%S4FNEzH8s{{8iM&u;o%uBs9aDp37zT z^SB2RKDX)d?s&RNi^CBy-_gvECfVMeo5=ht2KB= zoWbC(L8*AUJ@OvA`!J2==#iOi@Y|YIK0TQ&3OU!Nv%TH}xaREE-k-7&yPmcVKw`66 z)P46lE9(q?D0_K(_F!@=mZz#zV^h#a`HvyEAr*Ka9QyBP?|aR5TT=7YrfLrkr!uu( z5+uGa(1`eqK(6n|I5_F#V%R5dxom`B)aL4fKdgQ+yHr?3;d4;Wl{CwpgcxMOKl-6i zKZ#a#`v>Q%esGA??++`aym5v;AY9FllD=*HD8}XTTKMara;|zE6$9t(!6H@ZrG7qs zn%K71czo;|%=78QZ<>Qj`wiG*_4DXGC5E4AZu~vGE(ZuVZ)5iKX*Km0%EcyUR0>J+ ze1p;Wu@KnvIWo_P)XVsaV4JJxTD$b$#5tW>6(+JcNTfcGsl?Vdt^SmY>l^GJElC_T zx_a3t2n_r9AgEc8wesGSI(lUwHin^5tK29hi{3t8@g&usL&@Q`TV>Gpk`|7>lkxOy zqjveMxg)XMq6JNYDfDgNYGTd3#MW6kiB6r6Y_=5XrmM+9r98p*&TYI!LGDF09_$ruuSj~w6fuSygXe-_OT zUJ*n;$6pk!uc94iYx+z)@x0oQ9PVa7YqE1tjyM0>QQjE zjy`VLCo8arkEgR)3~lbq|B|)Z`_UERgbe%suv7CjRon6FV9gVte>%^pQe$`6YG&UF zoof>3N2~2tkEm{cvufi6J|~ui7~Pux6VhN3ZMqUA5@W{HLR!5AnV@%ub$dOE6Q0w} zIE&I(I-7vqq9RlD6dMM3uZJx$F}wg%?dom0+U#X+OV;#7O9A;#FP^0at5&#?!SRD% z?!byC2gR7Oxn@zr@R%@k8qAAhET#$%Td)(;*`i#~BCDqiorr7G6CwBRa5`_m0^?u{ z6xWJU27Ob@6LlQ)C`SmXTdFl-6Ew&7gn=TaaiLYOQ+=+l94t9`LX>l!?e|AcL7IoU zP#!IF)R*}h(&^G=SRHntm#dc+-qE2~*xdN3UPO}g1ng)MpK{?RP^#7BZ++|V;2;6)`1*npOK|ko z()@2&EE2CW|6|->p(ZPv|4syw6DXPj7FprhYB9N;K$ z4t6G1I!yU|)8f#8)Ot$PI#>fs#}{akWk=EI3MR#M%R{jQSbFFvUw&*L19|x#bOS`6 z>w=krE}!leb|8V)49@J9JlEm`dB9EHlK|MAaUQKzowY`r&Q<$K&jAkDn=YPnIo`*` zRrdgUt9brR2KUSsAQP{6BFqF7u8v{(UU z8Kg@yYq|h{r>?&g%hh{VckWml*br5rCqxQa(Y>v(mu7AJ8}T6_FxO2cQ!~wVK*8x0 z+7$hqTZf`)nIM0K)Vf>8Y-5iA2)Rig)HnB3w{S*$K3)B=m3<|tb8`{FV&U34XvVBG zxj@DDM4;i(ZPO{abd4?@vR~Qxa2odG*i2Q^gRlb+7>J^d5fI{#3Jn=5$WYrjM?Z|RruemDncz!}38Ph0W zFvWL&%U`Te*>OcV@T1T8UAY%CZ~Zut1`?d1jdDi6^PO{rVbUqH)eVn{_Vr1_IpWP{ zGc+Cy0^0SoCz%r;=fu~`^BJ3d422+)+44JvcF%)+xmIb?u&2h!etLmQW$PHmk&o}B z%gMwJg9{bk!zRQ&dz%jzm5$s8IVII#Sv;dCxJlQWJ(iv;ZcoADH(G>Sv97FM;CbtX zeDsL7(Q@4p_2PW&^F`|dOwJ@cw&gOtcH5#0fTWHAK5ORz#SdWqnzKXXyEm$mvhz^u z4#Y>Nqe*+4qJS4%dwYEzXMM|8!Mla3v~k(#u~wc44V5_l@l?xkaoH6B1Wm9kc;H`; zx3a@I?DusQsEdZ+a_z2nsjx>@&g&yIp;*jkzYbneWVCJaQs=Y^dY6x=lt=#Bf@Jm-80Q@vo zHf+cdvUltOWbS*p1;mY0hq0#`^vQorZ{as4^PvnShFxBd z&wR9eC3H$kgC4JUY!UI3CzYK)V+Bl3JU5n63uCb;WHWMFrqx`xLw`x{O4ilO$MhV* zzBRxWeHtv?66 zMWkP+=ffit2~SR-#_q_GPb~SJYtObb-g-Q_C!eEBgihj@Kez_uZ1Gt*mwEh{_UgwO zjW!sK)jePGRi)G8w|Ow)xihnbm9$tt{4#$?Xc~*f&}e1b@l|kNGUs~XLWvR$7V9a; z4#!hBGy!Lc^|xrPfWh{6U+Bo9KnP?VE(g;}{W@>(%cjZ><>UXj{6vG&&w#9}~y{A9mk57PK zCedn07nucUi4Iwmt#M@7E!FHv!x#quf(lrX6eQkp=P1FF1U~fv13@7(?#}k!aEASb zz7>oNLbSFjFZ!!oHgBdM6n8W2_w{2my#(4T45Lj@QkcXJ`l)7=!9NS&g-%3&cL_|H*b#Y z^}LJ+Psu?}+}_s*5Y{&0oXo9TIp@}i^&W`@H;wwE6CK#JhDFS+Tr5u}!YSl@_gnsh zNd*|hX&T^s;Vpl{qe(1mHajF^saZ;8$&;#QKL{ax)}1Z*$CqO5_OuA>yHJnkGe{=f z^HSv>AXp7S6NjH`f*GP|_y{q7wFQ=--4ARoQK6#+QagysZ4 zkKXA3ilPs1&2C#+T2xGU8qTuz>0qwUC;^m@7L1Gc-lV-i5F5D(1-IVT((@P-#GKU7 z!Y`~BEC!?Wg0yEQmXEO2W-t>U(sWqmCo*^_XjHz)m4{9ag4Skc+}Tpc3LmrDe99+~ z>D(`4xEy>T1boU-6RVG zEFuWex#QyGgDPYdqjfO@Xmy};OFMh1ySD53`v=7ChNIOTPhcT+L)wu;CNdruO3WMb z6afLv>A`v11qch1{wKU8WESi4qoAm?B>DSA?;SL>jwbpn#PeC>x*fcII$W#?^Db+a z@l`=oUhe1zFPM&(ZTnY!Zt67_kA~VvTnF z(GyW=eD^!iVv>_w7tD{rTF-s}BiDZQL2{X=VVJIm*m^II*pqp+FPS_CDp;2e-XxpE z&XuJ^Wsr7IlbP<1uyimQ&a2b&bsVnO`Ec5SG82nR$=B!joRIXB$U9__1V}C;g<^Oa z8UZu|(G6sNLj1zbwE~KPP*70ECS6{ABQqBCjb`bi3$!C;toO_MUN6Y!SbF6s$5{C5 zFzuGE8$CPAOT8TRuBxw`a_Cku9=x>1u*=FgE_?==UqgOXoOJzAv8S!=u(!=>!wQhi zkaJPs(ipi2Oh95G%4Zh7IG!s#Q7|)#_{C6x#{I?T>Z?vbZX^5(1|7Tm^AqCG%u88E z=0^MYC`J_W_SRk0>HrLMWH7{~A;H^`pSX1(O_O1DQAy=%Ym82Ih=H+PFaMZzNhtK}Arib6cV6JSc6>9inShI7aC#ff|70gt>7ZBX-3D+%DxaO>9PVN0hiF9S!yiHY%0epLLd*^0vIL+bN&EPgVmW zL$fk0v$?J>1g8W|zmF0Yy48S8qO{4QG~)GC!OFmU2!P)|7>%bSx4m76HSmW!7<)Ae z%1aM9y*F5@>N7PanTRIGHz>-Dbb(Kl8wS43c~G{5_q0B5H2X zGB0=Wy1OeZF>8~?b=LjqTrEdxuD*&L4z zhszg~Yxik!$G2R=39{?LI*}_i>P)Q#*)%fC)!UaXlfJ0^rk1>lf|kWUSx_-w*VG{3 zuJ*1VRPCL&QLS7rncfd1U^XdLt}l7q>Y5Qghesk))h~5?*%oPqwgZuiur$8cG$)Id zMj-|;JhAwW+uL!{hj0f33{|v>MLEsZw6Pe?YDDHO6T{Mi@4Dqf3t`+%NrNcy^P-bVBL5~@FM1iZ?#YSi16!qL+`qKpBtVU z*BRtqW(=|H#szbNx6Bw%cLtaBeLFM}9ti~n{Su?sKH~CS7G1nVhg#sACex_Q@0L|n zcuH9Y1()7MN7Mr8mNyB~NyMU3Uq&CqazMcNzzjC{*A%^ukDtE?D;W#{)8)T4MCHGo zbMo2Hw*9d!nC}S2jq7fuu7qGnN?q~TKfhMs9RxFgQtyoHY~0h4qW>ek1)7JDLA2;k zTEQ1uN2;ge4o`O(tf)tWDKGh2(m9n)L%-)PjRxy>LBN~&s;XMcDUul5R0T0h4;lP+ zwIH9sF1e^zvjs7vZ23SPWUr-KFS9@WUn85KMp1`_q|BYF-T-c%0Owt`<|ez zaPo12jc7FaKr@MM`%j{(Sx`Vr^nIGJ(cYB;6x;7~|Es;}k zRWgR^yMl;b%UVgI0eAcTl zsxn$3D2#XKbw3B%;_<@b@+g~mn}?#B`K;RM6uJKjA5Z6e&kSzquaRy#q5C>zey%er zo6SiDGp#AwJJu8bdk(b5Q{fhvdF~F*7asooC;ex~D&lf381a;8j_r`$<+{*d;e0tc zXt!KT?1}fW4khZlUM5`F5a04yg*JJFgTJUuzEZkVAw4c12r_wOG+(KX|3vd$lGEYH zSP*_Qg+j_oTJG$3X#JbhwW@T% zYK?c@Xp4!tmr2MDQ7!)Gw*Xe_jo!u#-rpbwAU<=1cO&vuZAQQGV6{o%uyxFpfhuX( z&mfUzhu4ej{;9>q75)JM8Fguwqk6f@+^27uivpBjt zk8YlrR3Iq+*M!$el(N%OCFsA?4XK&!VhVondH+1^j+s8`Kjx0`E0g-TmxW-vI9F;j z-Y#|T2U;blna%!gq0_0$Hk{M_a*)T8{8NE5T0xxzl~bd+X3wbm<&n|z*a5A0>QS{@ z@G;YQ{Ql=-GM-8W`=MC(NTPSzVFoD2v%HyNt;R^okw2}})i0X`7z#Y`Jn@CdP7*CK z+SEQuVaS`y<5cGHWE%xVoRR(u(R{MZ%e@*l?Zsv!8jlPg-Q#btsT#~?$oJRz&K=!w zxatLL5pC%hYXzR`q4Im-X;B_}QAlu!FY6JZ&)Om^h~D(M%bt7RJ&PVe*~XRF*fSm= z$|lNo3bgP}nzvF!F)Ms@9mV_(tG{#6sf}+ku0z8j z_XF!FUoYsF^wV3IN8kI4y`-^h`j<0KCuK08wb(}RU2&}1%&)XpPj{!$xWJo!)RgZ{ z-W&L8-0#lzxPn(NuAP!Qg|xe}GEWFAttQD0YPt=LR!gLF=JN?~=rrjIP_Kdy8ojUT zS%LHF@tK0QzgP>8*zw-G{Fl?@IGrqdsQATcer^-M=TgT!e`e)JTj5HVArO49J5_ZB zwgV$#=G)!b5Rlf!6$L1FL7aBl+0+zreOzO7Dt z>h*Vt3IYcjH<4dvHn{_hz%cI)#?wuNPxi;t)AxrG;RP`Hic&UgI$tX?G)Ap{V`iAHP%nGRHK<%*eByL=56oHZ@h)>?M#|yi-OmKDUj= z1>$;VGKpm!Z}5rGZL}Yc`8}6|Af>NsxC-t2na-UIE0-S{;2pm$EPZRROy%LOwRbgN zX+kOv`sce zTLzc&IW`kF#1z6VHR49zgc5Z>iXc$xRBed|GrVA(o;m_$M4YkZ7?sWsd%tJiYi6DoB2MarbX^_`%Kl-p zn@f|nj3Sl#Cku!~L7BNnSoce7yIVYuziH?t5fLhnCpO98b1}iC+(3EDCJ~f7<&`}m zVIx{V-p%r`K?A2zMywN{=+-``ZOL%b74e-Vt2Ci&#wx^u}tkZk4Pu-xMaGP@h@%{$;(LErww-hw@j zh0{dh$%*Erfm1Eqfkva%TvbQ1efrOKo<>u@N-Zq89?)h>ETk0>Bdj?+v4*?l2?KTs zBBJDXA#AN92()iMPELQsnBXE{TuFL3@c`IQrn5B)i-~-MRCG4R$hS}Z+MQLgQd$gR zCERa>7~StJoqKV7QP+pe7(&uu+Rx9I#rSk1>bc(iuZKvGMz8m1aM#x=c5)}{CVJmU zQV)^mw4VFC_z=N->>*9LqP)nT;SDr&TfqKaBIqb)_PqNVoU ztM*J#U0SvGZfmcU*eg`kh)u){wRaE^MCAS5&;7j5c;CBspMUcCeaJY!b6vl4u5*p= zwe7|JglYLI>4m>&2`wpFp;Eo~Mbk*JzW$F$w&iO6CC1VXId@9Ni1}5~*N1bVR9c(U zbZbvnq(R$?Xba!!V3Iuepz}lqgJ{&5UvWKOe7nkNw5a3PSy-L#Df#uwZV`66G>%QfJF7=XrWRT^smpvujPa;Ln zs}^72XF-2kwP6zak!O&u4lN(oI%4RIpm=dq648{y8>-9!R*d#VbxWpYf(aA)nX>*1 zeKLWasH1uM)?LF?7+H{DNBeWYT{4K-JhwI|gE6Gv9QG^2hGj>{PHdg8lb8hu3Embp zhM_Dou7&*~5;6-{C8Kcj@+y%=IJ=)Cqj0Zwws^$pW7Mi~{n~Q0&LC4*NJ>eh+q*ud z3o#n<9HNi-^$R!Dbfdhhi{_us$bI_wQK7QsqIKc#)REWjVf zL(I9J^VVkIU7>H9fc~8`15ZQVO{K6WYO}W-a+2lepMDW=I}v>FX!hHo=fx}rjFHV~ zw-=CIZdb1x%f}m^nzUM#ES4!B=DUjYdwnFMrKqn!hu#ZP=|a`Dz}PNO%ldC0O=vwr zB1*7`^C;&>B6jd!CwHMKU%Ir$%Dt7_`~(Ga5iLhv-(p<6bYuHAP{s8cykA9M%>Dp~ z-c3pjhr^Sz7O9Mz0>pvLgO_=@G%l4Uzoy&9T*Hs7qn>ElVziy1sN?hwaf%lKfn+lD zEt?Ts(b?Z970s7rvzR4W0(pL$FiU<>MTkO0gJuVn$9Fhn85&A{F;7$Gk5cI~RGU!V z7q+-#jdLHIo-tW^ZB5mQ?}-ix@R$`_5ZO;tym(SLWMkIEo@oN|Rqz(+5RjM0 zf*e|O{|BVb%@-*Oii$pG=;-QGza2nz@CXOub?`Fd92jYPob+BAnqSrOiJ6oT-POEFp zdcEo{`{fSyuS96hQ#37DCja1v4;s%^71FX@x^(LkzvhF1kKyIZnAegozC8MvYk*kN zX|&f?hM4;i*kyyBU+Fcw1+3n=+i5IQY8^n)YQhK=sK{ z*1O^i@^AM$9x03RZX;h>2mUf6>tC3LPEH`Dlr;~X;y_E-aFC`s%?4D3pIOUzZx46p ze>zl_N!AE~{gmaKJh;7Z=u~>lLrd;5e45tYAf2rF76b3eT>EqEV|DeOIf1#o-)G~_ z{@`S|bSdXkN?VVBoc^nzN?-*D@4a#}CMISrH?@Gk^wB;gIy(FM2UsM^F~CFB$xNlr z5jGx?9LX?o&+N<_o6$=nF}Y)Jp8u?54GY07hVsg7D?a!fKs`@!S@p`Bcg-SwBKr4# z>DT(_BNFX@hVwg2B_!aGx-iGDt2)R@HR=v#hNg~~6UKQDMyjCV9w$R0Q|BDtn# zy10;|$hanDYGCUP8zjG1ob^p$IZ7xt;^c0f$J4}L26O%c=3^{G5x#aIz2HWk$|#dC zl&Ou?!q^@j>P6OkL6aNxI}Fo52fb~^OSUPw4W|oT=lvfZvOfeI%KBP?b3S(6ly#7_%bDcFVTw)6cKi}cT-g!ocKLr&q8 zxca`W@Z!$TEu?;JpNNW zp(UCvuR{9PKFl8f{;B%EQ>sp5FHMGh@b?n__kTl6w1D5x{9(rOze+h;A(4k{c}C&W zxfhA@;pwvf>*qqqjDt`0n(hAkw|^(2&RxopQ}+lu8N){h*)~tdLbsjAbDqcCSUT4y zvf@2ObM9Q+ zoVemcIjE!n^htALdN@ud9t{YjY}d3vx*pVsLdo4Z&$ zjTjFbHNuE>(hoMb@v>qqZ(=Fs1#E{~^Y*PHX*^&V=ZLvY4K#9gA91CA1{Fu`^QTv1 zMiKt+E=~H*Alu%-Z+-WB8+k1u9CRH2IkGA)(L1|$_~9D^4PtWWitCIwVB65%ns2O)O3N?0CS_%p)RZa;{W z98eJD1BK(t(%40-S8Q>5rqy#oZx5V4Qrii}&mYX?20w4t$GIxWylJM`shOOnm$}#a za~F1@%_i>b!uFd-)1QP$SwWl!O*4&}1N`YP8%Ni%;_NXAi^nDd|>dp(JrS)vx+ zlPP27s8@VDlswdxP?!070OC@w17C_BeCcY)yLkH~l0|Mbh^7&zo%7*0`Rm^Hdb)nna-Z4t;?jI`R4nEg1lzkh|FtU)HV4zaZ{ zaX&KS9v}Tml7=DWtSt*Y+&q+mj{IOe$utSJyrGp~y>;2s=eK+Mss`sR0K3@W6F*T? z?cp{3l#PvK2s4GG%J#LPLNdc&VyXFFIe>8aG^+HQ) zo9o*LQjz&J$Rcfx&tI&r%?TZ4iQAQV4HV>d5If&Sbe05{nsNy>?pGflPd}g6zQ)MP z_3_!v1IXgOiyPMA48td*8fTtAUGGO?Fb#G^6jqx)2NbsQ-0_;3nLuR;C+* z=YPgck2knAS~ddCWS>=Bw&}*(9{FyQO|+@6X?Y}&kmH{oCW$dZHBWECkLI3;GQEl^ zo2;z5@U^KFbvEInG)*9FpF@?XU0-MRvT5w*T4dA{D&bf6OiRra2sNeRP$f@0mA-3I z3n?d2hywgfA`R_kqd!u|0URiB-_N8Li#-mchimJamiQUF8_1^Uo7! zS*k)|EVnfZ4Zg_Ay1wE>{Ly>&I4lMzGD=5AcQf5ll#X{hR~kc<~XK`smF*Cbk{zDYYeT=D5a*{%&SN*fC^s!9KaSkE zK+!*$iO0nlel+YFb&;{&kH_46+NQ;H`?&%-0Oy~w-DX{ASQEZFFF_klX2IOC!_Z=c ziHg^Gw$_<#0WEra0H7cr+i*SZOFhnT9%vjyu1m&sWJ2_|R*@#`!+zLI=Ycq)^(Yjc zCKNgyxZma3FS}T9_P%H9MivlC>{ki9;%guLFt>2Y4f1)WZTJ0;PiFaM6M_3%K!InH zCt9`+z!#6ueCper@P#XP$KSEAT>OI7;r^h8X{dM1Ts)t!9JSkd_S_J@rQqSj?l9<96y%RpsVYjyi7Ct9*<{PrE~YaCB{h=v zo$}<_ulVhSqF-Jmdb>86NI2elJIFFbzC*P5xQ6mZP<6EUE(E%<>?f>?C2IW$;>I-Y ze;$DvoqpRsj0?BwmmfAe^oj34Dm4|o3CffrNQ%<`mi?j-z)+x5_qjw@?)x|pm=^Lg z7_Cqe{c`3u$ur>@fcv1^gjTtXS8Jpe?325+jDL;#LtkZHY-h_#t@tA~A9wYr&VPJ3 zzRVHK*{QtY>{bY4zIGwm>l^nE-e{!RPS{(pwg&vf&OwYNQ7d?&gw_*RaO~$-8vnA{{_`RV*d|_)pr}e z${XJ)mOejJL7C#y${8>xmo<7$28c~I3%pqZZ|7s5a*mhVI+6KqM#~cWUwd9N?ISr- zcnFx?jg530uYc&;w0Ol}XW21o9d^{eg7>gN_$`noG3SC!WbVIo*^U#5EXbdz>jJ4&s*8!^}kXCXS0* zDBd;s7g}0}v-2W%gIjNG6um3SS`DbmO@5P{|I0*-KV38}Rh-+3aOGU9hRIwb%v)~$ zZm{8?lL-B?$lRSDU2*-gLNkz^aA~(DJ|o|a=x&yBTn5TaIF%vN?$v$XmUA%%1o@yW zuhjv)QRs8?^xDr~Hqe9f&BXiSq5gbtK*6h5&rVK_8XJl=J);Pt(1@StZ$5s6hG0At z$1l->r$Xh$X2+Qp!+cHkFZk8U;(DHhN7cLydvib-6%Z+NfQ$+}oBm^~WabKEq9oeN@&9-PvqdgAZ-=fegT01v; zFrLfmpjh9K!>Y=6|0efM%A6c3a%~+3hSlh;gJBh4mJk(R$Ymu z=^0{>t=&>P7z4KRwcXF~?Q^k!y%bFm(ALb5T3*j9LOR$ZaQo}yCOSM?u0LM%nV;V9 zxpLA$`N4M!y(wFThi^`BNmW?KT5$C~W5LlO>@--$KTzwW#-3K~Nx1Y=!{5A#U(u=0 z&9ti1dUh-B8BaFe8*}oRyO=e#N5>@~NED*+v-z_mQr=M;|+}T*B97Qujhl3A($>y*B!CK?k)pcgm9e zwKeoAgFBX8UMC$Wo*IH2p0oFD zU=wE99)VBWD?9UA?XgoRjO*M(!!-lO(|BU_`1jb0b^Va74i-w+;li}@V)th)cvLHu z(lmGbzbM=K^_N`)xn6S^D$1aIWjC-u$WLs1%TjxQ2^h8XzMkTb-r)k(O zi%#0{hiki|w=*ZIrVj-;vD?wmx$ZfnO@$r8k)nZia}Jd3(RW5eI#5E_X{v4&>z!6T zaaf;z2ddF=dQ`7oQ6!LbH#i->Z3gxbsO33jPS_*$+JQ`U9e-YtF+EdsFg<^2(mz|@-}s{3$iAaz1!(mBA`&Hrs?1p zGuHWUtE@B$I0CbT^VNp#=>~K!W;(D{z;nv?RR;}c2pPp+y?ioNC@#vAu=&XKC`g(n zRo;XpMPfa;nL60p-$V-)%%)*=+eW=dEo_)iG^L_DuiI`sw%uieTbKazY2!(wNj%8- zj@BDq`1@KjPVagg%xGS31ncYyn(%tYGDUR=T;0urG}RfjG9cX z-0p9L?Cmrn6vgl3@oJYP4TeFFsdUd8oo>`j;>r@>s#cJPI_qT-l_lp}m%3v#&L))D ziyC@E>MC-Z1h1gu-;Eya80w8XtPbbJTy`3r6!8g`%&i9rlY{edD^qCsAgG^q;U2`n z0oe>IzO+5$FPfv9w&Za;JZJbdF`EKJ>b$QT%Qa*x9DC``qWt)z)y(LyB*t&)<9&pYU2l5ptm1o8-LLG zXv~uTrHcJS?Swe9Q@+4XYE&jFo%f&eS=!kv&eWq8z2!_u&#H~3JUjJGCe*I#vQ>eT zpbH1?Q0qo39`tCl8vQB`$;9(>i$hlgFLT)bR2{50oZNqSF3I}vC3D`;O8#yKz?4Pb z1%FfV-=5_#aHIG>0U56|N=>ck2TWtwL$D)v{|_Fl^uBlo38G#opH=10WS^v3S| zcw&mQRbr*fxZ)eS4FQhP5{&$M9fuP`EqEE4V+P%-mUf4YZYo*P-A9V;uNR&oKaJ13 zuh*N2jO{*Wr;(WVk@a0<|GNHg$;<;PEMnx$mt50LM;{T|6~`wd&^&Ar(#1L~uRem> z5s_BWE}!N)$sG4wjc5I8Td&4PPf}{)`%zqw3|%h^#9*^Hljw}9hYK+ERkS`OO__Ru zE~d4VABY}2?o|Rkrj%%_o`$ayTo0H%pFNaMer$|r8)X`gjtF58?St=d8 z;S9>-gO_NW`UbchD)|oFEj!$85eF9@8Y67MqtH)pKW&fhyXN~le2 zDtZtWDXCt#MX%0wKykP`F<#vWW&UyiWtH5x*H);*<4cvsl~w_7>*btIx?|>D?=cNz zZm2^}ypw8Xp?xFSd5Mmr72n#_>t4`KKqkI6_f=5O>~FcgOj_T>8r5tigAPd4tTl|i z^jYhxdm?S8(>#XpiU+kzMND{K7s0W^PmJzMC|5h9kA}aCH61$Fr|YF7ai%tE4s)IlsrOrU($6bl+RXENOW0)X@8+;u}mF-l%rZ{O+z(GB$C{L!#-bXK(MH zoGugbgtOgKJgEU1gOFUb8hXOP@W;zOg1P!6j14x?#YdEdiJ0efTI!hK<5p8&%SFYbyOMLQRixq2bVb(-+6bV&j(?YSebLbm ziktM^dTgz$d9;uM)A`vp;ent$$jBpKZ34M{1v{7oW|dk^>g}%ba;O#c)Gda+U$lNQ z1MRKeniPrX&y*8N=uP07$g(}vpLtiW(#t!sv3+5-PcW7%_za^$^O}nsZN&P$ZseK- z0M@7P9J??xxiK$In^q?wXHkh_CGct(cF&T7k7eaiXPaf8)bN2K1xl&8)kfVH2@=2(4+R^?Jf0}YsW+zFVZ#xw!A3b7#RjIq1o8yb` zi`|)iS>_5l+CH`Fc;ZvBbms<3e3m5mWsX{g&PlVo+P9S5p0GDjF&w<=kUYfd!v>qK z-DlOW!V--KMB?;Q9Lcg$B~5z2?ql#UEmCX6K%?1BQ7hFM5jg7w4S!57LSHTB+uu6Wq*~l1(6jSk)wvC_LLZw9x3xVKlPa@H zyE0U(By&i_%Rzis5a}&jas7AJAjZ0uDSln}Hs!-~+z1xP?n(dwv{W9gjGYl#IFEhf zEseDikB#r{pi}v&3rVe%PHt(t{k+pf!AD3tl{9uCSL2<|#F~{jgnnk9_4&xt#VJkR z{|MBn)jbv`cwY-ni_j7bmG${{$?ZNmgfgT`3*S;2xjdO;Bf~7`m3>Kc(OjIh&br-^ z{7zw`_vT&J+EIx7Nly5R`%`{1Jx6`Qc1p)jN}vx3P;#04bSFfDRzk+=$Z_^4PFg&7 zj*xpyn_X;Bb|d#xkTLl*lZhtj`Il5qbn~gx#`=!;*4H^ady+*q@AVkwaaz2jlrSM} z4_fUeF+))`SEkP&Wqh~Rb21;{6DmQx77d+NXTCc&iwzAk&3ywd%i`00ROdgRSKL!6 zYKG?uV{SMo0K8v;`1CFzFK7pN&^l>GIOWHd242)~#QyW*N3c^px^%K;q4A8*>FPu8 zAP9~zne3|>_}*~VB6P8JI8T)eh{Of1;agVQ;DIJ)YElQdiKFT=>JZTs&WilQC+aCT zULR(RlSbCpXE6Euh43uCSnfE`J_d_D)luUjO8eL65C#R#y|c1fU!PgsJax1e-NDo4 z(mH3e6m)c7%5%R&O7bMqx;we=sSJ1~6SHOhSkSE2PXwsnfoW~A7V1@1fTaVsg796< z@^n2F%|0!PW>jZ9#Hu8rq(^fma zaq5~=(oh<_7RzO1tD452d+lzQlZ-FP;!WWcU-4)OR$S&DX9up4a4EXkUIFfbS_ z_xj>M>%yX?MplHcCG#lk&2LA|$wh=Rf8IW&FS!wgbS(!ZA?o+TFjrHEBkK{O__cNM z&pBl{rmCc@RIYAR$}(>D5I^9cRsQ`@URj17?99FI09vMGC0}{Ze9!CeCb1@E_B}Nkb{?&1uf^+dBzS*k;;3fAY5F!e8si ze@$9J$nqCufS&$uN&M=HhMXePqU{YW`Pamraow#V+@N;&&qe zR?gaUvLf8sZLiKSPSJ-Em47v3|DM!s&Qr-|6u%eocN)I{zhq(IUoP7J{wdDnbAB}B zv*srkD+s@v;E)im*~5b!t7Q@VZ{JEr(fYh)xYE3lup#|>RSQXKVZ44SdFFpmUqu>W z>KE_)n5AfbI}Zv}1*9I#??uLNgA&l28>WxwelN8r z0~ohDwJ>+~!uHRk*ctbUFf=qSQTBi+592saj-oo zX=}%&NF#*Dfc&iXJH}(Se|}TToRZ2hZ=Tv+8RRmkaVXxHBEFS9TIR5edE(9??)bDT zRVs`)4Hy^B1@~;}cEkOq;K8Kuleyu4ZAE{n7C}s%(Z#hGQPZIjLu8RkzC`cMfXV_b zm!CWHycL$c)DDAXA6Dj3$m&vqTRZdZ2>}|D-FbqOW++#`$ueq4{_#QBLSGDW(rdY^ z;@NtRO@okGaT-hrI@cV4q|Y&n(ySn&rgdNtn0~ox^-d~a#(WgaqzpF07Pl<@?d~ZV zdY>4eg4tqo-p21w2Y$YfxG6dKlV$tuJb!BB`Z^j6%%T;X)BxhflLjGQ@smKfV^y*VUx5jbAC1ukTT)=-7bb`?+QF>Ks1p_C!lrI1!n7M8%t zjcv*Ryn(+_nODN0Z*Y#Pi@Uj-+JZ+8Rh3%tFb#tZLZEkqI1F-RhuS|HVb3nWhS`(mZ~QVa()Su zXFr?drI~YYzdAo_jteM<{moLvU`Ykkck`G`RDJEJ!M9vD3xi904llXrj8K{ZkFI4{ zOC7rR#8CZ?N&WE;%HYe{Ug0-JKcPZ6GS8l@Ma=kYC0cypovcE8R74!X{ZEu6W&;?FbyL_$z;497A zg!aBa*ARwSD$Oi;!F3cg38)@MKtMD-<>-2EHQT;)J`v0B(aCx722aF@C!Zf;6vHoa z4@@_8)`z0U&Xq72?_;UxVxIknvPA>E4x7AQ^JE3xKEy7?1FTNf@Llo6!FgqfD|WPW zGWI&yP&GxiFh?uxv2BmRtcwBLxf=D=xm~5nrJ=8`8sS#mTv>9#gZrDepXWS*95(uM zb;WRhd3BCS^CCR1(;1Xx&uIv=5A<2T^?G+PHd*AeTKbrqYF@-Dci?DGh5uxK^*KQs z;~KqZTe& zG1Y=BY7L1RH?_9XJv!py^4>=gf}rUlgHuvIyOjWB2bVwU$+O~9dDA-y3QuUTHJO<} zpE}L0RFUJcvf^&gVDaI=?kj25LZ4~yi?U?L;Dc1Kp%x2s|Aqlf<@>t}3j;A$Wj1^F zfqpHE2)fe6z9s_Te^;4I%zB)(FUVaY<;&K#_cN)6P+gfdg;|Oo%@VIW~ zKs8un^I+@4L|)}EG76|dHh?T1$_JWr);tBevoYU$kuCRgefa_O#UlW`V-1k6J=R7F z^=qF=0D1%{u)@j}05=u`^dU4ctZ@|t+V|03`T>9wZn{iV-nAd#beO1g$R5SHSTv~! zx=z)`iaY5AYA3rP4#YQ=MsdacNg{H!8_S4$K1YiG+WKisD40Di2d|!PbX|2;O%y9X zu5@e)8qLFs*5uER+4dy^=WSAKn`@{Ft)u4EoebJHM;q2VyW&GdV0{^X1&pgs)eu3R zDF(<}K;XH8| zicjJ&!Nf>!%pA8@_hA9SPEH6O0EKl|u}o}%3ZCduvk0&Sj@!w+SSjF>oK|3$53;v& za9Cx}043qld{ts7>P(=m6CaHh0k6~H>ZY?=*aB-}f84E@8%I;|#$wE<*NCUyE(inj zgIW2{R;&bPM>&Th+0{LN-SLxk*Fpn$O23jHI|Y^3ZD*~fi!c8;x=>1n3K~^+hMqdT z;nM#8GC)tI2Pm2Gy<>$>KE14aziInICWoO89>=Sd2rLAQWb`r6JE#8;Xk!XRYHM+; znM(i($k%HeWkZ(C=#?6(w!JasrKNz03RSoQ0LPa%LpD8w!35{v7~(Lw3X57AcFM*c`YbdkEd&K zc7-z~84E%?p?hp%-}ICpn@75v&#5ljs*m%+xUQN}(m;Sld zNJmwqM1;Y>r6#?&7%KBY07VJBHI>j zT~Of6IdfAju_s|MP0T@u?`c7_-1iGyk5mZzn*bVvzrud-M))N@9Tk~9b8lJ0EuG1A z`9t=SfiZ(WOp8#aiw85L6FU@UDEC7tONAqnpU->`s0V8o5F1WJ`#iu@H33ZUmC4;SdU!hNtSml=zWSLvq^%UAYQU<6KM_) zYo;YcYrSDw)*}a99)v!&I`C2Cl<4C7stozSzC>dmiL%8cs0m?MzR72^zQ(`rZu7L$ z?$7S3v>esq$g**(A6g9w(aBmMvv)M3g%{m-B?4>O2T77dGHllkdFjiB=Qzbt?#B9?s&Bfv@n z@;)kwFL<~YEtSKlKm9NJ~OVns5Z-qts~B4}bES|;A0Z$!HO<&SNBzx-sZ z(xyT1C(#|(UFIN5BS&F@L2j@evOZU=9PVDzdYfO{rTo8>hsYF0{V{u2^x14 z^F9C&`FzFt(8RA6ZI1y^EqCDl#zh|Ws|`Isb+lp!)CpjaI;FyPfl7Qqp!PG4g6izp z#YkzQ0a0rG$OoExvBq_ppHPbQqx=)S*t%nm#y0(5_|9D84LqJ0+{~zVtZIab;XockWW^ti5O5or8XJ^%LJo`3T4{Ixqk0*zpCZ)LN{UGl#z(A-@Y0~NyW^_#@ zk)KT}x?$1d-|y#cHR&0$)oNy!L7P_H9rJ~kTVriOhtti~0OtBdRGRIjpfl<;J%>v94bvGyey{25*%N7*FU$xQMWXk^mX0zP^NJ*FihG2SxC^Hg z%r^erg&QhBczUC>wCjzW@6XsudlWLh4!!RDV}{T!Fxt83gtBf=rPUIYv^g?f9V&aR z7Z1u0h_Z{oXl7_ z*j0wGH+~Kcyj^-aU_Z$t_kEgm(bc}2TMxrS!b4w)^O0;XIsW0k0sI|eG$rKbaw+SS zA-Gq0nFk|r(E$&1jp%hHjsW!eWx?yEDc=His{z`^XKqGs13bX-lua{e^eRL#eGJ)K zZf+)6(GDaE=xd4Q}kIrcuekjwE* z3yDG+SI)c;EM0}151`P`Uk{%HOkf1SEO4GNyrYx9s%nkP{nKT)eV0-*L)r$!0d$IU z1^E`~mGT4JolC@sz;!{)yo$pU9U)SI`WZIf5wie# zV6tCgI#Lu^8Zk^q8P^Rf?TNZ{1{~`vpGa+}m)k&gD6q01SXS__Tf+=N9}O$jy7}iNn@CN29K36-XB0j~B%&WNS|p?&fH? zFb$VOz)q>5L4++ivySgL7ypbrpGdPuaVEy~irRceMaenpiQjD*`?KWtQ|8aefJxP! zH7govC-8V~PwH@1f?znCKw@2!&D|vNA4>_=)=WqEj=_1}=`8A%a%v2YRG1S!Q!WYD z>vrA0Z*-q=Wwe?Q1>Q&3a0Kpih*@1&-f_7y{Gguh%xVJJWA({`g zL6BhV0#nhpthqbEStXpm5{?b9{njG|+WU%?nk1m%O0Ie9%g-I~z#0@&QVV+=bykhb zq%C1czKWyw1R1fkGv6_3GDOyxd44pn!)I?r`z2g2jN>0COCdD{Sa21x=owceYRd|8Rd zNY2{8;eH!n?d_P((_ob;26mu*QxF1o75Q-4;msa4sM>dZw3j-#Y6f8KPhfZqr4(T? zNK<%JZ4JP2ehlDWHhr`*WgP_SRw@F>(*pj}he9JrA)N=UesI`$S7tgY5K*1QZIgJ6 zb{#*LSpLD@2IQjosK7QT^4ioSD$TadQ7wxnp(&UdlpZ_OjN3dfIN$Sui8m2kJD-%% zmy^HMZ&rISQW>H-<~tjGo<-X5v0lM-Dz)HScIwQIE=>(pH3kjjtND*}@ivR!m@5Ji z9x1Yp(dRd&d#z6-l6bO40*X>|hRw|5gmiNsTNDd8Ua2*MX4sM(J%7CKOXX_Qj-bf_B@S|!*tkqIPO0^I;lShMPaN`=AuQp+qxzlI# z=RY_7B<}21?NaqmZQ}N}B@%)bB?-J@VMroO6=o(r>^=Bd$%olQPxKW|R5$D+3ny-T zPZW75tC=cR=(RG?dM8Q3#gH}L`)J^{antr%l3Al!AqEBlvyb%;0rt@Z=0d=ZArPl@ zNeaN#ZPl>D`{-5HC-b4}gqJ5m1qOXbP0Fy>;(-Qf0YN+MOcGA|Lj1!ResY%pi)Vnx1PE*8{k`Z>C;LL2&oP<^NpnqWQm) zwzXw^Tq2!QD7@a;X!x7CNJo}dBlH_jglf&Z>FL?z1~swk9ib0Rtu~LuNzXJ;=H;1C zFu=?77`p~{6MQ3-(%&qIFL7Ab*v1q2y%+JN0-JScF&>(&2&oNJeXjccSL&du)q5?{ z)#Zs%%^o`;>Bqf|6VEZYKS42^WzxF&)>Zk`SIupdOQQXuii^C~RT%^IErjDEn1r2Ky1#UF zX5er+wzBSYIIfQI{Ah%wizbL%WM~m*?YVS21^20V)qPdfhb=7S&rn%Q@s)uO#a_lXB+&6n!JcVAAV(}vxnp8DkJrCy zHQZFreGTIIg9hE;3pIHp`pUbRaSs^|%y2ocueg(vk@*o!pq4lgVIpX^VuXn0{*bW| zZ!^kUFXTQa3&tf<8&yg1KHp@-as}+I^gRs37%u=-UJa5>pZl7q_LDDiKVE3AUoUqB zea+P{7Qp`LtqEmfc+m~&PZnvTQ_FrPw^wc6N-NfwH5+qx9eUYJlh}q;!le3 zF1q!H@K3;yzBRF|O@ENu+aS<)QcB7dHcqAG?RZ%!gVL|57 z@m`kiqaHn}_e=FcUHb56;eWIOvL}EKwm%g+2^ws|9HNcWY$gAn4FaKG76U(Nwon7i z2MN8;k3Oy~e_|3ceDePC?qeEeNryKkTO+HY5@BL`Jy&DJR4vU=YYsHIKIt1npb)(P2i!1YU_p@id+mk{;sy3%30{z-19r==U~ zELBC%%AqD%P?^a?)|!p4c)*sAeQqmjnWYgWu2E!XrZVd(mw7svTq~<;G>mlJof0su zC#WK?Nm;`DG9s?!E6xE()jt^I&1e=ed-L%>RJ)!-fezD#KZ&yqsHj-`K_YbQlD{wj+GV0h)KQmz#OkA8?khD$Jn|%(DsRTci z7M>N%JmA)Q^^9-fLSs;jB)%^O?a2Ltdlj%hR>~h&#Kc+Rz9~JO3uq zuyvjzB4rwX&UUkh=|IF{&6z+l1U!N^BlJXZ&p~7wb=_pO^G3kdOc+SRe<_Y+`@rqF zgtA9d8nLil)UL;Q3bYAhevM>3Ui zyRl=CX=2HfFsHz9yT{UQlRIf9uB6bg>f2R`*`~r+WjCJ-TuSj&qCc9RgiN@;;%L=s zF;-#I(~4lAq=P`&0)Y@m^EvyjZvfVMB1Nxde@z zVZF4e6v=n1TI)}9k*4uo2z9V{wv6QgO`-APYH?0SB%45~QBAegzGGEXi2-clC;J|W zlNj9DB-M7KRx41KeFh#p6G@&fZ1EuQe84qn35*tqXx+g(S{?~R5Lj2H*(98kal_85 z08y=OR6h&oKRxbjj`~g}`=S|P|19po>r8q79E0}0yW}g8O8VNyJQ#Pc{=0jwHV)v+|f5Nje@{N3G{&XD@*AJ{X=>3_$D*%JUd&c-%!+Z;9fwx#V+q* zJd~%pIsxRPX?D)p2y8Cxy$r1YO8F8Ku-rS7XG*7Itaxv#j|s=#HMmF!tbkzY_zZ=$ z%zdJ4!hq;44RcE?QOTNgL7{~^dmNNnSZqZGM1!8ClYNck%P+O-&(um_7>Ub~-pZgh z$jEEellvqMG;0YY%!biB`+O7yeFa=kt4)zoCAaaR8m9&|`Rwf=4L+NVHstGF zW`cwQLBOrmXDQ@Gy3w&P^CshB9Fvggo}0+hKM&6jC=AgCpofSAp=Bpf-}6e{JoR!c zKsay%MZpi3Sltj=@?gVe!j_ENFgl0VuuDAJrQ|bq>?2(@LZ=|1Egjsw$E|Z^VpR3$GTrnA6^XK?Ehb}LDHGqM2CdsfKHJFe@xTQ={j z9Y90+b&DgDNY=;QSNd>r5l>qRfgjy`;!V64m5-Jv5CQ{{hsBfJw)57(5Jip3$AD&U z6y+04vZSEG(q9z|?)!N$6e3O<$7gqKXo*`a1}c6!^#=CPb_Rzu)!Dh>7fsg1HXm7ecSUHDH1MIWw+zf=6)1^Hd$3T)%6*y#ukEwpU5c7ob@kR0rdgKoC{ zd`7r4y+I9yaPL#Hm-!6xjbh#yrdy*4lZQ7#&IVeXpX4+vH(X3$RR-d=wjvGYfQ7hKkf zM8#b;h?d_pY9QTcS7I(iKYX+UB%}9^fQk|upov-|_5lNTuLb?n8^8+aJ@rh|*$E1Yev{!ad zy1G0wA{%W28ZG59RB_=+DfL>rO>_o6q;p*TsR;h(K13&EwYBiSJp<>NH2^*^4Aw+< zch)2L_Z!X!Zw&U}cR>?q<|5G@iUnJsK7U&YG~41(N&=Oa?*7>vh_sUbQfVnG?TgT2;iAmPIR*$L@)%rNAHN5e^2ywsaLf0WVI)!K%#8b(6uszR|ScoPc0qm8sQ z0_kMw6Zxw}2B9-vsay_7ktF+;r&@-|0=(}*$&reXZVj(k5XvTE=MmV67<;DSc1yp~ z%1_9{v{cw{rk5%3-5}-GK$^aw0<&#jmB2;YRa)0QsO@3Pc zywh_(sBVgy#nAKiD==aqxf1NKtn57rrm>lcavDSF22$2L7Ae>B8s!?$5v3f~FBtZQ z9Zu8rh*|?YaF8G>6*Bo-B;HSLiKLH_a}_!*9#>?mGwU{#`{>=67|MxQ7gZS?<#kO0 zc@=F|j7m2k%6$CU_ZsL9Ir%uil}KoAxAsw;2t$6aw&8`?ph{mP;b?Mprl4m{Rw0Ac z??fR$zHuX{zi9ClB3f$mboQ>%WHA2~+x56EYi-PG#LXwU`t(XKWqPv~QKyJL#C|nh zb)LWWQb0jYeHRTv#A!a8a6s$=E@%1&h)s;{E1|R1WVOw$4Ob~RwkMKsfr=ccpVFjb zloNVeU6~G3&J_7>OPVmM7JPKw@G`9t+_{MyU z78miSkqEs{@Lpj0I@vnGeZR4?|13xc)rx-B>j{P*g2rz; zCZe=6`OP>o-nV+)5@@$LRV=o;8@@_nqT7%oY(!%biAsMN9QwsCKPEU!d|3fIjF2bV z1NaIoTrJ~PVV3dy!S7mIr(CB6(hI^rJ>!q3VSD%@qXZac`vN>ov#HoS)jFf>V(vV` zL%(VF@A7Gf?;q@zJW2(?fe@76Se_4PcV3(;Xn%o(p$zC`!XIOI6tTK)rd5d7diwkq zjW%at3J@SM?(x_qA*-ZfxAgs;8paKKZ?Hp;e(LB49b435Lwv^J31Zm67k?>G3Rp0ZABL$|=e(i#)Gpe=~ zKz0ZopcpiEj<5Ayk@zpMAumWR(>>jWUJhFNhYo31R$9-rb>UzGw=zh zLIU_ew7bV#uWtW0(7H;zQ(RC;P|%<+d5->n%`*UX!AAEG_B^2tYb_A?mv{M(^BxGM z5`BEMr)-h&KNb);o^senK4`uZbpLw{oi@N%f!cx1EOW*mfSLccrLj@1AB+NJ=Etr% zr~l>n{&{dm1CO#%>Ao8B@ss@T-!}Lh5UPxPZPqlw@K?h=LPkDb?TrJN)v`?=SK7{S z1>Bf-mhx+V#!!qdxMcsvmKh+^Kb(E@=3f@_-*zMfDKN%HBS~mMqd(d;(rok72PY5k zyEKtosc>Sh(xuj);gDni&mntaDih@5fc|`lfhC>m*@W=UGhi=;0^6P*3BkYo*UP-D zznsv*Rk)RMus$^hS8bFb=~pWcIt8#Crt&#DGtAjLzAKb|8Wm2SR&z}T*cYA`saNF+8@n>@uuFf9{757&s6c4-Q{Q9~glD;F7hFe35YY!%I2-}cPH>3++y8hV|F)$9$LL}iP58Y#0kr9h zuMcyqksA%qh>cRto~=<)Br?&((MZLIV%^6jleG>WLN5k3m^2R%6=j6!&!9t)YnJyr zCcoK3Haa~Qax%D_9Y^0B6ZY#o9M@(kkBk$}1bol$_bV({ke$vEkHGwbfI;F>>r|V( zvikEe`}fPGN@;n*tjfr4akd({4lkW?|IL=;BCn%7(q%5=qjovsnKC+n;~VQLXCc?t zR4vIFoK-Z~^u#}F=kS79P#7rqzbVz#cPrPWet2^l`!F>6*LZ&465$#Q-0e%}Efd+eLV0F-hPpxmrnfeXau zo=R%-S5ZKS1WvPq4P)m%unv*ajH7s`kO;UrvKvneO&4|_sTL^TAyo{%1$NdoieQ%5 zPeUl{ceB430x(zTDXelU2v6E$0hq^+?TOra()oI^iDHE{yA}tfbnbE*;O38$A-t94 zm3+6fnPFntj<#mgQ>X5D}1m>E^d zzYo6@_S*$H2ItH%q62o{2HS)BLqC!q#-T0lzgfIL*~25MkITy+tR^=|cIFDRTr8)5 za$R)h=Brj1aDtuh$(ePVHH5F4S`uICbV7haEC5T$N_;V%KR1miy%Tgkd}}q{R9R<# z9$ui7Ae&xy_|lnxIjJ-WlCm1oJ|Mj3=QHUZ>z`pk4r~{<2Uy3iT^Djv&*$Ak$bR1| z*>oQdyC2CUQTNmmcvoIcmuaRfG&1^|wrA>Wk#lt6*#J!tO~UjR~`$aM*U`of}FHP`^C(k4(c zlAs@Gs=hvy#u*O`i{$hcfRd~Oy>U4F5FXyp618F@->Ch6@l4x1kpV<4Ub~P(tNTT^ z-kZkG1|UrF)ZGB^{Z5;fAd-{S-u7uVgSC;`9c^UYFj_D zS6ryU5Y89@oor$N>wpt5wcU@dFJNq2qd6%+gBod|yX-p9kN1>(c)?| z*XCJ^&#d+Olg&quf3rW>qyXj+4X-z;mx9zS)%W*(4_uQSn*3tRkall5fGCvh&BJQY z!i+;B5eo#7m-BsDt?mkZyRCqO$85O8i39k53}4eg#E3JdL8yg8i>+vFh;JpyVm7qd zS@UqQmBG=t{rRF>)~)1n5~`*0#d?1Nhv_eD%}24!+BJgDPb>6W8+>EX6do)Ct%?Tc z;DiSkNzz65=Y!U4HUa=hF=dcGfm)^eG7k#>H*)1C+Q9EX^)uc;G5QfO@XrC}$dBor z$Etj1znit5Ub%f2^tc>j#RX5TP|R-G$RMJ@9$^6aH`(C!$zTY&fr-=4Fmi?6=}Ia< z@V|aAhLlINnbrLkgW+lPh44*{OI=DUF@E ziUdo_*;|jyq|A%QTL9k7i3W79F;S>+kltpyp+f{;qt$+G6Q6b7w%`bTf^kT%kgnzB za%co{-U)Hj2kkJly4|nL-_LUiC%!xLsB&z_25!uMv4g0a$mF;hRo{wGTweQ=&H18W zdz2sxKNKGYD$*D8*}hL?rva9|OfbI_*mSXeZ&CajlhpWO%a;6Dqe7p3{N55NM-!9> zcNj&OfLWhaB#P7~h?~8@G8fVPNPz9ktO%}5y}aN79fk885Ng7zS^zn>Um_FtQuD zQ8?7&Gy4@_fIE%SNyw`E0qS6!dSHbl4Rug0lxMy5c1;1KD}Gf9`Tsgn5lKJx8`Y$^`JQ4+8o~U4|;leVMwkH-BgZho-B8X)L0F26InN;ad>h#h%s6S z9pZRAM=n;(!o@heo>>g;AVxU@%%xp5l7ja%jO^O)7|?(^aXiD?0OU5z_J5X4Sae>f zels#ayKvy%|NKSwprZ=g0{WYp9*LauC0rvjaEin-VH-Fr^WBx zTFi&6&a)uhpD}DWjkC7A;vg$DeH&~EbGr9y#5Fz}#F~a3LZ}WkxW9=OcEE{|lPDIj z3!kg(fSztoD6Xc-qFno_)$Tq~EtjTB`%n{=RYVlsLZMUm72xgLe!m1m-7=8Q(JXKh z_(-SgVJ)_oNB4Gp^|3Ke6hR~GydM*u0ulQgRLjPr6K8<&Aw+~Lv(!{d*N?$Pz5ZmB3*M_gJb z{n)GPi|wlmN|^V?dPNgjHYgaWGVP7RV;SIR=<_RfyVGjMO+3)!{1_?uddnR>=~lTr z1dF2SxgmUlZE~H9VqPr+i-fyvokfLew*8$F*)L~>BW$G_{~WpKB!In&k4`I%eaZHU zczf(5Sl|1UOWRFkKt;`sJ(UtYE!F&0>+^7|3S?iFWp* z%s2*SWhkQWYLrWh-CP`=w%Tr3GkzE-NoF;W*Qi?9mOsDnJ62+%)18r5t&~Sg2=q2! z)jNHK2U`FD-uMYD3vOY_ywgP{2>bghO~x|=l^Z>OQem&D)0C^7i;P9Sjnkdl$_*dk zF^&qIN>6B@c%`=Y@ERHDz-g(jrK=XGr{5pw7=WiQ#G&&T&MQODIfX4s6B&InhTotV zgo^@n8ngSm+YWoV;yr}-=7k?nl29#GXEaRl&JM3yZC`7;@~bvm80FaWk2LL!#VH!{ zRvdqYG;a@SGh^+TWdlfico_Svw^sFguQNp*4wTXXE(ClSx|0(xsEsRQ$7=654{zh) z-Q&LyYtDiy(4nxnRuPsYdQ^SSeEpu7I3og`Nu#P3mY`E=$QG9SrzFc!5h|@*f{KnW zJouChB(38sByp{DCG%>^-k6ot-?OVXEG~c6Xl^i!$n!HA6YPCCqm0^(!3V~dOU~;| ztS|!R53TY^!jbq@T0+vu+a(<4DekHTt_RlTyl92%Pq^r=b-VWb!@s_9{Lq$*)h!i9 z>6-3Ob=^vKJ&#v~JC+OUq~#g*M{bA(#mk1!UqUjGPB{qR#Q{|a0tBvLlm1xg_aG+ z#X$e+M%g-9!cq$Fv4GRjFBSAv@Xhc7+Unc|34_OeMz;BImXLoJm(BjC65QU3^r7wC zhqT%gw3SyKLSv32t^T)At<7eIj zZ_XV=8i_u?g*u;_j+WMw&^9zHE8J&up)OY2C}jwhJy5SO<}h1)t(NE87DXn~198b* zPF5h9jnb9$$i8@-2wS&zE{3BFudle(9N%}_isOL?$sb2D$2*Z9@)Dgwl&$)+D$+t;p^|HmU&=;k!pJ~aFn?U*j=i`r*U?a!hLkQf$AHz%Ya;o-RyRNb8bKtPt zmcDWuQ~vFX0on0c)AL(Gbw_T;yHzvYmiKZ$CxAWG^XScsY6j8+hA0$?zb^Q>j#S2ZRKRy%VzYd=E;#uxl|u z}7I(mfy)GKLqAk9ESf#gu2AWYUxjV$m_x*DB( z1TR%el5Mr?mHiAZ5^OZ>5og8nBoL*68B59)>eTzCe2D}CLyY*ee8A27T(N$gB*#%? zXd%#UoK+NmwS+hUpOpb59qPFbuj;ZKQhH8q|L~LeH}CoRocF&Ek;1d|gxz%}O#L*4 z?T8bO3yS`s4QU>YrO8?MuWE}TIVvoW&Y5+9s9$6}ks9c9e@JF^ZRRgy#}Po0wl?yg4Big4Ve|h zH?};LM>6_w!Ay?)W;&%+CQ8fk05)Dg)n1^p#*NFVC^*+(9bT@>p?kR0Y}WXK(@n6} zaF|y7T+nH!n<&l9Z_CNt@jwUMPCV zT&F1WYJI=oM77k3MZ4Xy$X1I(3c?MdF)Q;m_QP`iRYLQ`^lDMY3DHTuQ14$AIcj%v zCg>G4>z1QdFh~s~L+Q|qzr-_bwU^poH-e&LMnRmQ2DrHbj|oyJy9xjJ-%=@e00p2T z0|T4Odg_;=OZHwnw?&IDZYPD#stX`K=4Lfq-k)=!Zw1K4VSL=~AE_%*`!%xUJ=8zr z?F98n8^^1(;jxY5qKC7^^-~rAkcTnU-?WNkAIx+J0w(#8WKbTP?)|GO++IM-Cqb|o znpB>~Ij_4CbY;4>S-~wZTb2E0;o_hXFa4{fTdQJ^liB(_5L1NpN1v4?v1(75w^?;+ z%e_qDCh9vJzP;`By-q*Npp}W2I8100f}9UJG)&nQom#aiU=1*!$t%k|RMqFNtLZAB zV7TY;b5gbs?Z8Hj3h+C$8FRlpT2XCvv-7SR&4NATfAbRnk0DPMf(ARHFQT7qc0VI~ zcyZaU)=&UX1-on*aJ*ykPleFeb`-Nyjy$!g!NevP(Ex0B`sDN|bzn|@;wym;Zj;7wZFyi3$7AgK0hzdG1# zztM2MI;CvN;ltK2*?ll6cc2c&fHeirU7LC6OvJq#F8Db4`g`R%o3;vp5($%tA6UV- z%LXcM+48)aYzK+Mj6lRz$+$>?lxgf*34QsS7%9u?b+u|E?N5E);B4P;y1Hj zy&HUb_62i9-iTHZK4@yxI>wboadr&gzQ1>2pg8X`CE z=i-;I*bW%?xORboh-NigN-mX}#B=vC*7Y^1Bqg2njr@Ua12VJ2Pajn47NznYc-sNE z#kl>e2-$ggz9IJWq$ORjoH~4-8+M+RDdadydSg-J55}~b-_Q*wgW0u41E*G>5=msY zeuougtH<&KWM#p*3xs?YY?fp1Zx;;v`fg#&NEzNH1ZaEk(mfvY(OI0|?i?4|{(SwQ zZon_kLsJZcHbA4xo1iDrJB!z~NWZG6zqi$<@~JLj@~MoaY`wL)5gp!CTB}is%T;Mi zo%M_~Ak#}NbmQp@pnN8oj>?(#c!RfC5skM;o8r8zCNqz&&x{_S_za@VSh3Bs>NnGm zD{$YB<1t9oIwBZF;YStvQG18|~>G<+b12UC6uXYdgpFHIRX11AUrv=Ii{V z7HV8W>+Gi0WizUMvl`~2j$5Zt-0PX^927V`;2Xnq4@098sY6aO$|0x_f-Fg?Y@-{mvfxqi(1SfQt%2Qte?f#PW_DVZI}i%kkXB zh8E&FZ~4U#9wTbrRrk!?R_aP#{sOZaQql-a!fua;tNl#RwOZXOXcf|@9{Xis@Hjy9 zT)co9OaZ{Z;V|k%N)z&KNCX7jgfEvL-pm~`lf>SDk0Fk0n|95X12j+$lMn}vxpvK} zTSjdT^Vf9i;MvY^-ZJY^L=&=oY#{bQNE}FhW`8VhjWz;pd2=v9O)A@1m9;5-dHTj zcdKqpD2!jX;_N3p8_d8R!P-Fd=hs-yzhKlWszDsX>961ek~?=+@0)KN%(qrc@(aca z%*>xH%sQ=Ib2YmhCLU~c@2oYT)SVj^Sql*@)`yhsA5T$S%NC2RnRTEM2p_AAZ;fRe z+I>Abpk+J|?g@J?1(C_tIlpVWD)|wX^0Hcw-+SKF`>+xIW|@di=d?H(Zr5EK{iD&< zCEfhQdd}Pqhgve3KI`Tet(u~4ZCU*Rq(RJYsc8ab7PhceCLhq`)NX!$JdJL#F*2NS z(ujuO_njrWzf&#)x-^76mWerpw=}r~p`~7sH+B+d3VU_3dT#!t(r&a&_5~ErE0N1f z%OhEnRJ(fL;_we^04Om9o0a$hr3- zuP0cDr){siooH({$+|mam0E-{UXYG480Ka0vzFnG+V?H!T!LKJ-=4}k^4nC8FSOc= zD$8V8Vs5=;vh$1jSf9H?5ND5|Vp4BNhxR?=(VD?`+}1u&LX?~3kGv8OC9P`N%)ZTL z8t=Hc#74qsE5S7!ljuj}!S!p!F7?+Q((AD7GdamDB}7|X+~U{P=S(#C(4vvZam8&q z()k7g!WbKRuBxvx_~Pc3x&6%@dOV!xmJ_zr` zlA;7)V{ab}4Fc3nDY={d?(b4t=c(-Oxyh)TF}Xm@z_hFM_S2ZeH8x+O-j9YIgos^+ zd@4R`tY+&lD^*7QPTrC^vgby(_Pkv)HBKz5)@LN9Ku{WUEAA&tLG)}OC}mm%w&yOdkC}G9dLS|k$uf!ZlptoK7VjNQVl4$>_gf~G~Ucb z=!=$B08~7kiAF#uAc(ZX?I4{p5C#QdDaJ%Ff+u3y+->Q>CkT@@Z=}{pcQ$=eg?X3H z0{Z^eOrjT$No;ILd0n1OGo^%Xo$0>dbxGt#$NNdPV6`X$SKRPOe75TyTvG+85N``u z^4{$WDbIyOyqPQX%=a=DXqjKRFL$?x>YVoXfw9_KTtz9Y8(oXpV7m3&cjlJV(G4a} zlGK#Iyh@|cA_EH+AR$V96pW3EhW747Tw{4q{ZC}_E9+YJIpS+7)M*l%#JbE_Ue8y& z33;hsmhk)Nf|`wXWTj;avVDU-8H2u5tSBqXd@CozDSRZGEam59GejWZMyjQ&{e!(Q z;UgJ~o8R4!0(i*Ulh`sNM=`|4fZ?<3Ek)|>TU}S~YQK$B6?OU^TWC1CWLm_ zEl{oeMdajo!rXUtD&06am==^p$ob-XEd>tbS=;TQjebbxN zqc5dB&ebRR;?b~AoeMqLO&BLA9=k-D$Y9pgNf$PH3Po5fw1U+(9kE0`i8kuG337h z0fmBd07V=FXv=g@PT~RR#m^}zDU`l|D$T1!38_~`#-Y`FA(1{+-l5lixlk@~yunn7 zCAeoY{}#pNMb0)H#Cmk*PrJJRYQSHS(^W9D@S$-w; zyLq?Mo6euy_xQT6e>~h@qFla$OCP-QTvz?KP0WdH^qz&uU9CVyj$OGb^`N_0$*@Li zVK$zh6NN43iKY7qKlld>Nce5M+X`HBJXmzRgn+(uGWfkzsZQpux{Ii<{}Q%vz3WLC zQWY^qUzN6%D~@W7XSuOB?n_O7Odhe49=gbwBlulBt6bk-(iFC=corU2`Am~KiqG#z z!^y-q*0`IvVRn9rNScf|X>E)E(bmoWS&g@_$4ND)v_kKo_hZ#u?5}NcR;>p% z;o3Nicj1b?eMfA(j?J&rXGeKo74^nl_+Fo?$aJ5fGaNer>$MFkf*eCRA?u%wF9Rfyd?3%^tF zrH7Gh!ar;T1~Tbn%#vU&a;ky~N^4V7m|c%6a~~oDPym><*rDjyYKd(XQ%j}{e0bSs zF8$o1v|Rr8e|<*N2?j)uD&M{!soc4kFYACJ45a8|-ccXfp=R>CK4sD<$A0Rn#>LKu z0Mu{C zFFgDFaD)q8qFJKAjmwm@GTKU>K8!~A^V-Zttx;oJ1mhXw!tREzst&#EYS>f(shRr} zyw~^mGJm9RZ{PZA9m2Ajk}%a>u|3Nx%+6#%rncuN;wrx(VXgC}%tkIo93%wU^xCW2 zjR?QxM|)Hm&p@Cc%3|1xMi51j=em*!Dxk-#j1NtzX3U{Nr)?ZwSwGa}%JTB7c*X`Iv! z5?K>oXhsn8sYSCv)=uHYlsq%qPFvytY`HE0MJsTGyfBKW)!$7q=aH1V{aAnZ+Ry|% zPi?I8bAp_C8>$DhYLmes9#bc%2=ur#rn=R*%mIbHA*IITZ!^6tK+|bjX)M2M5AD$3 zyGwiCg+Rx`BGgWmSUvp~50brgV*G8%v`{Y7>k|A$ZQ8f+unlzu%q z`JB@v*puY1^zc7jJpWw4UhLTpuLsQ*il07bT~lh8z6q23#GC+Px#L0Q8Ogz~%*04K zqj?Wo0JEvoYBS8v_DK3T9WUO&;w)$M@iZ8znNfCXOpAHj&%-QouG(f6IMBB z*B?3&yri0s`s$Etfe8}}#%zzJRKe9+n9uXcDtf{1O}481ja2+R96x_6dS5Jo)py17 z4kfOQ;+Ywqujxi%eZ}Uz{{G{ugN&4%wO%i(qy%(5^b~>}`<}(Pz^TEAGyzPyoV)m> zCOdDeA4PPW$C;~CoXs!&Bv5Nk(x;o#oLj<8oC|oHp6_u_maMYJEo_=ZubAEZM0m5#z za$`|{i=E~pJa=%xa9ymWzNo6}>r-%*UxZc7vU zW}!rxDE*Foxij)tkeu-&r(^(rMha)}f%@`7Z8e6^m! zgwUgK=~YKLcWic8K|(Ld-lhlTQ8mS5d4r&Fxdtk`qViO(yciW+Jy6j5+TN}TODlP; z>Xi9e1=B*ZhN|r190gm7g<2vd3Ly#TG5YK55m(e6_w$SMiOnyv`txreYoxv~8H4S`6E*BGj{kA zf!YkI#JxZ;1u6;f##vxR(_r(%=2M>e#&NN~srN>IzxM6f^nDt#|bFA-e8Z+@c0u2G;9M*@vq zHZs$!W3DK-#qOJz-DB0#w<3)wvr(Wj5k_g(gw0fD2w>XvzAQhg?)#F1)CGfXf6JeK z|C%7Gg5BgjdmjpSksr5);+z8^n{ba4CYQ%yqTs*;x4odmr;voL0cEq45sp``JNSsU zuB}e?LOkTKS&d$FCaWik+7X}(O`K|Pj3g7fDwdMg;9v-mEO!eVYf0K%d%duQ)o$DC z55g@LOoCV*rga8%fyb}kWamv7YD^pQc|Ws|IVMYBX*S`|k{pv95?(5l`NKp1r=k4+ zFTTI~@4x@`=QEVI;omTP)(JO8KJu9moZeMb(^ebCI=nhbaku_ux>iu!|5SmN!_g|L z&Gcj*JFr6C;*dAdM@9Rj`boRDT}>F{NeViLBY9D1PXE(0TnwB#vA!U)r=92GJZ(`Ux@@s%x{wM0fTE{+5|x=5xw65@ z5?>#$O!WWYt@%->y9I;Rva#4|SuV0%r|6A0x|#NQwRw?a`#$$;Oc(C@Sa1lN`C6{^ zhh_R>*8cO;e#bLkSj60o*|R@D*l02TG$$OJ_`eA1 zf3MH~J!x4}Y8fx|KIz!s|7`N*4ci|({_l_dm&oi-zuO6m5lvpailb_^i4q#4-msyx z7re3DGi+hzixt*Av@ByQ(pcANdJ|8tbSvIsVP^JFSZRiv8`|DZy-`Y4Z889ZMB9o| zJWxfFL)N0e|IhaLKfe&=CYADc^+)S0s`Q(#`_3`x?gvjof68=}`nw&kq8oE)lxxkO zTt&3xi-w=fA3$9Tt$%1pCr}My@d=XX>K=I-x8+ZgstM3&2CmM(GU=;++noL-j@23S zK)Cs24f}gM>sNo$*&4dP#iD!^qAEB9o2=%P2NNC{->c{U{kEB<$$u1=Mg( zOdn;!VmQQjqx1(HffjefPEAU4Z(EchV}JVsMc6m!eI~CXrOYx#aHH+NMZ$mabas?G zQelh*jS=zi2(GwRBecuK){v#WvtP3#sNP>5ol1&bs%BCF4kDrMH1RY&j+(WH-~3 zQgv39lCt@>_1DrHndHD<_iPq&9$)ux#u?;WNeN>41l6V0Q2DdF%ac!G++iJB51u^T z@;v`Zff)uFsBH0o&-RAgkykHV4eGres z!R?G@nZ>R2Zk=Y@x%y-}F8ca* zY!6doG%Enf-!Lw??-%3?leGSV@;95CH)RE{O0><&2f%>B9FRq zM#;8hGnwgh5PUDJCM>#}e|V!*MzQJ6%g>eEz58{jfjx>$ZUUaCYjQjPCOxSCjn-*{ z?Wu?Zn|k>q%1vYC=PZ4_#FD$vkM}|zy{~Nbg$YxX$lijf624~)crCRtf5h~;MlH#l z$TixDI67+yd8j+HT)Fi~h3%eEC-<@J`$}*#iSOK;8RRX3zSXb}Oz%gPVIHp`Sh+-|12> z#O&rY3rkvfY5kPi=Y<9DEL2Ha)Ynu}1a!hYU}K>QzPCbtSL7)$8{y>bt;pzuADR?_ z3!G-V-T~k>9?&hEu?6z}4qYpan1@5*&o;9Y79cv2s3`I2JTwMuN6pmwBL#Ii`nHc! z=0ttsI?R`@BP54UHh>ikWZqlTA$bAAwX*hkV6 za3?*ui4qmO`9v*b;>Pm^#x06nqxR%NRLkQ9|0%^6UX2;*tR)q&^$ZTo5cIP=&KG68 ztl|>>bgM+YifASIDQDu7F>imp&t~-@`mk=Me-<#!y z{)`@L|8qd`-8P4GFvivzd-u+k66w#V5A9Qu^?UMn9C0mn8^}WxJg8{6bkEnMGXgcN z8@HnXOY&j-XYQ|5POL?nH~9UeBu)A`D{mer zU46Hmx1{dIiWHRE&J?2Wc7vn_Hu)-u@6s#8J>_Xo*fj$69U1JE|@=3z5={tdtRu;--^ct{>$K=$P+X0|Zf$KA~Y*mP1= z<;7~dC^N(xUN+db9`uCuCU-(aH?DeERsGHaU6oru-~-br74z!N>n@QKV)FW3QgWTw z9ci?Xj}vm<=_3$~PdDEWJn-;Kmo9`fX2?LH`PnhdrPtF7%BT9@9Pv)`uPs#i@)*@% zR}8P+4m@Xnh9h!XEv?^b;0>c9MgX+!w<$0*&$n^R`HmG5BtigUxI}rUZ9$MHt6fGgIPhk=orS|&5 z8S0%pUDf_)U#7FL!VycgekIDu*IdFlFjmvyu}!<9=6X|$r_ z*cvDs9z={sPoqoDPjgCrG=>p0;SddON|dTts1~zQ*CY=b3V1&g&}5lPo;F-*c{O7f zy&>_!0Q3`Htd5xLMLLK{xIwbIqaZbB9w6#y&nb+?BK&CpaF^ z=IEY;&s3hJbp>O?m8R7mS3&rL!2(mE3~@_$!WZAa^ahwjOz}&+Ec>bigVN7WRHfdi zHv7pj=LkOrk5UlsE=VSPUKRc@^0UskoEhZ1!F;GE5Cde`QO&rEeG!*9Yu9ynubefO z$(NeNuv_`7eM-Ing$>C$FdK5n>UB>ok}BsDHOLLB-^DwtcNSMiU3R4NEuTNOvCtD} zsR@2uTNdOmnOGM*_WKa`N#Yv_s}Bn6BxMUNqx=zyYFNf%qfX!`Qqw0cjG^$ND&qyY3F1Uit2N~J+Nj32JCg`uyS8Il_-YtWCU@yifMXqn- zlp;L_(?XR?0yy_=%j<4B+~_(g>Lo9-p~vJ$^{HvE7@&;uxipji;^HA%P#3AXRqgig z_`VDhzEzWv%b;RfP2gzIfR7BYt_;la2|5|rJ)VA=;IePc%gWGR6grHcQ zlsTc(1k0b)R7b9leSh7(94Y%o@D0j-&D>VBIW^oe z*US=m1}gDCCn6^XH5|Uy!tN@{=Iw;JHO!0YGnrlcEATOIv6=N437PfL7$=)zS9SB< z8me#KLWDiHD@dDhrcX-_*`u-{8-_yjoV1K&g7yk7x&FMxv_Pz?)a9yYP@RIZdgk zy_y?@H(6m5o-(VcZ(EfrwW0%_b6)*3Fz9@Zd`p;vn_${o$4bxf9sL*ay?x&ifuu6E z)T{5*=KXgWN2i{c?%+2sChD+j_!tO0(~OAQ_Me^WpGx`&*i5meVPB z)`{RUyN-!F;{Q|1(BGw@Jn!mDyiHMVjXr~g(OBWr_g&~hZbB*s@ zUQD*U+~2vJ)AHs0a4WeI)Tl>paaz9iIY>L+<|eoNyY75*YvmdxP9c#hz37FGy``!S ze03n=-~&rUm&uPunXM!NKthYEm%2j#x!_hoKsRk8Iz>g?+>Au9QutdI^ey7vn zLl^{sN3V4zg-U0?W-TQg2@v~&9>4c|n{+U;kwTdx1&_XtEgBzft&0?Mu}_9f_QPyx zEwA?t<5P1*#lCb+d5-PYBiiPs>9(=MBH@w4L)Nfmb{jUM>6L}m%=f=i2!b;R@a-gb z>#U!YRlk7Urug6-@xSP)DegUGEsXowrEz*(dC$Zj9i!#H&kC8GwbP?#de?Xz^u(m? zv{PZx_mr3`vOsF39K=9193e+u#`Uz>a$GKnSJ>Q@C6fO~MsXHX0=a! zzI?IhjhU{{q!0{d(to(Nkc7SV-4>((b-{pN?஛=k-S7wPUk03VbtK<&E#-Fw} zk~iKVyW#B&Uu5x1oMYM;ydY8Z$Q?zj9pzSM)=oaHD@jEEvYl=xXwMTJUB*z#>$!&} z$ZdcNaGiGI^E{hvE#C`gNUyOwSPK%_fEU-X-5v@!PD(mvt&b+^c~`Bxo2zeY@$Ws_ zY}-FMg+90Vo^sf@LwdlT_nuvT!RwJ4e=_a`wsoA#6w=B>ccZ#q~%d(Es3rKx_{qP6Jj&AOl###jM5gqrQE2HS{cXNHPsJB4qc69DW3=Zlq?R>X-M4A-{IJITvz-mpz|iDL2(Q}` zh5im%-gRt6-fq0V9g<;j+4W9uvay->I4PrJtRPEyu{X6yV@AqoH0yikaHq@>6TiKk zo7t6}n1xB=+`TB8)X8>cw~6-M@mlv6gTG4>Lr&gy*R{|I$u2&NA+0^jgi>q z!z(FoPL>NU0AQBcrjrJ-Y4AF3EjEqo1Lbz&9U zgV)m4=`=8bNM~;u%|U}tD^1dID4AXYqH2v$EvIGu{&M~)NNenZuP;4{SDq&mXW0o> zH;oCej+R6pd*oWOSyAFfzd=sh7j?7)#SyIVyS2WUi8JXG8|k=Fy+57WhdT$GrbQve zv;nlNcw{(ee{RLjFwazkdwQlEZtOm;bWFK~P*Vq!Exa&=glIOD$?KD)hj7*Fl9ICd z6OH13QWJF{jT6T>^me!56MRdxlGcT*y4UtbFS6(FpXN`fc9C#N|E6SqHf}9qWTB@7 zr3Z`gwfMQ`>H*{h0}Me01LA7DVn`~HC}3A&Jye_iN2+7+HDfpZAHFf^`p+}y^%T`bNfseJf-22ZNB zgnM5oVxc($Cr7zx?o9I=jdlXDP4rW{0CY*Gjd~$R_fSiG%eO`CnG=?2!o4H00Z|k#dHh_iP5Q>Y9Gqr6t`}Qa|Ae)m{BHh);&l2Zb$GJZFORgSoT;-SPzYpHXs8yyRpJ*{l6*EtH_{*DWgL3Tb+w9+Y92fUzSa zr5u@T4~{O9fA#Lcs@rB+o^@xLtDNQ#mZsFgk6H?p`k@H!nGLc~r-9>^n^|S#>}>HZ zyfAaCjV#H8{0oQS9Tb^E6)hR4ts$Ho!_;?@BV(&leulaP%E0vOMUeIZ5qC2Dl3Kpc z3GXRMj0(uXLWZ!VlHU*>O~ocrr*m9H>iU(Ji-vdsxtO$4vqPT^w>|vLSePOAfUEmH zHBTh!_W1;AWfCa%gZlji6~+hT6^w_Q`NizW!|v?KOZ*(c@1`R=LbuXuC)+A{sUo7~ zYt7q@D(ce=xG+9+Csfj!H`I_?m-yIn8v(!Q`Z`E*E7&RZ8{b;6cV-9Lf;kN-E6!w{ z}lgLcdPM;BLw9aGM*di*wk~S?Y^* zN#wjG_W>Ift+cHy*|R7t^!$68Jm1eO*nvxd)E@))`?Wv(qM|nzd0>0{Fb?eayhD-R zReEdru?`7miJ$p{*_mBlbkyE`hMx2P$J_+j?Cvn>tY%Y|vni|O&wP_@#$_0buTByQ ze;lSp>^u`T>g&YkmcCRDgkiw#T65$)RP|Q->6|)h`eqH<;?l|R5I^$&v`UJ=+_?)c zBB4@R!>j9}db{EKK|qr}V_D}wu=Rr>C8T`Lmg9RHR$P3vWMnA;4`cM{vP4ZB8`KwD zV>2}!7Sn~HEs-#5H4sb8@L9-=*`!RYI?H7ZfBvf#%W`OxD06?3t~TwFF~7WIT#;HV zO{N{jU98Q2{W0A!~9Vy z9_Osd9Qi$bRf`d)vaq?0qL2tB5v;!Nao_@%;@D#82C&(rt9-0qaisVT?=56g@*xV* zfTijm7qyji()G&6#IjLle;^;K45rAHA(CMR1)*ScInOObf|kZ0SvJ2^{5h6dmYDG| zZ@()D*5a%FACUPO4C3zkXr<{NE>)qNX#;dlLvQUm)LPG(d2Zz0`i~eYHigSd>VpA2 z`D>XPSL&kRiDa4LnEhBOdd?q08Dj@eDLnF*sUf`xBD`D_;fzNY**m%|?<1d;ha#ec zC`Qgo2t%mVA_Ul@R{&Tep6WmE{)PrZS**XW)`ZP=mH!1wAZCpQpkQQ{dz;mouL-CY zH&s3__1!gavNTjG(6UzEO4ch;6k&`jw<&=Mo*R>WiA(B4x@A3ZC99? z+>$OHV@Kx35ReLYj#olN45RO;55aH}ELM zx7kvg3vT|djP7XwXp(SwxFR3V01tZuuT)k#Kz9cW>!R>4H@{rTA#_IiCB_5e}ybv7=?G9$Rap;HD5^B6#=Jn$K$W75tv+)OsG} zj+(9se12#4b#4}vz7Be7v-tS({mZJ2Ih`G0l-epqLujj%^5uq%Fd1ddN9GtSg%|w5 zYZ~1;V&=!X<)n$Bc)0@9pZ5;+CCqH9SxRP(1F=0wzG;X{iUF8YY~d!T$+gdW$f}N8 z!$71ZlcC!YDT`g7aSUYli1gQ&^klgS+z$QAb~Bo_R=xwG%04aCXvy1GwZl9e9|0Jj zIj>; zb=Xrc)XwnYjo&iu%Ah{CgyvRW=FAK5KHElCtJ(+!WlROX_rN9(tTE;SC32zbcS)00 z=*8T0w{-7fuk+I*M1dhq#<1x``*%d7j%#R3fK)gpu2& z3`OPF&3w)FJa{fgvil&(^enA0o~g&fPp;9n;~IXMJv12yw9$fYc+sJ8T&c7{5ZQif zJZ;5f*IC8(eJ`O{ql!jK{rz{yW}C%)y#S%Ff3X_b^=BB9F9uww|Is6U{#x_G9;@Uz z{=RPB`2A$reWm1?n#Y;8;jV&i*1z1n)q(ND*U%p~Hy$~nm-Nkuw>Lh>aHJ1me|U5$ z?=g-2D?95vl;nr1=*H>`T&=xk&e^78@TVe5I(TtYZ0H5aizr+Jb(U(8ia2w^n7?_a z>gd$b8aAAW8L6*dc+pHRRg^AkT)Jkgis20Pi=^;5y6X5*ib$#EqQuI~9^l_6h296| zx0xl?F>U}+?FxUZnjp8Xs7B~&JK>*vzsED2Xrg66=92dzJ&MqL{@$$U^e04@~Jel-qKv1k;f2y;qsNKj9h8)2Twy78b4^} zND15egz*fqrft>p=eY$8E(=N;n3!jI5LR=Iu~L?@(@x5es#+K%WeWSoxW0v@OJ#Fn zrm~Uih(v2z7gMXt40#xhkDSkO9X^>XY@35jLQudAKTy9)*qV=5qNQkt4#L)B6&=@s zZhrg!*?R5opIK<|N~o+GhMg`=BUe8zVKU1YXQ=7EhcR})cF_-^Oe~fx@CFyEZOkU8 zmPU_UnqXxX=Ollo0kfh?mL-2)!}sU67{6W6>-5&U&BEZV30moRQ`ouu$w!&cuzNK? za#6O8Ml{j4R%pAH=@#M?ZX#i3Q)$2vOe#)lFna}7tiqUjy*z^WXhD+80 zhz8u7IMafN0Q*^QuLyy{3g?Bl8w`#d*!E8L^i^G_nY)*EY5zB7`QLn5$f#Crx@7ai zVnewP47}w1Kx-cHE#YKMf^ZYJ5)!S*b{%1`SUJ{z^1%uE4RMK1ruax1=ySr1nDih11Y>PS8~5|%6^*^4JG-R zHLaRmY_g2D#r=vz=QH*|NHczbRa#9m;PNkbc()6&lI{klZ|eBDQh6E&LJ_hJNAkqt zIL`vZA5tK7SYlZisYpW>#Ojc@h)7dP#4nRRgX^%kroj96nZ79y@e@y}fjPx&G&T-B zVJy?=vbLtoOf88k4xQ{PC`z>)12k+5wFi!*>Orcheb-LhTUJ^i6e1SkIxFg^Nu7$P z`G&6{I`Dn>NB*W_!~#yS%3nal=sF_BEi*>VQLPSKYpq$*R5>ZPgq_^ILixxmIOxwB z4$D?kgD*t^O^wMyl&EK=D3;@A?cur5cyWN_w`V!3Xr!?{R+IA7I*PgQ&Y`Tol5xG8W(6s`*;|0Qa_fLAlKKFK0_Ot3?^H2@E4Ul zd>AJORP7cq1#T16eRfoyHy&pu$XTrSjUT3izthRj=avtsHj1gvcIqmOpm_xt<^^Xc zO;oj+xZui%87Orrv>oo-3k5)>M=Jv5#v=C#_q`|0rhR^pVQIb5(fkQe0jy9nG<;LO zV0Z0gQW4{mo{l|^JvP881$kA#(gHOBnbB<@99}Lygt1zWyW1=uL~7#Gj=^AqwA*Vg zZ|YBL2cUTH*Sz93zaAsu*9Pti&{*FyiyqI9?=^Zvp zbyye-cuE?%c>V5?+r)j2|8f=+e+JycgxWhBU_R9qJ*;~3LfOa@>J|8oDyDU`Z?1lA zBL&>c6hC@4%^%)H{+~GmKVttb<@aFRM|0Mk)i0ER<;(M<8QJRr`;Qpg1En)4-RCaw z`~xcEm?)fo1}B5LP2(felH~IAN325d1wQ(3bfTH2I2(JRUEt~-W2)DN?VgtHGc_3l zAy+7ZSv*hlZ3C3pz)maeX=<~D$5l+-BX*IRB_qs!Slj9u_UA#SJ*P|{8T3{T`jkpJ z>|>dWQrn3qip5q#8eHh8(DH_a8SyfX$jV@J1U_>W0q(>H_=3g$at*bWu@Uow^mMdC zV*mBBxWC7cRDJe&3R!hsc&~}NrUCVkQO8^+4 z_-_4CP15%PmF~&$vhSnq6U4WyKkio}9fZB9wIr+Y-oss-6!`@SQrW8YOT4f0li{ol zcfcdEazw-1R2?FccMY<&U4K&7PpAH*LQw{R6BVwC3N8^AF1W;SL@jiF({ z7G?rmQl})MZo?{rWc6X6?v|S3Nw$FZ7r`7uZYwhi*>-Tlqp^KPZh7wOHfih{b8TuW zR2bo2h9k2rTIzB7J4lv+5q0lq zr$wa3)2R%z*%0wquKvQkhAbjAafsCCv=l7Tj>a>U@|3T}R*G-ZxIA;GH&8(n89-|z zx8gN)G1AsTP<5Sbv{0KwEtacA;o@gh_!g!1L;JgIm@%x1GsJyaSDZldWmjd9}?Z|D@^s8&+x|9F8q&Lch>_p;mzD?qt6WT+< zIpKdj6Y8>8XX0&duXqIn4pIshDZ&Ou&Kaz*N@EK*wBNZZGSe}0$=GioDAuh{W zNIHDX5Z^@-eU-~GFtR9_H9vWsQ~28a-{P$QgvK`aNwekOQja3}w+20yBF`ZiRT-?j zbBioUMh8P^#6%1uRK`LKv?vO|eT_&_C`pyzP>~}DCVI4{&T0ns59$`AH#<`bB4vCo z@KzZw;ZLcUMN6PCuAubIQ(})F+^=G`yS9$ld9Mw>sLvnt*bSK&Qb1l&}cQAF4$683u~S4S;uDj5no(PCBi z2gw`-tr!+gAkKlY`EsJPgCBfk4>;(74=s~SEeoTDK+|+FbQ_-base@xw&M}}J>~v7 zct~)>S=XYvbqZpq2j)i02b46-QiZKiy(^ONaZhlP8_lsiaytysTr#T+B?YVa5m23v z=c(tu4W(6C%!J3{t5RZ9#cFH6B@geur+a;o?LSiu!`H112lWTCWJ4r)|1Y1Z{GFC^ zcm^juX|`~~Tn5?|f^-P3nd=@EWeg}s2_!R08h(JMAA`=FrhF@nyEE}QWlTyb)X1$7 z9YOeEiq;e1oE`G|usQg*1AmxvMx=~g*~a>9A7R=ASxCK8$n6#BFt}xT$e%`rw*kkD zhF!UFF__^&M|;3t^-L@THwqjqe>%uUh&Rd3d>C37yf)73Ze0zS|2?L)LXe z@kv#prA*rP_M#YC_XGy>qC2_>Z8Y#e?ORdEK&h|HUVq*1j7SIQi8%gbrh~f8068pu zFQj9YE05mk`C?P87c>ObHm_ZB{(2<9Ph1`|;*Q!Oo*Qu~g-KFn91?j2QpOWB zodjn(MdOu=FIkrIQ$!a?D=$=)aB~uI3sCi!w~}OH$h^6x`rt+)7UIN3$Rs3K3Via@ z-&(J^{oCk1e?Vq*)`6CA9DU3nVRwk4ce!9M*4hpR^Ofgw-oNl}J;QYp&rz9H70myk z%=)6^9V@XSLVDLw;`l-4>_V7)ka4bAQI6%!;D#eZ{9FWd0d z+CAU%oOjqRzMbMJU_B4&G7rS6%+_M?UUDxnAAp*OEOuM#bu zv_`qnuB6&Z(+rGot&PAh{FT(UJUaWyV6{@RO5<7vJ9-J?iZno>$gD$*jC&9 z#F{z>9FKqYZ)mZ`8$D1G%g4XM?LH!koJUQ5HrM={<&B!M%z$Bf9jko`QD(7j+38Zy z_6PX@QF0?Oe~heyg2drf*=QZ+<|9<9;UtENZ?Ys!&pLA&5*&wvb)%M=6Q4FBXH(u$ z5k_GSFZM()f{a*?llFZW4>U93w|o6Lcbx{Aq0HSFX-hMfDt>UwaY4492%GQAUQ0^+ zTvhnzV!-Ap#LCU-9?bC9*-y@tZfzzLh&i{}U4`6P(+f*((eMEdPMwO)`I{hnk!6R1 zcmCc(@b!CKv>0oMz0J0&&81-`5tk zk;q_7{Th>Vq-~u9s)UoyF~pt4`i+SK#Iizml+qwy^>k1d+x}mM?iICBznmZZi(j@( zt$0yLVZHL0TMZX1I;aIo4F#K&%GWe%K$H-{ZhrP+x2pL#*6FI*bnc)_%WhECF5Baz z&U%}u{;KOZ#tfUW_3x?7Wh1MrZYUZO;-BXu8!e@PfxV~B?*<>xm{i5u(e3KA)dmH3 z$de92zIn%AnpEnJn-}vAyD_YuyE}g@;^?SUs zYD)!EZ-JUOkLEOU{S&mn8MivUG>Nr_x@ZHvV5QP>{j-lp$A0JP ztSyOx^26@|l%AKYEmScbFFw8Og|f`4zuYne-W;MS>G+XUYiV79GPmY)0_eY=(Y`bfs%Bp?CtlaIq(q0$RR%?@ zr8<65fQ;1f`t8vPvjln32%13SgyZA;ZxV?=)gQ~UqMs+s%6*?6ZyPsY0!fKVVzJbl zo(_*r7tRM^>ngU4O$a$Ge>TT z2e_e}#L-|^scXFJF0k?0<4YQ=u={oUHcayZqja2VycZ4aibw27I7LO|c9-qf(g_{n zhywf;pUB<0jfsISzyMMt*=t+p6}$e9T#1tC^S76f``^j#=bHdy`H8em)l(VIA||(u z>YFEH7sic+_+uHam@%po;RJt%os^(B)m~IJMq+;|7Bd1VLsvh zO9#j9NfSXFcY;to)ZyPT+6j5#2@3{{nRV5sPW!RE zi^e4K=iuBpwr_GkxPZ-d#AZ|mjYy{|^<|fPw7$S@fB-PeU?;1o0x$pS*|fnvA0(h~ z6>PklSMI}g+tB=0c8#YMjhJJ3V*Gt!Bp|{Cb5Q@aBN-3>X+LSm6WDr3g6g}#A@kp8w~pyLgYiZ{_Op;oyA@jqXEE*k#%;gpqMr>X33 zR+wG*4)#^;)2!7q)P2`x+Z8evCIL$tnmH)ViWAb8VQEd~pdB1RF!wqJ{ZOl4&dc7B za*?>{+T^XZ<)eN3hURkA=>08xU!T_iedj{4WGR3rZEynlHYT;zUc>XXL{iP=_2{;$ z-M9ESWVj`#qe=jP@=k~JbE3t=kdOyCkh4j89j07?(#U`m!N) z^V$2-e47q?zLWFA(%NK$K#8yHRXROyxLMm7!saWSy!Yz>QhwXIvU7Ucy@S5t zQ*IM})54+;9e(25V3E>&UX^2F8%o5>-PezChXhTSSUo$lW$UC)DW zaM~V`{!bUc5OY22@*PnSz_+nim=Q_yJf5G`GdkmUQMC8g`zFH)2D#9Ej`G$y5U1Dc z2k4oj%FtFg>?{`RveR&e*-6CSeXZ$5_~}{S8m!(Fz@lY-fW_E)dk8CHgP`7`l)C&n&1m2@b!FS1zbO^U1S4q^t@aSLdG`Xcu)Zb||3mBx- z%iwrc+_!OED>Vb$?Db0ONCb1qA2tTRI>@Aal5Kg3bhF z)a-v45Sau|hg|-Rc2|Vfc}DE#=IPZyso3ug)NSXhKNCJx^j$J$F=L}`8Con}GLi#i zLTQVlQ4=vN^qb@a7I#^8_$>`^+JLu8`JkHG{*9e|<3sS)#eH+|>>-O%dqoU|8$a*W zWlxN5n!_VGP2vz@LP^AprlaB5DuPkkyGkK{b-1nC=6B%r^_`F?cNK)i=#M!){_EjE z!DWbg^$izNO;Z1yazi+s!!IN(+{fQ&WD0V8yqVg5aR0r+>0-bWi{Yo!?DKgP_s#{z z0g~YCma@Td>;=B|NCLbeC5F9y9fYE3ue{iXH3JH^@7q)oA)R6q{fq0iXhMii$~#Pk z=;F@!+u!;D5T9(Y{#H4iw;I~CbUu%HNqKg{H~GSDzCwWV<`ngiAO#P@v}{iN`~FRn z$0Hp(V@Bu#V`eCK3PIIcF70YqLVxQ*m82vP)r*4;>*(my>AU}ZJ3gnEMN-3Sd>4HKp38)41Qm|MW$Lh&E) zBNgZK?OSr~+3f?;ot~$~U!HGaUHsqlM&{SSGQrOj@vPKSyXQf3MNy#WLwB@Drn%N_=_RJ3k)CC+KEc;GsM z>HU2$aJJ{df65SEQI!asXG2JXWxd3z8BADdRmfG@L?V-~;iJu~AvZ^K(+iC0VLX{Q zFqdrhqd(g`a=y89#ay3kv>}F5B8&E?du;0E5)L1@HTnzTshf#r%$LR+4JTShHv%f_4)Ro&664+RPZkj zRpJF;Y)Kb>kBtQh^*U<6^;xQw`M zT-rqDR@xH`%04oEM)K^P^Z(9&h~PCmIFZ!0ShdA~>fL_knxy(r?_!Y4e1TIssRe9Y zVblLEZ@w~I`a2~;K)koc-M=3S&(q^UK8^8pB;fl>nmXgfH;HRsY>0&`x#cA({Z>lb zxhGCKE6r)%!t06}NmMPP2@yZJU37<=V8H@1oFuvBc=6E!I z^e@kpdPOFy?G%2WGPugDkm_xaI6U!}gl<7U%j+XYaIG6lq%I*_%xGASC6`Cg*5D*V^M?efL<1Bz&w$z{MZ&^QxAF0~uxCbg<7l8snB zj7={0A6o6r^nnb>c|+b(+r_H;f7oP=$Rl$AzZpGY4R!#j%;tTk;O1682jT|837p2a zJ&m?0;5cT)?$tgiNflu)gZ6IywbMX>jV>vwE<7Ed13r?JSwKdlsp7J1{`OA&viECJ?NA|jbKmS|8s!UZKYCPEsc3eHqzr}mOsAGDqj zE|R7z;^k@i<(n|BP5HKg=K{<2n$gpp^+Cr$Ktsga@FUYViqH#8uf40--qr&47l^U|KgRQaOaja=Ko)f~PM30jkY7{|MtK2}3 zdpj(2xoq=3vbed`2$9VX@V;9_5wzckCB5Pz5@=%_cKTw_@DElvUjNw$c>c+2maLwt zj4H*wT%(Z-t;#nJ{VG?&+{hLDGM0}$9ubTwz1XfMJc_q2uI&>ufEuAwEehTh7|KY$ zoSneoce4)x%5;F-B(i1I)igp!UX2p$NMhRLiTD*_i%&;HeooBUu?n`^_adgRcgmg^ zyV!86!N*-al9VJZ8G)iAkwiS5UUc{qAt_J8jlzUw&dTP`*uu*7ASI4ASm;1q3Crb1au#3JT{@kT4oI#v9eWJ%<99$cd>}aDb5xwUpiOuSh2* z6!*qGKG)MZ1qaP`=Gc%(S0xzAz2!J z3^*-mR3Vm({&qlwtH*aB6;_Ml4&qianN6|h5B>SdpARE{7?146#~6kHqq7v?a~f*0 z@FKvXv`gsyl7g}tIst&72p}D7jebL-M(SD*E5hI4WT7LATgRJD+#i3Yq zKYP*a-BICUm9EDms$Y>?UBpg7^CQvSvOnrhHM?6c!9dK@dSsswt(_Th*&&p&mF{_N z7KCvKPO$v)nOxW>%vhpw?E)^_F=)d_l^w)gqoq+(u+&uuu3+p?DbOD02y zxLqGGa< zUzK!5yJmIX^N|;{Xq2Q-TRN7qGC-(1tC=Mb<3NV#!ieDcdQbl{_IlQk?s#8~1@W^L zyxDq(;}iAwxUeQhnxqNhmFVyRl+E3`qI8oQzN<$iVjQU?8d@4Yj-&Yd766tdtI5>a zXZITg)a16`s&!i;be%p##j?JFmBAPk(v@p!RXW@r62-Z;hl|1j-l$<~OL@%f}G2vv+wPP zA}?tjhX_+7_OD!BFGbk8v*voFG7{;QkRICc)&DL6QdTmP=qZ1VMfP4)GfhQ)2s15j za1y`Eah`kf|6oSJROV=}%U3m?TSw#}VbT;A|51kM%_?NJ6)b5mdn&OlW={m)4C! zoic)ev5X@g7H~P~^@d;qkP4_MIyVk360noE4!EH~x4Q29j4TX#Ib^nFQ-OR~?JN7%#aa>P-Xr|)q|E+i8YHxddBpN*&DwWLUSMYL z8XHp35BdO`@ldiHV!S)FcRHTPF{*Y{JR(1aIbD~lRo|;mJM;QoXV?5fV@koZ9lfbI z{XrOd2dwX$-Uo}0u3CQV365J;+#UhpCTGXpWj;dmGwq4^sqy3G71xi)27l9=h+7#S zx*$P%%i_Z@yg(h1mYN2m8JWtYbD>GEbpS!r73vU1zvykxSEgcsVCrr`$@28hoB-qQ zbpfc_EHH*!(%Y6;J=S{AQk!9i8Thd48h^d{e!_y_H(I(sna68dWwKK`JcD2_Y6r&`qY2n;)dlo&GNdNmsG2G>)6ible5IJ5iUfhQSIk}(qEMt%m@YD6S-&eeL*-b-WY z{l)$fbcl7~;+3D$vmf!>o#_CrBzXiqUwj5;{S61UXEUNqgSoGLmj5Od==|AXDU z@rUMRxd?Rg@K2{WjwAZRj_HCHHuyb$WvKGRsKvN{9^2kS%fJ2KQO@O0@89@Hx&aKQ$^^m z(Zv+}{VvIl65x&h1mrP9c0nbP3K8+e^}$xFu{O9GSaYG9glnm*S`igggBuk^?T653 zG+!vE?}sD$2Sr5y?J+!o$3noLO<7d-ygkDmvkbS_)9pXY`8>rD@AmiS~E)be%WvG{J?94n+wm!Iek-QBuxFY zm;SBTdhzXt&a7|Pcz0=UBXesjPs5uw^gVH>*uqzF5v_-T=%&*mF6=j4J+B9375W4J zvO2sxr}|Y3j>iuS?tB;?99f>f_&-PG;Q%?!l{#3^o&uljVY0l0_fJrcpm$t-6+eF= z-JN>hBLL}IX={y&Zn`zO%B;EF#Heq~fa~+3qgE5)HKMxK#H*%;jfu75M<_D`6Arni z12XHqCphluW_^7f05TRgTw{Fbk4fnw&%@2tS(zS!-?kizLXGV$a+jtqZ>R2~p$<6T zaoY}4`s1`xBi6!U0JeRszMc!U8vn8)Hvx5d?e6nV8TfFjy`y%fGJE^p_Upk0i~Zpe z#|OVy(a?#Kt+DD9+)?SpZoClG3JCg&zTcXKv=U*jvZ@;cMh@tmeYBmf~k}L{aV_fq)3Ue)mWcN z6c#UF*C)&=jcC`3jTGzLG~+7K?jhzXxke?ePD*p^S=irSf+ABZjiSaF65=n|QPAGJ zmXdH_s?6YGFk;-N*TUP6I$M}9a$HI3?=x0XJB|kG_sB4JAm5ei`vcln5bu{x*i%o1 zFT8SM)vqpb<+qcSmVPwNc1GoTMc~`q;Jfzkb<_N(JOKhSZK6bBxasvyuVX3xisvhD z%lJ2@uK2ud8}h3Fl8Cm@V8NIsucJ!x&tkG0P^CN873^Ez<#f7!+mBMsCoY*}!2J zxso}n>%mmv0^t4OF#f>1O5(P>YvIil6~T%Drjat3~^wr5*2vt16hDVFMWpcIQLrPe& z1N7S-kVzR%y&N-+z1|ZQEc7@#h$Fdgu^EbY6jwFv^ljPFT!fMYF-ind?XyovA0W+n zn=PI5;{?lrR!A$5TscAhQ@x;{5>`yy_JAIx+su?{(gcCv85F4G>`ExIbk^(kgt@2s zmqX|%E|2nwp&DBjfAOP0P-irb5?6Cqy@MrS^xyWq!KGr2%{cMm`KF8z%h51R*MW?r zqb*m6a!OWu#q0J(y(F6ODM}1-mBFUhIksGE+D<36nS&!~Hpd}oVmG-f;*nGBS9O+FW9WdB=e^YB&Y*e>Os_7xnR8&)%N_#&cJSHl{dZW#yf zHf9bIpG5)tb3>Bfx7LFM+gfB;Xk(x6*ZOqd)o-S5q0eHTI5cjz>QwC3G_4?DbWg;o z`dGtUhm_qcQ78U0BR@=8!O=gF_w1d|_3;G*Bb>)sIMzyLIKzohnP4p*)oJ%_W0z@| zhUH#k$0UI$FV8w{ak$er>-(&4Au7Emd)hT+>wnLMn+|4npZ${8Rqx4IeZd?i1mX@q zWjW%XyrA*ohy5Krrn9xa;b>qqGyK<%FK;tvb=gpyglu{L0LX&Rh4Y)n#1^uBp<8w9 zUO1`yGg7=kOy?nIuH0^4jmH7^Y9oPGrqQnw8h{f)dz0DC?md^ZkB8Ay6iR>`{va`t z&uGfIf#qpe`*mc(a_NvzsHO@yuX$zHox_YNowWM#QOB#qG+;dPJ}Ka+8grc25!eWJaHBXN<6JumWshB4XFDM;y{fkmU3`}=ZRQHF~X?_|EU z{+Hx77&Z3!iw+=Ro!HG(3yvO}7w_^5Tl}&PMMu@CW+NuY`^36tm$-|w;)}?)Z|)9m zMG}B}>_$?M_Nvwv^0Hwm?B$KQDa^xhgj*xabEghvLa^ElK!ViF z^sV8m3s~t~kE4$~l!n<7D1!t$$^x9oB>DBf%|rUT=GnS5VRXYJnL@ZQ#yHC>-eH2T()*-EP%WeFd& z|5|uJ?pqL&r5NWpTT~zLW!G^%*Xo3t$oll=YCEc}rHMH*!i#jg`bB*ri-obr7OT@^ zTpwDYM$c*1eudC2;q#Gi{Ky%iOIpe@eF{Hbg=dt=JIx3Q3iN0elS+HlIzsxlb`)7? zO;ZK}fe^7%Pos^!xo`av4M$_*RhE-c|(NUA7e%Fx~r?`|IubsM_@ z^1FB5^pyCzrS(5de;r`t`?gQ^5uL>ELZoY9L0+?^tuD8|o*m9q)|i~w^y7B{HA=O4 z&OOOh?vdzv>$lr3Mlz1P%398`6m}vb8tE0)@0pnu;3EENkg6fNY5R_ync%XHW>HwO zb*3KeN7msL+}^$2nAte#Sn@q{zq@df#15m3cwtRwN$;&^dA(E)j$M$b;ks-1MV_OZ z4yQG#DLpus&aGYMHWu*L5osr%&VRj_I8ivYOlck5#8pC+Sk^T3IpNk;d{@sS)X|q- z;BSy?XV22S&#qRx55BL6Pj-VG@{zbbvB1a!c@;Pnx+Ka`~5>^6t%;&wbqzzo{v4$^WtdGyQY~w zRf?l8nAQn{I7*HRmA~#w-CO4+%N?HIUAy*Hr(Hqiml|yLVGY?xW6R|ONls|-D0ccq zB4aRG9}+q1<|Qi|L+VLcT4*_D7&8SH<~Im~%YV?rkBp64duA z-38g}7~AU}JJXsDifV0)`Y`-Vwh>O!nvZwy6`37owfq8goO4&1L@UQ`B94H-tfzNx z50nuNF)eQ~9;zsxqhQBp5+9w19_uY<@}E%T7a#-^ z8Eg!kthULT`#!>Ka^G8&o7#4D1xoSh`}sIUggR0qwIM3Cit}=LkZE?ACtl^;!zpg| zD$~0bW!~BI>WX+H2l1eOb4((+9bR~0Xt2zXQJzkvNg?HS?fm-s6`XmBx( z-tK@2Kv#oFTF8dilklFey+$2$76+gp1X^Dje>o{y#lja>q^LX}AxPclsDIG6WWWG~FGISuo%kDUOKnbLVxjlrvG_qv%N-kLiCvfNkf>2S*;VBSN!^)a6_dasv=eAzKa593> zu*x+0Ad1nQKukpiy4Ps`yW`}5OKaad_UQVy2w&HT;pFFo+@7nojKJq|jDb5bh`m^| zL;2`5esxhGe9>MSodOf?7XZYgZsxV#8GfcPAXwSY5BlKO|hO z_@0YEFzm2&nKLAidgB+t^tCDtp9|P{-eo_$4mUa&dnj6wMn8-I2T9vAFc;Bq5#{J zt5h{4B%&*Mj`{o^05&`|4{--XHjl{>%lUH@+@!3l)d{=<6O_D=P;j2MY zSSUm~r4+)oyvjoUYwo zOTRR4T~38_f<6RQT;0-o;3r?_hf|?W87&Vz%`u~1@J1tzIw2hJ_PMOsQ+c@e)w{%BZ z=yIB=lR1ubGuSowQD#!J%Wr2!0!pbqlrID5!&&0i07e6E*{wJc%U#;Dg%(B{O2Et* zatsEo9#9z*`{gu&7sUilWz#*N-U;hpWIdL*^*~cOZl1*wXQ!$znxWtZK%V(PHkd!8 zxhM84C(9!5#QM(hyChOj1oe?ZZSqe~mNt=;cWRYFYKS;`5IIkSsQM9SPok4#2$6Wl zN*K@3l=8-zw6O-9P@GW3qbT-_Mwgj7M6JtX$#p-%lX*kpd(^|)3pI!P<4JldwVqI* zy5oQd)Gg5zFT|{3)H-(F7)|WbVmz>Dn~vT-T*Mw${d_Z)y)ge%buZLR>C+L}amR82 z{S`|LyH6Rl{-hK(wb{{9gjS~H4Vo+N!U4FBG*VepMGQ12$sT)L2Q@j%$sKnJ&eVUa zIG7<{zm;(+_L&IG)%nz{CzW;+&VrmC)A18^cj+^q>7{lFL4!Cd(>udjx(|06@W<7% zhSkr{AE$O|93C>2Y3SO1J@n~b@#M|k;$ddvkD*X6oU5VgX$)uQTcSv3A8P{`oDN({ zj}!>!uNm`Q#A7};J&z#mS4-pbA-YCrbJL>oWWX3b z5w{)X#lmXdV1r7M?yE|dR#hk$%A0}h0#I?;)SKqR*d+kA%`IyYSsYflK!VQC+NhCk+;YCq z8SYpa2M=FcG78q$3X`b>y;H7HbPt#~bH_CgMt{_{I(Qw}-Gw?TSNd7>Hloqvp5JZ# zI+b2S1h!ty+gd)Uk+0%_KVh8Qg;}_YNAYIeSrQ)kif_zzE4gUgp=6YIiL_J*+SZ9c zk)C=DeF8g$uWiF4wr94bs72s_4lI`WT2j4$UR+(n&kY~`X;gg5L-_kNdfS08cC`>k zx=P~HV53{|Qm%aF)$K>WZ*~68dK&Wnda!q3t#sBWFN1ZWHBPQp`e$)XS*X&Pph#1? zXKaHyPt|zN1AU2Gmc@aHmd;U&XX1)W1`Me%hSG?y@Ee4qXv?FlI9S@4{*LYUP4ge8 z!W@P;Q*l?|=Qo4_8qP0;H2!4@rG;M+4m-;RK`cxWQsz8J5C@g_Sq9=Wq-lCQz-Q{>F6w7HtHgCIYL zueDF}ws-C_`1JezAM~odtXDbA9i0J~*q&Lf;aL_KU{_U<|3ec03n@-nP%02Uc|Y-f z{k~gn^Ps^5ANdMClD~)eR9Fq6z9bq^sG20yIb}0kbd3pRt@Q$H-FCM;x?@fI-I{<< zOSx@#ma08BnpwTmGIy5oAlV;4<57g%7uUe9S{**t)>xtgME|A7IYahc^2J)gHKli`8 z{DZyUx#-IvtjT8S6902?{KN0~SfzH1O*>gnZJv=AR9MfEa=TvN$3zUj{>vu+$A)i{ zg1UBe6n|?-^oQ5_=KwCgLzi^snA3x6o>iVw_jsx`LBQOI8S~4FHrV->LF&2azs`LTj;i2-I+WCFUnZWS58ZF3m>zL`inl0u5lFMgteR7Buf9J8k)j7qJaMnMweCquC zE-87um)?LeMr2qp^FMqkLH$mNF7oj}kFRH$DCxOKkJel2;osx!jZA8T+)t&ylsKVMlo*K7B$v;4nvU4dGm zLV4Ia@AohE8}dMjzWC5FUF)7CEhWwsy$9Wt`R63yf1eyfaP5Ennm>&Y=i?{~%JusP z_?SiXep}K{(>gXopq>AnSymB?2P&7o^w$?>uj= zX@y==`A?U>eO5@MA781z@}l6n{>|q6=?aX5)_Q2`47@6^)-kup75k`pVl#Bo8>Oap ziEZ*NWn&Dmh6547zCd!x-gq0X#x)tL<=quXU=N+iANS}Kde5fzv>)y&D*G4jx+f^UC(0z|jSN$hw zw)mnqOn<)1KgI+?9cMYN)&KAhf3i`6U!cn4GAV65$`nZYJ6=4Z=bK;_>8Ik`&|(2Q zxIa9V7LyYqx%H7<-P=Jc|%(CXaKH`>~KF$l(c#HEiAM%d zgX*c|nEX+4N2lH(xAX?vnjZeXvsH<-Xh1BW9p<)kV21NY7Kq{IKR0&7=8{qCk| z<9T3!jM8y70gqeRV#(9kij`@#^Nqo$qXxw}v2`glRMnm>_8}?9ycS9Zk4-Fd9veV? z`ya9P>e&~>T*l5YgO%|AVC=}(25#F4K1A^RWepIK`2>}PWUUWK1umajwRWOYU0Xa5 zFbW)cRFi~+S;kcnhSM@IysF&Y zr%E)!N*Ar%d*iW#?E(q!ZUq{xR&r>9NS{yr`vPPY)u2N@gK?(>Bjyb{G~6ll+_)<{?><6rGnjBZ=bx<}%BYqo5NZYVJ4iYXSQz&_0~+WRhx zl+uvkkAYwX;*Dt(hz@O!u#w_<`A33hqBvUbK~>4;%+mfrKZZ3*}8lhNkGta zL+q+H@(`Py!q9)7yvdMHMr@BQBTuU~Ewlbj#$l_FKz9PJjcla9Flr)I6GM7S}`2$r@EfAn@MEU_x2WLVjLr)+Wu3xVE z($+3bpzVcM!j_C;B_OVQhO1O-XIr7}y-#6w>VAfOJV~^KD|{~;vSDr;Vo^Q0)D_wH z4oH8{Mhlg3uo`X^LgXazPp@!V1gAfu^r(P{_KUsXdHDwg-AS9>1GNkvo8_|bCM%xJ z-W1yMcvBd|{05rYM5P-ai9|AVh@mt;nEj-yl-vCgngyq(C$41fOQsXX3{ZJ;_j;vr zJd)wo4dh;`&iy87s-U;^0ZFz9R8IIVk%zNhAcvEW&>;TtB#croPbv;&&-N4R?9<|! zk^M^!VxM$Uz#8a4Ko`l|CP&h_HclUX4U%gR?Gk<6<$FYsV~??A(BO#GW6Dr5uIK; zXDOai4tf;(DRiv$?PeFi=k7ettVz0O6=qmDt7MkDi80iR)qFW%1Z&E~W^wZ#ak<$& z3FTfj-&rT-6)u+}O+!=D_k*!kom9eK+h{}a!ZB}aEA+Qq-{Wz7@Ekg?_ee3k&9g+* zXxs3Cw8jYF7R3ZergEYVtCV%Oy4`fo$Bc@t>A}qpcJH3>Ej%#@RMGzz@lD51Was-w zEa_BEznp$)-N#>8iOdeHxVW*HsXK1d-ZI=gN>A`13G2G-jw6^2&S&qg54i#p=vyma zItUd@=00`zO9x=W@Veg)%$#)P4ZDtX+(GqAWU#W(;Vu`I>6XfoJZTOERp zDWmB!*bpy_TB=uAXeX$u?e9!A<+C(Q6r}Rso1Fd%BA3q-%?F_sD#g8I#R9c2>pW(~ z4Jxe;#5Fh?F+?28L}G*;q@GCijEo}0a(?czr1MpwGLc7s+-i(3uSvAiQ#siTADJyi zX)G(7udoYyPpUEKlA11$pJAmBmvVPvpMuTIh|4rNaqRNLd&94W?Vmb$moLE_cKS@o z{9xW;b?liUc*@mUPk8x)50~bYNIPRGtRHqhoOEoVx>dSOzsnVm>zZM+NLHM0ux$gP zN}N;@zfIi7jjg??IqSZfpK^LSZyd-KbRa`<&ftG_$X8%>aYL>T|6fc5zcGBSM1ndS zkK1#$5C*c;sJ8IdT}VO{t(ovh8Y_AdlfgtUhT_6VBZ3rqk1znxKRhIL8E7+YKdVFf zP2xE&0!f!rheLR#5M5KZxnJ;5m4Rl+JPq9FhNPuMFIOe{S~HE-%n;3Q4*sa=deV(T z!?i|ZfvOLR**zyv`5DF_j!9hEGJU=ze&eJyM)R`ll%5&k@O=$G5i7(d1&G z(nkO@1Px@IH=C!tJ$j_x*pyD;CpB?}Hgx^>qmWwT=}*S9S+M!kI$zqWoHc8$Iknoa zJEOfo**6`?r&m`jC%R1Z@s%4*s=9N>Jl3i^7Od~})(J_;c823LZ|;jsVOoKB&DR@u zcJzu0@oh_S6v;~PAaa?EX$#I8Q{OppXFBD&w~vC*>0e+Vr%ApJ_gz&OEJ;rv8kDek zuPeK*mji(Gg&dG>RqFQ6SXS)Nsuj^j zECKu}HKY7RWcs?}DrO7%4(b8a?0q44XX6wW=#k z%Edy3S4D>N&S^kAy|}0@UZeakmVQS9*+({MbG(E(xOJD!YdpoG<#}B!Cnd=RN+t~T z*fd@@tqg5B^cJDvXaB>6s>~m@>l%&jCNo8HYVe^8vQ6?9A-vk>f;74z=p%k4L8v*- zSuKl0%G-R_x9sl^Y-A%KDcF!v)yr1`Tfc2 zo7!R(!AEBs2K7dV_!>nu@&p?7R}Q91(>oC7GbduS#ksH(MgU=@OUAb%ex0niedoNn zxMABOz1W|n_Bh_aUdRSN_6!drRPWG&=!3GFUC-Xvn1DW(aoNv}YPGoEt)w;jqWfhhygh?@pCl=pj0 z)pbAgD<3BbVWK-NK)wwylBC>d^dUUybU8=EAc0Z~Gu;%gU|iGtaF!E`$hMJ@l<>Ha zG$_yZfmMpZ^^y6}?N?%5E`#vB4eKwM{%>AF5ehXhpk<*l|q&2t6eH_YTz{dG3u zcs;Oj`E2BCtGg>amrqNoK8!~4mI1Anb9uE|J;NjtfS3aQch0i#u^aKO+dXo(S&B9^ zi5#!T#(b2^vhd6wj^6}+8pLus;p-SqSZPY>xrs|`ly5tkK%SOm+NS@}z#GrraCg4l zbyAkBd-Ef2cQGrhD6G`zXiAOZK|l%5e_HXmW1<@RP>} zp1MPwqdtqKJ44dg9XViCv(r7E%Y}Qypx#g2?y3cf%xxsPFqLl}BZ-q&w_f9F`qg`y z26!}3&eUk#4KAB+3l-O4Y0|i`&^#%!U{<%N=P2i2w-#poPkersGMkdlhB9<$s*Ea-F@B=&SsQ+eT`RCz8OBay6q`pRkTTIjm z>4jpmsUbyDeC58}KX2~+9->nEwg31@w$_r#0EQcV`Nf$$me=h#AR`uFa-a{F_qw7C z^$xbhuJZBb5T;PBk^6jWFiTs8C*$h|D(vK}E0sr^V51vQsSK*_XwgWn-Db&>*+La( zSoy6wjKmh-5So3Ty~$*uw8KaWr*J)=%{?xs{ej-T+u0BctvDp#q5`QKV#;D{Qp`5~ z9*^ZiS6NpCsnli=FBzCjGAS#i?n#@iCyfEQeWomnHZIR|B|47GY(JCz$!)g~eV-W8 zXSo-E0dnoAzXx;xRAx82TxPSec5|UU#B^=IRUS@(OC~NXPi0Jf?~enDybwWGNCbRe z)A}qN+Jt^unl5`K?f2L=>W%wA81Bg{$SU84KGbS+H+cZETJ>`OewhAmpz4poU-x4v zyF0}jpdk}6{*}K+du21jkEWl*=kzQ8@tGhziu-x*=|d;E$k1!C&l!|jJW&Wf0JV$HS+GR%)GEiH%8YDxyB&d71t6McG~&Ug%1 zXh2^j zx_X90si6(Ag&AL!ib}%3B3{kt7|r80{6H%tWnuMf!T;)yU-X{_$YI_+A#rmB(kRp_ zTH1`u=n|XX$Q^+jMxjuGfVh>)U|i&cJX$(e@y7nfAI37qod=!v9V^shm+o(XFLl;E2;x# zcg~j9mM`1mb@_~gBr+-RUhmUOL3;(m9#8HzTO;u18mt%?()d#bq$M+7A&q_+fHH0# z`0;!uj5~rc(oYpeo8s)8oB?@e4OS_%UN*6%iU<(Zk@xt4igTIPPbZ7MWSD8mxhDmy zE)IE;8M9fe0;HoKtY%wpRaUfxUU83?Sl$Ethig8a7wnN8RK9Q zg36(XC@d#G5#j)1 z7B=CZyoeV`w>0Yt(Gl z>iP{1RqJv4uPtXSmag&rXkSM9_lO@PJ-jTJVN}M_x}HZ%Bn|2qb$i_v)EG`yPLbZl z78fSBpn6->CNN?oPz0hAqXRX$^-MSN{8uNJ0o^N;lZw|_Ng4qXdM_XcYX>`lv2UcY z^SpcmSuL&Ul+++8ibnwh{h_4Q{8y<|daQ~j6DT%uW;;y%NIDni9H!$KD>&7)C>076 z2zq=5FtpjvzkD>UZ?;+VM*+}6UZp<-AtjhL26I&F_1{`*wiZ4Z4Qj$A*==`^nkC3J z4UNyY*!rM%jKU)A-9tNBY?6xS3oZH_VJH3=iKLCNmD3dCv;N+afG&w0zTaNg8v}du z#YGlF0Wdh=Rw>_#p7=EP%Am}%NAc$N887tYf$uV-4p&T}U=FnX^@v=gGkQR~iaWZ< zk&RBf!*{UT8gh%%s%o|v!jrbMkt7D{&KjIQ^QQaPj>AgwT`xjUX_SFYb%MNZho6;4 zu*=JaMwbL{TU-M>R|ui*q#{l+LUBm`vpxI=0KVOZOTBkO+8~t-7Nw(eJ=yr7>OT?? zBmBKDP)jt1Bc-atJnH~AR+P%pxi{ck*?^XsDD~2;R0D}?-kJ-S*3i7>-i{rAwLPTW z^(p+#5-980YB`e5jE*P0Kd)Myf+bVL)C9tabhOvvJSx4|)n z_kwY?Xz$m67SCQDovY!$79pmqnOZ-O#9n2D@Zn^k_;lNO-zqvev( zC)SthS8o>CSrJE0@**ad3836T<>^&-V!Fk8 zNojbBBbM=nht*|Jq{tRp8hQuv`H-!`fAY+KypJ5v(OM4?`jJcf9Ee9gWx2iyRYb4U z)uc8VOZ3@g*5!<+HWLxkjcYVn;7(<)5XczET!4MbrxB@Q#<|XR&$iX%}~1@?CuhS0YNz32&$FI<<(PE*_WC3%m(0VswXPtKVJJ zWy`6e7)@%2&Zi2|zqoX3uRQhIw4!pgzIpqW5?N`gbkgIjTL$se?B|Mj&qZg@6IX>4 zy#I#sy3}`g+G@591x-`Jxx75uDKafc>YNY8&4N~>%7ZKwT?#)4fS+K_`jNHk@=jrC~Cs)aiEPG<4_T$<8LW~GS*2X(4waQm+Y|l9dTTgcyJaC^u zgAixA%eW~6f0eFTZJi7$efR87h@;ZVA~u9>{Kf2V*xiXn+~=f)@N9@E+2k0^B=9p#plS4KXqCl;5($p!3_Z(xj6}vc~`7k*B|o&4LX{9DXr+ z{0^1{{w$k5GE4Y07jkO{{@3aMyp$c( z@R85PwxQ@cI+Ykky_sAhWyLqjI5{PLbkd@<%v*^2M(*TA{7U!0_3ol?houMcx}TT( zJN9YGz`PICsftZjgXmv40RVJrTB`M)_*%?#-M$)xWI1(!DAQ7d{)7cz4MDNwRYkgI zftgo*dMwb85BG9l)l->oX;K-#sb#e>H#W0iDCUKwa&f{mDX`uTOm=dhxO>nBaPhTw z)crtS_c_8j(@ghW9JPwqfT^@S{&YdOGAx4!5*x91?;8AL^ zd1<^wfgKMI?ajL2_JaZq_$BcvNBueUfyC-pw!f%-SMMM5z$yeYXLc5~^>lXfhpVcU z2^@J~(bvzQx*p-tW^ zQKdIAi~`CvyR4tcUzWoThO(c0Z2dGV>`8MP(c_<1ptW!mbUtD=_G!rwtPkcqXa18kqrJ@(uf%U>}azcCy;j_I|Hrfm$W;VRuF#92UHOZam zsKhch#YVW)Xc9xqrIj4(#|}%p>8rhNw|DdoOR>e;kH@oTSbBM0WrJOdojz+S0w}0<%Z=p?n z05B>L4a6qS9}KWNgcy1OuKI>Hbhh!{*k%)(M)P#Wk^?kqNAP#GPjmphC1qN9h93oc zKoo^}ZWj&i5{-m{>W|OUhIjZ#$KVMh?!@|5pz{jkID{K(=@&Gdn}DJe-IZBR*PEp1Yq2;SB2GJG)te{Qv|{J~w}@y3+dqFfV;ODNbs zT4_J?rl*%vy1cR(f`fjCLN#v7R9Ap5mBy~ChD*=)%c!tUgqom}sOI^I$jWF6wkEXL z@6D?trWY%;NX|T7Kg6$6HrK|a`aWfG9&6NpLXlvTIjcfHBK=WvIQ>Za?J5mtE|SNK zVXLnmhNxM15|k$IW*$~DAJ-UFB1 z9KQa>2uH*Qa&*9^ka$RWVR96tM%lgKlQ07L&Qdv#nTH+NSh>m~J&I1Y=n#aeZCc;A z?N=SqdwMlnuCjW!lp#BiPcAe&daHuQ)@0jim=BATJHA3|CR^7>FIu!AKK!77IoJ)i zA9l7930qeta<|xC42vl*up+Tnz14Smmon=SdD~it-2C}f&8zTvmBi-1$f*9E(7X+X ziv_A3sz%tRx*J~5WVp9)e9v{Yq8Kr?0QYv%%W-&~n3=cyIEK5;m@HzRfC^$c7eDm$mg zyU&b8sT0bqQHgI_Qlaky#UZR5peFkoGw2d3T(_vQK9%1j6D+mkSv-Vy`igb0lpqwX zmD=9WWZOpATwJ!1Xq~<%{V|09E#|`!xKNLT?P#8W2tVYO$HQ1A1sE%UJr25p-&=fp zd#AHK(6-+?gT<#pNuX9Uhl!z`Dcb zHrFU2z(Gtc!i#V_gigNZB#x7;S6 z|5f~rN-6vQoQ53m`=qs>p2U;S2MNPX-rt1b9(~_AIw~uj_On@e3d>w$g$P5s_Om!74#5M|z+kY!m%ONg~=2{9B7$CjR`O z04pMbHa8~*-@KCF8;G#m78Ac#pAl@LG8skOTzuQ9m4Cm+sB7~e-iuuqCr&BHsq}+J z{ME63$$fQfqfi8V0)?e>p$1!sDwbAE+Ibx{dg+SRIInY1XC=~TCZT4=S|0}6Ae~(H zdfr^~&-dS$3d&gNg8fFWNMbF!!WaZcJn}T;aVitTY%g*`@se+w9ck512&1wa1Zlm0LB9-skV;qBdWx{!+ym7gV^=3%cZox+{gR*l^gpyn8C&5xg z=|`5ncs)_ElN}s1uQvpKMf)@R34!zMAgecPYf{4Rh2a)#&CPG_+8+;~n zL^kiQ4n(7fV2Txsd|O;DghxhgRsM(u{?hh5chrmX_pBOEX770+ z*!>#tktNb4IYffdb4#QLzBVxymfsqoS+^9+Hf|5$;|8YBzqb28?{&)^z^_D_G5}xX=3TDsLz<( zSkFVoqS56U^uSaOpRnQWjjlm~rc(P_SJ5P11c*5<93OqT%xHtU=tEMlOYQXDqn1(v z9jLPE(bNGoz8BZWMS1Mk!b0!v+YCY zyY4$d^n58QG$|cVf*(c09oapHp&8nez zGLNf0iF7Jv1OZ=N;^jaTaUzQux_X^?5iK+=m4J=>VSjPHGSM2DL_BnB)1wb0z5Bgu zCP}sCl0czVUqe%~(r^#AMG@4u{P4bOQjfg$lv;TJlA09QeCzPV^--}@wHB0}VZ&&2 z4^4}Xv%?+zdq-$AhwK8{-0wak=SroZpGVg?94K2LnjgLXJTR73dV1-ef3gF8wz+V4 zcAm$k)#eIwqsm|bs*e0qDDbDn@Qo;vNYn5ay&Ufhl(pz)K))+8<3ZiAZx7{^u6xD@ zXXl%EEyIo2y@{G(es&04@<>rmisyw8iq0DBdYo*G{Wn%32gNW~CM|-46LyE(H7|{y z+_hYAq`q+npDyIP>MW1-(qc0jj{DWMis-2B7&bVr+Xp`7bygQdNF798bXJ2lPLNxz zkLSN~AUa@!@{(2xRFBD|(vaKjZ&9Vk#^&$G-RFx|b8u>oe)tc{Qyac0j98jZayM2z zTTWWQW%mUJK}u{NGq%@i_J>z=>S22+x?T13(#C`%Zooq4D zd0er#4ql4CYnu5jDI5jDcbnIv|D$4FW%P-qR51Oj(zA-mp!?bF`p~%ytUb^eSVja2 zzU9&%_-MaBBR1znQU2PaiBPYJxxwXvRX8*y^n>P0Pqtdj)wEiXLS9FtCy$iC(h_Xb zj&_Yb>6msysEe13H=M}BGfyax=YiNOUBmcZLUZQjj(oYm8=8BF6>%v z?K4|K!!OVn2FnX}VE58gCN;pda0oI2-{Z`yBs&b>_Hv|aYO`^~5F>Tf=6qS@VkMk9 zVe_qX2&0&$%4%0Scw{B|gh1JyK(tj%XiK7!%HDu#hVfiRD`c<^l*`W6ZQUL?_weLa z0_8$#G4wo-p7f}Nj0T*OcxL|^P`DHcL@VFf{Fx6awwu~tD)$uITV|ljr{Fi9dUoyl zqq~_k>l7(2iqhy=7NH}Dl;`bckM3qj>-A?;PH$UYd-*l+8N35;qYbH0oHqF2KolrQKH8^);k ze*OjA?x1P)Qs3sKbMPi*~wiLP+>CZysJC@qQL^wbiBuAoLl$#dETLt@81%* z5ZBFrODYQll@G{evi~NS)p8!P`1gWEOTo-IlvY8u1M&56=u%iglzz>fJf1|fVDYEt zGP6p7#*0?FO{)C&`#&y*EXs3>!xleRp#+d$9R!kkMF>sW@_p@LCl-oW2MX;d*%f^6 z8(0xdr^~T@4el)bQ0MjI&?T-xg8)Lq^%tgFtQD7jAD%TWeF#=vuJE9zg^D=1fd$c~ z`V=Fo6NR8;^O2E?M`1fzxv2)W_c

6Lj|FRfkQ#{Gxd8uJv;z_|}egO6Um^;=L6W zk0p|d9D%G6coA6g9E! zr>VB@$*q2(s^5Crx^niR!EpL|(c((~sx2Iko{(zR!m=1?uk8zq{x{RqSRAjkMq~@U+lOV%)rtAx5Xl2T zzrAbg+;>P(EY)GV++XbcwAAc)HPqm|N3~6_^=pN_kNwDDQTpFhgPMt@e4D zgfBtIkqMG72)eV_8ZQk&(Q7NC2d52~ySgsd+HXFRnr0d&el`&gl-M6b9Ht(Y*9Nya z!c*@^E2CE_S)Z@bP2YDVi^bBQ9Xo5(?#ACeKn>+p`V}3{5Tc>pl)-{mAHo7&aJ&?j z{hHan6nYR(-xUl&%2^1FeanmF4@%pShVFaytf=?X0N3Uwl~v)xFFN(N2k7GynyAHQ zI-?_)3M=#xu?^G~$HhNke^(T#B%%Z4AQ!Wk=d6T?RJd*Ou{SxD1k5%}@-g@KBOPZ$ zSl(05y}Xq2WG3VQL!eMiV&&Q4AhBkh(}n3(+kDZ5*|v<(Qk&lA<5E(fwdjqIG3d0c z)+YD%p-j{EI00I;1O0X7H6DYmKhQN#*P7L2N-(5-xyeBQ(ygb`<-(Bml~<|K(OQ%q zvXI*ARRy3{2KSXSFbz4qKhGNT`mb0aax6!L|y;_0+ukbS9Rsl!gH4Okj&+pPCcsyq^UNuIG|wiRw5 zLa~Q11&_n&9AfNxSPL!%I|>0N^Zi@~?~gQU8$@T6@7QX$t97%ap z#`h}35OcZ9KDJ?KBbi$ziZL*Qz0sB&PxHnp{Fg5F&e)YxcALh7=vWjosjKAH!Z>Bm zLl-j~lNJ2@Bvj%F%x{nMh5R#8?|W=!xFUrDP_OQ58lnnZ(zn+A;do~ZMAl6;CXrt} zH6E*7dCrS4+t7yNS*0=vJ=8-9?xbI_Diww{c*G|J_Em4wf^_A5xL1jrQNj};c-fci zGkOa=qT1VnL3}?Zj1%dWqS_{Ik>r9or)A~ z1lRx?oyRjbd;#QlPF*5T;{y^`(D_mFq%uF$`IG`0e6_fZg5^et|JLz96Rs70>k$*B zd|`{%0<2OTs1e5%YT)$7nXM2v*ROXjrwkR(A&JE{;Ao(zG(&>7%^n}kA;eRMo?x8S z?8b=khpY;HX%PbtO3H?3nYbm}$FUo zj-K_-F^_`zYUb^@b`{jB!B}9Z_~-cBw)R@{(B&aU>z8&BidV2OVOu2Mh52<|ho>X5 zr0&JUW_J`w?{2aUYsK#!_KIPjvH4*9L>>dWU@nN%rbrrWF2>vxT)-`P0lF#)PBLG` zbNUcX^@gw0Fd1!evzr=Q=OSSKmQK*7_kne?G=0sk6|LXkaFx9h*w7)BND?;|VzwF` z=hat}<@HT>SsW#y8#LGn$<%+Mmt)Kwjee7c9m~|~hlZeGM9PXoSyx5j7=qrd|C-9u zC8zkJnXR0V`KcH0N^6TqRV&GMTLtPcgA-DG8cX8+g62WITEmC5BPpv)lI^eFfa(5`5MDRl1_y1G_>Q=PdT&)S9Lr{duV=An=K5QV z_X_W(*??N!{Q>+cU{DIWA>EL?#tJ<|QBqY%rw-)Aht&0_A1CKjw(oZ2sj!}ciy^IN zZgb!EU-=nWx&2UqNs$=SWl*A*QbncJY@KY0CeXU`E#+Gi{Wq?$`q{A$*926$?tsh& z0E8?FlzqiZwu>xVyUrch?ZyIqBZty?1~6+vl7+Mt)@EM>6uh>s@O;bIxZz zinvAUV&Be-;sgWbOE$#UNcr%3<+BV9n9GaomK&@*m(^_ZXfg$zNCoaOdO2eb?;<8n zT2M|;Cya~-QdXvu3E*FzgF1Id7M`Px0tu`cW7_W2JyE6-@H;8cIToA5YsM)WnS|Vb zN#+&*ba6a^de%NrGPN7aEH6XMf1?m5>r=&Jc!d!)D1&!`9f=~IG{pjH=S-x zd5PYqYdc-Zymuw*q(X5|WmAehinstCkA-2Z7d*>u4RP>+SP_DT1$RzE3JWqCj6qg) zH!pV;Ng1;9*wKwPnkL}=Gh;p60 zdcge;2PEwJ7Lu0Uq8#`hZ-^di@#^E zQ$ToQoTFxbXJ`8yuGL@=K2Km)ZqleagdfK#^{_r&`uR9Kv@0}w!S0-Y4s-|A2a!iI+%cs~Y(q785ohrAV6ae;)d8!QHO_z2IYCf@@2bV%Mr&>MW`^u8Ff!~6?W*@ads%pE43;8c~T&dWpKg(#so z+D?GRX}M+_L2|i{c#OjFTHceH*;BmV#W=xak?`4;bfs|*q4T2?!a}Wir1?a?!NjDW z{nj4pTcXpV5t?7J>4a(wiG6?kzEU}C1ad7_{72?w?U0&PKg_w!c`r(+Z%iGLpuzyg zm?7ZVUY*M0hd#+1Jp?fi4Z%G5eMg}6jgt`WhKXPgxynB**3{rPp}s2Nt^n=Ge?4(S z>FQW{v8za;oRks)pzUZ{eb@cvdasqFF~;g=alP}-`$A02cSvL_Ex6XtObomES=i1Y z!+H@w!PW&!=d^X~`92h!dpFg~Wj14omcVmZKvRWvuZB?rnfFqgNmQ*?`BDze(k5LO z>E+_d@*-)<95~?*wV(l;ZIuGF4{-8Njl+Cm^sakgUK3j8b~qk{TXn*+i>>J990NH> zyAlS&-mH_6Q??34GpxYp_EgXGm}*AGXx*+1{74gTTdh1WE{SB=hy zYU+)iyxdvbQ+hJ9I4al)YoI}`3CX{)Ou#Pi^j*noquzc&;Yeaf*g}wBzGkxUdtNUR zJ&!h3#3wAcwTXHw+D<#rSwGM-$F4Xxv#zdg^7~91P_VOsdwVI=N6+ySrwIfeFBJzO6@m)mceGDmuM|?M))jef` z6E^k@;y)r?=1*W%W@Cn3!RYS4PAC9w=^Pe@mHBWK)Y;Yb5y+2A&`Auapyh}yhpv-%PNZGf|ISdUcH=x?H9X7ck4&ObMzQCzw?I|m0OLA3l@{dg^u)%7!bX{(GfYE)Onu|F ze2oS+dNCAulX*w)gLg*-33WZf4@f*C`)bo*^c|FC2GMSwbeZAIH3x#!q-OFl4) zP4Q_#ht2u`M0>H8#wC1IvCoLvDhPV@_7r$5dXaFPm@gWF!G-)4;4c|X9QJsBoo|`= z`=()Z3=sRNP7Lvx#DDA~f?tWQv<`G{FnpHR6+G+*eR_F`H9$lyZLN&}=j|F?2}p$W zmRM?ZMnw_ze9-+#j->p}3(PaSDmhk3vJ8YWdz`oI1o zIU<(%<*tm`yF#DB7+Fk3Q!|AhZSZ=TLFQ{ofU}Go z@oZ6xN6kkvooBdJ%%J@uv9K0y|Hb2-{6k>$7rIBn1&irk{;4tt6S%%>u6HrBjYFdt zb>>32IoE2V?|j^VmfQ(SW^vd)3LE~MHxD0gFYYJAhiDRy~z-jm2qTj4OT zqr>eXbzSLZ-yI{wDOQQbd3v-@RWA{SHbxu6Y9%MM)aa>PcRMark{HWenScq`i)WK~ zRa~3Te8bWYvf{UhKVl!$97IlqxOUC5s$JMoZx-^8mB5o>eN(^`**y z!cSC2vT_^;?r-`08#Z4BwM@~TICww1zMmD+%QN2NaV1@N)5?=ai-jDykaD-^%=Tm zue*ZgAGK#i^dtt=n;A)0nvAf&N1cUwI{&&MQf6Vg{>LWyYzAdS$ZxD=Zq5t*{2HSJ z^+dse?&M3xMo;c58;jg=kvH$4Ru1qW<=le$Ccbw+klvh7i@mmk;+UochAHJp%|1P+ zHeBC4o(vyT9Uis(pO)sD4b7S(!*Y(9S)h`|2(Fe!o zd_G#JKpWas;L6Gj8d8X3=AKgb%i<`He2r*%k7n z(d3@wvMjj4sa@jBj)d~caPur^uGX@|0%S+v}(YIOQ%_T&fM*K*zuZ%@K71W+gv+!2K0p(?qP zxU6}K>ipjJ&(hgj=1&%OzhtrjO%QOqrGcM%bL^@mj%&2SpMb#sAH9dvr>5B4%~o&X z3#0-xDvjDw-$a_K9X#GjuV#zuq(?pJYOg<(Y?@Q#rF-&#Ecy>b3g`_-=L{&bd}HKz z&huQ)qYD+zyJ}A$DAHekb^gp+6m-?W1gPPp8x3jmk0G6g;;&{j z_CyDGiw}{@jy|M39U=g8lPug0?wD7$nh`Le%kJfLMP$9ltTH{0 zHwhtE!okEIO0^QAhc|Y|{$ksOD5u(9*Y)=QS)u>`WPAT-M1Kawv*!jexoqG}AcTS6IN;YtL{fU0SiA?GnV$73*pn4)DjmsA`a90dQ|XHUQM z=vE{rtJrdeqf1GPp|VFkMb-;u`ZR zh*fFMc^u7_@W|kJFQ;o@VXH$Qd^%FZoi4_tFkaYcirgMo5o{t zynOHG>MAS)+gNG5Qry%n&jqm?j|3UHiX`gegAJa&uBm~S=v@jS)5PtT@GYt)_^UpT z5c5Ip0cHAIiPXlr+YwC)O>FkI5&X`Ye>Xq=3wZc1P|@BbMedqWFLCh&DiLAxWCf@# z^Ge%P9U@z5GAfC_Iv2k`M5>a+>=4%{JJg}*81uBT+%BjR+_3S&UX98yI{S07+WKPG zJ37@jEWY4-dT0Q$Y?7q^hKYPCT0Ar5DGdNTuDbt8`dQ^GOr6LaL(8XUnpyv}CcJHd zHp&gJ8x|U)HPYD}S;Od?M=eDXFOAui>X~;k-~6iKO)@K@ee{F4c>fCO^F5oH4Z*2{ zc`@qL((QfjT#W-R+`kn` zr+*=UTb>;(%dFO*Q2!^{A&=1bSz2~j>4X%9MR@m`T>`y&g>Qxye!d5Gb&_WWRx)ek z@mq4)#4I645P^3>t?FI(e+-Cch>1vgb<4iarn+}9zUDGcyd-*|!S8Ms<3WhVZ|LW> zOK7@mhz}p8MpuFlM_DFaA{sXivdfE?9}#(w*Xj=Vu?~kA(#?_CABvS{HXkM|`09tQyit^i9ERU%l6T zp$)r}^`UMVaxF$Cec^880OJx|)VCngeT7#JKP ztu5oD2dO?gHx%a!txqi80`7UJw`D)<#HQ?*)6RKv1KYpattP-n3`VCOj@X!Qi3R#l%CZUlZ*j$18Ep=tMHJ>@8f#50 z;@YCCLE{xwQ_rTFUPNJx$g0D6+#p}oR+m7OWG)3avtd@SVQor}KkSFQ2}F9f=XFm< zhj8HnyMZApdZsW^JBbAK*BB*j*^NZ(gAsyMiyO1coaK6J-}(={MFM;-N8h(nFU7*0 z31}{y1?31WYYA!s?gzi= z5a`cnLTLU{kZ?aPDZ;l3V0&&2gfcT2p03adxRGRUWz;HMVUYEkey=-^d4!${tpClo zd>BHK@n*xsaOu;~hx)Z(2^87(g~=**l{D|g-;FHhh|8gRw-H2n2mVf^N<9R=N-?;hn)HGcVOh#see_ALN zJJUGdxUz0|_Vz`_2?bf1TO`w2piSus8TCZl*yQV#*SdHTFA9{`2sS=Jf|ZHgKggX8 z6JxlF)gEDU-S)xW&`8>(y#M`}k*Q-351!)w9FxPUXIQ%{!7IT^_lns@1hM|(5?cBWF^`RK)+lq=m74S z)<{3N3mMgkD)o4i{;7q`VWT%;srpJFri|p67>;efoS_y;1k^6eVd$77?Y3aMYPB|q zW1g(S+n<)HiyU34j1RBtJQHfMuXt4*b;&B5?8#s^GguS!{o4E+F-Ob(gi%_X#eqA~ z!_((H5wH7B90W#m;%dBrgJc1u?aaF$OlW?NqfO0l7J{mLNBnvB#s*B1D+NkXxl$r_oDWxEl znheM-)1}?L2612j0>WnmX)$5d-KZvi4Zj=21H7TrT79%x#s6`_d<*&9gPA6xPuMY| zezujx-;BvF@s>3!x^{%=*m~###(q?bHY6oaj^`{7I@eG;v*7_B6pA;@YBF(ℑHG zaJ76dq%yUcuQm2Mt&e)?{6ro!ZnY%`FFn$f##DSytZaa8OG+HalQ^qUcO>QI<@Mwq zL48-x3kM@S64B{?lPYHe%y=*i?rNhqaI1;M4n(INg!dE-(Z68x2*5m92xJ0vo{b7t z)zc!UUAvrBK$V)rX}zZ}y>9O0hvVVVi4pe8ch2Jt*A=Fm1?O)^PXnuXVKl2 zt_iybM1FW_Qzd~ZQU1fAv(ytbN-Us+bOg;UXQUUj(`v;+akY+gl=Dh-vs-QtZC!?g zTOX>tEUd9Pp3UI=p52yXPCu%lq_IrKbnK-Gmv*{yA2$croum5M6?cE@=J)Vz zOwy&E)q6K>@IJh>DqhH;Q6OC~V~TLEush^!!0cf)U&!8@Dgcl!HQ-bZ@Hw9#gna*w z0DJ4_W~kLB>K2kL0w?VK+r@>1j4!UDmF_l*#G6KYVn!keU(C5y!-itMU1Wz;T+d1r zx-V7^y>GN9<05GdY+jKbFEF*8rpp+e_F*%46K<*)U31U!E$em8%D`r>=B@RfDrZs2 z*pvlcp|9pdAT5UP#ruPx``yv0dJibY;I3jotNf~HXMN@AikVa8*8PE0`gGUDR9%v& zdj3~syXlWt7YooOOpyn?gj0K0A}R!JLPZS+)n0*spqAh3hXdN{%#>2bh!vg~W3ilS z{ns4)gFXOb4r1q8j`g6GUBLIm-8C+nap!Tg(#QRmT_Usi^J#q#E=cK^y4{gr`pzpo zVgWatL6?{@wXPvb zuc85_sJ=bYp~CdlY$gMq2zG~|jxgSc#kLyOmysy-Jzz?gonpRaTbYq<1w|X9Zp^i~ zeZ=nVcwz^hZkA@I(5Am)bUi&n_XhejHINa$(gE+syZ>OB_kHzJ8(;Eo1hM?>M)w)*tImk{}JIiz%I88L1%xF##ALt(8(H@$UXQ9XIb@(l@2Ki zrJR+>M#7aofJ1s?P^g@A2zv7A;Hm!Y>X1`KK0iCtSa>z5?Hp^dZT)U8T6@6maZlQd znyxa)WZ#+G^y+cto4RcRUn~?w-7Z9r4}~%h&LK|qqju|I5*Ft7=az?l*(*u1(x9XeWPdo^r*d=sVGac6R1)vld|dbp7uvS zJ%CU*lXxQMS-(&>x;m`FeUZQ_`;byeVfnyFOwP*S9SZ&DT|5Wg50FypudvqTx40i> z7!;Yx(}&nM?Bsg?RP&Hi7Nc4ARC(lvVY*DJYJbn6fG}}i=Bp_Y?1Eo=37(@4s%~C;M4pnE3;@Q zI64XVe4$&RF2;g+^nL}wB|-@I!sh-Ie$LWm&%8-cC-X72M;9)4spV4OqEX#zm& zRm8oXQMg^^G??;BUX8S+9+!U?<)=kK#DiG2Om%3rziQo2KLtK5a;o^!9Bp-R`?827 z@5lTUA4n{9b)cf*6B67w5AV9@uADeg8k%hcEiYp{c#@Oyx$$6%=X2n4RCD~6_L`@Y~{N~$f#MPE$>DM z4!)hy)ofzq4Je5I9q9U$5@60Z0OguCRoTG5M9aB6<&Ku0QturVuj& zX>S;-_qwkYv}_}{ROjL;@!k|rSSn~+HuZA5Xj==McU6v~F5lvdS)Es_o7|uK8WJ#vJHJzVbS))A{wJjxQz@?1=cisk*NgG)pR3h0W_jYGW~`w=*%0d zUWW(``(-;1=f1(z2;TZ1Pt{+r0z*=L8zuKJ(*t6~PrU(McKBmb;X+A+kc8@PKt`{M zrL4|@s7zIyxMgqJ^pWLxUHih3r;MndEd5$W`@8ka)N|Q?d(_CQ;%OtTL47mBWG>UX z__S?*mVUukBdl(R%ozBALVrWkzZPBVw*1krxdF{SmPfQ1Twk#fa|R+r{X=6|(?q1- zv3x&4)e02H;_=5^9s2><2ud5CBB_Zel-xaNU~}74@n@G@2;7^gj|Fhi<4XkylO?vh zrwd&jzD>h%+WRA#awRVQZusZB)MkEI8%BZsn9zIe7=7`8FR5a4JQljc@g!!a{_d}7 zqjyc2ssxmT#r0*~vc&HJhpe*A;fHg9NF=v?!c>gK_1`>furN()py_xLnb-sqQqBE3 zM3L+wkgH8~>kz+_P0hn+u1;{UfrTnBdtsR%p2Bz~$*JDw3P5&W**#HWkJT&$F+kM* zVnN{NQ08E$6g^f+O8PK+tqnWs;Q=D1n|e21C2d#{k&Z60_&1)&j!r!Rf0Yl>0i)PY zHMQ=FW3_<%`#MHbG1|+e8cy7N)%W_PA9kDSe|PWoixIS?Mj|AbbA@}+nfkR>Rq!73I*wI z07=B$F+68Y**BUVR9bq?9%Ss>w>gDvUHx|l67ju4k3sHdKFeICtq&)%M9|aB4VUuG zgR0zt_QX)y-1xe;B{6`^)xJp1J~k}~@dR1~u;~e6eGG)Tv-}pZ_k$;ya-^CsNH*f{G;dVoL!B4UKeZwsXRT@0%p$-t2BJpdiP}N z4{*$1+{!hy1D9xT-~Yq`nK;FGQY0tx@49Vk);ZV8h@BM)9+)S2-$yU6yYqb1hw`LhSxy6$K9dp8vX?;LmQy zg3yjs%HBNKG~SQ(APSVsnv*WpESrU0lja)emPDMw_vANsG5g@hO4!tF1#gn3CIG^R zPaoHwdBPNC0(J*nDq8>I%z75m<@^<#S!Qfo(rWqf7(DvxZr$UkVICC9lv_*F9!yApyev#8pizQcK< z%+4a?!>KO@8>UO9#r++&my2klT3E_yT2j5YLyFiA22$sQ-aopn#cI@M9yC)KLnMu5{+uhd7VloC`=y$qmF4`#pKL_J)vZ1=Iw_Q^UL#E;YO4tw zw}f<*(6M9LRA{B(OXu@GuT{orgNn^=+$8`Di~_q?M_e}3PopV*L!faaP@F3&t9d37 z9X)ec#QOUKdE;beWCt z_Of)D(<*Dkk3UJXA)^cp?p@u>T^mg4>^_u>%n&0pNtAcV{3DV+b`+Y-*R-z8*Bv$sMkv# zKX3r!sHz`t#s)h#yTaakJeNp%7_ZhKkyfr#kBKhdi2TRqSoa#FX(MMyf41H+^|Sqj zY-?Y81DCDf61gZT}u0-2d2#z8G@->mRK>-j;)vQ2+egMoTYXq+XJN<#+zw*yQRd~7T&!DwC7nzf4*OTdyLLD*UNz`v^kl+R zrsv#XfGi7~>QFr{ifF%3jfDtGcRO)Ah2O-v0JrJoia&$EUxrjdKRcX~p|+=`tCZb$ zGwrCX{@6-04K>RvLumFe7wArk9UP_f1|_=~L%`yIY5bTyp}?S>iJ4^28F3nSer#lo zAQScZC1We*(da)(MF19JRfC9bM#_Ypxn{Tt1xeS(>mDxgj`Igx_$uoYNn zNRQc{Ya^A%abFmO!2h)wwiupVQsyC@Ww!jCO{eqVK%k8&uT<;1yVLLeLgXmKZDnQRUTf_0#O*}qwqR-R5DVNR(VQ8IcO&UP;JqbVrO$0PL4SV5X zOrZ5Fqu;o*_fP`$F{{t`WuQq$zhM3XIY5()DaKYsn$m-G`44I`i38)M0)(r!uuTAW zL>pZ@ibM!lQ5Ft^xeU}~NuPk89-P?~3tj_I!XqSI&H+~P#TMvB0(nmR_D0&>8!2K^ z{b_gmrikd*LvOUQy&5bT0H8m|8HE9gGDe%dH@u`)cJ261J7U-zDmMl8IZS)1c#{IfR zxutp7y9!lv9h(uo_Cb~ZZD3|o7P@sJ;J&R;&)QIXsZ27aa}V^CJ=uJjPpGX=c;);f zuy4kG3{RHIN_U8uptPLN_xH+}WMiuK`f=k2_u(WIKNl3c*DeJh%6F`6d$i?Gb znDkS|z5r)P5Fb}P_eQn~Ys z0|^Yy<>5_wA|>k)wLqtuSgfH}@kNGBHhf-v0H6oA5W&zSL{mg;o_+c4FGKLQp0*%L zHNELspjCEptJPJnF~PsrV}FsOHR2ucM>n?KTu;xX38sx(b9HJJl@H`};_We4q=5mP z-i0?m`M=qCiWGUdr7c$$uy>mvTg~I&viZ`@QfLQyh{Ro(t^ZJ04qZ zHD@!%^=4k_D5lN&Ruk@SG&z{B;l;}6caq5WL`UmE;iv1^A8vc_m23dnbJ9Jo+T$up3@*b<1ElZvt+&5xw78;t}H;@!Txwu zba;j4HasE-e`+!`R80_skGHTj9cB35YGxkf1_19<6LaHyz+3jf3MAU-N#+p2_-2KY z{t~yh^(TKiy`xq1Y~uZBx)P`+!gH}Y{I0@3)B;}e!zPc-leJOANB#|*%jRt8%K6#O`q~ilql{Z6_dq=JP=`nNZQ~*W+|O>xb~-tuKCzGS2BC3q7B5RAw?D zG}=!^U2}@7K#K6Hn(P;K5;zq)NV^^azbUh}Mpab+lIK!-EjZt=orEq~YcZ%%Fchy& z`i+*fN*g}6B@$=1h;`5C)ZckOJ51;~>3Z5E&0e#+z6bA^_*kt6ZqYguvyC47Rg8F{kmJL0MjrB1#U2HbPRW;9myyqQ}`qpF?pqIhOOtf7^t`xq83fS+1p zVgr0hWAeZof6tL8K!f#lN?7EM3m1A6E`ZljvZCS5i$Y9z)#TQN^~ z)|3%AuU3v1aj;S{yU9P`_mCfWI&-h3b%tM8iOMqkyrM?z=HbjW5Zi3fG85Pq<*0BvE zaeS)LK@@1}&1;L3_EyKj#>i_8u!uZ^Rv_A^56PLh*DQ^DmA8lxCz)jepO(V%X=a;{jHulM`F!L~jwt)$?8_+k3z}V(hvoLq{kPQ>tvvK?R#`(hYe)JCu z{lU6OgBQE1pEpzsvuK{Ar zIx!)e>+&1S!54p>ZfnPs6&_M8@0seewsznUX|Ok9(=t$NF}j3ib`#FyIqdpkEs1AT zj$D=)#4#0;y0bT_b9e}-DNUS*x~hwO0KK;63$`J`jUXF?e$zIN7AlV;y8A*j{-@X@ z6^GBxBB8hEw{MM@_3K__fNXPuSMZRU#p3qS&!pO-F=Ei(Zx%1eSAOQp0#mz@S|4j@ zkck|6GKqd5J8ti>CLiDVl`W-6edxMdkJH@O_L(PjQ(t4d6nhhTxssb}gND@xf%4g7!%w&5;MT0h^npg^2jGA*NEaKxY4br=P%K#<+%9UquaG-iBxUe zWBS$Gw5CP(kgv(;F9e9D}w$`EYtW6u!&an<@?6&tG*}R-N4Ovl=eM z9%%qkWU6gd!#}7p5QfEKir?!)((G>*G`nh9p8kA7Gr?H=WiZc(?Z5_K`QnoEOAcp~ znW1VB^bdqQpXF!OsjVEV0J~FZ4P|4P%ftCZoSt&?^hAEP>HJMh^v06AmImoA7jB;W zy)RYoxFWN6yq}H}93eUf3w2YWmp5W}R=RbKw`@L_Z0tog-1aJ11L-F0{oM0!r>}Nv z`fnzjo!ZU^dJ}F}zjCf3qlBQuU0^Uk_@@UcMR!wgDuspV)c@z=ZC_L*H_OGQQ>kH= zxVT;NWcxT}CNjmm`c3NE^|jB%Yg|F|@Q0ou%5q6>;*~lQ*}^FQ=f?^^2B7&Ox|vV3 z;cjDF=_B7YX)x}2QP*z2;Pg#}fUQmwV^*`Ka*l@P!xHEXa~5j0Ba?pRb(GUUBtU4u zo5D0>Er;A%zv-=X^Otl^gXv|-Kvq`A3mkxX- z+jcorX?UVazM3DzEX?yEcg<%5VJO39?>%BD@VqC2d zhI=b@<_-oEEz~$209^j;OY3p6KRcaz{bQnR^;67tm2o1bQlpk`eNTMUh9k+)j$H3! zQ{6=FF1ayw?o-@Sn@9me|8)t2?cimhRQd}zMHhcK5(BU9D%Qb4b8RkV1kA}x;D0sw zv7|WZH$tClkM3c=0kvO(l*de~SwJL6F^W*Rv z^~S*);rdzTBNEY}f;$uXsV%Js z6HVKDCYv*M^K-cd32LD&JT-|hjM~W3TQDa}S$MBjIZba^qUQ7Vzu9o&d5Z$9|s zcMpl-2c_n30f;Zz+&b+K-zNoFd#s9rY*#lyrfPD^tFSi+4xe;BAVRmDiX5)G0o4x= zd>_$z(!{G`$t@hI^u{tz*{?!TD$#zEcwunq)ee~|is_667Ek?QcP-Tp-fmMA&cxGQ zJK8YZfm>|$o)Lz26-7I{2gVpC}h>|1b{Rl~^5~9)BJt<$JfQnEAQVQ7x{> z+#-X&8dD|s+CKus;Q7dlbC^8Rr`+I*D@;^Bu zT{6&V<-U@M=;t+CV!gfhe`yJXA_+B&YUZP8?$!{CaXv8tmJ@xh_=DhaCe{F->&=!= ziEzsR4XnI8y04R=@lt!I(%W=M9Z6p3iE#7PMf=NJW>ve3IGw(inul1_S2973WsPg2 zBSdqjyWTqIO$ct7n9gLpfF@vRZ@D}}oXi;&(0mo$=-wx)4f6y$xBk3 zVPjBSpNia%-8r5|sKSQ|o@EyK8(rJrTV0I(<^c`dG4X>`NHOvL8sF7c!~a6lF%2E9 zyu9+%CBibP;Ov|eODc^TkdfJ)`51RZnEOxf^n(-P2z9Bo77W2xS;cM^Wy7N}Pv0iI zjRozT>`I7%LTxGncoD-CK8G9pv|e#^l>`{KA9l1$1*kRK8FifQOB&-DsfcFSP8DKlhR?OCLv3tUPPkhQ_xsFCPNUK0y<26Hy_(<4jSk2JRl96`wRU+~c_xeXhlU=7%_zx^|6yVdyl;h(mW>aj*IdjpK*Zwo2OpVyDvccVJbV z{cfkb;nfhp@|`?wdfl&&Dw~5rPNxDM(?5T+lJ-$$@w77k0Jkl@#QhHUR&tr#!fmb# z1rZwDE`dC4_IH5|F(y`=9EcSke?sTF4O(C~VWt|5S^}EqsHG{u5mo(Gdpx-7Sz^V| zL~JcP43TCKRZ<@wquJ?hf0zQ#-qjLFwE$;-XYW~^^~0%0gg1=TWqj%dHJRU)^yzKP;SW$=Ziz9*U~Hy|6Wg)sbP zV?F^Mv}6>Cye8Xf@Z*~?-N|YL(|SQyV0Wm^(x&J8+%bKAQzqBj2hE|}+h(_WJa}Cu zeB~Xq4q$olDQpe29z64fULZ<*;uwWQCk zCAl+6np#Yn^_Cc+bgE_Zy1s|1+kInd^6r?Wf3h?nG$51|iJG;%QoVpE&6S9D=6x%S zhh~mVJg#@TW&cEf_3nF_l>VmwOM&nDR6Yv-9Mb&vVvwH~z;(}dBE@f;03n9}A#pOORz*8jcB+rKXS^S93k7{_ywaNj9q_Ev>1uT!4pbIlMw zfYQS^p+F%b!b~CUlO(NK!q6p{M`if7>pO~6JbwWX1X!j>N8yp(H!4NCdTqt=IsRP< zwXwYhso^n{8qs;*(#7H6CRZv%HtAE@K-r@1S=(>7KWUyvjp5uNSK&_X*}uL@2p-#) zF6knvgcp5RP*xj8y93Tro;cuR4SB|#mEr@Z(VGO-l~j8f{x3{PTq<1r^m?s8wA@b!SFy~nPF^)~rm+IkzVkRk>>q)fxMq;!3!Ct6t>28d1)In4R zvy%csC+e!d! zR#9mQS(@(>m7$7LtE}T&bvWV}xIe@mLZ_i_1DCDcN;Uhd#(*eu^ZmA-I zH_eR9zk2-&2BH~wSyTw%atQa|xiY4JsQsVG{jv3jyOP)qaxE^-hKyx`4dkKef|IfSq&rX?MlqT?GLpsSVe)qmdT{*fh<2Lldt0~7Lb zo&=lm50gg4vBB>OZKR~IdM};{M}+pno)YjPSW#iFD?h|lBm~)$f}KdGpbQ-MH`;I0 zYKJ3>D0hB*q{^}YA;Y7E#XkXtn=VMB=de$1;h^%y>cn`HqU_SEt~@AM=E-5ndsDha z?+lr2xk(okxxWA}Cn;widCo#Z1uu77Q(%XR8)}P0@D`HpORMBckpngN8ncNgJ8+DR zrt@K)4XDp5X&9SJfYto&Qy>y@`a>HI%Jm7Dx5aarY=}rz3krq#_++kl*gV_TEHm(&THjEcJUKt z!J1^ugxe1DL&G`82?|oap?`HeRW;qlpr-}(X)V?%kI!A=LvMs@W&~L}zIa1(2^M98 zoPus+fRO90;rP<>O^YP%&ryP_nAl?OnZsv%!sG$s{1H+o;Qjj-?vUBUJi>Sv>eFnh zoHp1`J-z3K!yV**>qKehL;8bHe-GA?E1f&Jad;4U0AJH8Db9r!u7`2bR*2t3X@DQe zkV8I;kKnUKk*7arO*8QhdD6IcN>{4phyQ3Ny@OhFtZ2iHM zF$nZm^7EgSTuc9`UAYMzN||qo2guz&)J}!gt#bM(J_Wq`%c9Ec$b~V99&8TJG?_mS zX&Jv$Y?mnbl*Zq&+-uW&@WKV|_Xu>FJ;|SF$CcP3S!t!_BD)lQGyo~Fs$heP)@hBn za=adY?5*a}eT~r2aawacN~k>lCHrn)hiS-6e5_>g?XQ7OXmW%;Z$ZYj>M(ne^vsm*S7UC#29 zc7B;I+4M0R?REN?G68_G8ninv{znz{w~*47{a=)Q1yo#Fwsx=p!6mqRa6*8>2~O}} zAy{zNLaK0g34!1eAi>?;-3d-`cMWd;P4~=uGd=x!`e(6N+*-G)&ONgCx4&%^vWKD! zDow1zcOEmvR|WXM{_xv)o+v@*DLExn?(qD!_*QTTjzqH|c!P0YlJ>#kwzqRH*l-6r z+VRA#%XuJHUUiy$;hp4irGCMdGE`7XTikI?%8L9%p#EISs)Wi*Rn$xTMB`St+|BSEAscgrxJEu4AbTb5kFi4ExtjERB!1gWP~V8*9LvL(3k!7`4ax07mG zB~+r7db2MC9eSB^LSX|0rxTV0xO-}{@R$TkmA*Fun*&`CFO`^s@CDZDi8!D;mt0ER zif?ie&%Ov_7~J4L51%e~CEep1#J2r#PLvIRgBg0IwWgmlE<4<0@J|hQhAXy(P4RE~ zyQ7*~UWsyC7fl~lOgN@2RWRmnDFnb2h3en8*4ssGgddsgcao)B{)!o6e$2pwHO9}E zgfE?`Uh_<-daw`Mv6<0#SL_76@#&F~;twtUejh0-de8mWHknQ*quuFX?`uk;Q86pFC!C)`+P7Ml zj7I4C08MqL$RGC^x6nLFUc|G?p0fDpc2k(6bb04qJN3|_U$-9ni#VSg3Np42Rq%$+({}FJY{cB9X$A3kZ=$RHpuc`?eGpw)Gjr8~PJH6>rsgz`3RSCg#I+F7&3M0@ z>WQXR`ugEeBCCBRc%c#BtP!U{bp*i!?aBi~l10$0U9ccnEhpC%UU*12U*CE`M~8X* z7&s~vus()cwuwVHJ271y=?(5JB`T4Z(@i5ea)nmBubIK~yMOw%1{g!<1*7v%% zE0?y6D;fTUibTTPRy=Oo_Eg9cihty3|52C1D@NqvPQrezpH{nW`_@)rpbl741My`J zk}iXZ*oa^Ia9M}u3uQ9#^EXlG)Kj7OdSChw<}By9QyR}(wQVd;I>(iYdn6d%{&3!Z z+5G_RCE3%;?>q!SHqaIHnl1a^(w?`Dfq4esQD#JO*mo%x;)XcY{nQotn3w#&4#aAevjkvZ z$S@1rV_P+(O7?1EO;MW8NcXhUHGyU}Z~1USO%-h#{j7Zg`i~Uk!IqEpr%0Yh$_Pu` zjKNQ7Ir}Rp8WA5}&oZpqt_cBG-186rWh)cCQniT~#B}7oe`^IE*l>^j-f#%Z zUcM0?z-%tEz#7&!vek`}Q!5Yj1^+OjI=n&5u-r)bWRKJPnH$S>sy|E{`zc@IgcE5C za{5F+HO2_L?FY2IIc4|+)Kt^VrIZD;_jH_Z!;lIM>n6yC5&SoGKt(G{+doX0=;imX#`a*ZwtXW%%1rFquv9%o_MP zkPloywI7dDHg@8lm3GlNx+Ti>DXF$|*A#rh+4&T((I@15h(~l^ zIS={CRQiC1qJMFNlW=SO+^l&?hrW1KY_>FG4CZj;4vFb9CB7GC5B0}Pu>uhH%?{{h z9Mhc2P9R6(czU?vSYZVj4W&dGK7T{*MKY+LG7)kh#ic=XLi1p-sy?{jh@E!IkNynm z-#iqW39Vx`VI{o{T5?`Jwf3S$^l3qrm`W?61j)vCa{*e)a z2*J*xeD-KYfBbTuBYYS7jk709yep|!Ef?P1Bhv7CKJSU&3>Q z(9c~XP4&yGi7~x8UtjgB=7j-S_^ay~9#@u&xGFZ9z8V&`s@5~l0DHZ4asa=~bwzG} z3Gbo?O&;g{ieEOvyK-3V1q0b@SLQ?xyo@MOITwIsnz$<^eHP0QB$bizPQdzA=)lc| zm8PF{L0qO?V!w1cFa2gXu|XpGXW~k6J0;91=A;my{~wj9PiC#%1uo;1x$mT`iD3sh zIGW1(DbVomSt58N=%~-|oscb~0y)1~H_#tS;$Rc#v-!?^z6FfvR#Vv3e?1@ zCD&1n806nRNn5Be!sX}=Am$OP^-{-4G}70Ul+HYU;#~0;4QCWeqCyRtfzK}U&bo0F zjbv?3?LG@o`tbZJXvFig+fMTjnv8?)*cPePqlZmIT8V=Du1Z9F>7GHUk>rhfu^)_% z*6OC3@T;9uwA0`(_0TRVz)ePqKXOA+<$~wwLC}POOz@fHeRcZE1W<&W{aT|# zQeq3J!O?rwfw}zHiqx0)-MDjgshbF<*75wNm$ChH=0Q(ObNA}CkI%bT*0|3;7l+@F zd-&o|$=PZ*kFKM6(yX5Hh|wpW0aO%~Oggj!lXxHpgy@pe`Qo?4nk*^%fYH*_81ykNE`;0qeHT{QrctnplT z^aIWgO0tk_*&T~>yN5yKJuaQ!T^)E)OE0@ELD2q65EQ~z99C04_p#(hJIAM%vV^DZ zfc{33YSq-a<+7#u(?|Q`lb&OPC}B&A%&)e9Dh--ptk-GPWP7jY!ZpKtO06?okF!sP ztwMjQbg62w^sdSzGiW}m?Q)Jk9@*&n0b=f~%F4pgPSN>D1AT)GS2Q+iu(IEmHMK%M z-vrkJUYY(-6u!QEil7I0DdfKgPdkuAUFFiIHRFY)Ga$bW^4HmlQS1Oc)8Au%fi-1pyHUr_`qzwERy}Y2D24vF?#S;5<(G}1p|GrMXIIPXyD>j| zZth>!gLE{pwMv9S+4dUi23~YvG9^{#grU=uQ3)SkgBo?I*&eON2B`NzJbeeWoMVT7 zdet%8)in_$xlN9Z?HRE`$3wp|ZYhZF;XL$!aj3a9QRiZ<9nz#VoxCV@Vbyg>O0+z< zB&C$^V138#Ec9}-*L3#^C~Si%2(zUmGb8`!Q0jk%``bT|d-%7ey{OMcSmOA^)U`q2 zeutaEoTI!JpvFMV`hmwTZy{OrR=U5Vkjo9Ie4UY?Sg*g5aAkR&Cap2&nBf(C&*hPS zpni1%t?RzW$%Wd;3eX%c$!-EjoU#v{AY_)0P)ebZu-L03qd9qFeu@*sRLMJbJd!tV z+TCq@X8nc>VOk|eVo0yiU{Uoff1welUxvZrJn)x2DX!S6%zUG6_4=Pj>Yrfw57Ot? z<#9i9Y{4v2q5~c#hOS0T507p1+nE**kFRHk?=JCyTG|mnKy0!Qf~=Z&q-fXI5y|l_ zZ-1q!yZSDs$w1k{chW4^KEn4a_%yPKk^+8WrH1#1CG5Z@lgSzF&715Mm&SIpstnoq}3ZqQ~O|u;iopmgGAzyNULi!1H{8 zCj+wB@96#k*8V-}?B8B&B^Lc~@{_}C^a2d|rVx~jO$6QSO)p%mxrnRP5~a6%n0MAY z_cLv$Ip+7mi+Ot44fs>#&h|V3IxmOp>dR!0+57{FK~fKKo3=VC)l#Lk$T=c zk9$EeL6<;LaR^YJ>atGK1svrU^ngKiu(nC2A<8YxuAfTH{Z>(H=$e=S6SJ&8rNHHA zO)3Kk89Ci{=KD78xyL)02+0I4S{PLS}=S^!?j-qZ{0kJjrcQGZ6a{6Y5rcBzX1 zgBa-T`TQL(-JcO~f424SZsBPVOJ`by*-Za#VMBn2=mX-2XWH6UXfgU%e%XtE@Att2 zb`^$%i0I~MWPtGhEZOoOpQZZ}28bEyqjmuO4aM>Mn)ytNiU@e8rljZ&Or*sA-SJ5; zZ(!TG|J)uMAdSxYQy1O8iHm^G|1pCi-oPWl@csQq(so8!tI)yO z5!cgrKboCzUKMI}9J<#>at76C>d{|VclLKxj21g_HXKkhF>7Wr)H~iWlv~6B)@`FB zz*J`J{LmdHWW!Go2u_c_w5R?TYB)Dsd6KyQ%nx% zX5lv&_7zrYl!j^4sQt3Dxw8{8t0J{FlXh;hUsQlkZJjicZL|2(K3Dz3qqe?YL@=0T z&wA>s5O>LYqp7_6=A&x~BiRo?U{)&_2NUDNv(^#$2-5#9a+dXIm1;kDqF$e9f%rfM z`|F8-Ck&6AJ2!$%&(mURtjQ980>=k3^3XxRtIziec0ahy1B)rHATRy#2WWjT4#BHP zH);w0KBS(Dse?tmaH~1cr$Z`9$e5fqo6zNR0M||FoiX#fT zC?a2~)jYTTD`X?Nbi9`PW~8MHQnOu+k{|vd)K7&kKiSFO3N{dK!(Wg#|0AUusbCX1vwJj8;;MpM_=XsDRgOJ=%u9AV66D1L_U1sb+WhbQuJuZTtS?)7ZY%N z6taI7U!o&c{xwA2!>Wee{(L3oB97x%I1owq6{lNup=9YoJveHo!e;hJI<+08d5di| z{0IRwz&HL4Z2-o~R*Cv;6xlPKfedP|uv`zAJQcc{=~!c6avwi>R~)uOn2Ud>C;rQJ z@!^SSHShbDENDpw3T-7KBBGm*V}38~u=8|nIFt7jfu(18?QM`cX}0V4#ss|f>0D}*?xXzS#&M{3=*OYyZTKL=1PLH{7*x@{#|V~ zlEA)b`jpibgw}ypt$wty=sZ7`ld|~?-@RE)RL9v z%S`!!Y(RuOZ89v4w1YeK2f*li*x@)GsLd2v2@R zn)uQ+ODC+G;P&hZ&VtkQqTlwFrb|T{o9jdHNz0u-mfvmC*|&k|FOnM$Qw*g}NtIpg ziECq7TBa&3kITxC97VsLjA*G4U>lYd(}F-V>l9?%4zSsS35MMef@4fWy8FP3qNERV z3HsexF5}fY>LZ5MuiO?59L;}_q&r&$SgiuJSfp_L7_{Ite15eeD*ERc9|jE0I@-aw(KsEFfx4Tv=7y+@ghz;74WBgVol z15?KnSUte0FGgy5n7V;SUQIr?yt4RpY6qx4zLKFDg}d4~S(pBE4DqkP0x)W|>m0bg z^WL%Qg;Xjq=>{z4M+OpM^D%}ofR^f=`13al0rh^>15dJ?%j*pUPIz4m!xrWjqI9H7 zBUQtOW-=J)<1;$D@H{1+u4rF^V_$v`v3tFLJ;U%6Y~Q;a4%9oAsDJ2XWMO7e{wm6nkeFOx zqI|&hI+KQD_gbR5hFk?-R#fDX;P_?1-x@k9vLgYAOl69+ZJjHK9eSXTn$rA?E7`?2 zQG%o+aOtGjV8Nu+We~H7(CY11(W%CO%*hKao_>u=REOsbWJUUCxG((tWeT0FQ>8(y znpLa@U7_EYZsYLN2WX9n=v8fIUVf)_9|ngwemPP~Yyi3#vS{CgNhugu2LaL8)UQ`R zn&M9C#?UE-yl3I_I&p9DlDzx62hC z_ce|VM6fXk#@b@73uh$VY?1?-SN5UbmdCvWjx45z)e;BxZ|qVLIoj5_A?g}_pt}lk zJ!&G-87S`fcH0jZeMc!n0c_M8N1SHkVsCfrstu}Ew5Cp$#WEUF!HicRr5ss_EG;eV z7^qE-FYiH(^`BRp1;C5(dF6Xhy23w^6O_{eA5M&=}AXZ*}op)*@a0k;r?e!*%uij6Pq_FVf?9`@x!IzW(X+y@!w} z@}@G|H3v0~ucGp}1*$0a%!Ou;L8aCk?(hgtGjF#w`ztE1m*9?9XK}*!#_@*hP}-|h zjRy(GqXhh{N9~tE%qbvTr}y^};#1Y2W;c`zTg`x|#iPlYkWX`&t})Jb!=K3&8_5hd z2FuJwoAEazcgPhg!-G+RSXdvWRE)lv;;tVB;UGGH*xC(vN<H93AnO$+^-c9RU^x+6>ArS0PK{Fui~ zm2-5P_Kn}^yRH}T;QmU5T90di>JrT_AaF7x+7HZ{9_fQvWT>{Rf#_poJ>NaYRsDxV zMh2at=A^BeG?seP80K$q?;Xi~vAlRDo%&KZg`dS21kPsNQ0caJZu7YlNq%A&F%>A} z#l{ypGk zu|!!``g$?wqf6E)J zIq%%y23I9C)Z?20M#tx3i+Y}Z(=Gc^8ZDVu0|r|o`NK)4eggX!oWfOFk#DXnD`K!5 zB?lc1JTv9)tZ@p28FN}66Vf`l6YC?bE%L=DfI?JESWZJJA$Bj6$)GPfUlC8&>~o>o z7p^jLxNaqNy}6T5q)W|LiTLSBu+~0F($9|`vH|qS=}fv-QEL27-dlvv&8tsk3ApHp zlH-TFeNX(N>9RO?t9G(^9JY!WH5V^+7QSY^$^Q++3n#XoWoCuFp^aUYz<*)K3LBGi6nr1VS`C zDD~Fca>u;M(jymBpTW7~^#!}8`%XFtrMR^Egp(AX;U0=B- z;GUBCK)BU1z=Rz`(EI)`6t;*Q<;U-B$i^10C`x)%`u%t`GCn~zuaWlpIAbH4a5Zz8 zG>cemboq+j_-L)uX3cEF%;$W}d9IpZI~mX86#IoBXMUX*RKl z?1Kp|x8!E`huB6R59ySKoc^~;tXk+^^9`EDRk|aHE~Vx>%(yIiaPf+y=sbpL?ca_+ zb}pq_;^pyOyit(4hJx$ydKS?JN$jUM2GP~m)K2Mn3`>vK5AaoV}_iB#f2#(E@?W)`kCA5DRU?_{}lCSL&=%h zoY#8TQd}73Uc$nTWUqww*>@KS@Qau2CV|dL)FtarHfB9;mRY|J#FCoIK#VJ>GQ4Jg zD*tpBoDg&~$N17zlE_0SbhMQJ@<8f^>!Oy%@Fdiejh>Zy%Qky{XlKQFkLXiO&w(}e zn_2MZgQdly57_+TBvaj^&9drwVGEY=yN#8z9dkZS9x|_{8VioJn?%}wG8cuN*xoFh ze(Iyf!dynYx60e>{p~RK{(pck(5?Z@C&l*HNzZMoHo+&e00yxnIkBI~15845s@272QKD z-zn!1a=y!@&X$;E>2`5P>v+U3?P6l6o}|aOh&=J@57cu~-n<{L<+l6WrErXs~D0tDevP>EpJE zluWbdayvOn7DEAn$mb4?Dql_ zfyYzAN5+z6pgZvDT;19YV8oEH*0~qaZF4!34U;VTfu$~1Cv!^2>Lr!6x5GNfXnf1f zhT=Yn+PKZPWZ`(#?Y9*%-JNT8uVx4yO!LO0tkO$||I`HMV=m5794$6ag|ujQ@12}H zh^$t?9WK-sa;Ypa%iUbx;?#VtKYb#s_nbQ!VL7XTCwlr+>G{j+_P!2~m!l#GOLK2~ zIW}4#WZJxD_x7+#R{9{}wZ`TS9@X*QvvBQWBF($W{k9>d+@<@UD-GM?Xx_z7<$aK%hv<#Xmr}+r1F{;$`s=kX64od&iFoTU^~+Skp&r3T8q#9DZGo3+ksaoAo`Xrxc#1$a+-_c-3W>_m9Kq!iV|x%4lUh^r>8gN>kuY@BH8-VxAB#9T@#MzrLBrK@bI*rfsU%9Wv8^w#eb2IKor=+jmOI0?jx*Lc_ge3wbY|}eZdj(^Zk`!QMz8j zR`&+Xdfng%cVVJtXWQ*G8*1%=c8$No*`l(1Nt$5bYwNu)B7JXYj5MU=`}G9EL?KI* z_MC+$oJdIE?7Mr9wv8l&Gq#j0cHM&J6Tu*@O;gheDqfeHgQ3BzLE^NXC)h1n<-uvD zs;03`+1IGx+A}#^2lt&yEXJCbTa^#ezIO*qTc4$%s%MWguFqgt$XbOjuzL3aP<;o1 zUE-XtsOEo=Y6p7NkJa?(^?s&KW*lBkK^AXkj--H4=XiJqN2m#%4 zKd+E*>!9lET|-b`mZ}!UG$C-nQ#yoms7f})eoUPf4Zq?@?y;HAV^bd_lDf9nQcj!G zVeBj`&R9eU$`h)#%(^k^TMISu!svwQS01E{pOs^nK8a7rxfMKxb$)yMU6g~LcPrR| z!GpDrfris9=ECB=%$Cfc?ifjza>dtm^C8}dr5Jj4;w`sG#Y+-A{=&MYCS*1Y9+HK+ z*BZ~`ID#@4>r66kRBgYLk#Nr?JlU5twBO!tST`;pbTc_iydjZIWsPYAo0upzk>ww+ zG?ROnmEhI0_Y2s3SG@MhO6u@y59S9MbDPC6>g2kVzm|AqlL?mO#dpdkx&GKYrw%o4 z@wz9fdffM|DbZc!gTnW?qXdRPp7h-H&@kZw7W5Xq-O=k_;bQI6K$@s&XVu1Fra<)V zZwu`QE7T;y9_p9+Fg%6UKdKPFy>N7_`$Q`(AcR}}-en~7Y?7EKI<0dxx3JXGbMmYH zne)49SHUzvuaHXfw^9YRs)n*ne`yad1-8=mXC-sE>DI>Picpmxyj5oSaN#UiX4}vP zk2ZI%YCsiaTf9__I7}}51;>BkXwlm;v7g31s{ zn~p3pX_*32j6ix3ENd52?k4RETyB)ipqdi3I)7rPZCOj?=kiq?P@#)bfG zK^lw81}<|=JI@!SMkE0sKFPUa+OxHMeg-t6vB{e4yZ@n_XM+MVvQu|I#bXI$zSg$) zX}9o-q@~^5*k;yz_g*>IPHMtx>UB{}&j&Uey_P5j-8^)_QXNZ_iJ~$tkhFvhlOCkx zJUIn$?{K5xCdqPRm!8EKZx#=08q6cmVpL}T{KOO3XL{{bjL#_w_$W5xuMdk`=8`wkck7GnD! zB7ltlVgejlIGm5sGBJKwqPtwmigDOba7@(na7eHMc^~lk8!g^QqHiDPrv61^*KWsk z2!E9|)<#kl$GX;EF-pIHQ9u`O)@_E#S)|$by!8I_2HWvB*mKo8bj}Si^8N4wl!gVSZnt#V)aDa6ANk zzZFFsyet1>ZKT+5S#hQC5(w>4336uSsD<4SBbzJ=*aiDHx}n4uMZV-B`p^T|*K7L= zmDp~V7x_{{zz#Qc~W7Tl5%*`&ww z18^wxsBOzTK>Vvs_`s$y;LKZM=vv?W44!F$a%~tZWDxras*I2e=e~R26;X1!I84kh zqEX^yf3u9<-FS^T7|dAc@wHIxCYx!j*^T&-Z@Mo5Ez}rF^metbmzbP1Yidc?i#7c@Ty@sj z8Q16`=B45F`BX9ftM-qyLzjq1;_+vOP&YRz(pvuPqW4qO@cyl~GRx8N_$ef$E!T@c z)7Wqpt?FF+D$9dp6yT3mi=0zvHwKRj#w*Kl!h-nW0H;r#l5Uk|m-wL{>21!s&z;T} zg@kfdD=0TdQi5OQsp#LNNW80Z_Uoq{j9}Gkio}*c$OaeL6E=(geNX;9xUT z***)33E#MnMzo#1oi{0k#K?vfWv(XM zWh#O!TSCG{E9?5dM1j&dYr=Zp5k#4lV4a5)-hoHV zj10TJ@@WxUCgch;bC|1@1q}9$?lYxJT#go^?pUGG z)LMAwFW%Qupp)`SK2|@wmLl4EVggfVJRXQkS5JV)ruXd@QN;+3T)-8!;WEJgCp!<1 zSkdlhrObEwDMI*3mz3oXZ|VP!1^A8LY4ws_&bkberZ<9Nr88FqIEY!e)Ey4`FuX5M; z<`GZ5M=T3zgchhVnkwO6Y;r~LLvKhrtI9>&?x4r52b2owyi%4-1l4)-f61<6iXqpk zNN?UhtO&9hZaa8|8kb266}xCX6I?)gzWa){#N0MT*!c-!P4UBvDYmdXLF5F7W@#LU z4E+BwYQKmv^~aptAp}MeVLW)TC0oMa-%U2FRnjfi8)A@+fVdl$?Wd%I%uEOx*1fULsiJV&%x2KQb1#;KcVOCz|L9tUx_X~=pJh-h#C$IGo z+&*i4WAp1&lW6?_Z+qBJa5EtM7tdk;l;$>g>XQ%$2QpELRs+5ea~x@AjwZn0JJMWH z;x`HC^=4TXe!fJ9LP|$^?cBB<4N~86 zFje*Fev+NN*{jt^GE6py?A3BaJjVyPK|o(!x&zj3spoU5QUzZckAw~6M70mL!1}8!(=TM=rKh z+Ok$EoX2O7llAD=y4y(n(h>x)WgjvFDA81?YyNZ+{!>`Cyzl|3pB#_uI{{I>{Bnkm z3DSX^H09ifqr?E==y{p*@nr-KBaao~ioj0A=IfEMis*?c^lTa>KCJjpYI{`bh41ne zyXuEN93@cx8nXU#VakVnJ|tVc7E~U&lCrZ==-;ZsL8Q??#OdJs z$L(Q5)Bc@Irg2sa?jwKK12^-UmJ=6txmg?*5a^00kU|Up&)5Htk3LG@3i|^GgNHd? z=zYsoP1CSiDJb$_F&rSdnVzy;Kvz4Ptsjx2%DISv2Ci_++MPFX0&4F&BjTR`Z2xEv zUWuN-WJx5)G%GD5h0Dq(g82;A#U{^hQ);<#YS_i}t6pB}K`kxhBric(Yu+a_m5z8O zQHO|LOHrj?o7AjFiFBmodO9DS+5v9g(amBa10(QpA-pk!`hsSP>*0XaAONX#72)oO zAau6$FH!&R>x|ifRLWych<)pT&lmzZ!CC3Q#+j)!3shL&JsmN!(iFn~)_XR=p~9^V zvd47eX6*s9VN~H1c`Z^Ie<17m)J{m=noY}zg-h^d7f1Qs0)QjlF6~1gM|kK*^T8-R z7XewVi%9_k8h*4;7>tB^68E0ix^MR%~Al0)IX z*wS;oc3TQQ7fn5*EP?*$M_^FNg86PT7%H2Cp2v~TV}pqQ?C9=vR;4hFlgy)4VJcWiS~RV-_#30)aDwN1PQRmlgwPD}Z*!&8(HA!XDl?^K zM92vvYRTI|h&F?6!y1LCo8dRgaND|JZOv+@TQ;GhugO~S;7gtwv z1k>op_6@pfnPRoQ=BWL-U_Y(t)-eC^S?b%B-C0A%6t=KLHR9^GzK7jioS00`R+ZZ) z7@tN7&U(;0=}se^Ug(lN->n{v=&~%HZG^k{?hZg|A8;}3w!D6rUJTFHS0A0@?KYOm zuN;|%$|o}My!3z<${*kt^-sUFn3Y_PdaTR61Hg6Q7G@jCoa^Rj+B;6KXUJV%A5ATG z7;C5CzD%nc1?0cF)wH|_WXxpfG%vzU(uTh6oZ+|Uv z$}7z`e2Cn!x0l6j+9`@PA|a=h*-aKL2yaxsPc|&n6~pTnc?aCN|9sneP=K$BB^+~O z-LQswV7mLUIG>rZdhVUxsmIAWrR(9fa)AW?yYYD|lGLloiXJ2yZY%U1POPA(7?1!XaG*o z>ft#GP*007PL9YY^SR}Vj5vf2qZg!h8-w=G8LxKh)(C*KwA#{kId8E8z$t;-EF?q& zyVMa>L#gi!jVLgQ&^Qx@t_R}gk*Y<0!1Fp@a|#o!rb<5CkfD;q9`mvD=8Lu|e}KHyJKTbxXz z5d{-iTVgvLkdCi`)8G`bh4rZ*UXYTsfQPTlFTvM_QHT%`(=J9EZ5@S$3Yur8^lYYh z+q0mQnjcNckdsao9(cFNbP6_YA^I`>g1svQe1V>Dtfpw*8kDDsdI6p|_*q6(H|$v? zBUa4)XR`2$=@IA1Tq*0U+&MxBd#LrZ~p=T z{VzEu?ENGNH`34?`Lbc!sp9qq5)-s2VwO`ZEVO&ex;TT>_*^xDw=X4D4TK8hhDQIJ`gL#ZZ7HaRv6&@Uv}P`GZjQ+ z>i63_ThXbdt&e4p^1Vg2duLfa;CS%1l4NSB1r6ib(fz{$@iVPdwI+s-G40A=h_Q#8 zTxPxrF!5FQpKLwR0UuZQ^HUl{G0L2?Pt#5Sk@Qi}mQrqe39GqTh!KPRXfmNIBHep9 zLjS?grbqO_hsARA>%e0yPhDAai)G@Af*Vv_xxY2|cG6>@Tr)PyMfg-DVUrK{&D@b8zufK; zGJs_!0;5i~^xhhbaZ><&A}NZZB#iNVxWP0uEZ#jC6V(SYB_c;gT z7M2bwC2i^%43K~3CUExCyT^~eR5n!-S`tJmB$4E}jE@pd=myzxV2ND){K&3;pB&fa zi;U80@HgS1aAa#mPZTC5fwxf}iDTIEe9OhU@$I(fgl_%_nHvQ=!xJ9Xl=cj3OQ-aX zb*uvD>}hLMubAV)T_1tmO8P9AhmY6J3FR4hMJAIp3|uBJ=70;XvJO?vpNv6jzYZj% znyVRGS(7x#5p(U(E(1yfs!{s#-Z=$Y)o~Qt@>}U}AWFH|wDO;Rx+mLISjlM;n0FlU zPDd0*912U`3O@v2iCBiqrj@-1nI0xb2$=GwfP3CcIHoI9h2^=|H%f62Yoz?Bg8y#Z zC!@Q;gF5VO@#T`5&+0rWsKIeC#ZS~{YvMC~wM<|zmh?&$id(T&^$rOJ=blFb zKJ;&ZH9*+6dMQ!_cP_=OS80~WyUNY5g zoc7usRe*;VPowAITidG%K~Fb~)vGl~#ryF3(`9;%nWiI|08H#Uo=<0d)7NpOcSd+B zoSSy94}H7NbFNIL=`W8&Hg3GAzWK!@g@KPFb(*E14W5T!z%-F9THvm0vhlc699wEgod;aQmuPm?Q-7H9s`e8myVN+B#ox z6S>@^uXeLNFU_6L-%^^dcgHg|Z^0tma$Bsir%g!_lQ6RZsBfXKC@Oek1TF9D@^rmD?eyUE2)Jhjyo?iq~s zVAwh~mw7gU&)|jJot;+MI9_Wbq|~o`t(lF@EzNP`87{{R3d^3j><;HFfto*HFq0B4kbX>(KjX3;nQ7jVLaLjr9vU8fiM3aaw5Svm$Rm7*9P$*6F0V#9c}tLM2DK#*6? zR;*()lg5UzaLaqFDK^+lY`-5_kF9h7?(0>)!m{I`I&j2ffc0(FWlG@Z>6QDH)Uk}r zTaxnMjSzO|2mGux_ks4{nd)6$RD4C!r(1ZDy{mN6xt-_14sXutDiz|e0k25l zEtqwyv$`5`c)ndQyxk(}qTjhRbiA5pDlYp#J78^HNQ-#Pc5P&=0!UzY69tfW zT)6t_Y>|6{;cra0k}+7}p8eZi{@05;OxQLbA3f%WQ(s;ZNAXp*>5417&r-BCw6sJ^!i^q&!2pZYyY;Clff{fFk_KUTZSI_?nZDA;w%mZ;H-xkdMh(%d%X|znV z1%gRBF3dk>&Sw38?VI0@{na(nAi)%zF2>P)X#hD8B?th5d;K%7zD~;s|GtM$_MXi` zZR{I&E&IN#zsInVB2PHT2RVb$JJznd(pX_o$|gK*-1BFSh6U}E_)Fv5ca!L_gRYemTw2;#for1>WmUthwb&Ydn=q=Yz&dZGAM(DGL%G%dvG|PU1z#u4D_OIbm@; zxHf2Ajp>b>BC1QZQt~*)^W$qaSMMeiwAk>SF2Kk?I0PS-%3|Ylq44IN+L7iix6&$x z>v*B(*a*EQqyJ%d0O~wvJ`G%yGAJ=Th2K_Xp(nzXRn%?oVNi7pp5BeDut^xpqY3>; zF>F11_VfGwR3BK2L$kt&bSv5C#5DB8Aq1SWv>q1gBwElrquYDWL1s&v18Q#AO;uws;s?KlHwwiFb()+{|i4()6 z*-dDrgJh|R#!FN(;c^FYVrM&$h-B9oD^V$-413N^*aR0>sBk|sqW%*#2SeJM8VgQS(tX@9|LA$e-MiWhS8u$0@x9c-5B2v1 z_65;UF?D-W^x8v{71g{Obc$rb%HAempfy4^qEyoWffH?fz87Vs7pD4v_CCM zT$P9161oI2R;9~$Ly3)-(740#zl?_>iWBT`J1eYuh05t-r)@aL8<~6%plF=L*9$da z7IKP8Gd>S()Nx(jMZ`(ID`kW)O?qdyqOVT8#9MAkL z#klA-ndwOdZ^{Iy`nY3ELhe%G;}Fa%6Sw-k?*r?fXVUWs)W#2rvPME{)XsuE;=Btj zg&v(RL8xT~sTQlziafG<-nb4dp~TOh{8+a5ciDFO^W(F0##ZiDiQ#=rW2oAF6l&}< za$5#BctgT^Ds#OvywC`~Ed)TvZ}Z%NNfUo&LB}7q`0Oo~`lDRzeU+SQq^Qf*B!Np! z`;^d}oHk)}=BdShZ4CJFQ*u=C+NtWK%U`yny56|i44rd6sR)zusSNq-Q~IB)BLEto zBl#q{H-nRJa|F2yNi@@rWC~D7rJ8hF+$qJjrOrF|lvZBzeq1|((?L>mLMgOZ-`Kq2 zD^Ti3_rM{_CK@_UoNl0Wj3|IOpQA2dcW=UAetdwc*jpDT$|V{ggj1%j5EM?1ax!lk zxR>d6=y{6b1X5YwnTx0USSs4DiIEI3P8}@%^x*ywIWy$@C5%i%L7+*{8C6iGdM9$u zEMi${B%}4JBPb$}tgS`c)BCzlL}~2%F&@TewHqPU#=Ix2{p7yAwj1a>G(Sbm{mu?9 zh&v{MQZw8MH5o%p`HJhPYuZxG1RU{|iez?4|r1a;< zu2pVI07k?B01qhwwHbO#_qiOI0M$)zS{$+VsVO3uw<T$389Nq35TWfp4zD4aQ_n<} zRjx3AX7%Cm&2mPopWS}Wplp1LCrvf;TTr#jMT&&;qp^!|@xB&G(nl)gj5f2)&X5%J zotdr3$-}JSjPA{n`2>M4ZqHG;%=po3nO;iP-qG@VBl2l<==5*r^dJIE4Lo(QP zSFtJ{uu|A?S-BEYN!ZGmq&FEKXhTZq(x5ne-{U~ctlUcYW-!mW2@ZaJX zSnukb!ZN7SQvU7?x~P@0Ne zKBb8k83Qv2k>%|S`GhO6i*y?qdU_a)1&-&?*4q++*7>5uq!W@Gq=&^HFOAHRSuzj6 zZ=RE3Othig=!0t17E9^D+izWnWgfzUo~MUoNmjB*+R@Wsj*`xuxk7duCad&PlrX}`M9{AD5*J5{ttU`x18foNjg_6(`IGj`!pIp z9b{>v!uFqD^?!OIFMX`uwbN^hA!XFT12`lH-u&<|kp~T< z?<6C34ND7xm{iWRSW&>M3SO+1GQT5!rO_55`=MUbp`_t>5s0;HC?5tW#*g=ZZ5*_6 z?_f+sKA(J^vRHm39Q(|Z%6n8{69fu2z&>m0t_MVQ)U#^?!&=wU6t@p))oLtr#2&d* zkA(@pUT^La-8CKcQ*S4r0hJT5)M9k%!);%rBqq|L*6KJXRC5060g)VN)hTm1@>zO} zW>pzMS0;(rtq*;AC35IAv4<@vlH_|5Ixk*`ym%ua`pK5$kZ>5kBXu-1Aw*^ZT+Q#4_p%&S2wXMD)~nNhgJY|!@vz77|t zu?`7-Qzd(?lj#ncS}3_=j-E!LPzl1S_+Xa6!_1?3c_dvnU~o%u9I3i5o|O`YU*|wU z=KgRO>U?n~s6E+iF4PLqn;gV2^zM6!(X4=W2bm`kRe_mdvm@>qN2{1y^-vtbo~aN{ z7N@K@s5+by%Pc-nOxeu~Ntipxzx1tkp9Y;hXHXT$`9WwOYvcvaca|7k&G2qD4+aTH zix|55c9s_seXr+fxE8)O>C|5%#Vq1-7O=#R0A#K0&M{uZR1`i$whgo;z%x)9+xhuu zZ2b;rFq0Mj>drJAXnhndQ7{zdKF_XiIlJN{RbELseH`Y0_6$S`_?NRuDmn+ zV*=llF|P`_U|v(HWI1#>uzs}OOvW{e=H4l0rm{CX%S9W9-%xK)7lSp+U^I=^w_Ma; zlhQ?J4R`6+a8%E2P&2u~M!8JMy3{9Xym+~%NG(Q{P;E_K!AHL6x`52=RpQ%s?i8Cp z7cUT)rYlR*IZ-p`>IJ@;Sm4RGP-W*1drwv=-pN#T@|dff(CJ*p?<(!E18$T|o4S-v zR>^?CP4-F-y0FWZ+RH#@{N5%E^#)?Qx$khf*lK$_Ygl-P@g#|H$^kpiywU%22$KSR zkYK<%mpgCA($opq)`De;yRre;#D@F1qH>SjPkA@2$K9{~0(5Uh^#Ks{_4aK}lL-sf zVa9cDaZqD?={J}SSPEL!DfspM229!y-yvOQ37#xRKx`C{evqf(60=r@xka4q6U&4) zU!EPQp3dq4h_L%-AqAaMTAO-)E)}m$B<;iM73Xp_*c2J_c^2Im6|7*m6Mhn>xs z60^5(5+FPAny12v|H3Qb%eV-b+RaCQ(?kL2!(1fxluO7mjnVqPQBsi zG+2jU3D#3)i%@aSo?6_Sy@BCV09$Q zuAJ>UiNXrtko;mL6+kqyG)>TFsg$bvyY-5Lx`Q90pn->hyk$OP!zc8hn5XQm6+A90 zsJm}E!L~Ii`vRV~ORS`|{%_3t?_JrPS+t~wU#+sfFKJnweiNU{qJ$tp_w&aeK)}qa3jOsX2FVNsYaD)S73wC=K4$&Rf#L*uB zVad;k%^g*TwCXwde#*ic#2EtpDruYO9#Xe-4 zy5$V47H^NJvHG)TOz@RFDI>u~<-6YUjo1%&hCkY^z{$Sc&A5jc+z$ZwM#|8(Iqwnp z5w@DPHa&`YZQ+n|u)AOYqVTHe{lU zj_7nwIUvs1g6xdTu;W~oF;pBwlS1BP)S}pG=U70J3!}Pi#+fCk%=yO&h{Xhj3CYe; zT3^g=chTj{dFy1#sKS7Z8GeDAgy4zd)-}<%Xhr2Yxr^tVX9pbkdi$=5JEOp8LJ(0j zBN5C84hK$cMDqjUbHcA^f7;>ByNpVGE5=kMNUi9_ za&2@caeFM>QvXx1eU%V%uF@Rm;qhUj+(757@{u%t49^c&UJ%*H<@pL5$aa(W{fzE+ zM!xl9ZFe*b)=mpgQRH15yLBq$|BcyD>TZ&{kA}zY|xf)x23Eg1n%Is{Em56&|vTggn6MVs@|lQgEGZUP8U;qa7zO zZftPe_83KL^RfEyHbU;ih-~IUvDJVx=#4q0Y*cb$AkRHzuE5XBMaMpTef%hH$6Qx0 za_+P^Z1z6=iS8@_UZp>r^yj%bNn~t2$Ajw+brow1cfQ+1U$`bMQXT+}9fllVo(LI5 z9OjP_t4x`L-oUT?2aik_{dSi9?8Vb17ZpDS}a&pwKMq)b#N^rxI5Jp#{5 zk9+QWQr+0zYL(T=$YlkpaSTg7)@q)$XBBDFe>-WFO;4?L3Bx zNcU=w%%!;uLU9^7QkMq>-NgpgK)ZPCYltF54U4U-B6JO}pTlA3HVyNAgU#-I7B5uy z;lOKpd>w%1sB+MFu?o{$%v(e_3gQw1j{2G9Q>`ywP1qcdn(|m0g^+p(kS5c-{8dq=!cpS4`7g!dLwdiKm=y^)_r0wjmNh{PJ)PX$tDz5DMrmV)+ zN*JPMzcjsZQ;6XCIESb-!t2j_UEaG{J4<+x?t}%1+J3s^rN!deY20e=7NlSX$kW4pjEM%5j=W0Zhbntd7aBXl>`lUTznjLd0kW`XP; zgMm%-de^JO{Vos1{2^#J!3gv0v`d6*bIsCpswxCCqL0m=YZ_I>Sny7Wu1j@tE|Dct z>$BvPSb$7s@0pruSJH;%j%_M-U2&2jc$sfTwJaHv;BpUl8R%BHjBBfDEVCW?j|R4G zpq)LxrbuSR3tNOBEra3Ke4_S>y;T_4K>zt~^PzFmzhNz{EIxbzY2!8^^zwx4tRCO3 z{F4@_{ie*L&Bt;p0r@zptcM!AC;HrE+#I6H#~)h~6%b@rbJ*mf_408!{zNMQ^FnseQn;5ouHvN8Ztug+rwG!?CNwA|#eCrTz{1vHk47W{? zDIA;YCH|I=25*yKUMB3o%beX zMCJNYi)bPmkj*=wi({vH(sILrCm88zg@~JI^AwsQY=E-aA5l08njOh+elOh8oI6-e zs<-m`XYbVhts8YfttF?P@asCY?^g-}9=KB*_mb#3P~Wr=eD1dx zJ|eh76&<~7&eOoZ(6UCt4s26)2~LT(kqUaNrnD2TRJn#102-Y$4QRAl-}e_j%i|>C ziEcuPt6U@7dc*wFqhvwR6k%SVK*5u!^{_DA>a(#=kcI46J$lHq?DcZ@Oo!4`!=RL2 zChrtikj1ek6Wp56vev3%;S2r+&;WU}72@any+0JP2;QB5Au) zzQ1X5S7s&ri}Nm6ww+hS^Xu_;#HUS0!_2VBejkm% zez#D=u2p}FmfQNmcAAgJu)E{v04d%&K_Es^v0i^f6)k)ah*ZASZX`?puD*yXPimGh zC#~In_+A6xNPOGB%4Y_?OyE1%bGR6|v113>Pr@j1L_wW|J4_!5T!dVBQX*%I@B?;h z$*g9t%aF5?ZSw)TJ9@?$rB@;Ga!ZJSQ_#o64aR!wGuBbkvSIA=jqzXH!R%gJhcdb0*a zkxM4|#zOG}f-s_A>pC?eT zNQ=ed)(RV0X9(aneplitSnchB%+5PU5dfTxw9jL2b?F;EI2nJS3Vy88>?gFwT*R8F zNt61XV$?g6J`2Bpee5p^+933`1&{Zlth{KhLiK*1uL|REjqH2%$r8_DGK9*ZEbhW3vR=nIK(ZnR z;D!@tj<@ET934c3h`nN=9$VW@Olp9E?B`cs`6t7VLbd7J$^AwU>Pfv5L+4$)_lsut zZI~3SE7iE8Vtlv}1zC%AamkL74lL<HO(;b!_&<(cGY@%B3x zm^0Wjp~&hf6*ZX$kMmKd)$@ypxP(cbQSZ*n%mi`{yWv#gqr0#U!?R5X8q{55ER1H> zE${GQ9tn;hq&5lPTLXC~S08eh&m&&zdu5a;-7`7T3H})ZU`K9-;=p=7k=m*$_t~EUvM2Xo0uEU z2Z~~Ke2j|>=_Kg;zCwzRm~U3RvJ<=9=7KFLqq?D4ao3xk50zi$%e$5GK81}T`H9op z6BYaJo)HJ$N~=MHEJxgS;){oTG;3;`vl@bm7SSwAIxJ%E1RFO(lQ}pMlxh0qY};LC zeWi{=TLmKpi>KTWc|Qub+{m)7tc%P;>PXqcxIbRyD59sYJ@dsj|;#PV}@g`mcVubtFhr7j2 zCsI}BhL=*sv=b!b-An7O*CS^r)mM5|yC1QH5+%o7Et&w07_SPCY4@CWX>th$H*=tp z0@ZvNBKDnEU@b&WubmFe*lQR!*bGXxbirCo%D3P5WoJ!-JJ^+jFh^@7D|ZRhZuC$l zQ`2lI*`-H&hMO-0Q=1~v--Fheu+4Dz)J(h{m+I$=M@$b>T8x3%va(>%4_tU4RuKU} zB6wy$27P574AI-PQ;zoq)Ox}q2mssyYHQx1%}Kn*rp*nibn`YXc&3}A4q#LgtU346 zLk>({A4UmYFgC^{loA)O(|j_&T!f9jt`iR&{u-v#>|X$?55own;R3P_iz26HuzXW= z-o4Cms0vP{7FUi-;59(O-DFnPGD1Qbq(3~ON(zkdJFeo@eMu{xkzgYdwuQH()nmmlFpBg zyl~y^aCX>YZB81ky7*mkR`d(;KNDw8_SIybR;R0azbo{4;_-wve>Y_zv27Z3f@AlZ+ zc~oRv@ny8Wyi3AmOs{n(j5rvmyq+yJiQ+5=s{6$h)IB>6?^hlpl5R+K%L4lcuyS&D zL7ib_2vr9$t6V0#U{2_@J0qpE$ia63WDn8W<^T$UQIlI+SCR+)u6EO*H+|5KrP>l> z+c~92wQ=VYd1EpFD}tDmuA1k(Upq6g6clEQuETHVJ5&F%h>rX6v_)ge#bVY9OJC7* zJ4I!*klow>cfYXs-~myy`|Rop=kglYIkFOl?1Uf9d2_h|O7jJH4 zZvHXeE#dv2@Q>IsyzN?}*}S53$~(zp`R6+5D1a1X5<{yi47o%~x^&@FNW`FCMnE8_ z{2O7!g0s|IuWB2R9XkHx^}Oy?@&&Jmi`Ia}z$0oY^NhiAZip|@_GX`7j)1|S%rxWW z*>xp1^G^BY8n1Nty8FHl89yiXm6efxCF=2H(6eLvfz7BFiMX@FpY4mDJCy1&K?tc$UuDhC*1JqjUh0)Z?pby7w;L*&8G;296;&a}=j(&ZLxcnliR)4>{>)9wzL z9vyAWHt1E-j?fpHW1tNkGxU(5h59J?sU_;eP=p5o3OCP%df|H~Mp9Bz*C^3cZ?7vj zx&vIuHYh;+^@%vW^6=D&`W3K0oKGgS{NWe}-$HvTp{@!@4i=#Gfxw`L@t{aRW(=sW z(y;+5A#DtQ*0&uke0VxKc-H@6GSYECrbTJjbO8z6&<`4M}fK&f(^20xW5$5;N4L$&mJs}xB{&7S9=N2H(?3LS-zRZ`D z^`@g9UMErOQY|hsez#2q<>RIi>}8*S%hE~L4XM;k?wm~f4ZIM0)r%es6Vome&E8jk zIq3V}@9_t1pM+u*sMg^h60^J4EyoqB2y|Mx}OuPLncSGZ1gq*G?Q z-0;Tpt|GJphgqOK$sjAlpmk3np90bg%_aZV*a7&Q>g8va4@f<90=EB_7W0^_yd+xi&*=A=fsQM+~YO$;)5VAqL@eXTk2lB!h`e z50+u#Nm7<{XM%5OlQoo;TXexaF+qj4S6?iFc+Jz%G7ezD1(j_|DTwlTMtGfjx?ck7j`wV znd;&ScpPR)Ecm|>6QiP$kHh(e)cTZ|v1aK=3wWHpJi3pV7A^P$mfvLlgux(Fpe501 znN_Kzm(lx1#R|O-14&gh)dNWO*yxWL)ev7j7|O2z z3zF_i@0RoJAscLUDRPE9Nn=XFut-$EB5i`@$7Z*41gfIxo(nJ6Re*u@r0sR-q|ofE{d13+GwN^Spl-HHVCl~ zbscb3$Q}zOjxUFRI23 zKs*dW6Q#ahU%hgG)o6A*!uY5N=#I2S2$ffV4Fbe7$#_GmPBADyZEfSk%X~3|w_`^X zsP}jucN&3ewbCBiJ=YINRC8<9z`>774$XeJrX&8@vGGLwO9cwmQfs0H*{wtDxaOQ@ zHIfZ6J!;4HEVgg}@s`v&*r?VXziJKO!Ra;OFW~|ea%XA>DxBBRv>L|l>txK+kmlcF zOJSsJZq}s2+AwbH@87PUcEL5?=hV`*2K!mQXh4G$rh~xr%ohLi^Z)*UR0^~VxWBYdNwIJ3CP(Oh467Ns`7r}wOmsXy4tR* zWdo=KIiPK!aj|huOrJ^?s|H@f#@X9#tmE@c5D@<~2E?UyPPe!0ZtiatkF_;;3`a!M z2X;CgCzfes_q>4^zbvEHwBP*4Fb3m8FCByYTMS&6S{y~|GN-V<-mbNz?6;cqEC6y5*tI38 zMaP&?#>GZs%>uDkF^q9gJoX$ZNlCmPbKd52-WFS-Y@`BubMdDb-YwSiKocEV9%(5d|2v8Q-EMz}o`ii$taAne4jUx_4TWXi1x@S-T zcEWeLgHoMlUMGw+(#Rm2qIB`7kh9xu24#}1FWe#a&A_>x{1xkq(p-kUGqja@YXaV1k zyVf#s%oG4j@B*M(KO2LY+lznH@_0Yz?~&;5TIS>uMnz(X=7@`Z$oT4;8T%9Cd$qmO zTiLJnPEIJq>`;nUC4%V(X?kDDjBmi#BTIH;^Trt-o;#y%V1wlv#{vUu8kq0d-r3c= zJXWJgkXV^7U}!}k5I%VS(Z_C8AFUK+gG*milXERnq_i3oB_Z>#@P3U$1a-zEfa|c1ZS1BWom`zppN*yGf#)&d(7E;WiI&^gr|g#X`$XrzI4FFa zd4BW&Q7?{3m-zY4PI?|h*mCacB|5ubZJ-T!rxWkypc_4PrqR|k<00W};Q8dnp{qWx9I-Xs+Qg%4VQ2G@ z+%QFPBQQ3;wls#`g4+Jx0io6GBer$~iNC`>;!2rpN^dwrZUO0XouPpAHu)N_b;DHu z%Sw5BbeZGyZ)Pu~6M}kNKK|iw{60u0jtKtP4@zOdzGbZu9_fVdWm)R>7D#dBNIK#< zy~st^DXB4?o@>LFFy?2ax^w1ZpF+mOdP=+lT#X%9txcF9(W1oH;e{<g0t{sXsHO z-``%uXB?UY)D{{LUc?0cU7SJUI)x5>Z)#Z92d{(L6=`%;7t zuNDwxU|D5Fr0?_l3IB5b{eIUg$Y_XnaR1}W!vhx|Zf;^>9{XSaw?Y3wd#fe5))&{; zUI*we|EKF;-66`rn;5;2e(vk%|AXiKx9z_8?fP@S|LyVAFMsP!@ptu@AgI4soqq2f z5uZBkb~aYllM?Ay|I={!B))1#T1`B`azXu{%txX<3gLgckQR*{n&|QI(V#c&g!ylb zK?Dic+QCM$!U>T=u@`9#WcqM(dTJRZW^H2wUq^}xOiMQT_KEp<-tA>p2<^{Sa|K?Hp?hBN;Gf&7~Mp}ZcwJq(^WgRusdH8!)w&C!vI5zwhRr~v1?`TW|YbT z1JZ|3xy@TBWH$yrf%stbYrUVss(@+y2VDdW&(q#v&cg>?vdx=gg^I--iM#BI+W8aP zY&D&cP6}nu4;`o#pznsddUr{9ljEs%vG>F@n%)*XDtN!Px@JJDR?O;Z|3n%;n(iHu zT+;{gfV1eHuQcl3vCvqrW5YDRCnEAKHCyaJ2BuXq(3qO6$aw^BvLLry0;E-;w_D|B ztG%@I+CNy?e=}U4I;~euqleCC-MH-AjI3CwBmyEH@Vhg&ne?ftsl%KO3QV72>6OBwTVeBo?sz*PI1MBuZw_!FPYJ|hOz zJ2Z1A5pUsS_dBX+YB~QbC|4N%UT&;Rkqd0=DU5ORKQMrQ&C%c9#H!RgA%R=H`kwTy z(k)tBL<;0sO5KNIYtM%&qpX3a7RRH&B=<8fs7^94<(p`UmrT_57jy9Ml4@RX7AcGE zee4lW@4W`vvj^lMFvgX&Zg#nxMic(X@cyISW(&v!7M+e&9qBC90901@3(ETX`b^Q~ z6T>R^^S+i7dxVf@9>R?+N-IwNf7duM}D z-&~ofPGS!?d#;$YI_o4Ci|JF1Vr?E`M570vLiU{va0&m1jmNhII{C22!p^~?!cRoo zBWXM~2$=a}l0&_hU58yzzs3d4+S@N8LS9!yM8rOkX-DQS?e7JQcBn`xIkxt`12Hi% zoAH?m7JcNb=FR}7&bM?{VsS500}zc&C)=e|=gYd3zrqH>ZCJUU4DZrrAqDBw{$Z~VAWE9W~j5JP4_ zQVM6DbP@+S@kZpimAJXNIq``wEuCQJ?O(oXI|HusJcRB+V6;r(_BZ^W2G{8xGo`v^ zHils!_gmYIud!}IUTga023J<=J)uEMyj2EctxzYImo>uO7QmE*MNE9b%IWnjQz*E``}TnzL1843&L}d zZN~qfTn4Km?~O&*jb^L=>1sCFSCLei1~X7w%fG+#Uv2AFRloUmi|)_OPJi?KlV14x z-md_Jqqt$AI>>HQiPf*u-0T87I$BDrya!cZ-`E;!ypF@q^;YHaBsc82Ip(TA5=QMD z%N3;O?1=|?H~*EGMjKDsCvhE&2mEZmKl@vEpuGsQKaz>Nbytw+k(#jsx5cxky^!HV zG`n`ZeRt?f#d}$0Z2f22*yZh1zqAgB7 zgZppx2tvvag)=mBl}2oiGB4)7T!TTq7wC@)mmBfcHlYE)St#2ef-I5Kh;_f_4$?`W z%)tB^Jo1$Meu2zt4Wr&|hZ?Ab=4)Y=px%|KTlkh%p)e4hT1UVGva0vVmoS0ll?JHO zWQmQoe#zp7gJ`Z5Ka@IxH8-DO*5X@dbVcD-!L`M*H`QeCtM-*KuDnMnM!F)6A{+iq z=8uq`@+1a3y_~ANPT!ee(E}cyukSQ&E z<7u>l!+DP%;#Oys{FGdyStv(=MX!VoPgQi?53)$S?khXFat)@xO#ZGJ7V&Oi<(R@~ z@S$&gD$3)g5iE1@oJW;@_r`GF=@DtV$&cQjyJZbxjx_y0>{wYm`xyUGN}`|d|tAn z3D5j-IBU#r(ftCzh`x(1S`DXOSuN6L)aZQH8MnKZ$xyqijU0O~nkwvq#jx|f7dVy= zpUgPthZBj!MI^UhpBk8}H=t?EGIC~&$wBoc4BQ|5iuE&%Fw?(e4{Ghz`=nw z^Kg((-OshYV^tw`WtNuC?fFKjXs$CtYF|ORSjNSZu2gKrdf1fktfm| zp5q&P4W8}ziNA5$*G?MsO5tPmV{034t;Mpfd8}Kj>nUUJB{*;c&Z+B7WvZfcPE|<> z(Zo`7D zfWPbATfL9Gs7u#*lgWoN_^HT@ze{v`;cPME-7ClM^~_0p2n16l^yd>eQErFMr%|+O zC$>ZI7}Qb^&>_RO75aUWLbu;?MkDbJ;Ox69pqfricZ3S&>r0?N2KD2j!UA;FR$Mxj z>5B-O9mktfTunYVvEyrY_*yIJrc$U#jt1%S)5?Bh!=I6$GsD1KlksiB+h!x*x`+f9 z*a(uPuF9khHZ7sCh4emozq(sPPrF{^9-gGO(j=`q7isp=?3k+Ccl)W*mq*`}&hx)3 z1dVPdUNVq-73o~K#UZVFdwcJ4Ta70EjWgH%3SwmQT`D85K$+&ZlTwZbtI@fvdY}Tu z-T|4}?w7(1<`J{R!x4rZIlY043TjqY>pr8UtLDICi~XcxRYASI>Iqswmxj*<)IE>< zb=Hq(F6@A8ukpLQ{P1Vb79+enacNCHyCtF&u4VvQPx20k_8{B^TV1KOaWH)H^ITS+ zj$)j?H@Z&Kn2n;EbSg~G{c(Js$4w%!fTWy!sEG^ zz0de9j`4GdY!tPkvSuXGr?j29CoP+Yb~!uu_3^5LPeo))`h8q#opn~A$ZsF4;2|^E z`n1H0g$a`yv;^FU4ibjBc8U}R=sBav%$QRJJzvsMvEOWF3D&zDRq{T`CdjYtVqv9n z@ps-Tl}ZT~r*n^d?*CHPfT8+6gDK$%DZ)M}Qf<)J$xLnN>af|%&40`dx$A2~OlDt~ zT^%ykXdrCVjy#^_>#v+ujZ_>-Lyh3cLvsIlM>l^iEc15?@zh@ASjE+zkXlhya3P6o8UtuvEuwM&!-q%`om%cB0 zpI4!+v7?o*HlBdRF)2{u-s@?iMhIYHyL!+vIOoO~?X~I>Ps(k(*ZVJacDYUe-VPXj z`aa&NRB3RUvA;g*jYF)pG<|IpXWOe?Fi~D(p|ieO7^(dFYmI540TY#GjyZPDyBG){ z>cDZYwXB~q8`&1!-vG>rDD#=Ow4W+{d4UZP8h4=4A%RhriP&Q0=jZ;ffDJABwz(dz z7aD9!*G(D^Fyl8P+E32N$>!NLzMD4(cP)&IoMP(eOnr%-nvs-!2gnyqJh~aB8_N?_ z5}!vSO;Ta3hMR{DFEM8zB9I&;60|qlJ|RuXK(%YkFp3_G3Ilb;mgZ=-s;({72#3j< zU2;ZZi4PJ+Y8)o0G~`JZf+U_{3l(K`p9fuR;%sM>6;*JJRnCdXZR51UWN%V*VTwi<6aX`)%EO=M}#!w7|=1VS@k6&+9h95Q{Z+|M&rVM#;2g7HCo zy^?d51YaZ=k>NT2>X(zHQ4u>3Ek~6!mpom}R0G|2$*b0(v+T6Jl38nM7TPadpkn_( zOsgXn?1Shk-6!3_tN4d!{olR8lPd%htX*4Q|KjWGM1^J&<}!3nZxlhOuc0&j zBw$P@TjDcOo;*4>cKxQjtF<-aeXU7(T2}LfLj)j8Y z1#hfg=MpxMMF|Z(=8S@bR37H!d@Z~?U)l=ohbr6aVDZ3VBvf5&|2erTG*m5OwzVt+ zx~XKFajZjr=Ag#v>Ek^22TzWdL{UwTF3%SHPjz$}J`j$iIl9v6p4rL-A8Ld!?-~_f zTVFp^b2`wq#!_vpZJ8R(PBvR(Lu{ zJYDTb)ma!nqnX}K9KrJF{EcIbh6&9Ptg8O@GgiO$7@)$YcuYe*wj!u2&MsKB ze2F(XbcQcit294?vQ!^bTS+KXMm3o;jd`*PI;~MHIq(e6%6>%9kX2i30{BRZuT#Lub{B{cB<~lJ zZtrDp^(_Vje>au3=hBq)_uq%(QoNP%BSinaop-uyVd-RVw9j>*?Xh`biYUTY%LVK zclIpQhWTF~s)7q4hm67!&W1zAGneaR;^~p?J(Q83xE@y`cq$J_6$g2%-DYc)Ntc@Z zk^$Z=r?Po>9L89N=PX>9F!)R3y=<+;ojGsT0a9N`0$NQ)h_i~TuM63OR*V=~SX^qCDcy+Ii= z#q?d_n%4|tSKkpISyC+i%qwRm_{15D35D0jewD&Cnl+_xMSbFkpM=Ji>_KU;;o_GO z8YvZ=oZ0Na=62~#Rk+Ra zc1DVu>|r_vY2A@OafzC{{fs#=woY537RG8DnL7h~T_lwi1m`cJUcQeC>#SP~vV>zBA zprXY&F4RZcj%*+yS}*if)8}X^O_kcCwpZv;Z7A`AI_RHA<5svVFXmoC1LNEmv(ET8 z#i;A5HS5H>a$_#n{RMq{ogtaop-Hmpw5Tj9njcPgKjIuhxOfb{;fHTGgvDc4MC;EEgK*qb(dVcVx^%iby zi~FeV$>d4N=~K}P5w~EOUYvX2r7YTl`xYYd#+_FL^;r@Tr59$zrj^;?NvjFjV+l@F z+f@;@j^Z~SdtwW1i(PnP3yWg#X=L+# zB4y#{8Ewv_QawQ5xGp^(zd5U9fRM)0{pQA*;#IeL!OSZl!L>5{Xm)D4SXqnVvZAhT znrTU8gk$=&B)W%;zb{%OKVmjJvWddI%r2A(f4C9W!sLCGALj?+8G*}siAf8ewejBI z{Uv)wK^3Ult7DRnwl`Izo*k$eGw?^YwH~5&PKXX3hD9+(diVElX3X%$m<60@jB#UZ z2(g_G?#Og5)oz@H5oXypd9T2ic81URAwM#1HM`%x083wYrG^@{3Zod8uH8Pr_Na-~ zwhGtaE}fHq4g<*@rz4FbJ+)4Z1;=Fn2M+PyA6J>MoXLIE8HO&^EOo{Zivq9(YVJ}J zqhe=jtVBCDGcS#gu7heyk`mTx3Sh|`EZ0VK{AToEyc~qCM32h1ai<&hoU_tW$Q@jX zKvkT@L-SW~Bg;cqLeb$ierEe3)#@%|DC3{RPJZ{N0CY`4*hfE3NaT>`bhx`}i^=~T?|I93%E>>_udh>y zAu+ikPwQJ0)7*{rmDk9;JMQ3k$)B4fWKQRTFxpijuZVQck(^zA+4&fy_2C>Fqjoc% zN||=plnYQJnd=d})PPy|q;O<^FvGOFq%G<8aKq*Z1|e%LrHB35H*s!)Nw(khy$OlI z^Cvf?&lah$<%$%`WuVZD{#<*lFyF6Lj`vo%6unSUQMSBZDtjhszbLJ8li8aisZzS} z`C!)sOn7!TO~8?GYmC^C0;W4jh9OdgyyFyx&`t+NN6N?Ld^=08D{yzY$th>le5N>Q zug&3LM&>f&OMZmV-AfomoNnM>*?bZ?V+XTkq;C_L!>}2YCGLnEbN<_6|KYOgx3&Z5 z6`R+(^B?WD;4kEar%KZ8L#t&3J;Y)fOwE za|PNhe9W5lx{#wR`U2si{{hqfpO4M>kSHi91)6ni2(sIyX2o-Jbq-MR?+E=~cnPzM zj%=4KU$C1fABPnsuo_IY>tdG8WMaF!HNNOEt^J?DA8c05n6*ezG;&?7Dr6{eG|skW7}wG^8!EQzS3_rtT?oOG^dfemqws1wJIcW3ocX`b#__r31$ zaj9j(i^1`_4;9>}XRanaVSSucA-vq&nOcQR3VGkCHCx@`3khXAqxgl^d7|Bx1S#4%AWcM(xIXEa5175s;hO}Vsmm=;CMVwX<8qy4E2 z#Rr-K=k&iz3Ki!TH79E$v>Xl7lcuJOpC@bmNchP7=|9nx_G5~c*C7cMFPf-P`AWPv z`CP4W293gdFIQm52ZWI1l-m{Y&g&Ju$$@~I;vE&-n-D*i>HYZzuy^(C41cEZ>I*@y z5d2D7;mT0n!XxiH^gYkJ8(O)tJ9=f3G?OQ!i^)(9no=c5q z_UfQHYU6D@>B8>tP3{aifnE5zQnFx`k~%8MF_j@H+U-aQ!`QcRbY2RJb+#|cH>leP zn*()^*41`l4j7tz9+<-|huF{1B*7S?%l2h3hr3MeDxxwwz%2MXV4lq)gqqL z)+8{k7QOKJo8V+zE<-kvQ6SLS#$9~qe0KG*C9 z)qA+Raz5Wg8%SV^_$40F(cRq*QHiixsKGi~sO{aG%pK-39!U{~6se782qShykgq_T zI=^F$fj#ffdb#X(pL1Bu`%6TS3C~cFaobYaZVd$3Mq`;*Cv#fqEj2jIRvP1+Zx4r@ zZw+)(RtGxFHieULVk4u!6q@)gRc^f|vdh!yad+8%w$VFWPizl7SyCU-H!|7Kc6+f? zZo4I&!s|4m`P7Lc*tIoy&cHG^Q)i=3B^lYaKcMngHPk9Ts)=ns>?%oj7l)KO0QDzK1JV^9 z&Tk_@A|Bd*N4Mly^!->|ccdvenhku^8I&4ifjyIP8rVv8ds=LdU&hOlhx`F#+ewp) zEw|Z!!Rp@=4Qb(!t)?M)I+V<##}x#H*bXaQ0Spw!7Grk z*=kdEcXC0FB?x`W8=?wfc>ASHyE{7wl199_Z3J7ioCb=QOdCwi{b&t#TnV?Qq|+|zxN=R8#A%Z)5eDru(wR`uFki`+Lilj<*&uzG zlk|5d8}rDa&CBzqh@+J^`bNU{VnJuC$E0eU+=51u%9gm$m*<%G)PpXC26L_?e8vfjKX;Snofk8ApIa=j*GcTaQ0{tViwH)dmx{k2g;H<#24C{II-Ny-?7vD zUAKEtss>ikgMQ#}w4SJI6ddS)4V6%!_#!O?UiBAVeTQ()hW=%>8qMU3@(hAjvnzXn za`D|19~)OuF|gvwBYQifYGGl5T}%oMD&u3WPfxv3Y$tz9>5ojWp5h4Jdvl#La5kr1 zH#?oAabFKIwf+R>$|Q_5rL%+M)nZf-+lZ&8&2^3n1sg#1wuTyTv*?-!L9WAVoqmwb zT}Uy>Wv`jG=Pk9p#ag}HP7>(&_?QN+2|UB-elg6M_azgd!g$oucQz_D7kjGGSRV8B zt8c`gdM__8{g>rK2pGod?Tk12;mWRF-uwlL>!U&$N#*jPE0|PbKu|>_Ga7$LoRm6d zLcBs~iQ12)8fm>GMNl8((JYX)7rSMwA7Isou0l<`dmWJ>G^ib>|A?HS%y3JYS8QeZ}~g46DFBL zk9@97Z^)vom_1C9W8;^qM~>M_1+Rgn1r8}V3F>t73-eM(=(r7uD?e&<+;cCZFy%q3 zNE2vSan$7eOwR9r&c2`QL2KBg{#8#SIlf(>!s^-a@_B60Dgqac&pcJX;`)8Qehz)z zNNewpx!D>^?rV#M9~imeNf*&Mr>M}%mlIm8@5*2&t&ycB@A2Rua8>c6*RIYuF2S*p zqf4e5rz%X=3MD7&hh_o^Z+cnqP`75PLv}>w20={XLM@1^eXA!jnOhkzGumVb#vH8K zZ-~_sjkUG4OgT4s{q*anCc?Ut&Q+4jm-vy?;fi z`Xtb<0@xzt+qBLKfJ6?BGK?1f8XY`dIpi8tAKx1LO;az< z1Xq&`%=hY3eN05DC zd)UK+N9N27rXlD=C9K-rA>1i=y{jF>^+Bzjd-;7Rrkbs^kY{Os3uN_vtmKHZc#0xLH4#REXzgSJ0Uc!%p6IRJr1NEcN^0Om~jk;}0G`Z-qy;%nv%0_VEFwyUG zKBv#^6Koe6y^yFe{bZUXt7dc8C$R7y*yX7!3+6QJHjFe~ZOmD!`uuR^MKSmM=jgB> z$G!UpCbeDZe*i_IisbmwTI^&MQo&z2;1%V92eclXeILa@=+EN->1V|!B;)n2d<88@ z?9KX}(hJlRMq?}kv=%3#mL@JIvU3+G3&o>nzYc33EtF>jM$SIJppuT)91z#;5Y0DJ zE1gcL^V;DEAK2+Mn}0ejX&oN@l}|Yi;NUKvbJkbBu}|1Ll?JI+PX%iIf7ptx9QFn0 ziLvGFviRHU$WQ~=D9ARVYJ2v|tFn&V%_Y$xsXC>)rt523(vbr?R~s{n7h9=aa1#em zMf#K>saKmBVh3PentfE7*_+O=?5lBuT2{)iT8h>HPbGrX_2ydUN|y>@SFN1vjjDb_ zqZ3u@9qoS46w39)HBP5_R)&YM*v+Qe<|>UdsSApg%XEantQ|8yw(}Fm)zj2LMXQzU zCg+>|3N_|x2MpSOg|PXk&;xp_BNF9I&*V~oAH}WfYb8?tJI=}8&WZJ0!1V@>yyj*R zNjF9+9t@hR@%^4D2Mvy08rG=uH~9USkbi4=YX*R?CPYnMS0{3OgM)EYl#_>v|GkD!vM1J5gW!^f? zATDrHI%VQkY+E67@o;y75W@H+MnuMKpBNBdLGB$u3xHF@AdcOZU3HJ~0M?>Xd!~4S z4wm@l?oi}3*c!nXAVaM0!JxKBt2WA5YQy!hUt_JI#o9iX3&p#-RG9&`CqB!}_b$aAxHL8i4%#5u)*k{g z^MVGGq06ZxDr+&LJTkXCM>GeTFiiocuMpFgpoU>T0jHoP#r2Jv)H40a4>3nY9*ijh z-*Mx>l5IQzYNM(?U{0YB&%pzf@6ppH4Acx6~?lbB6)Q(_9Nm<7GV|IoTNwa;uB=c7d*DX^SpH zp0I=7FcNo<;4kv`PR3#vd>%Y`i$Tl!CJgdv83%!mp|HQ6gPA@$9qOR1E(u3O*?ly) zBmSem(7$tJZvVE@EAVRO$<1IWiqJ2IzPA>!S8w{>^z`f+-A<1renfc0_atfP%h7M2 zRT|tPEIMv$0FIn}D+r*G-D8R1s6rv;qMy9u94+yIWJ^1|1_Mzzh=$k?wDjz~1hRKa zkX}zpU=Vn^J*hEe`#%^K2~xjHC2kNn>3YL&5Vod#<8aO6E@+SI_|QCpL{jY~c9DLq z;T2n`fYWJI+-WR`-F}x{c~vD@op-a}R@sC3=@w^Szi+5@EKihhh~!D540At{aE9IP z>D4S&9JPi^Udm+4fl!eK3(3L^Q>_EPg8>sRwH)JI3$FzX`(*9|^n@mHK=^6I&Hf4d zFxvYeg3?IGn=b@=>6!jS*)ol4A+5N970IlnenJsq3Nx5-mk_=vXViUjbmgtXclupU z8#K=whot(RqH(_>#-E&PD!(OD$vk=(gle^O2bzm2NH=;8URXQY+s9goeMG%?y>U?j z3OpC&`C6+8B32`v>%+Mms;UySqPb$ZBsMe$6D(rzbLC21kO6;4q+F#Hu)NlpOZQ#8 zy%12VFvwX*<#iG{@|1XtNaaY~7&K8gQ(pi&5>ZmE86u;^9pa|Q@Q&|lR z4xP1)5O`Y7V%HM85NvQFX<#c1pS8hjgN=J^R9b+(=v-Zg74Ck?{mmZh!tbd=+fZ9< zyg(RL3WSf4JhWQs)?D8=6Ihn(Zv;iimxUez7oxQCk=Isc@kPY(Lb!moWx~!YLIEw| zemMxU$21eXf6BoCcT9$$@?_|axZ_}Lflkj|9hUfq2dYV)uK7vzoCf2@2w9_s2m?AWhfN^Qfw{i@Q&ovYGD)cj#q%LJ zPyCT46y$fO6G-UzAN)H4Fp{r4*`0ZHJCN8+#^BI!sZg(y=Cv21C?(Q(JTJE86d`l_ z7vBPR7BhW7ATOttmM6$)8TrI~I;8|QY>CNSqx58L4ZaZ5vCqfp`2J+U;7!CC|6k$` z`$%lBh*@sHUJiH$F<07@3q`91My6&$T)%SKyT^Q6uXp&*&aesYs|5*cd4Ivmw~f8h z;`y=FmU&n%X6%~CmfiL>Vda$bp32&!^64slx1ZcobNyOhq~e{|p=gMrcpbsS3`#vn zsnov7eD#`WVO3`MiH=1U2mx9!&kr}?75Saa3thJ*1Wm*n2LKyP2e3hSge2r5@WFmz z4bVIXaGRCubq|%Fv_35xz%Mx6-yM_=FGN%Od0`B-j8>Y1Z;j;TM|E;z-zgw?J>Jqn zieV=U^7Fznice>~f34k5-?=p zwn}OXB=`LBry9=4LHW`-m@%Bh4(y~(8eH0PhhA+qEz<0ICEKQi_=`@tsLg4`Bajdu zj!~)LC0iMf^{UXfyf5tr9{`#9tidXW*%WHAdbNlT0ClUZbYs*s*eA3yq)U=3E-pY}N@0djQc_|v{??>8061YNBwgeE%(HGD zSsvdpiF{$ulPsVHq|v{b$d=h%Zsv8pJ~TZjU2&b*-rO`;fYOrywPzRaNt1rhRCzV^ z=ag4635?{;L~~QijJS`D%JhAR9%a$8-{W=eQ8(xZwQ(%ncMoQ* zbY<^Od11EP-8x++SG`1*QLZAL3Hu{I?cZ6FZDsPj!+M~73JT}{{uUIU(th{QvTEf% z_RhIl;V3d!^8FpiDT$G8^ ze7cYNJ(%W84fs1FsiIp0aSA}GK2@qsp7dh)FAaH3Wbh0ijYkd38B7O;d>UycTb7=H!`z3y4)D9?g>=fYU@@QK$Hrb}@A zC-2znr5aso<>6HGbsZN&?LRcbejog|+dMR2q3Owq!zgzc{<;v!(_3(*LE#sxAW-8) zTQhj8TaB~HZF;*JXGb&F5L=>7m_KR!r|jA!s`8D9AcXWkm`&;%7JB_qwY0I;>7}6u zJh3<@E8@XZd*fey6CPU3j7VsWADt8mq!~g<#-8x=C3a^M^l-+9%Xg!nq~fuxy?$XEuT zx`4{FYSSK=nLh%onIf+1RGz}k<)l2(y@!@+nmq7=zq51y);IX1XRIkA73@!`n2V}G5E?D28H(wE&5`Oz1J>PA}onNPwriWi@ucL#XUE^=< z5CkV=)?e02Z_!BaTMn$MZ|s4iH{Y+(w2N4r;n6a;Z_w??>)Wr47_AUCe}5yf?E7k| zv=Yg4=a=U}X(XQmj~)GgmHZaDENsI(t!*8dd#n9z{W!Gs4;t`a@n$5eIaMofIBx$$ zS!RoLSMnX&xO3_}^J^S_Z=2frW`d4^m3dT~l^@57GFZK8+ z+Y^q*1Yh8_IRDx%o_=aGe-foJRO66qyMFic@K=k6BY7^byI0@s4|EXXEgCPern}@2 z>ZPJ{KkDB~ttx_w$+6sNWK%GWBgi>=>84Eq&i8M!>aHx>ieQS!R)r~|zD?%1(pIDK|#O+V8 z3h2nlfy8Sv^I5!S(ACoshCIS<+|_mzw=FA#wLI#Kxqll_|J^VHS`@!;=EYwMeGJ}R zV|?IuvlslML{YWbn4?iz^Fsml=jEv>2AQ$EPLL%HpxYVEh%~S6>%bgiUe9J&T1pm1C=Cy z8C&x1;=%6PIVRI>tAl9hHT(1aufIP5NR>sxI7O9FVOkXI)^3};4^+pcsi`R!e|nV6 zNrgD2;ag(7`fw^RCB&+Sb2J|es25gG(~5V;TUz>KtFc~<^tjhsT{Qwb-(nx#lkjqu zd@|~%s!0kdOzt01N^YfwK{qp-D@uUG4b{q$QV?H6K#V6Rw85nt;c%`A!gG~Bv+>N4 zNCi*}4>MzR)-8gsj8An z6kU}N3YBHjsT&Oz7|WKEW+uN*s8*6x?;S|>YgTLD*;!Ey(k8=iChTlsKxoJ$gVuWm z1fs=n$7)VS!VYM>3wbAu`+aCzTJol;@{j6_(p)8pHZN#>k2m%MDr@wKsT^a3^fdd! zo9TQuBN@KK53Of->tk)A`LTbYVizvJ=)R2%m-y(z+o3wm5+ z#)aUi>m`M%phuT1DzJ`8Gzh4<}I<$NsiY@~Q85ffTbQj$XE-=_B*wD#_0MZMl#Mr z0YulKh8cHpnM5Xn&G@gAfSKeYby`f$X=J(HM#o;({4^UwqC21>RLr^W$bY)l=oNKq zc0AKGX{2mE8K_sJaZ_y-id5NF%U2eCnSgmjW^s^ce15tNdM}?!F?BZe?a4?)fcL?+ z;~ScJ?OhG$o%2gVo{413ax!77ZhZosG5D5)}@`=}H(mQjX{E7oK0o znIIxkVsA1gGZXkVR(0yXUn->j+4#VN-%w*^Fb^O~pJGNErM>5|W~d4;AXkO$pD|7` zJUw7S^=Vo0*TeF@3v29HUw4?aAoea25Rqk~Pd#NQjg7U1#t_z#Bwj(+(4;W*iX5w- zaxC`6IdcmhkE;IX`n7 z`!y6s>S{C7N(cv0rU_JKNdht>I-p4-?13Wen#tqgMEz%xk`Qc2=uYU|h0bD|rK5zO zm6^4IjO~HEWj}%CqY{o{^lf=WZT~%_Vj!u#h#}wLKwWOo75v3+Q+`FY@wdC3;)0_y zNrR^~otmKP%x$yBMypO;ZV~CxA8G^UxS9k~+dNHMGJU^B$|Nk}+?O<^W{clO^`4<* z_+s7=7@gCPc7^G5phnm3L#domeI}-QS{y}n4JRr`9QEAU*P&gd2GMav)TK^?^E317 zT{5Vg{oXA{F2hYs?+XzdI-pAYf%$tfmy1YD=}>>sPTP#t|EgM76#{8Px>) zns58QxS#minV)@GfpWuklLzGk+DO*$G-8eIJaFZ5qsL@uEQ=h3YvQ_aVa!3Nk9K#UwnUnb^z>RkbFb+M8iX%=LV8+0iDz>YUu)@=R<5a_?b%to+ zeLOQLp!XK8jA7STPZn5t20ft=+IE*gsEnYltkt}_+xB!Qo}W0Gt_s^=N~|JY{2WUk z;If^>ox+vac;xI%;*jJKZikDs#036$aKS9NUSybEYo%%^pq&INPLGpzAkLiGt;c=t z!9!6xQ`S0@kg&ro{ek-I=jf;+=$8yJLA!QT)kHzx8tKtD=q_q!VJj?{#c4PXRvi-l1ffnp7xEKYw>=NQ+q0T@QcKjz? zh$?ny5)gdoc4QpaIi3|o)xTHk0wfaRO9bgznMj-N1`H4f8RNmEO7QU6aCClhp$@Y> z8QU__W&AKP6Sz8!qxaW2c&8^WS~vxP)D1iCN(pa$&Cl&8Pm_EZ=HO}1H0Cg6V@>8k z+54ZomhWmFt1($WVf``siG)yqY4*0`FsRE&U#>E$7QBtlm9`i}swRi!82n<4J0Cx4 z#PW_QGIrC1hyaBUd@;kBo{`|w@s#9@562L(7HHQLu!eMzQRS z>Og28@yM+~>RNj52xnpqi9y&VrCUva3&VkTa?zPajXLO}=64aid3D$zq0UHMJD?~p z)2vK(hin6a@`C>ml$#s=Dt^2!g*2BWadvxhZcBeZqs5gx;xT7Q0fDTKZ*)WNZZ$N7 z+?V~e$1`DHE{ST}kiHv-6Z$RLwYHgP7wHyhl&a=*Gj4HA2nR1X*ndmIOX{2D*n zZ3?lOLm85uN*fJ0VWO5nJ!d-(>87 zaivt5UE8+e4#dBndv@9Nv;ibXho!Buq-&WEYCBNN zGV6S%LWeHkpZzJvC-uQO`L)J}u99)@z+iM~(LXTg&RpZAT841 zAX$JghLl5zVk|>}bb*P<&E_u@-!Yt*n}TR7R{2SQ+nW2|!AO#8| zv{NfSBYyd1EMyUM;O=kK6-D5|I%zdYs6oFTMDEP;&+rlxUR8?s4)3^hM*jojQR+ww z!DK$vd^-MHRoJwLKzu(O{eCi8KyD-gz(s$x0m(KuoMhs>X`g&{DcV}kc28ruB+$h` z-d-CDwHt{qe_$irasF%0)pHM|*Qtg!#(xQm{wKWQ|MCYOr$96f)SL1BT@i!EvIyYt z*Z!GHEg9QY#6V*7j0~uXeo;I}{p6L^)}#2pTpn#91iNGU?`*f2g4^1JkOsc0?To(e z&F_Sj84lw*Yt_Hn8^r$rQX5$a0X*-Y?ouL2FYxI?p!;kh06ev}arGR?H|Wg4zbMzc zrnP_O8#D2wJ&BV)B}M1fZ8hzvB86gre}m+Pe}sn*+5RziheF z{^uG1xTc@iwE6Z^p3eLtvbMGr0Z^sLlck9p%T@AMoEdbVUoVng2RTaYbHFh10`VmKV$qr40Bh?D@}&jcz8 zqPB*|QqLIw|Gmx}em<6Qa5cD7Zq)7)(>IfG#Q7&n!N^vf=k_5Y>5csO1nQ}WuLX&+ z-t=Ps4+gaczz7wmW5H5&Q^0HKqSYIDvXE$p9$(;(8)ru05&yc*`3}h_>;ajRQQLXn zh~2czvJx8r#{^RZDn=deA5%4qR@@wt+W=1(-`Uxx2l>|adPRvc{Rec7R_vufo$330 z8pL{aKjqW2vyO&t*5NZ*?(<(}4+MV_DFEF-&=HFjEN7zqRN%yG3v<^^2KRjQ|fkR$}3wq+g_5ELZ+b^qF_*_+X7=V7& zPvdab=(~f4yQMTs#O*EbQ1T7$fYK8x@#1=T53Oh(Rn~U)KgZ}^=H#P_g9V&l#gR=%OV?S9f%YZ4TT}*08P8${uph^D8 z8uXQPo*i93{@_!ee|C{tG-9_+hZvuMp`$kIV`P96uKFQAkBT}sB-RC|vKO=c_Z$8w zc6rY!%lYNYAW|lHAgcota6pNylxj#mhrLSjm|HTCen#SN1q_D_+%fxqv=#q((fxD& z`TLCO%!*v{)c6R*$`aWCw#A`zE=}Ji99g5ot&3a(tflF`6iRYOD=kWR2_5jgK<&5I=9{$4OYhuT4^&k zuj^TOz9z4kYXKIL0BNNzKu=e0J#mu50ytx79OQ%a@C%bC*fS!WJ*7rNmO0 zsQa4M?AjUtBv&^N^$_}9i*)+YZNm_9H0DF+J~s6xL}^9L67$OS&~s5!XCEa{^Sd~w zGr*+)Fo50C-#`FO`Q)rDJWT_#V<6K7=;G__uc=%^e}-8c#Oni~ex3Dx9y{M3*^o>d z{199X{a*<<|G5$_i}`#rWtF-XE2~M=#7KK^* zSzGU{)KSb;r_C^C0+x0WesLK#Fh>ars1(Fwd`JKDCfQ}$!R48}HUm-jj3~c~(|&y> zJ0T(G&7A=(b)Qlm6*>V^2$S0r z%>n93TOB;eLf_gC;Uy}rHAGMhW~K4`ueiT|*2=5EWYo6%EKGo%sFrwCDvF1{VvFI& z$Z$b&TiytZ#r7c(4u{5}JDgUno@jPe=DGJ8U>24SrGYlWVK-l8n4pDzuFJ5%(*y#m zK8a+I7k-cP09R{(rE4Lu93h)AyT7r!NuQl0it|Z38guzl%~l;W44xkXZ0qdr{-evB zMHqD*69~_SdO6RmY#WbHMeXuCwit*Q$3&0Qns59$AZu@Z)s-#^?7!l_{*nCCR*3(D zQ1A_~t_ht_?aB6UI|^coH2DX8v=H-Rec|OLX!3@+hxjrbVl!>)_m0M!;?-79rACW1`3690hw*F4nEY)l?z!+2B&*o@jCtBj$J$~r5 zRBpEXa-m*5np=DPD-B$+{*1}2I%EUOLwwnfmqX^-lDSEvh1kAHmU~4w0Ec*X`{-mR z@L$wtleu40Kkc%+mc(W`a?CurEQ(M)ayqAe3~DU0+kXqjU*XVy$)JsYGGk>)5<&9P zIT<+x5dmR&8@AK@KRe%rd^kU?@}o8ENAy^Y01>IQI7i)8E3kGM$o8}&(6`g4GAm2- z;CROv+>!}aNmRJ)bnwRo9+7cK)>x$ykSf%f8@jLDBlh2?{-{}j_5!i0RNxs5RtYSz z7Cl3Y2T{vi*N%;6_ugh$8ovO#0X4Rsug;4Qh<52{psqAu;w)9C@|n}yMz_C@Z; z94kV#kXamw?8S?cVesk35290#``Or_G%BC{Ke;&ygHp3CXi06y~gshI7THO;h%>ubwazGh1eiUTW z^3s_D{xM82nb1v`40czM9!HTngnwuDT5uh9Skp&AQf7^85QV|GC~r?-5ZMslaLb z<&m!wcjdKEMbPWlq;fQsfF@g>e4`>)KGC{&fi`lH|NT$+`)huut~8DF6Sv{c|e#&xNY%6@2e2JKClf|CJHD5p*+#*P0&pUwb2D?c{`^wYUTd)`(ia zcea7P?OuBSlJ2Dw@P~ojE*0rz)T#B^vV^+p_Sz+y^5BHK(d~F-gdWu!?!UR+Pa(BL z<6S>JlpKgLo@U-SX9xepjr(XXbJ_Gn3 z6jU!zk@3df?AiQc{J(w(t5506p8o!z4CH&(R-`;$RB!L7Ho#b80?^|2NAv6ro(~P@ z(s3W~vp=UM0j&9_AUs+QlwNLBZy{8g)#KyOBO@bk>b~a@|Hr99vOr)VCH~#b`Kb%UYP9kI&@k@uA9elE(G%o_qKwbLUzeDk|`*DpWp?yI9w&eQ`kC zlQmne|E#zplMOg)*e`Y`b8*DOh~qw}l*j@Kn(V$P%0g$W<)%zPia!SEb#_nJI@!#o z-UUvGQvBNi00(o!jar6s3Zy4`hBy3*|MMg01bCx*J;Qt@bh6x>#jM}+#TNs>zqB;H(>bL+y}dA=Z*(f1xW=!H*4y^D z+#QeosM`To7)#&P2wi>C9Yh-Q)*pvl=T`cY2^c<@BoX^Hl#87(u3ZW4-py_0mK@dPZ5}a?knX?({`>I9_tc-MPkW?(J>fFI3Ebzw(}A z0>4(7Y{?+fyVrll?|mqe!CTYY^DXJ!N`rn}>8Q5>;+PdDv*})s`D|Xi4*Oz&4K16` z<@`qS%eNv;2gUgbXpVeB5(@N4nAzFZbie4f%6!X<#)~`X74zF-?fM@{)km_NmWwR6 z-QoB#^VOE0#nGM3L|;u#`=CG%gpn_{axfA~##UeGOVLzxiH8se5YLo~PJc*HoY}q& zR%3scjMlUHp*(Y(EhN9CweNGdQNOil+ zYT-3qJxh&y5Id_yb^2K(}3NeIvDIw?#efl8z9M?$kiMW51BJBOp zcLrXQp2OQJ*FATuQPDK14DF@lZMB-E z#m!a<@`He+$3WBF^bOOEg6{q0syPgkz2IrHOh5OKts81OD4N$;D0hg-)jU>?b5D%8 zUC!5ENM{Jm#Yeo({wSCaF_03~U2*GX|AR{Bz1AwnTLdPHadp^mBnNx(LS6S%?~gC) zcL5XB!Cw=es+E-lz1DE64<>l45_Wx}ERtgk8la)J6v2e5YG4c?rKxu!jokKvB|Rq1Fy5p)mX*@LK5=bR^G7bp3Ck}WX9oA%;c#@?a!3x1`{x*RPJe} zOKKG@a@%i+uB~ z!ayp-4Q>!xr_H7IgUk8mRYCvCB11CAZ>TwUuFNProBNP+5{sq(@Y&AjSxj+Fcj5i8 zl|@rRcW_U+?P1qmmB$Vl@nY+3qdrs{7%1hUEA4 zm_#g*U03-b0|?Y=?dh}$;55>-A|=~{bCE@HvYwYp`Jej-mFd8*8% z=@)Yy{Spr|52iiq)?91|(~v@w84S58#mGYnXgE{ugFSmvBY)nO+^yr%s0L+AYB0MCU? zK}}!1?q0pQLRt%~cI%)eIr@ZFWx1rD=o;VbA&yrj9zw9}-})8(#bb#^?dJzm;W8^3 z+7tJccFO)v-|RSGeSQc3bvraIL>ia+M`=|c#`b6yo7t5c8IA0#Y~A0bhNYbuV~|Z) zIb4FnBqWuF@9nEb4;Z7;JBINE!oc@s15dVJ8`? zTyee80rfYZgV4Naq*eR;Twz^wAL(O7Y@{lbwFyDCi7*0&TyT@mEJB}1zC%8@~;QGry&-KlS3@0aUG3SHRu+H_|&(S?Q z!YRXfwtTr-LHGMSq!PV-RW^bykS@Ff2SLvMrGw7+B3?1C%rU;aC%?;GC~d7`Qj<)j zWBO0S_^R%H&yK+G*NQaiay~bkZJAjW1wYvF2cv)HFE{8%j3>)|-N{U$zd7dZQVi+X zb@5|=-L*Ax3+iK>5y#ldee>`bskmji(rmR(nL<_>*%uvS{Valv^MRE68k9Wt)maAG z)wJWuVb`cT=(yi*#BYe(BGjT8@4K$`kph_A#E5}0j?bVz^;?*7z6GeTzj@gn&y>i8 ztn~Q}OMXt}9@Ut5yNj>#R11-2ee9pyiGAzxm3{7jp=>TJV*>IaoD*K7RJtV=-#Ac7LC0X@9ofqo9A#J-9-cb~ghJ zVAf9Y``0l8hl=C`G$^66O+<3ID9{g?_kYU!RXl{_CLD`8>t$ML@m)H+zIAd=OHC^& z>DOl$`3s$H-pfsP2|zyoZ`r0edabTlf4XGzV@_-TJswVZIIQDI5a_Aa5|5PIGgzzH zQxu1+uAo^&oK>k2C@%IDsZZnhQOWt7C5bTI7Wd)MHhPt4l&giHcUQ?%4IAW_dy}*c zJv$ZhWy{*C>%Uj~%lw;mc6)|-XZ^v*>W!v!;LkL_sibf#-yfXBC>(HDsfcJ1sRR=; z$R2QO;xEh3?z%dfN>me9Wwa(d$h1}j@gH^F7jtDnDz90|3~{FY=uIFcN*6Ra52085 z+{epqZ8BV0=~+7!RuzYxH1qK%vwX!rvk7#`ZnGfeS%#uq>3C&RBh@vfziE7RS{+}g zqEfubble{f?^rfRQAop?xj5Sib`o#{^2L zIh++YRCNXqtAShV?H+!~O31))CmSBz9JV|L)5qE=d*`Y=c&4@cAN%GXB(?J=+n5b- zBUjik2tmpx43WJamWK7Z zEtG6onXge2Fsv~v5ts!ShpmkqX(;5c2ax7tcX$hxz0t8mP;X&ha$fuGUPvpSmU3@~ zqr-f~XS^m@72n#ZQmi(r|3!T|=WDKPiK-g7w`TrV*2Y6KtmH$)F8vPv~8hl95CgS$chj6+f*72H&HRh z$O6q|&AV6ExMLmN+r_+ni2=P)l;XeV&BNr~m&2TF zX&1LELZTDpa36d_DFdXB{*)e$MG%jvyAZ`;l<7g3B7Fe%0V_ z3SJfDXI<}EO*dzQzSZeyq4yeUKX~0(m zRraVC1i%JjAZ@9e-K1qs33-GaNjyEPJA z8v+TzA;H}ncXti$?hrgk?y$H z!E^gIWQ;w=m^SDqmm$+qYlA@+c@e$TNtUNZcX2TPK9GD|tWuSzWZKvH&BoJZdD%GX z=_Gq@mQ1-;?M8IDTTb8Z4QdgfT7ibD&rhOi_B$r(+>xs5xGC^zYnE8;ujbcj5#y<8=9`l!mr~f8%o(E(H9hRmYNz^GdYF0GtqIBs-G|Tjic7W~E2Gg_Wd) zff=+bYgI&|`>uj*(J?=*HJeY7@q7_RCeM&o5}>KaWb#_RGC!a!s;`&Zm1E$Td@A3Np(sj_bvu4`V{vU zhf&|5ex*d*qt1r66;p(F+3p0NPA%!f{j&Ba(&F^E z0J(V5sVrEW1%OH>e*SeDC7g|!@QD*(|4ss(-BK1WkbfdLQ%bq>p@X#`!u7uj#qS@x z)D;;mW|99yoM#AR*nb*@IBawn+G4}oDFs@H4W(j>y!?=gf+f9m4}y02-h|Uxz-XvL z$1+eUER~I>5;+!3xG?ma&G@+B+RP``7VjhUFz%CC-OGcYLmXGWaWD3|6Zl3G$A}p& zhs^e-e@lFzFkUy{w0h8pZ66)I!C&p#%R<|kkXIy{<%T8DX-4jJP%V_|OZ)WuX4=tU zJ8&&hUd5+e+Tf!FN`6oqwL>Rl8;?j)O8GtSw^KyZ;m#jTdRf)K>1O4%_aK>&q%bb4 zi{?U72OaD|@_rE;gWBPF{xx0x+to6$^xIF#pX;)O1KiXK%-!>o9+)Ev)&s={X0_f; zOU2ZD>y&S);}7jKR{;QQy>r5jq0k!qb`cc?m-fs*cMmY551b;x0K0}vX1zGisXyP z%*;$losB`lFEO#y6Un&#o$0y6_RV~`e!)En@2Ag)v&DIJ{=2jb!p=Vy%b;zNPqgo= z{6gef^;(@dsD3%9)VY($gBOkT-Q3!eoHqG8wKO+w;n^7lCS#T$FtO!^ba300X!v*w zMm)>X$>P`Z%JdHs__wbhlt5N?)trDyq!^04_`ebyPo!8%S+U?(xN6HCaOYt@+z~op zx5Yk0i7l5)$mmR2Va$*DjLT-ZKR*9>x6wJl6C8}!$(7Bt;t_j{X3!q){=P2Rh?JvD z`||D|F1)X;rR|lG(1{mw!5(A_QEnlL5m&UxK9EB z>#MUrdTrq%g&heFz2boKZYR)^3B~YdAg}PmE1k8PHo|W zv4*1-y&P`B;X&?yiMsgyJB_smy&A0uiLj4YaG6d6Wn-HBtnV*Y_0dS?gCa{;w7o2E zH5~&L2LMP82NaN4nd`E;=fI~A=3!wO=Jnel7*cKZ^LClKLrbC~2IvuG7~drH*Ey34 zz8n8q(f-k?xaE}Q6x`S$N;UXok$%!0(N;3=nClq47mO{`>Tzbq zZWz*si4??u051~ZfIPehmbm94R)o&^cZaf3MbVKS^p_v>g$GqDfLlLwUB+tx@0oT%v5=TWxarKpa0y~;uhK!HdUBwFi9>__Or@ZKr3FDi*e_$V*%Ar6LJt%XB%MoP$=P zq+_zGw03B?do#(Ka7lLGm?AxtMy>QxZu}MxK?lK`gq4)5?W74)Jz2qfE;i1c`T%U@ zvYgX8#q%V>#A-~YXvX14^pIEO$ahwMKW=N1c^XO9V2G@LfebGKf&fx}6ph^H>V9-; zFLgs`dM(%4);VuW zr8PyBO_yI8t8=J{9gdlGbAy}K$V{t#)WTnxint=_h0yX z*!%mnKR2RM%_-Gigup*QgMVi5|NA3RR+_HFtPG0{dEWp0=YM0h{(9N|zdjE%Q6k5| z1 zS&Tb=yT1FsT^Iaru_4fSbT{47^8Ygn{XcCP{PWK=ft5&aOxn=?XRZG~7jG$0O)!4F zJtt6o3E$2qssF-EVH!8FJzDTa&X!N-cbgI{mF6p>KZo=62TRqWW zj9^7e%T!{VCybU0I^^Z|D0URHRv-(vs$fLE?T!`w1lWd`0BBw&^MDC&ZGK;9bNSZY z@WfOLl-^ka5dj5tkr;zSrbqi@MQg{a$!cYM_Nt4yyA=UF=p!%il5j2He%{Ef_3;AC zVSOaoy7`uT=Pha(H>nG%Pi5FqM5EhbzxpZZquT&WXTaIorZUFMp}&`dNT>*l0qAc> z2a*0To><8Z=Yk2Rjq_&bbsIlQm5Xk(-LDQaTKhh|hwN2RNgtfl^2nX(+y5e34%p0@ zD~Ziv;rnM4}8lBGbjq2kY)kf{F?q2NJ^z zVL|%D3l1}FN_$`i)}#7#{O`v?PhD-Hc$N6}oeJ!_N1?S^p3++s3WHQCRY$MvIJ}eQ zUU_6T$J8|Fn$2yfVqm@?8t5%=BRH7PTZoo=%K1c7AyUws#&XLa<=RHR{Y&y_yQ%(b z0H-nF;C>DW*dPdw2Ujyszt)_lVF#}nB`jc#syr|+u{@W=+wL&sMhZA^f6V ztvc_7?qj*qb3q_`j1=m&HZ zCRN(-ijJ-|;pT11us)dz2QCDOtHSalk6viy2SQw-yiAw=vjZ>5gF4@e65rsLAtc&V z>hak?&q@#yH)8yrkP7{Ux;|5U_*+y6G?6D)_&u3(0Oz&48xU!E|Dq#N19_)<)H4j| zerE}37Xu*21^LVyzOufRA~}#db$aX|cWH-Qp*%aP^37R-a1FF zIc}x{t%7s2lcTQ8nh^J7389z@?OxgoPj z{6|j+sN_y<>hlY4TKnHBJLh){Z}j})=gFG8trA64hKysU$*<)JIAZ10>nL7u4J(^# zlNRP_lc~4-oU#==%8h4|WZY#TYRuD4&nCt&Rt+nYEmE(yk7sZ&*!BE1Elvcjv^Hd? zijaIGpI=B#HNzmX-RS~cn7q2~qqTe}a)X`|A9kL-brAGV!|Dr{_(HtT@j~-1QqMV^ zyL!o@Se^^72E#5nMC+teY0V0pw_*?yS^ij1V9BnUWT!i%bAx11c!;c~-bl2O)%SXS z+NcUfVF>&sm2g>*EqJiyJEUwJnle1jaiiwzPui}dzkeOaqmion#xXf5b|4}*XlYnp zq0O-AGM%)F`Z86Vrx;qVxwIt5FUReRL$42FsW zfD_i1nf-$W$h2ZjtZtA;d7tisvef`Zp(i>@s~bj&M>B3 zTn;(j`=4Sxm?P0Mv4%-Q+cWs|eK7IP`(_1j@6^4V@TZS4qC;0NZB81{jQU;Zc6&0k zWgbXfH_9{0t0#;^N(Xwa0vrkrwaJQ|0{zj@;pl8V9mUPf1u2t-=~YUaatV-|$Fo_VNUW=v%7Wm6?4jrl;`b)RO6 zeLV{9VvmSEB<@#@Vb6v^z(H@xIgf_(;|Z_JE_|D{L_t;)8(QthzyN{>h7FAr^`BbX zdF|FE+q#F+N4Su8#*JAm(!GnzVOI;yGktkO4&iFkygxK&vz+bcMiDOvu$G(7YUaM> zgQKDCtZ(#OzD67POEgda+&#TM)}YWg3tqm>xY=t*Nf`eTDfN&TLveZ2@Hk&&%$RX# zzSFAP!9?B2G`vx|*(OA<--%)0K66pd=x}&jKXB-&5~oI3vM(4l=ztG#xLM>W;8ISZB`XFQ=! z^C{!^YR9_INYP8u(;ie9^TxWpg%rjoAXkjUtmQ8VzW?CJgBCg==#n=kxKy%6G*3O2 zzN*ezdJ4R{m8cJKQcKbg(azI-V))oPv7Xm~hF;<;v`_L&DD-slH6B4&XEH?vmwn!l zMTx@MRc2$B&%zMs>#QyIK=ykw3+&TE<1j z;W%*`>e(Deu7Dhoz2iok+`H>M?VXeJl>9P?E&(Zb88(<$U0v}rx~`wZoccxFd(dOi zSQ!&Z^mWW*LV;_FcD0HOwU301aH;vrVJsFRFp=&Ty#&uq&Q6u~ANsU*NY=RiYAa@S zHc+b6mTq-@dFoh+#pSpokDbcpct%H!L!phhh#hM;wGMH{$&(T6^KE_Hoag5LpwP`; zm9-yf8qMHc$RSvo=<4)k=46yu$iQ5#m8F3O-*(N5s~V$Sn~w|A^ci}+Jh5@=v=GLf zlw&^6EFV@L=RMn5+V5&>X-#{aRFNdX9xm;d&6GR*P6a7x<1;1xxEjj<48uZ8-^%43 zngbD^JEfO*!+Qz^U*C-BhOV0)I&$4nKK6TB&itZ3G%V)-)b6$9=c)Pm?E_2IN>f`J zKxqTi`{4h$(uf#kqxKn}aCJ+P3`O@@N}7^#g?viWQxTin7j`bEQW9NXg~a8g>eE=y znW-Cd~@8amM2TX^r^1b(0F2iL#1ZB^`d1WC1#tKWaGX&bB}4bVOk&c_INPS&y` z7vQ;l?jeI)PkwlFE`VzSE3b|p*jOTwpSB~_J{1WNkMIm2i-E*dL7-jh0x$;n{HsAU zcBfGX3Emy2vlo=Nk_dQ|@tj=gr-g|N_YfCQ7qQ?spP{~Pq zVuH7j3n$Wjut4i~DcVA!<>Kf@bKH*w+ z<4((shty93e!F4QNU1#xS2L8M1$8bGsPCB6c&W%qYB$TU(swMFo}H6zl&~t>`Ai1N zZ;UB~sjqL)5ohx$U=WA_d8%mQ? zu==7UdQ3ShZL%8H)o)8itOZG19+HjP?quS0dv%iB5z+Lc9O(ggZ!F7AWNWkK5&O;u zcSCU2`Cj*hVg-uTYSr3UhSQDpmf`O72Nh2V5;AW}s_Rkf;QP&Jq*iYVhT%CN<9Sg> z5U0R<>kch{N#yzrdTkfmU}@S%p&3-fS?rEUr4r||G9tf)FNYg)e<9Iq^IF>aeM7u@ zy3KO<_0Jzp6LQlD_ahv-lOWD0%gbCJ40Us&3#Cx`IM&o@52fyd>(aEeikkjL8{Yy& z0z=!kvy92###g;cdi1sn1@DaC%zwvIM4J*k{7E!Bwa9sAG%xdg$a%Y!A~hd-8K=OE zG283${=<@wr~C4I2*IL*fole&6aFmfDum;5y;I!w5V}r#A^SjpS?mc;$H`hLVSI;a zeJV6*dzc5h2EneTXijt5FAg|&+v9EmnDzO;Uz4A%jYjv;1-aUgd`(So&#@F5rWZNK zcPJ2FrTMxUjMEa;7e<-Tb0UuL8U)>Ql108tpfm3tbq#@h2mM?8Rm&5une9g1! z{k00;OA})E1iu+Gudh82%Iv1jdD=nNX15wjo^KGNrgDQ>FL)x1R#D&0Ef$q~5zN#< z9*KJSriG*K25(tbbR_niwU-TajAX1zg$OR$GgnGC?2Q&VqX$fu*v0y332SRFn@Tc- z0`ws+J5cd`^Ml`m9QAWC(R=&!Xt19xa@Gw5mn`~Rm^U0T<%zs-mHWRA^5zXOi`8Ad ziQQbtAde5#qn7b-1cSxJtT52UFVcS$;n%S2oiYrCmz78nt0@!U{|wCm((yj>4bKCJPY|5=x%C^14A zi0aRSQ7#tTTCRc35$lM_jlgh6n2{Vd3ZM8Dbt5S=%^`~pnoEovdmV9L38pk~oL}b~ zI1jfz=M+f{9(!k(#OR2FRj$d#lEMS@%M=Y)Ak=?Ugy>`&ej5QPs>&G-pHbFN;r<%R zm5TGAecyE6AuQs1ltN$10eA*T`Y?7gWLQ0!%4VxzX?78cg%@DLk*rKh1%TzWlE+$sCp+IAcQX&Y~6$B`yHR1)BQrUx6mvx8bIdl6gf7PtGs2N>a zQ^kFxun-AYP?uj-BkK31c!=AN3#w14mmiuB!!4N`3UcLjnIRw?F^=#~BE?ck8FLz(Q{Z!8#u_r}*xp94DG;b&XCo!(k~ zyU5;HEhykdDPO(ukj3ANgo$?A7L1yT>|Bjk7O%-O}0v?4n>$~KdHhvkCam z;t^Lv-{Raxk65~U*?FeI-&)hX$@^rCTyrsj>AyFju(0rD?DdGR>DZF5XL%nQ-Ocmb zC^T_z={o2rM|N&`=2tEx^V<1HkMc#c++-WYE0vfhyZF~A8E$@|hL*o+X>evcK? zd!uFGJt<6&H!78U{HN7fOtt7@tZV*>>FB+HDfB?U(2{{gSH{)f2WVNKO+6#P!Y{M! z{dC2hz`o)~nm8Qh`!Pvl z(?Wq$zVeKR>zY0L_D=rj@d3T3oZh8j=^h;b21$vAYK$d+U z0~^m5Zeg3CDm?g<1O<2Y3kOp_`!@gO9sldNi3N?I#WKaU8QGz zt9mK#GMDQ?H8dbF;{&hXIRebFAdJ%1LJ2c2@wg?wZt@B#FF8WEi>Q~XhDNPmen)Tg zExRC z4Z{UI@zFXvsu}LEpcut!+}$DZIlj(BKO=2nm;G?E!^33k-u)y@d~H3NE=XOz959Eb zEDOzO4Phr_vgGrIg|R=CZIdT*%R_R{@v7mY{D10IX9(;FK6$({3K z2IB=A*4m6ZJ@@^o80T10wP0NPk4^pG-^`{e+AsT%#-bRUk&~Y~x~`vkzDT2p{)zaj z0L0-_UGcvDY;4iG^!k2n4!nmGlUHLI=uvnJ`^Q4z<7n;2cSl#{b0bkVm-P^sY1w{ECN!T6 zwlpwLW(l6Ty~spbfMZ3@00)^`-f{EvWTJ(WoqC!>Vx@llyYccw7Le@-(#y-lQ^}ob z@U)@w8&i^r`KkHS=lZ90k*8$0#D>E(0qu7C*~8E0r>5ICI(M01dy|&V@qy@0{&kv; zA2=?(gxlv~E$v>{nPKg>XDEL|YyZIWUB1ttPA_Cg{zTy*jKCLE>?)StTt~}YJtR?A@RbG)<=etRLz~+g^D=On#2P~?6=u2RWem$k zOR11?)WMK9DYxxyB67X>BVWn&58(A-9{s@KmxZ}UJ;-Ig#sY;64#LzRF3oI5L&J5? zoHyf!XGVBUv%0MMs?a*N4t2~{6iVBcJQp8Y_` zYR(n}p^ejdN~=yfj9R5wr`+3ZnOK+BCT+`C7Ibb;557E#URU#byNh9s>Djeo9 zyK~kw8X7P%nxE!apBOn9tc`V(B-YE6EtPF8QmVjdL#gvrbTBuN9tWS-CVWfI2gHkT z)u%S83pvqk6yeeoFRVV7OLjb6t55KdIY%qwb__FSn*WyI3FSVOXE^zT%+#IxfvSJ6 z+;p`8b%z(GeBr>Kg7~_x_mh70ftM}D^fxFoZxP-H8Qvu_P0qw?V>eAS_7f4H{Fg^z zdxapEYS~qi!qZaU=vl*SjQ9>oJg?){Uvj`CvOJGQW^eK! zEmOa=>D(#J`eBVMsXS#-ZbpYynt-5}c!FfCmtdf9e-GADe(QgyV88qPpZFSF)i?hP zeaz}gIN;GyYS#1Rd%VA3wgANXR|N$guGgyK>kJE}Xp)#6DI|$PK{Sm_av25H-+u&W zQ7C|a`l2^%%PNDzYg6{Yg{Nxcb?39?1V4IJQzh7FfmkEZk??1hmZul99>b1@jayCO z>KRPcOR1~rVd$h)CI~4cLFlEy1;@F< zj7WIhypEMaKV`m>UpK#hE&9sRveRCG(lHz0A(72;z+N!6i~`_sW|7s_NF)? z&SzmDyJ7KzT!FIab0T=hMSHcCSXWbQ zuQ{~*T%<9$qirb&8&{~-EBv=KC>;the@d9kQVWY@FDW2u3Yc5 z5bolcat|tCilU)!B9=}B7D?F*$DLG0&IRT!@e(O?;`XT!nWfha#eazH*qYQkmoqti zU2gHCu2;!HWtsFkJ-H*{j#r!l1uXJAUPU@JPU$m~wn%mCF*G|KCnI)@FNrw+u z<%vjBWtgW=V#1KS(JRLjw+zFNO7vuwZ9tRQlJv-5!G~b0*;X3;8`J!cj%3_M@F#;J%?|?DYe9Ze;!@ zqkK+kFd_7%lSkVs^=S2HZCm#6?6Qf?AcK9H(KJ6PS zehZt1=an<{5Cu1{e=^zRRj6NmZca#)IePxMN<=4VK5fh;s1?>#N@A;UVLCJxm90_l zjXn&ASw*`htKcw~EdZUUfEbxNmL)?vxWbMg6xC=I~`wizLm8}S1Wf`4Y zvRR>JQ6|NYu^YtmN^dtCsc0T^mR;a`9y(C*EhY@5$E4E+8r>aYvdWvbXimdzo&mZWy}h2iJjX82#vt{? zJYs&tJo^+qM&6{=xjs$|9Y74mdX`H+lJ*8TF3VYGE4Ac1w|d}aCiT)fm6PimrXuyr zJjDrqHBZ}tAH4#;@)JfzYD!BSi)sP6+j>5v{TExSo=8D@#5eE+#mLp$TF@)T*m~W` zNTB&9oNGH*vO(DDplVMVhOP%-=s)8xQ(o8hTpwB2(fW5})`LkfunL}Pmo%Dw=N;>J zDD+~Zn1;hI@$=bC0U06fK1=tHleQ1Pr+8pCDkShZA;HR9c~JsVBYMQ)+D_agccl$_ zZ4~pqW%UlW12rsFAm`C)EU#~|H>>~3NbHrdGdeHF0#Fv^)srTOo`ap>(@;qprmJ>o zEICdW9D?yhI(?|GFJ9UQ*lvfx=Q|vmkq7*tS5e&%nVZ8=d^n%pscaq?LM z%FD+;as!j7&kE!B^|h8#I!I9+xZL3?eN7=%-pxP`SqH$h>_sj0Lqc2q_W>PrK#v@W zaM_#9N;VRbb*;yS3wGs^<$5Qi^Z9A^PN$eGZ?8)Lq{45;hx;ZD0KZdAyV$Fz$VfE0 zZl(@F3yr6NpzGZgYtvG{JzG7jheOuuX|^m;kNlJQqX6d9 zG}UbSN0*64+ZEq{s5>ZSS+unge5*eWu!-oEZh(AQik>_5vewultY5sx0$a?S)Hw9+ zz1P&hx_*Gq3qG_wf{GWO=Sgo|tCgz(zoRp|9zEEUE=*uHmN|CEuu zqv@i~Hho-jHb8y(m-_3i+sn{z@Ye{}A20U56e(}dknHf~%13)DguOCItLctkw?X=@ zH|J>8*(aD-kMB_Y!hXQjk4%U)UOkDETs8@KAh_b_@46Y(Wh60Gf~A<<(#59#xS9h^ z6hx^<5AhX=BLSat|LxQeJ)`~uPsfS?;FgPeuO`PMn)&JVoWx%`rN2oX5ko&? zf9*t&AX@rgDbN3hUgeKuMFjg3)*F+5t9&YvuG2k1wT)T6?FSgzj!vt#-Pa>^*Mm$V z$}q4rMrti2H>g!um?gc5Jx#pH)Ief;t{AZlCw#wgFaa&qrWzq%BizZIg* zw#em-^xA)^bl$!HwIf6R_0J9lW<_4C9g%fBUDOM}*sMFn57Ny|@4F)~NM8AF-J7Sj-UW%9 zV#59B5nbW1h2r(Wv7M96rFF#8cklT&cA*thnXZXYQ_KfE;-+uc#-GZJtNVu*HwY%m zg9l*0#Jf8C&rRi{r`N2Vf-q3yagfAl5$ew0-kYd0{o`goLlF7!>k4is7mv$?W{lHA+aFVniS4XV81piHB@{(`L_3Rx7bAdd7c`SeyPal z2dB|A4!FC5p%Jnodrz(1^sb;*yGH`mA0SXfbB%QKaanSvvew4N{rs~$ z4S^EyQY-iTA^HT);R;_iY8anL=QH(1Vr35R%PGN*_D#W$DwGAm03nR8#sS~L zEo3FAYH2{FH^$RwCn9iliIj7mWHvl;{Bgze{7G0r*%+9ux5^R;*KUB`r^4o!BjZ)4 zFK!=#wT|26oJ71-)^ZPQyyE-{P-y27@>i!ZWLgQPX<2sVh_&ya`G)qEm&e=MS-E-k zk9xe<6>$-d$*oz}ktH7!b>`^Om#vyB&&85yH#2OfmuE^C;bN8ioTy8crYcjxtT#6k z;6gB{KBS^|{g(efy#V4|lP9l4{2d(_EY$pW?<;Z+z!Am%^S7_FIA+9Bu%>?FA=4Vw zg$~mPae2_cuMG%kjydX{$M$$#Ddkfb?@1=|qfHucubuz1S_rCKYt!@^u5uHK7TI<# zJx_L^G*7K7^hl^x0j&<bc?SHBGy2HD)FNo9fx=W^M|LUmXMK$TVj==wE6M^Y5@3jMwL9mjM zuh_`7fofofKUIl{9fxET!M|vEPgl^T zXLEYRFVxKJZu+MC9=jpV21;ZIU$f1}M)r7kqKVt+3bB_vQ+Js3p--h^%ScuUX_u?3A7V*$Cw}gj zcxkMHsIso7)ZLJU-oNz#nvJp!5mUL~B%8x;uEQI*%1Ao(()Ri6j06J0Iow<<+hR;K zC!EODny12o&1*SwS|zC>AbRultMj6#+z*V|KrytIM6FQ~|NDHn8L<0Hl7a0|#fY3v zo0TY4%jTgJT!*8z-f$87tP_2u=yoG{CG@^VrG5KK$Sq+Teq37K{mB=!$(mH-s`ZO8 zS&F_lXeI{NLYfRmeBW10JbZBr-DvJ?R&*p%3zlfQgnCWfK<8FsIDckk2j4RV6|HF-pfjlASuD_`XQk z-BrDn52^sIT=%~UA33!hlDa?7C_JUTo{kZy)rFNjUnq66Pw0;SvAKh9B>F+@347x< zY{A255u4fG1~uSKdeF43Xv{WA0eBCYQdUEtNSl#Fle9TZY0Xcnp@Zz;d|6?3Y{tV~Fi2YJt z=l=B~`9yX0%lw26pf^5$}^mz@nfBI_G+n04|QF0F7xSRIW!j4W0~1(FQ0h| zzamaobBQp~9Nk_PrMffvTYRn8D?&B=AZ09Z!v)6g;ltTp_(Lh@J{Q;&rXZa&k6%2rV3vR&nJmkz>OxDs4rWbb1!n z3Bfkz4e3?l)`gsTF`o32@IoeaK4S%h)aTUlxxb1#zq~EBn*0GVZI`)Q%#96Gf08ON zH9H22tt{pcK#!^<_BwDwN9PXkaiZbFDcr`UY(ob~@!Xdr%F|(3aM$_}PWBCtKsH~i(LZyUPPF~s24~ieS@`=;GayMh$cRe9Emdr-&h@t@2hVMqFNVohjwp`|| zIAsN^OtwS2mrDIQ6+ic^`DCDH?4|zrO!_#Lrt$k_7#CVW4hQ4cGGk7rB>+d5nWUOt zdu*c3t&>&v^l~qct^gaiqG3Sd9g6Btcg{%Mtz06R3AH*By&^uQWruhTfAR| zd7Yj{1rTYcVpsRLCs`3;t6tp};pWWg7l}|f*>83EF!1~CD8^&y!0k%4u7TJ^2p%Io z@~QP{HEP$+eQSZ^oaDSO-XjLGMMqG*^(O1%+S~S?FH1t!&`rZ7*WY;UN!nRp_h3kF zp7l_^i>yR;3rh$lXt;k*c=PLzk?=G3W390FEb+g|HGfA$)jTQCDB;XZ9*y$Bay_w! zX})4w>ekP2*^ubW?1?f9%y@%!LB4~rv_`j+wnBGNfaOR$+jGe&?j$s_hDc_W1G6=E=vxaWCy})g&`5PH>Zbk!Z)N+er@xV|29E<6PENc0#Sd| zooSzh5iS^nnNK$BPw0yC=V3bcOaz+zK>ou71jQ zJlBmcI@`p|zcD8#M92ruds5nOXyTSyzO_4j!`o$7pA+Gk;wFXu#QHcEMCYxlP6-lX zv?zy&*CfA0NAlPDQx+Y?Nw@#IEc&%itrq$Xe!Z`H za}27L%nk>W)#@6JvH(E$@p_V=NOhN>!*!2G?=sqCtFQEZxxbx@>m7zkZ^EzwC#v~z zSa5we#WytZ7~?jMnA{3Wmi4xb#2+^0xO+h1?X!+dPaW}bOd37WaYkBpe(U+Scmo$Y zyJEEkmCEU;@sU!bO(Q(bF=fs;a6OgWgM*04`S;7E^0rsuI~_L0Up=oaC>FLNEg`n; z0ty@QtxNfbmBBMA@ce{M{X)kr61ivj044&deb0%d+{mKQh zB$(uGvN%X1b-6g6L`E=4TQB1eeddTya@bk9+mz->}(Jz!X3#Y1{&RELJ<;z*`%*v0oEI`#fZb%qWlUBQsIx~0hHe%Y5pW+%iri8 z8_ad9WZG8_&$XZ4+}ne!>0wFjr`rDzf@icje)b#3^|Lq;1Z&114; zWy!>yYTJoBDD_NZ>|$muhJtNs7O&dLwgIzk zV@ph2hfW#JpsVw2Gi5~1)?jA;d9H8jIUWPbQ_ai$HrwqLuoL6%hly&(DYq*!v_sOW zerGU+EfaEYJGm%1YyUa+Y8H+)0B1R)?eQxVn5_$vL&=`gPdS z9T_;pP4^R0;CHEK4ubhYsc7gqDP4WJdfG zL3LZ8S>V3?`jaQVGLFC&{k~%Hb@l2`D_cn~=efBkpV6g>Ae4cd3>=Oj0i_L3N`c`d zVbZj^M;SN>rG-dP41KaX2Kjtw^on57lQ-wiv{YZrv=neXB7`+>HPNNF`_P1u1V>+V z3{`IUR?Sz}PP;7!OliIx{SRR%C!h7;bWiHj2mLZ^975+?$ARycub$*gl9s}<)_+|g&VzL@iq5y_=) zOeeI&agHT1+Gm_kH0iWm0~j&ex!A>3M)J@n4Hlc$_5EFN$U#}&>1mkLtY7}*@H9bu>5ODw$A9syaW`gYMP)W=Ps z5YV{hWquV7(ha=zS{K29^_dyV9qIQ+iw$J$!`26axnICQ2O!5_6EmKeMFgtz^up>M z8v{3s-dTh{Evn!v90a6?;AhKCohFjG|5G`4f9;A$%3S#n_Lnn{m& zqIugelVUzVv@Uywo#PlEN?MZ96bJ{B>iK#2jc@UFx4pq|@Uj+MZnvASiRK^*&}2j7 zN^bh{xbB>?=SFY(f%3c~ZM?2co6k=%_xcFSz9C`muFu7OqjJ zdY1%Kkmg~LV4}9yHYLVUgwLI!s-eoAVu-2z_xTdOx8mx&dt|OS?(Iis^wbZ9-M6;WcaUic7gEF%_W` zEWO=DKHX|nq{Hf??~YIHZfuE_yjX3s{Z>BuUblU!jU5Tm=Uvuh3#z;MKE59K} z3tBU>KLiPfgj?b5E_3xgPo@st(`*;kpI@cijMVQ?NJlHcJL7>$cI6pP^WYs9^KD(d zO5craGJ!UeZX|S!YVEv-k7J{w(|N_=EkFu6_RCmI7Q73Uv%cv)&VPP1Yz@0U5s;w@ z$1Q+&e6x_nnm(w-w;j5eb9>yfBrzaSuKMozFxy;bek06s1L>RDZ@A%@-~5Dx@I?Oh z5C*+z!{?gN`eS$gEc*7}M4-Zf69^}+cdKs6)LhCe_v#G*g-G#dDs%)djlG*esm``9 z%qrC>eTCRFd^ojcBlc1iD_4_pt%bYzP{v})FCO-d3EU1KHLkldle%-HU-<1*xqi~7 zUBZE8@l%%2U2p!7ZguT$dm2mH*INh(3C#U}7`qCfxVCHy1VWGi!Cis{clQv2G!on` zSa1vO5FiN}+#7d?27)ybAP^+DHxQ)p#vA8%X8!ws{``6Kri!9&QGGAB?>%Rqz1Lo6 ztp-I?$JRwXRRoul6c>6ASY3#rxS!}>uX8#6ZVH-5b+as@BWEk$gzG}o=1GKExu}n- z65$#b1p-0^hl^*?$CS2w{YP6+(N56yqCE*br!!CNHWdgzm{DycgE~m6-MpTiYHp|U z=$vpSUA!JvfhsCPRc#Hm(*2=1Ki(R`Q*vV{sM~S(&>@S)A5LD|RAKjXZ$T#>Hg;N) z5c5Qy$c>2)-Vfq>a;Zx(D~GY|YP6udaReE#^IO5@?T^H@@fkjk3RF1ldWMRUfe2^J zu82e`@HF03`dPeS&I_5);31r=-)$D@c*fO)iWYEcm15Y!6j)w9;m(qx>6V6Su_Gs6 zK1vB4hoZLUpku~ernwXL zEoynTMo)TYvUsLhSsoV`msPt+a=CNegQ7WOnspo)=9k{HL*NC;KY%}+=lZa>1R}I- zt%e;l7q95W6oy%#>>43C5)B;uj@>U8OpF&M@+?Y9of2n60SK{UrUOW0!gIMN&aEYC z2>CbV@EuRtl`uT+UZC zBdT1zGT@VZhiT1S^O%B;_P6p+oz*Ydd=O9}J~ihCUMh^*f5KorxubCGc+W7*-Jzaz+F$FB>dTXzr($B>D&NYeAVS@!SDdpIP>f*^5px%jjmot!p_c z!X)6cp~omvFOeK?<+tfL|0kTdFwz<{pJ#w<r@T65dp z&sT&3c3PU0^Ru2)xjDPn-*()mstu(WKbW+iM%(G28~X*|;T#^8}~EM)bF z_;~rx-t7ZdEerfn0)fo;UJ3XEpN6*VqcrC!6$7+8{K@d&lz-syCz=J*FaHWy*8CRt zT9zCNda-&6yZs{h$MZvsm_L5rp^$JH=OMHRH7knGZO`H%jqon>OoRF_&tDhj-L;l& zSBakIBP3aN3b(iG63;ev_!ud3Bm;#v**93;aGEwT5OHMzB?I4=BqZ%+u-)aiu)49#tovW-x+==7}IxWkvk*JpRT)AuRB*=BR2%v2wJ`X>c9LAte`gz;A zN6%_pR6R}srNuKt&6-2gv;K}y82O{Ui;^Lon6RVIHs!T7_o%f}MgokjXz??R7|m9D z1(Ux{o6Zy~s3<2rp|9{)yR+S!_u8jLxh5dD`2ACy|7lzQ9lHJ7?+_KUjX6bztw@CG zhYDazwMjnUryVdMy@Ft3;`g0 zP&I{tC(~ASzi-U@i$ueLVy>FA(gn>QaMl{ z4xtS=&+p7}Y|{Sz6koz)H%V!mRj4GEVz+PX_PMH&8gkQB!20=tA3P@2WE3}`LNnnN zM*Gq%2}^y$;~@89EBXrQ&;UXIHi=!>1y*luiD>)&Xjd}ducg>rpQ4j=!O z|2if=Hs!uA%EolgjwMG8CgHbL#=~x1b%R4@!PolrXij4lk=7ZsRvq?5?XkQFdN>U0 z<2IdVXcC^6CB;ESN$q-=Z^OMLuMxso1@C8*K6_sG7S>@8TPV0FH;_x6VC7Is&bVtX zx+o_udSM6akatMH!>79#qJDs8walt~cr|<^_Mn#Fw*qg6X2EYFf~0b8#G&RG?~>7e z`#}7Y4beaO6di40zQ1bW!f`TWok!`?+pS6%A<>`kHd3%R?(Q?A|1D#PYy)YQln6!z@;_rSnA zaSNNF)$4CR*iyKv#y!#(R5e9u`} zhnwX!V#Q5pz41uwOYBYRLp6sqPQw{?(tAg#7Au9gWxvE`uH$X)z%YrvS1lJ(HPzgqho3;Z=X)1j`sI_^W90%d?8?h)S$SN#;??{2MgbVx zao1-9m5LVQ753=wofc6B)PQoY2k}NT`eX;`>B^dSZG)R8W6l0#@yn3V?RWfn>Vl9y($dm*A} z6*uw&#w8{d!;pxo*42-2#q9YCly1ZmnociEZB*-h2rJ5zUOoX)3#*i;VQmD@+eQbw{g-1j7}+KaxZzL3$mVTcTml(l*?Yr7y= zh7d%_t(blD5H>1$Sy7*ZMe>~u*%5GQiuvjR7Wi%EpbejjS(FtMms8?=2e3+0mZjo`Ucck$JM=x`WRE^hNk-0aAw{7vKGFgEj`rW~-YvyAQl#6=nzbk0OD7nA(WV^!@6CDcJXtLSpFN zbSTzlcTf_PL7D%^j;GxTdH#`}Y53LMcC`YObo_afcNRAz9;Hs(6zcJ^IT_R1kms z^l9_HqlP^DwBr>r3-ZV+3~;vj^DpDY?+=)y2a^qt3=f;kzt8&kSH^*VH!L#A2^JA* z4D@NCO#e?#=&B$zdwf(ewPp05-B5n;6@W5R_AeZ_{(F3Mu@Q2|#l{APPr9=HMKJhp zgC}o&l&zGM6fS#*SioNmvcI2FqfAR|ZEkK}$HT)@p%h$g^(mny&<6M{e!pCxi=LW#Q*mkI|V+|RS1DTt8J=(`B--r zN)|ecFxDjWW<$kHs=n7*LU%?uH+r|BE%CPWXe0*<^6~k#!IHe>5%Z7;WtS$#-N&bc z2;P&4cx%)oN5`jOk!?5Y%-$dV7wIvh~FseI|Nzucl>m~N<-|GGes_w|cMU4zJ$z-FT^c21ZLlGe8p4i82w|NV6Q`j$)iT%wAaNM45SOOK2r;D3R zB;-+T&=0#;@^W-cqL}oO_74d<1LiF8*Qo4|aVUlP&p9&L5VH{?7vcwp(Hk!`!hThq z=CB;8K3Qt_z#3Img?;NdVL})Z{_`MG2tXky8S&YP;?x9HqrT)?zumEjpVAR;@j?$Q zX+PJ=@i2J*tx5Y~TcET`!d?g#jN2Lz(#j#)%K`A*@uK|lY_0zklmU_lh?s&sbS+1> z!rcdO=vjGD$Rz65Ce6WS0}EFDH-n~?>=~ZBF|uMpJ&eYW&nN`YNEj)a2ZRcK)*9o! zQj9;JIj`70*@VAaE|5AQVvCuShtnI?dulg$xx74$la$TOxl^y#Q2C~?Hje~uxXv_Y^Uyp;?Du3v^Gqv+jT-?;55IuPZ{D z_ji<6Q7PS5?#zwyIXy)`@AjVgj^gv`8q7xfahTfw=;TWzGs zz?40`y{?-RJ{`q9R;-HsF7|VSt!-xy)SLGF@d5*+vz21U6lfY{!C@U#!yFp~4F5s8 zS=@DeC-#NMU1x4?eg|#F3G$4{PN3(rREc^294wy6(HEl{Cp9;&NYb>yR!pmUk&x{^ zBV?xl9Oj;JhFITK4Gau>N9dytZ7!`XY~tQKrJosM0RSyFgCPh5J&W#*Iz5E#gqTED&u;f*W^Ii2ePfCX?E5_PQ6-{58FIpFT9;qkj zg-@*ZKr^IO5v7rAzjGR8b069;fU9)mLr@$;{-I6CnKp5FHEKDs;i-?VR!Yk)+qcGy&OYN1*01^!e7dyegfQ~F?; zOG6)P#ERgja$TBT0X034uID-rTI0xJU^KoMvL>f=3H3?`t9`u&!r3qsJU1Bip~9-r zb^`D)=yZVy`w)VXM+ZN@Av$F$n8?z86M|48b=vkQBak7vdynyMd2qw<`17>Y*oHKF z_GzYMdHomTiLd#->pHIq*z74VT;gOZ9TLkgs5x2pShQM0U7s@b9y6k8WCs^r%aX8J zq5iewI7A#-3|-#%@%8=kVnW#-Cs~WnKqQCXf0vH7Tl=9FT2nzhIwfP@ zOa8WihLOI%TqP7Z(SAbGU0e1*8&t>J;sqxj1xg$zLE?b*Z|oE9VpR~j%?#{)19&7+ zpFZ(B_5j;_&rp0_cSGC=DTS=r7F#Gq|0D6sHr|g zGo!J^>n24Kc)xZJz8>KVY}H=@~BHl-0s8pid-3jkVHVgmPljd&|)w06{R6|No-# z58p+>7$sJfQvsGj=0vUUfw7UZ5*tzgARo>YR$5VrMh${H>4(ioT<-*4;HbV^s3W0P z*nvg`mUwKzs?XVeN7xo6MGK-V$E7^xjDR_G^@K7%+4_&Y^gVAtfdFIcd08c4-SXuZ zeG05|zOYdae!CTdnI=z{c(;+|PV$foAhz;FnhY{Ye6xb$oLK8N_VV_37t0@sG))?h z4XErV^5omQ4zej`5tuTkotrXw zPoQs3YW7!er)-^T-}By|W4C;~&ruTP{n2Xt<+u&D$OYWPsVxTgR}v&<){HZsQ4ny< zKA`J*;UfH1!y9*&xwqn`F9T0jjv&nh&+rKa8%s-{RO<6nRJ8_$!~z5Dg&l6mD7~Le z%HzJQVshj$zkls7?!qs~VfwKnQ>%{7$~Jr9&pBO_wbPvj&xim?|Vfo`$S^ zNG&^^nVETl`&7kS$!1|eM3{c#K6_xa=qz{$DK_?VLd|!=r{?CiWi>)ZmGzmug_fWp zU)9Q;JAcMUm$o-liwk13$`@+Z1zl~=S5vx)QeZoi$|G> zZ?*5V-BzQp@5_1U8T4+%RnX#848Mo@6+6@TRS{4E-5z#P9tY7_tcNt41T*HYT+0tr z6#iwcgGj%qklnEqA8ZQ#gU=#Ny{EX$M(=_&A7nSE{%yke#|%UoHlrpWkj41t9PS^c z<3AiCD&l(d+D5G)iuQl?ZvMFFKb#SdZN|sPC;FH7(oTnhh^pVWzdY3uY*I7+Qq@5} z(PHgy2b^-7n3y2nv=jW03_7Q1+cAa-57Wyyl;~#Fx8y$@@%lK!je9#7)0lAlxaU{S%H%r13?=*-e=ElpJLhf^NO+<#u zLYD*p=hcTuI-^b;r;j- zG`k4p2TA5wGwAhZi|5tEBxe8}w9?VxXC53WOStyYmuJtoEQd*WK7I{nBnR!qY9!mq zY-%mciO>pfCj>I3`j54LWHGrR{dz(olYGtLVzwxHck52X9$Whp5A{1tS}9`(maubf zfqvE%-oUJkdwdRf#vQf^AWIGVT^A`>p^Tr<$ccF!enSA;czAfh2**r%c6Q~J@Tw{f zBV*&Wot;mZM69`Q`##s!)+$(eec$|j-akB@|2=eW?Lwysh#s-@$w z7k88ASc9ArwA>RgmhH*RX_z6~pUh_7dOs@=_SIDs7AXT_xg8>mdSEt&T3Xf|s z-i*o)S) zHx#3s??C$jRr%88K}fDt>|kk$I&&+rDC}6|JAz`+XJplhEYKvUlxj!+0abnd{Z6dau-8+<bRj)ZeO_J(1mDF+xEs{^ldUM2^HZtHww;H8H}KQslePOsAX{h$&LIx@MB2 zqG!uV4)xu0Kc3xe74)|RK~p44cnjn97d{m2O!U(Gaatg?D12otHK-90ciZUsY7trI zd+dOkBUMtvGwdOg1L#P}22Hnrl2Xi-#2*qgj!+hEwP7Rv-k{TcV{>3yY-hajUGoW6 zN$m7#nX`u#b+Td=%5}uCzp8J+C^{7$%L8$BrDsiC1J|2;an`AK!8;a@^Psuo-#N&- z#!tinx@F-jov^6SAo5+kU*%|g(d)XE(@*qI3A=7T6+{mt-1Mj_r3xKh@?d@p+P=FH zZucXY;mjy#h?^z_K7AnCOS%0-9%Ddfc!;vS(urf>d4o^luljanYr5ao7neE?^l(d; zz8kseRfAq_+!4)wqO7D{9nO|IYBBS{{5!M$!BYbd8SJ;9zL^JNikhHI(=@)z8$l@( z2x0)n8-*a)CA~vKN)H3kU(>&KD&L_;T=Vju2kq~|l&4c;{$#=t=5|F93pctG>^5zG zw8OuwcYi8gBYg&3{f%{6H z?;SzVcHA_UO@V%iiKuY;+_jw$>?0<=rO`2zf0*=!YNAjjuG;izl9h~0Qo7P- zTkv@8aD>8j4CHwWHPEU5z?CKGl*TzPrS0iMGN_{~1JhrRFuS4w%c9lpDaOzIoUl2O zMxX~S(lcf+%yXCm9^bWDY!?I;yZ=DVjJ&a|Sc!xi27LT+-q;ggxFAS+^Y? z#k}8h1PkVSh5%c+R#|9)Jk3K~g-!bY(6rgyP@3&Ti}(+#R%Zo;7cUF>LH4aJYPS-n zNQy=-QVgiLhf>n7o3a0fWBthjyefy~pnu#vnj2tRuaoI@|7wVnrYF|p;a9C?RyisW zywc=}j|+{L^NRBOUD_ZiZ|fh)+h3EY+pk?7R+Mu@T*jeE2A3jsqF40v`~;1L{lT4;b=L8-#ZA+I zVGbtktkVoKIPOxZUJQ4~y@6q_k}u0j9E3Bir7fQi4HOsLD zsWJfr{l#r|hvem|iwh2eCSJ_JESc2XU&g)?FOcZqhikE@flr}R?x3co{xvu&F%w zq14NiB_x0jB7oTY(^CpV?-%PA`q%jeq8u1P)4H-GLWMArWt3w(7q!M5tAow;c%-5@ zD{faIRC5dSdmSN9t@2fDG`nXF@!$D<85!+Xw>g7={S0|Im(OpDSf^hpvW!$_eEo`6 z$zByNQ!l7(cCZ>7gRfUv`lu^hNe6+GwK2R4zzrX2kNjAR$Nz3}x~095*CZv&CkFPE z>(^FYuiQ$Dbd4xAHF;%fNoa=WK?z4tQq@^R%|42k5L+?%c|Z@BL`lwYgZ(Jg~jyDG>2nNI|( zx6vD=*Fe$sF}+x^I-Y8Dv|qz@kNP#~GY zxq;%EZPxg6+GCtX31wPGvp=2COp}#zXCj)NkJSc|bb$HXw>=$wUPFez3ydQJw6_}s z?|S89WuuCpju!yXKMXw_`(KT`iKc{0DQ9qF0=C8`_7=a|&;(}lQ(DxbZO zndj=f#<5aL{r1p*izJVZe%ofp5qK3qZe*0Dj`04zgL_wJiEv*G(0%;)vF9SY?Ram4 z?0Y(Wn`F)7w!l;3(W^@t?Z6`uSe%ha{f>y%o2!QVm3(C~vIl0C4!Ze1wpoX+F|ex4 zmNJ=o>nol+8ugJ(e(5%yk#m9FOaqGFS>L}bNRED?J@0wx`fHW*u4qVR)?wncO})t% z+MLgOEv8FSu6lg$_0?f+8WfiW(YM>sY~O4FKe7W^S)Ujd4!CBz>(5xt?^h? zx&mi^rHPtD@>**~Og>cr0y>sjR6j!LUWTf}}cVSKlx2sKH2pOtBM zoO0K@kYP6M@zHZeZrg?Ye0nn$WJ!UX3^lYI73d<^kqQwEoZ1-#nK)_f#p74|tOiR3 zQIiO=KEriJh#0&5CW*<&s%OoFQ>Kg}0waK93o9V(ll=vP(ZNsqFBeO zU|OhIi-o;lzOt&4M zk^=(gf1$Di&yfjY5gAF>A}hDcsAXUfO)^x-}1J54c%blsxZO zQ$@WiVnMd!uR^~HaHPz#p@;K+&iE9 z`p>W(CVH}MfT$lb<=2H$_SX;u;BGaPK31g4jIa*lc8{H~+du6zw&Nii`pF|$z+xTE z)#hWS0<}X7+J5^6L*R$-$Ot8U$Z4qoIGht&fnp{BrV8`Zz&AkKo-*km2~Y#apO?yL z)q>31oHRaP=i+=fN$hYeu*i40{@&?b^krHU`kDXcsK99DbzkCHGMx2 zqG4bVQ!^xG^Ch8gQRj}JMy)5;ygd;EVwA!=gHvw}s&VPzoi@IS3x<(7JD7Ch-D?Em ze|-~9#%}xlCQ24z@%`}gP>H%_ALNQ9~ ze7#k7v#h+*)*<$#f{0bJZ3Rk?BB()9@`X-gEzhO>wy6oPCzlB}cRNDBuo^Y1#u!Y8 zY|6EMti30gYYiGI)$AY_bbwfk2VU=geKi0ZFOo12aCX>-2C?Az;Zh3*qM7=gDT-Jy zcm8j!`cEs021BmWba_JGfiJG~?@kPM>ig<Dlt|H9L{}9Z)BL~ zErcFzsgcLv6WJ@2ZLjHpT!^bBmKt;sdz{L4mU#zm>Jz0;6Eg}DqYn4Bz7Qeyc#U^k z$*1zC3ArY9zCzo-aMF)TS-$BJQ4!UfpUW*yGP(~uc~13HGh2%ZdS7Kf-ZVQa9B5Wj zDW+^%p@hED=8lC(Uq67Vs3;lI<_;>Jy?Yp3UmWgJIP!v}2q2nOK$EzRFoF!v{cv{tFu0g~5+fA|}a3_w4!ij%Rba>}FJ2eup=su9|%7rra%U5QdC%giC2UhcZMyQ8Y; z-{yWQN<4N;Bl?{jE4lnd&deKo+ZU&$ckZ~bg?#zBHXniA=G{;SAIxpZBG%d0G9p2i zV+BExDsRi7qEs)sJ-p)fUuZOaZh8aXPp)zKJb;}L5O+$N@QXX?Cni>Wc-EIf!}&`{PL`=jt%_PDe6ovA9~_W?=RO zIg&katf>F);Ci8%Tu^tFPn`}+5u5c53Fx)nN^Oj#q6bLrd@?eb2YwE`Iw9H^yLo4d zDGMD=Upfoyp*ty)n)VMETVEdor#vJ^hsL$*+@R`~A=vY6gm{H;c$INMxuxKu`P8MU z%h{zOA%_-{jBa&51){>Sc;+plV4BM1#p!>gzeG(Y=6TR}C~635dJgT!n`gjIxegr~CA`Ya!xAN^ z0if0kdL6VrlHF#;gPmFb0#aucbQ^mP*yA4t?jm6~jVZW9-x!~hBC9@?8Tr`}iI7n*;TEZb=qz`*Ag-pSx z#dm_EbB;eQj7dedg7ny|WU(E)R-e_cR6p&cnHqCZcrXGv(FZ0w{F(+l^c31Q)R|A} z@cV-WInmJo5ip(MY{6&x;>#W@Dqk z*Yyfl=jF~Y$NIS11)I0KH5TKW>tpI=FZKQB@>JRdj-H~K6b8ouX~dV-18qz7 z%bN4x`jy7IX8!_kSGit>ZI-M;nQex<@~xjIQRADPnwcz9U7p&ahyiBR(J6jr)KrUqU=J?-&C>+J5@2Xhyhj8T%%Q0N6I& z+Cxf&q2h8qM9TclER$&NLp!{b_j`dA5Xvmk{H?#au3=VYy~GNO*>kz={V+#Z#clc* zsodQ}%D=U=As!*AU(st>uiD+=k_g_DM(S|sn`*+BFGVfBrD0sdR!ai#n4O)t*!4Tf z^UTd_8$be*#W99W^!7Tq>o*7@;Vx3ubzB-fWPsb^5ZIr4cpa0e-lWX3@grLkt{Vb% z6!Tom3|J0CYL8#7F$l}6 z^l_UlxGuYkoln63hz5{mw)%F8$j!v)b1#r2T~?7&I1D2Yu;ugHwH$KR8$kpzfbTF; z4o8yoJ=^BX6jqquoIfSr-dphhsTKM)Pp{6GV`-W1mu$odSYN*&3tIWW>m|1L< z<^6*D{nEqgHkp*)fkE}TJh-aF5TnyX|Oqz{S9HJ4(B&&mWGjwVyv`oUIR&iyil}%94>8^ z(7}O5Nzx}e-IF}@k&^vRoQx{UhQ5O{RLmg|n0)MHQQEW!qG0uhl%OjSbJ;CS zU$$CLE2W8^;woneVv$Xh$Tm6XMdVieMBj%#)fuX-p#LMU0tpm_p8%N>5WbQ<@J zny;H}I{5YPU`t;s04m_h<3!LCPhT6f7W5;rn{A)<|F>3!Xwe<9!jV*oXpI;UrQB=G zZHZv*g{C1C>%WzDoLEVm#^fu3HyjW@n8e0#I)RguQ*9#tK=~lL6&?Z zP}Thf5Eh$m^*6+ZKr7%9w?qhQkT9rxpdseC?Y_rEM ztO1@zf`3+R{=EiuP7Mxgz%qbu$N9?v)<>)MTom^{Hw@ssy@wUJJ{&r(j}Wt&J{8Dp z4Dm~${1}hq+Tu^O)QQYSqAb7P>nLR}=or}^4tV>kwzGdP=OP~-kMZJ7CH7lJKW;&$ z$M=CYtM0#^cvhRn+NW|!988e7jV|;*rt-z@`t(Q)GL&2Fp7H+Db<3?Fd4cndKE*=9 zJlK5l1TCUj?Jj+EnhLaW1r%?j5UMtrp+R|}lP7xNWMoUpC%eV|JxzoymS*m1s?Pi5 zfUs8i66?Yf!UX&Luhye~Koi^NQE2?n$lfvp->5(%K7RfjJoMS|&Q+@MKmbWo{^`%C z8od&X-^nZsJvd~nj9zrLqCULsf+kiuXP(=A$qYe1E82dYI-LG7=Er?#x|Z_$9~+;Z zTd}QGj8F0$l!S=lW#`NId?9Y}yJ5QK_Zurco?OyS69{~vN;7XGp0Y%OK;gJsti`)%CZzbkvV&c{UvTC}FT7 z6kJ{g#43pnFK{=wGhsXz*!f9t8c6ZUeE@gYVs~+5l~$SZp7FJ^>0`AT@gpg}32#nD zE-m8YY3ZO!(_gp(|J&+0@6>6yc%fZJT2qU<_UpL&Pc_ncxU8eCx$UQ#fzb~`*n+5^D7-F7g{qCmw*rE) zKB${|%3Lg=jd!XoMLv3b$BPT`Yo@Rk<8} zeF@7f)VZ3~**u^HY-@V^a-MdBL&LyJ(}M8JITSx^emzq0>ps!D={o?+$bprpa=pn) zJ266y)_u#tb=LZLC~8>0<4x@=pY`*;AG$K}4?cPd5lyVH%Fqo2{IQc;wabs7+c{i0 z^ISL>gjb#LZBO_pQwThFB6&c`u^=N?B zW!K-^NZ>D`V_a7XQP3Xt0-Vtv&Nzo`~o&P_)mG%8MpL!u;07^Igm2s#hKkml+2KtAB;Z?o$*Z+Dgyh1Ub5Ve*+G z_Nu7YH(iX`Y~t=4{l@`fXA9c;zmT&j6w-ew7oH?`ejNER2Y* z!k?KAN`8wx6Vc|>w-Nr~ZAVoe4(LBXL1e^!#HJ?_^nOew@}O&)?PH}*-AAbmmu;TM zM$2%eF0_qdQ`FyG2dnCHnRJ`}%kKVw#B-iR=RU@p%Z1Iq^X|x3y>wPI&-<%3OyGMK zIx~WzZTeO!Y>D|+Yz1$2F!BA*`N&vyQg^R%65M_cDU1}i#bS*Eh-{Zw(iG-L($L=Y73gddd4DL{- zO-qcR@)yrU4*dDT`1iOch*5en$oixmz^mv^!XB&SL9t#C*jxKHkl0w~s=Gt17Su57 z_lRQXRVD;n)$zz_xy9Od`>W}qi12RL>RnrN-UYeNX==F^0VAVu`5(wTT_SquBd%Y+ zp?9RJ(y)(z0=*aiALxB@zAD0EEsyR=y9*Arh#T3XF3ADhz15n$xRtW?Qbw|&a;s!) z19TRusv#x{gN!q>c*b4zh_hs*2j<5?=cFPhT|zQVpqgRzDj7g;{1Z%J8s})lkojW5 zt}&rXBN2Str(|)n%XrcdZK)qOiEKuspi9;bRizxK!BZHVQ& zEK(_@Y{5pjKwdEWlfPl;Fb~gozblC<1=oBr-SyqQVukr3OB;3o+=j*oxo7nwRvE4?(VD)%U9%$5jLxKKF6VWTi?u^z7E^HZZ<3Ntz} zM*_vw5829T!7{`+MWkrXkgbFeiMeUI(u8I^V&j~k5<*n%Ty|zkWmEDuUmozBf#wDaR=AF+{)x&CO27! z*3a6!eGEUHi+JtsUVB{>JSu|lN&JsmjLA<`s(SG*&%_R5_;+}}n>2}dh^*Tdpzk=P z-UXa@_OrH4|J2Rq?wH|*fY?GlevW*LKB~@F(9+qHE!-Kmjwx0Od2g zFC2fNUKf${nL1V*%VsbKl=@!kT*a!p*y{%+?1w7Kh?*UOOxaqKDX)`Mts#$|N@YA@ z2KlCbjSm9vLpbcm>-|iHOB9*Y%LPkW7*oN(_9qc$1q*Rr4>O>5i+lEzQni{hRK+#g z-BW&EPI>A#IkiooFhqje8X$mjAVqdL=k^v6550bnHy)0w)eJzoF;v7ZdctnJT*qPR z!D4=$FH2cB)3p05S^pn&`KEgvJH5edyAbV#CJ172DzOvXyswhgtiX~~rlYoEpJH^y zKntNl@O}a(7{>x`r}8@9&9z3M&aYpPUWf${B1kJ?nQbMUA=z`NUJMl?Mp;-XzPQ%- zs5&ai>pGkD#zmAMOkZZ`Jfoms-0xKi3kyqeG7gxNBkCOfx1B~uTUZXqozW9Md|lm4 z)miA}G+%!`0j5wo+qpfLm(=?w?G5e4;d7Hzdh@F~Z)G`>>DS|3C%PB+S$5 z9;^mkvhOFGOJ{dw=hd1s4{fY$pW0(w92z~;)B%0J{5dvABy<8733sm_I7E<(p1-=F z7HpK$9LNg*%8>HzIHymNBi~D=ovb5Knem^A(B2@;zFT@-?_ooRqiAi9$`p5rNhNtj zTmY>O4d17#M5zQ=Bh;NNuHTL)9lu+=vB$pk!C|kea#wILu&*May7Pj=6%oMI~kdcZU-{sRRagKB_f^-@lmlUSMA^+o1X|qYj^{nT9pyx0CqcukaWn(#xX9ARQ5&r6fL#&Th(9I^ zf4FAoV!Mo5Rk2=O_}$~38ivN|Q>f;fHvBr~zMu0a-ZeU^!dpK!E~zoQP3WBqyyeBT z0>t46oaDKW_+R^jcf`~MOL+Z}H9@LBn~$%IT)TkG_x$qDscJ>Q7aPq6lkZ)%@z!n}4~vLf_&lgl$ve z#a*q~(c&RRd8gHMG7fNYuaga{H@U1EKF3~FG!RbNcBR*kv zMJ;vfC5~qk=or&w59jeX$8bn6tJh_+@t=dSbSCLE8P zI$S)*HU?gO=onkM-tzENi|f}Xx`QEF)0Dd*Mu6PMq6-x*YS z&6(cRtsTjsu3*T8-=-fM*03r-EgvP1_;DUJ-O5{G6FW;qqMLugFCB~elKq2XKpn?j zNC_kBhu6AH#O^IHrHhk-vi>f1p(D#K_4ie$cJ+Szh>F+SzK;(FzMD9*)*9hK>KH`bEm)d;}B8-!wPN76N=J;1mP-%fPpIay09d!UkSZbgfUQkNMh z?!A}Q#xSG4E)uwIei2e*b2ufVIK8ubfQM+sJn{foP0B1+dtk0jt~8SmoP&LJFE&2* zONBw?ACXyXL=x3*mTZY$wmHCk&VY4~fBthp%-?m<;4wh;$z%IRyyN4;-!Rp&2TErPN#BkvY&X~2({Y5acX zSqBUWE!YaJ*7`t^RJ@vLer9jtc{Bb$@ zs|L2?AYo(Cv85)%1jdYM)uAA04P?>;mO6?9TUlIPzYahD8rB*2AP{#)UZM$3ZHg&8{b3UUC(lycKlI>ZUVEF z?~Gx$)2^>)J1@zqQ>c2TOP}q|pV>lut*Vx{-S1rq`}5by$An~f*;KPU_CqmAJw8rv zDk1Vy<=Xe(F1dNCjzNtjDougcY53|=ZNih9k5?qE%5B}A%@4eGE3JCJe(1t`!ra4- zbMVr#kcj)zlGUH)Ai9}=)gx?9Lx~fmJayjGi}H8sEIW;-;3{54A;i7ar^*XNZq=QB z)4LZsoL^>5_%QxfdVBAe|?G9qmlpP4Eohul(hZ$bI^DaG5K%iLu(eh_K zwX&CkYmA?chr;%M5;h({^of}_#f-YA?7f};+ayCAEqklhtoKNDC0csdIr#Qw8U$H; zlwOTnk^5M8;ozB(^$&=%$g7_06L==iH52{AE8I}~25c8si@i^VcQeQF&RM%pdd%3l z&ZlFFo`Y6bW%w%p2fut{2PuTrX3_sfP?z=s%kPx1nV&*!-4na6T-5)?SD$h3$gu z)cWLO@3&u~YFE>=&(er@ZQ{~Z)r2%Dcs}q{X@HIn2~=p>@b-?R?LHaa+6T2zR@7%D z@eI!Z#T&_^H7mBhzi$}4G4N)q?=cg%iOE!4DJ0)oiY*wu!A481_0iaEqknebBd>E| z26LXZ{4rY~_-kMdgC5E_Mn{fps^2AHV(z~>Gr6L@QpqY5YI zZe~8pj$beg#f1nRzj3q7iTj#N0Wu82->GO2C+IE4PPmFQz{C?g=sQvMJ7Z1PcNKvJ zcDsWP=u1ltYtZbw&w>xPJFGt|xYJ^&-QH7ZF+i|SRQBr88LF_i)}^f6oAV)Z{7^=F_e?QzTWB~%VNu!xNRupujtv$cK(ymYmJ3_wPt0r z@)7E`Gbh>yoJD}C`xZ*9Qw=&^T-a3J`<$l-xF#haAm(V)Z7%J7r2$c3FR*ZM*s$u( zy~xjkQN9F=YB5H@ubW>S8_fR>(0-=~VDm~pA|SdZ3)DX22Q0s}o+usF9|LwbYU;ti z*oAdb3UU2to})98H^v_Iu1MF=Bkhx$D{Fb?Rcc>MS#}H1(oxKxUC_xzCvQ`)Syb<7 zbAbalV8iLoiOIXe^?9q6n>=U8$FPf=yQ7nvLf@`9owNzI(Ptj{IFQ;SA+bykUK|P> zp7I`JOQwMxzr@4yD;Jl5EkJE*?xLC!u4TpSI;{VKEnRsqN^l$2P3W^dm)N@@rgnVLl(jhI2`V$yv%i1TYq11N?kh{Q`1)$NmVTYYJPt|?} ztzGSn&2MHDgOuj4m)$ls=!q(}oqsCb$_nAww75E?1@TCLTRd&A>Z=($8@R?cT79et zNoF|0NIUL@CAGm~S}%6s&5q$iMmAb@Ze{UK&f+=P%60`8@MOOC#T!#eK!y*Jm|B~A zIO1*0`ocSPZyzvaXR%6ii3;&Oj!+>hDuV4LzD|vEx2R~@EkXqQi~toG=lj18Gq#@k zs4d(W8vtr@#{xaR{2Edekh4CzEs3;3|+?t>py*TEh?)tI)ja>Ieb|$_7 zuV;!vds!nkhb{`SJr)=osF!NY30FQ;HDhUe=W@mPeeV^)!PQLZC%%^RD!V5~gLcX6 zb9ws3j18Y2IH>r^XX9j_cWP#vqwaflbD*X3X^>K$1}mjxB7~*H`wtU`Ti%Ifoke0^ zjHEmz=!99UfBwlgrf`|TD;~lz?j%Vb@)3(j{7p7~N(oO9|NCy~SC@Q;^NFvO$MSTv z<`WzI*;kWybNf3A#T<{4Ysj1Jvf~hvP%8D-50LN`DH`418#*ZtI6Iboi^BPFZd`zv z`zW=^`Mqqp*R1NAw{wT(1|`sMe~Z%5q4c1a$}J48OhH-7MApbx5jUk^3XdwqBNo~| zGb6H)%i9~w!M*D5PMte<&R`@zC9!;~^G7{Y)$+yxcV7l~lkUCJ^=MBaA9PD_>a%uQ zCcj%A1UJQ&wH3n$)0<1=y@(fB%y;YjnrhTJosS9@gMTWR~f+LQx5l`ZWX zaiA!0yp19Nc`upgr$z}6Q!;FgG~@-7epk8#e{^HnEkU}h;mCi^A-2Vz36?nJ{`>dD zZRzu$LEz_IQ|enN*8&=1y35c=a)S)@kUPV$gG1~Y=Tsf{rr&GJVwB&tj{B#U!n_s; zJ@IFk9IF#s6#|zQMlAz+3(gl zQT)KTj1lRMVS%0Jc^)>HEjRJS^LPHP)mrr-^N01*t*_B}*gUm%LASN6M*UA__{6d3 z$vAlrw-^PMZf)zlcg&?JspgmPySzBe!^ho^N^ZMU|b0Jxxj9QOBt6B zeTWKln{ugc6z@kC&s}Lb`MQL3r_FK?i#TPpuYx>Hga!; zCD(^aaKJSG$+-sWHKi}dB8?rj1@grew&j;jOvq6qX8AQm(IUnjL_Swp#ZVB6gr%yrvd7ZK`h$S4d< zPfs%rECzb-^TBR)mxxbzW5lkAD?gUrO_{WGe3-@Me%mOl?5wPm|7%Tak28;`aDMs% zp{3jZR*vyWkwIm&}M%2;dyX<3ijPV1djgfaXIBqdO;U?|SdQ$Bs%K)SG z9(*;Hx9G?zOr$tp_jExc0Ho0t#jSIG}8Y-3&;8y(yH^sfj}QTWN{cu%$jKdzqX!wOKza3m13I zznOnC+ggT~lF3y&BoxBe~wwgE1v{{a7;b4_R3F3Hjt;n(doaZ#!*!{$+y{9_x9x=7y zi2~TXki_5|B3G5y-T2;M@;$xgn4m?Sx@fvRKD`Yg3y|}05|j0vHs~= z52~^|;|GQ0+FyE=mG_0tsiBABZgqbd7NiI5!%b*!YwOt$J?)a+Dh0G3jtIJC<^IRk z=vVD{qk#K;t592f>{#Cz76l*E_yA&EEzdZMFgY;bV5~ zP?APSh1vLZokh~khpb%lQ~?dpn2PrFInJ(zC!S$qvn##aHehr8BDq{Iz4uz@UDhMD zUp!$yHKd1bDGXOy0( z8l#+#UWI>L%BkMZ)+mTE0}Z_xC*6FilpdjU%|EMknO*iKTzy)wM_b(gwRD z!hbkNSCL)sh6&uR2F=omG2NDl1a|e&`$wA*hz|!D4pZ+;S_iM;JXto#bTKN@e!*IW z5sl7!1kW8;Qx~#@8@j!j0(<}y6S)!n2z z($UI!*~m~t1Cq1yF%PI%+@U6zZLG0gVDCRdYs;bzILfxO$c%{%d)P6-ZR1JvLYTsR zU{)Z?>k2}cEv%QARu-w5G+RqU;bSCW^r#RV@y5mVt1c&uI~^7U;Tg%!1)mX_?@OPi z1wIHkJ!cQey}qESGeYKWf7i&*6DrU`kLw$CT$4=21i>m?__BY9Xc(|TF(xGQ1tmYQ zV1y-Z>pX9>!s(L5KU1fzMQoy;Y_@7{8@Gn$;?D22)d-7&!t<}O*2%ZUI~fFWkXX`h z4fE=lyt%%fpd=bxb?xRv*s|fEH?mzsK79@8Dd@HtTo{X#5_9w#o79NWO2|wxWC<^{gI4uhYS65W~~aeAymIk@QopyUScm*Jyt7DJ|Q?cFN%=6 zToyJM^d?W~X{nyA`w8Dh2mGX9cV^MeLYMY}O-Hw+z7Jz-9K5l=P21IgmG(4In;I0D zsup?iyJ!Us+83YZ{PFkzL__3}j_yADo1k2oRo#o^+@YrjGG1Zw_2M}%@3y*^bAl)8 z^o0kNv7zc+m((ljAD`$IxP2P>VVrDA2ja}gN^;nGtmgP*Y5^Kzj#ctE@-Ji@q<~i) zA~xc~b}%us>^bmT z4W@(`j5oR(JkyB@ zLMUdLV@cVfIZeGqY{{2b*HD!criWakUB6}QpHD)aMbEy}%0~IY)~|-_2z_`^9Wzh& zpE=&SfRgzr>ob7s3@uj5OzBTO_q)&KaHe!Mc z>#4*3i77>;fr*hy)AR*%Ey}!X=l+-g^O|uh^hZwsWMt>mB1&I6d9xf{Lj|6n2tTXoF=`x?BNe^GlkTOP|ukm9JDGlMlIj0HIZ{a?B6@;g-o3KuCsk6uR7v5V4`> zYIXY+A!~*PD;?-6FRpjCF4Q*V?zM2HOth&8p~R(aFn;Cs_mlUIr+C+oxF^k!kl?<` zkRBNnd?CZ|SZ+r5%N7*eama$}l!e@?L;DnbIxe9bRfXd!t}t>YkK4krIyVCfk5`&X zuRo`Lf!Yt#`lO#%eMzPYCPawaBRwrJyhYvx^^5*0^>@l;CJ2PFmp)jq>=zVxq#}28 z;|i|$jgmEg>7jpVxM5-Q!s6yJ!vz-J6o%#>JMam!i4@gT`v$PVbOg}0;A z08DijoL+4*H)}@JiB#NTg;$wed;>0Oka4@cb5kMAR79d-{I1UNy587#8zxo_gx`_M zq$H$kyBrFTu<$+~mc14okL)gHN(tae5p`GHE`zAVFvGL!tM}O_c zo&y_lhpHX?!f44Sp6~y@?fPr%VX&pa6%$o)n*Hxg+mAeXmM5kS@EtSw*Hl}t(wIYg zon-R=`0d0$pMEPfmNU#zhd*9wVtJ?i?aRNPsERT#@H8*v0D12IWWHu*XI8iVb31n( zkl%h84hQdugk<;Ii}pUg|Ft>)_c!c3%ow?wx8Wn(zu$bt{2Q^Vrc8=_F(P}_Pk;5| z-|phTk)q~1k}rOf^^5(%_Wlp$X*{8otQC7F9D1@;x8?Vd3RM)d3F}f?@s2m!PI)7W zLF4E-P3g%!54K?Pc7{l*Yuet<3BQm1nV4W1o@l$vD03gh%RCbt^*nC~7%+p}=;u$gY_|28x(xDGF}DQRnwhtN&1k6A6|CDvLR?_0>UXIXcc7UaLjxEB{#)Bm~);BU=Q<=SGRLf#&=1K*Q^zrXEBCBqY)VN#se zP+i@2_kM;RLRIT_xhV+mxx8n4jvT4|69aXBaQK#@X(H!ier7uCMM8wOXK=`$=4Hu8vFHTnPn6Dbi8;O9TGEGQ$My>`3iuB^ zY*mRm`Mo{;hgJ$Xv)|!r7W;`E|I<9^8A z37($UNB;TnTeJRgZ{+EpaqsvRPjs#VUjKx$X4Wej8X7wV5Bvm=I2?Fx&t%?W|NBq8 z(KY)ghd&nW++Mf!PB5HXRr-?~EZbX)&|6#1m$SL0k>&EX*i}`D*+dly5mUSP_a=U; zP|Z(SZJpU?mVFehnZF5Izm}b}{h?XEb2WP=U%R+c zJ=leQ!Ed+oA5WUgn(xS`h@tQPB*bDlY-l+U-I~(yVtc<8(wzAx>}&nUcYMp29lv1T zzxOZn7wr3i4f+N9zR_;gZD8NugMUH2zj^Vs4f+N3zT(F(sP{u8{|oAUB`JTJXw_d( z?^~w)f_guIt-qk&FQ~_|=h?3e-@hW%|AP$Q-|*%a)cczkU)$ziQ12^#{DOMFpx$4q z@++(NFOwYdE35Z|9L%q*-q-oeU&Y)XXZe3Yy??uyUr_Ham-Dq2vHU9Fe(iyN6>$F@ zEB~cRs=o@j-*wHu3bc6gxc`(CMqBf*zP;~bxX}N<`u091Q`d^C z%+uhZMN=M6w(W`veYH2TZK}`AqgSnb)l`_ywcxe=H3pp8Hg6Xckm|-vyaX^9W7>AdRn|=`^J+S@moD( zDOzv-sU6?IwX=GmvmPc_jQ4!{zRdiW>d@B1GbXCq0&%U;Z|8r~A)MA_TQ2GFa@S9~ z{Pr+fnBAK=o7sanHGk4$dr+NyCi4tSfK4<6p;VRZHDZr(1d~nV$n)iOm&R7JWJT-n zjs7*{F}#I~Ps=fj7h&KJfiU+DAGN~{UD@|O zs2mIZ5|qR)USHzOs)U-p+NDfiK+_8A@qu^|Vs)ERhp4M>r+b9y=xc?ZEX|~j#3l-6 zB{n7?V;+vQz%X(6sMZLZ-VC18g$eU-H1F<^f|0uNJ@}{~pW#jmgGGNdIRQ)^1dGBb zUj#eD1yhaoD=szD?3=b{Y)M{oUwk};+7?V{>k~-qEQe9&-KjWmaQS_~(ibo#;%GVV z-FF8uT~dfMyLsK3^`QZphbarL!Tz#HU2P@!D9(a7;#8y(p#Y;(u$^=oo}QpEHmhZB zye`|g9948tzD*16v6#(s9->*cX@aaZAYs5X46Jp4ciS$aP}YOheM396Fs{(1+#K^4 zl$5G=$gluvTKd>=m5nzX9mGarLzGM7qL&~nzkXuNrq@d8-XhU5#K2VE;skkw6 zmdZFqnGirPAEA+vPOIKUvlT$Z^QMMgq-(^2mI`&Xtd~%oIh9FnHf-7)6m?#Q&;xoF zvQ?8}N)@hD_iJjQ&+E$bbZ8@fskV>p~r;)JU zDCI9FID|;UA2zVTH0(;+ZlxT`x+~o)aX{-fSwOa?-Bvu1GR2PuIzw|B*WtzHv9PhaN8SwK>7 zW_Gk((Q(?RHsvAUGI^i(r(0o*ezV1Oyo7t~Vl%MCAbKm9qE<6l=rY()6)iXY9Et=K z)s14rfXJ)DEnw=Lq3im~(VCILB7>FGnv+vxG(E_dkc1vByrdLNelibXSfqri`b_^~ zeDzecDY>uH36?O2tbQ-u(e2_@-l?YO!fe)Mr+ArEO}m0g75XG5t}jH*Qetd;gdLwnkm`RGJiHks2EKyLsV+KRL9x;dMD zm5Vj56T-&}bB*vZ5_ARfwYPHdwZ`lU$d456KUT)8aprh=IX62Iyf5{wIOXjM z_8Rm(Y2|H4Y+Kw%a}p`e7kHu0q)Zv<88i*U#UxR&ejSv4fk4cML=hRNs)bxJO{xrr z?I6o(t*0CzJ8xPQ+a%)@P9aAr>Suhxf;3XBHQh(iBB;r3KZ&<{Ed|l@{JKCK22%5~ zpF(r@A1`czSrtUy!r(@K*ajNZ$eI9E2bEcxM(J76HjC+-D~n|Y3ozO&ETLt;LV?DH z0H{|m*wr6}5_3&tu^Ty7EUQCexTbM$xtchDAzz&$+XTRtMn%&bGRh<&TEHb58IJ}I zMf>CL<_m};Uz&1CX#4fW)n@4B;ang|sAq`4MTqJ39fG8CgfjWrrTualyc?@KJEOeo z?j|Z~NaODFQzpcio;&o8(s2wS5@UKX)CBr0VBBDFZ?T@xBLcFvQ8@}?u`8Uai0m>&WNO2y@ATPRl* zp42wmT2eyGHDL@rQD>Sya}^tDol{=0*Tyw| zcsez`5eTJksaq#?(YtAJBU}(+_oXy|bSo(1vvfPPUQkk*VUE2A+iPGwJ4v@MD_gB( zLIW-Ukp13`gp{Hd)z4Z$vS+dXm?V9;A0J%kt3)Ru@SJnh&57K41h+}hGIwxkhP$G~ zI@_3sR5b$>HD=vab|kb$B7)Y-$8HhSVgtOsBsWey_kK2o4i1oQGphqF3Z6*1wr*sb z@tJ%##@~!+WTJqL*lCyIX;9_mYtn-4lsHN{)1AnZN=U|f5DjT@h^FM20DH_X+3E|4 z=MeJqRBAk`?uV__XCA4#AIvAVG%ypi)oj5xQFwv&+Je#~z>-5d+9?tF1&)gXD7tv0 z9}{(jS{pNDZ)yx5y2{stlqD<|Blso}T?1>A>VwLmT!m%>d#ZY7p?3p4?KrOwqpu+M zFYfn}ql1{ww@(LeW(Uu4KFfWqrEhg*It5^Gg%biwX23J5e&P_t=DC5!q9_zvxfQD4G z3T;D7P7eG~zKMFpMG!GnO1P*#oLG-ls_U2pG@#dt zygkEPNY-mtqEoI-0@QU6rak~ebA0icWt+&L1!V2qTypulTUeCZ(mc zHqZkAM{Xd4smLN>MVOQ?N0zh4mGva?6V^?}8Vbl(2dhHgZm^*UIPpk-qml$xYLNj5 z!3p^zhEiG>A5LnxQap|%`6f(-GPx+-)_|{e2?8Au>;VMo2BxBimBEw`(uwe=u;oZQ z5A50S8Nh=oFMvd6U3>JtEgrAc?-X5Fm$J;IBg#L z0&uH<45Tm&AVbK$(glr1ZXo~yNk;nl(r*RJ#d@jy~R*Z8pv}Y1s{NQi2xWMLcXv|9oEkguulriRDuk>)?5qB(}zT5 zFrPm~xW?u1SrbZMuWAY;ZtAtKpJG_Tly|O9i1SmCMUo0!M7bbN>`^&`ybn@P)rF~U z?wx&N0r1kHdIOY94Dc?)_lR3rTaem~I?sdVk~__?N5ci3C;FvNAdR8*_S6(I$OPL7 zJX&M0(i*78HuSl`~(fs`Be4@L5qPdqb9+|H0z9B(#piuv%t#eVIi^_Fr2-dK z3$Q@ToCi(MyQULNT+ya}VxTcVzobse%gg3q-(&++HY~`GXlAm!*gzcb#HHEp8wFgE zLawE1s1z*_kj!}!$H`=E5gybgQU0IjV1nm15PyME>e#KAr2lXbvrJ@ z3i`!PJjUe-^v?+ss+N6_6`T&07Tu% zB&_>!yYA0HT?mmXGIbS!_3p9HOX#O5g|7sNS=ZHbl;bHG4t2rW-AxQuF;LH_A7KuC zb-1x9#b}7f@u*!Y$c0F_ez8isJ_|$Or}pxDg3uKyS~h;Suj!nZU<7{)*^urcnN9KI zUupnZRp;EJfFN3YaX=;+=$LYxM|4}EYanhh$YdX} zeWf^X%@G)0)TkpB-lhDaY8)a*ziS~*)0;_kd&@Q^0VzxlrjhamocnFh3)tt5(la*`Wc5ny&g4v+=7h4DuNm?n#)r28oH>hfi)eQGZ=%ADAyI%0O7j8RE~v!3%4*Icy-|^j;9FMO=m@;^t|l$ z+e@eVW-=?e}-LN#_+FLz++msH#kVp<06DTCevw8@tP zU7t}$gJMvL3Zs+zw-OR_(}^`U4dV=Ckl_H9LphJq%FA?Cmt%qh5kE0JLi^lse2 zSsZZqOAS1--&O%fTZeR-xu@7z_y#JYRutF>dcjz)A+ALgSeceblYxgySeFtE4jRXf zJT@Ocbu5>;5p4P@6*1muBG2|E;L0?lp56yq|IzhBOw17{o+e-KH#pEUeTWTpd(IxW znX4U`r!nnpj|RD9nMk!q02E~;bwU&8J)w%GQa$px0{3?ZuX-%lVGq5^RlWXjek)4t z_DY1`DqnPjJsX{G*I&#Sk%nCBvrRuW7y^1Ocq{pfM&1&%9>iSEAI2Z#gEUYWOa&bY zggdjPJoq#Ea4lQJ13u{I6Kr>HKd@Gcg+w5F2W`!#v^T} z;~$Va+K>uY=w{O}L>Ou985uhyZ$S^B`xh>(Y&ej+aqcNuW2BDKmv<{c?lSGfc&s>9 zMn!}ePyMpfp9XQnl)49}qt=QwZ6Lzy<@-H?kH1@5`9vhICjBVERPDez67W_h@%mUo zzj~(4NP$3U(zWVAPJcNI&N$Eqf#FgU&uJ4#ZGC&aUL~~$PwBCNp0>w~7z+0AOU1|9 zOQqyZ!G#ZEtC|$KAdC=hI0-PUo4WGM#U=N#626{Lo8cTJc=vcGAQEb%9Wfpy5G>UM zr(m&-LkksU7R7x&7@ST`vOzloIjA%-8^=*+(qk4IH=u|al6dlJd5>LBj{<%az4pTe z7##@BDojW+KMK+0GKOXwYfBbF`iH8)9G?AInqz+H_DZ!#3x7|>g&uXE&w$$Rb0}}P z7dpT6lqi(crS7`+P5dc)%Gz`FHRz%dSlqm<0AP_s|INcKe~17 zz{W6p#*`goxs@Byr$y@E!KV7V$SqX#*RH>}g9UndKd(^G5}7KH@@+XbzSJSV`gW1{ zOjk?jDxa@4a<2^6$W|h{{blwKVbg!A3PXpg`-DS~3OU&S*80Br@a$vKogF)veL$Bl z=>1t=ute?6z^%-5xHQx|XFIuWQ*+pQRg%zr+HsBNXQMWlg+sXMyidLhCCEQd`k zQJU;O1fD5Tb#H{x=8xKVd=?V)6XP3ew$kcjjj~x0v=Cl zs#>vl(S*l~2M3eLooF!W`g48}y7xOYm_{$A@hlOmnwqdGBJL Date: Tue, 26 Apr 2022 13:12:59 +0530 Subject: [PATCH 263/628] Apex build --- commons/database/constants.go | 13 +- .../oraclerestdataservice_controller.go | 309 +++++++----------- 2 files changed, 129 insertions(+), 193 deletions(-) diff --git a/commons/database/constants.go b/commons/database/constants.go index c83a6329..b2193dfe 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -403,13 +403,16 @@ const ChownApex string = " chown oracle:oinstall /opt/oracle/oradata/${ORACLE_SI const InstallApex string = "if [ -f /opt/oracle/oradata/${ORACLE_SID^^}/apex/apexins.sql ]; then ( while true; do sleep 60; echo \"Installing Apex...\" ; done ) & " + " cd /opt/oracle/oradata/${ORACLE_SID^^}/apex && echo -e \"@apexins.sql SYSAUX SYSAUX TEMP /i/\" | %[1]s && kill -9 $!; else echo \"Apex Folder doesn't exist\" ; fi ;" -const InstallApexRemote string = "if [ -e ${ORDS_HOME}/config/apex/apexins.sql ]; then ( while true; do sleep 60; echo \"Installing Apex...\" ; done ) & " + - " cd ${ORDS_HOME}/config/apex/ && echo -e \"@apexins.sql SYSAUX SYSAUX TEMP /i/\" | %[1]s && kill -9 $!; else echo \"Apex Folder doesn't exist\" ; fi ;" +//const InstallApexRemote string = "if [ -e ${ORDS_HOME}/config/apex/apexins.sql ]; then ( while true; do sleep 60; echo \"Installing Apex...\" ; done ) & " + +// " cd ${ORDS_HOME}/config/apex/ && echo -e \"@apexins.sql SYSAUX SYSAUX TEMP /i/\" | %[1]s && kill -9 $!; else echo \"Apex Folder doesn't exist\" ; fi ;" -const InstallApexRemoteA string = "if [ -e ${ORDS_HOME}/config/apex/apxsilentins.sql ]; then ( while true; do sleep 60; echo \"Installing Apex...\" ; done ) & " + - " cd ${ORDS_HOME}/config/apex/ && echo -e \"@apxsilentins.sql SYSAUX SYSAUX TEMP /i/ %[2]s %[2]s %[2]s %[2]s\" | %[1]s && kill -9 $!; else echo \"Apex Folder doesn't exist\" ; fi ;" +// const InstallApexRemoteA string = "if [ -e ${ORDS_HOME}/config/apex/apxsilentins.sql ]; then ( while true; do sleep 60; echo \"Installing Apex...\" ; done ) & " + +// " cd ${ORDS_HOME}/config/apex/ && echo -e \"@apxsilentins.sql SYSAUX SYSAUX TEMP /i/ %[2]s %[2]s %[2]s %[2]s\" | %[1]s && kill -9 $!; else echo \"Apex Folder doesn't exist\" ; fi ;" + +const InstallApexRemote string = "if [ -e ${ORDS_HOME}/config/apex/apxsilentins.sql ]; then cd ${ORDS_HOME}/config/apex/ && echo -e \"@apxsilentins.sql SYSAUX SYSAUX TEMP /i/ %[2]s %[2]s %[2]s %[2]s\" | %[1]s; else echo \"Apex Folder doesn't exist\" ; fi ;" + +const InstallApexInContainer string = "cd ${ORDS_HOME}/config/apex/ && echo -e \"@apxsilentins.sql SYSAUX SYSAUX TEMP /i/ ${ORD_PWD} ${ORD_PWD} ${ORD_PWD} ${ORD_PWD}\" | sqlplus -s sys/${ORACLE_PWD}@${ORACLE_HOST}:${ORACLE_PORT}/${ORACLE_SERVICE} as sysdba;" -const InstallApexRemoteB string = "if [ -e ${ORDS_HOME}/config/apex/apxsilentins.sql ]; then cd ${ORDS_HOME}/config/apex/ && echo -e \"@apxsilentins.sql SYSAUX SYSAUX TEMP /i/ %[2]s %[2]s %[2]s %[2]s\" | %[1]s; else echo \"Apex Folder doesn't exist\" ; fi ;" const IsApexInstalled string = "select 'APEXVERSION:'||version as version FROM DBA_REGISTRY WHERE COMP_ID='APEX';" diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 753cd357..2999265f 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -625,6 +625,107 @@ func (r *OracleRestDataServiceReconciler) instantiatePodSpec(m *dbapi.OracleRest return pod } +//############################################################################# +// Instantiate POD spec from OracleRestDataService spec +//############################################################################# +func (r *OracleRestDataServiceReconciler) instantiateApexPodSpec(m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase) *corev1.Pod { + + pod := &corev1.Pod{ + TypeMeta: metav1.TypeMeta{ + Kind: "Pod", + }, + ObjectMeta: metav1.ObjectMeta{ + Name: m.Name + "-apex-" + dbcommons.GenerateRandomString(5), + Namespace: m.Namespace, + Labels: map[string]string{ + "app": m.Name+ "-apex", + "version": m.Spec.Image.Version, + }, + }, + Spec: corev1.PodSpec{ + RestartPolicy: "OnFailure", + Containers: []corev1.Container{{ + Name: m.Name, + Image: m.Spec.Image.PullFrom, + Command: []string{"/bin/sh", "-c", dbcommons.InstallApexInContainer}, + Env: []corev1.EnvVar{ + { + Name: "ORACLE_HOST", + Value: n.Name, + }, + { + Name: "ORACLE_PORT", + Value: "1521", + }, + { + Name: "ORACLE_SERVICE", + Value: func() string { + if m.Spec.OracleService != "" { + return m.Spec.OracleService + } + return n.Spec.Sid + }(), + }, + { + Name: "ORDS_PWD", + ValueFrom: &corev1.EnvVarSource{ + SecretKeyRef: &corev1.SecretKeySelector{ + LocalObjectReference: corev1.LocalObjectReference{ + Name: m.Spec.OrdsPassword.SecretName, + }, + Key: m.Spec.OrdsPassword.SecretKey, + }, + }, + }, + { + Name: "ORACLE_PWD", + ValueFrom: &corev1.EnvVarSource{ + SecretKeyRef: &corev1.SecretKeySelector{ + LocalObjectReference: corev1.LocalObjectReference{ + Name: m.Spec.AdminPassword.SecretName, + }, + Key: m.Spec.AdminPassword.SecretKey, + }, + }, + }, + }}, + }, + + TerminationGracePeriodSeconds: func() *int64 { i := int64(30); return &i }(), + + NodeSelector: func() map[string]string { + ns := make(map[string]string) + if len(m.Spec.NodeSelector) != 0 { + for key, value := range m.Spec.NodeSelector { + ns[key] = value + } + } + return ns + }(), + ServiceAccountName: func() string { + if m.Spec.ServiceAccountName != "" { + return m.Spec.ServiceAccountName + } + return "default" + }(), + SecurityContext: &corev1.PodSecurityContext{ + RunAsUser: func() *int64 { i := int64(dbcommons.ORACLE_UID); return &i }(), + RunAsGroup: func() *int64 { i := int64(dbcommons.DBA_GUID); return &i }(), + }, + + ImagePullSecrets: []corev1.LocalObjectReference{ + { + Name: m.Spec.Image.PullSecrets, + }, + }, + }, + } + + // Set oracleRestDataService instance as the owner and controller + ctrl.SetControllerReference(m, pod, r.Scheme) + return pod +} + //############################################################################# // Instantiate Persistent Volume Claim spec from SingleInstanceDatabase spec //############################################################################# @@ -1031,146 +1132,20 @@ func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataS if m.Status.ApexConfigured { return requeueN } - - // Obtain admin password of the referred database - adminPasswordSecret := &corev1.Secret{} - err := r.Get(ctx, types.NamespacedName{Name: m.Spec.AdminPassword.SecretName, Namespace: m.Namespace}, adminPasswordSecret) - if err != nil { - if apierrors.IsNotFound(err) { - m.Status.Status = dbcommons.StatusError - eventReason := "Waiting" - eventMsg := "waiting for secret : " + m.Spec.AdminPassword.SecretName + " to get created" - r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) - r.Log.Info("Secret " + m.Spec.AdminPassword.SecretName + " Not Found") - return requeueY - } - log.Error(err, err.Error()) - return requeueY - } - sidbPassword := string(adminPasswordSecret.Data[m.Spec.AdminPassword.SecretKey]) - log.Info("SIDB Password: " + sidbPassword) - - apexPasswordSecret := &corev1.Secret{} - err = r.Get(ctx, types.NamespacedName{Name: m.Spec.ApexPassword.SecretName, Namespace: m.Namespace}, apexPasswordSecret) - if err != nil { - if apierrors.IsNotFound(err) { - m.Status.Status = dbcommons.StatusError - eventReason := "Waiting" - eventMsg := "waiting for secret : " + m.Spec.ApexPassword.SecretName + " to get created" - r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) - r.Log.Info("Secret " + m.Spec.ApexPassword.SecretName + " Not Found") - return requeueY - } - log.Error(err, err.Error()) - return requeueY - } - - // APEX_LISTENER , APEX_REST_PUBLIC_USER , APEX_PUBLIC_USER passwords - apexPassword := string(apexPasswordSecret.Data[m.Spec.ApexPassword.SecretKey]) - + if !n.Status.ApexInstalled { - result := r.installApexSIDB(m, n, ordsReadyPod, sidbPassword, apexPassword, ctx, req) + result := r.installApex(m, n, ordsReadyPod, ctx, req) if result.Requeue { log.Info("Reconcile requeued because apex installation failed") return result } } - /* // If Seperate PV for ORDS, we need to unzip apex-latest.zip - if m.Spec.Persistence.AccessMode != "" { - ordsReadyPod, _, _, _, err := dbcommons.FindPods(r, m.Spec.Image.Version, - m.Spec.Image.PullFrom, m.Name, m.Namespace, ctx, req) - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - if ordsReadyPod.Name == "" { - eventReason := "Waiting" - eventMsg := "waiting for " + m.Name + " to be Ready" - r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) - return requeueY - } - - unzipApex := dbcommons.UnzipApexOnORDSPod - if n.Spec.Edition == "express" { - unzipApex += dbcommons.ChownApex - } - - // Unzip /opt/oracle/ords/config/ords/apex-latest.zip to /opt/oracle/ords/config/ords - out, err := dbcommons.ExecCommand(r, r.Config, ordsReadyPod.Name, ordsReadyPod.Namespace, "", ctx, req, false, "bash", "-c", - unzipApex) - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - log.Info(" UnzipApex Output : \n" + out) - - if strings.Contains(out, "apex-latest.zip not found") { - eventReason := "Waiting" - eventMsg := "apex-latest.zip doesn't exist in the location /opt/oracle/ords/config/ords" - r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) - return requeueY - } - - } else { - - // Copy APEX Images to ORACLE_SID only if PV for ORDS and SIDB is shared - out, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf(dbcommons.CopyApexImages)) - if err != nil { - log.Info(err.Error()) - return requeueY - } - log.Info(" CopyApexImages Output : \n" + out) - - } */ - + m.Status.Status = dbcommons.StatusUpdating r.Status().Update(ctx, m) - configureApexRestSqlClient := "sqlplus -s / as sysdba @apex_rest_config.sql" - configureApexRestSqlClient = fmt.Sprintf(dbcommons.SQLPlusRemoteCLI, sidbPassword, n.Status.PdbConnectString) + " @apex_rest_config.sql" - // Configure APEX - out, err := dbcommons.ExecCommand(r, r.Config, ordsReadyPod.Name, ordsReadyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf(dbcommons.ConfigureApexRest, apexPassword, configureApexRestSqlClient)) - if err != nil { - log.Info(err.Error()) - if !strings.Contains(err.Error(), "catcon.pl: completed") { - return requeueY - } - } - log.Info(" ConfigureApexRest Output : \n" + out) - - if strings.Contains(out, "Apex Folder doesn't exist") { - eventReason := "Waiting" - eventMsg := "apex Folder doesn't exist in the location /opt/oracle/oradata/" + strings.ToUpper(n.Spec.Sid) - r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) - return requeueY - } - - // Alter APEX_LISTENER,APEX_PUBLIC_USER,APEX_REST_PUBLIC_USER - /* out, err = dbcommons.ExecCommand(r, r.Config, ordsReadyPod.Name, ordsReadyPod.Namespace, "", ctx, req, true, - "bash", "-c", fmt.Sprintf(dbcommons.AlterApexUsers, apexPassword, fmt.Sprintf(dbcommons.SQLPlusRemoteCLI, sidbPassword, n.Status.ConnectString))) - if err != nil { - log.Info(err.Error()) - return requeueY - } - log.Info(" AlterApexUsers Output : \n" + out) - - pdbName := "orclpdb1" - if n.Spec.Pdbname != "" { - pdbName = n.Spec.Pdbname - } - - // Change APEX Admin Password - out, err = dbcommons.ExecCommand(r, r.Config, ordsReadyPod.Name, ordsReadyPod.Namespace, "", ctx, req, true, - "bash", "-c", fmt.Sprintf("echo -e \"%s\" | %s", fmt.Sprintf(dbcommons.ApexAdmin, apexPassword, pdbName), fmt.Sprintf(dbcommons.SQLPlusRemoteCLI, sidbPassword, n.Status.ConnectString))) - if err != nil { - log.Info(err.Error()) - return requeueY - } - log.Info("Change ApexAdmin Password Output : \n" + out) */ - + m.Status.ApexConfigured = true r.Status().Update(ctx, m) @@ -1187,20 +1162,7 @@ func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataS available = append(available, readyPod) } - // Set Apex users in apex_rt,apex_al,apex files - out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf(dbcommons.SetApexUsers, apexPassword)) - log.Info("SetApexUsers Output: \n" + out) - if strings.Contains(strings.ToUpper(out), "ERROR") { - return requeueY - } - if err != nil { - log.Info(err.Error()) - if strings.Contains(strings.ToUpper(err.Error()), "ERROR") { - return requeueY - } - } - + for _, pod := range available { // ORDS Needs to be restarted to configure APEX err = r.Delete(ctx, &pod, &client.DeleteOptions{}) @@ -1221,8 +1183,8 @@ func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataS //############################################################################# // Install APEX in SIDB //############################################################################# -func (r *OracleRestDataServiceReconciler) installApexSIDB(m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase, - ordsReadyPod corev1.Pod, sidbPassword string, apexPassword string, ctx context.Context, req ctrl.Request) ctrl.Result { +func (r *OracleRestDataServiceReconciler) installApex(m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase, + ordsReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) ctrl.Result { log := r.Log.WithValues("installApex", req.NamespacedName) // Initial Validation @@ -1230,54 +1192,25 @@ func (r *OracleRestDataServiceReconciler) installApexSIDB(m *dbapi.OracleRestDat return requeueN } + pod := r.instantiateApexPodSpec(m, n) + + log.Info("Creating a new "+m.Name+" POD", "POD.Namespace", pod.Namespace, "POD.Name", pod.Name) + err := r.Create(ctx, pod) + if err != nil { + log.Error(err, "Failed to create new "+m.Name+" POD", "pod.Namespace", pod.Namespace, "POD.Name", pod.Name) + return requeueY + } + log.Info("Succesfully Created new "+m.Name+" POD", "POD.NAME : ", pod.Name) + // Status Updation m.Status.Status = dbcommons.StatusUpdating r.Status().Update(ctx, m) eventReason := "Installing Apex" eventMsg := "Waiting for Apex Installation to complete in SIDB pod" r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) - - //Install Apex in SIDB ready pod - out, err := dbcommons.ExecCommand(r, r.Config, ordsReadyPod.Name, ordsReadyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf(dbcommons.InstallApexRemoteB, fmt.Sprintf(dbcommons.SQLPlusRemoteCLI, sidbPassword, n.Status.PdbConnectString), apexPassword)) - if err != nil { - log.Info(err.Error()) - } - log.Info(" InstallApex Output : \n" + out) - - if strings.Contains(out, "Apex Folder doesn't exist") { - eventReason := "Waiting" - eventMsg := "apex Folder doesn't exist in the location /opt/oracle/ords/config" - r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) - return requeueY - } - - // Checking if Apex is installed successfully or not - out, err = dbcommons.ExecCommand(r, r.Config, ordsReadyPod.Name, ordsReadyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf("echo -e \"%s\" | %s", dbcommons.IsApexInstalled, fmt.Sprintf(dbcommons.SQLPlusRemoteCLI, sidbPassword, n.Status.PdbConnectString))) - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - log.Info("IsApexInstalled Output: \n" + out) - - apexInstalled := "APEXVERSION:" - if !strings.Contains(out, apexInstalled) { - return requeueY - } - + n.Status.ApexInstalled = true - // Make sure m.Status.ApexInstalled is set to true . - for i := 0; i < 10; i++ { - err = r.Status().Update(ctx, n) - if err != nil { - log.Info(err.Error() + "\n updating m.Status.ApexInstalled = true") - time.Sleep(5 * time.Second) - continue - } - break - } - + r.Status().Update(ctx, n) return requeueN } @@ -1498,7 +1431,7 @@ func (r *OracleRestDataServiceReconciler) restEnableSchemas(m *dbapi.OracleRestD log.Error(err, err.Error()) return requeueY } else { - log.Info("getOrdsSchemaStatus Output") + log.Info("getOrdsSchemaStatus Output", "schema", m.Spec.RestEnableSchemas[i].Schema) log.Info(out) } From e4065c49fac302169ee29be4f955d94b0efd8875 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Tue, 26 Apr 2022 13:42:28 +0530 Subject: [PATCH 264/628] Fix POD cmd --- commons/database/constants.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/commons/database/constants.go b/commons/database/constants.go index b2193dfe..e2a8f03c 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -411,7 +411,7 @@ const InstallApex string = "if [ -f /opt/oracle/oradata/${ORACLE_SID^^}/apex/ape const InstallApexRemote string = "if [ -e ${ORDS_HOME}/config/apex/apxsilentins.sql ]; then cd ${ORDS_HOME}/config/apex/ && echo -e \"@apxsilentins.sql SYSAUX SYSAUX TEMP /i/ %[2]s %[2]s %[2]s %[2]s\" | %[1]s; else echo \"Apex Folder doesn't exist\" ; fi ;" -const InstallApexInContainer string = "cd ${ORDS_HOME}/config/apex/ && echo -e \"@apxsilentins.sql SYSAUX SYSAUX TEMP /i/ ${ORD_PWD} ${ORD_PWD} ${ORD_PWD} ${ORD_PWD}\" | sqlplus -s sys/${ORACLE_PWD}@${ORACLE_HOST}:${ORACLE_PORT}/${ORACLE_SERVICE} as sysdba;" +const InstallApexInContainer string = "cd ${ORDS_HOME}/config/apex/ && echo -e \"@apxsilentins.sql SYSAUX SYSAUX TEMP /i/ ${ORDS_PWD} ${ORDS_PWD} ${ORDS_PWD} ${ORDS_PWD};\n@apex_rest_config_core.sql;\n\" | sqlplus -s sys/${ORACLE_PWD}@${ORACLE_HOST}:${ORACLE_PORT}/${ORACLE_SERVICE} as sysdba;" const IsApexInstalled string = "select 'APEXVERSION:'||version as version FROM DBA_REGISTRY WHERE COMP_ID='APEX';" From 01f1cc757928beb09c29acf2836e110a13cf6388 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Tue, 26 Apr 2022 17:00:56 +0530 Subject: [PATCH 265/628] Test Pod status --- commons/database/utils.go | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/commons/database/utils.go b/commons/database/utils.go index 04699044..cf41535e 100644 --- a/commons/database/utils.go +++ b/commons/database/utils.go @@ -98,6 +98,11 @@ func ResourceEventHandler() predicate.Predicate { if oldStatus != newStatus { return true } + + if newPodObject.Status.Phase == corev1.PodSucceeded { + //if (oldPodObject.Status.Phase != + return true + } } // Ignore updates to CR status in which case metadata.Generation does not change // Reconcile if object Deletion Timestamp Set From b33aff99920ac5fe069c1f79d1995416f2e3c067 Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Tue, 26 Apr 2022 18:27:56 +0530 Subject: [PATCH 266/628] Some corrections Signed-off-by: abhisbyk --- docs/sidb/README.md | 10 ++-------- 1 file changed, 2 insertions(+), 8 deletions(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 78b7f529..b5e288a1 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -10,7 +10,7 @@ Oracle Database Operator for Kubernetes (the operator) includes the Single Insta - [Patch/Rollback Database](#patchrollback-database) - [Kind OracleRestDataService](#kind-oraclerestdataservice) - [REST Enable Database](#rest-enable-database) - - [Performing manual operations](#performing-manual-operations) + - [Performing maintenance operations](#performing-maintenance-operations) ## Prerequisites @@ -649,12 +649,6 @@ The Oracle Database Operator creates the OracleRestDataService (ORDS) kind as a **NOTE** * Apex Administrator for pdbs other than `.spec.databaseRef.pdbName` has to be created manually. More Info on creating Apex Administrator can be found at [APEX_UTIL.CREATE_USER] * By default, the full development runtime environment is initialized in APEX. It can be changed manually to the runtime environment. For this, `apxdevrm.sql` script should be run after connecting to the primary database from the ORDS pod as the sys user with sysdba privilage. Please click the [link](https://docs.oracle.com/en/database/oracle/application-express/21.2/htmig/converting-between-runtime-and-full-development-environments.html#GUID-B0621B40-3441-44ED-9D86-29B058E26BE9) for detailed instructions. - **Uninstall APEX:** - * Set `.spec.installApex` to false in [config/samples/sidb/singleinstancedatabase.yaml](config/samples/sidb/singleinstancedatabase.yaml) - * If you install APEX again, re-configure APEX with ORDS again. - - More info on Application Express can be found at - * ### Multiple Replicas Currently only single replica mode is supported @@ -666,7 +660,7 @@ The Oracle Database Operator creates the OracleRestDataService (ORDS) kind as a * A schema can be rest enabled or disabled by setting the `.spec.restEnableSchemas[].enable` to true or false respectively in ords sample .yaml file and apply using the kubectl apply command or edit/patch commands. This requires `.spec.ordsPassword` secret. -## Performing manual operations +## Performing maintenance operations If some manual operations are required to be performed, the procedure is as follows: - Exec into the pod from where you want to perform the manual operation using the similar command to the following command: From 0d56f49f482e18f02cceeb1610383bdd0576caac Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Tue, 26 Apr 2022 22:50:17 +0530 Subject: [PATCH 267/628] Update Apex command --- commons/database/constants.go | 12 +++------ commons/database/utils.go | 6 +---- .../oraclerestdataservice_controller.go | 26 +++++++------------ 3 files changed, 14 insertions(+), 30 deletions(-) diff --git a/commons/database/constants.go b/commons/database/constants.go index e2a8f03c..1bb9ce35 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -403,16 +403,12 @@ const ChownApex string = " chown oracle:oinstall /opt/oracle/oradata/${ORACLE_SI const InstallApex string = "if [ -f /opt/oracle/oradata/${ORACLE_SID^^}/apex/apexins.sql ]; then ( while true; do sleep 60; echo \"Installing Apex...\" ; done ) & " + " cd /opt/oracle/oradata/${ORACLE_SID^^}/apex && echo -e \"@apexins.sql SYSAUX SYSAUX TEMP /i/\" | %[1]s && kill -9 $!; else echo \"Apex Folder doesn't exist\" ; fi ;" -//const InstallApexRemote string = "if [ -e ${ORDS_HOME}/config/apex/apexins.sql ]; then ( while true; do sleep 60; echo \"Installing Apex...\" ; done ) & " + -// " cd ${ORDS_HOME}/config/apex/ && echo -e \"@apexins.sql SYSAUX SYSAUX TEMP /i/\" | %[1]s && kill -9 $!; else echo \"Apex Folder doesn't exist\" ; fi ;" - -// const InstallApexRemoteA string = "if [ -e ${ORDS_HOME}/config/apex/apxsilentins.sql ]; then ( while true; do sleep 60; echo \"Installing Apex...\" ; done ) & " + -// " cd ${ORDS_HOME}/config/apex/ && echo -e \"@apxsilentins.sql SYSAUX SYSAUX TEMP /i/ %[2]s %[2]s %[2]s %[2]s\" | %[1]s && kill -9 $!; else echo \"Apex Folder doesn't exist\" ; fi ;" - const InstallApexRemote string = "if [ -e ${ORDS_HOME}/config/apex/apxsilentins.sql ]; then cd ${ORDS_HOME}/config/apex/ && echo -e \"@apxsilentins.sql SYSAUX SYSAUX TEMP /i/ %[2]s %[2]s %[2]s %[2]s\" | %[1]s; else echo \"Apex Folder doesn't exist\" ; fi ;" -const InstallApexInContainer string = "cd ${ORDS_HOME}/config/apex/ && echo -e \"@apxsilentins.sql SYSAUX SYSAUX TEMP /i/ ${ORDS_PWD} ${ORDS_PWD} ${ORDS_PWD} ${ORDS_PWD};\n@apex_rest_config_core.sql;\n\" | sqlplus -s sys/${ORACLE_PWD}@${ORACLE_HOST}:${ORACLE_PORT}/${ORACLE_SERVICE} as sysdba;" - +const InstallApexInContainer string = "cd ${ORDS_HOME}/config/apex/ && echo -e \"@apxsilentins.sql SYSAUX SYSAUX TEMP /i/ ${ORDS_PWD} ${ORDS_PWD} ${ORDS_PWD} ${ORDS_PWD};"+ + "\n@apex_rest_config_core.sql;\nexec APEX_UTIL.set_workspace(p_workspace => 'INTERNAL');\n"+ + "exec APEX_UTIL.EDIT_USER(p_user_id => APEX_UTIL.GET_USER_ID('ADMIN'), p_user_name => 'ADMIN', p_change_password_on_first_use => 'Y');\n\"" + + " | sqlplus -s sys/${ORACLE_PWD}@${ORACLE_HOST}:${ORACLE_PORT}/${ORACLE_SERVICE} as sysdba;" const IsApexInstalled string = "select 'APEXVERSION:'||version as version FROM DBA_REGISTRY WHERE COMP_ID='APEX';" diff --git a/commons/database/utils.go b/commons/database/utils.go index cf41535e..875e392b 100644 --- a/commons/database/utils.go +++ b/commons/database/utils.go @@ -98,11 +98,7 @@ func ResourceEventHandler() predicate.Predicate { if oldStatus != newStatus { return true } - - if newPodObject.Status.Phase == corev1.PodSucceeded { - //if (oldPodObject.Status.Phase != - return true - } + } // Ignore updates to CR status in which case metadata.Generation does not change // Reconcile if object Deletion Timestamp Set diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 2999265f..de6556bd 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -135,6 +135,7 @@ func (r *OracleRestDataServiceReconciler) Reconcile(ctx context.Context, req ctr // Always refresh status before a reconcile defer r.Status().Update(ctx, oracleRestDataService) + defer r.Status().Update(ctx, singleInstanceDatabase) // Create Service result = r.createSVC(ctx, req, oracleRestDataService) @@ -160,27 +161,20 @@ func (r *OracleRestDataServiceReconciler) Reconcile(ctx context.Context, req ctr // setup DB for ORDS var cdbAdminPassword string result, cdbAdminPassword = r.setupDBPrequisites(oracleRestDataService, singleInstanceDatabase, sidbReadyPod, ctx, req) - if result.Requeue { - r.Log.Info("Reconcile queued") - return result, nil - } - - // Create ORDS Pods - result = r.createPods(oracleRestDataService, singleInstanceDatabase, sidbReadyPod, ctx, req) if result.Requeue { r.Log.Info("Reconcile queued") return result, nil } - // Validate ORDS pod readiness - result, ordsReadyPod := r.validateORDSReadiness(oracleRestDataService, ctx, req) + // Create ORDS Pods + result = r.createPods(oracleRestDataService, singleInstanceDatabase, sidbReadyPod, ctx, req) if result.Requeue { r.Log.Info("Reconcile queued") return result, nil } // Configure Apex - result = r.configureApex(oracleRestDataService, singleInstanceDatabase, ordsReadyPod, ctx, req) + result = r.configureApex(oracleRestDataService, singleInstanceDatabase, ctx, req) if result.Requeue { r.Log.Info("Reconcile queued") return result, nil @@ -1122,7 +1116,7 @@ func (r *OracleRestDataServiceReconciler) cleanupOracleRestDataService(req ctrl. // Configure APEX //############################################################################# func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase, - ordsReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) ctrl.Result { + ctx context.Context, req ctrl.Request) ctrl.Result { log := r.Log.WithValues("configureApex", req.NamespacedName) if m.Spec.ApexPassword.SecretName == "" { @@ -1134,7 +1128,8 @@ func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataS } if !n.Status.ApexInstalled { - result := r.installApex(m, n, ordsReadyPod, ctx, req) + m.Status.Status = dbcommons.StatusUpdating + result := r.installApex(m, n, ctx, req) if result.Requeue { log.Info("Reconcile requeued because apex installation failed") return result @@ -1142,9 +1137,6 @@ func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataS } - m.Status.Status = dbcommons.StatusUpdating - r.Status().Update(ctx, m) - m.Status.ApexConfigured = true r.Status().Update(ctx, m) @@ -1184,7 +1176,7 @@ func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataS // Install APEX in SIDB //############################################################################# func (r *OracleRestDataServiceReconciler) installApex(m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase, - ordsReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) ctrl.Result { + ctx context.Context, req ctrl.Request) ctrl.Result { log := r.Log.WithValues("installApex", req.NamespacedName) // Initial Validation @@ -1206,7 +1198,7 @@ func (r *OracleRestDataServiceReconciler) installApex(m *dbapi.OracleRestDataSer m.Status.Status = dbcommons.StatusUpdating r.Status().Update(ctx, m) eventReason := "Installing Apex" - eventMsg := "Waiting for Apex Installation to complete in SIDB pod" + eventMsg := "Waiting for Apex Installation to complete" r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) n.Status.ApexInstalled = true From 742977812139009341d02a668d7d6d55a6e8cfbd Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Wed, 27 Apr 2022 09:24:57 +0530 Subject: [PATCH 268/628] Removed unwanted files Signed-off-by: abhisbyk --- .DS_Store | Bin 8196 -> 0 bytes images/.DS_Store | Bin 6148 -> 0 bytes 2 files changed, 0 insertions(+), 0 deletions(-) delete mode 100644 .DS_Store delete mode 100644 images/.DS_Store diff --git a/.DS_Store b/.DS_Store deleted file mode 100644 index 32e70b56023cc58b7918ddca0681d4be3ea805f8..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 8196 zcmeHM&2G~`5T0#K;t&BrqM`>XB5^IDm1wENB|nWIKq)i04{21#Tg46mRYkc1 z@4zc?Vu)bpAbfS4 zF9!IDFMV|q>dKfWxUvX#C?eQ+h`fT6DEd;jngPwgGy}YMH^`!0>XRGWzn5^X@KY6~ zI7=;{oEYj$40Rr;5joT$o4Qn>b&}Mk9rTCv8u}QG&L9?IGo*o+G{8?{o|skQQw*I_ zXh+T6i_QEahCN3W+NAr`pe?FVnI556!|^THIiOBdo7n%)A^)?%l{|PN;RpLSjN3=d zM_B9hF&PN!7+{ZCOY%L8OHmIvwq3@p7wmfy*2U@!racvBL!KJ6jhMiy#c=Q~f=gjN z2SE0~MBG-a>e1NN*xe1j6n$;&+I`nsTKXntW{p#+w3#-u=8O8a9M?U|8@Xj`_>7Ml zQrfM9zSVl%?~a?f3l-^jR^RCkl|sMM1?2gYzSEK8vK%=bSLxRM6K2NDG;f?$`rIegJ(aB6!}=8UOgO{fJ(~{WtZYV@s#U zN~l@7htR4}72#D!TOg0#k{2bEgE>M{TG5Fl10G8oLt{xJXp=TTKBrbfy5>_D-Qc&>|)=w+_O}Yb(3N@luamYuk%BbsaT7wmSijvkIJ+$vnmuT8FT;he@M5 zCy9Ys-YrGiFn y$05~r9P;oFL-g~2vQ9CO6JKHk<*$DTi2Oxaf&t+tm45#J?tMZ(|5e!j^UW`52N&W1 diff --git a/images/.DS_Store b/images/.DS_Store deleted file mode 100644 index b07c0cc2d2a53d29089c95afca8976eb4a9ac6fb..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 6148 zcmeHK!AiqG5Z!I7-BN@c6g>vK7Hz@Sf|pq9!K)EHsMN$14aTfAsX3HFuKF_`{S$vj zXLh%R()OrOnSt3iJ3F(Oh(E;cf%r$}sgzHp5oytv#!AW13ynGUk3mH From a3b26ce31ee17231d79bacf69fdeb309436d775e Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Wed, 27 Apr 2022 10:57:31 +0530 Subject: [PATCH 269/628] Enhance ORDS install --- commons/database/constants.go | 15 +- .../oraclerestdataservice_controller.go | 160 ++++++++++-------- 2 files changed, 97 insertions(+), 78 deletions(-) diff --git a/commons/database/constants.go b/commons/database/constants.go index 1bb9ce35..9d0d5243 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -285,7 +285,8 @@ const SetupORDSCMD string = "$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-pr "\nsed -i 's,jetty.port=8888,jetty.secure.port=8443\\nssl.cert=\\nssl.cert.key=\\nssl.host=%[3]s,g' /opt/oracle/ords/config/ords/standalone/standalone.properties " + "\nsed -i 's,standalone.static.path=/opt/oracle/ords/doc_root/i,standalone.static.path=/opt/oracle/ords/config/apex/images,g' /opt/oracle/ords/config/ords/standalone/standalone.properties" -const InitORDSCMD string = "$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property database.api.enabled true" + +const InitORDSCMD string = "if [ -f $ORDS_HOME/config/ords/defaults.xml ]; then exit fi; \n" + + "$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property database.api.enabled true" + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property jdbc.auth.enabled true" + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property database.api.management.services.disabled false" + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property database.api.admin.enabled true" + @@ -300,18 +301,18 @@ const InitORDSCMD string = "$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-pro "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property jdbc.maxRows 1000" + "\numask 177" + "\necho db.cdb.adminUser=C##DBAPI_CDB_ADMIN AS SYSDBA > cdbAdmin.properties" + - "\necho db.cdb.adminUser.password=\"%[4]s\" >> cdbAdmin.properties" + + "\necho db.cdb.adminUser.password=\"${ORDS_PWD}\" >> cdbAdmin.properties" + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-properties --conf apex_pu cdbAdmin.properties" + "\nrm -f cdbAdmin.properties" + "\necho db.adminUser=C##_DBAPI_PDB_ADMIN > pdbAdmin.properties" + - "\necho db.adminUser.password=\"%[4]s\">> pdbAdmin.properties" + + "\necho db.adminUser.password=\"${ORDS_PWD}\">> pdbAdmin.properties" + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-properties --conf apex_pu pdbAdmin.properties" + "\nrm -f pdbAdmin.properties" + - "\necho -e \"%[1]s\n%[1]s\" > sqladmin.passwd" + + "\necho -e \"${ORDS_PWD}\n${ORDS_PWD}\" > sqladmin.passwd" + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war user ${ORDS_USER} \"SQL Administrator , System Administrator , SQL Developer , oracle.dbtools.autorest.any.schema \" < sqladmin.passwd" + "\nrm -f sqladmin.passwd" + "\numask 022" + - "\nsed -i 's,jetty.port=8888,jetty.secure.port=8443\\nssl.cert=\\nssl.cert.key=\\nssl.host=%[3]s,g' /opt/oracle/ords/config/ords/standalone/standalone.properties " + "\n$ORDS_HOME/runOrds.sh" const GetSessionInfoSQL string = "select s.sid || ',' || s.serial# as Info FROM v\\$session s, v\\$process p WHERE s.username = 'ORDS_PUBLIC_USER' AND p.addr(+) = s.paddr;" @@ -331,8 +332,8 @@ const UninstallORDSCMD string = "\numask 177" + "\nrm -rf /opt/oracle/ords/config/ords/standalone" + "\nrm -rf /opt/oracle/ords/config/ords/apex" -// To handle timing issue, checking if either of https://localhost:8443 or http://localhost:8888 is used for ORDS Installation -const GetORDSStatus string = " curl -sSkv -k -X GET https://localhost:8443/ords/_/db-api/stable/metadata-catalog/ || curl -sSkv -X GET http://localhost:8888/ords/_/db-api/stable/metadata-catalog/ " + +const GetORDSStatus string = "curl -sSkv -k -X GET https://localhost:8443/ords/_/db-api/stable/metadata-catalog/" const ValidateAdminPassword string = "conn sys/%s@${ORACLE_SID} as sysdba\nshow user" diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index de6556bd..eeb7e937 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -167,7 +167,7 @@ func (r *OracleRestDataServiceReconciler) Reconcile(ctx context.Context, req ctr } // Create ORDS Pods - result = r.createPods(oracleRestDataService, singleInstanceDatabase, sidbReadyPod, ctx, req) + result = r.createPods(oracleRestDataService, singleInstanceDatabase, cdbAdminPassword, ctx, req) if result.Requeue { r.Log.Info("Reconcile queued") return result, nil @@ -180,7 +180,7 @@ func (r *OracleRestDataServiceReconciler) Reconcile(ctx context.Context, req ctr return result, nil } - result = r.setupORDS(oracleRestDataService, singleInstanceDatabase, sidbReadyPod, cdbAdminPassword, ctx, req) + result = r.restEnableSchemas(oracleRestDataService, singleInstanceDatabase, sidbReadyPod, ctx, req) if result.Requeue { r.Log.Info("Reconcile queued") return result, nil @@ -446,7 +446,8 @@ func (r *OracleRestDataServiceReconciler) instantiateSVCSpec(m *dbapi.OracleRest //############################################################################# // Instantiate POD spec from OracleRestDataService spec //############################################################################# -func (r *OracleRestDataServiceReconciler) instantiatePodSpec(m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase, ordsInstalled bool) *corev1.Pod { +func (r *OracleRestDataServiceReconciler) instantiatePodSpec(m *dbapi.OracleRestDataService, + n *dbapi.SingleInstanceDatabase, ordsInstalled bool, cdbAdminPassword string) *corev1.Pod { pod := &corev1.Pod{ TypeMeta: metav1.TypeMeta{ @@ -475,20 +476,90 @@ func (r *OracleRestDataServiceReconciler) instantiatePodSpec(m *dbapi.OracleRest }, }, }}, - InitContainers: []corev1.Container{{ - Name: "init-permissions", - Image: m.Spec.Image.PullFrom, - Command: []string{"/bin/sh", "-c", fmt.Sprintf("chown %d:%d /opt/oracle/ords/config/ords", int(dbcommons.ORACLE_UID), int(dbcommons.DBA_GUID))}, - SecurityContext: &corev1.SecurityContext{ - // User ID 0 means, root user - RunAsUser: func() *int64 { i := int64(0); return &i }(), + InitContainers: []corev1.Container{ + { + Name: "init-permissions", + Image: m.Spec.Image.PullFrom, + Command: []string{"/bin/sh", "-c", fmt.Sprintf("chown %d:%d /opt/oracle/ords/config/ords", int(dbcommons.ORACLE_UID), int(dbcommons.DBA_GUID))}, + SecurityContext: &corev1.SecurityContext{ + // User ID 0 means, root user + RunAsUser: func() *int64 { i := int64(0); return &i }(), + }, + VolumeMounts: []corev1.VolumeMount{{ + MountPath: "/opt/oracle/ords/config/ords", + Name: "datamount", + SubPath: strings.ToUpper(n.Spec.Sid) + "_ORDS", + }}, }, - VolumeMounts: []corev1.VolumeMount{{ - MountPath: "/opt/oracle/ords/config/ords", - Name: "datamount", - SubPath: strings.ToUpper(n.Spec.Sid) + "_ORDS", - }}, - }}, + { + Name: "init-ords", + Image: m.Spec.Image.PullFrom, + Command: []string{"/bin/sh", "-c", dbcommons.InitORDSCMD}, + SecurityContext: &corev1.SecurityContext{ + RunAsUser: func() *int64 { i := int64(dbcommons.ORACLE_UID); return &i }(), + RunAsGroup: func() *int64 { i := int64(dbcommons.DBA_GUID); return &i }(), + }, + VolumeMounts: []corev1.VolumeMount{{ + MountPath: "/opt/oracle/ords/config/ords", + Name: "datamount", + SubPath: strings.ToUpper(n.Spec.Sid) + "_ORDS", + }}, + Env: []corev1.EnvVar { + { + Name: "SETUP_ONLY", + Value: "true", + }, + { + Name: "ORACLE_HOST", + Value: n.Name, + }, + { + Name: "ORACLE_PORT", + Value: "1521", + }, + { + Name: "ORACLE_SERVICE", + Value: func() string { + if m.Spec.OracleService != "" { + return m.Spec.OracleService + } + return n.Spec.Sid + }(), + }, + { + Name: "ORDS_USER", + Value: func() string { + if m.Spec.OrdsUser != "" { + return m.Spec.OrdsUser + } + return "ORDS_PUBLIC_USER" + }(), + }, + { + Name: "ORDS_PWD", + ValueFrom: &corev1.EnvVarSource{ + SecretKeyRef: &corev1.SecretKeySelector{ + LocalObjectReference: corev1.LocalObjectReference{ + Name: m.Spec.OrdsPassword.SecretName, + }, + Key: m.Spec.OrdsPassword.SecretKey, + }, + }, + }, + { + Name: "ORACLE_PWD", + ValueFrom: &corev1.EnvVarSource{ + SecretKeyRef: &corev1.SecretKeySelector{ + LocalObjectReference: corev1.LocalObjectReference{ + Name: m.Spec.AdminPassword.SecretName, + }, + Key: m.Spec.AdminPassword.SecretKey, + }, + }, + }, + }, + }, + }, Containers: []corev1.Container{{ Name: m.Name, Image: m.Spec.Image.PullFrom, @@ -499,59 +570,6 @@ func (r *OracleRestDataServiceReconciler) instantiatePodSpec(m *dbapi.OracleRest SubPath: strings.ToUpper(n.Spec.Sid) + "_ORDS", }}, Env: func() []corev1.EnvVar { - // ORDS_PWD, ORACLE_PWD is required only during the First ORDS Installation. - if !ordsInstalled { - return []corev1.EnvVar{ - { - Name: "ORACLE_HOST", - Value: n.Name, - }, - { - Name: "ORACLE_PORT", - Value: "1521", - }, - { - Name: "ORACLE_SERVICE", - Value: func() string { - if m.Spec.OracleService != "" { - return m.Spec.OracleService - } - return n.Spec.Sid - }(), - }, - { - Name: "ORDS_USER", - Value: func() string { - if m.Spec.OrdsUser != "" { - return m.Spec.OrdsUser - } - return "ORDS_PUBLIC_USER" - }(), - }, - { - Name: "ORDS_PWD", - ValueFrom: &corev1.EnvVarSource{ - SecretKeyRef: &corev1.SecretKeySelector{ - LocalObjectReference: corev1.LocalObjectReference{ - Name: m.Spec.OrdsPassword.SecretName, - }, - Key: m.Spec.OrdsPassword.SecretKey, - }, - }, - }, - { - Name: "ORACLE_PWD", - ValueFrom: &corev1.EnvVarSource{ - SecretKeyRef: &corev1.SecretKeySelector{ - LocalObjectReference: corev1.LocalObjectReference{ - Name: m.Spec.AdminPassword.SecretName, - }, - Key: m.Spec.AdminPassword.SecretKey, - }, - }, - }, - } - } // After ORDS is Installed, we DELETE THE OLD ORDS Pod and create new ones ONLY USING BELOW ENV VARIABLES. return []corev1.EnvVar{ { @@ -846,7 +864,7 @@ func (r *OracleRestDataServiceReconciler) createPVC(ctx context.Context, req ctr // Create the requested POD replicas //############################################################################# func (r *OracleRestDataServiceReconciler) createPods(m *dbapi.OracleRestDataService, - n *dbapi.SingleInstanceDatabase, sidbReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) ctrl.Result { + n *dbapi.SingleInstanceDatabase, cdbAdminPassword string, ctx context.Context, req ctrl.Request) ctrl.Result { log := r.Log.WithValues("createPods", req.NamespacedName) @@ -880,7 +898,7 @@ func (r *OracleRestDataServiceReconciler) createPods(m *dbapi.OracleRestDataServ if m.Status.OrdsInstalled { ordsInstalled = true } - pod := r.instantiatePodSpec(m, n, ordsInstalled) + pod := r.instantiatePodSpec(m, n, ordsInstalled, cdbAdminPassword) log.Info("Creating a new "+m.Name+" POD", "POD.Namespace", pod.Namespace, "POD.Name", pod.Name) err := r.Create(ctx, pod) if err != nil { From 427c5c3468300401d78b91a46464604568152bd7 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Wed, 27 Apr 2022 11:11:08 +0530 Subject: [PATCH 270/628] Fix init cmd --- commons/database/constants.go | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/commons/database/constants.go b/commons/database/constants.go index 9d0d5243..393b85a8 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -285,8 +285,8 @@ const SetupORDSCMD string = "$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-pr "\nsed -i 's,jetty.port=8888,jetty.secure.port=8443\\nssl.cert=\\nssl.cert.key=\\nssl.host=%[3]s,g' /opt/oracle/ords/config/ords/standalone/standalone.properties " + "\nsed -i 's,standalone.static.path=/opt/oracle/ords/doc_root/i,standalone.static.path=/opt/oracle/ords/config/apex/images,g' /opt/oracle/ords/config/ords/standalone/standalone.properties" -const InitORDSCMD string = "if [ -f $ORDS_HOME/config/ords/defaults.xml ]; then exit fi; \n" + - "$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property database.api.enabled true" + +const InitORDSCMD string = "if [ -f $ORDS_HOME/config/ords/defaults.xml ]; then exit ;fi;" + + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property database.api.enabled true" + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property jdbc.auth.enabled true" + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property database.api.management.services.disabled false" + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property database.api.admin.enabled true" + From 4f92fd79c37dfcd5135851995e4414ce85796005 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Wed, 27 Apr 2022 11:37:17 +0530 Subject: [PATCH 271/628] Fix init ORDS command --- commons/database/constants.go | 2 ++ 1 file changed, 2 insertions(+) diff --git a/commons/database/constants.go b/commons/database/constants.go index 393b85a8..10492ff0 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -299,6 +299,7 @@ const InitORDSCMD string = "if [ -f $ORDS_HOME/config/ords/defaults.xml ]; then "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property feature.sdw true" + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property security.verifySSL false" + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property jdbc.maxRows 1000" + + "\nmkdir -p $ORDS_HOME/config/ords/conf" + "\numask 177" + "\necho db.cdb.adminUser=C##DBAPI_CDB_ADMIN AS SYSDBA > cdbAdmin.properties" + "\necho db.cdb.adminUser.password=\"${ORDS_PWD}\" >> cdbAdmin.properties" + @@ -312,6 +313,7 @@ const InitORDSCMD string = "if [ -f $ORDS_HOME/config/ords/defaults.xml ]; then "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war user ${ORDS_USER} \"SQL Administrator , System Administrator , SQL Developer , oracle.dbtools.autorest.any.schema \" < sqladmin.passwd" + "\nrm -f sqladmin.passwd" + "\numask 022" + + "\nexport APEXI=$ORDS_HOME/config/apex/images" + "\n$ORDS_HOME/runOrds.sh" const GetSessionInfoSQL string = "select s.sid || ',' || s.serial# as Info FROM v\\$session s, v\\$process p WHERE s.username = 'ORDS_PUBLIC_USER' AND p.addr(+) = s.paddr;" From 0be3e589a2e80b1b3bb1f9d9cb01c6a4a3b1dee3 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Wed, 27 Apr 2022 12:06:02 +0530 Subject: [PATCH 272/628] Debug ORDS install --- commons/database/constants.go | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/commons/database/constants.go b/commons/database/constants.go index 10492ff0..10462b7b 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -314,7 +314,8 @@ const InitORDSCMD string = "if [ -f $ORDS_HOME/config/ords/defaults.xml ]; then "\nrm -f sqladmin.passwd" + "\numask 022" + "\nexport APEXI=$ORDS_HOME/config/apex/images" + - "\n$ORDS_HOME/runOrds.sh" + "\nset -x" + + "\n. $ORDS_HOME/runOrds.sh" const GetSessionInfoSQL string = "select s.sid || ',' || s.serial# as Info FROM v\\$session s, v\\$process p WHERE s.username = 'ORDS_PUBLIC_USER' AND p.addr(+) = s.paddr;" From 2f621f8410f5f945c758ede2c34da577a70695f5 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Wed, 27 Apr 2022 12:17:40 +0530 Subject: [PATCH 273/628] Fix init ORDS cmd --- commons/database/constants.go | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/commons/database/constants.go b/commons/database/constants.go index 10462b7b..2fcff64d 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -286,6 +286,9 @@ const SetupORDSCMD string = "$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-pr "\nsed -i 's,standalone.static.path=/opt/oracle/ords/doc_root/i,standalone.static.path=/opt/oracle/ords/config/apex/images,g' /opt/oracle/ords/config/ords/standalone/standalone.properties" const InitORDSCMD string = "if [ -f $ORDS_HOME/config/ords/defaults.xml ]; then exit ;fi;" + + "\nexport APEXI=$ORDS_HOME/config/apex/images" + + "\nset -x" + + "\n. $ORDS_HOME/runOrds.sh" + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property database.api.enabled true" + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property jdbc.auth.enabled true" + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property database.api.management.services.disabled false" + @@ -312,10 +315,7 @@ const InitORDSCMD string = "if [ -f $ORDS_HOME/config/ords/defaults.xml ]; then "\necho -e \"${ORDS_PWD}\n${ORDS_PWD}\" > sqladmin.passwd" + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war user ${ORDS_USER} \"SQL Administrator , System Administrator , SQL Developer , oracle.dbtools.autorest.any.schema \" < sqladmin.passwd" + "\nrm -f sqladmin.passwd" + - "\numask 022" + - "\nexport APEXI=$ORDS_HOME/config/apex/images" + - "\nset -x" + - "\n. $ORDS_HOME/runOrds.sh" + "\numask 022" const GetSessionInfoSQL string = "select s.sid || ',' || s.serial# as Info FROM v\\$session s, v\\$process p WHERE s.username = 'ORDS_PUBLIC_USER' AND p.addr(+) = s.paddr;" From 02feb32f6b989eff7f57518d46c071cb9d746365 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Wed, 27 Apr 2022 12:24:15 +0530 Subject: [PATCH 274/628] Fix runOrds cmd --- commons/database/constants.go | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/commons/database/constants.go b/commons/database/constants.go index 2fcff64d..f2b73aa0 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -287,8 +287,7 @@ const SetupORDSCMD string = "$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-pr const InitORDSCMD string = "if [ -f $ORDS_HOME/config/ords/defaults.xml ]; then exit ;fi;" + "\nexport APEXI=$ORDS_HOME/config/apex/images" + - "\nset -x" + - "\n. $ORDS_HOME/runOrds.sh" + + "\n$ORDS_HOME/runOrds.sh" + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property database.api.enabled true" + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property jdbc.auth.enabled true" + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property database.api.management.services.disabled false" + From 6eb4f75bba5d5419439a497552083f344c047b26 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Wed, 27 Apr 2022 12:36:38 +0530 Subject: [PATCH 275/628] Change reconcile order --- controllers/database/oraclerestdataservice_controller.go | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index eeb7e937..06e6a37b 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -180,13 +180,13 @@ func (r *OracleRestDataServiceReconciler) Reconcile(ctx context.Context, req ctr return result, nil } - result = r.restEnableSchemas(oracleRestDataService, singleInstanceDatabase, sidbReadyPod, ctx, req) + result = r.checkHealthStatus(oracleRestDataService, ctx, req) if result.Requeue { r.Log.Info("Reconcile queued") return result, nil } - result = r.checkHealthStatus(oracleRestDataService, ctx, req) + result = r.restEnableSchemas(oracleRestDataService, singleInstanceDatabase, sidbReadyPod, ctx, req) if result.Requeue { r.Log.Info("Reconcile queued") return result, nil From 33f4a40320780031ed89772cf1e1e0e65abef3bb Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Wed, 27 Apr 2022 16:24:56 +0530 Subject: [PATCH 276/628] Apex install change --- commons/database/constants.go | 9 +- .../oraclerestdataservice_controller.go | 230 ++++++------------ 2 files changed, 86 insertions(+), 153 deletions(-) diff --git a/commons/database/constants.go b/commons/database/constants.go index f2b73aa0..50e70a46 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -287,7 +287,7 @@ const SetupORDSCMD string = "$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-pr const InitORDSCMD string = "if [ -f $ORDS_HOME/config/ords/defaults.xml ]; then exit ;fi;" + "\nexport APEXI=$ORDS_HOME/config/apex/images" + - "\n$ORDS_HOME/runOrds.sh" + + "\n$ORDS_HOME/runOrds.sh --setuponly" + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property database.api.enabled true" + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property jdbc.auth.enabled true" + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-property database.api.management.services.disabled false" + @@ -408,12 +408,13 @@ const InstallApex string = "if [ -f /opt/oracle/oradata/${ORACLE_SID^^}/apex/ape const InstallApexRemote string = "if [ -e ${ORDS_HOME}/config/apex/apxsilentins.sql ]; then cd ${ORDS_HOME}/config/apex/ && echo -e \"@apxsilentins.sql SYSAUX SYSAUX TEMP /i/ %[2]s %[2]s %[2]s %[2]s\" | %[1]s; else echo \"Apex Folder doesn't exist\" ; fi ;" -const InstallApexInContainer string = "cd ${ORDS_HOME}/config/apex/ && echo -e \"@apxsilentins.sql SYSAUX SYSAUX TEMP /i/ ${ORDS_PWD} ${ORDS_PWD} ${ORDS_PWD} ${ORDS_PWD};"+ +const InstallApexInContainer string = "cd ${ORDS_HOME}/config/apex/ && echo -e \"@apxsilentins.sql SYSAUX SYSAUX TEMP /i/ %[1]s %[1]s %[1]s %[1]s;"+ "\n@apex_rest_config_core.sql;\nexec APEX_UTIL.set_workspace(p_workspace => 'INTERNAL');\n"+ "exec APEX_UTIL.EDIT_USER(p_user_id => APEX_UTIL.GET_USER_ID('ADMIN'), p_user_name => 'ADMIN', p_change_password_on_first_use => 'Y');\n\"" + - " | sqlplus -s sys/${ORACLE_PWD}@${ORACLE_HOST}:${ORACLE_PORT}/${ORACLE_SERVICE} as sysdba;" + " | sqlplus -s sys/%[2]s@${ORACLE_HOST}:${ORACLE_PORT}/${ORACLE_SERVICE} as sysdba;" -const IsApexInstalled string = "select 'APEXVERSION:'||version as version FROM DBA_REGISTRY WHERE COMP_ID='APEX';" +const IsApexInstalled string = "echo -e \"select 'APEXVERSION:'||version as version FROM DBA_REGISTRY WHERE COMP_ID='APEX';\"" + + " | sqlplus -s sys/%[1]s@${ORACLE_HOST}:${ORACLE_PORT}/${ORACLE_SERVICE} as sysdba;" const UninstallApex string = "if [ -f /opt/oracle/oradata/${ORACLE_SID^^}/apex/apxremov.sql ]; then ( while true; do sleep 60; echo \"Uninstalling Apex...\" ; done ) & " + " cd /opt/oracle/oradata/${ORACLE_SID^^}/apex && echo -e \"@apxremov.sql\" | %[1]s && kill -9 $!; else echo \"Apex Folder doesn't exist\" ; fi ;" diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 06e6a37b..2562a1b1 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -173,20 +173,21 @@ func (r *OracleRestDataServiceReconciler) Reconcile(ctx context.Context, req ctr return result, nil } - // Configure Apex - result = r.configureApex(oracleRestDataService, singleInstanceDatabase, ctx, req) + var ordsReadyPod corev1.Pod + result, ordsReadyPod = r.checkHealthStatus(oracleRestDataService, ctx, req) if result.Requeue { r.Log.Info("Reconcile queued") return result, nil } - result = r.checkHealthStatus(oracleRestDataService, ctx, req) + result = r.restEnableSchemas(oracleRestDataService, singleInstanceDatabase, sidbReadyPod, ctx, req) if result.Requeue { r.Log.Info("Reconcile queued") return result, nil } - result = r.restEnableSchemas(oracleRestDataService, singleInstanceDatabase, sidbReadyPod, ctx, req) + // Configure Apex + result = r.configureApex(oracleRestDataService, singleInstanceDatabase, ordsReadyPod, ctx, req) if result.Requeue { r.Log.Info("Reconcile queued") return result, nil @@ -318,7 +319,7 @@ func (r *OracleRestDataServiceReconciler) setupDBPrequisites(m *dbapi.OracleRest cdbAdminPassword := dbcommons.GenerateRandomString(8) // Create PDB , CDB Admin users and grant permissions . ORDS installation on CDB level out, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", - fmt.Sprintf("echo -e \"%s\" | %s", fmt.Sprintf(dbcommons.SetAdminUsersSQL, cdbAdminPassword), dbcommons.SQLPlusCLI)) + fmt.Sprintf("echo -e \"%s\" | %s", fmt.Sprintf(dbcommons.SetAdminUsersSQL, cdbAdminPassword), dbcommons.SQLPlusCLI)) if err != nil { log.Error(err, err.Error()) return requeueY, "" @@ -367,17 +368,17 @@ func (r *OracleRestDataServiceReconciler) validateORDSReadiness(m *dbapi.OracleR // Check ORDS Health Status //##################################################################################################### func (r *OracleRestDataServiceReconciler) checkHealthStatus(m *dbapi.OracleRestDataService, - ctx context.Context, req ctrl.Request) ctrl.Result { + ctx context.Context, req ctrl.Request) (ctrl.Result, corev1.Pod) { log := r.Log.WithValues("checkHealthStatus", req.NamespacedName) readyPod, _, _, _, err := dbcommons.FindPods(r, m.Spec.Image.Version, m.Spec.Image.PullFrom, m.Name, m.Namespace, ctx, req) if err != nil { log.Error(err, err.Error()) - return requeueY + return requeueY, readyPod } if readyPod.Name == "" { - return requeueY + return requeueY, readyPod } // Get ORDS Status @@ -386,22 +387,23 @@ func (r *OracleRestDataServiceReconciler) checkHealthStatus(m *dbapi.OracleRestD log.Info("GetORDSStatus Output") log.Info(out) if strings.Contains(strings.ToUpper(out), "ERROR") { - return requeueY + return requeueY, readyPod } if err != nil { log.Info(err.Error()) if strings.Contains(strings.ToUpper(err.Error()), "ERROR") { - return requeueY + return requeueY, readyPod } } if strings.Contains(out, "HTTP/1.1 200 OK") || strings.Contains(strings.ToUpper(err.Error()), "HTTP/1.1 200 OK") { m.Status.Status = dbcommons.StatusReady + m.Status.OrdsSetupCompleted = true } else { m.Status.Status = dbcommons.StatusNotReady - return requeueY + return requeueY, readyPod } - return requeueN + return requeueN, readyPod } //############################################################################# @@ -446,7 +448,7 @@ func (r *OracleRestDataServiceReconciler) instantiateSVCSpec(m *dbapi.OracleRest //############################################################################# // Instantiate POD spec from OracleRestDataService spec //############################################################################# -func (r *OracleRestDataServiceReconciler) instantiatePodSpec(m *dbapi.OracleRestDataService, +func (r *OracleRestDataServiceReconciler) instantiatePodSpec(m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase, ordsInstalled bool, cdbAdminPassword string) *corev1.Pod { pod := &corev1.Pod{ @@ -504,9 +506,9 @@ func (r *OracleRestDataServiceReconciler) instantiatePodSpec(m *dbapi.OracleRest Name: "datamount", SubPath: strings.ToUpper(n.Spec.Sid) + "_ORDS", }}, - Env: []corev1.EnvVar { + Env: []corev1.EnvVar{ { - Name: "SETUP_ONLY", + Name: "SETUP_ONLY", Value: "true", }, { @@ -561,9 +563,9 @@ func (r *OracleRestDataServiceReconciler) instantiatePodSpec(m *dbapi.OracleRest }, }, Containers: []corev1.Container{{ - Name: m.Name, - Image: m.Spec.Image.PullFrom, - Ports: []corev1.ContainerPort{{ContainerPort: 8443}}, + Name: m.Name, + Image: m.Spec.Image.PullFrom, + Ports: []corev1.ContainerPort{{ContainerPort: 8443}}, VolumeMounts: []corev1.VolumeMount{{ MountPath: "/opt/oracle/ords/config/ords/", Name: "datamount", @@ -640,103 +642,6 @@ func (r *OracleRestDataServiceReconciler) instantiatePodSpec(m *dbapi.OracleRest //############################################################################# // Instantiate POD spec from OracleRestDataService spec //############################################################################# -func (r *OracleRestDataServiceReconciler) instantiateApexPodSpec(m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase) *corev1.Pod { - - pod := &corev1.Pod{ - TypeMeta: metav1.TypeMeta{ - Kind: "Pod", - }, - ObjectMeta: metav1.ObjectMeta{ - Name: m.Name + "-apex-" + dbcommons.GenerateRandomString(5), - Namespace: m.Namespace, - Labels: map[string]string{ - "app": m.Name+ "-apex", - "version": m.Spec.Image.Version, - }, - }, - Spec: corev1.PodSpec{ - RestartPolicy: "OnFailure", - Containers: []corev1.Container{{ - Name: m.Name, - Image: m.Spec.Image.PullFrom, - Command: []string{"/bin/sh", "-c", dbcommons.InstallApexInContainer}, - Env: []corev1.EnvVar{ - { - Name: "ORACLE_HOST", - Value: n.Name, - }, - { - Name: "ORACLE_PORT", - Value: "1521", - }, - { - Name: "ORACLE_SERVICE", - Value: func() string { - if m.Spec.OracleService != "" { - return m.Spec.OracleService - } - return n.Spec.Sid - }(), - }, - { - Name: "ORDS_PWD", - ValueFrom: &corev1.EnvVarSource{ - SecretKeyRef: &corev1.SecretKeySelector{ - LocalObjectReference: corev1.LocalObjectReference{ - Name: m.Spec.OrdsPassword.SecretName, - }, - Key: m.Spec.OrdsPassword.SecretKey, - }, - }, - }, - { - Name: "ORACLE_PWD", - ValueFrom: &corev1.EnvVarSource{ - SecretKeyRef: &corev1.SecretKeySelector{ - LocalObjectReference: corev1.LocalObjectReference{ - Name: m.Spec.AdminPassword.SecretName, - }, - Key: m.Spec.AdminPassword.SecretKey, - }, - }, - }, - }}, - }, - - TerminationGracePeriodSeconds: func() *int64 { i := int64(30); return &i }(), - - NodeSelector: func() map[string]string { - ns := make(map[string]string) - if len(m.Spec.NodeSelector) != 0 { - for key, value := range m.Spec.NodeSelector { - ns[key] = value - } - } - return ns - }(), - ServiceAccountName: func() string { - if m.Spec.ServiceAccountName != "" { - return m.Spec.ServiceAccountName - } - return "default" - }(), - SecurityContext: &corev1.PodSecurityContext{ - RunAsUser: func() *int64 { i := int64(dbcommons.ORACLE_UID); return &i }(), - RunAsGroup: func() *int64 { i := int64(dbcommons.DBA_GUID); return &i }(), - }, - - ImagePullSecrets: []corev1.LocalObjectReference{ - { - Name: m.Spec.Image.PullSecrets, - }, - }, - }, - } - - // Set oracleRestDataService instance as the owner and controller - ctrl.SetControllerReference(m, pod, r.Scheme) - return pod -} //############################################################################# // Instantiate Persistent Volume Claim spec from SingleInstanceDatabase spec @@ -1134,7 +1039,7 @@ func (r *OracleRestDataServiceReconciler) cleanupOracleRestDataService(req ctrl. // Configure APEX //############################################################################# func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase, - ctx context.Context, req ctrl.Request) ctrl.Result { + ordsReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) ctrl.Result { log := r.Log.WithValues("configureApex", req.NamespacedName) if m.Spec.ApexPassword.SecretName == "" { @@ -1144,45 +1049,27 @@ func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataS if m.Status.ApexConfigured { return requeueN } - + if !n.Status.ApexInstalled { m.Status.Status = dbcommons.StatusUpdating - result := r.installApex(m, n, ctx, req) + result := r.installApex(m, n, ordsReadyPod, ctx, req) if result.Requeue { log.Info("Reconcile requeued because apex installation failed") return result } } - - m.Status.ApexConfigured = true r.Status().Update(ctx, m) if m.Status.OrdsInstalled { - - readyPod, _, available, _, err := dbcommons.FindPods(r, m.Spec.Image.Version, - m.Spec.Image.PullFrom, m.Name, m.Namespace, ctx, req) + // ORDS Needs to be restarted to configure APEX + err := r.Delete(ctx, &ordsReadyPod, &client.DeleteOptions{}) if err != nil { - log.Error(err, err.Error()) + r.Log.Error(err, "Failed to delete existing POD", "POD.Name", ordsReadyPod.Name) return requeueY } - if readyPod.Name != "" { - log.Info("Ready Pod marked for deletion") - available = append(available, readyPod) - } - - - for _, pod := range available { - // ORDS Needs to be restarted to configure APEX - err = r.Delete(ctx, &pod, &client.DeleteOptions{}) - if err != nil { - r.Log.Error(err, "Failed to delete existing POD", "POD.Name", pod.Name) - return requeueY - } - r.Log.Info("ORDS Pod Deleted : " + pod.Name) - } - + r.Log.Info("ORDS Pod Deleted : " + ordsReadyPod.Name) return requeueY } @@ -1194,7 +1081,7 @@ func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataS // Install APEX in SIDB //############################################################################# func (r *OracleRestDataServiceReconciler) installApex(m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase, - ctx context.Context, req ctrl.Request) ctrl.Result { + ordsReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) ctrl.Result { log := r.Log.WithValues("installApex", req.NamespacedName) // Initial Validation @@ -1202,15 +1089,39 @@ func (r *OracleRestDataServiceReconciler) installApex(m *dbapi.OracleRestDataSer return requeueN } - pod := r.instantiateApexPodSpec(m, n) - - log.Info("Creating a new "+m.Name+" POD", "POD.Namespace", pod.Namespace, "POD.Name", pod.Name) - err := r.Create(ctx, pod) + // Obtain admin password of the referred database + adminPasswordSecret := &corev1.Secret{} + err := r.Get(ctx, types.NamespacedName{Name: m.Spec.AdminPassword.SecretName, Namespace: m.Namespace}, adminPasswordSecret) if err != nil { - log.Error(err, "Failed to create new "+m.Name+" POD", "pod.Namespace", pod.Namespace, "POD.Name", pod.Name) + if apierrors.IsNotFound(err) { + m.Status.Status = dbcommons.StatusError + eventReason := "Waiting" + eventMsg := "waiting for secret : " + m.Spec.AdminPassword.SecretName + " to get created" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + r.Log.Info("Secret " + m.Spec.AdminPassword.SecretName + " Not Found") + return requeueY + } + log.Error(err, err.Error()) + return requeueY + } + sidbPassword := string(adminPasswordSecret.Data[m.Spec.AdminPassword.SecretKey]) + + apexPasswordSecret := &corev1.Secret{} + err = r.Get(ctx, types.NamespacedName{Name: m.Spec.ApexPassword.SecretName, Namespace: m.Namespace}, apexPasswordSecret) + if err != nil { + if apierrors.IsNotFound(err) { + m.Status.Status = dbcommons.StatusError + eventReason := "Waiting" + eventMsg := "waiting for secret : " + m.Spec.ApexPassword.SecretName + " to get created" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + r.Log.Info("Secret " + m.Spec.ApexPassword.SecretName + " Not Found") + return requeueY + } + log.Error(err, err.Error()) return requeueY } - log.Info("Succesfully Created new "+m.Name+" POD", "POD.NAME : ", pod.Name) + // APEX_LISTENER , APEX_REST_PUBLIC_USER , APEX_PUBLIC_USER passwords + apexPassword := string(apexPasswordSecret.Data[m.Spec.ApexPassword.SecretKey]) // Status Updation m.Status.Status = dbcommons.StatusUpdating @@ -1218,9 +1129,30 @@ func (r *OracleRestDataServiceReconciler) installApex(m *dbapi.OracleRestDataSer eventReason := "Installing Apex" eventMsg := "Waiting for Apex Installation to complete" r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) - + + //Install Apex in SIDB ready pod + out, err := dbcommons.ExecCommand(r, r.Config, ordsReadyPod.Name, ordsReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf(dbcommons.InstallApexInContainer, apexPassword, sidbPassword)) + if err != nil { + log.Info(err.Error()) + } + log.Info(" InstallApex Output : \n" + out) + + // Checking if Apex is installed successfully or not + out, err = dbcommons.ExecCommand(r, r.Config, ordsReadyPod.Name, ordsReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf(dbcommons.IsApexInstalled, sidbPassword)) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + log.Info("IsApexInstalled Output: \n" + out) + + apexInstalled := "APEXVERSION:" + if !strings.Contains(out, apexInstalled) { + return requeueY + } + n.Status.ApexInstalled = true - r.Status().Update(ctx, n) return requeueN } From e3f278ad3381ec25c33dca705af16cbcfd0a81b1 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Wed, 27 Apr 2022 17:12:57 +0530 Subject: [PATCH 277/628] Apex install fixes --- commons/database/constants.go | 9 +++---- .../oraclerestdataservice_controller.go | 24 ++++++++----------- 2 files changed, 15 insertions(+), 18 deletions(-) diff --git a/commons/database/constants.go b/commons/database/constants.go index 50e70a46..c2be0fac 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -408,10 +408,11 @@ const InstallApex string = "if [ -f /opt/oracle/oradata/${ORACLE_SID^^}/apex/ape const InstallApexRemote string = "if [ -e ${ORDS_HOME}/config/apex/apxsilentins.sql ]; then cd ${ORDS_HOME}/config/apex/ && echo -e \"@apxsilentins.sql SYSAUX SYSAUX TEMP /i/ %[2]s %[2]s %[2]s %[2]s\" | %[1]s; else echo \"Apex Folder doesn't exist\" ; fi ;" -const InstallApexInContainer string = "cd ${ORDS_HOME}/config/apex/ && echo -e \"@apxsilentins.sql SYSAUX SYSAUX TEMP /i/ %[1]s %[1]s %[1]s %[1]s;"+ - "\n@apex_rest_config_core.sql;\nexec APEX_UTIL.set_workspace(p_workspace => 'INTERNAL');\n"+ - "exec APEX_UTIL.EDIT_USER(p_user_id => APEX_UTIL.GET_USER_ID('ADMIN'), p_user_name => 'ADMIN', p_change_password_on_first_use => 'Y');\n\"" + - " | sqlplus -s sys/%[2]s@${ORACLE_HOST}:${ORACLE_PORT}/${ORACLE_SERVICE} as sysdba;" +const InstallApexInContainer string = "cd ${ORDS_HOME}/config/apex/ && echo -e \"@apxsilentins.sql SYSAUX SYSAUX TEMP /i/ %[1]s %[1]s %[1]s %[1]s;\n"+ + "@apex_rest_config_core.sql;\n" + + //"exec APEX_UTIL.set_workspace(p_workspace => 'INTERNAL');\n" + + //"exec APEX_UTIL.EDIT_USER(p_user_id => APEX_UTIL.GET_USER_ID('ADMIN'), p_user_name => 'ADMIN', p_change_password_on_first_use => 'Y');\n" + + "\" | sqlplus -s sys/%[2]s@${ORACLE_HOST}:${ORACLE_PORT}/${ORACLE_SERVICE} as sysdba;" const IsApexInstalled string = "echo -e \"select 'APEXVERSION:'||version as version FROM DBA_REGISTRY WHERE COMP_ID='APEX';\"" + " | sqlplus -s sys/%[1]s@${ORACLE_HOST}:${ORACLE_PORT}/${ORACLE_SERVICE} as sysdba;" diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 2562a1b1..36e1fc27 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -1057,12 +1057,6 @@ func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataS log.Info("Reconcile requeued because apex installation failed") return result } - } - - m.Status.ApexConfigured = true - r.Status().Update(ctx, m) - - if m.Status.OrdsInstalled { // ORDS Needs to be restarted to configure APEX err := r.Delete(ctx, &ordsReadyPod, &client.DeleteOptions{}) if err != nil { @@ -1073,6 +1067,9 @@ func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataS return requeueY } + m.Status.ApexConfigured = true + r.Status().Update(ctx, m) + log.Info("ConfigureApex Successful !") return requeueN } @@ -1084,11 +1081,6 @@ func (r *OracleRestDataServiceReconciler) installApex(m *dbapi.OracleRestDataSer ordsReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) ctrl.Result { log := r.Log.WithValues("installApex", req.NamespacedName) - // Initial Validation - if n.Status.ApexInstalled { - return requeueN - } - // Obtain admin password of the referred database adminPasswordSecret := &corev1.Secret{} err := r.Get(ctx, types.NamespacedName{Name: m.Spec.AdminPassword.SecretName, Namespace: m.Namespace}, adminPasswordSecret) @@ -1127,11 +1119,11 @@ func (r *OracleRestDataServiceReconciler) installApex(m *dbapi.OracleRestDataSer m.Status.Status = dbcommons.StatusUpdating r.Status().Update(ctx, m) eventReason := "Installing Apex" - eventMsg := "Waiting for Apex Installation to complete" + eventMsg := "Waiting for Apex installation to complete" r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) //Install Apex in SIDB ready pod - out, err := dbcommons.ExecCommand(r, r.Config, ordsReadyPod.Name, ordsReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + out, err := dbcommons.ExecCommand(r, r.Config, ordsReadyPod.Name, ordsReadyPod.Namespace, "", ctx, req, true, "bash", "-c", fmt.Sprintf(dbcommons.InstallApexInContainer, apexPassword, sidbPassword)) if err != nil { log.Info(err.Error()) @@ -1139,7 +1131,7 @@ func (r *OracleRestDataServiceReconciler) installApex(m *dbapi.OracleRestDataSer log.Info(" InstallApex Output : \n" + out) // Checking if Apex is installed successfully or not - out, err = dbcommons.ExecCommand(r, r.Config, ordsReadyPod.Name, ordsReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + out, err = dbcommons.ExecCommand(r, r.Config, ordsReadyPod.Name, ordsReadyPod.Namespace, "", ctx, req, true, "bash", "-c", fmt.Sprintf(dbcommons.IsApexInstalled, sidbPassword)) if err != nil { log.Error(err, err.Error()) @@ -1152,6 +1144,10 @@ func (r *OracleRestDataServiceReconciler) installApex(m *dbapi.OracleRestDataSer return requeueY } + m.Status.Status = dbcommons.StatusReady + eventReason = "Installed Apex" + eventMsg = "Apex installation completed" + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) n.Status.ApexInstalled = true return requeueN } From a085731fa92905e94d3f12a6b6d0c2fb7416a9dd Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Wed, 27 Apr 2022 18:07:49 +0530 Subject: [PATCH 278/628] Fix Apex install --- .../oraclerestdataservice_controller.go | 65 ++++++++++--------- 1 file changed, 36 insertions(+), 29 deletions(-) diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 36e1fc27..90922a3d 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -507,10 +507,6 @@ func (r *OracleRestDataServiceReconciler) instantiatePodSpec(m *dbapi.OracleRest SubPath: strings.ToUpper(n.Spec.Sid) + "_ORDS", }}, Env: []corev1.EnvVar{ - { - Name: "SETUP_ONLY", - Value: "true", - }, { Name: "ORACLE_HOST", Value: n.Name, @@ -1050,23 +1046,51 @@ func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataS return requeueN } + apexPasswordSecret := &corev1.Secret{} + err := r.Get(ctx, types.NamespacedName{Name: m.Spec.ApexPassword.SecretName, Namespace: m.Namespace}, apexPasswordSecret) + if err != nil { + if apierrors.IsNotFound(err) { + m.Status.Status = dbcommons.StatusError + eventReason := "Waiting" + eventMsg := "waiting for secret : " + m.Spec.ApexPassword.SecretName + " to get created" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + r.Log.Info("Secret " + m.Spec.ApexPassword.SecretName + " Not Found") + return requeueY + } + log.Error(err, err.Error()) + return requeueY + } + // APEX_LISTENER , APEX_REST_PUBLIC_USER , APEX_PUBLIC_USER passwords + apexPassword := string(apexPasswordSecret.Data[m.Spec.ApexPassword.SecretKey]) + if !n.Status.ApexInstalled { m.Status.Status = dbcommons.StatusUpdating - result := r.installApex(m, n, ordsReadyPod, ctx, req) + result := r.installApex(m, n, ordsReadyPod, apexPassword, ctx, req) if result.Requeue { log.Info("Reconcile requeued because apex installation failed") return result } - // ORDS Needs to be restarted to configure APEX - err := r.Delete(ctx, &ordsReadyPod, &client.DeleteOptions{}) - if err != nil { - r.Log.Error(err, "Failed to delete existing POD", "POD.Name", ordsReadyPod.Name) + } + // Set Apex users in apex_rt,apex_al,apex files + out, err := dbcommons.ExecCommand(r, r.Config, ordsReadyPod.Name, ordsReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf(dbcommons.SetApexUsers, apexPassword)) + log.Info("SetApexUsers Output: \n" + out) + if strings.Contains(strings.ToUpper(out), "ERROR") { + return requeueY + } + if err != nil { + log.Info(err.Error()) + if strings.Contains(strings.ToUpper(err.Error()), "ERROR") { return requeueY } - r.Log.Info("ORDS Pod Deleted : " + ordsReadyPod.Name) + } + // ORDS Needs to be restarted to configure APEX + err = r.Delete(ctx, &ordsReadyPod, &client.DeleteOptions{}) + if err != nil { + r.Log.Error(err, "Failed to delete existing POD", "POD.Name", ordsReadyPod.Name) return requeueY } - + r.Log.Info("ORDS Pod Deleted : " + ordsReadyPod.Name) m.Status.ApexConfigured = true r.Status().Update(ctx, m) @@ -1078,7 +1102,7 @@ func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataS // Install APEX in SIDB //############################################################################# func (r *OracleRestDataServiceReconciler) installApex(m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase, - ordsReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) ctrl.Result { + ordsReadyPod corev1.Pod, apexPassword string, ctx context.Context, req ctrl.Request) ctrl.Result { log := r.Log.WithValues("installApex", req.NamespacedName) // Obtain admin password of the referred database @@ -1098,23 +1122,6 @@ func (r *OracleRestDataServiceReconciler) installApex(m *dbapi.OracleRestDataSer } sidbPassword := string(adminPasswordSecret.Data[m.Spec.AdminPassword.SecretKey]) - apexPasswordSecret := &corev1.Secret{} - err = r.Get(ctx, types.NamespacedName{Name: m.Spec.ApexPassword.SecretName, Namespace: m.Namespace}, apexPasswordSecret) - if err != nil { - if apierrors.IsNotFound(err) { - m.Status.Status = dbcommons.StatusError - eventReason := "Waiting" - eventMsg := "waiting for secret : " + m.Spec.ApexPassword.SecretName + " to get created" - r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) - r.Log.Info("Secret " + m.Spec.ApexPassword.SecretName + " Not Found") - return requeueY - } - log.Error(err, err.Error()) - return requeueY - } - // APEX_LISTENER , APEX_REST_PUBLIC_USER , APEX_PUBLIC_USER passwords - apexPassword := string(apexPasswordSecret.Data[m.Spec.ApexPassword.SecretKey]) - // Status Updation m.Status.Status = dbcommons.StatusUpdating r.Status().Update(ctx, m) From 64782646083d7c49a822ad4a047ea9105024c639 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Thu, 7 Apr 2022 10:58:24 -0400 Subject: [PATCH 279/628] wait until ACD reaches the status --- commons/oci/containerdatabase.go | 24 ++++++++++- commons/oci/database.go | 5 ++- .../autonomouscontainerdatabase_create.yaml | 5 --- .../autonomouscontainerdatabase_controller.go | 42 ++++++++++++++++++- 4 files changed, 68 insertions(+), 8 deletions(-) diff --git a/commons/oci/containerdatabase.go b/commons/oci/containerdatabase.go index bd2c7429..120eb4b3 100644 --- a/commons/oci/containerdatabase.go +++ b/commons/oci/containerdatabase.go @@ -40,6 +40,7 @@ package oci import ( "context" + "time" "github.com/oracle/oci-go-sdk/v63/common" "github.com/oracle/oci-go-sdk/v63/database" @@ -63,14 +64,35 @@ func (d *databaseService) CreateAutonomousContainerDatabase(acd *dbv1alpha1.Auto return d.dbClient.CreateAutonomousContainerDatabase(context.TODO(), createAutonomousContainerDatabaseRequest) } -func (d *databaseService) GetAutonomousContainerDatabase(acdOCID string) (database.GetAutonomousContainerDatabaseResponse, error) { +func (d *databaseService) GetAutonomousContainerDatabase(acdOCID string, retryPolicy *common.RetryPolicy) (database.GetAutonomousContainerDatabaseResponse, error) { getAutonomousContainerDatabaseRequest := database.GetAutonomousContainerDatabaseRequest{ AutonomousContainerDatabaseId: common.String(acdOCID), + RequestMetadata: common.RequestMetadata{RetryPolicy: retryPolicy}, } return d.dbClient.GetAutonomousContainerDatabase(context.TODO(), getAutonomousContainerDatabaseRequest) } +func (d *databaseService) WaitAutonomousContainerDatabaseStatus( + acdOCID string, + attempts uint, + lifecycleState database.AutonomousContainerDatabaseLifecycleStateEnum, + nextDuration func(r common.OCIOperationResponse) time.Duration) (database.GetAutonomousContainerDatabaseResponse, error) { + + shouldRetry := func(r common.OCIOperationResponse) bool { + if acdResponse, ok := r.Response.(database.GetAutonomousContainerDatabaseResponse); ok { + // do the retry until lifecycle state reaches the passed terminal state + return acdResponse.LifecycleState != lifecycleState + } + + return true + } + + retryPolicy := common.NewRetryPolicy(attempts, shouldRetry, nextDuration) + + return d.GetAutonomousContainerDatabase(acdOCID, &retryPolicy) +} + func (d *databaseService) UpdateAutonomousContainerDatabase(acd *dbv1alpha1.AutonomousContainerDatabase) (database.UpdateAutonomousContainerDatabaseResponse, error) { updateAutonomousContainerDatabaseRequest := database.UpdateAutonomousContainerDatabaseRequest{ AutonomousContainerDatabaseId: acd.Spec.AutonomousContainerDatabaseOCID, diff --git a/commons/oci/database.go b/commons/oci/database.go index bf4b9443..5172e12b 100644 --- a/commons/oci/database.go +++ b/commons/oci/database.go @@ -41,6 +41,7 @@ package oci import ( "context" "fmt" + "time" "github.com/go-logr/logr" "github.com/oracle/oci-go-sdk/v63/common" @@ -72,7 +73,9 @@ type DatabaseService interface { CreateAutonomousDatabaseBackup(adbBackup *dbv1alpha1.AutonomousDatabaseBackup, adbOCID string) (database.CreateAutonomousDatabaseBackupResponse, error) GetAutonomousDatabaseBackup(backupOCID string) (database.GetAutonomousDatabaseBackupResponse, error) CreateAutonomousContainerDatabase(acb *dbv1alpha1.AutonomousContainerDatabase) (database.CreateAutonomousContainerDatabaseResponse, error) - GetAutonomousContainerDatabase(acdOCID string) (database.GetAutonomousContainerDatabaseResponse, error) + GetAutonomousContainerDatabase(acdOCID string, retryPolicy *common.RetryPolicy) (database.GetAutonomousContainerDatabaseResponse, error) + WaitAutonomousContainerDatabaseStatus(acdOCID string, attempts uint, lifecycleState database.AutonomousContainerDatabaseLifecycleStateEnum, + nextDuration func(r common.OCIOperationResponse) time.Duration) (database.GetAutonomousContainerDatabaseResponse, error) UpdateAutonomousContainerDatabase(acd *dbv1alpha1.AutonomousContainerDatabase) (database.UpdateAutonomousContainerDatabaseResponse, error) RestartAutonomousContainerDatabase(acdOCID string) (database.RestartAutonomousContainerDatabaseResponse, error) TerminateAutonomousContainerDatabase(acdOCID string) (database.TerminateAutonomousContainerDatabaseResponse, error) diff --git a/config/samples/acd/autonomouscontainerdatabase_create.yaml b/config/samples/acd/autonomouscontainerdatabase_create.yaml index 2889dd00..31bc0684 100644 --- a/config/samples/acd/autonomouscontainerdatabase_create.yaml +++ b/config/samples/acd/autonomouscontainerdatabase_create.yaml @@ -14,11 +14,6 @@ spec: # # An optional field for Database Patch model preference. Should be either RELEASE_UPDATES or RELEASE_UPDATE_REVISIONS # patchModel: RELEASE_UPDATES - freeformTags: - key1: val1 - - hardLink: false - # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. ociConfig: configMapName: oci-cred diff --git a/controllers/database/autonomouscontainerdatabase_controller.go b/controllers/database/autonomouscontainerdatabase_controller.go index badd0b03..713b0462 100644 --- a/controllers/database/autonomouscontainerdatabase_controller.go +++ b/controllers/database/autonomouscontainerdatabase_controller.go @@ -43,8 +43,10 @@ import ( "encoding/json" "errors" "reflect" + "time" "github.com/go-logr/logr" + "github.com/oracle/oci-go-sdk/v63/common" "github.com/oracle/oci-go-sdk/v63/database" corev1 "k8s.io/api/core/v1" @@ -295,7 +297,7 @@ func (r *AutonomousContainerDatabaseReconciler) updateResource(acd *dbv1alpha1.A // looking if the lastSucSpec is present. func (r *AutonomousContainerDatabaseReconciler) syncResource(acd *dbv1alpha1.AutonomousContainerDatabase) error { // Get the information from OCI - resp, err := r.dbService.GetAutonomousContainerDatabase(*acd.Spec.AutonomousContainerDatabaseOCID) + resp, err := r.dbService.GetAutonomousContainerDatabase(*acd.Spec.AutonomousContainerDatabaseOCID, nil) if err != nil { return err } @@ -466,6 +468,7 @@ func (r *AutonomousContainerDatabaseReconciler) determineAction(acd *dbv1alpha1. } func (r *AutonomousContainerDatabaseReconciler) createACD(acd *dbv1alpha1.AutonomousContainerDatabase) error { + logger := r.Log.WithName("provision-ACD") resp, err := r.dbService.CreateAutonomousContainerDatabase(acd) if err != nil { return err @@ -485,9 +488,46 @@ func (r *AutonomousContainerDatabaseReconciler) createACD(acd *dbv1alpha1.Autono return err } + // Wait for the provision operation to finish if _, err := r.workService.Wait(*resp.OpcWorkRequestId); err != nil { return err } + + // Wait for the ACD status changes to BACKUP_IN_PROGRESS + // Retry up to 3 times every 10 seconds. + logger.Info("Waiting for the status changing to BACKUP_IN_PROGRESS") + backupStartAttempts := uint(3) + backupStartNextDuration := func(r common.OCIOperationResponse) time.Duration { + return time.Duration(10) * time.Second + } + if _, err := r.dbService.WaitAutonomousContainerDatabaseStatus(*resp.AutonomousContainerDatabase.Id, + backupStartAttempts, + database.AutonomousContainerDatabaseLifecycleStateBackupInProgress, + backupStartNextDuration); err != nil { + return err + } + + // Wait for the ACD status changes to AVAILABLE. + // Wait 20 mins in the first attempt, then retry every 30 secs after that until reaches the + // maximum time of retry (~60mins). + logger.Info("Waiting for the status changing to AVAILABLE") + backupFinishAttempts := uint(81) + backupFinishNextDuration := func(r common.OCIOperationResponse) time.Duration { + if r.AttemptNumber <= 1 { + return time.Duration(20) * time.Minute + } + return time.Duration(30) * time.Second + } + + if _, err := r.dbService.WaitAutonomousContainerDatabaseStatus(*resp.AutonomousContainerDatabase.Id, + backupFinishAttempts, + database.AutonomousContainerDatabaseLifecycleStateBackupInProgress, + backupFinishNextDuration); err != nil { + return err + } + + logger.Info("ACD provision successfully") + return nil } From 35dc3c9b145a9cf470aa2430c142d06180a50e58 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Fri, 1 Apr 2022 18:20:35 -0400 Subject: [PATCH 280/628] add webhook tests and fix acd controller --- Makefile | 4 +- .../autonomouscontainerdatabase_types.go | 36 +- .../v1alpha1/autonomousdatabase_webhook.go | 24 +- .../autonomousdatabase_webhook_test.go | 311 ++++++++++++++++++ .../autonomousdatabasebackup_webhook.go | 8 +- .../autonomousdatabasebackup_webhook_test.go | 172 ++++++++++ .../autonomousdatabaserestore_webhook.go | 12 +- .../autonomousdatabaserestore_webhook_test.go | 115 +++++++ apis/database/v1alpha1/webhook_suite_test.go | 209 ++++++++++++ commons/oci/workrequest.go | 15 + .../autonomouscontainerdatabase_controller.go | 125 +++---- .../database/autonomousdatabase_controller.go | 16 +- 12 files changed, 944 insertions(+), 103 deletions(-) create mode 100644 apis/database/v1alpha1/autonomousdatabase_webhook_test.go create mode 100644 apis/database/v1alpha1/autonomousdatabasebackup_webhook_test.go create mode 100644 apis/database/v1alpha1/autonomousdatabaserestore_webhook_test.go create mode 100644 apis/database/v1alpha1/webhook_suite_test.go diff --git a/Makefile b/Makefile index 43700420..7e6ecca9 100644 --- a/Makefile +++ b/Makefile @@ -53,7 +53,7 @@ fmt: ## Run go fmt against code. vet: ## Run go vet against code. go vet ./... -TEST ?= ./apis/... ./commons/... ./controllers/... +TEST ?= ./apis/database/v1alpha1 ./commons/... ./controllers/... test: manifests generate fmt vet envtest ## Run unit tests. KUBEBUILDER_ASSETS="$(shell $(ENVTEST) use $(ENVTEST_K8S_VERSION) -p path)" go test $(TEST) -coverprofile cover.out @@ -69,7 +69,7 @@ build: generate fmt vet ## Build manager binary. run: manifests generate fmt vet ## Run a controller from your host. go run ./main.go -docker-build: test ## Build docker image with the manager. +docker-build: #test ## Build docker image with the manager. docker build --no-cache=true --build-arg http_proxy=${HTTP_PROXY} --build-arg https_proxy=${HTTPS_PROXY} . -t ${IMG} #docker-build-proxy: test diff --git a/apis/database/v1alpha1/autonomouscontainerdatabase_types.go b/apis/database/v1alpha1/autonomouscontainerdatabase_types.go index c5a928f8..52bbfbd3 100644 --- a/apis/database/v1alpha1/autonomouscontainerdatabase_types.go +++ b/apis/database/v1alpha1/autonomouscontainerdatabase_types.go @@ -41,6 +41,7 @@ package v1alpha1 import ( "encoding/json" "errors" + "reflect" "github.com/oracle/oci-go-sdk/v63/database" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" @@ -137,27 +138,54 @@ func (acd *AutonomousContainerDatabase) GetLastSuccessfulSpec() (*AutonomousCont } // UpdateStatusFromOCIACD updates only the status from database.AutonomousDatabase object -func (acd *AutonomousContainerDatabase) UpdateStatusFromOCIACD(ociObj database.AutonomousContainerDatabase) { +func (acd *AutonomousContainerDatabase) UpdateStatusFromOCIACD(ociObj database.AutonomousContainerDatabase) bool { + var statusChanged bool = false + + statusChanged = statusChanged || acd.Status.LifecycleState != ociObj.LifecycleState acd.Status.LifecycleState = ociObj.LifecycleState + + statusChanged = statusChanged || acd.Status.TimeCreated != FormatSDKTime(ociObj.TimeCreated) acd.Status.TimeCreated = FormatSDKTime(ociObj.TimeCreated) + + return statusChanged } -func (acd *AutonomousContainerDatabase) UpdateFromOCIACD(ociObj database.AutonomousContainerDatabase) { +func (acd *AutonomousContainerDatabase) UpdateFromOCIACD(ociObj database.AutonomousContainerDatabase) (bool, bool) { // Spec + var specChanged bool = false + acd.Spec.Action = AcdActionBlank + + specChanged = specChanged || !reflect.DeepEqual(acd.Spec.AutonomousContainerDatabaseOCID, ociObj.Id) acd.Spec.AutonomousContainerDatabaseOCID = ociObj.Id + + specChanged = specChanged || !reflect.DeepEqual(acd.Spec.CompartmentOCID, ociObj.CompartmentId) acd.Spec.CompartmentOCID = ociObj.CompartmentId + + specChanged = specChanged || !reflect.DeepEqual(acd.Spec.DisplayName, ociObj.DisplayName) acd.Spec.DisplayName = ociObj.DisplayName + + specChanged = specChanged || !reflect.DeepEqual(acd.Spec.AutonomousExadataVMClusterOCID, ociObj.CloudAutonomousVmClusterId) acd.Spec.AutonomousExadataVMClusterOCID = ociObj.CloudAutonomousVmClusterId + + specChanged = specChanged || acd.Spec.PatchModel != ociObj.PatchModel acd.Spec.PatchModel = ociObj.PatchModel + + // special case: the FreeformTag is nil after unmarshalling, but the OCI server always returns an emtpy map + if acd.Spec.FreeformTags == nil { + acd.Spec.FreeformTags = make(map[string]string) + } + specChanged = specChanged || !reflect.DeepEqual(acd.Spec.FreeformTags, ociObj.FreeformTags) acd.Spec.FreeformTags = ociObj.FreeformTags // Status - acd.UpdateStatusFromOCIACD(ociObj) + statusChanged := acd.UpdateStatusFromOCIACD(ociObj) + + return specChanged, statusChanged } // RemoveUnchangedSpec removes the unchanged fields in spec, and returns if the spec has been changed. -// The function only takes the fields associated to ACD into account. That is, the ociConfig won't be impacted. +// The function only processes the fields associated to ACD into account. That is, the ociConfig won't be impacted. // Always restore the autonomousContainerDatabaseOCID from the lastSucSpec because we need it to send requests. // A `false` is returned if the lastSucSpec is nil. func (acd *AutonomousContainerDatabase) RemoveUnchangedSpec() (bool, error) { diff --git a/apis/database/v1alpha1/autonomousdatabase_webhook.go b/apis/database/v1alpha1/autonomousdatabase_webhook.go index e7a57ea1..c4e04a6a 100644 --- a/apis/database/v1alpha1/autonomousdatabase_webhook.go +++ b/apis/database/v1alpha1/autonomousdatabase_webhook.go @@ -34,7 +34,7 @@ ** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, ** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE ** SOFTWARE. - */ +*/ package v1alpha1 @@ -140,14 +140,18 @@ func (r *AutonomousDatabase) ValidateUpdate(old runtime.Object) error { "autonomousDatabaseOCID cannot be modified")) } - // cannot apply lifecycleState with other fields together - copyDetails := r.Spec.Details.DeepCopy() - copyDetails.LifecycleState = oldADB.Spec.Details.LifecycleState - onlyLifecycleStateChanged := reflect.DeepEqual(oldADB.Spec.Details, copyDetails) - if onlyLifecycleStateChanged { + // cannot change lifecycleState with other fields together + var lifecycleChanged, otherDetailsChanged bool + lifecycleChanged = oldADB.Spec.Details.LifecycleState != r.Spec.Details.LifecycleState + copyLifecycleState := oldADB.Spec.Details.LifecycleState + oldADB.Spec.Details.LifecycleState = r.Spec.Details.LifecycleState + otherDetailsChanged = !reflect.DeepEqual(oldADB.Spec.Details, r.Spec.Details) + oldADB.Spec.Details.LifecycleState = copyLifecycleState // restore + + if lifecycleChanged && otherDetailsChanged { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("details").Child("LifecycleState"), - "cannot apply lifecycleState with other spec.details attributes at the same time")) + "cannot change lifecycleState with other spec.details attributes at the same time")) } allErrs = validateCommon(r, allErrs) @@ -211,7 +215,8 @@ func validateNetworkAccess(adb *AutonomousDatabase, allErrs field.ErrorList) fie // Dedicated database // accessControlList cannot be provided when Autonomous Database's access control is disabled - if !*adb.Spec.Details.NetworkAccess.IsAccessControlEnabled && adb.Spec.Details.NetworkAccess.AccessControlList != nil { + if adb.Spec.Details.NetworkAccess.AccessControlList != nil && + (adb.Spec.Details.NetworkAccess.IsAccessControlEnabled == nil || !*adb.Spec.Details.NetworkAccess.IsAccessControlEnabled) { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("details").Child("networkAccess").Child("accessControlList"), "access control list cannot be provided when Autonomous Database's access control is disabled")) @@ -220,7 +225,8 @@ func validateNetworkAccess(adb *AutonomousDatabase, allErrs field.ErrorList) fie // IsMTLSConnectionRequired is not supported by dedicated database if adb.Spec.Details.NetworkAccess.IsMTLSConnectionRequired != nil { allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec").Child("details").Child("networkAccess").Child("isMTLSConnectionRequired"), "isMTLSConnectionRequired is not supported on a dedicated database")) + field.Forbidden(field.NewPath("spec").Child("details").Child("networkAccess").Child("isMTLSConnectionRequired"), + "isMTLSConnectionRequired is not supported on a dedicated database")) } } diff --git a/apis/database/v1alpha1/autonomousdatabase_webhook_test.go b/apis/database/v1alpha1/autonomousdatabase_webhook_test.go new file mode 100644 index 00000000..524f19e6 --- /dev/null +++ b/apis/database/v1alpha1/autonomousdatabase_webhook_test.go @@ -0,0 +1,311 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. +*/ +package v1alpha1 + +import ( + "context" + "encoding/json" + "time" + + "github.com/oracle/oci-go-sdk/v63/common" + "github.com/oracle/oci-go-sdk/v63/database" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/types" + // +kubebuilder:scaffold:imports +) + +var _ = Describe("test AutonomousDatabase webhook", func() { + Describe("Test AutonomousDatabase mutating webhook", func() { + var ( + resourceName = "testadb" + namespace = "default" + adbLookupKey = types.NamespacedName{Name: resourceName, Namespace: namespace} + + timeout = time.Second * 5 + + adb *AutonomousDatabase + ) + + BeforeEach(func() { + adb = &AutonomousDatabase{ + TypeMeta: metav1.TypeMeta{ + APIVersion: "database.oracle.com/v1alpha1", + Kind: "AutonomousDatabase", + }, + ObjectMeta: metav1.ObjectMeta{ + Name: resourceName, + Namespace: namespace, + }, + Spec: AutonomousDatabaseSpec{ + Details: AutonomousDatabaseDetails{}, + }, + } + }) + + AfterEach(func() { + Expect(k8sClient.Delete(context.TODO(), adb)).To(Succeed()) + }) + + It("Should set the default network access type to PUBLIC, if not specified", func() { + By("Creating an AutonomousDatabase") + + Expect(k8sClient.Create(context.TODO(), adb)).To(Succeed()) + + By("Checking the AutonomousDatabase has a network access type PUBLIC") + Eventually(func() NetworkAccessTypeEnum { + err := k8sClient.Get(context.TODO(), adbLookupKey, adb) + if err != nil { + return "" + } + + return adb.Spec.Details.NetworkAccess.AccessType + }, timeout).Should(Equal(NetworkAccessTypePublic)) + }) + + It("Should set the default network access type to PRIVATE, if it's a dedicated ADB", func() { + By("Creating an AutonomousDatabase with ACD_OCID") + adb.Spec.Details.AutonomousContainerDatabase.OCIACD.OCID = common.String("ocid1.autonomouscontainerdatabase.oc1.dummy-acd-ocid") + + Expect(k8sClient.Create(context.TODO(), adb)).To(Succeed()) + + By("Checking the AutonomousDatabase has a network access type PRIVATE") + Eventually(func() NetworkAccessTypeEnum { + err := k8sClient.Get(context.TODO(), adbLookupKey, adb) + if err != nil { + return "" + } + + return adb.Spec.Details.NetworkAccess.AccessType + }, timeout).Should(Equal(NetworkAccessTypePrivate)) + }) + }) + + Describe("Test ValidateCreate of the AutonomousDatabase validating webhook", func() { + var ( + resourceName = "testadb" + namespace = "default" + + adb *AutonomousDatabase + ) + + BeforeEach(func() { + adb = &AutonomousDatabase{ + TypeMeta: metav1.TypeMeta{ + APIVersion: "database.oracle.com/v1alpha1", + Kind: "AutonomousDatabase", + }, + ObjectMeta: metav1.ObjectMeta{ + Name: resourceName, + Namespace: namespace, + }, + Spec: AutonomousDatabaseSpec{ + Details: AutonomousDatabaseDetails{ + CompartmentOCID: common.String("fake-compartment-ocid"), + DbName: common.String("fake-dbName"), + DisplayName: common.String("fake-displayName"), + CPUCoreCount: common.Int(1), + AdminPassword: PasswordSpec{ + K8sSecret: K8sSecretSpec{ + Name: common.String("fake-admin-password"), + }, + }, + DataStorageSizeInTBs: common.Int(1), + }, + }, + } + }) + + // Common validation + It("Should not apply values to adminPassword.k8sSecret and adminPassword.ociSecret at the same time", func() { + var errMsg string = "cannot apply k8sSecret.name and ociSecret.ocid at the same time" + + adb.Spec.Details.AdminPassword.K8sSecret.Name = common.String("test-admin-password") + adb.Spec.Details.AdminPassword.OCISecret.OCID = common.String("fake.ocid1.vaultsecret.oc1...") + + validateInvalidTest(adb, false, errMsg) + }) + + It("Should not apply values to wallet.password.k8sSecret and wallet.password.ociSecret at the same time", func() { + var errMsg string = "cannot apply k8sSecret.name and ociSecret.ocid at the same time" + + adb.Spec.Details.Wallet.Password.K8sSecret.Name = common.String("test-wallet-password") + adb.Spec.Details.Wallet.Password.OCISecret.OCID = common.String("fake.ocid1.vaultsecret.oc1...") + + validateInvalidTest(adb, false, errMsg) + }) + + // Network validation + Context("Shared Autonomous Database", func() { + It("AccessControlList cannot be empty when the network access type is RESTRICTED", func() { + var errMsg string = "accessControlList cannot be empty when the network access type is " + string(NetworkAccessTypeRestricted) + + adb.Spec.Details.NetworkAccess.AccessType = NetworkAccessTypeRestricted + adb.Spec.Details.NetworkAccess.AccessControlList = nil + + validateInvalidTest(adb, false, errMsg) + }) + + It("SubnetOCID and nsgOCIDs cannot be empty when the network access type is PRIVATE", func() { + var errMsg1 string = "subnetOCID cannot be empty when the network access type is " + string(NetworkAccessTypePrivate) + var errMsg2 string = "nsgOCIDs cannot be empty when the network access type is " + string(NetworkAccessTypePrivate) + + adb.Spec.Details.NetworkAccess.AccessType = NetworkAccessTypePrivate + adb.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID = nil + adb.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs = nil + + validateInvalidTest(adb, false, errMsg1, errMsg2) + }) + + It("IsAccessControlEnabled is not applicable on a shared Autonomous Database", func() { + var errMsg string = "isAccessControlEnabled is not applicable on a shared Autonomous Database" + + adb.Spec.Details.NetworkAccess.IsAccessControlEnabled = common.Bool(true) + + validateInvalidTest(adb, false, errMsg) + }) + }) + + Context("Dedicated Autonomous Database", func() { + BeforeEach(func() { + adb.Spec.Details.AutonomousContainerDatabase.K8sACD.Name = common.String("testACD") + adb.Spec.Details.AutonomousContainerDatabase.OCIACD.OCID = common.String("fake-acd-ocid") + }) + + It("AccessControlList cannot be empty when the network access type is RESTRICTED", func() { + var errMsg string = "access control list cannot be provided when Autonomous Database's access control is disabled" + + adb.Spec.Details.NetworkAccess.IsAccessControlEnabled = common.Bool(false) + adb.Spec.Details.NetworkAccess.AccessControlList = []string{"192.168.1.1"} + + validateInvalidTest(adb, false, errMsg) + }) + + It("AccessControlList cannot be empty when the network access type is RESTRICTED", func() { + var errMsg string = "isMTLSConnectionRequired is not supported on a dedicated database" + + adb.Spec.Details.NetworkAccess.IsMTLSConnectionRequired = common.Bool(true) + + validateInvalidTest(adb, false, errMsg) + }) + + }) + + // Others + It("Cannot apply lifecycleState to a provision operation", func() { + var errMsg string = "cannot apply lifecycleState to a provision operation" + + adb.Spec.Details.LifecycleState = database.AutonomousDatabaseLifecycleStateStopped + + validateInvalidTest(adb, false, errMsg) + }) + }) + + // Skip the common and network validations since they're already verified in the test for ValidateCreate + Describe("Test ValidateUpdate of the AutonomousDatabase validating webhook", func() { + var ( + resourceName = "testadb" + namespace = "default" + adbLookupKey = types.NamespacedName{Name: resourceName, Namespace: namespace} + + adb *AutonomousDatabase + + timeout = time.Second * 5 + ) + + BeforeEach(func() { + adb = &AutonomousDatabase{ + TypeMeta: metav1.TypeMeta{ + APIVersion: "database.oracle.com/v1alpha1", + Kind: "AutonomousDatabase", + }, + ObjectMeta: metav1.ObjectMeta{ + Name: resourceName, + Namespace: namespace, + }, + Spec: AutonomousDatabaseSpec{ + Details: AutonomousDatabaseDetails{ + CompartmentOCID: common.String("fake-compartment-ocid"), + AutonomousDatabaseOCID: common.String("fake-adb-ocid"), + DbName: common.String("fake-dbName"), + DisplayName: common.String("fake-displayName"), + CPUCoreCount: common.Int(1), + DataStorageSizeInTBs: common.Int(1), + LifecycleState: database.AutonomousDatabaseLifecycleStateAvailable, + }, + }, + } + + specBytes, err := json.Marshal(adb.Spec) + Expect(err).To(BeNil()) + + anns := map[string]string{ + LastSuccessfulSpec: string(specBytes), + } + adb.SetAnnotations(anns) + + Expect(k8sClient.Create(context.TODO(), adb)).To(Succeed()) + + // Make sure the object is created + Eventually(func() error { + createdADB := &AutonomousDatabase{} + return k8sClient.Get(context.TODO(), adbLookupKey, createdADB) + }, timeout).Should(BeNil()) + }) + + AfterEach(func() { + Expect(k8sClient.Delete(context.TODO(), adb)).To(Succeed()) + }) + + It("AutonomousDatabaseOCID cannot be modified", func() { + var errMsg string = "autonomousDatabaseOCID cannot be modified" + + adb.Spec.Details.AutonomousDatabaseOCID = common.String("modified-adb-ocid") + + validateInvalidTest(adb, true, errMsg) + }) + + It("Cannot change lifecycleState with other spec.details attributes at the same time", func() { + var errMsg string = "cannot change lifecycleState with other spec.details attributes at the same time" + + adb.Spec.Details.LifecycleState = database.AutonomousDatabaseLifecycleStateStopped + adb.Spec.Details.CPUCoreCount = common.Int(2) + + validateInvalidTest(adb, true, errMsg) + }) + }) +}) diff --git a/apis/database/v1alpha1/autonomousdatabasebackup_webhook.go b/apis/database/v1alpha1/autonomousdatabasebackup_webhook.go index 2407ea27..8b19940e 100644 --- a/apis/database/v1alpha1/autonomousdatabasebackup_webhook.go +++ b/apis/database/v1alpha1/autonomousdatabasebackup_webhook.go @@ -81,6 +81,11 @@ func (r *AutonomousDatabaseBackup) ValidateCreate() error { var allErrs field.ErrorList + if r.Spec.Target.K8sADB.Name == nil && r.Spec.Target.OCIADB.OCID == nil { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("target"), "target ADB is empty")) + } + if r.Spec.Target.K8sADB.Name != nil && r.Spec.Target.OCIADB.OCID != nil { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("target"), "specify either k8sADB or ociADB, but not both")) @@ -104,7 +109,8 @@ func (r *AutonomousDatabaseBackup) ValidateUpdate(old runtime.Object) error { if oldBackup.Spec.AutonomousDatabaseBackupOCID != nil && r.Spec.AutonomousDatabaseBackupOCID != nil && *oldBackup.Spec.AutonomousDatabaseBackupOCID != *r.Spec.AutonomousDatabaseBackupOCID { allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec").Child("autonomousDatabaseBackupOCID"), "cannot assign a new autonomousDatabaseBackupOCID to this backup")) + field.Forbidden(field.NewPath("spec").Child("autonomousDatabaseBackupOCID"), + "cannot assign a new autonomousDatabaseBackupOCID to this backup")) } if oldBackup.Spec.Target.K8sADB.Name != nil && r.Spec.Target.K8sADB.Name != nil && diff --git a/apis/database/v1alpha1/autonomousdatabasebackup_webhook_test.go b/apis/database/v1alpha1/autonomousdatabasebackup_webhook_test.go new file mode 100644 index 00000000..32407742 --- /dev/null +++ b/apis/database/v1alpha1/autonomousdatabasebackup_webhook_test.go @@ -0,0 +1,172 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. +*/ +package v1alpha1 + +import ( + "context" + "time" + + "github.com/oracle/oci-go-sdk/v63/common" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/types" + // +kubebuilder:scaffold:imports +) + +var _ = Describe("test AutonomousDatabaseBackup webhook", func() { + Describe("Test ValidateCreate of the AutonomousDatabaseBackup validating webhook", func() { + var ( + resourceName = "testadbbackup" + namespace = "default" + + backup *AutonomousDatabaseBackup + ) + + BeforeEach(func() { + backup = &AutonomousDatabaseBackup{ + TypeMeta: metav1.TypeMeta{ + APIVersion: "database.oracle.com/v1alpha1", + Kind: "AutonomousDatabaseBackup", + }, + ObjectMeta: metav1.ObjectMeta{ + Name: resourceName, + Namespace: namespace, + }, + Spec: AutonomousDatabaseBackupSpec{ + Target: TargetSpec{}, + }, + } + }) + + It("Should specify at least one of the k8sADB and ociADB", func() { + var errMsg string = "target ADB is empty" + + backup.Spec.Target.K8sADB.Name = nil + backup.Spec.Target.OCIADB.OCID = nil + + validateInvalidTest(backup, false, errMsg) + }) + + It("Should specify either k8sADB or ociADB, but not both", func() { + var errMsg string = "specify either k8sADB or ociADB, but not both" + + backup.Spec.Target.K8sADB.Name = common.String("fake-target-adb") + backup.Spec.Target.OCIADB.OCID = common.String("fake.ocid1.autonomousdatabase.oc1...") + + validateInvalidTest(backup, false, errMsg) + }) + }) + + Describe("Test ValidateUpdate of the AutonomousDatabaseBackup validating webhook", func() { + var ( + resourceName = "testadbbackup" + namespace = "default" + backupLookupKey = types.NamespacedName{Name: resourceName, Namespace: namespace} + + backup *AutonomousDatabaseBackup + + timeout = time.Second * 5 + ) + + BeforeEach(func() { + backup = &AutonomousDatabaseBackup{ + TypeMeta: metav1.TypeMeta{ + APIVersion: "database.oracle.com/v1alpha1", + Kind: "AutonomousDatabaseBackup", + }, + ObjectMeta: metav1.ObjectMeta{ + Name: resourceName, + Namespace: namespace, + }, + Spec: AutonomousDatabaseBackupSpec{ + AutonomousDatabaseBackupOCID: common.String("fake.ocid1.autonomousdatabasebackup.oc1..."), + DisplayName: common.String("fake-displayName"), + }, + } + }) + + JustBeforeEach(func() { + Expect(k8sClient.Create(context.TODO(), backup)).To(Succeed()) + + // Make sure the object is created + Eventually(func() error { + createdBackup := &AutonomousDatabaseBackup{} + return k8sClient.Get(context.TODO(), backupLookupKey, createdBackup) + }, timeout).Should(BeNil()) + }) + + AfterEach(func() { + Expect(k8sClient.Delete(context.TODO(), backup)).To(Succeed()) + }) + + Context("The bakcup is using target.k8sADB.name", func() { + BeforeEach(func() { + backup.Spec.Target.K8sADB.Name = common.String("fake-target-adb") + }) + + It("Cannot assign a new name to the target", func() { + var errMsg string = "cannot assign a new name to the target" + + backup.Spec.Target.K8sADB.Name = common.String("modified-target-adb") + + validateInvalidTest(backup, true, errMsg) + }) + + It("Cannot assign a new displayName to this backup", func() { + var errMsg string = "cannot assign a new displayName to this backup" + + backup.Spec.DisplayName = common.String("modified-displayName") + + validateInvalidTest(backup, true, errMsg) + }) + }) + + Context("The bakcup is using target.ociADB.ocid", func() { + BeforeEach(func() { + backup.Spec.Target.OCIADB.OCID = common.String("fake.ocid1.autonomousdatabase.oc1...") + }) + + It("Cannot assign a new ocid to the target", func() { + var errMsg string = "cannot assign a new ocid to the target" + + backup.Spec.Target.OCIADB.OCID = common.String("modified.ocid1.autonomousdatabase.oc1...") + + validateInvalidTest(backup, true, errMsg) + }) + }) + }) +}) diff --git a/apis/database/v1alpha1/autonomousdatabaserestore_webhook.go b/apis/database/v1alpha1/autonomousdatabaserestore_webhook.go index edc4db7a..ed6e2edb 100644 --- a/apis/database/v1alpha1/autonomousdatabaserestore_webhook.go +++ b/apis/database/v1alpha1/autonomousdatabaserestore_webhook.go @@ -71,6 +71,11 @@ func (r *AutonomousDatabaseRestore) ValidateCreate() error { var allErrs field.ErrorList // Validate the target ADB + if r.Spec.Target.K8sADB.Name == nil && r.Spec.Target.OCIADB.OCID == nil { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("target"), "target ADB is empty")) + } + if r.Spec.Target.K8sADB.Name != nil && r.Spec.Target.OCIADB.OCID != nil { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("target"), "specify either k8sADB.name or ociADB.ocid, but not both")) @@ -80,7 +85,7 @@ func (r *AutonomousDatabaseRestore) ValidateCreate() error { if r.Spec.Source.K8sADBBackup.Name == nil && r.Spec.Source.PointInTime.Timestamp == nil { allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec").Child("source"), "no retore source is chosen")) + field.Forbidden(field.NewPath("spec").Child("source"), "retore source is empty")) } if r.Spec.Source.K8sADBBackup.Name != nil && @@ -89,11 +94,6 @@ func (r *AutonomousDatabaseRestore) ValidateCreate() error { field.Forbidden(field.NewPath("spec").Child("source"), "cannot apply backupName and the PITR parameters at the same time")) } - if (r.Spec.Target.OCIADB.OCID == nil && r.Spec.Source.PointInTime.Timestamp != nil) || - (r.Spec.Target.OCIADB.OCID != nil && r.Spec.Source.PointInTime.Timestamp == nil) { - field.Forbidden(field.NewPath("spec").Child("source").Child("pointInTime").Child("timestamp"), "target.ociADB.ocid or source.pointInTime.timestamp cannot be empty") - } - // Verify the timestamp format if it's PITR if r.Spec.Source.PointInTime.Timestamp != nil { _, err := parseDisplayTime(*r.Spec.Source.PointInTime.Timestamp) diff --git a/apis/database/v1alpha1/autonomousdatabaserestore_webhook_test.go b/apis/database/v1alpha1/autonomousdatabaserestore_webhook_test.go new file mode 100644 index 00000000..f1e6bab0 --- /dev/null +++ b/apis/database/v1alpha1/autonomousdatabaserestore_webhook_test.go @@ -0,0 +1,115 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. +*/ +package v1alpha1 + +import ( + "github.com/oracle/oci-go-sdk/v63/common" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + // +kubebuilder:scaffold:imports +) + +var _ = Describe("test AutonomousDatabaseRestore webhook", func() { + Describe("Test ValidateCreate of the AutonomousDatabaseRestore validating webhook", func() { + var ( + resourceName = "testadbrestore" + namespace = "default" + + restore *AutonomousDatabaseRestore + ) + + BeforeEach(func() { + restore = &AutonomousDatabaseRestore{ + TypeMeta: metav1.TypeMeta{ + APIVersion: "database.oracle.com/v1alpha1", + Kind: "AutonomousDatabaseRestore", + }, + ObjectMeta: metav1.ObjectMeta{ + Name: resourceName, + Namespace: namespace, + }, + Spec: AutonomousDatabaseRestoreSpec{ + Target: TargetSpec{}, + }, + } + }) + + It("Should specify at least one of the k8sADB and ociADB", func() { + var errMsg string = "target ADB is empty" + + restore.Spec.Target.K8sADB.Name = nil + restore.Spec.Target.OCIADB.OCID = nil + + validateInvalidTest(restore, false, errMsg) + }) + + It("Should specify either k8sADB.name or ociADB.ocid, but not both", func() { + var errMsg string = "specify either k8sADB.name or ociADB.ocid, but not both" + + restore.Spec.Target.K8sADB.Name = common.String("fake-target-adb") + restore.Spec.Target.OCIADB.OCID = common.String("fake.ocid1.autonomousdatabase.oc1...") + + validateInvalidTest(restore, false, errMsg) + }) + + It("Should select at least one restore source", func() { + var errMsg string = "retore source is empty" + + restore.Spec.Source.K8sADBBackup.Name = nil + restore.Spec.Source.PointInTime.Timestamp = nil + + validateInvalidTest(restore, false, errMsg) + }) + + It("Cannot apply backupName and the PITR parameters at the same time", func() { + var errMsg string = "cannot apply backupName and the PITR parameters at the same time" + + restore.Spec.Source.K8sADBBackup.Name = common.String("fake-source-adb-backup") + restore.Spec.Source.PointInTime.Timestamp = common.String("2021-12-23 11:03:13 UTC") + + validateInvalidTest(restore, false, errMsg) + }) + + It("Invalid timestamp format", func() { + var errMsg string = "invalid timestamp format" + + restore.Spec.Source.PointInTime.Timestamp = common.String("12/23/2021 11:03:13") + + validateInvalidTest(restore, false, errMsg) + }) + }) +}) diff --git a/apis/database/v1alpha1/webhook_suite_test.go b/apis/database/v1alpha1/webhook_suite_test.go new file mode 100644 index 00000000..08079d97 --- /dev/null +++ b/apis/database/v1alpha1/webhook_suite_test.go @@ -0,0 +1,209 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v1alpha1 + +import ( + "context" + "crypto/tls" + "encoding/json" + "errors" + "fmt" + "net" + "path/filepath" + "testing" + "time" + + "github.com/onsi/ginkgo" + "github.com/onsi/gomega" + + admissionv1 "k8s.io/api/admission/v1" + //+kubebuilder:scaffold:imports + k8sErrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/client-go/rest" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/envtest" + "sigs.k8s.io/controller-runtime/pkg/envtest/printer" + logf "sigs.k8s.io/controller-runtime/pkg/log" + "sigs.k8s.io/controller-runtime/pkg/log/zap" +) + +// To avoid dot import +var ( + BeforeSuite = ginkgo.BeforeSuite + AfterSuite = ginkgo.AfterSuite + Describe = ginkgo.Describe + PDescribe = ginkgo.PDescribe + JustBeforeEach = ginkgo.JustBeforeEach + BeforeEach = ginkgo.BeforeEach + AfterEach = ginkgo.AfterEach + Context = ginkgo.Context + By = ginkgo.By + It = ginkgo.It + FIt = ginkgo.FIt + PIt = ginkgo.PIt + Eventually = gomega.Eventually + Expect = gomega.Expect + Succeed = gomega.Succeed + HaveOccurred = gomega.HaveOccurred + BeNil = gomega.BeNil + Equal = gomega.Equal + BeTrue = gomega.BeTrue + BeFalse = gomega.BeFalse + ContainSubstring = gomega.ContainSubstring +) + +// These tests use Ginkgo (BDD-style Go testing framework). Refer to +// http://onsi.github.io/ginkgo/ to learn more about Ginkgo. + +var cfg *rest.Config +var k8sClient client.Client +var testEnv *envtest.Environment +var ctx context.Context +var cancel context.CancelFunc + +func TestAPIs(t *testing.T) { + gomega.RegisterFailHandler(ginkgo.Fail) + + ginkgo.RunSpecsWithDefaultAndCustomReporters(t, + "Webhook Suite", + []ginkgo.Reporter{printer.NewlineReporter{}}) +} + +var _ = BeforeSuite(func() { + logf.SetLogger(zap.New(zap.WriteTo(ginkgo.GinkgoWriter), zap.UseDevMode(true))) + + ctx, cancel = context.WithCancel(context.TODO()) + + By("bootstrapping test environment") + testEnv = &envtest.Environment{ + CRDDirectoryPaths: []string{filepath.Join("..", "..", "..", "config", "crd", "bases")}, + ErrorIfCRDPathMissing: false, + WebhookInstallOptions: envtest.WebhookInstallOptions{ + Paths: []string{filepath.Join("..", "..", "..", "config", "webhook")}, + }, + } + + cfg, err := testEnv.Start() + Expect(err).NotTo(HaveOccurred()) + Expect(cfg).NotTo(BeNil()) + + scheme := runtime.NewScheme() + err = AddToScheme(scheme) + Expect(err).NotTo(HaveOccurred()) + + err = admissionv1.AddToScheme(scheme) + Expect(err).NotTo(HaveOccurred()) + + //+kubebuilder:scaffold:scheme + + k8sClient, err = client.New(cfg, client.Options{Scheme: scheme}) + Expect(err).NotTo(HaveOccurred()) + Expect(k8sClient).NotTo(BeNil()) + + // start webhook server using Manager + webhookInstallOptions := &testEnv.WebhookInstallOptions + mgr, err := ctrl.NewManager(cfg, ctrl.Options{ + Scheme: scheme, + Host: webhookInstallOptions.LocalServingHost, + Port: webhookInstallOptions.LocalServingPort, + CertDir: webhookInstallOptions.LocalServingCertDir, + LeaderElection: false, + MetricsBindAddress: "0", + }) + Expect(err).NotTo(HaveOccurred()) + + err = (&AutonomousDatabase{}).SetupWebhookWithManager(mgr) + Expect(err).NotTo(HaveOccurred()) + + err = (&AutonomousDatabaseBackup{}).SetupWebhookWithManager(mgr) + Expect(err).NotTo(HaveOccurred()) + + err = (&AutonomousDatabaseRestore{}).SetupWebhookWithManager(mgr) + Expect(err).NotTo(HaveOccurred()) + + //+kubebuilder:scaffold:webhook + + go func() { + defer ginkgo.GinkgoRecover() + err = mgr.Start(ctx) + Expect(err).NotTo(HaveOccurred()) + }() + + // wait for the webhook server to get ready + dialer := &net.Dialer{Timeout: time.Second} + addrPort := fmt.Sprintf("%s:%d", webhookInstallOptions.LocalServingHost, webhookInstallOptions.LocalServingPort) + Eventually(func() error { + conn, err := tls.DialWithDialer(dialer, "tcp", addrPort, &tls.Config{InsecureSkipVerify: true}) + if err != nil { + return err + } + conn.Close() + return nil + }).Should(Succeed()) +}, 60) + +var _ = AfterSuite(func() { + cancel() + By("tearing down the test environment") + err := testEnv.Stop() + Expect(err).NotTo(HaveOccurred()) +}) + +func validateInvalidTest(obj client.Object, isUpdate bool, expMsgList ...string) { + var err error + + if !isUpdate { + err = k8sClient.Create(context.TODO(), obj) + } else { + err = k8sClient.Update(context.TODO(), obj) + } + + jsonBytes, jsonErr := json.MarshalIndent(obj, "", " ") + Expect(jsonErr).ToNot(HaveOccurred()) + Expect(err).To(HaveOccurred(), "%s: %v", obj.GetObjectKind().GroupVersionKind().Kind, string(jsonBytes)) + + statusErr := &k8sErrors.StatusError{} + Expect(errors.As(err, &statusErr)).To(BeTrue()) + + for _, msg := range expMsgList { + Expect(statusErr.ErrStatus.Message).To(ContainSubstring(msg)) + } +} diff --git a/commons/oci/workrequest.go b/commons/oci/workrequest.go index 0ac02911..37fbb9be 100644 --- a/commons/oci/workrequest.go +++ b/commons/oci/workrequest.go @@ -53,6 +53,7 @@ import ( type WorkRequestService interface { Get(opcWorkRequestID string) (workrequests.GetWorkRequestResponse, error) Wait(opcWorkRequestID string) (workrequests.GetWorkRequestResponse, error) + List(compartmentID string, resourceID string) (workrequests.ListWorkRequestsResponse, error) } type workRequestService struct { @@ -143,3 +144,17 @@ func (w *workRequestService) Get(opcWorkRequestID string) (workrequests.GetWorkR return resp, nil } + +func (w *workRequestService) List(compartmentID string, resourceID string) (workrequests.ListWorkRequestsResponse, error) { + req := workrequests.ListWorkRequestsRequest{ + CompartmentId: common.String(compartmentID), + ResourceId: common.String(resourceID), + } + + resp, err := w.workClient.ListWorkRequests(context.TODO(), req) + if err != nil { + return resp, err + } + + return resp, nil +} diff --git a/controllers/database/autonomouscontainerdatabase_controller.go b/controllers/database/autonomouscontainerdatabase_controller.go index 713b0462..e4582663 100644 --- a/controllers/database/autonomouscontainerdatabase_controller.go +++ b/controllers/database/autonomouscontainerdatabase_controller.go @@ -42,6 +42,7 @@ import ( "context" "encoding/json" "errors" + "fmt" "reflect" "time" @@ -196,16 +197,17 @@ func (r *AutonomousContainerDatabaseReconciler) Reconcile(ctx context.Context, r } /****************************************************************** - * Determine which Database operations need to be executed by checking the changes to spec.details. + * Determine which Database operations need to be executed by checking the changes to spec. * There are three scenario: - * 1. provision operation. The AutonomousDatabaseOCID is missing, and the LastSucSpec annotation is missing. - * 2. bind operation. The AutonomousDatabaseOCID is provided, but the LastSucSpec annotation is missing. - * 3. update operation. Every changes other than the above two cases goes here. + * 1. provision operation. The ACD OCID is missing, and the LastSucSpec annotation is missing. + * 2. bind operation. The ACD OCID is provided, but the LastSucSpec annotation is missing. + * 3. sync operation. The action field is SYNC. + * 4. update operation. The changes which are not provision, bind or sync operations is an update operation. * Afterwards, update the resource from the remote database in OCI. This step will be executed right after * the above three cases during every reconcile. /******************************************************************/ // difACD is nil when the action is PROVISION or BIND. - // Use difACD to identify which fields are updated when it's UPDATE operation. + // Use difACD to identify which fields are updated when it's an UPDATE operation. action, difACD, err := r.determineAction(acd) if err != nil { return r.manageError(acd, err) @@ -230,11 +232,16 @@ func (r *AutonomousContainerDatabaseReconciler) Reconcile(ctx context.Context, r /***************************************************** * Sync resource and update the lastSucSpec *****************************************************/ - err = r.syncResource(acd) + specUpdated, err := r.syncResource(acd) if err != nil { return r.manageError(acd, err) } + r.Log.Info(fmt.Sprintf("=== specUpdated: %t", specUpdated)) + if specUpdated { + return emptyResult, nil + } + logger.Info("AutonomousContainerDatabase reconciles successfully") return emptyResult, nil @@ -264,11 +271,6 @@ func (r *AutonomousContainerDatabaseReconciler) updateResourceStatus(acd *dbv1al // updateResource updates the specification, the status of AutonomousContainerDatabase resource func (r *AutonomousContainerDatabaseReconciler) updateResource(acd *dbv1alpha1.AutonomousContainerDatabase) error { - // Update the status first to prevent unwanted Reconcile() - if err := r.updateResourceStatus(acd); err != nil { - return err - } - // Update the spec if err := retry.RetryOnConflict(retry.DefaultRetry, func() error { curACD := &dbv1alpha1.AutonomousContainerDatabase{} @@ -291,35 +293,53 @@ func (r *AutonomousContainerDatabaseReconciler) updateResource(acd *dbv1alpha1.A return nil } -// syncResource pulled information from OCI, update the LastSucSpec, and lastly, update the local resource. -// It's important to update the LastSucSpec prior than we update the local resource, -// because updating the local resource triggers the Reconcile, and the Reconcile determines the action by -// looking if the lastSucSpec is present. -func (r *AutonomousContainerDatabaseReconciler) syncResource(acd *dbv1alpha1.AutonomousContainerDatabase) error { +// syncResource pulled information from OCI, update the lastSucSpec, and lastly, update the local resource. +// It's important to update the lastSucSpec prior than we update the local resource, because updating +// the local resource triggers the Reconcile, and the Reconcile determines the action by looking if +// the lastSucSpec is present. +// The function returns if the spec is updated (and the Reconcile is triggered). +func (r *AutonomousContainerDatabaseReconciler) syncResource(acd *dbv1alpha1.AutonomousContainerDatabase) (bool, error) { // Get the information from OCI resp, err := r.dbService.GetAutonomousContainerDatabase(*acd.Spec.AutonomousContainerDatabaseOCID, nil) if err != nil { - return err + return false, err } - acd.UpdateFromOCIACD(resp.AutonomousContainerDatabase) + specChanged, statusChanged := acd.UpdateFromOCIACD(resp.AutonomousContainerDatabase) + r.Log.Info(fmt.Sprintf("==== specChanged = %t, statusChanged = %t", specChanged, statusChanged)) - if err := r.updateResource(acd); err != nil { - return err + // Validate the status change + if statusChanged { + r.Log.Info("==== update status") + if err := r.updateResourceStatus(acd); err != nil { + return false, err + } } - // Update the lastSucSpec - if err := r.updateLastSuccessfulSpec(acd); err != nil { - return err - } + // Validate the spec change + // If the spec changes, update the lastSucSpec and the resource, and then exit the Reconcile since it triggers another round of the Reconcile. + if specChanged { + r.Log.Info("==== update lastSucSpec") + if err := r.updateLastSuccessfulSpec(acd); err != nil { + return false, err + } - if err := r.updateResource(acd); err != nil { - return err + r.Log.Info("==== update resource") + if err := r.updateResource(acd); err != nil { + return false, err + } + + return true, nil } - return nil + return false, nil } +// updateLastSuccessfulSpec updates the lasSucSpec annotation, and returns the ORIGINAL object +// The object will NOT be updated with the returned content from the cluster since we want to +// update only the lasSucSpec. For example: After we get the ACD information from OCI, we want +// to update the lasSucSpec before updating the local resource. In this case, we want to keep +// the content returned from the OCI, not the one from the cluster. func (r *AutonomousContainerDatabaseReconciler) updateLastSuccessfulSpec(acd *dbv1alpha1.AutonomousContainerDatabase) error { specBytes, err := json.Marshal(acd.Spec) if err != nil { @@ -330,7 +350,9 @@ func (r *AutonomousContainerDatabaseReconciler) updateLastSuccessfulSpec(acd *db dbv1alpha1.LastSuccessfulSpec: string(specBytes), } - return annotations.PatchAnnotations(r.KubeClient, acd, anns) + copyACD := acd.DeepCopy() + + return annotations.PatchAnnotations(r.KubeClient, copyACD, anns) } func (r *AutonomousContainerDatabaseReconciler) manageError(acd *dbv1alpha1.AutonomousContainerDatabase, issue error) (ctrl.Result, error) { @@ -341,15 +363,11 @@ func (r *AutonomousContainerDatabaseReconciler) manageError(acd *dbv1alpha1.Auto var finalIssue = issue - if err := r.syncResource(acd); err != nil { + // Roll back + if _, err := r.syncResource(acd); err != nil { finalIssue = k8s.CombineErrors(finalIssue, err) } - if err := r.updateLastSuccessfulSpec(acd); err != nil { - finalIssue = k8s.CombineErrors(finalIssue, err) - } - - // r.updateResource has already triggered another Reconcile loop, so we simply log the error and return a nil return emptyResult, finalIssue } else { // Send event @@ -493,41 +511,6 @@ func (r *AutonomousContainerDatabaseReconciler) createACD(acd *dbv1alpha1.Autono return err } - // Wait for the ACD status changes to BACKUP_IN_PROGRESS - // Retry up to 3 times every 10 seconds. - logger.Info("Waiting for the status changing to BACKUP_IN_PROGRESS") - backupStartAttempts := uint(3) - backupStartNextDuration := func(r common.OCIOperationResponse) time.Duration { - return time.Duration(10) * time.Second - } - if _, err := r.dbService.WaitAutonomousContainerDatabaseStatus(*resp.AutonomousContainerDatabase.Id, - backupStartAttempts, - database.AutonomousContainerDatabaseLifecycleStateBackupInProgress, - backupStartNextDuration); err != nil { - return err - } - - // Wait for the ACD status changes to AVAILABLE. - // Wait 20 mins in the first attempt, then retry every 30 secs after that until reaches the - // maximum time of retry (~60mins). - logger.Info("Waiting for the status changing to AVAILABLE") - backupFinishAttempts := uint(81) - backupFinishNextDuration := func(r common.OCIOperationResponse) time.Duration { - if r.AttemptNumber <= 1 { - return time.Duration(20) * time.Minute - } - return time.Duration(30) * time.Second - } - - if _, err := r.dbService.WaitAutonomousContainerDatabaseStatus(*resp.AutonomousContainerDatabase.Id, - backupFinishAttempts, - database.AutonomousContainerDatabaseLifecycleStateBackupInProgress, - backupFinishNextDuration); err != nil { - return err - } - - logger.Info("ACD provision successfully") - return nil } @@ -607,8 +590,8 @@ func (r *AutonomousContainerDatabaseReconciler) updateLifecycleState(acd *dbv1al return errors.New("Unknown action") } - // Update the status and then erase the Action field - if err := r.updateResource(acd); err != nil { + // Update the status. The Action field will be erased at the sync operation before exiting the Reconcile. + if err := r.updateResourceStatus(acd); err != nil { return err } diff --git a/controllers/database/autonomousdatabase_controller.go b/controllers/database/autonomousdatabase_controller.go index 7cc37315..86aa9a0f 100644 --- a/controllers/database/autonomousdatabase_controller.go +++ b/controllers/database/autonomousdatabase_controller.go @@ -350,15 +350,11 @@ func (r *AutonomousDatabaseReconciler) manageError(adb *dbv1alpha1.AutonomousDat var finalIssue = issue + // Roll back if err := r.syncResource(adb); err != nil { finalIssue = k8s.CombineErrors(finalIssue, err) } - if err := r.updateLastSuccessfulSpec(adb); err != nil { - finalIssue = k8s.CombineErrors(finalIssue, err) - } - - // r.updateResource has already triggered another Reconcile loop, so we simply log the error and return a nil return emptyResult, finalIssue } else { // Send event @@ -475,6 +471,11 @@ func (r *AutonomousDatabaseReconciler) syncResource(adb *dbv1alpha1.AutonomousDa return err } + // Update the status first to prevent unwanted Reconcile() + if err := r.updateResourceStatus(adb); err != nil { + return err + } + if err := r.updateResource(adb); err != nil { return err } @@ -484,11 +485,6 @@ func (r *AutonomousDatabaseReconciler) syncResource(adb *dbv1alpha1.AutonomousDa // updateResource updates the specification, the status of AutonomousDatabase resource, and the lastSucSpec func (r *AutonomousDatabaseReconciler) updateResource(adb *dbv1alpha1.AutonomousDatabase) error { - // Update the status first to prevent unwanted Reconcile() - if err := r.updateResourceStatus(adb); err != nil { - return err - } - // Update the spec if err := retry.RetryOnConflict(retry.DefaultRetry, func() error { curADB := &dbv1alpha1.AutonomousDatabase{} From 04b864beb0f58df9b5065c258d2102873ce35a69 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Thu, 7 Apr 2022 02:30:45 +0000 Subject: [PATCH 281/628] Update autonomouscontainerdatabase_controller.go --- .../database/autonomouscontainerdatabase_controller.go | 5 ----- 1 file changed, 5 deletions(-) diff --git a/controllers/database/autonomouscontainerdatabase_controller.go b/controllers/database/autonomouscontainerdatabase_controller.go index e4582663..8a7e710d 100644 --- a/controllers/database/autonomouscontainerdatabase_controller.go +++ b/controllers/database/autonomouscontainerdatabase_controller.go @@ -237,7 +237,6 @@ func (r *AutonomousContainerDatabaseReconciler) Reconcile(ctx context.Context, r return r.manageError(acd, err) } - r.Log.Info(fmt.Sprintf("=== specUpdated: %t", specUpdated)) if specUpdated { return emptyResult, nil } @@ -306,11 +305,9 @@ func (r *AutonomousContainerDatabaseReconciler) syncResource(acd *dbv1alpha1.Aut } specChanged, statusChanged := acd.UpdateFromOCIACD(resp.AutonomousContainerDatabase) - r.Log.Info(fmt.Sprintf("==== specChanged = %t, statusChanged = %t", specChanged, statusChanged)) // Validate the status change if statusChanged { - r.Log.Info("==== update status") if err := r.updateResourceStatus(acd); err != nil { return false, err } @@ -319,12 +316,10 @@ func (r *AutonomousContainerDatabaseReconciler) syncResource(acd *dbv1alpha1.Aut // Validate the spec change // If the spec changes, update the lastSucSpec and the resource, and then exit the Reconcile since it triggers another round of the Reconcile. if specChanged { - r.Log.Info("==== update lastSucSpec") if err := r.updateLastSuccessfulSpec(acd); err != nil { return false, err } - r.Log.Info("==== update resource") if err := r.updateResource(acd); err != nil { return false, err } From 8e59393b20d19779f8370fb747996f21be3e8a0e Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Thu, 7 Apr 2022 02:35:40 +0000 Subject: [PATCH 282/628] Update autonomouscontainerdatabase_controller.go --- controllers/database/autonomouscontainerdatabase_controller.go | 1 - 1 file changed, 1 deletion(-) diff --git a/controllers/database/autonomouscontainerdatabase_controller.go b/controllers/database/autonomouscontainerdatabase_controller.go index 8a7e710d..2b6c9808 100644 --- a/controllers/database/autonomouscontainerdatabase_controller.go +++ b/controllers/database/autonomouscontainerdatabase_controller.go @@ -42,7 +42,6 @@ import ( "context" "encoding/json" "errors" - "fmt" "reflect" "time" From 6b45242a303fab95052fb16b5790e3ef68b35c33 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Wed, 27 Apr 2022 12:28:27 -0400 Subject: [PATCH 283/628] fix leadership election timed out --- Makefile | 2 +- .../v1alpha1/adbfamily_common_utils.go | 84 ++ .../autonomouscontainerdatabase_types.go | 94 +- .../autonomouscontainerdatabase_webhook.go | 108 ++ ...utonomouscontainerdatabase_webhook_test.go | 119 ++ .../v1alpha1/autonomousdatabase_types.go | 62 +- .../v1alpha1/autonomousdatabase_webhook.go | 33 +- .../autonomousdatabase_webhook_test.go | 17 +- .../autonomousdatabasebackup_types.go | 55 +- .../autonomousdatabasebackup_webhook_test.go | 2 +- .../autonomousdatabaserestore_types.go | 57 +- .../autonomousdatabaserestore_webhook_test.go | 2 +- apis/database/v1alpha1/webhook_suite_test.go | 13 +- commons/adb_family/utils.go | 18 +- commons/k8s/fetch.go | 4 +- commons/oci/containerdatabase.go | 34 +- commons/oci/database.go | 89 +- ....oracle.com_autonomousdatabasebackups.yaml | 10 +- ...oracle.com_autonomousdatabaserestores.yaml | 7 +- ...tabase.oracle.com_autonomousdatabases.yaml | 3 + .../acd/autonomouscontainerdatabase_sync.yaml | 16 - config/webhook/manifests.yaml | 21 +- .../autonomouscontainerdatabase_controller.go | 669 +++++----- .../database/autonomousdatabase_controller.go | 1138 +++++++++-------- .../autonomousdatabasebackup_controller.go | 154 +-- .../autonomousdatabaserestore_controller.go | 171 ++- controllers/database/suite_test.go | 13 +- docs/acd/README.md | 30 - go.mod | 13 +- go.sum | 21 +- main.go | 38 +- test/e2e/behavior/shared_behaviors.go | 2 +- test/e2e/suite_test.go | 14 +- 33 files changed, 1712 insertions(+), 1401 deletions(-) create mode 100644 apis/database/v1alpha1/autonomouscontainerdatabase_webhook.go create mode 100644 apis/database/v1alpha1/autonomouscontainerdatabase_webhook_test.go delete mode 100644 config/samples/acd/autonomouscontainerdatabase_sync.yaml diff --git a/Makefile b/Makefile index 7e6ecca9..15de15a8 100644 --- a/Makefile +++ b/Makefile @@ -69,7 +69,7 @@ build: generate fmt vet ## Build manager binary. run: manifests generate fmt vet ## Run a controller from your host. go run ./main.go -docker-build: #test ## Build docker image with the manager. +docker-build: manifests generate fmt vet #test ## Build docker image with the manager. Disable the test but keep the validations to fail fast docker build --no-cache=true --build-arg http_proxy=${HTTP_PROXY} --build-arg https_proxy=${HTTPS_PROXY} . -t ${IMG} #docker-build-proxy: test diff --git a/apis/database/v1alpha1/adbfamily_common_utils.go b/apis/database/v1alpha1/adbfamily_common_utils.go index b0c28f49..0411f1b5 100644 --- a/apis/database/v1alpha1/adbfamily_common_utils.go +++ b/apis/database/v1alpha1/adbfamily_common_utils.go @@ -47,6 +47,8 @@ import ( "sigs.k8s.io/controller-runtime/pkg/client" "github.com/oracle/oci-go-sdk/v63/common" + "github.com/oracle/oci-go-sdk/v63/database" + "github.com/oracle/oci-go-sdk/v63/workrequests" ) // LastSuccessfulSpec is an annotation key which maps to the value of last successful spec @@ -204,3 +206,85 @@ func parseDisplayTime(val string) (*common.SDKTime, error) { sdkTime := common.SDKTime{Time: parsedTime} return &sdkTime, nil } + +/************************ +* LifecycleState check +************************/ +func IsADBIntermediateState(state database.AutonomousDatabaseLifecycleStateEnum) bool { + if state == database.AutonomousDatabaseLifecycleStateProvisioning || + state == database.AutonomousDatabaseLifecycleStateUpdating || + state == database.AutonomousDatabaseLifecycleStateScaleInProgress || + state == database.AutonomousDatabaseLifecycleStateStarting || + state == database.AutonomousDatabaseLifecycleStateStopping || + state == database.AutonomousDatabaseLifecycleStateTerminating || + state == database.AutonomousDatabaseLifecycleStateRestoreInProgress || + state == database.AutonomousDatabaseLifecycleStateBackupInProgress || + state == database.AutonomousDatabaseLifecycleStateMaintenanceInProgress || + state == database.AutonomousDatabaseLifecycleStateRestarting || + state == database.AutonomousDatabaseLifecycleStateRecreating || + state == database.AutonomousDatabaseLifecycleStateRoleChangeInProgress || + state == database.AutonomousDatabaseLifecycleStateUpgrading { + return true + } + return false +} + +// NextADBStableState returns the next stable state if it's an intermediate state. +// Otherwise returns the same state. +func NextADBStableState(state database.AutonomousDatabaseLifecycleStateEnum) database.AutonomousDatabaseLifecycleStateEnum { + if state == database.AutonomousDatabaseLifecycleStateProvisioning || + state == database.AutonomousDatabaseLifecycleStateStarting || + state == database.AutonomousDatabaseLifecycleStateRestoreInProgress || + state == database.AutonomousDatabaseLifecycleStateBackupInProgress || + state == database.AutonomousDatabaseLifecycleStateScaleInProgress || + state == database.AutonomousDatabaseLifecycleStateUpdating || + state == database.AutonomousDatabaseLifecycleStateMaintenanceInProgress || + state == database.AutonomousDatabaseLifecycleStateRestarting || + state == database.AutonomousDatabaseLifecycleStateRecreating || + state == database.AutonomousDatabaseLifecycleStateRoleChangeInProgress || + state == database.AutonomousDatabaseLifecycleStateUpgrading { + + return database.AutonomousDatabaseLifecycleStateAvailable + } + + if state == database.AutonomousDatabaseLifecycleStateStopping { + return database.AutonomousDatabaseLifecycleStateStopped + } + + if state == database.AutonomousDatabaseLifecycleStateTerminating { + return database.AutonomousDatabaseLifecycleStateTerminated + } + + return state +} + +func IsBackupIntermediateState(state database.AutonomousDatabaseBackupLifecycleStateEnum) bool { + if state == database.AutonomousDatabaseBackupLifecycleStateCreating || + state == database.AutonomousDatabaseBackupLifecycleStateDeleting { + return true + } + return false +} + +func IsRestoreIntermediateState(state workrequests.WorkRequestStatusEnum) bool { + if state == workrequests.WorkRequestStatusAccepted || + state == workrequests.WorkRequestStatusInProgress || + state == workrequests.WorkRequestStatusCanceling { + return true + } + return false +} + +func IsACDIntermediateState(state database.AutonomousContainerDatabaseLifecycleStateEnum) bool { + if state == database.AutonomousContainerDatabaseLifecycleStateProvisioning || + state == database.AutonomousContainerDatabaseLifecycleStateUpdating || + state == database.AutonomousContainerDatabaseLifecycleStateTerminating || + state == database.AutonomousContainerDatabaseLifecycleStateBackupInProgress || + state == database.AutonomousContainerDatabaseLifecycleStateRestoring || + state == database.AutonomousContainerDatabaseLifecycleStateRestarting || + state == database.AutonomousContainerDatabaseLifecycleStateMaintenanceInProgress || + state == database.AutonomousContainerDatabaseLifecycleStateRoleChangeInProgress { + return true + } + return false +} diff --git a/apis/database/v1alpha1/autonomouscontainerdatabase_types.go b/apis/database/v1alpha1/autonomouscontainerdatabase_types.go index 52bbfbd3..ff400d5c 100644 --- a/apis/database/v1alpha1/autonomouscontainerdatabase_types.go +++ b/apis/database/v1alpha1/autonomouscontainerdatabase_types.go @@ -40,7 +40,6 @@ package v1alpha1 import ( "encoding/json" - "errors" "reflect" "github.com/oracle/oci-go-sdk/v63/database" @@ -57,7 +56,6 @@ type acdActionEnum string const ( AcdActionBlank acdActionEnum = "" - AcdActionSync acdActionEnum = "SYNC" AcdActionRestart acdActionEnum = "RESTART" AcdActionTerminate acdActionEnum = "TERMINATE" ) @@ -137,73 +135,75 @@ func (acd *AutonomousContainerDatabase) GetLastSuccessfulSpec() (*AutonomousCont return &sucSpec, nil } -// UpdateStatusFromOCIACD updates only the status from database.AutonomousDatabase object -func (acd *AutonomousContainerDatabase) UpdateStatusFromOCIACD(ociObj database.AutonomousContainerDatabase) bool { - var statusChanged bool = false +func (acd *AutonomousContainerDatabase) UpdateLastSuccessfulSpec() error { + specBytes, err := json.Marshal(acd.Spec) + if err != nil { + return err + } - statusChanged = statusChanged || acd.Status.LifecycleState != ociObj.LifecycleState - acd.Status.LifecycleState = ociObj.LifecycleState + anns := acd.GetAnnotations() - statusChanged = statusChanged || acd.Status.TimeCreated != FormatSDKTime(ociObj.TimeCreated) - acd.Status.TimeCreated = FormatSDKTime(ociObj.TimeCreated) + if anns == nil { + anns = map[string]string{ + LastSuccessfulSpec: string(specBytes), + } + } else { + anns[LastSuccessfulSpec] = string(specBytes) + } + + acd.SetAnnotations(anns) - return statusChanged + return nil } -func (acd *AutonomousContainerDatabase) UpdateFromOCIACD(ociObj database.AutonomousContainerDatabase) (bool, bool) { - // Spec - var specChanged bool = false +// UpdateStatusFromOCIACD updates the status subresource +func (acd *AutonomousContainerDatabase) UpdateStatusFromOCIACD(ociObj database.AutonomousContainerDatabase) { + acd.Status.LifecycleState = ociObj.LifecycleState + acd.Status.TimeCreated = FormatSDKTime(ociObj.TimeCreated) +} - acd.Spec.Action = AcdActionBlank +// UpdateFromOCIADB updates the attributes using database.AutonomousContainerDatabase object +func (acd *AutonomousContainerDatabase) UpdateFromOCIACD(ociObj database.AutonomousContainerDatabase) (specChanged bool) { + oldACD := acd.DeepCopy() - specChanged = specChanged || !reflect.DeepEqual(acd.Spec.AutonomousContainerDatabaseOCID, ociObj.Id) + /*********************************** + * update the spec + ***********************************/ + acd.Spec.Action = AcdActionBlank acd.Spec.AutonomousContainerDatabaseOCID = ociObj.Id - - specChanged = specChanged || !reflect.DeepEqual(acd.Spec.CompartmentOCID, ociObj.CompartmentId) acd.Spec.CompartmentOCID = ociObj.CompartmentId - - specChanged = specChanged || !reflect.DeepEqual(acd.Spec.DisplayName, ociObj.DisplayName) acd.Spec.DisplayName = ociObj.DisplayName - - specChanged = specChanged || !reflect.DeepEqual(acd.Spec.AutonomousExadataVMClusterOCID, ociObj.CloudAutonomousVmClusterId) acd.Spec.AutonomousExadataVMClusterOCID = ociObj.CloudAutonomousVmClusterId - - specChanged = specChanged || acd.Spec.PatchModel != ociObj.PatchModel acd.Spec.PatchModel = ociObj.PatchModel - // special case: the FreeformTag is nil after unmarshalling, but the OCI server always returns an emtpy map - if acd.Spec.FreeformTags == nil { - acd.Spec.FreeformTags = make(map[string]string) + // special case: an emtpy map will be nil after unmarshalling while the OCI always returns an emty map. + if len(ociObj.FreeformTags) != 0 { + acd.Spec.FreeformTags = ociObj.FreeformTags } - specChanged = specChanged || !reflect.DeepEqual(acd.Spec.FreeformTags, ociObj.FreeformTags) - acd.Spec.FreeformTags = ociObj.FreeformTags - // Status - statusChanged := acd.UpdateStatusFromOCIACD(ociObj) + /*********************************** + * update the status subresource + ***********************************/ + acd.UpdateStatusFromOCIACD(ociObj) - return specChanged, statusChanged + return !reflect.DeepEqual(oldACD.Spec, acd.Spec) } // RemoveUnchangedSpec removes the unchanged fields in spec, and returns if the spec has been changed. -// The function only processes the fields associated to ACD into account. That is, the ociConfig won't be impacted. -// Always restore the autonomousContainerDatabaseOCID from the lastSucSpec because we need it to send requests. -// A `false` is returned if the lastSucSpec is nil. -func (acd *AutonomousContainerDatabase) RemoveUnchangedSpec() (bool, error) { - lastSucSpec, err := acd.GetLastSuccessfulSpec() - if lastSucSpec == nil { - return false, errors.New("lastSucSpec is nil") - } - - oldConifg := acd.Spec.OCIConfig.DeepCopy() - - changed, err := removeUnchangedFields(*lastSucSpec, &acd.Spec) +func (acd *AutonomousContainerDatabase) RemoveUnchangedSpec(prevSpec AutonomousContainerDatabaseSpec) (bool, error) { + changed, err := removeUnchangedFields(prevSpec, &acd.Spec) if err != nil { return changed, err } - acd.Spec.AutonomousContainerDatabaseOCID = lastSucSpec.AutonomousContainerDatabaseOCID - - oldConifg.DeepCopyInto(&acd.Spec.OCIConfig) - return changed, nil } + +// A helper function which is useful for debugging. The function prints out a structural JSON format. +func (acd *AutonomousContainerDatabase) String() (string, error) { + out, err := json.MarshalIndent(acd, "", " ") + if err != nil { + return "", err + } + return string(out), nil +} diff --git a/apis/database/v1alpha1/autonomouscontainerdatabase_webhook.go b/apis/database/v1alpha1/autonomouscontainerdatabase_webhook.go new file mode 100644 index 00000000..6a9241fd --- /dev/null +++ b/apis/database/v1alpha1/autonomouscontainerdatabase_webhook.go @@ -0,0 +1,108 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v1alpha1 + +import ( + "reflect" + + apierrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/runtime/schema" + "k8s.io/apimachinery/pkg/util/validation/field" + ctrl "sigs.k8s.io/controller-runtime" + logf "sigs.k8s.io/controller-runtime/pkg/log" + "sigs.k8s.io/controller-runtime/pkg/webhook" +) + +// log is for logging in this package. +var autonomouscontainerdatabaselog = logf.Log.WithName("autonomouscontainerdatabase-resource") + +func (r *AutonomousContainerDatabase) SetupWebhookWithManager(mgr ctrl.Manager) error { + return ctrl.NewWebhookManagedBy(mgr). + For(r). + Complete() +} + +//+kubebuilder:webhook:verbs=create;update,path=/validate-database-oracle-com-v1alpha1-autonomouscontainerdatabase,mutating=false,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=autonomouscontainerdatabases,versions=v1alpha1,name=vautonomouscontainerdatabase.kb.io,admissionReviewVersions={v1} + +var _ webhook.Validator = &AutonomousContainerDatabase{} + +// ValidateCreate implements webhook.Validator so a webhook will be registered for the type +func (r *AutonomousContainerDatabase) ValidateCreate() error { + autonomouscontainerdatabaselog.Info("validate create", "name", r.Name) + + // TODO(user): fill in your validation logic upon object creation. + return nil +} + +// ValidateUpdate implements webhook.Validator so a webhook will be registered for the type +func (r *AutonomousContainerDatabase) ValidateUpdate(old runtime.Object) error { + var allErrs field.ErrorList + var oldACD *AutonomousContainerDatabase = old.(*AutonomousContainerDatabase) + + autonomouscontainerdatabaselog.Info("validate update", "name", r.Name) + + // skip the update of adding ADB OCID or binding + if oldACD.Status.LifecycleState == "" { + return nil + } + + // cannot update when the old state is in intermediate state, except for the terminate operatrion + if IsACDIntermediateState(oldACD.Status.LifecycleState) && + !reflect.DeepEqual(oldACD.Spec, r.Spec) { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec"), + "cannot change the spec when the lifecycleState is in an intermdeiate state")) + } + + if len(allErrs) == 0 { + return nil + } + return apierrors.NewInvalid( + schema.GroupKind{Group: "database.oracle.com", Kind: "AutonomousContainerDatabase"}, + r.Name, allErrs) +} + +// ValidateDelete implements webhook.Validator so a webhook will be registered for the type +func (r *AutonomousContainerDatabase) ValidateDelete() error { + autonomouscontainerdatabaselog.Info("validate delete", "name", r.Name) + + // TODO(user): fill in your validation logic upon object deletion. + return nil +} diff --git a/apis/database/v1alpha1/autonomouscontainerdatabase_webhook_test.go b/apis/database/v1alpha1/autonomouscontainerdatabase_webhook_test.go new file mode 100644 index 00000000..abdc8264 --- /dev/null +++ b/apis/database/v1alpha1/autonomouscontainerdatabase_webhook_test.go @@ -0,0 +1,119 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ +package v1alpha1 + +import ( + "context" + "encoding/json" + "time" + + "github.com/oracle/oci-go-sdk/v63/common" + "github.com/oracle/oci-go-sdk/v63/database" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/types" + // +kubebuilder:scaffold:imports +) + +var _ = Describe("test AutonomousContainerDatabase webhook", func() { + Describe("Test ValidateUpdate of the AutonomousContainerDatabase validating webhook", func() { + var ( + resourceName = "testacd" + namespace = "default" + acdLookupKey = types.NamespacedName{Name: resourceName, Namespace: namespace} + + acd *AutonomousContainerDatabase + + timeout = time.Second * 5 + ) + + BeforeEach(func() { + acd = &AutonomousContainerDatabase{ + TypeMeta: metav1.TypeMeta{ + APIVersion: "database.oracle.com/v1alpha1", + Kind: "AutonomousContainerDatabase", + }, + ObjectMeta: metav1.ObjectMeta{ + Name: resourceName, + Namespace: namespace, + }, + Spec: AutonomousContainerDatabaseSpec{ + AutonomousContainerDatabaseOCID: common.String("fake-acd-ocid"), + CompartmentOCID: common.String("fake-compartment-ocid"), + DisplayName: common.String("fake-displayName"), + AutonomousExadataVMClusterOCID: common.String("fake-vmcluster-ocid"), + PatchModel: database.AutonomousContainerDatabasePatchModelUpdates, + }, + } + + specBytes, err := json.Marshal(acd.Spec) + Expect(err).To(BeNil()) + + anns := map[string]string{ + LastSuccessfulSpec: string(specBytes), + } + acd.SetAnnotations(anns) + + Expect(k8sClient.Create(context.TODO(), acd)).To(Succeed()) + + // Change the lifecycleState to AVAILABLE + acd.Status.LifecycleState = database.AutonomousContainerDatabaseLifecycleStateAvailable + Expect(k8sClient.Status().Update(context.TODO(), acd)).To(Succeed()) + + // Make sure the object is created + Eventually(func() error { + createdACD := &AutonomousContainerDatabase{} + return k8sClient.Get(context.TODO(), acdLookupKey, createdACD) + }, timeout).Should(BeNil()) + }) + + AfterEach(func() { + Expect(k8sClient.Delete(context.TODO(), acd)).To(Succeed()) + }) + + It("Cannot change the spec when the lifecycleState is in an intermdeiate state", func() { + var errMsg string = "cannot change the spec when the lifecycleState is in an intermdeiate state" + + acd.Status.LifecycleState = database.AutonomousContainerDatabaseLifecycleStateProvisioning + Expect(k8sClient.Status().Update(context.TODO(), acd)).To(Succeed()) + + acd.Spec.DisplayName = common.String("modified-display-name") + + validateInvalidTest(acd, true, errMsg) + }) + }) +}) diff --git a/apis/database/v1alpha1/autonomousdatabase_types.go b/apis/database/v1alpha1/autonomousdatabase_types.go index c078aa73..b1cc6500 100644 --- a/apis/database/v1alpha1/autonomousdatabase_types.go +++ b/apis/database/v1alpha1/autonomousdatabase_types.go @@ -40,7 +40,7 @@ package v1alpha1 import ( "encoding/json" - "errors" + "reflect" "github.com/oracle/oci-go-sdk/v63/database" metaV1 "k8s.io/apimachinery/pkg/apis/meta/v1" @@ -184,6 +184,7 @@ type ConnectionStringSpec struct { // +kubebuilder:resource:shortName="adb";"adbs" // +kubebuilder:subresource:status // +kubebuilder:printcolumn:JSONPath=".spec.details.displayName",name="Display Name",type=string +// +kubebuilder:printcolumn:JSONPath=".spec.details.dbName",name="Db Name",type=string // +kubebuilder:printcolumn:JSONPath=".status.lifecycleState",name="State",type=string // +kubebuilder:printcolumn:JSONPath=".spec.details.isDedicated",name="Dedicated",type=string // +kubebuilder:printcolumn:JSONPath=".spec.details.cpuCoreCount",name="OCPUs",type=integer @@ -230,7 +231,28 @@ func (adb *AutonomousDatabase) GetLastSuccessfulSpec() (*AutonomousDatabaseSpec, return &sucSpec, nil } -// UpdateStatusFromOCIADB updates only the status from database.AutonomousDatabase object +func (adb *AutonomousDatabase) UpdateLastSuccessfulSpec() error { + specBytes, err := json.Marshal(adb.Spec) + if err != nil { + return err + } + + anns := adb.GetAnnotations() + + if anns == nil { + anns = map[string]string{ + LastSuccessfulSpec: string(specBytes), + } + } else { + anns[LastSuccessfulSpec] = string(specBytes) + } + + adb.SetAnnotations(anns) + + return nil +} + +// UpdateStatusFromOCIADB updates the status subresource func (adb *AutonomousDatabase) UpdateStatusFromOCIADB(ociObj database.AutonomousDatabase) { adb.Status.LifecycleState = ociObj.LifecycleState adb.Status.TimeCreated = FormatSDKTime(ociObj.TimeCreated) @@ -276,8 +298,10 @@ func (adb *AutonomousDatabase) UpdateStatusFromOCIADB(ociObj database.Autonomous } } -// UpdateFromOCIADB updates the attributes from database.AutonomousDatabase object -func (adb *AutonomousDatabase) UpdateFromOCIADB(ociObj database.AutonomousDatabase) { +// UpdateFromOCIADB updates the attributes using database.AutonomousDatabase object +func (adb *AutonomousDatabase) UpdateFromOCIADB(ociObj database.AutonomousDatabase) (specChanged bool) { + oldADB := adb.DeepCopy() + /*********************************** * update the spec ***********************************/ @@ -293,8 +317,11 @@ func (adb *AutonomousDatabase) UpdateFromOCIADB(ociObj database.AutonomousDataba adb.Spec.Details.CPUCoreCount = ociObj.CpuCoreCount adb.Spec.Details.IsAutoScalingEnabled = ociObj.IsAutoScalingEnabled adb.Spec.Details.IsDedicated = ociObj.IsDedicated - adb.Spec.Details.LifecycleState = ociObj.LifecycleState - adb.Spec.Details.FreeformTags = ociObj.FreeformTags + adb.Spec.Details.LifecycleState = NextADBStableState(ociObj.LifecycleState) + // special case: an emtpy map will be nil after unmarshalling while the OCI always returns an emty map. + if len(ociObj.FreeformTags) != 0 { + adb.Spec.Details.FreeformTags = ociObj.FreeformTags + } // Determine network.accessType if *ociObj.IsDedicated { @@ -310,35 +337,32 @@ func (adb *AutonomousDatabase) UpdateFromOCIADB(ociObj database.AutonomousDataba } adb.Spec.Details.NetworkAccess.IsAccessControlEnabled = ociObj.IsAccessControlEnabled - adb.Spec.Details.NetworkAccess.AccessControlList = ociObj.WhitelistedIps + if len(ociObj.WhitelistedIps) != 0 { + adb.Spec.Details.NetworkAccess.AccessControlList = ociObj.WhitelistedIps + } adb.Spec.Details.NetworkAccess.IsMTLSConnectionRequired = ociObj.IsMtlsConnectionRequired adb.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID = ociObj.SubnetId - adb.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs = ociObj.NsgIds + if len(ociObj.NsgIds) != 0 { + adb.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs = ociObj.NsgIds + } adb.Spec.Details.NetworkAccess.PrivateEndpoint.HostnamePrefix = ociObj.PrivateEndpointLabel /*********************************** * update the status subresource ***********************************/ - adb.UpdateStatusFromOCIADB(ociObj) + + return !reflect.DeepEqual(oldADB.Spec, adb.Spec) } // RemoveUnchangedDetails removes the unchanged fields in spec.details, and returns if the details has been changed. -// Always restore the details.autonomousDatabaseOCID from the lastSucSpec because we need it to send requests. -// A `false` is returned if the lastSucSpec is nil. -func (adb *AutonomousDatabase) RemoveUnchangedDetails() (bool, error) { - lastSucSpec, err := adb.GetLastSuccessfulSpec() - if lastSucSpec == nil { - return false, errors.New("lastSucSpec is nil") - } +func (adb *AutonomousDatabase) RemoveUnchangedDetails(prevSpec AutonomousDatabaseSpec) (bool, error) { - changed, err := removeUnchangedFields(lastSucSpec.Details, &adb.Spec.Details) + changed, err := removeUnchangedFields(prevSpec.Details, &adb.Spec.Details) if err != nil { return changed, err } - adb.Spec.Details.AutonomousDatabaseOCID = lastSucSpec.Details.AutonomousDatabaseOCID - return changed, nil } diff --git a/apis/database/v1alpha1/autonomousdatabase_webhook.go b/apis/database/v1alpha1/autonomousdatabase_webhook.go index c4e04a6a..eb5d9709 100644 --- a/apis/database/v1alpha1/autonomousdatabase_webhook.go +++ b/apis/database/v1alpha1/autonomousdatabase_webhook.go @@ -34,7 +34,7 @@ ** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, ** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE ** SOFTWARE. -*/ + */ package v1alpha1 @@ -42,6 +42,7 @@ import ( "fmt" "reflect" + "github.com/oracle/oci-go-sdk/v63/database" apierrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/runtime" "k8s.io/apimachinery/pkg/runtime/schema" @@ -60,8 +61,6 @@ func (r *AutonomousDatabase) SetupWebhookWithManager(mgr ctrl.Manager) error { Complete() } -// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! - //+kubebuilder:webhook:verbs=create;update,path=/mutate-database-oracle-com-v1alpha1-autonomousdatabase,mutating=true,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=autonomousdatabases,versions=v1alpha1,name=mautonomousdatabase.kb.io,admissionReviewVersions=v1 var _ webhook.Defaulter = &AutonomousDatabase{} @@ -82,8 +81,7 @@ func (r *AutonomousDatabase) Default() { } -// TODO(user): change verbs to "verbs=create;update;delete" if you want to enable deletion validation. -//+kubebuilder:webhook:verbs=create;update;delete,path=/validate-database-oracle-com-v1alpha1-autonomousdatabase,mutating=false,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=autonomousdatabases,versions=v1alpha1,name=vautonomousdatabase.kb.io,admissionReviewVersions={v1} +//+kubebuilder:webhook:verbs=create;update,path=/validate-database-oracle-com-v1alpha1-autonomousdatabase,mutating=false,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=autonomousdatabases,versions=v1alpha1,name=vautonomousdatabase.kb.io,admissionReviewVersions={v1} var _ webhook.Validator = &AutonomousDatabase{} @@ -117,20 +115,25 @@ func (r *AutonomousDatabase) ValidateCreate() error { // ValidateUpdate implements webhook.Validator so a webhook will be registered for the type func (r *AutonomousDatabase) ValidateUpdate(old runtime.Object) error { - // Validate creation instead of update if there is no last successful spec, i.e. the database isn't provisioned or bound yet - lastSucSpec, err := r.GetLastSuccessfulSpec() - if err != nil { - return err - } - if lastSucSpec == nil { - return r.ValidateCreate() - } - var allErrs field.ErrorList var oldADB *AutonomousDatabase = old.(*AutonomousDatabase) autonomousdatabaselog.Info("validate update", "name", r.Name) + // skip the update of adding ADB OCID or binding + if oldADB.Status.LifecycleState == "" { + return nil + } + + // cannot update when the old state is in intermediate, except for the terminate operatrion + if IsADBIntermediateState(oldADB.Status.LifecycleState) && + r.Spec.Details.LifecycleState != database.AutonomousDatabaseLifecycleStateTerminated && + !reflect.DeepEqual(oldADB.Spec.Details, r.Spec.Details) { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("details"), + "cannot change spec.details when the lifecycleState is in an intermdeiate state")) + } + // cannot modify autonomousDatabaseOCID if r.Spec.Details.AutonomousDatabaseOCID != nil && oldADB.Spec.Details.AutonomousDatabaseOCID != nil && @@ -142,7 +145,7 @@ func (r *AutonomousDatabase) ValidateUpdate(old runtime.Object) error { // cannot change lifecycleState with other fields together var lifecycleChanged, otherDetailsChanged bool - lifecycleChanged = oldADB.Spec.Details.LifecycleState != r.Spec.Details.LifecycleState + lifecycleChanged = oldADB.Spec.Details.LifecycleState != "" && oldADB.Spec.Details.LifecycleState != r.Spec.Details.LifecycleState copyLifecycleState := oldADB.Spec.Details.LifecycleState oldADB.Spec.Details.LifecycleState = r.Spec.Details.LifecycleState otherDetailsChanged = !reflect.DeepEqual(oldADB.Spec.Details, r.Spec.Details) diff --git a/apis/database/v1alpha1/autonomousdatabase_webhook_test.go b/apis/database/v1alpha1/autonomousdatabase_webhook_test.go index 524f19e6..d225c589 100644 --- a/apis/database/v1alpha1/autonomousdatabase_webhook_test.go +++ b/apis/database/v1alpha1/autonomousdatabase_webhook_test.go @@ -34,7 +34,7 @@ ** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, ** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE ** SOFTWARE. -*/ + */ package v1alpha1 import ( @@ -280,6 +280,10 @@ var _ = Describe("test AutonomousDatabase webhook", func() { Expect(k8sClient.Create(context.TODO(), adb)).To(Succeed()) + // Change the lifecycleState to AVAILABLE + adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateAvailable + Expect(k8sClient.Status().Update(context.TODO(), adb)).To(Succeed()) + // Make sure the object is created Eventually(func() error { createdADB := &AutonomousDatabase{} @@ -291,6 +295,17 @@ var _ = Describe("test AutonomousDatabase webhook", func() { Expect(k8sClient.Delete(context.TODO(), adb)).To(Succeed()) }) + It("Cannot change spec.details when the lifecycleState is in an intermdeiate state", func() { + var errMsg string = "cannot change spec.details when the lifecycleState is in an intermdeiate state" + + adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUpdating + Expect(k8sClient.Status().Update(context.TODO(), adb)).To(Succeed()) + + adb.Spec.Details.DbName = common.String("modified-db-name") + + validateInvalidTest(adb, true, errMsg) + }) + It("AutonomousDatabaseOCID cannot be modified", func() { var errMsg string = "autonomousDatabaseOCID cannot be modified" diff --git a/apis/database/v1alpha1/autonomousdatabasebackup_types.go b/apis/database/v1alpha1/autonomousdatabasebackup_types.go index 52113f4d..e6cec9d3 100644 --- a/apis/database/v1alpha1/autonomousdatabasebackup_types.go +++ b/apis/database/v1alpha1/autonomousdatabasebackup_types.go @@ -61,30 +61,17 @@ type AutonomousDatabaseBackupSpec struct { type BackupStateEnum string -const ( - BackupStateError BackupStateEnum = "ERROR" - BackupStateCreating BackupStateEnum = "CREATING" - BackupStateActive BackupStateEnum = "ACTIVE" - BackupStateDeleting BackupStateEnum = "DELETING" - BackupStateDeleted BackupStateEnum = "DELETED" - BackupStateFailed BackupStateEnum = "FAILED" -) - // AutonomousDatabaseBackupStatus defines the observed state of AutonomousDatabaseBackup type AutonomousDatabaseBackupStatus struct { - // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster - // Important: Run "make" to regenerate code after modifying this file - AutonomousDatabaseBackupOCID string `json:"autonomousDatabaseBackupOCID,omitempty"` - DisplayName string `json:"displayName"` - LifecycleState BackupStateEnum `json:"lifecycleState"` - Type database.AutonomousDatabaseBackupTypeEnum `json:"type"` - IsAutomatic bool `json:"isAutomatic"` - TimeStarted string `json:"timeStarted,omitempty"` - TimeEnded string `json:"timeEnded,omitempty"` - AutonomousDatabaseOCID string `json:"autonomousDatabaseOCID"` - CompartmentOCID string `json:"compartmentOCID"` - DBName string `json:"dbName"` - DBDisplayName string `json:"dbDisplayName"` + LifecycleState database.AutonomousDatabaseBackupLifecycleStateEnum `json:"lifecycleState"` + Type database.AutonomousDatabaseBackupTypeEnum `json:"type"` + IsAutomatic bool `json:"isAutomatic"` + TimeStarted string `json:"timeStarted,omitempty"` + TimeEnded string `json:"timeEnded,omitempty"` + AutonomousDatabaseOCID string `json:"autonomousDatabaseOCID"` + CompartmentOCID string `json:"compartmentOCID"` + DBName string `json:"dbName"` + DBDisplayName string `json:"dbDisplayName"` } //+kubebuilder:object:root=true @@ -119,15 +106,12 @@ func init() { } func (b *AutonomousDatabaseBackup) UpdateStatusFromOCIBackup(ociBackup database.AutonomousDatabaseBackup, ociADB database.AutonomousDatabase) { - b.Status.AutonomousDatabaseBackupOCID = *ociBackup.Id - b.Status.DisplayName = *ociBackup.DisplayName - b.Status.AutonomousDatabaseOCID = *ociBackup.AutonomousDatabaseId b.Status.CompartmentOCID = *ociBackup.CompartmentId b.Status.Type = ociBackup.Type b.Status.IsAutomatic = *ociBackup.IsAutomatic - b.Status.LifecycleState = b.ConvertBackupStatus(ociBackup.LifecycleState) + b.Status.LifecycleState = ociBackup.LifecycleState b.Status.TimeStarted = FormatSDKTime(ociBackup.TimeStarted) b.Status.TimeEnded = FormatSDKTime(ociBackup.TimeEnded) @@ -140,22 +124,3 @@ func (b *AutonomousDatabaseBackup) UpdateStatusFromOCIBackup(ociBackup database. func (b *AutonomousDatabaseBackup) GetTimeEnded() (*common.SDKTime, error) { return parseDisplayTime(b.Status.TimeEnded) } - -func (b *AutonomousDatabaseBackup) ConvertBackupStatus(state database.AutonomousDatabaseBackupLifecycleStateEnum) BackupStateEnum { - switch state { - case database.AutonomousDatabaseBackupLifecycleStateCreating: - return BackupStateCreating - case database.AutonomousDatabaseBackupLifecycleStateActive: - return BackupStateActive - - case database.AutonomousDatabaseBackupLifecycleStateDeleting: - return BackupStateDeleting - - case database.AutonomousDatabaseBackupLifecycleStateDeleted: - return BackupStateDeleted - case database.AutonomousDatabaseBackupLifecycleStateFailed: - return BackupStateFailed - default: - return "UNKNOWN" - } -} diff --git a/apis/database/v1alpha1/autonomousdatabasebackup_webhook_test.go b/apis/database/v1alpha1/autonomousdatabasebackup_webhook_test.go index 32407742..56c79bc0 100644 --- a/apis/database/v1alpha1/autonomousdatabasebackup_webhook_test.go +++ b/apis/database/v1alpha1/autonomousdatabasebackup_webhook_test.go @@ -34,7 +34,7 @@ ** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, ** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE ** SOFTWARE. -*/ + */ package v1alpha1 import ( diff --git a/apis/database/v1alpha1/autonomousdatabaserestore_types.go b/apis/database/v1alpha1/autonomousdatabaserestore_types.go index 6f65ee9a..c49556a6 100644 --- a/apis/database/v1alpha1/autonomousdatabaserestore_types.go +++ b/apis/database/v1alpha1/autonomousdatabaserestore_types.go @@ -44,6 +44,7 @@ import ( metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "github.com/oracle/oci-go-sdk/v63/common" + "github.com/oracle/oci-go-sdk/v63/database" "github.com/oracle/oci-go-sdk/v63/workrequests" ) @@ -72,26 +73,27 @@ type AutonomousDatabaseRestoreSpec struct { OCIConfig OCIConfigSpec `json:"ociConfig,omitempty"` } -type restoreStatusEnum string +type RestoreStatusEnum string const ( - RestoreStatusError restoreStatusEnum = "ERROR" - RestoreStatusInProgress restoreStatusEnum = "IN_PROGRESS" - RestoreStatusFailed restoreStatusEnum = "FAILED" - RestoreStatusSucceeded restoreStatusEnum = "SUCCEEDED" + RestoreStatusError RestoreStatusEnum = "ERROR" + RestoreStatusAccepted RestoreStatusEnum = "ACCEPTED" + RestoreStatusInProgress RestoreStatusEnum = "IN_PROGRESS" + RestoreStatusFailed RestoreStatusEnum = "FAILED" + RestoreStatusSucceeded RestoreStatusEnum = "SUCCEEDED" ) // AutonomousDatabaseRestoreStatus defines the observed state of AutonomousDatabaseRestore type AutonomousDatabaseRestoreStatus struct { // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster // Important: Run "make" to regenerate code after modifying this file - DisplayName string `json:"displayName"` - TimeAccepted string `json:"timeAccepted,omitempty"` - TimeStarted string `json:"timeStarted,omitempty"` - TimeEnded string `json:"timeEnded,omitempty"` - DbName string `json:"dbName"` - AutonomousDatabaseOCID string `json:"autonomousDatabaseOCID"` - Status restoreStatusEnum `json:"status"` + DisplayName string `json:"displayName"` + TimeAccepted string `json:"timeAccepted,omitempty"` + TimeStarted string `json:"timeStarted,omitempty"` + TimeEnded string `json:"timeEnded,omitempty"` + DbName string `json:"dbName"` + WorkRequestOCID string `json:"workRequestOCID"` + Status workrequests.WorkRequestStatusEnum `json:"status"` } //+kubebuilder:object:root=true @@ -131,23 +133,16 @@ func (r *AutonomousDatabaseRestore) GetPIT() (*common.SDKTime, error) { return parseDisplayTime(*r.Spec.Source.PointInTime.Timestamp) } -func (r *AutonomousDatabaseRestore) ConvertWorkRequestStatus(s workrequests.WorkRequestStatusEnum) restoreStatusEnum { - switch s { - case workrequests.WorkRequestStatusAccepted: - fallthrough - case workrequests.WorkRequestStatusInProgress: - return RestoreStatusInProgress - - case workrequests.WorkRequestStatusSucceeded: - return RestoreStatusSucceeded - - case workrequests.WorkRequestStatusCanceling: - fallthrough - case workrequests.WorkRequestStatusCanceled: - fallthrough - case workrequests.WorkRequestStatusFailed: - return RestoreStatusFailed - default: - return "UNKNOWN" - } +func (r *AutonomousDatabaseRestore) UpdateStatus( + adb database.AutonomousDatabase, + workResp workrequests.GetWorkRequestResponse) { + + r.Status.DisplayName = *adb.DisplayName + r.Status.DbName = *adb.DbName + + r.Status.WorkRequestOCID = *workResp.Id + r.Status.Status = workResp.Status + r.Status.TimeAccepted = FormatSDKTime(workResp.TimeAccepted) + r.Status.TimeStarted = FormatSDKTime(workResp.TimeStarted) + r.Status.TimeEnded = FormatSDKTime(workResp.TimeFinished) } diff --git a/apis/database/v1alpha1/autonomousdatabaserestore_webhook_test.go b/apis/database/v1alpha1/autonomousdatabaserestore_webhook_test.go index f1e6bab0..50974800 100644 --- a/apis/database/v1alpha1/autonomousdatabaserestore_webhook_test.go +++ b/apis/database/v1alpha1/autonomousdatabaserestore_webhook_test.go @@ -34,7 +34,7 @@ ** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, ** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE ** SOFTWARE. -*/ + */ package v1alpha1 import ( diff --git a/apis/database/v1alpha1/webhook_suite_test.go b/apis/database/v1alpha1/webhook_suite_test.go index 08079d97..dbbd02c9 100644 --- a/apis/database/v1alpha1/webhook_suite_test.go +++ b/apis/database/v1alpha1/webhook_suite_test.go @@ -49,7 +49,7 @@ import ( "testing" "time" - "github.com/onsi/ginkgo" + "github.com/onsi/ginkgo/v2" "github.com/onsi/gomega" admissionv1 "k8s.io/api/admission/v1" @@ -60,7 +60,6 @@ import ( ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/controller-runtime/pkg/envtest" - "sigs.k8s.io/controller-runtime/pkg/envtest/printer" logf "sigs.k8s.io/controller-runtime/pkg/log" "sigs.k8s.io/controller-runtime/pkg/log/zap" ) @@ -101,10 +100,7 @@ var cancel context.CancelFunc func TestAPIs(t *testing.T) { gomega.RegisterFailHandler(ginkgo.Fail) - - ginkgo.RunSpecsWithDefaultAndCustomReporters(t, - "Webhook Suite", - []ginkgo.Reporter{printer.NewlineReporter{}}) + ginkgo.RunSpecs(t, "Webhook Suite") } var _ = BeforeSuite(func() { @@ -159,6 +155,9 @@ var _ = BeforeSuite(func() { err = (&AutonomousDatabaseRestore{}).SetupWebhookWithManager(mgr) Expect(err).NotTo(HaveOccurred()) + err = (&AutonomousContainerDatabase{}).SetupWebhookWithManager(mgr) + Expect(err).NotTo(HaveOccurred()) + //+kubebuilder:scaffold:webhook go func() { @@ -178,7 +177,7 @@ var _ = BeforeSuite(func() { conn.Close() return nil }).Should(Succeed()) -}, 60) +}) var _ = AfterSuite(func() { cancel() diff --git a/commons/adb_family/utils.go b/commons/adb_family/utils.go index ed0e1efc..248784f3 100644 --- a/commons/adb_family/utils.go +++ b/commons/adb_family/utils.go @@ -39,7 +39,6 @@ package adbfamily import ( - "github.com/oracle/oci-go-sdk/v63/common" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" "github.com/oracle/oracle-database-operator/commons/k8s" "sigs.k8s.io/controller-runtime/pkg/client" @@ -49,31 +48,26 @@ import ( // The function returns two values in the following order: // ocid: the OCID of the target ADB. An empty string is returned if the ocid is nil. // ownerADB: the resource of the targetADB if it's found in the cluster -func VerifyTargetADB(kubeClient client.Client, target dbv1alpha1.TargetSpec, namespace string) (string, *dbv1alpha1.AutonomousDatabase, error) { +func VerifyTargetADB(kubeClient client.Client, target dbv1alpha1.TargetSpec, namespace string) (*dbv1alpha1.AutonomousDatabase, error) { var err error - var ocid *string var ownerADB *dbv1alpha1.AutonomousDatabase // Get the target ADB OCID if target.K8sADB.Name != nil { + // Find the target ADB using the name of the k8s ADB ownerADB = &dbv1alpha1.AutonomousDatabase{} if err := k8s.FetchResource(kubeClient, namespace, *target.K8sADB.Name, ownerADB); err != nil { - return "", nil, err + return nil, err } - ocid = ownerADB.Spec.Details.AutonomousDatabaseOCID } else { + // Find the target ADB using the ADB OCID ownerADB, err = k8s.FetchAutonomousDatabaseWithOCID(kubeClient, namespace, *target.OCIADB.OCID) if err != nil { - return "", nil, err + return nil, err } - ocid = target.OCIADB.OCID } - if ocid == nil { - ocid = common.String("") - } - - return *ocid, ownerADB, nil + return ownerADB, nil } diff --git a/commons/k8s/fetch.go b/commons/k8s/fetch.go index 6a41b4b1..b8d51b23 100644 --- a/commons/k8s/fetch.go +++ b/commons/k8s/fetch.go @@ -63,7 +63,9 @@ func FetchResource(kubeClient client.Client, namespace string, name string, obje } // Returns the first AutonomousDatabase resource that matches the AutonomousDatabaseOCID of the backup -// If the AutonomousDatabase doesn't exist, returns a nil +// Sometimes the AutonomousDatabase doesn't exist. It could happen if a user simply want to restore or +// backup the ADB without creating an ADB rersource in the cluster. +// If there isn't an AutonomousDatabase with the same OCID, a nil is returned. func FetchAutonomousDatabaseWithOCID(kubeClient client.Client, namespace string, ocid string) (*dbv1alpha1.AutonomousDatabase, error) { adbList, err := fetchAutonomousDatabases(kubeClient, namespace) if err != nil { diff --git a/commons/oci/containerdatabase.go b/commons/oci/containerdatabase.go index 120eb4b3..b8a0ad1e 100644 --- a/commons/oci/containerdatabase.go +++ b/commons/oci/containerdatabase.go @@ -40,7 +40,6 @@ package oci import ( "context" - "time" "github.com/oracle/oci-go-sdk/v63/common" "github.com/oracle/oci-go-sdk/v63/database" @@ -64,42 +63,21 @@ func (d *databaseService) CreateAutonomousContainerDatabase(acd *dbv1alpha1.Auto return d.dbClient.CreateAutonomousContainerDatabase(context.TODO(), createAutonomousContainerDatabaseRequest) } -func (d *databaseService) GetAutonomousContainerDatabase(acdOCID string, retryPolicy *common.RetryPolicy) (database.GetAutonomousContainerDatabaseResponse, error) { +func (d *databaseService) GetAutonomousContainerDatabase(acdOCID string) (database.GetAutonomousContainerDatabaseResponse, error) { getAutonomousContainerDatabaseRequest := database.GetAutonomousContainerDatabaseRequest{ AutonomousContainerDatabaseId: common.String(acdOCID), - RequestMetadata: common.RequestMetadata{RetryPolicy: retryPolicy}, } return d.dbClient.GetAutonomousContainerDatabase(context.TODO(), getAutonomousContainerDatabaseRequest) } -func (d *databaseService) WaitAutonomousContainerDatabaseStatus( - acdOCID string, - attempts uint, - lifecycleState database.AutonomousContainerDatabaseLifecycleStateEnum, - nextDuration func(r common.OCIOperationResponse) time.Duration) (database.GetAutonomousContainerDatabaseResponse, error) { - - shouldRetry := func(r common.OCIOperationResponse) bool { - if acdResponse, ok := r.Response.(database.GetAutonomousContainerDatabaseResponse); ok { - // do the retry until lifecycle state reaches the passed terminal state - return acdResponse.LifecycleState != lifecycleState - } - - return true - } - - retryPolicy := common.NewRetryPolicy(attempts, shouldRetry, nextDuration) - - return d.GetAutonomousContainerDatabase(acdOCID, &retryPolicy) -} - -func (d *databaseService) UpdateAutonomousContainerDatabase(acd *dbv1alpha1.AutonomousContainerDatabase) (database.UpdateAutonomousContainerDatabaseResponse, error) { +func (d *databaseService) UpdateAutonomousContainerDatabase(acdOCID string, difACD *dbv1alpha1.AutonomousContainerDatabase) (database.UpdateAutonomousContainerDatabaseResponse, error) { updateAutonomousContainerDatabaseRequest := database.UpdateAutonomousContainerDatabaseRequest{ - AutonomousContainerDatabaseId: acd.Spec.AutonomousContainerDatabaseOCID, + AutonomousContainerDatabaseId: common.String(acdOCID), UpdateAutonomousContainerDatabaseDetails: database.UpdateAutonomousContainerDatabaseDetails{ - DisplayName: acd.Spec.DisplayName, - PatchModel: database.UpdateAutonomousContainerDatabaseDetailsPatchModelEnum(acd.Spec.PatchModel), - FreeformTags: acd.Spec.FreeformTags, + DisplayName: difACD.Spec.DisplayName, + PatchModel: database.UpdateAutonomousContainerDatabaseDetailsPatchModelEnum(difACD.Spec.PatchModel), + FreeformTags: difACD.Spec.FreeformTags, }, } diff --git a/commons/oci/database.go b/commons/oci/database.go index 5172e12b..93c97f22 100644 --- a/commons/oci/database.go +++ b/commons/oci/database.go @@ -41,7 +41,6 @@ package oci import ( "context" "fmt" - "time" "github.com/go-logr/logr" "github.com/oracle/oci-go-sdk/v63/common" @@ -55,15 +54,15 @@ import ( type DatabaseService interface { CreateAutonomousDatabase(adb *dbv1alpha1.AutonomousDatabase) (database.CreateAutonomousDatabaseResponse, error) GetAutonomousDatabase(adbOCID string) (database.GetAutonomousDatabaseResponse, error) - UpdateAutonomousDatabaseGeneralFields(adb *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) - UpdateAutonomousDatabaseDBWorkload(adb *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) - UpdateAutonomousDatabaseLicenseModel(adb *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) - UpdateAutonomousDatabaseAdminPassword(adb *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) - UpdateAutonomousDatabaseScalingFields(adb *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) + UpdateAutonomousDatabaseGeneralFields(adbOCID string, difADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) + UpdateAutonomousDatabaseDBWorkload(adbOCID string, difADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) + UpdateAutonomousDatabaseLicenseModel(adbOCID string, difADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) + UpdateAutonomousDatabaseAdminPassword(adbOCID string, difADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) + UpdateAutonomousDatabaseScalingFields(adbOCID string, difADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) UpdateNetworkAccessMTLSRequired(adbOCID string) (resp database.UpdateAutonomousDatabaseResponse, err error) - UpdateNetworkAccessMTLS(adb *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) - UpdateNetworkAccessPublic(lastSucSpec dbv1alpha1.NetworkAccessTypeEnum, adbOCID string) (resp database.UpdateAutonomousDatabaseResponse, err error) - UpdateNetworkAccess(adb *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) + UpdateNetworkAccessMTLS(adbOCID string, difADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) + UpdateNetworkAccessPublic(lastAccessType dbv1alpha1.NetworkAccessTypeEnum, adbOCID string) (resp database.UpdateAutonomousDatabaseResponse, err error) + UpdateNetworkAccess(adbOCID string, difADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) StartAutonomousDatabase(adbOCID string) (database.StartAutonomousDatabaseResponse, error) StopAutonomousDatabase(adbOCID string) (database.StopAutonomousDatabaseResponse, error) DeleteAutonomousDatabase(adbOCID string) (database.DeleteAutonomousDatabaseResponse, error) @@ -72,11 +71,9 @@ type DatabaseService interface { ListAutonomousDatabaseBackups(adbOCID string) (database.ListAutonomousDatabaseBackupsResponse, error) CreateAutonomousDatabaseBackup(adbBackup *dbv1alpha1.AutonomousDatabaseBackup, adbOCID string) (database.CreateAutonomousDatabaseBackupResponse, error) GetAutonomousDatabaseBackup(backupOCID string) (database.GetAutonomousDatabaseBackupResponse, error) - CreateAutonomousContainerDatabase(acb *dbv1alpha1.AutonomousContainerDatabase) (database.CreateAutonomousContainerDatabaseResponse, error) - GetAutonomousContainerDatabase(acdOCID string, retryPolicy *common.RetryPolicy) (database.GetAutonomousContainerDatabaseResponse, error) - WaitAutonomousContainerDatabaseStatus(acdOCID string, attempts uint, lifecycleState database.AutonomousContainerDatabaseLifecycleStateEnum, - nextDuration func(r common.OCIOperationResponse) time.Duration) (database.GetAutonomousContainerDatabaseResponse, error) - UpdateAutonomousContainerDatabase(acd *dbv1alpha1.AutonomousContainerDatabase) (database.UpdateAutonomousContainerDatabaseResponse, error) + CreateAutonomousContainerDatabase(acd *dbv1alpha1.AutonomousContainerDatabase) (database.CreateAutonomousContainerDatabaseResponse, error) + GetAutonomousContainerDatabase(acdOCID string) (database.GetAutonomousContainerDatabaseResponse, error) + UpdateAutonomousContainerDatabase(acdOCID string, difACD *dbv1alpha1.AutonomousContainerDatabase) (database.UpdateAutonomousContainerDatabaseResponse, error) RestartAutonomousContainerDatabase(acdOCID string) (database.RestartAutonomousContainerDatabaseResponse, error) TerminateAutonomousContainerDatabase(acdOCID string) (database.TerminateAutonomousContainerDatabaseResponse, error) } @@ -218,47 +215,47 @@ func (d *databaseService) GetAutonomousDatabase(adbOCID string) (database.GetAut return d.dbClient.GetAutonomousDatabase(context.TODO(), getAutonomousDatabaseRequest) } -func (d *databaseService) UpdateAutonomousDatabaseGeneralFields(adb *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { +func (d *databaseService) UpdateAutonomousDatabaseGeneralFields(adbOCID string, difADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ - AutonomousDatabaseId: adb.Spec.Details.AutonomousDatabaseOCID, + AutonomousDatabaseId: common.String(adbOCID), UpdateAutonomousDatabaseDetails: database.UpdateAutonomousDatabaseDetails{ - DisplayName: adb.Spec.Details.DisplayName, - DbName: adb.Spec.Details.DbName, - DbVersion: adb.Spec.Details.DbVersion, - FreeformTags: adb.Spec.Details.FreeformTags, + DisplayName: difADB.Spec.Details.DisplayName, + DbName: difADB.Spec.Details.DbName, + DbVersion: difADB.Spec.Details.DbVersion, + FreeformTags: difADB.Spec.Details.FreeformTags, }, } return d.dbClient.UpdateAutonomousDatabase(context.TODO(), updateAutonomousDatabaseRequest) } -func (d *databaseService) UpdateAutonomousDatabaseDBWorkload(adb *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { +func (d *databaseService) UpdateAutonomousDatabaseDBWorkload(adbOCID string, difADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ - AutonomousDatabaseId: adb.Spec.Details.AutonomousDatabaseOCID, + AutonomousDatabaseId: common.String(adbOCID), UpdateAutonomousDatabaseDetails: database.UpdateAutonomousDatabaseDetails{ - DbWorkload: database.UpdateAutonomousDatabaseDetailsDbWorkloadEnum(adb.Spec.Details.DbWorkload), + DbWorkload: database.UpdateAutonomousDatabaseDetailsDbWorkloadEnum(difADB.Spec.Details.DbWorkload), }, } return d.dbClient.UpdateAutonomousDatabase(context.TODO(), updateAutonomousDatabaseRequest) } -func (d *databaseService) UpdateAutonomousDatabaseLicenseModel(adb *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { +func (d *databaseService) UpdateAutonomousDatabaseLicenseModel(adbOCID string, difADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ - AutonomousDatabaseId: adb.Spec.Details.AutonomousDatabaseOCID, + AutonomousDatabaseId: common.String(adbOCID), UpdateAutonomousDatabaseDetails: database.UpdateAutonomousDatabaseDetails{ - LicenseModel: database.UpdateAutonomousDatabaseDetailsLicenseModelEnum(adb.Spec.Details.LicenseModel), + LicenseModel: database.UpdateAutonomousDatabaseDetailsLicenseModelEnum(difADB.Spec.Details.LicenseModel), }, } return d.dbClient.UpdateAutonomousDatabase(context.TODO(), updateAutonomousDatabaseRequest) } -func (d *databaseService) UpdateAutonomousDatabaseAdminPassword(adb *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { - adminPassword, err := d.readPassword(adb.Namespace, adb.Spec.Details.AdminPassword) +func (d *databaseService) UpdateAutonomousDatabaseAdminPassword(adbOCID string, difADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { + adminPassword, err := d.readPassword(difADB.Namespace, difADB.Spec.Details.AdminPassword) if err != nil { return resp, err } updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ - AutonomousDatabaseId: adb.Spec.Details.AutonomousDatabaseOCID, + AutonomousDatabaseId: common.String(adbOCID), UpdateAutonomousDatabaseDetails: database.UpdateAutonomousDatabaseDetails{ AdminPassword: adminPassword, }, @@ -266,13 +263,13 @@ func (d *databaseService) UpdateAutonomousDatabaseAdminPassword(adb *dbv1alpha1. return d.dbClient.UpdateAutonomousDatabase(context.TODO(), updateAutonomousDatabaseRequest) } -func (d *databaseService) UpdateAutonomousDatabaseScalingFields(adb *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { +func (d *databaseService) UpdateAutonomousDatabaseScalingFields(adbOCID string, difADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ - AutonomousDatabaseId: adb.Spec.Details.AutonomousDatabaseOCID, + AutonomousDatabaseId: common.String(adbOCID), UpdateAutonomousDatabaseDetails: database.UpdateAutonomousDatabaseDetails{ - DataStorageSizeInTBs: adb.Spec.Details.DataStorageSizeInTBs, - CpuCoreCount: adb.Spec.Details.CPUCoreCount, - IsAutoScalingEnabled: adb.Spec.Details.IsAutoScalingEnabled, + DataStorageSizeInTBs: difADB.Spec.Details.DataStorageSizeInTBs, + CpuCoreCount: difADB.Spec.Details.CPUCoreCount, + IsAutoScalingEnabled: difADB.Spec.Details.IsAutoScalingEnabled, }, } return d.dbClient.UpdateAutonomousDatabase(context.TODO(), updateAutonomousDatabaseRequest) @@ -288,18 +285,20 @@ func (d *databaseService) UpdateNetworkAccessMTLSRequired(adbOCID string) (resp return d.dbClient.UpdateAutonomousDatabase(context.TODO(), updateAutonomousDatabaseRequest) } -func (d *databaseService) UpdateNetworkAccessMTLS(adb *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { +func (d *databaseService) UpdateNetworkAccessMTLS(adbOCID string, difADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ - AutonomousDatabaseId: adb.Spec.Details.AutonomousDatabaseOCID, + AutonomousDatabaseId: common.String(adbOCID), UpdateAutonomousDatabaseDetails: database.UpdateAutonomousDatabaseDetails{ - IsMtlsConnectionRequired: adb.Spec.Details.NetworkAccess.IsMTLSConnectionRequired, + IsMtlsConnectionRequired: difADB.Spec.Details.NetworkAccess.IsMTLSConnectionRequired, }, } return d.dbClient.UpdateAutonomousDatabase(context.TODO(), updateAutonomousDatabaseRequest) } -func (d *databaseService) UpdateNetworkAccessPublic(lastAccessType dbv1alpha1.NetworkAccessTypeEnum, +func (d *databaseService) UpdateNetworkAccessPublic( + lastAccessType dbv1alpha1.NetworkAccessTypeEnum, adbOCID string) (resp database.UpdateAutonomousDatabaseResponse, err error) { + updateAutonomousDatabaseDetails := database.UpdateAutonomousDatabaseDetails{} if lastAccessType == dbv1alpha1.NetworkAccessTypeRestricted { @@ -316,15 +315,15 @@ func (d *databaseService) UpdateNetworkAccessPublic(lastAccessType dbv1alpha1.Ne return d.dbClient.UpdateAutonomousDatabase(context.TODO(), updateAutonomousDatabaseRequest) } -func (d *databaseService) UpdateNetworkAccess(adb *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { +func (d *databaseService) UpdateNetworkAccess(adbOCID string, difADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ - AutonomousDatabaseId: adb.Spec.Details.AutonomousDatabaseOCID, + AutonomousDatabaseId: common.String(adbOCID), UpdateAutonomousDatabaseDetails: database.UpdateAutonomousDatabaseDetails{ - IsAccessControlEnabled: adb.Spec.Details.NetworkAccess.IsAccessControlEnabled, - WhitelistedIps: adb.Spec.Details.NetworkAccess.AccessControlList, - SubnetId: adb.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID, - NsgIds: adb.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs, - PrivateEndpointLabel: adb.Spec.Details.NetworkAccess.PrivateEndpoint.HostnamePrefix, + IsAccessControlEnabled: difADB.Spec.Details.NetworkAccess.IsAccessControlEnabled, + WhitelistedIps: difADB.Spec.Details.NetworkAccess.AccessControlList, + SubnetId: difADB.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID, + NsgIds: difADB.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs, + PrivateEndpointLabel: difADB.Spec.Details.NetworkAccess.PrivateEndpoint.HostnamePrefix, }, } diff --git a/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml b/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml index cd9e3729..0541b985 100644 --- a/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml +++ b/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml @@ -90,11 +90,6 @@ spec: description: AutonomousDatabaseBackupStatus defines the observed state of AutonomousDatabaseBackup properties: - autonomousDatabaseBackupOCID: - description: 'INSERT ADDITIONAL STATUS FIELD - define observed state - of cluster Important: Run "make" to regenerate code after modifying - this file' - type: string autonomousDatabaseOCID: type: string compartmentOCID: @@ -103,11 +98,11 @@ spec: type: string dbName: type: string - displayName: - type: string isAutomatic: type: boolean lifecycleState: + description: 'AutonomousDatabaseBackupLifecycleStateEnum Enum with + underlying type: string' type: string timeEnded: type: string @@ -122,7 +117,6 @@ spec: - compartmentOCID - dbDisplayName - dbName - - displayName - isAutomatic - lifecycleState - type diff --git a/config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml b/config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml index bf1b2d51..5e9f2c73 100644 --- a/config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml +++ b/config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml @@ -101,8 +101,6 @@ spec: description: AutonomousDatabaseRestoreStatus defines the observed state of AutonomousDatabaseRestore properties: - autonomousDatabaseOCID: - type: string dbName: type: string displayName: @@ -111,6 +109,7 @@ spec: this file' type: string status: + description: 'WorkRequestStatusEnum Enum with underlying type: string' type: string timeAccepted: type: string @@ -118,11 +117,13 @@ spec: type: string timeStarted: type: string + workRequestOCID: + type: string required: - - autonomousDatabaseOCID - dbName - displayName - status + - workRequestOCID type: object type: object served: true diff --git a/config/crd/bases/database.oracle.com_autonomousdatabases.yaml b/config/crd/bases/database.oracle.com_autonomousdatabases.yaml index 1701111c..c70b9dd6 100644 --- a/config/crd/bases/database.oracle.com_autonomousdatabases.yaml +++ b/config/crd/bases/database.oracle.com_autonomousdatabases.yaml @@ -23,6 +23,9 @@ spec: - jsonPath: .spec.details.displayName name: Display Name type: string + - jsonPath: .spec.details.dbName + name: Db Name + type: string - jsonPath: .status.lifecycleState name: State type: string diff --git a/config/samples/acd/autonomouscontainerdatabase_sync.yaml b/config/samples/acd/autonomouscontainerdatabase_sync.yaml deleted file mode 100644 index b758fe9d..00000000 --- a/config/samples/acd/autonomouscontainerdatabase_sync.yaml +++ /dev/null @@ -1,16 +0,0 @@ -# -# Copyright (c) 2021, Oracle and/or its affiliates. -# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. -# -apiVersion: database.oracle.com/v1alpha1 -kind: AutonomousContainerDatabase -metadata: - name: autonomouscontainerdatabase-sample -spec: - autonomousContainerDatabaseOCID: ocid1.autonomouscontainerdatabase... - action: SYNC - - # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. - ociConfig: - configMapName: oci-cred - secretName: oci-privatekey diff --git a/config/webhook/manifests.yaml b/config/webhook/manifests.yaml index c7fd1c9d..61d66981 100644 --- a/config/webhook/manifests.yaml +++ b/config/webhook/manifests.yaml @@ -138,6 +138,26 @@ metadata: creationTimestamp: null name: validating-webhook-configuration webhooks: +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: webhook-service + namespace: system + path: /validate-database-oracle-com-v1alpha1-autonomouscontainerdatabase + failurePolicy: Fail + name: vautonomouscontainerdatabase.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + resources: + - autonomouscontainerdatabases + sideEffects: None - admissionReviewVersions: - v1 clientConfig: @@ -155,7 +175,6 @@ webhooks: operations: - CREATE - UPDATE - - DELETE resources: - autonomousdatabases sideEffects: None diff --git a/controllers/database/autonomouscontainerdatabase_controller.go b/controllers/database/autonomouscontainerdatabase_controller.go index 2b6c9808..a71267ca 100644 --- a/controllers/database/autonomouscontainerdatabase_controller.go +++ b/controllers/database/autonomouscontainerdatabase_controller.go @@ -43,18 +43,14 @@ import ( "encoding/json" "errors" "reflect" - "time" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v63/common" "github.com/oracle/oci-go-sdk/v63/database" corev1 "k8s.io/api/core/v1" apiErrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/runtime" - "k8s.io/apimachinery/pkg/types" "k8s.io/client-go/tools/record" - "k8s.io/client-go/util/retry" ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/controller-runtime/pkg/controller" @@ -75,9 +71,7 @@ type AutonomousContainerDatabaseReconciler struct { Scheme *runtime.Scheme Recorder record.EventRecorder - dbService oci.DatabaseService - workService oci.WorkRequestService - lastSucSpec *dbv1alpha1.AutonomousContainerDatabaseSpec + dbService oci.DatabaseService } // SetupWithManager sets up the controller with the Manager. @@ -89,20 +83,6 @@ func (r *AutonomousContainerDatabaseReconciler) SetupWithManager(mgr ctrl.Manage Complete(r) } -func (r *AutonomousContainerDatabaseReconciler) isIntermediateState(state database.AutonomousContainerDatabaseLifecycleStateEnum) bool { - if state == database.AutonomousContainerDatabaseLifecycleStateProvisioning || - state == database.AutonomousContainerDatabaseLifecycleStateUpdating || - state == database.AutonomousContainerDatabaseLifecycleStateTerminating || - state == database.AutonomousContainerDatabaseLifecycleStateBackupInProgress || - state == database.AutonomousContainerDatabaseLifecycleStateRestoring || - state == database.AutonomousContainerDatabaseLifecycleStateRestarting || - state == database.AutonomousContainerDatabaseLifecycleStateMaintenanceInProgress || - state == database.AutonomousContainerDatabaseLifecycleStateRoleChangeInProgress { - return true - } - return false -} - func (r *AutonomousContainerDatabaseReconciler) eventFilterPredicate() predicate.Predicate { pred := predicate.Funcs{ UpdateFunc: func(e event.UpdateEvent) bool { @@ -110,28 +90,20 @@ func (r *AutonomousContainerDatabaseReconciler) eventFilterPredicate() predicate if acdOk { oldACD := e.ObjectOld.(*dbv1alpha1.AutonomousContainerDatabase) - oldSucSpec, oldSucSpecOk := oldACD.GetAnnotations()[dbv1alpha1.LastSuccessfulSpec] - newSucSpec, newSucSpecOk := desiredACD.GetAnnotations()[dbv1alpha1.LastSuccessfulSpec] - sucSpecChanged := oldSucSpecOk != newSucSpecOk || oldSucSpec != newSucSpec - - if !reflect.DeepEqual(oldACD.Status, desiredACD.Status) || sucSpecChanged { + if !reflect.DeepEqual(oldACD.Status, desiredACD.Status) || + (controllerutil.ContainsFinalizer(oldACD, dbv1alpha1.ACDFinalizer) != controllerutil.ContainsFinalizer(desiredACD, dbv1alpha1.ACDFinalizer)) { // Don't enqueue if the status or the lastSucSpec changes return false } return true } - return true }, DeleteFunc: func(e event.DeleteEvent) bool { // Do not trigger reconciliation when the object is deleted from the cluster. _, acdOk := e.Object.(*dbv1alpha1.AutonomousContainerDatabase) - if acdOk { - return false - } - - return true + return !acdOk }, } @@ -148,9 +120,10 @@ func (r *AutonomousContainerDatabaseReconciler) eventFilterPredicate() predicate // For more details, check Reconcile and its Result here: // - https://pkg.go.dev/sigs.k8s.io/controller-runtime@v0.6.4/pkg/reconcile func (r *AutonomousContainerDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { - logger := r.Log.WithValues("Namespaced/Name", req.NamespacedName) + logger := r.Log.WithValues("Namespace/Name", req.NamespacedName) var err error + var ociACD *dbv1alpha1.AutonomousContainerDatabase // Get the autonomousdatabase instance from the cluster acd := &dbv1alpha1.AutonomousContainerDatabase{} @@ -164,80 +137,89 @@ func (r *AutonomousContainerDatabaseReconciler) Reconcile(ctx context.Context, r return emptyResult, err } - r.lastSucSpec, err = acd.GetLastSuccessfulSpec() - if err != nil { - return r.manageError(acd, err) - } - /****************************************************************** - * Get OCI database client and work request client + * Get OCI database client ******************************************************************/ - if err := r.setupOCIClients(acd); err != nil { + if err := r.setupOCIClients(logger, acd); err != nil { logger.Error(err, "Fail to setup OCI clients") - return r.manageError(acd, err) + return r.manageError(logger, acd, err) } logger.Info("OCI clients configured succesfully") /****************************************************************** - * Register/unregister finalizer + * Get OCI AutonomousDatabase + ******************************************************************/ + + if acd.Spec.AutonomousContainerDatabaseOCID != nil { + resp, err := r.dbService.GetAutonomousContainerDatabase(*acd.Spec.AutonomousContainerDatabaseOCID) + if err != nil { + return r.manageError(logger, acd, err) + } + + ociACD = &dbv1alpha1.AutonomousContainerDatabase{} + ociACD.UpdateFromOCIACD(resp.AutonomousContainerDatabase) + } + + /****************************************************************** + * Requeue if the ACD is in an intermediate state + * No-op if the ACD OCID is nil + * To get the latest status, execute before all the reconcile logic + ******************************************************************/ + needsRequeue, err := r.validateLifecycleState(logger, acd, ociACD) + if err != nil { + return r.manageError(logger, acd, err) + } + + if needsRequeue { + return requeueResult, nil + } + + /****************************************************************** + * Cleanup the resource if the resource is to be deleted. * Deletion timestamp will be added to a object before it is deleted. * Kubernetes server calls the clean up function if a finalizer exitsts, and won't delete the real object until * all the finalizers are removed from the object metadata. * Refer to this page for more details of using finalizers: https://kubernetes.io/blog/2021/05/14/using-finalizers-to-control-deletion/ ******************************************************************/ - isACDDeleteTrue, err := r.validateFinalizer(acd) + exitReconcile, err := r.validateCleanup(logger, acd) if err != nil { - return r.manageError(acd, err) + return r.manageError(logger, acd, err) } - if isACDDeleteTrue { + + if exitReconcile { return emptyResult, nil } /****************************************************************** - * Determine which Database operations need to be executed by checking the changes to spec. - * There are three scenario: - * 1. provision operation. The ACD OCID is missing, and the LastSucSpec annotation is missing. - * 2. bind operation. The ACD OCID is provided, but the LastSucSpec annotation is missing. - * 3. sync operation. The action field is SYNC. - * 4. update operation. The changes which are not provision, bind or sync operations is an update operation. - * Afterwards, update the resource from the remote database in OCI. This step will be executed right after - * the above three cases during every reconcile. - /******************************************************************/ - // difACD is nil when the action is PROVISION or BIND. - // Use difACD to identify which fields are updated when it's an UPDATE operation. - action, difACD, err := r.determineAction(acd) - if err != nil { - return r.manageError(acd, err) + * Register/unregister the finalizer + ******************************************************************/ + if err := r.validateFinalizer(acd); err != nil { + return r.manageError(logger, acd, err) } - switch action { - case acdRecActionProvision: - if err := r.createACD(acd); err != nil { - return r.manageError(acd, err) - } - case acdRecActionBind: - break - case acdRecActionUpdate: - // updateACD contains downloadWallet - if err := r.updateACD(acd, difACD); err != nil { - return r.manageError(acd, err) - } - case acdRecActionSync: - break + /****************************************************************** + * Validate operations + ******************************************************************/ + exitReconcile, result, err := r.validateOperation(logger, acd, ociACD) + if err != nil { + return r.manageError(logger, acd, err) + } + if exitReconcile { + return result, nil } - /***************************************************** - * Sync resource and update the lastSucSpec - *****************************************************/ - specUpdated, err := r.syncResource(acd) - if err != nil { - return r.manageError(acd, err) + /****************************************************************** + * Update the status and requeue if it's in an intermediate state + ******************************************************************/ + if err := r.KubeClient.Status().Update(context.TODO(), acd); err != nil { + return r.manageError(logger, acd, err) } - if specUpdated { - return emptyResult, nil + if dbv1alpha1.IsACDIntermediateState(acd.Status.LifecycleState) { + logger.WithName("IsIntermediateState").Info("Current lifecycleState is " + string(acd.Status.LifecycleState) + "; reconcile queued") + return requeueResult, nil } logger.Info("AutonomousContainerDatabase reconciles successfully") @@ -245,124 +227,55 @@ func (r *AutonomousContainerDatabaseReconciler) Reconcile(ctx context.Context, r return emptyResult, nil } -// updateResourceStatus updates only the status of the resource, not including the lastSucSpec. -// This function should not be called by the functions associated with the OCI update requests. -// The OCI update requests should use updateResource() to ensure all the spec, resource and the -// lastSucSpec are updated. -func (r *AutonomousContainerDatabaseReconciler) updateResourceStatus(acd *dbv1alpha1.AutonomousContainerDatabase) error { - return retry.RetryOnConflict(retry.DefaultRetry, func() error { - curACD := &dbv1alpha1.AutonomousContainerDatabase{} - - namespacedName := types.NamespacedName{ - Namespace: acd.GetNamespace(), - Name: acd.GetName(), - } - - if err := r.KubeClient.Get(context.TODO(), namespacedName, curACD); err != nil { - return err - } - - curACD.Status = acd.Status - return r.KubeClient.Status().Update(context.TODO(), curACD) - }) -} - -// updateResource updates the specification, the status of AutonomousContainerDatabase resource -func (r *AutonomousContainerDatabaseReconciler) updateResource(acd *dbv1alpha1.AutonomousContainerDatabase) error { - // Update the spec - if err := retry.RetryOnConflict(retry.DefaultRetry, func() error { - curACD := &dbv1alpha1.AutonomousContainerDatabase{} - - namespacedName := types.NamespacedName{ - Namespace: acd.GetNamespace(), - Name: acd.GetName(), - } - - if err := r.KubeClient.Get(context.TODO(), namespacedName, curACD); err != nil { - return err - } +func (r *AutonomousContainerDatabaseReconciler) setupOCIClients(logger logr.Logger, acd *dbv1alpha1.AutonomousContainerDatabase) error { + var err error - curACD.Spec = acd.Spec - return r.KubeClient.Update(context.TODO(), curACD) - }); err != nil { - return err + authData := oci.APIKeyAuth{ + ConfigMapName: acd.Spec.OCIConfig.ConfigMapName, + SecretName: acd.Spec.OCIConfig.SecretName, + Namespace: acd.GetNamespace(), } - return nil -} - -// syncResource pulled information from OCI, update the lastSucSpec, and lastly, update the local resource. -// It's important to update the lastSucSpec prior than we update the local resource, because updating -// the local resource triggers the Reconcile, and the Reconcile determines the action by looking if -// the lastSucSpec is present. -// The function returns if the spec is updated (and the Reconcile is triggered). -func (r *AutonomousContainerDatabaseReconciler) syncResource(acd *dbv1alpha1.AutonomousContainerDatabase) (bool, error) { - // Get the information from OCI - resp, err := r.dbService.GetAutonomousContainerDatabase(*acd.Spec.AutonomousContainerDatabaseOCID, nil) + provider, err := oci.GetOCIProvider(r.KubeClient, authData) if err != nil { - return false, err - } - - specChanged, statusChanged := acd.UpdateFromOCIACD(resp.AutonomousContainerDatabase) - - // Validate the status change - if statusChanged { - if err := r.updateResourceStatus(acd); err != nil { - return false, err - } - } - - // Validate the spec change - // If the spec changes, update the lastSucSpec and the resource, and then exit the Reconcile since it triggers another round of the Reconcile. - if specChanged { - if err := r.updateLastSuccessfulSpec(acd); err != nil { - return false, err - } - - if err := r.updateResource(acd); err != nil { - return false, err - } - - return true, nil + return err } - return false, nil -} - -// updateLastSuccessfulSpec updates the lasSucSpec annotation, and returns the ORIGINAL object -// The object will NOT be updated with the returned content from the cluster since we want to -// update only the lasSucSpec. For example: After we get the ACD information from OCI, we want -// to update the lasSucSpec before updating the local resource. In this case, we want to keep -// the content returned from the OCI, not the one from the cluster. -func (r *AutonomousContainerDatabaseReconciler) updateLastSuccessfulSpec(acd *dbv1alpha1.AutonomousContainerDatabase) error { - specBytes, err := json.Marshal(acd.Spec) + r.dbService, err = oci.NewDatabaseService(logger, r.KubeClient, provider) if err != nil { return err } - anns := map[string]string{ - dbv1alpha1.LastSuccessfulSpec: string(specBytes), - } - - copyACD := acd.DeepCopy() - - return annotations.PatchAnnotations(r.KubeClient, copyACD, anns) + return nil } -func (r *AutonomousContainerDatabaseReconciler) manageError(acd *dbv1alpha1.AutonomousContainerDatabase, issue error) (ctrl.Result, error) { - // Rollback if lastSucSpec exists - if r.lastSucSpec != nil { +func (r *AutonomousContainerDatabaseReconciler) manageError(logger logr.Logger, acd *dbv1alpha1.AutonomousContainerDatabase, issue error) (ctrl.Result, error) { + l := logger.WithName("manageError") + + // Has synced at least once + if acd.Status.LifecycleState != "" { // Send event - r.Recorder.Event(acd, corev1.EventTypeWarning, "ReconcileIncompleted", issue.Error()) + r.Recorder.Event(acd, corev1.EventTypeWarning, "UpdateFailed", issue.Error()) var finalIssue = issue // Roll back - if _, err := r.syncResource(acd); err != nil { + specChanged, err := r.getACD(logger, acd) + if err != nil { finalIssue = k8s.CombineErrors(finalIssue, err) } - return emptyResult, finalIssue + // We don't exit the Reconcile if the spec has changed + // becasue it will exit anyway after the manageError is called. + if specChanged { + if err := r.KubeClient.Update(context.TODO(), acd); err != nil { + finalIssue = k8s.CombineErrors(finalIssue, err) + } + } + + l.Error(finalIssue, "UpdateFailed") + + return emptyResult, nil } else { // Send event r.Recorder.Event(acd, corev1.EventTypeWarning, "CreateFailed", issue.Error()) @@ -371,238 +284,370 @@ func (r *AutonomousContainerDatabaseReconciler) manageError(acd *dbv1alpha1.Auto } } -func (r *AutonomousContainerDatabaseReconciler) setupOCIClients(acd *dbv1alpha1.AutonomousContainerDatabase) error { - var err error - - authData := oci.APIKeyAuth{ - ConfigMapName: acd.Spec.OCIConfig.ConfigMapName, - SecretName: acd.Spec.OCIConfig.SecretName, - Namespace: acd.GetNamespace(), +// validateLifecycleState gets and validates the current lifecycleState +func (r *AutonomousContainerDatabaseReconciler) validateLifecycleState(logger logr.Logger, acd *dbv1alpha1.AutonomousContainerDatabase, ociACD *dbv1alpha1.AutonomousContainerDatabase) (needsRequeue bool, err error) { + if ociACD == nil { + return false, nil } - provider, err := oci.GetOCIProvider(r.KubeClient, authData) - if err != nil { - return err + l := logger.WithName("validateLifecycleState") + + // Special case: Once the status changes to AVAILABLE after the provision operation, the reconcile stops. + // The backup starts right after the provision operation and the controller is not able to track the operation in this case. + // To prevent this issue, requeue the reconcile if the previous status is PROVISIONING and we ignore the status change + // until it becomes BACKUP_IN_PROGRESS. + if acd.Status.LifecycleState == database.AutonomousContainerDatabaseLifecycleStateProvisioning && + ociACD.Status.LifecycleState != database.AutonomousContainerDatabaseLifecycleStateBackupInProgress { + l.Info("Provisioning the ACD and waiting for the backup to start; reconcile queued") + return true, nil } - r.dbService, err = oci.NewDatabaseService(r.Log, r.KubeClient, provider) - if err != nil { - return err + acd.Status = ociACD.Status + + if err := r.KubeClient.Status().Update(context.TODO(), acd); err != nil { + return false, err } - r.workService, err = oci.NewWorkRequestService(r.Log, r.KubeClient, provider) - if err != nil { - return err + if dbv1alpha1.IsACDIntermediateState(ociACD.Status.LifecycleState) { + l.Info("LifecycleState is " + string(acd.Status.LifecycleState) + "; reconcile queued") + return true, nil } - return nil + return false, nil } -func (r *AutonomousContainerDatabaseReconciler) validateFinalizer(acd *dbv1alpha1.AutonomousContainerDatabase) (isACDDeleteTrue bool, err error) { - logger := r.Log.WithName("finalizer") +func (r *AutonomousContainerDatabaseReconciler) validateCleanup(logger logr.Logger, acd *dbv1alpha1.AutonomousContainerDatabase) (exitReconcile bool, err error) { + l := logger.WithName("validateCleanup") isACDToBeDeleted := acd.GetDeletionTimestamp() != nil - if isACDToBeDeleted { - if controllerutil.ContainsFinalizer(acd, dbv1alpha1.ACDFinalizer) { - if (r.lastSucSpec != nil && r.lastSucSpec.AutonomousContainerDatabaseOCID != nil) && // lastSucSpec exists and the ACD_OCID isn't nil - (acd.Status.LifecycleState != database.AutonomousContainerDatabaseLifecycleStateTerminating && acd.Status.LifecycleState != database.AutonomousContainerDatabaseLifecycleStateTerminated) { - // Run finalization logic for finalizer. If the finalization logic fails, don't remove the finalizer so - // that we can retry during the next reconciliation. - logger.Info("Terminating Autonomous Container Database: " + *acd.Spec.DisplayName) - if err := r.terminateACD(acd); err != nil { - return false, err - } - } else { - logger.Info("Missing AutonomousContaineratabaseOCID to terminate Autonomous Container Database", "Name", acd.Name, "Namespace", acd.Namespace) + + if !isACDToBeDeleted { + return false, nil + } + + if controllerutil.ContainsFinalizer(acd, dbv1alpha1.ACDFinalizer) { + if acd.Status.LifecycleState == database.AutonomousContainerDatabaseLifecycleStateTerminating { + l.Info("Resource is already in TERMINATING state") + // Delete in progress, continue with the reconcile logic + return false, nil + } + + if acd.Status.LifecycleState == database.AutonomousContainerDatabaseLifecycleStateTerminated { + // The acd has been deleted. Remove the finalizer and exit the reconcile. + // Once all finalizers have been removed, the object will be deleted. + l.Info("Resource is already in TERMINATED state; remove the finalizer") + if err := k8s.RemoveFinalizerAndPatch(r.KubeClient, acd, dbv1alpha1.ACDFinalizer); err != nil { + return false, err } + return true, nil + } - // Remove finalizer. Once all finalizers have been - // removed, the object will be deleted. + if acd.Spec.AutonomousContainerDatabaseOCID == nil { + l.Info("Missing AutonomousContainerDatabaseOCID to terminate Autonomous Container Database; remove the finalizer anyway", "Name", acd.Name, "Namespace", acd.Namespace) + // Remove finalizer anyway. if err := k8s.RemoveFinalizerAndPatch(r.KubeClient, acd, dbv1alpha1.ACDFinalizer); err != nil { - return false, nil + return false, err } + return true, nil } - // Send true because delete is in progress and it is a custom delete message - // We don't need to print custom err stack as we are deleting the topology - return true, nil + + if acd.Spec.Action != dbv1alpha1.AcdActionTerminate { + // Run finalization logic for finalizer. If the finalization logic fails, don't remove the finalizer so + // that we can retry during the next reconciliation. + l.Info("Terminating Autonomous Database: " + *acd.Spec.DisplayName) + acd.Spec.Action = dbv1alpha1.AcdActionTerminate + if err := r.KubeClient.Update(context.TODO(), acd); err != nil { + return false, err + } + // Exit the reconcile since we have updated the spec + return true, nil + } + + // Continue with the reconcile logic + return false, nil } + // Exit the Reconcile since the to-be-deleted resource doesn't has a finalizer + return true, nil +} + +func (r *AutonomousContainerDatabaseReconciler) validateFinalizer(acd *dbv1alpha1.AutonomousContainerDatabase) error { // Delete is not schduled. Update the finalizer for this CR if hardLink is present if acd.Spec.HardLink != nil { if *acd.Spec.HardLink { if err := k8s.AddFinalizerAndPatch(r.KubeClient, acd, dbv1alpha1.ACDFinalizer); err != nil { - return false, nil + return err } } else { if err := k8s.RemoveFinalizerAndPatch(r.KubeClient, acd, dbv1alpha1.ACDFinalizer); err != nil { - return false, nil + return err } } } - return false, nil + return nil } -type acdRecActionEnum string +func (r *AutonomousContainerDatabaseReconciler) validateOperation( + logger logr.Logger, + acd *dbv1alpha1.AutonomousContainerDatabase, + ociACD *dbv1alpha1.AutonomousContainerDatabase) (exitReconcile bool, result ctrl.Result, err error) { -const ( - acdRecActionProvision acdRecActionEnum = "PROVISION" - acdRecActionBind acdRecActionEnum = "BIND" - acdRecActionUpdate acdRecActionEnum = "UPDATE" - acdRecActionSync acdRecActionEnum = "SYNC" -) + l := logger.WithName("validateOperation") -func (r *AutonomousContainerDatabaseReconciler) determineAction(acd *dbv1alpha1.AutonomousContainerDatabase) (acdRecActionEnum, *dbv1alpha1.AutonomousContainerDatabase, error) { - if r.lastSucSpec == nil { + lastSpec, err := acd.GetLastSuccessfulSpec() + if err != nil { + return false, emptyResult, err + } + + // If lastSucSpec is nil, then it's CREATE or BIND opertaion + if lastSpec == nil { if acd.Spec.AutonomousContainerDatabaseOCID == nil { - return acdRecActionProvision, nil, nil + l.Info("Create operation") + + err := r.createACD(logger, acd) + if err != nil { + return false, emptyResult, err + } + + // Update the ACD OCID + if err := r.updateCR(acd); err != nil { + return false, emptyResult, err + } + + l.Info("AutonomousContainerDatabaseOCID updated; exit reconcile") + return true, emptyResult, nil } else { - return acdRecActionBind, nil, nil + l.Info("Bind operation") + + _, err := r.getACD(logger, acd) + if err != nil { + return false, emptyResult, err + } + + if err := r.updateCR(acd); err != nil { + return false, emptyResult, err + } + + l.Info("spec updated; exit reconcile") + return false, emptyResult, nil } - } else { - // Pre-process step for the UPDATE. Remove the unchanged fields from spec.details, + } + + // If it's not CREATE or BIND opertaion, then UPDATE or SYNC + // Compare with the lastSucSpec.details. If the details are different, it means that the user updates the spec. + lastDifACD := acd.DeepCopy() + + lastDetailsChanged, err := lastDifACD.RemoveUnchangedSpec(*lastSpec) + if err != nil { + return false, emptyResult, err + } + + if lastDetailsChanged { + l.Info("Update operation") + + // Double check if the user input spec is actually different from the spec in OCI. If so, then update the resource. + difACD := acd.DeepCopy() - specChanged, err := difACD.RemoveUnchangedSpec() + + ociDetailsChanged, err := difACD.RemoveUnchangedSpec(ociACD.Spec) if err != nil { - return "", nil, err + return false, emptyResult, err } - if specChanged { - // Return SYNC if the spec.action is SYNC - if difACD.Spec.Action == dbv1alpha1.AcdActionSync { - return acdRecActionSync, nil, nil + if ociDetailsChanged { + ociReqSent, specChanged, err := r.updateACD(logger, acd, difACD) + if err != nil { + return false, emptyResult, err } - return acdRecActionUpdate, difACD, nil + + // Requeue the k8s request if an OCI request is sent, since OCI can only process one request at a time. + if ociReqSent { + if specChanged { + if err := r.KubeClient.Update(context.TODO(), acd); err != nil { + return false, emptyResult, err + } + + l.Info("spec updated; exit reconcile") + return false, emptyResult, nil + + } else { + l.Info("reconcile queued") + return true, requeueResult, nil + } + } + } + + // Stop the update and patch the lastSpec when the current ACD matches the oci ACD. + if err := r.patchLastSuccessfulSpec(acd); err != nil { + return false, emptyResult, err + } + + return false, emptyResult, nil + + } else { + l.Info("No operation specified; sync the resource") + + // The user doesn't change the spec and the controller should pull the spec from the OCI. + specChanged, err := r.getACD(logger, acd) + if err != nil { + return false, emptyResult, err } - return acdRecActionSync, nil, nil + if specChanged { + l.Info("The local spec doesn't match the oci's spec; update the CR") + if err := r.updateCR(acd); err != nil { + return false, emptyResult, err + } + + return true, emptyResult, nil + } + return false, emptyResult, nil } } -func (r *AutonomousContainerDatabaseReconciler) createACD(acd *dbv1alpha1.AutonomousContainerDatabase) error { - logger := r.Log.WithName("provision-ACD") - resp, err := r.dbService.CreateAutonomousContainerDatabase(acd) - if err != nil { +func (r *AutonomousContainerDatabaseReconciler) updateCR(acd *dbv1alpha1.AutonomousContainerDatabase) error { + // Update the lastSucSpec + if err := acd.UpdateLastSuccessfulSpec(); err != nil { return err } - // Update the ACD OCID and the status - // The trick is to update the status first to prevent unwanted reconcile - acd.Spec.AutonomousContainerDatabaseOCID = resp.AutonomousContainerDatabase.Id - acd.UpdateStatusFromOCIACD(resp.AutonomousContainerDatabase) - - if err := r.updateResourceStatus(acd); err != nil { + if err := r.KubeClient.Update(context.TODO(), acd); err != nil { return err } + return nil +} - // Patching is faster - if err := k8s.Patch(r.KubeClient, acd, "/spec/autonomousContainerDatabaseOCID", acd.Spec.AutonomousContainerDatabaseOCID); err != nil { +func (r *AutonomousContainerDatabaseReconciler) patchLastSuccessfulSpec(acd *dbv1alpha1.AutonomousContainerDatabase) error { + specBytes, err := json.Marshal(acd.Spec) + if err != nil { return err } - // Wait for the provision operation to finish - if _, err := r.workService.Wait(*resp.OpcWorkRequestId); err != nil { - return err + anns := map[string]string{ + dbv1alpha1.LastSuccessfulSpec: string(specBytes), } + annotations.PatchAnnotations(r.KubeClient, acd, anns) + return nil } -func (r *AutonomousContainerDatabaseReconciler) updateGeneralFields(acd *dbv1alpha1.AutonomousContainerDatabase, difACD *dbv1alpha1.AutonomousContainerDatabase) error { - if difACD.Spec.DisplayName == nil && - difACD.Spec.PatchModel == "" && - difACD.Spec.FreeformTags == nil { - return nil - } +func (r *AutonomousContainerDatabaseReconciler) createACD(logger logr.Logger, acd *dbv1alpha1.AutonomousContainerDatabase) error { + logger.WithName("createACD").Info("Sending CreateAutonomousContainerDatabase request to OCI") - resp, err := r.dbService.UpdateAutonomousContainerDatabase(difACD) + resp, err := r.dbService.CreateAutonomousContainerDatabase(acd) if err != nil { return err } - // If the OpcWorkRequestId is nil (such as when the displayName is changed), - // no need to update the resource and wail until the work is done - if resp.OpcWorkRequestId == nil { - return nil - } + acd.UpdateFromOCIACD(resp.AutonomousContainerDatabase) - acd.UpdateStatusFromOCIACD(resp.AutonomousContainerDatabase) - if err := r.updateResourceStatus(acd); err != nil { - return err + return nil +} + +func (r *AutonomousContainerDatabaseReconciler) getACD(logger logr.Logger, acd *dbv1alpha1.AutonomousContainerDatabase) (bool, error) { + if acd == nil { + return false, errors.New("AutonomousContainerDatabase OCID is missing") } - if _, err := r.workService.Wait(*resp.OpcWorkRequestId); err != nil { - return err + logger.WithName("getACD").Info("Sending GetAutonomousContainerDatabase request to OCI") + + // Get the information from OCI + resp, err := r.dbService.GetAutonomousContainerDatabase(*acd.Spec.AutonomousContainerDatabaseOCID) + if err != nil { + return false, err } - return nil + + specChanged := acd.UpdateFromOCIACD(resp.AutonomousContainerDatabase) + + return specChanged, nil } -func (r *AutonomousContainerDatabaseReconciler) terminateACD(acd *dbv1alpha1.AutonomousContainerDatabase) error { +// updateACD returns true if an OCI request is sent. +// The AutonomousContainerDatabase is updated with the returned object from the OCI requests. +func (r *AutonomousContainerDatabaseReconciler) updateACD( + logger logr.Logger, + acd *dbv1alpha1.AutonomousContainerDatabase, + difACD *dbv1alpha1.AutonomousContainerDatabase) (ociReqSent bool, specChanged bool, err error) { - resp, err := r.dbService.TerminateAutonomousContainerDatabase(*acd.Spec.AutonomousContainerDatabaseOCID) - if err != nil { - return err + validations := []func(logr.Logger, *dbv1alpha1.AutonomousContainerDatabase, *dbv1alpha1.AutonomousContainerDatabase) (bool, bool, error){ + r.validateGeneralFields, + r.validateDesiredLifecycleState, } - acd.Status.LifecycleState = database.AutonomousContainerDatabaseLifecycleStateTerminating + for _, op := range validations { + ociReqSent, specChanged, err := op(logger, acd, difACD) + if err != nil { + return false, false, err + } - if err := r.updateResourceStatus(acd); err != nil { - return err + if ociReqSent { + return true, specChanged, nil + } } - if _, err := r.workService.Wait(*resp.OpcWorkRequestId); err != nil { - return err + return false, false, nil +} + +func (r *AutonomousContainerDatabaseReconciler) validateGeneralFields( + logger logr.Logger, + acd *dbv1alpha1.AutonomousContainerDatabase, + difACD *dbv1alpha1.AutonomousContainerDatabase) (sent bool, requeue bool, err error) { + + if difACD.Spec.DisplayName == nil && + difACD.Spec.PatchModel == "" && + difACD.Spec.FreeformTags == nil { + return false, false, nil } - return nil + + logger.WithName("validateGeneralFields").Info("Sending UpdateAutonomousDatabase request to OCI") + + resp, err := r.dbService.UpdateAutonomousContainerDatabase(*acd.Spec.AutonomousContainerDatabaseOCID, difACD) + if err != nil { + return false, false, err + } + + acd.UpdateStatusFromOCIACD(resp.AutonomousContainerDatabase) + + return true, false, nil } -func (r *AutonomousContainerDatabaseReconciler) updateLifecycleState(acd *dbv1alpha1.AutonomousContainerDatabase, difACD *dbv1alpha1.AutonomousContainerDatabase) error { +func (r *AutonomousContainerDatabaseReconciler) validateDesiredLifecycleState( + logger logr.Logger, + acd *dbv1alpha1.AutonomousContainerDatabase, + difACD *dbv1alpha1.AutonomousContainerDatabase) (sent bool, specChanged bool, err error) { + if difACD.Spec.Action == dbv1alpha1.AcdActionBlank { - return nil + return false, false, nil } - var opcWorkRequestId string + l := logger.WithName("validateDesiredLifecycleState") switch difACD.Spec.Action { case dbv1alpha1.AcdActionRestart: + l.Info("Sending RestartAutonomousContainerDatabase request to OCI") + resp, err := r.dbService.RestartAutonomousContainerDatabase(*acd.Spec.AutonomousContainerDatabaseOCID) if err != nil { - return err + return false, false, err } acd.Status.LifecycleState = resp.LifecycleState - opcWorkRequestId = *resp.OpcWorkRequestId case dbv1alpha1.AcdActionTerminate: - resp, err := r.dbService.TerminateAutonomousContainerDatabase(*acd.Spec.AutonomousContainerDatabaseOCID) + l.Info("Sending TerminateAutonomousContainerDatabase request to OCI") + + _, err := r.dbService.TerminateAutonomousContainerDatabase(*acd.Spec.AutonomousContainerDatabaseOCID) if err != nil { - return err + return false, false, err } acd.Status.LifecycleState = database.AutonomousContainerDatabaseLifecycleStateTerminating - opcWorkRequestId = *resp.OpcWorkRequestId default: - return errors.New("Unknown action") - } - - // Update the status. The Action field will be erased at the sync operation before exiting the Reconcile. - if err := r.updateResourceStatus(acd); err != nil { - return err + return false, false, errors.New("Unknown lifecycleState") } - if _, err := r.workService.Wait(opcWorkRequestId); err != nil { - return err - } - return nil -} + acd.Spec.Action = dbv1alpha1.AcdActionBlank -func (r *AutonomousContainerDatabaseReconciler) updateACD(acd *dbv1alpha1.AutonomousContainerDatabase, difACD *dbv1alpha1.AutonomousContainerDatabase) error { - if err := r.updateGeneralFields(acd, difACD); err != nil { - return err - } - - if err := r.updateLifecycleState(acd, difACD); err != nil { - return err - } - - return nil + return true, true, nil } diff --git a/controllers/database/autonomousdatabase_controller.go b/controllers/database/autonomousdatabase_controller.go index 86aa9a0f..186ac058 100644 --- a/controllers/database/autonomousdatabase_controller.go +++ b/controllers/database/autonomousdatabase_controller.go @@ -46,8 +46,10 @@ import ( "reflect" "regexp" "strings" + "time" "github.com/go-logr/logr" + "github.com/oracle/oci-go-sdk/v63/common" "github.com/oracle/oci-go-sdk/v63/database" corev1 "k8s.io/api/core/v1" @@ -55,7 +57,6 @@ import ( "k8s.io/apimachinery/pkg/runtime" "k8s.io/apimachinery/pkg/types" "k8s.io/client-go/tools/record" - "k8s.io/client-go/util/retry" ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/controller-runtime/pkg/controller" @@ -72,6 +73,7 @@ import ( "github.com/oracle/oracle-database-operator/commons/oci" ) +var requeueResult ctrl.Result = ctrl.Result{Requeue: true, RequeueAfter: 15 * time.Second} var emptyResult ctrl.Result = ctrl.Result{} // *AutonomousDatabaseReconciler reconciles a AutonomousDatabase object @@ -81,9 +83,7 @@ type AutonomousDatabaseReconciler struct { Scheme *runtime.Scheme Recorder record.EventRecorder - dbService oci.DatabaseService - workService oci.WorkRequestService - lastSucSpec *dbv1alpha1.AutonomousDatabaseSpec + dbService oci.DatabaseService } // SetupWithManager function @@ -131,24 +131,6 @@ func (r *AutonomousDatabaseReconciler) watchPredicate() predicate.Predicate { } } -func (r *AutonomousDatabaseReconciler) isIntermediateState(state database.AutonomousDatabaseLifecycleStateEnum) bool { - if state == database.AutonomousDatabaseLifecycleStateProvisioning || - state == database.AutonomousDatabaseLifecycleStateUpdating || - state == database.AutonomousDatabaseLifecycleStateStarting || - state == database.AutonomousDatabaseLifecycleStateStopping || - state == database.AutonomousDatabaseLifecycleStateTerminating || - state == database.AutonomousDatabaseLifecycleStateRestoreInProgress || - state == database.AutonomousDatabaseLifecycleStateBackupInProgress || - state == database.AutonomousDatabaseLifecycleStateMaintenanceInProgress || - state == database.AutonomousDatabaseLifecycleStateRestarting || - state == database.AutonomousDatabaseLifecycleStateRecreating || - state == database.AutonomousDatabaseLifecycleStateRoleChangeInProgress || - state == database.AutonomousDatabaseLifecycleStateUpgrading { - return true - } - return false -} - func (r *AutonomousDatabaseReconciler) eventFilterPredicate() predicate.Predicate { return predicate.Funcs{ UpdateFunc: func(e event.UpdateEvent) bool { @@ -157,28 +139,12 @@ func (r *AutonomousDatabaseReconciler) eventFilterPredicate() predicate.Predicat if adbOk { oldADB := e.ObjectOld.(*dbv1alpha1.AutonomousDatabase) - oldSucSpec, oldSucSpecOk := oldADB.GetAnnotations()[dbv1alpha1.LastSuccessfulSpec] - newSucSpec, newSucSpecOk := desiredADB.GetAnnotations()[dbv1alpha1.LastSuccessfulSpec] - sucSpecChanged := oldSucSpecOk != newSucSpecOk || oldSucSpec != newSucSpec - - if !reflect.DeepEqual(oldADB.Status, desiredADB.Status) || sucSpecChanged { + if !reflect.DeepEqual(oldADB.Status, desiredADB.Status) || + (controllerutil.ContainsFinalizer(oldADB, dbv1alpha1.ADBFinalizer) != controllerutil.ContainsFinalizer(desiredADB, dbv1alpha1.ADBFinalizer)) { // Don't enqueue if the status or the lastSucSpec changes return false } - oldState := oldADB.Status.LifecycleState - desiredState := desiredADB.Spec.Details.LifecycleState - - if r.isIntermediateState(oldState) { - // Except for the case that the ADB is already terminating, we should let the terminate requests to be enqueued - if oldState != database.AutonomousDatabaseLifecycleStateTerminating && - desiredState == database.AutonomousDatabaseLifecycleStateTerminated { - return true - } - - // All the requests other than the terminate request, should be discarded during the intermediate states - return false - } return true } return true @@ -186,11 +152,7 @@ func (r *AutonomousDatabaseReconciler) eventFilterPredicate() predicate.Predicat DeleteFunc: func(e event.DeleteEvent) bool { // Do not trigger reconciliation when the object is deleted from the cluster. _, adbOk := e.Object.(*dbv1alpha1.AutonomousDatabase) - if adbOk { - return false - } - - return true + return !adbOk }, } } @@ -207,9 +169,10 @@ func (r *AutonomousDatabaseReconciler) eventFilterPredicate() predicate.Predicat // It go to the beggining of the reconcile if an error is returned. We won't return a error if it is related // to OCI, because the issues cannot be solved by re-run the reconcile. func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { - logger := r.Log.WithValues("Namespaced/Name", req.NamespacedName) + logger := r.Log.WithValues("Namespace/Name", req.NamespacedName) var err error + var ociADB *dbv1alpha1.AutonomousDatabase // Get the autonomousdatabase instance from the cluster adb := &dbv1alpha1.AutonomousDatabase{} @@ -223,91 +186,104 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R return emptyResult, err } - r.lastSucSpec, err = adb.GetLastSuccessfulSpec() - if err != nil { - return r.manageError(adb, err) - } - /****************************************************************** - * Get OCI database client and work request client + * Get OCI database client ******************************************************************/ - if err := r.setupOCIClients(adb); err != nil { + if err := r.setupOCIClients(logger, adb); err != nil { logger.Error(err, "Fail to setup OCI clients") - return r.manageError(adb, err) + return r.manageError(logger, adb, err) } logger.Info("OCI clients configured succesfully") /****************************************************************** - * Register/unregister finalizer + * Get OCI AutonomousDatabase + ******************************************************************/ + + if adb.Spec.Details.AutonomousDatabaseOCID != nil { + resp, err := r.dbService.GetAutonomousDatabase(*adb.Spec.Details.AutonomousDatabaseOCID) + if err != nil { + return r.manageError(logger, adb, err) + } + + ociADB = &dbv1alpha1.AutonomousDatabase{} + ociADB.UpdateFromOCIADB(resp.AutonomousDatabase) + } + + /****************************************************************** + * Requeue if the ADB is in an intermediate state + * No-op if the ADB OCID is nil + * To get the latest status, execute before all the reconcile logic + ******************************************************************/ + needsRequeue, err := r.validateLifecycleState(adb, ociADB) + if err != nil { + return r.manageError(logger, adb, err) + } + + if needsRequeue { + logger.WithName("validateLifecycleState").Info("Current lifecycleState is " + string(adb.Status.LifecycleState) + "; reconcile queued") + return requeueResult, nil + } + + /****************************************************************** + * Cleanup the resource if the resource is to be deleted. * Deletion timestamp will be added to a object before it is deleted. * Kubernetes server calls the clean up function if a finalizer exitsts, and won't delete the real object until * all the finalizers are removed from the object metadata. * Refer to this page for more details of using finalizers: https://kubernetes.io/blog/2021/05/14/using-finalizers-to-control-deletion/ ******************************************************************/ - isADBDeleteTrue, err := r.validateFinalizer(adb) + exitReconcile, err := r.validateCleanup(logger, adb) if err != nil { - return r.manageError(adb, err) + return r.manageError(logger, adb, err) } - if isADBDeleteTrue { + + if exitReconcile { return emptyResult, nil } /****************************************************************** - * Determine which Database operations need to be executed by checking the changes to spec.details. - * There are three scenario: - * 1. provision operation. The AutonomousDatabaseOCID is missing, and the LastSucSpec annotation is missing. - * 2. bind operation. The AutonomousDatabaseOCID is provided, but the LastSucSpec annotation is missing. - * 3. update operation. Every changes other than the above two cases goes here. - * Afterwards, update the resource from the remote database in OCI. This step will be executed right after - * the above three cases during every reconcile. - /******************************************************************/ - // difADB is nil when the action is PROVISION or BIND. - // Use difADB to identify which fields are updated when it's UPDATE operation. - action, difADB, err := r.determineAction(adb) - if err != nil { - return r.manageError(adb, err) + * Register/unregister the finalizer + ******************************************************************/ + if err := r.validateFinalizer(adb); err != nil { + return r.manageError(logger, adb, err) } - switch action { - case adbRecActionProvision: - if err := r.createADB(adb); err != nil { - return r.manageError(adb, err) - } - - if err := r.downloadWallet(adb); err != nil { - return r.manageError(adb, err) - } - case adbRecActionBind: - if err := r.downloadWallet(adb); err != nil { - return r.manageError(adb, err) - } - case adbRecActionUpdate: - // updateADB contains downloadWallet - if err := r.updateADB(adb, difADB); err != nil { - return r.manageError(adb, err) - } - case adbRecActionSync: - // SYNC action needs to make sure the wallet is present - if err := r.downloadWallet(adb); err != nil { - return r.manageError(adb, err) - } + /****************************************************************** + * Validate operations + ******************************************************************/ + exitReconcile, result, err := r.validateOperation(logger, adb, ociADB) + if err != nil { + return r.manageError(logger, adb, err) + } + if exitReconcile { + return result, nil } /***************************************************** * Sync AutonomousDatabase Backups from OCI *****************************************************/ - if err := r.syncBackupResources(adb); err != nil { - return r.manageError(adb, err) + if err := r.syncBackupResources(logger, adb); err != nil { + return r.manageError(logger, adb, err) } /***************************************************** - * Sync resource and update the lastSucSpec + * Validate Wallet *****************************************************/ - err = r.syncResource(adb) - if err != nil { - return r.manageError(adb, err) + if err := r.validateWallet(logger, adb); err != nil { + return r.manageError(logger, adb, err) + } + + /****************************************************************** + * Update the status and requeue if it's in an intermediate state + ******************************************************************/ + if err := r.KubeClient.Status().Update(context.TODO(), adb); err != nil { + return r.manageError(logger, adb, err) + } + + if dbv1alpha1.IsADBIntermediateState(adb.Status.LifecycleState) { + logger.WithName("IsIntermediateState").Info("LifecycleState is " + string(adb.Status.LifecycleState) + "; reconcile queued") + return requeueResult, nil } logger.Info("AutonomousDatabase reconciles successfully") @@ -315,7 +291,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R return emptyResult, nil } -func (r *AutonomousDatabaseReconciler) setupOCIClients(adb *dbv1alpha1.AutonomousDatabase) error { +func (r *AutonomousDatabaseReconciler) setupOCIClients(logger logr.Logger, adb *dbv1alpha1.AutonomousDatabase) error { var err error authData := oci.APIKeyAuth{ @@ -329,12 +305,7 @@ func (r *AutonomousDatabaseReconciler) setupOCIClients(adb *dbv1alpha1.Autonomou return err } - r.dbService, err = oci.NewDatabaseService(r.Log, r.KubeClient, provider) - if err != nil { - return err - } - - r.workService, err = oci.NewWorkRequestService(r.Log, r.KubeClient, provider) + r.dbService, err = oci.NewDatabaseService(logger, r.KubeClient, provider) if err != nil { return err } @@ -342,20 +313,32 @@ func (r *AutonomousDatabaseReconciler) setupOCIClients(adb *dbv1alpha1.Autonomou return nil } -func (r *AutonomousDatabaseReconciler) manageError(adb *dbv1alpha1.AutonomousDatabase, issue error) (ctrl.Result, error) { - // Rollback if lastSucSpec exists - if r.lastSucSpec != nil { +func (r *AutonomousDatabaseReconciler) manageError(logger logr.Logger, adb *dbv1alpha1.AutonomousDatabase, issue error) (ctrl.Result, error) { + l := logger.WithName("manageError") + // Has synced at least once + if adb.Status.LifecycleState != "" { // Send event - r.Recorder.Event(adb, corev1.EventTypeWarning, "ReconcileFailed", issue.Error()) + r.Recorder.Event(adb, corev1.EventTypeWarning, "UpdateFailed", issue.Error()) var finalIssue = issue // Roll back - if err := r.syncResource(adb); err != nil { + specChanged, err := r.getADB(l, adb) + if err != nil { finalIssue = k8s.CombineErrors(finalIssue, err) } - return emptyResult, finalIssue + // We don't exit the Reconcile if the spec has changed + // becasue it will exit anyway after the manageError is called. + if specChanged { + if err := r.KubeClient.Update(context.TODO(), adb); err != nil { + finalIssue = k8s.CombineErrors(finalIssue, err) + } + } + + l.Error(finalIssue, "UpdateFailed") + + return emptyResult, nil } else { // Send event r.Recorder.Event(adb, corev1.EventTypeWarning, "CreateFailed", issue.Error()) @@ -364,459 +347,451 @@ func (r *AutonomousDatabaseReconciler) manageError(adb *dbv1alpha1.AutonomousDat } } -func (r *AutonomousDatabaseReconciler) validateFinalizer(adb *dbv1alpha1.AutonomousDatabase) (isADBDeleteTrue bool, err error) { - logger := r.Log.WithName("finalizer") +func (r *AutonomousDatabaseReconciler) validateOperation( + logger logr.Logger, + adb *dbv1alpha1.AutonomousDatabase, + ociADB *dbv1alpha1.AutonomousDatabase) (exitReconcile bool, result ctrl.Result, err error) { - isADBToBeDeleted := adb.GetDeletionTimestamp() != nil + l := logger.WithName("validateOperation") - if isADBToBeDeleted { - if controllerutil.ContainsFinalizer(adb, dbv1alpha1.ADBFinalizer) { - if (r.lastSucSpec != nil && r.lastSucSpec.Details.AutonomousDatabaseOCID != nil) && // lastSucSpec exists and the ADB_OCID isn't nil - (adb.Status.LifecycleState != database.AutonomousDatabaseLifecycleStateTerminating && adb.Status.LifecycleState != database.AutonomousDatabaseLifecycleStateTerminated) { - // Run finalization logic for finalizer. If the finalization logic fails, don't remove the finalizer so - // that we can retry during the next reconciliation. - logger.Info("Terminating Autonomous Database: " + *adb.Spec.Details.DbName) - if err := r.deleteAutonomousDatabase(adb); err != nil { - return false, err - } - } else { - logger.Info("Missing AutonomousDatabaseOCID to terminate Autonomous Database", "Name", adb.Name, "Namespace", adb.Namespace) - } + lastSpec, err := adb.GetLastSuccessfulSpec() + if err != nil { + return false, emptyResult, err + } - // Remove finalizer. Once all finalizers have been - // removed, the object will be deleted. - if err := k8s.RemoveFinalizerAndPatch(r.KubeClient, adb, dbv1alpha1.ADBFinalizer); err != nil { - return false, nil + // If lastSucSpec is nil, then it's CREATE or BIND opertaion + if lastSpec == nil { + if adb.Spec.Details.AutonomousDatabaseOCID == nil { + l.Info("Create operation") + err := r.createADB(logger, adb) + if err != nil { + return false, emptyResult, err } - } - // Send true because delete is in progress and it is a custom delete message - // We don't need to print custom err stack as we are deleting the topology - return true, nil - } - // Delete is not schduled. Update the finalizer for this CR if hardLink is present - if adb.Spec.HardLink != nil { - if *adb.Spec.HardLink { - if err := k8s.AddFinalizerAndPatch(r.KubeClient, adb, dbv1alpha1.ADBFinalizer); err != nil { - return false, nil + // Update the ADB OCID + if err := r.updateCR(adb); err != nil { + return false, emptyResult, err } + + l.Info("AutonomousDatabaseOCID updated; exit reconcile") + return true, emptyResult, nil } else { - if err := k8s.RemoveFinalizerAndPatch(r.KubeClient, adb, dbv1alpha1.ADBFinalizer); err != nil { - return false, nil + l.Info("Bind operation") + _, err := r.getADB(logger, adb) + if err != nil { + return false, emptyResult, err } + + if err := r.updateCR(adb); err != nil { + return false, emptyResult, err + } + + l.Info("spec updated; exit reconcile") + return false, emptyResult, nil } } - return false, nil -} + // If it's not CREATE or BIND opertaion, then UPDATE or SYNC + // Compare with the lastSucSpec.details. If the details are different, it means that the user updates the spec. + lastDifADB := adb.DeepCopy() -// updateLastSuccessfulSpec updates the lasSucSpec annotation, and returns the ORIGINAL object -// The object will NOT be updated with the returned content from the cluster since we want to -// update only the lasSucSpec. For example: After we get the ADB information from OCI, we want -// to update the lasSucSpec before updating the local resource. In this case, we want to keep -// the content returned from the OCI, not the one from the cluster. -func (r *AutonomousDatabaseReconciler) updateLastSuccessfulSpec(adb *dbv1alpha1.AutonomousDatabase) error { - specBytes, err := json.Marshal(adb.Spec) + lastDetailsChanged, err := lastDifADB.RemoveUnchangedDetails(*lastSpec) if err != nil { - return err + return false, emptyResult, err } - anns := map[string]string{ - dbv1alpha1.LastSuccessfulSpec: string(specBytes), - } + if lastDetailsChanged { + l.Info("Update operation") - copyADB := adb.DeepCopy() + // Double check if the user input spec is actually different from the spec in OCI. If so, then update the resource. + difADB := adb.DeepCopy() - return annotations.PatchAnnotations(r.KubeClient, copyADB, anns) -} + // Compare the secret associated fields with the ones from lastSucSpec since they are missing in the oci ADB + ociADB.Spec.Details.AdminPassword = lastSpec.Details.AdminPassword + ociADB.Spec.Details.Wallet = lastSpec.Details.Wallet + ociDetailsChanged, err := difADB.RemoveUnchangedDetails(ociADB.Spec) + if err != nil { + return false, emptyResult, err + } + + if ociDetailsChanged { + ociReqSent, err := r.updateADB(logger, adb, difADB, ociADB) + if err != nil { + return false, emptyResult, err + } -// updateResourceStatus updates only the status of the resource, not including the lastSucSpec. -// This function should not be called by the functions associated with the OCI update requests. -// The OCI update requests should use updateResource() to ensure all the spec, resource and the -// lastSucSpec are updated. -func (r *AutonomousDatabaseReconciler) updateResourceStatus(adb *dbv1alpha1.AutonomousDatabase) error { - return retry.RetryOnConflict(retry.DefaultRetry, func() error { - curADB := &dbv1alpha1.AutonomousDatabase{} - - namespacedName := types.NamespacedName{ - Namespace: adb.GetNamespace(), - Name: adb.GetName(), + // Requeue the k8s request if an OCI request is sent, since OCI can only process one request at a time. + if ociReqSent { + l.Info("reconcile queued") + return true, requeueResult, nil + } } - if err := r.KubeClient.Get(context.TODO(), namespacedName, curADB); err != nil { - return err + // Stop the update and patch the lastSpec when the current ADB matches the oci ADB. + if err := r.patchLastSuccessfulSpec(adb); err != nil { + return false, emptyResult, err } - curADB.Status = adb.Status + return false, emptyResult, nil - return r.KubeClient.Status().Update(context.TODO(), curADB) - }) -} + } else { + l.Info("No operation specified; sync the resource") -// syncResource pulled information from OCI, update the LastSucSpec, and lastly, update the local resource. -// It's important to update the LastSucSpec prior than we update the local resource, -// because updating the local resource triggers the Reconcile, and the Reconcile determines the action by -// looking if the lastSucSpec is present. -func (r *AutonomousDatabaseReconciler) syncResource(adb *dbv1alpha1.AutonomousDatabase) error { - // Get the information from OCI - resp, err := r.dbService.GetAutonomousDatabase(*adb.Spec.Details.AutonomousDatabaseOCID) - if err != nil { - return err + // The user doesn't change the spec and the controller should pull the spec from the OCI. + specChanged, err := r.getADB(logger, adb) + if err != nil { + return false, emptyResult, err + } + + if specChanged { + l.Info("The local spec doesn't match the oci's spec; update the CR") + + if err := r.updateCR(adb); err != nil { + return false, emptyResult, err + } + + return true, emptyResult, nil + } + return false, emptyResult, nil } +} - adb.UpdateFromOCIADB(resp.AutonomousDatabase) +func (r *AutonomousDatabaseReconciler) validateCleanup(logger logr.Logger, adb *dbv1alpha1.AutonomousDatabase) (exitReconcile bool, err error) { + l := logger.WithName("validateCleanup") - // Update the lastSucSpec - if err := r.updateLastSuccessfulSpec(adb); err != nil { - return err + isADBToBeDeleted := adb.GetDeletionTimestamp() != nil + + if !isADBToBeDeleted { + return false, nil } - // Update the status first to prevent unwanted Reconcile() - if err := r.updateResourceStatus(adb); err != nil { - return err + if controllerutil.ContainsFinalizer(adb, dbv1alpha1.ADBFinalizer) { + if adb.Status.LifecycleState == database.AutonomousDatabaseLifecycleStateTerminating { + l.Info("Resource is already in TERMINATING state") + // Delete in progress, continue with the reconcile logic + return false, nil + } + + if adb.Status.LifecycleState == database.AutonomousDatabaseLifecycleStateTerminated { + // The adb has been deleted. Remove the finalizer and exit the reconcile. + // Once all finalizers have been removed, the object will be deleted. + l.Info("Resource is already in TERMINATED state; remove the finalizer") + if err := k8s.RemoveFinalizerAndPatch(r.KubeClient, adb, dbv1alpha1.ADBFinalizer); err != nil { + return false, err + } + return true, nil + } + + if adb.Spec.Details.AutonomousDatabaseOCID == nil { + l.Info("Missing AutonomousDatabaseOCID to terminate Autonomous Database; remove the finalizer anyway", "Name", adb.Name, "Namespace", adb.Namespace) + // Remove finalizer anyway. + if err := k8s.RemoveFinalizerAndPatch(r.KubeClient, adb, dbv1alpha1.ADBFinalizer); err != nil { + return false, err + } + return true, nil + } + + if adb.Spec.Details.LifecycleState != database.AutonomousDatabaseLifecycleStateTerminated { + // Run finalization logic for finalizer. If the finalization logic fails, don't remove the finalizer so + // that we can retry during the next reconciliation. + l.Info("Terminating Autonomous Database: " + *adb.Spec.Details.DbName) + adb.Spec.Details.LifecycleState = database.AutonomousDatabaseLifecycleStateTerminated + if err := r.KubeClient.Update(context.TODO(), adb); err != nil { + return false, err + } + // Exit the reconcile since we have updated the spec + return true, nil + } + + // Continue with the reconcile logic + return false, nil } - if err := r.updateResource(adb); err != nil { - return err + // Exit the Reconcile since the to-be-deleted resource doesn't has a finalizer + return true, nil +} + +func (r *AutonomousDatabaseReconciler) validateFinalizer(adb *dbv1alpha1.AutonomousDatabase) error { + // Delete is not schduled. Update the finalizer for this CR if hardLink is present + if adb.Spec.HardLink != nil { + if *adb.Spec.HardLink { + if err := k8s.AddFinalizerAndPatch(r.KubeClient, adb, dbv1alpha1.ADBFinalizer); err != nil { + return err + } + } else { + if err := k8s.RemoveFinalizerAndPatch(r.KubeClient, adb, dbv1alpha1.ADBFinalizer); err != nil { + return err + } + } } return nil } -// updateResource updates the specification, the status of AutonomousDatabase resource, and the lastSucSpec -func (r *AutonomousDatabaseReconciler) updateResource(adb *dbv1alpha1.AutonomousDatabase) error { - // Update the spec - if err := retry.RetryOnConflict(retry.DefaultRetry, func() error { - curADB := &dbv1alpha1.AutonomousDatabase{} +// validateLifecycleState gets and validates the current lifecycleState +func (r *AutonomousDatabaseReconciler) validateLifecycleState(adb *dbv1alpha1.AutonomousDatabase, ociADB *dbv1alpha1.AutonomousDatabase) (needsRequeue bool, err error) { + if ociADB == nil { + return false, nil + } - namespacedName := types.NamespacedName{ - Namespace: adb.GetNamespace(), - Name: adb.GetName(), - } + adb.Status = ociADB.Status - if err := r.KubeClient.Get(context.TODO(), namespacedName, curADB); err != nil { - return err - } + if err := r.KubeClient.Status().Update(context.TODO(), adb); err != nil { + return false, err + } + + if dbv1alpha1.IsADBIntermediateState(ociADB.Status.LifecycleState) { + return true, nil + } + + return false, nil +} - curADB.Spec.Details = adb.Spec.Details - return r.KubeClient.Update(context.TODO(), curADB) - }); err != nil { +func (r *AutonomousDatabaseReconciler) updateCR(adb *dbv1alpha1.AutonomousDatabase) error { + // Update the lastSucSpec + if err := adb.UpdateLastSuccessfulSpec(); err != nil { return err } + if err := r.KubeClient.Update(context.TODO(), adb); err != nil { + return err + } return nil } -type adbRecActionEnum string - -const ( - adbRecActionProvision adbRecActionEnum = "PROVISION" - adbRecActionBind adbRecActionEnum = "BIND" - adbRecActionUpdate adbRecActionEnum = "UPDATE" - adbRecActionSync adbRecActionEnum = "SYNC" -) +func (r *AutonomousDatabaseReconciler) patchLastSuccessfulSpec(adb *dbv1alpha1.AutonomousDatabase) error { + specBytes, err := json.Marshal(adb.Spec) + if err != nil { + return err + } -func (r *AutonomousDatabaseReconciler) determineAction(adb *dbv1alpha1.AutonomousDatabase) (adbRecActionEnum, *dbv1alpha1.AutonomousDatabase, error) { - if r.lastSucSpec == nil { - if adb.Spec.Details.AutonomousDatabaseOCID == nil { - return adbRecActionProvision, nil, nil - } else { - return adbRecActionBind, nil, nil - } - } else { - // Pre-process step for the UPDATE. Remove the unchanged fields in spec.details, - difADB := adb.DeepCopy() - detailsChanged, err := difADB.RemoveUnchangedDetails() - if err != nil { - return "", nil, err - } + anns := map[string]string{ + dbv1alpha1.LastSuccessfulSpec: string(specBytes), + } - if detailsChanged { - return adbRecActionUpdate, difADB, nil - } + annotations.PatchAnnotations(r.KubeClient, adb, anns) - return adbRecActionSync, nil, nil - } + return nil } -func (r *AutonomousDatabaseReconciler) createADB(adb *dbv1alpha1.AutonomousDatabase) error { +func (r *AutonomousDatabaseReconciler) createADB(logger logr.Logger, adb *dbv1alpha1.AutonomousDatabase) error { + logger.WithName("createADB").Info("Sending CreateAutonomousDatabase request to OCI") resp, err := r.dbService.CreateAutonomousDatabase(adb) if err != nil { return err } - // Update the ADB OCID and the status - // The trick is to update the status first to prevent unwanted reconcile - adb.Spec.Details.AutonomousDatabaseOCID = resp.AutonomousDatabase.Id - adb.UpdateStatusFromOCIADB(resp.AutonomousDatabase) + adb.UpdateFromOCIADB(resp.AutonomousDatabase) - if err := r.updateResourceStatus(adb); err != nil { - return err + return nil +} + +func (r *AutonomousDatabaseReconciler) getADB(logger logr.Logger, adb *dbv1alpha1.AutonomousDatabase) (bool, error) { + if adb == nil { + return false, errors.New("AutonomousDatabase OCID is missing") } - // Patching is faster - if err := k8s.Patch(r.KubeClient, adb, "/spec/details/autonomousDatabaseOCID", adb.Spec.Details.AutonomousDatabaseOCID); err != nil { - return err + // Get the information from OCI + logger.WithName("getADB").Info("Sending GetAutonomousDatabase request to OCI") + resp, err := r.dbService.GetAutonomousDatabase(*adb.Spec.Details.AutonomousDatabaseOCID) + if err != nil { + return false, err } - if _, err := r.workService.Wait(*resp.OpcWorkRequestId); err != nil { - return err + specChanged := adb.UpdateFromOCIADB(resp.AutonomousDatabase) + + return specChanged, nil +} + +// updateADB returns true if an OCI request is sent. +// The AutonomousDatabase is updated with the returned object from the OCI requests. +func (r *AutonomousDatabaseReconciler) updateADB( + logger logr.Logger, + adb *dbv1alpha1.AutonomousDatabase, + difADB *dbv1alpha1.AutonomousDatabase, + ociADB *dbv1alpha1.AutonomousDatabase) (ociReqSent bool, err error) { + + validations := []func(logr.Logger, *dbv1alpha1.AutonomousDatabase, *dbv1alpha1.AutonomousDatabase, *dbv1alpha1.AutonomousDatabase) (bool, error){ + r.validateGeneralFields, + r.validateAdminPassword, + r.validateDbWorkload, + r.validateLicenseModel, + r.validateScalingFields, + r.validateGeneralNetworkAccess, + r.validateDesiredLifecycleState, + } + + for _, op := range validations { + ociReqSent, err := op(logger, adb, difADB, ociADB) + if err != nil { + return false, err + } + + if ociReqSent { + return true, nil + } } - return nil + + return false, nil } -func (r *AutonomousDatabaseReconciler) updateGeneralFields(adb *dbv1alpha1.AutonomousDatabase, difADB *dbv1alpha1.AutonomousDatabase) error { +func (r *AutonomousDatabaseReconciler) validateGeneralFields( + logger logr.Logger, + adb *dbv1alpha1.AutonomousDatabase, + difADB *dbv1alpha1.AutonomousDatabase, + ociADB *dbv1alpha1.AutonomousDatabase) (sent bool, err error) { + if difADB.Spec.Details.DisplayName == nil && difADB.Spec.Details.DbName == nil && difADB.Spec.Details.DbVersion == nil && difADB.Spec.Details.FreeformTags == nil { - return nil + return false, nil } - resp, err := r.dbService.UpdateAutonomousDatabaseGeneralFields(difADB) + logger.WithName("validateGeneralFields").Info("Sending UpdateAutonomousDatabase request to OCI") + resp, err := r.dbService.UpdateAutonomousDatabaseGeneralFields(*adb.Spec.Details.AutonomousDatabaseOCID, difADB) if err != nil { - return err - } - - // If the OpcWorkRequestId is nil (such as when the displayName is changed), - // no need to update the resource and wail until the work is done - if resp.OpcWorkRequestId == nil { - return nil + return false, err } adb.UpdateStatusFromOCIADB(resp.AutonomousDatabase) - if err := r.updateResourceStatus(adb); err != nil { - return err - } - if _, err := r.workService.Wait(*resp.OpcWorkRequestId); err != nil { - return err - } - return nil + return true, nil } -func (r *AutonomousDatabaseReconciler) updateAdminPassword(adb *dbv1alpha1.AutonomousDatabase, difADB *dbv1alpha1.AutonomousDatabase) error { +// Special case: compare with lastSpec but not ociSpec +func (r *AutonomousDatabaseReconciler) validateAdminPassword( + logger logr.Logger, + adb *dbv1alpha1.AutonomousDatabase, + difADB *dbv1alpha1.AutonomousDatabase, + ociADB *dbv1alpha1.AutonomousDatabase) (sent bool, err error) { + if difADB.Spec.Details.AdminPassword.K8sSecret.Name == nil && difADB.Spec.Details.AdminPassword.OCISecret.OCID == nil { - return nil + return false, nil } - _, err := r.dbService.UpdateAutonomousDatabaseAdminPassword(difADB) + logger.WithName("validateAdminPassword").Info("Sending UpdateAutonomousDatabase request to OCI") + resp, err := r.dbService.UpdateAutonomousDatabaseAdminPassword(*adb.Spec.Details.AutonomousDatabaseOCID, difADB) if err != nil { - return err + return false, err } - // UpdateAdminPassword request doesn't return the workrequest ID - return nil + adb.UpdateStatusFromOCIADB(resp.AutonomousDatabase) + + return true, nil } -func (r *AutonomousDatabaseReconciler) updateDbWorkload(adb *dbv1alpha1.AutonomousDatabase, difADB *dbv1alpha1.AutonomousDatabase) error { +func (r *AutonomousDatabaseReconciler) validateDbWorkload( + logger logr.Logger, + adb *dbv1alpha1.AutonomousDatabase, + difADB *dbv1alpha1.AutonomousDatabase, + ociADB *dbv1alpha1.AutonomousDatabase) (sent bool, err error) { + if difADB.Spec.Details.DbWorkload == "" { - return nil + return false, nil } - resp, err := r.dbService.UpdateAutonomousDatabaseDBWorkload(difADB) + logger.WithName("validateDbWorkload").Info("Sending UpdateAutonomousDatabase request to OCI") + resp, err := r.dbService.UpdateAutonomousDatabaseDBWorkload(*adb.Spec.Details.AutonomousDatabaseOCID, difADB) if err != nil { - return err + return false, err } adb.UpdateStatusFromOCIADB(resp.AutonomousDatabase) - if err := r.updateResourceStatus(adb); err != nil { - return err - } - if _, err := r.workService.Wait(*resp.OpcWorkRequestId); err != nil { - return err - } - return nil + return true, nil } -func (r *AutonomousDatabaseReconciler) updateLicenseModel(adb *dbv1alpha1.AutonomousDatabase, difADB *dbv1alpha1.AutonomousDatabase) error { +func (r *AutonomousDatabaseReconciler) validateLicenseModel( + logger logr.Logger, + adb *dbv1alpha1.AutonomousDatabase, + difADB *dbv1alpha1.AutonomousDatabase, + ociADB *dbv1alpha1.AutonomousDatabase) (sent bool, err error) { + if difADB.Spec.Details.LicenseModel == "" { - return nil + return false, nil } - resp, err := r.dbService.UpdateAutonomousDatabaseLicenseModel(difADB) + logger.WithName("validateLicenseModel").Info("Sending UpdateAutonomousDatabase request to OCI") + resp, err := r.dbService.UpdateAutonomousDatabaseLicenseModel(*adb.Spec.Details.AutonomousDatabaseOCID, difADB) if err != nil { - return err + return false, err } adb.UpdateStatusFromOCIADB(resp.AutonomousDatabase) - if err := r.updateResourceStatus(adb); err != nil { - return err - } - if _, err := r.workService.Wait(*resp.OpcWorkRequestId); err != nil { - return err - } - return nil + return true, nil } -func (r *AutonomousDatabaseReconciler) updateScalingFields(adb *dbv1alpha1.AutonomousDatabase, difADB *dbv1alpha1.AutonomousDatabase) error { +func (r *AutonomousDatabaseReconciler) validateScalingFields( + logger logr.Logger, + adb *dbv1alpha1.AutonomousDatabase, + difADB *dbv1alpha1.AutonomousDatabase, + ociADB *dbv1alpha1.AutonomousDatabase) (sent bool, err error) { + if difADB.Spec.Details.DataStorageSizeInTBs == nil && difADB.Spec.Details.CPUCoreCount == nil && difADB.Spec.Details.IsAutoScalingEnabled == nil { - return nil + return false, nil } - resp, err := r.dbService.UpdateAutonomousDatabaseScalingFields(difADB) + logger.WithName("validateScalingFields").Info("Sending UpdateAutonomousDatabase request to OCI") + resp, err := r.dbService.UpdateAutonomousDatabaseScalingFields(*adb.Spec.Details.AutonomousDatabaseOCID, difADB) if err != nil { - return err + return false, err } adb.UpdateStatusFromOCIADB(resp.AutonomousDatabase) - if err := r.updateResourceStatus(adb); err != nil { - return err - } - if _, err := r.workService.Wait(*resp.OpcWorkRequestId); err != nil { - return err - } - return nil + return true, nil } -func (r *AutonomousDatabaseReconciler) deleteAutonomousDatabase(adb *dbv1alpha1.AutonomousDatabase) error { - - resp, err := r.dbService.DeleteAutonomousDatabase(*adb.Spec.Details.AutonomousDatabaseOCID) - if err != nil { - return err - } +func (r *AutonomousDatabaseReconciler) validateDesiredLifecycleState( + logger logr.Logger, + adb *dbv1alpha1.AutonomousDatabase, + difADB *dbv1alpha1.AutonomousDatabase, + ociADB *dbv1alpha1.AutonomousDatabase) (sent bool, err error) { - adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateTerminating - - if err := r.updateResourceStatus(adb); err != nil { - return err - } - - _, err = r.workService.Wait(*resp.OpcWorkRequestId) - - return err -} - -func (r *AutonomousDatabaseReconciler) updateLifecycleState(adb *dbv1alpha1.AutonomousDatabase, difADB *dbv1alpha1.AutonomousDatabase) error { if difADB.Spec.Details.LifecycleState == "" { - return nil + return false, nil } - var opcWorkRequestId string + l := logger.WithName("validateDesiredLifecycleState") switch difADB.Spec.Details.LifecycleState { case database.AutonomousDatabaseLifecycleStateAvailable: - resp, err := r.dbService.StartAutonomousDatabase(*difADB.Spec.Details.AutonomousDatabaseOCID) + l.Info("Sending StartAutonomousDatabase request to OCI") + + resp, err := r.dbService.StartAutonomousDatabase(*adb.Spec.Details.AutonomousDatabaseOCID) if err != nil { - return err + return false, err } adb.Status.LifecycleState = resp.LifecycleState - opcWorkRequestId = *resp.OpcWorkRequestId case database.AutonomousDatabaseLifecycleStateStopped: - resp, err := r.dbService.StopAutonomousDatabase(*difADB.Spec.Details.AutonomousDatabaseOCID) + l.Info("Sending StopAutonomousDatabase request to OCI") + + resp, err := r.dbService.StopAutonomousDatabase(*adb.Spec.Details.AutonomousDatabaseOCID) if err != nil { - return err + return false, err } adb.Status.LifecycleState = resp.LifecycleState - opcWorkRequestId = *resp.OpcWorkRequestId case database.AutonomousDatabaseLifecycleStateTerminated: - resp, err := r.dbService.DeleteAutonomousDatabase(*difADB.Spec.Details.AutonomousDatabaseOCID) + l.Info("Sending DeleteAutonomousDatabase request to OCI") + + _, err := r.dbService.DeleteAutonomousDatabase(*adb.Spec.Details.AutonomousDatabaseOCID) if err != nil { - return err + return false, err } adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateTerminating - opcWorkRequestId = *resp.OpcWorkRequestId default: - return errors.New("Unknown state") - } - - if err := r.updateResourceStatus(adb); err != nil { - return err - } - - if _, err := r.workService.Wait(opcWorkRequestId); err != nil { - return err - } - return nil -} - -func (r *AutonomousDatabaseReconciler) setMTLSRequired(adb *dbv1alpha1.AutonomousDatabase) error { - resp, err := r.dbService.UpdateNetworkAccessMTLSRequired(*adb.Spec.Details.AutonomousDatabaseOCID) - if err != nil { - return err - } - - adb.UpdateStatusFromOCIADB(resp.AutonomousDatabase) - if err := r.updateResourceStatus(adb); err != nil { - return err - } - - if _, err := r.workService.Wait(*resp.OpcWorkRequestId); err != nil { - return err - } - return nil -} - -func (r *AutonomousDatabaseReconciler) updateMTLS(adb *dbv1alpha1.AutonomousDatabase, difADB *dbv1alpha1.AutonomousDatabase) error { - if difADB.Spec.Details.NetworkAccess.IsMTLSConnectionRequired == nil { - return nil - } - - resp, err := r.dbService.UpdateNetworkAccessMTLS(difADB) - if err != nil { - return err - } - - adb.UpdateStatusFromOCIADB(resp.AutonomousDatabase) - if err := r.updateResourceStatus(adb); err != nil { - return err + return false, errors.New("Unknown lifecycleState") } - if _, err := r.workService.Wait(*resp.OpcWorkRequestId); err != nil { - return err - } - return nil -} - -func (r *AutonomousDatabaseReconciler) setNetworkAccessPublic(adb *dbv1alpha1.AutonomousDatabase) error { - resp, err := r.dbService.UpdateNetworkAccessPublic(r.lastSucSpec.Details.NetworkAccess.AccessType, *adb.Spec.Details.AutonomousDatabaseOCID) - if err != nil { - return err - } - - adb.UpdateStatusFromOCIADB(resp.AutonomousDatabase) - if err := r.updateResourceStatus(adb); err != nil { - return err - } - - if _, err := r.workService.Wait(*resp.OpcWorkRequestId); err != nil { - return err - } - return nil -} - -func (r *AutonomousDatabaseReconciler) updateNetworkAccess(adb *dbv1alpha1.AutonomousDatabase, difADB *dbv1alpha1.AutonomousDatabase) error { - if difADB.Spec.Details.NetworkAccess.AccessType == "" && - difADB.Spec.Details.NetworkAccess.IsAccessControlEnabled == nil && - difADB.Spec.Details.NetworkAccess.AccessControlList == nil && - difADB.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID == nil && - difADB.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs == nil && - difADB.Spec.Details.NetworkAccess.PrivateEndpoint.HostnamePrefix == nil { - return nil - } - resp, err := r.dbService.UpdateNetworkAccess(difADB) - if err != nil { - return err - } - - adb.UpdateStatusFromOCIADB(resp.AutonomousDatabase) - if err := r.updateResourceStatus(adb); err != nil { - return err - } - - if _, err := r.workService.Wait(*resp.OpcWorkRequestId); err != nil { - return err - } - return nil + return true, nil } // The logic of updating the network access configurations is as follows: @@ -836,7 +811,12 @@ func (r *AutonomousDatabaseReconciler) updateNetworkAccess(adb *dbv1alpha1.Auton // // 2. Dedicated databases: // Apply the configs directly -func (r *AutonomousDatabaseReconciler) determineNetworkAccessUpdate(adb *dbv1alpha1.AutonomousDatabase, difADB *dbv1alpha1.AutonomousDatabase) error { +func (r *AutonomousDatabaseReconciler) validateGeneralNetworkAccess( + logger logr.Logger, + adb *dbv1alpha1.AutonomousDatabase, + difADB *dbv1alpha1.AutonomousDatabase, + ociADB *dbv1alpha1.AutonomousDatabase) (sent bool, err error) { + if difADB.Spec.Details.NetworkAccess.AccessType == "" && difADB.Spec.Details.NetworkAccess.IsAccessControlEnabled == nil && difADB.Spec.Details.NetworkAccess.AccessControlList == nil && @@ -844,89 +824,212 @@ func (r *AutonomousDatabaseReconciler) determineNetworkAccessUpdate(adb *dbv1alp difADB.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID == nil && difADB.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs == nil && difADB.Spec.Details.NetworkAccess.PrivateEndpoint.HostnamePrefix == nil { - return nil + return false, nil } + l := logger.WithName("validateGeneralNetworkAccess") + if !*adb.Spec.Details.IsDedicated { - var lastAccessType = r.lastSucSpec.Details.NetworkAccess.AccessType + var lastAccessType = ociADB.Spec.Details.NetworkAccess.AccessType var difAccessType = difADB.Spec.Details.NetworkAccess.AccessType if difAccessType != "" { switch difAccessType { case dbv1alpha1.NetworkAccessTypePublic: + l.Info("Configuring network access type to PUBLIC") // OCI validation requires IsMTLSConnectionRequired to be enabled before changing the network access type to PUBLIC - if !*r.lastSucSpec.Details.NetworkAccess.IsMTLSConnectionRequired { - if err := r.setMTLSRequired(adb); err != nil { - return err + if !*ociADB.Spec.Details.NetworkAccess.IsMTLSConnectionRequired { + if err := r.setMTLSRequired(logger, adb); err != nil { + return false, err } + return true, nil } - if err := r.setNetworkAccessPublic(adb); err != nil { - return err + if err := r.setNetworkAccessPublic(logger, ociADB.Spec.Details.NetworkAccess.AccessType, adb); err != nil { + return false, err } + return true, nil case dbv1alpha1.NetworkAccessTypeRestricted: + l.Info("Configuring network access type to RESTRICTED") // If the access type was PRIVATE, then OCI validation requires IsMTLSConnectionRequired - // to be enabled before setting ACL. Also we can only change the network access type from - // PRIVATE to PUBLIC. + // to be enabled before setting ACL. Also, we can only change the network access type from + // PRIVATE to PUBLIC, so the steps are PRIVATE->(requeue)->PUBLIC->(requeue)->RESTRICTED. if lastAccessType == dbv1alpha1.NetworkAccessTypePrivate { - if !*r.lastSucSpec.Details.NetworkAccess.IsMTLSConnectionRequired { - var oldMTLS bool = *adb.Spec.Details.NetworkAccess.IsMTLSConnectionRequired - if err := r.setMTLSRequired(adb); err != nil { - return err + if !*ociADB.Spec.Details.NetworkAccess.IsMTLSConnectionRequired { + if err := r.setMTLSRequired(logger, adb); err != nil { + return false, err } - // restore IsMTLSConnectionRequired - adb.Spec.Details.NetworkAccess.IsMTLSConnectionRequired = &oldMTLS + return true, nil } - if err := r.setNetworkAccessPublic(adb); err != nil { - return err + if err := r.setNetworkAccessPublic(logger, ociADB.Spec.Details.NetworkAccess.AccessType, adb); err != nil { + return false, err } + return true, nil } - if err := r.updateNetworkAccess(adb, difADB); err != nil { - return err + sent, err := r.validateNetworkAccess(logger, adb, difADB, ociADB) + if err != nil { + return false, err + } + if sent { + return true, nil } - if err := r.updateMTLS(adb, difADB); err != nil { - return err + sent, err = r.validateMTLS(logger, adb, difADB, ociADB) + if err != nil { + return false, err + } + if sent { + return true, nil } case dbv1alpha1.NetworkAccessTypePrivate: - if err := r.updateNetworkAccess(adb, difADB); err != nil { - return err + l.Info("Configuring network access type to PRIVATE") + + sent, err := r.validateNetworkAccess(logger, adb, difADB, ociADB) + if err != nil { + return false, err + } + if sent { + return true, nil } - if err := r.updateMTLS(adb, difADB); err != nil { - return err + sent, err = r.validateMTLS(logger, adb, difADB, ociADB) + if err != nil { + return false, err + } + if sent { + return true, nil } } } else { // Access type doesn't change - if err := r.updateNetworkAccess(adb, difADB); err != nil { - return err + sent, err := r.validateNetworkAccess(logger, adb, difADB, ociADB) + if err != nil { + return false, err + } + if sent { + return true, nil } - if err := r.updateMTLS(adb, difADB); err != nil { - return err + sent, err = r.validateMTLS(logger, adb, difADB, ociADB) + if err != nil { + return false, err + } + if sent { + return true, nil } } } else { // Dedicated database - if err := r.updateNetworkAccess(adb, difADB); err != nil { - return err + sent, err := r.validateNetworkAccess(logger, adb, difADB, ociADB) + if err != nil { + return false, err + } + if sent { + return true, nil } } + return false, nil +} + +// Set the mTLS to true but not changing the spec +func (r *AutonomousDatabaseReconciler) setMTLSRequired(logger logr.Logger, adb *dbv1alpha1.AutonomousDatabase) error { + logger.WithName("setMTLSRequired").Info("Sending request to OCI to set IsMtlsConnectionRequired to true") + + adb.Spec.Details.NetworkAccess.IsMTLSConnectionRequired = common.Bool(true) + + resp, err := r.dbService.UpdateNetworkAccessMTLSRequired(*adb.Spec.Details.AutonomousDatabaseOCID) + if err != nil { + return err + } + + adb.UpdateStatusFromOCIADB(resp.AutonomousDatabase) + return nil } -func (r *AutonomousDatabaseReconciler) downloadWallet(adb *dbv1alpha1.AutonomousDatabase) error { +func (r *AutonomousDatabaseReconciler) validateMTLS( + logger logr.Logger, + adb *dbv1alpha1.AutonomousDatabase, + difADB *dbv1alpha1.AutonomousDatabase, + ociADB *dbv1alpha1.AutonomousDatabase) (sent bool, err error) { + + if difADB.Spec.Details.NetworkAccess.IsMTLSConnectionRequired == nil { + return false, nil + } + + logger.WithName("validateMTLS").Info("Sending request to OCI to configure IsMtlsConnectionRequired") + + resp, err := r.dbService.UpdateNetworkAccessMTLS(*adb.Spec.Details.AutonomousDatabaseOCID, difADB) + if err != nil { + return false, err + } + + adb.UpdateStatusFromOCIADB(resp.AutonomousDatabase) + + return true, nil +} + +func (r *AutonomousDatabaseReconciler) setNetworkAccessPublic(logger logr.Logger, lastAcessType dbv1alpha1.NetworkAccessTypeEnum, adb *dbv1alpha1.AutonomousDatabase) error { + adb.Spec.Details.NetworkAccess.AccessType = dbv1alpha1.NetworkAccessTypePublic + adb.Spec.Details.NetworkAccess.AccessControlList = nil + adb.Spec.Details.NetworkAccess.PrivateEndpoint.HostnamePrefix = common.String("") + adb.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs = nil + adb.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID = nil + + logger.WithName("setNetworkAccessPublic").Info("Sending request to OCI to configure network access options to PUBLIC") + + resp, err := r.dbService.UpdateNetworkAccessPublic(lastAcessType, *adb.Spec.Details.AutonomousDatabaseOCID) + if err != nil { + return err + } + + adb.UpdateStatusFromOCIADB(resp.AutonomousDatabase) + + return nil +} + +func (r *AutonomousDatabaseReconciler) validateNetworkAccess( + logger logr.Logger, + adb *dbv1alpha1.AutonomousDatabase, + difADB *dbv1alpha1.AutonomousDatabase, + ociADB *dbv1alpha1.AutonomousDatabase) (sent bool, err error) { + + if difADB.Spec.Details.NetworkAccess.AccessType == "" && + difADB.Spec.Details.NetworkAccess.IsAccessControlEnabled == nil && + difADB.Spec.Details.NetworkAccess.AccessControlList == nil && + difADB.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID == nil && + difADB.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs == nil && + difADB.Spec.Details.NetworkAccess.PrivateEndpoint.HostnamePrefix == nil { + return false, nil + } + + logger.WithName("validateNetworkAccess").Info("Sending request to OCI to configure network access options") + + resp, err := r.dbService.UpdateNetworkAccess(*adb.Spec.Details.AutonomousDatabaseOCID, difADB) + if err != nil { + return false, err + } + + adb.UpdateStatusFromOCIADB(resp.AutonomousDatabase) + + return true, nil +} + +func (r *AutonomousDatabaseReconciler) validateWallet(logger logr.Logger, adb *dbv1alpha1.AutonomousDatabase) error { if adb.Spec.Details.Wallet.Name == nil && adb.Spec.Details.Wallet.Password.K8sSecret.Name == nil && adb.Spec.Details.Wallet.Password.OCISecret.OCID == nil { return nil } - logger := r.Log.WithName("download-wallet") + if adb.Status.LifecycleState == database.AutonomousDatabaseLifecycleStateProvisioning { + return nil + } + + l := logger.WithName("validateWallet") // lastSucSpec may be nil if this is the first time entering the reconciliation loop var walletName string @@ -942,7 +1045,7 @@ func (r *AutonomousDatabaseReconciler) downloadWallet(adb *dbv1alpha1.Autonomous val, ok := secret.Labels["app"] if !ok || val != adb.Name { // Overwrite if the fetched secret has a different label - logger.Info("wallet existed but has a different label; skip the download") + l.Info("wallet existed but has a different label; skip the download") } // No-op if Wallet is already downloaded return nil @@ -966,42 +1069,58 @@ func (r *AutonomousDatabaseReconciler) downloadWallet(adb *dbv1alpha1.Autonomous return err } - logger.Info(fmt.Sprintf("Wallet is stored in the Secret %s", walletName)) + l.Info(fmt.Sprintf("Wallet is stored in the Secret %s", walletName)) return nil } -func (r *AutonomousDatabaseReconciler) updateADB(adb *dbv1alpha1.AutonomousDatabase, difADB *dbv1alpha1.AutonomousDatabase) error { - if err := r.updateGeneralFields(adb, difADB); err != nil { - return err - } +// updateBackupResources get the list of AutonomousDatabasBackups and +// create a backup object if it's not found in the same namespace +func (r *AutonomousDatabaseReconciler) syncBackupResources(logger logr.Logger, adb *dbv1alpha1.AutonomousDatabase) error { + l := logger.WithName("syncBackupResources") - if err := r.updateAdminPassword(adb, difADB); err != nil { + // Get the list of AutonomousDatabaseBackupOCID in the same namespace + backupList, err := k8s.FetchAutonomousDatabaseBackups(r.KubeClient, adb.Namespace) + if err != nil { return err } - if err := r.updateDbWorkload(adb, difADB); err != nil { - return err - } + curBackupNames := make(map[string]bool) + curBackupOCIDs := make(map[string]bool) - if err := r.updateLicenseModel(adb, difADB); err != nil { - return err - } + for _, backup := range backupList.Items { + // mark the backup name that exists + curBackupNames[backup.Name] = true - if err := r.updateScalingFields(adb, difADB); err != nil { - return err + // mark the backup ocid that exists + if backup.Spec.AutonomousDatabaseBackupOCID != nil { + curBackupOCIDs[*backup.Spec.AutonomousDatabaseBackupOCID] = true + } } - if err := r.determineNetworkAccessUpdate(adb, difADB); err != nil { + resp, err := r.dbService.ListAutonomousDatabaseBackups(*adb.Spec.Details.AutonomousDatabaseOCID) + if err != nil { return err } - if err := r.updateLifecycleState(adb, difADB); err != nil { - return err - } + for _, backupSummary := range resp.Items { + // Create the resource if the backup doesn't exist + if !r.ifBackupExists(backupSummary, curBackupOCIDs, backupList) { + validBackupName, err := r.getValidBackupName(*backupSummary.DisplayName, curBackupNames) + if err != nil { + return err + } - if err := r.downloadWallet(adb); err != nil { - return err + if err := k8s.CreateAutonomousBackup(r.KubeClient, validBackupName, backupSummary, adb); err != nil { + return err + } + + // Add the used name and ocid + curBackupNames[validBackupName] = true + curBackupOCIDs[*backupSummary.AutonomousDatabaseId] = true + + l.Info("Create AutonomousDatabaseBackup " + validBackupName) + } } return nil @@ -1037,14 +1156,13 @@ func (r *AutonomousDatabaseReconciler) ifBackupExists(backupSummary database.Aut } // Special case: when a Backup is creating and hasn't updated the OCID, a duplicated Backup might be created by mistake. - // To handle this case, skip creating the backup if the current backupSummary is with CREATING state, and there is - // another AutonomousBackup with the same displayName in the cluster is also at CREATING state. + // To handle this case, skip creating the AutonomousDatabaseBackup resource if the current backupSummary is with CREATING state, + // and there is another AutonomousBackup with the same displayName in the cluster is also at CREATING state. if backupSummary.LifecycleState == database.AutonomousDatabaseBackupSummaryLifecycleStateCreating { for _, backup := range backupList.Items { if (backup.Spec.DisplayName != nil && *backup.Spec.DisplayName == *backupSummary.DisplayName) && (backup.Status.LifecycleState == "" || - backup.Status.LifecycleState == dbv1alpha1.BackupStateError || - backup.Status.LifecycleState == dbv1alpha1.BackupStateCreating) { + backup.Status.LifecycleState == database.AutonomousDatabaseBackupLifecycleStateCreating) { return true } } @@ -1052,55 +1170,3 @@ func (r *AutonomousDatabaseReconciler) ifBackupExists(backupSummary database.Aut return false } - -// updateBackupResources get the list of AutonomousDatabasBackups and -// create a backup object if it's not found in the same namespace -func (r *AutonomousDatabaseReconciler) syncBackupResources(adb *dbv1alpha1.AutonomousDatabase) error { - logger := r.Log.WithName("update-backups") - - // Get the list of AutonomousDatabaseBackupOCID in the same namespace - backupList, err := k8s.FetchAutonomousDatabaseBackups(r.KubeClient, adb.Namespace) - if err != nil { - return err - } - - curBackupNames := make(map[string]bool) - curBackupOCIDs := make(map[string]bool) - - for _, backup := range backupList.Items { - // mark the backup name that exists - curBackupNames[backup.Name] = true - - // mark the backup ocid that exists - if backup.Status.AutonomousDatabaseBackupOCID != "" { - curBackupOCIDs[backup.Status.AutonomousDatabaseBackupOCID] = true - } - } - - resp, err := r.dbService.ListAutonomousDatabaseBackups(*adb.Spec.Details.AutonomousDatabaseOCID) - if err != nil { - return err - } - - for _, backupSummary := range resp.Items { - // Create the resource if the backup doesn't exist - if !r.ifBackupExists(backupSummary, curBackupOCIDs, backupList) { - validBackupName, err := r.getValidBackupName(*backupSummary.DisplayName, curBackupNames) - if err != nil { - return err - } - - if err := k8s.CreateAutonomousBackup(r.KubeClient, validBackupName, backupSummary, adb); err != nil { - return err - } - - // Add the used name and ocid - curBackupNames[validBackupName] = true - curBackupOCIDs[*backupSummary.AutonomousDatabaseId] = true - - logger.Info("Create AutonomousDatabaseBackup " + validBackupName) - } - } - - return nil -} diff --git a/controllers/database/autonomousdatabasebackup_controller.go b/controllers/database/autonomousdatabasebackup_controller.go index 9ee1576f..cc233d5f 100644 --- a/controllers/database/autonomousdatabasebackup_controller.go +++ b/controllers/database/autonomousdatabasebackup_controller.go @@ -52,13 +52,11 @@ import ( "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/controller-runtime/pkg/controller" "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" - "sigs.k8s.io/controller-runtime/pkg/event" "sigs.k8s.io/controller-runtime/pkg/predicate" databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" "github.com/oracle/oracle-database-operator/commons/adb_family" - "github.com/oracle/oracle-database-operator/commons/k8s" "github.com/oracle/oracle-database-operator/commons/oci" ) @@ -69,54 +67,24 @@ type AutonomousDatabaseBackupReconciler struct { Scheme *runtime.Scheme Recorder record.EventRecorder - adbService oci.DatabaseService - workService oci.WorkRequestService -} - -func (r *AutonomousDatabaseBackupReconciler) eventFilterPredicate() predicate.Predicate { - pred := predicate.Funcs{ - CreateFunc: func(e event.CreateEvent) bool { - return true - }, - UpdateFunc: func(e event.UpdateEvent) bool { - oldStatus := e.ObjectOld.(*dbv1alpha1.AutonomousDatabaseBackup).Status.LifecycleState - - if oldStatus == dbv1alpha1.BackupStateCreating || - oldStatus == dbv1alpha1.BackupStateDeleting { - return false - } - - return true - }, - } - - return pred + dbService oci.DatabaseService } // SetupWithManager sets up the controller with the Manager. func (r *AutonomousDatabaseBackupReconciler) SetupWithManager(mgr ctrl.Manager) error { return ctrl.NewControllerManagedBy(mgr). For(&databasev1alpha1.AutonomousDatabaseBackup{}). - WithEventFilter(r.eventFilterPredicate()). + WithEventFilter(predicate.GenerationChangedPredicate{}). WithOptions(controller.Options{MaxConcurrentReconciles: 100}). // ReconcileHandler is never invoked concurrently with the same object. Complete(r) } -func (r *AutonomousDatabaseBackupReconciler) backupStarted(backup *dbv1alpha1.AutonomousDatabaseBackup) bool { - return backup.Spec.AutonomousDatabaseBackupOCID != nil || - (backup.Status.LifecycleState == dbv1alpha1.BackupStateCreating || - backup.Status.LifecycleState == dbv1alpha1.BackupStateActive || - backup.Status.LifecycleState == dbv1alpha1.BackupStateDeleting || - backup.Status.LifecycleState == dbv1alpha1.BackupStateDeleted || - backup.Status.LifecycleState == dbv1alpha1.BackupStateFailed) -} - //+kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabasebackups,verbs=get;list;watch;create;delete //+kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabasebackups/status,verbs=get;update;patch //+kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabases,verbs=get;list func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { - logger := r.Log.WithValues("Namespaced/Name", req.NamespacedName) + logger := r.Log.WithValues("Namespace/Name", req.NamespacedName) backup := &dbv1alpha1.AutonomousDatabaseBackup{} if err := r.KubeClient.Get(context.TODO(), req.NamespacedName, backup); err != nil { @@ -139,7 +107,7 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req } /****************************************************************** - * Get OCI database client and work request client + * Get OCI database client ******************************************************************/ if err := r.setupOCIClients(backup); err != nil { return r.manageError(backup, err) @@ -148,67 +116,85 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req logger.Info("OCI clients configured succesfully") /****************************************************************** - * If the Spec.AutonomousDatabaseBackupOCID is empty and the LifecycleState is never assigned , create a backup. - * LifecycleState is checked to avoid sending a duplicated backup request when the backup is creating. - * Otherwise, bind to an exisiting backup if the Spec.AutonomousDatabaseBackupOCID isn't empty. + * Get status from OCI AutonomousDatabaseBackup + ******************************************************************/ + if backup.Spec.AutonomousDatabaseBackupOCID != nil { + backupResp, err := r.dbService.GetAutonomousDatabaseBackup(*backup.Spec.AutonomousDatabaseBackupOCID) + if err != nil { + return r.manageError(backup, err) + } + + adbResp, err := r.dbService.GetAutonomousDatabase(*backupResp.AutonomousDatabaseId) + if err != nil { + return r.manageError(backup, err) + } + + backup.UpdateStatusFromOCIBackup(backupResp.AutonomousDatabaseBackup, adbResp.AutonomousDatabase) + } + + /****************************************************************** + * Requeue if the Backup is in an intermediate state + * No-op if the ADB OCID is nil + * To get the latest status, execute before all the reconcile logic + ******************************************************************/ + if dbv1alpha1.IsBackupIntermediateState(backup.Status.LifecycleState) { + logger.WithName("validateLifecycleState").Info("Reconcile queued") + return requeueResult, nil + } + + /****************************************************************** + * If the spec.autonomousDatabaseBackupOCID is empty. + * Otherwise, bind to an exisiting backup. ******************************************************************/ - if !r.backupStarted(backup) { + if backup.Spec.AutonomousDatabaseBackupOCID == nil { // Create a new backup - backupResp, err := r.adbService.CreateAutonomousDatabaseBackup(backup, adbOCID) + backupResp, err := r.dbService.CreateAutonomousDatabaseBackup(backup, adbOCID) if err != nil { return r.manageError(backup, err) } - adbResp, err := r.adbService.GetAutonomousDatabase(*backupResp.AutonomousDatabaseId) + logger.Info("AutonomousDatabaseBackup " + *backupResp.DisplayName + " started") + + // After the creation, update the status first + adbResp, err := r.dbService.GetAutonomousDatabase(*backupResp.AutonomousDatabaseId) if err != nil { return r.manageError(backup, err) } - // update the Backup status backup.UpdateStatusFromOCIBackup(backupResp.AutonomousDatabaseBackup, adbResp.AutonomousDatabase) if err := r.KubeClient.Status().Update(context.TODO(), backup); err != nil { return r.manageError(backup, err) } - // Wait until the work is done - if _, err := r.workService.Wait(*backupResp.OpcWorkRequestId); err != nil { - return r.manageError(backup, err) - } + // Then update the OCID + backup.Spec.AutonomousDatabaseBackupOCID = backupResp.Id + backup.UpdateStatusFromOCIBackup(backupResp.AutonomousDatabaseBackup, adbResp.AutonomousDatabase) - logger.Info("AutonomousDatabaseBackup " + *backupResp.DisplayName + " created successfully") - } + if err := r.KubeClient.Update(context.TODO(), backup); err != nil { + // Do no requeue otherwise it will create multiple backups + r.Recorder.Event(backup, corev1.EventTypeWarning, "ReconcileFailed", err.Error()) + logger.Error(err, "cannot update AutonomousDatabaseBackupOCID; stop reconcile", "AutonomousDatabaseBackupOCID", *backupResp.Id) - /****************************************************************** - * Sync the resource status - *******************************************************************/ - // get the backup ID - var backupID string - if backup.Spec.AutonomousDatabaseBackupOCID != nil { - backupID = *backup.Spec.AutonomousDatabaseBackupOCID - } else if backup.Status.AutonomousDatabaseBackupOCID != "" { - backupID = backup.Status.AutonomousDatabaseBackupOCID - } else { - // Send the event and exit the Reconcile - err := errors.New("the backup is incomplete and missing the OCID; the resource should be removed") - logger.Error(err, "Reconcile stopped") - r.Recorder.Event(backup, corev1.EventTypeWarning, "ReconcileFailed", err.Error()) + return emptyResult, nil + } + + logger.WithName("createAutonomousDatabaseBackup").Info("AutonomousDatabaseBackupOCID updated") return emptyResult, nil } - backupResp, err := r.adbService.GetAutonomousDatabaseBackup(backupID) - if err != nil { + /****************************************************************** + * Update the status and requeue if it's in an intermediate state + ******************************************************************/ + if err := r.KubeClient.Status().Update(context.TODO(), backup); err != nil { return r.manageError(backup, err) } - adbResp, err := r.adbService.GetAutonomousDatabase(*backupResp.AutonomousDatabaseId) - if err != nil { - return r.manageError(backup, err) + if dbv1alpha1.IsBackupIntermediateState(backup.Status.LifecycleState) { + logger.WithName("IsIntermediateState").Info("Reconcile queued") + return requeueResult, nil } - backup.UpdateStatusFromOCIBackup(backupResp.AutonomousDatabaseBackup, adbResp.AutonomousDatabase) - if err := r.KubeClient.Status().Update(context.TODO(), backup); err != nil { - return r.manageError(backup, err) - } + logger.Info("AutonomousDatabaseBackup reconciles successfully") return emptyResult, nil } @@ -230,7 +216,7 @@ func (r *AutonomousDatabaseBackupReconciler) setOwnerAutonomousDatabase(backup * // The function returns the OCID of the target ADB. func (r *AutonomousDatabaseBackupReconciler) verifyTargetADB(backup *dbv1alpha1.AutonomousDatabaseBackup) (string, error) { // Get the target ADB OCID and the ADB resource - ocid, ownerADB, err := adbfamily.VerifyTargetADB(r.KubeClient, backup.Spec.Target, backup.Namespace) + ownerADB, err := adbfamily.VerifyTargetADB(r.KubeClient, backup.Spec.Target, backup.Namespace) if err != nil { return "", err @@ -243,7 +229,14 @@ func (r *AutonomousDatabaseBackupReconciler) verifyTargetADB(backup *dbv1alpha1. } } - return ocid, nil + if backup.Spec.Target.OCIADB.OCID != nil { + return *backup.Spec.Target.OCIADB.OCID, nil + } + if ownerADB != nil && ownerADB.Spec.Details.AutonomousDatabaseOCID != nil { + return *ownerADB.Spec.Details.AutonomousDatabaseOCID, nil + } + + return "", errors.New("cannot get the OCID of the targetADB") } func (r *AutonomousDatabaseBackupReconciler) setupOCIClients(backup *dbv1alpha1.AutonomousDatabaseBackup) error { @@ -260,12 +253,7 @@ func (r *AutonomousDatabaseBackupReconciler) setupOCIClients(backup *dbv1alpha1. return err } - r.adbService, err = oci.NewDatabaseService(r.Log, r.KubeClient, provider) - if err != nil { - return err - } - - r.workService, err = oci.NewWorkRequestService(r.Log, r.KubeClient, provider) + r.dbService, err = oci.NewDatabaseService(r.Log, r.KubeClient, provider) if err != nil { return err } @@ -277,11 +265,5 @@ func (r *AutonomousDatabaseBackupReconciler) manageError(backup *dbv1alpha1.Auto // Send event r.Recorder.Event(backup, corev1.EventTypeWarning, "ReconcileFailed", issue.Error()) - // Change the status to ERROR - backup.Status.LifecycleState = dbv1alpha1.BackupStateError - if statusErr := r.KubeClient.Status().Update(context.TODO(), backup); statusErr != nil { - return emptyResult, k8s.CombineErrors(issue, statusErr) - } - return emptyResult, issue } diff --git a/controllers/database/autonomousdatabaserestore_controller.go b/controllers/database/autonomousdatabaserestore_controller.go index 7ad1f591..876ac5dc 100644 --- a/controllers/database/autonomousdatabaserestore_controller.go +++ b/controllers/database/autonomousdatabaserestore_controller.go @@ -47,9 +47,7 @@ import ( corev1 "k8s.io/api/core/v1" apiErrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/runtime" - "k8s.io/apimachinery/pkg/types" "k8s.io/client-go/tools/record" - "k8s.io/client-go/util/retry" ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" @@ -70,7 +68,7 @@ type AutonomousDatabaseRestoreReconciler struct { Scheme *runtime.Scheme Recorder record.EventRecorder - adbService oci.DatabaseService + dbService oci.DatabaseService workService oci.WorkRequestService } @@ -82,13 +80,6 @@ func (r *AutonomousDatabaseRestoreReconciler) SetupWithManager(mgr ctrl.Manager) Complete(r) } -func (r *AutonomousDatabaseRestoreReconciler) restoreStarted(restore *dbv1alpha1.AutonomousDatabaseRestore) bool { - return restore.Status.TimeAccepted != "" || - (restore.Status.Status == dbv1alpha1.RestoreStatusInProgress || - restore.Status.Status == dbv1alpha1.RestoreStatusFailed || - restore.Status.Status == dbv1alpha1.RestoreStatusSucceeded) -} - //+kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabaserestores,verbs=get;list;watch;create;delete //+kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabaserestores/status,verbs=get;update;patch //+kubebuilder:rbac:groups=database.oracle.com,resources=autonomousdatabases,verbs=get;list @@ -103,7 +94,7 @@ func (r *AutonomousDatabaseRestoreReconciler) restoreStarted(restore *dbv1alpha1 // For more details, check Reconcile and its Result here: // - https://pkg.go.dev/sigs.k8s.io/controller-runtime@v0.6.4/pkg/reconcile func (r *AutonomousDatabaseRestoreReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { - logger := r.Log.WithValues("Namespaced/Name", req.NamespacedName) + logger := r.Log.WithValues("Namespace/Name", req.NamespacedName) restore := &dbv1alpha1.AutonomousDatabaseRestore{} if err := r.KubeClient.Get(context.TODO(), req.NamespacedName, restore); err != nil { @@ -116,12 +107,6 @@ func (r *AutonomousDatabaseRestoreReconciler) Reconcile(ctx context.Context, req return emptyResult, err } - if r.restoreStarted(restore) { - return emptyResult, nil - } - - // ===================== Run Restore for the Target ADB ============================ - /****************************************************************** * Look up the owner AutonomousDatabase and set the ownerReference * if the owner hasn't been set yet. @@ -149,10 +134,68 @@ func (r *AutonomousDatabaseRestoreReconciler) Reconcile(ctx context.Context, req logger.Info("OCI clients configured succesfully") /****************************************************************** - * Start restore + * Get status from OCI WorkRequest + ******************************************************************/ + if restore.Status.WorkRequestOCID != "" { + resp, err := r.workService.Get(restore.Status.WorkRequestOCID) + if err != nil { + return r.manageError(restore, err) + } + + restore.Status.Status = resp.Status + + if dbv1alpha1.IsRestoreIntermediateState(resp.Status) { + logger.WithName("validateStatus").Info("Reconcile queued") + return requeueResult, nil + } + } + + /****************************************************************** + * Start the restore or update the status ******************************************************************/ - if err := r.restoreAutonomousDatabase(restore, restoreTime, adbOCID); err != nil { - return r.manageError(restore, err) + if restore.Status.WorkRequestOCID == "" { + // Start restore + adbResp, err := r.dbService.RestoreAutonomousDatabase(adbOCID, *restoreTime) + if err != nil { + return r.manageError(restore, err) + } + + workResp, err := r.workService.Get(*adbResp.OpcWorkRequestId) + if err != nil { + return r.manageError(restore, err) + } + + restore.UpdateStatus(adbResp.AutonomousDatabase, workResp) + if err := r.KubeClient.Update(context.TODO(), restore); err != nil { + return r.manageError(restore, err) + } + + } else { + // Update the status + adbResp, err := r.dbService.GetAutonomousDatabase(adbOCID) + if err != nil { + return r.manageError(restore, err) + } + + workResp, err := r.workService.Get(restore.Status.WorkRequestOCID) + if err != nil { + return r.manageError(restore, err) + } + + restore.UpdateStatus(adbResp.AutonomousDatabase, workResp) + if err := r.KubeClient.Update(context.TODO(), restore); err != nil { + return r.manageError(restore, err) + } + + if err := r.KubeClient.Update(context.TODO(), restore); err != nil { + return r.manageError(restore, err) + } + } + + // Requeue if it's in intermediate state + if dbv1alpha1.IsRestoreIntermediateState(restore.Status.Status) { + logger.WithName("validateStatus").Info("Reconcile queued") + return requeueResult, nil } logger.Info("AutonomousDatabaseRestore reconciles successfully") @@ -201,7 +244,7 @@ func (r *AutonomousDatabaseRestoreReconciler) setOwnerAutonomousDatabase(restore // The function returns the OCID of the target ADB. func (r *AutonomousDatabaseRestoreReconciler) verifyTargetADB(restore *dbv1alpha1.AutonomousDatabaseRestore) (string, error) { // Get the target ADB OCID and the ADB resource - ocid, ownerADB, err := adbfamily.VerifyTargetADB(r.KubeClient, restore.Spec.Target, restore.Namespace) + ownerADB, err := adbfamily.VerifyTargetADB(r.KubeClient, restore.Spec.Target, restore.Namespace) if err != nil { return "", err @@ -214,25 +257,14 @@ func (r *AutonomousDatabaseRestoreReconciler) verifyTargetADB(restore *dbv1alpha } } - return ocid, nil -} - -func (r *AutonomousDatabaseRestoreReconciler) updateResourceStatus(restore *dbv1alpha1.AutonomousDatabaseRestore) error { - return retry.RetryOnConflict(retry.DefaultRetry, func() error { - curBackup := &dbv1alpha1.AutonomousDatabaseRestore{} - - namespacedName := types.NamespacedName{ - Namespace: restore.GetNamespace(), - Name: restore.GetName(), - } - - if err := r.KubeClient.Get(context.TODO(), namespacedName, curBackup); err != nil { - return err - } + if restore.Spec.Target.OCIADB.OCID != nil { + return *restore.Spec.Target.OCIADB.OCID, nil + } + if ownerADB != nil && ownerADB.Spec.Details.AutonomousDatabaseOCID != nil { + return *ownerADB.Spec.Details.AutonomousDatabaseOCID, nil + } - curBackup.Status = restore.Status - return r.KubeClient.Status().Update(context.TODO(), curBackup) - }) + return "", errors.New("cannot get the OCID of the targetADB") } func (r *AutonomousDatabaseRestoreReconciler) setupOCIClients(restore *dbv1alpha1.AutonomousDatabaseRestore) error { @@ -249,7 +281,7 @@ func (r *AutonomousDatabaseRestoreReconciler) setupOCIClients(restore *dbv1alpha return err } - r.adbService, err = oci.NewDatabaseService(r.Log, r.KubeClient, provider) + r.dbService, err = oci.NewDatabaseService(r.Log, r.KubeClient, provider) if err != nil { return err } @@ -264,65 +296,8 @@ func (r *AutonomousDatabaseRestoreReconciler) setupOCIClients(restore *dbv1alpha // manageError doesn't return the error so that the request won't be requeued func (r *AutonomousDatabaseRestoreReconciler) manageError(restore *dbv1alpha1.AutonomousDatabaseRestore, issue error) (ctrl.Result, error) { - nsn := types.NamespacedName{ - Namespace: restore.Namespace, - Name: restore.Name, - } - logger := r.Log.WithValues("Namespaced/Name", nsn) - // Send event r.Recorder.Event(restore, corev1.EventTypeWarning, "ReconcileFailed", issue.Error()) - // Change the status to ERROR - var combinedErr error = issue - - restore.Status.Status = dbv1alpha1.RestoreStatusError - if statusErr := r.updateResourceStatus(restore); statusErr != nil { - combinedErr = k8s.CombineErrors(issue, statusErr) - } - - logger.Error(combinedErr, "Fail to restore Autonomous Database") - - return emptyResult, nil -} - -func (r *AutonomousDatabaseRestoreReconciler) restoreAutonomousDatabase( - restore *dbv1alpha1.AutonomousDatabaseRestore, - restoreTime *common.SDKTime, - adbOCID string) error { - var err error - - resp, err := r.adbService.RestoreAutonomousDatabase(adbOCID, *restoreTime) - if err != nil { - return err - } - - // Update status and wait for the work finish if a request is sent. Note that some of the requests (e.g. update displayName) won't return a work request ID. - // It's important to update the status by reference otherwise Reconcile() won't be able to get the latest values - restore.Status.DisplayName = *resp.AutonomousDatabase.DisplayName - restore.Status.DbName = *resp.AutonomousDatabase.DbName - restore.Status.AutonomousDatabaseOCID = *resp.AutonomousDatabase.Id - - workStart, err := r.workService.Get(*resp.OpcWorkRequestId) - if err != nil { - return err - } - restore.Status.Status = restore.ConvertWorkRequestStatus(workStart.Status) - restore.Status.TimeAccepted = dbv1alpha1.FormatSDKTime(workStart.TimeAccepted) - - r.updateResourceStatus(restore) - - workEnd, err := r.workService.Wait(*resp.OpcWorkRequestId) - if err != nil { - return err - } - - // Update status when the work is finished - restore.Status.Status = restore.ConvertWorkRequestStatus(workEnd.Status) - restore.Status.TimeStarted = dbv1alpha1.FormatSDKTime(workEnd.TimeStarted) - restore.Status.TimeEnded = dbv1alpha1.FormatSDKTime(workEnd.TimeFinished) - - r.updateResourceStatus(restore) - - return nil + return emptyResult, issue } diff --git a/controllers/database/suite_test.go b/controllers/database/suite_test.go index fdebedce..60a0d659 100644 --- a/controllers/database/suite_test.go +++ b/controllers/database/suite_test.go @@ -42,13 +42,12 @@ import ( "path/filepath" "testing" - . "github.com/onsi/ginkgo" + . "github.com/onsi/ginkgo/v2" . "github.com/onsi/gomega" "k8s.io/client-go/kubernetes/scheme" "k8s.io/client-go/rest" "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/controller-runtime/pkg/envtest" - "sigs.k8s.io/controller-runtime/pkg/envtest/printer" logf "sigs.k8s.io/controller-runtime/pkg/log" "sigs.k8s.io/controller-runtime/pkg/log/zap" @@ -66,12 +65,10 @@ var testEnv *envtest.Environment func TestAPIs(t *testing.T) { RegisterFailHandler(Fail) - RunSpecsWithDefaultAndCustomReporters(t, - "Controller Suite", - []Reporter{printer.NewlineReporter{}}) + RunSpecs(t, "Controller Suite") } -var _ = BeforeSuite(func(done Done) { +var _ = BeforeSuite(func() { logf.SetLogger(zap.New(zap.WriteTo(GinkgoWriter), zap.UseDevMode(true))) By("bootstrapping test environment") @@ -95,9 +92,7 @@ var _ = BeforeSuite(func(done Done) { k8sClient, err = client.New(cfg, client.Options{Scheme: scheme.Scheme}) Expect(err).ToNot(HaveOccurred()) Expect(k8sClient).ToNot(BeNil()) - - close(done) -}, 60) +}) var _ = AfterSuite(func() { By("tearing down the test environment") diff --git a/docs/acd/README.md b/docs/acd/README.md index b0cc05d3..cd3a4303 100644 --- a/docs/acd/README.md +++ b/docs/acd/README.md @@ -21,7 +21,6 @@ After you create the resource, you can use the operator to perform the following * [Change the display name](#change-the-display-name) of an Autonomous Container Database * [Restart/Terminate](#restartterminate) an Autonomous Container Database -* [Sync](#sync-the-resource-manually) of an Autonomous Container Database manually * [Delete the resource](#delete-the-resource) from the cluster ## Provision an Autonomous Container Database @@ -197,35 +196,6 @@ Here's a list of the values you can set for `action`: autonomouscontainerdatabase.database.oracle.com/autonomouscontainerdatabase-sample configured ``` -## Sync the resource manually - -> Note: this operation requires an `AutonomousContainerDatabase` object to be in your cluster. This example assumes the provision operation or the bind operation has been done by the users and the operator is authorized with API Key Authentication. - -Users can sync the resource manually by setting the value of the `action` attribute to SYNC. The Operator may not response immediately if it is still waiting for a work to finish. - -1. A sample .yaml file is available here: [config/samples/acd/autonomouscontainerdatabase_sync.yaml](./../../config/samples/acd/autonomouscontainerdatabase_sync.yaml) - - ```yaml - --- - apiVersion: database.oracle.com/v1alpha1 - kind: AutonomousContainerDatabase - metadata: - name: autonomouscontainerdatabase-sample - spec: - autonomousContainerDatabaseOCID: ocid1.autonomouscontainerdatabase... - action: SYNC - ociConfig: - configMapName: oci-cred - secretName: oci-privatekey - ``` - -2. Apply the change to sync the database. - - ```sh - kubectl apply -f config/samples/acd/autonomouscontainerdatabase_sync.yaml - autonomouscontainerdatabase.database.oracle.com/autonomouscontainerdatabase-sample configured - ``` - ## Delete the resource > Note: this operation requires an `AutonomousContainerDatabase` object to be in your cluster. This example assumes the provision operation or the bind operation has been done by the users and the operator is authorized with API Key Authentication. diff --git a/go.mod b/go.mod index df92e9e5..d5974bfa 100644 --- a/go.mod +++ b/go.mod @@ -4,13 +4,13 @@ go 1.17 require ( github.com/go-logr/logr v1.2.2 - github.com/onsi/ginkgo v1.16.5 - github.com/onsi/gomega v1.18.1 + github.com/onsi/ginkgo/v2 v2.1.3 + github.com/onsi/gomega v1.19.0 github.com/oracle/oci-go-sdk/v63 v63.0.0 gopkg.in/yaml.v2 v2.4.0 - k8s.io/api v0.23.3 - k8s.io/apimachinery v0.23.3 - k8s.io/client-go v0.23.3 + k8s.io/api v0.23.5 + k8s.io/apimachinery v0.23.5 + k8s.io/client-go v0.23.5 sigs.k8s.io/controller-runtime v0.11.0 sigs.k8s.io/yaml v1.3.0 ) @@ -38,6 +38,7 @@ require ( github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect github.com/modern-go/reflect2 v1.0.2 // indirect github.com/nxadm/tail v1.4.8 // indirect + github.com/onsi/ginkgo v1.16.5 // indirect github.com/pkg/errors v0.9.1 // indirect github.com/prometheus/client_golang v1.12.1 // indirect github.com/prometheus/client_model v0.2.0 // indirect @@ -48,7 +49,7 @@ require ( go.uber.org/atomic v1.9.0 // indirect go.uber.org/multierr v1.7.0 // indirect go.uber.org/zap v1.21.0 // indirect - golang.org/x/net v0.0.0-20220127200216-cd36cc0744dd // indirect + golang.org/x/net v0.0.0-20220225172249-27dd8689420f // indirect golang.org/x/oauth2 v0.0.0-20211104180415-d3ed0bb246c8 // indirect golang.org/x/sys v0.0.0-20220209214540-3681064d5158 // indirect golang.org/x/term v0.0.0-20210927222741-03fcf44c2211 // indirect diff --git a/go.sum b/go.sum index bf657b15..d2aeef54 100644 --- a/go.sum +++ b/go.sum @@ -386,14 +386,14 @@ github.com/onsi/ginkgo v1.14.0/go.mod h1:iSB4RoI2tjJc9BBv4NKIKWKya62Rps+oPG/Lv9k github.com/onsi/ginkgo v1.16.4/go.mod h1:dX+/inL/fNMqNlz0e9LfyB9TswhZpCVdJM/Z6Vvnwo0= github.com/onsi/ginkgo v1.16.5 h1:8xi0RTUf59SOSfEtZMvwTvXYMzG4gV23XVHOZiXNtnE= github.com/onsi/ginkgo v1.16.5/go.mod h1:+E8gABHa3K6zRBolWtd+ROzc/U5bkGt0FwiG042wbpU= -github.com/onsi/ginkgo/v2 v2.0.0 h1:CcuG/HvWNkkaqCUpJifQY8z7qEMBJya6aLPx6ftGyjQ= -github.com/onsi/ginkgo/v2 v2.0.0/go.mod h1:vw5CSIxN1JObi/U8gcbwft7ZxR2dgaR70JSE3/PpL4c= +github.com/onsi/ginkgo/v2 v2.1.3 h1:e/3Cwtogj0HA+25nMP1jCMDIf8RtRYbGwGGuBIFztkc= +github.com/onsi/ginkgo/v2 v2.1.3/go.mod h1:vw5CSIxN1JObi/U8gcbwft7ZxR2dgaR70JSE3/PpL4c= github.com/onsi/gomega v0.0.0-20170829124025-dcabb60a477c/go.mod h1:C1qb7wdrVGGVU+Z6iS04AVkA3Q65CEZX59MT0QO5uiA= github.com/onsi/gomega v1.7.1/go.mod h1:XdKZgCCFLUoM/7CFJVPcG8C1xQ1AJ0vpAezJrB7JYyY= github.com/onsi/gomega v1.10.1/go.mod h1:iN09h71vgCQne3DLsj+A5owkum+a2tYe+TOCB1ybHNo= github.com/onsi/gomega v1.17.0/go.mod h1:HnhC7FXeEQY45zxNK3PPoIUhzk/80Xly9PcubAlGdZY= -github.com/onsi/gomega v1.18.1 h1:M1GfJqGRrBrrGGsbxzV5dqM2U2ApXefZCQpkukxYRLE= -github.com/onsi/gomega v1.18.1/go.mod h1:0q+aL8jAiMXy9hbwj2mr5GziHiwhAIQpFmmtT5hitRs= +github.com/onsi/gomega v1.19.0 h1:4ieX6qQjPP/BfC3mpsAtIGGlxTWPeA3Inl/7DtXw1tw= +github.com/onsi/gomega v1.19.0/go.mod h1:LY+I3pBVzYsTBU1AnDwOSxaYi9WoWiqgwooUqq9yPro= github.com/opentracing/opentracing-go v1.1.0/go.mod h1:UkNAQd3GIcIGf0SeVgPpRdFStlNbqXla1AfSYxPUl2o= github.com/oracle/oci-go-sdk/v51 v51.0.0 h1:eDUVMsAzvf+jfq4xbtpQrxznYNjccwKKyIhmRvZLgwI= github.com/oracle/oci-go-sdk/v51 v51.0.0/go.mod h1:d9KSNXwE64drofxoor+y/JWofJqLqRF9D1/AtfYIE10= @@ -636,8 +636,8 @@ golang.org/x/net v0.0.0-20210525063256-abc453219eb5/go.mod h1:9nx3DQGgdP8bBQD5qx golang.org/x/net v0.0.0-20210805182204-aaa1db679c0d/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= golang.org/x/net v0.0.0-20210825183410-e898025ed96a/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= golang.org/x/net v0.0.0-20211209124913-491a49abca63/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= -golang.org/x/net v0.0.0-20220127200216-cd36cc0744dd h1:O7DYs+zxREGLKzKoMQrtrEacpb0ZVXA5rIwylE2Xchk= -golang.org/x/net v0.0.0-20220127200216-cd36cc0744dd/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk= +golang.org/x/net v0.0.0-20220225172249-27dd8689420f h1:oA4XRj0qtSt8Yo1Zms0CUlsT3KG69V2UGQWPBxujDmc= +golang.org/x/net v0.0.0-20220225172249-27dd8689420f/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk= golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U= golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= @@ -1021,19 +1021,22 @@ honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt honnef.co/go/tools v0.0.1-2020.1.3/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k= honnef.co/go/tools v0.0.1-2020.1.4/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k= k8s.io/api v0.23.0/go.mod h1:8wmDdLBHBNxtOIytwLstXt5E9PddnZb0GaMcqsvDBpg= -k8s.io/api v0.23.3 h1:KNrME8KHGr12Ozjf8ytOewKzZh6hl/hHUZeHddT3a38= k8s.io/api v0.23.3/go.mod h1:w258XdGyvCmnBj/vGzQMj6kzdufJZVUwEM1U2fRJwSQ= +k8s.io/api v0.23.5 h1:zno3LUiMubxD/V1Zw3ijyKO3wxrhbUF1Ck+VjBvfaoA= +k8s.io/api v0.23.5/go.mod h1:Na4XuKng8PXJ2JsploYYrivXrINeTaycCGcYgF91Xm8= k8s.io/apiextensions-apiserver v0.23.0/go.mod h1:xIFAEEDlAZgpVBl/1VSjGDmLoXAWRG40+GsWhKhAxY4= k8s.io/apiextensions-apiserver v0.23.3 h1:JvPJA7hSEAqMRteveq4aj9semilAZYcJv+9HHFWfUdM= k8s.io/apiextensions-apiserver v0.23.3/go.mod h1:/ZpRXdgKZA6DvIVPEmXDCZJN53YIQEUDF+hrpIQJL38= k8s.io/apimachinery v0.23.0/go.mod h1:fFCTTBKvKcwTPFzjlcxp91uPFZr+JA0FubU4fLzzFYc= -k8s.io/apimachinery v0.23.3 h1:7IW6jxNzrXTsP0c8yXz2E5Yx/WTzVPTsHIx/2Vm0cIk= k8s.io/apimachinery v0.23.3/go.mod h1:BEuFMMBaIbcOqVIJqNZJXGFTP4W6AycEpb5+m/97hrM= +k8s.io/apimachinery v0.23.5 h1:Va7dwhp8wgkUPWsEXk6XglXWU4IKYLKNlv8VkX7SDM0= +k8s.io/apimachinery v0.23.5/go.mod h1:BEuFMMBaIbcOqVIJqNZJXGFTP4W6AycEpb5+m/97hrM= k8s.io/apiserver v0.23.0/go.mod h1:Cec35u/9zAepDPPFyT+UMrgqOCjgJ5qtfVJDxjZYmt4= k8s.io/apiserver v0.23.3/go.mod h1:3HhsTmC+Pn+Jctw+Ow0LHA4dQ4oXrQ4XJDzrVDG64T4= k8s.io/client-go v0.23.0/go.mod h1:hrDnpnK1mSr65lHHcUuIZIXDgEbzc7/683c6hyG4jTA= -k8s.io/client-go v0.23.3 h1:23QYUmCQ/W6hW78xIwm3XqZrrKZM+LWDqW2zfo+szJs= k8s.io/client-go v0.23.3/go.mod h1:47oMd+YvAOqZM7pcQ6neJtBiFH7alOyfunYN48VsmwE= +k8s.io/client-go v0.23.5 h1:zUXHmEuqx0RY4+CsnkOn5l0GU+skkRXKGJrhmE2SLd8= +k8s.io/client-go v0.23.5/go.mod h1:flkeinTO1CirYgzMPRWxUCnV0G4Fbu2vLhYCObnt/r4= k8s.io/code-generator v0.23.0/go.mod h1:vQvOhDXhuzqiVfM/YHp+dmg10WDZCchJVObc9MvowsE= k8s.io/code-generator v0.23.3/go.mod h1:S0Q1JVA+kSzTI1oUvbKAxZY/DYbA/ZUb4Uknog12ETk= k8s.io/component-base v0.23.0/go.mod h1:DHH5uiFvLC1edCpvcTDV++NKULdYYU6pR9Tt3HIKMKI= diff --git a/main.go b/main.go index 4f79b45f..1839893a 100644 --- a/main.go +++ b/main.go @@ -190,6 +190,22 @@ func main() { setupLog.Error(err, "unable to create webhook", "webhook", "CDB") os.Exit(1) } + if err = (&databasev1alpha1.AutonomousDatabase{}).SetupWebhookWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create webhook", "webhook", "AutonomousDatabase") + os.Exit(1) + } + if err = (&databasev1alpha1.AutonomousDatabaseBackup{}).SetupWebhookWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create webhook", "webhook", "AutonomousDatabaseBackup") + os.Exit(1) + } + if err = (&databasev1alpha1.AutonomousDatabaseRestore{}).SetupWebhookWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create webhook", "webhook", "AutonomousDatabaseRestore") + os.Exit(1) + } + if err = (&databasev1alpha1.AutonomousContainerDatabase{}).SetupWebhookWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create webhook", "webhook", "AutonomousContainerDatabase") + os.Exit(1) + } } // PDB Reconciler @@ -216,28 +232,6 @@ func main() { setupLog.Error(err, "unable to create controller", "controller", "CDB") os.Exit(1) } - - if err = (&databasev1alpha1.AutonomousDatabase{}).SetupWebhookWithManager(mgr); err != nil { - setupLog.Error(err, "unable to create webhook", "webhook", "AutonomousDatabase") - os.Exit(1) - } - if err = (&databasev1alpha1.AutonomousDatabaseBackup{}).SetupWebhookWithManager(mgr); err != nil { - setupLog.Error(err, "unable to create webhook", "webhook", "AutonomousDatabaseBackup") - os.Exit(1) - } - if err = (&databasev1alpha1.AutonomousDatabaseRestore{}).SetupWebhookWithManager(mgr); err != nil { - setupLog.Error(err, "unable to create webhook", "webhook", "AutonomousDatabaseRestore") - os.Exit(1) - } - if err = (&databasecontroller.DbcsSystemReconciler{ - KubeClient: mgr.GetClient(), - Logger: ctrl.Log.WithName("controllers").WithName("database").WithName("DbcsSystem"), - Scheme: mgr.GetScheme(), - Recorder: mgr.GetEventRecorderFor("DbcsSystem"), - }).SetupWithManager(mgr); err != nil { - setupLog.Error(err, "unable to create controller", "controller", "DbcsSystem") - os.Exit(1) - } // +kubebuilder:scaffold:builder // Add index for PDB CR to enable mgr to cache PDBs diff --git a/test/e2e/behavior/shared_behaviors.go b/test/e2e/behavior/shared_behaviors.go index b5497f52..7b4616c3 100644 --- a/test/e2e/behavior/shared_behaviors.go +++ b/test/e2e/behavior/shared_behaviors.go @@ -45,7 +45,7 @@ import ( "reflect" "time" - "github.com/onsi/ginkgo" + "github.com/onsi/ginkgo/v2" "github.com/onsi/gomega" "github.com/oracle/oci-go-sdk/v63/common" "github.com/oracle/oci-go-sdk/v63/database" diff --git a/test/e2e/suite_test.go b/test/e2e/suite_test.go index 3d478e8b..f173ce1e 100644 --- a/test/e2e/suite_test.go +++ b/test/e2e/suite_test.go @@ -45,7 +45,7 @@ import ( "testing" "time" - "github.com/onsi/ginkgo" + "github.com/onsi/ginkgo/v2" "github.com/onsi/gomega" "github.com/onsi/gomega/gexec" "github.com/oracle/oci-go-sdk/v63/common" @@ -56,7 +56,6 @@ import ( ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/controller-runtime/pkg/envtest" - "sigs.k8s.io/controller-runtime/pkg/envtest/printer" logf "sigs.k8s.io/controller-runtime/pkg/log" "sigs.k8s.io/controller-runtime/pkg/log/zap" @@ -121,13 +120,10 @@ const SharedWalletPassSecretName = "adb-wallet-password" func TestAPIs(t *testing.T) { gomega.RegisterFailHandler(ginkgo.Fail) - - ginkgo.RunSpecsWithDefaultAndCustomReporters(t, - "Controller Suite", - []ginkgo.Reporter{printer.NewlineReporter{}}) + ginkgo.RunSpecs(t, "Controller Suite") } -var _ = BeforeSuite(func(done ginkgo.Done) { +var _ = BeforeSuite(func() { logf.SetLogger(zap.New(zap.WriteTo(ginkgo.GinkgoWriter), zap.UseDevMode(true))) By("bootstrapping test environment") @@ -261,9 +257,7 @@ var _ = BeforeSuite(func(done ginkgo.Done) { Expect(err).ToNot(HaveOccurred()) Expect(k8sClient.Create(context.TODO(), walletSecret)).To(Succeed()) }) - - close(done) -}, 60) +}) var _ = AfterSuite(func() { /* From 4756541ca8a9e59e8fe1392f1e79541e2113b264 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Wed, 27 Apr 2022 12:57:47 -0400 Subject: [PATCH 284/628] remove unused method --- commons/oci/workrequest.go | 58 -------------------------------------- 1 file changed, 58 deletions(-) diff --git a/commons/oci/workrequest.go b/commons/oci/workrequest.go index 37fbb9be..6257a4c8 100644 --- a/commons/oci/workrequest.go +++ b/commons/oci/workrequest.go @@ -40,8 +40,6 @@ package oci import ( "context" - "math" - "time" "github.com/go-logr/logr" "sigs.k8s.io/controller-runtime/pkg/client" @@ -52,7 +50,6 @@ import ( type WorkRequestService interface { Get(opcWorkRequestID string) (workrequests.GetWorkRequestResponse, error) - Wait(opcWorkRequestID string) (workrequests.GetWorkRequestResponse, error) List(compartmentID string, resourceID string) (workrequests.ListWorkRequestsResponse, error) } @@ -77,61 +74,6 @@ func NewWorkRequestService( }, nil } -func (w workRequestService) getRetryPolicy() common.RetryPolicy { - shouldRetry := func(r common.OCIOperationResponse) bool { - if _, isServiceError := common.IsServiceError(r.Error); isServiceError { - // Don't retry if it's service error. Sometimes it could be network error or other errors which prevents - // request send to server; we do the retry in these cases. - return false - } - - if converted, ok := r.Response.(workrequests.GetWorkRequestResponse); ok { - // do the retry until WorkReqeut Status is Succeeded - ignore case (BMI-2652) - return converted.Status != workrequests.WorkRequestStatusSucceeded && - converted.Status != workrequests.WorkRequestStatusFailed && - converted.Status != workrequests.WorkRequestStatusCanceled - } - - return true - } - - // maximum times of retry (~60mins) - attempts := uint(124) - - nextDuration := func(r common.OCIOperationResponse) time.Duration { - // Wait longer for next retry when your previous one failed - // this function will return the duration as: - // 1s, 2s, 4s, 8s, 16s, 30s, 30s etc... - if r.AttemptNumber <= 5 { - return time.Duration(math.Pow(float64(2), float64(r.AttemptNumber-1))) * time.Second - } - return time.Duration(30) * time.Second - } - - return common.NewRetryPolicy(attempts, shouldRetry, nextDuration) -} - -func (w *workRequestService) Wait(opcWorkRequestID string) (workrequests.GetWorkRequestResponse, error) { - w.logger.Info("Waiting for the work request to finish. opcWorkRequestID = " + opcWorkRequestID) - - // retries until the work status is SUCCEEDED, FAILED or CANCELED - retryPolicy := w.getRetryPolicy() - - workRequest := workrequests.GetWorkRequestRequest{ - WorkRequestId: common.String(opcWorkRequestID), - RequestMetadata: common.RequestMetadata{ - RetryPolicy: &retryPolicy, - }, - } - - resp, err := w.workClient.GetWorkRequest(context.TODO(), workRequest) - if err != nil { - return resp, err - } - - return resp, nil -} - func (w *workRequestService) Get(opcWorkRequestID string) (workrequests.GetWorkRequestResponse, error) { workRequest := workrequests.GetWorkRequestRequest{ WorkRequestId: common.String(opcWorkRequestID), From 9ecaadcbe7ea8d10210ce1013a537e04b59af435 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Wed, 27 Apr 2022 19:17:17 -0400 Subject: [PATCH 285/628] fix staticcheck and adb restore --- PROJECT | 3 ++ .../v1alpha1/autonomousdatabase_webhook.go | 3 +- .../autonomousdatabaserestore_types.go | 2 +- apis/database/v1alpha1/webhook_suite_test.go | 2 - .../autonomouscontainerdatabase_controller.go | 2 +- .../database/autonomousdatabase_controller.go | 22 ++++++++- .../autonomousdatabasebackup_controller.go | 10 ++-- .../autonomousdatabaserestore_controller.go | 47 +++++-------------- go.mod | 3 -- go.sum | 3 -- 10 files changed, 43 insertions(+), 54 deletions(-) diff --git a/PROJECT b/PROJECT index 9150c705..67db6526 100644 --- a/PROJECT +++ b/PROJECT @@ -111,6 +111,9 @@ resources: kind: AutonomousContainerDatabase path: github.com/oracle/oracle-database-operator/apis/database/v1alpha1 version: v1alpha1 + webhooks: + validation: true + webhookVersion: v1 - api: crdVersion: v1beta1 namespaced: true diff --git a/apis/database/v1alpha1/autonomousdatabase_webhook.go b/apis/database/v1alpha1/autonomousdatabase_webhook.go index eb5d9709..1986769f 100644 --- a/apis/database/v1alpha1/autonomousdatabase_webhook.go +++ b/apis/database/v1alpha1/autonomousdatabase_webhook.go @@ -102,7 +102,6 @@ func (r *AutonomousDatabase) ValidateCreate() error { field.Forbidden(field.NewPath("spec").Child("details").Child("LifecycleState"), "cannot apply lifecycleState to a provision operation")) } - } else { // binding operation } if len(allErrs) == 0 { @@ -212,7 +211,7 @@ func validateNetworkAccess(adb *AutonomousDatabase, allErrs field.ErrorList) fie if adb.Spec.Details.NetworkAccess.IsAccessControlEnabled != nil { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("details").Child("networkAccess").Child("IsAccessControlEnabled"), - fmt.Sprintf("isAccessControlEnabled is not applicable on a shared Autonomous Database"))) + "isAccessControlEnabled is not applicable on a shared Autonomous Database")) } } else { // Dedicated database diff --git a/apis/database/v1alpha1/autonomousdatabaserestore_types.go b/apis/database/v1alpha1/autonomousdatabaserestore_types.go index c49556a6..6668685d 100644 --- a/apis/database/v1alpha1/autonomousdatabaserestore_types.go +++ b/apis/database/v1alpha1/autonomousdatabaserestore_types.go @@ -128,7 +128,7 @@ func init() { // GetPIT returns the spec.pointInTime.timeStamp in SDKTime format func (r *AutonomousDatabaseRestore) GetPIT() (*common.SDKTime, error) { if r.Spec.Source.PointInTime.Timestamp == nil { - return nil, errors.New("The timestamp is empty") + return nil, errors.New("the timestamp is empty") } return parseDisplayTime(*r.Spec.Source.PointInTime.Timestamp) } diff --git a/apis/database/v1alpha1/webhook_suite_test.go b/apis/database/v1alpha1/webhook_suite_test.go index dbbd02c9..b0cdde57 100644 --- a/apis/database/v1alpha1/webhook_suite_test.go +++ b/apis/database/v1alpha1/webhook_suite_test.go @@ -56,7 +56,6 @@ import ( //+kubebuilder:scaffold:imports k8sErrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/runtime" - "k8s.io/client-go/rest" ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/controller-runtime/pkg/envtest" @@ -92,7 +91,6 @@ var ( // These tests use Ginkgo (BDD-style Go testing framework). Refer to // http://onsi.github.io/ginkgo/ to learn more about Ginkgo. -var cfg *rest.Config var k8sClient client.Client var testEnv *envtest.Environment var ctx context.Context diff --git a/controllers/database/autonomouscontainerdatabase_controller.go b/controllers/database/autonomouscontainerdatabase_controller.go index a71267ca..4f4557ad 100644 --- a/controllers/database/autonomouscontainerdatabase_controller.go +++ b/controllers/database/autonomouscontainerdatabase_controller.go @@ -644,7 +644,7 @@ func (r *AutonomousContainerDatabaseReconciler) validateDesiredLifecycleState( acd.Status.LifecycleState = database.AutonomousContainerDatabaseLifecycleStateTerminating default: - return false, false, errors.New("Unknown lifecycleState") + return false, false, errors.New("unknown lifecycleState") } acd.Spec.Action = dbv1alpha1.AcdActionBlank diff --git a/controllers/database/autonomousdatabase_controller.go b/controllers/database/autonomousdatabase_controller.go index 186ac058..0a82e400 100644 --- a/controllers/database/autonomousdatabase_controller.go +++ b/controllers/database/autonomousdatabase_controller.go @@ -56,6 +56,7 @@ import ( apiErrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/runtime" "k8s.io/apimachinery/pkg/types" + "k8s.io/client-go/tools/record" ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" @@ -125,9 +126,26 @@ func (r *AutonomousDatabaseReconciler) watchPredicate() predicate.Predicate { CreateFunc: func(e event.CreateEvent) bool { _, backupOk := e.Object.(*dbv1alpha1.AutonomousDatabaseBackup) _, restoreOk := e.Object.(*dbv1alpha1.AutonomousDatabaseRestore) - // Don't enqueue if it's a create Backup event or a create Restore event + // Don't enqueue if the event is from Backup or Restore return !(backupOk || restoreOk) }, + UpdateFunc: func(e event.UpdateEvent) bool { + // Enqueue the update event only when the status changes the first time + desiredBackup, backupOk := e.ObjectNew.(*dbv1alpha1.AutonomousDatabaseBackup) + if backupOk { + oldBackup := e.ObjectOld.(*dbv1alpha1.AutonomousDatabaseBackup) + return oldBackup.Status.LifecycleState == "" && desiredBackup.Status.LifecycleState != "" + } + + desiredRestore, restoreOk := e.ObjectNew.(*dbv1alpha1.AutonomousDatabaseRestore) + if restoreOk { + oldRestore := e.ObjectOld.(*dbv1alpha1.AutonomousDatabaseRestore) + return oldRestore.Status.Status == "" && desiredRestore.Status.Status != "" + } + + // Enqueue if the event is not from Backup or Restore + return true + }, } } @@ -788,7 +806,7 @@ func (r *AutonomousDatabaseReconciler) validateDesiredLifecycleState( adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateTerminating default: - return false, errors.New("Unknown lifecycleState") + return false, errors.New("unknown lifecycleState") } return true, nil diff --git a/controllers/database/autonomousdatabasebackup_controller.go b/controllers/database/autonomousdatabasebackup_controller.go index cc233d5f..4632ffff 100644 --- a/controllers/database/autonomousdatabasebackup_controller.go +++ b/controllers/database/autonomousdatabasebackup_controller.go @@ -54,7 +54,6 @@ import ( "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" "sigs.k8s.io/controller-runtime/pkg/predicate" - databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" "github.com/oracle/oracle-database-operator/commons/adb_family" "github.com/oracle/oracle-database-operator/commons/oci" @@ -73,7 +72,7 @@ type AutonomousDatabaseBackupReconciler struct { // SetupWithManager sets up the controller with the Manager. func (r *AutonomousDatabaseBackupReconciler) SetupWithManager(mgr ctrl.Manager) error { return ctrl.NewControllerManagedBy(mgr). - For(&databasev1alpha1.AutonomousDatabaseBackup{}). + For(&dbv1alpha1.AutonomousDatabaseBackup{}). WithEventFilter(predicate.GenerationChangedPredicate{}). WithOptions(controller.Options{MaxConcurrentReconciles: 100}). // ReconcileHandler is never invoked concurrently with the same object. Complete(r) @@ -138,7 +137,7 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req * To get the latest status, execute before all the reconcile logic ******************************************************************/ if dbv1alpha1.IsBackupIntermediateState(backup.Status.LifecycleState) { - logger.WithName("validateLifecycleState").Info("Reconcile queued") + logger.WithName("IsIntermediateState").Info("Current lifecycleState is " + string(backup.Status.LifecycleState) + "; reconcile queued") return requeueResult, nil } @@ -148,13 +147,12 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req ******************************************************************/ if backup.Spec.AutonomousDatabaseBackupOCID == nil { // Create a new backup + logger.Info("Sending CreateAutonomousDatabaseBackup request to OCI") backupResp, err := r.dbService.CreateAutonomousDatabaseBackup(backup, adbOCID) if err != nil { return r.manageError(backup, err) } - logger.Info("AutonomousDatabaseBackup " + *backupResp.DisplayName + " started") - // After the creation, update the status first adbResp, err := r.dbService.GetAutonomousDatabase(*backupResp.AutonomousDatabaseId) if err != nil { @@ -178,7 +176,7 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req return emptyResult, nil } - logger.WithName("createAutonomousDatabaseBackup").Info("AutonomousDatabaseBackupOCID updated") + logger.Info("AutonomousDatabaseBackupOCID updated") return emptyResult, nil } diff --git a/controllers/database/autonomousdatabaserestore_controller.go b/controllers/database/autonomousdatabaserestore_controller.go index 876ac5dc..561d4884 100644 --- a/controllers/database/autonomousdatabaserestore_controller.go +++ b/controllers/database/autonomousdatabaserestore_controller.go @@ -54,7 +54,6 @@ import ( "sigs.k8s.io/controller-runtime/pkg/predicate" "github.com/oracle/oci-go-sdk/v63/common" - databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" "github.com/oracle/oracle-database-operator/commons/adb_family" "github.com/oracle/oracle-database-operator/commons/k8s" @@ -75,7 +74,7 @@ type AutonomousDatabaseRestoreReconciler struct { // SetupWithManager sets up the controller with the Manager. func (r *AutonomousDatabaseRestoreReconciler) SetupWithManager(mgr ctrl.Manager) error { return ctrl.NewControllerManagedBy(mgr). - For(&databasev1alpha1.AutonomousDatabaseRestore{}). + For(&dbv1alpha1.AutonomousDatabaseRestore{}). WithEventFilter(predicate.GenerationChangedPredicate{}). Complete(r) } @@ -116,14 +115,6 @@ func (r *AutonomousDatabaseRestoreReconciler) Reconcile(ctx context.Context, req return r.manageError(restore, err) } - /****************************************************************** - * Extract the restoreTime from the spec - ******************************************************************/ - restoreTime, err := r.getRestoreSDKTime(restore) - if err != nil { - return r.manageError(restore, err) - } - /****************************************************************** * Get OCI database client and work request client ******************************************************************/ @@ -134,44 +125,36 @@ func (r *AutonomousDatabaseRestoreReconciler) Reconcile(ctx context.Context, req logger.Info("OCI clients configured succesfully") /****************************************************************** - * Get status from OCI WorkRequest - ******************************************************************/ - if restore.Status.WorkRequestOCID != "" { - resp, err := r.workService.Get(restore.Status.WorkRequestOCID) + * Start the restore or update the status + ******************************************************************/ + if restore.Status.WorkRequestOCID == "" { + logger.Info("Start restoring the database") + // Extract the restoreTime from the spec + restoreTime, err := r.getRestoreSDKTime(restore) if err != nil { return r.manageError(restore, err) } - restore.Status.Status = resp.Status - - if dbv1alpha1.IsRestoreIntermediateState(resp.Status) { - logger.WithName("validateStatus").Info("Reconcile queued") - return requeueResult, nil - } - } - - /****************************************************************** - * Start the restore or update the status - ******************************************************************/ - if restore.Status.WorkRequestOCID == "" { - // Start restore + logger.Info("Sending RestoreAutonomousDatabase request to OCI") adbResp, err := r.dbService.RestoreAutonomousDatabase(adbOCID, *restoreTime) if err != nil { return r.manageError(restore, err) } + // Update the restore status workResp, err := r.workService.Get(*adbResp.OpcWorkRequestId) if err != nil { return r.manageError(restore, err) } restore.UpdateStatus(adbResp.AutonomousDatabase, workResp) - if err := r.KubeClient.Update(context.TODO(), restore); err != nil { + if err := r.KubeClient.Status().Update(context.TODO(), restore); err != nil { return r.manageError(restore, err) } } else { // Update the status + logger.Info("Update the status of the restore session") adbResp, err := r.dbService.GetAutonomousDatabase(adbOCID) if err != nil { return r.manageError(restore, err) @@ -183,18 +166,14 @@ func (r *AutonomousDatabaseRestoreReconciler) Reconcile(ctx context.Context, req } restore.UpdateStatus(adbResp.AutonomousDatabase, workResp) - if err := r.KubeClient.Update(context.TODO(), restore); err != nil { - return r.manageError(restore, err) - } - - if err := r.KubeClient.Update(context.TODO(), restore); err != nil { + if err := r.KubeClient.Status().Update(context.TODO(), restore); err != nil { return r.manageError(restore, err) } } // Requeue if it's in intermediate state if dbv1alpha1.IsRestoreIntermediateState(restore.Status.Status) { - logger.WithName("validateStatus").Info("Reconcile queued") + logger.WithName("IsIntermediateState").Info("Current status is " + string(restore.Status.Status) + "; reconcile queued") return requeueResult, nil } diff --git a/go.mod b/go.mod index d5974bfa..f779db45 100644 --- a/go.mod +++ b/go.mod @@ -37,8 +37,6 @@ require ( github.com/moby/spdystream v0.2.0 // indirect github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect github.com/modern-go/reflect2 v1.0.2 // indirect - github.com/nxadm/tail v1.4.8 // indirect - github.com/onsi/ginkgo v1.16.5 // indirect github.com/pkg/errors v0.9.1 // indirect github.com/prometheus/client_golang v1.12.1 // indirect github.com/prometheus/client_model v0.2.0 // indirect @@ -59,7 +57,6 @@ require ( google.golang.org/appengine v1.6.7 // indirect google.golang.org/protobuf v1.27.1 // indirect gopkg.in/inf.v0 v0.9.1 // indirect - gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 // indirect gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b // indirect k8s.io/apiextensions-apiserver v0.23.3 // indirect k8s.io/component-base v0.23.3 // indirect diff --git a/go.sum b/go.sum index d2aeef54..8a2308c2 100644 --- a/go.sum +++ b/go.sum @@ -395,8 +395,6 @@ github.com/onsi/gomega v1.17.0/go.mod h1:HnhC7FXeEQY45zxNK3PPoIUhzk/80Xly9PcubAl github.com/onsi/gomega v1.19.0 h1:4ieX6qQjPP/BfC3mpsAtIGGlxTWPeA3Inl/7DtXw1tw= github.com/onsi/gomega v1.19.0/go.mod h1:LY+I3pBVzYsTBU1AnDwOSxaYi9WoWiqgwooUqq9yPro= github.com/opentracing/opentracing-go v1.1.0/go.mod h1:UkNAQd3GIcIGf0SeVgPpRdFStlNbqXla1AfSYxPUl2o= -github.com/oracle/oci-go-sdk/v51 v51.0.0 h1:eDUVMsAzvf+jfq4xbtpQrxznYNjccwKKyIhmRvZLgwI= -github.com/oracle/oci-go-sdk/v51 v51.0.0/go.mod h1:d9KSNXwE64drofxoor+y/JWofJqLqRF9D1/AtfYIE10= github.com/oracle/oci-go-sdk/v63 v63.0.0 h1:OOGCUmaDzrd5zTG8pljcnkR1ZHxg/991uEiQJi95/4E= github.com/oracle/oci-go-sdk/v63 v63.0.0/go.mod h1:n6V9PcyRW5wtHaNd2TltbV3sWvbNy3PNqLmLrcT23Fg= github.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc= @@ -456,7 +454,6 @@ github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1 github.com/smartystreets/goconvey v1.6.4/go.mod h1:syvi0/a8iFYH4r/RixwvyeAJjdLS9QV7WQ/tjFTllLA= github.com/soheilhy/cmux v0.1.4/go.mod h1:IM3LyeVVIOuxMH7sFAkER9+bJ4dT7Ms6E4xg4kGIyLM= github.com/soheilhy/cmux v0.1.5/go.mod h1:T7TcVDs9LWfQgPlPsdngu6I6QIoyIFZDDC6sNE1GqG0= -github.com/sony/gobreaker v0.4.2-0.20210216022020-dd874f9dd33b/go.mod h1:ZKptC7FHNvhBz7dN2LGjPVBz2sZJmc0/PkyDJOjmxWY= github.com/sony/gobreaker v0.5.0 h1:dRCvqm0P490vZPmy7ppEk2qCnCieBooFJ+YoXGYB+yg= github.com/sony/gobreaker v0.5.0/go.mod h1:ZKptC7FHNvhBz7dN2LGjPVBz2sZJmc0/PkyDJOjmxWY= github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA= From c688f9b9294980361df9efef7f9a883672e1e689 Mon Sep 17 00:00:00 2001 From: omar_salazar Date: Thu, 28 Apr 2022 02:23:27 +0000 Subject: [PATCH 286/628] Add backup/restore tests --- Makefile | 2 +- .../autonomousdatabasebackup_types.go | 2 - .../autonomousdatabaserestore_types.go | 10 -- ...autonomousdatabase_controller_bind_test.go | 2 +- ...tonomousdatabase_controller_create_test.go | 78 ++++++++++ test/e2e/backup.sql | 20 +++ test/e2e/behavior/shared_behaviors.go | 141 +++++++++++++----- test/e2e/resource/test_config.yaml | 10 +- test/e2e/suite_test.go | 15 ++ test/e2e/util/oci_db_request.go | 33 +++- test/e2e/util/util.go | 3 + 11 files changed, 260 insertions(+), 56 deletions(-) create mode 100644 test/e2e/backup.sql diff --git a/Makefile b/Makefile index 15de15a8..a29214db 100644 --- a/Makefile +++ b/Makefile @@ -59,7 +59,7 @@ test: manifests generate fmt vet envtest ## Run unit tests. E2ETEST ?= ./test/e2e/ e2e: manifests generate fmt vet envtest ## Run e2e tests. - KUBEBUILDER_ASSETS="$(shell $(ENVTEST) use $(ENVTEST_K8S_VERSION) -p path)" go test $(E2ETEST) -v -timeout 50m -ginkgo.v -ginkgo.failFast + KUBEBUILDER_ASSETS="$(shell $(ENVTEST) use $(ENVTEST_K8S_VERSION) -p path)" go test $(E2ETEST) -v -timeout 80m -ginkgo.v -ginkgo.failFast ##@ Build diff --git a/apis/database/v1alpha1/autonomousdatabasebackup_types.go b/apis/database/v1alpha1/autonomousdatabasebackup_types.go index e6cec9d3..586a27a3 100644 --- a/apis/database/v1alpha1/autonomousdatabasebackup_types.go +++ b/apis/database/v1alpha1/autonomousdatabasebackup_types.go @@ -59,8 +59,6 @@ type AutonomousDatabaseBackupSpec struct { OCIConfig OCIConfigSpec `json:"ociConfig,omitempty"` } -type BackupStateEnum string - // AutonomousDatabaseBackupStatus defines the observed state of AutonomousDatabaseBackup type AutonomousDatabaseBackupStatus struct { LifecycleState database.AutonomousDatabaseBackupLifecycleStateEnum `json:"lifecycleState"` diff --git a/apis/database/v1alpha1/autonomousdatabaserestore_types.go b/apis/database/v1alpha1/autonomousdatabaserestore_types.go index 6668685d..3a22fb35 100644 --- a/apis/database/v1alpha1/autonomousdatabaserestore_types.go +++ b/apis/database/v1alpha1/autonomousdatabaserestore_types.go @@ -73,16 +73,6 @@ type AutonomousDatabaseRestoreSpec struct { OCIConfig OCIConfigSpec `json:"ociConfig,omitempty"` } -type RestoreStatusEnum string - -const ( - RestoreStatusError RestoreStatusEnum = "ERROR" - RestoreStatusAccepted RestoreStatusEnum = "ACCEPTED" - RestoreStatusInProgress RestoreStatusEnum = "IN_PROGRESS" - RestoreStatusFailed RestoreStatusEnum = "FAILED" - RestoreStatusSucceeded RestoreStatusEnum = "SUCCEEDED" -) - // AutonomousDatabaseRestoreStatus defines the observed state of AutonomousDatabaseRestore type AutonomousDatabaseRestoreStatus struct { // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster diff --git a/test/e2e/autonomousdatabase_controller_bind_test.go b/test/e2e/autonomousdatabase_controller_bind_test.go index 6be0c4e5..e8185e77 100644 --- a/test/e2e/autonomousdatabase_controller_bind_test.go +++ b/test/e2e/autonomousdatabase_controller_bind_test.go @@ -196,7 +196,7 @@ var _ = Describe("test ADB binding with hardLink=true", func() { Describe("bind to a terminated adb", func() { //Wait until remote state is terminated - It("Should check that OCI adb state is terminated", e2ebehavior.AssertRemoteStateOCID(&k8sClient, &dbClient, &terminatedAdbID, database.AutonomousDatabaseLifecycleStateTerminated)) + It("Should check that OCI adb state is terminated", e2ebehavior.AssertRemoteStateOCID(&k8sClient, &dbClient, &terminatedAdbID, database.AutonomousDatabaseLifecycleStateTerminated, time.Second*300)) It("Should create a AutonomousDatabase resource", func() { adb := &dbv1alpha1.AutonomousDatabase{ diff --git a/test/e2e/autonomousdatabase_controller_create_test.go b/test/e2e/autonomousdatabase_controller_create_test.go index feba24d3..82e11462 100644 --- a/test/e2e/autonomousdatabase_controller_create_test.go +++ b/test/e2e/autonomousdatabase_controller_create_test.go @@ -43,6 +43,7 @@ import ( "time" "github.com/oracle/oci-go-sdk/v63/common" + "github.com/oracle/oci-go-sdk/v63/database" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/types" @@ -68,6 +69,8 @@ var _ = Describe("test ADB provisioning", func() { const downloadedWallet = "instance-wallet-secret-1" const resourceName = "createadb1" + const backupName = "adb-backup" + const restoreName = "adb-restore" duplicateAdbResourceName := "duplicateadb" var adbLookupKey = types.NamespacedName{Name: resourceName, Namespace: ADBNamespace} @@ -170,6 +173,81 @@ var _ = Describe("test ADB provisioning", func() { Expect(k8sClient.Delete(context.TODO(), duplicateAdb)).To(Succeed()) }) + It("Should create an Autonomous Database Backup", func() { + e2ebehavior.AssertState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateAvailable)() + + // Get adb ocid + adb := &dbv1alpha1.AutonomousDatabase{} + Expect(k8sClient.Get(context.TODO(), adbLookupKey, adb)).To(Succeed()) + databaseOCID := adb.Spec.Details.AutonomousDatabaseOCID + tnsEntry := dbName + "_high" + err := e2ebehavior.ConfigureADBBackup(&dbClient, databaseOCID, &tnsEntry, &SharedPlainTextAdminPassword, &SharedPlainTextWalletPassword, &SharedBucketUrl, &SharedAuthToken, &SharedOciUser) + Expect(err).ShouldNot(HaveOccurred()) + + adbBackup := &dbv1alpha1.AutonomousDatabaseBackup{ + TypeMeta: metav1.TypeMeta{ + APIVersion: "database.oracle.com/v1alpha1", + Kind: "AutonomousDatabaseBackup", + }, + ObjectMeta: metav1.ObjectMeta{ + Name: backupName, + Namespace: ADBNamespace, + }, + Spec: dbv1alpha1.AutonomousDatabaseBackupSpec{ + Target: dbv1alpha1.TargetSpec{ + OCIADB: dbv1alpha1.OCIADBSpec{ + OCID: common.String(*databaseOCID), + }, + }, + DisplayName: common.String(backupName), + OCIConfig: dbv1alpha1.OCIConfigSpec{ + ConfigMapName: common.String(SharedOCIConfigMapName), + SecretName: common.String(SharedOCISecretName), + }, + }, + } + + Expect(k8sClient.Create(context.TODO(), adbBackup)).To(Succeed()) + + backupLookupKey := types.NamespacedName{Name: backupName, Namespace: ADBNamespace} + e2ebehavior.AssertBackupRestore(&k8sClient, &dbClient, &backupLookupKey, &adbLookupKey, database.AutonomousDatabaseLifecycleStateBackupInProgress)() + }) + + It("Should restore a database", func() { + e2ebehavior.AssertState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateAvailable)() + + adbRestore := &dbv1alpha1.AutonomousDatabaseRestore{ + TypeMeta: metav1.TypeMeta{ + APIVersion: "database.oracle.com/v1alpha1", + Kind: "AutonomousDatabaseRestore", + }, + ObjectMeta: metav1.ObjectMeta{ + Name: restoreName, + Namespace: ADBNamespace, + }, + Spec: dbv1alpha1.AutonomousDatabaseRestoreSpec{ + Target: dbv1alpha1.TargetSpec{ + K8sADB: dbv1alpha1.K8sADBSpec{ + Name: common.String(resourceName), + }, + }, + Source: dbv1alpha1.SourceSpec{ + K8sADBBackup: dbv1alpha1.K8sADBBackupSpec{ + Name: common.String(backupName), + }, + }, + OCIConfig: dbv1alpha1.OCIConfigSpec{ + ConfigMapName: common.String(SharedOCIConfigMapName), + SecretName: common.String(SharedOCISecretName), + }, + }, + } + + Expect(k8sClient.Create(context.TODO(), adbRestore)).To(Succeed()) + restoreLookupKey := types.NamespacedName{Name: restoreName, Namespace: ADBNamespace} + e2ebehavior.AssertBackupRestore(&k8sClient, &dbClient, &restoreLookupKey, &adbLookupKey, database.AutonomousDatabaseLifecycleStateRestoreInProgress)() + }) + It("Should download an instance wallet using the password from K8s Secret "+SharedWalletPassSecretName, e2ebehavior.AssertWallet(&k8sClient, &adbLookupKey)) It("Should delete the resource in cluster and terminate the database in OCI", e2ebehavior.AssertHardLinkDelete(&k8sClient, &dbClient, &adbLookupKey)) diff --git a/test/e2e/backup.sql b/test/e2e/backup.sql new file mode 100644 index 00000000..3663e6ff --- /dev/null +++ b/test/e2e/backup.sql @@ -0,0 +1,20 @@ +set cloudconfig -proxy=&1 &2 +connect ADMIN/&3@&4 +ALTER DATABASE PROPERTY SET default_backup_bucket='&5'; + +BEGIN +DBMS_CLOUD.DROP_CREDENTIAL( credential_name => 'DEF_CRED_NAME' ); +END; +/ + +BEGIN + DBMS_CLOUD.CREATE_CREDENTIAL( + credential_name => 'DEF_CRED_NAME', + username => '&6', + password => '&7' +); +END; +/ + +ALTER DATABASE PROPERTY SET DEFAULT_CREDENTIAL = 'ADMIN.DEF_CRED_NAME'; +exit \ No newline at end of file diff --git a/test/e2e/behavior/shared_behaviors.go b/test/e2e/behavior/shared_behaviors.go index 7b4616c3..8a29c504 100644 --- a/test/e2e/behavior/shared_behaviors.go +++ b/test/e2e/behavior/shared_behaviors.go @@ -49,6 +49,7 @@ import ( "github.com/onsi/gomega" "github.com/oracle/oci-go-sdk/v63/common" "github.com/oracle/oci-go-sdk/v63/database" + "github.com/oracle/oci-go-sdk/v63/workrequests" corev1 "k8s.io/api/core/v1" k8sErrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/types" @@ -56,6 +57,9 @@ import ( dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" "github.com/oracle/oracle-database-operator/test/e2e/util" + "os" + "os/exec" + "strings" ) /************************************************************** @@ -66,24 +70,29 @@ import ( **************************************************************/ var ( - Describe = ginkgo.Describe - By = ginkgo.By - GinkgoWriter = ginkgo.GinkgoWriter - Expect = gomega.Expect - BeNil = gomega.BeNil - Eventually = gomega.Eventually - Equal = gomega.Equal - Succeed = gomega.Succeed - BeNumerically = gomega.BeNumerically - BeTrue = gomega.BeTrue + Describe = ginkgo.Describe + By = ginkgo.By + GinkgoWriter = ginkgo.GinkgoWriter + Expect = gomega.Expect + BeNil = gomega.BeNil + Eventually = gomega.Eventually + Equal = gomega.Equal + Succeed = gomega.Succeed + BeNumerically = gomega.BeNumerically + BeTrue = gomega.BeTrue + changeTimeout = time.Second * 300 + provisionTimeout = time.Second * 15 + bindTimeout = time.Second * 30 + backupTimeout = time.Minute * 20 + intervalTime = time.Second * 10 + updateTimeout = time.Minute * 7 + changeLocalStateTimeout = time.Second * 600 ) func AssertProvision(k8sClient *client.Client, adbLookupKey *types.NamespacedName) func() { return func() { - // Set the timeout to 15 minutes. The provision operation might take up to 10 minutes + // Set provisionTimeout to 15 minutes. The provision operation might take up to 10 minutes // if we have already send too many requests to OCI. - provisionTimeout := time.Minute * 15 - provisionInterval := time.Second * 10 Expect(k8sClient).NotTo(BeNil()) Expect(adbLookupKey).NotTo(BeNil()) @@ -100,7 +109,7 @@ func AssertProvision(k8sClient *client.Client, adbLookupKey *types.NamespacedNam } return createdADB.Spec.Details.AutonomousDatabaseOCID, nil - }, provisionTimeout, provisionInterval).ShouldNot(BeNil()) + }, provisionTimeout, intervalTime).ShouldNot(BeNil()) fmt.Fprintf(GinkgoWriter, "AutonomousDatabase DbName = %s, and AutonomousDatabaseOCID = %s\n", *createdADB.Spec.Details.DbName, *createdADB.Spec.Details.AutonomousDatabaseOCID) @@ -109,8 +118,6 @@ func AssertProvision(k8sClient *client.Client, adbLookupKey *types.NamespacedNam func AssertBind(k8sClient *client.Client, adbLookupKey *types.NamespacedName) func() { return func() { - bindTimeout := time.Second * 30 - Expect(k8sClient).NotTo(BeNil()) Expect(adbLookupKey).NotTo(BeNil()) @@ -223,8 +230,6 @@ func UpdateDetails(k8sClient *client.Client, dbClient *database.DatabaseClient, return func() *dbv1alpha1.AutonomousDatabase { // Considering that there are at most two update requests will be sent during the update // From the observation per request takes ~3mins to finish - updateTimeout := time.Minute * 7 - updateInterval := time.Second * 20 Expect(k8sClient).NotTo(BeNil()) Expect(dbClient).NotTo(BeNil()) @@ -251,7 +256,7 @@ func UpdateDetails(k8sClient *client.Client, dbClient *database.DatabaseClient, } return database.AutonomousDatabaseLifecycleStateEnum(listResp.Items[0].LifecycleState), nil - }, updateTimeout, updateInterval).Should(Equal(database.AutonomousDatabaseLifecycleStateAvailable)) + }, updateTimeout, intervalTime).Should(Equal(database.AutonomousDatabaseLifecycleStateAvailable)) // Update var newDisplayName = *expectedADB.Spec.Details.DisplayName + "_new" @@ -284,8 +289,6 @@ func AssertADBDetails(k8sClient *client.Client, dbClient *database.DatabaseClien return func() { // Considering that there are at most two update requests will be sent during the update // From the observation per request takes ~3mins to finish - updateTimeout := time.Minute * 7 - updateInterval := time.Second * 20 Expect(k8sClient).NotTo(BeNil()) Expect(dbClient).NotTo(BeNil()) @@ -380,7 +383,7 @@ func AssertADBDetails(k8sClient *client.Client, dbClient *database.DatabaseClien compareString(expectedADBDetails.NetworkAccess.PrivateEndpoint.HostnamePrefix, resp.AutonomousDatabase.PrivateEndpointLabel) return same, nil - }, updateTimeout, updateInterval).Should(BeTrue()) + }, updateTimeout, intervalTime).Should(BeTrue()) // IMPORTANT: make sure the local resource has finished reconciling, otherwise the changes will // be conflicted with the next test and cause unknow result. @@ -483,8 +486,6 @@ func AssertState(k8sClient *client.Client, dbClient *database.DatabaseClient, ad // AssertHardLinkDelete asserts the database is terminated in OCI when hardLink is set to true func AssertHardLinkDelete(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName) func() { return func() { - changeStateTimeout := time.Second * 300 - Expect(k8sClient).NotTo(BeNil()) Expect(dbClient).NotTo(BeNil()) Expect(adbLookupKey).NotTo(BeNil()) @@ -501,7 +502,7 @@ func AssertHardLinkDelete(k8sClient *client.Client, dbClient *database.DatabaseC Eventually(func() (database.AutonomousDatabaseLifecycleStateEnum, error) { retryPolicy := e2eutil.NewLifecycleStateRetryPolicy(database.AutonomousDatabaseLifecycleStateTerminating) return returnRemoteState(derefK8sClient, derefDBClient, adb.Spec.Details.AutonomousDatabaseOCID, &retryPolicy) - }, changeStateTimeout).Should(Equal(database.AutonomousDatabaseLifecycleStateTerminating)) + }, changeTimeout).Should(Equal(database.AutonomousDatabaseLifecycleStateTerminating)) AssertSoftLinkDelete(k8sClient, adbLookupKey)() } @@ -510,9 +511,6 @@ func AssertHardLinkDelete(k8sClient *client.Client, dbClient *database.DatabaseC // AssertSoftLinkDelete asserts the database remains in OCI when hardLink is set to false func AssertSoftLinkDelete(k8sClient *client.Client, adbLookupKey *types.NamespacedName) func() { return func() { - changeStateTimeout := time.Second * 300 - changeStateInterval := time.Second * 10 - Expect(k8sClient).NotTo(BeNil()) Expect(adbLookupKey).NotTo(BeNil()) @@ -532,15 +530,13 @@ func AssertSoftLinkDelete(k8sClient *client.Client, adbLookupKey *types.Namespac return } return - }, changeStateTimeout, changeStateInterval).Should(Equal(true)) + }, changeTimeout, intervalTime).Should(Equal(true)) } } // AssertLocalState asserts the lifecycle state of the local resource using adbLookupKey func AssertLocalState(k8sClient *client.Client, adbLookupKey *types.NamespacedName, state database.AutonomousDatabaseLifecycleStateEnum) func() { return func() { - changeLocalStateTimeout := time.Second * 600 - Expect(k8sClient).NotTo(BeNil()) Expect(adbLookupKey).NotTo(BeNil()) @@ -553,10 +549,8 @@ func AssertLocalState(k8sClient *client.Client, adbLookupKey *types.NamespacedNa } } -// AssertRemoteState asserts the lifecycle state in OCI using adbLookupKey func AssertRemoteState(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName, state database.AutonomousDatabaseLifecycleStateEnum) func() { return func() { - Expect(k8sClient).NotTo(BeNil()) Expect(dbClient).NotTo(BeNil()) Expect(adbLookupKey).NotTo(BeNil()) @@ -566,16 +560,28 @@ func AssertRemoteState(k8sClient *client.Client, dbClient *database.DatabaseClie adb := &dbv1alpha1.AutonomousDatabase{} Expect(derefK8sClient.Get(context.TODO(), *adbLookupKey, adb)).To(Succeed()) By("Checking if the lifecycleState of remote resource is " + string(state)) - AssertRemoteStateOCID(k8sClient, dbClient, adb.Spec.Details.AutonomousDatabaseOCID, state)() + AssertRemoteStateOCID(k8sClient, dbClient, adb.Spec.Details.AutonomousDatabaseOCID, state, changeTimeout)() } } -// AssertRemoteStateOCID asserts the lifecycle state in OCI using autonomousDatabaseOCID -func AssertRemoteStateOCID(k8sClient *client.Client, dbClient *database.DatabaseClient, adbID *string, state database.AutonomousDatabaseLifecycleStateEnum) func() { +// Backup takes ~15 minutes to complete, this function waits 20 minutes until ADB state is AVAILABLE +func AssertRemoteStateForBackupRestore(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName, state database.AutonomousDatabaseLifecycleStateEnum) func() { return func() { - changeRemoteStateTimeout := time.Second * 300 - changeRemoteStateInterval := time.Second * 10 + Expect(k8sClient).NotTo(BeNil()) + Expect(dbClient).NotTo(BeNil()) + Expect(adbLookupKey).NotTo(BeNil()) + + derefK8sClient := *k8sClient + adb := &dbv1alpha1.AutonomousDatabase{} + Expect(derefK8sClient.Get(context.TODO(), *adbLookupKey, adb)).To(Succeed()) + By("Checking if the lifecycleState of remote resource is " + string(state)) + AssertRemoteStateOCID(k8sClient, dbClient, adb.Spec.Details.AutonomousDatabaseOCID, state, backupTimeout)() + } +} + +func AssertRemoteStateOCID(k8sClient *client.Client, dbClient *database.DatabaseClient, adbID *string, state database.AutonomousDatabaseLifecycleStateEnum, timeout time.Duration) func() { + return func() { Expect(k8sClient).NotTo(BeNil()) Expect(dbClient).NotTo(BeNil()) Expect(adbID).NotTo(BeNil()) @@ -588,7 +594,7 @@ func AssertRemoteStateOCID(k8sClient *client.Client, dbClient *database.Database By("Checking if the lifecycleState of the ADB in OCI is " + string(state)) Eventually(func() (database.AutonomousDatabaseLifecycleStateEnum, error) { return returnRemoteState(derefK8sClient, derefDBClient, adbID, nil) - }, changeRemoteStateTimeout, changeRemoteStateInterval).Should(Equal(state)) + }, timeout, intervalTime).Should(Equal(state)) } } @@ -625,3 +631,60 @@ func returnRemoteState(k8sClient client.Client, dbClient database.DatabaseClient } return resp.LifecycleState, nil } + +/* Runs a script that connects to an ADB and configures the backup bucket */ +func ConfigureADBBackup(dbClient *database.DatabaseClient, databaseOCID *string, tnsEntry *string, adminPassword *string, walletPassword *string, bucket *string, authToken *string, ociUser *string) error { + + By("Downloading wallet zip") + walletZip, err := e2eutil.DownloadWalletZip(*dbClient, databaseOCID, walletPassword) + if err != nil { + fmt.Fprint(GinkgoWriter, err) + panic(err) + } + fmt.Fprint(GinkgoWriter, walletZip+" successfully downloaded.\n") + + By("Installing SQLcl") + cmd := exec.Command("wget", "https://download.oracle.com/otn_software/java/sqldeveloper/sqlcl-latest.zip") + stdout, err := cmd.Output() + cmd = exec.Command("unzip", "sqlcl-latest.zip") + stdout, err = cmd.Output() + + proxy := os.Getenv("HTTP_PROXY") + + By("Configuring adb backup bucket") + cmd = exec.Command("./sqlcl/bin/sql", "/nolog", "@backup.sql", proxy, walletZip, *adminPassword, strings.ToLower(*tnsEntry), *bucket, *ociUser, *authToken) + stdout, err = cmd.Output() + + fmt.Fprint(GinkgoWriter, string(stdout)) + + return err +} + +func AssertBackupRestore(k8sClient *client.Client, dbClient *database.DatabaseClient, backupRestoreLookupKey *types.NamespacedName, adbLookupKey *types.NamespacedName, state database.AutonomousDatabaseLifecycleStateEnum) func() { + return func() { + // After creating a backup, ADB status will change to BACKUP IN PROGRESS + // for ~7 minutes. After that time, the state should return to AVAILBLE + derefK8sClient := *k8sClient + + AssertRemoteState(k8sClient, dbClient, adbLookupKey, state)() + + By("Wait until ADB state returns to AVAILABLE") + AssertRemoteStateForBackupRestore(k8sClient, dbClient, adbLookupKey, database.AutonomousDatabaseLifecycleStateAvailable)() + + if state == database.AutonomousDatabaseLifecycleStateBackupInProgress { + By("Checking adb backup State is ACTIVE") + createdBackup := &dbv1alpha1.AutonomousDatabaseBackup{} + Eventually(func() (database.AutonomousDatabaseBackupLifecycleStateEnum, error) { + derefK8sClient.Get(context.TODO(), *backupRestoreLookupKey, createdBackup) + return createdBackup.Status.LifecycleState, nil + }, backupTimeout, time.Second*20).Should(Equal(database.AutonomousDatabaseBackupLifecycleStateActive)) + } else { + By("Checking adb restore State is SUCCEEDED") + createdRestore := &dbv1alpha1.AutonomousDatabaseRestore{} + Eventually(func() (workrequests.WorkRequestStatusEnum, error) { + derefK8sClient.Get(context.TODO(), *backupRestoreLookupKey, createdRestore) + return createdRestore.Status.Status, nil + }, backupTimeout, time.Second*20).Should(Equal(workrequests.WorkRequestStatusSucceeded)) + } + } +} diff --git a/test/e2e/resource/test_config.yaml b/test/e2e/resource/test_config.yaml index 1a4c6faf..7191045c 100644 --- a/test/e2e/resource/test_config.yaml +++ b/test/e2e/resource/test_config.yaml @@ -9,7 +9,7 @@ ociConfigFile: ~/.oci/config profile: DEFAULT # Compartment OCID where the database creates -compartmentOCID: ocid1.compartment.. +compartmentOCID: ocid1.compartment... # The OCID of the OCI Vault Secret that holds the password of the ADMIN account (should start with ocid1.vaultsecret...) adminPasswordOCID: ocid1.vaultsecret... # The OCID of the OCI Vault Secret that holds the password of the wallet (should start with ocid1.vaultsecret...) @@ -17,4 +17,10 @@ instanceWalletPasswordOCID: ocid1.vaultsecret... # The OCID of the subnet used to test the network access settings subnetOCID: ocid1.subnet... # The OCID of the network security group used to test the network access settings -nsgOCID: ocid1.networksecuritygroup... \ No newline at end of file +nsgOCID: ocid1.networksecuritygroup... +# The URL of the bucket used for configure ADB manual backup +bucketURL: https://swiftobjectstorage.region.oraclecloud.com/v1/namespace-string/bucket_name +# The auth token generated in OCI Console > Profile > User Settings > Auth Token +authToken: token +# The OCI user used to login to OCI Console +ociUser: user \ No newline at end of file diff --git a/test/e2e/suite_test.go b/test/e2e/suite_test.go index f173ce1e..9e0a37f6 100644 --- a/test/e2e/suite_test.go +++ b/test/e2e/suite_test.go @@ -41,6 +41,7 @@ package e2etest import ( "context" "fmt" + "os" "path/filepath" "testing" "time" @@ -115,6 +116,10 @@ var SharedInstanceWalletPasswordOCID string var SharedSubnetOCID string var SharedNsgOCID string +var SharedBucketUrl string +var SharedAuthToken string +var SharedOciUser string + const SharedAdminPassSecretName string = "adb-admin-password" const SharedWalletPassSecretName = "adb-wallet-password" @@ -211,6 +216,9 @@ var _ = BeforeSuite(func() { SharedInstanceWalletPasswordOCID = testConfig.InstanceWalletPasswordOCID SharedSubnetOCID = testConfig.SubnetOCID SharedNsgOCID = testConfig.NsgOCID + SharedBucketUrl = testConfig.BucketURL + SharedAuthToken = testConfig.AuthToken + SharedOciUser = testConfig.OciUser By("checking if the required parameters exist") Expect(testConfig.OCIConfigFile).ToNot(Equal("")) @@ -219,6 +227,9 @@ var _ = BeforeSuite(func() { Expect(testConfig.InstanceWalletPasswordOCID).ToNot(Equal("")) Expect(testConfig.SubnetOCID).ToNot(Equal("")) Expect(testConfig.NsgOCID).ToNot(Equal("")) + Expect(testConfig.BucketURL).ToNot(Equal("")) + Expect(testConfig.AuthToken).ToNot(Equal("")) + Expect(testConfig.OciUser).ToNot(Equal("")) By("getting OCI provider") ociConfigUtil, err := e2eutil.GetOCIConfigUtil(testConfig.OCIConfigFile, testConfig.Profile) @@ -288,4 +299,8 @@ var _ = AfterSuite(func() { Expect(e2eutil.DeleteAutonomousDatabase(dbClient, adb.Spec.Details.AutonomousDatabaseOCID)).Should(Succeed()) } } + + // Delete sqlcl-latest.zip and sqlcl folder if exists + os.Remove("sqlcl-latest.zip") + os.RemoveAll("sqlcl") }) diff --git a/test/e2e/util/oci_db_request.go b/test/e2e/util/oci_db_request.go index 3d9b09cf..d4ed212a 100644 --- a/test/e2e/util/oci_db_request.go +++ b/test/e2e/util/oci_db_request.go @@ -43,7 +43,8 @@ import ( "github.com/oracle/oci-go-sdk/v63/common" "github.com/oracle/oci-go-sdk/v63/database" - + "io" + "io/ioutil" "time" ) @@ -140,3 +141,33 @@ func NewLifecycleStateRetryPolicy(lifecycleState database.AutonomousDatabaseLife } return generateRetryPolicy(shouldRetry) } + +func DownloadWalletZip(dbClient database.DatabaseClient, databaseOCID *string, walletPassword *string) (string, error) { + + req := database.GenerateAutonomousDatabaseWalletRequest{ + AutonomousDatabaseId: common.String(*databaseOCID), + GenerateAutonomousDatabaseWalletDetails: database.GenerateAutonomousDatabaseWalletDetails{ + Password: common.String(*walletPassword), + }, + } + + resp, err := dbClient.GenerateAutonomousDatabaseWallet(context.TODO(), req) + if err != nil { + return "", err + } + + // Create a temp file wallet*.zip + const walletFileName = "wallet*.zip" + outZip, err := ioutil.TempFile("", walletFileName) + if err != nil { + return "", err + } + defer outZip.Close() + + // Save the wallet in wallet*.zip + if _, err := io.Copy(outZip, resp.Content); err != nil { + return "", err + } + + return outZip.Name(), nil +} diff --git a/test/e2e/util/util.go b/test/e2e/util/util.go index 7e5097ce..b1791df6 100644 --- a/test/e2e/util/util.go +++ b/test/e2e/util/util.go @@ -121,6 +121,9 @@ type testConfiguration struct { InstanceWalletPasswordOCID string `yaml:"instanceWalletPasswordOCID"` SubnetOCID string `yaml:"subnetOCID"` NsgOCID string `yaml:"nsgOCID"` + BucketURL string `yaml:"bucketURL"` + AuthToken string `yaml:"authToken"` + OciUser string `yaml:"ociUser"` } func GetTestConfig(filename string) (*testConfiguration, error) { From 3404321de3fc6e3b5603bf70e05448440e177f7d Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Thu, 28 Apr 2022 11:19:47 +0530 Subject: [PATCH 287/628] Fix config samples --- commons/database/constants.go | 4 +- .../samples/sidb/oraclerestdataservice.yaml | 16 ++--- .../sidb/oraclerestdataservice_create.yaml | 22 ++++--- .../samples/sidb/singleinstancedatabase.yaml | 8 +-- .../sidb/singleinstancedatabase_clone.yaml | 4 +- .../sidb/singleinstancedatabase_create.yaml | 6 +- .../sidb/singleinstancedatabase_minikube.yaml | 61 ------------------- .../sidb/singleinstancedatabase_patch.yaml | 4 +- .../singleinstancedatabase_prebuiltdb.yaml | 11 ++-- .../oraclerestdataservice_controller.go | 6 +- 10 files changed, 38 insertions(+), 104 deletions(-) delete mode 100644 config/samples/sidb/singleinstancedatabase_minikube.yaml diff --git a/commons/database/constants.go b/commons/database/constants.go index c2be0fac..5e915ca9 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -412,10 +412,10 @@ const InstallApexInContainer string = "cd ${ORDS_HOME}/config/apex/ && echo -e \ "@apex_rest_config_core.sql;\n" + //"exec APEX_UTIL.set_workspace(p_workspace => 'INTERNAL');\n" + //"exec APEX_UTIL.EDIT_USER(p_user_id => APEX_UTIL.GET_USER_ID('ADMIN'), p_user_name => 'ADMIN', p_change_password_on_first_use => 'Y');\n" + - "\" | sqlplus -s sys/%[2]s@${ORACLE_HOST}:${ORACLE_PORT}/${ORACLE_SERVICE} as sysdba;" + "\" | sqlplus -s sys/%[2]s@${ORACLE_HOST}:${ORACLE_PORT}/%[3]s as sysdba;" const IsApexInstalled string = "echo -e \"select 'APEXVERSION:'||version as version FROM DBA_REGISTRY WHERE COMP_ID='APEX';\"" + - " | sqlplus -s sys/%[1]s@${ORACLE_HOST}:${ORACLE_PORT}/${ORACLE_SERVICE} as sysdba;" + " | sqlplus -s sys/%[1]s@${ORACLE_HOST}:${ORACLE_PORT}/%[2]s as sysdba;" const UninstallApex string = "if [ -f /opt/oracle/oradata/${ORACLE_SID^^}/apex/apxremov.sql ]; then ( while true; do sleep 60; echo \"Uninstalling Apex...\" ; done ) & " + " cd /opt/oracle/oradata/${ORACLE_SID^^}/apex && echo -e \"@apxremov.sql\" | %[1]s && kill -9 $!; else echo \"Apex Folder doesn't exist\" ; fi ;" diff --git a/config/samples/sidb/oraclerestdataservice.yaml b/config/samples/sidb/oraclerestdataservice.yaml index 83a684ea..28d56ec9 100644 --- a/config/samples/sidb/oraclerestdataservice.yaml +++ b/config/samples/sidb/oraclerestdataservice.yaml @@ -25,8 +25,8 @@ spec: secretKey: keepSecret: true - ## Secret containing a common password for APEX_PUBLIC_USER, APEX_REST_PUBLIC_USER, APEX_LISTENER and Apex administrator (username: ADMIN) mapped to secretKey - ## Mention a non-null string for apexPassword.secretName to configure APEX with ORDS. + ## To configure APEX with ORDS, specfiy the apexPassword secret details. Leave empty if Apex is not needed. + ## This is a secret containing a common password for APEX_PUBLIC_USER, APEX_REST_PUBLIC_USER, APEX_LISTENER and Apex administrator (username: ADMIN) mapped to secretKey ## This secret will be deleted after ORDS Installation unless keepSecret set to true. apexPassword: secretName: @@ -38,17 +38,13 @@ spec: pullFrom: pullSecrets: - ## Uncomment only if you want to use seperate pvc for ords. else same pvc for sidb is used here - ## apex-latest.zip file should be present in the location '/opt/oracle/ords/config/ords' - ## Download using `wget https://download.oracle.com/otn_software/apex/apex-latest.zip` - ## use `kubectl cp :/opt/oracle/ords/config/ords ## size : Minimum size of pvc | class : PVC storage Class ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. - # persistence: - # size: 100Gi - # storageClass: "oci" - # accessMode: "ReadWriteOnce" + persistence: + size: 100Gi + storageClass: "oci-bv" + accessMode: "ReadWriteOnce" ## Type of service Applicable on cloud enviroments only. ## if loadBalService: false, service type = "NodePort". else "LoadBalancer" diff --git a/config/samples/sidb/oraclerestdataservice_create.yaml b/config/samples/sidb/oraclerestdataservice_create.yaml index c225212c..e352f800 100644 --- a/config/samples/sidb/oraclerestdataservice_create.yaml +++ b/config/samples/sidb/oraclerestdataservice_create.yaml @@ -6,7 +6,7 @@ apiVersion: v1 kind: Secret metadata: - name: ords-secret-1 + name: ords-secret type: Opaque stringData: oracle_pwd: "ChangeOnInstall_1" @@ -20,34 +20,33 @@ metadata: spec: ## Database ref. This can be of kind SingleInstanceDatabase. - databaseRef: "sidb-sample" + ## Make sure the source database has been created by applying singeinstancedatabase_prebuilt.yaml + databaseRef: "prebuiltdb-sample" ## Secret containing databaseRef password mapped to secretKey. ## This secret will be deleted after ORDS Installation unless keepSecret set to true. adminPassword: - secretName: admin-secret-1 + secretName: prebuiltdb-admin-secret secretKey: oracle_pwd keepSecret: true ## Secret containing ORDS_PUBLIC_USER password mapped to secretKey. ## This secret will be deleted after ORDS Installation unless keepSecret set to true. ordsPassword: - secretName: ords-secret-1 + secretName: ords-secret secretKey: oracle_pwd keepSecret: true - ## ORDS image details image: - pullFrom: - pullSecrets: + pullFrom: container-registry.oracle.com/database/ords:21.4.2-gh ## size : Minimum size of pvc | class : PVC storage Class ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. persistence: size: 100Gi - storageClass: "oci" + storageClass: "oci-bv" accessMode: "ReadWriteOnce" @@ -57,6 +56,11 @@ spec: - schema: schema1 enable: true urlMapping: - pdb: + pdb: xepdb1 + - schema: schema2 + enable: true + urlMapping: + pdb: xepdb1 + \ No newline at end of file diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index bc998370..3e08b67e 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -42,12 +42,6 @@ spec: ## Enable/Disable ForceLogging forceLog: false - ## apex-latest.zip file should be present in the location '/opt/oracle/oradata' - ## Download using `wget https://download.oracle.com/otn_software/apex/apex-latest.zip` - ## use `kubectl cp :/opt/oracle/oradata - ## Deploy OracleRestDataService CR to front APEX - installApex: false - ## NA if cloning from a SourceDB (cloneFrom is set) ## Specify both sgaSize and pgaSize (in MB) or dont specify both ## Specify Non-Zero value to use @@ -74,7 +68,7 @@ spec: ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. persistence: size: 100Gi - storageClass: "oci" + storageClass: "oci-bv" accessMode: "ReadWriteOnce" ## Type of service . Applicable on cloud enviroments only diff --git a/config/samples/sidb/singleinstancedatabase_clone.yaml b/config/samples/sidb/singleinstancedatabase_clone.yaml index 9fa611d3..dbe7a3dc 100644 --- a/config/samples/sidb/singleinstancedatabase_clone.yaml +++ b/config/samples/sidb/singleinstancedatabase_clone.yaml @@ -21,7 +21,7 @@ spec: ## Secret containing SIDB password mapped to secretKey ## This secret will be deleted after creation of the database unless keepSecret is set to true adminPassword: - secretName: admin-secret-1 + secretName: db-admin-secret secretKey: oracle_pwd keepSecret: true @@ -38,5 +38,5 @@ spec: ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. persistence: size: 100Gi - storageClass: "oci" + storageClass: "oci-bv" accessMode: "ReadWriteOnce" diff --git a/config/samples/sidb/singleinstancedatabase_create.yaml b/config/samples/sidb/singleinstancedatabase_create.yaml index 672d3b61..be8d7630 100644 --- a/config/samples/sidb/singleinstancedatabase_create.yaml +++ b/config/samples/sidb/singleinstancedatabase_create.yaml @@ -6,7 +6,7 @@ apiVersion: v1 kind: Secret metadata: - name: admin-secret-1 + name: db-admin-secret type: Opaque stringData: oracle_pwd: "ChangeOnInstall_1" @@ -31,7 +31,7 @@ spec: ## Secret containing SIDB password mapped to secretKey ## This secret will be deleted after creation of the database unless keepSecret is set to true adminPassword: - secretName: admin-secret-1 + secretName: db-admin-secret secretKey: oracle_pwd keepSecret: true @@ -54,5 +54,5 @@ spec: ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. persistence: size: 100Gi - storageClass: "oci" + storageClass: "oci-bv" accessMode: "ReadWriteOnce" diff --git a/config/samples/sidb/singleinstancedatabase_minikube.yaml b/config/samples/sidb/singleinstancedatabase_minikube.yaml deleted file mode 100644 index 0d81ec3a..00000000 --- a/config/samples/sidb/singleinstancedatabase_minikube.yaml +++ /dev/null @@ -1,61 +0,0 @@ -# -# Copyright (c) 2022, Oracle and/or its affiliates. -# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. -# - -apiVersion: v1 -kind: Secret -metadata: - name: admin-secret-3 -type: Opaque -stringData: - oracle_pwd: "ChangeOnInstall_3" - ---- - -apiVersion: v1 -kind: PersistentVolume -metadata: - name: minikube-pv -spec: - accessModes: - - ReadWriteOnce - capacity: - storage: 5Gi - storageClassName: minikube-sc - hostPath: - path: /data/oradata - ---- - -apiVersion: database.oracle.com/v1alpha1 -kind: SingleInstanceDatabase -metadata: - name: xedb-minikube-sample - namespace: default -spec: - - ## Use only alphanumeric characters for sid - sid: XE - - ## DB edition. NA if cloning from a SourceDB (cloneFrom is set) - edition: express - - ## Secret containing SIDB password mapped to secretKey - ## This secret will be deleted after creation of the database unless keepSecret is set to true - adminPassword: - secretName: admin-secret-3 - secretKey: oracle_pwd - keepSecret: true - - ## Database image details - image: - pullFrom: container-registry.oracle.com/database/express:latest - prebuiltDB: true - - ## size : Minimum size of pvc | class : PVC storage Class - ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany - persistence: - size: 5Gi - storageClass: "minikube-sc" - accessMode: "ReadWriteOnce" diff --git a/config/samples/sidb/singleinstancedatabase_patch.yaml b/config/samples/sidb/singleinstancedatabase_patch.yaml index f843568f..704e1058 100644 --- a/config/samples/sidb/singleinstancedatabase_patch.yaml +++ b/config/samples/sidb/singleinstancedatabase_patch.yaml @@ -20,7 +20,7 @@ spec: ## Secret containing SIDB password mapped to secretKey ## This secret will be deleted after creation of the database unless keepSecret is set to true adminPassword: - secretName: admin-secret-1 + secretName: db-admin-secret secretKey: oracle_pwd keepSecret: true @@ -46,5 +46,5 @@ spec: ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. persistence: size: 100Gi - storageClass: "oci" + storageClass: "oci-bv" accessMode: "ReadWriteOnce" diff --git a/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml b/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml index dadfcddb..db2f5034 100644 --- a/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml +++ b/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml @@ -6,29 +6,30 @@ apiVersion: v1 kind: Secret metadata: - name: admin-secret-4 + name: prebuiltdb-admin-secret type: Opaque stringData: - oracle_pwd: "ChangeOnInstall_4" + oracle_pwd: "ChangeOnInstall_1" --- apiVersion: database.oracle.com/v1alpha1 kind: SingleInstanceDatabase metadata: - name: xe-prebuilt-sample + name: prebuiltdb-sample namespace: default spec: ## Use only alphanumeric characters for sid sid: XE + ## DB edition edition: express ## Secret containing SIDB password mapped to secretKey ## This secret will be deleted after creation of the database unless keepSecret is set to true adminPassword: - secretName: admin-secret-4 + secretName: prebuiltdb-admin-secret secretKey: oracle_pwd keepSecret: true @@ -41,5 +42,5 @@ spec: ## if specified, the prebuilt DB datafiles are copied over to the persistant volume before DB startup #persistence: # size: 100Gi - # storageClass: "oci" + # storageClass: "oci-bv" # accessMode: "ReadWriteOnce" diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 90922a3d..e68a69c7 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -1072,7 +1072,7 @@ func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataS } } // Set Apex users in apex_rt,apex_al,apex files - out, err := dbcommons.ExecCommand(r, r.Config, ordsReadyPod.Name, ordsReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + out, err := dbcommons.ExecCommand(r, r.Config, ordsReadyPod.Name, ordsReadyPod.Namespace, "", ctx, req, true, "bash", "-c", fmt.Sprintf(dbcommons.SetApexUsers, apexPassword)) log.Info("SetApexUsers Output: \n" + out) if strings.Contains(strings.ToUpper(out), "ERROR") { @@ -1131,7 +1131,7 @@ func (r *OracleRestDataServiceReconciler) installApex(m *dbapi.OracleRestDataSer //Install Apex in SIDB ready pod out, err := dbcommons.ExecCommand(r, r.Config, ordsReadyPod.Name, ordsReadyPod.Namespace, "", ctx, req, true, "bash", "-c", - fmt.Sprintf(dbcommons.InstallApexInContainer, apexPassword, sidbPassword)) + fmt.Sprintf(dbcommons.InstallApexInContainer, apexPassword, sidbPassword, n.Status.Pdbname)) if err != nil { log.Info(err.Error()) } @@ -1139,7 +1139,7 @@ func (r *OracleRestDataServiceReconciler) installApex(m *dbapi.OracleRestDataSer // Checking if Apex is installed successfully or not out, err = dbcommons.ExecCommand(r, r.Config, ordsReadyPod.Name, ordsReadyPod.Namespace, "", ctx, req, true, "bash", "-c", - fmt.Sprintf(dbcommons.IsApexInstalled, sidbPassword)) + fmt.Sprintf(dbcommons.IsApexInstalled, sidbPassword, n.Status.Pdbname)) if err != nil { log.Error(err, err.Error()) return requeueY From 43197a0c9a10e8a4ada649f3e16d944e295974e6 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Thu, 28 Apr 2022 13:58:09 +0530 Subject: [PATCH 288/628] Fix Apex Status URLs --- .../v1alpha1/oraclerestdataservice_types.go | 7 +- commons/database/constants.go | 6 +- .../sidb/oraclerestdataservice_apex.yaml | 81 +++++++ .../sidb/singleinstancedatabase_express.yaml | 46 ++++ .../oraclerestdataservice_controller.go | 226 +++--------------- 5 files changed, 170 insertions(+), 196 deletions(-) create mode 100644 config/samples/sidb/oraclerestdataservice_apex.yaml create mode 100644 config/samples/sidb/singleinstancedatabase_express.yaml diff --git a/apis/database/v1alpha1/oraclerestdataservice_types.go b/apis/database/v1alpha1/oraclerestdataservice_types.go index 79647f57..bfe57c02 100644 --- a/apis/database/v1alpha1/oraclerestdataservice_types.go +++ b/apis/database/v1alpha1/oraclerestdataservice_types.go @@ -105,13 +105,13 @@ type OracleRestDataServiceStatus struct { // Important: Run "make" to regenerate code after modifying this file Status string `json:"status,omitempty"` DatabaseApiUrl string `json:"databaseApiUrl,omitempty"` - ClusterDbApiUrl string `json:"clusterDbApiUrl,omitempty"` LoadBalancer string `json:"loadBalancer,omitempty"` DatabaseRef string `json:"databaseRef,omitempty"` ServiceIP string `json:"serviceIP,omitempty"` DatabaseActionsUrl string `json:"databaseActionsUrl,omitempty"` OrdsInstalled bool `json:"ordsInstalled,omitempty"` ApexConfigured bool `json:"apexConfigured,omitempty"` + ApxeUrl string `json:"apexUrl,omitempty"` CommonUsersCreated bool `json:"commonUsersCreated,omitempty"` OrdsSetupCompleted bool `json:"ordsSetupCompleted,omitempty"` Replicas int `json:"replicas,omitempty"` @@ -123,8 +123,9 @@ type OracleRestDataServiceStatus struct { //+kubebuilder:subresource:status // +kubebuilder:printcolumn:JSONPath=".status.status",name="Status",type="string" // +kubebuilder:printcolumn:JSONPath=".spec.databaseRef",name="Database",type="string" -// +kubebuilder:printcolumn:JSONPath=".status.databaseApiUrl",name="Database Api & Apex Url ",type="string" -// +kubebuilder:printcolumn:JSONPath=".status.databaseActionsUrl",name="Database Actions Url",type="string" +// +kubebuilder:printcolumn:JSONPath=".status.databaseApiUrl",name="Database API",type="string" +// +kubebuilder:printcolumn:JSONPath=".status.databaseActionsUrl",name="Database Actions URL",type="string" +// +kubebuilder:printcolumn:JSONPath=".status.apexUrl",name="Apex URL",type="string" // OracleRestDataService is the Schema for the oraclerestdataservices API type OracleRestDataService struct { diff --git a/commons/database/constants.go b/commons/database/constants.go index 5e915ca9..1c50ae81 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -304,11 +304,11 @@ const InitORDSCMD string = "if [ -f $ORDS_HOME/config/ords/defaults.xml ]; then "\nmkdir -p $ORDS_HOME/config/ords/conf" + "\numask 177" + "\necho db.cdb.adminUser=C##DBAPI_CDB_ADMIN AS SYSDBA > cdbAdmin.properties" + - "\necho db.cdb.adminUser.password=\"${ORDS_PWD}\" >> cdbAdmin.properties" + + "\necho db.cdb.adminUser.password=\"${ORACLE_PWD}\" >> cdbAdmin.properties" + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-properties --conf apex_pu cdbAdmin.properties" + "\nrm -f cdbAdmin.properties" + "\necho db.adminUser=C##_DBAPI_PDB_ADMIN > pdbAdmin.properties" + - "\necho db.adminUser.password=\"${ORDS_PWD}\">> pdbAdmin.properties" + + "\necho db.adminUser.password=\"${ORACLE_PWD}\">> pdbAdmin.properties" + "\n$JAVA_HOME/bin/java -jar $ORDS_HOME/ords.war set-properties --conf apex_pu pdbAdmin.properties" + "\nrm -f pdbAdmin.properties" + "\necho -e \"${ORDS_PWD}\n${ORDS_PWD}\" > sqladmin.passwd" + @@ -369,6 +369,8 @@ const StatusReady string = "Healthy" const StatusError string = "Error" +const StatusUnavailable string = "Unavailable" + const ValueUnavailable string = "Unknown" const NoExternalIp string = "Node ExternalIP unavailable" diff --git a/config/samples/sidb/oraclerestdataservice_apex.yaml b/config/samples/sidb/oraclerestdataservice_apex.yaml new file mode 100644 index 00000000..372f4a32 --- /dev/null +++ b/config/samples/sidb/oraclerestdataservice_apex.yaml @@ -0,0 +1,81 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +apiVersion: v1 +kind: Secret +metadata: + name: ords-secret +type: Opaque +stringData: + oracle_pwd: "ChangeOnInstall_1" + +--- + +apiVersion: v1 +kind: Secret +metadata: + name: apex-secret +type: Opaque +stringData: + oracle_pwd: "ChangeOnApexInstall_1" + +--- + +apiVersion: database.oracle.com/v1alpha1 +kind: OracleRestDataService +metadata: + name: ords-sample +spec: + + ## Database ref. This can be of kind SingleInstanceDatabase. + ## Make sure the source database has been created by applying singeinstancedatabase_prebuilt.yaml + databaseRef: "prebuiltdb-sample" + + ## Secret containing databaseRef password mapped to secretKey. + ## This secret will be deleted after ORDS Installation unless keepSecret set to true. + adminPassword: + secretName: prebuiltdb-admin-secret + secretKey: oracle_pwd + keepSecret: true + + ## Secret containing ORDS_PUBLIC_USER password mapped to secretKey. + ## This secret will be deleted after ORDS Installation unless keepSecret set to true. + ordsPassword: + secretName: ords-secret + secretKey: oracle_pwd + keepSecret: true + + ## To configure APEX with ORDS, specfiy the apexPassword secret details. Leave empty if Apex is not needed. + ## This is a secret containing a common password for APEX_PUBLIC_USER, APEX_REST_PUBLIC_USER, APEX_LISTENER and Apex administrator (username: ADMIN) mapped to secretKey + ## This secret will be deleted after ORDS Installation unless keepSecret set to true. + apexPassword: + secretName: apex-secret + secretKey: oracle_pwd + keepSecret: true + + ## ORDS image details + image: + pullFrom: container-registry.oracle.com/database/ords:21.4.2-gh + + ## size : Minimum size of pvc | class : PVC storage Class + ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany + ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. + persistence: + size: 100Gi + storageClass: "oci-bv" + accessMode: "ReadWriteOnce" + + ## PDB Schemas to be ORDS Enabled. + ## Schema will be created (if not exists) with username as schema and password as .spec.ordsPassword. + restEnableSchemas: + - schema: schema1 + enable: true + urlMapping: + pdb: xepdb1 + - schema: schema2 + enable: true + urlMapping: + pdb: xepdb1 + \ No newline at end of file diff --git a/config/samples/sidb/singleinstancedatabase_express.yaml b/config/samples/sidb/singleinstancedatabase_express.yaml new file mode 100644 index 00000000..cc8da754 --- /dev/null +++ b/config/samples/sidb/singleinstancedatabase_express.yaml @@ -0,0 +1,46 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +apiVersion: v1 +kind: Secret +metadata: + name: xe-admin-secret +type: Opaque +stringData: + oracle_pwd: "ChangeOnInstall_1" + +--- + +apiVersion: database.oracle.com/v1alpha1 +kind: SingleInstanceDatabase +metadata: + name: xedb-minikube-sample + namespace: default +spec: + + ## Use only alphanumeric characters for sid + sid: XE + + ## DB edition + edition: express + + ## Secret containing SIDB password mapped to secretKey + ## This secret will be deleted after creation of the database unless keepSecret is set to true + adminPassword: + secretName: xe-admin-secret + secretKey: oracle_pwd + keepSecret: true + + ## Database image details + image: + pullFrom: container-registry.oracle.com/database/express:latest + prebuiltDB: true + + ## size : Minimum size of pvc | class : PVC storage Class + ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany + persistence: + size: 5Gi + storageClass: "oci-bv" + accessMode: "ReadWriteOnce" diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index e68a69c7..a95eb724 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -138,7 +138,7 @@ func (r *OracleRestDataServiceReconciler) Reconcile(ctx context.Context, req ctr defer r.Status().Update(ctx, singleInstanceDatabase) // Create Service - result = r.createSVC(ctx, req, oracleRestDataService) + result = r.createSVC(ctx, req, oracleRestDataService, singleInstanceDatabase) if result.Requeue { r.Log.Info("Reconcile queued") return result, nil @@ -152,22 +152,15 @@ func (r *OracleRestDataServiceReconciler) Reconcile(ctx context.Context, req ctr } // Validate if Primary Database Reference is ready - result, sidbReadyPod := r.validateSidbReadiness(oracleRestDataService, singleInstanceDatabase, ctx, req) + result, sidbReadyPod := r.validateSIDBReadiness(oracleRestDataService, singleInstanceDatabase, ctx, req) if result.Requeue { r.Log.Info("Reconcile queued") return result, nil } - // setup DB for ORDS - var cdbAdminPassword string - result, cdbAdminPassword = r.setupDBPrequisites(oracleRestDataService, singleInstanceDatabase, sidbReadyPod, ctx, req) - if result.Requeue { - r.Log.Info("Reconcile queued") - return result, nil - } // Create ORDS Pods - result = r.createPods(oracleRestDataService, singleInstanceDatabase, cdbAdminPassword, ctx, req) + result = r.createPods(oracleRestDataService, singleInstanceDatabase, ctx, req) if result.Requeue { r.Log.Info("Reconcile queued") return result, nil @@ -246,7 +239,7 @@ func (r *OracleRestDataServiceReconciler) validate(m *dbapi.OracleRestDataServic //##################################################################################################### // Validate Readiness of the primary DB specified //##################################################################################################### -func (r *OracleRestDataServiceReconciler) validateSidbReadiness(m *dbapi.OracleRestDataService, +func (r *OracleRestDataServiceReconciler) validateSIDBReadiness(m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase, ctx context.Context, req ctrl.Request) (ctrl.Result, corev1.Pod) { log := r.Log.WithValues("validateSidbReadiness", req.NamespacedName) @@ -304,39 +297,23 @@ func (r *OracleRestDataServiceReconciler) validateSidbReadiness(m *dbapi.OracleR return requeueY, sidbReadyPod } - return requeueN, sidbReadyPod -} - -//##################################################################################################### -// Set up DB prereqs -//##################################################################################################### -func (r *OracleRestDataServiceReconciler) setupDBPrequisites(m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase, - sidbReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) (ctrl.Result, string) { - log := r.Log.WithValues("setupDBPrequisites", req.NamespacedName) - - if !m.Status.CommonUsersCreated { - - cdbAdminPassword := dbcommons.GenerateRandomString(8) - // Create PDB , CDB Admin users and grant permissions . ORDS installation on CDB level - out, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", - fmt.Sprintf("echo -e \"%s\" | %s", fmt.Sprintf(dbcommons.SetAdminUsersSQL, cdbAdminPassword), dbcommons.SQLPlusCLI)) - if err != nil { - log.Error(err, err.Error()) - return requeueY, "" - } - log.Info("SetAdminUsers Output :\n" + out) - - if !strings.Contains(out, "ERROR") || !strings.Contains(out, "ORA-") || - strings.Contains(out, "ERROR") && strings.Contains(out, "ORA-01920") { - m.Status.CommonUsersCreated = true - } - - return requeueN, cdbAdminPassword + // Create PDB , CDB Admin users and grant permissions. ORDS installation on CDB level + out, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | %s", fmt.Sprintf(dbcommons.SetAdminUsersSQL, adminPassword), dbcommons.SQLPlusCLI)) + if err != nil { + log.Error(err, err.Error()) + return requeueY, sidbReadyPod } + log.Info("SetAdminUsers Output :\n" + out) - return requeueN, "" + if !strings.Contains(out, "ERROR") || !strings.Contains(out, "ORA-") || + strings.Contains(out, "ERROR") && strings.Contains(out, "ORA-01920") { + m.Status.CommonUsersCreated = true + } + return requeueN, sidbReadyPod } + //##################################################################################################### // Validate Readiness of the ORDS pods //##################################################################################################### @@ -449,7 +426,7 @@ func (r *OracleRestDataServiceReconciler) instantiateSVCSpec(m *dbapi.OracleRest // Instantiate POD spec from OracleRestDataService spec //############################################################################# func (r *OracleRestDataServiceReconciler) instantiatePodSpec(m *dbapi.OracleRestDataService, - n *dbapi.SingleInstanceDatabase, ordsInstalled bool, cdbAdminPassword string) *corev1.Pod { + n *dbapi.SingleInstanceDatabase) *corev1.Pod { pod := &corev1.Pod{ TypeMeta: metav1.TypeMeta{ @@ -679,7 +656,7 @@ func (r *OracleRestDataServiceReconciler) instantiatePVCSpec(m *dbapi.OracleRest // Create a Service for OracleRestDataService //############################################################################# func (r *OracleRestDataServiceReconciler) createSVC(ctx context.Context, req ctrl.Request, - m *dbapi.OracleRestDataService) ctrl.Result { + m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase) ctrl.Result { log := r.Log.WithValues("createSVC", req.NamespacedName) // Check if the Service already exists, if not create a new one @@ -710,19 +687,27 @@ func (r *OracleRestDataServiceReconciler) createSVC(ctx context.Context, req ctr m.Status.ServiceIP = "" if m.Spec.LoadBalancer { if len(svc.Status.LoadBalancer.Ingress) > 0 { - m.Status.DatabaseApiUrl = "https://" + svc.Status.LoadBalancer.Ingress[0].IP + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/ords/" + m.Status.DatabaseApiUrl = "https://" + svc.Status.LoadBalancer.Ingress[0].IP + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/ords/_/db-api/stable/" m.Status.ServiceIP = svc.Status.LoadBalancer.Ingress[0].IP m.Status.DatabaseActionsUrl = "https://" + svc.Status.LoadBalancer.Ingress[0].IP + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/ords/sql-developer" + if m.Status.ApexConfigured { + m.Status.ApxeUrl = "https://" + svc.Status.LoadBalancer.Ingress[0].IP + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/ords/" + n.Status.Pdbname + "/apex" + } else { + m.Status.ApxeUrl = dbcommons.StatusUnavailable + } } - m.Status.ClusterDbApiUrl = "https://" + svc.Name + "." + svc.Namespace + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/ords/" return requeueN } - m.Status.ClusterDbApiUrl = "https://" + svc.Name + "." + svc.Namespace + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/ords/" nodeip := dbcommons.GetNodeIp(r, ctx, req) if nodeip != "" { m.Status.ServiceIP = nodeip - m.Status.DatabaseApiUrl = "https://" + nodeip + ":" + fmt.Sprint(svc.Spec.Ports[0].NodePort) + "/ords/" + m.Status.DatabaseApiUrl = "https://" + nodeip + ":" + fmt.Sprint(svc.Spec.Ports[0].NodePort) + "/ords/_/db-api/stable/" m.Status.DatabaseActionsUrl = "https://" + nodeip + ":" + fmt.Sprint(svc.Spec.Ports[0].NodePort) + "/ords/sql-developer" + if m.Status.ApexConfigured { + m.Status.ApxeUrl = "https://" + nodeip + ":" + fmt.Sprint(svc.Spec.Ports[0].NodePort) + "/ords/" + n.Status.Pdbname + "/apex" + } else { + m.Status.ApxeUrl = dbcommons.StatusUnavailable + } } return requeueN } @@ -765,7 +750,7 @@ func (r *OracleRestDataServiceReconciler) createPVC(ctx context.Context, req ctr // Create the requested POD replicas //############################################################################# func (r *OracleRestDataServiceReconciler) createPods(m *dbapi.OracleRestDataService, - n *dbapi.SingleInstanceDatabase, cdbAdminPassword string, ctx context.Context, req ctrl.Request) ctrl.Result { + n *dbapi.SingleInstanceDatabase, ctx context.Context, req ctrl.Request) ctrl.Result { log := r.Log.WithValues("createPods", req.NamespacedName) @@ -782,8 +767,8 @@ func (r *OracleRestDataServiceReconciler) createPods(m *dbapi.OracleRestDataServ replicasReq := m.Spec.Replicas if replicasFound == 0 { m.Status.Status = dbcommons.StatusNotReady - m.Status.DatabaseApiUrl = dbcommons.ValueUnavailable - m.Status.DatabaseActionsUrl = dbcommons.ValueUnavailable + m.Status.DatabaseApiUrl = dbcommons.StatusUnavailable + m.Status.DatabaseActionsUrl = dbcommons.StatusUnavailable } if replicasFound == replicasReq { @@ -791,15 +776,7 @@ func (r *OracleRestDataServiceReconciler) createPods(m *dbapi.OracleRestDataServ } else if replicasFound < replicasReq { // Create New Pods , Name of Pods are generated Randomly for i := replicasFound; i < replicasReq; i++ { - if i > 0 && !m.Status.OrdsInstalled { - // Wait till the first created pod to installs ORDS . - break - } - ordsInstalled := false - if m.Status.OrdsInstalled { - ordsInstalled = true - } - pod := r.instantiatePodSpec(m, n, ordsInstalled, cdbAdminPassword) + pod := r.instantiatePodSpec(m, n) log.Info("Creating a new "+m.Name+" POD", "POD.Namespace", pod.Namespace, "POD.Name", pod.Name) err := r.Create(ctx, pod) if err != nil { @@ -1206,137 +1183,6 @@ func (r *OracleRestDataServiceReconciler) deleteSecrets(m *dbapi.OracleRestDataS } -//############################################################################# -// Setup ORDS in CDB , enable ORDS for PDBs Specified -//############################################################################# -func (r *OracleRestDataServiceReconciler) setupORDS(m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase, - sidbReadyPod corev1.Pod, cdbAdminPassword string, ctx context.Context, req ctrl.Request) ctrl.Result { - - log := r.Log.WithValues("setupORDS", req.NamespacedName) - - readyPod, replicasFound, _, _, err := dbcommons.FindPods(r, m.Spec.Image.Version, - m.Spec.Image.PullFrom, m.Name, m.Namespace, ctx, req) - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - if readyPod.Name == "" { - eventReason := "Waiting" - eventMsg := "waiting for " + m.Name + " to be Ready" - r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) - return requeueY - } - OrdsPasswordSecret := &corev1.Secret{} - if !m.Status.OrdsInstalled { - err := r.Get(ctx, types.NamespacedName{Name: m.Spec.OrdsPassword.SecretName, Namespace: m.Namespace}, OrdsPasswordSecret) - if err != nil { - if apierrors.IsNotFound(err) { - m.Status.Status = dbcommons.StatusError - eventReason := "Waiting" - eventMsg := "waiting for secret : " + m.Spec.OrdsPassword.SecretName + " to get created" - r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) - r.Log.Info("Secret " + m.Spec.OrdsPassword.SecretName + " Not Found") - return requeueY - } - log.Error(err, err.Error()) - return requeueY - } - ordsPassword := string(OrdsPasswordSecret.Data[m.Spec.OrdsPassword.SecretKey]) - - // APEX_LISTENER , APEX_REST_PUBLIC_USER , APEX_PUBLIC_USER , APEX ADMIN passwords - apexPassword := ordsPassword - if m.Spec.ApexPassword.SecretName != "" { - apexPasswordSecret := &corev1.Secret{} - err = r.Get(ctx, types.NamespacedName{Name: m.Spec.ApexPassword.SecretName, Namespace: m.Namespace}, apexPasswordSecret) - if err != nil { - if apierrors.IsNotFound(err) { - m.Status.Status = dbcommons.StatusError - eventReason := "Waiting" - eventMsg := "waiting for secret : " + m.Spec.ApexPassword.SecretName + " to get created" - r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) - r.Log.Info("Secret " + m.Spec.ApexPassword.SecretName + " Not Found") - return requeueY - } - log.Error(err, err.Error()) - return requeueY - } - apexPassword = string(apexPasswordSecret.Data[m.Spec.ApexPassword.SecretKey]) - } - - // Get ORDS Status - out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", - dbcommons.GetORDSStatus) - log.Info("GetORDSStatus Output") - log.Info(out) - if strings.Contains(strings.ToUpper(out), "ERROR") { - return requeueY - } - if err != nil { - log.Info(err.Error()) - if strings.Contains(strings.ToUpper(err.Error()), "ERROR") { - return requeueY - } - } - - if strings.Contains(out, "HTTP/1.1 200 OK") || strings.Contains(strings.ToUpper(err.Error()), "HTTP/1.1 200 OK") { - - if !m.Status.OrdsSetupCompleted { - - // Setup ORDS - out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, true, "bash", "-c", - fmt.Sprintf(dbcommons.SetupORDSCMD, ordsPassword, apexPassword, m.Name, cdbAdminPassword)) - log.Info("SetupORDSCMD Output") - log.Info(out) - if strings.Contains(strings.ToUpper(out), "ERROR") { - return requeueY - } - if err != nil { - log.Info(err.Error()) - if strings.Contains(strings.ToUpper(err.Error()), "ERROR") { - return requeueY - } - } - - if !strings.Contains(strings.ToUpper(out), "ERROR") && !strings.Contains(strings.ToUpper(err.Error()), "ERROR") && - !strings.Contains(strings.ToUpper(out), "could not execute") && !strings.Contains(strings.ToUpper(err.Error()), "could not execute") { - m.Status.OrdsSetupCompleted = true - } - } - - // ORDS Needs to be restarted to ensure ORDS Installation works fine - err := r.Delete(ctx, &readyPod, &client.DeleteOptions{}) - if err != nil { - r.Log.Error(err, "Failed to delete existing POD", "POD.Name", readyPod.Name) - return requeueY - } - r.Log.Info("ORDS Installation Pod Deleted : " + readyPod.Name) - - eventReason := "ORDS Installed" - eventMsg := "" - r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) - m.Status.OrdsInstalled = true - r.Status().Update(ctx, m) - - // Wait for 15 sec to update OrdsInstalled to true - time.Sleep(15 * time.Second) - - } - return requeueY - } - - result := r.restEnableSchemas(m, n, sidbReadyPod, ctx, req) - if result.Requeue { - r.Log.Info("Reconcile queued") - return result - } - - if replicasFound < m.Spec.Replicas { - // Requeue if all replicas not created - return requeueY - } - return requeueN -} - //############################################################################# // Rest Enable/Disable Schemas //############################################################################# @@ -1357,7 +1203,6 @@ func (r *OracleRestDataServiceReconciler) restEnableSchemas(m *dbapi.OracleRestD } for i := 0; i < len(m.Spec.RestEnableSchemas); i++ { - // If the PDB mentioned in yaml doesnt contain in the database , continue if !strings.Contains(strings.ToUpper(availablePDBS), strings.ToUpper(m.Spec.RestEnableSchemas[i].Pdb)) { eventReason := "Warning" @@ -1427,7 +1272,6 @@ func (r *OracleRestDataServiceReconciler) restEnableSchemas(m *dbapi.OracleRestD } return requeueN - } //############################################################################# From 27fa096f7d2c2d3cb7c5a6c7cf87c5b2a27cdf19 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Thu, 28 Apr 2022 14:01:04 +0530 Subject: [PATCH 289/628] Sample file fix --- config/samples/sidb/oraclerestdataservice.yaml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/config/samples/sidb/oraclerestdataservice.yaml b/config/samples/sidb/oraclerestdataservice.yaml index 28d56ec9..69eee4b2 100644 --- a/config/samples/sidb/oraclerestdataservice.yaml +++ b/config/samples/sidb/oraclerestdataservice.yaml @@ -34,6 +34,8 @@ spec: keepSecret: true ## ORDS image details + ## Build the ORDS image following instructions at + ## https://github.com/oracle/docker-images/tree/main/OracleRestDataServices image: pullFrom: pullSecrets: From c3570ec790d7a96f9eb2b6242cf2254b08b25d3f Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Thu, 28 Apr 2022 16:52:49 +0530 Subject: [PATCH 290/628] Fix URLs --- .../v1alpha1/oraclerestdataservice_types.go | 2 +- .../v1alpha1/singleinstancedatabase_types.go | 1 - .../oraclerestdataservice_controller.go | 18 +- .../singleinstancedatabase_controller.go | 174 +----------------- 4 files changed, 19 insertions(+), 176 deletions(-) diff --git a/apis/database/v1alpha1/oraclerestdataservice_types.go b/apis/database/v1alpha1/oraclerestdataservice_types.go index bfe57c02..2a29f86e 100644 --- a/apis/database/v1alpha1/oraclerestdataservice_types.go +++ b/apis/database/v1alpha1/oraclerestdataservice_types.go @@ -123,7 +123,7 @@ type OracleRestDataServiceStatus struct { //+kubebuilder:subresource:status // +kubebuilder:printcolumn:JSONPath=".status.status",name="Status",type="string" // +kubebuilder:printcolumn:JSONPath=".spec.databaseRef",name="Database",type="string" -// +kubebuilder:printcolumn:JSONPath=".status.databaseApiUrl",name="Database API",type="string" +// +kubebuilder:printcolumn:JSONPath=".status.databaseApiUrl",name="Database API URL",type="string" // +kubebuilder:printcolumn:JSONPath=".status.databaseActionsUrl",name="Database Actions URL",type="string" // +kubebuilder:printcolumn:JSONPath=".status.apexUrl",name="Apex URL",type="string" diff --git a/apis/database/v1alpha1/singleinstancedatabase_types.go b/apis/database/v1alpha1/singleinstancedatabase_types.go index f9dd6da9..b35b117b 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_types.go +++ b/apis/database/v1alpha1/singleinstancedatabase_types.go @@ -57,7 +57,6 @@ type SingleInstanceDatabaseSpec struct { // +k8s:openapi-gen=true // +kubebuilder:validation:Pattern=`^[a-zA-Z0-9]+$` Sid string `json:"sid,omitempty"` - InstallApex bool `json:"installApex,omitempty"` Charset string `json:"charset,omitempty"` Pdbname string `json:"pdbName,omitempty"` LoadBalancer bool `json:"loadBalancer,omitempty"` diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index a95eb724..7352f0dd 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -687,11 +687,14 @@ func (r *OracleRestDataServiceReconciler) createSVC(ctx context.Context, req ctr m.Status.ServiceIP = "" if m.Spec.LoadBalancer { if len(svc.Status.LoadBalancer.Ingress) > 0 { - m.Status.DatabaseApiUrl = "https://" + svc.Status.LoadBalancer.Ingress[0].IP + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/ords/_/db-api/stable/" + m.Status.DatabaseApiUrl = "https://" + svc.Status.LoadBalancer.Ingress[0].IP + ":" + + fmt.Sprint(svc.Spec.Ports[0].Port) + "/ords/"+n.Status.Pdbname+"_/db-api/stable/" m.Status.ServiceIP = svc.Status.LoadBalancer.Ingress[0].IP - m.Status.DatabaseActionsUrl = "https://" + svc.Status.LoadBalancer.Ingress[0].IP + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/ords/sql-developer" + m.Status.DatabaseActionsUrl = "https://" + svc.Status.LoadBalancer.Ingress[0].IP + ":" + + fmt.Sprint(svc.Spec.Ports[0].Port) + "/ords/sql-developer" if m.Status.ApexConfigured { - m.Status.ApxeUrl = "https://" + svc.Status.LoadBalancer.Ingress[0].IP + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/ords/" + n.Status.Pdbname + "/apex" + m.Status.ApxeUrl = "https://" + svc.Status.LoadBalancer.Ingress[0].IP + ":" + + fmt.Sprint(svc.Spec.Ports[0].Port) + "/ords/" + n.Status.Pdbname + "/apex" } else { m.Status.ApxeUrl = dbcommons.StatusUnavailable } @@ -701,10 +704,13 @@ func (r *OracleRestDataServiceReconciler) createSVC(ctx context.Context, req ctr nodeip := dbcommons.GetNodeIp(r, ctx, req) if nodeip != "" { m.Status.ServiceIP = nodeip - m.Status.DatabaseApiUrl = "https://" + nodeip + ":" + fmt.Sprint(svc.Spec.Ports[0].NodePort) + "/ords/_/db-api/stable/" - m.Status.DatabaseActionsUrl = "https://" + nodeip + ":" + fmt.Sprint(svc.Spec.Ports[0].NodePort) + "/ords/sql-developer" + m.Status.DatabaseApiUrl = "https://" + nodeip + ":" + fmt.Sprint(svc.Spec.Ports[0].NodePort) + + "/ords/"+n.Status.Pdbname+"_/db-api/stable/" + m.Status.DatabaseActionsUrl = "https://" + nodeip + ":" + fmt.Sprint(svc.Spec.Ports[0].NodePort) + + "/ords/sql-developer" if m.Status.ApexConfigured { - m.Status.ApxeUrl = "https://" + nodeip + ":" + fmt.Sprint(svc.Spec.Ports[0].NodePort) + "/ords/" + n.Status.Pdbname + "/apex" + m.Status.ApxeUrl = "https://" + nodeip + ":" + fmt.Sprint(svc.Spec.Ports[0].NodePort) + "/ords/" + + n.Status.Pdbname + "/apex" } else { m.Status.ApxeUrl = dbcommons.StatusUnavailable } diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 02f14e9c..b415938d 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -215,22 +215,6 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct } } - /* - // Install Apex - result = r.installApex(singleInstanceDatabase, readyPod, ctx, req) - if result.Requeue { - r.Log.Info("Reconcile queued") - return result, nil - } - - // Uninstall Apex - result = r.uninstallApex(singleInstanceDatabase, readyPod, ctx, req) - if result.Requeue { - r.Log.Info("Reconcile queued") - return result, nil - } - */ - // If LoadBalancer = true , ensure Connect String is updated if singleInstanceDatabase.Status.ConnectString == dbcommons.ValueUnavailable { return requeueY, nil @@ -238,7 +222,7 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct // update status to Ready after all operations succeed singleInstanceDatabase.Status.Status = dbcommons.StatusReady - r.updateORDSStatus(singleInstanceDatabase, false, ctx, req) + r.updateORDSStatus(singleInstanceDatabase, ctx, req) completed = true r.Log.Info("Reconcile completed") @@ -1355,7 +1339,7 @@ func (r *SingleInstanceDatabaseReconciler) validateDBReadiness(m *dbapi.SingleIn if m.Spec.Edition == "express" { eventReason = "Database Unhealthy" m.Status.Status = dbcommons.StatusNotReady - r.updateORDSStatus(m, false, ctx, req) + r.updateORDSStatus(m, ctx, req) } out, err := dbcommons.ExecCommand(r, r.Config, runningPod.Name, runningPod.Namespace, "", ctx, req, false, "bash", "-c", dbcommons.GetCheckpointFileCMD) @@ -1370,7 +1354,7 @@ func (r *SingleInstanceDatabaseReconciler) validateDBReadiness(m *dbapi.SingleIn eventMsg = "datafiles exists" m.Status.DatafilesCreated = "true" m.Status.Status = dbcommons.StatusNotReady - r.updateORDSStatus(m, false, ctx, req) + r.updateORDSStatus(m, ctx, req) } } @@ -1759,151 +1743,10 @@ func (r *SingleInstanceDatabaseReconciler) updateDBConfig(m *dbapi.SingleInstanc return requeueN, nil } -//############################################################################# -// Install APEX to CDB -//############################################################################# -func (r *SingleInstanceDatabaseReconciler) installApex(m *dbapi.SingleInstanceDatabase, - readyPod corev1.Pod, ctx context.Context, req ctrl.Request) ctrl.Result { - log := r.Log.WithValues("installApex", req.NamespacedName) - - // No APEX for Pre-built db - if m.Spec.Image.PrebuiltDB { - return requeueN - } - - if !m.Spec.InstallApex || m.Status.ApexInstalled { - return requeueN - } - - m.Status.Status = dbcommons.StatusUpdating - r.Status().Update(ctx, m) - eventReason := "Installing Apex" - eventMsg := "Waiting for Apex Installation to complete" - r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) - - unzipApex := dbcommons.UnzipApexOnSIDBPod - if m.Spec.Edition == "express" { - unzipApex += dbcommons.ChownApex - } - - // Unzip /opt/oracle/oradata/apex-latest.zip to /opt/oracle/oradata/${ORACLE_SID^^} - out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", - unzipApex) - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - log.Info(" UnzipApex Output : \n" + out) - - if strings.Contains(out, "apex-latest.zip not found") { - eventReason := "Waiting" - eventMsg := "apex-latest.zip doesn't exist in the location /opt/oracle/oradata/" - r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) - return requeueY - } - - // Install APEX - out, err = dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf(dbcommons.InstallApex, dbcommons.SQLPlusCLI)) - if err != nil { - log.Info(err.Error()) - } - log.Info(" InstallApex Output : \n" + out) - - if strings.Contains(out, "Apex Folder doesn't exist") { - eventReason := "Waiting" - eventMsg := "apex Folder doesn't exist in the location /opt/oracle/oradata/" + strings.ToUpper(m.Spec.Sid) - r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) - return requeueY - } - - out, err = dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf("echo -e \"%s\" | %s", dbcommons.IsApexInstalled, dbcommons.SQLPlusCLI)) - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - log.Info("IsApexInstalled Output: \n" + out) - - apexInstalled := "APEXVERSION:" - if !strings.Contains(out, apexInstalled) { - return requeueY - } - - m.Status.ApexInstalled = true - // Make sure m.Status.ApexInstalled is set to true . - for i := 0; i < 10; i++ { - err = r.Status().Update(ctx, m) - if err != nil { - log.Info(err.Error() + "\n updating m.Status.ApexInstalled = true") - time.Sleep(5 * time.Second) - continue - } - break - } - - return requeueN -} - -//############################################################################# -// Uninstall APEX from CDB -//############################################################################# -func (r *SingleInstanceDatabaseReconciler) uninstallApex(m *dbapi.SingleInstanceDatabase, - readyPod corev1.Pod, ctx context.Context, req ctrl.Request) ctrl.Result { - log := r.Log.WithValues("uninstallApex", req.NamespacedName) - - // No APEX for Pre-built db - if m.Spec.Image.PrebuiltDB { - return requeueN - } - - if m.Spec.InstallApex || !m.Status.ApexInstalled { - return requeueN - } - - m.Status.Status = dbcommons.StatusUpdating - r.Status().Update(ctx, m) - eventReason := "Uninstalling Apex" - eventMsg := "Waiting for Apex Uninstallation to complete" - r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) - - // Uninstall APEX - out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf(dbcommons.UninstallApex, dbcommons.SQLPlusCLI)) - if err != nil { - log.Info(err.Error()) - if !strings.Contains(err.Error(), "catcon.pl: completed") { - return requeueY - } - } - log.Info(" UninstallApex Output : \n" + out) - - if strings.Contains(out, "Apex Folder doesn't exist") { - eventReason := "Waiting" - eventMsg := "apex Folder doesn't exist in the location /opt/oracle/oradata/" + strings.ToUpper(m.Spec.Sid) - r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) - return requeueY - } - - m.Status.ApexInstalled = false - // Make sure m.Status.ApexInstalled is set to false. - for i := 0; i < 10; i++ { - err = r.Status().Update(ctx, m) - if err != nil { - log.Info(err.Error() + "\n updating m.Status.ApexInstalled = false") - time.Sleep(5 * time.Second) - continue - } - break - } - r.updateORDSStatus(m, true, ctx, req) - return requeueN -} - //############################################################################# // Update ORDS Status //############################################################################# -func (r *SingleInstanceDatabaseReconciler) updateORDSStatus(m *dbapi.SingleInstanceDatabase, updateApexStatus bool, ctx context.Context, req ctrl.Request) { +func (r *SingleInstanceDatabaseReconciler) updateORDSStatus(m *dbapi.SingleInstanceDatabase, ctx context.Context, req ctrl.Request) { if m.Status.OrdsReference == "" { return @@ -1913,13 +1756,8 @@ func (r *SingleInstanceDatabaseReconciler) updateORDSStatus(m *dbapi.SingleInsta if err != nil { return } - if updateApexStatus && n.Status.ApexConfigured && !m.Status.ApexInstalled { - n.Status.ApexConfigured = false - r.Status().Update(ctx, n) - n.Spec.ApexPassword.SecretName = "" - r.Update(ctx, n) - } - if !updateApexStatus && n.Status.OrdsInstalled { + + if n.Status.OrdsInstalled { // Update Status to Healthy/Unhealthy when SIDB turns Healthy/Unhealthy after ORDS is Installed n.Status.Status = m.Status.Status r.Status().Update(ctx, n) From f7c16a54ba4f8e76c57ef2b5070bf7409ebf2dae Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Fri, 29 Apr 2022 06:14:47 +0530 Subject: [PATCH 291/628] Fix missing status --- apis/database/v1alpha1/oraclerestdataservice_types.go | 1 - controllers/database/oraclerestdataservice_controller.go | 6 +++--- 2 files changed, 3 insertions(+), 4 deletions(-) diff --git a/apis/database/v1alpha1/oraclerestdataservice_types.go b/apis/database/v1alpha1/oraclerestdataservice_types.go index 2a29f86e..0c90e2b3 100644 --- a/apis/database/v1alpha1/oraclerestdataservice_types.go +++ b/apis/database/v1alpha1/oraclerestdataservice_types.go @@ -113,7 +113,6 @@ type OracleRestDataServiceStatus struct { ApexConfigured bool `json:"apexConfigured,omitempty"` ApxeUrl string `json:"apexUrl,omitempty"` CommonUsersCreated bool `json:"commonUsersCreated,omitempty"` - OrdsSetupCompleted bool `json:"ordsSetupCompleted,omitempty"` Replicas int `json:"replicas,omitempty"` Image OracleRestDataServiceImage `json:"image,omitempty"` diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 7352f0dd..f2b04047 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -375,7 +375,7 @@ func (r *OracleRestDataServiceReconciler) checkHealthStatus(m *dbapi.OracleRestD if strings.Contains(out, "HTTP/1.1 200 OK") || strings.Contains(strings.ToUpper(err.Error()), "HTTP/1.1 200 OK") { m.Status.Status = dbcommons.StatusReady - m.Status.OrdsSetupCompleted = true + m.Status.OrdsInstalled = true } else { m.Status.Status = dbcommons.StatusNotReady return requeueY, readyPod @@ -688,7 +688,7 @@ func (r *OracleRestDataServiceReconciler) createSVC(ctx context.Context, req ctr if m.Spec.LoadBalancer { if len(svc.Status.LoadBalancer.Ingress) > 0 { m.Status.DatabaseApiUrl = "https://" + svc.Status.LoadBalancer.Ingress[0].IP + ":" + - fmt.Sprint(svc.Spec.Ports[0].Port) + "/ords/"+n.Status.Pdbname+"_/db-api/stable/" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/ords/"+n.Status.Pdbname+"/_/db-api/stable/" m.Status.ServiceIP = svc.Status.LoadBalancer.Ingress[0].IP m.Status.DatabaseActionsUrl = "https://" + svc.Status.LoadBalancer.Ingress[0].IP + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/ords/sql-developer" @@ -705,7 +705,7 @@ func (r *OracleRestDataServiceReconciler) createSVC(ctx context.Context, req ctr if nodeip != "" { m.Status.ServiceIP = nodeip m.Status.DatabaseApiUrl = "https://" + nodeip + ":" + fmt.Sprint(svc.Spec.Ports[0].NodePort) + - "/ords/"+n.Status.Pdbname+"_/db-api/stable/" + "/ords/"+n.Status.Pdbname+"/_/db-api/stable/" m.Status.DatabaseActionsUrl = "https://" + nodeip + ":" + fmt.Sprint(svc.Spec.Ports[0].NodePort) + "/ords/sql-developer" if m.Status.ApexConfigured { From 162771f4c556709c7247954535985c2d01906832 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Fri, 29 Apr 2022 06:51:30 +0530 Subject: [PATCH 292/628] Fix ORDS uninstall --- commons/database/constants.go | 13 ++++++------- config/samples/sidb/oraclerestdataservice_apex.yaml | 2 +- 2 files changed, 7 insertions(+), 8 deletions(-) diff --git a/commons/database/constants.go b/commons/database/constants.go index 1c50ae81..699793c9 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -46,8 +46,6 @@ const DBA_GUID int64 = 54322 const SQLPlusCLI string = "sqlplus -s / as sysdba" -const SQLPlusRemoteCLI string = "sqlplus -s sys/%[1]s@%[2]s as sysdba" - const NoCloneRef string = "Unavailable" const GetVersionSQL string = "SELECT VERSION_FULL FROM V\\$INSTANCE;" @@ -316,12 +314,13 @@ const InitORDSCMD string = "if [ -f $ORDS_HOME/config/ords/defaults.xml ]; then "\nrm -f sqladmin.passwd" + "\numask 022" -const GetSessionInfoSQL string = "select s.sid || ',' || s.serial# as Info FROM v\\$session s, v\\$process p WHERE s.username = 'ORDS_PUBLIC_USER' AND p.addr(+) = s.paddr;" +const GetSessionInfoSQL string = "select s.sid || ',' || s.serial# as Info FROM v\\$session s, v\\$process p "+ + "WHERE (s.username = 'ORDS_PUBLIC_USER' or s.username = 'C##_DBAPI_PDB_ADMIN' ) AND p.addr(+) = s.paddr;" const KillSessionSQL string = "alter system kill session '%[1]s';" -const DropAdminUsersSQL string = "drop user C##DBAPI_CDB_ADMIN;" + - "\ndrop user C##_DBAPI_PDB_ADMIN;" +const DropAdminUsersSQL string = "drop user C##DBAPI_CDB_ADMIN cascade;" + + "\ndrop user C##_DBAPI_PDB_ADMIN cascade;" const UninstallORDSCMD string = "\numask 177" + "\necho -e \"1\n${ORACLE_HOST}\n${ORACLE_PORT}\n1\n${ORACLE_SERVICE}\nsys\n%[1]s\n%[1]s\n1\" > ords.cred" + @@ -412,8 +411,8 @@ const InstallApexRemote string = "if [ -e ${ORDS_HOME}/config/apex/apxsilentins. const InstallApexInContainer string = "cd ${ORDS_HOME}/config/apex/ && echo -e \"@apxsilentins.sql SYSAUX SYSAUX TEMP /i/ %[1]s %[1]s %[1]s %[1]s;\n"+ "@apex_rest_config_core.sql;\n" + - //"exec APEX_UTIL.set_workspace(p_workspace => 'INTERNAL');\n" + - //"exec APEX_UTIL.EDIT_USER(p_user_id => APEX_UTIL.GET_USER_ID('ADMIN'), p_user_name => 'ADMIN', p_change_password_on_first_use => 'Y');\n" + + "exec APEX_UTIL.set_workspace(p_workspace => 'INTERNAL');\n" + + "exec APEX_UTIL.EDIT_USER(p_user_id => APEX_UTIL.GET_USER_ID('ADMIN'), p_user_name => 'ADMIN', p_change_password_on_first_use => 'Y');\n" + "\" | sqlplus -s sys/%[2]s@${ORACLE_HOST}:${ORACLE_PORT}/%[3]s as sysdba;" const IsApexInstalled string = "echo -e \"select 'APEXVERSION:'||version as version FROM DBA_REGISTRY WHERE COMP_ID='APEX';\"" + diff --git a/config/samples/sidb/oraclerestdataservice_apex.yaml b/config/samples/sidb/oraclerestdataservice_apex.yaml index 372f4a32..0f6fe8e4 100644 --- a/config/samples/sidb/oraclerestdataservice_apex.yaml +++ b/config/samples/sidb/oraclerestdataservice_apex.yaml @@ -19,7 +19,7 @@ metadata: name: apex-secret type: Opaque stringData: - oracle_pwd: "ChangeOnApexInstall_1" + oracle_pwd: "ChangeOnInstall_2" --- From 10da97a9d1a2525d3451f42f11a5980dbc513533 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Fri, 29 Apr 2022 09:26:03 +0530 Subject: [PATCH 293/628] Remove commented code --- .../oraclerestdataservice_controller.go | 31 ------------------- 1 file changed, 31 deletions(-) diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index f2b04047..b4c4515f 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -214,10 +214,6 @@ func (r *OracleRestDataServiceReconciler) validate(m *dbapi.OracleRestDataServic if m.Status.LoadBalancer != "" && m.Status.LoadBalancer != strconv.FormatBool(m.Spec.LoadBalancer) { eventMsgs = append(eventMsgs, "service patching is not avaiable currently") } - //Not needed as apex will be installed from ORDS now - /* if !n.Status.ApexInstalled && m.Spec.ApexPassword.SecretName != "" { - eventMsgs = append(eventMsgs, "apex is not installed yet") - } */ if m.Status.Image.PullFrom != "" && m.Status.Image != m.Spec.Image { eventMsgs = append(eventMsgs, "image patching is not avaiable currently") } @@ -314,33 +310,6 @@ func (r *OracleRestDataServiceReconciler) validateSIDBReadiness(m *dbapi.OracleR } -//##################################################################################################### -// Validate Readiness of the ORDS pods -//##################################################################################################### -func (r *OracleRestDataServiceReconciler) validateORDSReadiness(m *dbapi.OracleRestDataService, ctx context.Context, req ctrl.Request) (ctrl.Result, corev1.Pod) { - log := r.Log.WithValues("validateORDSReadiness", req.NamespacedName) - - // ## FETCH THE ORDS REPLICAS . - ordsReadyPod, _, _, _, err := dbcommons.FindPods(r, m.Spec.Image.Version, - m.Spec.Image.PullFrom, m.Name, m.Namespace, ctx, req) - if err != nil { - log.Error(err, err.Error()) - return requeueY, ordsReadyPod - } - - if ordsReadyPod.Name == "" { - eventReason := "Waiting" - eventMsg := "waiting for " + m.Name + " to be Ready" - r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) - // Logging the event messages - log.Info(eventMsg) - return requeueY, ordsReadyPod - } - - return requeueN, ordsReadyPod - -} - //##################################################################################################### // Check ORDS Health Status //##################################################################################################### From 04bc64c5de4cd88c32c1a44feec9a2a013fb4db6 Mon Sep 17 00:00:00 2001 From: Mohammed Yunus Date: Fri, 29 Apr 2022 04:30:43 +0000 Subject: [PATCH 294/628] Update README.md --- docs/sidb/README.md | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index b5e288a1..40eeca1e 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -2,15 +2,14 @@ Oracle Database Operator for Kubernetes (the operator) includes the Single Instance Database Controller that enables provisioning, cloning, and patching of Oracle Single Instance Databases on Kubernetes. The following sections explain the setup and functionality of the operator -- [Managing Oracle Single Instance Databases with Oracle Database Operator for Kubernetes](#managing-oracle-single-instance-databases-with-oracle-database-operator-for-kubernetes) - - [Prerequisites](#prerequisites) - - [Kind SingleInstanceDatabase Resource](#kind-singleinstancedatabase-resource) - - [Provision New Database](#provision-new-database) - - [Clone Existing Database](#clone-existing-database) - - [Patch/Rollback Database](#patchrollback-database) - - [Kind OracleRestDataService](#kind-oraclerestdataservice) - - [REST Enable Database](#rest-enable-database) - - [Performing maintenance operations](#performing-maintenance-operations) + * [Prerequisites](#prerequisites) + * [Kind SingleInstanceDatabase Resource](#kind-singleinstancedatabase-resource) + * [Provision New Database](#provision-new-database) + * [Clone Existing Database](#clone-existing-database) + * [Patch/Rollback Database](#patchrollback-database) + * [Kind OracleRestDataService](#kind-oraclerestdataservice) + * [REST Enable Database](#rest-enable-database) + * [Performing maintenance operations](#performing-maintenance-operations) ## Prerequisites From 629f197e58dc57261c16abb66f2ffae6ae6fd0b7 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Fri, 29 Apr 2022 15:13:58 +0530 Subject: [PATCH 295/628] Fix namespace in config xmls --- config/samples/sidb/oraclerestdataservice_apex.yaml | 2 ++ config/samples/sidb/oraclerestdataservice_create.yaml | 1 + config/samples/sidb/singleinstancedatabase_create.yaml | 1 + config/samples/sidb/singleinstancedatabase_express.yaml | 7 ++++--- config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml | 1 + controllers/database/singleinstancedatabase_controller.go | 5 +++-- 6 files changed, 12 insertions(+), 5 deletions(-) diff --git a/config/samples/sidb/oraclerestdataservice_apex.yaml b/config/samples/sidb/oraclerestdataservice_apex.yaml index 0f6fe8e4..97a19c7c 100644 --- a/config/samples/sidb/oraclerestdataservice_apex.yaml +++ b/config/samples/sidb/oraclerestdataservice_apex.yaml @@ -7,6 +7,7 @@ apiVersion: v1 kind: Secret metadata: name: ords-secret + namespace: default type: Opaque stringData: oracle_pwd: "ChangeOnInstall_1" @@ -17,6 +18,7 @@ apiVersion: v1 kind: Secret metadata: name: apex-secret + namespace: default type: Opaque stringData: oracle_pwd: "ChangeOnInstall_2" diff --git a/config/samples/sidb/oraclerestdataservice_create.yaml b/config/samples/sidb/oraclerestdataservice_create.yaml index e352f800..7440c8d1 100644 --- a/config/samples/sidb/oraclerestdataservice_create.yaml +++ b/config/samples/sidb/oraclerestdataservice_create.yaml @@ -7,6 +7,7 @@ apiVersion: v1 kind: Secret metadata: name: ords-secret + namespace: default type: Opaque stringData: oracle_pwd: "ChangeOnInstall_1" diff --git a/config/samples/sidb/singleinstancedatabase_create.yaml b/config/samples/sidb/singleinstancedatabase_create.yaml index be8d7630..81fc9a3e 100644 --- a/config/samples/sidb/singleinstancedatabase_create.yaml +++ b/config/samples/sidb/singleinstancedatabase_create.yaml @@ -7,6 +7,7 @@ apiVersion: v1 kind: Secret metadata: name: db-admin-secret + namespace: default type: Opaque stringData: oracle_pwd: "ChangeOnInstall_1" diff --git a/config/samples/sidb/singleinstancedatabase_express.yaml b/config/samples/sidb/singleinstancedatabase_express.yaml index cc8da754..159ac91c 100644 --- a/config/samples/sidb/singleinstancedatabase_express.yaml +++ b/config/samples/sidb/singleinstancedatabase_express.yaml @@ -6,7 +6,8 @@ apiVersion: v1 kind: Secret metadata: - name: xe-admin-secret + name: xedb-admin-secret + namespace: default type: Opaque stringData: oracle_pwd: "ChangeOnInstall_1" @@ -16,7 +17,7 @@ stringData: apiVersion: database.oracle.com/v1alpha1 kind: SingleInstanceDatabase metadata: - name: xedb-minikube-sample + name: xedb-sample namespace: default spec: @@ -29,7 +30,7 @@ spec: ## Secret containing SIDB password mapped to secretKey ## This secret will be deleted after creation of the database unless keepSecret is set to true adminPassword: - secretName: xe-admin-secret + secretName: xedb-admin-secret secretKey: oracle_pwd keepSecret: true diff --git a/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml b/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml index db2f5034..2ed4c8c7 100644 --- a/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml +++ b/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml @@ -7,6 +7,7 @@ apiVersion: v1 kind: Secret metadata: name: prebuiltdb-admin-secret + namespace: default type: Opaque stringData: oracle_pwd: "ChangeOnInstall_1" diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index b415938d..c3338c22 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -235,6 +235,9 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct func (r *SingleInstanceDatabaseReconciler) updateReconcileStatus(m *dbapi.SingleInstanceDatabase, ctx context.Context, result *ctrl.Result, err *error, blocked *bool, completed *bool) { + // Always refresh status before a reconcile + defer r.Status().Update(ctx, m) + errMsg := func() string { if *err != nil { return (*err).Error() @@ -285,8 +288,6 @@ func (r *SingleInstanceDatabaseReconciler) updateReconcileStatus(m *dbapi.Single meta.RemoveStatusCondition(&m.Status.Conditions, condition.Type) } meta.SetStatusCondition(&m.Status.Conditions, condition) - // Always refresh status before a reconcile - r.Status().Update(ctx, m) } //############################################################################# From dfdb28cd15106932e842bf7e48fd6eb2489c6863 Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Fri, 29 Apr 2022 15:20:58 +0530 Subject: [PATCH 296/628] Readme changes: ORDS with APEX Signed-off-by: abhisbyk --- docs/sidb/README.md | 174 +++++++++++++------------------------------- 1 file changed, 51 insertions(+), 123 deletions(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 40eeca1e..9b04bbf8 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -109,7 +109,7 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI ## Provision New Database - - Easily provision a new database instance on **minikube** using [singleinstancedatabase_minikube.yaml](../../config/samples/sidb/singleinstancedatabase_minikube.yaml). + - Easily provision a new database instance on the Kubernetes cluster using [singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml). Sign into [Oracle Container Registry](https://container-registry.oracle.com/ords/f?p=113:4:7154182141811:::4:P4_REPOSITORY,AI_REPOSITORY,AI_REPOSITORY_NAME,P4_REPOSITORY_NAME,P4_EULA_ID,P4_BUSINESS_AREA_ID:9,9,Oracle%20Database%20Enterprise%20Edition,Oracle%20Database%20Enterprise%20Edition,1,0&cs=3Y_90hkCQLfJzrvTLiEipIGgWGUytfrtAPuHFocuWd0NDSacbBPlamohfLuiJA-bAsVL6Z_yKEMsTbb52bm6IRA) and accept the license agreement for the Database image, ignore if you have accepted already. @@ -121,10 +121,10 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI secret/oracle-container-registry-secret created ``` - Now, Easily provision a new database instance on minikube by using following one command. + Now, easily provision a new database instance on the cluster by using the following command. ```sh - $ kubectl create -f singleinstancedatabase_minikube.yaml + $ kubectl create -f singleinstancedatabase_create.yaml singleinstancedatabase.database.oracle.com/sidb-sample created ``` @@ -137,14 +137,16 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI singleinstancedatabase.database.oracle.com/sidb-sample created ``` - **NOTE:** Make sure you have created the required `.spec.adminPassword` [secret](https://kubernetes.io/docs/tasks/configmap-secret/managing-secret-using-kubectl/) and `.spec.persistence` [persistent volume](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) + **NOTE:** + - Make sure you have created the required `.spec.adminPassword` [secret](https://kubernetes.io/docs/tasks/configmap-secret/managing-secret-using-kubectl/) and `.spec.persistence` [persistent volume](https://kubernetes.io/docs/concepts/storage/persistent-volumes/). + - For the ease of use, the storage class **oci-bv** is specified in the [singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml). This storage class facilitates dynamic provisioning of the OCI block volume for the persistent storage of the database. For other cloud providers, there dynamic provisioning storage class can be used similarly. * ### Provision a Pre-built Database Provision a new Pre-built database instance by specifying appropriate values for the attributes in the the example `.yaml` file, and running the following command: ```sh - $ kubectl create -f singleinstancedatabase_prov_prebuilt_db.yaml + $ kubectl create -f singleinstancedatabase_prebuiltdb.yaml singleinstancedatabase.database.oracle.com/sidb-sample created ``` @@ -155,8 +157,8 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI Some limitations are listed as follows: - External volume can not be used for database persistence (as data files are inside the image itself). - Only the single replica mode (i.e. replicas=1) can be used. + - External volume can not be used for database persistence (as data files are inside the image itself). + - Only the single replica mode (i.e. replicas=1) can be used. * ### Creation Status @@ -219,8 +221,8 @@ $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.status}" Multiple database pod replicas can be provisioned when the persistent volume access mode is ReadWriteMany. Database is open and mounted by one of the replicas. Other replicas will have instance started but not mounted and serve to provide quick cold fail-over in case the active pod dies. Update the replica attribute in the .yaml and apply using the kubectl apply command or edit/patch commands - Note: This functionality requires the [K8s extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/k8s) - Pre-built images from container-registry.oracle.com include the K8s extension + **Note:** This functionality requires the [K8s extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/k8s) + extended images from container-registry.oracle.com include the K8s extension * ### Patch Attributes @@ -259,30 +261,30 @@ $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.status}" Quickly create copies of your existing database using this cloning functionality. A cloned database is an exact, block-for-block copy of the source database. This is much faster than creating a fresh new database and copying over the data. - To clone, specify the source database reference as value for the cloneFrom attribute in the sample .yaml. - The source database must have archiveLog mode set to true. + To clone, specify the source database reference as value for the cloneFrom attribute in the sample [singleinstancedatabase_clone.yaml](../../config/samples/sidb/singleinstancedatabase_clone.yaml) file. + **The source database must have archiveLog mode set to true.** ```sh - $ grep 'cloneFrom:' singleinstancedatabase.yaml + $ grep 'cloneFrom:' singleinstancedatabase_clone.yaml cloneFrom: "sidb-sample" - $ kubectl create -f singleinstancedatabase.yaml + $ kubectl create -f singleinstancedatabase_clone.yaml singleinstancedatabase.database.oracle.com/sidb-sample-clone created ``` - Note: The clone database can specify a database image different from the source database. In such cases, cloning is supported only between databases of the same major release. + **Note:** The clone database can specify a database image different from the source database. In such cases, cloning is supported only between databases of the same major release. ## Patch/Rollback Database Databases running in your cluster and managed by this operator can be patched or rolled back between release updates of the same major release. To patch databases, specify an image of the higher release update, and to roll back, specify an image of the lower release update. - Patched Oracle Docker images can be built using this [patching extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/patching) + Patched Oracle Docker images can be built using this [patching extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/patching). * ### Patch existing Database - Edit and apply the `singleinstancedatabase.yaml` file of the database resource/object by specifying a new release update for image attributes or run the following command. + Edit and apply the [singleinstancedatabase_patch.yaml](../../config/samples/sidb/singleinstancedatabase_patch.yaml) file of the database resource/object by specifying a new release update for image attributes or run the following command. ```sh kubectl --type=merge -p '{"spec":{"image":{"pullFrom":"patched-image:tag","pullSecrets":"pull-secret"}}}' patch singleinstancedatabase sidb-sample @@ -340,8 +342,9 @@ The Oracle Database Operator creates the OracleRestDataService (ORDS) kind as a ```sh $ kubectl get oraclerestdataservice ords-sample - NAME STATUS DATABASE DATABASE API URL DATABASE ACTIONS URL - ords-sample Healthy sidb-sample https://10.0.25.54:8443/ords/ https://10.0.25.54:8443/ords/sql-developer + NAME STATUS DATABASE DATABASE API URL DATABASE ACTIONS URL APEX URL + ords-sample Healthy sidb-sample https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/ https://10.0.25.54:8443/ords/sql-developer https://10.0.25.54:8443/ords/ORCLPDB1/apex + ``` * ### Detailed Status @@ -358,9 +361,10 @@ The Oracle Database Operator creates the OracleRestDataService (ORDS) kind as a Metadata: ... Spec: ... Status: - Cluster Db API URL: https://ords21c-1.default:8443/ords/ + Cluster Db API URL: https://ords21c-1.default:8443/ords/ORCLPDB1/_/db-api/stable/ Database Actions URL: https://10.0.25.54:8443/ords/sql-developer - Database API URL: https://10.0.25.54:8443/ords/ + Database API URL: https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/ + Apex URL: https://10.0.25.54:8443/ords/ORCLPDB1/apex Database Ref: sidb21c-1 Image: Pull From: ... @@ -403,7 +407,7 @@ The Oracle Database Operator creates the OracleRestDataService (ORDS) kind as a ```sh $ kubectl get oraclerestdataservice/ords-sample --template={{.status.databaseApiUrl}} - https://10.0.25.54:8443/ords/ + https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/ ``` All the REST Endpoints can be found at @@ -429,90 +433,28 @@ The Oracle Database Operator creates the OracleRestDataService (ORDS) kind as a Note : Browser may not prompt for credentials while accessing certain REST Endpoints and in such case one can use clients like curl and pass credentials while calling REST Endpoints . -* #### Some Usecases - -* ##### PDB Lifecycle Management - - * To Enable PDB Lifecycle Management, Grant SYSDBA to CDB Administrator - +* #### Some use cases + Some generic use cases for the Database API are as follows: +* ##### Getting all Database components ```sh - $ echo "GRANT SYSDBA TO C##DBAPI_CDB_ADMIN CONTAINER = ALL;" | sqlplus -s sys/@ as sysdba - - Grant succeeded. + curl -s -k -X GET -u 'ORDS_PUBLIC_USER:<.spec.ordsPassword>' https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/database/components/ | python -m json.tool + ``` +* ##### Getting all Database users + ```sh + curl -s -k -X GET -u 'ORDS_PUBLIC_USER:<.spec.ordsPassword>' https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/database/security/users/ | python -m json.tool + ``` +* ##### Getting all tablespaces + ```sh + curl -s -k -X GET -u 'ORDS_PUBLIC_USER:<.spec.ordsPassword>' https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/database/storage/tablespaces/ | python -m json.tool + ``` +* ##### Getting all Database parameters + ```sh + curl -s -k -X GET -u 'ORDS_PUBLIC_USER:<.spec.ordsPassword>' https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/database/parameters/ | python -m json.tool + ``` +* ##### Getting all feature usage statitics + ```sh + curl -s -k -X GET -u 'ORDS_PUBLIC_USER:<.spec.ordsPassword>' https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/database/feature_usage/ | python -m json.tool ``` - - The Oracle REST Data Services (ORDS) database API allows us to manage the lifecycle of PDBs via REST web service calls. - Few APIs : - * List PDB's - - ```sh - `$ curl -s -k -X GET -u 'ORDS_PUBLIC_USER:<.spec.ordsPassword>' https://10.0.25.54:8443/ords/_/db-api/stable/database/pdbs/ | python -m json.tool` - ``` - - * Create PDB - - Create a file called "pdbsample.json" with the following contents. - - ```sh - { "method": "CREATE", - "adminName": "pdbsample_admin", - "adminPwd": "", - "pdb_name": "pdbsample", - "fileNameConversions": "('/opt/oracle/oradata/<.spec.databaseRef.sid>/pdbseed/', '/opt/oracle/oradata/<.spec.databaseRef.sid>/pdbsample/')", - "reuseTempFile": true, - "totalSize": "10G", - "tempSize": "100M", - "getScript": false - } - ``` - - Execute the follwing API to run the above json script to create pdb. - - ```sh - $ curl -k --request POST --url https://10.0.25.54:8443/ords/_/db-api/latest/database/pdbs/ \ - --user 'ORDS_PUBLIC_USER:<.spec.ordsPassword>' \ - --header 'content-type: application/json' \ - --data @pdbsample.json - ``` - - * Open/Close PDB - - ```sh - $ curl -k --request POST --url https://10.0.25.54:8443/ords/_/db-api/latest/database/pdbs/pdbsample/status \ - --user 'ORDS_PUBLIC_USER:<.spec.ordsPassword>'\ - --header 'content-type: application/json' \ - --data ' { - "state": "OPEN/CLOSE", - "modifyOption": "NORMAL" - } ' - ``` - - * Clone PDB - - Create a file called "pdbclone.json" with the following contents. - - ```sh - { - "method": "CLONE", - "clonePDBName": "pdbclone", - "fileNameConversions": "('/opt/oracle/oradata/<.spec.databaseRef.sid>/pdbsample/', '/opt/oracle/oradata/<.spec.databaseRef.sid>/pdbclone/')", - "unlimitedStorage": true, - "reuseTempFile": true, - "totalSize": "UNLIMITED", - "tempSize": "UNLIMITED" - } - ``` - - Execute the follwing API to run the above json script to clone pdb from pdbsample. - - ```sh - $ curl -k --request POST --url https://10.0.25.54:8443/ords/_/db-api/latest/database/pdbs/pdbsample/ \ - --user 'ORDS_PUBLIC_USER:<.spec.ordsPassword>' \ - --header 'content-type: application/json' \ - --data @pdbclone.json - ``` - - More REST APIs for PDB Lifecycle Management can be found at [https://docs.oracle.com/en/database/oracle/oracle-database/21/dbrst/api-pluggable-database-lifecycle-management.html](https://docs.oracle.com/en/database/oracle/oracle-database/21/dbrst/api-pluggable-database-lifecycle-management.html) * #### REST Enabled SQL @@ -566,7 +508,7 @@ The Oracle Database Operator creates the OracleRestDataService (ORDS) kind as a * ### Database Actions - Database Actions is a web-based interface that uses Oracle REST Data Services to provide development, data tools, administration and monitoring features for Oracle Database . + Database Actions is a web-based interface that uses Oracle REST Data Services to provide development, data tools, administration and monitoring features for Oracle Database. * To use Database Actions, one must sign in as a database user whose schema has been REST-enabled. * This can be done by specifying appropriate values for the `.spec.restEnableSchemas` attributes details in the sample yaml [config/samples/sidb/oraclerestdataservice.yaml](config/samples/sidb/oraclerestdataservice.yaml) which are needed for authorising Database Actions. @@ -600,25 +542,11 @@ The Oracle Database Operator creates the OracleRestDataService (ORDS) kind as a Using APEX, developers can quickly develop and deploy compelling apps that solve real problems and provide immediate value. Developers won't need to be an expert in a vast array of technologies to deliver sophisticated solutions. Focus on solving the problem and let APEX take care of the rest. - **Download APEX:** - * Download latest version of apex using - * Copy apex-latest.zip file to the location '/opt/oracle/oradata' of SingleInstanceDatabase pod . - * Use `kubectl cp :/opt/oracle/oradata` to copy apex-latest.zip - - **Install APEX:** - * Set `.spec.installApex` to true in [config/samples/sidb/singleinstancedatabase.yaml](config/samples/sidb/singleinstancedatabase.yaml) - * Status of SIDB turns to 'Updating' during apex installation and turns 'Healthy' after successful installation. You can also check status using below cmd - - ```sh - $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath=[{.status.apexInstalled}]" - - [true] - ``` - - * To access APEX , You need to configure APEX with ORDS + To access APEX, You need to configure APEX with ORDS. The following section will explain configuring APEX with ORDS in details: **Configure APEX with ORDS:** - * Set `.spec.apexPassword.secretName` to a non-null string in [config/samples/sidb/oraclerestdataservice.yaml](config/samples/sidb/oraclerestdataservice.yaml) + * For quick provisioning, apply the [config/samples/sidb/oraclerestdataservice_apex.yaml](../../confi/samples/sidb/oraclerestdataservice_apex.yaml) file. First, it creates `ords-secret`, `apex-secret`, and then provision the ORDS configured with Oracle APEX. It uses the ORDS image hosted on the [Oracle Container Registry](https://container-registry.oracle.com/ords/f?p=113:4:113387942129427:::4:P4_REPOSITORY,AI_REPOSITORY,AI_REPOSITORY_NAME,P4_REPOSITORY_NAME,P4_EULA_ID,P4_BUSINESS_AREA_ID:1183,1183,Oracle%20REST%20Data%20Services%20(ORDS)%20with%20Application%20Express,Oracle%20REST%20Data%20Services%20(ORDS)%20with%20Application%20Express,1,0&cs=3_y-KlneZIxRRfXzerC_0ro7P1MGh-B_9lTEQObVTdoQCWkmsQ3lHpFs90Z8QFheteVQEzPvtUVHEQAqqXegYbA). + * On the other hand, to provision ORDS step by step, set `.spec.apexPassword.secretName` to a non-null string in [config/samples/sidb/oraclerestdataservice.yaml](../../config/samples/sidb/oraclerestdataservice.yaml) * This is used as a common password for APEX_PUBLIC_USER, APEX_REST_PUBLIC_USER, APEX_LISTENER and Apex administrator (username: ADMIN) mapped to secretKey * Status of ORDS turns to 'Updating' during apex configuration and turns 'Healthy' after successful configuration. You can also check status using below cmd @@ -635,7 +563,7 @@ The Oracle Database Operator creates the OracleRestDataService (ORDS) kind as a ```sh $ kubectl get oraclerestdataservice/ords-sample --template={{.status.databaseApiUrl}} - https://10.0.25.54:8888/ords/ + https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/ ``` Sign in to Administration servies using \ @@ -654,7 +582,7 @@ The Oracle Database Operator creates the OracleRestDataService (ORDS) kind as a * ### Patch Attributes - The following attributes cannot be patched post SingleInstanceDatabase instance Creation : databaseRef, loadBalancer, image, ordsPassword, adminPassword, apexPassword. + * The following attributes cannot be patched post SingleInstanceDatabase instance Creation : databaseRef, loadBalancer, image, ordsPassword, adminPassword, apexPassword. * A schema can be rest enabled or disabled by setting the `.spec.restEnableSchemas[].enable` to true or false respectively in ords sample .yaml file and apply using the kubectl apply command or edit/patch commands. This requires `.spec.ordsPassword` secret. From 73b1ee5ccfc3985d8316a80ef8ad4b5eb41efede Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Fri, 29 Apr 2022 15:21:34 +0530 Subject: [PATCH 297/628] Fix status string --- commons/database/constants.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/commons/database/constants.go b/commons/database/constants.go index 699793c9..4d5fae8c 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -370,7 +370,7 @@ const StatusError string = "Error" const StatusUnavailable string = "Unavailable" -const ValueUnavailable string = "Unknown" +const ValueUnavailable string = "Unavailable" const NoExternalIp string = "Node ExternalIP unavailable" From 6cda2eac284a9cc311735f1db6543437751f152e Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Sat, 30 Apr 2022 07:21:53 +0530 Subject: [PATCH 298/628] Fix deletion policy --- config/samples/sidb/oraclerestdataservice.yaml | 1 + .../sidb/oraclerestdataservice_apex.yaml | 3 ++- .../sidb/oraclerestdataservice_create.yaml | 1 + .../oraclerestdataservice_controller.go | 18 +++++++++++++----- .../singleinstancedatabase_controller.go | 7 ++++--- 5 files changed, 21 insertions(+), 9 deletions(-) diff --git a/config/samples/sidb/oraclerestdataservice.yaml b/config/samples/sidb/oraclerestdataservice.yaml index 69eee4b2..72a309ff 100644 --- a/config/samples/sidb/oraclerestdataservice.yaml +++ b/config/samples/sidb/oraclerestdataservice.yaml @@ -6,6 +6,7 @@ apiVersion: database.oracle.com/v1alpha1 kind: OracleRestDataService metadata: name: ords-sample + namespace: default spec: ## Database ref. This can be of kind SingleInstanceDatabase. diff --git a/config/samples/sidb/oraclerestdataservice_apex.yaml b/config/samples/sidb/oraclerestdataservice_apex.yaml index 97a19c7c..9c9734ab 100644 --- a/config/samples/sidb/oraclerestdataservice_apex.yaml +++ b/config/samples/sidb/oraclerestdataservice_apex.yaml @@ -29,10 +29,11 @@ apiVersion: database.oracle.com/v1alpha1 kind: OracleRestDataService metadata: name: ords-sample + namespace: default spec: ## Database ref. This can be of kind SingleInstanceDatabase. - ## Make sure the source database has been created by applying singeinstancedatabase_prebuilt.yaml + ## Make sure the source database has been created by applying singeinstancedatabase_prebuiltdb.yaml databaseRef: "prebuiltdb-sample" ## Secret containing databaseRef password mapped to secretKey. diff --git a/config/samples/sidb/oraclerestdataservice_create.yaml b/config/samples/sidb/oraclerestdataservice_create.yaml index 7440c8d1..2f6d9591 100644 --- a/config/samples/sidb/oraclerestdataservice_create.yaml +++ b/config/samples/sidb/oraclerestdataservice_create.yaml @@ -18,6 +18,7 @@ apiVersion: database.oracle.com/v1alpha1 kind: OracleRestDataService metadata: name: ords-sample + namespace: default spec: ## Database ref. This can be of kind SingleInstanceDatabase. diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index b4c4515f..0efe7ee4 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -109,8 +109,11 @@ func (r *OracleRestDataServiceReconciler) Reconcile(ctx context.Context, req ctr err = r.Get(ctx, types.NamespacedName{Namespace: req.Namespace, Name: oracleRestDataService.Spec.DatabaseRef}, singleInstanceDatabase) if err != nil { if apierrors.IsNotFound(err) { - r.Log.Info("Resource deleted") - return requeueN, nil + eventReason := "Waiting" + eventMsg := "waiting for database " + oracleRestDataService.Spec.DatabaseRef + r.Recorder.Eventf(oracleRestDataService, corev1.EventTypeNormal, eventReason, eventMsg) + r.Log.Info("Resource not found", "DatabaseRef", oracleRestDataService.Spec.DatabaseRef) + return requeueY, nil } return requeueN, err } @@ -158,7 +161,6 @@ func (r *OracleRestDataServiceReconciler) Reconcile(ctx context.Context, req ctr return result, nil } - // Create ORDS Pods result = r.createPods(oracleRestDataService, singleInstanceDatabase, ctx, req) if result.Requeue { @@ -774,7 +776,10 @@ func (r *OracleRestDataServiceReconciler) createPods(m *dbapi.OracleRestDataServ break } r.Log.Info("Deleting Pod : ", "POD.NAME", pod.Name) - err := r.Delete(ctx, &pod, &client.DeleteOptions{}) + var gracePeriodSeconds int64 = 0 + policy := metav1.DeletePropagationForeground + err := r.Delete(ctx, &pod, &client.DeleteOptions{ + GracePeriodSeconds: &gracePeriodSeconds, PropagationPolicy: &policy }) noDeleted += 1 if err != nil { r.Log.Error(err, "Failed to delete existing POD", "POD.Name", pod.Name) @@ -1037,7 +1042,10 @@ func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataS } } // ORDS Needs to be restarted to configure APEX - err = r.Delete(ctx, &ordsReadyPod, &client.DeleteOptions{}) + var gracePeriodSeconds int64 = 0 + policy := metav1.DeletePropagationForeground + err = r.Delete(ctx, &ordsReadyPod, &client.DeleteOptions{ + GracePeriodSeconds: &gracePeriodSeconds, PropagationPolicy: &policy }) if err != nil { r.Log.Error(err, "Failed to delete existing POD", "POD.Name", ordsReadyPod.Name) return requeueY diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index c3338c22..5fd4b5de 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -954,11 +954,12 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex pdbName := strings.ToUpper(m.Spec.Pdbname) sid := m.Spec.Sid if m.Spec.Image.PrebuiltDB { - sid, pdbName, m.Status.Edition = dbcommons.GetSidPdbEdition(r, r.Config, ctx, req) - if sid == "" || pdbName == "" || m.Status.Edition == "" { + edition := "" + sid, pdbName, edition = dbcommons.GetSidPdbEdition(r, r.Config, ctx, req) + if sid == "" || pdbName == "" || edition == "" { return requeueN, nil } - m.Status.Edition = strings.Title(m.Status.Edition) + m.Status.Edition = strings.Title(edition) } if m.Spec.LoadBalancer { From 66a5004f3a4eb9b36f8cb5e5e4c648e1d4838ae0 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Sat, 30 Apr 2022 11:44:13 +0530 Subject: [PATCH 299/628] Add nodeSelector to ORDS yaml --- .../samples/sidb/oraclerestdataservice.yaml | 7 ++++ .../samples/sidb/singleinstancedatabase.yaml | 10 ++--- .../oraclerestdataservice_controller.go | 39 +++++++++++++------ .../singleinstancedatabase_controller.go | 20 +++++++++- 4 files changed, 59 insertions(+), 17 deletions(-) diff --git a/config/samples/sidb/oraclerestdataservice.yaml b/config/samples/sidb/oraclerestdataservice.yaml index 72a309ff..a63abc7a 100644 --- a/config/samples/sidb/oraclerestdataservice.yaml +++ b/config/samples/sidb/oraclerestdataservice.yaml @@ -53,6 +53,13 @@ spec: ## if loadBalService: false, service type = "NodePort". else "LoadBalancer" loadBalancer: false + ## Deploy only on nodes having required labels. Format label_name : label_value + ## The same lables are applied to the created PVC + ## For instance if the Pods and storage need to be restricted to a particular AD + ## Leave commented if there is no such requirement. + # nodeSelector: + # failure-domain.beta.kubernetes.io/zone: PHX-AD-1 + ## PDB Schemas to be ORDS Enabled. ## Schema will be created (if not exists) with username as schema and password as .spec.ordsPassword. restEnableSchemas: diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index 3e08b67e..771d005c 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -76,11 +76,11 @@ spec: loadBalancer: false ## Deploy only on nodes having required labels. Format label_name : label_value - ## Leave empty if there is no such requirement. - ## Uncomment to use - # nodeSelector: - # failure-domain.beta.kubernetes.io/zone: bVCG:PHX-AD-1 - # pool: sidb + ## The same lables are applied to the created PVC + ## For instance if the Pods and storage need to be restricted to a particular AD + ## Leave commented if there is no such requirement. + # nodeSelector: + # failure-domain.beta.kubernetes.io/zone: PHX-AD-1 ## If deploying on OpenShift, change service account name to 'sidb-sa' after you run `$ oc apply -f openshift_rbac.yaml` serviceAccountName: default diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 0efe7ee4..7d8382fc 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -616,6 +616,17 @@ func (r *OracleRestDataServiceReconciler) instantiatePVCSpec(m *dbapi.OracleRest }, }, StorageClassName: &m.Spec.Persistence.StorageClass, + Selector: &metav1.LabelSelector{ + MatchLabels: func() map[string]string { + ns := make(map[string]string) + if len(m.Spec.NodeSelector) != 0 { + for key, value := range m.Spec.NodeSelector { + ns[key] = value + } + } + return ns + }(), + }, }, } // Set SingleInstanceDatabase instance as the owner and controller @@ -866,7 +877,15 @@ func (r *OracleRestDataServiceReconciler) cleanupOracleRestDataService(req ctrl. return err } - if m.Status.OrdsInstalled && readyPod.Name != "" { + if readyPod.Name == "" { + eventReason := "Waiting" + eventMsg := "waiting for " + m.Name + " to be Ready" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + err = errors.New(eventMsg) + return err + } + + if m.Status.OrdsInstalled { // ## FETCH THE SIDB REPLICAS . sidbReadyPod, _, _, _, err := dbcommons.FindPods(r, n.Spec.Image.Version, n.Spec.Image.PullFrom, n.Name, n.Namespace, ctx, req) @@ -911,14 +930,6 @@ func (r *OracleRestDataServiceReconciler) cleanupOracleRestDataService(req ctrl. } log.Info("KillSession Output : " + out) - if readyPod.Name == "" { - eventReason := "Waiting" - eventMsg := "waiting for " + m.Name + " to be Ready" - r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) - err = errors.New(eventMsg) - return err - } - // Fetch admin Password of database to uninstall ORDS adminPasswordSecret := &corev1.Secret{} adminPasswordSecretFound := false @@ -964,6 +975,12 @@ func (r *OracleRestDataServiceReconciler) cleanupOracleRestDataService(req ctrl. log.Info("UninstallORDSCMD Output : " + out) + //Delete ORDS pod + var gracePeriodSeconds int64 = 0 + policy := metav1.DeletePropagationForeground + r.Delete(ctx, &readyPod, &client.DeleteOptions{ + GracePeriodSeconds: &gracePeriodSeconds, PropagationPolicy: &policy }) + //Delete Database Admin Password Secret if !m.Spec.AdminPassword.KeepSecret { err = r.Delete(ctx, adminPasswordSecret, &client.DeleteOptions{}) @@ -1244,13 +1261,13 @@ func (r *OracleRestDataServiceReconciler) restEnableSchemas(m *dbapi.OracleRestD strconv.FormatBool(m.Spec.RestEnableSchemas[i].Enable), urlMappingPattern, m.Spec.RestEnableSchemas[i].Pdb) // Create users,schemas and grant enableORDS for PDB - out, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", + _, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", fmt.Sprintf("echo -e \"%s\" | %s", enableORDSSchema, dbcommons.SQLPlusCLI)) if err != nil { log.Error(err, err.Error()) return requeueY } - log.Info("getOrdsSchemaStatus Output : " + out) + log.Info("REST Enabled", "schema", m.Spec.RestEnableSchemas[i].Schema) } diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 5fd4b5de..625b49b3 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -839,6 +839,17 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePVCSpec(m *dbapi.SingleIns }, }, StorageClassName: &m.Spec.Persistence.StorageClass, + Selector: &metav1.LabelSelector{ + MatchLabels: func() map[string]string { + ns := make(map[string]string) + if len(m.Spec.NodeSelector) != 0 { + for key, value := range m.Spec.NodeSelector { + ns[key] = value + } + } + return ns + }(), + }, }, } // Set SingleInstanceDatabase instance as the owner and controller @@ -1302,7 +1313,14 @@ func (r *SingleInstanceDatabaseReconciler) deletePods(ctx context.Context, req c break } r.Log.Info("Deleting Pod : ", "POD.NAME", availablePod.Name) - err := r.Delete(ctx, &availablePod, &client.DeleteOptions{}) + var delOpts *client.DeleteOptions = &client.DeleteOptions{} + if replicasRequired == 0 { + var gracePeriodSeconds int64 = 0 + policy := metav1.DeletePropagationForeground + delOpts.GracePeriodSeconds = &gracePeriodSeconds + delOpts.PropagationPolicy = &policy + } + err := r.Delete(ctx, &availablePod, delOpts) noDeleted += 1 if err != nil { r.Log.Error(err, "Failed to delete existing POD", "POD.Name", availablePod.Name) From 8bca4db0a1b4a2e06bfa552640276bd93322e144 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Sat, 30 Apr 2022 14:30:01 +0530 Subject: [PATCH 300/628] Fix PVC selector --- .../samples/sidb/oraclerestdataservice.yaml | 2 +- .../sidb/oraclerestdataservice_apex.yaml | 2 +- .../sidb/oraclerestdataservice_create.yaml | 2 +- .../sidb/singleinstancedatabase_express.yaml | 2 +- .../singleinstancedatabase_prebuiltdb.yaml | 2 +- .../oraclerestdataservice_controller.go | 25 ++++--- .../singleinstancedatabase_controller.go | 67 ++++++++++++------- 7 files changed, 62 insertions(+), 40 deletions(-) diff --git a/config/samples/sidb/oraclerestdataservice.yaml b/config/samples/sidb/oraclerestdataservice.yaml index a63abc7a..0e6d66a9 100644 --- a/config/samples/sidb/oraclerestdataservice.yaml +++ b/config/samples/sidb/oraclerestdataservice.yaml @@ -45,7 +45,7 @@ spec: ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. persistence: - size: 100Gi + size: 50Gi storageClass: "oci-bv" accessMode: "ReadWriteOnce" diff --git a/config/samples/sidb/oraclerestdataservice_apex.yaml b/config/samples/sidb/oraclerestdataservice_apex.yaml index 9c9734ab..073fc073 100644 --- a/config/samples/sidb/oraclerestdataservice_apex.yaml +++ b/config/samples/sidb/oraclerestdataservice_apex.yaml @@ -66,7 +66,7 @@ spec: ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. persistence: - size: 100Gi + size: 50Gi storageClass: "oci-bv" accessMode: "ReadWriteOnce" diff --git a/config/samples/sidb/oraclerestdataservice_create.yaml b/config/samples/sidb/oraclerestdataservice_create.yaml index 2f6d9591..dc00429f 100644 --- a/config/samples/sidb/oraclerestdataservice_create.yaml +++ b/config/samples/sidb/oraclerestdataservice_create.yaml @@ -47,7 +47,7 @@ spec: ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. persistence: - size: 100Gi + size: 50Gi storageClass: "oci-bv" accessMode: "ReadWriteOnce" diff --git a/config/samples/sidb/singleinstancedatabase_express.yaml b/config/samples/sidb/singleinstancedatabase_express.yaml index 159ac91c..02840981 100644 --- a/config/samples/sidb/singleinstancedatabase_express.yaml +++ b/config/samples/sidb/singleinstancedatabase_express.yaml @@ -42,6 +42,6 @@ spec: ## size : Minimum size of pvc | class : PVC storage Class ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany persistence: - size: 5Gi + size: 50Gi storageClass: "oci-bv" accessMode: "ReadWriteOnce" diff --git a/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml b/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml index 2ed4c8c7..c22b14f3 100644 --- a/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml +++ b/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml @@ -42,6 +42,6 @@ spec: ## Persistence is optional for prebuilt DB image ## if specified, the prebuilt DB datafiles are copied over to the persistant volume before DB startup #persistence: - # size: 100Gi + # size: 50Gi # storageClass: "oci-bv" # accessMode: "ReadWriteOnce" diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 7d8382fc..18c0e5a3 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -616,17 +616,22 @@ func (r *OracleRestDataServiceReconciler) instantiatePVCSpec(m *dbapi.OracleRest }, }, StorageClassName: &m.Spec.Persistence.StorageClass, - Selector: &metav1.LabelSelector{ - MatchLabels: func() map[string]string { - ns := make(map[string]string) - if len(m.Spec.NodeSelector) != 0 { - for key, value := range m.Spec.NodeSelector { - ns[key] = value + Selector: func() *metav1.LabelSelector { + if m.Spec.Persistence.StorageClass == "oci-bv" { + return nil + } + return &metav1.LabelSelector{ + MatchLabels: func() map[string]string { + ns := make(map[string]string) + if len(m.Spec.NodeSelector) != 0 { + for key, value := range m.Spec.NodeSelector { + ns[key] = value + } + } + return ns + }(), } - } - return ns - }(), - }, + }(), }, } // Set SingleInstanceDatabase instance as the owner and controller diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 625b49b3..ab73eb3d 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -560,7 +560,6 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, }, }, - ImagePullPolicy: corev1.PullAlways, Ports: []corev1.ContainerPort{{ContainerPort: 1521}, {ContainerPort: 5500}}, ReadinessProbe: &corev1.Probe{ @@ -839,17 +838,22 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePVCSpec(m *dbapi.SingleIns }, }, StorageClassName: &m.Spec.Persistence.StorageClass, - Selector: &metav1.LabelSelector{ - MatchLabels: func() map[string]string { - ns := make(map[string]string) - if len(m.Spec.NodeSelector) != 0 { - for key, value := range m.Spec.NodeSelector { - ns[key] = value + Selector: func() *metav1.LabelSelector { + if m.Spec.Persistence.StorageClass == "oci-bv" { + return nil + } + return &metav1.LabelSelector{ + MatchLabels: func() map[string]string { + ns := make(map[string]string) + if len(m.Spec.NodeSelector) != 0 { + for key, value := range m.Spec.NodeSelector { + ns[key] = value + } + } + return ns + }(), } - } - return ns - }(), - }, + }(), }, } // Set SingleInstanceDatabase instance as the owner and controller @@ -1058,7 +1062,31 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplacePods(m *dbapi.SingleIn // Version/Image changed // PATCHING START (Only Software Patch) // call FindPods() to find pods of newer version . if running , delete the older version replicas. - readyPod, replicasFound, available, _, err = dbcommons.FindPods(r, m.Spec.Image.Version, + + // call FindPods() to find pods of older version . delete all the Pods + readyPod, oldReplicasFound, available, _, err := dbcommons.FindPods(r, oldVersion, + oldImage, m.Name, m.Namespace, ctx, req) + if err != nil { + log.Error(err, err.Error()) + return requeueY, err + } + if readyPod.Name != "" { + log.Info("Ready pod marked for deletion", "name", readyPod.Name) + available = append(available, readyPod) + } + + // For ReadWriteOnce, delete first and then create + if m.Spec.Persistence.Size == "" || m.Spec.Persistence.AccessMode == "ReadWriteOnce" { + result, err := r.deletePods(ctx, req, m, available, corev1.Pod{}, oldReplicasFound, 0) + if result.Requeue { + return result, err + } + result, err = r.createPods(m, n, ctx, req, 0) + return result, err + } + + // For ReadWriteMany + readyPod, newReplicasFound, available, _, err := dbcommons.FindPods(r, m.Spec.Image.Version, m.Spec.Image.PullFrom, m.Name, m.Namespace, ctx, req) if err != nil { log.Error(err, err.Error()) @@ -1066,7 +1094,7 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplacePods(m *dbapi.SingleIn } // create new Pods with the new Version and no.of Replicas required - result, err := r.createPods(m, n, ctx, req, replicasFound) + result, err := r.createPods(m, n, ctx, req, newReplicasFound) if result.Requeue { return result, err } @@ -1084,18 +1112,7 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplacePods(m *dbapi.SingleIn return requeueY, errors.New(eventMsg) } - // call FindPods() to find pods of older version . delete all the Pods - readyPod, replicasFound, available, _, err = dbcommons.FindPods(r, oldVersion, - oldImage, m.Name, m.Namespace, ctx, req) - if err != nil { - log.Error(err, err.Error()) - return requeueY, err - } - if readyPod.Name != "" { - log.Info("Ready pod marked for deletion", "name", readyPod.Name) - available = append(available, readyPod) - } - return r.deletePods(ctx, req, m, available, corev1.Pod{}, replicasFound, 0) + return r.deletePods(ctx, req, m, available, corev1.Pod{}, oldReplicasFound, 0) // PATCHING END } From e9794528cabb0c89027a49ad08543c8b78e62317 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Sat, 30 Apr 2022 15:12:54 +0530 Subject: [PATCH 301/628] Fix claim selector --- controllers/database/oraclerestdataservice_controller.go | 2 +- controllers/database/singleinstancedatabase_controller.go | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 18c0e5a3..07280cf2 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -617,7 +617,7 @@ func (r *OracleRestDataServiceReconciler) instantiatePVCSpec(m *dbapi.OracleRest }, StorageClassName: &m.Spec.Persistence.StorageClass, Selector: func() *metav1.LabelSelector { - if m.Spec.Persistence.StorageClass == "oci-bv" { + if m.Spec.Persistence.StorageClass != "oci" { return nil } return &metav1.LabelSelector{ diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index ab73eb3d..c9a1949b 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -839,7 +839,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePVCSpec(m *dbapi.SingleIns }, StorageClassName: &m.Spec.Persistence.StorageClass, Selector: func() *metav1.LabelSelector { - if m.Spec.Persistence.StorageClass == "oci-bv" { + if m.Spec.Persistence.StorageClass != "oci" { return nil } return &metav1.LabelSelector{ From f4874211fce4d7f70d9bba511e271685b8dd6e6b Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Sat, 30 Apr 2022 16:44:01 +0530 Subject: [PATCH 302/628] Change default storage class to oci --- commons/database/constants.go | 2 +- config/samples/sidb/oraclerestdataservice.yaml | 2 +- config/samples/sidb/oraclerestdataservice_apex.yaml | 2 +- config/samples/sidb/oraclerestdataservice_create.yaml | 2 +- config/samples/sidb/singleinstancedatabase.yaml | 2 +- config/samples/sidb/singleinstancedatabase_clone.yaml | 2 +- config/samples/sidb/singleinstancedatabase_create.yaml | 2 +- config/samples/sidb/singleinstancedatabase_express.yaml | 2 +- config/samples/sidb/singleinstancedatabase_patch.yaml | 2 +- config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml | 2 +- 10 files changed, 10 insertions(+), 10 deletions(-) diff --git a/commons/database/constants.go b/commons/database/constants.go index 4d5fae8c..f381dcfe 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -208,7 +208,7 @@ const GetSqlpatchStatusSQL string = "select status from dba_registry_sqlpatch or const GetSqlpatchVersionSQL string = "select SOURCE_VERSION || ':' || TARGET_VERSION as versions from dba_registry_sqlpatch order by action_time desc;" -const GetCheckpointFileCMD string = "find ${ORACLE_BASE}/oradata -name .${ORACLE_SID}${CHECKPOINT_FILE_EXTN} " +const GetCheckpointFileCMD string = "find ${ORACLE_BASE}/oradata -name .${ORACLE_SID}${CHECKPOINT_FILE_EXTN} 2> /dev/null" const GetEnterpriseEditionFileCMD string = "if [ -f ${ORACLE_BASE}/oradata/dbconfig/$ORACLE_SID/.docker_enterprise ]; then ls ${ORACLE_BASE}/oradata/dbconfig/$ORACLE_SID/.docker_enterprise; fi " diff --git a/config/samples/sidb/oraclerestdataservice.yaml b/config/samples/sidb/oraclerestdataservice.yaml index 0e6d66a9..7cca08d7 100644 --- a/config/samples/sidb/oraclerestdataservice.yaml +++ b/config/samples/sidb/oraclerestdataservice.yaml @@ -46,7 +46,7 @@ spec: ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. persistence: size: 50Gi - storageClass: "oci-bv" + storageClass: "oci" accessMode: "ReadWriteOnce" ## Type of service Applicable on cloud enviroments only. diff --git a/config/samples/sidb/oraclerestdataservice_apex.yaml b/config/samples/sidb/oraclerestdataservice_apex.yaml index 073fc073..b497bd34 100644 --- a/config/samples/sidb/oraclerestdataservice_apex.yaml +++ b/config/samples/sidb/oraclerestdataservice_apex.yaml @@ -67,7 +67,7 @@ spec: ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. persistence: size: 50Gi - storageClass: "oci-bv" + storageClass: "oci" accessMode: "ReadWriteOnce" ## PDB Schemas to be ORDS Enabled. diff --git a/config/samples/sidb/oraclerestdataservice_create.yaml b/config/samples/sidb/oraclerestdataservice_create.yaml index dc00429f..2a35f97c 100644 --- a/config/samples/sidb/oraclerestdataservice_create.yaml +++ b/config/samples/sidb/oraclerestdataservice_create.yaml @@ -48,7 +48,7 @@ spec: ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. persistence: size: 50Gi - storageClass: "oci-bv" + storageClass: "oci" accessMode: "ReadWriteOnce" diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index 771d005c..5811c6c0 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -68,7 +68,7 @@ spec: ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. persistence: size: 100Gi - storageClass: "oci-bv" + storageClass: "oci" accessMode: "ReadWriteOnce" ## Type of service . Applicable on cloud enviroments only diff --git a/config/samples/sidb/singleinstancedatabase_clone.yaml b/config/samples/sidb/singleinstancedatabase_clone.yaml index dbe7a3dc..6dd6903d 100644 --- a/config/samples/sidb/singleinstancedatabase_clone.yaml +++ b/config/samples/sidb/singleinstancedatabase_clone.yaml @@ -38,5 +38,5 @@ spec: ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. persistence: size: 100Gi - storageClass: "oci-bv" + storageClass: "oci" accessMode: "ReadWriteOnce" diff --git a/config/samples/sidb/singleinstancedatabase_create.yaml b/config/samples/sidb/singleinstancedatabase_create.yaml index 81fc9a3e..64e000e0 100644 --- a/config/samples/sidb/singleinstancedatabase_create.yaml +++ b/config/samples/sidb/singleinstancedatabase_create.yaml @@ -55,5 +55,5 @@ spec: ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. persistence: size: 100Gi - storageClass: "oci-bv" + storageClass: "oci" accessMode: "ReadWriteOnce" diff --git a/config/samples/sidb/singleinstancedatabase_express.yaml b/config/samples/sidb/singleinstancedatabase_express.yaml index 02840981..62b81b7d 100644 --- a/config/samples/sidb/singleinstancedatabase_express.yaml +++ b/config/samples/sidb/singleinstancedatabase_express.yaml @@ -43,5 +43,5 @@ spec: ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany persistence: size: 50Gi - storageClass: "oci-bv" + storageClass: "oci" accessMode: "ReadWriteOnce" diff --git a/config/samples/sidb/singleinstancedatabase_patch.yaml b/config/samples/sidb/singleinstancedatabase_patch.yaml index 704e1058..17061897 100644 --- a/config/samples/sidb/singleinstancedatabase_patch.yaml +++ b/config/samples/sidb/singleinstancedatabase_patch.yaml @@ -46,5 +46,5 @@ spec: ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. persistence: size: 100Gi - storageClass: "oci-bv" + storageClass: "oci" accessMode: "ReadWriteOnce" diff --git a/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml b/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml index c22b14f3..cb9febef 100644 --- a/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml +++ b/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml @@ -43,5 +43,5 @@ spec: ## if specified, the prebuilt DB datafiles are copied over to the persistant volume before DB startup #persistence: # size: 50Gi - # storageClass: "oci-bv" + # storageClass: "oci" # accessMode: "ReadWriteOnce" From 5561e45e153abf0a4bd5aee44622b6b0e21c87de Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Sat, 30 Apr 2022 17:54:42 +0530 Subject: [PATCH 303/628] Check for image pull secrets --- .../oraclerestdataservice_controller.go | 31 +++++++++++++++---- .../singleinstancedatabase_controller.go | 27 +++++++++++----- 2 files changed, 45 insertions(+), 13 deletions(-) diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 07280cf2..765466eb 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -125,8 +125,12 @@ func (r *OracleRestDataServiceReconciler) Reconcile(ctx context.Context, req ctr return result, nil } + // Always refresh status before a reconcile + defer r.Status().Update(ctx, oracleRestDataService) + defer r.Status().Update(ctx, singleInstanceDatabase) + // First validate - result, err = r.validate(oracleRestDataService, singleInstanceDatabase) + result, err = r.validate(oracleRestDataService, singleInstanceDatabase, ctx) if result.Requeue { r.Log.Info("Spec validation failed, Reconcile queued") return result, nil @@ -136,10 +140,6 @@ func (r *OracleRestDataServiceReconciler) Reconcile(ctx context.Context, req ctr return result, nil } - // Always refresh status before a reconcile - defer r.Status().Update(ctx, oracleRestDataService) - defer r.Status().Update(ctx, singleInstanceDatabase) - // Create Service result = r.createSVC(ctx, req, oracleRestDataService, singleInstanceDatabase) if result.Requeue { @@ -201,11 +201,30 @@ func (r *OracleRestDataServiceReconciler) Reconcile(ctx context.Context, req ctr //############################################################################# // Validate the CRD specs //############################################################################# -func (r *OracleRestDataServiceReconciler) validate(m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase) (ctrl.Result, error) { +func (r *OracleRestDataServiceReconciler) validate(m *dbapi.OracleRestDataService, + n *dbapi.SingleInstanceDatabase, ctx context.Context) (ctrl.Result, error) { var err error eventReason := "Spec Error" var eventMsgs []string + + //First check image pull secrets + if m.Spec.Image.PullSecrets != "" { + secret := &corev1.Secret{} + err = r.Get(ctx, types.NamespacedName{Name: m.Spec.Image.PullSecrets, Namespace: m.Namespace}, secret) + if err != nil { + if apierrors.IsNotFound(err) { + // Secret not found + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, err.Error()) + r.Log.Info(err.Error()) + m.Status.Status = dbcommons.StatusError + return requeueY, err + } + r.Log.Error(err, err.Error()) + return requeueY, err + } + } + // If using same pvc for ords as sidb, ensure sidb has ReadWriteMany Accessmode if n.Spec.Persistence.AccessMode == "ReadWriteOnce" && m.Spec.Persistence.AccessMode == "" { eventMsgs = append(eventMsgs, "ords can be installed only on ReadWriteMany Access Mode of : "+m.Spec.DatabaseRef) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index c9a1949b..2c74b7fa 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -303,6 +303,24 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab r.Log.Info("Entering reconcile validation") + //First check image pull secrets + if m.Spec.Image.PullSecrets != "" { + secret := &corev1.Secret{} + err = r.Get(ctx, types.NamespacedName{Name: m.Spec.Image.PullSecrets, Namespace: m.Namespace}, secret) + if err != nil { + if apierrors.IsNotFound(err) { + // Secret not found + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, err.Error()) + r.Log.Info(err.Error()) + m.Status.Status = dbcommons.StatusError + return requeueY, err + } + r.Log.Error(err, err.Error()) + return requeueY, err + } + } + + // If Express Edition , Ensure Replicas=1 if m.Spec.Edition == "express" && m.Spec.Replicas > 1 { eventMsgs = append(eventMsgs, "XE supports only one replica") @@ -1373,16 +1391,11 @@ func (r *SingleInstanceDatabaseReconciler) validateDBReadiness(m *dbapi.SingleIn eventReason = "Database Creating" eventMsg = "waiting for database to be ready" m.Status.Status = dbcommons.StatusCreating - if m.Spec.Edition == "express" { - eventReason = "Database Unhealthy" - m.Status.Status = dbcommons.StatusNotReady - r.updateORDSStatus(m, ctx, req) - } + out, err := dbcommons.ExecCommand(r, r.Config, runningPod.Name, runningPod.Namespace, "", ctx, req, false, "bash", "-c", dbcommons.GetCheckpointFileCMD) if err != nil { - r.Log.Error(err, err.Error()) - return requeueY, readyPod, err + r.Log.Info(err.Error()) } r.Log.Info("GetCheckpointFileCMD Output : \n" + out) From 3927d2260125edc46f04f3694798a652b5eae3b2 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Sat, 30 Apr 2022 22:50:55 +0530 Subject: [PATCH 304/628] Fix replica validation --- .../singleinstancedatabase_webhook.go | 32 +++++++-------- .../singleinstancedatabase_controller.go | 41 +++++++------------ 2 files changed, 28 insertions(+), 45 deletions(-) diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index bac7571d..d5ae69a0 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -102,20 +102,6 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { singleinstancedatabaselog.Info("validate create", "name", r.Name) var allErrs field.ErrorList - if r.Spec.Replicas > 1 { - valMsg := "" - if r.Spec.Edition == "express" { - valMsg = "should be 1 for express edition" - } - if r.Spec.Image.PrebuiltDB { - valMsg = "should be 1 for prebuiltDB" - } - if valMsg != "" { - allErrs = append(allErrs, - field.Invalid(field.NewPath("spec").Child("replicas"), r.Spec.Replicas, valMsg)) - } - } - // Persistence spec validation if r.Spec.Persistence.Size == "" && (r.Spec.Persistence.AccessMode != "" || r.Spec.Persistence.StorageClass != "" ) || (r.Spec.Persistence.Size != "" && r.Spec.Persistence.AccessMode == "" ) { @@ -131,11 +117,21 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { r.Spec.Persistence.AccessMode, "should be either \"ReadWriteOnce\" or \"ReadWriteMany\"")) } - if r.Spec.Persistence.AccessMode == "ReadWriteOnce" && r.Spec.Replicas != 1 { - allErrs = append(allErrs, - field.Invalid(field.NewPath("spec").Child("replicas"), r.Spec.Replicas, - "should be 1 for accessMode \"ReadWriteOnce\"")) + // Replica validation + if r.Spec.Replicas > 1 { + valMsg := "" + if r.Spec.Edition == "express" { + valMsg = "should be 1 for express edition" + } + if r.Spec.Persistence.Size == "" { + valMsg = "should be 1 if no persistence is specified" + } + if valMsg != "" { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("replicas"), r.Spec.Replicas, valMsg)) + } } + if r.Spec.Edition == "express" && r.Spec.CloneFrom != "" { allErrs = append(allErrs, field.Invalid(field.NewPath("spec").Child("cloneFrom"), r.Spec.CloneFrom, diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 2c74b7fa..c815c34f 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1080,31 +1080,7 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplacePods(m *dbapi.SingleIn // Version/Image changed // PATCHING START (Only Software Patch) // call FindPods() to find pods of newer version . if running , delete the older version replicas. - - // call FindPods() to find pods of older version . delete all the Pods - readyPod, oldReplicasFound, available, _, err := dbcommons.FindPods(r, oldVersion, - oldImage, m.Name, m.Namespace, ctx, req) - if err != nil { - log.Error(err, err.Error()) - return requeueY, err - } - if readyPod.Name != "" { - log.Info("Ready pod marked for deletion", "name", readyPod.Name) - available = append(available, readyPod) - } - - // For ReadWriteOnce, delete first and then create - if m.Spec.Persistence.Size == "" || m.Spec.Persistence.AccessMode == "ReadWriteOnce" { - result, err := r.deletePods(ctx, req, m, available, corev1.Pod{}, oldReplicasFound, 0) - if result.Requeue { - return result, err - } - result, err = r.createPods(m, n, ctx, req, 0) - return result, err - } - - // For ReadWriteMany - readyPod, newReplicasFound, available, _, err := dbcommons.FindPods(r, m.Spec.Image.Version, + readyPod, replicasFound, available, _, err = dbcommons.FindPods(r, m.Spec.Image.Version, m.Spec.Image.PullFrom, m.Name, m.Namespace, ctx, req) if err != nil { log.Error(err, err.Error()) @@ -1112,7 +1088,7 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplacePods(m *dbapi.SingleIn } // create new Pods with the new Version and no.of Replicas required - result, err := r.createPods(m, n, ctx, req, newReplicasFound) + result, err := r.createPods(m, n, ctx, req, replicasFound) if result.Requeue { return result, err } @@ -1130,7 +1106,18 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplacePods(m *dbapi.SingleIn return requeueY, errors.New(eventMsg) } - return r.deletePods(ctx, req, m, available, corev1.Pod{}, oldReplicasFound, 0) + // call FindPods() to find pods of older version . delete all the Pods + readyPod, replicasFound, available, _, err = dbcommons.FindPods(r, oldVersion, + oldImage, m.Name, m.Namespace, ctx, req) + if err != nil { + log.Error(err, err.Error()) + return requeueY, err + } + if readyPod.Name != "" { + log.Info("Ready pod marked for deletion", "name", readyPod.Name) + available = append(available, readyPod) + } + return r.deletePods(ctx, req, m, available, corev1.Pod{}, replicasFound, 0) // PATCHING END } From fc7252fa010e4a90350c069a82f5893467302513 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Sat, 30 Apr 2022 23:00:08 +0530 Subject: [PATCH 305/628] Fix status setting --- .../v1alpha1/singleinstancedatabase_webhook.go | 5 +++++ .../singleinstancedatabase_controller.go | 16 ++++++++-------- 2 files changed, 13 insertions(+), 8 deletions(-) diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index d5ae69a0..bbf7dab5 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -90,6 +90,11 @@ func (r *SingleInstanceDatabase) Default() { } + r.Status.Role = dbcommons.ValueUnavailable + r.Status.ConnectString = dbcommons.ValueUnavailable + r.Status.PdbConnectString = dbcommons.ValueUnavailable + r.Status.OemExpressUrl = dbcommons.ValueUnavailable + r.Status.ReleaseUpdate = dbcommons.ValueUnavailable } // TODO(user): change verbs to "verbs=create;update;delete" if you want to enable deletion validation. diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index c815c34f..913cba1e 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -980,9 +980,9 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex } log.Info("Found Existing Service ", "Service Name ", svc.Name) - m.Status.ConnectString = dbcommons.ValueUnavailable - m.Status.PdbConnectString = dbcommons.ValueUnavailable - m.Status.OemExpressUrl = dbcommons.ValueUnavailable + //m.Status.ConnectString = dbcommons.ValueUnavailable + //m.Status.PdbConnectString = dbcommons.ValueUnavailable + //m.Status.OemExpressUrl = dbcommons.ValueUnavailable pdbName := strings.ToUpper(m.Spec.Pdbname) sid := m.Spec.Sid @@ -1257,11 +1257,11 @@ func (r *SingleInstanceDatabaseReconciler) createPods(m *dbapi.SingleInstanceDat m.Status.Status = dbcommons.StatusPending m.Status.DatafilesCreated = "false" m.Status.DatafilesPatched = "false" - m.Status.Role = dbcommons.ValueUnavailable - m.Status.ConnectString = dbcommons.ValueUnavailable - m.Status.PdbConnectString = dbcommons.ValueUnavailable - m.Status.OemExpressUrl = dbcommons.ValueUnavailable - m.Status.ReleaseUpdate = dbcommons.ValueUnavailable + //m.Status.Role = dbcommons.ValueUnavailable + //m.Status.ConnectString = dbcommons.ValueUnavailable + //m.Status.PdbConnectString = dbcommons.ValueUnavailable + //m.Status.OemExpressUrl = dbcommons.ValueUnavailable + //m.Status.ReleaseUpdate = dbcommons.ValueUnavailable } // if Found < Required , Create New Pods , Name of Pods are generated Randomly for i := replicasFound; i < replicasReq; i++ { From 683d266fd1c4a203f973f896d58dfab6b243f28d Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Sat, 30 Apr 2022 23:24:06 +0530 Subject: [PATCH 306/628] Enhance status updates --- .../v1alpha1/singleinstancedatabase_webhook.go | 7 ------- .../singleinstancedatabase_controller.go | 17 +++++++++++++++++ 2 files changed, 17 insertions(+), 7 deletions(-) diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index bbf7dab5..652d1f70 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -87,14 +87,7 @@ func (r *SingleInstanceDatabase) Default() { } else { r.Spec.Pdbname = "ORCLPDB1" } - } - - r.Status.Role = dbcommons.ValueUnavailable - r.Status.ConnectString = dbcommons.ValueUnavailable - r.Status.PdbConnectString = dbcommons.ValueUnavailable - r.Status.OemExpressUrl = dbcommons.ValueUnavailable - r.Status.ReleaseUpdate = dbcommons.ValueUnavailable } // TODO(user): change verbs to "verbs=create;update;delete" if you want to enable deletion validation. diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 913cba1e..86b3248d 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -303,6 +303,23 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab r.Log.Info("Entering reconcile validation") + /* Initialize statuses */ + if m.Status.Role == "" { + m.Status.Role = dbcommons.ValueUnavailable + } + if m.Status.ConnectString == "" { + m.Status.ConnectString = dbcommons.ValueUnavailable + } + if m.Status.PdbConnectString == "" { + m.Status.PdbConnectString = dbcommons.ValueUnavailable + } + if m.Status.OemExpressUrl == "" { + m.Status.OemExpressUrl = dbcommons.ValueUnavailable + } + if m.Status.ReleaseUpdate == "" { + m.Status.ReleaseUpdate = dbcommons.ValueUnavailable + } + //First check image pull secrets if m.Spec.Image.PullSecrets != "" { secret := &corev1.Secret{} From 98c677fdf43ff61d1de17b8da8079862bc319400 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Sat, 30 Apr 2022 23:26:09 +0530 Subject: [PATCH 307/628] Revert to oci-bv --- config/samples/sidb/oraclerestdataservice.yaml | 2 +- config/samples/sidb/oraclerestdataservice_apex.yaml | 2 +- config/samples/sidb/oraclerestdataservice_create.yaml | 2 +- config/samples/sidb/singleinstancedatabase.yaml | 2 +- config/samples/sidb/singleinstancedatabase_clone.yaml | 2 +- config/samples/sidb/singleinstancedatabase_create.yaml | 2 +- config/samples/sidb/singleinstancedatabase_express.yaml | 2 +- config/samples/sidb/singleinstancedatabase_patch.yaml | 2 +- config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml | 2 +- 9 files changed, 9 insertions(+), 9 deletions(-) diff --git a/config/samples/sidb/oraclerestdataservice.yaml b/config/samples/sidb/oraclerestdataservice.yaml index 7cca08d7..0e6d66a9 100644 --- a/config/samples/sidb/oraclerestdataservice.yaml +++ b/config/samples/sidb/oraclerestdataservice.yaml @@ -46,7 +46,7 @@ spec: ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. persistence: size: 50Gi - storageClass: "oci" + storageClass: "oci-bv" accessMode: "ReadWriteOnce" ## Type of service Applicable on cloud enviroments only. diff --git a/config/samples/sidb/oraclerestdataservice_apex.yaml b/config/samples/sidb/oraclerestdataservice_apex.yaml index b497bd34..073fc073 100644 --- a/config/samples/sidb/oraclerestdataservice_apex.yaml +++ b/config/samples/sidb/oraclerestdataservice_apex.yaml @@ -67,7 +67,7 @@ spec: ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. persistence: size: 50Gi - storageClass: "oci" + storageClass: "oci-bv" accessMode: "ReadWriteOnce" ## PDB Schemas to be ORDS Enabled. diff --git a/config/samples/sidb/oraclerestdataservice_create.yaml b/config/samples/sidb/oraclerestdataservice_create.yaml index 2a35f97c..dc00429f 100644 --- a/config/samples/sidb/oraclerestdataservice_create.yaml +++ b/config/samples/sidb/oraclerestdataservice_create.yaml @@ -48,7 +48,7 @@ spec: ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. persistence: size: 50Gi - storageClass: "oci" + storageClass: "oci-bv" accessMode: "ReadWriteOnce" diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index 5811c6c0..771d005c 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -68,7 +68,7 @@ spec: ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. persistence: size: 100Gi - storageClass: "oci" + storageClass: "oci-bv" accessMode: "ReadWriteOnce" ## Type of service . Applicable on cloud enviroments only diff --git a/config/samples/sidb/singleinstancedatabase_clone.yaml b/config/samples/sidb/singleinstancedatabase_clone.yaml index 6dd6903d..dbe7a3dc 100644 --- a/config/samples/sidb/singleinstancedatabase_clone.yaml +++ b/config/samples/sidb/singleinstancedatabase_clone.yaml @@ -38,5 +38,5 @@ spec: ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. persistence: size: 100Gi - storageClass: "oci" + storageClass: "oci-bv" accessMode: "ReadWriteOnce" diff --git a/config/samples/sidb/singleinstancedatabase_create.yaml b/config/samples/sidb/singleinstancedatabase_create.yaml index 64e000e0..81fc9a3e 100644 --- a/config/samples/sidb/singleinstancedatabase_create.yaml +++ b/config/samples/sidb/singleinstancedatabase_create.yaml @@ -55,5 +55,5 @@ spec: ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. persistence: size: 100Gi - storageClass: "oci" + storageClass: "oci-bv" accessMode: "ReadWriteOnce" diff --git a/config/samples/sidb/singleinstancedatabase_express.yaml b/config/samples/sidb/singleinstancedatabase_express.yaml index 62b81b7d..02840981 100644 --- a/config/samples/sidb/singleinstancedatabase_express.yaml +++ b/config/samples/sidb/singleinstancedatabase_express.yaml @@ -43,5 +43,5 @@ spec: ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany persistence: size: 50Gi - storageClass: "oci" + storageClass: "oci-bv" accessMode: "ReadWriteOnce" diff --git a/config/samples/sidb/singleinstancedatabase_patch.yaml b/config/samples/sidb/singleinstancedatabase_patch.yaml index 17061897..704e1058 100644 --- a/config/samples/sidb/singleinstancedatabase_patch.yaml +++ b/config/samples/sidb/singleinstancedatabase_patch.yaml @@ -46,5 +46,5 @@ spec: ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. persistence: size: 100Gi - storageClass: "oci" + storageClass: "oci-bv" accessMode: "ReadWriteOnce" diff --git a/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml b/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml index cb9febef..c22b14f3 100644 --- a/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml +++ b/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml @@ -43,5 +43,5 @@ spec: ## if specified, the prebuilt DB datafiles are copied over to the persistant volume before DB startup #persistence: # size: 50Gi - # storageClass: "oci" + # storageClass: "oci-bv" # accessMode: "ReadWriteOnce" From 4c60e6404086f126d7e5a174edb4cda1276a3507 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Sun, 1 May 2022 06:20:17 +0530 Subject: [PATCH 308/628] Fix init --- commons/database/constants.go | 2 +- .../singleinstancedatabase_controller.go | 24 +++++++++++-------- 2 files changed, 15 insertions(+), 11 deletions(-) diff --git a/commons/database/constants.go b/commons/database/constants.go index f381dcfe..216b00a7 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -386,7 +386,7 @@ const WalletEntriesCMD string = "umask 177\ncat > wallet.passwd < /dev/null; do sleep 0.5; done; fi " -const InitPrebuiltDbCMD string = "if [ ! -d /mnt/oradata/${ORACLE_SID} ]; then cp -v $ORACLE_BASE/oradata/.${ORACLE_SID}$CHECKPOINT_FILE_EXTN /mnt/oradata && " + +const InitPrebuiltDbCMD string = "if [ ! -d /mnt/oradata/${ORACLE_SID} -a -d $ORACLE_BASE/oradata/${ORACLE_SID} ]; then cp -v $ORACLE_BASE/oradata/.${ORACLE_SID}$CHECKPOINT_FILE_EXTN /mnt/oradata && " + " cp -vr $ORACLE_BASE/oradata/${ORACLE_SID} /mnt/oradata && cp -vr $ORACLE_BASE/oradata/dbconfig /mnt/oradata; fi " diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 86b3248d..af782152 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -304,6 +304,13 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab r.Log.Info("Entering reconcile validation") /* Initialize statuses */ + if m.Status.Edition == "" { + if m.Spec.Edition != "" { + m.Status.Edition = strings.Title(m.Spec.Edition) + } else { + m.Status.Edition = dbcommons.ValueUnavailable + } + } if m.Status.Role == "" { m.Status.Role = dbcommons.ValueUnavailable } @@ -342,18 +349,10 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab if m.Spec.Edition == "express" && m.Spec.Replicas > 1 { eventMsgs = append(eventMsgs, "XE supports only one replica") } - // If Block Volume , Ensure Replicas=1 - if m.Spec.Persistence.AccessMode == "ReadWriteOnce" && m.Spec.Replicas > 1 { - eventMsgs = append(eventMsgs, "accessMode ReadWriteOnce supports only one replica") - } if m.Status.Sid != "" && !strings.EqualFold(m.Spec.Sid, m.Status.Sid) { eventMsgs = append(eventMsgs, "sid cannot be updated") } - edition := m.Spec.Edition - if m.Spec.Edition == "" { - edition = "Enterprise" - } - if m.Spec.CloneFrom == "" && m.Status.Edition != "" && !strings.EqualFold(m.Status.Edition, edition) { + if m.Spec.CloneFrom == "" && m.Status.Edition != "" && !strings.EqualFold(m.Status.Edition, m.Spec.Edition) { eventMsgs = append(eventMsgs, "edition cannot be updated") } if m.Status.Charset != "" && !strings.EqualFold(m.Status.Charset, m.Spec.Charset) { @@ -400,7 +399,6 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab // update status fields m.Status.Sid = m.Spec.Sid - m.Status.Edition = strings.Title(edition) m.Status.Charset = m.Spec.Charset m.Status.Pdbname = m.Spec.Pdbname m.Status.Persistence = m.Spec.Persistence @@ -531,6 +529,12 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns MountPath: "/mnt/oradata", Name: "datamount", }}, + Env: []corev1.EnvVar{ + { + Name: "ORACLE_SID", + Value: strings.ToUpper(m.Spec.Sid), + }, + }, }) } } From 3d66ae9679a53165944411a8e6869c1cbbb6c2a4 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Sun, 1 May 2022 07:12:44 +0530 Subject: [PATCH 309/628] Add pod affinity --- .../database/singleinstancedatabase_controller.go | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index af782152..b639317d 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -473,6 +473,19 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, }, Spec: corev1.PodSpec{ + Affinity: func() *corev1.Affinity { + if m.Spec.Persistence.AccessMode == "ReadWriteOnce" { + return &corev1.Affinity{ + PodAffinity: &corev1.PodAffinity{ + RequiredDuringSchedulingIgnoredDuringExecution: []corev1.PodAffinityTerm{{ + TopologyKey: "kubernetes.io/hostname", + }, + }, + }, + } + } + return nil + }(), Volumes: []corev1.Volume{{ Name: "datamount", VolumeSource: func() corev1.VolumeSource { From 48891f356e44193064b42617b22130965e853e92 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Sun, 1 May 2022 07:37:06 +0530 Subject: [PATCH 310/628] Fix pod affinity --- controllers/database/singleinstancedatabase_controller.go | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index b639317d..8626b6d1 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -478,6 +478,12 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns return &corev1.Affinity{ PodAffinity: &corev1.PodAffinity{ RequiredDuringSchedulingIgnoredDuringExecution: []corev1.PodAffinityTerm{{ + LabelSelector: &metav1.LabelSelector{ + MatchExpressions: []metav1.LabelSelectorRequirement{{ + Key: "kubernetes.io/hostname", + Operator: metav1.LabelSelectorOpExists, + }}, + }, TopologyKey: "kubernetes.io/hostname", }, }, From 066edb3c8088ce3f0d01df51fa8c178e85a4b18e Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Sun, 1 May 2022 08:22:59 +0530 Subject: [PATCH 311/628] Fix affinity --- .../singleinstancedatabase_controller.go | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 8626b6d1..026f51bb 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -476,14 +476,18 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns Affinity: func() *corev1.Affinity { if m.Spec.Persistence.AccessMode == "ReadWriteOnce" { return &corev1.Affinity{ + NodeAffinity: &corev1.NodeAffinity{ + RequiredDuringSchedulingIgnoredDuringExecution: &corev1.NodeSelector{ + NodeSelectorTerms: []corev1.NodeSelectorTerm{{ + MatchExpressions: []corev1.NodeSelectorRequirement{{ + Key: "kubernetes.io/hostname", + Operator: corev1.NodeSelectorOpExists, + }}, + }}, + }, + } , PodAffinity: &corev1.PodAffinity{ RequiredDuringSchedulingIgnoredDuringExecution: []corev1.PodAffinityTerm{{ - LabelSelector: &metav1.LabelSelector{ - MatchExpressions: []metav1.LabelSelectorRequirement{{ - Key: "kubernetes.io/hostname", - Operator: metav1.LabelSelectorOpExists, - }}, - }, TopologyKey: "kubernetes.io/hostname", }, }, From fe2dd2a5880d2f0e8e8c06dea7e303f7ea8f2597 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Sun, 1 May 2022 08:38:43 +0530 Subject: [PATCH 312/628] Add pod affinity --- .../database/singleinstancedatabase_controller.go | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 026f51bb..11ad7ee6 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -487,8 +487,18 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, } , PodAffinity: &corev1.PodAffinity{ - RequiredDuringSchedulingIgnoredDuringExecution: []corev1.PodAffinityTerm{{ - TopologyKey: "kubernetes.io/hostname", + PreferredDuringSchedulingIgnoredDuringExecution: []corev1.WeightedPodAffinityTerm{{ + Weight: 50, + PodAffinityTerm: corev1.PodAffinityTerm { + LabelSelector: &metav1.LabelSelector{ + MatchExpressions: []metav1.LabelSelectorRequirement{{ + Key: "app", + Operator: metav1.LabelSelectorOpIn, + Values: []string{m.Name}, + }}, + }, + TopologyKey: "kubernetes.io/hostname", + }, }, }, }, From fecc3356a334a2e394bc0a2c8d735154c6cacafe Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Sun, 1 May 2022 11:38:43 +0530 Subject: [PATCH 313/628] Fix wrong secret --- .../singleinstancedatabase_controller.go | 98 +++++++++---------- 1 file changed, 45 insertions(+), 53 deletions(-) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 11ad7ee6..4aad76a0 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -344,7 +344,6 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab } } - // If Express Edition , Ensure Replicas=1 if m.Spec.Edition == "express" && m.Spec.Replicas > 1 { eventMsgs = append(eventMsgs, "XE supports only one replica") @@ -458,7 +457,7 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab // Instantiate POD spec from SingleInstanceDatabase spec //############################################################################# func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleInstanceDatabase, n *dbapi.SingleInstanceDatabase) *corev1.Pod { - + // POD spec pod := &corev1.Pod{ TypeMeta: metav1.TypeMeta{ @@ -476,31 +475,21 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns Affinity: func() *corev1.Affinity { if m.Spec.Persistence.AccessMode == "ReadWriteOnce" { return &corev1.Affinity{ - NodeAffinity: &corev1.NodeAffinity{ - RequiredDuringSchedulingIgnoredDuringExecution: &corev1.NodeSelector{ - NodeSelectorTerms: []corev1.NodeSelectorTerm{{ - MatchExpressions: []corev1.NodeSelectorRequirement{{ - Key: "kubernetes.io/hostname", - Operator: corev1.NodeSelectorOpExists, - }}, - }}, - }, - } , PodAffinity: &corev1.PodAffinity{ PreferredDuringSchedulingIgnoredDuringExecution: []corev1.WeightedPodAffinityTerm{{ - Weight: 50, - PodAffinityTerm: corev1.PodAffinityTerm { - LabelSelector: &metav1.LabelSelector{ - MatchExpressions: []metav1.LabelSelectorRequirement{{ - Key: "app", - Operator: metav1.LabelSelectorOpIn, - Values: []string{m.Name}, - }}, - }, - TopologyKey: "kubernetes.io/hostname", + Weight: 100, + PodAffinityTerm: corev1.PodAffinityTerm{ + LabelSelector: &metav1.LabelSelector{ + MatchExpressions: []metav1.LabelSelectorRequirement{{ + Key: "app", + Operator: metav1.LabelSelectorOpIn, + Values: []string{m.Name}, + }}, }, + TopologyKey: "kubernetes.io/hostname", }, }, + }, }, } } @@ -508,10 +497,10 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }(), Volumes: []corev1.Volume{{ Name: "datamount", - VolumeSource: func() corev1.VolumeSource { + VolumeSource: func() corev1.VolumeSource { if m.Spec.Persistence.Size == "" { return corev1.VolumeSource{EmptyDir: &corev1.EmptyDirVolumeSource{}} - } + } /* Persistence is specified */ return corev1.VolumeSource{ PersistentVolumeClaim: &corev1.PersistentVolumeClaimVolumeSource{ @@ -534,9 +523,9 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, }}, InitContainers: func() []corev1.Container { - initContainers := []corev1.Container {} + initContainers := []corev1.Container{} if m.Spec.Persistence.Size != "" { - initContainers = append(initContainers, corev1.Container { + initContainers = append(initContainers, corev1.Container{ Name: "init-permissions", Image: m.Spec.Image.PullFrom, Command: []string{"/bin/sh", "-c", fmt.Sprintf("chown %d:%d /opt/oracle/oradata", int(dbcommons.ORACLE_UID), int(dbcommons.ORACLE_GUID))}, @@ -550,11 +539,11 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }}, }) if m.Spec.Image.PrebuiltDB { - initContainers = append(initContainers, corev1.Container { + initContainers = append(initContainers, corev1.Container{ Name: "init-prebuiltdb", Image: m.Spec.Image.PullFrom, Command: []string{"/bin/sh", "-c", dbcommons.InitPrebuiltDbCMD}, - SecurityContext: &corev1.SecurityContext { + SecurityContext: &corev1.SecurityContext{ RunAsUser: func() *int64 { i := int64(dbcommons.ORACLE_UID); return &i }(), RunAsGroup: func() *int64 { i := int64(dbcommons.ORACLE_GUID); return &i }(), }, @@ -573,7 +562,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns } /* Wallet only for non-express edition, non-prebuiltDB */ if m.Spec.Edition != "express" && !m.Spec.Image.PrebuiltDB { - initContainers = append(initContainers, corev1.Container{ + initContainers = append(initContainers, corev1.Container{ Name: "init-wallet", Image: m.Spec.Image.PullFrom, Env: []corev1.EnvVar{ @@ -632,7 +621,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, }, }, - Ports: []corev1.ContainerPort{{ContainerPort: 1521}, {ContainerPort: 5500}}, + Ports: []corev1.ContainerPort{{ContainerPort: 1521}, {ContainerPort: 5500}}, ReadinessProbe: &corev1.Probe{ ProbeHandler: corev1.ProbeHandler{ @@ -653,15 +642,15 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns VolumeMounts: func() []corev1.VolumeMount { mounts := []corev1.VolumeMount{} if m.Spec.Persistence.Size != "" { - mounts = append(mounts, corev1.VolumeMount { + mounts = append(mounts, corev1.VolumeMount{ MountPath: "/opt/oracle/oradata", Name: "datamount", }) } if m.Spec.Edition == "express" || m.Spec.Image.PrebuiltDB { // mounts pwd as secrets for express edition - // or prebuilt db - mounts = append(mounts, corev1.VolumeMount { + // or prebuilt db + mounts = append(mounts, corev1.VolumeMount{ MountPath: "/run/secrets/oracle_pwd", ReadOnly: true, Name: "oracle-pwd-vol", @@ -669,7 +658,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }) } return mounts - }(), + }(), Env: func() []corev1.EnvVar { // adding XE support, useful for dev/test/CI-CD if m.Spec.Edition == "express" { @@ -915,16 +904,16 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePVCSpec(m *dbapi.SingleIns return nil } return &metav1.LabelSelector{ - MatchLabels: func() map[string]string { - ns := make(map[string]string) - if len(m.Spec.NodeSelector) != 0 { - for key, value := range m.Spec.NodeSelector { - ns[key] = value - } - } - return ns - }(), + MatchLabels: func() map[string]string { + ns := make(map[string]string) + if len(m.Spec.NodeSelector) != 0 { + for key, value := range m.Spec.NodeSelector { + ns[key] = value + } } + return ns + }(), + } }(), }, } @@ -939,7 +928,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePVCSpec(m *dbapi.SingleIns func (r *SingleInstanceDatabaseReconciler) createOrReplacePVC(ctx context.Context, req ctrl.Request, m *dbapi.SingleInstanceDatabase) (ctrl.Result, error) { - // Don't create PVC if persistence is not chosen + // Don't create PVC if persistence is not chosen if m.Spec.Persistence.Size == "" { return requeueN, nil } @@ -1157,6 +1146,16 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplacePods(m *dbapi.SingleIn eventMsg := "waiting for newer version/image DB pods get to running state" r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) log.Info(eventMsg) + + for i := 0; i < len(available); i++ { + if strings.Contains(available[i].Status.Reason, "ImagePullBackOff") { + var gracePeriodSeconds int64 = 0 + policy := metav1.DeletePropagationForeground + r.Delete(ctx, &available[i], &client.DeleteOptions{ + GracePeriodSeconds: &gracePeriodSeconds, PropagationPolicy: &policy}) + } + } + return requeueY, errors.New(eventMsg) } @@ -1546,16 +1545,11 @@ func (r *SingleInstanceDatabaseReconciler) runDatapatch(m *dbapi.SingleInstanceD return requeueN, nil } - // No Patching for Pre-built db - if m.Spec.Image.PrebuiltDB { - return requeueN, nil - } - m.Status.Status = dbcommons.StatusPatching - r.Status().Update(ctx, m) eventReason := "Datapatch Executing" - eventMsg := "datapatch begin execution" + eventMsg := "datapatch begins execution" r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + r.Status().Update(ctx, m) //RUN DATAPATCH out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", @@ -1601,7 +1595,6 @@ func (r *SingleInstanceDatabaseReconciler) updateInitParameters(m *dbapi.SingleI readyPod corev1.Pod, ctx context.Context, req ctrl.Request) (ctrl.Result, error) { log := r.Log.WithValues("updateInitParameters", req.NamespacedName) - if m.Status.InitParams == m.Spec.InitParams { return requeueN, nil } @@ -1647,7 +1640,6 @@ func (r *SingleInstanceDatabaseReconciler) updateDBConfig(m *dbapi.SingleInstanc log := r.Log.WithValues("updateDBConfig", req.NamespacedName) - m.Status.Status = dbcommons.StatusUpdating r.Status().Update(ctx, m) var forceLoggingStatus bool From f31fa9ffb968337173f2fa5953bc42d9bf7a54d1 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Sun, 1 May 2022 11:46:10 +0530 Subject: [PATCH 314/628] Debug stms --- controllers/database/singleinstancedatabase_controller.go | 3 +++ 1 file changed, 3 insertions(+) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 4aad76a0..46f1bb98 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1148,6 +1148,9 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplacePods(m *dbapi.SingleIn log.Info(eventMsg) for i := 0; i < len(available); i++ { + r.Log.Info("Unavailable reason: ", "reason", available[i].Status.Reason) + r.Log.Info("Unavailable reason: ", "phase", available[i].Status.Phase) + r.Log.Info("Unavailable reason: ", "msg", available[i].Status.Message) if strings.Contains(available[i].Status.Reason, "ImagePullBackOff") { var gracePeriodSeconds int64 = 0 policy := metav1.DeletePropagationForeground From f81ac86501cebf8a7ad4f03da5dc93e05849fcea Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Sun, 1 May 2022 12:05:16 +0530 Subject: [PATCH 315/628] Add debug statements --- controllers/database/singleinstancedatabase_controller.go | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 46f1bb98..ea09f08a 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1148,9 +1148,10 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplacePods(m *dbapi.SingleIn log.Info(eventMsg) for i := 0; i < len(available); i++ { - r.Log.Info("Unavailable reason: ", "reason", available[i].Status.Reason) - r.Log.Info("Unavailable reason: ", "phase", available[i].Status.Phase) - r.Log.Info("Unavailable reason: ", "msg", available[i].Status.Message) + for j :=0; j < len(available[i].Status.InitContainerStatuses); j++ { + r.Log.Info("Unavailable reason: ", "state", available[i].Status.InitContainerStatuses[i]) + r.Log.Info("Unavailable reason: ", "reason", available[i].Status.InitContainerStatuses[i].State.Waiting.Reason) + } if strings.Contains(available[i].Status.Reason, "ImagePullBackOff") { var gracePeriodSeconds int64 = 0 policy := metav1.DeletePropagationForeground From 1322f4b52686049d0f3f283f96f363b753151598 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Sun, 1 May 2022 12:15:06 +0530 Subject: [PATCH 316/628] Fix condition --- .../database/singleinstancedatabase_controller.go | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index ea09f08a..2d163cf6 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1148,11 +1148,10 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplacePods(m *dbapi.SingleIn log.Info(eventMsg) for i := 0; i < len(available); i++ { - for j :=0; j < len(available[i].Status.InitContainerStatuses); j++ { - r.Log.Info("Unavailable reason: ", "state", available[i].Status.InitContainerStatuses[i]) - r.Log.Info("Unavailable reason: ", "reason", available[i].Status.InitContainerStatuses[i].State.Waiting.Reason) - } - if strings.Contains(available[i].Status.Reason, "ImagePullBackOff") { + waitingReason := available[i].Status.InitContainerStatuses[i].State.Waiting.Reason + r.Log.Info("Pod unavailable reason: ", "reason", waitingReason) + if strings.Contains(waitingReason, "ImagePullBackOff") || strings.Contains(waitingReason, "ErrImagePull") { + r.Log.Info("Deleting pod", "name", available[0].Name) var gracePeriodSeconds int64 = 0 policy := metav1.DeletePropagationForeground r.Delete(ctx, &available[i], &client.DeleteOptions{ From 4476dc6ac5db3834317e8ec5f3efb92354d4782f Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Sun, 1 May 2022 14:21:32 +0530 Subject: [PATCH 317/628] Fix create and replace PODs --- .../samples/sidb/singleinstancedatabase.yaml | 5 +- .../sidb/singleinstancedatabase_clone.yaml | 3 + .../sidb/singleinstancedatabase_create.yaml | 3 + .../sidb/singleinstancedatabase_patch.yaml | 3 + .../singleinstancedatabase_controller.go | 80 ++++++++++--------- 5 files changed, 55 insertions(+), 39 deletions(-) diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index 771d005c..5034993c 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -85,5 +85,8 @@ spec: ## If deploying on OpenShift, change service account name to 'sidb-sa' after you run `$ oc apply -f openshift_rbac.yaml` serviceAccountName: default - ## Count of Database Pods. Applicable only for "ReadWriteMany" AccessMode + ## Count of Database Pods. Only one pod will have the DB mounted and open. + ## The other replica pods will have instance up and will mount and open the DB if the primary pod dies + ## For "ReadWriteOnce" AccessMode, all the replicas will schedule on the same node that has the storage attached + ## Express edition can only have one replica replicas: 1 diff --git a/config/samples/sidb/singleinstancedatabase_clone.yaml b/config/samples/sidb/singleinstancedatabase_clone.yaml index dbe7a3dc..3a5d89cf 100644 --- a/config/samples/sidb/singleinstancedatabase_clone.yaml +++ b/config/samples/sidb/singleinstancedatabase_clone.yaml @@ -40,3 +40,6 @@ spec: size: 100Gi storageClass: "oci-bv" accessMode: "ReadWriteOnce" + + ## Count of Database Pods. + replicas: 1 diff --git a/config/samples/sidb/singleinstancedatabase_create.yaml b/config/samples/sidb/singleinstancedatabase_create.yaml index 81fc9a3e..a214624f 100644 --- a/config/samples/sidb/singleinstancedatabase_create.yaml +++ b/config/samples/sidb/singleinstancedatabase_create.yaml @@ -57,3 +57,6 @@ spec: size: 100Gi storageClass: "oci-bv" accessMode: "ReadWriteOnce" + + ## Count of Database Pods. + replicas: 1 diff --git a/config/samples/sidb/singleinstancedatabase_patch.yaml b/config/samples/sidb/singleinstancedatabase_patch.yaml index 704e1058..0eda6063 100644 --- a/config/samples/sidb/singleinstancedatabase_patch.yaml +++ b/config/samples/sidb/singleinstancedatabase_patch.yaml @@ -48,3 +48,6 @@ spec: size: 100Gi storageClass: "oci-bv" accessMode: "ReadWriteOnce" + + ## Count of Database Pods. + replicas: 1 diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 2d163cf6..a1df78ce 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1023,10 +1023,6 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex } log.Info("Found Existing Service ", "Service Name ", svc.Name) - //m.Status.ConnectString = dbcommons.ValueUnavailable - //m.Status.PdbConnectString = dbcommons.ValueUnavailable - //m.Status.OemExpressUrl = dbcommons.ValueUnavailable - pdbName := strings.ToUpper(m.Spec.Pdbname) sid := m.Spec.Sid if m.Spec.Image.PrebuiltDB { @@ -1072,7 +1068,7 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplacePods(m *dbapi.SingleIn oldImage := "" // call FindPods() to fetch pods all version/images of the same SIDB kind - readyPod, replicasFound, available, podsMarkedToBeDeleted, err := dbcommons.FindPods(r, "", "", m.Name, m.Namespace, ctx, req) + readyPod, replicasFound, allAvailable, podsMarkedToBeDeleted, err := dbcommons.FindPods(r, "", "", m.Name, m.Namespace, ctx, req) if err != nil { log.Error(err, err.Error()) return requeueY, err @@ -1082,10 +1078,10 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplacePods(m *dbapi.SingleIn return requeueY, err } if readyPod.Name != "" { - available = append(available, readyPod) + allAvailable = append(allAvailable, readyPod) } - for _, pod := range available { + for _, pod := range allAvailable { if pod.Labels["version"] != m.Spec.Image.Version { oldVersion = pod.Labels["version"] } @@ -1117,63 +1113,76 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplacePods(m *dbapi.SingleIn eventMsg = "from " + strconv.Itoa(replicasFound) + " pods to " + strconv.Itoa(m.Spec.Replicas) r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) // Delete extra PODs - return r.deletePods(ctx, req, m, available, readyPod, replicasFound, m.Spec.Replicas) + return r.deletePods(ctx, req, m, allAvailable, readyPod, replicasFound, m.Spec.Replicas) } // Version/Image changed // PATCHING START (Only Software Patch) + + // call FindPods() to find pods of older version. Delete all the Pods + readyPod, oldReplicasFound, oldAvailable, _, err := dbcommons.FindPods(r, oldVersion, + oldImage, m.Name, m.Namespace, ctx, req) + if err != nil { + log.Error(err, err.Error()) + return requeueY, err + } + if readyPod.Name != "" { + log.Info("Ready pod marked for deletion", "name", readyPod.Name) + oldAvailable = append(oldAvailable, readyPod) + } + + if m.Spec.Edition == "express" { + r.deletePods(ctx, req, m, oldAvailable, corev1.Pod{}, oldReplicasFound, 0) + } + // call FindPods() to find pods of newer version . if running , delete the older version replicas. - readyPod, replicasFound, available, _, err = dbcommons.FindPods(r, m.Spec.Image.Version, + readyPod, newReplicasFound, newAvailable, _, err := dbcommons.FindPods(r, m.Spec.Image.Version, m.Spec.Image.PullFrom, m.Name, m.Namespace, ctx, req) if err != nil { log.Error(err, err.Error()) return requeueY, nil } + // Findpods() only returns non ready pods + if readyPod.Name != "" { + log.Info("New ready pod found", "name", readyPod.Name) + newAvailable = append(newAvailable, readyPod) + } // create new Pods with the new Version and no.of Replicas required - result, err := r.createPods(m, n, ctx, req, replicasFound) + result, err := r.createPods(m, n, ctx, req, newReplicasFound) if result.Requeue { return result, err } - // Findpods() only returns non ready pods - if readyPod.Name != "" { - log.Info("New ready pod found", "name", readyPod.Name) - available = append(available, readyPod) - } - if ok, _ := dbcommons.IsAnyPodWithStatus(available, corev1.PodRunning); !ok { + if ok, _ := dbcommons.IsAnyPodWithStatus(newAvailable, corev1.PodRunning); !ok { eventReason := "Database Pending" eventMsg := "waiting for newer version/image DB pods get to running state" r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) log.Info(eventMsg) - for i := 0; i < len(available); i++ { - waitingReason := available[i].Status.InitContainerStatuses[i].State.Waiting.Reason + for i := 0; i < len(newAvailable); i++ { + waitingReason := "" + if (len(newAvailable[i].Status.InitContainerStatuses) > 0) { + waitingReason = newAvailable[i].Status.InitContainerStatuses[0].State.Waiting.Reason + } else { + waitingReason = newAvailable[i].Status.ContainerStatuses[0].State.Waiting.Reason + } r.Log.Info("Pod unavailable reason: ", "reason", waitingReason) if strings.Contains(waitingReason, "ImagePullBackOff") || strings.Contains(waitingReason, "ErrImagePull") { - r.Log.Info("Deleting pod", "name", available[0].Name) + r.Log.Info("Deleting pod", "name", newAvailable[i].Name) var gracePeriodSeconds int64 = 0 policy := metav1.DeletePropagationForeground - r.Delete(ctx, &available[i], &client.DeleteOptions{ + r.Delete(ctx, &newAvailable[i], &client.DeleteOptions{ GracePeriodSeconds: &gracePeriodSeconds, PropagationPolicy: &policy}) } } return requeueY, errors.New(eventMsg) } - - // call FindPods() to find pods of older version . delete all the Pods - readyPod, replicasFound, available, _, err = dbcommons.FindPods(r, oldVersion, - oldImage, m.Name, m.Namespace, ctx, req) - if err != nil { - log.Error(err, err.Error()) - return requeueY, err - } - if readyPod.Name != "" { - log.Info("Ready pod marked for deletion", "name", readyPod.Name) - available = append(available, readyPod) + if m.Spec.Edition == "express" { + return requeueN, nil } - return r.deletePods(ctx, req, m, available, corev1.Pod{}, replicasFound, 0) + return r.deletePods(ctx, req, m, oldAvailable, corev1.Pod{}, oldReplicasFound, 0) // PATCHING END } @@ -1313,11 +1322,6 @@ func (r *SingleInstanceDatabaseReconciler) createPods(m *dbapi.SingleInstanceDat m.Status.Status = dbcommons.StatusPending m.Status.DatafilesCreated = "false" m.Status.DatafilesPatched = "false" - //m.Status.Role = dbcommons.ValueUnavailable - //m.Status.ConnectString = dbcommons.ValueUnavailable - //m.Status.PdbConnectString = dbcommons.ValueUnavailable - //m.Status.OemExpressUrl = dbcommons.ValueUnavailable - //m.Status.ReleaseUpdate = dbcommons.ValueUnavailable } // if Found < Required , Create New Pods , Name of Pods are generated Randomly for i := replicasFound; i < replicasReq; i++ { @@ -1375,7 +1379,7 @@ func (r *SingleInstanceDatabaseReconciler) deletePods(ctx context.Context, req c } } - // For deleting all pods , call with readyPod as nil ( corev1.Pod{} ) and append readyPod to avaiable while calling deletePods() + // For deleting all pods , call with readyPod as nil ( corev1.Pod{} ) and append readyPod to available while calling deletePods() // if Found > Required , Delete Extra Pods if replicasFound > len(available) { // if available does not contain readyPOD, add it From 7e784e02f2e2c5b7023cfd1c86abba390ce16383 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Sun, 1 May 2022 15:52:04 +0530 Subject: [PATCH 318/628] Default keepSecret --- .../v1alpha1/singleinstancedatabase_types.go | 2 +- .../v1alpha1/singleinstancedatabase_webhook.go | 5 +++-- config/samples/sidb/singleinstancedatabase.yaml | 15 ++++++++------- .../sidb/singleinstancedatabase_clone.yaml | 2 -- .../sidb/singleinstancedatabase_create.yaml | 2 -- .../sidb/singleinstancedatabase_express.yaml | 2 -- .../sidb/singleinstancedatabase_patch.yaml | 2 -- .../sidb/singleinstancedatabase_prebuiltdb.yaml | 2 -- .../database/singleinstancedatabase_controller.go | 15 +++++++++------ 9 files changed, 21 insertions(+), 26 deletions(-) diff --git a/apis/database/v1alpha1/singleinstancedatabase_types.go b/apis/database/v1alpha1/singleinstancedatabase_types.go index b35b117b..3919738c 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_types.go +++ b/apis/database/v1alpha1/singleinstancedatabase_types.go @@ -105,7 +105,7 @@ type SingleInstanceDatabaseImage struct { type SingleInstanceDatabaseAdminPassword struct { SecretName string `json:"secretName"` SecretKey string `json:"secretKey"` - KeepSecret bool `json:"keepSecret,omitempty"` + KeepSecret *bool `json:"keepSecret,omitempty"` } // SingleInstanceDatabaseStatus defines the observed state of SingleInstanceDatabase diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index 652d1f70..96c6e683 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -71,8 +71,9 @@ var _ webhook.Defaulter = &SingleInstanceDatabase{} func (r *SingleInstanceDatabase) Default() { singleinstancedatabaselog.Info("default", "name", r.Name) - if r.Spec.Replicas == 0 { - r.Spec.Replicas = 1 + if r.Spec.AdminPassword.KeepSecret == nil { + keepSecret := true + r.Spec.AdminPassword.KeepSecret = &keepSecret } if r.Spec.Edition == "express" { diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index 5034993c..e4e8c53a 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -12,16 +12,17 @@ spec: ## Use only alphanumeric characters for sid sid: ORCL1 - ## A source database ref to clone from, leave empty to create a fresh database - ## If cloning from on-prem source database, mention connect string `:/`. + ## Specify a source database ref to copy/clone from any SIDB in current K8s cluster instead of creating a fresh one. + ## If cloning from an external containerized DB which could be either standalone or in any K8s cluster, + ## specify connect string as `:/` instead of source database ref cloneFrom: "" ## DB edition. N/A if cloning from a SourceDB (cloneFrom is set) edition: enterprise - ## Should refer to SourceDB secret if cloning from a SourceDB (cloneFrom is set) + ## Should refer to SourceDB secret if cloning from a Source DB (cloneFrom is set) ## Secret containing SIDB password mapped to secretKey - ## This secret will be deleted after creation of the database unless keepSecret is set to true + ## This secret will be deleted after creation of the database unless keepSecret is set to true which is the default adminPassword: secretName: secretKey: @@ -54,10 +55,9 @@ spec: ## Database image details ## Base DB images are available at container-registry.oracle.com or build from https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance ## Build patched DB images from https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/patching - ## If cloning from on-prem DB, build on-prem DB image following the instructions at - ## https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance#containerizing-an-on-premise-database-supported-from-version-1930-release ## Prebuilt DB support (https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/prebuiltdb) ## Specify prebuiltDB as true if the image includes a prebuilt DB + ## If cloning specify an image that is of same major version as the source DB at same or different patch levels. image: pullFrom: pullSecrets: @@ -88,5 +88,6 @@ spec: ## Count of Database Pods. Only one pod will have the DB mounted and open. ## The other replica pods will have instance up and will mount and open the DB if the primary pod dies ## For "ReadWriteOnce" AccessMode, all the replicas will schedule on the same node that has the storage attached - ## Express edition can only have one replica + ## For minimal downtime during patching set the count of replicas > 1 + ## Express edition can only have one replica and does not support patching replicas: 1 diff --git a/config/samples/sidb/singleinstancedatabase_clone.yaml b/config/samples/sidb/singleinstancedatabase_clone.yaml index 3a5d89cf..acf6a4c2 100644 --- a/config/samples/sidb/singleinstancedatabase_clone.yaml +++ b/config/samples/sidb/singleinstancedatabase_clone.yaml @@ -19,11 +19,9 @@ spec: ## Should refer to SourceDB secret ## Secret containing SIDB password mapped to secretKey - ## This secret will be deleted after creation of the database unless keepSecret is set to true adminPassword: secretName: db-admin-secret secretKey: oracle_pwd - keepSecret: true ## Database image details ## This image should be the same as the source DB image being cloned diff --git a/config/samples/sidb/singleinstancedatabase_create.yaml b/config/samples/sidb/singleinstancedatabase_create.yaml index a214624f..a8f7430f 100644 --- a/config/samples/sidb/singleinstancedatabase_create.yaml +++ b/config/samples/sidb/singleinstancedatabase_create.yaml @@ -30,11 +30,9 @@ spec: edition: enterprise ## Secret containing SIDB password mapped to secretKey - ## This secret will be deleted after creation of the database unless keepSecret is set to true adminPassword: secretName: db-admin-secret secretKey: oracle_pwd - keepSecret: true ## DB character set charset: AL32UTF8 diff --git a/config/samples/sidb/singleinstancedatabase_express.yaml b/config/samples/sidb/singleinstancedatabase_express.yaml index 02840981..175d6290 100644 --- a/config/samples/sidb/singleinstancedatabase_express.yaml +++ b/config/samples/sidb/singleinstancedatabase_express.yaml @@ -28,11 +28,9 @@ spec: edition: express ## Secret containing SIDB password mapped to secretKey - ## This secret will be deleted after creation of the database unless keepSecret is set to true adminPassword: secretName: xedb-admin-secret secretKey: oracle_pwd - keepSecret: true ## Database image details image: diff --git a/config/samples/sidb/singleinstancedatabase_patch.yaml b/config/samples/sidb/singleinstancedatabase_patch.yaml index 0eda6063..cdb676c7 100644 --- a/config/samples/sidb/singleinstancedatabase_patch.yaml +++ b/config/samples/sidb/singleinstancedatabase_patch.yaml @@ -18,11 +18,9 @@ spec: edition: enterprise ## Secret containing SIDB password mapped to secretKey - ## This secret will be deleted after creation of the database unless keepSecret is set to true adminPassword: secretName: db-admin-secret secretKey: oracle_pwd - keepSecret: true ## DB character set charset: AL32UTF8 diff --git a/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml b/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml index c22b14f3..33ed1316 100644 --- a/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml +++ b/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml @@ -28,11 +28,9 @@ spec: edition: express ## Secret containing SIDB password mapped to secretKey - ## This secret will be deleted after creation of the database unless keepSecret is set to true adminPassword: secretName: prebuiltdb-admin-secret secretKey: oracle_pwd - keepSecret: true ## Database Image image: diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index a1df78ce..d9098ab4 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1073,7 +1073,7 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplacePods(m *dbapi.SingleIn log.Error(err, err.Error()) return requeueY, err } - if m.Spec.Edition == "express" && podsMarkedToBeDeleted > 0 { + if m.Spec.Replicas == 1 && podsMarkedToBeDeleted > 0 { // Recreate new pods only after earlier pods are terminated completely return requeueY, err } @@ -1131,7 +1131,7 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplacePods(m *dbapi.SingleIn oldAvailable = append(oldAvailable, readyPod) } - if m.Spec.Edition == "express" { + if m.Spec.Replicas == 1 { r.deletePods(ctx, req, m, oldAvailable, corev1.Pod{}, oldReplicasFound, 0) } @@ -1161,12 +1161,16 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplacePods(m *dbapi.SingleIn log.Info(eventMsg) for i := 0; i < len(newAvailable); i++ { + r.Log.Info("Pod status: ", "name", newAvailable[i].Name, "phase", newAvailable[i].Status.Phase) waitingReason := "" if (len(newAvailable[i].Status.InitContainerStatuses) > 0) { waitingReason = newAvailable[i].Status.InitContainerStatuses[0].State.Waiting.Reason - } else { + } else if len(newAvailable[i].Status.ContainerStatuses) > 0 { waitingReason = newAvailable[i].Status.ContainerStatuses[0].State.Waiting.Reason } + if waitingReason == "" { + continue + } r.Log.Info("Pod unavailable reason: ", "reason", waitingReason) if strings.Contains(waitingReason, "ImagePullBackOff") || strings.Contains(waitingReason, "ErrImagePull") { r.Log.Info("Deleting pod", "name", newAvailable[i].Name) @@ -1176,10 +1180,9 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplacePods(m *dbapi.SingleIn GracePeriodSeconds: &gracePeriodSeconds, PropagationPolicy: &policy}) } } - return requeueY, errors.New(eventMsg) } - if m.Spec.Edition == "express" { + if m.Spec.Replicas == 1 { return requeueN, nil } return r.deletePods(ctx, req, m, oldAvailable, corev1.Pod{}, oldReplicasFound, 0) @@ -1510,7 +1513,7 @@ func (r *SingleInstanceDatabaseReconciler) deleteWallet(m *dbapi.SingleInstanceD // Deleting the secret and then deleting the wallet // If the secret is not found it means that the secret and wallet both are deleted, hence no need to requeue - if !m.Spec.AdminPassword.KeepSecret { + if !*m.Spec.AdminPassword.KeepSecret { r.Log.Info("Querying the database secret ...") secret := &corev1.Secret{} err := r.Get(ctx, types.NamespacedName{Name: m.Spec.AdminPassword.SecretName, Namespace: m.Namespace}, secret) From 12a0f4a893001a5c4efdc1cf5263c487d8e4e29b Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Sun, 1 May 2022 17:38:37 +0530 Subject: [PATCH 319/628] Fix pod relocation --- .../sidb/singleinstancedatabase_express.yaml | 3 +++ .../sidb/singleinstancedatabase_prebuiltdb.yaml | 3 +++ .../database/singleinstancedatabase_controller.go | 13 +++++++++++-- 3 files changed, 17 insertions(+), 2 deletions(-) diff --git a/config/samples/sidb/singleinstancedatabase_express.yaml b/config/samples/sidb/singleinstancedatabase_express.yaml index 175d6290..3f1a3927 100644 --- a/config/samples/sidb/singleinstancedatabase_express.yaml +++ b/config/samples/sidb/singleinstancedatabase_express.yaml @@ -43,3 +43,6 @@ spec: size: 50Gi storageClass: "oci-bv" accessMode: "ReadWriteOnce" + + ## Count of Database Pods. Should be 1 for express edition. + replicas: 1 diff --git a/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml b/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml index 33ed1316..ae0e15df 100644 --- a/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml +++ b/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml @@ -43,3 +43,6 @@ spec: # size: 50Gi # storageClass: "oci-bv" # accessMode: "ReadWriteOnce" + + ## Count of Database Pods. + replicas: 1 diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index d9098ab4..ccfeb310 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1073,9 +1073,18 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplacePods(m *dbapi.SingleIn log.Error(err, err.Error()) return requeueY, err } - if m.Spec.Replicas == 1 && podsMarkedToBeDeleted > 0 { + if podsMarkedToBeDeleted > 0 { // Recreate new pods only after earlier pods are terminated completely - return requeueY, err + for i := 0; i < len(allAvailable); i++ { + if allAvailable[i].ObjectMeta.DeletionTimestamp != nil { + r.Log.Info("Force deleting pod ", "name", allAvailable[i].Name, "phase", allAvailable[i].Status.Phase) + var gracePeriodSeconds int64 = 0 + policy := metav1.DeletePropagationForeground + r.Delete(ctx, &allAvailable[i], &client.DeleteOptions{ + GracePeriodSeconds: &gracePeriodSeconds, PropagationPolicy: &policy }) + } + } + return requeueY, nil } if readyPod.Name != "" { allAvailable = append(allAvailable, readyPod) From ee9efd476a44d51b08568633bc1d783e7a832698 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Sun, 1 May 2022 18:12:04 +0530 Subject: [PATCH 320/628] Add debug msgs --- controllers/database/singleinstancedatabase_controller.go | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index ccfeb310..27f9486e 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1076,7 +1076,9 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplacePods(m *dbapi.SingleIn if podsMarkedToBeDeleted > 0 { // Recreate new pods only after earlier pods are terminated completely for i := 0; i < len(allAvailable); i++ { - if allAvailable[i].ObjectMeta.DeletionTimestamp != nil { + r.Log.Info("This pod ", "name", allAvailable[i].Name, "phase", allAvailable[i].Status.Phase) + r.Log.Info("This pod ", "name", allAvailable[i].Name, "status", allAvailable[i].Status) + if strings.Contains(allAvailable[i].Status.Message, "Terminating") { r.Log.Info("Force deleting pod ", "name", allAvailable[i].Name, "phase", allAvailable[i].Status.Phase) var gracePeriodSeconds int64 = 0 policy := metav1.DeletePropagationForeground From f4e01b75411a419f7b2f17cc961aa3bc88106acb Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Sun, 1 May 2022 18:33:19 +0530 Subject: [PATCH 321/628] Fix pod deletion --- commons/database/utils.go | 8 +++---- .../singleinstancedatabase_controller.go | 23 ++++++++----------- 2 files changed, 13 insertions(+), 18 deletions(-) diff --git a/commons/database/utils.go b/commons/database/utils.go index 875e392b..90b2256b 100644 --- a/commons/database/utils.go +++ b/commons/database/utils.go @@ -295,13 +295,14 @@ func GenerateRandomString(n int) string { // retuns Ready Pod,No of replicas ( Only running and Pending Pods) ,available pods , Total No of Pods of a particular CRD func FindPods(r client.Reader, version string, image string, name string, namespace string, ctx context.Context, - req ctrl.Request) (corev1.Pod, int, []corev1.Pod, int, error) { + req ctrl.Request) (corev1.Pod, int, []corev1.Pod, []corev1.Pod, error) { log := ctrllog.FromContext(ctx).WithValues("FindPods", req.NamespacedName) // "available" stores list of pods which can be deleted while scaling down i.e the pods other than one of Ready Pods // There are multiple ready pods possible in OracleRestDataService , while others have atmost one readyPod var available []corev1.Pod + var podsMarkedToBeDeleted []corev1.Pod var readyPod corev1.Pod // To Store the Ready Pod ( Pod that Passed Readiness Probe . Will be shown as 1/1 Running ) podList := &corev1.PodList{} @@ -310,18 +311,17 @@ func FindPods(r client.Reader, version string, image string, name string, namesp // List retrieves list of objects for a given namespace and list options. if err := r.List(ctx, podList, listOpts...); err != nil { log.Error(err, "Failed to list pods of "+name, "Namespace", namespace, "Name", name) - return readyPod, 0, available, 0, err + return readyPod, 0, available, podsMarkedToBeDeleted, err } // r.List() lists all the pods in running, pending,terminating stage matching listOpts . so filter them // Fetch the Running and Pending Pods - podsMarkedToBeDeleted := 0 for _, pod := range podList.Items { // Return pods having Image = image (or) if image = ""(Needed in case when called findpods with "" image) if pod.Spec.Containers[0].Image == image || image == "" { if pod.ObjectMeta.DeletionTimestamp != nil { - podsMarkedToBeDeleted += 1 + podsMarkedToBeDeleted = append(podsMarkedToBeDeleted, pod) continue } if pod.Status.Phase == corev1.PodRunning || pod.Status.Phase == corev1.PodPending { diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 27f9486e..17403e01 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1073,21 +1073,16 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplacePods(m *dbapi.SingleIn log.Error(err, err.Error()) return requeueY, err } - if podsMarkedToBeDeleted > 0 { - // Recreate new pods only after earlier pods are terminated completely - for i := 0; i < len(allAvailable); i++ { - r.Log.Info("This pod ", "name", allAvailable[i].Name, "phase", allAvailable[i].Status.Phase) - r.Log.Info("This pod ", "name", allAvailable[i].Name, "status", allAvailable[i].Status) - if strings.Contains(allAvailable[i].Status.Message, "Terminating") { - r.Log.Info("Force deleting pod ", "name", allAvailable[i].Name, "phase", allAvailable[i].Status.Phase) - var gracePeriodSeconds int64 = 0 - policy := metav1.DeletePropagationForeground - r.Delete(ctx, &allAvailable[i], &client.DeleteOptions{ - GracePeriodSeconds: &gracePeriodSeconds, PropagationPolicy: &policy }) - } - } - return requeueY, nil + + // Recreate new pods only after earlier pods are terminated completely + for i := 0; i < len(podsMarkedToBeDeleted); i++ { + r.Log.Info("Force deleting pod ", "name", allAvailable[i].Name, "phase", allAvailable[i].Status.Phase) + var gracePeriodSeconds int64 = 0 + policy := metav1.DeletePropagationForeground + r.Delete(ctx, &allAvailable[i], &client.DeleteOptions{ + GracePeriodSeconds: &gracePeriodSeconds, PropagationPolicy: &policy }) } + if readyPod.Name != "" { allAvailable = append(allAvailable, readyPod) } From 21c64eaef41e4518e467ba0c71160e7de5ac52c8 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Sun, 1 May 2022 18:58:52 +0530 Subject: [PATCH 322/628] Fix index exception --- controllers/database/singleinstancedatabase_controller.go | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 17403e01..6882710d 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1076,10 +1076,10 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplacePods(m *dbapi.SingleIn // Recreate new pods only after earlier pods are terminated completely for i := 0; i < len(podsMarkedToBeDeleted); i++ { - r.Log.Info("Force deleting pod ", "name", allAvailable[i].Name, "phase", allAvailable[i].Status.Phase) + r.Log.Info("Force deleting pod ", "name", podsMarkedToBeDeleted[i].Name, "phase", podsMarkedToBeDeleted[i].Status.Phase) var gracePeriodSeconds int64 = 0 policy := metav1.DeletePropagationForeground - r.Delete(ctx, &allAvailable[i], &client.DeleteOptions{ + r.Delete(ctx, &podsMarkedToBeDeleted[i], &client.DeleteOptions{ GracePeriodSeconds: &gracePeriodSeconds, PropagationPolicy: &policy }) } From f14947c478c44a66ae30d55da1a2dad2a5419497 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Sun, 1 May 2022 23:16:30 +0530 Subject: [PATCH 323/628] Doc changes --- config/samples/sidb/singleinstancedatabase.yaml | 5 ++--- controllers/database/oraclerestdataservice_controller.go | 2 +- 2 files changed, 3 insertions(+), 4 deletions(-) diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index e4e8c53a..0ba28d8d 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -20,8 +20,8 @@ spec: ## DB edition. N/A if cloning from a SourceDB (cloneFrom is set) edition: enterprise - ## Should refer to SourceDB secret if cloning from a Source DB (cloneFrom is set) ## Secret containing SIDB password mapped to secretKey + ## Should refer to adminPassword of Source DB if cloning from a Source DB (i.e if cloneFrom is set) ## This secret will be deleted after creation of the database unless keepSecret is set to true which is the default adminPassword: secretName: @@ -76,8 +76,7 @@ spec: loadBalancer: false ## Deploy only on nodes having required labels. Format label_name : label_value - ## The same lables are applied to the created PVC - ## For instance if the Pods and storage need to be restricted to a particular AD + ## For instance if the Pods need to be restricted to a particular AD ## Leave commented if there is no such requirement. # nodeSelector: # failure-domain.beta.kubernetes.io/zone: PHX-AD-1 diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 765466eb..6076fbf6 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -226,7 +226,7 @@ func (r *OracleRestDataServiceReconciler) validate(m *dbapi.OracleRestDataServic } // If using same pvc for ords as sidb, ensure sidb has ReadWriteMany Accessmode - if n.Spec.Persistence.AccessMode == "ReadWriteOnce" && m.Spec.Persistence.AccessMode == "" { + if n.Spec.Persistence.AccessMode == "ReadWriteOnce" && m.Spec.Persistence.Size == "" { eventMsgs = append(eventMsgs, "ords can be installed only on ReadWriteMany Access Mode of : "+m.Spec.DatabaseRef) } if m.Status.DatabaseRef != "" && m.Status.DatabaseRef != m.Spec.DatabaseRef { From f8d58b9151a53130b26a8fc005f2aeec73525979 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Sun, 1 May 2022 23:41:25 +0530 Subject: [PATCH 324/628] ORDS controller refactor --- .../oraclerestdataservice_controller.go | 62 +++++++------------ 1 file changed, 23 insertions(+), 39 deletions(-) diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 6076fbf6..96afb4a7 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -894,20 +894,6 @@ func (r *OracleRestDataServiceReconciler) cleanupOracleRestDataService(req ctrl. m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase) error { log := r.Log.WithValues("cleanupOracleRestDataService", req.NamespacedName) - readyPod, _, _, _, err := dbcommons.FindPods(r, m.Spec.Image.Version, - m.Spec.Image.PullFrom, m.Name, m.Namespace, ctx, req) - if err != nil { - log.Error(err, err.Error()) - return err - } - - if readyPod.Name == "" { - eventReason := "Waiting" - eventMsg := "waiting for " + m.Name + " to be Ready" - r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) - err = errors.New(eventMsg) - return err - } if m.Status.OrdsInstalled { // ## FETCH THE SIDB REPLICAS . @@ -976,28 +962,36 @@ func (r *OracleRestDataServiceReconciler) cleanupOracleRestDataService(req ctrl. break } } - if !adminPasswordSecretFound { - log.Info("AdminPassword Secret not found . Omitting OracleRestDataService uninstallation") - return nil + // Find ORDS ready pod + readyPod, _, _, _, err := dbcommons.FindPods(r, m.Spec.Image.Version, + m.Spec.Image.PullFrom, m.Name, m.Namespace, ctx, req) + if err != nil { + log.Error(err, err.Error()) + return err } + if adminPasswordSecretFound && readyPod.Name != "" { + adminPassword := string(adminPasswordSecret.Data[m.Spec.AdminPassword.SecretKey]) + uninstallORDS := fmt.Sprintf(dbcommons.UninstallORDSCMD, adminPassword) - adminPassword := string(adminPasswordSecret.Data[m.Spec.AdminPassword.SecretKey]) - uninstallORDS := fmt.Sprintf(dbcommons.UninstallORDSCMD, adminPassword) - - out, err = dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, true, "bash", "-c", + out, err = dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, true, "bash", "-c", uninstallORDS) - log.Info("UninstallORDSCMD Output : " + out) - if strings.Contains(strings.ToUpper(out), "ERROR") { - return errors.New(out) + log.Info("UninstallORDSCMD Output : " + out) + if strings.Contains(strings.ToUpper(out), "ERROR") { + return errors.New(out) + } + if err != nil { + log.Info(err.Error()) + } + log.Info("UninstallORDSCMD Output : " + out) } + + // Drop Admin Users + out, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | %s ", dbcommons.DropAdminUsersSQL, dbcommons.SQLPlusCLI)) if err != nil { log.Info(err.Error()) - if strings.Contains(strings.ToUpper(err.Error()), "ERROR") { - return err - } } - - log.Info("UninstallORDSCMD Output : " + out) + log.Info("DropAdminUsersSQL Output : " + out) //Delete ORDS pod var gracePeriodSeconds int64 = 0 @@ -1012,16 +1006,6 @@ func (r *OracleRestDataServiceReconciler) cleanupOracleRestDataService(req ctrl. r.Log.Info("Deleted Admin Password Secret :" + adminPasswordSecret.Name) } } - - // Drop Admin Users - out, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf("echo -e \"%s\" | %s ", dbcommons.DropAdminUsersSQL, dbcommons.SQLPlusCLI)) - if err != nil { - log.Error(err, err.Error()) - return err - } - log.Info("DropAdminUsersSQL Output : " + out) - } // Cleanup steps that the operator needs to do before the CR can be deleted. From 5364ba8b3d01d10f33b20bd9678e188bf8bcb2b3 Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Mon, 2 May 2022 15:10:55 +0530 Subject: [PATCH 325/628] Readme changes Signed-off-by: abhisbyk --- docs/sidb/README.md | 721 ++++++++++++++++++++++---------------------- 1 file changed, 356 insertions(+), 365 deletions(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 9b04bbf8..74fc51d2 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -1,6 +1,6 @@ # Managing Oracle Single Instance Databases with Oracle Database Operator for Kubernetes -Oracle Database Operator for Kubernetes (the operator) includes the Single Instance Database Controller that enables provisioning, cloning, and patching of Oracle Single Instance Databases on Kubernetes. The following sections explain the setup and functionality of the operator +Oracle Database Operator for Kubernetes (a.k.a. OraOperator) includes the Single Instance Database Controller that enables provisioning, cloning, and patching of Oracle Single Instance Databases on Kubernetes. The following sections explain the setup and functionality of the operator * [Prerequisites](#prerequisites) * [Kind SingleInstanceDatabase Resource](#kind-singleinstancedatabase-resource) @@ -17,402 +17,400 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI ## Kind SingleInstanceDatabase Resource - The Oracle Database Operator creates the SingleInstanceDatabase kind as a custom resource that enables Oracle Database to be managed as a native Kubernetes object + The Oracle Database Operator creates the SingleInstanceDatabase kind as a custom resource that enables Oracle Database to be managed as a native Kubernetes object. -* ### SingleInstanceDatabase Sample YAML +### SingleInstanceDatabase template YAML - For the use cases detailed below a sample .yaml file is available at - * Enterprise, Standard Editions - [config/samples/sidb/singleinstancedatabase.yaml](./../../config/samples/sidb/singleinstancedatabase.yaml) +The template `.yaml` file for Single Instance Database (Enterprise & Standard Editions) including all the configurable options is as follows: +[config/samples/sidb/singleinstancedatabase.yaml](./../../config/samples/sidb/singleinstancedatabase.yaml) - **Note:** The `adminPassword` field of the above `singleinstancedatabase.yaml` yaml contains a secret for Single Instance Database creation (Provisioning a new database or cloning an existing database). This secret gets deleted after the database pod becomes ready for security reasons. +**Note:** +The `adminPassword` field of the above `singleinstancedatabase.yaml` file refers to a secret for SYS, SYSTEM and PDBADMIN users of the Single Instance Database. It is required while provisioning a new database or cloning an existing one. - More info on creating Kubernetes Secret available at [https://kubernetes.io/docs/tasks/configmap-secret/managing-secret-using-kubectl/](https://kubernetes.io/docs/tasks/configmap-secret/managing-secret-using-kubectl/) +This secret can be created using the following sample command: -* ### List Databases + kubectl create secret generic pwd-secret --from-literal=oracle_pwd="SamplePWD#123" - ```sh - $ kubectl get singleinstancedatabases -o name +The above command will create a secret having name as `pwd-secret` with key `oracle_pwd` mapped to the actual password specified in the command. - singleinstancedatabase.database.oracle.com/sidb-sample - singleinstancedatabase.database.oracle.com/sidb-sample-clone +### List Databases - ``` +```sh +$ kubectl get singleinstancedatabases -o name -* ### Quick Status - - ```sh - $ kubectl get singleinstancedatabase sidb-sample + singleinstancedatabase.database.oracle.com/sidb-sample + singleinstancedatabase.database.oracle.com/sidb-sample-clone - NAME EDITION STATUS ROLE VERSION CLUSTER CONNECT STR CONNECT STR OEM EXPRESS URL - sidb-sample Enterprise Healthy PRIMARY 19.3.0.0.0 (29517242) sidb-sample.default:1521/ORCL1 10.0.25.54:1521/ORCL https://10.0.25.54:5500/em - ``` +``` + +### Quick Status + +```sh +$ kubectl get singleinstancedatabase sidb-sample -* ### Detailed Status +NAME EDITION STATUS VERSION CONNECT STR OEM EXPRESS URL +sidb-sample Enterprise Healthy 19.3.0.0.0 10.0.25.54:1521/ORCLCDB https://10.0.25.54:5500/em +``` - ```sh - $ kubectl describe singleinstancedatabase sidb-sample-clone - - Name: sidb-sample-clone - Namespace: default - Labels: - Annotations: - API Version: database.oracle.com/v1alpha1 - Kind: SingleInstanceDatabase - Metadata: .... - Spec: .... - Status: - Cluster Connect String: sidb-sample-clone.default:1521/ORCL1C - Conditions: - Last Transition Time: 2021-06-29T15:45:33Z - Message: Waiting for database to be ready - Observed Generation: 2 - Reason: LastReconcileCycleQueued - Status: True - Type: ReconcileQueued - Last Transition Time: 2021-06-30T11:07:56Z - Message: processing datapatch execution - Observed Generation: 3 - Reason: LastReconcileCycleBlocked - Status: True - Type: ReconcileBlocked - Last Transition Time: 2021-06-30T11:16:58Z - Message: no reconcile errors - Observed Generation: 3 - Reason: LastReconcileCycleCompleted - Status: True - Type: ReconcileComplete - Connect String: 10.0.25.54:1521/ORCL1 - Datafiles Created: true - Datafiles Patched: true - Edition: Enterprise - Flash Back: true - Force Log: false - Oem Express URL: https://10.0.25.54:5500/em - Pdb Name: orclpdb1 - Release Update: 19.11.0.0.0 (32545013) - Replicas: 2 - Role: PRIMARY - Sid: ORCL1C - Status: Healthy - Events: - Type Reason Age From Message - ---- ------ ---- ---- ------- - Normal Database Pending 35m (x2 over 35m) SingleInstanceDatabase Waiting for database pod to be ready - Normal Database Creating 27m (x24 over 34m) SingleInstanceDatabase Waiting for database to be ready - Normal Database Ready 22m SingleInstanceDatabase database open on pod sidb-sample-clone-133ol scheduled on node 10.0.10.6 - Normal Datapatch Pending 21m SingleInstanceDatabase datapatch execution pending - Normal Datapatch Executing 20m SingleInstanceDatabase datapatch begin execution - Normal Datapatch Done 8s SingleInstanceDatabase Datapatch from 19.3.0.0.0 to 19.11.0.0.0 : SUCCESS +### Detailed Status + +```sh +$ kubectl describe singleinstancedatabase sidb-sample-clone + + Name: sidb-sample-clone + Namespace: default + Labels: + Annotations: + API Version: database.oracle.com/v1alpha1 + Kind: SingleInstanceDatabase + Metadata: .... + Spec: .... + Status: + Cluster Connect String: sidb-sample-clone.default:1521/ORCL1C + Conditions: + Last Transition Time: 2021-06-29T15:45:33Z + Message: Waiting for database to be ready + Observed Generation: 2 + Reason: LastReconcileCycleQueued + Status: True + Type: ReconcileQueued + Last Transition Time: 2021-06-30T11:07:56Z + Message: processing datapatch execution + Observed Generation: 3 + Reason: LastReconcileCycleBlocked + Status: True + Type: ReconcileBlocked + Last Transition Time: 2021-06-30T11:16:58Z + Message: no reconcile errors + Observed Generation: 3 + Reason: LastReconcileCycleCompleted + Status: True + Type: ReconcileComplete + Connect String: 10.0.25.54:1521/ORCL1 + Datafiles Created: true + Datafiles Patched: true + Edition: Enterprise + Flash Back: true + Force Log: false + Oem Express URL: https://10.0.25.54:5500/em + Pdb Name: orclpdb1 + Release Update: 19.11.0.0.0 (32545013) + Replicas: 2 + Role: PRIMARY + Sid: ORCL1C + Status: Healthy + Events: + Type Reason Age From Message + ---- ------ ---- ---- ------- + Normal Database Pending 35m (x2 over 35m) SingleInstanceDatabase Waiting for database pod to be ready + Normal Database Creating 27m (x24 over 34m) SingleInstanceDatabase Waiting for database to be ready + Normal Database Ready 22m SingleInstanceDatabase database open on pod sidb-sample-clone-133ol scheduled on node 10.0.10.6 + Normal Datapatch Pending 21m SingleInstanceDatabase datapatch execution pending + Normal Datapatch Executing 20m SingleInstanceDatabase datapatch begin execution + Normal Datapatch Done 8s SingleInstanceDatabase Datapatch from 19.3.0.0.0 to 19.11.0.0.0 : SUCCESS - ``` +``` ## Provision New Database - - Easily provision a new database instance on the Kubernetes cluster using [singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml). + Easily provision a new database instance on the Kubernetes cluster using [singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml). - Sign into [Oracle Container Registry](https://container-registry.oracle.com/ords/f?p=113:4:7154182141811:::4:P4_REPOSITORY,AI_REPOSITORY,AI_REPOSITORY_NAME,P4_REPOSITORY_NAME,P4_EULA_ID,P4_BUSINESS_AREA_ID:9,9,Oracle%20Database%20Enterprise%20Edition,Oracle%20Database%20Enterprise%20Edition,1,0&cs=3Y_90hkCQLfJzrvTLiEipIGgWGUytfrtAPuHFocuWd0NDSacbBPlamohfLuiJA-bAsVL6Z_yKEMsTbb52bm6IRA) and accept the license agreement for the Database image, ignore if you have accepted already. + Firstly, sign into [Oracle Container Registry](https://container-registry.oracle.com/ords/f?p=113:4:7154182141811:::4:P4_REPOSITORY,AI_REPOSITORY,AI_REPOSITORY_NAME,P4_REPOSITORY_NAME,P4_EULA_ID,P4_BUSINESS_AREA_ID:9,9,Oracle%20Database%20Enterprise%20Edition,Oracle%20Database%20Enterprise%20Edition,1,0&cs=3Y_90hkCQLfJzrvTLiEipIGgWGUytfrtAPuHFocuWd0NDSacbBPlamohfLuiJA-bAsVL6Z_yKEMsTbb52bm6IRA) and accept the license agreement for the Database image, ignore if you have accepted already. - Create an image pull secret for Oracle Container Registry, ignore if you have created already: + Create an image pull secret for the Oracle Container Registry, ignore if you have created already: - ```sh - $ kubectl create secret docker-registry oracle-container-registry-secret --docker-server=container-registry.oracle.com --docker-username='' --docker-password='' --docker-email='' - - secret/oracle-container-registry-secret created - ``` + ```sh + $ kubectl create secret docker-registry oracle-container-registry-secret --docker-server=container-registry.oracle.com --docker-username='' --docker-password='' --docker-email='' + + secret/oracle-container-registry-secret created + ``` + This secret can also be created using the docker config file as follows: - Now, easily provision a new database instance on the cluster by using the following command. + kubectl create secret generic oracle-container-registry-secret --from-file=.dockerconfigjson=.docker/config.json --type=kubernetes.io/dockerconfigjson - ```sh - $ kubectl create -f singleinstancedatabase_create.yaml - - singleinstancedatabase.database.oracle.com/sidb-sample created - ``` + Now, easily provision a new database instance on the cluster by using the following command. - - Provision a new database instance on any K8s cluster by specifying appropriate values for the attributes in the [singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml) and running the following command: + ```sh + $ kubectl create -f singleinstancedatabase_create.yaml - ```sh - $ kubectl create -f singleinstancedatabase.yaml - - singleinstancedatabase.database.oracle.com/sidb-sample created - ``` + singleinstancedatabase.database.oracle.com/sidb-sample created + ``` - **NOTE:** - - Make sure you have created the required `.spec.adminPassword` [secret](https://kubernetes.io/docs/tasks/configmap-secret/managing-secret-using-kubectl/) and `.spec.persistence` [persistent volume](https://kubernetes.io/docs/concepts/storage/persistent-volumes/). - - For the ease of use, the storage class **oci-bv** is specified in the [singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml). This storage class facilitates dynamic provisioning of the OCI block volume for the persistent storage of the database. For other cloud providers, there dynamic provisioning storage class can be used similarly. + **NOTE:** + For the ease of use, the storage class **oci-bv** is specified in the [singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml). This storage class facilitates dynamic provisioning of the OCI block volume for the persistent storage of the database. For other cloud providers, there dynamic provisioning storage class can be used similarly. -* ### Provision a Pre-built Database +### Provision a pre-built database - Provision a new Pre-built database instance by specifying appropriate values for the attributes in the the example `.yaml` file, and running the following command: +Provision a new pre-built database instance by specifying appropriate values for the attributes in the example `.yaml` file, and running the following command: - ```sh - $ kubectl create -f singleinstancedatabase_prebuiltdb.yaml - - singleinstancedatabase.database.oracle.com/sidb-sample created - ``` +```sh +$ kubectl create -f singleinstancedatabase_prebuiltdb.yaml - This Pre-built image includes an already setup database inside the image itself. Although the image size is larger, the startup time of the container includes only the database startup itself, which makes the container startup duration just a couple of seconds. + singleinstancedatabase.database.oracle.com/prebuiltdb-sample created +``` - This Pre-built database would be very useful in CI/CD scenarios, where database would be used for conducting tests, experiments and the workflow is simple. +This pre-built image includes the data files of the database inside the image itself. So, the database startup time of the container is reduced to a couple of seconds only. This pre-built database would be very useful in CI/CD scenarios, where database would be used for conducting tests, experiments and the workflow is simple. - Some limitations are listed as follows: +Please follow the [link](https://github.com/oracle/docker-images/blob/main/OracleDatabase/SingleInstance/extensions/prebuiltdb/README.md) to create the pre-built database image for the Enterprise/Standard edition. - - External volume can not be used for database persistence (as data files are inside the image itself). - - Only the single replica mode (i.e. replicas=1) can be used. +The only limitation with the pre-built database is that it can only be used in the single replica (i.e. replicas=1) mode. -* ### Creation Status +### Creation Status - Creating a new database instance takes a while. When the 'status' status returns the response "Healthy", the Database is open for connections. +Creating a new database instance takes a while. When the 'status' column returns the response "Healthy", the Database is open for connections. - ```sh +```sh $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.status}" - + Healthy ``` -* ### Connection Information +### Connection Information - External and internal (running in Kubernetes pods) clients can connect to the database using .status.connectString and .status.clusterConnectString - respectively in the following command +External and internal (running in Kubernetes pods) clients can connect to the database using `.status.connectString` in the following command: - ```sh - $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.connectString}" +```sh +$ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.connectString}" - 10.0.25.54:1521/ORCL - ``` + 10.0.25.54:1521/ORCL +``` - The Oracle Database inside the container also has Oracle Enterprise Manager Express configured. To access OEM Express, start the browser and follow the URL: +The Oracle Database inside the container also has Oracle Enterprise Manager Express configured. To access OEM Express, start the browser and paste the following URL: - ```sh - $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.oemExpressUrl}" +```sh +$ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.oemExpressUrl}" - https://10.0.25.54:5500/em - ``` + https://10.0.25.54:5500/em +``` - * ### Update Database Config +### Update Database Config - The following database parameters can be updated post database creation: flashBack, archiveLog, forceLog. Change their attribute values and apply using - kubectl apply or edit/patch commands . Enable archiveLog before turning ON flashBack . Turn OFF flashBack before disabling the archiveLog +The following database parameters can be updated post database creation: **flashBack, archiveLog, forceLog**. Change their attribute values and apply using +kubectl **apply** or **edit/patch** commands . Enable archiveLog mode before turning ON flashBack, and turn OFF flashBack before disabling the archiveLog. - ```sh - $ kubectl patch singleinstancedatabase sidb-sample --type merge -p '{"spec":{"forceLog": true}}' +```sh +$ kubectl patch singleinstancedatabase sidb-sample --type merge -p '{"spec":{"forceLog": true}}' - singleinstancedatabase.database.oracle.com/sidb-sample patched - ``` + singleinstancedatabase.database.oracle.com/sidb-sample patched +``` - * #### Database Config Status +#### Database Config Status - Check the Database Config Status using the following command +Check the Database Config Status using the following command - ```sh - $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath=[{.status.archiveLog}, {.status.flashBack}, {.status.forceLog}]" +```sh +$ kubectl get singleinstancedatabase sidb-sample -o "jsonpath=[{.status.archiveLog}, {.status.flashBack}, {.status.forceLog}]" - [true, true, true] - ``` + [true, true, true] +``` - * ### Update Initialization Parameters +### Update Initialization Parameters - The following database initialization parameters can be updated post database creation: `sgaTarget, pgaAggregateTarget, cpuCount, processes`. Change their attribute values and apply using kubectl apply or edit/patch commands. +The following database initialization parameters can be updated post database creation: `sgaTarget, pgaAggregateTarget, cpuCount, processes`. Change their attribute values and apply using kubectl **apply** or **edit/patch** commands. - **NOTE** - * `sgaTarget` should be in range [sga_min_size, sga_max_size], else initialization parameter `sga_target` would not be updated to specified `sgaTarget`. +**NOTE:** +`sgaTarget` should be in range [sga_min_size, sga_max_size], else initialization parameter `sga_target` would not be updated to specified `sgaTarget`. - * ### Multiple Replicas +### Multiple Replicas - Multiple database pod replicas can be provisioned when the persistent volume access mode is ReadWriteMany. Database is open and mounted by one of the replicas. Other replicas will have instance started but not mounted and serve to provide quick cold fail-over in case the active pod dies. Update the replica attribute in the .yaml and apply using the kubectl apply command or edit/patch commands +In multiple replicas mode, more than one pod are created for the database. The database is open and mounted by one of the replica pod. Other replica pods will have instance started but not mounted and serve to provide a quick cold fail-over in case the active pod dies. Update the replica attribute in the .yaml and apply using the `kubectl apply` or `edit/patch` commands. - **Note:** This functionality requires the [K8s extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/k8s) - extended images from container-registry.oracle.com include the K8s extension +**Note:** +- This functionality requires the [k8s extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/k8s) extended images. The database image from the container-registry.oracle.com includes the K8s extension. +- The Oracle database express edition (XE) does not support the [k8s extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/k8s). Hence, it does not support multiple replicas. - * ### Patch Attributes +### Patch Attributes - The following attributes cannot be patched post SingleInstanceDatabase instance Creation : sid, edition, charset, pdbName, cloneFrom. +The following attributes cannot be patched post the Single Instance Database instance creation: sid, edition, charset, pdbName, cloneFrom. - ```sh - $ kubectl --type=merge -p '{"spec":{"sid":"ORCL1"}}' patch singleinstancedatabase sidb-sample +```sh +$ kubectl --type=merge -p '{"spec":{"sid":"ORCL1"}}' patch singleinstancedatabase sidb-sample - The SingleInstanceDatabase "sidb-sample" is invalid: spec.sid: Forbidden: cannot be changed - ``` + The SingleInstanceDatabase "sidb-sample" is invalid: spec.sid: Forbidden: cannot be changed +``` - * #### Patch Persistence Volume Claim +#### Patch Persistence Volume Claim - Persistence Volume Claim (PVC) can be patched post SingleInstanceDatabase instance Creation . This will **delete all the database pods, PVC** and new database pods are created using the new PVC . +Persistence Volume Claim (PVC) can be patched post SingleInstanceDatabase instance Creation . This will **delete all the database pods, PVC** and new database pods are created using the new PVC . - ```sh - $ kubectl --type=merge -p '{"spec":{"persistence":{"accessMode":"ReadWriteMany","size":"110Gi","storageClass":""}}}' patch singleinstancedatabase sidb-sample +```sh +$ kubectl --type=merge -p '{"spec":{"persistence":{"accessMode":"ReadWriteMany","size":"110Gi","storageClass":""}}}' patch singleinstancedatabase sidb-sample - singleinstancedatabase.database.oracle.com/sidb-sample patched - ``` + singleinstancedatabase.database.oracle.com/sidb-sample patched +``` - * #### Patch Service +#### Patch Service - Service can be patched post SingleInstanceDatabase instance Creation . This will **replace the Service with a new type** . - * NodePort - '{"spec":{"loadBalancer": false}}' - * LoadBalancer - '{"spec":{"loadBalancer": true }}' +Service can be patched post SingleInstanceDatabase instance Creation. This will **replace the Service with a new type**. +* NodePort - '{"spec":{"loadBalancer": false}}' +* LoadBalancer - '{"spec":{"loadBalancer": true }}' - ```sh - $ kubectl --type=merge -p '{"spec":{"loadBalancer": false}}' patch singleinstancedatabase sidb-sample +```sh +$ kubectl --type=merge -p '{"spec":{"loadBalancer": false}}' patch singleinstancedatabase sidb-sample - singleinstancedatabase.database.oracle.com/sidb-sample patched - ``` + singleinstancedatabase.database.oracle.com/sidb-sample patched +``` ## Clone Existing Database - Quickly create copies of your existing database using this cloning functionality. A cloned database is an exact, block-for-block copy of the source database. - This is much faster than creating a fresh new database and copying over the data. - - To clone, specify the source database reference as value for the cloneFrom attribute in the sample [singleinstancedatabase_clone.yaml](../../config/samples/sidb/singleinstancedatabase_clone.yaml) file. - **The source database must have archiveLog mode set to true.** +Quickly create copies of your existing database using this cloning functionality. A cloned database is an exact, block-for-block copy of the source database. This is much faster than creating a fresh database and copying over the data. - ```sh - $ grep 'cloneFrom:' singleinstancedatabase_clone.yaml +To clone, specify the source database reference as value for the `cloneFrom` attribute in the sample [singleinstancedatabase_clone.yaml](../../config/samples/sidb/singleinstancedatabase_clone.yaml) file. + +**The source database must have archiveLog mode set to true.** + +```sh +$ grep 'cloneFrom:' singleinstancedatabase_clone.yaml + + cloneFrom: "sidb-sample" - cloneFrom: "sidb-sample" - - $ kubectl create -f singleinstancedatabase_clone.yaml +$ kubectl create -f singleinstancedatabase_clone.yaml - singleinstancedatabase.database.oracle.com/sidb-sample-clone created - ``` + singleinstancedatabase.database.oracle.com/sidb-sample-clone created +``` - **Note:** The clone database can specify a database image different from the source database. In such cases, cloning is supported only between databases of the same major release. +**Note:** The clone database can specify a database image different from the source database. In such cases, cloning is supported only between databases of the same major release. ## Patch/Rollback Database - Databases running in your cluster and managed by this operator can be patched or rolled back between release updates of the same major release. To patch databases, specify an image of the higher release update, and to roll back, specify an image of the lower release update. - - Patched Oracle Docker images can be built using this [patching extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/patching). +Databases running in your cluster and managed by this operator can be patched or rolled back between release updates of the same major release. To patch databases, specify an image of the higher release update, and to roll back, specify an image of the lower release update. - * ### Patch existing Database +Patched Oracle Docker images can be built using this [patching extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/patching). - Edit and apply the [singleinstancedatabase_patch.yaml](../../config/samples/sidb/singleinstancedatabase_patch.yaml) file of the database resource/object by specifying a new release update for image attributes or run the following command. - - ```sh - kubectl --type=merge -p '{"spec":{"image":{"pullFrom":"patched-image:tag","pullSecrets":"pull-secret"}}}' patch singleinstancedatabase sidb-sample +### Patch existing Database - singleinstancedatabase.database.oracle.com/sidb-sample patched +Edit and apply the [singleinstancedatabase_patch.yaml](../../config/samples/sidb/singleinstancedatabase_patch.yaml) file of the database resource/object by specifying a new release update for image attributes or run the following command. - ``` +```sh +kubectl --type=merge -p '{"spec":{"image":{"pullFrom":"patched-image:tag","pullSecrets":"pull-secret"}}}' patch singleinstancedatabase sidb-sample - The database pods will be restarted with the new release update image. For minimum downtime, ensure that you have mutiple replicas of the database pods running. +singleinstancedatabase.database.oracle.com/sidb-sample patched - * ### Clone and Patch Database +``` + +The database pods will be restarted with the new release update image. For minimum downtime, ensure that you have multiple replicas of the database pods running. + +### Clone and Patch Database - Clone your source database using the method of [cloning existing database](README.md#clone-existing-database) and specify a new release image for the cloned database. Use this method to enusure there are no patching related issues impacting your database performance/functionality + Clone your source database using the method of [cloning existing database](#clone-existing-database) and specify a new release image for the cloned database. Use this method to ensure there are no patching related issues impacting your database performance/functionality. - * ### Datapatch status +### Datapatch status - Patching/Rollback operations are complete when the datapatch tool completes patching or rollback of the data files. Check the data files patching status - and current release update version using the following commands +Patching/Rollback operations are complete when the datapatch tool completes patching or rollback of the data files. Check the data files patching status +and current release update version using the following commands - ```sh - $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.datafilesPatched}" +```sh +$ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.datafilesPatched}" - true - - $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.releaseUpdate}" + true + +$ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.releaseUpdate}" - 19.3.0.0.0 (29517242) - ``` + 19.3.0.0.0 (29517242) +``` ## Kind OracleRestDataService -The Oracle Database Operator creates the OracleRestDataService (ORDS) kind as a custom resource that enables RESTful API access to the Oracle Database in K8s +The Oracle Database Operator creates the OracleRestDataService (ORDS) kind as a custom resource that enables RESTful API access to the Oracle Database in K8s. -* ### OracleRestDataService Sample YAML +### OracleRestDataService template YAML - For the use cases detailed below a sample .yaml file is available at - [config/samples/sidb/oraclerestdataservice.yaml](config/samples/sidb/oraclerestdataservice.yaml) +The template `.yaml` file for Oracle Rest Data Services (OracleRestDataService kind) is available at [config/samples/sidb/oraclerestdataservice.yaml](config/samples/sidb/oraclerestdataservice.yaml) - **Note:** - - The `adminPassword` , `ordsPassword` fields of the above `oraclerestdataservice.yaml` yaml contains secrets for authenticating Single Instance Database and for ORDS user with roles `SQL Administrator , System Administrator , SQL Developer , oracle.dbtools.autorest.any.schema` respectively . These secrets gets delete after the first deployed ORDS pod REST enables the database successfully for security reasons. - - To build the ORDS image, please follow the these [instructions](https://github.com/oracle/docker-images/tree/main/OracleRestDataServices#building-oracle-rest-data-services-install-images). - - By default, the ORDS uses self-signed certificates. To use certificates from the Certificate Authority, the ORDS image needs to be rebuilt after specifying the values of `ssl.cert` and `ssl.cert.key` in the [standalone.properties](https://github.com/oracle/docker-images/blob/main/OracleRestDataServices/dockerfiles/standalone.properties.tmpl) file. This newly built ORDS image should be used in the [config/samples/sidb/oraclerestdataservice.yaml](config/samples/sidb/oraclerestdataservice.yaml) file. +**Note:** +- For **quick provisioning** of the ORDS, apply the [config/samples/sidb/oraclerestdataservice_create.yaml](../../config/samples/sidb/oraclerestdataservice_create.yaml) file using the command below: -* ### List OracleRestDataServices + kubectl apply -f oraclerestdataservice_create.yaml - ```sh - $ kubectl get oraclerestdataservice -o name +- The `adminPassword` , `ordsPassword` fields of the above `oraclerestdataservice.yaml` file contains secrets for authenticating Single Instance Database and for ORDS user with roles `SQL Administrator, System Administrator, SQL Developer, oracle.dbtools.autorest.any.schema` respectively. +- To build the ORDS image, please follow the these [instructions](https://github.com/oracle/docker-images/tree/main/OracleRestDataServices#building-oracle-rest-data-services-install-images). +- By default, the ORDS uses self-signed certificates. To use certificates from the Certificate Authority, the ORDS image needs to be rebuilt after specifying the values of `ssl.cert` and `ssl.cert.key` in the [standalone.properties](https://github.com/oracle/docker-images/blob/main/OracleRestDataServices/dockerfiles/standalone.properties.tmpl) file. This newly built ORDS image should be used in the [config/samples/sidb/oraclerestdataservice.yaml](config/samples/sidb/oraclerestdataservice.yaml) file. - oraclerestdataservice.database.oracle.com/ords-sample +### List OracleRestDataServices - ``` +```sh +$ kubectl get oraclerestdataservice -o name -* ### Quick Status + oraclerestdataservice.database.oracle.com/ords-sample - ```sh - $ kubectl get oraclerestdataservice ords-sample +``` - NAME STATUS DATABASE DATABASE API URL DATABASE ACTIONS URL APEX URL - ords-sample Healthy sidb-sample https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/ https://10.0.25.54:8443/ords/sql-developer https://10.0.25.54:8443/ords/ORCLPDB1/apex - - ``` +### Quick Status -* ### Detailed Status +```sh +$ kubectl get oraclerestdataservice ords-sample - ```sh - $ kubectl describe oraclerestdataservice ords-sample - - Name: ords-sample - Namespace: default - Labels: - Annotations: - API Version: database.oracle.com/v1alpha1 - Kind: OracleRestDataService - Metadata: ... - Spec: ... - Status: - Cluster Db API URL: https://ords21c-1.default:8443/ords/ORCLPDB1/_/db-api/stable/ - Database Actions URL: https://10.0.25.54:8443/ords/sql-developer - Database API URL: https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/ - Apex URL: https://10.0.25.54:8443/ords/ORCLPDB1/apex - Database Ref: sidb21c-1 - Image: - Pull From: ... - Pull Secrets: ... - Load Balancer: true - Ords Installed: true - Persistence: - Access Mode: ReadWriteMany - Size: 100Gi - Storage Class: - Service IP: 10.0.25.54 - Status: Healthy +NAME STATUS DATABASE DATABASE API URL DATABASE ACTIONS URL APEX URL +ords-sample Healthy sidb-sample https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/ https://10.0.25.54:8443/ords/sql-developer https://10.0.25.54:8443/ords/ORCLPDB1/apex - ``` +``` + +### Detailed Status + +```sh +$ kubectl describe oraclerestdataservice ords-sample + + Name: ords-sample + Namespace: default + Labels: + Annotations: + API Version: database.oracle.com/v1alpha1 + Kind: OracleRestDataService + Metadata: ... + Spec: ... + Status: + Cluster Db API URL: https://ords21c-1.default:8443/ords/ORCLPDB1/_/db-api/stable/ + Database Actions URL: https://10.0.25.54:8443/ords/sql-developer + Database API URL: https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/ + Apex URL: https://10.0.25.54:8443/ords/ORCLPDB1/apex + Database Ref: sidb21c-1 + Image: + Pull From: ... + Pull Secrets: ... + Load Balancer: true + Ords Installed: true + Persistence: + Access Mode: ReadWriteMany + Size: 100Gi + Storage Class: + Service IP: 10.0.25.54 + Status: Healthy + +``` ## REST Enable Database - Provision a new ORDS instance by specifying appropriate values for the attributes in the the sample .yaml file and executing the following command . ORDS is installed in the root container(CDB) of SingleInstanceDatabase to enable PDB Lifecycle Management . +Provision a new ORDS instance by specifying appropriate values for the attributes in the the sample .yaml file and executing the following command . ORDS is installed in the root container(CDB) of the respective Single Instance Database. - ```sh - $ kubectl create -f oraclerestdataservice.yaml - - oraclerestdataservice.database.oracle.com/ords-sample created - ``` +```sh +$ kubectl create -f oraclerestdataservice_create.yaml + + oraclerestdataservice.database.oracle.com/ords-sample created +``` -* ### Creation Status +### Creation Status - Creating a new ORDS instance takes a while. ORDS is open for connections when the 'status' status returns a "Healthy" +Creating a new ORDS instance takes a while. ORDS is open for connections when the 'status' column returns "Healthy". - ```sh - $ kubectl get oraclerestdataservice/ords-sample --template={{.status.status}} +```sh +$ kubectl get oraclerestdataservice/ords-sample --template={{.status.status}} - Healthy - ``` + Healthy +``` -* ### REST Endpoints +### REST Endpoints - External and internal (running in Kubernetes pods) clients can access the REST Endpoints using .status.databaseApiUrl and .status.clusterDbApiUrl respectively in the following command . +External and internal (running in Kubernetes pods) clients can access the REST Endpoints using .status.databaseApiUrl and .status.clusterDbApiUrl respectively in the following command . - ```sh - $ kubectl get oraclerestdataservice/ords-sample --template={{.status.databaseApiUrl}} +```sh +$ kubectl get oraclerestdataservice/ords-sample --template={{.status.databaseApiUrl}} - https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/ - ``` + https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/ +``` - All the REST Endpoints can be found at +All the REST Endpoints can be found at - There are two basic approaches for authentication to the REST Endpoints. Certain APIs are specific about which authentication method they will accept. +There are two basic approaches for authentication to the REST Endpoints. Certain APIs are specific about which authentication method they will accept. * #### Default Administrator @@ -433,7 +431,7 @@ The Oracle Database Operator creates the OracleRestDataService (ORDS) kind as a Note : Browser may not prompt for credentials while accessing certain REST Endpoints and in such case one can use clients like curl and pass credentials while calling REST Endpoints . -* #### Some use cases +#### Some use cases Some generic use cases for the Database API are as follows: * ##### Getting all Database components ```sh @@ -456,136 +454,129 @@ The Oracle Database Operator creates the OracleRestDataService (ORDS) kind as a curl -s -k -X GET -u 'ORDS_PUBLIC_USER:<.spec.ordsPassword>' https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/database/feature_usage/ | python -m json.tool ``` -* #### REST Enabled SQL - - The REST Enabled SQL functionality allows REST calls to send DML, DDL and scripts to any REST enabled schema by exposing the same SQL engine used in SQL Developer and SQLcl. +#### REST Enabled SQL - * Run a Script +The REST Enabled SQL functionality allows REST calls to send DML, DDL and scripts to any REST enabled schema by exposing the same SQL engine used in SQL Developer and SQLcl. - Create a file called "/tmp/table.sql" with the following contents. +**Run a Script:** - ```sh - CREATE TABLE DEPT ( - DEPTNO NUMBER(2) CONSTRAINT PK_DEPT PRIMARY KEY, - DNAME VARCHAR2(14), - LOC VARCHAR2(13) - ) ; +Create a file called "/tmp/table.sql" with the following contents. - INSERT INTO DEPT VALUES (10,'ACCOUNTING','NEW YORK'); - INSERT INTO DEPT VALUES (20,'RESEARCH','DALLAS'); - INSERT INTO DEPT VALUES (30,'SALES','CHICAGO'); - INSERT INTO DEPT VALUES (40,'OPERATIONS','BOSTON'); - COMMIT; - ``` +```sh + CREATE TABLE DEPT ( + DEPTNO NUMBER(2) CONSTRAINT PK_DEPT PRIMARY KEY, + DNAME VARCHAR2(14), + LOC VARCHAR2(13) + ) ; - Execute the follwing API to run the above script. + INSERT INTO DEPT VALUES (10,'ACCOUNTING','NEW YORK'); + INSERT INTO DEPT VALUES (20,'RESEARCH','DALLAS'); + INSERT INTO DEPT VALUES (30,'SALES','CHICAGO'); + INSERT INTO DEPT VALUES (40,'OPERATIONS','BOSTON'); + COMMIT; +``` - ```sh - curl -s -k -X "POST" "https://10.0.25.54:8443/ords/<.spec.restEnableSchemas[].pdb>/<.spec.restEnableSchemas[].urlMapping>/_/sql" \ - -H "Content-Type: application/sql" \ - -u '<.spec.restEnableSchemas[].schema>:<.spec.ordsPassword>' \ - -d @/tmp/table.sql - ``` +Execute the follwing API to run the above script. - * Basic Call +```sh + curl -s -k -X "POST" "https://10.0.25.54:8443/ords/<.spec.restEnableSchemas[].pdb>/<.spec.restEnableSchemas[].urlMapping>/_/sql" \ + -H "Content-Type: application/sql" \ + -u '<.spec.restEnableSchemas[].schema>:<.spec.ordsPassword>' \ + -d @/tmp/table.sql +``` - Fetch all entries from 'DEPT' table by calling the following API +**Basic Call:** - ```sh - curl -s -k -X "POST" "https://10.0.25.54:8443/ords/<.spec.restEnableSchemas[].pdb>/<.spec.restEnableSchemas[].urlMapping>/_/sql" \ - -H "Content-Type: application/sql" \ - -u '<.spec.restEnableSchemas[].schema>:<.spec.ordsPassword>' \ - -d $'select * from dept;' | python -m json.tool - ``` +Fetch all entries from 'DEPT' table by calling the following API - **NOTE:** `.spec.restEnableSchema[].urlMapping` is optional and is defaulted to `.spec.restEnableSchema[].schema` +```sh + curl -s -k -X "POST" "https://10.0.25.54:8443/ords/<.spec.restEnableSchemas[].pdb>/<.spec.restEnableSchemas[].urlMapping>/_/sql" \ + -H "Content-Type: application/sql" \ + -u '<.spec.restEnableSchemas[].schema>:<.spec.ordsPassword>' \ + -d $'select * from dept;' | python -m json.tool +``` -* #### Data Pump +**NOTE:** `.spec.restEnableSchema[].urlMapping` is optional and is defaulted to `.spec.restEnableSchema[].schema` - The Oracle REST Data Services (ORDS) database API allows user to create Data Pump export and import jobs via REST web service calls. +#### Data Pump - REST APIs for Data Pump Jobs can be found at [https://docs.oracle.com/en/database/oracle/oracle-database/21/dbrst/op-database-datapump-jobs-post.html](https://docs.oracle.com/en/database/oracle/oracle-database/21/dbrst/op-database-datapump-jobs-post.html) +The Oracle REST Data Services (ORDS) database API allows user to create Data Pump export and import jobs via REST web service calls. -* ### Database Actions +REST APIs for Data Pump Jobs can be found at [https://docs.oracle.com/en/database/oracle/oracle-database/21/dbrst/op-database-datapump-jobs-post.html](https://docs.oracle.com/en/database/oracle/oracle-database/21/dbrst/op-database-datapump-jobs-post.html). - Database Actions is a web-based interface that uses Oracle REST Data Services to provide development, data tools, administration and monitoring features for Oracle Database. +### Database Actions - * To use Database Actions, one must sign in as a database user whose schema has been REST-enabled. - * This can be done by specifying appropriate values for the `.spec.restEnableSchemas` attributes details in the sample yaml [config/samples/sidb/oraclerestdataservice.yaml](config/samples/sidb/oraclerestdataservice.yaml) which are needed for authorising Database Actions. - * Schema will be created (if not exists) with username as `.spec.restEnableSchema[].schema` and password as `.spec.ordsPassword.`. - * UrlMapping `.spec.restEnableSchema[].urlMapping` is optional and is defaulted to `.spec.restEnableSchema[].schema`. +Database Actions is a web-based interface that uses Oracle REST Data Services to provide development, data tools, administration and monitoring features for Oracle Database. - Database Actions can be accessed via browser using `.status.databaseActionsUrl` in the following command +* To use Database Actions, one must sign in as a database user whose schema has been REST-enabled. +* This can be done by specifying appropriate values for the `.spec.restEnableSchemas` attributes details in the sample yaml [config/samples/sidb/oraclerestdataservice.yaml](config/samples/sidb/oraclerestdataservice.yaml) which are needed for authorising Database Actions. +* Schema will be created (if not exists) with username as `.spec.restEnableSchema[].schema` and password as `.spec.ordsPassword.`. +* UrlMapping `.spec.restEnableSchema[].urlMapping` is optional and is defaulted to `.spec.restEnableSchema[].schema`. - ```sh - $ kubectl get oraclerestdataservice/ords-sample --template={{.status.databaseActionsUrl}} +Database Actions can be accessed via browser using `.status.databaseActionsUrl` in the following command - https://10.0.25.54:8443/ords/sql-developer - ``` +```sh +$ kubectl get oraclerestdataservice/ords-sample --template={{.status.databaseActionsUrl}} - Sign in to Database Actions using - * First Page: \ - PDB Name: `.spec.restEnableSchema[].pdb` \ - Username: `.spec.restEnableSchema[].urlMapping` + https://10.0.25.54:8443/ords/sql-developer +``` - * Second Page: \ - Username: `.spec.restEnableSchema[].schema` \ - Password: `.spec.ordsPassword` +Sign in to Database Actions using +* First Page: \ +PDB Name: `.spec.restEnableSchema[].pdb` \ +Username: `.spec.restEnableSchema[].urlMapping` - ![database-actions-home](/images/sidb/database-actions-home.png) +* Second Page: \ +Username: `.spec.restEnableSchema[].schema` \ +Password: `.spec.ordsPassword` - More info on Database Actions can be found at +![database-actions-home](/images/sidb/database-actions-home.png) -* ### Application Express +More info on Database Actions can be found at - Oracle Application Express (APEX) is a low-code development platform that enables developers to build scalable, secure enterprise apps, with world-class features, that can be deployed anywhere. +### Application Express - Using APEX, developers can quickly develop and deploy compelling apps that solve real problems and provide immediate value. Developers won't need to be an expert in a vast array of technologies to deliver sophisticated solutions. Focus on solving the problem and let APEX take care of the rest. +Oracle Application Express (APEX) is a low-code development platform that enables developers to build scalable, secure enterprise apps, with world-class features, that can be deployed anywhere. - To access APEX, You need to configure APEX with ORDS. The following section will explain configuring APEX with ORDS in details: +Using APEX, developers can quickly develop and deploy compelling apps that solve real problems and provide immediate value. Developers won't need to be an expert in a vast array of technologies to deliver sophisticated solutions. Focus on solving the problem and let APEX take care of the rest. - **Configure APEX with ORDS:** - * For quick provisioning, apply the [config/samples/sidb/oraclerestdataservice_apex.yaml](../../confi/samples/sidb/oraclerestdataservice_apex.yaml) file. First, it creates `ords-secret`, `apex-secret`, and then provision the ORDS configured with Oracle APEX. It uses the ORDS image hosted on the [Oracle Container Registry](https://container-registry.oracle.com/ords/f?p=113:4:113387942129427:::4:P4_REPOSITORY,AI_REPOSITORY,AI_REPOSITORY_NAME,P4_REPOSITORY_NAME,P4_EULA_ID,P4_BUSINESS_AREA_ID:1183,1183,Oracle%20REST%20Data%20Services%20(ORDS)%20with%20Application%20Express,Oracle%20REST%20Data%20Services%20(ORDS)%20with%20Application%20Express,1,0&cs=3_y-KlneZIxRRfXzerC_0ro7P1MGh-B_9lTEQObVTdoQCWkmsQ3lHpFs90Z8QFheteVQEzPvtUVHEQAqqXegYbA). - * On the other hand, to provision ORDS step by step, set `.spec.apexPassword.secretName` to a non-null string in [config/samples/sidb/oraclerestdataservice.yaml](../../config/samples/sidb/oraclerestdataservice.yaml) - * This is used as a common password for APEX_PUBLIC_USER, APEX_REST_PUBLIC_USER, APEX_LISTENER and Apex administrator (username: ADMIN) mapped to secretKey - * Status of ORDS turns to 'Updating' during apex configuration and turns 'Healthy' after successful configuration. You can also check status using below cmd +To access APEX, You need to configure APEX with the ORDS. The following section will explain configuring APEX with ORDS in details: - ```sh - $ kubectl get oraclerestdataservice ords-sample -o "jsonpath=[{.status.apexConfigured}]" +#### Configure APEX with ORDS - [true] - ``` +* For quick provisioning, apply the [config/samples/sidb/oraclerestdataservice_apex.yaml](../../confi/samples/sidb/oraclerestdataservice_apex.yaml) file. First, it creates `ords-secret`, `apex-secret`, and then provision the ORDS configured with Oracle APEX. It uses the ORDS image hosted on the [Oracle Container Registry](https://container-registry.oracle.com/ords/f?p=113:4:113387942129427:::4:P4_REPOSITORY,AI_REPOSITORY,AI_REPOSITORY_NAME,P4_REPOSITORY_NAME,P4_EULA_ID,P4_BUSINESS_AREA_ID:1183,1183,Oracle%20REST%20Data%20Services%20(ORDS)%20with%20Application%20Express,Oracle%20REST%20Data%20Services%20(ORDS)%20with%20Application%20Express,1,0&cs=3_y-KlneZIxRRfXzerC_0ro7P1MGh-B_9lTEQObVTdoQCWkmsQ3lHpFs90Z8QFheteVQEzPvtUVHEQAqqXegYbA). - * If you configure APEX after ORDS is installed, ORDS pods will be deleted and recreated + kubectl apply -f oraclerestdataservice_apex.yaml - Application Express can be accessed via browser using `.status.databaseApiUrl` in the following command .\ +* On the other hand, to provision ORDS step by step, set `.spec.apexPassword.secretName` to a non-null string in [config/samples/sidb/oraclerestdataservice.yaml](../../config/samples/sidb/oraclerestdataservice.yaml) +* This is used as a common password for APEX_PUBLIC_USER, APEX_REST_PUBLIC_USER, APEX_LISTENER and Apex administrator (username: ADMIN) mapped to secretKey +* Status of ORDS turns to 'Updating' during apex configuration and turns 'Healthy' after successful configuration. You can also check status using below cmd - ```sh - $ kubectl get oraclerestdataservice/ords-sample --template={{.status.databaseApiUrl}} + ```sh + $ kubectl get oraclerestdataservice ords-sample -o "jsonpath=[{.status.apexConfigured}]" - https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/ - ``` + [true] + ``` - Sign in to Administration servies using \ - workspace: `INTERNAL` \ - username: `ADMIN` \ - password: `.spec.apexPassword` +* If you configure APEX after ORDS is installed, ORDS pods will be deleted and recreated - ![application-express-admin-home](/images/sidb/application-express-admin-home.png) +Application Express can be accessed via browser using `.status.databaseApiUrl` in the following command .\ - **NOTE** - * Apex Administrator for pdbs other than `.spec.databaseRef.pdbName` has to be created manually. More Info on creating Apex Administrator can be found at [APEX_UTIL.CREATE_USER] - * By default, the full development runtime environment is initialized in APEX. It can be changed manually to the runtime environment. For this, `apxdevrm.sql` script should be run after connecting to the primary database from the ORDS pod as the sys user with sysdba privilage. Please click the [link](https://docs.oracle.com/en/database/oracle/application-express/21.2/htmig/converting-between-runtime-and-full-development-environments.html#GUID-B0621B40-3441-44ED-9D86-29B058E26BE9) for detailed instructions. -* ### Multiple Replicas - - Currently only single replica mode is supported +```sh +$ kubectl get oraclerestdataservice/ords-sample --template={{.status.databaseApiUrl}} -* ### Patch Attributes + https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/ +``` - * The following attributes cannot be patched post SingleInstanceDatabase instance Creation : databaseRef, loadBalancer, image, ordsPassword, adminPassword, apexPassword. +Sign in to Administration servies using \ +workspace: `INTERNAL` \ +username: `ADMIN` \ +password: `.spec.apexPassword` - * A schema can be rest enabled or disabled by setting the `.spec.restEnableSchemas[].enable` to true or false respectively in ords sample .yaml file and apply using the kubectl apply command or edit/patch commands. This requires `.spec.ordsPassword` secret. +![application-express-admin-home](/images/sidb/application-express-admin-home.png) +**NOTE:** +By default, the full development runtime environment is initialized in APEX. It can be changed manually to the runtime environment. For this, `apxdevrm.sql` script should be run after connecting to the primary database from the ORDS pod as the sys user with sysdba privilage. Please click the [link](https://docs.oracle.com/en/database/oracle/application-express/21.2/htmig/converting-between-runtime-and-full-development-environments.html#GUID-B0621B40-3441-44ED-9D86-29B058E26BE9) for detailed instructions. ## Performing maintenance operations If some manual operations are required to be performed, the procedure is as follows: From 53b08104592fca9fc86004ab10e0b2fd783205b0 Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Mon, 2 May 2022 17:34:23 +0530 Subject: [PATCH 326/628] More readme changes Signed-off-by: abhisbyk --- docs/sidb/README.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 74fc51d2..16585749 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -86,13 +86,13 @@ $ kubectl describe singleinstancedatabase sidb-sample-clone Reason: LastReconcileCycleCompleted Status: True Type: ReconcileComplete - Connect String: 10.0.25.54:1521/ORCL1 + Connect String: 10.0.25.58:1521/ORCL1C Datafiles Created: true Datafiles Patched: true Edition: Enterprise Flash Back: true Force Log: false - Oem Express URL: https://10.0.25.54:5500/em + Oem Express URL: https://10.0.25.58:5500/em Pdb Name: orclpdb1 Release Update: 19.11.0.0.0 (32545013) Replicas: 2 @@ -137,7 +137,8 @@ $ kubectl describe singleinstancedatabase sidb-sample-clone ``` **NOTE:** - For the ease of use, the storage class **oci-bv** is specified in the [singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml). This storage class facilitates dynamic provisioning of the OCI block volume for the persistent storage of the database. For other cloud providers, there dynamic provisioning storage class can be used similarly. + - For the ease of use, the storage class **oci-bv** is specified in the [singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml). This storage class facilitates dynamic provisioning of the OCI block volume for the persistent storage of the database. For other cloud providers, there dynamic provisioning storage class can be used similarly. + - Oracle Database Express edition is supported from version 21.3.0 onwards, and the other editions (i.e. Enterprise and Standard editions) are supported from version 19.3.0 onwards. ### Provision a pre-built database From ac976cee14692666698d42548c11acabd9564965 Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Mon, 2 May 2022 17:55:01 +0530 Subject: [PATCH 327/628] Added section for express edition Signed-off-by: abhisbyk --- docs/sidb/README.md | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 16585749..0f085d8f 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -138,7 +138,17 @@ $ kubectl describe singleinstancedatabase sidb-sample-clone **NOTE:** - For the ease of use, the storage class **oci-bv** is specified in the [singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml). This storage class facilitates dynamic provisioning of the OCI block volume for the persistent storage of the database. For other cloud providers, there dynamic provisioning storage class can be used similarly. - - Oracle Database Express edition is supported from version 21.3.0 onwards, and the other editions (i.e. Enterprise and Standard editions) are supported from version 19.3.0 onwards. + - Oracle Database Enterprise and Standard editions are supported from version 19.3.0 onwards. + +### Provisioning a new XE database +To provision a new XE database, use the [config/samples/sidb/singleinstancedatabase_express.yaml](../../config/samples/sidb/singleinstancedatabase_express.yaml) file. The sample command is as follows: + + kubectl apply -f singleinstancedatabase_express.yaml + +It pulls the XE image uploaded on the [Oracle Container Registry](https://container-registry.oracle.com/ords/f?p=113:4:7460390069267:::4:P4_REPOSITORY,AI_REPOSITORY,AI_REPOSITORY_NAME,P4_REPOSITORY_NAME,P4_EULA_ID,P4_BUSINESS_AREA_ID:803,803,Oracle%20Database%20Express%20Edition,Oracle%20Database%20Express%20Edition,1,0&cs=3-UN6D9nAfyqxcYnrks18OAmfFcri96NZojBQALxMdakix8wgYRBxhD8rpTFd2ak1FAtfOVFexbuOM2opsjxT9w). + +**NOTE:** +Oracle Database XE edition is supported version 21.3.0 onwards. ### Provision a pre-built database From 5e3e06daa8e0c3c2ff44b61e28c1fc0bffb9a997 Mon Sep 17 00:00:00 2001 From: Mohammed Yunus Date: Mon, 2 May 2022 18:08:09 +0000 Subject: [PATCH 328/628] Update README.md --- docs/sidb/README.md | 123 ++++++++++++++++++-------------------------- 1 file changed, 51 insertions(+), 72 deletions(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 0f085d8f..bb905e94 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -1,6 +1,6 @@ # Managing Oracle Single Instance Databases with Oracle Database Operator for Kubernetes -Oracle Database Operator for Kubernetes (a.k.a. OraOperator) includes the Single Instance Database Controller that enables provisioning, cloning, and patching of Oracle Single Instance Databases on Kubernetes. The following sections explain the setup and functionality of the operator +Oracle Database Operator for Kubernetes (a.k.a. OraOperator) includes the Single Instance Database Controller that performs provisioning, cloning, and patching of Oracle Single Instance Databases on Kubernetes and enables configuring them for Oracle REST Data Services with Oracle APEX development platform. The following sections explain the setup and functionality of the operator * [Prerequisites](#prerequisites) * [Kind SingleInstanceDatabase Resource](#kind-singleinstancedatabase-resource) @@ -21,7 +21,7 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI ### SingleInstanceDatabase template YAML -The template `.yaml` file for Single Instance Database (Enterprise & Standard Editions) including all the configurable options is as follows: +The template `.yaml` file for Single Instance Database including all the configurable options is available at: [config/samples/sidb/singleinstancedatabase.yaml](./../../config/samples/sidb/singleinstancedatabase.yaml) **Note:** @@ -29,9 +29,9 @@ The `adminPassword` field of the above `singleinstancedatabase.yaml` file refers This secret can be created using the following sample command: - kubectl create secret generic pwd-secret --from-literal=oracle_pwd="SamplePWD#123" + kubectl create secret generic admin-secret --from-literal=oracle_pwd="ChangeOnInstall_1" -The above command will create a secret having name as `pwd-secret` with key `oracle_pwd` mapped to the actual password specified in the command. +The above command will create a secret having name as `admin-secret` with key `oracle_pwd` mapped to the actual password specified in the command. ### List Databases @@ -48,8 +48,8 @@ $ kubectl get singleinstancedatabases -o name ```sh $ kubectl get singleinstancedatabase sidb-sample -NAME EDITION STATUS VERSION CONNECT STR OEM EXPRESS URL -sidb-sample Enterprise Healthy 19.3.0.0.0 10.0.25.54:1521/ORCLCDB https://10.0.25.54:5500/em +NAME EDITION STATUS VERSION CONNECT STR OEM EXPRESS URL +sidb-sample Enterprise Healthy 19.3.0.0.0 10.0.25.54:1521/ORCLCDB https://10.0.25.54:5500/em ``` ### Detailed Status @@ -113,7 +113,7 @@ $ kubectl describe singleinstancedatabase sidb-sample-clone ## Provision New Database - Easily provision a new database instance on the Kubernetes cluster using [singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml). + A sample **[singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml)** is provided to quickly and easily provision a single instance database in Kubernetes Firstly, sign into [Oracle Container Registry](https://container-registry.oracle.com/ords/f?p=113:4:7154182141811:::4:P4_REPOSITORY,AI_REPOSITORY,AI_REPOSITORY_NAME,P4_REPOSITORY_NAME,P4_EULA_ID,P4_BUSINESS_AREA_ID:9,9,Oracle%20Database%20Enterprise%20Edition,Oracle%20Database%20Enterprise%20Edition,1,0&cs=3Y_90hkCQLfJzrvTLiEipIGgWGUytfrtAPuHFocuWd0NDSacbBPlamohfLuiJA-bAsVL6Z_yKEMsTbb52bm6IRA) and accept the license agreement for the Database image, ignore if you have accepted already. @@ -124,24 +124,25 @@ $ kubectl describe singleinstancedatabase sidb-sample-clone secret/oracle-container-registry-secret created ``` - This secret can also be created using the docker config file as follows: - - kubectl create secret generic oracle-container-registry-secret --from-file=.dockerconfigjson=.docker/config.json --type=kubernetes.io/dockerconfigjson - + This secret can also be created using the docker config file after a successful docker login + ```sh + $ docker login container-registry.oracle.com + $ kubectl create secret generic oracle-container-registry-secret --from-file=.dockerconfigjson=.docker/config.json --type=kubernetes.io/dockerconfigjson + ``` Now, easily provision a new database instance on the cluster by using the following command. ```sh - $ kubectl create -f singleinstancedatabase_create.yaml + $ kubectl apply -f singleinstancedatabase_create.yaml singleinstancedatabase.database.oracle.com/sidb-sample created ``` **NOTE:** - - For the ease of use, the storage class **oci-bv** is specified in the [singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml). This storage class facilitates dynamic provisioning of the OCI block volume for the persistent storage of the database. For other cloud providers, there dynamic provisioning storage class can be used similarly. + - For the ease of use, the storage class **oci-bv** is specified in the [singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml). This storage class facilitates dynamic provisioning of the block volumes on Oracle OKE for the persistent storage of the database. For other cloud providers, an appropriate dynamic provisioning storage class must be provided. - Oracle Database Enterprise and Standard editions are supported from version 19.3.0 onwards. ### Provisioning a new XE database -To provision a new XE database, use the [config/samples/sidb/singleinstancedatabase_express.yaml](../../config/samples/sidb/singleinstancedatabase_express.yaml) file. The sample command is as follows: +To provision a XE database, use the sample **[config/samples/sidb/singleinstancedatabase_express.yaml](../../config/samples/sidb/singleinstancedatabase_express.yaml)** file. Use the following command: kubectl apply -f singleinstancedatabase_express.yaml @@ -152,20 +153,18 @@ Oracle Database XE edition is supported version 21.3.0 onwards. ### Provision a pre-built database -Provision a new pre-built database instance by specifying appropriate values for the attributes in the example `.yaml` file, and running the following command: +Provision a pre-built database using the sample yaml **[config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml](../../config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml)** with the following command: ```sh -$ kubectl create -f singleinstancedatabase_prebuiltdb.yaml +$ kubectl apply -f singleinstancedatabase_prebuiltdb.yaml singleinstancedatabase.database.oracle.com/prebuiltdb-sample created ``` -This pre-built image includes the data files of the database inside the image itself. So, the database startup time of the container is reduced to a couple of seconds only. This pre-built database would be very useful in CI/CD scenarios, where database would be used for conducting tests, experiments and the workflow is simple. +This pre-built database image includes the data files of the database inside the image itself. This enables for quick startup of the database in the container. This pre-built database would be very useful in CI/CD scenarios, where database would be used for conducting tests, experiments requiring a new database instance for every run. Please follow the [link](https://github.com/oracle/docker-images/blob/main/OracleDatabase/SingleInstance/extensions/prebuiltdb/README.md) to create the pre-built database image for the Enterprise/Standard edition. -The only limitation with the pre-built database is that it can only be used in the single replica (i.e. replicas=1) mode. - ### Creation Status Creating a new database instance takes a while. When the 'status' column returns the response "Healthy", the Database is open for connections. @@ -224,11 +223,11 @@ The following database initialization parameters can be updated post database cr ### Multiple Replicas -In multiple replicas mode, more than one pod are created for the database. The database is open and mounted by one of the replica pod. Other replica pods will have instance started but not mounted and serve to provide a quick cold fail-over in case the active pod dies. Update the replica attribute in the .yaml and apply using the `kubectl apply` or `edit/patch` commands. +Specifying a replicas count greater than one creates multiple replica pods for the database. The database is open and mounted by one of the replica pod. Other replica pods will have instance started but not mounted and serve to provide a fast fail over in case the active pod dies. Update the replica attribute in the .yaml and apply using the `kubectl apply` or `edit/patch` commands. **Note:** - This functionality requires the [k8s extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/k8s) extended images. The database image from the container-registry.oracle.com includes the K8s extension. -- The Oracle database express edition (XE) does not support the [k8s extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/k8s). Hence, it does not support multiple replicas. +- The Oracle database express edition (XE) does not include the [k8s extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/k8s). Hence, it does not support multiple replicas. ### Patch Attributes @@ -240,16 +239,6 @@ $ kubectl --type=merge -p '{"spec":{"sid":"ORCL1"}}' patch singleinstancedatabas The SingleInstanceDatabase "sidb-sample" is invalid: spec.sid: Forbidden: cannot be changed ``` -#### Patch Persistence Volume Claim - -Persistence Volume Claim (PVC) can be patched post SingleInstanceDatabase instance Creation . This will **delete all the database pods, PVC** and new database pods are created using the new PVC . - -```sh -$ kubectl --type=merge -p '{"spec":{"persistence":{"accessMode":"ReadWriteMany","size":"110Gi","storageClass":""}}}' patch singleinstancedatabase sidb-sample - - singleinstancedatabase.database.oracle.com/sidb-sample patched -``` - #### Patch Service Service can be patched post SingleInstanceDatabase instance Creation. This will **replace the Service with a new type**. @@ -264,23 +253,24 @@ $ kubectl --type=merge -p '{"spec":{"loadBalancer": false}}' patch singleinstanc ## Clone Existing Database -Quickly create copies of your existing database using this cloning functionality. A cloned database is an exact, block-for-block copy of the source database. This is much faster than creating a fresh database and copying over the data. +Create copies of your existing database using the cloning functionality. A cloned database is an exact, block-for-block copy of the source database. This is much faster than creating a fresh database and copying over the data. Specify the source database reference in the `cloneFrom` attribute -To clone, specify the source database reference as value for the `cloneFrom` attribute in the sample [singleinstancedatabase_clone.yaml](../../config/samples/sidb/singleinstancedatabase_clone.yaml) file. +$ grep 'cloneFrom:' singleinstancedatabase_clone.yaml + + cloneFrom: "sidb-sample" **The source database must have archiveLog mode set to true.** -```sh -$ grep 'cloneFrom:' singleinstancedatabase_clone.yaml +Quicly clone using the sample **[singleinstancedatabase_clone.yaml](../../config/samples/sidb/singleinstancedatabase_clone.yaml)** file by executing the following command - cloneFrom: "sidb-sample" +```sh -$ kubectl create -f singleinstancedatabase_clone.yaml +$ kubectl apply -f singleinstancedatabase_clone.yaml singleinstancedatabase.database.oracle.com/sidb-sample-clone created ``` -**Note:** The clone database can specify a database image different from the source database. In such cases, cloning is supported only between databases of the same major release. +**Note:** The database image specified in the yaml for cloning can be at a different patch level but should be of the same major release version ## Patch/Rollback Database @@ -290,7 +280,7 @@ Patched Oracle Docker images can be built using this [patching extension](https: ### Patch existing Database -Edit and apply the [singleinstancedatabase_patch.yaml](../../config/samples/sidb/singleinstancedatabase_patch.yaml) file of the database resource/object by specifying a new release update for image attributes or run the following command. +Edit and apply the sample **[singleinstancedatabase_patch.yaml](../../config/samples/sidb/singleinstancedatabase_patch.yaml)** file of the database resource/object by specifying a new release update for image attributes or run the following command. ```sh kubectl --type=merge -p '{"spec":{"image":{"pullFrom":"patched-image:tag","pullSecrets":"pull-secret"}}}' patch singleinstancedatabase sidb-sample @@ -322,19 +312,14 @@ $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.releaseUp ## Kind OracleRestDataService -The Oracle Database Operator creates the OracleRestDataService (ORDS) kind as a custom resource that enables RESTful API access to the Oracle Database in K8s. +The Oracle Database Operator creates the OracleRestDataService (ORDS) kind as a custom resource that configures Oracle REST Data Services and enables RESTful API access to the Oracle Database in K8s. ### OracleRestDataService template YAML The template `.yaml` file for Oracle Rest Data Services (OracleRestDataService kind) is available at [config/samples/sidb/oraclerestdataservice.yaml](config/samples/sidb/oraclerestdataservice.yaml) -**Note:** -- For **quick provisioning** of the ORDS, apply the [config/samples/sidb/oraclerestdataservice_create.yaml](../../config/samples/sidb/oraclerestdataservice_create.yaml) file using the command below: - - kubectl apply -f oraclerestdataservice_create.yaml - -- The `adminPassword` , `ordsPassword` fields of the above `oraclerestdataservice.yaml` file contains secrets for authenticating Single Instance Database and for ORDS user with roles `SQL Administrator, System Administrator, SQL Developer, oracle.dbtools.autorest.any.schema` respectively. -- To build the ORDS image, please follow the these [instructions](https://github.com/oracle/docker-images/tree/main/OracleRestDataServices#building-oracle-rest-data-services-install-images). +- The `adminPassword` field of the above `oraclerestdataservice.yaml` file contains secrets for authenticating Single Instance Database and the `ordsPassword` field contains secret for ORDS_PUBLIC_USER user with roles `SQL Administrator, System Administrator, SQL Developer, oracle.dbtools.autorest.any.schema` respectively. +- To build the ORDS image, please follow the these [GitHub](https://github.com/oracle/docker-images/tree/main/OracleRestDataServices#building-oracle-rest-data-services-install-images) instructions. - By default, the ORDS uses self-signed certificates. To use certificates from the Certificate Authority, the ORDS image needs to be rebuilt after specifying the values of `ssl.cert` and `ssl.cert.key` in the [standalone.properties](https://github.com/oracle/docker-images/blob/main/OracleRestDataServices/dockerfiles/standalone.properties.tmpl) file. This newly built ORDS image should be used in the [config/samples/sidb/oraclerestdataservice.yaml](config/samples/sidb/oraclerestdataservice.yaml) file. ### List OracleRestDataServices @@ -351,8 +336,8 @@ $ kubectl get oraclerestdataservice -o name ```sh $ kubectl get oraclerestdataservice ords-sample -NAME STATUS DATABASE DATABASE API URL DATABASE ACTIONS URL APEX URL -ords-sample Healthy sidb-sample https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/ https://10.0.25.54:8443/ords/sql-developer https://10.0.25.54:8443/ords/ORCLPDB1/apex +NAME STATUS DATABASE DATABASE API URL DATABASE ACTIONS URL APEX URL +ords-sample Healthy sidb-sample https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/ https://10.0.25.54:8443/ords/sql-developer https://10.0.25.54:8443/ords/ORCLPDB1/apex ``` @@ -391,10 +376,10 @@ $ kubectl describe oraclerestdataservice ords-sample ## REST Enable Database -Provision a new ORDS instance by specifying appropriate values for the attributes in the the sample .yaml file and executing the following command . ORDS is installed in the root container(CDB) of the respective Single Instance Database. +For quick provisioning of the Oracle REST Data Services, apply the sample **[config/samples/sidb/oraclerestdataservice_create.yaml](../../config/samples/sidb/oraclerestdataservice_create.yaml)** file using the command below: ```sh -$ kubectl create -f oraclerestdataservice_create.yaml +$ kubectl apply -f oraclerestdataservice_create.yaml oraclerestdataservice.database.oracle.com/ords-sample created ``` @@ -411,7 +396,7 @@ $ kubectl get oraclerestdataservice/ords-sample --template={{.status.status}} ### REST Endpoints -External and internal (running in Kubernetes pods) clients can access the REST Endpoints using .status.databaseApiUrl and .status.clusterDbApiUrl respectively in the following command . +Clients can access the REST Endpoints using .status.databaseApiUrl as shown in the following command. ```sh $ kubectl get oraclerestdataservice/ords-sample --template={{.status.databaseApiUrl}} @@ -423,27 +408,19 @@ All the REST Endpoints can be found at ' https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/database/components/ | python -m json.tool @@ -467,6 +444,9 @@ There are two basic approaches for authentication to the REST Endpoints. Certain #### REST Enabled SQL +The REST Enable SQL functionality is available to all the schemas specified in the `.spec.restEnableSchemas` atrribute of the sample yaml. +Only these schemas will have access SQL Developer Web Console specified by the Database Actions URL. + The REST Enabled SQL functionality allows REST calls to send DML, DDL and scripts to any REST enabled schema by exposing the same SQL engine used in SQL Developer and SQLcl. **Run a Script:** @@ -555,12 +535,11 @@ To access APEX, You need to configure APEX with the ORDS. The following section #### Configure APEX with ORDS -* For quick provisioning, apply the [config/samples/sidb/oraclerestdataservice_apex.yaml](../../confi/samples/sidb/oraclerestdataservice_apex.yaml) file. First, it creates `ords-secret`, `apex-secret`, and then provision the ORDS configured with Oracle APEX. It uses the ORDS image hosted on the [Oracle Container Registry](https://container-registry.oracle.com/ords/f?p=113:4:113387942129427:::4:P4_REPOSITORY,AI_REPOSITORY,AI_REPOSITORY_NAME,P4_REPOSITORY_NAME,P4_EULA_ID,P4_BUSINESS_AREA_ID:1183,1183,Oracle%20REST%20Data%20Services%20(ORDS)%20with%20Application%20Express,Oracle%20REST%20Data%20Services%20(ORDS)%20with%20Application%20Express,1,0&cs=3_y-KlneZIxRRfXzerC_0ro7P1MGh-B_9lTEQObVTdoQCWkmsQ3lHpFs90Z8QFheteVQEzPvtUVHEQAqqXegYbA). +* For quick provisioning, apply the **[config/samples/sidb/oraclerestdataservice_apex.yaml](../../confi/samples/sidb/oraclerestdataservice_apex.yaml)** file. It configured Oracle REST Data Services with Oracle APEX. It uses the ORDS image hosted on the [Oracle Container Registry](https://container-registry.oracle.com/ords/f?p=113:4:113387942129427:::4:P4_REPOSITORY,AI_REPOSITORY,AI_REPOSITORY_NAME,P4_REPOSITORY_NAME,P4_EULA_ID,P4_BUSINESS_AREA_ID:1183,1183,Oracle%20REST%20Data%20Services%20(ORDS)%20with%20Application%20Express,Oracle%20REST%20Data%20Services%20(ORDS)%20with%20Application%20Express,1,0&cs=3_y-KlneZIxRRfXzerC_0ro7P1MGh-B_9lTEQObVTdoQCWkmsQ3lHpFs90Z8QFheteVQEzPvtUVHEQAqqXegYbA). kubectl apply -f oraclerestdataservice_apex.yaml -* On the other hand, to provision ORDS step by step, set `.spec.apexPassword.secretName` to a non-null string in [config/samples/sidb/oraclerestdataservice.yaml](../../config/samples/sidb/oraclerestdataservice.yaml) -* This is used as a common password for APEX_PUBLIC_USER, APEX_REST_PUBLIC_USER, APEX_LISTENER and Apex administrator (username: ADMIN) mapped to secretKey +* The secret apexPassword contains the common password for APEX_PUBLIC_USER, APEX_REST_PUBLIC_USER, APEX_LISTENER and Apex administrator (username: ADMIN) * Status of ORDS turns to 'Updating' during apex configuration and turns 'Healthy' after successful configuration. You can also check status using below cmd ```sh @@ -571,10 +550,10 @@ To access APEX, You need to configure APEX with the ORDS. The following section * If you configure APEX after ORDS is installed, ORDS pods will be deleted and recreated -Application Express can be accessed via browser using `.status.databaseApiUrl` in the following command .\ +Application Express can be accessed via browser using `.status.apexUrl` in the following command .\ ```sh -$ kubectl get oraclerestdataservice/ords-sample --template={{.status.databaseApiUrl}} +$ kubectl get oraclerestdataservice/ords-sample --template={{.status.apexUrl}} https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/ ``` From 4bfb002ae080f60d8fa256d289e2f092402c381a Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Tue, 3 May 2022 10:15:40 +0530 Subject: [PATCH 329/628] General enhancements --- .../v1alpha1/oraclerestdataservice_types.go | 9 ++-- .../v1alpha1/oraclerestdataservice_webhook.go | 11 ++++- .../v1alpha1/singleinstancedatabase_types.go | 3 +- .../singleinstancedatabase_webhook.go | 7 ++- commons/database/constants.go | 2 - ...ase.oracle.com_oraclerestdataservices.yaml | 23 +++++----- ...se.oracle.com_singleinstancedatabases.yaml | 15 +++---- .../samples/sidb/oraclerestdataservice.yaml | 2 +- .../sidb/oraclerestdataservice_apex.yaml | 6 --- .../sidb/oraclerestdataservice_create.yaml | 3 -- .../samples/sidb/singleinstancedatabase.yaml | 11 ++--- .../sidb/singleinstancedatabase_clone.yaml | 1 - .../sidb/singleinstancedatabase_create.yaml | 1 - .../sidb/singleinstancedatabase_express.yaml | 1 - .../sidb/singleinstancedatabase_patch.yaml | 1 - .../singleinstancedatabase_prebuiltdb.yaml | 1 - .../oraclerestdataservice_controller.go | 44 +++++++++++++------ .../singleinstancedatabase_controller.go | 18 ++++---- 18 files changed, 87 insertions(+), 72 deletions(-) diff --git a/apis/database/v1alpha1/oraclerestdataservice_types.go b/apis/database/v1alpha1/oraclerestdataservice_types.go index 0c90e2b3..7666a6a3 100644 --- a/apis/database/v1alpha1/oraclerestdataservice_types.go +++ b/apis/database/v1alpha1/oraclerestdataservice_types.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** @@ -86,9 +86,10 @@ type OracleRestDataServiceImage struct { // OracleRestDataServicePassword defines the secret containing Password mapped to secretKey type OracleRestDataServicePassword struct { - SecretName string `json:"secretName,omitempty"` - SecretKey string `json:"secretKey"` - KeepSecret bool `json:"keepSecret"` + SecretName string `json:"secretName"` + // +kubebuilder:default:="oracle_pwd" + SecretKey string `json:"secretKey,omitempty"` + KeepSecret *bool `json:"keepSecret,omitempty"` } // OracleRestDataServicePDBSchemas defines the PDB Schemas to be ORDS Enabled diff --git a/apis/database/v1alpha1/oraclerestdataservice_webhook.go b/apis/database/v1alpha1/oraclerestdataservice_webhook.go index 3a546d46..bb889943 100644 --- a/apis/database/v1alpha1/oraclerestdataservice_webhook.go +++ b/apis/database/v1alpha1/oraclerestdataservice_webhook.go @@ -70,7 +70,16 @@ func (r *OracleRestDataService) Default() { oraclerestdataservicelog.Info("default", "name", r.Name) // OracleRestDataService Currently supports single replica r.Spec.Replicas = 1 - // TODO(user): fill in your defaulting logic. + keepSecret := true + if r.Spec.OrdsPassword.KeepSecret == nil { + r.Spec.OrdsPassword.KeepSecret = &keepSecret + } + if r.Spec.ApexPassword.KeepSecret == nil { + r.Spec.ApexPassword.KeepSecret = &keepSecret + } + if r.Spec.AdminPassword.KeepSecret == nil { + r.Spec.AdminPassword.KeepSecret = &keepSecret + } } // TODO(user): change verbs to "verbs=create;update;delete" if you want to enable deletion validation. diff --git a/apis/database/v1alpha1/singleinstancedatabase_types.go b/apis/database/v1alpha1/singleinstancedatabase_types.go index 3919738c..9687fb1b 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_types.go +++ b/apis/database/v1alpha1/singleinstancedatabase_types.go @@ -104,7 +104,8 @@ type SingleInstanceDatabaseImage struct { // SingleInsatnceAdminPassword defines the secret containing Admin Password mapped to secretKey for Database type SingleInstanceDatabaseAdminPassword struct { SecretName string `json:"secretName"` - SecretKey string `json:"secretKey"` + // +kubebuilder:default:="oracle_pwd" + SecretKey string `json:"secretKey,omitempty"` KeepSecret *bool `json:"keepSecret,omitempty"` } diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index 96c6e683..eba392ab 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -139,13 +139,18 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { if r.Spec.Edition == "express" && strings.ToUpper(r.Spec.Sid) != "XE" { allErrs = append(allErrs, field.Invalid(field.NewPath("spec").Child("sid"), r.Spec.Sid, - "Express edition SID must be XE")) + "Express edition SID must only be XE")) } if r.Spec.Edition == "express" && strings.ToUpper(r.Spec.Pdbname) != "XEPDB1" { allErrs = append(allErrs, field.Invalid(field.NewPath("spec").Child("pdbName"), r.Spec.Pdbname, "Express edition PDB must be XEPDB1")) } + if r.Spec.Edition != "express" && r.Spec.Sid == "XE" { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("sid"), r.Spec.Sid, + "XE is reserved as the SID for Express edition of the database")) + } if r.Spec.CloneFrom != "" { if r.Spec.Image.PrebuiltDB { diff --git a/commons/database/constants.go b/commons/database/constants.go index 216b00a7..f9257179 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -368,8 +368,6 @@ const StatusReady string = "Healthy" const StatusError string = "Error" -const StatusUnavailable string = "Unavailable" - const ValueUnavailable string = "Unavailable" const NoExternalIp string = "Node ExternalIP unavailable" diff --git a/config/crd/bases/database.oracle.com_oraclerestdataservices.yaml b/config/crd/bases/database.oracle.com_oraclerestdataservices.yaml index 13e4edd8..88fdbcf5 100644 --- a/config/crd/bases/database.oracle.com_oraclerestdataservices.yaml +++ b/config/crd/bases/database.oracle.com_oraclerestdataservices.yaml @@ -24,10 +24,13 @@ spec: name: Database type: string - jsonPath: .status.databaseApiUrl - name: 'Database Api & Apex Url ' + name: Database API URL type: string - jsonPath: .status.databaseActionsUrl - name: Database Actions Url + name: Database Actions URL + type: string + - jsonPath: .status.apexUrl + name: Apex URL type: string name: v1alpha1 schema: @@ -57,12 +60,12 @@ spec: keepSecret: type: boolean secretKey: + default: oracle_pwd type: string secretName: type: string required: - - keepSecret - - secretKey + - secretName type: object apexPassword: description: OracleRestDataServicePassword defines the secret containing @@ -71,12 +74,12 @@ spec: keepSecret: type: boolean secretKey: + default: oracle_pwd type: string secretName: type: string required: - - keepSecret - - secretKey + - secretName type: object databaseRef: type: string @@ -108,12 +111,12 @@ spec: keepSecret: type: boolean secretKey: + default: oracle_pwd type: string secretName: type: string required: - - keepSecret - - secretKey + - secretName type: object ordsUser: type: string @@ -166,7 +169,7 @@ spec: properties: apexConfigured: type: boolean - clusterDbApiUrl: + apexUrl: type: string commonUsersCreated: type: boolean @@ -193,8 +196,6 @@ spec: type: string ordsInstalled: type: boolean - ordsSetupCompleted: - type: boolean replicas: type: integer serviceIP: diff --git a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml index 02666bfc..7b59758f 100644 --- a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml +++ b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml @@ -68,11 +68,11 @@ spec: keepSecret: type: boolean secretKey: + default: oracle_pwd type: string secretName: type: string required: - - secretKey - secretName type: object archiveLog: @@ -95,6 +95,8 @@ spec: description: SingleInstanceDatabaseImage defines the Image source and pullSecrets for POD properties: + prebuiltDB: + type: boolean pullFrom: type: string pullSecrets: @@ -116,8 +118,6 @@ spec: sgaTarget: type: integer type: object - installApex: - type: boolean loadBalancer: type: boolean nodeSelector: @@ -136,9 +136,6 @@ spec: type: string storageClass: type: string - required: - - size - - storageClass type: object readinessCheckPeriod: type: integer @@ -290,10 +287,9 @@ spec: type: string storageClass: type: string - required: - - size - - storageClass type: object + prebuiltDB: + type: boolean releaseUpdate: type: string replicas: @@ -310,7 +306,6 @@ spec: type: string required: - persistence - - replicas type: object type: object served: true diff --git a/config/samples/sidb/oraclerestdataservice.yaml b/config/samples/sidb/oraclerestdataservice.yaml index 0e6d66a9..3d406fbd 100644 --- a/config/samples/sidb/oraclerestdataservice.yaml +++ b/config/samples/sidb/oraclerestdataservice.yaml @@ -19,7 +19,7 @@ spec: secretKey: keepSecret: true - ## Secret containing ORDS_PUBLIC_USER password mapped to secretKey. + ## Secret containing ORDS_PUBLIC_USER password mapped to secretKey. secretKey defaults to oracle_pwd ## This secret will be deleted after ORDS Installation unless keepSecret set to true. ordsPassword: secretName: diff --git a/config/samples/sidb/oraclerestdataservice_apex.yaml b/config/samples/sidb/oraclerestdataservice_apex.yaml index 073fc073..4bf5a476 100644 --- a/config/samples/sidb/oraclerestdataservice_apex.yaml +++ b/config/samples/sidb/oraclerestdataservice_apex.yaml @@ -37,18 +37,13 @@ spec: databaseRef: "prebuiltdb-sample" ## Secret containing databaseRef password mapped to secretKey. - ## This secret will be deleted after ORDS Installation unless keepSecret set to true. adminPassword: secretName: prebuiltdb-admin-secret - secretKey: oracle_pwd - keepSecret: true ## Secret containing ORDS_PUBLIC_USER password mapped to secretKey. - ## This secret will be deleted after ORDS Installation unless keepSecret set to true. ordsPassword: secretName: ords-secret secretKey: oracle_pwd - keepSecret: true ## To configure APEX with ORDS, specfiy the apexPassword secret details. Leave empty if Apex is not needed. ## This is a secret containing a common password for APEX_PUBLIC_USER, APEX_REST_PUBLIC_USER, APEX_LISTENER and Apex administrator (username: ADMIN) mapped to secretKey @@ -56,7 +51,6 @@ spec: apexPassword: secretName: apex-secret secretKey: oracle_pwd - keepSecret: true ## ORDS image details image: diff --git a/config/samples/sidb/oraclerestdataservice_create.yaml b/config/samples/sidb/oraclerestdataservice_create.yaml index dc00429f..940c6e9f 100644 --- a/config/samples/sidb/oraclerestdataservice_create.yaml +++ b/config/samples/sidb/oraclerestdataservice_create.yaml @@ -29,15 +29,12 @@ spec: ## This secret will be deleted after ORDS Installation unless keepSecret set to true. adminPassword: secretName: prebuiltdb-admin-secret - secretKey: oracle_pwd - keepSecret: true ## Secret containing ORDS_PUBLIC_USER password mapped to secretKey. ## This secret will be deleted after ORDS Installation unless keepSecret set to true. ordsPassword: secretName: ords-secret secretKey: oracle_pwd - keepSecret: true ## ORDS image details image: diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index 0ba28d8d..23a83fd1 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -9,7 +9,7 @@ metadata: namespace: default spec: - ## Use only alphanumeric characters for sid + ## Use only alphanumeric characters for sid up to a maximum of 8 characters sid: ORCL1 ## Specify a source database ref to copy/clone from any SIDB in current K8s cluster instead of creating a fresh one. @@ -17,10 +17,11 @@ spec: ## specify connect string as `:/` instead of source database ref cloneFrom: "" - ## DB edition. N/A if cloning from a SourceDB (cloneFrom is set) + ## DB edition. N/A if cloning from a Source DB (if cloneFrom is set) + ## Valid values for edition are enterprise, standard or express edition: enterprise - ## Secret containing SIDB password mapped to secretKey + ## Secret containing SIDB password mapped to secretKey. secretKey defaults to oracle_pwd ## Should refer to adminPassword of Source DB if cloning from a Source DB (i.e if cloneFrom is set) ## This secret will be deleted after creation of the database unless keepSecret is set to true which is the default adminPassword: @@ -28,10 +29,10 @@ spec: secretKey: keepSecret: true - ## DB character set. N/A if cloning from a SourceDB (cloneFrom is set) + ## DB character set. N/A if cloning from a Source DB (if cloneFrom is set) charset: AL32UTF8 - ## PDB name. N/A if cloning from a SourceDB (cloneFrom is set) + ## PDB name. N/A if cloning from a Source DB (if cloneFrom is set) pdbName: orclpdb1 ## Enable/Disable Flashback diff --git a/config/samples/sidb/singleinstancedatabase_clone.yaml b/config/samples/sidb/singleinstancedatabase_clone.yaml index acf6a4c2..1fab477d 100644 --- a/config/samples/sidb/singleinstancedatabase_clone.yaml +++ b/config/samples/sidb/singleinstancedatabase_clone.yaml @@ -21,7 +21,6 @@ spec: ## Secret containing SIDB password mapped to secretKey adminPassword: secretName: db-admin-secret - secretKey: oracle_pwd ## Database image details ## This image should be the same as the source DB image being cloned diff --git a/config/samples/sidb/singleinstancedatabase_create.yaml b/config/samples/sidb/singleinstancedatabase_create.yaml index a8f7430f..1038614d 100644 --- a/config/samples/sidb/singleinstancedatabase_create.yaml +++ b/config/samples/sidb/singleinstancedatabase_create.yaml @@ -32,7 +32,6 @@ spec: ## Secret containing SIDB password mapped to secretKey adminPassword: secretName: db-admin-secret - secretKey: oracle_pwd ## DB character set charset: AL32UTF8 diff --git a/config/samples/sidb/singleinstancedatabase_express.yaml b/config/samples/sidb/singleinstancedatabase_express.yaml index 3f1a3927..68c7e785 100644 --- a/config/samples/sidb/singleinstancedatabase_express.yaml +++ b/config/samples/sidb/singleinstancedatabase_express.yaml @@ -30,7 +30,6 @@ spec: ## Secret containing SIDB password mapped to secretKey adminPassword: secretName: xedb-admin-secret - secretKey: oracle_pwd ## Database image details image: diff --git a/config/samples/sidb/singleinstancedatabase_patch.yaml b/config/samples/sidb/singleinstancedatabase_patch.yaml index cdb676c7..134642a0 100644 --- a/config/samples/sidb/singleinstancedatabase_patch.yaml +++ b/config/samples/sidb/singleinstancedatabase_patch.yaml @@ -20,7 +20,6 @@ spec: ## Secret containing SIDB password mapped to secretKey adminPassword: secretName: db-admin-secret - secretKey: oracle_pwd ## DB character set charset: AL32UTF8 diff --git a/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml b/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml index ae0e15df..ab6047d4 100644 --- a/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml +++ b/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml @@ -30,7 +30,6 @@ spec: ## Secret containing SIDB password mapped to secretKey adminPassword: secretName: prebuiltdb-admin-secret - secretKey: oracle_pwd ## Database Image image: diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 96afb4a7..41869942 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -208,6 +208,21 @@ func (r *OracleRestDataServiceReconciler) validate(m *dbapi.OracleRestDataServic eventReason := "Spec Error" var eventMsgs []string + /* Initialize Status */ + if m.Status.Status == "" { + m.Status.Status = dbcommons.StatusPending + } + if m.Status.ApxeUrl == "" { + m.Status.ApxeUrl = dbcommons.ValueUnavailable + } + if m.Status.DatabaseApiUrl == "" { + m.Status.DatabaseApiUrl = dbcommons.ValueUnavailable + } + if m.Status.DatabaseActionsUrl == "" { + m.Status.DatabaseActionsUrl = dbcommons.ValueUnavailable + } + + //First check image pull secrets if m.Spec.Image.PullSecrets != "" { secret := &corev1.Secret{} @@ -701,8 +716,6 @@ func (r *OracleRestDataServiceReconciler) createSVC(ctx context.Context, req ctr if m.Status.ApexConfigured { m.Status.ApxeUrl = "https://" + svc.Status.LoadBalancer.Ingress[0].IP + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/ords/" + n.Status.Pdbname + "/apex" - } else { - m.Status.ApxeUrl = dbcommons.StatusUnavailable } } return requeueN @@ -717,8 +730,6 @@ func (r *OracleRestDataServiceReconciler) createSVC(ctx context.Context, req ctr if m.Status.ApexConfigured { m.Status.ApxeUrl = "https://" + nodeip + ":" + fmt.Sprint(svc.Spec.Ports[0].NodePort) + "/ords/" + n.Status.Pdbname + "/apex" - } else { - m.Status.ApxeUrl = dbcommons.StatusUnavailable } } return requeueN @@ -766,21 +777,28 @@ func (r *OracleRestDataServiceReconciler) createPods(m *dbapi.OracleRestDataServ log := r.Log.WithValues("createPods", req.NamespacedName) - readyPod, replicasFound, available, _, err := dbcommons.FindPods(r, m.Spec.Image.Version, + readyPod, replicasFound, available, podsMarkedToBeDeleted, err := dbcommons.FindPods(r, m.Spec.Image.Version, m.Spec.Image.PullFrom, m.Name, m.Namespace, ctx, req) if err != nil { log.Error(err, err.Error()) return requeueY } + // Recreate new pods only after earlier pods are terminated completely + for i := 0; i < len(podsMarkedToBeDeleted); i++ { + r.Log.Info("Force deleting pod ", "name", podsMarkedToBeDeleted[i].Name, "phase", podsMarkedToBeDeleted[i].Status.Phase) + var gracePeriodSeconds int64 = 0 + policy := metav1.DeletePropagationForeground + r.Delete(ctx, &podsMarkedToBeDeleted[i], &client.DeleteOptions{ + GracePeriodSeconds: &gracePeriodSeconds, PropagationPolicy: &policy }) + } + log.Info(m.Name, " pods other than one of Ready Pods : ", dbcommons.GetPodNames(available)) log.Info(m.Name, " Ready Pod : ", readyPod.Name) replicasReq := m.Spec.Replicas if replicasFound == 0 { m.Status.Status = dbcommons.StatusNotReady - m.Status.DatabaseApiUrl = dbcommons.StatusUnavailable - m.Status.DatabaseActionsUrl = dbcommons.StatusUnavailable } if replicasFound == replicasReq { @@ -825,9 +843,7 @@ func (r *OracleRestDataServiceReconciler) createPods(m *dbapi.OracleRestDataServ n.Status.OrdsReference = m.Name r.Status().Update(ctx, n) m.Status.Replicas = m.Spec.Replicas - if !m.Status.OrdsInstalled { - m.Status.Replicas = 1 - } + return requeueN } @@ -1000,7 +1016,7 @@ func (r *OracleRestDataServiceReconciler) cleanupOracleRestDataService(req ctrl. GracePeriodSeconds: &gracePeriodSeconds, PropagationPolicy: &policy }) //Delete Database Admin Password Secret - if !m.Spec.AdminPassword.KeepSecret { + if !*m.Spec.AdminPassword.KeepSecret { err = r.Delete(ctx, adminPasswordSecret, &client.DeleteOptions{}) if err == nil { r.Log.Info("Deleted Admin Password Secret :" + adminPasswordSecret.Name) @@ -1150,7 +1166,7 @@ func (r *OracleRestDataServiceReconciler) installApex(m *dbapi.OracleRestDataSer func (r *OracleRestDataServiceReconciler) deleteSecrets(m *dbapi.OracleRestDataService, ctx context.Context, req ctrl.Request) { log := r.Log.WithValues("deleteSecrets", req.NamespacedName) - if !m.Spec.AdminPassword.KeepSecret { + if !*m.Spec.AdminPassword.KeepSecret { // Fetch adminPassword Secret adminPasswordSecret := &corev1.Secret{} err := r.Get(ctx, types.NamespacedName{Name: m.Spec.AdminPassword.SecretName, Namespace: m.Namespace}, adminPasswordSecret) @@ -1163,7 +1179,7 @@ func (r *OracleRestDataServiceReconciler) deleteSecrets(m *dbapi.OracleRestDataS } } - if !m.Spec.OrdsPassword.KeepSecret { + if !*m.Spec.OrdsPassword.KeepSecret { // Fetch ordsPassword Secret ordsPasswordSecret := &corev1.Secret{} err := r.Get(ctx, types.NamespacedName{Name: m.Spec.OrdsPassword.SecretName, Namespace: m.Namespace}, ordsPasswordSecret) @@ -1176,7 +1192,7 @@ func (r *OracleRestDataServiceReconciler) deleteSecrets(m *dbapi.OracleRestDataS } } - if !m.Spec.ApexPassword.KeepSecret { + if !*m.Spec.ApexPassword.KeepSecret { // Fetch apexPassword Secret apexPasswordSecret := &corev1.Secret{} err := r.Get(ctx, types.NamespacedName{Name: m.Spec.ApexPassword.SecretName, Namespace: m.Namespace}, apexPasswordSecret) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 6882710d..a73cec36 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -303,7 +303,10 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab r.Log.Info("Entering reconcile validation") - /* Initialize statuses */ + /* Initialize Status */ + if m.Status.Status == "" { + m.Status.Status = dbcommons.StatusPending + } if m.Status.Edition == "" { if m.Spec.Edition != "" { m.Status.Edition = strings.Title(m.Spec.Edition) @@ -344,9 +347,13 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab } } - // If Express Edition , Ensure Replicas=1 + // If Express Edition, ensure Replicas=1 if m.Spec.Edition == "express" && m.Spec.Replicas > 1 { - eventMsgs = append(eventMsgs, "XE supports only one replica") + eventMsgs = append(eventMsgs, "express edition supports only one replica") + } + // If no persistence, ensure Replicas=1 + if m.Spec.Persistence.Size == "" && m.Spec.Replicas > 1 { + eventMsgs = append(eventMsgs, "replicas should be 1 if no persistence is specified") } if m.Status.Sid != "" && !strings.EqualFold(m.Spec.Sid, m.Status.Sid) { eventMsgs = append(eventMsgs, "sid cannot be updated") @@ -1946,11 +1953,6 @@ func (r *SingleInstanceDatabaseReconciler) cleanupSingleInstanceDatabase(req ctr for _, pod := range podList.Items { podNames += pod.Name + " " } - eventReason := "Waiting" - eventMsg := "waiting for " + req.Name + " database pods ( " + podNames + " ) to terminate" - r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) - r.Log.Info(eventMsg) - time.Sleep(15 * time.Second) } log.Info("Successfully cleaned up SingleInstanceDatabase") From 917d6c934d0b7037e983ad006e1fa3d2e99e6157 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Tue, 3 May 2022 20:25:52 +0530 Subject: [PATCH 330/628] Default few ORDS params --- .../sidb/oraclerestdataservice_apex.yaml | 9 +- .../sidb/oraclerestdataservice_create.yaml | 4 - .../oraclerestdataservice_controller.go | 94 ++++++++++++++----- 3 files changed, 75 insertions(+), 32 deletions(-) diff --git a/config/samples/sidb/oraclerestdataservice_apex.yaml b/config/samples/sidb/oraclerestdataservice_apex.yaml index 4bf5a476..2a586cc8 100644 --- a/config/samples/sidb/oraclerestdataservice_apex.yaml +++ b/config/samples/sidb/oraclerestdataservice_apex.yaml @@ -36,21 +36,18 @@ spec: ## Make sure the source database has been created by applying singeinstancedatabase_prebuiltdb.yaml databaseRef: "prebuiltdb-sample" - ## Secret containing databaseRef password mapped to secretKey. + ## Secret containing databaseRef password adminPassword: secretName: prebuiltdb-admin-secret - ## Secret containing ORDS_PUBLIC_USER password mapped to secretKey. + ## Secret containing ORDS_PUBLIC_USER password ordsPassword: secretName: ords-secret - secretKey: oracle_pwd ## To configure APEX with ORDS, specfiy the apexPassword secret details. Leave empty if Apex is not needed. - ## This is a secret containing a common password for APEX_PUBLIC_USER, APEX_REST_PUBLIC_USER, APEX_LISTENER and Apex administrator (username: ADMIN) mapped to secretKey - ## This secret will be deleted after ORDS Installation unless keepSecret set to true. + ## This is a secret containing a common password for APEX_PUBLIC_USER, APEX_REST_PUBLIC_USER, APEX_LISTENER and Apex administrator (username: ADMIN) apexPassword: secretName: apex-secret - secretKey: oracle_pwd ## ORDS image details image: diff --git a/config/samples/sidb/oraclerestdataservice_create.yaml b/config/samples/sidb/oraclerestdataservice_create.yaml index 940c6e9f..c442efae 100644 --- a/config/samples/sidb/oraclerestdataservice_create.yaml +++ b/config/samples/sidb/oraclerestdataservice_create.yaml @@ -26,15 +26,12 @@ spec: databaseRef: "prebuiltdb-sample" ## Secret containing databaseRef password mapped to secretKey. - ## This secret will be deleted after ORDS Installation unless keepSecret set to true. adminPassword: secretName: prebuiltdb-admin-secret ## Secret containing ORDS_PUBLIC_USER password mapped to secretKey. - ## This secret will be deleted after ORDS Installation unless keepSecret set to true. ordsPassword: secretName: ords-secret - secretKey: oracle_pwd ## ORDS image details image: @@ -48,7 +45,6 @@ spec: storageClass: "oci-bv" accessMode: "ReadWriteOnce" - ## PDB Schemas to be ORDS Enabled. ## Schema will be created (if not exists) with username as schema and password as .spec.ordsPassword. restEnableSchemas: diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 41869942..1d3ec3ae 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -431,7 +431,24 @@ func (r *OracleRestDataServiceReconciler) instantiateSVCSpec(m *dbapi.OracleRest // Instantiate POD spec from OracleRestDataService spec //############################################################################# func (r *OracleRestDataServiceReconciler) instantiatePodSpec(m *dbapi.OracleRestDataService, - n *dbapi.SingleInstanceDatabase) *corev1.Pod { + n *dbapi.SingleInstanceDatabase) (*corev1.Pod, *corev1.Secret) { + + initSecret := &corev1.Secret { + TypeMeta: metav1.TypeMeta{ + Kind: "Secret", + }, + ObjectMeta: metav1.ObjectMeta{ + Name: m.Name, + Namespace: m.Namespace, + Labels: map[string]string{ + "app": m.Name, + }, + }, + Type: corev1.SecretTypeOpaque, + StringData: map[string]string { + "init-cmd": dbcommons.InitORDSCMD, + }, + } pod := &corev1.Pod{ TypeMeta: metav1.TypeMeta{ @@ -446,20 +463,35 @@ func (r *OracleRestDataServiceReconciler) instantiatePodSpec(m *dbapi.OracleRest }, }, Spec: corev1.PodSpec{ - Volumes: []corev1.Volume{{ - Name: "datamount", - VolumeSource: corev1.VolumeSource{ - PersistentVolumeClaim: &corev1.PersistentVolumeClaimVolumeSource{ - ClaimName: func() string { - if m.Spec.Persistence.AccessMode != "" { - return m.Name - } - return n.Name - }(), - ReadOnly: false, + Volumes: []corev1.Volume{ + { + Name: "datamount", + VolumeSource: corev1.VolumeSource{ + PersistentVolumeClaim: &corev1.PersistentVolumeClaimVolumeSource{ + ClaimName: func() string { + if m.Spec.Persistence.AccessMode != "" { + return m.Name + } + return n.Name + }(), + ReadOnly: false, + }, }, }, - }}, + { + Name: "init-ords-vol", + VolumeSource: corev1.VolumeSource{ + Secret: &corev1.SecretVolumeSource{ + SecretName: m.Name, + Optional: func() *bool { i := true; return &i }(), + Items: []corev1.KeyToPath{{ + Key: "init-cmd", + Path: "init-cmd", + }}, + }, + }, + }, + }, InitContainers: []corev1.Container{ { Name: "init-permissions", @@ -478,16 +510,24 @@ func (r *OracleRestDataServiceReconciler) instantiatePodSpec(m *dbapi.OracleRest { Name: "init-ords", Image: m.Spec.Image.PullFrom, - Command: []string{"/bin/sh", "-c", dbcommons.InitORDSCMD}, + Command: []string{"/bin/sh", "-c", "/run/secrets/init-cmd"}, SecurityContext: &corev1.SecurityContext{ RunAsUser: func() *int64 { i := int64(dbcommons.ORACLE_UID); return &i }(), RunAsGroup: func() *int64 { i := int64(dbcommons.DBA_GUID); return &i }(), }, - VolumeMounts: []corev1.VolumeMount{{ - MountPath: "/opt/oracle/ords/config/ords", - Name: "datamount", - SubPath: strings.ToUpper(n.Spec.Sid) + "_ORDS", - }}, + VolumeMounts: []corev1.VolumeMount{ + { + MountPath: "/opt/oracle/ords/config/ords", + Name: "datamount", + SubPath: strings.ToUpper(n.Spec.Sid) + "_ORDS", + }, + { + MountPath: "/run/secrets/init-cmd", + ReadOnly: true, + Name: "init-ord-vol", + SubPath: "init-cmd", + }, + }, Env: []corev1.EnvVar{ { Name: "ORACLE_HOST", @@ -613,8 +653,9 @@ func (r *OracleRestDataServiceReconciler) instantiatePodSpec(m *dbapi.OracleRest } // Set oracleRestDataService instance as the owner and controller + ctrl.SetControllerReference(m, initSecret, r.Scheme) ctrl.SetControllerReference(m, pod, r.Scheme) - return pod + return pod, initSecret } //############################################################################# @@ -806,9 +847,18 @@ func (r *OracleRestDataServiceReconciler) createPods(m *dbapi.OracleRestDataServ } else if replicasFound < replicasReq { // Create New Pods , Name of Pods are generated Randomly for i := replicasFound; i < replicasReq; i++ { - pod := r.instantiatePodSpec(m, n) + pod, initSecret := r.instantiatePodSpec(m, n) + // Check if init-secret is present + err := r.Get(ctx, types.NamespacedName{Name: m.Name, Namespace: m.Namespace}, &corev1.Secret{}) + if err != nil && apierrors.IsNotFound(err) { + log.Info("Creating a new secret", "name", m.Name) + if err = r.Create(ctx, initSecret); err != nil { + log.Error(err, "Failed to create secret ", "Namespace", initSecret.Namespace, "Name", initSecret.Name) + return requeueY + } + } log.Info("Creating a new "+m.Name+" POD", "POD.Namespace", pod.Namespace, "POD.Name", pod.Name) - err := r.Create(ctx, pod) + err = r.Create(ctx, pod) if err != nil { log.Error(err, "Failed to create new "+m.Name+" POD", "pod.Namespace", pod.Namespace, "POD.Name", pod.Name) return requeueY From 720505154f38a1d866f96d2036618692b2509737 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Tue, 3 May 2022 20:58:40 +0530 Subject: [PATCH 331/628] Fix vol name --- controllers/database/oraclerestdataservice_controller.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 1d3ec3ae..86d9b872 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -524,7 +524,7 @@ func (r *OracleRestDataServiceReconciler) instantiatePodSpec(m *dbapi.OracleRest { MountPath: "/run/secrets/init-cmd", ReadOnly: true, - Name: "init-ord-vol", + Name: "init-ords-vol", SubPath: "init-cmd", }, }, From 37c20e68a59f1a9345445de59ba733c676097c94 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Tue, 3 May 2022 21:06:20 +0530 Subject: [PATCH 332/628] Fix init command --- controllers/database/oraclerestdataservice_controller.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 86d9b872..d38a7e51 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -510,7 +510,7 @@ func (r *OracleRestDataServiceReconciler) instantiatePodSpec(m *dbapi.OracleRest { Name: "init-ords", Image: m.Spec.Image.PullFrom, - Command: []string{"/bin/sh", "-c", "/run/secrets/init-cmd"}, + Command: []string{"/bin/sh", "/run/secrets/init-cmd"}, SecurityContext: &corev1.SecurityContext{ RunAsUser: func() *int64 { i := int64(dbcommons.ORACLE_UID); return &i }(), RunAsGroup: func() *int64 { i := int64(dbcommons.DBA_GUID); return &i }(), From f08fb3dc498aa15924d115238095851b4dd52327 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Tue, 3 May 2022 22:57:21 +0530 Subject: [PATCH 333/628] Update ORDS samples --- .../v1alpha1/oraclerestdataservice_types.go | 8 ++-- commons/database/constants.go | 3 ++ ...ase.oracle.com_oraclerestdataservices.yaml | 7 ++- .../samples/sidb/oraclerestdataservice.yaml | 11 ++--- .../sidb/oraclerestdataservice_apex.yaml | 17 ++++--- .../sidb/oraclerestdataservice_create.yaml | 17 ++++--- .../oraclerestdataservice_controller.go | 45 +++++++++++++------ 7 files changed, 63 insertions(+), 45 deletions(-) diff --git a/apis/database/v1alpha1/oraclerestdataservice_types.go b/apis/database/v1alpha1/oraclerestdataservice_types.go index 7666a6a3..7a88d8bf 100644 --- a/apis/database/v1alpha1/oraclerestdataservice_types.go +++ b/apis/database/v1alpha1/oraclerestdataservice_types.go @@ -94,10 +94,10 @@ type OracleRestDataServicePassword struct { // OracleRestDataServicePDBSchemas defines the PDB Schemas to be ORDS Enabled type OracleRestDataServiceRestEnableSchemas struct { - Pdb string `json:"pdb"` - Schema string `json:"schema"` - UrlMapping string `json:"urlMapping,omitempty"` - Enable bool `json:"enable"` + PdbName string `json:"pdbName,omitempty"` + SchemaName string `json:"schemaName"` + UrlMapping string `json:"urlMapping,omitempty"` + Enable bool `json:"enable"` } // OracleRestDataServiceStatus defines the observed state of OracleRestDataService diff --git a/commons/database/constants.go b/commons/database/constants.go index f9257179..5dc54d25 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -216,6 +216,9 @@ const GetStandardEditionFileCMD string = "if [ -f ${ORACLE_BASE}/oradata/dbconfi const GetPdbsSQL string = "select name from v\\$pdbs where name not like 'PDB\\$SEED' and open_mode like 'READ WRITE';" +const OpenPDBSeed = "alter pluggable database pdb\\$seed close;" + + "\nalter pluggable database pdb\\$seed open;" + const SetAdminUsersSQL string = "CREATE USER C##DBAPI_CDB_ADMIN IDENTIFIED BY \\\"%[1]s\\\" ACCOUNT UNLOCK CONTAINER=ALL;" + "\nalter user C##DBAPI_CDB_ADMIN identified by \\\"%[1]s\\\" account unlock;" + "\nGRANT DBA TO C##DBAPI_CDB_ADMIN CONTAINER = ALL;" + diff --git a/config/crd/bases/database.oracle.com_oraclerestdataservices.yaml b/config/crd/bases/database.oracle.com_oraclerestdataservices.yaml index 88fdbcf5..8c426b34 100644 --- a/config/crd/bases/database.oracle.com_oraclerestdataservices.yaml +++ b/config/crd/bases/database.oracle.com_oraclerestdataservices.yaml @@ -144,16 +144,15 @@ spec: properties: enable: type: boolean - pdb: + pdbName: type: string - schema: + schemaName: type: string urlMapping: type: string required: - enable - - pdb - - schema + - schemaName type: object type: array serviceAccountName: diff --git a/config/samples/sidb/oraclerestdataservice.yaml b/config/samples/sidb/oraclerestdataservice.yaml index 3d406fbd..20c3fb13 100644 --- a/config/samples/sidb/oraclerestdataservice.yaml +++ b/config/samples/sidb/oraclerestdataservice.yaml @@ -60,13 +60,14 @@ spec: # nodeSelector: # failure-domain.beta.kubernetes.io/zone: PHX-AD-1 - ## PDB Schemas to be ORDS Enabled. - ## Schema will be created (if not exists) with username as schema and password as .spec.ordsPassword. + ## Schemas to be ORDS Enabled. + ## Schema will be created (if not exists) with password as .spec.ordsPassword. + ## pdbName defaults to the singleinstancedatabase .spec.pdbName restEnableSchemas: - - schema: + - schemaName: enable: true urlMapping: - pdb: + pdbName: ## If deploying on OpenShift, change service account name to 'sidb-sa' after you run `$ oc apply -f openshift_rbac.yaml` - serviceAccountName: default \ No newline at end of file + serviceAccountName: default diff --git a/config/samples/sidb/oraclerestdataservice_apex.yaml b/config/samples/sidb/oraclerestdataservice_apex.yaml index 2a586cc8..1fb6c0f0 100644 --- a/config/samples/sidb/oraclerestdataservice_apex.yaml +++ b/config/samples/sidb/oraclerestdataservice_apex.yaml @@ -33,12 +33,12 @@ metadata: spec: ## Database ref. This can be of kind SingleInstanceDatabase. - ## Make sure the source database has been created by applying singeinstancedatabase_prebuiltdb.yaml - databaseRef: "prebuiltdb-sample" + ## Make sure the source database has been created by applying singeinstancedatabase_express.yaml + databaseRef: "xedb-sample" ## Secret containing databaseRef password adminPassword: - secretName: prebuiltdb-admin-secret + secretName: xedb-admin-secret ## Secret containing ORDS_PUBLIC_USER password ordsPassword: @@ -62,14 +62,13 @@ spec: accessMode: "ReadWriteOnce" ## PDB Schemas to be ORDS Enabled. - ## Schema will be created (if not exists) with username as schema and password as .spec.ordsPassword. + ## Schema will be created (if not exists) with password as .spec.ordsPassword. restEnableSchemas: - - schema: schema1 + - schemaName: schema1 enable: true urlMapping: - pdb: xepdb1 - - schema: schema2 + - schemaName: schema2 enable: true - urlMapping: - pdb: xepdb1 + urlMapping: + \ No newline at end of file diff --git a/config/samples/sidb/oraclerestdataservice_create.yaml b/config/samples/sidb/oraclerestdataservice_create.yaml index c442efae..c2b204a7 100644 --- a/config/samples/sidb/oraclerestdataservice_create.yaml +++ b/config/samples/sidb/oraclerestdataservice_create.yaml @@ -22,12 +22,12 @@ metadata: spec: ## Database ref. This can be of kind SingleInstanceDatabase. - ## Make sure the source database has been created by applying singeinstancedatabase_prebuilt.yaml - databaseRef: "prebuiltdb-sample" + ## Make sure the source database has been created by applying singeinstancedatabase_express.yaml + databaseRef: "xedb-sample" ## Secret containing databaseRef password mapped to secretKey. adminPassword: - secretName: prebuiltdb-admin-secret + secretName: xedb-admin-secret ## Secret containing ORDS_PUBLIC_USER password mapped to secretKey. ordsPassword: @@ -46,16 +46,15 @@ spec: accessMode: "ReadWriteOnce" ## PDB Schemas to be ORDS Enabled. - ## Schema will be created (if not exists) with username as schema and password as .spec.ordsPassword. + ## Schema will be created (if not exists) with password as .spec.ordsPassword. restEnableSchemas: - - schema: schema1 + - schemaName: schema1 enable: true urlMapping: - pdb: xepdb1 - - schema: schema2 + - schemaName: schema2 enable: true - urlMapping: - pdb: xepdb1 + urlMapping: + \ No newline at end of file diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index d38a7e51..73910dc3 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -169,7 +169,7 @@ func (r *OracleRestDataServiceReconciler) Reconcile(ctx context.Context, req ctr } var ordsReadyPod corev1.Pod - result, ordsReadyPod = r.checkHealthStatus(oracleRestDataService, ctx, req) + result, ordsReadyPod = r.checkHealthStatus(oracleRestDataService, sidbReadyPod, ctx, req) if result.Requeue { r.Log.Info("Reconcile queued") return result, nil @@ -322,7 +322,7 @@ func (r *OracleRestDataServiceReconciler) validateSIDBReadiness(m *dbapi.OracleR } else if strings.Contains(out, "ORA-01017") { m.Status.Status = dbcommons.StatusError eventReason := "Logon denied" - eventMsg := "invalid databaseRef admin password. secret: " + m.Spec.AdminPassword.SecretName + eventMsg := "invalid databaseRef admin password secret: " + m.Spec.AdminPassword.SecretName r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) return requeueY, sidbReadyPod } else { @@ -350,7 +350,7 @@ func (r *OracleRestDataServiceReconciler) validateSIDBReadiness(m *dbapi.OracleR // Check ORDS Health Status //##################################################################################################### func (r *OracleRestDataServiceReconciler) checkHealthStatus(m *dbapi.OracleRestDataService, - ctx context.Context, req ctrl.Request) (ctrl.Result, corev1.Pod) { + sidbReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) (ctrl.Result, corev1.Pod) { log := r.Log.WithValues("checkHealthStatus", req.NamespacedName) readyPod, _, _, _, err := dbcommons.FindPods(r, m.Spec.Image.Version, @@ -380,7 +380,17 @@ func (r *OracleRestDataServiceReconciler) checkHealthStatus(m *dbapi.OracleRestD if strings.Contains(out, "HTTP/1.1 200 OK") || strings.Contains(strings.ToUpper(err.Error()), "HTTP/1.1 200 OK") { m.Status.Status = dbcommons.StatusReady - m.Status.OrdsInstalled = true + if !m.Status.OrdsInstalled { + m.Status.OrdsInstalled = true + out, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", + ctx, req, false, "bash", "-c", fmt.Sprintf("echo -e \"%s\" | %s", dbcommons.OpenPDBSeed, dbcommons.SQLPlusCLI)) + if err != nil { + log.Error(err, err.Error()) + } else { + log.Info("Close PDB seed") + log.Info(out) + } + } } else { m.Status.Status = dbcommons.StatusNotReady return requeueY, readyPod @@ -923,7 +933,7 @@ func (r *OracleRestDataServiceReconciler) manageOracleRestDataServiceDeletion(re err := r.Status().Update(ctx, n) if err != nil { log.Info(err.Error() + "\n updating n.Status.OrdsInstalled = false") - time.Sleep(5 * time.Second) + time.Sleep(1 * time.Second) continue } break @@ -938,7 +948,7 @@ func (r *OracleRestDataServiceReconciler) manageOracleRestDataServiceDeletion(re return requeueY } } - return requeueY + return requeueN } // Add finalizer for this CR @@ -1277,16 +1287,23 @@ func (r *OracleRestDataServiceReconciler) restEnableSchemas(m *dbapi.OracleRestD } for i := 0; i < len(m.Spec.RestEnableSchemas); i++ { + + pdbName := m.Spec.RestEnableSchemas[i].PdbName + if pdbName == "" { + pdbName = n.Spec.Pdbname + r.Log.Info("Defaulting PDB name", "name", pdbName) + } + // If the PDB mentioned in yaml doesnt contain in the database , continue - if !strings.Contains(strings.ToUpper(availablePDBS), strings.ToUpper(m.Spec.RestEnableSchemas[i].Pdb)) { + if !strings.Contains(strings.ToUpper(availablePDBS), strings.ToUpper(pdbName)) { eventReason := "Warning" - eventMsg := "enabling ORDS schema for PDB : " + m.Spec.RestEnableSchemas[i].Pdb + " failed ; as pdb not found" + eventMsg := "enabling ORDS schema for PDB : " + pdbName + " failed. PDB not found." log.Info(eventMsg) r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) continue } - getOrdsSchemaStatus := fmt.Sprintf(dbcommons.GetUserOrdsSchemaStatusSQL, m.Spec.RestEnableSchemas[i].Schema, m.Spec.RestEnableSchemas[i].Pdb) + getOrdsSchemaStatus := fmt.Sprintf(dbcommons.GetUserOrdsSchemaStatusSQL, m.Spec.RestEnableSchemas[i].SchemaName, pdbName) // Get ORDS Schema status for PDB out, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", @@ -1295,7 +1312,7 @@ func (r *OracleRestDataServiceReconciler) restEnableSchemas(m *dbapi.OracleRestD log.Error(err, err.Error()) return requeueY } else { - log.Info("getOrdsSchemaStatus Output", "schema", m.Spec.RestEnableSchemas[i].Schema) + log.Info("getOrdsSchemaStatus Output", "schema", m.Spec.RestEnableSchemas[i].SchemaName) log.Info(out) } @@ -1327,12 +1344,12 @@ func (r *OracleRestDataServiceReconciler) restEnableSchemas(m *dbapi.OracleRestD password := string(OrdsPasswordSecret.Data[m.Spec.OrdsPassword.SecretKey]) urlMappingPattern := "" if m.Spec.RestEnableSchemas[i].UrlMapping == "" { - urlMappingPattern = strings.ToLower(m.Spec.RestEnableSchemas[i].Schema) + urlMappingPattern = strings.ToLower(m.Spec.RestEnableSchemas[i].SchemaName) } else { urlMappingPattern = strings.ToLower(m.Spec.RestEnableSchemas[i].UrlMapping) } - enableORDSSchema := fmt.Sprintf(dbcommons.EnableORDSSchemaSQL, strings.ToUpper(m.Spec.RestEnableSchemas[i].Schema), password, - strconv.FormatBool(m.Spec.RestEnableSchemas[i].Enable), urlMappingPattern, m.Spec.RestEnableSchemas[i].Pdb) + enableORDSSchema := fmt.Sprintf(dbcommons.EnableORDSSchemaSQL, strings.ToUpper(m.Spec.RestEnableSchemas[i].SchemaName), password, + strconv.FormatBool(m.Spec.RestEnableSchemas[i].Enable), urlMappingPattern, pdbName) // Create users,schemas and grant enableORDS for PDB _, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", @@ -1341,7 +1358,7 @@ func (r *OracleRestDataServiceReconciler) restEnableSchemas(m *dbapi.OracleRestD log.Error(err, err.Error()) return requeueY } - log.Info("REST Enabled", "schema", m.Spec.RestEnableSchemas[i].Schema) + log.Info("REST Enabled", "schema", m.Spec.RestEnableSchemas[i].SchemaName) } From 7061309cb7de73ccfaf96e2b5731c95926da3197 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Tue, 3 May 2022 23:01:06 +0530 Subject: [PATCH 334/628] Fix clean up --- controllers/database/singleinstancedatabase_controller.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index a73cec36..bdd4e7a3 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1901,7 +1901,7 @@ func (r *SingleInstanceDatabaseReconciler) manageSingleInstanceDatabaseDeletion( } log.Info("Successfully Removed SingleInstanceDatabase Finalizer") } - return requeueY, errors.New("deletion pending") + return requeueN, nil } // Add finalizer for this CR From 132e8999bf0ec0899ef67c68dce4d6413132add4 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Tue, 3 May 2022 23:14:30 +0530 Subject: [PATCH 335/628] Fix requeue --- controllers/database/singleinstancedatabase_controller.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index bdd4e7a3..a73cec36 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1901,7 +1901,7 @@ func (r *SingleInstanceDatabaseReconciler) manageSingleInstanceDatabaseDeletion( } log.Info("Successfully Removed SingleInstanceDatabase Finalizer") } - return requeueN, nil + return requeueY, errors.New("deletion pending") } // Add finalizer for this CR From 6f3f4a8117a578d2d6761982d49dfa0d708e04d4 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Tue, 3 May 2022 23:34:37 +0530 Subject: [PATCH 336/628] Fix initialize status --- .../oraclerestdataservice_controller.go | 35 ++++++------ .../singleinstancedatabase_controller.go | 55 ++++++++++--------- 2 files changed, 46 insertions(+), 44 deletions(-) diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 73910dc3..1092edc3 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -101,7 +101,22 @@ func (r *OracleRestDataServiceReconciler) Reconcile(ctx context.Context, req ctr r.Log.Info("Resource deleted") return requeueN, nil } - return requeueN, err + r.Log.Error(err, err.Error()) + return requeueY, err + } + + /* Initialize Status */ + if oracleRestDataService.Status.Status == "" { + oracleRestDataService.Status.Status = dbcommons.StatusPending + } + if oracleRestDataService.Status.ApxeUrl == "" { + oracleRestDataService.Status.ApxeUrl = dbcommons.ValueUnavailable + } + if oracleRestDataService.Status.DatabaseApiUrl == "" { + oracleRestDataService.Status.DatabaseApiUrl = dbcommons.ValueUnavailable + } + if oracleRestDataService.Status.DatabaseActionsUrl == "" { + oracleRestDataService.Status.DatabaseActionsUrl = dbcommons.ValueUnavailable } // Fetch Primary Database Reference @@ -115,7 +130,8 @@ func (r *OracleRestDataServiceReconciler) Reconcile(ctx context.Context, req ctr r.Log.Info("Resource not found", "DatabaseRef", oracleRestDataService.Spec.DatabaseRef) return requeueY, nil } - return requeueN, err + r.Log.Error(err, err.Error()) + return requeueY, err } // Manage OracleRestDataService Deletion @@ -208,21 +224,6 @@ func (r *OracleRestDataServiceReconciler) validate(m *dbapi.OracleRestDataServic eventReason := "Spec Error" var eventMsgs []string - /* Initialize Status */ - if m.Status.Status == "" { - m.Status.Status = dbcommons.StatusPending - } - if m.Status.ApxeUrl == "" { - m.Status.ApxeUrl = dbcommons.ValueUnavailable - } - if m.Status.DatabaseApiUrl == "" { - m.Status.DatabaseApiUrl = dbcommons.ValueUnavailable - } - if m.Status.DatabaseActionsUrl == "" { - m.Status.DatabaseActionsUrl = dbcommons.ValueUnavailable - } - - //First check image pull secrets if m.Spec.Image.PullSecrets != "" { secret := &corev1.Secret{} diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index a73cec36..1e8b851c 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -114,9 +114,37 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct r.Log.Info("Resource not found") return requeueN, nil } + r.Log.Error(err, err.Error()) return requeueY, err } + /* Initialize Status */ + if singleInstanceDatabase.Status.Status == "" { + singleInstanceDatabase.Status.Status = dbcommons.StatusPending + } + if singleInstanceDatabase.Status.Edition == "" { + if singleInstanceDatabase.Spec.Edition != "" { + singleInstanceDatabase.Status.Edition = strings.Title(singleInstanceDatabase.Spec.Edition) + } else { + singleInstanceDatabase.Status.Edition = dbcommons.ValueUnavailable + } + } + if singleInstanceDatabase.Status.Role == "" { + singleInstanceDatabase.Status.Role = dbcommons.ValueUnavailable + } + if singleInstanceDatabase.Status.ConnectString == "" { + singleInstanceDatabase.Status.ConnectString = dbcommons.ValueUnavailable + } + if singleInstanceDatabase.Status.PdbConnectString == "" { + singleInstanceDatabase.Status.PdbConnectString = dbcommons.ValueUnavailable + } + if singleInstanceDatabase.Status.OemExpressUrl == "" { + singleInstanceDatabase.Status.OemExpressUrl = dbcommons.ValueUnavailable + } + if singleInstanceDatabase.Status.ReleaseUpdate == "" { + singleInstanceDatabase.Status.ReleaseUpdate = dbcommons.ValueUnavailable + } + // Manage SingleInstanceDatabase Deletion result, err = r.manageSingleInstanceDatabaseDeletion(req, ctx, singleInstanceDatabase) if result.Requeue { @@ -303,33 +331,6 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab r.Log.Info("Entering reconcile validation") - /* Initialize Status */ - if m.Status.Status == "" { - m.Status.Status = dbcommons.StatusPending - } - if m.Status.Edition == "" { - if m.Spec.Edition != "" { - m.Status.Edition = strings.Title(m.Spec.Edition) - } else { - m.Status.Edition = dbcommons.ValueUnavailable - } - } - if m.Status.Role == "" { - m.Status.Role = dbcommons.ValueUnavailable - } - if m.Status.ConnectString == "" { - m.Status.ConnectString = dbcommons.ValueUnavailable - } - if m.Status.PdbConnectString == "" { - m.Status.PdbConnectString = dbcommons.ValueUnavailable - } - if m.Status.OemExpressUrl == "" { - m.Status.OemExpressUrl = dbcommons.ValueUnavailable - } - if m.Status.ReleaseUpdate == "" { - m.Status.ReleaseUpdate = dbcommons.ValueUnavailable - } - //First check image pull secrets if m.Spec.Image.PullSecrets != "" { secret := &corev1.Secret{} From 3c83c9f277ac2f281c006d69f51e069a8a17602d Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Tue, 3 May 2022 23:42:02 +0530 Subject: [PATCH 337/628] Change order --- .../database/oraclerestdataservice_controller.go | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 1092edc3..05fe6e02 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -105,6 +105,9 @@ func (r *OracleRestDataServiceReconciler) Reconcile(ctx context.Context, req ctr return requeueY, err } + // Always refresh status before a reconcile + defer r.Status().Update(ctx, oracleRestDataService) + /* Initialize Status */ if oracleRestDataService.Status.Status == "" { oracleRestDataService.Status.Status = dbcommons.StatusPending @@ -134,6 +137,9 @@ func (r *OracleRestDataServiceReconciler) Reconcile(ctx context.Context, req ctr return requeueY, err } + // Always refresh status before a reconcile + defer r.Status().Update(ctx, singleInstanceDatabase) + // Manage OracleRestDataService Deletion result := r.manageOracleRestDataServiceDeletion(req, ctx, oracleRestDataService, singleInstanceDatabase) if result.Requeue { @@ -141,10 +147,6 @@ func (r *OracleRestDataServiceReconciler) Reconcile(ctx context.Context, req ctr return result, nil } - // Always refresh status before a reconcile - defer r.Status().Update(ctx, oracleRestDataService) - defer r.Status().Update(ctx, singleInstanceDatabase) - // First validate result, err = r.validate(oracleRestDataService, singleInstanceDatabase, ctx) if result.Requeue { From 931188ffa535d7a2de66a9cdaf6895224dfee319 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Tue, 3 May 2022 23:54:14 +0530 Subject: [PATCH 338/628] Fix SQL --- commons/database/constants.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/commons/database/constants.go b/commons/database/constants.go index 5dc54d25..9b2917c5 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -217,7 +217,7 @@ const GetStandardEditionFileCMD string = "if [ -f ${ORACLE_BASE}/oradata/dbconfi const GetPdbsSQL string = "select name from v\\$pdbs where name not like 'PDB\\$SEED' and open_mode like 'READ WRITE';" const OpenPDBSeed = "alter pluggable database pdb\\$seed close;" + - "\nalter pluggable database pdb\\$seed open;" + "\nalter pluggable database pdb\\$seed open read only;" const SetAdminUsersSQL string = "CREATE USER C##DBAPI_CDB_ADMIN IDENTIFIED BY \\\"%[1]s\\\" ACCOUNT UNLOCK CONTAINER=ALL;" + "\nalter user C##DBAPI_CDB_ADMIN identified by \\\"%[1]s\\\" account unlock;" + From bab2df965219881c42b304b83b0b238a6e24943f Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Wed, 4 May 2022 00:02:48 +0530 Subject: [PATCH 339/628] Fix status --- controllers/database/oraclerestdataservice_controller.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 05fe6e02..62ded137 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -852,7 +852,7 @@ func (r *OracleRestDataServiceReconciler) createPods(m *dbapi.OracleRestDataServ replicasReq := m.Spec.Replicas if replicasFound == 0 { - m.Status.Status = dbcommons.StatusNotReady + m.Status.Status = dbcommons.StatusPending } if replicasFound == replicasReq { From 5253008e4259b5a93b97fdb044af6e03b93c26be Mon Sep 17 00:00:00 2001 From: Mohammed Yunus Date: Wed, 4 May 2022 05:00:58 +0000 Subject: [PATCH 340/628] Update README.md --- docs/sidb/README.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index bb905e94..33d44306 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -470,9 +470,9 @@ Create a file called "/tmp/table.sql" with the following contents. Execute the follwing API to run the above script. ```sh - curl -s -k -X "POST" "https://10.0.25.54:8443/ords/<.spec.restEnableSchemas[].pdb>/<.spec.restEnableSchemas[].urlMapping>/_/sql" \ + curl -s -k -X "POST" "https://10.0.25.54:8443/ords/<.spec.restEnableSchemas[].pdbName>/<.spec.restEnableSchemas[].urlMapping>/_/sql" \ -H "Content-Type: application/sql" \ - -u '<.spec.restEnableSchemas[].schema>:<.spec.ordsPassword>' \ + -u '<.spec.restEnableSchemas[].schemaName>:<.spec.ordsPassword>' \ -d @/tmp/table.sql ``` @@ -481,13 +481,13 @@ Execute the follwing API to run the above script. Fetch all entries from 'DEPT' table by calling the following API ```sh - curl -s -k -X "POST" "https://10.0.25.54:8443/ords/<.spec.restEnableSchemas[].pdb>/<.spec.restEnableSchemas[].urlMapping>/_/sql" \ + curl -s -k -X "POST" "https://10.0.25.54:8443/ords/<.spec.restEnableSchemas[].pdbName>/<.spec.restEnableSchemas[].urlMapping>/_/sql" \ -H "Content-Type: application/sql" \ - -u '<.spec.restEnableSchemas[].schema>:<.spec.ordsPassword>' \ + -u '<.spec.restEnableSchemas[].schemaName>:<.spec.ordsPassword>' \ -d $'select * from dept;' | python -m json.tool ``` -**NOTE:** `.spec.restEnableSchema[].urlMapping` is optional and is defaulted to `.spec.restEnableSchema[].schema` +**NOTE:** `.spec.restEnableSchema[].urlMapping` is optional and is defaulted to `.spec.restEnableSchemas[].schemaName` #### Data Pump @@ -501,8 +501,8 @@ Database Actions is a web-based interface that uses Oracle REST Data Services to * To use Database Actions, one must sign in as a database user whose schema has been REST-enabled. * This can be done by specifying appropriate values for the `.spec.restEnableSchemas` attributes details in the sample yaml [config/samples/sidb/oraclerestdataservice.yaml](config/samples/sidb/oraclerestdataservice.yaml) which are needed for authorising Database Actions. -* Schema will be created (if not exists) with username as `.spec.restEnableSchema[].schema` and password as `.spec.ordsPassword.`. -* UrlMapping `.spec.restEnableSchema[].urlMapping` is optional and is defaulted to `.spec.restEnableSchema[].schema`. +* Schema will be created (if not exists) with username as `.spec.restEnableSchemas[].schemaName` and password as `.spec.ordsPassword.`. +* UrlMapping `.spec.restEnableSchema[].urlMapping` is optional and is defaulted to `.spec.restEnableSchemas[].schemaName`. Database Actions can be accessed via browser using `.status.databaseActionsUrl` in the following command @@ -514,11 +514,11 @@ $ kubectl get oraclerestdataservice/ords-sample --template={{.status.databaseAct Sign in to Database Actions using * First Page: \ -PDB Name: `.spec.restEnableSchema[].pdb` \ -Username: `.spec.restEnableSchema[].urlMapping` +PDB Name: `.spec.restEnableSchemas[].pdbName` \ +Username: `.spec.restEnableSchemas[].urlMapping` * Second Page: \ -Username: `.spec.restEnableSchema[].schema` \ +Username: `.spec.restEnableSchemas[].schemaName` \ Password: `.spec.ordsPassword` ![database-actions-home](/images/sidb/database-actions-home.png) From 6e284746494714434b1d9534126d3051e6ed4033 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Wed, 4 May 2022 12:22:22 +0530 Subject: [PATCH 341/628] Fix ORDS deletion --- controllers/database/oraclerestdataservice_controller.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 62ded137..e2192010 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -951,7 +951,7 @@ func (r *OracleRestDataServiceReconciler) manageOracleRestDataServiceDeletion(re return requeueY } } - return requeueN + return requeueY } // Add finalizer for this CR From 03f63303c83a46900ea8ae71cb43d87c97b644d7 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Wed, 4 May 2022 12:50:35 +0530 Subject: [PATCH 342/628] Fix OrdsReference --- .../database/oraclerestdataservice_controller.go | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index e2192010..472b8e57 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -95,6 +95,9 @@ func (r *OracleRestDataServiceReconciler) Reconcile(ctx context.Context, req ctr _ = log.FromContext(ctx) oracleRestDataService := &dbapi.OracleRestDataService{} + // Always refresh status before a reconcile + defer r.Status().Update(ctx, oracleRestDataService) + err := r.Get(ctx, types.NamespacedName{Namespace: req.Namespace, Name: req.Name}, oracleRestDataService) if err != nil { if apierrors.IsNotFound(err) { @@ -105,9 +108,6 @@ func (r *OracleRestDataServiceReconciler) Reconcile(ctx context.Context, req ctr return requeueY, err } - // Always refresh status before a reconcile - defer r.Status().Update(ctx, oracleRestDataService) - /* Initialize Status */ if oracleRestDataService.Status.Status == "" { oracleRestDataService.Status.Status = dbcommons.StatusPending @@ -124,6 +124,9 @@ func (r *OracleRestDataServiceReconciler) Reconcile(ctx context.Context, req ctr // Fetch Primary Database Reference singleInstanceDatabase := &dbapi.SingleInstanceDatabase{} + // Always refresh status before a reconcile + defer r.Status().Update(ctx, singleInstanceDatabase) + err = r.Get(ctx, types.NamespacedName{Namespace: req.Namespace, Name: oracleRestDataService.Spec.DatabaseRef}, singleInstanceDatabase) if err != nil { if apierrors.IsNotFound(err) { @@ -137,9 +140,6 @@ func (r *OracleRestDataServiceReconciler) Reconcile(ctx context.Context, req ctr return requeueY, err } - // Always refresh status before a reconcile - defer r.Status().Update(ctx, singleInstanceDatabase) - // Manage OracleRestDataService Deletion result := r.manageOracleRestDataServiceDeletion(req, ctx, oracleRestDataService, singleInstanceDatabase) if result.Requeue { @@ -933,9 +933,10 @@ func (r *OracleRestDataServiceReconciler) manageOracleRestDataServiceDeletion(re n.Status.OrdsReference = "" // Make sure n.Status.OrdsInstalled is set to false or else it blocks .spec.databaseRef deletion for i := 0; i < 10; i++ { + log.Info("Clearing the OrdsReference from DB", "name", n.Name) err := r.Status().Update(ctx, n) if err != nil { - log.Info(err.Error() + "\n updating n.Status.OrdsInstalled = false") + log.Error(err, err.Error()) time.Sleep(1 * time.Second) continue } From 525cad94b27c4a67beadf2462fc01ea7057cdbc7 Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Wed, 4 May 2022 12:52:48 +0530 Subject: [PATCH 343/628] Included changes suggested by the doc review team Signed-off-by: abhisbyk --- docs/sidb/README.md | 252 ++++++++++++++++++++++++++------------------ 1 file changed, 151 insertions(+), 101 deletions(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index bb905e94..b40aa2ad 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -1,6 +1,6 @@ # Managing Oracle Single Instance Databases with Oracle Database Operator for Kubernetes -Oracle Database Operator for Kubernetes (a.k.a. OraOperator) includes the Single Instance Database Controller that performs provisioning, cloning, and patching of Oracle Single Instance Databases on Kubernetes and enables configuring them for Oracle REST Data Services with Oracle APEX development platform. The following sections explain the setup and functionality of the operator +Oracle Database Operator for Kubernetes (`OraOperator`) includes the Single Instance Database Controller, which enables provisioning, cloning, and patching of Oracle Single Instance Databases on Kubernetes. It also enables configuring the database for Oracle REST Data Services with Oracle APEX development platform. The following sections explain the setup and functionality of the operator * [Prerequisites](#prerequisites) * [Kind SingleInstanceDatabase Resource](#kind-singleinstancedatabase-resource) @@ -17,23 +17,24 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI ## Kind SingleInstanceDatabase Resource - The Oracle Database Operator creates the SingleInstanceDatabase kind as a custom resource that enables Oracle Database to be managed as a native Kubernetes object. +The Oracle Database Operator creates the `SingleInstanceDatabase` kind as a custom resource. Doing this enables Oracle Database to be managed as a native Kubernetes object. ### SingleInstanceDatabase template YAML -The template `.yaml` file for Single Instance Database including all the configurable options is available at: -[config/samples/sidb/singleinstancedatabase.yaml](./../../config/samples/sidb/singleinstancedatabase.yaml) +The template `.yaml` file for Single Instance Database (Enterprise and Standard Editions), including all the configurable options, is available at: +**[config/samples/sidb/singleinstancedatabase.yaml](./../../config/samples/sidb/singleinstancedatabase.yaml)** **Note:** -The `adminPassword` field of the above `singleinstancedatabase.yaml` file refers to a secret for SYS, SYSTEM and PDBADMIN users of the Single Instance Database. It is required while provisioning a new database or cloning an existing one. +The `adminPassword` field in the above `singleinstancedatabase.yaml` file refers to a secret for the SYS, SYSTEM and PDBADMIN users of the Single Instance Database. This secret is required when you provision a new database, or when you clone an existing database. -This secret can be created using the following sample command: +Create this secret using the following command as an example: kubectl create secret generic admin-secret --from-literal=oracle_pwd="ChangeOnInstall_1" -The above command will create a secret having name as `admin-secret` with key `oracle_pwd` mapped to the actual password specified in the command. +This command creates a secret named `admin-secret`, with the key `oracle_pwd` mapped to the actual password specified in the command. ### List Databases +To list databases, use the following command as an example, where the database names are `sidb-sample` and `sidb-sample-clone`, which are the names we will use as database names in command examples: ```sh $ kubectl get singleinstancedatabases -o name @@ -44,7 +45,8 @@ $ kubectl get singleinstancedatabases -o name ``` ### Quick Status - +To obtain a quick database status, use the following command as an example: + ```sh $ kubectl get singleinstancedatabase sidb-sample @@ -53,6 +55,7 @@ sidb-sample Enterprise Healthy 19.3.0.0.0 10.0.25.54:1521/ORCLCDB ``` ### Detailed Status +To obtain a detailed database status, use the following command as an example: ```sh $ kubectl describe singleinstancedatabase sidb-sample-clone @@ -113,47 +116,47 @@ $ kubectl describe singleinstancedatabase sidb-sample-clone ## Provision New Database - A sample **[singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml)** is provided to quickly and easily provision a single instance database in Kubernetes +You can easily provision a new database instance on the Kubernetes cluster by using [singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml). - Firstly, sign into [Oracle Container Registry](https://container-registry.oracle.com/ords/f?p=113:4:7154182141811:::4:P4_REPOSITORY,AI_REPOSITORY,AI_REPOSITORY_NAME,P4_REPOSITORY_NAME,P4_EULA_ID,P4_BUSINESS_AREA_ID:9,9,Oracle%20Database%20Enterprise%20Edition,Oracle%20Database%20Enterprise%20Edition,1,0&cs=3Y_90hkCQLfJzrvTLiEipIGgWGUytfrtAPuHFocuWd0NDSacbBPlamohfLuiJA-bAsVL6Z_yKEMsTbb52bm6IRA) and accept the license agreement for the Database image, ignore if you have accepted already. +1. Sign into [Oracle Container Registry](https://container-registry.oracle.com/ords/f?p=113:4:7154182141811:::4:P4_REPOSITORY,AI_REPOSITORY,AI_REPOSITORY_NAME,P4_REPOSITORY_NAME,P4_EULA_ID,P4_BUSINESS_AREA_ID:9,9,Oracle%20Database%20Enterprise%20Edition,Oracle%20Database%20Enterprise%20Edition,1,0&cs=3Y_90hkCQLfJzrvTLiEipIGgWGUytfrtAPuHFocuWd0NDSacbBPlamohfLuiJA-bAsVL6Z_yKEMsTbb52bm6IRA) and accept the license agreement for the Database image, ignore if you have accepted already. - Create an image pull secret for the Oracle Container Registry, ignore if you have created already: +2. If you have not already done so, create an image pull secret for the Oracle Container Registry: - ```sh - $ kubectl create secret docker-registry oracle-container-registry-secret --docker-server=container-registry.oracle.com --docker-username='' --docker-password='' --docker-email='' - - secret/oracle-container-registry-secret created - ``` - This secret can also be created using the docker config file after a successful docker login - ```sh - $ docker login container-registry.oracle.com - $ kubectl create secret generic oracle-container-registry-secret --from-file=.dockerconfigjson=.docker/config.json --type=kubernetes.io/dockerconfigjson - ``` - Now, easily provision a new database instance on the cluster by using the following command. + ```sh + $ kubectl create secret docker-registry oracle-container-registry-secret --docker-server=container-registry.oracle.com --docker-username='' --docker-password='' --docker-email='' + + secret/oracle-container-registry-secret created + ``` + This secret can also be created using the docker config file after a successful docker login + ```sh + $ docker login container-registry.oracle.com + $ kubectl create secret generic oracle-container-registry-secret --from-file=.dockerconfigjson=.docker/config.json --type=kubernetes.io/dockerconfigjson + ``` +3. Provision a new database instance on the cluster by using the following command: - ```sh - $ kubectl apply -f singleinstancedatabase_create.yaml + ```sh + $ kubectl apply -f singleinstancedatabase_create.yaml singleinstancedatabase.database.oracle.com/sidb-sample created - ``` + ``` - **NOTE:** - - For the ease of use, the storage class **oci-bv** is specified in the [singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml). This storage class facilitates dynamic provisioning of the block volumes on Oracle OKE for the persistent storage of the database. For other cloud providers, an appropriate dynamic provisioning storage class must be provided. - - Oracle Database Enterprise and Standard editions are supported from version 19.3.0 onwards. +**NOTE:** +- For ease of use, the storage class **oci-bv** is specified in the [singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml). This storage class facilitates dynamic provisioning of the OCI block volumes on the Oracle OKE for persistent storage of the database. For other cloud providers, you can similarly use their dynamic provisioning storage class. +- Supports Oracle Database Enterprise Edition (19.3.0), and later releases. ### Provisioning a new XE database -To provision a XE database, use the sample **[config/samples/sidb/singleinstancedatabase_express.yaml](../../config/samples/sidb/singleinstancedatabase_express.yaml)** file. Use the following command: +To provision new Oracle Database Express Edition (XE) database, use the sample **[config/samples/sidb/singleinstancedatabase_express.yaml](../../config/samples/sidb/singleinstancedatabase_express.yaml)** file. For example: kubectl apply -f singleinstancedatabase_express.yaml -It pulls the XE image uploaded on the [Oracle Container Registry](https://container-registry.oracle.com/ords/f?p=113:4:7460390069267:::4:P4_REPOSITORY,AI_REPOSITORY,AI_REPOSITORY_NAME,P4_REPOSITORY_NAME,P4_EULA_ID,P4_BUSINESS_AREA_ID:803,803,Oracle%20Database%20Express%20Edition,Oracle%20Database%20Express%20Edition,1,0&cs=3-UN6D9nAfyqxcYnrks18OAmfFcri96NZojBQALxMdakix8wgYRBxhD8rpTFd2ak1FAtfOVFexbuOM2opsjxT9w). +This command pulls the XE image uploaded on the [Oracle Container Registry](https://container-registry.oracle.com/ords/f?p=113:4:7460390069267:::4:P4_REPOSITORY,AI_REPOSITORY,AI_REPOSITORY_NAME,P4_REPOSITORY_NAME,P4_EULA_ID,P4_BUSINESS_AREA_ID:803,803,Oracle%20Database%20Express%20Edition,Oracle%20Database%20Express%20Edition,1,0&cs=3-UN6D9nAfyqxcYnrks18OAmfFcri96NZojBQALxMdakix8wgYRBxhD8rpTFd2ak1FAtfOVFexbuOM2opsjxT9w). **NOTE:** -Oracle Database XE edition is supported version 21.3.0 onwards. +Oracle Database XE edition is supported Release 21c (21.3.0) and later releases. ### Provision a pre-built database -Provision a pre-built database using the sample yaml **[config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml](../../config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml)** with the following command: +Provision a new pre-built database instance by specifying appropriate values for the attributes in the example **[config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml](../../config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml)** file, and running the following command: ```sh $ kubectl apply -f singleinstancedatabase_prebuiltdb.yaml @@ -161,13 +164,13 @@ $ kubectl apply -f singleinstancedatabase_prebuiltdb.yaml singleinstancedatabase.database.oracle.com/prebuiltdb-sample created ``` -This pre-built database image includes the data files of the database inside the image itself. This enables for quick startup of the database in the container. This pre-built database would be very useful in CI/CD scenarios, where database would be used for conducting tests, experiments requiring a new database instance for every run. +This pre-built image includes the data files of the database inside the image itself. As a result, the database startup time of the container is reduced, down to a couple of seconds. The pre-built database image can be very useful in contiguous integration/continuous delivery (CI/CD) scenarios, in which databases are used for conducting tests or experiments, and the workflow is simple. -Please follow the [link](https://github.com/oracle/docker-images/blob/main/OracleDatabase/SingleInstance/extensions/prebuiltdb/README.md) to create the pre-built database image for the Enterprise/Standard edition. +To create the pre-built database image for the Enterprise/Standard edition using a pre-built image, please follow these instructions: [Pre-built Database (prebuiltdb) Extension](https://github.com/oracle/docker-images/blob/main/OracleDatabase/SingleInstance/extensions/prebuiltdb/README.md). ### Creation Status -Creating a new database instance takes a while. When the 'status' column returns the response "Healthy", the Database is open for connections. +Creating a new database instance takes a while. When the `status` column returns the response `Healthy`, the Database is open for connections. ```sh $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.status}" @@ -177,7 +180,7 @@ $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.status}" ### Connection Information -External and internal (running in Kubernetes pods) clients can connect to the database using `.status.connectString` in the following command: +External and internal (running in Kubernetes pods) clients can connect to the database by using `.status.connectString`. For example: ```sh $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.connectString}" @@ -185,7 +188,7 @@ $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.connectSt 10.0.25.54:1521/ORCL ``` -The Oracle Database inside the container also has Oracle Enterprise Manager Express configured. To access OEM Express, start the browser and paste the following URL: +The Oracle Database inside the container also has Oracle Enterprise Manager Express (OEM Express) configured. To access OEM Express, start the browser, and paste in a URL similar to the following example: ```sh $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.oemExpressUrl}" @@ -195,8 +198,18 @@ $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.oemExpres ### Update Database Config -The following database parameters can be updated post database creation: **flashBack, archiveLog, forceLog**. Change their attribute values and apply using -kubectl **apply** or **edit/patch** commands . Enable archiveLog mode before turning ON flashBack, and turn OFF flashBack before disabling the archiveLog. +The following database parameters can be updated after the database is created: + +- `flashBack` +- `archiveLog` +- `forceLog` + +To change these parameters, change their attribute values, and apply the change by using the +`kubectl` `apply` or `edit`/`patch` commands. + +**Caution**: Enable `archiveLog` mode before setting `flashback` to `ON`, and set `flashback` to `OFF` before disabling `archiveLog` mode. + +For example: ```sh $ kubectl patch singleinstancedatabase sidb-sample --type merge -p '{"spec":{"forceLog": true}}' @@ -206,7 +219,7 @@ $ kubectl patch singleinstancedatabase sidb-sample --type merge -p '{"spec":{"fo #### Database Config Status -Check the Database Config Status using the following command +Check the Database Config Status by using the following command: ```sh $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath=[{.status.archiveLog}, {.status.flashBack}, {.status.forceLog}]" @@ -216,22 +229,35 @@ $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath=[{.status.archiveL ### Update Initialization Parameters -The following database initialization parameters can be updated post database creation: `sgaTarget, pgaAggregateTarget, cpuCount, processes`. Change their attribute values and apply using kubectl **apply** or **edit/patch** commands. +The following database initialization parameters can be updated after the database is created: + +- sgaTarget +- pgaAggregateTarget +- cpuCount +- processes. Change their attribute values and apply using kubectl **apply** or **edit/patch** commands. **NOTE:** -`sgaTarget` should be in range [sga_min_size, sga_max_size], else initialization parameter `sga_target` would not be updated to specified `sgaTarget`. +The value for the initialization parameter `sgaTarget` that you provide should be within the range set by [sga_min_size, sga_max_size]. If the value you provide is not in that range, then `sga_target` is not updated to the value you specify for `sgaTarget`. ### Multiple Replicas -Specifying a replicas count greater than one creates multiple replica pods for the database. The database is open and mounted by one of the replica pod. Other replica pods will have instance started but not mounted and serve to provide a fast fail over in case the active pod dies. Update the replica attribute in the .yaml and apply using the `kubectl apply` or `edit/patch` commands. +In multiple replicas mode, more than one pod is created for the database. The database is open and mounted by one of the replica pods. Other replica pods have instances started but not mounted, and serve to provide a quick cold fail-over in case the active pod goes down. To enable multiple replicas, Update the replica attribute in the `.yaml`, and apply by using the `kubectl apply` or `edit/patch` commands. **Note:** -- This functionality requires the [k8s extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/k8s) extended images. The database image from the container-registry.oracle.com includes the K8s extension. -- The Oracle database express edition (XE) does not include the [k8s extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/k8s). Hence, it does not support multiple replicas. +- This functionality requires the [k8s extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/k8s) extended images. The database image from the container registry `container-registry.oracle.com` includes the K8s extension. +- Because Oracle Database Express Edition (XE) does not support [k8s extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/k8s), it does not support multiple replicas. ### Patch Attributes -The following attributes cannot be patched post the Single Instance Database instance creation: sid, edition, charset, pdbName, cloneFrom. +The following attributes cannot be patched after creating the Single Instance Database instance: + +- `sid` +- `edition` +- `charset` +- `pdbName` +- `cloneFrom` + +If you attempt to patch one of these attributes, then you receive an error similar to the following: ```sh $ kubectl --type=merge -p '{"spec":{"sid":"ORCL1"}}' patch singleinstancedatabase sidb-sample @@ -241,10 +267,17 @@ $ kubectl --type=merge -p '{"spec":{"sid":"ORCL1"}}' patch singleinstancedatabas #### Patch Service -Service can be patched post SingleInstanceDatabase instance Creation. This will **replace the Service with a new type**. +You can patch the Service can be patched after creating the Single Instance Database instance. + +**Caution**: Patching the service **replaces the existing Service with a new type**. + +Options: + * NodePort - '{"spec":{"loadBalancer": false}}' * LoadBalancer - '{"spec":{"loadBalancer": true }}' +For example: + ```sh $ kubectl --type=merge -p '{"spec":{"loadBalancer": false}}' patch singleinstancedatabase sidb-sample @@ -253,15 +286,13 @@ $ kubectl --type=merge -p '{"spec":{"loadBalancer": false}}' patch singleinstanc ## Clone Existing Database -Create copies of your existing database using the cloning functionality. A cloned database is an exact, block-for-block copy of the source database. This is much faster than creating a fresh database and copying over the data. Specify the source database reference in the `cloneFrom` attribute - -$ grep 'cloneFrom:' singleinstancedatabase_clone.yaml +To create copies of your existing database quickly, you can use the cloning functionality. A cloned database is an exact, block-for-block copy of the source database. Cloning is much faster than creating a fresh database and copying over the data. - cloneFrom: "sidb-sample" +To clone an existing database, specify the source database reference as the value for the `cloneFrom` attribute in the sample [singleinstancedatabase_clone.yaml](../../config/samples/sidb/singleinstancedatabase_clone.yaml) file. -**The source database must have archiveLog mode set to true.** +**Note**: To clone a database, The source database must have archiveLog mode set to true. -Quicly clone using the sample **[singleinstancedatabase_clone.yaml](../../config/samples/sidb/singleinstancedatabase_clone.yaml)** file by executing the following command +For example: ```sh @@ -270,17 +301,17 @@ $ kubectl apply -f singleinstancedatabase_clone.yaml singleinstancedatabase.database.oracle.com/sidb-sample-clone created ``` -**Note:** The database image specified in the yaml for cloning can be at a different patch level but should be of the same major release version +**Note:** The clone database can specify a database image that is different from the source database. In such cases, cloning is supported only between databases of the same major release. ## Patch/Rollback Database -Databases running in your cluster and managed by this operator can be patched or rolled back between release updates of the same major release. To patch databases, specify an image of the higher release update, and to roll back, specify an image of the lower release update. +Databases running in your cluster and managed by the Oracle Database operator can be patched or rolled back between release updates of the same major release. To patch databases, specify an image of the higher release update. To roll back databases, specify an image of the lower release update. -Patched Oracle Docker images can be built using this [patching extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/patching). +Patched Oracle Docker images can be built by using this [patching extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/patching). ### Patch existing Database -Edit and apply the sample **[singleinstancedatabase_patch.yaml](../../config/samples/sidb/singleinstancedatabase_patch.yaml)** file of the database resource/object by specifying a new release update for image attributes or run the following command. +To patch an existing database, edit and apply the [singleinstancedatabase_patch.yaml](../../config/samples/sidb/singleinstancedatabase_patch.yaml) file of the database resource/object either by specifying a new release update for image attributes, or by running the following command: ```sh kubectl --type=merge -p '{"spec":{"image":{"pullFrom":"patched-image:tag","pullSecrets":"pull-secret"}}}' patch singleinstancedatabase sidb-sample @@ -289,11 +320,11 @@ singleinstancedatabase.database.oracle.com/sidb-sample patched ``` -The database pods will be restarted with the new release update image. For minimum downtime, ensure that you have multiple replicas of the database pods running. +After patching is complete, the database pods are restarted with the new release update image. For minimum downtime, ensure that you have multiple replicas of the database pods running before you start the patch operation. ### Clone and Patch Database - Clone your source database using the method of [cloning existing database](#clone-existing-database) and specify a new release image for the cloned database. Use this method to ensure there are no patching related issues impacting your database performance/functionality. +To clone and patch the database at the same time, clone your source database by using the [cloning existing database](#clone-existing-database) method, and specify a new release image for the cloned database. Use this method to ensure there are no patching related issues impacting your database performance or functionality. ### Datapatch status @@ -312,17 +343,25 @@ $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.releaseUp ## Kind OracleRestDataService -The Oracle Database Operator creates the OracleRestDataService (ORDS) kind as a custom resource that configures Oracle REST Data Services and enables RESTful API access to the Oracle Database in K8s. +The Oracle Database Operator creates the `OracleRestDataService` (ORDS) kind as a custom resource. Creating ORDS as a custom resource enables the RESTful API access to the Oracle Database in K8s. ### OracleRestDataService template YAML -The template `.yaml` file for Oracle Rest Data Services (OracleRestDataService kind) is available at [config/samples/sidb/oraclerestdataservice.yaml](config/samples/sidb/oraclerestdataservice.yaml) +The template `.yaml` file for Oracle Rest Data Services (`OracleRestDataService` kind) is available at [config/samples/sidb/oraclerestdataservice.yaml](config/samples/sidb/oraclerestdataservice.yaml). + +**Note:** +- For **quick provisioning** of the ORDS, apply the [config/samples/sidb/oraclerestdataservice_create.yaml](../../config/samples/sidb/oraclerestdataservice_create.yaml) file, using the following command: + + kubectl apply -f oraclerestdataservice_create.yaml -- The `adminPassword` field of the above `oraclerestdataservice.yaml` file contains secrets for authenticating Single Instance Database and the `ordsPassword` field contains secret for ORDS_PUBLIC_USER user with roles `SQL Administrator, System Administrator, SQL Developer, oracle.dbtools.autorest.any.schema` respectively. -- To build the ORDS image, please follow the these [GitHub](https://github.com/oracle/docker-images/tree/main/OracleRestDataServices#building-oracle-rest-data-services-install-images) instructions. -- By default, the ORDS uses self-signed certificates. To use certificates from the Certificate Authority, the ORDS image needs to be rebuilt after specifying the values of `ssl.cert` and `ssl.cert.key` in the [standalone.properties](https://github.com/oracle/docker-images/blob/main/OracleRestDataServices/dockerfiles/standalone.properties.tmpl) file. This newly built ORDS image should be used in the [config/samples/sidb/oraclerestdataservice.yaml](config/samples/sidb/oraclerestdataservice.yaml) file. + +Note the following: +- The `adminPassword` and `ordsPassword` fields in the `oraclerestdataservice.yaml` file contains secrets for authenticating the Single Instance Database and the ORDS user with the following roles: `SQL Administrator, System Administrator, SQL Developer, oracle.dbtools.autorest.any.schema`. +- To build the ORDS image, use the following instructions: [Building Oracle REST Data Services Install Images](https://github.com/oracle/docker-images/tree/main/OracleRestDataServices#building-oracle-rest-data-services-install-images). +- By default, ORDS uses self-signed certificates. To use certificates from the Certificate Authority, the ORDS image needs to be rebuilt after specifying the values of `ssl.cert` and `ssl.cert.key` in the [standalone.properties](https://github.com/oracle/docker-images/blob/main/OracleRestDataServices/dockerfiles/standalone.properties.tmpl) file. After you rebuild the ORDS image, use the rebuilt image in the [config/samples/sidb/oraclerestdataservice.yaml](config/samples/sidb/oraclerestdataservice.yaml) file. ### List OracleRestDataServices +To list the ORDS service, use the following command: ```sh $ kubectl get oraclerestdataservice -o name @@ -332,6 +371,7 @@ $ kubectl get oraclerestdataservice -o name ``` ### Quick Status +To obtain a quick status check of the ORDS service, use the following command: ```sh $ kubectl get oraclerestdataservice ords-sample @@ -342,6 +382,7 @@ ords-sample Healthy sidb-sample https://10.0.25.54:8443/ords/ORCLPDB1 ``` ### Detailed Status +To obtain a detaile status check of the ORDS service, use the following command: ```sh $ kubectl describe oraclerestdataservice ords-sample @@ -376,27 +417,29 @@ $ kubectl describe oraclerestdataservice ords-sample ## REST Enable Database -For quick provisioning of the Oracle REST Data Services, apply the sample **[config/samples/sidb/oraclerestdataservice_create.yaml](../../config/samples/sidb/oraclerestdataservice_create.yaml)** file using the command below: +To provision a new ORDS instance, specify the appropriate values for the attributes in the the example [config/samples/sidb/oraclerestdataservice_create.yaml](../../config/samples/sidb/oraclerestdataservice_create.yaml) file, and run the following command: ```sh $ kubectl apply -f oraclerestdataservice_create.yaml oraclerestdataservice.database.oracle.com/ords-sample created ``` +After this command completes, ORDS is installed in the container database (CDB) of the Single Instance Database. ### Creation Status -Creating a new ORDS instance takes a while. ORDS is open for connections when the 'status' column returns "Healthy". +Creating a new ORDS instance takes a while. To check the status of the ORDS insteance, use the following command: ```sh $ kubectl get oraclerestdataservice/ords-sample --template={{.status.status}} Healthy ``` +ORDS is open for connections when the `status` column returns `Healthy`. ### REST Endpoints -Clients can access the REST Endpoints using .status.databaseApiUrl as shown in the following command. +Clients can access the REST Endpoints using `.status.databaseApiUrl` as shown in the following command. ```sh $ kubectl get oraclerestdataservice/ords-sample --template={{.status.databaseApiUrl}} @@ -404,23 +447,23 @@ $ kubectl get oraclerestdataservice/ords-sample --template={{.status.databaseApi https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/ ``` -All the REST Endpoints can be found at +All the REST Endpoints can be found in [_REST APIs for Oracle Database_](https://docs.oracle.com/en/database/oracle/oracle-database/21/dbrst/rest-endpoints.html). There are two basic approaches for authentication to the REST Endpoints. Certain APIs are specific about which authentication method they will accept. #### Database API - ORDS_PUBLIC_USER User with role "SQL Administrator", `.spec.ordsPassword` credentials are required to call Database API REST Endpoints. +To call certain REST endpoints, you must use the ORDS_PUBLIC_USER user with role `SQL Administrator`, and `.spec.ordsPassword` credentials. - This user has also given the additional roles `System Administrator, SQL Developer, oracle.dbtools.autorest.any.schema`. +The ORDS user also has the following additional roles: `System Administrator , SQL Developer , oracle.dbtools.autorest.any.schema`. - This user can now be used to authenticate - * Database APIs - * Any Protected AutoRest Enabled Object APIs - * Database Actions of any REST Enabled Schema +Use this ORDS user to authenticate the following: +* Database APIs +* Any Protected AutoRest Enabled Object APIs +* Database Actions of any REST Enabled Schema #### Database API examples - Some examples for the Database API usage are as follows: +Some examples for the Database API usage are as follows: * ##### Getting all Database components ```sh curl -s -k -X GET -u 'ORDS_PUBLIC_USER:<.spec.ordsPassword>' https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/database/components/ | python -m json.tool @@ -447,7 +490,9 @@ There are two basic approaches for authentication to the REST Endpoints. Certain The REST Enable SQL functionality is available to all the schemas specified in the `.spec.restEnableSchemas` atrribute of the sample yaml. Only these schemas will have access SQL Developer Web Console specified by the Database Actions URL. -The REST Enabled SQL functionality allows REST calls to send DML, DDL and scripts to any REST enabled schema by exposing the same SQL engine used in SQL Developer and SQLcl. +The REST Enabled SQL functionality enables REST calls to send DML, DDL and scripts to any REST enabled schema by exposing the same SQL engine used in SQL Developer and Oracle SQLcl (SQL Developer Command Line). + +For example: **Run a Script:** @@ -467,7 +512,7 @@ Create a file called "/tmp/table.sql" with the following contents. COMMIT; ``` -Execute the follwing API to run the above script. +Run the following API to run the script created in the previous example: ```sh curl -s -k -X "POST" "https://10.0.25.54:8443/ords/<.spec.restEnableSchemas[].pdb>/<.spec.restEnableSchemas[].urlMapping>/_/sql" \ @@ -489,22 +534,22 @@ Fetch all entries from 'DEPT' table by calling the following API **NOTE:** `.spec.restEnableSchema[].urlMapping` is optional and is defaulted to `.spec.restEnableSchema[].schema` -#### Data Pump +#### Oracle Data Pump -The Oracle REST Data Services (ORDS) database API allows user to create Data Pump export and import jobs via REST web service calls. +The Oracle REST Data Services (ORDS) database API enables you to create Oracle Data Pump export and import jobs by using REST web service calls. -REST APIs for Data Pump Jobs can be found at [https://docs.oracle.com/en/database/oracle/oracle-database/21/dbrst/op-database-datapump-jobs-post.html](https://docs.oracle.com/en/database/oracle/oracle-database/21/dbrst/op-database-datapump-jobs-post.html). +REST APIs for Oracle Data Pump Jobs can be found at [https://docs.oracle.com/en/database/oracle/oracle-database/21/dbrst/op-database-datapump-jobs-post.html](https://docs.oracle.com/en/database/oracle/oracle-database/21/dbrst/op-database-datapump-jobs-post.html). ### Database Actions Database Actions is a web-based interface that uses Oracle REST Data Services to provide development, data tools, administration and monitoring features for Oracle Database. -* To use Database Actions, one must sign in as a database user whose schema has been REST-enabled. -* This can be done by specifying appropriate values for the `.spec.restEnableSchemas` attributes details in the sample yaml [config/samples/sidb/oraclerestdataservice.yaml](config/samples/sidb/oraclerestdataservice.yaml) which are needed for authorising Database Actions. -* Schema will be created (if not exists) with username as `.spec.restEnableSchema[].schema` and password as `.spec.ordsPassword.`. +* To use Database Actions, you must sign in as a database user whose schema has been REST-enabled. +* To enable a schema for REST, you can specify appropriate values for the `.spec.restEnableSchemas` attributes details in the sample `yaml` [config/samples/sidb/oraclerestdataservice.yaml](config/samples/sidb/oraclerestdataservice.yaml), which are needed for authorizing Database Actions. +* Schema are created (if they exist) with the username as `.spec.restEnableSchema[].schema` and password as `.spec.ordsPassword.`. * UrlMapping `.spec.restEnableSchema[].urlMapping` is optional and is defaulted to `.spec.restEnableSchema[].schema`. -Database Actions can be accessed via browser using `.status.databaseActionsUrl` in the following command +Database Actions can be accessed with a browser by using `.status.databaseActionsUrl`. For example: ```sh $ kubectl get oraclerestdataservice/ords-sample --template={{.status.databaseActionsUrl}} @@ -512,7 +557,8 @@ $ kubectl get oraclerestdataservice/ords-sample --template={{.status.databaseAct https://10.0.25.54:8443/ords/sql-developer ``` -Sign in to Database Actions using +To access Database Actions, sign in by using the following code as a database user whose schema has been REST-enabled: + * First Page: \ PDB Name: `.spec.restEnableSchema[].pdb` \ Username: `.spec.restEnableSchema[].urlMapping` @@ -523,11 +569,11 @@ Password: `.spec.ordsPassword` ![database-actions-home](/images/sidb/database-actions-home.png) -More info on Database Actions can be found at +For more information about Database Actions, see: [Oracle Database Actions](https://docs.oracle.com/en/database/oracle/sql-developer-web/21.2/index.html). ### Application Express -Oracle Application Express (APEX) is a low-code development platform that enables developers to build scalable, secure enterprise apps, with world-class features, that can be deployed anywhere. +Oracle APEX (previously known as Oracle Application Express) is a low-code development platform that enables developers to build scalable, secure enterprise apps, with world-class features that can be deployed anywhere. Using APEX, developers can quickly develop and deploy compelling apps that solve real problems and provide immediate value. Developers won't need to be an expert in a vast array of technologies to deliver sophisticated solutions. Focus on solving the problem and let APEX take care of the rest. @@ -535,12 +581,14 @@ To access APEX, You need to configure APEX with the ORDS. The following section #### Configure APEX with ORDS -* For quick provisioning, apply the **[config/samples/sidb/oraclerestdataservice_apex.yaml](../../confi/samples/sidb/oraclerestdataservice_apex.yaml)** file. It configured Oracle REST Data Services with Oracle APEX. It uses the ORDS image hosted on the [Oracle Container Registry](https://container-registry.oracle.com/ords/f?p=113:4:113387942129427:::4:P4_REPOSITORY,AI_REPOSITORY,AI_REPOSITORY_NAME,P4_REPOSITORY_NAME,P4_EULA_ID,P4_BUSINESS_AREA_ID:1183,1183,Oracle%20REST%20Data%20Services%20(ORDS)%20with%20Application%20Express,Oracle%20REST%20Data%20Services%20(ORDS)%20with%20Application%20Express,1,0&cs=3_y-KlneZIxRRfXzerC_0ro7P1MGh-B_9lTEQObVTdoQCWkmsQ3lHpFs90Z8QFheteVQEzPvtUVHEQAqqXegYbA). +* For quick provisioning, apply the [config/samples/sidb/oraclerestdataservice_apex.yaml](../../confi/samples/sidb/oraclerestdataservice_apex.yaml) file. First, it creates `ords-secret`, `apex-secret`, and then provision the ORDS configured with Oracle APEX. It uses the ORDS image hosted on the [Oracle Container Registry](https://container-registry.oracle.com/ords/f?p=113:4:113387942129427:::4:P4_REPOSITORY,AI_REPOSITORY,AI_REPOSITORY_NAME,P4_REPOSITORY_NAME,P4_EULA_ID,P4_BUSINESS_AREA_ID:1183,1183,Oracle%20REST%20Data%20Services%20(ORDS)%20with%20Application%20Express,Oracle%20REST%20Data%20Services%20(ORDS)%20with%20Application%20Express,1,0&cs=3_y-KlneZIxRRfXzerC_0ro7P1MGh-B_9lTEQObVTdoQCWkmsQ3lHpFs90Z8QFheteVQEzPvtUVHEQAqqXegYbA). kubectl apply -f oraclerestdataservice_apex.yaml -* The secret apexPassword contains the common password for APEX_PUBLIC_USER, APEX_REST_PUBLIC_USER, APEX_LISTENER and Apex administrator (username: ADMIN) -* Status of ORDS turns to 'Updating' during apex configuration and turns 'Healthy' after successful configuration. You can also check status using below cmd +* To provision ORDS step by step for APEX, set `.spec.apexPassword.secretName` to a non-null string in [config/samples/sidb/oraclerestdataservice.yaml](../../config/samples/sidb/oraclerestdataservice.yaml) +* The APEX Password is used as a common password for APEX_PUBLIC_USER, APEX_REST_PUBLIC_USER, APEX_LISTENER and Apex administrator (username: ADMIN) mapped to secretKey +* The status of ORDS turns to `Updating` during APEX configuration, and changes to `Healthy` after successful configuration. You can also check status by using the following command: + ```sh $ kubectl get oraclerestdataservice ords-sample -o "jsonpath=[{.status.apexConfigured}]" @@ -548,9 +596,9 @@ To access APEX, You need to configure APEX with the ORDS. The following section [true] ``` -* If you configure APEX after ORDS is installed, ORDS pods will be deleted and recreated +* If you configure APEX after ORDS is installed, then ORDS pods will be deleted and recreated. -Application Express can be accessed via browser using `.status.apexUrl` in the following command .\ +Application Express can be accessed via browser using `.status.apexUrl` in the following command . ```sh $ kubectl get oraclerestdataservice/ords-sample --template={{.status.apexUrl}} @@ -558,23 +606,25 @@ $ kubectl get oraclerestdataservice/ords-sample --template={{.status.apexUrl}} https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/ ``` -Sign in to Administration servies using \ -workspace: `INTERNAL` \ -username: `ADMIN` \ +Sign in to Administration servies using +workspace: `INTERNAL` +username: `ADMIN` password: `.spec.apexPassword` ![application-express-admin-home](/images/sidb/application-express-admin-home.png) **NOTE:** -By default, the full development runtime environment is initialized in APEX. It can be changed manually to the runtime environment. For this, `apxdevrm.sql` script should be run after connecting to the primary database from the ORDS pod as the sys user with sysdba privilage. Please click the [link](https://docs.oracle.com/en/database/oracle/application-express/21.2/htmig/converting-between-runtime-and-full-development-environments.html#GUID-B0621B40-3441-44ED-9D86-29B058E26BE9) for detailed instructions. +By default, the full development environment is initialized in APEX. After deployment, you can change it manually to the runtime environment. To change environments, run the script `apxdevrm.sql` after connecting to the primary database from the ORDS pod as the `SYS` user with `SYSDBA` privilege. For detailed instructions, see: [Converting a Full Development Environment to a Runtime Environment](https://docs.oracle.com/en/database/oracle/application-express/21.2/htmig/converting-between-runtime-and-full-development-environments.html#GUID-B0621B40-3441-44ED-9D86-29B058E26BE9). ## Performing maintenance operations -If some manual operations are required to be performed, the procedure is as follows: -- Exec into the pod from where you want to perform the manual operation using the similar command to the following command: +If you need to perform some maintenance operations manually, then the procedure is as follows: +1. Use `kubectl exec` to access the pod where you want to perform the manual operation, a command similar to the following: + + kubectl exec -it /bin/bash + +2. The important locations, such as like ORACLE_HOME, ORDS_HOME, and so on, can be seen in the environment, by using the `env` command. - kubectl exec -it /bin/bash +3. Log In to `sqlplus` to perform manual operations by using the following command: -- The important locations like ORACLE_HOME, ORDS_HOME etc. can be seen in the environment, by using the `env` command. -- Login to `sqlplus` to perform manual operations using the following command: sqlplus / as sysdba From ede67fdc457841a11a187c2fdc861305191a8061 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Wed, 4 May 2022 13:16:14 +0530 Subject: [PATCH 344/628] Fix formating --- .../oraclerestdataservice_controller.go | 63 +++++++++---------- 1 file changed, 30 insertions(+), 33 deletions(-) diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 472b8e57..ac50bb14 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -121,6 +121,9 @@ func (r *OracleRestDataServiceReconciler) Reconcile(ctx context.Context, req ctr if oracleRestDataService.Status.DatabaseActionsUrl == "" { oracleRestDataService.Status.DatabaseActionsUrl = dbcommons.ValueUnavailable } + oracleRestDataService.Status.DatabaseRef = oracleRestDataService.Spec.DatabaseRef + oracleRestDataService.Status.LoadBalancer = strconv.FormatBool(oracleRestDataService.Spec.LoadBalancer) + oracleRestDataService.Status.Image = oracleRestDataService.Spec.Image // Fetch Primary Database Reference singleInstanceDatabase := &dbapi.SingleInstanceDatabase{} @@ -251,21 +254,17 @@ func (r *OracleRestDataServiceReconciler) validate(m *dbapi.OracleRestDataServic eventMsgs = append(eventMsgs, "databaseRef cannot be updated") } if m.Status.LoadBalancer != "" && m.Status.LoadBalancer != strconv.FormatBool(m.Spec.LoadBalancer) { - eventMsgs = append(eventMsgs, "service patching is not avaiable currently") + eventMsgs = append(eventMsgs, "service patching is not available currently") } if m.Status.Image.PullFrom != "" && m.Status.Image != m.Spec.Image { - eventMsgs = append(eventMsgs, "image patching is not avaiable currently") + eventMsgs = append(eventMsgs, "image patching is not available currently") } - m.Status.DatabaseRef = m.Spec.DatabaseRef - m.Status.LoadBalancer = strconv.FormatBool(m.Spec.LoadBalancer) - m.Status.Image = m.Spec.Image - if len(eventMsgs) > 0 { r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, strings.Join(eventMsgs, ",")) r.Log.Info(strings.Join(eventMsgs, "\n")) err = errors.New(strings.Join(eventMsgs, ",")) - return requeueN, err + return requeueY, err } return requeueN, err @@ -348,7 +347,6 @@ func (r *OracleRestDataServiceReconciler) validateSIDBReadiness(m *dbapi.OracleR return requeueN, sidbReadyPod } - //##################################################################################################### // Check ORDS Health Status //##################################################################################################### @@ -386,7 +384,7 @@ func (r *OracleRestDataServiceReconciler) checkHealthStatus(m *dbapi.OracleRestD if !m.Status.OrdsInstalled { m.Status.OrdsInstalled = true out, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", - ctx, req, false, "bash", "-c", fmt.Sprintf("echo -e \"%s\" | %s", dbcommons.OpenPDBSeed, dbcommons.SQLPlusCLI)) + ctx, req, false, "bash", "-c", fmt.Sprintf("echo -e \"%s\" | %s", dbcommons.OpenPDBSeed, dbcommons.SQLPlusCLI)) if err != nil { log.Error(err, err.Error()) } else { @@ -444,9 +442,9 @@ func (r *OracleRestDataServiceReconciler) instantiateSVCSpec(m *dbapi.OracleRest // Instantiate POD spec from OracleRestDataService spec //############################################################################# func (r *OracleRestDataServiceReconciler) instantiatePodSpec(m *dbapi.OracleRestDataService, - n *dbapi.SingleInstanceDatabase) (*corev1.Pod, *corev1.Secret) { + n *dbapi.SingleInstanceDatabase) (*corev1.Pod, *corev1.Secret) { - initSecret := &corev1.Secret { + initSecret := &corev1.Secret{ TypeMeta: metav1.TypeMeta{ Kind: "Secret", }, @@ -454,11 +452,11 @@ func (r *OracleRestDataServiceReconciler) instantiatePodSpec(m *dbapi.OracleRest Name: m.Name, Namespace: m.Namespace, Labels: map[string]string{ - "app": m.Name, + "app": m.Name, }, }, Type: corev1.SecretTypeOpaque, - StringData: map[string]string { + StringData: map[string]string{ "init-cmd": dbcommons.InitORDSCMD, }, } @@ -709,16 +707,16 @@ func (r *OracleRestDataServiceReconciler) instantiatePVCSpec(m *dbapi.OracleRest return nil } return &metav1.LabelSelector{ - MatchLabels: func() map[string]string { - ns := make(map[string]string) - if len(m.Spec.NodeSelector) != 0 { - for key, value := range m.Spec.NodeSelector { - ns[key] = value - } - } - return ns - }(), + MatchLabels: func() map[string]string { + ns := make(map[string]string) + if len(m.Spec.NodeSelector) != 0 { + for key, value := range m.Spec.NodeSelector { + ns[key] = value + } } + return ns + }(), + } }(), }, } @@ -763,10 +761,10 @@ func (r *OracleRestDataServiceReconciler) createSVC(ctx context.Context, req ctr if m.Spec.LoadBalancer { if len(svc.Status.LoadBalancer.Ingress) > 0 { m.Status.DatabaseApiUrl = "https://" + svc.Status.LoadBalancer.Ingress[0].IP + ":" + - fmt.Sprint(svc.Spec.Ports[0].Port) + "/ords/"+n.Status.Pdbname+"/_/db-api/stable/" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/ords/" + n.Status.Pdbname + "/_/db-api/stable/" m.Status.ServiceIP = svc.Status.LoadBalancer.Ingress[0].IP m.Status.DatabaseActionsUrl = "https://" + svc.Status.LoadBalancer.Ingress[0].IP + ":" + - fmt.Sprint(svc.Spec.Ports[0].Port) + "/ords/sql-developer" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/ords/sql-developer" if m.Status.ApexConfigured { m.Status.ApxeUrl = "https://" + svc.Status.LoadBalancer.Ingress[0].IP + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/ords/" + n.Status.Pdbname + "/apex" @@ -778,7 +776,7 @@ func (r *OracleRestDataServiceReconciler) createSVC(ctx context.Context, req ctr if nodeip != "" { m.Status.ServiceIP = nodeip m.Status.DatabaseApiUrl = "https://" + nodeip + ":" + fmt.Sprint(svc.Spec.Ports[0].NodePort) + - "/ords/"+n.Status.Pdbname+"/_/db-api/stable/" + "/ords/" + n.Status.Pdbname + "/_/db-api/stable/" m.Status.DatabaseActionsUrl = "https://" + nodeip + ":" + fmt.Sprint(svc.Spec.Ports[0].NodePort) + "/ords/sql-developer" if m.Status.ApexConfigured { @@ -844,7 +842,7 @@ func (r *OracleRestDataServiceReconciler) createPods(m *dbapi.OracleRestDataServ var gracePeriodSeconds int64 = 0 policy := metav1.DeletePropagationForeground r.Delete(ctx, &podsMarkedToBeDeleted[i], &client.DeleteOptions{ - GracePeriodSeconds: &gracePeriodSeconds, PropagationPolicy: &policy }) + GracePeriodSeconds: &gracePeriodSeconds, PropagationPolicy: &policy}) } log.Info(m.Name, " pods other than one of Ready Pods : ", dbcommons.GetPodNames(available)) @@ -895,7 +893,7 @@ func (r *OracleRestDataServiceReconciler) createPods(m *dbapi.OracleRestDataServ var gracePeriodSeconds int64 = 0 policy := metav1.DeletePropagationForeground err := r.Delete(ctx, &pod, &client.DeleteOptions{ - GracePeriodSeconds: &gracePeriodSeconds, PropagationPolicy: &policy }) + GracePeriodSeconds: &gracePeriodSeconds, PropagationPolicy: &policy}) noDeleted += 1 if err != nil { r.Log.Error(err, "Failed to delete existing POD", "POD.Name", pod.Name) @@ -974,7 +972,6 @@ func (r *OracleRestDataServiceReconciler) cleanupOracleRestDataService(req ctrl. m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase) error { log := r.Log.WithValues("cleanupOracleRestDataService", req.NamespacedName) - if m.Status.OrdsInstalled { // ## FETCH THE SIDB REPLICAS . sidbReadyPod, _, _, _, err := dbcommons.FindPods(r, n.Spec.Image.Version, @@ -1054,7 +1051,7 @@ func (r *OracleRestDataServiceReconciler) cleanupOracleRestDataService(req ctrl. uninstallORDS := fmt.Sprintf(dbcommons.UninstallORDSCMD, adminPassword) out, err = dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, true, "bash", "-c", - uninstallORDS) + uninstallORDS) log.Info("UninstallORDSCMD Output : " + out) if strings.Contains(strings.ToUpper(out), "ERROR") { return errors.New(out) @@ -1075,9 +1072,9 @@ func (r *OracleRestDataServiceReconciler) cleanupOracleRestDataService(req ctrl. //Delete ORDS pod var gracePeriodSeconds int64 = 0 - policy := metav1.DeletePropagationForeground + policy := metav1.DeletePropagationForeground r.Delete(ctx, &readyPod, &client.DeleteOptions{ - GracePeriodSeconds: &gracePeriodSeconds, PropagationPolicy: &policy }) + GracePeriodSeconds: &gracePeriodSeconds, PropagationPolicy: &policy}) //Delete Database Admin Password Secret if !*m.Spec.AdminPassword.KeepSecret { @@ -1150,7 +1147,7 @@ func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataS var gracePeriodSeconds int64 = 0 policy := metav1.DeletePropagationForeground err = r.Delete(ctx, &ordsReadyPod, &client.DeleteOptions{ - GracePeriodSeconds: &gracePeriodSeconds, PropagationPolicy: &policy }) + GracePeriodSeconds: &gracePeriodSeconds, PropagationPolicy: &policy}) if err != nil { r.Log.Error(err, "Failed to delete existing POD", "POD.Name", ordsReadyPod.Name) return requeueY @@ -1196,7 +1193,7 @@ func (r *OracleRestDataServiceReconciler) installApex(m *dbapi.OracleRestDataSer //Install Apex in SIDB ready pod out, err := dbcommons.ExecCommand(r, r.Config, ordsReadyPod.Name, ordsReadyPod.Namespace, "", ctx, req, true, "bash", "-c", - fmt.Sprintf(dbcommons.InstallApexInContainer, apexPassword, sidbPassword, n.Status.Pdbname)) + fmt.Sprintf(dbcommons.InstallApexInContainer, apexPassword, sidbPassword, n.Status.Pdbname)) if err != nil { log.Info(err.Error()) } From 4b69e509efa088630d0d14d5ab622f5879ef5364 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Wed, 4 May 2022 13:26:31 +0530 Subject: [PATCH 345/628] Fix status --- controllers/database/oraclerestdataservice_controller.go | 1 - 1 file changed, 1 deletion(-) diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index ac50bb14..740b8e7d 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -301,7 +301,6 @@ func (r *OracleRestDataServiceReconciler) validateSIDBReadiness(m *dbapi.OracleR err = r.Get(ctx, types.NamespacedName{Name: m.Spec.AdminPassword.SecretName, Namespace: m.Namespace}, adminPasswordSecret) if err != nil { if apierrors.IsNotFound(err) { - m.Status.Status = dbcommons.StatusError eventReason := "Waiting" eventMsg := "waiting for secret : " + m.Spec.AdminPassword.SecretName + " to get created" r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) From 5a4d11534bf75b0a245057431501e4ff98b2cbd2 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Wed, 4 May 2022 13:39:12 +0530 Subject: [PATCH 346/628] Enhance status updates --- controllers/database/oraclerestdataservice_controller.go | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 740b8e7d..b5cadaf9 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -109,16 +109,10 @@ func (r *OracleRestDataServiceReconciler) Reconcile(ctx context.Context, req ctr } /* Initialize Status */ - if oracleRestDataService.Status.Status == "" { + if strings.TrimSpace(oracleRestDataService.Status.Status) == "" { oracleRestDataService.Status.Status = dbcommons.StatusPending - } - if oracleRestDataService.Status.ApxeUrl == "" { oracleRestDataService.Status.ApxeUrl = dbcommons.ValueUnavailable - } - if oracleRestDataService.Status.DatabaseApiUrl == "" { oracleRestDataService.Status.DatabaseApiUrl = dbcommons.ValueUnavailable - } - if oracleRestDataService.Status.DatabaseActionsUrl == "" { oracleRestDataService.Status.DatabaseActionsUrl = dbcommons.ValueUnavailable } oracleRestDataService.Status.DatabaseRef = oracleRestDataService.Spec.DatabaseRef @@ -301,6 +295,7 @@ func (r *OracleRestDataServiceReconciler) validateSIDBReadiness(m *dbapi.OracleR err = r.Get(ctx, types.NamespacedName{Name: m.Spec.AdminPassword.SecretName, Namespace: m.Namespace}, adminPasswordSecret) if err != nil { if apierrors.IsNotFound(err) { + m.Status.Status = dbcommons.StatusPending eventReason := "Waiting" eventMsg := "waiting for secret : " + m.Spec.AdminPassword.SecretName + " to get created" r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) From 7a695121b075374a542aa57d651626b9696b0ebb Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Wed, 4 May 2022 13:45:42 +0530 Subject: [PATCH 347/628] Add missing status save --- controllers/database/oraclerestdataservice_controller.go | 1 + 1 file changed, 1 insertion(+) diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index b5cadaf9..a6bdc42f 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -114,6 +114,7 @@ func (r *OracleRestDataServiceReconciler) Reconcile(ctx context.Context, req ctr oracleRestDataService.Status.ApxeUrl = dbcommons.ValueUnavailable oracleRestDataService.Status.DatabaseApiUrl = dbcommons.ValueUnavailable oracleRestDataService.Status.DatabaseActionsUrl = dbcommons.ValueUnavailable + r.Status().Update(ctx, oracleRestDataService) } oracleRestDataService.Status.DatabaseRef = oracleRestDataService.Spec.DatabaseRef oracleRestDataService.Status.LoadBalancer = strconv.FormatBool(oracleRestDataService.Spec.LoadBalancer) From a4b9aa16d81f85c194ce923984500d71fb1b3874 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Wed, 4 May 2022 13:50:44 +0530 Subject: [PATCH 348/628] Fix if check --- controllers/database/oraclerestdataservice_controller.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index a6bdc42f..9d21f57d 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -109,7 +109,7 @@ func (r *OracleRestDataServiceReconciler) Reconcile(ctx context.Context, req ctr } /* Initialize Status */ - if strings.TrimSpace(oracleRestDataService.Status.Status) == "" { + if oracleRestDataService.Status.Status == "" { oracleRestDataService.Status.Status = dbcommons.StatusPending oracleRestDataService.Status.ApxeUrl = dbcommons.ValueUnavailable oracleRestDataService.Status.DatabaseApiUrl = dbcommons.ValueUnavailable From 8001a93bc256db9738003ce44c0a5ddc1d3ed2e2 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Wed, 4 May 2022 13:54:19 +0530 Subject: [PATCH 349/628] Fix status update --- controllers/database/oraclerestdataservice_controller.go | 1 + 1 file changed, 1 insertion(+) diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 9d21f57d..1fa21970 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -356,6 +356,7 @@ func (r *OracleRestDataServiceReconciler) checkHealthStatus(m *dbapi.OracleRestD return requeueY, readyPod } if readyPod.Name == "" { + m.Status.Status = dbcommons.StatusNotReady return requeueY, readyPod } From d94130582e91012dd8c043bca3d3c7322d8a8f95 Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Wed, 4 May 2022 14:46:09 +0530 Subject: [PATCH 350/628] Some corrections Signed-off-by: abhisbyk --- docs/sidb/README.md | 44 +++++++++++++++++++++----------------------- 1 file changed, 21 insertions(+), 23 deletions(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index b40aa2ad..d6e78a6a 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -116,9 +116,9 @@ $ kubectl describe singleinstancedatabase sidb-sample-clone ## Provision New Database -You can easily provision a new database instance on the Kubernetes cluster by using [singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml). +You can easily provision a new database instance on the Kubernetes cluster by using **[singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml)**. -1. Sign into [Oracle Container Registry](https://container-registry.oracle.com/ords/f?p=113:4:7154182141811:::4:P4_REPOSITORY,AI_REPOSITORY,AI_REPOSITORY_NAME,P4_REPOSITORY_NAME,P4_EULA_ID,P4_BUSINESS_AREA_ID:9,9,Oracle%20Database%20Enterprise%20Edition,Oracle%20Database%20Enterprise%20Edition,1,0&cs=3Y_90hkCQLfJzrvTLiEipIGgWGUytfrtAPuHFocuWd0NDSacbBPlamohfLuiJA-bAsVL6Z_yKEMsTbb52bm6IRA) and accept the license agreement for the Database image, ignore if you have accepted already. +1. Log into [Oracle Container Registry](https://container-registry.oracle.com/ords/f?p=113:4:7154182141811:::4:P4_REPOSITORY,AI_REPOSITORY,AI_REPOSITORY_NAME,P4_REPOSITORY_NAME,P4_EULA_ID,P4_BUSINESS_AREA_ID:9,9,Oracle%20Database%20Enterprise%20Edition,Oracle%20Database%20Enterprise%20Edition,1,0&cs=3Y_90hkCQLfJzrvTLiEipIGgWGUytfrtAPuHFocuWd0NDSacbBPlamohfLuiJA-bAsVL6Z_yKEMsTbb52bm6IRA) and accept the license agreement for the Database image, ignore if you have accepted already. 2. If you have not already done so, create an image pull secret for the Oracle Container Registry: @@ -141,11 +141,11 @@ You can easily provision a new database instance on the Kubernetes cluster by us ``` **NOTE:** -- For ease of use, the storage class **oci-bv** is specified in the [singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml). This storage class facilitates dynamic provisioning of the OCI block volumes on the Oracle OKE for persistent storage of the database. For other cloud providers, you can similarly use their dynamic provisioning storage class. +- For ease of use, the storage class **oci-bv** is specified in the **[singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml)**. This storage class facilitates dynamic provisioning of the OCI block volumes on the Oracle OKE for persistent storage of the database. For other cloud providers, you can similarly use their dynamic provisioning storage class. - Supports Oracle Database Enterprise Edition (19.3.0), and later releases. ### Provisioning a new XE database -To provision new Oracle Database Express Edition (XE) database, use the sample **[config/samples/sidb/singleinstancedatabase_express.yaml](../../config/samples/sidb/singleinstancedatabase_express.yaml)** file. For example: +To provision new Oracle Database Express Edition (XE) database, use the template **[config/samples/sidb/singleinstancedatabase_express.yaml](../../config/samples/sidb/singleinstancedatabase_express.yaml)** file. For example: kubectl apply -f singleinstancedatabase_express.yaml @@ -156,8 +156,7 @@ Oracle Database XE edition is supported Release 21c (21.3.0) and later releases. ### Provision a pre-built database -Provision a new pre-built database instance by specifying appropriate values for the attributes in the example **[config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml](../../config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml)** file, and running the following command: - +To provision a new pre-built database instance, use the template **[config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml](../../config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml)** file. For example: ```sh $ kubectl apply -f singleinstancedatabase_prebuiltdb.yaml @@ -274,7 +273,7 @@ You can patch the Service can be patched after creating the Single Instance Data Options: * NodePort - '{"spec":{"loadBalancer": false}}' -* LoadBalancer - '{"spec":{"loadBalancer": true }}' +* LoadBalancer - '{"spec":{"loadBalancer": true}}' For example: @@ -288,7 +287,7 @@ $ kubectl --type=merge -p '{"spec":{"loadBalancer": false}}' patch singleinstanc To create copies of your existing database quickly, you can use the cloning functionality. A cloned database is an exact, block-for-block copy of the source database. Cloning is much faster than creating a fresh database and copying over the data. -To clone an existing database, specify the source database reference as the value for the `cloneFrom` attribute in the sample [singleinstancedatabase_clone.yaml](../../config/samples/sidb/singleinstancedatabase_clone.yaml) file. +To clone an existing database, specify the source database reference as the value for the `cloneFrom` attribute in the sample **[singleinstancedatabase_clone.yaml](../../config/samples/sidb/singleinstancedatabase_clone.yaml)** file. **Note**: To clone a database, The source database must have archiveLog mode set to true. @@ -311,7 +310,7 @@ Patched Oracle Docker images can be built by using this [patching extension](htt ### Patch existing Database -To patch an existing database, edit and apply the [singleinstancedatabase_patch.yaml](../../config/samples/sidb/singleinstancedatabase_patch.yaml) file of the database resource/object either by specifying a new release update for image attributes, or by running the following command: +To patch an existing database, edit and apply the **[singleinstancedatabase_patch.yaml](../../config/samples/sidb/singleinstancedatabase_patch.yaml)** file of the database resource/object either by specifying a new release update for image attributes, or by running the following command: ```sh kubectl --type=merge -p '{"spec":{"image":{"pullFrom":"patched-image:tag","pullSecrets":"pull-secret"}}}' patch singleinstancedatabase sidb-sample @@ -347,10 +346,10 @@ The Oracle Database Operator creates the `OracleRestDataService` (ORDS) kind as ### OracleRestDataService template YAML -The template `.yaml` file for Oracle Rest Data Services (`OracleRestDataService` kind) is available at [config/samples/sidb/oraclerestdataservice.yaml](config/samples/sidb/oraclerestdataservice.yaml). +The template `.yaml` file for Oracle Rest Data Services (`OracleRestDataService` kind) is available at **[config/samples/sidb/oraclerestdataservice.yaml](config/samples/sidb/oraclerestdataservice.yaml)**. **Note:** -- For **quick provisioning** of the ORDS, apply the [config/samples/sidb/oraclerestdataservice_create.yaml](../../config/samples/sidb/oraclerestdataservice_create.yaml) file, using the following command: +- For **quick provisioning** of the ORDS, apply the **[config/samples/sidb/oraclerestdataservice_create.yaml](../../config/samples/sidb/oraclerestdataservice_create.yaml)** file, using the following command: kubectl apply -f oraclerestdataservice_create.yaml @@ -358,7 +357,7 @@ The template `.yaml` file for Oracle Rest Data Services (`OracleRestDataService` Note the following: - The `adminPassword` and `ordsPassword` fields in the `oraclerestdataservice.yaml` file contains secrets for authenticating the Single Instance Database and the ORDS user with the following roles: `SQL Administrator, System Administrator, SQL Developer, oracle.dbtools.autorest.any.schema`. - To build the ORDS image, use the following instructions: [Building Oracle REST Data Services Install Images](https://github.com/oracle/docker-images/tree/main/OracleRestDataServices#building-oracle-rest-data-services-install-images). -- By default, ORDS uses self-signed certificates. To use certificates from the Certificate Authority, the ORDS image needs to be rebuilt after specifying the values of `ssl.cert` and `ssl.cert.key` in the [standalone.properties](https://github.com/oracle/docker-images/blob/main/OracleRestDataServices/dockerfiles/standalone.properties.tmpl) file. After you rebuild the ORDS image, use the rebuilt image in the [config/samples/sidb/oraclerestdataservice.yaml](config/samples/sidb/oraclerestdataservice.yaml) file. +- By default, ORDS uses self-signed certificates. To use certificates from the Certificate Authority, the ORDS image needs to be rebuilt after specifying the values of `ssl.cert` and `ssl.cert.key` in the [standalone.properties](https://github.com/oracle/docker-images/blob/main/OracleRestDataServices/dockerfiles/standalone.properties.tmpl) file. After you rebuild the ORDS image, use the rebuilt image in the **[config/samples/sidb/oraclerestdataservice.yaml](config/samples/sidb/oraclerestdataservice.yaml)** file. ### List OracleRestDataServices To list the ORDS service, use the following command: @@ -382,7 +381,7 @@ ords-sample Healthy sidb-sample https://10.0.25.54:8443/ords/ORCLPDB1 ``` ### Detailed Status -To obtain a detaile status check of the ORDS service, use the following command: +To obtain a detailed status check of the ORDS service, use the following command: ```sh $ kubectl describe oraclerestdataservice ords-sample @@ -417,7 +416,7 @@ $ kubectl describe oraclerestdataservice ords-sample ## REST Enable Database -To provision a new ORDS instance, specify the appropriate values for the attributes in the the example [config/samples/sidb/oraclerestdataservice_create.yaml](../../config/samples/sidb/oraclerestdataservice_create.yaml) file, and run the following command: +To provision a new ORDS instance, use the template **[config/samples/sidb/oraclerestdataservice_create.yaml](../../config/samples/sidb/oraclerestdataservice_create.yaml)** file. For example: ```sh $ kubectl apply -f oraclerestdataservice_create.yaml @@ -428,7 +427,7 @@ After this command completes, ORDS is installed in the container database (CDB) ### Creation Status -Creating a new ORDS instance takes a while. To check the status of the ORDS insteance, use the following command: +Creating a new ORDS instance takes a while. To check the status of the ORDS instance, use the following command: ```sh $ kubectl get oraclerestdataservice/ords-sample --template={{.status.status}} @@ -453,7 +452,7 @@ There are two basic approaches for authentication to the REST Endpoints. Certain #### Database API -To call certain REST endpoints, you must use the ORDS_PUBLIC_USER user with role `SQL Administrator`, and `.spec.ordsPassword` credentials. +To call certain REST endpoints, you must use the ORDS_PUBLIC_USER with role `SQL Administrator`, and `.spec.ordsPassword` credentials. The ORDS user also has the following additional roles: `System Administrator , SQL Developer , oracle.dbtools.autorest.any.schema`. @@ -480,14 +479,14 @@ Some examples for the Database API usage are as follows: ```sh curl -s -k -X GET -u 'ORDS_PUBLIC_USER:<.spec.ordsPassword>' https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/database/parameters/ | python -m json.tool ``` -* ##### Getting all feature usage statitics +* ##### Getting all feature usage statistics ```sh curl -s -k -X GET -u 'ORDS_PUBLIC_USER:<.spec.ordsPassword>' https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/database/feature_usage/ | python -m json.tool ``` #### REST Enabled SQL -The REST Enable SQL functionality is available to all the schemas specified in the `.spec.restEnableSchemas` atrribute of the sample yaml. +The REST Enable SQL functionality is available to all the schemas specified in the `.spec.restEnableSchemas` attribute of the sample yaml. Only these schemas will have access SQL Developer Web Console specified by the Database Actions URL. The REST Enabled SQL functionality enables REST calls to send DML, DDL and scripts to any REST enabled schema by exposing the same SQL engine used in SQL Developer and Oracle SQLcl (SQL Developer Command Line). @@ -545,7 +544,7 @@ REST APIs for Oracle Data Pump Jobs can be found at [https://docs.oracle.com/en/ Database Actions is a web-based interface that uses Oracle REST Data Services to provide development, data tools, administration and monitoring features for Oracle Database. * To use Database Actions, you must sign in as a database user whose schema has been REST-enabled. -* To enable a schema for REST, you can specify appropriate values for the `.spec.restEnableSchemas` attributes details in the sample `yaml` [config/samples/sidb/oraclerestdataservice.yaml](config/samples/sidb/oraclerestdataservice.yaml), which are needed for authorizing Database Actions. +* To enable a schema for REST, you can specify appropriate values for the `.spec.restEnableSchemas` attributes details in the sample `yaml` **[config/samples/sidb/oraclerestdataservice.yaml](config/samples/sidb/oraclerestdataservice.yaml)**, which are needed for authorizing Database Actions. * Schema are created (if they exist) with the username as `.spec.restEnableSchema[].schema` and password as `.spec.ordsPassword.`. * UrlMapping `.spec.restEnableSchema[].urlMapping` is optional and is defaulted to `.spec.restEnableSchema[].schema`. @@ -581,11 +580,10 @@ To access APEX, You need to configure APEX with the ORDS. The following section #### Configure APEX with ORDS -* For quick provisioning, apply the [config/samples/sidb/oraclerestdataservice_apex.yaml](../../confi/samples/sidb/oraclerestdataservice_apex.yaml) file. First, it creates `ords-secret`, `apex-secret`, and then provision the ORDS configured with Oracle APEX. It uses the ORDS image hosted on the [Oracle Container Registry](https://container-registry.oracle.com/ords/f?p=113:4:113387942129427:::4:P4_REPOSITORY,AI_REPOSITORY,AI_REPOSITORY_NAME,P4_REPOSITORY_NAME,P4_EULA_ID,P4_BUSINESS_AREA_ID:1183,1183,Oracle%20REST%20Data%20Services%20(ORDS)%20with%20Application%20Express,Oracle%20REST%20Data%20Services%20(ORDS)%20with%20Application%20Express,1,0&cs=3_y-KlneZIxRRfXzerC_0ro7P1MGh-B_9lTEQObVTdoQCWkmsQ3lHpFs90Z8QFheteVQEzPvtUVHEQAqqXegYbA). +* For quick provisioning, apply the **[config/samples/sidb/oraclerestdataservice_apex.yaml](../../confi/samples/sidb/oraclerestdataservice_apex.yaml)** file. First, it creates `ords-secret`, `apex-secret`, and then provision the ORDS configured with Oracle APEX. It uses the ORDS image hosted on the [Oracle Container Registry](https://container-registry.oracle.com/ords/f?p=113:4:113387942129427:::4:P4_REPOSITORY,AI_REPOSITORY,AI_REPOSITORY_NAME,P4_REPOSITORY_NAME,P4_EULA_ID,P4_BUSINESS_AREA_ID:1183,1183,Oracle%20REST%20Data%20Services%20(ORDS)%20with%20Application%20Express,Oracle%20REST%20Data%20Services%20(ORDS)%20with%20Application%20Express,1,0&cs=3_y-KlneZIxRRfXzerC_0ro7P1MGh-B_9lTEQObVTdoQCWkmsQ3lHpFs90Z8QFheteVQEzPvtUVHEQAqqXegYbA). kubectl apply -f oraclerestdataservice_apex.yaml -* To provision ORDS step by step for APEX, set `.spec.apexPassword.secretName` to a non-null string in [config/samples/sidb/oraclerestdataservice.yaml](../../config/samples/sidb/oraclerestdataservice.yaml) * The APEX Password is used as a common password for APEX_PUBLIC_USER, APEX_REST_PUBLIC_USER, APEX_LISTENER and Apex administrator (username: ADMIN) mapped to secretKey * The status of ORDS turns to `Updating` during APEX configuration, and changes to `Healthy` after successful configuration. You can also check status by using the following command: @@ -598,7 +596,7 @@ To access APEX, You need to configure APEX with the ORDS. The following section * If you configure APEX after ORDS is installed, then ORDS pods will be deleted and recreated. -Application Express can be accessed via browser using `.status.apexUrl` in the following command . +Application Express can be accessed via browser using `.status.apexUrl` in the following command. ```sh $ kubectl get oraclerestdataservice/ords-sample --template={{.status.apexUrl}} @@ -606,7 +604,7 @@ $ kubectl get oraclerestdataservice/ords-sample --template={{.status.apexUrl}} https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/ ``` -Sign in to Administration servies using +Sign in to Administration services using workspace: `INTERNAL` username: `ADMIN` password: `.spec.apexPassword` From c9c10fb8267f61964a606a06b207a826327d8be5 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Wed, 4 May 2022 15:46:28 +0530 Subject: [PATCH 351/628] Using DB PVC --- controllers/database/oraclerestdataservice_controller.go | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 1fa21970..056b4119 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -242,9 +242,9 @@ func (r *OracleRestDataServiceReconciler) validate(m *dbapi.OracleRestDataServic } // If using same pvc for ords as sidb, ensure sidb has ReadWriteMany Accessmode - if n.Spec.Persistence.AccessMode == "ReadWriteOnce" && m.Spec.Persistence.Size == "" { - eventMsgs = append(eventMsgs, "ords can be installed only on ReadWriteMany Access Mode of : "+m.Spec.DatabaseRef) - } + //if n.Spec.Persistence.AccessMode == "ReadWriteOnce" && m.Spec.Persistence.Size == "" { + // eventMsgs = append(eventMsgs, "ords can be installed only on ReadWriteMany Access Mode of : "+m.Spec.DatabaseRef) + //} if m.Status.DatabaseRef != "" && m.Status.DatabaseRef != m.Spec.DatabaseRef { eventMsgs = append(eventMsgs, "databaseRef cannot be updated") } @@ -256,6 +256,7 @@ func (r *OracleRestDataServiceReconciler) validate(m *dbapi.OracleRestDataServic } if len(eventMsgs) > 0 { + m.Status.Status = dbcommons.StatusError r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, strings.Join(eventMsgs, ",")) r.Log.Info(strings.Join(eventMsgs, "\n")) err = errors.New(strings.Join(eventMsgs, ",")) From d958a58394443ea6669c37f91b32c2f2591284a4 Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Wed, 4 May 2022 15:50:28 +0530 Subject: [PATCH 352/628] using jsonpath uniformly, some enhancements Signed-off-by: abhisbyk --- docs/sidb/README.md | 33 +++++++++++++-------------------- 1 file changed, 13 insertions(+), 20 deletions(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 3a76a957..a8037b8d 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -145,7 +145,7 @@ You can easily provision a new database instance on the Kubernetes cluster by us - Supports Oracle Database Enterprise Edition (19.3.0), and later releases. ### Provisioning a new XE database -To provision new Oracle Database Express Edition (XE) database, use the template **[config/samples/sidb/singleinstancedatabase_express.yaml](../../config/samples/sidb/singleinstancedatabase_express.yaml)** file. For example: +To provision new Oracle Database Express Edition (XE) database, use the sample **[config/samples/sidb/singleinstancedatabase_express.yaml](../../config/samples/sidb/singleinstancedatabase_express.yaml)** file. For example: kubectl apply -f singleinstancedatabase_express.yaml @@ -156,7 +156,7 @@ Oracle Database XE edition is supported Release 21c (21.3.0) and later releases. ### Provision a pre-built database -To provision a new pre-built database instance, use the template **[config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml](../../config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml)** file. For example: +To provision a new pre-built database instance, use the sample **[config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml](../../config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml)** file. For example: ```sh $ kubectl apply -f singleinstancedatabase_prebuiltdb.yaml @@ -264,21 +264,14 @@ $ kubectl --type=merge -p '{"spec":{"sid":"ORCL1"}}' patch singleinstancedatabas The SingleInstanceDatabase "sidb-sample" is invalid: spec.sid: Forbidden: cannot be changed ``` -#### Patch Service +#### Enable LoadBalancer Service -You can patch the Service can be patched after creating the Single Instance Database instance. - -**Caution**: Patching the service **replaces the existing Service with a new type**. - -Options: - -* NodePort - '{"spec":{"loadBalancer": false}}' -* LoadBalancer - '{"spec":{"loadBalancer": true}}' +For the Single Instance Database, the default service is the `NodePort` service. You can enable the `LoadBalancer` service by using `kubectl patch` command. For example: ```sh -$ kubectl --type=merge -p '{"spec":{"loadBalancer": false}}' patch singleinstancedatabase sidb-sample +$ kubectl --type=merge -p '{"spec":{"loadBalancer": true}}' patch singleinstancedatabase sidb-sample singleinstancedatabase.database.oracle.com/sidb-sample patched ``` @@ -416,7 +409,7 @@ $ kubectl describe oraclerestdataservice ords-sample ## REST Enable Database -To provision a new ORDS instance, use the template **[config/samples/sidb/oraclerestdataservice_create.yaml](../../config/samples/sidb/oraclerestdataservice_create.yaml)** file. For example: +To provision a new ORDS instance, use the sample **[config/samples/sidb/oraclerestdataservice_create.yaml](../../config/samples/sidb/oraclerestdataservice_create.yaml)** file. For example: ```sh $ kubectl apply -f oraclerestdataservice_create.yaml @@ -430,7 +423,7 @@ After this command completes, ORDS is installed in the container database (CDB) Creating a new ORDS instance takes a while. To check the status of the ORDS instance, use the following command: ```sh -$ kubectl get oraclerestdataservice/ords-sample --template={{.status.status}} +$ kubectl get oraclerestdataservice/ords-sample -o "jsonpath={.status.status}" Healthy ``` @@ -441,7 +434,7 @@ ORDS is open for connections when the `status` column returns `Healthy`. Clients can access the REST Endpoints using `.status.databaseApiUrl` as shown in the following command. ```sh -$ kubectl get oraclerestdataservice/ords-sample --template={{.status.databaseApiUrl}} +$ kubectl get oraclerestdataservice/ords-sample -o "jsonpath={.status.databaseApiUrl}" https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/ ``` @@ -551,7 +544,7 @@ Database Actions is a web-based interface that uses Oracle REST Data Services to Database Actions can be accessed with a browser by using `.status.databaseActionsUrl`. For example: ```sh -$ kubectl get oraclerestdataservice/ords-sample --template={{.status.databaseActionsUrl}} +$ kubectl get oraclerestdataservice/ords-sample -o "jsonpath={.status.databaseActionsUrl}" https://10.0.25.54:8443/ords/sql-developer ``` @@ -580,7 +573,7 @@ To access APEX, You need to configure APEX with the ORDS. The following section #### Configure APEX with ORDS -* For quick provisioning, apply the **[config/samples/sidb/oraclerestdataservice_apex.yaml](../../confi/samples/sidb/oraclerestdataservice_apex.yaml)** file. First, it creates `ords-secret`, `apex-secret`, and then provision the ORDS configured with Oracle APEX. It uses the ORDS image hosted on the [Oracle Container Registry](https://container-registry.oracle.com/ords/f?p=113:4:113387942129427:::4:P4_REPOSITORY,AI_REPOSITORY,AI_REPOSITORY_NAME,P4_REPOSITORY_NAME,P4_EULA_ID,P4_BUSINESS_AREA_ID:1183,1183,Oracle%20REST%20Data%20Services%20(ORDS)%20with%20Application%20Express,Oracle%20REST%20Data%20Services%20(ORDS)%20with%20Application%20Express,1,0&cs=3_y-KlneZIxRRfXzerC_0ro7P1MGh-B_9lTEQObVTdoQCWkmsQ3lHpFs90Z8QFheteVQEzPvtUVHEQAqqXegYbA). +* For quick provisioning, use the sample **[config/samples/sidb/oraclerestdataservice_apex.yaml](../../confi/samples/sidb/oraclerestdataservice_apex.yaml)** file. For example: kubectl apply -f oraclerestdataservice_apex.yaml @@ -589,7 +582,7 @@ To access APEX, You need to configure APEX with the ORDS. The following section ```sh - $ kubectl get oraclerestdataservice ords-sample -o "jsonpath=[{.status.apexConfigured}]" + $ kubectl get oraclerestdataservice ords-sample -o "jsonpath={.status.apexConfigured}" [true] ``` @@ -599,9 +592,9 @@ To access APEX, You need to configure APEX with the ORDS. The following section Application Express can be accessed via browser using `.status.apexUrl` in the following command. ```sh -$ kubectl get oraclerestdataservice/ords-sample --template={{.status.apexUrl}} +$ kubectl get oraclerestdataservice/ords-sample -o "jsonpath={.status.apexUrl}" - https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/ + https://10.0.25.54:8443/ords/ORCLPDB1/apex ``` Sign in to Administration services using From cb016ae2555ac61d5312ddb9ddd1e0211249e58a Mon Sep 17 00:00:00 2001 From: Mohammed Yunus Date: Wed, 4 May 2022 10:40:09 +0000 Subject: [PATCH 353/628] Update README.md --- docs/sidb/README.md | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index a8037b8d..a91cf9c4 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -116,7 +116,7 @@ $ kubectl describe singleinstancedatabase sidb-sample-clone ## Provision New Database -You can easily provision a new database instance on the Kubernetes cluster by using **[singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml)**. +You can easily provision a new database instance on the Kubernetes cluster by using **[config/samples/sidb/singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml)**. 1. Log into [Oracle Container Registry](https://container-registry.oracle.com/ords/f?p=113:4:7154182141811:::4:P4_REPOSITORY,AI_REPOSITORY,AI_REPOSITORY_NAME,P4_REPOSITORY_NAME,P4_EULA_ID,P4_BUSINESS_AREA_ID:9,9,Oracle%20Database%20Enterprise%20Edition,Oracle%20Database%20Enterprise%20Edition,1,0&cs=3Y_90hkCQLfJzrvTLiEipIGgWGUytfrtAPuHFocuWd0NDSacbBPlamohfLuiJA-bAsVL6Z_yKEMsTbb52bm6IRA) and accept the license agreement for the Database image, ignore if you have accepted already. @@ -280,7 +280,7 @@ $ kubectl --type=merge -p '{"spec":{"loadBalancer": true}}' patch singleinstance To create copies of your existing database quickly, you can use the cloning functionality. A cloned database is an exact, block-for-block copy of the source database. Cloning is much faster than creating a fresh database and copying over the data. -To clone an existing database, specify the source database reference as the value for the `cloneFrom` attribute in the sample **[singleinstancedatabase_clone.yaml](../../config/samples/sidb/singleinstancedatabase_clone.yaml)** file. +To quickly clone the existing database sidb-sample created above, use the sample **[config/samples/sidb/singleinstancedatabase_clone.yaml](../../config/samples/sidb/singleinstancedatabase_clone.yaml)** file. **Note**: To clone a database, The source database must have archiveLog mode set to true. @@ -303,7 +303,7 @@ Patched Oracle Docker images can be built by using this [patching extension](htt ### Patch existing Database -To patch an existing database, edit and apply the **[singleinstancedatabase_patch.yaml](../../config/samples/sidb/singleinstancedatabase_patch.yaml)** file of the database resource/object either by specifying a new release update for image attributes, or by running the following command: +To patch an existing database, edit and apply the **[config/samples/sidb/singleinstancedatabase_patch.yaml](../../config/samples/sidb/singleinstancedatabase_patch.yaml)** file of the database resource/object either by specifying a new release update for image attributes, or by running the following command: ```sh kubectl --type=merge -p '{"spec":{"image":{"pullFrom":"patched-image:tag","pullSecrets":"pull-secret"}}}' patch singleinstancedatabase sidb-sample @@ -341,8 +341,7 @@ The Oracle Database Operator creates the `OracleRestDataService` (ORDS) kind as The template `.yaml` file for Oracle Rest Data Services (`OracleRestDataService` kind) is available at **[config/samples/sidb/oraclerestdataservice.yaml](config/samples/sidb/oraclerestdataservice.yaml)**. -**Note:** -- For **quick provisioning** of the ORDS, apply the **[config/samples/sidb/oraclerestdataservice_create.yaml](../../config/samples/sidb/oraclerestdataservice_create.yaml)** file, using the following command: +For **quick provisioning** of ORDS, apply the sample **[config/samples/sidb/oraclerestdataservice_create.yaml](../../config/samples/sidb/oraclerestdataservice_create.yaml)** file, using the following command: kubectl apply -f oraclerestdataservice_create.yaml @@ -447,7 +446,7 @@ There are two basic approaches for authentication to the REST Endpoints. Certain To call certain REST endpoints, you must use the ORDS_PUBLIC_USER with role `SQL Administrator`, and `.spec.ordsPassword` credentials. -The ORDS user also has the following additional roles: `System Administrator , SQL Developer , oracle.dbtools.autorest.any.schema`. +The ORDS user also has the following additional roles: `System Administrator, SQL Developer, oracle.dbtools.autorest.any.schema`. Use this ORDS user to authenticate the following: * Database APIs From 913ff88915c4e74eb16a91e9422588f3fa80d72a Mon Sep 17 00:00:00 2001 From: Mohammed Yunus Date: Wed, 4 May 2022 15:25:56 +0000 Subject: [PATCH 354/628] Update README.md --- docs/sidb/README.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index a91cf9c4..f435fe87 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -187,6 +187,11 @@ $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.connectSt 10.0.25.54:1521/ORCL ``` +Use any supported client or SQLPlus to connect to the database as follows +```sh +$ sqlplus sys/<.spec.adminPassword>@10.0.25.54:1521/ORCL as sysdba +``` + The Oracle Database inside the container also has Oracle Enterprise Manager Express (OEM Express) configured. To access OEM Express, start the browser, and paste in a URL similar to the following example: ```sh From 8ed4113af5ae69862961ae1b7eb62e7fa5f7b9ad Mon Sep 17 00:00:00 2001 From: Mohammed Yunus Date: Wed, 4 May 2022 16:02:22 +0000 Subject: [PATCH 355/628] Update README.md --- docs/sidb/README.md | 21 +++++++++++++++++++-- 1 file changed, 19 insertions(+), 2 deletions(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index f435fe87..f92171c6 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -179,17 +179,34 @@ $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.status}" ### Connection Information -External and internal (running in Kubernetes pods) clients can connect to the database by using `.status.connectString`. For example: +Clients can get the connect string to the CDB from `.status.connectString` and PDB from `.status.pdbConnectString`. For example: ```sh $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.connectString}" 10.0.25.54:1521/ORCL ``` +```sh +$ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.pdbConnectString}" + + 10.0.25.54:1521/ORCLPDB +``` -Use any supported client or SQLPlus to connect to the database as follows +Use any supported client or SQLPlus to connect to the database using the above connect strings as follows ```sh $ sqlplus sys/<.spec.adminPassword>@10.0.25.54:1521/ORCL as sysdba + +SQL*Plus: Release 19.0.0.0.0 - Production on Wed May 4 16:00:49 2022 +Version 19.14.0.0.0 + +Copyright (c) 1982, 2021, Oracle. All rights reserved. + + +Connected to: +Oracle Database 21c Express Edition Release 21.0.0.0.0 - Production +Version 21.3.0.0.0 + +SQL> ``` The Oracle Database inside the container also has Oracle Enterprise Manager Express (OEM Express) configured. To access OEM Express, start the browser, and paste in a URL similar to the following example: From 6ac6abb42cb82fcb73b62319ecfdc1cdd8aff7c1 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Wed, 4 May 2022 21:36:58 +0530 Subject: [PATCH 356/628] PVC enhancement for ORDS --- .../v1alpha1/oraclerestdataservice_webhook.go | 4 -- .../samples/sidb/oraclerestdataservice.yaml | 9 +-- .../sidb/oraclerestdataservice_apex.yaml | 8 --- .../sidb/oraclerestdataservice_create.yaml | 8 --- .../oraclerestdataservice_controller.go | 62 +++++++++++++++---- .../singleinstancedatabase_controller.go | 13 +--- 6 files changed, 55 insertions(+), 49 deletions(-) diff --git a/apis/database/v1alpha1/oraclerestdataservice_webhook.go b/apis/database/v1alpha1/oraclerestdataservice_webhook.go index bb889943..eae7860c 100644 --- a/apis/database/v1alpha1/oraclerestdataservice_webhook.go +++ b/apis/database/v1alpha1/oraclerestdataservice_webhook.go @@ -123,10 +123,6 @@ func (r *OracleRestDataService) ValidateUpdate(oldRuntimeObject runtime.Object) allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("databaseRef"), "cannot be changed")) } - if old.Status.LoadBalancer != "" && old.Status.LoadBalancer != strconv.FormatBool(r.Spec.LoadBalancer) { - allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec").Child("loadBalancer"), "cannot be changed")) - } if old.Status.Image.PullFrom != "" && old.Status.Image != r.Spec.Image { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("image"), "cannot be changed")) diff --git a/config/samples/sidb/oraclerestdataservice.yaml b/config/samples/sidb/oraclerestdataservice.yaml index 20c3fb13..60cb7ed2 100644 --- a/config/samples/sidb/oraclerestdataservice.yaml +++ b/config/samples/sidb/oraclerestdataservice.yaml @@ -41,13 +41,14 @@ spec: pullFrom: pullSecrets: + ## Dedicated persistence storage is optional. If not specified, ORDS will use the databaseRef persistence storage ## size : Minimum size of pvc | class : PVC storage Class ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. - persistence: - size: 50Gi - storageClass: "oci-bv" - accessMode: "ReadWriteOnce" + # persistence: + # size: 50Gi + # storageClass: "oci-bv" + # accessMode: "ReadWriteOnce" ## Type of service Applicable on cloud enviroments only. ## if loadBalService: false, service type = "NodePort". else "LoadBalancer" diff --git a/config/samples/sidb/oraclerestdataservice_apex.yaml b/config/samples/sidb/oraclerestdataservice_apex.yaml index 1fb6c0f0..bb632bac 100644 --- a/config/samples/sidb/oraclerestdataservice_apex.yaml +++ b/config/samples/sidb/oraclerestdataservice_apex.yaml @@ -53,14 +53,6 @@ spec: image: pullFrom: container-registry.oracle.com/database/ords:21.4.2-gh - ## size : Minimum size of pvc | class : PVC storage Class - ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany - ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. - persistence: - size: 50Gi - storageClass: "oci-bv" - accessMode: "ReadWriteOnce" - ## PDB Schemas to be ORDS Enabled. ## Schema will be created (if not exists) with password as .spec.ordsPassword. restEnableSchemas: diff --git a/config/samples/sidb/oraclerestdataservice_create.yaml b/config/samples/sidb/oraclerestdataservice_create.yaml index c2b204a7..007f41e4 100644 --- a/config/samples/sidb/oraclerestdataservice_create.yaml +++ b/config/samples/sidb/oraclerestdataservice_create.yaml @@ -37,14 +37,6 @@ spec: image: pullFrom: container-registry.oracle.com/database/ords:21.4.2-gh - ## size : Minimum size of pvc | class : PVC storage Class - ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany - ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. - persistence: - size: 50Gi - storageClass: "oci-bv" - accessMode: "ReadWriteOnce" - ## PDB Schemas to be ORDS Enabled. ## Schema will be created (if not exists) with password as .spec.ordsPassword. restEnableSchemas: diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 056b4119..57f86c18 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -241,16 +241,13 @@ func (r *OracleRestDataServiceReconciler) validate(m *dbapi.OracleRestDataServic } } - // If using same pvc for ords as sidb, ensure sidb has ReadWriteMany Accessmode - //if n.Spec.Persistence.AccessMode == "ReadWriteOnce" && m.Spec.Persistence.Size == "" { - // eventMsgs = append(eventMsgs, "ords can be installed only on ReadWriteMany Access Mode of : "+m.Spec.DatabaseRef) - //} + // If ORDS has no peristence specified, ensure SIDB has persistence configured + if m.Spec.Persistence.Size == "" && n.Spec.Persistence.AccessMode == "" { + eventMsgs = append(eventMsgs, "ORDS cannot be configured for database " + m.Spec.DatabaseRef + " that has not attached persistent volume") + } if m.Status.DatabaseRef != "" && m.Status.DatabaseRef != m.Spec.DatabaseRef { eventMsgs = append(eventMsgs, "databaseRef cannot be updated") } - if m.Status.LoadBalancer != "" && m.Status.LoadBalancer != strconv.FormatBool(m.Spec.LoadBalancer) { - eventMsgs = append(eventMsgs, "service patching is not available currently") - } if m.Status.Image.PullFrom != "" && m.Status.Image != m.Spec.Image { eventMsgs = append(eventMsgs, "image patching is not available currently") } @@ -471,6 +468,29 @@ func (r *OracleRestDataServiceReconciler) instantiatePodSpec(m *dbapi.OracleRest }, }, Spec: corev1.PodSpec{ + Affinity: func() *corev1.Affinity { + if m.Spec.Persistence.Size == "" && n.Spec.Persistence.AccessMode == "ReadWriteOnce" { + return &corev1.Affinity{ + PodAffinity: &corev1.PodAffinity{ + PreferredDuringSchedulingIgnoredDuringExecution: []corev1.WeightedPodAffinityTerm{{ + Weight: 100, + PodAffinityTerm: corev1.PodAffinityTerm{ + LabelSelector: &metav1.LabelSelector{ + MatchExpressions: []metav1.LabelSelectorRequirement{{ + Key: "app", + Operator: metav1.LabelSelectorOpIn, + Values: []string{n.Name}, // Schedule on same host as DB Pod + }}, + }, + TopologyKey: "kubernetes.io/hostname", + }, + }, + }, + }, + } + } + return nil + }(), Volumes: []corev1.Volume{ { Name: "datamount", @@ -731,11 +751,29 @@ func (r *OracleRestDataServiceReconciler) createSVC(ctx context.Context, req ctr log := r.Log.WithValues("createSVC", req.NamespacedName) // Check if the Service already exists, if not create a new one svc := &corev1.Service{} - // Get retrieves an obj for the given object key from the Kubernetes Cluster. - // obj must be a struct pointer so that obj can be updated with the response returned by the Server. - // Here foundsvc is the struct pointer to corev1.Service{} + svcDeleted := false + // Check if the Service already exists, if not create a new one + // Get retrieves an obj ( a struct pointer ) for the given object key from the Kubernetes Cluster. err := r.Get(ctx, types.NamespacedName{Name: m.Name, Namespace: m.Namespace}, svc) - if err != nil && apierrors.IsNotFound(err) { + if err == nil { + log.Info("Found Existing Service ", "Service.Name", svc.Name) + svcType := corev1.ServiceType("NodePort") + if m.Spec.LoadBalancer { + svcType = corev1.ServiceType("LoadBalancer") + } + + if svc.Spec.Type != svcType { + log.Info("Deleting SVC", " name ", svc.Name) + err = r.Delete(ctx, svc) + if err != nil { + r.Log.Error(err, "Failed to delete svc", " Name", svc.Name) + return requeueN + } + svcDeleted = true + } + } + + if svcDeleted || (err != nil && apierrors.IsNotFound(err)) { // Define a new Service svc = r.instantiateSVCSpec(m) log.Info("Creating a new Service", "Service.Namespace", svc.Namespace, "Service.Name", svc.Name) @@ -750,8 +788,6 @@ func (r *OracleRestDataServiceReconciler) createSVC(ctx context.Context, req ctr } else if err != nil { log.Error(err, "Failed to get Service") return requeueY - } else if err == nil { - log.Info("Found Existing Service ", "Service.Name", svc.Name) } m.Status.ServiceIP = "" diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 1e8b851c..442f89b3 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -121,28 +121,17 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct /* Initialize Status */ if singleInstanceDatabase.Status.Status == "" { singleInstanceDatabase.Status.Status = dbcommons.StatusPending - } - if singleInstanceDatabase.Status.Edition == "" { if singleInstanceDatabase.Spec.Edition != "" { singleInstanceDatabase.Status.Edition = strings.Title(singleInstanceDatabase.Spec.Edition) } else { singleInstanceDatabase.Status.Edition = dbcommons.ValueUnavailable } - } - if singleInstanceDatabase.Status.Role == "" { singleInstanceDatabase.Status.Role = dbcommons.ValueUnavailable - } - if singleInstanceDatabase.Status.ConnectString == "" { singleInstanceDatabase.Status.ConnectString = dbcommons.ValueUnavailable - } - if singleInstanceDatabase.Status.PdbConnectString == "" { singleInstanceDatabase.Status.PdbConnectString = dbcommons.ValueUnavailable - } - if singleInstanceDatabase.Status.OemExpressUrl == "" { singleInstanceDatabase.Status.OemExpressUrl = dbcommons.ValueUnavailable - } - if singleInstanceDatabase.Status.ReleaseUpdate == "" { singleInstanceDatabase.Status.ReleaseUpdate = dbcommons.ValueUnavailable + r.Status().Update(ctx, singleInstanceDatabase) } // Manage SingleInstanceDatabase Deletion From c1bc571fddce3d122bff99c7e726b5db09cb02e9 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Wed, 4 May 2022 21:39:22 +0530 Subject: [PATCH 357/628] Fix compilation error --- apis/database/v1alpha1/oraclerestdataservice_webhook.go | 1 - 1 file changed, 1 deletion(-) diff --git a/apis/database/v1alpha1/oraclerestdataservice_webhook.go b/apis/database/v1alpha1/oraclerestdataservice_webhook.go index eae7860c..ed73c9e9 100644 --- a/apis/database/v1alpha1/oraclerestdataservice_webhook.go +++ b/apis/database/v1alpha1/oraclerestdataservice_webhook.go @@ -39,7 +39,6 @@ package v1alpha1 import ( - "strconv" apierrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/runtime" From 4e6ccae187991ed63b69005131866cee8cadc32a Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Wed, 4 May 2022 23:53:04 +0530 Subject: [PATCH 358/628] Fix ORDS order for relocation --- .../database/oraclerestdataservice_controller.go | 16 +++++++++++++--- .../singleinstancedatabase_controller.go | 6 ++++++ 2 files changed, 19 insertions(+), 3 deletions(-) diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 57f86c18..d469876e 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -279,15 +279,17 @@ func (r *OracleRestDataServiceReconciler) validateSIDBReadiness(m *dbapi.OracleR return requeueY, sidbReadyPod } + if m.Status.OrdsInstalled || m.Status.CommonUsersCreated { + return requeueN, sidbReadyPod + } + if sidbReadyPod.Name == "" || n.Status.Status != dbcommons.StatusReady { eventReason := "Waiting" eventMsg := "waiting for " + n.Name + " to be Ready" r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + m.Status.Status = dbcommons.StatusNotReady return requeueY, sidbReadyPod } - if m.Status.OrdsInstalled || m.Status.CommonUsersCreated { - return requeueN, sidbReadyPod - } // Validate databaseRef Admin Password adminPasswordSecret := &corev1.Secret{} @@ -1309,6 +1311,14 @@ func (r *OracleRestDataServiceReconciler) restEnableSchemas(m *dbapi.OracleRestD log := r.Log.WithValues("restEnableSchemas", req.NamespacedName) + if sidbReadyPod.Name == "" || n.Status.Status != dbcommons.StatusReady { + eventReason := "Waiting" + eventMsg := "waiting for " + n.Name + " to be Ready" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + m.Status.Status = dbcommons.StatusNotReady + return requeueY + } + // Get Pdbs Available availablePDBS, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", fmt.Sprintf("echo -e \"%s\" | %s", dbcommons.GetPdbsSQL, dbcommons.SQLPlusCLI)) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 442f89b3..e6f24939 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1666,6 +1666,7 @@ func (r *SingleInstanceDatabaseReconciler) updateDBConfig(m *dbapi.SingleInstanc flashBackStatus, archiveLogStatus, forceLoggingStatus, result := dbcommons.CheckDBConfig(readyPod, r, r.Config, ctx, req, m.Spec.Edition) if result.Requeue { + m.Status.Status = dbcommons.StatusNotReady return result, nil } m.Status.ArchiveLog = strconv.FormatBool(archiveLogStatus) @@ -1725,6 +1726,7 @@ func (r *SingleInstanceDatabaseReconciler) updateDBConfig(m *dbapi.SingleInstanc if m.Spec.FlashBack && !flashBackStatus { _, archiveLogStatus, _, result := dbcommons.CheckDBConfig(readyPod, r, r.Config, ctx, req, m.Spec.Edition) if result.Requeue { + m.Status.Status = dbcommons.StatusNotReady return result, nil } if archiveLogStatus { @@ -1732,6 +1734,7 @@ func (r *SingleInstanceDatabaseReconciler) updateDBConfig(m *dbapi.SingleInstanc fmt.Sprintf("echo -e \"%s\" | %s", dbcommons.FlashBackTrueSQL, dbcommons.SQLPlusCLI)) if err != nil { log.Error(err, err.Error()) + m.Status.Status = dbcommons.StatusNotReady return requeueY, err } log.Info("FlashBackTrue Output") @@ -1765,6 +1768,7 @@ func (r *SingleInstanceDatabaseReconciler) updateDBConfig(m *dbapi.SingleInstanc if !m.Spec.ArchiveLog && archiveLogStatus { flashBackStatus, _, _, result := dbcommons.CheckDBConfig(readyPod, r, r.Config, ctx, req, m.Spec.Edition) if result.Requeue { + m.Status.Status = dbcommons.StatusNotReady return result, nil } if !flashBackStatus { @@ -1773,6 +1777,7 @@ func (r *SingleInstanceDatabaseReconciler) updateDBConfig(m *dbapi.SingleInstanc fmt.Sprintf(dbcommons.ArchiveLogFalseCMD, dbcommons.SQLPlusCLI)) if err != nil { log.Error(err, err.Error()) + m.Status.Status = dbcommons.StatusNotReady return requeueY, err } log.Info("ArchiveLogFalse Output") @@ -1805,6 +1810,7 @@ func (r *SingleInstanceDatabaseReconciler) updateDBConfig(m *dbapi.SingleInstanc flashBackStatus, archiveLogStatus, forceLoggingStatus, result = dbcommons.CheckDBConfig(readyPod, r, r.Config, ctx, req, m.Spec.Edition) if result.Requeue { + m.Status.Status = dbcommons.StatusNotReady return result, nil } From ec4b31e4455433c5bf1889dd1ce1a4810bf180de Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Thu, 5 May 2022 00:29:50 +0530 Subject: [PATCH 359/628] Fix ready status --- config/samples/sidb/oraclerestdataservice_apex.yaml | 2 -- .../samples/sidb/oraclerestdataservice_create.yaml | 4 ---- .../database/oraclerestdataservice_controller.go | 13 ++++++++----- 3 files changed, 8 insertions(+), 11 deletions(-) diff --git a/config/samples/sidb/oraclerestdataservice_apex.yaml b/config/samples/sidb/oraclerestdataservice_apex.yaml index bb632bac..6af87955 100644 --- a/config/samples/sidb/oraclerestdataservice_apex.yaml +++ b/config/samples/sidb/oraclerestdataservice_apex.yaml @@ -62,5 +62,3 @@ spec: - schemaName: schema2 enable: true urlMapping: - - \ No newline at end of file diff --git a/config/samples/sidb/oraclerestdataservice_create.yaml b/config/samples/sidb/oraclerestdataservice_create.yaml index 007f41e4..f8ab5041 100644 --- a/config/samples/sidb/oraclerestdataservice_create.yaml +++ b/config/samples/sidb/oraclerestdataservice_create.yaml @@ -46,7 +46,3 @@ spec: - schemaName: schema2 enable: true urlMapping: - - - - \ No newline at end of file diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index d469876e..0540f482 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -185,7 +185,7 @@ func (r *OracleRestDataServiceReconciler) Reconcile(ctx context.Context, req ctr } var ordsReadyPod corev1.Pod - result, ordsReadyPod = r.checkHealthStatus(oracleRestDataService, sidbReadyPod, ctx, req) + result, ordsReadyPod = r.checkHealthStatus(oracleRestDataService, singleInstanceDatabase, sidbReadyPod, ctx, req) if result.Requeue { r.Log.Info("Reconcile queued") return result, nil @@ -345,7 +345,7 @@ func (r *OracleRestDataServiceReconciler) validateSIDBReadiness(m *dbapi.OracleR //##################################################################################################### // Check ORDS Health Status //##################################################################################################### -func (r *OracleRestDataServiceReconciler) checkHealthStatus(m *dbapi.OracleRestDataService, +func (r *OracleRestDataServiceReconciler) checkHealthStatus(m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase, sidbReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) (ctrl.Result, corev1.Pod) { log := r.Log.WithValues("checkHealthStatus", req.NamespacedName) @@ -375,8 +375,11 @@ func (r *OracleRestDataServiceReconciler) checkHealthStatus(m *dbapi.OracleRestD } } + m.Status.Status = dbcommons.StatusNotReady if strings.Contains(out, "HTTP/1.1 200 OK") || strings.Contains(strings.ToUpper(err.Error()), "HTTP/1.1 200 OK") { - m.Status.Status = dbcommons.StatusReady + if n.Status.Status == dbcommons.StatusReady || n.Status.Status == dbcommons.StatusUpdating || n.Status.Status == dbcommons.StatusPatching { + m.Status.Status = dbcommons.StatusReady + } if !m.Status.OrdsInstalled { m.Status.OrdsInstalled = true out, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", @@ -388,8 +391,8 @@ func (r *OracleRestDataServiceReconciler) checkHealthStatus(m *dbapi.OracleRestD log.Info(out) } } - } else { - m.Status.Status = dbcommons.StatusNotReady + } + if m.Status.Status == dbcommons.StatusNotReady { return requeueY, readyPod } return requeueN, readyPod From f1dd3a12ac9020d3fe1a02f64dec705ea3a4ef39 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Thu, 5 May 2022 00:56:54 +0530 Subject: [PATCH 360/628] Fix typos --- config/samples/sidb/oraclerestdataservice.yaml | 2 +- controllers/database/oraclerestdataservice_controller.go | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/config/samples/sidb/oraclerestdataservice.yaml b/config/samples/sidb/oraclerestdataservice.yaml index 60cb7ed2..9a34d2fa 100644 --- a/config/samples/sidb/oraclerestdataservice.yaml +++ b/config/samples/sidb/oraclerestdataservice.yaml @@ -41,7 +41,7 @@ spec: pullFrom: pullSecrets: - ## Dedicated persistence storage is optional. If not specified, ORDS will use the databaseRef persistence storage + ## Dedicated persistent storage is optional. If not specified, ORDS will use the databaseRef persistent storage ## size : Minimum size of pvc | class : PVC storage Class ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 0540f482..4c500c51 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -243,7 +243,7 @@ func (r *OracleRestDataServiceReconciler) validate(m *dbapi.OracleRestDataServic // If ORDS has no peristence specified, ensure SIDB has persistence configured if m.Spec.Persistence.Size == "" && n.Spec.Persistence.AccessMode == "" { - eventMsgs = append(eventMsgs, "ORDS cannot be configured for database " + m.Spec.DatabaseRef + " that has not attached persistent volume") + eventMsgs = append(eventMsgs, "ORDS cannot be configured for database " + m.Spec.DatabaseRef + " that has no attached persistent volume") } if m.Status.DatabaseRef != "" && m.Status.DatabaseRef != m.Spec.DatabaseRef { eventMsgs = append(eventMsgs, "databaseRef cannot be updated") From 382c741d741aedc104c01a71304d4f0ed3ca3e6c Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Thu, 5 May 2022 12:35:40 +0530 Subject: [PATCH 361/628] Bug fixes --- config/samples/sidb/singleinstancedatabase.yaml | 2 +- .../database/singleinstancedatabase_controller.go | 13 ++++++++++++- 2 files changed, 13 insertions(+), 2 deletions(-) diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index 23a83fd1..4388d802 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -17,7 +17,7 @@ spec: ## specify connect string as `:/` instead of source database ref cloneFrom: "" - ## DB edition. N/A if cloning from a Source DB (if cloneFrom is set) + ## DB edition. N/A if cloning from a Source DB in current K8s cluster (if cloneFrom is set to a database ref) ## Valid values for edition are enterprise, standard or express edition: enterprise diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index e6f24939..30f19507 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -679,6 +679,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns } } if m.Spec.CloneFrom == "" { + // For new DB use case return []corev1.EnvVar{ { Name: "SVC_HOST", @@ -703,7 +704,12 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, { Name: "WALLET_DIR", - Value: "/opt/oracle/oradata/dbconfig/$(ORACLE_SID)/.wallet", + Value: func() string { + if m.Spec.Image.PrebuiltDB { + return "" // No wallets for prebuilt DB + } + return "/opt/oracle/oradata/dbconfig/$(ORACLE_SID)/.wallet" + }(), }, { Name: "ORACLE_PDB", @@ -741,6 +747,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, } } + // For clone DB use case return []corev1.EnvVar{ { Name: "SVC_HOST", @@ -1014,6 +1021,10 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex log.Error(err, "Failed to create new Service", "Service.Namespace", svc.Namespace, "Service.Name", svc.Name) return requeueY, err } + // Reset connect strings whenever service is recreated /* + m.Status.ConnectString = dbcommons.ValueUnavailable + m.Status.PdbConnectString = dbcommons.ValueUnavailable + m.Status.OemExpressUrl = dbcommons.ValueUnavailable } else if err != nil { log.Error(err, "Failed to get Service") return requeueY, err From ba7566871156355e63ea72a01251992bb3c286c4 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Thu, 5 May 2022 13:38:27 +0530 Subject: [PATCH 362/628] Set Defaults --- apis/database/v1alpha1/singleinstancedatabase_webhook.go | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index eba392ab..9a72f8f9 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -76,9 +76,17 @@ func (r *SingleInstanceDatabase) Default() { r.Spec.AdminPassword.KeepSecret = &keepSecret } + if r.Spec.Edition == "" { + if r.Spec.CloneFrom == "" && !r.Spec.Image.PrebuiltDB { + r.Spec.Edition = "enterprise" + } + } + if r.Spec.Edition == "express" { if r.Spec.Sid == "" { r.Spec.Sid = "XE" + } else { + r.Spec.Sid = "ORCLCDB" } } From ee40e146a31c071a41fb06c92a719c53f44a1d66 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Thu, 5 May 2022 13:53:34 +0530 Subject: [PATCH 363/628] Fix defaults --- apis/database/v1alpha1/singleinstancedatabase_webhook.go | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index 9a72f8f9..ba3818e2 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -82,8 +82,8 @@ func (r *SingleInstanceDatabase) Default() { } } - if r.Spec.Edition == "express" { - if r.Spec.Sid == "" { + if r.Spec.Sid == "" { + if r.Spec.Edition == "express" { r.Spec.Sid = "XE" } else { r.Spec.Sid = "ORCLCDB" From 78f81299e9ceb6e10267dc14358c702a515342e7 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Thu, 5 May 2022 15:31:43 +0530 Subject: [PATCH 364/628] Fix config yamls --- config/samples/sidb/oraclerestdataservice_apex.yaml | 6 ++++-- config/samples/sidb/oraclerestdataservice_create.yaml | 3 ++- config/samples/sidb/singleinstancedatabase_create.yaml | 3 ++- config/samples/sidb/singleinstancedatabase_express.yaml | 3 ++- config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml | 6 ++---- 5 files changed, 12 insertions(+), 9 deletions(-) diff --git a/config/samples/sidb/oraclerestdataservice_apex.yaml b/config/samples/sidb/oraclerestdataservice_apex.yaml index 6af87955..f44f4010 100644 --- a/config/samples/sidb/oraclerestdataservice_apex.yaml +++ b/config/samples/sidb/oraclerestdataservice_apex.yaml @@ -10,7 +10,8 @@ metadata: namespace: default type: Opaque stringData: - oracle_pwd: "ChangeOnInstall_1" + ## Specify your ORDS password here + oracle_pwd: --- @@ -21,7 +22,8 @@ metadata: namespace: default type: Opaque stringData: - oracle_pwd: "ChangeOnInstall_2" + ## Specify your APEX password here + oracle_pwd: --- diff --git a/config/samples/sidb/oraclerestdataservice_create.yaml b/config/samples/sidb/oraclerestdataservice_create.yaml index f8ab5041..9d72fcc7 100644 --- a/config/samples/sidb/oraclerestdataservice_create.yaml +++ b/config/samples/sidb/oraclerestdataservice_create.yaml @@ -10,7 +10,8 @@ metadata: namespace: default type: Opaque stringData: - oracle_pwd: "ChangeOnInstall_1" + ## Specify your ORDS password here + oracle_pwd: --- diff --git a/config/samples/sidb/singleinstancedatabase_create.yaml b/config/samples/sidb/singleinstancedatabase_create.yaml index 1038614d..22dee813 100644 --- a/config/samples/sidb/singleinstancedatabase_create.yaml +++ b/config/samples/sidb/singleinstancedatabase_create.yaml @@ -10,7 +10,8 @@ metadata: namespace: default type: Opaque stringData: - oracle_pwd: "ChangeOnInstall_1" + # Specify your DB password here + oracle_pwd: --- diff --git a/config/samples/sidb/singleinstancedatabase_express.yaml b/config/samples/sidb/singleinstancedatabase_express.yaml index 68c7e785..4bc5bf0b 100644 --- a/config/samples/sidb/singleinstancedatabase_express.yaml +++ b/config/samples/sidb/singleinstancedatabase_express.yaml @@ -10,7 +10,8 @@ metadata: namespace: default type: Opaque stringData: - oracle_pwd: "ChangeOnInstall_1" + ## Specify your DB password here + oracle_pwd: --- diff --git a/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml b/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml index ab6047d4..ddd9b581 100644 --- a/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml +++ b/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml @@ -10,7 +10,8 @@ metadata: namespace: default type: Opaque stringData: - oracle_pwd: "ChangeOnInstall_1" + ## Specify your DB password here + oracle_pwd: --- @@ -21,9 +22,6 @@ metadata: namespace: default spec: - ## Use only alphanumeric characters for sid - sid: XE - ## DB edition edition: express From f2b5e6ef9147926112e95a91a743278ccb976a67 Mon Sep 17 00:00:00 2001 From: Mohammed Yunus Date: Thu, 5 May 2022 10:14:38 +0000 Subject: [PATCH 365/628] Update README.md --- docs/sidb/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index f92171c6..39cfd6bc 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -29,7 +29,7 @@ The `adminPassword` field in the above `singleinstancedatabase.yaml` file refers Create this secret using the following command as an example: - kubectl create secret generic admin-secret --from-literal=oracle_pwd="ChangeOnInstall_1" + kubectl create secret generic admin-secret --from-literal=oracle_pwd= This command creates a secret named `admin-secret`, with the key `oracle_pwd` mapped to the actual password specified in the command. From 0396b81f0185a689236ac977b145477fa870f24f Mon Sep 17 00:00:00 2001 From: Mohammed Yunus Date: Thu, 5 May 2022 14:39:46 +0000 Subject: [PATCH 366/628] Update README.md --- docs/sidb/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 39cfd6bc..3cc16d33 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -152,7 +152,7 @@ To provision new Oracle Database Express Edition (XE) database, use the sample * This command pulls the XE image uploaded on the [Oracle Container Registry](https://container-registry.oracle.com/ords/f?p=113:4:7460390069267:::4:P4_REPOSITORY,AI_REPOSITORY,AI_REPOSITORY_NAME,P4_REPOSITORY_NAME,P4_EULA_ID,P4_BUSINESS_AREA_ID:803,803,Oracle%20Database%20Express%20Edition,Oracle%20Database%20Express%20Edition,1,0&cs=3-UN6D9nAfyqxcYnrks18OAmfFcri96NZojBQALxMdakix8wgYRBxhD8rpTFd2ak1FAtfOVFexbuOM2opsjxT9w). **NOTE:** -Oracle Database XE edition is supported Release 21c (21.3.0) and later releases. +Provisioning Oracle Database express edition is supported for release 21c (21.3.0) and later releases. ### Provision a pre-built database From 64bed0298e70b1a6f3ef582049c65d7e1d901e7a Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Sun, 8 May 2022 21:41:03 -0400 Subject: [PATCH 367/628] fix reconcile bug --- .../v1alpha1/adbfamily_common_utils.go | 14 + .../autonomouscontainerdatabase_types.go | 2 + .../autonomouscontainerdatabase_webhook.go | 11 +- .../v1alpha1/autonomousdatabase_types.go | 17 +- .../v1alpha1/autonomousdatabase_webhook.go | 57 ++- .../autonomousdatabase_webhook_test.go | 24 +- .../v1alpha1/zz_generated.deepcopy.go | 7 +- commons/database/utils.go | 2 +- commons/k8s/finalizer.go | 18 +- commons/oci/database.go | 6 +- commons/oci/vault.go | 2 +- commons/oci/workrequest.go | 2 +- .../autonomouscontainerdatabase_controller.go | 11 +- .../database/autonomousdatabase_controller.go | 434 ++++++++++++------ docs/adb/ADB_PREREQUISITES.md | 4 +- go.mod | 12 +- go.sum | 41 +- ...autonomousdatabase_controller_bind_test.go | 2 +- test/e2e/behavior/shared_behaviors.go | 71 ++- test/e2e/suite_test.go | 13 +- test/e2e/verify_connection.sql | 4 + 21 files changed, 494 insertions(+), 260 deletions(-) create mode 100644 test/e2e/verify_connection.sql diff --git a/apis/database/v1alpha1/adbfamily_common_utils.go b/apis/database/v1alpha1/adbfamily_common_utils.go index 0411f1b5..61377d5c 100644 --- a/apis/database/v1alpha1/adbfamily_common_utils.go +++ b/apis/database/v1alpha1/adbfamily_common_utils.go @@ -229,6 +229,20 @@ func IsADBIntermediateState(state database.AutonomousDatabaseLifecycleStateEnum) return false } +func ValidADBTerminateState(state database.AutonomousDatabaseLifecycleStateEnum) bool { + if state == database.AutonomousDatabaseLifecycleStateProvisioning || + state == database.AutonomousDatabaseLifecycleStateUpdating || + state == database.AutonomousDatabaseLifecycleStateScaleInProgress || + state == database.AutonomousDatabaseLifecycleStateRestoreInProgress || + state == database.AutonomousDatabaseLifecycleStateBackupInProgress || + state == database.AutonomousDatabaseLifecycleStateMaintenanceInProgress || + state == database.AutonomousDatabaseLifecycleStateRoleChangeInProgress || + state == database.AutonomousDatabaseLifecycleStateUpgrading { + return true + } + return false +} + // NextADBStableState returns the next stable state if it's an intermediate state. // Otherwise returns the same state. func NextADBStableState(state database.AutonomousDatabaseLifecycleStateEnum) database.AutonomousDatabaseLifecycleStateEnum { diff --git a/apis/database/v1alpha1/autonomouscontainerdatabase_types.go b/apis/database/v1alpha1/autonomouscontainerdatabase_types.go index ff400d5c..1d812253 100644 --- a/apis/database/v1alpha1/autonomouscontainerdatabase_types.go +++ b/apis/database/v1alpha1/autonomouscontainerdatabase_types.go @@ -179,6 +179,8 @@ func (acd *AutonomousContainerDatabase) UpdateFromOCIACD(ociObj database.Autonom // special case: an emtpy map will be nil after unmarshalling while the OCI always returns an emty map. if len(ociObj.FreeformTags) != 0 { acd.Spec.FreeformTags = ociObj.FreeformTags + } else { + acd.Spec.FreeformTags = nil } /*********************************** diff --git a/apis/database/v1alpha1/autonomouscontainerdatabase_webhook.go b/apis/database/v1alpha1/autonomouscontainerdatabase_webhook.go index 6a9241fd..03b09d1a 100644 --- a/apis/database/v1alpha1/autonomouscontainerdatabase_webhook.go +++ b/apis/database/v1alpha1/autonomouscontainerdatabase_webhook.go @@ -39,8 +39,6 @@ package v1alpha1 import ( - "reflect" - apierrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/runtime" "k8s.io/apimachinery/pkg/runtime/schema" @@ -84,8 +82,13 @@ func (r *AutonomousContainerDatabase) ValidateUpdate(old runtime.Object) error { } // cannot update when the old state is in intermediate state, except for the terminate operatrion - if IsACDIntermediateState(oldACD.Status.LifecycleState) && - !reflect.DeepEqual(oldACD.Spec, r.Spec) { + var copiedSpec *AutonomousContainerDatabaseSpec = r.Spec.DeepCopy() + changed, err := removeUnchangedFields(oldACD.Spec, copiedSpec) + if err != nil { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec"), err.Error())) + } + if IsACDIntermediateState(oldACD.Status.LifecycleState) && changed { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec"), "cannot change the spec when the lifecycleState is in an intermdeiate state")) diff --git a/apis/database/v1alpha1/autonomousdatabase_types.go b/apis/database/v1alpha1/autonomousdatabase_types.go index b1cc6500..f16ca76f 100644 --- a/apis/database/v1alpha1/autonomousdatabase_types.go +++ b/apis/database/v1alpha1/autonomousdatabase_types.go @@ -318,9 +318,11 @@ func (adb *AutonomousDatabase) UpdateFromOCIADB(ociObj database.AutonomousDataba adb.Spec.Details.IsAutoScalingEnabled = ociObj.IsAutoScalingEnabled adb.Spec.Details.IsDedicated = ociObj.IsDedicated adb.Spec.Details.LifecycleState = NextADBStableState(ociObj.LifecycleState) - // special case: an emtpy map will be nil after unmarshalling while the OCI always returns an emty map. + // Special case: an emtpy map will be nil after unmarshalling while the OCI always returns an emty map. if len(ociObj.FreeformTags) != 0 { adb.Spec.Details.FreeformTags = ociObj.FreeformTags + } else { + adb.Spec.Details.FreeformTags = nil } // Determine network.accessType @@ -339,14 +341,27 @@ func (adb *AutonomousDatabase) UpdateFromOCIADB(ociObj database.AutonomousDataba adb.Spec.Details.NetworkAccess.IsAccessControlEnabled = ociObj.IsAccessControlEnabled if len(ociObj.WhitelistedIps) != 0 { adb.Spec.Details.NetworkAccess.AccessControlList = ociObj.WhitelistedIps + } else { + adb.Spec.Details.NetworkAccess.AccessControlList = nil } adb.Spec.Details.NetworkAccess.IsMTLSConnectionRequired = ociObj.IsMtlsConnectionRequired adb.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID = ociObj.SubnetId if len(ociObj.NsgIds) != 0 { adb.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs = ociObj.NsgIds + } else { + adb.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs = nil } adb.Spec.Details.NetworkAccess.PrivateEndpoint.HostnamePrefix = ociObj.PrivateEndpointLabel + // The admin password is not going to be updated in a bind operation. Erase the field if the lastSucSpec is nil. + // Leave the wallet field as is because the download wallet operation is independent from the update operation. + lastSucSpec, _ := adb.GetLastSuccessfulSpec() + if lastSucSpec == nil { + adb.Spec.Details.AdminPassword = PasswordSpec{} + } else { + adb.Spec.Details.AdminPassword = lastSucSpec.Details.AdminPassword + } + /*********************************** * update the status subresource ***********************************/ diff --git a/apis/database/v1alpha1/autonomousdatabase_webhook.go b/apis/database/v1alpha1/autonomousdatabase_webhook.go index 1986769f..5841fe5f 100644 --- a/apis/database/v1alpha1/autonomousdatabase_webhook.go +++ b/apis/database/v1alpha1/autonomousdatabase_webhook.go @@ -40,8 +40,8 @@ package v1alpha1 import ( "fmt" - "reflect" + "github.com/oracle/oci-go-sdk/v63/common" "github.com/oracle/oci-go-sdk/v63/database" apierrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/runtime" @@ -71,8 +71,18 @@ func (r *AutonomousDatabase) Default() { if !isDedicated(r) { // Shared database // AccessType is PUBLIC by default - if r.Spec.Details.NetworkAccess.AccessType == "" { - r.Spec.Details.NetworkAccess.AccessType = NetworkAccessTypePublic + if r.Spec.Details.NetworkAccess.AccessType == NetworkAccessTypePublic { + r.Spec.Details.NetworkAccess.IsMTLSConnectionRequired = common.Bool(true) + r.Spec.Details.NetworkAccess.AccessControlList = nil + r.Spec.Details.NetworkAccess.PrivateEndpoint.HostnamePrefix = nil + r.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs = nil + r.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID = nil + } else if r.Spec.Details.NetworkAccess.AccessType == NetworkAccessTypeRestricted { + r.Spec.Details.NetworkAccess.PrivateEndpoint.HostnamePrefix = nil + r.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs = nil + r.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID = nil + } else if r.Spec.Details.NetworkAccess.AccessType == NetworkAccessTypePrivate { + r.Spec.Details.NetworkAccess.AccessControlList = nil } } else { // Dedicated database // AccessType can only be PRIVATE for a dedicated database @@ -99,7 +109,7 @@ func (r *AutonomousDatabase) ValidateCreate() error { if r.Spec.Details.LifecycleState != "" { allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec").Child("details").Child("LifecycleState"), + field.Forbidden(field.NewPath("spec").Child("details").Child("lifecycleState"), "cannot apply lifecycleState to a provision operation")) } } @@ -124,13 +134,20 @@ func (r *AutonomousDatabase) ValidateUpdate(old runtime.Object) error { return nil } - // cannot update when the old state is in intermediate, except for the terminate operatrion - if IsADBIntermediateState(oldADB.Status.LifecycleState) && - r.Spec.Details.LifecycleState != database.AutonomousDatabaseLifecycleStateTerminated && - !reflect.DeepEqual(oldADB.Spec.Details, r.Spec.Details) { + // cannot update when the old state is in intermediate, except for the terminate operatrion during valid lifecycleState + var copySpec *AutonomousDatabaseSpec = r.Spec.DeepCopy() + specChanged, err := removeUnchangedFields(oldADB.Spec, copySpec) + if err != nil { allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec").Child("details"), - "cannot change spec.details when the lifecycleState is in an intermdeiate state")) + field.Forbidden(field.NewPath("spec"), err.Error())) + } + + terminateOp := ValidADBTerminateState(oldADB.Status.LifecycleState) && copySpec.Details.LifecycleState == database.AutonomousDatabaseLifecycleStateTerminated + + if specChanged && IsADBIntermediateState(oldADB.Status.LifecycleState) && !terminateOp { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec"), + "cannot change the spec when the lifecycleState is in an intermdeiate state")) } // cannot modify autonomousDatabaseOCID @@ -143,17 +160,23 @@ func (r *AutonomousDatabase) ValidateUpdate(old runtime.Object) error { } // cannot change lifecycleState with other fields together - var lifecycleChanged, otherDetailsChanged bool + var lifecycleChanged, otherFieldsChanged bool + lifecycleChanged = oldADB.Spec.Details.LifecycleState != "" && oldADB.Spec.Details.LifecycleState != r.Spec.Details.LifecycleState - copyLifecycleState := oldADB.Spec.Details.LifecycleState - oldADB.Spec.Details.LifecycleState = r.Spec.Details.LifecycleState - otherDetailsChanged = !reflect.DeepEqual(oldADB.Spec.Details, r.Spec.Details) - oldADB.Spec.Details.LifecycleState = copyLifecycleState // restore + var copiedADB *AutonomousDatabaseSpec = r.Spec.DeepCopy() + copiedADB.Details.LifecycleState = oldADB.Spec.Details.LifecycleState + copiedADB.OCIConfig = oldADB.Spec.OCIConfig + + otherFieldsChanged, err = removeUnchangedFields(oldADB.Spec, copiedADB) + if err != nil { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec"), err.Error())) + } - if lifecycleChanged && otherDetailsChanged { + if lifecycleChanged && otherFieldsChanged { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("details").Child("LifecycleState"), - "cannot change lifecycleState with other spec.details attributes at the same time")) + "cannot change lifecycleState with other spec attributes at the same time")) } allErrs = validateCommon(r, allErrs) diff --git a/apis/database/v1alpha1/autonomousdatabase_webhook_test.go b/apis/database/v1alpha1/autonomousdatabase_webhook_test.go index d225c589..69d25c31 100644 --- a/apis/database/v1alpha1/autonomousdatabase_webhook_test.go +++ b/apis/database/v1alpha1/autonomousdatabase_webhook_test.go @@ -81,22 +81,6 @@ var _ = Describe("test AutonomousDatabase webhook", func() { Expect(k8sClient.Delete(context.TODO(), adb)).To(Succeed()) }) - It("Should set the default network access type to PUBLIC, if not specified", func() { - By("Creating an AutonomousDatabase") - - Expect(k8sClient.Create(context.TODO(), adb)).To(Succeed()) - - By("Checking the AutonomousDatabase has a network access type PUBLIC") - Eventually(func() NetworkAccessTypeEnum { - err := k8sClient.Get(context.TODO(), adbLookupKey, adb) - if err != nil { - return "" - } - - return adb.Spec.Details.NetworkAccess.AccessType - }, timeout).Should(Equal(NetworkAccessTypePublic)) - }) - It("Should set the default network access type to PRIVATE, if it's a dedicated ADB", func() { By("Creating an AutonomousDatabase with ACD_OCID") adb.Spec.Details.AutonomousContainerDatabase.OCIACD.OCID = common.String("ocid1.autonomouscontainerdatabase.oc1.dummy-acd-ocid") @@ -295,8 +279,8 @@ var _ = Describe("test AutonomousDatabase webhook", func() { Expect(k8sClient.Delete(context.TODO(), adb)).To(Succeed()) }) - It("Cannot change spec.details when the lifecycleState is in an intermdeiate state", func() { - var errMsg string = "cannot change spec.details when the lifecycleState is in an intermdeiate state" + It("Cannot change the spec when the lifecycleState is in an intermdeiate state", func() { + var errMsg string = "cannot change the spec when the lifecycleState is in an intermdeiate state" adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateUpdating Expect(k8sClient.Status().Update(context.TODO(), adb)).To(Succeed()) @@ -314,8 +298,8 @@ var _ = Describe("test AutonomousDatabase webhook", func() { validateInvalidTest(adb, true, errMsg) }) - It("Cannot change lifecycleState with other spec.details attributes at the same time", func() { - var errMsg string = "cannot change lifecycleState with other spec.details attributes at the same time" + It("Cannot change lifecycleState with other spec attributes at the same time", func() { + var errMsg string = "cannot change lifecycleState with other spec attributes at the same time" adb.Spec.Details.LifecycleState = database.AutonomousDatabaseLifecycleStateStopped adb.Spec.Details.CPUCoreCount = common.Int(2) diff --git a/apis/database/v1alpha1/zz_generated.deepcopy.go b/apis/database/v1alpha1/zz_generated.deepcopy.go index 04d45f54..700da778 100644 --- a/apis/database/v1alpha1/zz_generated.deepcopy.go +++ b/apis/database/v1alpha1/zz_generated.deepcopy.go @@ -2011,6 +2011,11 @@ func (in *SingleInstanceDatabase) DeepCopyObject() runtime.Object { // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *SingleInstanceDatabaseAdminPassword) DeepCopyInto(out *SingleInstanceDatabaseAdminPassword) { *out = *in + if in.KeepSecret != nil { + in, out := &in.KeepSecret, &out.KeepSecret + *out = new(bool) + **out = **in + } } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SingleInstanceDatabaseAdminPassword. @@ -2110,7 +2115,7 @@ func (in *SingleInstanceDatabaseSpec) DeepCopyInto(out *SingleInstanceDatabaseSp (*out)[key] = val } } - out.AdminPassword = in.AdminPassword + in.AdminPassword.DeepCopyInto(&out.AdminPassword) out.Image = in.Image out.Persistence = in.Persistence out.InitParams = in.InitParams diff --git a/commons/database/utils.go b/commons/database/utils.go index 90b2256b..1231cdf0 100644 --- a/commons/database/utils.go +++ b/commons/database/utils.go @@ -98,7 +98,7 @@ func ResourceEventHandler() predicate.Predicate { if oldStatus != newStatus { return true } - + } // Ignore updates to CR status in which case metadata.Generation does not change // Reconcile if object Deletion Timestamp Set diff --git a/commons/k8s/finalizer.go b/commons/k8s/finalizer.go index 256597fd..0344feb7 100644 --- a/commons/k8s/finalizer.go +++ b/commons/k8s/finalizer.go @@ -75,24 +75,18 @@ func patchFinalizer(kubeClient client.Client, obj client.Object) error { return kubeClient.Patch(context.TODO(), obj, patch) } -// No-op if the obj already has the finalizer func AddFinalizerAndPatch(kubeClient client.Client, obj client.Object, finalizer string) error { - if !controllerutil.ContainsFinalizer(obj, finalizer) { - controllerutil.AddFinalizer(obj, finalizer) - if err := patchFinalizer(kubeClient, obj); err != nil { - return err - } + controllerutil.AddFinalizer(obj, finalizer) + if err := patchFinalizer(kubeClient, obj); err != nil { + return err } return nil } -// No-op if the obj doesn't have the finalizer func RemoveFinalizerAndPatch(kubeClient client.Client, obj client.Object, finalizer string) error { - if controllerutil.ContainsFinalizer(obj, finalizer) { - controllerutil.RemoveFinalizer(obj, finalizer) - if err := patchFinalizer(kubeClient, obj); err != nil { - return err - } + controllerutil.RemoveFinalizer(obj, finalizer) + if err := patchFinalizer(kubeClient, obj); err != nil { + return err } return nil } diff --git a/commons/oci/database.go b/commons/oci/database.go index 93c97f22..7d93bfba 100644 --- a/commons/oci/database.go +++ b/commons/oci/database.go @@ -101,7 +101,7 @@ func NewDatabaseService( } return &databaseService{ - logger: logger, + logger: logger.WithName("dbService"), kubeClient: kubeClient, dbClient: dbClient, vaultService: vaultService, @@ -115,7 +115,7 @@ func NewDatabaseService( // ReadPassword reads the password from passwordSpec, and returns the pointer to the read password string. // The function returns a nil if nothing is read func (d *databaseService) readPassword(namespace string, passwordSpec dbv1alpha1.PasswordSpec) (*string, error) { - logger := d.logger.WithName("read-password") + logger := d.logger.WithName("readPassword") if passwordSpec.K8sSecret.Name != nil { logger.Info(fmt.Sprintf("Getting password from Secret %s", *passwordSpec.K8sSecret.Name)) @@ -254,6 +254,8 @@ func (d *databaseService) UpdateAutonomousDatabaseAdminPassword(adbOCID string, return resp, err } + d.logger.Info("==== test: new admin password = " + *adminPassword) + updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ AutonomousDatabaseId: common.String(adbOCID), UpdateAutonomousDatabaseDetails: database.UpdateAutonomousDatabaseDetails{ diff --git a/commons/oci/vault.go b/commons/oci/vault.go index bc72a9d3..4b5e5982 100644 --- a/commons/oci/vault.go +++ b/commons/oci/vault.go @@ -66,7 +66,7 @@ func NewVaultService( } return &vaultService{ - logger: logger, + logger: logger.WithName("vaultService"), secretClient: secretClient, }, nil } diff --git a/commons/oci/workrequest.go b/commons/oci/workrequest.go index 6257a4c8..03b97398 100644 --- a/commons/oci/workrequest.go +++ b/commons/oci/workrequest.go @@ -69,7 +69,7 @@ func NewWorkRequestService( } return &workRequestService{ - logger: logger, + logger: logger.WithName("workRequestService"), workClient: workClient, }, nil } diff --git a/controllers/database/autonomouscontainerdatabase_controller.go b/controllers/database/autonomouscontainerdatabase_controller.go index 4f4557ad..c74ac6da 100644 --- a/controllers/database/autonomouscontainerdatabase_controller.go +++ b/controllers/database/autonomouscontainerdatabase_controller.go @@ -91,8 +91,9 @@ func (r *AutonomousContainerDatabaseReconciler) eventFilterPredicate() predicate oldACD := e.ObjectOld.(*dbv1alpha1.AutonomousContainerDatabase) if !reflect.DeepEqual(oldACD.Status, desiredACD.Status) || + (controllerutil.ContainsFinalizer(oldACD, dbv1alpha1.LastSuccessfulSpec) != controllerutil.ContainsFinalizer(desiredACD, dbv1alpha1.LastSuccessfulSpec)) || (controllerutil.ContainsFinalizer(oldACD, dbv1alpha1.ACDFinalizer) != controllerutil.ContainsFinalizer(desiredACD, dbv1alpha1.ACDFinalizer)) { - // Don't enqueue if the status or the lastSucSpec changes + // Don't enqueue if the status, lastSucSpec, or the finalizler changes return false } @@ -222,6 +223,10 @@ func (r *AutonomousContainerDatabaseReconciler) Reconcile(ctx context.Context, r return requeueResult, nil } + if err := r.patchLastSuccessfulSpec(acd); err != nil { + return r.manageError(logger, acd, err) + } + logger.Info("AutonomousContainerDatabase reconciles successfully") return emptyResult, nil @@ -374,11 +379,11 @@ func (r *AutonomousContainerDatabaseReconciler) validateCleanup(logger logr.Logg func (r *AutonomousContainerDatabaseReconciler) validateFinalizer(acd *dbv1alpha1.AutonomousContainerDatabase) error { // Delete is not schduled. Update the finalizer for this CR if hardLink is present if acd.Spec.HardLink != nil { - if *acd.Spec.HardLink { + if *acd.Spec.HardLink && !controllerutil.ContainsFinalizer(acd, dbv1alpha1.ACDFinalizer) { if err := k8s.AddFinalizerAndPatch(r.KubeClient, acd, dbv1alpha1.ACDFinalizer); err != nil { return err } - } else { + } else if !*acd.Spec.HardLink && controllerutil.ContainsFinalizer(acd, dbv1alpha1.ACDFinalizer) { if err := k8s.RemoveFinalizerAndPatch(r.KubeClient, acd, dbv1alpha1.ACDFinalizer); err != nil { return err } diff --git a/controllers/database/autonomousdatabase_controller.go b/controllers/database/autonomousdatabase_controller.go index 0a82e400..bfbd4bc1 100644 --- a/controllers/database/autonomousdatabase_controller.go +++ b/controllers/database/autonomousdatabase_controller.go @@ -158,8 +158,9 @@ func (r *AutonomousDatabaseReconciler) eventFilterPredicate() predicate.Predicat oldADB := e.ObjectOld.(*dbv1alpha1.AutonomousDatabase) if !reflect.DeepEqual(oldADB.Status, desiredADB.Status) || + (controllerutil.ContainsFinalizer(oldADB, dbv1alpha1.LastSuccessfulSpec) != controllerutil.ContainsFinalizer(desiredADB, dbv1alpha1.LastSuccessfulSpec)) || (controllerutil.ContainsFinalizer(oldADB, dbv1alpha1.ADBFinalizer) != controllerutil.ContainsFinalizer(desiredADB, dbv1alpha1.ADBFinalizer)) { - // Don't enqueue if the status or the lastSucSpec changes + // Don't enqueue if the status, lastSucSpec, or the finalizler changes return false } @@ -193,8 +194,8 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R var ociADB *dbv1alpha1.AutonomousDatabase // Get the autonomousdatabase instance from the cluster - adb := &dbv1alpha1.AutonomousDatabase{} - if err := r.KubeClient.Get(context.TODO(), req.NamespacedName, adb); err != nil { + desiredADB := &dbv1alpha1.AutonomousDatabase{} + if err := r.KubeClient.Get(context.TODO(), req.NamespacedName, desiredADB); err != nil { // Ignore not-found errors, since they can't be fixed by an immediate requeue. // No need to change the since we don't know if we obtain the object. if apiErrors.IsNotFound(err) { @@ -207,43 +208,14 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R /****************************************************************** * Get OCI database client ******************************************************************/ - if err := r.setupOCIClients(logger, adb); err != nil { + if err := r.setupOCIClients(logger, desiredADB); err != nil { logger.Error(err, "Fail to setup OCI clients") - return r.manageError(logger, adb, err) + return r.manageError(logger.WithName("setupOCIClients"), desiredADB, err) } logger.Info("OCI clients configured succesfully") - /****************************************************************** - * Get OCI AutonomousDatabase - ******************************************************************/ - - if adb.Spec.Details.AutonomousDatabaseOCID != nil { - resp, err := r.dbService.GetAutonomousDatabase(*adb.Spec.Details.AutonomousDatabaseOCID) - if err != nil { - return r.manageError(logger, adb, err) - } - - ociADB = &dbv1alpha1.AutonomousDatabase{} - ociADB.UpdateFromOCIADB(resp.AutonomousDatabase) - } - - /****************************************************************** - * Requeue if the ADB is in an intermediate state - * No-op if the ADB OCID is nil - * To get the latest status, execute before all the reconcile logic - ******************************************************************/ - needsRequeue, err := r.validateLifecycleState(adb, ociADB) - if err != nil { - return r.manageError(logger, adb, err) - } - - if needsRequeue { - logger.WithName("validateLifecycleState").Info("Current lifecycleState is " + string(adb.Status.LifecycleState) + "; reconcile queued") - return requeueResult, nil - } - /****************************************************************** * Cleanup the resource if the resource is to be deleted. * Deletion timestamp will be added to a object before it is deleted. @@ -251,9 +223,9 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R * all the finalizers are removed from the object metadata. * Refer to this page for more details of using finalizers: https://kubernetes.io/blog/2021/05/14/using-finalizers-to-control-deletion/ ******************************************************************/ - exitReconcile, err := r.validateCleanup(logger, adb) + exitReconcile, err := r.validateCleanup(logger, desiredADB) if err != nil { - return r.manageError(logger, adb, err) + return r.manageError(logger.WithName("validateCleanup"), desiredADB, err) } if exitReconcile { @@ -263,16 +235,22 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R /****************************************************************** * Register/unregister the finalizer ******************************************************************/ - if err := r.validateFinalizer(adb); err != nil { - return r.manageError(logger, adb, err) + exit, err := r.validateFinalizer(logger, desiredADB) + if err != nil { + return r.manageError(logger.WithName("validateFinalizer"), desiredADB, err) + } + + if exit { + return emptyResult, nil } /****************************************************************** * Validate operations ******************************************************************/ - exitReconcile, result, err := r.validateOperation(logger, adb, ociADB) + modifiedADB := desiredADB.DeepCopy() // the ADB which stores the changes + exitReconcile, result, err := r.validateOperation(logger, modifiedADB, ociADB) if err != nil { - return r.manageError(logger, adb, err) + return r.manageError(logger.WithName("validateOperation"), modifiedADB, err) } if exitReconcile { return result, nil @@ -281,32 +259,66 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R /***************************************************** * Sync AutonomousDatabase Backups from OCI *****************************************************/ - if err := r.syncBackupResources(logger, adb); err != nil { - return r.manageError(logger, adb, err) + if err := r.syncBackupResources(logger, modifiedADB); err != nil { + return r.manageError(logger.WithName("syncBackupResources"), modifiedADB, err) } /***************************************************** * Validate Wallet *****************************************************/ - if err := r.validateWallet(logger, adb); err != nil { - return r.manageError(logger, adb, err) + if err := r.validateWallet(logger, modifiedADB); err != nil { + return r.manageError(logger.WithName("validateWallet"), modifiedADB, err) } /****************************************************************** - * Update the status and requeue if it's in an intermediate state + * Requeue if it's in an intermediate state. Update the status right before + * exiting the reconcile, otherwise the modifiedADB will be overwritten + * by the object returned from the cluster. ******************************************************************/ - if err := r.KubeClient.Status().Update(context.TODO(), adb); err != nil { - return r.manageError(logger, adb, err) - } + if dbv1alpha1.IsADBIntermediateState(modifiedADB.Status.LifecycleState) { + logger.WithName("IsADBIntermediateState").Info("LifecycleState is " + string(modifiedADB.Status.LifecycleState) + "; reconcile queued") + + if err := r.KubeClient.Status().Update(context.TODO(), modifiedADB); err != nil { + return r.manageError(logger.WithName("IsADBIntermediateState"), modifiedADB, err) + } - if dbv1alpha1.IsADBIntermediateState(adb.Status.LifecycleState) { - logger.WithName("IsIntermediateState").Info("LifecycleState is " + string(adb.Status.LifecycleState) + "; reconcile queued") return requeueResult, nil } - logger.Info("AutonomousDatabase reconciles successfully") + /****************************************************************** + * Update the lastSucSpec and the status, and then finish the reconcile. + * Requeue if it's in an intermediate state or the modifiedADB + * doesn't match the desiredADB. + ******************************************************************/ + // Do the comparison before updating the status to avoid being overwritten + var requeue bool = false + if !reflect.DeepEqual(modifiedADB.Spec, desiredADB.Spec) { + requeue = true + } + + if modifiedADB.GetDeletionTimestamp() != nil && + controllerutil.ContainsFinalizer(modifiedADB, dbv1alpha1.ADBFinalizer) && + modifiedADB.Status.LifecycleState == database.AutonomousDatabaseLifecycleStateTerminated { + logger.Info("The ADB is TERMINATED. The CR is to be deleted but finalizer is not yet removed; reconcile queued") + requeue = true + } + + if err := r.patchLastSuccessfulSpec(modifiedADB); err != nil { + return r.manageError(logger.WithName("patchLastSuccessfulSpec"), modifiedADB, err) + } + + if err := r.KubeClient.Status().Update(context.TODO(), modifiedADB); err != nil { + return r.manageError(logger.WithName("Status().Update"), modifiedADB, err) + } + + if requeue { + logger.Info("Reconcile queued") + return requeueResult, nil - return emptyResult, nil + } else { + logger.Info("AutonomousDatabase reconciles successfully") + return emptyResult, nil + } } func (r *AutonomousDatabaseReconciler) setupOCIClients(logger logr.Logger, adb *dbv1alpha1.AutonomousDatabase) error { @@ -341,14 +353,22 @@ func (r *AutonomousDatabaseReconciler) manageError(logger logr.Logger, adb *dbv1 var finalIssue = issue // Roll back - specChanged, err := r.getADB(l, adb) + ociADB := adb.DeepCopy() + specChanged, err := r.getADB(l, ociADB) if err != nil { finalIssue = k8s.CombineErrors(finalIssue, err) } - // We don't exit the Reconcile if the spec has changed - // becasue it will exit anyway after the manageError is called. + // Will exit the Reconcile anyway after the manageError is called. if specChanged { + // Clear the lifecycleState first to avoid the webhook error when update during an intermediate state + adb.Status.LifecycleState = "" + if err := r.KubeClient.Status().Update(context.TODO(), adb); err != nil { + finalIssue = k8s.CombineErrors(finalIssue, err) + } + + adb.Spec = ociADB.Spec + if err := r.KubeClient.Update(context.TODO(), adb); err != nil { finalIssue = k8s.CombineErrors(finalIssue, err) } @@ -368,15 +388,15 @@ func (r *AutonomousDatabaseReconciler) manageError(logger logr.Logger, adb *dbv1 func (r *AutonomousDatabaseReconciler) validateOperation( logger logr.Logger, adb *dbv1alpha1.AutonomousDatabase, - ociADB *dbv1alpha1.AutonomousDatabase) (exitReconcile bool, result ctrl.Result, err error) { - - l := logger.WithName("validateOperation") + ociADB *dbv1alpha1.AutonomousDatabase) (exit bool, result ctrl.Result, err error) { lastSpec, err := adb.GetLastSuccessfulSpec() if err != nil { return false, emptyResult, err } + l := logger.WithName("validateOperation") + // If lastSucSpec is nil, then it's CREATE or BIND opertaion if lastSpec == nil { if adb.Spec.Details.AutonomousDatabaseOCID == nil { @@ -405,12 +425,15 @@ func (r *AutonomousDatabaseReconciler) validateOperation( } l.Info("spec updated; exit reconcile") - return false, emptyResult, nil + return true, emptyResult, nil } } - // If it's not CREATE or BIND opertaion, then UPDATE or SYNC - // Compare with the lastSucSpec.details. If the details are different, it means that the user updates the spec. + // If it's not CREATE or BIND opertaion, then it's UPDATE or SYNC operation. + // In most of the case the user changes the spec, and we update the oci ADB, but when the user updates on + // the Cloud Console, the controller cannot tell the direction and how to update the resource. + // Thus we compare the current spec with the lastSucSpec. If the details are different, it means that + // the user updates the spec (UPDATE operation), otherwise it's a SYNC operation. lastDifADB := adb.DeepCopy() lastDetailsChanged, err := lastDifADB.RemoveUnchangedDetails(*lastSpec) @@ -419,42 +442,24 @@ func (r *AutonomousDatabaseReconciler) validateOperation( } if lastDetailsChanged { - l.Info("Update operation") - // Double check if the user input spec is actually different from the spec in OCI. If so, then update the resource. - difADB := adb.DeepCopy() - - // Compare the secret associated fields with the ones from lastSucSpec since they are missing in the oci ADB - ociADB.Spec.Details.AdminPassword = lastSpec.Details.AdminPassword - ociADB.Spec.Details.Wallet = lastSpec.Details.Wallet - ociDetailsChanged, err := difADB.RemoveUnchangedDetails(ociADB.Spec) - if err != nil { - return false, emptyResult, err - } + // When the update completes and the status changes from UPDATING to AVAILABLE, the lastSucSpec is not updated yet, + // so we compare with the oci ADB again to make sure that the updates are completed. - if ociDetailsChanged { - ociReqSent, err := r.updateADB(logger, adb, difADB, ociADB) - if err != nil { - return false, emptyResult, err - } - - // Requeue the k8s request if an OCI request is sent, since OCI can only process one request at a time. - if ociReqSent { - l.Info("reconcile queued") - return true, requeueResult, nil - } - } + l.Info("Update operation") - // Stop the update and patch the lastSpec when the current ADB matches the oci ADB. - if err := r.patchLastSuccessfulSpec(adb); err != nil { + exit, err := r.updateADB(logger, adb) + if err != nil { return false, emptyResult, err } - return false, emptyResult, nil + return exit, emptyResult, nil } else { l.Info("No operation specified; sync the resource") + testOldADB := adb.DeepCopy() + // The user doesn't change the spec and the controller should pull the spec from the OCI. specChanged, err := r.getADB(logger, adb) if err != nil { @@ -464,6 +469,14 @@ func (r *AutonomousDatabaseReconciler) validateOperation( if specChanged { l.Info("The local spec doesn't match the oci's spec; update the CR") + // Erase the status.lifecycleState temporarily to avoid the webhook error. + oldADB := adb.DeepCopy() + adb.Status.LifecycleState = "" + r.KubeClient.Status().Update(context.TODO(), adb) + adb.Spec = oldADB.Spec + + adb.DeepCopy().RemoveUnchangedDetails(testOldADB.Spec) + if err := r.updateCR(adb); err != nil { return false, emptyResult, err } @@ -485,7 +498,6 @@ func (r *AutonomousDatabaseReconciler) validateCleanup(logger logr.Logger, adb * if controllerutil.ContainsFinalizer(adb, dbv1alpha1.ADBFinalizer) { if adb.Status.LifecycleState == database.AutonomousDatabaseLifecycleStateTerminating { - l.Info("Resource is already in TERMINATING state") // Delete in progress, continue with the reconcile logic return false, nil } @@ -493,7 +505,7 @@ func (r *AutonomousDatabaseReconciler) validateCleanup(logger logr.Logger, adb * if adb.Status.LifecycleState == database.AutonomousDatabaseLifecycleStateTerminated { // The adb has been deleted. Remove the finalizer and exit the reconcile. // Once all finalizers have been removed, the object will be deleted. - l.Info("Resource is already in TERMINATED state; remove the finalizer") + l.Info("Resource is in TERMINATED state; remove the finalizer") if err := k8s.RemoveFinalizerAndPatch(r.KubeClient, adb, dbv1alpha1.ADBFinalizer); err != nil { return false, err } @@ -512,7 +524,7 @@ func (r *AutonomousDatabaseReconciler) validateCleanup(logger logr.Logger, adb * if adb.Spec.Details.LifecycleState != database.AutonomousDatabaseLifecycleStateTerminated { // Run finalization logic for finalizer. If the finalization logic fails, don't remove the finalizer so // that we can retry during the next reconciliation. - l.Info("Terminating Autonomous Database: " + *adb.Spec.Details.DbName) + l.Info("Terminating Autonomous Database") adb.Spec.Details.LifecycleState = database.AutonomousDatabaseLifecycleStateTerminated if err := r.KubeClient.Update(context.TODO(), adb); err != nil { return false, err @@ -529,42 +541,43 @@ func (r *AutonomousDatabaseReconciler) validateCleanup(logger logr.Logger, adb * return true, nil } -func (r *AutonomousDatabaseReconciler) validateFinalizer(adb *dbv1alpha1.AutonomousDatabase) error { +func (r *AutonomousDatabaseReconciler) validateFinalizer(logger logr.Logger, adb *dbv1alpha1.AutonomousDatabase) (exit bool, err error) { + l := logger.WithName("validateFinalizer") + // Delete is not schduled. Update the finalizer for this CR if hardLink is present + var finalizerChanged = false if adb.Spec.HardLink != nil { - if *adb.Spec.HardLink { + if *adb.Spec.HardLink && !controllerutil.ContainsFinalizer(adb, dbv1alpha1.ADBFinalizer) { + l.Info("Finalizer added") if err := k8s.AddFinalizerAndPatch(r.KubeClient, adb, dbv1alpha1.ADBFinalizer); err != nil { - return err - } - } else { - if err := k8s.RemoveFinalizerAndPatch(r.KubeClient, adb, dbv1alpha1.ADBFinalizer); err != nil { - return err + return false, err } - } - } - return nil -} + finalizerChanged = true -// validateLifecycleState gets and validates the current lifecycleState -func (r *AutonomousDatabaseReconciler) validateLifecycleState(adb *dbv1alpha1.AutonomousDatabase, ociADB *dbv1alpha1.AutonomousDatabase) (needsRequeue bool, err error) { - if ociADB == nil { - return false, nil - } + } else if !*adb.Spec.HardLink && controllerutil.ContainsFinalizer(adb, dbv1alpha1.ADBFinalizer) { + l.Info("Finalizer removed") - adb.Status = ociADB.Status + if err := k8s.RemoveFinalizerAndPatch(r.KubeClient, adb, dbv1alpha1.ADBFinalizer); err != nil { + return false, err + } - if err := r.KubeClient.Status().Update(context.TODO(), adb); err != nil { - return false, err + finalizerChanged = true + } } - if dbv1alpha1.IsADBIntermediateState(ociADB.Status.LifecycleState) { + // If the finalizer is changed during an intermediate state, e.g. set hardLink to true and + // delete the resource, then there must be another ongoing reconcile. In this case we should + // exit the reconcile. + if finalizerChanged && dbv1alpha1.IsADBIntermediateState(adb.Status.LifecycleState) { + l.Info("Finalizer changed during an intermediate state, exit the reconcile") return true, nil } return false, nil } +// updateCR updates the lastSucSpec and the CR func (r *AutonomousDatabaseReconciler) updateCR(adb *dbv1alpha1.AutonomousDatabase) error { // Update the lastSucSpec if err := adb.UpdateLastSuccessfulSpec(); err != nil { @@ -578,6 +591,8 @@ func (r *AutonomousDatabaseReconciler) updateCR(adb *dbv1alpha1.AutonomousDataba } func (r *AutonomousDatabaseReconciler) patchLastSuccessfulSpec(adb *dbv1alpha1.AutonomousDatabase) error { + copyADB := adb.DeepCopy() + specBytes, err := json.Marshal(adb.Spec) if err != nil { return err @@ -589,6 +604,9 @@ func (r *AutonomousDatabaseReconciler) patchLastSuccessfulSpec(adb *dbv1alpha1.A annotations.PatchAnnotations(r.KubeClient, adb, anns) + adb.Spec = copyADB.Spec + adb.Status = copyADB.Status + return nil } @@ -599,18 +617,24 @@ func (r *AutonomousDatabaseReconciler) createADB(logger logr.Logger, adb *dbv1al return err } + // Restore the admin password after updating from OCI ADB + adminPass := adb.Spec.Details.AdminPassword adb.UpdateFromOCIADB(resp.AutonomousDatabase) + adb.Spec.Details.AdminPassword = adminPass return nil } +// getADB gets the information from OCI and overwrites the spec and the status, but not update the CR in the cluster func (r *AutonomousDatabaseReconciler) getADB(logger logr.Logger, adb *dbv1alpha1.AutonomousDatabase) (bool, error) { if adb == nil { return false, errors.New("AutonomousDatabase OCID is missing") } + l := logger.WithName("getADB") + // Get the information from OCI - logger.WithName("getADB").Info("Sending GetAutonomousDatabase request to OCI") + l.Info("Sending GetAutonomousDatabase request to OCI") resp, err := r.dbService.GetAutonomousDatabase(*adb.Spec.Details.AutonomousDatabaseOCID) if err != nil { return false, err @@ -625,28 +649,75 @@ func (r *AutonomousDatabaseReconciler) getADB(logger logr.Logger, adb *dbv1alpha // The AutonomousDatabase is updated with the returned object from the OCI requests. func (r *AutonomousDatabaseReconciler) updateADB( logger logr.Logger, - adb *dbv1alpha1.AutonomousDatabase, - difADB *dbv1alpha1.AutonomousDatabase, - ociADB *dbv1alpha1.AutonomousDatabase) (ociReqSent bool, err error) { + adb *dbv1alpha1.AutonomousDatabase) (exit bool, err error) { + + l := logger.WithName("updateADB") - validations := []func(logr.Logger, *dbv1alpha1.AutonomousDatabase, *dbv1alpha1.AutonomousDatabase, *dbv1alpha1.AutonomousDatabase) (bool, error){ - r.validateGeneralFields, - r.validateAdminPassword, - r.validateDbWorkload, - r.validateLicenseModel, - r.validateScalingFields, - r.validateGeneralNetworkAccess, - r.validateDesiredLifecycleState, + // Get OCI AutonomousDatabase and update the lifecycleState of the CR, + // so that the validatexx functions know when the state changes back to AVAILABLE + ociADB := adb.DeepCopy() + _, err = r.getADB(logger, ociADB) + if err != nil { + return false, err } - for _, op := range validations { - ociReqSent, err := op(logger, adb, difADB, ociADB) + adb.Status.LifecycleState = ociADB.Status.LifecycleState + + // Start update + difADB := adb.DeepCopy() + + ociDetailsChanged, err := difADB.RemoveUnchangedDetails(ociADB.Spec) + if err != nil { + return false, err + } + + // Do the update request only if the current ADB is actually different from the OCI ADB + if ociDetailsChanged { + // Special case: if the oci ADB is terminating, then update the spec and exit the reconcile. + // This happens when the lifecycleState changes to TERMINATED during an intermediate state, + // whatever is in progress should be abandonded and the desired spec should the same as oci ADB. + if ociADB.Status.LifecycleState == database.AutonomousDatabaseLifecycleStateTerminating { + l.Info("OCI ADB is in TERMINATING state; update the spec and exit the reconcile") + + adb.Status.LifecycleState = "" + if err := r.KubeClient.Status().Update(context.TODO(), adb); err != nil { + return false, err + } + + adb.Spec = ociADB.Spec + if err := r.KubeClient.Update(context.TODO(), adb); err != nil { + return false, err + } + return true, nil + } + + // Special case: if the lifecycleState is changed, it might have to exit the reconcile in some cases. + sent, exit, err := r.validateDesiredLifecycleState(logger, adb, difADB, ociADB) if err != nil { return false, err } + if sent { + return exit, nil + } - if ociReqSent { - return true, nil + validations := []func(logr.Logger, *dbv1alpha1.AutonomousDatabase, *dbv1alpha1.AutonomousDatabase, *dbv1alpha1.AutonomousDatabase) (bool, error){ + r.validateGeneralFields, + r.validateAdminPassword, + r.validateDbWorkload, + r.validateLicenseModel, + r.validateScalingFields, + r.validateGeneralNetworkAccess, + } + + for _, op := range validations { + sent, err := op(logger, adb, difADB, ociADB) + if err != nil { + return false, err + } + + if sent { + return false, nil + } } } @@ -666,13 +737,19 @@ func (r *AutonomousDatabaseReconciler) validateGeneralFields( return false, nil } - logger.WithName("validateGeneralFields").Info("Sending UpdateAutonomousDatabase request to OCI") + if ociADB.Status.LifecycleState != database.AutonomousDatabaseLifecycleStateAvailable { + return false, nil + } + + l := logger.WithName("validateGeneralFields") + + l.Info("Sending UpdateAutonomousDatabase request to OCI") resp, err := r.dbService.UpdateAutonomousDatabaseGeneralFields(*adb.Spec.Details.AutonomousDatabaseOCID, difADB) if err != nil { return false, err } - adb.UpdateStatusFromOCIADB(resp.AutonomousDatabase) + adb.UpdateFromOCIADB(resp.AutonomousDatabase) return true, nil } @@ -689,13 +766,21 @@ func (r *AutonomousDatabaseReconciler) validateAdminPassword( return false, nil } - logger.WithName("validateAdminPassword").Info("Sending UpdateAutonomousDatabase request to OCI") + if ociADB.Status.LifecycleState != database.AutonomousDatabaseLifecycleStateAvailable { + return false, nil + } + + l := logger.WithName("validateAdminPassword") + + l.Info("Sending UpdateAutonomousDatabase request to OCI") resp, err := r.dbService.UpdateAutonomousDatabaseAdminPassword(*adb.Spec.Details.AutonomousDatabaseOCID, difADB) if err != nil { return false, err } - adb.UpdateStatusFromOCIADB(resp.AutonomousDatabase) + adb.UpdateFromOCIADB(resp.AutonomousDatabase) + // Update the admin password fields because they are missing in the ociADB + adb.Spec.Details.AdminPassword = difADB.Spec.Details.AdminPassword return true, nil } @@ -710,13 +795,19 @@ func (r *AutonomousDatabaseReconciler) validateDbWorkload( return false, nil } - logger.WithName("validateDbWorkload").Info("Sending UpdateAutonomousDatabase request to OCI") + if ociADB.Status.LifecycleState != database.AutonomousDatabaseLifecycleStateAvailable { + return false, nil + } + + l := logger.WithName("validateDbWorkload") + + l.Info("Sending UpdateAutonomousDatabase request to OCI") resp, err := r.dbService.UpdateAutonomousDatabaseDBWorkload(*adb.Spec.Details.AutonomousDatabaseOCID, difADB) if err != nil { return false, err } - adb.UpdateStatusFromOCIADB(resp.AutonomousDatabase) + adb.UpdateFromOCIADB(resp.AutonomousDatabase) return true, nil } @@ -731,13 +822,19 @@ func (r *AutonomousDatabaseReconciler) validateLicenseModel( return false, nil } - logger.WithName("validateLicenseModel").Info("Sending UpdateAutonomousDatabase request to OCI") + if ociADB.Status.LifecycleState != database.AutonomousDatabaseLifecycleStateAvailable { + return false, nil + } + + l := logger.WithName("validateLicenseModel") + + l.Info("Sending UpdateAutonomousDatabase request to OCI") resp, err := r.dbService.UpdateAutonomousDatabaseLicenseModel(*adb.Spec.Details.AutonomousDatabaseOCID, difADB) if err != nil { return false, err } - adb.UpdateStatusFromOCIADB(resp.AutonomousDatabase) + adb.UpdateFromOCIADB(resp.AutonomousDatabase) return true, nil } @@ -754,13 +851,19 @@ func (r *AutonomousDatabaseReconciler) validateScalingFields( return false, nil } - logger.WithName("validateScalingFields").Info("Sending UpdateAutonomousDatabase request to OCI") + if ociADB.Status.LifecycleState != database.AutonomousDatabaseLifecycleStateAvailable { + return false, nil + } + + l := logger.WithName("validateScalingFields") + + l.Info("Sending UpdateAutonomousDatabase request to OCI") resp, err := r.dbService.UpdateAutonomousDatabaseScalingFields(*adb.Spec.Details.AutonomousDatabaseOCID, difADB) if err != nil { return false, err } - adb.UpdateStatusFromOCIADB(resp.AutonomousDatabase) + adb.UpdateFromOCIADB(resp.AutonomousDatabase) return true, nil } @@ -769,10 +872,20 @@ func (r *AutonomousDatabaseReconciler) validateDesiredLifecycleState( logger logr.Logger, adb *dbv1alpha1.AutonomousDatabase, difADB *dbv1alpha1.AutonomousDatabase, - ociADB *dbv1alpha1.AutonomousDatabase) (sent bool, err error) { + ociADB *dbv1alpha1.AutonomousDatabase) (sent bool, exit bool, err error) { if difADB.Spec.Details.LifecycleState == "" { - return false, nil + return false, false, nil + } + + if difADB.Spec.Details.LifecycleState == database.AutonomousDatabaseLifecycleStateTerminated { + // OCI only allows terminate operation when the ADB is in an valid state, otherwise requeue the reconcile. + if !dbv1alpha1.ValidADBTerminateState(adb.Status.LifecycleState) { + return false, false, nil + } + } else if dbv1alpha1.IsADBIntermediateState(ociADB.Status.LifecycleState) { + // Other lifecycle management operation; requeue the reconcile if it's in an intermediate state + return false, false, nil } l := logger.WithName("validateDesiredLifecycleState") @@ -783,7 +896,7 @@ func (r *AutonomousDatabaseReconciler) validateDesiredLifecycleState( resp, err := r.dbService.StartAutonomousDatabase(*adb.Spec.Details.AutonomousDatabaseOCID) if err != nil { - return false, err + return false, false, err } adb.Status.LifecycleState = resp.LifecycleState @@ -792,7 +905,7 @@ func (r *AutonomousDatabaseReconciler) validateDesiredLifecycleState( resp, err := r.dbService.StopAutonomousDatabase(*adb.Spec.Details.AutonomousDatabaseOCID) if err != nil { - return false, err + return false, false, err } adb.Status.LifecycleState = resp.LifecycleState @@ -801,15 +914,22 @@ func (r *AutonomousDatabaseReconciler) validateDesiredLifecycleState( _, err := r.dbService.DeleteAutonomousDatabase(*adb.Spec.Details.AutonomousDatabaseOCID) if err != nil { - return false, err + return false, false, err } adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateTerminating + + // The controller allows terminate during some intermediate states. + // Exit the reconcile because there is already another ongoing reconcile. + if dbv1alpha1.IsADBIntermediateState(ociADB.Status.LifecycleState) { + l.Info("Terminating an ADB which is in an intermediate state; exit reconcile") + return true, true, nil + } default: - return false, errors.New("unknown lifecycleState") + return false, false, errors.New("unknown lifecycleState") } - return true, nil + return true, false, nil } // The logic of updating the network access configurations is as follows: @@ -845,6 +965,10 @@ func (r *AutonomousDatabaseReconciler) validateGeneralNetworkAccess( return false, nil } + if ociADB.Status.LifecycleState != database.AutonomousDatabaseLifecycleStateAvailable { + return false, nil + } + l := logger.WithName("validateGeneralNetworkAccess") if !*adb.Spec.Details.IsDedicated { @@ -954,7 +1078,9 @@ func (r *AutonomousDatabaseReconciler) validateGeneralNetworkAccess( // Set the mTLS to true but not changing the spec func (r *AutonomousDatabaseReconciler) setMTLSRequired(logger logr.Logger, adb *dbv1alpha1.AutonomousDatabase) error { - logger.WithName("setMTLSRequired").Info("Sending request to OCI to set IsMtlsConnectionRequired to true") + l := logger.WithName("setMTLSRequired") + + l.Info("Sending request to OCI to set IsMtlsConnectionRequired to true") adb.Spec.Details.NetworkAccess.IsMTLSConnectionRequired = common.Bool(true) @@ -963,7 +1089,7 @@ func (r *AutonomousDatabaseReconciler) setMTLSRequired(logger logr.Logger, adb * return err } - adb.UpdateStatusFromOCIADB(resp.AutonomousDatabase) + adb.UpdateFromOCIADB(resp.AutonomousDatabase) return nil } @@ -978,14 +1104,16 @@ func (r *AutonomousDatabaseReconciler) validateMTLS( return false, nil } - logger.WithName("validateMTLS").Info("Sending request to OCI to configure IsMtlsConnectionRequired") + l := logger.WithName("validateMTLS") + + l.Info("Sending request to OCI to configure IsMtlsConnectionRequired") resp, err := r.dbService.UpdateNetworkAccessMTLS(*adb.Spec.Details.AutonomousDatabaseOCID, difADB) if err != nil { return false, err } - adb.UpdateStatusFromOCIADB(resp.AutonomousDatabase) + adb.UpdateFromOCIADB(resp.AutonomousDatabase) return true, nil } @@ -997,14 +1125,16 @@ func (r *AutonomousDatabaseReconciler) setNetworkAccessPublic(logger logr.Logger adb.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs = nil adb.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID = nil - logger.WithName("setNetworkAccessPublic").Info("Sending request to OCI to configure network access options to PUBLIC") + l := logger.WithName("setNetworkAccessPublic") + + l.Info("Sending request to OCI to configure network access options to PUBLIC") resp, err := r.dbService.UpdateNetworkAccessPublic(lastAcessType, *adb.Spec.Details.AutonomousDatabaseOCID) if err != nil { return err } - adb.UpdateStatusFromOCIADB(resp.AutonomousDatabase) + adb.UpdateFromOCIADB(resp.AutonomousDatabase) return nil } @@ -1024,14 +1154,16 @@ func (r *AutonomousDatabaseReconciler) validateNetworkAccess( return false, nil } - logger.WithName("validateNetworkAccess").Info("Sending request to OCI to configure network access options") + l := logger.WithName("validateNetworkAccess") + + l.Info("Sending request to OCI to configure network access options") resp, err := r.dbService.UpdateNetworkAccess(*adb.Spec.Details.AutonomousDatabaseOCID, difADB) if err != nil { return false, err } - adb.UpdateStatusFromOCIADB(resp.AutonomousDatabase) + adb.UpdateFromOCIADB(resp.AutonomousDatabase) return true, nil } diff --git a/docs/adb/ADB_PREREQUISITES.md b/docs/adb/ADB_PREREQUISITES.md index 56c47445..be53afe5 100644 --- a/docs/adb/ADB_PREREQUISITES.md +++ b/docs/adb/ADB_PREREQUISITES.md @@ -46,7 +46,9 @@ After creating the ConfigMap and the Secret, use their names as the values of `o ## Authorized with Instance Principal -Instance principal authorization enables the operator to make API calls from an instance (that is, a node) without requiring the `ociConfigMap`, and `ociSecret` attributes in the `.yaml` file. This approach applies only to instances that are running in the Oracle Cloud Infrastructure (OCI). In general, you will have to: +Instance principal authorization enables the operator to make API calls from an instance (that is, a node) without requiring the `ociConfigMap`, and `ociSecret` attributes in the `.yaml` file. This approach applies only to instances that are running in the Oracle Cloud Infrastructure (OCI). In addition, this approach grants permissions to the nodes that matche the rules, which means that all the pods in the nodes can make the service calls. + +To set up the instance princials, you will have to: * [Define dynamic group that includes the nodes in which the operator runs](#define-dynamic-group) * [Define policies that grant to the dynamic group the required permissions for the operator to its OCI interactions](#define-policies) diff --git a/go.mod b/go.mod index f779db45..50ae07e5 100644 --- a/go.mod +++ b/go.mod @@ -8,10 +8,10 @@ require ( github.com/onsi/gomega v1.19.0 github.com/oracle/oci-go-sdk/v63 v63.0.0 gopkg.in/yaml.v2 v2.4.0 - k8s.io/api v0.23.5 - k8s.io/apimachinery v0.23.5 - k8s.io/client-go v0.23.5 - sigs.k8s.io/controller-runtime v0.11.0 + k8s.io/api v0.23.6 + k8s.io/apimachinery v0.23.6 + k8s.io/client-go v0.23.6 + sigs.k8s.io/controller-runtime v0.11.2 sigs.k8s.io/yaml v1.3.0 ) @@ -58,8 +58,8 @@ require ( google.golang.org/protobuf v1.27.1 // indirect gopkg.in/inf.v0 v0.9.1 // indirect gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b // indirect - k8s.io/apiextensions-apiserver v0.23.3 // indirect - k8s.io/component-base v0.23.3 // indirect + k8s.io/apiextensions-apiserver v0.23.5 // indirect + k8s.io/component-base v0.23.5 // indirect k8s.io/klog/v2 v2.40.1 // indirect k8s.io/kube-openapi v0.0.0-20220124234850-424119656bbf // indirect k8s.io/utils v0.0.0-20220127004650-9b3446523e65 // indirect diff --git a/go.sum b/go.sum index 8a2308c2..d08c77ba 100644 --- a/go.sum +++ b/go.sum @@ -1017,28 +1017,21 @@ honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWh honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg= honnef.co/go/tools v0.0.1-2020.1.3/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k= honnef.co/go/tools v0.0.1-2020.1.4/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k= -k8s.io/api v0.23.0/go.mod h1:8wmDdLBHBNxtOIytwLstXt5E9PddnZb0GaMcqsvDBpg= -k8s.io/api v0.23.3/go.mod h1:w258XdGyvCmnBj/vGzQMj6kzdufJZVUwEM1U2fRJwSQ= -k8s.io/api v0.23.5 h1:zno3LUiMubxD/V1Zw3ijyKO3wxrhbUF1Ck+VjBvfaoA= k8s.io/api v0.23.5/go.mod h1:Na4XuKng8PXJ2JsploYYrivXrINeTaycCGcYgF91Xm8= -k8s.io/apiextensions-apiserver v0.23.0/go.mod h1:xIFAEEDlAZgpVBl/1VSjGDmLoXAWRG40+GsWhKhAxY4= -k8s.io/apiextensions-apiserver v0.23.3 h1:JvPJA7hSEAqMRteveq4aj9semilAZYcJv+9HHFWfUdM= -k8s.io/apiextensions-apiserver v0.23.3/go.mod h1:/ZpRXdgKZA6DvIVPEmXDCZJN53YIQEUDF+hrpIQJL38= -k8s.io/apimachinery v0.23.0/go.mod h1:fFCTTBKvKcwTPFzjlcxp91uPFZr+JA0FubU4fLzzFYc= -k8s.io/apimachinery v0.23.3/go.mod h1:BEuFMMBaIbcOqVIJqNZJXGFTP4W6AycEpb5+m/97hrM= -k8s.io/apimachinery v0.23.5 h1:Va7dwhp8wgkUPWsEXk6XglXWU4IKYLKNlv8VkX7SDM0= +k8s.io/api v0.23.6 h1:yOK34wbYECH4RsJbQ9sfkFK3O7f/DUHRlzFehkqZyVw= +k8s.io/api v0.23.6/go.mod h1:1kFaYxGCFHYp3qd6a85DAj/yW8aVD6XLZMqJclkoi9g= +k8s.io/apiextensions-apiserver v0.23.5 h1:5SKzdXyvIJKu+zbfPc3kCbWpbxi+O+zdmAJBm26UJqI= +k8s.io/apiextensions-apiserver v0.23.5/go.mod h1:ntcPWNXS8ZPKN+zTXuzYMeg731CP0heCTl6gYBxLcuQ= k8s.io/apimachinery v0.23.5/go.mod h1:BEuFMMBaIbcOqVIJqNZJXGFTP4W6AycEpb5+m/97hrM= -k8s.io/apiserver v0.23.0/go.mod h1:Cec35u/9zAepDPPFyT+UMrgqOCjgJ5qtfVJDxjZYmt4= -k8s.io/apiserver v0.23.3/go.mod h1:3HhsTmC+Pn+Jctw+Ow0LHA4dQ4oXrQ4XJDzrVDG64T4= -k8s.io/client-go v0.23.0/go.mod h1:hrDnpnK1mSr65lHHcUuIZIXDgEbzc7/683c6hyG4jTA= -k8s.io/client-go v0.23.3/go.mod h1:47oMd+YvAOqZM7pcQ6neJtBiFH7alOyfunYN48VsmwE= -k8s.io/client-go v0.23.5 h1:zUXHmEuqx0RY4+CsnkOn5l0GU+skkRXKGJrhmE2SLd8= +k8s.io/apimachinery v0.23.6 h1:RH1UweWJkWNTlFx0D8uxOpaU1tjIOvVVWV/bu5b3/NQ= +k8s.io/apimachinery v0.23.6/go.mod h1:BEuFMMBaIbcOqVIJqNZJXGFTP4W6AycEpb5+m/97hrM= +k8s.io/apiserver v0.23.5/go.mod h1:7wvMtGJ42VRxzgVI7jkbKvMbuCbVbgsWFT7RyXiRNTw= k8s.io/client-go v0.23.5/go.mod h1:flkeinTO1CirYgzMPRWxUCnV0G4Fbu2vLhYCObnt/r4= -k8s.io/code-generator v0.23.0/go.mod h1:vQvOhDXhuzqiVfM/YHp+dmg10WDZCchJVObc9MvowsE= -k8s.io/code-generator v0.23.3/go.mod h1:S0Q1JVA+kSzTI1oUvbKAxZY/DYbA/ZUb4Uknog12ETk= -k8s.io/component-base v0.23.0/go.mod h1:DHH5uiFvLC1edCpvcTDV++NKULdYYU6pR9Tt3HIKMKI= -k8s.io/component-base v0.23.3 h1:q+epprVdylgecijVGVdf4MbizEL2feW4ssd7cdo6LVY= -k8s.io/component-base v0.23.3/go.mod h1:1Smc4C60rWG7d3HjSYpIwEbySQ3YWg0uzH5a2AtaTLg= +k8s.io/client-go v0.23.6 h1:7h4SctDVQAQbkHQnR4Kzi7EyUyvla5G1pFWf4+Od7hQ= +k8s.io/client-go v0.23.6/go.mod h1:Umt5icFOMLV/+qbtZ3PR0D+JA6lvvb3syzodv4irpK4= +k8s.io/code-generator v0.23.5/go.mod h1:S0Q1JVA+kSzTI1oUvbKAxZY/DYbA/ZUb4Uknog12ETk= +k8s.io/component-base v0.23.5 h1:8qgP5R6jG1BBSXmRYW+dsmitIrpk8F/fPEvgDenMCCE= +k8s.io/component-base v0.23.5/go.mod h1:c5Nq44KZyt1aLl0IpHX82fhsn84Sb0jjzwjpcA42bY0= k8s.io/gengo v0.0.0-20210813121822-485abfe95c7c/go.mod h1:FiNAH4ZV3gBg2Kwh89tzAEV2be7d5xI0vBa/VySYy3E= k8s.io/klog/v2 v2.0.0/go.mod h1:PBfzABfn139FHAV07az/IF9Wp1bkk3vpT2XSJ76fSDE= k8s.io/klog/v2 v2.2.0/go.mod h1:Od+F08eJP+W3HUb4pSrPpgp9DGU4GzlpG/TmITuYh/Y= @@ -1049,23 +1042,19 @@ k8s.io/kube-openapi v0.0.0-20211115234752-e816edb12b65/go.mod h1:sX9MT8g7NVZM5lV k8s.io/kube-openapi v0.0.0-20220124234850-424119656bbf h1:M9XBsiMslw2lb2ZzglC0TOkBPK5NQi0/noUrdnoFwUg= k8s.io/kube-openapi v0.0.0-20220124234850-424119656bbf/go.mod h1:sX9MT8g7NVZM5lVL/j8QyCCJe8YSMW30QvGZWaCIDIk= k8s.io/utils v0.0.0-20210802155522-efc7438f0176/go.mod h1:jPW/WVKK9YHAvNhRxK0md/EJ228hCsBRufyofKtW8HA= -k8s.io/utils v0.0.0-20210930125809-cb0fa318a74b/go.mod h1:jPW/WVKK9YHAvNhRxK0md/EJ228hCsBRufyofKtW8HA= k8s.io/utils v0.0.0-20211116205334-6203023598ed/go.mod h1:jPW/WVKK9YHAvNhRxK0md/EJ228hCsBRufyofKtW8HA= k8s.io/utils v0.0.0-20220127004650-9b3446523e65 h1:ONWS0Wgdg5wRiQIAui7L/023aC9+IxrIrydY7l8llsE= k8s.io/utils v0.0.0-20220127004650-9b3446523e65/go.mod h1:jPW/WVKK9YHAvNhRxK0md/EJ228hCsBRufyofKtW8HA= rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8= rsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0= rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA= -sigs.k8s.io/apiserver-network-proxy/konnectivity-client v0.0.25/go.mod h1:Mlj9PNLmG9bZ6BHFwFKDo5afkpWyUISkb9Me0GnK66I= -sigs.k8s.io/apiserver-network-proxy/konnectivity-client v0.0.27/go.mod h1:tq2nT0Kx7W+/f2JVE+zxYtUhdjuELJkVpNz+x/QN5R4= -sigs.k8s.io/controller-runtime v0.11.0 h1:DqO+c8mywcZLFJWILq4iktoECTyn30Bkj0CwgqMpZWQ= -sigs.k8s.io/controller-runtime v0.11.0/go.mod h1:KKwLiTooNGu+JmLZGn9Sl3Gjmfj66eMbCQznLP5zcqA= +sigs.k8s.io/apiserver-network-proxy/konnectivity-client v0.0.30/go.mod h1:fEO7lRTdivWO2qYVCVG7dEADOMo/MLDCVr8So2g88Uw= +sigs.k8s.io/controller-runtime v0.11.2 h1:H5GTxQl0Mc9UjRJhORusqfJCIjBO8UtUxGggCwL1rLA= +sigs.k8s.io/controller-runtime v0.11.2/go.mod h1:P6QCzrEjLaZGqHsfd+os7JQ+WFZhvB8MRFsn4dWF7O4= sigs.k8s.io/json v0.0.0-20211020170558-c049b76a60c6/go.mod h1:p4QtZmO4uMYipTQNzagwnNoseA6OxSUutVw05NhYDRs= sigs.k8s.io/json v0.0.0-20211208200746-9f7c6b3444d2 h1:kDi4JBNAsJWfz1aEXhO8Jg87JJaPNLh5tIzYHgStQ9Y= sigs.k8s.io/json v0.0.0-20211208200746-9f7c6b3444d2/go.mod h1:B+TnT182UBxE84DiCz4CVE26eOSDAeYCpfDnC2kdKMY= sigs.k8s.io/structured-merge-diff/v4 v4.0.2/go.mod h1:bJZC9H9iH24zzfZ/41RGcq60oK1F7G282QMXDPYydCw= -sigs.k8s.io/structured-merge-diff/v4 v4.1.2/go.mod h1:j/nl6xW8vLS49O8YvXW1ocPhZawJtm+Yrr7PPRQ0Vg4= -sigs.k8s.io/structured-merge-diff/v4 v4.2.0/go.mod h1:j/nl6xW8vLS49O8YvXW1ocPhZawJtm+Yrr7PPRQ0Vg4= sigs.k8s.io/structured-merge-diff/v4 v4.2.1 h1:bKCqE9GvQ5tiVHn5rfn1r+yao3aLQEaLzkkmAkf+A6Y= sigs.k8s.io/structured-merge-diff/v4 v4.2.1/go.mod h1:j/nl6xW8vLS49O8YvXW1ocPhZawJtm+Yrr7PPRQ0Vg4= sigs.k8s.io/yaml v1.2.0/go.mod h1:yfXDCHCao9+ENCvLSE62v9VSji2MKu5jeNfTrofGhJc= diff --git a/test/e2e/autonomousdatabase_controller_bind_test.go b/test/e2e/autonomousdatabase_controller_bind_test.go index e8185e77..9f2176eb 100644 --- a/test/e2e/autonomousdatabase_controller_bind_test.go +++ b/test/e2e/autonomousdatabase_controller_bind_test.go @@ -130,7 +130,7 @@ var _ = Describe("test ADB binding with hardLink=true", func() { It("Should download an instance wallet using the password from K8s Secret "+SharedWalletPassSecretName, e2ebehavior.AssertWallet(&k8sClient, &adbLookupKey)) - It("should update ADB", e2ebehavior.UpdateAndAssertDetails(&k8sClient, &dbClient, &adbLookupKey)) + It("should update ADB", e2ebehavior.UpdateAndAssertDetails(&k8sClient, &dbClient, &adbLookupKey, SharedNewAdminPassSecretName, &SharedPlainTextNewAdminPassword, &SharedPlainTextWalletPassword)) It("Should stop ADB", e2ebehavior.UpdateAndAssertState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateStopped)) diff --git a/test/e2e/behavior/shared_behaviors.go b/test/e2e/behavior/shared_behaviors.go index 8a29c504..8c84d3e4 100644 --- a/test/e2e/behavior/shared_behaviors.go +++ b/test/e2e/behavior/shared_behaviors.go @@ -78,6 +78,7 @@ var ( Eventually = gomega.Eventually Equal = gomega.Equal Succeed = gomega.Succeed + HaveOccurred = gomega.HaveOccurred BeNumerically = gomega.BeNumerically BeTrue = gomega.BeTrue changeTimeout = time.Second * 300 @@ -226,7 +227,7 @@ func compareStringMap(obj1 map[string]string, obj2 map[string]string) bool { } // UpdateDetails updates spec.details from local resource and OCI -func UpdateDetails(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName) func() *dbv1alpha1.AutonomousDatabase { +func UpdateDetails(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName, newSecretName string, newAdminPassword *string) func() *dbv1alpha1.AutonomousDatabase { return func() *dbv1alpha1.AutonomousDatabase { // Considering that there are at most two update requests will be sent during the update // From the observation per request takes ~3mins to finish @@ -234,6 +235,7 @@ func UpdateDetails(k8sClient *client.Client, dbClient *database.DatabaseClient, Expect(k8sClient).NotTo(BeNil()) Expect(dbClient).NotTo(BeNil()) Expect(adbLookupKey).NotTo(BeNil()) + Expect(newAdminPassword).NotTo(BeNil()) derefK8sClient := *k8sClient derefDBClient := *dbClient @@ -277,6 +279,7 @@ func UpdateDetails(k8sClient *client.Client, dbClient *database.DatabaseClient, expectedADB.Spec.Details.DisplayName = common.String(newDisplayName) expectedADB.Spec.Details.CPUCoreCount = common.Int(newCPUCoreCount) expectedADB.Spec.Details.FreeformTags = map[string]string{newKey: newVal} + expectedADB.Spec.Details.AdminPassword.K8sSecret.Name = common.String(newSecretName) Expect(derefK8sClient.Update(context.TODO(), expectedADB)).To(Succeed()) @@ -285,7 +288,10 @@ func UpdateDetails(k8sClient *client.Client, dbClient *database.DatabaseClient, } // AssertADBDetails asserts the changes in spec.details -func AssertADBDetails(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName, expectedADB *dbv1alpha1.AutonomousDatabase) func() { +func AssertADBDetails(k8sClient *client.Client, + dbClient *database.DatabaseClient, + adbLookupKey *types.NamespacedName, + expectedADB *dbv1alpha1.AutonomousDatabase) func() { return func() { // Considering that there are at most two update requests will be sent during the update // From the observation per request takes ~3mins to finish @@ -403,6 +409,37 @@ func TestNetworkAccessRestricted(k8sClient *client.Client, dbClient *database.Da } } +/* Runs a script that connects to an ADB */ +func AssertAdminPassword(dbClient *database.DatabaseClient, databaseOCID *string, tnsEntry *string, adminPassword *string, walletPassword *string) error { + By("Downloading wallet zip") + walletZip, err := e2eutil.DownloadWalletZip(*dbClient, databaseOCID, walletPassword) + if err != nil { + fmt.Fprint(GinkgoWriter, err) + panic(err) + } + fmt.Fprint(GinkgoWriter, walletZip+" successfully downloaded.\n") + + By("Installing SQLcl") + if _, err := os.Stat("sqlcl-latest.zip"); errors.Is(err, os.ErrNotExist) { + cmd := exec.Command("wget", "https://download.oracle.com/otn_software/java/sqldeveloper/sqlcl-latest.zip") + _, err = cmd.Output() + Expect(err).To(BeNil()) + cmd = exec.Command("unzip", "sqlcl-latest.zip") + _, err = cmd.Output() + Expect(err).To(BeNil()) + } + + proxy := os.Getenv("HTTP_PROXY") + + By("Verify the adb connection") + cmd := exec.Command("./sqlcl/bin/sql", "/nolog", "@verify_connection.sql", proxy, walletZip, *adminPassword, strings.ToLower(*tnsEntry)) + stdout, err := cmd.Output() + + fmt.Fprint(GinkgoWriter, string(stdout)) + + return err +} + func TestNetworkAccessPrivate(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName, isMTLSConnectionRequired bool, subnetOCID *string, nsgOCIDs *string) func() { return func() { Expect(*subnetOCID).ToNot(Equal("")) @@ -456,11 +493,19 @@ func TestNetworkAccess(k8sClient *client.Client, dbClient *database.DatabaseClie } } -// UpdateAndAssertDetails changes the displayName from "foo" to "foo_new", and scale the cpuCoreCount to 2 -func UpdateAndAssertDetails(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName) func() { +// UpdateAndAssertDetails changes the below fields: +// displayName: "bar" -> "bar_new" +// adminPassword: "foo" -> "foo_new", +// cpuCoreCount: from 1 to 2, or from 2 to 1 +func UpdateAndAssertDetails(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName, newSecretName string, newAdminPassword *string, walletPassword *string) func() { return func() { - expectedADB := UpdateDetails(k8sClient, dbClient, adbLookupKey)() + expectedADB := UpdateDetails(k8sClient, dbClient, adbLookupKey, newSecretName, newAdminPassword)() AssertADBDetails(k8sClient, dbClient, adbLookupKey, expectedADB)() + + ocid := expectedADB.Spec.Details.AutonomousDatabaseOCID + tnsEntry := *expectedADB.Spec.Details.DbName + "_high" + err := AssertAdminPassword(dbClient, ocid, &tnsEntry, newAdminPassword, walletPassword) + Expect(err).ShouldNot(HaveOccurred()) } } @@ -644,16 +689,20 @@ func ConfigureADBBackup(dbClient *database.DatabaseClient, databaseOCID *string, fmt.Fprint(GinkgoWriter, walletZip+" successfully downloaded.\n") By("Installing SQLcl") - cmd := exec.Command("wget", "https://download.oracle.com/otn_software/java/sqldeveloper/sqlcl-latest.zip") - stdout, err := cmd.Output() - cmd = exec.Command("unzip", "sqlcl-latest.zip") - stdout, err = cmd.Output() + if _, err := os.Stat("sqlcl-latest.zip"); errors.Is(err, os.ErrNotExist) { + cmd := exec.Command("wget", "https://download.oracle.com/otn_software/java/sqldeveloper/sqlcl-latest.zip") + _, err = cmd.Output() + Expect(err).To(BeNil()) + cmd = exec.Command("unzip", "sqlcl-latest.zip") + _, err = cmd.Output() + Expect(err).To(BeNil()) + } proxy := os.Getenv("HTTP_PROXY") By("Configuring adb backup bucket") - cmd = exec.Command("./sqlcl/bin/sql", "/nolog", "@backup.sql", proxy, walletZip, *adminPassword, strings.ToLower(*tnsEntry), *bucket, *ociUser, *authToken) - stdout, err = cmd.Output() + cmd := exec.Command("./sqlcl/bin/sql", "/nolog", "@backup.sql", proxy, walletZip, *adminPassword, strings.ToLower(*tnsEntry), *bucket, *ociUser, *authToken) + stdout, err := cmd.Output() fmt.Fprint(GinkgoWriter, string(stdout)) diff --git a/test/e2e/suite_test.go b/test/e2e/suite_test.go index 9e0a37f6..a849a664 100644 --- a/test/e2e/suite_test.go +++ b/test/e2e/suite_test.go @@ -107,6 +107,7 @@ const ADBNamespace string = "default" var SharedOCIConfigMapName = "oci-cred" var SharedOCISecretName = "oci-privatekey" var SharedPlainTextAdminPassword = "Welcome_1234" +var SharedPlainTextNewAdminPassword = "Welcome_1234_new" var SharedPlainTextWalletPassword = "Welcome_1234" var SharedCompartmentOCID string @@ -121,7 +122,8 @@ var SharedAuthToken string var SharedOciUser string const SharedAdminPassSecretName string = "adb-admin-password" -const SharedWalletPassSecretName = "adb-wallet-password" +const SharedNewAdminPassSecretName string = "new-adb-admin-password" +const SharedWalletPassSecretName string = "adb-wallet-password" func TestAPIs(t *testing.T) { gomega.RegisterFailHandler(ginkgo.Fail) @@ -260,6 +262,15 @@ var _ = BeforeSuite(func() { Expect(k8sClient.Create(context.TODO(), adminSecret)).To(Succeed()) }) + By("Creating a k8s secret to hold new admin password", func() { + data := map[string]string{ + SharedNewAdminPassSecretName: SharedPlainTextNewAdminPassword, + } + newAdminSecret, err := e2eutil.CreateKubeSecret(ADBNamespace, SharedNewAdminPassSecretName, data) + Expect(err).ToNot(HaveOccurred()) + Expect(k8sClient.Create(context.TODO(), newAdminSecret)).To(Succeed()) + }) + By("Creating a k8s secret to hold wallet password", func() { data := map[string]string{ SharedWalletPassSecretName: SharedPlainTextWalletPassword, diff --git a/test/e2e/verify_connection.sql b/test/e2e/verify_connection.sql new file mode 100644 index 00000000..85ded8ad --- /dev/null +++ b/test/e2e/verify_connection.sql @@ -0,0 +1,4 @@ +set cloudconfig -proxy=&1 &2 +connect ADMIN/&3@&4 +select 1 from dual; +exit \ No newline at end of file From 4266275b9f322c31ceef13477e838ad18052ed24 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Mon, 9 May 2022 11:12:28 -0400 Subject: [PATCH 368/628] fix terminate operation --- .../v1alpha1/adbfamily_common_utils.go | 28 +++++++++++++++++-- .../v1alpha1/autonomousdatabase_webhook.go | 8 ++++-- test/e2e/suite_test.go | 1 + 3 files changed, 32 insertions(+), 5 deletions(-) diff --git a/apis/database/v1alpha1/adbfamily_common_utils.go b/apis/database/v1alpha1/adbfamily_common_utils.go index 61377d5c..2dc6c69b 100644 --- a/apis/database/v1alpha1/adbfamily_common_utils.go +++ b/apis/database/v1alpha1/adbfamily_common_utils.go @@ -140,6 +140,25 @@ func traverse(lastSpec interface{}, curSpec interface{}) bool { } else { fieldChanged := hasChanged(lastField, curField) + // if fieldChanged { + // if curField.Kind() == reflect.Ptr { + // fmt.Printf("== field %s changed\n", field.Name) + // if lastField.IsZero() { + // fmt.Printf("=== lastField is nil\n") + // } else { + // fmt.Printf("=== lastField = %v\n", lastField.Elem().Interface()) + // } + // if curField.IsZero() { + // fmt.Printf("===== curField is nil\n") + // } else { + // fmt.Printf("===== curField = %v\n", curField.Elem().Interface()) + // } + // } else { + // fmt.Printf("=== lastField = %v\n", lastField.Interface()) + // fmt.Printf("===== curField = %v\n", curField.Interface()) + // } + // } + if fieldChanged && !changed { changed = true } @@ -231,10 +250,15 @@ func IsADBIntermediateState(state database.AutonomousDatabaseLifecycleStateEnum) func ValidADBTerminateState(state database.AutonomousDatabaseLifecycleStateEnum) bool { if state == database.AutonomousDatabaseLifecycleStateProvisioning || - state == database.AutonomousDatabaseLifecycleStateUpdating || - state == database.AutonomousDatabaseLifecycleStateScaleInProgress || + state == database.AutonomousDatabaseLifecycleStateAvailable || + state == database.AutonomousDatabaseLifecycleStateStopped || + state == database.AutonomousDatabaseLifecycleStateUnavailable || state == database.AutonomousDatabaseLifecycleStateRestoreInProgress || + state == database.AutonomousDatabaseLifecycleStateRestoreFailed || state == database.AutonomousDatabaseLifecycleStateBackupInProgress || + state == database.AutonomousDatabaseLifecycleStateScaleInProgress || + state == database.AutonomousDatabaseLifecycleStateAvailableNeedsAttention || + state == database.AutonomousDatabaseLifecycleStateUpdating || state == database.AutonomousDatabaseLifecycleStateMaintenanceInProgress || state == database.AutonomousDatabaseLifecycleStateRoleChangeInProgress || state == database.AutonomousDatabaseLifecycleStateUpgrading { diff --git a/apis/database/v1alpha1/autonomousdatabase_webhook.go b/apis/database/v1alpha1/autonomousdatabase_webhook.go index 5841fe5f..5b0e34f3 100644 --- a/apis/database/v1alpha1/autonomousdatabase_webhook.go +++ b/apis/database/v1alpha1/autonomousdatabase_webhook.go @@ -134,7 +134,7 @@ func (r *AutonomousDatabase) ValidateUpdate(old runtime.Object) error { return nil } - // cannot update when the old state is in intermediate, except for the terminate operatrion during valid lifecycleState + // cannot update when the old state is in intermediate, except for the change to the hardLink or the terminate operatrion during valid lifecycleState var copySpec *AutonomousDatabaseSpec = r.Spec.DeepCopy() specChanged, err := removeUnchangedFields(oldADB.Spec, copySpec) if err != nil { @@ -142,9 +142,11 @@ func (r *AutonomousDatabase) ValidateUpdate(old runtime.Object) error { field.Forbidden(field.NewPath("spec"), err.Error())) } + hardLinkChanged := copySpec.HardLink != nil + terminateOp := ValidADBTerminateState(oldADB.Status.LifecycleState) && copySpec.Details.LifecycleState == database.AutonomousDatabaseLifecycleStateTerminated - if specChanged && IsADBIntermediateState(oldADB.Status.LifecycleState) && !terminateOp { + if specChanged && IsADBIntermediateState(oldADB.Status.LifecycleState) && !terminateOp && !hardLinkChanged { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec"), "cannot change the spec when the lifecycleState is in an intermdeiate state")) @@ -159,7 +161,7 @@ func (r *AutonomousDatabase) ValidateUpdate(old runtime.Object) error { "autonomousDatabaseOCID cannot be modified")) } - // cannot change lifecycleState with other fields together + // cannot change lifecycleState with other fields together (except the oci config) var lifecycleChanged, otherFieldsChanged bool lifecycleChanged = oldADB.Spec.Details.LifecycleState != "" && oldADB.Spec.Details.LifecycleState != r.Spec.Details.LifecycleState diff --git a/test/e2e/suite_test.go b/test/e2e/suite_test.go index a849a664..3d98330b 100644 --- a/test/e2e/suite_test.go +++ b/test/e2e/suite_test.go @@ -83,6 +83,7 @@ var ( AfterSuite = ginkgo.AfterSuite Describe = ginkgo.Describe PDescribe = ginkgo.PDescribe + FDescribe = ginkgo.FDescribe AfterEach = ginkgo.AfterEach By = ginkgo.By It = ginkgo.It From d2d8d78be9a1169e921a5d8949d2cd686caae07c Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Mon, 9 May 2022 16:46:27 +0000 Subject: [PATCH 369/628] Update commons/oci/database.go --- commons/oci/database.go | 2 -- 1 file changed, 2 deletions(-) diff --git a/commons/oci/database.go b/commons/oci/database.go index 7d93bfba..2686a86d 100644 --- a/commons/oci/database.go +++ b/commons/oci/database.go @@ -254,8 +254,6 @@ func (d *databaseService) UpdateAutonomousDatabaseAdminPassword(adbOCID string, return resp, err } - d.logger.Info("==== test: new admin password = " + *adminPassword) - updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ AutonomousDatabaseId: common.String(adbOCID), UpdateAutonomousDatabaseDetails: database.UpdateAutonomousDatabaseDetails{ From 410b1819d9e4ce37c7a4a49377d4a0e857bdedb6 Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Tue, 10 May 2022 11:54:47 +0530 Subject: [PATCH 370/628] Resolve sidb not getting unhealthy issue, added log on enable ords schema command Signed-off-by: abhisbyk --- .../database/v1alpha1/zz_generated.deepcopy.go | 11 ++++++++--- commons/database/constants.go | 8 +++----- .../oraclerestdataservice_controller.go | 8 ++++---- .../singleinstancedatabase_controller.go | 18 +++++++++--------- 4 files changed, 24 insertions(+), 21 deletions(-) diff --git a/apis/database/v1alpha1/zz_generated.deepcopy.go b/apis/database/v1alpha1/zz_generated.deepcopy.go index 700da778..ef295aa1 100644 --- a/apis/database/v1alpha1/zz_generated.deepcopy.go +++ b/apis/database/v1alpha1/zz_generated.deepcopy.go @@ -1451,6 +1451,11 @@ func (in *OracleRestDataServiceList) DeepCopyObject() runtime.Object { // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *OracleRestDataServicePassword) DeepCopyInto(out *OracleRestDataServicePassword) { *out = *in + if in.KeepSecret != nil { + in, out := &in.KeepSecret, &out.KeepSecret + *out = new(bool) + **out = **in + } } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new OracleRestDataServicePassword. @@ -1504,9 +1509,9 @@ func (in *OracleRestDataServiceSpec) DeepCopyInto(out *OracleRestDataServiceSpec } } out.Image = in.Image - out.OrdsPassword = in.OrdsPassword - out.ApexPassword = in.ApexPassword - out.AdminPassword = in.AdminPassword + in.OrdsPassword.DeepCopyInto(&out.OrdsPassword) + in.ApexPassword.DeepCopyInto(&out.ApexPassword) + in.AdminPassword.DeepCopyInto(&out.AdminPassword) if in.RestEnableSchemas != nil { in, out := &in.RestEnableSchemas, &out.RestEnableSchemas *out = make([]OracleRestDataServiceRestEnableSchemas, len(*in)) diff --git a/commons/database/constants.go b/commons/database/constants.go index 9b2917c5..9cbbe4da 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -208,7 +208,7 @@ const GetSqlpatchStatusSQL string = "select status from dba_registry_sqlpatch or const GetSqlpatchVersionSQL string = "select SOURCE_VERSION || ':' || TARGET_VERSION as versions from dba_registry_sqlpatch order by action_time desc;" -const GetCheckpointFileCMD string = "find ${ORACLE_BASE}/oradata -name .${ORACLE_SID}${CHECKPOINT_FILE_EXTN} 2> /dev/null" +const GetCheckpointFileCMD string = "find ${ORACLE_BASE}/oradata -name .${ORACLE_SID}${CHECKPOINT_FILE_EXTN}" const GetEnterpriseEditionFileCMD string = "if [ -f ${ORACLE_BASE}/oradata/dbconfig/$ORACLE_SID/.docker_enterprise ]; then ls ${ORACLE_BASE}/oradata/dbconfig/$ORACLE_SID/.docker_enterprise; fi " @@ -317,7 +317,7 @@ const InitORDSCMD string = "if [ -f $ORDS_HOME/config/ords/defaults.xml ]; then "\nrm -f sqladmin.passwd" + "\numask 022" -const GetSessionInfoSQL string = "select s.sid || ',' || s.serial# as Info FROM v\\$session s, v\\$process p "+ +const GetSessionInfoSQL string = "select s.sid || ',' || s.serial# as Info FROM v\\$session s, v\\$process p " + "WHERE (s.username = 'ORDS_PUBLIC_USER' or s.username = 'C##_DBAPI_PDB_ADMIN' ) AND p.addr(+) = s.paddr;" const KillSessionSQL string = "alter system kill session '%[1]s';" @@ -336,7 +336,6 @@ const UninstallORDSCMD string = "\numask 177" + "\nrm -rf /opt/oracle/ords/config/ords/standalone" + "\nrm -rf /opt/oracle/ords/config/ords/apex" - const GetORDSStatus string = "curl -sSkv -k -X GET https://localhost:8443/ords/_/db-api/stable/metadata-catalog/" const ValidateAdminPassword string = "conn sys/%s@${ORACLE_SID} as sysdba\nshow user" @@ -390,7 +389,6 @@ const InitWalletCMD string = "if [ ! -f $ORACLE_BASE/oradata/.${ORACLE_SID}${CHE const InitPrebuiltDbCMD string = "if [ ! -d /mnt/oradata/${ORACLE_SID} -a -d $ORACLE_BASE/oradata/${ORACLE_SID} ]; then cp -v $ORACLE_BASE/oradata/.${ORACLE_SID}$CHECKPOINT_FILE_EXTN /mnt/oradata && " + " cp -vr $ORACLE_BASE/oradata/${ORACLE_SID} /mnt/oradata && cp -vr $ORACLE_BASE/oradata/dbconfig /mnt/oradata; fi " - const AlterSgaPgaCpuCMD string = "echo -e \"alter system set sga_target=%dM scope=both; \n alter system set pga_aggregate_target=%dM scope=both; \n alter system set cpu_count=%d; \" | %s " const AlterProcessesCMD string = "echo -e \"alter system set processes=%d scope=spfile; \" | %s && " + CreateChkFileCMD + " && " + @@ -410,7 +408,7 @@ const InstallApex string = "if [ -f /opt/oracle/oradata/${ORACLE_SID^^}/apex/ape const InstallApexRemote string = "if [ -e ${ORDS_HOME}/config/apex/apxsilentins.sql ]; then cd ${ORDS_HOME}/config/apex/ && echo -e \"@apxsilentins.sql SYSAUX SYSAUX TEMP /i/ %[2]s %[2]s %[2]s %[2]s\" | %[1]s; else echo \"Apex Folder doesn't exist\" ; fi ;" -const InstallApexInContainer string = "cd ${ORDS_HOME}/config/apex/ && echo -e \"@apxsilentins.sql SYSAUX SYSAUX TEMP /i/ %[1]s %[1]s %[1]s %[1]s;\n"+ +const InstallApexInContainer string = "cd ${ORDS_HOME}/config/apex/ && echo -e \"@apxsilentins.sql SYSAUX SYSAUX TEMP /i/ %[1]s %[1]s %[1]s %[1]s;\n" + "@apex_rest_config_core.sql;\n" + "exec APEX_UTIL.set_workspace(p_workspace => 'INTERNAL');\n" + "exec APEX_UTIL.EDIT_USER(p_user_id => APEX_UTIL.GET_USER_ID('ADMIN'), p_user_name => 'ADMIN', p_change_password_on_first_use => 'Y');\n" + diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 4c500c51..1296ea1b 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -242,8 +242,8 @@ func (r *OracleRestDataServiceReconciler) validate(m *dbapi.OracleRestDataServic } // If ORDS has no peristence specified, ensure SIDB has persistence configured - if m.Spec.Persistence.Size == "" && n.Spec.Persistence.AccessMode == "" { - eventMsgs = append(eventMsgs, "ORDS cannot be configured for database " + m.Spec.DatabaseRef + " that has no attached persistent volume") + if m.Spec.Persistence.Size == "" && n.Spec.Persistence.AccessMode == "" { + eventMsgs = append(eventMsgs, "ORDS cannot be configured for database "+m.Spec.DatabaseRef+" that has no attached persistent volume") } if m.Status.DatabaseRef != "" && m.Status.DatabaseRef != m.Spec.DatabaseRef { eventMsgs = append(eventMsgs, "databaseRef cannot be updated") @@ -484,7 +484,7 @@ func (r *OracleRestDataServiceReconciler) instantiatePodSpec(m *dbapi.OracleRest MatchExpressions: []metav1.LabelSelectorRequirement{{ Key: "app", Operator: metav1.LabelSelectorOpIn, - Values: []string{n.Name}, // Schedule on same host as DB Pod + Values: []string{n.Name}, // Schedule on same host as DB Pod }}, }, TopologyKey: "kubernetes.io/hostname", @@ -1399,7 +1399,7 @@ func (r *OracleRestDataServiceReconciler) restEnableSchemas(m *dbapi.OracleRestD strconv.FormatBool(m.Spec.RestEnableSchemas[i].Enable), urlMappingPattern, pdbName) // Create users,schemas and grant enableORDS for PDB - _, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", + _, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", fmt.Sprintf("echo -e \"%s\" | %s", enableORDSSchema, dbcommons.SQLPlusCLI)) if err != nil { log.Error(err, err.Error()) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 30f19507..d9350bc6 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -703,12 +703,12 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns Value: strings.ToUpper(m.Spec.Sid), }, { - Name: "WALLET_DIR", + Name: "WALLET_DIR", Value: func() string { - if m.Spec.Image.PrebuiltDB { - return "" // No wallets for prebuilt DB - } - return "/opt/oracle/oradata/dbconfig/$(ORACLE_SID)/.wallet" + if m.Spec.Image.PrebuiltDB { + return "" // No wallets for prebuilt DB + } + return "/opt/oracle/oradata/dbconfig/$(ORACLE_SID)/.wallet" }(), }, { @@ -1088,7 +1088,7 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplacePods(m *dbapi.SingleIn var gracePeriodSeconds int64 = 0 policy := metav1.DeletePropagationForeground r.Delete(ctx, &podsMarkedToBeDeleted[i], &client.DeleteOptions{ - GracePeriodSeconds: &gracePeriodSeconds, PropagationPolicy: &policy }) + GracePeriodSeconds: &gracePeriodSeconds, PropagationPolicy: &policy}) } if readyPod.Name != "" { @@ -1177,7 +1177,7 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplacePods(m *dbapi.SingleIn for i := 0; i < len(newAvailable); i++ { r.Log.Info("Pod status: ", "name", newAvailable[i].Name, "phase", newAvailable[i].Status.Phase) waitingReason := "" - if (len(newAvailable[i].Status.InitContainerStatuses) > 0) { + if len(newAvailable[i].Status.InitContainerStatuses) > 0 { waitingReason = newAvailable[i].Status.InitContainerStatuses[0].State.Waiting.Reason } else if len(newAvailable[i].Status.ContainerStatuses) > 0 { waitingReason = newAvailable[i].Status.ContainerStatuses[0].State.Waiting.Reason @@ -1196,7 +1196,7 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplacePods(m *dbapi.SingleIn } return requeueY, errors.New(eventMsg) } - if m.Spec.Replicas == 1 { + if m.Spec.Replicas == 1 { return requeueN, nil } return r.deletePods(ctx, req, m, oldAvailable, corev1.Pod{}, oldReplicasFound, 0) @@ -1457,7 +1457,7 @@ func (r *SingleInstanceDatabaseReconciler) validateDBReadiness(m *dbapi.SingleIn m.Status.Status = dbcommons.StatusCreating out, err := dbcommons.ExecCommand(r, r.Config, runningPod.Name, runningPod.Namespace, "", - ctx, req, false, "bash", "-c", dbcommons.GetCheckpointFileCMD) + ctx, req, false, "bash", "-c", dbcommons.GetCheckpointFileCMD, "2> /dev/null") if err != nil { r.Log.Info(err.Error()) } From 0047cf308042e38444b4b158fac82424f962eb17 Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Tue, 10 May 2022 14:02:25 +0530 Subject: [PATCH 371/628] Correcting the GetCheckpointFileCMD Signed-off-by: abhisbyk --- commons/database/constants.go | 2 +- controllers/database/singleinstancedatabase_controller.go | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/commons/database/constants.go b/commons/database/constants.go index 9cbbe4da..8cf98b77 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -208,7 +208,7 @@ const GetSqlpatchStatusSQL string = "select status from dba_registry_sqlpatch or const GetSqlpatchVersionSQL string = "select SOURCE_VERSION || ':' || TARGET_VERSION as versions from dba_registry_sqlpatch order by action_time desc;" -const GetCheckpointFileCMD string = "find ${ORACLE_BASE}/oradata -name .${ORACLE_SID}${CHECKPOINT_FILE_EXTN}" +const GetCheckpointFileCMD string = "find ${ORACLE_BASE}/oradata -maxdepth 1 -name .${ORACLE_SID}${CHECKPOINT_FILE_EXTN}" const GetEnterpriseEditionFileCMD string = "if [ -f ${ORACLE_BASE}/oradata/dbconfig/$ORACLE_SID/.docker_enterprise ]; then ls ${ORACLE_BASE}/oradata/dbconfig/$ORACLE_SID/.docker_enterprise; fi " diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index d9350bc6..ce006c7f 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1457,7 +1457,7 @@ func (r *SingleInstanceDatabaseReconciler) validateDBReadiness(m *dbapi.SingleIn m.Status.Status = dbcommons.StatusCreating out, err := dbcommons.ExecCommand(r, r.Config, runningPod.Name, runningPod.Namespace, "", - ctx, req, false, "bash", "-c", dbcommons.GetCheckpointFileCMD, "2> /dev/null") + ctx, req, false, "bash", "-c", dbcommons.GetCheckpointFileCMD) if err != nil { r.Log.Info(err.Error()) } From 57849b8735613050b1fae3b4b54428e6ab1d4f04 Mon Sep 17 00:00:00 2001 From: douglas_williams Date: Tue, 10 May 2022 20:05:33 +0000 Subject: [PATCH 372/628] Doc style edit to network_access_options.md --- docs/adb/NETWORK_ACCESS_OPTIONS.md | 76 ++++++++++++++++-------------- 1 file changed, 41 insertions(+), 35 deletions(-) diff --git a/docs/adb/NETWORK_ACCESS_OPTIONS.md b/docs/adb/NETWORK_ACCESS_OPTIONS.md index 908fde32..e029b52d 100644 --- a/docs/adb/NETWORK_ACCESS_OPTIONS.md +++ b/docs/adb/NETWORK_ACCESS_OPTIONS.md @@ -1,78 +1,83 @@ -# Configuring Network Access of Autonomous Database +# Configuring Network Access for Oracle Autonomous Database -This documentation describes how to configure network access with public access, access control rules (ACLs), or private endpoints. Also describes how to configure the TLS connections (require mutual TLS only or allow both 1 way TLS and mutual TLS). For more information, please visit [this page](https://docs.oracle.com/en/cloud/paas/autonomous-database/adbsa/autonomous-network-access.html#GUID-D2D468C3-CA2D-411E-92BC-E122F795A413). +To configure network access for Oracle Autonomous Database (Autonomous Database), review and complete the procedures in this document. + +Network access for Autonomous Database includes public access, and configuring secure access, either over public networks using access control rules (ACLs), or by using using private endpoints inside a Virtual Cloud Network (VCN) in your tenancy. This document also describes procedures to configure the Transport Layer Security (TLS) connections, with the option either to require mutual TLS only, or to allow both one-way TLS and mutual TLS. + +For more information about these options, see: [Configuring Network Access with Access Control Rules (ACLs) and Private Endpoints ](https://docs.oracle.com/en/cloud/paas/autonomous-database/adbsa/autonomous-network-access.html#GUID-D2D468C3-CA2D-411E-92BC-E122F795A413). ## Supported Features +Review the network access configuration options available to you with Autonomous Database. ### Types of Network Access There are three types of network access supported by Autonomous Database: -* **PUBLIC**: +* **PUBLIC** - This option allows secure access from anywhere. The network access type is PUBLIC if no option is specified in the spec. With this option, mutual TLS (mTLS) authentication is always required to connect to the database. This option is available only for databases on shared Exadata infrastructure. + The Public option permits secure access from anywhere. The network access type is PUBLIC if no option is specified in the specification. With this option, mutual TLS (mTLS) authentication is always required to connect to the database. This option is available only for databases on shared Exadata infrastructure. -* **RESTRICTED**: +* **RESTRICTED** - This option restricts connections to the database according to the access control lists (ACLs) you specify. This option is available only for databases on shared Exadata infrastructure. + The Restricted option permits connections to the database only as specified by the access control lists (ACLs) that you create. This option is available only for databases on shared Exadata infrastructure. You can add the following to your ACL: - * **IP Address**: Specify one or more individual public IP address. Use commas to separate your addresses in the input field. + * **IP Address**: Specify one or more individual public IP addresses. Use commas to delimit your addresses in the input field. * **CIDR Block**: Specify one or more ranges of public IP addresses using CIDR notation. Use commas to separate your CIDR block entries in the input field. - * **Virtual Cloud Network (OCID)** (applies to Autonomous Databases on shared Exadata infrastructure): Specify the OCID of a virtual cloud network (VCN). If you want to specify multiple IP addresses or CIDR ranges within the same VCN, then do not create multiple access control list entries. Use one access control list entry with the values for the multiple IP addresses or CIDR ranges separated by commas. + * **Virtual Cloud Network (OCID)** (applies to Autonomous Databases on shared Exadata infrastructure): Specify the Oracle Cloud Identifier (OCID) of a virtual cloud network (VCN). If you want to specify multiple IP addresses or CIDR ranges within the same VCN, then do not create multiple access control list entries. Instead, use one access control list entry with the values for the multiple IP addresses or CIDR ranges, separated by commas. -* **PRIVATE**: +* **PRIVATE** - This option creates a private endpoint for your database within a specified VCN. This option is available for databases on shared Exadata infrastructure and is the only available option for databases on dedicated Exadata infrastructure. + The Private option creates a private endpoint for your database within a specified VCN. This option is available for databases on shared Exadata infrastructure, and is the only available option for databases on dedicated Exadata infrastructure. Review the private options for your configuration: * **Autonomous Databases on shared Exadata infrastructure**: - This option allows the access through private enpoints by specifying the OCIDs of a subnet and network security groups (NSGs) under the same VCN in the spec. + This option permits access through private enpoints by specifying the OCIDs of a subnet and the network security groups (NSGs) under the same VCN in the specification. * **Autonomous Databases on dedicated Exadata infrastructure**: - The network path to a dedicated Autonomous Database is through a VCN and subnet defined by the dedicated infrastucture hosting the database. Usually, the subnet is defined as private, meaning that there is no public Internet access to databases. + The network path to a dedicated Autonomous Database is through a VCN and subnet defined by the dedicated infrastucture hosting the database. Usually, the subnet is defined as private, which means that there is no public Internet access to the databases. - Autonomous Database supports restricted access using a ACL. You can optionally enabling an ACL by setting the `isAccessControlEnabled` parameter. If disabled, database access is defined by the network security rules. If enabled, database access is restricted to the IP addresses and CIDR blocks defined in the ACL. Note that enabling an ACL with an empty list of IP addresses makes the database inaccessible. See [Autonomous Database with Private Endpoint](https://docs.oracle.com/en-us/iaas/Content/Database/Concepts/adbsprivateaccess.htm) for overview and examples for private endpoint. + Autonomous Database supports restricted access using an ACL. You have the option to enable an ACL by setting the `isAccessControlEnabled` parameter. If access is disabled, then database access is defined by the network security rules. If enabled, then database access is restricted to the IP addresses and CIDR blocks defined in the ACL. Note that enabling an ACL with an empty list of IP addresses makes the database inaccessible. See [Autonomous Database with Private Endpoint](https://docs.oracle.com/en-us/iaas/Content/Database/Concepts/adbsprivateaccess.htm) for overview and examples for private endpoint. ### Allowing TLS or Require Only Mutual TLS (mTLS) Authentication -If your Autonomous Database instance is configured to only allow mTLS connections, you can update the instance to allow both mTLS and TLS connections. When you update your configuration to allow both mTLS and TLS, you can use both authentication types at the same time and connections are no longer restricted to require mTLS authentication. +If your Autonomous Database instance is configured to allow only mTLS connections, then you can reconfigure the instance to permit both mTLS and TLS connections. When you reconfigure the instance to permit both mTLS and TLS, you can use both authentication types at the same time, so that connections are no longer restricted to require mTLS authentication. -This option only applies to Autonomous Databases on shared Exadata infrastructure, and you can allow TLS connections when network access type is configured as follows: +This option only applies to Autonomous Databases on shared Exadata infrastructure. You can permit TLS connections when network access type is configured by using one of the following options: * **RESTRICTED**: with ACLs defined. * **PRIVATE**: with a private endpoint defined. -## Sample YAML +## Example YAML -You can always configure the network access options when you create an Autonomous Database, or update the settings after the creation. Following are some sample YAMLs which configure the networking with different newtwork access options. +You can always configure the network access options when you create an Autonomous Database, or update the settings after you create the database. Following are some example YAMLs that show how to configure the networking with different network access options. -For Autonomous Databases on shared Exadata infrastructure, you can: +For Autonomous Databases on shared Exadata infrastructure, review the following examples: * Configure network access [with PUBLIC access type](#autonomous-database-with-public-access-type-on-shared-exadata-infrastructure) * Configure network access [with RESTRICTED access type](#autonomous-database-with-restricted-access-type-on-shared-exadata-infrastructure) * Configure network access [with PRIVATE access type](#autonomous-database-with-private-access-type-on-shared-exadata-infrastructure) * [Change the mutual TLS (mTLS) authentication setting](#allow-both-tls-and-mutual-tls-mtls-authentication-of-autonomous-database-on-shared-exadata-infrastructure) -For Autonomous Databases on dedicated Exadata infrastructure, you can: +For Autonomous Databases on dedicated Exadata infrastructure, refiew the following examples: * Configure network access [with access control list enabled](#autonomous-database-with-access-control-list-enabled-on-dedicated-exadata-infrastructure) > Note: > -> * The above operations require an `AutonomousDatabase` object to be in your cluster. This example assumes either the provision operation or the bind operation has been done by the users and the operator is authorized with API Key Authentication. -> * If you are creating an Autonomous Database, see step 4 of [Provision an Autonomous Database](./README.md#provision-an-autonomous-database) in [Managing Oracle Autonomous Databases with Oracle Database Operator for Kubernetes](./README.md) topic to return to provisioning instructions. +> * Operations on Exadata infrastructure require an `AutonomousDatabase` object to be in your cluster. These examples assume either the provision operation or the bind operation has been done before you begin, and the operator is authorized with API Key Authentication. +> * If you are creating an Autonomous Database, then see step 4 of [Provision an Autonomous Database](./README.md#provision-an-autonomous-database) in [Managing Oracle Autonomous Databases with Oracle Database Operator for Kubernetes](./README.md) topic to return to provisioning instructions. ### Autonomous Database with PUBLIC access type on shared Exadata infrastructure -Follow the steps to configure the network with PUBLIC access type. +To configure the network with PUBLIC access type, complete this procedure. -1. Add the following parameters to the spec. An example file is availble here: [`config/samples/adb/autonomousdatabase_update_network_access.yaml`](./../../config/samples/adb/autonomousdatabase_update_network_access.yaml): +1. Add the following parameters to the specification. An example file is availble here: [`config/samples/adb/autonomousdatabase_update_network_access.yaml`](./../../config/samples/adb/autonomousdatabase_update_network_access.yaml): | Attribute | Type | Description | Required? | |----|----|----|----| - | `networkAccess.accessType` | string | An enum value which defines how the database can be accessed. The value can be PUBLIC, RESTRICTED or PRIVATE. See [Types of Network Access](#types-of-network-access) for more descriptions. | Yes | + | `networkAccess.accessType` | string | An enumeration (enum) value that defines how the database can be accessed. The value can be PUBLIC, RESTRICTED or PRIVATE. See [Types of Network Access](#types-of-network-access) for more descriptions. | Yes | ```yaml --- @@ -100,14 +105,15 @@ Follow the steps to configure the network with PUBLIC access type. ### Autonomous Database with RESTRICTED access type on shared Exadata infrastructure -Follow the steps to configure the network with RESTRICTED access type. +To configure the network with RESTRICTED access type, complete this procedure. -1. Add the following parameters to the spec. An example file is availble here: [`config/samples/adb/autonomousdatabase_update_network_access.yaml`](./../../config/samples/adb/autonomousdatabase_update_network_access.yaml): + +1. Add the following parameters to the specification. An example file is availble here: [`config/samples/adb/autonomousdatabase_update_network_access.yaml`](./../../config/samples/adb/autonomousdatabase_update_network_access.yaml): | Attribute | Type | Description | Required? | |----|----|----|----| - | `networkAccess.accessType` | string | An enum value which defines how the database can be accessed. The value can be PUBLIC, RESTRICTED or PRIVATE. See [Types of Network Access](#types-of-network-access) for more descriptions. | Yes | - | `networkAccess.accessControlList` | []string | The client IP access control list (ACL). This feature is available for autonomous databases on [shared Exadata infrastructure](https://docs.cloud.oracle.com/Content/Database/Concepts/adboverview.htm#AEI) and on Exadata Cloud@Customer.
Only clients connecting from an IP address included in the ACL may access the Autonomous Database instance.

For shared Exadata infrastructure, this is an array of CIDR (Classless Inter-Domain Routing) notations for a subnet or VCN OCID.
Use a semicolon (;) as a deliminator between the VCN-specific subnets or IPs.
Example: `["1.1.1.1","1.1.1.0/24","ocid1.vcn.oc1.sea.","ocid1.vcn.oc1.sea.;1.1.1.1","ocid1.vcn.oc1.sea.;1.1.0.0/16"]`

For Exadata Cloud@Customer, this is an array of IP addresses or CIDR (Classless Inter-Domain Routing) notations.
Example: `["1.1.1.1","1.1.1.0/24","1.1.2.25"]`

For an update operation, if you want to delete all the IPs in the ACL, use an array with a single empty string entry. | Yes | + | `networkAccess.accessType` | string | An enumerated (enum) that defines how the database can be accessed. The value can be PUBLIC, RESTRICTED or PRIVATE. See [Types of Network Access](#types-of-network-access) for more descriptions. | Yes | + | `networkAccess.accessControlList` | []string | The client IP access control list (ACL). This feature is available for Autonomous Databases on [shared Exadata infrastructure](https://docs.cloud.oracle.com/Content/Database/Concepts/adboverview.htm#AEI) and on Exadata Cloud@Customer.
Only clients connecting from an IP address included in the ACL may access the Autonomous Database instance.

For shared Exadata infrastructure, this is an array of CIDR (Classless Inter-Domain Routing) notations for a subnet or VCN OCID.
Use a semicolon (;) as a deliminator between the VCN-specific subnets or IPs.
Example: `["1.1.1.1","1.1.1.0/24","ocid1.vcn.oc1.sea.","ocid1.vcn.oc1.sea.;1.1.1.1","ocid1.vcn.oc1.sea.;1.1.0.0/16"]`

For Exadata Cloud@Customer, this is an array of IP addresses or CIDR (Classless Inter-Domain Routing) notations.
Example: `["1.1.1.1","1.1.1.0/24","1.1.2.25"]`

For an update operation, if you want to delete all the IPs in the ACL, use an array with a single empty string entry. | Yes | ```yaml --- @@ -140,7 +146,7 @@ Follow the steps to configure the network with RESTRICTED access type. ### Autonomous Database with PRIVATE access type on shared Exadata infrastructure -Follow the steps to configure the network with RESTRICTED access type. +To configure the network with PRIVATE access type, complete this procedure 1. Visit [Overview of VCNs and Subnets](https://docs.oracle.com/en-us/iaas/Content/Network/Tasks/managingVCNs_topic-Overview_of_VCNs_and_Subnets.htm#console) and [Network Security Groups](https://docs.oracle.com/en-us/iaas/Content/Network/Concepts/networksecuritygroups.htm#working) to see how to create VCNs, subnets, and network security groups (NSGs) if you haven't created them yet. The subnet and the NSG has to be in the same VCN. @@ -148,7 +154,7 @@ Follow the steps to configure the network with RESTRICTED access type. | Attribute | Type | Description | Required? | |----|----|----|----| - | `networkAccess.accessType` | string | An enum value which defines how the database can be accessed. The value can be PUBLIC, RESTRICTED or PRIVATE. See [Types of Network Access](#types-of-network-access) for more descriptions. | Yes | + | `networkAccess.accessType` | string | An enumeration (enum) value that defines how the database can be accessed. The value can be PUBLIC, RESTRICTED or PRIVATE. See [Types of Network Access](#types-of-network-access) for more descriptions. | Yes | | `networkAccess.privateEndpoint.subnetOCID` | string | The [OCID](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm) of the subnet the resource is associated with.

**Subnet Restrictions:**
- For bare metal DB systems and for single node virtual machine DB systems, do not use a subnet that overlaps with 192.168.16.16/28.
- For Exadata and virtual machine 2-node RAC systems, do not use a subnet that overlaps with 192.168.128.0/20.
- For Autonomous Database, setting this will disable public secure access to the database.
These subnets are used by the Oracle Clusterware private interconnect on the database instance.
Specifying an overlapping subnet will cause the private interconnect to malfunction.
This restriction applies to both the client subnet and the backup subnet. | Yes | | `networkAccess.privateEndpoint.nsgOCIDs` | string[] | A list of the [OCIDs](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm) of the network security groups (NSGs) that this resource belongs to. Setting this to an empty array after the list is created removes the resource from all NSGs. For more information about NSGs, see [Security Rules](https://docs.cloud.oracle.com/Content/Network/Concepts/securityrules.htm).

**NsgOCIDs restrictions:**
- Autonomous Databases with private access require at least 1 Network Security Group (NSG). The nsgOCIDs array cannot be empty. | Yes | | `networkAccess.privateEndpoint.hostnamePrefix` | string | The hostname prefix for the resource. | No | @@ -176,9 +182,9 @@ Follow the steps to configure the network with RESTRICTED access type. ### Allow both TLS and mutual TLS (mTLS) authentication of Autonomous Database on shared Exadata infrastructure -If you are using the RESTRICTED or the PRIVATE network access option, you can choose whether to allow both TLS and mutual TLS (mTLS) authentication, or to allow only mTLS authentication. Follow the steps to change the mTLS authentication setting. +If you are using either the RESTRICTED or the PRIVATE network access option, then you can choose whether to permit both TLS and mutual TLS (mTLS) authentication, or to permit only mTLS authentication. To change the mTLS authentication setting, complete the following steps: -1. Add the following parameters to the spec. An example file is availble here: [`config/samples/adb/autonomousdatabase_update_mtls.yaml`](./../../config/samples/adb/autonomousdatabase_update_mtls.yaml): +1. Add the following parameters to the specification. An example file is availble here: [`config/samples/adb/autonomousdatabase_update_mtls.yaml`](./../../config/samples/adb/autonomousdatabase_update_mtls.yaml): | Attribute | Type | Description | Required? | |----|----|----|----| @@ -209,13 +215,13 @@ If you are using the RESTRICTED or the PRIVATE network access option, you can ch ### Autonomous Database with access control list enabled on dedicated Exadata infrastructure -Follow the steps to configure the network with RESTRICTED access type. +To configure the network with RESTRICTED access type using an access control list (ACL), complete this procedure. -1. Add the following parameters to the spec. An example file is availble here: [`config/samples/adb/autonomousdatabase_update_network_access.yaml`](./../../config/samples/adb/autonomousdatabase_update_network_access.yaml): +1. Add the following parameters to the specification. An example file is availble here: [`config/samples/adb/autonomousdatabase_update_network_access.yaml`](./../../config/samples/adb/autonomousdatabase_update_network_access.yaml): | Attribute | Type | Description | Required? | |----|----|----|----| - | `networkAccess.isAccessControlEnabled` | boolean | Indicates if the database-level access control is enabled.

If disabled, database access is defined by the network security rules.

If enabled, database access is restricted to the IP addresses defined by the rules specified with the `accessControlList` property. While specifying `accessControlList` rules is optional, if database-level access control is enabled and no rules are specified, the database will become inaccessible. The rules can be added later using the `UpdateAutonomousDatabase` API operation or edit option in console.

When creating a database clone, the desired access control setting should be specified. By default, database-level access control will be disabled for the clone.
This property is applicable only to Autonomous Databases on the Exadata Cloud@Customer platform. | Yes | + | `networkAccess.isAccessControlEnabled` | boolean | Indicates if the database-level access control is enabled.

If disabled, then database access is defined by the network security rules.

If enabled, then database access is restricted to the IP addresses defined by the rules specified with the `accessControlList` property. While specifying `accessControlList` rules is optional, if database-level access control is enabled, and no rules are specified, then the database will become inaccessible. The rules can be added later by using the `UpdateAutonomousDatabase` API operation, or by using the edit option in console.

When creating a database clone, you should specify the access control setting that you want the clone database to use. By default, database-level access control will be disabled for the clone.
This property is applicable only to Autonomous Databases on the Exadata Cloud@Customer platform. | Yes | | `networkAccess.accessControlList` | []string | The client IP access control list (ACL). This feature is available for autonomous databases on [shared Exadata infrastructure](https://docs.cloud.oracle.com/Content/Database/Concepts/adboverview.htm#AEI) and on Exadata Cloud@Customer.
Only clients connecting from an IP address included in the ACL may access the Autonomous Database instance.

For shared Exadata infrastructure, this is an array of CIDR (Classless Inter-Domain Routing) notations for a subnet or VCN OCID.
Use a semicolon (;) as a deliminator between the VCN-specific subnets or IPs.
Example: `["1.1.1.1","1.1.1.0/24","ocid1.vcn.oc1.sea.","ocid1.vcn.oc1.sea.;1.1.1.1","ocid1.vcn.oc1.sea.;1.1.0.0/16"]`

For Exadata Cloud@Customer, this is an array of IP addresses or CIDR (Classless Inter-Domain Routing) notations.
Example: `["1.1.1.1","1.1.1.0/24","1.1.2.25"]`

For an update operation, if you want to delete all the IPs in the ACL, use an array with a single empty string entry. | Yes | ```yaml From bd5c6bce4bbb2d7833c7117da133ee2a857301c5 Mon Sep 17 00:00:00 2001 From: douglas_williams Date: Tue, 10 May 2022 21:13:53 +0000 Subject: [PATCH 373/628] Doc style edit update as requested by Ting-Lan Wang --- docs/adb/ADB_MANUAL_BACKUP.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/docs/adb/ADB_MANUAL_BACKUP.md b/docs/adb/ADB_MANUAL_BACKUP.md index 7b803e45..6d831915 100644 --- a/docs/adb/ADB_MANUAL_BACKUP.md +++ b/docs/adb/ADB_MANUAL_BACKUP.md @@ -1,28 +1,28 @@ -# Backing up an Autonomous Database Manually +# Backing up an Oracle Autonomous Database Manually -This document describes how to create manual backups of Autonomous Databases. +To create manual backups of Autonomous Databases, use this procedure. -Oracle Cloud Infrastructure automatically backs up your Autonomous Databases and retains these backups for 60 days. You can restore and recover your database to any point-in-time in this retention period. Automatic backups are full backups taken every 60 days and daily incremental backups. You can also create manual backups to supplement your automatic backups. Manual backups are stored in an Object Storage bucket that you create, and are retained for 60 days. For more information, please visit [this page](https://docs.oracle.com/en-us/iaas/Content/Database/Tasks/adbbackingup.htm). +Oracle Cloud Infrastructure (OCI) automatically backs up your Autonomous Databases, and retains these backups for 60 days. You can restore and recover your database to any point-in-time in this retention period. Automatic backups are full backups taken every 60 days, with daily incremental backups. You can also create manual backups to supplement your automatic backups. Manual backups are stored in an Object Storage bucket that you create, and are retained for 60 days. For more information, please visit [this page](https://docs.oracle.com/en-us/iaas/Content/Database/Tasks/adbbackingup.htm). -The operator gets the list of the `AutonomousBackupOCIDs` from OCI in every reconciliation loop, and creates the `AutonomousDatabaseBackup` resource automatically if the resource with the same `AutonomousBackupOCID` doesn't exist in the cluster. +In every OCI reconciliation loop, the operator obtains the list of the `AutonomousBackupOCIDs`. If an Autonomous Database backup resource with the same OCID (`AutonomousBackupOCID`) doesn't exist in the cluster, then OCI also creates the `AutonomousDatabaseBackup` resource automatically. ## Prerequisites -You must create an Oracle Cloud Infrastructure Object Storage bucket to hold your Autonomous Database manual backups and configure your database to connect to it. Please follow the steps [in this page](https://docs.oracle.com/en-us/iaas/Content/Database/Tasks/adbbackingup.htm#creatingbucket) to finish the setup. This is a one-time operation. +To hold your Autonomous Database manual backups, you must create an Oracle Cloud Infrastructure Object Storage bucket, and configure your database to connect to it. To finish setting up manual backup storage, follow the steps in this page: [Setting Up a Bucket to Store Manual Backups](https://docs.oracle.com/en-us/iaas/Content/Database/Tasks/adbbackingup.htm#creatingbucket). Creating an Autonomous Database manual backup object storage bucket is a one-time operation. ## Create Manual Backup -Follow the steps to back up an Autonomous Database. +To back up an Autonomous Database, complete this procedure. 1. Add the following fields to the AutonomousDatabaseBackup resource definition. An example `.yaml` file is available here: [`config/samples/adb/autonomousdatabase_backup.yaml`](./../../config/samples/adb/autonomousdatabase_backup.yaml) | Attribute | Type | Description | Required? | |----|----|----|----| - | `spec.target.k8sADB.name` | string | The name of custom resource of the target AutonomousDatabase. Choose either the `spec.target.k8sADB.name` or the `spec.target.ociADB.ocid`, but not both. | Conditional | + | `spec.target.k8sADB.name` | string | The name of custom resource of the target Autonomous Database. Choose either the `spec.target.k8sADB.name` or the `spec.target.ociADB.ocid`, but not both. | Conditional | | `spec.target.ociADB.ocid` | string | The [OCID](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm) of the target AutonomousDatabase. Choose either the `spec.target.k8sADB.name` or the `spec.target.ociADB.ocid`, but not both. | Conditional | - | `spec.displayName` | string | The user-friendly name for the backup. The name does not have to be unique. | Yes | - | `spec.ociConfig` | dictionary | Not required when the Operator is authorized with [Instance Principal](./ADB_PREREQUISITES.md#authorized-with-instance-principal). Otherwise, you will need the values from the [Authorized with API Key Authentication](./ADB_PREREQUISITES.md#authorized-with-api-key-authentication) section. | Conditional | + | `spec.displayName` | string | The user-friendly name for the backup. This name does not have to be unique. | Yes | + | `spec.ociConfig` | dictionary | Not required when the Operator is authorized with [Instance Principal](./ADB_PREREQUISITES.md#authorized-with-instance-principal). Otherwise, you will need the values from this section: [Authorized with API Key Authentication](./ADB_PREREQUISITES.md#authorized-with-api-key-authentication). | Conditional | | `spec.ociConfig.configMapName` | string | Name of the ConfigMap that holds the local OCI configuration | Conditional | - | `spec.ociConfig.secretName`| string | Name of the K8s Secret that holds the private key value | Conditional | + | `spec.ociConfig.secretName`| string | Name of the Kubernetes (K8s) Secret that holds the private key value | Conditional | ```yaml --- From 3fcf1a4304c45f44bf1c607d13b6cf598e261e42 Mon Sep 17 00:00:00 2001 From: douglas_williams Date: Tue, 10 May 2022 21:23:30 +0000 Subject: [PATCH 374/628] Documentation style edit requested by Ting-Lan Wang --- docs/adb/ADB_RESTORE.md | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/docs/adb/ADB_RESTORE.md b/docs/adb/ADB_RESTORE.md index 933287a8..8fc51592 100644 --- a/docs/adb/ADB_RESTORE.md +++ b/docs/adb/ADB_RESTORE.md @@ -1,23 +1,23 @@ -# Restoring an Autonomous Database Manually +# Restoring an Oracle Autonomous Database Manually -This document describes how to restore an Autonomous Database from a backup. +To restore an Autonomous Database from a backup, use this document. -You can use any existing manual or automatic backup to restore your database, or you can restore and recover your database to any point in time in the 60-day retention period of your automatic backups. For point-in-time restores, you specify a timestamp, and your Autonomous Database decides which backup to use for the fastest restore. +You can either use any existing manual or automatic backup to restore your database, or you can restore and recover your database to any point in time in the 60-day retention period of your automatic backups. For point-in-time restores, you specify a timestamp. Your Autonomous Database identifies which backup to use for the fastest restore. ## Restore an Autonomous Database -Follow the steps to restore an Autonomous Database from a backup or using point-in-time restore. +To restore an Autonomous Database from a backup, or by using point-in-time restore, complete this procedure. 1. Add the following fields to the AutonomousDatabaseBackup resource definition. An example `.yaml` file is available here: [`config/samples/adb/autonomousdatabase_restore.yaml`](./../../config/samples/adb/autonomousdatabase_restore.yaml) | Attribute | Type | Description | Required? | |----|----|----|----| - | `spec.target.k8sADB.name` | string | The name of custom resource of the target AutonomousDatabase. Choose either the `spec.target.k8sADB.name` or the `spec.target.ociADB.ocid`, but not both. | Conditional | - | `spec.target.ociADB.ocid` | string | The [OCID](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm) of the target AutonomousDatabase. Choose either the `spec.target.k8sADB.name` or the `spec.target.ociADB.ocid`, but not both. | Conditional | - | `spec.source.k8sADBBackup.name` | string | The name of custom resource of the AutonomousDatabaseBackup that you want to restore from. Choose either the `spec.source.k8sADBBackup.name` or the `spec.source.pointInTime.timestamp`, but not both. | Conditional | - | `spec.source.pointInTime.timestamp` | string | The timestamp to specify which time to restore the database to. Your Autonomous Database decides which backup to use for the fastest restore. The timestamp must follow this format: YYYY-MM-DD HH:MM:SS GMT. Choose either the `spec.source.k8sADBBackup.name` or the `spec.source.pointInTime.timestamp`, but not both. | Conditional | - | `spec.ociConfig` | dictionary | Not required when the Operator is authorized with [Instance Principal](./ADB_PREREQUISITES.md#authorized-with-instance-principal). Otherwise, you will need the values from the [Authorized with API Key Authentication](./ADB_PREREQUISITES.md#authorized-with-api-key-authentication) section. | Conditional | - | `spec.ociConfig.configMapName` | string | Name of the ConfigMap that holds the local OCI configuration | Conditional | - | `spec.ociConfig.secretName`| string | Name of the K8s Secret that holds the private key value | Conditional | + | `spec.target.k8sADB.name` | string | The name of custom resource of the target Autonomous Database (`AutonomousDatabase`). Choose either the `spec.target.k8sADB.name` or the `spec.target.ociADB.ocid`, but not both. | Conditional | + | `spec.target.ociADB.ocid` | string | The [OCID](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm) of the target `AutonomousDatabase`. Choose either the `spec.target.k8sADB.name` or the `spec.target.ociADB.ocid`, but not both. | Conditional | + | `spec.source.k8sADBBackup.name` | string | The name of custom resource of the `AutonomousDatabaseBackup` that you want to restore from. Choose either the `spec.source.k8sADBBackup.name` or the `spec.source.pointInTime.timestamp`, but not both. | Conditional | + | `spec.source.pointInTime.timestamp` | string | The timestamp to specify the point in time to which you want the database restored. Your Autonomous Database identifies which backup to use for the fastest restore. The timestamp must follow this format: YYYY-MM-DD HH:MM:SS GMT. Choose either the `spec.source.k8sADBBackup.name` or the `spec.source.pointInTime.timestamp`, but not both. | Conditional | + | `spec.ociConfig` | dictionary | Not required when the Operator is authorized with [Instance Principal](./ADB_PREREQUISITES.md#authorized-with-instance-principal). Otherwise, you will need the values from this section: [Authorized with API Key Authentication](./ADB_PREREQUISITES.md#authorized-with-api-key-authentication). | Conditional | + | `spec.ociConfig.configMapName` | string | Name of the `ConfigMap` that holds the local OCI configuration | Conditional | + | `spec.ociConfig.secretName`| string | Name of the Kubernetes (K8s) Secret that holds the private key value | Conditional | ```yaml --- From cc50abd7bc47ec94d1005ebd3b34ec6388c6d4f2 Mon Sep 17 00:00:00 2001 From: omar_salazar Date: Wed, 11 May 2022 18:41:37 +0000 Subject: [PATCH 375/628] Add acd tests --- Makefile | 2 +- .../autonomouscontainerdatabase_types.go | 10 +- test/e2e/autonomouscontainerdatabase_test.go | 142 +++++++++ ...autonomousdatabase_controller_bind_test.go | 8 +- ...tonomousdatabase_controller_create_test.go | 6 +- test/e2e/behavior/shared_behaviors.go | 285 ++++++++++++++++-- test/e2e/resource/test_config.yaml | 4 +- test/e2e/suite_test.go | 3 + test/e2e/util/oci_acd_request.go | 78 +++++ test/e2e/util/oci_db_request.go | 13 +- test/e2e/util/util.go | 11 +- 11 files changed, 518 insertions(+), 44 deletions(-) create mode 100644 test/e2e/autonomouscontainerdatabase_test.go create mode 100644 test/e2e/util/oci_acd_request.go diff --git a/Makefile b/Makefile index a29214db..3d963755 100644 --- a/Makefile +++ b/Makefile @@ -59,7 +59,7 @@ test: manifests generate fmt vet envtest ## Run unit tests. E2ETEST ?= ./test/e2e/ e2e: manifests generate fmt vet envtest ## Run e2e tests. - KUBEBUILDER_ASSETS="$(shell $(ENVTEST) use $(ENVTEST_K8S_VERSION) -p path)" go test $(E2ETEST) -v -timeout 80m -ginkgo.v -ginkgo.failFast + KUBEBUILDER_ASSETS="$(shell $(ENVTEST) use $(ENVTEST_K8S_VERSION) -p path)" ginkgo -v --timeout=2h30m --fail-fast $(E2ETEST) ##@ Build diff --git a/apis/database/v1alpha1/autonomouscontainerdatabase_types.go b/apis/database/v1alpha1/autonomouscontainerdatabase_types.go index 1d812253..b69156d4 100644 --- a/apis/database/v1alpha1/autonomouscontainerdatabase_types.go +++ b/apis/database/v1alpha1/autonomouscontainerdatabase_types.go @@ -52,12 +52,12 @@ import ( // name of our custom finalizer const ACDFinalizer = "database.oracle.com/acd-finalizer" -type acdActionEnum string +type AcdActionEnum string const ( - AcdActionBlank acdActionEnum = "" - AcdActionRestart acdActionEnum = "RESTART" - AcdActionTerminate acdActionEnum = "TERMINATE" + AcdActionBlank AcdActionEnum = "" + AcdActionRestart AcdActionEnum = "RESTART" + AcdActionTerminate AcdActionEnum = "TERMINATE" ) // AutonomousContainerDatabaseSpec defines the desired state of AutonomousContainerDatabase @@ -71,7 +71,7 @@ type AutonomousContainerDatabaseSpec struct { // +kubebuilder:validation:Enum:="RELEASE_UPDATES";"RELEASE_UPDATE_REVISIONS" PatchModel database.AutonomousContainerDatabasePatchModelEnum `json:"patchModel,omitempty"` // +kubebuilder:validation:Enum:="SYNC";"RESTART";"TERMINATE" - Action acdActionEnum `json:"action,omitempty"` + Action AcdActionEnum `json:"action,omitempty"` FreeformTags map[string]string `json:"freeformTags,omitempty"` OCIConfig OCIConfigSpec `json:"ociConfig,omitempty"` diff --git a/test/e2e/autonomouscontainerdatabase_test.go b/test/e2e/autonomouscontainerdatabase_test.go new file mode 100644 index 00000000..7b9fb49e --- /dev/null +++ b/test/e2e/autonomouscontainerdatabase_test.go @@ -0,0 +1,142 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package e2etest + +import ( + "context" + "time" + + "github.com/oracle/oci-go-sdk/v63/common" + "github.com/oracle/oci-go-sdk/v63/database" + + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/types" + + dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + "github.com/oracle/oracle-database-operator/test/e2e/behavior" + "github.com/oracle/oracle-database-operator/test/e2e/util" + // +kubebuilder:scaffold:imports +) + +var _ = Describe("test ACD binding", func() { + var acdLookupKey types.NamespacedName + var acdID string + + AfterEach(func() { + // IMPORTANT: The operator might have to call reconcile multiple times to finish an operation. + // If we do the update immediately, the previous reconciliation will overwrite the changes. + By("Sleeping 20 seconds to wait for reconciliation to finish") + time.Sleep(time.Second * 20) + }) + + Describe("ACD Provisioning", func() { + It("Should create an AutonomousContainerDatabase resource and in OCI", func() { + provisionAcd := &dbv1alpha1.AutonomousContainerDatabase{ + TypeMeta: metav1.TypeMeta{ + APIVersion: "database.oracle.com/v1alpha1", + Kind: "AutonomousContainerDatabase", + }, + ObjectMeta: metav1.ObjectMeta{ + Name: "provisionacd", + Namespace: ADBNamespace, + }, + Spec: dbv1alpha1.AutonomousContainerDatabaseSpec{ + DisplayName: common.String(e2eutil.GenerateACDName()), + CompartmentOCID: common.String(SharedCompartmentOCID), + AutonomousExadataVMClusterOCID: common.String(SharedExadataVMClusterOCID), + PatchModel: database.AutonomousContainerDatabasePatchModelUpdates, + OCIConfig: dbv1alpha1.OCIConfigSpec{ + ConfigMapName: common.String(SharedOCIConfigMapName), + SecretName: common.String(SharedOCISecretName), + }, + }, + } + + acdLookupKey = types.NamespacedName{Name: provisionAcd.Name, Namespace: provisionAcd.Namespace} + + Expect(k8sClient.Create(context.TODO(), provisionAcd)).Should(Succeed()) + }) + + It("Should check ACD status is BACKUP IN PROGRESS", e2ebehavior.AssertACDState(&k8sClient, &dbClient, &acdLookupKey, database.AutonomousContainerDatabaseLifecycleStateBackupInProgress, time.Minute*35)) + + It("Should check ACD status is AVAILABLE", e2ebehavior.AssertACDState(&k8sClient, &dbClient, &acdLookupKey, database.AutonomousContainerDatabaseLifecycleStateAvailable, time.Minute*60)) + + It("Should save ACD ocid for next test", func() { + acd := &dbv1alpha1.AutonomousContainerDatabase{} + Expect(k8sClient.Get(context.TODO(), acdLookupKey, acd)).To(Succeed()) + acdID = *acd.Spec.AutonomousContainerDatabaseOCID + }) + + It("Should delete ACD local resource", e2ebehavior.AssertACDLocalDelete(&k8sClient, &dbClient, &acdLookupKey)) + }) + + Describe("ACD Binding", func() { + It("Should create an AutonomousContainerDatabase resource", func() { + acd := &dbv1alpha1.AutonomousContainerDatabase{ + TypeMeta: metav1.TypeMeta{ + APIVersion: "database.oracle.com/v1alpha1", + Kind: "AutonomousContainerDatabase", + }, + ObjectMeta: metav1.ObjectMeta{ + Name: "bindacd", + Namespace: ADBNamespace, + }, + Spec: dbv1alpha1.AutonomousContainerDatabaseSpec{ + AutonomousContainerDatabaseOCID: common.String(acdID), + OCIConfig: dbv1alpha1.OCIConfigSpec{ + ConfigMapName: common.String(SharedOCIConfigMapName), + SecretName: common.String(SharedOCISecretName), + }, + }, + } + + acdLookupKey = types.NamespacedName{Name: acd.Name, Namespace: acd.Namespace} + + Expect(k8sClient.Create(context.TODO(), acd)).Should(Succeed()) + }) + + It("Should bind to an ACD", e2ebehavior.AssertACDBind(&k8sClient, &dbClient, &acdLookupKey, database.AutonomousContainerDatabaseLifecycleStateAvailable)) + + It("Should update the ACD", e2ebehavior.UpdateAndAssertACDSpec(&k8sClient, &dbClient, &acdLookupKey)) + + It("Should restart the ACD", e2ebehavior.AssertACDRestart(&k8sClient, &dbClient, &acdLookupKey)) + + It("Should terminate the ACD", e2ebehavior.AssertACDTerminate(&k8sClient, &dbClient, &acdLookupKey)) + }) +}) diff --git a/test/e2e/autonomousdatabase_controller_bind_test.go b/test/e2e/autonomousdatabase_controller_bind_test.go index 9f2176eb..22f8468c 100644 --- a/test/e2e/autonomousdatabase_controller_bind_test.go +++ b/test/e2e/autonomousdatabase_controller_bind_test.go @@ -132,9 +132,9 @@ var _ = Describe("test ADB binding with hardLink=true", func() { It("should update ADB", e2ebehavior.UpdateAndAssertDetails(&k8sClient, &dbClient, &adbLookupKey, SharedNewAdminPassSecretName, &SharedPlainTextNewAdminPassword, &SharedPlainTextWalletPassword)) - It("Should stop ADB", e2ebehavior.UpdateAndAssertState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateStopped)) + It("Should stop ADB", e2ebehavior.UpdateAndAssertADBState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateStopped)) - It("Should restart ADB", e2ebehavior.UpdateAndAssertState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateAvailable)) + It("Should restart ADB", e2ebehavior.UpdateAndAssertADBState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateAvailable)) It("Should change to RESTRICTED network access", e2ebehavior.TestNetworkAccessRestricted(&k8sClient, &dbClient, &adbLookupKey, false)) @@ -196,7 +196,7 @@ var _ = Describe("test ADB binding with hardLink=true", func() { Describe("bind to a terminated adb", func() { //Wait until remote state is terminated - It("Should check that OCI adb state is terminated", e2ebehavior.AssertRemoteStateOCID(&k8sClient, &dbClient, &terminatedAdbID, database.AutonomousDatabaseLifecycleStateTerminated, time.Second*300)) + It("Should check that OCI adb state is terminated", e2ebehavior.AssertADBRemoteStateOCID(&k8sClient, &dbClient, &terminatedAdbID, database.AutonomousDatabaseLifecycleStateTerminated, time.Second*300)) It("Should create a AutonomousDatabase resource", func() { adb := &dbv1alpha1.AutonomousDatabase{ @@ -225,7 +225,7 @@ var _ = Describe("test ADB binding with hardLink=true", func() { Expect(k8sClient.Create(context.TODO(), adb)).Should(Succeed()) }) - It("Should check for TERMINATED state in local resource", e2ebehavior.AssertLocalState(&k8sClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateTerminated)) + It("Should check for TERMINATED state in local resource", e2ebehavior.AssertADBLocalState(&k8sClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateTerminated)) It("Should delete local resource", e2ebehavior.AssertSoftLinkDelete(&k8sClient, &adbLookupKey)) }) diff --git a/test/e2e/autonomousdatabase_controller_create_test.go b/test/e2e/autonomousdatabase_controller_create_test.go index 82e11462..22f940c9 100644 --- a/test/e2e/autonomousdatabase_controller_create_test.go +++ b/test/e2e/autonomousdatabase_controller_create_test.go @@ -157,7 +157,7 @@ var _ = Describe("test ADB provisioning", func() { Expect(k8sClient.Create(context.TODO(), duplicateAdb)).To(Succeed()) }) - It("Should check for local resource state \"\"", e2ebehavior.AssertLocalState(&k8sClient, &dupAdbLookupKey, "")) + It("Should check for local resource state \"\"", e2ebehavior.AssertADBLocalState(&k8sClient, &dupAdbLookupKey, "")) It("Should cleanup the resource with duplicated db name", func() { duplicateAdb := &dbv1alpha1.AutonomousDatabase{ @@ -174,7 +174,7 @@ var _ = Describe("test ADB provisioning", func() { }) It("Should create an Autonomous Database Backup", func() { - e2ebehavior.AssertState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateAvailable)() + e2ebehavior.AssertADBState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateAvailable)() // Get adb ocid adb := &dbv1alpha1.AutonomousDatabase{} @@ -214,7 +214,7 @@ var _ = Describe("test ADB provisioning", func() { }) It("Should restore a database", func() { - e2ebehavior.AssertState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateAvailable)() + e2ebehavior.AssertADBState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateAvailable)() adbRestore := &dbv1alpha1.AutonomousDatabaseRestore{ TypeMeta: metav1.TypeMeta{ diff --git a/test/e2e/behavior/shared_behaviors.go b/test/e2e/behavior/shared_behaviors.go index 8c84d3e4..4dc2f834 100644 --- a/test/e2e/behavior/shared_behaviors.go +++ b/test/e2e/behavior/shared_behaviors.go @@ -86,8 +86,9 @@ var ( bindTimeout = time.Second * 30 backupTimeout = time.Minute * 20 intervalTime = time.Second * 10 - updateTimeout = time.Minute * 7 + updateADBTimeout = time.Minute * 7 changeLocalStateTimeout = time.Second * 600 + updateACDTimeout = time.Minute * 3 ) func AssertProvision(k8sClient *client.Client, adbLookupKey *types.NamespacedName) func() { @@ -258,7 +259,7 @@ func UpdateDetails(k8sClient *client.Client, dbClient *database.DatabaseClient, } return database.AutonomousDatabaseLifecycleStateEnum(listResp.Items[0].LifecycleState), nil - }, updateTimeout, intervalTime).Should(Equal(database.AutonomousDatabaseLifecycleStateAvailable)) + }, updateADBTimeout, intervalTime).Should(Equal(database.AutonomousDatabaseLifecycleStateAvailable)) // Update var newDisplayName = *expectedADB.Spec.Details.DisplayName + "_new" @@ -305,7 +306,7 @@ func AssertADBDetails(k8sClient *client.Client, expectedADBDetails := expectedADB.Spec.Details Eventually(func() (bool, error) { // Fetch the ADB from OCI when it's in AVAILABLE state, and retry if its attributes doesn't match the new ADB's attributes - retryPolicy := e2eutil.NewLifecycleStateRetryPolicy(database.AutonomousDatabaseLifecycleStateAvailable) + retryPolicy := e2eutil.NewLifecycleStateRetryPolicyADB(database.AutonomousDatabaseLifecycleStateAvailable) resp, err := e2eutil.GetAutonomousDatabase(derefDBClient, expectedADB.Spec.Details.AutonomousDatabaseOCID, &retryPolicy) if err != nil { return false, err @@ -389,11 +390,11 @@ func AssertADBDetails(k8sClient *client.Client, compareString(expectedADBDetails.NetworkAccess.PrivateEndpoint.HostnamePrefix, resp.AutonomousDatabase.PrivateEndpointLabel) return same, nil - }, updateTimeout, intervalTime).Should(BeTrue()) + }, updateADBTimeout, intervalTime).Should(BeTrue()) // IMPORTANT: make sure the local resource has finished reconciling, otherwise the changes will // be conflicted with the next test and cause unknow result. - AssertLocalState(k8sClient, adbLookupKey, database.AutonomousDatabaseLifecycleStateAvailable)() + AssertADBLocalState(k8sClient, adbLookupKey, database.AutonomousDatabaseLifecycleStateAvailable)() } } @@ -484,7 +485,7 @@ func TestNetworkAccess(k8sClient *client.Client, dbClient *database.DatabaseClie derefK8sClient := *k8sClient adb := &dbv1alpha1.AutonomousDatabase{} - AssertState(k8sClient, dbClient, adbLookupKey, database.AutonomousDatabaseLifecycleStateAvailable)() + AssertADBState(k8sClient, dbClient, adbLookupKey, database.AutonomousDatabaseLifecycleStateAvailable)() Expect(derefK8sClient.Get(context.TODO(), *adbLookupKey, adb)).To(Succeed()) adb.Spec.Details.NetworkAccess = networkSpec @@ -509,22 +510,22 @@ func UpdateAndAssertDetails(k8sClient *client.Client, dbClient *database.Databas } } -// UpdateAndAssertState updates adb state and then asserts if change is propagated to OCI -func UpdateAndAssertState(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName, state database.AutonomousDatabaseLifecycleStateEnum) func() { +// UpdateAndAssertADBState updates adb state and then asserts if change is propagated to OCI +func UpdateAndAssertADBState(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName, state database.AutonomousDatabaseLifecycleStateEnum) func() { return func() { UpdateState(k8sClient, adbLookupKey, state)() - AssertState(k8sClient, dbClient, adbLookupKey, state)() + AssertADBState(k8sClient, dbClient, adbLookupKey, state)() } } -// AssertState asserts local and remote state -func AssertState(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName, state database.AutonomousDatabaseLifecycleStateEnum) func() { +// AssertADBState asserts local and remote state +func AssertADBState(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName, state database.AutonomousDatabaseLifecycleStateEnum) func() { return func() { // Waits longer for the local resource to reach the desired state - AssertLocalState(k8sClient, adbLookupKey, state)() + AssertADBLocalState(k8sClient, adbLookupKey, state)() // Double-check the state of the DB in OCI so the timeout can be shorter - AssertRemoteState(k8sClient, dbClient, adbLookupKey, state)() + AssertADBRemoteState(k8sClient, dbClient, adbLookupKey, state)() } } @@ -545,8 +546,8 @@ func AssertHardLinkDelete(k8sClient *client.Client, dbClient *database.DatabaseC By("Checking if the ADB in OCI is in TERMINATING state") // Check every 10 secs for total 60 secs Eventually(func() (database.AutonomousDatabaseLifecycleStateEnum, error) { - retryPolicy := e2eutil.NewLifecycleStateRetryPolicy(database.AutonomousDatabaseLifecycleStateTerminating) - return returnRemoteState(derefK8sClient, derefDBClient, adb.Spec.Details.AutonomousDatabaseOCID, &retryPolicy) + retryPolicy := e2eutil.NewLifecycleStateRetryPolicyADB(database.AutonomousDatabaseLifecycleStateTerminating) + return returnADBRemoteState(derefK8sClient, derefDBClient, adb.Spec.Details.AutonomousDatabaseOCID, &retryPolicy) }, changeTimeout).Should(Equal(database.AutonomousDatabaseLifecycleStateTerminating)) AssertSoftLinkDelete(k8sClient, adbLookupKey)() @@ -579,8 +580,8 @@ func AssertSoftLinkDelete(k8sClient *client.Client, adbLookupKey *types.Namespac } } -// AssertLocalState asserts the lifecycle state of the local resource using adbLookupKey -func AssertLocalState(k8sClient *client.Client, adbLookupKey *types.NamespacedName, state database.AutonomousDatabaseLifecycleStateEnum) func() { +// AssertADBLocalState asserts the lifecycle state of the local resource using adbLookupKey +func AssertADBLocalState(k8sClient *client.Client, adbLookupKey *types.NamespacedName, state database.AutonomousDatabaseLifecycleStateEnum) func() { return func() { Expect(k8sClient).NotTo(BeNil()) Expect(adbLookupKey).NotTo(BeNil()) @@ -589,12 +590,12 @@ func AssertLocalState(k8sClient *client.Client, adbLookupKey *types.NamespacedNa By("Checking if the lifecycleState of local resource is " + string(state)) Eventually(func() (database.AutonomousDatabaseLifecycleStateEnum, error) { - return returnLocalState(derefK8sClient, *adbLookupKey) + return returnADBLocalState(derefK8sClient, *adbLookupKey) }, changeLocalStateTimeout).Should(Equal(state)) } } -func AssertRemoteState(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName, state database.AutonomousDatabaseLifecycleStateEnum) func() { +func AssertADBRemoteState(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName, state database.AutonomousDatabaseLifecycleStateEnum) func() { return func() { Expect(k8sClient).NotTo(BeNil()) Expect(dbClient).NotTo(BeNil()) @@ -605,12 +606,12 @@ func AssertRemoteState(k8sClient *client.Client, dbClient *database.DatabaseClie adb := &dbv1alpha1.AutonomousDatabase{} Expect(derefK8sClient.Get(context.TODO(), *adbLookupKey, adb)).To(Succeed()) By("Checking if the lifecycleState of remote resource is " + string(state)) - AssertRemoteStateOCID(k8sClient, dbClient, adb.Spec.Details.AutonomousDatabaseOCID, state, changeTimeout)() + AssertADBRemoteStateOCID(k8sClient, dbClient, adb.Spec.Details.AutonomousDatabaseOCID, state, changeTimeout)() } } // Backup takes ~15 minutes to complete, this function waits 20 minutes until ADB state is AVAILABLE -func AssertRemoteStateForBackupRestore(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName, state database.AutonomousDatabaseLifecycleStateEnum) func() { +func AssertADBRemoteStateForBackupRestore(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName, state database.AutonomousDatabaseLifecycleStateEnum) func() { return func() { Expect(k8sClient).NotTo(BeNil()) Expect(dbClient).NotTo(BeNil()) @@ -621,11 +622,11 @@ func AssertRemoteStateForBackupRestore(k8sClient *client.Client, dbClient *datab adb := &dbv1alpha1.AutonomousDatabase{} Expect(derefK8sClient.Get(context.TODO(), *adbLookupKey, adb)).To(Succeed()) By("Checking if the lifecycleState of remote resource is " + string(state)) - AssertRemoteStateOCID(k8sClient, dbClient, adb.Spec.Details.AutonomousDatabaseOCID, state, backupTimeout)() + AssertADBRemoteStateOCID(k8sClient, dbClient, adb.Spec.Details.AutonomousDatabaseOCID, state, backupTimeout)() } } -func AssertRemoteStateOCID(k8sClient *client.Client, dbClient *database.DatabaseClient, adbID *string, state database.AutonomousDatabaseLifecycleStateEnum, timeout time.Duration) func() { +func AssertADBRemoteStateOCID(k8sClient *client.Client, dbClient *database.DatabaseClient, adbID *string, state database.AutonomousDatabaseLifecycleStateEnum, timeout time.Duration) func() { return func() { Expect(k8sClient).NotTo(BeNil()) Expect(dbClient).NotTo(BeNil()) @@ -638,7 +639,7 @@ func AssertRemoteStateOCID(k8sClient *client.Client, dbClient *database.Database By("Checking if the lifecycleState of the ADB in OCI is " + string(state)) Eventually(func() (database.AutonomousDatabaseLifecycleStateEnum, error) { - return returnRemoteState(derefK8sClient, derefDBClient, adbID, nil) + return returnADBRemoteState(derefK8sClient, derefDBClient, adbID, nil) }, timeout, intervalTime).Should(Equal(state)) } } @@ -660,7 +661,7 @@ func UpdateState(k8sClient *client.Client, adbLookupKey *types.NamespacedName, s } } -func returnLocalState(k8sClient client.Client, adbLookupKey types.NamespacedName) (database.AutonomousDatabaseLifecycleStateEnum, error) { +func returnADBLocalState(k8sClient client.Client, adbLookupKey types.NamespacedName) (database.AutonomousDatabaseLifecycleStateEnum, error) { adb := &dbv1alpha1.AutonomousDatabase{} err := k8sClient.Get(context.TODO(), adbLookupKey, adb) if err != nil { @@ -669,7 +670,7 @@ func returnLocalState(k8sClient client.Client, adbLookupKey types.NamespacedName return adb.Status.LifecycleState, nil } -func returnRemoteState(k8sClient client.Client, dbClient database.DatabaseClient, adbID *string, retryPolicy *common.RetryPolicy) (database.AutonomousDatabaseLifecycleStateEnum, error) { +func returnADBRemoteState(k8sClient client.Client, dbClient database.DatabaseClient, adbID *string, retryPolicy *common.RetryPolicy) (database.AutonomousDatabaseLifecycleStateEnum, error) { resp, err := e2eutil.GetAutonomousDatabase(dbClient, adbID, retryPolicy) if err != nil { return "", err @@ -677,6 +678,23 @@ func returnRemoteState(k8sClient client.Client, dbClient database.DatabaseClient return resp.LifecycleState, nil } +func returnACDLocalState(k8sClient client.Client, acdLookupKey types.NamespacedName) (database.AutonomousContainerDatabaseLifecycleStateEnum, error) { + acd := &dbv1alpha1.AutonomousContainerDatabase{} + err := k8sClient.Get(context.TODO(), acdLookupKey, acd) + if err != nil { + return "", err + } + return acd.Status.LifecycleState, nil +} + +func returnACDRemoteState(k8sClient client.Client, dbClient database.DatabaseClient, acdID *string, retryPolicy *common.RetryPolicy) (database.AutonomousContainerDatabaseLifecycleStateEnum, error) { + resp, err := e2eutil.GetAutonomousContainerDatabase(dbClient, acdID, retryPolicy) + if err != nil { + return "", err + } + return resp.LifecycleState, nil +} + /* Runs a script that connects to an ADB and configures the backup bucket */ func ConfigureADBBackup(dbClient *database.DatabaseClient, databaseOCID *string, tnsEntry *string, adminPassword *string, walletPassword *string, bucket *string, authToken *string, ociUser *string) error { @@ -715,10 +733,10 @@ func AssertBackupRestore(k8sClient *client.Client, dbClient *database.DatabaseCl // for ~7 minutes. After that time, the state should return to AVAILBLE derefK8sClient := *k8sClient - AssertRemoteState(k8sClient, dbClient, adbLookupKey, state)() + AssertADBRemoteState(k8sClient, dbClient, adbLookupKey, state)() By("Wait until ADB state returns to AVAILABLE") - AssertRemoteStateForBackupRestore(k8sClient, dbClient, adbLookupKey, database.AutonomousDatabaseLifecycleStateAvailable)() + AssertADBRemoteStateForBackupRestore(k8sClient, dbClient, adbLookupKey, database.AutonomousDatabaseLifecycleStateAvailable)() if state == database.AutonomousDatabaseLifecycleStateBackupInProgress { By("Checking adb backup State is ACTIVE") @@ -737,3 +755,214 @@ func AssertBackupRestore(k8sClient *client.Client, dbClient *database.DatabaseCl } } } + +func AssertACDState(k8sClient *client.Client, dbClient *database.DatabaseClient, acdLookupKey *types.NamespacedName, state database.AutonomousContainerDatabaseLifecycleStateEnum, timeout time.Duration) func() { + return func() { + AssertACDLocalState(k8sClient, acdLookupKey, state, timeout)() + AssertACDRemoteState(k8sClient, dbClient, acdLookupKey, state, timeout)() + } +} + +func AssertACDLocalState(k8sClient *client.Client, acdLookupKey *types.NamespacedName, state database.AutonomousContainerDatabaseLifecycleStateEnum, timeout time.Duration) func() { + return func() { + Expect(k8sClient).NotTo(BeNil()) + Expect(acdLookupKey).NotTo(BeNil()) + + derefK8sClient := *k8sClient + + By("Checking if the lifecycleState of local resource is " + string(state)) + Eventually(func() (database.AutonomousContainerDatabaseLifecycleStateEnum, error) { + return returnACDLocalState(derefK8sClient, *acdLookupKey) + }, timeout).Should(Equal(state)) + } +} + +func AssertACDRemoteState(k8sClient *client.Client, dbClient *database.DatabaseClient, acdLookupKey *types.NamespacedName, state database.AutonomousContainerDatabaseLifecycleStateEnum, timeout time.Duration) func() { + return func() { + derefK8sClient := *k8sClient + + acd := &dbv1alpha1.AutonomousContainerDatabase{} + Expect(derefK8sClient.Get(context.TODO(), *acdLookupKey, acd)).To(Succeed()) + By("Checking if the lifecycleState of remote resource is " + string(state)) + AssertACDRemoteStateOCID(k8sClient, dbClient, acd.Spec.AutonomousContainerDatabaseOCID, state, timeout)() + } +} + +func AssertACDRemoteStateOCID(k8sClient *client.Client, dbClient *database.DatabaseClient, acdID *string, state database.AutonomousContainerDatabaseLifecycleStateEnum, timeout time.Duration) func() { + return func() { + Expect(k8sClient).NotTo(BeNil()) + Expect(dbClient).NotTo(BeNil()) + Expect(acdID).NotTo(BeNil()) + + fmt.Fprintf(GinkgoWriter, "ACD ID is %s\n", *acdID) + + derefK8sClient := *k8sClient + derefDBClient := *dbClient + + By("Checking if the lifecycleState of the ACD in OCI is " + string(state)) + Eventually(func() (database.AutonomousContainerDatabaseLifecycleStateEnum, error) { + return returnACDRemoteState(derefK8sClient, derefDBClient, acdID, nil) + }, timeout, intervalTime).Should(Equal(state)) + } +} + +func AssertACDBind(k8sClient *client.Client, dbClient *database.DatabaseClient, acdLookupKey *types.NamespacedName, state database.AutonomousContainerDatabaseLifecycleStateEnum) func() { + return func() { + + // ACD state should be AVAILABLE + acdBindTimeout := time.Minute * 3 + + By("Wait until ACD is in state AVAILABLE") + AssertACDState(k8sClient, dbClient, acdLookupKey, state, acdBindTimeout)() + } +} + +func UpdateAndAssertACDSpec(k8sClient *client.Client, dbClient *database.DatabaseClient, acdLookupKey *types.NamespacedName) func() { + return func() { + expectedACD := UpdateACDSpec(k8sClient, dbClient, acdLookupKey)() + AssertACDSpec(k8sClient, dbClient, acdLookupKey, expectedACD)() + } +} + +func UpdateACDSpec(k8sClient *client.Client, dbClient *database.DatabaseClient, acdLookupKey *types.NamespacedName) func() *dbv1alpha1.AutonomousContainerDatabase { + return func() *dbv1alpha1.AutonomousContainerDatabase { + Expect(k8sClient).NotTo(BeNil()) + Expect(dbClient).NotTo(BeNil()) + + derefK8sClient := *k8sClient + expectedAcd := &dbv1alpha1.AutonomousContainerDatabase{} + Expect(derefK8sClient.Get(context.TODO(), *acdLookupKey, expectedAcd)).To(Succeed()) + + expectedAcd.Spec.DisplayName = common.String(*expectedAcd.Spec.DisplayName + "_new") + + Expect(derefK8sClient.Update(context.TODO(), expectedAcd)).To(Succeed()) + return expectedAcd + } +} + +func AssertACDSpec(k8sClient *client.Client, dbClient *database.DatabaseClient, acdLookupKey *types.NamespacedName, expectedACD *dbv1alpha1.AutonomousContainerDatabase) func() { + return func() { + Expect(k8sClient).NotTo(BeNil()) + Expect(dbClient).NotTo(BeNil()) + Expect(acdLookupKey).NotTo(BeNil()) + + derefDBClient := *dbClient + + expectedACDSpec := expectedACD.Spec + Eventually(func() (bool, error) { + // Fetch the ACD from OCI when it's in AVAILABLE state, and retry if its attributes doesn't match the new ACD's attributes + retryPolicy := e2eutil.NewLifecycleStateRetryPolicyACD(database.AutonomousContainerDatabaseLifecycleStateAvailable) + resp, err := e2eutil.GetAutonomousContainerDatabase(derefDBClient, expectedACD.Spec.AutonomousContainerDatabaseOCID, &retryPolicy) + if err != nil { + return false, err + } + + debug := true + if debug { + if !compareString(expectedACDSpec.AutonomousContainerDatabaseOCID, resp.AutonomousContainerDatabase.Id) { + fmt.Fprintf(GinkgoWriter, "Expected OCID: %v\nGot: %v\n", expectedACDSpec.AutonomousContainerDatabaseOCID, resp.AutonomousContainerDatabase.Id) + } + if !compareString(expectedACDSpec.CompartmentOCID, resp.AutonomousContainerDatabase.CompartmentId) { + fmt.Fprintf(GinkgoWriter, "Expected CompartmentOCID: %v\nGot: %v\n", expectedACDSpec.CompartmentOCID, resp.CompartmentId) + } + if !compareString(expectedACDSpec.DisplayName, resp.AutonomousContainerDatabase.DisplayName) { + fmt.Fprintf(GinkgoWriter, "Expected DisplayName: %v\nGot: %v\n", expectedACDSpec.DisplayName, resp.AutonomousContainerDatabase.DisplayName) + } + if !compareString(expectedACDSpec.AutonomousExadataVMClusterOCID, resp.AutonomousContainerDatabase.CloudAutonomousVmClusterId) { + fmt.Fprintf(GinkgoWriter, "Expected AutonomousExadataVMClusterOCID: %v\nGot: %v\n", expectedACDSpec.AutonomousExadataVMClusterOCID, resp.AutonomousContainerDatabase.CloudAutonomousVmClusterId) + } + if !compareStringMap(expectedACDSpec.FreeformTags, resp.AutonomousContainerDatabase.FreeformTags) { + fmt.Fprintf(GinkgoWriter, "Expected FreeformTags: %v\nGot: %v\n", expectedACDSpec.FreeformTags, resp.AutonomousContainerDatabase.FreeformTags) + } + if expectedACDSpec.PatchModel != resp.AutonomousContainerDatabase.PatchModel { + fmt.Fprintf(GinkgoWriter, "Expected PatchModel: %v\nGot: %v\n", expectedACDSpec.PatchModel, resp.AutonomousContainerDatabase.PatchModel) + } + } + + // Compare the elements one by one rather than doing reflect.DeelEqual(adb1, adb2), since some parameters + // (e.g. adminPassword, wallet) are missing from e2eutil.GetAutonomousDatabase(). + // We don't compare LifecycleState in this case. We only make sure that the ADB is in AVAIABLE state before + // proceeding to the next test. + same := compareString(expectedACDSpec.AutonomousContainerDatabaseOCID, resp.AutonomousContainerDatabase.Id) && + compareString(expectedACDSpec.CompartmentOCID, resp.AutonomousContainerDatabase.CompartmentId) && + compareString(expectedACDSpec.DisplayName, resp.AutonomousContainerDatabase.DisplayName) && + compareString(expectedACDSpec.AutonomousExadataVMClusterOCID, resp.AutonomousContainerDatabase.CloudAutonomousVmClusterId) && + compareStringMap(expectedACDSpec.FreeformTags, resp.AutonomousContainerDatabase.FreeformTags) && + expectedACDSpec.PatchModel == resp.AutonomousContainerDatabase.PatchModel + + return same, nil + }, updateACDTimeout, intervalTime).Should(BeTrue()) + + // IMPORTANT: make sure the local resource has finished reconciling, otherwise the changes will + // be conflicted with the next test and cause unknow result. + AssertACDLocalState(k8sClient, acdLookupKey, database.AutonomousContainerDatabaseLifecycleStateAvailable, time.Minute*2)() + } +} + +func AssertACDRestart(k8sClient *client.Client, dbClient *database.DatabaseClient, acdLookupKey *types.NamespacedName) func() { + return func() { + Expect(k8sClient).NotTo(BeNil()) + Expect(dbClient).NotTo(BeNil()) + Expect(acdLookupKey).NotTo(BeNil()) + + derefK8sClient := *k8sClient + acd := &dbv1alpha1.AutonomousContainerDatabase{} + Expect(derefK8sClient.Get(context.TODO(), *acdLookupKey, acd)).To(Succeed()) + + acd.Spec.Action = dbv1alpha1.AcdActionRestart + + Expect(derefK8sClient.Update(context.TODO(), acd)) + + // Check ACD status is RESTARTING + AssertACDState(k8sClient, dbClient, acdLookupKey, database.AutonomousContainerDatabaseLifecycleStateRestarting, time.Minute*2)() + // Wait until restart is completed + AssertACDState(k8sClient, dbClient, acdLookupKey, database.AutonomousContainerDatabaseLifecycleStateAvailable, time.Minute*7)() + } +} + +func AssertACDTerminate(k8sClient *client.Client, dbClient *database.DatabaseClient, acdLookupKey *types.NamespacedName) func() { + return func() { + Expect(k8sClient).NotTo(BeNil()) + Expect(dbClient).NotTo(BeNil()) + Expect(acdLookupKey).NotTo(BeNil()) + + derefK8sClient := *k8sClient + acd := &dbv1alpha1.AutonomousContainerDatabase{} + Expect(derefK8sClient.Get(context.TODO(), *acdLookupKey, acd)) + + acd.Spec.Action = dbv1alpha1.AcdActionTerminate + Expect(derefK8sClient.Update(context.TODO(), acd)) + + // Check ACD status is TERMINATING + AssertACDState(k8sClient, dbClient, acdLookupKey, database.AutonomousContainerDatabaseLifecycleStateTerminating, time.Minute*2)() + // Wait until status is TERMINATED + AssertACDState(k8sClient, dbClient, acdLookupKey, database.AutonomousContainerDatabaseLifecycleStateTerminated, time.Minute*40)() + } +} + +func AssertACDLocalDelete(k8sClient *client.Client, dbClient *database.DatabaseClient, acdLookupKey *types.NamespacedName) func() { + return func() { + Expect(k8sClient).NotTo(BeNil()) + Expect(dbClient).NotTo(BeNil()) + Expect(acdLookupKey).NotTo(BeNil()) + + derefK8sClient := *k8sClient + existingAcd := &dbv1alpha1.AutonomousContainerDatabase{} + Expect(derefK8sClient.Get(context.TODO(), *acdLookupKey, existingAcd)).To(Succeed()) + Expect(derefK8sClient.Delete(context.TODO(), existingAcd)) + + By("Checking if the AutonomousContainerDatabase resource is deleted") + Eventually(func() (isDeleted bool) { + acd := &dbv1alpha1.AutonomousContainerDatabase{} + isDeleted = false + err := derefK8sClient.Get(context.TODO(), *acdLookupKey, acd) + if err != nil && k8sErrors.IsNotFound(err) { + isDeleted = true + return + } + return + }, changeTimeout, intervalTime).Should(Equal(true)) + + AssertACDRemoteState(k8sClient, dbClient, acdLookupKey, database.AutonomousContainerDatabaseLifecycleStateAvailable, time.Minute*2) + } +} diff --git a/test/e2e/resource/test_config.yaml b/test/e2e/resource/test_config.yaml index 7191045c..475b3351 100644 --- a/test/e2e/resource/test_config.yaml +++ b/test/e2e/resource/test_config.yaml @@ -23,4 +23,6 @@ bucketURL: https://swiftobjectstorage.region.oraclecloud.com/v1/namespace-string # The auth token generated in OCI Console > Profile > User Settings > Auth Token authToken: token # The OCI user used to login to OCI Console -ociUser: user \ No newline at end of file +ociUser: user +# The Autonomous Exadata VM Cluster used for AutonomousContainerDatabase provision +exadataVMClusterOCID: ocid1.autonomousexainfrastructure... \ No newline at end of file diff --git a/test/e2e/suite_test.go b/test/e2e/suite_test.go index 3d98330b..f5cd5af5 100644 --- a/test/e2e/suite_test.go +++ b/test/e2e/suite_test.go @@ -121,6 +121,7 @@ var SharedNsgOCID string var SharedBucketUrl string var SharedAuthToken string var SharedOciUser string +var SharedExadataVMClusterOCID string const SharedAdminPassSecretName string = "adb-admin-password" const SharedNewAdminPassSecretName string = "new-adb-admin-password" @@ -222,6 +223,7 @@ var _ = BeforeSuite(func() { SharedBucketUrl = testConfig.BucketURL SharedAuthToken = testConfig.AuthToken SharedOciUser = testConfig.OciUser + SharedExadataVMClusterOCID = testConfig.ExadataVMClusterOCID By("checking if the required parameters exist") Expect(testConfig.OCIConfigFile).ToNot(Equal("")) @@ -233,6 +235,7 @@ var _ = BeforeSuite(func() { Expect(testConfig.BucketURL).ToNot(Equal("")) Expect(testConfig.AuthToken).ToNot(Equal("")) Expect(testConfig.OciUser).ToNot(Equal("")) + Expect(testConfig.ExadataVMClusterOCID).ToNot(Equal("")) By("getting OCI provider") ociConfigUtil, err := e2eutil.GetOCIConfigUtil(testConfig.OCIConfigFile, testConfig.Profile) diff --git a/test/e2e/util/oci_acd_request.go b/test/e2e/util/oci_acd_request.go new file mode 100644 index 00000000..a798d29f --- /dev/null +++ b/test/e2e/util/oci_acd_request.go @@ -0,0 +1,78 @@ +/* +** Copyright (c) 2021 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package e2eutil + +import ( + "context" + + "github.com/oracle/oci-go-sdk/v63/common" + "github.com/oracle/oci-go-sdk/v63/database" + // "io" + // "io/ioutil" + // "time" +) + +func CreateAutonomousContainerDatabase(dbClient database.DatabaseClient, compartmentId *string, acdName *string, exadataVmClusterID *string) (response database.CreateAutonomousContainerDatabaseResponse, err error) { + acdDetails := database.CreateAutonomousContainerDatabaseDetails{ + DisplayName: acdName, + CloudAutonomousVmClusterId: exadataVmClusterID, + CompartmentId: compartmentId, + PatchModel: database.CreateAutonomousContainerDatabaseDetailsPatchModelUpdates, + } + + createACDRequest := database.CreateAutonomousContainerDatabaseRequest{ + CreateAutonomousContainerDatabaseDetails: acdDetails, + } + + return dbClient.CreateAutonomousContainerDatabase(context.Background(), createACDRequest) +} + +func GetAutonomousContainerDatabase(dbClient database.DatabaseClient, acdOCID *string, retryPolicy *common.RetryPolicy) (database.GetAutonomousContainerDatabaseResponse, error) { + getRequest := database.GetAutonomousContainerDatabaseRequest{ + AutonomousContainerDatabaseId: acdOCID, + } + + if retryPolicy != nil { + getRequest.RequestMetadata = common.RequestMetadata{ + RetryPolicy: retryPolicy, + } + } + + return dbClient.GetAutonomousContainerDatabase(context.TODO(), getRequest) +} diff --git a/test/e2e/util/oci_db_request.go b/test/e2e/util/oci_db_request.go index d4ed212a..f22b0be8 100644 --- a/test/e2e/util/oci_db_request.go +++ b/test/e2e/util/oci_db_request.go @@ -131,7 +131,7 @@ func generateRetryPolicy(retryFunc func(r common.OCIOperationResponse) bool) com return common.NewRetryPolicy(attempts, retryFunc, nextDuration) } -func NewLifecycleStateRetryPolicy(lifecycleState database.AutonomousDatabaseLifecycleStateEnum) common.RetryPolicy { +func NewLifecycleStateRetryPolicyADB(lifecycleState database.AutonomousDatabaseLifecycleStateEnum) common.RetryPolicy { shouldRetry := func(r common.OCIOperationResponse) bool { if databaseResponse, ok := r.Response.(database.GetAutonomousDatabaseResponse); ok { // do the retry until lifecycle state reaches the passed terminal state @@ -142,6 +142,17 @@ func NewLifecycleStateRetryPolicy(lifecycleState database.AutonomousDatabaseLife return generateRetryPolicy(shouldRetry) } +func NewLifecycleStateRetryPolicyACD(lifecycleState database.AutonomousContainerDatabaseLifecycleStateEnum) common.RetryPolicy { + shouldRetry := func(r common.OCIOperationResponse) bool { + if databaseResponse, ok := r.Response.(database.GetAutonomousContainerDatabaseResponse); ok { + // do the retry until lifecycle state reaches the passed terminal state + return databaseResponse.LifecycleState != lifecycleState + } + return true + } + return generateRetryPolicy(shouldRetry) +} + func DownloadWalletZip(dbClient database.DatabaseClient, databaseOCID *string, walletPassword *string) (string, error) { req := database.GenerateAutonomousDatabaseWalletRequest{ diff --git a/test/e2e/util/util.go b/test/e2e/util/util.go index b1791df6..340b2e07 100644 --- a/test/e2e/util/util.go +++ b/test/e2e/util/util.go @@ -56,12 +56,20 @@ import ( // GenerateDBName returns a string DB concatenate 14 digits of the date time // E.g., DB060102150405 if the curret date-time is 2006.01.02 15:04:05 func GenerateDBName() string { + return "DB" + getDateTimeString() +} + +func GenerateACDName() string { + return "ACD" + getDateTimeString() +} + +func getDateTimeString() string { timeString := time.Now().Format("2006.01.02 15:04:05") trimmed := strings.ReplaceAll(timeString, ":", "") // remove colons trimmed = strings.ReplaceAll(trimmed, ".", "") // remove dots trimmed = strings.ReplaceAll(trimmed, " ", "") // remove spaces trimmed = trimmed[2:] // remove the first two digits of year (2006 -> 06) - return "DB" + trimmed + return trimmed } func unmarshalFromYamlBytes(bytes []byte, obj interface{}) error { @@ -124,6 +132,7 @@ type testConfiguration struct { BucketURL string `yaml:"bucketURL"` AuthToken string `yaml:"authToken"` OciUser string `yaml:"ociUser"` + ExadataVMClusterOCID string `yaml:"exadataVMClusterOCID"` } func GetTestConfig(filename string) (*testConfiguration, error) { From d62e6dc4ef2415d5506a52a023b73c25967f637e Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Wed, 11 May 2022 19:45:26 +0000 Subject: [PATCH 376/628] Remove irrelevant sentence. --- docs/adb/ADB_MANUAL_BACKUP.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/docs/adb/ADB_MANUAL_BACKUP.md b/docs/adb/ADB_MANUAL_BACKUP.md index 6d831915..ecf75fea 100644 --- a/docs/adb/ADB_MANUAL_BACKUP.md +++ b/docs/adb/ADB_MANUAL_BACKUP.md @@ -4,8 +4,6 @@ To create manual backups of Autonomous Databases, use this procedure. Oracle Cloud Infrastructure (OCI) automatically backs up your Autonomous Databases, and retains these backups for 60 days. You can restore and recover your database to any point-in-time in this retention period. Automatic backups are full backups taken every 60 days, with daily incremental backups. You can also create manual backups to supplement your automatic backups. Manual backups are stored in an Object Storage bucket that you create, and are retained for 60 days. For more information, please visit [this page](https://docs.oracle.com/en-us/iaas/Content/Database/Tasks/adbbackingup.htm). -In every OCI reconciliation loop, the operator obtains the list of the `AutonomousBackupOCIDs`. If an Autonomous Database backup resource with the same OCID (`AutonomousBackupOCID`) doesn't exist in the cluster, then OCI also creates the `AutonomousDatabaseBackup` resource automatically. - ## Prerequisites To hold your Autonomous Database manual backups, you must create an Oracle Cloud Infrastructure Object Storage bucket, and configure your database to connect to it. To finish setting up manual backup storage, follow the steps in this page: [Setting Up a Bucket to Store Manual Backups](https://docs.oracle.com/en-us/iaas/Content/Database/Tasks/adbbackingup.htm#creatingbucket). Creating an Autonomous Database manual backup object storage bucket is a one-time operation. From 632ae29d16802a280a532da637c1f81c3795cba2 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Thu, 12 May 2022 09:34:25 -0400 Subject: [PATCH 377/628] update copyright --- Dockerfile | 2 +- LICENSE.txt | 2 +- Makefile | 2 +- README.md | 2 +- apis/database/v1alpha1/adbfamily_common_utils.go | 2 +- apis/database/v1alpha1/autonomouscontainerdatabase_types.go | 2 +- apis/database/v1alpha1/autonomouscontainerdatabase_webhook.go | 2 +- .../v1alpha1/autonomouscontainerdatabase_webhook_test.go | 2 +- apis/database/v1alpha1/autonomousdatabase_types.go | 2 +- apis/database/v1alpha1/autonomousdatabase_webhook.go | 2 +- apis/database/v1alpha1/autonomousdatabase_webhook_test.go | 2 +- apis/database/v1alpha1/autonomousdatabasebackup_types.go | 2 +- apis/database/v1alpha1/autonomousdatabasebackup_webhook.go | 2 +- .../v1alpha1/autonomousdatabasebackup_webhook_test.go | 2 +- apis/database/v1alpha1/autonomousdatabaserestore_types.go | 2 +- apis/database/v1alpha1/autonomousdatabaserestore_webhook.go | 2 +- .../v1alpha1/autonomousdatabaserestore_webhook_test.go | 2 +- apis/database/v1alpha1/cdb_types.go | 2 +- apis/database/v1alpha1/cdb_webhook.go | 2 +- apis/database/v1alpha1/dbcssystem_types.go | 2 +- apis/database/v1alpha1/groupversion_info.go | 2 +- apis/database/v1alpha1/oraclerestdataservice_webhook.go | 2 +- apis/database/v1alpha1/pdb_types.go | 2 +- apis/database/v1alpha1/pdb_webhook.go | 2 +- apis/database/v1alpha1/shardingdatabase_types.go | 2 +- apis/database/v1alpha1/singleinstancedatabase_types.go | 2 +- apis/database/v1alpha1/webhook_suite_test.go | 2 +- apis/database/v1alpha1/zz_generated.deepcopy.go | 2 +- bundle.Dockerfile | 2 +- commons/adb_family/utils.go | 2 +- commons/annotations/annotations.go | 2 +- commons/database/utils.go | 2 +- commons/dbcssystem/dbcs_reconciler.go | 2 +- commons/dbcssystem/dcommon.go | 2 +- commons/finalizer/finalizer.go | 2 +- commons/k8s/create.go | 2 +- commons/k8s/fetch.go | 2 +- commons/k8s/finalizer.go | 2 +- commons/k8s/utils.go | 2 +- commons/oci/containerdatabase.go | 2 +- commons/oci/database.go | 2 +- commons/oci/provider.go | 2 +- commons/oci/vault.go | 2 +- commons/oci/wallet.go | 2 +- commons/oci/workrequest.go | 2 +- commons/sharding/catalog.go | 2 +- commons/sharding/exec.go | 2 +- commons/sharding/gsm.go | 2 +- commons/sharding/provstatus.go | 2 +- commons/sharding/scommon.go | 2 +- commons/sharding/shard.go | 2 +- config/certmanager/certificate.yaml | 2 +- config/certmanager/kustomization.yaml | 2 +- config/certmanager/kustomizeconfig.yaml | 2 +- config/crd/kustomization.yaml | 2 +- config/crd/kustomizeconfig.yaml | 2 +- config/crd/patches/cainjection_in_autonomousdatabases.yaml | 2 +- config/crd/patches/cainjection_in_shardingdatabases.yaml | 2 +- .../crd/patches/cainjection_in_singleinstancedatabases.yaml | 2 +- config/crd/patches/webhook_in_autonomousdatabases.yaml | 2 +- config/crd/patches/webhook_in_shardingdatabases.yaml | 2 +- config/crd/patches/webhook_in_singleinstancedatabases.yaml | 2 +- config/default/kustomization.yaml | 2 +- config/default/manager_auth_proxy_patch.yaml | 2 +- config/default/manager_webhook_patch.yaml | 2 +- config/default/webhookcainjection_patch.yaml | 2 +- config/manager/kustomization.yaml | 2 +- config/manager/manager.yaml | 2 +- config/manifests/kustomization.yaml | 2 +- config/rbac/auth_proxy_client_clusterrole.yaml | 2 +- config/rbac/auth_proxy_role.yaml | 2 +- config/rbac/auth_proxy_role_binding.yaml | 2 +- config/rbac/auth_proxy_service.yaml | 2 +- config/rbac/autonomousdatabase_editor_role.yaml | 2 +- config/rbac/autonomousdatabase_viewer_role.yaml | 2 +- config/rbac/kustomization.yaml | 2 +- config/rbac/leader_election_role.yaml | 2 +- config/rbac/leader_election_role_binding.yaml | 2 +- config/rbac/provshard_editor_role.yaml | 2 +- config/rbac/provshard_viewer_role.yaml | 2 +- config/rbac/role_binding.yaml | 2 +- config/rbac/shardingdatabase_editor_role.yaml | 2 +- config/rbac/shardingdatabase_viewer_role.yaml | 2 +- config/rbac/singleinstancedatabase_editor_role.yaml | 2 +- config/rbac/singleinstancedatabase_viewer_role.yaml | 2 +- config/samples/acd/autonomouscontainerdatabase_bind.yaml | 2 +- .../acd/autonomouscontainerdatabase_change_displayname.yaml | 2 +- config/samples/acd/autonomouscontainerdatabase_create.yaml | 2 +- .../acd/autonomouscontainerdatabase_delete_resource.yaml | 2 +- .../acd/autonomouscontainerdatabase_restart_terminate.yaml | 2 +- config/samples/adb/autonomousdatabase_backup.yaml | 2 +- config/samples/adb/autonomousdatabase_bind.yaml | 2 +- config/samples/adb/autonomousdatabase_create.yaml | 2 +- config/samples/adb/autonomousdatabase_delete_resource.yaml | 2 +- config/samples/adb/autonomousdatabase_rename.yaml | 2 +- config/samples/adb/autonomousdatabase_restore.yaml | 4 ++-- config/samples/adb/autonomousdatabase_scale.yaml | 2 +- .../samples/adb/autonomousdatabase_stop_start_terminate.yaml | 2 +- .../samples/adb/autonomousdatabase_update_admin_password.yaml | 2 +- config/samples/adb/autonomousdatabase_update_mtls.yaml | 2 +- .../samples/adb/autonomousdatabase_update_network_access.yaml | 2 +- config/samples/adb/autonomousdatabase_wallet.yaml | 2 +- config/samples/kustomization.yaml | 2 +- config/samples/onpremdb/cdb.yaml | 2 +- config/samples/onpremdb/cdb_secret.yaml | 2 +- config/samples/onpremdb/pdb.yaml | 2 +- config/samples/onpremdb/pdb_clone.yaml | 2 +- config/samples/onpremdb/pdb_delete.yaml | 2 +- config/samples/onpremdb/pdb_map.yaml | 2 +- config/samples/onpremdb/pdb_modify.yaml | 2 +- config/samples/onpremdb/pdb_plug.yaml | 2 +- config/samples/onpremdb/pdb_secret.yaml | 2 +- config/samples/onpremdb/pdb_unplug.yaml | 2 +- config/samples/sharding/sharding_v1alpha1_provshard.yaml | 2 +- .../sharding/sharding_v1alpha1_provshard_clonespec.yaml | 2 +- .../sharding/sharding_v1alpha1_provshard_clonespec1.yaml | 2 +- config/samples/sharding/sharding_v1alpha1_provshard_orig.yaml | 2 +- config/samples/sharding/shardingdatabase.yaml | 2 +- config/samples/sidb/openshift_rbac.yaml | 2 +- config/samples/sidb/oraclerestdataservice.yaml | 2 +- config/samples/sidb/singleinstancedatabase.yaml | 2 +- config/scorecard/bases/config.yaml | 2 +- config/scorecard/kustomization.yaml | 2 +- config/scorecard/patches/basic.config.yaml | 2 +- config/scorecard/patches/olm.config.yaml | 2 +- config/webhook/kustomization.yaml | 2 +- config/webhook/kustomizeconfig.yaml | 2 +- config/webhook/service.yaml | 2 +- .../database/autonomouscontainerdatabase_controller.go | 4 ++-- controllers/database/autonomousdatabase_controller.go | 4 ++-- controllers/database/autonomousdatabasebackup_controller.go | 2 +- controllers/database/autonomousdatabaserestore_controller.go | 2 +- controllers/database/cdb_controller.go | 2 +- controllers/database/dbcssystem_controller.go | 2 +- controllers/database/oraclerestdataservice_controller.go | 2 +- controllers/database/pdb_controller.go | 2 +- controllers/database/shardingdatabase_controller.go | 2 +- controllers/database/suite_test.go | 2 +- docs/adb/ADB_RESTORE.md | 2 +- docs/sharding/provisioning/oraclesi.yaml | 2 +- docs/sharding/provisioning/oraclesi_pvc_commented.yaml | 2 +- docs/sharding/provisioning/shard_prov.yaml | 2 +- docs/sharding/provisioning/shard_prov_clone.yaml | 2 +- docs/sharding/provisioning/shard_prov_clone_across_ads.yaml | 2 +- docs/sharding/provisioning/shard_prov_delshard.yaml | 2 +- docs/sharding/provisioning/shard_prov_extshard.yaml | 2 +- docs/sharding/provisioning/shard_prov_memory_cpu.yaml | 2 +- docs/sharding/provisioning/shard_prov_send_notification.yaml | 2 +- hack/boilerplate.go.txt | 2 +- main.go | 2 +- ords/setupwebuser.sh | 2 +- set_ocicredentials.sh | 2 +- test/e2e/autonomouscontainerdatabase_test.go | 2 +- test/e2e/autonomousdatabase_controller_bind_test.go | 2 +- test/e2e/autonomousdatabase_controller_create_test.go | 2 +- test/e2e/behavior/shared_behaviors.go | 2 +- test/e2e/resource/test_config.yaml | 2 +- test/e2e/suite_test.go | 2 +- test/e2e/util/oci_acd_request.go | 2 +- test/e2e/util/oci_config_util.go | 2 +- test/e2e/util/oci_db_request.go | 2 +- test/e2e/util/oci_work_request.go | 2 +- test/e2e/util/util.go | 2 +- 163 files changed, 166 insertions(+), 166 deletions(-) diff --git a/Dockerfile b/Dockerfile index dc6742f9..01c2bf7b 100644 --- a/Dockerfile +++ b/Dockerfile @@ -1,4 +1,4 @@ -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl. # diff --git a/LICENSE.txt b/LICENSE.txt index 4ac08f59..56dbf679 100644 --- a/LICENSE.txt +++ b/LICENSE.txt @@ -1,4 +1,4 @@ -Copyright (c) 2021 Oracle and/or its affiliates. +Copyright (c) 2022 Oracle and/or its affiliates. The Universal Permissive License (UPL), Version 1.0 diff --git a/Makefile b/Makefile index 3d963755..d7a42e32 100644 --- a/Makefile +++ b/Makefile @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # diff --git a/README.md b/README.md index 0b6433f4..8476dbd4 100644 --- a/README.md +++ b/README.md @@ -155,5 +155,5 @@ See [Reporting security vulnerabilities](./SECURITY.md) ## License -Copyright (c) 2021 Oracle and/or its affiliates. +Copyright (c) 2022 Oracle and/or its affiliates. Released under the Universal Permissive License v1.0 as shown at [https://oss.oracle.com/licenses/upl/](https://oss.oracle.com/licenses/upl/) diff --git a/apis/database/v1alpha1/adbfamily_common_utils.go b/apis/database/v1alpha1/adbfamily_common_utils.go index 2dc6c69b..a5dd3018 100644 --- a/apis/database/v1alpha1/adbfamily_common_utils.go +++ b/apis/database/v1alpha1/adbfamily_common_utils.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/apis/database/v1alpha1/autonomouscontainerdatabase_types.go b/apis/database/v1alpha1/autonomouscontainerdatabase_types.go index b69156d4..cdc037ef 100644 --- a/apis/database/v1alpha1/autonomouscontainerdatabase_types.go +++ b/apis/database/v1alpha1/autonomouscontainerdatabase_types.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/apis/database/v1alpha1/autonomouscontainerdatabase_webhook.go b/apis/database/v1alpha1/autonomouscontainerdatabase_webhook.go index 03b09d1a..3e9f5e1f 100644 --- a/apis/database/v1alpha1/autonomouscontainerdatabase_webhook.go +++ b/apis/database/v1alpha1/autonomouscontainerdatabase_webhook.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/apis/database/v1alpha1/autonomouscontainerdatabase_webhook_test.go b/apis/database/v1alpha1/autonomouscontainerdatabase_webhook_test.go index abdc8264..1decc7b9 100644 --- a/apis/database/v1alpha1/autonomouscontainerdatabase_webhook_test.go +++ b/apis/database/v1alpha1/autonomouscontainerdatabase_webhook_test.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/apis/database/v1alpha1/autonomousdatabase_types.go b/apis/database/v1alpha1/autonomousdatabase_types.go index f16ca76f..bfa1fe4b 100644 --- a/apis/database/v1alpha1/autonomousdatabase_types.go +++ b/apis/database/v1alpha1/autonomousdatabase_types.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/apis/database/v1alpha1/autonomousdatabase_webhook.go b/apis/database/v1alpha1/autonomousdatabase_webhook.go index 5b0e34f3..cc805bec 100644 --- a/apis/database/v1alpha1/autonomousdatabase_webhook.go +++ b/apis/database/v1alpha1/autonomousdatabase_webhook.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/apis/database/v1alpha1/autonomousdatabase_webhook_test.go b/apis/database/v1alpha1/autonomousdatabase_webhook_test.go index 69d25c31..ce812938 100644 --- a/apis/database/v1alpha1/autonomousdatabase_webhook_test.go +++ b/apis/database/v1alpha1/autonomousdatabase_webhook_test.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/apis/database/v1alpha1/autonomousdatabasebackup_types.go b/apis/database/v1alpha1/autonomousdatabasebackup_types.go index 586a27a3..52974622 100644 --- a/apis/database/v1alpha1/autonomousdatabasebackup_types.go +++ b/apis/database/v1alpha1/autonomousdatabasebackup_types.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/apis/database/v1alpha1/autonomousdatabasebackup_webhook.go b/apis/database/v1alpha1/autonomousdatabasebackup_webhook.go index 8b19940e..c08c8c58 100644 --- a/apis/database/v1alpha1/autonomousdatabasebackup_webhook.go +++ b/apis/database/v1alpha1/autonomousdatabasebackup_webhook.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/apis/database/v1alpha1/autonomousdatabasebackup_webhook_test.go b/apis/database/v1alpha1/autonomousdatabasebackup_webhook_test.go index 56c79bc0..d382ee60 100644 --- a/apis/database/v1alpha1/autonomousdatabasebackup_webhook_test.go +++ b/apis/database/v1alpha1/autonomousdatabasebackup_webhook_test.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/apis/database/v1alpha1/autonomousdatabaserestore_types.go b/apis/database/v1alpha1/autonomousdatabaserestore_types.go index 3a22fb35..05583305 100644 --- a/apis/database/v1alpha1/autonomousdatabaserestore_types.go +++ b/apis/database/v1alpha1/autonomousdatabaserestore_types.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/apis/database/v1alpha1/autonomousdatabaserestore_webhook.go b/apis/database/v1alpha1/autonomousdatabaserestore_webhook.go index ed6e2edb..8b4dd974 100644 --- a/apis/database/v1alpha1/autonomousdatabaserestore_webhook.go +++ b/apis/database/v1alpha1/autonomousdatabaserestore_webhook.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/apis/database/v1alpha1/autonomousdatabaserestore_webhook_test.go b/apis/database/v1alpha1/autonomousdatabaserestore_webhook_test.go index 50974800..332aa47b 100644 --- a/apis/database/v1alpha1/autonomousdatabaserestore_webhook_test.go +++ b/apis/database/v1alpha1/autonomousdatabaserestore_webhook_test.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/apis/database/v1alpha1/cdb_types.go b/apis/database/v1alpha1/cdb_types.go index 1d323c20..ccf5b0f5 100644 --- a/apis/database/v1alpha1/cdb_types.go +++ b/apis/database/v1alpha1/cdb_types.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/apis/database/v1alpha1/cdb_webhook.go b/apis/database/v1alpha1/cdb_webhook.go index 756b5b07..2d591223 100644 --- a/apis/database/v1alpha1/cdb_webhook.go +++ b/apis/database/v1alpha1/cdb_webhook.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/apis/database/v1alpha1/dbcssystem_types.go b/apis/database/v1alpha1/dbcssystem_types.go index dd03b28c..28773fb2 100644 --- a/apis/database/v1alpha1/dbcssystem_types.go +++ b/apis/database/v1alpha1/dbcssystem_types.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/apis/database/v1alpha1/groupversion_info.go b/apis/database/v1alpha1/groupversion_info.go index 60029108..d72ee908 100644 --- a/apis/database/v1alpha1/groupversion_info.go +++ b/apis/database/v1alpha1/groupversion_info.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/apis/database/v1alpha1/oraclerestdataservice_webhook.go b/apis/database/v1alpha1/oraclerestdataservice_webhook.go index ed73c9e9..37259007 100644 --- a/apis/database/v1alpha1/oraclerestdataservice_webhook.go +++ b/apis/database/v1alpha1/oraclerestdataservice_webhook.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/apis/database/v1alpha1/pdb_types.go b/apis/database/v1alpha1/pdb_types.go index 03f63f01..c5b95c81 100644 --- a/apis/database/v1alpha1/pdb_types.go +++ b/apis/database/v1alpha1/pdb_types.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/apis/database/v1alpha1/pdb_webhook.go b/apis/database/v1alpha1/pdb_webhook.go index 12ea899a..b02b797d 100644 --- a/apis/database/v1alpha1/pdb_webhook.go +++ b/apis/database/v1alpha1/pdb_webhook.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/apis/database/v1alpha1/shardingdatabase_types.go b/apis/database/v1alpha1/shardingdatabase_types.go index 50058e48..b2f7142e 100644 --- a/apis/database/v1alpha1/shardingdatabase_types.go +++ b/apis/database/v1alpha1/shardingdatabase_types.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/apis/database/v1alpha1/singleinstancedatabase_types.go b/apis/database/v1alpha1/singleinstancedatabase_types.go index 9687fb1b..d0391a4b 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_types.go +++ b/apis/database/v1alpha1/singleinstancedatabase_types.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/apis/database/v1alpha1/webhook_suite_test.go b/apis/database/v1alpha1/webhook_suite_test.go index b0cdde57..343f279c 100644 --- a/apis/database/v1alpha1/webhook_suite_test.go +++ b/apis/database/v1alpha1/webhook_suite_test.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/apis/database/v1alpha1/zz_generated.deepcopy.go b/apis/database/v1alpha1/zz_generated.deepcopy.go index 700da778..56a312e0 100644 --- a/apis/database/v1alpha1/zz_generated.deepcopy.go +++ b/apis/database/v1alpha1/zz_generated.deepcopy.go @@ -2,7 +2,7 @@ // +build !ignore_autogenerated /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/bundle.Dockerfile b/bundle.Dockerfile index 4ca3c52f..d591c4ef 100644 --- a/bundle.Dockerfile +++ b/bundle.Dockerfile @@ -1,4 +1,4 @@ -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl. # diff --git a/commons/adb_family/utils.go b/commons/adb_family/utils.go index 248784f3..8218502e 100644 --- a/commons/adb_family/utils.go +++ b/commons/adb_family/utils.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/commons/annotations/annotations.go b/commons/annotations/annotations.go index 9c354c0c..b0196156 100644 --- a/commons/annotations/annotations.go +++ b/commons/annotations/annotations.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/commons/database/utils.go b/commons/database/utils.go index 1231cdf0..c93f7696 100644 --- a/commons/database/utils.go +++ b/commons/database/utils.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/commons/dbcssystem/dbcs_reconciler.go b/commons/dbcssystem/dbcs_reconciler.go index 753a2667..878dd94e 100644 --- a/commons/dbcssystem/dbcs_reconciler.go +++ b/commons/dbcssystem/dbcs_reconciler.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/commons/dbcssystem/dcommon.go b/commons/dbcssystem/dcommon.go index 51dedb00..373951df 100644 --- a/commons/dbcssystem/dcommon.go +++ b/commons/dbcssystem/dcommon.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/commons/finalizer/finalizer.go b/commons/finalizer/finalizer.go index bafe110e..169cbaef 100644 --- a/commons/finalizer/finalizer.go +++ b/commons/finalizer/finalizer.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/commons/k8s/create.go b/commons/k8s/create.go index ad522504..68258139 100644 --- a/commons/k8s/create.go +++ b/commons/k8s/create.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/commons/k8s/fetch.go b/commons/k8s/fetch.go index b8d51b23..05792cad 100644 --- a/commons/k8s/fetch.go +++ b/commons/k8s/fetch.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/commons/k8s/finalizer.go b/commons/k8s/finalizer.go index 0344feb7..015bbc61 100644 --- a/commons/k8s/finalizer.go +++ b/commons/k8s/finalizer.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/commons/k8s/utils.go b/commons/k8s/utils.go index 092357a4..37ec1a3f 100644 --- a/commons/k8s/utils.go +++ b/commons/k8s/utils.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/commons/oci/containerdatabase.go b/commons/oci/containerdatabase.go index b8a0ad1e..b8ba6ce6 100644 --- a/commons/oci/containerdatabase.go +++ b/commons/oci/containerdatabase.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/commons/oci/database.go b/commons/oci/database.go index 2686a86d..372c9631 100644 --- a/commons/oci/database.go +++ b/commons/oci/database.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/commons/oci/provider.go b/commons/oci/provider.go index 83dee0a0..1609103d 100644 --- a/commons/oci/provider.go +++ b/commons/oci/provider.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/commons/oci/vault.go b/commons/oci/vault.go index 4b5e5982..83722eca 100644 --- a/commons/oci/vault.go +++ b/commons/oci/vault.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/commons/oci/wallet.go b/commons/oci/wallet.go index 818ce7c0..27ebb150 100644 --- a/commons/oci/wallet.go +++ b/commons/oci/wallet.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/commons/oci/workrequest.go b/commons/oci/workrequest.go index 03b97398..541902c5 100644 --- a/commons/oci/workrequest.go +++ b/commons/oci/workrequest.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/commons/sharding/catalog.go b/commons/sharding/catalog.go index 406250e9..f87efea8 100644 --- a/commons/sharding/catalog.go +++ b/commons/sharding/catalog.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/commons/sharding/exec.go b/commons/sharding/exec.go index 24e912b6..48fc18cf 100644 --- a/commons/sharding/exec.go +++ b/commons/sharding/exec.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/commons/sharding/gsm.go b/commons/sharding/gsm.go index ae1f94e2..b6c1f0a8 100644 --- a/commons/sharding/gsm.go +++ b/commons/sharding/gsm.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/commons/sharding/provstatus.go b/commons/sharding/provstatus.go index 4245fc07..6c34b007 100644 --- a/commons/sharding/provstatus.go +++ b/commons/sharding/provstatus.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/commons/sharding/scommon.go b/commons/sharding/scommon.go index 04cd5079..9392fba0 100644 --- a/commons/sharding/scommon.go +++ b/commons/sharding/scommon.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/commons/sharding/shard.go b/commons/sharding/shard.go index 6a000a14..ecebf606 100644 --- a/commons/sharding/shard.go +++ b/commons/sharding/shard.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/config/certmanager/certificate.yaml b/config/certmanager/certificate.yaml index 3aadc474..6b331894 100644 --- a/config/certmanager/certificate.yaml +++ b/config/certmanager/certificate.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # diff --git a/config/certmanager/kustomization.yaml b/config/certmanager/kustomization.yaml index 8dd6746c..0a7c8089 100644 --- a/config/certmanager/kustomization.yaml +++ b/config/certmanager/kustomization.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # resources: diff --git a/config/certmanager/kustomizeconfig.yaml b/config/certmanager/kustomizeconfig.yaml index f8e232fa..ce54850c 100644 --- a/config/certmanager/kustomizeconfig.yaml +++ b/config/certmanager/kustomizeconfig.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # diff --git a/config/crd/kustomization.yaml b/config/crd/kustomization.yaml index 19983b29..1cee363a 100644 --- a/config/crd/kustomization.yaml +++ b/config/crd/kustomization.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # # This kustomization.yaml is not intended to be run by itself, diff --git a/config/crd/kustomizeconfig.yaml b/config/crd/kustomizeconfig.yaml index fb9995dc..344a1576 100644 --- a/config/crd/kustomizeconfig.yaml +++ b/config/crd/kustomizeconfig.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # # This file is for teaching kustomize how to substitute name and namespace reference in CRD diff --git a/config/crd/patches/cainjection_in_autonomousdatabases.yaml b/config/crd/patches/cainjection_in_autonomousdatabases.yaml index 072e3f9e..05842d0b 100644 --- a/config/crd/patches/cainjection_in_autonomousdatabases.yaml +++ b/config/crd/patches/cainjection_in_autonomousdatabases.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # # The following patch adds a directive for certmanager to inject CA into the CRD diff --git a/config/crd/patches/cainjection_in_shardingdatabases.yaml b/config/crd/patches/cainjection_in_shardingdatabases.yaml index 6ef22218..45d35376 100644 --- a/config/crd/patches/cainjection_in_shardingdatabases.yaml +++ b/config/crd/patches/cainjection_in_shardingdatabases.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # # The following patch adds a directive for certmanager to inject CA into the CRD diff --git a/config/crd/patches/cainjection_in_singleinstancedatabases.yaml b/config/crd/patches/cainjection_in_singleinstancedatabases.yaml index 4e454c1a..11114339 100644 --- a/config/crd/patches/cainjection_in_singleinstancedatabases.yaml +++ b/config/crd/patches/cainjection_in_singleinstancedatabases.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # # The following patch adds a directive for certmanager to inject CA into the CRD diff --git a/config/crd/patches/webhook_in_autonomousdatabases.yaml b/config/crd/patches/webhook_in_autonomousdatabases.yaml index 55540503..230f9f68 100644 --- a/config/crd/patches/webhook_in_autonomousdatabases.yaml +++ b/config/crd/patches/webhook_in_autonomousdatabases.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # # The following patch enables conversion webhook for CRD diff --git a/config/crd/patches/webhook_in_shardingdatabases.yaml b/config/crd/patches/webhook_in_shardingdatabases.yaml index fccda7d0..b006a011 100644 --- a/config/crd/patches/webhook_in_shardingdatabases.yaml +++ b/config/crd/patches/webhook_in_shardingdatabases.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # # The following patch enables conversion webhook for CRD diff --git a/config/crd/patches/webhook_in_singleinstancedatabases.yaml b/config/crd/patches/webhook_in_singleinstancedatabases.yaml index 66687d8e..aecc7ba9 100644 --- a/config/crd/patches/webhook_in_singleinstancedatabases.yaml +++ b/config/crd/patches/webhook_in_singleinstancedatabases.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # # The following patch enables conversion webhook for CRD diff --git a/config/default/kustomization.yaml b/config/default/kustomization.yaml index 04f37b2c..d41001b0 100644 --- a/config/default/kustomization.yaml +++ b/config/default/kustomization.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # diff --git a/config/default/manager_auth_proxy_patch.yaml b/config/default/manager_auth_proxy_patch.yaml index c88bb7a7..9bd3bbc9 100644 --- a/config/default/manager_auth_proxy_patch.yaml +++ b/config/default/manager_auth_proxy_patch.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # diff --git a/config/default/manager_webhook_patch.yaml b/config/default/manager_webhook_patch.yaml index cda36c4c..b34405d2 100644 --- a/config/default/manager_webhook_patch.yaml +++ b/config/default/manager_webhook_patch.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: apps/v1 diff --git a/config/default/webhookcainjection_patch.yaml b/config/default/webhookcainjection_patch.yaml index 51b1069f..c6b7ea34 100644 --- a/config/default/webhookcainjection_patch.yaml +++ b/config/default/webhookcainjection_patch.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # diff --git a/config/manager/kustomization.yaml b/config/manager/kustomization.yaml index 30ed1f75..b8ec1f58 100644 --- a/config/manager/kustomization.yaml +++ b/config/manager/kustomization.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # resources: diff --git a/config/manager/manager.yaml b/config/manager/manager.yaml index 18412546..90df4158 100644 --- a/config/manager/manager.yaml +++ b/config/manager/manager.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: v1 diff --git a/config/manifests/kustomization.yaml b/config/manifests/kustomization.yaml index 2a0f628b..39275249 100644 --- a/config/manifests/kustomization.yaml +++ b/config/manifests/kustomization.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # resources: diff --git a/config/rbac/auth_proxy_client_clusterrole.yaml b/config/rbac/auth_proxy_client_clusterrole.yaml index b3e55375..1a478f0d 100644 --- a/config/rbac/auth_proxy_client_clusterrole.yaml +++ b/config/rbac/auth_proxy_client_clusterrole.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: rbac.authorization.k8s.io/v1 diff --git a/config/rbac/auth_proxy_role.yaml b/config/rbac/auth_proxy_role.yaml index fd7483d6..e275bf6d 100644 --- a/config/rbac/auth_proxy_role.yaml +++ b/config/rbac/auth_proxy_role.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: rbac.authorization.k8s.io/v1 diff --git a/config/rbac/auth_proxy_role_binding.yaml b/config/rbac/auth_proxy_role_binding.yaml index 5632b87b..55665e32 100644 --- a/config/rbac/auth_proxy_role_binding.yaml +++ b/config/rbac/auth_proxy_role_binding.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: rbac.authorization.k8s.io/v1 diff --git a/config/rbac/auth_proxy_service.yaml b/config/rbac/auth_proxy_service.yaml index 72b1ecb8..8cca79bf 100644 --- a/config/rbac/auth_proxy_service.yaml +++ b/config/rbac/auth_proxy_service.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: v1 diff --git a/config/rbac/autonomousdatabase_editor_role.yaml b/config/rbac/autonomousdatabase_editor_role.yaml index bd6172fd..4cc7959a 100644 --- a/config/rbac/autonomousdatabase_editor_role.yaml +++ b/config/rbac/autonomousdatabase_editor_role.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # diff --git a/config/rbac/autonomousdatabase_viewer_role.yaml b/config/rbac/autonomousdatabase_viewer_role.yaml index af24143f..089bf01e 100644 --- a/config/rbac/autonomousdatabase_viewer_role.yaml +++ b/config/rbac/autonomousdatabase_viewer_role.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # diff --git a/config/rbac/kustomization.yaml b/config/rbac/kustomization.yaml index 6325cf8c..7a20231c 100644 --- a/config/rbac/kustomization.yaml +++ b/config/rbac/kustomization.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # resources: diff --git a/config/rbac/leader_election_role.yaml b/config/rbac/leader_election_role.yaml index 083cd887..68f2270f 100644 --- a/config/rbac/leader_election_role.yaml +++ b/config/rbac/leader_election_role.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # diff --git a/config/rbac/leader_election_role_binding.yaml b/config/rbac/leader_election_role_binding.yaml index 01f5d630..180bd383 100644 --- a/config/rbac/leader_election_role_binding.yaml +++ b/config/rbac/leader_election_role_binding.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: rbac.authorization.k8s.io/v1 diff --git a/config/rbac/provshard_editor_role.yaml b/config/rbac/provshard_editor_role.yaml index 473a41c0..1df44f2c 100644 --- a/config/rbac/provshard_editor_role.yaml +++ b/config/rbac/provshard_editor_role.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # diff --git a/config/rbac/provshard_viewer_role.yaml b/config/rbac/provshard_viewer_role.yaml index 36b5bb4b..a4ef06ed 100644 --- a/config/rbac/provshard_viewer_role.yaml +++ b/config/rbac/provshard_viewer_role.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # diff --git a/config/rbac/role_binding.yaml b/config/rbac/role_binding.yaml index 1ec61dc5..1abe0ec7 100644 --- a/config/rbac/role_binding.yaml +++ b/config/rbac/role_binding.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: rbac.authorization.k8s.io/v1 diff --git a/config/rbac/shardingdatabase_editor_role.yaml b/config/rbac/shardingdatabase_editor_role.yaml index f9660cb7..efe6ad75 100644 --- a/config/rbac/shardingdatabase_editor_role.yaml +++ b/config/rbac/shardingdatabase_editor_role.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # diff --git a/config/rbac/shardingdatabase_viewer_role.yaml b/config/rbac/shardingdatabase_viewer_role.yaml index 717c61a4..08b8ca26 100644 --- a/config/rbac/shardingdatabase_viewer_role.yaml +++ b/config/rbac/shardingdatabase_viewer_role.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # diff --git a/config/rbac/singleinstancedatabase_editor_role.yaml b/config/rbac/singleinstancedatabase_editor_role.yaml index 90a19c43..918ef991 100644 --- a/config/rbac/singleinstancedatabase_editor_role.yaml +++ b/config/rbac/singleinstancedatabase_editor_role.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # diff --git a/config/rbac/singleinstancedatabase_viewer_role.yaml b/config/rbac/singleinstancedatabase_viewer_role.yaml index 84bea03d..c1f0f469 100644 --- a/config/rbac/singleinstancedatabase_viewer_role.yaml +++ b/config/rbac/singleinstancedatabase_viewer_role.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # diff --git a/config/samples/acd/autonomouscontainerdatabase_bind.yaml b/config/samples/acd/autonomouscontainerdatabase_bind.yaml index 6a270f9b..3d28ba4d 100644 --- a/config/samples/acd/autonomouscontainerdatabase_bind.yaml +++ b/config/samples/acd/autonomouscontainerdatabase_bind.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: database.oracle.com/v1alpha1 diff --git a/config/samples/acd/autonomouscontainerdatabase_change_displayname.yaml b/config/samples/acd/autonomouscontainerdatabase_change_displayname.yaml index 5e844ede..dd75250d 100644 --- a/config/samples/acd/autonomouscontainerdatabase_change_displayname.yaml +++ b/config/samples/acd/autonomouscontainerdatabase_change_displayname.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: database.oracle.com/v1alpha1 diff --git a/config/samples/acd/autonomouscontainerdatabase_create.yaml b/config/samples/acd/autonomouscontainerdatabase_create.yaml index 31bc0684..5f42a136 100644 --- a/config/samples/acd/autonomouscontainerdatabase_create.yaml +++ b/config/samples/acd/autonomouscontainerdatabase_create.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: database.oracle.com/v1alpha1 diff --git a/config/samples/acd/autonomouscontainerdatabase_delete_resource.yaml b/config/samples/acd/autonomouscontainerdatabase_delete_resource.yaml index 499ee21b..5be06b5a 100644 --- a/config/samples/acd/autonomouscontainerdatabase_delete_resource.yaml +++ b/config/samples/acd/autonomouscontainerdatabase_delete_resource.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: database.oracle.com/v1alpha1 diff --git a/config/samples/acd/autonomouscontainerdatabase_restart_terminate.yaml b/config/samples/acd/autonomouscontainerdatabase_restart_terminate.yaml index 436a6185..0e884f6e 100644 --- a/config/samples/acd/autonomouscontainerdatabase_restart_terminate.yaml +++ b/config/samples/acd/autonomouscontainerdatabase_restart_terminate.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: database.oracle.com/v1alpha1 diff --git a/config/samples/adb/autonomousdatabase_backup.yaml b/config/samples/adb/autonomousdatabase_backup.yaml index 750fa763..ac3fb779 100644 --- a/config/samples/adb/autonomousdatabase_backup.yaml +++ b/config/samples/adb/autonomousdatabase_backup.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: database.oracle.com/v1alpha1 diff --git a/config/samples/adb/autonomousdatabase_bind.yaml b/config/samples/adb/autonomousdatabase_bind.yaml index d52b9c18..8d1de0fe 100644 --- a/config/samples/adb/autonomousdatabase_bind.yaml +++ b/config/samples/adb/autonomousdatabase_bind.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: database.oracle.com/v1alpha1 diff --git a/config/samples/adb/autonomousdatabase_create.yaml b/config/samples/adb/autonomousdatabase_create.yaml index 9db481ad..3ecf966f 100644 --- a/config/samples/adb/autonomousdatabase_create.yaml +++ b/config/samples/adb/autonomousdatabase_create.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: database.oracle.com/v1alpha1 diff --git a/config/samples/adb/autonomousdatabase_delete_resource.yaml b/config/samples/adb/autonomousdatabase_delete_resource.yaml index e075787e..60a8fe5c 100644 --- a/config/samples/adb/autonomousdatabase_delete_resource.yaml +++ b/config/samples/adb/autonomousdatabase_delete_resource.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: database.oracle.com/v1alpha1 diff --git a/config/samples/adb/autonomousdatabase_rename.yaml b/config/samples/adb/autonomousdatabase_rename.yaml index 8aa2ae8b..d3a29998 100644 --- a/config/samples/adb/autonomousdatabase_rename.yaml +++ b/config/samples/adb/autonomousdatabase_rename.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: database.oracle.com/v1alpha1 diff --git a/config/samples/adb/autonomousdatabase_restore.yaml b/config/samples/adb/autonomousdatabase_restore.yaml index 74c36061..3db8a1b6 100644 --- a/config/samples/adb/autonomousdatabase_restore.yaml +++ b/config/samples/adb/autonomousdatabase_restore.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: database.oracle.com/v1alpha1 @@ -21,7 +21,7 @@ spec: # # Uncomment the following field to perform point-in-time restore # pointInTime: # # The timestamp must follow this format: YYYY-MM-DD HH:MM:SS GMT - # timestamp: 2021-12-23 11:03:13 UTC + # timestamp: 2022-12-23 11:03:13 UTC # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. ociConfig: diff --git a/config/samples/adb/autonomousdatabase_scale.yaml b/config/samples/adb/autonomousdatabase_scale.yaml index 4a7c85e9..cd100675 100644 --- a/config/samples/adb/autonomousdatabase_scale.yaml +++ b/config/samples/adb/autonomousdatabase_scale.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: database.oracle.com/v1alpha1 diff --git a/config/samples/adb/autonomousdatabase_stop_start_terminate.yaml b/config/samples/adb/autonomousdatabase_stop_start_terminate.yaml index 83d831cd..a41f8d0b 100644 --- a/config/samples/adb/autonomousdatabase_stop_start_terminate.yaml +++ b/config/samples/adb/autonomousdatabase_stop_start_terminate.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: database.oracle.com/v1alpha1 diff --git a/config/samples/adb/autonomousdatabase_update_admin_password.yaml b/config/samples/adb/autonomousdatabase_update_admin_password.yaml index 70b952b5..67fd142d 100644 --- a/config/samples/adb/autonomousdatabase_update_admin_password.yaml +++ b/config/samples/adb/autonomousdatabase_update_admin_password.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: database.oracle.com/v1alpha1 diff --git a/config/samples/adb/autonomousdatabase_update_mtls.yaml b/config/samples/adb/autonomousdatabase_update_mtls.yaml index f067f4c7..93db0d8a 100644 --- a/config/samples/adb/autonomousdatabase_update_mtls.yaml +++ b/config/samples/adb/autonomousdatabase_update_mtls.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: database.oracle.com/v1alpha1 diff --git a/config/samples/adb/autonomousdatabase_update_network_access.yaml b/config/samples/adb/autonomousdatabase_update_network_access.yaml index ef7145eb..0762efcb 100644 --- a/config/samples/adb/autonomousdatabase_update_network_access.yaml +++ b/config/samples/adb/autonomousdatabase_update_network_access.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: database.oracle.com/v1alpha1 diff --git a/config/samples/adb/autonomousdatabase_wallet.yaml b/config/samples/adb/autonomousdatabase_wallet.yaml index 64925483..15fa6ca0 100644 --- a/config/samples/adb/autonomousdatabase_wallet.yaml +++ b/config/samples/adb/autonomousdatabase_wallet.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: database.oracle.com/v1alpha1 diff --git a/config/samples/kustomization.yaml b/config/samples/kustomization.yaml index a726efa1..d3d73391 100644 --- a/config/samples/kustomization.yaml +++ b/config/samples/kustomization.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # diff --git a/config/samples/onpremdb/cdb.yaml b/config/samples/onpremdb/cdb.yaml index a7f71a7d..3a5452b5 100644 --- a/config/samples/onpremdb/cdb.yaml +++ b/config/samples/onpremdb/cdb.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. All rights reserved. +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: database.oracle.com/v1alpha1 diff --git a/config/samples/onpremdb/cdb_secret.yaml b/config/samples/onpremdb/cdb_secret.yaml index 856a653b..5cc8b351 100644 --- a/config/samples/onpremdb/cdb_secret.yaml +++ b/config/samples/onpremdb/cdb_secret.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. All rights reserved. +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: v1 diff --git a/config/samples/onpremdb/pdb.yaml b/config/samples/onpremdb/pdb.yaml index 71641e9c..2be31acf 100644 --- a/config/samples/onpremdb/pdb.yaml +++ b/config/samples/onpremdb/pdb.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. All rights reserved. +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: database.oracle.com/v1alpha1 diff --git a/config/samples/onpremdb/pdb_clone.yaml b/config/samples/onpremdb/pdb_clone.yaml index c88db391..f36e904d 100644 --- a/config/samples/onpremdb/pdb_clone.yaml +++ b/config/samples/onpremdb/pdb_clone.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. All rights reserved. +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: database.oracle.com/v1alpha1 diff --git a/config/samples/onpremdb/pdb_delete.yaml b/config/samples/onpremdb/pdb_delete.yaml index 3532c023..6c5299c0 100644 --- a/config/samples/onpremdb/pdb_delete.yaml +++ b/config/samples/onpremdb/pdb_delete.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. All rights reserved. +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: database.oracle.com/v1alpha1 diff --git a/config/samples/onpremdb/pdb_map.yaml b/config/samples/onpremdb/pdb_map.yaml index be543f75..1501d690 100644 --- a/config/samples/onpremdb/pdb_map.yaml +++ b/config/samples/onpremdb/pdb_map.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. All rights reserved. +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: database.oracle.com/v1alpha1 diff --git a/config/samples/onpremdb/pdb_modify.yaml b/config/samples/onpremdb/pdb_modify.yaml index 7fc298ef..feac2dbf 100644 --- a/config/samples/onpremdb/pdb_modify.yaml +++ b/config/samples/onpremdb/pdb_modify.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. All rights reserved. +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: database.oracle.com/v1alpha1 diff --git a/config/samples/onpremdb/pdb_plug.yaml b/config/samples/onpremdb/pdb_plug.yaml index 4a9fef98..b48c4ffc 100644 --- a/config/samples/onpremdb/pdb_plug.yaml +++ b/config/samples/onpremdb/pdb_plug.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. All rights reserved. +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: database.oracle.com/v1alpha1 diff --git a/config/samples/onpremdb/pdb_secret.yaml b/config/samples/onpremdb/pdb_secret.yaml index 65a5fabe..ca0d21dd 100644 --- a/config/samples/onpremdb/pdb_secret.yaml +++ b/config/samples/onpremdb/pdb_secret.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. All rights reserved. +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: v1 diff --git a/config/samples/onpremdb/pdb_unplug.yaml b/config/samples/onpremdb/pdb_unplug.yaml index 89b4d086..21d7b187 100644 --- a/config/samples/onpremdb/pdb_unplug.yaml +++ b/config/samples/onpremdb/pdb_unplug.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. All rights reserved. +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: database.oracle.com/v1alpha1 diff --git a/config/samples/sharding/sharding_v1alpha1_provshard.yaml b/config/samples/sharding/sharding_v1alpha1_provshard.yaml index 60f5f007..a51993e6 100644 --- a/config/samples/sharding/sharding_v1alpha1_provshard.yaml +++ b/config/samples/sharding/sharding_v1alpha1_provshard.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: sharding.oracle.com/v1alpha1 diff --git a/config/samples/sharding/sharding_v1alpha1_provshard_clonespec.yaml b/config/samples/sharding/sharding_v1alpha1_provshard_clonespec.yaml index 414544bd..acbf5a9c 100644 --- a/config/samples/sharding/sharding_v1alpha1_provshard_clonespec.yaml +++ b/config/samples/sharding/sharding_v1alpha1_provshard_clonespec.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: sharding.oracle.com/v1alpha1 diff --git a/config/samples/sharding/sharding_v1alpha1_provshard_clonespec1.yaml b/config/samples/sharding/sharding_v1alpha1_provshard_clonespec1.yaml index 67988b2c..5bca5b85 100644 --- a/config/samples/sharding/sharding_v1alpha1_provshard_clonespec1.yaml +++ b/config/samples/sharding/sharding_v1alpha1_provshard_clonespec1.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: sharding.oracle.com/v1alpha1 diff --git a/config/samples/sharding/sharding_v1alpha1_provshard_orig.yaml b/config/samples/sharding/sharding_v1alpha1_provshard_orig.yaml index fc7ba66d..0300d6ce 100644 --- a/config/samples/sharding/sharding_v1alpha1_provshard_orig.yaml +++ b/config/samples/sharding/sharding_v1alpha1_provshard_orig.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: sharding.oracle.com/v1alpha1 diff --git a/config/samples/sharding/shardingdatabase.yaml b/config/samples/sharding/shardingdatabase.yaml index 40453b49..b639800f 100644 --- a/config/samples/sharding/shardingdatabase.yaml +++ b/config/samples/sharding/shardingdatabase.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: database.oracle.com/v1alpha1 diff --git a/config/samples/sidb/openshift_rbac.yaml b/config/samples/sidb/openshift_rbac.yaml index 30fde8fb..ded5ef69 100644 --- a/config/samples/sidb/openshift_rbac.yaml +++ b/config/samples/sidb/openshift_rbac.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # diff --git a/config/samples/sidb/oraclerestdataservice.yaml b/config/samples/sidb/oraclerestdataservice.yaml index 9a34d2fa..f32fbe56 100644 --- a/config/samples/sidb/oraclerestdataservice.yaml +++ b/config/samples/sidb/oraclerestdataservice.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. All rights reserved. +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: database.oracle.com/v1alpha1 diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index 4388d802..8d6ed3c2 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: database.oracle.com/v1alpha1 diff --git a/config/scorecard/bases/config.yaml b/config/scorecard/bases/config.yaml index 0650fef2..fbd8c506 100644 --- a/config/scorecard/bases/config.yaml +++ b/config/scorecard/bases/config.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # kind: Configuration diff --git a/config/scorecard/kustomization.yaml b/config/scorecard/kustomization.yaml index d9c19e9b..bf4c1e7c 100644 --- a/config/scorecard/kustomization.yaml +++ b/config/scorecard/kustomization.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # resources: diff --git a/config/scorecard/patches/basic.config.yaml b/config/scorecard/patches/basic.config.yaml index 67fa78c9..516ab755 100644 --- a/config/scorecard/patches/basic.config.yaml +++ b/config/scorecard/patches/basic.config.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # - op: add diff --git a/config/scorecard/patches/olm.config.yaml b/config/scorecard/patches/olm.config.yaml index 521bc8e6..40e4fbe8 100644 --- a/config/scorecard/patches/olm.config.yaml +++ b/config/scorecard/patches/olm.config.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # - op: add diff --git a/config/webhook/kustomization.yaml b/config/webhook/kustomization.yaml index ef3ca64c..f78631f3 100644 --- a/config/webhook/kustomization.yaml +++ b/config/webhook/kustomization.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # resources: diff --git a/config/webhook/kustomizeconfig.yaml b/config/webhook/kustomizeconfig.yaml index afc69aba..972d71bb 100644 --- a/config/webhook/kustomizeconfig.yaml +++ b/config/webhook/kustomizeconfig.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # diff --git a/config/webhook/service.yaml b/config/webhook/service.yaml index 08333c36..f93ad4d2 100644 --- a/config/webhook/service.yaml +++ b/config/webhook/service.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: v1 diff --git a/controllers/database/autonomouscontainerdatabase_controller.go b/controllers/database/autonomouscontainerdatabase_controller.go index c74ac6da..bd0a0ff5 100644 --- a/controllers/database/autonomouscontainerdatabase_controller.go +++ b/controllers/database/autonomouscontainerdatabase_controller.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** @@ -182,7 +182,7 @@ func (r *AutonomousContainerDatabaseReconciler) Reconcile(ctx context.Context, r * Deletion timestamp will be added to a object before it is deleted. * Kubernetes server calls the clean up function if a finalizer exitsts, and won't delete the real object until * all the finalizers are removed from the object metadata. - * Refer to this page for more details of using finalizers: https://kubernetes.io/blog/2021/05/14/using-finalizers-to-control-deletion/ + * Refer to this page for more details of using finalizers: https://kubernetes.io/blog/2022/05/14/using-finalizers-to-control-deletion/ ******************************************************************/ exitReconcile, err := r.validateCleanup(logger, acd) if err != nil { diff --git a/controllers/database/autonomousdatabase_controller.go b/controllers/database/autonomousdatabase_controller.go index bfbd4bc1..b0552e10 100644 --- a/controllers/database/autonomousdatabase_controller.go +++ b/controllers/database/autonomousdatabase_controller.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** @@ -221,7 +221,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R * Deletion timestamp will be added to a object before it is deleted. * Kubernetes server calls the clean up function if a finalizer exitsts, and won't delete the real object until * all the finalizers are removed from the object metadata. - * Refer to this page for more details of using finalizers: https://kubernetes.io/blog/2021/05/14/using-finalizers-to-control-deletion/ + * Refer to this page for more details of using finalizers: https://kubernetes.io/blog/2022/05/14/using-finalizers-to-control-deletion/ ******************************************************************/ exitReconcile, err := r.validateCleanup(logger, desiredADB) if err != nil { diff --git a/controllers/database/autonomousdatabasebackup_controller.go b/controllers/database/autonomousdatabasebackup_controller.go index 4632ffff..7fac9e04 100644 --- a/controllers/database/autonomousdatabasebackup_controller.go +++ b/controllers/database/autonomousdatabasebackup_controller.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/controllers/database/autonomousdatabaserestore_controller.go b/controllers/database/autonomousdatabaserestore_controller.go index 561d4884..26dc33d0 100644 --- a/controllers/database/autonomousdatabaserestore_controller.go +++ b/controllers/database/autonomousdatabaserestore_controller.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/controllers/database/cdb_controller.go b/controllers/database/cdb_controller.go index c2a3d967..9be4477b 100644 --- a/controllers/database/cdb_controller.go +++ b/controllers/database/cdb_controller.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/controllers/database/dbcssystem_controller.go b/controllers/database/dbcssystem_controller.go index 9f731061..fdefd40d 100644 --- a/controllers/database/dbcssystem_controller.go +++ b/controllers/database/dbcssystem_controller.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 4c500c51..35ada6c2 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/controllers/database/pdb_controller.go b/controllers/database/pdb_controller.go index 931ec556..42a21b51 100644 --- a/controllers/database/pdb_controller.go +++ b/controllers/database/pdb_controller.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/controllers/database/shardingdatabase_controller.go b/controllers/database/shardingdatabase_controller.go index 6f2a54cd..4258276d 100644 --- a/controllers/database/shardingdatabase_controller.go +++ b/controllers/database/shardingdatabase_controller.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/controllers/database/suite_test.go b/controllers/database/suite_test.go index 60a0d659..6c6772b4 100644 --- a/controllers/database/suite_test.go +++ b/controllers/database/suite_test.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/docs/adb/ADB_RESTORE.md b/docs/adb/ADB_RESTORE.md index 8fc51592..a7f67888 100644 --- a/docs/adb/ADB_RESTORE.md +++ b/docs/adb/ADB_RESTORE.md @@ -37,7 +37,7 @@ To restore an Autonomous Database from a backup, or by using point-in-time resto name: autonomousdatabasebackup-sample # # Uncomment the following field to perform point-in-time restore # pointInTime: - # timestamp: 2021-12-23 11:03:13 UTC + # timestamp: 2022-12-23 11:03:13 UTC ociConfig: configMapName: oci-cred secretName: oci-privatekey diff --git a/docs/sharding/provisioning/oraclesi.yaml b/docs/sharding/provisioning/oraclesi.yaml index ffc5734b..2e0607b7 100644 --- a/docs/sharding/provisioning/oraclesi.yaml +++ b/docs/sharding/provisioning/oraclesi.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: v1 diff --git a/docs/sharding/provisioning/oraclesi_pvc_commented.yaml b/docs/sharding/provisioning/oraclesi_pvc_commented.yaml index a6facda0..89707a08 100644 --- a/docs/sharding/provisioning/oraclesi_pvc_commented.yaml +++ b/docs/sharding/provisioning/oraclesi_pvc_commented.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # diff --git a/docs/sharding/provisioning/shard_prov.yaml b/docs/sharding/provisioning/shard_prov.yaml index 508192d7..032abd68 100644 --- a/docs/sharding/provisioning/shard_prov.yaml +++ b/docs/sharding/provisioning/shard_prov.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: database.oracle.com/v1alpha1 diff --git a/docs/sharding/provisioning/shard_prov_clone.yaml b/docs/sharding/provisioning/shard_prov_clone.yaml index 0acc0ec5..f9d3f826 100644 --- a/docs/sharding/provisioning/shard_prov_clone.yaml +++ b/docs/sharding/provisioning/shard_prov_clone.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: database.oracle.com/v1alpha1 diff --git a/docs/sharding/provisioning/shard_prov_clone_across_ads.yaml b/docs/sharding/provisioning/shard_prov_clone_across_ads.yaml index 98d9ee56..6b815541 100644 --- a/docs/sharding/provisioning/shard_prov_clone_across_ads.yaml +++ b/docs/sharding/provisioning/shard_prov_clone_across_ads.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: database.oracle.com/v1alpha1 diff --git a/docs/sharding/provisioning/shard_prov_delshard.yaml b/docs/sharding/provisioning/shard_prov_delshard.yaml index 91dfdd28..09cbd549 100644 --- a/docs/sharding/provisioning/shard_prov_delshard.yaml +++ b/docs/sharding/provisioning/shard_prov_delshard.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: database.oracle.com/v1alpha1 diff --git a/docs/sharding/provisioning/shard_prov_extshard.yaml b/docs/sharding/provisioning/shard_prov_extshard.yaml index 953fe23f..449ba90b 100644 --- a/docs/sharding/provisioning/shard_prov_extshard.yaml +++ b/docs/sharding/provisioning/shard_prov_extshard.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: database.oracle.com/v1alpha1 diff --git a/docs/sharding/provisioning/shard_prov_memory_cpu.yaml b/docs/sharding/provisioning/shard_prov_memory_cpu.yaml index a542ebab..0e66c563 100644 --- a/docs/sharding/provisioning/shard_prov_memory_cpu.yaml +++ b/docs/sharding/provisioning/shard_prov_memory_cpu.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: database.oracle.com/v1alpha1 diff --git a/docs/sharding/provisioning/shard_prov_send_notification.yaml b/docs/sharding/provisioning/shard_prov_send_notification.yaml index af723c8e..19f3c02c 100644 --- a/docs/sharding/provisioning/shard_prov_send_notification.yaml +++ b/docs/sharding/provisioning/shard_prov_send_notification.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: database.oracle.com/v1alpha1 diff --git a/hack/boilerplate.go.txt b/hack/boilerplate.go.txt index dd467e1f..1a8fcae1 100644 --- a/hack/boilerplate.go.txt +++ b/hack/boilerplate.go.txt @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/main.go b/main.go index 1839893a..9bece017 100644 --- a/main.go +++ b/main.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/ords/setupwebuser.sh b/ords/setupwebuser.sh index c90deeda..ad2294ae 100644 --- a/ords/setupwebuser.sh +++ b/ords/setupwebuser.sh @@ -1,5 +1,5 @@ #!/bin/bash -# Copyright (c) 2021, Oracle and/or its affiliates. All rights reserved. +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. WEBSERVER_USER=`cat $ORDS_HOME/secrets/$WEBSERVER_USER_KEY` diff --git a/set_ocicredentials.sh b/set_ocicredentials.sh index b21742f4..dddf62c4 100755 --- a/set_ocicredentials.sh +++ b/set_ocicredentials.sh @@ -1,5 +1,5 @@ #!/bin/bash -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # Parse command line arguments diff --git a/test/e2e/autonomouscontainerdatabase_test.go b/test/e2e/autonomouscontainerdatabase_test.go index 7b9fb49e..6289b150 100644 --- a/test/e2e/autonomouscontainerdatabase_test.go +++ b/test/e2e/autonomouscontainerdatabase_test.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/test/e2e/autonomousdatabase_controller_bind_test.go b/test/e2e/autonomousdatabase_controller_bind_test.go index 22f8468c..3bd83710 100644 --- a/test/e2e/autonomousdatabase_controller_bind_test.go +++ b/test/e2e/autonomousdatabase_controller_bind_test.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/test/e2e/autonomousdatabase_controller_create_test.go b/test/e2e/autonomousdatabase_controller_create_test.go index 22f940c9..c98fc04f 100644 --- a/test/e2e/autonomousdatabase_controller_create_test.go +++ b/test/e2e/autonomousdatabase_controller_create_test.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/test/e2e/behavior/shared_behaviors.go b/test/e2e/behavior/shared_behaviors.go index 4dc2f834..14ee546d 100644 --- a/test/e2e/behavior/shared_behaviors.go +++ b/test/e2e/behavior/shared_behaviors.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/test/e2e/resource/test_config.yaml b/test/e2e/resource/test_config.yaml index 475b3351..1907af47 100644 --- a/test/e2e/resource/test_config.yaml +++ b/test/e2e/resource/test_config.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2021, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # diff --git a/test/e2e/suite_test.go b/test/e2e/suite_test.go index f5cd5af5..832d2633 100644 --- a/test/e2e/suite_test.go +++ b/test/e2e/suite_test.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/test/e2e/util/oci_acd_request.go b/test/e2e/util/oci_acd_request.go index a798d29f..cb21fe2b 100644 --- a/test/e2e/util/oci_acd_request.go +++ b/test/e2e/util/oci_acd_request.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/test/e2e/util/oci_config_util.go b/test/e2e/util/oci_config_util.go index 84770a96..3780d2cd 100644 --- a/test/e2e/util/oci_config_util.go +++ b/test/e2e/util/oci_config_util.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/test/e2e/util/oci_db_request.go b/test/e2e/util/oci_db_request.go index f22b0be8..6ba74ad9 100644 --- a/test/e2e/util/oci_db_request.go +++ b/test/e2e/util/oci_db_request.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/test/e2e/util/oci_work_request.go b/test/e2e/util/oci_work_request.go index cc5ca82d..a20b45d7 100644 --- a/test/e2e/util/oci_work_request.go +++ b/test/e2e/util/oci_work_request.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/test/e2e/util/util.go b/test/e2e/util/util.go index 340b2e07..81b5259d 100644 --- a/test/e2e/util/util.go +++ b/test/e2e/util/util.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2021 Oracle and/or its affiliates. +** Copyright (c) 2022 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** From 87850c4fbe448c6d39d838a1944b3136f00c13e2 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Thu, 12 May 2022 15:13:25 +0000 Subject: [PATCH 378/628] fix logs --- controllers/database/autonomouscontainerdatabase_controller.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/controllers/database/autonomouscontainerdatabase_controller.go b/controllers/database/autonomouscontainerdatabase_controller.go index bd0a0ff5..e564f9df 100644 --- a/controllers/database/autonomouscontainerdatabase_controller.go +++ b/controllers/database/autonomouscontainerdatabase_controller.go @@ -359,7 +359,7 @@ func (r *AutonomousContainerDatabaseReconciler) validateCleanup(logger logr.Logg if acd.Spec.Action != dbv1alpha1.AcdActionTerminate { // Run finalization logic for finalizer. If the finalization logic fails, don't remove the finalizer so // that we can retry during the next reconciliation. - l.Info("Terminating Autonomous Database: " + *acd.Spec.DisplayName) + l.Info("Terminating Autonomous Container Database") acd.Spec.Action = dbv1alpha1.AcdActionTerminate if err := r.KubeClient.Update(context.TODO(), acd); err != nil { return false, err From 087d747535d91085b33f1e487871877cbdebc10b Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Fri, 13 May 2022 09:42:55 +0530 Subject: [PATCH 379/628] Bugfixes: enable replicas>1 for XE, keepSecret false condition handled for ORDS, added apex password validator Signed-off-by: abhisbyk --- .../singleinstancedatabase_webhook.go | 27 +++++++--------- commons/database/utils.go | 30 +++++++++++++++++ .../samples/sidb/oraclerestdataservice.yaml | 6 ++++ .../sidb/oraclerestdataservice_apex.yaml | 5 +++ .../oraclerestdataservice_controller.go | 32 ++++++++++++++++++- .../singleinstancedatabase_controller.go | 4 --- 6 files changed, 84 insertions(+), 20 deletions(-) diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index ba3818e2..ae8026ef 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -110,15 +110,15 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { var allErrs field.ErrorList // Persistence spec validation - if r.Spec.Persistence.Size == "" && (r.Spec.Persistence.AccessMode != "" || r.Spec.Persistence.StorageClass != "" ) || - (r.Spec.Persistence.Size != "" && r.Spec.Persistence.AccessMode == "" ) { - allErrs = append(allErrs, - field.Invalid(field.NewPath("spec").Child("persistence"), r.Spec.Persistence, - "invalid specification, size and/or accessMode missing")) + if r.Spec.Persistence.Size == "" && (r.Spec.Persistence.AccessMode != "" || r.Spec.Persistence.StorageClass != "") || + (r.Spec.Persistence.Size != "" && r.Spec.Persistence.AccessMode == "") { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("persistence"), r.Spec.Persistence, + "invalid specification, size and/or accessMode missing")) } if r.Spec.Persistence.Size != "" && - r.Spec.Persistence.AccessMode != "ReadWriteMany" && r.Spec.Persistence.AccessMode != "ReadWriteOnce" { + r.Spec.Persistence.AccessMode != "ReadWriteMany" && r.Spec.Persistence.AccessMode != "ReadWriteOnce" { allErrs = append(allErrs, field.Invalid(field.NewPath("spec").Child("persistence").Child("accessMode"), r.Spec.Persistence.AccessMode, "should be either \"ReadWriteOnce\" or \"ReadWriteMany\"")) @@ -127,9 +127,6 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { // Replica validation if r.Spec.Replicas > 1 { valMsg := "" - if r.Spec.Edition == "express" { - valMsg = "should be 1 for express edition" - } if r.Spec.Persistence.Size == "" { valMsg = "should be 1 if no persistence is specified" } @@ -163,13 +160,13 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { if r.Spec.CloneFrom != "" { if r.Spec.Image.PrebuiltDB { allErrs = append(allErrs, - field.Invalid(field.NewPath("spec").Child("cloneFrom"), r.Spec.CloneFrom, - "cannot clone to create a prebuilt db")) + field.Invalid(field.NewPath("spec").Child("cloneFrom"), r.Spec.CloneFrom, + "cannot clone to create a prebuilt db")) } else if strings.Contains(r.Spec.CloneFrom, ":") && strings.Contains(r.Spec.CloneFrom, "/") && r.Spec.Edition == "" { //Edition must be passed when cloning from a source database other than same k8s cluster allErrs = append(allErrs, - field.Invalid(field.NewPath("spec").Child("edition"), r.Spec.CloneFrom, - "Edition must be passed when cloning from a source database other than same k8s cluster")) + field.Invalid(field.NewPath("spec").Child("edition"), r.Spec.CloneFrom, + "Edition must be passed when cloning from a source database other than same k8s cluster")) } } @@ -210,7 +207,7 @@ func (r *SingleInstanceDatabase) ValidateUpdate(oldRuntimeObject runtime.Object) allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("image").Child("prebuiltDB"), "cannot be changed")) } - if r.Spec.CloneFrom == "" && old.Status.Edition != "" && !strings.EqualFold(old.Status.Edition, r.Spec.Edition) { + if r.Spec.CloneFrom == "" && old.Status.Edition != "" && !strings.EqualFold(old.Status.Edition, r.Spec.Edition) { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("edition"), "cannot be changed")) } @@ -218,7 +215,7 @@ func (r *SingleInstanceDatabase) ValidateUpdate(oldRuntimeObject runtime.Object) allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("charset"), "cannot be changed")) } - if old.Status.Sid != "" && !strings.EqualFold(r.Spec.Sid, old.Status.Sid) { + if old.Status.Sid != "" && !strings.EqualFold(r.Spec.Sid, old.Status.Sid) { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("sid"), "cannot be changed")) } diff --git a/commons/database/utils.go b/commons/database/utils.go index 1231cdf0..9ae5ecda 100644 --- a/commons/database/utils.go +++ b/commons/database/utils.go @@ -47,6 +47,7 @@ import ( "math/rand" "strings" "time" + "unicode" corev1 "k8s.io/api/core/v1" @@ -645,3 +646,32 @@ func IsSourceDatabaseOnCluster(cloneFrom string) bool { } return true } + +// Apex password validation function +func ApexPasswordValidator(pwd string) bool { + var ( + hasMinLen = false + hasUpper = false + hasLower = false + hasNumber = false + hasSpecial = false + ) + if len(pwd) > 7 { + hasMinLen = true + } + + for _, c := range pwd { + switch { + case unicode.IsUpper(c): + hasUpper = true + case unicode.IsLower(c): + hasLower = true + case unicode.IsNumber(c): + hasNumber = true + case unicode.IsPunct(c): + hasSpecial = true + } + } + + return hasMinLen && hasUpper && hasLower && hasNumber && hasSpecial +} diff --git a/config/samples/sidb/oraclerestdataservice.yaml b/config/samples/sidb/oraclerestdataservice.yaml index 9a34d2fa..26f47b79 100644 --- a/config/samples/sidb/oraclerestdataservice.yaml +++ b/config/samples/sidb/oraclerestdataservice.yaml @@ -29,6 +29,12 @@ spec: ## To configure APEX with ORDS, specfiy the apexPassword secret details. Leave empty if Apex is not needed. ## This is a secret containing a common password for APEX_PUBLIC_USER, APEX_REST_PUBLIC_USER, APEX_LISTENER and Apex administrator (username: ADMIN) mapped to secretKey ## This secret will be deleted after ORDS Installation unless keepSecret set to true. + ## This password should complete the following requirements: + ## 1. Contain at least 6 characters. + ## 2. Contain at least one numeric character (0123456789). + ## 3. Contain at least one punctuation character (!"#$%&()``*+,-/:;?_). + ## 4. Contain at least one uppercase alphabetic character. + apexPassword: secretName: secretKey: diff --git a/config/samples/sidb/oraclerestdataservice_apex.yaml b/config/samples/sidb/oraclerestdataservice_apex.yaml index f44f4010..afd40f7a 100644 --- a/config/samples/sidb/oraclerestdataservice_apex.yaml +++ b/config/samples/sidb/oraclerestdataservice_apex.yaml @@ -23,6 +23,11 @@ metadata: type: Opaque stringData: ## Specify your APEX password here + ## This password should complete the following requirements: + ## 1. Contain at least 6 characters. + ## 2. Contain at least one numeric character (0123456789). + ## 3. Contain at least one punctuation character (!"#$%&()``*+,-/:;?_). + ## 4. Contain at least one uppercase alphabetic character. oracle_pwd: --- diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 1296ea1b..19e615f2 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -252,6 +252,35 @@ func (r *OracleRestDataServiceReconciler) validate(m *dbapi.OracleRestDataServic eventMsgs = append(eventMsgs, "image patching is not available currently") } + // Check for the apex ADMIN password + // + apexPasswordSecret := &corev1.Secret{} + err = r.Get(ctx, types.NamespacedName{Name: m.Spec.ApexPassword.SecretName, Namespace: m.Namespace}, apexPasswordSecret) + if err != nil { + if apierrors.IsNotFound(err) { + m.Status.Status = dbcommons.StatusError + eventReason := "Waiting" + eventMsg := "waiting for secret : " + m.Spec.ApexPassword.SecretName + " to get created" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + r.Log.Info("Secret " + m.Spec.ApexPassword.SecretName + " Not Found") + return requeueY, nil + } + r.Log.Error(err, err.Error()) + return requeueY, err + } + // APEX_LISTENER , APEX_REST_PUBLIC_USER , APEX_PUBLIC_USER passwords + apexPassword := string(apexPasswordSecret.Data[m.Spec.ApexPassword.SecretKey]) + + // Validate apexPassword + if !dbcommons.ApexPasswordValidator(apexPassword) { + m.Status.Status = dbcommons.StatusError + eventReason := "Apex Password does not conform to the requirements, please update the secret according to the requirements" + eventMsg := "Password should contain: at least 6 chars, at least one numeric character, at least one punctuation character (!\"#$%&()``*+,-/:;?_), at least one upper-case alphabet" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + r.Log.Info("APEX password does not conform to the requirements") + return requeueY, nil + } + if len(eventMsgs) > 0 { m.Status.Status = dbcommons.StatusError r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, strings.Join(eventMsgs, ",")) @@ -1195,7 +1224,8 @@ func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataS r.Status().Update(ctx, m) log.Info("ConfigureApex Successful !") - return requeueN + // Can not return requeueN as the secrets will be deleted if keepSecert is false, which cause problem in pod restart + return requeueY } //############################################################################# diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index ce006c7f..fff66e87 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -337,10 +337,6 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab } } - // If Express Edition, ensure Replicas=1 - if m.Spec.Edition == "express" && m.Spec.Replicas > 1 { - eventMsgs = append(eventMsgs, "express edition supports only one replica") - } // If no persistence, ensure Replicas=1 if m.Spec.Persistence.Size == "" && m.Spec.Replicas > 1 { eventMsgs = append(eventMsgs, "replicas should be 1 if no persistence is specified") From 785f21758c33bac6b74723091f5d307c3d929665 Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Fri, 13 May 2022 14:25:02 +0530 Subject: [PATCH 380/628] Readme changes for the bugs Signed-off-by: abhisbyk --- docs/sidb/README.md | 31 +++++++++++++++++++++++++++++-- oracle-database-operator.yaml | 2 +- 2 files changed, 30 insertions(+), 3 deletions(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 3cc16d33..41c5ac2a 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -152,7 +152,15 @@ To provision new Oracle Database Express Edition (XE) database, use the sample * This command pulls the XE image uploaded on the [Oracle Container Registry](https://container-registry.oracle.com/ords/f?p=113:4:7460390069267:::4:P4_REPOSITORY,AI_REPOSITORY,AI_REPOSITORY_NAME,P4_REPOSITORY_NAME,P4_EULA_ID,P4_BUSINESS_AREA_ID:803,803,Oracle%20Database%20Express%20Edition,Oracle%20Database%20Express%20Edition,1,0&cs=3-UN6D9nAfyqxcYnrks18OAmfFcri96NZojBQALxMdakix8wgYRBxhD8rpTFd2ak1FAtfOVFexbuOM2opsjxT9w). **NOTE:** -Provisioning Oracle Database express edition is supported for release 21c (21.3.0) and later releases. +- Provisioning Oracle Database express edition is supported for release 21c (21.3.0) and later releases. +- For XE database `SGA_TARGET + PGA_AGGREGATE_TARGET <= 2GB`. The default values for these parameters are 1536M and 512M respectively. +- If you want to set the `sgaTarget` and the `pgaAggregateTarget` for the XE edition, set the `sgaTarget` to the required value in the first go. After this, you can set the `pgaAggregateTarget` next. You need to add the following section to modify the `init-parameters` in the **[config/samples/sidb/singleinstancedatabase_express.yaml](../../config/samples/sidb/singleinstancedatabase_express.yaml)**: + + initParams: + cpuCount: + processes: + sgaTarget: + pgaAggregateTarget: ### Provision a pre-built database @@ -372,6 +380,7 @@ Note the following: - The `adminPassword` and `ordsPassword` fields in the `oraclerestdataservice.yaml` file contains secrets for authenticating the Single Instance Database and the ORDS user with the following roles: `SQL Administrator, System Administrator, SQL Developer, oracle.dbtools.autorest.any.schema`. - To build the ORDS image, use the following instructions: [Building Oracle REST Data Services Install Images](https://github.com/oracle/docker-images/tree/main/OracleRestDataServices#building-oracle-rest-data-services-install-images). - By default, ORDS uses self-signed certificates. To use certificates from the Certificate Authority, the ORDS image needs to be rebuilt after specifying the values of `ssl.cert` and `ssl.cert.key` in the [standalone.properties](https://github.com/oracle/docker-images/blob/main/OracleRestDataServices/dockerfiles/standalone.properties.tmpl) file. After you rebuild the ORDS image, use the rebuilt image in the **[config/samples/sidb/oraclerestdataservice.yaml](config/samples/sidb/oraclerestdataservice.yaml)** file. +- If you want to install ORDS in a [prebuilt database](#provision-a-pre-built-database), make sure to attach the **database persistence** by uncommenting the `persistence` section in the **[config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml](../../config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml)** file, while provisioning the prebuilt database. ### List OracleRestDataServices To list the ORDS service, use the following command: @@ -428,6 +437,13 @@ $ kubectl describe oraclerestdataservice ords-sample ``` +### Delete ORDS +- To delete ORDS run the following command: + + kubectl delete oraclerestdataservice ords-samples + +- You can not delete referred Single Instance Database (SIDB) before deleting its ORDS resource. + ## REST Enable Database To provision a new ORDS instance, use the sample **[config/samples/sidb/oraclerestdataservice_create.yaml](../../config/samples/sidb/oraclerestdataservice_create.yaml)** file. For example: @@ -626,7 +642,8 @@ password: `.spec.apexPassword` ![application-express-admin-home](/images/sidb/application-express-admin-home.png) **NOTE:** -By default, the full development environment is initialized in APEX. After deployment, you can change it manually to the runtime environment. To change environments, run the script `apxdevrm.sql` after connecting to the primary database from the ORDS pod as the `SYS` user with `SYSDBA` privilege. For detailed instructions, see: [Converting a Full Development Environment to a Runtime Environment](https://docs.oracle.com/en/database/oracle/application-express/21.2/htmig/converting-between-runtime-and-full-development-environments.html#GUID-B0621B40-3441-44ED-9D86-29B058E26BE9). +- By default, the full development environment is initialized in APEX. After deployment, you can change it manually to the runtime environment. To change environments, run the script `apxdevrm.sql` after connecting to the primary database from the ORDS pod as the `SYS` user with `SYSDBA` privilege. For detailed instructions, see: [Converting a Full Development Environment to a Runtime Environment](https://docs.oracle.com/en/database/oracle/application-express/21.2/htmig/converting-between-runtime-and-full-development-environments.html#GUID-B0621B40-3441-44ED-9D86-29B058E26BE9). +- Deleting ORDS resource will not delete APEX from the SIDB. Currently, performing APEX uninstall is not supported through the Operator. ## Performing maintenance operations If you need to perform some maintenance operations manually, then the procedure is as follows: @@ -640,3 +657,13 @@ If you need to perform some maintenance operations manually, then the procedure sqlplus / as sysdba + + +## Additional use-cases +- If you use **oci-bv** storage class for dynamic provisioning of the persistent volume, this volume gets deleted with the deletion of its associated resource (Database/ORDS). This happens because the Reclaim Policy of the provisioned volume is Delete by default. If you want to retain this dynamically provisioned volume, the following command should be used: + + kubectl patch pv -p '{"spec":{"persistentVolumeReclaimPolicy":"Retain"}}' + + To make this retained PV available for the next Database/ORDS deployment, you can run the following command: + + kubectl patch pv -p '{"spec":{"claimRef":null}}' \ No newline at end of file diff --git a/oracle-database-operator.yaml b/oracle-database-operator.yaml index 2c47c600..d29964aa 100644 --- a/oracle-database-operator.yaml +++ b/oracle-database-operator.yaml @@ -1178,7 +1178,7 @@ spec: type: string type: object replicas: - minimum: 1 + minimum: 3 type: integer restEnableSchemas: items: From d3522711345eea8f66511f9fa3a2745b2d73e40a Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Fri, 13 May 2022 17:55:03 +0530 Subject: [PATCH 381/628] Added back conditions for XE edition to support only 1 replica Signed-off-by: abhisbyk --- apis/database/v1alpha1/singleinstancedatabase_webhook.go | 3 +++ controllers/database/singleinstancedatabase_controller.go | 4 ++++ docs/sidb/README.md | 1 + oracle-database-operator.yaml | 2 +- 4 files changed, 9 insertions(+), 1 deletion(-) diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index ae8026ef..9eaa5df1 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -127,6 +127,9 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { // Replica validation if r.Spec.Replicas > 1 { valMsg := "" + if r.Spec.Edition == "express" { + valMsg = "should be 1 for express edition" + } if r.Spec.Persistence.Size == "" { valMsg = "should be 1 if no persistence is specified" } diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index fff66e87..ce006c7f 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -337,6 +337,10 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab } } + // If Express Edition, ensure Replicas=1 + if m.Spec.Edition == "express" && m.Spec.Replicas > 1 { + eventMsgs = append(eventMsgs, "express edition supports only one replica") + } // If no persistence, ensure Replicas=1 if m.Spec.Persistence.Size == "" && m.Spec.Replicas > 1 { eventMsgs = append(eventMsgs, "replicas should be 1 if no persistence is specified") diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 41c5ac2a..56fba38f 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -153,6 +153,7 @@ This command pulls the XE image uploaded on the [Oracle Container Registry](http **NOTE:** - Provisioning Oracle Database express edition is supported for release 21c (21.3.0) and later releases. +- For XE database, only single replica mode (i.e. `replicas: 1`) is supported. - For XE database `SGA_TARGET + PGA_AGGREGATE_TARGET <= 2GB`. The default values for these parameters are 1536M and 512M respectively. - If you want to set the `sgaTarget` and the `pgaAggregateTarget` for the XE edition, set the `sgaTarget` to the required value in the first go. After this, you can set the `pgaAggregateTarget` next. You need to add the following section to modify the `init-parameters` in the **[config/samples/sidb/singleinstancedatabase_express.yaml](../../config/samples/sidb/singleinstancedatabase_express.yaml)**: diff --git a/oracle-database-operator.yaml b/oracle-database-operator.yaml index d29964aa..2c47c600 100644 --- a/oracle-database-operator.yaml +++ b/oracle-database-operator.yaml @@ -1178,7 +1178,7 @@ spec: type: string type: object replicas: - minimum: 3 + minimum: 1 type: integer restEnableSchemas: items: From 5d4362acbb5710297d8ba3505b12a80c35e74ad7 Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Fri, 13 May 2022 18:15:55 +0530 Subject: [PATCH 382/628] Readme update Signed-off-by: abhisbyk --- docs/sidb/README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 56fba38f..3bc1c6d7 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -163,6 +163,8 @@ This command pulls the XE image uploaded on the [Oracle Container Registry](http sgaTarget: pgaAggregateTarget: +- Updating the init-parameters like `sgaTarget` and `pgaAggregateTarget` requires restart of the database to apply the new values. This database restart is handled by the Operator automatically. + ### Provision a pre-built database To provision a new pre-built database instance, use the sample **[config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml](../../config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml)** file. For example: From 71a170254606b8206b843b92cb84c3c7315afef6 Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Tue, 17 May 2022 17:09:51 +0530 Subject: [PATCH 383/628] Added the apex password validation if secret is there and apex is not configured Signed-off-by: abhisbyk --- .../oraclerestdataservice_controller.go | 48 ++++++++++--------- 1 file changed, 25 insertions(+), 23 deletions(-) diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 31a99c75..afb1e0c0 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -252,33 +252,35 @@ func (r *OracleRestDataServiceReconciler) validate(m *dbapi.OracleRestDataServic eventMsgs = append(eventMsgs, "image patching is not available currently") } - // Check for the apex ADMIN password - // - apexPasswordSecret := &corev1.Secret{} - err = r.Get(ctx, types.NamespacedName{Name: m.Spec.ApexPassword.SecretName, Namespace: m.Namespace}, apexPasswordSecret) - if err != nil { - if apierrors.IsNotFound(err) { + // Validate the apex ADMIN password if it is specified + + if m.Status.ApexConfigured && m.Spec.ApexPassword.SecretName != "" { + apexPasswordSecret := &corev1.Secret{} + err = r.Get(ctx, types.NamespacedName{Name: m.Spec.ApexPassword.SecretName, Namespace: m.Namespace}, apexPasswordSecret) + if err != nil { + if apierrors.IsNotFound(err) { + m.Status.Status = dbcommons.StatusError + eventReason := "Waiting" + eventMsg := "waiting for secret : " + m.Spec.ApexPassword.SecretName + " to get created" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + r.Log.Info("Secret " + m.Spec.ApexPassword.SecretName + " Not Found") + return requeueY, nil + } + r.Log.Error(err, err.Error()) + return requeueY, err + } + // APEX_LISTENER , APEX_REST_PUBLIC_USER , APEX_PUBLIC_USER passwords + apexPassword := string(apexPasswordSecret.Data[m.Spec.ApexPassword.SecretKey]) + + // Validate apexPassword + if !dbcommons.ApexPasswordValidator(apexPassword) { m.Status.Status = dbcommons.StatusError - eventReason := "Waiting" - eventMsg := "waiting for secret : " + m.Spec.ApexPassword.SecretName + " to get created" + eventReason := "Apex Password does not conform to the requirements, please update the secret according to the requirements" + eventMsg := "Password should contain: at least 6 chars, at least one numeric character, at least one punctuation character (!\"#$%&()``*+,-/:;?_), at least one upper-case alphabet" r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) - r.Log.Info("Secret " + m.Spec.ApexPassword.SecretName + " Not Found") + r.Log.Info("APEX password does not conform to the requirements") return requeueY, nil } - r.Log.Error(err, err.Error()) - return requeueY, err - } - // APEX_LISTENER , APEX_REST_PUBLIC_USER , APEX_PUBLIC_USER passwords - apexPassword := string(apexPasswordSecret.Data[m.Spec.ApexPassword.SecretKey]) - - // Validate apexPassword - if !dbcommons.ApexPasswordValidator(apexPassword) { - m.Status.Status = dbcommons.StatusError - eventReason := "Apex Password does not conform to the requirements, please update the secret according to the requirements" - eventMsg := "Password should contain: at least 6 chars, at least one numeric character, at least one punctuation character (!\"#$%&()``*+,-/:;?_), at least one upper-case alphabet" - r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) - r.Log.Info("APEX password does not conform to the requirements") - return requeueY, nil } if len(eventMsgs) > 0 { From de6ad9fad8ea92a7f5a452d1d35d956fd1a9b5be Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Tue, 17 May 2022 18:03:00 +0530 Subject: [PATCH 384/628] Correction in the condition Signed-off-by: abhisbyk --- controllers/database/oraclerestdataservice_controller.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index afb1e0c0..3d168f1f 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -254,7 +254,7 @@ func (r *OracleRestDataServiceReconciler) validate(m *dbapi.OracleRestDataServic // Validate the apex ADMIN password if it is specified - if m.Status.ApexConfigured && m.Spec.ApexPassword.SecretName != "" { + if !m.Status.ApexConfigured && m.Spec.ApexPassword.SecretName != "" { apexPasswordSecret := &corev1.Secret{} err = r.Get(ctx, types.NamespacedName{Name: m.Spec.ApexPassword.SecretName, Namespace: m.Namespace}, apexPasswordSecret) if err != nil { From 615251fc897d5ab331d7ad79f43720b83d8f0cde Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Wed, 18 May 2022 15:32:27 +0530 Subject: [PATCH 385/628] Ords enbale schema command log disabled, added validation for XE init-params change, capturing the ORA-error while changing the init-params and notify the user through event Signed-off-by: abhisbyk --- .gitignore | 1 + apis/database/v1alpha1/singleinstancedatabase_webhook.go | 6 ++++++ controllers/database/oraclerestdataservice_controller.go | 2 +- controllers/database/singleinstancedatabase_controller.go | 6 ++++++ 4 files changed, 14 insertions(+), 1 deletion(-) diff --git a/.gitignore b/.gitignore index a78b60d5..0711d342 100644 --- a/.gitignore +++ b/.gitignore @@ -5,3 +5,4 @@ testbin/* onpremtest/* ords/*zip .gitattributes +.vscode \ No newline at end of file diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index 9eaa5df1..878d1e92 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -154,6 +154,12 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { field.Invalid(field.NewPath("spec").Child("pdbName"), r.Spec.Pdbname, "Express edition PDB must be XEPDB1")) } + if r.Spec.Edition == "express" && + (r.Spec.InitParams.CpuCount != 0 || r.Spec.InitParams.Processes != 0 || r.Spec.InitParams.SgaTarget != 0 || r.Spec.InitParams.PgaAggregateTarget != 0) { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("initParams"), r.Spec.Pdbname, + "Express edition does not support changing init-parameters.")) + } if r.Spec.Edition != "express" && r.Spec.Sid == "XE" { allErrs = append(allErrs, field.Invalid(field.NewPath("spec").Child("sid"), r.Spec.Sid, diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 3d168f1f..739d7245 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -1431,7 +1431,7 @@ func (r *OracleRestDataServiceReconciler) restEnableSchemas(m *dbapi.OracleRestD strconv.FormatBool(m.Spec.RestEnableSchemas[i].Enable), urlMappingPattern, pdbName) // Create users,schemas and grant enableORDS for PDB - _, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + _, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", fmt.Sprintf("echo -e \"%s\" | %s", enableORDSSchema, dbcommons.SQLPlusCLI)) if err != nil { log.Error(err, err.Error()) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index ce006c7f..a55b705d 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1630,6 +1630,12 @@ func (r *SingleInstanceDatabaseReconciler) updateInitParameters(m *dbapi.SingleI log.Error(err, err.Error()) return requeueY, err } + // Notify the user about unsucessfull init-parameter value change + if strings.Contains(out, "ORA-") { + eventReason := "Invalid init-param value" + eventMsg := "Unable to change the init-param as specified. Error log: \n" + out + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) + } log.Info("AlterSgaPgaCpuCMD Output:" + out) if m.Status.InitParams.Processes != m.Spec.InitParams.Processes { From 9d8de7f0800523a509d441fd0ecace4cd376f529 Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Wed, 18 May 2022 17:56:24 +0530 Subject: [PATCH 386/628] Now scheduling pods on the nodes where sidb older pods are running is mandatory for ORDS and patching usecase Signed-off-by: abhisbyk --- .../oraclerestdataservice_controller.go | 20 ++++---- .../singleinstancedatabase_controller.go | 46 +++++++++++++------ 2 files changed, 41 insertions(+), 25 deletions(-) diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 739d7245..32c2f505 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -506,20 +506,18 @@ func (r *OracleRestDataServiceReconciler) instantiatePodSpec(m *dbapi.OracleRest Spec: corev1.PodSpec{ Affinity: func() *corev1.Affinity { if m.Spec.Persistence.Size == "" && n.Spec.Persistence.AccessMode == "ReadWriteOnce" { + // Only allowing pods to be scheduled on the node where SIDB pods are running return &corev1.Affinity{ PodAffinity: &corev1.PodAffinity{ - PreferredDuringSchedulingIgnoredDuringExecution: []corev1.WeightedPodAffinityTerm{{ - Weight: 100, - PodAffinityTerm: corev1.PodAffinityTerm{ - LabelSelector: &metav1.LabelSelector{ - MatchExpressions: []metav1.LabelSelectorRequirement{{ - Key: "app", - Operator: metav1.LabelSelectorOpIn, - Values: []string{n.Name}, // Schedule on same host as DB Pod - }}, - }, - TopologyKey: "kubernetes.io/hostname", + RequiredDuringSchedulingIgnoredDuringExecution: []corev1.PodAffinityTerm{{ + LabelSelector: &metav1.LabelSelector{ + MatchExpressions: []metav1.LabelSelectorRequirement{{ + Key: "app", + Operator: metav1.LabelSelectorOpIn, + Values: []string{n.Name}, // Schedule on same host as DB Pod + }}, }, + TopologyKey: "kubernetes.io/hostname", }, }, }, diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index a55b705d..4d8ca4f2 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -453,7 +453,7 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab //############################################################################# // Instantiate POD spec from SingleInstanceDatabase spec //############################################################################# -func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleInstanceDatabase, n *dbapi.SingleInstanceDatabase) *corev1.Pod { +func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleInstanceDatabase, n *dbapi.SingleInstanceDatabase, patching bool) *corev1.Pod { // POD spec pod := &corev1.Pod{ @@ -471,11 +471,10 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns Spec: corev1.PodSpec{ Affinity: func() *corev1.Affinity { if m.Spec.Persistence.AccessMode == "ReadWriteOnce" { - return &corev1.Affinity{ - PodAffinity: &corev1.PodAffinity{ - PreferredDuringSchedulingIgnoredDuringExecution: []corev1.WeightedPodAffinityTerm{{ - Weight: 100, - PodAffinityTerm: corev1.PodAffinityTerm{ + if patching { + return &corev1.Affinity{ + PodAffinity: &corev1.PodAffinity{ + RequiredDuringSchedulingIgnoredDuringExecution: []corev1.PodAffinityTerm{{ LabelSelector: &metav1.LabelSelector{ MatchExpressions: []metav1.LabelSelectorRequirement{{ Key: "app", @@ -484,10 +483,28 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }}, }, TopologyKey: "kubernetes.io/hostname", - }, + }}, }, + } + } else { + return &corev1.Affinity{ + PodAffinity: &corev1.PodAffinity{ + PreferredDuringSchedulingIgnoredDuringExecution: []corev1.WeightedPodAffinityTerm{{ + Weight: 100, + PodAffinityTerm: corev1.PodAffinityTerm{ + LabelSelector: &metav1.LabelSelector{ + MatchExpressions: []metav1.LabelSelectorRequirement{{ + Key: "app", + Operator: metav1.LabelSelectorOpIn, + Values: []string{m.Name}, + }}, + }, + TopologyKey: "kubernetes.io/hostname", + }, + }, + }, }, - }, + } } } return nil @@ -1121,7 +1138,7 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplacePods(m *dbapi.SingleIn r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) } // If version is same , call createPods() with the same version , and no of Replicas required - return r.createPods(m, n, ctx, req, replicasFound) + return r.createPods(m, n, ctx, req, replicasFound, false) } eventReason = "Scaling In" eventMsg = "from " + strconv.Itoa(replicasFound) + " pods to " + strconv.Itoa(m.Spec.Replicas) @@ -1163,7 +1180,7 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplacePods(m *dbapi.SingleIn } // create new Pods with the new Version and no.of Replicas required - result, err := r.createPods(m, n, ctx, req, newReplicasFound) + result, err := r.createPods(m, n, ctx, req, newReplicasFound, true) if result.Requeue { return result, err } @@ -1319,13 +1336,14 @@ func (r *SingleInstanceDatabaseReconciler) createWallet(m *dbapi.SingleInstanceD return requeueN, nil } -//############################################################################# +//############################################################################## // Create the requested POD replicas // m = SingleInstanceDatabase // n = CloneFromDatabase -//############################################################################# +// patching = Boolean variable to differentiate normal usecase with patching +//############################################################################## func (r *SingleInstanceDatabaseReconciler) createPods(m *dbapi.SingleInstanceDatabase, n *dbapi.SingleInstanceDatabase, - ctx context.Context, req ctrl.Request, replicasFound int) (ctrl.Result, error) { + ctx context.Context, req ctrl.Request, replicasFound int, patching bool) (ctrl.Result, error) { log := r.Log.WithValues("createPods", req.NamespacedName) @@ -1342,7 +1360,7 @@ func (r *SingleInstanceDatabaseReconciler) createPods(m *dbapi.SingleInstanceDat } // if Found < Required , Create New Pods , Name of Pods are generated Randomly for i := replicasFound; i < replicasReq; i++ { - pod := r.instantiatePodSpec(m, n) + pod := r.instantiatePodSpec(m, n, patching) log.Info("Creating a new "+m.Name+" POD", "POD.Namespace", pod.Namespace, "POD.Name", pod.Name) err := r.Create(ctx, pod) if err != nil { From 528d39993a60d2984265710b8910686ea80f02d4 Mon Sep 17 00:00:00 2001 From: Mohammed Yunus Date: Thu, 19 May 2022 01:42:07 +0000 Subject: [PATCH 387/628] Mqureshi bug fixes 1 --- .../singleinstancedatabase_webhook.go | 7 + .../singleinstancedatabase_controller.go | 169 +++++++++++------- 2 files changed, 113 insertions(+), 63 deletions(-) diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index 878d1e92..e952ab01 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -97,6 +97,13 @@ func (r *SingleInstanceDatabase) Default() { r.Spec.Pdbname = "ORCLPDB1" } } + + if r.Spec.Edition == "express" { + if r.Status.Replicas == 1 { + // default the replicas for XE + r.Spec.Replicas = 1 + } + } } // TODO(user): change verbs to "verbs=create;update;delete" if you want to enable deletion validation. diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 4d8ca4f2..0b465b83 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -173,19 +173,6 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct return result, nil } - if singleInstanceDatabase.Status.DatafilesCreated != "true" { - // Creation of Oracle Wallet for Single Instance Database credentials - result, err = r.createWallet(singleInstanceDatabase, ctx, req) - if result.Requeue { - r.Log.Info("Reconcile queued") - return result, nil - } - if err != nil { - r.Log.Info("Spec validation failed") - return result, nil - } - } - // Validate readiness var readyPod corev1.Pod result, readyPod, err = r.validateDBReadiness(singleInstanceDatabase, ctx, req) @@ -1128,28 +1115,68 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplacePods(m *dbapi.SingleIn if !imageChanged { eventReason := "" eventMsg := "" - if replicasFound == m.Spec.Replicas { - return requeueN, nil + if replicasFound > m.Spec.Replicas { + eventReason = "Scaling in pods" + eventMsg = "from " + strconv.Itoa(replicasFound) + " to " + strconv.Itoa(m.Spec.Replicas) + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + // Delete extra PODs + return r.deletePods(ctx, req, m, allAvailable, readyPod, replicasFound, m.Spec.Replicas) } - if replicasFound < m.Spec.Replicas { - if replicasFound != 0 { - eventReason = "Scaling Out" - eventMsg = "from " + strconv.Itoa(replicasFound) + " pods to " + strconv.Itoa(m.Spec.Replicas) + if replicasFound != 0 { + if replicasFound == 1 { + if m.Status.DatafilesCreated != "true" { + log.Info("No datafiles created, single replica found, creating wallet") + // Creation of Oracle Wallet for Single Instance Database credentials + r.createWallet(m, ctx, req) + } + } + if ok, _ := dbcommons.IsAnyPodWithStatus(allAvailable, corev1.PodRunning); !ok { + eventReason = "Database Pending" + eventMsg = "waiting for a pod to get to running state" + log.Info(eventMsg) r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + for i := 0; i < len(allAvailable); i++ { + r.Log.Info("Pod status: ", "name", allAvailable[i].Name, "phase", allAvailable[i].Status.Phase) + waitingReason := "" + var stateWaiting *corev1.ContainerStateWaiting + if len(allAvailable[i].Status.InitContainerStatuses) > 0 { + stateWaiting = allAvailable[i].Status.InitContainerStatuses[0].State.Waiting + } else if len(allAvailable[i].Status.ContainerStatuses) > 0 { + stateWaiting = allAvailable[i].Status.ContainerStatuses[0].State.Waiting + } + if stateWaiting != nil { + waitingReason = stateWaiting.Reason + } + if waitingReason == "" { + continue + } + r.Log.Info("Pod unavailable reason: ", "reason", waitingReason) + if strings.Contains(waitingReason, "ImagePullBackOff") || strings.Contains(waitingReason, "ErrImagePull") { + r.Log.Info("Deleting pod", "name", allAvailable[i].Name) + var gracePeriodSeconds int64 = 0 + policy := metav1.DeletePropagationForeground + r.Delete(ctx, &allAvailable[i], &client.DeleteOptions{ + GracePeriodSeconds: &gracePeriodSeconds, PropagationPolicy: &policy}) + } + } + return requeueY, err } - // If version is same , call createPods() with the same version , and no of Replicas required - return r.createPods(m, n, ctx, req, replicasFound, false) } - eventReason = "Scaling In" - eventMsg = "from " + strconv.Itoa(replicasFound) + " pods to " + strconv.Itoa(m.Spec.Replicas) - r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) - // Delete extra PODs - return r.deletePods(ctx, req, m, allAvailable, readyPod, replicasFound, m.Spec.Replicas) + if replicasFound == m.Spec.Replicas { + return requeueN, nil + } + if replicasFound != 0 && replicasFound < m.Spec.Replicas { + eventReason = "Scaling out pods" + eventMsg = "from " + strconv.Itoa(replicasFound) + " to " + strconv.Itoa(m.Spec.Replicas) + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + } + // If version is same , call createPods() with the same version , and no of Replicas required + return r.createPods(m, n, ctx, req, replicasFound, false) } // Version/Image changed // PATCHING START (Only Software Patch) - + log.Info("Pod image change detected") // call FindPods() to find pods of older version. Delete all the Pods readyPod, oldReplicasFound, oldAvailable, _, err := dbcommons.FindPods(r, oldVersion, oldImage, m.Name, m.Namespace, ctx, req) @@ -1162,7 +1189,7 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplacePods(m *dbapi.SingleIn oldAvailable = append(oldAvailable, readyPod) } - if m.Spec.Replicas == 1 { + if m.Status.Replicas == 1 { r.deletePods(ctx, req, m, oldAvailable, corev1.Pod{}, oldReplicasFound, 0) } @@ -1179,41 +1206,47 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplacePods(m *dbapi.SingleIn newAvailable = append(newAvailable, readyPod) } + if newReplicasFound != 0 { + if ok, _ := dbcommons.IsAnyPodWithStatus(newAvailable, corev1.PodRunning); !ok { + eventReason := "Database Pending" + eventMsg := "waiting for pod with changed image to get to running state" + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) + log.Info(eventMsg) + + for i := 0; i < len(newAvailable); i++ { + r.Log.Info("Pod status: ", "name", newAvailable[i].Name, "phase", newAvailable[i].Status.Phase) + waitingReason := "" + var stateWaiting *corev1.ContainerStateWaiting + if len(newAvailable[i].Status.InitContainerStatuses) > 0 { + stateWaiting = newAvailable[i].Status.InitContainerStatuses[0].State.Waiting + } else if len(newAvailable[i].Status.ContainerStatuses) > 0 { + stateWaiting = newAvailable[i].Status.ContainerStatuses[0].State.Waiting + } + if stateWaiting != nil { + waitingReason = stateWaiting.Reason + } + if waitingReason == "" { + continue + } + r.Log.Info("Pod unavailable reason: ", "reason", waitingReason) + if strings.Contains(waitingReason, "ImagePullBackOff") || strings.Contains(waitingReason, "ErrImagePull") { + r.Log.Info("Deleting pod", "name", newAvailable[i].Name) + var gracePeriodSeconds int64 = 0 + policy := metav1.DeletePropagationForeground + r.Delete(ctx, &newAvailable[i], &client.DeleteOptions{ + GracePeriodSeconds: &gracePeriodSeconds, PropagationPolicy: &policy}) + } + } + return requeueY, errors.New(eventMsg) + } + } + // create new Pods with the new Version and no.of Replicas required result, err := r.createPods(m, n, ctx, req, newReplicasFound, true) if result.Requeue { return result, err } - - if ok, _ := dbcommons.IsAnyPodWithStatus(newAvailable, corev1.PodRunning); !ok { - eventReason := "Database Pending" - eventMsg := "waiting for newer version/image DB pods get to running state" - r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) - log.Info(eventMsg) - - for i := 0; i < len(newAvailable); i++ { - r.Log.Info("Pod status: ", "name", newAvailable[i].Name, "phase", newAvailable[i].Status.Phase) - waitingReason := "" - if len(newAvailable[i].Status.InitContainerStatuses) > 0 { - waitingReason = newAvailable[i].Status.InitContainerStatuses[0].State.Waiting.Reason - } else if len(newAvailable[i].Status.ContainerStatuses) > 0 { - waitingReason = newAvailable[i].Status.ContainerStatuses[0].State.Waiting.Reason - } - if waitingReason == "" { - continue - } - r.Log.Info("Pod unavailable reason: ", "reason", waitingReason) - if strings.Contains(waitingReason, "ImagePullBackOff") || strings.Contains(waitingReason, "ErrImagePull") { - r.Log.Info("Deleting pod", "name", newAvailable[i].Name) - var gracePeriodSeconds int64 = 0 - policy := metav1.DeletePropagationForeground - r.Delete(ctx, &newAvailable[i], &client.DeleteOptions{ - GracePeriodSeconds: &gracePeriodSeconds, PropagationPolicy: &policy}) - } - } - return requeueY, errors.New(eventMsg) - } - if m.Spec.Replicas == 1 { + if m.Status.Replicas == 1 { return requeueN, nil } return r.deletePods(ctx, req, m, oldAvailable, corev1.Pod{}, oldReplicasFound, 0) @@ -1252,12 +1285,12 @@ func (r *SingleInstanceDatabaseReconciler) createWallet(m *dbapi.SingleInstanceD return requeueY, nil } - // Iterate through the avaialableFinal (list of pods) to find out the pod whose status is updated about the init containers + // Iterate through the availableFinal (list of pods) to find out the pod whose status is updated about the init containers // If no required pod found then requeue the reconcile request var pod corev1.Pod var podFound bool for _, pod = range availableFinal { - // Check if pod status contianer is updated about init containers + // Check if pod status container is updated about init containers if len(pod.Status.InitContainerStatuses) > 0 { podFound = true break @@ -1296,9 +1329,9 @@ func (r *SingleInstanceDatabaseReconciler) createWallet(m *dbapi.SingleInstanceD if err == nil && out != "" { m.Status.Status = dbcommons.StatusError - eventMsg := "wrong edition" + eventMsg := "incorrect database edition" r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) - return requeueN, errors.New("wrong Edition") + return requeueY, errors.New(eventMsg) } } @@ -1353,10 +1386,12 @@ func (r *SingleInstanceDatabaseReconciler) createPods(m *dbapi.SingleInstanceDat log.Info("No of " + m.Name + " replicas found are same as required") return requeueN, nil } + waitForFirstPod := false if replicasFound == 0 { m.Status.Status = dbcommons.StatusPending m.Status.DatafilesCreated = "false" m.Status.DatafilesPatched = "false" + waitForFirstPod = true } // if Found < Required , Create New Pods , Name of Pods are generated Randomly for i := replicasFound; i < replicasReq; i++ { @@ -1367,6 +1402,10 @@ func (r *SingleInstanceDatabaseReconciler) createPods(m *dbapi.SingleInstanceDat log.Error(err, "Failed to create new "+m.Name+" POD", "pod.Namespace", pod.Namespace, "POD.Name", pod.Name) return requeueY, err } + if waitForFirstPod { + log.Info("Wait for first pod to get to running state", "POD.Namespace", pod.Namespace, "POD.Name", pod.Name) + return requeueY, err + } } readyPod, _, availableFinal, _, err := dbcommons.FindPods(r, m.Spec.Image.Version, @@ -1624,7 +1663,11 @@ func (r *SingleInstanceDatabaseReconciler) runDatapatch(m *dbapi.SingleInstanceD m.Status.DatafilesPatched = "true" status, versionFrom, versionTo, _ := dbcommons.GetSqlpatchStatus(r, r.Config, readyPod, ctx, req) - eventMsg = "data files patched from " + versionFrom + " to " + versionTo + " : " + status + if versionTo != "" { + eventMsg = "data files patched from " + versionFrom + " to " + versionTo + " : " + status + } else { + eventMsg = "datapatch execution completed" + } r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) return requeueN, nil From ec6aa52e82b4aceba2d589512aca3d9b34823037 Mon Sep 17 00:00:00 2001 From: Abhishek Kumar Date: Thu, 19 May 2022 09:55:08 +0000 Subject: [PATCH 388/628] Readme updates, improving XE init-params validation --- .../singleinstancedatabase_webhook.go | 67 ++++++++++++------- .../samples/sidb/singleinstancedatabase.yaml | 1 + docs/sidb/README.md | 15 ++--- 3 files changed, 47 insertions(+), 36 deletions(-) diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index e952ab01..d2b384c7 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -146,31 +146,48 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { } } - if r.Spec.Edition == "express" && r.Spec.CloneFrom != "" { - allErrs = append(allErrs, - field.Invalid(field.NewPath("spec").Child("cloneFrom"), r.Spec.CloneFrom, - "Cloning not supported for Express edition")) - } - if r.Spec.Edition == "express" && strings.ToUpper(r.Spec.Sid) != "XE" { - allErrs = append(allErrs, - field.Invalid(field.NewPath("spec").Child("sid"), r.Spec.Sid, - "Express edition SID must only be XE")) - } - if r.Spec.Edition == "express" && strings.ToUpper(r.Spec.Pdbname) != "XEPDB1" { - allErrs = append(allErrs, - field.Invalid(field.NewPath("spec").Child("pdbName"), r.Spec.Pdbname, - "Express edition PDB must be XEPDB1")) - } - if r.Spec.Edition == "express" && - (r.Spec.InitParams.CpuCount != 0 || r.Spec.InitParams.Processes != 0 || r.Spec.InitParams.SgaTarget != 0 || r.Spec.InitParams.PgaAggregateTarget != 0) { - allErrs = append(allErrs, - field.Invalid(field.NewPath("spec").Child("initParams"), r.Spec.Pdbname, - "Express edition does not support changing init-parameters.")) - } - if r.Spec.Edition != "express" && r.Spec.Sid == "XE" { - allErrs = append(allErrs, - field.Invalid(field.NewPath("spec").Child("sid"), r.Spec.Sid, - "XE is reserved as the SID for Express edition of the database")) + if r.Spec.Edition == "express" { + if r.Spec.CloneFrom != "" { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("cloneFrom"), r.Spec.CloneFrom, + "Cloning not supported for Express edition")) + } + if strings.ToUpper(r.Spec.Sid) != "XE" { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("sid"), r.Spec.Sid, + "Express edition SID must only be XE")) + } + if strings.ToUpper(r.Spec.Pdbname) != "XEPDB1" { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("pdbName"), r.Spec.Pdbname, + "Express edition PDB must be XEPDB1")) + } + if r.Spec.InitParams.CpuCount != 0 { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("initParams").Child("cpuCount"), r.Spec.InitParams.CpuCount, + "Express edition does not support changing init parameter cpuCount.")) + } + if r.Spec.InitParams.Processes != 0 { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("initParams").Child("processes"), r.Spec.InitParams.Processes, + "Express edition does not support changing init parameter process.")) + } + if r.Spec.InitParams.SgaTarget != 0 { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("initParams").Child("sgaTarget"), r.Spec.InitParams.SgaTarget, + "Express edition does not support changing init parameter sgaTarget.")) + } + if r.Spec.InitParams.PgaAggregateTarget != 0 { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("initParams").Child("pgaAggregateTarget"), r.Spec.InitParams.PgaAggregateTarget, + "Express edition does not support changing init parameter pgaAggregateTarget.")) + } + } else { + if r.Spec.Sid == "XE" { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("sid"), r.Spec.Sid, + "XE is reserved as the SID for Express edition of the database")) + } } if r.Spec.CloneFrom != "" { diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index 8d6ed3c2..538a9e4c 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -47,6 +47,7 @@ spec: ## NA if cloning from a SourceDB (cloneFrom is set) ## Specify both sgaSize and pgaSize (in MB) or dont specify both ## Specify Non-Zero value to use + ## You cannot change these initParams for Oracle Database Express (XE) edition initParams: cpuCount: 0 processes: 0 diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 3bc1c6d7..39e8e198 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -143,6 +143,8 @@ You can easily provision a new database instance on the Kubernetes cluster by us **NOTE:** - For ease of use, the storage class **oci-bv** is specified in the **[singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml)**. This storage class facilitates dynamic provisioning of the OCI block volumes on the Oracle OKE for persistent storage of the database. For other cloud providers, you can similarly use their dynamic provisioning storage class. - Supports Oracle Database Enterprise Edition (19.3.0), and later releases. +- To pull the database image faster from the container registry in order to bring up the SIDB instance quickly, you can use container-registry mirror of the corresponding cluster's region. For example, if the cluster exists in Mumbai region, you can use `container-registry-bom.oracle.com` mirror. For more information on container-registry mirrors, please follow the link [https://blogs.oracle.com/wim/post/oracle-container-registry-mirrors-in-oracle-cloud-infrastructure](https://blogs.oracle.com/wim/post/oracle-container-registry-mirrors-in-oracle-cloud-infrastructure). +- Updating the init parameters like `sgaTarget` and `pgaAggregateTarget` (please refer [singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml)) requires restart of the database to apply the new values. This database restart is handled by the Operator automatically. ### Provisioning a new XE database To provision new Oracle Database Express Edition (XE) database, use the sample **[config/samples/sidb/singleinstancedatabase_express.yaml](../../config/samples/sidb/singleinstancedatabase_express.yaml)** file. For example: @@ -154,16 +156,7 @@ This command pulls the XE image uploaded on the [Oracle Container Registry](http **NOTE:** - Provisioning Oracle Database express edition is supported for release 21c (21.3.0) and later releases. - For XE database, only single replica mode (i.e. `replicas: 1`) is supported. -- For XE database `SGA_TARGET + PGA_AGGREGATE_TARGET <= 2GB`. The default values for these parameters are 1536M and 512M respectively. -- If you want to set the `sgaTarget` and the `pgaAggregateTarget` for the XE edition, set the `sgaTarget` to the required value in the first go. After this, you can set the `pgaAggregateTarget` next. You need to add the following section to modify the `init-parameters` in the **[config/samples/sidb/singleinstancedatabase_express.yaml](../../config/samples/sidb/singleinstancedatabase_express.yaml)**: - - initParams: - cpuCount: - processes: - sgaTarget: - pgaAggregateTarget: - -- Updating the init-parameters like `sgaTarget` and `pgaAggregateTarget` requires restart of the database to apply the new values. This database restart is handled by the Operator automatically. +- For XE database, you **cannot change** the init parameters i.e. `cpuCount, processes, sgaTarget or pgaAggregateTarget`. ### Provision a pre-built database @@ -445,7 +438,7 @@ $ kubectl describe oraclerestdataservice ords-sample kubectl delete oraclerestdataservice ords-samples -- You can not delete referred Single Instance Database (SIDB) before deleting its ORDS resource. +- You cannot delete referred Single Instance Database (SIDB) before deleting its ORDS resource. ## REST Enable Database From 92421f451313dca9135ea2804d9e999b097a0ad7 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Thu, 19 May 2022 18:01:05 +0530 Subject: [PATCH 389/628] Fix status replica updation --- controllers/database/singleinstancedatabase_controller.go | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 0b465b83..343162b8 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1402,6 +1402,7 @@ func (r *SingleInstanceDatabaseReconciler) createPods(m *dbapi.SingleInstanceDat log.Error(err, "Failed to create new "+m.Name+" POD", "pod.Namespace", pod.Namespace, "POD.Name", pod.Name) return requeueY, err } + m.Status.Replicas += 1 if waitForFirstPod { log.Info("Wait for first pod to get to running state", "POD.Namespace", pod.Namespace, "POD.Name", pod.Name) return requeueY, err @@ -1418,8 +1419,6 @@ func (r *SingleInstanceDatabaseReconciler) createPods(m *dbapi.SingleInstanceDat availableFinal = append(availableFinal, readyPod) } - m.Status.Replicas = m.Spec.Replicas - podNamesFinal := dbcommons.GetPodNames(availableFinal) log.Info("Final "+m.Name+" Pods After Deleting (or) Adding Extra Pods ( Including The Ready Pod ) ", "Pod Names", podNamesFinal) log.Info(m.Name+" Replicas Available", "Count", len(podNamesFinal)) @@ -1481,11 +1480,11 @@ func (r *SingleInstanceDatabaseReconciler) deletePods(ctx context.Context, req c if err != nil { r.Log.Error(err, "Failed to delete existing POD", "POD.Name", availablePod.Name) // Don't requeue + } else { + m.Status.Replicas -= 1 } } - m.Status.Replicas = m.Spec.Replicas - return requeueN, nil } From b965c1d5fdc6ae01b8b1d28b311f2db0070ed96a Mon Sep 17 00:00:00 2001 From: Mohammed Yunus Date: Thu, 19 May 2022 17:17:50 +0000 Subject: [PATCH 390/628] Mqureshi bug 34189847 --- commons/database/constants.go | 2 ++ commons/database/utils.go | 6 ++---- .../database/singleinstancedatabase_controller.go | 10 +++++++++- 3 files changed, 13 insertions(+), 5 deletions(-) diff --git a/commons/database/constants.go b/commons/database/constants.go index 8cf98b77..3559837e 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -214,6 +214,8 @@ const GetEnterpriseEditionFileCMD string = "if [ -f ${ORACLE_BASE}/oradata/dbcon const GetStandardEditionFileCMD string = "if [ -f ${ORACLE_BASE}/oradata/dbconfig/$ORACLE_SID/.docker_standard ]; then ls ${ORACLE_BASE}/oradata/dbconfig/$ORACLE_SID/.docker_standard; fi " +const CreateSIDlinkCMD string = "cd ${ORACLE_BASE}/oradata && test ! -e $ORACLE_SID && ln -s $(basename $PRIMARY_DB_CONN_STR)/$ORACLE_SID" + const GetPdbsSQL string = "select name from v\\$pdbs where name not like 'PDB\\$SEED' and open_mode like 'READ WRITE';" const OpenPDBSeed = "alter pluggable database pdb\\$seed close;" + diff --git a/commons/database/utils.go b/commons/database/utils.go index be3ee5f4..c5ebbefd 100644 --- a/commons/database/utils.go +++ b/commons/database/utils.go @@ -246,9 +246,7 @@ func ExecCommand(r client.Reader, config *rest.Config, podName string, namespace pod := &corev1.Pod{} err := r.Get(ctx, types.NamespacedName{Name: podName, Namespace: namespace}, pod) if err != nil { - return "", fmt.Errorf("could not get pod info: %v", err) - } else { - log.Info("Pod Found", "Name : ", podName) + return "", fmt.Errorf("could not find pod to execute command: %v", err) } client, err := kubernetes.NewForConfig(config) if err != nil { @@ -275,7 +273,7 @@ func ExecCommand(r client.Reader, config *rest.Config, podName string, namespace Tty: false, }) if err != nil { - return "", fmt.Errorf("could not execute: %v", err) + return "", err } if execErr.Len() > 0 { return "", fmt.Errorf("stderr: %v", execErr.String()) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 343162b8..7fe0b57e 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1511,7 +1511,15 @@ func (r *SingleInstanceDatabaseReconciler) validateDBReadiness(m *dbapi.SingleIn eventReason = "Database Creating" eventMsg = "waiting for database to be ready" m.Status.Status = dbcommons.StatusCreating - + if m.Spec.CloneFrom != "" { + // Required since clone creates the datafiles under primary database SID folder + r.Log.Info("Creating the SID directory link for clone database", "name", m.Spec.Sid) + _, err := dbcommons.ExecCommand(r, r.Config, runningPod.Name, runningPod.Namespace, "", + ctx, req, false, "bash", "-c", dbcommons.CreateSIDlinkCMD) + if err != nil { + r.Log.Info(err.Error()) + } + } out, err := dbcommons.ExecCommand(r, r.Config, runningPod.Name, runningPod.Namespace, "", ctx, req, false, "bash", "-c", dbcommons.GetCheckpointFileCMD) if err != nil { From a3ade4809ab43d4877cd399acbfa3c23980af994 Mon Sep 17 00:00:00 2001 From: Abhishek Kumar Date: Fri, 20 May 2022 11:23:06 +0000 Subject: [PATCH 391/628] Added bugfixes --- .../v1alpha1/singleinstancedatabase_types.go | 46 ++++++++++--------- .../singleinstancedatabase_webhook.go | 16 +++++++ ...se.oracle.com_singleinstancedatabases.yaml | 2 + .../singleinstancedatabase_controller.go | 12 ++--- docs/sidb/README.md | 2 +- 5 files changed, 48 insertions(+), 30 deletions(-) diff --git a/apis/database/v1alpha1/singleinstancedatabase_types.go b/apis/database/v1alpha1/singleinstancedatabase_types.go index d0391a4b..066f4eb7 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_types.go +++ b/apis/database/v1alpha1/singleinstancedatabase_types.go @@ -106,7 +106,7 @@ type SingleInstanceDatabaseAdminPassword struct { SecretName string `json:"secretName"` // +kubebuilder:default:="oracle_pwd" SecretKey string `json:"secretKey,omitempty"` - KeepSecret *bool `json:"keepSecret,omitempty"` + KeepSecret *bool `json:"keepSecret,omitempty"` } // SingleInstanceDatabaseStatus defines the observed state of SingleInstanceDatabase @@ -114,31 +114,33 @@ type SingleInstanceDatabaseStatus struct { // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster // Important: Run "make" to regenerate code after modifying this file - Nodes []string `json:"nodes,omitempty"` - Role string `json:"role,omitempty"` - Status string `json:"status,omitempty"` - Replicas int `json:"replicas,omitempty"` - ReleaseUpdate string `json:"releaseUpdate,omitempty"` + Nodes []string `json:"nodes,omitempty"` + Role string `json:"role,omitempty"` + Status string `json:"status,omitempty"` + Replicas int `json:"replicas,omitempty"` + ReleaseUpdate string `json:"releaseUpdate,omitempty"` + // +kubebuilder:default:="false" DatafilesPatched string `json:"datafilesPatched,omitempty"` ConnectString string `json:"connectString,omitempty"` ClusterConnectString string `json:"clusterConnectString,omitempty"` StandbyDatabases map[string]string `json:"standbyDatabases,omitempty"` - DatafilesCreated string `json:"datafilesCreated,omitempty"` - Sid string `json:"sid,omitempty"` - Edition string `json:"edition,omitempty"` - Charset string `json:"charset,omitempty"` - Pdbname string `json:"pdbName,omitempty"` - InitSgaSize int `json:"initSgaSize,omitempty"` - InitPgaSize int `json:"initPgaSize,omitempty"` - CloneFrom string `json:"cloneFrom,omitempty"` - FlashBack string `json:"flashBack,omitempty"` - ArchiveLog string `json:"archiveLog,omitempty"` - ForceLogging string `json:"forceLog,omitempty"` - OemExpressUrl string `json:"oemExpressUrl,omitempty"` - OrdsReference string `json:"ordsReference,omitempty"` - PdbConnectString string `json:"pdbConnectString,omitempty"` - ApexInstalled bool `json:"apexInstalled,omitempty"` - PrebuiltDB bool `json:"prebuiltDB,omitempty"` + // +kubebuilder:default:="false" + DatafilesCreated string `json:"datafilesCreated,omitempty"` + Sid string `json:"sid,omitempty"` + Edition string `json:"edition,omitempty"` + Charset string `json:"charset,omitempty"` + Pdbname string `json:"pdbName,omitempty"` + InitSgaSize int `json:"initSgaSize,omitempty"` + InitPgaSize int `json:"initPgaSize,omitempty"` + CloneFrom string `json:"cloneFrom,omitempty"` + FlashBack string `json:"flashBack,omitempty"` + ArchiveLog string `json:"archiveLog,omitempty"` + ForceLogging string `json:"forceLog,omitempty"` + OemExpressUrl string `json:"oemExpressUrl,omitempty"` + OrdsReference string `json:"ordsReference,omitempty"` + PdbConnectString string `json:"pdbConnectString,omitempty"` + ApexInstalled bool `json:"apexInstalled,omitempty"` + PrebuiltDB bool `json:"prebuiltDB,omitempty"` // +patchMergeKey=type // +patchStrategy=merge diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index d2b384c7..1d031789 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -203,6 +203,22 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { } } + if r.Status.FlashBack == "true" { + if !r.Spec.ArchiveLog { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("archiveLog"), r.Spec.ArchiveLog, + "Cannot disable Archivelog. Please disable Flashback first.")) + } + } + + if !r.Spec.ArchiveLog { + if r.Spec.FlashBack { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("flashBack"), r.Spec.FlashBack, + "Cannot enable Flashback. Please enable Archivelog first.")) + } + } + if len(allErrs) == 0 { return nil } diff --git a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml index 7b59758f..941aa627 100644 --- a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml +++ b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml @@ -240,8 +240,10 @@ spec: connectString: type: string datafilesCreated: + default: "false" type: string datafilesPatched: + default: "false" type: string edition: type: string diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 7fe0b57e..8d494868 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1389,8 +1389,6 @@ func (r *SingleInstanceDatabaseReconciler) createPods(m *dbapi.SingleInstanceDat waitForFirstPod := false if replicasFound == 0 { m.Status.Status = dbcommons.StatusPending - m.Status.DatafilesCreated = "false" - m.Status.DatafilesPatched = "false" waitForFirstPod = true } // if Found < Required , Create New Pods , Name of Pods are generated Randomly @@ -1515,7 +1513,7 @@ func (r *SingleInstanceDatabaseReconciler) validateDBReadiness(m *dbapi.SingleIn // Required since clone creates the datafiles under primary database SID folder r.Log.Info("Creating the SID directory link for clone database", "name", m.Spec.Sid) _, err := dbcommons.ExecCommand(r, r.Config, runningPod.Name, runningPod.Namespace, "", - ctx, req, false, "bash", "-c", dbcommons.CreateSIDlinkCMD) + ctx, req, false, "bash", "-c", dbcommons.CreateSIDlinkCMD) if err != nil { r.Log.Info(err.Error()) } @@ -1826,11 +1824,11 @@ func (r *SingleInstanceDatabaseReconciler) updateDBConfig(m *dbapi.SingleInstanc log.Info(out) } else { - // Occurs when flashback is attermpted to be turned on without turning on archiving first + // Occurs when flashback is attempted to be turned on without turning on archiving first eventReason := "Waiting" eventMsg := "enable ArchiveLog to turn ON Flashback" - log.Info(eventMsg) r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) + log.Info(eventMsg) changeArchiveLog = true } @@ -1869,11 +1867,11 @@ func (r *SingleInstanceDatabaseReconciler) updateDBConfig(m *dbapi.SingleInstanc log.Info(out) } else { - // Occurs when archiving is attermpted to be turned off without turning off flashback first + // Occurs when archiving is attempted to be turned off without turning off flashback first eventReason := "Waiting" eventMsg := "turn OFF Flashback to disable ArchiveLog" - log.Info(eventMsg) r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) + log.Info(eventMsg) changeArchiveLog = true } diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 39e8e198..477a0037 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -144,7 +144,7 @@ You can easily provision a new database instance on the Kubernetes cluster by us - For ease of use, the storage class **oci-bv** is specified in the **[singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml)**. This storage class facilitates dynamic provisioning of the OCI block volumes on the Oracle OKE for persistent storage of the database. For other cloud providers, you can similarly use their dynamic provisioning storage class. - Supports Oracle Database Enterprise Edition (19.3.0), and later releases. - To pull the database image faster from the container registry in order to bring up the SIDB instance quickly, you can use container-registry mirror of the corresponding cluster's region. For example, if the cluster exists in Mumbai region, you can use `container-registry-bom.oracle.com` mirror. For more information on container-registry mirrors, please follow the link [https://blogs.oracle.com/wim/post/oracle-container-registry-mirrors-in-oracle-cloud-infrastructure](https://blogs.oracle.com/wim/post/oracle-container-registry-mirrors-in-oracle-cloud-infrastructure). -- Updating the init parameters like `sgaTarget` and `pgaAggregateTarget` (please refer [singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml)) requires restart of the database to apply the new values. This database restart is handled by the Operator automatically. +- To update the init parameters like `sgaTarget` and `pgaAggregateTarget`, please refer the `initParams` section of the [singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml) file. ### Provisioning a new XE database To provision new Oracle Database Express Edition (XE) database, use the sample **[config/samples/sidb/singleinstancedatabase_express.yaml](../../config/samples/sidb/singleinstancedatabase_express.yaml)** file. For example: From de990ca65a5cac00339561cb6b14a0debf174471 Mon Sep 17 00:00:00 2001 From: Mohammed Yunus Date: Fri, 20 May 2022 14:31:44 +0000 Subject: [PATCH 392/628] Update pod affinity logic --- .../singleinstancedatabase_controller.go | 21 +++++++++++-------- 1 file changed, 12 insertions(+), 9 deletions(-) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 8d494868..d76a646c 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -440,7 +440,8 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab //############################################################################# // Instantiate POD spec from SingleInstanceDatabase spec //############################################################################# -func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleInstanceDatabase, n *dbapi.SingleInstanceDatabase, patching bool) *corev1.Pod { +func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleInstanceDatabase, n *dbapi.SingleInstanceDatabase, + requiredAffinity bool) *corev1.Pod { // POD spec pod := &corev1.Pod{ @@ -458,7 +459,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns Spec: corev1.PodSpec{ Affinity: func() *corev1.Affinity { if m.Spec.Persistence.AccessMode == "ReadWriteOnce" { - if patching { + if requiredAffinity { return &corev1.Affinity{ PodAffinity: &corev1.PodAffinity{ RequiredDuringSchedulingIgnoredDuringExecution: []corev1.PodAffinityTerm{{ @@ -1242,7 +1243,8 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplacePods(m *dbapi.SingleIn } // create new Pods with the new Version and no.of Replicas required - result, err := r.createPods(m, n, ctx, req, newReplicasFound, true) + // if m.Status.Replicas > 1, then it is replica based patching + result, err := r.createPods(m, n, ctx, req, newReplicasFound, m.Status.Replicas > 1) if result.Requeue { return result, err } @@ -1376,7 +1378,7 @@ func (r *SingleInstanceDatabaseReconciler) createWallet(m *dbapi.SingleInstanceD // patching = Boolean variable to differentiate normal usecase with patching //############################################################################## func (r *SingleInstanceDatabaseReconciler) createPods(m *dbapi.SingleInstanceDatabase, n *dbapi.SingleInstanceDatabase, - ctx context.Context, req ctrl.Request, replicasFound int, patching bool) (ctrl.Result, error) { + ctx context.Context, req ctrl.Request, replicasFound int, replicaPatching bool) (ctrl.Result, error) { log := r.Log.WithValues("createPods", req.NamespacedName) @@ -1386,14 +1388,15 @@ func (r *SingleInstanceDatabaseReconciler) createPods(m *dbapi.SingleInstanceDat log.Info("No of " + m.Name + " replicas found are same as required") return requeueN, nil } - waitForFirstPod := false + firstPod := false if replicasFound == 0 { m.Status.Status = dbcommons.StatusPending - waitForFirstPod = true + firstPod = true } // if Found < Required , Create New Pods , Name of Pods are generated Randomly for i := replicasFound; i < replicasReq; i++ { - pod := r.instantiatePodSpec(m, n, patching) + // mandatory pod affinity if it is replica based patching or not the first pod + pod := r.instantiatePodSpec(m, n, replicaPatching || !firstPod) log.Info("Creating a new "+m.Name+" POD", "POD.Namespace", pod.Namespace, "POD.Name", pod.Name) err := r.Create(ctx, pod) if err != nil { @@ -1401,8 +1404,8 @@ func (r *SingleInstanceDatabaseReconciler) createPods(m *dbapi.SingleInstanceDat return requeueY, err } m.Status.Replicas += 1 - if waitForFirstPod { - log.Info("Wait for first pod to get to running state", "POD.Namespace", pod.Namespace, "POD.Name", pod.Name) + if firstPod { + log.Info("Requeue for first pod to get to running state", "POD.Namespace", pod.Namespace, "POD.Name", pod.Name) return requeueY, err } } From 42fbf8a95c5116513fb63774670eddb86bb91bb6 Mon Sep 17 00:00:00 2001 From: Mohammed Yunus Date: Mon, 23 May 2022 04:36:09 +0000 Subject: [PATCH 393/628] Mqureshi ords enhancements --- .../singleinstancedatabase_webhook.go | 2 +- commons/database/constants.go | 16 +- .../oraclerestdataservice_controller.go | 198 +++++++++++------- 3 files changed, 128 insertions(+), 88 deletions(-) diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index 1d031789..749e6414 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -297,7 +297,7 @@ func (r *SingleInstanceDatabase) ValidateDelete() error { var allErrs field.ErrorList if r.Status.OrdsReference != "" { allErrs = append(allErrs, - field.Forbidden(field.NewPath("status").Child("ordsInstalled"), "uninstall ORDS to cleanup this SIDB")) + field.Forbidden(field.NewPath("status").Child("ordsReference"), "delete " + r.Status.OrdsReference+ " to cleanup this SIDB")) } if len(allErrs) == 0 { return nil diff --git a/commons/database/constants.go b/commons/database/constants.go index 3559837e..f82126fb 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -231,18 +231,16 @@ const SetAdminUsersSQL string = "CREATE USER C##DBAPI_CDB_ADMIN IDENTIFIED BY \\ "\nalter pluggable database pdb\\$seed close;" + "\nalter pluggable database pdb\\$seed open read write force;" -const GetUserOrdsSchemaStatusSQL string = "alter session set container=%[2]s;" + +const GetUserORDSSchemaStatusSQL string = "alter session set container=%[2]s;" + "\nselect 'STATUS:'||status as status from ords_metadata.ords_schemas where upper(parsing_schema) = upper('%[1]s');" -const EnableORDSSchemaSQL string = "\nALTER SESSION SET CONTAINER=%[5]s;" + +const CreateORDSSchemaSQL = "\nALTER SESSION SET CONTAINER=%[3]s;" + "\nCREATE USER %[1]s IDENTIFIED BY \\\"%[2]s\\\";" + - "\nGRANT CONNECT, RESOURCE, DBA, PDB_DBA TO %[1]s;" + - "\nCONN %[1]s/\\\"%[2]s\\\"@localhost:1521/%[5]s;" + - "\nBEGIN" + - "\nORDS.enable_schema(p_enabled => %[3]s ,p_schema => '%[1]s',p_url_mapping_type => 'BASE_PATH',p_url_mapping_pattern => '%[4]s',p_auto_rest_auth => FALSE);" + - "\nCOMMIT;" + - "\nEND;" + - "\n/" + "\nGRANT CONNECT, RESOURCE, DBA, PDB_DBA TO %[1]s;" + +const EnableORDSSchemaSQL string = "\nALTER SESSION SET CONTAINER=%[4]s;" + + "\nGRANT INHERIT PRIVILEGES ON USER SYS TO ORDS_METADATA;" + + "\nexec ORDS.enable_schema(p_enabled => %[2]s ,p_schema => '%[1]s',p_url_mapping_type => 'BASE_PATH',p_url_mapping_pattern => '%[3]s',p_auto_rest_auth => FALSE);" // SetupORDSCMD is run only for the FIRST TIME, ORDS is installed. Once ORDS is installed, we delete the pod that ran SetupORDSCMD and create new ones. // Newly created pod doesn't run this SetupORDSCMD. diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 32c2f505..48ab8ae8 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -116,7 +116,6 @@ func (r *OracleRestDataServiceReconciler) Reconcile(ctx context.Context, req ctr oracleRestDataService.Status.DatabaseActionsUrl = dbcommons.ValueUnavailable r.Status().Update(ctx, oracleRestDataService) } - oracleRestDataService.Status.DatabaseRef = oracleRestDataService.Spec.DatabaseRef oracleRestDataService.Status.LoadBalancer = strconv.FormatBool(oracleRestDataService.Spec.LoadBalancer) oracleRestDataService.Status.Image = oracleRestDataService.Spec.Image @@ -128,14 +127,24 @@ func (r *OracleRestDataServiceReconciler) Reconcile(ctx context.Context, req ctr err = r.Get(ctx, types.NamespacedName{Namespace: req.Namespace, Name: oracleRestDataService.Spec.DatabaseRef}, singleInstanceDatabase) if err != nil { if apierrors.IsNotFound(err) { - eventReason := "Waiting" - eventMsg := "waiting for database " + oracleRestDataService.Spec.DatabaseRef - r.Recorder.Eventf(oracleRestDataService, corev1.EventTypeNormal, eventReason, eventMsg) - r.Log.Info("Resource not found", "DatabaseRef", oracleRestDataService.Spec.DatabaseRef) + oracleRestDataService.Status.Status = dbcommons.StatusError + oracleRestDataService.Status.DatabaseRef = "" + eventReason := "Error" + eventMsg := "database reference " + oracleRestDataService.Spec.DatabaseRef + " not found" + r.Recorder.Eventf(oracleRestDataService, corev1.EventTypeWarning, eventReason, eventMsg) + r.Log.Info(eventMsg) return requeueY, nil } r.Log.Error(err, err.Error()) return requeueY, err + } else { + if oracleRestDataService.Status.DatabaseRef == "" { + oracleRestDataService.Status.Status = dbcommons.StatusPending + oracleRestDataService.Status.DatabaseRef = oracleRestDataService.Spec.DatabaseRef + eventReason := "Database Check" + eventMsg := "database reference " + oracleRestDataService.Spec.DatabaseRef + " found" + r.Recorder.Eventf(oracleRestDataService, corev1.EventTypeNormal, eventReason, eventMsg) + } } // Manage OracleRestDataService Deletion @@ -147,11 +156,7 @@ func (r *OracleRestDataServiceReconciler) Reconcile(ctx context.Context, req ctr // First validate result, err = r.validate(oracleRestDataService, singleInstanceDatabase, ctx) - if result.Requeue { - r.Log.Info("Spec validation failed, Reconcile queued") - return result, nil - } - if err != nil { + if result.Requeue || err != nil { r.Log.Info("Spec validation failed") return result, nil } @@ -191,7 +196,7 @@ func (r *OracleRestDataServiceReconciler) Reconcile(ctx context.Context, req ctr return result, nil } - result = r.restEnableSchemas(oracleRestDataService, singleInstanceDatabase, sidbReadyPod, ctx, req) + result = r.restEnableSchemas(oracleRestDataService, singleInstanceDatabase, sidbReadyPod, ordsReadyPod, ctx, req) if result.Requeue { r.Log.Info("Reconcile queued") return result, nil @@ -243,7 +248,10 @@ func (r *OracleRestDataServiceReconciler) validate(m *dbapi.OracleRestDataServic // If ORDS has no peristence specified, ensure SIDB has persistence configured if m.Spec.Persistence.Size == "" && n.Spec.Persistence.AccessMode == "" { - eventMsgs = append(eventMsgs, "ORDS cannot be configured for database "+m.Spec.DatabaseRef+" that has no attached persistent volume") + eventMsgs = append(eventMsgs, "cannot configure ORDS for database "+m.Spec.DatabaseRef+" that has no attached persistent volume") + } + if !m.Status.OrdsInstalled && n.Status.OrdsReference != "" { + eventMsgs = append(eventMsgs, "database "+m.Spec.DatabaseRef+ " is already configured with ORDS "+n.Status.OrdsReference) } if m.Status.DatabaseRef != "" && m.Status.DatabaseRef != m.Spec.DatabaseRef { eventMsgs = append(eventMsgs, "databaseRef cannot be updated") @@ -275,7 +283,7 @@ func (r *OracleRestDataServiceReconciler) validate(m *dbapi.OracleRestDataServic // Validate apexPassword if !dbcommons.ApexPasswordValidator(apexPassword) { m.Status.Status = dbcommons.StatusError - eventReason := "Apex Password does not conform to the requirements, please update the secret according to the requirements" + eventReason := "Apex Password Invalid" eventMsg := "Password should contain: at least 6 chars, at least one numeric character, at least one punctuation character (!\"#$%&()``*+,-/:;?_), at least one upper-case alphabet" r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) r.Log.Info("APEX password does not conform to the requirements") @@ -314,12 +322,16 @@ func (r *OracleRestDataServiceReconciler) validateSIDBReadiness(m *dbapi.OracleR return requeueN, sidbReadyPod } + m.Status.Status = dbcommons.StatusPending if sidbReadyPod.Name == "" || n.Status.Status != dbcommons.StatusReady { eventReason := "Waiting" - eventMsg := "waiting for " + n.Name + " to be Ready" - r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) - m.Status.Status = dbcommons.StatusNotReady + eventMsg := "waiting for database " + n.Name + " status to be Ready" + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) return requeueY, sidbReadyPod + } else { + eventReason := "Database Healthy" + eventMsg := "database " + n.Name + " status is Ready" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) } // Validate databaseRef Admin Password @@ -327,7 +339,6 @@ func (r *OracleRestDataServiceReconciler) validateSIDBReadiness(m *dbapi.OracleR err = r.Get(ctx, types.NamespacedName{Name: m.Spec.AdminPassword.SecretName, Namespace: m.Namespace}, adminPasswordSecret) if err != nil { if apierrors.IsNotFound(err) { - m.Status.Status = dbcommons.StatusPending eventReason := "Waiting" eventMsg := "waiting for secret : " + m.Spec.AdminPassword.SecretName + " to get created" r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) @@ -364,8 +375,6 @@ func (r *OracleRestDataServiceReconciler) validateSIDBReadiness(m *dbapi.OracleR log.Error(err, err.Error()) return requeueY, sidbReadyPod } - log.Info("SetAdminUsers Output :\n" + out) - if !strings.Contains(out, "ERROR") || !strings.Contains(out, "ORA-") || strings.Contains(out, "ERROR") && strings.Contains(out, "ORA-01920") { m.Status.CommonUsersCreated = true @@ -413,6 +422,11 @@ func (r *OracleRestDataServiceReconciler) checkHealthStatus(m *dbapi.OracleRestD } if !m.Status.OrdsInstalled { m.Status.OrdsInstalled = true + n.Status.OrdsReference = m.Name + r.Status().Update(ctx, n) + eventReason := "ORDS Installation" + eventMsg := "installation of ORDS completed" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) out, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", fmt.Sprintf("echo -e \"%s\" | %s", dbcommons.OpenPDBSeed, dbcommons.SQLPlusCLI)) if err != nil { @@ -968,8 +982,7 @@ func (r *OracleRestDataServiceReconciler) createPods(m *dbapi.OracleRestDataServ } } } - n.Status.OrdsReference = m.Name - r.Status().Update(ctx, n) + m.Status.Replicas = m.Spec.Replicas return requeueN @@ -1092,9 +1105,10 @@ func (r *OracleRestDataServiceReconciler) cleanupOracleRestDataService(req ctrl. if err != nil { log.Error(err, err.Error()) if apierrors.IsNotFound(err) { - eventReason := "Waiting" - eventMsg := "waiting for admin password secret : " + m.Spec.AdminPassword.SecretName + " to get created" - r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + m.Status.Status = dbcommons.StatusError + eventReason := "Error" + eventMsg := "database admin password secret " + m.Spec.AdminPassword.SecretName + " required for ORDS uninstall not found, retrying..." + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) r.Log.Info(eventMsg) if i < 4 { time.Sleep(15 * time.Second) @@ -1210,21 +1224,21 @@ func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataS return requeueY } } - // ORDS Needs to be restarted to configure APEX + + m.Status.ApexConfigured = true + r.Status().Update(ctx, m) + log.Info("ConfigureApex Successful !") + + // ORDS needs to be restarted to configure APEX + r.Log.Info("Restarting ORDS Pod to complete APEX configuration: " + ordsReadyPod.Name) var gracePeriodSeconds int64 = 0 policy := metav1.DeletePropagationForeground err = r.Delete(ctx, &ordsReadyPod, &client.DeleteOptions{ GracePeriodSeconds: &gracePeriodSeconds, PropagationPolicy: &policy}) if err != nil { - r.Log.Error(err, "Failed to delete existing POD", "POD.Name", ordsReadyPod.Name) - return requeueY + r.Log.Error(err, err.Error()) } - r.Log.Info("ORDS Pod Deleted : " + ordsReadyPod.Name) - m.Status.ApexConfigured = true - r.Status().Update(ctx, m) - - log.Info("ConfigureApex Successful !") - // Can not return requeueN as the secrets will be deleted if keepSecert is false, which cause problem in pod restart + // Cannot return requeue as the secrets will be deleted if keepSecert is false, which cause problem in pod restart return requeueY } @@ -1255,9 +1269,9 @@ func (r *OracleRestDataServiceReconciler) installApex(m *dbapi.OracleRestDataSer // Status Updation m.Status.Status = dbcommons.StatusUpdating r.Status().Update(ctx, m) - eventReason := "Installing Apex" - eventMsg := "Waiting for Apex installation to complete" - r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) + eventReason := "Apex Installation" + eventMsg := "waiting for Apex installation to complete" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) //Install Apex in SIDB ready pod out, err := dbcommons.ExecCommand(r, r.Config, ordsReadyPod.Name, ordsReadyPod.Namespace, "", ctx, req, true, "bash", "-c", @@ -1282,9 +1296,9 @@ func (r *OracleRestDataServiceReconciler) installApex(m *dbapi.OracleRestDataSer } m.Status.Status = dbcommons.StatusReady - eventReason = "Installed Apex" - eventMsg = "Apex installation completed" - r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) + eventReason = "Apex Installation" + eventMsg = "installation of Apex completed" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) n.Status.ApexInstalled = true return requeueN } @@ -1340,7 +1354,7 @@ func (r *OracleRestDataServiceReconciler) deleteSecrets(m *dbapi.OracleRestDataS // Rest Enable/Disable Schemas //############################################################################# func (r *OracleRestDataServiceReconciler) restEnableSchemas(m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase, - sidbReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) ctrl.Result { + sidbReadyPod corev1.Pod, ordsReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) ctrl.Result { log := r.Log.WithValues("restEnableSchemas", req.NamespacedName) @@ -1352,35 +1366,36 @@ func (r *OracleRestDataServiceReconciler) restEnableSchemas(m *dbapi.OracleRestD return requeueY } - // Get Pdbs Available + // Get available PDBs availablePDBS, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", - ctx, req, false, "bash", "-c", fmt.Sprintf("echo -e \"%s\" | %s", dbcommons.GetPdbsSQL, dbcommons.SQLPlusCLI)) + ctx, req, true, "bash", "-c", fmt.Sprintf("echo -e \"%s\" | %s", dbcommons.GetPdbsSQL, dbcommons.SQLPlusCLI)) if err != nil { log.Error(err, err.Error()) return requeueY } else { - log.Info("GetPdbsSQL Output") + log.Info("PDBs found:") log.Info(availablePDBS) } + restartORDS := false + for i := 0; i < len(m.Spec.RestEnableSchemas); i++ { pdbName := m.Spec.RestEnableSchemas[i].PdbName if pdbName == "" { pdbName = n.Spec.Pdbname - r.Log.Info("Defaulting PDB name", "name", pdbName) } // If the PDB mentioned in yaml doesnt contain in the database , continue if !strings.Contains(strings.ToUpper(availablePDBS), strings.ToUpper(pdbName)) { - eventReason := "Warning" - eventMsg := "enabling ORDS schema for PDB : " + pdbName + " failed. PDB not found." + eventReason := "PDB check failed" + eventMsg := "PDB "+ pdbName +" not found for specified schema " + m.Spec.RestEnableSchemas[i].SchemaName log.Info(eventMsg) - r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) continue } - getOrdsSchemaStatus := fmt.Sprintf(dbcommons.GetUserOrdsSchemaStatusSQL, m.Spec.RestEnableSchemas[i].SchemaName, pdbName) + getOrdsSchemaStatus := fmt.Sprintf(dbcommons.GetUserORDSSchemaStatusSQL, m.Spec.RestEnableSchemas[i].SchemaName, pdbName) // Get ORDS Schema status for PDB out, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", @@ -1388,57 +1403,84 @@ func (r *OracleRestDataServiceReconciler) restEnableSchemas(m *dbapi.OracleRestD if err != nil { log.Error(err, err.Error()) return requeueY - } else { - log.Info("getOrdsSchemaStatus Output", "schema", m.Spec.RestEnableSchemas[i].SchemaName) - log.Info(out) } // if ORDS already enabled for given PDB - if strings.Contains(out, "STATUS:ENABLED") && m.Spec.RestEnableSchemas[i].Enable { - continue - } - - // if ORDS already disabled for given PDB - if !strings.Contains(out, "STATUS:ENABLED") && !m.Spec.RestEnableSchemas[i].Enable { - continue - } - - OrdsPasswordSecret := &corev1.Secret{} - // Fetch the secret to get password for database user . Secret has to be created in the same namespace of OracleRestDataService - err = r.Get(ctx, types.NamespacedName{Name: m.Spec.OrdsPassword.SecretName, Namespace: m.Namespace}, OrdsPasswordSecret) - if err != nil { - if apierrors.IsNotFound(err) { - eventReason := "No Secret" - eventMsg := "secret " + m.Spec.OrdsPassword.SecretName + " Not Found" - r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) - r.Log.Info(eventMsg) + if strings.Contains(out, "STATUS:ENABLED") { + if m.Spec.RestEnableSchemas[i].Enable { + log.Info("Schema already enabled", "schema", m.Spec.RestEnableSchemas[i].SchemaName) + continue + } + } else if strings.Contains(out, "STATUS:DISABLED") { + if !m.Spec.RestEnableSchemas[i].Enable { + log.Info("Schema already disabled", "schema", m.Spec.RestEnableSchemas[i].SchemaName) + continue + } + } else if m.Spec.RestEnableSchemas[i].Enable { + OrdsPasswordSecret := &corev1.Secret{} + // Fetch the secret to get password for database user . Secret has to be created in the same namespace of OracleRestDataService + err = r.Get(ctx, types.NamespacedName{Name: m.Spec.OrdsPassword.SecretName, Namespace: m.Namespace}, OrdsPasswordSecret) + if err != nil { + if apierrors.IsNotFound(err) { + eventReason := "No Secret" + eventMsg := "secret " + m.Spec.OrdsPassword.SecretName + " Not Found" + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) + r.Log.Info(eventMsg) + return requeueY + } + log.Error(err, err.Error()) return requeueY } - log.Error(err, err.Error()) - return requeueY + password := string(OrdsPasswordSecret.Data[m.Spec.OrdsPassword.SecretKey]) + // Create users,schemas and grant enableORDS for PDB + createSchemaSQL := fmt.Sprintf(dbcommons.CreateORDSSchemaSQL, m.Spec.RestEnableSchemas[i].SchemaName, password, pdbName) + log.Info("Creating schema", "schema", m.Spec.RestEnableSchemas[i].SchemaName) + _, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | %s", createSchemaSQL, dbcommons.SQLPlusCLI)) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + } else { + log.Info("Noop, ignoring", "schema", m.Spec.RestEnableSchemas[i].SchemaName) + continue } - - password := string(OrdsPasswordSecret.Data[m.Spec.OrdsPassword.SecretKey]) urlMappingPattern := "" if m.Spec.RestEnableSchemas[i].UrlMapping == "" { urlMappingPattern = strings.ToLower(m.Spec.RestEnableSchemas[i].SchemaName) } else { urlMappingPattern = strings.ToLower(m.Spec.RestEnableSchemas[i].UrlMapping) } - enableORDSSchema := fmt.Sprintf(dbcommons.EnableORDSSchemaSQL, strings.ToUpper(m.Spec.RestEnableSchemas[i].SchemaName), password, + enableORDSSchema := fmt.Sprintf(dbcommons.EnableORDSSchemaSQL, m.Spec.RestEnableSchemas[i].SchemaName, strconv.FormatBool(m.Spec.RestEnableSchemas[i].Enable), urlMappingPattern, pdbName) - // Create users,schemas and grant enableORDS for PDB - _, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", + // EnableORDS for Schema + out, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", fmt.Sprintf("echo -e \"%s\" | %s", enableORDSSchema, dbcommons.SQLPlusCLI)) if err != nil { log.Error(err, err.Error()) return requeueY } - log.Info("REST Enabled", "schema", m.Spec.RestEnableSchemas[i].SchemaName) - + log.Info(out) + if m.Spec.RestEnableSchemas[i].Enable { + log.Info("REST Enabled", "schema", m.Spec.RestEnableSchemas[i].SchemaName) + } else { + log.Info("REST Disabled", "schema", m.Spec.RestEnableSchemas[i].SchemaName) + restartORDS = true + } } + if restartORDS { + r.Log.Info("Restarting ORDS Pod "+ordsReadyPod.Name+" to clear disabled schemas cache") + var gracePeriodSeconds int64 = 0 + policy := metav1.DeletePropagationForeground + err = r.Delete(ctx, &ordsReadyPod, &client.DeleteOptions{ + GracePeriodSeconds: &gracePeriodSeconds, PropagationPolicy: &policy}) + if err != nil { + r.Log.Error(err, err.Error()) + } + return requeueY + } return requeueN } From 32d4a9c414b5f172dc60fe56ccba5af25d8f4518 Mon Sep 17 00:00:00 2001 From: Mohammed Yunus Date: Mon, 23 May 2022 08:22:19 +0000 Subject: [PATCH 394/628] Mqureshi bug fixes 3 --- commons/database/constants.go | 2 +- .../database/singleinstancedatabase_controller.go | 14 ++++++++++---- 2 files changed, 11 insertions(+), 5 deletions(-) diff --git a/commons/database/constants.go b/commons/database/constants.go index f82126fb..1c436b42 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -202,7 +202,7 @@ const GetDatabaseRoleCMD string = "SELECT DATABASE_ROLE FROM V\\$DATABASE; " const RunDatapatchCMD string = " ( while true; do sleep 60; echo \"Installing patches...\" ; done ) & if ! $ORACLE_HOME/OPatch/datapatch -skip_upgrade_check;" + " then echo \"Datapatch execution has failed.\" ; else echo \"DONE: Datapatch execution.\" ; fi ; kill -9 $!;" -const GetSqlpatchDescriptionSQL string = "select TARGET_VERSION || ' (' || PATCH_ID || ')' as patchinfo from dba_registry_sqlpatch order by action_time desc;" +const GetSqlpatchDescriptionSQL string = "select TARGET_VERSION || ' (' || ACTION || ' of ' || PATCH_ID || ')' as patchinfo from dba_registry_sqlpatch order by action_time desc;" const GetSqlpatchStatusSQL string = "select status from dba_registry_sqlpatch order by action_time desc;" diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index d76a646c..c14ebe50 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1177,7 +1177,8 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplacePods(m *dbapi.SingleIn // Version/Image changed // PATCHING START (Only Software Patch) - log.Info("Pod image change detected") + log.Info("Pod image change detected, datapatch to be rerun...") + m.Status.DatafilesPatched = "false" // call FindPods() to find pods of older version. Delete all the Pods readyPod, oldReplicasFound, oldAvailable, _, err := dbcommons.FindPods(r, oldVersion, oldImage, m.Name, m.Namespace, ctx, req) @@ -1566,7 +1567,7 @@ func (r *SingleInstanceDatabaseReconciler) validateDBReadiness(m *dbapi.SingleIn } version, out, err := dbcommons.GetDatabaseVersion(readyPod, r, r.Config, ctx, req, m.Spec.Edition) if err == nil { - if !strings.Contains(out, "ORA-") && m.Status.DatafilesPatched != "true" { + if !strings.Contains(out, "ORA-") { m.Status.ReleaseUpdate = version } } @@ -1631,6 +1632,10 @@ func (r *SingleInstanceDatabaseReconciler) runDatapatch(m *dbapi.SingleInstanceD // Datapatch not supported for XE Database if m.Spec.Edition == "express" { + eventReason := "Datapatch Check" + eventMsg := "datapatch not supported for express edition" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + r.Log.Info(eventMsg) return requeueN, nil } @@ -1653,12 +1658,13 @@ func (r *SingleInstanceDatabaseReconciler) runDatapatch(m *dbapi.SingleInstanceD // Get Sqlpatch Description out, err = dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", fmt.Sprintf("echo -e \"%s\" | sqlplus -s / as sysdba ", dbcommons.GetSqlpatchDescriptionSQL)) + releaseUpdate := "" if err == nil { r.Log.Info("GetSqlpatchDescriptionSQL Output") r.Log.Info(out) SqlpatchDescriptions, _ := dbcommons.StringToLines(out) if len(SqlpatchDescriptions) > 0 { - m.Status.ReleaseUpdate = SqlpatchDescriptions[0] + releaseUpdate = SqlpatchDescriptions[0] } } @@ -1672,7 +1678,7 @@ func (r *SingleInstanceDatabaseReconciler) runDatapatch(m *dbapi.SingleInstanceD m.Status.DatafilesPatched = "true" status, versionFrom, versionTo, _ := dbcommons.GetSqlpatchStatus(r, r.Config, readyPod, ctx, req) if versionTo != "" { - eventMsg = "data files patched from " + versionFrom + " to " + versionTo + " : " + status + eventMsg = "data files patched from release update " + versionFrom + " to " + versionTo + ", "+ status + ": " +releaseUpdate } else { eventMsg = "datapatch execution completed" } From aeaa9fd0ca40403a28392fb02434ab7a59f57a9e Mon Sep 17 00:00:00 2001 From: Abhishek Kumar Date: Mon, 23 May 2022 10:53:24 +0000 Subject: [PATCH 395/628] Added support for OCI flex volume cloning --- .../database/v1alpha1/singleinstancedatabase_types.go | 7 ++++--- .../v1alpha1/singleinstancedatabase_webhook.go | 11 ++++++++++- .../database.oracle.com_singleinstancedatabases.yaml | 4 ++++ .../database/singleinstancedatabase_controller.go | 9 +++++++++ 4 files changed, 27 insertions(+), 4 deletions(-) diff --git a/apis/database/v1alpha1/singleinstancedatabase_types.go b/apis/database/v1alpha1/singleinstancedatabase_types.go index 066f4eb7..f400cc8a 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_types.go +++ b/apis/database/v1alpha1/singleinstancedatabase_types.go @@ -80,9 +80,10 @@ type SingleInstanceDatabaseSpec struct { // SingleInstanceDatabasePersistence defines the storage size and class for PVC type SingleInstanceDatabasePersistence struct { - Size string `json:"size,omitempty"` - StorageClass string `json:"storageClass,omitempty"` - AccessMode string `json:"accessMode,omitempty"` + Size string `json:"size,omitempty"` + StorageClass string `json:"storageClass,omitempty"` + AccessMode string `json:"accessMode,omitempty"` + VolClaimAnnotation string `json:"volClaimAnnotation,omitempty"` } // SingleInstanceDatabaseInitParams defines the Init Parameters diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index 749e6414..8b2e478c 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -219,6 +219,15 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { } } + if r.Spec.Persistence.VolClaimAnnotation != "" { + strParts := strings.Split(r.Spec.Persistence.VolClaimAnnotation, ":") + if len(strParts) != 2 { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("persistence").Child("volClaimAnnotation"), r.Spec.Persistence.VolClaimAnnotation, + "volClaimAnnotation should be in : format.")) + } + } + if len(allErrs) == 0 { return nil } @@ -297,7 +306,7 @@ func (r *SingleInstanceDatabase) ValidateDelete() error { var allErrs field.ErrorList if r.Status.OrdsReference != "" { allErrs = append(allErrs, - field.Forbidden(field.NewPath("status").Child("ordsReference"), "delete " + r.Status.OrdsReference+ " to cleanup this SIDB")) + field.Forbidden(field.NewPath("status").Child("ordsReference"), "delete "+r.Status.OrdsReference+" to cleanup this SIDB")) } if len(allErrs) == 0 { return nil diff --git a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml index 941aa627..3af3672e 100644 --- a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml +++ b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml @@ -136,6 +136,8 @@ spec: type: string storageClass: type: string + volClaimAnnotation: + type: string type: object readinessCheckPeriod: type: integer @@ -289,6 +291,8 @@ spec: type: string storageClass: type: string + volClaimAnnotation: + type: string type: object prebuiltDB: type: boolean diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index c14ebe50..a0edbc57 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -894,6 +894,15 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePVCSpec(m *dbapi.SingleIns Labels: map[string]string{ "app": m.Name, }, + Annotations: func() map[string]string { + if m.Spec.Persistence.VolClaimAnnotation != "" { + strParts := strings.Split(m.Spec.Persistence.VolClaimAnnotation, ":") + annotationMap := make(map[string]string) + annotationMap[strParts[0]] = strParts[1] + return annotationMap + } + return nil + }(), }, Spec: corev1.PersistentVolumeClaimSpec{ AccessModes: func() []corev1.PersistentVolumeAccessMode { From 4c65ed6d87530e2790b998bb6b9b24e30c674b36 Mon Sep 17 00:00:00 2001 From: Mohammed Yunus Date: Tue, 24 May 2022 03:09:54 +0000 Subject: [PATCH 396/628] Mqureshi bug fixes 4 --- commons/database/constants.go | 14 ++- .../oraclerestdataservice_controller.go | 111 +++++++++++------- .../singleinstancedatabase_controller.go | 42 +++++-- docs/sidb/README.md | 1 - 4 files changed, 108 insertions(+), 60 deletions(-) diff --git a/commons/database/constants.go b/commons/database/constants.go index 1c436b42..ce944c78 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -406,8 +406,6 @@ const ChownApex string = " chown oracle:oinstall /opt/oracle/oradata/${ORACLE_SI const InstallApex string = "if [ -f /opt/oracle/oradata/${ORACLE_SID^^}/apex/apexins.sql ]; then ( while true; do sleep 60; echo \"Installing Apex...\" ; done ) & " + " cd /opt/oracle/oradata/${ORACLE_SID^^}/apex && echo -e \"@apexins.sql SYSAUX SYSAUX TEMP /i/\" | %[1]s && kill -9 $!; else echo \"Apex Folder doesn't exist\" ; fi ;" -const InstallApexRemote string = "if [ -e ${ORDS_HOME}/config/apex/apxsilentins.sql ]; then cd ${ORDS_HOME}/config/apex/ && echo -e \"@apxsilentins.sql SYSAUX SYSAUX TEMP /i/ %[2]s %[2]s %[2]s %[2]s\" | %[1]s; else echo \"Apex Folder doesn't exist\" ; fi ;" - const InstallApexInContainer string = "cd ${ORDS_HOME}/config/apex/ && echo -e \"@apxsilentins.sql SYSAUX SYSAUX TEMP /i/ %[1]s %[1]s %[1]s %[1]s;\n" + "@apex_rest_config_core.sql;\n" + "exec APEX_UTIL.set_workspace(p_workspace => 'INTERNAL');\n" + @@ -417,14 +415,18 @@ const InstallApexInContainer string = "cd ${ORDS_HOME}/config/apex/ && echo -e \ const IsApexInstalled string = "echo -e \"select 'APEXVERSION:'||version as version FROM DBA_REGISTRY WHERE COMP_ID='APEX';\"" + " | sqlplus -s sys/%[1]s@${ORACLE_HOST}:${ORACLE_PORT}/%[2]s as sysdba;" -const UninstallApex string = "if [ -f /opt/oracle/oradata/${ORACLE_SID^^}/apex/apxremov.sql ]; then ( while true; do sleep 60; echo \"Uninstalling Apex...\" ; done ) & " + - " cd /opt/oracle/oradata/${ORACLE_SID^^}/apex && echo -e \"@apxremov.sql\" | %[1]s && kill -9 $!; else echo \"Apex Folder doesn't exist\" ; fi ;" +const UninstallApex string = "cd ${ORDS_HOME}/config/apex/ && echo -e \"@apxremov.sql\n\" | sqlplus -s sys/%[1]s@${ORACLE_HOST}:${ORACLE_PORT}/%[2]s as sysdba;" const ConfigureApexRest string = "if [ -f ${ORDS_HOME}/config/apex/apex_rest_config.sql ]; then cd ${ORDS_HOME}/config/apex && " + "echo -e \"%[1]s\n%[1]s\" | %[2]s ; else echo \"Apex Folder doesn't exist\" ; fi ;" -const AlterApexUsers string = "echo -e \" ALTER USER APEX_PUBLIC_USER IDENTIFIED BY \\\"%[1]s\\\" ACCOUNT UNLOCK; \n ALTER USER APEX_REST_PUBLIC_USER IDENTIFIED BY \\\"%[1]s\\\" ACCOUNT UNLOCK;" + - " \n ALTER USER APEX_LISTENER IDENTIFIED BY \\\"%[1]s\\\" ACCOUNT UNLOCK;\" | %[2]s" +const AlterApexUsers string = "\nALTER SESSION SET CONTAINER=%[2]s;" + + "\n ALTER USER APEX_PUBLIC_USER IDENTIFIED BY \\\"%[1]s\\\" ACCOUNT UNLOCK; "+ + "\n ALTER USER APEX_REST_PUBLIC_USER IDENTIFIED BY \\\"%[1]s\\\" ACCOUNT UNLOCK;" + + "\n ALTER USER APEX_LISTENER IDENTIFIED BY \\\"%[1]s\\\" ACCOUNT UNLOCK;" + + "\nexec APEX_UTIL.set_workspace(p_workspace => 'INTERNAL');" + + "\nexec APEX_UTIL.EDIT_USER(p_user_id => APEX_UTIL.GET_USER_ID('ADMIN'), p_user_name => 'ADMIN', p_web_password => '%[1]s', p_new_password => '%[1]s');\n" + const CopyApexImages string = " ( while true; do sleep 60; echo \"Copying Apex Images...\" ; done ) & mkdir -p /opt/oracle/oradata/${ORACLE_SID^^}_ORDS/apex/images && " + " cp -R /opt/oracle/oradata/${ORACLE_SID^^}/apex/images/* /opt/oracle/oradata/${ORACLE_SID^^}_ORDS/apex/images; chown -R oracle:oinstall /opt/oracle/oradata/${ORACLE_SID^^}_ORDS/apex; kill -9 $!;" diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 48ab8ae8..ba1924e1 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -203,7 +203,7 @@ func (r *OracleRestDataServiceReconciler) Reconcile(ctx context.Context, req ctr } // Configure Apex - result = r.configureApex(oracleRestDataService, singleInstanceDatabase, ordsReadyPod, ctx, req) + result = r.configureApex(oracleRestDataService, singleInstanceDatabase, sidbReadyPod, ordsReadyPod, ctx, req) if result.Requeue { r.Log.Info("Reconcile queued") return result, nil @@ -268,10 +268,10 @@ func (r *OracleRestDataServiceReconciler) validate(m *dbapi.OracleRestDataServic if err != nil { if apierrors.IsNotFound(err) { m.Status.Status = dbcommons.StatusError - eventReason := "Waiting" - eventMsg := "waiting for secret : " + m.Spec.ApexPassword.SecretName + " to get created" - r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) - r.Log.Info("Secret " + m.Spec.ApexPassword.SecretName + " Not Found") + eventReason := "Apex Password" + eventMsg := "password secret " + m.Spec.ApexPassword.SecretName + " not found, retrying..." + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) + r.Log.Info(eventMsg) return requeueY, nil } r.Log.Error(err, err.Error()) @@ -283,9 +283,9 @@ func (r *OracleRestDataServiceReconciler) validate(m *dbapi.OracleRestDataServic // Validate apexPassword if !dbcommons.ApexPasswordValidator(apexPassword) { m.Status.Status = dbcommons.StatusError - eventReason := "Apex Password Invalid" - eventMsg := "Password should contain: at least 6 chars, at least one numeric character, at least one punctuation character (!\"#$%&()``*+,-/:;?_), at least one upper-case alphabet" - r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + eventReason := "Apex Password" + eventMsg := "password for Apex is invalid, it should contain at least 6 chars, at least one numeric character, at least one punctuation character (!\"#$%&()``*+,-/:;?_), at least one upper-case alphabet" + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) r.Log.Info("APEX password does not conform to the requirements") return requeueY, nil } @@ -324,13 +324,13 @@ func (r *OracleRestDataServiceReconciler) validateSIDBReadiness(m *dbapi.OracleR m.Status.Status = dbcommons.StatusPending if sidbReadyPod.Name == "" || n.Status.Status != dbcommons.StatusReady { - eventReason := "Waiting" - eventMsg := "waiting for database " + n.Name + " status to be Ready" + eventReason := "Database Check" + eventMsg := "status of database " + n.Name + " is not ready, retrying..." r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) return requeueY, sidbReadyPod } else { - eventReason := "Database Healthy" - eventMsg := "database " + n.Name + " status is Ready" + eventReason := "Database Check" + eventMsg := "status of database " + n.Name + " is ready" r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) } @@ -339,10 +339,10 @@ func (r *OracleRestDataServiceReconciler) validateSIDBReadiness(m *dbapi.OracleR err = r.Get(ctx, types.NamespacedName{Name: m.Spec.AdminPassword.SecretName, Namespace: m.Namespace}, adminPasswordSecret) if err != nil { if apierrors.IsNotFound(err) { - eventReason := "Waiting" - eventMsg := "waiting for secret : " + m.Spec.AdminPassword.SecretName + " to get created" - r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) - r.Log.Info("Secret " + m.Spec.AdminPassword.SecretName + " Not Found") + eventReason := "Database Password" + eventMsg := "password secret " + m.Spec.AdminPassword.SecretName + " not found, retrying..." + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) + r.Log.Info(eventMsg) return requeueY, sidbReadyPod } log.Error(err, err.Error()) @@ -1062,8 +1062,8 @@ func (r *OracleRestDataServiceReconciler) cleanupOracleRestDataService(req ctrl. } if sidbReadyPod.Name == "" { - eventReason := "No Ready Pod" - eventMsg := "ommitting ORDS uninstallation as No Ready Pod of " + n.Name + " available" + eventReason := "ORDS Uninstallation" + eventMsg := "skipping ORDS uninstallation as no ready pod for " + n.Name + " is available" r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) return nil } @@ -1103,7 +1103,6 @@ func (r *OracleRestDataServiceReconciler) cleanupOracleRestDataService(req ctrl. for i := 0; i < 5; i++ { err := r.Get(ctx, types.NamespacedName{Name: m.Spec.AdminPassword.SecretName, Namespace: n.Namespace}, adminPasswordSecret) if err != nil { - log.Error(err, err.Error()) if apierrors.IsNotFound(err) { m.Status.Status = dbcommons.StatusError eventReason := "Error" @@ -1114,6 +1113,8 @@ func (r *OracleRestDataServiceReconciler) cleanupOracleRestDataService(req ctrl. time.Sleep(15 * time.Second) continue } + } else { + log.Error(err, err.Error()) } } else { adminPasswordSecretFound = true @@ -1129,18 +1130,33 @@ func (r *OracleRestDataServiceReconciler) cleanupOracleRestDataService(req ctrl. } if adminPasswordSecretFound && readyPod.Name != "" { adminPassword := string(adminPasswordSecret.Data[m.Spec.AdminPassword.SecretKey]) + //Uninstall Apex + eventReason := "Apex Uninstallation" + eventMsg := "Uninstalling Apex..." + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) + log.Info(eventMsg) + out, err = dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, true, "bash", "-c", + fmt.Sprintf(dbcommons.UninstallApex, adminPassword, n.Status.Pdbname)) + if err != nil { + log.Info(err.Error()) + } + n.Status.ApexInstalled = false // To reinstall Apex when ORDS is reinstalled + log.Info("Apex Uninstall : " + out) + //Uninstall Apex + eventReason = "ORDS Uninstallation" + eventMsg = "Uninstalling ORDS..." + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) + log.Info(eventMsg) uninstallORDS := fmt.Sprintf(dbcommons.UninstallORDSCMD, adminPassword) - out, err = dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, true, "bash", "-c", uninstallORDS) - log.Info("UninstallORDSCMD Output : " + out) + log.Info("ORDS Uninstall : " + out) if strings.Contains(strings.ToUpper(out), "ERROR") { return errors.New(out) } if err != nil { log.Info(err.Error()) } - log.Info("UninstallORDSCMD Output : " + out) } // Drop Admin Users @@ -1175,7 +1191,7 @@ func (r *OracleRestDataServiceReconciler) cleanupOracleRestDataService(req ctrl. // Configure APEX //############################################################################# func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase, - ordsReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) ctrl.Result { + sidbReadyPod corev1.Pod, ordsReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) ctrl.Result { log := r.Log.WithValues("configureApex", req.NamespacedName) if m.Spec.ApexPassword.SecretName == "" { @@ -1191,10 +1207,10 @@ func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataS if err != nil { if apierrors.IsNotFound(err) { m.Status.Status = dbcommons.StatusError - eventReason := "Waiting" - eventMsg := "waiting for secret : " + m.Spec.ApexPassword.SecretName + " to get created" - r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) - r.Log.Info("Secret " + m.Spec.ApexPassword.SecretName + " Not Found") + eventReason := "Apex Password" + eventMsg := "password secret " + m.Spec.ApexPassword.SecretName + " not found, retrying..." + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) + r.Log.Info(eventMsg) return requeueY } log.Error(err, err.Error()) @@ -1210,7 +1226,18 @@ func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataS log.Info("Reconcile requeued because apex installation failed") return result } + } else { + // Alter Apex Users + log.Info("Alter APEX Users") + _, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", + ctx, req, true, "bash", "-c", fmt.Sprintf("echo -e \"%s\" | %s", + fmt.Sprintf(dbcommons.AlterApexUsers, apexPassword, n.Spec.Pdbname), dbcommons.SQLPlusCLI)) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } } + // Set Apex users in apex_rt,apex_al,apex files out, err := dbcommons.ExecCommand(r, r.Config, ordsReadyPod.Name, ordsReadyPod.Namespace, "", ctx, req, true, "bash", "-c", fmt.Sprintf(dbcommons.SetApexUsers, apexPassword)) @@ -1255,10 +1282,10 @@ func (r *OracleRestDataServiceReconciler) installApex(m *dbapi.OracleRestDataSer if err != nil { if apierrors.IsNotFound(err) { m.Status.Status = dbcommons.StatusError - eventReason := "Waiting" - eventMsg := "waiting for secret : " + m.Spec.AdminPassword.SecretName + " to get created" - r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) - r.Log.Info("Secret " + m.Spec.AdminPassword.SecretName + " Not Found") + eventReason := "Database Password" + eventMsg := "password secret " + m.Spec.AdminPassword.SecretName + " not found, retrying..." + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) + r.Log.Info(eventMsg) return requeueY } log.Error(err, err.Error()) @@ -1270,7 +1297,7 @@ func (r *OracleRestDataServiceReconciler) installApex(m *dbapi.OracleRestDataSer m.Status.Status = dbcommons.StatusUpdating r.Status().Update(ctx, m) eventReason := "Apex Installation" - eventMsg := "waiting for Apex installation to complete" + eventMsg := "Performing install of Apex in database " + m.Spec.DatabaseRef r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) //Install Apex in SIDB ready pod @@ -1292,12 +1319,16 @@ func (r *OracleRestDataServiceReconciler) installApex(m *dbapi.OracleRestDataSer apexInstalled := "APEXVERSION:" if !strings.Contains(out, apexInstalled) { + eventReason = "Apex Installation" + eventMsg = "Unable to determine Apex version, retrying install..." + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) return requeueY } m.Status.Status = dbcommons.StatusReady eventReason = "Apex Installation" - eventMsg = "installation of Apex completed" + outArr := strings.Split(out, apexInstalled) + eventMsg = "installation of Apex "+ strings.TrimSpace(outArr[len(outArr)-1]) +" completed" r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) n.Status.ApexInstalled = true return requeueN @@ -1317,7 +1348,7 @@ func (r *OracleRestDataServiceReconciler) deleteSecrets(m *dbapi.OracleRestDataS //Delete Database Admin Password Secret . err := r.Delete(ctx, adminPasswordSecret, &client.DeleteOptions{}) if err == nil { - log.Info("Database Admin Password secret Deleted : " + adminPasswordSecret.Name) + log.Info("Database admin password secret deleted : " + adminPasswordSecret.Name) } } } @@ -1330,7 +1361,7 @@ func (r *OracleRestDataServiceReconciler) deleteSecrets(m *dbapi.OracleRestDataS //Delete ORDS Password Secret . err := r.Delete(ctx, ordsPasswordSecret, &client.DeleteOptions{}) if err == nil { - log.Info("ORDS Password secret Deleted : " + ordsPasswordSecret.Name) + log.Info("ORDS password secret deleted : " + ordsPasswordSecret.Name) } } } @@ -1343,7 +1374,7 @@ func (r *OracleRestDataServiceReconciler) deleteSecrets(m *dbapi.OracleRestDataS //Delete APEX Password Secret . err := r.Delete(ctx, apexPasswordSecret, &client.DeleteOptions{}) if err == nil { - log.Info("APEX Password secret Deleted : " + apexPasswordSecret.Name) + log.Info("APEX password secret deleted : " + apexPasswordSecret.Name) } } } @@ -1359,9 +1390,9 @@ func (r *OracleRestDataServiceReconciler) restEnableSchemas(m *dbapi.OracleRestD log := r.Log.WithValues("restEnableSchemas", req.NamespacedName) if sidbReadyPod.Name == "" || n.Status.Status != dbcommons.StatusReady { - eventReason := "Waiting" - eventMsg := "waiting for " + n.Name + " to be Ready" - r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + eventReason := "Database Check" + eventMsg := "status of database " + n.Name + " is not ready, retrying..." + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) m.Status.Status = dbcommons.StatusNotReady return requeueY } @@ -1388,7 +1419,7 @@ func (r *OracleRestDataServiceReconciler) restEnableSchemas(m *dbapi.OracleRestD // If the PDB mentioned in yaml doesnt contain in the database , continue if !strings.Contains(strings.ToUpper(availablePDBS), strings.ToUpper(pdbName)) { - eventReason := "PDB check failed" + eventReason := "PDB Check" eventMsg := "PDB "+ pdbName +" not found for specified schema " + m.Spec.RestEnableSchemas[i].SchemaName log.Info(eventMsg) r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index a0edbc57..bd15014b 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -412,7 +412,7 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab if n.Status.Status != dbcommons.StatusReady { m.Status.Status = dbcommons.StatusPending eventReason := "Source Database Pending" - eventMsg := "waiting for source database " + m.Spec.CloneFrom + " to be Ready" + eventMsg := "status of database " + m.Spec.CloneFrom + " is not ready, retrying..." r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) err = errors.New(eventMsg) return requeueY, err @@ -420,8 +420,8 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab if !n.Spec.ArchiveLog { m.Status.Status = dbcommons.StatusPending - eventReason := "Source Database Pending" - eventMsg := "waiting for ArchiveLog to turn ON " + n.Name + eventReason := "Source Database Check" + eventMsg := "enable ArchiveLog for database " + n.Name r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) r.Log.Info(eventMsg) err = errors.New(eventMsg) @@ -489,13 +489,29 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, TopologyKey: "kubernetes.io/hostname", }, - }, - }, + }}, }, } } } - return nil + // For ReadWriteMany Access, spread out the PODs + return &corev1.Affinity{ + PodAntiAffinity: &corev1.PodAntiAffinity{ + PreferredDuringSchedulingIgnoredDuringExecution: []corev1.WeightedPodAffinityTerm{{ + Weight: 100, + PodAffinityTerm: corev1.PodAffinityTerm{ + LabelSelector: &metav1.LabelSelector{ + MatchExpressions: []metav1.LabelSelectorRequirement{{ + Key: "app", + Operator: metav1.LabelSelectorOpIn, + Values: []string{m.Name}, + }}, + }, + TopologyKey: "kubernetes.io/hostname", + }, + }}, + }, + } }(), Volumes: []corev1.Volume{{ Name: "datamount", @@ -1221,7 +1237,7 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplacePods(m *dbapi.SingleIn if ok, _ := dbcommons.IsAnyPodWithStatus(newAvailable, corev1.PodRunning); !ok { eventReason := "Database Pending" eventMsg := "waiting for pod with changed image to get to running state" - r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) log.Info(eventMsg) for i := 0; i < len(newAvailable); i++ { @@ -1513,14 +1529,14 @@ func (r *SingleInstanceDatabaseReconciler) validateDBReadiness(m *dbapi.SingleIn } if readyPod.Name == "" { eventReason := "Database Pending" - eventMsg := "waiting for database pod to be ready" + eventMsg := "status of database is not ready, retrying..." m.Status.Status = dbcommons.StatusPending if ok, _ := dbcommons.IsAnyPodWithStatus(available, corev1.PodFailed); ok { eventReason = "Database Failed" eventMsg = "pod creation failed" } else if ok, runningPod := dbcommons.IsAnyPodWithStatus(available, corev1.PodRunning); ok { eventReason = "Database Creating" - eventMsg = "waiting for database to be ready" + eventMsg = "database creation in progress..." m.Status.Status = dbcommons.StatusCreating if m.Spec.CloneFrom != "" { // Required since clone creates the datafiles under primary database SID folder @@ -1843,8 +1859,8 @@ func (r *SingleInstanceDatabaseReconciler) updateDBConfig(m *dbapi.SingleInstanc } else { // Occurs when flashback is attempted to be turned on without turning on archiving first - eventReason := "Waiting" - eventMsg := "enable ArchiveLog to turn ON Flashback" + eventReason := "Database Check" + eventMsg := "enable ArchiveLog to turn on Flashback" r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) log.Info(eventMsg) @@ -1886,8 +1902,8 @@ func (r *SingleInstanceDatabaseReconciler) updateDBConfig(m *dbapi.SingleInstanc } else { // Occurs when archiving is attempted to be turned off without turning off flashback first - eventReason := "Waiting" - eventMsg := "turn OFF Flashback to disable ArchiveLog" + eventReason := "Database Check" + eventMsg := "turn off Flashback to disable ArchiveLog" r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) log.Info(eventMsg) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 477a0037..616ece12 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -639,7 +639,6 @@ password: `.spec.apexPassword` **NOTE:** - By default, the full development environment is initialized in APEX. After deployment, you can change it manually to the runtime environment. To change environments, run the script `apxdevrm.sql` after connecting to the primary database from the ORDS pod as the `SYS` user with `SYSDBA` privilege. For detailed instructions, see: [Converting a Full Development Environment to a Runtime Environment](https://docs.oracle.com/en/database/oracle/application-express/21.2/htmig/converting-between-runtime-and-full-development-environments.html#GUID-B0621B40-3441-44ED-9D86-29B058E26BE9). -- Deleting ORDS resource will not delete APEX from the SIDB. Currently, performing APEX uninstall is not supported through the Operator. ## Performing maintenance operations If you need to perform some maintenance operations manually, then the procedure is as follows: From 8bb3ca6a1be48497bbdeadeab5fe9cc65319080e Mon Sep 17 00:00:00 2001 From: Mohammed Yunus Date: Tue, 24 May 2022 03:56:12 +0000 Subject: [PATCH 397/628] Fix Apex Uninstall --- .../oraclerestdataservice_controller.go | 37 ++++++++++--------- 1 file changed, 20 insertions(+), 17 deletions(-) diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index ba1924e1..d564c935 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -360,9 +360,10 @@ func (r *OracleRestDataServiceReconciler) validateSIDBReadiness(m *dbapi.OracleR log.Info("validated Admin password successfully") } else if strings.Contains(out, "ORA-01017") { m.Status.Status = dbcommons.StatusError - eventReason := "Logon denied" - eventMsg := "invalid databaseRef admin password secret: " + m.Spec.AdminPassword.SecretName + eventReason := "Database Check" + eventMsg := "login denied, invalid database admin password in secret " + m.Spec.AdminPassword.SecretName r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) + log.Info(eventMsg) return requeueY, sidbReadyPod } else { return requeueY, sidbReadyPod @@ -1130,27 +1131,29 @@ func (r *OracleRestDataServiceReconciler) cleanupOracleRestDataService(req ctrl. } if adminPasswordSecretFound && readyPod.Name != "" { adminPassword := string(adminPasswordSecret.Data[m.Spec.AdminPassword.SecretKey]) - //Uninstall Apex - eventReason := "Apex Uninstallation" - eventMsg := "Uninstalling Apex..." - r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) - log.Info(eventMsg) - out, err = dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, true, "bash", "-c", - fmt.Sprintf(dbcommons.UninstallApex, adminPassword, n.Status.Pdbname)) - if err != nil { - log.Info(err.Error()) + if n.Status.ApexInstalled { + //Uninstall Apex + eventReason := "Apex Uninstallation" + eventMsg := "Uninstalling Apex..." + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) + log.Info(eventMsg) + out, err = dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, true, "bash", "-c", + fmt.Sprintf(dbcommons.UninstallApex, adminPassword, n.Status.Pdbname)) + if err != nil { + log.Info(err.Error()) + } + n.Status.ApexInstalled = false // To reinstall Apex when ORDS is reinstalled + log.Info("Apex uninstall output: " + out) } - n.Status.ApexInstalled = false // To reinstall Apex when ORDS is reinstalled - log.Info("Apex Uninstall : " + out) - //Uninstall Apex - eventReason = "ORDS Uninstallation" - eventMsg = "Uninstalling ORDS..." + //Uninstall ORDS + eventReason := "ORDS Uninstallation" + eventMsg := "Uninstalling ORDS..." r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) log.Info(eventMsg) uninstallORDS := fmt.Sprintf(dbcommons.UninstallORDSCMD, adminPassword) out, err = dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, true, "bash", "-c", uninstallORDS) - log.Info("ORDS Uninstall : " + out) + log.Info("ORDS uninstall output: " + out) if strings.Contains(strings.ToUpper(out), "ERROR") { return errors.New(out) } From 5e35fdaa1b259eb21034a08a0e09b816bb4b1731 Mon Sep 17 00:00:00 2001 From: Abhishek Kumar Date: Tue, 24 May 2022 08:34:42 +0000 Subject: [PATCH 398/628] Readme update --- docs/sidb/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 616ece12..c44edb6c 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -97,7 +97,7 @@ $ kubectl describe singleinstancedatabase sidb-sample-clone Force Log: false Oem Express URL: https://10.0.25.58:5500/em Pdb Name: orclpdb1 - Release Update: 19.11.0.0.0 (32545013) + Release Update: 19.11.0.0.0 Replicas: 2 Role: PRIMARY Sid: ORCL1C @@ -356,7 +356,7 @@ $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.datafiles $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.releaseUpdate}" - 19.3.0.0.0 (29517242) + 19.3.0.0.0 ``` ## Kind OracleRestDataService From 2718bfac9eefbc6bd5316fc82b6eb42e168f25be Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Tue, 24 May 2022 19:52:07 +0530 Subject: [PATCH 399/628] Add more logging --- .../oraclerestdataservice_controller.go | 30 ++++++++++++------- .../singleinstancedatabase_controller.go | 12 +++++--- 2 files changed, 27 insertions(+), 15 deletions(-) diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index d564c935..faa1b44f 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -828,10 +828,13 @@ func (r *OracleRestDataServiceReconciler) createSVC(ctx context.Context, req ctr log.Info("Creating a new Service", "Service.Namespace", svc.Namespace, "Service.Name", svc.Name) err = r.Create(ctx, svc) if err != nil { - log.Error(err, "Failed to create new Service", "Service.Namespace", svc.Namespace, "Service.Name", svc.Name) + log.Error(err, "Failed to create new service", "Service.Namespace", svc.Namespace, "Service.Name", svc.Name) return requeueY } else { - log.Info("Succesfully Created New Service ", "Service.Name : ", svc.Name) + eventReason := "Service creation" + eventMsg := "successfully created service type " + string(svc.Spec.Type) + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + log.Info(eventMsg) } } else if err != nil { @@ -1163,12 +1166,12 @@ func (r *OracleRestDataServiceReconciler) cleanupOracleRestDataService(req ctrl. } // Drop Admin Users - out, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + out, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", fmt.Sprintf("echo -e \"%s\" | %s ", dbcommons.DropAdminUsersSQL, dbcommons.SQLPlusCLI)) if err != nil { log.Info(err.Error()) } - log.Info("DropAdminUsersSQL Output : " + out) + log.Info("Drop admin users: " + out) //Delete ORDS pod var gracePeriodSeconds int64 = 0 @@ -1255,10 +1258,6 @@ func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataS } } - m.Status.ApexConfigured = true - r.Status().Update(ctx, m) - log.Info("ConfigureApex Successful !") - // ORDS needs to be restarted to configure APEX r.Log.Info("Restarting ORDS Pod to complete APEX configuration: " + ordsReadyPod.Name) var gracePeriodSeconds int64 = 0 @@ -1268,6 +1267,14 @@ func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataS if err != nil { r.Log.Error(err, err.Error()) } + + m.Status.ApexConfigured = true + r.Status().Update(ctx, m) + eventReason := "Apex Configuration" + eventMsg := "configuration of Apex completed!" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + log.Info(eventMsg) + // Cannot return requeue as the secrets will be deleted if keepSecert is false, which cause problem in pod restart return requeueY } @@ -1300,7 +1307,7 @@ func (r *OracleRestDataServiceReconciler) installApex(m *dbapi.OracleRestDataSer m.Status.Status = dbcommons.StatusUpdating r.Status().Update(ctx, m) eventReason := "Apex Installation" - eventMsg := "Performing install of Apex in database " + m.Spec.DatabaseRef + eventMsg := "performing install of Apex in database " + m.Spec.DatabaseRef r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) //Install Apex in SIDB ready pod @@ -1309,7 +1316,7 @@ func (r *OracleRestDataServiceReconciler) installApex(m *dbapi.OracleRestDataSer if err != nil { log.Info(err.Error()) } - log.Info(" InstallApex Output : \n" + out) + log.Info("Apex installation output : \n" + out) // Checking if Apex is installed successfully or not out, err = dbcommons.ExecCommand(r, r.Config, ordsReadyPod.Name, ordsReadyPod.Namespace, "", ctx, req, true, "bash", "-c", @@ -1318,7 +1325,7 @@ func (r *OracleRestDataServiceReconciler) installApex(m *dbapi.OracleRestDataSer log.Error(err, err.Error()) return requeueY } - log.Info("IsApexInstalled Output: \n" + out) + log.Info("Is Apex installed: \n" + out) apexInstalled := "APEXVERSION:" if !strings.Contains(out, apexInstalled) { @@ -1334,6 +1341,7 @@ func (r *OracleRestDataServiceReconciler) installApex(m *dbapi.OracleRestDataSer eventMsg = "installation of Apex "+ strings.TrimSpace(outArr[len(outArr)-1]) +" completed" r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) n.Status.ApexInstalled = true + r.Status().Update(ctx, n) return requeueN } diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index bd15014b..10edc369 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1033,10 +1033,10 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex } if svc.Spec.Type != svcType { - log.Info("Deleting SVC", " name ", svc.Name) + log.Info("Deleting service", "name", svc.Name) err = r.Delete(ctx, svc) if err != nil { - r.Log.Error(err, "Failed to delete svc", " Name", svc.Name) + r.Log.Error(err, "Failed to delete service", "name", svc.Name) return requeueN, err } svcDeleted = true @@ -1045,12 +1045,16 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex if svcDeleted || err != nil && apierrors.IsNotFound(err) { // Define a new Service svc = r.instantiateSVCSpec(m) - log.Info("Creating a new Service", "Service.Namespace", svc.Namespace, "Service.Name", svc.Name) + log.Info("Creating a new service", "Service.Namespace", svc.Namespace, "Service.Name", svc.Name) err = r.Create(ctx, svc) if err != nil { - log.Error(err, "Failed to create new Service", "Service.Namespace", svc.Namespace, "Service.Name", svc.Name) + log.Error(err, "Failed to create new service", "Service.Namespace", svc.Namespace, "Service.Name", svc.Name) return requeueY, err } + eventReason := "Service creation" + eventMsg := "successfully created service type " + string(svc.Spec.Type) + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + log.Info(eventMsg) // Reset connect strings whenever service is recreated /* m.Status.ConnectString = dbcommons.ValueUnavailable m.Status.PdbConnectString = dbcommons.ValueUnavailable From 82943726e62f73918dc5a6168cddd7c0a2fc0212 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Tue, 24 May 2022 21:10:56 +0530 Subject: [PATCH 400/628] Enhance kill session query --- commons/database/constants.go | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/commons/database/constants.go b/commons/database/constants.go index ce944c78..3c899e51 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -318,7 +318,12 @@ const InitORDSCMD string = "if [ -f $ORDS_HOME/config/ords/defaults.xml ]; then "\numask 022" const GetSessionInfoSQL string = "select s.sid || ',' || s.serial# as Info FROM v\\$session s, v\\$process p " + - "WHERE (s.username = 'ORDS_PUBLIC_USER' or s.username = 'C##_DBAPI_PDB_ADMIN' ) AND p.addr(+) = s.paddr;" + "WHERE (s.username = 'ORDS_PUBLIC_USER' or "+ + "s.username = 'APEX_PUBLIC_USER' or "+ + "s.username = 'APEX_REST_PUBLIC_USER' or "+ + "s.username = 'APEX_LISTENER' or "+ + "s.username = 'C##_DBAPI_CDB_ADMIN' or "+ + "s.username = 'C##_DBAPI_PDB_ADMIN' ) AND p.addr(+) = s.paddr;" const KillSessionSQL string = "alter system kill session '%[1]s';" From 26f62975581ec57799ab51cc5d63993628495d00 Mon Sep 17 00:00:00 2001 From: Abhishek Kumar Date: Wed, 25 May 2022 08:36:30 +0000 Subject: [PATCH 401/628] Correction in the validation condition for flashback and archivelog --- apis/database/v1alpha1/singleinstancedatabase_webhook.go | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index 8b2e478c..4b2319f1 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -203,7 +203,7 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { } } - if r.Status.FlashBack == "true" { + if r.Status.FlashBack == "true" && r.Spec.FlashBack { if !r.Spec.ArchiveLog { allErrs = append(allErrs, field.Invalid(field.NewPath("spec").Child("archiveLog"), r.Spec.ArchiveLog, @@ -211,7 +211,7 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { } } - if !r.Spec.ArchiveLog { + if r.Status.ArchiveLog == "false" && !r.Spec.ArchiveLog { if r.Spec.FlashBack { allErrs = append(allErrs, field.Invalid(field.NewPath("spec").Child("flashBack"), r.Spec.FlashBack, From 3579d6d51d6fd3092551cbe3fb7d5f5c57c2cc5e Mon Sep 17 00:00:00 2001 From: Mohammed Yunus Date: Wed, 25 May 2022 10:09:14 +0000 Subject: [PATCH 402/628] Fix status.Replicas --- controllers/database/singleinstancedatabase_controller.go | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 10edc369..8e8c9635 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1423,7 +1423,8 @@ func (r *SingleInstanceDatabaseReconciler) createPods(m *dbapi.SingleInstanceDat m.Status.Status = dbcommons.StatusPending firstPod = true } - // if Found < Required , Create New Pods , Name of Pods are generated Randomly + m.Status.Replicas = replicasFound + // if Found < Required, create new pods, name of pods are generated randomly for i := replicasFound; i < replicasReq; i++ { // mandatory pod affinity if it is replica based patching or not the first pod pod := r.instantiatePodSpec(m, n, replicaPatching || !firstPod) From 28d7e6f404e88e751843fa5324d576d0e089e690 Mon Sep 17 00:00:00 2001 From: Abhishek Kumar Date: Thu, 26 May 2022 05:26:55 +0000 Subject: [PATCH 403/628] Added length validation for ORACLE_SID --- apis/database/v1alpha1/singleinstancedatabase_types.go | 3 ++- .../bases/database.oracle.com_singleinstancedatabases.yaml | 5 +++-- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/apis/database/v1alpha1/singleinstancedatabase_types.go b/apis/database/v1alpha1/singleinstancedatabase_types.go index f400cc8a..00289d80 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_types.go +++ b/apis/database/v1alpha1/singleinstancedatabase_types.go @@ -53,9 +53,10 @@ type SingleInstanceDatabaseSpec struct { // +kubebuilder:validation:Enum=standard;enterprise;express Edition string `json:"edition,omitempty"` - // SID can only have a-z , A-Z, 0-9 . It cant have any special characters + // SID must be alphanumeric (no special characters, only a-z, A-Z, 0-9), and no longer than 12 characters. // +k8s:openapi-gen=true // +kubebuilder:validation:Pattern=`^[a-zA-Z0-9]+$` + // +kubebuilder:validation:MaxLength:=12 Sid string `json:"sid,omitempty"` Charset string `json:"charset,omitempty"` Pdbname string `json:"pdbName,omitempty"` diff --git a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml index 3af3672e..3d230bd9 100644 --- a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml +++ b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml @@ -146,8 +146,9 @@ spec: serviceAccountName: type: string sid: - description: SID can only have a-z , A-Z, 0-9 . It cant have any special - characters + description: SID must be alphanumeric (no special characters, only + a-z, A-Z, 0-9), and no longer than 12 characters. + maxLength: 12 pattern: ^[a-zA-Z0-9]+$ type: string required: From 088fa2eb6403ec5dd87e8032a80c8fe13e0c3eef Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Thu, 26 May 2022 15:39:47 +0530 Subject: [PATCH 404/628] Update status.replicas condition --- controllers/database/singleinstancedatabase_controller.go | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 8e8c9635..56347257 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1423,7 +1423,9 @@ func (r *SingleInstanceDatabaseReconciler) createPods(m *dbapi.SingleInstanceDat m.Status.Status = dbcommons.StatusPending firstPod = true } - m.Status.Replicas = replicasFound + if !replicaPatching { + m.Status.Replicas = replicasFound + } // if Found < Required, create new pods, name of pods are generated randomly for i := replicasFound; i < replicasReq; i++ { // mandatory pod affinity if it is replica based patching or not the first pod From c1014e633ee6f8a931887264bc184670a2fdbcd5 Mon Sep 17 00:00:00 2001 From: douglas_williams Date: Fri, 27 May 2022 16:17:22 +0000 Subject: [PATCH 405/628] Documentation style edit --- docs/dbcs/README.md | 38 +++++++++++++++++++++----------------- 1 file changed, 21 insertions(+), 17 deletions(-) diff --git a/docs/dbcs/README.md b/docs/dbcs/README.md index 2d40a52a..1adf0944 100644 --- a/docs/dbcs/README.md +++ b/docs/dbcs/README.md @@ -1,8 +1,10 @@ # Using the DB Operator DBCS Controller -The Oracle Cloud Infrastructure's Database Service furnishes [co-managed Oracle Database cloud solutions](https://docs.oracle.com/en-us/iaas/Content/Database/Concepts/overview.htm). A single-node DB systems on either bare metal or virtual machines, and 2-node RAC DB systems on virtual machines. To manage the life cycle of an OCI DBCS system, we can use the OCI Console, the REST API or the Oracle Cloud Infrastructure CLI and at the granular level, the Database CLI (DBCLI), Enterprise Manager, or SQL Developer. +Oracle Cloud Infastructure (OCI) Database Cloud Service (DBCS) provides single-node Database (DB) systems, deployed either bare metal or virtual machines, and provides two-node Oracle Real Appliation Clusters (Oracle RAC) database systems on virtual machines. -The Oracle DB Operator DBCS Controller is a feature of the Oracle DB Operator for Kubernetes (a.k.a. OraOperator) which supports the life cycle management of the Database Systems deployed using OCI's DBCS Service. +The single-node DB systems and Oracle RAC systems on virtual machines are [co-managed Oracle Database cloud solutions](https://docs.oracle.com/en-us/iaas/Content/Database/Concepts/overview.htm). To manage the lifecycle of an OCI DBCS system, you can use the OCI Console, the REST API, or the Oracle Cloud Infrastructure command-line interface (CLI). At the granular level, you can use the Oracle Database CLI (DBCLI), Oracle Enterprise Manager, or Oracle SQL Developer. + +The Oracle DB Operator DBCS Controller is a feature of the Oracle DB Operator for Kubernetes (OraOperator) which uses OCI's DBCS service to support lifecycle management of the database systems. # Supported Database Editions and Versions @@ -16,7 +18,7 @@ All single-node OCI Oracle RAC DB systems support the following Oracle Database Two-node Oracle RAC DB systems require Oracle Enterprise Edition - Extreme Performance. -For standard provisioning of DB systems (using Oracle Automatic Storage Management (ASM) as your storage management software), the supported database versions are: +For standard provisioning of DB systems (using Oracle Automatic Storage Management (ASM) as your storage management software), the following database releases are supported: - Oracle Database 21c - Oracle Database 19c @@ -26,7 +28,7 @@ For standard provisioning of DB systems (using Oracle Automatic Storage Manageme - Oracle Database 11g Release 2 (11.2) -For fast provisioning of single-node virtual machine database systems (using Logical Volume Manager as your storage management software), the supported database versions are: +For fast provisioning of single-node virtual machine database systems (using Logical Volume Manager as your storage management software), the following database releases are supported: - Oracle Database 21c - Oracle Database 19c @@ -36,9 +38,9 @@ For fast provisioning of single-node virtual machine database systems (using Log # Oracle DB Operator DBCS Controller Deployment -The step by step procedure to deploy the OraOperator is documented [here](https://github.com/oracle/oracle-database-operator/blob/main/README.md). +To deploy OraOperator, use this [Oracle Database Operator for Kubernetes](https://github.com/oracle/oracle-database-operator/blob/main/README.md) step-by-step procedure. -Once the Oracle DB Operator has been deployed, we can see the DB operator pods running in the Kubernetes Cluster. The DBCS Controller will deployed as part of the Oracle DB Operator Deployment as a CRD (Custom Resource Definition). Below is an example of such a deployment: +After the Oracle Database Operator is deployed, you can see the DB operator pods running in the Kubernetes Cluster. As part of the OraOperator deployment, the DBCS Controller is deployed as a CRD (Custom Resource Definition). The following screen output is an example of such a deployment: ``` [root@test-server oracle-database-operator]# kubectl get ns NAME STATUS AGE @@ -87,11 +89,11 @@ singleinstancedatabases.database.oracle.com 2022-02-22T23:23:25Z # Prerequsites to deploy a DBCS system using Oracle DB Operator DBCS Controller -Complete the following steps before deploying a DBCS system in OCI using the Oracle DB Operator DBCS Controller: +Before you deploy a DBCS system in OCI using the Oracle DB Operator DBCS Controller, complete the following procedure. -**IMPORTANT :** You must make the changes specified in this section before you proceed to the next section. +**CAUTION :** You must make the changes specified in this section before you proceed to the next section. -## 1. Create a Kubernetes Configmap. For example: We are creating a Kubernetes Configmap named "oci-cred" using the OCI account we are using as below: +## 1. Create a Kubernetes Configmap. For example: We are creating a Kubernetes Configmap named `oci-cred` using the OCI account we are using as below: ``` kubectl create configmap oci-cred \ @@ -102,7 +104,7 @@ kubectl create configmap oci-cred \ ``` -## 2. Create a Kubernetes secret "oci-privatekey" using the OCI Pem key taken from OCI console for the account you are using: +## 2. Create a Kubernetes secret `oci-privatekey` using the OCI Pem key taken from OCI console for the account you are using: ``` -- assuming the OCI Pem key to be "/root/.oci/oci_api_key.pem" @@ -111,7 +113,8 @@ kubectl create secret generic oci-privatekey --from-file=privatekey=/root/.oci/o ``` -## 3. Create a Kubernetes secret named "admin-password". This passward needs to satisfy the minimum passward requirements for the OCI DBCS Service. For example: +## 3. Create a Kubernetes secret named `admin-password`; This passward must meet the minimum passward requirements for the OCI DBCS Service. +For example: ``` -- assuming the passward has been added to a text file named "admin-password": @@ -120,7 +123,8 @@ kubectl create secret generic admin-password --from-file=./admin-password -n def ``` -## 4. Create a Kubernetes secret named "tde-password". This passward needs to satisfy the minimum passward requirements for the OCI DBCS Service. For example: +## 4. Create a Kubernetes secret named `tde-password`; this passward must meet the minimum passward requirements for the OCI DBCS Service. +For example: ``` -- assuming the passward has been added to a text file named "tde-password": @@ -129,7 +133,7 @@ kubectl create secret generic tde-password --from-file=./tde-password -n default ``` -## 5. Create an ssh key pair and use its public key to create a Kubernetes secret named "oci-publickey". The private key for this public key can be used later to access the DBCS system's host machine using ssh: +## 5. Create an ssh key pair, and use its public key to create a Kubernetes secret named `oci-publickey`; the private key for this public key can be used later to access the DBCS system's host machine using ssh: ``` [root@test-server DBCS]# ssh-keygen -N "" -C "DBCS_System"-`date +%Y%m` -P "" @@ -159,9 +163,9 @@ The key's randomart image is: -# Use Cases to manage life cycle of a OCI DBCS System using Oracle DB Operator DBCS Controller +# Use Cases to manage the lifecycle of an OCI DBCS System with Oracle DB Operator DBCS Controller -There are multiple use cases to deploy and manage the OCI DBCS Service based database using the Oracle DB Operator DBCS Controller. +For more informatoin about the multiple use cases available to you to deploy and manage the OCI DBCS Service-based database using the Oracle DB Operator DBCS Controller, review this list: [1. Deploy a DB System using OCI DBCS Service with minimal parameters](./provisioning/dbcs_service_with_minimal_parameters.md) [2. Binding to an existing DBCS System already deployed in OCI DBCS Service](./provisioning/bind_to_existing_dbcs_system.md) @@ -176,8 +180,8 @@ There are multiple use cases to deploy and manage the OCI DBCS Service based dat ## Connecting to OCI DBCS database deployed using Oracle DB Operator DBCS Controller -After the OCI DBCS database has been deployed using Oracle DB Operator DBCS Controller, you can follow the steps in this document to connect to this Database: [Database Connectivity](./provisioning/database_connection.md) +After you have deployed the OCI DBCS database with the Oracle DB Operator DBCS Controller, you can connect to the database. To see how to connect and use the database, refer to the steps in [Database Connectivity](./provisioning/database_connection.md). ## Known Issues -Please refer to the list of [Known Issues](./provisioning/known_issues.md) for an OCI DBCS System deployed using Oracle DB Operator DBCS Controller. +If you encounter any issues with deployment, refer to the list of [Known Issues](./provisioning/known_issues.md) for an OCI DBCS System deployed using Oracle DB Operator DBCS Controller. From cab94bec10b42b925b836592c1e9eac1c549d180 Mon Sep 17 00:00:00 2001 From: douglas_williams Date: Fri, 27 May 2022 16:55:34 +0000 Subject: [PATCH 406/628] Documentation style edit --- CONTRIBUTING.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 722965b9..87d9b60b 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,16 +1,16 @@ # Contributing to This Repository -We welcome your contributions! There are multiple ways to contribute. +Oracle welcomes your contributions! There are multiple ways that you can contribute. ## Opening issues -For bugs or enhancement requests, please file a GitHub issue unless the problem is security-related. When filing a bug, remember that the more specific the bug is, the more likely it is to be fixed. If you think you've found a security -vulnerability, then do not raise a GitHub issue. Instead, follow the instructions in our -[security policy](./SECURITY.md). +For bugs or enhancement requests, please file a GitHub issue, unless the problem is security-related. + +When filing a bug, remember that the more specific the bug is, the more likely it is to be fixed. If you think you've found a security vulnerability, then do not raise a GitHub issue. Instead, follow the instructions in our [security policy](./SECURITY.md). ## Contributing code -We welcome your code contributions. Before submitting code by using a pull request, +Oracle welcomes your code contributions. Before submitting code by using a pull request, you must sign the [Oracle Contributor Agreement][OCA] (OCA), and your commits must include the following line, using the name and e-mail address you used to sign the OCA: ```text @@ -29,22 +29,22 @@ can be accepted. ## Pull request process -1. Ensure there is an issue created to track and discuss the fix or enhancement that you intend to submit. +1. Ensure that there is an issue created to track and discuss the fix or enhancement that you intend to submit. 1. Fork this repository. 1. Create a branch in your fork to implement the changes. Oracle recommends using the issue number as part of your branch name. For example: `1234-fixes` 1. Ensure that any documentation is updated with the changes that are required by your change. -1. Ensure that any samples are updated, if the base image has been changed. +1. If the base image has been changed, then ensure that any examples are updated. 1. Submit the pull request. *Do not leave the pull request blank*. Explain exactly what your changes are meant to do, and provide simple steps to indicate how to validate your changes. Ensure that you reference the issue that you created as well. -1. Before the changes are merged, Oracle will assign the pull request to 2 or 3 people for review. +1. Before the changes are merged, Oracle will assign the pull request to two or three people for review. ## Code of conduct Follow the [Golden Rule](https://en.wikipedia.org/wiki/Golden_Rule). If you'd -like more specific guidelines, see the [Contributor Covenant Code of Conduct][COC]. +like more specific guidelines, then see the [Contributor Covenant Code of Conduct][COC]. [OCA]: https://oca.opensource.oracle.com [COC]: https://www.contributor-covenant.org/version/1/4/code-of-conduct/ From 2f1303327858bde2f98f8c8912e9d943cfb4109a Mon Sep 17 00:00:00 2001 From: douglas_williams Date: Fri, 27 May 2022 16:58:47 +0000 Subject: [PATCH 407/628] Documentation style edit --- PREREQUISITES.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/PREREQUISITES.md b/PREREQUISITES.md index 7fc3cf21..e386c6bf 100644 --- a/PREREQUISITES.md +++ b/PREREQUISITES.md @@ -2,13 +2,14 @@ ## Prerequisites for Using Oracle Database Operator for Kubernetes -Oracle Database operator for Kubernetes (OraOperator) manages all Cloud deployments of Oracle Database, including: +Oracle Database Operator for Kubernetes (OraOperator) manages all Cloud deployments of Oracle Database, including: * Oracle Autonomous Database (ADB) * Containerized Oracle Database Single Instance (SIDB) * Containerized Sharded Oracle Database (SHARDING) ### Setting Up a Kubernetes Cluster and Volumes +Review and complete each step as needed. #### Setting Up an OKE Cluster on Oracle Cloud Infrastructure (OCI) From 23fb90fc6101693d007c392acf706dffa626826b Mon Sep 17 00:00:00 2001 From: Mohammed Yunus Date: Mon, 30 May 2022 06:02:15 +0000 Subject: [PATCH 408/628] Mqureshi pvc vol name --- .../v1alpha1/oraclerestdataservice_types.go | 1 + .../v1alpha1/oraclerestdataservice_webhook.go | 21 ++++++++++++++++ .../v1alpha1/singleinstancedatabase_types.go | 2 ++ .../singleinstancedatabase_webhook.go | 24 ++++++++++++------- ...ase.oracle.com_oraclerestdataservices.yaml | 2 ++ ...se.oracle.com_singleinstancedatabases.yaml | 10 ++++++++ .../samples/sidb/oraclerestdataservice.yaml | 21 ++++++++-------- .../sidb/oraclerestdataservice_apex.yaml | 2 +- .../sidb/oraclerestdataservice_create.yaml | 2 +- .../samples/sidb/singleinstancedatabase.yaml | 15 +++++++----- .../sidb/singleinstancedatabase_clone.yaml | 7 +++--- .../sidb/singleinstancedatabase_create.yaml | 7 +++--- .../sidb/singleinstancedatabase_express.yaml | 6 +++-- .../sidb/singleinstancedatabase_patch.yaml | 7 +++--- .../oraclerestdataservice_controller.go | 1 + .../singleinstancedatabase_controller.go | 2 ++ 16 files changed, 92 insertions(+), 38 deletions(-) diff --git a/apis/database/v1alpha1/oraclerestdataservice_types.go b/apis/database/v1alpha1/oraclerestdataservice_types.go index 7a88d8bf..59d38621 100644 --- a/apis/database/v1alpha1/oraclerestdataservice_types.go +++ b/apis/database/v1alpha1/oraclerestdataservice_types.go @@ -75,6 +75,7 @@ type OracleRestDataServicePersistence struct { // +kubebuilder:validation:Enum=ReadWriteOnce;ReadWriteMany AccessMode string `json:"accessMode,omitempty"` + VolName string `json:"volName,omitempty"` } // OracleRestDataServiceImage defines the Image source and pullSecrets for POD diff --git a/apis/database/v1alpha1/oraclerestdataservice_webhook.go b/apis/database/v1alpha1/oraclerestdataservice_webhook.go index 37259007..858fb920 100644 --- a/apis/database/v1alpha1/oraclerestdataservice_webhook.go +++ b/apis/database/v1alpha1/oraclerestdataservice_webhook.go @@ -92,6 +92,27 @@ func (r *OracleRestDataService) ValidateCreate() error { var allErrs field.ErrorList + // Persistence spec validation + if r.Spec.Persistence.Size == "" && (r.Spec.Persistence.AccessMode != "" || + r.Spec.Persistence.StorageClass != "" || r.Spec.Persistence.VolName != "") { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("persistence").Child("size"), r.Spec.Persistence, + "invalid persistence specification, specify required size")) + } + + if r.Spec.Persistence.Size != "" { + if r.Spec.Persistence.AccessMode == "" { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("persistence").Child("size"), r.Spec.Persistence, + "invalid persistence specification, specify accessMode")) + } + if r.Spec.Persistence.AccessMode != "ReadWriteMany" && r.Spec.Persistence.AccessMode != "ReadWriteOnce" { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("persistence").Child("accessMode"), + r.Spec.Persistence.AccessMode, "should be either \"ReadWriteOnce\" or \"ReadWriteMany\"")) + } + } + if len(allErrs) == 0 { return nil } diff --git a/apis/database/v1alpha1/singleinstancedatabase_types.go b/apis/database/v1alpha1/singleinstancedatabase_types.go index 00289d80..7fd55697 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_types.go +++ b/apis/database/v1alpha1/singleinstancedatabase_types.go @@ -83,7 +83,9 @@ type SingleInstanceDatabaseSpec struct { type SingleInstanceDatabasePersistence struct { Size string `json:"size,omitempty"` StorageClass string `json:"storageClass,omitempty"` + // +kubebuilder:validation:Enum=ReadWriteOnce;ReadWriteMany AccessMode string `json:"accessMode,omitempty"` + VolName string `json:"volName,omitempty"` VolClaimAnnotation string `json:"volClaimAnnotation,omitempty"` } diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index 4b2319f1..f94ede5b 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -117,18 +117,24 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { var allErrs field.ErrorList // Persistence spec validation - if r.Spec.Persistence.Size == "" && (r.Spec.Persistence.AccessMode != "" || r.Spec.Persistence.StorageClass != "") || - (r.Spec.Persistence.Size != "" && r.Spec.Persistence.AccessMode == "") { + if r.Spec.Persistence.Size == "" && (r.Spec.Persistence.AccessMode != "" || + r.Spec.Persistence.StorageClass != "" || r.Spec.Persistence.VolName != "") { allErrs = append(allErrs, - field.Invalid(field.NewPath("spec").Child("persistence"), r.Spec.Persistence, - "invalid specification, size and/or accessMode missing")) + field.Invalid(field.NewPath("spec").Child("persistence").Child("size"), r.Spec.Persistence, + "invalid persistence specification, specify required size")) } - if r.Spec.Persistence.Size != "" && - r.Spec.Persistence.AccessMode != "ReadWriteMany" && r.Spec.Persistence.AccessMode != "ReadWriteOnce" { - allErrs = append(allErrs, - field.Invalid(field.NewPath("spec").Child("persistence").Child("accessMode"), - r.Spec.Persistence.AccessMode, "should be either \"ReadWriteOnce\" or \"ReadWriteMany\"")) + if r.Spec.Persistence.Size != "" { + if r.Spec.Persistence.AccessMode == "" { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("persistence").Child("size"), r.Spec.Persistence, + "invalid persistence specification, specify accessMode")) + } + if r.Spec.Persistence.AccessMode != "ReadWriteMany" && r.Spec.Persistence.AccessMode != "ReadWriteOnce" { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("persistence").Child("accessMode"), + r.Spec.Persistence.AccessMode, "should be either \"ReadWriteOnce\" or \"ReadWriteMany\"")) + } } // Replica validation diff --git a/config/crd/bases/database.oracle.com_oraclerestdataservices.yaml b/config/crd/bases/database.oracle.com_oraclerestdataservices.yaml index 8c426b34..d4db3340 100644 --- a/config/crd/bases/database.oracle.com_oraclerestdataservices.yaml +++ b/config/crd/bases/database.oracle.com_oraclerestdataservices.yaml @@ -133,6 +133,8 @@ spec: type: string storageClass: type: string + volName: + type: string type: object replicas: minimum: 1 diff --git a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml index 3d230bd9..df8adf70 100644 --- a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml +++ b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml @@ -131,6 +131,9 @@ spec: size and class for PVC properties: accessMode: + enum: + - ReadWriteOnce + - ReadWriteMany type: string size: type: string @@ -138,6 +141,8 @@ spec: type: string volClaimAnnotation: type: string + volName: + type: string type: object readinessCheckPeriod: type: integer @@ -287,6 +292,9 @@ spec: size and class for PVC properties: accessMode: + enum: + - ReadWriteOnce + - ReadWriteMany type: string size: type: string @@ -294,6 +302,8 @@ spec: type: string volClaimAnnotation: type: string + volName: + type: string type: object prebuiltDB: type: boolean diff --git a/config/samples/sidb/oraclerestdataservice.yaml b/config/samples/sidb/oraclerestdataservice.yaml index a7e02513..220a1942 100644 --- a/config/samples/sidb/oraclerestdataservice.yaml +++ b/config/samples/sidb/oraclerestdataservice.yaml @@ -48,33 +48,34 @@ spec: pullSecrets: ## Dedicated persistent storage is optional. If not specified, ORDS will use the databaseRef persistent storage - ## size : Minimum size of pvc | class : PVC storage Class - ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany - ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. + ## size is the required minimum size of the persistent volume + ## storageClass is used for automatic volume provisioning + ## accessMode can only accept one of ReadWriteOnce, ReadWriteMany + ## volName is optional. Specify for binding to a specific PV and set storageClass to an empty string to disable automatic volume provisioning # persistence: # size: 50Gi + ## oci-bv applies to OCI block volumes. Update as appropriate for other cloud service providers # storageClass: "oci-bv" # accessMode: "ReadWriteOnce" + # volName: "" ## Type of service Applicable on cloud enviroments only. - ## if loadBalService: false, service type = "NodePort". else "LoadBalancer" + ## if loadBalService: false, service type = "NodePort" else "LoadBalancer" loadBalancer: false ## Deploy only on nodes having required labels. Format label_name : label_value ## The same lables are applied to the created PVC - ## For instance if the Pods and storage need to be restricted to a particular AD + ## For instance if the pods need to be restricted to a particular AD ## Leave commented if there is no such requirement. # nodeSelector: - # failure-domain.beta.kubernetes.io/zone: PHX-AD-1 + # topology.kubernetes.io/zone: PHX-AD-1 - ## Schemas to be ORDS Enabled. - ## Schema will be created (if not exists) with password as .spec.ordsPassword. - ## pdbName defaults to the singleinstancedatabase .spec.pdbName + ## PDB (.spec.pdbName) schemas to be ORDS Enabled. + ## schema will be created (if not exists) with password as .spec.ordsPassword. restEnableSchemas: - schemaName: enable: true urlMapping: - pdbName: ## If deploying on OpenShift, change service account name to 'sidb-sa' after you run `$ oc apply -f openshift_rbac.yaml` serviceAccountName: default diff --git a/config/samples/sidb/oraclerestdataservice_apex.yaml b/config/samples/sidb/oraclerestdataservice_apex.yaml index afd40f7a..f56437e6 100644 --- a/config/samples/sidb/oraclerestdataservice_apex.yaml +++ b/config/samples/sidb/oraclerestdataservice_apex.yaml @@ -68,4 +68,4 @@ spec: urlMapping: - schemaName: schema2 enable: true - urlMapping: + urlMapping: myschema diff --git a/config/samples/sidb/oraclerestdataservice_create.yaml b/config/samples/sidb/oraclerestdataservice_create.yaml index 9d72fcc7..dfba1626 100644 --- a/config/samples/sidb/oraclerestdataservice_create.yaml +++ b/config/samples/sidb/oraclerestdataservice_create.yaml @@ -46,4 +46,4 @@ spec: urlMapping: - schemaName: schema2 enable: true - urlMapping: + urlMapping: myschema diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index 538a9e4c..e9844858 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -65,23 +65,26 @@ spec: pullSecrets: prebuiltDB: false - ## size : Minimum size of pvc | class : PVC storage Class - ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany - ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. + ## size is the required minimum size of the persistent volume + ## storageClass is specified for automatic volume provisioning + ## accessMode can only accept one of ReadWriteOnce, ReadWriteMany + ## volName is optional. Specify for binding to a specific PV and set storageClass to an empty string to disable automatic volume provisioning persistence: size: 100Gi + ## oci-bv applies to OCI block volumes. Update as appropriate for other cloud service providers storageClass: "oci-bv" accessMode: "ReadWriteOnce" + volName: "" ## Type of service . Applicable on cloud enviroments only - ## if loadBalService : false, service type = "NodePort". else "LoadBalancer" + ## if loadBalService : false, service type = "NodePort" else "LoadBalancer" loadBalancer: false ## Deploy only on nodes having required labels. Format label_name : label_value - ## For instance if the Pods need to be restricted to a particular AD + ## For instance if the pods need to be restricted to a particular AD ## Leave commented if there is no such requirement. # nodeSelector: - # failure-domain.beta.kubernetes.io/zone: PHX-AD-1 + # topology.kubernetes.io/zone: PHX-AD-1 ## If deploying on OpenShift, change service account name to 'sidb-sa' after you run `$ oc apply -f openshift_rbac.yaml` serviceAccountName: default diff --git a/config/samples/sidb/singleinstancedatabase_clone.yaml b/config/samples/sidb/singleinstancedatabase_clone.yaml index 1fab477d..67ae08fc 100644 --- a/config/samples/sidb/singleinstancedatabase_clone.yaml +++ b/config/samples/sidb/singleinstancedatabase_clone.yaml @@ -30,11 +30,12 @@ spec: pullFrom: container-registry.oracle.com/database/enterprise:latest pullSecrets: oracle-container-registry-secret - ## size : Minimum size of pvc | class : PVC storage Class - ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany - ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. + ## size is the required minimum size of the persistent volume + ## storageClass is specified for automatic volume provisioning + ## accessMode can only accept one of ReadWriteOnce, ReadWriteMany persistence: size: 100Gi + ## oci-bv applies to OCI block volumes. Update as appropriate for other cloud service providers storageClass: "oci-bv" accessMode: "ReadWriteOnce" diff --git a/config/samples/sidb/singleinstancedatabase_create.yaml b/config/samples/sidb/singleinstancedatabase_create.yaml index 22dee813..342637f1 100644 --- a/config/samples/sidb/singleinstancedatabase_create.yaml +++ b/config/samples/sidb/singleinstancedatabase_create.yaml @@ -48,11 +48,12 @@ spec: pullFrom: container-registry.oracle.com/database/enterprise:latest pullSecrets: oracle-container-registry-secret - ## size : Minimum size of pvc | class : PVC storage Class - ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany - ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. + ## size is the required minimum size of the persistent volume + ## storageClass is specified for automatic volume provisioning + ## accessMode can only accept one of ReadWriteOnce, ReadWriteMany persistence: size: 100Gi + ## oci-bv applies to OCI block volumes. Update as appropriate for other cloud service providers storageClass: "oci-bv" accessMode: "ReadWriteOnce" diff --git a/config/samples/sidb/singleinstancedatabase_express.yaml b/config/samples/sidb/singleinstancedatabase_express.yaml index 4bc5bf0b..d1100c9b 100644 --- a/config/samples/sidb/singleinstancedatabase_express.yaml +++ b/config/samples/sidb/singleinstancedatabase_express.yaml @@ -37,10 +37,12 @@ spec: pullFrom: container-registry.oracle.com/database/express:latest prebuiltDB: true - ## size : Minimum size of pvc | class : PVC storage Class - ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany + ## size is the required minimum size of the persistent volume + ## storageClass is specified for automatic volume provisioning + ## accessMode can only accept one of ReadWriteOnce, ReadWriteMany persistence: size: 50Gi + ## oci-bv applies to OCI block volumes. Update as appropriate for other cloud service providers storageClass: "oci-bv" accessMode: "ReadWriteOnce" diff --git a/config/samples/sidb/singleinstancedatabase_patch.yaml b/config/samples/sidb/singleinstancedatabase_patch.yaml index 134642a0..aa04e317 100644 --- a/config/samples/sidb/singleinstancedatabase_patch.yaml +++ b/config/samples/sidb/singleinstancedatabase_patch.yaml @@ -38,11 +38,12 @@ spec: pullFrom: pullSecrets: - ## size : Minimum size of pvc | class : PVC storage Class - ## AccessMode can only accept one of ReadWriteOnce, ReadWriteMany - ## Below mentioned storageClass/accessMode applies to OCI block volumes. Update appropriately for other types of persistent volumes. + ## size is the required minimum size of the persistent volume + ## storageClass is specified for automatic volume provisioning + ## accessMode can only accept one of ReadWriteOnce, ReadWriteMany persistence: size: 100Gi + ## oci-bv applies to OCI block volumes. Update as appropriate for other cloud service providers storageClass: "oci-bv" accessMode: "ReadWriteOnce" diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index faa1b44f..cd59381d 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -768,6 +768,7 @@ func (r *OracleRestDataServiceReconciler) instantiatePVCSpec(m *dbapi.OracleRest }, }, StorageClassName: &m.Spec.Persistence.StorageClass, + VolumeName: m.Spec.Persistence.VolName, Selector: func() *metav1.LabelSelector { if m.Spec.Persistence.StorageClass != "oci" { return nil diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 56347257..7be70fc2 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -933,6 +933,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePVCSpec(m *dbapi.SingleIns }, }, StorageClassName: &m.Spec.Persistence.StorageClass, + VolumeName: m.Spec.Persistence.VolName, Selector: func() *metav1.LabelSelector { if m.Spec.Persistence.StorageClass != "oci" { return nil @@ -976,6 +977,7 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplacePVC(ctx context.Contex if err == nil { if *pvc.Spec.StorageClassName != m.Spec.Persistence.StorageClass || pvc.Spec.Resources.Requests["storage"] != resource.MustParse(m.Spec.Persistence.Size) || + (m.Spec.Persistence.VolName != "" && pvc.Spec.VolumeName != m.Spec.Persistence.VolName) || pvc.Spec.AccessModes[0] != corev1.PersistentVolumeAccessMode(m.Spec.Persistence.AccessMode) { // call deletePods() with zero pods in avaiable and nil readyPod to delete all pods result, err := r.deletePods(ctx, req, m, []corev1.Pod{}, corev1.Pod{}, 0, 0) From 879cedca332025794c0a0c187899895a6600f3ab Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Mon, 30 May 2022 13:12:37 +0530 Subject: [PATCH 409/628] Update comments in sample yamls --- config/samples/sidb/oraclerestdataservice.yaml | 16 ++++++++-------- config/samples/sidb/singleinstancedatabase.yaml | 6 +++--- 2 files changed, 11 insertions(+), 11 deletions(-) diff --git a/config/samples/sidb/oraclerestdataservice.yaml b/config/samples/sidb/oraclerestdataservice.yaml index 220a1942..111d4b87 100644 --- a/config/samples/sidb/oraclerestdataservice.yaml +++ b/config/samples/sidb/oraclerestdataservice.yaml @@ -13,20 +13,20 @@ spec: databaseRef: "singleinstancedatabase-sample" ## Secret containing databaseRef password mapped to secretKey. - ## This secret will be deleted after ORDS Installation unless keepSecret set to true. + ## This secret will be deleted after ORDS Installation unless keepSecret set to true adminPassword: secretName: secretKey: keepSecret: true ## Secret containing ORDS_PUBLIC_USER password mapped to secretKey. secretKey defaults to oracle_pwd - ## This secret will be deleted after ORDS Installation unless keepSecret set to true. + ## This secret will be deleted after ORDS Installation unless keepSecret set to true ordsPassword: secretName: secretKey: keepSecret: true - ## To configure APEX with ORDS, specfiy the apexPassword secret details. Leave empty if Apex is not needed. + ## To configure APEX with ORDS, specfiy the apexPassword secret details. Leave empty if Apex is not needed ## This is a secret containing a common password for APEX_PUBLIC_USER, APEX_REST_PUBLIC_USER, APEX_LISTENER and Apex administrator (username: ADMIN) mapped to secretKey ## This secret will be deleted after ORDS Installation unless keepSecret set to true. ## This password should complete the following requirements: @@ -47,7 +47,7 @@ spec: pullFrom: pullSecrets: - ## Dedicated persistent storage is optional. If not specified, ORDS will use the databaseRef persistent storage + ## Dedicated persistent storage is optional. If not specified, ORDS will use persistent storage from .spec.databaseRef ## size is the required minimum size of the persistent volume ## storageClass is used for automatic volume provisioning ## accessMode can only accept one of ReadWriteOnce, ReadWriteMany @@ -63,15 +63,15 @@ spec: ## if loadBalService: false, service type = "NodePort" else "LoadBalancer" loadBalancer: false - ## Deploy only on nodes having required labels. Format label_name : label_value + ## Deploy only on nodes having required labels. Format label_name: label_value ## The same lables are applied to the created PVC ## For instance if the pods need to be restricted to a particular AD - ## Leave commented if there is no such requirement. + ## Leave commented if there is no such requirement # nodeSelector: # topology.kubernetes.io/zone: PHX-AD-1 - ## PDB (.spec.pdbName) schemas to be ORDS Enabled. - ## schema will be created (if not exists) with password as .spec.ordsPassword. + ## Schemas to be ORDS Enabled in PDB of .spec.databaseRef (.spec.pdbName) + ## Schema will be created (if not exists) with password as .spec.ordsPassword restEnableSchemas: - schemaName: enable: true diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index e9844858..89bbe23f 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -12,7 +12,7 @@ spec: ## Use only alphanumeric characters for sid up to a maximum of 8 characters sid: ORCL1 - ## Specify a source database ref to copy/clone from any SIDB in current K8s cluster instead of creating a fresh one. + ## Specify a source database ref to copy/clone from any SIDB in current K8s cluster instead of creating a fresh one ## If cloning from an external containerized DB which could be either standalone or in any K8s cluster, ## specify connect string as `:/` instead of source database ref cloneFrom: "" @@ -59,7 +59,7 @@ spec: ## Build patched DB images from https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/patching ## Prebuilt DB support (https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/prebuiltdb) ## Specify prebuiltDB as true if the image includes a prebuilt DB - ## If cloning specify an image that is of same major version as the source DB at same or different patch levels. + ## If cloning specify an image that is of same major version as the source DB at same or different patch levels image: pullFrom: pullSecrets: @@ -80,7 +80,7 @@ spec: ## if loadBalService : false, service type = "NodePort" else "LoadBalancer" loadBalancer: false - ## Deploy only on nodes having required labels. Format label_name : label_value + ## Deploy only on nodes having required labels. Format label_name: label_value ## For instance if the pods need to be restricted to a particular AD ## Leave commented if there is no such requirement. # nodeSelector: From 64d5538bc733fe91538551bcd9ce334ff0ae05db Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Mon, 30 May 2022 13:36:32 +0530 Subject: [PATCH 410/628] Name change in PVC --- apis/database/v1alpha1/oraclerestdataservice_types.go | 4 ++-- .../database/v1alpha1/oraclerestdataservice_webhook.go | 2 +- apis/database/v1alpha1/singleinstancedatabase_types.go | 4 ++-- .../v1alpha1/singleinstancedatabase_webhook.go | 10 +++++----- .../database.oracle.com_oraclerestdataservices.yaml | 2 +- .../database.oracle.com_singleinstancedatabases.yaml | 8 ++++---- config/samples/sidb/oraclerestdataservice.yaml | 4 ++-- config/samples/sidb/singleinstancedatabase.yaml | 4 ++-- .../database/oraclerestdataservice_controller.go | 2 +- .../database/singleinstancedatabase_controller.go | 8 ++++---- 10 files changed, 24 insertions(+), 24 deletions(-) diff --git a/apis/database/v1alpha1/oraclerestdataservice_types.go b/apis/database/v1alpha1/oraclerestdataservice_types.go index 59d38621..a705604a 100644 --- a/apis/database/v1alpha1/oraclerestdataservice_types.go +++ b/apis/database/v1alpha1/oraclerestdataservice_types.go @@ -74,8 +74,8 @@ type OracleRestDataServicePersistence struct { StorageClass string `json:"storageClass,omitempty"` // +kubebuilder:validation:Enum=ReadWriteOnce;ReadWriteMany - AccessMode string `json:"accessMode,omitempty"` - VolName string `json:"volName,omitempty"` + AccessMode string `json:"accessMode,omitempty"` + VolumeName string `json:"volumeName,omitempty"` } // OracleRestDataServiceImage defines the Image source and pullSecrets for POD diff --git a/apis/database/v1alpha1/oraclerestdataservice_webhook.go b/apis/database/v1alpha1/oraclerestdataservice_webhook.go index 858fb920..0676c7b3 100644 --- a/apis/database/v1alpha1/oraclerestdataservice_webhook.go +++ b/apis/database/v1alpha1/oraclerestdataservice_webhook.go @@ -94,7 +94,7 @@ func (r *OracleRestDataService) ValidateCreate() error { // Persistence spec validation if r.Spec.Persistence.Size == "" && (r.Spec.Persistence.AccessMode != "" || - r.Spec.Persistence.StorageClass != "" || r.Spec.Persistence.VolName != "") { + r.Spec.Persistence.StorageClass != "" || r.Spec.Persistence.VolumeName != "") { allErrs = append(allErrs, field.Invalid(field.NewPath("spec").Child("persistence").Child("size"), r.Spec.Persistence, "invalid persistence specification, specify required size")) diff --git a/apis/database/v1alpha1/singleinstancedatabase_types.go b/apis/database/v1alpha1/singleinstancedatabase_types.go index 7fd55697..a0ac9f33 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_types.go +++ b/apis/database/v1alpha1/singleinstancedatabase_types.go @@ -85,8 +85,8 @@ type SingleInstanceDatabasePersistence struct { StorageClass string `json:"storageClass,omitempty"` // +kubebuilder:validation:Enum=ReadWriteOnce;ReadWriteMany AccessMode string `json:"accessMode,omitempty"` - VolName string `json:"volName,omitempty"` - VolClaimAnnotation string `json:"volClaimAnnotation,omitempty"` + VolumeName string `json:"volumeName,omitempty"` + VolumeClaimAnnotation string `json:"volumeClaimAnnotation,omitempty"` } // SingleInstanceDatabaseInitParams defines the Init Parameters diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index f94ede5b..2b527083 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -118,7 +118,7 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { // Persistence spec validation if r.Spec.Persistence.Size == "" && (r.Spec.Persistence.AccessMode != "" || - r.Spec.Persistence.StorageClass != "" || r.Spec.Persistence.VolName != "") { + r.Spec.Persistence.StorageClass != "" || r.Spec.Persistence.VolumeName != "") { allErrs = append(allErrs, field.Invalid(field.NewPath("spec").Child("persistence").Child("size"), r.Spec.Persistence, "invalid persistence specification, specify required size")) @@ -225,12 +225,12 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { } } - if r.Spec.Persistence.VolClaimAnnotation != "" { - strParts := strings.Split(r.Spec.Persistence.VolClaimAnnotation, ":") + if r.Spec.Persistence.VolumeClaimAnnotation != "" { + strParts := strings.Split(r.Spec.Persistence.VolumeClaimAnnotation, ":") if len(strParts) != 2 { allErrs = append(allErrs, - field.Invalid(field.NewPath("spec").Child("persistence").Child("volClaimAnnotation"), r.Spec.Persistence.VolClaimAnnotation, - "volClaimAnnotation should be in : format.")) + field.Invalid(field.NewPath("spec").Child("persistence").Child("volumeClaimAnnotation"), r.Spec.Persistence.VolumeClaimAnnotation, + "volumeClaimAnnotation should be in : format.")) } } diff --git a/config/crd/bases/database.oracle.com_oraclerestdataservices.yaml b/config/crd/bases/database.oracle.com_oraclerestdataservices.yaml index d4db3340..e48e7db4 100644 --- a/config/crd/bases/database.oracle.com_oraclerestdataservices.yaml +++ b/config/crd/bases/database.oracle.com_oraclerestdataservices.yaml @@ -133,7 +133,7 @@ spec: type: string storageClass: type: string - volName: + volumeName: type: string type: object replicas: diff --git a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml index df8adf70..e00a2db1 100644 --- a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml +++ b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml @@ -139,9 +139,9 @@ spec: type: string storageClass: type: string - volClaimAnnotation: + volumeClaimAnnotation: type: string - volName: + volumeName: type: string type: object readinessCheckPeriod: @@ -300,9 +300,9 @@ spec: type: string storageClass: type: string - volClaimAnnotation: + volumeClaimAnnotation: type: string - volName: + volumeName: type: string type: object prebuiltDB: diff --git a/config/samples/sidb/oraclerestdataservice.yaml b/config/samples/sidb/oraclerestdataservice.yaml index 111d4b87..c1fa782d 100644 --- a/config/samples/sidb/oraclerestdataservice.yaml +++ b/config/samples/sidb/oraclerestdataservice.yaml @@ -51,13 +51,13 @@ spec: ## size is the required minimum size of the persistent volume ## storageClass is used for automatic volume provisioning ## accessMode can only accept one of ReadWriteOnce, ReadWriteMany - ## volName is optional. Specify for binding to a specific PV and set storageClass to an empty string to disable automatic volume provisioning + ## volumeName is optional. Specify for binding to a specific PV and set storageClass to an empty string to disable automatic volume provisioning # persistence: # size: 50Gi ## oci-bv applies to OCI block volumes. Update as appropriate for other cloud service providers # storageClass: "oci-bv" # accessMode: "ReadWriteOnce" - # volName: "" + # volumeName: "" ## Type of service Applicable on cloud enviroments only. ## if loadBalService: false, service type = "NodePort" else "LoadBalancer" diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index 89bbe23f..44f26cf4 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -68,13 +68,13 @@ spec: ## size is the required minimum size of the persistent volume ## storageClass is specified for automatic volume provisioning ## accessMode can only accept one of ReadWriteOnce, ReadWriteMany - ## volName is optional. Specify for binding to a specific PV and set storageClass to an empty string to disable automatic volume provisioning + ## volumeName is optional. Specify for binding to a specific PV and set storageClass to an empty string to disable automatic volume provisioning persistence: size: 100Gi ## oci-bv applies to OCI block volumes. Update as appropriate for other cloud service providers storageClass: "oci-bv" accessMode: "ReadWriteOnce" - volName: "" + volumeName: "" ## Type of service . Applicable on cloud enviroments only ## if loadBalService : false, service type = "NodePort" else "LoadBalancer" diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index cd59381d..106be615 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -768,7 +768,7 @@ func (r *OracleRestDataServiceReconciler) instantiatePVCSpec(m *dbapi.OracleRest }, }, StorageClassName: &m.Spec.Persistence.StorageClass, - VolumeName: m.Spec.Persistence.VolName, + VolumeName: m.Spec.Persistence.VolumeName, Selector: func() *metav1.LabelSelector { if m.Spec.Persistence.StorageClass != "oci" { return nil diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 7be70fc2..8945a154 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -911,8 +911,8 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePVCSpec(m *dbapi.SingleIns "app": m.Name, }, Annotations: func() map[string]string { - if m.Spec.Persistence.VolClaimAnnotation != "" { - strParts := strings.Split(m.Spec.Persistence.VolClaimAnnotation, ":") + if m.Spec.Persistence.VolumeClaimAnnotation != "" { + strParts := strings.Split(m.Spec.Persistence.VolumeClaimAnnotation, ":") annotationMap := make(map[string]string) annotationMap[strParts[0]] = strParts[1] return annotationMap @@ -933,7 +933,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePVCSpec(m *dbapi.SingleIns }, }, StorageClassName: &m.Spec.Persistence.StorageClass, - VolumeName: m.Spec.Persistence.VolName, + VolumeName: m.Spec.Persistence.VolumeName, Selector: func() *metav1.LabelSelector { if m.Spec.Persistence.StorageClass != "oci" { return nil @@ -977,7 +977,7 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplacePVC(ctx context.Contex if err == nil { if *pvc.Spec.StorageClassName != m.Spec.Persistence.StorageClass || pvc.Spec.Resources.Requests["storage"] != resource.MustParse(m.Spec.Persistence.Size) || - (m.Spec.Persistence.VolName != "" && pvc.Spec.VolumeName != m.Spec.Persistence.VolName) || + (m.Spec.Persistence.VolumeName != "" && pvc.Spec.VolumeName != m.Spec.Persistence.VolumeName) || pvc.Spec.AccessModes[0] != corev1.PersistentVolumeAccessMode(m.Spec.Persistence.AccessMode) { // call deletePods() with zero pods in avaiable and nil readyPod to delete all pods result, err := r.deletePods(ctx, req, m, []corev1.Pod{}, corev1.Pod{}, 0, 0) From 8dcf6e867f346d4350fa043881d3b11e7c9742ac Mon Sep 17 00:00:00 2001 From: Abhishek Kumar Date: Tue, 31 May 2022 12:08:30 +0000 Subject: [PATCH 411/628] Added discussed readme changes --- docs/sidb/README.md | 122 ++++++++++++++++++++++++++++++-------------- 1 file changed, 83 insertions(+), 39 deletions(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index c44edb6c..f746679e 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -9,7 +9,7 @@ Oracle Database Operator for Kubernetes (`OraOperator`) includes the Single Inst * [Patch/Rollback Database](#patchrollback-database) * [Kind OracleRestDataService](#kind-oraclerestdataservice) * [REST Enable Database](#rest-enable-database) - * [Performing maintenance operations](#performing-maintenance-operations) + * [Performing Maintenance Operations](#performing-maintenance-operations) ## Prerequisites @@ -19,7 +19,7 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI The Oracle Database Operator creates the `SingleInstanceDatabase` kind as a custom resource. Doing this enables Oracle Database to be managed as a native Kubernetes object. -### SingleInstanceDatabase template YAML +### SingleInstanceDatabase Template YAML The template `.yaml` file for Single Instance Database (Enterprise and Standard Editions), including all the configurable options, is available at: **[config/samples/sidb/singleinstancedatabase.yaml](./../../config/samples/sidb/singleinstancedatabase.yaml)** @@ -118,7 +118,7 @@ $ kubectl describe singleinstancedatabase sidb-sample-clone You can easily provision a new database instance on the Kubernetes cluster by using **[config/samples/sidb/singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml)**. -1. Log into [Oracle Container Registry](https://container-registry.oracle.com/ords/f?p=113:4:7154182141811:::4:P4_REPOSITORY,AI_REPOSITORY,AI_REPOSITORY_NAME,P4_REPOSITORY_NAME,P4_EULA_ID,P4_BUSINESS_AREA_ID:9,9,Oracle%20Database%20Enterprise%20Edition,Oracle%20Database%20Enterprise%20Edition,1,0&cs=3Y_90hkCQLfJzrvTLiEipIGgWGUytfrtAPuHFocuWd0NDSacbBPlamohfLuiJA-bAsVL6Z_yKEMsTbb52bm6IRA) and accept the license agreement for the Database image, ignore if you have accepted already. +1. Log into [Oracle Container Registry](https://container-registry.oracle.com/) and accept the license agreement for the Database image, ignore if you have accepted already. 2. If you have not already done so, create an image pull secret for the Oracle Container Registry: @@ -141,24 +141,61 @@ You can easily provision a new database instance on the Kubernetes cluster by us ``` **NOTE:** -- For ease of use, the storage class **oci-bv** is specified in the **[singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml)**. This storage class facilitates dynamic provisioning of the OCI block volumes on the Oracle OKE for persistent storage of the database. For other cloud providers, you can similarly use their dynamic provisioning storage class. +- For ease of use, the storage class **oci-bv** is specified in the **[singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml)**. This storage class facilitates dynamic provisioning of the OCI block volumes on the Oracle OKE for persistent storage of the database. The supported access mode for this class is `ReadWriteOnce`. For other cloud providers, you can similarly use their dynamic provisioning storage classes. +- It is beneficial to have the database replica pods more than or equal to the number of available nodes if `ReadWriteMany` access mode is used with the OCI NFS volume. By doing so, the pods get distributed on different nodes and the database image is downloaded on all those nodes. This helps in reducing time for the database fail-over if the active database pod dies. - Supports Oracle Database Enterprise Edition (19.3.0), and later releases. - To pull the database image faster from the container registry in order to bring up the SIDB instance quickly, you can use container-registry mirror of the corresponding cluster's region. For example, if the cluster exists in Mumbai region, you can use `container-registry-bom.oracle.com` mirror. For more information on container-registry mirrors, please follow the link [https://blogs.oracle.com/wim/post/oracle-container-registry-mirrors-in-oracle-cloud-infrastructure](https://blogs.oracle.com/wim/post/oracle-container-registry-mirrors-in-oracle-cloud-infrastructure). - To update the init parameters like `sgaTarget` and `pgaAggregateTarget`, please refer the `initParams` section of the [singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml) file. -### Provisioning a new XE database -To provision new Oracle Database Express Edition (XE) database, use the sample **[config/samples/sidb/singleinstancedatabase_express.yaml](../../config/samples/sidb/singleinstancedatabase_express.yaml)** file. For example: - - kubectl apply -f singleinstancedatabase_express.yaml - -This command pulls the XE image uploaded on the [Oracle Container Registry](https://container-registry.oracle.com/ords/f?p=113:4:7460390069267:::4:P4_REPOSITORY,AI_REPOSITORY,AI_REPOSITORY_NAME,P4_REPOSITORY_NAME,P4_EULA_ID,P4_BUSINESS_AREA_ID:803,803,Oracle%20Database%20Express%20Edition,Oracle%20Database%20Express%20Edition,1,0&cs=3-UN6D9nAfyqxcYnrks18OAmfFcri96NZojBQALxMdakix8wgYRBxhD8rpTFd2ak1FAtfOVFexbuOM2opsjxT9w). - -**NOTE:** -- Provisioning Oracle Database express edition is supported for release 21c (21.3.0) and later releases. -- For XE database, only single replica mode (i.e. `replicas: 1`) is supported. -- For XE database, you **cannot change** the init parameters i.e. `cpuCount, processes, sgaTarget or pgaAggregateTarget`. - -### Provision a pre-built database +### Database Persistence +The database persistence can be achieved in the following two ways: +- Dynamic Persistence Provisioning +- Static Persistence Provisioning + +In **Dynamic Persistence Provisioning**, a persistent volume is provisioned by mentioning a storage class. For example, **oci-bv** storage class is specified in the **[singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml)** file. This storage class facilitates dynamic provisioning of the OCI block volumes. The supported access mode for this class is `ReadWriteOnce`. For other cloud providers, you can similarly use their dynamic provisioning storage classes. +**Note:** Generally, the `Reclaim Policy` of such dynamically provisioned volumes is `Delete`, hence, these volumes get deleted when their corresponding database deployment is deleted. To retain volumes, please use static provisioning as explained in the section below. + +In **Static Persistence Provisioning**, you have to create a volume manually (either using Block Volume or NFS), and then use the name of this volume with the `<.spec.persistence.volumeName>` field (corresponds to the `volumeName` field of the persistence section in the **[singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml)**). The `Reclaim Policy` of such volume can be set to `Retain`. So, this volume does not get deleted with the deletion of its corresponding deployment. The access modes supported with block volume and NFS are `ReadWriteOnce` and `ReadWriteMany` respectively. + +Static Persistence Provisioning in OCI explained in the following subsections: + +#### Block Volume Static Provisioning +You have to manually create a block volume resource from the OCI console, and fetch its `OCID`. Further, you can use the following YAML file to create the persistent volume: +```yaml +apiVersion: v1 +kind: PersistentVolume +metadata: + name: block-vol +spec: + capacity: + storage: 1024Gi + accessModes: + - ReadWriteOnce + persistentVolumeReclaimPolicy: Retain + csi: + driver: blockvolume.csi.oraclecloud.com + volumeHandle: +``` +#### NFS Volume Static Provisioning +Similar to the block volume static provisioning, you have to manually create a file system resource from the OCI console, and fetch its `OCID, Mount Target and Export Path`. Mention these values in the following YAML file to create the persistent volume: + +```yaml +apiVersion: v1 +kind: PersistentVolume +metadata: + name: nfs-vol +spec: + capacity: + storage: 1024Gi + volumeMode: Filesystem + accessModes: + - ReadWriteMany + persistentVolumeReclaimPolicy: Retain + csi: + driver: fss.csi.oraclecloud.com + volumeHandle: ":/" +``` +### Provision a Pre-built Database To provision a new pre-built database instance, use the sample **[config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml](../../config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml)** file. For example: ```sh @@ -171,6 +208,18 @@ This pre-built image includes the data files of the database inside the image it To create the pre-built database image for the Enterprise/Standard edition using a pre-built image, please follow these instructions: [Pre-built Database (prebuiltdb) Extension](https://github.com/oracle/docker-images/blob/main/OracleDatabase/SingleInstance/extensions/prebuiltdb/README.md). +### Provisioning a new XE Database +To provision new Oracle Database Express Edition (XE) database, use the sample **[config/samples/sidb/singleinstancedatabase_express.yaml](../../config/samples/sidb/singleinstancedatabase_express.yaml)** file. For example: + + kubectl apply -f singleinstancedatabase_express.yaml + +This command pulls the XE image uploaded on the [Oracle Container Registry](https://container-registry.oracle.com/). + +**NOTE:** +- Provisioning Oracle Database express edition is supported for release 21c (21.3.0) and later releases. +- For XE database, only single replica mode (i.e. `replicas: 1`) is supported. +- For XE database, you **cannot change** the init parameters i.e. `cpuCount, processes, sgaTarget or pgaAggregateTarget`. + ### Creation Status Creating a new database instance takes a while. When the `status` column returns the response `Healthy`, the Database is open for connections. @@ -212,6 +261,7 @@ Version 21.3.0.0.0 SQL> ``` +**Note:** The `<.spec.adminPassword>` above refers to the database password for SYS, SYSTEM and PDBADMIN users, which in turn represented by `spec` section's `adminPassword` field of the **[/config/samples/sidb/singleinstancedatabase.yaml](../config/samples/sidb/../../../../config/samples/sidb/singleinstancedatabase.yaml)** file. The Oracle Database inside the container also has Oracle Enterprise Manager Express (OEM Express) configured. To access OEM Express, start the browser, and paste in a URL similar to the following example: @@ -259,7 +309,9 @@ The following database initialization parameters can be updated after the databa - sgaTarget - pgaAggregateTarget - cpuCount -- processes. Change their attribute values and apply using kubectl **apply** or **edit/patch** commands. +- processes. + +Change their attribute values and apply using kubectl **apply** or **edit/patch** commands. **NOTE:** The value for the initialization parameter `sgaTarget` that you provide should be within the range set by [sga_min_size, sga_max_size]. If the value you provide is not in that range, then `sga_target` is not updated to the value you specify for `sgaTarget`. @@ -270,7 +322,9 @@ In multiple replicas mode, more than one pod is created for the database. The da **Note:** - This functionality requires the [k8s extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/k8s) extended images. The database image from the container registry `container-registry.oracle.com` includes the K8s extension. -- Because Oracle Database Express Edition (XE) does not support [k8s extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/k8s), it does not support multiple replicas. +- Because Oracle Database Express Edition (XE) does not support [k8s extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/k8s), it does not support multiple replicas. +- If the `ReadWriteOnce` access mode is used, all the replicas will be scheduled on the same node where the persistent volume would be mounted. +- If the `ReadWriteMany` access mode is used, all the replicas will be distributed on different nodes. So, it is recommended to have replicas more than or equal to the number of the nodes as the database image is downloaded on all those nodes. This is beneficial in quick cold fail-over scenario (when the active pod dies) as the image would already be available on that node. ### Patch Attributes @@ -308,7 +362,7 @@ To create copies of your existing database quickly, you can use the cloning func To quickly clone the existing database sidb-sample created above, use the sample **[config/samples/sidb/singleinstancedatabase_clone.yaml](../../config/samples/sidb/singleinstancedatabase_clone.yaml)** file. -**Note**: To clone a database, The source database must have archiveLog mode set to true. +**Note**: To clone a database, the source database must have archiveLog mode set to true. For example: @@ -344,7 +398,7 @@ After patching is complete, the database pods are restarted with the new release To clone and patch the database at the same time, clone your source database by using the [cloning existing database](#clone-existing-database) method, and specify a new release image for the cloned database. Use this method to ensure there are no patching related issues impacting your database performance or functionality. -### Datapatch status +### Datapatch Status Patching/Rollback operations are complete when the datapatch tool completes patching or rollback of the data files. Check the data files patching status and current release update version using the following commands @@ -363,7 +417,7 @@ $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.releaseUp The Oracle Database Operator creates the `OracleRestDataService` (ORDS) kind as a custom resource. Creating ORDS as a custom resource enables the RESTful API access to the Oracle Database in K8s. -### OracleRestDataService template YAML +### OracleRestDataService Template YAML The template `.yaml` file for Oracle Rest Data Services (`OracleRestDataService` kind) is available at **[config/samples/sidb/oraclerestdataservice.yaml](config/samples/sidb/oraclerestdataservice.yaml)**. @@ -487,25 +541,25 @@ Use this ORDS user to authenticate the following: * Any Protected AutoRest Enabled Object APIs * Database Actions of any REST Enabled Schema -#### Database API examples +#### Database API Examples Some examples for the Database API usage are as follows: -* ##### Getting all Database components +* ##### Getting all Database Components ```sh curl -s -k -X GET -u 'ORDS_PUBLIC_USER:<.spec.ordsPassword>' https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/database/components/ | python -m json.tool ``` -* ##### Getting all Database users +* ##### Getting all Database Users ```sh curl -s -k -X GET -u 'ORDS_PUBLIC_USER:<.spec.ordsPassword>' https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/database/security/users/ | python -m json.tool ``` -* ##### Getting all tablespaces +* ##### Getting all Tablespaces ```sh curl -s -k -X GET -u 'ORDS_PUBLIC_USER:<.spec.ordsPassword>' https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/database/storage/tablespaces/ | python -m json.tool ``` -* ##### Getting all Database parameters +* ##### Getting all Database Parameters ```sh curl -s -k -X GET -u 'ORDS_PUBLIC_USER:<.spec.ordsPassword>' https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/database/parameters/ | python -m json.tool ``` -* ##### Getting all feature usage statistics +* ##### Getting all Feature Usage Statistics ```sh curl -s -k -X GET -u 'ORDS_PUBLIC_USER:<.spec.ordsPassword>' https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/database/feature_usage/ | python -m json.tool ``` @@ -640,7 +694,7 @@ password: `.spec.apexPassword` **NOTE:** - By default, the full development environment is initialized in APEX. After deployment, you can change it manually to the runtime environment. To change environments, run the script `apxdevrm.sql` after connecting to the primary database from the ORDS pod as the `SYS` user with `SYSDBA` privilege. For detailed instructions, see: [Converting a Full Development Environment to a Runtime Environment](https://docs.oracle.com/en/database/oracle/application-express/21.2/htmig/converting-between-runtime-and-full-development-environments.html#GUID-B0621B40-3441-44ED-9D86-29B058E26BE9). -## Performing maintenance operations +## Performing Maintenance Operations If you need to perform some maintenance operations manually, then the procedure is as follows: 1. Use `kubectl exec` to access the pod where you want to perform the manual operation, a command similar to the following: @@ -652,13 +706,3 @@ If you need to perform some maintenance operations manually, then the procedure sqlplus / as sysdba - - -## Additional use-cases -- If you use **oci-bv** storage class for dynamic provisioning of the persistent volume, this volume gets deleted with the deletion of its associated resource (Database/ORDS). This happens because the Reclaim Policy of the provisioned volume is Delete by default. If you want to retain this dynamically provisioned volume, the following command should be used: - - kubectl patch pv -p '{"spec":{"persistentVolumeReclaimPolicy":"Retain"}}' - - To make this retained PV available for the next Database/ORDS deployment, you can run the following command: - - kubectl patch pv -p '{"spec":{"claimRef":null}}' \ No newline at end of file From f363259e3bda295d2b391754e27609d84830f3c7 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Tue, 31 May 2022 18:03:19 +0530 Subject: [PATCH 412/628] Update ReadMe --- docs/sidb/README.md | 12 ++++++------ docs/sidb/SIDB_PREREQUISITES.md | 9 +++------ 2 files changed, 9 insertions(+), 12 deletions(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index f746679e..0828bbf2 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -206,7 +206,7 @@ $ kubectl apply -f singleinstancedatabase_prebuiltdb.yaml This pre-built image includes the data files of the database inside the image itself. As a result, the database startup time of the container is reduced, down to a couple of seconds. The pre-built database image can be very useful in contiguous integration/continuous delivery (CI/CD) scenarios, in which databases are used for conducting tests or experiments, and the workflow is simple. -To create the pre-built database image for the Enterprise/Standard edition using a pre-built image, please follow these instructions: [Pre-built Database (prebuiltdb) Extension](https://github.com/oracle/docker-images/blob/main/OracleDatabase/SingleInstance/extensions/prebuiltdb/README.md). +To build the pre-built database image for the Enterprise/Standard edition, please follow these instructions: [Pre-built Database (prebuiltdb) Extension](https://github.com/oracle/docker-images/blob/main/OracleDatabase/SingleInstance/extensions/prebuiltdb/README.md). ### Provisioning a new XE Database To provision new Oracle Database Express Edition (XE) database, use the sample **[config/samples/sidb/singleinstancedatabase_express.yaml](../../config/samples/sidb/singleinstancedatabase_express.yaml)** file. For example: @@ -543,23 +543,23 @@ Use this ORDS user to authenticate the following: #### Database API Examples Some examples for the Database API usage are as follows: -* ##### Getting all Database Components +* ##### Get all Database Components ```sh curl -s -k -X GET -u 'ORDS_PUBLIC_USER:<.spec.ordsPassword>' https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/database/components/ | python -m json.tool ``` -* ##### Getting all Database Users +* ##### Get all Database Users ```sh curl -s -k -X GET -u 'ORDS_PUBLIC_USER:<.spec.ordsPassword>' https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/database/security/users/ | python -m json.tool ``` -* ##### Getting all Tablespaces +* ##### Get all Tablespaces ```sh curl -s -k -X GET -u 'ORDS_PUBLIC_USER:<.spec.ordsPassword>' https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/database/storage/tablespaces/ | python -m json.tool ``` -* ##### Getting all Database Parameters +* ##### Get all Database Parameters ```sh curl -s -k -X GET -u 'ORDS_PUBLIC_USER:<.spec.ordsPassword>' https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/database/parameters/ | python -m json.tool ``` -* ##### Getting all Feature Usage Statistics +* ##### Get all Feature Usage Statistics ```sh curl -s -k -X GET -u 'ORDS_PUBLIC_USER:<.spec.ordsPassword>' https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/database/feature_usage/ | python -m json.tool ``` diff --git a/docs/sidb/SIDB_PREREQUISITES.md b/docs/sidb/SIDB_PREREQUISITES.md index 7cf61165..3ae417c6 100644 --- a/docs/sidb/SIDB_PREREQUISITES.md +++ b/docs/sidb/SIDB_PREREQUISITES.md @@ -6,17 +6,14 @@ To deploy Oracle Database Operator for Kubernetes on Oracle Docker images, compl Build SingleInstanceDatabase Docker Images from source, following the instructions at [https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance), or use the pre-built images available at [https://container-registry.oracle.com](https://container-registry.oracle.com) by signing in and accepting the required license agreement. - Oracle Database Releases Supported: Oracle Database 19c Enterprise Edition or Standard Edition, and later releases. + Oracle Database Releases Supported: Oracle Database 19c Enterprise Edition or Standard Edition, and later releases. Oracle Database 21.3 Express Edition. - Prepare the pre-built database docker image by following the [instructions](https://github.com/oracle/docker-images/blob/main/OracleDatabase/SingleInstance/extensions/prebuiltdb/README.md) - Build OracleRestDataService Docker Images from source following the instructions at [https://github.com/oracle/docker-images/tree/main/OracleRestDataServices](https://github.com/oracle/docker-images/tree/main/OracleRestDataServices). - OracleRestDataService version 20.4.1 onwards are supported. + Supported OracleRestDataService version is 21.4.2 * ### Set Up Kubernetes and Volumes - Set up an on-premises Kubernetes cluster, or subscribe to a managed Kubernetes service, such as Oracle Cloud Infrastructure Container Engine for Kubernetes, configured with persistent volumes. The persistent volumes are required for storage of the database files. + Set up an on-premises Kubernetes cluster, or subscribe to a managed Kubernetes service, such as Oracle Cloud Infrastructure Container Engine for Kubernetes. Use a dynamic volume provisioner or pre-provision static persistent volumes manually. These volumes are required for persistent storage of the database files. More info on creating persistent volumes available at [https://kubernetes.io/docs/concepts/storage/persistent-volumes/](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) - More info on creating persistent volumes on OCI is available at [https://docs.oracle.com/en-us/iaas/Content/ContEng/Tasks/contengcreatingpersistentvolumeclaim.htm](https://docs.oracle.com/en-us/iaas/Content/ContEng/Tasks/contengcreatingpersistentvolumeclaim.htm) From 4975b95e5913d66d2d4b3e1448776dc0b0ff4d0c Mon Sep 17 00:00:00 2001 From: Mohammed Yunus Date: Tue, 31 May 2022 12:48:57 +0000 Subject: [PATCH 413/628] Update SIDB_PREREQUISITES.md --- docs/sidb/SIDB_PREREQUISITES.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/sidb/SIDB_PREREQUISITES.md b/docs/sidb/SIDB_PREREQUISITES.md index 3ae417c6..4588a84d 100644 --- a/docs/sidb/SIDB_PREREQUISITES.md +++ b/docs/sidb/SIDB_PREREQUISITES.md @@ -3,13 +3,13 @@ To deploy Oracle Database Operator for Kubernetes on Oracle Docker images, compl * ### Prepare Oracle Docker Images - Build SingleInstanceDatabase Docker Images from source, following the instructions at [https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance), or + Build Single Instance Database Docker Images from source, following the instructions at [https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance), or use the pre-built images available at [https://container-registry.oracle.com](https://container-registry.oracle.com) by signing in and accepting the required license agreement. Oracle Database Releases Supported: Oracle Database 19c Enterprise Edition or Standard Edition, and later releases. Oracle Database 21.3 Express Edition. - Build OracleRestDataService Docker Images from source following the instructions at [https://github.com/oracle/docker-images/tree/main/OracleRestDataServices](https://github.com/oracle/docker-images/tree/main/OracleRestDataServices). - Supported OracleRestDataService version is 21.4.2 + Build Oracle REST Data Service Docker Images from source following the instructions at [https://github.com/oracle/docker-images/tree/main/OracleRestDataServices](https://github.com/oracle/docker-images/tree/main/OracleRestDataServices). + Supported Oracle REST Data Service version is 21.4.2 * ### Set Up Kubernetes and Volumes From 4b23139755a132b0c35906b8867d071408d8542c Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Tue, 31 May 2022 14:44:36 +0000 Subject: [PATCH 414/628] Update docs/adb/README.md --- docs/adb/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/adb/README.md b/docs/adb/README.md index dd86015f..08cc0b0a 100644 --- a/docs/adb/README.md +++ b/docs/adb/README.md @@ -2,7 +2,7 @@ Before you use the Oracle Database Operator for Kubernetes (the operator), ensure your system meets all of the Oracle Autonomous Database (ADB) Prerequisites [ADB_PREREQUISITES](./ADB_PREREQUISITES.md). -To interact with OCI services, either the cluster has to be authorized using Principal Instance, or using the API Key Authentication by specifying the configMap and the secret under the `ociConfig` field. +As indicated in the prerequisites (see above), to interact with OCI services, either the cluster has to be authorized using Principal Instance, or using the API Key Authentication by specifying the configMap and the secret under the `ociConfig` field. ## Required Permissions @@ -12,7 +12,7 @@ The permission to view the workrequests is also required, so that the operator w ## Supported Features -After the operator is deployed, choose either one of the following operations to create an `AutonomousDatabase` custom resource for Oracle Autonomous Database in your cluster. +After the operator is deployed, choose one of the following operations to create an `AutonomousDatabase` custom resource for Oracle Autonomous Database in your cluster. * [Provision](#provision-an-autonomous-database) an Autonomous Database * [Bind](#bind-to-an-existing-autonomous-database) to an existing Autonomous Database From b4a7313096f86b9af96b4a7ed059759bb3625bac Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Tue, 31 May 2022 14:45:18 +0000 Subject: [PATCH 415/628] Update docs/acd/README.md --- docs/acd/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/acd/README.md b/docs/acd/README.md index cd3a4303..08eb2954 100644 --- a/docs/acd/README.md +++ b/docs/acd/README.md @@ -2,7 +2,7 @@ Before you use the Oracle Database Operator for Kubernetes (the operator), ensure your system meets all of the Oracle Autonomous Database (ADB) Prerequisites [ADB_PREREQUISITES](./../adb/ADB_PREREQUISITES.md). -To interact with OCI services, either the cluster has to be authorized using Principal Instance, or using the API Key Authentication by specifying the configMap and the secret under the `ociConfig` field. +As indicated in the prerequisites (see above), to interact with OCI services, either the cluster has to be authorized using Principal Instance, or using the API Key Authentication by specifying the configMap and the secret under the `ociConfig` field. ## Required Permissions @@ -12,7 +12,7 @@ The permission to view the workrequests is also required, so that the operator w ## Supported Features -After the operator is deployed, choose either one of the following operations to create an `AutonomousContainerDatabase` custom resource for Oracle Autonomous Container Database in your cluster. +After the operator is deployed, choose one of the following operations to create an `AutonomousContainerDatabase` custom resource for Oracle Autonomous Container Database in your cluster. * [Provision](#provision-an-autonomous-container-database) an Autonomous Container Database * [Bind](#bind-to-an-existing-autonomous-container-database) to an existing Autonomous Container Database From 7b418cfe5f04852729c2cfd91d3f62e8e81c57c3 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Tue, 31 May 2022 11:51:16 -0400 Subject: [PATCH 416/628] update 3rd party libs --- apis/database/v1alpha1/adbfamily_common_utils.go | 6 +++--- .../v1alpha1/autonomouscontainerdatabase_types.go | 2 +- .../v1alpha1/autonomouscontainerdatabase_webhook_test.go | 4 ++-- apis/database/v1alpha1/autonomousdatabase_types.go | 2 +- apis/database/v1alpha1/autonomousdatabase_webhook.go | 4 ++-- apis/database/v1alpha1/autonomousdatabase_webhook_test.go | 4 ++-- apis/database/v1alpha1/autonomousdatabasebackup_types.go | 4 ++-- .../v1alpha1/autonomousdatabasebackup_webhook_test.go | 2 +- apis/database/v1alpha1/autonomousdatabaserestore_types.go | 6 +++--- .../v1alpha1/autonomousdatabaserestore_webhook_test.go | 2 +- commons/dbcssystem/dbcs_reconciler.go | 8 ++++---- commons/dbcssystem/dcommon.go | 6 +++--- commons/k8s/create.go | 4 ++-- commons/oci/containerdatabase.go | 4 ++-- commons/oci/database.go | 4 ++-- commons/oci/provider.go | 4 ++-- commons/oci/vault.go | 4 ++-- commons/oci/workrequest.go | 4 ++-- commons/sharding/scommon.go | 4 ++-- .../database/autonomouscontainerdatabase_controller.go | 2 +- controllers/database/autonomousdatabase_controller.go | 4 ++-- .../database/autonomousdatabaserestore_controller.go | 2 +- controllers/database/dbcssystem_controller.go | 6 +++--- controllers/database/shardingdatabase_controller.go | 4 ++-- go.mod | 4 ++-- go.sum | 7 ++++--- test/e2e/autonomouscontainerdatabase_test.go | 4 ++-- test/e2e/autonomousdatabase_controller_bind_test.go | 6 +++--- test/e2e/autonomousdatabase_controller_create_test.go | 4 ++-- test/e2e/behavior/shared_behaviors.go | 6 +++--- test/e2e/suite_test.go | 4 ++-- test/e2e/util/oci_acd_request.go | 4 ++-- test/e2e/util/oci_config_util.go | 2 +- test/e2e/util/oci_db_request.go | 4 ++-- test/e2e/util/oci_work_request.go | 4 ++-- 35 files changed, 73 insertions(+), 72 deletions(-) diff --git a/apis/database/v1alpha1/adbfamily_common_utils.go b/apis/database/v1alpha1/adbfamily_common_utils.go index a5dd3018..8dcb13d8 100644 --- a/apis/database/v1alpha1/adbfamily_common_utils.go +++ b/apis/database/v1alpha1/adbfamily_common_utils.go @@ -46,9 +46,9 @@ import ( metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "sigs.k8s.io/controller-runtime/pkg/client" - "github.com/oracle/oci-go-sdk/v63/common" - "github.com/oracle/oci-go-sdk/v63/database" - "github.com/oracle/oci-go-sdk/v63/workrequests" + "github.com/oracle/oci-go-sdk/v64/common" + "github.com/oracle/oci-go-sdk/v64/database" + "github.com/oracle/oci-go-sdk/v64/workrequests" ) // LastSuccessfulSpec is an annotation key which maps to the value of last successful spec diff --git a/apis/database/v1alpha1/autonomouscontainerdatabase_types.go b/apis/database/v1alpha1/autonomouscontainerdatabase_types.go index cdc037ef..28f36445 100644 --- a/apis/database/v1alpha1/autonomouscontainerdatabase_types.go +++ b/apis/database/v1alpha1/autonomouscontainerdatabase_types.go @@ -42,7 +42,7 @@ import ( "encoding/json" "reflect" - "github.com/oracle/oci-go-sdk/v63/database" + "github.com/oracle/oci-go-sdk/v64/database" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" ) diff --git a/apis/database/v1alpha1/autonomouscontainerdatabase_webhook_test.go b/apis/database/v1alpha1/autonomouscontainerdatabase_webhook_test.go index 1decc7b9..11a8d5b2 100644 --- a/apis/database/v1alpha1/autonomouscontainerdatabase_webhook_test.go +++ b/apis/database/v1alpha1/autonomouscontainerdatabase_webhook_test.go @@ -42,8 +42,8 @@ import ( "encoding/json" "time" - "github.com/oracle/oci-go-sdk/v63/common" - "github.com/oracle/oci-go-sdk/v63/database" + "github.com/oracle/oci-go-sdk/v64/common" + "github.com/oracle/oci-go-sdk/v64/database" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/types" // +kubebuilder:scaffold:imports diff --git a/apis/database/v1alpha1/autonomousdatabase_types.go b/apis/database/v1alpha1/autonomousdatabase_types.go index bfa1fe4b..f39d10bf 100644 --- a/apis/database/v1alpha1/autonomousdatabase_types.go +++ b/apis/database/v1alpha1/autonomousdatabase_types.go @@ -42,7 +42,7 @@ import ( "encoding/json" "reflect" - "github.com/oracle/oci-go-sdk/v63/database" + "github.com/oracle/oci-go-sdk/v64/database" metaV1 "k8s.io/apimachinery/pkg/apis/meta/v1" ) diff --git a/apis/database/v1alpha1/autonomousdatabase_webhook.go b/apis/database/v1alpha1/autonomousdatabase_webhook.go index cc805bec..8aeaa00d 100644 --- a/apis/database/v1alpha1/autonomousdatabase_webhook.go +++ b/apis/database/v1alpha1/autonomousdatabase_webhook.go @@ -41,8 +41,8 @@ package v1alpha1 import ( "fmt" - "github.com/oracle/oci-go-sdk/v63/common" - "github.com/oracle/oci-go-sdk/v63/database" + "github.com/oracle/oci-go-sdk/v64/common" + "github.com/oracle/oci-go-sdk/v64/database" apierrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/runtime" "k8s.io/apimachinery/pkg/runtime/schema" diff --git a/apis/database/v1alpha1/autonomousdatabase_webhook_test.go b/apis/database/v1alpha1/autonomousdatabase_webhook_test.go index ce812938..639507ef 100644 --- a/apis/database/v1alpha1/autonomousdatabase_webhook_test.go +++ b/apis/database/v1alpha1/autonomousdatabase_webhook_test.go @@ -42,8 +42,8 @@ import ( "encoding/json" "time" - "github.com/oracle/oci-go-sdk/v63/common" - "github.com/oracle/oci-go-sdk/v63/database" + "github.com/oracle/oci-go-sdk/v64/common" + "github.com/oracle/oci-go-sdk/v64/database" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/types" // +kubebuilder:scaffold:imports diff --git a/apis/database/v1alpha1/autonomousdatabasebackup_types.go b/apis/database/v1alpha1/autonomousdatabasebackup_types.go index 52974622..27531fd5 100644 --- a/apis/database/v1alpha1/autonomousdatabasebackup_types.go +++ b/apis/database/v1alpha1/autonomousdatabasebackup_types.go @@ -41,8 +41,8 @@ package v1alpha1 import ( metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" - "github.com/oracle/oci-go-sdk/v63/common" - "github.com/oracle/oci-go-sdk/v63/database" + "github.com/oracle/oci-go-sdk/v64/common" + "github.com/oracle/oci-go-sdk/v64/database" ) // EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! diff --git a/apis/database/v1alpha1/autonomousdatabasebackup_webhook_test.go b/apis/database/v1alpha1/autonomousdatabasebackup_webhook_test.go index d382ee60..481f4ac7 100644 --- a/apis/database/v1alpha1/autonomousdatabasebackup_webhook_test.go +++ b/apis/database/v1alpha1/autonomousdatabasebackup_webhook_test.go @@ -41,7 +41,7 @@ import ( "context" "time" - "github.com/oracle/oci-go-sdk/v63/common" + "github.com/oracle/oci-go-sdk/v64/common" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/types" // +kubebuilder:scaffold:imports diff --git a/apis/database/v1alpha1/autonomousdatabaserestore_types.go b/apis/database/v1alpha1/autonomousdatabaserestore_types.go index 05583305..bbc4e1c0 100644 --- a/apis/database/v1alpha1/autonomousdatabaserestore_types.go +++ b/apis/database/v1alpha1/autonomousdatabaserestore_types.go @@ -43,9 +43,9 @@ import ( metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" - "github.com/oracle/oci-go-sdk/v63/common" - "github.com/oracle/oci-go-sdk/v63/database" - "github.com/oracle/oci-go-sdk/v63/workrequests" + "github.com/oracle/oci-go-sdk/v64/common" + "github.com/oracle/oci-go-sdk/v64/database" + "github.com/oracle/oci-go-sdk/v64/workrequests" ) // EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! diff --git a/apis/database/v1alpha1/autonomousdatabaserestore_webhook_test.go b/apis/database/v1alpha1/autonomousdatabaserestore_webhook_test.go index 332aa47b..cd92779f 100644 --- a/apis/database/v1alpha1/autonomousdatabaserestore_webhook_test.go +++ b/apis/database/v1alpha1/autonomousdatabaserestore_webhook_test.go @@ -38,7 +38,7 @@ package v1alpha1 import ( - "github.com/oracle/oci-go-sdk/v63/common" + "github.com/oracle/oci-go-sdk/v64/common" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" // +kubebuilder:scaffold:imports ) diff --git a/commons/dbcssystem/dbcs_reconciler.go b/commons/dbcssystem/dbcs_reconciler.go index 878dd94e..61170ec1 100644 --- a/commons/dbcssystem/dbcs_reconciler.go +++ b/commons/dbcssystem/dbcs_reconciler.go @@ -46,10 +46,10 @@ import ( "time" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v63/common" - "github.com/oracle/oci-go-sdk/v63/core" - "github.com/oracle/oci-go-sdk/v63/database" - "github.com/oracle/oci-go-sdk/v63/workrequests" + "github.com/oracle/oci-go-sdk/v64/common" + "github.com/oracle/oci-go-sdk/v64/core" + "github.com/oracle/oci-go-sdk/v64/database" + "github.com/oracle/oci-go-sdk/v64/workrequests" "k8s.io/client-go/tools/record" "k8s.io/client-go/util/retry" diff --git a/commons/dbcssystem/dcommon.go b/commons/dbcssystem/dcommon.go index 373951df..76ea74f4 100644 --- a/commons/dbcssystem/dcommon.go +++ b/commons/dbcssystem/dcommon.go @@ -44,9 +44,9 @@ import ( "strconv" "strings" - "github.com/oracle/oci-go-sdk/v63/common" - "github.com/oracle/oci-go-sdk/v63/database" - "github.com/oracle/oci-go-sdk/v63/workrequests" + "github.com/oracle/oci-go-sdk/v64/common" + "github.com/oracle/oci-go-sdk/v64/database" + "github.com/oracle/oci-go-sdk/v64/workrequests" "sigs.k8s.io/controller-runtime/pkg/client" databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" diff --git a/commons/k8s/create.go b/commons/k8s/create.go index 68258139..3c058156 100644 --- a/commons/k8s/create.go +++ b/commons/k8s/create.go @@ -43,8 +43,8 @@ import ( dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" - "github.com/oracle/oci-go-sdk/v63/common" - "github.com/oracle/oci-go-sdk/v63/database" + "github.com/oracle/oci-go-sdk/v64/common" + "github.com/oracle/oci-go-sdk/v64/database" corev1 "k8s.io/api/core/v1" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "sigs.k8s.io/controller-runtime/pkg/client" diff --git a/commons/oci/containerdatabase.go b/commons/oci/containerdatabase.go index b8ba6ce6..9fa9ec91 100644 --- a/commons/oci/containerdatabase.go +++ b/commons/oci/containerdatabase.go @@ -41,8 +41,8 @@ package oci import ( "context" - "github.com/oracle/oci-go-sdk/v63/common" - "github.com/oracle/oci-go-sdk/v63/database" + "github.com/oracle/oci-go-sdk/v64/common" + "github.com/oracle/oci-go-sdk/v64/database" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" ) diff --git a/commons/oci/database.go b/commons/oci/database.go index 372c9631..1e08cc66 100644 --- a/commons/oci/database.go +++ b/commons/oci/database.go @@ -43,8 +43,8 @@ import ( "fmt" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v63/common" - "github.com/oracle/oci-go-sdk/v63/database" + "github.com/oracle/oci-go-sdk/v64/common" + "github.com/oracle/oci-go-sdk/v64/database" "sigs.k8s.io/controller-runtime/pkg/client" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" diff --git a/commons/oci/provider.go b/commons/oci/provider.go index 1609103d..a5ded3cc 100644 --- a/commons/oci/provider.go +++ b/commons/oci/provider.go @@ -41,8 +41,8 @@ package oci import ( "errors" - "github.com/oracle/oci-go-sdk/v63/common" - "github.com/oracle/oci-go-sdk/v63/common/auth" + "github.com/oracle/oci-go-sdk/v64/common" + "github.com/oracle/oci-go-sdk/v64/common/auth" "sigs.k8s.io/controller-runtime/pkg/client" diff --git a/commons/oci/vault.go b/commons/oci/vault.go index 83722eca..7a83e4f9 100644 --- a/commons/oci/vault.go +++ b/commons/oci/vault.go @@ -43,8 +43,8 @@ import ( "encoding/base64" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v63/common" - "github.com/oracle/oci-go-sdk/v63/secrets" + "github.com/oracle/oci-go-sdk/v64/common" + "github.com/oracle/oci-go-sdk/v64/secrets" ) type VaultService interface { diff --git a/commons/oci/workrequest.go b/commons/oci/workrequest.go index 541902c5..4eb0b2d0 100644 --- a/commons/oci/workrequest.go +++ b/commons/oci/workrequest.go @@ -44,8 +44,8 @@ import ( "github.com/go-logr/logr" "sigs.k8s.io/controller-runtime/pkg/client" - "github.com/oracle/oci-go-sdk/v63/common" - "github.com/oracle/oci-go-sdk/v63/workrequests" + "github.com/oracle/oci-go-sdk/v64/common" + "github.com/oracle/oci-go-sdk/v64/workrequests" ) type WorkRequestService interface { diff --git a/commons/sharding/scommon.go b/commons/sharding/scommon.go index 9392fba0..7f6d5da2 100644 --- a/commons/sharding/scommon.go +++ b/commons/sharding/scommon.go @@ -49,8 +49,8 @@ import ( "strings" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v63/common" - "github.com/oracle/oci-go-sdk/v63/ons" + "github.com/oracle/oci-go-sdk/v64/common" + "github.com/oracle/oci-go-sdk/v64/ons" appsv1 "k8s.io/api/apps/v1" corev1 "k8s.io/api/core/v1" "k8s.io/apimachinery/pkg/api/errors" diff --git a/controllers/database/autonomouscontainerdatabase_controller.go b/controllers/database/autonomouscontainerdatabase_controller.go index e564f9df..660bf0f8 100644 --- a/controllers/database/autonomouscontainerdatabase_controller.go +++ b/controllers/database/autonomouscontainerdatabase_controller.go @@ -45,7 +45,7 @@ import ( "reflect" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v63/database" + "github.com/oracle/oci-go-sdk/v64/database" corev1 "k8s.io/api/core/v1" apiErrors "k8s.io/apimachinery/pkg/api/errors" diff --git a/controllers/database/autonomousdatabase_controller.go b/controllers/database/autonomousdatabase_controller.go index b0552e10..b3ca889d 100644 --- a/controllers/database/autonomousdatabase_controller.go +++ b/controllers/database/autonomousdatabase_controller.go @@ -49,8 +49,8 @@ import ( "time" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v63/common" - "github.com/oracle/oci-go-sdk/v63/database" + "github.com/oracle/oci-go-sdk/v64/common" + "github.com/oracle/oci-go-sdk/v64/database" corev1 "k8s.io/api/core/v1" apiErrors "k8s.io/apimachinery/pkg/api/errors" diff --git a/controllers/database/autonomousdatabaserestore_controller.go b/controllers/database/autonomousdatabaserestore_controller.go index 26dc33d0..5a25759c 100644 --- a/controllers/database/autonomousdatabaserestore_controller.go +++ b/controllers/database/autonomousdatabaserestore_controller.go @@ -53,7 +53,7 @@ import ( "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" "sigs.k8s.io/controller-runtime/pkg/predicate" - "github.com/oracle/oci-go-sdk/v63/common" + "github.com/oracle/oci-go-sdk/v64/common" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" "github.com/oracle/oracle-database-operator/commons/adb_family" "github.com/oracle/oracle-database-operator/commons/k8s" diff --git a/controllers/database/dbcssystem_controller.go b/controllers/database/dbcssystem_controller.go index fdefd40d..29030601 100644 --- a/controllers/database/dbcssystem_controller.go +++ b/controllers/database/dbcssystem_controller.go @@ -48,9 +48,9 @@ import ( "github.com/oracle/oracle-database-operator/commons/oci" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v63/core" - "github.com/oracle/oci-go-sdk/v63/database" - "github.com/oracle/oci-go-sdk/v63/workrequests" + "github.com/oracle/oci-go-sdk/v64/core" + "github.com/oracle/oci-go-sdk/v64/database" + "github.com/oracle/oci-go-sdk/v64/workrequests" "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/runtime" "k8s.io/client-go/tools/record" diff --git a/controllers/database/shardingdatabase_controller.go b/controllers/database/shardingdatabase_controller.go index 4258276d..a67401d0 100644 --- a/controllers/database/shardingdatabase_controller.go +++ b/controllers/database/shardingdatabase_controller.go @@ -47,8 +47,8 @@ import ( "time" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v63/common" - "github.com/oracle/oci-go-sdk/v63/ons" + "github.com/oracle/oci-go-sdk/v64/common" + "github.com/oracle/oci-go-sdk/v64/ons" appsv1 "k8s.io/api/apps/v1" corev1 "k8s.io/api/core/v1" "k8s.io/apimachinery/pkg/api/errors" diff --git a/go.mod b/go.mod index 50ae07e5..4e950301 100644 --- a/go.mod +++ b/go.mod @@ -3,10 +3,10 @@ module github.com/oracle/oracle-database-operator go 1.17 require ( - github.com/go-logr/logr v1.2.2 + github.com/go-logr/logr v1.2.3 github.com/onsi/ginkgo/v2 v2.1.3 github.com/onsi/gomega v1.19.0 - github.com/oracle/oci-go-sdk/v63 v63.0.0 + github.com/oracle/oci-go-sdk/v64 v64.0.0 gopkg.in/yaml.v2 v2.4.0 k8s.io/api v0.23.6 k8s.io/apimachinery v0.23.6 diff --git a/go.sum b/go.sum index d08c77ba..62d4fc56 100644 --- a/go.sum +++ b/go.sum @@ -163,8 +163,9 @@ github.com/go-logfmt/logfmt v0.5.0/go.mod h1:wCYkCAKZfumFQihp8CzCvQ3paCTfi41vtzG github.com/go-logr/logr v0.1.0/go.mod h1:ixOQHD9gLJUVQQ2ZOR7zLEifBX6tGkNJF4QyIY7sIas= github.com/go-logr/logr v0.2.0/go.mod h1:z6/tIYblkpsD+a4lm/fGIIU9mZ+XfAiaFtq7xTgseGU= github.com/go-logr/logr v1.2.0/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A= -github.com/go-logr/logr v1.2.2 h1:ahHml/yUpnlb96Rp8HCvtYVPY8ZYpxq3g7UYchIYwbs= github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A= +github.com/go-logr/logr v1.2.3 h1:2DntVwHkVopvECVRSlL5PSo9eG+cAkDCuckLubN+rq0= +github.com/go-logr/logr v1.2.3/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A= github.com/go-logr/zapr v1.2.0/go.mod h1:Qa4Bsj2Vb+FAVeAKsLD8RLQ+YRJB8YDmOAKxaBQf7Ro= github.com/go-logr/zapr v1.2.2 h1:5YNlIL6oZLydaV4dOFjL8YpgXF/tPeTbnpatnu3cq6o= github.com/go-logr/zapr v1.2.2/go.mod h1:eIauM6P8qSvTw5o2ez6UEAfGjQKrxQTl5EoK+Qa2oG4= @@ -395,8 +396,8 @@ github.com/onsi/gomega v1.17.0/go.mod h1:HnhC7FXeEQY45zxNK3PPoIUhzk/80Xly9PcubAl github.com/onsi/gomega v1.19.0 h1:4ieX6qQjPP/BfC3mpsAtIGGlxTWPeA3Inl/7DtXw1tw= github.com/onsi/gomega v1.19.0/go.mod h1:LY+I3pBVzYsTBU1AnDwOSxaYi9WoWiqgwooUqq9yPro= github.com/opentracing/opentracing-go v1.1.0/go.mod h1:UkNAQd3GIcIGf0SeVgPpRdFStlNbqXla1AfSYxPUl2o= -github.com/oracle/oci-go-sdk/v63 v63.0.0 h1:OOGCUmaDzrd5zTG8pljcnkR1ZHxg/991uEiQJi95/4E= -github.com/oracle/oci-go-sdk/v63 v63.0.0/go.mod h1:n6V9PcyRW5wtHaNd2TltbV3sWvbNy3PNqLmLrcT23Fg= +github.com/oracle/oci-go-sdk/v64 v64.0.0 h1:ALQDxoKLeNfxLbbOkjnNxMf3/AyWhnHhMJxc8tZXRgI= +github.com/oracle/oci-go-sdk/v64 v64.0.0/go.mod h1:eLqTg9MSXE/9naair0yFKqiP1RlYghuzBDvEgZvwjYs= github.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc= github.com/pelletier/go-toml v1.2.0/go.mod h1:5z9KED0ma1S8pY6P1sdut58dfprrGBbd/94hg7ilaic= github.com/pelletier/go-toml v1.9.3/go.mod h1:u1nR/EPcESfeI/szUZKdtJ0xRNbUoANCkoOuaOx1Y+c= diff --git a/test/e2e/autonomouscontainerdatabase_test.go b/test/e2e/autonomouscontainerdatabase_test.go index 6289b150..7e8d7191 100644 --- a/test/e2e/autonomouscontainerdatabase_test.go +++ b/test/e2e/autonomouscontainerdatabase_test.go @@ -42,8 +42,8 @@ import ( "context" "time" - "github.com/oracle/oci-go-sdk/v63/common" - "github.com/oracle/oci-go-sdk/v63/database" + "github.com/oracle/oci-go-sdk/v64/common" + "github.com/oracle/oci-go-sdk/v64/database" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/types" diff --git a/test/e2e/autonomousdatabase_controller_bind_test.go b/test/e2e/autonomousdatabase_controller_bind_test.go index 3bd83710..ce5de55a 100644 --- a/test/e2e/autonomousdatabase_controller_bind_test.go +++ b/test/e2e/autonomousdatabase_controller_bind_test.go @@ -42,9 +42,9 @@ import ( "context" "time" - "github.com/oracle/oci-go-sdk/v63/common" - "github.com/oracle/oci-go-sdk/v63/database" - "github.com/oracle/oci-go-sdk/v63/workrequests" + "github.com/oracle/oci-go-sdk/v64/common" + "github.com/oracle/oci-go-sdk/v64/database" + "github.com/oracle/oci-go-sdk/v64/workrequests" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/types" diff --git a/test/e2e/autonomousdatabase_controller_create_test.go b/test/e2e/autonomousdatabase_controller_create_test.go index c98fc04f..789d124e 100644 --- a/test/e2e/autonomousdatabase_controller_create_test.go +++ b/test/e2e/autonomousdatabase_controller_create_test.go @@ -42,8 +42,8 @@ import ( "context" "time" - "github.com/oracle/oci-go-sdk/v63/common" - "github.com/oracle/oci-go-sdk/v63/database" + "github.com/oracle/oci-go-sdk/v64/common" + "github.com/oracle/oci-go-sdk/v64/database" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/types" diff --git a/test/e2e/behavior/shared_behaviors.go b/test/e2e/behavior/shared_behaviors.go index 14ee546d..244f07ba 100644 --- a/test/e2e/behavior/shared_behaviors.go +++ b/test/e2e/behavior/shared_behaviors.go @@ -47,9 +47,9 @@ import ( "github.com/onsi/ginkgo/v2" "github.com/onsi/gomega" - "github.com/oracle/oci-go-sdk/v63/common" - "github.com/oracle/oci-go-sdk/v63/database" - "github.com/oracle/oci-go-sdk/v63/workrequests" + "github.com/oracle/oci-go-sdk/v64/common" + "github.com/oracle/oci-go-sdk/v64/database" + "github.com/oracle/oci-go-sdk/v64/workrequests" corev1 "k8s.io/api/core/v1" k8sErrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/types" diff --git a/test/e2e/suite_test.go b/test/e2e/suite_test.go index 832d2633..d0e195ed 100644 --- a/test/e2e/suite_test.go +++ b/test/e2e/suite_test.go @@ -49,8 +49,8 @@ import ( "github.com/onsi/ginkgo/v2" "github.com/onsi/gomega" "github.com/onsi/gomega/gexec" - "github.com/oracle/oci-go-sdk/v63/common" - "github.com/oracle/oci-go-sdk/v63/database" + "github.com/oracle/oci-go-sdk/v64/common" + "github.com/oracle/oci-go-sdk/v64/database" corev1 "k8s.io/api/core/v1" "k8s.io/client-go/kubernetes/scheme" "k8s.io/client-go/rest" diff --git a/test/e2e/util/oci_acd_request.go b/test/e2e/util/oci_acd_request.go index cb21fe2b..1bd8f345 100644 --- a/test/e2e/util/oci_acd_request.go +++ b/test/e2e/util/oci_acd_request.go @@ -41,8 +41,8 @@ package e2eutil import ( "context" - "github.com/oracle/oci-go-sdk/v63/common" - "github.com/oracle/oci-go-sdk/v63/database" + "github.com/oracle/oci-go-sdk/v64/common" + "github.com/oracle/oci-go-sdk/v64/database" // "io" // "io/ioutil" // "time" diff --git a/test/e2e/util/oci_config_util.go b/test/e2e/util/oci_config_util.go index 3780d2cd..5fb5527f 100644 --- a/test/e2e/util/oci_config_util.go +++ b/test/e2e/util/oci_config_util.go @@ -48,7 +48,7 @@ import ( "regexp" "strings" - "github.com/oracle/oci-go-sdk/v63/common" + "github.com/oracle/oci-go-sdk/v64/common" corev1 "k8s.io/api/core/v1" ) diff --git a/test/e2e/util/oci_db_request.go b/test/e2e/util/oci_db_request.go index 6ba74ad9..5a835803 100644 --- a/test/e2e/util/oci_db_request.go +++ b/test/e2e/util/oci_db_request.go @@ -41,8 +41,8 @@ package e2eutil import ( "context" - "github.com/oracle/oci-go-sdk/v63/common" - "github.com/oracle/oci-go-sdk/v63/database" + "github.com/oracle/oci-go-sdk/v64/common" + "github.com/oracle/oci-go-sdk/v64/database" "io" "io/ioutil" "time" diff --git a/test/e2e/util/oci_work_request.go b/test/e2e/util/oci_work_request.go index a20b45d7..a58ce3cf 100644 --- a/test/e2e/util/oci_work_request.go +++ b/test/e2e/util/oci_work_request.go @@ -41,8 +41,8 @@ package e2eutil import ( "context" - "github.com/oracle/oci-go-sdk/v63/common" - "github.com/oracle/oci-go-sdk/v63/workrequests" + "github.com/oracle/oci-go-sdk/v64/common" + "github.com/oracle/oci-go-sdk/v64/workrequests" "time" ) From a879a2c26da57c7b5e9e79e8d00179da0630016f Mon Sep 17 00:00:00 2001 From: Abhishek Kumar Date: Wed, 1 Jun 2022 07:53:52 +0000 Subject: [PATCH 417/628] Discussed readme changes --- docs/sidb/README.md | 30 ++++++++++++++++++++++-------- 1 file changed, 22 insertions(+), 8 deletions(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 0828bbf2..86c5da4c 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -105,18 +105,18 @@ $ kubectl describe singleinstancedatabase sidb-sample-clone Events: Type Reason Age From Message ---- ------ ---- ---- ------- - Normal Database Pending 35m (x2 over 35m) SingleInstanceDatabase Waiting for database pod to be ready - Normal Database Creating 27m (x24 over 34m) SingleInstanceDatabase Waiting for database to be ready + Normal Database Pending 35m (x2 over 35m) SingleInstanceDatabase waiting for database pod to be ready + Normal Database Creating 27m (x24 over 34m) SingleInstanceDatabase waiting for database to be ready Normal Database Ready 22m SingleInstanceDatabase database open on pod sidb-sample-clone-133ol scheduled on node 10.0.10.6 Normal Datapatch Pending 21m SingleInstanceDatabase datapatch execution pending Normal Datapatch Executing 20m SingleInstanceDatabase datapatch begin execution - Normal Datapatch Done 8s SingleInstanceDatabase Datapatch from 19.3.0.0.0 to 19.11.0.0.0 : SUCCESS + Normal Datapatch Done 8s SingleInstanceDatabase datafiles patched from 19.3.0.0.0 to 19.11.0.0.0 : SUCCESS ``` ## Provision New Database -You can easily provision a new database instance on the Kubernetes cluster by using **[config/samples/sidb/singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml)**. +To provision a new database instance on the Kubernetes cluster, use the sample **[config/samples/sidb/singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml)**. 1. Log into [Oracle Container Registry](https://container-registry.oracle.com/) and accept the license agreement for the Database image, ignore if you have accepted already. @@ -152,7 +152,7 @@ The database persistence can be achieved in the following two ways: - Dynamic Persistence Provisioning - Static Persistence Provisioning -In **Dynamic Persistence Provisioning**, a persistent volume is provisioned by mentioning a storage class. For example, **oci-bv** storage class is specified in the **[singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml)** file. This storage class facilitates dynamic provisioning of the OCI block volumes. The supported access mode for this class is `ReadWriteOnce`. For other cloud providers, you can similarly use their dynamic provisioning storage classes. +In **Dynamic Persistence Provisioning**, a persistent volume is provisioned by mentioning a storage class. For example, **oci-bv** storage class is specified in the **[singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml)** file. This storage class facilitates dynamic provisioning of the OCI block volumes. The supported access mode for this class is `ReadWriteOnce`. For other cloud providers, you can similarly use their dynamic provisioning storage classes. **Note:** Generally, the `Reclaim Policy` of such dynamically provisioned volumes is `Delete`, hence, these volumes get deleted when their corresponding database deployment is deleted. To retain volumes, please use static provisioning as explained in the section below. In **Static Persistence Provisioning**, you have to create a volume manually (either using Block Volume or NFS), and then use the name of this volume with the `<.spec.persistence.volumeName>` field (corresponds to the `volumeName` field of the persistence section in the **[singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml)**). The `Reclaim Policy` of such volume can be set to `Retain`. So, this volume does not get deleted with the deletion of its corresponding deployment. The access modes supported with block volume and NFS are `ReadWriteOnce` and `ReadWriteMany` respectively. @@ -176,6 +176,15 @@ spec: driver: blockvolume.csi.oraclecloud.com volumeHandle: ``` + +**Note:** OCI block volumes are AD (Availability Domain) specific. Please make sure that the database is deployed in the same AD as that of its statically provisioned block volume. This is handled automatically in dynamic provisioning. +To provision the database in a specific AD, please uncomment the following line from the **[singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml)** file: + +```yaml +nodeSelector: + topology.kubernetes.io/zone: PHX-AD-1 +``` + #### NFS Volume Static Provisioning Similar to the block volume static provisioning, you have to manually create a file system resource from the OCI console, and fetch its `OCID, Mount Target and Export Path`. Mention these values in the following YAML file to create the persistent volume: @@ -195,6 +204,9 @@ spec: driver: fss.csi.oraclecloud.com volumeHandle: ":/" ``` + +**Note:** Whenever a mount target is provisioned in OCI, its `Reported Size (GiB)` values are very large. This is visible on the mount target page when logged in to the OCI console. Some applications will fail to install if the results of a space requirements check show too much available disk space. So please specify, in gibibytes (GiB), the maximum capacity reported by file systems exported through this mount target. This setting does not limit the actual amount of data you can store. + ### Provision a Pre-built Database To provision a new pre-built database instance, use the sample **[config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml](../../config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml)** file. For example: @@ -280,7 +292,7 @@ The following database parameters can be updated after the database is created: - `forceLog` To change these parameters, change their attribute values, and apply the change by using the -`kubectl` `apply` or `edit`/`patch` commands. +`kubectl apply` or `kubectl edit/patch` commands. **Caution**: Enable `archiveLog` mode before setting `flashback` to `ON`, and set `flashback` to `OFF` before disabling `archiveLog` mode. @@ -311,14 +323,16 @@ The following database initialization parameters can be updated after the databa - cpuCount - processes. -Change their attribute values and apply using kubectl **apply** or **edit/patch** commands. +Change their attribute values and apply using `kubectl apply` or `kubectl edit/patch` commands. **NOTE:** The value for the initialization parameter `sgaTarget` that you provide should be within the range set by [sga_min_size, sga_max_size]. If the value you provide is not in that range, then `sga_target` is not updated to the value you specify for `sgaTarget`. ### Multiple Replicas -In multiple replicas mode, more than one pod is created for the database. The database is open and mounted by one of the replica pods. Other replica pods have instances started but not mounted, and serve to provide a quick cold fail-over in case the active pod goes down. To enable multiple replicas, Update the replica attribute in the `.yaml`, and apply by using the `kubectl apply` or `edit/patch` commands. +In multiple replicas mode, more than one pod is created for the database. The database is open and mounted by one of the replica pods. Other replica pods have instances started but not mounted, and serve to provide a quick cold fail-over in case the active pod goes down. Multiple replicas are also helpful in [patching](#patch-existing-database) operation. Please ensure that you have multiple replicas of the database pods running before you start the patching operation for minimum downtime. + +To enable multiple replicas, update the replica attribute in the `.yaml`, and apply by using the `kubectl apply` or `kubectl scale` commands. **Note:** - This functionality requires the [k8s extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/k8s) extended images. The database image from the container registry `container-registry.oracle.com` includes the K8s extension. From 9a5104f9d92fd73cf9bcbd6702d3859ffdfcbc1e Mon Sep 17 00:00:00 2001 From: kuassi_mensah Date: Wed, 1 Jun 2022 14:35:04 +0000 Subject: [PATCH 418/628] Made minor updates to reflect the supported database configurations in v0.2.0 --- README.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 8476dbd4..b29c7906 100644 --- a/README.md +++ b/README.md @@ -26,7 +26,7 @@ Upcoming releases will support new configurations, operations and capabilities. ## Release Status -**CAUTION:** The current release of `OraOperator` (v0.1.0) is for development and test only. DO NOT USE IN PRODUCTION. +**CAUTION:** The current release of `OraOperator` (v0.2.0) is for development and test only. DO NOT USE IN PRODUCTION. This release can be deployed on the following platforms: @@ -82,9 +82,11 @@ For more details, see [Oracle Database Operator Installation Instrunctions](./do The quickstarts are designed for specific database configurations, including: +* [Oracle Autonomous Container Database](./docs/acd/README.md) * [Oracle Autonomous Database](./docs/adb/README.md) * [Oracle Database Single Instance configuration](./docs/sidb/README.md) * [Oracle Database configured with Oracle Sharding](./docs/sharding/README.md) +* [Oracle On-Premises Database](./docs/onpremdb/README.md) * [Oracle Database Cloud Services](./docs/dbcs/README.md) YAML file templates are available under [`/config/samples`](./config/samples/). You can copy and edit these template files to configure them for your use cases. @@ -101,6 +103,7 @@ YAML file templates are available under [`/config/samples`](./config/samples/). kubectl delete singleinstancedatabase.database.oracle.com --all -n kubectl delete shardingdatabase.database.oracle.com --all -n kubectl delete autonomousdatabase.database.oracle.com --all -n + ``` After all CRD instances are deleted, it is safe to remove the CRDs, APISerivces and operator deployment. @@ -119,7 +122,7 @@ YAML file templates are available under [`/config/samples`](./config/samples/). kubectl delete deployment.apps/oracle-database-operator-controller-manager -n oracle-database-operator-system ``` -## Documentation +## Docs of the supported Oracle Database configurations * [Oracle Autonomous Database](https://docs.oracle.com/en-us/iaas/Content/Database/Concepts/adboverview.htm) * [Oracle Database Single Instance](https://docs.oracle.com/en/database/oracle/oracle-database/) From c03d76437dd937f24d2d23fe652be08595787a5b Mon Sep 17 00:00:00 2001 From: Mohammed Yunus Date: Wed, 1 Jun 2022 15:45:15 +0000 Subject: [PATCH 419/628] Update README.md --- docs/sidb/README.md | 88 ++++++++++++++++++++++----------------------- 1 file changed, 43 insertions(+), 45 deletions(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 86c5da4c..349be81f 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -5,10 +5,13 @@ Oracle Database Operator for Kubernetes (`OraOperator`) includes the Single Inst * [Prerequisites](#prerequisites) * [Kind SingleInstanceDatabase Resource](#kind-singleinstancedatabase-resource) * [Provision New Database](#provision-new-database) + * [Provision Pre-built Database](#provision-pre-built-database) + * [Provision XE Database](#provision-xe-database) * [Clone Existing Database](#clone-existing-database) * [Patch/Rollback Database](#patchrollback-database) * [Kind OracleRestDataService](#kind-oraclerestdataservice) - * [REST Enable Database](#rest-enable-database) + * [REST Enabled Database](#rest-enabled-database) + * [APEX Enabled Database](#apex-enabled-database) * [Performing Maintenance Operations](#performing-maintenance-operations) ## Prerequisites @@ -17,21 +20,7 @@ Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISI ## Kind SingleInstanceDatabase Resource -The Oracle Database Operator creates the `SingleInstanceDatabase` kind as a custom resource. Doing this enables Oracle Database to be managed as a native Kubernetes object. - -### SingleInstanceDatabase Template YAML - -The template `.yaml` file for Single Instance Database (Enterprise and Standard Editions), including all the configurable options, is available at: -**[config/samples/sidb/singleinstancedatabase.yaml](./../../config/samples/sidb/singleinstancedatabase.yaml)** - -**Note:** -The `adminPassword` field in the above `singleinstancedatabase.yaml` file refers to a secret for the SYS, SYSTEM and PDBADMIN users of the Single Instance Database. This secret is required when you provision a new database, or when you clone an existing database. - -Create this secret using the following command as an example: - - kubectl create secret generic admin-secret --from-literal=oracle_pwd= - -This command creates a secret named `admin-secret`, with the key `oracle_pwd` mapped to the actual password specified in the command. +The Oracle Database Operator creates the `SingleInstanceDatabase` kind as a custom resource. Doing this enables Oracle Database to be managed as a native Kubernetes object as shown below. ### List Databases To list databases, use the following command as an example, where the database names are `sidb-sample` and `sidb-sample-clone`, which are the names we will use as database names in command examples: @@ -114,6 +103,20 @@ $ kubectl describe singleinstancedatabase sidb-sample-clone ``` +### SingleInstanceDatabase Template YAML + +The template `.yaml` file for Single Instance Database (Enterprise and Standard Editions), including all the configurable options, is available at: +**[config/samples/sidb/singleinstancedatabase.yaml](./../../config/samples/sidb/singleinstancedatabase.yaml)** + +**Note:** +The `adminPassword` field in the above `singleinstancedatabase.yaml` file refers to a secret for the SYS, SYSTEM and PDBADMIN users of the Single Instance Database. This secret is required when you provision a new database, or when you clone an existing database. + +Create this secret using the following command as an example: + + kubectl create secret generic admin-secret --from-literal=oracle_pwd= + +This command creates a secret named `admin-secret`, with the key `oracle_pwd` mapped to the actual password specified in the command. + ## Provision New Database To provision a new database instance on the Kubernetes cluster, use the sample **[config/samples/sidb/singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml)**. @@ -207,7 +210,7 @@ spec: **Note:** Whenever a mount target is provisioned in OCI, its `Reported Size (GiB)` values are very large. This is visible on the mount target page when logged in to the OCI console. Some applications will fail to install if the results of a space requirements check show too much available disk space. So please specify, in gibibytes (GiB), the maximum capacity reported by file systems exported through this mount target. This setting does not limit the actual amount of data you can store. -### Provision a Pre-built Database +## Provision Pre-built Database To provision a new pre-built database instance, use the sample **[config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml](../../config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml)** file. For example: ```sh @@ -220,7 +223,7 @@ This pre-built image includes the data files of the database inside the image it To build the pre-built database image for the Enterprise/Standard edition, please follow these instructions: [Pre-built Database (prebuiltdb) Extension](https://github.com/oracle/docker-images/blob/main/OracleDatabase/SingleInstance/extensions/prebuiltdb/README.md). -### Provisioning a new XE Database +## Provision XE Database To provision new Oracle Database Express Edition (XE) database, use the sample **[config/samples/sidb/singleinstancedatabase_express.yaml](../../config/samples/sidb/singleinstancedatabase_express.yaml)** file. For example: kubectl apply -f singleinstancedatabase_express.yaml @@ -429,25 +432,10 @@ $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.releaseUp ## Kind OracleRestDataService -The Oracle Database Operator creates the `OracleRestDataService` (ORDS) kind as a custom resource. Creating ORDS as a custom resource enables the RESTful API access to the Oracle Database in K8s. - -### OracleRestDataService Template YAML - -The template `.yaml` file for Oracle Rest Data Services (`OracleRestDataService` kind) is available at **[config/samples/sidb/oraclerestdataservice.yaml](config/samples/sidb/oraclerestdataservice.yaml)**. - -For **quick provisioning** of ORDS, apply the sample **[config/samples/sidb/oraclerestdataservice_create.yaml](../../config/samples/sidb/oraclerestdataservice_create.yaml)** file, using the following command: - - kubectl apply -f oraclerestdataservice_create.yaml - - -Note the following: -- The `adminPassword` and `ordsPassword` fields in the `oraclerestdataservice.yaml` file contains secrets for authenticating the Single Instance Database and the ORDS user with the following roles: `SQL Administrator, System Administrator, SQL Developer, oracle.dbtools.autorest.any.schema`. -- To build the ORDS image, use the following instructions: [Building Oracle REST Data Services Install Images](https://github.com/oracle/docker-images/tree/main/OracleRestDataServices#building-oracle-rest-data-services-install-images). -- By default, ORDS uses self-signed certificates. To use certificates from the Certificate Authority, the ORDS image needs to be rebuilt after specifying the values of `ssl.cert` and `ssl.cert.key` in the [standalone.properties](https://github.com/oracle/docker-images/blob/main/OracleRestDataServices/dockerfiles/standalone.properties.tmpl) file. After you rebuild the ORDS image, use the rebuilt image in the **[config/samples/sidb/oraclerestdataservice.yaml](config/samples/sidb/oraclerestdataservice.yaml)** file. -- If you want to install ORDS in a [prebuilt database](#provision-a-pre-built-database), make sure to attach the **database persistence** by uncommenting the `persistence` section in the **[config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml](../../config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml)** file, while provisioning the prebuilt database. +The Oracle Database Operator creates the `OracleRestDataService` (ORDS) kind as a custom resource. Creating ORDS as a custom resource enables the RESTful API access to the Oracle Database in K8s and enables it to be managed as a native Kubernetes object as shown below. ### List OracleRestDataServices -To list the ORDS service, use the following command: +To list ORDS services, use the following command: ```sh $ kubectl get oraclerestdataservice -o name @@ -501,16 +489,19 @@ $ kubectl describe oraclerestdataservice ords-sample ``` -### Delete ORDS -- To delete ORDS run the following command: - - kubectl delete oraclerestdataservice ords-samples +### OracleRestDataService Template YAML + +The template `.yaml` file for Oracle Rest Data Services (`OracleRestDataService` kind), including all the configurable options, is available at **[config/samples/sidb/oraclerestdataservice.yaml](config/samples/sidb/oraclerestdataservice.yaml)**. -- You cannot delete referred Single Instance Database (SIDB) before deleting its ORDS resource. +**Note:** +- The `adminPassword` and `ordsPassword` fields in the `oraclerestdataservice.yaml` file contains secrets for authenticating the Single Instance Database and the ORDS user with the following roles: `SQL Administrator, System Administrator, SQL Developer, oracle.dbtools.autorest.any.schema`. +- To build the ORDS image, use the following instructions: [Building Oracle REST Data Services Install Images](https://github.com/oracle/docker-images/tree/main/OracleRestDataServices#building-oracle-rest-data-services-install-images). +- By default, ORDS uses self-signed certificates. To use certificates from the Certificate Authority, the ORDS image needs to be rebuilt after specifying the values of `ssl.cert` and `ssl.cert.key` in the [standalone.properties](https://github.com/oracle/docker-images/blob/main/OracleRestDataServices/dockerfiles/standalone.properties.tmpl) file. After you rebuild the ORDS image, use the rebuilt image in the **[config/samples/sidb/oraclerestdataservice.yaml](config/samples/sidb/oraclerestdataservice.yaml)** file. +- If you want to install ORDS in a [prebuilt database](#provision-a-pre-built-database), make sure to attach the **database persistence** by uncommenting the `persistence` section in the **[config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml](../../config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml)** file, while provisioning the prebuilt database. -## REST Enable Database +## REST Enabled Database -To provision a new ORDS instance, use the sample **[config/samples/sidb/oraclerestdataservice_create.yaml](../../config/samples/sidb/oraclerestdataservice_create.yaml)** file. For example: +To quickly provision a new ORDS instance, use the sample **[config/samples/sidb/oraclerestdataservice_create.yaml](../../config/samples/sidb/oraclerestdataservice_create.yaml)** file. For example: ```sh $ kubectl apply -f oraclerestdataservice_create.yaml @@ -664,15 +655,22 @@ Password: `.spec.ordsPassword` For more information about Database Actions, see: [Oracle Database Actions](https://docs.oracle.com/en/database/oracle/sql-developer-web/21.2/index.html). -### Application Express +### Delete ORDS +- To delete ORDS run the following command: + + kubectl delete oraclerestdataservice ords-sample + +- You cannot delete referred Single Instance Database (SIDB) before deleting its ORDS resource. + +## APEX Enabled Database -Oracle APEX (previously known as Oracle Application Express) is a low-code development platform that enables developers to build scalable, secure enterprise apps, with world-class features that can be deployed anywhere. +Oracle APEX is a low-code development platform that enables developers to build scalable, secure enterprise apps, with world-class features that can be deployed anywhere. Using APEX, developers can quickly develop and deploy compelling apps that solve real problems and provide immediate value. Developers won't need to be an expert in a vast array of technologies to deliver sophisticated solutions. Focus on solving the problem and let APEX take care of the rest. To access APEX, You need to configure APEX with the ORDS. The following section will explain configuring APEX with ORDS in details: -#### Configure APEX with ORDS +#### Provision APEX with ORDS * For quick provisioning, use the sample **[config/samples/sidb/oraclerestdataservice_apex.yaml](../../confi/samples/sidb/oraclerestdataservice_apex.yaml)** file. For example: From 210f924731d44f9fdbb8833d824abd5d29723765 Mon Sep 17 00:00:00 2001 From: Mohammed Yunus Date: Wed, 1 Jun 2022 17:23:08 +0000 Subject: [PATCH 420/628] Mqureshi readme fixes 2 --- README.md | 1 + docs/sidb/README.md | 20 ++++++++++---------- 2 files changed, 11 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index b29c7906..8180d2da 100644 --- a/README.md +++ b/README.md @@ -101,6 +101,7 @@ YAML file templates are available under [`/config/samples`](./config/samples/). ```sh kubectl delete singleinstancedatabase.database.oracle.com --all -n + kubectl delete oraclerestdataservices.database.oracle.com --all -n kubectl delete shardingdatabase.database.oracle.com --all -n kubectl delete autonomousdatabase.database.oracle.com --all -n diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 349be81f..f3ff3d1a 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -147,16 +147,16 @@ To provision a new database instance on the Kubernetes cluster, use the sample * - For ease of use, the storage class **oci-bv** is specified in the **[singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml)**. This storage class facilitates dynamic provisioning of the OCI block volumes on the Oracle OKE for persistent storage of the database. The supported access mode for this class is `ReadWriteOnce`. For other cloud providers, you can similarly use their dynamic provisioning storage classes. - It is beneficial to have the database replica pods more than or equal to the number of available nodes if `ReadWriteMany` access mode is used with the OCI NFS volume. By doing so, the pods get distributed on different nodes and the database image is downloaded on all those nodes. This helps in reducing time for the database fail-over if the active database pod dies. - Supports Oracle Database Enterprise Edition (19.3.0), and later releases. -- To pull the database image faster from the container registry in order to bring up the SIDB instance quickly, you can use container-registry mirror of the corresponding cluster's region. For example, if the cluster exists in Mumbai region, you can use `container-registry-bom.oracle.com` mirror. For more information on container-registry mirrors, please follow the link [https://blogs.oracle.com/wim/post/oracle-container-registry-mirrors-in-oracle-cloud-infrastructure](https://blogs.oracle.com/wim/post/oracle-container-registry-mirrors-in-oracle-cloud-infrastructure). -- To update the init parameters like `sgaTarget` and `pgaAggregateTarget`, please refer the `initParams` section of the [singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml) file. +- To pull the database image faster from the container registry in order to bring up the SIDB instance quickly, you can use container-registry mirror of the corresponding cluster's region. For example, if the cluster exists in Mumbai region, you can use `container-registry-bom.oracle.com` mirror. For more information on container-registry mirrors, follow the link [https://blogs.oracle.com/wim/post/oracle-container-registry-mirrors-in-oracle-cloud-infrastructure](https://blogs.oracle.com/wim/post/oracle-container-registry-mirrors-in-oracle-cloud-infrastructure). +- To update the init parameters like `sgaTarget` and `pgaAggregateTarget`, refer the `initParams` section of the [singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml) file. ### Database Persistence -The database persistence can be achieved in the following two ways: +Database persistence can be achieved in the following two ways: - Dynamic Persistence Provisioning - Static Persistence Provisioning In **Dynamic Persistence Provisioning**, a persistent volume is provisioned by mentioning a storage class. For example, **oci-bv** storage class is specified in the **[singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml)** file. This storage class facilitates dynamic provisioning of the OCI block volumes. The supported access mode for this class is `ReadWriteOnce`. For other cloud providers, you can similarly use their dynamic provisioning storage classes. -**Note:** Generally, the `Reclaim Policy` of such dynamically provisioned volumes is `Delete`, hence, these volumes get deleted when their corresponding database deployment is deleted. To retain volumes, please use static provisioning as explained in the section below. +**Note:** Generally, the `Reclaim Policy` of such dynamically provisioned volumes is `Delete`, hence, these volumes get deleted when their corresponding database deployment is deleted. To retain volumes, use static provisioning as explained in the section below. In **Static Persistence Provisioning**, you have to create a volume manually (either using Block Volume or NFS), and then use the name of this volume with the `<.spec.persistence.volumeName>` field (corresponds to the `volumeName` field of the persistence section in the **[singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml)**). The `Reclaim Policy` of such volume can be set to `Retain`. So, this volume does not get deleted with the deletion of its corresponding deployment. The access modes supported with block volume and NFS are `ReadWriteOnce` and `ReadWriteMany` respectively. @@ -180,8 +180,8 @@ spec: volumeHandle: ``` -**Note:** OCI block volumes are AD (Availability Domain) specific. Please make sure that the database is deployed in the same AD as that of its statically provisioned block volume. This is handled automatically in dynamic provisioning. -To provision the database in a specific AD, please uncomment the following line from the **[singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml)** file: +**Note:** OCI block volumes are AD (Availability Domain) specific. Make sure that the database is deployed in the same AD as that of its statically provisioned block volume. This is handled automatically in dynamic provisioning. +To provision the database in a specific AD, uncomment the following line from the **[singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml)** file: ```yaml nodeSelector: @@ -208,7 +208,7 @@ spec: volumeHandle: ":/" ``` -**Note:** Whenever a mount target is provisioned in OCI, its `Reported Size (GiB)` values are very large. This is visible on the mount target page when logged in to the OCI console. Some applications will fail to install if the results of a space requirements check show too much available disk space. So please specify, in gibibytes (GiB), the maximum capacity reported by file systems exported through this mount target. This setting does not limit the actual amount of data you can store. +**Note:** Whenever a mount target is provisioned in OCI, its `Reported Size (GiB)` values are very large. This is visible on the mount target page when logged in to the OCI console. Some applications will fail to install if the results of a space requirements check show too much available disk space. So specify, in gibibytes (GiB), the maximum capacity reported by file systems exported through this mount target. This setting does not limit the actual amount of data you can store. ## Provision Pre-built Database @@ -221,7 +221,7 @@ $ kubectl apply -f singleinstancedatabase_prebuiltdb.yaml This pre-built image includes the data files of the database inside the image itself. As a result, the database startup time of the container is reduced, down to a couple of seconds. The pre-built database image can be very useful in contiguous integration/continuous delivery (CI/CD) scenarios, in which databases are used for conducting tests or experiments, and the workflow is simple. -To build the pre-built database image for the Enterprise/Standard edition, please follow these instructions: [Pre-built Database (prebuiltdb) Extension](https://github.com/oracle/docker-images/blob/main/OracleDatabase/SingleInstance/extensions/prebuiltdb/README.md). +To build the pre-built database image for the Enterprise/Standard edition, follow these instructions: [Pre-built Database (prebuiltdb) Extension](https://github.com/oracle/docker-images/blob/main/OracleDatabase/SingleInstance/extensions/prebuiltdb/README.md). ## Provision XE Database To provision new Oracle Database Express Edition (XE) database, use the sample **[config/samples/sidb/singleinstancedatabase_express.yaml](../../config/samples/sidb/singleinstancedatabase_express.yaml)** file. For example: @@ -276,7 +276,7 @@ Version 21.3.0.0.0 SQL> ``` -**Note:** The `<.spec.adminPassword>` above refers to the database password for SYS, SYSTEM and PDBADMIN users, which in turn represented by `spec` section's `adminPassword` field of the **[/config/samples/sidb/singleinstancedatabase.yaml](../config/samples/sidb/../../../../config/samples/sidb/singleinstancedatabase.yaml)** file. +**Note:** The `<.spec.adminPassword>` above refers to the database password for SYS, SYSTEM and PDBADMIN users, which in turn represented by `spec` section's `adminPassword` field of the **[config/samples/sidb/singleinstancedatabase.yaml](../config/samples/sidb/../../../../config/samples/sidb/singleinstancedatabase.yaml)** file. The Oracle Database inside the container also has Oracle Enterprise Manager Express (OEM Express) configured. To access OEM Express, start the browser, and paste in a URL similar to the following example: @@ -333,7 +333,7 @@ The value for the initialization parameter `sgaTarget` that you provide should b ### Multiple Replicas -In multiple replicas mode, more than one pod is created for the database. The database is open and mounted by one of the replica pods. Other replica pods have instances started but not mounted, and serve to provide a quick cold fail-over in case the active pod goes down. Multiple replicas are also helpful in [patching](#patch-existing-database) operation. Please ensure that you have multiple replicas of the database pods running before you start the patching operation for minimum downtime. +In multiple replicas mode, more than one pod is created for the database. The database is open and mounted by one of the replica pods. Other replica pods have instances started but not mounted, and serve to provide a quick cold fail-over in case the active pod goes down. Multiple replicas are also helpful in [patching](#patch-existing-database) operation. Ensure that you have multiple replicas of the database pods running before you start the patching operation for minimum downtime. To enable multiple replicas, update the replica attribute in the `.yaml`, and apply by using the `kubectl apply` or `kubectl scale` commands. From 9f323164673d3ecc140c3c4cf12580bbcc0b0344 Mon Sep 17 00:00:00 2001 From: douglas_williams Date: Wed, 1 Jun 2022 23:00:41 +0000 Subject: [PATCH 421/628] Minor style changes doc --- docs/sidb/README.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index f3ff3d1a..f19d3184 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -119,9 +119,9 @@ This command creates a secret named `admin-secret`, with the key `oracle_pwd` ma ## Provision New Database -To provision a new database instance on the Kubernetes cluster, use the sample **[config/samples/sidb/singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml)**. +To provision a new database instance on the Kubernetes cluster, use the example **[config/samples/sidb/singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml)**. -1. Log into [Oracle Container Registry](https://container-registry.oracle.com/) and accept the license agreement for the Database image, ignore if you have accepted already. +1. Log into [Oracle Container Registry](https://container-registry.oracle.com/) and accept the license agreement for the Database image; ignore if you have accepted the license agreement already. 2. If you have not already done so, create an image pull secret for the Oracle Container Registry: @@ -147,7 +147,7 @@ To provision a new database instance on the Kubernetes cluster, use the sample * - For ease of use, the storage class **oci-bv** is specified in the **[singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml)**. This storage class facilitates dynamic provisioning of the OCI block volumes on the Oracle OKE for persistent storage of the database. The supported access mode for this class is `ReadWriteOnce`. For other cloud providers, you can similarly use their dynamic provisioning storage classes. - It is beneficial to have the database replica pods more than or equal to the number of available nodes if `ReadWriteMany` access mode is used with the OCI NFS volume. By doing so, the pods get distributed on different nodes and the database image is downloaded on all those nodes. This helps in reducing time for the database fail-over if the active database pod dies. - Supports Oracle Database Enterprise Edition (19.3.0), and later releases. -- To pull the database image faster from the container registry in order to bring up the SIDB instance quickly, you can use container-registry mirror of the corresponding cluster's region. For example, if the cluster exists in Mumbai region, you can use `container-registry-bom.oracle.com` mirror. For more information on container-registry mirrors, follow the link [https://blogs.oracle.com/wim/post/oracle-container-registry-mirrors-in-oracle-cloud-infrastructure](https://blogs.oracle.com/wim/post/oracle-container-registry-mirrors-in-oracle-cloud-infrastructure). +- To pull the database image faster from the container registry, so that you can bring up the SIDB instance quickly, you can use the container-registry mirror of the corresponding cluster's region. For example, if the cluster exists in Mumbai region, then you can use the `container-registry-bom.oracle.com` mirror. For more information on container-registry mirrors, follow the link [https://blogs.oracle.com/wim/post/oracle-container-registry-mirrors-in-oracle-cloud-infrastructure](https://blogs.oracle.com/wim/post/oracle-container-registry-mirrors-in-oracle-cloud-infrastructure). - To update the init parameters like `sgaTarget` and `pgaAggregateTarget`, refer the `initParams` section of the [singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml) file. ### Database Persistence @@ -156,14 +156,14 @@ Database persistence can be achieved in the following two ways: - Static Persistence Provisioning In **Dynamic Persistence Provisioning**, a persistent volume is provisioned by mentioning a storage class. For example, **oci-bv** storage class is specified in the **[singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml)** file. This storage class facilitates dynamic provisioning of the OCI block volumes. The supported access mode for this class is `ReadWriteOnce`. For other cloud providers, you can similarly use their dynamic provisioning storage classes. -**Note:** Generally, the `Reclaim Policy` of such dynamically provisioned volumes is `Delete`, hence, these volumes get deleted when their corresponding database deployment is deleted. To retain volumes, use static provisioning as explained in the section below. +**Note:** Generally, the `Reclaim Policy` of such dynamically provisioned volumes is `Delete`. These volumes are deleted when their corresponding database deployment is deleted. To retain volumes, use static provisioning, as explained in the Block Volume Static Provisioning section. In **Static Persistence Provisioning**, you have to create a volume manually (either using Block Volume or NFS), and then use the name of this volume with the `<.spec.persistence.volumeName>` field (corresponds to the `volumeName` field of the persistence section in the **[singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml)**). The `Reclaim Policy` of such volume can be set to `Retain`. So, this volume does not get deleted with the deletion of its corresponding deployment. The access modes supported with block volume and NFS are `ReadWriteOnce` and `ReadWriteMany` respectively. Static Persistence Provisioning in OCI explained in the following subsections: #### Block Volume Static Provisioning -You have to manually create a block volume resource from the OCI console, and fetch its `OCID`. Further, you can use the following YAML file to create the persistent volume: +With block volume static provisioning, you must manually create a block volume resource from the OCI console, and fetch its `OCID`. To create the persistent volume, you can use the following YAML file: ```yaml apiVersion: v1 kind: PersistentVolume @@ -180,7 +180,7 @@ spec: volumeHandle: ``` -**Note:** OCI block volumes are AD (Availability Domain) specific. Make sure that the database is deployed in the same AD as that of its statically provisioned block volume. This is handled automatically in dynamic provisioning. +**Note:** OCI block volumes are AD (Availability Domain) specific. Ensure that the database is deployed in the same AD as that of its statically provisioned block volume. In dynamic provisioning, this is done automatically. To provision the database in a specific AD, uncomment the following line from the **[singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml)** file: ```yaml From 5415baf7ce3f018732ee876d374d49200335646a Mon Sep 17 00:00:00 2001 From: psaini Date: Thu, 2 Jun 2022 07:28:47 +0000 Subject: [PATCH 422/628] Moved dbcs sample file in dbcs --- config/samples/database_v1alpha1_dbcssystem.yaml | 7 ------- config/samples/kustomization.yaml | 2 +- 2 files changed, 1 insertion(+), 8 deletions(-) delete mode 100644 config/samples/database_v1alpha1_dbcssystem.yaml diff --git a/config/samples/database_v1alpha1_dbcssystem.yaml b/config/samples/database_v1alpha1_dbcssystem.yaml deleted file mode 100644 index 0ca38ddf..00000000 --- a/config/samples/database_v1alpha1_dbcssystem.yaml +++ /dev/null @@ -1,7 +0,0 @@ -apiVersion: database.oracle.com/v1alpha1 -kind: DbcsSystem -metadata: - name: dbcssystem-sample -spec: - # Add fields here - foo: bar diff --git a/config/samples/kustomization.yaml b/config/samples/kustomization.yaml index d3d73391..fd4781bf 100644 --- a/config/samples/kustomization.yaml +++ b/config/samples/kustomization.yaml @@ -15,5 +15,5 @@ resources: - sidb/singleinstancedatabase.yaml - sharding/shardingdatabase.yaml - sharding/sharding_v1alpha1_provshard.yaml -- database_v1alpha1_dbcssystem.yaml + - dbcs/database_v1alpha1_dbcssystem.yaml # +kubebuilder:scaffold:manifestskustomizesamples From ec3c6aa7e47fbaa0cf45f111d0104ed754d06d5c Mon Sep 17 00:00:00 2001 From: psaini Date: Thu, 2 Jun 2022 07:28:58 +0000 Subject: [PATCH 423/628] Moved dbcs sample file in dbcs --- config/samples/dbcs/database_v1alpha1_dbcssystem.yaml | 7 +++++++ 1 file changed, 7 insertions(+) create mode 100644 config/samples/dbcs/database_v1alpha1_dbcssystem.yaml diff --git a/config/samples/dbcs/database_v1alpha1_dbcssystem.yaml b/config/samples/dbcs/database_v1alpha1_dbcssystem.yaml new file mode 100644 index 00000000..0ca38ddf --- /dev/null +++ b/config/samples/dbcs/database_v1alpha1_dbcssystem.yaml @@ -0,0 +1,7 @@ +apiVersion: database.oracle.com/v1alpha1 +kind: DbcsSystem +metadata: + name: dbcssystem-sample +spec: + # Add fields here + foo: bar From c72f4cda5d837fb9eb40570b5c4c832ab0b316e4 Mon Sep 17 00:00:00 2001 From: Abhishek Kumar Date: Thu, 2 Jun 2022 11:53:47 +0000 Subject: [PATCH 424/628] Restructured SIDB readme --- PREREQUISITES.md | 2 +- ...SIDB_PREREQUISITES.md => PREREQUISITES.md} | 0 docs/sidb/README.md | 368 +++++++++++------- 3 files changed, 218 insertions(+), 152 deletions(-) rename docs/sidb/{SIDB_PREREQUISITES.md => PREREQUISITES.md} (100%) diff --git a/PREREQUISITES.md b/PREREQUISITES.md index e386c6bf..01bb94b4 100644 --- a/PREREQUISITES.md +++ b/PREREQUISITES.md @@ -27,7 +27,7 @@ If you intent to use `OraOperator` to handle Oracle Autonomous Database lifecycl ### Prerequites for Single Instance Databases (SIDB) -If you intent to use `OraOperator` to handle Oracle Database Single Instance lifecycles, then read [Single Instance Database Prerequisites](./docs/sidb/SIDB_PREREQUISITES.md) +If you intent to use `OraOperator` to handle Oracle Database Single Instance lifecycles, then read [Single Instance Database Prerequisites](./docs/sidb/PREREQUISITES.md) ### Prerequites for Sharded Databases (SHARDING) diff --git a/docs/sidb/SIDB_PREREQUISITES.md b/docs/sidb/PREREQUISITES.md similarity index 100% rename from docs/sidb/SIDB_PREREQUISITES.md rename to docs/sidb/PREREQUISITES.md diff --git a/docs/sidb/README.md b/docs/sidb/README.md index f19d3184..2e198088 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -3,26 +3,63 @@ Oracle Database Operator for Kubernetes (`OraOperator`) includes the Single Instance Database Controller, which enables provisioning, cloning, and patching of Oracle Single Instance Databases on Kubernetes. It also enables configuring the database for Oracle REST Data Services with Oracle APEX development platform. The following sections explain the setup and functionality of the operator * [Prerequisites](#prerequisites) - * [Kind SingleInstanceDatabase Resource](#kind-singleinstancedatabase-resource) - * [Provision New Database](#provision-new-database) - * [Provision Pre-built Database](#provision-pre-built-database) - * [Provision XE Database](#provision-xe-database) - * [Clone Existing Database](#clone-existing-database) - * [Patch/Rollback Database](#patchrollback-database) - * [Kind OracleRestDataService](#kind-oraclerestdataservice) - * [REST Enabled Database](#rest-enabled-database) - * [APEX Enabled Database](#apex-enabled-database) - * [Performing Maintenance Operations](#performing-maintenance-operations) + * [SingleInstanceDatabase Resource](#singleinstancedatabase-resource) + * [Resource Details](#resource-details) + * [Database List](#database-list) + * [Quick Status](#quick-status) + * [Detailed Status](#detailed-status) + * [Template YAML](#template-yaml) + * [New Database Provisioning](#new-database-provisioning) + * [Pre-built Database Provisioning](#pre-built-database-provisioning) + * [XE Database Provisioning](#xe-database-provisioning) + * [Database Connection](#database-connection) + * [Database Configuration](#database-configuration) + * [Database Persistence](#database-persistence) + * [Dynamic Persistence Provisioning](#dynamic-persistence-provisioning) + * [Static Persistence Provisioning](#static-persistence-provisioning) + * [Block Volume Static Provisioning](#block-volume-static-provisioning) + * [NFS Volume Static Provisioning](#nfs-volume-static-provisioning) + * [Database Modes: Flashback, Archivelog and Forcelog](#database-modes-flashback-archivelog-and-forcelog) + * [Initialization Parameters](#initialization-parameters) + * [Multiple Replicas](#multiple-replicas) + * [Load Balancer Service](#load-balancer-service) + * [Immutable YAML Attributes](#immutable-yaml-attributes) + * [Clone Existing Database](#clone-existing-database) + * [Patch/Rollback Database](#patchrollback-database) + * [Patch](#patch) + * [Patch after Cloning](#patch-after-cloning) + * [Datapatch Status](#datapatch-status) + * [Rollback](#rollback) + * [Deleting Database](#deleting-database) + * [OracleRestDataService Resource](#oraclerestdataservice-resource) + * [Resource details](#resource-details) + * [ORDS List](#ords-list) + * [Quick Status](#quick-status) + * [Detailed Status](#detailed-status) + * [Template YAML](#template-yaml) + * [REST Enable Database](#rest-enable-database) + * [ORDS Provisioning](#ords-provisioning) + * [Creation Status](#creation-status) + * [Database API](#database-api) + * [Examples](#examples) + * [REST Enabled SQL](#rest-enabled-sql) + * [Oracle Data Pump](#oracle-data-pump) + * [Database Actions](#database-actions) + * [APEX Installation](#apex-installation) + * [Deleting ORDS](#deleting-ords) + * [Maintenance Operations](#maintenance-operations) ## Prerequisites -Oracle strongly recommends that you follow the [Prerequisites](./SIDB_PREREQUISITES.md). +Oracle strongly recommends that you follow the [prerequisites](./PREREQUISITES.md). -## Kind SingleInstanceDatabase Resource +## SingleInstanceDatabase Resource -The Oracle Database Operator creates the `SingleInstanceDatabase` kind as a custom resource. Doing this enables Oracle Database to be managed as a native Kubernetes object as shown below. +The Oracle Database Operator creates the `SingleInstanceDatabase` as a custom resource. Doing this enables Oracle Database to be managed as a native Kubernetes object. We will refer `SingleInstanceDatabase` resource as Database from now onwards. -### List Databases +### Resource Details + +#### Database List To list databases, use the following command as an example, where the database names are `sidb-sample` and `sidb-sample-clone`, which are the names we will use as database names in command examples: ```sh @@ -33,7 +70,7 @@ $ kubectl get singleinstancedatabases -o name ``` -### Quick Status +#### Quick Status To obtain a quick database status, use the following command as an example: ```sh @@ -43,7 +80,7 @@ NAME EDITION STATUS VERSION CONNECT STR sidb-sample Enterprise Healthy 19.3.0.0.0 10.0.25.54:1521/ORCLCDB https://10.0.25.54:5500/em ``` -### Detailed Status +#### Detailed Status To obtain a detailed database status, use the following command as an example: ```sh @@ -60,7 +97,7 @@ $ kubectl describe singleinstancedatabase sidb-sample-clone Status: Cluster Connect String: sidb-sample-clone.default:1521/ORCL1C Conditions: - Last Transition Time: 2021-06-29T15:45:33Z + Last Transition Time: (YYYY-MM-DD)T(HH:MM:SS)Z Message: Waiting for database to be ready Observed Generation: 2 Reason: LastReconcileCycleQueued @@ -72,7 +109,7 @@ $ kubectl describe singleinstancedatabase sidb-sample-clone Reason: LastReconcileCycleBlocked Status: True Type: ReconcileBlocked - Last Transition Time: 2021-06-30T11:16:58Z + Last Transition Time: (YYYY-MM-DD)T(HH:MM:SS)Z Message: no reconcile errors Observed Generation: 3 Reason: LastReconcileCycleCompleted @@ -103,7 +140,7 @@ $ kubectl describe singleinstancedatabase sidb-sample-clone ``` -### SingleInstanceDatabase Template YAML +### Template YAML The template `.yaml` file for Single Instance Database (Enterprise and Standard Editions), including all the configurable options, is available at: **[config/samples/sidb/singleinstancedatabase.yaml](./../../config/samples/sidb/singleinstancedatabase.yaml)** @@ -117,7 +154,7 @@ Create this secret using the following command as an example: This command creates a secret named `admin-secret`, with the key `oracle_pwd` mapped to the actual password specified in the command. -## Provision New Database +### New Database Provisioning To provision a new database instance on the Kubernetes cluster, use the example **[config/samples/sidb/singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml)**. @@ -150,67 +187,7 @@ To provision a new database instance on the Kubernetes cluster, use the example - To pull the database image faster from the container registry, so that you can bring up the SIDB instance quickly, you can use the container-registry mirror of the corresponding cluster's region. For example, if the cluster exists in Mumbai region, then you can use the `container-registry-bom.oracle.com` mirror. For more information on container-registry mirrors, follow the link [https://blogs.oracle.com/wim/post/oracle-container-registry-mirrors-in-oracle-cloud-infrastructure](https://blogs.oracle.com/wim/post/oracle-container-registry-mirrors-in-oracle-cloud-infrastructure). - To update the init parameters like `sgaTarget` and `pgaAggregateTarget`, refer the `initParams` section of the [singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml) file. -### Database Persistence -Database persistence can be achieved in the following two ways: -- Dynamic Persistence Provisioning -- Static Persistence Provisioning - -In **Dynamic Persistence Provisioning**, a persistent volume is provisioned by mentioning a storage class. For example, **oci-bv** storage class is specified in the **[singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml)** file. This storage class facilitates dynamic provisioning of the OCI block volumes. The supported access mode for this class is `ReadWriteOnce`. For other cloud providers, you can similarly use their dynamic provisioning storage classes. -**Note:** Generally, the `Reclaim Policy` of such dynamically provisioned volumes is `Delete`. These volumes are deleted when their corresponding database deployment is deleted. To retain volumes, use static provisioning, as explained in the Block Volume Static Provisioning section. - -In **Static Persistence Provisioning**, you have to create a volume manually (either using Block Volume or NFS), and then use the name of this volume with the `<.spec.persistence.volumeName>` field (corresponds to the `volumeName` field of the persistence section in the **[singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml)**). The `Reclaim Policy` of such volume can be set to `Retain`. So, this volume does not get deleted with the deletion of its corresponding deployment. The access modes supported with block volume and NFS are `ReadWriteOnce` and `ReadWriteMany` respectively. - -Static Persistence Provisioning in OCI explained in the following subsections: - -#### Block Volume Static Provisioning -With block volume static provisioning, you must manually create a block volume resource from the OCI console, and fetch its `OCID`. To create the persistent volume, you can use the following YAML file: -```yaml -apiVersion: v1 -kind: PersistentVolume -metadata: - name: block-vol -spec: - capacity: - storage: 1024Gi - accessModes: - - ReadWriteOnce - persistentVolumeReclaimPolicy: Retain - csi: - driver: blockvolume.csi.oraclecloud.com - volumeHandle: -``` - -**Note:** OCI block volumes are AD (Availability Domain) specific. Ensure that the database is deployed in the same AD as that of its statically provisioned block volume. In dynamic provisioning, this is done automatically. -To provision the database in a specific AD, uncomment the following line from the **[singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml)** file: - -```yaml -nodeSelector: - topology.kubernetes.io/zone: PHX-AD-1 -``` - -#### NFS Volume Static Provisioning -Similar to the block volume static provisioning, you have to manually create a file system resource from the OCI console, and fetch its `OCID, Mount Target and Export Path`. Mention these values in the following YAML file to create the persistent volume: - -```yaml -apiVersion: v1 -kind: PersistentVolume -metadata: - name: nfs-vol -spec: - capacity: - storage: 1024Gi - volumeMode: Filesystem - accessModes: - - ReadWriteMany - persistentVolumeReclaimPolicy: Retain - csi: - driver: fss.csi.oraclecloud.com - volumeHandle: ":/" -``` - -**Note:** Whenever a mount target is provisioned in OCI, its `Reported Size (GiB)` values are very large. This is visible on the mount target page when logged in to the OCI console. Some applications will fail to install if the results of a space requirements check show too much available disk space. So specify, in gibibytes (GiB), the maximum capacity reported by file systems exported through this mount target. This setting does not limit the actual amount of data you can store. - -## Provision Pre-built Database +### Pre-built Database Provisioning To provision a new pre-built database instance, use the sample **[config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml](../../config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml)** file. For example: ```sh @@ -223,7 +200,7 @@ This pre-built image includes the data files of the database inside the image it To build the pre-built database image for the Enterprise/Standard edition, follow these instructions: [Pre-built Database (prebuiltdb) Extension](https://github.com/oracle/docker-images/blob/main/OracleDatabase/SingleInstance/extensions/prebuiltdb/README.md). -## Provision XE Database +### XE Database Provisioning To provision new Oracle Database Express Edition (XE) database, use the sample **[config/samples/sidb/singleinstancedatabase_express.yaml](../../config/samples/sidb/singleinstancedatabase_express.yaml)** file. For example: kubectl apply -f singleinstancedatabase_express.yaml @@ -235,8 +212,8 @@ This command pulls the XE image uploaded on the [Oracle Container Registry](http - For XE database, only single replica mode (i.e. `replicas: 1`) is supported. - For XE database, you **cannot change** the init parameters i.e. `cpuCount, processes, sgaTarget or pgaAggregateTarget`. -### Creation Status - +### Database Connection + Creating a new database instance takes a while. When the `status` column returns the response `Healthy`, the Database is open for connections. ```sh @@ -244,10 +221,8 @@ $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.status}" Healthy ``` - -### Connection Information -Clients can get the connect string to the CDB from `.status.connectString` and PDB from `.status.pdbConnectString`. For example: +Clients can get the connect-string to the CDB from `.status.connectString` and PDB from `.status.pdbConnectString`. For example: ```sh $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.connectString}" @@ -276,7 +251,7 @@ Version 21.3.0.0.0 SQL> ``` -**Note:** The `<.spec.adminPassword>` above refers to the database password for SYS, SYSTEM and PDBADMIN users, which in turn represented by `spec` section's `adminPassword` field of the **[config/samples/sidb/singleinstancedatabase.yaml](../config/samples/sidb/../../../../config/samples/sidb/singleinstancedatabase.yaml)** file. +**Note:** The `<.spec.adminPassword>` above refers to the database password for SYS, SYSTEM and PDBADMIN users, which in turn represented by `spec` section's `adminPassword` field of the **[config/samples/sidb/singleinstancedatabase.yaml](../config/samples/sidb/../../../../config/samples/sidb/singleinstancedatabase.yaml)** file. The Oracle Database inside the container also has Oracle Enterprise Manager Express (OEM Express) configured. To access OEM Express, start the browser, and paste in a URL similar to the following example: @@ -286,8 +261,73 @@ $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.oemExpres https://10.0.25.54:5500/em ``` -### Update Database Config - +### Database Configuration +The `OraOperator` facilitates you to configure the database. Various database configuration options are explained in the following subsections: + +#### Database Persistence +The database persistence can be achieved in the following two ways: +- Dynamic Persistence Provisioning +- Static Persistence Provisioning + +##### Dynamic Persistence Provisioning +In **Dynamic Persistence Provisioning**, a persistent volume is provisioned by mentioning a storage class. For example, **oci-bv** storage class is specified in the **[singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml)** file. This storage class facilitates dynamic provisioning of the OCI block volumes. The supported access mode for this class is `ReadWriteOnce`. For other cloud providers, you can similarly use their dynamic provisioning storage classes. + +**Note:** Generally, the `Reclaim Policy` of such dynamically provisioned volumes is `Delete`. These volumes are deleted when their corresponding database deployment is deleted. To retain volumes, use static provisioning, as explained in the Block Volume Static Provisioning section. + +##### Static Persistence Provisioning +In **Static Persistence Provisioning**, you have to create a volume manually (either using Block Volume or NFS), and then use the name of this volume with the `<.spec.persistence.volumeName>` field (corresponds to the `volumeName` field of the persistence section in the **[singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml)**). The `Reclaim Policy` of such volume can be set to `Retain`. So, this volume does not get deleted with the deletion of its corresponding deployment. The access modes supported with block volume and NFS are `ReadWriteOnce` and `ReadWriteMany` respectively. + +Static Persistence Provisioning in OCI is explained in the following subsections: + +###### Block Volume Static Provisioning +With block volume static provisioning, you must manually create a block volume resource from the OCI console, and fetch its `OCID`. To create the persistent volume, you can use the following YAML file: +```yaml +apiVersion: v1 +kind: PersistentVolume +metadata: + name: block-vol +spec: + capacity: + storage: 1024Gi + accessModes: + - ReadWriteOnce + persistentVolumeReclaimPolicy: Retain + csi: + driver: blockvolume.csi.oraclecloud.com + volumeHandle: +``` + +**Note:** OCI block volumes are AD (Availability Domain) specific. Ensure that the database is deployed in the same AD as that of its statically provisioned block volume. In dynamic provisioning, this is done automatically. +To provision the database in a specific AD, uncomment the following line from the **[singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml)** file: + +```yaml +nodeSelector: + topology.kubernetes.io/zone: PHX-AD-1 +``` + +###### NFS Volume Static Provisioning +Similar to the block volume static provisioning, you have to manually create a file system resource from the OCI console, and fetch its `OCID, Mount Target and Export Path`. Mention these values in the following YAML file to create the persistent volume: + +```yaml +apiVersion: v1 +kind: PersistentVolume +metadata: + name: nfs-vol +spec: + capacity: + storage: 1024Gi + volumeMode: Filesystem + accessModes: + - ReadWriteMany + persistentVolumeReclaimPolicy: Retain + csi: + driver: fss.csi.oraclecloud.com + volumeHandle: ":/" +``` + +**Note:** Whenever a mount target is provisioned in OCI, its `Reported Size (GiB)` values are very large. This is visible on the mount target page when logged in to the OCI console. Some applications will fail to install if the results of a space requirements check show too much available disk space. So specify, in gibibytes (GiB), the maximum capacity reported by file systems exported through this mount target. This setting does not limit the actual amount of data you can store. + +#### Database Modes: Flashback, Archivelog and Forcelog The following database parameters can be updated after the database is created: - `flashBack` @@ -306,9 +346,6 @@ $ kubectl patch singleinstancedatabase sidb-sample --type merge -p '{"spec":{"fo singleinstancedatabase.database.oracle.com/sidb-sample patched ``` - -#### Database Config Status - Check the Database Config Status by using the following command: ```sh @@ -317,7 +354,7 @@ $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath=[{.status.archiveL [true, true, true] ``` -### Update Initialization Parameters +#### Initialization Parameters The following database initialization parameters can be updated after the database is created: @@ -331,8 +368,8 @@ Change their attribute values and apply using `kubectl apply` or `kubectl edit/p **NOTE:** The value for the initialization parameter `sgaTarget` that you provide should be within the range set by [sga_min_size, sga_max_size]. If the value you provide is not in that range, then `sga_target` is not updated to the value you specify for `sgaTarget`. -### Multiple Replicas - +#### Multiple Replicas + In multiple replicas mode, more than one pod is created for the database. The database is open and mounted by one of the replica pods. Other replica pods have instances started but not mounted, and serve to provide a quick cold fail-over in case the active pod goes down. Multiple replicas are also helpful in [patching](#patch-existing-database) operation. Ensure that you have multiple replicas of the database pods running before you start the patching operation for minimum downtime. To enable multiple replicas, update the replica attribute in the `.yaml`, and apply by using the `kubectl apply` or `kubectl scale` commands. @@ -343,9 +380,22 @@ To enable multiple replicas, update the replica attribute in the `.yaml`, and ap - If the `ReadWriteOnce` access mode is used, all the replicas will be scheduled on the same node where the persistent volume would be mounted. - If the `ReadWriteMany` access mode is used, all the replicas will be distributed on different nodes. So, it is recommended to have replicas more than or equal to the number of the nodes as the database image is downloaded on all those nodes. This is beneficial in quick cold fail-over scenario (when the active pod dies) as the image would already be available on that node. -### Patch Attributes -The following attributes cannot be patched after creating the Single Instance Database instance: +#### Load Balancer Service + +For the Single Instance Database, the default service is the `NodePort` service. You can enable the `LoadBalancer` service by using `kubectl patch` command. + +For example: + +```sh +$ kubectl --type=merge -p '{"spec":{"loadBalancer": true}}' patch singleinstancedatabase sidb-sample + + singleinstancedatabase.database.oracle.com/sidb-sample patched +``` + +#### Immutable YAML Attributes + +The following attributes cannot be modified after creating the Single Instance Database instance: - `sid` - `edition` @@ -353,7 +403,7 @@ The following attributes cannot be patched after creating the Single Instance Da - `pdbName` - `cloneFrom` -If you attempt to patch one of these attributes, then you receive an error similar to the following: +If you attempt to changing one of these attributes, then you receive an error similar to the following: ```sh $ kubectl --type=merge -p '{"spec":{"sid":"ORCL1"}}' patch singleinstancedatabase sidb-sample @@ -361,19 +411,8 @@ $ kubectl --type=merge -p '{"spec":{"sid":"ORCL1"}}' patch singleinstancedatabas The SingleInstanceDatabase "sidb-sample" is invalid: spec.sid: Forbidden: cannot be changed ``` -#### Enable LoadBalancer Service - -For the Single Instance Database, the default service is the `NodePort` service. You can enable the `LoadBalancer` service by using `kubectl patch` command. - -For example: - -```sh -$ kubectl --type=merge -p '{"spec":{"loadBalancer": true}}' patch singleinstancedatabase sidb-sample - - singleinstancedatabase.database.oracle.com/sidb-sample patched -``` -## Clone Existing Database +### Clone Existing Database To create copies of your existing database quickly, you can use the cloning functionality. A cloned database is an exact, block-for-block copy of the source database. Cloning is much faster than creating a fresh database and copying over the data. @@ -392,13 +431,13 @@ $ kubectl apply -f singleinstancedatabase_clone.yaml **Note:** The clone database can specify a database image that is different from the source database. In such cases, cloning is supported only between databases of the same major release. -## Patch/Rollback Database +### Patch/Rollback Database Databases running in your cluster and managed by the Oracle Database operator can be patched or rolled back between release updates of the same major release. To patch databases, specify an image of the higher release update. To roll back databases, specify an image of the lower release update. Patched Oracle Docker images can be built by using this [patching extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/patching). -### Patch existing Database +#### Patch To patch an existing database, edit and apply the **[config/samples/sidb/singleinstancedatabase_patch.yaml](../../config/samples/sidb/singleinstancedatabase_patch.yaml)** file of the database resource/object either by specifying a new release update for image attributes, or by running the following command: @@ -411,11 +450,11 @@ singleinstancedatabase.database.oracle.com/sidb-sample patched After patching is complete, the database pods are restarted with the new release update image. For minimum downtime, ensure that you have multiple replicas of the database pods running before you start the patch operation. -### Clone and Patch Database +#### Patch after Cloning To clone and patch the database at the same time, clone your source database by using the [cloning existing database](#clone-existing-database) method, and specify a new release image for the cloned database. Use this method to ensure there are no patching related issues impacting your database performance or functionality. -### Datapatch Status +#### Datapatch Status Patching/Rollback operations are complete when the datapatch tool completes patching or rollback of the data files. Check the data files patching status and current release update version using the following commands @@ -429,12 +468,38 @@ $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.releaseUp 19.3.0.0.0 ``` - -## Kind OracleRestDataService -The Oracle Database Operator creates the `OracleRestDataService` (ORDS) kind as a custom resource. Creating ORDS as a custom resource enables the RESTful API access to the Oracle Database in K8s and enables it to be managed as a native Kubernetes object as shown below. +#### Rollback +You can roll back to a prior database version by specifying the old image in the `image` field of the **[config/samples/sidb/singleinstancedatabase_patch.yaml](../../config/samples/sidb/singleinstancedatabase_patch.yaml)** file, and applying it by the following command: + +```bash +kubectl apply -f singleinstancedatabase_patch.yaml +``` + +This can also be done using the following command: + +```sh +kubectl --type=merge -p '{"spec":{"image":{"pullFrom":"old-image:tag","pullSecrets":"pull-secret"}}}' patch singleinstancedatabase sidb-sample + +singleinstancedatabase.database.oracle.com/sidb-sample patched + +``` + +### Deleting Database +Please run the following command to delete the database: + +```bash +kubectl delete singleinstancedatabase.database.oracle.com sidb-sample +``` +The command above will delete the database pods and associated service. + +## OracleRestDataService Resource + +The Oracle Database Operator creates the `OracleRestDataService` as a custom resource. We will refer `OracleRestDataService` as ORDS from now onwards. Creating ORDS as a custom resource enables the RESTful API access to the Oracle Database in K8s and enables it to be managed as a native Kubernetes object. + +### Resource Details -### List OracleRestDataServices +#### ORDS List To list ORDS services, use the following command: ```sh @@ -444,7 +509,7 @@ $ kubectl get oraclerestdataservice -o name ``` -### Quick Status +#### Quick Status To obtain a quick status check of the ORDS service, use the following command: ```sh @@ -455,7 +520,7 @@ ords-sample Healthy sidb-sample https://10.0.25.54:8443/ords/ORCLPDB1 ``` -### Detailed Status +#### Detailed Status To obtain a detailed status check of the ORDS service, use the following command: ```sh @@ -489,7 +554,7 @@ $ kubectl describe oraclerestdataservice ords-sample ``` -### OracleRestDataService Template YAML +### Template YAML The template `.yaml` file for Oracle Rest Data Services (`OracleRestDataService` kind), including all the configurable options, is available at **[config/samples/sidb/oraclerestdataservice.yaml](config/samples/sidb/oraclerestdataservice.yaml)**. @@ -499,7 +564,9 @@ The template `.yaml` file for Oracle Rest Data Services (`OracleRestDataService` - By default, ORDS uses self-signed certificates. To use certificates from the Certificate Authority, the ORDS image needs to be rebuilt after specifying the values of `ssl.cert` and `ssl.cert.key` in the [standalone.properties](https://github.com/oracle/docker-images/blob/main/OracleRestDataServices/dockerfiles/standalone.properties.tmpl) file. After you rebuild the ORDS image, use the rebuilt image in the **[config/samples/sidb/oraclerestdataservice.yaml](config/samples/sidb/oraclerestdataservice.yaml)** file. - If you want to install ORDS in a [prebuilt database](#provision-a-pre-built-database), make sure to attach the **database persistence** by uncommenting the `persistence` section in the **[config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml](../../config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml)** file, while provisioning the prebuilt database. -## REST Enabled Database +### REST Enable Database + +#### ORDS Provisioning To quickly provision a new ORDS instance, use the sample **[config/samples/sidb/oraclerestdataservice_create.yaml](../../config/samples/sidb/oraclerestdataservice_create.yaml)** file. For example: @@ -510,7 +577,7 @@ $ kubectl apply -f oraclerestdataservice_create.yaml ``` After this command completes, ORDS is installed in the container database (CDB) of the Single Instance Database. -### Creation Status +#### Creation Status Creating a new ORDS instance takes a while. To check the status of the ORDS instance, use the following command: @@ -521,7 +588,7 @@ $ kubectl get oraclerestdataservice/ords-sample -o "jsonpath={.status.status}" ``` ORDS is open for connections when the `status` column returns `Healthy`. -### REST Endpoints +#### REST Endpoints Clients can access the REST Endpoints using `.status.databaseApiUrl` as shown in the following command. @@ -546,25 +613,25 @@ Use this ORDS user to authenticate the following: * Any Protected AutoRest Enabled Object APIs * Database Actions of any REST Enabled Schema -#### Database API Examples +##### Examples Some examples for the Database API usage are as follows: -* ##### Get all Database Components +- **Get all Database Components** ```sh curl -s -k -X GET -u 'ORDS_PUBLIC_USER:<.spec.ordsPassword>' https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/database/components/ | python -m json.tool ``` -* ##### Get all Database Users +- **Get all Database Users** ```sh curl -s -k -X GET -u 'ORDS_PUBLIC_USER:<.spec.ordsPassword>' https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/database/security/users/ | python -m json.tool ``` -* ##### Get all Tablespaces +- **Get all Tablespaces** ```sh curl -s -k -X GET -u 'ORDS_PUBLIC_USER:<.spec.ordsPassword>' https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/database/storage/tablespaces/ | python -m json.tool ``` -* ##### Get all Database Parameters +- **Get all Database Parameters** ```sh curl -s -k -X GET -u 'ORDS_PUBLIC_USER:<.spec.ordsPassword>' https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/database/parameters/ | python -m json.tool ``` -* ##### Get all Feature Usage Statistics +- **Get all Feature Usage Statistics** ```sh curl -s -k -X GET -u 'ORDS_PUBLIC_USER:<.spec.ordsPassword>' https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/database/feature_usage/ | python -m json.tool ``` @@ -624,7 +691,7 @@ The Oracle REST Data Services (ORDS) database API enables you to create Oracle D REST APIs for Oracle Data Pump Jobs can be found at [https://docs.oracle.com/en/database/oracle/oracle-database/21/dbrst/op-database-datapump-jobs-post.html](https://docs.oracle.com/en/database/oracle/oracle-database/21/dbrst/op-database-datapump-jobs-post.html). -### Database Actions +#### Database Actions Database Actions is a web-based interface that uses Oracle REST Data Services to provide development, data tools, administration and monitoring features for Oracle Database. @@ -655,28 +722,19 @@ Password: `.spec.ordsPassword` For more information about Database Actions, see: [Oracle Database Actions](https://docs.oracle.com/en/database/oracle/sql-developer-web/21.2/index.html). -### Delete ORDS -- To delete ORDS run the following command: - - kubectl delete oraclerestdataservice ords-sample - -- You cannot delete referred Single Instance Database (SIDB) before deleting its ORDS resource. - -## APEX Enabled Database +### APEX Installation Oracle APEX is a low-code development platform that enables developers to build scalable, secure enterprise apps, with world-class features that can be deployed anywhere. Using APEX, developers can quickly develop and deploy compelling apps that solve real problems and provide immediate value. Developers won't need to be an expert in a vast array of technologies to deliver sophisticated solutions. Focus on solving the problem and let APEX take care of the rest. -To access APEX, You need to configure APEX with the ORDS. The following section will explain configuring APEX with ORDS in details: - -#### Provision APEX with ORDS +The `OraOperator` facilitates installation of APEX in the database and also configures ORDS for it. The following section will explain installing APEX with configured ORDS: * For quick provisioning, use the sample **[config/samples/sidb/oraclerestdataservice_apex.yaml](../../confi/samples/sidb/oraclerestdataservice_apex.yaml)** file. For example: kubectl apply -f oraclerestdataservice_apex.yaml -* The APEX Password is used as a common password for APEX_PUBLIC_USER, APEX_REST_PUBLIC_USER, APEX_LISTENER and Apex administrator (username: ADMIN) mapped to secretKey +* The APEX Password is used as a common password for `APEX_PUBLIC_USER, APEX_REST_PUBLIC_USER, APEX_LISTENER` and Apex administrator (username: `ADMIN`) mapped to secretKey. * The status of ORDS turns to `Updating` during APEX configuration, and changes to `Healthy` after successful configuration. You can also check status by using the following command: @@ -706,15 +764,23 @@ password: `.spec.apexPassword` **NOTE:** - By default, the full development environment is initialized in APEX. After deployment, you can change it manually to the runtime environment. To change environments, run the script `apxdevrm.sql` after connecting to the primary database from the ORDS pod as the `SYS` user with `SYSDBA` privilege. For detailed instructions, see: [Converting a Full Development Environment to a Runtime Environment](https://docs.oracle.com/en/database/oracle/application-express/21.2/htmig/converting-between-runtime-and-full-development-environments.html#GUID-B0621B40-3441-44ED-9D86-29B058E26BE9). -## Performing Maintenance Operations -If you need to perform some maintenance operations manually, then the procedure is as follows: +### Deleting ORDS +- To delete ORDS run the following command: + + kubectl delete oraclerestdataservice ords-sample + +- You cannot delete the referred Database before deleting its ORDS resource. +- APEX, if installed, also gets uninstalled from the database when ORDS gets deleted. + + +## Maintenance Operations +If you need to perform some maintenance operations (Database/ORDS) manually, then the procedure is as follows: 1. Use `kubectl exec` to access the pod where you want to perform the manual operation, a command similar to the following: kubectl exec -it /bin/bash -2. The important locations, such as like ORACLE_HOME, ORDS_HOME, and so on, can be seen in the environment, by using the `env` command. +2. The important locations, such as ORACLE_HOME, ORDS_HOME, and so on, can be found in the environment, by using the `env` command. -3. Log In to `sqlplus` to perform manual operations by using the following command: - - - sqlplus / as sysdba +3. Log In to `sqlplus` to perform manual operations by using the following command: + + sqlplus / as sysdba From f1e49dbfc09d4f506482a80f889a05295a8d627b Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Thu, 2 Jun 2022 14:02:40 +0000 Subject: [PATCH 425/628] update docs --- README.md | 5 ++++- docs/acd/README.md | 6 ++++-- docs/adb/README.md | 30 ++++++++++++++++++++++++------ 3 files changed, 32 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index 8180d2da..1eb3770d 100644 --- a/README.md +++ b/README.md @@ -84,6 +84,7 @@ The quickstarts are designed for specific database configurations, including: * [Oracle Autonomous Container Database](./docs/acd/README.md) * [Oracle Autonomous Database](./docs/adb/README.md) +* [Oracle Autonomous Container Database](./docs/acd/README.md) * [Oracle Database Single Instance configuration](./docs/sidb/README.md) * [Oracle Database configured with Oracle Sharding](./docs/sharding/README.md) * [Oracle On-Premises Database](./docs/onpremdb/README.md) @@ -104,7 +105,9 @@ YAML file templates are available under [`/config/samples`](./config/samples/). kubectl delete oraclerestdataservices.database.oracle.com --all -n kubectl delete shardingdatabase.database.oracle.com --all -n kubectl delete autonomousdatabase.database.oracle.com --all -n - + kubectl delete autonomousdatabasebackup.database.oracle.com --all -n + kubectl delete autonomousdatabaserestore.database.oracle.com --all -n + kubectl delete autonomouscontainerdatabase.database.oracle.com --all -n ``` After all CRD instances are deleted, it is safe to remove the CRDs, APISerivces and operator deployment. diff --git a/docs/acd/README.md b/docs/acd/README.md index 08eb2954..ffd09116 100644 --- a/docs/acd/README.md +++ b/docs/acd/README.md @@ -1,5 +1,7 @@ # Managing Oracle Autonomous Container Databases on Dedicated Exadata Infrastructure +Oracle Database Operator for Kubernetes (`OraOperator`) includes the Oracle Autonomous Container Database Controller. Autonomous Container Database is one of the resources of Oracle Autonomous Database dedicated Exadata infrastructure feature. You can create multiple Autonomous Container Database resources in a single Autonomous Exadata VM Cluster resource, but you must create at least one before you can create any Autonomous Databases. + Before you use the Oracle Database Operator for Kubernetes (the operator), ensure your system meets all of the Oracle Autonomous Database (ADB) Prerequisites [ADB_PREREQUISITES](./../adb/ADB_PREREQUISITES.md). As indicated in the prerequisites (see above), to interact with OCI services, either the cluster has to be authorized using Principal Instance, or using the API Key Authentication by specifying the configMap and the secret under the `ociConfig` field. @@ -29,7 +31,7 @@ Follow the steps to provision an Autonomous Database that will map objects in yo 1. Get the `Compartment OCID`. - Login cloud console and click `Compartment`. + Login Cloud Console and click `Compartment`. ![compartment-1](/images/adb/compartment-1.png) @@ -39,7 +41,7 @@ Follow the steps to provision an Autonomous Database that will map objects in yo 2. Get the `AutonomousExadataVMCluster OCID`. - Login cloud console. Go to `Autonomous Database`, and click the `Autonomous Exadata VM Cluster` under the Dedicated Infrastructure. + Login Cloud Console. Go to `Autonomous Database`, and click the `Autonomous Exadata VM Cluster` under the Dedicated Infrastructure. ![aei-1](/images/adb/adb-id-1.png) diff --git a/docs/adb/README.md b/docs/adb/README.md index 08cc0b0a..d682250f 100644 --- a/docs/adb/README.md +++ b/docs/adb/README.md @@ -34,7 +34,7 @@ Follow the steps to provision an Autonomous Database that will map objects in yo 1. Get the `Compartment OCID`. - Login cloud console and click `Compartment`. + Login Cloud Console and click `Compartment`. ![compartment-1](/images/adb/compartment-1.png) @@ -42,7 +42,23 @@ Follow the steps to provision an Autonomous Database that will map objects in yo ![compartment-2](/images/adb/compartment-2.png) -2. Create a Kubernetes Secret to hold the password of the ADMIN user. +2. To create an Autonomous Database on Dedicated Exadata Infrastructure (ADB-D), the OCID of Oracle Autonomous Container Database is required. + + You can skip this step if you want to create a Autonomous Database on Shared Exadata Infrastructure (ADB-S). + + Go to the Cloud Console and click `Autonomous Database`. + + ![acd-id-1](/images/adb/adb-id-1.png) + + Under `Dedicated Infrastructure`, click `Autonomous Container Database`. + + ![acd-id-2](/images/adb/acd-id-1.png) + + Click on the name of Autonomous Container Database and copy the `Autonomous Container Database OCID` from Cloud Console. + + ![acd-id-3](/images/adb/acd-id-2.png) + +3. Create a Kubernetes Secret to hold the password of the ADMIN user. You can create this secret with the following command (as an example): @@ -50,7 +66,7 @@ Follow the steps to provision an Autonomous Database that will map objects in yo kubectl create secret generic admin-password --from-literal=admin-password='password_here' ``` -3. Add the following fields to the AutonomousDatabase resource definition. An example `.yaml` file is available here: [`config/samples/adb/autonomousdatabase_create.yaml`](./../../config/samples/adb/autonomousdatabase_create.yaml) +4. Add the following fields to the AutonomousDatabase resource definition. An example `.yaml` file is available here: [`config/samples/adb/autonomousdatabase_create.yaml`](./../../config/samples/adb/autonomousdatabase_create.yaml) | Attribute | Type | Description | Required? | |----|----|----|----| | `spec.details.compartmentOCID` | string | The [OCID](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm) of the compartment of the Autonomous Database. | Yes | @@ -62,7 +78,9 @@ Follow the steps to provision an Autonomous Database that will map objects in yo |`spec.details.adminPassword.ociSecret.ocid` | string | The **[OCID](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm)** of the [OCI Secret](https://docs.oracle.com/en-us/iaas/Content/KeyManagement/Tasks/managingsecrets.htm) where you want to hold the password for the ADMIN user. | Conditional | | `spec.details.dataStorageSizeInTBs` | int | The size, in terabytes, of the data volume that will be created and attached to the database. This storage can later be scaled up if needed. | Yes | | `spec.details.isAutoScalingEnabled` | boolean | Indicates if auto scaling is enabled for the Autonomous Database OCPU core count. The default value is `FALSE` | No | - | `spec.details.isDedicated` | boolean | True if the database is on dedicated [Exadata infrastructure](https://docs.cloud.oracle.com/Content/Database/Concepts/adbddoverview.htm) | No | + | `spec.details.isDedicated` | boolean | True if the database is on dedicated [Exadata infrastructure](https://docs.cloud.oracle.com/Content/Database/Concepts/adbddoverview.htm). `spec.details.autonomousContainerDatabase.k8sACD.name` or `spec.details.autonomousContainerDatabase.ociACD.ocid` has to be provided if the value is true. | No | + | `spec.details.autonomousContainerDatabase.k8sACD.name` | string | The **name** of the K8s Autonomous Container Database resource | No | + | `spec.details.autonomousContainerDatabase.ociACD.ocid` | string | The Autonomous Container Database [OCID](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm). | No | | `spec.details.freeformTags` | dictionary | Free-form tags for this resource. Each tag is a simple key-value pair with no predefined name, type, or namespace. For more information, see [Resource Tag](https://docs.cloud.oracle.com/Content/General/Concepts/resourcetags.htm).

Example:
`freeformTags:`
    `key1: value1`
    `key2: value2`| No | | `spec.details.dbWorkload` | string | The Oracle Autonomous Database workload type. The following values are valid:
- OLTP - indicates an Autonomous Transaction Processing database
- DW - indicates an Autonomous Data Warehouse database
- AJD - indicates an Autonomous JSON Database
- APEX - indicates an Autonomous Database with the Oracle APEX Application Development workload type. | No | | `spec.details.dbVersion` | string | A valid Oracle Database release for Oracle Autonomous Database. | No | @@ -91,11 +109,11 @@ Follow the steps to provision an Autonomous Database that will map objects in yo secretName: oci-privatekey ``` -4. Choose the type of network access (optional): +5. Choose the type of network access (optional): By default, the network access type is set to PUBLIC, which allows secure connections from anywhere. Uncomment the code block if you want configure the netowrk acess. See [Configuring Network Access of Autonomous Database](./NETWORK_ACCESS_OPTIONS.md) for more information. -5. Apply the yaml: +6. Apply the yaml: ```sh kubectl apply -f config/samples/adb/autonomousdatabase_create.yaml From 62d675f7a164118a6c581a9e4387117bb4fdfa92 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Thu, 2 Jun 2022 14:11:08 +0000 Subject: [PATCH 426/628] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 1eb3770d..dd79843d 100644 --- a/README.md +++ b/README.md @@ -82,7 +82,6 @@ For more details, see [Oracle Database Operator Installation Instrunctions](./do The quickstarts are designed for specific database configurations, including: -* [Oracle Autonomous Container Database](./docs/acd/README.md) * [Oracle Autonomous Database](./docs/adb/README.md) * [Oracle Autonomous Container Database](./docs/acd/README.md) * [Oracle Database Single Instance configuration](./docs/sidb/README.md) @@ -129,6 +128,7 @@ YAML file templates are available under [`/config/samples`](./config/samples/). ## Docs of the supported Oracle Database configurations * [Oracle Autonomous Database](https://docs.oracle.com/en-us/iaas/Content/Database/Concepts/adboverview.htm) +* [Components of Dedicated Autonomous Database](https://docs.oracle.com/en-us/iaas/autonomous-database/doc/components.html) * [Oracle Database Single Instance](https://docs.oracle.com/en/database/oracle/oracle-database/) * [Oracle Database Sharding](https://docs.oracle.com/en/database/oracle/oracle-database/21/shard/index.html) From 2e6b6712da7c82ebdfa7305775c3e86fa038a8dc Mon Sep 17 00:00:00 2001 From: kuassi_mensah Date: Thu, 2 Jun 2022 14:21:54 +0000 Subject: [PATCH 427/628] Update README.md --- README.md | 7 ------- 1 file changed, 7 deletions(-) diff --git a/README.md b/README.md index dd79843d..f41e20f9 100644 --- a/README.md +++ b/README.md @@ -117,13 +117,6 @@ YAML file templates are available under [`/config/samples`](./config/samples/). Note: If the CRD instances are not deleted, and the operator is deleted by using the preceding command, then operator deployment and instance objects (pods,services,PVCs, and so on) are deleted. However, the CRD deletion stops responding, because the CRD instances have finalizers that can only be removed by the operator pod, which is deleted when the APIServices are deleted. -* ### Retaining the CRDs and APIservices - - To delete the operator deployment and retain the CRDs, run the following commands: - - ```sh - kubectl delete deployment.apps/oracle-database-operator-controller-manager -n oracle-database-operator-system - ``` ## Docs of the supported Oracle Database configurations From fc135eb129f4e099730c122273069eb74595d5e7 Mon Sep 17 00:00:00 2001 From: Abhishek Kumar Date: Fri, 3 Jun 2022 10:57:00 +0000 Subject: [PATCH 428/628] Restructured SIDB readme --- docs/sidb/README.md | 174 ++++++++++++++++++++------------------------ 1 file changed, 79 insertions(+), 95 deletions(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 2e198088..33439d2d 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -4,49 +4,33 @@ Oracle Database Operator for Kubernetes (`OraOperator`) includes the Single Inst * [Prerequisites](#prerequisites) * [SingleInstanceDatabase Resource](#singleinstancedatabase-resource) - * [Resource Details](#resource-details) - * [Database List](#database-list) - * [Quick Status](#quick-status) - * [Detailed Status](#detailed-status) - * [Template YAML](#template-yaml) - * [New Database Provisioning](#new-database-provisioning) - * [Pre-built Database Provisioning](#pre-built-database-provisioning) - * [XE Database Provisioning](#xe-database-provisioning) - * [Database Connection](#database-connection) - * [Database Configuration](#database-configuration) - * [Database Persistence](#database-persistence) - * [Dynamic Persistence Provisioning](#dynamic-persistence-provisioning) - * [Static Persistence Provisioning](#static-persistence-provisioning) - * [Block Volume Static Provisioning](#block-volume-static-provisioning) - * [NFS Volume Static Provisioning](#nfs-volume-static-provisioning) - * [Database Modes: Flashback, Archivelog and Forcelog](#database-modes-flashback-archivelog-and-forcelog) - * [Initialization Parameters](#initialization-parameters) - * [Multiple Replicas](#multiple-replicas) - * [Load Balancer Service](#load-balancer-service) - * [Immutable YAML Attributes](#immutable-yaml-attributes) - * [Clone Existing Database](#clone-existing-database) - * [Patch/Rollback Database](#patchrollback-database) - * [Patch](#patch) - * [Patch after Cloning](#patch-after-cloning) - * [Datapatch Status](#datapatch-status) - * [Rollback](#rollback) - * [Deleting Database](#deleting-database) + * [Create a Database](#create-a-database) + * [New Database](#new-database) + * [Pre-built Database](#pre-built-database) + * [XE Database](#xe-database) + * [Connecting to Database](#connecting-to-database) + * [Database Persistence (Storage) Configuration Options](#database-persistence-storage-configuration-options) + * [Dynamic Persistence](#dynamic-persistence) + * [Static Persistence](#static-persistence) + * [Configuring a Database](#configuring-a-database) + * [Configure ArchiveLog, Flashback and ForceLog](#configure-archivelog-flashback-and-forcelog) + * [Change Init Parameters](#change-init-parameters) + * [Clone a Database](#clone-a-database) + * [Patch a Database](#patch-a-database) + * [Delete a Database](#delete-a-database) + * [Advanced Database Configurations](#advanced-database-configurations) + * [Run Database with Multiple Replicas](#run-database-with-multiple-replicas) + * [Setup Database with LoadBalancer](#setup-database-with-loadbalancer) * [OracleRestDataService Resource](#oraclerestdataservice-resource) - * [Resource details](#resource-details) - * [ORDS List](#ords-list) - * [Quick Status](#quick-status) - * [Detailed Status](#detailed-status) - * [Template YAML](#template-yaml) - * [REST Enable Database](#rest-enable-database) - * [ORDS Provisioning](#ords-provisioning) - * [Creation Status](#creation-status) + * [REST Enable a Database](#rest-enable-a-database) + * [Provision ORDS](#provision-ords) * [Database API](#database-api) - * [Examples](#examples) - * [REST Enabled SQL](#rest-enabled-sql) - * [Oracle Data Pump](#oracle-data-pump) - * [Database Actions](#database-actions) + * [Advanced Usages](#advanced-usages) + * [Oracle Data Pump](#oracle-data-pump) + * [REST Enabled SQL](#rest-enabled-sql) + * [Database Actions](#database-actions) * [APEX Installation](#apex-installation) - * [Deleting ORDS](#deleting-ords) + * [Delete ORDS](#delete-ords) * [Maintenance Operations](#maintenance-operations) ## Prerequisites @@ -154,7 +138,9 @@ Create this secret using the following command as an example: This command creates a secret named `admin-secret`, with the key `oracle_pwd` mapped to the actual password specified in the command. -### New Database Provisioning +### Create a Database + +#### New Database To provision a new database instance on the Kubernetes cluster, use the example **[config/samples/sidb/singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml)**. @@ -187,7 +173,7 @@ To provision a new database instance on the Kubernetes cluster, use the example - To pull the database image faster from the container registry, so that you can bring up the SIDB instance quickly, you can use the container-registry mirror of the corresponding cluster's region. For example, if the cluster exists in Mumbai region, then you can use the `container-registry-bom.oracle.com` mirror. For more information on container-registry mirrors, follow the link [https://blogs.oracle.com/wim/post/oracle-container-registry-mirrors-in-oracle-cloud-infrastructure](https://blogs.oracle.com/wim/post/oracle-container-registry-mirrors-in-oracle-cloud-infrastructure). - To update the init parameters like `sgaTarget` and `pgaAggregateTarget`, refer the `initParams` section of the [singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml) file. -### Pre-built Database Provisioning +#### Pre-built Database To provision a new pre-built database instance, use the sample **[config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml](../../config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml)** file. For example: ```sh @@ -196,11 +182,11 @@ $ kubectl apply -f singleinstancedatabase_prebuiltdb.yaml singleinstancedatabase.database.oracle.com/prebuiltdb-sample created ``` -This pre-built image includes the data files of the database inside the image itself. As a result, the database startup time of the container is reduced, down to a couple of seconds. The pre-built database image can be very useful in contiguous integration/continuous delivery (CI/CD) scenarios, in which databases are used for conducting tests or experiments, and the workflow is simple. +This pre-built image includes the data files of the database inside the image itself. As a result, the database startup time of the container is reduced, down to a couple of seconds. The pre-built database image can be very useful in continuous integration/continuous delivery (CI/CD) scenarios, in which databases are used for conducting tests or experiments, and the workflow is simple. To build the pre-built database image for the Enterprise/Standard edition, follow these instructions: [Pre-built Database (prebuiltdb) Extension](https://github.com/oracle/docker-images/blob/main/OracleDatabase/SingleInstance/extensions/prebuiltdb/README.md). -### XE Database Provisioning +#### XE Database To provision new Oracle Database Express Edition (XE) database, use the sample **[config/samples/sidb/singleinstancedatabase_express.yaml](../../config/samples/sidb/singleinstancedatabase_express.yaml)** file. For example: kubectl apply -f singleinstancedatabase_express.yaml @@ -212,7 +198,7 @@ This command pulls the XE image uploaded on the [Oracle Container Registry](http - For XE database, only single replica mode (i.e. `replicas: 1`) is supported. - For XE database, you **cannot change** the init parameters i.e. `cpuCount, processes, sgaTarget or pgaAggregateTarget`. -### Database Connection +### Connecting to Database Creating a new database instance takes a while. When the `status` column returns the response `Healthy`, the Database is open for connections. @@ -261,25 +247,22 @@ $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.oemExpres https://10.0.25.54:5500/em ``` -### Database Configuration -The `OraOperator` facilitates you to configure the database. Various database configuration options are explained in the following subsections: - -#### Database Persistence +### Database Persistence (Storage) Configuration Options The database persistence can be achieved in the following two ways: - Dynamic Persistence Provisioning - Static Persistence Provisioning -##### Dynamic Persistence Provisioning +#### Dynamic Persistence In **Dynamic Persistence Provisioning**, a persistent volume is provisioned by mentioning a storage class. For example, **oci-bv** storage class is specified in the **[singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml)** file. This storage class facilitates dynamic provisioning of the OCI block volumes. The supported access mode for this class is `ReadWriteOnce`. For other cloud providers, you can similarly use their dynamic provisioning storage classes. **Note:** Generally, the `Reclaim Policy` of such dynamically provisioned volumes is `Delete`. These volumes are deleted when their corresponding database deployment is deleted. To retain volumes, use static provisioning, as explained in the Block Volume Static Provisioning section. -##### Static Persistence Provisioning +#### Static Persistence In **Static Persistence Provisioning**, you have to create a volume manually (either using Block Volume or NFS), and then use the name of this volume with the `<.spec.persistence.volumeName>` field (corresponds to the `volumeName` field of the persistence section in the **[singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml)**). The `Reclaim Policy` of such volume can be set to `Retain`. So, this volume does not get deleted with the deletion of its corresponding deployment. The access modes supported with block volume and NFS are `ReadWriteOnce` and `ReadWriteMany` respectively. Static Persistence Provisioning in OCI is explained in the following subsections: -###### Block Volume Static Provisioning +##### Block Volume Static Provisioning With block volume static provisioning, you must manually create a block volume resource from the OCI console, and fetch its `OCID`. To create the persistent volume, you can use the following YAML file: ```yaml apiVersion: v1 @@ -305,7 +288,7 @@ nodeSelector: topology.kubernetes.io/zone: PHX-AD-1 ``` -###### NFS Volume Static Provisioning +##### NFS Volume Static Provisioning Similar to the block volume static provisioning, you have to manually create a file system resource from the OCI console, and fetch its `OCID, Mount Target and Export Path`. Mention these values in the following YAML file to create the persistent volume: ```yaml @@ -327,7 +310,10 @@ spec: **Note:** Whenever a mount target is provisioned in OCI, its `Reported Size (GiB)` values are very large. This is visible on the mount target page when logged in to the OCI console. Some applications will fail to install if the results of a space requirements check show too much available disk space. So specify, in gibibytes (GiB), the maximum capacity reported by file systems exported through this mount target. This setting does not limit the actual amount of data you can store. -#### Database Modes: Flashback, Archivelog and Forcelog +### Configuring a Database +The `OraOperator` facilitates you to configure the database. Various database configuration options are explained in the following subsections: + +#### Configure ArchiveLog, Flashback, and ForceLog The following database parameters can be updated after the database is created: - `flashBack` @@ -354,7 +340,7 @@ $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath=[{.status.archiveL [true, true, true] ``` -#### Initialization Parameters +#### Change Init Parameters The following database initialization parameters can be updated after the database is created: @@ -368,31 +354,6 @@ Change their attribute values and apply using `kubectl apply` or `kubectl edit/p **NOTE:** The value for the initialization parameter `sgaTarget` that you provide should be within the range set by [sga_min_size, sga_max_size]. If the value you provide is not in that range, then `sga_target` is not updated to the value you specify for `sgaTarget`. -#### Multiple Replicas - -In multiple replicas mode, more than one pod is created for the database. The database is open and mounted by one of the replica pods. Other replica pods have instances started but not mounted, and serve to provide a quick cold fail-over in case the active pod goes down. Multiple replicas are also helpful in [patching](#patch-existing-database) operation. Ensure that you have multiple replicas of the database pods running before you start the patching operation for minimum downtime. - -To enable multiple replicas, update the replica attribute in the `.yaml`, and apply by using the `kubectl apply` or `kubectl scale` commands. - -**Note:** -- This functionality requires the [k8s extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/k8s) extended images. The database image from the container registry `container-registry.oracle.com` includes the K8s extension. -- Because Oracle Database Express Edition (XE) does not support [k8s extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/k8s), it does not support multiple replicas. -- If the `ReadWriteOnce` access mode is used, all the replicas will be scheduled on the same node where the persistent volume would be mounted. -- If the `ReadWriteMany` access mode is used, all the replicas will be distributed on different nodes. So, it is recommended to have replicas more than or equal to the number of the nodes as the database image is downloaded on all those nodes. This is beneficial in quick cold fail-over scenario (when the active pod dies) as the image would already be available on that node. - - -#### Load Balancer Service - -For the Single Instance Database, the default service is the `NodePort` service. You can enable the `LoadBalancer` service by using `kubectl patch` command. - -For example: - -```sh -$ kubectl --type=merge -p '{"spec":{"loadBalancer": true}}' patch singleinstancedatabase sidb-sample - - singleinstancedatabase.database.oracle.com/sidb-sample patched -``` - #### Immutable YAML Attributes The following attributes cannot be modified after creating the Single Instance Database instance: @@ -411,8 +372,7 @@ $ kubectl --type=merge -p '{"spec":{"sid":"ORCL1"}}' patch singleinstancedatabas The SingleInstanceDatabase "sidb-sample" is invalid: spec.sid: Forbidden: cannot be changed ``` - -### Clone Existing Database +### Clone a Database To create copies of your existing database quickly, you can use the cloning functionality. A cloned database is an exact, block-for-block copy of the source database. Cloning is much faster than creating a fresh database and copying over the data. @@ -431,7 +391,7 @@ $ kubectl apply -f singleinstancedatabase_clone.yaml **Note:** The clone database can specify a database image that is different from the source database. In such cases, cloning is supported only between databases of the same major release. -### Patch/Rollback Database +### Patch a Database Databases running in your cluster and managed by the Oracle Database operator can be patched or rolled back between release updates of the same major release. To patch databases, specify an image of the higher release update. To roll back databases, specify an image of the lower release update. @@ -485,7 +445,7 @@ singleinstancedatabase.database.oracle.com/sidb-sample patched ``` -### Deleting Database +### Delete a Database Please run the following command to delete the database: ```bash @@ -493,6 +453,31 @@ kubectl delete singleinstancedatabase.database.oracle.com sidb-sample ``` The command above will delete the database pods and associated service. +### Advanced Database Configurations +Some advanced database configuration scenarios are as follows: + +#### Run Database with Multiple Replicas +In multiple replicas mode, more than one pod is created for the database. The database is open and mounted by one of the replica pods. Other replica pods have instances started but not mounted, and serve to provide a quick cold fail-over in case the active pod goes down. Multiple replicas are also helpful in [patching](#patch-existing-database) operation. Ensure that you have multiple replicas of the database pods running before you start the patching operation for minimum downtime. + +To enable multiple replicas, update the replica attribute in the `.yaml`, and apply by using the `kubectl apply` or `kubectl scale` commands. + +**Note:** +- This functionality requires the [k8s extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/k8s) extended images. The database image from the container registry `container-registry.oracle.com` includes the K8s extension. +- Because Oracle Database Express Edition (XE) does not support [k8s extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/k8s), it does not support multiple replicas. +- If the `ReadWriteOnce` access mode is used, all the replicas will be scheduled on the same node where the persistent volume would be mounted. +- If the `ReadWriteMany` access mode is used, all the replicas will be distributed on different nodes. So, it is recommended to have replicas more than or equal to the number of the nodes as the database image is downloaded on all those nodes. This is beneficial in quick cold fail-over scenario (when the active pod dies) as the image would already be available on that node. + +#### Setup Database with LoadBalancer +For the Single Instance Database, the default service is the `NodePort` service. You can enable the `LoadBalancer` service by using `kubectl patch` command. + +For example: + +```sh +$ kubectl --type=merge -p '{"spec":{"loadBalancer": true}}' patch singleinstancedatabase sidb-sample + + singleinstancedatabase.database.oracle.com/sidb-sample patched +``` + ## OracleRestDataService Resource The Oracle Database Operator creates the `OracleRestDataService` as a custom resource. We will refer `OracleRestDataService` as ORDS from now onwards. Creating ORDS as a custom resource enables the RESTful API access to the Oracle Database in K8s and enables it to be managed as a native Kubernetes object. @@ -564,9 +549,9 @@ The template `.yaml` file for Oracle Rest Data Services (`OracleRestDataService` - By default, ORDS uses self-signed certificates. To use certificates from the Certificate Authority, the ORDS image needs to be rebuilt after specifying the values of `ssl.cert` and `ssl.cert.key` in the [standalone.properties](https://github.com/oracle/docker-images/blob/main/OracleRestDataServices/dockerfiles/standalone.properties.tmpl) file. After you rebuild the ORDS image, use the rebuilt image in the **[config/samples/sidb/oraclerestdataservice.yaml](config/samples/sidb/oraclerestdataservice.yaml)** file. - If you want to install ORDS in a [prebuilt database](#provision-a-pre-built-database), make sure to attach the **database persistence** by uncommenting the `persistence` section in the **[config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml](../../config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml)** file, while provisioning the prebuilt database. -### REST Enable Database +### REST Enable a Database -#### ORDS Provisioning +#### Provision ORDS To quickly provision a new ORDS instance, use the sample **[config/samples/sidb/oraclerestdataservice_create.yaml](../../config/samples/sidb/oraclerestdataservice_create.yaml)** file. For example: @@ -635,8 +620,13 @@ Some examples for the Database API usage are as follows: ```sh curl -s -k -X GET -u 'ORDS_PUBLIC_USER:<.spec.ordsPassword>' https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/database/feature_usage/ | python -m json.tool ``` +#### Advanced Usages + +##### Oracle Data Pump +The Oracle REST Data Services (ORDS) database API enables you to create Oracle Data Pump export and import jobs by using REST web service calls. -#### REST Enabled SQL +REST APIs for Oracle Data Pump Jobs can be found at [https://docs.oracle.com/en/database/oracle/oracle-database/21/dbrst/op-database-datapump-jobs-post.html](https://docs.oracle.com/en/database/oracle/oracle-database/21/dbrst/op-database-datapump-jobs-post.html). +##### REST Enabled SQL The REST Enable SQL functionality is available to all the schemas specified in the `.spec.restEnableSchemas` attribute of the sample yaml. Only these schemas will have access SQL Developer Web Console specified by the Database Actions URL. @@ -685,13 +675,7 @@ Fetch all entries from 'DEPT' table by calling the following API **NOTE:** `.spec.restEnableSchema[].urlMapping` is optional and is defaulted to `.spec.restEnableSchemas[].schemaName` -#### Oracle Data Pump - -The Oracle REST Data Services (ORDS) database API enables you to create Oracle Data Pump export and import jobs by using REST web service calls. - -REST APIs for Oracle Data Pump Jobs can be found at [https://docs.oracle.com/en/database/oracle/oracle-database/21/dbrst/op-database-datapump-jobs-post.html](https://docs.oracle.com/en/database/oracle/oracle-database/21/dbrst/op-database-datapump-jobs-post.html). - -#### Database Actions +##### Database Actions Database Actions is a web-based interface that uses Oracle REST Data Services to provide development, data tools, administration and monitoring features for Oracle Database. @@ -764,7 +748,7 @@ password: `.spec.apexPassword` **NOTE:** - By default, the full development environment is initialized in APEX. After deployment, you can change it manually to the runtime environment. To change environments, run the script `apxdevrm.sql` after connecting to the primary database from the ORDS pod as the `SYS` user with `SYSDBA` privilege. For detailed instructions, see: [Converting a Full Development Environment to a Runtime Environment](https://docs.oracle.com/en/database/oracle/application-express/21.2/htmig/converting-between-runtime-and-full-development-environments.html#GUID-B0621B40-3441-44ED-9D86-29B058E26BE9). -### Deleting ORDS +### Delete ORDS - To delete ORDS run the following command: kubectl delete oraclerestdataservice ords-sample From 0463e7bf3056f375f9dc7fcdb9d87f42a800f90b Mon Sep 17 00:00:00 2001 From: psaini Date: Sun, 5 Jun 2022 02:25:06 +0000 Subject: [PATCH 429/628] Fixed main.go --- main.go | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/main.go b/main.go index 9bece017..865f7b39 100644 --- a/main.go +++ b/main.go @@ -153,6 +153,15 @@ func main() { setupLog.Error(err, "unable to create controller", "controller", "ShardingDatabase") os.Exit(1) } + if err = (&databasecontroller.DbcsSystemReconciler{ + Client: mgr.GetClient(), + Log: ctrl.Log.WithName("controllers").WithName("database").WithName("DbcsSystem"), + Scheme: mgr.GetScheme(), + Recorder: mgr.GetEventRecorderFor("DbcsSystem"), + }).SetupWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create controller", "controller", "DbcsSystem") + os.Exit(1) + } if err = (&databasecontroller.OracleRestDataServiceReconciler{ Client: mgr.GetClient(), Log: ctrl.Log.WithName("controllers").WithName("OracleRestDataService"), From 6a259eba6210dee06994ae49740b00bbc622e8f0 Mon Sep 17 00:00:00 2001 From: psaini Date: Sun, 5 Jun 2022 02:33:15 +0000 Subject: [PATCH 430/628] Fixed main.go --- main.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/main.go b/main.go index 865f7b39..8c872097 100644 --- a/main.go +++ b/main.go @@ -154,7 +154,7 @@ func main() { os.Exit(1) } if err = (&databasecontroller.DbcsSystemReconciler{ - Client: mgr.GetClient(), + KubeClient: mgr.GetClient(), Log: ctrl.Log.WithName("controllers").WithName("database").WithName("DbcsSystem"), Scheme: mgr.GetScheme(), Recorder: mgr.GetEventRecorderFor("DbcsSystem"), From 874893a9f77c17390da58b60521086da90e30c74 Mon Sep 17 00:00:00 2001 From: psaini Date: Sun, 5 Jun 2022 02:36:22 +0000 Subject: [PATCH 431/628] Fixed main.go --- main.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/main.go b/main.go index 8c872097..9c78c4d5 100644 --- a/main.go +++ b/main.go @@ -155,7 +155,7 @@ func main() { } if err = (&databasecontroller.DbcsSystemReconciler{ KubeClient: mgr.GetClient(), - Log: ctrl.Log.WithName("controllers").WithName("database").WithName("DbcsSystem"), + Logger: ctrl.Log.WithName("controllers").WithName("database").WithName("DbcsSystem"), Scheme: mgr.GetScheme(), Recorder: mgr.GetEventRecorderFor("DbcsSystem"), }).SetupWithManager(mgr); err != nil { From c2d464d6d3fb44da15b74fdc4adaadff7c06e09d Mon Sep 17 00:00:00 2001 From: Mohammed Yunus Date: Mon, 6 Jun 2022 08:28:25 +0000 Subject: [PATCH 432/628] Update README.md --- README.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index f41e20f9..4b1ea423 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # Oracle Database Operator for Kubernetes -## Make Oracle Database Kubernetes-Native +## Make Oracle Database Kubernetes Native - An Overview As part of Oracle's resolution to make Oracle Database Kubernetes-native (that is, observable and operable by Kubernetes), Oracle is announcing _Oracle Database Operator for Kubernetes_ (`OraOperator`). @@ -52,7 +52,7 @@ Oracle strongly recommends that you ensure your system meets the following [Prer ## Quick Install of the Operator - To install the operator in the cluster quickly, you can use a single [oracle-database-operator.yaml](https://github.com/oracle/oracle-database-operator/blob/main/oracle-database-operator.yaml) file. Operator pod replicas are set to a default of 3 for High Availability, which can be scaled up and down. + To install the operator in the cluster quickly, you can use a single [oracle-database-operator.yaml](https://github.com/oracle/oracle-database-operator/blob/main/oracle-database-operator.yaml) file. Run the following command @@ -60,7 +60,7 @@ Oracle strongly recommends that you ensure your system meets the following [Prer kubectl apply -f oracle-database-operator.yaml ``` - Ensure that operator pods are up and running + Ensure that operator pods are up and running. Operator pod replicas are set to a default of 3 for High Availability, which can be scaled up and down. ```sh $ kubectl get pods -n oracle-database-operator-system @@ -80,14 +80,14 @@ For more details, see [Oracle Database Operator Installation Instrunctions](./do ## Getting Started with the Operator (Quickstart) -The quickstarts are designed for specific database configurations, including: +The quickstarts are designed for specific database configurations: * [Oracle Autonomous Database](./docs/adb/README.md) * [Oracle Autonomous Container Database](./docs/acd/README.md) -* [Oracle Database Single Instance configuration](./docs/sidb/README.md) -* [Oracle Database configured with Oracle Sharding](./docs/sharding/README.md) +* [Containerized Oracle Single Instance Database](./docs/sidb/README.md) +* [Containerized Oracle Shared Database](./docs/sharding/README.md) * [Oracle On-Premises Database](./docs/onpremdb/README.md) -* [Oracle Database Cloud Services](./docs/dbcs/README.md) +* [Oracle Cloud Co-managed Database](./docs/dbcs/README.md) YAML file templates are available under [`/config/samples`](./config/samples/). You can copy and edit these template files to configure them for your use cases. @@ -100,8 +100,8 @@ YAML file templates are available under [`/config/samples`](./config/samples/). To delete all the CRD instances deployed to cluster by the operator, run the following commands, where is the namespace of the cluster object: ```sh - kubectl delete singleinstancedatabase.database.oracle.com --all -n kubectl delete oraclerestdataservices.database.oracle.com --all -n + kubectl delete singleinstancedatabase.database.oracle.com --all -n kubectl delete shardingdatabase.database.oracle.com --all -n kubectl delete autonomousdatabase.database.oracle.com --all -n kubectl delete autonomousdatabasebackup.database.oracle.com --all -n @@ -115,7 +115,7 @@ YAML file templates are available under [`/config/samples`](./config/samples/). kubectl delete -f oracle-database-operator.yaml --ignore-not-found=true ``` - Note: If the CRD instances are not deleted, and the operator is deleted by using the preceding command, then operator deployment and instance objects (pods,services,PVCs, and so on) are deleted. However, the CRD deletion stops responding, because the CRD instances have finalizers that can only be removed by the operator pod, which is deleted when the APIServices are deleted. + Note: If the CRD instances are not deleted, and the operator is deleted by using the preceding command, then operator deployment and instance objects (pods,services, PVCs, and so on) are deleted. However, the CRD deletion stops responding, because the CRD instances have properties that prevent its deletion and that can only be removed by the operator pod, which is deleted when the APIServices are deleted. ## Docs of the supported Oracle Database configurations From 4de7ae02427b363e6687bfba04fa700f1070bc36 Mon Sep 17 00:00:00 2001 From: psaini Date: Tue, 7 Jun 2022 08:41:52 +0000 Subject: [PATCH 433/628] Added fix for the object patching issues in DBCS controller --- commons/dbcssystem/dbcs_reconciler.go | 9 ++--- commons/dbcssystem/dcommon.go | 36 ++++++++++--------- controllers/database/dbcssystem_controller.go | 18 ++++++++-- main.go | 18 +++++----- oracle-database-operator.yaml | 4 +-- 5 files changed, 51 insertions(+), 34 deletions(-) diff --git a/commons/dbcssystem/dbcs_reconciler.go b/commons/dbcssystem/dbcs_reconciler.go index 61170ec1..f923ec86 100644 --- a/commons/dbcssystem/dbcs_reconciler.go +++ b/commons/dbcssystem/dbcs_reconciler.go @@ -476,7 +476,7 @@ func UpdateDbcsSystemId(kubeClient client.Client, dbcs *databasev1alpha1.DbcsSys payload := []annotations.PatchValue{{ Op: "replace", Path: "/spec/details", - Value: dbcs.Spec.DbSystem, + Value: dbcs.Spec, }} payloadBytes, err := json.Marshal(payload) if err != nil { @@ -741,9 +741,10 @@ func ValidateSpex(logger logr.Logger, kubeClient client.Client, dbClient databas } if dbcs.Spec.DbSystem.NodeCount != nil { - if *dbcs.Spec.DbSystem.NodeCount == 1 { - } else if *dbcs.Spec.DbSystem.NodeCount == 2 { - } else { + switch *dbcs.Spec.DbSystem.NodeCount { + case 1: + case 2: + default: eventMsg = "DBCS CRD resource " + "NodeCount " + GetFmtStr(dbcs.Name) + GetFmtStr("dbcs.Spec.DbSystem.NodeCount") + " can be either 1 or 2." eRecord.Eventf(dbcs, corev1.EventTypeWarning, eventErr, eventMsg) return err diff --git a/commons/dbcssystem/dcommon.go b/commons/dbcssystem/dcommon.go index 76ea74f4..9ba66079 100644 --- a/commons/dbcssystem/dcommon.go +++ b/commons/dbcssystem/dcommon.go @@ -255,17 +255,18 @@ func getRecoveryWindowsInDays(dbcs *databasev1alpha1.DbcsSystem) (int, error) { var days int - if *dbcs.Spec.DbSystem.DbBackupConfig.RecoveryWindowsInDays == 7 { + switch *dbcs.Spec.DbSystem.DbBackupConfig.RecoveryWindowsInDays { + case 7: return *dbcs.Spec.DbSystem.DbBackupConfig.RecoveryWindowsInDays, nil - } else if *dbcs.Spec.DbSystem.DbBackupConfig.RecoveryWindowsInDays == 15 { + case 15: return *dbcs.Spec.DbSystem.DbBackupConfig.RecoveryWindowsInDays, nil - } else if *dbcs.Spec.DbSystem.DbBackupConfig.RecoveryWindowsInDays == 30 { + case 30: return *dbcs.Spec.DbSystem.DbBackupConfig.RecoveryWindowsInDays, nil - } else if *dbcs.Spec.DbSystem.DbBackupConfig.RecoveryWindowsInDays == 45 { + case 45: return *dbcs.Spec.DbSystem.DbBackupConfig.RecoveryWindowsInDays, nil - } else if *dbcs.Spec.DbSystem.DbBackupConfig.RecoveryWindowsInDays == 60 { + case 60: return *dbcs.Spec.DbSystem.DbBackupConfig.RecoveryWindowsInDays, nil - } else { + default: days = 30 return days, nil } @@ -278,11 +279,12 @@ func GetDBSystemopts( dbSystemOpt := database.DbSystemOptions{} if dbcs.Spec.DbSystem.StorageManagement != "" { - if dbcs.Spec.DbSystem.StorageManagement == "LVM" { + switch dbcs.Spec.DbSystem.StorageManagement { + case "LVM": dbSystemOpt.StorageManagement = database.DbSystemOptionsStorageManagementLvm - } else if dbcs.Spec.DbSystem.StorageManagement == "ASM" { + case "ASM": dbSystemOpt.StorageManagement = database.DbSystemOptionsStorageManagementAsm - } else { + default: dbSystemOpt.StorageManagement = database.DbSystemOptionsStorageManagementAsm } } else { @@ -340,15 +342,16 @@ func GetDBEdition(dbcs *databasev1alpha1.DbcsSystem) database.LaunchDbSystemDeta } if dbcs.Spec.DbSystem.DbEdition != "" { - if dbcs.Spec.DbSystem.DbEdition == "STANDARD_EDITION" { + switch dbcs.Spec.DbSystem.DbEdition { + case "STANDARD_EDITION": return database.LaunchDbSystemDetailsDatabaseEditionStandardEdition - } else if dbcs.Spec.DbSystem.DbEdition == "ENTERPRISE_EDITION" { + case "ENTERPRISE_EDITION": return database.LaunchDbSystemDetailsDatabaseEditionEnterpriseEdition - } else if dbcs.Spec.DbSystem.DbEdition == "ENTERPRISE_EDITION_HIGH_PERFORMANCE" { + case "ENTERPRISE_EDITION_HIGH_PERFORMANCE": return database.LaunchDbSystemDetailsDatabaseEditionEnterpriseEditionHighPerformance - } else if dbcs.Spec.DbSystem.DbEdition == "ENTERPRISE_EDITION_EXTREME_PERFORMANCE" { + case "ENTERPRISE_EDITION_EXTREME_PERFORMANCE": return database.LaunchDbSystemDetailsDatabaseEditionEnterpriseEditionExtremePerformance - } else { + default: return database.LaunchDbSystemDetailsDatabaseEditionEnterpriseEdition } } @@ -363,9 +366,10 @@ func GetDBbDiskRedundancy( return database.LaunchDbSystemDetailsDiskRedundancyHigh } - if dbcs.Spec.DbSystem.DiskRedundancy == "HIGH" { + switch dbcs.Spec.DbSystem.DiskRedundancy { + case "HIGH": return database.LaunchDbSystemDetailsDiskRedundancyHigh - } else if dbcs.Spec.DbSystem.DiskRedundancy == "NORMAL" { + case "NORMAL": return database.LaunchDbSystemDetailsDiskRedundancyNormal } diff --git a/controllers/database/dbcssystem_controller.go b/controllers/database/dbcssystem_controller.go index 29030601..d279bf84 100644 --- a/controllers/database/dbcssystem_controller.go +++ b/controllers/database/dbcssystem_controller.go @@ -188,9 +188,10 @@ func (r *DbcsSystemReconciler) Reconcile(ctx context.Context, req ctrl.Request) return ctrl.Result{}, nil } - dbcsInst.Spec.Id = &dbcsID + assignDBCSID(dbcsInst, dbcsID) if err := dbcsv1.UpdateDbcsSystemId(r.KubeClient, dbcsInst); err != nil { // Change the status to Failed + assignDBCSID(dbcsInst, dbcsID) if statusErr := dbcsv1.SetLifecycleState(r.KubeClient, r.dbClient, dbcsInst, databasev1alpha1.Failed, r.nwClient, r.wrClient); statusErr != nil { return ctrl.Result{}, statusErr } @@ -198,10 +199,11 @@ func (r *DbcsSystemReconciler) Reconcile(ctx context.Context, req ctrl.Request) } r.Logger.Info("DbcsSystem system provisioned succesfully") - + assignDBCSID(dbcsInst, dbcsID) if err := dbcsInst.UpdateLastSuccessfulSpec(r.KubeClient); err != nil { return ctrl.Result{}, err } + assignDBCSID(dbcsInst, dbcsID) } else { if lastSucSpec == nil { if err := dbcsv1.GetDbSystemId(r.Logger, r.dbClient, dbcsInst); err != nil { @@ -216,8 +218,10 @@ func (r *DbcsSystemReconciler) Reconcile(ctx context.Context, req ctrl.Request) return ctrl.Result{}, err } + dbcsInstId := *dbcsInst.Spec.Id if err := dbcsv1.UpdateDbcsSystemId(r.KubeClient, dbcsInst); err != nil { // Change the status to Failed + assignDBCSID(dbcsInst, dbcsInstId) if statusErr := dbcsv1.SetLifecycleState(r.KubeClient, r.dbClient, dbcsInst, databasev1alpha1.Failed, r.nwClient, r.wrClient); statusErr != nil { return ctrl.Result{}, statusErr } @@ -226,9 +230,11 @@ func (r *DbcsSystemReconciler) Reconcile(ctx context.Context, req ctrl.Request) r.Logger.Info("Sync information from remote DbcsSystem System successfully") + dbcsInstId = *dbcsInst.Spec.Id if err := dbcsInst.UpdateLastSuccessfulSpec(r.KubeClient); err != nil { return ctrl.Result{}, err } + assignDBCSID(dbcsInst, dbcsInstId) } else { if dbcsInst.Spec.Id == nil { dbcsInst.Spec.Id = lastSucSpec.Id @@ -255,11 +261,13 @@ func (r *DbcsSystemReconciler) Reconcile(ctx context.Context, req ctrl.Request) //r.updateWalletSecret(dbcs) // Update the last succesful spec + dbcsInstId := *dbcsInst.Spec.Id if err := dbcsInst.UpdateLastSuccessfulSpec(r.KubeClient); err != nil { return ctrl.Result{}, err } - + //assignDBCSID(dbcsInst,dbcsI) // Change the phase to "Available" + assignDBCSID(dbcsInst, dbcsInstId) if statusErr := dbcsv1.SetLifecycleState(r.KubeClient, r.dbClient, dbcsInst, databasev1alpha1.Available, r.nwClient, r.wrClient); statusErr != nil { return ctrl.Result{}, statusErr } @@ -267,6 +275,10 @@ func (r *DbcsSystemReconciler) Reconcile(ctx context.Context, req ctrl.Request) return ctrl.Result{}, nil } +func assignDBCSID(dbcsInst *databasev1alpha1.DbcsSystem, dbcsID string) { + dbcsInst.Spec.Id = &dbcsID +} + func (r *DbcsSystemReconciler) eventFilterPredicate() predicate.Predicate { return predicate.Funcs{ CreateFunc: func(e event.CreateEvent) bool { diff --git a/main.go b/main.go index 9c78c4d5..56de731c 100644 --- a/main.go +++ b/main.go @@ -153,15 +153,15 @@ func main() { setupLog.Error(err, "unable to create controller", "controller", "ShardingDatabase") os.Exit(1) } - if err = (&databasecontroller.DbcsSystemReconciler{ - KubeClient: mgr.GetClient(), - Logger: ctrl.Log.WithName("controllers").WithName("database").WithName("DbcsSystem"), - Scheme: mgr.GetScheme(), - Recorder: mgr.GetEventRecorderFor("DbcsSystem"), - }).SetupWithManager(mgr); err != nil { - setupLog.Error(err, "unable to create controller", "controller", "DbcsSystem") - os.Exit(1) - } + if err = (&databasecontroller.DbcsSystemReconciler{ + KubeClient: mgr.GetClient(), + Logger: ctrl.Log.WithName("controllers").WithName("database").WithName("DbcsSystem"), + Scheme: mgr.GetScheme(), + Recorder: mgr.GetEventRecorderFor("DbcsSystem"), + }).SetupWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create controller", "controller", "DbcsSystem") + os.Exit(1) + } if err = (&databasecontroller.OracleRestDataServiceReconciler{ Client: mgr.GetClient(), Log: ctrl.Log.WithName("controllers").WithName("OracleRestDataService"), diff --git a/oracle-database-operator.yaml b/oracle-database-operator.yaml index 2c47c600..c6e88988 100644 --- a/oracle-database-operator.yaml +++ b/oracle-database-operator.yaml @@ -2768,7 +2768,7 @@ spec: - --enable-leader-election command: - /manager - image: container-registry.oracle.com/database/operator:0.1.0 + image: phx.ocir.io/intsanjaysingh/db-operator/sharding/dbcs-operator:0.2.0 imagePullPolicy: Always name: manager ports: @@ -3099,4 +3099,4 @@ webhooks: - DELETE resources: - singleinstancedatabases - sideEffects: None \ No newline at end of file + sideEffects: None From a8ebfebd46e5bd7c45069bf63ea778763e9d37c7 Mon Sep 17 00:00:00 2001 From: psaini Date: Wed, 8 Jun 2022 15:10:36 +0000 Subject: [PATCH 434/628] Fixed Documentation issue --- docs/dbcs/README.md | 2 +- .../terminate_dbcs_system_sample_output.log | 310 +----------------- 2 files changed, 2 insertions(+), 310 deletions(-) diff --git a/docs/dbcs/README.md b/docs/dbcs/README.md index 1adf0944..275db9c0 100644 --- a/docs/dbcs/README.md +++ b/docs/dbcs/README.md @@ -1,6 +1,6 @@ # Using the DB Operator DBCS Controller -Oracle Cloud Infastructure (OCI) Database Cloud Service (DBCS) provides single-node Database (DB) systems, deployed either bare metal or virtual machines, and provides two-node Oracle Real Appliation Clusters (Oracle RAC) database systems on virtual machines. +Oracle Cloud Infastructure (OCI) Database Cloud Service (DBCS) provides single-node Database (DB) systems, deployed on virtual machines, and provides two-node Oracle Real Appliation Clusters (Oracle RAC) database systems on virtual machines. The single-node DB systems and Oracle RAC systems on virtual machines are [co-managed Oracle Database cloud solutions](https://docs.oracle.com/en-us/iaas/Content/Database/Concepts/overview.htm). To manage the lifecycle of an OCI DBCS system, you can use the OCI Console, the REST API, or the Oracle Cloud Infrastructure command-line interface (CLI). At the granular level, you can use the Oracle Database CLI (DBCLI), Oracle Enterprise Manager, or Oracle SQL Developer. diff --git a/docs/dbcs/provisioning/terminate_dbcs_system_sample_output.log b/docs/dbcs/provisioning/terminate_dbcs_system_sample_output.log index 8f0d5a36..ff8afc96 100644 --- a/docs/dbcs/provisioning/terminate_dbcs_system_sample_output.log +++ b/docs/dbcs/provisioning/terminate_dbcs_system_sample_output.log @@ -234,314 +234,6 @@ dbcssystem.database.oracle.com "dbcssystem-terminate" deleted 2022-03-09T01:25:06.920Z INFO controller-runtime.manager.controller.dbcssystem Finalizer unregistered successfully. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-terminate", "namespace": "default"} - - -[root@docker-test-server test]# kubectl describe dbcssystems.database.oracle.com dbcssystem-terminate -Error from server (NotFound): dbcssystems.database.oracle.com "dbcssystem-terminate" not found -[root@docker-test-server test]# - - - -[root@docker-test-server test]# kubectl describe dbcssystems.database.oracle.com dbcssystem-existing -Name: dbcssystem-existing -Namespace: default -Labels: -Annotations: lastSuccessfulSpec: - {"dbSystem":{"compartmentId":"ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya","availabilityDomain":"O... -API Version: database.oracle.com/v1alpha1 -Kind: DbcsSystem -Metadata: - Creation Timestamp: 2022-03-08T23:27:48Z - Generation: 5 - Managed Fields: - API Version: database.oracle.com/v1alpha1 - Fields Type: FieldsV1 - fieldsV1: - f:metadata: - f:annotations: - f:lastSuccessfulSpec: - f:spec: - f:dbSystem: - f:cpuCoreCount: - f:dbBackupConfig: - f:dbEdition: - f:dbName: - f:dbUniqueName: - f:dbVersion: - f:diskRedundancy: - f:displayName: - f:faultDomains: - f:nodeCount: - f:status: - .: - f:availabilityDomain: - f:cpuCoreCount: - f:dataStoragePercentage: - f:dataStorageSizeInGBs: - f:dbEdition: - f:dbInfo: - f:displayName: - f:id: - f:licenseModel: - f:network: - .: - f:clientSubnet: - f:domainName: - f:hostName: - f:listenerPort: - f:scanDnsName: - f:vcnName: - f:nodeCount: - f:recoStorageSizeInGB: - f:shape: - f:state: - f:storageManagement: - f:subnetId: - f:timeZone: - f:workRequests: - Manager: manager - Operation: Update - Time: 2022-03-08T23:32:55Z - API Version: database.oracle.com/v1alpha1 - Fields Type: FieldsV1 - fieldsV1: - f:metadata: - f:annotations: - .: - f:kubectl.kubernetes.io/last-applied-configuration: - f:spec: - .: - f:dbSystem: - .: - f:availabilityDomain: - f:compartmentId: - f:dbAdminPaswordSecret: - f:hostName: - f:licenseModel: - f:shape: - f:sshPublicKeys: - f:subnetId: - f:id: - f:ociConfigMap: - f:ociSecret: - Manager: kubectl-client-side-apply - Operation: Update - Time: 2022-03-09T01:15:19Z - Resource Version: 55226409 - UID: 96d7bc49-33e9-42cc-8dd0-ada9a5a4c7e5 -Spec: - Db System: - Availability Domain: OLou:PHX-AD-1 - Compartment Id: ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya - Db Admin Pasword Secret: admin-password - Host Name: host0130 - License Model: BRING_YOUR_OWN_LICENSE - Shape: VM.Standard2.1 - Ssh Public Keys: - oci-publickey - Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a - Id: ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa - Oci Config Map: oci-cred - Oci Secret: oci-privatekey -Status: - Availability Domain: OLou:PHX-AD-1 - Cpu Core Count: 1 - Data Storage Percentage: 80 - Data Storage Size In G Bs: 512 - Db Edition: ENTERPRISE_EDITION - Db Info: - Db Home Id: ocid1.dbhome.oc1.phx.anyhqljr5gy3jhqat52milqwt3gq6lwohhacwg5yi4mtzq7c7hag53lrkugq - Db Name: db0130 - Db Unique Name: db0130_phx1zn - Db Workload: OLTP - Id: ocid1.database.oc1.phx.anyhqljrabf7htyackgmsaqjfexoqgrzuuk33ju2q25z2al43tnd5mhhvkra - Display Name: dbsystem20220308221302 - Id: ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa - License Model: BRING_YOUR_OWN_LICENSE - Network: - Client Subnet: k8test-pubvcn - Domain Name: k8testpubvcn.k8test.oraclevcn.com - Host Name: host0130 - Listener Port: 1521 - Scan Dns Name: host0130-scan.k8testpubvcn.k8test.oraclevcn.com - Vcn Name: k8test - Node Count: 1 - Reco Storage Size In GB: 256 - Shape: VM.Standard2.1 - State: AVAILABLE - Storage Management: ASM - Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a - Time Zone: UTC - Work Requests: - Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrxivzvgzel47zuoyke5yk36o7mrgjl27vscd5z3bqptmyh3rxwbqq - Operation Type: Create DB System - Percent Complete: 100 - Time Accepted: 2022-03-08 22:13:02.999 +0000 UTC - Time Finished: 2022-03-08 23:11:50.46 +0000 UTC - Time Started: 2022-03-08 22:13:16.995 +0000 UTC - Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrc3fx6kwq4yxerk3ngztdbbngm7w4dnlddcdhxqxjn6e4kcyux5ca - Operation Type: Update Shape - Percent Complete: 100 - Time Accepted: 2022-03-08 23:33:42.807 +0000 UTC - Time Finished: 2022-03-08 23:46:21.126 +0000 UTC - Time Started: 2022-03-08 23:33:52.109 +0000 UTC - Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljr5sveun3f6k3zuz23py7mm7jncmpq5vwyajbo5ezhc765347defwq - Operation Type: Update Shape - Percent Complete: 100 - Time Accepted: 2022-03-09 00:25:03.644 +0000 UTC - Time Finished: 2022-03-09 00:38:59.526 +0000 UTC - Time Started: 2022-03-09 00:25:15.578 +0000 UTC - Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrbaqah6qktukvdlnx66fp2hlevegryfuppsshkqemfcdjtwfwaq3q - Operation Type: Scale Storage - Percent Complete: 100 - Time Accepted: 2022-03-09 00:48:54.849 +0000 UTC - Time Finished: 2022-03-09 01:03:10.885 +0000 UTC - Time Started: 2022-03-09 00:49:05.911 +0000 UTC - Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrvhcpme5ijlsxup22kuumjuzn367vdxwhblv2nxpwshfwnig5au7a - Operation Type: Update DB System License Type - Percent Complete: 100 - Time Accepted: 2022-03-09 01:16:16.991 +0000 UTC - Time Finished: 2022-03-09 01:17:05.025 +0000 UTC - Time Started: 2022-03-09 01:16:24.716 +0000 UTC -Events: -[root@docker-test-server test]# -[root@docker-test-server test]# -[root@docker-test-server test]# -[root@docker-test-server test]# kubectl describe dbcssystems.database.oracle.com dbcssystem-create -Name: dbcssystem-create -Namespace: default -Labels: -Annotations: lastSuccessfulSpec: - {"dbSystem":{"compartmentId":"ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya","availabilityDomain":"O... -API Version: database.oracle.com/v1alpha1 -Kind: DbcsSystem -Metadata: - Creation Timestamp: 2022-03-08T22:12:57Z - Generation: 1 - Managed Fields: - API Version: database.oracle.com/v1alpha1 - Fields Type: FieldsV1 - fieldsV1: - f:metadata: - f:annotations: - .: - f:kubectl.kubernetes.io/last-applied-configuration: - f:spec: - .: - f:dbSystem: - .: - f:availabilityDomain: - f:compartmentId: - f:dbAdminPaswordSecret: - f:dbName: - f:dbVersion: - f:dbWorkload: - f:hostName: - f:shape: - f:sshPublicKeys: - f:subnetId: - f:ociConfigMap: - f:ociSecret: - Manager: kubectl-client-side-apply - Operation: Update - Time: 2022-03-08T22:12:57Z - API Version: database.oracle.com/v1alpha1 - Fields Type: FieldsV1 - fieldsV1: - f:metadata: - f:annotations: - f:lastSuccessfulSpec: - f:spec: - f:dbSystem: - f:dbBackupConfig: - f:id: - f:status: - .: - f:availabilityDomain: - f:cpuCoreCount: - f:dataStoragePercentage: - f:dataStorageSizeInGBs: - f:dbEdition: - f:dbInfo: - f:displayName: - f:id: - f:licenseModel: - f:network: - .: - f:clientSubnet: - f:domainName: - f:hostName: - f:listenerPort: - f:scanDnsName: - f:vcnName: - f:nodeCount: - f:recoStorageSizeInGB: - f:shape: - f:state: - f:storageManagement: - f:subnetId: - f:timeZone: - f:workRequests: - Manager: manager - Operation: Update - Time: 2022-03-08T23:13:34Z - Resource Version: 55187354 - UID: 0cdbebc0-3aeb-43b1-ae7f-eb36e6c56000 -Spec: - Db System: - Availability Domain: OLou:PHX-AD-1 - Compartment Id: ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya - Db Admin Pasword Secret: admin-password - Db Name: db0130 - Db Version: 21c - Db Workload: OLTP - Host Name: host0130 - Shape: VM.Standard2.1 - Ssh Public Keys: - oci-publickey - Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a - Oci Config Map: oci-cred - Oci Secret: oci-privatekey -Status: - Availability Domain: OLou:PHX-AD-1 - Cpu Core Count: 1 - Data Storage Percentage: 80 - Data Storage Size In G Bs: 256 - Db Edition: ENTERPRISE_EDITION - Db Info: - Db Home Id: ocid1.dbhome.oc1.phx.anyhqljr5gy3jhqat52milqwt3gq6lwohhacwg5yi4mtzq7c7hag53lrkugq - Db Name: db0130 - Db Unique Name: db0130_phx1zn - Db Workload: OLTP - Id: ocid1.database.oc1.phx.anyhqljrabf7htyackgmsaqjfexoqgrzuuk33ju2q25z2al43tnd5mhhvkra - Display Name: dbsystem20220308221302 - Id: ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa - License Model: LICENSE_INCLUDED - Network: - Client Subnet: k8test-pubvcn - Domain Name: k8testpubvcn.k8test.oraclevcn.com - Host Name: host0130 - Listener Port: 1521 - Scan Dns Name: host0130-scan.k8testpubvcn.k8test.oraclevcn.com - Vcn Name: k8test - Node Count: 1 - Reco Storage Size In GB: 256 - Shape: VM.Standard2.1 - State: AVAILABLE - Storage Management: ASM - Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a - Time Zone: UTC - Work Requests: - Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrxivzvgzel47zuoyke5yk36o7mrgjl27vscd5z3bqptmyh3rxwbqq - Operation Type: Create DB System - Percent Complete: 100 - Time Accepted: 2022-03-08 22:13:02.999 +0000 UTC - Time Finished: 2022-03-08 23:11:50.46 +0000 UTC - Time Started: 2022-03-08 22:13:16.995 +0000 UTC -Events: -[root@docker-test-server test]# - - [root@docker-test-server test]# kubectl delete dbcssystems.database.oracle.com dbcssystem-existing dbcssystem.database.oracle.com "dbcssystem-existing" deleted [root@docker-test-server test]# @@ -553,4 +245,4 @@ dbcssystem.database.oracle.com "dbcssystem-create" deleted [root@docker-test-server test]# [root@docker-test-server test]# kubectl delete dbcssystems.database.oracle.com dbcssystem-create Error from server (NotFound): dbcssystems.database.oracle.com "dbcssystem-create" not found -[root@docker-test-server test]# \ No newline at end of file +[root@docker-test-server test]# From 32571a6e69f999f3ce2e3c8840595167bf96a501 Mon Sep 17 00:00:00 2001 From: psaini Date: Wed, 8 Jun 2022 15:39:48 +0000 Subject: [PATCH 435/628] Added Correct DB edition type for RAC --- docs/dbcs/provisioning/dbcs_service_with_2_node_rac.md | 2 +- docs/dbcs/provisioning/dbcs_service_with_2_node_rac.yaml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/dbcs/provisioning/dbcs_service_with_2_node_rac.md b/docs/dbcs/provisioning/dbcs_service_with_2_node_rac.md index 8401a0e7..1cfbe006 100644 --- a/docs/dbcs/provisioning/dbcs_service_with_2_node_rac.md +++ b/docs/dbcs/provisioning/dbcs_service_with_2_node_rac.md @@ -15,7 +15,7 @@ This example uses `dbcs_service_with_2_node_rac.yaml` to deploy a 2 Node RAC VMD - Enable flag for Automatic Backup for DBCS Database as `True` - Auto Backup Window for DBCS Database as `SLOT_FOUR` - Recovery Windows for Backup retention in days as `15` -- Oracle Database Edition as `STANDARD_EDITION` +- Oracle Database Edition as `ENTERPRISE_EDITION_EXTREME_PERFORMANCE` - Database Name as `db0130` - Oracle Database Software Image Version as `21c` - Database Workload Type as Transaction Processing i.e. `OLTP` diff --git a/docs/dbcs/provisioning/dbcs_service_with_2_node_rac.yaml b/docs/dbcs/provisioning/dbcs_service_with_2_node_rac.yaml index 183246f8..168cb427 100644 --- a/docs/dbcs/provisioning/dbcs_service_with_2_node_rac.yaml +++ b/docs/dbcs/provisioning/dbcs_service_with_2_node_rac.yaml @@ -14,7 +14,7 @@ spec: autoBackupEnabled: True autoBackupWindow: "SLOT_FOUR" recoveryWindowsInDays: 15 - dbEdition: "STANDARD_EDITION" + dbEdition: "ENTERPRISE_EDITION_EXTREME_PERFORMANCE" dbName: "db0130" dbVersion: "21c" dbWorkload: "OLTP" From f1bb4052333ed6fb99a3f8c4caaa98c29f2fc9c3 Mon Sep 17 00:00:00 2001 From: psaini Date: Wed, 8 Jun 2022 16:02:53 +0000 Subject: [PATCH 436/628] Added RAC DB Edition Fix --- .../dbcs_service_with_2_node_rac_sample_output.log | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/dbcs/provisioning/dbcs_service_with_2_node_rac_sample_output.log b/docs/dbcs/provisioning/dbcs_service_with_2_node_rac_sample_output.log index c4c5efcd..e33e7a86 100644 --- a/docs/dbcs/provisioning/dbcs_service_with_2_node_rac_sample_output.log +++ b/docs/dbcs/provisioning/dbcs_service_with_2_node_rac_sample_output.log @@ -1,4 +1,4 @@ -[root@docker-test-server test]# cat dbcs_service_with_2_node_rac.yaml +NTERPRISE_EDITION_EXTREME_PERFORMANCE[root@docker-test-server test]# cat dbcs_service_with_2_node_rac.yaml apiVersion: database.oracle.com/v1alpha1 kind: DbcsSystem metadata: @@ -15,7 +15,7 @@ spec: autoBackupEnabled: True autoBackupWindow: "SLOT_FOUR" recoveryWindowsInDays: 15 - dbEdition: "STANDARD_EDITION" + dbEdition: "ENTERPRISE_EDITION_EXTREME_PERFORMANCE" dbName: "db0130" dbVersion: "21c" dbWorkload: "OLTP" @@ -271,7 +271,7 @@ Spec: Auto Backup Enabled: true Auto Backup Window: SLOT_FOUR Recovery Windows In Days: 15 - Db Edition: STANDARD_EDITION + Db Edition: ENTERPRISE_EDITION_EXTREME_PERFORMANCE Db Name: db0130 Db Version: 21c Db Workload: OLTP @@ -349,4 +349,4 @@ Last login: Wed Mar 9 18:23:10 CET 2022 [grid@host01301 ~]$ [grid@host01301 ~]$ cemutlo -n -dbSys3zthw55a \ No newline at end of file +dbSys3zthw55a From 26c58b9ac8a05f3245b31b8a9dcdcd802809e78a Mon Sep 17 00:00:00 2001 From: psaini Date: Thu, 9 Jun 2022 04:06:37 +0000 Subject: [PATCH 437/628] Added the known issues --- docs/dbcs/provisioning/known_issues.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/docs/dbcs/provisioning/known_issues.md b/docs/dbcs/provisioning/known_issues.md index 2bba6bf7..2dc54a48 100644 --- a/docs/dbcs/provisioning/known_issues.md +++ b/docs/dbcs/provisioning/known_issues.md @@ -3,3 +3,10 @@ Below are the known issues using the Oracle DB Operator DBCS Controller: 1. There is a known issue related to the DB Version 19c, 12c and 11g when used with the Oracle DB Operator DBCS Controller. DB Version 21c and 18c work with the controller. +2. In order to scale up storage of an existing DBCS system, the steps will be: + * Bind the existing DBCS System to DBCS Controller. + * Apply the change to scale up its storage. + This causes issue. The actual real step sequence that work is + * Bind + * Apply Shape change + * Apply scale storage change From c3f2920139093a1129e7a5b0f16c58506d65f5ab Mon Sep 17 00:00:00 2001 From: Ruggero Citton Date: Thu, 9 Jun 2022 15:59:56 +0200 Subject: [PATCH 438/628] ords dockerfile fix, oneprem readme change --- docs/onpremdb/README.md | 48 ++++++++++++++++++++++++++++++----------- ords/Dockerfile | 2 +- 2 files changed, 36 insertions(+), 14 deletions(-) diff --git a/docs/onpremdb/README.md b/docs/onpremdb/README.md index 1b27ad22..78e1a80e 100644 --- a/docs/onpremdb/README.md +++ b/docs/onpremdb/README.md @@ -3,8 +3,8 @@ The On-Premise Database Controller enables provisioning of Oracle Databases (PDBs) both on a Kubernetes cluster or outside of a Kubernetes cluster. The following sections explain the setup and functionality of this controller: * [Prerequisites for On-Premise Database Controller](ORACLE_ONPREMDB_CONTROLLER_README.md#prerequisites-and-setup) -* [Kubernetes Secrets](ORACLE_ONPREMDB_CONTROLLER_README.md#kubernetes-secrets) -* [Kubernetes CRD for CDB](ORACLE_ONPREMDB_CONTROLLER_README.md#kubernetes-crd-for-cdb) +* [Kubernetes Secrets](ORACLE_ONPREMDB_CONTROLLER_README.md#kubernetes-secrets) +* [Kubernetes CRD for CDB](ORACLE_ONPREMDB_CONTROLLER_README.md#kubernetes-crd-for-cdb) * [Kubernetes CRD for PDB](ORACLE_ONPREMDB_CONTROLLER_README.md#kubernetes-crd-for-pdb) * [PDB Lifecycle Management Operations](ORACLE_ONPREMDB_CONTROLLER_README.md#pdb-lifecycl-management-operations) * [Validation and Errors](ORACLE_ONPREMDB_CONTROLLER_README.md#validation-and-errors) @@ -14,7 +14,7 @@ The On-Premise Database Controller enables provisioning of Oracle Databases (PDB + ### Prepare CDB for PDB Lifecycme Management (PDB-LM) Pluggable Database management is performed in the Container Database (CDB) and includes create, clone, plug, unplug, delete, modify and map operations. - You cannot have an ORDS enabled schema in the container database. To perform the PDB lifecycle management operations, the default CDB administrator credentials must be defined. + You cannot have an ORDS enabled schema in the container database. To perform the PDB lifecycle management operations, the default CDB administrator credentials must be defined. To define the default CDB administrator credentials, perform the following steps on the target CDB(s) where PDB-LM operations are to be performed: @@ -26,26 +26,40 @@ The On-Premise Database Controller enables provisioning of Oracle Databases (PDB ``` + ### Building the Oracle REST Data Service (ORDS) Image - Oracle On-Premise Database controller enhances the Oracle REST Data Services (ORDS) image to enable it for PDB Lifecycle Management. You can build this image by following the instructions below: - * After cloning the repository, go to the "ords" folder, and run: + Oracle On-Premise Database controller enhances the Oracle REST Data Services (ORDS) image to enable it for PDB Lifecycle Management. You can build this image by following the instructions below: + * After cloning the repository, go to the "ords" folder, and run: ```sh docker build -t oracle/ords-dboper:latest . ``` - * Once the image is ready, you need to push it to your Docker Images Repository to pull it during CDB Resource creation. + + > **_NOTE1:_** Required file to build this image is Oracle Rest Data Services 'ords-< version >.zip', + you can download such file from http://www.oracle.com/technetwork/developer-tools/rest-data-services/downloads/index.html + + > **_NOTE2:_** to build the ords image you need to Accept the Oracle Standard Terms and Restrictions for Java/jdk repository on Oracle Container Registry at https://container-registry.oracle.com + + > **_NOTE3:_** docker needs access to container-registry.oracle.com for which following command may be required: + ```sh + docker login container-registry.oracle.com + ``` + > **_NOTE4:_** if you are going to deploy inside minikube, you need to switch the env to it, issuing: + ```sh + eval $(minikube docker-env) + ``` + * Once the image is ready, you may need to push it to your Docker Images Repository to pull it during CDB controller resource creation. + ### Install cert-manager Validating webhook is an endpoint Kubernetes can invoke prior to persisting resources in ETCD. This endpoint returns a structured response indicating whether the resource should be rejected or accepted and persisted to the datastore. Webhook requires a TLS certificate that the apiserver is configured to trust . Install the cert-manager with the following command: - + ```sh kubectl apply -f https://github.com/jetstack/cert-manager/releases/latest/download/cert-manager.yaml ``` ## Kubernetes Secrets - Both CDBs and PDBs make use of Kubernetes Secrets to store usernames and passwords. + Both CDBs and PDBs make use of Kubernetes Secrets to store usernames and passwords. For CDB, create a secret file as shown here: [config/samples/onpremdb/cdb_secret.yaml](../../config/samples/onpremdb/cdb_secret.yaml) @@ -53,7 +67,15 @@ The On-Premise Database Controller enables provisioning of Oracle Databases (PDB $ kubectl apply -f cdb_secret.yaml secret/cdb1-secret created ``` - On successful creation of the CDB Resource, the CDB secrets would be deleted from the Kubernetes system. + > **_NOTE:_** the entries such user and password must be 'base64 encoded', example: + "cdbadmin_user=C##DBAPI_CDB_ADMIN" become cdbadmin_user: "QyMjREJBUElfQ0RCX0FETUlOCg==" using command: + + ```sh + echo C##DBAPI_CDB_ADMIN | base64 + QyMjREJBUElfQ0RCX0FETUlOCg== + ``` + + On successful creation of the CDB Resource, the CDB secrets would be deleted from the Kubernetes system. For PDB, create a secret file as shown here: [config/samples/onpremdb/pdb_secret.yaml](../../config/samples/onpremdb/pdb_secret.yaml) @@ -68,7 +90,7 @@ The On-Premise Database Controller enables provisioning of Oracle Databases (PDB $ kubectl create secret generic pdb1-secret --from-literal sysadmin_user=pdbadmin --from-literal sysdamin_pwd=WE2112#232# secret/pdb1-secret created ``` - + ## Kubernetes CRD for CDB The Oracle Database Operator creates the CDB kind as a custom resource that models a target CDB as a native Kubernetes object. This is only used to create Pods to connect to the target CDB to perform PDB-LM operations. These CDB resources can be scaled up and down based on the expected load using replicas. Each CDB resource follows the CDB CRD as defined here: [config/crd/bases/database.oracle.com_cdbs.yaml](../../config/crd/bases/database.oracle.com_cdbs.yaml) @@ -77,7 +99,7 @@ The On-Premise Database Controller enables provisioning of Oracle Databases (PDB A sample .yaml file is available here: [config/samples/onpremdb/cdb.yaml](../../config/samples/onpremdb/cdb.yaml) - **Note:** The password and username fields in the above `cdb.yaml` yaml are Kubernetes Secrets. Please see the section [Kubernetes Secrets](ORACLE_ONPREMDB_CONTROLLER_README.md#kubernetes-secrets) for more information. + **Note:** The password and username fields in the above `cdb.yaml` yaml are Kubernetes Secrets. Please see the section [Kubernetes Secrets](ORACLE_ONPREMDB_CONTROLLER_README.md#kubernetes-secrets) for more information. + ### Check the status of the all CDBs ```sh @@ -153,7 +175,7 @@ The On-Premise Database Controller enables provisioning of Oracle Databases (PDB + ### Map PDB This is used to map an existing PDB in the CDB as a Kubernetes Custom Resource. - A sample .yaml file is available here: [config/samples/onpremdb/pdb_map.yaml](../../config/samples/onpremdb/pdb_map.yaml) + A sample .yaml file is available here: [config/samples/onpremdb/pdb_map.yaml](../../config/samples/onpremdb/pdb_map.yaml) ## Validation and Errors @@ -210,4 +232,4 @@ $ kubectl get pdbs -A NAMESPACE NAME CONNECT STRING CDB NAME PDB NAME PDB SIZE STATUS MESSAGE oracle-database-operator-system pdb1 democdb demotest1 Failed Secret not found:pdb12-secret oracle-database-operator-system pdb2 democdb demotest2 Failed ORA-65016: FILE_NAME_CONVERT must be specified... -``` \ No newline at end of file +``` diff --git a/ords/Dockerfile b/ords/Dockerfile index eeff5cba..1d137cab 100644 --- a/ords/Dockerfile +++ b/ords/Dockerfile @@ -20,7 +20,7 @@ # # Pull base image # --------------- -FROM container-registry.oracle.com/java/serverjre:8 +FROM container-registry.oracle.com/java/jdk:latest # Environment variables required for this build (do NOT change) # ------------------------------------------------------------- From 0bb9617e3e9cea82982ac01cba53559b52d0f809 Mon Sep 17 00:00:00 2001 From: kuassi_mensah Date: Thu, 9 Jun 2022 15:09:00 +0000 Subject: [PATCH 439/628] Update README.md --- README.md | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 4b1ea423..a79dd9ec 100644 --- a/README.md +++ b/README.md @@ -2,17 +2,21 @@ ## Make Oracle Database Kubernetes Native - An Overview -As part of Oracle's resolution to make Oracle Database Kubernetes-native (that is, observable and operable by Kubernetes), Oracle is announcing _Oracle Database Operator for Kubernetes_ (`OraOperator`). +Starting with release 19c, Oracle Database images have been supported in containers (Docker, Podman) for production use and Kubernetes deployment with Helm Charts. +As part of Oracle's resolution to make Oracle Database Kubernetes-native (that is, observable and operable by Kubernetes), Oracle released _Oracle Database Operator for Kubernetes_ (`OraOperator`). +OraOperator extends the Kubernetes API with custom resources and controllers for automating Oracle Database lifecycle management. -Since Oracle Database 19c, Oracle Database images have been supported in containers (Docker, Podman) for production use and Kubernetes deployment with Helm Charts. This release includes Oracle Database Operator, which is a new open source product that extends the Kubernetes API with custom resources and controllers for automating Oracle Database lifecycle management. - -In this release, `OraOperator` supports the following Oracle Database configurations: +In this release, `OraOperator` supports the following Oracle Database configurations and infrastructure: * Oracle Autonomous Database on shared Oracle Cloud Infrastructure (OCI), also known as ADB-S +* Oracle Autonomous Database on dedicated Cloud infrastructure, also known as ADB-D * Containerized Single Instance databases (SIDB) deployed in the Oracle Kubernetes Engine (OKE) * Containerized Sharded databases (SHARDED) deployed in OKE +* Oracle On-Premises Databases +* Oracle Database Cloud Service +* Oracle Autonomous Container Database (infrastructure) -Oracle will continue to expand Oracle Database Operator support for additional Oracle Database configurations. +Oracle will continue to expand OraOperator support for additional Oracle Database configurations. ## Features Summary @@ -87,7 +91,7 @@ The quickstarts are designed for specific database configurations: * [Containerized Oracle Single Instance Database](./docs/sidb/README.md) * [Containerized Oracle Shared Database](./docs/sharding/README.md) * [Oracle On-Premises Database](./docs/onpremdb/README.md) -* [Oracle Cloud Co-managed Database](./docs/dbcs/README.md) +* [Oracle Database Cloud Service](./docs/dbcs/README.md) YAML file templates are available under [`/config/samples`](./config/samples/). You can copy and edit these template files to configure them for your use cases. From 214b4892d319d73900fb5d87d907b5c1ca5add43 Mon Sep 17 00:00:00 2001 From: kuassi_mensah Date: Thu, 9 Jun 2022 15:15:10 +0000 Subject: [PATCH 440/628] Update README.md --- README.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index a79dd9ec..ba3724ae 100644 --- a/README.md +++ b/README.md @@ -23,10 +23,13 @@ Oracle will continue to expand OraOperator support for additional Oracle Databas This release of Oracle Database Operator for Kubernetes (the operator) supports the following lifecycle operations: * ADB-S: provision, bind, start, stop, terminate (soft/hard), scale (down/up) +* ADB-D: provision, bind, start, stop, terminate (soft/hard), scale (down/up) * SIDB: provision, clone, patch (in-place/out-of-place), update database initialization parameters, update database configuration (Flashback, archiving), Oracle Enterprise Manager (EM) Express (a basic observability console), Oracle REST Data Service (ORDS) to support REST based SQL, PDB management, SQL Developer Web, and Application Express (Apex) * SHARDED: provision/deploy sharded databases and the shard topology, add a new shard, delete an existing shard +* On-Premises Database: Bind to a CDB, Create a  PDB, Plug a  PDB, Unplug a PDB, Delete a PDB, Clone a PDB, Open/Close a PDB +* Database Cloud Service: Provision, Bind, Scale shapeUp/Down, Liveness Probe, On Demand backup -Upcoming releases will support new configurations, operations and capabilities. +The upcoming releases will support new configurations, operations and capabilities. ## Release Status From 257d7df2d88ad8a7e99589024e39fcafc857ea66 Mon Sep 17 00:00:00 2001 From: kuassi_mensah Date: Thu, 9 Jun 2022 15:30:38 +0000 Subject: [PATCH 441/628] Update README.md --- README.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index ba3724ae..2ee0f487 100644 --- a/README.md +++ b/README.md @@ -35,11 +35,13 @@ The upcoming releases will support new configurations, operations and capabiliti **CAUTION:** The current release of `OraOperator` (v0.2.0) is for development and test only. DO NOT USE IN PRODUCTION. -This release can be deployed on the following platforms: +This release has been installed and tested on the following platforms: * [Oracle Container Engine for Kubernetes (OKE)](https://www.oracle.com/cloud-native/container-engine-kubernetes/) with Kubernetes 1.17 or later -* In an on-premises [Oracle Linux Cloud Native Environment(OLCNE)](https://docs.oracle.com/en/operating-systems/olcne/) 1.3 or later +* [Oracle Linux Cloud Native Environment(OLCNE)](https://docs.oracle.com/en/operating-systems/olcne/) 1.3 or later * [Minikube](https://minikube.sigs.k8s.io/docs/) with version v1.21.0 or later +* [Azure Kubernetes Service](https://azure.microsoft.com/en-us/services/kubernetes-service/) +* [Amazon Elastic Kubernetes Service](https://aws.amazon.com/eks/) In upcoming releases, the operator will be certified against third-party Kubernetes clusters. From f9bcbf90391772abf0910ea36a0518adbaab8823 Mon Sep 17 00:00:00 2001 From: kuassi_mensah Date: Thu, 9 Jun 2022 15:51:57 +0000 Subject: [PATCH 442/628] Update README.md --- README.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/README.md b/README.md index 2ee0f487..e4b55ef5 100644 --- a/README.md +++ b/README.md @@ -113,9 +113,7 @@ YAML file templates are available under [`/config/samples`](./config/samples/). kubectl delete singleinstancedatabase.database.oracle.com --all -n kubectl delete shardingdatabase.database.oracle.com --all -n kubectl delete autonomousdatabase.database.oracle.com --all -n - kubectl delete autonomousdatabasebackup.database.oracle.com --all -n - kubectl delete autonomousdatabaserestore.database.oracle.com --all -n - kubectl delete autonomouscontainerdatabase.database.oracle.com --all -n + kubectl delete acd -n ``` After all CRD instances are deleted, it is safe to remove the CRDs, APISerivces and operator deployment. From 9d5df47d44f1921ad2d12f7b79d09db58d311620 Mon Sep 17 00:00:00 2001 From: psaini Date: Thu, 9 Jun 2022 15:56:27 +0000 Subject: [PATCH 443/628] Added delete info DBCS --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index e4b55ef5..78b6246c 100644 --- a/README.md +++ b/README.md @@ -113,6 +113,7 @@ YAML file templates are available under [`/config/samples`](./config/samples/). kubectl delete singleinstancedatabase.database.oracle.com --all -n kubectl delete shardingdatabase.database.oracle.com --all -n kubectl delete autonomousdatabase.database.oracle.com --all -n + kubectl delete dbcssystem.database.oracle.com --all -n kubectl delete acd -n ``` From c38c0f3f4cb279b0f58ebb92b5757011cda93bdd Mon Sep 17 00:00:00 2001 From: psaini Date: Thu, 9 Jun 2022 15:59:55 +0000 Subject: [PATCH 444/628] Added DBCS Documentation Link --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 78b6246c..0d760ca0 100644 --- a/README.md +++ b/README.md @@ -132,6 +132,7 @@ YAML file templates are available under [`/config/samples`](./config/samples/). * [Components of Dedicated Autonomous Database](https://docs.oracle.com/en-us/iaas/autonomous-database/doc/components.html) * [Oracle Database Single Instance](https://docs.oracle.com/en/database/oracle/oracle-database/) * [Oracle Database Sharding](https://docs.oracle.com/en/database/oracle/oracle-database/21/shard/index.html) +* [Oracle Database Cloud Service](https://docs.oracle.com/en/database/database-cloud-services.html) ## Contributing From 8a899e765761c13d828f5062111c38696e4d897a Mon Sep 17 00:00:00 2001 From: Ruggero Citton Date: Fri, 10 Jun 2022 09:38:26 +0200 Subject: [PATCH 445/628] how to manage ords image --- docs/onpremdb/README.md | 33 ++++++++++++++++++++++++++++++--- 1 file changed, 30 insertions(+), 3 deletions(-) diff --git a/docs/onpremdb/README.md b/docs/onpremdb/README.md index 78e1a80e..494b2a42 100644 --- a/docs/onpremdb/README.md +++ b/docs/onpremdb/README.md @@ -41,11 +41,38 @@ The On-Premise Database Controller enables provisioning of Oracle Databases (PDB ```sh docker login container-registry.oracle.com ``` - > **_NOTE4:_** if you are going to deploy inside minikube, you need to switch the env to it, issuing: + + * Once the image is ready, you may need to push it to your Docker Images Repository to pull it during CDB controller resource creation. + + The steps + 1. check the ords docker image creation issuing: ```sh - eval $(minikube docker-env) + docker images ``` - * Once the image is ready, you may need to push it to your Docker Images Repository to pull it during CDB controller resource creation. + example: + ``` + $ docker images + REPOSITORY TAG IMAGE ID CREATED SIZE + oracle/ords-dboper latest 887652b3e87f 17 hours ago 777MB + ``` + + 2. make the required docker tag issuing: + ``` + docker tag oracle/ords-dboper + ``` + example: + ``` + docker tag oracle/ords-dboper lin.ocir.io/mytenancy/mycontainer/myrepo/ords-dboper + ``` + + 3. Push the image over the repository + ``` + docker push + ``` + example: + ``` + docker push lin.ocir.io/mytenancy/mycontainer/myrepo/ords-dboper:latest + ``` + ### Install cert-manager From f3ea2d91399a5a2ef8c7e05ee425e2e53bd21740 Mon Sep 17 00:00:00 2001 From: Abhishek Kumar Date: Mon, 13 Jun 2022 10:26:33 +0000 Subject: [PATCH 446/628] Readme changes regarding minikube persistence --- .../samples/sidb/oraclerestdataservice.yaml | 2 +- .../samples/sidb/singleinstancedatabase.yaml | 2 +- .../sidb/singleinstancedatabase_clone.yaml | 2 +- .../sidb/singleinstancedatabase_create.yaml | 2 +- .../sidb/singleinstancedatabase_express.yaml | 2 +- .../sidb/singleinstancedatabase_patch.yaml | 2 +- docs/sidb/PREREQUISITES.md | 8 +++++++ docs/sidb/README.md | 23 +++++++++++++++++-- 8 files changed, 35 insertions(+), 8 deletions(-) diff --git a/config/samples/sidb/oraclerestdataservice.yaml b/config/samples/sidb/oraclerestdataservice.yaml index c1fa782d..2b381cdb 100644 --- a/config/samples/sidb/oraclerestdataservice.yaml +++ b/config/samples/sidb/oraclerestdataservice.yaml @@ -54,7 +54,7 @@ spec: ## volumeName is optional. Specify for binding to a specific PV and set storageClass to an empty string to disable automatic volume provisioning # persistence: # size: 50Gi - ## oci-bv applies to OCI block volumes. Update as appropriate for other cloud service providers + ## oci-bv applies to OCI block volumes. Use "standard" storageClass for dynamic provisioning in Minikube. Update as appropriate for other cloud service providers # storageClass: "oci-bv" # accessMode: "ReadWriteOnce" # volumeName: "" diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index 44f26cf4..7728d91c 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -71,7 +71,7 @@ spec: ## volumeName is optional. Specify for binding to a specific PV and set storageClass to an empty string to disable automatic volume provisioning persistence: size: 100Gi - ## oci-bv applies to OCI block volumes. Update as appropriate for other cloud service providers + ## oci-bv applies to OCI block volumes. Use "standard" storageClass for dynamic provisioning in Minikube. Update as appropriate for other cloud service providers storageClass: "oci-bv" accessMode: "ReadWriteOnce" volumeName: "" diff --git a/config/samples/sidb/singleinstancedatabase_clone.yaml b/config/samples/sidb/singleinstancedatabase_clone.yaml index 67ae08fc..de575050 100644 --- a/config/samples/sidb/singleinstancedatabase_clone.yaml +++ b/config/samples/sidb/singleinstancedatabase_clone.yaml @@ -35,7 +35,7 @@ spec: ## accessMode can only accept one of ReadWriteOnce, ReadWriteMany persistence: size: 100Gi - ## oci-bv applies to OCI block volumes. Update as appropriate for other cloud service providers + ## oci-bv applies to OCI block volumes. Use "standard" storageClass for dynamic provisioning in Minikube. Update as appropriate for other cloud service providers storageClass: "oci-bv" accessMode: "ReadWriteOnce" diff --git a/config/samples/sidb/singleinstancedatabase_create.yaml b/config/samples/sidb/singleinstancedatabase_create.yaml index 342637f1..6435d850 100644 --- a/config/samples/sidb/singleinstancedatabase_create.yaml +++ b/config/samples/sidb/singleinstancedatabase_create.yaml @@ -53,7 +53,7 @@ spec: ## accessMode can only accept one of ReadWriteOnce, ReadWriteMany persistence: size: 100Gi - ## oci-bv applies to OCI block volumes. Update as appropriate for other cloud service providers + ## oci-bv applies to OCI block volumes. Use "standard" storageClass for dynamic provisioning in Minikube. Update as appropriate for other cloud service providers storageClass: "oci-bv" accessMode: "ReadWriteOnce" diff --git a/config/samples/sidb/singleinstancedatabase_express.yaml b/config/samples/sidb/singleinstancedatabase_express.yaml index d1100c9b..61a0b404 100644 --- a/config/samples/sidb/singleinstancedatabase_express.yaml +++ b/config/samples/sidb/singleinstancedatabase_express.yaml @@ -42,7 +42,7 @@ spec: ## accessMode can only accept one of ReadWriteOnce, ReadWriteMany persistence: size: 50Gi - ## oci-bv applies to OCI block volumes. Update as appropriate for other cloud service providers + ## oci-bv applies to OCI block volumes. Use "standard" storageClass for dynamic provisioning in Minikube. Update as appropriate for other cloud service providers storageClass: "oci-bv" accessMode: "ReadWriteOnce" diff --git a/config/samples/sidb/singleinstancedatabase_patch.yaml b/config/samples/sidb/singleinstancedatabase_patch.yaml index aa04e317..be2de1cf 100644 --- a/config/samples/sidb/singleinstancedatabase_patch.yaml +++ b/config/samples/sidb/singleinstancedatabase_patch.yaml @@ -43,7 +43,7 @@ spec: ## accessMode can only accept one of ReadWriteOnce, ReadWriteMany persistence: size: 100Gi - ## oci-bv applies to OCI block volumes. Update as appropriate for other cloud service providers + ## oci-bv applies to OCI block volumes. Use "standard" storageClass for dynamic provisioning in Minikube. Update as appropriate for other cloud service providers storageClass: "oci-bv" accessMode: "ReadWriteOnce" diff --git a/docs/sidb/PREREQUISITES.md b/docs/sidb/PREREQUISITES.md index 4588a84d..d442c11a 100644 --- a/docs/sidb/PREREQUISITES.md +++ b/docs/sidb/PREREQUISITES.md @@ -17,3 +17,11 @@ To deploy Oracle Database Operator for Kubernetes on Oracle Docker images, compl More info on creating persistent volumes available at [https://kubernetes.io/docs/concepts/storage/persistent-volumes/](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) +* ### Minikube Cluster Environment + + By default, Minikube creates a node with 2GB RAM, 2 CPUs, and 20GB disk space when a cluster is created using `minikube start` command. However, these resources (particularly disk space and RAM) may not be sufficient for running and managing Oracle Database using the OraOperator. It is recommended to have larger RAM and disk space for better performance. For example, the following command creates a Minikube cluster with 6GB RAM and 50GB disk space for the Minikube VM: + + ``` + minikube start --memory=6g --disk-size=50g + ``` + diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 33439d2d..f7e32613 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -255,12 +255,31 @@ The database persistence can be achieved in the following two ways: #### Dynamic Persistence In **Dynamic Persistence Provisioning**, a persistent volume is provisioned by mentioning a storage class. For example, **oci-bv** storage class is specified in the **[singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml)** file. This storage class facilitates dynamic provisioning of the OCI block volumes. The supported access mode for this class is `ReadWriteOnce`. For other cloud providers, you can similarly use their dynamic provisioning storage classes. -**Note:** Generally, the `Reclaim Policy` of such dynamically provisioned volumes is `Delete`. These volumes are deleted when their corresponding database deployment is deleted. To retain volumes, use static provisioning, as explained in the Block Volume Static Provisioning section. +**Note:** +- Generally, the `Reclaim Policy` of such dynamically provisioned volumes is `Delete`. These volumes are deleted when their corresponding database deployment is deleted. To retain volumes, use static provisioning, as explained in the Block Volume Static Provisioning section. +- In **Minikube**, the dynamic persistence provisioning class is **standard**. #### Static Persistence In **Static Persistence Provisioning**, you have to create a volume manually (either using Block Volume or NFS), and then use the name of this volume with the `<.spec.persistence.volumeName>` field (corresponds to the `volumeName` field of the persistence section in the **[singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml)**). The `Reclaim Policy` of such volume can be set to `Retain`. So, this volume does not get deleted with the deletion of its corresponding deployment. The access modes supported with block volume and NFS are `ReadWriteOnce` and `ReadWriteMany` respectively. +**Note:** +In **Minikube**, a persistent volume can be provisioned using the sample yaml file below: +```yaml +apiVersion: v1 +kind: PersistentVolume +metadata: + name: db-vol +spec: + capacity: + storage: 10Gi + accessModes: + - ReadWriteMany + persistentVolumeReclaimPolicy: Retain + hostPath: + path: +``` +Further, the volume name (i.e. db-vol) can be mentioned in the `volumeName` field of the **[singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml)**. `storageClass` field is not required in this case, and can be left empty. -Static Persistence Provisioning in OCI is explained in the following subsections: +Static Persistence Provisioning in Oracle Cloud Infrastructure (OCI) is explained in the following subsections: ##### Block Volume Static Provisioning With block volume static provisioning, you must manually create a block volume resource from the OCI console, and fetch its `OCID`. To create the persistent volume, you can use the following YAML file: From 7ed14b76cd9f52e3bb485a4438e7c32baea49923 Mon Sep 17 00:00:00 2001 From: Mohammed Yunus Date: Mon, 13 Jun 2022 11:46:31 +0000 Subject: [PATCH 447/628] Update README.md --- docs/sidb/README.md | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index f7e32613..e55580ca 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -260,9 +260,8 @@ In **Dynamic Persistence Provisioning**, a persistent volume is provisioned by m - In **Minikube**, the dynamic persistence provisioning class is **standard**. #### Static Persistence -In **Static Persistence Provisioning**, you have to create a volume manually (either using Block Volume or NFS), and then use the name of this volume with the `<.spec.persistence.volumeName>` field (corresponds to the `volumeName` field of the persistence section in the **[singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml)**). The `Reclaim Policy` of such volume can be set to `Retain`. So, this volume does not get deleted with the deletion of its corresponding deployment. The access modes supported with block volume and NFS are `ReadWriteOnce` and `ReadWriteMany` respectively. -**Note:** -In **Minikube**, a persistent volume can be provisioned using the sample yaml file below: +In **Static Persistence Provisioning**, you have to create a volume manually, and then use the name of this volume with the `<.spec.persistence.volumeName>` field which corresponds to the `volumeName` field of the persistence section in the **[singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml)**. The `Reclaim Policy` of such volume can be set to `Retain`. So, this volume does not get deleted with the deletion of its corresponding deployment. +For example in **Minikube**, a persistent volume can be provisioned using the sample yaml file below: ```yaml apiVersion: v1 kind: PersistentVolume @@ -275,13 +274,13 @@ spec: - ReadWriteMany persistentVolumeReclaimPolicy: Retain hostPath: - path: + path: /data/oradata ``` -Further, the volume name (i.e. db-vol) can be mentioned in the `volumeName` field of the **[singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml)**. `storageClass` field is not required in this case, and can be left empty. +The persistent volume name (i.e. db-vol) can be mentioned in the `volumeName` field of the **[singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml)**. `storageClass` field is not required in this case, and can be left empty. Static Persistence Provisioning in Oracle Cloud Infrastructure (OCI) is explained in the following subsections: -##### Block Volume Static Provisioning +##### OCI Block Volume Static Provisioning With block volume static provisioning, you must manually create a block volume resource from the OCI console, and fetch its `OCID`. To create the persistent volume, you can use the following YAML file: ```yaml apiVersion: v1 @@ -307,7 +306,7 @@ nodeSelector: topology.kubernetes.io/zone: PHX-AD-1 ``` -##### NFS Volume Static Provisioning +##### OCI NFS Volume Static Provisioning Similar to the block volume static provisioning, you have to manually create a file system resource from the OCI console, and fetch its `OCID, Mount Target and Export Path`. Mention these values in the following YAML file to create the persistent volume: ```yaml From 73b85bccddd8cd3aa3751afa30700b12cb2592c3 Mon Sep 17 00:00:00 2001 From: Abhishek Kumar Date: Tue, 14 Jun 2022 10:10:48 +0000 Subject: [PATCH 448/628] Logger time format change from Epoch to RFC3339 format --- main.go | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/main.go b/main.go index 56de731c..2b09f7f0 100644 --- a/main.go +++ b/main.go @@ -45,6 +45,7 @@ import ( "strconv" "time" + "go.uber.org/zap/zapcore" "k8s.io/apimachinery/pkg/runtime" utilruntime "k8s.io/apimachinery/pkg/util/runtime" clientgoscheme "k8s.io/client-go/kubernetes/scheme" @@ -79,7 +80,13 @@ func main() { "Enabling this will ensure there is only one active controller manager.") flag.Parse() - ctrl.SetLogger(zap.New(zap.UseDevMode(true))) + // Initialize new logger Opts + options := &zap.Options{ + Development: true, + TimeEncoder: zapcore.RFC3339TimeEncoder, + } + + ctrl.SetLogger(zap.New(func(o *zap.Options) { *o = *options })) mgr, err := ctrl.NewManager(ctrl.GetConfigOrDie(), ctrl.Options{ Scheme: scheme, From d7227373005f81aefe4868629749789a6aa080fe Mon Sep 17 00:00:00 2001 From: Abhishek Kumar Date: Thu, 16 Jun 2022 14:14:59 +0000 Subject: [PATCH 449/628] AbhiK_Bugfix_34265574 --- Makefile | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/Makefile b/Makefile index d7a42e32..78cf72ac 100644 --- a/Makefile +++ b/Makefile @@ -23,6 +23,8 @@ IMG ?= controller:latest CRD_OPTIONS ?= "crd:trivialVersions=true,preserveUnknownFields=false" # ENVTEST_K8S_VERSION refers to the version of kubebuilder assets to be downloaded by envtest binary. ENVTEST_K8S_VERSION = 1.21 +# Operator YAML file +OPERATOR_YAML=$$(basename $$(pwd)).yaml # Get the currently used golang install path (in GOPATH/bin, unless GOBIN is set) ifeq (,$(shell go env GOBIN)) @@ -90,9 +92,14 @@ deploy: manifests kustomize ## Deploy controller to the K8s cluster specified in cd config/manager && $(KUSTOMIZE) edit set image controller=${IMG} $(KUSTOMIZE) build config/default | kubectl apply -f - +# Bug:34265574 +# Used sed to reposition the controller-manager Deployment after the certificate creation in the OPERATOR_YAML operator-yaml: manifests kustomize cd config/manager && $(KUSTOMIZE) edit set image controller=${IMG} - $(KUSTOMIZE) build config/default > $$(basename $$(pwd)).yaml + $(KUSTOMIZE) build config/default > "${OPERATOR_YAML}" + sed -i.bak -e '/^apiVersion: apps\/v1/,/---/d' "${OPERATOR_YAML}" + (echo --- && sed '/^apiVersion: apps\/v1/,/---/!d' "${OPERATOR_YAML}.bak") >> "${OPERATOR_YAML}" + rm "${OPERATOR_YAML}.bak" undeploy: ## Undeploy controller from the K8s cluster specified in ~/.kube/config. $(KUSTOMIZE) build config/default | kubectl delete -f - From f9d3cf5078c539cf0a1b413673b76bbc513515b4 Mon Sep 17 00:00:00 2001 From: Abhishek Kumar Date: Wed, 22 Jun 2022 08:39:04 +0000 Subject: [PATCH 450/628] Added service annotations for SIDB & ORDS controllers --- .../v1alpha1/oraclerestdataservice_types.go | 15 +++++----- .../v1alpha1/singleinstancedatabase_types.go | 21 +++++++------- .../v1alpha1/zz_generated.deepcopy.go | 14 +++++++++ ...ase.oracle.com_oraclerestdataservices.yaml | 4 +++ ...se.oracle.com_singleinstancedatabases.yaml | 4 +++ .../samples/sidb/oraclerestdataservice.yaml | 4 +++ .../samples/sidb/singleinstancedatabase.yaml | 3 ++ .../oraclerestdataservice_controller.go | 29 ++++++++++++------- .../singleinstancedatabase_controller.go | 13 +++++++-- 9 files changed, 78 insertions(+), 29 deletions(-) diff --git a/apis/database/v1alpha1/oraclerestdataservice_types.go b/apis/database/v1alpha1/oraclerestdataservice_types.go index a705604a..dad990ee 100644 --- a/apis/database/v1alpha1/oraclerestdataservice_types.go +++ b/apis/database/v1alpha1/oraclerestdataservice_types.go @@ -52,6 +52,7 @@ type OracleRestDataServiceSpec struct { DatabaseRef string `json:"databaseRef"` LoadBalancer bool `json:"loadBalancer,omitempty"` + ServiceAnnotations map[string]string `json:"serviceAnnotations,omitempty"` NodeSelector map[string]string `json:"nodeSelector,omitempty"` Image OracleRestDataServiceImage `json:"image,omitempty"` OrdsPassword OracleRestDataServicePassword `json:"ordsPassword"` @@ -74,8 +75,8 @@ type OracleRestDataServicePersistence struct { StorageClass string `json:"storageClass,omitempty"` // +kubebuilder:validation:Enum=ReadWriteOnce;ReadWriteMany - AccessMode string `json:"accessMode,omitempty"` - VolumeName string `json:"volumeName,omitempty"` + AccessMode string `json:"accessMode,omitempty"` + VolumeName string `json:"volumeName,omitempty"` } // OracleRestDataServiceImage defines the Image source and pullSecrets for POD @@ -90,15 +91,15 @@ type OracleRestDataServicePassword struct { SecretName string `json:"secretName"` // +kubebuilder:default:="oracle_pwd" SecretKey string `json:"secretKey,omitempty"` - KeepSecret *bool `json:"keepSecret,omitempty"` + KeepSecret *bool `json:"keepSecret,omitempty"` } // OracleRestDataServicePDBSchemas defines the PDB Schemas to be ORDS Enabled type OracleRestDataServiceRestEnableSchemas struct { - PdbName string `json:"pdbName,omitempty"` - SchemaName string `json:"schemaName"` - UrlMapping string `json:"urlMapping,omitempty"` - Enable bool `json:"enable"` + PdbName string `json:"pdbName,omitempty"` + SchemaName string `json:"schemaName"` + UrlMapping string `json:"urlMapping,omitempty"` + Enable bool `json:"enable"` } // OracleRestDataServiceStatus defines the observed state of OracleRestDataService diff --git a/apis/database/v1alpha1/singleinstancedatabase_types.go b/apis/database/v1alpha1/singleinstancedatabase_types.go index a0ac9f33..11ee82db 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_types.go +++ b/apis/database/v1alpha1/singleinstancedatabase_types.go @@ -57,13 +57,14 @@ type SingleInstanceDatabaseSpec struct { // +k8s:openapi-gen=true // +kubebuilder:validation:Pattern=`^[a-zA-Z0-9]+$` // +kubebuilder:validation:MaxLength:=12 - Sid string `json:"sid,omitempty"` - Charset string `json:"charset,omitempty"` - Pdbname string `json:"pdbName,omitempty"` - LoadBalancer bool `json:"loadBalancer,omitempty"` - FlashBack bool `json:"flashBack,omitempty"` - ArchiveLog bool `json:"archiveLog,omitempty"` - ForceLogging bool `json:"forceLog,omitempty"` + Sid string `json:"sid,omitempty"` + Charset string `json:"charset,omitempty"` + Pdbname string `json:"pdbName,omitempty"` + LoadBalancer bool `json:"loadBalancer,omitempty"` + ServiceAnnotations map[string]string `json:"serviceAnnotations,omitempty"` + FlashBack bool `json:"flashBack,omitempty"` + ArchiveLog bool `json:"archiveLog,omitempty"` + ForceLogging bool `json:"forceLog,omitempty"` CloneFrom string `json:"cloneFrom,omitempty"` ReadinessCheckPeriod int `json:"readinessCheckPeriod,omitempty"` @@ -81,10 +82,10 @@ type SingleInstanceDatabaseSpec struct { // SingleInstanceDatabasePersistence defines the storage size and class for PVC type SingleInstanceDatabasePersistence struct { - Size string `json:"size,omitempty"` - StorageClass string `json:"storageClass,omitempty"` + Size string `json:"size,omitempty"` + StorageClass string `json:"storageClass,omitempty"` // +kubebuilder:validation:Enum=ReadWriteOnce;ReadWriteMany - AccessMode string `json:"accessMode,omitempty"` + AccessMode string `json:"accessMode,omitempty"` VolumeName string `json:"volumeName,omitempty"` VolumeClaimAnnotation string `json:"volumeClaimAnnotation,omitempty"` } diff --git a/apis/database/v1alpha1/zz_generated.deepcopy.go b/apis/database/v1alpha1/zz_generated.deepcopy.go index da87f055..17eb7052 100644 --- a/apis/database/v1alpha1/zz_generated.deepcopy.go +++ b/apis/database/v1alpha1/zz_generated.deepcopy.go @@ -1501,6 +1501,13 @@ func (in *OracleRestDataServiceRestEnableSchemas) DeepCopy() *OracleRestDataServ // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *OracleRestDataServiceSpec) DeepCopyInto(out *OracleRestDataServiceSpec) { *out = *in + if in.ServiceAnnotations != nil { + in, out := &in.ServiceAnnotations, &out.ServiceAnnotations + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } if in.NodeSelector != nil { in, out := &in.NodeSelector, &out.NodeSelector *out = make(map[string]string, len(*in)) @@ -2113,6 +2120,13 @@ func (in *SingleInstanceDatabasePersistence) DeepCopy() *SingleInstanceDatabaseP // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *SingleInstanceDatabaseSpec) DeepCopyInto(out *SingleInstanceDatabaseSpec) { *out = *in + if in.ServiceAnnotations != nil { + in, out := &in.ServiceAnnotations, &out.ServiceAnnotations + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } if in.NodeSelector != nil { in, out := &in.NodeSelector, &out.NodeSelector *out = make(map[string]string, len(*in)) diff --git a/config/crd/bases/database.oracle.com_oraclerestdataservices.yaml b/config/crd/bases/database.oracle.com_oraclerestdataservices.yaml index e48e7db4..121383fd 100644 --- a/config/crd/bases/database.oracle.com_oraclerestdataservices.yaml +++ b/config/crd/bases/database.oracle.com_oraclerestdataservices.yaml @@ -159,6 +159,10 @@ spec: type: array serviceAccountName: type: string + serviceAnnotations: + additionalProperties: + type: string + type: object required: - adminPassword - databaseRef diff --git a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml index e00a2db1..a4124691 100644 --- a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml +++ b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml @@ -150,6 +150,10 @@ spec: type: integer serviceAccountName: type: string + serviceAnnotations: + additionalProperties: + type: string + type: object sid: description: SID must be alphanumeric (no special characters, only a-z, A-Z, 0-9), and no longer than 12 characters. diff --git a/config/samples/sidb/oraclerestdataservice.yaml b/config/samples/sidb/oraclerestdataservice.yaml index 2b381cdb..f2ab93ab 100644 --- a/config/samples/sidb/oraclerestdataservice.yaml +++ b/config/samples/sidb/oraclerestdataservice.yaml @@ -62,6 +62,10 @@ spec: ## Type of service Applicable on cloud enviroments only. ## if loadBalService: false, service type = "NodePort" else "LoadBalancer" loadBalancer: false + ## Service Annotations (Cloud provider specific), for configuring the service (e.g. private LoadBalancer service) + #serviceAnnotations: + # service.beta.kubernetes.io/oci-load-balancer-internal: "true" + ## Deploy only on nodes having required labels. Format label_name: label_value ## The same lables are applied to the created PVC diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index 7728d91c..f0502697 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -79,6 +79,9 @@ spec: ## Type of service . Applicable on cloud enviroments only ## if loadBalService : false, service type = "NodePort" else "LoadBalancer" loadBalancer: false + ## Service Annotations (Cloud provider specific), for configuring the service (e.g. private LoadBalancer service) + #serviceAnnotations: + # service.beta.kubernetes.io/oci-load-balancer-internal: "true" ## Deploy only on nodes having required labels. Format label_name: label_value ## For instance if the pods need to be restricted to a particular AD diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 106be615..9021a6ec 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -251,7 +251,7 @@ func (r *OracleRestDataServiceReconciler) validate(m *dbapi.OracleRestDataServic eventMsgs = append(eventMsgs, "cannot configure ORDS for database "+m.Spec.DatabaseRef+" that has no attached persistent volume") } if !m.Status.OrdsInstalled && n.Status.OrdsReference != "" { - eventMsgs = append(eventMsgs, "database "+m.Spec.DatabaseRef+ " is already configured with ORDS "+n.Status.OrdsReference) + eventMsgs = append(eventMsgs, "database "+m.Spec.DatabaseRef+" is already configured with ORDS "+n.Status.OrdsReference) } if m.Status.DatabaseRef != "" && m.Status.DatabaseRef != m.Spec.DatabaseRef { eventMsgs = append(eventMsgs, "databaseRef cannot be updated") @@ -458,6 +458,15 @@ func (r *OracleRestDataServiceReconciler) instantiateSVCSpec(m *dbapi.OracleRest Labels: map[string]string{ "app": m.Name, }, + Annotations: func() map[string]string { + annotations := make(map[string]string) + if len(m.Spec.ServiceAnnotations) != 0 { + for key, value := range m.Spec.ServiceAnnotations { + annotations[key] = value + } + } + return annotations + }(), }, Spec: corev1.ServiceSpec{ Ports: []corev1.ServicePort{ @@ -768,7 +777,7 @@ func (r *OracleRestDataServiceReconciler) instantiatePVCSpec(m *dbapi.OracleRest }, }, StorageClassName: &m.Spec.Persistence.StorageClass, - VolumeName: m.Spec.Persistence.VolumeName, + VolumeName: m.Spec.Persistence.VolumeName, Selector: func() *metav1.LabelSelector { if m.Spec.Persistence.StorageClass != "oci" { return nil @@ -1237,8 +1246,8 @@ func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataS // Alter Apex Users log.Info("Alter APEX Users") _, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", - ctx, req, true, "bash", "-c", fmt.Sprintf("echo -e \"%s\" | %s", - fmt.Sprintf(dbcommons.AlterApexUsers, apexPassword, n.Spec.Pdbname), dbcommons.SQLPlusCLI)) + ctx, req, true, "bash", "-c", fmt.Sprintf("echo -e \"%s\" | %s", + fmt.Sprintf(dbcommons.AlterApexUsers, apexPassword, n.Spec.Pdbname), dbcommons.SQLPlusCLI)) if err != nil { log.Error(err, err.Error()) return requeueY @@ -1339,7 +1348,7 @@ func (r *OracleRestDataServiceReconciler) installApex(m *dbapi.OracleRestDataSer m.Status.Status = dbcommons.StatusReady eventReason = "Apex Installation" outArr := strings.Split(out, apexInstalled) - eventMsg = "installation of Apex "+ strings.TrimSpace(outArr[len(outArr)-1]) +" completed" + eventMsg = "installation of Apex " + strings.TrimSpace(outArr[len(outArr)-1]) + " completed" r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) n.Status.ApexInstalled = true r.Status().Update(ctx, n) @@ -1432,7 +1441,7 @@ func (r *OracleRestDataServiceReconciler) restEnableSchemas(m *dbapi.OracleRestD // If the PDB mentioned in yaml doesnt contain in the database , continue if !strings.Contains(strings.ToUpper(availablePDBS), strings.ToUpper(pdbName)) { eventReason := "PDB Check" - eventMsg := "PDB "+ pdbName +" not found for specified schema " + m.Spec.RestEnableSchemas[i].SchemaName + eventMsg := "PDB " + pdbName + " not found for specified schema " + m.Spec.RestEnableSchemas[i].SchemaName log.Info(eventMsg) r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) continue @@ -1450,10 +1459,10 @@ func (r *OracleRestDataServiceReconciler) restEnableSchemas(m *dbapi.OracleRestD // if ORDS already enabled for given PDB if strings.Contains(out, "STATUS:ENABLED") { - if m.Spec.RestEnableSchemas[i].Enable { + if m.Spec.RestEnableSchemas[i].Enable { log.Info("Schema already enabled", "schema", m.Spec.RestEnableSchemas[i].SchemaName) continue - } + } } else if strings.Contains(out, "STATUS:DISABLED") { if !m.Spec.RestEnableSchemas[i].Enable { log.Info("Schema already disabled", "schema", m.Spec.RestEnableSchemas[i].SchemaName) @@ -1479,7 +1488,7 @@ func (r *OracleRestDataServiceReconciler) restEnableSchemas(m *dbapi.OracleRestD createSchemaSQL := fmt.Sprintf(dbcommons.CreateORDSSchemaSQL, m.Spec.RestEnableSchemas[i].SchemaName, password, pdbName) log.Info("Creating schema", "schema", m.Spec.RestEnableSchemas[i].SchemaName) _, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", - fmt.Sprintf("echo -e \"%s\" | %s", createSchemaSQL, dbcommons.SQLPlusCLI)) + fmt.Sprintf("echo -e \"%s\" | %s", createSchemaSQL, dbcommons.SQLPlusCLI)) if err != nil { log.Error(err, err.Error()) return requeueY @@ -1514,7 +1523,7 @@ func (r *OracleRestDataServiceReconciler) restEnableSchemas(m *dbapi.OracleRestD } if restartORDS { - r.Log.Info("Restarting ORDS Pod "+ordsReadyPod.Name+" to clear disabled schemas cache") + r.Log.Info("Restarting ORDS Pod " + ordsReadyPod.Name + " to clear disabled schemas cache") var gracePeriodSeconds int64 = 0 policy := metav1.DeletePropagationForeground err = r.Delete(ctx, &ordsReadyPod, &client.DeleteOptions{ diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 8945a154..714c07c0 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -865,6 +865,15 @@ func (r *SingleInstanceDatabaseReconciler) instantiateSVCSpec(m *dbapi.SingleIns Labels: map[string]string{ "app": m.Name, }, + Annotations: func() map[string]string { + annotations := make(map[string]string) + if len(m.Spec.ServiceAnnotations) != 0 { + for key, value := range m.Spec.ServiceAnnotations { + annotations[key] = value + } + } + return annotations + }(), }, Spec: corev1.ServiceSpec{ Ports: []corev1.ServicePort{ @@ -933,7 +942,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePVCSpec(m *dbapi.SingleIns }, }, StorageClassName: &m.Spec.Persistence.StorageClass, - VolumeName: m.Spec.Persistence.VolumeName, + VolumeName: m.Spec.Persistence.VolumeName, Selector: func() *metav1.LabelSelector { if m.Spec.Persistence.StorageClass != "oci" { return nil @@ -1712,7 +1721,7 @@ func (r *SingleInstanceDatabaseReconciler) runDatapatch(m *dbapi.SingleInstanceD m.Status.DatafilesPatched = "true" status, versionFrom, versionTo, _ := dbcommons.GetSqlpatchStatus(r, r.Config, readyPod, ctx, req) if versionTo != "" { - eventMsg = "data files patched from release update " + versionFrom + " to " + versionTo + ", "+ status + ": " +releaseUpdate + eventMsg = "data files patched from release update " + versionFrom + " to " + versionTo + ", " + status + ": " + releaseUpdate } else { eventMsg = "datapatch execution completed" } From 5aab47bac09f9edc4af94dbf3a220d793947964c Mon Sep 17 00:00:00 2001 From: kuassi_mensah Date: Fri, 24 Jun 2022 15:16:50 +0000 Subject: [PATCH 451/628] Update README.md --- README.md | 27 ++++++++++++--------------- 1 file changed, 12 insertions(+), 15 deletions(-) diff --git a/README.md b/README.md index 0d760ca0..bda72904 100644 --- a/README.md +++ b/README.md @@ -2,21 +2,19 @@ ## Make Oracle Database Kubernetes Native - An Overview -Starting with release 19c, Oracle Database images have been supported in containers (Docker, Podman) for production use and Kubernetes deployment with Helm Charts. -As part of Oracle's resolution to make Oracle Database Kubernetes-native (that is, observable and operable by Kubernetes), Oracle released _Oracle Database Operator for Kubernetes_ (`OraOperator`). -OraOperator extends the Kubernetes API with custom resources and controllers for automating Oracle Database lifecycle management. +As part of Oracle's resolution to make Oracle Database Kubernetes-native (that is, observable and operable by Kubernetes), Oracle released _Oracle Database Operator for Kubernetes_ (`OraOperator`). OraOperator extends the Kubernetes API with custom resources and controllers for automating Oracle Database lifecycle management. -In this release, `OraOperator` supports the following Oracle Database configurations and infrastructure: +In this v0.2.0 release, `OraOperator` supports the following database configurations and infrastructure: -* Oracle Autonomous Database on shared Oracle Cloud Infrastructure (OCI), also known as ADB-S -* Oracle Autonomous Database on dedicated Cloud infrastructure, also known as ADB-D -* Containerized Single Instance databases (SIDB) deployed in the Oracle Kubernetes Engine (OKE) -* Containerized Sharded databases (SHARDED) deployed in OKE -* Oracle On-Premises Databases -* Oracle Database Cloud Service -* Oracle Autonomous Container Database (infrastructure) +* Oracle Autonomous Database on shared Oracle Cloud Infrastructure (OCI) (a.k.a. ADB-S) +* Oracle Autonomous Database on dedicated Cloud infrastructure (a.k.a. ADB-D) +* Containerized Single Instance databases (SIDB) deployed in the Oracle Kubernetes Engine (OKE) and any k8s where OraOperator is deployed +* Containerized Sharded databases (SHARDED) deployed in OKE and any k8s where OraOperator is deployed +* Oracle On-Premises Databases (CDB/PDBs, Exadata) +* Oracle Database Cloud Service (DBCS) (a.k.a. VMDB) +* Oracle Autonomous Container Database (infrastructure) the infrastructure for provisionning Autonomous Databases. -Oracle will continue to expand OraOperator support for additional Oracle Database configurations. +Oracle will continue to extent OraOperator to support additional Oracle Database configurations. ## Features Summary @@ -35,15 +33,14 @@ The upcoming releases will support new configurations, operations and capabiliti **CAUTION:** The current release of `OraOperator` (v0.2.0) is for development and test only. DO NOT USE IN PRODUCTION. -This release has been installed and tested on the following platforms: +This release has been installed and tested (not certified) on the following Kubernetes platforms: * [Oracle Container Engine for Kubernetes (OKE)](https://www.oracle.com/cloud-native/container-engine-kubernetes/) with Kubernetes 1.17 or later * [Oracle Linux Cloud Native Environment(OLCNE)](https://docs.oracle.com/en/operating-systems/olcne/) 1.3 or later * [Minikube](https://minikube.sigs.k8s.io/docs/) with version v1.21.0 or later * [Azure Kubernetes Service](https://azure.microsoft.com/en-us/services/kubernetes-service/) * [Amazon Elastic Kubernetes Service](https://aws.amazon.com/eks/) - -In upcoming releases, the operator will be certified against third-party Kubernetes clusters. +* [Red Hat OpenShift](https://www.redhat.com/en/technologies/cloud-computing/openshift/) ## Prerequisites From bd525ed9767da80396f98cb45da44bdf5c355403 Mon Sep 17 00:00:00 2001 From: kuassi_mensah Date: Fri, 24 Jun 2022 16:12:00 +0000 Subject: [PATCH 452/628] Update README.md --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index bda72904..6a525b0b 100644 --- a/README.md +++ b/README.md @@ -6,12 +6,12 @@ As part of Oracle's resolution to make Oracle Database Kubernetes-native (that i In this v0.2.0 release, `OraOperator` supports the following database configurations and infrastructure: -* Oracle Autonomous Database on shared Oracle Cloud Infrastructure (OCI) (a.k.a. ADB-S) -* Oracle Autonomous Database on dedicated Cloud infrastructure (a.k.a. ADB-D) +* Oracle Autonomous Database on shared Oracle Cloud Infrastructure (OCI) (ADB-S) +* Oracle Autonomous Database on dedicated Cloud infrastructure (ADB-D) * Containerized Single Instance databases (SIDB) deployed in the Oracle Kubernetes Engine (OKE) and any k8s where OraOperator is deployed * Containerized Sharded databases (SHARDED) deployed in OKE and any k8s where OraOperator is deployed * Oracle On-Premises Databases (CDB/PDBs, Exadata) -* Oracle Database Cloud Service (DBCS) (a.k.a. VMDB) +* Oracle Database Cloud Service (DBCS) (VMDB) * Oracle Autonomous Container Database (infrastructure) the infrastructure for provisionning Autonomous Databases. Oracle will continue to extent OraOperator to support additional Oracle Database configurations. From c0a2e2370dc036b041ed4b4a5506ba1e57d1d63c Mon Sep 17 00:00:00 2001 From: kuassi_mensah Date: Fri, 24 Jun 2022 16:26:09 +0000 Subject: [PATCH 453/628] Update README.md --- README.md | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 6a525b0b..abfed9c8 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ In this v0.2.0 release, `OraOperator` supports the following database configurat * Containerized Sharded databases (SHARDED) deployed in OKE and any k8s where OraOperator is deployed * Oracle On-Premises Databases (CDB/PDBs, Exadata) * Oracle Database Cloud Service (DBCS) (VMDB) -* Oracle Autonomous Container Database (infrastructure) the infrastructure for provisionning Autonomous Databases. +* Oracle Autonomous Container Database (ACD) (infrastructure) the infrastructure for provisionning Autonomous Databases. Oracle will continue to extent OraOperator to support additional Oracle Database configurations. @@ -20,12 +20,13 @@ Oracle will continue to extent OraOperator to support additional Oracle Databas This release of Oracle Database Operator for Kubernetes (the operator) supports the following lifecycle operations: -* ADB-S: provision, bind, start, stop, terminate (soft/hard), scale (down/up) -* ADB-D: provision, bind, start, stop, terminate (soft/hard), scale (down/up) -* SIDB: provision, clone, patch (in-place/out-of-place), update database initialization parameters, update database configuration (Flashback, archiving), Oracle Enterprise Manager (EM) Express (a basic observability console), Oracle REST Data Service (ORDS) to support REST based SQL, PDB management, SQL Developer Web, and Application Express (Apex) -* SHARDED: provision/deploy sharded databases and the shard topology, add a new shard, delete an existing shard +* ADB-S: Provision, Bind, Start, Stop, terminate (soft/hard), scale (up/down), Manual Backup, Manual Restore +* ADB-D: provision, bind, start, stop, terminate (soft/hard), scale (up/down), Manual Backup, Manual Restore +* ACD: provision, bind, restart, terminate (soft/hard) +* SIDB: Provision, clone, patch (in-place/out-of-place), update database initialization parameters, update database configuration (Flashback, archiving), Oracle Enterprise Manager (EM) Express (a basic observability console), Oracle REST Data Service (ORDS) to support REST based SQL, PDB management, SQL Developer Web, and Application Express (Apex) +* SHARDED: Provision/deploy sharded databases and the shard topology, Add a new shard, Delete an existing shard * On-Premises Database: Bind to a CDB, Create a  PDB, Plug a  PDB, Unplug a PDB, Delete a PDB, Clone a PDB, Open/Close a PDB -* Database Cloud Service: Provision, Bind, Scale shapeUp/Down, Liveness Probe, On Demand backup +* Database Cloud Service: Provision, Bind, Scale Up/Down, Liveness Probe, Manual Backup The upcoming releases will support new configurations, operations and capabilities. @@ -33,7 +34,7 @@ The upcoming releases will support new configurations, operations and capabiliti **CAUTION:** The current release of `OraOperator` (v0.2.0) is for development and test only. DO NOT USE IN PRODUCTION. -This release has been installed and tested (not certified) on the following Kubernetes platforms: +This release has been installed and tested on the following Kubernetes platforms: * [Oracle Container Engine for Kubernetes (OKE)](https://www.oracle.com/cloud-native/container-engine-kubernetes/) with Kubernetes 1.17 or later * [Oracle Linux Cloud Native Environment(OLCNE)](https://docs.oracle.com/en/operating-systems/olcne/) 1.3 or later @@ -91,7 +92,7 @@ The quickstarts are designed for specific database configurations: * [Oracle Autonomous Database](./docs/adb/README.md) * [Oracle Autonomous Container Database](./docs/acd/README.md) * [Containerized Oracle Single Instance Database](./docs/sidb/README.md) -* [Containerized Oracle Shared Database](./docs/sharding/README.md) +* [Containerized Oracle Sharded Database](./docs/sharding/README.md) * [Oracle On-Premises Database](./docs/onpremdb/README.md) * [Oracle Database Cloud Service](./docs/dbcs/README.md) From e64990ea6f33fe58b501a9f8065a177eaa632d14 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Fri, 24 Jun 2022 16:33:05 +0000 Subject: [PATCH 454/628] Update README.md --- README.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index abfed9c8..2c0033e5 100644 --- a/README.md +++ b/README.md @@ -111,8 +111,10 @@ YAML file templates are available under [`/config/samples`](./config/samples/). kubectl delete singleinstancedatabase.database.oracle.com --all -n kubectl delete shardingdatabase.database.oracle.com --all -n kubectl delete autonomousdatabase.database.oracle.com --all -n + kubectl delete autonomousdatabasebackup.database.oracle.com --all -n + kubectl delete autonomousdatabaserestore.database.oracle.com --all -n + kubectl delete autonomouscontainerdatabase.database.oracle.com --all -n kubectl delete dbcssystem.database.oracle.com --all -n - kubectl delete acd -n ``` After all CRD instances are deleted, it is safe to remove the CRDs, APISerivces and operator deployment. From 773c403bbbdd429ad870f9e1e2d013aabcc1fd48 Mon Sep 17 00:00:00 2001 From: kuassi_mensah Date: Sat, 25 Jun 2022 14:50:31 +0000 Subject: [PATCH 455/628] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 2c0033e5..7766330f 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # Oracle Database Operator for Kubernetes -## Make Oracle Database Kubernetes Native - An Overview +## Make Oracle Database Kubernetes Native - Take 2 As part of Oracle's resolution to make Oracle Database Kubernetes-native (that is, observable and operable by Kubernetes), Oracle released _Oracle Database Operator for Kubernetes_ (`OraOperator`). OraOperator extends the Kubernetes API with custom resources and controllers for automating Oracle Database lifecycle management. From dcc0992b382407bce99d6502213ac6dcb7f8e03a Mon Sep 17 00:00:00 2001 From: kuassi_mensah Date: Sat, 25 Jun 2022 14:54:58 +0000 Subject: [PATCH 456/628] Update README.md --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 7766330f..d8d63691 100644 --- a/README.md +++ b/README.md @@ -41,6 +41,7 @@ This release has been installed and tested on the following Kubernetes platforms * [Minikube](https://minikube.sigs.k8s.io/docs/) with version v1.21.0 or later * [Azure Kubernetes Service](https://azure.microsoft.com/en-us/services/kubernetes-service/) * [Amazon Elastic Kubernetes Service](https://aws.amazon.com/eks/) +* [Red Hat OKD](https://www.okd.io/) * [Red Hat OpenShift](https://www.redhat.com/en/technologies/cloud-computing/openshift/) ## Prerequisites From a117ea84943a8545e6fadce73bc1c2d1711a8306 Mon Sep 17 00:00:00 2001 From: Kuassi Mensah Date: Mon, 27 Jun 2022 06:26:14 -0700 Subject: [PATCH 457/628] Update README.md Fixed a minor typo. --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 99845ab9..160ba65f 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,7 @@ As part of Oracle's resolution to make Oracle Database Kubernetes-native (that is, observable and operable by Kubernetes), Oracle is announcing _Oracle Database Operator for Kubernetes_ (`OraOperator`). -Since Oracle Database 19c, Oracle Database images have been supported in containers (Docker, Podman) for production use and Kubernetes deployment with Helm Charts. This release includes Oracle Database Operator, which is a new open source product that extends the Kubernetes API with custom resources and controllers for automating Oracle Database lifecycle management. +Since Oracle Database 19c, the Oracle Database images have been supported in containers (Docker, Podman) for production use and Kubernetes deployment with Helm Charts. This release includes Oracle Database Operator, which is a new open source product that extends the Kubernetes API with custom resources and controllers for automating Oracle Database lifecycle management. In this release, `OraOperator` supports the following Oracle Database configurations: From f1c813d590ae4d2207597684ea7e1cd3d8d0f4c4 Mon Sep 17 00:00:00 2001 From: kuassi_mensah Date: Mon, 27 Jun 2022 15:29:47 +0000 Subject: [PATCH 458/628] Fixed a minor typo. --- docs/dbcs/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/dbcs/README.md b/docs/dbcs/README.md index 275db9c0..b73bcc21 100644 --- a/docs/dbcs/README.md +++ b/docs/dbcs/README.md @@ -87,7 +87,7 @@ singleinstancedatabases.database.oracle.com 2022-02-22T23:23:25Z ``` -# Prerequsites to deploy a DBCS system using Oracle DB Operator DBCS Controller +# Prerequisites to deploy a DBCS system using Oracle DB Operator DBCS Controller Before you deploy a DBCS system in OCI using the Oracle DB Operator DBCS Controller, complete the following procedure. From fdaf049ad9a7126e2a91460f4e881d2f22eeeac9 Mon Sep 17 00:00:00 2001 From: Jyoti Verma Date: Tue, 28 Jun 2022 04:00:23 +0000 Subject: [PATCH 459/628] Jyoti onprem controller branch --- docs/onpremdb/README.md | 339 ++++++++---------- docs/onpremdb/images/K8S_SECURE1.png | Bin 0 -> 129940 bytes docs/onpremdb/images/K8S_SECURE2.png | Bin 0 -> 130742 bytes docs/onpremdb/images/K8S_SECURE3.png | Bin 0 -> 130498 bytes docs/onpremdb/images/K8S_SECURE4.png | Bin 0 -> 132342 bytes docs/onpremdb/provisioning/add_replica.log | 192 ++++++++++ docs/onpremdb/provisioning/add_replica.md | 36 ++ docs/onpremdb/provisioning/add_replica.yaml | 41 +++ docs/onpremdb/provisioning/cdb.log | 279 ++++++++++++++ docs/onpremdb/provisioning/cdb.yaml | 42 +++ .../onpremdb/provisioning/cdb_crd_resource.md | 38 ++ docs/onpremdb/provisioning/cdb_secret.log | 0 docs/onpremdb/provisioning/cdb_secret.yaml | 17 + docs/onpremdb/provisioning/clone_pdb.log | 137 +++++++ docs/onpremdb/provisioning/clone_pdb.md | 36 ++ docs/onpremdb/provisioning/clone_pdb.yaml | 20 ++ docs/onpremdb/provisioning/create_pdb.log | 139 +++++++ docs/onpremdb/provisioning/create_pdb.md | 37 ++ docs/onpremdb/provisioning/create_pdb.yaml | 27 ++ docs/onpremdb/provisioning/delete_pdb.log | 157 ++++++++ docs/onpremdb/provisioning/delete_pdb.md | 35 ++ docs/onpremdb/provisioning/delete_pdb.yaml | 16 + .../example_setup_using_oci_oke_cluster.md | 55 +++ docs/onpremdb/provisioning/known_issues.md | 49 +++ docs/onpremdb/provisioning/modify_pdb.log | 181 ++++++++++ docs/onpremdb/provisioning/modify_pdb.md | 67 ++++ .../provisioning/modify_pdb_close.yaml | 18 + .../provisioning/modify_pdb_open.yaml | 18 + docs/onpremdb/provisioning/ords_image.log | 263 ++++++++++++++ docs/onpremdb/provisioning/ords_image.md | 68 ++++ docs/onpremdb/provisioning/pdb.log | 0 docs/onpremdb/provisioning/pdb.yaml | 27 ++ .../onpremdb/provisioning/pdb_crd_resource.md | 0 docs/onpremdb/provisioning/pdb_secret.log | 0 docs/onpremdb/provisioning/pdb_secret.yaml | 13 + docs/onpremdb/provisioning/plug_pdb.log | 100 ++++++ docs/onpremdb/provisioning/plug_pdb.md | 42 +++ docs/onpremdb/provisioning/plug_pdb.yaml | 22 ++ docs/onpremdb/provisioning/unplug_pdb.log | 165 +++++++++ docs/onpremdb/provisioning/unplug_pdb.md | 37 ++ docs/onpremdb/provisioning/unplug_pdb.yaml | 17 + .../onpremdb/provisioning/validation_error.md | 73 ++++ 42 files changed, 2604 insertions(+), 199 deletions(-) create mode 100644 docs/onpremdb/images/K8S_SECURE1.png create mode 100644 docs/onpremdb/images/K8S_SECURE2.png create mode 100644 docs/onpremdb/images/K8S_SECURE3.png create mode 100644 docs/onpremdb/images/K8S_SECURE4.png create mode 100644 docs/onpremdb/provisioning/add_replica.log create mode 100644 docs/onpremdb/provisioning/add_replica.md create mode 100644 docs/onpremdb/provisioning/add_replica.yaml create mode 100644 docs/onpremdb/provisioning/cdb.log create mode 100644 docs/onpremdb/provisioning/cdb.yaml create mode 100644 docs/onpremdb/provisioning/cdb_crd_resource.md create mode 100644 docs/onpremdb/provisioning/cdb_secret.log create mode 100644 docs/onpremdb/provisioning/cdb_secret.yaml create mode 100644 docs/onpremdb/provisioning/clone_pdb.log create mode 100644 docs/onpremdb/provisioning/clone_pdb.md create mode 100644 docs/onpremdb/provisioning/clone_pdb.yaml create mode 100644 docs/onpremdb/provisioning/create_pdb.log create mode 100644 docs/onpremdb/provisioning/create_pdb.md create mode 100644 docs/onpremdb/provisioning/create_pdb.yaml create mode 100644 docs/onpremdb/provisioning/delete_pdb.log create mode 100644 docs/onpremdb/provisioning/delete_pdb.md create mode 100644 docs/onpremdb/provisioning/delete_pdb.yaml create mode 100644 docs/onpremdb/provisioning/example_setup_using_oci_oke_cluster.md create mode 100644 docs/onpremdb/provisioning/known_issues.md create mode 100644 docs/onpremdb/provisioning/modify_pdb.log create mode 100644 docs/onpremdb/provisioning/modify_pdb.md create mode 100644 docs/onpremdb/provisioning/modify_pdb_close.yaml create mode 100644 docs/onpremdb/provisioning/modify_pdb_open.yaml create mode 100644 docs/onpremdb/provisioning/ords_image.log create mode 100644 docs/onpremdb/provisioning/ords_image.md create mode 100644 docs/onpremdb/provisioning/pdb.log create mode 100644 docs/onpremdb/provisioning/pdb.yaml create mode 100644 docs/onpremdb/provisioning/pdb_crd_resource.md create mode 100644 docs/onpremdb/provisioning/pdb_secret.log create mode 100644 docs/onpremdb/provisioning/pdb_secret.yaml create mode 100644 docs/onpremdb/provisioning/plug_pdb.log create mode 100644 docs/onpremdb/provisioning/plug_pdb.md create mode 100644 docs/onpremdb/provisioning/plug_pdb.yaml create mode 100644 docs/onpremdb/provisioning/unplug_pdb.log create mode 100644 docs/onpremdb/provisioning/unplug_pdb.md create mode 100644 docs/onpremdb/provisioning/unplug_pdb.yaml create mode 100644 docs/onpremdb/provisioning/validation_error.md diff --git a/docs/onpremdb/README.md b/docs/onpremdb/README.md index 494b2a42..2f728455 100644 --- a/docs/onpremdb/README.md +++ b/docs/onpremdb/README.md @@ -1,262 +1,203 @@ -# Oracle On-Premise Database Controller +# Oracle On-Prem Database Controller -The On-Premise Database Controller enables provisioning of Oracle Databases (PDBs) both on a Kubernetes cluster or outside of a Kubernetes cluster. The following sections explain the setup and functionality of this controller: +CDBs and PDBs are the part of the Oracle Database's [Multitenant Architecture](https://docs.oracle.com/en/database/oracle/oracle-database/21/multi/introduction-to-the-multitenant-architecture.html#GUID-AB84D6C9-4BBE-4D36-992F-2BB85739329F). The On-Prem Database Controller is a feature of Oracle DB Operator for Kubernetes (OraOperator) which helps to manage the life cycle of Pluggable Databases (PDBs) in an Oracle Container Database(CDB). -* [Prerequisites for On-Premise Database Controller](ORACLE_ONPREMDB_CONTROLLER_README.md#prerequisites-and-setup) -* [Kubernetes Secrets](ORACLE_ONPREMDB_CONTROLLER_README.md#kubernetes-secrets) -* [Kubernetes CRD for CDB](ORACLE_ONPREMDB_CONTROLLER_README.md#kubernetes-crd-for-cdb) -* [Kubernetes CRD for PDB](ORACLE_ONPREMDB_CONTROLLER_README.md#kubernetes-crd-for-pdb) -* [PDB Lifecycle Management Operations](ORACLE_ONPREMDB_CONTROLLER_README.md#pdb-lifecycl-management-operations) -* [Validation and Errors](ORACLE_ONPREMDB_CONTROLLER_README.md#validation-and-errors) +The target CDB (for which the PDB life cycle management is needed) can be running on an on-prem machine and to manage its PDBs, the Oracle DB Operator can run on an on-prem Kubernetes system (For Example: [Oracle Linux Cloud Native Environment or OLCNE](https://docs.oracle.com/en/operating-systems/olcne/)). -## Prerequisites for On-Premise Database Controller +NOTE: The target CDB (for which the PDB life cycle management is needed) **can also** run in a Cloud environment as well (For Example: OCI's [Oracle Base Database Service](https://docs.oracle.com/en-us/iaas/dbcs/doc/bare-metal-and-virtual-machine-db-systems.html)) and to manage its PDBs, the Oracle DB Operator can run on a Kubernetes Cluster running in cloud (For Example: OCI's [Container Engine for Kubernetes or OKE](https://docs.oracle.com/en-us/iaas/Content/ContEng/Concepts/contengoverview.htm#Overview_of_Container_Engine_for_Kubernetes)) -+ ### Prepare CDB for PDB Lifecycme Management (PDB-LM) - Pluggable Database management is performed in the Container Database (CDB) and includes create, clone, plug, unplug, delete, modify and map operations. - You cannot have an ORDS enabled schema in the container database. To perform the PDB lifecycle management operations, the default CDB administrator credentials must be defined. - To define the default CDB administrator credentials, perform the following steps on the target CDB(s) where PDB-LM operations are to be performed: +# Oracle DB Operator On-Prem Database Controller Deployment - Create the CDB administrator user and grant the SYSDBA privilege. In this example, the user is called C##DBAPI_CDB_ADMIN. However, any suitable common user name can be used. +To deploy OraOperator, use this [Oracle Database Operator for Kubernetes](https://github.com/oracle/oracle-database-operator/blob/main/README.md) step-by-step procedure. - ```sh - CREATE USER C##DBAPI_CDB_ADMIN IDENTIFIED BY ; - GRANT SYSOPER TO C##DBAPI_CDB_ADMIN CONTAINER = ALL; - ``` -+ ### Building the Oracle REST Data Service (ORDS) Image +After the Oracle Database Operator is deployed, you can see the DB Operator Pods running in the Kubernetes Cluster. As part of the OraOperator deployment, the On-Prem Database Controller is deployed as a CRD (Custom Resource Definition). The following output is an example of such a deployment: +``` +[root@test-server oracle-database-operator]# kubectl get ns +NAME STATUS AGE +cert-manager Active 32h +default Active 245d +kube-node-lease Active 245d +kube-public Active 245d +kube-system Active 245d +oracle-database-operator-system Active 24h <<<< namespace to deploy the Oracle Database Operator + + +[root@test-server oracle-database-operator]# kubectl get all -n oracle-database-operator-system +NAME READY STATUS RESTARTS AGE +pod/oracle-database-operator-controller-manager-665874bd57-dlhls 1/1 Running 0 28s +pod/oracle-database-operator-controller-manager-665874bd57-g2cgw 1/1 Running 0 28s +pod/oracle-database-operator-controller-manager-665874bd57-q42f8 1/1 Running 0 28s + +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +service/oracle-database-operator-controller-manager-metrics-service ClusterIP 10.96.130.124 8443/TCP 29s +service/oracle-database-operator-webhook-service ClusterIP 10.96.4.104 443/TCP 29s + +NAME READY UP-TO-DATE AVAILABLE AGE +deployment.apps/oracle-database-operator-controller-manager 3/3 3 3 29s + +NAME DESIRED CURRENT READY AGE +replicaset.apps/oracle-database-operator-controller-manager-665874bd57 3 3 3 29s +[root@docker-test-server oracle-database-operator]# + + +[root@test-server oracle-database-operator]# kubectl get crd +NAME CREATED AT +autonomouscontainerdatabases.database.oracle.com 2022-06-22T01:21:36Z +autonomousdatabasebackups.database.oracle.com 2022-06-22T01:21:36Z +autonomousdatabaserestores.database.oracle.com 2022-06-22T01:21:37Z +autonomousdatabases.database.oracle.com 2022-06-22T01:21:37Z +cdbs.database.oracle.com 2022-06-22T01:21:37Z <<<< CRD for CDB +certificaterequests.cert-manager.io 2022-06-21T17:03:46Z +certificates.cert-manager.io 2022-06-21T17:03:47Z +challenges.acme.cert-manager.io 2022-06-21T17:03:47Z +clusterissuers.cert-manager.io 2022-06-21T17:03:48Z +dbcssystems.database.oracle.com 2022-06-22T01:21:38Z +issuers.cert-manager.io 2022-06-21T17:03:49Z +oraclerestdataservices.database.oracle.com 2022-06-22T01:21:38Z +orders.acme.cert-manager.io 2022-06-21T17:03:49Z +pdbs.database.oracle.com 2022-06-22T01:21:39Z <<<< CRD for PDB +shardingdatabases.database.oracle.com 2022-06-22T01:21:39Z +singleinstancedatabases.database.oracle.com 2022-06-22T01:21:40Z +``` - Oracle On-Premise Database controller enhances the Oracle REST Data Services (ORDS) image to enable it for PDB Lifecycle Management. You can build this image by following the instructions below: - * After cloning the repository, go to the "ords" folder, and run: - ```sh - docker build -t oracle/ords-dboper:latest . - ``` +The following sections explain the setup and functionality of this controller. - > **_NOTE1:_** Required file to build this image is Oracle Rest Data Services 'ords-< version >.zip', - you can download such file from http://www.oracle.com/technetwork/developer-tools/rest-data-services/downloads/index.html - > **_NOTE2:_** to build the ords image you need to Accept the Oracle Standard Terms and Restrictions for Java/jdk repository on Oracle Container Registry at https://container-registry.oracle.com +# Prerequsites to manage PDB Life Cycle using Oracle DB Operator On-Prem Database Controller - > **_NOTE3:_** docker needs access to container-registry.oracle.com for which following command may be required: - ```sh - docker login container-registry.oracle.com - ``` +Before you want to manage the life cycle of a PDB in a CDB using the Oracle DB Operator On-Prem Database Controller, complete the following steps. - * Once the image is ready, you may need to push it to your Docker Images Repository to pull it during CDB controller resource creation. +**CAUTION :** You must make the changes specified in this section before you proceed to the next section. - The steps - 1. check the ords docker image creation issuing: - ```sh - docker images - ``` - example: - ``` - $ docker images - REPOSITORY TAG IMAGE ID CREATED SIZE - oracle/ords-dboper latest 887652b3e87f 17 hours ago 777MB - ``` +* [Prepare CDB for PDB Lifecycle Management or PDB-LM](#prepare-cdb-for-pdb-lifecycle-management-pdb-lm) +* [Oracle REST Data Service or ORDS Image](#oracle-rest-data-service-ords-image) +* [Kubernetes Secrets](#kubernetes-secrets) +* [Kubernetes CRD for CDB](#kubernetes-crd-for-cdb) +* [Kubernetes CRD for PDB](#kubernetes-crd-for-pdb) - 2. make the required docker tag issuing: - ``` - docker tag oracle/ords-dboper - ``` - example: - ``` - docker tag oracle/ords-dboper lin.ocir.io/mytenancy/mycontainer/myrepo/ords-dboper - ``` - 3. Push the image over the repository - ``` - docker push - ``` - example: - ``` - docker push lin.ocir.io/mytenancy/mycontainer/myrepo/ords-dboper:latest - ``` ++ ## Prepare CDB for PDB Lifecycle Management (PDB-LM) -+ ### Install cert-manager +Pluggable Database management operation is performed in the Container Database (CDB) and it includes create, clone, plug, unplug, delete, modify and map operations. - Validating webhook is an endpoint Kubernetes can invoke prior to persisting resources in ETCD. This endpoint returns a structured response indicating whether the resource should be rejected or accepted and persisted to the datastore. +You cannot have an ORDS enabled schema in the container database. To perform the PDB lifecycle management operations, the default CDB administrator credentials must be defined by performing the below steps on the target CDB(s): - Webhook requires a TLS certificate that the apiserver is configured to trust . Install the cert-manager with the following command: +Create the CDB administrator user and grant the required privileges. In this example, the user is `C##DBAPI_CDB_ADMIN`. However, any suitable common user name can be used. - ```sh - kubectl apply -f https://github.com/jetstack/cert-manager/releases/latest/download/cert-manager.yaml - ``` +Also, create a user named SQL_ADMIN. You can use the below set of SQL commands for this purpose: -## Kubernetes Secrets +``` +SQL> conn /as sysdba + +-- Create the below users at the database level: + +DROP USER SQL_ADMIN CASCADE; +CREATE USER SQL_ADMIN IDENTIFIED BY welcome1; +ALTER SESSION SET "_oracle_script"=true; +DROP USER C##DBAPI_CDB_ADMIN cascade; +CREATE USER C##DBAPI_CDB_ADMIN IDENTIFIED BY CONTAINER=ALL ACCOUNT UNLOCK; +GRANT SYSOPER TO C##DBAPI_CDB_ADMIN CONTAINER = ALL; +GRANT SYSDBA TO C##DBAPI_CDB_ADMIN CONTAINER = ALL; +GRANT CREATE SESSION TO C##DBAPI_CDB_ADMIN CONTAINER = ALL; +GRANT CREATE SESSION TO SQL_ADMIN; +GRANT SYSDBA TO SQL_ADMIN; + + +-- Verify the account status of the below usernames. They should not be in locked status: + +col username for a30 +col account_status for a30 +select username, account_status from dba_users where username in ('ORDS_PUBLIC_USER','SQL_ADMIN','C##DBAPI_CDB_ADMIN','APEX_PUBLIC_USER','APEX_REST_PUBLIC_USER'); +``` - Both CDBs and PDBs make use of Kubernetes Secrets to store usernames and passwords. - For CDB, create a secret file as shown here: [config/samples/onpremdb/cdb_secret.yaml](../../config/samples/onpremdb/cdb_secret.yaml) - ```sh - $ kubectl apply -f cdb_secret.yaml - secret/cdb1-secret created - ``` - > **_NOTE:_** the entries such user and password must be 'base64 encoded', example: - "cdbadmin_user=C##DBAPI_CDB_ADMIN" become cdbadmin_user: "QyMjREJBUElfQ0RCX0FETUlOCg==" using command: +### Reference Setup: Example of a setup using OCI OKE(Kubernetes Cluster) and a CDB in Cloud (OCI Exadata Database Cluster) - ```sh - echo C##DBAPI_CDB_ADMIN | base64 - QyMjREJBUElfQ0RCX0FETUlOCg== - ``` +Please refer [here](./provisioning/example_setup_using_oci_oke_cluster.md) for steps to configure a Kubernetes Cluster and a CDB. This example uses an OCI OKE Cluster as the Kubernetes Cluster and a CDB in OCI Exadata Database service. - On successful creation of the CDB Resource, the CDB secrets would be deleted from the Kubernetes system. - For PDB, create a secret file as shown here: [config/samples/onpremdb/pdb_secret.yaml](../../config/samples/onpremdb/pdb_secret.yaml) ++ ## Oracle REST Data Service (ORDS) Image - ```sh - $ kubectl apply -f pdb_secret.yaml - secret/pdb1-secret created - ``` - **Note:** Don't leave plaintext files containing sensitive data on disk. After loading the Secret, remove the plaintext file or move it to secure storage. + Oracle DB Operator On-Prem Database controller requires the Oracle REST Data Services (ORDS) image for PDB Lifecycle Management in the target CDB. + + You can build this image by using the ORDS [Dockerfile](../../../ords/Dockerfile) + + > **_NOTE:_** Download the required binaries to build this image i.e. Oracle Rest Data Services 'ords-< version >.zip', from http://www.oracle.com/technetwork/developer-tools/rest-data-services/downloads/index.html - Another option is to use "kubectl create secret" command as shown below for the PDB: - ```sh - $ kubectl create secret generic pdb1-secret --from-literal sysadmin_user=pdbadmin --from-literal sysdamin_pwd=WE2112#232# - secret/pdb1-secret created - ``` + > **_NOTE:_** The current version of Oracle DB Operator On-Prem Controller has been tested with `ORDS 21.4.3` version. -## Kubernetes CRD for CDB +Please refer [here](./provisioning/ords_image.md) for the steps to build ORDS Docker Image with `ORDS 21.4.3` version. - The Oracle Database Operator creates the CDB kind as a custom resource that models a target CDB as a native Kubernetes object. This is only used to create Pods to connect to the target CDB to perform PDB-LM operations. These CDB resources can be scaled up and down based on the expected load using replicas. Each CDB resource follows the CDB CRD as defined here: [config/crd/bases/database.oracle.com_cdbs.yaml](../../config/crd/bases/database.oracle.com_cdbs.yaml) - + ### CDB Sample YAML ++ ## Kubernetes Secrets - A sample .yaml file is available here: [config/samples/onpremdb/cdb.yaml](../../config/samples/onpremdb/cdb.yaml) + Oracle DB Operator On-Prem Database Controller uses Kubernetes Secrets to store usernames and passwords to manage the life cycle operations of a PDB in the target CDB. - **Note:** The password and username fields in the above `cdb.yaml` yaml are Kubernetes Secrets. Please see the section [Kubernetes Secrets](ORACLE_ONPREMDB_CONTROLLER_README.md#kubernetes-secrets) for more information. +### Secrets for CDB CRD - + ### Check the status of the all CDBs - ```sh - $ kubectl get cdbs -A + Create a secret file as shown here: [config/samples/onpremdb/cdb_secret.yaml](../../config/samples/onpremdb/cdb_secret.yaml). Modify this file with the `base64` encoded values of the required passwords for CDB and use it to create the required secrets. - NAMESPACE NAME CDB NAME DB SERVER DB PORT SCAN NAME REPLICAS STATUS MESSAGE - oracle-database-operator-system cdb-dev devdb 172.17.0.4 1521 devdb 1 Ready Success - ``` - + ### Scale the CDB resource ```sh - $ kubectl patch --type=merge cdb cdb-dev -p '{"spec":{"replicas":3}}' -n oracle-database-operator-system + $ kubectl apply -f cdb_secret.yaml ``` + + **Note:** In order to get the base64 encoded value for a password, please use the below command like below at the command prompt. The value you get is the base64 encoded value for that password string. -## Kubernetes CRD for PDB - - The Oracle Database Operator creates the PDB kind as a custom resource that models a PDB as a native Kubernetes object. There is a one-to-one mapping between the actual PDB and the Kubernetes PDB Custom Resource. You cannot have more than one Kubernetes resource for a target PDB. This PDB resource can be used to perform PDB-LM operations by specifying the action attribute in the PDB specs. Each PDB resource follows the PDB CRD as defined here: [config/crd/bases/database.oracle.com_pdbs.yaml](../../config/crd/bases/database.oracle.com_pdbs.yaml) - - - + ### PDB Sample YAML + ```sh + echo -n "" | base64 + ``` - A sample .yaml file is available here: [config/samples/onpremdb/pdb.yaml](../../config/samples/onpremdb/pdb.yaml) + **Note:** On successful creation of the CDB Resource, the CDB secrets would be deleted from the Kubernetes system. - **Note:** The password and username fields in the above `pdb.yaml` yaml are Kubernetes Secrets. Please see the section [Kubernetes Secrets](ORACLE_ONPREMDB_CONTROLLER_README.md#kubernetes-secrets) for more information. +### Secrets for PDB CRD + Create a secret file as shown here: [config/samples/onpremdb/pdb_secret.yaml](../../config/samples/onpremdb/pdb_secret.yaml). Modify this file with the `base64` encoded values of the required passwords for PDB and use it to create the required secrets. - + ### Check the status of the all PDBs ```sh - $ kubectl get pdbs -A - - NAMESPACE NAME CONNECT STRING CDB NAME PDB NAME PDB SIZE STATUS MESSAGE - oracle-database-operator-system pdb1 devdb:1521/pdbdev cdb-dev pdbdev 2G Ready Success - oracle-database-operator-system pdb2 testdb:1521/pdbtets cdb-test pdbtes 1G Ready Success + $ kubectl apply -f pdb_secret.yaml ``` + **NOTE:** Refer to command provided above to encode the password using base64. + + **NOTE:** Don't leave plaintext files containing sensitive data on disk. After loading the Secret, remove the plaintext file or move it to secure storage. + -## PDB Lifecycle Management Operations ++ ## Kubernetes CRD for CDB - Using ORDS, you can perform the following PDB-LM operations: CREATE, CLONE, PLUG, UNPLUG and DELETE +The Oracle Database Operator On-Prem Controller creates the CDB kind as a custom resource that models a target CDB as a native Kubernetes object. This is used only to create Pods to connect to the target CDB to perform PDB-LM operations. These CDB resources can be scaled up and down based on the expected load using replicas. Each CDB resource follows the CDB CRD as defined here: [config/crd/bases/database.oracle.com_cdbs.yaml](../../config/crd/bases/database.oracle.com_cdbs.yaml) -+ ### Create PDB +To create a CDB CRD, a sample .yaml file is available here: [config/samples/onpremdb/cdb.yaml](../../config/samples/onpremdb/cdb.yaml) - A sample .yaml file is available here: [config/samples/onpremdb/pdb.yaml](../../config/samples/onpremdb/pdb.yaml) +**Note:** The password and username fields in this `cdb.yaml` yaml are Kubernetes Secrets created earlier. Please see the section [Kubernetes Secrets](ORACLE_ONPREMDB_CONTROLLER_README.md#kubernetes-secrets) for more information. -+ ### Clone PDB +1. [Use Case: Create a CDB CRD Resource](./provisioning/cdb_crd_resource.md) +2. [Use Case: Add another replica to an existing CDB CRD Resource](./provisioning/add_replica.md) - A sample .yaml file is available here: [config/samples/onpremdb/pdb_clone.yaml](../../config/samples/onpremdb/pdb_clone.yaml) -+ ### Plug PDB ++ ## Kubernetes CRD for PDB - A sample .yaml file is available here: [config/samples/onpremdb/pdb_plug.yaml](../../config/samples/onpremdb/pdb_plug.yaml) +The Oracle Database Operator On-Prem Controller creates the PDB kind as a custom resource that models a PDB as a native Kubernetes object. There is a one-to-one mapping between the actual PDB and the Kubernetes PDB Custom Resource. You cannot have more than one Kubernetes resource for a target PDB. This PDB resource can be used to perform PDB-LM operations by specifying the action attribute in the PDB Specs. Each PDB resource follows the PDB CRD as defined here: [config/crd/bases/database.oracle.com_pdbs.yaml](../../../config/crd/bases/database.oracle.com_pdbs.yaml) -+ ### Unplug PDB +To create a PDB CRD Resource, a sample .yaml file is available here: [config/samples/onpremdb/pdb.yaml](../../../config/samples/onpremdb/pdb.yaml) - A sample .yaml file is available here: [config/samples/onpremdb/pdb_unplug.yaml](../../config/samples/onpremdb/pdb_unplug.yaml) +# Use Cases for PDB Lifecycle Management Operations using Oracle DB Operator On-Prem Controller -+ ### Delete PDB +Using Oracle DB Operator On-Prem Controller, you can perform the following PDB-LM operations: CREATE, CLONE, MODIFY, DELETE, STATUS, PLUG, UNPLUG - A sample .yaml file is available here: [config/samples/onpremdb/pdb_delete.yaml](../../config/samples/onpremdb/pdb_delete.yaml) +1. [Create PDB](./provisioning/create_pdb.md) +2. [Clone PDB](./provisioning/clone_pdb.md) +3. [Modify PDB](./provisioning/modify_pdb.md) +4. [Delete PDB](./provisioning/delete_pdb.md) +5. [Unplug PDB](./provisioning/unplug_pdb.md) +6. [Plug PDB](./provisioning/plug_pdb.md) - You can also use the following cmd to delete an existing PDB: - ```sh - $ kubectl patch --type=merge pdb pdb1 -p '{"spec":{"action":"Delete","dropAction":"INCLUDING"}}' -n oracle-database-operator-system - ``` - -+ ### Modify PDB - - This is used to open/close a target PDB. - A sample .yaml file is available here: [config/samples/onpremdb/pdb_modify.yaml](../../config/samples/onpremdb/pdb_modify.yaml) - - You can also use the following cmd to modify an existing PDB: - ```sh - $ kubectl patch --type=merge pdb pdb1 -p '{"spec":{"action":"Modify","modifyOption":"IMMEDIATE","pdbState":"CLOSE"}}' -n oracle-database-operator-system - ``` - -+ ### Map PDB - - This is used to map an existing PDB in the CDB as a Kubernetes Custom Resource. - A sample .yaml file is available here: [config/samples/onpremdb/pdb_map.yaml](../../config/samples/onpremdb/pdb_map.yaml) ## Validation and Errors -You can check Kubernetes events for any errors or status updates as shown below: -```sh -$ kubectl get events -A -NAMESPACE LAST SEEN TYPE REASON OBJECT MESSAGE -oracle-database-operator-system 58m Warning Failed pod/cdb-dev-ords-qiigr Error: secret "cdb1-secret" not found -oracle-database-operator-system 56m Normal DeletedORDSPod cdb/cdb-dev Deleted ORDS Pod(s) for cdb-dev -oracle-database-operator-system 56m Normal DeletedORDSService cdb/cdb-dev Deleted ORDS Service for cdb-dev -... -oracle-database-operator-system 26m Warning OraError pdb/pdb1 ORA-65016: FILE_NAME_CONVERT must be specified... -oracle-database-operator-system 24m Warning OraError pdb/pdb2 ORA-65011: Pluggable database DEMOTEST does not exist. -... -oracle-database-operator-system 20m Normal Created pdb/pdb1 PDB 'demotest' created successfully -... -oracle-database-operator-system 17m Warning OraError pdb/pdb3 ORA-65012: Pluggable database DEMOTEST already exists... -``` - -+ ### CDB Validation and Errors +Please check [here](./provisioning/validation_error.md) for the details to look for any validation error. -Validation is done at the time of CDB resource creation as shown below: -```sh -$ kubectl apply -f cdb1.yaml -The PDB "cdb-dev" is invalid: -* spec.dbServer: Required value: Please specify Database Server Name or IP Address -* spec.dbPort: Required value: Please specify DB Server Port -* spec.ordsImage: Required value: Please specify name of ORDS Image to be used -``` -Apart from events, listing of CDBs will also show the possible reasons why a particular CDB CR could not be created as shown below: -```sh - $ kubectl get cdbs -A +## Known issues - NAMESPACE NAME CDB NAME DB SERVER DB PORT SCAN NAME STATUS MESSAGE - oracle-database-operator-system cdb-dev devdb 172.17.0.4 1521 devdb Failed Secret not found:cdb1-secret -``` - -+ ### PDB Validation and Errors - -Validation is done at the time of PDB resource creation as shown below: -```sh -$ kubectl apply -f pdb1.yaml -The PDB "pdb1" is invalid: -* spec.cdbResName: Required value: Please specify the name of the CDB Kubernetes resource to use for PDB operations -* spec.pdbName: Required value: Please specify name of the PDB to be created -* spec.adminPwd: Required value: Please specify PDB System Administrator Password -* spec.fileNameConversions: Required value: Please specify a value for fileNameConversions. Values can be a filename convert pattern or NONE -``` - -Similarly, for PDBs, listing of PDBs will also show the possible reasons why a particular PDB CR could not be created as shown below: -```sh -$ kubectl get pdbs -A -NAMESPACE NAME CONNECT STRING CDB NAME PDB NAME PDB SIZE STATUS MESSAGE -oracle-database-operator-system pdb1 democdb demotest1 Failed Secret not found:pdb12-secret -oracle-database-operator-system pdb2 democdb demotest2 Failed ORA-65016: FILE_NAME_CONVERT must be specified... -``` +Please refer [here](./provisioning/known_issues.md) for the known issues related to Oracle DB Operator On-Prem Controller. diff --git a/docs/onpremdb/images/K8S_SECURE1.png b/docs/onpremdb/images/K8S_SECURE1.png new file mode 100644 index 0000000000000000000000000000000000000000..292c9335d1ebace9276875171c30c70566174673 GIT binary patch literal 129940 zcmd?RhgVbIwg!r5KtV*MNk_3DRS`iznkXnudM{E!54}UE3Mx%SdIt%;cS8N4bO^l# z2t7a`bOMCH+njsOy?4BK)j!}lMj&Ko@3q!kvwq)Peo|JFr66M zI}s7_&Setdlg?Q~c_JdRX)9@Ih{OlmQG&L;uF}tA9O^7TX9vX3LZp1>pH3foKB_s*mqt5x z2f2xqWpBV+%D6i(p~OHT#6un+wk%D ziF)_9>j4E<7f-M-zALIF&x8eR6JDLX zIeSCfbulA5{=u$NoVobegV&`(>;fJMYA=16UL|oW4GWN)HGUo9Hn^GI5q)Rh{OuDh zktnScmckNp5Ywv@CS&z=0gF(R7uh;7Ec5~k3xy@}OE*3rgG~I2RMbqB`|SGr0FLXFv=>;e2P&|Ex|=ym zt}O<=D4|#<&PpR*CnasU;c!(TKy8+(=2B6Bk^|#&qK0=w^+fEm%(sHrE|aRTG5ol& z`a4G5sWa)QusPo=ir?aL<^$V|%4TGkGj?-?h zJl(q@(_$8P&8D?&OFs@&KF9Wq%E5%dr*Uv z?^0jP3MqS8syvhPNS&AybnjK66iwP^6^$G(b@DuBSE`Xv1{vr}(*XeiD#$0dSEOl~ z1JCzV_ZarfT^=5@)!l$zfKp9|ut@p+Qhua+i#&`xIOtRB50e<xLVp}RNVq%JBXyuJC=>CL?#nLoOJh!4t^aZV(y zD=OXX?$qmex~RD$w!=vF!9-#E$6e(z4n2;_@4MgIzf;Ny+~c|zans1EP&HmPQ?+_f z9voQqm0!>Et5bwi>5k(PV^^y>@krd7;*#o^a(B*yT(MylnuO2ca&dC7F7s$?|XJhTShiX-7dZ(>^}B#k-E;0Hy2sSA}zNQw;K57u<|e%WF5k@XRi z;h6mB++v-l6DZd-;rGvMT`Zjs z=T{avTnpTpocLYjoH*Urj0be)3VI8=`)*nAp)32clO*b%)tS{YIUlu-3_hLS|ALu) z9_1P6NxHlbUxhnPb546r$7A1MQE;5sil>8E-t!a>29F)Fe6iHh-f^q-*7aD!QA5>Y zD#OhNgL*CR>fr#|6Dkj7#joPCjz^Bd4@D1)_v|T}FI@uJQm)KdI(sDP z_PtJ1n_X)rms(8gNE=Lh(6ZbzI2&{Q+1*EGvK_6;naYlqYPT*gyx+L&O>K9xQQ!o9 zG=4O<+r2CiUh+wsW`N~Qs6LY?$Bw~wt>Mue(gE1O&Pvs-rMRDQlX19G@zPhN4UWzW zn(YRmV_|f^gdWV^G`X2aB|~?}J6`#YZQ%A}iS51@Q6IYL40V3ga+nUR{^|;?4UK$Y z{LLil+CAuUuw#BrW@pmk&f<*}A$y@s5GtxEO5&~q74?;rtePz9tgEoPEbMFMn_ibg zKeXPlSIJ$`(3RAEHHqIV*-O~l4jT^(R`4-bU4B!V_sqb@_*37vCHcC$wBLN*Y^ug) z!DAv`2mKh~e_f?Kn)FJRBn#uDb|kdZXp37fTc_+ciD5P>D$}n&IVRhsNz{Daz1@9e ze9!BiJs8Q^pn=icFElSM(!W^v#&D==^c`eoGC==qDhTrtGdM;loAH7U99J`%2vsav zp_i8T{T9v?4H{J&1)3}}F>Rmi2T-3IH^$AXtq`zc&`DtahtUteKBNlm3K|jmmf_zv z+p2%qu8yzOxS|Bt1-Gk2kz!S>+^P?|96a$f=rzKt z@7DbAjGRk>c#Fr9$=kec+f<*hsvOcoTt_S>(@P{eyH#imiumyE>-!<+t4PrceW`w9T|wA`s^gUCpLC+xp=jd48aDFx@C{T?Yh=|s-|ZrP8qGGncKlb;mbhu|t>-E&dbARmUJb{-MAagr#)rIP{oJ<<+ZwcR<(rXW9ng=S+FKuwa}G;K z7xIjHeIYBZ>tfUBA$P)?#VArj{V82jr!W5b7u@%qio0St$bxd|%hQ#oVQFznLiOhmxuO7Y{k~xK)MIY8OK%Vq=P>j94QqA|mB$RI!U3{x7JN-dvbrVBaSm)J;JKI#*7JutuiGWv$zvka9KqN;MWm8O?5CKjSJ zp~LUP?*Cd?u>W#&!-AwaHf4Le6t=z1H}bj^_f>d#dRSlL_Hb)Qq5_NPKerEWCyOT= z-Bx>YX5@8(CcDpBLVBiucFCoJZt#8WPz;doYJ#=jm@6t0aRToz6I}?gBDw^;y8yfx zfEN+b#c#nxq`>c+!0Tli@qez8x~EhwvJk*_(iQJs=L}w-7z|5CPsoz%IsY9uQkQXAuvvJOBEI2=M;= zW4=3V|N4rHjo2M+MP)W=dnYiPAnzmIM|Z@@*x1;fJDHh_sJ@i>_wB%6Vs|WETpUFB z_}tyydEKAz+B;eBJr))g=6l4?$Is6Le1pf?)6T`%gU8O9{l6aapXa;;JDWOLIk;Ha z+p(QL*Vx3~)kW;io%4bI``>@f6YOF2KO@;W|9e@$0{PCb@IB^z#P{FN25xY@JYxP6=Ma>M%!^@pl7)E5{J3`-qoM72xCY>x6h8mhPscwhY@9xn+XR;3~5 z0A%x8afNyH>3hl3-9(Rh^9hQ?FYmu^9&*FIyOQD&@yELA4!ZdAc~p@V z(}(x;AS+4X>2R}E=L#<^yBVtojj5K=Mv35TpR)cgF|ppdaD4Yza@2jsZ*DqMPlj-o zyKt(pLvcRcCU$6YpQnE8!&Ou!w4;0bMj^!!#B%}bY}=Oi<7Bkp>W0cATz%xSUT-zk zMiPQn_g8kdUhx$he%q)aqZjQ?n#EbT2Zfd8$9>4mH|eg}(w$AvKwHjQ7#^V>iV>MM zqH`?nDa-cLFCAmm`Z;*e@3xvEI694V%6q>3lWH_ep3Zx3Kz7K;$Y=YNl16Hw*FVU; zf5`d9`I=73#_Cu+W&;}RT1kE{P-@zM8gADg>W*-eEq;8y%By@;!2<*MXViXL`XBL}9ZBZoAQ0l9FOQiZGSCijqYaD4dj(;PqIw% z`axe6@?f}B4L-P|pw%O}Y7a1?=s}h_h;D;dWy9W}{Ad|iK3_dom6ti%`o?cRTwpif|4`e>M8r|)%Ve!c{|bF*EGd|jV?)SG_*~0|DlNUdea0t7z*SWT~4kQ ze>N}VcE%ySH(5-pliF#+W4v+A+jq~pX>E;DyqHE>^noRKSP)+@Wn=SF8l1x)Q0Xz}xjmeRO}THt|Hfi!5xE7X0A2ss>sY zE?=%Fq1+L2jb|PbBpNxujT01?>6u=6A{E4mAUHHqB*ZV{yZw$$`Didg*2? ze!F0afpwCPBUMQc-EO%u-r&8ZI^@eaq-lqX&l%FX3wB7vz?K^4cR%>^_A-n#6|pU3Lgv zq2kxJ)OFrdmjG0D067!sSoPYqh8VkBCF3%1Z)s&0qfbwz#dFl~Yb+`nW692|vt?ha zPoXS9-f^(hzJlQb?ZIOMO@vpt-a_HinZcrJdbd1A%H6SzEnHB4Rla!m#bk@{8~d+d z=^JDb@oEb~IedfoA8(?3lr`eZB~A}B_cu`}`_b&Ttnvy96+$x}t+afGwnH$*@>nC^ ziHhkQmCO@CWVL-Y4&{fcbK6*peVdvRvvh{vY(hFC#4O%Q5GwXJkYnGXGEs;?L5*U= z={)sZ6GlPX5)e7_Bo#Nv1(mPgOLxaqXcK5DYj zT4VAoagb}_;_rhR=rx;7i@{jw14Irum2bv`ooX;JP|E1cApnMKGO}uNwX@u{v$vL} z5<=1qjUm58Q1o_1mpl{_EQ688l#1|5KEGpJj@rnKPh3x?w^E}%TO|Jew7Y%6ESBKD zlm-^uSv!t#F>+3I^}^3qNUpx7`|0H}EQj&*WfslQ)G?C%7RsBv0ShiPs{LVrf+av4 zThb&>uQi7m#hYd7H(SKmAmX|)df5T;1aUsNRa7>ivUPOI`6ESLS)4n?>vvLQr!zAj8Dosqn#wh$`U;g%CRK<&bNxBTlG!Di0UTZF(7}4 zJ!a(hj%rH13JCgmA3Aq?o4zQ^bo$xmDvU(yjbp$7`axki)tc7jP1T>9>9{Z@LhSf= z41?~@MV@0Odetx{+w*89^&si}Ish&2+&3G3#X~GoZP3qjb zzCScQBW(I@)M1KtU9A>;3C9zK)iZ0cHVwqBU%RGioe6chYT1)xMt+cJF0HDdYa$)g z0;ibb3f5PaO^S(v6K&!2+10F1%)<(G7R*0+tlain2Xcx*`zIb-??g`z7YL=1Vh7qo zZhuI4$?_y)g5X(G!(FDlHt*1W4R@tu(ur$(^=A*Dbn2_vlzrW9=;=5}l&dB;NrSV_ zKOwPltjOuYfW1LgPKt(JI|xb{$_MsR!YEg-=e5oko`vln^s8toLp`6s_S!^Kg%T#iVmE*To}$q6fkcyO${ALs)@CaaBXIY=fdu`iu>7>$QmdY zDCj@V*Q>3VUJ5Ln%X<0grnrrQj!ihDP!*;4A)60XD4*IteRiq?jgA+rCGPFk_m0kO ziDBwc`xql0PGkV@iiTiOO|%w zhITZJ_i(?7Di5jXM>-fpVM5P(kEt`<&gW_>e8Si5LUDV9XtPX>Re^@M;`sbk9MmUi zI4uy0y^1|@%OzLeED?4Wph$>2^<7C1Nl1}kU}d$Zy07pWL3CB{<0xHt`DT&bJE3a^ zc7AnNce49lGe(^`o9@!1VYWkb-!&Qusm@0wj2Kcao{em7ze>9^pr_4&Sm2fRaCS+V&GUO4uk3s~L3ons9-Jx_P$wGdkOS`iJU zXElM9P7+`Gg~gB~rT>7eTIGSbv@z|Ddg|$OvfW+rIw7z zF7nebJQ?~v&CttHF8F~}f&fp%oDIM@2mF@W!i%H{TZ^h= z4-8Uuc(Mia-qF>vKCc&cs}x%aL@^sQ%;8Gn)?DZe-w3e;!)76SYOGE zJTPrC>-^@{^)UNVA|Cgo?^&z;89LQZ_t$5djcSzdE`7yufxm5QFa@M7j9&avz9a^U zn?~b?w2!x1DG8JyC9!>V(Y2gR{$WNRm0;}pL^-rXJHQa+7D^ZCTYU9k2p-r?#&otf z^!BuT@&}t!xp}XTOmUf4G{sc9bV&d7Vc2i0M#+GiA2*7ZjSW=O_70>MHf{@5Ix2rN zn63EM?UJus_1-UeE>O0*wpqhtpxnGkw`AmPv5#Bpj$08qEG(h=SEFNhAB4dk=?<&o zmZhs9z`ygX0<<1Su4I;6YaTBbF9)x`)8^xmg0k+|WRF?5U%;i+Wtd%Ih(+4-3WzWi{7`uk4SXW?DH z!0?uxG#=3;joPbdQ^Y9LjQ2}fx}B$vK}&lmE&$b_a}eUO{>AsW2k zE|wxSSPf;wCg$*23s{A>u{yre#oQjwdPm(wZO-m;;x`Y%@xHjJH(DuH7z2B+@r9$p zY&3R*x5kq9nM5y0{%Kb6XV|jcCUJ1$lw*IwKKxFky)R>MyX3r$G*ZRAYBQS5K#O0D; zJF4T_j3*l_*f$C0cHxh+txi)=22EkqoF|)5JIBuHSTjH6KuZV3ogOZ^x3Gr0P zvsN+KeKwDtP07Kfpoh!yj1lZCtv^`=-(9YH>Hk^a+F#i@z2_R-K!M!&4USz2|XUKceX$Cb!mWmCm zu+b0JMQ7$s@bLDR&w$ECLG+(_efQm`1}(0q08m9LVV_U5Td~#KZ9#5yNGzxd9Ifg* zO<$CP<$Yagu$oh}=ZffoV2FS5RxO6gkK?mz__UZ+Mo8RpEZnad*~tr@P^}+ZWQs`` z%ts8ggVNLmvYc>VdS>6*fZX+}?I2`VnSiQ$wA8fyoklRuEkwP_NZ)NWv8k{!(I8Do zRYk^HdCzeS^>yu#-%)S|B&DIBG)Ypst{%NDU!`Ls%6}smvYp|BaHZfMgbL&uJ_+EXRw{`o0g_nAA*=S#KK3xC4t!(U3L0R#F zv`j17wklD7%ic0OSp2%Egl(Rc80B7sD0b_Dq9PBlyy-N?Q!!m5>gN)bs5^{aR$_2O zC2lz&lyD%XU$WinY=bwA(QKoaryjQ@WG;*Ou2lY5a<8?@K*iOJ%D`8V zZ(VU=1+-#Fc%^qrwK<7Ln}Ij9FCWr6Mfuj}N^J;Ob5qq_>2!a|1T*hDn>InlP~)8p z%N!Fc4eI`O_Oq8kxzaYt*Yv9|>#bSh!AO=H_Bgo0oar@Ai>BJyEo{@~jk)%&Puq-! z7Av^>iV24WS|EE@%3*=-^m`o{BW?+tIv3OxJqr-XD##PX{es-2%9A!m`@Mj98ZOO4 z^q!#Qthvv#GavX$MyP^Ltv450+gkh6gg~7O$dUY^6kMU(L%2{qhp6_3n)lw|W4UO~ zXmLy+b}7M5B*rvktk?bb$ly+>WSe?DcZFQQKXbevU9qB9e2aJWM>k&MlBW5K`zmC3 zJsiJuS24n726?2&!o z1H(<&V&UN$x9Ipt=@1C!TeJMGiFZ;WZ`TpWjqIM@H8XOPif^D16RK%Nlk;Gsbu@>- zS%z4CyKXM&XFoVz_XFrk(+`b&&D%EozUs?MG`yx;&U7zDH;-lLwPX%Q+r$|6P*~yO z1n~nNp%Gi2Se;xq0f+dI_PXYLIb_|E?Pv0v%m<_TsIwvcX7nq%3a@66-yIr|8kL=; zC7n-I2Mc3UTS9GEi?ChZ+FRb7g^SDS_^XamV^POE71{a@V#dy&#awMQSbJ279;xc_ zIv@laekPBRVe8j?nC4Qv8Re~X$_vES%kqqFMaLY)MQ=N1k#DU{w64VxBMeCR%8QG0 z(3J4vg#~G$tnG9tdIHnu98O+ls0~kIR(Z1EtWGJsc-@a%@d{(^OH6kwdyI}KAo21< z^|C0+B1@W?I4i=bwuL*~s_8AJx?i`hU^m##3-e6K@Tp?49AU@qcvA>HE`{})T=DIH z)F;L&q&pxiGM2m$so$@HaAjWU1Mj9dPdnRnC#HbBBk{(+3Q^pphIH9Ft`795*3-uO z`S9)OQL&KfsKg@S6VHi(2)ZzDu!jsMqL$S|e>J62@Aw0~+YfKvy-iHih8)h62*m#s z;`9R_P}as?`DS01!S8DXBR6?N%nZZ`Az|coI%?q7KDWIpL2{*vV#d!YfKXF36jZEP zVC#s_mA-gpFwumDAJ`PIF8SK>cY^Yo;CFlZPgr6M1MpuAv*C2=PHIx_jxGCVqNTwo zGk^Ttr=se~B5W^~^k#9gW-Ic2RYMd$pldaiyX5w6Ro&xFY%G>#yMmDwQHNLeSe=F4 zM~!VW4o?7O-Bq1B9^HzP{Nl#Rn^aUA6BemY6}t;;j5j7LSg&8guNtViV&aFpdU&*o zk~DIZTS%Rw_fG-eF<#@iVA~$Sd>V3tY|Yx*dnC8V2;6GqxfGo~@`_N+bKm z`#=CQJvdw}I%Q)&l!Gla2bBqYuvxNwF636o?Z%)(dp#r5&R&&1nceRrmgqCO+6<~8*(1(FJ9R~`c)GMB z+~Ys2`B{D%rY+Q3F6t{Jeur1xX+2a{U9`5yf7ryXn8|E&RyIhNO!ubyZH$TES_qBi z00>7@fr|TXLR#aT%=oHZ?Y?F0Wwr0zxTS=I~W7*&`;59IK^5$ z>jb@za8XOM{;Wqv?8#GNt5z26aG@f>jseouc%@*feh5}Hj)iR(Svnd|tl%}B2atM* zB%sS6g>q%Ir~zk%4NbW&Ci7eV<=xp8>WNwGDLcC3_xS$5=_Scg(80!qJRJ!Z?>`ex z|GH`>?LTWL*gp>XZSM7-3mrB9LFLKQxBYhl$QGcedPx}Ie;?nIN76y4_?Ixt?|1v( z3;sd@3_l@hO818y`7a2ctr&C}yIlFKUHZ53{Lh85+raRQmg|APt+W1fVX=~^WxAdg z?eTZ{3j#Em`NeR9zssqfmaQgnY@UhoKQjI;2hkA>3~!rOO8$2_jRCaZu5Qx8-zmol zfcQ(K5O9jKJ{a0d2g0my;nmpd~hr5h4GP@o&}T|L;nA zJvJPAZlRy!nu94AZCe6CRVKg4@>R1wGrROlX#%M_lg4e~ zd(;Blw-}U?DHHy=ZWB}7WM|*79}~@^H#*~shu8o~?{U?YD_72miNq%s5tM0yw&VT9 zO(#3ug#hZdiUhGz_iqejhEUM+|D^Vz;Nj7$s_;9*qud7HJov!dw}T!YR>!ZGBi0Ex zR5cKzEykPlYn|NoN3@Utm}w1oWBO6o_3?rGN=b3XIu(4IjrgKupPf#6P5a4;PJrmJ zHf;+lt+W|4*Ul!|1kf2fI)Eo-;RWED#@<9B#Ekc5g95(MX-Q6W>ARXw0~r%(<8+;C zId&Ru@yu|AZeJ~6w;UmkEP`W2xYS?@$7}Dd{^09aY@G{XwKIb5Yy9MK+nj;K8o0%; zo-_g6;f-qJ+kY^|MG3%N9c?D{-?@}Q#&JEbS-)aur<+f&*s!7C#RoE&|MFK^miWe1 zV19OQpKrfMHR)Db)6&sd0BF)~@hJh1Y-jaLy8WdFz=Yw;2{yL92~S6B98EV3a@wvX|?CEPyB4k`hvpvQdmTQ6)TD{>#I9RfpC{rvf| z$qqn@4+dmej|4+Bc__g*LP&k3s|PGR-^WOtoeYt}M%wc<@{b!I-bzSHvQ^7gss!p< z-B@0O6ZoW7fp+QMv}@07bmOBgQV3nHrRP#tG&I=Z@ECwK;wOcm$1aS|JxVD1gGoFp z?K{8SYfw9EOh?hwglh&WQqfs@%F z%Jq`gYw{9p8LvXubF))+q`Jloy?ZAsZO~)AagSR8NJ5G=N26*JK;B{-5t)(VYZu{N zNzk3n2Sc%#*1lxXLJLHScs?$Gl%-@KQ#P^8wpzILC=e^h;ekf!JP|6&o?H_C)p^uJg(JAG?Hh{+! zTNV@T8z)F87|aqFblW>tohGfaM zc-TZZNJMQUIrnv5AE#JTeYX$jVj|U2uMkQA-=FW(Bsu?sARQyZ^Ef(-B;LxqMY0go@3!jJ(E+%ZC@43=C_En5plpJ;EK1P zfty|=C1q~awi7qGy9V)qha(&$+!4_go_kMqQqeildG%Vc^XbWv;Jap`vlFB@w)TOK zYw%`vidP=9bI)Vn&z9hR7O$0b`cd=&9mr&#dme(U+SpB}-qU&~!I5wvQjQiRMQ;B| zQIF)=yk6e(tBe{2))MDW4EG8yBK8Lav>#y5Aj_FRtUb{H_@Zq!z7P_%pj9ZqxbE#4 z&~Ox~WCmN2EdbUy^LACg(dTGyza=EUsIF_T%T9RyBl~(cU(+5UM&rN^aHhB%ImFS( z_W@y*`wDThhvVasUrmTA(|)DIJN^Ix#T8(ne#paz)M=Kx2zQQFS38=#_rU&}8(CbF-P z9?sW7VC#34&XV*MTc=7*Wrv2l&ixag;zEH1GMH8jy)SWg6zPX82>gfaI#*~FU?K4= zTon-w(9z-6gaeP!0-e*tgljtV<+7~M3O{p)cz*DK<_AQ9L8-~B;SF`!Xikk2FR@mw zEUO{dsaVr#R9xIj_JpSY(RVeaiOhJo+eTT>aRKwS=UbBG6!V)ZSIGYmveKHbQ0LBK z!!+0-$9YMfb`}AmLq|0&HdRIZ$%(X2`}*1odn6Rq-f1DyHQEXT{kmqMt68z@t&|cy zE@WUacCD7)+tR_>TLpUia`d%c*9rpz(w7G1`6|1jxz6!0+vT2kUlCJ9s{&QH8$@*V zN7Q;3SMeTVgJPPGD-|dytL=^ad`85;UDRi)%w*!#Rzq!l;&@w3e|iuQDFsbNy0}NV zbcF%j6soL;dCF5Rmn`CXfr2pf`-nv)k3~GPcd_TXbn|vmL-MoT$T- zMcoAzg{CrPDdBi5oL#fbYUuuXXVE#yC~ldz|f1Eef8{wF892!cGqNyDKQ2=D3ewiBHV4}jy@0Nm`B6dLeJiNvX799)+R zwGb&{8{tATTY$SNu6Y$o8?RJ4RR-?fGYCYJZBErbflizrr&+}Y<;JpCO6d#&Nb~24 zF4DSvg_$0?4`ej_dc7il$T~UviC^8X!7l{5PrLMllFvTmWKbNf1#py&*~Czp5L{_& z0>csDe4l_D#R+>u;yL+5IVp&|hvBW=Nc*OZi|}@4_ar}GNUhV-pwJP3Wv{1)umDQR z>A_gXJD&3<>OO1?hyiH2-V`z9F>ceI6}tBoir=UfS1ushXru;T$J92pl=}f5$pi?a z<*u7E(|~8RsGfF9%q>hX5vg!l8}8(-UHsPnN=0I<*a!vS;&iFZjINnqXcA7f;|BrH zhH%>w`B8V3l}12IQna4ipl;Y=og}`etKwR!VXgH@o_MB+k5ME0@B95E=V&<$Yv8e? za79r~WXDfj=nE}!7gpB@mm-dM@|B~~gf8s>gJ|bI;dWm5I=>p*&MAHqhe)F+o&#dw zPUtDF>1^XhJp|R*K$*jt_%yCa2jRmEq!)#T%tt%x-sRC0D zc>v#DwgL?F#HGLMokj^Rgw^+Bg;h^?U~>uoSJQ{pQ%<3&Zn^>MX_tni?U7YH+SJ=( z+Qk;9oX?mHT}~Ni8AnNx9L$D8iev#2nQg2S+z7FT@`rHYKBE` z0u7`*E{4@}@vATVv1!}*w|nyZ)3Pl)fYUmHM2H=X??C|Geno<7D?vx6-lNp0(7)_B zAzHQ0#flteFf%4@u~cIl`lk@Dm)1|}PFIj>xe#-;6J-%Fnnk{%znY{{c0bL3ldHIX zK6_KcEuY^zD8u!s^>E1svKl{YU-Enp3dFioo2{_1S>ej_?jR5%mttD{u;5);U>bls zR{%*RDAd$Ti>Mr!YvUY*}=1F*#%JSpPg zHI%tMiV}!0O`UbFn1iZ^nBVsTN~V3h{F7$cS4sak`d(3@VH{kW0}peCIlJ#kF_VGM zoXXXNrd^_}x{Sp{wE(8~QFLv-RU*xEjef_zBw@Ev9Sa>FpZfE}l@HnAU5%2VZ{#F1 zY$O~YPjwnEl3o)4Z>2yFCSVVP7v|?}fu4{9fIgov@*^yz(lfFUg;z!b5n~5(>(M*^ z)D)eTrO`q?z5TTjf;uAVzT(b?CO*Yu{))|L!32xOIU6T_v=}886Tk~sPzK6!=xq3LW=qR8bV8Cu`{E9Y*bJY~vHEMyGY`mvZx?I?0Jyrw(;!us)UcdAx z+})n>srC0{kNHn-{Zgby+()j=0sq8No3?1Wyd?6hirw7fZMpXo_EHhF)>`=bOrMcFl}Di7@g# z0<(5d_z5}!o8z;EtKaPlt?*XpUM#-4Fi~!S_xG`Ur#JD>P`1TsD~%Xw{GZUK5Atbo z-YMqRb8D))_QCh+tx*Vk%_FzMj=j|$8Rm`p`#8Gz(`60W{rU-TcgzG3Y6!#KKo&KL zNss#W&8Mm;Fo%qdi>pE^h(r@;>}$X>Vn}jxb75iXU{JA6d7A)FX_;~Kuf)Vx?FN9S zx_AFRpaUa5{hB(xrL-WG5%YNKrsI7KUXbgN{2y^4C)G4%5_c{8cniK0Cmkgbl`-F& z9)~~q3cWta;8$0e1oMf34y|xcIz{)SXg2E}n({xD%WTT8L1arw)$C|jt{>U?E3WeB zvV|MPbPW-7{VcfEp$E$`tUx(%$=!6+MEVcufMjz)KN#D0f+6)j!<{Wm(cTGQjP#jE zVD<32y>`Rkk1M(^$p%ot<&E;EGp37EEJyHw3f^wwq#^F8(m(V$0;!nfZE6nnmg&dj z-pFsgR-QqtozS!zb}kIxjD|6uI{O zbzx5fh=Je{zon!iYVoSEDQWkP#3kT3$8VhnkP`fm47i4kQ1BTVwKdYX`&M5CThUkB zC-xUSJ0Rn(^1C3oO%+esjZe_eKn78Kn^*+LaPbd4ZvU!0K7W=BpuqaZ_BZ4tu^-dF z@_&3C6#VPYviP8yCQ0IsE~1A-X@+rrfAC)E7jN3*v{K2x3jQ*6$1Ha1C(En3ss8p` zmu`_iW-Gn&hv*Aza{6@+hZf*00n{v1@GdoQG_!P*n1P&u>Te4pgG&aGVefBDnX#iA z^u`fa2;CX}ZSUJi{wB$PxRuia;eg?m^K|Nw?H;AsPh9Vh>mUCU2JkQKj{}h@IJmFE zBsS0=2;{r9Y4&)ww9@p)Fa9Q$5;E$zS!_qGnK|vKRGb>YBy92f-Tgls+oEg$@r@~h z>5_6{skw>oUZ&#FYf00WrXT)hG{)2%*ITCNu85Imib1v4 zS(o=KM)YFQ^nCb6^Vw#?${S6E@|zOFIw;_nA6JLF3hr|ClKn?I=r{ z%9?)st59xqRR2pTWTB4<<8O(>(g~Lk#(zkgfLS`dVlyIfZ|7UJ*d0UkL|;?08o1Ga zYfwAmZ%a6S~WHQVAgqM>Hl8mKFG@Nc2gZ`QSJG&TLYBk8{t@(dbNwA(7Zn z=@qFUuNSd2KM`aszm8VMX#SO+q; zz20Gh@!D|iX*pK7h4S@_SbX%v$pcYbU_X>!6sKdAxmF9n6dM30bpVwBV7_yltjE0A z1j(~^nmDrk6hQjm3lX9R&y+2by$epfrB*z{swleuPi5p-AwIfaBg&m_Ayxf@aHOA> z7FjwxJZyV~>(R;nn307I3Oa2C^l}RNo*V*#Yx48M3}r`BT2nAR>re4Lw!)7!oi1Gr zBW3k*P}pZaFDaYi1p8XT4@N}cXkk+LY`&`UDbP4trDX&i0Wj*=jb>vvWV8`e8S*q~ zicd3CyPDiy&#ic9%d2eHV9d)fw)qV%TKbysPu5cax|XBF2J@m~tH5Z(>y2;wN4MO} zvciFywH7GldV;SpTLVNybZ+rEU+^$Ws9J7?ASAlLrP?Ho2^+=s@7{ti#{8ckfpR}+ z=amDl9ke<5nJ1LI?%oChP0V1*U;}F47+MDwUM6POzq9<*ChZBKTy3}g|sXG6*`2~Sc4zmr;5kinw$B0{M>2r^=7`C1YlV)6BX?u z_FesI65uc2zb_&MAd>>;?2M0Is@K{>ztfY?C4R$*5EieF>P-MkI)G@=8UfZ%IJztd z;O$67`XH&JBXsHpfUs8p3|1fMFijdOz?89|m50o^)UX+$ygNE0WS6-(oC*75iGk{z z=Pa8mCPr|n@t9LS`t?G8amQk+!&?J#0GgLOW$*op1FpladhPD00IYEs>0p(pXkCo1mKoq@5>xDV*AR?FUAR2sXAEt z`StG?*hr&wpIo$LT4}tYj~*x*vuu-En!^$EM7r^H+Oiggx?qC@W$HFb_d%NgWwnP> zm=@BYzDtqJED2-KIzDRy&L*+0FK>>->N|aE!jayXk02Trki^ zb$LzwBK!1EE5g)E_VE9N!X$4~jc>M0{}%-2cN(YN;j^kFPSvkH6vx#82D)5F0f3hE zPgsX&IEMF9$l^H0$Lm~en)IbqKq1oI08eENuvCSV z3g@(=Zz{k$A;(Kip@{%{xe`R$HKo?9rDkW{1qNXN%552$&&{nUTgaAE_L_!U)nBcf zMp>0Mft&i}=`KP-nsPG;Ce^H@M$Xld1qO$lp%1S6vbDm1``R9WpaFPFaJYiSR7D2nooSMxAmq^09+;-)8uOwT@%2}XW$=QniI++8ZexxQTFW5z<ulpB@^H?7!KoB*E~}O_AfWh~KG?gv|azG+;eE4RSnVjrR51 z!A&?TD9{;BIM7X**k|Z{mh=3?@XJKQ-AQp>mW?S=NTdsja*cfE-0p@d9Rbn`k8d(} zdIq%j7dS0-30m|qVP+cOfVOm#j+pS+~N5_hj;WDz<5E zAs@iOSJhr%#Pje?rxG{iudEGMeZ{WMp&!SHOpyFv{M`_oub7+OW8#7 zH2=1QHhn3+P?Lfr`UhDG|c90g{Vd&UK}x>!<1?Z7ESgx>39#Sr;eK zGy9o(m&ux%XIut#7514k8q~AO%?d3hBU#OflW9a@fgxR|oO)FSD9w$Le2?vU-}net z>Pjj4w8?|!iHu;Nl#la}I2bpDlH~LI(0tTPds4xHDFy&X*oF%e$c3h0*aJHRq)h|rGe$^h+6gamt z;HfgJgD3{bkpN0JoA|(LYwsT!j~!38YX&Hq{_}qUwBw=CwFR?rf$3c@G`1A(2}FtH zNJECS2M)>l3`%$_B*;9tw2O$~lQ5!E{x%wab-|Rtw&NB_fOLmymh&4`@|FY2`(V<0 zWjhl>Rrg$%F;W!!?Cc^5;iFW_ap?=oD}fsB)2^`+Xy9nk0eu()j`M3#OFtnqxyce5 zDM+G`T@ zcCz+bYtA*s9AlcA(uJL-T+PN7PMfTnQ!3->Nd^_xhNWh#vYqzms@%Y5c>F$nj!=VT zI9aS4|Cv|V5N{_7z_p~kn7^n>M7Ia}cpYD=Y~83iUw?~7M8UCJr(~4Wq(%Di`f`t` zxk7i^3rEu!O*ze+GuG~J^v_KR@)TBzWlxDq2c6l~3G@~y{7y+mOxDuhC@8+pePfZb zAY-nLSA=6|G%C7Td#O4x*)bz`b3@{Ey6x=@bJEwTs=_VL<0VEhn0_cWYXTK=yFF1pATGZv6_y_I>WGzF)_}o?`+kPp4J^}OddEXgqF}E|FL>5iJowE2tX3P>v%mM_$hF_>jn0C zQ!}{l@AS1puVfH+&yyA6j=(;;gYe9B=1{QxZ&o20igodL!#-txtzJTB zCU-#MG-_Qws!LWWGS?P)Yj?c6Bzj3}GQ1;;*>WHkOE8EjXsAT7&}Ut6 zEB-2qKQZ#4xQoy(Yg|J7c^#RNqmsanNS6y*CrAeJ?ck*c4V`Zv6L)>ussmujJfASKQ1;N`Utt{&N z-s^=BU^0LcHHo^+XK!$7&2w6~AOComKV+6S>ShOtjArP4;MzV1KmM&f ze0QT&b2q(HrE}~d&Z~H+KizF!-?$HE9%CY6q1vU}Pj|eh*|(47oAIT|ZmUWSfts(F zyAA63geRtAMwrXqsTXv*y!yHE21rR2-T4}ee7xOR-Kz6&N(dEMcs4-V94N$BFp_>D z3+D$RN7lJpf|aS#M@b1iIdufHG}*AqId5<_Qtg7`=wf;Jb_?6>EECw}iRnA_5>nYC zUn|Hy_V0w=Co_^@RZ)+2GXEmo{XoQ8`65it8PP7f$Ew`3BRQ9H^_9``GP+jcL6VQ4 zTbgl-NrzMxYcIEm*I_kgeaHJQo|?SS(qHH8%CPJ=HLDJ~CGslH7SdK^cp>+KCFP>y z!+U`q!iys;H8<%?n*4Rno{QPTFfryN@p=#e} z){HwRWp_`a+&YmSyQ}D=*Z)0L7MrjAJy(vvsqIO2(4pW6T+G3+Lo3N7omE@?RIl2j zoH&amDw|Gp@%bI0jbq!8rgTs0l(`sw%{E6d%i;WFGif8T@pEp}{FBg}uH>1O0LFxe zBF^e!ygfWsj@Iv0cO@Cd6@0$78;qZ;a--7y6PjNfyiF3F_~*lT3r^!$m|Ldnxc7J_ z^Zshr0{S?m#1(ED;xAqy9?m_mDwck==x5{ps5N!(=w}r=(#meSj&RNFO#b_1c%*z) znP>0w#1Pi#LWVRGS{9Uy*DGSqo(O+yiX{|IQR{nE`WG;DMIkNJ{k1&g3h`+Vchi#t zr8k-q>T)!cFn4suwND8bGZ&h(;supR{~EI8P2HLZ)amcqgq7&*PosN3EvrE6ZK~hy z5$-#!kD(QUDcRcAQO94C3VhBAB345hUhLs_?FW-H^@@TVb|-1&nIGYs=d7)-`=eT< z&wVQH#n-eSC!lX%T^@@(&^V5ZU*QIyWzS1*BrzIlJmi~ucbtN(xZV6m>mH@dxS%U% z&sP6N)%Zb4{CdgpCm&<$D>Yr0`#Aj1ZJvpZeR5sg?ayM3u#g{p#(FFfQW&r|^JXZ75$KsusF}vaNP(V@Dm@T^1w=3Jpj`|-HiR~is1bk<) zvl-pkLIVd+{$$uQ*(ArDFF9B?s6e+5`j1!r`}=$QJWzbhN%eRqkM7FqJcB@U;7f5a zP~D#L2aNG!BHlJH08p($Pb?)y1M@E!7F|j7sQ7g~6_$m$;r*CsJw$VOG{RC>QC!2+ zSc#;2xc~Mq5Iij8{r9Q$qaNP&Uu7sh*Z6CR2fChoNzf}U7lW08)qbNi`TNo%C3*z_ zwBJH$_I9ov6`U6ncsu;xUdT~ORpQd1igDbk9TSzU&ZyBSFn|>4z=WBq%hen=jAS5% zcBw4dKYpgdp0|?ANTV%BRgfc1DQZXq`>*Vil`T%|(w!8r^qXcll-bV*s)v`=tnhx% zL|ni*lBOR5N0wWth3xAK4u-7}Y$iDt@m4VN(jvF5=Ng%1+gx<__#8`f(8!kg6TYT2 z#BJl9p^ie;gURX$Zd;?(@5hl%s&eMb98`SgMNvq07Amv$2@le~_Y~nfa$^6usr(mN zpOw8|ljXo@CRuT%|jF#giENK-Rw zPholb-RSs%5vnuGC<{Gfa&n{Z4CS;@jPFN7)`xs%_AOgm)av=5m0nndtlsZ#ei3)O zTs&74kl6LYN0k^&>jZ8`PBYIu5sy}S3$>LKYNFLFhVa(@{G!+~!*dbG_cZEx@noD4 z+3Acc@tjZGNt+6^edXz(&wfXI5vGdgJ|lkqpxVp=na84PTEfbo|6Km8NnZ5uGKEV= z)twyH2+`VHffG&C&T#3Ny?YxOOM_4EonUCN{vT)BGkSF&lv~K^`|qMhsiA_sA(Gck zjSi^K3%YVUJ`oybq$rgjow{t{2)k$&F3XMwg6v6LhyvL^5I6K`q33m~f520WO9ty9F4%t|pQ1dOeHlnhy*6V*iq|HowX-@^l_ zs}TJE=GSbXldm4Hf{lXB{`B=R+E{&21Bv_U zhUnblR-VvLFDtFt7<8_^`m(%?vpbHX_g=dPH#-p(s#N{jNEqR!9b2&+8FirF5O=eC*h*#P9;-^B za;NGG_CHWvua)SV(~M{Z^I~DPnpy8s!$MdMVdi?ID$Y)*sWnEG^DZOj>l63$sd9We ze#F9t$0qnq2km?K1IkV^D%S3EnCPz)np#?XQ2F=P|4;cU_`hL29Kl;-(26AmjH~|n z>43kEpw+mv96~j~j%z>!lh?ys9 zVEVxV@yFZHVQ)htSh{J%3t?Aho!|MC?_}9NDm7yB)WBM0-Ed&xWsLpKfc zJeg^OgsNU^+Xl=S)G*;sM3pWPcwc9iCta0R_6gLS)$QI~bm`{k;UbT3YB6O_9>r?z zYM_wJW_oI;`UQt89Zv$G*sWv+bFI^wIZ9ht>CeoUg zdz)lu{}X>)rAL%|gq3C2;rPd57G*NFw)+=@U4u{isK!4%e?I&050fU!NVp8;iLs-K zIFv}{8;jaiZ--l`q*nz)cf)gr3;&vj*FVxJH1lRZ+L_yI^yo^*p~U1^l$Np>SuFIC z_p<%|8`MAexCUrt*Kted<@NgcPZNo9nk^yhdWD7K&Oa&kp$5bl*XhWJxWfBYw4$W) z7lYx@OX}47ItdOoGie?ChMgXx#YQp-)Oi`SDa!8%2fu1c2IZ-8-IGufC5}d?$VZuv zZk#2io_xB|lf%UJLHKJx_}YFMnQ)q#(jo`^m*$IrUTstc|BBpJU00eky9k_(9ZPXa z;L&A;gL{ZX_FR`Gwi01Mqii`{uHiHeIx7N;baZY%RC7d(kv~@}Ek?w%iI_{R#Nt5S z{nfA%7pzUnop8)nme@jduS`UJcJp%q$$7pIA8~QaK=zzXhbje2zTKXdHgcO&_-yh1 z6N+>%j@E^{Jh8B5UFDQZsJaVEZ-AZMC7-;?e_&Gi}HU{(#efiHZMVI&LK+6#a`Z)}IQ*pK-$ zzMesn)sBdjvx%2OPh&^ca#RBfF(_tgx?k*H|H@~QZV*jh0Mq1uL5x$#?-+76BQIFjt7UcTZ-Qj19MQ#Cr zAl3@v@X8rC2h{a6+alOu6P2z?#GqcCTo{zA-p@S;jMy5G%KXS*?^6&f!FmGKXaOq_ zpm_d#?g5lSuZKCrJTrHvcOb4kp%B*}T@a&%_TsK}_eR>#ISuG1^#m3F3Rvk;@##fy|tr6;o&!T6hp5dJc39iMY}=V8Ea!F9D=yG;)b?lteM z?U;oQBL>7YTGwZFV0M|e=2k2Ct+jFp9qso7R1-ulLa8dG_MMplxjx~T&EE_oHCrOhRaEO2gXSwsEz8_>haC#hBaX1gak4QtnBV& ze+cRvkL9J7Mseu?5d9i{yjuV$=RR!B>}aKH5h&y~9IS@BqX6v<1GTrFDg&8s7O-%5 zk#ec^2n3MPkJLzA^G9Pg zZ^dff;uG&LXt$N*=q4|~t(#ky_8Mo}v{Q8zG*9}i5xtMbbDmaFp4}kBh+3yA2-N?4 zUTvKkVx}Kn?J{mM2RW;%QKKB|lcpIT`&omjW$B_cpQBq|Jz$2P{YXz%i!{I(s-+9#XNIos`bAxEX0i2g11oZWX;HU8Emw}x~L$P#PBayUw5 zF=nxL=haP|GjzWW!26}*?R8MTT?D8cPE`&+aE{&S*!rUU*>dc2@_y%c%&ORAyeXV< z680sMxHu(DuDn+Y&&67(B0Zd4lS6$uy?2#hF{ZoZlU^WB3L$+KQ(1cL5>ThbPCUjt zBNaFSeq3un1yY11-TkiKG04#>*{+kqvsVLhJPWao9Z*WF56X=9FE%VBxOajQY|TE> z?qs8xreJL8bGA--A@MxR$E;&gjrHMzZ{t{2cZV_2qZglQK-B1t#j-x0v7XB`p8MI+ zLXOw@83b?z=TKoL0hD2L$gU(-e(MF<+{ad007`eVQ)q<^tG$8Wa0Z&5!8AE4_xWfO z5BdXuB=S^+p@Bc`-g^~Dx_ZJ}I)}%2`9RHJ4k)1=bV-)yNU_;IK*Y2U*2m@g`28>E zI*O8W=qz^Cf%+UH!l-7gVrwvUPhHiMAl}+!cnfMKHJ+!k3-Wcdd=`Vlz)|p|>|^O* zFs;Z)1iLo*%CwyWiqKlbh}n>Fva%a|&pvfpw*i**R|Y}9UC61ns}8_|EiRJ*_~HOFOb8B#guZQK~784%P}-0w&Q zoB0$u+PcGZng!o#;#-`G2M(~= zNIM{f!c2{)$;Da%D@;-GY;ki%(p9wx3&5YUiHc5OMWS?5SiiUdAdka(FcJ=8$!3#_ zp|`o}@eM$o2uTeT6jiSRX$@hPj8~p^X>Oj(#ST!&gWi3tX*3L$`p{r5)PjhdE#ELi zu!$DFt3<~H=~l1Pl;c6HDy`{#eM&$_4WT-K006g7Ubtf@8$MvY<^^g(FydRgOMUQ& z4d6#Qe+ifvg;ERJSr4r59M8_aR&hcUX3VLk%g66#M$g&xJfqMA)I9=t*oo59P%Ewg z3HNKj25(8&8F4pjS(VR`qt=k4m8tFFf;soJ2Q;%(nX@&E(&C+UN5WcwR{T+kN_0=_ z8^}YAwe}Td#A^ap+qxy=Eiw-$%du&>4s+^PwsO73OfTGWN*NSl%rb>l*c=-k$Q<8W zJ4uyIgfUNkD2BET4X>9_b)0D}_@Ioq`Mumf!#0Fl+B@8;b}$8I5_+YLO*>Y|uuPtu zRlYUEt{R0_LExOuDTvQR-4*evxEI(X@5b?`!kkw0Yv2^78mZ+=F)a$IfZcZL6kGrWdkvO9zXy0q zPjv4ylRz{RB$RG&w}hw-=HVpCM6hjntHZuAzhhO+cm-;=c2TCWXcIRU_}zsjZ8y7F>A=mTZ94pdPq8Rz<_`ES0h5* ztlT6TH)8JhlBFF>U4#)V7ieGK$@Vl_W#Qq^zZ!C9#41PS)gz|l#J5I06F_t_mG3hf zg7K9;W3l0? zsYJ`42411ryZa2MJKv&T7&hwIJW>ug-*%LN7EKtfgdl0)_!>*7ym7-KgY&KlcbPc&*6Dsm|qvT+S&6f{m~uU6>;6VVCx=g$yHyI(8a zMO?ei{EdWpP&~pzU%Dz(Or4uwPIhN%) z^1&fwN_z8;Kji4UlUm0SVsA0jA4PVZMR3#P63OXKcOheY6L_}0B-)Lc3YN+@I}>R2 zLR`Pm+IKyEi9*^pEsLCQ&}Q){OtZ={C?XQ^o{0L58Xb;bymYy}wkJ=oMpd&l$k#wM zT3JbE^wryIKbO)IuV*;|g-_ouWuJ;sMF;0e>5nGCYtMEe^l z2eff#+|~tRKNUv~3D#+z6=S$&tp_NNOG^&VEN1Y1{lb(|@}4nn z5qdT@kI&FKd+}pSg!Hv0Z@Qjv@%yM{=@VSdob7-CIi?;pQ0EnZ8asS7B*^OmEU&!s zGoRNjSB>6Z?780|MejbW@82jdycV|Zk7}Z{NSWeyj#i_i1EqszG;E>t_Tb6LlNGb( z6e|;I-NA{`Pb3f{c$_)UaAh5MY7jC2v%+G4DmqUH%Jl(RE`1#MHSl4s^o(xB7;y1R zK!GB?EWPql&e8L9u~TLkRbHP<)yH7>#akJ9pxBTH;g=jmpyYp@CMGMQbi!?=IOi&3 zJReK6t#Qqb^}?z>*ws>x?>C>Pz?(jHxhQqCRU23Np5ku%R;j{B^6f)k4?cUg18l$z z&QH&s6-JJ8dGz|^hBp?Yc4BzGo|bLhEGiB9ab78K?MqzV+yaNmvG0$q(Nl%*W1QpU zIa24;HKGSv#adQBOiKp?xpMBkwsKyT;8OQT8?GkG2cR$K(#Gr62&bJ32z~w9dk`5< z`~w;mN~S1&5s&nrewdPb2Y%v6R|{k1s|-#M;FdqzAg{ae9jQrTN|s~LFo9}pf)i(6 z=`z0h(_CwNF<|ohR_L|d{k7b+g#g1rFT}_}R`ubEwaQF4{;?Vyz2JDIICog#d5*~A z3Wr7A=CC7IOkHzgz}{fXhNr=zfzRae_4Bq}+_##ImT=?;_Zs7ddnSD(f7yxYu6M*z zJwDe`N`2YLEGfRg???WCzO%~OW*vhzWKoN*S$4opooNnDG%H7YsCkKQNbI_VWWOf_lC1%BX6PbuCa~VUViz;H8(Gi9C9Fs&;9&N#JWR4 z)K*`UD(SK)JW4&a8<3y;NTCf3)lc^ApVf}y<~RkpZXIyFOz9= zz+I~NXUlJves8-^mpN2iBF9^GPwmCQJdDTv>aqv|`I2=q7Io?&QEOVJ`?5EMSYSVv zWthF+PmO|Y5Si;I-&L4(8C5A2r2|cpJ=EYEHgfa3GoPL-rKkgbr+w179HBk5O?V`F zfHR+70U764eFPmCvO7>fTKX8UID?bY&LJpjRX&WzL$&M4a>$Xb7LM1P?$)>;Z@U~H zE^l=PqWmr(5HPpPm&87EN5M@73LDcM0u{ zi8$Wl+?Zz=>xek-_E&E-7jga|U(^*@r8=`Qm^1QytjIZ_=Zh!8mU5EmL#>CslR~qn ze#E*mSxe%D1D@)ifvi8l(tIh&qy7ltcs4ly`5$NSzh-q(H_jf_rLqX!tLuG_56lp$ zGDW+C!IjpSxP`l}Uh!uZ9;Z8_dphnZt>bPAtG=!zre@6^6R&g>l?ZKWrX2@0 z(?>956h{7QVgDGI{uQzMnI5)HuM-kk-fDAL(VHQZUJEYx;+1*(ukx5!PDs&R87B<1 zeK2i(+8{f+q$$!f;RG7Ip$4B>c)3UXHM?_Vr-{G@Vc$Obo9}{MtET4On%A9|Hj}FN zPzshR);flR&H3$yj9z+qnxiH0JdmYoiK?`mji);fi<)^a8Rbs}L7ar-nZ+}C;jk}h zbR`DLDF15F|2<@;gx;0DC=nrcsui{hn96SH%fYSAQsNqN%iYytm#XwUSiSt3 zQ1e$PBop?z^`M&STyxHljyL#1vp}<(eX%@^a8t%cWsP0(8dpJ74!7BT3h@L#=>lmv|bmnFU;j_1PxG3l<2M`=E(UE zI}gbGM5H@^0MjIE=<$GDKT>UU*sA;+i=V5N-uNg_)&5|{dHZe*C#igrAu6PfYeEMW z%|IUNEPBX0ff-h;`G|S5Nxl-JCAi@~Vks2oEGtXx^hDNu=3C67*2xMkN?`cc4}vL0 zJR4}O2R1Gj?JYJIfy%kj$y0TaG`97ZTF30334%yTti=uqGsP0^X*K(+c_<0OlpQ}uj_m^xfA zugmN9fsqH_{!S%FQ6>d)t=IoMCDF!Uf3_q1mzmPP{-8=Y%Qt3}xgc6&=)sk?n_W7% zQDSddJ$o};aSy&Z&}H2`lc+Nwz`uB%s z_3B)1KZm%Cdou#>MhdI?mO%|V1L>U$I8Iqu-l86vqGcaf-FsMl8t=vHB5Gn9MZco0+qwg{{M4Q}aA(S>~k6%u{863EWb>vz#rH(lIW+cN(IRS zg55TH?hPkU+3M+>k0g^Fbj1%MMt3@f4`zHQ_Jadd7riUvsM2+#{(t6 zn{rU`XBo^n%sX*=ua=E7yP7C!d&ReGWt?PY z)#TuVtxD6NGk<#9TX7j7r=8mAd%s?Ve}@|Jq@?L}mhsX+22_p?ZftZS#|iIJbJ=qE z{h##|PN8{TuEJtOImXO=p0U)NUhwB9I{_Ut0bN!>OMO8z2{i)_CC{qAbF~#%sQiBR zCPm?nkXP_f?TJ2|IM>g6l9%=}@_d@u`umc?8s+$c)?de*#b$`&?uef9QyegSN`_(| z7HTR5b&r?NZf{gMnxKmch^C@Sew{S`{n!;PadH4E0og3<7M>6hO}Jz0TmW;QIPPQ7 zYV;wKu2L{HH@v0x(P2m}8j}0sW^QT=RRk zx#n{YJ$6xCPOY3gvMXQGC^YP+*$rb;Lt)2cD5+mZ@V^Ir`G)Cd8|Ra-#GkY2{CQV+ z83%jfBMzKmAuxjw^!$7NoY5vP=sY?b$xJ*#F?5~5yuKxD@ zy(9io>~>!_R`fM`>%R6yggtw*2BNwg63@-wBufd)*PeAxFzkO-+T#3q{I}1hRU_$T zJ=9}J%b{ygaLB+&b}fyf=tA>@=8Qj1O)}F$8;Um)aAGD1K1OXGhc?#8^FrCt@Ag%3 zf-AgcWvJcCONOaNpA_P<;A-rP4#2W_wsz;G`iHsWGYtcENOwLP`_ zpB{qr%OuSPXwv09T5@1A%*w!n(W=LxwxG8EdXrN+TwU<0`JGHh=UGJ5h34DMF@GEvQl>x~in3tLs$QVyez`C1 z<0ffS6SUgz7gaQF;Da%!@tH)y*oj4Zzl5G5Jsl3UHnrg&zo1%@X+87;{=pqcR>(f8 z1@AF3&;uVW^84Ef;RUjcL6c@~aoH5mtgIH3Fm>O!7{VOF{-+6=%GPZTQ0~4UvrB0w zLR|8$c72CNtx*Qc5pZTDZ)|G$X=RLJZN!18hy75m)SsA6xA(MkhmbzMkzD&q@^7MU z5q-B4`8<0I!qj&71-pnHcX>#xEs zX{@-?wYIov#Gw6xMrB(sTE&aCo9SZ6-4KR9eLHrEB&G)FEBph&B!T#o zbG3e(mC$eCJfm9QU#@UNj2M2aw9Y4DBfc0y6LR;DcYBs(+5mlnzeh<9(<(7z9<+K+ zLSWNwv-+D1_FswqOPpsk>rPAEB9{Se!r+Qxm!q%cYA7)L?S4zDn8ZLp>R3j!TdRv{ z_OPeyFG+cu`*F0spq}fGSEdq>!pn#br$qf6+1F}W`}bWCq|$?Pzs-3Fc}vAd=hwyI zgEjRyvERAJ*F&5=T{1<#2+Ktivi0S7*&x6-O)?^sjT7Jk4oB*1O+cB<4=6O{HTUX_ zU(OYAnVU=jHWaH8aDO#l6snZex%Arw9P0bX3OoSmMY;e6(h!KI3%=iy`Tm56UG+fL zHQ^@^CVwcOk6{&ZUVkhjL(66GR7>75SyyzB%d9u$5C~eo0!t&VW@`KX>^vYngB4p( ziq0pvue-@jl9J3dd}Th?91_xXTS|2~oc8Z~y&4PJHl>|*5dk>Rni?3rI9DR;UN46E&e8RSSARhH3yr76l{b*^PiQzipBT zcVa`W7IJ}BAA|09aNXO*T(@WY&$D>ubH6RSmv?biEPg!%jcxjl?e!M60O>_HYK4Ni z<$+EgYLf{|N5B$_9K4asV_?cwfR!2Vda6_@I>s(rD7Bo;&D_>~{pp#(RIT??I?(%a zIjs&009jz)4nSr}3o#tpzg#WeUvDr)w?d0SEG5tT;K9@+m+7g|`SnH#i5y3F0kqk{ zlJ_W3#_}C#l=L2$OmXv%*)$viFRAD!JwPw{2-v??ECQD{OC;J@4*5)7$M#uP@#}Z@ zpND(^E}w$$I>Ce2n>hZ63K=1nW>^I6H>ri!%M4*(2972`b3Dp}ac4bHT`#%?-m<)N ziAYdl0G5^a>@G%i(TGQfXY>@572rwo=2a|b6#(2eW#9~Od0U4wq0LykSC%Zy!3G_0 zm{Yn+`&d9Kl$PfFsLRWsKSOCr#K-2z>))*QMkYn>aMV-@z=Db`#`z~O(?lm7+>OHVX+kEZ*?-&jTzN0=R<`%!aG$24Df}K@G!T z)ke2A_R^<7@;qbhh0#8iyVC3-cDk#g1LWX!`E^&lWcN1J)S=-K5?-dN`w6y9cX|M^ ze#cMfK|X4;p2+YW86*Q~cK{r|ySaFub9l1L&hgieTy2 z^J1Wm;|sAQArCHp1Ad&X5W?S z3QRlCc+Z75DoFsR=cdJvNd^A6%wd0(+xF_h{xnzw3KW+SV24;T$_mfC3|zhT3=>@EUp6ycEYi=kXI(fQ`rBe0?zlFN8pQ9LM)#5Ia&s$L}?Y;+|qUrgcM2&-86M=Eq@PfVMs^z$vSCO38>587p8`PLxTk4yz+Bj*YEH5+MAlx$C zL2b)|XO!EnGr^DKCHPkuP16e4Ruh2}G`&u9z>Uu~vHFEu$&8ep&G-yDVv>}r`hLS395-ze`8n&5xa=l7wQn2P$6UX$f+oh_14zCBxgImG5mg3+ zp&OVjyvV9&hysRlaC;PF#xN_PNI#N5B=gAw+)0bPs?Rf87CGVd+a&S=ch%;V+1o3y z9uLD{?R>fK%^tkD3-(Uw(IyLEp8Zm-)wf~y2!s&n@N(6QDsw?1-VA-%0-j7l6o`QZ z{mL?_7vhusRz@DwyR(E5Y@6X$`o{P!uBuRyYg?5YWi8KZ=nor&d@-={-5HZP2A}yQ z*_5hyUq)`4c^nZzVbe2QRn5LDKLpP2qv-e7kWRbWHm3%=#R_sv zwOq9Yud)+sf$yElyXCw7)St5+Wg;2aQaREmgagz{EkrMRMd2QW^rm^aY1mPtdB5`G}0_P2Pr*} zh3gq?hWkcyhPKW(e}K;i@vK8)3efgAc9abdRm|XPxgF}1x{_HHLcW+~|2|NhOk3*w{ZdAmMRF$@!W>+>9B z$&OepOYY8k)CHbLv$G{*O($2$bP}cB&G}JTX_R}em&njjjV@Azw*co&2g3C$&Y4fD zf!`_>s>QA2x#^3C@uM#Cc(#KXO)%LR+NLqb>~*p#|X^F zTOX7d+`wL(+HEmbW^0x){Y61kFvqtxJlQ?k6NG77kVZ9vwG#1!E6XMG0_dF?s^Zcl zqPsGh0ZCQrk$%SzoE%NCc-uJgArjdnRIXsTPJB^!Fhjt2SRY4m9oBZE3&DfRV+lnk zWJy@^dl`CJ7yfv#Bw7I|XB9#sI^Z~?%GJJ7G$Iv39gX9N^lw7;Wh|!-Amy$G2eYd6 zjD<$&m)V+>Ei1!B-1o0~mN9bavrn_D_4!Vz%7Tr0Vkx0{#?0Wby*O=fm^LZ#NUZYOb{V$69>+1q>F?4XRjtl zDIbWtw85?K__+GhIKALeQIZ+{v043i9`2XHfJo(PXP4ycyL}wl?Er$&i{xQYkLZ*C zLF(ZlHU~0}g+;8xq~Elhny&QSUXI43V!N7~jRPY=IsF}wXNNseA5_-{d-Z}(;+RxM zBwv;%=yQBt2HFg$G;-+EN$l~DT7G$O*HUD!Uq$6Y+_*nqtTV*Z9=-6G=-*xdoZ31B z%>*ahICZ0%sNDPQ0@c=@1x&Yxuhpe4bM`Y2VDt7vQP(Xk|Vc5Zr+B#+M5-t(GNSzH%QLx@>J@P ze*At44>az37>(mbg0eDcweV|Dh+|*TiKWKuswWcdFlG@2%i8GfL3`{Sbrfb68&yU)(gKMpc9nL7WHNp00hnx1of zb_J=laf!8lyecr?#EE$lXVpTxjXcTCdHNkihzv)0ik-`1L{*~|G#O7I?RMbkuOWN! zOrqlVHqO(XtW7aQ)_1x-l%e_${7>_qy@9t^BTFnVt`D_^{(20s=apAd-h(`%?Bga> zScu7FMX`eFRhI?XtI5Jp*sqhizuAJYZ1Erh=LO3lzRL$A={g67L6cVd?3m$Sf8M`O zw&LZ4#ZfkqG-ZQd-KDz9$ytZsXoJ#Ncx&|^vF1)3c9t=t=hv6r#RWII$jjH(`^d|+ z$C-YQ`CKL}E(4Z+uJ}#TM7iVQml-)B7xx)3{eS$NMBute8ljA#e7ng!c4N$VB)hSC z#ogZzOA%A|y@v54q5Z&XZfpEOyqyzcIIEKTslR^=0L#Cp1=?o7++=(YVaLS+M?ZoM zcT?!Jf6kX2!j=IGCtx0r|J5-Y$?mDWQsfO*Twd2}`ES3(3b21ryoZC$OW(SW7<}P* zQU?eaibqv&ySGYl$wzp@&X-X<>;6q1wI(_~UBAu<|0XRYk_z6z+3hsuE&X>l3ZT2| z6*~LzrU7Laq1jb%b~2z`1-p2)Hy6CRQgFR9C8cwL;epSN`ZI~u`|U= zuNY*b+>E5CqN0f~KX2INPrx8)lu}ZoKgx1FX`Xj5jDKNj zwe2`YH`kqF;MIZMHXG5U6Y7D|{|^@yqF30F)@;MJ)6sYysZ6-Tk5=#hCIkG}JW}OU zcFSDQs8hQ1Dye z5IL=qty2B*#{F!HkZF#q?g$6Z4$A7^&m`>dkYF6+VtOxq5A~9vZIE0_0KO=On)$VX`GX57yt-;2d7M#%+Y$oq`jxhjx1m~#Stlu4}^}aL%ls<{mr(=x3cF+`2 zGN`o$?8+h49-y(sci^xXsk=}GP}A?giH-(10m{V8zrP=nu_`$yOGcd4*<5*Yb|^!mH$WRsHi2H?Fh?T1nlEVyum@X!3aakI0BFc^SYe@E2U@Pg zSy};0L;=u5rPss^K+9aPkzxUGV6uoUq2Jirr~~Sq8Q`-N5^Gh#EdO}xdy;j@yJNhi z_e5&{L&hh0*4>J$LntwgcMFf1T_XKkb; z=$T31XP%s8Q$BT#gaD$p_&_7fe;vT|r1TK^xCMOb6)aM%1JGMrfXOt1 zR!j*jMv+?PnWVb&u8w=UW1x7C1X$=R+a%*%JO&g53GN&gx6_g`hHT)?>(>KE(}z%v z`gha-`FlkTiei`U1p`F-F|myE&HvdK2}z}rmdo2fGuC(x*||p)tmN7=l_|^RF3HEk zVJj^C{AD`-In+^n-pC?;l>GvsB7-xQTL91!4Ho?)bcz5phiXU5it3%-1c2RX=)WT~ z03#vSmH8NpB^m-9je=U}wH(nDZfhWsDd?4EZNz#4v2geKWS{uIDg~-5W<>UOghHG6 z8%`>0?es9f`;;&Icwh~n`+@<8c541EvDcxbj^ltDmwvr3>ArG~+uP$Jj^TKn_^ z*G2zpR1wJNN#7>x>LX(;y3G7={vJL@89g=hl{cRG+9RuCgj<5>8hGm-a;?V7?9CvSQa$)))@Mp=|mL*ygdEr4&D?% z@B+az5)nBN`;Uh{E16VbPtnO!_Hi404!P{96W<`ENpMWtb>>~T0*q5_k5Mr}nn$D2 zb2b-Q7npTE{9uxdb568tWyX?SDMa)`01AJFn*h23H4kcALtU-x!ShYGcR!oQwsiQL zDi8{t1h`{g#xp>*$Wq8}JBvw5*;m)QUh1>!Vf@B$^C@Rx%RSrLi})!)7SaAU-wPZh z+1?cR=BSif3DleEM3zUGL`R^EVZ`=VbD0ZQ?=^ju9`B^3CBI%#mVfeDhU4WV-_(~; zQLx4E`Vi}W7=C(QzSb#rwiW8&&ABqob)SC=jze*3r(1v2Zl0YJ!Etp?*ca@4obim) z4UKirX&hCB6lN+{z}EEvEC3Pc)%Fa9_O^EoF1NbvMH`6o>}@#)I)JV$kiB$_JD$~6 zwCP>_2Z1Ixfzt2uA9y-nU|a!y`;wy5&Zj7z~x2T6vU}2hs(TU$2M1bv9S|1wE_s5zsZ{_sX1#&DsK+P=T{T4UArI& ztA%l)qQ^|}Ww#1%Qra8DzH2YZVT=v^qU$N3UmT7Uhe*V{Lt0*A9DKYW3i17>8e@mk z_0oOtiZg*lwx{ODszgletB;)5Z`a2XI?7icJxS#X=DY(WPa~~?jv@9Oum0xrY~=-7 z7d{Xg9?O=s@^>HJyMJ(2=rP@Z?5zPC=>xfj1-|Tzg*FNwiwFtD4vMQ-600}z5f-Vr z+9u7TzC(wjSp8;tr~c!LYUR#50zKNELvCM3YNuiS%+>@)lGfN^e#X_+h(&^ zVW-uoYBqh{eWJR@!D}Yvj(pW8iy)x31FDioww2<6j)QVv1;MAIQbLky7oY@v+jruD z?yYh^c-{ zSzC3+E}9^(b5zbnC{%WpUn)(@tl;3No+TGz<1Dr1Xj+tsGt3u>yL96^5c~z4e zPgx!nP%@y+y2@7%V*QFO@*JIoqoBf)}gs)4Q*@wR4YhicBI3- z%NJh3o4d%E-}H{tEaL(_ZhD{#5O2%&Z3@o8t9hrF9dmsdN#_F{j@yvilp6yBWy%R~ zb?8jh$Y96gvAz!vV>q)8>;1hf7OC<~&ZW4X@^}3whBK>@lx9!C@nqINh^_|Mt*Z|F zXzU;J^sUhr6lO6%2&lX5APkfGbctw z2W1|K#;u3Mw;$3AP)vP&-Wgxiv}RrgB|81-qb0TS^;US(6(P3BnmpG;dm0>#`d-UX zK@eYWG)9nYDoQy009+Ixfh@*2;-CkOmW?6fhj=TmPQ`LaW-Vrm%D!vhDnF zUoKop9ftOo*DqdGk2hvy!%|X0N`@~~aIz5FA)7%Lf?QofKQd+?<|vewxrMt%B__Tj z*J6S)$hb((;C^w(Y&ZP)t6{*HQMZI$%vp54BWE806P;w$^=37@`(YL+i(Yt5!&IuaKDROA1-xK&IjCF;8_h@{$>7k)cv>aWTfHx0fioaA_fPy24vpUKLGH-Onp!)k1&Q;MG(0VIT|E5}E*QRkNNws+LlPGD; z>i!?ql|L%Eaz`}iq{`W;27@^z4nO-#LperXo#)3tUXTf`pp;)F_Ozo?pW1n*3R^Kg zwq!D)?Q>2`zo|&FtX$Q$f@uB^kW5?@FH6{Y`sYhOjt8slQcHJjQnnuPdezu*ISv-? z+~~YxiryKkv3C{GzI>O_3sZDeYHJ$jxfJH^Fv6{l(JRs3N?FBF6Fqr^r~y50X*-); z5^J&vl7Uw0Z2iGhiqZWoRp;pkEBa_Ozwv3E{~k%)cEgc~q*kB6qp#c1b4o`tGt>dy z`#XWjxp(B4h%(DHM0WIrI?19VgOoF8jb`t0Xe~UwG_b$77EUnkcR=C*JwJ^Mr6-1FdVU(Iwk*$z4wf2YF)dAw+I%npi%@Buu}t41*FLa3(^Ta z^w2{ONEcMFp;+iuklq6Xl2AoKdaof=0U;1Nq4VB3XPxU6VXKU_4-tivC;XYO(s-8?yUU7FdP~w%x{JiGtud2 zCK}nQm0YhaIkh_YhT?-^)a%d5xLU(F4g;sjwSlgTHimg@32pOJ-ySa_>(DdpbN*G2P2-&5&{uIsT;TlIEcGXT zf9d0Ig#d*+m1~xm75IWn8wb@@LtENK8tUzs?Heu&msHV5-C1VsL0_+J3To_2#^RJ^ zVqN&`kcBNM6{udDn)wp9Pk+aCpF2-%dSl#J)mK7(g6hzxrmRrGNT12m&8|KT?nvF!1HLYwDl#`bg9K8~x`z7Q%|;MbLj$_L)0res zo>)^7Od!rxkI-^V`iQ zNwJl#*ZCNm?kk5ig>^k9m0g!kp>r`4Aa6|?g+7N>#*QXyrol&CAhj5pu07ydnQyOU z{UFgzG@?m@Z#r-qbbcLM*#7}V%k}Q@Li+}N6YV9jVwajj z_%`UO?|~Xc`WJktI~7a{Hm%IE)VSmN;yqgf&hu*q8V9d7ZhcCbGP7L+{%tiBu@n^kUjiC$VpN2p~7UA-7ee^p}vZo`0cdI;EGMrBK zs3j)Mg;HyNALGE@+cdIf#MSsD0x25IYb6QsM%ITRpJ~kalkrk=Dedv&tk4Nf%^QUpMT19zvxBp9N-va(WX&XZ zVYlielzWhUqH2D9U%XWOf?&Oo;HjIAJ7gPjbg6e3!LnyA>QW8r+xE<->+2`1{7?Z= z&jKk{#B4&*2>eQWf42BNmS5OC&G;2LI?WE3R_BNQkvIMx`x~>cn8(LN&5?CY zVt@iqhB5*_m;VDSA9~7$5EUJF``>f^XI~}R(j4=Q?Zy9H?GJ$Np{E6h5Dc-xKVX?Z zf#Owo0jgd+ocF^v{*%wQc@K{Nl>WH8Kl%9|{a4_pHUEAk1~$CARu80UP64(d|MR{6qcqR&ngZE*f0^gDA^prh z>>@t|2cVhm9{8|(v>aUR%QQg5KK-nuyG>5n`13-*`&0sIFynx>+Dr@v%&>VEFz3Y| zM1u^~Xca`Zy_=-6iC20+8N@udq&I7401zXrVx|F4^aX$b?9=+}y~hB$`w~FTRcWPc zJ6rF$wakH)RQL+T#l3Jc!mxF*2rxFGl_rFPR2yg1ZeAA~uzV*nqHt1LTVeo+Au$3x z*VdtYsgB~HQ4au{E*}am2}}=8*b*oRHpN+bX6?7k^{3OcGr`r9T z0A?QG(|7z>MQqfK0CI$Jg39;<_MmltFHkqgT9#(lE*%;=ng{%d`psYPcdo~SCukQ4WJruE!fL?-}|DauS=M&sZ3^IyJ7lAswBn{I>wvXMyqZwE#})Om)2vTjuX zz8eL;WTnuin*r^z5Xi1fr3wQ?B)}IyVYcUw>zgvE_lmYH_>*z03SS(I(V?cr7ji3! z`G{!nH1@Pj9Dog`y&0{7`+D)kXTZ6QmJS8vNhzgJR_JCq_VD8j3b+q~A?YQM;~z2; z4E#m`vYUn5?r-M-u(S~98l~Cx0?z3T0FXS|1c>_?pjoCF2LMEu=Iiy;V?6SLW^)Hm z-V95#knQOPMTcw#yUw=Z3LjlnB^&<{Kk^sAB8=FQH?069@YJwk*W~AiDL5wmo@;rB zLFwKI@^?{wah28m{T1MfPCZ7Vu!s?ObSRM3y7Xj^y6f>G(8n^p2>>|moR;`}nb!dF z9(Vw>usS~iDJ545oO2zBjIYM+wmqS`VRnZN5a?;Wfhkaf9>9UR?Du}Y#|2I?6u6d3 ztbw6d;aUXtG#?uksl>A+Gmlh&>|5&ydA@X{4q)K#sEm0SJpn|1W4^dcbMjivD3+KP z8?ZREGsqt(L;ech~}c%|lw*>%PoU}WQx75s59BI^#%Wn1ku zb^wtV0vV@|Ja`}}1*vEaIC3?!HOGlf*58+w?66>FMhljHTw4XKvpjBzAgj~YW0bGtLae)^Z?I@V^^9Acx^XlrU3wd>>R}g zp^=>JBbU^KjrIhRKA}L6?8ZM(POyS3D%N^mP#Dp`rHg~YMBf2q{^LPil#0)3Wxh9` zw9`JAN7YGEXrQZ6^k)+XKg~%@MP1SX$RoqK^yt0)0B4!ehX;emz{#7H!VkJ2;8av~ zBY@s#gQUR1>Du^85aTa( z+~%M;zzIzb*laSOX&gD@d+I4At#B2=dnsGu_F(?GWudP; z&C5JBg!%`;9G=SCr75a1aqB>Y+TgA0s!c_Tp*X}7vCQk8eNYx}I=%_Yzu zaUO*6##xrRtxp8S?X`EttJT{v&)nvhEJM~nW`$#C1~pyo zwbvCi9E;y>xR&^F5m4h>0*)Bj0$IJ> zw5A&fp5|XfU<{8yICp)HW;;pl%n|34y{QofJP=?8xdAK^!?P?io8!e01`b6nV&T$X zc^HmrfG}q1tdxfX?Q#V0%N98&K=$?nEghYL)xdM?^DzbLBKoOoz8@t$LG9*5kO5Uy zE_#qBW5_bfm`av(vPt9^^1kU;^%ngJ0FkCHt7G>yrPeA;DzR4a?|!sIl7J*2D@A>Xl7+B1g7Q>CfhqGhfxd(c+aUR zXi_TK!pACFd>@m+rt7;y$t;igdR3$$SB#ONSth|ttvp7VLsoa-tgl&K#R3Qi3QrNNd=bHna=$oJ9twwtiZQiHJNd;RnJk zUDaERB7t6xP}j*=nSTepfBh}b$f)T;-X>Z?+g%m9cc@EdzoZ2fq22)vcaLmpOI4QlVK5IzYbsac#YrW>t;dGzz z)4yBXV_c(<6VUJWtV6fQgnk$5sTx|jVXu9+NY>84Qw-KK^0>A(`1lKo$W?VT?P&8F zpirF5a*$9$aYk2a=r{X?x$zcdLn1F=KvC+he|;iiy&vO72>s^zBP{=a zWex{u3?5T5{}8z2DcndXNF`#Zx5x`7U51KF-frvp5`8d+4Ubzt&ENu0pRs|N@0Q^H zb8hoP5Fn=y^0A0XN6(a>KGPp}medV$JY~e|0h6EQJO8>JE@hDSpoZohi~lamdLIKe zugv-D+dos9`7?=xQ}|^OI6|n-UxNL=ejBKHMsSF~MA|CgZy$&c;JEW|3CiG`&95i`XmqWU-uD%o4%(4a@OwksNElUG=F`Te++0KiUB-O6M>}k z(>CZoMoBL7D0r8H>_t`o@!tL%OF)y201pJm;0ZQ=`z}SnyVS&6tn2>k?f!Lx{MSu4 zYJvyi^Te_J{51dLTu;3U-ld&S^WG0w&<|7chtvPRlk>lm^A|Ie>%Wup|7GWFNAr~q zhUz3y(*y&JhM)>`)35i3&C@obfCV_SX%N-)bCl&@>z;w@<-=Mb=DMELzgx}^sk|5k zhgx{k>3)9x|Ldx>m%tuzRyQ*1-@Cz2zWN5Ob{5Z%gWUhS4T^H3V~%DCiWU6DF_^F&RIBr%NXFmPqr2n7a27aPsiT@qY|9B?+ zdlvk6K>s^K|89N$Z8iNjDgAnsFNcz4mDSkQj2ddjrr4&4Ky%5`Zy317mYmLqrLb zbosbaxcOI<Di z8f)(}+fj5XS^*_j&ng;jjz^TLBrZ_6)x%&w%D%l-<`JI-SZC-y03Bv_DLf~BCyF=+ zaPY2jWo5q~E5=Xhf?Zt!?g6&}N*gpI4p21h>l7DnmbB^NPXOEX0UYXU@=y4gU*`aJ zxb63EK(E6s*w%XT=&oh|6eu&u(v1Pt z_V(g^;f-I2bJM<$m#e6SS$eO}u>g#E;{F}&N+7pk>{5GN{P&(DtF&i+@}QVz^srzZ zTw9F@-!LPc+6LtRH0KD;6+q5@P0?MZ$pl^sp#Z^vP)bVQfvd+H0Pn8@J^SzN9~5E& z08nkqWf2^lW*$I+I1UIjaqduxRu*p*neKL-Ze@3_KClEz1HXgTKg~N0s9`|xY()Np zh{O)GPc-5HQF}&Wj8}dU=+Z=O-5XV|I+C{1QFh~r&qAlbE#1Ly4ez=n`$@B(2mnF4 zQ6UTH);%mF8NfhUimckXQn|mk&G|lUgqjHI8v1kKt+;&u5TfIFsB4*ZY?%U@QyTWW zmL>P|mK73zEb{(!v8oSK2}d-Gb_TVZ1Jvif{9V$;N}z1bdF;b%sa=X>*u#+7n?Y$= zp?0!7vl$*ui2KSnv`el_6C3;z>#5uHDK&_4@cWsAH+=ZX;_?i&MV+Ntq3}X}Z2?WF zwN?^_`bKLNfRqgaHf~zZQ@T!SNK|xu^rb_t%H^OKF*6O;ylM?Mw4_HgYx$@Z1-iQd zhHe5N)c4+jci}aYQ1!S<`{0=&&5kXA_lf$UDU~|vY7r{S;dI&!uuafr`E9DGD=Si} zKv6Q&_nW3kpw#q?0?hU`3@=Y9X|UW$84QyPmO1sQfIW+JBu{^zhS`Xo(g8Ba9-IN!2eqWzF@5`7 z?eD2lltQ0VG=+Pi%Pe|8sikXr4iwYwcNv4$jv3*GGdzx9VAeRb&w-L|Ns~AD&@S(d z@@22&>+XVy)-tyN|Jo|cdlET30!ocRW>51$*_Xjw=%X;Sv=|f(B{Jw+2SM$V8K7>; z7X%WKTcC!Nc&MistR}U@S_gwY*p1nYl<~)BRHJ6hUpvIa_$PV{+6rrLp>_ajdj+O< z%cK;XN)$0B`{kU0Ig>l!-V?^8kVz;4rKX-GiI z+4V{+odFN??7?50oLp=aabfM$Q$vE~+>CjYVkxaTC~PHig2Q3u2H0RVIYj`IFWpY! z){C6u=WYl>OI_4PD8)PeY0%a5b@&yiT3t{DibIsXA|8|M`I>;el|)njPghx2PK%tn z0}mCP42pk1`!6j3AT$&TN}hrgl@*3_XNoODd0;%GBk`cFIkmApvY!wH8@`tFVRnJi z9tGfgtD+6Vz^kWBAAAGb7p@nnv3oCNF#N^2BU+gLWINF1u-0iE*`|k_2j%SSV;)|K zbQ~G9!(hO99}$K&H{wo_ge-+$V>p8XPEsTX+?VQf3=!cx9?>BehTTHE(qFH}%0*R& zzqp3_@RU*=)kenQ^0sdJ1N{#BxR>o`Z{O018*_`Ml%P_~n|>ocS+P9(o(~vZk;vkBjv$ecJ@paC6YBm3>U}C~wC5 zmLP(IK`?V_Ojva==d(uGCpuPgTl69syC3{*zy)$L!` zY=^WW>XraglYL-_LtLuYEq%;w0LVfOMRraUWN^d zbX1KRaCPdmer*~~CxvP|)l+qBzq#uAF^-m#3gY>wMjr0VR%gElpGa(0@AUO|V^b*6 z3cvr|w_W})O;5!@l`N)9bp5(QgWFjBzQqFMw{O74`DK;MNEuG*Ld{)g&m)c@>@~2H z6^|C2Z) z6^p1E8c9u=Tn`wYV|`VQsXsBJjmx;7aC6{r_NVPJjlf8+m8k&`c8%iR>J;3S9|wJK z++uV>_v?xJ{JhR=FC0fRlOv%Ub@ zdn|zT(lb9$hZ3{v=Hj@lqvQ+BihM`KsRd3-%?VaXtFP|@O(ov~Etb43twWW2?-6m1 z8lzbKA}hm}dGlai0e>tk86khj^AQFOpEZ|+cHH0@+^YaBNtM%7qGB3-OO0=P6NB2Z z0r}X%v|so*SEKpEZjQf8dm+@^y}JYgpovE~T88srWy?qjxG-I){y3Vk$0T4p_edPf z^4K7@(=<#apc*>P#mZBHdBa-FWwh#C33pk`ct1xXR2#u{# z3ig{0-h@(vf%@4XeL`UsIA8U#{z1#(-n(v{!h$5>N*3g*T4E(pUSMgf63%r#>%zB07&4@4lLNPLS#TN>3+0!5mzw(`Xk zpiOL=I-E^Nyp3S%exQIqwoW(FzT3kFWfiG5(FILG`0+n zpE1FwX4p%yAy!6WhDu7o954b(W`Ad;<1S*VIl5BW8abA=y}%Qe{m_AnHkm?d8iS*6 z)4a9o&A}*!P4f1w*@ClvrwpWXXi$md#Je{#9UBNVFfI$RIeYiOIhr7@BMjCD;5FbD+C^2(l}BvHrL^^vqXGf0YspCDmElf0y&1F76F_W{whuWn}6G~hYq z02&k%mEf$>76?62fb>=G{#unrkD6M7JctAWi5r_YsDY>LH9< zk-tDgMDF(F9_Y7SdK7+P3utmhV{Z+Ew9=1U32dASL82Ywf3h(__t8uD=Rw&(hUs0K zao>U+=wwE*OM6FQd3&W@!>?3ergL;Cre1ov<3q*Do9D3)KntW@2kmrYOc+IrdPNjX7hFYtT&oI zThJ|+i6wTbCl`{El7b%2Fo0V$*69mU+_?riyLwkV4e-qA&6K4>m z?%2g{ug|j$FCQ=p-3HmB;n%qyIT|lYuPzQqTcQYBQud!7Uz=spSFC};>#y>H#4Yrs z5bs$wtZknwQWim25L>0^Wac;2M_b1uBVeE{d#_}QjWsxf4o>j7$&gC~GGDepaWaUT zljAwIvDcLhy6G1qOkSb9!pCwT9~3miim-5<-V#&+_s>1*I13PT40g_*_%!(ZDXs)6 zY*h|QQQ^^Zz>VyFe>b~%aM2hms(XnJ{we!SFiK^DRpqf)rjI%NOFj(OdzeR|^t;)E zRgD8W$of26hJNTldj1ry7gz-yP9yz2JNmbY$x0UtJagT?(@Hw}+nFXWF^uLKS74oN zlDDaT8EhOlN!b>?_SdfHST#po`X!D&SC$RC$2h* zrZHk}R?CN){_9=2f#O-$%{n$G5kjqjBH=jEIB)irz9KpNSLZiy)@5)yAmsUiJgqQK z|FyIln6A!&eUn^cL^y~us|`k5Hk|JAdjLGuMnhmAH`+8lA^cB!+p6A1=9ZI8Uol+2Ehhx!NS7Qn+4@1f*q#>*mdZ2zq$8|*aFu_f7lBD>K_I=@SFd==>NxK8n`^uYhE4s zsOxCfMOJw*+*lB_~!ds7wIvZFOF_4R-yU@vr4F6lcBLqDRYJX6%Zv*0^( z>*-%V80~%2t!4?COsCTGKHt0(+?(S5xCq7OyYxFj|KhKnLA{;o`R2EhQk@_i&91GW z?9V-9|5*M>^wdX4OjJBYKebfq^}2 z6^WJh`_h~Fn{tarl17o;y3Pm_+OTPsyB=0&DH-p^`VxMdjU=VXpnf{Q6AW&hPfe<-hjxvn1+E zCcYE$AmTu>jPd>&Y_7dPg_5klL3HM9L! z`1LDlrgHU|rCy5vdI5on9muIJbF*Q=VspE`U$-8OlF7}Q^wgn|*=@*IE8I(ojDilG zYw~zKjchCk;23Uh4CjC6?uCzfhe9 zov%@d>3_2yYd*4H;@7)yy!+J~^7u9W_gz;aVyW;_@+(Bt82mOQW{I#8c! z)S3pvw;1eL{EjB8^=IZudm8Z?mbv<&$+5dWanaky9YWvgrHAGfegUCyQI)R{dY(u!@_#0t@@_z9tDa0Y|MhU{>GL_gtS5cC0$Wmoh}!2Yf$01#qC?Yh4L%-fK1 zKIa;iwobLi+k@&?iG~#qBsXmrvCy2Pff0h|m}7)`KX@o^JAXPYm1EJe19>%b>W=d0I&5~}(nw1Q zou_S>P+4sG%p-l2*}*#8NKX-j)9B?rv?OL^=+SM_QXjINWnZ~4zb$`>KTW~WJCgfB zXVq~Y2qMS{42lMey&m;u>k7zL2H|MwnUT)AbPK~3IB32xq0gcx3o7Qj;hb$!RSC^d zuc@k+{l)f8FQ7hJL%5)&+rJG_mq0d1`uNSd;?w=%J`bX zue6TZi!xsVZ}=ksN~WPYoHHLYas zwLi=bC!RIlw;wr>J@*hT>JVh;SoCsPC*Qz|bE5n_abTn>LB&}-_J#nhN;N!tKh+v$ zKZBaTZPec|E1qq&l9s>t!hSe8MmpekVe)Q{4Tp-}EE~sTZ?1#ugVL<}lgseAXtR}C z{EVy)uaf|K*C%hWOgja_GE?rLsqTqrYdloI$)_~Ku7t3Fa#^GG*xL^1KXY5GlwhD; zI>MY=m@*PGTjYNdU-@dcc6TZpJF-a*;43W}c3!~A&F{Z*f=jdN7Bbq44?+`Li{(kB zF{rYUoscU2B4yY^!^xwiev(e3d_^?T0}Y9=_Kp(|d~dDmi{zMdU>-Ub#W2ptayQK- zd+rGuR&_W^f6R<;&qbf4k|lTf;_{;`H+h`5fh^C!@?=x)y>K3#JR^y0gOV(hYZW7f z7Pr|z`p8`%Y5w^L5oji_uknA<%}Nvs1uhKpk8l+KX<#uC6V!2>Rit35B^D3L8Hl}| zO|-|nfg#!Xn3f#xTZ>g%ovAykW1K@pdv0{xrSS4ykex6aL1)F z;fMt)F>%ZiWZyYBews>|jLQqxg>?+*N2QllFwHODg7rWY}QUrYPd`H9uy(y{Y0u=$Z<4Ned`b_j6>dL;uhmx&HiE5Q_O& z93rI+VGrebU!-6XXNj2AF20)=I}jHk?K^dks9l+d25T%@`|sYiTK)OM08ci$cMzSPIEd|-KP-DV`>u6;jY*Kxrkuw8wM&(Qj$ z?{)mJi+p#Rk>}U+s+$eo=#~PrY0u)r0S2bWvm!hP=0fK=65CK!0aAU3?PZ;h|K92b zi(2TuP#c`sGwh9#TWx|oH6&JcSULJEg&FR>fe@9`uv`YoLY%CWDEkP;UQb$I(d&H?${c1%Kiltzr)VPl1`D8 z3ULTil&e?x>98~7H|?W=_h}yVaC zBc}VfbS_f*6HWMQV%z{>dEQ67 zSlY0!;287ov9m&_@iI4tTRjdGZ^a5fV9qg$3~!0eft0_N^QVCjN-9;{$?b%$-C>5a zyUGI0pOFJ#-~P?Byvc||*f!eaGg~cF|H5Ib;lpdI5Ejr$fkMKRDjT0b1ie2`EtCXY zT5h|=EV4HUUkpLntQ|@;!v$=|c&xfrZCAbuNGR(LMzk{ZKFDfIfjlyaF1w9s-mdA2 zz<(i;lRFE7@%J%^dS=I%HrV2NK%UP^(;>|EDE8RKI*A>doD z0uG^z4lymCp>Yq-%$_rn)@_F(HG1=?DvGSFho|*w?BH`^4*l(ivlZ40+1C}rl;+uzm z8lDE4UsmU;A=tg)D=WyG-75>C3%ksVu+v?Bq?12l8r~F+gk6hV$zR(jqnpbzoc^Ng zQK$xM_k2jyT#Ex;vf|byCBu0>leUqb#jooGO6p;YB@4JN#NK3lZUmdGLF)O|12jG0%E?L|@%#q|r-XCE9pmOcy6e`7^r~^S?9OR93^u zvJ+T8MA?d|*~o#DxuXJP}$+a}tIf$D^8F$0Eb?PrI;VURN6m^ z>H&oRz@{Y3;R~%68}iYGaoeAVp6-*M$m9xRy(bb;#}jX0G4WGaz1=z*4bqPHtK(4x&8>vY1V49gwh`$p zta5ASs;%aiJ!k!=w3m4=Js5u8{POe3s}5dOt$r}2MPKH22|SWNSCadT%LkwMRusN3 z&!)=hgi(p#gCLdR9p7@SzsHI#H2rOCpSK%h1MSX~wrB9AFx^1=p@nZ(`jCrPoK#!3 zaGNX99&d;-UW**O&7ODXDvNlK1gFAm;;i1}nZ74m+|owqiK#iVe}#}R$8NRzo(s7a zS~_#MY`ZJ#yG)go5?0ojBd8K$vr|O(`IWmqW`EnPvaM5k0O>ckrB>QTsBo??dpR87 zFVqFub;h3W&I^W(qkLX3yf@3=y)mNdz2~W=@DiJcUx{l}?^E=4WB#tAKkgs0-Q5(I z<`&DU(?XM+GU*d(W>kyZ-K*+&;TP}kf+S9QU;REsO57bCYnh`X?=d>56zxH3O~}i` zgNlLo;ki6dGg1D#-W=U8vm=G9OFcAx(;?%ay*b#oI)35d_zaCp`$qX)+-4QiEwR(L zb=7qbeUP#P`^TxAW+VRF6QH&Nx7Tl6?%B9t6W1!l@{6^f%*6G+nNSw4&fJ)Pu4`ye zMxsN=Xz`mn(q$GauFAJ&#Qa47lY^~$y)%ow)q1S5+{MU^gSuiqdq8L3WFqGo-D8N0 zeA2#D(wkg&`IJ^j#!CJNI)_Rh|KHzb^`>Y{ejadK!ag%xe+{Wl`Iyh^k?$1Ban>o` z=%IC6T69g$cj}Nbl94PDR;lA}np2XiIUED!b>hIup|CowwgZQ;YrX@>ONOoRgO#17 zn38~TaiVc{a;U5R+l9pbP9!6&SE-j8YF+mmQ=;-1IQ>Fs2{;9YMO@mOLdu8RMwudHO^;*M$L zshhjK>|?0`4(l8Fz1uwvVxN)Oz9aG~4yf4uzITy>eB{{qD8&6DKchb4NMh$;ME-zH zxRK!Ilz>$N!m>WrFOuSl7K9*p=tLA`6SvRmfV>Ief*ZH{a0&g$=3M6WWm>TcLyb{x3 zoUgV-?nU6#F0pyQb(uMhouGu>G6C1X>14iH^9@iLYC?YnRiq7{udmp|TRp{d?i8V$8g+=kC#AS&K^Pn}1Seg27?NAs{kwXie0I(t1cu!ppC)pUv7lQwZ7t!tMes=` z-lS~DNGL;lkF-b0OLbteu^y1xiE4~ppu_oMk*78s-!#$&o7`B zyA=zmM0EXxnueuHkU62J8zaiL1Fw4Im-oX5a`6S%i83WIdK0b4=u)VWvm)WzbSfV* z@Z)ome`2_}_0_r-1(;aAOF+-1Xg>*lL!!ZEO`+HbO>=iiv5zgb)Um8BFT#z4@0z2I zeCC@(m!Lua^vv1CNmD6@ed&oq@Cx0jtPlEaD{G%=Zie5@dpOIaSap&tb)D?ffEcJn z#_xoZrg50;=?jtWC;|t2B>y8>{LLOwAM^9wlek&*5PUF%KXR7HY7~Wi-l|ZsBUGXq zQ;M&=%y00jX_R1ouIF|aj?)m8(vNcy1)xLv}U+!nO)Q#l!$!>D@a~d+o7eSBvxUDA39xOK7>!>f2NP# z5_T={A{{BqL!lMnG@@cSW-c5%S69KdEoYrz2m5qY(kBFa7gq>fB=<}hye|+<#vb`* zJUSd$G&ijABDWNmzEJy!;HLqLY-ac+z8kno^IW8iI3CrDgO{w@>)xx=_HYXEHXL5j z%+|#dY9oWH#V!e2hadL{_?rt2(S4ggRDv-%q4o2J`e#80yyR~R01A~^jAHm{WcIHg zaPa`-Q1MEOD?OKGpi&FE0@6koU94`E*gE|lps+251!d|$pD8+vJxfhB^+u0GI(EwX z0hWDJ|4`S(KR@rUw-k2&AzS6ucVm9k_oAu|{S5U+O4A1BW_a)EwUoxs@)TsZai@1y zxJj{zIwq^cNlK?{(D{`Gq5qd3G3qG9GJHGf@zJ?n6t)GMtb89{;g^r*KgXRepGq>Ex-3{8`m?q1=R-1Fqm;B}x{@G2zuF%IA$K~k_z`Y0 z|9pZa)BnSjIo12AY!7?V;Vw3F{OiU100#c0V#A36xa>EvWAcCb|Nr^z^HA`c<|`WN zr+zs}DIfX2xv1Qxxz-sTut{^%$eg>uHR*il=QDvadhZ>xZ-RIgurk$fj%6n43pk2;X=fWMnQ1HgCh&}_^za(e~} zyll&_z8if3Tu<--`q7oSgDF_>XDV{|f^r5dy!8cZ*I%dMpRexEA1LwxNd0Mz{<)tA zuYbJUFlKPmUK&mHB4?O$(6UVV&*E!4McL*f^|Ku&()w%4pZ(p0!57E{65L96j?2a( z`h|EfdkMyd@t%v_jOe~gXHb1923|PtK5vgcc@|l2nXhcK#_!GB_3t6-))--G^sz zJt%p|=($F5HywOXG;gkL=yUY#`eQh+cB3HNfsosOH?0<_Xjq338Yw{p^hiW>!EEqG zxA=a+g(BSneVF|qhjO1Gc`JOblQ66A6q)JWqKcfas^UW(G|(xDCEHp?`U=(>$Cfq6 z066pHLf_X{bDB+c&HC{4irtV<*Fg1b%~sMtMI~$?Jus7ZqG&kAZb8+^QKxQh6(Txc zcq!I!a+#!fYv6+mqm+FJc8?GWG4<(@>OzoE$o1Z+FCG}WRex_yUSf9=ia>5T=d&@b zxK4a>zNp3Q5m8|CRlXH|zLt5D>@)+#_mmd$yVZ=l&pIz;D568?E{nUXI+ejQbC3hA zP>yr<;^*4Gdf4gosJgCIeGj!2gLkmnLve38LIQpc#r(0t1F4RaRNW5rZ|$>2rh2C} z-KrpmrFj_b+pc%L3|}AWXhD3WTa=NqtPGzwZla)r!s+?NyhhdQg3PP)gyfj>D^=WL z4{BI_)}E&I1~i{0Ztq-X^MYIB9IEfLi8_^pVv{N=zEjoo`HR-S@(5I?u^K@J_LBi7 z*aS-=fAdVo`{!okZKX?_IoYR3*4y3fn=)?X*#%hXS0wLdGXb#dwO=z5#kR0&hN#vm zo!@R4kUUsQN&~LfvrD6*TTC!Dj+-{YJ)Q)qeCHZM+-#DISPRVJ{2=^yYgehad+ZIX&$v_k;c;B6pRBA21#;{a5|ZB@$@s}WiOh7pjNYLRRX1MMiC@i zy!9a?4rg*fXv7(GJ|h3zNGDUCfzvh2{o#4S&EOV2IMOIM53@1bbctC%768)OaOJr& z@x20zLEX}^X`u0sz%tYZUY9} zA#_Y5e$HojO~iz|%Dj}U{OUjvA;kS_;k}ptb3^*eQG$P3o%p7XO3#eOfI}i#A1#sV z31a~vbR1)LUMRn?yqJAC334O{)d?@na&TrTVl*-;O`uB{b~e3)C%{+PyDQVqkCCFc z_cl^J`cBf#~tEGpzZiwYlKxdKTPL*mJSy& zU4J5o@WFSPDAXsgq9j!Z=L|vJ^;5=;$A5~w`RikW^I^4^fO=95!IrdF60mQsBi5b0 zB~lE-_Z8?G1%nmKI6plAar5}H*DaZg;{mJ=+nI5YD)c#z zpB{Jy(&0NJQY9tl2R{U#Cl6hYog*nrjh0^Vn0~4#w!L%0Bef|B;ahLB?f0SSt%^n? zh9+T@Dr*5Fxm#&kvS6T}BD>#oi$l+USDe{BlsmmN+f3kuq0M$lOTiXo(RX-Tcy1_x zREaxiJ*5|-1exgLi7b+h*ujtz)jY^YJ)*IVawp(fIN8E`O)V@9V}9(XzUEesUr|P^ zZahOAe_wh4-BUZktfy0~1`eV=kEQEkMB*P?oPBAY!1E z@W)lR=Sd|0%G;iq@D<&hqT5Dwj+dCb*SO1&JcCFdOOJQw#P=7Cx4okWK2>2yNuoX? zD7(lK9{pVf8$|5XQ_jtyfk80x&pnSRX4jk(z;eN zbRB|QkAjr682#oZ75M>QL!&3!g7n#A@ty7Y_&m1?o83~Lip#|yq;SNftCb!h?isfB zj&oM;@@LH5h6?1Ti;HF%n+&oC!yU#Z;}OAZVryFvbgK#;BOmTy-;INdR-HpOoNGLx zlD$QC#OFVQx~=HDDq6>SxQQFOxp}fZP%@Og595~gbf7Z~@U*JVPQb}7lsaPQ_J(Vj za8>!Oj@FD#;dir77?4VE-9?EoxxSHrz6#?2gl_BumwNRp`y0()zw|aD6$s_Mi38;W zN~1JXwzOOp;_=maR_RNf4T~yn9Ro^+7}r!vBNTe#mDC{M96EtUxJS~9)d%$_ZUTKh zG1KaYIeiTYfUJCyLRB09g|&B~<0Dl|b!@o`A)Q{ev#Rlu`gk;AH!*<&_#UhSUyRh% zuePS$vQ-{{Y`vV=0XVVucCFvMxv0ovPQzll<1(qMV zx!!LvfQAC^_ZOf+1ukaY8du)js`jp|rEx(lEoqt=R`_{D-wcs`Ccb39eLc4ffln5@ zDPR-`SiJ&xCxN)g??(A$5kS_iy-&ZY;WNs4>=rQZNklFT6F_f=2@$k1NH2fA^ekm^ zL$XK6HSPiL%L2O|9y}=CR4CtY(h?mG*!SBX20efLRnt!l%}VqxgWxkPMQ;3hH?WSe z22R$`cL8ruLe-G|(nlP^4>(270d20;7jaUac_U)`fb7ftlt=I+H9Z%ciY68lOKqLs z+cTe`n?9IbRhDDuxj=|Fsv|=ESTD0kt;`=H8Ka*m4pg@EAl>fUo)z|Qw&5PJ)4b$A z{OnARJIfcBE9sJP!$q&w*aW+bcf3BpD(Vvt%nRl>(Dn77?SrluTBDCTfd%`F`N3+( z?Rnt*QM$ifwJ!|Q+XJd{M70>?1y$xMV32)O3Ox2arF#dHC)wYrc=h}PxaCKZjKP4` z6KFRI^jM6jN^D!alBnZ>@@yPpH4uQyP>iqB?%TkOaG*c#AEXHRxc7`NSkA7=0tZxz z`tCCp-ip%qg-j6Y*;^L=(~-lPEOZB`$Ek}ej^IBnJ^)&4SXI@=4FpP+2{_9jt|dGe zXL@HZl~i|WWx5S?q|!mkst+A!wS+gNja(HJ$ItQegiW!TRdk-KP``}>nwl!thTEjc zap3|c!;Zi=0|&fV=Rvy^DhvAJwiDm7T}Sd0icWhP9S(Hzy@dg#t+W@cz{;G|FN|Gy z`snz`1<^%HBipWj03b~Z?`qJ0&6nGlC(q;iV#J*sLDOdmAjnsPepCcdVJw8^L=8*8 zirJsxbAU_J!se_kWzyx^KVbK_eo~Tqq7?{T=E(-1Y{Gz!0+pJ?aTJ1%{U%doHBj}O z_c3P`bBL0K{2$8xIxOmK?H|VlK}iJxk;WnglxCz;k?tBoK)PgxZb?a%knT=thLn`< z98!?(4uS8Qea=4T{GQKq_I`fXHGhnv%zIYc>t6T$3KW0s@+uv$2>cp%(RFSK*NyE z68?nYcy~i%SZ#J>CacL^(CxXOoOtrC)yd%5kEj{1+#NnkPoU?XTl#?pQ9Mj$6#S0Z_7ue<~Oy- z^2``P=3M~Y`qFKL{r54_rfPjOA1QQwCLjOYzz4oLF$m;d?+bZEN@<$m&;|57!Gxl; z6o=aY_PSvb@H&5fATiG^T!X(*C|rPju6^5v1E4DLGeDSqtb8q5^BQN$+!-;DX`Pjx zsaB|U4EUTR()v8lq(~`P5vz}`mC#6y6>|J)*bk$h*`3fjevb$GVA_CMeCcFvP#nTu zI0rDf=8)&VyY;T#0`;=?`&awaytXO*C2DG8r+bT+ZFPYDH+<;KI4nP@i?Gd0sTV}L z9IZQ}%)2^?rxlz55#TrGO)~#87QKe)Z6LUcvcjo5Ae5q9*vPhi)6cg;X))JOf1LpT zmIFBFf;TmiUtpqP;K=;}t4C%7fzlSB)yhG^_uk0TR=v^QicOVHHg1W{p)a1N|LVA^ z@eJ4NeCry}qmRLFo&r|&8vz?UMNKpKJk$E90l$y5ylnqsbu&vgez~fmi(riV7jPW< z2B?Jtx3d?QQG6R8F~$_W&I|z2Xg~`qMUII1u|7ZnyBm`&1ILAr;g-x9RDfMD(u1#dQ+d^>eTpPznw zdulxIb{IGv+O;Yz%?901P%PEz2aUy8-HMmxJ9;Jh-ZX61H17dpEPI-iVNv?RZEIzu z>9v5z>f5@~%p>TShr)rn8sgXyfQ*o7a&?O57_et3)!1{AVH>efH9#bJtlYE)XwAU_ za=#CU<+lI~i~EQ>t*InTXjGqdkM{r&O!ttFjg#nK$_<`8n^_!N{hcKf`*R~pzdUR_ zk}qecbUkD45XE;0uQ=2~T8+LadV*BT*R?}xa|s7otB+`~8*TY5S{Sp^-REk@>jSPl zyYS%`Nr;qdV7jvqo_$mYnz0_*MO$K}M+4l79b@LFxtBg+x$wF(5b!T%pA5}JKOOrt zF7Y@ec!=q|Fn2Jfq3Rs3Vs7kM^Wr(z@d)3?_R(YAix7Z%Iq%o(z@DO6eIPVx!-=4S z(DZCM$_#H@WXzRYV`kV#R__DqQ(V|fm~~EjDO1-(BZ^Akg7)fId$hVU`qSob4(sHC zGt1C4`Ccfd%Opsmbms53L3=x|b`(?S>GKkr)1bu`M+& zKWj`&f9(l^GYeFyAhX!xqp;arLjh>_FgHup_-GpP)mByWSzuUHS_c%8JbYAb8!FQs zf|Od1%OJ-t(ygCLu3{|7->`w)R{Ah)bU~L$WL5`wI%7P%(W#aF=H;>?H@t_K6^sz>F9CUW6ELHbWs?PG^k~PuV|)miAR;4v&oLR`@=uJV)^11_!wndD|y z)L(WO(=<85jRP9f5jL=N_pg-YHm@qE*;Vr`Nian!3RQ)r>cv-U(BsTU(w)t;DJs(u zDMdF*b?K4gZH2KuyXTgT0Q5K)@p>^wyH^E1awVKxIJOocc4zM*dbQ!DH6PBc{L~Q; zAm%w7NyLZ!!i+8m-0yzg!fo~=WRbey8P(&)S-#ruX1huY@{vab76cz5Udc`#V;mxt z6`Nko9aJxhV^S5@ZbLjN^9&zY*$@+lA>XY593y>{(OfSqT)P4v??BmB8OeWHbQr%4 zus1^7Lwnk%Gl0T6+@MY|Dv#VyxT_cYL#VTIw6n6x;L4yjt!Hj^uVS4{z4=(QBivj# zNAeTo{`j{9_AGcpuAwJZ zVhh!8I^qW{j5?1$nNMUJvL{OAvEoo$xaLW^#{x8nB%Y*r70Eq-muAy?qx+k;xdXlOiyj>KK`!n%1WkR=PZL_L3O1iM6ie;K z3msdf6UHvsfOqH}2bjn!(yuUY^!jlPS zJ=D4Y~(-K3<@~%sX@+Fv%>U z8Ewkjk9pttrxt+2*3^|NQPthSw!2Z)?%u3Qs~W4oGO71 z{*Fvc`)payCDSq7<)w1>4o*aXc5co?KUQJ$#1Yvt>95I29fbjXn2~k`R#_i)Hx86u zJ4{nou^bTX%6}tNXwks5I+__fkzWyCClr;qfV(HFHrJ#yoY*xgCU&ol38XIldYR>3 z;&B_&KkS^yYR#O^`Hi7PP2!@=6noBKOZoGq8rQW^H^cUvVWP>}1JUGQ^n)pNrZ4Kq zbfc$}b|HA9)zbLF<%L6g$>hJ8nkYOrx}@JYu*O+B*M);xK5>OGug!oUEjZI6pQmZh zt=36P;GZ-LN*1SgK&PZ6-b@QBt?)w}M$W>El+8WKr1KVEJsrr&Il<#<9ji9)$vLR5 zn=XCw>!R*-hKN-M$Q6UCB6D_(BOPo-i;dC5qSio4jwKZN-Lx-`_e5?iDiUrJamQ^Btm zp?F&XL6CC8rt}_qoi=kxe(*A4b*2j%G4dMm5j|H+JyD}uT*RKrGA|7BLq*c>4lCT7 zBReg`0#iffFvpo$p`i*lkok-FjH92Ib)NGY$3UAf^Z}m4ftms@G1O8OyL-5l0UY=x z=u-{uzsS9FC`663#67;`Y#WTuP7HT~_aixE>10ddo0vB^q|!$dX3n`J5;K{#(2 zK8PZPxX=e)r1}W)MOE*-Lv$1lItT37P3W$><}Wh3$A~FD#*DI+Ar9MX-raprRVY_c zk3a0UCGC_roQ}S*a8h7sZ%k>yw;n1LDluaEWZPrPZX*>K(G9#$ZycSr?0%oe(GsjC zEaG7m5EWF|E`S0}35+T1$|u|B&6$3c1NMPHt3Gun)52MUyAaR=)9D9-pdQ z{_fmx^_+R0G4aaIx5}qexAGF!`aDPVr5k@0PuqQO_QSoru)EHHNGd332#A^q6-&;T zuV(2NlfPfPPsHRbKpL8lyYmp#6fs#+ZkbOxY&>A64F9!^i0eCRt|?g0y7cce6MGGx zCaDU?8*UTGn&R=>TMW}|c}`6zoN@?Ql`QVbj26Da-|i_edt%CwOsKXO%2C7Lk|A!6 z`rW7q8S{H+thDM=!HQTjvIVef&5g7@6=F+;2qpSW2-&E>C2ThX7dwbFX%t89VBj^P z6c6@E^R|4~Gv#@;nV*ci1;OSJ#?hc0Rwc&Z*HCjt(}ELa5=^IHc0+X$3{2w@s_gBm zoH4hqsUb|xSSB&)jl7in($_Hk>`ko19~*)#=50kcM3?8FyME#`r2rEh(uw)X59p)k3%7Cr1^g_(14A>P$Cpt#>-4N+YvbNZ z=J=E&SjJcEZjvrSs#IM37>7qXPiNc%+K1Z=N!+EBw#0tFdT|W{^`sMWl%qp8Med*m zLtfRf**cK4zqx)Xa*>(PuxQ{y{if2(#c%L38r~=apPTC9a!@W&Ym5Rpl5w+nwFg^Q z!f2t`$kSY(+J%!8tLU)IAx1#?mt4s-Y_#DC>adDQxxhb^ZRYUkI%U$-Hk7$8dyP zc2TTY5VfP>9hsYk!|_W9*B}GYb}+uE1b9$ z6{D=}3Sef#?`g*lBg(lcj|m?k31L;C^SHmBCtEnL%Hw|f*kQ7<%QhY4be$vvZ4gZI zqns~tRySeabe|%6)rxUI>Vos$6q)II%w(Ff70k|GUN6se{WBx$GI;xOE^bC@zO&WM zQCA;UhmA+J?DSi%<=g9(8p!v(%$nH#U7tUNv-X)cX5Da}ZeX2s`#IZR2#3w3vQmRx zJgq`Jpm>tkdHm2L<16s4b?L*zBd$|+Oh&90;j$xEQ{53hYh96Qi%&7*V5$D4R#?6} zNvy4mrul{H_i!dQ-|MjY3^bxN@ z|JS_L=3E~^oDgO)$z<)-Xue51*SxbA1*cd!-#>v<>pI2Vl_51>+TG;va&U(eT(I|k zKV+MG!J;?3<|OWk^Rt~~ODxI#Skp847!(|vZ~Im>4!YNj4u6p@oy!Yz{>+lj>lsec zqdK_vq0d#dd~U=g?_o)0fZfy47e3^KXDj)X5R2HalQAZDHgvD17AgVKfG?y?ALJo;-e`JvDqD`{IMmCo zZ0A#SWWONyRAZ-Q(hK}BaqpiJ)wdEk!ntz2lJEEuyQy&Ju{Yl1@Qd8b-&W$R_g|}6 zCaX8SA5iXg_TKYzF43{+V7d8ZVX*t4v6j3BT0GRZdxN|yKoK%8-(VRcJlJu{O> zrP29v4PCGNiFFy*vr%qgq!HKp>t~(I$-KG-b$x|Zm1SMcZ+b#hXFeAOw2$lktmKH!I3LL-c*4Ho$zp%TY@uwGD7o0TL&kfGOxshiOTzS!DbSIOx(%3+k zKZ`9Qc`k`P7`8#h)vDN}f*-Fk;w6H#HtuLoEPHwpDru33+|l@|c7c&NmucUZI`@tI zx_zsbtI?=&RHd6?eaFs2uu(sH*w8wR@ZJi^@ul5LTF~M?IcRJB2o@Hxxoz(Xl?@$N z8JgTNtxGumG-bWc&)`y!wjj_K(rmaq}ekTPnHTL;mvx+E&=s~@OZ)K%Xj6~|hn zkEgR1Y}=}@#-GRCu>%=>H`p66`#+vs*OdApw@qXGc~6#cxn<8m8R9<(-8 zkX}Z}C8Gnju#e`%kp)k*??P8c^bk*D4MaApK7(?abUuk%v*UF=0aDHf?%iFHZ! zD%AmrJI4G0O-1d>izP;^27>(7uIQa%MDT*xiOYQ0&q__c++WMpTz0y-m83z7%fH}= z#vm{;Ir5xh9Cb14-$2n#uvS;#hpBc7o}BS}$u}aF-DwAl&I`CX>JMi;r0Q?^HCe-Q zJ3WFsXT#;|KHKZwI#@oGL{|2%FRwl!H6qr!15B=}y!PB?|)4la%{|7Lm1S^1S z{pi%B{d4I4YtuZ!6?}XRk0H+A!IFeieqaL1Js%(L-&5}2vkBYsoV=6q{yVrA7BLE! z_lVxa`>*@@s4mPlW>W4IlmGc%Ys=#S}G@g*

{$Kdx|9JL& zK&R)Cx0LbyyXV9rE(9}~cJ$8dufGp!CX;}6!*Wn)(Es?X|M!pHX(SAK<-vyFcPgl$ z2gf|C7+j>E`fS{7uVd6pK3b@q?Vy(bPBLtf9sMtdNBF%rQ1j`(%MY5bU&13h9%48i zV09Q_5j&z`I%sp*nD@qU_I%45HtE~*o)@e=?6T956b8y>GN3jNnu(?)Oeze0qZ!u! zKVPkh`(X)SGyXxu9bY||E^_n8>fI`R@Gms#kK!iK-`_;ZI+c+LxRk@Urnnl*dbkB< z*i9Az*da7Rv*$+} zFtZ4=zD}+>C<9x`#aabP1G_tAGF--^LcOqC{$%BpL(5q(;x* zAfT4|`_c1E+kVIuwEG2G(|uYfWkxHYveQLw1>{8?)-q$XCQFSk4|B)dfi`a?7-bXL zZXyin^d?{%X1+C5QHGLMwm1Uj0M?*!mE~F3vAUydljkiCHOSznWFzfS-kBXFZq!hMg2>OtiTrZbFSpH)IpNgk|rtnyey~%a< z3MLe6g>r@iu^Zl&=r|21O+sx9g3{2|U`kA4+Zu2iaZ`|Fi59B?T@N?rRe8a)*6o@V z>OWfX;F|*`;B!~dcijSwByLP`ptwtM0Sc@wMw;P3#qNX9WCH+?Sb}lT++m(S0BzGr z;FoEDqJ4OxkP3_sP^MPr^SR3xK&~0-g?6(v3s-*DWrO8nL+^T>4SiKyFcQ@(l;itF zOaU6ER4%Kjf~Lg&Yl`sS+qFBTqfhaYA@IBz(3L(0+FNnx{SdOdP;JwM;)e6Ck*~sV z`HF4GH#L0?Z#x|HOif|@7jlAUM#F|WLEK@1cZz`uL*r@|t_=|X^=o{PY4zH=3n;R_ z1MY;~y-&Ji?9Go|Jum;q*kH`|b_yvH_AkNbV1ktLWX+}dw!(MKi0?})^%25SdHI|g_ zNJc}dmkx)XLRY8hj2x;WBXL|7aSqf#bv0f#j*}5t?xF~KPe|~U-281|^Ptzg#^KB5 z{;Ml$3#{V`6sP^!Oh^b`zH)Y0vJC*#lp6Q@J9FC1PRsG{wOY=|qWH(CLfmaiTQ8a| zFtJrAgrJ%{1x35l?+3k%u$9dyVj;b>b*r@u!UItgBQlNZye!!XNh%$C{00yBG@NuXK z7zTE#H|B1b{Hm}pC*ag}o|FjF%|AezD&CxZPQx`I>!6eCUpu%svdVv`(@}s}PJztA zhGjhjgBDxs$7kq0F}coy~?1=y7)3!;m1&pwG4TB zKFHZ%(7mU%UOu_&T^cOO>6s!QZfQb`x@74?aKadNb6Ja3VC=ae8t}3>jS=Jg<$P<6 zbP4bIWRLSs7`p3(LBLs2a(5K735#~+;dlMSgmnzv?vdB;-zo}l>0yS8ycuOdPyL?VaNDdLR>n7E^9vNtP_g=J}f`u zE*+gd-KQUg!k zukIKmF!ptmAUPc6%XLW`vS(qQ^4kNRr=PfJxU^&(bYh(n$W;{O4mJlP79bcZgTKz@9Lt`=1Y$`r$u~a zI@|ImMYUs6P-%-AVhrgrFPGN?HtNth#&lJrCbZ{G6~k&?f%awY#Zlw$Uu+&P)7D6w zMt7=!@$J>muiG(MxXN9d>Wh@n=lLqRpZi19{p%~GT-dDnw{Di>qxRILgD{d2(|mZ6 z=Wu(XG!lESLj3NlKeTz zi+u(KBB3GJ8A$!Wqb8aE(dP-`qzLiJ9GodG9Mk;KrI)~5R{mO=z7&G3dOz8x2GvJT zM47g>5UUJc+7C0L@`U~yDi$li$@1cvzA=puKyqDGAaw7r#6}ht+@DWnaLRDK8&InM z9X%gfTe-ueHR%V=c-#+f$Dty5K=n9Q%PpTlzfgY&q%hm^V`X&9*yvAngz;IyRyF}9 z_kCJK@O;Y4OC$V1=d@sj)cm?teN6WXs{vaQt9*DWYorl!2kgJA1p{3JY8SRd_YXd` zT`2=?T=12vaL?Z@jbjb||K?jpKxrF00w!19HSFm-?fBrb?yeX^JU5Y5#dnYW4RWdB zJ=tC+<#wfEJVfDl(+^&O@M)LUkvnJxlPl;kfPX8vxLjwONwZ%ty@YM%LzXhBwX*tp zRg)iz`Jdj3T22SauQ^nGBXG3 z)P-H^s=Epz($X&IDD>VAoitQ;janL_i1LVZtDFTQ`R?kukcXl7G2Pb3i)|iaQBvJc z%D-Svw50|KrJxgesXmF5B3+7>rbPqTV+o@c@WleJLUjJlkK3Pei8L@XNi0>finiBs zdyo-+`FLW9OTdIfPnYZzB^SPvGmun5iQ!%DVe_qN#=YJC&)6VTd z*cRuC?VZd6F|L78zN}g0s!lrrR^wvXWoAsf(UuVMjkQX?^=-|SkLGm=1y5cUjhv#6 zxI>7L3h5ByR_gisBMaV>kTB8{osI#>j*8(s+5Dcj@QQ+s`<-DQh&EDtpe*vk8+Fi? zU4&ndOY`6!2y(p6pu_gZl?#SNEUCcrgJ2r}9)S1@f5MsuI9AOQQ`NCd#HGqf10p7G zJJ|(HQ-)=(_LHb*BDjj(rf{OnFQS;-OSxZ+z}pL~HIJCo)hj6+A02?OY_KqbSHu;V ze)SPUm}JYDlGF{>!IbjmgSXQQE|BZus+v{Z(JW&yp_eNfcAs<`C z5KpVN5Wve4`Wi%_XiVCAXckb0jj4)!6fBV~{y)e*SHui~i!H0)W|Hl>J-2fYIJj_% zG|RGtY*f+>x$ekiPlIIZ%1IGT88aM?n@GSL7B2__9lOUD5-L(X06#@tPR?$Wgi$2B ztTTFs#97AeHn79tMVBCNIKLGHvEBlhMw<*uM@mZdyIe|RHRTB3JN+ahYBp*R&+I?T zLwiK8s=jIE#cgmCm=9#`qtFr?lez8vTP5tO`SVz*h34gf-&L zzskhJg;Iz`#=vZS>Zdm7Q<@Ap`T7fZC`!3vHLZ7JcZy1}zp@`3fQ@}bA??X3~8wIQ8EZMv65GL^^XItBLcp!NeV zGP2}XP89`|L&>?icpzin%*%t{u`pL14RAtY#?DF*pX7S>M{WNMHpGR(A{g%%%?p-r zsovzQI*H=<+zSqiA&%h#1(l!=@7-jVt;57aK;MKoM>8EEo6Oe{$JJSPZJt*#;6(0e zaUZ?*Y*>D!m6PqwtFg9HA6`Dp_ev~m81IxrzgT`Py*wn9W*)EL04UAMB;mY^Gzp6+ zOMd9cN8C;GDV2*`R5v!#9)KVIGt<%Ez{y#OQus`D;J08GUSq|XfHYRgQFypft<(`w zBj73LAQOjlF`YSZWu;o_K;d>#M{C4O?`E7;NsCndZE(+AoJ0L`uXVz!_ruTShVM;B zp<{Y0F$@$czoZcGNDF=N@?l2>sNgRCuUfi2y6#7+5B|(oiLsaHbK>L8#8OZfbs258 z9wC|dVP6bK`+IMB(VwcMsP*D)l)Wjwr{$XWzwl%&1-y+kxwVTWpOO9ZEABrcK+oor zjo)@UiRdYD7cf3QTvt! zM@$q9-BU=n_ZjPtH?}<395#O|n%GJB>VEYoc`JT5^#J@6viCc(+OA%7-)ZyyD^2f7 zS>x%_mibNZ*^-0;yB5=U8ZLgj2j6Efp6LD^0ZO>@9+vk|t5!!njZY!-xK-)if!dUR z!Mw%t$U_kx>#5c=`i48-Cww)v_;`lpj`B4|aSXOS<;bFmvM%jky;Fdk3x1!;xzqo5 z_0|*>EZ-C}jo5;e?ekEF1y|3BH$u-!v}E%%ez9xc(rMwuJgS~zaog)rb(ms>Zw)?< zueF@*37K6@Q=K!V<*}Z`a9mVzy8EvM{9h(d+E@53fqImj8ykkhDDe8YzB0dEUSN%D zvtw3~AIV$_*C{vuaJe;1=5cn{p--n^JxbY&hy0L5+o-}=Ynu<1tZS#w=qOZ3LdhSHK)AG zLaflT@A6+18Fyy*E`Alm_^NSuYXxH~?jXjumt#Dx7V2AZ?oG=QFL0r^smzyLo;=Iq zO4s_UQ4$`+wIrLkEPJ!q8-$33(&U>jgvxvNsY=XI*8k8w@eJ;r7O*cP8Se#jK=Vm& zD~`>C#C2ne$m8H%LMTlS{L`wb7mpn^dBD7N8cy-wuBle+C%0VT;ZQy9+YUql*;?QG zJ@w2l8X=R$=S3CnIyX*|ZpLdAXQ^5Vk^V@mvkjHiDsT7i*DzxX(X}kTn`pV>0cp9F zG%|Hpo@;LPIn4l^tX7i{dcvuqxoC_ZE+cNsJhL>5 z-&OKi$i(EnQu3mUWkMc@(lI!b#O;f57kTeli(|{B4|Om3pSK@-!h8}NO~hO`bM8;J z%O5`)k%F=_H{~n9-k$UX4I}yz{z@7BAF&5C*}?8DxivUGxMJW`+Cik7xOB6Yyuor= z`r`MAai`?b4%>Su*Um{?_wI}iH4SHn9PHqSY|t3nxz52WH=qE?QyLQdU)TFTf6%5z z%WR>}HJwlqe>P&=ZoQq!r}=A#iW7N$7aC!;s|OwLk;hL=K1xCusX6vA;cQkqTh_76 z{)U7cBH{x|J?)wDAwU>EG{+Y=A&-ME6?d^S!EoI*9)IO1#9!i60 zS?qoiKCytadxgYwlA?RJH}-@(x&oEo~>S3s5iA8ckHLU^~Cu}bv%ZD z)!#0kQ(WXh#zq#T#0{1k`Av{Q8(QHD~I_WX%W+X4~Mny5e4pA^R%r;zkHqcOo&s$>zXaCh+ zK@F|~JEmK4dwe;~KwUD;jjq!0Qe~b#QfA2IlrH1!Pe1Ro znW>e#NN{aO*24o<3hn!;$Fy9x=4MwC>c`4v9fQ2E0{`MAtV>Vu&^Jp#Q(TGuP)k@C ze~IFSVWy_vJqOXPQZ<7rsHNHuWTnh6JdxuCjyz)$hnvoX?yo;!4lCU^zLQkK83B3! z;)#$6&cG)mQQ_-pS#hZgabb-A%WEIs8|J4cw|Ru!y_I+Vt7Rpy$yH=)NG_p1b5d|& zYwx~JTtvi6!!`fFCy{4+A)|yyC1{W%HBNT|lSn3!x0|H?*~chugMFCxH>R@~$r|>| zLXR1UT7!-$agfa4Kf(SB@17q$geDO`>>Wj4Mb`7uZhy8r{0%4cu@N_Tp|;KICcnXB z=O;ij;iwU%bllwf%bVp>ux=B}JrY+wwM%!opxPwgS&0-cLg9%*&BBkWg*uFDvAv0n z1K8hX!z~jy3-1LAv!jFheheU;Fl(}Jk)jb!1YwL7K~V~jrCm|X+Iv6S9N)^qObH3l z{(O1Y3C{y=Kv|9bidNf463{95(tkVV3w*Z3;Dak2i&l9KFcD_t%+?`fR(p2I+O5yW zrzsTm6#WtHAFqfLY$tEdMJi2B4h(z!J}6J3)uT4pq-1asDac-J0+UR5(89D_Bg=ms z+!xQ#pM`28UcA+gj#8-q%_qLuP3$(=!SUHsU1J$bT8q%zN9;!UjEXO`?<*|V>GWNI z9H?b%p)3o3oKF~rPulkjkbaY8&L#ZH?_3H6mKqDKKv?zLY`rV;HGQHHimT`jC^-x6 z-~B+Lx&;MCdb!|BgmN}4`_)|*Myyq86ojhvWSWf^C87$~8sbi8AMOg4pzdx|2Y|}? z@pLd&&>bOgw)90Z9RVtMV+gp#OOzJ~;3?s*c!EYs%NdQ^uE-*#5f}nKF|Yn}VBEB* zixPeH$EWZg2NX`TU{bcqH*EDIQ&g3oZfw`1k#(2e{!4y`!!9_Fq$rtpKEYW|6&0}tw|gVZ=?QzcpjTd-FlZv zz%nxdw2U%WK~!M^?;GixqqB%(qeFWIrWlbqx_rp(vbeDi+ zsm$1%i5>mreHNu0FQ9)=aG9%AzWG%LZqYg~frs1{is7o%2YMqWJhM)Q(Ocy$+U2H% zvw|jllp7~Fo9;7Y#1h~&tqP(K8e=G%6 z0{g5<;86PZ<}Ey1{B6)i>#e_lVO4D*t9T%9)Qfb?Km)D}MFvClqqKqeL2Xi*o=y@F|>fCNX zpnW+14fL}{K*8uXvigN#IcNX#WZ)OJ_l(PQfcC_)nL8F0d5JmEgOcu^AZ>(P0Yerg zKzs}sn6*1Cpz}CM;EO8D20R*FX#u6pnw=5Y*&fwzz=^BfaX7zYI{e6*oq<0wP~rRg zp+Yga%w%4>w2Q({Ip@RJfv{yqRPW7U2XqZ>su`w+itF}20=Y%jx=_qk=UdQX%`&>W zOVBEFc=W=`HR+PfiG}DNQO3JjjP^+w-JsOs(cnliv^}kD@?6!SYVBe8tcq37?}{;Y zrt*X0gE5WRuD!?Ut@-k&U$@;J$xHvV{u*{4tl{^;O`!N{UFT+^waW`c+TglIPDu{i z)06M|lc=T(f61Wxwm~*aS`rk9eD}fr;eBB+4t0d5v55Ap4yJ#2Z3*i2tgR%EzhTeV zf(~M>zpHrK0FLpuyDzU!7iV5swN9s?JlCw}0BzCm>6>VnDC^ntW|M~mg9}~oi~>N+ zZX<{^AIVt|H-Tt@N{)ml7z(B&4|DFI(9UXAyS^Pe7Fu%O;!XH;O@qE6`<+opch(h#FY$X7L-TyI^1(e(D9?OSScW zxab;*Rx>!4xfy)o7RcK(KM(pPlYXSStG<#A$zp@h7we?K zqL|hDpf~HjcE_;o^bx3uiyL|=mlN{kgY8VHTu3`blpFATT#C}p*JCa%)$EVxC7z-{ z^lF3@YF7o_>=>Y|osIV6w_Yi_?!OhP`IbZMV2_St2rsT=`c6*7p3bjfma$|yNrX+SUKqAN zNFB!BYZu1XbVCRBqfv&hZcme$BoL0f@_1lRJqGa}#~Yl;NL2OyWb+3tdlIADsc{{A ztvvjW`sF<3Sy3tY4=iuL|0xa= z^*xH-JhKJ7TDZ^Cq)PF~=J-gFCOm$+eWKYWta7JicH_IcdA>T5;>)gy{4yl(L5CCx z%6}^uual(KIP?*;b-~!l{rZqwn*?Tbrr+PA5DwLscz6V9f`NtnWtgFrtf>)Z-FK)U zoYO_JBQ}K3!bLn1MN=cNnS0C8#Nh_Q99v5$hvsRTW$*dZ+ZrS(obzql) zmXpaT8Rg)EZ`>2d^hEs|zpq~WcxejYv9=#J_Iw()Dv=?jsQvPfS0}cPLZeS8UQk&e zqR#v%o-$5|=kdpPi|X06RzcEcocah&hB@51D+{(z&CSnBeLn{3>LU}s(An|Oo8>(N zyQUiB>tcw1!j9MD1rVyOL2ne^jJDC!`_Zc@M?=#bBTekiXJ!}eLx@?L6#6&ndKNX! zpdmbJ{s}*iM>LXJejS z)xL;#B~LyT?&7sS(CFMt51r%HJ&d>Q^R1EWc~^1>TAZbUgBcPd$BiEbf7#s;UlPse zs8#>tp;1#8Q6bMC*U5)flFH{mLsmg6I`Ne15!+jvXg+7LiwU)by`V}a;?I-FsGBAb z@0ikOoXRoG$eoY_>iiu6IVI-%HUX8v;94OpcQ0i@Jewk2a!tPl>+U**!Iefq88n(4 zjXVNa%wf~N$2x;E>}BO>nG_2Y94C+rJg+-gCXto1OYi+Au+J=)ppMQT?$N zDO23@K;O6Rz-rK*^yf$0j}P7ohU$nmcA*ZPm>UnJ$b1Q{oaj~Z)q4E~3~Ao79%yta zJ@z{JQEj(~7fu4%ecph(C!P9O34F^4dyGJ%<`a?yx)=Obgjl;*ZyW4{a^3>nb?XS6 zH1s6h9-hvTdmiU^yB7Qunj9YwOGU)oIYLp~b^KI6qYmyRJ9Ip^?}kz1zz&V7h^3#i zd@;%6+^mY_Wq*Lvl1u7IjGl*G>Q_SJyL?LoCpa|EI??C)BY)u+>u0&`8((Z)31|x4 z{AtDi_(=tiuU1!=wR2!?V42hNmV5pM9rvR6=vF3xma?PMRHqabJTf?o1DIc|6!wfT z0gI6LA3iyGtnh^7Kn34ozI|@C`+k`((o_?xvo5gp!wFYoD*M0g3#S-NzRm1&gO)ZM zmlCfcLJhF$WJMoYpAGQWZHXNmyuB%=TQdth5^>RsjiX;r@|o>V_r8u}dILi7Lo~_k zA!q+=%&`&!F`k&mp?G-3y8L{1dFO9b6$y1>Kfk+EV$h-^ilIs#X4xVZKA_I&yYSjD z@hrRqmH1>D#LCGvLk+%{V!2CveJ80o{OXBiw7l4ROgH=T?enC92Ft0hz9Zo#o#$ltWO+m{#;V}hy*-CY zYmv8adg}7qbCd+;sLCk%NwGERedl)CrF2p=Nx3IUhVsRwf(?jRo1``U9qxDLnSR^)c6g!jzh>16!`iH}I4U=RFvtiw7LrMd z+%ajKg_Ts~t4=lGV%OJcP`jBWRy*%vWc4I)J8v&@Dn`+RRUdJ~;P5wPJ}2uc7Ck@P zLG>eHL2$dVlbWY{RvobokVolokf^=L{4Uu-82*yq_9c+g_0=Kw47o?gVt+WGu`}1$ z_!tQM+~`Ow>GZWntmow?(2CrQICC~VeRPjQ>h8hy^YCpTII{_nSi{mUf3922ITWk@ zgW|EDS5{1Jc!6=sPjoB%tO+Z&?*#3@cke7L zEG^=qg7JZr@G}}t(+J$V5BSr8OH8dPNlzjjDkAftyM^x+I02V<-hO1}n;5NRL3f=1 zzMEL}dx`BFIMf;CWDBRjf6s1}eCo@PAeH8cnMrfboKk{yFbS$6p;WI&_rXJvv zpmRtlJ_-wApeuik4VQoQJB5(^6y({C3p-}?>^skzpXg?GhO976MS$zIojWGpv_apr zQ>DNq-cn}oN8yh@-n`^X+$S(3<1Ghz*z{q;4a=IJxOrD(B3XRZ&jlG?n)W=MtFTaa zbPIC_iXDhbM};Q=Z14m7om=%@LG8ahanGzWguGsnI-Rhe@NfO&5m9v+;rHlH6CPCe z?BWj-Eh`0EM1pj%YSk%|2rKr_n2=M+4Xuk7F%{&g{ND$9uxcB15skmf$5}g`-NP{{ z9i`FP0kPcqOOwfCDQ%=0*@ClD;ljW|`cT&w>vNTjPv!`H-g%V+h8b>jUz10cDu?{` zNTsh7YYAO}P=5H@5+3)X55G!e48JI7aMC%`^K*orsf4z(p1D;os~j*Uf~V^v&=2Ei za6Fr)J1j8vMa~XC{Mqqr#qQ84r@llRRd8RV8 zGoSpeM~W*d{ZXshc;ZvD7B`rujsu5XX2TmkYk@l*D3mZTV4M~nEyv7cJ&HZp@K>AY zv*?Zqf4de#13&gUWw*w z)hG?wKc-r9%xW{|-Re1CsxrJSwb`Ek5?`F$q4QZ3Cy36^$rIPGYL`;&;^op4_(72N zPIxT5qLfD^!{?|H^#qkM@JZ3q`$EmrcbXGw>6l)FA@d|^I^cM)MVsC4U((tzBOhRd zm}{IQe0jMy#7D~8+D&TXTDQ;FKkxQ^ZwRw$MMaznV+IyZRS*y@@sCPdN(qs0QGortHT|qd2k~9^xAC_^t;+9yrdI;Qd3?H4t96r_G1c)@ z=3S4`&%A^|&TQGdw03-v!T8!Dxa5t?AA9KkEgTYaqXSH*J$0GiRvo+nl}P;Y+QX*p zE5gIB16iJ>x`$xDi>KgRSAh837p9$e6&w@g!uYBueK2j7wEw(>i`*+Zmr+#r6 z54wQDWt8l>V`8u7w?WmCpWAJl-iF7OMCLKyZZbzt8$?A{*S08Z2>)g^rNdyWU~a0QlsJ>%cJ0S9qVE%u&i_FqEwDks_lowSfG6{*jsGM>X3dhg3K;~&W@nw?|NAqO| z-K=xg2VK6>5u~b=}@yZj=bBJ==-PZTE{5wnprdDrG>d{_fGd-Cof$ zY@ZuvpZK@UM5ItmHCf>PWCby5D?jLUfAz0#Qmrz|wUVlfBvv4uuX1uu8@Ap-w9afj z&H4QoPyC-h!t`(2u8j(DW?i!m*ZkpNJ(5t1#8#O6XH58Wd;GVhw@>H&NWvD_Ho$Sm znN)<-`*;0MardV~`z2lf+kg2Ve~&}v%{Y90CwWiMb;s;iSG3*6_bp4_5;@r__qHz8 zm3b=}oQwmR`rR^tVP_KkdL)wra z0#Eyt)`S=s4 z`|Wu%#sA~;B;R|LHj~AivCQq4aTH`H2)5uwUXI(}EyaKA9&a5w0x@j3RLGQWL15uz z_0yP*&(%(jzta?tN>|}}l1mPL#>YB`tmv2_P5UIwR^TwGd^@_HVWL(fW0p^Y^HiZh zXEMVq>hIPS7`c6N?~syivI-oRR)4V6se$P4kefRJ0-&!me zBIC?c_kCTTx)wYqKgl))(j^w6G2*+SxdD8>^;O>&lED;du)QK>w7`B;Y}8E;Ml9x& zuxr3`o+<49_1+k)#Q=OP?MnYzZ<(!0d(ZPU6F+K=b9*pKe8w&-f1En@ zNFPb4+?eQ7$Hg@Nv64R69BVQ!_FZl(Rwx?@-o#3c(QskL81S|br~-L7Cz#=mqAxaP zyo>&UNue0%$Zp3W0Fc>VVQ2m`o+?aHJ}*f0j33Efx&fdXD1^_DDbB(j1Y*7^c0Amg zG%G=ZdC|)~Y5sD(+3G^EJjPEWcuYVxC*I2J2L;<^M^J;(-%J7p^WRY?k-GpI_TCl+ zuW1ta2CkZwH(KH@l)qR>!DFNa&JRvbXPiLqHFgb?NR77{MW!SD*G1rl1tAw{_Rr6UbpOZa?>8s^AqlHdbJoWfm?qSQs47Tl_}fW_VeORkkMIEv6``W{a|aC?f33-7>!16q(?hr zYFJ-oy%bsUk|L}K3?|5zty0wz?aymmLDWxwiMb{e6R4Izz#xP10pJV;D*%z? z418JTXkRfKDyLR|9KhXSv50RrQetin8L6_=%6b4_W?((vW?9Fz3l!(6!kZ05d!W@* z`5wv$a$T>=A4o?_{5tv^Qi%`we(`MvK9ud!ghK zv$1mTD7hHm4rm7`yrInwE){>*(axf-2PW?DSqsSq;lbIA5X4sH1yTzc2)rCT>JC@6 zedPfP8j3nc0jM4csZi55kzWmj`l6AaLI)k?Vl(H{)Wfvr!=s zu80z_-k7W{1|PIc+!qhN;Ytl+Yi%0{Wck^rm0D;Y<;E#(&)PuwvV!Xj`?d1T{}t+TQA|gR>BeDr3ybo^7`@ORaU}#<#!*rv>R!g z$x*7^rXhSC`+T@hL?AuCXfUVH>1-Fr`gS3UheYZXY=f(JCT|Tf6n==)s{DDI22!b2LB*xhy<8@I?zFTA+|;WUy!BszA!65txWH5J>H>qe!ukjewb>|tfQmfiaq-4|%m#;hKc0t%2x*OO7 z%207$*rYOHfz4z(7|_7`ip5W{quiJtDmua+HYx#CC;+B-hC4u}pX}{e<{Q4q?+LN= zSF=f=J@Z>S2H2eq&8ts~IX@4S^pSh|-#gf!a4{74ish)GoV&k-!eDSn{YR zM<5q~|H&o@T77fT!#*1hr1|O}(*bqZf9;Nn7G!2}=>NF-Zh!-wvGWgWC z%EeH8;Vvac`n57Z@YK)2ffHseI5p zMhkV+3Dzo36sTdej6e-TYX|E(sAC9dQtL9h*W+7%yzS3kWufiC2K*To2kBbFbyWdm z4`dmqo_g#7sGlN%XS<>ITv=+* z;r_-ey$e#arbGE@%VCuZDgL8HHw(i6JxjZfg+I;u9OIIpGf?th&qf>Hv!Q=a&fSDn z0mKr)xs;!X9Ndg9bKFR|;Gw=T{*wy2dLh9JaZ{(}WtaC=YKpywDIN*y#ZsL&Y#g_p znrkHLxVMM&huxzi2{ZfSbyN z-<~@Xq;Wafv~Yo;1_AgznLK8NZ2-!d-inhI z=5hCw=g!f4Ys8N1&w?`vAAiygZc?jX(w&heZ=&CqhU@z!)8{;IMFUvnszEiPplcRP zern;vBiJzC9v`VHFGIt=g87vwJ9uQ__IxDQ5}8J;N-)Zd7(WXvy0cd=AO)glILVwN z?b~7311wxQNpZGkId1PId0e#`*zQ`zp5G56OUrkf_ezn`4D2*=c8m_+i zM}1;`?Ud4N9+O4*0Wju{#UXrY6@+)gUgC|Did2t-w#F&~&4WKhw3o`Q^EUhS?Drja zMRE|5_$~2=l|MNhUs4_o_FUZre&>B)F(_#uF9MZ5;2SeR5UsF3NA-=V=*q^oH??PZ z67-$Bu+iDa^vuZ;LOWSnu3GjXV#Y=cfm{%FzjRT>J8FhCg ziV+i!eu?YA4w$v33>mjJrV&+Cow0ii)>~jQ@sMd9cjV_M;@h2SVYTLBNN3C~j?yS` ziM#;gz!PFw1bw3Ye6;P>ccyeTnkRG!?|JGAr0nV`lat!5H4Fi@vO}EnH|9&Om>Zmr zNDIK$;dWfX!?FvDAf=L$ti$?Nrm;DenxAm(tDFTL6N827k^mcW{WA3vtSBKFC35GI z26Y7q2t+J<;A(%}*cUQqOm%;xv|Og#=Acut6A)4C=P_XBbI4QpEFf9mM7E?FZq--kV93 z$!<{hk#&)xRmS=OhA+1pUUR}n65$KLl6|IFYvU^Nf50Y%p*YieU9YmGzmPe(x-fi^ zotW3zmfhkTO}ojnGr}M6ZFvsbJY_l1B|u zFUDlOLBFEoI}MMe-)dT-gq>YGvih@DyyPKvkE9)^eMPFL>%Ki5pQI!%Rt79R`t6sI z$g&k_jjcoYA_twQlPQj1{pj{({o^gkIRjWlGOpPWqqWPZZD8w*0|kR7)9(+pqc3t4 zG6Ph-0)vdY)u;y)&7IVrwd=jjG$dqVbUd#f0pqcnWP@o};zx|A%ilUb$e|crQ@^)~O2FzyV^g8tDNgN#?*h8f#1SCHqoXx$$8Tv(cKRExfla?3vWAj| zk_TxdRC<_hEX*{&@EHS%+^g^T#(nW^qXYfyy|Btwywnq~gY;QVuH`bu+nt4}I$BFwnfE>99iv~{X5OKF zhYN4YOwPZZ+nTx{XuD$MsYYM;14swe|3%--&xntv6?`ac3z1_8JTt<4 zUf2Wf(1|3bMMwPTkvAVlmi2+>78KZXDM7SfKIuwvCO>MSngy;obd=*H} zELNrF;~DiVZ_~ZA&rZP|`t*o$K)GiB?zBF9Bmu0uJqBqC=pm7|R)TY!mBiy6pz^ny z=3jlCKw|7+-GTT{FMrI7Kw87hP<8?fdej++rP9N;BMbyxZq6WF%g}&L;j*8*#5ppNGGdL-^2k15=I4o z4rV`<(b-2#xFqD{f)=yvGsT%o)g3*x;zLI^OM|hvx>&kx1GA3L1uwBqZvif6`s-@R zB9M#F1`!;X%o;&J=!+r>K}V$*_C1dhpX)!pM}>1aPQr@OkG|%f8pUhI2VuU$1*YL4 z9X!RxaW}c(Z?HtksEn!O6TI1yb78j{g{Gp7+tOe0*nUUV-%XEEScD!_Z#3(;&B|63 zZ?~baVc@5yKiA52LY2z5ZxGH6TC9prW>@CB|0sDpg)~60IwjgKB1bPlEkQhb$w&T) zAlzwTQmN(RsjspLhmBRf5r zy-B)Db&Adv7e&9VS%xe8M!ak>-HdAQmagrnr7wGX{&+k$h}t)QIeru0XRlmw!&n82 zts?%`37rmO@442)oElmVDfK&&ZSDBBqLhd6@jp(Ya=*29*V9{wb-u@Ul{B6$!@eI4 zBNjuSYY8hZ+a`U{*AgC@K)racnw#}`y|H0GQ#G|vxV^*1uzz2!zX-^~qP?+QXLOLx zuoCKnX)pNRYU?Kw5{MQrCsgin3-u}|^yeu;!SMZVM5G;!60_tKL-3H8^Nq)uDx7?L zYG!@$KS>*uUU_HA5J4P7+A#eaX*1Vki)qpHdL+HaC7ANf)9BxOcWx?dgL37PDwQ?? z;*@usbMJUA;I01c3~{?b*V34fo4eIg1A4b53#Z>+?z1L8-Q@a`+pZhAihN39d+Xt} z+0uZ-EooX!g3p>Ifsdl#W6p^@jK-A&w)fc~MBzWb-0EbKz+@Ey>tfR4?SVt6P5yZ# zHzz%xpi(H;??~huGs6oMnuV_2+K~BDF2cI#zQ`SKJ`~p{A$?G|0xvRc;t?aqBKW`^ z;cncr9z%S-hy^ZZnRx1qniHlBrJ?wxj%+xsI(?NQvTk5$2F}&#nK*pPfeJc z-BgPBy^MlpkWdV5Y`(J9Y5@^F{dI(9c%9msL3(`Q?V-{L(j`^#V3%IHB&~h0_)!<` zTp+%4J-Ruw+{awXTlhY`haP%Wa-;9jQ=a{|>-)*n5+V3OXTt zsY>j{Z>OTNoL|bko1C~O&KK3+$I6}28dOKgwY13kyv#WkluGxZQt6+3U2=CNdcrNq zYY0Qqk%Z&7AFK-24wP-@lWBkV)L_q-(R)`w_;7iN=mDzojk<7LE~UU_3l4=$9mch$ zloZo{@($p44#Atq>$X)=j;=}f=8~F+eQWiT!LA%}*jhV?_Ch=v$f%J2u{&Nq61cQX z<}^BK>sQfj1LEud(SIMD1LLQ=>sQl?b-2;oP0xO;_Dw(075Bo(&00)e12%(KMXI)* zm(=%i3Gv_f6Z^H15LxDskn_6$!@U z_9w5*IC2!GCsHq|uuqqtUJxHh$Ez~@PB}>aIHPJUQG&;Wd zr*%u178YcWT#rfQGa{%nx47IIB429eA^O~=$8Ffs;U0T^$o7Z zx;XF-Lf3` z&{Y9RnM_~aZJoKRvJn@PZj{{gx9>tp=?<}C)bF(TO1 zUxBeoB|wC}maF^8Itj?LI)TD;A^&kRu-~VB5RT(jkw>IL8uH&?wAYCMHyFg8@a6AU zB|n%Wl+yE^G1Bt`6bJ_8I8{Ih(6T3yd@j)MmYzb!h`wy~k*dS_Sc|vU0A#PdyVO?z zjL*HM)gYG%%KplOxQ!SdV;vBRR1C5j90=@BT0moO2+SCJUvE{q0CYb)5LBAL&n+@G zTxhHtBy#jw7=WG7Se#Evn11eaALDL`*8w)-Ki*w7MCR-MCLhW&0e%|$X6eT* zp+>#$)*zo}2%z=!#iC%`+8*F?WK2c~SU@|T3nGNzS(54&wU>Cc>pbt+7yCTfpqUhz zY&5*h{&GXaU%3-V<$rQ6<6}wU9vk_;PjhjQSl()D_UQleSlCCq?supo5qavCjU-T- z)-aUA^aTz^vM&Sp;XRG;aF#OQ&aeX8WBKqBdqBvV4zMfW<|vK>jl32_4~^`B<5Ou4 zq>krM1r~e@!D|{0t9K#SMV3n5j|w^gnmLdf(~lJo{4C|rm@IFVLZEb)3QII%RFO}y z$ykv@rXoSRID!8fpgu|v&6H_5({#T8Afokv^RgPr*OzN6Y^Ju0tK6u=HoHCm+jInz z0BwQirQuZreMHIq#rmCQ6aHD0`S-?*s?N_t=DReyQzF*Uk zEqA*aZ`^RRhaI%LP8tVhHIPgWalh!7%$@cTdWPt;G}YWtNaOl9U$fNc5cp5@urgP! z9sltrp?c-pqOZ)y_;jttoO?}u*W$5NRMq!!^SRdqi zqWcWSoG5tnaihF*TkNO9{h6r?fg@%RWEP1+67#XIU~HfKEbb@2qJaI^fb?0dxri4~ zB-C15xi{R}NA!e5#&-xYG40lV$Y50vv<*b@$L}|WXBPn<2L*Y_e>A*~nyG)m5NKTm z>;5=oS~9j$d-=M}n>ua^ThE10ju`sc`)g*JcLJd?Xdw); z<;!09ftUHm`@3~qB8UQqEdYuq(0u}6ZY$KJ^cgS-$&3ocT|~8=*4jTY=VuUk92zzNhc-4gU;*T{F%^kr z8ov7(WcyU(TOgx3r4o!&lV{e~!lc>rq=Zs;#)@WWv?(=KO;9v4Q9fAk!-rODb-rf=hhsoE@iiw84mhLYMP1g8d;tXsib|9&-75d`2??%;d6~FdK41 zV&?7BL_k7}^o2Ma0pnT$z4W`%BMmTNu%ljd+$rV?dm6+D0Sk$kb~JWxUw_%(6*RcLHgMFD&FuQZ zhEiqmn2baK>j&NUhY)M&8*?ffOTe{8lsa_HNrACkv(%hi1uaoe?WBxW!`j@+lwTw#X$(La@)j~k z&&&JR-cb~ty|LSp_=vNZM$mQ&W>E5(Ld31sx}`wA2HxQxf!pNeY<(X6`{5!gJcm ze=51zv68HgP@};2K4NLw?HwuIxC=r%v>LHFZALj#IYsA7_E1F+hc2@0vB$MLKC8ZZ zFs&cD;9%^mQO>)4${iz-`JW?8jOa$pQPg=$5Zyez09b0RN3p!PYk?|Fa)f!DD*L_| zSkLkIv|KHG2iP^D$pY|oq9aMz<>6*ilQ{&)Z=}?iv8KMYWN+q2IUD#rrRyFskWI5b z&}a~BnI4NJrPQ_b#7@Y9lCP=t*9F(&SPkFg$wShUUqqjej;52Y!O_X#+ zak3W^J)=C*4r zqUL)bPi_jql}q1$cX!{rx)a}l98qhBUdNZvsz!wTD!}#zk7A#krreQT3vs}(#zjjv z?nKd+^Sg24wqILf`>z)g`k|YI7#ingI;LTKOZ-ISl)is1pMDE5)4^0NxhbsLlWiX4 z!}2jj$)x)g$s=Ci1l_Nzrtct-kM1hSph)=G?M^T}8^z%LO^#m!>03F~7xRwpEu>Uodw4g6p5_xUIKEL$&7q|HVEJ$^%9HWt4m(W- zSsmLOU(#%Tl~2|3hGg0Mu-4z3*!k9UzxTEq>;}UIsA(8YGjFm;9z*0{LJbfk zsTtzX75~ef1}>S`2Q+K{Juq(7&j;oSMjvAMGN9T z;5C9xKsk-4z)euw-hYcn;noz?Q>=Lk!f#{w2RtE_z|yqomY#y=Hnt8>)-ltxhzChm z<5_;-Xq~Nf2*t0U!mDQ>y+9RouYqg+ENv8;H3mh(l{~zs)}!V7vF-33M`By)2R!OpV;Q z>pC|EuFo9x_WOZLYYi@KT^^Z|>$i82Z1ij%H{EH55Km{?-dxl00Icfx`M61Eo1{jd6XeHhAaEIY=b1A6zXd+G;|rr7P#h z-)T9j3aoqHc~*&Wd+W~WW&AWI$znP3S``+V_T9!2Ee3 zSPVGQEC>9&sjRc00Xk{#+58E@PmlNQ-kVcg;|Vs>(g_et+uR4KRw|IyWX843tF)ho zkV7W8CnX?Ja+0+i`F*e>49XvtF>Tv|J{ z?>Ce+>>J-aJ@nvua15@BE%iN{&<%m~UjGiPI{ofwDao3tXJ3$E|bC1*)$%dOY-Es;le0a$A{pe+PA&gpW=h=#l+h?IjNUndOFLv&hp3k zo1>0-y4OcqLte{{m7vb*mp?dompjPW{CfP+neHI29|#`Uc|K}5isO;cwZ`j7uRm18 z3#>l@aFQC|k$Nd#3MyItIiBb9h6O1UaZbP_%AM=;We*ZU@;B+C%>AxDQip{|w0uv44JR7s;vGd zyp<^4yMxN}WrM0zG8Dm0Ux?*cEVDlP%`7 zNa&i~DeKzPt}ZHVyBKwFUK1XQ5}#wy8o_Ot;;fTW!Bi=yO_87&Cn5Q4ih z%!cCW?l7m|F@#6UdgmJ%fl%FE`$IZG?gtS~xT*PvPg1GaB`yU)kg{7Gool4%R5t9T zPj|E%Dce-}iQ&30;5;Q7F;GepD~u~I;jg=r+>TEOABp|BeLvp7;z(rNw%oc?TX0eP z#RqZ?T`3&;2fhz6t_WVJ8~#M^d(GymV0FACT1ec514q$qbz!5fBnwIl)Q@|zBtPM5 zS2U+tq*eWmwRy~Ekgot=+_g*uKuLrN{!Qz8JA3+(>IRW&R({y~_xZIOX4CDB%YDkbsOY1ai)@GGBhSx++NRWCx?IlIc`$ths_|aL~atzKB-yl9T~i zem#mHlVpN*MG_{%b5lyyW~g=tib42QLo)OKq?R$4xq@avQ$9b>@W|E{&b|>qV3KpA zHT4v#Y`}`fAjWKpNe*Ql46%rn{aTl-!q8IKF5a-8^R8jBcl2>2q~gG$H0>Tq8MwbLyyvLVpIVZ~E@+;@<9SM_6wB9B z1rRw~Ft>gfpl>Mn8z`U!N->bL>y{=R0{F*FzY|R*0A6PAT)=$Xa5O4|m!iF#Lz>cSX zEV4h>8e3X79ia^sL10G^qq|u{#SE~(9Ll+Nsz-xB02>hEC9WI43J!wfyfu_juFg+2 zkS2|s1=o58<9#5-AiHK29BzjVDb?DR3FCSP(zSO8tkVt-t)fT|fEE8ua!NGmU?=xB zNF;v=lqsvA>9lL4$AOfuAj>{%Qu`;_FNv}ABV%SB&V*A?0x%dAC@97e>HHEXD%B2b z1+_u(%#CVX7Pfbafl~0jy7!l%NR!{!lL~|E$6bKVkvjR@QbBNvMIPFGefISlQ5}#- zn1h(iLP(7b0weTswr-T;X;lB|7XJA^FfU+yQO;RQa1&67upXW*i|v(qmHK&ob34E-3)iRUo-rSZxN zVSw$}g3zFf3_GI_APo9elvL~QSN?u|5*F>!ymu~k;Wv#+^!HDgAXhCZj6?ZDqAp3! zV+lvXgh+ce0$QwnzssNR+(n1+#0%NkK3vqIyJH1pwQ-lte6CXCY=#Ghf#dA-6KJ#Z zf{lVf%XUB(jmf{}`RJz2-ZN!&eIP0gBpQlV!0vn@H4OpPKs*nrGiY1`e#o9=05ijO zo&NZ2tXcUjHDmbsyKAhkB)xI4%pLZI3|B!=b$)lMEaf(ba>mof_QRnP^9c(IR)AE3 zXQpn99fbFLNoBq$`oao|pN;uSfJ5a|!SLQzv6xlv_TM=!6#wIiNZKG|))!2BD7hXuvkXiIf-e;!T(z$7SdSiVZO;^c`gmSI^H{BIqh{ zABX=q`}Q$tA=_6$sE@L2Ts#nHho|nc<^!8RmcWY?XyN!~iJthL$`o)>a$iRQr?50;ktwqhP8_J=~=$?YQQcfV?K4s73Wn=6>Q0_$W-DSemb*zy9e#%Z*e5K`# z48thzx~JhdDBx!pJM*1D0ybMB#cS0rBM;c@l?!lN;A%qEWw)~MDdT^A{Kqr1fl+KS zG@Ww9ac`i6WKgD5AM;(y7MmML&o6W9!>QAEWxKY7XLHd5OtUW^BS{x-DZGv|`>4-% zt3gapRA1rCVQ8AavKc^G$HWXVS@3HO0Mr2q+yyYdw*-__13JJghHj5rE2g%myW7%N z4FaY;(?#i7Jw~Fdm~&nrt-r4xanU*goq4voHkcV==6c&h!JqgR;eOY3my^7{9L)(_ zxL@o&q2Cpu78CwtNx%fN37DW@3*gn}%uD4DVM^{q7?Qq%Yw9_I6JAr_o7xnG0UiYyOp!2Xl3^P99dqA zJ~YF6zCjIB#N}`F8gLp&#ipmCmm=pNBA%Wek{nDAh0$}W*6Y?3nWltC{XZFnZ zR;(}m9uUe8LLnKRh);VGVJKlxj*;`oD_!_wDW7FT_bn6oq7T=$b68@rv~yUvl>?jA zM&DiT&%^qVKC=Gd(~T^~I<`gq|MB4ddVGU#yA2n;PLEMqwB@>hkq=VThJj{$@>8o{ z_Eq{JXn6!HZ9Z73#-J7}^2WF={3*0Dk%ss!+93&8-`#SCj3<1h!T96AVPUQ4wX-tR z3$nrV?!NkAk&LyO@SZntAHe8BmO-_CZDVa(8_PM>;}m>mD@#EK`@n!R>~@(Z%Brk( zhSp}r_ykDLhtE$Z&Q(EL7%tvCh(@b0;)7zSB>1goE5Q_(4#oJl?a|!LkNj)^+Fdpc zs3W}nz|V3#`C#K80y${WptvXAK{&2d=KoEW_v;q^EU9fE;H!Lj|Kt#Jh zo|p*c56g1QcAq}zXz}wp9og{h?}M4MV@_x+ORgpS`^5Xyian5;2Ku69Czy=BT>?Za zW+BhY=yN|{{nm*fm5d&>Bd*($_|m+x4?plVfWaH&;TrgYK6uy7plS;we+>gnaG{Qt0X_Yx*#=pyz69gh zMm=7!*_dF(N`7k`0<4kPsCNHv_sNz4T2Q);0QIeUzoNWABV^2hCaMrU8iHZNA8x)2 zB*DU-BC8#toW7}Z6J-fdQ6YjzWNx!HNgnAYxh2SK2Q-CM3U4nR-7tMV{cvc+Q4Pk` zT*-i&<=1x0c6#KfTcI#CC}a(mnk&^6Xs5B+;dM_8v1a{<;0Zx&bT*ddKETO&J=YKe zLCYN{qu|n8+XM5}EG9k%1NKiB3+R<8PJ-ZD_k^Z>8K^F<-Tm}N0`3d~(^eo|HZuBh zoW%yn^()}+g!e){e~|JYBA3U^uzRDd#F45@f5&$}j>4J@d6(;bnKdD793N{lTS}=B ze@WzYN0CXZX;@wWFkZI&ahy6EperAPSNS@F`X(PDP{X|*3jolvUXg^8D5MNXOXW+g zvCv)c8G))@7S(+O4Fv-3i0LJ1LOB2F3DUQY>N0z$QVT8>y-tX|&#eIma)vuPzL9N0 z%P0neF$KoHB=L(N<1R#ZhS2A1Qvcn7pJ8TV(D3>UHQgy-f3eIZ&=N#ls;0;TJ_pcg zoPm`$XG$*v4+>SU|r=^Vt+HUPq@AM4ACVfse-a79zjYx z=|~evx80eq-x?>&gsuS%0BRsoz*%qldUu$gCPgdXDs}sC7Z9!F$&CV?3rD#fws#QO zBIbhY;Q7igHXZTSL`ds;e%;#xntUN`YoL4wvq@{oJno{%T2o4zbv*e@fd#B?z;hX7 zYx$H<_Xx8i-3Uerer#qpGLUj6nEDoPW?vt;&UwvTFDlWd=RU9P%Al{Jj_CN1Z)YxW zg+)R@oP5~4poQ64l{-^H%OAxma~sBuwQKWVbFBaC0s$5aW*!WD46JA>9HM~7&2ed- zlubgmV9{nXu}E3<9F*p7fF1hNT_fZrRe01=8%xxw*uY;$Ml?^B>bNev#H)8EexSRi zB5_LSehB<1wo^VH#_p}K+>9{Fwt6nQ_mJfd&;nsa<`d&lCAQFxEw8>u#7n_GeDNy3 zYhM_S`O}t1&F<58lEVq=2sAfk7?pVDEOB9v-V?M8aX;MUpG}8nUuO5TW~XXqVHhu~ zsQ~4IjPEPYvD6@uVN0n@15M`)w!4+*}3&Idr=~))3Rt zlAD&Kb zq54t_7vWC!C)mMPBUp(dvIBLDS%OyL%_HG(D<_)yFVa#{qYb)tKn~1?bHUWLv>w#=GCGIz3DxdDu-6i@$CQf{TB#)Om z=;c9uyuC4j>1OWxMiG?>DU^h8x!t-MMeuvrwJB8_t*2#+!3c7eA2T2a*qosMVsOVGAE0MpU8Zm9U$2W-U z^j>QVsK%$mW|jY_Oj~pMlq-KfmcP#RAAfG~U#O#E_$K2YvswHh!l`>v;8BA>-Ga=> z)?90R(z`CQ){W&`*!cFQ;#&mjiHh&c zNaHkrdG@t@qYbf7e?TvvuG>~EXEc*kVj*~__$YSaXezw*v@WiGGl1gCcZTOAX#3dTk*ZinV_C)@ZQA>#wG z9F=QwgN*82Q-xqWP%$dO!P{SIR<1ELJoW+k$L|?dPlYu&s1v@7!<8FM;dYzIc(8cV z^?YJkd1`Ry!uWL-rU@eTY9%RE_*dX@qbYdcJC+efz+J%mhphAh1FdC=tNP>kz`qqx z_==q|)gX%@4DJ+gqHK3KtqEIy-`~(VpUz(UqvQQ+W$@?s-#jp)4EMlnvQCe?is660 z(Ds^JiqLY6VD{g?@xLiL#FoHH^Qq&(t^a!2|NM{t^^I0KL2zkBZ`X3ozddt*``2JW z5b=>w@btmoUZFn^>(@7f#^5up@$3lZ{?o(wC=ax0OPOK+6YcodhyLR>{_`ic6yP%n z;Z|P#qka3^Pyfd@lmV%=1Xg`gf8XU$H5=VD)qnFMT%nzHf)p&!zG|j{DCO@cWy8 zB0~Egr{*yNs*B(QFAQ)00XzJ0hW^Lz{rSV-Ac$!MtkJW-+sgiR&VE5OaA@yKc%8-l z3xWK%fBw(Ef<_R)lQaJ5U;Y=x(qB&-F%3wm!R{>TkN!KU$zOjPd_SuT#oBB(D)#-8 z>jzHl6+CbWy-+WfAO6D^fJAPc!_u`cS2(=&*{rU!UbcEpyh==ph)XGx{g5 zNS|xp`aHJG#$#0@<@2*!FX^B3I93N)#}}D+^nK3#O2YC&FFip)d%8+doHWTJ2b74{ zSYPOYFxBE|U>Hf_?#=k?iPQ6`0erQgHL9Zsj4&MrnF|T=^!f3IpiWK-Q~94o!GI_YvGR$2<*9B@+VoUxe8KC#37$B0R@0Lu`gi#?RVTMa%40xjK42I)KDAn`n@lMQiQ8~BVtvk*^xJ0OP9jtGZYm#vpN0Y2?`@|owO+4St^Rw+L_+a{dxZT;Vc)|k} zxhgTOr$#K8Ltk=6XlTCDPI)%+Dqb{7ztQ2;Ed6BS;r>;kVwC4D73TyZ?NwQ$@Yvgr zFEo1|Y37eZkzF3=Dik5Bu7+oYPq>w(P5d?Ntyc(R^NpN#=2tb?tx!25>M|P4lb~hI z{r?bVp=LV>}c=|i*M zKa5oY@#heTsL;-OQIxN!AlwJU5nu1v|AO7FaQ$Bq;1)o8r<&mLbn3xG4K$S&N)p5c z*?LaOGjly4?&BDAFaMbU{=+u1W&QaMH=|b-Ku{ro-MjT1+;D2?mcV;IR-}t&SO~?L zLUL9a6QFAiS_L(IiPXNgxnkE%{)HruI@5QM0+V?gYM4sMJ4B96l=-f@*Ep{h4U{iK zE@V4V2)-p~9{8C92X1s?Omqm%s7^i!%#Z7hIO8ad%XVu4P#<#_256aIU3Wf6e*JTJ z>!l+#yN2((XT~AwRyNYhq;Fl;0+v*-l3U$;Z(6aY^RnXl^j?u2b2V2Z`8G1gPw*_|zoDF+q){0b= zZM~@G9a}rKa&CJoqbHx=^RZ=(cLN!7|EX#78b69Ar{T3vTe1CD-l3xkilOWZV#}1Z zA@g&6Wheet4PnFM<@;}*?q2M8w5A^Tg$z%;!vxRXE|E_!kTpxOw^i9f79%GVK5j(ZE6} zT{}R{3;^cMEqH`^mCDHJXeSQ;p<0Op)>SK_A;WGpYfswvMbGU2R5k#X^43V2%wZ~-;C?Ntx)4zi93RXdlPoNLxvNmhj&jGPU z(U+0HUw6wpvcVI6Y!1cs<+MEQ5T`XO@Q>%Psh{ zn_WY&ASPZ>GsWR_KJHUz7GB`sr5`CBZR&~_F%?^aH@tx@&&e&HJ!`!Gai zm9Uwg@X@M7Y&oJ*_+VF*?bLj`|HTJXx=P6SOjcuSubPUUZ39ap*KN2+_d7VxIKI|^ z=Mq`x2BK=Upp1MMCrwnlhDQ>HN0Cs)ex5(5AzI>1WINvZh< zsmBVqV;5aiwv|5#QK)-=Vwb|)7~YR7Vw z3aEWewi*1HX{G*WP`k|&GSIKCN$^!rN4vPswZ+_JF%zu^Pw5KgF@EjMVeSa=_di08 z7@j`GTgRfznV?sJlG@LJP+A2V`U?=6KrB=n+yMfdik!fl@c_4bc_veJUh=fn0KTNR z-FCb$RS2Ld?-K6d#T0+Gd@%dD{B=K|j+xqgh*oMYN$Fr1q!>JAUuwnH?P6ZV{R3Dh zmO(e7E=#kM+Up>dDHk*Z4~`m>1%L+ygE%N2{CgNX15dCy21f6zp-l5{FH2tD!}+RI zUP?{9W#A-w`=hCZ3|%8`UI3}K;{Ucf;d#r<0vd|JA6aG?PAWR$dWeZ_=sU`}g7; zKKE4>9~NWU);XuGgzj^D)*|LK`HdppR}==G+P#)!Ar%b5yUEu2)UI0l@Th%nDgOuky0a3I`zOJV$C+p6kca)4do3ByGN1Xf*QJz9Na~#3 z)nv;}%=xVAqW*HgTmGf-dO#44Zd~jrDtpvuTOtQBI;0`0gsL+AI@A=q{IbugakV_y zXPRKkPBP7Z(IJUQ9*W65+`@OqDfHFq3#d%oesn=WL(V`i^7E7@VbrR(mHC{L-Y zz&nq9jno!$>vyJu$h$Vw65HX*%-WH>Lo6z3=y?wf2L zBe+OZ@fhn;9ya#Nyo(i1t7~hH=e7)=ncwM<^4ik6`?_nTY$Z-?#xIZR?~hkDW=#Nd zb%l_SwZXhlYh}T9+jd+dX^{A!%?o>!4^X!HU66C0&;XBl9(o&is3~OvsmQ@h8sr8@ z_Gv(4eO3vj!LTqtwu+|R0<8V@NbS$~O({aanW5XyhDufArk$AsKgR=Pm_9N(5REz9 zAfI&vkG#e!o7)k{U@RSA#h-L1(@q@$!MsTo*n$rf)`5JNHj!Xd>d;x~_n=rmGIT9~ zkhy(f8`Wwk3IxG2@&OxYAM2I=(>mfrjv_j1)MiL?F-K$^?-&L9xh?c(L8 zZQBTK;gj~(fzKM^GB!t&Xyso_0X1kjHu~e+UPxKH9WHt=vU$R|h3`XqdoH`d>hou# zb0m+&E?ioQMH3$bHEd&I3~b{?L|K&+{iw#Jr3^IPrVV}1JGq4bmA@T9{GeG**O8~` z`&72dtL~9A650iu_GO~_XR&t(X&#`5sn5TrVS9UcoLm2H`&>v-we!;>yb7kFz`{$R zp~dH_hRM0@TiX$L@~lP#>ZZ3lo^KtcnV(blM*1Z_eCTd{pL}e;y7T%+-2}-2GwQt; zZ)*7sZr2v|g^I1z=ut@<9|7+_0hvTVPQ5g4R z9nTb>4%>HZY<)D=&brEquDjNwaq#MuitX8uL{7&!MnKR+k}1DYR<({;;p1~k8?brZ zZ;@UPF=^%;BV{WITNm^%!eKdO?;0@NjLW=Q$oHWlw1a!uA9v3o>l@YPvgox7> zf_6=mcTgQl^qf3lOxMpb_T_rI<;=qupLi;`jWdcrJvuK#1z}sM`O-54TK1NS>9f3a z)nZfa($mvw=C?cSsmbTI7Yy&Iad!d&qOdBIy6Qa8EzG*frr>jd}< zz3B=nM#)}V?ejdBh#x0C!Ll%mT0giM zzgSa9LQN+8*5*!0Th$z|`-~n%5E$xCfr`*KI0H;;Bd1lbKKZyS4(bb!`*-6airc#Q zai|09R5bnKGE3Jak-2EB&=2J8m&{_(uP`UMqF+%8a;+U(vMeOM^#9s>>!>R8?`@b= zK`=lX1Vu`^B?K&#MiHbtq>hAiqku{&AaH1d5Rf=1Eh(J>3Wr7nMCpTcy!&=$obP;p zGr#%gUF%uvSI%+IP5eCVU^iJE=Z#jPnNgza zwNEEr@Dgtsu)|7MicF7%4Z`|rx=8mVJxkKi$-E{-of&Q ze1e#frT)j9?c&2YHf-LThG>8L!6W(>0S;FwYV>(YPBT~cRM%RKX7(*qw)mP4b@}aR z-Z&{?v|sP4cUn4;C^Yq2V5AZon|GHACOA?#urf$D^vrE-Ei>tfhYIN1iuhJVZXA0n zR5D3j_YJ(O{F?5~JK{K=sYG2V_n`PH&C*gSGrzeQWjAmpDn9oSp2eUq zdhOQjC)nz-==OoKXuHX~)t+ntRv5FQ*J2}nN~m-3IanUHp(aJ1I zkI&?|uAA$RJn`Z?PWS_Bua-(#GbYPL>3jJ`tAu^fmg<-q^ER)j+SmQB@dfi%8g=ra zC5JkqLNCwZE+Rtm z&59EjU#j2}XRG-uvtj~z=JYZ5IYs@zDe(PrV@uxicaYQ+db3?1PH`qY#m2fhAv%>< zg0)QiHYDJ6138Lc;RRe4(%NhpU9FQKr;CGHsP%ABYZ5cXU1e^zN@(U%ctThwnLDr4 z^p%{M$K4%NX8xPw@h>ZV2i+vk`|vDEUXrx5m-JhE64htTDyL-hbUt#Nuio`>Z5Uf{ z<@DgSDtSvGzf53Bmqo3F!=+@zh>OLk?2W3Rh1d?LNcfG}Jz*{~bOO0cPTb}pID_v{ zL>WHwaiq2Ck->;B&KJ0~$5MdZHT%oSRpNa5|vN5|kY7vqPqUUYXhJ?GslE*L6ov|q$dSIB*a?RmfM z8;f(cFV-fCU&bt&DU9cz4fkJ2QNZ);->%}J!^PidtsG_J+Ei+>8|2yF{;nwJc{%Ee z=XNoZS?rGzjlD3lo*UoFb)vX-zlKDA-Z46po-}0eH0&^g(JoG{>&&IiZS&mE=WiLX zl<&re?Y9loVYLz$a^9v2@9b=+n&iB!6mN2A`La><)ZAM(+1#Ev!CSsqRVc3I4bx-u zewqC#S!_SnzDq{7jjZh z7I7OU(#9&ap2dxR3vH(|RBXMRX(p}l%~N@$?2K>{+zQL4O=l?gCeB2=SAVe#&Sw{` zOjP~^?y_HS;^$-Eh2piq^~WS;r`@7`d-g7KC=H8nK+W;Vv~*ehn?)kd%fX$mkmdBx$EAu@+mtA8eu9JcP{!v;1 zmXxo~B}*Vd)mu6;tUJ1@t9L%&Re#kco89Q!yX@Z$6{x=k@9#?6CB?ja{mK;^Iu`Uw zA$Ioo_dVQ|u(?szPr3qju`!aqUi;}6DP+U#Ct{0&y`s|wNENuI*``q`?^_b?AB~>- zlzD)Mdd$yZ=!i+Ox!%y33mvZ%8fUJYr{;V5HUTAz&KYvGIX;@aaaEg#D>Ko4KFT*= zh2YIR-!98tMNE9l=XX-Gak2S+LifZCbFakjR=U+?D;VXgZm~+g?U2P;ZNL7C;}$a# zVO(9hWwItLACw+NHgeQ&gLe*1B!o^G8b~!%P#BlsbLZ!IV%HVSG)=n5hToVOh4&RV zl?W1@`TNDng=rHVRTzuWvXk~cpfzN4) zSO#`{l-eTNt&s?~?U%%b`4K>QOUStWF_@UJ#v(%(B2|CqxHGkP!8P5?_TEIp3SEjH1G^eJa2Uz-P&+4_we zsdfga;Zy4RS(;Olgl(9`+>kH>j5>D@bG_1&`JJIV)eYn4%sxBiUBip@F?4ESBdoQ~ z^XYagzOKMwAEwq;=CyF{?FPm^sR?%}m`B_}NM2}Bu=tLBN&OKi9xXdo0hMXKF>x(D z`?v9$`a?t0TR-!M6Y*#Q?|t)O5Bf_Q0%5mYirHI6+In+}E^XhC3;0GuE9q_iwLixz zNb;n#-Q}RzS6_Fi2IEg#PA_y0^YdNdndWg(_P^gM7`@-(uK)6*okGFu)dNxBNSh(; zJnI(f)gYMx!8Xr+V#Q@>$g2_8JR* zk6R?opx^0`-B#P&U8L1lwqLP#E@lwAN3s`7^*C3>4O79v(PWl#=5eBZbd>U>wcoBm zwztdLu4xL*w~BZR=b23zi33F8_WOq%0$Uah`06Z*&aPfZW+xl38rQS@p5dEJxe;<= zX%c0jkd}UpX(rE>+j4sL-&ps4K3CzrICDUPAcd-|S;10Jrx4rIGG(uJB~SQR44Z?Z z7LQrwOohz#Cwl{8YdD~8Js8q|;=_BV8nb9+P}FO?Eiooes~8~YZfi4U>&0wP{aZX~ zL84xG7i$q0UZievkKx^l=w~4xF>R?aJquM~=Mv?UV>i?GNZ;6CPlwS-$-KEUR6d++ zEHjy0S(baEElw{bXic{_9q~+>DPIuM>s6(rttHm{>zUn8T`hA zw?7{bdCwnL(yV>3>LnU~dC_lg`tzNC{kYvuaKMcL_rY!Kw`Ko#ulfC9Mi0>rzZMT^ zCD;7*xBvdt0gXJy4kd!jKI;bm_W9q!A*1kg&Kg$#<2(P;d;cGQP`M({qI6UiLAbv0 zQo-?X{hE`RQCUzdt^cVz7*t61+=pN(1|U3 zoYnhplOY!%c?QN9X!qh*{%wp;a3U{#9QOa-^naT#B=-6LYfNva5kQgX5-nK%$IJM5 zCJ)y`2M6vQApAHE4Ky10?Y_M+gr3j5!BqPAkJ(pmA|hM(R7m)F`aceaKNp9(0RbX~ z0Cx>r1vhZ=bo~k@_Ot@VayzqNJMRuG+sSRDv#_@Jazi};eR@K&YM>HjjMcG3I}AHE zHOxZ8y_?$LbyKfv`+}xDcV^q3pG<{Bsx-W0)YEnY z%y=$>EVK^(gc&?N>Kq`uqWO4G*kYT@^O=xkC3sU#78|=xJ-rnxXf<1cXs`>wk31N0 z=3ixK;T|q%aDqbW5eEA=*GmyAaJWn?<4fX65N_@U0E?5kpkLG+&lH*m#}iV({~v@_ z0YluRWW+T`SzMT>sP$~leZ(@nZ&kSS9IHNjz4VNZFd8Vod1-^$C-<9ds5uK(!uJs7 z(V%)UXXW38FT0bP-Nw5u!Tlzgl0%gYOh~+1hT3>b^jn-$e~#`=l{B4?)Na;)f97Ph z5m>t}KvkF-=`-p8-2M<4DqhqI=pyTZI!K9ts7CyyC_?7eRp0YOpix+(jPE zu0je-3s7Q%ICR9JL$v?>`Z;-`E)M1RW2(~2?<;X}qRYU=4Qt8<6w3rAhu`*-&=|DH zUayN6G`k3`b37MNN6{oxl-&WbqLH27Q74_LGvN{eCxVs>BYv&yT+k#-}1h&Nc9Q=?WF=YB4A~Z#~G}=LZD3o%Yj*s$X$gtT}nnPKa$JBU5w7{$2-9rzf{Fs z-t6i_)D}ltyCCa8`wG!ZSSf7|FsnhJX^^Oj>skhCHCcVg(a5GyxpQ>m=6W*+wyg#x zl1QLfahEJ%45D{O<{p-PlMp7YZ$kT3@&gN~f zR-}gAC&LBRTf^zh2Q*OLu-w9$BDxOsHrrT9rkW7o;>TO|6(X32ym}Nk8tim2krl4< z^pVH+OB|Gh8#gN*D6c&;^MunNb}Q+}*AJ|Dp}SM;N3sb7=7E#{Xb8IGND)VH4B^K1 z_ry`Gsj~pA1cM0W{rm`Y6EF?-d)8$wXxRAz?n@DN{5B!+j2H7&Xa77QfBhz+i{DC+ z|E5P#cuux9Er?o|QXAJ? zME6K)q9l}x1p`zAZxH1lO0!_V7vG{2xvk#LDFf1$Ng0YWYy~Z1-f&0@KJWowognCK zBhCo?hG)Q;5sZ(W51k%+T^z+H_hU&`W1R0)#&O`J6N;}(Fhca5 z#Pr4%dR&}bJVLe!PlLdX!LH!K054a)@-Zgaz(wm6))kxso>a4IH zG%lK|5<&ibhGQr~c5j2$f!+P#!BiUh+P;hE(K%-PiPeyC?x}XJC#qVzFa$IP8-6WT z0aYYbz997~R2IRv4=f^bEPKXs-q5{=rWZ?1@LC$NMiNZh?@{7!NEF(rZtXY$$DNfs zoEe`xh=L-l7iPlt$BH_{^yZ?wCWvYDkAaIFWwZ9ozQlg@ZS##@&X4UXGD>E9jK`Za&8MBZ)^LFdS>oFM5fN|InO?-1y^MBaDzW?ePZ|`dXNC zEu{^o&da+O>Z9oFGLDmF6n4x>g&Q*lTxnII6gZ<7V?pxi15Kp0(!KCaM#ob{E>LWb zD5Bh8FVIKJyNai>p{QAgM0VRVQkgKXThynV>yM@i65J=M!ukB1YdY>vY$G&*5Cg7iHP03HS=(=wfT;vPh!Q-hGhKCwt0y??<~6ki{M-PBK?wCoZe^ZpRcz`erM zyISX!u zL|6Ed{+>qtHSD`h1Q5odFJHt0&k?2e$j9;qyaQcnlRq!6p$@dHO0 zt}h(L30D6GEdFPfLZ4TI#6`kiIK2{!-;kX(Yqn%SrxCRuI-_-WxX(_O=y6ZGQbs2B z9TB=k#9Udk6_2u@wKHy>b$t`!BT8XkE^?I@H+k7Y4~qs~1*n~iD^;nObtg0w zD2;!yb!1vhu{EkN0mYh*>Qz=q(IAi*YTjd`Tmk>g2eRgeqn}+{v(O3mrR8|)f#oZH zA}4j>yzil4N@-Ty6Zyw+PL<}zGsAN~nbKeA0w9bp_7!{9wcy#++1|ULteG4?9#L73K2ZNEtofl!Nr%r)O)N_d{r%}J z!!mPW65XhG?aNj}oIjLj?S!1UiaN*%sI09UrP>}Bc90*B;cW8nY4bk08TPnL`ElH2 zQ|U)4YaX>?-u)S@CB}o!xuP=Gw^M+{Uqvpy>ugi({Ab~q&gl8jAA)Pb;qy}-2|=mJ zT{8%h<3nRCeb?9KIb@XVH|I41Bw3agLQ?NXguXdEKUB=;X4@FX#2oLJ{ITMxRK76h zV^VL>Sx@MBsKZSphK#=G@ZMA5L zedTiFM4yIQxJ8ijSWdHTV}1JYS*#CM{?h~m-wd2|N_e*soQExqL@Tzx+3Dr5VC_pd ztOZhLgd$Kn;mfjgO>xR*t;8CQX#2DnyW-;;GSE$_4wiaKGNY?fBUk#)9Gt;4gf8Yc zZ&0Hfi`5m~Yy9!q+KiQA?+hLai+YySp{+c|TB?Ayh&|;MMNyF(DKF25?;v!VBvx>Z ze-~9gn_-gwZtYPQ*TXhxk-(Q>n~KI&m5o-$pk$0R`%d9~=CN9_m_;OsIz_u>rEaTw z=Ih6-=#GXrPzz>Hg3xJMxK`3SY%(#HlrQ5b#uzIb_=M4qzA1H45_TTdW@XG&U3uv% ze->_?a=62+PbWT6VHd^Dt=8}JN1VbIiI3dX*qKY8Wa_yYb;x_WmYu&vUk8XD0bF-C{SG^p{3fo1OdIerMnq=pUdTE^4 zyufk0eTGA7PiQ}K$wYz-ExTJelh@S$bKdoKGDREHjNGEt?G8d#+^YUkC%YtOl8`dQ zMvy;7A&F>4Y{3;N*-g@zscvnq)WucXO`^=I()K~o6;N@D>+n;(lUbEN95cC$R=~CP zi@V+CO49~UWri9Qk^55c-0Yk=q-kE?4ohxW90Bgx`xPs5GIjb09>Rr_Uz9eW5d|&( zjdZ;ZgF)N8GRsQ;_g5%<;~YMOhEm3~eKw^q>g#oU*E8jf0Lw_m9?T_Y$uu_0nXc%t zgv9qB#66?Ru8lv!9n~A5V3BKBzf#rqQERe-1C4>6aIvp6GPO9B;-^|Lxz{WOXUf}C zEd#XfjjzH)0Q~zL}e083|r}Qo~-m_02A(aay?9 zg>fx)-uQf2wUU3*r2I5rYWT2YR@=}Kg4aYZ+6FT(`k3z7oq*Qb8zA(f?uUsFy<&@)3oTIY}v%HNC z6L~|!TBBPV&ud0LS|z(^8Mm`55`uq+z*_Q9Tt1b$@$}&(zvM>pQ%Jf_wge`e@0O2v z2UcJ!Hp+UFL-VlmIu&zQ4sEeg`cpWV3piYeOT1kA*!1^A8Xx}Ej^dW zuydMIx7Wf#$7;b9moTaQw1#h67I?$4Vc1sBp2_R6Dw<3Zu9~|$g~`|&z9GF&R@u)} z<1qRTt-Fk0JT0mgBy%-C2B!i?R(YZ zIBi4YIA86!fRT&ORd2!Q^TFt6OD@&pzM&4d<>>Y8olrMEfUhdBInnuU80oRtuxuhW z>`-6OCA0m0`Kr^*vdba()sAQRy+JvVITxX*)0gHOBE#fkFRn&v6NE&Y1?Ka_*&`;n z(zO|3ltu(#i5j=mF5G2q^{}f=m)Y$h#qdOg`SgpU^xvmD)dSnbh)-lO5cKOitMIs8 z!AjE82shWI>))csGqivEks#d^V|MoVT>6~evtmdZ8^%O|e{S+jC+TM#1X&UJHUDKE zy2xl}l1?VhUR}e<3VhALEDBSNv}cFaKNZLO-@fGLuy1qKpG87^6Ss{X`v`8RYS*d> zKXp=z-j31A?LSm!65G@46 z!7?kdV;Q)OjYzynd<(U94^@)ac4j9V#T3B2gLmlBY{+I?LS#=a9CAlugiq7PMBJ%& z+WgdV7das!@M`;q3v){K9w*B7gsMq9;rrVe&|CDk;C&q^H1kg0t6yXsWj&_c9Zz9V z=97ur%K9BZC081U7NQTuzY9ftfmvZ{V&%PaRKjSSk#+lgMx}+f(a(_Ii6EwDn*API zoIb{hsWb)&UgX}A9_yPrpx|qBy2jR_ME7W1GFKX4v5<9&#aeY*dLT@(i2gT#bu~2{ zTA)y8N$l6s^_R0jwI#<|PFT#eC3CEc;;4%lap`S0Mnj_Pcy7&Mcwr+y_}VIze_)Howx+`ekr; zy3z7ESJo|=?atO^d}7OTxtY#nD_YzRyzi0Wo6&oGmM?&>e@v;^`MB%X?q?>^Dq?68 zJvWemH~_r%J{9rn7I6K&k_-@d6`r^Y;v>79!53^f(KrHlinKrq>X|E~!XbrbO~H^) zNj;w52Fg+pq?wuBbFk%p1&V1GNO;P>K3`5k<1ABdxe1r*jXYn=^d}%D_C#{!L)NYX zAm(6h^U9zhZvMsu0MS?A_t(nMe`mkAFXKA(56Gp$dOZ+7=s2RL z5HdQ7c=^fBO{lzF)l=ky{JF`|)Ab}lNsN9vmF^fIcuZUCi_9}E$&s}aT)mYL$j4PK zQo>#|z*ihIf(&$dUsmrT^&+e#3fEDO8TWhguHTzKod@p;vsY7yKx%BI?Dr`XF%1Ah z3@_5{j&1DJM&}yjNezjTMPpYnm%&-w2^z!V>Pa|~j)ja)NR5J1NNjQg62tv5oz5VxZzy!lFa7*z!*^kewj^loJ4iU5+A+G#AB8(&w599H$%WR; zH^mU@e+$N&S^#)nEU;ktI2bHKO@7|oZSEXQ6+Wc9( z-D3Jn@iybl9Ta-vG_~#BB4wZsIm~#2_08c0DT~+ppd6Cpn!6ajEY@2wf4HN5b#Cix zG^Pz0wrG>%3j>*z&b=Ab_v7)KNJ#te@r7Kg!Igi*>r?Kqq=Vcr(Qi2Ow`Vn=uV>;u z=opRi9PJWm<#q1Oe7!&lQ11EVMw{TVadu<+OLraWmjKkeIYmC>WrkqyU?UN_5NhWj zR9-TKs(2p9W!ZRfHxJ$H~qP6@&Gk17OaL4P^=m45_nHLs|N+ z5Yxq$j}`*KKLjXxvQm1djxyzn%!7wM_&@Dai0wc%1C z6i*rWErZTdBG}=bymKNvI{~`sbX0h}x7e~Pa2;K5P0-#7dyy;QqXrZIR4yl$w=TYL z^3rzk>$VZpwAtRE-2vCoUVd)l#5K94xKs>1_H9F0c%lI_{*-qB7>FnvS=dJ|!g1R{ z>TuzHyTKnsLr!%ae1dY2?fAWFpt}^Ns9apmTNCcIIfWNvBGcsv#6Yl>#(;wrJHg#1IdU z7prjj!p%pzIvKe>uU?Ux#`86$M(9kjKCDb`2T>+TEHCzoWzsdT2tlF;>4#rSYkt%3=FaflI2&%UO z2MHFusO@p3eP?J`I|S(9f$)Q|GPvY}G@ zQ~a}}HP|6?uDjqCMg?RiijJe_#{Ih?Pzc=Cylr&IoRX{>Iyryt7(w{xJGqQb&8L>N zKk#2VKPBZGqg@57+-#h>00GTVdhfT#WTDk~MAycSqc_)7=dl}57xPNmgM!HW!fK>o zAbU`|klZ{JhyLu`^jPHg>G~FhH*-_;BNXFea(B?AZV-x5rW zW8ttsI&KV=G9yJj{ktHOJOXAwE?=Qj1X+U6@xwyOEsr{HeSFl@PgL0}?+|UwzFi0f zmPQu7HdD3fv$ckST|{mqX?}nY2xfH%C=l6`U?|>L7#1I-WGrB%Rr$zUYz6%P$XLYX z-W|YP!hq{MB)4e>^Dk~+WMhdCZHi$zTWcA7|8(g#2oHz|MuH-ko06BypfP-5X*(=3 z>N_mWTGYB0Pc`eHpu@BT{!}09fwoCazL5)0ekWjj`3jA}iC9C=*{>bv#$)awCvrWt zp51Xz#5nf%C{XSN#6?4#M~b9g{f~(v<*CYyqeHFv1Y82g22@qY#7NsCd6^ zZbj+c_zfb+qd#LLMQ6r=l2zv*4s`49rw4rz!us%D9dY~b!yifInrz)gI()ff}8Er(t>9@p*zEz1wn(M-Tn68whFAj6 zAfG|`5++MtjB8ixrNzCX7%lc=s!LDpZYo~sWd>}cPef57Gq@oyWNJ2Yvi4{IPQ*b! zEArxY;Q9?EUMkM(D#mU|!7qX+$0NUcnU&TX?u|y{^VnipV|*RY33vAfYRu-C(h8BR%q;6fXc(-gyJ%+{e=|8OeEV;}k$g-(8; zP~HLAxTyf9lI8Xwe`FY)5}nCfq`&hO2DYf9Qga2`*=T!~0%fBZ+l-q?9V3g#mOIKZ zRCXauNvS)QYzUy!d0tzife55tdv|_NbZs5fjqeMe=v3uSuEQbHMA2h@+<6C^VF$MO zx6=5xINaMh4wy$90by}jSRsQ?oLSv7!4>S@8k;}@)!>-ok$Pj|k5v^a7ya^n%W5O0 zXd#o}hoJ)!9GmGtoNJl+wEvFlC?=&>O|lB5gZud2%RCd)hP2M5>MzJd@qHB~W+K|8K;^yR z!6Ik(8vvB{>=4;WvD2$q<9X7>2y_J%yYsIs@6UV>exr}YqbtTP+xEWW7D4O@0#dzs zjf~?qTMs88I@NaA%WMV|U$`^iX$5@*)LedO%xz(CPhKSy_dOjKLsyuOMleZ<9eeru zwsD*cj(EJ+MRUeXrzt-^jR*ZPl&}2usLJU917&6p8rSE)B@E=sHRC8kskDWNU+MK< z&a*y0{KmJ4#GxKy6v{RVVfMmb1r34H?^b_#{{IO0vhERt2@e70BUx&Jiu(P%`(ovK z2Xza-|K0NcYpi&J(-59(S*!E!ZnD3gxP1w5=IFj~Yxw)4_&;7Ow+wZf&=M!r|FLHN z>%-O4J|T1@`ghz~|MdNS{f$R7fJIb3b^g0r{9o^ex9q?}`}EbI@b5?ZKVGa}318{+ z>!g3@FJ_4bSiUW`yp`)uKt(}|q;u15zxb1X`|JV1Z15N+Ols>H_=i`&e|(3(m|XEI zAN`Mq_%(1MCx0jAEqlukYy8{TWi`Pde_yV3{#U+T+A=UI?WLOr&;Q$c2=GK0l_!~{ z#s8I2Nsnl>8l7`T{;zt_j&IAw6|+F!4X%0FA{*o;yK_c4Md)o<*`7y^W*fB$HeE8r zL$||Vp|bPWwEf*wa;j^O!@Uu42UU!<46@4pB6Yxxn`Fk6w9f^fpn{3Unr5)X-e4gk z0vN{!0ujP(8BX#Z!K)fTQ>Y=_EL=^G{O@fRL$LwbOkI+5CtapRYFxsr2Q->G#Sa#e ze=#$&v;x4Mgx?N&^%ZtOl~){Yh_|}iB|Oe=od(eaE?51-MobAR>;!RCAk9F((*p>; z&N?$$%jN?hA1wMn+e^>bd~O#2xQbK~q){gI15GBG|DEOpn+mMRT z;~;GoG#M^+OrPdnB(8oT1Blj&WzN#}W^q|va_O%lwfh?JS!drvniie&8+iknYN-wY zyoDh2NFc$9nE>U9oFAHp4ZtXc$`Dw~0z|6+q3wjQna4~+!|L^RY5naDzy(6lA3c6Q zY}4?O{xpI-kn}ExfIy3ELF*?|TaF7qFchJ7k>TIJA-wrqKL81j972v7W#+ipI z|MP#2Obr=|Ik=dRfNQLhJH#@WqW3cv zukQdSX98lNqew~5SY|7&5Vg?E-VDJ(z#tVOP~F*g2eX&P#ETn#dgxE!$GzdVjqpK{ z(EZ$($Cv65eq}LmAqMtgr=1kTnn(YnnSy8UDXqh$koHe5a%77+Tq3QeS?Oq6a|o0& zuJJF8K;Y#thzC%X9)1ik_?hm)Bp=CWOYSo$+x@wVgNV<1`Nn&pnk1NO4N25 z^10L(LMLV{AcwKpNx3Gl2DQ;c&_>SU9Do*tR4>oPnfnr?h-6X8T+JktBUb6gUBzcWRS;Zrc%oZiGgd9Q-}%cajK zPW3qAPmq8XKs&!C{+rnOFQM&7Jw{1*IX!Y5&&YS zy&*Tz8_zg7Jwq^YHYb;&=-i{tp_)C%e%9><;;VcE*gs<^(!;O^1lETa5F)}%AdwT@ z)d-$p-r|ANqjpVYBY+Jv(56@;RecNeE4)i`F6mFTND=$pcu5x#Z0E#=RFP{$pd}gLu2&V;$lv3>sQQ~YO1NEzv zhD|`UeDzjc^#Rg(28Nr<-sP7}M}gVkn@9wqiU~zqS zJx5Oe!oBb5aGec3t1=*hW|$tK^Ksn!faXoNUh96z#buhmO}Ah za`W^hw-ZC>W?07+@I>v0&Eppjd|utXIPR}9X)TH9OKSTxQR%y#MMWkfn@{g5$<<+klcAnmS+JbbQ=>MF5>qpKJwc5 z!KJV6FXhE`FR!0|YAq5dJ=qi)SsbH}_^nN@&PfxYxat+T-^gJ0d&sJWH)*A! zg*wbvVWX$55j88H1D>}c=1xlu+lB{RYd1Y!V9+PP;N7!*E?tAK?#Yyr*u}vfitV6j z*7`}l)_1SB@`mSFM z0(Cprm`(B@2?hCO|1^{ek?X|f`daCMz$p|IoJXk7O~q)Kl5iA9k*Nh9Av{3#z^sAd zt~Au$t(w1N4|$0=7f0|1gc)7l%mJPN*8!13VFF^uC)W_Y1%Q%T4<_~UQe(oIldAsbp9-s|oYd@e@^5>nM zOJ(zj$BNJ`qG`Gi&fVumLeXkKZpa`xHvT08opRnQR+nRP@diPJV_SBPAp0P48@07u ze-gkm2q+D&tD5CJa2%PN*tkWQu{ArW({gPCBj$TfCI#`;YYg`bN3{7_ubQ z`p0;{O4Ch~>^xdD0a4&h9dGqvo@i;g|8o85So3Y?nPo)_o{1eX#<x_sEb*=>r1lKnmI6EIT6kZ5qx!=Y+^ zym1x@ile$`odXBMM%sQH`=h=jqKqFLVC;T~zW$Pc@+Cyv*-$$B5&O&sN>M)!lQGE% zA&KEdrlTjto?mRcd3TT0UirAN#-MmfyTkEN(Q+M5DMh*a%b}#A3w)N!Y>Fl{Hr|fx z*YpiZrv5ApKGPF=z_qCV6*Wso?GOmGbz1zuoSm_**xQ{Y&Oy^k!gQrvY$Z>BlmytB zC=M2mev`S-Ida5^`&R;Uf_i+L1w0^3+9jC9=bsen+D%r>LY!bQH|NOIkmQ%aq?#zk z8&T9Ida&~pw$ZecISwL}B_R5|fA-+qt&Z5+8H*QZN@hKA=PB@}y~HeD?(QY2HCz7{ z`iPL>NAZ_(g~@BGpak0bAG9IxjQ5?V5*_U$LLS~40bAVN*%qM!mUA23RrcSV+yXb_ ztHlxnMn12$kq%^>KenUN)TEp_CF3N1*CW6nwk0pp->IsG4@cEYgF`m=1j$RYrPvbC zl1x6=fXd2(yp#?PFqFvSI@0}QraeU+rGV?Ce$g%J~9gL3ui zwJ7R6Tg1du51h-96X=SQe+-Y@E+tF+HZ;F$Am=|Z@VeEIZus@1l(w_#A;hr;+0O3k z^=frl>sR9{o)WpD$NpS>zXq4{SY5K)DA&C|un2YQ060c+d)i`4(rQOm?t2>rc%B8> zW?8*tj$2Nq-lFB76Z0#xe9NIusU!BbL_AODcR-6iO0GBfjqpO~AMg9;2maS@-Ri_- z3u1k&-+liaj$d#5XLJg`$=#!XvPRmGOsW6ifc(m+{_?iunt&QzI*(b1`?t>*;lzgs z@|#1s@LxXbZ{GI*_=5&mlH1%FrX!{=s2Mk5rC%B?v;E(nd5VN~m{+ZNohpl!VAsxVE*;vYm{N-NJXa^{Eh1PFY5-5Whz(? zKXuH1d91&|X^q)oc#{>n^W^{L5C8hmS&smW%KloG`){j}Qy>}!s`Je4TmQPmgNTs~ zO#rADb;5mxZUdhn`@EZhdRjYzZk=yMsB(VLR%!!yKlK12+!7aSe*Mp5Z_G}roFKyG z4b8VFK=GIQ(AGuMf&`K3Or*ydk|I!vPLbx27gAb#x^1cl6pQ&KAko|^%Quw*+%q=@ z$EALh=p;0d>xQyf!9*}aFoeMz%D~(RevPD@mw@bB5AGlgNY>h2?YCdUT!#@E6JP}!M@M?#lBLj;e!;IuAy>0IC1<=C{ zj6FEcT6f_f)6Lnc!@Q+HE1>esx!4OS%r=rP$8NQd-#-i4zEI4Eih)sp>Pg#nCGriBi2#DWP+PoOKCEOH)-;M$DjGy3%lEK&x0{ zWUn$=6L%ScNHQBY(!N+cbSV%}3`^nD>VOQ!h-8Qx@gpY#-$A=@^HSgq*Zb{nk1fIE zgn*`0_Lx6!&eUOwI8nc~cW`y3*VP)t=C?`~eP92BIrTT&_EilW+;m7ULPYiC0Jv3% z)Iuq1lKba}Yzpe4j`mwndQrY*l->mCozO6h^70JO{ttnFY*;|UuxhW*27J9qndvH{ zenUT~R45wG-#*4876kJL?pZ$FyDkt7EtiwgQi;#5l{hgS=N9b zCwd1geJ!>8`^Qm&R~tM|WE~`c09GI3n5){r532{6)XCC&Gr+{+eLL0Xo7pa>o1Y?U zyz>ECfZR%ol)4!qv-PzbFtDFw4yjgh_Qg7-5br6C`hv&S&rdr|WGOBv*BEn;>&`N)*d{rr1i^L#brB)27Lie?^w!1dq38^ud>-YCWbglQM98_2 zICLbW8PJ#NS)wgWFWwy7N5qc8Ug87B_X?o)9}HCK#u9kG7X<)!HMOFFp`}Ll<@jYp z^h0prCVg$c5?|RFnHgPaTyNU-zLMRbp&YPCelKv{ zN1vd=+IFP;2btNo6{S6B>lIpEurbxfOxG%u3*tOSL0>fVrr;%Gk`&nsoT2e7UOv2r zs0c3@ETB9=oJPQQaw8p@1=|7H-rj9Bf>`ZUr$P_4qyNR% z(A<6kqEhlu-eT=iXrX7y66e%6hXejBb=-M`*e6qvF0D5tQ9;gF?V~7zajLNCT~y{sCs9i{jML6bB43SyO zA$IJ;Gd!4!Bn!)iC^H>`upI$v{s7Coch2EH2zjc%rR4x$Uo#JTL4qYQO+utCPCwsD zs!Xd(=J9Lg9%_lCm0uUp#GGoS+*zgvZDiA0gNZdWkc=7)KZ; z2n{LJaa}FpLi@_jMKc}W(n}9yiHIKd;!m~6S=!#C<%|4ZNw z-Le^|A9mq`7-j-4ufyO!;o=?I8pqR{t`tYRYGCs!-BgS>zP=ZHle)mMEf;kQK#PeNDmUS^=l&Qw2L zszaxsbO_IhdLf%1J5oC-AlA$uz1`BpIl?S7~_^Uhe!A>ld#gc zgdAJBgPDjtbg#i`k^1ydoWC7}T&D8;Y;>IUHPWnCBf@%hV(IsgYgAtp-|~P}W;@|7 z6A%VX;GBBE#=OxWZ?FwGkgRIv9HC$7A%Wk${{DOEApva`Dga?eGWx832(X$@8A?$NBIL>Dj__a-O=iXWO2~`d-U0GuqHUwtub2tq6Cs3CFIm z**E*~!xJCKK{`nOKs2(b9+xG?%JRLzL%%^lx;wa#2NiCZaHF*tRldF+OxrAT-d&kb zN@JU?iZ5|X*L-|iCBMXaB6zyK7=wBz-;DD=XD8X0{CTt*E7-bQ9PT%W3Hi~n7fLc* zxX?IjL3=5$Ke|k*aUM!v+Q~T&eV!OMymb#&&%JGy=nLll@s2~49=Jz*h7NloUU7nt zbA)RW^9tA_bOznFHh>k|P{W69ZaCn2@F>G^gm^r78{J@OOZxdD} zDPaZ+16+Nrq;nXu+J0~R0aHKp%z>cZ`K+(B__Izj|D4vT9|y8A70I^@pIdGq{%Ki~ zh8o*vhll#rnO_7og>9c7m(Ei*jv^i?syU-GvsM0z`l4`;{vYR>r)^P~)B{9rQ^4?2~WXvvh>YqM2ot&(4}_q$Bx zpT8Vv(fEL=@GW@lztYRU`Om&mNDk>aT#5hr-u(S+llurziPlK>$v-2YfA}qco@T@E Z&B+Jld>=kvz=MAjuBcwlyJ#Hv{{cF13`_t3 literal 0 HcmV?d00001 diff --git a/docs/onpremdb/images/K8S_SECURE2.png b/docs/onpremdb/images/K8S_SECURE2.png new file mode 100644 index 0000000000000000000000000000000000000000..b9713d7cbbaa2cb1d7dc6634e8fef304aba79a99 GIT binary patch literal 130742 zcmd?Rhd*53w+4)uNG3&sL_|%b=n}mSB8V_Z^fCw;eTd$B5{a6I=$+^cMj17NsH2x* z5WS4vjp03h_xHW`zVH1d`2*g3KEuiEv(G+z?S0mI*7L06y_$-`CGbr!2?@z1#V3!S zlaNrjk&uuboI4AA^PFo3L_z|_S<1?)Day*Ss5#h~S=zu!NS?fpjHb{~AGt2Q=DB!@ z?EKXaR36mn)Y4BEQj{p3omXT@`SJCNhRwH&;gdRQS4rX06pv{b7)j|am^28T(Bg9Q+Ir_oRs$&J=9v6HMV0r=uV;*P)VhDRvE;i9)CA%eb1!1>HY(I zRnqIE$9(Q@Dz5KytN`Pdxg{BL{x$_9{Rb$9t?AqkB)o*~t4Uh&VnICHd`$$S2Jk%*{( zO|0CJ;)&w*wlk?&G55DrzM4UX?msCOW)*OYRe$WmAQ#W2(l2n~W$nj4E*MqH&&b<5 zW~%(!VxP4WnevM+fEeVC7+z@L1wH&{q22m&y<*|uug{e+ltLOQL>-<8;@+A0dcSuSUi9h{rZK3ygm zVDuYwOEid;gv+XWy&$=IV3d63Qu3LLe(Xh$mmJ9p>Z!9uuDteVyFz)Ll<7)nW zI_csg`46C%7m8V2f6{{A-cEL7JOA{}j}HubAcP+{<@Pt4oYz(<%qgF{MHj&RAY9;K z@~s8I?ep>tFTY;4ZfssR_zEid#UcVWJeT!d=@)|tdHn)M5A zG$6G*G-on`kdKSi#=kw#AmaetmCKj8mK^*{^P7jpg&am_>Vc4(^4^b4djtfi5$|2( zD3a59B(|S#-`qBHVn1Z5y6R2pO^pj;lJRX)d!TpYLgxZe}p4SWVp|>-y4qa7xxAP<9N6-)9J5-8EGs>~5R39A_@201vx2HpT6_6Yw z@p!1po%UA!pMo=5o06OK;5R198|in{kZk&FWnZ?ww0xm_DsY$cZa9^ZW&ZP+=V{N& zdzIh;$d7#b?jIe(9f~*YXX)D-HOL0OE|(yPqxD|Lj_9aM;tYPcz%hB4s(W zHO^|<=QQWk=6DTA3=nhXH3^n%v80`G&1r~a#$;IfrS4)UYO$b1dERv96xVKrevJYA zfJuK;Ze(_X)-pW#M)SqXtbxUm_M!Gc8(I5b;4r_0=!n2F_{>bgw8nF1Hv$#pjQ-Csr7pBX+BYJ zs7$`hxs+IT7wcJyv8G&p;3VPP>*TT&-$U1%u~@Yn?Bu$XIN;r-VTz2Q>!6wg&Am&E zGW1I{D12ThYo|Q%SW3^g#wYNY^-L;hJ4=&smP}m3yOe?#?|yMKiZv4C$yuX5c3YG` zL-%-^Qcvqlhc!c+1Me-}WBAPQS?qIACXG5}Uq+^8W_d@E8CQu@NsU>;Tv26UWp{PQ z(OZzsFN<5qVe1}23jP-Na5t5KcqS#}-BG?#WET<>{NkET*?3Ot(yT#a<$1?)3?hjz zfNce{NBifbF%1b5gR89 z$Ni~=88+uUR|W?@r>71au1ha^bbsY_=Cya-Fz1~w>&l9ksuHPsS;^pd&^XX5h}-$F zJSp+nJ;0q}ZU?=Hw#RYcJa93qimMaoJ&y(VSCTmriEcODHYIZheBfAAYcPiD{N0_w)>*B@W8U-yb3#d5_p z_Kwq9EwGTGP}(Np`;$~AR5{f0wEH~6WyUN$w;oDubUpg~rk&PMH@%Y0v}duYEu=Ce z;{J&{<>v7i(KFU#}xvaI5Z&p}fa3){Tun#q8j2OrI8=Q;yfJ-mAk%jr#o zOJ^+SXXkc&r%#};+UK?Zy&B1m(6T!7B!!OG%F`0#+qk!BugKwJNBnfW`84wRQT7Mk6|K z1I=n?w3O#b-UKSQBs~3bCY;fMJN@;u_Bp$KP zU|CR}S7>7_K*I!5m{mwR`MN!wU@Fss4KE2FI(fOResCW9K0yAPz1;MZ&qdkgqP3OF zFDF}3*N390rSypgj!)M#Ml8$v^f6a3GYNE3agHvfn!RFPJUa%yqf;n~`RT5Sss|^J zTAGqWqInNsViR4qQl29#o7tQ^+A4-{7@}&>Hg z_KOFnbBsED5DU(DN!)y&D^YRgGX=5wn6|FfhamA`@5^TCe#~kKR%Wz8UDii70YfXn z>~_ojp-}t@x@~Qi7ix689W>88q%b9a)=<$XtZw{Bej3`TPbf#E{Ys0NfY;J{8m-Gr zH#S7%MXV&5NJ-M`@6zp6&V-CaM-|E#F^8LnM<`W1U!7s|dqkt6NIL0!iI2JEjPx?O zUxl$%Z6b}uQiAcPdf%yRIJdC=Eer^z?0&=kD_z9egmmn{f$54l$pf8-`c;G z07)>qrOtn1n?o>+K`?|6(3c$Nsa2-W6D3pW)I6g;08e~aA4jhpJ zpPRskgyhVpwezzb%4Wd9WCsfzF;Y3 zflCcpYUnuWK$XQz?GQXKUfP+!dE5}MPP>puxQPLW2)NS=7B_^At)rNmHqce#^jk4C zOE?U>;x$h_KK@iGQ8{e_H-$#Q)V(=YM)W6n)6|zq|ggrvGnOO-HzctQ`Uv z(+ToFOY`r}|GV+u9VK{Ar~bc8@n3fS*SCP7Az%sK|6Vi*m{5AuOhO_}qWD-w!;KV= zqj1xZ8*Sd8I9C(+Jmbk#(hJ6oL{EfysKlaL)1_^(R=R86IIoT7c}Hoq)ABs_{0A- zEnCbwce?*M7C#cQKmRcFBjG(m`seEbt!#J5$lWfqzE6$$^W{!kNe!F-HtIPRwI}4~ z)~|n}mgB$rx9O0QRfqod2BnifoVzl~edj*MpqcHHQQKC|=iG?Va2?kd;^Yo@i%h1q z;%X%ZgtF`~nv(d?Ki=Pm8tD$w@h_3tQBIho!IQOt7%N1NV^vse$*n%*I&Va%&P)BR zTa#b6Ctt72j_^7PV}IG*T|-I~(sJR720pDO8G~)#M3qRzBY2OhJ8qXjM~Py-X_pdf zg`A=pyu-EDKJDnQd@EPa!z!t-3{%eQylJ_wxA(B!8|Q9`yyqNuSMs4{v{2mv0X#8Z znOpucuTNBUNenXWSxN~Kg`}3<>WLCweu%rr3x$@N!L1_5sS~#R2 ztsW7ukCt)O$5dibjvm}KlC~KMN2?vMSQ@=~9?lcpD~QmXuJjXVQM&EtBj=CX-!(js zf|^E->u*P)f>26(r9G?TQQEV=f|yYSFf^99H<7e)xZS6^m2+6Jbl$FyA0uHZb#mm0 za>h9i!A5Kw&;EW>(A@P;XK>C`Rz$Bazd==7!gCEFyD&5n`F&M1Gdz+<^R_}zi1{>X zr%k_Z>t=I_b@*;JEW4Bu=_*GsY12dRevmw>jYHYGHjFy&9**p+gnDx*sB=0U1q|XXX4z9oe2j^X0>3@%$~PU z?}=Bz4$(2=ZsC}TYJ|a@Hf(dmV0o+Msr4!{^mAg83GrBOHrmdyr^GXer}QbWN*SsU z^Po(YFjs0Yn)$Xy+#cF&8i-|)*R{DTMHUl?!XbGJR;M%WXam`nPErloH& zidIcLP z#DLA7ADTX)Z^Lo=b%w@RRru7WP6(@W?^M6CE&mt|j@E~-#ct~(y}DbV)U@QS58kvBFD!3$iKnM5(L!wO1$LuyAz^0{%)_qk$YrLifWu?7 zr-i@q_7W=Wr@f3d=}M)J)&qCeCMN6#%k7Mm6cvlL4c(jOI^)c#@2ix~C;5(-;@(|l zJrar&wnLRq_>5LLFV97*#+LGroiK$Qc^*4U zD6`I+agv@W@0|hd1`c)l#u87#)wh=!mm;=$m3T+Ku&W>KUZCYEjW+TbDOW@oRXNR{ z#PM-VpZUXcqs(GXYB*G>pps{u)G=z@_}TUB*}&uD*#M}pPx%j@<=!7=;}sE?RCMi< zRP+#mV!{t|)!LekaQ(X@CGi8UGE*;Fmx;D(s=DTO%HyK2jR>A6x>+Wlh>J)P-b1UD zOLgfVGsb(`%#+6Vce;!hU(x=o^8G=z>@4psUS#WTq>vNxAYm%=JwhYD6gEp1DqgR6B1dGy?n~Y#Z%Kki&d$6LtDwN>SeCq|_gFR8}UVOl;e3ojyx%Klt#AdK>mVn_5^5lBnsMXSEB#8nHS)MV`ht*qoHaJdj zEGy*gm7>0mKwhBN9EJ}l^7I`}9>zo)9UVwmtBJPuuG(0h5feo6OT{b)e(1rbX1^?S zo%1yu=oiA(7G~tN;b}a&Kk;bpJJOC-HScJv>jXvVG`A>r=3vXWgbp23m`T0n!-?7L zAG8*w;ALgsbcKedr^AXl^Q}gnM;-g-<0)KM?d(m3szaG1u{1QX5br7wX-+pHb!V0; zT1K)hY{C^60kS=`s%7je3SjR(#{aylRqLdv0AAo@~*N zn3?A)FN>dDV@MMl#|!Es!t|NMT-HQAREOr8GKCm$cHuZzC&Xx-tterTQClim?Z9oy zYP4`rCAU3@8)d{ZIy!h8}d&uUZPLQ-g(U-uJ5^ZRG#ahPgrN~7x?ct+oCYBSmPxj8hCVH?mi+WF7hbEFO zmyd`qf4&LhD_ASp&5vEw6YB`uGPI3-73oCF;O#NSA&z{OvCboR#N>gg$UE_(_m!ho zLg$uga_aZO7YbXQD6MN1D&IN3=E*b42u07|9Z{wotLp%*L?ASJ;G*R&^Epvv-mXz^ z%3)G^?n{LOsKY^TL?0|4&%bB-b|XaCd|R+|wfaV#jQ521Uc#jt4_Kzlj2p<}q%+#vxCbu+yf>O?wes}@N39EMl>Mh? z+g*obOMy#w`XZKO<-aDACi0froOu)=00(Sb;sCnk*RaO_{PiP9i@z* z-vU7Y?_3|IN$lu^J2$^=(T~y@bzr_Az8TwIpsn^E1!;LynAZ(DCrP~`#XPmuGmmWJ z`8H>^UKRpV+oq2e%rPGtboHMg=0rV|EO#!_ciHgK%s-hKaKSVPdKq)IuB^|rINAEWsyU6snsOHdaNQ@2osH#f_3Fj@`mJ(NPed~Qj9x(<(Dy7te5q6M8|}#O)6LA^j|( zYyPszP~k~wfthDjz4k%+C#}IU*ji68w(Piph!|;hN?JLnz2IZ_BqZ_0BGVfyp7-w2 zsgIxT2(z_09glIcItdPV|WwKeI*FiE$M z#1+0)W=Z?`uU;Kto%iG8kSa;fTn^@ZF|#$cd26t=yrkMA`*LVf-N_|AoX>uNsrK|v zU-%_F%UHU7#1{v39YYRwF*{pSW_nWAMDaL}Y$u`Z(}1xcqL>}U zA#;^>dno6~$f^QmVYs20E1Ja3T-$D6k}FdT_eL8V7L>9I_|~p+Gl6cJL3-hI0<*?z z6j8^#9G`hz$f3(a7!swkgcV5IVO$h(f9MnFu{@A#9!4+R<))dZ({p^d%qu?gLH1~v z7gklu=!q|Gm^&u^pjO=;JpquoR;2@+uu?wZbuS;n$u^p4381`skbloQk zM2&l{NuHScPlt%`jRb^n-JRFb;@bDYKq~S=oY9W!xsTsiTo+P zajP%9vqtx;NO~{vxLq^c#*bjk?Y7mqlX}xr6XwV`Sj}vUcyY+S-E6D4^3_)(;XG~x zRdypR$kM?0dBfAldx8co25JUZW_M++^#r3C51NM)Mm!sGCTmjlX$^&{Q1{@}gdV3P zun@kug$a_&jpb)4wAIIrJJNlQy}aub{g5+oz4r%b*o0lbd?h|ndn7l~D8{x?eNcCL zDPyI8=yK_1&|rDeg?=9iVaIjo@)`rO-s6~mwxl+T9gL67s#jOzh+_ZMadJuIY&f8pN9FVlXfQm5QFeIk!}_m6#4$hyX2y`m4h-WkiRCF=Ux4Epwo}*(0&r!(f)@Bk<*grNGXyuPibw#7Q?(q_3uas{(G+m*bz!=5LT{MFZ}|GewBsoQ@=``$`xM}n z#nGOhGRJ%{@W%0VjAOBE`TNlUWr;Ftetx@Pu z9@QB2&iHbFl@=@h;78h=O_5&BCz!EU2xCtXl5x(bo+^K)Y5&ra?Lbasne`CQbjIR` zzP>)Fle>&QI#7qvmM%bRwM zWrtT#bQj1|3vE1@!=m2PRCN3vcBuTF+F)+-a~)UtXWW0ImUiSw@Z@auG0tL0b-v1U z+gXO~8nkeX^^UB_upa3+3NacRaK>}BGRt{&OjIXwv8J(h0B;@*KU}ioT+yS&hiQC? zxk=+awyRoNWmS2HFaUa_X^@AlbsOy?|iNFc8;S4fYbPIf-u-u)Vwnc)5eeP4N z`iaDz&Zwow@apFwJzCoR3&BW{JqQ0z)#KLf{w*jPZG-tdCi-shI{aPjTvt$Ln3lV% z*LHSl;Pe*<(wPR^r+6jYmPsJh^1cYlUXFf-kI~mDlVXV1j&ZEAstNwX#f7e$qmq%C z@1~e{4Q!gsFOb)IDzLcZduGDmmW*B1;yvd6eu0upHwDgB{hCPIkt5xnKI|-Om=`P+ zH}EWE6v5a{^nns4zxouexc=jkK@wHx*Nq=mmd$EMQBS*w?B%tz2}2EOi|29hs5pO84d!^jD3>=WTg zg8gdZ$G%i^?{a6%-rz-2)e7svwZ$EqcZ7}iytnp>@|IweU9G;5ICB^-f2t7py_s2v`k$;Bq{nC^Gia17h=NLLe+JRk2DZ<-vv z9ils|ao?JQ5Hv{*aAyQ(9W2fxEz1%zp;}io`UP(8TA6@_#hY~#c^iqQMBm(`wQs_KJdN{PF64L`NV67Mtfu0 z(7S}#I2Tkwdy$6tP4vYdjRxzQ8Qy_lp>a^~t|#K{+Q|mb!dBh|1Kcc`^v7VboK|hY zVt%PaYs%xQ?|Ky-4U4weB}z=*THss?;i7~8P#U4LSOMlI>F!-6>7jFpx%>H*2ha7p zoG@aNLkUAqSF4x27#0!(>6Jv2Rou}-aBV4m6Z5k#RhnDmcz))ZU(GcFLtKd-Dc<%k zWPOLn!Xx&^wwt*p5-fSeG1qvyJ+x}TF2)HR7)_k(gg3ERK+-0iDxY`nvc3i5n4_zz zCBQO|pplv$@ZcSSbtG;nlSHa^fx@x9c#}r?;2( z_Tj2~WlwNdkL4;u-5ObhN7u5nE06w8?5{*m{hn=`;nA@O&Vmz%+8u~_mgfEF1Id{r zYh)Dt#0t*Qo!3!aI7muL9J>)3k^LztwyH{MLuNkW*oqg03PVoG@yR z+VRyB_a6`GOKR_V?6JO&4li6mJR0?hA$)8*&)kKQ;TNuYxV9R-%Cu2I&#YQS&~RIv z$z;c@03iuKUgMc*gB#4(AH5rGV8YJMzB}_-H8#^?u&l^owu|{HIbjjj=dw&U(AJ%= zmJ)EvM9z|RFme^!Nz9Bwa~Vju({sk6^zp0^OQC{t`OVbqr~1OI{P&`o~b-rsgx zSYRI3Cs3q6#))RvEXmbv;DXNUafi=am}&2=@rakk#%-3rZ}-r+vQh0*LM(yHCT+yP z7~Ly3YjVA(vcm!q5SOcFA(>_MxiK1)W6poV+p3IFz2Ouw3D?UVm!GYz|MO z8j+IwBQ|xXZE~Pe^9q;n-K)((pIDmQPo72+^Nw7pkA=`56uQ{KDl7?Y+R>x^=>F{y?BCEDxqqvbIUc;X<0MaQl9qbAwZ7!z1|cl6xok3MU&q4KyWB*$9LXoOP9u>l5TmjvqJ% zb1e_cpo+swya_H7_g3g318x#4#>~GLSX{EC6DizW&CN|DcU}31MlE>)_ft>`AhHJ>0gzC|rBcfuz-;Mv406sYSk4dTt&bOFS{@#TDC|Ld?pmXcZ z=UZ%luOIwd!IGA_0H`|__TfQ)lx|K_mLKfWe!?muF0#J`bpJGf_54WeWI`Mz{(7e@ zDu4)O|Gk+0uk%;yJ{8LJ;-_!@Hhuc1!5(~NGK@ML_m4GxA7TTpv()Q%Pccyb_LP4b z-l+pqihqTF`&Tp3l#-B+2V9?b`@8MqfhkSDHWmJ>nUa%%^}Jpx((-rPPtmZ{)8A)6 z|J_U=ZfU>r@!Mj$Y?Av>C|NE(db2_L~A0AF{|sQHZS;og!m@#0av9In92$S*zHjNkUR3 zy{*5HT|H9=P`>h?rO5+gm^Y{Hr-{19NE5n0tR~(NSRM3GQX1=(lKSCd(-mYp&z3*t zDy#fs+}dok5o*ZLv$HLl*A8F_tIF&qAf}@;GcT7%%L}1N5FEv`I04J4A){9fAgvsY z?9i0}O7PMyex&d0rJFCo^>3rjTo$Q6CQO`QdlZ-%t3+CoT=By%n0=3mYlyp}LU!ZT z$^`9iY7IPUz@f!5bLJw=- z#-qA#N#MjJhcZs6 z1V7G9Pv3V2xa0D@3Ezow(NZgSwm1=qefp{}NIPrk5x{RjMDihn-<~z@K}mZ`O9f8p z<1j!MsB&3Zni#01y8HAczQFWsVd$G@5DWAZx%OPm+&TU?jYEZcWo`Ynil6U|3>o>Z z1eT`A7_R77Ia#jJ)Y>h$g4P04Ll@9%wWFuv5-q!59!cq;m~lM`8d&lbfYv@IH2?b) z2QZ`#U`VnghS;~#D&&dob970d-1=YQYt^Oz7iZMG>Nww3roimoPr+g2v3Pg6J0;La z*nRUQ^S*kKmTsNvn7u+X1ekIi zmUP0l@syJ}oy^|bA1UupIFA9ehmd&}1NvyQeFad{*8OMVY^&IH5;lC=xyy$EE5~g} zqA8W4d3OAlJL7~2ew*s5l5lw2m)6oavNqsJ$v+j>GA&b`L#ugOn0>1koV zI1$aEbdU@Bq{H3MbG5h$Qz5*MXyFICx})G#;4v5*qE#-;Cqa^pv>nwUt^8YamtPv8u%xNww*dC$x?gx3gB}ZR=crdLZ z%5hP%J^-m3kFgkV8f;7FUZ9fTcLT_*VDW2keggpd!>!(A1HB*jLcT<>z*%oJD9TQrzszHbC%y-b_k?SOnmkI zJYC2&E-h)zm;_ZRVkw|_Jv4upiI zMys^!AD}Y-AhZ7)mi>RwzDpE}-bX~_zVU9EFbNUv2(>nHX=C=Uij2iZB0OhQXXQ$@ zK!yO@FGIkWV7w@eyf&JY)z{v-qF*$=bpd=fVSdi)>0DQWcwD9~6W_~szLrnTC6|Rt z7b*)+fOv4^XmRa^)(4%DQUtcNbyFwB`Q@H&-xbaWlFmySzFyFQ%xahwpn61+J&n_%Q3YaACd9IVDf|eckec-R_3#80S+Inlnd~6QShX) z#zfCm%eK%#0qk1M-Z+ml`GvEW85X3@HtZUU{&D5Lk~MMEi@PtriYpbcOPr4RnpN@& zW!2Q2k?Gi1wafmditN4J_Dls2oM($v@oZSzTtKYKB5PC zcmpVJ{&eo@#&ioIdP$9>3Wj!ZX=BgQ$x=zk^E%kPnydA;*s!Nf@qWV0nNht$_E^oO zp~AHZ*Lf-5<)=%&JC%W5e50^MEWKLX$ zF&$PaL@@34sY=Ck&1Xa#-mD77{Aoic|H~|QDzyU=&&Fmr%K8bT?2~OYL9Y6x7B5sK zcfL3;_BILZyuu_(t^hGOOs5qnO~4-$*2;k$aZsOvw7*uZDXmH1@WTQ@T>XJS>B2#k z$KCt)M;Ch1Rse^+*PsBpC3P@EeImBi!SC&-izy4s`o>5px&7g^^a-+_mmPcUNJCB6 z-=YSv9K5g^P}}d>?myHkw@sLJYWLW+Q45&AXoss3)F~Yu7X&#zxa62F&!%5x*_ZiI z>%iExls!6-kwY_w#%pXAri+P?*nD@1Nn)5{cfxHt>>KGe%*BlKPf$tA--LRS_+mHC z)@sfjeJw6ffwsG{M^wyb%{tBQN9;vX=;tF~>M?xA0K&0n9@e~~fzcb-ZvHj9Gi0>a zfNP|b8k54H9=!q6#d#gfM!R4n_rCIZU8}zYqqXU;c6aPs4(2RaKXor=M8e*`0=HC} z(7E2Tqb*wn$|jQS!fuIcZ!gh%xnv`i&AZ|)g~wh6X6MJ+ij^9;jKE!1E9c+x1Op;^ zn*1&R76@D;-{{}ub!+@=`>JK;R85Niu*B!BF=-&PvW^6V=%h;emjayAdg|OkQ8xvx z_B)ok;l)0KrDGA2yI%o_)l1Ql_ z+kGVr%PJ0CK9Hx|XNE~!In#TU@#HIozp$P4@zuG(9<8Aon#J8~`g2i@h0Wi6KpYNsvuh z1fLFTWE8jlVUzC(vE27$F3!#iwF+Pw8q2LvIJlmpL{nXw*P!BosXG_*_R(Z+VJ%k7 zb=3yEE6-`=`oMX+KU+NwalLOd@UqxXJ7KGSHlU=yCyKy!P7HvCb)7GQ7^8|2pYK7B z%t1ya99wgpTe;UXw2T~|w?%PVH_&ucu9Sr5PVPr)8(PW+UEWc*AvUU>~{ zvRZJ{~LBvcY7TKxO|8mik=d?4|cw z00a=P<9HB;f|bs;;Wc}<931rxy*A`d5v`@z@@WcVPBD*C+5Hpbz_F^$oT--m?1^X6 z4Zm4VGS3C60uU?4wvOmhw*}mlwR)y<>o>Na(%w_=@4K8IUWzGge0zx>dY9p41rq?+ zAVt~u9>OXbBV;@F^_lkiEpG14?IW#glB2?j=KYDThAs4*PiRckCbee%& z3D{^0>jR?41(tb{&vNGvmgp*$6x^P@U2f;C!scRKjQrG9*AJov`JNVo z*7>fFO|2u4gxdK;&qF9s?-7a1)GbI7cP-di9>UxdwzZ*@SeK3(MN<7?EvbxH*f-*C z_+IceTCdE=i>2`e;VbpYow}S>FOTH**^2tSfGh|O8JNcU=wNy^i?n5w;pt%mz%r>NjDCJJe^>Hoy>T#A zE(DgdI#QD5z5go+%|!`g31JZ#%hBWnl>F$^M{5^wuBS019~viwIyoi~vO^;mbEb2~ z0H^LU^HJfg7JO%Epd^UN<0s%BnU?#qBEoJw{2ng0#CBRFcowWbV(GZ8~ytfWljgKWJgyuVz?Ia@kho~QK9$npig&|u#Ol>b0Bv^ zb;j_wP{{3^#<*P|3qABp3dpYU-0S+L(#HhU5#e0M>|5YmeH7e7C2kWQ%iKv}IidaC z&w-jBb?F1Fb>YrVP(gdEIs6*ueopc-5Q%;nr4TsdbTCVj$NST=wB}xez`JXniK|Nk zx%{Z1AZk^Kp4$DX3!d9O3W%fq4ak#RX@r`9b7??@m`i#pm!ag={=t0GUc=bB`85Cu zPf+*)5Ow>Es*m3A-t{iO_l#74X~W1~kl+Nq@Y!&NKR;FY(~u zs2Y`8i(`4U_AE&^|0&e9@uTQV39>*6%5ZmKsCWElX{(NFCAa>7BUsd4gMw#{${WpH zc)w@mxj94Ukz4+}w1q0PQ{%fUqr!88Ye4!;?q)h@BKrsHNgHQLe{7#jKpuH|5+Y8f zkFMYH&!zMj*IRLabam+}><7; z`Ipxw)Z#(G%6_goPhFL8Rh1%1?YD8R3Hk}9f6!|Fb^lGe$xq1L=JR8W>pJLl;xVSh z3+S%mF2O%hI6x-kKgFST2Z-m-J!pouwDE_Xl{_w4@t1#JpL0quzdlEENV3KpQd?@t zd3PPx2(S=D_& z%f?VIPu18SU)@#NCHdE(?x|Uvspb^wEPKK39Wr{5T6c18Va)oX+W*b|{$CM;O@-`@ zP(152u=%S+mdM@!`JtDY>*xnm%C{uy!)RI3u=8P; z{g8fQO_!q|Nece%pNRZ)Qp5QnLWIr1^&w-7gd-hTCBeKhXyF z`~S&dv3&SR_U3jyVdztXkbBeGd;F78Drsrc$EI)o=;wgP^w~8w8q)EU)|%iPDPo%` zY4cO2z~_vA@@9V0kIvum^4lZC)YyzP%bkBCZ%U;tZ7OH__OD~nJXs)FTj~t4v3Q!4 zx}cJ@`AphW-Zbd1{isj-t;gVimyQr}o$<#rC4SAJ&6odb$a^1(NXKJxn}>%%jlr?P zM0OU^FvhT3e_c66mKXm*hS%?!A0p|=Z^)ua6J&4yqb>bMe>syY{phTlZ^8t|EI;v6 zA!F8NDsS?`^y0^V+n3+Y-7)YJMvR)#M;)0x+sI%>>q?s{ng;*XC|90TkgS!n7hg zhOQb8OIrE{fFGc!wtu|r$>Y{X4Ff@pc>R=f-QM2jLYZm$oVUT2ve)Hg8Uk;(N&BC2 zIrxzh=$_VZEKg(W|N8v8ilNxpJ#I!=g`lPPcd&(xW$3j*q{ohx^+1lML8norL&hm2 zv|ouEO*k#&1atw7@Z3cJ6(#^s`xImomU0X6@s3@^rxngTwfiPzAfPrn-2luAEsUo*=tvM3?&Cq8iV^(W?X9#sfUeM0bXs@~0CiTN3T)m1 z={Yt!*L-U`Sw=FY9XxFgn#PvfB_aBPE;CvI6ssm`R1V8BzAW$@CJKu(nrH77OhKI@ z&ed9i?SwLp`JVFENvY*T>Z5meoGOjz=A|~qE^a}G=QEQ6Z7pRY9U2}uO_8Ek2YFX zq)rYZDEU?a_Uz%n5E8CbXQWI&|4Y7dCwxz0{spaE zvHiPpqI;!L-t}Tz+HsUyVc)vP)IouDO|sHtI-%{9tfd4%`kFbpn{4!aCIXoI_r3Gj zo8#f}r5iwXMlm2e3{N%$t^oXn@XijvEDOQ6=d{n)oHX;--rf-}?Xp<8EE&;*dLjBB zosygk%Pq2MsScmM>vmtYX(Se9$u3xgTIOBbO${wMeYIs3y#^6qDed#h>;sq-8$eXr zoj5rjQjsD$Ee{u`c>}r=%A)6X9T&hV@}Tj26FY0Qgf8EXcUzerqEG&rDV6w!j>`{w4{g+vHfWR6#I@lua zHb{we#o5(b8u=V#NCGq$%6?iN4U`YtbSKLUW~<+?(*^i5Z_k0K({u%3GUo^-ZW|`2 zYLD0TwIttTOG?OoG9^vjMl*e;Ggjp+SYm1aBj1HB)aQ)Vd=3}ydjg}a09oITj5BC4=pr4mU9_JD+xAr);Elaq$^yQF zI}F@sBQvEV+kj-u42NEe_c8L0%y^n^zo7kcL$xBS08ks;Zka~d)gClMfD9&d?eI!= zm|npaw`>9hj6SwXsM-QgVS4oa0&gi`p%FW(m75nQq_77egL@XHYCBfd%VOtigc>7g zy&?pJ%6Oh9H5RHPXcwzGaxgtu#{3g8c{Lgfjf2TRO0Ud{X7nK)x2Z%dtkn9EZe@)>h}M#ud=)#11&_ zL4i52qi&r~O$|UqKyGAPhFnwY6KIZgC^2aYhFsYL_%{Lz6xj6MuQYWF!Qq)18M;FS zuoZv+<^%EE6tY>^)4Zar9t}_m1FCQGNJ7Y5IlwOk5ZL|`8Cb(3=uUR>d$MpKQ{QR7 zc0=+*|-IlQ200%x!g#PShd&R75OaJf(C=$>Hnl3%QHJfl3*h&+S2Ts9 zGBdSUG)B_FH`sq>&w@GF>7m!bcTNlLQ-b#V`&cZpEZ<(U&`^_>U$VA%mH-~N(o=&v zdv?oE#E{(tf8glO^I@yco-7X-0e|_%y(9~~_K61ykvB)a{~C7yuasbqfC~P7$kG^l zuF3@KwX#LjsO=W1C5Q(I9B{y)=B-6z+0_#o&rnKqm*0KPXq6Nov{a5QkhTR#x_p4O zI(-eNE(DuC8PyT!(Em)~pdJ};XcZVx2hI!HQ}EViQC82f>X6(WHbntCi_s8o-O*Go z0dQ4b544T?69FpqF=J4}Mp{h}v(eC_h5r7-;_m=Yz`w$j9fp;c;RltyOvAqd-DqqF z=DdcRE!=VfG>XGxdGuM?y$F}%R9Ik7bMlf#QMC{8@HqaBVO&bf1tNo`jg@ygB*-YM zr=Z=!JA81}f=G$Tf=~>t0Q3PKAo6$Uraguvp~Q3CJ^1ZLe~Y2lWGN+Potvf_QpN|s z>hR_CJFF1eI1BeZlLi6W@TXnjtS+puCp56FaSf!4Q%YS)o9*fH?%sW}M@+7rB&LS& z@WJTs$G-}wJF%7z3Z#S#=V<1d^<}A+GXeFMYvVQEe%^p0ea!T&mebQ{d0X~6j31av zq8xhedm;2-@t7V`drHqglq_9Mt2$XeVJ-fWtb(9lOl0z5GR^*s_>buaTOJu&vR+&w z1D%0T;>`0YMSnPNk!RTf_;9H%YpZ9V<;#-WY9}oO7bq{>{-3`Eu!4=@<4y88v;EpyZ}Fb^RA9$=+mc2K>j}zT7t>RTn2U0JkWxHYqKwbzq9#$ z<0?9>nnw&6);x{kT8v)Ymw872<$MleZ_H7ZIo%p~r>n>#dhhhTDsv=9pLT->gO7-Lmho~db*DsgKYuMC?;?8I41l_|uyae(73JS7!0Vv512AP?)fZU`r<7M(pf z&sqwkdM!On!^!4=lZA&lffkgsBhlU*Mo;i7k7D=BMJaIHq*h?Ce?6g1*8%>(&XQxJ z0qWLx#Gex=)-M!xEv#^I47B`+kd|CdHU~!LBEVFfjs`MM;Ir(n@|Z6HoN|k~Fy|Sq zW#rAE2K&^SASNlS(Er2UTfarwu2I8+ASHrGNs2yzv~-6e7?gCkG(&fXVjy79LyB}W zbR!Z{Gtw}04c(pJHP7Dr+57zt+Wxzz94n&`&>#>;`^V7prO z$!$Y@IjYX;`rwb_dsN?rhBfzIP7us}7^r)A@^T~1Y$WmI@qzCeg^o*v8ltukx^IA7 zNpkLe!hCcoPhtZtTDeqkQtTIyYNLIiHLeQ)FVqGMKZ&(jIG1E;^eVlfMWSnTL*^3B zrRs{M3q!!s=*Ss;y>ffJFD%1@KWmVE{0HW z1W9N$Fy_2Z%vVa(5u}o#4))+PUUz=RY|NrmnGvH74Yla#bBDsE*TE!o?R`$z9Auja z`laB{l3g}WEx@&Np4Uby=QPS|$Y4mFT{oCVGY~L>g`F1;FJ<#61H|en&YmTG8MCG8 z7ZJYsyo^GP3$j(^lsly_hB94sGR|M=uGB>5z9`%wXg^7=bI49u<5jD-jgOu~l`KB#WHRVf&+MZ}_Qu0^fVdk0x#Cg_Q6q{y`Wm-n# zfRNZitm)|y0M+t{rtVUvbv?n}*_#c_Yc*C3r?1ZsDw$@~p)Pdkd(2ju>$nT3yT?}% z2w(oBuWj{Wrs4-bdJIeJA5jNetA4TJEzSQR@rVjKvZuZG+*MyThgYX2ZBJ4~H`r`? zmS2mm_7SYF3V%}YkglFng+A!Gd2--Y3SB-yLGqMWx0U`ha>e^z($@%DANcxMo73cw zkY;P4xf3FCJHC4gEckrllX^q1Fp*8Zvb>CoA{s>krFGTir4SFix`hWDgwsguN!L1B zl({>C=zio)dif#xyof#V8gH+>y`rC=Z(ph#Rb26S#dL5t&kig>1f(s+nM*QJ4B3)B zD@WNK^?HDX1;kFS{(=&5&a7U6r{4c3Z_ z=wMgV$lz&;%)hA<`GH)=-_kANys_||xPpHzh=?ta(FiZH{%7VHZ;vx9vO^@K1zdEo zQ(UfMzmMY1Md7_t?8|}d$#($S&5^vNuck(B228yMpq?MMe2f(=a#FH_7Js}wrt$cD z$v|Yg;oPUi)`}#{_Wc?fo?p(^r`U?$vsl%xBLQOm_j8 z;*eqE!m2-nl#Nt5Q=6O`ctZWZo}^7RjAq9838kdGH|WS3R2xMrdg9XJ$u&%$tT>5W zlZM{#@-aKdR^_uv!0vnS%8qO2F{Gb}L@g*KGl`c*r(ZDTgEPJjM##>kp$@IhG4z{{mEV^S`QD!u zAAWRze@6LNt_|nz=v8sQ%6y(={<`{g3;esN(&w&9M3NecE9TxmMVRnI8yl=VvBzd6 zDSYf$a+%0Bm4#_AMX~wz`K}O+M$_usq=Q_e{GiAV1Uha`Q&i2{XL+4tRD$_# zxgn{LT8DJ;*_5U#g-x{z_Cdd-!cq=|469$jD?DAS1poE&2sF^$Bg5hSNKSc{9X8{h z*xX~85k$wZdTZk77n0H^@MN`Tx~lYS%}Qr))po^{g;{QB-K_D_aCm<`D_pad;dUl@=i}D%q@tWL@MEP1MZqfz=(x9k4e`p&6sgDAw-9@qt; zNzJ+R(8ZgR)q)RMkBdL9Gv8MZDPH$y)gJGYhMRslx&Ms*=iCy2mXO$a)zM;)zX6$` zweQJ_6$9C91jh&(qQI+x%k$Qw7vZ!fAdD-Y6ns))jCJ@ zjKR=fZxRri??p^}{zIWqSM5+I=6ERo$E&>Vx<2 z58VKdL+KM0d+33_ukw^u;Dwf4?d(03vnH5d8g^)@(ieecu|=;BsX3MqZ}O`KV}6g{kNX9}wuMLP5#T-&61U$4N3{*<>_50gkF_qo%$uv8 zZ6(n!ABFo@hSbxiZ{Z7lkw6nbER~Xi1%jICOdH_k(LCGnnk&oAiRPzk42gb9LouEo`j&>=B$m1dlq)_{oh?~yE@VH8KmPe!N)dh}dwYnI_PKb|cn}s!iyLnd z2Zsv!C)X-)RVk2mu-KJ%AsW?`$EJj~?0)Nm{PYh0-P$Sd@kq4--1`$G1pLodAGAt4 zhlU2UX0+b@6T0}2{OZ&sy{#G9sSrcu9W>i(VA*IE2J`26r9?f+p>*C8b+fG?gD#Bj&ZpU)&M*kr45%Te&pRP3 zF={>G)-DF}1FOTu-LhTBLK%?IPFX705M5W~Ov}7apVDu;<{lo}tPJk^ko#%Gh1PDm zFW*-2T^oXfnNhM8Ve-2{Q^xqw$dCrBHr8neXqQ<&|ao?;+wAF==mDZeyg;<>yx9mC=k z18Z&@#fI9@YKu8-2hIE}m4fob-cteZsml9q9?jQ4+Pgj}DZ#Zi7NSquX2&i1(uLxm zG5r%#;@yv!N96ZulD2d5higk8rw-(f4X=wX$cdV7^kecON)`=hm95^3)Ll* zY?eB(x^XuT!(5F5uXo->+6MTRS8do;f4T_$7pQqm8t8+!=?5LVqkJ60yk@DBR;KU0 zsH?%5?Ud9X^XcibhMn<+_~&GGR@KcmK)DHBE|-C@Qw;1pZ1~+T2hQAnkHVO^$tF8m z*>dOYJ3|>knQb$OVW|q^(~WWobuPmpg^|-KJ5fJJiK5n$R=z*ZPU7~ROB;yDq&J8{ zF`MQV9EpXD#|1aseUWaqWwh%JW6@_T3D#@W3DHRFVaaQOp9L~zZy8biZa7{F#Jc{t zqtoqO(m`Oi{-=Z(_4UDODh(U+lK~0gM-Z(ltNO6GA$SG{HN>VouPDrlI`8?Lj~Z3B zWVtFCAp==j@X^7Qsbz7NHti;hvRZg6$6-&e`xjJ}f^C8*MwnID+8rvl9(|eBuwQiP ze?&~2*xT%l&}!F2Lb;{H&V|1Q*B|kTNiJMj8!jG~I5}aU=Dnas7TE_UHZ7HW7yi*`}52gc2{*Ac^Zng_OQn$=Gy z)E;yEVT65ScyFP~N0dA=E>LMZ@L8pUDGKJc*At#CiqyY!(2RG1DiLd#V63Fh58AU@ zfwaGq*dOT6{0t#+JZd5ok=5(@dP;^E9;D3tZ|3Ev?}|jRxBqIaF&dxT?@KtZfzf;V ztzJ=&*NsylGs$ru-Tok8_v3YJIpO5ui8eiwh<6g#kY>-jQ>RTk?n{aC5bX8^7GEp> z;GZVUDrcY=-sZJr6pE zyKha!>0`5gM?(Tkgk&ry+Nm>}??%wtnFAl|iS^uaohlJ-dY^?HNz(~|-jM7iC}etc zm{&SCx{xfJDw+)BwyImj{xrVF8Q!me@jYiylQw?j*OfEB|L%yAzoRW_jXJ+KGO9sV zx2$uURoD{ADGIwagel|Ng`x_<|J@YCz}nWo#Ul9V(9fJpx7<GOlYtx@4h%W+HCPrdWI2pR=y{q zv$wrRQu>mQ;CJh(RLWiaZ?sdi60jP`WO3cy_(O^O&im1}ExiM);b{_O_Nrc^O3o6n z(p?Uj!I$A{q=?}meurs4B{JOaqpmBE?OpA1yXFu~VFO3;$5BtC@?1lKzs}S7*H|q_ zV$aQBrmI*;pB2;Sb4{m*6*EQmmKqlTK~Gv^1|iUrTC;ooSNCQli2t|i=7+vnxE_D2 zS6H2JW(`@+S|?*vg&p$+dJ9{~8n;@aUW2z7B45Zzb9l}_M3$TChVR2e!)zjRt%wp2 zedX05$oheGDkledWXX>LOhMYwKF~8hLu8({&v!Tcwa-MnbrN;>cPh;kGW}136lPZLY=AGX$jlWhq_y zK{ZE`bk7siqg@;&h_pvYQC;H*A- z)JC;JdJMt}Rr`J|%Q?qC`y+q09J|@PrJTW5u6GzrJ)3Q|07`p9T52e$;LScEr{cRS zKk$FQmfz-hpLWT9w7gq@{iypkA<1afocayOY7l%4Q77_}GaNXYIk~P*9ixt8ShcIH zlDhSdpxt`nJ?88e;~FsdCR_3OelGSrU7N3Mqde%XF@70Zcq>P%o>T{osY zgR0^S1~ z*u6a@8ejKEX7AR_w!`zCp|A3t;ySRM^sapr4V>`_n^O$0`(@Any>6J0y%uhyws|Sz zT2q#7vl_)cq?PO_IIjvV&g%_oo*P!V0tYy`0vF7=^{yQ~@OAI`B%%dvXSNC7s3kHb z7H0=GHg~5J4>z;l$h`5@ZY^Beux(-M-B~tO)bw1)X zXYGE2)q@P!PGZ+clBI6z-63PsC`Bh{(Ta~BZ?s|NBY-T*Q@@jHgmvXAr4f+c)h~(X zmYBAMXXQ2z={tRA6fpUQdtp787gx92D|hbf*7n*f08_7ceY>DMT1AcaIU;1c zVk>ztAGc*-HF@TQogQVDw+0|8bHJF`?o@wNR7mauV1bH9kaLi`UjsSxa)3?`qlF4X zdd^{RmHPQ;U0Yy(;^&WD0oWE^Cvg9BQ#6S_9psMZkIs~woJxpFayOhn>d~f^0dRvC z344T5G$(VP{F7~xl9JY8g=MFJNl*@u=sTv#s}k+GHbVvHVmAJ9S&Mor2?>DH;WnVI zd9;J(E1c})&yQ*pB6gC-i?_XezvqzUKizngzx;wLzavIGPpx_MoH%OIceI1fGew>D zygW*{v`qbI*DN_vcP-*vV}8XzW8uAq#val=Z?wNjV=IZ%FS*2m#?&k7?x(qmLWedp zjXgMq#vY0&-zB&1OyCIzmG(fYy-pcRaNekWvQFHcYegj%RcGsU6HoR13e^3MdlFr% z%tn=vYKI%?em5_E8l(M97Lf`#Y(I8o^lPuwjRw+rO0J*ki41NdEqgR_H$98s$lD;19BT*A=)mLzM z;iArHN!3C-`y#;vxxiVfj5h;O%a`(HzAp5@Sqir_Y18gZZ z3iPaC7VvUuBJuw9h6_J>c8w`b3>^jGJa&~e!yGjh`db>*Fg7ppwXHV#jg^7=<%<0V zpqK@IO1)p6NFWc;lGG+!?C@7a^hx? z)E~u3558+xA|J zuzjZfLh$CVeHH?PC?0n#F(3sUR9|sYJ%?#*bbaYuI&U#PSraWsCh+r~+IV!FM62EY zzF#oWt1~3gTBlep&bFSw%4ZPTyubWBIHO^K=cT)%J+-~~YK(Uv(|~SV!JCJyKto>& z^vweySC9Pn*DovPEU(TUN?*T!9iupP_%8XXNrXOzcbL9uSTVC(STUn@KczQ?_l5~t zZb{Ah`F4sL#`O5)RvpxMBr(wp+10Z#2U}kqb6Y={VDv`$Cp*)>)->>BB!okelB_1# z`(0G`pKCD6?QtT)%%{9hidx4F&~6bWti>k#N+FJ zeV$_n#2b6JTo@z&@_CMw@1!#R3z>I(s-0mm&{Eq|-nRK|RoZS=98-QUYA<+0o6&K- zmhddTF_Ks;SHS#CZC!-{s%?(!S`+NUb%#ZthlH-^4s8ir>} zXXfi9+c8W&lfu9z{pm}}8?0ruCg`4((Z(4p#*a(NXuJRW*7w)$GX+GPuRkS^-B$E= zN_!iy|L)MMUl*HQ{EU5Vw1+Bp|8SC5Ju|u4J^pEmmOlmVF{M{bst4$Rc`Kpgy_FbN zdCyNq@2pi>ZLC%b8~nsTn(6+)I&fJ^m@xIrQK%TT&fS#}f>O(8#Ic@#!hVDG>8hMO z|Gmy4MOG(moTz4CSKyM<%V^6OV2-rqB8Wqe0=_ODvI?vZSBVr%wU$=n>P{0N_Qfsa zp+HhDNGRTQB8EDs-RTJh#!o!ro$i1mr*M{H9c3{q;AT2(nsiy-)pq|ShO~jISyw%Y zL|+kn6rU}RcvY0co5@KazkFkrmqBD&v`6TAi1R9?A5>PxUE^k$sIjO(8;!HPh^&1b zzv%_tz+i<>UuJkRrRv1-tqh-I^_=pvz4bPejQlI#`jgvE>$O^lX9-6rTdUMB_-A9z z6yM^xi;?G}3MhN3X2j(3XWUa~zC7h(mfUkHT^7@Q*Kl6nU#J1o92-NVt4h1IxU!@K zVB+8X*~`6VOxJ=bGdb~^0xGs{$y)~J3#m6J9E75u#|A{Q7rH= z@XmNT3L-Ynm-$z5x``23DKC=L_DHU{FZD8Ff67eu^U|5WCc6550kCN`i4i@*7BaUq z)3p$2l15&&A?tARK_m{!h0h$w2 zG3L}+T6mQsk)Ji*aiNlUv!vmrK!}0Fa#mU|x9OopZPxO<7EB-0|A8l_VNM0!h^ILZ z@^gvj*B5xjlYR}7Y2^rrK)=uIzOaCK-Gbx~zpmleambd!i+Ul#xUpv|aD8~cV?%4? zN%Ff9!zvD)uv1|_@98khvVM~vc420wPBMtG^}}G3@@u?pi@{1^afAB&A`PDI@@H1h zAIh|BaEa1VA9B*o2Kg)#eUGpY~Z{^2$rEfKg?-wST zHcPxx!?NrPapceUoIBi@`PPP`AzEc~B0N*MM_1^kGa%>>^W>2f3}lLni@AhRX{_;snK2YDN)c|LTHLX%j=IARXd0_7E8Wq#$7gR)hGBs# zc?NrHyUxW?RR~y;Z??~1x zthsaXLhO!^xwS6$nSb;Bp}{)E8xz?O^2}d1>@%*wYDQdP=bme*@qv&grjH=@h$XFr~!&?Vsb{%p21=Z0U>GY?9AS8e|!Zj2t+9#80RE`eTayBVHs1jHyF zY^V-9n?g--R;m?3Ea}cxBCt6I3)b?A9~|rv#8)8V78Vg{DcWsH$HGiXiF%0Io_+^R zjwL;NA+n(_zgo8cHSN_f2}w44Cau0}*`($}bKqh1i3j!mU#1|o;|GU}Kde3kq2Pk1 znQn=ushn(c&~&F!BGZN}pS948>r*r+J^S#%Y|}Q%{b0DRovrt~;)3j`9mjTyKDW3D z{trXUYe zCE_ zwEEt-)8hbsl>)ZEuY)%qxl1CMPx*emE}1yUQR4*FdSGeTYTq zk;tBcp{s!q9Fxn>*YxVmJ*y<)g#P!K37rXj0>21V{}aQ$y`SQ0>S7W+fy38o4fRQ! z1i6aOI{PnTb+#HzEGN&m4X&PVo|-H^9obDyPv|srZX8qo-A|^xB;b46h=fFAn@Saq z>tKx&NUw*kObV6GcvgRcy872spBd|I_D*p7^;Vj%Jraf_ILA7VWc+r%r$`m!f*dIih1_H{r=46$r^e=K z{@B#=m|^6bC7;f`H&o?MbPDTh51QE=mp@Ur={D144S6k>Y<`<;ZPDMj;O+7AJD}us zbkQewo$$gh{o-7aM9Nw4+k1cZ5HpNwvjE{6s+EV_mMN{-Tj_X>C)pBJNJ~ahUuxrX z(pj^otDTh2taQY0aJC3@`pIyNTs4!tjW=M&rAkI&CSiW8>`x=c7862UBYoM6 zmYDw9WLmLc-%e$;F=Do*&^}zVzDlz7 zIB(p`yJrz!$O7_*NlTh0mHYh@v!IvfBJD;pC7iIThVr1{{&7F6PD?GBv3#a+76SRW{B0D7^k{xLs0PF&I?nu! zKt8gy^82jG%$%X_;I;CUIx9)lS3>4t-X?=?`nm{%Dz@MMKF5<3<>V&l{p+!Wt5ur& zrrN$2`JYYPcfbAMw*LCKdHlBIc0yW6jf~&>)ui8TiO&`=6-YKsYFi)FJc<}0=XADn zZ_vUP5(F9d-`@Z}DNO1UcI~a*4VXW-?r;SU`Q{k=Z16_GuLC!*crgBj_=2%P;#@%8 z*3QAL_C!+H&*(m0kn##K(!h4qX8AzVrO#EA>Zg+(siw}Cii#aJF-aX6HcwO z>*~#wgR!a?-&eilNT&4W68?BGd`tzp%^;qXq5D6;o|8UhO0QdV33Utg`PaMJ-`icF z%5}@=fC|^#a}ePe{N=|`#u0=Kp+yk{`R+hE{oYswaGbV)}mF`)@t@`t$;lE5{d?u^vB1$VL2E; zk#}Bq?VTHBG%6l?#-WVS!`~zPr>HCaE@BnKJ~IxLgFCj*E+cxP7lfH$li>WP4dAJA z%E)B{#GtmvMr_3_X8NK4kjE|ye;?+tM!eK@Eka!yegrz{LRJkUpLb4PATB0d-Siue$}2* z-1wve#ND8dfVkUhnQHbAXTaal9Q565=kI@dI=Sn2I|3-i0_zU!Sz6a0@#1-0v=;7_A&#w{hFO{}d$!0f#nIM`V{iiXe^HJ)@mD*be8=gj`Rs$J~ zo;6$ww;F|;t4vjYd%3F2BjmUKA1Il5WgJ|>hbH0*GsH>>(-}kyyU^LUSehjYR zmHjcUh4_J@g7id>urk2eP2m0Gy?&PZfm548vu zCGxS5zOxIayu0S+UMqMSo}5=}lYTWxJt3x|SMvuw=W% z+q3>MB1L?}dU|^byVgJXG8^WEzV64mlio(mggfP<*nU9P=K)|O)0uB?iWfhJwt|8U znkNqk#o7QqOAG53&~;fKt1_?w_I{6blw6w(q-{+BZnFi_j1PgGlS#C;IZIX{+MBBs z_{N$i`|bxV*uRaV`udsWSQ{w-pvNc=;Lp=*O9KU>hV)rDwKG7PXj~BY%ksZ2{kkzC zYo(xohQ9tnWjIuk7f#loNJnQFcAP?5Z9@;$GRl8h9u5MLj5igdF3^+3vy@#xj9{U!v>an{3t0R@U4+610h1 zg75xF@!Utb);w5`&ZY?Nfz~;Ac!jgVYNK_59L3JFBcPFc3zxhQgGe6u8Z;cvgcbu` zaHbU~k&-9kS)EJ-k|aXg?RRDYa2H1FHf&tIq_8JD;LFt-Mgllrclyge!-+g zy2_(uKiQRH2?8^!N;G0<8cZng+8=l?vQUZxb=qw8N+`4)jqu5}B@C~JIR10q04=obp)rCdPvTX?u)c%id$ z8YV3L?`P^CUNK)Z-#-B=XS@&_1FQ5Xlb2)~kD-Y?W>3YozTMaYnZt~ZWim}YZV8Yb zq!e;6dPgJ7L+5umO(|fj6)#Jv_}=!{b2dc_`)NiHMPoU{JlqXLUpUQ%>>5fZ16MJj zyCah1OVuj8wpB|4Bg6v{FLZ%Do^&4YY$^u&=?vo+G5T&Jd|hh~2slVX*-IYhy$U7;Kl~oy2FpLPJ5Z6al#YTVx5~G zouh4EH6OhQ43kfRqnFYo-i(&rx7XJlN6ccdrEz&ldDOoFY~4%;=PXru`Xu?{lo?bV z;O#DId*KGu%*O1nE}j96O#?v1u%cF*s0UTx9FLbW+d!*a2d02}UBK>^8!oonj{y{z63~kifKl)k$j{@PeOCU^ zwT-rZ-8S|x4rw;OYfQCbGK@mI*%kmV>F3=xK_-JQ{R<81X7Vd0-u5Yqau|RTKBcRm zW@=)zKfsk{fbGHKm4!+dP@oF?(~AT31D2tnmq1^Aq{gpM8DZ~6gRXf}%5nT%*kL8V zB7+(u)?jjw80BC`Uk~haU7Q;M;~;4w!#XZ!6HEv$AR`NR2`AZhsx1H=;=(R;a8m+$ zW|HO911I3DyjAVC79VZuDL-i7y=Dm<`_>nH@_!i=O8vks<%~}v9sV#tF~&%~I#$zI zm6)GS;K1piIBZRE>$M(umf4SToNpRV=ebz4PJu#y7F_?yG`Wosa3P!pIJr_21q1z# z_-5l}t9i6{6IdPuaVfK%&{8bbGS(Yt<3rN9jGJud(ue3Qc&5W9kiWhLQ-Fh`)7i>% zpEH#OgRv0S=yZOBiU3}MnT;PL%)`u*fznkz|+9K5(U`jx-`4HbRM#nmv^dm{UD)&>edtnYAh+`05V1<=ymb9;& z4q@psKY7v3H~8`lVzyKPQA zu#W969XB~Ax_<#CJM?(G!hVu;u0%3n7G&?OM=g?syNf1r3#zT>3+{arB*!{W7*aE` zVDO!num3rcg^s6FUSmDAe!b8IF)BQp{iq3b$L-JK^0PO<=)u-C#(eSq#RpA8Uso}( zL`4}hsMo65^zik`XMc=DOuH_Z+c3AR!D}p5fX%2;`OyWMq^Tmr59CzuvS&;R`A9!F#aOzionTqrU>7{4( zU>SHaQ27D|yyM*MhOB)Q_M8Qb^3_ zK@NOzNk8+hIzUby1}LTT7)yk`k1_gmE7A06vk74DC=HA!uw6Co@>J;x3rMn%^9Q|? zO_K1`l1hD0Ey9`i{$S;Y0Bakn`@$&tqy+n`lU<%xzh4^*f+*T0#s}Z|?G$L{W$fp^ z!4kp|qOg65Cm+wpqMkzs3_G&le#+6slxBUZ1Qy_3pI%!JqX$px>3lYu!0yG>ThydB zZ;o+G_xYjj1No3E1^L{&P6N{sHaa21k;X1~5jU6P-s-Z-Ta+rtJ@I(-_`3uIDm!*a z6zhjG4%9!Q^L!q!f%MOkpZ73_XPEi1W{qT`h5Algc~o(T20%o~KCW^yxnAFH_t2;T zWTuML0uvHrUM#9HD!uSsToO_;}>|sU`sb1OVbK*f_RB#7Iz4_2-A3Rx4I+pv|c4eG! zwe;o4)CfFo=U~N&P6#`^sOcpiD~I5)QEgP+FLu@`qDkjGVC<9uQ+a2lhfw zSxs$(6V|I2eN|z}*?;v=saNs#dSr0JC)0bQ@n`aGSC+e$*^P@pl5+<0uKR4WKTnd@ z&4J9de^{b1)3VzC;E>2riC5}5!K2=zcKgO+Nnrv3pWlLK+Y98FTH^-JwCBK1g_{0q z#V0Dnb6m3y-MTv3f^**!kOW5Fgh zEJvLxDql^fmog*neK>3&JDO*C9?C}(!@7{w)1pDyv?B_&KjQG=W2we zuz8E!6Ip6VWKym-aJNZs+)RB%_cW@x)^Jk5Wf4J|1C!+2Pn31#;+T}+mKP{cd$i+Y z5;E7Cn3^kBhBo#;oVnS8)h++V|36*;r8heKc3)vJAfh&Z-y*4{L~Fu-ZCnUo-IZd1}_j8Qo7?j&6tG+v(? zMlLf+1#~()+zj_bLMjvpRJG-%zCUZIH5X|fAQ|qZ+>SPmwYqv)-l2NL9GY%qJ0GQa z4U?B@dJ3>AuS0E1o_GziFx-Y`a(v-l<%KJW13G4JIamoMNrm7b7-R7;^FA6S;Ho9| znHLDE%Tqq9ne;np0YOO;t2^E)ZZ0}G&qolrVwIT~{a|XfDsQA@@2Cd`ab?D5yL zF<@%j3-efk8AOkRGU=-P+u$`O9DDsLMfZM|Mo)!OSM&Q;G4JU2(A1Rd(S<4)n!)E= zGx)sIxf#=t&lX*^K+RZ@k&|p*4P_oxDRwy?#dKqFTZZ3Wqj;>)ut5tP6@on}uR!O2 z12+MrDfjKPa3;Akj?D_vMtb8d^~AY&Kn}zpv*KzXZHxq`#$r-K4$N$I>0;kxbE=6& z>uge#_x$a%h_~vjE+EFC8xRkTo`&GH;}YfyM4M_&+CD~&Enzi^okTX~7lfro;tk;% zjR;Je0S_HjS2O(~_3_g@N$T6e-t`axdN`eVGCr-Wh;G6AjXdaE-H`M2pz2hrm2`aY z*f3NV$r#DIK5Hbimj_Jg3J_CHZdOwPq0*vIRu00Uep_>!&JHI*9OJ(Cl`ZN+8 zniZ~-(O=_A?0~6=K6#tn5bCNcvA^!_$?cl`Isv8-n-NU4EzY7wOh+I@R=Y+vUUD#8XJd=e!xA$88@w7pf~E7Yu|^7IX1a$h z5P=FA8z%{WbMh?sQd{ibKPuJ`!}XhW6`WTZcQNxb{d4&yZz5J2Ede0fyINaUpdMh_ z$^DxP|0^fagUP`(l~|@fL^_VMJN6Va?GElv|!iDbyeB1 z3vR^owojY1Ff|s>#g6{|YoyO^eFop7=|2}I+8r(@#)vQ@5PrWN@~htcI|7yp#w~3R za4!fKH$6)d6~ukdW+j0U&d}ETher#*Z1^|66#bgpY1kjLbZn9{H8d|PZC5e9uT z)IA7>WeYd#54bgYNP{n$#he_7+#4;k5ep%3 zwd(>l$iptRy*4%bbr|=rb_-m_?zlZ=HZ=5a)O3LD&<;l6U@Ij1=Sym**kT&s3ar){ zc9SStX<$Ssz_x1~f0t8}3M9F1seAj(_PyP|fzzo&yDkgD5(&-Tdzd?Jt2b$=|JW&W z6nAq2DBg}oi%K@UR{ceC`@b3Gz}sYZa~a}8cPWQCetU0!pCWEyT%HJuP52$j;a`)_ z(hq!cLGsG}Z^z%?Q~B=W#(;`jOcuW%s_2i#8-h`o8ox=J?@tu z_|HEA7bxycLC#H$i2n1J|F+Hl?_FRJ)>RJz6DOuWx6={!RS)O|Hlj| zQcG=20La*&P20IaB_k?#+x{%&t!}}|xx~!Ymz_SCZg*tg-DRzLbA!2cbGmsJfPZW| zX|wD9O%O(YyfnmV=)0#5j+ohz)eyt~W9A3Sl7UffG{IGWc$-|`26y0vFmaiWoll@R&0f}POIS1w*_qTzsN+0EN68{%oXG=JF$0C1g7Md zV&l@H18OZ$CBqaLH-@4SJ)l2Hu)e@%N75e{xv3|$?q6(9;R=lY)DY0aOj zbEf@w6O;Oc2h^*!MNLVIcKP{4!_;R)_WD?!ic8j-isi&i^hLRH|UH z3UsvxcQx#HvNj8lpeHs-rs2Z9F994bp;MKPcZ2C0FpQ(6hLkFMZ2)Lm;XeEHNc)F7 zY^Y9OZrA1hsazm&+XICuS^*tW!{n(L*wTANc+Nsih3?4r6d}Q1V`C-w4&wp zx$)+ng9jB2XB_88aQ`dQHmIIN)nV~&dDqkF3GWSsro%uSI!aJy!ei6>TLYeU=U7QH z^@%952IvaXKw0=ELRuQ;@Oc3GQ2?QonW_E6-hapQ-FwoQr?gP-H79UenNZcEiT6WP zy!}st5 z%bMO8IIHn%h>)z4m|Pzo=}?;b{&w>M4fuh?1habX?V|5Y&~N0rwjXj7E&$e-dCy_2 z(vf8|-5OdB(400fC7_s11^P7^2Tm8fK&#aP21Lw+$++4TbfMg(1T+R8oTHKR$fC3u z^3@c`pk!sl$kx@$RE5VtB4ilU66rl^Xulo$%?$~mK&M1DF^JVm)i?}+oQ@j|@(lIO z9lM&f=$z5)%FKLNk4ScJ)R5ZM8>fC?!f|)9!@d+j-v*9Sxm8KuDy&}4;ReidP0oGb ze3&VP_6}Ui2w!E20Nu{)d+Xb#^R7QY6%?cP(e*1IE0tP zzUh1F4SBDiT2+eE_TIA8Gu0>ks!9s}D0<*>v?LPHU{!_T0y-RO{EiVv zDdNG_LBoZ(ejAz!zeM$X5s_+DN>@;L1U*3~90b*Qxb9$cTa);Ua)EhL3~=X6YFq7) z>n|!9B3r|z-i^wX@eExXd)~=UfLt7}arXfz^cGlA(v_QlS?z9V(ip_iS$Ig;XD~p? zZZ>`)QdRMZ!ReXeg(nuERqhqKD13GE=x$f=Noovh@Ud&!>p~~BfKtb&`hEgb`eq_7 zy&bUxz~xOq(T6R-C%>cb?6}sb@a5h4w~{fjcgzgpLykUM=PeY52{{S6W!7Y97z?cl zZR)Nad(qKuYWvDY&h~sK`QnS8ec6Dtb}*G%bg4 zU8|&SfAa)86dRX^_Bn6~enL@*1;J|58#BM}T)$`0={VgM7`IZH*rWh zICjIaLa%ZuTDFH>(yO!x3y5hfU&gsIvOBhc73Aa5=rEy{?Hs}*jb zuL`qPV;wvq4uIaFBuR9Ln)*n@gsyNbv|7~M3rD9skBOP7c?fYZO+wh=A@Sz1E`AY* z&tAHLiZB_I*ek! zb83NSV$%?SROx~1sFMz(osWn5xIm6wA1I~sPr!RuEv4FqBIw}*O*_s+AxD#AQQ!9W z43401NIFyn4#x!s0>|-(k}$|U5+cDgp2S;+z$`ecZewEm&5cys+cAP zmn)4Urvz&Zw1J$bZ*o^p_xj_VL^#*5QuyT|fyzm+K`z0r*on=Lz6pqh3*ed-f8NTX zik@~RqkcNT}?S>@``|zQx*thV?(UxZ*vpWRZ2&<(tcC> zsyra-bGYd;{k26cH7t~o(+$^}ayf2Ul69k`wS|4)tJ;AIS^K_orp1o>SrSKEiI3xT zOEySgUP-Czik1HwEIF=+f}+#MEG*P61gSpI?D1UR&D3`tP`64v z_1pHAZ`64_>mDciU0ZTxHkd$Sb*jgOs^QfsQcC?E2Aj0bj%e-7Teb{q?QrZ1fzA6j z{CpliWZVV;OVavhl=NRzf=2i9%Ug?c?X)Z>W^*$j7Ax}^*I7vxxtbMtC8Kb6-^@x$ z9tvjqp6rj@_S5`lusutez~tAjap>bS9Ed~GR z2|2%h?xe~gn4$9FDK$tA>u~w+@Fqebc*c10Fkj{|MzMur9=?L3T)R*|r?vNC$Dhpe z4v7TKOX%qOTtI=>FE*iFA<(B)4AG78)CNnUX=K;yNxt$(O6XlS4q!-j2H-LS_SB2v zvHngAO0{Rs;;zkvf?pJPFVG`(!R#9k-OFY_2N;KXzaD{cS2t>euu@NuyM#1fDOuwF z)H|-UM~;^r-Duy^wiN(NWR`o(l>ZBGfq~V>BtFls4EME&&9c3#%IShDRe8c!V>#l| zr_k3z1)pGiKl3$2_LA(~79M$B+u{~hHh+R868GYgtDQSCmfyxpw6mIJn3^lMTd*0Y zRo%9t?9Zw?i*4-&Eb8YIG-@%maXZH6v}eVpYfURBR_*M*wvRoFCa{orJf*lcN(CkZ zM)L!*=8ulA#HVjfT6}SQB_t@miGf9bg_vh25lVV^>o08do(&NO})k$)vDp`KS>mm%z4)a0Y%*7q$h(twQIq569;k*(!O zQG3-&Gn9y>q*cs^vbi|*IEKg&*W;)tm`J)(G+B!OMDQ=&g6fS^x}7G?fmR$2HBA&! zpmT0NX6QwAtOMs(Di!>^=fQ5y|Hs~2$3?lVZ{vbs5H=zLf?GvN5g59Zjclc3Kw=1y z9=bae+$yD%%#b2A3=K1Mh%`eF-6=VA$Ghh26Q1)u>hJgG`+2|r_(7O?Vm?qFUDPdQj~Y)Z1#<6b9#r`qsj3@ zIdQw(Ougqhnu9WQvL6KU3DmAdlL%yAs z3-Dcd$>{e9TEO=>nPM6LuN?x1hVrP%s5_?Dn!i8xu}u}4Rd zdyI+r>V8A$3$+Xr=SKUNC#Bx68j$Zdy{0=@woMgw4loHie!~f3Hd0HnVDrF8Jd@;t+E|iQ}}@JEe#PrmTp(r$&+E zkCn45cHjxX8+h*eoeY=h>whLDy9RR#bmbif^4HB?P$SJ}cz8b^X0!BFsYCRB)RLW8 zt)0%KEp;vAs-7q+z~ZLg=%Sndf+#AfSbZwMfYc!CxCk==A>fm3hx4>+8)RRV>bz5u z{=QI^5CgCt`mt~JzK;^VuDjXM(>?=Q-H*Vc>g)J#5jWqtjGPa8SB%2@QVD&nWov6p zA1F#|%1pb(H$%!w=^P#Zu6=R0k}CbFQiT5tGMgcTW@*iyRzMEA#%HRuWk}&$bgvo4 zg>3L=BnivpBj35;XNZ$OBTD&((#lWDPbcoIY;(Z%TM5%ORM(kathC$_S4-y_Guq(_ z*q#`gI+*BJ80wb1g`TEUgVMKIPP3efn{Il2NoR-Gfv3jBC|gs*2`k)u*~_@iZCdKh ztzg051ij~&n0Raw6}fS=HOL1#*PIi1gQjy5gl2R)lNzEv(IGqm4QFmU?x?#vt!nnnI}wy5(edE&M$XP^%z zAeG)!5L{@NKVRg{f>cGd#fc{;6rZ)cmf6MN_WHSMUf5^Z!1GLac~wq~b1^ZjT7ORi z^bNR@HaFmJ(qvq-Ye)TsIzfkEmsyo2IuzbSo#tpp7J&}a4|80_3Wl%{_kLZYl?RM1 zqQz2t8n9y@l9MW?jq8TR>6p;ETjAFZ|D0{ITdGy6({aZ4*kNmfO7O5!-nH|+(xN3S z`(-|sHj{S+K3|%DUPoO5k6$=xbz&Mp;w~OGHt-(#l5Qrt!mjInyao4t1FjlHUbkyj zEp52uz*k>`Fs~}qmhI%FH+k3X7K`3Qg~qwMPAR~yKZGnyUylSUC(QRHx^QAmP9n?V zzC63h&25D>+%_{ocCfqb}ot$%UZY5Cb~ zz#9*PTTB}loYbAye|Ow}81caO9+Ssd)#>ZIJ@KXI0^ZbpJg;8VRUY~{r`g7&Uz{dS z4mOi0v3S}ORl6EH% zV0rVkwf9~_hhHpyld96c0GMuiCh8}>WBM6$h+Gj-p#yv%kurjT^4Tg z7r<(ycreRsWbyD3wifqoQDZqMzh4;T)wXoH*>)*Z>}tT0EiTkO^NZ+Edy{*=N561C z%R@NcLcXWby?CK9vT-|>J8JVHg@%~IZlpcpyIo@~ms0#A#8MlpP~_YeP9rSSV$ zCXOij0d=g{OrvMFyj46MF0Dkp!4c}%Y(#)M$K;v)+X8Qh3_Zq)y|sK4`B~C)dE#Cp zBUeI~=%_i^(h^}Q2La~_>5oyR6RrW_ncO6(+dB#A^RI=->ZzJ>HXTqx!+kieHTgSG zYn@Wbx5=jZKwcYWWV62Ae&YoxktQnqxc}Ru{{4SDbvOBZ*x|Ek?RlBeBX*?}QnEmQ z@yjLcyG`MF1H)N!#noyju4s`fExq+r{TBoV;y8h{Fsp`@_93=8kJ2&yGnXv0^S>~% z1P*eP_l~iqN>SH$L-^k-Uy`ujV@c`lHD{B-VHQ@BbRkpSTKY1k00#gkOL3 zDX3;VrO^BD7x%9*^&ze0)UVPWs;?jUZg2C?y4}BjJp$}_BCh;|v-OX~{abB-Bo>8) zl>e$)MMBO7<|1}hkHqgU>A#NH6)YGfTP7y=)L(v7jpEvhy2{R@e?8eB{B+L>jM9X- z_?Ck2#TPm^lsFF6$Tynrk;G*!oktkH*KD0A-OrMKem`-=psPXlNfI zwM3Slw9*5mwZ}deYyfQh7FZiGdji5-?IeKkf`H+!$P74a9{@ni7)kSRz#Q*5i+T!x(}c)m7FIM?wj+FVfe-@;BNV8=qrWbA0?9FgOB>$JzA965R-tnPsGSUO0h`<> zRE7YNF)sfyH4Y+Mp2$p2rBM1(4b&Y({Plp2gv&f<_2fkvhW98YV0eaCJAAr%tGNtB zF>Bdv?vxLkptKf%nL;Lz*_ICsRAsHO(zK7e{21&Osh>i|>*yX&Oi0QGaIO zf8|E1zul44pSab;FA%J4*_jYC;1?`EO9ibfw(QX~2?A9rh|Q-fk7b_LXW6zY9fq7I`k)(lzpZtVe5_tV+il}ws7F{AH+#G8i;%lZ9c0}x1 zP3-AMH;~AZXLpEZZVw%A*DM3W-*>FsDTr|Mu3@J^PWkq$}QAnZ?v>ahHcQF!A;$-Wty-1(>=)a{>k*Zrvyldg z_8H2+O*@KI|IxufKv8QCbW@}rc_MOlZB2A==y4y+G)eg5{4lUy;};8w(ifzBeRd%o zF@5Uf^|r}7YHe%*3+B&E<97}l99tr5klCn8fqsX|?@DF=9;o-)yGij;A0dwShd{e3 zU&#rn(_MKcp=*eh2sTBOWquFP0o(X6uYEm{woUR*QSw^&SkO&3R~8N$#4a1dSr}M0 zHw*7=Eg^fVZEDG4i={?qDp(Gh4}~(Pj*q3L@72hf=mFNW9^#AeS!YU1st2)~*(L3K z+KN{L*LkIW>rGvi(_-_cG$YF+D^QD62||*`foa4&@Y;hWLgCoCiH!{|=BVfPug8sr zNDlz7^#Q=JC7=NnWqMthXmyz3N6WbUme66*cSX*Gk%$^D)#2v+m~{xMk^t(%e3Y|{ z%T#Tmc2f--i^eP$#%9hb;zGO0zG!B&awF@U{cHlxr9vMG3b0Be5CjUE}=7^34LT|B#G)N{rNU$Vd*YJwWODc zukAB#=T{J!{HzS0wIV4RK(Ejpe>bu(U@7LU;v;J+XM==BtS$C{q$>{H0rHB2tms>x z%RuYabI=qdUvrvQVw@+D`QgolloA2TGiSXl+)i2dU>{YkmuY-aB<&i)5UI!&V~?)6 z0RC@EBe6N;x*y_gE3h{dBo8fa#(i5L_W2l~(?8)XA)5pu2im}OdELJMwBMHR$O?ojm2Vw6#lhqrp zQ}1&@hEKM?jZB?)_8WhBPWT>3%aI9>qGy*cJtkUI*n+iP9T-Xmk>w)86}WD|iPSYD zl1h+Z3Za{^dTk%}78bWH3*n}z0cpZ%WpUOc$lS!rYi!$>&BV_H6;b@gcOCy@G9pk` zC_trsLTGWs<}y*-Q!1|ljjXC;(8KPMO4-f3HN2`%kkpe%YtC0#D(V=K`)xzX6!}2= zvy1boi>U3y*CzeIo1t`1i-1mH7fyS)Bo&!{!&-^4=|ZWf0LDB?@0E+%o!Mpj8X3di zeFpkC^^Vspkw*qPgY7lDJr9xwTd}Q=QZ9C{i5dhfWu+^-Hg6yT=CjO-hqi_4&zdg` zobjK#HSQ`OG42RUUH@@r3uOlz{(2t)jm}<*E9sJ+UXAY@q7Z0pBK-q(E~|On6^XW| zp*L?mg3Mx&oqd)+-i0Z_NjDq47(NyE1*aS;OZuLod1hNnL+{{8U7k0ysAq3gn#c%? z{|;>UkCnmnZ|x;IPThxU|Gu*OSNZRsKRtzk?OS(9!r^z-n*aJG-@J=pLyDAI3jT4Y z=wGQ`G97G_BKIy99RJ5v{g3}D@q=Bak$u|mPsKclmvhe%?A}x){I8k>0g`AtW zOpkwgLvX+H#q+{IKO|r z{v$CZ{BLmnf61J_nBGfT-$U>JJy!t{ zq*GC z5hby{$VgTC=^wxOkCp1b{yTM>Ul;VFpLW6N*XIO(p_oCO>3;(;@|*BAC+T0!4p$oP>Q^BZ>&^cA z@%R8>0Vp&GOwF<@^75hz`skzkyfj-VI-%PA>$A;8A|U1w_Oiv`4D9 zmvP#rNYla!>W0mM{D1>tMC_vp&;Shr;>3A%u9VT-Kp;c#aHDGyK+AyuMMw&jj*w#EpgJ zsj*Z}1?fEJ`iW5p%f6$5OD$hK-ja|w&w$I!UcLGZmkP(?}67E>U~ zOGp6nd^({Er!*Zz1gfn75yb^i!=HoxZej)XnNbAvc@U&0jmP9Gh)zu*0+5tpuy1)F z!vPeOa{*SJVsc6m>tp?+3{1>}niN+URz7tBu(=NqvQu7uBDD$;+xnB0`VB5pjuC3SJVr7tzkjg7qERWz}@0ywVcSmhW2bf5zYPPrYA>7{`{$lt0y zp!wr5kVENQ16m4-HP?CN_#*Z}kXci>YMv%p0d)a-AmX8qi1Kj)C}_ytcn_X|0<$o` zr#&Dttf5xxz@HhtK`hox03DsD5|^tko$?08tm)aWDmLEPl-#^7kha31yC=Q!l<4l! z#otGmCmm6gIOLO$x5V_DQ@iK^MGqngcY^3N;AgfcU*rW|;96k1d-?SZH`8e=?)!v` z+yK~H>;S$MeD?|K1U9{mpKDafX$uRyyg^3W%vB1fkhRlT1;EN_-!u41C_U8gm8ja> zMSiHI+M%+h3|`yHP?z)*DrmP~Vc9|Vvbc#Q@M>HIHYh2m$;I*^!YgGe{L@hse@|_N zQtcm(vGTq%m&Qbo!3Wm7R`){-0SqnfUkVuNia-yWa}~TJpR)jaG zi}%tKKqtmT2UH`>Fko=+i=F&<;v-7`i#D5fh~UJZ8>NNdO37|=2VIDsQ#&aEj`k3L zfAp+HdJ9p3RJ(>2Qzl<*|YAiG8t$~IaC`1N_b>W6AR& z<`}rC1`4CK+d{^bEFD;t@a8snv=_Pc)9FjAXRzsFeqLkoa$95Pr58d07D zin{E6jZGoi0^sek&3jDb2o-%$63hiXN7KW%D^C)&ME1l=0ZF#AjOdlL7dv&D&6-%@ zVtzIR3ZEIUUhvQmm>L3mr6OOz#hLBs(IjGb5VUelylWdI1^quJK)+8ibJ%!ndPOp4 z<-yv+WSgEyAnqavlx9@hn+d|6L_Lg`r~#s6%xDc<_Xs8kIQSEv!xRw90U_`>LA%BZHqz8oZ-sDESD^#unEMi?gP9xb(6udTe*Mgnz)BcjBOpAEwGZ&sE8ue zf=(X>4#lzG4K=^8JW5AjOd}dj*;bL9{t9}J5MKyx+1tc(Y$OJbd=7f?uhWjmXgOc)N(p`PU-Lp*4p-W7nCV zF+aRSYO67^%w>IOp(;!GY~rMz-+#ask83H|c23z$Pu1$LCi(}c+Xx89VR*@1@E-Xi z87i-~PJ?WY$*v-TrF$)~pro0RB>4&py22p}3ifX(iYtkl6F_8UXL0gdS*PnL=ob6@ zlGTPohFpcR-$Md>G2xY?O!Ape)#e0GHezg8RXm*&)e_JR4CBFi7 zN&K;S5O5HB$gDx43(-SMTHd4B07d)};Ik8m)<=zRvJz`Ynl$ z@wGi&TvNu{>mGK^ltM&+y7CH)M7QuNw_zO>aq>7kQmZytvUVEvbUx;IA$A<&_cNo= zKfq{?fRe{v%h0`8#7(?CAI+ljiWA806|P2NxA?TKv_re{DBOtBS`~B3!pO<%F$40^I|7k|=@=LYEc)z=BVRsd`NHW^R%Gh>ztVcwNmTDgP zofLwsN@XC2e>Ij0Sap$3C_3{>@_Bz=qktpiB7??tet41etWk61u9f*Y7|B@Cr<4Jy zu32-dXPQ;@jLtDcrpOiu`99Ev>6!2>dX-bFV5NHWLY3&mpSKQ;9I zj&}bz1ARm8zy-shE0So>)xa~4c)|39ZXr?B(&8jLP$Ymsi3Ii zc`55HU2i~$i*7^p+M^@<(K8?MeEY;i4bcTf>?P2sx$Ui00SbGFI41PWAW_PS=^zkT zN1gq=`xtkZy(+^}YyE6ng*>Dr&)%MmVxFxk(=&<9|Fd?1LX%2Q{9Crvw6(yvL=JYy zBpR+q@naLXGw@MT?UbgO4k;39auxiQ#qM;~Lwn9mc=HM^g6R;DGvwN^P~}qzaVErS z+g?gE-V$75E=fP2zYUvAn2THVpgR)!-y)QP*_%eX*1krxw`7uH4TO@iK_}|!!+22A z!-Zt&=khFZm90EF_E0vut#aXBD=BC{*rd79eLELHt-x{e^tnLGI3e4j6(P}jjX_%- z6t{}Px1{;t0GZZaf>ha-JE2nNa)I>HS7#7;AH5GjneQM6C3*3!UzRy)2dtIX zagc1S!lu(Nq>c~D%tsR_7zSq3?vT=u8$sII13e}-3YnVH(;`RDcR(y)P|O7|m<4HxrAG$%w1Z(9iPAOS~7lHfUYx zAJVT&BZty0-@ED?X$|8|qrT3!QnR(#-z9#uGYOw$wcvhn*4%46`jY`Ox$PCIPs~a& z5w8to9 z_qpv)d``LB=32i!k;>KP*TriCIRY~=dPKZbfrQ<28Q856QcZN{iz^zNK*U-d@AX!C z>$p*fy|%%J=82B@oB0P>U1Ty!i&-aVg;SzQ!y<$BZ~Z@#RK%QhE-C>pVV z&@}f2>v3bt`;9Z1v$ID)jBd3&f^`q9hY<+r5%1$ePt{7A9ao*<$^a^%6<;0;HzQiP6F&sR5i$ccX~gWc z`w#8Hf=CCF9tW*uQW3XYN1DY#^4iT@77+GKqaOR0o3Z!8wEElZ%D_JL;7Bg>6mhk) z1(ed<1gNt;gV-9sMc3}dT_HQ)M%89EvLW+dEX@Dz4DS1ZQ>n?Q&XNrcRnu4Z4GfK? zj}0L$HOFZBs*%f2#(AepqIeb`u~<5I7;Tt&7|jO2t4)uKwh^f*>5}hw+%Lb|13h+w zr^_$x&*k^X^nYnOp!Q8H8vhm`X>jHqcbcTWurv(~QGOBqFyhHRXocMKN%YGZFy3H* z8+~MJPdkxKTCiuC(XUU=59E}VU*>CGep41L>Y6tUHp$}ly3vJf11VP+%I8rTdE^E% zAvY`mvoJIso97{=Y;-}a5`1YSG3@+=c^_aGH9QK)c4KRI2m81|uzm?!q8rq_#Zsnx zxsi<`hfS7Wvly@dJrXoY`rBWW&ScD5EbL+vlTQ?zjx=G!W@GFZ0o`IX9x|*X^gE7q zL31k66b3{W>D&RyPyXSqQa2nk16X+b0SaE?P4m!{e8lT>(waE)Pn!L`O?4?hT_gEl zAM*#E@^9mP;si3#9ZKmFU~?{D)Xi4I?;z`(4()hEw~9(jM0XXvy=kY~afd^_)$K%ZSEyYS~_CG|{%UIo}-%eNJ%D|gknVKNY^>K*nezwq#` z)$^emnK)+5eRXC^R=+>b$O#bt_IvLl7B$3 zzkTd;!n#9{mzE5JssfIn9zNUPx#bJl&pdnlIH1t9(FtfUKFPM~^S z<-~6D%?p^RIGWV{JQ67M61oBDT7c#V{_*55N6tsmi`?{+P^Dks&lVVe-Pe`(V3dFJ zGv@#GZF+%+Jfji1`twUAhQn`Q8vcFV030*)yzd>^sQxtDFnD9$^zKKBq@M>f+|qX* zkJ3tY%0#<8E)?aYAQ=(OeQ=3Xnmu96>Rn1u=98p7X%WMjfS<;>QUYn0N$ctxPxr+k zPQ|X|H#7e@2+?6&+M;#n$Bv@l^1obAYPl%C%MZYr|FK>r#EMAmv+IoHfW?J>}NhkO1@-)E)I|2;2|?m z)pM0DjVkowZhoAw3ejmB=?>l`bLhfAZV)~#?!BE;ty#1xm#MgW%EJ4$es19(57HA5 zVJyjsr%4ARPgD<29DB0vWD(4IAh9Z3Xur};^WL1+C9Tg4S@+ypwR8GTBew_2Wt#eF zQjHXf6LuG)vt-0gYQFtAkZ;o@yqWpTw;`!Nc!}BGd2pwqb2l!$Si6_p<(*WA-RH0; zIx&&O+{2xl_gakf2Yz^U>5ke{lY{NYcm3CSz-9>~F#;jJ z@K;&P*YXda69oPZqujI~UgF>XkmE^tVT{v{;N8WN|Atb%zSXR8o|@mfwxrL?;nx$u zCsLy%rnU8o#hP17gOl3Nvadqdc%vWDZFgf+p2c63ZE3O&HYx3;6nW8C83KQ9^fc}oM4t!jnmg#=g|Ev$T19^@MJ^jG;FVl9i zu0Mct@w~+GLB8cs(E|(q>E+Q&_l=J!TVvwXivioIcVJI5^+*;;KPXf(yLF>AyGj5VyXP~% z0x@T7Id2{;_|*!h+u<; zN4Sz94Ng-JGiOWmP^B>?~V(GEX{b%r$Pvd^_+!b^lEDh$NP&= zC#3~*{vI6m$z#n7)jPZ&ZO2<37p+HeELBMPnDhQDV|hKto#hyav*xzc{Uv=zq*|_P zm&Q&%#8SOhR~YqX-!l@cSkJXdbK#5Ky~|s5<=3Cp<|QxgR@#29Kh@d@IzSgPu|MYt%^ zlUG@4S&=u`>b+-J9J`KLSoSMC7RmK@w0PF2jvhAc^?cB^S853{vK>6QI~mm$8sR_iN#pzJK4$`m`8?fB*KpMxG)}sp!e(+SXyXkyHP^_s7gpQ|B8!eG2m` zU8nu@4je1Q)a3F!Y04NSyvjKBDu&pgs24ycEf<8wzN59RSG7q2SSuO=m0k+U+xvUSBQ1`5(SVI|!&ZSmG%!%+ypK25MN=D2mM z1}vOs@_m8IjuEJ^ZfQ1#vBquuHM z=65O#mXBsC9$VWs>CUHKUQeLQ$jebj559OgYwWGNtVH{<{PQTB-qy<^LoF+i@u=yx z1vH~f-cG)ChL3QQS3~Rd!1UpUw;jo{;*dvavHq5cYUE(jzCaH9R+`1qLL}Cjg~OhN zI*KN};2{O6|GgaV)OoI|l1}*aDR+Yg(o!6`aJGsM_L15CY7D>ApZlI6F=Z_Zed!n& zK0i+H!&R=nz4N^~uE_fOkDPkeiW^pO4rfbCmbScTsi=)w^BlJtdh?e&Y5zFvgamoa zScIY77pUkxY~9^AddbzZ?KGqCBke~=c#i$S^Y)%qTZS4>rXKai>~byfz#D`tGhb6a zuACt)hTOPrFtbH_Y_~2kU^FU_q1zgn)vq7B>^uy+u$8K+J+S>9aUL?*f_2k~0HtDEAe z=?PcfvjWriW?lfrRaLgO>fJhk}?)JA=JOdV<&Nx5<`Mv4%@XMEfg z9Yk(f>2G@q^qx;T&T^EeWtt`BAC+P?A3C@k^f zgUvN?6%rW{c>8`Y_rf%DY;oY=02l9$WYn(6R;QC@S7601p4^hXm^rXXkoKdcPC6*H z2+QSdTNm$Qoizj6sGX|mHLjOzXP?uC3v8mICd*!zRIQz5)={0+*mQ2Ib&OuI9NzMNXR&sm8?`#v3;ob{D8Y4tQFXr5v-ZIf z54I-oZL;U%!w}ZEO8VH}X*F&pqnq8tlPn>;+o{G##s0)d*!mn*6i#PA*T#3@JPfn0 zdp~fyn((d2|4>&G)%1c|+E-*^J ziu#)uHU67W)rM)E(A%13(0o5aq1)l9jAtnY7LSVqkI}uUIyM^eqW3E>VV>ipm7cJ5 z)T}38O@%~i;iiK;r($VIxu>vky4W;nPPxQKtM%+~4_vV?F$HHAU7XP~ z*SlxZLC{Dp%V5ZLkp8Uq#l}0>fWHQkc{E3ej<}gu5_LazS5!MWy*~U2B`;D2gWLa; zJZpDGDeD*>(9ExEU9K0S9}6RYW2h7A*d)EQBu;z5tLYHp9qq0MoX;y`7zJ$- zAZDFkPwUeZP`6D$^P99}BECAd+prPO;2=nyG?#(oaim##eMRUFrDGSY9LF|L`h4ec7){dn%y}~`! zVl~cb?iE=)?%9rX_T{ji@veIQLS7yFjL}lbrt(o-ZiQh~|88k_u3{T}8u=93)hHC9 z8^qSn#6_=M{le%}oA&$Y=$+L*&iK*d%4VuKf}ikgiE&DyT+<&q-^5dS2^dK8;8$FA zMo&mkVIy)hM47kUN|@6eYSX1pm;(N&6)qFFNuw-=Z92OKMBhRu5FjrJD$N3=`teN113 zlJh`&vh@fjk+#jrR0El?E3feD))8VE#>s_cO|6#g3BKOhZFAu{BkMD`^_T29>@VB- z3jIo&sMG_pW|GEfUi6ef?i=l!?_3a6Fo~mj`9mb7s^`MJaH^3a8fE*YY^L6BEeYOg z9-R}@auDZvgs=jKBhb&xE?F!I2nn&r{bq7Xe?&Yn{rszu%_Vroep^RR1G!XGk!1TZ zV|SH$NtLn;ZQt%}0y+O^xdcktXrU+J`NRwC&QJqxpO$t?d94@X-cuM55}13At0CWc zvc*T;;Z==JSC(GUj2^o{c;@2&GZJ#J?sZlL8@82gv~F1L z43)Z8XS&R=`wL?{&Aof@kw!}1c$veRZ+EsF2dU_trD%~0UZsbx>OT38m*5i|py(q*^MoP_VS1$B{amEOXfnO{ zIk{QZzDJ~|i(B(EraO_Gwr!{7;pVQn%@}R0fnkrm1tc#Zc7ex+z4*jCgi5Qd=DgpN z>EU$=cB@vUKx6>IT6gjY^_b_t4uS0{oJdN7S5P9-qs_KEHqgT>LonRz!FEdj6Rumk z`m~Zs%opyqZzD;ZzWTb?Z#K=V&zC&ZB&=`Nk&8trXC=&El^%ROsL*10znIg#*f_kh zcimG!U|)3Q?K#Y>VN$-)nh7!mVWy~a3UZufq`l0uToWhTyub$ghzM$}vDxCb5kw%Zo{N+}PxxThuDG{}1(d=kLUO`ly_wn?%<|`s1z-FgTv_R- zWt^#@!`n57w!ZdQvyx(fg@2S`?{0;;i&&fB@P}ZTlCI~1KUx*~E~!dxR9h(MAvoxpOx2;!ZPn;!$Ya*`=;hCd{btLbo@WBsp z{T1(9i`|e4`b9C?rw^rN71q^fmR0xpa#D3J=LsyqMkSu}T_1`ikak^$sdkV#C!xwl zQKS?xET`0KNYlmtxUukRahb=M;6Rtn6s!w($EFz*0-tS|g!0Qt>|MuFQB2I=*D5IE z$`I76g$~)wq0{ds?tUfrnJh_;Q+dA`B(8^PFxEQ3+hl#!(#o$;YiQ`4-ieHc+y4z% z`RjKL`I7I+T?${FY_qnIP|zNq;6*O>c+&6bo5V$V&|hC2iZ#D$*{M!4CZJ%7q0Xd! z_f~R*?8c%%^9_cIKDQE+19AQ;vAcLko7}@6&V$51G{j22Kji}B=yK!F{!n^{gG7qG z84HW$jQ354X3WLS$v)ICz4$S|=~R)#&$tlGKH&6gTU7&PxOcffzN^32MoGxlZh|rm zlS|;+AExJj{NYM6KqEhO8|$9A@(G}_gL><;{tcR@VG6VEO=(*A8O~=qrp=+qvlc=H zqCyc91ua6H7G8U)3-9Ayxw(D}=A@IkoPe`2?%m9xK3g_e8olY|jKIZJJg3*1U$Ri9 z8NzRetIh$4mzqhennf-)t}DTRM9){7Q&AyVkkekF84-Up=Qd?>Bk(~GgoWNDR&~gM+58e3QjKj+*dCWe>`dp4}Dk= z_9s3G{P=eMzQk41Av?2jHOks=`ky2H!^7HL_(P;hd#moIklfpmA1{&rd0YOhp-w(n zZdyOHTR*GS{dJ@%4)Ae`q@qXUDw^P(3(qCHSE;L-yM>c;Nqb8MRtHNk}9wHa#t*9iuYvgx^IVb3B z4%1Lw1@~1@oX$II$`slA=rYbV!cg?rSN|utl&cQ}^QC_J>JrCxjSY-*ww)`3^%p<< zR{aDd*YG|pzr!oOu!-O5?UKCAuTIw2N=~szS7v0ri-XrhzHi(FUN!SyPo`w$c-~+_ zdJ2Ih$%1pX;WlIEJ@@TbiYAr3I9BgrqS&?qh0+w_|3Sy5uM(=_Zt9#5)a=@nZ!&#+@`22yZEjngF84=j> z0h7L^!`Vw?)PiNdDsV|gg0^fIB$VC6nK8%cF&c4O{4iDmJ_e9ch1vP7`p(UsLM!r8 zyAa`?E+h0vj;!TAf$(lIVJ2c}>+pIX4O3f=z+H}f7HIQ;;c32{BoTbhdwcbMI>Mnr z4UD^+Eh6{y78>AuC=>IZofzlTfzy{Gkr6y1`KpfYYu{4Ta{;eN$;td#V{3mphEUQ$ zZkg@wU4nyQT1Oa*`?8#>NA4?_88FU63tm^m8-^TW8tPyJ;)7TCi!fn?T_?=2*o*fB zFPRX8l`H)K4=uNX2_2XO8n*1Jr}kRsECudI6ml&TuO%fdM!vk>#|3shtX8}C-Q5xL zngw-Q-RQB`<3GV;{+>T1Ny)E|#m@NUI35cL%WlD=g7(G?f|g#1cP;kPu`kscA_ur> zr#ubDJlKZ|12VNL8;uq8%zBFwmM(Iw4CZUnh%9@n!mIosTXo7onQaYsdiIl_@GlMo zJ3&XI`8`I=>Ajw4yVsJxv_5bOW?b+~q-ga5Lg`GgqXUtp`h;{QIqXR-^+Mhf>*yz< zePW|q{SVhYPZ2gHidjBbXspnc5lVJVg#fhpEy{D?gX2-YUsZ=)Wiz^be?e&LYll4i zsEOLCK9v`pWijvSPq{KvKy}@?YX|~%fsY+NIE^~z<6>%J_XReFwYxb8dNQSscdC#< zZ_ocCDZ#gi<5EarwD|cC`}$ZNel(AocWlv6t9^fo$>K*k2hXo!UHn=K1Gy>XUeE2d zL3Jc{g2&C-q3fEcVFcbTx@6`#I-6SiMJ6f=YaPXzdwJxjCfGt79}qKIr-eu9<8<=d z*<^~VmZcd{gY|bISgApy^Pd7-sXyj@Id$)(3;t9-o__H~l#mYgX>q$qeXlo_rs+(c z^<7Raq!4_XXRCmcYqT>z>o~@oZiYi|wl~?VE`EwPDzmV%)MxeZHGf`SGB0m1()a~e z+ecT6#T+Lgu1hEXne=~f`&CZ2SAd7W33aXNp862;50+0CVEMpDnct)y6>Zoo7mmB0 zl|f0Cfp*Z$Vx!lYfeX^Wrm3AKqRkEzKT<@3%q{ zJyZaYIgg^)3T~b{-2U^<)+k|iAoew6L;mjm8|kCcCh}+|C%Rg#fXkyX&#g-&*NLE- z3R2r%cap~pI>UZ~K~+85xp}q@pO-$=nUMuSYjI^7rRii9Fqs5Vy1ZW7Ei%dqnhA1> zY5x73QP@DVQ>}=9@GmpO2~vdw$&+PWL+kM4txYsMr*{)&U5d>#&MuYfbt-Hgw9bc} zT6(+LAJ89DW-hN2nT5~LSA{t(rE}Sc7_DlR9=z*!vg_5tj8a}kzoq=(<>DqQQU-&F zjMqNl?-xtO5O(f35!$8s_a~g3GMhhJy0o1^7PT8Sr`$6dqEfVb2-dzl@;W&K0{Q4_ zy*1=1Sl+KJTmi;Vwccx{kJywIvjy;E1-hN>0$Ei8vWg-+#WGF_w-M4 z*<*bv7x&bLw5FF*%6X7-!AP%fJs1V!o)tmbv2ug#Q|ieEcigS2MBQ7(Mx95mLKn5M zTyLhUrLGUS^op+gDhBb|H;*=xXDK@46xwu3_%n@6mwF68yI0uUvzF7c;=Dv!JSng} zT5dMYeJ%UU)Dy|-;j8lW+F_43?HRoHT7<-}4Ow7s2%SUrl3eA~dO<|Vi51e?x6y-o zx~a7;f4w4C1bpSQ$PxYdCWC;z>U9lNK;-r$;b!gsZuQMjv*`W~Pyrd5%>W2Hn21}?xgm!7)6uodiLvQx6t0mfe zx3lMyF}b#jqYQFAg4$XVwV6>*9L8H!*fGuA@+{afN| zIN-ZBo?0EYF5grKHMSSxRVAPosi(xz;y861m@U652V!%qVs`U%n{C;{XwCcsXFi}i zbmnp38 z4U0efIOL|V!VTbczXb^Rvt}PU3}qZh-&Ec_489by-!bkQBB@Ij2fU==`zRpag99y8 zY)D=c1znce`X@ss)6Pt)?OpG(o=rHZKii-G-GuS~F!t7QQFUwou%I9*Al;x6(vs4k zfYLd1OG$T1mw-YzJ>X)Y~K+2i)S!Gt)8NWKAMAT?=A8=T9!%Q==z9@V7bF z*dPo@(P4qhXZ)i=jualuSmglzo(5i)jk0yN!^ZD`5nt0Z=!&p3j|b}*RVUGb^Tx+q zu_gWy;D}`iOyE+05t1E$ln(CQ&kc46XUuJ2XSZLOCg@f=&vy&>zE9sv8j_*WC?TU@ z@CNds4~Vc9O+B-dtLF1@o2=c@f;ZIljBk35%5plJt$OZE)`UrFHOr5FaFQ#a=6cs- z_*9UmkVtg9{ZnB>&r)5?FlX9iV}jm@8?^CY`bC3JnP}jYk#;O`>DN!ClPhlhwuA!v z;Vr!Edq8nG0dl2HZ%=)1&0+Od^S3wS2G2Lm(ro=ec*?LVxUp&Au$1Dc=hFIxTwdV3 z`}^zJxT=bLjH3LIQ^3YdbUZ+)E+F|zd-2>=`rUpaAURx~bP)E<6@&y#bVia_BJ)2b zW&vhdw?JZ03&w%nEdNO0sT1oA)<|#Xl>&QXJtqM&T06RhFWyP;;xPy&a*0fjJGeB_ z_Q}c@U$O6VIwM22!XL8h~JKd)&acx;sANRT0a9B~bSu%nYHa4M5~jI12)Lq2%Nx z+RHmLT%KF+1F@TFAqSoy;Off37brtHu9b1Wmx)cBc%AOp0L#g0utQA(eWf)B{3-`h z%AW2-#oZJ(7)97+F~SA&+|mJ>l}D^MwkKk39$a1apbrR1n?yr#4(oxbu{=`NQiUBR z6`ZK41sH{G>X?uTxtEKwBZO<$`?&ze4i-H`&It6G*QVYUWvg322Cz>oaM;vodgOS0 zaeVmB2GSCt#TYxx^Si<{IhsxtS9}~q*lyijx){U;$GC|E6j$JbYVgbrbmLAIL#Z#k ztg|Z=j7l|Nr{Aks5R#j&JfJX{1{vwgf$-pZYdLbHdo{q+8hwvyunATIFDa6D=!X$S0O z+am4i!s}nux8+~obZK9nY;S`Qti85Z9&zbVKgaJ%DhtPeNzOZKk1!Oo0gkXDhS_A( zWCncqAj}HHFaXMy16A1sAoahAB)?6*E^8@S$xGFZd(MF zuZPrx&dT(`mMP4tc0V4T$~h9Zka8IMA*d`kc_}B|EW;O(!0)7 zAB~dc4wipz1HpF{(8Y}jQUfiZ&-qBuI@qMm5n`12N?Q`ZDe5&x-V_96ZLx3Y2yVU4 zZ50rpOah7XTL9WcVDDK9j^C)50tNg{$SU09a0`5VQ8RkJ%`I>lE$5qaZ_{^QM4303 zt_yGjLg)yD(VWg!WpyOO^nJNh-rilS_@~9*-Gmyt0FZuHj-%8i->hDujS{LNrWzyt zMJE0o9ffIq!555KZv6(=lHCkz;i}u=JxA36NPLn>+13=0kB;n4#{&ijIf_{|^$k$$ zUVW9l^v2GW5pn1eKsm&Z*lKkvTIc~Dq^p6(Q@0E{Qj|E(#SmEv#8%E@H#d{hwFBH( zX&iWOH1Sq)mxsUlehWbHsjGYQJ&YkE=|D59hne!wB~6u%q53MD{GdB!%ucIIT;5=E zU0@DUpkf+fn)C7p{y=>@*dMkvl7X2MPUl$jh(N6LHn%}|JuzUq3**wHZr8yB=x}>Xt=?}?1tco zgU+QnWt-Zf>Nq2r`lS~o7gQmAquD&2Va+|u>B?lXj}~Gv8clpm3&(Zcw}&?^)Cq-q+@yyjnw0#0hHr`rKN* zdGy>ibQMnzsuX{2dsnZCsC1Rs3+Fg}S@TFNU3C^^n|VAe^egHUjG-Vrph%-{e>6aS5^=Fa=NJqfGO!E17InamBJsFbT9iL%zxL{D3#2y0!J^7yo=Q zW}vHV&aaHiD2*(DJ}7UO4qrpo`KE<}>7x3IK@DT?>BmBHWY}abxoKXSWB4@~@yhk# zyvx+z0_Q4XFC>Y>V5Cr*FBzXcvph|gSg*8YlZ>#H*u3}Z*V@Y06n7t>ob~2d0tUhy zvvFgQhP@hm5iVT%d=>F{IV>!*>iJFgB*1`zHN7@T0y4;@)fpTpop-W;o%VvT}yt2?aS4#Y@cPLYL=a{6Av*$%d+O?*TE=Imy zWpU;GO75@J1Or}-wHr{p-E@W_E5SY$lb^tp+njupij z<^OQPhk@0#U3o-4Rv#-eN2Et$F>vi`KCz&oRc*=Cqh(Di6lO28L$`Wh6Qt2(uV z2XB2T6WZA2em9$3ZkSw7b6l4jD5KgRU1|R$)o8tWu^L?{kstm^fOxze|VYVhp46w?s+)CdBh4)cbMXN8P*$85s zY=b&iWUXjE!Zw3pS_;2x`QU(D6-6H<_+Sv`LwpzVNn<<;CM8o{0!?}4 za7(#4r9fXuldE2(_H7x+xyCjaSby~4#hn*nkX`gaq~*UE%j=A17T(F4_b)w+)r+7{ zrxgZ`BmYMP1J7iBPzi*p{n!S}F}k4R1xmP!aL$$v#0+!w*;ZC z1St9=qhLc)FF}!TLYlr>i1)_}fXMPdm9NYNuQVsM66R5KClk3AMp{f`T(|xisus7O z6-%DJw+8)a%ehWp))_&RK9QfPkfX*8hmvPZ=f|dd>!B9g%B-U%vbo0(ekPmrEW1-+ zo6|*@p7!*6(Zbj8xgs6qL?v=v_3kK0uX{2h^UH6@rWI?b*)#FpdE{Fd`kQ7AH#s*j z_h_QyVfj`-OgWbmMuD520r$-_QF&E9NF#w#tKAsRcV9Yvnlzks$hXA0 zLkWoEpopBf%=JpO?+`C^`&BbDq06ux&)4>AUdUJ0^uw{WBj+fnwG@PLBvQ<#3-&Dk zTH?9rD3B$WIPsg>ucf^veD+Xwrb&A<)7qE3?`@lpdx@fv0GrkHdKjUjE zs-m230vLLMi%M)SlaZ$nJ4F4# zzp^X}xctp8ytA z%`+s^>HL)%PIp4hk;rlT!>10ABOw}7d`6mLvrmM6AD9bii*wf3BqVtP<9VQB5>*z@ z(>33cTt6krPW{y_HYERSc5Mh{ZL-ugwc;VS;;5KbkIiUT4nq0h)I)-g^C6%7kK@6d=~T;^)ES_Pvdvm zDl95JK$1r>a(OoxTt6mHyw6_}$_9h9$}g`K=ATOtn)1A@+v|}%|E|OLIEHdFXBd6= z_=C82Eo{GV-apEXiJvuhg*>3aNV`BE;vu$KntI%gtAg@~vdmE+q@5D8wf7YFUX+>p z+j^I)*uW_7!n&-4N6tOj)&|HI@uLr0pcFsDzy-h56rcmEO;1E=HUCvnn`SuEMr*c-wFz#%4DO5*> z>>s5h&L>6TU8KLARvF}%)E(ScqVufO8}}{T%6j8@mfmSoRIs9)=G!%uXhEItSkrI= z`SR4}=RE8%AQ+SQV#ne3i@+--LrvC~>(9%=XA8HWeq~Hf>kjx#9*sniIz%v90LG!P zX31DixouK#rbREWG^LcDw^kP?$ai7u4Tr>Rz?pYQNIN6E zO^Xg^v&I;PzwzU&sT+(8rn;gY9X+v)B~EVa!c?R}+LF|l-T4FZRItk)kx}^e&h0ZmZj8qpGy*wcqN*S1`Q7U9I_e zg`DGSB`}58?!^^krydSs{-~>G&d*vLN%JvA|HkVw&RcBus@h<@NyT4Yuf-BxE;=Bh zL)?w1mHI7xy4@knNS2-KCKsPme3l$-diyQe47%7U8>%u@X}GEwv#%WskCjX*P_!ZI zQoW=M2@56H`5FF2>qX1=w2B}>h*QLR7v<)|yf6I%t{V-b`<(AyZDd%Ij*8wpKGvt3 zeG?iojBP@AEu}*F@=EO4Eux!LOSRwcdoxU4u%yYTZA&>k0t@Q9KY|eyCCh)^*>(oE zokq5#lwXXMwPEkh8hs-%B2<73H}V<~kzG|;k8|$V z$sHc%hbFvarCrmNPTrq-wNaZk{jj!7VKt3;OM|<4rnr{LlQC9;kFz>`9j=w!Ye`Fg z&&f&qzNsw}vx>N}&F~=9$n@h5nAzt8%@!zeC}q$nm&UyLIHuT;94s4Cw+-dzmOBRW zeE%wYhFvmC@gSs-X-#8HNu&Cak-7jlnL<1XcUoxy0URPrT2M2pcF5S*{02qOg+*i< z#qF2`<$4xaI{Bo#=zl7H_dVM_nTwr{N$CUe= zcJ%90xmXR*1i87_ZR=aqL-yEDOc`S>Ncx-FXp8!wys@y8EYb@?l*gmovypsrYnc&H zvyu_+L|cM2XuU1GU@btI&z>hOpv!+d*ZCBvutpC zkyz$X8u51aNcE#c8zU|=pXp&?XQpjw!4IPN{AT1;e!}E}f)d~bH87urC6BuCTh*VE z2QxP{K5RZX#YWY}oi)|x?j?-H&=dwXZo_H90c=4O^5a5G#qGP!ZTfVue)UZZFu5_D zE#|+NpE22IT}N;VS2Zj0TrYiw{))(8E=c{HJ?XK1cQib})_qgTo|A#nt7^xxRrAtO zo>>b$q&wkLG$=~ypfnycU)d&|mtd=70-jq+PSN0$XA2BJ@(V^yx^P96dVDdM8i7Os zW1(POan^lYo^X`@a%_M+OzY*O$h}rOMZRCaT%kUzhPL|qL${24C)sP% zV;xV>Y4tWd&WT_yy3#IDS>cu# zQ_Swgn-VDNrbX8k$aqWzrINo^UhyHOaXCus<2NG}L8V`ug^8$Gv%HP#+~WJZdFb-i zv=vK;Hbl}(`<y^mKRHl zLK7cfh*jSHf!6en7BdpC`YU#ciV&JS#rvI~Or@*%6 zRbG7NX}yD-xd^NtYOWtrj1`%Ctj1f7JdS}SAv+>EB|Le_!CK#Sz9y)$BIkT#q5-qZZ zw_|Cs>Qo#L>|Myt@G@JGU=y+ckvwbmCcD9z(>% zAhv@(2Bf{&1no^MauMT4CPupHcFDiqVggm5PS~8xbYF9nMdWJ`*!iF0*Qhyy8 z?BF6daI7aVPB9e)L3+M3QL;`qt*+XeoA29oetr|6rBGJS|LS#``ty@6P^UilX%6NL z4LcY#>K<_FgBYqxuh!)*ItoZxHWzGWt<@KHV$w~1eR{M?2&zt2r0uo`2xcFVfJcg0 z27T9SfFJ1IE7T~m1S03J^pTvIzv3{A!oat8PWbRcz&f7#)!z{852xphzNy6&6}1p}E* zR&4s+jB93H|Cl!5V?3E=U`X|GP*dy3GNGIFTrD ziq@!Zukf6)?X~>I1|BACFuP|JcV^VSkS3vrPA+tOTajkcBd3oCKQ$m;0X!gv>Y21! zdIjdun@0{4@OKJsso#HkhAxMhS%%ma=zATlEmm%g#(X$I>He)IWnP7NBm%;7#7%zM z#6!96Bc4iVJ5sAD);e-y!UDFozHf|X)(&xB)|!+}MZJ`=n&J5I*ZV1~`PWR^<0k?u zCli^^MKb?!MEDN+UCwtK!Ztr`Y35%;Pa1y6RJ>8GSX>9)F**&J?K|v~4RoikT2kWW z{Mjg{Cf+~8w_eNt%3qol&_I^)FFPmzY61tV9=`qjRi_QDq2Ty^&6%;(W$~=f=oXjP zz@^jr;%Hr@jZUj0Bc^7K&f9w=Cm`Hb0r<1(TvMXIbaRB;i5IW#q8^m)Nst3=6G)EH$ID z80Cl=8}c1#Y-0Rz-Gh|{}J%A(E0tl2b`n+{M~=}>3b^pdU*XaqJP}(f4K-HVBhH; zk(NgKcLb(8$R7B>N`sHD^Tpp1#=>=hfybv9||d_ zSoCpGKK}c=mShIL1Qw0SrWd`|xd?j3^T*zf$| z{l90-fB1=@1i@iFHnaUNPx~M2M6nF;^()Cyc>jH0ahbARdYQExa_~vEWcNR zuNNAA_ULcNNvR6iG{V&t@tNT=hWjyJc#V7R=~P&IL{%2+mlV3J4|D?R>URfC)c^jC zui5qxm|X@iigP~jKNt+XV<)mlLg>H5@^7CDezujPWT$r=b>p_ejuZ(<2xqh5JjGf- zpU|myTn;}puCV@p#;e7Zw%l=9rbxHG!eD&ua;Eg1PiOBooP_Gn005EQ_fL0Ti z4XRVAHW6UZI_X&uGKkKQW(e{FYe1?QSJ6B$?lcXhbWO_K0U>iZ({&DS)SKJ+091|0 zf3(v|XHd+r2JT2vl%K5(I`DcnREYzZ8Z#yj09PaR>#0;)fo6`)d2C@+DSoF z;gi7{8yy6-;d&nAD_ergtLz2#Qqi$HzSQ1FpWb@0uRW~oK@rA*hW?qvacchXH0th0 z?5nPrL@ebmafC-~0ncK%I;E7gUtp7k9B8xjK!W32{Lu9^SS^jL!fFQ!5Buz;Ca@L~ zmycTiC%2`A_t8L(B(o3RgM)v4)0#i6nWF+Aao0ELETtpO`zyG-vBEfUtvV*pT} z5YGTp=;#u?2Hv8E01<$g77GIcz{!kL|M@uZCQCuC0A^PCI9Zl3Kn#dI-k#>Og72*W ztF~(3Y1}f8Hqh~oF;U;M0MJg}g3V7<5JV<$hGL*-pQvq?Mp&VQ5>n*RD8-xjJH4uv$zF<@-uQf z*!c|qNalurlj zm&ue2O}P8l-ej(>2Lfv}q7wj8wE)w@OFav6O+Yf{*hPp^fbgy!7OIv*Kxts|yJ-b( zm<6vS55qP)TR>QN}6~d$LGCDU(pj!iVu=g*A#E}#~ZMy+@EOs8sP}mM2xU8>c%g}O}_Z|IA9{y?s z7)P!kitjAp-VsGCU|ujOZX!z7e{A0^CyzdkAGWnS2}16uw`i0w6fD!}`L4g)0VExc z^h7`|_->H54%-A)?d8B>t`As))02z5t6SGqX=7e1$gye%sas>Ud!IM8_2#7_nD#@~ z^J8o)4rL&7kr{bXdD*7|=A***S$_c=`<1-d=W;XVdZ~ys$R@`FFhA`uc#A$aP{4e~ z1vUgACDUL~8K8$|diVxzvo19NfvB4t`t!PwdG8lPjqw=i(cVAI%KXH#(T{*}BAECL zzRHB@VQgj~9$t4|E|%@eIXf4VMvq0g0+TVT&@2*iS3nzZFA_(u3TK=L^Y4TOWLnhaATe7_0hK8-Wa+&q)tp;GMY}7jn_zHqG;2kWOK3zb_SY7N4+*>c{ zQ$`Iwa62tgL7YvYa>+}?YW@a#v4xUMjpOi~UeaCQrmi)t+|gmO8XvY=po{|X zQ%8}kW`U07jVuiVbTv2diFcV_CA35)0zqzC?YA@UO^H8oytS#OAt1X z{VyPX8sZYBw#q~Jb-+(>8*mej{gqL4!3)O%sA+r2w1_`rbTV-%5Ra@A={avI&L<daB>M&}6RvhJqQ%oO9>p9VKe zr;o`DzG>Jr1DE|1zVAm@+_PyaAFs^wnn_W3+s4+D#JWkELC={ zwYJ`6JiEbMCVn9gQ#x@1o8DArW(b`c%Ise-^4aiS;gc)CVmMoD#pnS~N?dhu%0f1y-)D2Y z&z3}Xx{ z)jCIEW9%`-zTle}sRHx-#_d+Ci|pU(Gci~Hs|;$h%#?GCpzLXo*|Aji)A@R$!Of0( z5BplBaU)Gpyx#)FG~JFqgIX%@(-vH%O`=~l_t#pACTY8+tycf-r~yoJymfGSxV|ib zmi=e8$HtWIPcHifF@rD_)|96Mvi}s#`NuG1eMY5|Em-q5B0z=*XLobsiP5Nw@w|iN zYV^B74!;R*l?O&r@DGism1Rs8Zt0`o66SyY%&-TDUKjMaBUkYm1)n22K#kdNzKu=J z+ryVy7%^m_JdkHQ3nN1QHt4h1n9@2Vxr(SK@8;8ztA8Uly;oRM4K3Q0L(=wEkL|;l zV)`Ciyq25{dk-*gK_z5!&@R2vxCtLnC6A@k8Wb*J*e$dVOf zw`rT&VoW<$gBcjJCPl!QwutG~#tYFwhRKRyN`D(kF;a|iU8hL9hwLnswl9v?hZ*L3 zW38sKB&XAB-c&!x!t!d=F#V`hWuPR@n+7_tC521DC(@Pziqn-f`)N_w{X&Nyn#1zO z?j7Z=Car7cyjA)4A*2G6p#9wFdF^mq8&6y*4n$gGJZS?;ANNaJzq^S$Ig{W!M3$?t zJnz+7{<^syb0M-506xJ(dWCpMn-@yG(wkI42^xIbZPn(^b#r(BFyLXL$-5nL|Q$k|6)-)$`D0dz#4>ui+25F+G~T}+>Qq)RJ& z%Sj-NBklhYq>&8r#TJ`s#1bB>`n~So8<8wQex8!RCP(Nu=oEhp*U>9xx&%CSQ`e1x zyo-o=H6%~H@=d;S+gXA6po9Q!CXBt%967yNih&~FpbiC*A$-kP7*|Y;Gx^Go=N0?U zM+_JX#)MO5^HL@}A3Q|yk9m@H?H7=HpTb_OW^|~gcx6G&BR4BN;}hUTj-Nx9kE}Uu zF3lTQya`PwHj<~NctODuPBumTw)yj&GN#b{AIo4FSrRE}U1Df&nrW@}rI&bLp7_jN z{j0vG*~=3XPotLV-@Zw!IJZ*N%PcUq2S!HMF1bupMFI8#%Yg85x*CCz!sJ)nwH~zK zFmv{!k2(H4wRjDF24XF$faH4czQ4=d8F&+4IE|S63fk$O7_B}5{r?4Kx(#@>Y$97z zj^m7n{fuLNPDSaLQ6XpdvX&a{>Y-!7KQF5AOfeYYm6?(Re9utUBh-l7Cw>*O% zF)!PGx9R@*{sR#4Oa2ixC-*Kx1;>HVxPSC{J*CL(@@u!}C%fp|HuO4aGm3O%5UBdqdlqhs;PXS06G7WGc~Bw&xtub(kXhqsC=bBC+36iSX~z zmT#l0m8p+H`_taYlbMP{^Nf;NJjZV4!TdIRwVvnXkdny%#L8C7KP@eQzhH$ppt2xD zdQ4Hv_X6`vu+>U3xDOpgno*~r9NBp5=d%;xm^+p;YF()5_cl98{yY@`&TL}jqa-NB zq#p*~IF!|wJWreUh-21Z+k~sEeynfwQE*jAnO+`%t)6NN^hN(^ZG05;Z^hVGd z3rh8QH}K(wp$U(#mZ6TiNjO07@QoRnPA*F}lo!!jAQ!N-ju32KO zicD*mVx8@@K~2e?Zsdf~1Ku^3n%Iej%?S~A?2e`vh5z;=Dv<}pEN5h%Y-Ub2s?9K# zPOt1ZH1e57EiF|@cPAiC_ju4zeiO~qz?UUj1A4Tel)gVFjsIty{>RvrZ=*fIoU_DU zx-iArfURw>+_9Z0*QfQ6bH)5zys!mTv7O6yJ2A9NQXDHcD(@}ZpdOFDV~0#Y{dWx~ zU@D3BZK(ZA)Fu^jDf75Z#j`{8sgUA3)+-aV%6zsn?duFa50EB|8{;u7~a{&bIfhO3zT5ACxOUoBZ-?ck!iwj5dgj)OtKnhaX6nxI7a~=&|F$J)jgK7W zugmIeT#Gvc;p&;a73_OpsXt0IH#=*o&ZKIxS!Xeddbvr=b#+C(oy4!hs-Yl0*WO~c z{z%8zEa*Rt{=X!~1X*InG#arATPuMLY~>9ka3wD_U0J&^jEw z$|N@!V~SIq=>28yEzdYXVYSCLG_PoH!Ll0 zsC&AtEhg_fy$OI95PhUiQC%Ov@i%_5C*6G0VewY;PuKi^{Y=A6OzY(BjVYly-J`VW z9<0gU*`tc`a2G2zeyo_y@h?`qXAYc9zZ>70i{8ew;b@4PkBd}}iTBUVUJWXWI!EY; zA;TS*oL^>q?Jbz32_ey);()FGAUUevY;_ zaT$MVT)rhr1@b^M=F%U_v9OWyKppc5kA{&SW6aw)sktzbtj(OkK^nUy?iT}zckFVo z5U|I89WujI{MT4#8_5k#)(>{*PZX;I4CHk!?*oGq)2q_zYhBz^|3GrB=+5}*YzNed~$!E&5cFhL&-3!FzsQ@_EOHC@C{$iT4*1<4&9)_qQ2i;$N|MltqW5}n$ zb~jXxI@mvWn(bRL9y^!xJFsD+*?L9h(sHT7B=BMS@UMUW*|={OEV@2@MkZ zfBexU`3|z>LH9i|Hh0GNH4G=0w7aenb5@*>b{XpD;Fvs-PDd-u67N^_1Xu`Gw|i@a zrDZYU&(xeMvcESV+u=R8jrp4|KwN<(pMTYB8Cfkw=tBD7fz)!Nsk!xg*~_C#5ogUB z50aGqZaQwKQu*E|GZoWA$>(ldIkufml!veCm>yp|=z*`2oln#lK>MO`be_36xeC|B zp1Y|MO;b*bJ{IQ7M7G07iul|6eE3mp_6hD|3PlZ9JjqxM$yb4=J1rmFY;W7t(b9E# ziKp~M>(ahQ7XE+kYI~|8Z5->0TsT_2-rqmo zhxz#Tn3!)ImE5o?h##z%i=CX|dYo$3Xc1T_uEE@S8wXzW#Ns(MQ*nLKCE9qmv>CQM zB%}A#l_JD~Ek5V`iBcEZ2XawbEDE6~A8xCc*rSMgQ-dEc*Wh4Qeg8l|_+O_M7;;K_ z4q+Lm`p1kb-vl`x_w1xUbboqwu_wJOG4B|^ti^9y65NIz*cnQ$Iz47BY*6=hpiWKZ zyhEbhCJsN8-{5kXj?jIWTQ@;W_>o`TwC#_vZ26)NCZR&3wpUWm>V68OGxu1bqN_

FcZ;+!m)=0*A!M^af8enjqQ3Ut|;QL7H6xvnR*~ z4BjklDAi42{(^@HMn>eyRV3#rq>};VSwMLQkxo!a?VNxRXlqQ`tOp~a2Cp^^ZZT4A zYP3Ms-s9Ff2d-!mGGJVUNF6lJ$Po4l0nh=9o}}wJs5G&4Y^otMs~|8`Z70+m=s_33 z*$2zH7%1r|K|&N&4bW`vffJC~Ptmi5z{vy9bo5yu;YNR*<%zbZg^1nzRRr46*{XR8 zlgB_n#ruJbhiR*J-X|M`ZY%=T4i>n2+q%;^uS&sri`>XoZL1Q(pT%E0$riH$A7CH= zAlC-WVDy$+aDZFP@Y;Er7w53$W<2Gi&ZIxwshL1?b_seVnoiN<~`6}S6b zF++%d&da_(&I&ZkW$OFaz#x#-0CdTH*9jDwf*ypwOn(VJnAM;-kT5m zf+Wy#MEHk1@8pv$&=K_)pGf&L6-{&OBq>DB_D>JARx%#H<1%wRuX<$lhT)oLc+4TT zd#A{3>LXuvm8*)HKOegoJ{^t!#-o<^F+_;V;>6=< zp@*msC`f{5b!_YD(f~%`Fyc~??$(Et!)B~-dvpDAbF8GTgV-Rg?_nZdILMlsL^N}x z@RB*}d7ak=?0?WD1_|!eJ$8j59`{+s@0ajd(2hsOkz10e!Ipy9391nD&q0j$V-pkqow zowd&oJjw7J&YKYVu5U=GD)z;qx3xR&o!v|W*yB*-LoT;95Bh7vF?5=p(%L!CeX2Lh z@fr^RuIE%QQ{QDoqqB3T!aT!+9ubsWmY&V_ zRo=a*LQBKKDw8wGcNUJfdFirD&26D{RZ<}1EyR^g-Mc#lhJ|R}7c|4IAxB@p%s=6H zJM`I_3={1L3x2a*J72L55?Vb8!q=yrcKa8s57?ub+#AB+9(vJlA;2;%^c$m! zQoEl##6-wTw%RT9mWuZW`r*}0baU`Z3C63}Pe(GgM1Jk}yi>w}YrR0cYPkD z$K1DEzeeC|gQ-|JBS1m*{2fRvUGHU6WDo-_$#Gq-aGtk}h=?!(0z{N93l27s^p^c|=g06EvO}mSgh^Nct`DVGJCZq=BrvXw!RLk0Qwqyjhp(tav zWsVnbX&>$n#2>!a11s^Bn40BujSWGD$ArM`)nRBbkv3xwcs%}|Ojk&>XW-pznXJbAv6pIaf=Y88GNOQ^0%o7#!IjOz%JlA^5za9VExK;6#qc_wke zUojoAru;!L%GW*6GmJns9EK(j_>=zfiW9G|pZ{Xb?Ckqqtmn=m0@K<8qF!a|-BpBxu}@^~9)rE}K{m*5S_HAL z;okk4>qnyNo@;GHo_&7y7p<7=?ce(A=Q;t{2HMyZr)1Xh@ zde|>Y-L}4H1O50Bs#enz_)4HMNI{)kC4#g zUMzq#9!$`FvxmEx(2$-INHdzjT|MkG$zd@HoW^%>Hb~bJNbb(p)jrDk)_i-tdxq(2+ksjpxaK`oQp2~2TtTu07_GIOakpUj~7mq{RhL;olDjCmo7fHBx~t4 z2rGvNAvDG4i8`hO;Syenvc$@|yB!n8B*J^7e;$0=hlS(d8x(>maaJOHSD z*b>ra&uIAjmNm^1ZshvK`%mLj6B-mUX7I#|SE0*AMVc8PvY0W%KZim}UtnFx!c^pm zK&_CNVLSIB2JPMrFcWZSEDrJo@2!+4Eq>uI=I^gNQ+kw8vQI5#s z<_g>uSW*0CltmBbCc8ffX+~{jJX|1CWS|;yiVjQnSV>+4J_uN0_q8j`TYVm8#JqoW zj`b`c^lBz|;TdA@cxdtZUfq4U`TNjl;dK4MYudXOD*)b}$Yi)6=xoGQJ*FYzQcAw0 zyC;qMi{nnxYmB&WjBS|F!=E$iXb(eg9y=X5mUZw#@5sC|p)r6_waOBE^|jo2gL=jd zQZFYZPbeMYE8LlWc}G0g8FY&H+=N1`%YjRn2O!3MO1#AKajc{#t%OvqB!SG6zNP!* zT@kDF6kN1rcvYkv*Wi8I^W=3bv60|8Wlu@;8yZgH7sC2Y|0)BxMDhDgE4Ut6oAu4g zqJCgx*onc(CjRCI<@e;8eyy{2Aig**=f2YZ0Cqdgc%Y*3y=RJWQ-PPrz5jHpQEV(p z$oy-hZ7ev7De2u?6WU*h9LyCmX+tASac+;fE?0o5lOYLw0%6*HW@h|+aq3M%LEbXt z>7o-{7kubP|8r?4GyB zp#uIxVe}n5`Lm`+M^8e!SRWzj8}^81U=^Q%H`$UHVGxQ?Q<87YTl$y+vHVbCW2E^= zIXZDIszNuwN4NlOZNwC}agi(PzNXEn51Y6Iz-~u* z4_akWNsab9Q#bW*$n*Up@g#HKdGFlwK3`__6SvSYTA-vkNrv-8G7h>InC!+>2_6H|Ys@Tq#75<4lRXwz z0^vaeG6%Tf+<1$zFf>ug`<&0D&{PdFMZ7DP*rwO2I4x68jM5H|0)vv)sKFK~g@zjy z_sP%Q@JZ(>J5FRw{a}pqyexyCg~B}yct-5FQ7ENZ(#jBCiYhW4{=yty^q1drpS&>I z(IvEZ*pqKRjstQRani}%U0sn}w8qa)?BRKYII#V_7j>_NM)p1)WPJXuY9s}|C~o(pU-82o@V!8q+Z#<2sQFSuT0Z)ZQ@fbF z0l@$csuJnhqi5)42H%-H`;@FV@Gfel4H7we<#-^;7lt1xh)-v**dBQj4nr0vf9`6) za56+tcScEzgOs}fK2NEx{r1cA-a9$0&2O%12Ak*k@G(;O!;<^?&bMDaHzQpnmy*8C zG-v6#pC;%o*l3qX_s<4OlK4VUscg_nN%y0ep4Y58Fw@#EZI@l84?Wq2S+3Y0pBxyU zXJ1bdvjo24a2X3)|E7blR-%ghkny$ON6#N=Ma~@VBntWZD=v~j@Ux`|%&2x&&9`^w z27or-9wa`#f;vyF2`zbqGVFkNAytR2fZqoqdUmIi0i$M6G6ls;<*cy zmrUK{3tSUYA#RU@-;5uJn^*Kdi9GL;t72`}$naKjn@ZBT0gTD8OY?aUqwBfll^5)I zHdJH6Hdn>Rp_M^!B`GdD!DR7{ytn@3#ZQrIcU>P9$MQpNq0q(T!NrPo$eekLm-Kg_2>?(yh6Wy_p4&OF|j zOsz&-b-oX>1lf;t#~dcF#Y^1iq8HgBMJjowe;tG*k%L62o*7FJ;6xlV7kOHPbBlSJ zTcS8GR?N$zG`psAnbN46#~CH-9NmP3r9DolL0UeTBzGjL(58~D;tQfHC>R1iSvVDR z$q=xiKJMM%_apT7x@Wn2>eT=FA62Q60otBUs%8a6ESY?!9be@OfwBTsuHCEDrn#*T z;zm=ZR&*Wap62?q62FB_Bj{SsXE7BIKN3(0uJ4jQVqe-ZVh-s`w(sjG9(33Z**7mE zx%_e@e?DI4u)_W`Sz|?U(rZOYtHaoHMGt50eVgdReBXQ8+;&7E*wmy`Roca^fsd)r z&GVKwg?jr=MykE@UhNQB3Z_d4xDjeS^}vnwFme`Re!Qa+^1#+qU?mPTulzS2x7>xQ zP;)}&^-H7?&4OdR{09z+V`qAr-mC2?;9tPx@8AqLj|6Urki8&axb~rvQ9rq?WePB_ zXzr%Kq`-)0j?>TgiQ*FWwa76&dSn-b|epWWVSkm=M9i+9;4GWzZ ze&VF2furs8sr=}LX@9%Z`8z=~nY!gh<;C%@s>XFI(OgEUn9IGaI-@QqbA>ULY2dBr zk7&5vy^n)V?Rz2_%qlrU*Nm8@UZ`|W|9AoX|2n;7Wd5MdBh^0{;yOcB_NUtrj7~G4 z@cli!wn>nQ;%Fv%F<><$6os>Om#Aa~{ig!4XLBD~LqAR?L8qyZPa9}XvQD$dV=wF0 zt!zx@wCD&^lt?01N3|fcuHfj6t?O=1nFjuEEkH4wM<08vbW*2GbpHH0pYmRkSK>Hv zv)luApKGhLtySs8(0$+b&gwL`KKr?gh>+d{8!B}R{!`H@LzFpRf?7dDIrGn*_dj;A zH$c4PjIVY(YiFLrKP#ZW`!C?3PQ4WKuRTe_)B$ITzi-}VF(RSN<+#r<{-6Hw39iyY ziSTM`!ry-|cxa3=hJ86Yv;Vh`M#3WzH8QTxwRz@SS*JgF3bN(2!ELAo1h1e8vtkroL-X~_*DDUE;@RMkG+|NDM{*>XR-P7X7xBq2r{hOy!go5gW{CY*FBvD$O(5v_|@MY0g zu|Lnv|M2vrGZ}e7i=pG&n_Q@FM||-TAu&#sDs3m>A8@sy;}SI*a%83;e-~uo2WdO1!Z9===Z$Z z)9uSWrs2Ic4#Zwd4*)bQo6BGCU1Ihys8)>qLD>IZbIf;mF#E&@r@Z6xD%ZR@-h|4p z%)Ubp*Jbz%>tj?BHRk%IZK_&BBCo*;5Zb|FdA@HchFx*xoah;VP^zh@2LKe(84_|Mf=walu86XrX>b z{x!Rph=}vuG2^*YleUHRyABUnx_2cW78G1FlDm~6w-fnS?o2kO4(r+OrpR_rjV8hL zgiG>N;a~YN4lM9t^gnu#PU^BZJyNJDmkIJDo7LC(D_4JxhK$b?a(JMEw^iw}f;bT? z9hYZ6kn+7}RnZyGLhZ_HF-WlLX!sv1 z8G3R-0fEveg0~i54^FV3{pcDE$dL%~gdNR@6h5rq>LVNMCkO9Sm9yb<>2pS2$FpEw zXXJC{e%UZJy`XCk`ixDC1pddy5j3fo!-?jmUoxCkF;U((@^~0ub$}}C zI-GlR(k6Gp19LOrf^0;=a`KC{Vxx^OJbCS%DxK;u*i?gqcZiCBdC~J5*MwNL&3nBN zB_aFcPweQP`fLGeC)gak&(Q9oHG^8SypO;4^-OZ0NE^0E@;zpLAU$< zop+7W-=EAZ9qa$0QZkR|zMuXRkwQslIUs7aM5kY{aiT=flKCfQ!TSwMp0E1TRD(?r zNySn7V=`QcY)UdA>oLyitPg8a(-(AXXRxCHwT*6njxiOTR@522{Q1gI9X$~~1T=x$^dK}K!L(hlxx9;C&;`Kt#I)7paGZ)az6caRb z{&>wKimL__SXUl~J43)8Q@&O6EfnBs zLIz_1mQCe?Fk*~Y8K?^myOMnfRO3?3U``o@*yHuYhnIkFXajTiTEX0Ma-V}$SS)<+ zQ~7qoMkq*b6mHy)5t?Lw`~K^LioWZ1Z(+bXkb?xmHLKdjJE>RvKwu~xz-O~u(?kfy z@r0)cjhBQPz_F&2)&i&<8cE>qus;E)cbD&LAn1idz-N#z>)UaOZUBu??r7KDpF0G~ z`Y!(bDdCRj3uqI%gMq2z9&ik@7(;*wBN+TrUh06e-Q5c$7+8;k*{t{GDC~gEZRgqT zk8M>!G@es@2;la~HU99#1Taomz~$qWqxlmUTT<>S^d*!QF3Nt%*abHY8@P*7^B=gK zp6Q_9uN&Ga{E#^g%ooH!rJmB_G{e_LP23qrM<@J|>%?K<9N+@6Yx$B=%IinJdCog2 zSoFaI)T4Eq;o~)KVb@x*Xh8j4dd6wEw;qoCdh{X0GwjP*)OoqokKT($&L5s69{Uxs z8ck_6WR*PxVH`RMYfy{6NBHPgpIMv)nK%>F5M)uaCdmfA7*NeTUJqSB)Z)=JJwiA|t)yE<<>N%)IAaRyXV7~mIfBzuEqbws}v zG&OBNv`aL;gNnmR_+!EcxTcYj5Pk!7~!y1Q@o~nL@@_u(Rm11*RT%QaQCpdfgcV!Gg}CT1p*{n{j}Zp-_Zk z5x8W+ZNN+FPvU#u(4`|VM>PxFR)M#Oy`G>je+ZAR1GX2rnPz{hCYM75dh_M6hVN_; zhHRNplcm^$@d73Nfy9q!peJWWoq7fbPOn`fmUI1uIl&`h5c#EzxRkzP8K|G@8xNR9 zPKZF*rL~O}T@F8GeI@heNS#IStJ*Jk1|`V!DC}08uLOEG_}d)0HX{WldyCyB;8l#7 zDa5_PeR$!#IbKnboqFNOy@aeE^aByMt&CBeL>P=&fT%J7yb>;;9Xt+Lx%2V&UT@rZ zul9I_3Xm=bPq!!E+vS&VkFEp&+gK|wG||+JwU>B7~FXw#PbGCXSF}X%v?PLwklk}ec?bgK~P$ zOmaiWB4sfSkuA~?c$d`z_CHdIiFgc&%@;f00DUfsq=~nKOUrVa@M6+ZM_h?dA3P*H zK~i8H=$YVstXMJ z1CWuY#TE~pGobQ4i(K1>DFmJNLLj(LGV}sz=^?=9U6_5e%`pb9o?>?mm>LN{zANVG z&W8ZR*z29hBVI<9qkB`e!`bF(DB{Fy8n{&UsQtrkef$6nh8!WU0VW^)~g?Fl!`u3L-*B{uoNHA`6rt$W}^E8aC-%c#yM)@@= zON|WN56DxanWCzQdBVxa-P;}IpUB0wW1n7;A#rW@vuFT6m0KTSzo@if5;$41uh8(q z+g6A2V+NRML=nwgsmkL15JNsCg5$ue1&Y5|JBQFui@B)tl`INj20-bPC zwnOUJOot36fuZ8f+b|-6wG&4U56#v#r{*0BL**h%$woC3tsZ}EpAXysMHE39uz9_FJ36~hDYE^vZ9jY2E%M3loi~s$!-!WQZS_L zxCN^e;F$zcUyXqo*V8Sfp=e_~^jXX28zyPxyy_P&l2GF=fqRqtAOq!N$`AY6?;g9p zQV%g^E~nZ)rz{DxoZ#K~F~#_B282kH{{XuVMN_nlqSw>_;JSA8yMVqV%b&KD&jT27 zn>OkUiv|aCY_rlHooT>2p5D-3HFle#c*=juM6=MG{mD=b)*9Gvew!>{fsyv|SCJ=u zmA+-i>+5hYAK%RaEoTeCWp%8Qw^o-G)W(adMk)`QkC-*Xul9T4*G?9fgwTbZi!^0b z@<-IAJ8IO(@bL+8>ff|8^Kj^VN{bNCt78UtmL3HmmbTAF6Pe;YfXNc=704%yyT8mU z=8>PI8)*-!WGKO`L+8B@4B?Fn96A1vPIOh*F!w8wAvZp+s02F46>7{P)cTQ&2%6G5Rx zpT~DxW1p}}(Q^CV_XOUj*9of3XK@&Lj4Z>k z>lM8rYFyf%xuwaZ`BW}8>EP}Tfj6?-p8#fzl(#_gwypy2BWw)zq~6%{3PUUR)xZE? z!rE*ivhb?-@H#74_*`ju)d5(7)LP|IR<&$Yx9f|Z%>>F0QnI%J1_ygdpa6cmK)jX7 z8@QD*A-zlYA&lm!uk|#{W%nz~b^NRvT*+37Kpj%ESkosZNL!63jYSx{nWq})+H5a; z@FkCDMMN@A@9K@rSQaPyt%w;vxHoA<`5P&8pl^{-X}C;wlBms`IK!+PX);&Iv*(X+ z%IjR($d9d)sI6REnjqV_{g}n<`eVxrHr?d1x21IvQa7SbKfPgkFUpQAu&Lt&kB3>#z!&g;J768b8aoTvZRXu4s2az5xU8BYI{N@Y2$0PVOzKw2HHwV7|ZRtBA&g zFdbExsIG_^B8p(~#tcxldii;nfi zcsmCOZd{&JO=ZsJ2M)=INZ^)XEzf?r%@j!94V>*HNJEotvFIIHgtiyO8&I-e;-bh# zTmZ*Xtd=aYN4>R&yn&4Y&E{to*5D)g;O*~?gYHgAzB31d)GvI06_WdC;M68(BA=b8 zUwsv2>G??l@4OBaU1A0ImM#bf4S$AH?0?$(u2wV$kZZ2y`B0(@dXmCCp;;O=ipAI4 zQ#S#;{~WWfeo1eomVr>P63W+7{k+Ch`h+1#l26?O3*& zu#!#w$dUAEO>KKl|>%9muUhd&_-7}`td(D-{+4|w~v*!!t<7?+dRiYWO= zcikHbuGgv?YXlqhuvpw|3*LjkJ@8&EWAffk*2;&(KT|3YbpsEH7U{rb0{o{`$AGdLQZocCm7G0II0Yls zomM4JTZw{Gtj&FUwLIYP%l!&TS+N%wa;Yo_q{*dGyn4|MthdI-E6mI=-!T*9Digyb z-$XgzS@RA6&hQ?+-{j5GTB^XmkT<;9%)@9xhDsR{ej-?INzdJ8ga5FryffW8R)H) zU>E{}1y=JXL2twU(pB$(!wIq5NwHkVAJ*OW2@o~)?WZXguQCY9((bguG3YC`-(=-s zw&J#^tQW*V?6$UpQOT;P+)rV~TGW3fYRZ~^JQ%k~d1&fo3HPl((m49)c9iWzsgWhZ z_axWUksj=kMc$}>Sx${xH7#e4;F$BLf!=M$erzUNw#}r6b_AHq;C;pm##W=E4U~U3 z8}#xIf``_WcyAxHZoS*MJe;lh&`WEJ@>vm6ZtxtFRtxl)p1AMySDSAb)Z_fRjW86r z-CChj^=HN0hm9U4t3$bb*jTN$7SYzM2P$bJXTWH7^M2xqUmWO^kjv>#olh9|hlrZ% z5}iAy?tc!0;vE*MP@BtOyf_nY;Z`}E{<>Kz^asP5rE30af+1u3i`@x%(i)d9eHDJr zQVUF1VvefNA%^%~kDSmf2**D0GKF8At;Knf@+81?<876OM+?Jwz|W05#hAzu^4S-L zt0ywQ4kPHPOFv|8aWb9ktvT~iyqUS(k$$q<$C;wiwD^&oJf#ZT*#tF$Fk||@v2(wWyEysX$Tk>@8vXzYLT{fxDgzr?AzH;?6rG?&2 zfxXp_=|jkesE5(g0OV>dpJS3s*NwX8WxmLCxmoR(SvWC~)ljo@cX{`tFOTsG_mQ4z z_p{VKoV*#wCgBU_>?EI=#Im=5 zmISXqV-RSK1O`a&Ydf&XyA1EF1esgxD3maYv7MIm@qNs_Q!l#~vc4iRvX4RDbDww( zcb;>!mS^ciM!mZGPptZq1m(#yVlAPnRDEt@brmzJWFv07`}#lK{c8>HPTXF)AhJlX z!F%Jhl%yLQlHi{rXcD@@y#_KJ4EZ##%{vZ*n3CTpo7mfNNk}&BcQ!#zN=rGXSlrq#0E z_J2~nD|YG5Hf2BHildd9$xUJ%Blo+E(p z(jVAl-0vw^5ES<>TMC(;Dqya^YvER4@Nbk!Vx4K+y}|`GBB$mywq&tb(5LSYY|21 z161FHB|=fR8ji_M4G5;gkpjB}&eS0yrqYuFNC5-CgLvFzIesdTA6!xnmjR@jJ>qEv`^``5?#&wp+d z3sno=FF@`jCIQo^Vn}xjHp0J^cdA>`ajsd$2lL2lr3(38#A8)6Fm5;8IirI`ue-yu9 z=fP)t{Q4;U^^$KHJMf4|+WwM={p))F=Su{ioH2fx*m(paX;F3fW>5Aa$C5*>8&b>(C3pa&W`O0}i$ z5g|f=#Q(3CGx`Mez09Wn&yKO4kjPS8;Q8no;{O|!`(J+cP9fH73Cl~B3gM<1eUzdW zagGB{j=!>O^-=R9OFM#R?`{-vD`O+RC=C1j*LCv8H3a#n%x06hM9J72+UI@y3qJsD z-~V4}Qb5?U(+dBUr9f09fkxtGH&|cJ@?Y-J-(T2BqC2x(kzFx=>6>i{02e&z#%&+* z7icZ|pjY5ZqUK%l{?a%9FIGAzAh-p>oq*Z8t=eYf;>Gzr4>q7v_5^7^Ql@;9B0zlJ zqYK+F+<0^M8rk1{S?`|*qB?QGZwNsOhkx1c!TSFqGYiN7tkcaZ&6;9O$rEP#_g{2P zKw%zP)-Uvp69KAlms9XJXpo^!m)HeI^S+CQlw;%Cp%BLPZ!;?MQ z{@w;sf+;S}_6O7UlH;LiS;3GE#j2P2VK38NCkQYd5X9Ke>4T z`@P%2>SxHR((Skn`ZSM;YrhVK)W@i913si*|Gv2X&)YDniyD!#4RJj|7_=0nAmpnY zK$>04&^WHS*=Gd!P-Kds6Np(Lp%I?6&fA=WgYY900M`wcYR_mX(U1bJ=tJrcKpljF z93cglO8qL!2|vy$z*Ectpbyh+K>w$i@(-nR37>;-(-m;P$RUm)@X@UiqmHuM8*Ig@+wURzKZg1j*X2YOUoaZL(vf5A&|XU>eYIo!Iwn zo2eRKt!m`T-<0ytB<+#%*c?Cho@+#bh5%t(2>x;86-(zFh?m^dafJP+vo*(q5NZbV ztY3yVR%63RojwDBaHnD=o(nf%7@ z80I21kLfJ{xmjg7gVoX{G{=A^-uhTu>J|SA?#Hyf@L?bxGy^Go7_ItNnWVK#3qamW z7Yd%JXckYqGyLOj!vN`-mDp@XaDs-|4eS-*)^dV}%CAZ-r<(OV6sNx0)pz*O<+Gy* z6MWf78~4E{&n8X;vejkBD^y^}Biwobvl4zdx+30ZV5|q6@z4h7+k7=L{ z%nW1|4X8c>LkOdd-g_~W0RjFl_3p$|gL%<`!&d3%vMUFm|K#?}bZA&Ba51Llxhfit zFFgrUqK18|SjqB5A7I<62UxTzReb?S(VG|oZ{GMcm~$rO^OL+mLv_xA7Fuw4F@X#( zsU|6py#B0r|9P9d&Ws|925cki0yN@Jhd?Cu^%0mMq5rOK2jLaia|q%ZOOWzPt;FM_ z;3ITunrbMkUw+9>#H8`w;hPZT&(V4dp9>~#$mV)hnt)h+Fwwb)2ZJ5z!TX2UK%Z%>X}O zH=hB$w#mn9ab`Vsx8C+q@NJ3;K|L|-uXYhh|2 zW3cD?*eJyFUh4*YIXiah=!<5LIxwT8EJQvl9w8wleZKya(7qI zKXP+$bsU!}g9c>UXt^u41Y?ZhgyuZ>w2;0aN1|bbSspCJW+*m&m}2Ovve>#3Zx_@6 zdjzg?pMw9o7F7@#LyF9`?vtl%k6&mQ5E^(x418YE=o3Md|9t8m|pzW5gCf*t8xg;Q9dG6Lq6LR64jW3Jj#X zfo)Vjm$2)RZv>ayZS6pgnq3&-x?t_-qp`p}cI7+L33;Rud^cuZt0bf<`j~oQGhz}m zQp)vNIF455Z!pm0;=-amI0510!u=g$qJ7Tql5gGSyx*rW?5$J|?KpYTKf#L<3a0P9 z7EUqfG$8uO_x*<}W#n7>?T=S>33rSU0(=wBW6?Mm<*B54{h?3Og-A2+LwwcjHV4$aVc6hZq-{R z*kP2}!SB^Odbz|f5HUVkX)K6*$YW@h@2V1ejRC&9sL?(Q8P6O;VfObQKK{a<8viLL z7@U4$K2xlZ7n_-y*|KqJ7?5T_pB0%^wn@+#{wh2VvoXVWfE<-KtG$~ZZ}O@0ON!iv zx$eiowf_8sS9$E@4H1*|rSaa?{mVa&w{#G~Tpb$zA$@Fy#22-OI zwIgd3rcMF<`f4?d&8n2)f~Yff3}T8DMyB8=U$O2*&55e8>!FFsauA2s=(OeCx_^uD z-E-K}>n6fnK8(x^w5BTQ*hy?OWy%N!ld7d;14Ws-@r+l-{)hMAH6RL=AA1OMB3Or2 z$5{aYN^>u;eEWU(m;251Y-Cm4N(om}FDV9XL}m}Y!FnQlC&V>cATa;%=S;L$-w*KG z+-R~XSc(*3T)k%ww(pc0knP4mYZk9tl(h(VnYQM6yxaPWylhE!Os8Ta|EA%v#qbzx zOKG+vzGgR;{MKA{6lGW7ir=0sAf@{>DApDqSP`O`9(d09lXuL@6E0s@G5%J6s^X+t zD?vGmqS^lSb-x#wbz>4Tpx{I2^#&*!cD3v~so*wFVwyf_=O4NFe!TD_8`)B=`DMnE zMm@n|M=r0J7Nf1hgIpHaf-jrSWgzJww(%mJ25#srLBiMSVBY zh2XNVenMgn@i^NCTKU)QR}&7}d&zSt%=EVE{{K(?}^<&Dz z3)etdVfuM$U(r;7T6KbrArk9qwAIbfI#nv{5N?gJvDxrIAnQBaD#=hb21;pr0T}nL>GIi#P^%=dA{D_ySw{io9R6glW^MFA2XqC(Qfl;I zlVzX9>mQ#{=BDg-!M`T)^(;K@waOA&_|i)~MXK+b`SA*jN-Nba$xXGfE;Js#)-X=z zCSz8eveZr`l{pBkn$oC`Gq3FD?HAt=katsm@!kll&k_}q9B(Dt)BF-wC*x8{@<0FZ z&z}1?!c@f5z~XBfUCL02*GWF!P8*C#5v{e05%uxv5JTuQ#qzhGkRHqjd{6K8CUY(L zbMLQ{34m!5M#k`1EE+Xst4V3SiE4%?*?>i!1IM~1TbRGVelk&{M_!Cjq$y2SQm}HT zHI7Nh(WG6OJ>~fPRyxHzdsdIlV23#sDjP$}jc4R6a~>0Pp;p|CB3UmmE&HyiHGz88 zK?K=yb%B4*rTZB}B0mP_yq`j+U^#Gech!5q=wbW2ldNd`j4V%u_Q#0g#VTZ}cb&OA zpb@xGGt+7J)ZeskBRv=os8|ptqLOGT15%wQg=N~SFN^yQ??&UCnm$62<(O{3tjEnz z_O)Elin%X-h&-96o>%Y%L5S^kz)6$ZaMKJj@RR(2?Zcj=jih+Ho!G-_(grEVDBUy( zsD$1w7B_lqUB|$mO~W@3adjxfz+2P>%}E^@_`Y@OGB zOTs-lv;pd^>VUi7!24h~S^!XA^_!KwreUE}V((nhKxQ$u_;j#ID+SCOShGT)s?viD z^IbchYboDwYU~N(X)aWToc{o8S z@AB}xuV(~Ha(M0#lA2zPdKx{N@ih}vYj(-!r*929UVajm!Ik$mlJhXQR#iB57 zyMImaZvH!g#C%5zYt}?5JGQ(SGZ?7&xV`VN(ME^gp9-*kA9aP1)L-;xKvnPR$UI+E zj=TY@Muy%ECzGB92YpHcv+c*-*cCxhAMSa5B8hE%57G z7*QOBupTDxAhbwJ?4Nf~ctY=ByFy^K^&X%OMO!vwTpR>bEQxv-m*;};=ebb1T8BV3@*VCzv`kDzV}YlM_?>_g zr6~m^@-c5XuQr^*)chEX1SXb4BEbA)7svuZd2(&TkzYcH?h;%ERm{43@izugd~c7>-c$ zDUL32nN?*glJ9oHFR4+suUaV>mc%AAZqH(+hessAY$v3aI1JGFZgIuy{t)q!PUi?@ zi2mvxuKy$Lo8wo}xGKhN*E5KK_rSmZBOxQs)ZQ|QX~j{uOlmTX%+toH!mhY-OqGw@ zxu*%=)jR)Gr~X@UL#*{n-0_v;dRi4TB;2IOds%iS4&@|CuoTR(d2=UoZSdNdnZi8x zVL4C7cUgILu!*jVp)`|6Y_~)$z2v<%{W8Zk6EIw&s8i;9rf9~cke!=2(HR9eOx6HVGtlaJnWO(J?8g;&kdkb{S`+PiHQ!#AL-5obD>Yw=D@C3%>?fc)WR}(Ok%9y{U*xVh1;`r1TJaZx(5LHyw{J`OLc%?Ido`a}7Bpko7Sc+< zB6kk*Xk7zAA%7g?>ryDowdPMWwpr>t>CNpFLXRk3JPRj#rIOQ3OyBNs`ZtKcv+Ri#?jxA3Es|ppifwT_vWb`cpu110u*a$&|uRg$1%!#sm0ijKmZX?W8 z6dg^VCkg{?9$V-Ib2Tq2D*e(nlu%Uz)beqJD|$cTK#UGpe_qQgoP>jO>!4w zNcR+a1=W7rP#3rKk+b67l}Jxo)YGxImW{?)}@)kV%&(!;!JEr`NXmhR>5Cd+L6}A&P(ldA4fG!JvrtQz`*LL32-I6H86RvGVuWKm2)$sW$aP4_p=4jW zRIpTCcu?n92gJr;@-v8Rp@MWjsZOAR!3F{H z+J>4vzrs9tP|&p&Pp6jgUz>1$XI2x)Uh;F>;19pbLoJISehHkDGZTM%_3)lu$rX=S zmKz_8TpqD_z5=>k7!;Wviqe$_1m%B2#9E}(9t{L|1#AP4f^`mBD)AE3=jS!0Rbi#` zTikvWY%;0G_7K>K_diC{N{FSWElquXd9ufQUt`is@9HT~Xj`nf=5qHG#CRJGyPdIm zQ-rin-RM3O?IVVgQGj>3k2s@Gd><$uLVz|U3}B9ex=|wTP~Qi|p;}BS7 zA{b%}X~f{Pm}vzY9OcT2`tW3p1C?C;K0PGhjhRM7C^^Mj4N4qEGaM#7p7|3eu%)OX?YESN8Wn zD)NZS)mJz7=i);>r=j*5^BJCID?217;hPZ{gcI+VKjl^ZbPrg86Fv}EdIOPljENq= z0^97fpeB){mF=+vfV*~sH2V@y+QOf#Z-D6(x>FY4K$HdbGzej_r*IrN0cgidl6{B? zw?qy82nlQ#uId(drXK-aAa;|_AvW%&GL=yyTZXeNZsQaXiX5>1%`a=+f(FHoco}CO zu&q`>N(p>)i=NQeWzQAezv^3sfYIBxGC-b;%XQkH%mCeU5{v5nTc-sbSz~B~9(K2< zt{I-N`5s=+$6x~zDRk7n?}>f&k)G^xl2~7QAzeoBMLV$$SSXDBV2TBG+TQaJ!fwz# zxgNBC2Or}qDkp(9RBf`a@?Mh4B7qE^+S3`7S@PS#cUYr;K&KTgK*rds1wN3`)?juN zX8G89fmO1AXz)ViH>E=vpLe_u^{OWIO zEs78L!fL+`pujpjC}7hEaY&&`_9F%AVD5%mOUY3svdM4XRzZ*bQg`e9gIus1 zz5Bbk;mZjiwUTaXPn;#$s$r## zka$CyqR*fhwiWU5Q}=;HesT%Bajp*pl8cO|q2S0@>H~*xE7W~c>gZVIZULyvAj36$ z3-_gEi@ReYHDhX~ODE;To9CZ;uXq;l7r)b8)w6AA!o>~%&GUR!h|Dwe4w+(HX@}7w zt&b0(i11vHP2z%7`Kadj8xHJ4M?Fj4r3%5L7U6}$K)eltxT}^}sTRNMYQ3UGkiuLW z(bn06polaOP!=lmj;sWU`AO+w!lrMl_+I7}S#ahFJ8ow*ob)8dWALSshZY$^G6yiV z#M7_hEhxbRH3P#Ky^w#45>U=5Mk9Qj=E~yG_fA!w^_>UtqjI`)8CgrPpKb{b>;2Hu zFS(s|_s5Geqv34;F0rrjAD;kOIi~z8S$cWrcaix=a^FG3a*G-&&mmMv&4Ik5;vQ~G z)il$tyV`iBKUrTzNC42xB+Gj_!W2H@$I!QL^hqQRmzu31@Hf?6q>e`Dp0Ju6(*9lF zp$v8BEm1WB&GyFsaiz-tcBTHYkpJA}*^^oTnr2|DyWek#8gAYeiiZQTv#c}3*lI;F zEg*sLF^HUL`^8hhku9nDcf_*ZfZUb8pIFbYKN0MlvaFzM1|OXxOdFCtZg{yi< zc9+H^rN~^u_XjTI^ieTgfPw^|$rKb8Q8(@p&g>q9P#Lx0W<0vh+HSq>2iic>*Y5z{ z){gQOc#71`hrw$9W$pkmYqv^`!6)Hs=Nl2G@fbHj;W?#ptYgrak+=TTMj!=u2X28` z7g8~@c6+qllG28o_|#U#ETu_cipR|?j&%#{BEOY5=U#9cc#}Ax)Bp^wYZpbe;|RK^-u zc&V^BhfFVqEHVeGDz0rm_*bqlXj1wLfjwFuDAonA{nUCBD5euY#=81}Gt1qPfQ$X? zAux%AU1wR9?J4mZWIH)n2O_$vgyQ*(KCnq|UjR6o`~o~%C6v8jVhv0v6^;aRVg{nw zKBWj1T#AC1?&|z51e0IWgc}ozpC0Yd2~C(MAi)V60>zny$}3`W=T9F<4)+GD`vJ9n z8^AM+QVVHBA})z)pOr?SBno$M>lI=e<9IO3BHz9;H9(@ z6&~Uhy_!jZ%J!7C%LQxziVP{0!RY;qOVT!SuaVkFLM++K$ zx!iNs99U*6b~To5uM7OaQ4H1*hVmr(~I-ZJ}`@9LtBJU__x)=w_gbF zKPA3?0Twnk$l#Y7x3B6aLj0-TxQ+{GXmqz?i93Iu9DlxWzqYEN(A!rAb=n2tHoHY? zxVeNps=@bgE1+2~5sa42?&}laFsaOqfdzx}Wl5xrr*#HW`Ges{U-igMm!N_Tsi!a` zZ*Angl$7U1F7Wz-Uac84XP>JM!Cl6L1~=K0>mwJ{mNyhAETz1BlkYm&k}r{^g{Ars zJDIgB;G!gWTPK`EfW*(FbwH3pNsNr?siONBPTFX9;N}S@ZxS33Oa*1YM>oMNKMM?M z81P_NG9OwW@Z#NwtfBKm${Z-O(3p_K&6(T2u1>j`8uV) z=$&X6NnETihUjf9|UeQ)|i`!Y{O-i!gcJG7Tc`?C(!Nz!F> z{YT+XB24 z#MbnF>LpYrc^ddXDqLK%Z$0vKA4Q!sZ;Yyi#e`WzFxa@;UK8p5DcqheM(N4eQ z^sG3T(R*l)MHcbE2Iv!MOG-9f=18MgV~b)_xU%Xo_g*thk%5&OfToX#H))0vC4rQ= ze2SrhhQ|TTS`V6^0FtjnwNjVi&i37p!Rwj)Th1me$ClIbZK&)@obkVI8IaOLhESS> z+0RwXoY$1bEtSZJf@cC^h(ZaulQrrdw#!8$KTtenU~u0Ze@%-J+m%*$&h(m(A&zfd z6QOL@dLR`Z(8r=WZrsWj00g#tBrX6=ri9fb`ixlUu_AVV#T^1}AG!%G@S11nc#&?K zm6?RYUL)L~T78>r?v-@c$%2z>h{%$Xj~a!nOs7>*n{FkL_X~wdCUI!~cG%u%*U4;% zS5|me$U3__C(_8JgY@cLG0YQk8GV9%eCZG}K=%|02xaPPz6i&9Zr5u@lggNur7`W8 zOpqC{YyTNCW_FN|v*}>?GY7mo%oB3CK&!_(91ZvL6yab)7bN2R=c5sNQ-|oW2rG}k z*x~zipjBVk^#t)l*a2R8M3Je;`Y|J42NHA`T2y{Qt;4q({aDN{P_-J8z+i!CW@H2g ztvifU^KW`Gir4s60QSbT&d|U($)~gp66{D;@HF-on^W#O-3_g`gAu_*cBi(0RTNjS z?+2(hgaW}DCmdb`Csqq~4f6ifl8 z1+(W^@Wu(K6VT*p<;to#887^1u&HO=1+mGx`W8ugGvK79UAbN0SFwKtPO*09VJ~90 z3mff8a43)!Mg4?1Q#uFUKY{EdWh!tF%0Ouf+f5+Zh}+2&_l(__tja8(``tt4Fu<%6 zv8?#tG0>EE-3)3_o}EZBCIG1f@1oRe`~R^+_+6NdqM!!D6%+eOg>do{QXJ|}M0Eh^ zvc5c7=d$psC~`b?5cz;*l&6)?8-R8ca`2Vjbd8dtY4>T<-qD1lFiljiwjQXHeI<>h211M(i^x)Ztw`sg_(tm3&i^6~;X=6_1@cBQT z82_+3LF7rGx8n{Vfsugt=p^ju(}GIV-yZ;z9qB{!Z&vbe$x9KiIe2=Es0+Uf;i^*k z^F#M(pvG9O41BNsd-wkjK)}EJGf@Mmz1($|{}C8h8x=jWnQpwr%+5CH}h? z?N3FH^cwj1s#X>n|LvRp_P45%0#SGDJ9Y;C?Dv1Kw?F^(pTG5wANc>#Ww&HhJ>G%t z2zy$K_Blpk##r>9&*I--JNVzj1RxVThnLablVFDQ?mk{RwDJGrfmwT@S$0$m687Wm z`0R%Ng}W3ab{jm~*esEBu76l^|LLE%;HcOLyqyf+_WybH;Gm{q0ht#{ni>7Sc-;T= zJAZ%v>$ISP3iTY2F#QWF0k~B`s7oH(faPzi$$xrxe*ICrIB+A|#8+r5{>R;eR?{6N zuzbIMCbp{jD|atc3EaK8Ot(0_zp&Vg^3ayw=55tXfO~TdturQ@5FMYEsPj^+;%Np$ zwe>WoyIzIAUQWju>9XLn|8=R-Ca93Hf2s~x*hvR&;TVv3S)pVpu-r)p0lhT_m5;4Q zY%8$SMStf>6uO0Y_tOWNqFuZctZ{+)lg*~PN6ATAmoY9qnuH(U_m>vXTU z$TjVTc0s%EFtlR=1&F{c+fnwu-*FRfT+KN3xWK2UB8O_NQz}U0@qX0iw>C7WLH^ND zj*JO_{^HbWMVue$StQ*nZaN9IF7+fCqzMA?KH(Eshhr; zhbH6aK~8N13zy*&*E5e?j-8Kp))-B8nrA{Jc?bvwpNLA{5<6|yrF*_VkBbdef}Vabk1hEeu|qok zQp&q$W~B9F96p}uY`L=C!c6EMRG50a48tWh~sYw6j&dvBU1_*IXI+I>g{Ftr;P z>6U*S9ZYX+*EU(&Z#kTe6RJt$QJ1W50yErBVr)KCaHl;lz8F#K9&4xrvx zK$d_5wbovs*S5TA_7tq(pr2|vuTI;5VwB?mRIK(8-~q(3aYtc=nH~?MksVObGf+mV z9m(W7FHN+7nyi?-msPO(x#TOaQIAU zjM`*Uzof%qf$fq8zYjC<{s&2Z!^*>W2J;?s4Q-=ut|w$B*|9jGZCS?*@VamOxtGte z=4w7Jv;{ASCH88b(D0kCu!WV8h;d75zAZb=lxt1n!5#cI;2OfAr=NF-n7asgUqU~| zY0vGv8Da}Z%pA>i8DHWiXc`Fi2`c5WZ*IHK2+uw53Jty7hHq@W7R~>8b&UC_?W==6 zrp)1MD+J>5%T5??fqzu_EZi;8aODz=Dw=#gby9ObH{N0qKxlpWsDy(+F=U z6M9~3rVH*j0m3xeKm-YF9_91_r`&Zg0=2*y zC{vUC0>W!q=Z%|Bb4*j~{#*2gu22)#Lrt;sY{O1%44zdmwNTf)7R;kl^MU{mp!mCA z8VhN8rohV0RZkba_#ho?*8D!n8Mh;VX$D^fet`PEYh-%#1t-7reTE1ze`R{le0+He zuoLm6kHG=>;z}mIQNWqjGT+P2JN+J0z{_}ILCXKt{sswb+U7Jn>2 zq@ag3j|)WGZzA?k=kj-~XuqdB4UXAY>8zgiw3$G)T<2v{miy?jO&Kwvj@tYGc_kwLN=QG_zb@Tr@5D8_Q7U(rHAF7lSkKm6h63v*V*)s=_2PbZil?OULpDE zMh{1-?XyQ}6r#I>&T}PF3c3>87h;BoKL<^QKB-Iy@%VID%_a5intCbhq>5fBl0P9z zuK&4@oFB7Tb6Gf$;x+zRJa~19J9U7*sfo4z=+xeCHiUG3pkhvNn%T{0_h(jOy|lN7 zbL)syup&7@blHKZdacB*bb*ZO<<`0KuKj@J{V)9|-eL>)nzzhm!WyM0XudYAMYV{2 z93Ol$h!@pMEU@BzZqOXvn{A2sKGt~B+b4*$8DITT_Uee2YItl~T&j-{^Pkd~su=fMcW41u*O8z?#kui0F0+yrzXm$OD}ImN*a(qMU;Qqwhg! zeYi8-v~a`}HgL2`df!lMyYZIH-PH@ipkqLZU_!FDanPMET%ks5VaxX8$LD>024%Ca z9y!%aSk3{hcHEH~h?};i05O)o652797by^7Fv5r@twPa0080&{i1`)IZj!MOx)v~e zh)W}Oi^ONZ8K46lC{ps3-G|RD9x*w*KF3U|X4(zs%9o7A(mI2NV2~R+%cSy}$eOqE z3G7URSD(Vb7u*$ebf%>$0HG?r@B{1*du`gODq|iE$iJU(M;F)vR|k1ycQf8KtQTu# z3|byzkP}A2(ju~C8m)PFqJ#1dgn#qj(sIm!U$qi^lup?yB0imjyh(s$I3r&!_QHorhChx^V+r@syrBZ8bU24}g zE2R(F9frv6MAN_YVlX}*C@R0WyB}g)sD_tqF1yD_TlxaF%DrAAf4uQJ)}$(b1?Log zmldC^IAE|@u1)FmJwZVB_>JmM)LnVcItNC7Fq=#*erWBusu-FeWXP!|*~}E%Px;xy zy;8K9iHsZZ>9W?79Cdb8%R zY(qQAMwxHMdPX9t+PU=IalEelex+1n=`ibNMxxWSly`>9CiYw_-ejn@qFQvXV>Vjv zeYMKWOA*@>=Qc|k+3fa|S<^{HHk9-8?4M$Y%KKx;wV>W$6FPM~&)6FehwE6scd4Iq z`6Zo^xLWzdnWvXglAh^yK!21hC1N_5`ok z75>;F+&-S~Ldp|W_t(h!XAc+puO|2v0D%S1GhIVocjc$;ntmbzyia-?wOa+d2^{2! zIE~$>m-%u&$P%qeI5sJ?MlT9fdAh14-*)7E)HlCr<<#**po`-4Fpjd@MY)s5I7eaJ zKPjX#eUv@R?IVjZO9(^}9<1(q>Pk+aBue@N1{w0jm|=}hSn){qHfTM%=gh`_63L*C z6m2z2pw^7#S`QDgvLnD)3`0%iDAKoql|5>rF(b~kKd>(afLKx>^on#hRP1Sv zPDXc)y_7jBdBXq_cs)9{7-nC_Mx^`z=jKeio5)RhTq+^XX^Qs!%2EUK(*;E-M|M+@ zKnA%`wBknR1lX7uG*5>DZuxN23{-+6Uy|YRTB^vGEv}AV$Wgm+D~3VpQv3-Nxdvsy zcuf!RzYY7~eRBOC=cj0m08H8Y@6})4x>1@=MKvMGUw2X}MuYUk$n z;>bn>xFI>xa>VrI-@t^QV{L62Bh_2N)tVvSO8y^rZy8qQw!IG%3nWEBNd=UW?vPfu zK`RP^w3J9UNTY~~G%P?`1q>t>N=OPym!O0!LMf3B38{BX_jd1derKPvKmM=l{o;kN zSUl^Q^BHrDd)(u`m5C3UQZZFUMNwmmL3plV>EzV9&HaS>(&PBUYW6n*UqW*aM)))u9$r~Eks*vU=|d^rE8<) zspk@OT+O(8<@>Ptme}Ixd<=n+w)R@3wByA&YbDijKxKkVwyeq+tyU~=)VQj<-Mr2QknXy5Hl;2-TQDH``yU% zuCCemF3IK(!G6B`T0|-FHle~w0e!C<)Q5XljbzEUb`IqQNH%TxZH6vv=LggijNU1{ zSVvdxMqMPGENB;nX$z2X2w9gm8cuDh7>ps9aL;e25-U+l87I?P2CU{gn?d|0u{2E~YAdX5+tZWZA(}011f{B+Rd4X^YJ}xRh zdRzYbX1iG`XZejQ*)yJ*Rrv$i7q3MNh_4-Lt3A{-8lkeW;4_qb=!bioQh2fIy_px=;;Vk zLP9<#hIN#KD9F{*6%|N{*A?OiG-anAdmTxl8uSepTd6pEJ_E_|MW3c`l1}bEa9%lx zo2#bGtZMnxzC`1hjU$9A?LMWvQOVc9vz?as5l@=eRbP#fgk>&=t|J9%^}?2`-L72N z&!OM5wjb{a9^o`!q3wK(bd!oK7OXZi4Mi;PFB@5#dW&6WN>DHv0PtN+l|43Ey~ha* zhQj6|pUI@0T0XZg!OqAb(GV)`(N3P+*ku0YSEz{>VWW1&p|}zXw^utd_pXi`n-$mSdNbElc*DcX`>x&L~;0u2|}XHxa-{K zHE&BWl-a`r6^O!&9}%h7^Ny~x1LrfS2YJ9^)G^BVcwr5S&B^;6KQw7Y%7vS#6rXx-7EE=$812YZsqrinZ@87IqGIzY zILTQz|J(2pfkcbSF06WRv5nldk0G_QD*If&b4K}YH3#g;@J`2-Q*v%b+biv?TB|%A z^WCNywtu={M=RnlMy@Or8@H?=-NdRWM!rIST`;!BOh0@vq*_h&%td;c8f zRj`W`DyBcuEbjMlF>Lwx_60O*?Tc)l7H#E_c&CjfZ!eo{?#1&*lv8 zQVEK0uGz*Mku!;LPno-%&bh9RwJZLZwJF@Bv1eH-z0zW{uiZ9gM2M><=Wv;CYqOs~ zQ{JEgleOoEG_`l8ejWqelbA)CbE>!5qT}2(4Kk~`*SZ@BTUyyj6~%VO+jLV1^f?_Xzw5HQjw?0wxl4$hZPfgH z^)Vbt&$I&XbKruz1dMPY$K(P)#PjW9Llf>{I4IO(ahwG;t`0{Z`*JE(WXCPTe!c9+ zVx@&bG4Vn6#B&pTO8qP*GC4<N#$E|A-|SjaX9Q+o)bcmZsju z^AqzQAg{fw-Ruk;+56@rE>%s#<)9eyu7-DV9$^>)W^G>8TOwA%<~);>3oZ^F)N3{z zUM3$2gtKzHMZLQHy~ncsg1=zClYfPIR1qt#W$Bbwt7LSR1QKbS4*ZS4!Rz5-1^1Qy z%Q)Ms;j2&KySjnp7IN<24$Q%v&9xe%H2k7U84vULsj}&pU}CiRYIjRexp*#h`%`Fg z@vPxekzY)v*LVKdowTQKrwY>nL& z7ZuTT`c#z?V^Ph00#7pO!1|g~zi^N~O?)O9)3y_Bkuij|$Pr7$-gUkmq|!e)t(Zvb zpc!1@GZxoqBuTEUlyE;@*8fv$?x%s_Eulj@eV0X(#Z7iyIg<+cxyK*&8dvDZ*v3YD z?s=R$|43yapD#RPE=^_Mjb0xmVJ6}vTyw<`7w@jxhTW%`-OXv$^EWJFG;Rzi$u`(c z=jUwiZ+Mb5Hz+%dK=Y|YR<^Xuvc(trtH)GscZ`YS>|@@j*Uye{wybV+SX-G{ zmndv*;dc(#eVo@KXX85IawW6Mo92;Htf_$g5GVT(&xRefM(~7RdGTHr9;<}vRUA_+cwKMcF$WY#9u_Krn1hv*$el*(zT1EYtQ z>Q1|`mfdOHu#`;=!&7a(SwW+s#(LI=F(i$_&7`9Dj7pYyVdsQj$gpg|imJuK7F%p$ z@>~TPz&*p&jRLpO>Zdm%ZdWMDTjf(Z1H}0(U zUtgi?r8w640Yf&^bIFN8P24N_6HoKDPDTU?QoLM09nz*}8btEt-NKujdm3W>=9m@N z@UE0YcMTenjjf;a$bA2FSt&%YJfZ#CAb%6-;QDZSh(_$vOUaKyev8Q>3QbZ9S1XfD z>Km67uFi(k(W~^2itZ7hi($NO7gqd$10Rd2*Hv#86P7dy4=ySTk) z=RI$3vYl-?XTfoWrmc#CwTD@_%iXg)H@~wPUAgU9wJ9hN9oN%VyjP{Vm-ED@y$Ka; zuQ~f$EtZGkSJYa1m6&Rih2D#`x(^MQy?UH~nP-kZH9@SjF?HS8H+O4v0W#fcV?Ss3c zF-XSuObFjGYM`j4ezoBMhYEbBx#(#g~3;H9w)elQ)(qoOGYM(r%jm z^Y-Jh)kk-Iq%J;CEV3^HHQ$Rd>W=m2{B6=}_2>I7nVVgOtV9*LMsHp?87^AEC_Muf zamiT^PM+$|c|D{lq=$%|g_le{435Q~{df;KUdr`PLJn{ma;5&(oQbyNhU7g{*krQq zGp89%<> zzR-3ngTs74+d*bP3Mht1Kl&CL+r^q$9lGMs$9pMHrd(iB=l-~ww;#@<%sbT7Ia4=& zno6Lhs)$w0v+2x>a{*MqQeOu8A}#UQ{pgmmqbKxMy#cGl z9}pGy^q%nd=DjLv{nVO>Q#T@6Onlk7u*a2~u%wIS9O*ZrB<;4WVZphB(}}Co>4gWJ z=4dy5>>ei<*;!~|GkcE}R3+*vYtfmI)PHQkd1Cb4q!!U9o5$tKXXD(`0`ewApD4`s zR$mPiebrRb*6+}`kDI_y_dHH$uxTkR=1}BDtf@n>hNf6V!0vLsc2;iDJi~Iu`Qdj0 z$#NYfO`_~hcFDXoD%{ncW&0A^@;cAL=e(9nwAHTGz~Gp>`PCnV0G<6A&2K=O`Y>^5 zNWFNz>Mmje9r8?=-^O#E?JPRVAloFuEhlAgrVL7;Mt4%8B8*m3gk|fRZ!Zr7rGa{X zf53yCllgJ^w3Hyj&aAkjvea6G69mJ74RxfSmi=&O*Nd7tg?r#A+dfDq$`6GKKFpgVL2d z_Mo)w55Y@$mg4?#oR7-!$xlYEZTaD00>;;#dRzCxNBVxY#VLnV?`W1GDjSA)|#k>o>9E6AbQ1qK8b3m>0SKS30fNRSMB+W1;#2wz1mTzLFJk((tr z7oi+w4tjPyz`7M!7Fl20{*jNv3x0j2{E>vQyC?hCHzYWNtf+0w2X$c||J=;~^`8u0 zO2!fU=Ci|ozwi40`a>L~?+B0M|MLrNWBJhYx{yM`EC{&AQlBe5d;j+}?-Qw+o`lf@?%{_X|;_Pq}rgv1r!znYfv{@Mcn_T&Ha#-v^? zU5g6;v>4|zSjibjUYy-ir*YwbE$`38Z>ANgF~CO5|DHAc%U1l4>-c`IhQLulmeJwf z|NQ*u_s(NsU(9UwLUC=yXITNrt>XbqFN6v%?s zcQ-SXl>(`Nc)>`G+QeiNI7HMUw>ap+mdb2Dc@#mx|Kg>#9sq)Me9v=f0rF3F0;(zY z3b&s#jJ}0}Ei8wh{)bU0r5H*UVNM79khXr#VaNc3y$)%zpk~{{AEw!MN~eV{)!#=R z5P14l)D#q(5ZLZPhcRMVf(#H_oIi(m2urbE9*m)y#cD{A!^oG;It&q8CzT|UBOzDh z_wXeb(Y;4<#Aab+h=Sv6#QiX2>TCu=(a+VTiv)_kJ$rr|^wldqtia5TxFIwTv$8+d zharYzE|6OfUitX5$qf2D`q-!UF06{zOa`*;XnmZ-Zl@gl)DB%aAv~0$T#j&UOI#g( zKbQ8rd(ec=d@bwWZ=Q@QipTn|VFJLu4`%LaK{$k77kVe9Wr!AmS_lY7^{s?vWkbd& zOdgrw%H7=((@LkZTMK915^GMOkl)I+9u6c z!YGg#OxR>z&Nmh%(AZZjTD-D*#%SJPRs4MKfOhMTqV%GZ%ex@{zPL+S&2kO(aWnN( z^L{>}7gT>n5lZ(>J!-vZQmi_h_rTAz>HTdBxw7Dz#rMERkYz2 zIUP498)mY!GQkR6RBa7fzZ5^a!Jq}eZtn5r?cGd}YRWtvMsGIcJsJK+E%-U2QOkh9 zX)PE494v~^QuzJ>&7!$|3~Dv|p;8nJX^X$uMRHv>(2fnt_=J438?RdpljY&kEhcFb z+lyiQiy}hB&*2RvgqZ{l&B$-0#gfPYFFzzJ!B}>9ph(>S&h0NAw0iIE1WM_bT3jvK z#k}^8+=ltSZsM$B{Rx=oCPQ?6st?Poo3YqZZsqnJ+WS^%$c6&-&m8EOy815UCGU{) zxwewY6v>~MLi2M!HK$PDd4AmWClw#5lP(@*)Z~`Y?=2(T)%TFGu0;qO@g%glh&iY<5{gR(Cbj;~`HLYZ%Wm|#6WBK}Vf7R|rN5GdK zjRf^paI1t~G4#>xV*?ZWEyK~`zN=yHI>Z~o2>eCEI}YhDBjH<`zP*4q+?V^*DAr??}UB_Da z0k&%*PK~F02ZpJ_gtn^3ij;>jrXUcutta*%LyGVUQ5pVH6sxfORM?aqdMTp!iExzAZ;S61?-oFkiXeQfZbdSBv7pqd!x=# zkX#H#^irgH{L^2R`k<`^-76gHH!|~ z^GD4S)Gy)4n}|4BV#g<87D^_3%7)`&RGiA8b%w=I&Y%ydGe*!*c0_MZ8a zB3Q$SPA63L<-HsyE$W_dozcez9`;*I zES4Ba?AXsBW(X|1ei^wpj=?SQHWuE?`d@An;xxNE?0)ngnSP+IqbR+#z{GLx+=AS& z^|Ra#%JjmkMqPeK_N7_5BmTSfJ;WzG{L-M@hE{hxAToy(t(pB683T5>SH_h_peTcGg|1=z2$nI z+My#Gu^LP%#HdhKc63-&Ly2W-@_wMOw8?^8(1mfQ^uD+ik%Sn-LH;%R1 zV*+^1TRR{Ku-?orTORY#czT67DK;*cOZlvgWpgL}K;yACYG%UG-%fut4#am z)Ros7Ay{i!{b-9*SG$4h#<)+&R{cE;;vJhCfO)h7#GM!%?^E@sBTLLo<#A`TXCxzE zTzPfRGK*KRt?;OF!+L{-=BRSdBoK}+Tb1-H+k2eBsX3|!H&j%EK%~X+gK=-~2+T^e z?F{i|f;|isS>hYqu4MXmjlOj@_n8V~6Z2}iBD9jkH@&RQlrSUK-?4u{D=-1;z!GV3 z;^IzOZoOjE@$?-x5PfOaWSUwjYS-9@d9(|mw%*i*q9%l+I=^(nl-=q1fBj-CXnU( zGvx4fj*xI`B{8Q3Y=nydSJfpTF>>Qq2kEcX==37-fMYt5_bfU7Wfs>3ri)SojbvQc z&WIE~9DQQ7dE`!}XuJGQcp1ZZ(3Zm1<)W45&AH8Fg$ zU-?_l3j-BiBOEL5YzkISv*hkLIj!~&sgeuE*w&A3EWAF+R_aSBE{*+X=tY>K(IoZO z5!*!d(>Rfq>nnAoPg`3&3d1o2lv4;bHE};fn}UP7Ku2MsyY|CGu^Nl)F0wfsb%-j1^`?3x9nVoQk1JVaBu2n0LTTVkzjgDq*i5x1dk5Ml=hVc9^>4S@JKW1K zJWd*CHnH4jeT&M9sPd3fePP}?b8no(hEj)A?8g>H{P?RC*V}|A?1F23b&nU@F{Ed@ zrP-O}Miyq+*a>%Z+ilJGDx!Lx`8C1lqveYi+i2S)xooQ`J`~o-vH2d774A9WW;a}n z7i5z5ziPVU-uqR5uA)6dnGX#lU^m}Mr1qW#sKh4->Iy#Ns7yQ-`agf<${98kB0Y#) zw4eA6hMc^9-+`<0z4+p(?Bv_snsS0z#&*B_qxnp130==EHY{2CLm^Xl&+rvEk-@5P zn871tJHa2sR1&7Hc!0sXGm^4 zH20QsjbgR$dB9N^`9@ZZpEp|F+~63$Jg`+`EDN{>r0p?hn=Z9m31!dkA9*|`))IAE zrOc@3EQfdkyG&9vrj?O0(H|smmJ8tUeK#4`V9ClHq+K7#C1YYdDqir0l;v%q2e~qB zE01k5Z6k)4{F=DGAlK7w-NtCD$*Z5xv`#z3u=OOkI%ad4N(H0fNCr1zuL7JIraRgJ(5of z@(bR;_%n4UIJ<6CpYFHXNnbo}J_9Va`h$b+U>)`)oY*4UJd7iKc1DA^EY(uU+Ls<) z_|z5GUX~PWf3zjNMl?Nr#;bKpsWx<6#A0VI^Cjq!UqGO0F5@I1@tXcSGlil9(qy4!}!c4|B%ekn)3&M!bzVz4JXk;GM8U zN@xE)x{>wbM(VIH#1Zs4Nz8tDgrTM%KWKr0I1w~YElRA`#VDTnayn$eM%syEzaQ zV<51cY~!eZ9kujZN}VZP*OJvMwl59}A9c>WHOytW{)LDntEe>NZNhiAlzg}2Y=59H z8O{_@W(}UQ&Ch7rP+zElz_f(2?Qtjs)Uz}^75F66`B&cGTs(LL%;H5 zf31L4vE^y8`^S4NcO+O|KHSutI5yB+=KP7#HNJ*M zu$uuql|kHOO*i@y@V4Yr6;tLqhmU*M`RFJHSOhF3nGJO7_E}VmInhJot@cb2nJA zZ?A*ivFC-em}kp_E9ZM$enLSK7(#zxv54d0$xdC)4X!1rrRvqxy-lfxA8gem7~YMk z19GBIs42y36q{dh!E)2?Ud#TB&zk9>Z2mHaQ0+VWrsB&*%lcd(;=YqRN&X4!MOcdB z0!O@%%*|qiRn+;<|t0&{+EggU9vHWSG0603= zNsfuCWphq$TQIo@omb}&X$`lq;qAGKbx!+4HzHra_vY2NKJ8#Ri`wirZ|`BQ>npWf zc4G?8>}R(*y>^rGWf`Fe(IZKtt62952kHFd5jx~^xLrzt0_7&gyj&gyR?GfNAEV7L zf9zlzNP^O{($X)R?}s;U3rP;X<3!%&RXLqGpLjowPKipD^zF&zMhUS;ycad46oY-@cQ|`*UV73ovrDNMIZom^73$7}7v%ir1_=3AqAlacuZfl9^Y2;(mT+I1y+}x+ zO70ZXMB}+IUJO8nNCZO>+|i%PI1y)KzJAfh@zf}fA~3XeW=o> z!9^y8oi5O_%x)7(yDutIT&e3v3tEKAZ|!bPD)sAsS;qA=3_iqom5L{Ae7#x!y6jY( zauI~zZ=p4)zE?B3P#aNCLXC6g9gHUpg39*0a)06sqZ-DSm+#@xJ=>5V)HV7X z6bewi#6fH7t5!cbW*F?cN3|H+yY8U;iv3wg-+^m0L~*5p+{Zrk9mZF`-p_u%J$z@& zf5P)S^8f3a4|Fd&ou`9BKLKIA@nlZ+1Hz#PF|SNW4jkKWIuog*tDj|I_h5y3?#mDq zI<<(6trno|;)P^fPSm=wvRPAzD4t&(OPP%8ZU(EB+}M|*G~KutiT;Vzxf{p2KYsfdt1?^%B7G2g&vYwjFM{S58_+$XQA7j)Q@)@l)1?;8gX%_Efos-YKQ6 z_Rk8zZ!i(Om$3t$viXs}wpb|!H7dR8gho|a{Z`&P`*tm@MlF-Hi{ggOn+@vMent#F z9%X%d2MImk+VJ-Vsr|$+FFhv+GIEF+%gGL|0E)}^sQx4(iUYJVfVi1IovBA9Co|%|s~a9dZxwge zN6nx@PqsAh1fxtRAOSX{BYg!CETS;}%omue38Xy+0i0W`7rJfpd14S~hyo&24w)WE zA$oDNAry&}Mt$D|#ljFcXL34cH2=wBFyuYYga~A?}4Y$&ZyO zx|?J+*gjN*srh9ykT%Yx0=xJt z>oo6dPURgK?I6j1UwYFYu%q|br4}~7l?y3psi7l+#Ct=3K}_7NZz~2ldP&p1N6U#`jWO4W4L6YW~u`_S2j(V&$nc#XoeTju=R|<&m8Vo z+I4|)U*C(TBSKDXEU(h!phNIoGq5mOYRGG})X3t7_M9t#!Ok;!M3HejoTonMUI{d9 z1#t+v+0w+5cgH~aR0vLXI_ItD(ArbT1-upk$fDwrBkG&0v*5rxvrkSI4Sgggm(SqT zWM*G>Uv&yCEFhyd0mQ~R4J9uTl|FzHH>|n#HpmEBn9(qaJ+>Gv^$9>{gy)KDW@z~` zZ;|~)<)9x&l&3b4Ks@#=@XyTGVRWBZzY=27f&fb?B#QOJG+R{#e|8R@)*Nze-QNyZ zyFM5!`WB|r4v>v3rQV$9Kp=t_&{w(5`%ORqr0qy`$C<15D%I~DILrjJ9<5A$UxzoL*=yn2bT zGv{*1k3&+v79rTDU$+i^N<3sA7 zGJH{*;l-!31Oy_RD!-k$)LlMD*;cnQ^W$qQZ7b-CBwOyE!{>Z(EwZA&ok-F^PVs@5 zP|UOX$G7L}K0Pz|8&1l2%pZ?NMh&FK2l>gIA359@>X1+sb{)-mu|t6exOZ{l`t2_x zXH1#v+?~1?;2hT*r=IjNulC=rg~R0DTv)D?SUG@xn-3Fst?Ey1BMtvWQfmi2)6yfi zSQmeoDJn4k4~iuxysr&o6gS&Z=oRjvR|%}P+zNkq!Klb*ZO&Ud^7njX z)7YC%E!{m%x@4X|L;2?ZYlF`$8KJ=V+E-;?Hqh$ZYLd`4;^6$llEl87m5gBdqAo{E zablrM(KN@C6yegA8~3K3=gy)&)f0U}p?gtpZ2xSo@bN33FM&y%EP88oQ9iN$ZzTlM zn~R5|V)_v#8Ez*q{p8y7RizM7?R<_igBjt?Kn@v3FRDkh#Qq_yMG6Z?EQc6Z}Bvp}m zQIa1wLNP5P4@349sn|Pz4+$>U8I&Tf&^hSlP4exja-`+!pPk?P{*HEK2Irg-{&Qo8tQpbeSZ<&>OGH?fP3&~?#M z;2U6zf5`aS8VP)oo@(|M29VadFADiXRKO@;=EWS%LiHp1FU!Ke8zK3dng;57_s3hQ zKZhQ2IrkO(wekK$fgrg=O~vuniSyk`vJuT22eF8B$mD+4>nW()8szJlzb^xWF>!;LHbPXSyemUfOf37v*!W7we~y=f#8Ef?A5tXj(5!0C&#U52dEG z4OG)^HH3RGxEbWbhi30iFB8+qi>!S6_%$BW8`|9)r%{ot%QuA>19EBVxRnBqfb&DM)a%IUoKMSDwXbpA-Geny(# z!;RlQcH(4$3FOB?>`PS`D~01hcyi)rRmmqvn+}04q|Pu z7TjbiQ*$s3sW<}^3sC}?>O6Z}ve!_YluUmZ^|TU&2N>zUqP86-G66x+3!KX#rjy8> znG(ibTl2kM44oOnB#o5W{4yT#yf`@E4YE+bCxBDQ1Uu93QEabTvA1F+C zji>_i9Yx{iApe6$nj#r12@U{_U(_#{y{%b1YM=GEzqiy2c0S6ppj!(%qz;- z%b9+eNB1>6Fjr-d4-~IEwSNRz7!bXgJJ}F9+U_-o#HtLfpxYNFz2AbL3XS=grMmZ! zujd!mGn>JrL?o$i4~Xa_C=+bJ5kLi*vfYZ6p_6J|0E~#5J%3S5fd*ikryu0J@j-Lbzh#Bs!(O{78ya zZ+ELOCaTK~wd=V0qs>U`#{=g(Q_^ix+@QTpg@&fO>-jm}k=61&q7SK|XDbMtJe&6C{X>E!->ii+uY zy_&@4Lv{GV$ZmhXOFw(w8KRUDwykoLRev7jf8^o67acX~ghcmXHv$x6m`&@;A)`_g z#fIZ)Iv^;KT66u1Y{EG2Q#B4#Rkh8}&Y+`}iFlHM3DmV&EpJKSxrI|_pST?PmsgI% zifFEM%pPap$}-I%qGw!P){>P!40EZoJo1r6Y=1tnG2`;TaMcjD5%*gjo zV$8y(q|2!~cpZJ=R1bk^y4%p}C8;ljRKjzVvAjZy+y-X@GOhJ351I9|Rt%!C)ScQ^ zNv1M{dN^%f7L9;I#Lbtwk4>jA)0eYibCe?XC}BFfN1QViDP}2$S_F#JGYlo)45B$E zS=xe$N_cn!7a&_;Azrl@TqGv(ALUA6q#!@QEXyF~y`O<`y z=@d4mc82Z5M0a<^p-2F6T0R0G=raTO*iMyfITW zV9--la2O^lUA{u8KQj-_yhwAdB47Q6d3fUQk3VkWo>(u^=?^n(IQA-+1tp6 ze9uR%W9}O{C}&Vf+JhbFVxe_Mnls*dD95h!k3%V&Q^Y{K-Qvu}PiAX6LAQ@Btlzm{ zrkrj}?c6Y?Tw=>*z6rOY7E>9dgiL{`l6%;wi4f>uQTD?yfURatekFpfzs%Q|@n-4yN1T!;{F`MbBf90l4RhU_{hZzJ zNnA!D8Pc~${65v$2&RhotFA}>7rpmbkB6Ue2SDbu8E zdjvr5Kl`y7k>9(QOFsm&X(pcL`QI}#@lX`g?Z&>Q9pS#2K!u00au3m~gVb|f)>Z7j zuk*h73QVsbq)X(gy8)G2N4W~7B5aCdzE<;(K(?O(IB>W2d^FPjYb*ZCS3$Q0Yh+5` zh%)ek8o>PS)%s(_f5tpA#*y!?zel{ka)Mu9eKjjsmC-eK5>HzF(|Ra((}~m!tVERk|LQ_Byom=Lqzwp3n)fHlTVD5? zyct;+XVcx1Bz~*s#g*H?vuxAy@!qK?{#v)c`6{EJ1_`(gLiTqfN~Nwk9y2VS7?E$x)ruhzq#`{Nkb0_e7cfIAqE zW{aY*v2c55Mdl!IX6mM4p5YT>vORYa>fL>bBm_iz| zu|qanW{~_wD1ngX4H+;qs|Bi-YTM-@O8>K<#BM#_ru%tq+1@oo&iT*9$hb>!X<}>N zv*wh6MWaF(q(mYzG{B3r?sF)g{fQ$ZVoL!490&a{UH*u`_!H?^Y7YyWdX(s0x(e)| zfCJ*sm;ZR=0OJ0NFLTxN=(ilD>9BpsH1dL6XY-PB5G^6mec-Rv2%NYT!{nEq%V(89Sv)c{Yvqu&Z^T*2xZ*5Ef zKIrv1Jj@a4j8w3@!^%ufKOA|9*m+R0F^~-B&#;6Pm*KX{Qhsfk&NvL8kH21$YdOm3 zKuc2)Id+%<`_l|L*&!ueJ#dGr1G-L0j=S{Aypsm_Z;HYs}W>e?q7R_L# zbn*+gIOgwVOxFfKc=8E?e;h6QW>`Ah;TIp=jFFzZ8M0Sr%0c|ba16#u--bA40SKD) z%55M_$#fHkREZGr@u`A0{PmL{G*O3~+K>P_WEp}P>o4uPh|Zc{sL zdcvzJe} z<2Bu+>!mwiDrNvk3?HT=Z$$JKGbDM7RQ)qz4ot?hYIVk*k$ehtuUp_5_?BuIWP{5- z>|HRDoDHSW32End?YOAs9#b)57`J?;coB4Y?`)5?$IgF8C4L^j9D1j1;i;m=B-^}m zXn<6eX)XHtzF9TvYbx>EQIKI;S0}%^4b5wchlm@5rV|h}qi{ucH zGU_p)_?}wO%=i4Quugycf~6J6Y22}Vu>R$2V4$$FuRboGejPl@P7V|YMCP`w9uHfS8i8=ibcAoSIla2}o|D&#Nf1tV*<5O^5(!ZWCBGYKmw zhP^K=F^rhPrG=pzW#>wP;ahJPadk(kAOOGJ=^LVUcGC@_h#QEBhb{XLxy|VTIsAih~h%c zwa^lE6fxweh>G&;j+;BEYi@q7e|K^+zHIWTehA{Z4P~6OEQFXMugl;Ak+i7jfUjmv zQdOBvG~bn!VSk_=oOW!0dY}19iNoQJ95nZNOdKgWG;EI>VxL|4b-~wCh%g@IZd0@Y z*ZCT4152F!J@EV_-%@J|8(3M?Bm;mM%{3r(W94gSp?Skk>VXZ$%No?S6F(Y=#cDky zAWXnx4Iu1?2!fBarGclJvpo>`?`_;XLL~4WNnTvm-k3$1<<87{KR6ZlC z8gf9|3{4Z+Zj&8N#?F5iyz=tS)~N&*+A`7T8FRk+g79||?Uu)fE#olI$qjKC`0{LW2_rj?^$zBah9Y_~fDJO$ZLy zAHip(F`S7G{DK8FM3lGSB+j!q+Cl!rf+gQ|(yYv`rI*I@z+|!S@_QvumPIc2%n*ucrm)h{JY?gQWB99Lh6B{;l{s~87LSr#rB-Sd-?Es2qWv`dbdLnb6O^D0C!-)o^Tt3G*G2In zQlLg6R@hUqzU#4tc#6;ujGk7i-OJF0*YAlQ#?urav zHB~{YO-SQ3i-R5>4*XGXZU+7CZr@72{N}sBj0=YaDtGrCa zVYQJ((0WScdi30X{=%%?VG|5f#~p1w!1lvWPzQ6|*Lf?W2<_b)dLYR%>XpF4$(0Y1 z-AvF)M;JwDX^^k?A&G0}E7aDJii?Q=lW+peghu~1I%gK~2DHRjhdYdhkr!T3K`8h` zPfoI_IwC=-!m)#Iyz9ACt7j5g$LL^5Bng94@G?b^IL2S$8C}%g8{ypg?78TfPly_+ z)^Z^!$nFzJ1`2v;-JXSHjIDu}UxH2l&6dN_b1iy976h)&1+X$(LSvpZCv5#A-RFlC z7UY`$98o)V{z`^yJ5AA~NBWJ_ z8OeB%bY)5>Q!^B5%1lc>*?^n)k%gF{VrFY&vzzf8&a~-SoIupX|$s}G+YgtGs zStU<*OejCa#lvq!g?z$e_ETGzVA9*T8$xcqD*}`ZYsWk(}Q#j7}NMxtfuB5$2DQWlaJm975fWb?!@*I&%`@UM*ZMq$JGQ%G0`nb)}1R zry5r6TLuXy%y(Yfk$Q%83-ez~UR5TLIV>!mirs89dJn`IeeQC_;O33s^481hCau=c z4X)>jifA$zN;~)t4uLhU*&uS#JX+CfDD4>$&!Frlo%-!&P@kjMdtrZwJhY}3wit|n-&wMNcOP&P_ z?bI}Yv}&y2QuPE1!hd9<*_P>c_J6-4B*_2IFSL!NEv=M;Lc-0+uHxW=4OCF4AP?J`Oj-% z(1Wa1{I>01L7o4+i%;$;>(Firyf-}Po$!0@^k3HvesKK)pge+C%lZE4CZEsXgS-6v z;Uw;#K%NX`K>#CPoVL51^*`XH-{0lZY54OxD~A6#k9*+BX#kmC-I;y%zwa0PH0T)s znlJ99wV=uU|9!Eqj=^rEFEon(hu1+YJqUkZ=e?``Ki?;Mz_@yj7C--cB=)Zb3)&(= z{Y*4YxLmDIlY@H!b0R7)9Fqu#uJCwDALKRzD2Nh8Rr-85!W&R;W7yzV#rM}o|MM#dRTwvfZFt$IOn0I@z7#;Cs{t@X zNrluxvo?iZj(`uZ$=4SY8f=gh^g$0xZF`LL6dSmmBVt6oe>F61k#?ibwZra`;%|(C zAA>NX7lKj3KHUd!KoIrbo_M=je*);E4wbA>KR?*i_1*3W*eQBaN;c3el{1iDQpB8i zEwF_2{H%7+>wWATYcSnIiOh$*!1|>YAxCUQ11D?N8aX1VkfY7|TJ`Q>V>qQcL6Tp2 zUvVy0_`G4yR{xp)VJp=UeLt71vnY+PKr}4c_tJtqNRxgHm{+>xNr*JfM1&)n1-dR z<}59T;yJF|`x0*@nmz=0f$n)cwp_WL3N>O8|6!yQiM3`8jf)oo;UX0AYYWn@!i0^M zJ*hYK0==t6%Gn@N_65k`LIB?jMOH+u(9#c*^G*XSOxr+H&`Wji)uXE(>*0jJ|25@aqwg0Zkrd1JF@S-m;!UVYRp%txH+yqj{S zjeEJ(615p%BVMNB&Pl4nQ=jIjOSnI(A=S@`7$u(pv=E%?<@wwD_Y*g}lt9p%OODTa zQ;rB?+^4g)5zkMw&J#G;RB$&<5@B}JcN%sg72IE++>A1y@4Yo;4f=qfoqgxB_6H(+ zj#1aVcj8Ln(+e*VS%ft1=>wH4jl8g&@VA0QP$QOEs*lM6lG$CN$_k{pH;FGL0R+2+ z41SPm@XKrOAC7I$&;=)2aQy7)Lu2)xWAtpx>!%o`PZ$b!p3AreN|^0B@R?>J1v8S8 zqDSPZCnUayzqtGU7<~}nE`jq<0(uVP6Cv)>7TLS@PWVP#s(Hk53WqaQ z=BqG~TTdXQcykX;nq@!ST!DIt*Pio=Ek2eS>|rHRH-Y}yBFkKrqfAU|DsL2AZeu%8 zJS$*6vlO-iXlPjV%XN}gHfB|cTwo_Si~47-LA@*wJjlfa`RWfv;U+i;7&cThE(zW6 z(e@o%s}WvB0*X5iC-bmCI3X4DOe%4o?)eO$30bPF`#t?S3uf2Lt!T6waaU#LQO6ot z4=#XAb-j0!z+auF|5(QJgG3l19$?EMYKPN@3uoU^^&*fEjdwchAdhUBuNgOMVY@K# zYq$*l0E}}lJosk?{94_bIb%nD)27oAL#27Cz$s2K%#m5RyW|)P8{mC-_##KeI>0QM z5J-%s#OrOpV2p;LHcE`X0o=`ZYbp_Z{DG?RnzSW%h!0!`>Ki4JZ3mtTS1r<%y;&BQ zZIuq3%J-!+2#r|_@Ni~wutp?O((!@6qtg{vb8+dPRC$YZP3n&^EA-bF1_he08A zL`Xm1#_44rNwTCZ{Tdr_+N5nr1XDwj7pZ6%LYlFsHv~F`8(uvW9Hd#j`?x})Kkwo8 z0r7yHN+BziNuaLY;EKW;HaZ*#Fmz$@e7w+Lg=mUNMe;5@`}ALKEhb6XU!fYIZ5`Ld zYAJ3!eoX1#wA_-aBp>HB5lKgAjJwfPVGoX0i9n7rH%(BA!JmGcDuJkkTU;${)M*z_ z29JN8hQ@Kk^mrskQo~Oo`Z@s@|LWcUejI*eFF-TnHC+8KEey4vCD{#1WV{AuV1uY> z6BXC-dt=X#DkZ^JqyD{X9WeDNEP!;?`Qit88L^$@8r-OZZ>q5q2nEW=A zn8mp!BejZ#k!8%a?zN3mfT<3=D^I{xlabSlmwG#xGn*4VZYi2UI^y$JkCY}$?R@R~ z?>Wj7hahO9%j?F?bY-atyi=p+VM<1auAxrVbhjZ(M{2umb}YGik8nIMIT@Nz`HToU z-3X7?ms0W(l=q9ZodlOAEfeg=^4eC6HB199SmN=3_F>htIWgTEAw}TqO*&VR*-GR|?>tG{-CfVd zQ$J-%Wjt|YAi<;-v42{>Xfz1O{GkB$G?QjB2mv&{@j9-UV!x1LB7E>{E&#d#cjGrkkmf` zp=~;9P6;FYcXbZSx+QWQOBrzd@xXS{u$!l|Wfj36QwBxyF@hCf$1fBGcQl=JD^)gY7-6udY|at`I><$C_&7 zADs!m*R79zu!BlEL&+0Xq;@%D#M_5&e9c76v3uie>*HrqH&OdbGY5kSs2~#?_@CPk zh|&UwAfyG^PGOECQg7m|IBu?S12z?HD)+{HcjAPy?Zih1vfs$jW*^hX!?Y)> zG)MrnD)0Z>C%?rmUx1~T)cU29CHH_XrUR}3eL8o#*WA>p1?TjRSnN^B&E5HQTH&5} zqcYD2UYpZ^vre<-?mhE7X8(QQ2=^n?+NRqvuh)AipSZod{Jq2j)f3)}_Y{_H{nQs( zx>o%*KWFJ1?)lu0J;v+TUd&J9oL@Q|MgP&J|W?kHa0=)=L_-gOqB%!BynRePePl|nsaUm6AG&(FHLy;HRZ#k%H5CqtizsXCB>^m z7V6G>D58Dq#xmfgBX()`wgH<0i@L?<*q?m_Y%{##`Ls-XO=t47Z*%SHfJ4=WJG{e+ zHXghSy85$Ddfnu*z=iI{71~Fhp5eMu&&ag==c~u*+vl8#eRoIm$|Lzn z46hnzFA{WE2b`eIRm{{r`)}>JT;&bk$``zwBvjY0-hB0l;px2iRNzp>Q|-@p81hrE z#JIuR=~%mvjadQ-6-R-~S2d<(<^k{7lmM2MreAJt*yYi@XUg_vQ5wlF%vz>ROI_#j zEU)yS%yER_@-L_2HqhQ{qODs#W zi_Y9FseQSa@m|Ow#0V)4x2`a5Y@4)mHk05L-_8A(Gq--cvBoUykrnPN0`Fa~(gSu5 zU$uQhFO)Dll~=leLlsepWvUqM8+5(EL#}|Gt-BX+oZ5mE%#Bfkz@ESMnRNsQN<>zG zmg}vOzSWGo+J>bMZD1chyY=Q&jR4a|(29e?$Le8gg{NZ~4?+r{6~&B6c1P;8Zm0bQ zcJ9(bfJepy$HZ<4&M+#Cs(63cAGCxaJH2(CD~>3Dgy;e{77pO}UT)`ws(oQoU)^WO zUwR|P1FsfPZ~(XLd7E6{w=t<8 literal 0 HcmV?d00001 diff --git a/docs/onpremdb/images/K8S_SECURE3.png b/docs/onpremdb/images/K8S_SECURE3.png new file mode 100644 index 0000000000000000000000000000000000000000..b70123c12a2035af0533c17eaff662d2f11acb01 GIT binary patch literal 130498 zcmeFZXH-*Nw>AuDL_q`YJIMa@H)+dfC z#CLLmo=BeYsQhIT987xYRrYvuH&TQsQO7|~`3D|{IxooG1M%G_QhoUSmk%B9?FH~F zqrJyNTtupJH<7L7T)!@(UdHptkiGJF8>o?Uw11gZg)dQf{Wi@)ce z_(Z~FgXZHKLX0N1PmLbSzMwU6PmSu9yZeiL#oU4?Vcz7O(oMf>MAX9H3zwRID4yz{ zn9KO^2x_%zz?I16~AXXNHhpNF|1x6(Uf9%3z2 z__c+jwNvjGm6C%PWlk7PG||s2Lrq`i=*Hfsd$zb(RI0FiGXxJZ^(#qe@C}*chx)B+ z|4>8s-jW5ed9C)1@KRM+ns)uBuG$5OH}Q)PlKIKU#yyOC{K#N|w{u5(0y_CQq*1rC zO{hZ9!106iOv{e-74a6@%a54;x=LEza&3ah(qA))MJM(7U>NS!FS0x~t@w8HM}o>N zOCLSK&hgb_)zw6bGpIbd-@UWHystgo;TJfQB|`VyeB{^>d&oeP<8}$*N#HH*0?4_?ZoG%x*+|GMJ^)7&oit-NeeX2l3W>9wv zN9m2FpqHf-Xp$@h37V9&_2%2_&jK{&8EY>W2PnT~5G86fCTbvhG=J}I5c5@1HD>xu z;`NuZQ6O{jGG_NqTCxuh5gu&U-n{=2#i$2@`IDtT%%#qMXOqs99z7tmMm7SP4Z^*Idx9PdoBhv%x zAzHb2ClqR|RH{5dWG5;>fxLyJL()$B!`j_U2xS>B7?m$s1~|99U#P9+G7z_?+phPw;15aNVu;knkZf3^R)4?O^(z*6O~Pw(#?+X5h`%_f)d;x4~4^w{B4BT@}1~ z>D|+BWP|V4WWye2-yFZG@DckX^hfXy!N<4c)0PwyRc?KDl4r@x$n4G(ACfEQm`p}1 zDL?N1rQi8{3A`t^$3XVpRB<=+v1&P+K3mn-{jWd2Qoeb{!pRbG%gDM&JwZJ~y=F)O z8d(0BPv7&iQ-o95p5rn@SDPluX#9rKvii7ccP?w5*oYcUVo3O#_&3>I7BSKs+L~9i z9P>N!oAP}|MMqEbSG5S%Y>C9ZNgWw5#61KglVYIE<+gBesw{6NQ@ZSYoSubhNWx2SBbd(tOf*Oh?qn8>Yqkm z(P-0t1TjO(Mt3|?s?4ri@#%#o5}xte-008lcT4F_f;=Hipb2`Vv31~gV{eL1^~S4Y zt6VEjt5~MIEBkFJH=nqOx(>OxZ6puU4P~v@Z-%(IZ={aG`!vnU6X<$wt$gSOIQ<5?C+Hvl?1mgJyH{b+g?{+%lZhpCCu$CQ+^kp?5FrT&$dr zey=UDxfZ%JI`O%@apG{_Fd5WcDC{ll?z?NryIR$klPppHqTalY(fPP-bm;jkHfnQT zG}<%JlXL}(Tt_<2a?ERmRMN51vZw_PvaL-%V?%n_0(ZHn`s26$!hY& zH2MY$eC2~uTux zwucUCd23pFl6o>zgoDz9#Dm?iiLeifJ{IaL@@4riAU-Ca`o1hH)IYxS#YY~a9+!oT zjd&fDIm-9CT6HX0M(#@1rjy38;9ip*4qc9>>^6Hj)nmr6xv2ny^O>N{kDEi|r{!~A@IibHgQ;NUlJ)BH z3f6D&TnW;o-t?^5GGntN#9(zG>UUSr*#T>?4@EO=WOcS|?W;lF!PKkU85xu%)#^7PYaZ8T zdPXj!!o0=f$P^snJY$}JwT*G$@N151uJKM=ZNigng?tZDL4`Io>#rM4>Oc*&8kmq0 z-si)Yo{~=a8K0`^7wJNDPP1eUhP|Kso`0{GGc_fx7hFBo>bh5--u5Y2B=U&Sy11sW z#LoB`^?i`svRvAE{J~<1nbgmzh>D2ubMpg@<7-o&0%da@WfosYm*&+7HPvmJ&;P2v zGah?eLjTmj`3**M(zr4A3b$;KH_ta%ElJ+ zje31yYp!Uq+0|k9Q~9N6($j`B+U8%r1kotm*S*T4gzbtcsp(b?X9fT)Yrg6dVwP5l?&)KZD6D>T-cU&WeQiqM3P zybJrQeR0tt>iDMRm6o{F-QBY6-Cf?%*JZfRLMyW)1`-cO+B%aI?~8nYaP%NWJjLjN z2LHK{*U2i`UmT^R=LYAOT`FmZ-qj7q0`V>wsv~cqq(sC49A70O4z?z`3>*;yA9~^PrGFoj99`Tfr6DaZ4}5EwIYFWJ&Q=aCfonv-725Eq6v|xhcIvmgSwb7d%*1MorOKb9{%eNVc_`UH19*^ zf8F9@EA~)FNtIdJ!3oMN!1ILX$wP56W@ctlCvyv7^;fe0*&O&w?4gy5%UfYyUUzqQ z9(R5o2PaG3r$RzPyifRe`S`ejJGh-a?OjYfxb2-E{YNMN?&lTM+04oMt&6pTJ@ZAs zCZ-OqE@BTKUJUeKKmYNZP!H?>9?9PMKc@vukoV#W?^B*9y#LiV&{Xu|tgx!J2h>jY zl{E|y4={$fpuiK+f8GCoT>0-2|3^!m|8DtI=!ww(Z2CVg{r{V4IYXVK9bmwiF5>^a zH2>N7e_s60hN8R|Q~w{L_zyY%>ntE>aWYZf|5`M0vQul}uS7(WMDnksG(CvXv!ot0 z?f9R&v{xI0)U$%v*hmBlCm8Nfqm8A?jO${XL&j}?&Sg!2I0aNC^G%6CG+)2@M-7M! z*^YaC*Ti#QONbvIy>z&}2$V$}t)}x4IyPvHLtSGLj-IMMQj=O7f4tI-^Ns z_RUMys7bCe`xE`~mp?6YZuhG{cXx4huarn?R@BFFjsB0_T?{6v8YlD*qXM@rnGgro zdmjBBc>Yhbk@QFKIRA0J|9bq+Hj>M(`#bHw-i!-mqf39p{E97yTY_(_-o5<$j!vC* zQtt`tWOe5DM}p3cKhL+*fO!RBKa_8-P>Z8v*cnfBDH7Nc8}8_vjpUo*N0vM zTCdjzX&F^WyEXxlvy{gL2Se9w34@}X`$w9aLT5}UH@Cz>r8)a+DkpVvX{f}~p-9iO z#3E~gOYiM^l{W;Dso7vnVY@uF&|Kw)-O)rLMHaBi!8vsRuU}HIcdIK zl|@KQlgO~J=HCmHxtaX&kLUikWVVJLgl?tDE*CkiPf%{{?!Q6ee$ay(bUxQA$HjCN z*~CL|$YY+7%W%S^^8b#{c5Q}2f`rGb^1 z8}7ka+0x3%$0`*<{al1kl-21JFQ<|QCw4nXY&?@iAKjVirynfMp*zY~Cc*T(>_%gA zg%hugDFuvQ&xkKEgY`ML1Ilo=lPRB9+hA|fN59_WGghcsxpIXowki0q8)c-XAL{p0 zd3+vLGiy<0J05X9&nWz`L4^}sSh||(-KSS+WkSQD*_mE3#P#I~GIN|sXS{99K_3rftxvf%mZ;hg~ zr~N+g>?OG4uAh(Zwf!n6>gPq4+q(9g@%ER#v&_+Du&g$helz9yT_P;Cq7|IET}hY* zt4xp9+ORV?Dm~)V5{18VA6?Om4&@_@fr^=qdHIEv6J=m!CO-6(Sr+Xlq)A0vHcI@wqJml`3^ypb;p__<6PRqPGpFj9`gZrF_Mcfv zys#{6eiXiZDqm0(npDD(*q=xsD=u;0uL(***2g;exy}^Zp5RM3w^|+Ij*bhr5}AZ~ z*qBrxTdP)UGmWs$hr^B>8ga1Qgd9H}CFNCtZtiCIdd|L2O5|*^@cd{q_d=cZ7`6Z; z9;7*rKT}h1>kV4XITjeJyN>qgwy|8#PFeR=SEF(ocuoQF_Zvrj4%6h-OMBQWF$vk( zHSkk}(>}Se?ND{kix(pDHXz)w{{v#J?v|~r&p>v>DF3&RZIRWXK`4{M6#QP0WqfQx zU+xH-(G1?mV0@0Tc}U;3X{#QUJt=Q5-lN+%Dna>@P<;4|(k11&+_O6g9}YQRSi1G1 z5RI>!S(TI5g7Uh)o}Ljfuoi#fCcS3gQ!aM)UxuE`H2S5+0ZJ*tqp5ypo*IpV8FB<+ z{c5{1_c_m>UI%LnHRs2x#f4`mzw_BtuMKwZlii>hV4`ixxiHJ4@hf^ahQgI|N4qU+++!aehQsLS!p8AfUb}j zpBPZ4=c6&qyFJL4u{RFMM`A~A9%}nxH+et zEk`R^BT*=Om$WqTir6A0h|iK~TxH4<6%!qH6ay6mD~*DMHMG zHK(Djqn%lY&Z}3dw)<$(whxopN=bG- z@lS7FGMSx1TRP7sa~~f(L!MN49u||IrRg*`Y-v~5VvI6Z1GX_%@DLZ?%zAHyL z&5yFML))1B8~O3uhQd?5`{MT3-o|LsQo+4`+lUk!*6loyIT1NX81!s;ndIUstx$)o z)8fz_f&MO0!dcy~%|C@R3Jvkp&Xb_tZOpn1E4lr==YHf39m#yDFP-l1}av>#QJ9@YOxwS&eWB zWSb=22#Hp046UvvKfJGA_HJFQux+Ngn=;SpQg%_9cVzDGAd_p`{;Vn`)@dil4g%;P zR&a%YYgZyZ-AZ%1N&Ya7JXzdVuju1pP0b`~r0=O~cfD1=vWB`zbMx5LC7!!rj`WYC zd74A^u2N$=>V-N_(^#^(syf0Mrh#~g!LXepmczpG8M}i~T?-Iyc^+Qb^jJ%U%`SJi zn(V)+ZYFn&QD(DYfuOJ=dTPxAq8_F=}6*!1-)+w=?g#L! zVJ93%(fC>71H?J0i4wrVQFnh`H6%dAr2&N6VM{lqOoH#AJb zYol1`I6f&!<%J+LPCSEakNe!Pn=oVGpqiCwlz8H(or5ttzmmWG!lJ*f;25t<5O%Be zUfqV-8>S4s)t)a$B4wHCdJJO(Aq?ZSytLO{a9^s&3k-(3>ItewrM06~c}@mFPGdvx zr!U73u8()9(_N)CJEeXxA4W^UgF3!gG0(UEZ1vozgi7 z<*5mYpx}cGqmG>D`?Lq!OH&knAKFKBmQi=xhi^pdy`%qKCAx)D*PTg$3keIQSzl*=BJOXsxY8}9-zYyW2W840%%~})hRp$gCINixS@84 zaR=xT4MNjerybe~Y6mHAzsS$ma?*`*IH~4T;GLj3;n_TVAE#7YaxLM9cjea{lIwuB zrS%(z9xDIb)YGcQK`b(3y~L)W9bL=~&s?Ocbh@gykg;=Gu{Our7E`&ISa7qGBa#p$ zcBwsm>)!9cinJaTrnkdq$Pc{bbI3U7F%)~qN&Pna1ohW)mXH_wenDZahX`K%6s-i8 zakKti(9g@w80P?-JWp;OvxTZ@3R7G?;>ehGo=MCyBV{!6+dcKXckKGLZh7Ct*_vwh z7hl9ZubYp^9{ZlqxZUKhI>+sa6tG)gcy2p(bbSjpZIE54kJoBVANVq;Q(cSrR);vz zqb+4~7+WRkS zp)DL_Y*Pt|a=xd>8rz>yY-3(h%Z2xw;al!SDVMMQ>}b|>_@;k0l|`=aT0>p&W=C-Z zBW^iX$zg(FxG&<6p|u&k7Pz3lHBrC3WrfR%v0@QfaTsu`u;?vx-5j%)(A#+N{Q2m? z`&g^2H!+ZYmr+jL^2`_>$kh2Wn~~11plO8ErOmlU`iaK&`n&E>^6JSyXjt>rQWad&ND_egL>(_h zs|chRosP!2%=zs1?noaq{Czkn${XeRj;D*N(q|`$XVLL+dpStOv2`+r=TTAjO|7jf zwDY=)Bsv8_VXtUwrtXy-&;{kxUTfWK;^KEnD$dzGg z_&iQbo%`=P-+^F)9TtPi3*3F zaGmz+zuv0qOU;6d^b$x&fH%F_bz_8mB4*ni8N@NE!jp6`YNV~JO*0TFpYyp92Nu-$nWAM7u4shdF4bIr{q{OxdL8+W%L zHP&}a2$tgN*xH0()88ke?KyFAm(7{eo*IU_rmhrARW`BRK6vgfR{6NH%RBlT@1=Xm zIq~H5l+N7{qxHFS`mZ{xa?^MyEVtY0q&4Bb8nUv=d3*yLcRckADqMixI*A(mn81Mc zqCH?3t4pVB>AR+{u7@F@$BL3D&26qt3cGGxZrf?j2$zxCkH+`cnB3c#N-1uJkaF)} zy^+j|1(u_Gt6X{4VYGG>#W{9Cb!TM-If^}DA;p{6jM%A@=|sJ6-D@L+2(7PM(<9I( ziTPNI?Ugz2BBx}+UOR!Ow0QZWR(&0`C%D#IVC)+&pUe7CJGCToIac4Nh3jBqRNrmF ztdHL#r#QPR)}vS>y;g=lWGu&Ozdwz4N#SbxDrI9_hPcIbUl-r-=Q}JQC&NsfhFz7=cI!dy2uPpX4`qJp zchVXF_e*l;FzN*s>W&V7=_E_Gu{MLxHF#E7%Ug0!7H*A0+*2s2cp>#8&A~Ok@S5*! zLaP(Ut7#5h-+K$$t`_i)TIE4OHrj;~SaAoe!eqmO%w)?(8a{(5CF9knTh;|!Yr4hI zEHfc@kLi=}zjXyIO?O_WdC|%_P%|?w1ZrD0eqM_do`z0FIAI34vAd7^=e68!1Trp4>xUmwl9vj}cT_EG;Q8Xm}R={)J!h7}PlFzIgPc)9c=D_|K zfEF%|cL>l%;kUc4_!?E7=k^BAAZ>UZF`@7|kQpDR7w_AjDyzER$@vREbiZu;0iB4| zl70JH?JYtk^p3AWcPzK`VdiDN`HTgq?iM?@wH5z=Y>nBZN5$M%E7X@`c7kd~J8>N^FZG4h!)kZpWpDT>Y6ZlKGe*LMFid0TOArI`TVA`Y(~(jtjZZPD3&1)5-F4N z;cy;&tbuwhQGw4E4~$+cQ3+P7NH`u6k&bQHnKbEx|g4@UYDTcQ$P$ zGrq><9xm~=RkpFlH3}Kqi|=v5lyTd7jc;yklhX+LRt?NG7{_VJ#Vv^W;KX!$docpr zV#t))@}31uTZCJa@f>Fczq)4^yljyV$20q-U$LTX);rD5DB3ixj+dCtKX3W-BQeUa zEcf$NY$S$GLS5#J_178?z=Zm<(?MqeSdD!!Sdnn+wW{FJX%P7u>%kHK(QwFD9vcal zLd-e18Y5JEp13Jj%bTF+LcHO?GXp`m^j*9gl&!$osr)pTd}$3U6rS31Z_5Qn}nwK%A6J|?t;kg9I>Rwyr0gH zqXaNGC`oYKu!_Vi{&33{09Ff8S)K*FOICT7Sb1X86#W1w*&mQWCB(<>&Ms!sXJ-pm zpT>)}Su2*+`E7NHxwuF5*vT>NJ%wA>c*rZi==as6ciU^umr9@4ok7adG3cJcaP&-b z7dJK^-N?+K3qvj>#hMha%(WG4Z5pgNb78J(9d2#(P8SyI`;Jp^+o*kTl8KYg-3^X? z4IlJ{Zfs0(>{JP|@b*x$NEN@W)|(O;a+)lQ;7u)CtADNMTTV^LKyg>?@#K~mpVlvF zPWMALli7sR_$0bt>4b=i^yQRsPe++PpAm6u6U(5J+0>{!rnQyeY^oKy_d+>wQ0h9H}&WOr~#tSOjEC}WXE zajY(=&@{#JXT-fG6=4A%y4}KqAhSQ@ny>&n%8ivnqH?Du2TO<23WMw&=^~EfWmDSv z!U2ckKGALg&eh*{FMqoxx)bhgSwz0!=78iZ;MWtYw&dAKB7FN$uFqK;wi?M(9Cu7S z@{tXZ6=4KGX&SNBBspCjjc54HBGdMeo0}Ck|27~YzU(B2P`%YrMzWPV{}P-pRfB#D@B0DO|8u^P=EWcMG7+X(~itmE!G{`- zM}UmhKaXAmy1KDa{bE)6AE)%2iaD2o^p4ieznk;FA1h$y4s&kO^zUTKf1J|)6X^dF z=s&0v^Z$*8zW$iO11KmWimVWV99!v2u~mrUsk%VyiAfoafyCn%IDBimhTwbwA6DDX zZuF})`@%T^2B12XRo#q!{9t{!$o$tAaEYN<0R&l1!`_)xcu;t!haWCXnG+eyx{ zm&+gT0QjWtYF`Sx)cIg-P;J`dcsXvXYTU45C`(~_v!vM`HdQHKkQ5(pb&AJUB=B3} zTddnYP@JAA@YMYXq2Y`L^By52?SaG)wlKj16_W`v1@b`*Zw9$?shO-7~=KAi;4cF7$0S?Unw!?waG2A`{VZT zmLY&$V%G-SIwJ1b7J!T1)Tji^COI@s0zhe?m_eV=Dr1tEcXi|G;WS+Y9DsGks$u%5 zVv>#khyGNtBJC2x^*&mBa7Uj1F`Z?yiMA2EDuzRg1LppObV$H_XJJ5^vWmCq=wT7i zzX-abU%bL`u|ro~{fkzvYHQ*%>)-u@wT_FGUOsmWi-^p%pIFrO&wsMNixRGC@gb`JPqKGadQ>yp25EHZ3 z&R)ZQU&SR#(SfgYCyS91`1y>$oan!l$|=Q@ENj|NfcHZ1v_%$OUdpDvor=Z2J}X$%IhLix{k4_ZmL@WB2zQ`^m&WGfgSw8z-moW-t%}T++4B$X0v`>F$>} zpGgJqgUx|-Y1I_e-}^r1sGorCUnwJh=}y&|-@@N7x>y0B;qt-Fx^G2E-;lSpIXgK% zT+4{W&o3$PHkxluRqYc1TEGH;BqAvXIFPtL5sae9N&xBo9byQyOMI{Mk@Ymz1OWBM za#Wbo+P#6ErsJQQDe6}3#(uiZiuW(Z11uR6Vcv0|1CE=u1me(&g5vrv-}6&VUu{oS6;Rw4ov6HA z+4p&N`2z2NRJ@i+XdE|i4)0BT-fgLma{PI3JzuK;+za6I-7eF7cxl}MvNaA{{e&4Z zbI!NLyX=U9%?sL2AGz2m5hD?;`Oj0Gpln+ms-i;xN-cE+^Rto3(_@Ha~ zJ^*3h%_X+wzBKQb_}tdQGQ~AkLb^~v(Q+gOH|O1NY0`Z3J5(G|P*0Z0X%g?SrL<&>AA>JFHxqO* zDh_~d zzj;qab0YLr3t1zhK;?jHnx!rs^@j3uctsZ50c&R~ax@>rpP%@MP-$Kr6010u^bL2- zmN8P|?C5?kTNG1m_q%_ZfS17>8e@pdM`h~6);lJ>XKzrLr%CoYg`*R}jpZe+F9*H^ z76B#&0Y5ul^{vzcYX{6)W!cZxyLEHd9O33|>Xv~CC&a4W7x$?>A9U)~3lXeb6DvMH z#rU0iYtvi@8|&y+aO+q1>=>lO%nx-HuG@Mp((zjy?b}Y8foLN|_gfsVcF ze+j8~u^#cV$#7ck@@<|QEV#vPeabT0qq&0e=PXX#8eY+B4KtHnjuY>fB0kg5y`apV+AI1joK`?+=_nkE5h7_s;^n=L4yW8mUZl3*a{D=Fr{g%)L zn{}%cBbZ9!vtQ@A6rdFS8he;!vf~F=^u*wsIG%fD*3esk%v^RgbY>*)>koC*g#{@c6s}mY7E%Z)pYwQIArPf^D$9_R20$ z5Rg>8{+*onpp{Tb@kf6q%Bq?hSl@P=uvAAb8SHj#ZHj6n945dftwB5sSluZN6)`BV zKu@_jXOQj=$gS$>hXUokr(WLk0iZato*5#BdxF05~m~Q$==&_;<*NeI%NaWW@m-Iq6Y)gZh*Ze7+1tz0=H-c zn1j-=!+;t0UXIZof+k;K9YC7$>%FN1uqUa}%B-9k*mP_)%m_ZP*VTHf`!gO-)+YV%b5I52K%!BO+4zs?6ml1E>3>UAt7;Wk zkLrhab0Zs=aqor96iMm@>Sb94*XT7)zZ=RcFgF9iN4BDNBd1oY0Z1Hsro|0mG&J-( zb!8e5TZuP2Ay{{RWse3g7nC&ZB_wq%X$=Ut7^&vTNB@mun1;oS7&>$%pdFD-QbAj#8G2oT#awotpFTNffu4Z4$zYgZcGyq{#kHpI% zq$A5X51%}_*RH4MHCyir&QVTrY29JY6#+mKlyc>n55PnEY8%rR8CGs}{npY&y1^CD3ipIB2T9=$Gry30i{TI>DwqSIV#BR9k6iH7bsFU$sD z4u_`5#4~}P#vm=sMsJshrHISgd=dfUcWw(<2K@n-dX(Nbp{r4;Z@*t#KZGt*a;p{f z=t7G(QmX0YJm>&-*$9MfO{uYkH@NkR^@vp`J9WFk!L%SBAbcYC^Vzxodq+6owDJ55 zs0{@8#lAwD4+Lxk6e>bV|E%*u$}X-N8XBj2-Fj}v6K0X&A{j3rW+wrEFls`7MMar$ zuLgWz<-&je0zOY5^?FJB`{uy&;VO|m|A<{iSqSS}pcX1A_Rbj6k!D_NUOk~+FI87c z1H@q2muwBhEcMotv)04e$1o8$^p5M-!GdpyWgu7FMvWF2Blz6{a9fFc76EgHWDMwz-Rt3oI$WpuiDN!_`X-Jp95y;SM;f; z>q)O53LVFD0|q3EdWS%IHwidCTfi+=0cCU3K*;Yv%J|m5KR<;J`eq9G>3I_n>IdJy zzi$b-6r)>VvE>-XXB*^ybYH~XL3PEY#IVulN)^c@*_v(MCXl=KTfQ#Qt>_@#uTrcm zCS*j4(-*NlX1lY`^)c+_o3Rmpa$o(TY{9exlQ3k4w5fA6h@N1aZXW+nmEh zWRolB-GSf|hN@i?@Ob2XBmds{!w+50i1jd>}cxZZ&mZs|q3|4pNOtr=#Y! z_URYHO~|vRP}{vB6wMdS{mL&K-U%B6{H`v5vKHBn7lZ4ZmMhJEpCYoNXzaj<=TDzH zMz+N~FLcMG`t1E$Km7Gj&&{DeWIJ*-&2KL0o&C6oD@|weGwUy~iP8ryag<&hZws8| zILYSnA)o=kihd4kvP^@!9q}}yD`I_{cD|Dj^QGW&mjrs!Bv7P7RSrAzt>TD#Jpr$= zZ8htszsMR$Wm+wvcY>S9YXPoH{KZU>z97>C#m3&>lfXdnd#q#t#l{U4pTVY-QkEU328h$o#RCVVOvd$ z5;Zo<^fZaPE4Y9g3<6v$4;bBN934R_X7=$RfABE=lEk;!qHL!l5@##?A=%$W7$M!t z!HotU#CwyS1ned)knIYuF$dN^wN?8QY)w`n8PGkFa@OyD*oQgSjxVteU?8{#OE~l% zN@TT!hY~;ddzHN~8&si^gE}0`Il>th*glontdgf!-zu3&e)|JY0?bM>fQC&~YPM`2 zKlPo$nH=5DQFv+i`Dj(vKWvLoy$d2!RCU+rvSD~i*!u24LEawk3Udu-|C6ME$D|er zAh|tqFJWZ(OAUC~M#saaH<4^S|ElA&tY@co4q z#1`WGXF(E?yq94{k-EyUG^yzn(=5(#w>-Dx__GwL{|fNAH*zoLAirp)yx|Fiw`1e7 zORwwJKML_mfDgk6q|UP`TyaDDZt+dCFlK~piIC(!NdrN?G>zC6{M1N>uQKsTxcY>Y zkJwc3?}@a3DF{_(oYbr?=g)~BSsCO%K*m(Mv;&L-28ch;jafAcvln;w|dMpaZ>f^_*Rm#WwZZE;L2stA7D{{4G0M2h$;!J z|1~`}ojd73Gh*9XTh4|MO0SdpCy6kVEfpZPb0@`MW?dbD9{27J8(y_iwz60KH^=Zd z1G9Z3IsPjbX?1FJxY$4p+NFPzZAUdQGr*YsPhuIkyr-n?b$4Z%fZnr62!eUy>$Pi{~#G=6;ju6J6+R9U+{ORZ96_I zkR<{%|m9@XSMy0OK_FMQy^Ew8|y)U4R!_^ z_uM@U@L!2oq53CfIpK-2{8uQs>aLG(js(R^6OKsUdi^1H0yNC-W@oh2tkIpr8X?nYnR1OafSM=eONE*V4K1S4K3d#I~W->REE!6?< zs(i^5{%h6oXteL@Q{rBhUiN=lhGQ*e6_9HjEZG{>d-|+dZwMpGfe^~7`^Nz#$d>XD z+aG_V9uG!|uU;Zv0gKnZHi694yB2Isl(jj`R>P9Ava>&o&iBtyO*D8q+b4f;22Aq} z@_$nxUe)UVyB;&2Ut3B*Y*#L4p!=5>_tKcf`e;cK^%KpGs|tb83;^t6v$jk0s=6*h zN3MIw8d9YMzIFiKXK?3-;}6Pi+!LN~Xg;+FBBiV*=c#S^aGlOH&d9g!6sX*MS|w;b z^mr5CJrrFlD%8P^0LQY9+{JH}B>Dbk-%F72^R1UQcV7$_%$~l}&qV)IXf@gFoPat< z1HgMV0J`#|wpAFoJ21Xfp|8AM&n@8#CMs4+5=d}^|M+;*4Iou4^mx_Z7tZB;o(gf- zSBB)g#=XM}cpX?FfyO?9Sf6-N&+_v^OXZ#R&jVY&mUqshV=SfCp3*@G9{5~0d*qe? z@^=X5$wVZ8YZ8F^?TQ!q6F11=XB{HVKs;>)Bu>+duhL=LWzM({1Dkh?ZcP8L%x7l_ ziRWLrBBEs~zRxpJ$FG>1`^!>9+%4qwh!=R9PxusmZf{8em8um+zQ-)v7FVv{H3qn! z{nkRMpevKWTXMECQ)4UfV4JM?e{A{AnBW)&`t#M6vz{m=kMn5M@NM?k;|FfN|KM8zYkd1?F8o>vw+Y zHl}NN9F`4a7gKob=9a|yabyboaB>U)D+oa9VRwNo>jMmt>-NkfoSq&kg_=HU~EO0R*Nu`+6(l)bps71|40ilr0PSnPqhXYH^`l3gofVY0_;W?Tz zm49%%ZpWM70uk2u>pOpbS|Lf+g0FsBL{XzAcHN~ZB z^5VX4)Ktl3-5x@7%J-8jpG_y{+q_JMZ1{=9H%`f1 zs3;AaT2$FZK93AcCnF1vjaC-^Rb0Qp!4%bSfvh|=>j*0Y0N`VBrT8bUq!eHCawJP) z$3}*_$cD6v^hR+&O|71rUaJZ;oa1$7OLjG9RAwQ+E}O>*zog-BUl1z!E0>4!Y zAfmG_OqYRi7#FZeRE2E;1gkxnDH~hfXPise15;o}XyG@uh@qV6C zJ*UnctQ~a+IfT;k+K(1!*L;2?hnF5<1y!2g{|cDGInrpFPHiUY9jzx~KBDx=tCQD3VcJ?<7uR`4Pj8p+`(X+no z5mwEWnoWaQL7}rrkUl(G@m!gAXmR*l#bVr8N{0t1szhW4=PeKyv{by`pBM>xHvVjN&wEg!({@r7v=$+} z!FEo0SfpPg0M!u(On|s87$c=MPzvm*0yfh;<-DjZWK;W2ZKn?q-2L`i9beDV4P>y? zOG8h|HrQX}__=7qp^|wNDcT9zWF(wAIezon7IA@PWG4U~FA^^B;6dw~Sgv(KE{cQ9 zj!iY2xok?kMG3AtFC!%l^}eX9AJ_83YE)XOH7l)BhTsulDx()}25wvo;AgGdV!lj( z%@jA&qH83R1qkK<8EJZHB0K{GeX{`jfF8qeD@~K)!N=P6}ehDs+;}V$o zP}O1d+Pw-sqnP7tJ9JAZ$ggf zvY&Xfn7l$nd?2DrSZhF$<(advl{9?VVDN;j7*fjXB1-1M`MS8J9-rM(%J+lDL-Q~t zupR$RGMxVZWAClsqWs$KVMS00MF~ODLXeaONkKuSVQ7Yw9=aq3L=co#q#LBW1{g{} zLOP^NsiB9iA>KWo=f3afeviX={($c(o zLisKYW~}>wC2dFNgQ$Ox++^uL>#bBZhf~NiqmUuA*=*?dVg0yu4m_$|hql}PBcKY# z2YTwlQysL959O$})Yvx$#SteIJpH`Or(uQb3|+>P(~rNcdIrCwp~>BoNW$A{ThvUQ zZ4B*0bQZurqv}Ek+1#X<6{mXG%nOg)7UG@9%taS-da$Y%Vz(97JPzoh?hv*&-GY*KF^=Ma9b03&1gIx6wm&a13petijg8BoGS-gTQZ%JJ1<41TtGm3p%B^xA;uYf<84pi^N(j&R6sxTKje4KGrix1_m0M3AyBklR zfK1ENb>T;Jr>B32G@L`P&O?j7ye5C)CqTy+3(+8-Uhv@~_`)QqeBJ|Ya0|pp&tcOF z^EGPS)ZI<0iYH|_$XICw(ZkO@J47OC+Yda}JGh4|>DP{)QoEN4W{EyU@=Ic``9N!;b!BPC^n&Xvs>$t zUJp0khgr6pA_Gl5>7+$?Thg*asdvdpAta3%~z;uaree#ypZBVe{TcGI&qQ?iw8&}Ea!OMgb*J8)2@ zle^5kk@?7tBVCw_;LS2L*4yQklx*v87rB+TOtA z3<$`yR;U)CsmMJ7scf{C)}vCAwIg%aaj=2#T;di|5XPhh3+{+o4HrPaVwClQ-a%3) zh5B0Pvx7A%jCpKQCrRDZ5er$7^y&|}50oVj1b6AI4i_g&9=}I>ljiNGi zjr674{>GT(%SW;4P=@j}GtO5|c1dFqQlG`Dv57O7-{o=$FLqEO*N>D#7BcI|KCfxf zX^!s-gCl5)`O-^rkuxqNN}?yfCuMsDB%pk?>gswA`-G}@VecS2cnl&6Z&GCqMb02x zAY0dMeb(cXEaLe~vM7umFQ77g+j+sc^}jiyH5E{b-H-dfB+;npwAHf}8E3d+uf&-~ zfvgr=_RVajuqf^@r ziL|uYBF3DThJ{87Z`nk#Lyt+a8W4G>LmFhPYDHp09|v72vrm3Z*7o5%d30Zy-ig++ zzD>~^ZT2)KFqU*Ph{KGff49SI^=4N?a#!skZAWyeJ$6=q-bBltahG%~CiaSaMy0-q z*o=I&KCCg;-EI25JJZf|Z$e!}#Bz5YH-G5bY5bK)^{Z!%0d+-pw|kAV2Gv;i#F=2 zN(cv8h2LFeNYWcc#KZqLLSPP5P+b8K>Yxw@RWBoMP}R44f2e)M-cT-1)s648LWPBF z6A$?dAN3YWuqRh#3%>XpDp`m>_u@aaPYs{*k&pPJ^Ib|^s(*teU7z1le}lXm?xwYc zU~@H}*unD_!R3+WRsZX~F?jpNuXnyF4_`y!`Pd)vxTc=mc^aS*%vHYnI-n~r00Tw7 z!edcM{YcKC)sM)qE}Q#)1;OAHo-d0!hJ4}iosgH`%a5OWAX*t z?Z4fs!sWSFiY_me3xzp*Swe$5TH<1IzxN|(SdF}{C| zns*Jm;yqqoCv@67C8ueFP2UjZ=9BF zs8GB-X*1rtZ(7yGoSri#ndwO}usQAmE3NjYzpgUb2w4iaK|K1ie2gOFHn-ZJv2J1J zurxF{tsO|w(K-)Rr*}>S#!edSm(#-7EEB`6&O>*zB(j&CrNYP8ycn;P8!aPBQ=i)J zY`vsSxkjUT4P|}15=QhVuJamG7>?i#UYdU~U*ai)%zD%7Vw)^N#SP``KVltN)T216 zlAvyj;m#d>g;o1okq;F84viniR>wB|jZfGs7Z1U8+D#n!@cd+Nccf(Z_|n>NzqQz& zWRhH5{uR?+jkFxA)jgvl&2;4m?yTm5c#>Y4A)SoB5I02}c=q?QFmi^4aKQ|im`fUS zSN@8Ujdr&0*65V8$(142>F=iwFfSJMJNrEen$rYLr*HV82%C6~f_{1E_}9;gKPGM4 zOJ2{5FoMnW_u;i<=V_l-I^s`3Q?#(3)M;lFhM zhjxSWM$!nP=lw+%BE89^rO z72FPy`c>ZczpvdQ#2LNuU2Y07C3U|x<}Lf&q-03GyIp6(IGBg<(_u+)cpFAIm>zXb ztbu7aHphEv+Hf=q})UboRI=)veDt$RmCOn@c>|L)Y>)MA9N8mb97k6PeCa6b)WibkN0DMY_o#oyrJcfU%fs`YNPNq}jDB#y ziJgsI`P+=KSGlGA2AO6;UPIKvY;4iuK}(_@99U)jIpA*iQEUm^=s|M;Sw#-ReqcJY zlf8KIt?peK{nY1wUPNGIBP+@4pNyMeW9|5(Bk47LkaT5U!}Ye+L9eem{L0O9-`Nna zpsP2N$z{|7zY~qyKNfd@IwbzB62KaN)#)Efb7I-DATiT0`@~38FQUKLbXUrf$9#~Z z&v~u5R)PjTKC92SS}y)LVcY$R8@@KLvKLiYJ`^1Y*BAR9Z2>fcB@cjab41SkgN;H23#fmiq66dK4OQzXTil0_+(u#WI(N<>3orgldS#n4UFYsh24JD z6A4YQaB@{IEqdRTrPUUfj23S|7Tj+L4S}!xlCS0TJCS&{XQZN}fUa8;JY|lnc9SUBsd`6Vd|_W`YF&7p zaOL}!0gGO$M?rj4_-cZl$9FF524Ym)g_EFfTRuP>-KBT@OMR=4$$SiIIgN zMU|Vyv%l~Eau2-iyyYrXI|RG5hJe2d6X`$K?|~qh^;#}`Xls$$*2O?gaF}DE;@$mE5p0Szkiprg-_jcWnllYN7RG@e z%5(wW8>8|RKuFh#EpG$D*%T|zg9B5CJpfgY?+)vsOFv*QMH}guEv9>Z4d7aR>@^z_ z?tn376BHb^=~+6vcmMv;7i0_ju+=ysJ;A;=`SBE}ePu5KLXmxu`Rwd$(GIO~K-Ak~ zK()p|$hTFW8YB=d7M6xYy$>iTFL(~5IIClpg}cKpZDF@h-7Kq@ml=1Erp^VcQ+j2F znzS{l8b=C5$oxB3J&>~h0p+cR^A#SZNXa*;kJ@!g^8uNEB=W-u@4vi2f ziBUaXfadZTMMu2|MS|HI_&h# zy_z_6;GC7_oNtu@{*wn8$GPUAn^?I>zq9@ z#dRQ7m<0O9x}&?H!ZSWW%Sj~x?Q?pn`Y^>163X^PbV;99GrIbHWr}4yc*yQ5o`< z!qj4m9t1k-*~-nad{)b)=7Y!B!UGUn6}q)UpdLp!X9_3Xx;@9Od z1vdZHl2>O}UaYY_13Twh`~{n_aT5ViYCHh7PqnekgKCeR$JJnQ7zaXV4}G=s-;Cs+<^Z<$sX=q)FCX)mj%!7v{q5JrQKO1;#o zldZFNCZ|8bw5OWd$x$eq)0QI(+*pXk_0U!Av&YThQ|IxRc*Cd4 zh^p!FyeF6eSnF1VTF$ns8q!jF+VlO@>ec=v-}>JtYi+dNzONJpGD5U(IDS)%D>YGX z(Wz;r@H-N6UN3UBG(AjGdQPt)@d4LT@lfAV_vFdG19DBzd6+9%vs~clbqJs1f+hRi zSHu0~npXa^d%DR6^ClVYcQ_(|ykjiJG|?PuX^KyJ{~+B(5|i1fZtQgNX+!HRYBkL| z0uOeb%HB&ls?su-qJWDJa8^VUD62lHa|47odOPJCie+ z%xVCrQ4z_av}EN(JPD>ULa&wY`-$vLHBYfHJ3r2DBtV~rRK*#L5R_JPm)3{(l&Dw> z?z^L-Yq#rrs#NPs6~hdC%|~~HGd@P1i&-86x_{owTKISHX*BuK#=+-4IxS2qMDCX| z8`7Ds7%dtgZdYtlr7(1~A2p!!3sx}3D6WJM?xA+B1a!NV7lCKk(wRw z_*%6A8)hf`xA?6$E#jE_+R189yF1{4SWsSGH=N6}c6l$_ zwT*p1b|?ovk2314Y`=B5?5XvD{s~cLCneZ7pu4$C6dsZ#IT&8mH zLIs~}#njt2)WGSci3?CBs@UzIr^Qr1Zrw4UR85EU_JbO^gjYvPnLf)CVBS2eZP|fD zSQj_nSn?O=;}4Q0RXq)V>+k!4NKkf=W4lO}*+%msc_Z+Pc$w3}pwHlFGMUFg7>Nd& z79L-&h;rXhq7zx8w#8Mt8sVOyipA8x7Hyl_2tV=#r}c4=+MuL>8FqiI>#&w({&(UW zm8xMYBYr=gg}OROgNYdB!+74ouP_#G?kku)wu%Y+2E;&#JB=xNa5v8Cz5t_b>rF-+ z+dW!I%Ym$kZsue=M*Y?WtZ)vqxI3JQu9tS&%}4BK1|?C3>v{3t_u6sI$vN3AkT%R< z8wz|N7J3LrwEt`(9HmUH8vHUi6LlgVgkK>>l8?%v$*I~BkjdXFRdiKM_a|4Vq0E2O zb7Qu!ntwNUa5aL0$8ibLR#3_M%R%~Q^r-Ij{&)e6oT^@VQ7qB3JJZZT59@7@KXCCb zZ1p79^K!Hjh8zx5ugnzxymP5p0_sIPxNKQAC+ss*tYDLko|7NGjN@_ffR)VXu44L8 zL~>aBf@&`D8syuxO>&F{$gnaTQF8C7#2NgjaV7K2pq6|7>?dC9Ya zrLe}H0Ok-Cb9PWU8zQr!?}{4@9=d5xG}I)=;#MRU<#U+X9=o z8+8jF%sUiq!An#n3z~Xt{D7p_94^FVS{aWl<|$PLabw;pMy=)B7-`Ide1dC@+t0Zr za2dMCyTUNX_>OTd{Y9L9g8QvfcIIeC+30JSwSA0;=}L9CgdD&1w2|RZ?iu-7Yfx}> z74-P1C$QhP?X8yQx_w3vhWGyT{;ST@{`@|Rxt`5&ho;m)l0d0y)u@nwcH62+yGt!F zT9c4IFHr^)Ez9)9<1r}beCvnn5fQcZFt2CpsUCT+aR``_fAx=eH~}QYOjAiS@%sj! z{qZ}2!*6nLQv9sA@AoomH6!890lO5UpSgJA9=*;r|8}?4iq@*=ZeIK6#!o4U4%CvDNr_NX6axDvtDF(p zK&3@{ercrGC{xp#Pwy1nng|!Xm|ykcv}4INW$(hcWU~BX!t^3bx79e`YAp30F&klj zw&Q|l%K@kCtgk;{7gbg)Iz@XKu(3`0&r-E9s;B77+N*ul7H z95>%-Sf0~(XM~KdBDNeZdsAHy;=p^R&?e$#l!1sqpV?%gkjP>4#yBUr8O(2<(i51U#)5^Br%i{Cc0=SSgi>{>w$YpTxu8Y@+f`Ujq+3n3>N1dh~}O*w>QmfGr)nWiAGV=ugZN1fsx z>7#{9NZB6S#^djUm)~t)fG*sg$g{3;Vk_jFM(NfcLxyw4{gyL2yx3gMUUuBSm7$C+ z0S_sxKYjFeN;A3RxOuK0;>|uTa+o=d9_^TYV3b?GZ|!EIJipO%Jo4LBn#|0^d+Dmv z(!N&heqa=+ESe6c?dyzFsvIQ!xR+?O>>f5$wo~Zont3b58gYC{tddJ@wcN@vw#t1^ zIk;ZC!h*E4XM#aW{-C(U+#}5n7uhp*mlhLT>KK(H{u8%SZxiI#bf7fHnd(gy@@Fn# z1z*(XU&ZggUJGVWtS|Po)Wjs$bve+tXd!tdZGsaau4iA}5Z!omZu=Dv>YLTY>Q#Hn zrE|XL?|+roc<5~}v_N=q_Dz8DYYaYc3RxbU3aw-kf@S{xrWN(6yl-B0ii&mj(Wn{U z8epx2=uw`ma76grR(Eh{w%{}Cv-S##8T&563gayuI_U4I_f(&CKeRa(PB}gAcv>Tx zBziu?Da1AIck}UI_an#fuqlO|mKFe^>lel$)Vz0B3=Zb|>OJ)epO#q)S+WirCAi6H zPrqMS?9J5;4@uf_0NZ5NaG5+rz}x=O5Mk!b$3NT#*m|i{lEKjQ$e$SQ;vrz2o_tFs z)2DkWZiO}-&7pz12{&@~cgp2PvY{xApv(_8a5$A*+#7g z3qt6miIF`@BdChyH%tm&971L{|9#%m+5*YIu}SN^-`r>KXB;cSa)fuS|5ui~bU~b? ztz;G-x)AYE^+QVT&?G`c1KO`Mwe7Fnv>_xjFhj3 zvePbPyGYmHJb65^H#9xllr)vKFFfiC9qYO7z9Yus)>Zd%PdTAp!Of%gz;l&D`yEp}sEG;(tfTqr`v}x{cDDM;(VvHr z?A;qbN@*3ghv;NMJi=GQ*8+|GA{Z%H}{4l(wP{E@i}4*B2$^g{rgr z5x~!TNIxp#I;!mVK?l75o z5b=1^%H}MsVOy|ihtO7b{4dYNjNSW?Dckca_*w4%BYr2}BBJSp9yD|7T0(i4tFF z(bz=7iOqtV`3|}2=`gCss;irv>04gSy>&HwOIe1tYaPuR2JQ|1SB( zh8-7~*(b${an|itH@GtkVFAi-ntJ$(P}$aLC3zQ8L@(5nqua{>O(|xsQ@WSNkRNXp zZ8Vzkhs6=xfVcwz&x@>%i&s`1^c`0osykR~Hw>L@y_25ViaH|;_oUV?TP;SsI9%F2%ScYyy5#+>uW3{N7JY4=>qecS7vu@);UJF zR?R$=-6{`MlaK&gJyq$Q{ zQ@t4G(17U!372rhd7HvfiuZ}79CRW=Y&RS_A9*>NjXyB481T&5l}p^x4l1OiCB1(_ zA{IhTmlW#8<`?S5ElPbTAC!2|0U0>@yo9F~V>2*fSUel+*<7IX+tux#KX_B&#@++v zFPoVHhuOg_KAoGqj%glx)&n_mom>N~eHl*UlZrA6w1+%1uG;P7xU}Z2 zAtMhWN8rCd9AG2ap2Y0C`?86m?@J#ybcBeNQa#A;I%!4h9D`yX_KR1v?F+YNd0T%# z->U|0V85SdW#ZoV@ty3TPTrGg!!SaGxFLTP8Lj}IpwwT7G)_GHEfRL{0un2Y}FkLBUW(JkW? z>n2@DlfeiE41V=2zji+4amdrZl>KQ^qU4NYS?TX2ol9PDV ztn@6qV?{)olb;Q5{D__%DgE0Z5a4uyLEtIs2{RkcfIiR|ZRG+FbT<00A7sYqqFZ(b z8LDv`!ZiEUXb%a+|NV0~2ph~N6^x7qPksrfq_MmZHHVazEcO|71Ptb9o^ReQs|0uUcs{U;UM z1h*usE0}GH%u#ZJl<{DG4ZX8J1*4L6O@-#*y>I5|sllAYG=YqDr{luzk+ukZi;bI{kr%e8ur zNrSHoXs#kMMlObshur+z(lUQXaVLen*XM|;KceIO@W}Yj;{K6FcD>&X!aq@zoQ*fe zw@Z*|3YY1seat$wR`3erAC+P1j55EC6X>EXDxd}+I?4R%g^ouZ_N{ReS*~9ElBRK_v_Hk2f@5euxI@ z#g)YDFDf3J%@myDu+pahUN<%S27qCOj%@Z$mamV1yX&=S_&(HeWrz(ROObB7yFfl0 zytg*q3anRk3zy;yQzbNbo{Ft#wE~+@GGdV(#n>P?w|W2PvNKl$VD=gWd`^oBo*vZw zey3x2(7OO!kVNXp+@{PI);BGii9uTLB9l_RSNtANO;5i=!*eP>_Eo-I1riXGVt7T| z$d>ALB#ZN;({LrQa+?Lls_%gU_Yu|_Oas0-ehM;yo2zVjh z>VCA6w>Vs=*9rdL)xGFVw|xe*;@M}1+zSPF&uS-7K%Hs;2n$B@ld8h)1JD*C6yTFw zwZp(|^|&I_75fXVpn1@Bz;8Oj0Gc*XGY#RGvr&PwQI&eV74QS}?5T!jetEwF5K$Wf zcV4?r&eqSU3m1vZI@haVzh4|m!;t{=`;(FI#UA49-wKj+#9n@^Ex{%?vtYC9gWoSJe*08mo; zT)YwhtAFjS#VLz;|g+E(Q}Pd55xI1>Ddp!(i;hAD4vycjIypRkp@ zwuil>YQXS;=7&A^vYcrdI5 z6zJ*LK4vZdt~{u=tZV;`!JHN{0v>Oi9ROTG)ahKuy7K{_R$)%Su4md}-csnk1BPem zVWG^u6D=Fv5PqAYhKmjMSp@0Rz;Z~s$m3`N;J=hY&aimkKeh=NRos(QCD+blAX%5% zGFa62Lz+Yb3d8Uj3M(`z0D9|}T}F)tk#D_{MBy`lS`7tukGx<`RPms79J(SFynu9W z7D%ZTI!x*vL06o&A8_{$$3Iy%bOPTrSWd|O-d6tcP!PDTS`;U9l$D^_i}FDahI@>_ zjtI5)6L>7kP~hg3v@bkF$8-)liJZoiYV8^T=R|;e>Z;X;sibvV~Zlex_ z0i}`#DpAZu7g}Ee==dGb#LpFUxBfbP#BRkrx#BA(P>f@qN5r8^$f3}FLWp2OIt`w{ z4T1YGC&Fu@SnFszut>Ego$h3k-N&}Uvja5f$ zN&l7~4L&}KA1qpc0RV?;`qlz?BoU3{JO=Y(xYhtf*CxOP>)K=`=z=y-GhrI(zC`kg zT)>F_pvxml?-a3d88wbN0G7K0 zTtM}|Kz2Ou2bkh*uTnX$!wx-V+v=|*GN1K7CC9fb=D^Ky18^&F2jgHR&#A^-fk~Uf zdA=1!{fuJPRnV*H{VhoA`;8BZaX*VTQ>Jl;aig#$7m%w_Ned@{a5Vxw1%m)i$IJ(0 z_>ML+z}J8gFKV^o*s7g1I_3lc?(b3)1u>1l!!@DE#}B*gmnfQ^kuq`9@C{BD*6wL( z*>IjoR@Qg%jeqt8w%IceYK+|lT7XH!d|B6oWbEx~buIJs965snYu3w$f(6-S-ER(a z-h2E$LlWmE{PcaJBW5sJ6w}(w!H?d^(9}DgtihonL>%E?6DASYbsALN1o}gDJYUF) za)jW3EO&Z<(z;DG@CZ+TtK9ISnJwjBL>Q5}Oq=xG?{jrE5aFT0%R^Zk#)4H=eNQRm z3Y89#U|=%Et#k7cFW`5{7#0~IQ?F)&Gpp@Co?>fgkd8|q8%aAuU2`*EDs*@yb z>!5z73YxpgsoFid(e9f7p(XW115x)kyFpcTx!Z?8_!!M>mwnX)DzsW839J0+_w7S+ zq?DN(D#0bgYPKAhOlRc0lrT8*0DZDDUg+d_W-m7uh#FoYC8#?*-)ps;;*?6fR-`XH^fCP1&^A zSY|u+TWwSn#EPnNcWF=uP2sb_DKUcVe>lvMg^PbvU4BzxTQQglTGdd5r@_rHo)0-6 zf~S3gdtWa06SFd~Kn#QAKX+ogeLEtJF~Lz8Ic>(b!_jb0j97R>+yn4^=FjwVY)X4W zdAZ8i8W@^RhXyIt1ij2=V;628ynLsFW8`O-Q4;y0f5{4>Pcm{{wnQ$@3Za=J&#)Mm zD76vAyMA8wKnr?TpCO2$H{IZih^`pu5SH_8+y9W^Rga%(MovuR70IrLjXcaUqxXk8 z>OSY}zq=dq3{ZK*La&lD>M#@ai39g92OY|PL1XY4kfF4`d&w~9S5o^Au_%Vh3^~#<| zax~4F4EN<%L_9IzC1xnt)>eUiF`bu*ForcGg2Z3Q(413T;` zy4%-pkUT^2C8}`7^XAxOX4-`su4l?TWs7LkoVS5?Sb*{hEwRB|&AY2}Q!Nwf2*@}n zuo&QM59@>7^?uZ3NoweD_A)0XTj2gzV{O43X2Ph`X52BIXJ1f0Uxd#N*iOflZmGCy5MC9 z6?q;6(ogv&MnB}IBff#9WnMVu%fjik3FBY_BR`7!2Xj;9MDv}GN--H*(QDs`^-B*k z_xCSS!@}0X-Lb|yVZa}TUY5MEMl{U|nCd)uX6kMR6AEVjy618I-(CQ^T2|vhH z<8__*-L$55@k?hP1f~RqKm6UVlJ=8~DKp3&$i3YGyIRcp^Ujm7o1U@m%Z5+R)l52% zH>XGGQk@;Nip@kDg*>O=c!*p22Y}m5Alter8%ON-$<_WYbUjYbx#oQzP@nv*$SQTD_epXu+&VEAxi4T#vFP? zD30g&8f)^D=BuYYG0nzN&`4%S{KLyx6(fd$JovX)KsSNdH`A-4xuZ4QTXF7#GcTCr zN1nZ_9Jj7ccVFplvyX8BPTsPrt(Jwg&SMmn++;i}Y0Om!D?=(jmId7s+;O69Stq{z z42gG*!~c#^=Ch`+2*C-2L(PRM5mm`X}_?;wrd zA5zm7kCW#lp&6eynm-Yq*weiyg&CCn+RE-tk~Pz1 zMfSK^=Q;c$o6$dI_5WaFTCw7zzQ|I<3KalC2mAN*0!MZC*f1l%kEwqaJ8>COrZ~kY z+KOeIYU#q!8!&TB(P~8$yuGgcFAzm930^z%cjHS}bzYrCR?>I4#Z6`!*Sb#%{QlI1 z1#BPTwzGeS!n(^1V_em8jY{Fw5ZsZdRh2(%H~-|aY^uqt5>V&4FH@3Q%crc>YtDL+ zlk84De+CArm=q&ZtoK7|jS{AWc*7dM13@w zrX|=WLC1!Ok(KWJUV``fcTC!C`qgaIPR#GZ^?wG86Q8FnxscUguiLq8OQa$|6dKi* zcvSracf@p6=x>>aHyYWZN0 zCltB@=l94loqvg}*EscrpWs;?CRbOv?~Rq4qeMZ&=Dq8f_P>7ajMVIUigjj8wYuM1 z_fwl(IOG$~>fODF-)}Sj0)I{!d+$g<`CPj2_Cs7Bb??`+?JFsP*)t;jhlS^V0w`K5 zx!D&9+dG2q$GNS*)p&5$<5EY-ouB*}S8Br@yP@etRW20R!}~BY+S1HJWMNhc8cJvr z{Y9A8{Sv0-uXj5@P?n__)irGuvz?*3LH@_Fz+O!9GdwMDBzTG>UXA=?q+r8EG3|AV zbudrepYYEAIV$qhgLiJDFS!5yF!Jy0=sS~smiFCXn_rjv{QF(lzjkrxXN@0YzR>;s zt-J)3%R|p*segZP{nvMhZxd23^PPznUHbc5gK_+jy!{7$e|=89$Kah^^~9@xW1s)u zBmIAm6dR?V3crBkCKXWrJe5rpu4)OU;kAHx0Nk!zUS+mZZl?uQgU6c54HlY!#x|Qk zjUoN?>C;_Oo~B$#(;Mq9TpgLV0;fO z-8iwR&dwMp5G6;0qdNqR5)}lEM2x^`J%PEc3gEg%?-?SW6>d+xK7YXwSf>YJtu4FU zKnHlW-P)6IMV;crdepn2T28*h>0bnFB6IIhW)z$v&WB%5H{S*0NNp@{Q?|6XJina)d2N<2Sx5dxMG8@Edx>=Y%0Jf{$_!PrX2P5q|0>kRiV|e90jav#sO$8cN28M2?T7wLoBA2bEP}vsb>#(oDEM2 zKvg+vs|r=QaY|yTAj%Ip@2bwFsAS3y7R(TP9Y`L7zC;S>>(0uxZs6GS zxz40eUC2m87aN>dyx(rbz@H&b>|{ydv0iMhee{VfXg(ta5Yab*yd(6hrC5X;nCg|f z!9Ill*V?ZrjB|)$2aT}QkhdbHgW(07A_TyCa)}2uGf*0T)H(2JB}c(4Uh2O$-B|f# ztAYHS~urGg2?vZzXq^* zM>J4CC`TBnK{fP>u_z^wvl#(e<#`G@dGsJJ&SZCjP-a*Gavz>wgLnCO+HEC&1n9DS z+O5Z=sIADbzzi1IF9sv>SwBqwV_4KOjOyni!2iP=8w#M+8N>ExOH;M(s{qnpVug-&sxvGc8f6gPKf~~Iz~O(9 z=LvOEe*MXs%>&!?l1L8!ET(R{(tH3#>VT<4-3RRL$Ye?7hX-3i>684@GurEbN(gjo z2ps{WLJoV4{yTx8$MpEMARNcdrt9FqK$L%}{<=<09{|xHf}LDgoOT49GW#uJLYZ+- z#~J>JxoZX0N(GE}!JxF}NUz>2bDW7%yw1ir~J zg@>Y8d9uU4(l%n7>@?CCa=W`)Ff-aXQga5J*Wd9D>u}S#Ev7iK7Xx@iJz+c^kY!5~ zi607XXy|(%tmvJ^cF*4AeCI%LI%LYj!v1F7W>cCT&I8r|Ru)9`b@3T)FR+&NS_SvH zl&ZBbY7%E<2=r~{)RhNueN|a6bj{Nv-Vj=XxEs=^#kPRLLcCxGMerOAn`b6OMuu0j zvom44u?@IfIh?+L?c8PuXemYxDYMkG30)NZ*?}~&C~9{j`g-hvsCOXzaOw@3S53#R z*)j2}%jLlYOFPl8u0vq$xCSPf12XxOQet$J!*gN5{h5gXdfj?Zvaig`P0@;_hl&A; zoH;&X`Z-+1%S88FuLGo5^cw-uGn)uCub)w=fRWG zgF`D(xOy*Y-UD!IOTH1d!9~%eW2`y_RwAPw@vB=+?^E;3@Gd_y*L>D$B%kFffB9SQ zPh6MyO41;cV8y5~zB8FOt0xoFzCIoXU!thNnZCb^;=#Z`bP|UNFgX5E32thD;yi9} z9+vwE1;fa(8@P6dXF(=P2wlTXdxo2ai-3{p-P&NlZ{TTokVIP7X)vB`Juh=vG+fx^ z-NF%1-CQzrOwUEVhc$rTDh9g73=aM1q}{D5pp0r7h`Q-X9I1 zBMZ9oC5n^BFRH+r_HbpV>c3J$Zf3>FDnhS{ZGJI|Pg@nd4XBNuO|yV3m7_={xk~by z#0^U|E%80@O$=IMq@a=o&|;_6riD zL%cPo z0Gs-}>mVv|Zlct*rwp!h(B3A=8h+Bae+{=SzO+cbyMB=$Wgk!a-{m)E<{$&ao0#>j zgsj@(wBcW6gvP&iTpi)W6Rb(ZgX-YR+|{{5lR}GO7<%YpAU&4Ya)0F{6%$^`(c=QS|c1uyi)@%I9e%JQc=0u1WOrVTa$S7{?%y zLoQ}NVY;5%>0frHh9d~c4*GpI!bRXlSIs$ITl($tZQTJQ4vwV>*ZafwGx*JIm%FO`_X!w)u#~ z!cT{1Z`t+MH=@r)7a@R9w;0UYjOKoNgnti*-Z{Gn3}ceV$ASP9;T7+=K>#+pi5&EJYWw972E%?u&iU%Pshho=Z>LR8{ z+j3#1V6f+8NJDQ_@EaHmRc^#=`>syJ)nl@$7{upN+wwN4$m<{__TGCy?D3J^Y+l$1 zh<)Ty#PN*R`b~m|@tN@>FA-6Ob<1U)TbK~EoQU*0?c-qoY<%x7vx0e0YLhEI>OxJ= z(-ho7-_N56(nb5a#-{~5-G|%5&2KAk2S&5R%p2RiDxZbV+Y;XIcF<5T!Mcim1?g0h zgoX9Drm7ov@vc{r)NqN2>ey-xRNbK1+M>M}(I0QA1NH~=LD_oX%bu~#)y)W;zw5|k z9qJ8o(hn1Hjk?!L+^=m$#NO z^3EPexKZ}=?G&W(1MEOq-ukjxWjl|M(kP&!2Jor&`_tP)!*5chQ2Zp`-k|eVQ2VK! zBx+yIy!}v23^H%03cImkYxu1<|Jat~+qI{o`07?2Jqe+%4!J^>aXtG1>bD3Nsw{*p zX&7l-i2CB7g47QU9+dcSkl)B6dQ|l>F*$5{V4$)Qgw?Wzj=|! zE_kim2BvVjB8+nKqn-^Umw+Bg(N~Qn-i|9hQ=8-9*cVJvulwF^qBoX#* zVU=@WV1%#HNkX?Sd8Xsmf1^3UZ2v4yUffSOZ!@YQ%2tJ6*XQD6DwDY_M_r#e%YP6t zMNUMr`Phcn*K7V$5kSVyMMXgYOvDFSTp}HRnpk;Jm*%b7cn#@IA~lIoMm0Utv%Wo$ zbrY{ZQdoaw@~4)!@rS{?MXNCC()~??6Wokr+q$sHH~vLY+YFJdZ{v8njVd4RmA3zX z?7e3|Q`^=BiU@)fL8^!p<)}yrN>{oF7NiJ-9uSdU1w!v476fecF498pB%w<02!s{_ zDpCVMq(u5#IrqEw_?>&y_vih3-@in*JA1Fa_FQX@ImVc$3y?XIwAuDoMn026Rd3#V zVgu2glQ|5ze>SK`F8iMG_jH9(EBZADo>`(?S!Y&Hz+7GLw8K6C{RRK?9`N@wQP9C@ z%VR;s1hozf5puT8sk4RBEcwsBhoo35i!jHyFsVK=ljPAyxxyj(EyEb;#?COA&ROfY zGbfvdW~JUkG&C-!msQ9fBCxh-b;g>w-aJ_tXI}k%ASiDFsh#k8Wl8G%ZfJFA3SCdj zO{w#X7>PO)bFbO5-YbIiBCV`Tr*^~(Or>U<VOIg2b)TiwSKeNmuzbEqGk&P4|r}z`~e4vU~l9 ztw1YVyMpJZ_m;gd>FmsVulr+*zX6Yf+f`;rKrqocp*D~0Z@rm}AssaZ8t;yNUc{4e#oNv1j-_IM(JJ z;55DBMXtA9N2(LA(L-djSN9G#sg^nFE;B*I5GSUrwN|@oLec95S#H8%saEeU_GVP~ z(9da~VA^!$R-WE{E@0EER&DTxF|HNK9OYebI^x!}Q2UH*KO2go+eyGy4{QP_e9iNobd8-)FMMJquB0FT>I4>IWZAn?Ag&} zg;TVcsHmy93`?(e6Xw)jv^>?*OK(5QsKl18KP-J{T0Yg#*djt=;Hd$@Z_IaM@Nsd6 zUIMw_YD3+uygHpi_f1J9h_Vmp-u#E7uH^F*thl5vgVmzRzWNUsj5XTVI6wVCjQxBq zWjfqfVl|T@;^GTlvHIhmx!t-iEyx#`eCX+rH+!qKl^|zj=ojFTmmhLgcUHcht}%M* zd(vtqZBw`ySE_`-G>5B@{e*S6NbvW8$0GMfO(9Mn!)eJKu4+~$!^u`*@TtH7?Nz=4(x}U)G)YM#++X#Z0jTH(?=&4oT&W9vRj2>#f#jSsBT` z#9%gi@&3tZq)Crvs>|K2-JwdSVKx4~OO5jFgxuS#hxrZ_M{zv+k;t#8Lp|h9crSOa zEZyfa)n<8Qg*dR*ABRvcGlxV5@vmdAIcTRjuA+5*2#^*sr_?FvbAE<969JLJQSDrs~j zw_ikX@P(HI^m108zcj}*x|U2a7ny|C^(<5g>gpc|U9P?)$DZJ-Btralb}92}?8-Um zk$Xm)Tp4`FLAy*)C^j0&(%6un;}eQ?GpNH#lZZY(Q|EtNODKZ%q64)wZI5y;N(yLo zl2iB@KJq+W`SA7Ku<$~nKMul>hf*K_$@R=Jx@-xwp@_Pi!__>;}29ZF68Pv9_gaG>!re^K*D^}fc>r(ELJovNiGp1FTJy^1O?4Hv+@|q;i zoZN0i@6^hM;|7eKJ*^Bj#@JgStWd?%-F8{UqxV%{VT)^A*qtZR2c216F@0Aq*LNz6 z+>ft`NiMS-9N9=Di=8=6F}IS9n%(%3WB<%BEqzQAdvEf@4uGyB_!M16z~0b^T|=uw z(@i1-DqxRHND|N^P$tpiO<=&yLVt2PoC{%dRM5>BZf5l*;LuC-^NB529=E_T&@o(q zn5i0gJM#hDzfaE(xK?BdOf7l)2Lig+0`#@^t56!seVz3QoSg{SAg0R=j(Da{Q+FRrQUE0r;WAPtWu#7U$={JRr)NiF)?;)L_#7 z%;oGK5}F?l_-h9Wlsbnf!P5I=;q9+$XQXP~Re-g#84b<+Zy@rY&P&O1aY&dq>o-ap zQiYR77{Hh|cVd63$N2MlS+p9os)FJmxBgo2e)5UG98@pTRbVNEP#JV=78l+uHx&X&p~_7zU!}T z(Pw^kC&Am)pr&%Y!%C^(6sX9Bua(3{Qz z-VhJ1NR3Ou8V=yoC4gAh3=B++Hp(!1#_D2yzy$bSl-y?A>_&nq(89~y1DxEvh2ID&!_tQ#!j;Qa?M)rDqsNW3+Gi*-ELdehFO@M za&142sfBKg13*v;Q}Pw$)-4MAt5HJ=+kGBG!{0_$0THwr&|+RQn5ekiwMc}_ z2P8IOg;45`@6wOeE*k#+Q{;|`7MXa$E<*rSO|SxOwjv)g$V<0-ZK4khLy@fz})>Nwu>B zH2_N*u?1@Y>}JJ3-!}n}*$1#1*iKGAVmI@i-x=rFy@0hD=AlUYbkaF+!0bnzJs;o!Sm=0{d z|Uo27HvJ0EM+#yll=F{YtmH zuzcWFZfCT>^;vH)QrdSrN^=bMfE|YR2H&7I<;|Sy8B7~s(u8C5&E%jMQlISw`2Oh^ z`FS+ucAq<`0IOMDX$F*761O}02au=j2<31&dEncl{UkyCx+Na8;hhFm6Lm>lG7Hs5 z4gvPS+DE8z^xi4}?Jrv;*k&QBu;sSQU{`s4dkegdhuiofrQa^@Epjv3l{(G-$-gEM z%1IDr3Rp!A+0F(aXC;U>mRTNf;WV z`<*F`6do4pYCvhhQ;V{rxPsLDm*#2i$xMVhh%{;|06-5^8)Tj=5kjZ}S32Ez&io6B@+DwW_O;sXy^lO^=sB0(aS%h2xwQ?| z(zXK+h=B{2p;>g4NyZ(w;+}Cuv|%JFNeNbEWehrEJ-Abk+_>^}=VJb9oMI9&=VRva zt7TEt6SlfLR?At5PR{@v{Svi~yl|>WN??WnLq{}J`SQnt+K-Itpa>!Komt?Fcx=dNQCO>O@ z`@tUQ9+vqjZq2Oi7laB$H(K4)E8?C-CiUbrEG9vV*Np;3+x_*+e2c!xW1<0@+6u8v z+LmbteFuo?HPC}h+u(-8IZXaU)tg1F_pp-~G@TAMQ!Q5aX2-=2T?Hm-(c2fhcpIko z3&8}mlieJ(SlF+ocpsWU=bI=&=-L7GC`--kZFPkhBfDG#PmDUJLV({cpzue4N{Oa~ z|LIhR@(-%E#s=q4OJX|cZ7C*SpHqJZoEYj%%kHSF&4@#bLD+R7DMm1QqfDQzB52G) z+>>1MG&t(wPUjx=%FRhhQLWwuB*%9yl7Ru6r<+VihI6!LrN{1t3u~u!)jmhY1zn8u zogA<;rdk1H;7oCL@-I^|vBd1E)vb1TcTAIf4q@YegAsqh5`TW`X8|j}div587Jl+W zVU$^xM3$LL!Gf41U5u45tx9fb=5d=#Z!Ve(vu5dj72Smpy)v1~DsQQV=$(Y73t0fl zm}VDm9N$DY*RUk<0g$F-`G^RAsXx#<-P!+PtPiYZiIe8>x{JeAm;Bs^>&L^+^uBkA z=_;-l8>ns*Vy)Y-4J{5W)qSB250`NS&prw)Nx62%=?c2*(&fIj^Ob;n_lVNvLr!}1 zw_Q5FcF~>pYoeEWcemW`5WSZR`1x$D6u66~k$`s|Czw$xOv7U9QcxrvXOHuC64uU$ zigoA|+CdQ7s4J@jPnx1G%O&r1H1TWLbF;VCWnLRR8x*zV&kL3 zP(4FSM9a`nJ+RX|j~j{k`YKJ-9&j%;0ehYC1BtwyKmh++Q(4oD6i(?&di+8+&nLND zT>^RwsFFz0NV(HwUg4!sNoXBiTTksH;QPplUMf{fctc}M=rJ+o^U5}gZ*3vh-Sb$P zlv9m76;EjC#!D1Dq%n)6eeZM_2$z4IQ+FU(!wGXWGvVb#88=Ds0ISiWb$)3=Qdm&bTiB2q- z7h0lw#g#!fME2^(z&*U#h4Qof=kj|H*OBaaATCK1@SXyQs{4U)NC#;f2I%)G;@3n6 znfaN!j~^>_KzIRju;QvL-^E#CN#Wc?V06Rl7vfdRpoE?)Qq4$4Dw{1VgKGT+dJ*-R zS>QeyceQO|D~q@a+8++|H|ciGRwir{fjMSYkSsBv+M(%{8iWHVm2MGh&IYx*rNyGhuoRUr43gUe8L4Y$m*&2A5DI)j%d~fk#*r@*#+9hJ z-n`#-$Z@QEWD)*x9n{;Of(^(6M4F^0SN`K=QxdMzawN27OfJ-Iq^;zpCB$4T8bex( z*IY#A#!o_fOujbE1>0G@w+f?~druju|0A~gzc%xLwRE8$@%HV#TW%f=aKSW19J%u2 zV+EV?S@1r?Jr-SW(RESQvN6V5jONx(9z`YHPNIx(y3tIa!dsxj8SK7wR~BrB5f`6- zir_dGx05zowHTAE(DPh2o94|YH!22h*`9mz?2TNQHYB#SzyHMX1WAv)fd1oXE4=K;+`XN>Q`?8-zX%xHGiu^@1 z`JdgKVlGHGW45`=5B~N<{_CsOFo@8i@Qc{OhZtA?b!T2=M8DPBH%)sk9-GldJhNj0gVwdH(Ah{Py66 zN+)Zbl75%#zCnQ)ox7s6Q}n;Hl>fd|OHxWcj2?;j`BVJYr3P!WgEW6b&gS6P3UWYG z%m+8rgHu9!+``~(L0k39Mo3_Qyr zV%nbmUqk$lpMp!-zzyMX_}1S+9RD{t|8H{s>OK0u$@%{(a`smaNj2;X_G?oBW94ye zNzC#O1jv7QMca>)9I`H*okKGGF64m!{p?1=r`T{C>(SrysCmsAw4}_M=YA7&09<T22$Yt1_J(?ieKTcUXnjAh2DliV zCjJ+I=$od?h06P{R@9jhoIhV{I$AU# z35e<6zjcXNeE<51q&jZ{7}`k^QK=LE>U&dn|CW+oMP zNTmr`ARAuT1Ggh3>mhza-U+3?8(wl2ijdiK3qYoRF8W z2~1dkKSdlLse1Bf`9>}%JHCY-%sKN;`e^xG{NTq`D za@*JMZ^mo@zZ3m}H4+u+TlLrXw9}w^1eoNoqd^MaKRz7N4>8?V;e|bERk<)>92;=F za)4BnD?e6=Mrjd&iT|yYK1r+tJPZZ26DFqseoYd6P??MGoztoZN(l>KfDxh0AXjcD z78f%@9Su52Y|c+0-4bF(wpIib);oTs6BE|CyU2uG5a-P|02sKA;w$tMQaOWKDd$^{ zv+iN0C9R0-4R5A$2pJ)fiiEOISdDGG$?1VhZ}{WWP+L%Ad_oJU=A{;)R@+}W*cUlMfAUS3(2V~ipYG}T(?gl4 zC`5(Bs2*YKSX>QRM0uzJgL^Gm!F_;sv(ogg}T~aPt60H}7mo?St0izZ_+)6f7v$W;}O{ zTef=Kl4K2JE9YmrCHvvTnYcc&)C{p+X+9|O+m#g(wW!ghDb_5&`pw0zW2+N2WEyu3F6_Oa> z%MCJ()KQPkqfPqO#Pf6Ya=TK`J}u{=Uu4>)%l{{V|L^D#T*}UWBS($(WH5bGoc0GF zV6mS`mIc`SM_ov6(?B#NNvT9)-ykHxpvqfR4$^xs5?#moYmv zz$eLl;3l*Pv-wfsU`z9&!fECR@1t?lZ%b9508gRV=0jAc7{Q?~G|!J3l&V4@tRA5~ zz=lF+msDX2hO96TXw47@H5F7q^er>oEl_+-eUa{)U0#769+l&fc?Z|oLLZuAa{uMj zFt7w#ADa==gUNuBavnKsaxR1l5d=;)h)|UBl$W;#EVrI+3rOh4dMc8r2xVgd3ZZJO zsO}~$0? zPD%H?H54HZWnN7{EU@YsoCaPAF9hO_jsh>PkGgW*fttwq&~TF<>B#?+8Z;b~d9u5> zve@#pr7~7UpwY}sNfe%sKQbQk^psvdHrezkQs#?`0&n#ZYBuT^#wSWwWt(xZ$?cJ> zn8}-cOAa0$W#}!ad}92wF@SSC1RJn}b|u~O%Q z5WJ5>EtD0dp65ATi=P-t%V|z-~W&Yk6#gp~>r7 z8b8I*3q=@jwvKzB0qfr6^fN+m!Om3Jl#lzVg4ktjRn)O>_~0Y_SX~bEQU~($t{v$eG-)Cv+9AR(s#J#A*jSWHU4F(Kc=Uw^3vitqrkZ_@ zyi5_7dvhntDYEYhUWjK18Z-~sW?#p=p1!6IK1->xQRVXKDo;6*)MrW67}4-7i<5nc ze$4>sYU!2mdMo?T_H**i+-Idh88DH@4_2?T8|pQ;mSMoN&~`1=Pdc~nkw|#Lw8k)*J7ekHhOv@zZ@I+V0E2)QiF<0 zbd;Ky9uh%e&tCvu)cKVi#YQs4QYYp??_{MCGfrj=B3s)dwo2lU;`46{Z+65U7JRHxW-RYj^Ow{R;b+JJ=%<Gv;}RK|V8Q7cFa{_Esg{3-!2w@RB38HYPfH z=DI#HkF+aX&ye)rcAtWmq3kewfF%g!m1LpR3;>eKuc9LvT1)I`R}LE^7jwyINv_t@&0l=wTF{S ze?#vplwPo$t6lfQ=C!jFHJHSp$~%iHyeCbDrAh7c=ucc!)Lat+oADH$>vgDX&+;6# z86dzLV(HeqYq*A=s;_4VI`3gz)b~hL=!4Tt#^ChjpLO2^{oCFI1iQu?UBg}13D$&{ zARyW+)zimPEAkcu72n=1An@b{-3AM*|JUopHmJu6C@cES=Cv;4&{ch}&_Gt`Uztd- z&Fdp66n(JPA1>U;9FgjzMWUm%4)bH@1F%LuBeK0&sfXQ;?ckJ!%L?g$Dph z-!fn+UBW%o31+2qg!8CH13KK$RF@8H3}Fuy&YsPsyGBE$%!IRFuK)`rs9H@hbx$1UJV2WRKL;Ymx#urrii! z!PaCIpt_7;s(HOAhT=MA%iDrohLM1B7Cx1d>lp<2LF3?U!Q;g1DkbbrWcV`TeYHQz zOdOUi@?Mkw)x(kx7KI#kQZ^Tk2`Qm54HcT0_hadX#&*-B2xy)3R8XVTjK0Y(YL$_y zKbs<7BsB4Y^@P6y0Xds%D^UHGZnF6;(YiNj;V|s!%0flVb;HKauk?*+)vG7M6wRsm zPvm?wyP^`oEDsjKq&v}<0J(>_&6`eMWMu7bW^egQpgJe5Fr;+bF;DF-rI;^R$_|%a zW_~Ig4;nJ!KzbstY|2qCV3f_mSi4l(v+v7?=u1jJf*gAJ@SF6VwXXr|cY>jCbYUNV zT3#4W!u30BaNm5g>e6it<2zoJ2&M8RquKr$w~6(k;%n0vij515B&Hg!5%)5G(T%-f zSXfcZ6TiQ2xLu&AXqdi;GKkN4dgwinQ;!2?l?_ODN=VyQ!T!6)e-S3hf?u64R`19Y zl~5%M-lgXE)q@F=SO{6Zx_LRq=D0j4>}=kA-G^TdkRn;|=L?EiPOH17IjrOy%m$_R zpOUDM@=x-7O{8r48`y-6?xXsgw!Y2m`I6K?t+Q>RKiv`h`3F%!MJ=hal-u;d_b!07 z_6X0=9!RwuhVu8Aa!GxFvbKd(Q92>WTj%zyuZacD;BXYHf^nguKtheG~;c!~V6 zPPpyBOZ>Py91Ot)dLPqjeH|;hdPDw`$^7F3V~JFZudp9+;!nCM2L1UC^xqRPIE1Eu z&sB+XOZJ!3iS)p5V= zhjd4UjB1(#_WVPHjY`Fpb?%G);|>1j1jbj>R5$E~k^c61FQIbFW6VS9iO-H7A9W?C znXhyhZcdPNDxMK&O!`$EI7C4sbP>KSl-gZ|KZYAjVZ(`JtG7I%cDQ`Va-OLl=epm` z@r!*tWBZ$+d-7Y$$IB0`5<|Gvflx?V*HhzUq(uVBRvR^Bw@-Fg|y zi4D1)>bGv4odRjb^&0B$fkXKkLrDVHN(ExZuO$BI$SP zr;j$+gxtM8Df}Tj^&68zqV8&~p0T~ecXT)(NRIKTXNxa2T3x`EnxiuDcjDl8$A3DC z#;M@Z6s*WtKmGMAWK>L}ILbIAtYpRp{dJRbu({ zgfE|zDE;ob%e2$=Sf_{WkD@{7j^sL~_Mj}@`0coNuNzoxH$j$TjC4gxO#?6J;#<)I zGuyl0SEmc?Fo@w|b7c}WJRex9S6cO)OqBDjl&~H6W2E}g-e*7;F9w}==eo15KB5`U zxx;5aQT?FzV#iC6*cE*%!RtStzS+I?z zbJ?-Dd-!n&ms2=RAq(YpaSmvgx$O%NQ5}i0ac6yt4s5DS<)+CSTXMc2UK`JG%a+FD z(ALW6+2sp2?Q>_k8qV=9tc$Ix78q6JlkB|J?V4KYjX*QXz3()O0vAn&lwdtIpi-j; zc5P-+tb%%@fE`-^GDN*PKW~NS2mfAQ{^1Sdtx{}#h?>$cwOT=v&3sX@U#jR(b6oL> zUfN(bGNO+ilXBmtky+Wx-RydW@z9AH-`sj9Ht~O zOH0J8@fRw2%Wq;Z%P;v7jAmT1qZ>S$Wjoh}H`QF+q7ppt@(~WfwEZdDtt@y2w@l21 z`UtsCAywDiHMnyykL~p#)>P;v><2f!u|+tigC={>HeqS$_Shviz47G3*4VN(@xHSg zvpJ7EHs;D4;%Ns=OJKt2fW9NPJv&WP>{usy!JU}{%UK$1y*iw8dKN35mxL)u;A)We ze2m(xDzai4>N(pRewEo}Y$0&rPMcBn(zZFfN4bxB-gr`O*X~k+M3G$|*W}Qiyx{0# z3$`6qs&wv7FTg4e)5|sC@bzh*xG25XhL~A;l#ys0)e3bePMiv1#V>U_&y7}MI>$_V z@(hHlOeGO|lf8i}48FThtSar->?8vk1!NY13b>2dzifw{!d)?IDVUo&C>iSY!qIyk z8b9C~89+%P9J=(1s`BLHoetwr38mHW{f8xLOWN$CdfpR`sl8kmn;c9%B)?^|yBF{A zaK83sq8GqEhA>W)t5WXx3^1YRmle8)3#*J7obI)G7R)@X@7ot2Aqbu`tfd(R#gyR~ z2+S^8P`?uNm^O+QfbNRgdVzA-5-6=nQ0kJ}-Q&=neYG_z-*0W`Sgm$O3;Jt$PS$u! zdVjmG!-xVUWbi(U8P|A~^d!rS=oOhds>KMEfJ&;do?eW4yl9K{UZoRmd~qmux-1H` z82#9$f5fdTMp!-AnLf|3EbAbNO}$8>t~55fCEpft40O@J{n3S_z7WUy^PqN)X#BmXKB*JV_P}263 z9eTCRd$Ik(U3R#CLcQV@^N=%P=zlK zK8MH_MH<*h2*gxW=zCV0Sr-&PG)+FmIkaave9O0s(svlG*m!MZX3-atwhN!yuI#Pl zmUUD))?X^Y>CnD3L9FMR4Jl!#i?ikHbFzr;imgyKjk{Z5uU~w$P-d)VR?dCf6>pFo zlDbcXRq<0=9^yyYY{L1|jAkQ=?s#608}2c6s`7!6vpkGQR&@6Y*o^kfhEv!YR|?6c%8 z=}Yq=6Ub-2&R=XD^Q-cRSF7ZwiUaw4`zh;IJh1u9C`@TDF+$WWm{8QWe3yuHgx&EF ztMvow;Wu;}&@B74*&aurf6>cOrN;3>jNt7^6iMU_2(R7LhizH_uCnHKH{9v%eM!$84Z+- zgrh=6fI)Y? zX%M1IV!JA#ih46a!x)D!+C{(VkT~o}(HpnGar~x@jihFbvm7FBcH1S` z)bqpMbV>ML>b)e*D1*C1?k|L@dC_T&MX9l4&Stne`GN|Jr=dT(D&I;9**$&*a@b2D z_LI!)`_0RP&*ir|DD4E%4dIZSeC zVI$~(lr`?zkd8KwPJ-UnX6?yQ?gE5y3~h?K2gegf?o*D|s0PE@2oCgGOi@S2q`Dq^ z7@w9_u$pJC_4I_CE71j?Nd7i(VIianB2$@7I}zy-znF1OXtm7Czg?CyP;t{=LvuN4 zCg1=fGh+EM&t#tb?PKR%*v_Gs2eatAtNIWNg*TLfBU{hM_r@{B87W*VJL3D^6Zbev zFUc58=gLP_Klits1*Hzn)=(y3d7bDrRa7oEf+ zHx7=BByS}{%Bt$^Pn2{8n)F}J5_iEqqcKkE}0yepd)}WiS z3ExXmc>|*=C)~wby&0{T9jwEewW7>i&((mpiiM(X(GhubG?fTQ=KwUwbGCbF=Y-&m z0y-S1A%EwEw@VIZgxPYi6vL{VlFbNPYjZn_m06jejGX8P<$R-)1vx35B@|eDihMQ} zH`{n1YvZhzcmjeqcdcn~9=6PD-Svw9Nl=NbMeMB-grDs;5pqBVxl9$FGU3dFU_gQzADTQs_S! zZ|5ema$f@X{e!DsRl;TB5az2!)bo=4dXZA)E3WL@6GH{f^Z?w_Ji9F6Dq6_G~%1XLKO6y@$|sp+CbW5?&K~aWXaaV&W`H{orft z(h!ZzI?! zA<(yt?jQ8|Pz5a6$;n_pQmzg%pL*>u>6&#U$T9^<8`BBR~ zmCrJeNPJf$*u!^X@Fv4f^fI2=sNoN|*Pd}`vqDj?BKPy8#KLpPN=1X_gt~ea{Q|qQh|?2omSlbK z;3P2^`vBs#D8n3jWT6`kE1;1KuxLQ^C|fvuZk@}(w3H=kbd9s(%7+v3ZfG<}6`=sO zoG9oLVXSHD4x=#p*7_x>K2agji5&@vquML!_8Vy;Nabm_6@Q$+kNUl8KDqF(v%#AO47GFfjI&15RZB$O5~5RZ8LckV3-#C};I55HfyYv;-EeD7ocbjMdcm_A9}U-6D(_f!;Iit&{j%p(={nN$H4lF?m;gJ84F$F8Wy-s#=5 zLspd}f+XDQLM_$D?ueIpL`79NNeGTo$Fgos9u+o;?Z=LOQY&O01Z?hQ)^9n!^q+)u1=MHd z^?20dK#SkU*Rz>Zc)o}RWlxUF)vQOU^X%Y!YDGWP!ExuOJnvd0&SxE9np-5B3d*=# zBWx?H@b~t1UK@`MK=B1{Y0pgUZqbR2Jo&6PBD{V=D$sJH(gokYTBKR|5+djoik>4- zIw28IetYFC;WBTr{hI3@@87wZY8aIc`=HV)oJxKB(#D2YLiHr?_034OQaWS{YuK=q zIuyV@hqTw5jdysSEqc1NHU3B7%7mn^RQYLrj|ah@D)d@#&O2w=T_Bxr3q$JxeaNJ2 z2~>>guP=c*=q#F5Z?=#;_G#w5mS^(e8-TDf{+aj^Dp^R#&bwQPiUi#)iu>ZAY>uFV z++K0>c3UlBv?Rk8nXR5U`>>hj3&PhDRD|ISupYiQpn2SmIn9^On;gwFo7z9$zwkSe zTbi)xzIs(<`ZLA&*1vXb1r*(UMkYTGf?dVnwcjm^1Ny-aW=$MhlH*j)k;u za%pXe1MVvmEK9_@i7RogLkvgg%3;19 z>X$XHI2Rh0o#VqD4U}@fpHg3c8)MkTI`YW8y4Z|1G@QYN-IKkydng;?N+ z@Nc~aPW$U)8To?%yh#3W_0I1OR7@+Tq+VpJyf}CrF#bqqI1?<+2vW`}$NF+W`nPW2YMR2j^81i*_Ol4ZTYn23=GT}{@?&SUg zspO=b4~S1C|M~Cy8!~-P8mMWJ$B5IvtxVZc!Mk8GEW*DXeX`&)Y&T?bjGH&^dHj4K z{SQp25tIf|IQ-kQrk$V#ka4ToF{tuyrwaArH_o52k^Y)w&q@@L_e`WnmAm7?AOtYk%F=P_+8(Z%6IRvIw4N)+i07sD~!bRchCv(rM7<%eb$P(}Q89*Rf+M3{f4hD6s!nge{K4 zs3d#kWItgRoNt$~Q}vZqcy8SAI)5ON4KZ@$;yLBWaB94{@RYdq>!URRHWo6QTTh3J zoCRf2YCYSj*r2>oK5%>A*R!`940 z^wP4wDHs$-)7}@)!PfYAqaCNz#mq3cuNRzm&8K=z%&pyX%>#u@?Qx6Hay8Ea`&tvL zMG2YVl7s!yhE3-?El!i@blA6jabNfZlf6+vl3cm7vdmjIbxsNUM;rcl2F_39X?nmu3YR_U)vWrusb`j9N6@H|pm%E=bxo3fLRw zz2$lQD~$Wk_wg>CwcSR`^BHZ;@Xnm`<%Dc>{-7@jC*4fEr=V`^)W?nK4nsE(#HM;H z98s8ZR(q+@_1vyy8ulEyAxd9gfD2zgaDGLfP}OOAx#sBxXGQzAtW;x|*;oezuYgnS zHnin~*jS5HlYOl)>n+VNA=5oH=4nE6~XbT-BDv(WE@FXP0_P zcEozobgve_U#+2%n!A&g7`0g9(Bor9ybwP8eS~ILwF?J%xVhP9J;rxkyaL%ZGA7KU zQRY*~IcEvUNz~^O?+qKv%lF6T&;PMpSY(xO`LQw4_RHg{d}yP>{%j1WgF*yM{}agm zIo1_JX~5jEXYCqaeOdTi2A!{O`F3ACAUAoY)OE)xmdnP!0hb|Z6ggEGQ|o6!n^=FZ zAbDQYv&wDogs9^qj0XpD_A^3+-VA`8y2T=~LZudF2ThSq`aNMOxM5%!WYYtS?Gt#6 zzDw}tvLD@Pt15mtK6flB%$q);y!*(7LC=;eWzqO0irCX8MRr?XJyN0$PF$Jlwj9$i zD8Ax5Az-=ufL*IDU0?3t3Y5!A7&aZ7X*itvWiU3svL4O5ildUKbj9AcW|QjbEbN=V zoVWCG(_8Q9@!!Rrv^TW79Om5(Ap#BCUq7k4soNIdOARbrT7?YY<_=>=OD<2t-5SlJ zXCR4C%jB*X!YiMVa9IC(*x{4oPLPMh1`mlP7f;OzqN+$GIv~#4lRnPY3`uwywW8N$ zWTy9F(uD(kz>RF*2dFiA59TU(C1)_O=@Pkx?*y5^1gn#o8Hj<$3M zEuMiwuk4F2egO;oYamALC`C^WC4UAy&8j}@FYjv*o`mN}2#nJ}_0V;`0r8b(bjpH|I^|39f-Q@s~t*lMuu!h3!uH$r!#!fHiVN3cn zik}kd%J$kIr819&i{t02ssv%J4jD44M#0tKfP^{bX}J=E^prJ{l+hW&@U_G!W#9Q zTs0(k`z2NVy4*{1hc`nB=B6}?-I+VKm{Mf{^PLW5^FX2xmfY$gm4jET$7JAw3FLVX zX4+mGV>LuHiyc=yiL-E}{ybP(rTxQO9<^Q1ZdZ?lYKuCKS`4I-2|t4?K1?1dX|mr|E$67#|nFjXgC zb_0xcA$GbiO{cFl+Hn~JbJ!NfIba;Iq;-0;LZ;fE-WYD7pXjA-oj5be>7{3!r)SPd ziuFD9HLum(R@Ce2$_gH&F6-927M@+iyhWr&p-hii?wo_}7GpOZ9iTXOHrR)btceMX z9&V!NNYZd?eKzEEs$3SbpjJkvA3A~yByvhkc?aU1XmL}q)|C}BmR;|lR6WG;Wc+v^ zYU!e(_7y@1b)HGj-SQlgasM*1XRB?l)x;P(5f1n97Vz4-f@pnEwLiHhj3`njb; zb+w7q4c>mL&VBL)3K-K2XQ^`@WX}Q4at^5*_650bK4ObzT4)Y2tM4l|plp%3k8baZ zCLSLhARkvf1(?2dCrQ$k-pUM6R09$}^cdigTH=Ri>Tu1V7VOqKSeXS(>jO>#wtYMC z^E-4;FELaqKyl|a=*A3pQ%G6uivX69$1ZOhV;OL28w34fEJ*c#P}P@yP%dkhop3#cXOf&R-=O3@xL=tSre3UW z1mJM?F>nF|gmt~fg;t2IjFh&0W4R$4HV!I2%RsPH39R=o_w`xb3w5t50EUDplA1Ii z2KW|kVrNNg-uN8yK|t%!KJ#MiHtr98-%5nl^*Yfp)2$ZEF_bT!FfVFSvvLDysz2B(QZO8*l!mPb0JX(3 zP$LwR`Zvv#F7m3e$GSH}M@OR)Z7kJT_3o$<0LfBt<_pQE@c&`#tHYvR^S((X1f)ws zK|pB)k)c6Qx|<;cq)R#lB&4OgrAr!y5CK6#LWb^;?rz_Ee9zfE&p!L?+5fmM$6ww3 z{Zz3g=ONCMYTX`3q4ke3vPlC#a`c^NZ2aPQ)4`AT5i`l*Yxpl9AgBNyg#52dj&3~? zXVpc5qj&nrvo|+zk)U97V48X0*9I&g(#{>zl4JL$jMM0AEk=6epDp;y36*X8P=xmZ z^tXH%q^GCxKbMS2- z4Xy)bW!*)pkEol=zZC1=*VQ&@pN$6ATIg_}UIoHGZ6X8$ zm)8hZ^wA5^%uE!+ncd<1NFK}4J|MBw@Bm%X8i-F%hF;KU1z@eG4NGiB)qlU3n!8cY zOgG^YY&L_oAAb7X>z)Bs<-`q{P~R%ZkLO1B3hAw7b=}iELjg{AdAn}rAQa}Ne!d_l z;@~neo>hnBrtgWs?dz5rfl2RFU=23f;9v^$q+^~-O8~yHI%!Y~EY@O*8RlD#emGHg zPi0)dZkKy|`8h zr{LKFIW332Gi@c5!+>@d{ZNr!bVi&X{^=*LQ7w5QQ8Em}zN0{OIG>*rKM#g=BZzML4lGjNjwf;&l>l2SXW#>Q zwx1)=Bc^hf&-lTkad)6bNMYjwrlQ|Zz_fNj8=(Dq^ALtnFlq~FfYw?iIc-ml>;S&L z<&Wh7mbd7ZJi^d1nReXpOV(8(RkdpujgQ)7>O zGBk}zTXfKkLCD)<8jRr+VJ)f};lUPlK^pjM60@KxL>aftaTW0r~t@MZBJ6V=1IF&UngVgxSWUIFNB@ zJA^_ikfqtqvsCtLN48|*7dMcO-C_gjRM9v@^ylmS>h#mCiDBE`8n2hN_a+%O#FZRgQr5Lkd|YnYrR`D zkSO3*`77NiHS^jxVn3!dXXxS3xq)s z1DrggVvaMcpd($qJcj*MHLjvJ0e}b;eDG*ubBQ|?pe;R zdrOoQy^2$&JfnGlaJo*&RJm4!lDuK#W{I;Fm?6)XU|sULKKsj?Y+A)+G{^ZY>=}r` zkW>vyCDz>yu?XC3BDl()Dk-v~?+el{d4c`sAT{sC6a>K?lXg0_=F<9y$lq~7mi3jI zIea;x2(kNN2~`XsfqIULh709jcW$(t10&pH~AMN2@uZtiSZ!^Im z(-CZwF1=(N=cssT(0xZlHh?4cX%Y-8eI4aRrPq7!+ubMV3x4lca;{T0*eTKN-=6;@ zeOvKdlo2X0D&Y=I%>xxH?Ne1ErXRZkJI|E9bOLFAKTXgEndB5D`n#oCtqYvZajAzB zwohi3)U8q5kBH##rwW^Kqw5HLsSzF3=YZoIAl*M`9xw2o)C+Py>v?`7Q(G!u+l(`e zH$`l6AsJb&Hc(GcOA7H;Z5QPUgjdu@{EWR{x+F`bvUw)gHDM@Z-<#$2zVg&b2MKBn zKfJ5iKTyk3mVRf73q>yF>G52{3I1b<^Wal$oCnoAOs#c5B6uNE?#&J}NVlSr6G)g! zolB(Heth>Hg?M3>mafu_s@OYpL5(G78vn+S_wemUN{==BGP=>VowuqvM1XHG0EP;UIox*4^ z03kFg1O8aJo1V|LYAsqxB7)lN6C!wX#=b>w(~V>*bQwb;RiQ-OqbI82hsxV2jy50n zW8v9LENSBAZ(jm7_K!y&n)vkAb@k%0kkLXkMSOMBm^SDYz5vVn5sOP;s(qS7w9&6Z zTH>?_pvOv>L2+ifUkwbYq}8x~@S}}#>_jFi1=aGJ*atS2^|mr0QdUUfkO#z0dFU+J zrXDW`*Y4y^QW@-SZm2e-TWuv;TW_<$FB@h@t>Gw=*II_s`PjY8Py#AeY^(JMTb*^@ zBzp@R9V50PJJj!uMtmAu5wk%l_SQw{#iqZY_E9?99YF7!!SesP@TV7m>GGp*tnI+{ zW9XN8GHkHGr;=^27+Zn3rW1EAL9dt^E1B6rvhma4Dw~f`Nd^O{ko-x#&k%fW*Ho*Z-Bo?F z=8~Ijcu~p_Uq6>eQI+d3O_7!!F%(-Z;~gii4#rMj8!VmzSgStpZj$3sC!gRPIy8fB8jZKED|K;?oTz`lm}7puYz*td+1Anwz-UaHj{U_I;4F?x~qYYmcnl92Tg5}~>n z7UJHkh-wvHEExugoJ!S+@pn6TkA(77h4vKRYQKc8MhwZ9fF-kpqGnJ%ugotPHcr9z z>>*qH&TdW#GME~FR`^BOOR)N?aZ^#H7NCEsY$x)3PZFO0?ooJ-FgmTHsBV%Z>~3DD z4rEK0$CJ_ZGWp}^H@vwT`2K!`ezbvy! ztXo0Zp(T$Ey^~3S^qPw~b9*YGY(EG_smdVjm=@MA<%<=bzn#P7Q3J)5@I|-wP^-qZ zgyKFFh%B3OXCHYgGP6+Zzw>4U^_^dM@(bATDdD)^FQB>xQM+4QG##q*YL}#GVU`n) zi|J!;)}HG;)O{Jj76Ox0RXU08#6JBL8`r3x^TTmR#z%t(~OgWJr%bHDSv_9+I~@DIEg1d!Gz|?T(DIrlai@p)-NBOIcn)REf|+LECOabE z-}kH1ZZ@X?9Z}tvjcqeiD#;koTQB+1Q=K^F7;_-VBVQkxuoiBzpoj7Zs3{rLF+Fi4 z7Et7noUz&P4zXy?XJLgtiz(#Ft}el)D zW;l_o`e^}--S!AYaW& zhZRc3;yp{t_G*6pXpx!XLuVHsz@KEx*T&^(RkBmVv4*SJzrviIjMzx8O>S|2`Ju*L zI1z~QGrXr;*zC34OTF@2bhg>sT|O=pZWNq~)9MeX{g`nv6M%B7582X>2z5jf9qI#? zrYAc-qR7)6MG+d1?QWTor$_H7kU2|Tt>3fBOD!~D)J>wu&H>A)+zBeQ(bj=U+F<># zfClr+tF9E!j$LHSYRx=&4vlwAxZx&p_2pOi5!CUKRCNUS@SX%WtskKZ}g znYeuqIt^79nb_Ir)LiwrkSjzsXX@!tC&{;Q$SJ!TVRfi<;@;P_aqu>R><;r~Q)F6D z$!rBP!DVLNGk+@HhueuRt*1&O>m0w2vVh9*L@r+Ot7ocRE)GLQH0oXs&H;C&`2?*l z;_i_6M-%($C;J!uBQ>=YE=&#@&E{tO_^_`EPHf3m>33d$vKf|&$mK(*#_)Lx0iWa5 zN4>MBt1#_x$)czaCr&W=e6~a|>RbX07dUNb3Qv#7sVvu1xsvxjSj-XSb%Wk+MTPRS zHsY)CBYY798O$|WDP+}}YW1N2!x?cIQBh)a+L7&PP-WcQb;x5YhHuBS;jW1_u+ z(JPT!^Q5DcZA2)O`X6aIt$*|}F_YRBp1&FPByLh=0JIbfzLL;876GN$O9)>SVa*@t zHt4R$1xTf^&%@}3RO9P4L^VzoWLRsWT`jDxVj{MPrKN_%QH zJ(Fs^LQq8&7na9LZ_#@XA9JbjYXA13U?PX1!jbd+_%;qCNSO#fV>@-{sqYr0c%ynW zSm~Qo1KvpHGS=FIG?XxRcz0ecGVEnJ!OZJo(g5QrBy`^D?H__}#4Lj+6692qv-!#1 z_R7OI5~soNfT6+Mrdr<(R#DOhr+wa}m=i&5XD5Xc>xR6z5;Sp>uisvfn$+m47zo(t zCs;x3+Yc?NW?TxGyHq87p90<%Jlq0o?r9#a2%|@*JNcrZZ^KLT*=je%!q}Y6GRkOS zTsb3VQpgO@%q-9V3w8O}c37hunu?nr-1R^&hF9czzq?=+1V&9liSDy*tLK4C!BZuU zXifqt{q~mCjO0`q0oLhkC`B!hydTx@mvn>=t(XZS8`?OX9I1nQON}~IB{s`Lb}&mq z<6!|(z@yL0PIgL=3NP<5%%I315@oUH{m>~1Okdu4I^Dv(Ny8HjKKK$xy5{X*i(u^F z!(8tt7uRNjaB3h1wz7v~tBwNM!sxb3At(0-1P;SK<6Qz@za7*slXGh=QyUZD z2m&iIN;f`a>i=wnZm;Hem3gQ~yvwNzG70mUFnDS zxUgShX{Cz;z!rdO)3$8@f9Kzy>nKEG1GbW{>Yd~Iop>I8T+>x(CftCe^$$2U5xPvr z)$7_{z6{T~^#r1>{3Dn`F(`WDnR58YM;Nr9jxl2GS@EkbxB85d0LuhEIy?L%vci=T z3={FpU!?E02Cq-PjeQg*^g&U3=14nk+AM};C8xIRRVswah9 z%>+;0#V!=vC>^;q=Z_`7K6<5f@5hf^nd|Ocd3cUQhMJ5GbUCzNW~sP^B7FnJ!%-XE zEuE&zaU}=a0v2TQt!2=(I<+TR{gd_=AG6?#7eS03)Oin2pV4@|==l)bFDxrFS2`$XBjnZJRSZmHHDmLt~e<$4Ykk2EnCsqOsyBfqzv& zzkz2;OnI5G1w{v2YbBCIu45A!&!{j&*;M4*dWIESBpMS!!Dcp9YBgT(P;0ztRy}%t z9Ud@Xk&$`W=-AI!OJKdr)EK^&S2OJQ7MPMIXqgORiA3B76cvkvpHLyU>LjeCxT<2MRi_uzxRu!pEZ$iXO&gaAtN;}wpioKygi{vP~@#dtH$9s+toAI zfVe?KCSweqgZfPb3Ic&F95&S&?wmONDXsP*Jbxd%Y7M3OW--LiGi$BoMK{b&Mu2l{ zvk3*3T&6wuHLiYMluUTSbTLn5I`Au0bNK5oJKOM%3(0H3xloTZ3vCMF3DNx7CBH@@ zCX84%`O6qGXP(xd&8oHzmH0-9Xp$_UBske_VEiPP9qIlbN z&L{cOv2V9Uc>l0T60AER+O(g?Mt)U2;Dst)3^J9S0=d==GM5Jht7yc~#0ZiZaSiDHpLAH-I;6{Q)6`Ai zka30ikqZ8O&X^nw``T)e?-W|29;PtaxgT?=hlmvuAfdi~*i4&++^>a*bpszEx(Csj zOdPRRWqjbIG%aT$r7AoW}~*Vjx| zPMwwD|M=Y$U!we`<337|@sO2#4m9+S5j;Apq4OsW|MBDa_F;H(7#OrLJU{I{Uwmc! zAAgK^`b;wfv$2%d;qTdk2M<9*>g$Gu{l`Z~bP}Rcz(CDDer@wNT?3rNTYwgQB{VeZ z?>~Ar!akyUwh!lTKl-->aC=XCPOiW29o2~5F`Clw_q5J)g!2MBn36n*)dR#Js+B7tt<-0@tKN%rfZV|m0|M~NDqtAMsLVvB`zql9W z`O&jQT#+}!AS^;e3SUvA&ZLTAwA%iiK(PQYs-QvIX?7S8=rTb=3#n=avq`DwyIVkW zeZHtGxA8v>SFDIpdK1{|9;+3dnS=`s?6#3ESDmMkF8#y41D%+R$0C*rBp!Kl(Y=VM zW`n|r9HVazf%5^rB!i#GKA~cu4FnlFqPvtEGt|f@GtuZ`i*USh@Hzt3!#V^oKE|Q< zk&(mw$P8rn9KMm%1Bl!;NY|PtJ)8xhwPrz7&WUe#Gk#r}_V3j*Qk(*_ zo^QwT+5}X9NHChOK;r@IQylVF9fXGcyqC7z0#L-^JM)7;Cj1Bqo=Y7^$=5cYFWHL`w3GiN{WLgQ|NcTT3{5J zPUSc#>I}?4llw;;ZQMY_-vq!sErCC=_hg451hV_xxO=nynabghnn=KZd2R7V_VyoF zl724&M9jc>Oa4Y-;ECFwrX3ZB~F0r?RnQkm1&CKCt%l^iI;2vn|u4K`FM9uc-3K{JH@ue9N|eR zd_IQh$pkK+mF9WS{nVfpi66ModVYGjix6i6*X!4S@!;N{;a4@p1ti_S5mA6ya)P=6 z0X6K?peu0g%PqwB-AK2Xa1tx^`Gm4ageG0(Lt%jMM9u4}C$J{M_xcQ+7j9mqu+@73 zIQZs4Z0*DIM~DnE{qn=ZB*#uU^;Jiq-hW7xT1VMFHFO1~I9T!8&5KOgcTruRFYPU~ zm%jPg0}yIqz*X7a0=Zyvm&F*v-NjjNCJDpHD>x!K5zw6=fL0Y=41)AY`PHl!r)(%R)nLYmz?5ffY{zD$Z6( zRew(n8CFlxSBE4n69l|{vkOYW+$PzNT>%{mXIS274C+#YNyxB#QKARNa@WbMhi%3X zrPpCN@9ZA=mkZBa>A0;lm)VMB9?Of8>5>wbVrXM+hE01Qj)O^WpgA|-Bp^r-Bwd5Qb7u$qtYOsT< zAA$zqb?bB9FEFONY=vV@%5TxaXir*E@e|_FCXs+;K{lc^T~q(&SQVJUub~96{sSXirbiIQGZoyF5V=EP?*`!^cB*s)20) zjOQ&?etCEM@{l{t#S0+&&PPKprhq4>DIgeeP{~+$qC?aiBU}M0evzK)s8@VAWv0|V z!$HNjTgimXIu6TuLxPt&nn(GL0gGsZlD^pV5qhH9kJ6z2 zxtcJDf*Si%8qYo7-6ZMN4h^VNT&ZTo1Do1Ae=MGWt&{nj;o79qPGLsYf(u9h-V?8l z6ekR%!-Hex$Hj&)`#?80?;%jrsE3`E$55Mi1A@oXu+AJp#k*=y66Q|?^7QB`pS`{2 zYAm`UsIG%^<%x)ihNPUBAi=8fWtHxPp|IB@a{zjDsG(Lzst1lDEi$js6=2TW8#u;$Bj8-9#w%t&vx`2?=he+FJ1 zDR7OXwhLV*m`hn;A(Lha;U36(l}6@yGl z4$E`~=z3mF$;O55W>yHU02p`XZ++#pb}U|INiy^#k4XrWs%)e~MVEk|-_*cvb|~Ay zRBlcLq(}dm%+qw(l8OK=pD*76ti=v;4#!ZkT3>~kp|zlc=HL9sY0aWA}G zaqzP@B|-^!^wq5~No6p9E4lRprk#Ww z>~IoHAm2|7ayz5`^oL{eLV@Lt7O63RHh{gJ0|bXK7WyO@xuNj`P`6EJMHImlyos8y zO#qKzwAx&SW^jZft&b_O_ztVII|@jStC#`4`YqTdKqmKtOx}kFJNE~WrGa1QCkskn-rTGCaPQUrzPnqLyvMvb7d zcv}5_HQwM;g9NDa{uHuxek^8me@m9>Kzd#u4pbE`TDgPw;-416?@5`y{chlu%)n*~ z$CAKCgjb={&h&&MHuVhk3#}?hmkO!@6=9KRW(2mA2NhLDV76q#Z#a56e}ClLF4#`& zEd|(T4=V{qQ)yUSt=U_gHBs|G(X)vC)#58CMDsnEsi}H5o~a)zid*nJP~Cer<8C%F zQXC(VP;&*@X$DLPFT0DdC;fY4BH2zq|6vXKApt1~=7zZs>?e(%i!gd1%qnej8$oNI zo0n$pEire8G8g>@I5xIq5JG?-o#_y$tM46lN zT_#o(3p1rU8z?2;8{_^5k9vh+H_J`WH^SYDIU8&2&n!9kb}CycsDK zPH*E4V+{{}ay-ESmOe&eq$7^V`@lxO@olLW&Yvena8j%j5sk!iQ26ed^MylpqNYb| z@S(A11LHJan-> zuHiJE>TZ?};nnbh`)J`XjyNQN(!w7bY%qYNjTQZ3rSmEsd04reazy}u`v6~2M$c)k z__C(Rs&TvasRf*A?q``EbEt->I65O}1p%;(<*|%_@W*Z!(GtSv=mv3Vyq;%v9t#1a zPchdX59aKIs1Q3j5ObLjbE}OO!|X7+nAgQ@~$2r_U) z*v6c0sjg(~uz$=ZbdxIv^woyE;sGYtVnQ=wC!JhF7|^j_V=+#3$3thV3OzK^1~(PH z&m6p9>GDQ>>mXtnd%vT0hpE%1~@`elX$M zvo`1~9nPu>XHBQb`B-J^3NmkHPbbvNkvAswwP-h5()|8$^!ijN!VrPiUXhkZ-Jb=g z1-So)phVq8BEL(SFxzzXtrT3PJ$1pJf~fSKYV+X|&?i$O$jGBXW8{FwprhGe?Sze$ z!{kTp^Vk5?osqrtRdB2!zn@>Nx}=AG7UZg5pPN=L&3Cvn0e;i)YK<6>rWC;e-X z7}51T-tdpJi=s3dQQvoC5e*jOjreGq+o8VTd@|@|)F!AolC1INmeW$PM-S7u3$Pac zXqBX+9A8ziQ5ZkCfh+yxE;{#eVT_T}?c7T=D!{hoQyXe#CPo_S&Vh-j_s$zj>^@@p z=<#ObxjK~BQX4|9c316GYV_LD$?xJDUF3N5Lz%6QG4cnDO@)337QrQRK0BWakv^e3 zdAvcO1(o^?LI!Tce7Jm_Y#+}2dMc|?t`^jMGTD3EkCNABYi_ui`}jjy7Lc z?!yDpYhO&VjTCDPoWe7){nocAt7(*+TTn(|B6*UuR1n9#!xiCwoECkB(YfqB$pmXJ z9eP#<1f9G={zi}%pk7|NEdi#ZXU_bZYj-+`;F!5_-3R=pI@`hG!?wE|5tp)+OVT`d z~v9ocHs9Y&PIe ziSq<>A&)-+X#V=fOb__IVaG3B1kiWk&1@!zXw@-_92%mdvLxtwPRga1_a#E*)t&V7 zN=i!B=BqQ9G4A1f3dAdtF^1lGX*F|ib}YxQFWKd+31&Vjg|I=d4fF9U(l*V3a(0c- zPq^N9xW6<1bfiV}_W0rI!0F+LR}a%akH0*s>Ai#d*^(-BipVA4mL?! z^;{D_ij^w*lbpVbc73?;gmguM(_Ixlo#a_(Cqx~mSv=r;xsLiGwLjNMs6_9Ra@uL2 z+uom5s{rN?!ALf`IB||yMwrA@3I&E*uZH%c7r7ACm;KU@<=t>O>2^wtSSw$Ok>UyX zdR2+}($-~ho;C$2rPdkW&(z_#c9eg~Yl;gYk7UN0T_$b~zSJcnL8)ADa#an~)03F+ zy1)+bvH38M@5@T}VWY)#h~goo0%_GE3hctF$m<=hrxRz`U&gfwdiJV)cf`5!2!}8- zRF3iU;b1AG%|18c{-MXsJ)xBPJLjcALi&Ra>m86iia&P92keTPzSCxML>*pW+j5;* z?^8CbDA=YC!&$`Gkc@MS5P3wfHd#>`eowGJ@EXD}kgF`BvGi}%%YDGeoS5v73P&ap zCJ}vNO2$AS(l(+DQLC5eeP&gOM*x>&XtzU?uK)kV$JO+Qo{tLtjX|^G5lVNZMMw)n zAO0UtiuK?wN}ih>-ON4~S6Y)_Z5D2KCtX2Gg3#_iDiHr{_W$vhNH770=nvz#{>|&P z)gR=%S{zSh86n}SOBPM_r#YToMWMw&>wDCvCxKg$ z^_o`EmF6Fwdgw_B&+pb12TQzbM01MS<0AX>2=Z_0Jiot@bc9HyPiZ{uPV=;{E@{OK zS#>z&7&CojyNzEg;fEQ;&hPw9Y5c8}D=|-O$gG!gX7P6M;c~4D5C5(<70cp#msYOC zX?pLjF|_6NyYTd?>CvBP&?;dnf;US-DbKDu#_nUJa7Zz4GE7$JnG`Dv$zBgA1fKg3 z+?>lhb7;3?Zb+)R{@-Sqe=H@rVzg*7?QP3w4r66?*mKwIdRK<&BUZW-j~g=04HM#Q zRrS7$a(pW?(I@5L&nD}qn{V$bbL^&{7v(?KknZ^wM;M(VS^+m28Umv@&g*ImO_`GIlULv$&m<{}!$%1By=BUc#| zI{jlWL|0OMswCe$thk|&RMf!wbFcA#weMXfClo&9neeFL%fSAsO?Te$?B2rj&rj$Y z#Z72h!!frQ)IaPb{Fb6~5}z5gw!YGSQWWr?p4p1rSFTDM`nHoYbVny84YScck$Od= zW+%MZU+Lhw-gN}>E2$(TWzM=P8}6lXpzV{PQ?j_pzuhsc*zB=bT~M0?Lv3f9#3>&z5-U+d*5tleO8~ z>0`5aSx9nFwI6EzS%U%AH#<^Whdn*(AY!yR@iAn36Q!PTwY(A&TYX~Cym$LN3VXZe z3*q*bLf(WAq0afKiX87nI^Htu&P?jdG7pNPlKnTcE(4ew(*FOPD*unCcqmD$<}=!5 zC+@>dFerDNwC4Bgc&X5NmGMalY%|YoJe;ajZ98PB?%EPZ#pg6xkm6!~=L9dT@|v35 zXmxj}m0GaUt!zK08RJ-KtQTwRFB2WW=YcD-WvGAfs5#UKq@g2 zu|zDbGzmK0uT*XKHS*WarU_63mw|wZbwxF(6U;QP_%p1Io)#JNr3eba2OqTGe`cQC zeTkg|Pd+x5o>?z7`>;vO@AX1=vV7n)Yvw?%Z7}SV)iB|3JEJdf>O}MDlMEZlZ{Np1 zRJ`@KGNobitxH^+;g@3E$-08bj(OFNrBg-Vz%ODxGqLsi5I-z zUm&q&y?LLbG>;R2KAX^zMAr7;(4RG4pNWO*m#(2%XJf5YDA)ayA5=O6Olx@&!Q(O% zENStTOovg(Mk<94q)(z9fPJS1Gh;=z4KA+6zkcL@xg+A!`%X${R{F7TH<`Ac*=aTg zRLMz>&sd&!D#Z#m97a!UeXua4PxD6X5q%}z zgfvTnf+uKq9jB^o$hB+C^bAu1!^w!z=jAt@N;pa}=@s#P5uq?FpejK8(BU&0JqIP` z6|hxX0QG!aR_$sGu)R+X=h+|pOCvd;_y z@c7>Y|NHeynC0D-^u~iomf@eOrq?Voe{^VHz6_BEVe!p$LGsu&0@gw^Oe+Ml^&gZe zR7`KeF6ovBJbLdncC8HLqQVD8tC%wKcr9y>yuHi-HGypmd?p=&QuzYs|IrnW(u!WRHb z0U+>$CFKkV7%_o>?5hHSodM#weDnyJ=fpJ|5hK!Et@TtOfJ@jJ z@mG->*Yi>qnN9PV2|kGAV#F%pt^w#kQd`jg$oe66s!DPHij*KS#~6K4?tYF z*O#Z|ZwjkDfW{6Ru+L-=Mby_!tAI{`1G2ED?@X-+*}~TICbj_rCO!`OR{QG|YZ6dd z&uDG6l>4(Uab?v#z@@?c#md|5=@4MsCEq=x-alF2{r3AOm#gK<%OY}TqG?5ML1HIt zF^3%RI#4t+(F2gJ=MT`L0$4~%!cUG@7a$=tg6fID76HKRy4rk@QL__0rmX7%5P5|W zW%4J7{d^+dBwE+U%Jt1ADoyG^)a0<#BG=*`P-4;m1S*&5UJT{_!>9 z1i(;ab1{w>wcw&J=I^g}1QWJ{!JwsW0fOcIukU`#RU2pXQd!3Htw{wfq>h3P5I@D>z4g?jg5cLS+z8gEZ9WDs! zObejBvEE)OJ%uEDB!c1{#EyLg1hcu?wmn6BVi72v;_USTBDIhW)FU`vb-dpL*&Us` z?cfN{ON%xKa$fNzKo0;S_t3e*xSw4vd*A^GKTo+ii)4@}#eA8M3r31#%dd1QC_6&E zjT`ovN9|FhDZ^3DIkXW-8UfFDA@wq*3>d_sSRjnXm2GwoGaEAH-j25=IFOidW+laV_k_ABchZB-8$0D^fZdJ3 zS$URJ=wg)OW6#$_Q!1lO5;MvJ{_9{)e@ysnyLuRdTx5xFKWq+i()TVfV)h>*SY2qV?`^ue5ek=hH!?J9lpQeH_b(>Z>7;!f(KgAuOzo(52 z9^Zp>3Eq^qWqZ!Aac4R@O$4lhTPyxgjBh@6Ffj{qC2p6{buh%sxAP$qlXL9)UM>Kg zpCChfOrrb^fbs8rg^GzRh`uCaijZN{00Y`09VzQ2=s(}daXUo90FrM~w}dW&Q3s%V z_PY<#6m9Md4u%lA0--3KICT>OFqLR_3Qd@&2G2}{0!njGO_1xl*i84Xt3Ll_8_QB! zZ~Mj7r&z}@M7J84uk9e)>TD*C!j5>h`2K9(69i$)IA5|M`i9=k9Hnq=qC4x_K*KnBxX}-gkMH#{@kRRoAhu zu>(9=iTlx)O6lYrtaoQS{JTx>QlUD%XNjqG(J_*J9n~$-j_^Q<%YMg-H%-F z1zOPdik`$X1ELaj3g4?izvZ5}_|Njm zS5d+xtG3Ll>Tc%LU_U{&80<-AzMgn!7k3`M24Wk1qK5hi{R1e5^ZE#8E(^roMl2U?9hwfrN*oQ*DMX$Z=L?B*aaPWC7;vV*O&L!~&7Nyz|;o+Peeh4rEL?Bg!;t-Y)8} zwSsN=+RX>iCx{5=tpP$4tR=no=B{_&lc)Aw6Fi_twIe%{=|DR$SY8M)HCL9Ojyf0r z6~*V$F!qXO`0^ti`v*2CZdOjOM}KSPt6Rt1z$TpU`3aS1T$3yPl1V77&X7as$0i99 zE&99Hd0-9Pr&@^rFnTANKAJeal??SUpLfeXWmo7bLv@nf==b3lXsxWDt@pr2#a<~7X}Ny8h1 zvxkK4d==u<_Xq-@Jy=QP4X-A;`ycEEq&KfJrt!ahw~5%^TKFMRF;LfXO2^)|Br_C= z>DPE8J33JoC+QP`sBc#WYxU#Bs4?N(Px~Pmgm)5lASWiEMB+BwDYSA5&NpsN68+F@-^MXII}TM#yB{jZ3aA>5e-9`Xt0ap2qH9S#ilN5 zir0JnZGWyX8WHw#x!wPy}={7n_ z1sMktyT*BnU@FpIq@~}PJIA4u)*#42!B6CN<#C{qc4R>_)_XqtWh!}2@gLi{MKqFb ze|@3mLxP2DQZ6woWbtt*Gu5dd$VeB@WM0ExejiB{H9V#ML zaTEfkV5A%+H?u%f^ihzT_Dq8=4f_teo&Yb{{(r6C$q2#Z7}!ByOw9apLp}Ow*ST$Fqgr)$liZt;&-o-e&aB9OFqc#;%l>e|+|EA{m|U zqjHc1rzM);<)})YCU?!z?{={uI=x2odr`P+5sUxURhz!^lpQZ5x}iUMyCp#$DJ%6Q zp(`(AIbEc{kGp-jblkaU(hRN5bm>yv&2*kARfv|sA9FPj7>49Tq6*F)t^V6SL|vDCv& zXgl*UT3&rK%F|nHOkKn1KDCX`8|Qz(kzOn{am0z4rr-D2ypKoVjI&E*rH(;msCopiXmmi z&vANBt^3}YwsIKxjMN|QWZ9ilk@S{z!Q>{)9*B&(u8Vz|2#IzGTOaa%@vCw zF^PJSXUnPsvyL=rzs*=e0%IS0cEwJ&ckt>w3U!j?`@2pErcWcza6)%}A|vIz z+lS>k^f!qHb8;ohdxFIKXt@fy&@qa`$D9Li0GhfaAmKV_1}56?g!6~8_53TB54a(7 zxBE~#SNVl4#J-Q^+1x{pY64L@oOcXh62KQDeCSakr5sqkiz1QcjHa1^f6UCz9H+==vqMGTX@mbWXu&y*w*gFav|+!0vHYW%%FYrz-6_?Xdu zdLPvUaoETSr_EFEe6KB*5&Pl9zeR!azE33IN+Q`xy7z;_o(C@%CA6dL^(AQDfPH_& z02bM&_SE{XR8DX%A(VPC_MzK6Y2!*r9&kh~ zk#`MnpR~A~Psg8Xm8a)ug&>MO8;l~)qpsHJi{22m;O&<@Ch4qtY?2K6;0FV$kYiIErISz*qi(9UpdQRNwZQkUF~2kcX5 zrpIzmmJx%hgWKqQ$7!3tgGR))@oWZ1%9#3;FE6(fq`&|z`b^< z&l$I45E9bKI6|eC8T1(2D;heXM$`;So+zflLM!T$jGQRF!&+^APZx`*Z z(Pu&Xnok$dckL48zlexO#7Ff6SQ^bHp6q)HiFe*m+jzWnJTsZz>}O#0CG^~cI; zX&)@x#>6a19i`^INMqboAo9CgK?*{@D%-T_)}r;mApGBjzvFNbP}0c>vhuR5+XlO> z-@nLwhWvaYnB4FbZ1{r2(`l&L+`XS`KBGIa8F_W{>5AL_G$kzYvh97hp=W0mFAJAK zy>(Y^m0@ydg6^?k2u#h?R35W$2#v%BrnpwE*Tn~mP?!IG{1A9wzxJmWz`s08pAAa4 zKuuL6HMRclpMqY=7m4JsesFQH`e-57)*Cb97dPJ($3mp$&}-~IDz2`*?;#X|D5g-2X&?dpKO~3*Xc)pbJgG9z~F>H5uVn3B?$XB?eo{G{(d;$H<*3U z9X%K6|MiIf@*q^o?BEo0i~rx&UEZ{EWxsP)(EHOEw< z{AO7a^v1g*a^cdDZt>zrskZ+Kc1>EXt;N|~BaO+h)8-GpMqMa&pne%!bUfXE^VXTS zFxConP!(B~QFuf%!4$`%RATZsFXgK(VE}@m3NPYW71AbY{ri5^v)f!SR;6nFGPN+! z&`rNkV3=zwY^pI;QmQ*iWwz8Fz=Zo3dkem>IDGgY`$j3bzsVRD=KT<)YoUu=+WYRU6}iR^Yn<9~D0 z3yJyOFLNf8P0IwK0iXv2Z?11!X%@s8<^v!CR+2~ zne9*@ll)8MfBDWF1u({2r8;Uw6*nz;7WVFcthkHl!`|kgB8%y9)rev z{(4W%a>+SOw=$>>TL{QY|-G^1@s9XaQjnCgwtyF71^CN;0Q{`Qp-(u0bsflidzTt^nr~gV>W8W_sz!4!@Y%vk z0Tcvg>-{i4tl^5U9b=oa=B2x{yv zQZFsrZ6^(mQ53j0MVaollyohci23Jb)TPO#m-0PVSDrG!jsyeEE5pI+5Q8NkKV|`1 zgn1y_0FoP@gd!{k7h_Z3B;!G0m03Plm)xo^yfZSIXE2CPFAl2kJdP$^T7c1m@OV=Y z_*q)u=y-E%)pQM3bs50TW9RXv5Bv3=7sT5@h>jVu$^z>k(&6Tq^{lP^S)h1PktlY; zXCAJ)i;L3P&xQ}K=J?1W=k3oGzh1pEvd(%dy*w$eJEV4EXtO%O1|5!tzDPNG=|IQ% z@D*${9bVdPtY z{o%})D4PVY-3+)h3VOvb!43+^_tAl{-co1z6C~sgxh}Q^(820WU?+)0-i82H96MPDa7}|% z)pZ~2_-;4dq{IZmb-R_xPe(oKC9e`1k0_@tu!RqlR|axAFy95TrlCS_!jC)=0O@ph z_2@1%=lvcW;Q<0wuI@_gz*JfDy~k@Ha1MsZf+-0Q8mjaDi6k52&w7g`EkkQ)tNQRt z%Rst z7Czl~YZ>}fiI&1P&L?!W#!K>jRz6fi+LmIGr=5K=C zAT&L{B^VpqYODg?Jk3c>1F&jCvO2vuch3pP$J+?CSTr<3W>ajTo7w^^6^nNxr7o#0H{;SpoKJ7SJ|f)LUHLEHMhF;OXq}0{f52M3AqX zh@@__$?Sj&h@|X*jgSQg=ccN6WyU;~@yZG$#0X=neU8t|7#xfOVQ>2=keGPoeZ0`b zEW&eo%6R*Tr^*A4x`@JC+2037+OdE#3e(#_y(k(Ywmz=V0@?bGN_N zY27u0`^9i!>fkPp1iZ1@KI5NH(3Vk6Sy5xg_== zuq<+ih5)|Siwvj7YY}4qXapfd(h$VE&H^V>_)+qK?k=u$;zE7snc8U#3-=@Q!CYCx zoCu!KzT1OO5o91!dDI@FDP8-zGZ)X7F@cv(rtWI7{`LSLs~C4;wBMl3=bvhx1y>5X zD`7+)*3oQ{-fVicm?rHs@s-h6ARILOdNRx)BvneV{S`T>BL8A6S$*5-`8k`NPp1tb zs$jm*oJ&}tAZBmtO3O4z6t+t;reBTMdhv$uZVGvxkSOBD@gGZ24o%W#!oM$`yD#PG zcw z+(Sh6p2UnMakj9_U|vT50@Ksjgc?^`^CGZl(|GHI^&#ShVkI*Gt1HOualyOoZbYU~ zCN1D@yuk|bEs!;0;Z`ymdn_x_k*uuHlMzd78qI4}%;3*9K4ifc4ELJrK(m>Fm>aV; zz}$$WDYQ<0)<_c6#2jC4*yw_NrjP`A;+CCKt5{XVPe6pMhqMN`)sJ_5Vwm;gop!-w zq{ZaU(9JJkT&%z6rIP?v_h8SLr+i4sF^QckENLt5D_@*_CIOeoe#i0Ibxm(E8 z*I$JkNG2Mh4@tU99UNVZ!EO3wa`7ny1A1lu|;|k?4`_f$52YthYoLv<|ibwT=n4SD)}F?Tyqz zFM(sJ>9}|vY_&Zi*jW+eg3j`k0&GO<``l5<=Jdo3PbG^CY6(uiSCC3bF1cEERBtmN z6Ak%rJmOBWZkw(KmCSD&5jNV6VEV@c`w3#zLB{g(J45-c&qh96X}n?|q$@Si&N`x1 zKm>HVVpX-Ty@b!U1Sv5WuqaWl~lP2*JOc&=iVGW$I%PR94X-{Ta1&8XQo-F|ppx0pv|LP);Q z`l!3mrum_r-YVc~W`MFG++m8!8*p+D?1a5^X-}Zz_iX}F0PmjY5Sd>eQzU<5xT972B2sv%RdjFNl-Z3PfM5(2`Q3rgcG|1VL^ve ztKL=u@qI*McjmOd&&30eQb`IuGWL56vkDMV12IOtvmtV^g%xy1I>-ewgzs;oRk<$W{zQ$`LM1cA6nGoTxy;d^>dMa0 zpc@-7)9b}>y)XtL1aj7&li~Dw-78+T;l3@G<)HgqTXoGGu#Zqr(wHjd>+#z0>RrFP zxA2Emm{oyWOL;^cCyPD3L_ko@;uGnK4E-M!=NnMunHWc z%}ROg2p{klrtyJkb9PbNjQx=LfoOewAI_6|{cGX#@Zh)b6l&W0`-eE87#^Txou3eq zb(IvjaI_|HqM^6~WG{M!)i_VDdb|%#9SnyTEiQPLyeYNzuXIh1RPe4Yf(Qfq?gR8k z1|v9kLQv)bJ3Q>7-qOn{>#R=E!di%l+QR_-L{~lA0xV-m6bl08qdXCn8LO-GR5HG`S zCa%k{f6NsX!|x1pSK5+;3gMM%GR`g;pmNQ%PsVxWKQf=t9&J}bi<+sSL8I74yo^pBiBFvE^0W7A?Y?p4Y6f3-w(b84RBRXOO4l>6kH z_E>VA+qW(Ng2X@#$KXCojtnJ`RJm4zQohygCW&S!jCuw5hr`LEs+&X!a1GKkOel0H zv>=X#oL^jL;99>I)OqQ3`$WC7oPI!|+Oyyc`3iGPat?6|K!2BtRx9hR!4U91$tU4t-Yqe8 zYri0alJ>;&kEVbQ`JBzTPR7b{%)XqwEq3uZ^5}qhJ}C?-WvczNg8Szm_5$S1) zn*AMoQai!0w;DxQLpd^$V$bQsA1S2raXJ(4mbJ_~gMu@BfAj9zUQ|p)kDuIfglzzE zR`jf*tBE)dIWQEdAjEW*A2N{ZwHiuadIN2nUAj)Q5{xV5CQ|rm+j@L2|2`^RQ1Bg5 z1FzEqS)R66U?x*fQL9k5>+arstB&YzCEKykG%(2foz%M?w;WyD_uV~F6d(EK$+|WI z!|C)m+9F50p2#C~flzFcxDTDPZpcE_^r_3h+?*s#Jf#K~haOo+1he$11l@s1F2BfHHFa19E*Sj5TbuN67pxyO&zJ?A@(Fr4o5Jd740% z{n&d?vl$hyl0BqOXDUcaH6&MavGs~?uWp{d+9j0{72-_4wlf}BCf6YP9%MtX3x0{Z z6utO4kR#La27O0}?eP3==k}-Hp==)98D7p!g_KX}(J}>Y>t^0U+});oY!)1QZHvWg z+NomHm-&%#83NkY((#K&8E4GJp!MVom9{U=Wm(V$xdcXY7EWPZ)=71ORt}nJq$bVr zwBI5G&jHBU+)iFKqlG3W>S6E z7c+rT$H(2=LJsKMi4>Y?8R=MTRHbO-ZGwi0tE~wy9PK+3;HGR3G9a-Q=>EyfJh%61 zShqju3Cn!IhfWd)Ha`%1Zn`Wg|Gj+pEZ>phkID!BfCBhL$&S_a8p_{Vfo){#VbWl5 zaF)=Dyf{k26Z#98I)fpq*UVk&Nz0gs_lO!v&zUN!qOjqlt*^`TlaGNrao$~>E3zPhDtRc@#@vACkSTgBqW6q#NN3-WERz_431lFRGjKU#nFb89b^ksb}x<*A$OubEo?Py1J&q)TEv(zEV%(Y0A>5 zHyRI%QxY87zeXgdjyFzHtyyfN-?bl*Tp05QAZ1`%WLpk< z7KD@xgb{83uu%O%;Ih9Smz%t`nUX=RY141(K2~lu6gVNjMb^a2ykSSz?>*!;*tYg2C4UwO6#q{y z;TL*PgeKWk(Ku1Nx*4L8{B!Ipo{NQTeWTz9oVncsg4c)|m2VM7)^lls7lpFq`;pzw zQj?xcztN%pbXk6X(~qHI_{z*U&KAnJOMLnhruvxgQfUn&y7J{p-$V=%7FbkpZ+yY1 z*)XJ|TjF^hl~N@1FkvIKRv7?_|B8?MX(?n1872fR_67@neiK4SCPB@Rvj1$taf()U zPbA=O%E-&7;?}Q4E<_}zhI5J9KkHn(SIz#y5(B$-Uu>e!%(Lcmuh80id~E+h2L1XrA5+BKn%P+8ZsngK z+gx70js8LakKr2ZP^ctuXOC4>slvHe^)F_F-(Dvk=<*41t%f2V57Lb@JjcE@ihty%oP1kF}%IHd7aLtg&{Ujmi_ zhX*>KAN4LNev_mA%bFCcLBV(GZkt#9FFp-`76IwHwXwK8@L#ORYv{3ZVk%WSLR+!_ z^1S`}UjKi7Y>sa9aB@MOaAHns`7{Bic*;+)*DnFSIU6Z@iI9CWzt9;Y*U2N1?8wi( z;TZ7e(kzq$*Y4#&XZhQIxHE;AegL82*#HSvqbyJc%nx8PXa+N~@bn>S!*%=f(cC3N zh9?&bkiXA@maIG9+^`vrZum`A{kD0ko%Dn^B&&qlGE3kwF?{!N1ml;P0q_V8d$i2F z6Sfk^pm%xw?e!Gs#D8jZcx=@$U{VP2SpsS8EI|FRLG5pP47AwVPe^DVCp@*!x5tG5 z$MsnLFf(0~(V79ImWDM0(xicqxdh0|`Om?~3hrE1(-zPW z69fTvBnAos@>5sc_}yp`sR>LJEoC6qNj)!-WUm9nx)yCSNA2_M8T!56ex#hrZ)C~|% z#TY-l`5UlB?Kilc|CpuUD~;*Z%Wd)OCgxj7A#3BT(vT#c_af13L$WFeiwI8!H28UO^x~jXo7{MYEvi z#hYFfNV?OitS79dy1K)46d6d=)j%{KtSFAf5^T7>p{@E~UbqXu=MH$X>bfmHzm z0QHemZNeUhn7|_T2#b6NCqK3I774q?;Jja`m>1yVP~a61$u;v?ejMkS1%qBSV!+}I zaC9vVR~Or5jmWoJ)d3ZkQ=Ovt^f`&mcOYM6huDcg=e{y4F!Ckxr=)x!jGAYv(p6{3 zyVdG&0J;UDIMOLXpWT0AfT!LSe|c2KU)$_ zNe)3@>|o;vpZKh+@|@`f|Bk@&+jvQBzkqCbdcUQ?)veYTG%q8k8lDP z+pdHN)dyd&0L-d;+3QpZu3U%lfF7z1&JABH0IfI{cYxJG&X+N~?01s_;k1zKy;ZX; z4z>;>FKnTm#+3p64FT4`1Pkcc8NP`O)GFBKqdb3KXAFBF!>@D!vS~i!i&oK!E7!V- zU_txrnrDRYMo2@T)p{qTknCtMQaBPqUC(tkA4t z9h%V@B(diphgv1f4*y|{fG6O!RO;%%!`n<9#)1n*lvVeGY%0d=ylx|sYt_JFqEan1 zhoH$uss>!mB15;P4B2PE?Hbka3o>bBVGReLQu)J5h}mUYJ0Sc5BMEX~{Q&$}3s^&x z5p96ZH0gH^*;VeII;;VThM5&M1ozW4kmf0@zx;M&qc!Iaq^ zxPvvs9Uz*HE;Mw(V=2jePUmZCU)P=Q!+_BvmHGnl-l6fa(e=*Ngci(mj2-^p0d7nt z^}KRX|LLMGyG|7*qpy;b&g;s%S`JqB)_fwR6={I+RCz940l)eERuDp~=R?XN6$%5p zqJZE{=wq4@eEl zj=BPt&W#5?WDjYZAqthiswEiw%#X&aW9^&LP8qRFv-eAXzD*2ka0 zjv4749@T?cnW>0*f0n!RV>6P!(RYg;Nb3v`FEsZxci2DlFqsrMV)yUiBPUuRYC<+4 zL&h^HB=?1A8RT^kln&1fa{^w+P@R*k=-n52IznQ*m2Ep;j79V?AEL8sO415B?DDm} z#IX^rW9ys-8&P${X2>iIfhK=!jCn1gMsX;fEqD`QjHW+q97v#saG0+L{Mo>#gz`dl zqZ4H3@-!F8w-AXOnhnuZD>&aYTNE6z%csKRk zvDOwstWJJ^qLaNA-4C~6#5uhc(8*xR8))` zL*q)rsNS}-w4`Meg*27d`5;9{&{cgty3460>?NilPSdCU2dYy)n;)n#CFHUqrb9ZM zsiVMoRd2@|Ztw~4zB19V5tx$404U5hw|XLdDd_L9S7Sh#EyClrx3DtXwQCTL6M8o^ zvAbs3lj?MiGZnKZlMRs<0t+Hd*O77k#t^{1lP_J}=8?9}ltkn)AVm}*M53&+l1l$w z+CGhHk88$%8wJHiD2|`O^{Io-S~^b&w~+mbUgOS+%&_TQ@%QWZ)24x)NeuaCl5cwL z&y$UD=fABo{oEyxU^lNZOyk(#q~W#{5kB08I6iSoijL#vBxTlFxBXzvMMG_QFZ+U+iL*>;;_-<6t^{F)A zuyn0y%WoTX6t8=9l>!SNZ2(V2uFKal9hi({U1&H@e|^OTkPW<#=&Uqaq+Jf#m-f0B zMwDLb%0Tofzw49U+h)2n1}}0IA4y%7H>BIHf_P&!LOHixzc7@7V8iwWI6|?+3035- zm=6#Ms30jy8k9sId`$C6J*an_jPWDeku*?P3$Nn+PNKG>OT!s=#GSJvYCp%?BU1Lf|Nn^7pLmS)a0vxJ%E^! z|0u`T*!+OXT{4odd7M5TE@=AdN5$+(h;opx#;BXh(BfS(y7YKUr8+lQ!Y+$R@3ZC$ z>SuXFdEH<1^_;$`<^;1-WpauS0^~7~>dreL^cm%-aplFW?3t@P>jG#n+(Ua^m^+N;J@#K~4v7z|+oL?x_%9CIt z!;m3U`epP!+g_^c*9^)b%KI8xTgjU)J@!SrED;ON4BL^H?>?$kCL`ipnH?_QkW!_g zM@KNQ%~p^QE0f3xO>_nX^IWGwlft?#&f*|!A2IVLyt@0uSZ}S1_++ z#iK0UMTWpBE3GQbcUt#L%hXz=!!dHVfr1xfC>a>VV8>u9W!}hDdlqyCtQczy9AKv9 zF;0-O7C@f=Xkf)IXd5T8Ah8SnZ5S%d0LMQu$0{jexVBZS<+t_rsD;co6RQH^=A ze&3llCEyF^#;HE10?^&n0e5#+1Cw4%r;V9tH@*qi0^3rkU{|kzOhi9bqRtd@?YVB! z?UOuJGqnt3twb1!Ad48WI~;Ifvt_V~bM5ob3mCJl8ny(*={PR;rp+5%6Y4j-B~!9_ zAm(@8JougDJ9P$YbK<;iFA~NYoLAI{^ShF}x2cHC4izP)Y=+MENrxTOZ;M&cHR2O= zZZiZl9QT%By^K1UVDOUUQWYXIuAp+~Iyf17l+;iEGpnXEaa~m>Ya`uo=Aj5rIihKwB9qU18 zb?>n)6F*~4IqwgHfa4iNFpX!g+mAL0^k$=7oDeK`E*tgNeJ+1&Hf!|PgjQFEU|a+$ zWgy{_4tt^XDXb|J%$Dw@_T`Y$e|0&%!F5{U+D=Wak^rE0)h_15b5~Wh_LxKu-9ZMu zj5sSnitOPykNMU_2?EXtkCkMuG1=#1j-OZ6hx%m5-gVV|meu9I33rutUEr&rTU}3mIDz3iCYaq)_wsG|cR4qm8F*f4KrU9Npg%pem(v}pvKmyke2KF+ zH#-cyb>VisqxNB>~`Dt6l!9_ zkVjo#+H&W`=3y_1SNB0^bRdY!WbMs~5xo=RU<-Qtj)7G8HzPel>5-*zDI#6`MEww3 z5lA%I6O$1X%!gKqC`UY`^gcFJL_-(A3NJUyWlReN~3Ld&4u; z$R9}cH_bnQ;V{$m+I#M)^I#)C*CKvrDd@_W!tPELG~#%y@hVB*W`sYIn`UM365`VZ z8JIg{du7}#&A?5wHY*ifOCof9W8?PynQ3XlUUlQWD}gV|aa&Ipw~N*U#4WfUWgI#P zmq%RNhu>XJy5%K`I;%cX{fha;)Aw9M8MjJ0LQ!rln!oaYz#; zA;hUg^gfU41QB!@QT}R|RMHUD7RJEzhgQ%M z!Hdjx!XX_-eaOL;TGpNBkmld)nN{d6iV(c+17#sJDqs&F_HBGGZRHZuu-}du{5ohxwWBhXQE|)nCQH-9~PE zy>Awwwclii01x5kkCNDJ44|g2-Ul(XMiqyk(IW<@QUmcwOj#2m-tqFI4l*KBx7M*U z$nw!47q&9>NLhrb`18DXvG+M-`em20Wg#7eh73X>d>%L)zt%3sYq!5*p~?cIkbfp( zDj78+y{p^nQkj^v{P0yAcw^>uB*=&!Lh6$jK}gvzFDIU178;02@AZec2wb`@8^KkS zP)%}Qwz=Fo!|{cMp-aWTq#XAnBP@A`G&!f@as5z#(b?+2UU9St)ij2N|lyUOdvp*+R##gFpCq@ zVXSB!YO?n^t$=`vlQf%69C_*!q|5O9T6J!9vYx9Xbnvqx1y*dn!%<^JhTZQ9tal+Uzeqrh7NbS#>NowjPdg(?w*JgpgH@X z^Snd{)q2KC;@G*FZU1CRQ2P%|uD^T+cCTWfF=}?tcpW$lA^gpN)^JS29LQL%ExC|zo5}NVc7t5nTX}|C zDHXfEsE6|%k$t^i#WfccNcSsqmp2p!ImQq*%CAps6OWF-$n{XFtfV8}B$*`u31Wk4 z;x;5HWg&dH0x4U;CT4ZY0SSA*NoHD&SK28$G_x`sg1S2h%6;jrO0m>7OENV|bf%~m z0Y1usm%O_Z3WHGI1E-F6`PT1rGH4e9kOFE>4>w73HaWgwu%j+<>07Ulyt^e{4d93t z(6KdB1(V@=1EFu@jAHi5%wo}BalyOH}gV~IA|hqfj&WO#K02wj<%Pz`N=d6fLJ1YOwu01b2L z5$RM7K!)ux-%J2x?||(W2@O^6g&inM0CIBG&p2RM`2a~Du#ijzf^tivq`~l{WjN@Z zgi=9mZ5K*@N()Kl1Va9NP=5XW5DJb%(bN7LYFk5f0ug=f;MzGLUt2PIyrkTAD^@uj zK5Lq;2A64epWA&CCyivd)GvV@A@5cuemoD!nB6A;;FErBAnqpPa+-@CzFlG;#_kbR zYuu~`vx%u)nrZN;&p?vKElAgl|2>KQmXcQMEO##-4U{DW^cF&w#7U7WL)lVcD^A^Y zUl>wt#VZG0PajzR;DA)?pEiM*78~eH9U#yad7q^Uh;e&Y-M~N0hN{65f5v{Tc`TAb zSRhEm^4nWgVo(P)dK>}NEqzwZwP#<~4q)azWqr5*Jhk+ZLCh2eB)rb_X-YK!2&{ux zMV_b_Z8~slbJqdfn%@_c!tuHV8RC?EHBKKM>3{<-6lccm#&sD4a=hd@)9c}{zrCCv ze=<_^-iW4onubhjDg<4=cw-X)VBVK6KE|3*fpr!EbKQ#L6$eZdv#R&B_XTrYLHR1P zY7#DPLCa-))C8>?3YtYb+9?2F7;O3uRO?(~U_XbddaGNXdo**?bP;lkn_K}6I%hG@ zHr!C}#hbiPx(vfJAlNYS7PR2bv!g;3JjZ$F#g!CDEKe&Zbr_fY_bnnBza z+*Bj0?`H*9HS;axS;hr=a~Cq@uQS@HOGZh{es*HeQ2GZ$1O=j92Nakytlto8tw;+d zFVdgvtHzdS)TCIJ%aYCX+`U0PQe4)IF$+|e;h0QJAAVedgl-x|hE{LX$J~zuMDJil zclClUVB@|_uBo!%5dPOUYQ?M%boEjFMfbkY?2a}!`dy!Ie1@9F^SnQEB`X6&VD$r9 zfo~@FIkgM48o_loyME2P`#7Q*()mL<9{wN25=PxWL||+&SEwjWia(p6_Y)Ayw1{ap z1n!t$KAFlO$J~sxcAa-x-_St-L~e7B>*P{jv^L|@dlooRZiZ-`kfD?tL7J(Xlj)^u=hH4ujJWPf zunh=?R%5z)1K{{NtN`Z*OtZ*nbzb@K;Z5@U0fllkq&9VS)!>MGYmXRxT~Jp!{46jr zIi%2#y>3T55jbFSkoxfT5W96m+`TjbZ6VZ|zwcH{w8)5RS^RDze{0pXqC#zg7faD% ztatmZ7b^`LGn-~(`WlMMD$#0lzyI0!{n@$a;c9^WPfp=Q?WjDGp(U{O%z`BoIW)JR ze$#FP6ew8Wjm-f|LEa3uX0jOk@mDw3phO;GzYq&&Tp9uVrzc?c2!ow6RaPw+b3k5z zvI@*H$LITfEkMx@8oD{w6dSVbv6rP0*%Se|NhwJWXFA9n4P~E&TMt3OQoh_br@$spDm?V&JIE-*DG(6= zP9I}CND1#L(?fTv0Y#pHI!yO<<8iOxHXJxF?n@6D)#9VL2)WM&Rj~)qC0e6jOXdO? z=pwO75s03!Q~n8r-9$-HteFrPKw}s}7mrv6E(}df{hncY845R(#+1wa6zkBZ+_1x4 zaCTv&3A(V2l^9t)Y?MZ6SW135XJD7FGN;9V_``Uce4+2!Qh(Od!vu`R;Xa2b)aHXkS!llJb3Y}4$DA_doJlPY{B9P_s`?~-0rZ#R+`xZRNUW9;kDdGGH zSxoj$u#>X6m*YmSPG4QRK%rpx47aQpp5Tpx@fuAlpS{2yOfS{O9}lBI@4f(}2RoTv zD@?^v2zZBD!1hTg_3%~IcRi0AgLx5eydZ(RuR$FM0g{Tw!zM!M1uD%zNI?!}ee+I$ z%g8ywIcIZkF+Va|kDhi8KxPDxCjblJ`HXiU@%VrTkzI`KH!$xge6t#qT1F#~U26k0 z)mh-2S79-wVcB79yH%^az>H!H-)hBq0OPdi09GrCHtoG;3MExZJxOy(UHCo}1{gXj z?*Z2cs}9wW(^|WZ$JdAjOMb33yzF}&3&8>`Sa%>pkndd#IrxUrCesKZ_0`OqK7J`7 zDHAg-dP$&^1JPi%M1nr4E=$wi01P)u2TZLf7TF^iTrDsQsYRO93V)^nfb)SMl!2FP zM$7n=UUDAflrt5VcQHlt?t9`LLV`~$!Oe=NfNEjIt*if?!e6@&9LNG*?pCS)1GfD6 z$U!$(yzx50!`qRXLJW8kWUVQoy~n8qMDJ|? z_lXV2Uk{dBexP7$z;w;0Hlk)LtzCPoQWl2(#0Q6F8ghR$oq{Je2ndZuN5W`bE$R2S zsZkxs>ICHO@j4+=rIrZtCt8b$AHpOmO!P)g>J3L??&XYC*-%1^*%x~H-t(p+`~f_r zA13vyCY0M~D=&-ROFQE=ZzEtd$mxxUPU7rEKsONwj)A9-!#%y6oD;uq#$+s~f|toK zIL4_iJ_Ve9!VG9-z#*lL|70|nFWgQ9VpFPhignw@2Em}E#01;Hijc}dETQyK^^J48 z&v%iAUeV$^%`K8Kzdi{#x-Y&7VUvCLjLyK=C_LeEi(3@kI(T)3l;0Tw&=YBr91=wn z?4FeThEm@3tCh|))O{JX2>^#bAB6w-(Tvs?aSZOPxy0;igCU?WPIOn@=zLtl0R=Bw~F)k zbGLZh?El#!pey6&E`vlB)&FJHwh_&-NA|IXVV$85C_srRs_Qo4gHjx=Xq#bFxVRKV zeezvR9(YPmB{vahFaF2^;Am|-PZ`HnUop^Y&DYC{8eO0b-5qN#rf=Cn#WZ7xz z;-6RxV0XjWyg^Q(_|@e_3x&r9EHS|uIKS+Abq?_;?v#!asOv<3dyToR()BJpPfAmq zk;a#%cFybZqvu`oWH|nJXiOZM%ZBHpGWPf=>u`GTnyNKO+QyK#ffFO5A4#~FZ}RKTV)i8!w?1fW&K)U>QXo$K%L zwU`%`!SUj$eZEz}->x08&kSOGQ$82#`m=y$4`+dB^EMHod+6Nq&nLkB6bbc1_%2#8 zY(my8(@+ghDB^AwXCQ&!csG}>g3XAP;!RI`Bl}cW^XsfSPiD-tz+j@r%-R3IYS`t9Uvn8${8sK;vb^AM(P6&bvL<{JK^C?cw#%`bv&K| zp=ErL+-KCj*WGEXGEiO!`Yr3Z(KxOT)>QJgoc@i5i6a>IZYrm^ zWqqQirJ%)VLPq)$RTqqnce*{2k}~iaO1pTF3zMNIEKHUglwid5HOgZLm7!e<1*|qn z_ODMIx`ER3Y)3lp>kgc?+x2$Kdtlw<#N?i;L;H*vjN@GTP6xLe6RhSskyFr~I%&C; zupY=C7)COi4?9F3O%>hV0>s+9-|>zYE-Zmu11=`Kzo(TSWm@Al z=I{uMS`Bs?jYF#I#-b>v(9St$X?jDqi-oq1q2q1u(O$3qJtLO%Vc*a491>ni`p$GqHV%+|p{@MVGzEP+g zh;dHHbR>%YC}SIosrzl!$iMtoZg|j{ECOMeT6z-y-*348>$g2z;6r`D{CE=z`1*r+ z_{SfW-N65jGdKH%c>eLd{&%7C=U;ya2P74X#HQcBzcAGArGuo8#iS&V!Ufe=|MzEU zWTD9{6zo>^_*=>C|MRzT44H-6Ze~mW{=(p$Il9q$gu7oYfRF8$MBQJdf0fi_^Y@$e zU#dL6Fz$C%v$4Zk=q5j(!`_Fe97KWaL?bI>qe_G3b zVIVWHV9LYqBp8PB|8c4O<(4}|f^{${SWx-szgP%`5c! z)cfYYywrX*>&n`e;7UhPb9gtmav1c_aLgvizIhi&l)_^{Hui3Uhh@Sss~LUb_s0C6 zmzOfGA0zw~W0ExT$0#I!>y!~zi6X+U%*7XFu$*k935(CK2=X8J zp>)ygdOQr8rO5f44{2Wsy=dzx$27X^#dg3Un4gCaE)0NagV{)&d|P&@<{cCTRew219^@ zhO1I=7>H?Def;(|?-Z_KIMp%F^*}8z4sf~{Aoj{zaRdNz)Ds{t zf`44+UPPmU%@x_5)B`=@o%sdJperxld&NHM33h2myj<%R5+-y?0qK+ppLs!L}RjbEXI>czVn;8h<~A!|^xze(}cuiZa7H&4Y-Mol>&Z+c>*PLSAzb717I`NHhT-5vp^Gx z4RC#Q5KgTNFhtoEE|C^~c$<)J8xc21fpMGM1hncfsY{+3z zPf+##q3uLIY7f=e525s5HSUQnC8#PrlZ$>fErv>%x34ChmeysR9{)UhC%Yw6 zGF?pUzc!k_yL73ZLD~GHQlW)5A9DQ-d$YJI`!CF10{}PErgCn>LPd~<7Ie}tX0A;KV2cp5f%3E; zC_qo)?;l$&8h(RM;QDjbk6>*t>S&wOn&4NOzVn1koH8je;v6 zA3x;mxSoc(2_i>szTqfa1S3V?zP|Yb^QHr&vG&sv%d9(-c-!}R3RGjwkOC5P`{|7{ ziBSGDlxRE)tduM#mpG;JC=KFA4;KyR{;wp9y1G@bRr|ql^F=NSepf-4t%;hhV8FVY zw?Kf0nucLuDi??*2(|^Vb`Z$dZQDevE=Y6M!uLtgvG3$Fz}t;?i09dQ=(b2Y&Q#0u zWvEoY!FA#$h3j2y6fh07$%ZM2ug7g0#LeKl)GR|ocjxQDm`0({LjY>3YJd=cd7i7^ z&$+ehejwJ5V@FV9GQYsu3Gbs2@UNmGJKlG_*p*2#e=g7j09pb;~hA zSIxdXKz~1p_h@+@yYAW5F~j+#`snp%j$z|2FUXy()JH5xc~7>>WZDI*n@q2ZFON51 zOq$ZL4?Wkf&1g?oo|%Jr`W+)gd?>l z-TgD!N-g;=p5!f*A96HKV&8v>yDkwRCvqFDw83mmqi4rfCc|r>A8~9>&UKS zI~$+pF9v4QC*wo>+0JmBw@00lY>$GCTMA`;g2MFAILp_Zw|QridA%{N%P91YxP=sn zV3(SWh0R{nIpkm5X1l!4;<8BuLhHIJJJN=v!Edi_X2IMW?KT@Zj{d4ca)}2R@XP14+14DbM0nXl#FVnXFW>Ir zdB#wWedNtYU+mDG;OF3Pm&%<^NtnQ3+PfcAE+Cavd%Pp8FoYG z+|ti3K8b7?=Sem39zsI26F-|)5*yeqesS+#-0E!bv_YfW0%e6_1WJOKbz~0g0$imC z+KlZQl$n5z>r-efE%Ffm`Kl4%pyZSk%fXQyf-gYK$SsJlJe+aN1@O=nU!niX4K%{Q zE!htw5Jy3)9EpbW73qBm&{gc7Md$Aey94XMqGtmAqtHo8Y7EE=Jqux7fO&Eua|~EH z0$Y5?!@vc|@q7K^5*k4Pkwv@dt3bzw4FEn&u%Hd_FA+Ea2^`r}DvCdXReBW#H}P}W zK(abNr=-IK=cZC^D#*YMGU_AQ0vk5pmte$$lk_r7v)ot1lyKdEIMLj(i-%SQGaIF*KweR4Nm`Iw)z>3q$T6DPF|z5xK5)w4&oh$ z`TQby#u(jw2Od5qC4VGf0|XFI1_zN*p^!Kd9q-j@1zWJKal7DAYdG3Son21Z!*-9K z@9X+ME^>7-S|Lp8(7t=LzszZ#8bVRuOS@Y8;(PPOp8N9^$LL$vj`Fswnyp=Y@UV10 zTKurA2{hvyu5;i;Q{U_tD2{>Yz@EdtODmi*Th@DOj(zsPMo+1-O;Z5pKD{8e-WXvz&~O;&=ce(epKAYJFDPJHec(TvFnxudVb;7!OQ=;C^tNhZD`KBkO_Rs zYfX`#I!tkMDO_2Qew2I~hDN0AS>JEjB+Wlgq;0U(Pd(3nx*FLKNR{N~?6NM%LY8XL z=lznE(A`9wn=;xh!`(gD_FMkfmYUM=P&b!8w1pz`y5uW?Jg;hpPw`oDLG1oUMj0lO z=QMg`BvZoGR>y~haT3#|3_-ynUvI1H#7qP$oYP?T)rH!bdwh~lv)LFCIJbN9ssF7- zj;&*R&e_FmLr}LUDdi=f-A9huAF3_aNemNlW&(?s<5{m7KA$1QB)=f#@DyuBHIfNM z_SYygvMEGh7}*#!mHfDMxMki1(DG@UZAPZh>>p)W*+%RL>g>B3IB}wVyWPp;VDM38 zW4X|BZG2KEEkkFTA#V)twnwwB{snHjJo3j;wZiS8+K-r%ult6DJD2m0LT|y9i z;B2A4;(x%8)#Pp4}BW(`AEckfd zG^@(F4{z3QsD%hV;ML2W0AjT@ z+{>ZN-w)5-BMFiw6*HJKeUY7HeP?NiB@!|$UxK&*%=-KxCP7ilfov(vXM1OdfCq>q z&`$?M5C>pMY{7vP3@8CsRI@j84{T=|H8}kSyHFyb2$gC~34e}}2@upo@xhUwaxwWD z2^;wR!|JuSMRN*{C?aoIVys5JzZ|nVrp*ErB4k`5>H+yY+vYN85F(dtpZNnbk60ZE zVwpg<>dRrrSFZ_4$v0PkX1MXT+SVBMT>#2?BjzyE-7p5~x}k6J!ms(|8j}s|t(}Pc z+b}Z*fyBQ2M7z?G1fu=>#x;X}?(nkf=V;v^>0ib6TVAUKa8|pM12@gMOg_)_1GGo` z6g~y_hU5=9MthNewb19}1jdE$C_ z)X%s|Fu$T>@ZqZK^KhSbKE8B7oN78X;H7#R8 zY`51{@Ocye62E8Kl3~U-!dJ%GxX-kO(HjpHM(j=q!-5%JH%_^S@!sp^NlT1nznnSq z8L{g+{o10{=>bMKG_FMT`6xc->m%1A7o{tn%$GHg*Bz77SHS3I<&#+%navQ}4I!<> z3@*M<$AlBE`Ltjuv@FSLu6%yI$yd^y9T&f$>k^W0xOjErxVFmXjx@u=uWu`85;E8= zKTL?$mmcXdp&wonT9d__9M_*cJ+|A^=PcxNUFe(boD_hM-r+$brS;nble4Krl)k&T zaM^UwK6h&=^FjXq*n9JEs@FDtG?Iu0ROYA*$vk8pmXyp1iIACOp2tpZhbkaz;*; z9NuADzHaE}6uYjaGk2Lev)JP9fWAuDCAEG&@dSdT4f)8z+EDqfjkoQQ4hHfKE-#t* zJz6s-4sx?duUw3zC0jg)O%9st6U>b=xpKKJdYP+d?O=Z~acGq3MB;11cnS9b5w?Tg zLz`nVYhg=)nvR{vkK1;gF|$2PC34c;whANF%E5D~&nUr%ufbK%{Y{s7SBc-SIOitr z^n>cTP=>pmJ{$bv)gL1_1YNVz8V>SXh`gWac)|={Yisg`6jz-+-UCwMhKrMLqhA=| z`InkVMEUvAkJT)=+UAu_JadyCx{C3OBmjv2=-xeBsV&yo$qovYZb~dRRkHWpKc7C* z_1a@Czv)!)m!x)!1834}IGAh746Bx8j>O-X**GaJ*Wpo0_K?<`MJfJBKn7xPcq-?| zEiv1)os3sB<7Qs&c4Uqn30leAezM=+m&tgJyzYq}Of|brzL{-i9OYqtu%u^h;Lc}G zZ7!h?jGEb&7TPFxB^Ns&JUo~S|0a|yvm`eho2zRRi9CVE@>z1DXwNRaLJY#ND~c1= z#V~30KxSM8xqO?k{X`F!Hh`dmdVJ7uo zr1P;klAbImM0%txo0R{JJzqvyzO_1fq*O6}--xMtUZkcevZrGr@I$7X>WjfUPtf~{ zVL@{gYn<18#fp(us&4D?eRDqC&0SWXge&>J4Uv8_ln*q{AZ=PyWPKUwJ7XztSD@OT zJLZ|;7`dHP%qp20FBvM0tqKo6Fnt^Oq#^` zf#WA#EP*?v*mO#l6dR?o4kN$Kw{sHL@9vA7(Yy|R(@Dhv*YwT|BoJ_2yX@b{nzv=! zm9WP4RLlp4@QI0$?)bQqE)h%G$tpdX9hpA%UjFIlrAUsE$5F#J8NE!z$2EJX zxZQNs4kbobw}>DB+So;tQ+I zzK-cgU#{`jtS4T3wpDqZYJN1xPc7v7^|NBb9xs>v?n#2V!%8iF3^CDGDmoce-D};A zsfwMHwLMcjcYQS*sLhe5( zlLNwX7INNh$_W!7_p6!^Y)eIpCPBk@&nfC>TPC(HBguiJtw>Nbr+u~KFtK1(3>k`X z&~ixN=!nt0{#4hufKSxBr2E<@A~`i#sadgJt`l(7|kMibt1nmm(9f~H_d;MUp8Sm9LWXBf|=P(?u z2inf!4{UWRs|cii)Of!B%~|pO|qk=_ylBFDv7liZLZ*xiV?z zC}o?DCT3hyQ(Fr>%z8&Bs%Gwn$hhq_W&aLtCgZfQ)b-1&Gby&AB$p%?l-oYFecYLm zC!Bd8&-V2dr7caz=OX7)2{zoO8~OL=DyJV-xRSel-u}tm+8~f+!uQfuQDIY!CP}b= z=Vv+jy%Uipp`Qz6`s1%|&(=we#TP~w*+iSjO0ABp3-3_MH&19@V;!q%xVE!ZcqKAo zONQ_G!sq9lM+dzX8rDo_hOLq!=AM_Nif!?2fc8?YA!)2!!@+Y%g|9MA8C!JtvgGN0 z=7biL-c@<5h5puJn$`47v3XkB$H&g4s{zC#V@y0M>^(yBhRfH;eKXsm2;NjMr@iZHZ#Hh(>qzfQl6;r% zj^ju9Bk@*)Me~S~h7{bc#n_KAdgeJdwIV?M^k{~~QGrK&JR|xikK9`DGjf8PW%uu8 z8Se=H}6|>H}>CexUp1^cFGXp8&AN z4X+YX_ZA4!DgruzCFSor&@rr>X84?yapi{)Fg!(LI&}wGn|};^sZI-Ok9=SF>I0Y0 z#wHJ2^I3^ouYIa~BicQBK9K}*MnyKv)0H~Ojb;^aesWs1(F+ja(|esh79KN8NeZTl{2Os?|=UW9NzM%zjIL6|QGW9#BtJ zsge)>v~>_aE+5wTzIh>CBHuWvORzxF`MaxOMzHLiK{ zBxC4Wl_P5eU#JdMHukmYuo^QD<>RFA=Jg_zXjhdQG+DlOUW_i0-66O6{3Fd+`pD1J zFYzUp8*bmo(jNORt){ZMP$!W@rksW=wpuECYauTyWPG9;t3NWcI>X1`>d0ll`&l5x-ZHBFGmzI9pLOeP8}>5)ZLu;rXRst_;Pb^-0KR&=KYAMb4X&j3&k8ZlDCGiwmgZdXi|t@n@7l?*yDa=oyT58_K-zAZk`rk z#oKMCQs4DY{5Yn%+d`z`d+wO%y3@$0^5|YuE+T9mY*Dv-bb1qZ{!bKC!frRe;$A4q zvqSExoP2AvsE<6YdR^NPUepG!JTjExz>@F`DrMjLYN z*x0|+YOHJaspqRMQ~WZ$Q^J(9-aYRVcri(Ui{$(y*R`0BCc)k@1P>_FxQUIj-_7FF zU&(W?5}~cftiu1+m~M0{$~LKR@FB5MSIHyHmk=k`B#-kB;uDJar8q&!W{%Xwk3-`v z8scDp~ZhG#@s=3FTnQ+)? zT=Km|2*Utt(&NWz(kex_FDLLZhdpjx^R@{&-KE`9ugrJ9*}pwjF3-wCp{xVL6YI;S zogXriI{)JW+jfEV%p_tMm6Sz$y|Ajr&YSBMcAGCdR@w5H;2Y2M_EvR%^>W(&<6PBX z!rXlitQ@Y6^A``=d~f-9uhnVDF2&P!JjE=_=EiZ>vph>A0+yecLTDsdifnL|?^6sV zC(egDazSq+bsvI@CocebTd+lyNh3b701f&{QS`b-vK3a zzh@@j!qUDwy3N;KetYNsv2fGoydzzDHdBWTtIS?qYj72fDicMO^zK&s*FVzF_<`9) zU1TE(XOZfaC9x>{Kd#RiRSJBwZ@qUOd7V)`OhD9?S0OujJD+vV|JTpar`fGg&Zv6e zhg`Azvrfm`_pKaLTVCa7llJT9zkl;@uXxx=sCrUj!mBp({(7^&{mQR@b-q!PZj&N) z!6yBGy=z+wKB2Su_~ZF11Wx?<5&!WOCWVqGM`!YI1Z~cL|3zQ?LaeX!p!UYsiuwQc zhUi^XC-;$l>)nv~LU{Hc|LE_(RFlhLpz5kA`D3o)zkW>kKYHB#WSzZ91~L1N-b2#O zE9d{Ze}B83yZ>r%{3uV37|97ISNDmduu-(Om5tcE`mcWkBjTqPwZT!X8`eqrzrG@@ zV{*7~C1F82kAHui|L`iB0)92D_Wys=|FIhWZ@Z>FK7Igs^P;IO*a*E+nVSJqo@1y+ zEVZ^81MaJ4=4{)ldgdWrxnGMY=QKPr(@lPM{QtH+p!Jkv;RnUvshO(HIn%)^7_nzu z3O|9;j~C3rPx}Bks07a=rhNwZ+>F3{pF1gym4Zcy2%0S`Z~>;Ij%-#l0~dGXv=FsP@`@0 z1C$oWxIq8v(4Ey740uBjq(!`Va&;C&s3QP%B$*|AqcWA+>v?E>WJg2K2G^;bF-s?8 zqYIM>v6jy_09t8H-MoOVcls^-&}YT*io3h{AN%LOtc;pA#)oEOKVb}Fh{g{{iL2$c z@?k9NFcl9Ya&SVHMhh!%Nx13uhN3wI#)hD{=JZhZLF&vtF#H%RLIh<*jpk*< z!B7>u;fC_j0^y^I3Ea}yGL`GSv;)>QA$T3&nazi0*?x@gVAQRdxAWkJ$>!6WRYspr z-@N1lZYmf?xe8D`1;HGe-wDQ*@9Rq$~4)QG8dXVjPtd%lGRHIzN1jW#@HP`FS3f zWX7+UOPSxD*Z~Z9bcfPgs0+<;&lig7?Or)KztPUD2Qyu`e_W-i>2t&bz+>iBO(b-} zoyX5MxvP>Wz-GfUPu6u>tOOV(_B081kP`#qpo~oq_%zk%IK8`Q%yKp%-r{t;B2OWt z9`aHw!Q`F18NgwahA@yl3xI40 zv40^iT%6ZH9ONaAYGNh>`;HV zk`5#*kGha6)!$3NS~3!=xQcCpM#EU$UI(_(|?- zmWm9c&=kOiBB87A%7dhz#7vHDM0|zfeqT{a-~d>~jDRLMbbj2?6IRaK;Wq(e3#ay| zGzj57lu7@za1l0tThMrdHeehqkjw{y1WJAC$Sy^1&g37S;ZHlf8a&&OU|>9F{($z_ zqSp1Pqd%Zd>C?!vBI-MwBx?!2@V<%KiK3G#UW-;& zxsNL}p>>7emlLOd+K$AU-CJ@WnFv4MxD1a2`k{M+ek z5KEc9g)%q9L-$IjBT=ymYt#AkxLgeyX(@kMAjE{e?DlD58kfMYA=uO0(mJ~dZAZAD zA-)>Qgn&Wm6;jm`FfQE{lDy%>uLN$3l9uN}eJ!%C+G|2!Ks-uD^a+-^F;6)8djdb= z%1Aav7hXIS|0@J9h>}fd{RDrY2=_KdN!aqoB+2QR)0+i7VS>r)<^=-6AXxiSDEuG; znC86qVrPRV34AM@r;qQA+fQeQA>ET#&_GjG7~`RAw~Tnp+p!*xBPp%+_SdeG5j@uA z)ea+IxC~DRRZ}I9f@?wiNtFsawbrvZwGVDbqRofS6e!mq##XqWHvRy?6!8tU(S&rN zgF<0ajeFGbU1)7e4f{@mhxT_mDgQYO`N{KP6s#&C(idYrVD37m)lS!`Mw|9@cSLj& z1$R&tJqe#N59&O%sbClFJmQR;1sDao9i-HcdEHnaaFD^1j46AvdA=*Llch0mWhu%U z*|d-fP!!b#bdrQJ-pXjRSq;b7{gCO3!H!3tXAMS2O*q$9^_N1-5P_Z)cuQy)Q=sG590iZj|ywWUAK@&K_>Dq&k#VDUfSD6r<(urm+V z$ER1DC&87A8ySTWndB_(B=6`8@cpoeHkmLZDZWpuye`#jewUW$fS1ac|Igi$=}248 zip;<4`@f5)8V*KG7n44*K^9q?az9cR1w140MUFZ((*v@q_k6jmkspwB-oX1NlUMJ# zslh4reN&UM_zfC7W~>SEsfYx{1~%;glg$X{=0#A8dL%b27e$cC9f@CT6u>VHaZZFG zTD*!~NDXCw0T5yu3hd#i!Nm_M>RVm=1?=!W(}kfGykLu#eh<%bYS7+@5); zr!0Fmjn;cTgp7)5IAxyy>buFiomvH4Q&mT{RQQX#_e`O5prHn$evkq163Qnug%TlO z!?s-I%_c9V;nbhi!xH{KilC9?G%LkQh7jK&Kf)Z}S}^HGw#;lS{F*nw`lME(V2j;Y z^W?qUAIlRY*r&t;k*_-@&(c$R0PpZ#>VRs*e0)|~=(3~i!#G;5^%P7^l9Fi{F|H>! z&bm`j(%j_BqKBWXtdz8#&4n||BJEuBj<6pXXddSQ=;}Yz^gs*790}T^VsqL}ri;zm zlw1{7XD>84C|ZI_<==ENe4JPov;kex9hT?SmGtpeMgmnXl(-}$dHRLh4_X#R{~#6 z%sDPf=D~Fwyv;YS=yn`$^;dv4++6lu{<&B2#p~qFa=CmJ4T!O*>tj zmd?dFD0rgTC9BBsVyG9RjO4ZjC}fjU$lpzwr_eGGVSGTu$h#+*&~GEeM5@1Ry16QD zaCKSskT`LER=-lQP#XXB4C4sB{kX4}=$$o4d7@TQY;0XpXB>NjbU@Mn!-u!RYid(+ zx53(CH&9*ECrM*deM!)8d0nJ({aX?U7f;6g5T8 zY|6ShWCGtnhNzvOi=}1jPmaB;QgVNsm{hG-u;hwpEa?X)S~iO$Gug#twMLw5F?T{Z zYl@TF)=9DWkPBE&_FK!f!Z~fNE(M_keZ&(BGDnirsT0Bk{41gdnJtB_O$v^86S-8u zu2MjE?#%^0g>01+8p?JF8>QT~f_P^&?|AZRll1pQU$F=8wvd}|Duq)djL^y05Y#*Y zLHq*Yo%3l=)ugzOmE}v|3?#>MuW6gcVB49r%pW>C?w;tJmeM&1-D6D~SB=}`R*y;HmV@%GPjUFpa>K6=OA%XB zS{}PzVs&FmGZa7-o}F?_r#z<{=BkXx5cTxzHeX`L^kW!#xcPy`faS)WjOf@ufMImh z8uT7DnCU_aI>Lj*q~yLQkJPQ_mRT3J3z9%c_-zh`gas|w|?!X4iDw(~^xzt4z#+Sv8x6wkA=66uAk1;Go zu`6PaBn}r1BNzE2NeegGgi?ynhnetEcEjOXY6nSJb)5v$Pk2>qualR?OE%p$-61;f zuw_B#Q#F0`hf>U<`_rbo&i4FxZ?0k;h(9|NUJ%@y)MkPGI2E*`PbY&lg=;ynj#F&p z&w8o$3BjEK8wB5pSNUYQ$A%Yuf+O3uq{bW@wB7?F`)v6%mtJ{Y(!M54O#v$=j6QYe>!enRbDn@0BD3AfA9Xo2PG<;*`C_+IVoyq^=t$WQZ-nNZjf1vy^$4+Y8q) zSntV;Wa@iepR4_3Dg$a?Q}qnEOvg258?7tuqToi6z&8a^`fbl9I(kL#436Yh?&-jAp0-*s$O zJwp<^!~Hd2c9`Yv1DbvnJJF%gL&rELJ$4^s#+vzy4b0j0W6GjoZga&HaZ^x8^}t5;n10=u%X&#&SsIN#3H#4APb`PN@zJ0n)Yt6fIxcBlH+vOtnr!}q*zSZ@itlx0D>giOz8l?p zjM(02_jWF3a>?uwprzt;mMT4yM7w;7>U#gUv9|e~%TnSP{65zv657RuHn-eD2LZhU8P$eaaTSUP=1ct zLJA%sRbL84?^1Q{5@-@G*>A-u3*wj%E@1B&G2re={}$PMDvH=&%T10oy zwc2&|)4AC4ktgx#?5(E#uv^kx&Sm>p*Sv%3DD;%ga4fiAoF2$Wb-iK&niLL! zVEFL6eNp-eVxAk!W7ZB_Z_4(LYq48;3Z6d2tY5^lH8hKP4;#ll33ZQ;wXR>T-8$BOiMTgZ z=(DhD@jXUWQSXDT<2tO}Fl^^BCLk)G2ve>f^cz@?xZyM!!Tr-@^8J~Y9rQbYHg4-F zwH^e%#K+YiaFonD-C+3jJ-vyM+wcO5Uc-HHq~mvc95Z`zk|f#Kq`Fa~IAneoh}KZ{ zjl>P;=J%*6UVX7QPHh)iyMpImHbB*US((t~;J4ndhFJc6N6#?XK`YeA*M`f@LqxHK zctJ2YE2Waw!zVVG|I|Wq1k-L^*8M)WV#X8v^sCH18^sz|Icc}7tKJSBf1npTj=s(I zzNUxGYdg_XxQ4z`3u3dM;ghuNakQdM#Th1Irg6wV&7!5+baT1v*Mq{1*W#d-5)_#l z6B^#y#fxaTh%>%@-p=-4%BXlh1g|qzJ=ZkQBsU0&QdbdS{`_GZg~jWoiMu-ttGu-u zJBN5URE?(K8e>Y~o*I@Bsq)2nPMjmodgF&7UGhMzt>ll%yi*P08AXRForQ0J;}5q^ z^2%gh;f!(`)DKRS0CUv}7$@#P1LBut@qVb@N&^a4BMTRT;0o{jy$slpbn>;A?z{BCn%cuH9ImK&UV z_R(lA8HN!ecnwCMK>0K367f!+trM!Dx&i}YLABl}B~=}FJbWHRbuJpiKQJDSYUb|% z2R1C^eg?r#{^$etKrs5OBNfNzp_q_U_-~awyV50fj(VUTC2An-Brv6V~Hc_x!cIwLy@5v zkh(3vy!~bdWe$m$Bpw_e;_b(~5!@3J*XxJx#JgatiaxFCtQB0R6qCY%n~WkzroK#k?YA(Dg7fmd*KRq+r&) z1h$bR9hK61^=4(-h&fcMUS&Vlux~A8@dL2eFQtER&S-E;oEluYJ7IX+#cNsKef&7* zb)`Zg*RSdgWri{&7^Q$Qf_>k7_fVGugsr?KVR_t!UYOD5&uJZhMh)=x>Dl+$JKEg- z+$qp<*0C`A#oU1bVB{|Z-U`GGPw3Og*xNn+4%oQH?7>M7V`PQ`fltA6!85jer4WwU z_$ZFetl|829;9dIN)LCJnOOCbtUgoqhEh-Qp?C??S zxSrFy?~ur8V*XsclVmF<8-LFpp8fI{q%~$0t!=+R2Tz8;4Yzi~x_e{`4H`BIwg9cZ zxr*$pP~tSnfSu7Fqs4SC;_oFP)mH?b-E;wTZ+?Pow;XXFY85KjN|3D*NrvN!KNC}! zrTjMMo$>kU*mm|o?+N3d)Hu?w$FudXqzlgarE^OiJBPIhrrs2+ki>7^z%S3uj;}iV z>BJDQ%sw3(K2MZT4&~yX2iP-w0tZA*b_nK|rHYBOa@O&s-C92+Co%)#wWpebvCh$- zo?(@Er<*=mhFMmj z9>DI-bmUWzW)M2a38wlB5e8fz8X@>@cOLTyKjdZAk6UgqhA`U4x$5UDM*X@pVz#xN zoV51BoGa3c2v$T>5zEaon6lNS zpQxRD63WqxY}d@NYAaLVe}3r2^;ZwxG<$YG9jyt%xrd~a7EU~y+>?WLV zq`Qc(=q}!jTtuX$drM%tw|}BlFmMZSO=En#G3HpUQ-mVTb|-5@K%kiqW`coudkODA zM=Ys&Dt_3OciZe^gqMnZ9WtInq40)Kkv)*EMZK-rJo&&A-4&=#Y1hVJa}N%-frxZN zU__Ga&NI|ljHG4Rw7WK*XpFCo7%cA6FBRG#s}kD(X}vE%Kvi}9noTF7WOPhPkp^c&_s}P#5O6IiXQzRWQE{#nuQUu<_BYz!CSFAz)^Mq9?s<`= z(*Z)r$RESV^2W5GfcYi>g|bzkgyaoa$&9;k0Mv}Pb)}X7Kn$x^9+DHAVOYWYyrXj#VpdldcK%Ws*Mc-4w}mx~R*^|EYzxZfv$L zS12PJ=WI|TvY-NzeOqDG0~qn+2K85bNFR~O6#0uxfNS@dy>#$n5S}Vo+L?>)&5pA2*nO>`UWh@x;AzE zdVySAqbH~9ZOPMEb0}l4yC@0gK{$uQOkU6Ssm(Zyn4c5YqPd2ds8hiI)!i_L#xzC_ zKhHKHoJkbQRgCFp{>)07l+LdSm)1-ejK++4631$)GSk?_o0XvQL0_DQ{Q!_@j)j+R z=KfxHf4zcmp(8|31RRR#c<@9Um@;#DaQSHPGVaHosKTa)A1I)(`JRYe?L=Ig(vOoK zQA+Cmb|%|u9r3a|7R5J{zgPPZt@8?UFnh7R+Jdd4j;|Kyj_o9_Jl<{VLY=B!z7GDZ zL^6S-#2`#K6rv~wb6#OHu6gwniMhBida*@pj_m50?b!ENX>Elw6nj{|Dy*_GXH10~ zV~B6jFejdHDoGg|Qp<^~ZRre@pI0XzfLaQU9E=y`&4+*xwZf>5k2PdTiof>%gqx5_O1vhmpV)k_~(7^q4dKG2)r{#o2R}!7ynd8Acp@Rd0DDb z`x$2TujK-+Gp2^f4|a zysf7wGmqd^a}RU33)(h^t)AlP{b-|fyZrHLRQNIaY9s#w6E(0M7tfBD<+Qzap|+})v{j{Utj{y%>Wea}pwC2|h7G5^mCo5xZ5aSr=$Ptw0!Zq*$E1?34CrEG7K zmR|Z_msvHO0QiTG1E%u+aj-Xi<1&d0HQu|Jkb~lS*S*F;Xjl}jOAHwW^ z++6;;{70F;gIh%1Mll#iJeuG(V+km)KO+*3&!4deZ?HJ{)qxi}f-jw>FNL0<*jqE5 z_pa={dy2H5#xRWWNfbE>F8p|pNbUc}Bhx~>1+~{MOQ;*$+1jpA7yOMw=fYaYqhB|* zC#MEkSAeVSu}x2CM9;$(dFzQ!EB8mD$0=u2*9e-rI9bwLWBK)Ysb62kwW4`{vV8;f z`0>{;f2;-QMcXEk`|R;`IHqcG=4teTP?qH5%#!XRBHcVbq8ELIOy=bI0msgR;EI}J zR<%n%1N&t|L=Z$^Kb4gCmcWuman)tX4JL51U^WD7a0GCH5U-VQAJ|bN*}XhY0D~G{ ze25RY2Kb-MI^Z|0g2U}XL!=%v1(!VA$AG6>z|)2{&BcTG^Id495gB2vu$uLsiL__3UpjguNzMM}mhj1a^D9xGI+c+KBl z?GHkh9*Evy2=QCoKZu@eC)>RxzR0cK77~*}N!fjgcgb#c4BRw0%rqyj{BaljZZma3 z%bMTD*ILH_5-t859regn;CQ(2BRVSyfo$CbhZZ|#>iqE$m2NofM3oYBv@0XIg&hyx z#LeIXZf+_J(}Y_1mUL%kYvOu}>SRP28UrwM89^s#2oKU6z5eRfG4P|UT%N@?w)}w( zBELV3Bee9jP`*w9HZ zqV!;5WD&6b&z!-;Qc3}=)hMv!2}2A+l$M^5i7>+#1bA86Q%b^!L>m4k0a0rRjQ~a6 zM$&Ny57cekL3xm<(rW{ELf)?}4!Bhy3kJfMlgNcXpCKJUh%)3!;7xU1V%IAlttbPo z0g@2f-Q3EyOAG?N?eu*Fal45BKG`rj>K&;Qz=v8eJMwc#GxHoMrhXz+xTKc5dW-y! z*i8O+z#i@*tJzhb`eWDbz>N~ODyTocYwP#{4{6HRH}9W4IB@Un{9Vtb0*U0lRWK0G zGltgbAXw0ZC28Y^0ztbZ^7+E~75oO~kp}W#!ztK=4bi2ZNbR@pxCBI#yYs*p#T;Yp zn9L!lcgKNG;ON=~|NP4mD;j+$wG&b9a;WOdn3gEMKDl!4nSQl*2G}z3My#-&xxr z#krRlQF^yd#SE%c)yzPEXaNC*{wmRd@0mf|a>a}I&_#RQ{Jb@^5aBKf>EjOkyod1y zi+B%rV_(F3$5%hv5Z3f0&b+V$7RPT4p_ZMcRJMt!)QfHGwwXwqgyxjXb>J?dK<7A`wkHM9pC{A?=oAl?XXcf zaqQQ>S%n&UJO0BLfuJ_xZlJkbKXM%if({@EVKXYHbHw_s3it69S&i=_!hR_pu-H(L z9q)+ROq2}P4%>KdI*(9U%5$y9DmYq!M`Q4VNcEUsux51W>42D_Lfty{Vlpe{)>wvy zDHhta8GIVNkMLDRR9#YnjY5JN_B{p2fctZFt-#p>s${+o@Q}@lFoVzD;}GMQ55Ct8 zL)#|b$kRC|Y-PZV!8pe6=k{i!*a>Qej$exnrhWS9qLV>olPh8?G~8%{FcfK{A1xiZDbuG%mdblemVZZYPqm1O#w}2yO66WhU+itL%7gV7>K) zou5o=OOv>)95)%^T}B*{MApog#Yft&r8K4+yQEia3>{}!Le;<*tr?g6d5xw~&xGyK z%H7NfX~VOz_6O_!`f$;wzRxG9u_=nYgpx@0(H{L(WN_`aW_^2d7Ls~fy`DZ+^gp6h z5<`jH*K6TJwy-QFGb$7>h`OGLZ7ufPwwtuGX%%AFcDzQWuS9wMW(HO6h}#uDcI#Xz z*OVy_pf}YllQB-m>^u&sRc;og8IQUI=U&>i)x$fDZ1)=vt+L49IPH@k6uj?rf z3tHdx_MuFihYgc+nte9F0@gNTj$*@w#xVM^eRYTiuLMC2vkkv%2)_`FbLU}xz546{ zf%xLRhlClCqt$|E#7WTy*2zm`UzzL@1#P(6T*vHwcxNmreC|xm&RSYphXXSRg%5g0 znC$0I<(8@!1V-BI*g%|hPK!&vih@X#{T2Xh4YaHXb2SpG-}tj?F<_x!jPM(w{6B}< zPxrVksyTcGFNF7$;!~Rm)L&Og4Z5MT#arv6Py zK2hM4gpV+ougQk&$q^M)*@Rb zp@-Zdfcu2G?nR9Mg;u7qp~d=0-te0P%x%1-@S(;w%cgZ8L&B)ox!QYsbZbjSdlpUa z95e~OP@A9G_SYATg25W}eMkDfm?eO5QqF)VwSJtMZHS_#dMIyanD}ComSETcQj_*M zI0Fkj_$)4qFVqRoX=Q9Q{Tz3t@Y8OXVCjw{@rOk}?0T}=gGgh^BNvKMzo9F?x@MrO zEaIQD_l}n+XV@OHc<;1sBJ=z%007>Eg56BLWkfrWc1;Nm8yw zj_Bmr-|&Q3ra%O86MBo-RsuMS?{&4escpPLriwkBUQ95xgsWu00XfmTcLpq3?-x>9 zg9ztXZ?V!EgZ{YR*70Y{4Kt@4ZeVKZ=dZe9LsK={G{ zNQ}&!>fg4Qt(D05)!z!0B-@b%w>uQ7-iXE!jZqxCHK5!UFCSgW)wBLq$*vTu){I+u z$2uOPQ*@J8xx|DNCWBpr@47jLp{mXi!Y8=8QJuzFpKLpMpgqov%Z^{c=J{9Ky|_2g zdB(5LIU6Nccmvz!SJR-uRkJ2t<)ij2-ciz~_q5Ebz6RHBH16!fjfp-G*-;Wp<}+2G zyA)@6C|L#@TAwV1j;RqifJ0>t+2@AX~I(J*?<{;KRQ>@mrP3m+K z-r(pm`+=S;i6hI(jUay?mvmTjG{w`4ZO#kFI8^y^EgkEoTmpGW`cL8~#xt75pd}eI z(u5yuUtKQoBe|IG&+q3xsP_eXt@C+n+J)lXLu-xI7*y?zJCr% zyE8H$p6g^u{FSfzH-qta-^O?cnB7wt z$>#sWO;*&P^?xm%I~V;wTYpN_`j5+%1J&=Zb@Y#~RJoBxvbFY`%I~}JkG}}`;CE8D@1q>AzoORr)`rm-N=l{0^b+UqX#jeID;+u5>q zfaX76`>cSP^@XuoAWoftoD*^S-ifmh$kx9-fB6>0wgY)0%E;{HsyGUDdbMewGX$IO zcTnVLRJ?2vg?o_jaGsrqf><2nD@cxM7^N&o4527C@uJS(1ax+QPaTN-2kD^EHUxXs z!zfaIYvEbWEF7OBU;+8;W~~3bp8I&r2m&QWo&wt&wtsl+u2SC6%0)djmP;yUEba?pPn6pH4431&gKFIP^TLjqA71}ii;g*K2X)$A zxKMYHT{a}s>>UAQh7=iWX)mf<>n3?amGJ(+4o?BZMe~Oa8cNAW<(+oRIeE&Pf=MGe_ zR%?-0gKVr1dmm38ru4}z2dUTOSXeX+z6G7;KTE(mkz9zoUUn#w67r6m`k=ph8>?eO zZ3DT1*b)Z7-SSs@bsDQwBvyJ#56wB9B#E~&XwgDO-MqfZW<-u3DMFQiKo9Kc?tDn| zxr5ScSEecc^SZyK;A`1SF#X1RlvlNGsc~G_Hcxx-XJ{xd?=>QZq%}Dhr~L+xG3%EHt<>6og8u zLP^nOno=hyY-7@mBFXj%ogU)dga{&|uH$LIlT#|3K@oh$G+;EIIPv-c%flBs{itC- zMvph7Zooor;HisWZejVkazns)VzG4_qzLz_bJjCk87N!0bD=Y_=k>l1rokelK!W^f zG)d`Ryb!#CG=y(-=gi=;#NlVst($ry)qwuoz-vA*#Kr`2EeTrof){+ZG9v5nf&|5A z&b#dEqu{rw7npnv=vXb#Z;m4zOqiRx?svib7jpXR?6`H^*I*oEFT43pVSv!ZHK*|tYwbVB}c|qZpz7+X&9GM@?S<<&AKGDPs*d2n0 zh|?B7qV#P1TzzqFWw*UQ=>Rx=<7wkwi zBhM+CF`LFWwaSwj6&zv*-MC z)=G~Xe9FaJ^PpE=usH?~+t}gI|8m?V9!LGF@puzr6!2A;SVksn0J*B+>Uf1@`>Tb4 zsY)|Z>S~iUHNQy4?+~0xeFKhEVQZiQCYdc2etg?4ROa#-!a|o#?}if?eFWih7wGX1 zOtyMOS*JkH-jdfDHSU4ZmT^Q|X0%J=g8offWF*4VxrIg+-v- z$2EmLH~%~puh9hr8;$8`l$#^~Bc#D#{=FhMeW5w=S8_Y%1h9nE>?ZJc+*HimqEwUz)YF0L!q;UHqD+B?ccJoS2cM;TlXuB~}By(6opo*zf zaj_3-k7TckCzEx=L~ay5a{Mle-_q|Tu#1{qT*kacj%Cv{g9gUkd4a5{<*GUkpFeb}A`W8(gCt`@?o7%vB{kwQL5D)SwPR_vQAekH&d-!kFFSefHoZ%qp&_t8 z!!qgD@5&DTKy*Wu?8?19`9vK5u2TH#(t`B{Kvbvb;2id|oFWjchJ7~PEt=)jxDLbp zE2PFv-vgtWvzYkDK4jsEMLs!mY2c}Pw%hC`_~H|xs-!A!QRVU05}G5ta^my-2tF3b zexY(nEu!$zVjJ!*)wQ?_FVJxrHl%3AC9h-QVg+YpVd1s)h&G0n`QM9E*6G6f442v?rm^xFO`e?b7~x70%dD{l#cGuAcXcEmI7+PLjQfLmMrZ zo-BQr+xBV^`V#9DFJPgjD@ZHX!8M80wSQff|MS(Y^S-%P8w#0gn)3td1^K+F4;8S` zzQHt%n3kT;`xy09F>n^DG~KA@_q`&t>R-79Dsa3iP=;gQxGmor#*j~gDU+qzdrd>e z5=m#W*mFaTV^>ooy^&@oJd)~bFT$wBMkPhT> z{d)A)v3*Y3f$5f2HI@a{b&dgVrd*;$2{Gl#{{jjAt(Q6weG0=^5^Z4f$yIs3(^OnY z2e^fB9SUw*@lc>uXD$?^ksfDp>uGI1M>X)4_9dS2e-6*Gbk9`zeH&Hm!X(Bm>r#~3@jhgqv^|ZI(Jo?n!?rW zrbY9(8lThia(zPQ=VvD!{{D&ocE0rZ?lGQnnNl0z2|4$FZNo0Bo2-qoo2CIzr3_Zz zu7BT6aKiEw6UP^`G~_`?7w+ZY8}_K>}$4BIj}R9iygfqm_-yl)*bnnCEctS|-k zwYR!uAIIT5gulsgsKZ%K=t*SDo>^^C#~aU$l1dI|+Hd%4u005vbKE+Gbxt@V>$3d^ zTIQ>_eYgLm<1@n|^Vj-2rubJXT$uq}k&tSwCphcu?JY0tcwdS6KJ6jsIpAKs4Szm; iF)!~WsMO&i(|`Up3M0%OF=808k1(>bGqep9T^!B zAs|++%|MmJFQ}vA<@AymB=$#L{^+(n#*0EVxbCvT+1DTrjg-5QYr7_Gt@j@}shnae z0PaL7$g6ViO=u|9>F0S^R2Nd5B1zXlU-|pN?!<9%!3K!`28HUaZ$Erl_^$lYeh%5V z^OJ`{RsIsPxsvC{nTe-~yf3JqdpZVdXB(`{k@mFN|O$tx`1a2izpk3tf z7hHqpor|K^-`GBe+>v|2{Kg~gW0(BZA2f4j7QA03-@H}26mXt`LF`+}4C?!9vcaL* z3twIlt!52BCF*hcJ27UCI)~}rT@n!(1cKuA=tQCXRL|HCpYxB`1cZ;|C|0^J-F9k=-(jW5cR!us?AB)S`69&- zTOh_W%`i#Q{F#dH8;VQ&kc`u`8K*Ay^SM^lyVW>FJnG-JlPC%>nBA z%2j@GCgf>3?FwaX2IUGBRr4js3xYu!lhU$xfW)LxRJf`mA~0f64EZ-<7e@8?phds*m)q(nQd_5BboNV-o*n<8f=EP51fk z7v0cqia9o>aC2j&a^+zPI|*T0ptFUHPR23rO=hqkZ6^@><%njBvad!|R9{GjI0 z%85 z*2lu-;BM7q_}vkoYP2og@*`IXx1X-=ODX*NBjCO7|+S^DOmUtBF>EhM=!x8pzc zS=Fc^`+ZEWOzTcZwkfrR+*`VLJ?46hSWIXhg9hC|Zk|?NO?SBkPnBy`qebdmd0j|d zUqkoddyw5PtDBX>w*5j>0_|Q=p2|ZhHxw)H;`#BFy_M*&H;i`h@uD9~vxY5o=Ur;h z&~(z!NV8bWLE!djR;^^2`g7qaJrRvTzBSoedWyMvGk^2?^j7rNmuVxOTKw{Z+?n&{gZmLy6Z0cEki$r z#yZyWR_4j*>+ZX`a_SHzbQC(<^$0tyfxB z5{xiL>Sb4qh>hR|9iN(m4BtaPYF{y5Ex&9(ML%}ZA}RM^Xg_jKe6MWFf%fZ}Gax&< zg~GCNUT9`pvy}Dz44`kutL}b16;4L4*jJh(X0j-*LRca6T=!HkTpyLSB9D z>PxoE-si&KwcK)0D_qdhm(hPQLfR_dO4?eF7>;=V+SfvTPNAac3E20|hu+V#iuHGx zKKm*V)e~}&@lh{Ba)$U{B2+OcFXYeWE<0=Ni)=L6?XFa=&~=%_vq4HL4I2&*s5cpt zwI#aNyY}DQ^}g$1j^}FBTGrl%S(KF;o`xwH4InVa(D9KV!{gDA<$socV#$@`-hTZD zHLNBg)ic)fvvb=4)5l8SCiNyklx6mETbM)tL|7AH*sR7HomU1r3@(0$dDr?bU1U=j zLhhYIekQincd13r(YCg17jkBmIi4@F>_-8SkoTRwz} zNAF#?F0CmkvojWCxB-%%l}|rT+?q}`m2Dr1s*1uMn{8?ApC9=UELY(4V)|uFd0~xc zQ{A%J4&su5Xyv&J+3XQ^mr z33kt$UXO)Sj**?jSw1Dm(N^d@J63*5?yQjlBoZ}#C^xOtV?e5bX8+2Lo-l7>^?|I( zPPa5imqZiNO{67R4R%;|>t@1n3GrpJ5cVj`sA$Dn_0<{9z^4q#3a2LBX!+UOPs=Qy z39L1?X-Z?TT1qwk{54>zki}R1`l*8TGaP4||1qTKyl+AOJnoXglPt9~3PhF00_u6Z ziKRGW_~6@!d#%&c4j=a~S)ToxkhZ>Fk+;6iH}tY%_mk+{*r1{G&B2zA-~x`%QJf03rZ@u} zodQ1BfDZ-5>Cf*esDR&>fzR^{%Kx0D^3FK@pJPh$$%V2S&lD7Z-x{XQ=H~V;Rt~Ox zU7^5zPGPMzbzOCpUW=JJKzZMoIhdI9dO{sfl2Ax^iUEgEbJsTd)o=ogDa|# zS4S~EJ`WEMUJn6Y2WLyZ$D*R5e2@6~`1v0KS3Gp_vUh#s`Ox0w_W!-ff9~_#+{M({ z+R@e8!JgyfzHdw%+*~DZ-8y;DfBpNv?`iI7{ohZrclmd=fDH1ToZ)-S`-tzq?hPcB zIQdpg)!NhCPVc!j6eu3x8B!v`LK6SF{{K1i-%tD>DRuulc6Yz---X{$$ux5;5*6s|0s(8Th9Ob7AR;bY6-sosx&F;2-!*_3JMtt zh3B%Go~KsEs5~`a;M>+iKlX{gjJuV=$w?`^(rEHaYsHvIfctVS*|v}87U+I0xX1>5uB#_(KKVF&rXV{07_uq6YRC~4fb-SS@#RQ-IWW8tBKPB`M$pI-ntD}E-i{!8XxFFP0e=oDq+-Id_k!O_SE zi}AJmIQ5_W&LeT?OPusUamff=cfe<;qK$XZp#0rG=LAPHW3x1uqxFL+0GlqnM-UZp;wjnW*iVA|{(rJHLw; zm`JxplN-L>f-8~8^^P&Lsdc9Lb6=%-Q$3?K3C*cs=@ca=k=dtq`}O z55utJ_LwMUZRn2pezuo3x&Hws;INa2#ITjzSCNAES?oDqyz;%qk~o;tC+2bB*#6$i zRMV|(Lj1>+kJ9??i$j+ILO*-_+}?|D-H{t3ypjWL^_B**aa8QSecn4O*1KaKkU4aQ zYzTGHfyB_2@x1&r?@s@t{rT(5_0v(}&ZD7hzi*tJP|C%{MW*y(WvzdzHc|R^$JIsl zaPD@0t1^i82v}aI`t8Q7vs-<`%4mg?IcY7$!b?<8r(Lih-k6G@HYAmd=q;5hNRxKo zM7T142RoFd;l3;5i@m=e_(Q63bEOIu10#bGdtOGu7PT{DAynOn<=rx< zoUgY$DzZqo?wh3uym_iqFZw18bSp~u`$$68b@mE`wGJ`Ci1KnJWZx~<+A4RvB6>cl z)-0TD(C2+>mBYhggS$KP&+JQ{5s3|>il20!Ek|JDSK1aHLyPbp35bo*B^u^=+9U@> z5uQqaOW{cE)YhLneeU}tZX0ObF-MD*=Vp(HLCifR=K#l>Y?sFK7`e5F1<(?v1bj`+ zH6a^@pL&H%zwOmQpC70;`FeGY4fYI85lT-l4J$$TqK2n$YGC!d#sy%_iUKP?(&I^f zQU2UN>|B!It0G+%B`*=rj8v)v+|8P$4$11Ii5kU2|0HK!mO`%d`jR(8SB^(DkxNw| z#GB=v2XD|vj-})$9@%c*(0IDoz0@)3&g+dc1N*{jXs!5Kn_@{(?{~)wSt@h*Kxu7U z8t?LxBb@clFGw+YN09ji(r3@4c<0b{d2fkFL)_KQzH5kWX&4`awYag<6XE7=Jivvi z^#&oof+V!kt~HL%IKS`OYP4IUL=xI(cQ)9I38)F44Hj>YizMtuqe3xkq6}Z<=+B$BqFJ*Sv$vKSW;f31l}TzbLK^jyATwHjOyocxyF9m0=CJLB^#IDAkf~F zf*9MK_ex%gV$AG&3hXgHRn{I1K z88RSma#RzFRHkLHdZy3)K3Q8KgI$;g_h^^Gw|u_6JumgOC}M=&f3Nv0-HjCw)+de` zckkZCDe^U%(J=9p|A@JduR=@NjyMb!sQITqG(Zd|J2Y80dT(#w^IzlA$j9yM0mD~R zB;G_z9TYT@ckuC_FO$8Pstf(e2iiz?r~%R)){`oZZ93fOn!K&S+EV2wwECWoeHFd= zjfQWy+IrydD4dxO!3^>3uTg+PntVye$^6{Ypx;k4cU3taK75!EWQ6?@8s%Wd&Npmz z`$`Sfxe)*3*Sn)l1Dqu?Yr4%QYSq6J!TI@n*P96qsX( z)%Oqq*_};MX3W-NKa|+<4cftAOkpY1H7FD1e#X=v9B-FxxMo>zi%#53GKl75+BZ#e z)8JM)vMh!*m^+X*g6DP$XgYa}(uLMIQuvzozQyhwZet+^&Ktd_ZkY2PEd^Updloh< z2RC6|OQs4GXgO*HqDr2lt&cbE@(=0_wH>RhA*-Y06dD;f$< zW^LnN`>AVBCWh~n4Oh@O!44S zUBI(o|3Tl*1|u{6|7>|syh2_ZC~di@Dr=F#@Qx%N6qjh z?J}ES2T0C*&??ai8s^HEQ%@o$vn-Q0&4%}T1c*d|ozgF}Mq=M1H~sAo9OE>Z@Ax@? zvk`Cbg0Jzu@MS;fUU=-8snk%Nn>*UyYoit6!a}mGl4#i}(RpCPmdTntSTLF9wJwYY z?fcYDlKXp#>r3X{SM`T#q8!i_Yl!78Zk0oL8)mej2HKPj^P6fKeB`tk-`(^c1E!xA zDIg+=O;tCay*E4J(XZ}c;x)8tc~4tqTUys6tMd2qUiT(cF-T@O5TsfLf#?0+KvN!(V{i)Qg?9QcyOWAO9&^~ zQ4w%pZnOa_;*$?KjmB5oWk9l6)1z5Pq^b=U6@{Tb?LHd4dI)g z?Z{-I?MmtCDiGOXsC zM@3RxCtkrvy9Ny|A5pxpJ;(yO`uj(bX8a>;eZWR0A^jCKx^B* z+Qy|5>pAw?Vi#Bv>#=jU6CZ%}6)v4s}xEz7J5nk6}9cRTfID z(*C{{7RKezs++7*{Cw1Ckj@L2NqF;kY2U-0$NT@VCP#~km}g4m+HF2OHtIr-=}u%T zzmSIQXWOTr3OHtMsNe2OZ7*$G&L~wPP}IC9_ShS;VV!**Rple)h{y}ZaYv<5U8Bqw z-`Dym;FGbXR$60z1$^MqnI%n8-)I|5v!C8m?A9Hfc6H`*T0S2FJfZREVEJD3?zj(H z*#1z&RGY-*BDvlsEWEvhagT5zf88$I83;(OG|FZ%$L6hVS~huTWTBSwWXJQ5cKAH3 zkaD)sjHic=!=3<7p4eDq z!em>EqlW{`32Pv5-O6TJiB*Hl%>5MaUtMM3^o*5N(jkPEy^F^S9EPThc1;Vuijr-N zWEDX$J=8Zl(uC;`0k6y_zK{`eIIQyWWqEeAl-TgtvDDGtH2Emtm|UY(W;EtcB8a5x z6=SJ;5q7wrdCI9vA=T9NIXS_#CD$N)t#n{rPeR1E}5>J4W0^qP;I zfw~%~3k%$SL>CnCRMFiJH$@paN4XsXp41#TDzZ9QE#>vB9Z#LFR9;Ox$8wu`iRy(O zDsS2&_NA!~J8WX9Gy#deSGFg@df0h-X+$6fyI&Z@TFwOn7zQK2bV8LvgJaq>3!ItRBMmj*FQB!iM1=k%kCw~Z1pNUW%5QpJB^%iJIA#Tdhv~) z%+OIk$bzU3_DgmN+IF;~g1ubL0%|By>S)K(TNp&bVqC6xg0COcQ8V(UrwD&@HJqpa zm2AXB#wr4e*Uwi@bJUg7^&Tz|# zDvi7*v>`j$e#58Ra@xpmd&vD@F`sl{y~&Ss@F4Sc>hw&E3SzTY9NjB@?3@0eq%b>9 z>xPlw^i$7%a*>QfUtVvjIO*$VMf(j~G1N#g-|7}|Li$j3{Pu52gX+{7L2jpBH6-WQ z&%qWl*kWddO@oeoo)WL_=5gvR_m$y(hC9R1nz0g}-PLA!J%0}z zGM>CrsSG)S#(@dJO`pQ}wL-EIti`i1LfozRGG30NDHF_2ZkN8+zS}R0<%K+*)meTA zwBxqzD+9k9u#|co+Fi92$qG}j86n@=)JFwusZ&c~tQtF1hoo`zZ@9gwv7El+j!VB^ zYfBG%*m>2j6-=jeAGWdceL>?T)%*h@<&S4$nmA_UwpqyK<-I5MFSJN)=2Axsuj%a6 zarL#fZZTaS39I~BS{ve0W`e$RXOW#C%FrFF#gIyuZ+Y36gG`Z8#}GPun*&^)Qu5ic zHB|QYi=Zu2sT7B2Xlsj_x*a2ODyp6wvKP+UbPtv&Tt*{CF| zZGQr5Y3&rXhAN?r)nQAvy3bU{L7_%I#j;$k>^(d$)L#E~a~2CA!>ZTk(aamIZ~Hbw z#}!BXW0GGT=58VdNbGA zZltR)>8k9SQ}vkieUn@kG;d7%FlM-ONU|3W5{sKAhlqGyyx`QlNZKj0u0dXy3h^j2 zLDk7P9Yk3^->aiGIed?0Iq*kI?r!1Asb`j03JMdSDSiYOlLb>NgA0?G^vXmW#_f_W z>JLcTlHzoX7DL(;6L`0m4-i(rEP|g91jRSI1(b00qKqJm?H8;{10|D$Fh)F9MWAlZ( z%yg$zqOkR%h0kX97`X{-*hVzB9~oaltJFvuvf&LsE|J1_*V12*PH}$CB7kP}0!?c-S?sXl|Lj`zWVV9tjoMUAUl-y|X+9{{0 zzHk2$9!gg@ecFKftej2PqW#0csxdd%TxoG>`;mR>sO~G$`#6Gq0+c*hKw$Y)$HLBB zYg~rm(nc}7UELotA*~!*998eLu@h^pqj$Z05RD5GI`x6eERn^-N3>O?$a)~KA-PzO z48Odq4_UoYFC(~0Yy8-l%JOIeYg^s6Yc?Y|TVqtn9}D7uUtD2c)khFSx?tf3L)Gpp4_6hL zu>MVb2kG#!qU5eK?1<^K5@MT&WsSW9ok*_B-oZ$JFLQ*CxxXBjN-dv{;Us1bwl}CL z{uOqc+@8dM{Ob2uMpj7)c74$+!*1gxRGs{(4U)5Y7XOr(i$w}=E~}w&u@)jLIs%*F zH=CQ9iR|Ey(oNXdNmyL*bxUb?+g}sh+%2P31~$6UA?TwMkd<~e2uIXQQH4%ge2G(?t-bm2Cn zg!94ETLuGOwzRh9E82zXtwmbJbElD=?*j;Mtlem>6T)p-`}pp8kf#v2aY<9IymE%G zb~3ncDXU3aZa)}AbLjfB6GIp(PB{J+#Q(mf>%;vttJ7yMl;@}Wj|Aj674t1xC0W50 zWKB~Ink}gRz&gM(AG~J`=d^|Evg(Lgd7)!V{?17@LitIIgFgk!;2Y(=7RqkNtCcTo z>k0!6>Umy*uAuiFOCdhOuC#qAOPn2OojPlixLIbihsE|-zX68c?Xmi%k)~erm+h5v z1(mKk4&xR*ql1m5wrb)0Uu^3mo^4B4Keq{&(vd4Gg$2Laf6)5Ims8LAUDZ(a{9m?|u`U135`;LvimJY!@&q zw7U@&3^1ksHb@DfWdECMviA#a{1|9(Q-AAj1bo$7utfVN-iK3UyUc0V0O}Df&)siTE5SbFnUB6|_SGS9lV}#18L)ZrLk>o_rfk3+O&Dm?t_BNz#Y`qPsKbtdotZqJ){oUEpDhp zPatm^cmB8P_tcp%xs2|!-mS>*<{Zk7&6%f11q+kTCIri<(Eu39WET7gmYHmm{8>84 zoE3q~$u$_&oEUm2#iJ-0Ae7LU4VT+q!BzP=);LY4YZbYV~ zGyTOq0wSq8=Dv-veFI0K-T9ewzn?n(Q^Y~O=kDCOL%+}y${zZ6XXpj5jGV)#7?-Di zy)j1sB?}9S+16b2$LznlV?kMr|3pXM?=z^LNum91G5qI5NBX~9ukpi6&Vs-1qW?CW zRI4e^c!y(>t8V@_%KXoXeWsT$UvgxNOoVCv=d0hlRt%XkHa1*Aj_%|NptK>^UNWQXZ!=23fuEdx%k^G{nrT&cFNh#IMby$ z-#sVJzt`Q&iID|VpJD2izqieOdZ6yuJ+FuTwIEiwPGJ_tv^G+h|LetmR~#ML6hY$> z-<+=g^<~Ss0oT+^r@%$|uLTB_0+25`IR_~UQHH<10;f&T#%$M>g@-n-y`g{2+l)Lw z;d`bwHBPnP3Y5Xo>;4<`$Mb@1;I>%An8(Cek}y<1BNcmEVP3b|>JxOVekeWrML3f- z+-8VUQYEKzu*`ZOPwzwodTq{OaulwFhns#2d8uvp?03Z^LjjCY8A9IYw^k39WoYRC zQ7}H-TgPO})3H;q3E5~4ni~18zwnuj((|vmr>RD|zXKJ}IT7uuzr4(XmW;QI(?q}xE zw4e7-x16JEU8_j5BAmCrqqK2HzZgKE@#uOFqCM*P$R{4!Ki&Ya9rr;sdGmcw^1(Ky zSci{Qz#;;s59jK#q~0+KAaA>4^ehQ8&Lh_OIACNZM2d`eAc$S@532wUG=eSWaGt~? zJsg$?@>SwD-!-X8Gd`DI%!yg~smKR^6~|@ne?20on-= zQEayCzH323_M=bme*5%)sPC|7ps`K4JhVDb`S?61XHhFWJI*rQ4|A|fbO11}07!)H z$z)(RSo&zs)Mu@kioEW&jDhd%d9n)IB_6X#c$PN;VX0nl7!5OTbvR$suy(1THO!y} zF&!;EZZlZ$Cg5m49{^sx_f*pCPG&i22z_6-gdVSv1fWf%tcn8XSS(ZCoY1=mA$vef zbP|9a;{fJN0GiD<^de0EMknt@I?B-1Z0yFlhbK&oPQ5tVZE0}bPs3R7?%h^^I?sfp zOZx0=p@yU+c|x*W&Pq2UtWa=BP8I=ZUmIUqex;WgHI586J(-=#`Y4luO%X(S$#bbN-&ebY z@*juB?RWw}+P`e~rb|!U@ZWpGUhezr9dikw$&RWDHS$MNJ;oP95S^4w1O%Qxx{laH z^8Kj80;iMQQk`>e97aHsx0kVKOVYSgI~!m)SZVA)?YrKdzv9cG1J%|pr0kTfBdsE# z;|*T!KkgjiHFBi-HiXA_ol$$Qh$M>R)k96DcMT%UrAJ2{7{d#Jl@ zsLjkX=5h~r@D4}iF)C7`iXTM^6lF0!t99+DA%+%^nvws$xyHvDTWsmhs0*6FFf$}7 z=DuRpIiuxuOx_KbxH^J>8i>!vY8>vHrlyS$ueV4ayhnpQ#%A+vq*iOEO`ZgU|4`K# zwdXlMh08cNA-Srlf(*}DU7L`<;q`sop9HlXF3+Ufuvwa3`<#-pBJ$vhV8aT~6Z~T+ zn`Qfx;9_k7tl#}^-_h?+YEJs;HQg53@BYIb$*-s-80lK=y!R`V6|GKx`T-@HYHmYBz?q472vV9|nZHF;SVDGx!~ zqKtotiex@fKUc}-{%OH5_T#K_!;g6T6tL_4nb)G!f<>54-sR!@?d0KKkW(OIBI?+m z1i-(;w)ymcaVnqVvD?eV7ClKgU<_Tr4h?Dcj3GUCaVdb&B|x?+vV^^3)vx9~siI|U zp;n-7vxzy@eWC9r-sK(U(}1m#N=f0Te3kT7blHO##Nq3uFeWy3CVxu1e19Q3dc068De5LtYB_Ms9sf``w zm3pOeElfQYgl2i%)H2iuW9p0?MRKyP*I+nE#uV-59d={ zqE4(h@7>iC3v*NBhV6)HTw5e7{*(L-Yi2{oAjrJPR_$nmHlj2y6|=9k3keH#l`+aQQKEwDURQORRS=ykXzL&V7<#B5xgIrr?QHxvW&lH zRd1(cNyb6~zU1LPxf|8v!Akay_a9d13Hf(vkM?0XppJ7&W#DbyPX>w&g3U|d5^g>k>uxV%fiW9^&v z=l+Q+0er?)BvFd>vB_IH4lZ#WSwoPvTM1<$uuOp;^x|1KW@1Ri#3U<{O&sHYO!i=Z ztNCaHn5uT9fvKmSPSBQjuYmzR9Yl3w4VOnPV@qubl4Uf+_>i}Nf!WA=@h-~3eou4+ zZks?5>;{4bw8S0VP)|Gp`UUSkM5EMm{Wbb}A#G=xJy7mB(Y}4+x-BA87wt`fX^3C? zCtO1Y95id?X0h5$^M{CybBFR(BDX-$IQv}FYMz~u>I&a0J!nR0WNCkek0Dt5yz`KQ zQu-mlE))4p*!{ND*eA6fX)f20CCO%2Qf4B#u63@Q_p=h$J@Equw06K&4Qwy zdGF7(QazvMljP6axTTI#ux-3}Lt=Q`Zx@Vu52Y`m-9dLbm^0bJCX5>=fJu8qMVf5p zcd%6fR50ER3!bl16QXsqTI|mbP4(NsWhQ4e*^f8Wkq1 zJ()@<7m{9yHk>%lmK$eTdON>HMWYW2N;UGM(A5{Eg@2NONCF2;@=&T9o*7_wocSmB zH>`h%d4Z*Wt&gT&=_YRuj&2+140q&+Y`XGLF{TEVekx_W>vIYwQoQmSt3HcGBIjHPMU6#&@*b^6B zMG2PSt{VIx2f(q{?ZSX<3Wm04xK*zkxsvs3>>CH7F`TOtDAW+l06W*4x_o-zSNq%? znY*mPq$`1McfgqLnjojZ6$2JrQm5YRav5p|@1nmeA~J6d`BN*AIMr(I9wo&vzVEKr zb34Y{;FW$SuekL6C?y%}OKjNZ#z21i5M$Hw{5*rPYYAb?;nzdz9wj8Y=E#^ z51KUgSs;qK%ciA)w`v_p9s;+Ia<(SwqD-HFA;HK*^>DFylERAHwW1 z9e-3dTY*rJmY|=jf7n4TMbXG8(qjk0)mJ^3v{QQ`S14y>Gd%Xzel?|85%z&$=TJV# zx|*B?)j^cKqA!RDLrh zm_pncY_7b&zw==5ps-;QRBPcS0Q46>!pXV}Drxf8fG~8Vx;rd5Vh}LM?DD_DFePlj zz-Jxyi(^{rlnm%eoRT--9t;Ea3jZ>-QBF?Iu!f4{?1T*gV!ktnNO9>EJ^J`B!4S>P zzV4_*ynaS0T4JN)AJU9Pc3zG4dl=>(=33p7yeA6~?KkdCh5Jh@d9djV4#D%{;v-Xp zX5Bp1==T8SvE5F!d@Yn1Ge80+zaC_e3Z!gYZ$*9aa`;6ImxAXEX>1ssl3%@i3M>FdhPV?EioKA}D&9j{I$!ecV)T*J z2V9AT5k^Xc5}6{(j{=sx)xi3*ej7}QwMM@3Hh$|-pH%0emKw+AkPc+q_Sk>Dl{v3_ z{A6Miq^-!KJdvV)Btp#&TKafT$f6}Av*Oy$LVrumn~sk+{eiJRNgRkKftY-EKvpI3 zQFDy};Y0($fEYO7;4Nb?RjBTS`=$latdYSKmLN>dWwf@6ImS^epPA2WnR+$je zowVJR`^sC*`XIZItOvP_-L!ea7ZFsbxS?X9PbWfzeUKK?RjUwlPm10b&|7CiXAWu( zoTdjF*3(Cm?aexVF-`FH)&} zMab0pd4hQ!>cx;6OaV2DrZn3-^!&m#T&xsNopV&8q4>o0hbS2!L9ZeM`MnjjGCaB)P9Ypy zM%7({7CPKdfi=Jee=N`w7HP-KDHJrW^q?5Jw<1zonyV>;)y5_U|M_KQ^q4udG(@6b zE6GW+okYjjU85C-UDQ~sJi zCeHPs@lPMh;%1k1?vZ+h_+k=Lm482=oRs7WJ?A=5Ca=A5+V*g(*f7`s$ znFTQP`nchU@wnu9{69HC=IizSmiIGk%K)qJ_muCXEoPLQkL80j&7Ac{DXnN=N}0*D zSBx*b4O;vCvig4|dwnh^H#e?ZhFB7p4gL00&3x)s|BnUAYp1W#{Pk@=%X|Q5dw8l( zRJx`v*a}ldyt8xMJNKuaA(Np=9or#8|ZUt-2zJQP*!Gd~9g z6fuK6KkyiQ2d*q3ZuyzCDMMdk-|%14GMxcH|10ebkC~Nm#A9<)y;|X|$IO)1u3TgN zYbJwb^!4?BoZsTrKdvDQIymap@@_q*qrAp??e^c&zCxa$eMM#9xjg*Bilo49hvTca z_ZeqA{}8kS(Dw5!dY#MH4zQJ(!fvy~yP1X6PR!I+5mqItfA0G9oCM0dxBA2-Dj?RV zi(6%ZXRdOC&L{}|0fhi{CGImH_@kf{pd^QPx*%Foz*cR4y;ZLCm@Ge0= zxjugUsM>J4(H6l|@k`%zHe01}6JQhCewu0pL@taJO*q!_aT z+alU7{`IMxA3uFk^F7^Ir4wDX9xfLv=W(EUhVq*DwP$~=l4KcQU*C1wt&1uv1|toN zDwk*;y$`#ZGVllF0(57F4?xHCUcaz)RKNBtxynNS6NlsHpWGRbb1I(ytz!zFXXuh~ zcs0Vr`f1PG+BuI8sVZZtU>f$fVaNIsV0>}B|A2b<$jd4i2EOi^&gSvzUhg=+>VeD*$*JK{O>`BnHrKw%!Q!|Gk-J%+N3Z zOOx|?*{wC=qaW@UZDr_ETiIDh#0^$j^jxzA7(rSR%4Ue`Bxc-nwezPt0F<5`AfD<< z^B^dH_g>NY;l1@~0>GiGt9r${09MP51Y{AkErR9okciXN^Ap-(GE&3g(OK*P1eU%6 zz=Alf0Tuf&-#K)8`Yb&T3}AFU#Oz$;fzyelaD22YO_~g$TLHqlA3eXk#zmwrX5Ob= zIzOg;8N$tpt#g^bo-F0N=ZFM$WVnp-{k-VasB(KK4iA0jmU?UKiVeO%EqkR`oVe99 zxcLT%V2|KwqRj~kPishYn|mroX`sw@GY^8xfZUJ^Aad|8n87n7e;~ZY13Kq(yHTsV z?!u->&9p5LO}n?yimbS1{d;Fx-||)HZ=zOJ{WJXh{9!6`#PaX{xvg>j3>rb7rP^Fu z=5je-9s<$g>};PeyMWu0CJERq;Q~faHitkbzXnNt-EL)u3fhL#06TQ6I>O4<iv<0=1?&Uc zpSrX_ezUraaGehRIrU09tT3=Gu}xoruT#bcP%nfupVbNtWzkkkDd=czKcF`pj>*IT z!k~rBB$43t9RQ=y6IM6p)A-Mz92C19pII8R zN~>heY9zQ5^7v$E5%m6AoT1`&2(J@D>_(OMg3_{A-?OT>e_}0grlm z3sHl>yx7!TitZm_gB^>>t1>uhdJ!9UedlW1kOwReLnM;G)jm)4>WdVydR1p$CqBCQ z1Rfx@;E!p{w1u&p7QDDMdTw>oYS&h9I%8waG6*5>6L01bArQ>xbTI(=?T%OI(ggZf>(+b(JdfT-^GVJ>D+@}a zt>`_VA1pj8=baY{Y446&qx@gnx;j?|s=b}C)C8NT|0Z{jObTrUkKO2n)Pn>uf^)B& zae(wX*+M48S-2SCcH7q5RH;=DZllHi(FahBe&u;;*>X}HMLfD71r2re5=(U24q#X$ zQeFj`@@6~N)w~BvwwRK|{FmApFHT7?f&}&9HnYd&r5Txd6k2P~gW99m#19D6AGuVo z>qm*sT}94(R`mV2aROG_BW_5J01oOhev3pzkz|7T4;T)Is#B8U^@5N94?o!95?9pF z=H@6`Q3pjatPHw?5@J*D(STxs#!c)SAW_)RHw125H08xqocMo!-6Kgp)3^idXQ=R7a)ME@27cwv!o04>3UF&w%G)L-5jaFHlIhl zFbK5siuh|bSD{BlIZXsniadYGqY75N)JoJ)5g~_|J{L#yts-7tODE#Z#Y}$zx*Z7|82aL`vIja(27s>pJ2PkMSPVfKHwKE z8)TH)UyWCTBEF?t+*Njq63C z#8;mjA7U@iouK9uOT?l##rgzn189RJL#lO}J`_;((-B5V4KVAwq8oq(0ZfE68kERv z{=CE9o(;AXJXhWC9ksN7^lO0dpT7lAb{zd(Jy;CxkYnk+=N6Hxyz{?COBp#dD;qMz zj-2uM&++O%*{lQ3U+U8xNaS985s-t8&88k*yKI#^# z-_Ekos$IQga#iCg29+d@NUpSrQ` zDG72vgvFS;kL&6apPiN0BY<09tXq-gknLa0Ih;B9g|fX!QT7R3w)raPavY;}owGTr zY@|57w)P21X`U`@B%@oEfW+G)_l?2H!qDJoj|0Dso1uH%+U7@GoUjZ_D>Y$bz)w;l zE&d>(6senpKHz?iRCAbl#r8`8Lp&4Lkl6~I|%w9a0`DWve&?elb^$lRyrloi7uS}Sl6k7USA zamy;-$6hPp(2u;Id8sN`;igi<*fX)ks5;2pcAvw`o9ZYnxN%?O)!H&?Z_P46`MwyQ z)xs~QMeWAQDN^1q4Kf)+GYH~dQ4FTSP4bN0v4&X>nM;LnJejQXb*siQbc;CJhyxX) zk|3?e+aiGK^MIFdyxvoDoO{S4TRNS(7y_4F+Bdck*zJ9C7JX;e_lnC(0B$PHHO)!; z0qOI&a?8mvBdF;_?o-774BI`yvD=D?2>iy4u3kGaR|+AhLy!21-J!?bwc0`62YbBg zRcU*|@)~a~5OqZEgFUq;>t;+U`PYTE>(WLIoMe|=P1asdMg!H$zRp;ye6}04&ZOks z4n*DCuz|C?Z9F{6d%i-r-#<>4JZqPHo)V9zbApPsYs98I z6v@*b2Gsf0#ytTfDZ4!z>Q1haB;b+H?36_n4Frr-9!$o6n?nV1c5n7P0+4wR^z0%R zta77DWLqWAdUlP6**_Ley<6fVBN_O!>98ijYrzMO-x6RfaFN6(?Msf01R`N@NL z1jjwn*R6%Z>Cp2XGIg$T`l^Gk6@GW-j6dY4z5fJYj%S124eY#UcxxeJ0C(Q&gqoUa z27O5j=4;oP-kNJU5efF;X#cDzr=#$?wV4c>bnvl!sit| z*KPA3>pkK=Q=Jd)Fxey`T_2e1PE(@)1Cn%4HNsT=e)ji8&E!z{rJStZT z4IO(Gcj6{zz$ua&=9Pc)1Y>=MH!YUXo|4I!Y)pGnV~o4{?~o_CvPJ_m^DV4D)b zCEWY%5%?Kf_(CCjPBJ~as;F1OKfQUsD1JlW?r4(Yk+hpBw;vG3aZ6%AvGxt|2oN)xDi8N(CKdkd}$Ne5J;*^QB?URUA?xr_||_-1hmSQ5liAE!4FSw5=H#K%Jb&tL< z$1LL$v%Z0Cx%IG=*xXYLnaaPfbe;K)o)S(`dSHvJ4TRK+q|AhRM!bSC!1?^ zY2`JC33aUTT(^)}I*P=;Ej5{ELn1{=fvD@GE_ktnSBCnw5y?Zs1XXR~C~i z=`hob?FAqYjP)Ky*)<_#ONIQVGAnD?evAb@3$xE>pMER{k~a`NibN?x5z8N6WQlIN zSgsGx#Pm+5_CU(mql}&}v8OtSxlycDQX=Y*OGCT+4R+7*;utk_?tKhF9t}xdM)>W` zQ>_&&`ZT*)2PP38%Rh5G&|Ipy<9cqbVer^&%tNq;NVVIdZ9Vp$XZ{4T>-?v8=ZL>B z&IDJdAJsgtJo+DA#s4rb0nd94a@Xzd+pB9+u)$DzeigIA75t^+ClyHeBp~P9_89zzgUP_~czN^*wlP%X|CQy^Jl2w8=ssmF6};`NZ@acB*Z48dkT zBPD|sBPd|kwGC#nr5Q!zONG*X#^VUo?s%=JeEzg2KL5E>*I?6>}kK0XL~BAuHU-(TzD z=EV|GlYc29Lj{kmZ!F6!g+14#l(>30nzAtUZ!!6WW6v(Qpf9`J9Bi@r`HB)C!pprj)ou5V#CD^f zQ+{nLS*DDuEf?oVN-;#l-g_)^WvoU5zof#pYV`!Kk1V=HmZ-ItvNzHucbSKp7>kq( zE9X3=K)BdyqJlkxiy(3@T^Q~cZ9M+vAYfNpRNdeOBmHMMlL!TD@?)^d_@2atajbr7 z3;jsmrMwK%Ab6em;JasL;-)ofWhGSpkOjVw48iW~R)_wU(qO=eAm1KWwj@Uq3?k~g$x_Ext)Ek%Q9muN=-qS9fZI`=;&wZI+uRBE)n!$ zVwWw8sei8{e`7SLA*K8DG&sG-r4l{0th^nb{uF1K7>0&fUO#mRfwJkK&^ zEKR1N`*QAB077tgOwMRbM}ya5G=;YFFNA~8m+QF9>qnnV>3%NgPWkJaF}&^p{%-BT zFbrz5yc9O`gskJeNQ=0H+QFF)Gj&aR{7)CDe;{%iEs+kzW?}=njI$j<(Wy_Ei<<1% zO95B!#{Uuj`~he8T>+@;iUxtEvECgbUGu1h0Ga6Da$^LHKt>$Eoc3>0iQt7;iDcEA z=8R_gKjDe9taAGQa7zAFSJ#;4Aa~R5=KkqP{Y%i0Yu^u+2Td#y8R3q-dqequ`%6vD zgeFqQ%l{^p3S||TS5*A%zc2<_9K>0`f={$~Q{O3N%XJXUGL9h!gQ5};4e-yF#SU#9 z7R&Gc(kOmaYauDgKw^=3YaHv5GQPQc6j$yL2Ek9~`hUide>$2h_y#cTzxEiS64|oE zm1MGBQ;<;@CEj%^GUPfa$JtWS%jct}o>aC>vf{kBqemExSHhc{sqyZFtu9^OoreF@ z5ih$^_{WciX$$T|1GNSQbU~xJ`1(0p#kMuebiWb*P|5XUwnwvB&C4q&>`dFWR3=&W*4z z?3WV$3=#R~XH-Vsj|2V+{uj5FWQY5r_g+wLd7)eI@kH?0oGt0!EY3t-v>~)Efs)FI7a$WUK0Yd}*yGk&8?Dm5S~XvI;EuGY zWo&P1fUP*fHSj2m%NU2cn_jM}%R%mbx!du_AssIA`0?X!m_I1GgyYx^cxlc6NcB3r z0fI3Eryb_uKH@{;BVPC?y*Wzs$99WwSh~$cKQu~mj_Z@`@0J7+HNt0-l1zvH(0J+1 z#X0eOl@cwBF!##8wUC)(kz|$G2^-S}i)C13hHP&A6@6Ob+j5^Dprw{%)YJD#>XYZnX&%4N@V>c{Jh7&!?R2w zlv^E(OddK?sCId^V_Ej;IL++-<#@-Sio_sarj#NWyUb59mj7IHbyaYIxr|6E#i{SO zZY|U0t&@Wn&hDwS*{TU~yUK^`?mHKOaBNBOrc$dzndzRRR?nC-^}~g%je~5jK&AHu zxA*LX=mju>Y=RvAMDkV1qq#+0NDZ}7WUW*SpVHkXcMAfQpdgp&XZ*p>?&`MV1gA%^ zMe8~+Ap;Cl_8Bm}`+1PK-X9o$w;cf+)JrN>R@Mvesk*2eW!X$#Y+nu6<8sWDOmFXe zFe*6bT=1Y?i%CEkWOvH8`rD(B4C zbt*2>F`)9CFH*1PJV@E3KSw9cqjX^ZZl&?K%+-?n(~xwB&PB3!e=v~wKc%9k{xZ8` zm$6}VP(mCZ^*z)7wcBcj1?sh>MlRpgvCrvxucw6jgS}YqEFhQ$AfR{Pt)#|B`l z24n5a&zgh8Y-@{M!|o2{s8vB(V1_{}x3Jr{9a_6-^-lj9x80$RI#zl8w7j^G-Sa{a zX1j4UZ=!*s{jt+B^{^=^vV=woM%sOLgQh{sJo$GJe5Zeu{ zM+{2uFZLIh_lJE0p?<~b2A}6w4t+){WzrG*^;^>|ic0Fo&?>zVZ}zng8GhUeYik`d zrN!PvVfq;sJ+F(7xJMIRcaa4e^;`;CS@p%lx#nBNUaN&~_wgqaju6Z`eyiqCN?XTR zyzMKOpEgV}*4@CK+9vyDYkEH>q*uvLXqr#o|MtOS$5!+=u{&mK1VmJ*}op3-ImsB zDeLQf#ka#hU)aG>)t4bODJkiM%hgNkFttmK6ZUXi$$aJ9U`hg+`Z8LeJoB|Xm)Z2D z{zcjgBrRLXXC*xroB4uuBOjQIcH|<0x6t|8l6J3PU*)oIQ<`Y$PYH~FK6T02V83j` zf4%TG_Sb6hjp88AwUbIpKML*RX;avgkRDqR5b%(?8j&fQSUPo>hGdtwrW3 zqaF*>(b3<=m?T3;{E5dWg1^bI4CX~EI-FjWXO18X%Iw`js=hAdpVvGaQ|)da)!qv( z9LKvDL^wJ3=BQ+8|Ft;(*AXFo$HBAn?>u+^>4TrmV=8X6-?8YbtDS<4$%K>Ewz`|i zgc4M@5op{qc*Xj~u-wZsHr6Dr7dH=S5Vi60K5rvd1yl$SjC<|_dUIp&1KQ1ouwO17 zaD);i)dPQkDG>aZ11Y?OU>h(|to=!{1FRUhEQ^s-Jaktc$vMEqgxL^-jv86`GO97)FcCpBS+eJbhxO za*^_Ymy)h6Uc{zCB1ai!ispjT)6)$cUw0?)!u8w2IuEMp?|Oq|xRJ`mB(vj0t+v+I z_s2GeMxZ;{l6I$#cafTQ1hx4I{9 z7(a8gC48f3nW?sXa5v(yg{kkh(rUj9n?{@*41~z|pz5Q1cO8z_Haitc?v>R&R*|;pfL`B?rbsm*zW0 zhvq{d1oLnWM}`TddUCKb)B&}cS}o57rEx4OfK!;SMcVtcgaAOXH9F9P` ze3Unj_UZw5>`I7A=m~hzvWwG2;eEiqY1rBpPiv1{{6$N9tXp-=It(<$21(msByL_Y zN_2?wINhp4C1qx09Gjb4pf&Sr0IO~lpv@%RUKuUPtn22}re#DR5wvLQkCx}r{Q0rx z-jK?~@N%U=E~R>CI4_|wy-!R`Z$ zQWF?*~*wE=dOD>m3mQoE0$4jbdAeiGUBKPLTL4mCXe(urY z&D^7%lc-)t{5o0p_yuLVLzJ@F7`HO2TvYj6;lA?Lb*G8RVf7=)ZupAqRBa7wxpa$3 zGUG+fh}9hBuaC0+0V4UxC8ac3bB}h2IDFeEXqTZUuZ<$>4G-Prs*NGKFa`yEH}5D%K#6MW{ryq;PXDLP4g@Jjv$` z&VyRO^!<=RfN5{`L&dPv+i>+B@rcsVYDv$jV(;54H;S=vNe*Ye{YW+i%z#PUQg5;4 zBp+I|jiwT>Iua`?dEjt30r`Ms&7N}I$rF6R@0UYNlZ!*R1dDu1Wd-*l1OcfBb>7q+ zc>8=qi70Cl;gIVE^8&j31s$f)li2_^pw~$>1QhV6N%x&Dp_CIa0?|oXt01{*M|2X9 zab3G!fYTTs0X367fvS&2iDf1`O#0r}N$WMAq%6Ur#RX%W>*&KSg0zTk@crs9?pz8vZ%TSMFdtuh^AmudV zo&^YD4c@cILgyFOwG%JF3Vv5yn4b zJ`UCYTd8*AijLLL6uxS^o6g7>T#NZ=bQsp3IIP;fslz;zsPggph zmWF@pL|IH02{NA_eu4FS1+(;eD%|}E83*>R&aZLLYCyDXJ6x$J9XU4{9y*JY(6tSpW6*hKiS!x8M=9Il+m!Qbmav+{UA zGDO*`QRi8Mc8^^(BUZfO52>owPCu#yAtK8t_h| zt05p)^QytL*M&!&4(R$dU`D#0({VakYNDVM?Gq229N{KsY6er?K!>9YQ#Y_%>@ItY zFEoO7vQa)#zNG4{E-jXqCr?pu^gR+Q7rLNWs=Nk91`0J)v{D{iw_4oc-U|fq3_>Rd zU?A^^S_p262b0GtlpUQ50Ecu0SMI7QFPBOGUv|fal?g5jv1pBe6^Ue@su~H(Cx);8?K{l41X_*YI1pfRCM@BvM*gl-= zxrt(0?KBbP+nJO`$|Z3jIt}f2q}d(^ z(#NrvtT-6jPdbKY9d5@j!H1Rm&gv-!Kpg63_+%dcRyYKQKIP3%zpPP<%xaq`IMpnC zQquAduH&Cd-c&HuQW)O2-l3wX{MC%gr z=qqK2U{VVLj7Vg{ba6|Oasp-i_Q~sFMACsGF#k%Yx>2oynEG# zu}zg7@Le}ZCgzutRXw;YE!&-vSOa*^gezwVm6fvXw9&wdD@|Io2s?%yVwqMWrV)zm zIn0cC-o|&2-_iW|{@XZaJ;a&ifc&w-BqQB9QHDC~25<65D4lsqC8-l@o@Z-w zQo7MHw)Dg`35~yykrTndBKqlFF2F|pM= zJu9mR58`Yw*Cx&qL=TzVG7J6$VwE+2Uv@F-}$O$Z&{Ie;%ZHgU;7J!>VI zIpx`#uyy}7JVOfj^LMP{RN>A|#41v1RhU}XPfM)!qp{*Q5UBjYGmbkIn13xp;~qN= zjKXa&&wJwxT%1Uxs{hmvo%J`4$1$W@eEm_n2{-B7U*z9scq%+{yc#G7m?K zNAa)N`${9(EoiPuHwZE1!r7%U3v=QZb9R^`bEbWB!WnB=+;(H>R|I9HPKGD5wZ(Rn1;6)Z zsz@p~Sup%!8sX4Clz?2H9{nDuz0Xs)p`tdoDu~}Z5KW>P>giu9gQ9zX@)3r_h`i;jzBtCf{zChsnhu{g@UeT*f zKp#NKzgDUc>cQ{4In#>3&fqs{M$(+v*T2Cq;qobIA-9uS1sz~=IsrtA{8%=2`qpbR8Tv8A==@m6V^0svtgOm>EM-F@@WGRBW+T8DzbR*MP3OlNp3zlpNv`pQ4&=IZI<%xYt zm_WBF3FU5ZY*sYX(dUy*781nc{hrBgPFB?urEa+*N+%wYdqFTsEBhfP*X;Cgu`V+| zLc{c=ay&cXSWj_+lXii^Ns>tVP^dlG^S-~bDlHN{JF;Fs*l4c{!2nrdX9Oqlg z7~wF2h`l6=#~b5>lG*AjQCT2UG_^F69J=B=NhOB&Y|fm0qib54?7{gB=24E44Y%)YBi3tCf-?42Qa3A#v@lBqL>rmI;p{AW zc_VqK@8&K({(;jSs9Z6dzux9~Hm%W}Gi#R#SbfbkUkeD1x~{M*Ptn^_(pCFgdqR z1EyEB=62@#953Er8aM=U<-SBMBFsIP;E!P)+y<|=ss%sIp5*9Skwwhb)_k(A)@_4rabtt>i)G*z__Z9d)~8k`?h?xJEFxW+t8qDgyGjkC35+g_|w z_K9zb4fr`~(=Q*s1}&{YCiILnoDM3Rnmw`T7)pq8b1$|ad_1|GH=Cds8meq;}h+IHmhSLsHM)aaWWC7uSEL^vYU! zRvFcB*UY4z&4x5d3cl(Dq;65L}!S%U?FfLlUs;h%T7y}TlS3|$jblp^E1l7 z-&@2DJ0d!|$0a4YJ>+(Og^j^@ZmRSsn-;h1@fJ!P)?ZdCk%#mO7iSqC@m)aG8mEyY zLU&H+bD7Nsi&+P4A-?wccEtR?F)qTaYP&-hb95&Mt z>Rnpfc)0srwRBiMNV~Qc%gtepLULCPRx6gX=)pDrcIBYfgbbtGrh9LnzQ%Pwkge8M zU;znlxs5`+FcQMs8;!_+RKZ#-!=>dX<4|Fd_lwT!&4&QAt<*KPLAO=ru~Gj^Be z?@?p=RThh*J?p_`B;#o>Pd4QH1@Sb``gsmMXuWRS-MCu@w)#kt$mU?C1i2w%`|>fv z9_U&m+d1d>Bf8Qgk)C|Pw|3Pv?Y-&I{=-wQ!c)$}m%PmGU4~Su4lD8P*ktu&ocPi; zm41;8D2|@W4#=O$B>djg@arBC(YU6lsQC1ynwr|&mCqJWgOZL9$13*r>Y%UNPPISN z`_Fz3S`QMMB(<~`tBcunoH$;goK4)&w%p|#_3=P-k)GV(XCp~;YBHC9Ti|tu^1nPAmE%&Jdq!Q;`EFV8`nhhc2gNh*W))Wn`T>Wq+5o1(nh zQO3-ocBs6{d85JRYaBQH*mJ`B7d`v$vqR7=dIkoQ%KZ2eir~BU#+t|GD8u)LC#GG) z5>s{T%G$;ys-*`nq{q76xI~B4C=0SVGR6~4%bfAl#vhwV#q6s|C7g>Xm3L0&LJ%IG zB$VWw+JwI3aJ)O=HW_EuR3jFs-76D@x(KQm*IrsI75mlc@?S0F?S3n3rL1!Pa?O)t z80i<1DW0UW(^Pj8qmmU-=(+>jJxU3flSBlGlf^a^5!p=`uzNSoGp^cJkSXx&`|+Xk zY}Fp~x4bB#8O?Fe1H0~-dJb$ntDigsa^N0ZiC;f<3FDE=r1`nwNX$q>@M%? z-dlYICq74mQ-;Iyqg(h*#69O6hEcL+`_KjbrV!fNn_gsYB2tO$1~Sk&g!IjsiFxur zHiZ=gl&OJ{E~OVyU!jb!vaodY&^|HMOkg(*a9$2@i?0dF$mdCOH=Wozrkt(L+z)Ew zq~&sOh)%xj{g}^b@lAsg@lRUgKXYMQE=7sc)nvo;tUSoz7#8OzZKWMq8-H%0c{E{T zv*mRV+~et7=0axjRo#GCxVa>GlEmv?iBjOvOw^2s+L+86yC}QiUr+xZE44^1P6XVs zNgR)ldFs@#4ml+qZ+Wq1rR`rrIZb`sQ{sGGM?XN-;G@MSr*hwg3OpStSN>*CTfM-O z-K3aV)oQ!d(TN+@-Rl2&>Yskss^b0W((G0U{| z?eAf|{y<7YH6*?TTVeO>@vr+Gi4*($jv@cvZGN#AJ(7QO&wKo@UVejSC!6Cr4J9n@ zOs-;6-xAg5U23)ztL&mlF20o%RwU$=R_|~aE@HxB9G3MK97bkl|*v-W((J3|Q)6c(n#(z!2 z&}4`O!BYA%inYj!nXf18C|eiCS958Rb}{QuYeFzpuc5d0L-FgprPML1Ei&{om801B z2x;Zt&crN{moHzI0?BUW5x2h*iFUzTTeY>v#4`!sCrbaYGW6cUWMpI;CDy`DcAlmrJRrsPMR4Be{R)o*4^53>Vra&CpwTX_0&}>5tD#FQ=_N zRqL(YD}J3%01;)s_TCd&c^St#-=5_-Vkx9wyRK07_p!jd zl5m}a=Z`RoU4J)_-avu~=)nSW0e0m$JT2>|d4)4$qY^sA#xF6tS-RQ(IGD6jfG6{a za0$x}^bJ2Ed#^_UqX#A+_)uB$w;OOu8)SZd{s{Pjvljb{`ogLnRo>4}5%GW|C?sh7 zsf}<%Q;Lf@{Bw@2@;ft|JL!{2j06y3rw{dB3n_IV?B%?*wcJ{^P}3Azk_e|>l2gUC zzBCD^B@hgab#ufO1@rx9i2U!4^d`AU&&nFyPckACVCnN_qP5{_b5}v)f>jDBV{9oX7P41T#Us}Ar82qOx^Oy*EN?9>18Pg_y!uAG}eSMCwu?mje*b~#k9yR?Y282HCGCC)Z&O zy6!osd)X=0w_p#~Ie&0tPV=`D%{!P)PF|iMog(!P6HBSf^))A>>iw>YU^y++hz#V( z$qoE&V$fDZ8TFvl-D{APRR+xm9$7%xD4+!B82^2y{a-b)TZIk*Pxi<0A zWL0ml_nL_)M)xhrFAy|~Qw@MXcD?36RZyEt@1q!riE}!M_AG#dKLDto4#fWnN@xIL zI%L4n6nC@it5Ak)@;wmNqXa_j*cO1JSbS&?xXuGXA)qE;eH+i3AAg-1RcIXG3M~x* zU~?0wD-=pHH_{2})lzuTb_!pd+I+nw@u(Dgk?Ge@{SZP!muTP5K#|IYzL|S6pU?`$ z9g0h2h1Z0g_%;ALPsYDm_#}?odZs?E#OTDmlfC6Dpi{_X1-729 z!pAL3x~y`sxyFH637MxOneYYhug>PBed5cRKxxfx(*K!f>nq{FA{SCMjVpHp{_Epo zD-*^n9|dh1Y2~fU_#unU)0E*xgHsMh+RN&(P{O6rG9y0jF>^l(dkK(N6$G48LQvpL zJKRaWNPv6>0ga-X0Hr{q%p_Fi;h-JC&yHEJMQ~6%X_F;&DFry z*3}=7g;5QIY(dg^^Wx`uAbvL6?Hed%!t3Bw^_4*KtGpyjWU)CKKcTvG3K;Ut>*EtC zqz%O&L{b>bZ*&T@3n@tY#D{!ftKNK^hgiwMUI1E#09>}}he&|&L`=F4NOOzzBni$r z)tzX$(onUy0vUvsCdeBU9d5M$*f~H4^6d=9uI#0TZktP{(dF`<3XB zTH{f9FV`crUQi*^rI4anBKrWy5bl{L^WonU;K`7RB>=^BP0{(#qBZpT^d;xKiwWWG zOO7#?#mlc_8sSTL$M482--7{8tG1K$hJC!zt2ae-3wYt1K$gQD5X)$3u? z^pcSTZ@_ z)6?Ee6Ap!^X+Jc`a_Ve*n&-$5bC38{tb8=--C2*daU|u`$ZEVx-}ju z1j;b-9L~IbD?+Unf|lL1Ya1+pd6NV|m^9>gU2B{KdojbaLy&Ipm8EBL z{30dnKyAhrp!|JxUKd2ppM%w2O@bx!kqh%Tb4q=V4KQXJV)E_3blygQpa&;+5M^r; zZ`Z}8f^%izN@Q_gkMmjGoy83s@q;cgnn^ zOI|MijL<%#kYyFP{rgluG4=)%VmyaT+NTHM@6pCcKSXEI1?A-L4c zCQ!{Sx|#31iOut)Pb_(PoJBjcYU~^wWJF)lq67r;0}6h)Xkpx%JR2f~wpB`HOy9c> zj8Bugqf>m03y>_$JLI<1o63+P>BiFnBGX0(4Vyn_RBeW9#Yoe;{OtM3^FBHz@8Wm$tv|`R`ziNE1*hy zH_bD*8SkqL%CPPO{U=6RoP?KX_+#H30(LE$d>CvLL_ET4r33QKtZH9Z&43j>gatnG zfR~^;{pPb8P`z4yJO&Cj$cP%Llo-Bb5FpK<7rm-m>{UeP+-3{hq?+yBXl{nNYPFal zM#kyujj|6SLa`l~dkmOv*wzGci!t(rBZl!7@^J1&94#`tr0;h3<$6X2K%j<8`*k|b zxW`DY(eDC^p~Uy-sZQb^*C*`Y%XV`k92Tm7a~TQUfKafek@ns{Lk7IUwMy6k0fLcO zJXd*0nS%&pcV`8=YTtqXC3R<*4MIPXb8%Dd=_ObqiR z^Ucy!>p~8R0NZWo2b8JD+(XgemRi6YjbrX&iUE;PaE|3p~W=Z@`|=9tcG-IP}^N zH24`dhNAgX?)(h;IdBY8k z1xB?j`wvnNIk{)rN+%kR?dc|B9xu#i#9OM@cr`Di2Y0ay23(&90-Og}Ti15O65Z#5 zxS>wrLFyrw9*C|?G#YUPlst~Iw8PQ6$0@`XTdq%JbVv4e%%^{60W{vQ$OQ)awPfbu zyqlK{^7Nat*Rl*iQRQXw^eU6FXQ7GtmDuK-_c+yh^l1r4(qRd>L_(bRI7BV)vCbU4 zrE=}sDNjSFTK(#T>D@>?bI;kYb7?+wH{NaQHm`AVqaYtVPa7jxaPdw07DPWJEHyT$ zvi&gMo=mDf%0?19?4zJ7h@Q^4H1fSIT*2upP%yfVVK=BC4h)V!x#0&gyz>!Y^CFdh z6S%6)@fICzL-b5fYNVN@Sd~E^5$5td^~BJ}!M;gS7hc;abVmhx;HRst-1;=b?6ps7 zpsyr%?!1Ed-$SSRkw8T0#^ z>%8!=_;g~f*%oDZ`Wms)oX85}H&wIm6MIY9{E*gKk}BHV6f!H&R|>*3GTDRg=)~sd z-bo{+_GttaY0c2P_EwAVIlGL&*4JH;%ahz1|NFZ2?~7Bb5k}Z5-*}fD59~JOl8tU z7-jSdKExP=l~3_uwIWsZf~c&;?>{>P`p&OW8kVumk7;6B56glW?5}nm=J-)Ks*W9i z#W4ZW6Tn>j45z-h?zkNUq*9!ICxN)D-6uN?eS-G^oVEoFwh0cYjnN?|nPS(flfX%x z;dVmMcpQwMt{~7OfXUky5{gN30QQDQ${Pvh(GE@{ulynstGAlTGWERqC!duF7e}Ma z2s?U**!2boRx<*Oi!1I#?;kBDClBP5jUEeB)Z=4~QwQ@rFh~fGm3+rLicm9T)qBdB z&*81**}C4v5#x7*j3hT7VOn^7Qm>@;cN;mWE>HKT97}$totjU3LmQP_PG23^E{NXe zSn)0r2iLDNB@yNB6$p2fSgW5XNCFjEA|#KckJZObzc8$59PGJS7J2H38X7tA%|gRh zf{2qYgll)AELBX=OYCZ{vfd^RQ7EKnHJ)k#=r0xE1`6d$z-oxcdSw1c zE)oiR;&!ENtTa`LzY6r)z+pAGWjL>h^L4OVqFgKLqKJ)q>|#2#yXhe70wW?B#{?I3 z2Vu|M?(i-Z=7?-~N~^c{@@HIX8IvD@=LoV8KhO!+mdFJMt!0V5(d#UN z6TvUdoows<`VfGA-YhKub@BL{B@b<71e4o4+oSqx=9DJxa=&<^*TMdqh5E-Opm!9j zy`uw{u6I1mZ2C4pV7on~aN%d<8o@=s9%esgu6H*pxJJwKcx&S=AS74G0=i4|jfC^D zhCPEBT9ALh(ceIXv5BOHyGAi6*uVF#!V0h+%&@bBUN+(U^*Vt@Zq%hxfK#fYtL|l9 z@un-wmGAGyRNm;dv4hms-y*{)^|BBH0|Vb72RlbyG`!)4GP_E;sp`0S*?8~Q_rf1< zsfot9eD5r2r7YHu=+7jv;4NGkSAL_{TJ)PE!@nM<{sI%UZdw*2tbE%m9A{BW%s?&S ze94gg)p5D+K$gs4n9$@x03ouD+dtq1|I1vI<;qfD8Y*e|50E(?T_HHia#wW|6BEBI zeEA6SH}zEtBJOHe*Aj2dqBBe$$15&Id&%KC`)cLi>3+3W{i_@IiDE#Fv;O}6##IC_ zKmC`htGC}C=ZAXimc6}Z-O1zloALXvzKYQT)u3ia=}3_gFW=%b`K{|KxHHgAL z{CXav`>Ts&iO743=Hz-v8Oh|Q(P#N1a0oe5aO@uj9tM^v=~92X>Qu6Zh>@Z#3y2?b9F4`D-k$-&S6%qh*t&@9e zUxR9H*Q-`6zIHI<$<)H)fs#}I30}h)Wb0*UUh1M5G2gsjN$HU+0l611z;|2u_wPzo zfCXrvv_}PIBaM+{h?)1QuNwNcj=mqLJ9k)G%3zYL^FHK({_#^AtEPZhaXT#6`=dsY z0e7Qpuzx!$&_jbt7OQoMrF#IO#`4GC74hc7yuORXnA88`uToKB`TP5KE_JnB)=T{T zYzMD;BhVB8_}9L3_WzfEjsCxBum2DK_X;L?S~&)l(XWxQf3GMxA^*FGk?i?`a|c!injN50Pl*`?bdI-_PKHCVg%%!M-%N{eDh?S4FgO zfM|qlsjGupFa6IA%!h6WyxVmDhlXGT4WSbfxc7TK$&ZG(nU$3V+JetKA@9#^kuG9m zQ{+8WEx@7t;>V)&yDwQ!0EsmrVnqR!`7Js3CBHuHQFG4i{7K#DKrEE0!Y|<<3q0gpN0im z9z?bT!);l{Au6s1=bYQg?%U6SOl5Gi^8+DWGq_8rZ*Vx|uLj0&7`#BEC8ERFNAf&V zz(XNzefr6i?l z(UCmhqY>?8E3~cC0B3om$Bom1X1lnj6AV1p z{Xo_eBNAW&RIbU9z&XL9toq9LQKE}kUXl1J5O8jRw7t(_AQG-(g-;)~sct8TocF6w zN!P=52f%?1$n9(aS&rmrBn*uw+5lTokL{0QZbbm2XhLJ92~KG23mq$#{SA#f0>tny zkl&iH>jEH{WzGTT9#^2lcZ(VX1zvLu9gI53DY)1D-zQJ+eNuWB7Ovj$e!p+v=JElw5SJxl zIiqy>Xd#|WD-`(G-VRm833<2*fXZM+;D?zeXha;RmEdXo!&i^@Y%8tax9=!u9D$s} zIS{592uvzMz|_Rq@j&=wv}YEGYh{Ij+eHzqDT>3eT8oAJXjt9Sr-1Ha3t@X^jNk)| ztK^~Gx3sa5pkUtER(ahS3v2ae<6uld?J=VxxPV4{a0LSDW;6q^1d8mC1DatADBhL0 zdH6|Pw7_Nc?Nw6nY#}j^OytP{2*L1PwZ@vM23<+*_3B6wngdZDd$I$SOaxoOEbv^( zSk|4+vkiP91o*ne)D;a~mOST?zy0{}00F=QJUIhz$w0x2XSn8*(*JeR>n(hWvp)nzLqQ75J_Xoc2)ZyLsTFbJMYY8Wb5Vqp|7fQk&RNl^?}xG>|(Q3hGek#7&KO zwGM*t#;1A3?Tj{n_sEp$8bi{wf--;`L>0LfwF5XBZ~QKS;NKx0P@~G{x^+9~$~$0` zTWH!e5pMV#z6?mPQtuqF>UTk<>@Hy&#S!x_eL1^>e2wTvGfph9&$-Z*6U( zEgB@>o>{(!bGekTx8;NUc&*E``MQfUs}+nlR2Kje(E~N~q`Z2Z>3!D>7q5At(3S|k zi~Y}uhavy9o(2(Rt&RDjuIKAjkfd09G``Jt4_Q3n9OjI-fUCJW$8EH_8yWHobhbO- zOsgz=yhmB)6QVh_-J9P54ssh`U{mC-t3lNoJhv8J3Fbn7nV{fnmZGl&qE?SJTPU4p zytBEfYkLMz#Q4p&U7Ki%<0_MyBPcALO-kd_kEWi;U6)dqMKzG4E{lnC;1^^qX1uLk zw;s5;z{-#~_u%tx!Y8){oUPpf53mL5;H7IfR>-{KuRNJ`T>{X$^ugWDHWn~NELZxj zvboxRI=|kvg9cFqS35kNWo2b1WM?pR*EOtl^t!l)rSSc1@xKB3M`k$Sy7ZT~jdTgt zkFC4DI1G+k)lr}kao76DsT9>{v*!ZXNIMF^AE0@U%siue{*wS9eGR6u^p}av-iLx8 zizxG1ijF#T&clK^hhJ&pPl4d!E0 zy0JR&I)rv5!(&6vI_`;3XV6^=Zz9MTh!T$CtpNx58Lls?up&b%IIygK0_L1i+;oXi zkNss|2VNme6a%{C9RfQ-XmAI=VFE-OHUWnZXNUa6({G|g5O~=dnUhrI^QNw>>C**n z6A#OJ=CVO~QCUoKTpUqrOWpq#1{h;|uU0v&m*HZF`4kA~w>zo&q=I6SHVO9q&zHX` zLf&DDVgFqV5{;KnguLZyaqTQbLc?_*?p))uz46tpwqyVo{|iGO3rB;D4_p&}Q!vKw zF?IzQ^Oe~q*R$6TAAN_O-!kqEAA%F*F#*|Idkui%l#KvDmLj?n;&u%va%-aR8JblZk{a0y-j(DztArP+24Kq70S!18FEW; zLFmfK_qzdm6?g(Ct0i{vt8V|b-1DVhdal)!SUe9KG=@~^Z`izS6~iZG)Qq^9b^pzm zUFO3k!V;n+dRoa~W*rEJX1(>xp^-*0Le$Na2b+&Cy_5QyXZ=#rI`GXfiH7;d{bYV& zjr;{O_&wD26Oq>bA}nYBxW`c@mTf|u$&{>dc9*j>-NOyWM)Lhi99-tbF}R^|3M6X! zABhCMf5Wm23w3yLDI)%=z*_Coq3~@k+&fhQcVy~gud-dMAo%*777JfVe^S=&i*G#p z1W#&*9lXQHvbqot6}DX6`xrNUb(mSNa^c+cL`o!*3_9`jQ}Ai<4silw)-z*!B|U50PD-~sNtKZ5-PpSGbc;XZ zZ-p<#b^6#u-ZT&si?8W?uu`D>X#9+%GKWOGg^*ZsJzfm4Mb|Iw%y{oXzI#Cp50_7) z>vVGGYf3i;(J5)$a5PRfx^ZDThK5;|-t}=VEQlBRs88LP&;b)i=63a?S5LrmUQx<- z$hDNCkes%9!(uSV)`0tX`=!&lMOeF~3x*_+ZG{&Y-YRN9%vz6K}Y;_`a zUK`fur9|Eiro}P*ImnOK!8-QhHZt)Lh_UPK$h6m`W7PPy-CqWNG^=Vs!`TL}#zqEK zu^>Xkuim?bG@$fJU?DBxbejCCiZt-_!Vg~Eu!=CR|3PPjvtH{*J4|6hZmmrJ5 z^?YAexfvH(g{zU9kk(vK1FM16E8QCC9zHN=INxTE(|R5+VaWQ6?dg{k=Vg@f1Yi>h z)-A43>v6+`*qcLEp`qTLu2Da)>dP11(o8~{xBW!Z=XyJym`~ibgJ!)UQ*mOULm!Y}5!|C%Fnmttfo%`muOAj2mbSH0%}`Whjl8E&J0NWzh8rw$ zN;;Gq(WVb09BoGLA2q}Cm$w(I;GPAyvg@PTx2H@eMy#f-~s)xrz- zQpwVfzP&y)>yywPHLKmec#7^$3}&^O&d-tzx8#h)sxg$2@I#G?KPJt?64nbecM&|0iHv+>Ct89pN)_MS!S85jZ#c48^v| zPECwy%4QRGdD8kX1Md@(JMo@YaD4Fn0rTWs4~w?;>L(jjd~3Vp4|uTXwja;<#5jf@ zbjLeH1j*RGrnfE?Xl?>Z>Y-)uRxz90Z)fJ`hlV!uVcKKw9i^0fwg&tjd!-jO=g-#}Ps~U|eRO@_2de!jZQT9QuOKocPW?>gVxB2A zLv!u9Kul9Bf%PelE0;BL2i)Ng4)Zy^3)1Ea)ZmSBm#hVhqsvP7laC-S17{TCNn5ga zH@ueCN|$$sZ)Zfa7+MGNm>g*Mc0*F;>jFpRa7P{6^|J5D3yi;XtY~&oDtj_Bu1t9!Dow55$VXsnkC9D^)U|Z& zcK~6Jd>;c#7|O|^72|NedK+37IXymDcR8DP?!5oN7=U$;=(z3PiL!r6IYZ9$PVy{K57;v$i7)Bmb`Da(Db6jn5%$= z_)9k{FY(Ak-I~ZF*~q5A^Rg?Evbsxhwug23H-xhdN>cSTDAS?R<1sHV_MY_>=W&OA z#&v}dr&b$3rn3)($KR~kVg{sy6sZt-@r=8u0JGMC5w&GE(>XK;AE2_vMcgbcxCo#;}88V!;LEjPH zZ8=kj(`@o3E*cADK&%ENuONFkM)Ovr8Wzj?1JF`TqQ5rT;2m269VT;$dv zd?vp%u~Bn++{(8=q(uO>Hl+cM=qe8v)5W2^GV?ZgW6|m6X|cHY){N!w=M1B@mhP@a z*_+o9?ACi7*_r#ZW$^lqA-7!0KHq_D?=7iRacekdbH|-a-TBk`m;wddeBK=10lIGc z;;<~GQRaI(a4Fl88M|~F(b=EyIxr~?*Z0S-gQmtFcbkw3KB^a~T&1>LJ!vW0zgC(E z)s)YiftgciWFCYXprUHj%gFUS?2N8J^GsZyR#oaBVEJ8TNtO#2u!=wnDkaIz(kjak z4MD(J@y+gj#;&lj?kJ|P!#(`-GL|8<#k@ylD6~SmnLFaH_&8Fx zynKS`s?T3?Tc)lpgv*WxBNHUDmq81O=-?;F^G?*oz3{)L&XLV0;>}@HC#tmVnpOr# z&RiU;BcAMZhCfx~b2QtWv#KB6`VaTjkEQx_kQ~OeG*oo06e`!4**5E}&i`-wz~<>R zpQB?YB1XQTej3SN7WChro{9RreM@+{BC8%9@fURV56HDo1UW}a@wD9V$De}#zsJNo z7I2-1hZeps{rBzuG~)mM-7^iG(-o}fdE`Ib^*^>R?xRWuaBbCTr^&y+>Aw+sED+yJ z1d!O5*D6?-7k`h$`oXjiqFjV++uBA(bLUEsuC*c<@}ES4!|x;p?^tr@d@Td(vFZXK<>?gupHZZ{Ah zcyAiI4B1Ip0T(21z%=#;V0hdN4-YpV0c}VOLFXm;eX)fvDHDJ{qtJ~7zk-FGfJRis z1Yn?~kEnXus0XiqhS%f?0A`qFESVF=1_A(_@D7p;rQ>pU3g^b?W^HrxaGV`T3XNkpjs6Zd}*nThw zrC~3C!`BO;GfA-oI~LU1i+^4jyHudGh4S;R;w3bAk6G;E94lDbG5?6`oeDO9-C?23 zdH`RM6S0-(gBBkbSfBa>yntQ|cnXCfsa70Bs(giQ?Ot023uAgZ&0T!HpX3;WhP^#e zQz1+~>!585mB~xfW;Ou-{lUE5zY3bsnt6bZSiOms?vlCduWxz1@jl)kT?JYV5FZ2I zus4@dxL$27)EA zjTGLf|C+r9nq#G&qTjs1^Q^-v;7KCr*3EV1+*J;g2whK6)$xASaR8f6*>@}n50txm z5T=8Tz2J%FUez(#rB78@-ZcPbA46%yD`#RX>)tYF_Z!#7glhp!u1PFT9tPmajVO=+ zn7{&hiGoKWJmM=iB%6RVP@HdNGfROiHuqp2bNqKJ!Q5@Sn(OgCm~eR3}l7W!33 zaT+~+c;}mLzG&m+L%^q_G?oLmi(CcpSeicjwLrvjr8}0p%y{QR((oAyt59Ekpk8O2 z&VJ{8kJ`_>pf%Owdg`#qSJ}&#Y6V1SSbuVo`T(jym*_+-L7`9z$gSLfq4!rYU&ZB=G!23 z4E<>h!FmY*$0tjI>x6$lNRD7x*Ke^k3&6lgK(jaXi*Y(eLA=-$(uFch{cHL9jEfFw z@y^oA1o%J>Umw`pY%s{+u;RVzmKHp|*fO&dx*DS;pbL4&=!D4uB1m~a7w)XQv8fAY zkT?ELJLvJXSt$?!Fo$P~uPZ(G>#)?M>SDoHrJI&>hgfTKlwaF_N7k}?Ga68}^B-6K z=OFt(yHY4W?y*ctQ!JClB~TfE9)}CY2+=N(IEDTgz$SgGvBnc@OA~;_OW^Bc972iZ zOJ=;&Uj`yjQMhF;!%k{VbhKaG223J9+j#*I<+J9?U`r$lL%a_6egI*y2o+}NizHL1-(I|3-L>9=e){Oo5MhD~e?UE?d_)Xt z?EMY=kW>N*vuS``%Gfkk(>6$}GMQk-95A3DA|g-?H?#kVtDaVJJj9nk8>wcGdUO;2 zJL&h&@y7aQe%p}7XsJ|oziOU=LC83xF`n5pU|`E$%ZF&8nIP)s^)l=g%JDd{totF} z2X{niLbcA$*y&yhP3s$T-o7rQP4N-XYfrdCt4Sl>FDOVL-Fe1sef5hvJ)Y*h=d!ZWt*4Cnf-T`PbzZa$ z4I8x=gjw+LK#T3oInel>km`NWVg*Wb+8$ zD(r{0R$rYHCYEZaC7Rm^^on<4Esp1^p4y=YyR`GR}_Slvx7#?f($>EtRz9XfdD-YkX%JMCPoMBiUnpQjx&%Xf5 z0`~a%8!60ILs!`&yOe9K+|O44+eAmPhVb9z^|s01{e^LMP-00;z^9>HSyJ-Zx|k$m zAkFuXahagz{pRB>sOlV3qy!(E-x&F1I!QGxX}GLj*^&vM36Hq5`79B{ggiCBO zps*dO2hR*&C0%L3knnGd-m4yJFO8)O`(T|`koe$=22HDJECdErY}|x0^q}9mt%Nn6 z$F)0#i_lL4Ba?})4jMS)^AxwpIOhGpTVWGW))NHM7TIp3j$ti%$=GzO9T4&PJSJ## z*-WF>*s|M+Tjvs~KwY}vG`55QboWW~uD~5c8BuAs$sOTC{cd& zrG~qkJxY_aK&Z@VC2}4ZF06>GY}E9IY1x`-=ES!bXVW-cG$O3@>fEqvH~*VW)!|m* zx!Idbjwlg>nwLCqZtezwRN++?LzeScGKrQ8&5zlr;cs)VF?$eW6OCkNqigcjbQ=5z5mFqfAG)JqR=s7`N1 z83hxR(a^1E6vSbv{m-PfSgyUp)(sFYUAlUN%|GFmLP7|}Of^yiF?*m_=I6js5UvP1 zotDd81D@R*KXP3GC+3zlNHoF3B@Sbh@jlQ?!>x7->eMO-KJRHNqVAgnk#}eYiSNV( zU!r^8u--uH2o@ab>oIoi^0c)M<#ahrtxOcKGh7&n$s8Pds&xl95R~sm5xf;2*!n_uOR;gv#{w8{;_)6-dB&>Qk~k&6;~XKBF<2E2mD)D)6Z; zW2@4zsm~Shdpi4f#63XS-FM);UN&aoW@EoMkJSye!)DC6+eN~BI|GLioewEKpW+{= zAIQsS=Exw;iHkK};iBZY)^B_f3*0br3%HcVEH`@1P~&^uYH*t=39D$SJr|erGa&Fk zfyCP5=OfYq>G|nQ>tCNDLB`IM{Hmy%gjI6WgWy zz(yHW)6q&#Gp%r{HNX!@KW%)FmmDw-j-yt%6%>^KO%{w2hcoCbQ-aQS1yDb?OD%U~ z1dE}L%t4oYdXQ|Jy*ZVhN_PV(7T%XsLU~+9H=zZb>-ha#BOJrhJu3JeUDE(|nanO_ zZCjmzEDZ2F(Y(=BOLcAa-2KP8S#h6K{1)@Y*ZX~E++uGpg4kR2_t*;8r4hJ1Zv&=SZX(%FNFRdSK&W_(7#+i@nixHk&iwj62j?lsecID{u6r=YW|r# zxOdBd0zB8RFOEmAsI6_4R6G`(e*Prxm5=Wg5bODhmGjA;_zX`#Ep=Dmh=fQk{6roT zq1vBJs&PtPt<;kq*pET~p$#>q3hLK-4PXZG*cG)`SPOlAJptga;y47C$;bwr9B^(b z#Quc7{QaZEizj{jSR-yjFVM#KS2}Qt82|e9-+xlv!G;}Y z+=z}jP!c~uGyk(|{P`m&H~#}p@N*Q0{+#3b+e=@r1ha7Eh!gJjw}1XA5>nI${vFZc z^TA*9nZaZ;cwjLggf-hitrgLUlN$FXQHiB_T=|QcsUr*1#1K&;P?ODuKs!_LO$Xe8qqrZF20jiq$ke9a|jSV zH%uyi9-Jok6aM|**w(-H93OdX>~Eb^OnSpC_P@D(e=~2u)HF2AfB(vwniju*s*|D(%vlb`qLrFkL55Su2ux)^BpYUZK zN$f>^W|&U+9+)q8!awm{3=A?`iNh-_23ygV819nZNv3@H*~dQtUjELOAHQJ|WPjqg*;3W<|Ljry+cM=Y zGqAJsccAWblmC=i`rC~Q2X9d2Pu~MUiQ{te2zexvO0J}j(t0`cJk|RCj>eyKg4a^DNp^58LRxsXjkgg);;;9D1dE48mv6nso$|VWuLCD zr@-cZbLIU}YXP6du^l~mkpCaM#T8Jm6L75p<8f~2XZpn7-=l-(&&^qm#KGX?brO&F zhbsVI=d|7a`|JJN8QZT3+|IN38J953^0+V0rg#S-l6=ZJI{ujKac{=THTr6wS8vI^krIsnDq01&@E zRslW_;CeG`U7xlWnfJ}R-m?9&oPtN``FB2Uw^)C$B^E|j;}Wr4#PZwSF_n^-ZNL_G z2f(;KT!?g06eKh!0Q48ABrizI2ZX5sz}9EgEO_r%_SYbV^S&eI@|-7N#haM}m6so-vLZPLP{t51{4YW0J8qqu(_-t+B?{+%Yq3&C-$QN z;BE-yto^Y95bnAgmPXgTxn;SfM4$w7)pr{~;Y z3p|$;Xu7rH;ScG8qHgml$0xEv$0;FVF}Ot#UaZPSKR~yhDmrz2dTa;i5p_V1gc9`F zcq_Wqf5)^X>Jrd$Q9vsI2?!?_+3n7QOB&#HDCI^%FXChKD25>6qtHz9MHE1RMlDgC z_w-f|_qP>LVCo~&w_yuk(^TF6sILG}e`-6#kbn@tQ>d@m5bv+#_ZA`;M84FhSM&s& zVwna@L3TfBxCjrc1~bg#4WMvyj*k6}ZiV_zsarPzHtcjUDRl%gzz*OSEY*JEOGAyDd>EBuaf8+N4YX^CL$}U(#X+CCN;uJOy zDSt0)NX`SKXzuC1v>J=?%e?0+YK*z$n6A-)OVb_9QZiJN4~e(95@B|?bRDK}QHWV7 zRn{LEFs})#aF*$%Y<%E->BO1>eRg9rKG~9MB)CAPo zFChxP2*m3L+JzCzO`Jtux)#xeFcJA(B zwe*F`;bLnQPbbh;5clHqQxK|CIQZh&kK#U}ZtIe%wwd|!-DxP0FEu9Np-DP-L!e^| zz^1pBl=q7iBm%zQBB)ry{ota7zss~-)EGvO0$IQLAMYNA(GAc>YEM)Qeb@pN*~!`# zrYwDuhu3+Yja3AbX{uP-%13kkc>PEvN`-)S?E`wKw7~QDa9fuqjDt4hUUd0I+O{vU zB*q^?%2#O&nUAr(i1xNH=o~2nt8gSpaW1w`--1YoRfD7gG*ZX;arEaUACQjR*I z+^81=8Q`OYp0mRTpvo!bu|Af?Y}9>S$ni_N)ON-aeRK^G+6(FF2oPph8+B;k5l|=? zzMQ@Tnis72rq+lxQ>Y=+6mEsh+U9~*5`(~?mhtq4(M)I{<#4!23Oig7JD2J4;b zJ6IyAX?+ubv)dnKHy3&j8PDEy3>^k3Lfspn?N!Y7$FS#@tN_}*t!txsdIyYka~``4 z42oOhh5JCc4%-fv6n^hn zP!yG(*kA_^6gLIk_!N$nPQ?|blqOa?^@$9+r!Q@P0J*c3k%rFlF4$c_ivRTbvG^4! zg7hEIHDlG18E}rczxWiMgPaSeFDlO{Mp+A^z>HU<72Mwj$=!bUYZ+p__bCz>kXQ3BiwRwd1nAj!#`ANd3I+>{@ zQ#ToCPc=?O$1==G-?s+MiMEgAWcw07tk?i?vXvTtzGhN+N{=V+)KFx>e3{K(J92^M zR|el~3?!Ax>v7b(*}NL1v)#O}*Iq-*C6ajXpV|A=(5D$`zEDjab&oTbgJz{sc%eIu z9QB+RdKk+ZN6!vYN!7`* z2xrWKsFNhFn~LkxtOCYPsoP5+aqrP@l0|);mjFR1$hN?wP~r5jI? zc>u3w0FWu3ffKMO2=px?s$2VDgMw#~`@9H+RpXr9*%Q~@)$`i@OfELPn|>EIgI3UO zn2))?2pShbZA%tN2LLa|T9dh0z;C2l+bOz6F^%iy9i_u$o~pDN5U4g>XER z>rulIICTzUDhNk!NKCzQaPh6U6`i`eoeQ@2f&&rY+l3lGRGU@O`t0pn2kiLHH3alKcf)P^a}82A{m77qAh*AE`F~hyHU<4kH?Pf@g6v#c8qx6-H!UM=Eva$RJ}Z|&XO^! zx*G5;vNvTcoROPSj8AERaQPQBaledIkk#8%^h$lJ@&q!|8XKwAN1Ko$q!34wuMj8D zsaFgc{$h^X1Ox-kLA(_?8Ye%}xeg=#*BSoz6Ifj*BVJ~eM^FK}x2^j=26-wu-ylfz zmqsH6%L$9tRsr_nGAHz2Em_$n2%3TuEC7{g+aQr_Mg_eo-4iZAA?w=&9DWk98Cdui z6~Cn`kp^sa?YQ)Pt+#XTtT=9e8fga7ul=c*bW20nfoEcx;x)w z)Scc0dxFEDyw7GsJzUE-lTPakB?Ft1=WU|fB_A(*34U7W^JOg42vDSmu!q&ij3uBK zrQUt$QS{KY=W_7w`0HDI1VbdT;iI56O*uot)ivE&KY!NF(2dRw*Au2JRZ2YAGWv(p zfM%qU;g^~Mrt$@la$cx#53jsWN>HL1^7OV`2&HH_)u&D8&wL@EbEJF2LjTd>zPn8s z2y%Y7|1b~Ij`s%L56z3W;bFF)+dcO-R`(0(Qjv#prhxR8fF&$#jVr6Pfh@jOypR*i z8%{#rLm*m;0-=eyX19`Iz)S6{07h&o`(QAM@I^*$W|{mnqr!u8xS!?pLF&77TPDA} zsB#GqbM&gT5>l{j)flzcKq=#rIKOSUE*;>LVkPvj}EAOpHFLHJAHiwRY%H2+dP(}3#aAY)_!G`SWch&r8Bet`f z(%qlxR)Id5YYBOpDQd7ApqkTb;-Qd<+-wwwh9ug%8Jt?YrPCO8Yr{JR`ygaqe{vR- z-d?PVu=X;~STxMC?Pe-Kk|AAgw_Cs@118V&4~?O@A6x>_^z-w;)pY1yO}FIHcxZVY z?f(8MCGv7RyUmv5|9GnZZRNSEq@f*Jjj*OH)~_<*;>`E%xS}7fzpKu$Z6i^pO-SeP zr@N;@>GG|BuS}29OGs~|?3+7B+1;gzHRTu*?Wy_LXM=YgW@OpDXqURpbVABsPZQjL zd@PylKjZVBipodPS6?WVfCyVHxRkt>Spg!-a$5#Y)$KrJV?_N|h!jBuiNz*LRltBC z4unWsx1rXz-|>SOG#Bs;M%=rayH!)&rEkxJjC+(+;31HE#DEvH1#3q0N74B`^IQhS z`65!|221@Dv9z5{h+27=9A-kIIn5i}!Up1Qb^)2F^T|WN z(-}qXl~CJ1FJZ8OjV9PPF*<_+%l&p~ep!+yQ6^a`l(Kz$09zRV;EWN;>DK0y)~ z@3s4|52=jC`Sx4{|cP`?qXyw zp5IO;DJwPfQ2RiF>qnOhTn}M@b(QC4@D>0`HdEk`aAl#iK1-8TErZf{cE`Oq{1e^) zm0}qLe^|S$yzxA)$wP4>`{ox|$2M1^z*GY?SNI)AXChwk`6}v$T{w$#rNm@$l@EDm{4y-5Wu+n zP8jPS!&ZER8_uluaE_Hm7_^%IjUg-2K?)z*n3f3=?^!)6du@LM!l!||zl5|2t8ivJ z@uep@V&4dosdT5+QifjG{v_ap?sjjfo-U=wVrDv`W-_# z1$m1#^tp47aen9P;$JT*&dszuZ4L^gIt5y~s~5WZA8ysVY84urCwGtCKgra5^63?G zKhs`0tnfv>^WZqx73o z5;g=rv>GRk_Kz)#_TKZ3R@!^vJiqt)sC?q$ZXoAa?^V*AKWWRuu9c*I0%K!is*Nkc zDkLv|%Zz;?`2_`+++;%8AOG5S`1ewD#??&WFX&gIy$5b5i|-+9{`Kb3o^h<8lScHE zLh3H*vyt?O3NKd*l->FuBe8DDv%H`k&rDX`GM;I>&v^ybm_cNL<=G6q9P6gVCZ%Yl zD}`@_yQiOgw-UJdE29g259V1~Wbvi=kK4~1w2rFS{a5x9OtpttO$=z)_@K(8WiA%6 zYb9O%f@UKU>_W?hl8GM%!qd3YSe{*SNo`JfGhi$9tA$NPi=Au<{xq8J^7zVBxUOre zf$Yrj%a0Sv<>ETDC}pP~)fbeAgqPBker`^oX%-*+XQKeip(&jC&c+(|Um@8qzODEU z-`LFT&W7qI_Q3q#+7TNX8pP+O%}zR`|J)2eUizZlNhk~ZEt|7i06VkHIC-vf)<6l2 z*~me&kP9`#qD6`b3Hrql{(2G;0Eyc(s+Aeem?UuZ13ks~UtcAov?6>=5v8334{axqV zwAw7US`2#b7ZtuZ=I^ZBcdsA9l>3PLlRkUeYM(vSCGTzYQB=@azH9N7Zr3mzW|}!L zqyh{1nqUx?GiLy`ri?J#??mzC`$)lycAE>nkhk9IfXvj`0yzP7Q_~TPa?r(56$At#vmq4xZ5F-hCatmj z#Xy68aBq9r#uAyCH&9@P0yRDnv;FUnR)7zyB`+^;&8L!~T6Bd~Q@uHY)uDLx%&#o{ zlur*NUHR)%4+)!v%qHiYZl}eqiWroMFz-5Ebk*(VJrObF)`91{nCYsd7fqvmDc+RRJZqBXQXvn8P>F`X8hqUOPr1T1Y(|0#JN_C$UOnTsddN3A& zp+LPDZG$_&ZDO9z$%&n>VHv2rmon_R`GNFx^Sf!rFvRSV@Ta2TA{ht7L2Khpqdq470?!7pJ1cji5?==yX66o%z252ds3pB6? zVtC9A-53mhZ9G#7RKmiD9lsG9I@SW0xMzaqo!8xAT~AS2kDc5%qu{FTuv14Oo__JW z09|)#v_NFLNbWmMaP3d@TyU@f( zd&@z$EM+z`zjweAN=`5B>AoivGRWzmItbNkgZoGoHcGB+J|V*7E0_&%(GTgCtDHty zM4FmUA0FKb$g)|#ezfNuUs|(ep;up5;-S^sT3A2 z#tIkWDfrTkHJQ5;L^Fdntu?y69CumO-$5fixa;AW@#YBlSM&2jo&z79$%N7Ij9YWx zvY_>RMbIH%73Vp0SULX+juvlW?69a__wekfoQ6eO`2t+HbIV(*s7RO9qCdY^(0N{} z8zbYAlgi`LUbQ_-f~fW%&X>|TbgUb@X1b1qv3h$ro1xI}IALMWRBK0ECTug@R4^I- zGK)j}X;CKA{XzRsQ>pD|V|e2}cUy}P&Hmdt_lK}3aRi%=1+PU4*eYuWj0#Xa;qcHF=@VAv^L4{?<0z+ufpU2WxKw5QA?si{M#A83+9I2jx75$-Nm71{#r0 z1*4`}xt@A4mzg(?ge*T-j6;%cxO9?gmmpo@8V-7&r$%7Jjs>6(mhf1$3Xreb+7Jm# z9P}LtqOPeyA{7qqpu3@I{V~FY4 ziuRpttY??|zYFnywMY!onRt1AmX?uU;v6pRBmHgb&A5)TAuAYMbaPz<(MFI!Ev-v%X?EpKar3eMY|v<`E}2$26RK=@94^ecg_wlr z-ZXqUGUJrdZ!Nmu&OSxQR<&MiV;JczGEAQLI`)l$yvPN37HQ>r6pDZC(-v|x+&vkQ zm@|^$H$3}&MOZuDNw)nA60`KgRQ}*-A+XK4I0G%seSdRpy17E}S`%|fKOc0kABgB? z0p%rRk)@gz+$x&ff!4Lj_X}g5T5{sFj@BUDc3eWM$RZs4euBxPK*WPq7K>8ChwNq8 z<*}d`Q4UGESxb}wHOpp4vSh^^0+gdgcfIFe8CUJQbI%<}igD&$6Z0sUze-THrW+v+ zFbP5p)=TPf-yg7CVNus_vH{vxY|tB1XMg1>GH~DsijW~XGM;WY=$4Y0o46NeK)K^O z){98_imf$8T`Ew_F4oq?-khjgDc+GKp>?kX$7c#wfB)k;<*jX`6Xsh=zCiCn15$Y=xifDt4@!2QTxC<$aan3q6$#L(^(9@3~Z4Cn# ziRnA6Rqxa$-V&u7*pJjXijVIbCzXtUGnwh>uQ0xwN#n#P9#f z8$AipnSFq&_b*L}m)ifbD&uh_=R&;M;!$+fvs4;-;KyTSJ9YiW$tvDX>mf|mMMv#w<$1fEw{Z{{(dtZT6V_2gis+F zop9H?lDs%P0*)>#lCwP39H3VJs6)kzYg#Q${$?!K57;x|u=-rcL} z+Z4jFT&qz*7JViC5zq02m)jsKHPe^r?J0|nirGU@cZ#j_x#QB7 z-tz7h9hvMMHl>$6CJBgqutj@gyYHH5w|2wkwzm#H<82rB-X=5>5sq>T3`(qGv&zrP zxieFm+xCrZUq5DDtS!IxY;e(%^~dkj@>cP#re=s?g4&Rhs@A35)?mAj>jQ1*S9%QOtrKxtdNo?69X8l>xA_9O%IeJI@e{bPnOU3NF`^!Ait zojXvMStpOs{r$VMKgthDjiA0n8hV;^Pn0cD59P=ZEKFk$N2NkoGq%@+E?90ze>w!| zwoKrV>b?ngr9cA32@{$QlGj}fwOUT>Z+oq%@@V=X z)4TaP-wznQv~on+N}LH=glzO?!T4SRLRewERDMXX*Dp~udMa69)Ki{*Irw}!Xq9|h zb^mN#|I5u~DQ1yEnKWpv1zQ%2170JU&pNa4KDDAcZX{xVN|0Q4=b(>n`bg)v(NVb1 z&xVrABziaZ#}YrY<^EK}j0uC=IE zG!k!wj^wr=Gk&$a_|Xsl{Mz{9rQ^y(bcIWZ!}vR$K&SohS@FV6u)QO6?Y#3hcO_`z z1*b)i*-dxl$g-_2Vh2orT75K~36Fhp=X${{ovWKj`Hcux`|WY-puYI-BAuHV)A#4U zc+QH1pi5R(jwwnzBIniA;9_is_lS!}wmi?m^A(1-0`wBCas=AFvkrBTbWfBKLju|VcX5sT(QnmP+lZCE$cP+=3Z%`i4GwY?V??1xU^HVeM1jhl1zWoR#n%QPcP zh|YA%BrPAa%{1+@-pyN(DaW;`tG`_H8ka)vDEYf?)aiuQuvFQqTUs9Xt7o8Jejp_~ z)9WWLUs%T$xa!b`%Kq`fyTrB;{W3}|#0rDg87r1(F1K_%tSR)w@g3>Qqf#{ottH5( zZ27U$<9ZlhU&Y-oCUWTBayP?#3J>e~wWDwTqV{JHB{rm%cFWJu6W?+OEa&u&}{V*=Bmxk>@+l!MyB4I~Z{+#tY%yrd2E&Bkx+3vwk_j%_wK3vkLAu zMU38_fb7@#=~lWYM(}<6eEj3(55C&rg820t=I~ge_M_odt^O_*LDg1*=00zs+7`LY zogu#ul#^?b+siXcltO#GkNLV_#38npI|-QV4Cmf+6hhGG*4Wj)7!<+dS+BSfn&v$L zYOR2+5y1Vcuy47-XBaunSLa{SeT823vIbxa3LET(5S8t5Y)BhNM$+#UxuTYlL)e(v z|A&TZch;z9Hfgd<_vKeRK+>}{%mkz`4`K=c9S}kMIP;n)2{&n$W|74RY7;QAnAV-l zr;9Q+VYDkuiyJ@)dQQ}GGjsu&6+0TCepJ$qRh8D+1`Q3%ZCUKplA}#XT#D#&*6@*E zmFTPFCb9(@yVY?yehKI@EJLOkR2mPN$v`^isNMgBj-^x2+qgjn@jXHg*^yvP9wmZeINHs4TmU6)R{iRig_mX=1+*#^AC+D_h z4C!Is)a!Y{(3%;e+p(=q6VWvah>+AG*V?l3rr8aCI#g$yB+Q2XlflI|cR1z>sGX}5 zgzGU)!>aRLw;V>v=YvS^P8+br((fjlN^+lh4AN#@6OY{-@2;e7Rs40El832m89rL;oZULu4m3|BJ`+$e@awI=VKH(w* z4sNR!)Xkt%=9G~?V7hH97dt~^xor&mBWdH$DPIVuzssy((w39?U?7n#You;Oys5WNK`X65W!T@m|Du=h0lxRI*WSf?~m*iDZ2JgP$`Siw?12hLu^InAfPmjL5}aB*<~#*j}_nz(Jt*^3D3Q$E_bJ>;bf($nY5W$?J8#i=4sZELd; zj)*5I6WA_?Gc6gm2QKIIzk5#wZvG8lkytgCJ#Yey6o&uh0`uGHsi}))r3|M8$$w=Q z{|33m%}Kv#mvTwim`l?CG79?7t9?Kn8Xq6jp>_GWpP^vY zTu~lk;KZCZ_U_f-C9xEg2pH=g254-(A~Zq9d%k53kVxY;?&3mCf8&I;=Y*R_VmkV* zN7GD3Fa(M6UeJ8e?Mad5vI>`)1YhsCvS&q3r8b4b?$xHwjiG`RG%E`@te6Hfx_;)< z@Z8$6zEShn^^~rbQT65S6uJ!0?^@Z zu>)pF!E`X;=0bl02K|X`eegWqVh{cIWzV_&Qm=YW(FIhbyvK#D-pp`5n>~9}ro0O3 ztAh^Ws2La)&V`&4&En$o>NAkHDsxb!w70{B^{*)N2~|95ER_~xqnQ^!ENxmfO|YN` zKE>$d8X2KyMbz0x;qMMdu?TUi<=`fQw|IRt-jqE^$1uynZ1A)@Pw@*xZ}F1?)&A%F zsr~r$Z#9F?Aqs3r5ud!3lu9O*bQ6tjf^;lY8yCM@Z!V6%(9+tJ-KPEqw|-J9KT!5= zG-mQUN@g_9q98IeuTP4#Fd%MjzcUY-CDXm&yscKS=%aU~|JJNyTbKkq;Hw9EmKzh1 zNWO#N-!LC*Le)eFj0?nb4e=WKV5aU*n+A7hu(1zV%;M>+SB@?&0Q4W8WdYpFgV{=< z0XjBy6DWo9di@IVZy769Xn43vIb}sy{8iM{(WvB}bN;$~g=JOiVuKnTN})RfRV}H7 zvZdb=+ica&7%HgDYS6UoPGc($h<9~=PWpVm=&Q-=Lz|E1ql=q#MV@iHoAlfkKA2zS zJ9wY%Jf1YqXV@l9VdpI6j`Xil^AKyk;Pmm!{lVfmhW)mA4D>6hXNbFHIwtP{l%m63 zvV9`i`Vdv`9fuZmW8ZJ{leboGb>FC5T6-cGk=3|rl34yM=$M65z`{j90i`7tZ$f-? zsA-0+9ZoTlXey_TKW8F8*P3%q$P}}vg8V8u{OrB+J~?;mjn@<|Hsi&`P{rP=qor)J zWPV~Mv7b!!A3p|*hcl_XRhuiaxaRCgbIzWB_9k;kZ(<#5QFq+-((^2&f6jIpuT6^c zhU)o6zZAaWUFQ5n_1SSmxQx+UAsMm*wiMH?nRu(RdayUcB9UFgSgKO!#|5-H%1f-` zMUka?>tKm(Xa0-0YIoB}x2>hlSdh|Md;66-hnh5U;35o?)4-<{4XqbYn4l|4e>uEc zd4!(bHEpACIq>a82WLeagb2OcUGS1Sm{m53Dsl{}iaOL?6~uT}J0AA2PXi@BzH zj!Q7#8ZPiDLgdC&%!s%jbK8S8r&fTeJJj3>Z4%3Cs3MB*_Xt_%P`wuHX8y5a3NlON z{LC;MDFxG=7O^yO0EFZICKfd@m+U4wM430K_&TT1}Dt&O)em!?gquTBN%G_g=$kh$&VQX%3XmK-BwVW<@&NAnXohpZhIN&aQ_shVPhx|Xr-a4wvZEYVGM5M(9 z2vUlq($d`_-JQ}c-5@PWcXxMpgHj68-QC^w&2`S}e$P9`cYb^Sr(-N&J@c7y&-*IF zX?iZaX-nzgYBeb;_Zweo^9L#W-5#$M3vcyqhu^8+c58Hz&DT|vcOe_9Gt1-Q%$6-D zDU!M`X*x5L>#l1kM*yzserZ5(e_DFd+;e-JD)l+I9D`%D+$dn6mTP*VnoP=R@r}Dki=D1g>WLC9}bh_QZWJTy06&$zrsTP1y%aF1@Ms9j@q32Q4BQ8CYa& z>YDJ(?&%EDeaD#4u|oM77rQN;vz_m*og237;F$cYK=}XP;Kjmu4Gj&Ckeb(hr_|xs zuv`XbYs#A)ey%%&s}G)UExefdFtqthrD9Z?>ej2DuvVhGZ5T) z6>AHVE_V57C2O`>P)^LI&+T^}2&ao%ereYHAhb(q`jyl=<;j|Uah#l>Ui^Y-H;~A77qRyKpv1y#YGYpwvRC4+HknFeev%S;tGnK}uz)y8 zUiWEq1iMtK5M^)9KFDV0*;7~RTRF4D*PHkC9uH{6X8xUkw4IJ1x@s}w!eBO5q1v(` zoy^_;t>dotAo8Hxk4vZB`w6ruZ#Ijf{zoHlS9o zsR{}@Bhu+p=X?9?PX%5N2T1K}sS7R+X6_9ChkAz<2x_m8;&0>{DA@xI#kB@=Q${{K z&K2_~0K(V}vC*mxus73L@A^rdykSXBUbCG<>3A>xWqYK;-#~MGE@$EQh|*kD_FRj! zxOT%G`&gyo#CaKFtYf9BK&K|cBPpga*>~xj>u;f}O z)oeMrAL@V6&^jo+r=!U5E?W_P&Gq|v@E$h#{`Hy5+(r>)!_4Ffd%&Bk3Gx8isu~&v zz=HO4zFKGd{0v<*5-@B=o;yb;g@|{bu>2}I!LM<VlW4=MLE~S>_`nzR5Svj zw{0jikjES(me;37Hbc{bpd8Pd@87V=Il8Q%iRH!exxuzxJ3z6da&x|SmpcFIQya`V zFcr*Q4PO|bc*v+oq}TI*2oVm(tse5kzOpvpf$K~7y!{1;Er+pJRwJbciaFTrHg!N0 zc{ONa%vJe|jCug!+w6+Mgx$gXTrT~`t}mOqCe6~d15%XbaXzPz;FCdU$mDa2VVCD=UvKoUX`#TAn`=l=R!aREk>s;*SvaXw=tnF>V0T+)v z4@xV0^(fQ>@t8JXY8bvTk9-F?JL@ZiDs)Qsvje4|)wgwSC2+GzWHk#~S>tlHZ*GH}z%g|OP}Cz1>c3m_0y2AHX^SavNQ@h_0X*pz%X=W%RmT^4jf%jlLrnTQy;enl zg?y!||3&p)7&iITw^nv5jasFHngNXS;d{wQg8f<+d$R?ygr?8xlufolOyAhJ*UO?3 z)y4y*IgYYi21Zy1jww3>oAIXOeR5_h0YuvGa4kNIW+EBa+9zE9X`nQ9md1P;3l=M- z4XUIASz>{AC#esNsP7*Z9_~^Pc+F&_9xnm$RUQE_go1{i=QJrXH8Cz`39HOCvsCB6nOY7EYzW@&psQpKXU&iDI+YFS*e>^%ss z@rIfg(u-u*en>Ibm67oY5t5{B)?h<|1r49H>o=o*OK0E%w(ZjIuGm!xxXe|3Y1A80KseeAWoE&?S-&y&gYJ z;9s8w-_Bu32zv^6d+IZp#e`B&FOTI&;z23v5gVi(L6HdGb=0sV6;>ga3P=C0J z(cQ|c2MRg3gk>pOwedslStC+K<);InXV+lW8Q=y?Um=m`3R>GWZ9WqbwjKG3J@wQsYOEPtmFH&6r==P0w~Fpl+v4f`ap5q40zQz`pN6cKI78CF6+( z|3QY3qEay};a}Mf}zlm-xS|9eFKI@{(sgQl3mUymMrAE};3%Wuh6UcmUrJLQrcKCifXu=I z=-tctOZRQZVk>|IAXf?QY`=>DfxD z43CUJ%}YztVW`wz3t#>&wJ6Aia8tFg2Ok(JW?2B|;ur1?3&3VLaO%=Xs7GXNq|$S~ z$@#vn&ZtLLgdqA@^|dq=*U4GZh(5Ux>`+@%Q{I{SxUI}qzq;mNBj|Zs$dECcyTOgd zOoCr>=eITg8!iSoHZPEH-vJG5`Cb=PA4=%72qTXP3G6cUG(Fb%$Z-OWfWxzJ&f&;v z`BM(@+BsZFtL3i17IR3>m{z1nx<~)J$o;LyAPlXx`c(C-rVq!cciS|YvZQ59Kgzf{ zBQUZ(k+!OIZ79w-A?_vXGd$*>v62>?e)+B72y%B-I{@%jv6W|Iq zyr)M^t+?+_N4Wb08|+m-zP!?4hhyZW9%;@&)VZG9<`}hAKdVg9O}NbG{u-j_UTX9s z>qF%2MUjJd%2U%7@N3RlH$vVLOK2ZsIlBD=nh(b|z!-D;lEA7KIUWcj2Fh%MF@aA)*m7nlrV&w&^@ z{^6b}Tw+h8nts#>&Hb#a#kk^j5)D+PX<3MQ%@{w%Mh_I!rS+T(Mv&M`UKA6b%_7}B zua3^xqdD7PB*gyF6J<2ZAg33RKv!q8CbnP|*;9DLa^}Uz#xhm;!Tk+^BQQa{?Ct)T zcdZIyZ@>|@LY{W7lK-{@!iH7gERW^fIrw(OnRJJCph`7~4#8XkZWL!|u@;wp+gLm+ z+BmPBfT@V&^n=V;+h=%pMjeOU^6qI6VaAilZeQ?&%(V1D8(PME;-Hp z@dCIY(hy_Rj1~#zMQm*M?!jZ>eG&3@e*zRXS7C7wjZuWC06$;3z!lR+ri=UppHR=i z;vUd$gnu?BGs+ft`(3>wRHEY>ZW?3?3)tR`Ek@N*(=z>9 zWWp0b~`3z`-u-Ar~1UyZKAOVDyA{p}52Zcs862 z#%xw{(7ZzIOkh0w$uQ^?s)cBZdn$U4GIa2Is`52T1jQ?^1qF}t@Uj~qEV_olM3lvq z8EiCZ=5ickGd zN$d&+M%=Sq8-P%x#Ukrmz3w2@X&Ld=p75HyJ3-$PGR9SGQHhk&g8!*N^PA8`;~TZL zph_9-yt}+!--OrsHFeWA$5~1o@)oAm3!?*QS6!L`DMZJ|6A`euPNO%knX$8IzEP6T z^!f7w#pSJIiIJ~_#nZ8&1d?=Dk&FePJzM^+eW%6vC1saTawCdf-oEuu|e6mWpH11N-GU4(nL1S~{V`MSqOU&+G>ApY16?ZH!y zoaT3>sA?RY4+;qSo;fZY zrWfrMFgWpnQAUJ8h@yiDY~1E7SRrrcT~0T2gj$@=ukd4q!j=T}=@t7YBR7a@?e0-dOgQ z<4?mV7R|9T1?{J-^2CxKO9l1g=grhj)Z++fl#XM@B%JBPWBmmmJJ%Se$;_?==XyB= zPU6~4Xh*fOAIXkzqrL!WWH(5%ynN9K3jOwnx00ybV*2B0tsS0yjAk5e+Sf%kqN1?@ z!G_p%VzGzWs;*$3%EohU`$Aj z>SK@TLMYD?mG81#B`!iel;2y7STPMPdn6_-#z-|XC8SInA6t%g@uMPabB{kt-wyoE ztr$;lL}?!?AfL`=#t)?3-w1sT z+o~^FsH8f5a@H8yC^8m1a?IV1bb>d+pmO1L?Yuo`Du*x%(~wvspMrA!U2bnH#I>`T#;RK1eMtYR z9I8|RAOON0JlA7zP?2psrw+~Zh?a=F>|hX1&D^8m%FK9i5u}hRVj*(o$tab-VNj;9 zC(1U`gQ+MGSz%v9#$519InF?mKLyIE0TQ?$^CvZ?l*09INkJ9th0t>&%uv5JMCm2_ zxJ(!Q-O>9ht*|C!5)?{}3f~^qks#gpVObv`2ER2SGQKya>{6&y1erX65L}X2kqG#1 z#>MQA&0foz#3szBTtgD^PYQZsKClqiNL6&#ezDcDM-A|8GY#z@>?aKhChd#2^}hwe z*)ee{3Nx{;nWX#Yz;m21IDZs358qNW5ocT&3tTy-ll@ZqdOt@@e8CGIo=Fr(!bJQv zT8@{pYLB!tpmy^fL1gaE>l4v4Wc-d*0_P)#l1Q{RS;s<;A1O8COMLX%){5WPo$GJ! zxY=wjcF*5_%DbQ;RTuLP!kkon6`Yc3Wvb|OSzr3+oRT0daD*+0^@~#x+Lbbldh*z zaABeRSnX1iOsVhSRkIl8vx~~<^CklK5PS^G^7G?XZ-Q!Q#y%^2+}Dzf7ID5OD5Q}kpx~G~^JH=syExt`(WLXdcWdR^*Dhf8hWjY=b=X!5 zqlsxtvZIvD-sB~cs_spOwSg`k6i{9!OY1c)XdLrL*(>)BD$F0mzOHzen1Z)G0xv$c z|J7C4|72N9o-48A$l7AOEL0g-$(9? zGM4t)P{J%sl<);^)A*w#c7Jd^{?!amKpuJcx?L{W(O#k&J`Gn6S3wltcYK#MI~}dD z=^5ieE*B$_NPdi2Eu@&$+9YQnE*oR=-C3WnQtD(WC&D!gmZUX?9e6T+-f z;J2!KrC^j#9UYQAV;P*2%be4f-h*ID*(gYKPcEsYMhvwlHEP7AfY_ ztEWDnqV`6l{}Dx&ZG|giIjwTsLfp0cHnFK(!G3wy7ol_wBtDvQ` z7-cmm%dg3%cy>dyVk0-Myd_&kA+RduUjHDyG6$y-%d=|Ly=4W#~J#8=0MileAw z7(`hf1>owyIrXor0}0}0eU>%T*~02<`gvr=8J3|2^=NnkYnD;+k1fm=HkGP5tQxNzdskDGliA(q>_-4}UZN-dY{tc5bm0+&oV%Zc4GnT%`LI(1i z9HCj%~8%Ln+U&on^#EjcUvvIZ>05brN~6d z?;i#I(Ms=;QP#MKxpDKxY$ZQB&sNf?vRls}_px8^%HZ@UkR3^qS&I;q zEqnQL`oWDoYJxXne3W6-;A`(JWA-LekQ{YdClTK3aAqVe2~MN>iT$&;{NfrMvKHcZ z@Mr2SAdbAb%?0#4uY2DH+AFccjipHD&fx~LW}d6&oiNhn9T!MuS20>=vQmv=1zME+ z&Iwr$cbYRL&UruA`o5UH$V-CSbGT5MmNqVbUfz{1+5}w@HcZgQd0^8OIXyA5dPY9G zy9FfbMuswi^vb7PGd_NwVy=H}US!@dPH`nndQ$wflDOL8tQI*X=VKW-MS+^f-8Sq) zdyv=xXH7%w^Isp4iaV4R{EDOsJ-+3LDWjCX;k(m{Pf2W$l0;H+$1%#EMo1_lJJNbn zaUBTR`L1+U0f(px=B$Q?Ih2_tGYr=u_x>wmdP~%`o^CJ9SfK9iyJTx7w*0)(Tnc~O zT=@ryz2rIvliVPtBYJQqd?lFTPaN^2g`1^}HWiSuXowljVP}D8_jw5nK5?pR8X=X@4ZwTB0Dm z&zB`3lG7?UMi4)JEtj1g_Kscq=2X>0`04NE*VLK~qmHa&&Y=kdV?kD^w?ZewNAF5& za&hfKa}=X4o-D;lapUTbB*e9g3v6t%uW3@lsKfG&X541ZL^eM&33SDO{o&I<`qVo6?V%tM~j{e4$D z3Xg}nUZZvxqhCWCFZ^EozNRv&IftdkeG;*{qKOm7B57$C8U>W1<5CLh@@K*0Jl|?Q ztZWx4Y?*2EO&156FkyJ`Sd;D3V=%MqzOS!nPxHkq@ zCK@O215Lrj+7f;d>vl!hi_$F&ioRy@P-5!A%nxi3X{O!}IlU+ajcZJJ6Gq|=p~B*t zzZX{LWOm5r4i$YRsj%@^jbvGRZS z&cT0;q9TX-JV>T#B~LTkUsIWl+03zP=l5l<*tva3fX_a|HiF!Hn-FB{FD(GSh3ll$ zo?sQ;b%w?e5=_Z{djAsb+r4o1yy!#75;hW^a!*_IB?7~ zunik4`!`+e>dMIlXrfBwtUS(C3Ue^~ zsFuI|txFI2gl*0>gKmPL594eWWk#;~d~2lJb&j$kXdBkqp|6v{^t)`V(8!8=FNRDN z5%o}}F+bmG@Or|omP8GgizVo2EzNGa1nR0+D9?4bcQWnA>0g)@a5Dc`u;!aa2*gLe z-j(J|9fVR(#1eVIJ$M+)38R#Fs#4}GXL}Aj(^S;1Qw1tyzdA>}?{P(wCd-;v+Ye#h ziv!@_rz4*$bw)_rOLl2)KG9sR9r}i#bO@;8Xk?+Km7Ijun-qY5V%xKZ6ihkeSQk) zM*eo_^us?el|bBWRBz=ss&m*=>!&)Mg4(1+z4L+|0tzR@41dcsjUYOlU!?sO^$FOc zsbf7_966KecCnepQ<2zqy9r1j8c#*(8AR>N5xujs5;vl%H>N!{@;o5->DU8Rfzvm z_s1~X9sjdAh6xKAjzXK%NW268jJ!Dw_nv`)y0VIa6wncYxd@d;)ueC;q1m?>s$VRf zF7~GCp)>?VfCYO+*bno!ue2Aj2|$Y;8|_bDCEcdI+5fjK#~T5u0N#4i*WtAmJHLt) z`wg#RthY~9wJ`Tv-BaB{*B&868Nq^$44`cZ@h{T5|Lj74jhi_lm3GUl>3Go=@^g3X z-*Q-Mv)=39kN)~PK*R@H-v&_VN!oC=Cwv@uL}ie@bZzmS?<;qm1e0083HEqZQe<_^ zZ@bQp(-Zk|w&%mW1HkQ8+0Ig2_shXnN;k;Noh1SdHh`k$>x50_Q|}vrJFqzvbb1y- zB;dcg4-NaTL}A52m|A2+SJy8_Zavl@ZLN;Gpo?&hK;|D3Vk^aRE@e z8K{vf&kmD|r)egcxZUpL5Y*%K0U**GP+Zv@a8vG+>>ip=1~_tVn(L@betu7Wy7gx> zf8-S$&8s^d@KH9dX0FNk#2gqIR|Biw@lj>@h2yM9sS4oj{9^#*cUCbuOqgDaXcs7Pg^&zT~uTJs^U8DRfSVFh*| zL6ZZI(O72RdFdBu@LNLD7VkcPWGu7Q7&H{0P#nkN#xVK{o)Sj6!$GOJ#1U}iA6uRlKScYn$J4& zO&Bh?;A8Um% z^|N?IRpphc7UYfwu1daDcM3(M&Z~FGo%j$Ig_S;!@C?sGw^{ijYT4@@yz5DCiST zX~<_^ML&+1NeFu^>Dnuh{@BK`kHe5;s5OwKZDY6JzU0J&Rr**gI0qcH zu8-f5RKp`zNfJB&vuH~hEoaP~CLOhJ$}{?qu(7Fm^U7aMkMKi2G&!KM8_z()TG5uU zVDU&{B;9_eC6L)tNCd%8p|LK0t}0Q~9kGAaH;Us9GG_16k_{0|ZT!emdK$GJyEf!v zPHXZ*Q+fAKIG~kwvmG$Qyl$`3?|vJVN_LrNDVLm10eid;JQ$xwaN~76*}P^96q?Ac zc$eS>zyBW^951K;dGA$jq0^qwwvPos9Jx^{V{XFH3lP1&BAxXRhhEmCPn$rTv8=_y z<1w?vp2v|%C2b<-y8~aCV+(O1b9MlhyN~Am{Au4A&9PS{Z^@p}7FVR1-`hXrU>fzA zjK5IhD`U=Yf78r+u<;EEM6|>n7JV4OE&Odmfy7m8HwqtY_LMv(Q|Q}^mV^9rJ79wE zHou)%sO>r;$#NyCoIn@&XQwYadj30Z`h~jr60L;Zxy5wV+wgDt!Sj%Iu?%oA!1n^| zu4%_FB%4q0n9bv=O~?AaHBINBg}`Y!+vsjRX9a=oyWmVYS&AjyFCc!&{#}fZKt8Yn zxQ+v}eV4$7OyJb9{V*CB{iso=9d@C9OyEN*(~R2HhpMp+JBfv2Uk~SHEs&L(v*1hk zX}Schoc&&{brZPWTo73K1qZ8Xuojnvl=5@;3$U67PL!C*O`5EIVck2x3`WLsDc-bi8Er2)svD?NTx2jw)YOXJWIDb`AZPdDMjtMSbbLV+kr0d~9~t<%j=<^y(-G}{^`4?KjL^5CzkmzNLG_$Pkoytmwn6R6x|;6`DZmi+N< z&F4X;@HHdYD5G;TbAz`pO*mo*Vf<`k=-yd3fg&rVoKw>PBEe{@mYy?RZ5FHLy%QRN zy$DX*U7mdTLbQBCJ@LsoaM1gsNW}-p*hHBMHb^$3HxgJJ#W&&)<}Zf zvmXL#M5q?x8&j{-l~b^AjYra!f3YlpGDp?tKi<_Usm$oP?)>OaX%du~RX<8A#j{sy zQ*QMg7 zcJafz8Jm7;3&2d`)+pufpHb$|fD@W?TO%f29;I9 z_@n56v~kLaeB070V8EF|u9-~Cr<`mW&#z^YI1a1WQ}!E18gK%KvVWItukJ{DhiN@` z=90R%5QJMT{{&c28!l^o49XacfSeKiMmrmR#e1t|1P&?Q@Y#0rcB)Q#3NTINO)Ves6ICdVO|sb$Gv1SjMuvAnZS%xegJnA*;c+}4h5we znLaqSTQuu*5t(P-@F|BIFe{ZqNrYBSRYRgSC)z{E3U1@U%ly)>U>NS6+ zL0LF@Setspn^L-^p~moAl{ypz1rk`b0J5U30@YT?YfM-bvr*AUP#=ByfyH9c2{l}U zu^i|2N zOGwW*?d59tUySrw0950)y8kmoU{|{pN@JZ47^<1mew%pUTHxsbjyE-M2)VUdg8&qd zj8CU^`UeMfD`*Q^w}YPdE96%N+Bkx%b+!@JluW9u@%fr=?N6{Lf%ovvcb#fHCKwfy zHKXH2MQ!zd6cd|_>0-cUq?1$$F?-q_Ef5(w7OFV8d8h?jo1u~a@Ry;z%soOEHId41 zIn#UuHlyCvQ4*uvhzc=F9ZQQ3q*A~{k@pB4=X464Lub%l*KS1esJUJH#b%N>VX7tD zFqf2f3jfJRKjO5f0dQJ~rQb~;dSgT5*J85rPnC8!$BNZi8rb$OApV&dW~{PHQB1sH ztjO%U6NSD;YXk$zUySUl0Lv@Q96{80j0+ES1z$Nrzx&CvfI4`q0wW*Xxs#2-i!0N) z|0L*x59htgy^kjwDfpF3%7rw{GA76UWJ{gQOqs0FMUqh|RaFnoRABqoZaS39^}Omi zvG65gk7=Plg&a|;cZ|SBK5g-RS%yLO_Ju#E)1Ui!nm!_CKpvhlKikJVY9e`8{>Wbt z$vx7o_rMdB)>?TwHTs7(vTl|;?o%k<@5S59UTpzx&*KGa4QFcd(W>2BmlceH^BF_Z zxshsd_@jKG42Yo7&2GKnXTg_+n%Y^&d3DU-46{v(^Tz6RJ0%?83>DaqW{`>jOVqk$ zL9OJ(?Wr{XBKV&FtY#`m(f7JZ%}%o4>GBC`Wy}BOtokWptXzPq`upCNmE{N*NewMo zd$h&whP!oedhqfb;Rl`hI6&R8Unb7V*Xg=_4=)jW0UmQ3yC*P zBo>X?=2Qjmd?wCKobvy4JpYee(2%y}@(s?PcEIEp1Wcht>YjSODz3{#pOGBw;b$8; zt{EXBo073Fy7|()pZ~j=*&_XflIi=O@%=x)(Lnlj{DtIHY!Kv+u1f|TmQV24L6Ti4 zr5O7aVtqQw6#+K&V}7eB`2Rg@|9dCP3rYGlLT)b&N$cIv$uJu;j~8=-aDkq^M#0^X z4biEv$syrC=!E5!!DV=x&I*ytv|74WoY_oRh0ZiPXwpcfN`Aycs@*vjbMe-0RSwLC zxK%$(XF;@K{wJ23SOL86pDQb)*V2~O?I_Hi@mtQ8r|{cYI6M}uf3r6&MsrZcQM!qD z+{?v`UvsZ`6S>z9wNaH^{4Vn668~rH|FmNMV=%#RBV?F-EMKX4x@3QqP|fTzVADT< z^>HGZ0N-(CKvII|$QIoYBcci=);N}osmL%o>{{E%->swGb5E6ArrTl=>O>jrz0CDT z81v_L^5@n3M`Nta8XW=Fv@Suzi-od#z}fXH*To%Y;X;MKqN8q@__R{PPi!e|-N!Xn z`atzJ?R-^oavE${e0hkORFLQ-%lP4>JvtWOT<1}y@kQyt2-+lw~;3K+rI!lkoFuA z6S+Y_GwFrUd{P?++r>g)Cl-k<@oMT(FU--p-Tv~4zw}A&PPWbcy>?Zw#g`0A#PV9X$K;?* zvbio6#!2I~&cEmd^Egpl-aO#H4$Qxw%lHLc&bi00@BAf9;xZM~{-JI+v zLzim!GCSvJ-SN@f>{M{dc9xHvG|vn}*ZgL(_Sf;lZEZq z4qmy^Vm?_IqjBdrt-o$QY@A2-H>=UEdhJ=CJWANXywdlLN)x}MzRq0f_SX~7@&C3M z{`%|C4^(NueqfYT4x1aa%w1WVcdS3jq&T*@ZCv8<@8-EJ(HM{8euq8a9NB1-(pv4X z*H+5Eph0j}ZhliSGb-DZTCAMTBHn57Pe%Sbq{sEd=1KK@6H_8v=+w06z#7edhL^U# z2W>C~tD6js+et93d!jV4IRjB5z2H1)$xD3UiTzv&$Bpea%DRZ56j&k_rm`RHwFAo_ zeSpr+Hrbbx;{QC%|Nj!IQh`Ot--<>K(l}>Fw+cOYaPD3C(yYANHP7;`*>xkCv}8%W z06jg`ai88wW1@ESC@KcaMGs-aF0I2AMEhX6P|_Wj{mlx3?mfSQ5cR*yCG>izWjKme zsskCO6dQW&`PPH#X1RXAp~--FY}l{w%x_~qb(VbF7~l96*vNL{v^uzSoOH=SaiL|t zx-#ie&%VHFH?`b(bvG#mKB8joX#Nm&Dt85X6?wlCm)3zL%inHyzxffndb+ETXqSy$ z6dUlHoS&{W*+0c^kl37Z9w?g}^Fi*?@|}4+5!YIlKq??XT*N@w+AF68x5Ta7irvY1 zvHMxD z^pin+QXed0&dOTE)b*xM9Cjtojc8{*!t;gZB~;;bj&-J*Mun0uP;ldWHS6LdGt9nk zcASd2YKYI&SYWwOdL7sOlShA#Fsclmw}hkym_EX79oEp}*kd2`)VH$fVfI=F3VjSi6t^_rhtOmHvb$u28A!ZHtx#}B+d)r^|6C`miXqhbNg3i zCc8^zSBb`m0}hL^*JJE1s;K#tNBk#; zIu6(FH*cS!5j^30jWFghsS?dCT%?e(udVniFW)xjat7_x3tMF@HNqLvK^?n^k8P$_ z3xAx{5`A$w+s1+-9NrnR8a5y50BC|VJewcVw%G2|Nfdw|G59km8C)M}&1F>L6pFNxgQ6Y$hk{3Aph$Wqb!bEcmUW(C%BY>%G#>63wcTVVy$D$>HQde@XdEpE#_t!hRi)~)0CDA+c zKvUHT9u5a2?v>QCi^}x=;P7kV|@6{6zhkC62Ozd}quT44>L_{7GPBfUtydn_Csu{4>RHf2ch7L7h zA;u8cpL9?yat9O_BRLzSihf^t4e~`olbJY=BIZE8q%Ke?%Puy9eq0CJMiPdRZthBx z?K5=8=k_4w4!yx{3j)~69p*80!8%Q=5#s(p-^cm2H|lK-fQJ}0K*Z@12z(4ohNj9g zwdA^ese>YZI*S0zD_;WGDhg;X3$X$asuO$wYxC1Rh>;C^{0QL(cvLqpgrw~w=-0%7 z5&RDNc+4$0_#BI$A0OX<{E}}S)cl8$(%dhf4qtMtHj)O`yJ^pYWTJ8u9}ARSYmGvQ z&j+m_;H`Ky4L@7A@B=UGh}5HS$7K8Qz(E{<#<|sjp+|nV_A&s`Lc+g)8niwz0e)J4 zcrwUTndj_(1)2pIPBGgPrP^-HM8GW!j)X4rdEU+ywBPg$;561a+-^=zfEBlVu>_Q0 zIOqYT;Mu-pOmB3`W(t2l7`;AT;kX2idSs`WSDKALthszMq3xMm`nBa0v;^@zuD5dCaPd&~-^1JhL3K~=!-(v7Y&?4QC5SA^y%3DdzGAO4zf`GqVW+t;AUsrTsq*l$<fp=L+*6_F&xO)ir2D`bl$MW>$^qOb8iJsM>`azO>ITqadXyIoRfq>=K(*a1 z<;IsYi)L?)G6N*PD=3z}b46T0jCV8%jamh0Tn)vx92#-gi;vHJN@qp3O+jEFuZ9{} zav6_)rR`|g{=!-Zn+x4Wk}dpp{-r>S#pfBYdVLV-%3-Zso9rT?@w(RNfd^q9&A@pU zgxU64@pzMBV8sI!<`P|Y37QaHq>}?iC|LUQ2fa>zRAi7+E8b=si|_>Gi|OABI>{cn zRs4M4zdACRhVQ)(1jH%QM!dJrJTC=Mr2A|U39m-;p#vNjkMJ7z9terQ@GnwTuJR+T z2OY?gZM^V|wJDlCA>yaU@Z$kD__A(7SHYq110%B0*V7-BwVVxtkc6V%`iiEcdR#Ov zeD6WIIcW7f-OlC-WTD5f{o)^y!7MXq<#CeuJ6AgO)3izM_ZFj~s`FHHu2bFqV20Oe z+~<>e))2e$iz6@)FK0w=0wshvhDmlzw^tI-yW>TBNleMOJAKPTfP$MwDle!0OMNz( z)=IJXX-NnH#}C)fjwiFWg9N4MN`hi~74qfJWzB%Tno1?)2}~*iH_B~E{d&TE3zmSb z_P=M5DXe|4kYhr#q!V{7nN@`>oAhk@3K_JTM0m9^eo(Bxy2WB_r{a>5uu%||%C1cA zW=o)?+_+8gf}D-=bX=%M54@mEbQJ)Q!V>XrO&SDCkrFhxpHGo3G}{kbTRK>l!B7vs5Zc62@U5g`dSm$SOO)({Jq(50Le*FA;*w!W>x% zL*b)7%2NEwC2NUp&nbi0U~k%`%tJ;JU(Ai{73j*Al>SVYQ&`fltw8?KMi}I*9d9`y zqu=j=E+dk!GC2xIaC#E=Q zmUKux(cND_Crxv}8iRn=gsdKE)NwA!b6aXPW|Beo@2HGiXb zmc4a!4pTT)IlcC6k6JF+GqGUC1vBMi;a7SNK`yy(iDsklb1(OC8zBjunj$3vujMk| zsCQD7^W@S;C`9MqM#Mk{9&{fj_YcUx7uI6mL8+_If+TCeL*n>;H3%Ry10sRJZ0Xyj zRxsh@Avzz0>JNO(*&katq0puIKKiMXsKf#T|AYfnZCp&3C3@~m@K2b)CTET42GYm8 z4*O7aR<62qxCY}UQ-Kct6W}fctcJ%7NU+LuC_!|yqKL%kCI=ai2=x>G^@0N*4St^5 zXxpte8lmKR^*-DXfE=NB&Ka`0oXF+A&obySVw|=3 zY0%{8q`8`n4o3v^v7aU-l1P@Z2s?2isn|TrkMO#`S!0yM%S{6W8pdtHaxXpsc9j)^Yc+)7pZTkYeRoM>D- zT+(3Iuubtfdf(5*FWjae{iAMkIrtem z{x3Obs_aZRS!QF;(-_!*jfk6J%{Kw_=sp3~yrLmQU#?0oc_)hP4D)f;A1>{^a)j(9 zc0c2}f}byM1Dx{PQWj#gXr1A1xCmjOZW%)Tmm>ke07p=Je!aOThCyT`fa!>MPn0=% zsHUPHF7z>uD}-pbVn17n0H#R{in4wtNA_{XS@3fzq@#dt(ZM305y;)NpGQDCA7ht4 zDy17Im&Tx)cRTsTs{6zX<{901uKrbEl_0{|AwT0{V>l@pMgO7;Q2OJRwvR(QQEBlQ zIuMkUqNW!rI$myO&N=C?E`}30#(^1{^J3^zsY(Vlr%w{#WdZw-cSI#mz{$Wm1 z_e-bCDf1K6ov6=nVi2A2caKe+baW_-tK@VD1laL$=3AVcXqr!ltrtAsVd%`wLy2RC zf*RQvA8=D#eY-Q{QKZ@Ti(-ADS@QJHW>B0U5aj-GAtx$+Hinx7G~)DXIYRy@GJ5VY zA(eOR5xsHr67HZ#bI@8{!%0>)v>xFOa!nzS`4afA*NO89*6MeFj6N z&{5aBT)EaD(`wF76=7?#jpPFKD0+?e{me(2QkZ_HcZtVQt|B-;I+?~KUqzPPtQ8Hj zK7M1$mg$*z-Qa6wh*)qv`qalxlhc-=_lvO-m7ZTNZL9Vaw^I!PXCZ?LVrB@omJ<pcNpLp&IK;rxd|9ORuR*wV`bDKmd{d2-M% z^$z?50tT3#i{)gEo6mu6*iy8f5`ufI@vHk-40q>p_d$=j{+M=wIQGQh(@hJphH+|e zYZCe_<7y(Q#Ey&d6{928Ui&#QEhxn+7;JM3e)^Veq`3M6KNQyY*(bu}5NS?MTSwqU zLa{ykT>qTTS#=~Js;&QJBt9^3^!<$GJJ` z+S&JLVecv0s6IB?Zk1J{$-Xy4A1OarX!R8PnylV5qmmnFp9m6KAro93Bf_9iPDpv= zIq2J`cSqMzaN^$9@bT6iTSVFTdi*Dj5ijcjsHjnh;+iu+jms&V%@y+Uj~76>PB9ah z@5Y?4ZlOm-P@al#=xjK#h(F*0@}Q+K#M8Z8FyYKxjE_Fkdh;I9{;U zSS0^VG_cl$bfx|)_bMQeJnu$|>*~a9Tr*eu@sdJ36{{SxJ=Vj0Pk*mq+Yk!^2XFrDwgq5A2oBW*)u#C05`t3`$*xjC1G&^bbH)rW~!(kfM!P`(Wp zQc*Njky+k)K)Xv-C9A~KZCj(b=J$?_-@v7?SgTNx&zqZW`|c?#VNzFeyes#Y{KssI zfMdj&xkVGOEI&KPl#Z@@_yK)R-To#u2}iMk`Fe80g!`jpc5<$&39jX2Nq-z=c-$R> zc!vj3IF{m?+%d%12B^YilwEcrc7r<%dPI~8)97=G?e#iu78I%pzE)4TEQ9+<7IMWl zYp1Sz#$MlVBhr=6_9cYN9ifB@tJM<7NcA-x(o!zC(2A3*<(U6OKc$i3(iijL>{+2& znhm9yqd!2O)#B!vXxD5e*aE^!Qp(oXSHXe#vl`Lie4)3pEE&!b z?Q%&~e2~>UY;ae#I^i`>!QmB+{qMNBKGW>=F0dXI{z>PSf(9vI2e4U|Zo zasb3A({|P@b(eC^CL8(x%nv>fyWsUW&Tfm8^L zN6t`(o(Hn%D_501+3JznjGu=@h+=Wlc$Ow<>Qe+jdU>14! zc(-!5Xg~RlxZ9o5p)AiV)03j~9h_z+K z#t*MwnAUBTE1B+g%A0jN*^bQ;fU8*%;S%mi#rNmm5Pqsvs)0D?U_yuro#k){tsqe> zC*GPWKF>PF>0h#Sn4-T?7uh{H!7yNl*7MD|aKN@8t@oV>pY)%b-@ko2j~ue~x5?!( zzR6!aQse%u#);b6=~D|m*je*-(>=pW0#WKcSGz|!f)l~6Bl2f za&Ee--1)ad{sYpDf{a!@sL;dO9x~8d&`rq$pa|%}{Tl=P&p#$n2Y?|>%=>%|lP2N+ z`A`1OZ(lPF{Ny=yVNgr*pU>uBf17cPn;_tvCF!_V!&-6wcg{-O6FL9OWW||r^uHeG z&o|Nk_X{OdqeYOO`h33Tk&@>#RpjU;aVxU+&A?=bc!Bb7d>nu`FAnGPj&H7(FXtYA z?+^AP6(tuE-eR99chzz(+$wKp47sd3(h;2j?AA(UvW@B2xAMUb4tJ_X?*7*IXGZ(z z@NHtCR|#oE=+*!M3Ke+mG3S;m|Bq`okcFn0CM-BOo%FpkMmOlisTfr_czsM(pUcNw z4VnM3C|~X3YS~pW)eE5AJNp$=la}3&K$(KECLmRRG_2vUItm8(V^*x&g&rE)% ztz|rzlH}@A!^AnFWs;B`_U2%OCDF38^%JT=>j>zA9u_J(Pq$4t4!(<&bTs|^@qZC~ z&`ru%!cQ>$G5)|n+GkKf<7?>i0`d~Jyx*b5peO62O;EC9nYHp-@ZtpVr1pu`_)vv; zH0&Q2!N0EcMn&1#I@o|vHYYZ%db-Iew+&^j%OQZDje3x(cy(RJO z6cac;gIm)$eCOAz;a`vRf39Hw(F3Cnrt~GMq{>amT!H zgOO`|Ph3B2p}TjY6;X*~m+<_D|iypFF{e&-4$@dre?ma|Vby0mv? z#}`DG2^|bv^rJoIvem10%rewfhAOQVcC>U>-n(is`zv#{Q-Ub+zO7cL3;@Y>AyQa^ z4C#pj7$Bn3U<_!#n>2Lo2_X~CS;=M_$jl=21?t~mCBAV63g4qSW8mT$Z+3p#5{f$q znxsfiXy=8Pf)Bed5WtUf*AC=a?xYawIZ1q;KadK>KnFTo?TkQBTv@`Q3fm5NF6QP! zsQX6UKy>CxcRIjU0&EAsw{D|toH+qh1G(Yd!YUIj>+IOwQ)q7e5ujX}eNVp=gS^rW zClDqE2P4hzO#yerag1Z}XH?@z-jPsqaMtBc0}|JR0NQ!WZz^&ZWp)zuxlSPEkl1Wq z)M!-CRmq!>;t6t1}V$MN<8|F`}NS(yA z_Dc-U8rvC#P`aam7D#dxv?gN!Wo~M?Jl_^i-v@nm!Mm^oCS8@`ATq%PV3(mn#i9^U zDknNT{(w>FlsTi~cW)fR@ZG_cZFHQ0%K5ekQhe;stmhRwzVpwS+V*YQvDMD&e-@$s zC^}q7C||yQj&hAEt2VuNkZ|wJxwTqyw0jzz`1~*&ab_t7j}PWM58rbV?dv{%$|}HF zkwdVtx|nQhtKTMFeWDVbmnRzybpp5P8aES8w`$nkA83H?A?wYShJt_M@RKd0YS8j7 zh5yaL$TW^y=cFR101`c^zga|lO2$bJCYMqN(8jBDwj($Zfa2?PIvCarVM>G1#d!i? zt4zBI^3kvXw7k(1G=y8l{i3HW%Fqp_-lLs|c&0f3%3*-~LriSZBcmfg{k-W=tbvkZ z5%CRYw?IzhC3;yepY9mo@3PP$?nA5t;89d}KbsG`kOszZ@tsY_N8m!Z5fxW<>`aMPkQN>z#WGLnXj()C&2^(CO2)IXdZ{261d`>wh?dOwKg`BQ{)4foDjez zb1Q$c_{!LJg+rtFNTO#|?2mFXfHY3LU|;vteIocK9Fr)Serk6j!Kt&GzKOlMB-urS z`W)NT%M zvha&8a+o~SYTz5qMiNiIdqPq61NbcDqKAx|gM)zRKS=C+O~n|bvG!Ux-;?tn(0<~+ zlI&RjjI;?nG=6W2IX3dkfvhanNkTXOjrEM70IG50MDf%#1JV!hDe)1bj2I2{cGh5^1&Pd2Wx{&6s|p-JEVc=tl4SuN7EZ) zC8kHfxq)CU)ov_c;M) zOY4eZ+9b}avMe5aibUaaz~QudoXmU-s)toOhuit=yY4KTyKrAE)FWR4CbT+%PEyr4 zG+n)R5P#bY6bpupm3ryvGq=*%k9(Vp+Fj(&9B@-Z(344gj>sLVXdPVpUONKESX9hM z!G~ZC<~RO)$%1E%6)wI%!75mhLhji2zQ|3fR?hAl2XTEO=lZ^+ZLi~s)@vadcg31V z=LIfq2)h=}lakPCyBy+`_v6aj6v?Hj4cDAkQ~X@6hezxRdkiVA-4R+Xgr34S25~tI z6;2^{?KkT3Ay`ENa@0ON!9+q3jZ2$V>S_7I9O@%I=h~s+mPguPX5U&|Ec-s0@ z>4+JEki3xL0e1spj=V4W1!&4qKkR)98D$^a6fd9aP0#IPT|C| z50_6V0$wdiC){p_n@Vf_LHuMwS)#IOk{RY%*tk}uaRnirtxEtO@wM)EpR^}#I4pBR zocD86_-Om8e|5b7llh4f`7u@u$on=c3q`)Bs~*7k8DUBZ-Z|F3!f@M7FoXNYkaG=K z4xPghiAHD2xWmt~byf{szmZ(6O^JRIvEobTR`LW5pMOw|^`cEi7{Y16O5jF9@@?OX z`xf+EUuG#sWBPOQqVt~|K0RLJm%x;o?Y+VSN@ zasqtQ=C2w`Y1*|~x1KKc_X#{WZMA+&9A9u3d#+-Yhl!rOtQq}Qt50@Jw|En%7>Gs( zq;RSMB){PM!}Vp?DI2(TY)Fhx2riXeQ7<$+vtKx1eB3z|uJX&LSfWR>0`e&< zYFdLc7U=P6ho2tlNKIIh{?P-AyV5jr%U4`#*fdn79-pCAmm8G5Rpn#doy>JJt{cLQ zCf3x@O*1ViRMg~pr)s%%X~uhOvmO?5#ALcdA>(F-)vh#=JD{uB3Ur_u@@U1%45_d1 zl?7T4NJPhg_v#3kUkL`+S63zC8&ps33hT|G>{$zn< zJD!^{V;keHJ>q^!W||TE@G(5bf|905%Q#wi_rBiU+d&#;H2zpcl8!A4Vt7_+$iQ|N zy^&6sj#6|D1-og}7^Ye*TlB94knNB0dw*{@J+D%BvG-QX_XO4p{6RtXr^kLaHQcW*Rlxia6`D6+b zHE2<1rga3;4*cSMER69>ZGjhz^P@(WROnZRwwzCB__6xLM?mToevk^de$SWjyH5rZ zTa`{ZwmjX%X&FzV!QRe*qUHE1WbmUqG1J^>JQAJ7ku4^@IL}o&HK;{89?cA)vN>7~ zp8NP3g-p@E9Gg^nhM$;^`+4mnjjcrZ$`{u@K_)-Zv`h^>=T@Qg1S=xZx zN;uG<Z`uWDsT_(r9L+kX3*T@iPX?Mgd`%=p^dV8sJdOub!kG=Nzsz7?Y zzsf4p#seP2PP7zoUYw8cn2h+CuP)>dcYP`feVMmHbV@dx9> z|Luvo(K!by{7BSDpYx(#y{hJ&V{3#PDtyoP{N$P>ngwRGc$OANF(k|Zl8w)Hk^yNh zUChTZbFnfc2QK{#(wsCWXme|xRG+16Kq(zX%ik@+zjW6_3cwJJ7}8f%bWhgE?TRH< zK{tN{L<^>m@K8ohNWLho#*{7}gBC^O;p+q6XpldFfC?vHx{V>ck*Xw)Het;#AyX#d ztn*q(GoxsUzv!JG{wT0UPgdn4QFSe~VXJupO6KP#%KGw3!*lJT z$)@uZGo6j$J#Zcw;!>4wt1As^-gYEHWqwnF{UpaoQHlLR{wYqytZC*WQnq!w`5`9QUr) z2#BQ`G@0YNhj~Z}43-p)tRAVc1wARONICuk8%VP9-rn`l9i8!7Na@vqiew%`=RUxfTk%BcByhT{XaX8QVR653dyNNT8J>aaPR^|zND z;ud_hF4-;zq?Y0tGPAe1P#Q3;<=<=yuB%foRrLXquZ?eXMW<}VZj~=T^y}l?OG8ke zO7+_c{`nN`(>!FUw>W%LLC!m!@;=FyqW$X*^Q3yq{L~j1Iy3gY@$azWFuzc5%j@!!f%+x4mcVmeK95|W7pdG+97wbW}PjhI#?#c`h99Re=m^V@Po3!PPs&K^4k8<;W%4u}ypa_w+^2Maw?{GHd zb6-@(jZ>BhoE^c`BGFNl5SJwYI_9Ns^c1q|u^Fnrnb5H7%7!`LGZ>4IM z(>XdT5O{?@uVN1%g^Nd<*Mvv}hQRr!J&WcEX?I^4{tJNiH%`k<9stbOMEN_SW%>=H ze}eL$;LW;Erti_MFQY8k8|0*X3~Rs9YG5{EroQ{?Rj=X3>trsX`jy=Ko`S`_BfJXN zhRr0k3dg&a(K9W+&*XQI56=lHKhiRYo+it^xEnzFqH&6HF4_;cA&rXkzNM1uNLnVK^eHWLZj@C)7)! z|6XY$a?4R^g20(y^OHDAUwC;nT@amPrdCRSyt3MUpR7nuZL{m^GN_F5}sCHy(>@!ClJrf*-!@*DEq%zw42%Th%|z&6pj z9QnPIQpKRQgi>{pRgYUh_uczX&%OcN61=Dt3cn|lc>tvp> z6YH^@DBp#{E+!tlY9x=HzKvjHpJujHvIo53hy9-<2vMNRFP!whQ8QNBC|U1yBsQ1A z)It|cKF6C{V;WY3+lNu;dcSQ`@Bnb^bj=RQyo2nl{cx^(PK6OOXS|eKc8#nftMJT! zxi-JB@_;yn47QT^ygPb^9$jvXJ_5n&%AGP@4i$D?ucmYGOv}g|WT%vl%eK{Hdnxjt zC}#3s?N<7=VE)VTGnVjfGyQOpkr!YkvX6;({Im`eu1s?rxgkBfiBwV6{7cyT-h1@!9iya&f>I@xE;hP>OYZkl)0)6ejklpn1Y zpbRbGS+V_O!>V~&-9ZWX-;NnTB$L5;ZK|yvae`L6HU)AsR9BTayyL}PnTp-gr+e$3 zSE&5g1ArajDRQsGz7Jw2=ceR+eiWU_V1m+!zwr3~u+)E@Q9#^%nhl9l+pzlaJw8@E6V?NvVUG^@o@Tw1xGsZG*ln>wny%pT`p~qsthW%y$Thgftxg^PCM%JHKM;cyWZF0KOK?DYjq{s|8O=57RcBl`wz!}0B!f678p*9K_VIkWRdW; zFYRQU14`hqb_m!E0jWe>kcPc?#nKc61Q;JL#bbj^5En%;APQ@S8om+WL@9<$C6pdh zoM&JR7t8G!cOWj#^{*kh3o7 zW>&k?MsBKWf_{H_b`5g`~+RAWRMp3C@J~ zdbph%Kt7AA_moi}=5G8h(AhHe03iL?XZ;?SboG;48U)-@0VGnBeOU^I)E#@;tK%I;4g3FuA`bNj6;-BA@F6uLZ8B4TmV80uC5|d{^*C$Aec2* z0xKdi^xi23Q=vu@D+>c1L3$gB&=adwfItvr;hI`XEBKKQo=%je2dsUIv*}AePl@do zvl)7SblBu^)g5?wV!LfXjy+Txzqrz1IxJ?)i(nx7j};ZPaNmgqK$MiKuo4|RtE6J< z(|v#lM`JFM&`}%IQr>fqu!d1wEb}*@141V9wDofKL*;@E(9{4k-Wt_q-F%^=i^1X8GqPk`_5V zEfR0_fTg0^B5mda*vBToA`%Baq!o^d|!I%)*vpH-(Gyt;8dAn^-i`=Lm7tXHO2}=cLrzWm$ z7WH++BWobs^Rr_@z8is4vD4McoxCB3SSA_u8_~ABCh4TU48|#8U7n0KWEu5y7BU6Z z%idn`!p-jF23%hV;9~eI(4(!#aNCkkaO0~j(fVJ@rA!-#P&R?*lTaCCO2P68+5zax zDh#sAc6@2W7D%Ir-AfQ+!`Fy!r8qUAtBOM-vc1QM1 zui5^k4AJv+%PnF_U*0xLEcO#6{%<@}025HapRLs}3gn9`u$^AVo)z z4|rMBIu&XE!=}JZjwE@#2)xVG_4%3xkZ-c!4B{H#`0W_9;vkbsZ-TyhO679axrb}CF_Z8h@QGq!aeikD)XjJbEV$R$rDq_&7YpDQGeHWaDv#7_ znK3Df39zutW%s2sBs(M(yUFIs4|x z`?l4LQ!}PcKJ`(PBz(R$!}UU;(;d5TJC?cYNA}IBQ2+=|RDmXj#O+U_8#} z_>eKR!b7xA{j*q32=k|^Q>W$pC*cP-BNF_z+FYQ#)T~kk52-w0J)Y#LqU`r;jt&)27;0`!HYZD$-E$rRAeyAqRbmD7Akf*6gFG89Y zh}H~OZo_fYzQ+qJFZbm*q%hu4DzKJ6(z87TZ?{l`#vz%SSA^jc%&8vmJmB6Dy^#&L zV!c%mlljBxGv9O!TJ(4YvDtgm>o2iULqlO#7Yd$#MpKP*)7#lhh43UZc3GSyYm%7D~w(b0n>M=UNUi)c9LEWCNY@tj!O{m zC$>51dDo9+Fa|5!t)6SO#3;eRtsSz=E$qLUUSd=I)>Z;XCGDF$4@A^27?XU4auB1M zn4c>uAAvZ-H`g6<-yBimik|B`VPsODg6biQiDe~_bN)6RGcx;O+G!9@gr##*zJxB$ z!x-Eq@-*)#M7;_=7vhs_PiuE-|6?7?r$CetDV%7gx}Nr~U-l%0_KXWXa2rM1NjRyR z38wFAFp``)(94t9T^i4+QBEoTv+T=>FrT0I|p$8REPj(LSsg60O-V-&V84-T;| zfl2EK((~{lJ8GOsKa`YiQA_TMWuj!4ucbFGYbDZ^Hm)kyKxM9Nbl1SEJ~9qYe{9-BAj(CYEl&&f!S*(fEq3AvnhLK&qqJ~&aeA<2gMAK zpOl5Xn<#9-H7c=}{!oKEc#J z>?Q{nv0!wc-tjFE_N`#t^t1-rO=|OgyX>b zcL^P#UpSzf7A?mxkcA#W#m#rN9i-WO^(v}{_si%r$WHWsdjW2?0I=t9d-dUx@oV3R z?Mf=sZk8h0!B8!!-^jPvFuua&Y4$3u))zNrN8AikL-HEEf%%^K#_eeX-{Yk>OdNA> zIGe@+#F;|NzzVD!$2c`G3A|mwAOmIlEMXH`TIcY{;xbt2J3c%te$R)cN(qbkZrwq| zZBn}`ekbbQb21StbOwoTqzA=?cRa0$?WLc{%`OCSM7;=>V+fvA=rE`akD2?f6G`h2 z%~Iri&5KN*mIL>10SRZ;u>!AhUg3U9Ks3T~O+srwX6oHvu=80Hos;MjF>|@12vt@B z%$ZY`x%KJspQ;AZ@*7odZ-Ts6))_?<#oDCm%a4&x1Mtx{)A)x|ibU_DZ>hk#W;=Tq z(V7yP{SiIhRL2psUm|9^&rMS|PDqw^nwA{X_!gGB$-c$1`1Be1CqxSLSjIk8s5!#O z79Rg3Ky52y`Z50dVrqKtN(H#s0ddOy2ZNrx**iXy70fC|-vEfbPtRe6crjrAlI^FK zo6lqwQG|T_v|zZ*#DM#L1|B3gdvDTSu0}}(-1O3mf1JfcPnKCEHCE4fYk>St!cV*UoP5+?ag-_X_z`o z^@_Idta)2W^VI=oPw+QtJ{-j4=+iaC?^hma{EgUx9b$8j>7by5%`t>g&TupolVEy1 z1))g7?ao#;`wRi}AZ^N5Y2#rJ+4wA|1|_ILJ8Q&vcP=Ze88woGlWv+x$SuFu1+w$j zwz_c+?t<2XY9o!`}e?uJ7jUVj|qbB&@4#J!(SyPZrl6e(mh; zdl;E`_ERv2WZ}Xsm4?I;QC@L0tq;m?(@-8^bTsOaL!`1BYcGKQvDdlrPU`FNNg&fv z2E)Y~;SH{+jU?}B-V$9^crr04Wb`3r`&YBvlYoM#dZQv&^%&j+#^$_P?@Wd9(3$L< zW%9Bkhd$n4TD??af1?t99cuB@so>TK0_v=bkdQ;}{PspCU>XwTZ4WXRZ-5 z8KVW#Pah&S81n5azU}9&t1|QW#ZBcVvoK$Ic8SbIsNFeIQ`{^_3_yp zW|w7f1ykMz_v4v<1%16)xVhN%514C|FY)Kzbw6}R?^Y&4yWmjF73@^_M`pqw@6#Ul4vi;1f9}+U(LR)T1q~UC z`x(@6gW{7PSXII-?Om{;ybFPim=}8adrcTqCig%5rMq3;y5`6AgA`HrltKKS;ET zLjXN%+jYC9dtcDUK-tv{TKdN?LMYegNh$#9(mtM!R-if$OpUH?>q&ZL zH`GB2KpbVp8dUry_RP7i@vysvB$vL=xiWr1wj4lXcdjt9y{P@n^15WT23KuoD1-Dq z4q%qR;Eov$`l!BDirV@$sVkpTI*+`?_Vm92@uN4j)5*tUb3n==W*xjO<@Pdu4;W5(rz^0;vpQpj#H@WVlTmg3CpUGjIc)CIXCpVWQrmr8L*i zWSD)?#y3}43OVWNci)!VtLeY)B|8l*>l85TlF}N5`c!uZbS5Ztz1~U>gH53W{Y*8@oxentqCFAZuRpi zz|%rNMgz+-6v^HMl6goE)kz`$T-&WivoJNk%2s?EEnEH|$7RRrp@|yB<;J@q1KV)B zYY<%Y$-vj8=O@M@*_aL?B0hjGx^00)o>%|)9qy5k$}!Zq?BEbyU)uXe?1Y(r}qZQGObQu^XXGYvcaGRxh4=oPUQ}6 zrWUK1w8ylkA?){5b$md5riey-p+w%8d?1?b|dd>tPEv1Y4Ds0XSUz;WhQ zzar~5h9#DWS8`b40a)?sxHS75CX%~k zxJO z@rJP8o@Oj$)=aGuJAD>BmD?*2t{ZIW^Syj@*Aq&5s3l2X{{#@wg+`U0*QX(*7egLE z`?Vrt5CuR$ZwL_X3RziBV=rz*{*VB6N^yF88sQ7@YqIM)AIP?+fkZ}l6QmRcpi^l# zhaX1!ota{^PXSVNllJdb{~Wysq$}-IiyknomPJzD<5Rw2&KxqN*+C}+=E^sL!jIx& zKS!)}$#XRZ0&d4yekuol5hF&yjQ)+2uj$wkTCWP6;Eutv>#g9e(i?Ea_Fgux_72f4 zjN7!{+J}UkuU=%pf^CLz#q$M0GexhCs*9Fd3s66~{wa`g!%EY__1*={`j9maIViz_ z1104N>=_W(-Ev|2$Te4vEx7o?2?Q+3d`>;6L{!Oe01azQ^WCCaywPv1+%HJa&?Hjt zc|=WD=a+c{*Tf=uq#P7ND0O*pzGn@knSdon*#v}M_*#|KAS8b$WEE_EeJjpDm|>-d z3~#vvigMGj7GP@%22?(3tv6U~h!4teOOY8ufDoNMYFGU0J1^fF0DtEMbk^5Ti1Rsr zfZKrr*E_i?y(zf#TD=KecfBj`L1ldGbHcp8qnF1Y-GT|W5DcWYy9tVtABzb{^A$B> z{^V79ldj+%jA)rgj>%G_FQ>TnFZY6lviSuyX$s3-9{il}H_JF*Hiy(z zdf8t}8!&z|U$ME5gHLw*>8-br$_ z@1};G2;_*3EH^9<#us-gFoUfS6;gmc{>8~z`f>Q42KJTOZ;7wIvbHzov9H2&=+_j6eX*r2ljpUg{Z0NUS#Sd$(#HcFLYh%Vw(!&lg6pbsPHzETHD7w0TmV^GYRy{t;o!S;wGqP<+cXU8f`F#-6w&@mI zLnLhv-vQ6P#+B!)zp?;?m1!)|8Ab0xaZR=BSQH&fvI6!QZ?1UmJRF(3NQ_v_$aO$q}4r6*7@dzlLJV z_F=bN2B|L)Ap_Y8@wc_&9x!F$uR$B|{w{=;s6Te@wP?97>2(PPn%;!2_ZysJ;4k2W zLhMv;FM5Oh;3DaKKmI0lU&o$gMEucX|Hi1^8!jS$0Dd6-j+R;62z80&{CBw4Oj|e$d>F)#BzHR#wU(@euqKejl8^W#N1}*!L-R{N|OCsDq zd^XHq!C>MlzCa_EangI;kCvdB31_NuWQn3T67NaHH9vtl5iT~ljqHq3m4aBypCM6a5R_jGviW3Dy+^c8&}om^ z{f-s1002wpID*M!b_R__0s&bwaL5WbckrkAnnNAe2sv4DThXn%PU~+>=3judfW((q z(sD!XU?cO!i5Q=8iWE>ZFurU7m1&rX#fK;B(3qb++GyNh2w*n6%SRh%JxL@3yZz!9w6;nmCsLdeWiE!LfH^GEN{h7XEP_ofQDbqEMH7u0le(vx=gVLIizXQrSzXMh2Sf8a z?>~4}jTq_ui;sX8Y^izGqk8yWZN4b6g(xro1o=r<%S%ZMhP;0 zbOhMFr#2^B?DK2j(dD&yw7)OivjL2pV;~fNFJ#A^m9LMuPRk`TDke6UpYpCR2>cnn zI@ey^H}A3;6J=ghUKM5Y1DG8@Ly?dxhA;(F_(x@T;G2ds9+%-Ly>+abdnsl1M{5< zS_UY3!kXHyyt<5pfTNhFn!h_wX}b#f_nh_TUgsRuC3&fnj#S7zlwNI#sWfcxew(IZY;fa8z;x^(17 zY4$oxL(`qylR;1G@T{y+sC;)5AvqRn!Fa`g@O}pnNnVM@X&E;ODAN=26x2vb@0NK_ zKjcsnNhKl_mKPe8jlr%Vd4--P26CT8XQnGG^}c(Y%b#3XXumGMu`?1|#2htO-o@LD zUtzbPkI@cl^ERA5*g4$=-J{WVt)h5DyEgm$$?GCMNBu)=M}1NDd{w14$@+&M-s9e|Vnzi$+)q2Jt@Iv8 zh&iOs74>xNZgF@EgVkhvb%m#zk>cT+UjfppQ!0-(Gd!ePTKxRa-JkTezvMQCmuVf# zf80C3Hj-hEZ$vo-Nn>ikD~9t4r!C3f1s@G=DE{u(sK0>Kzc_)v_n!f8v|_gJz*Rft zO=k9&6v~PZMT;-ft;{+S^S1<8Gy!2KAS#=V3$n8Qeei$yF<@E7|KR(=V;!hFQxJZfM|9zVO<7urk%#zi)&F^IV-Zzjl+Fk4f zfo1=HhNd*WskGm-jbjDcvIn>@qsB(H&i^gE1n>=kmt|A()~!lKWZlEHZ(QUKdH?WQ z`};!NKvT>M1hWzgFuv)D4^jT!HBM=KD<1>u(qoZQ&)>SnH_%qU==wMR-r-pT#W&)< zDCz=8>OZX8zfb+o1^3)v(vc2?O8?IKBNA^M=w+Q{9GCvBLu(_)Fe=^f+P8RK1%k}} z_sM{WKC7QODclWmeE&<6`txw&NKiWWo!^y4gGP_3k*v->*^}Hp4Ya7@{}zj3f=PMr zBeqD~cdF$r0nOH0i(gpAzda%Udc$t3%mUcjL%0fnGqL?Vk#YoHIk*|Ew+2$s|0RXI zmqO?)@Ai#DojgnEpyK){jEYGVqF_uy52wP$fO(OOSL60#!&?yJ`vkzMcWWI5e9xfK-#pQBG|Fx|=UA?vUwO z{98gfx^rWx{ke~N=>bm_Cy^Zi@*BBmtT(ceVds(BmZ{~c>mTv{IH0%!9Rd^@A&~xp z&l99V1N*gx)vINdB%V%?_98j*;MM>J+H9w8WQT(KA2AqgU}C1h97qbjbK9%)Ry-VV zd$}~dsq1KZduRNa!_MqM5{4JA*%1vY_1LzCmj+WS=ZRCkBRB@kgX4*v!l!C@cL1FHw9~|u|}qW)#!V z!8qaXc)0bg8QcK_C)BO0?!CODEt;PtfPp;#6`zF5?EV;Fv2yN+0cffi2vmbX+DN-8 z)F=w9y^WDf7xii4o3z7+z|jv~3DEOyB+K}4?br#ZsOKOl9tg(pg+{&3@69piq~PKB zme{0Y%SDiI4!b_&)gvYUi2~kauf;hz+@@~G+wonStaOBdRFS+7bjv`L>6R(mcLXTU zkbtY#m0b(Y90BTxh(2vROl} zVhPX;9gN@ixp9I5`;4p+6n54bmY-*-rwXl@=}cJI<1?mLJP_uym}+q{QhtB7!EG26UZKMP*GrtT zzhkY-@{Mz|Y+iV7;8YCTWBZdd35S_ol{F%Px& zsit&la)Z*;Ye9E6^GD31oK_rb9(V9!hIH#attP|XxYQ>3@r~VhiurDc_TNsaFi~Zy z`mFm&Qo%YGx4P^fk?96pSKUDqga+X#Q!A$l5(xzr6L&yZKP%^V(9VWuZ3C4YIY|p` zKUl0}ai9ln%$WJtkd8!j)4;-5qOp1y?Hou72crG4)gQY9 z=`Ct)Xu@#Bj*r~pGYF+q)-x{c1_&6@1Ye`NU2aSwwD2;n;eaId7t4G3U~V7+0YJ+j zNS(@IYynh4=%B697XYcFdA_18Pw{b!hg=`(C&2n3Z(|ckD4}LzNCTcwW6(HY13bxM zpRF*@?uS-In%zlJb>s?`!55H2LXo6rK+$H8a6naZXCr!xns% zh-U~WketbxZq=o~w2k|AkqLdlSIa;y$hNz34#1k7%kST=)e~aYmWH*4FE(Wb1Y7N0C4B4a8Dly4Yn(qC^rQDW{3_nM ztxJ~&FvLQ7^E>&9t(!hI@DbOzi@hDD@q9mJY(K?RPt{nhHl;~cBKJ+vz3{322Bn)m zF)Z_p#6A>WQ#^FuiDsmO6*Z^f97o;q#<8|yH*=UI=D4{M^DmE$ysM64osCv849g3_8h)K4vE+O48%t!HGD(HQpu}skn-y5}j0? z+Ic48NJ)woW=T%k$ltJMT4kEk$JxyH#3anb8UzGU?bbitgfVrsxtVmlMKFpIW!pOng5Hf~(EZ>Ei2 zx)e(5ryJIa?~v?OSJh4hRM|X5}xHE zjdAUQ;8391&SS==ZKTW;fu`ZpGaCm2r(DNX&&5oZO8@D`{mX$`P%KhQn7 zL$Duba|F4=l$Su9->HoEZWbVp8S$iM)Uqu~2txvK?0e)81ig`Q#A2P!%F{69(_EJS<1A%RI1G9G;OfTp&cS?U;aGJVLGUOcBW8>QDP@5e04FqB@fSXVnhg$&9Dk;zm1Bx#?r5!I{> zZkH0=>hOaB_qLPt4H>K#SEch0ZtEM6uzsW+1%l?VL?`+7ZiFM--tRbANo3V`^C6L5dt|w zVsG`yq}s6|mJ6ENj1^>#pWXIY@ue$BVNs2nA*Sv(Mkd?{S%9 z=o9F6e}YLN#ic$55lWA_JVBUsqxTgmlAcjD|GiCq4xO|QsBwMExb;gPN+C^2Zag`~ zh$z?Rqg;uww?l_Q@~=z^t4PV3%_ly*uB^J8weAEob-+My=uo-8UR#!bsLU~7^rN^4 zp>7aqO>7utbSdaPb4F9jv^+Kt=578qZK?KbX!M+j2Fd=ca<;KVgN=wv-X}_yjq_gl zZWnc9E=$>&Yer$x887W>qM<||kzD&kQpI_LUMArphnjxLzFh$QLUPgE0e|zU2pT7W z7%}A#Yy%g~qZQZ2Lus0wj4~6ncw+ga$)o8Wy%<8``zEhKdFRi~A7;d>ww~?HInmi88o8yB z-X_z%5*kZ@zq_b5dMLx%wmL!IHZ@_q?A-WKI>mi@ZsCPnu=&yrAxDfA`jNp1?)f^b zJ@@{riH#I(Ty|wo$<*mWE9s4zs1FS#m#PP=mc0{)z@ketbEWBZQp|^54t0a#@YFz<3>^OpT`~imf=)P`yBr}4kZt;;N z=#ll$pc-*G?C`et?~v7dgML|nAF;N;q`87Jn1WT!kVhjVqGWCBteNM7%^T8$0_F+XVzj04`Bh56F9`d7BT|@m z=i&z*ScfSg`7L9-ti?mSex&sBidSApV)O|Zu@Q)_Cd+teK7p<9QLUHv#1Hw?^Smk~&bwmB_*)op= zcsG>{m!IP3d-q}6WxlP&R=Ul~;aY!xyNHMV#i(4f`PyZ70`kSfQDrr5VoYSL<$oi_ zc4VyY64>Eow((2zvNS-2Fu|Ws+2RCDvNvnAfQ2c#hS`v7PESz{VO#%R@=DQ z7Un*c3b-0f&yK;lnGlb`%+iKit32bJndtDL2B(M-6;z!`n0SxqBBGL4aX1tD!(KT9 zA?X*9kDW(1kXws=8UWpZpo2GJtpX4ZV=x7y@3oi+xu*xy(9<9G*Yqzh8#oV<_8~0$ zkBUMgKPiycs~>+b_k_EUe)sJ(W<|_7<_6E18M5v;{s0tsX6CK(IvCa$gzfFgxsCC; zwg>~HsGS2!hv6I1rN+X#9whxS1yMVwF1O1Tn@@_beaU;45`#aqOj2!HD{Q+XT5scX z0(Wo(9J$!^&Pi*oM2nv=+3C@*YfpM6VTKzMqm}OE7A@@bVnU3hfkU->jg#Ep>bRRV z#+jKBZuJa-{_E0VJ0n`3{iTu)OAUj@pGRRacz5>eP!N;c6R?<6kSo-S&|Qbw@#AVm zu#Th?z2&Beh4-AlF?>G}W%!XRG5VFgzgDQ7c`s zcy)+jwzYZE_`ILHP4gTw6SwsI-tThdeJXj%PsOwCWrCvOt5P5{N}CtNxdmYrFludg~LSiJ3#u_qs4MtMI*k&Q~EP^p=IEH zNK#^t>ub=wGWE_RRc>a?9-oBaBmde7AgW>bUcv27{4{s4oy#4`OVj0dDRNr+=P2KoFF~y_*@ty~-GTtDI~@^zK?2 zxSCYdVh$A6K}@6C{CrYKty{%30m#WteLY%VK&Xjcy^x)mbHACz9z1*EXniPuZ9AP? z^FX#xivOhNv0le}a|`(r7M%-GIq5>h&~e>`7_LH(P%X#QeEEl0otJKOyr7FF8@>5S zui0MdZ+MA zw!8o4^ZP!_iU%92&j?GFYjp8<&lPQji4;OwxZ1W-M2Z0%PhTMFzB!yfMc>%)PTFDP z3mx}SRhLP$8cWGmoDO=sXgoznSGX?!C#A`%<|jR)^adQ0#q*sshwE)PxUP%_ymqum z#Hth2XR_t?m2oXTb*Jua2%VdxLM|hGb}tko9*!w8tdDe>9(R_`fA6B~zRy)lDs>c$ zf@xB35|ho*Q<2ql7!^c)*Oa&UP+zkkLP5|p`UmEoQSviap$cnz0JpZ_`;1tmx>2dH zLh|ud!(^5eaY9t|Rb%edxBOOARyW^kSTe|w6Y4WY{@>jGB$3n-KgvBR7= z1nLE)lmaX~@i|vPgAWeUTyAZR6fmq>MFV1`K*94)H+8mRC4nS0Vh=J1_~2Z z_Qlt5gONo*&lSYWTKIm&-GrRbp|9DgG@)`a#4M0wwAoo$%&^;*o6+Fb4w(V+kb}^V ztTcUkexCuu-EDUR0F?)gG?!FcY z?T9*!WgI$M`*S&r#lAeE1M-#F)0W~VHw7GDxt(1^!^u3)-Jd;q|BZs)dwFdB){c-` z`XpWbtilex_ZrWPRtl|*NDu$x9g?x6)0SP5r*anC7IvlNT>cpF7{Y`cdB?&UMurxH z8oI1_3xaU$x!MeQ4wV_f6Bk4e)}B^pVa3jAECmXzwKNWvV7^_^fm&f+>L&_;98!c+ z+#HmiRjKP~C+SeJcgO3)6@%uWSIA!^XA>W%UjVK!OGrn@frn)@?V;KmtRzE<2)1~( zJk5#yM{|NG$m%{>h8G`Y3NaP(Cu5!j{( z3)T6fL9&1&BCSl2ZcWOYj3`U)?#7E(5e+TL?oN&4Zp?~KbE2^)c$#F2(M*5^O@XT?x##EyE)bU@!uX2=0GVDX|=qWE3IbjzdTRygTAd+r4;yrEiFmPm7H_>L1a0WX(Esg$Dy7^>QvdMg{rnoHhq}kR%h8tXegDT7Kjpl%87C2Oc^-=}xSKe}*qm?z%C6bc zf?&X|)krVp9#u>iw7&7{22w&We@-k)5kAZ4za26aA}EAd5@#$^js%jq()$kX{v-^5QgX z4Ldx6`ij1xz;j)%EmwH!3xOVJXRsY5o`(o;w_Zjhn*tM$2T%ltepsQ(M^*ZrtvNui zzy*pWtEUrHhaF)#q30*N%LKkjKsg}~wL?+7(?F*p>PKykPT^T3uMQ&bT5DQu0F?_9 zUimnrJ_mYmN+?9ambe!1HX8xK3pP)vM%9;QvosBu{z5_^% zI8+4~aYzB^xg()0hd2QE0o@HIBXrqax`y%+4bZ3<m zkb6bmY6d@Qov{EZ&8t{U?8H_iz!h~tA*!38eQ@C6Xm@B+91 z@oEAxHpr}1K!*}6vQCVKLv|e!P>7?g0rM@G5ym1&eD5tZ>LB?- z*Ln>4tS^9T3WpmY1R$`EKr0M;ZM01BF=Z=VMlz2^oVnTpqjFTrr2`UrZZ%-pc(a=9 z2h*^eNSCrB|1duR!4C~U7DWUY!bykypESxyHoLkbHg2mG{A$4MR10?$qY`U!S~4Q5 z292W6O;Lr_q z5ylv}G78dy_|FJ57hUz#H|WnFGTo zN=18aH63jnzT+|NA12wd>rZaUv?n@h7$HCNd05ACDMcNr9};7iQ$kvaDW^gaz$NjV z;oDAvveBggiEL(n9k3V>JGH$h#Uvby55Qth_EEgQ`Sa|0+25{~!O~uR^&fKz+gU5+ zbe}>|2>lpUc&GHPQFm#=#2*50keQ8TF~fvyG6_8&AH$9LeRr_vHR~cIK6EDv1pwQ$ zFDdg7iypGKAlbaRbOIO&0SbwNeo(LhS{Hgd{xGdTF<=KbAyLm~Vqg;l(WgbT8u}!` zv(OE%$#wAgSnb*YNluI0EPj|I2pu(R%atbnt>QyBu_f_ zo9uiVs$KS8*LOnpYG{`pNMj>1X9{(@fFclb7VVOnpfikC8n8yoc{kgj@CL^HB%5YZ zM7c9$N(Z72HzMZKxQy%lL?%i3Zld`~Ap2*g&zl$F2QYVnm!OlSXuxdhGYWr(v1Dr} z;Zn8vo+G1(4X)+v1|*#Z))&G26EFFnZhA(Mvw&%nvIqgZ5 zN85FJmYBN>RI?|2=v;5)Wp!U+YjY{LvD2}Ht;$7PeSLQ)p?SCX&F%ZTv3+ekKA(dU z-&%}##46MU>5_~Vfc>?(zyWYJ{9;k%bj%-rxApzL{G19y(*j6+5K6}o=L)QSVkfrQ zi-wB$PMwyn*`)1o3>p#s*Cf=MZRjf_frjds@b zqjz32d{4()FI9M7ExL}Y~93@Gq!OlLK~Gt;qg%Hgd3xLu#l+&`IBsc%UNKy$^YpiBz=RoJ7b^k}sbG)p79aZN`b?B?K9VV5kk{ z#8~w3O_&(2Z&!%Z?j(zI@H1X_Nj8o-sDPG*x=GmSI3n%93OAYg?<)9ST>ymO4YXOn zpQHbl9=xjI;|Aqc)r)Zn*^x`=wv+Dw6SI~tQ>N>}M;@6k*n;I`WWL~MrOk28n+94K z#x9_R#;RsL07k`De23EFuV`+vjb})9a3``#7t$6i1F-`(b`0;r5=d>CtpP)9gus&* zkAK(^iN~Jzre1c$uw8Fw__O`W6U(>Z{S+)P`=C%c2HvZN-c*2ZVdM~xGi+=^H@aI& z_p);ZSR+d9r>iB!EZ*16t7;a__Z-;=E$C)MatTO?s<0HaEav$5O|MZN z4mF${typQRi5}nIUAQWY@i}8uPE}xoTRqj_jn);l6+rbe#}@q(+~R7y<8nV#rE+5bEU@kYYS z`e=b7-KIY}TE?hz#GF2*la+qD4Qhi~7s?eeU|cyW`y462vO*7bQja#ZL( zC#nPacrkF!rCFO0$CzD`sDt;j%B_A;d)7#nFYS~?(nv^mbCNM z{n%jfVYofoipe0g9jUDFO3d(DM8$noW3^xplE+cbBkZMb0?Xlthc`GCXo7?8lOgNK z?1yV5@D*wWn!?3(oadvSw7*57jmzXV?tB@zK*i4xz4)Y4kXcL4S(i)26z)9YFG(L< za{QOvt-lxZHvPv=V^WRfyra)=u-tm|dk-W=|tP?^#tNHkSfv{TnX|hRt+qO~A z`nLM9Xrq@dU%lSMM+%M7zyl>g#Zr#JmM33%N@EsH9XuRF0p#Ze+tc#e>OabEYhKJr zk2iZbTtk?3$2(N)MEJQ{j?~g{jaG#Q;@NI_-RVkR$+L^ zO4&yRG@(i-J%27uzF+4SfHgDMJ$Vl~m}1l6v(hn1S-;%Uwi|_XXWc-D^Tb8qpyc&w z_hCh(A^i%+XzN`HAmmKRVI^jU0HcoI2+CRW-kjZ+HiXGyu@x%`_ZW?hgBG42*E$Dj zYvlEe0q#-p3h+Lld-O>&`au#Lq1&rbL1r?qzT@zZaiXoY7QcVa6>O0@W)03lGST(X ztCPjo=;-Zk$G64S+)asi*vTrTN^j((Mjb4t##KPgM!A5dt6C@ri=;JlI381n|1ZE z?;|Z=FoivwkZrk_k@vb75UrY##3bBdbwNiR<)2R{-z9Kg^f)CW`1Z0Mq`hAM z42&YS&|qBO3Yy*=IP5QYk98Mra8Cq+55fi2mc|~{^S#+qM2b65bf_{ur=D)KCE@Zs z_UO?Ht-XBoDqhoUeYMGBc+Dmb@;sKYrr~a}-Bt1*83uKe0ZzjPQ^cjhKi>WEAho7- z?o%rlVPfrYu-+K)y0-A%fU+#n5pZRstR1%2Hv#=cp&J~)B#F~$bLw#3t4}5YZ;@}^ zrwXW3XJQO%kT&o5bNGxjYqh$mU4Xh3=2-Gj@K)IiCFhmdtB-0n6G26{HN1T%Y)kVn zQ55v&lw4tBZC#Yxnsx6GgG9-dpd@(8`sOYJ^89S8=v1y`P1qzLv2OXOeEv38T@Bu; zFM-mUoYO@wuyi%EQr;60M<4bjg`h;UFQ)xo2=d4wbFCjL+^bN5`9Yy&U-{j`zuw0S z+`-it$ESJHba1Dp&Sc{N)#sCKbVeGt z`4z|w3h{MkDL!YAok**Vt~wCVELMsj_qPY3Bv`Vvcy0H>Y1ME~!d=;JB=WoGe4uR! zvB+yb2@M>rG8x`4j@nE5uym5nN}TX?hFnaEqTg}rd2I@AsV_Nk`!jPP^F_bTYX@3Y?jl3mo?p; z4EcDDf$T+_)5Nh#?usQ_P$)GC{f0471r8cTk^WOoP#WddRi$=2!~&M%>_X}GU`q_z z`e#wd9mzoWf#?!201k3r+@IgTWa2zQJ*-ujT?m7u?`W>nAW+N6MAUm?b^L4$xd(OF zJZ)EN-v@s7=7xBj02g_Ac?S-y=A(k&w*c`*;*?$1xQK*<2>aFwaYQf@f{3cRYJ4j8bHgfoXDtrKzE$Q{Y8%zNPS!&ais z6rxVljXW8h-cIE7obPxTlEw8!axe-dZ+MKI>U?cJDX((BHDI!Tfx6MM?Nhbe_!)bf z2;crQz+)YeMTxWb>u`I_KG+aMcu&0TE73u{&U9_cFoJr0sl$TyZqUowHuKUEW610H z0_^LK8~6$7j|PB*UFNHG%qc^>Iey2S^*cG5==s7ma2|WfI2!3P zgH;a`ihJWDH={v%2ASquX`QLub&wUrxVtea77b6oz6Evs;$|UdSjjhw1ZcHx{uH3=C^qBFlpEF0tN63SPFO zW7hfLSv}50iFpW}3$j#^VO3+SZHn?}uGuNNcdbgO1cgAWLJ|f-IQ3bl6S~fi0W*H~ zP=VZ{_$pXD4qOk6JQg9Tx6dz>DIzYr0-YX4D zU(*@^@Il=QJBI*5fU_t)b;$e^D&Ly|7qtIzy(jLr)q=A;ufHmyS3~*pJ^;37q%wkusWO%LRY&A%?*+-!#|gh~84m>a0Eijde{=@e zVN4j2VqgdKHsx8*q6WD^#p$d&P%*!|Dh#(_?vPjXGXvPTJ#dm0_9AlA1~V9%d${-?b# zxRV)ID;AZCKs^7`JB|u~U7CPoAO!5XjcqDhPv2udg+SxsLO2aGhD>wTr(z_DI7pH6 z-R6NG@FoO$947#E%^W^~e2Jm7;7jv+?mhHANY=j!vSbU=0Cp=H(gu#;uKq>cf-Gpx zBrJg`Tc9ArTdV+T+S%bM@CfbQL>%3Nb|h8z0ew`f&}OX5bISWlv(Rb9RO9$@;h~h z)CJqZpOu2Wj@N4!$U}duw`|fwsMvg@@Gjrm_aO|@BQ>PFeU*u~Qp8Izghib0FZF-C z=XXplur!a!ru_AYr5JIiQ|@ST?BSO?tbs1uCRn%0AV_A*$tYHe|B^myZN)Q z{^5%>0!uisYqWHk=;4v!vO~%Gw2nCjmftM~GykP2e_gUhC+nTfm)B@NR86~Ct%8## zBnI}xhmiod;hkegS!bk&Pz>Sbm5b>hFsijZG zhI!h%k?hPQ^&c)R!1fED3g~H8&Fz0Ur@Ch^2Uq?0@bS{0B=0shIUnmpPJ8eV5PHL0 z{sSr{o&!gCYl7Fm^G-g~)lG7p-~F0Z3I#VVjtkFlRr70eY?w94&21Pi7lrCa+fsqs zCWcw4mtm&e4aK4yEn{(XjSD?8De|EviOeTt?SYL%y68DJKEC09UilT>a3=S zaE~exsR1hnFH4;?HEH!9??g%=4DL8uYN7GWrkNs5GUAgZmUTb%BrCTtZw?~(pg`}d;F`}^HKgjGsfPvd{l}M?*U;Hzk(nn^?i*P6*;FIi_wUU!`R^V< zyHK!y`pW|k}P~?N4`=CAS?ZpP;x@jl{oTCccFankek;S z@tgpxhUQPK9~-WJ+?5-U!cW=?Od4c<`U?SVUsLA-gr%$2FN1v+`!#pQ1SBEj9v;aB z*T{D@lLv$m439&Q4K>E*!*p{<5kdp3Ft)XmPY}M^R}LRc1RW}Gnr&dkLxqxT*nqc7!ShsU7!>$Wl1L)}Oi7qHXcn3gp1EWhutDK45x=bT!6aAMhvJzL&}$+(>s*%-3z)M;z&{c zsBxKwvWp)b?l_zKOq$a6O6x|P34q81eeoEh0>-qcIu7rruxfZeH1;G@#!U3Ea=0dQ ze0P;OE`xyXJ>9p@G|hW}-#8^3Afc>mw$g^eecnI8A7d}p@!R*ql%OyRVMbPg^5) z#nD*2|{rNQjjiCq=Oyc0&>8-2zR! z@y65yhVQSn(uF02=yYx@TPw5VB*N(g3~>{-N(KQ9y{`bJ40C*mOxw(^V_`h8`qR{K14*TQ z;D{1jucA$>D>v%|*{W|jr=Hy^=76$kG@~C5tP-?@pV4r?9m1p(o8W-^jf55l*}ui1 z2 zoqj7$-FSa;Nw)c4{PhMJQ7!1Vg>!w5#d|?}UwSHxx{$ezf{iB;$xQuW%sg|6EUmSS z?8)>Ni1oPoNxkH?ca=FOyA}icX00cbT_3wF&6x*MW%R%)fz3J2d}yrs7#%Nq$ZFko zd4t+*QzGS5;3Te64=(Y4_Dkg_h7p9v=7AuSbS3(Nk1n}a@b}G?3 z$u+)p?;NttT;#slQEI<3NZ;~e_H+BKk6+erebzC=AaG+WR~TfA`wU-*yNguTyES`A z5KCVkdIZThC&2Ffu=2}_p#{Oq<{uFA6Ew$Ev6bv7hRb%{{TN9*f9+Z5TWCxu)infv zs?`5OM|cW{!LKlXOkksz>kUCDFiXn~u0lj;)9JztW&aqOo56*3Jg|VsL3-Z^kJfEE z+Ne5;k@`riEQf_P$dY;0MsU!{&mrgVKPAwTgK!nmoVebJTS<@ z1Q|rOOF-1En=lGu;+qgo&WU|}hC}Qn0}oHtAJKM1da(2rdaJi{5L{=(la0PL|9&~t zK>X+#*?f|6qyA9i_HN78Q~-?&*>CIv_fp;KUE_?RF(L~XS}OrS^V>bxP?hS^;k8R! zA%m$TR?P|G3UBSwH+`K|{C#(wm!V!S6*X)Mem)Wa`OvU#Jr^x>|3%g|29ynp&U=&Y zkTUA8V+3quBex;N?lnl#R6Z%`D=T-Thrhjavb z!LRCngVhH~$TqVXIb%$PSG31|95L$iW44mB4xM7D7i#Wx2ZiZOdBd`BMJSk$?2W^v z@aq`Q!`oah`IBF&JV5`cWy=hM)dTzmf>TXfoM$cOUUO{v0O!fQ&5C!yTw@l&-KLJYx@0z7#adv*HFe7kqh-o>XBdr>HDenC2J1AbrsUHId zfB-vBp0dK&&w~`uIX(@KydC$I#Bt1BHAVD~%GLe5VVTPR4CRkENa}Sn?*? z+8v3>S`13}vd72x^cr4}Xwomm=Dlf>!O(@Zc5)lFM?N{eJy|tkI=>g6SeCQ68xhi? zG3ds-MW*&~l{;y8J7HCy0vxPJWCbGjWH)2P?eCDy?mavgh`*!xW>I?* zOzZCc46sS^!XjF`0wR| z2OF8os%UmR!&T+!F>t)NcGxzwjp`hiS4Hy-&7frGp+d} zIVMe5JZv7Si+|hO{qNN!3kz`n8t#@uEYq60Ith*#u?V|={sBBr-axmFhEk1l&ROY; zb0O-ig^svZA4e1E-}0U>WNoB(5Uyw`=J z`rlXmKV#*6myOe)85^ZK_?U94`%fszf6t#^?@KiSAR$$FPq;V^jOqT-12AB9 z#@`Dr8~Oj|vXN1$_twKI>i>E}e;w7{VN~_vr;t*?lc4sN^!6r;Oz44^Lw8_P6~hh;r}%*d$L{#m-wMPc?UDl@Ot8nLBI%jWhcUN z3Bu`PcfJ=}5`O0BXX_j?p0HiCNBhU|@@{|usOO2BO}s@Zv#>IuP8`C(-+ST~+2?YY zy{Z?DRZNrMi;%U^sLEh)5Dt`>-WyUc8FNmQ@P0G}NO0JY-`?@Z(7#@9B>o{dn>GTb z2SKJ$a?IQ8mpUH`&999M1^&7)BU`JuwYV9qhbaI_*R$@J3L&5(7V(qWt-2 z4S(p&zX6PvPpu*P;fxz#IZI51B#l83j~^5mmTJ## zQTN(m?l5RMT5q>827u}0m+0`F8G@#(z>#V?#rNv1WAyiD3D5ha?{Fx02F)iD6ZhXk z>{zcE0Csd!=MUlE9|Qo+KR;i_j|R>B!$u${)4Z@!S56$w1EcSSM9mWlXn(BraQj;f zo4r!FsOzWGSo_c+49fCr@{%i!0hZl@mjZ4;lEjJhn+h~jr@EOKIxAs)@RBwJ-4_i9 z6S8rEG&#q%Z3fOO05v_4gaqT8&addB(ZOiC32fG(7);V99{z@m`b@>^A`@&mW>W~A z4o6z_of7jAE^V*bn`4lH4~N`1ICma>W)fI~01se6B40nmaw3^(k!$YB{zqBP!{FVX zM6J3owhq`@S0O~K9Dv`Az)`9ZJQ_+UHzOa&`vUqE#j2m~7gJmECx%_@d?@DBqeR;brs!VG=4c7S7I_f0PSFE1}ik=O$OaZI@j zLn+EBrc!VM{3ZQ5dLU7>z7v(T>hX4}DtEdq&xQq=IeVP+5RDvM@HsTZqRDWD*stYm5 zwm69Y<>o~x*C#;9+`8Dm^q>VbmNlR``U6D&=4yW9)q)GCo-{A{rm6=RtXBbmVOLeL z7%t2HDp~Q7-PFQ$QR3~cmVN8OQJ|zM4*`czu^@0Hvov=0B!_s7R;;P*$g>I7MX!Fs zW6)BSk}q!5gKO8bCa_4m+0oVLIBx$q|n=!D3#dftz>*-}lLS_1jBU+@V99UE@1$Ku*AW6LQY*gF-7jD%x|sIC#O;DzZ*~Ar^sV@qT%jBcKZV z15}qAYE(~yRR{+VWuK#sgN9W#q1B`J zeZycTQGN_|2kgcXHB3>)O}WXjs{1Nz-Bv9@q{_Pvnt)`IDj(0c<@OmgQoOhO!NJ{q zOsCr@yU+*{tAm}++n*VT{D8;NRwjdQwPp)y>tso^1 zt@s|C-S7cbsN7>wlZyPu=Cnw%eC%RDC-Q@$YUn&Z`I$*B&r%G9h}%hYf&(8xq|&Mf z0j7_rQf!f;P=UTo+gkJ-?Ky}AhupBYijsr0eM*~7EuPqaQ!(K60aKTMg0#)a?$^;6 z+}9~g<+fO-NSOhSVN1B*C}2lHj%hEz?!(S`iYs;HjaJ721hvy95p;v7f~~52u$*t5 z@{CJbZh8Q&_-wFer()b9;K^QDkOf{BrD?rKj5{Vh%Kh`s?{zHY=FsjGzR9L(i`?e4u0x0yMt>ws*pm6q;A^jfhCJYj+kTOvKc_B>Q=AO2t{9{1+BO0$>d4D1&9zWdn|LJs}A06^xbf^0?^)S z3VaRWIv|4C_IvQ3>IhQr4@uO@{5aG0O-X4^UUthD*R68VcqHVQS?pYvQRsn(p0X|dO>CsdE;&1 zMStBY1M*W19KXK!>qT{85Vu8<;@e-p(#_+hxsWqr;ewH!%?*P`};`lI1liKWr-kj5rqi;$O_NV z4kn^{Dj}$_SPwb!k4QPb-V9#erD6 zx|-Wi>C3pxojq!$>GUwWJ!Hx(FRh(lSwMTSn_)RbELU>J}hx_dz7L9VK9nl3=Ka95gn6^NkyKrnFt#K*U!(YXmMUdT7dL_sdu46$5u z|9l670n^v=_*jEWr{^KNp)ycIwB)R&g^CJx_lQN#i9%%yGpo;TkVd4(GDln+BWQh( z!qFXUQRIPyMfOv~mJ2gMi-5_uZzNJ{5N<=im^C2oum+SMgq4i5`=E^xG8;?yHk4IQ zG#-7BYmlF`UXzY>!3rhW&ut&CYQfGNO8IuX`zN)J4A!HTBfb4G_EDjF^%_r0u$$Yk zsy9zl0%Hwb^HKfppr4G_L!5n6iY9I&#>hRH^|`h*gLP`N^A=>-2#HxquotQzoeSnJ zKu&#Q;;$X$o`MeLmz_c2W*xw;^?6PNx4VpT?6+#*#PFr?>#`keENHJ|5AWC30tiX| zuXWKe$fWLbMoIzx)h)Ad&Di({1c{@WK&Y8q*Y6wfLbTfDQC1@mo#pmI^eahn{ zssTJZV|^##qj85ozhmmo==16$Dh#w8k@$!G9-iHdb38xi2$?_9E0QsqyJ_DU_~ZXe zC0Y^uPbyJ9->RnN7{V%fR~o#8kjyxR4);0sWkASbjnJa5pLdhcUSP@KSSSMhQVk`&)Ip;MZ4{` zAwptebw8uSI8((~=!t*)^^QsjF310j(4L}K-8y)m$@r_3!pa2qU$2~^xqOuB6TP10 z3mcDFCK5eg-PxRGZ0r~p^r`9uJLu(B{a?SJPlFcX8;M_kmsMN`4p+*{MuA#k?TKhE zf!}OJQGdkMxmhgWKtxQD(P#XRe|`8rr~NE<@3);b*=N;uSidhnDL2rSKD{@U8oqY$ z$Cv--^MCo>Q&t$73wg&I2eCg<2mbPzpMOlLN0&<$9S@p7v*Q`R_QzWHzgKkmaS*!* zsutefQKpb$p??($;@s8!UTg|3r1R_Zd%srE|KppLQXZi5m<@IU>8U{9`uksp-Tykh z|LLsE!{nWT^UP|D^xv*nDdieE$&v;9%81s#fkfW^w4C-o270TeS3_Lii8m6F06u_! zweNc3KVFUa0~k220GVZEBsa6t6*xq*yi(dA|F^^a#|xA7L0pC46aZsHtmB&B(x$fm zbaQ_XVbqyl2U8uHxpa^ z^Ia5jZPWU-694_kU0yRU=sJ9Qn=JcpkQrbVY3Vixks9xIJJ>(|MJ)S2pH2}v&^0rh zfbF7G43@F0jdd#B2mr&eT)=3;hdBI=e>2Mi!oSdhXI5_p+8%W78t-$PuaqYK&i zf_N27NSnB-Q(@N0vI)9aFF*TG!0)1Aqs7gZ(kh)AZ0nRhG4phEg?Oghl z6}!Wt)PhO{IG*`nw|W7EF@iD_)%#+uVhyy~(M4SMu^~zu6fV~Q+$Ql4eN|O2!}K61 zW{?$Cr}B59C{|6 zW%G+cbeJA+`eA#zB3`R?Ip}OV4BF!jpq8=Kd0l$}4*RUP*c(a z{@^@{=2Ku);)Xn1xX@GM0FLL61$sNSRn$xz&Mr;-{{H=?AuI)%x(rr>$Tijg zo(KRwFeP+1J}~6Ui*2{kF8}zr>jcazUO;4y^90*==2XyrM6||^ntsLwhY7o}ma(Cg z?;C(lXDS`$2&xzs$DJ&77f2up*(0fPusM1sklqcYqO1WFyE>aVpud!@Zggn=M(iZ79;P6x95X<9N*O%B3>* zZ(blUr$5MgnGM1>`)DQrD~vcIt?%3u)OxX7|F5xYkB2&6!^sf2jLQ~^IAW5}vZ63! z3dN47mL#`XI~xVKiQA(vrtt6Z{7(RqLE>FoKO z-PX?Ezt8+`-{1Rw-{*ba=V603Voa2@iI#1Kfo6^2Cx9vL(i-I;yyUp-fj+1)g@qh9 zP~K#UfUFpxd?S_6T&I9}L25 zn$$_bTB(jZ=U;GEo_1`G`pOiu!U$1s%5q8aI5qwV>LYMwHRd=y}9y6Adu8DN%&0E5XB)W2fPa5PM0zq@*D6r$dSH1C&|(U@y6 zJ>f#F)tQ3bSlrJbb0QA(^}urAVLk-5jNTp zQG1EjYZ1Xn`Qg&McMot$S!tL)$I{E5eB-P4j%?+Bb7hm*wLHa=%qG@hvwjA?aww8UV*_>v}Md~sDcuhr* zFOoQJRuLV*Wi&X|GxIo#pF$_@5>Oy73{w_2+hArOR(cqm!gJzI0x|M4X$!wYOb4ug z^v~FY>MEtD1Yu4XOBEpf_F&o(11p84LoW*jZ_P3O> zi~esXg`Hb8f|4TYCtR7#gEi?>xro($FxO>%bE_K0G`ugQTMByKjs?$p!G{fC)SIPE zqTv_Zd~-;9qrcjOIv-x}iY|I0_I67GF>v|Iu#nfOOHdKSOT5`fo#d8IXI=3tn-*$N z*&T5lF=+p?!|d&-5{+RlZHVjWybSB8lBb7pCnZu=-yNHH|KPbFlQGn1+l#i0Y>g`^OrtLhn-`2eWtrv))=HO=ZOsgUa+h0yf{Ftt&ordTGL=xFF<}S^ctI{zP4@ z{O}~I-#<4sq~)0B$zGiJ0#h^3-{e*|KJaiqf9Og=N-{19!pS+TmsVt%`716YfaTgLZ4Hr$Df zw}XxPuqSt2iqCqMnQDMeGdfIqdLQ*nu!$lzgF~|-{#7~jdwjT3DbiS_Nz8C>AH6i! zQ%oXA#iz5|5Y9HjIgNl^JFThhS<-Tdpu0}E77T=JTS_z=2{LrlEuhb}$SJo?880#X z`L3N?2Ab!&FeWxeV3{RjtAK9SZh(D(Dr#rG!>wQ66(B3IZirD8`sSyw+L`375C{&)fq6SVyIh!5S zI5QFzW3DA>c)9w*c^zUT3#2s?GHOzOxZ)r@i)j1W`8pq44g`b6;4W0AQV-*h%v>1K zlRz}K>E{M45=<6il*!gvc79^Z=R<5FTmOCuUcZ{K7U?0l61Ma#f<8dK*EPdkTVDvQ z>7HVi(R;65(;vxF`d}nAl(f@o>o78^@BVHXIM|PX4t(kMt*TBf71&J#KK38;T6_Cm zKjmBCq=xhJ$iB{b43>nx-FA)((q~jgr%SraRs5oZ2u&E9p2%pgICUS=dcD5@RZ;QO zBX#ecFlgk&Qr#n~4VyMOIco-lj81lQ5R?_W& z_NMg-I({LyuH*0uiIP6zo4n|lbM(Dqgc)W=tT; z5i8wd<*MOb8($VyUd4XCUi<_+u=f9sVfbd2ZCjbWa&z~UkA>f@2T$}?xC%gXd$oyU zqrFX2O)DRIQ$i6gIOm^TFm$x(0ogx1E3&3)iitx)$ZSgmx@HfoK}RF);b)$5_jU;{; U`ptZk@GAJ(SlL+??IA|~8+Afp&;S4c literal 0 HcmV?d00001 diff --git a/docs/onpremdb/provisioning/add_replica.log b/docs/onpremdb/provisioning/add_replica.log new file mode 100644 index 00000000..53971443 --- /dev/null +++ b/docs/onpremdb/provisioning/add_replica.log @@ -0,0 +1,192 @@ +-- Check the status of CDB CRD Pod(s): + +% kubectl get all -n oracle-database-operator-system +NAME READY STATUS RESTARTS AGE +pod/cdb-dev-ords-rs-q2b68 1/1 Running 0 29s +pod/oracle-database-operator-controller-manager-76cb674c5c-4nrh8 1/1 Running 0 4d10h +pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd 1/1 Running 1 4d10h +pod/oracle-database-operator-controller-manager-76cb674c5c-xsv9g 1/1 Running 2 4d10h + +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +service/oracle-database-operator-controller-manager-metrics-service ClusterIP 10.96.98.47 8443/TCP 5d1h +service/oracle-database-operator-webhook-service ClusterIP 10.96.166.163 443/TCP 5d1h + +NAME READY UP-TO-DATE AVAILABLE AGE +deployment.apps/oracle-database-operator-controller-manager 3/3 3 3 5d1h + +NAME DESIRED CURRENT READY AGE +replicaset.apps/cdb-dev-ords-rs 1 1 1 31s +replicaset.apps/oracle-database-operator-controller-manager-76cb674c5c 3 3 3 5d1h + + +-- .yaml file for the add replica use case: + +% cat add_replica.yaml +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: CDB +metadata: + name: cdb-dev + namespace: oracle-database-operator-system +spec: + cdbName: "goldcdb" + scanName: "goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com" + dbServer: "goldhost1.lbsub52b3b1cae.okecluster.oraclevcn.com" + ordsImage: phx.ocir.io//oracle/ords:21.4.3 + dbPort: 1521 + replicas: 2 + serviceName: "goldcdb_phx1pw.lbsub52b3b1cae.okecluster.oraclevcn.com" + sysAdminPwd: + secret: + secretName: "cdb1-secret" + key: "sysadmin_pwd" + ordsPwd: + secret: + secretName: "cdb1-secret" + key: "ords_pwd" + cdbAdminUser: + secret: + secretName: "cdb1-secret" + key: "cdbadmin_user" + cdbAdminPwd: + secret: + secretName: "cdb1-secret" + key: "cdbadmin_pwd" + webServerUser: + secret: + secretName: "cdb1-secret" + key: "webserver_user" + webServerPwd: + secret: + secretName: "cdb1-secret" + key: "webserver_pwd" + + + + +-- Apply the .yaml file: + +% kubectl apply -f add_replica.yaml +cdb.database.oracle.com/cdb-dev configured + + + +-- Check the status of the CDB CRD Pod(s): + +% kubectl get all -n oracle-database-operator-system +NAME READY STATUS RESTARTS AGE +pod/cdb-dev-ords-rs-5bztb 1/1 Running 0 21s << New Pod Added +pod/cdb-dev-ords-rs-q2b68 1/1 Running 0 7m40s +pod/oracle-database-operator-controller-manager-76cb674c5c-4nrh8 1/1 Running 0 4d10h +pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd 1/1 Running 1 4d10h +pod/oracle-database-operator-controller-manager-76cb674c5c-xsv9g 1/1 Running 2 4d10h + +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +service/cdb-dev-ords ClusterIP None 6m25s +service/oracle-database-operator-controller-manager-metrics-service ClusterIP 10.96.98.47 8443/TCP 5d2h +service/oracle-database-operator-webhook-service ClusterIP 10.96.166.163 443/TCP 5d2h + +NAME READY UP-TO-DATE AVAILABLE AGE +deployment.apps/oracle-database-operator-controller-manager 3/3 3 3 5d2h + +NAME DESIRED CURRENT READY AGE +replicaset.apps/cdb-dev-ords-rs 2 2 2 7m42s +replicaset.apps/oracle-database-operator-controller-manager-76cb674c5c 3 3 3 5d2h + + + + + +-- Logs from Oracle DB Operator Pod: + +% kubectl logs -f pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd -n oracle-database-operator-system +. +. +2022-06-27T03:24:34Z DEBUG controller-runtime.webhook.webhooks received request {"webhook": "/validate-database-oracle-com-v1alpha1-cdb", "UID": "19a3fbb6-57e4-4ad2-92c9-a90bb66cefae", "kind": "database.oracle.com/v1alpha1, Kind=CDB", "resource": {"group":"database.oracle.com","version":"v1alpha1","resource":"cdbs"}} +2022-06-27T03:24:34Z INFO cdb-webhook validate update {"name": "cdb-dev"} +2022-06-27T03:24:34Z DEBUG controller-runtime.webhook.webhooks wrote response {"webhook": "/validate-database-oracle-com-v1alpha1-cdb", "code": 200, "reason": "", "UID": "19a3fbb6-57e4-4ad2-92c9-a90bb66cefae", "allowed": true} +2022-06-27T03:24:34Z INFO controllers.CDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:24:34Z INFO controllers.CDB Res Status: {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "Ready", "Status": "true"} +2022-06-27T03:24:34Z INFO controllers.CDB Existing Replicas: 1, New Replicas: 2 {"evaluateSpecChange": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:24:34Z INFO controllers.CDB Current Phase:ValidatingPods {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev"} +2022-06-27T03:24:34Z INFO controllers.CDB Validating Pod creation for :cdb-dev {"validateORDSPod": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:24:34Z INFO controller.cdb Executing Command : {"reconciler group": "database.oracle.com", "reconciler kind": "CDB", "name": "cdb-dev", "namespace": "oracle-database-operator-system", "ExecCommand": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:24:34Z INFO controller.cdb bash -c curl -sSkv -k -X GET https://localhost:8888/ords/_/db-api/stable/metadata-catalog/ || curl -sSkv -X GET http://localhost:8888/ords/_/db-api/stable/metadata-catalog/ {"reconciler group": "database.oracle.com", "reconciler kind": "CDB", "name": "cdb-dev", "namespace": "oracle-database-operator-system", "ExecCommand": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:24:35Z INFO controllers.CDB Replicas: 2 {"validateORDSPod": "oracle-database-operator-system/cdb-dev", "Ready Pods: ": 1} +2022-06-27T03:24:35Z INFO controllers.CDB Reconcile queued {"onpremdboperator": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:24:35Z INFO controllers.CDB DEFER {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "ValidatingPods", "Status": "false"} +2022-06-27T03:24:50Z INFO controllers.CDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:24:50Z INFO controllers.CDB Res Status: {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "ValidatingPods", "Status": "false"} +2022-06-27T03:24:50Z INFO controllers.CDB Current Phase:ValidatingPods {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev"} +2022-06-27T03:24:50Z INFO controllers.CDB Validating Pod creation for :cdb-dev {"validateORDSPod": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:24:50Z INFO controller.cdb Executing Command : {"reconciler group": "database.oracle.com", "reconciler kind": "CDB", "name": "cdb-dev", "namespace": "oracle-database-operator-system", "ExecCommand": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:24:50Z INFO controller.cdb bash -c curl -sSkv -k -X GET https://localhost:8888/ords/_/db-api/stable/metadata-catalog/ || curl -sSkv -X GET http://localhost:8888/ords/_/db-api/stable/metadata-catalog/ {"reconciler group": "database.oracle.com", "reconciler kind": "CDB", "name": "cdb-dev", "namespace": "oracle-database-operator-system", "ExecCommand": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:24:50Z INFO controller.cdb Executing Command : {"reconciler group": "database.oracle.com", "reconciler kind": "CDB", "name": "cdb-dev", "namespace": "oracle-database-operator-system", "ExecCommand": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:24:50Z INFO controller.cdb bash -c curl -sSkv -k -X GET https://localhost:8888/ords/_/db-api/stable/metadata-catalog/ || curl -sSkv -X GET http://localhost:8888/ords/_/db-api/stable/metadata-catalog/ {"reconciler group": "database.oracle.com", "reconciler kind": "CDB", "name": "cdb-dev", "namespace": "oracle-database-operator-system", "ExecCommand": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:24:50Z INFO controllers.CDB Replicas: 2 {"validateORDSPod": "oracle-database-operator-system/cdb-dev", "Ready Pods: ": 1} +2022-06-27T03:24:50Z INFO controllers.CDB Reconcile queued {"onpremdboperator": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:24:50Z INFO controllers.CDB DEFER {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "ValidatingPods", "Status": "false"} +2022-06-27T03:25:05Z INFO controllers.CDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:25:05Z INFO controllers.CDB Res Status: {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "ValidatingPods", "Status": "false"} +2022-06-27T03:25:05Z INFO controllers.CDB Current Phase:ValidatingPods {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev"} +2022-06-27T03:25:05Z INFO controllers.CDB Validating Pod creation for :cdb-dev {"validateORDSPod": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:25:05Z INFO controller.cdb Executing Command : {"reconciler group": "database.oracle.com", "reconciler kind": "CDB", "name": "cdb-dev", "namespace": "oracle-database-operator-system", "ExecCommand": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:25:05Z INFO controller.cdb bash -c curl -sSkv -k -X GET https://localhost:8888/ords/_/db-api/stable/metadata-catalog/ || curl -sSkv -X GET http://localhost:8888/ords/_/db-api/stable/metadata-catalog/ {"reconciler group": "database.oracle.com", "reconciler kind": "CDB", "name": "cdb-dev", "namespace": "oracle-database-operator-system", "ExecCommand": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:25:05Z INFO controller.cdb Executing Command : {"reconciler group": "database.oracle.com", "reconciler kind": "CDB", "name": "cdb-dev", "namespace": "oracle-database-operator-system", "ExecCommand": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:25:05Z INFO controller.cdb bash -c curl -sSkv -k -X GET https://localhost:8888/ords/_/db-api/stable/metadata-catalog/ || curl -sSkv -X GET http://localhost:8888/ords/_/db-api/stable/metadata-catalog/ {"reconciler group": "database.oracle.com", "reconciler kind": "CDB", "name": "cdb-dev", "namespace": "oracle-database-operator-system", "ExecCommand": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:25:06Z INFO controllers.CDB DEFER {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "CreatingService", "Status": "false"} +2022-06-27T03:25:21Z INFO controllers.CDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:25:21Z INFO controllers.CDB Res Status: {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "CreatingService", "Status": "false"} +2022-06-27T03:25:21Z INFO controllers.CDB Current Phase:CreatingService {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev"} +2022-06-27T03:25:21Z INFO controllers.CDB ORDS Cluster Service already exists {"createORDSSVC": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:25:21Z INFO controllers.CDB DEFER {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "Ready", "Status": "false"} +2022-06-27T03:25:36Z INFO controllers.CDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:25:36Z INFO controllers.CDB Res Status: {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "Ready", "Status": "false"} +2022-06-27T03:25:36Z INFO controllers.CDB Current Phase:Ready {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev"} +2022-06-27T03:25:36Z INFO controllers.CDB DEFER {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "Ready", "Status": "true"} + + + +-- Logs of the newly added CDB CRD Pod: + +% kubectl logs -f pod/cdb-dev-ords-rs-5bztb -n oracle-database-operator-system + +Retrieving information. +Requires to login with administrator privileges to verify Oracle REST Data Services schema. + +Connecting to database user: SYS AS SYSDBA url: jdbc:oracle:thin:@//goldhost1.lbsub52b3b1cae.okecluster.oraclevcn.com:1521/goldcdb_phx1pw.lbsub52b3b1cae.okecluster.oraclevcn.com + +Retrieving information.. +Completed verifying Oracle REST Data Services schema version 21.4.3.r1170405. +2022-06-27T03:24:40.351Z INFO reloaded pools: [] +2022-06-27T03:24:40.353Z INFO Oracle REST Data Services schema version 21.4.3.r1170405 is installed. +spawn java -jar /opt/oracle/ords/ords.war user sql_admin SQL Administrator +Enter a password for user sql_admin: +Confirm password for user sql_admin: +2022-06-27T03:24:42.034Z INFO Created user: sql_admin in file: /opt/oracle/ords/config/ords/credentials +2022-06-27T03:24:43.666Z INFO Modified: /opt/oracle/ords/config/ords/conf/apex_pu.xml, updated properties: database.api.admin.enabled, db.cdb.adminUser, db.cdb.adminUser.password +2022-06-27T03:24:45.455Z INFO HTTP and HTTP/2 cleartext listening on host: localhost port: 8888 +2022-06-27T03:24:45.520Z INFO The document root is serving static resources located in: /opt/oracle/ords/doc_root +2022-06-27T03:24:47.515Z INFO Configuration properties for: |apex|pu| +db.servicename=goldcdb_phx1pw.lbsub52b3b1cae.okecluster.oraclevcn.com +db.hostname=goldhost1.lbsub52b3b1cae.okecluster.oraclevcn.com +database.api.admin.enabled=true +db.password=****** +db.cdb.adminUser.password=****** +database.api.enabled=true +db.cdb.adminUser=C##DBAPI_CDB_ADMIN as SYSDBA +db.username=ORDS_PUBLIC_USER +restEnabledSql.active=true +resource.templates.enabled=true +db.port=1521 +feature.sdw=true +db.connectionType=basic + +2022-06-27T03:24:47.517Z WARNING *** jdbc.MaxLimit in configuration |apex|pu| is using a value of 10, this setting may not be sized adequately for a production environment *** +2022-06-27T03:24:47.517Z WARNING *** jdbc.InitialLimit in configuration |apex|pu| is using a value of 3, this setting may not be sized adequately for a production environment *** +2022-06-27T03:24:51.761Z INFO Oracle REST Data Services initialized +Oracle REST Data Services version : 21.4.3.r1170405 +Oracle REST Data Services server info: jetty/9.4.44.v20210927 diff --git a/docs/onpremdb/provisioning/add_replica.md b/docs/onpremdb/provisioning/add_replica.md new file mode 100644 index 00000000..1315dc9f --- /dev/null +++ b/docs/onpremdb/provisioning/add_replica.md @@ -0,0 +1,36 @@ +# Add a new replicate to an existing CDB CRD Resource using Oracle DB Operator On-Prem Controller + +In this use case, using the Oracle Database Operator On-Prem Controller, you will add a new replica to an existing CDB CRD resource. + +**NOTE:** It is assumed that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-manage-pdb-life-cycle-using-oracle-db-operator-on-prem-database-controller) steps. + +This example uses `add_replica.yaml` with: + +- CDB CRD resource Name as `cdb-dev` +- Container Database (CDB) Name as `goldcdb` +- Scan Name as `goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com` +- Database Server Name as `goldhost1.lbsub52b3b1cae.okecluster.oraclevcn.com` +- ORDS Docker Image as `phx.ocir.io//oracle/ords:21.4.3` +- Database Listener Port as `1521` +- Number of replicas for CDB CRD Resource as 2. + +**NOTE:** For the details of the parameters to be used in the .yaml file, please refer [config/crd/bases/database.oracle.com_cdbs.yaml](../../../config/crd/bases/database.oracle.com_cdbs.yaml) + +Use the file: [add_replica.yaml](./add_replica.yaml) for this use case as below: + +1. Deploy the .yaml file: +```sh +[root@test-server oracle-database-operator]# kubectl apply -f add_replica.yaml +``` + +2. Monitor the Oracle DB Operator Pod for the progress of the CDB CRD Resource creation. + +NOTE: Check the DB Operator Pod name in your environment. + +``` +[root@test-server oracle-database-operator]# kubectl logs -f pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd -n oracle-database-operator-system +``` + +## Sample Output + +[Here](./add_replica.log) is the sample output for a PDB created using Oracle DB Operator On-Prem Controller using file [add_replica.yaml](./add_replica.yaml) diff --git a/docs/onpremdb/provisioning/add_replica.yaml b/docs/onpremdb/provisioning/add_replica.yaml new file mode 100644 index 00000000..2bc54495 --- /dev/null +++ b/docs/onpremdb/provisioning/add_replica.yaml @@ -0,0 +1,41 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: CDB +metadata: + name: cdb-dev + namespace: oracle-database-operator-system +spec: + cdbName: "goldcdb" + scanName: "goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com" + dbServer: "goldhost1.lbsub52b3b1cae.okecluster.oraclevcn.com" + ordsImage: phx.ocir.io//oracle/ords:21.4.3 + dbPort: 1521 + replicas: 2 + serviceName: "goldcdb_phx1pw.lbsub52b3b1cae.okecluster.oraclevcn.com" + sysAdminPwd: + secret: + secretName: "cdb1-secret" + key: "sysadmin_pwd" + ordsPwd: + secret: + secretName: "cdb1-secret" + key: "ords_pwd" + cdbAdminUser: + secret: + secretName: "cdb1-secret" + key: "cdbadmin_user" + cdbAdminPwd: + secret: + secretName: "cdb1-secret" + key: "cdbadmin_pwd" + webServerUser: + secret: + secretName: "cdb1-secret" + key: "webserver_user" + webServerPwd: + secret: + secretName: "cdb1-secret" + key: "webserver_pwd" diff --git a/docs/onpremdb/provisioning/cdb.log b/docs/onpremdb/provisioning/cdb.log new file mode 100644 index 00000000..8c3cbdc5 --- /dev/null +++ b/docs/onpremdb/provisioning/cdb.log @@ -0,0 +1,279 @@ +-- Check the status of the Oracle DB Operator Pods: + +% kubectl get all -n oracle-database-operator-system +NAME READY STATUS RESTARTS AGE +pod/oracle-database-operator-controller-manager-76cb674c5c-4nrh8 1/1 Running 0 29h +pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd 1/1 Running 1 29h +pod/oracle-database-operator-controller-manager-76cb674c5c-xsv9g 1/1 Running 2 29h + +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +service/oracle-database-operator-controller-manager-metrics-service ClusterIP 10.96.98.47 8443/TCP 45h +service/oracle-database-operator-webhook-service ClusterIP 10.96.166.163 443/TCP 45h + +NAME READY UP-TO-DATE AVAILABLE AGE +deployment.apps/oracle-database-operator-controller-manager 3/3 3 3 45h + +NAME DESIRED CURRENT READY AGE +replicaset.apps/oracle-database-operator-controller-manager-76cb674c5c 3 3 3 45h + + + +-- Get the base64 values for the required passwords as below: + +% echo -n "WElcome_21##" | base64 +V0VsY29tZV8yMSMj + +% echo -n "C##DBAPI_CDB_ADMIN" | base64 +QyMjREJBUElfQ0RCX0FETUlO + +% echo -n "sql_admin" | base64 +c3FsX2FkbWlu + +% echo -n "welcome1" | base64 +d2VsY29tZTE= + + +-- Add the base64 encoded values against the required variables in cdb_secret.yaml file for this use case: + +% cat cdb_secret.yaml +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: v1 +kind: Secret +metadata: + name: cdb1-secret + namespace: oracle-database-operator-system +type: Opaque +data: + ords_pwd: "V0VsY29tZV8yMSMj" + sysadmin_pwd: "V0VsY29tZV8yMSMj" + cdbadmin_user: "QyMjREJBUElfQ0RCX0FETUlO" + cdbadmin_pwd: "V0VsY29tZV8yMSMj" + webserver_user: "c3FsX2FkbWlu" + webserver_pwd: "d2VsY29tZTE=" + + +-- Check the contents of the cdb.yaml file: +% cat cdb.yaml +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: CDB +metadata: + name: cdb-dev + namespace: oracle-database-operator-system +spec: + cdbName: "goldcdb" + scanName: "goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com" + dbServer: "goldhost1.lbsub52b3b1cae.okecluster.oraclevcn.com" + ordsImage: phx.ocir.io//oracle/ords:21.4.3 + ordsImagePullSecret: "container-registry-secret" + dbPort: 1521 + replicas: 1 + serviceName: "goldcdb_phx1pw.lbsub52b3b1cae.okecluster.oraclevcn.com" + sysAdminPwd: + secret: + secretName: "cdb1-secret" + key: "sysadmin_pwd" + ordsPwd: + secret: + secretName: "cdb1-secret" + key: "ords_pwd" + cdbAdminUser: + secret: + secretName: "cdb1-secret" + key: "cdbadmin_user" + cdbAdminPwd: + secret: + secretName: "cdb1-secret" + key: "cdbadmin_pwd" + webServerUser: + secret: + secretName: "cdb1-secret" + key: "webserver_user" + webServerPwd: + secret: + secretName: "cdb1-secret" + key: "webserver_pwd" + + + +-- Apply the .yaml files: + +% kubectl apply -f cdb_secret.yaml +secret/cdb1-secret created + +% kubectl apply -f cdb.yaml +cdb.database.oracle.com/cdb-dev created + + + +-- Monitor the Oracle DB Operator Pod during the period when the CDB CRD is getting deployed: + +% kubectl logs -f pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd -n oracle-database-operator-system +. +. +2022-06-27T03:16:44Z INFO controllers.CDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:16:44Z INFO controllers.CDB Res Status: {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "", "Status": "false"} +2022-06-27T03:16:44Z INFO controllers.CDB Adding finalizer {"manageCDBDeletion": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:16:44Z INFO controllers.CDB Current Phase: {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev"} +2022-06-27T03:16:44Z INFO controllers.CDB DEFAULT: {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "", "Status": "false"} +2022-06-27T03:16:44Z INFO controllers.CDB DEFER {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "Initializing", "Status": "false"} +2022-06-27T03:17:00Z INFO controllers.CDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:17:00Z INFO controllers.CDB Res Status: {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "Initializing", "Status": "false"} +2022-06-27T03:17:00Z INFO controllers.CDB Current Phase:Initializing {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev"} +2022-06-27T03:17:00Z INFO controllers.CDB Verified secrets successfully {"verifySecrets": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:17:00Z INFO controllers.CDB DEFER {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "CreatingPod", "Status": "false"} +2022-06-27T03:17:15Z INFO controllers.CDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:17:15Z INFO controllers.CDB Res Status: {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "CreatingPod", "Status": "false"} +2022-06-27T03:17:15Z INFO controllers.CDB Current Phase:CreatingPod {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev"} +2022-06-27T03:17:15Z INFO controllers.CDB Creating ORDS Replicaset: cdb-dev-ords-rs {"createORDSInstances": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:17:15Z INFO controllers.CDB Created ORDS ReplicaSet successfully {"createORDSInstances": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:17:15Z DEBUG events Normal {"object": {"kind":"CDB","namespace":"oracle-database-operator-system","name":"cdb-dev","uid":"c36e8d5f-6103-4a70-a840-16c6683755ec","apiVersion":"database.oracle.com/v1alpha1","resourceVersion":"101443216"}, "reason": "CreatedORDSReplicaSet", "message": "Created ORDS Replicaset (Replicas - 1) for cdb-dev"} +2022-06-27T03:17:15Z INFO controllers.CDB DEFER {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "ValidatingPods", "Status": "false"} +2022-06-27T03:17:30Z INFO controllers.CDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:17:30Z INFO controllers.CDB Res Status: {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "ValidatingPods", "Status": "false"} +2022-06-27T03:17:30Z INFO controllers.CDB Current Phase:ValidatingPods {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev"} +2022-06-27T03:17:30Z INFO controllers.CDB Validating Pod creation for :cdb-dev {"validateORDSPod": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:17:30Z INFO controller.cdb Executing Command : {"reconciler group": "database.oracle.com", "reconciler kind": "CDB", "name": "cdb-dev", "namespace": "oracle-database-operator-system", "ExecCommand": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:17:30Z INFO controller.cdb bash -c curl -sSkv -k -X GET https://localhost:8888/ords/_/db-api/stable/metadata-catalog/ || curl -sSkv -X GET http://localhost:8888/ords/_/db-api/stable/metadata-catalog/ {"reconciler group": "database.oracle.com", "reconciler kind": "CDB", "name": "cdb-dev", "namespace": "oracle-database-operator-system", "ExecCommand": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:17:30Z INFO controllers.CDB Replicas: 1 {"validateORDSPod": "oracle-database-operator-system/cdb-dev", "Ready Pods: ": 0} +2022-06-27T03:17:30Z INFO controllers.CDB Reconcile queued {"onpremdboperator": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:17:30Z INFO controllers.CDB DEFER {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "ValidatingPods", "Status": "false"} +2022-06-27T03:17:45Z INFO controllers.CDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:17:45Z INFO controllers.CDB Res Status: {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "ValidatingPods", "Status": "false"} +2022-06-27T03:17:45Z INFO controllers.CDB Current Phase:ValidatingPods {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev"} +2022-06-27T03:17:45Z INFO controllers.CDB Validating Pod creation for :cdb-dev {"validateORDSPod": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:17:45Z INFO controller.cdb Executing Command : {"reconciler group": "database.oracle.com", "reconciler kind": "CDB", "name": "cdb-dev", "namespace": "oracle-database-operator-system", "ExecCommand": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:17:45Z INFO controller.cdb bash -c curl -sSkv -k -X GET https://localhost:8888/ords/_/db-api/stable/metadata-catalog/ || curl -sSkv -X GET http://localhost:8888/ords/_/db-api/stable/metadata-catalog/ {"reconciler group": "database.oracle.com", "reconciler kind": "CDB", "name": "cdb-dev", "namespace": "oracle-database-operator-system", "ExecCommand": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:17:45Z INFO controllers.CDB Replicas: 1 {"validateORDSPod": "oracle-database-operator-system/cdb-dev", "Ready Pods: ": 0} +2022-06-27T03:17:45Z INFO controllers.CDB Reconcile queued {"onpremdboperator": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:17:45Z INFO controllers.CDB DEFER {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "ValidatingPods", "Status": "false"} +2022-06-27T03:18:00Z INFO controllers.CDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:18:00Z INFO controllers.CDB Res Status: {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "ValidatingPods", "Status": "false"} +2022-06-27T03:18:00Z INFO controllers.CDB Current Phase:ValidatingPods {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev"} +2022-06-27T03:18:00Z INFO controllers.CDB Validating Pod creation for :cdb-dev {"validateORDSPod": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:18:00Z INFO controller.cdb Executing Command : {"reconciler group": "database.oracle.com", "reconciler kind": "CDB", "name": "cdb-dev", "namespace": "oracle-database-operator-system", "ExecCommand": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:18:00Z INFO controller.cdb bash -c curl -sSkv -k -X GET https://localhost:8888/ords/_/db-api/stable/metadata-catalog/ || curl -sSkv -X GET http://localhost:8888/ords/_/db-api/stable/metadata-catalog/ {"reconciler group": "database.oracle.com", "reconciler kind": "CDB", "name": "cdb-dev", "namespace": "oracle-database-operator-system", "ExecCommand": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:18:01Z INFO controllers.CDB Replicas: 1 {"validateORDSPod": "oracle-database-operator-system/cdb-dev", "Ready Pods: ": 0} +2022-06-27T03:18:01Z INFO controllers.CDB Reconcile queued {"onpremdboperator": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:18:01Z INFO controllers.CDB DEFER {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "ValidatingPods", "Status": "false"} +2022-06-27T03:18:16Z INFO controllers.CDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:18:16Z INFO controllers.CDB Res Status: {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "ValidatingPods", "Status": "false"} +2022-06-27T03:18:16Z INFO controllers.CDB Current Phase:ValidatingPods {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev"} +2022-06-27T03:18:16Z INFO controllers.CDB Validating Pod creation for :cdb-dev {"validateORDSPod": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:18:16Z INFO controller.cdb Executing Command : {"reconciler group": "database.oracle.com", "reconciler kind": "CDB", "name": "cdb-dev", "namespace": "oracle-database-operator-system", "ExecCommand": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:18:16Z INFO controller.cdb bash -c curl -sSkv -k -X GET https://localhost:8888/ords/_/db-api/stable/metadata-catalog/ || curl -sSkv -X GET http://localhost:8888/ords/_/db-api/stable/metadata-catalog/ {"reconciler group": "database.oracle.com", "reconciler kind": "CDB", "name": "cdb-dev", "namespace": "oracle-database-operator-system", "ExecCommand": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:18:16Z INFO controllers.CDB DEFER {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "CreatingService", "Status": "false"} +2022-06-27T03:18:31Z INFO controllers.CDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:18:31Z INFO controllers.CDB Res Status: {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "CreatingService", "Status": "false"} +2022-06-27T03:18:31Z INFO controllers.CDB Current Phase:CreatingService {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev"} +2022-06-27T03:18:31Z INFO controllers.CDB Creating a new Cluster Service for: cdb-dev {"createORDSSVC": "oracle-database-operator-system/cdb-dev", "Svc.Namespace": "oracle-database-operator-system", "Service.Name": "cdb-dev-ords"} +2022-06-27T03:18:31Z INFO controllers.CDB Created ORDS Cluster Service successfully {"createORDSSVC": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:18:31Z DEBUG events Normal {"object": {"kind":"CDB","namespace":"oracle-database-operator-system","name":"cdb-dev","uid":"c36e8d5f-6103-4a70-a840-16c6683755ec","apiVersion":"database.oracle.com/v1alpha1","resourceVersion":"101443627"}, "reason": "CreatedORDSService", "message": "Created ORDS Service for cdb-dev"} +2022-06-27T03:18:31Z INFO controllers.CDB DEFER {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "Ready", "Status": "false"} +2022-06-27T03:18:46Z INFO controllers.CDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/cdb-dev"} +2022-06-27T03:18:46Z INFO controllers.CDB Res Status: {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "Ready", "Status": "false"} +2022-06-27T03:18:46Z INFO controllers.CDB Current Phase:Ready {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev"} +2022-06-27T03:18:46Z INFO controllers.CDB DEFER {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "Ready", "Status": "true"} + + + + +-- Check the status of the CDB CRD Pod after few minutes of running above commands: + +% kubectl get all -n oracle-database-operator-system +NAME READY STATUS RESTARTS AGE +pod/cdb-dev-ords-rs-q2b68 1/1 Running 0 29s <<< CDB CRD Resource Pod +pod/oracle-database-operator-controller-manager-76cb674c5c-4nrh8 1/1 Running 0 4d10h +pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd 1/1 Running 1 4d10h +pod/oracle-database-operator-controller-manager-76cb674c5c-xsv9g 1/1 Running 2 4d10h + +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +service/oracle-database-operator-controller-manager-metrics-service ClusterIP 10.96.98.47 8443/TCP 5d1h +service/oracle-database-operator-webhook-service ClusterIP 10.96.166.163 443/TCP 5d1h + +NAME READY UP-TO-DATE AVAILABLE AGE +deployment.apps/oracle-database-operator-controller-manager 3/3 3 3 5d1h + +NAME DESIRED CURRENT READY AGE +replicaset.apps/cdb-dev-ords-rs 1 1 1 31s +replicaset.apps/oracle-database-operator-controller-manager-76cb674c5c 3 3 3 5d1h + + + + +-- Check the logs of the CDB CRD Pod created above: + +% kubectl logs -f pod/cdb-dev-ords-rs-q2b68 -n oracle-database-operator-system +Requires to login with administrator privileges to verify Oracle REST Data Services schema. + +Connecting to database user: SYS AS SYSDBA url: jdbc:oracle:thin:@//goldhost1.lbsub52b3b1cae.okecluster.oraclevcn.com:1521/goldcdb_phx1pw.lbsub52b3b1cae.okecluster.oraclevcn.com + +Retrieving information.. +Your database connection is to a CDB. ORDS common user ORDS_PUBLIC_USER will be created in the CDB. ORDS schema will be installed in the PDBs. +Root CDB$ROOT - create ORDS common user +PDB PDB$SEED - install ORDS 21.4.3.r1170405 + +2022-06-27T03:17:22.015Z INFO reloaded pools: [] +2022-06-27T03:17:22.024Z INFO + +2022-06-27T03:17:23.200Z INFO Installing Oracle REST Data Services version 21.4.3.r1170405 in CDB$ROOT +2022-06-27T03:17:23.234Z INFO ... Log file written to /home/oracle/ords_cdb_install_core_CDB_ROOT_2022-06-27_031723_00234.log +2022-06-27T03:17:24.352Z INFO ... Verified database prerequisites +2022-06-27T03:17:24.662Z INFO ... Created Oracle REST Data Services proxy user +2022-06-27T03:17:24.708Z INFO Completed installation for Oracle REST Data Services version 21.4.3.r1170405. Elapsed time: 00:00:01.504 + +2022-06-27T03:17:24.722Z INFO Installing Oracle REST Data Services version 21.4.3.r1170405 in PDB$SEED +2022-06-27T03:17:24.731Z INFO ... Log file written to /home/oracle/ords_cdb_install_core_PDB_SEED_2022-06-27_031724_00731.log +2022-06-27T03:17:24.863Z INFO ... Verified database prerequisites +2022-06-27T03:17:25.123Z INFO ... Created Oracle REST Data Services proxy user +2022-06-27T03:17:25.568Z INFO ... Created Oracle REST Data Services schema +2022-06-27T03:17:26.252Z INFO ... Granted privileges to Oracle REST Data Services +2022-06-27T03:17:34.493Z INFO ... Created Oracle REST Data Services database objects +2022-06-27T03:17:43.730Z INFO ... Log file written to /home/oracle/ords_cdb_install_datamodel_PDB_SEED_2022-06-27_031743_00730.log +2022-06-27T03:17:45.883Z INFO ... Log file written to /home/oracle/ords_cdb_install_scheduler_PDB_SEED_2022-06-27_031745_00883.log +2022-06-27T03:17:49.296Z INFO ... Log file written to /home/oracle/ords_cdb_install_apex_PDB_SEED_2022-06-27_031749_00296.log +2022-06-27T03:17:51.492Z INFO Completed installation for Oracle REST Data Services version 21.4.3.r1170405. Elapsed time: 00:00:26.768 + +2022-06-27T03:17:51.492Z INFO Completed CDB installation for Oracle REST Data Services version 21.4.3.r1170405. Total elapsed time: 00:00:28.297 + +spawn java -jar /opt/oracle/ords/ords.war user sql_admin SQL Administrator +Enter a password for user sql_admin: +Confirm password for user sql_admin: +2022-06-27T03:17:53.192Z INFO Created user: sql_admin in file: /opt/oracle/ords/config/ords/credentials +2022-06-27T03:17:54.816Z INFO Modified: /opt/oracle/ords/config/ords/conf/apex_pu.xml, updated properties: database.api.admin.enabled, db.cdb.adminUser, db.cdb.adminUser.password +2022-06-27T03:17:56.583Z INFO HTTP and HTTP/2 cleartext listening on host: localhost port: 8888 +2022-06-27T03:17:56.647Z INFO The document root is serving static resources located in: /opt/oracle/ords/doc_root +2022-06-27T03:17:58.593Z INFO Configuration properties for: |apex|pu| +db.servicename=goldcdb_phx1pw.lbsub52b3b1cae.okecluster.oraclevcn.com +db.hostname=goldhost1.lbsub52b3b1cae.okecluster.oraclevcn.com +database.api.admin.enabled=true +db.password=****** +db.cdb.adminUser.password=****** +database.api.enabled=true +db.cdb.adminUser=C##DBAPI_CDB_ADMIN as SYSDBA +db.username=ORDS_PUBLIC_USER +restEnabledSql.active=true +resource.templates.enabled=true +db.port=1521 +feature.sdw=true +db.connectionType=basic + +2022-06-27T03:17:58.595Z WARNING *** jdbc.MaxLimit in configuration |apex|pu| is using a value of 10, this setting may not be sized adequately for a production environment *** +2022-06-27T03:17:58.595Z WARNING *** jdbc.InitialLimit in configuration |apex|pu| is using a value of 3, this setting may not be sized adequately for a production environment *** +2022-06-27T03:18:02.803Z INFO Oracle REST Data Services initialized +Oracle REST Data Services version : 21.4.3.r1170405 +Oracle REST Data Services server info: jetty/9.4.44.v20210927 + + + +-- Check the CDB CRD Resource and it should be in "Ready" status for a successful deployment: + +% kubectl get cdbs -A +NAMESPACE NAME CDB NAME DB SERVER DB PORT SCAN NAME REPLICAS STATUS MESSAGE +oracle-database-operator-system cdb-dev goldcdb goldhost1.lbsub52b3b1cae.okecluster.oraclevcn.com 1521 goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com 1 Ready diff --git a/docs/onpremdb/provisioning/cdb.yaml b/docs/onpremdb/provisioning/cdb.yaml new file mode 100644 index 00000000..4282040e --- /dev/null +++ b/docs/onpremdb/provisioning/cdb.yaml @@ -0,0 +1,42 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: CDB +metadata: + name: cdb-dev + namespace: oracle-database-operator-system +spec: + cdbName: "goldcdb" + scanName: "goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com" + dbServer: "goldhost1.lbsub52b3b1cae.okecluster.oraclevcn.com" + ordsImage: phx.ocir.io//oracle/ords:21.4.3 + ordsImagePullSecret: "container-registry-secret" + dbPort: 1521 + replicas: 1 + serviceName: "goldcdb_phx1pw.lbsub52b3b1cae.okecluster.oraclevcn.com" + sysAdminPwd: + secret: + secretName: "cdb1-secret" + key: "sysadmin_pwd" + ordsPwd: + secret: + secretName: "cdb1-secret" + key: "ords_pwd" + cdbAdminUser: + secret: + secretName: "cdb1-secret" + key: "cdbadmin_user" + cdbAdminPwd: + secret: + secretName: "cdb1-secret" + key: "cdbadmin_pwd" + webServerUser: + secret: + secretName: "cdb1-secret" + key: "webserver_user" + webServerPwd: + secret: + secretName: "cdb1-secret" + key: "webserver_pwd" diff --git a/docs/onpremdb/provisioning/cdb_crd_resource.md b/docs/onpremdb/provisioning/cdb_crd_resource.md new file mode 100644 index 00000000..f1f36404 --- /dev/null +++ b/docs/onpremdb/provisioning/cdb_crd_resource.md @@ -0,0 +1,38 @@ +# Create a CDB CRD Resource using Oracle DB Operator On-Prem Controller + +In this use case, using the Oracle Database Operator On-Prem Controller, you will create the CDB kind as a custom resource that will model a CDB as a native Kubernetes object. + +**NOTE:** It is assumed that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-manage-pdb-life-cycle-using-oracle-db-operator-on-prem-database-controller) steps. + +This example uses `create_cdb.yaml` with: + +- CDB CRD resource Name as `cdb-dev` +- Container Database (CDB) Name as `goldcdb` +- Scan Name as `goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com` +- Database Server Name as `goldhost1.lbsub52b3b1cae.okecluster.oraclevcn.com` +- ORDS Docker Image as `phx.ocir.io//oracle/ords:21.4.3` +- Image Pull Secret as `container-registry-secret` +- Database Listener Port as `1521` +- Database Service Name as `goldcdb_phx1pw.lbsub52b3b1cae.okecluster.oraclevcn.com` +- Number of replicas for CDB CRD Resource as 1 + +**NOTE:** For the details of the parameters to be used in the .yaml file, please refer [config/crd/bases/database.oracle.com_cdbs.yaml](../../../config/crd/bases/database.oracle.com_cdbs.yaml) + +Use the file: [cdb.yaml](./cdb.yaml) for this use case as below: + +1. Deploy the .yaml file: +```sh +[root@test-server oracle-database-operator]# kubectl apply -f cdb.yaml +``` + +2. Monitor the Oracle DB Operator Pod for the progress of the CDB CRD Resource creation. + +NOTE: Check the DB Operator Pod name in your environment. + +``` +[root@test-server oracle-database-operator]# kubectl logs -f pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd -n oracle-database-operator-system +``` + +## Sample Output + +[Here](./cdb.log) is the sample output for a PDB created using Oracle DB Operator On-Prem Controller using file [cdb.yaml](./cdb.yaml) diff --git a/docs/onpremdb/provisioning/cdb_secret.log b/docs/onpremdb/provisioning/cdb_secret.log new file mode 100644 index 00000000..e69de29b diff --git a/docs/onpremdb/provisioning/cdb_secret.yaml b/docs/onpremdb/provisioning/cdb_secret.yaml new file mode 100644 index 00000000..f6c5031f --- /dev/null +++ b/docs/onpremdb/provisioning/cdb_secret.yaml @@ -0,0 +1,17 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: v1 +kind: Secret +metadata: + name: cdb1-secret + namespace: oracle-database-operator-system +type: Opaque +data: + ords_pwd: "V0VsY29tZV8yMSMj" + sysadmin_pwd: "V0VsY29tZV8yMSMj" + cdbadmin_user: "QyMjREJBUElfQ0RCX0FETUlO" + cdbadmin_pwd: "V0VsY29tZV8yMSMj" + webserver_user: "c3FsX2FkbWlu" + webserver_pwd: "d2VsY29tZTE=" diff --git a/docs/onpremdb/provisioning/clone_pdb.log b/docs/onpremdb/provisioning/clone_pdb.log new file mode 100644 index 00000000..54b54ef6 --- /dev/null +++ b/docs/onpremdb/provisioning/clone_pdb.log @@ -0,0 +1,137 @@ +-- Check the Oracle DB Operator Pod and CDB CRD Pod status: + +% kubectl get all -n oracle-database-operator-system +NAME READY STATUS RESTARTS AGE +pod/cdb-dev-ords-rs-5bztb 1/1 Running 0 5m23s +pod/cdb-dev-ords-rs-q2b68 1/1 Running 0 12m +pod/oracle-database-operator-controller-manager-76cb674c5c-4nrh8 1/1 Running 0 4d10h +pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd 1/1 Running 1 4d10h +pod/oracle-database-operator-controller-manager-76cb674c5c-xsv9g 1/1 Running 2 4d10h + +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +service/cdb-dev-ords ClusterIP None 11m +service/oracle-database-operator-controller-manager-metrics-service ClusterIP 10.96.98.47 8443/TCP 5d2h +service/oracle-database-operator-webhook-service ClusterIP 10.96.166.163 443/TCP 5d2h + +NAME READY UP-TO-DATE AVAILABLE AGE +deployment.apps/oracle-database-operator-controller-manager 3/3 3 3 5d2h + +NAME DESIRED CURRENT READY AGE +replicaset.apps/cdb-dev-ords-rs 2 2 2 12m +replicaset.apps/oracle-database-operator-controller-manager-76cb674c5c 3 3 3 5d2h + + +-- Check the current PDB CRD resource: + +% kubectl get pdbs -A +NAMESPACE NAME CONNECT STRING CDB NAME PDB NAME PDB STATE PDB SIZE STATUS MESSAGE +oracle-database-operator-system pdb1 goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com:1521/pdbnew goldcdb pdbnew READ WRITE 1G Ready Success + + +-- Check the current PDBs in the target CDB: + +SQL> show pdbs + + CON_ID CON_NAME OPEN MODE RESTRICTED +---------- ------------------------------ ---------- ---------- + 2 PDB$SEED READ WRITE NO + 3 PDBNEW READ WRITE NO + + + +-- .yaml file used in this use case to clone a PDB: + +% cat clone_pdb.yaml +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1-clone + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbName: "goldcdb" + pdbName: "pdbnewclone" + srcPdbName: "pdbnew" + fileNameConversions: "NONE" + totalSize: "UNLIMITED" + tempSize: "UNLIMITED" + action: "Clone" + + +-- Apply the .yaml file: + +% kubectl apply -f clone_pdb.yaml +pdb.database.oracle.com/pdb1-clone created + + + +- Monitor the logs from the Oracle DB Operator Pod: + +% kubectl logs -f pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd -n oracle-database-operator-system +. +. +2022-06-27T03:37:21Z DEBUG controller-runtime.webhook.webhooks received request {"webhook": "/mutate-database-oracle-com-v1alpha1-pdb", "UID": "7fbbd983-0309-4603-9c3e-77f7ffded000", "kind": "database.oracle.com/v1alpha1, Kind=PDB", "resource": {"group":"database.oracle.com","version":"v1alpha1","resource":"pdbs"}} +2022-06-27T03:37:21Z INFO pdb-webhook Setting default values in PDB spec for : pdb1-clone +2022-06-27T03:37:21Z INFO pdb-webhook - reuseTempFile : true +2022-06-27T03:37:21Z INFO pdb-webhook - unlimitedStorage : true +2022-06-27T03:37:21Z INFO pdb-webhook - tdeImport : false +2022-06-27T03:37:21Z INFO pdb-webhook - tdeExport : false +2022-06-27T03:37:21Z INFO pdb-webhook - asClone : false +2022-06-27T03:37:21Z INFO pdb-webhook - getScript : false +2022-06-27T03:37:21Z DEBUG controller-runtime.webhook.webhooks wrote response {"webhook": "/mutate-database-oracle-com-v1alpha1-pdb", "code": 200, "reason": "", "UID": "7fbbd983-0309-4603-9c3e-77f7ffded000", "allowed": true} +2022-06-27T03:37:21Z DEBUG controller-runtime.webhook.webhooks received request {"webhook": "/validate-database-oracle-com-v1alpha1-pdb", "UID": "12aa43ea-df81-4931-b29b-d665c121590f", "kind": "database.oracle.com/v1alpha1, Kind=PDB", "resource": {"group":"database.oracle.com","version":"v1alpha1","resource":"pdbs"}} +2022-06-27T03:37:21Z INFO pdb-webhook ValidateCreate-Validating PDB spec for : pdb1-clone +2022-06-27T03:37:21Z INFO pdb-webhook validateCommon {"name": "pdb1-clone"} +2022-06-27T03:37:21Z INFO pdb-webhook Valdiating PDB Resource Action : CLONE +2022-06-27T03:37:21Z INFO pdb-webhook PDB Resource : pdb1-clone successfully validated for Action : CLONE +2022-06-27T03:37:21Z DEBUG controller-runtime.webhook.webhooks wrote response {"webhook": "/validate-database-oracle-com-v1alpha1-pdb", "code": 200, "reason": "", "UID": "12aa43ea-df81-4931-b29b-d665c121590f", "allowed": true} +2022-06-27T03:37:21Z INFO controllers.PDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/pdb1-clone"} +2022-06-27T03:37:21Z INFO controllers.PDB Adding finalizer {"managePDBDeletion": "oracle-database-operator-system/pdb1-clone"} +2022-06-27T03:37:21Z DEBUG controller-runtime.webhook.webhooks received request {"webhook": "/mutate-database-oracle-com-v1alpha1-pdb", "UID": "b3bbaa0d-6865-4dd1-9ca9-d54f916a8d66", "kind": "database.oracle.com/v1alpha1, Kind=PDB", "resource": {"group":"database.oracle.com","version":"v1alpha1","resource":"pdbs"}} +2022-06-27T03:37:21Z INFO pdb-webhook Setting default values in PDB spec for : pdb1-clone +2022-06-27T03:37:21Z DEBUG controller-runtime.webhook.webhooks wrote response {"webhook": "/mutate-database-oracle-com-v1alpha1-pdb", "code": 200, "reason": "", "UID": "b3bbaa0d-6865-4dd1-9ca9-d54f916a8d66", "allowed": true} +2022-06-27T03:37:21Z DEBUG controller-runtime.webhook.webhooks received request {"webhook": "/validate-database-oracle-com-v1alpha1-pdb", "UID": "5924d376-7a5c-4d4a-8ca9-040c585fb4b6", "kind": "database.oracle.com/v1alpha1, Kind=PDB", "resource": {"group":"database.oracle.com","version":"v1alpha1","resource":"pdbs"}} +2022-06-27T03:37:21Z INFO pdb-webhook ValidateUpdate-Validating PDB spec for : pdb1-clone +2022-06-27T03:37:21Z INFO pdb-webhook validateCommon {"name": "pdb1-clone"} +2022-06-27T03:37:21Z INFO pdb-webhook Valdiating PDB Resource Action : CLONE +2022-06-27T03:37:21Z DEBUG controller-runtime.webhook.webhooks wrote response {"webhook": "/validate-database-oracle-com-v1alpha1-pdb", "code": 200, "reason": "", "UID": "5924d376-7a5c-4d4a-8ca9-040c585fb4b6", "allowed": true} +2022-06-27T03:37:21Z INFO controllers.PDB Found PDB: pdb1-clone {"checkDuplicatePDB": "oracle-database-operator-system/pdb1-clone"} +2022-06-27T03:37:21Z INFO controllers.PDB Validating PDB phase for: pdb1-clone {"validatePhase": "oracle-database-operator-system/pdb1-clone", "Action": "CLONE"} +2022-06-27T03:37:21Z INFO controllers.PDB Validation complete {"validatePhase": "oracle-database-operator-system/pdb1-clone"} +2022-06-27T03:37:21Z INFO controllers.PDB PDB: {"onpremdboperator": "oracle-database-operator-system/pdb1-clone", "Name": "pdb1-clone", "Phase": "Cloning", "Status": "false"} +2022-06-27T03:37:21Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1-clone", "Name": "cdb-dev", "CR Name": "cdb-dev"} +2022-06-27T03:37:21Z INFO controllers.PDB Issuing REST call {"callAPI": "oracle-database-operator-system/pdb1-clone", "URL": "http://cdb-dev-ords:8888/ords/_/db-api/latest/database/pdbs/pdbnew/", "Action": "POST"} +2022-06-27T03:37:21Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1-clone", "Name": "cdb-dev", "CR Name": "cdb-dev"} +2022-06-27T03:38:01Z INFO controllers.PDB Cloned PDB successfully {"clonePDB": "oracle-database-operator-system/pdb1-clone", "Source PDB Name": "pdbnew", "Clone PDB Name": "pdbnewclone"} +2022-06-27T03:38:01Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1-clone", "Name": "cdb-dev", "CR Name": "cdb-dev"} +2022-06-27T03:38:01Z DEBUG events Normal {"object": {"kind":"PDB","namespace":"oracle-database-operator-system","name":"pdb1-clone","uid":"16276c26-60a3-463b-bdd5-10bd328f9d43","apiVersion":"database.oracle.com/v1alpha1","resourceVersion":"101449482"}, "reason": "Created", "message": "PDB 'pdbnewclone' cloned successfully"} +2022-06-27T03:38:01Z INFO controllers.PDB Issuing REST call {"callAPI": "oracle-database-operator-system/pdb1-clone", "URL": "http://cdb-dev-ords:8888/ords/_/db-api/latest/database/pdbs/pdbnewclone/status", "Action": "GET"} +2022-06-27T03:38:01Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1-clone", "Name": "cdb-dev", "CR Name": "cdb-dev"} +2022-06-27T03:38:01Z INFO controllers.PDB Successfully obtained PDB state {"getPDBState": "oracle-database-operator-system/pdb1-clone", "PDB Name": "pdbnewclone", "State": "READ WRITE"} +2022-06-27T03:38:01Z INFO controllers.PDB Reconcile completed {"onpremdboperator": "oracle-database-operator-system/pdb1-clone"} + + + +-- Check the status of the new PDB CRD resource created by cloning: + +% kubectl get pdbs -A +NAMESPACE NAME CONNECT STRING CDB NAME PDB NAME PDB STATE PDB SIZE STATUS MESSAGE +oracle-database-operator-system pdb1 goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com:1521/pdbnew goldcdb pdbnew READ WRITE 1G Ready Success +oracle-database-operator-system pdb1-clone goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com:1521/pdbnewclone goldcdb pdbnewclone READ WRITE Ready Success + + +-- Verify the new PDB created from the target CDB: + +SQL> show pdbs + + CON_ID CON_NAME OPEN MODE RESTRICTED +---------- ------------------------------ ---------- ---------- + 2 PDB$SEED READ WRITE NO + 3 PDBNEW READ WRITE NO + 4 PDBNEWCLONE READ WRITE NO diff --git a/docs/onpremdb/provisioning/clone_pdb.md b/docs/onpremdb/provisioning/clone_pdb.md new file mode 100644 index 00000000..225c35ea --- /dev/null +++ b/docs/onpremdb/provisioning/clone_pdb.md @@ -0,0 +1,36 @@ +# Clone a PDB using Oracle DB Operator On-Prem Controller in a target CDB + +In this use case, a PDB is cloned using Oracle DB Operator On-Prem controller. + +**NOTE:** It is assumed that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-manage-pdb-life-cycle-using-oracle-db-operator-on-prem-database-controller) steps. + +This example uses `clone_pdb.yaml` to clone a PDB using Oracle DB Operator On-Prem Controller with: + +- PDB CRD resource Name as `pdb1-clone` +- Pluggable Database (PDB) Name as `pdbnewclone` +- Total Size of the PDB as `UNLIMITED` +- Total size for temporary tablespace as `UNLIMITED` +- Target CDB CRD Resource Name as `cdb-dev` +- Target CDB name as `goldcdb` +- Source PDB Name as `pdbnew` + +**NOTE:** For the details of the parameters to be used in the .yaml file, please refer [config/crd/bases/database.oracle.com_pdbs.yaml](../../../config/crd/bases/database.oracle.com_pdbs.yaml) + +Use the file: [clone_pdb.yaml](./clone_pdb.yaml) for this use case as below: + +1. Deploy the .yaml file: +```sh +[root@test-server oracle-database-operator]# kubectl apply -f clone_pdb.yaml +``` + +2. Monitor the Oracle DB Operator Pod for the progress of the PDB creation. + +NOTE: Check the DB Operator Pod name in your environment. + +``` +[root@test-server oracle-database-operator]# kubectl logs -f pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd -n oracle-database-operator-system +``` + +## Sample Output + +[Here](./clone_pdb.log) is the sample output for a PDB created using Oracle DB Operator On-Prem Controller using file [clone_pdb.yaml](./clone_pdb.yaml) diff --git a/docs/onpremdb/provisioning/clone_pdb.yaml b/docs/onpremdb/provisioning/clone_pdb.yaml new file mode 100644 index 00000000..7d3cfff9 --- /dev/null +++ b/docs/onpremdb/provisioning/clone_pdb.yaml @@ -0,0 +1,20 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1-clone + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbName: "goldcdb" + pdbName: "pdbnewclone" + srcPdbName: "pdbnew" + fileNameConversions: "NONE" + totalSize: "UNLIMITED" + tempSize: "UNLIMITED" + action: "Clone" diff --git a/docs/onpremdb/provisioning/create_pdb.log b/docs/onpremdb/provisioning/create_pdb.log new file mode 100644 index 00000000..7ae2d6f0 --- /dev/null +++ b/docs/onpremdb/provisioning/create_pdb.log @@ -0,0 +1,139 @@ +-- Check the Oracle DB Operator Pod and CDB CRD Pod status: + +% kubectl get all -n oracle-database-operator-system +NAME READY STATUS RESTARTS AGE +pod/cdb-dev-ords-rs-5bztb 1/1 Running 0 5m23s +pod/cdb-dev-ords-rs-q2b68 1/1 Running 0 12m +pod/oracle-database-operator-controller-manager-76cb674c5c-4nrh8 1/1 Running 0 4d10h +pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd 1/1 Running 1 4d10h +pod/oracle-database-operator-controller-manager-76cb674c5c-xsv9g 1/1 Running 2 4d10h + +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +service/cdb-dev-ords ClusterIP None 11m +service/oracle-database-operator-controller-manager-metrics-service ClusterIP 10.96.98.47 8443/TCP 5d2h +service/oracle-database-operator-webhook-service ClusterIP 10.96.166.163 443/TCP 5d2h + +NAME READY UP-TO-DATE AVAILABLE AGE +deployment.apps/oracle-database-operator-controller-manager 3/3 3 3 5d2h + +NAME DESIRED CURRENT READY AGE +replicaset.apps/cdb-dev-ords-rs 2 2 2 12m +replicaset.apps/oracle-database-operator-controller-manager-76cb674c5c 3 3 3 5d2h + + + +-- PDB secrets in this use case were created using the below file: + +% cat pdb_secret.yaml +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: v1 +kind: Secret +metadata: + name: pdb1-secret + namespace: oracle-database-operator-system +type: Opaque +data: + sysadmin_user: "cGRiYWRtaW4=" + sysadmin_pwd: "V0VsY29tZV8yMSMj" + + +-- This is the .yaml file used to create a PDB CRD resource in this use case: + +% cat pdb.yaml +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbName: "goldcdb" + pdbName: "pdbnew" + adminName: + secret: + secretName: "pdb1-secret" + key: "sysadmin_user" + adminPwd: + secret: + secretName: "pdb1-secret" + key: "sysadmin_pwd" + fileNameConversions: "NONE" + totalSize: "1G" + tempSize: "100M" + action: "Create" + + + + +-- Apply the .yaml files: + +% kubectl apply -f pdb_secret.yaml +secret/pdb1-secret created + +% kubectl apply -f pdb.yaml +pdb.database.oracle.com/pdb1 created + + +-- Monitor the logs from the Oracle DB Operator Pod: + +% kubectl logs -f pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd -n oracle-database-operator-system +. +. +2022-06-27T03:28:30Z DEBUG controller-runtime.webhook.webhooks received request {"webhook": "/mutate-database-oracle-com-v1alpha1-pdb", "UID": "6fca0e37-8fd9-4ccd-86ad-2edec604a28b", "kind": "database.oracle.com/v1alpha1, Kind=PDB", "resource": {"group":"database.oracle.com","version":"v1alpha1","resource":"pdbs"}} +2022-06-27T03:28:30Z INFO pdb-webhook Setting default values in PDB spec for : pdb1 +2022-06-27T03:28:30Z INFO pdb-webhook - reuseTempFile : true +2022-06-27T03:28:30Z INFO pdb-webhook - unlimitedStorage : true +2022-06-27T03:28:30Z INFO pdb-webhook - tdeImport : false +2022-06-27T03:28:30Z INFO pdb-webhook - tdeExport : false +2022-06-27T03:28:30Z INFO pdb-webhook - asClone : false +2022-06-27T03:28:30Z DEBUG controller-runtime.webhook.webhooks wrote response {"webhook": "/mutate-database-oracle-com-v1alpha1-pdb", "code": 200, "reason": "", "UID": "6fca0e37-8fd9-4ccd-86ad-2edec604a28b", "allowed": true} +2022-06-27T03:28:30Z DEBUG controller-runtime.webhook.webhooks received request {"webhook": "/validate-database-oracle-com-v1alpha1-pdb", "UID": "6ae20043-fa1a-4eba-b943-a2266183da48", "kind": "database.oracle.com/v1alpha1, Kind=PDB", "resource": {"group":"database.oracle.com","version":"v1alpha1","resource":"pdbs"}} +2022-06-27T03:28:30Z INFO pdb-webhook ValidateUpdate-Validating PDB spec for : pdb1 +2022-06-27T03:28:30Z INFO pdb-webhook validateCommon {"name": "pdb1"} +2022-06-27T03:28:30Z INFO pdb-webhook Valdiating PDB Resource Action : CREATE +2022-06-27T03:28:30Z DEBUG controller-runtime.webhook.webhooks wrote response {"webhook": "/validate-database-oracle-com-v1alpha1-pdb", "code": 200, "reason": "", "UID": "6ae20043-fa1a-4eba-b943-a2266183da48", "allowed": true} +2022-06-27T03:28:30Z INFO controllers.PDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/pdb1"} +2022-06-27T03:28:30Z INFO controllers.PDB Validating PDB phase for: pdb1 {"validatePhase": "oracle-database-operator-system/pdb1", "Action": "CREATE"} +2022-06-27T03:28:30Z INFO controllers.PDB Validation complete {"validatePhase": "oracle-database-operator-system/pdb1"} +2022-06-27T03:28:30Z INFO controllers.PDB PDB: {"onpremdboperator": "oracle-database-operator-system/pdb1", "Name": "pdb1", "Phase": "Creating", "Status": "false"} +2022-06-27T03:28:30Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} +2022-06-27T03:28:30Z INFO controllers.PDB Issuing REST call {"callAPI": "oracle-database-operator-system/pdb1", "URL": "http://cdb-dev-ords:8888/ords/_/db-api/latest/database/pdbs/", "Action": "POST"} +2022-06-27T03:28:30Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} +2022-06-27T03:29:04Z INFO controllers.PDB Created PDB Resource {"createPDB": "oracle-database-operator-system/pdb1", "PDB Name": "pdbnew"} +2022-06-27T03:29:04Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} +2022-06-27T03:29:04Z DEBUG events Normal {"object": {"kind":"PDB","namespace":"oracle-database-operator-system","name":"pdb1","uid":"81f2e686-6e1b-4e2c-8a2f-e20c2f99d6b9","apiVersion":"database.oracle.com/v1alpha1","resourceVersion":"101446779"}, "reason": "Created", "message": "PDB 'pdbnew' created successfully"} +2022-06-27T03:29:04Z INFO controllers.PDB Issuing REST call {"callAPI": "oracle-database-operator-system/pdb1", "URL": "http://cdb-dev-ords:8888/ords/_/db-api/latest/database/pdbs/pdbnew/status", "Action": "GET"} +2022-06-27T03:29:04Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} +2022-06-27T03:29:04Z INFO controllers.PDB Successfully obtained PDB state {"getPDBState": "oracle-database-operator-system/pdb1", "PDB Name": "pdbnew", "State": "READ WRITE"} +2022-06-27T03:29:04Z INFO controllers.PDB Reconcile completed {"onpremdboperator": "oracle-database-operator-system/pdb1"} + + + + +-- Check the status of the PDB CRD Resource created: + +% kubectl get pdbs -A +NAMESPACE NAME CONNECT STRING CDB NAME PDB NAME PDB STATE PDB SIZE STATUS MESSAGE +oracle-database-operator-system pdb1 goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com:1521/pdbnew goldcdb pdbnew READ WRITE 1G Ready Success + + +-- The status as "Ready" and message as "Success" confirms that the resource has been created successfully. + + +-- Verify that the PDB is created from CDB: + +SQL> show pdbs + + CON_ID CON_NAME OPEN MODE RESTRICTED +---------- ------------------------------ ---------- ---------- + 2 PDB$SEED READ WRITE NO + 3 PDBNEW READ WRITE NO diff --git a/docs/onpremdb/provisioning/create_pdb.md b/docs/onpremdb/provisioning/create_pdb.md new file mode 100644 index 00000000..d6296baf --- /dev/null +++ b/docs/onpremdb/provisioning/create_pdb.md @@ -0,0 +1,37 @@ +# Create a PDB using Oracle DB Operator On-Prem Controller in a target CDB + +The Oracle Database Operator On-Prem Controller creates the PDB kind as a custom resource that models a PDB as a native Kubernetes object. There is a one-to-one mapping between the actual PDB and the Kubernetes PDB Custom Resource. Each PDB resource follows the PDB CRD as defined here: [config/crd/bases/database.oracle.com_pdbs.yaml](../../../config/crd/bases/database.oracle.com_pdbs.yaml) + +To create a PDB CRD Resource, a sample .yaml file is available here: [config/samples/onpremdb/pdb.yaml](../../../config/samples/onpremdb/pdb.yaml) + +**NOTE:** It is assumed that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-manage-pdb-life-cycle-using-oracle-db-operator-on-prem-database-controller) steps. + +This example uses `create_pdb.yaml` to create a PDB using Oracle DB Operator On-Prem Controller with: + +- PDB CRD resource Name as `pdb1` +- Pluggable Database (PDB) Name as `pdbnew` +- Total Size of the PDB as `1GB` +- Total size for temporary tablespace as `100M` +- Target CDB CRD Resource Name as `cdb-dev` +- Target CDB name as `goldcdb` + +**NOTE:** For the details of the parameters to be used in the .yaml file, please refer [config/crd/bases/database.oracle.com_pdbs.yaml](../../../config/crd/bases/database.oracle.com_pdbs.yaml) + +Use the file: [create_pdb.yaml](./create_pdb.yaml) for this use case as below: + +1. Deploy the .yaml file: +```sh +[root@test-server oracle-database-operator]# kubectl apply -f create_pdb.yaml +``` + +2. Monitor the Oracle DB Operator Pod for the progress of the PDB creation. + +NOTE: Check the DB Operator Pod name in your environment. + +``` +[root@test-server oracle-database-operator]# kubectl logs -f pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd -n oracle-database-operator-system +``` + +## Sample Output + +[Here](./create_pdb.log) is the sample output for a PDB created using Oracle DB Operator On-Prem Controller using file [create_pdb.yaml](./create_pdb.yaml) diff --git a/docs/onpremdb/provisioning/create_pdb.yaml b/docs/onpremdb/provisioning/create_pdb.yaml new file mode 100644 index 00000000..82941185 --- /dev/null +++ b/docs/onpremdb/provisioning/create_pdb.yaml @@ -0,0 +1,27 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbName: "goldcdb" + pdbName: "pdbnew" + adminName: + secret: + secretName: "pdb1-secret" + key: "sysadmin_user" + adminPwd: + secret: + secretName: "pdb1-secret" + key: "sysadmin_pwd" + fileNameConversions: "NONE" + totalSize: "1G" + tempSize: "100M" + action: "Create" diff --git a/docs/onpremdb/provisioning/delete_pdb.log b/docs/onpremdb/provisioning/delete_pdb.log new file mode 100644 index 00000000..7f361871 --- /dev/null +++ b/docs/onpremdb/provisioning/delete_pdb.log @@ -0,0 +1,157 @@ +-- Check the existing PDB CRD resources: + +% kubectl get pdbs -A +NAMESPACE NAME CONNECT STRING CDB NAME PDB NAME PDB STATE PDB SIZE STATUS MESSAGE +oracle-database-operator-system pdb1 goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com:1521/pdbnew goldcdb pdbnew READ WRITE 1G Ready Success +oracle-database-operator-system pdb1-clone goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com:1521/pdbnewclone goldcdb pdbnewclone READ WRITE Ready Success + + +-- Also check from the database as well: + +SQL> show pdbs + + CON_ID CON_NAME OPEN MODE RESTRICTED +---------- ------------------------------ ---------- ---------- + 2 PDB$SEED READ WRITE NO + 3 PDBNEW READ WRITE NO + 4 PDBNEWCLONE READ WRITE NO + + + +% cat modify_pdb_close.yaml +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1-clone + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbName: "goldcdb" + pdbName: "pdbnewclone" + pdbState: "CLOSE" + modifyOption: "IMMEDIATE" + action: "Modify" + + +% kubectl apply -f modify_pdb_close.yaml +pdb.database.oracle.com/pdb1-clone configured + + + +% kubectl logs -f pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd -n oracle-database-operator-system +. +. +2022-06-27T04:19:36Z DEBUG controller-runtime.webhook.webhooks received request {"webhook": "/mutate-database-oracle-com-v1alpha1-pdb", "UID": "24842cc8-0047-46cc-86a5-2782a95e3e36", "kind": "database.oracle.com/v1alpha1, Kind=PDB", "resource": {"group":"database.oracle.com","version":"v1alpha1","resource":"pdbs"}} +2022-06-27T04:19:36Z INFO pdb-webhook Setting default values in PDB spec for : pdb1-clone +2022-06-27T04:19:36Z DEBUG controller-runtime.webhook.webhooks wrote response {"webhook": "/mutate-database-oracle-com-v1alpha1-pdb", "code": 200, "reason": "", "UID": "24842cc8-0047-46cc-86a5-2782a95e3e36", "allowed": true} +2022-06-27T04:19:36Z INFO controllers.PDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/pdb1-clone"} +2022-06-27T04:19:36Z INFO controllers.PDB Validating PDB phase for: pdb1-clone {"validatePhase": "oracle-database-operator-system/pdb1-clone", "Action": "MODIFY"} +2022-06-27T04:19:36Z INFO controllers.PDB Validation complete {"validatePhase": "oracle-database-operator-system/pdb1-clone"} +2022-06-27T04:19:36Z INFO controllers.PDB PDB: {"onpremdboperator": "oracle-database-operator-system/pdb1-clone", "Name": "pdb1-clone", "Phase": "Modifying", "Status": "false"} +2022-06-27T04:19:36Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1-clone", "Name": "cdb-dev", "CR Name": "cdb-dev"} +2022-06-27T04:19:36Z INFO controllers.PDB Issuing REST call {"callAPI": "oracle-database-operator-system/pdb1-clone", "URL": "http://cdb-dev-ords:8888/ords/_/db-api/latest/database/pdbs/pdbnewclone/status", "Action": "GET"} +2022-06-27T04:19:36Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1-clone", "Name": "cdb-dev", "CR Name": "cdb-dev"} +2022-06-27T04:19:38Z INFO controllers.PDB Successfully obtained PDB state {"getPDBState": "oracle-database-operator-system/pdb1-clone", "PDB Name": "pdbnewclone", "State": "READ WRITE"} +2022-06-27T04:19:38Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1-clone", "Name": "cdb-dev", "CR Name": "cdb-dev"} +2022-06-27T04:19:38Z INFO controllers.PDB Issuing REST call {"callAPI": "oracle-database-operator-system/pdb1-clone", "URL": "http://cdb-dev-ords:8888/ords/_/db-api/latest/database/pdbs/pdbnewclone/status", "Action": "POST"} +2022-06-27T04:19:38Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1-clone", "Name": "cdb-dev", "CR Name": "cdb-dev"} +2022-06-27T04:19:39Z INFO controllers.PDB Successfully modified PDB state {"modifyPDB": "oracle-database-operator-system/pdb1-clone", "PDB Name": "pdbnewclone"} +2022-06-27T04:19:39Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1-clone", "Name": "cdb-dev", "CR Name": "cdb-dev"} +2022-06-27T04:19:39Z DEBUG events Normal {"object": {"kind":"PDB","namespace":"oracle-database-operator-system","name":"pdb1-clone","uid":"309dd711-198b-45b6-a34b-da5069af70fb","apiVersion":"database.oracle.com/v1alpha1","resourceVersion":"101462484"}, "reason": "Modified", "message": "PDB 'pdbnewclone' modified successfully"} +2022-06-27T04:19:39Z INFO controllers.PDB Issuing REST call {"callAPI": "oracle-database-operator-system/pdb1-clone", "URL": "http://cdb-dev-ords:8888/ords/_/db-api/latest/database/pdbs/pdbnewclone/status", "Action": "GET"} +2022-06-27T04:19:39Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1-clone", "Name": "cdb-dev", "CR Name": "cdb-dev"} +2022-06-27T04:19:39Z INFO controllers.PDB Successfully obtained PDB state {"getPDBState": "oracle-database-operator-system/pdb1-clone", "PDB Name": "pdbnewclone", "State": "MOUNTED"} +2022-06-27T04:19:39Z INFO controllers.PDB Reconcile completed {"onpremdboperator": "oracle-database-operator-system/pdb1-clone"} + + + +SQL> show pdbs + + CON_ID CON_NAME OPEN MODE RESTRICTED +---------- ------------------------------ ---------- ---------- + 2 PDB$SEED READ WRITE NO + 3 PDBNEW READ WRITE NO + 4 PDBNEWCLONE MOUNTED + + +-- Check the .yaml file to be used to delete a PDB: + +% cat delete_pdb.yaml +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1-clone + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + pdbName: "pdbnewclone" + action: "Delete" + dropAction: "INCLUDING" + + + +-- Apply the .yaml file: + +% kubectl apply -f delete_pdb.yaml +pdb.database.oracle.com/pdb1-clone configured + + +-- Monitor the Oracle DB Operator Pod logs: + +% kubectl logs -f pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd -n oracle-database-operator-system +. +. +2022-06-27T04:21:37Z DEBUG controller-runtime.webhook.webhooks received request {"webhook": "/validate-database-oracle-com-v1alpha1-pdb", "UID": "64148dda-d0df-4e03-88e3-98b1ce7b7aaf", "kind": "database.oracle.com/v1alpha1, Kind=PDB", "resource": {"group":"database.oracle.com","version":"v1alpha1","resource":"pdbs"}} +2022-06-27T04:21:37Z INFO pdb-webhook ValidateUpdate-Validating PDB spec for : pdb1-clone +2022-06-27T04:21:37Z INFO pdb-webhook validateCommon {"name": "pdb1-clone"} +2022-06-27T04:21:37Z INFO pdb-webhook Valdiating PDB Resource Action : DELETE +2022-06-27T04:21:37Z DEBUG controller-runtime.webhook.webhooks wrote response {"webhook": "/validate-database-oracle-com-v1alpha1-pdb", "code": 200, "reason": "", "UID": "64148dda-d0df-4e03-88e3-98b1ce7b7aaf", "allowed": true} +2022-06-27T04:21:37Z INFO controllers.PDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/pdb1-clone"} +2022-06-27T04:21:37Z INFO controllers.PDB Validating PDB phase for: pdb1-clone {"validatePhase": "oracle-database-operator-system/pdb1-clone", "Action": "DELETE"} +2022-06-27T04:21:37Z INFO controllers.PDB Validation complete {"validatePhase": "oracle-database-operator-system/pdb1-clone"} +2022-06-27T04:21:37Z INFO controllers.PDB PDB: {"onpremdboperator": "oracle-database-operator-system/pdb1-clone", "Name": "pdb1-clone", "Phase": "Deleting", "Status": "false"} +2022-06-27T04:21:37Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1-clone", "Name": "cdb-dev", "CR Name": "cdb-dev"} +2022-06-27T04:21:37Z INFO controllers.PDB Issuing REST call {"callAPI": "oracle-database-operator-system/pdb1-clone", "URL": "http://cdb-dev-ords:8888/ords/_/db-api/latest/database/pdbs/pdbnewclone/", "Action": "DELETE"} +2022-06-27T04:21:37Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1-clone", "Name": "cdb-dev", "CR Name": "cdb-dev"} +2022-06-27T04:21:39Z INFO controllers.PDB Successfully dropped PDB {"deletePDBInstance": "oracle-database-operator-system/pdb1-clone", "PDB Name": "pdbnewclone"} +2022-06-27T04:21:39Z INFO controllers.PDB Removing finalizer {"deletePDB": "oracle-database-operator-system/pdb1-clone"} +2022-06-27T04:21:39Z DEBUG controller-runtime.webhook.webhooks received request {"webhook": "/validate-database-oracle-com-v1alpha1-pdb", "UID": "15bd320b-3f9f-46a7-8493-c586310b7d84", "kind": "database.oracle.com/v1alpha1, Kind=PDB", "resource": {"group":"database.oracle.com","version":"v1alpha1","resource":"pdbs"}} +2022-06-27T04:21:39Z INFO pdb-webhook ValidateUpdate-Validating PDB spec for : pdb1-clone +2022-06-27T04:21:39Z INFO pdb-webhook validateCommon {"name": "pdb1-clone"} +2022-06-27T04:21:39Z INFO pdb-webhook Valdiating PDB Resource Action : DELETE +2022-06-27T04:21:39Z DEBUG controller-runtime.webhook.webhooks wrote response {"webhook": "/validate-database-oracle-com-v1alpha1-pdb", "code": 200, "reason": "", "UID": "15bd320b-3f9f-46a7-8493-c586310b7d84", "allowed": true} +2022-06-27T04:21:39Z INFO controllers.PDB Successfully deleted PDB resource {"deletePDB": "oracle-database-operator-system/pdb1-clone"} +2022-06-27T04:21:39Z INFO controllers.PDB Reconcile completed {"onpremdboperator": "oracle-database-operator-system/pdb1-clone"} +2022-06-27T04:21:39Z DEBUG events Normal {"object": {"kind":"PDB","namespace":"oracle-database-operator-system","name":"pdb1-clone","uid":"309dd711-198b-45b6-a34b-da5069af70fb","apiVersion":"database.oracle.com/v1alpha1","resourceVersion":"101463106"}, "reason": "Deleted", "message": "PDB 'pdbnewclone' dropped successfully"} + + + + + +-- Check the PDB CRD resources: + +% kubectl get pdbs -A +NAMESPACE NAME CONNECT STRING CDB NAME PDB NAME PDB STATE PDB SIZE STATUS MESSAGE +oracle-database-operator-system pdb1 goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com:1521/pdbnew goldcdb pdbnew READ WRITE 1G Ready Success + + +-- Verify from the CDB: + +SQL> show pdbs + + CON_ID CON_NAME OPEN MODE RESTRICTED +---------- ------------------------------ ---------- ---------- + 2 PDB$SEED READ WRITE NO + 3 PDBNEW READ WRITE NO diff --git a/docs/onpremdb/provisioning/delete_pdb.md b/docs/onpremdb/provisioning/delete_pdb.md new file mode 100644 index 00000000..360793f4 --- /dev/null +++ b/docs/onpremdb/provisioning/delete_pdb.md @@ -0,0 +1,35 @@ +# Delete a PDB using Oracle DB Operator On-Prem Controller in a target CDB + +In this use case, a PDB is deleted using Oracle DB Operator On-Prem controller. + +**NOTE:** It is assumed that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-manage-pdb-life-cycle-using-oracle-db-operator-on-prem-database-controller) steps. + +This example uses `delete_pdb.yaml` to delete a PDB using Oracle DB Operator On-Prem Controller with: + +- Pluggable Database (PDB) Name as `pdbnewclone` +- Target CDB CRD Resource Name as `cdb-dev` +- Action to be taken on the PDB as `Delete` +- Option to specify if datafiles should be removed as `INCLUDING` + +**NOTE:** You need to *modify* the PDB status to MOUNTED, as described earlier, on the target CDB before you want to delete that PDB. + +**NOTE:** For the details of the parameters to be used in the .yaml file, please refer [config/crd/bases/database.oracle.com_pdbs.yaml](../../../config/crd/bases/database.oracle.com_pdbs.yaml) + +Use the file: [delete_pdb.yaml](./delete_pdb.yaml) for this use case as below: + +1. Deploy the .yaml file: +```sh +% kubectl apply -f delete_pdb.yaml +``` + +2. Monitor the Oracle DB Operator Pod `pod/oracle-database-operator-controller-manager-665874bd57-g2cgw` for the progress of the PDB deletion. + +NOTE: Check the DB Operator Pod name in your environment. + +```sh +% kubectl logs -f pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd -n oracle-database-operator-system +``` + +## Sample Output + +[Here](./delete_pdb.log) is the sample output for a PDB created using Oracle DB Operator On-Prem Controller using file [delete_pdb.yaml](./delete_pdb.yaml) diff --git a/docs/onpremdb/provisioning/delete_pdb.yaml b/docs/onpremdb/provisioning/delete_pdb.yaml new file mode 100644 index 00000000..d16084bb --- /dev/null +++ b/docs/onpremdb/provisioning/delete_pdb.yaml @@ -0,0 +1,16 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1-clone + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + pdbName: "pdbnewclone" + action: "Delete" + dropAction: "INCLUDING" diff --git a/docs/onpremdb/provisioning/example_setup_using_oci_oke_cluster.md b/docs/onpremdb/provisioning/example_setup_using_oci_oke_cluster.md new file mode 100644 index 00000000..ddf275a1 --- /dev/null +++ b/docs/onpremdb/provisioning/example_setup_using_oci_oke_cluster.md @@ -0,0 +1,55 @@ +# Example of a working setup using OCI OKE(Kubernetes Cluster) and a CDB in Cloud (OCI Exadata Database Cluster) + +In this example, the target CDB (for which the PDB life cycle management is needed) is running in a Cloud environment (OCI's [Oracle Exadata Database Service](https://docs.oracle.com/en-us/iaas/exadatacloud/index.html)) and to manage its PDBs, the Oracle DB Operator is running on a Kubernetes Cluster running in cloud (OCI's [Container Engine for Kubernetes or OKE](https://docs.oracle.com/en-us/iaas/Content/ContEng/Concepts/contengoverview.htm#Overview_of_Container_Engine_for_Kubernetes)). + + +## Environment details + +| Component | Region Name |VCN (Virtual Cloud Network) Name| +| ------------- | ------------- |--------------------------------| +| OKE Cluster | Phoenix Region|exaphxvcn| +| ExaCS Cluster | Phoenix Region|exaphxvcn| + + +## High Level plans for this setup + +Below are the main steps that will be involved in this setup: + +- Setup VCN, Add security lists +- Setup OKE cluster with custom settings +- Install Oracle Database Operator on OKE Cluster +- Install ords controller definition +- Manager pdb life cycle management. + + +## Reference security lists for Ingress/Egress rules for VPN "exaphxvcn" + +Refer to the below screen shots to create security lists for Ingress/Egress rules as follows for client & backup network in VPN exaphxvcn to allow k8s Worker node to talk to ExaCS db hosts. + +|Network Type|Reference Screenshots| +|------------|---------------------| +|client network| ![](../images/K8S_SECURE1.png),![](../images/K8S_SECURE2.png)| +|backup network| ![](../images/K8S_SECURE3.png),![](../images/K8S_SECURE4.png)| + + +## OKE Cluster + +Check the [Oracle Documentation](https://docs.oracle.com/en-us/iaas/Content/ContEng/Concepts/contengnetworkconfigexample.htm#example-privatek8sapi-privateworkers-publiclb) for the OKE rules settings. + +Create OKE cluster with CUSTOM option to use same VCN where ExaCS is provisioned. + +**NOTE:** Make sure you choose same VCN exaphxvcn where ExaCS is provisioned. + +After this, setup kubeconfig & validate cluster access as well as worker node access via ssh. + +For example, you should be able to check the available OKE nodes using "kubectl" as below: + +``` +% kubectl get nodes -o wide +NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME +192.168.194.163 Ready node 3d19h v1.23.4 192.168.194.163 XX.XX.XX.XX Oracle Linux Server 7.9 5.4.17-2136.306.1.3.el7uek.x86_64 cri-o://1.23.2 +192.168.194.169 Ready node 3d19h v1.23.4 192.168.194.169 XX.XX.XX.XX Oracle Linux Server 7.9 5.4.17-2136.306.1.3.el7uek.x86_64 cri-o://1.23.2 +192.168.194.241 Ready node 3d19h v1.23.4 192.168.194.241 XX.XX.XX.XX Oracle Linux Server 7.9 5.4.17-2136.306.1.3.el7uek.x86_64 cri-o://1.23.2 +``` + +Once this setup is ready, you can proceed with the installation of [Oracle Database Operator for Kubernetes](https://github.com/oracle/oracle-database-operator/blob/main/README.md) to use the Oracle On-prem controller to manage PDBs in this CDB. diff --git a/docs/onpremdb/provisioning/known_issues.md b/docs/onpremdb/provisioning/known_issues.md new file mode 100644 index 00000000..3ab19897 --- /dev/null +++ b/docs/onpremdb/provisioning/known_issues.md @@ -0,0 +1,49 @@ +# Known Issues + +Please refer to the below list of known issues related to Oracle DB Operator On-Prem Controller: + +1. **ORA-20002: ERROR: The user ORDS_PUBLIC_USER already exists in the logs of CDB CRD Pods.** + +This error is expected when you are deploying `2` replicas of CDB CRD during the deployment of the CDB CRD. Below is the snippet of a possible error: +``` +2022-06-22T20:06:32.616Z INFO Installing Oracle REST Data Services version 21.4.2.r0621806 in CDB$ROOT +2022-06-22T20:06:32.663Z INFO ... Log file written to /home/oracle/ords_cdb_install_core_CDB_ROOT_2022-06-22_200632_00662.log +2022-06-22T20:06:33.835Z INFO CDB restart file created in /home/oracle/ords_restart_2022-06-22_200633_00835.properties +2022-06-22T20:06:33.837Z SEVERE Error executing script: ords_prereq_env.sql Error: ORA-20002: ERROR: The user ORDS_PUBLIC_USER already exists. You must first uninstall ORDS using ords_uninstall.sql prior to running the install scripts. +ORA-06512: at line 8 +ORA-06512: at line 8 + + Refer to log file /home/oracle/ords_cdb_install_core_CDB_ROOT_2022-06-22_200632_00662.log for details + +java.io.IOException: Error executing script: ords_prereq_env.sql Error: ORA-20002: ERROR: The user ORDS_PUBLIC_USER already exists. You must first uninstall ORDS using ords_uninstall.sql prior to running the install scripts. +ORA-06512: at line 8 +ORA-06512: at line 8 +``` +This error is seen in the logs of one of the two CDB CRD pods. The other Pod `does not` show this error and the ORDS installation is done successfully. + +To avoid this error, you need to initially deploy the CDB CRD with a single replica and later add another replica as per need. + +2. **PDB create failure with error "Failed: Unauthorized"** + +It was observed that PDB creation fails with the below error when special characters like "_" or "#" were used in the password for user SQL_ADMIN: +``` +2022-06-22T20:10:09Z INFO controllers.PDB ORDS Error - HTTP Status Code :401 {"callAPI": "oracle-database-operator-system/pdb1", "Err": "\n{\n \"code\": \"Unauthorized\",\n \"message\": \"Unauthorized\",\n \"type\": \"tag:oracle.com,2020:error/Unauthorized\",\n \"instance\": \"tag:oracle.com,2020:ecid/OoqA0Zw3oBWdabzP8wUMcQ\"\n}"} +2022-06-22T20:10:09Z INFO controllers.PDB Reconcile completed {"onpremdboperator": "oracle-database-operator-system/pdb1"} +2022-06-22T20:10:09Z DEBUG events Warning {"object": {"kind":"PDB","namespace":"oracle-database-operator-system","name":"pdb1","uid":"19fc98b1-ca7f-4e63-a6c7-fdeb14b8c275","apiVersion":"database.oracle.com/v1alpha1","resourceVersion":"99558229"}, "reason": "ORDSError", "message": "Failed: Unauthorized"} +``` + +In testing, we have used the password `welcome1` for the user SQL_ADMIN. + +To avoid this error, please avoid password `welcome1` for SQL_ADMIN user. + + +3. **After cloning a PDB from another PDB, PDB SIZE field is show as empty even if the .yaml file used during the PDB cloning specifies the PDB size:** + +```sh +% kubectl get pdbs -A +NAMESPACE NAME CONNECT STRING CDB NAME PDB NAME PDB STATE PDB SIZE STATUS MESSAGE +oracle-database-operator-system pdb1 goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com:1521/pdbnew goldcdb pdbnew READ WRITE 1G Ready Success +oracle-database-operator-system pdb1-clone goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com:1521/pdbnewclone goldcdb pdbnewclone READ WRITE Ready Success +``` + +In the above example the PDB `pdbnewclone` is cloned from PDB `pdbnew` and is showing the size column as EMPTY. This will be fixed in future version. diff --git a/docs/onpremdb/provisioning/modify_pdb.log b/docs/onpremdb/provisioning/modify_pdb.log new file mode 100644 index 00000000..151393a7 --- /dev/null +++ b/docs/onpremdb/provisioning/modify_pdb.log @@ -0,0 +1,181 @@ +----- Closing a PDB ------ + +-- Check the existing PDB CRD resources + +jyotiprakashverma@jyotiprakashverma-mac onprem_test % kubectl get pdbs -A +NAMESPACE NAME CONNECT STRING CDB NAME PDB NAME PDB STATE PDB SIZE STATUS MESSAGE +oracle-database-operator-system pdb1 goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com:1521/pdbnew goldcdb pdbnew READ WRITE 1G Ready Success +oracle-database-operator-system pdb1-clone goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com:1521/pdbnewclone goldcdb pdbnewclone READ WRITE Ready Success + + + + +-- Check the status of the PDBs in the CDB: + +SQL> show pdbs + + CON_ID CON_NAME OPEN MODE RESTRICTED +---------- ------------------------------ ---------- ---------- + 2 PDB$SEED READ WRITE NO + 3 PDBNEW READ WRITE NO + 5 PDBNEWCLONE READ WRITE NO + + + +-- Check the file to modify the PDB state to CLOSE: + +% cat modify_pdb_close.yaml +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbName: "goldcdb" + pdbName: "pdbnew" + pdbState: "CLOSE" + modifyOption: "IMMEDIATE" + action: "Modify" + + +-- Apply the file: + +% kubectl apply -f modify_pdb_close.yaml +pdb.database.oracle.com/pdb1 configured + + +-- Monitor the logs from the Oracle DB Operator Pod: + +% kubectl logs -f pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd -n oracle-database-operator-system +. +. +2022-06-27T03:44:36Z DEBUG controller-runtime.webhook.webhooks received request {"webhook": "/mutate-database-oracle-com-v1alpha1-pdb", "UID": "08f95926-6bf1-4c70-b319-1b17015ce22a", "kind": "database.oracle.com/v1alpha1, Kind=PDB", "resource": {"group":"database.oracle.com","version":"v1alpha1","resource":"pdbs"}} +2022-06-27T03:44:36Z INFO pdb-webhook Setting default values in PDB spec for : pdb1 +2022-06-27T03:44:36Z DEBUG controller-runtime.webhook.webhooks wrote response {"webhook": "/mutate-database-oracle-com-v1alpha1-pdb", "code": 200, "reason": "", "UID": "08f95926-6bf1-4c70-b319-1b17015ce22a", "allowed": true} +2022-06-27T03:44:36Z DEBUG controller-runtime.webhook.webhooks received request {"webhook": "/validate-database-oracle-com-v1alpha1-pdb", "UID": "64ebebe2-87f2-4237-8928-532365f3cca9", "kind": "database.oracle.com/v1alpha1, Kind=PDB", "resource": {"group":"database.oracle.com","version":"v1alpha1","resource":"pdbs"}} +2022-06-27T03:44:36Z INFO pdb-webhook ValidateUpdate-Validating PDB spec for : pdb1 +2022-06-27T03:44:36Z INFO pdb-webhook validateCommon {"name": "pdb1"} +2022-06-27T03:44:36Z INFO pdb-webhook Valdiating PDB Resource Action : MODIFY +2022-06-27T03:44:36Z DEBUG controller-runtime.webhook.webhooks wrote response {"webhook": "/validate-database-oracle-com-v1alpha1-pdb", "code": 200, "reason": "", "UID": "64ebebe2-87f2-4237-8928-532365f3cca9", "allowed": true} +2022-06-27T03:44:36Z INFO controllers.PDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/pdb1"} +2022-06-27T03:44:36Z INFO controllers.PDB Validating PDB phase for: pdb1 {"validatePhase": "oracle-database-operator-system/pdb1", "Action": "MODIFY"} +2022-06-27T03:44:36Z INFO controllers.PDB Validation complete {"validatePhase": "oracle-database-operator-system/pdb1"} +2022-06-27T03:44:36Z INFO controllers.PDB PDB: {"onpremdboperator": "oracle-database-operator-system/pdb1", "Name": "pdb1", "Phase": "Modifying", "Status": "false"} +2022-06-27T03:44:36Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} +2022-06-27T03:44:36Z INFO controllers.PDB Issuing REST call {"callAPI": "oracle-database-operator-system/pdb1", "URL": "http://cdb-dev-ords:8888/ords/_/db-api/latest/database/pdbs/pdbnew/status", "Action": "GET"} +2022-06-27T03:44:36Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} +2022-06-27T03:44:36Z INFO controllers.PDB Successfully obtained PDB state {"getPDBState": "oracle-database-operator-system/pdb1", "PDB Name": "pdbnew", "State": "READ WRITE"} +2022-06-27T03:44:36Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} +2022-06-27T03:44:36Z INFO controllers.PDB Issuing REST call {"callAPI": "oracle-database-operator-system/pdb1", "URL": "http://cdb-dev-ords:8888/ords/_/db-api/latest/database/pdbs/pdbnew/status", "Action": "POST"} +2022-06-27T03:44:36Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} +2022-06-27T03:44:38Z INFO controllers.PDB Successfully modified PDB state {"modifyPDB": "oracle-database-operator-system/pdb1", "PDB Name": "pdbnew"} +2022-06-27T03:44:38Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} +2022-06-27T03:44:38Z DEBUG events Normal {"object": {"kind":"PDB","namespace":"oracle-database-operator-system","name":"pdb1","uid":"81f2e686-6e1b-4e2c-8a2f-e20c2f99d6b9","apiVersion":"database.oracle.com/v1alpha1","resourceVersion":"101451707"}, "reason": "Modified", "message": "PDB 'pdbnew' modified successfully"} +2022-06-27T03:44:38Z INFO controllers.PDB Issuing REST call {"callAPI": "oracle-database-operator-system/pdb1", "URL": "http://cdb-dev-ords:8888/ords/_/db-api/latest/database/pdbs/pdbnew/status", "Action": "GET"} +2022-06-27T03:44:38Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} +2022-06-27T03:44:38Z INFO controllers.PDB Successfully obtained PDB state {"getPDBState": "oracle-database-operator-system/pdb1", "PDB Name": "pdbnew", "State": "MOUNTED"} +2022-06-27T03:44:38Z INFO controllers.PDB Reconcile completed {"onpremdboperator": "oracle-database-operator-system/pdb1"} + + + + +-- Check the status of PDB CRD resources + +jyotiprakashverma@jyotiprakashverma-mac onprem_test % kubectl get pdbs -A +NAMESPACE NAME CONNECT STRING CDB NAME PDB NAME PDB STATE PDB SIZE STATUS MESSAGE +oracle-database-operator-system pdb1 goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com:1521/pdbnew goldcdb pdbnew MOUNTED 1G Ready Success +oracle-database-operator-system pdb1-clone goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com:1521/pdbnewclone goldcdb pdbnewclone READ WRITE Ready Success + + + +-- Confirm the status of the PDB in the CDB: + +SQL> show pdbs + + CON_ID CON_NAME OPEN MODE RESTRICTED +---------- ------------------------------ ---------- ---------- + 2 PDB$SEED READ WRITE NO + 3 PDBNEW MOUNTED + 4 PDBNEWCLONE READ WRITE NO + + + + + + + + +----- Opening a PDB ------ + +-- Check the .yaml file to open the PDB: + +% cat modify_pdb_open.yaml +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbName: "goldcdb" + pdbName: "pdbnew" + pdbState: "OPEN" + modifyOption: "READ WRITE" + action: "Modify" + + + +-- Apply the file: + +% kubectl apply -f modify_pdb_open.yaml +pdb.database.oracle.com/pdb1 configured + + +-- Monitor the logs from the Oracle DB Operator Pod: + +% kubectl logs -f pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd -n oracle-database-operator-system +. +. +2022-06-27T03:48:38Z INFO controllers.PDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/pdb1"} +2022-06-27T03:48:38Z INFO controllers.PDB Validating PDB phase for: pdb1 {"validatePhase": "oracle-database-operator-system/pdb1", "Action": "MODIFY"} +2022-06-27T03:48:38Z INFO controllers.PDB Validation complete {"validatePhase": "oracle-database-operator-system/pdb1"} +2022-06-27T03:48:38Z INFO controllers.PDB PDB: {"onpremdboperator": "oracle-database-operator-system/pdb1", "Name": "pdb1", "Phase": "Modifying", "Status": "false"} +2022-06-27T03:48:38Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} +2022-06-27T03:48:38Z INFO controllers.PDB Issuing REST call {"callAPI": "oracle-database-operator-system/pdb1", "URL": "http://cdb-dev-ords:8888/ords/_/db-api/latest/database/pdbs/pdbnew/status", "Action": "GET"} +2022-06-27T03:48:38Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} +2022-06-27T03:48:38Z INFO controllers.PDB Successfully obtained PDB state {"getPDBState": "oracle-database-operator-system/pdb1", "PDB Name": "pdbnew", "State": "MOUNTED"} +2022-06-27T03:48:38Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} +2022-06-27T03:48:38Z INFO controllers.PDB Issuing REST call {"callAPI": "oracle-database-operator-system/pdb1", "URL": "http://cdb-dev-ords:8888/ords/_/db-api/latest/database/pdbs/pdbnew/status", "Action": "POST"} +2022-06-27T03:48:38Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} +2022-06-27T03:48:41Z INFO controllers.PDB Successfully modified PDB state {"modifyPDB": "oracle-database-operator-system/pdb1", "PDB Name": "pdbnew"} +2022-06-27T03:48:41Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} +2022-06-27T03:48:41Z DEBUG events Normal {"object": {"kind":"PDB","namespace":"oracle-database-operator-system","name":"pdb1","uid":"81f2e686-6e1b-4e2c-8a2f-e20c2f99d6b9","apiVersion":"database.oracle.com/v1alpha1","resourceVersion":"101452939"}, "reason": "Modified", "message": "PDB 'pdbnew' modified successfully"} +2022-06-27T03:48:41Z INFO controllers.PDB Issuing REST call {"callAPI": "oracle-database-operator-system/pdb1", "URL": "http://cdb-dev-ords:8888/ords/_/db-api/latest/database/pdbs/pdbnew/status", "Action": "GET"} +2022-06-27T03:48:41Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} +2022-06-27T03:48:41Z INFO controllers.PDB Successfully obtained PDB state {"getPDBState": "oracle-database-operator-system/pdb1", "PDB Name": "pdbnew", "State": "READ WRITE"} +2022-06-27T03:48:41Z INFO controllers.PDB Reconcile completed {"onpremdboperator": "oracle-database-operator-system/pdb1"} + + + +-- Verify the status of the PDB: + +SQL> show pdbs + + CON_ID CON_NAME OPEN MODE RESTRICTED +---------- ------------------------------ ---------- ---------- + 2 PDB$SEED READ WRITE NO + 3 PDBNEW READ WRITE NO + 4 PDBNEWCLONE READ WRITE NO diff --git a/docs/onpremdb/provisioning/modify_pdb.md b/docs/onpremdb/provisioning/modify_pdb.md new file mode 100644 index 00000000..7e22d420 --- /dev/null +++ b/docs/onpremdb/provisioning/modify_pdb.md @@ -0,0 +1,67 @@ +# Modify a PDB using Oracle DB Operator On-Prem Controller in a target CDB + +In this use case, the state of an existing PDB is modified using Oracle DB Operator On-Prem controller. + +**NOTE:** It is assumed that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-manage-pdb-life-cycle-using-oracle-db-operator-on-prem-database-controller) steps. + +Subcase 1: This example uses `modify_pdb_close.yaml` to close a PDB using Oracle DB Operator On-Prem Controller with: + +- PDB CRD resource Name as `pdb1` +- Pluggable Database (PDB) Name as `pdbnew` +- Target CDB CRD Resource Name as `cdb-dev` +- Target CDB name as `goldcdb` +- Action to be taken on the PDB as `MODIFY` +- Target state of the PDB as `CLOSE` +- Option to close the state (i.e. modify) as `IMMEDIATE` + + +**NOTE:** For the details of the parameters to be used in the .yaml file, please refer [config/crd/bases/database.oracle.com_pdbs.yaml](../../../config/crd/bases/database.oracle.com_pdbs.yaml) + +Use the file: [modify_pdb_close.yaml](./modify_pdb_close.yaml) for this use case as below: + +1. Deploy the .yaml file: +```sh +% kubectl apply -f modify_pdb_close.yaml +pdb.database.oracle.com/pdb1 configured +``` + +2. Monitor the Oracle DB Operator Pod for the progress of the PDB creation. + +NOTE: Check the DB Operator Pod name in your environment. + +``` +[root@test-server oracle-database-operator]# kubectl logs -f pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd -n oracle-database-operator-system +``` + +Subcase 2: This example uses `modify_pdb_open.yaml` to open a PDB using Oracle DB Operator On-Prem Controller with: + +- PDB CRD resource Name as `pdb1` +- Pluggable Database (PDB) Name as `pdbnew` +- Target CDB CRD Resource Name as `cdb-dev` +- Target CDB name as `goldcdb` +- Action to be taken on the PDB as `MODIFY` +- Target state of the PDB as `OPEN` +- Option to close the state (i.e. modify) as `READ WRITE` + + +**NOTE:** For the details of the parameters to be used in the .yaml file, please refer [config/crd/bases/database.oracle.com_pdbs.yaml](../../config/crd/bases/database.oracle.com_pdbs.yaml) + +Use the file: [modify_pdb_open.yaml](./modify_pdb_open.yaml) for this use case as below: + +1. Deploy the .yaml file: +```sh +% kubectl apply -f modify_pdb_open.yaml +pdb.database.oracle.com/pdb1 configured +``` + +2. Monitor the Oracle DB Operator Pod for the progress of the PDB creation. + +NOTE: Check the DB Operator Pod name in your environment. + +``` +[root@test-server oracle-database-operator]# kubectl logs -f pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd -n oracle-database-operator-system +``` + +## Sample Output + +[Here](./modify_pdb.log) is the sample output for a PDB created using Oracle DB Operator On-Prem Controller using file [modify_pdb_close.yaml](./modify_pdb_close.yaml) and [modify_pdb_open.yaml](./modify_pdb_open.yaml) diff --git a/docs/onpremdb/provisioning/modify_pdb_close.yaml b/docs/onpremdb/provisioning/modify_pdb_close.yaml new file mode 100644 index 00000000..8897b482 --- /dev/null +++ b/docs/onpremdb/provisioning/modify_pdb_close.yaml @@ -0,0 +1,18 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbName: "goldcdb" + pdbName: "pdbnew" + pdbState: "CLOSE" + modifyOption: "IMMEDIATE" + action: "Modify" diff --git a/docs/onpremdb/provisioning/modify_pdb_open.yaml b/docs/onpremdb/provisioning/modify_pdb_open.yaml new file mode 100644 index 00000000..0ec640d4 --- /dev/null +++ b/docs/onpremdb/provisioning/modify_pdb_open.yaml @@ -0,0 +1,18 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbName: "goldcdb" + pdbName: "pdbnew" + pdbState: "OPEN" + modifyOption: "READ WRITE" + action: "Modify" diff --git a/docs/onpremdb/provisioning/ords_image.log b/docs/onpremdb/provisioning/ords_image.log new file mode 100644 index 00000000..7cce536a --- /dev/null +++ b/docs/onpremdb/provisioning/ords_image.log @@ -0,0 +1,263 @@ +-- Clone the software using git + +% git clone git@orahub.oci.oraclecorp.com:rac-docker-dev/oracle-database-operator.git +Cloning into 'oracle-database-operator'... +Use of the Oracle network and applications is intended solely for Oracle's authorized users. The use of these resources by Oracle employees and contractors is subject to company policies, including the Code of Conduct, Acceptable Use Policy and Information Protection Policy; access may be monitored and logged, to the extent permitted by law, in accordance with Oracle policies. Unauthorized use may result in termination of your access, disciplinary action and/or civil and criminal penalties. +remote: Enumerating objects: 11244, done. +remote: Counting objects: 100% (506/506), done. +remote: Compressing objects: 100% (205/205), done. +remote: Total 11244 (delta 305), reused 493 (delta 296), pack-reused 10738 +Receiving objects: 100% (11244/11244), 4.11 MiB | 397.00 KiB/s, done. +Resolving deltas: 100% (7510/7510), done. +% cd oracle-database-operator/ords/ + + +-- Download the ORDS Version 21.4.3 software and copy that to the current location: + +% ls -rlt +total 79968 +-rw-rw-rw- 1 root root 81861484 Jun 26 04:07 ords-21.4.3.117.0405.zip +-rw-r--r-- 1 root root 266 Jun 26 04:13 standalone.properties.tmpl +-rw-r--r-- 1 root root 639 Jun 26 04:13 setupwebuser.sh +-rw-r--r-- 1 root root 3096 Jun 26 04:13 runOrds.sh +-rw-r--r-- 1 root root 427 Jun 26 04:13 ords_params.properties.tmpl +-rw-r--r-- 1 root root 2230 Jun 26 04:13 Dockerfile +-rw-r--r-- 1 root root 123 Jun 26 04:13 cdbadmin.properties.tmpl + + +-- Login to repository: login container-registry.oracle.com +% docker login container-registry.oracle.com +Username: +Password: +WARNING! Your password will be stored unencrypted in /root/.docker/config.json. +Configure a credential helper to remove this warning. See +https://docs.docker.com/engine/reference/commandline/login/#credentials-store + +Login Succeeded + + +-- Login to a repository where you want to push the ORDS docker image: +% docker login phx.ocir.io +Username (/>): +Password: +WARNING! Your password will be stored unencrypted in /root/.docker/config.json. +Configure a credential helper to remove this warning. See +https://docs.docker.com/engine/reference/commandline/login/#credentials-store + +Login Succeeded + + +-- Build the ORDS Docker Image: +[root@docker-test-server ords]# docker build -t oracle/ords-dboper:ords-21.4.3 . +Sending build context to Docker daemon 81.88MB +Step 1/10 : FROM container-registry.oracle.com/java/jdk:latest + ---> b94ea1fc0f64 +Step 2/10 : ENV ORDS_HOME=/opt/oracle/ords INSTALL_FILE=ords*.zip CONFIG_PROPS="ords_params.properties.tmpl" STANDALONE_PROPS="standalone.properties.tmpl" CDBADMIN_PROPS="cdbadmin.properties.tmpl" SETUP_WEBUSER="setupwebuser.sh" RUN_FILE="runOrds.sh" + ---> Running in baf28954867a +Removing intermediate container baf28954867a + ---> 40d169e44c21 +Step 3/10 : COPY $INSTALL_FILE $CONFIG_PROPS $STANDALONE_PROPS $CDBADMIN_PROPS $RUN_FILE $SETUP_WEBUSER $ORDS_HOME/ + ---> 1d6a27f6ce13 +Step 4/10 : RUN yum -y install bind-utils net-tools zip unzip tar wget vim-minimal which sudo expect procps && yum clean all + ---> Running in e0879fd25edc +Oracle Linux 8 BaseOS Latest (x86_64) 74 MB/s | 46 MB 00:00 +Oracle Linux 8 Application Stream (x86_64) 66 MB/s | 36 MB 00:00 +Last metadata expiration check: 0:00:15 ago on Mon 27 Jun 2022 04:34:12 PM UTC. +Package vim-minimal-2:8.0.1763-16.0.2.el8_5.13.x86_64 is already installed. +Package procps-ng-3.3.15-6.0.1.el8.x86_64 is already installed. +Dependencies resolved. +================================================================================ + Package Arch Version Repository Size +================================================================================ +Installing: + bind-utils x86_64 32:9.11.36-3.el8 ol8_appstream 452 k + expect x86_64 5.45.4-5.el8 ol8_baseos_latest 266 k + net-tools x86_64 2.0-0.52.20160912git.el8 ol8_baseos_latest 322 k + sudo x86_64 1.8.29-8.el8 ol8_baseos_latest 925 k + tar x86_64 2:1.30-5.el8 ol8_baseos_latest 838 k + unzip x86_64 6.0-46.el8 ol8_baseos_latest 196 k + wget x86_64 1.19.5-10.0.1.el8 ol8_appstream 734 k + which x86_64 2.21-17.el8 ol8_baseos_latest 49 k + zip x86_64 3.0-23.el8 ol8_baseos_latest 270 k +Installing dependencies: + bind-libs x86_64 32:9.11.36-3.el8 ol8_appstream 175 k + bind-libs-lite x86_64 32:9.11.36-3.el8 ol8_appstream 1.2 M + bind-license noarch 32:9.11.36-3.el8 ol8_appstream 103 k + fstrm x86_64 0.6.1-2.el8 ol8_appstream 29 k + libmaxminddb x86_64 1.2.0-10.el8 ol8_appstream 33 k + libmetalink x86_64 0.1.3-7.el8 ol8_baseos_latest 32 k + protobuf-c x86_64 1.3.0-6.el8 ol8_appstream 37 k + python3-bind noarch 32:9.11.36-3.el8 ol8_appstream 150 k + python3-ply noarch 3.9-9.el8 ol8_baseos_latest 111 k + tcl x86_64 1:8.6.8-2.el8 ol8_baseos_latest 1.1 M + +Transaction Summary +================================================================================ +Install 19 Packages + +Total download size: 6.9 M +Installed size: 21 M +Downloading Packages: +(1/19): libmetalink-0.1.3-7.el8.x86_64.rpm 683 kB/s | 32 kB 00:00 +(2/19): net-tools-2.0-0.52.20160912git.el8.x86_ 5.7 MB/s | 322 kB 00:00 +(3/19): expect-5.45.4-5.el8.x86_64.rpm 4.3 MB/s | 266 kB 00:00 +(4/19): python3-ply-3.9-9.el8.noarch.rpm 5.4 MB/s | 111 kB 00:00 +(5/19): sudo-1.8.29-8.el8.x86_64.rpm 23 MB/s | 925 kB 00:00 +(6/19): unzip-6.0-46.el8.x86_64.rpm 12 MB/s | 196 kB 00:00 +(7/19): tar-1.30-5.el8.x86_64.rpm 14 MB/s | 838 kB 00:00 +(8/19): which-2.21-17.el8.x86_64.rpm 3.9 MB/s | 49 kB 00:00 +(9/19): tcl-8.6.8-2.el8.x86_64.rpm 16 MB/s | 1.1 MB 00:00 +(10/19): zip-3.0-23.el8.x86_64.rpm 12 MB/s | 270 kB 00:00 +(11/19): bind-libs-9.11.36-3.el8.x86_64.rpm 7.2 MB/s | 175 kB 00:00 +(12/19): bind-license-9.11.36-3.el8.noarch.rpm 9.6 MB/s | 103 kB 00:00 +(13/19): fstrm-0.6.1-2.el8.x86_64.rpm 4.8 MB/s | 29 kB 00:00 +(14/19): libmaxminddb-1.2.0-10.el8.x86_64.rpm 4.6 MB/s | 33 kB 00:00 +(15/19): bind-utils-9.11.36-3.el8.x86_64.rpm 17 MB/s | 452 kB 00:00 +(16/19): protobuf-c-1.3.0-6.el8.x86_64.rpm 3.4 MB/s | 37 kB 00:00 +(17/19): python3-bind-9.11.36-3.el8.noarch.rpm 16 MB/s | 150 kB 00:00 +(18/19): bind-libs-lite-9.11.36-3.el8.x86_64.rp 19 MB/s | 1.2 MB 00:00 +(19/19): wget-1.19.5-10.0.1.el8.x86_64.rpm 24 MB/s | 734 kB 00:00 +-------------------------------------------------------------------------------- +Total 31 MB/s | 6.9 MB 00:00 +Running transaction check +Transaction check succeeded. +Running transaction test +Transaction test succeeded. +Running transaction + Preparing : 1/1 + Installing : protobuf-c-1.3.0-6.el8.x86_64 1/19 + Installing : libmaxminddb-1.2.0-10.el8.x86_64 2/19 + Running scriptlet: libmaxminddb-1.2.0-10.el8.x86_64 2/19 + Installing : fstrm-0.6.1-2.el8.x86_64 3/19 + Installing : bind-license-32:9.11.36-3.el8.noarch 4/19 + Installing : bind-libs-lite-32:9.11.36-3.el8.x86_64 5/19 + Installing : bind-libs-32:9.11.36-3.el8.x86_64 6/19 + Installing : unzip-6.0-46.el8.x86_64 7/19 + Installing : tcl-1:8.6.8-2.el8.x86_64 8/19 + Running scriptlet: tcl-1:8.6.8-2.el8.x86_64 8/19 + Installing : python3-ply-3.9-9.el8.noarch 9/19 + Installing : python3-bind-32:9.11.36-3.el8.noarch 10/19 + Installing : libmetalink-0.1.3-7.el8.x86_64 11/19 + Installing : wget-1.19.5-10.0.1.el8.x86_64 12/19 + Running scriptlet: wget-1.19.5-10.0.1.el8.x86_64 12/19 + Installing : bind-utils-32:9.11.36-3.el8.x86_64 13/19 + Installing : expect-5.45.4-5.el8.x86_64 14/19 + Installing : zip-3.0-23.el8.x86_64 15/19 + Installing : which-2.21-17.el8.x86_64 16/19 + Installing : tar-2:1.30-5.el8.x86_64 17/19 + Running scriptlet: tar-2:1.30-5.el8.x86_64 17/19 + Installing : sudo-1.8.29-8.el8.x86_64 18/19 + Running scriptlet: sudo-1.8.29-8.el8.x86_64 18/19 + Installing : net-tools-2.0-0.52.20160912git.el8.x86_64 19/19 + Running scriptlet: net-tools-2.0-0.52.20160912git.el8.x86_64 19/19 + Verifying : expect-5.45.4-5.el8.x86_64 1/19 + Verifying : libmetalink-0.1.3-7.el8.x86_64 2/19 + Verifying : net-tools-2.0-0.52.20160912git.el8.x86_64 3/19 + Verifying : python3-ply-3.9-9.el8.noarch 4/19 + Verifying : sudo-1.8.29-8.el8.x86_64 5/19 + Verifying : tar-2:1.30-5.el8.x86_64 6/19 + Verifying : tcl-1:8.6.8-2.el8.x86_64 7/19 + Verifying : unzip-6.0-46.el8.x86_64 8/19 + Verifying : which-2.21-17.el8.x86_64 9/19 + Verifying : zip-3.0-23.el8.x86_64 10/19 + Verifying : bind-libs-32:9.11.36-3.el8.x86_64 11/19 + Verifying : bind-libs-lite-32:9.11.36-3.el8.x86_64 12/19 + Verifying : bind-license-32:9.11.36-3.el8.noarch 13/19 + Verifying : bind-utils-32:9.11.36-3.el8.x86_64 14/19 + Verifying : fstrm-0.6.1-2.el8.x86_64 15/19 + Verifying : libmaxminddb-1.2.0-10.el8.x86_64 16/19 + Verifying : protobuf-c-1.3.0-6.el8.x86_64 17/19 + Verifying : python3-bind-32:9.11.36-3.el8.noarch 18/19 + Verifying : wget-1.19.5-10.0.1.el8.x86_64 19/19 + +Installed: + bind-libs-32:9.11.36-3.el8.x86_64 + bind-libs-lite-32:9.11.36-3.el8.x86_64 + bind-license-32:9.11.36-3.el8.noarch + bind-utils-32:9.11.36-3.el8.x86_64 + expect-5.45.4-5.el8.x86_64 + fstrm-0.6.1-2.el8.x86_64 + libmaxminddb-1.2.0-10.el8.x86_64 + libmetalink-0.1.3-7.el8.x86_64 + net-tools-2.0-0.52.20160912git.el8.x86_64 + protobuf-c-1.3.0-6.el8.x86_64 + python3-bind-32:9.11.36-3.el8.noarch + python3-ply-3.9-9.el8.noarch + sudo-1.8.29-8.el8.x86_64 + tar-2:1.30-5.el8.x86_64 + tcl-1:8.6.8-2.el8.x86_64 + unzip-6.0-46.el8.x86_64 + wget-1.19.5-10.0.1.el8.x86_64 + which-2.21-17.el8.x86_64 + zip-3.0-23.el8.x86_64 + +Complete! +17 files removed +Removing intermediate container e0879fd25edc + ---> 824de67afe8c +Step 5/10 : RUN mkdir -p $ORDS_HOME/doc_root && chmod ug+x $ORDS_HOME/*.sh && groupadd -g 54322 dba && useradd -u 54321 -d /home/oracle -g dba -m -s /bin/bash oracle && cd $ORDS_HOME && jar -xf $INSTALL_FILE && rm $INSTALL_FILE && mkdir -p $ORDS_HOME/config/ords && mkdir -p $ORDS_HOME/secrets && java -jar $ORDS_HOME/ords.war configdir $ORDS_HOME/config && chown -R oracle:dba $ORDS_HOME + ---> Running in 623f0b8b31a0 +2022-06-27T16:34:43.462Z INFO Set config.dir to /opt/oracle/ords/config in: /opt/oracle/ords/ords.war +Removing intermediate container 623f0b8b31a0 + ---> b5b4693d5ecf +Step 6/10 : USER oracle + ---> Running in dd481ff2fc2d +Removing intermediate container dd481ff2fc2d + ---> e4d3603d79ec +Step 7/10 : WORKDIR /home/oracle + ---> Running in 5aba61a54f70 +Removing intermediate container 5aba61a54f70 + ---> bf0501caf083 +Step 8/10 : VOLUME ["$ORDS_HOME/config/ords"] + ---> Running in 26e4dbd9c1ed +Removing intermediate container 26e4dbd9c1ed + ---> dc4134ac42aa +Step 9/10 : EXPOSE 8888 + ---> Running in a5fbea15a1ee +Removing intermediate container a5fbea15a1ee + ---> cf615843f041 +Step 10/10 : CMD $ORDS_HOME/$RUN_FILE + ---> Running in 444eaf7568e8 +Removing intermediate container 444eaf7568e8 + ---> 7c95be92a842 +Successfully built 7c95be92a842 +Successfully tagged oracle/ords-dboper:ords-21.4.3 + + + +-- Check the docker image details: + +docker images +REPOSITORY TAG IMAGE ID CREATED SIZE +oracle/ords-dboper ords-21.4.3 7c95be92a842 44 seconds ago 771M + + +-- Tag and push the image to your docker repository +-- NOTE: We have used the repo as "phx.ocir.io//oracle/ords" +% docker tag oracle/ords-dboper:ords-21.4.3 phx.ocir.io//oracle/ords:21.4.3 +% docker push phx.ocir.io//oracle/ords:21.4.3 +The push refers to repository [phx.ocir.io//oracle/ords] +98222b4c695a: Pushed +d8fd7c53f02f: Pushed +2023d4f36fb3: Pushed +a14158bad0d3: Layer already exists +0c1b321d1256: Layer already exists +74a1be7b5642: Layer already exists +21.4.3: digest: sha256:0f75cba13605944d2ea970c38330b65bd81509aa2cfc63d63f454769d61c1d5b size: 1589 + + +-- Verify the image pushed to your docker repository. + +For example, in above example, you can check the image on the cloud console with signature "sha256:0f75cba13605944d2ea970c38330b65bd81509aa2cfc63d63f454769d61c1d5b" + + +-- Create a Kubernetes Secret for the docker repository (if its private) + +% kubectl create secret generic container-registry-secret --from-file=.dockerconfigjson=./.docker/config.json --type=kubernetes.io/dockerconfigjson -n oracle-database-operator-system +secret/container-registry-secret created + +% kubectl get secret -n oracle-database-operator-system +NAME TYPE DATA AGE +container-registry-secret kubernetes.io/dockerconfigjson 1 26s +default-token-x86d6 kubernetes.io/service-account-token 3 5d16h +webhook-server-cert kubernetes.io/tls 3 5d16h diff --git a/docs/onpremdb/provisioning/ords_image.md b/docs/onpremdb/provisioning/ords_image.md new file mode 100644 index 00000000..4399f3fb --- /dev/null +++ b/docs/onpremdb/provisioning/ords_image.md @@ -0,0 +1,68 @@ +# Build ORDS Docker Image + +In the below steps, we are building an ORDS Docker Image for ORDS Software. + +The image built can be later pushed to a local repository to be used later for a deployment. + +**NOTE:** It is assumed that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-manage-pdb-life-cycle-using-oracle-db-operator-on-prem-database-controller) steps. + +1. Clone the software using git: +```sh +[root@test-server oracle-database-operator]# git clone git@orahub.oci.oraclecorp.com:rac-docker-dev/oracle-database-operator.git +[root@test-server oracle-database-operator]# cd oracle-database-operator/ords/ +``` + +2. Download the ORDS Software for required ORDS version. For Example: For ORDS Version 21.4.3, use this [link](https://www.oracle.com/tools/ords/ords-downloads-2143.html) + +3. Copy the downloaded software .zip file to the current location. + +4. Login to the registry: container-registry.oracle.com + +**NOTE:** To login to this registry, you will need to the URL https://container-registry.oracle.com , Sign in, then click on "Java" and then accept the agreement. + +```sh +docker login container-registry.oracle.com +``` + +5. Login to a repo where you want to push your docker image (if needed) to pull during deployment in your environment. + +```sh +docker login +``` + +6. Build the docker image by using below command: + +```sh +docker build -t oracle/ords-dboper:ords-21.4.3 . +``` + +7. Check the docker image details using: + +```sh +docker images +``` + +8. Tag and push the image to your docker repository. + +NOTE: We have the repo as `phx.ocir.io//oracle/ords:21.4.3`. Please change as per your environment. + +```sh +docker tag oracle/ords-dboper:ords-21.4.3 phx.ocir.io//oracle/ords:21.4.3 +docker push phx.ocir.io//oracle/ords:21.4.3 +``` + +9. Verify the image pushed to your docker repository. + +You can refer to below sample output for above steps as well. + +10. Create a Kubernetes Secret for your docker repository to pull the image during deployment using the below command: + +```sh +kubectl create secret generic container-registry-secret --from-file=.dockerconfigjson=./.docker/config.json --type=kubernetes.io/dockerconfigjson -n oracle-database-operator-system +``` + +This Kubernetes secret will be provided in the .yaml file against the parameter `ordsImagePullSecret` to pull the ORDS Docker Image from your docker repository (if its a private repository). + +## Sample Output + +[Here](./ords_image.log) is the sample output for docker image created for ORDS version 21.4.3 diff --git a/docs/onpremdb/provisioning/pdb.log b/docs/onpremdb/provisioning/pdb.log new file mode 100644 index 00000000..e69de29b diff --git a/docs/onpremdb/provisioning/pdb.yaml b/docs/onpremdb/provisioning/pdb.yaml new file mode 100644 index 00000000..82941185 --- /dev/null +++ b/docs/onpremdb/provisioning/pdb.yaml @@ -0,0 +1,27 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbName: "goldcdb" + pdbName: "pdbnew" + adminName: + secret: + secretName: "pdb1-secret" + key: "sysadmin_user" + adminPwd: + secret: + secretName: "pdb1-secret" + key: "sysadmin_pwd" + fileNameConversions: "NONE" + totalSize: "1G" + tempSize: "100M" + action: "Create" diff --git a/docs/onpremdb/provisioning/pdb_crd_resource.md b/docs/onpremdb/provisioning/pdb_crd_resource.md new file mode 100644 index 00000000..e69de29b diff --git a/docs/onpremdb/provisioning/pdb_secret.log b/docs/onpremdb/provisioning/pdb_secret.log new file mode 100644 index 00000000..e69de29b diff --git a/docs/onpremdb/provisioning/pdb_secret.yaml b/docs/onpremdb/provisioning/pdb_secret.yaml new file mode 100644 index 00000000..22a84dc9 --- /dev/null +++ b/docs/onpremdb/provisioning/pdb_secret.yaml @@ -0,0 +1,13 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: v1 +kind: Secret +metadata: + name: pdb1-secret + namespace: oracle-database-operator-system +type: Opaque +data: + sysadmin_user: "cGRiYWRtaW4=" + sysadmin_pwd: "V0VsY29tZV8yMSMj" diff --git a/docs/onpremdb/provisioning/plug_pdb.log b/docs/onpremdb/provisioning/plug_pdb.log new file mode 100644 index 00000000..3a0be247 --- /dev/null +++ b/docs/onpremdb/provisioning/plug_pdb.log @@ -0,0 +1,100 @@ +-- Check the status of the PDB CRD Resources: + +% kubectl get pdbs -A +No resources found + + + +-- Verify from the CDB: + +SQL> show pdbs + + CON_ID CON_NAME OPEN MODE RESTRICTED +---------- ------------------------------ ---------- ---------- + 2 PDB$SEED READ WRITE NO + + + +-- Confirm the availability of the required .xml file: + +[oracle@goldhost1 ~]$ ls -lrt /tmp/pdbnewclone.xml +-rw-r--r-- 1 oracle asmadmin 9920 Jun 27 06:26 /tmp/pdbnewclone.xml + + +-- Use the below .yaml file for the plug in operation: + +% cat plug_pdb.yaml +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbName: "goldcdb" + pdbName: "pdbnew" + xmlFileName: "/tmp/pdbnewclone.xml" + fileNameConversions: "NONE" + sourceFileNameConversions: "NONE" + copyAction: "MOVE" + totalSize: "1G" + tempSize: "100M" + action: "Plug" + + + +-- Apply the .yaml file: + +% kubectl apply -f plug_pdb.yaml +pdb.database.oracle.com/pdb1 created + + +-- Monitor the logs from the Oracle DB Operator Pod: + +% kubectl logs -f pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd -n oracle-database-operator-system +. +. +2022-06-27T04:28:36Z DEBUG controller-runtime.webhook.webhooks received request {"webhook": "/validate-database-oracle-com-v1alpha1-pdb", "UID": "bfad69af-36be-4792-87e3-639323300167", "kind": "database.oracle.com/v1alpha1, Kind=PDB", "resource": {"group":"database.oracle.com","version":"v1alpha1","resource":"pdbs"}} +2022-06-27T04:28:36Z INFO pdb-webhook ValidateCreate-Validating PDB spec for : pdb1 +2022-06-27T04:28:36Z INFO pdb-webhook validateCommon {"name": "pdb1"} +2022-06-27T04:28:36Z INFO pdb-webhook Valdiating PDB Resource Action : PLUG +2022-06-27T04:28:36Z INFO pdb-webhook PDB Resource : pdb1 successfully validated for Action : PLUG +2022-06-27T04:28:36Z DEBUG controller-runtime.webhook.webhooks wrote response {"webhook": "/validate-database-oracle-com-v1alpha1-pdb", "code": 200, "reason": "", "UID": "bfad69af-36be-4792-87e3-639323300167", "allowed": true} +2022-06-27T04:28:36Z INFO controllers.PDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/pdb1"} +2022-06-27T04:28:36Z INFO controllers.PDB Adding finalizer {"managePDBDeletion": "oracle-database-operator-system/pdb1"} +2022-06-27T04:28:36Z INFO controllers.PDB Found PDB: pdb1 {"checkDuplicatePDB": "oracle-database-operator-system/pdb1"} +2022-06-27T04:28:36Z INFO controllers.PDB Validating PDB phase for: pdb1 {"validatePhase": "oracle-database-operator-system/pdb1", "Action": "PLUG"} +2022-06-27T04:28:36Z INFO controllers.PDB Validation complete {"validatePhase": "oracle-database-operator-system/pdb1"} +2022-06-27T04:28:36Z INFO controllers.PDB PDB: {"onpremdboperator": "oracle-database-operator-system/pdb1", "Name": "pdb1", "Phase": "Plugging", "Status": "false"} +2022-06-27T04:28:36Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} +2022-06-27T04:28:36Z INFO controllers.PDB Issuing REST call {"callAPI": "oracle-database-operator-system/pdb1", "URL": "http://cdb-dev-ords:8888/ords/_/db-api/latest/database/pdbs/", "Action": "POST"} +2022-06-27T04:28:36Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} +2022-06-27T04:29:07Z INFO controllers.PDB Successfully plugged PDB {"plugPDB": "oracle-database-operator-system/pdb1", "PDB Name": "pdbnew"} +2022-06-27T04:29:07Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} +2022-06-27T04:29:07Z DEBUG events Normal {"object": {"kind":"PDB","namespace":"oracle-database-operator-system","name":"pdb1","uid":"dd9bef3c-e493-4d5a-ae82-b24cbf5d0be3","apiVersion":"database.oracle.com/v1alpha1","resourceVersion":"101465242"}, "reason": "Created", "message": "PDB 'pdbnew' plugged successfully"} +2022-06-27T04:29:07Z INFO controllers.PDB Issuing REST call {"callAPI": "oracle-database-operator-system/pdb1", "URL": "http://cdb-dev-ords:8888/ords/_/db-api/latest/database/pdbs/pdbnew/status", "Action": "GET"} +2022-06-27T04:29:07Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} +2022-06-27T04:29:07Z INFO controllers.PDB Successfully obtained PDB state {"getPDBState": "oracle-database-operator-system/pdb1", "PDB Name": "pdbnew", "State": "READ WRITE"} +2022-06-27T04:29:07Z INFO controllers.PDB Reconcile completed {"onpremdboperator": "oracle-database-operator-system/pdb1"} + + + +-- Confirm the PDB CRD resource has been created and the PDB has been plugged in to the target CDB: + +% kubectl get pdbs -A +NAMESPACE NAME CONNECT STRING CDB NAME PDB NAME PDB STATE PDB SIZE STATUS MESSAGE +oracle-database-operator-system pdb1 goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com:1521/pdbnew goldcdb pdbnew READ WRITE 1G Ready Success + + +SQL> show pdbs + + CON_ID CON_NAME OPEN MODE RESTRICTED +---------- ------------------------------ ---------- ---------- + 2 PDB$SEED READ WRITE NO + 4 PDBNEW READ WRITE NO diff --git a/docs/onpremdb/provisioning/plug_pdb.md b/docs/onpremdb/provisioning/plug_pdb.md new file mode 100644 index 00000000..059cc91d --- /dev/null +++ b/docs/onpremdb/provisioning/plug_pdb.md @@ -0,0 +1,42 @@ +# Plug in a PDB using Oracle DB Operator On-Prem Controller in a target CDB + +In this use case, a PDB is plugged in using Oracle DB Operator On-Prem controller using an existing .xml file which was generated when the PDB was unplugged from this target CDB or another CDB. + +**NOTE:** It is assumed that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-manage-pdb-life-cycle-using-oracle-db-operator-on-prem-database-controller) steps. + +This example uses `plug_pdb.yaml` to plug in a PDB to a target CDB using Oracle DB Operator On-Prem Controller with: + +- Pluggable Database CRD Resource Name as `pdb1` +- Pluggable Database (PDB) Name as `pdbnew` +- Target CDB CRD Resource Name as `cdb-dev` +- CDB Name as `goldcdb` +- Action to be taken on the PDB as `Plug` +- XML metadata filename as `/tmp/pdbnewclone.xml` +- Source File Name Conversion as `NONE` +- File Name Conversion as `NONE` +- Copy Action as `MOVE` +- PDB Size as `1G` +- Temporary tablespace Size as `100M` + +**NOTE:** For the details of the parameters to be used in the .yaml file, please refer [config/crd/bases/database.oracle.com_pdbs.yaml](../../../config/crd/bases/database.oracle.com_pdbs.yaml) + +**NOTE:** Before performing the plug inoperation, you will first need to confirm the availability of the .xml file and the PDB datafiles. + +Use the file: [plug_pdb.yaml](./plug_pdb.yaml) for this use case as below: + +1. Deploy the .yaml file: +```sh +% kubectl apply -f plug_pdb.yaml +``` + +2. Monitor the Oracle DB Operator Pod for the progress of the PDB Unplug operation: + +NOTE: Check the DB Operator Pod name in your environment. + +```sh +% kubectl logs -f pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd -n oracle-database-operator-system +``` + +## Sample Output + +[Here](./plug_pdb.log) is the sample output for a PDB created using Oracle DB Operator On-Prem Controller using file [plug_pdb.yaml](./plug_pdb.yaml) diff --git a/docs/onpremdb/provisioning/plug_pdb.yaml b/docs/onpremdb/provisioning/plug_pdb.yaml new file mode 100644 index 00000000..a27d3255 --- /dev/null +++ b/docs/onpremdb/provisioning/plug_pdb.yaml @@ -0,0 +1,22 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbName: "goldcdb" + pdbName: "pdbnew" + xmlFileName: "/tmp/pdbnewclone.xml" + fileNameConversions: "NONE" + sourceFileNameConversions: "NONE" + copyAction: "MOVE" + totalSize: "1G" + tempSize: "100M" + action: "Plug" diff --git a/docs/onpremdb/provisioning/unplug_pdb.log b/docs/onpremdb/provisioning/unplug_pdb.log new file mode 100644 index 00000000..c0995f83 --- /dev/null +++ b/docs/onpremdb/provisioning/unplug_pdb.log @@ -0,0 +1,165 @@ +-- Check the status of the PDB CRD resource: + +% kubectl get pdbs -A +NAMESPACE NAME CONNECT STRING CDB NAME PDB NAME PDB STATE PDB SIZE STATUS MESSAGE +oracle-database-operator-system pdb1 goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com:1521/pdbnew goldcdb pdbnew READ WRITE 1G Ready Success + + +-- Verify the status from the CDB: + +SQL> show pdbs + + CON_ID CON_NAME OPEN MODE RESTRICTED +---------- ------------------------------ ---------- ---------- + 2 PDB$SEED READ WRITE NO + 3 PDBNEW READ WRITE NO + + +-- Use the below .yaml file to close the PDB: + +% cat modify_pdb_close.yaml +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbName: "goldcdb" + pdbName: "pdbnew" + pdbState: "CLOSE" + modifyOption: "IMMEDIATE" + action: "Modify" + + +-- Apply the .yaml file: + +% kubectl apply -f modify_pdb_close.yaml +pdb.database.oracle.com/pdb1 configured + + +-- Monitor the Oracle DB Operator Pod logs: + +% kubectl logs -f pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd -n oracle-database-operator-system +. +. +2022-06-27T04:25:00Z DEBUG controller-runtime.webhook.webhooks received request {"webhook": "/mutate-database-oracle-com-v1alpha1-pdb", "UID": "1623a6e2-d7dc-4b0f-8aa8-efada76cac13", "kind": "database.oracle.com/v1alpha1, Kind=PDB", "resource": {"group":"database.oracle.com","version":"v1alpha1","resource":"pdbs"}} +2022-06-27T04:25:00Z INFO pdb-webhook Setting default values in PDB spec for : pdb1 +2022-06-27T04:25:00Z DEBUG controller-runtime.webhook.webhooks wrote response {"webhook": "/mutate-database-oracle-com-v1alpha1-pdb", "code": 200, "reason": "", "UID": "1623a6e2-d7dc-4b0f-8aa8-efada76cac13", "allowed": true} +2022-06-27T04:25:00Z INFO controllers.PDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/pdb1"} +2022-06-27T04:25:00Z INFO controllers.PDB Validating PDB phase for: pdb1 {"validatePhase": "oracle-database-operator-system/pdb1", "Action": "MODIFY"} +2022-06-27T04:25:00Z INFO controllers.PDB Validation complete {"validatePhase": "oracle-database-operator-system/pdb1"} +2022-06-27T04:25:00Z INFO controllers.PDB PDB: {"onpremdboperator": "oracle-database-operator-system/pdb1", "Name": "pdb1", "Phase": "Modifying", "Status": "false"} +2022-06-27T04:25:00Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} +2022-06-27T04:25:00Z INFO controllers.PDB Issuing REST call {"callAPI": "oracle-database-operator-system/pdb1", "URL": "http://cdb-dev-ords:8888/ords/_/db-api/latest/database/pdbs/pdbnew/status", "Action": "GET"} +2022-06-27T04:25:00Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} +2022-06-27T04:25:00Z INFO controllers.PDB Successfully obtained PDB state {"getPDBState": "oracle-database-operator-system/pdb1", "PDB Name": "pdbnew", "State": "READ WRITE"} +2022-06-27T04:25:00Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} +2022-06-27T04:25:01Z INFO controllers.PDB Issuing REST call {"callAPI": "oracle-database-operator-system/pdb1", "URL": "http://cdb-dev-ords:8888/ords/_/db-api/latest/database/pdbs/pdbnew/status", "Action": "POST"} +2022-06-27T04:25:01Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} +2022-06-27T04:25:02Z INFO controllers.PDB Successfully modified PDB state {"modifyPDB": "oracle-database-operator-system/pdb1", "PDB Name": "pdbnew"} +2022-06-27T04:25:02Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} +2022-06-27T04:25:02Z DEBUG events Normal {"object": {"kind":"PDB","namespace":"oracle-database-operator-system","name":"pdb1","uid":"447346c7-cfb0-43ed-abb2-a0fac844a3e4","apiVersion":"database.oracle.com/v1alpha1","resourceVersion":"101464133"}, "reason": "Modified", "message": "PDB 'pdbnew' modified successfully"} +2022-06-27T04:25:02Z INFO controllers.PDB Issuing REST call {"callAPI": "oracle-database-operator-system/pdb1", "URL": "http://cdb-dev-ords:8888/ords/_/db-api/latest/database/pdbs/pdbnew/status", "Action": "GET"} +2022-06-27T04:25:02Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} +2022-06-27T04:25:02Z INFO controllers.PDB Successfully obtained PDB state {"getPDBState": "oracle-database-operator-system/pdb1", "PDB Name": "pdbnew", "State": "MOUNTED"} +2022-06-27T04:25:02Z INFO controllers.PDB Reconcile completed {"onpremdboperator": "oracle-database-operator-system/pdb1"} + + + +-- Confirm the PDB is now in MOUNT status: + +% kubectl get pdbs -A +NAMESPACE NAME CONNECT STRING CDB NAME PDB NAME PDB STATE PDB SIZE STATUS MESSAGE +oracle-database-operator-system pdb1 goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com:1521/pdbnew goldcdb pdbnew MOUNTED 1G Ready Success + + +SQL> show pdbs + + CON_ID CON_NAME OPEN MODE RESTRICTED +---------- ------------------------------ ---------- ---------- + 2 PDB$SEED READ WRITE NO + 3 PDBNEW MOUNTED + + + + +-- Use the below .yaml file to unplug the PDB: + +% cat unplug_pdb.yaml +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbName: "goldcdb" + pdbName: "pdbnew" + xmlFileName: "/tmp/pdbnewclone.xml" + action: "Unplug" + + +-- Apply the .yaml file: + +% kubectl apply -f unplug_pdb.yaml +pdb.database.oracle.com/pdb1 configured + + +-- Monitor the Oracle DB Operator Pod logs: + +% kubectl logs -f pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd -n oracle-database-operator-system +. +. +2022-06-27T04:26:10Z DEBUG controller-runtime.webhook.webhooks received request {"webhook": "/mutate-database-oracle-com-v1alpha1-pdb", "UID": "0f292426-8839-46b6-ba30-b3ffeee7e644", "kind": "database.oracle.com/v1alpha1, Kind=PDB", "resource": {"group":"database.oracle.com","version":"v1alpha1","resource":"pdbs"}} +2022-06-27T04:26:10Z INFO pdb-webhook Setting default values in PDB spec for : pdb1 +2022-06-27T04:26:10Z DEBUG controller-runtime.webhook.webhooks wrote response {"webhook": "/mutate-database-oracle-com-v1alpha1-pdb", "code": 200, "reason": "", "UID": "0f292426-8839-46b6-ba30-b3ffeee7e644", "allowed": true} +2022-06-27T04:26:10Z INFO controllers.PDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/pdb1"} +2022-06-27T04:26:10Z INFO controllers.PDB Validating PDB phase for: pdb1 {"validatePhase": "oracle-database-operator-system/pdb1", "Action": "UNPLUG"} +2022-06-27T04:26:10Z INFO controllers.PDB Validation complete {"validatePhase": "oracle-database-operator-system/pdb1"} +2022-06-27T04:26:10Z INFO controllers.PDB PDB: {"onpremdboperator": "oracle-database-operator-system/pdb1", "Name": "pdb1", "Phase": "Unplugging", "Status": "false"} +2022-06-27T04:26:10Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} +2022-06-27T04:26:10Z INFO controllers.PDB Issuing REST call {"callAPI": "oracle-database-operator-system/pdb1", "URL": "http://cdb-dev-ords:8888/ords/_/db-api/latest/database/pdbs/pdbnew/", "Action": "POST"} +2022-06-27T04:26:10Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} +2022-06-27T04:26:18Z INFO controllers.PDB Removing finalizer {"unplugPDB": "oracle-database-operator-system/pdb1"} +2022-06-27T04:26:19Z DEBUG controller-runtime.webhook.webhooks received request {"webhook": "/validate-database-oracle-com-v1alpha1-pdb", "UID": "ce9e1a52-a372-4e3b-b148-c1e31bcb26f8", "kind": "database.oracle.com/v1alpha1, Kind=PDB", "resource": {"group":"database.oracle.com","version":"v1alpha1","resource":"pdbs"}} +2022-06-27T04:26:19Z INFO pdb-webhook ValidateUpdate-Validating PDB spec for : pdb1 +2022-06-27T04:26:19Z INFO pdb-webhook validateCommon {"name": "pdb1"} +2022-06-27T04:26:19Z INFO pdb-webhook Valdiating PDB Resource Action : UNPLUG +2022-06-27T04:26:19Z DEBUG controller-runtime.webhook.webhooks wrote response {"webhook": "/validate-database-oracle-com-v1alpha1-pdb", "code": 200, "reason": "", "UID": "ce9e1a52-a372-4e3b-b148-c1e31bcb26f8", "allowed": true} +2022-06-27T04:26:19Z INFO controllers.PDB Successfully unplugged PDB resource {"unplugPDB": "oracle-database-operator-system/pdb1"} +2022-06-27T04:26:19Z INFO controllers.PDB Reconcile completed {"onpremdboperator": "oracle-database-operator-system/pdb1"} +2022-06-27T04:26:19Z DEBUG events Normal {"object": {"kind":"PDB","namespace":"oracle-database-operator-system","name":"pdb1","uid":"447346c7-cfb0-43ed-abb2-a0fac844a3e4","apiVersion":"database.oracle.com/v1alpha1","resourceVersion":"101464533"}, "reason": "Unplugged", "message": "PDB 'pdbnew' unplugged successfully"} + + + +-- Confirm the PDB has been unplugged: + +% kubectl get pdbs -A +No resources found + + +SQL> show pdbs + + CON_ID CON_NAME OPEN MODE RESTRICTED +---------- ------------------------------ ---------- ---------- + 2 PDB$SEED READ WRITE NO + + + +-- Confirm the .xml file generated in the CDB host: + +[oracle@goldhost1 ~]$ ls -lrt /tmp/pdbnewclone.xml +-rw-r--r-- 1 oracle asmadmin 9920 Jun 27 06:26 /tmp/pdbnewclone.xml diff --git a/docs/onpremdb/provisioning/unplug_pdb.md b/docs/onpremdb/provisioning/unplug_pdb.md new file mode 100644 index 00000000..eb317ef7 --- /dev/null +++ b/docs/onpremdb/provisioning/unplug_pdb.md @@ -0,0 +1,37 @@ +# Unplug a PDB using Oracle DB Operator On-Prem Controller in a target CDB + +In this use case, a PDB is unplugged using Oracle DB Operator On-Prem controller. + +**NOTE:** It is assumed that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-manage-pdb-life-cycle-using-oracle-db-operator-on-prem-database-controller) steps. + +This example uses `unplug_pdb.yaml` to unplug a PDB from a target CDB using Oracle DB Operator On-Prem Controller with: + +- Pluggable Database CRD Resource Name as `pdb1` +- Pluggable Database (PDB) Name as `pdbnew` +- Target CDB CRD Resource Name as `cdb-dev` +- CDB Name as `goldcdb` +- Action to be taken on the PDB as `Unplug` +- XML metadata filename as `/tmp/pdbnewclone.xml` + +**NOTE:** For the details of the parameters to be used in the .yaml file, please refer [config/crd/bases/database.oracle.com_pdbs.yaml](../../../config/crd/bases/database.oracle.com_pdbs.yaml) + +**NOTE:** Before performing the unplug operation on the PDB CRD Resource, you will first need to perform the Modify Operation on that PDB CRD resource to Close the the PDB. After that you will be able to perform the Unplug operation. Please refer to the use case to modify the PDB state to Close. + +Use the file: [unplug_pdb.yaml](./unplug_pdb.yaml) for this use case as below: + +1. Deploy the .yaml file: +```sh +% kubectl apply -f unplug_pdb.yaml +``` + +2. Monitor the Oracle DB Operator Pod for the progress of the PDB Unplug operation: + +NOTE: Check the DB Operator Pod name in your environment. + +```sh +% kubectl logs -f pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd -n oracle-database-operator-system +``` + +## Sample Output + +[Here](./unplug_pdb.log) is the sample output for a PDB created using Oracle DB Operator On-Prem Controller using file [unplug_pdb.yaml](./unplug_pdb.yaml) diff --git a/docs/onpremdb/provisioning/unplug_pdb.yaml b/docs/onpremdb/provisioning/unplug_pdb.yaml new file mode 100644 index 00000000..c9915b28 --- /dev/null +++ b/docs/onpremdb/provisioning/unplug_pdb.yaml @@ -0,0 +1,17 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbName: "goldcdb" + pdbName: "pdbnew" + xmlFileName: "/tmp/pdbnewclone.xml" + action: "Unplug" diff --git a/docs/onpremdb/provisioning/validation_error.md b/docs/onpremdb/provisioning/validation_error.md new file mode 100644 index 00000000..ec527cdb --- /dev/null +++ b/docs/onpremdb/provisioning/validation_error.md @@ -0,0 +1,73 @@ +#Validation and Errors + +## Kubernetes Events +You can check Kubernetes events for any errors or status updates as shown below: +```sh +$ kubectl get events -A +NAMESPACE LAST SEEN TYPE REASON OBJECT MESSAGE +oracle-database-operator-system 58m Warning Failed pod/cdb-dev-ords-qiigr Error: secret "cdb1-secret" not found +oracle-database-operator-system 56m Normal DeletedORDSPod cdb/cdb-dev Deleted ORDS Pod(s) for cdb-dev +oracle-database-operator-system 56m Normal DeletedORDSService cdb/cdb-dev Deleted ORDS Service for cdb-dev +... +oracle-database-operator-system 26m Warning OraError pdb/pdb1 ORA-65016: FILE_NAME_CONVERT must be specified... +oracle-database-operator-system 24m Warning OraError pdb/pdb2 ORA-65011: Pluggable database DEMOTEST does not exist. +... +oracle-database-operator-system 20m Normal Created pdb/pdb1 PDB 'demotest' created successfully +... +oracle-database-operator-system 17m Warning OraError pdb/pdb3 ORA-65012: Pluggable database DEMOTEST already exists... +``` + +In case of successfull operation, you can see messages like below: + +```sh +% kubectl get events -A +NAMESPACE LAST SEEN TYPE REASON OBJECT MESSAGE +kube-system 33s Warning BackOff pod/kube-apiserver Back-off restarting failed container +oracle-database-operator-system 59m Normal CreatedORDSService cdb/cdb-dev Created ORDS Service for cdb-dev +oracle-database-operator-system 51m Normal Created pdb/pdb1-clone PDB 'pdbnewclone' cloned successfully +oracle-database-operator-system 49m Normal Modified pdb/pdb1-clone PDB 'pdbnewclone' modified successfully +oracle-database-operator-system 47m Normal Deleted pdb/pdb1-clone PDB 'pdbnewclone' dropped successfully +oracle-database-operator-system 53m Normal Created pdb/pdb1 PDB 'pdbnew' created successfully +oracle-database-operator-system 44m Normal Modified pdb/pdb1 PDB 'pdbnew' modified successfully +oracle-database-operator-system 42m Normal Unplugged pdb/pdb1 PDB 'pdbnew' unplugged successfully +oracle-database-operator-system 39m Normal Created pdb/pdb1 PDB 'pdbnew' plugged successfully +``` + +## CDB Validation and Errors + +Validation is done at the time of CDB resource creation as shown below: +```sh +$ kubectl apply -f cdb1.yaml +The PDB "cdb-dev" is invalid: +* spec.dbServer: Required value: Please specify Database Server Name or IP Address +* spec.dbPort: Required value: Please specify DB Server Port +* spec.ordsImage: Required value: Please specify name of ORDS Image to be used +``` + +Apart from events, listing of CDBs will also show the possible reasons why a particular CDB CR could not be created as shown below: +```sh + $ kubectl get cdbs -A + + NAMESPACE NAME CDB NAME DB SERVER DB PORT SCAN NAME STATUS MESSAGE + oracle-database-operator-system cdb-dev devdb 172.17.0.4 1521 devdb Failed Secret not found:cdb1-secret +``` + +## PDB Validation and Errors + +Validation is done at the time of PDB resource creation as shown below: +```sh +$ kubectl apply -f pdb1.yaml +The PDB "pdb1" is invalid: +* spec.cdbResName: Required value: Please specify the name of the CDB Kubernetes resource to use for PDB operations +* spec.pdbName: Required value: Please specify name of the PDB to be created +* spec.adminPwd: Required value: Please specify PDB System Administrator Password +* spec.fileNameConversions: Required value: Please specify a value for fileNameConversions. Values can be a filename convert pattern or NONE +``` + +Similarly, for PDBs, listing of PDBs will also show the possible reasons why a particular PDB CR could not be created as shown below: +```sh +$ kubectl get pdbs -A +NAMESPACE NAME CONNECT STRING CDB NAME PDB NAME PDB SIZE STATUS MESSAGE +oracle-database-operator-system pdb1 democdb demotest1 Failed Secret not found:pdb12-secret +oracle-database-operator-system pdb2 democdb demotest2 Failed ORA-65016: FILE_NAME_CONVERT must be specified... +``` From e3b0b4f5ff5bf10a79c06e889932f217dd767096 Mon Sep 17 00:00:00 2001 From: Saurabh Ahuja Date: Tue, 28 Jun 2022 07:49:53 +0000 Subject: [PATCH 460/628] Fix issue 1 --- config/samples/onpremdb/cdb.yaml | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/config/samples/onpremdb/cdb.yaml b/config/samples/onpremdb/cdb.yaml index 3a5452b5..7b566cd7 100644 --- a/config/samples/onpremdb/cdb.yaml +++ b/config/samples/onpremdb/cdb.yaml @@ -12,7 +12,11 @@ spec: scanName: "devdb" dbServer: "172.17.0.4" dbPort: 1521 - replicas: 2 + replicas: 1 + ordsImage: "" + ordsImagePullPolicy: "Always" + # Uncomment Below Secret Format for accessing ords image from private docker registry + # ordsImagePullSecret: "" serviceName: "devdb.example.com" sysAdminPwd: secret: From d310a67eb379be1ca2b4a40fd1cb64d3036edeaf Mon Sep 17 00:00:00 2001 From: racpack Date: Tue, 28 Jun 2022 09:20:28 +0000 Subject: [PATCH 461/628] Update example_setup_using_oci_oke_cluster.md --- .../example_setup_using_oci_oke_cluster.md | 18 ------------------ 1 file changed, 18 deletions(-) diff --git a/docs/onpremdb/provisioning/example_setup_using_oci_oke_cluster.md b/docs/onpremdb/provisioning/example_setup_using_oci_oke_cluster.md index ddf275a1..f3ed2489 100644 --- a/docs/onpremdb/provisioning/example_setup_using_oci_oke_cluster.md +++ b/docs/onpremdb/provisioning/example_setup_using_oci_oke_cluster.md @@ -3,14 +3,6 @@ In this example, the target CDB (for which the PDB life cycle management is needed) is running in a Cloud environment (OCI's [Oracle Exadata Database Service](https://docs.oracle.com/en-us/iaas/exadatacloud/index.html)) and to manage its PDBs, the Oracle DB Operator is running on a Kubernetes Cluster running in cloud (OCI's [Container Engine for Kubernetes or OKE](https://docs.oracle.com/en-us/iaas/Content/ContEng/Concepts/contengoverview.htm#Overview_of_Container_Engine_for_Kubernetes)). -## Environment details - -| Component | Region Name |VCN (Virtual Cloud Network) Name| -| ------------- | ------------- |--------------------------------| -| OKE Cluster | Phoenix Region|exaphxvcn| -| ExaCS Cluster | Phoenix Region|exaphxvcn| - - ## High Level plans for this setup Below are the main steps that will be involved in this setup: @@ -22,16 +14,6 @@ Below are the main steps that will be involved in this setup: - Manager pdb life cycle management. -## Reference security lists for Ingress/Egress rules for VPN "exaphxvcn" - -Refer to the below screen shots to create security lists for Ingress/Egress rules as follows for client & backup network in VPN exaphxvcn to allow k8s Worker node to talk to ExaCS db hosts. - -|Network Type|Reference Screenshots| -|------------|---------------------| -|client network| ![](../images/K8S_SECURE1.png),![](../images/K8S_SECURE2.png)| -|backup network| ![](../images/K8S_SECURE3.png),![](../images/K8S_SECURE4.png)| - - ## OKE Cluster Check the [Oracle Documentation](https://docs.oracle.com/en-us/iaas/Content/ContEng/Concepts/contengnetworkconfigexample.htm#example-privatek8sapi-privateworkers-publiclb) for the OKE rules settings. From b837db26c7c19ffc65b9337499a69f8b9924c28c Mon Sep 17 00:00:00 2001 From: Saurabh Ahuja Date: Tue, 28 Jun 2022 09:54:32 +0000 Subject: [PATCH 462/628] Add reference to standard k8s private docker registry documentation --- docs/onpremdb/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/onpremdb/README.md b/docs/onpremdb/README.md index 2f728455..a061b92b 100644 --- a/docs/onpremdb/README.md +++ b/docs/onpremdb/README.md @@ -169,7 +169,7 @@ The Oracle Database Operator On-Prem Controller creates the CDB kind as a custom To create a CDB CRD, a sample .yaml file is available here: [config/samples/onpremdb/cdb.yaml](../../config/samples/onpremdb/cdb.yaml) -**Note:** The password and username fields in this `cdb.yaml` yaml are Kubernetes Secrets created earlier. Please see the section [Kubernetes Secrets](ORACLE_ONPREMDB_CONTROLLER_README.md#kubernetes-secrets) for more information. +**Note:** The password and username fields in this `cdb.yaml` yaml are Kubernetes Secrets created earlier. Please see the section [Kubernetes Secrets](ORACLE_ONPREMDB_CONTROLLER_README.md#kubernetes-secrets) for more information. Please see [Kubernetes Private Registry Documenation]( https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/) for creating secrets for pulling images from docker private registry. 1. [Use Case: Create a CDB CRD Resource](./provisioning/cdb_crd_resource.md) 2. [Use Case: Add another replica to an existing CDB CRD Resource](./provisioning/add_replica.md) From 223d2c49b4a71f9bd3cc6760a50c56b71928ca0d Mon Sep 17 00:00:00 2001 From: Jyoti Verma Date: Tue, 28 Jun 2022 15:13:46 +0000 Subject: [PATCH 463/628] Fix issue 3 --- .../onpremdb/{pdb.yaml => pdb_create.yaml} | 0 config/samples/onpremdb/pdb_map.yaml | 16 ---------------- docs/onpremdb/README.md | 10 +++++----- docs/onpremdb/provisioning/clone_pdb.md | 2 ++ docs/onpremdb/provisioning/create_pdb.log | 4 ++-- docs/onpremdb/provisioning/create_pdb.md | 2 +- docs/onpremdb/provisioning/delete_pdb.md | 2 ++ docs/onpremdb/provisioning/modify_pdb.md | 4 +++- docs/onpremdb/provisioning/plug_pdb.md | 2 ++ docs/onpremdb/provisioning/unplug_pdb.md | 2 ++ 10 files changed, 19 insertions(+), 25 deletions(-) rename config/samples/onpremdb/{pdb.yaml => pdb_create.yaml} (100%) delete mode 100644 config/samples/onpremdb/pdb_map.yaml diff --git a/config/samples/onpremdb/pdb.yaml b/config/samples/onpremdb/pdb_create.yaml similarity index 100% rename from config/samples/onpremdb/pdb.yaml rename to config/samples/onpremdb/pdb_create.yaml diff --git a/config/samples/onpremdb/pdb_map.yaml b/config/samples/onpremdb/pdb_map.yaml deleted file mode 100644 index 1501d690..00000000 --- a/config/samples/onpremdb/pdb_map.yaml +++ /dev/null @@ -1,16 +0,0 @@ -# -# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. -# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. -# -apiVersion: database.oracle.com/v1alpha1 -kind: PDB -metadata: - name: pdb3 - namespace: oracle-database-operator-system - labels: - cdb: cdb-dev -spec: - cdbResName: "cdb-dev" - cdbName: "democdb" - pdbName: "demotest" - action: "Map" \ No newline at end of file diff --git a/docs/onpremdb/README.md b/docs/onpremdb/README.md index a061b92b..3c77c1b4 100644 --- a/docs/onpremdb/README.md +++ b/docs/onpremdb/README.md @@ -12,7 +12,7 @@ NOTE: The target CDB (for which the PDB life cycle management is needed) **can a To deploy OraOperator, use this [Oracle Database Operator for Kubernetes](https://github.com/oracle/oracle-database-operator/blob/main/README.md) step-by-step procedure. -After the Oracle Database Operator is deployed, you can see the DB Operator Pods running in the Kubernetes Cluster. As part of the OraOperator deployment, the On-Prem Database Controller is deployed as a CRD (Custom Resource Definition). The following output is an example of such a deployment: +After the Oracle Database Operator is deployed, you can see the DB Operator Pods running in the Kubernetes Cluster. As part of the OraOperator deployment, the On-Prem Database Controller is deployed and we can see the CRDs (Custom Resource Definition) for CDB and PDB in the list of CRDs. The following output is an example of such a deployment: ``` [root@test-server oracle-database-operator]# kubectl get ns NAME STATUS AGE @@ -48,7 +48,7 @@ autonomouscontainerdatabases.database.oracle.com 2022-06-22T01:21:36Z autonomousdatabasebackups.database.oracle.com 2022-06-22T01:21:36Z autonomousdatabaserestores.database.oracle.com 2022-06-22T01:21:37Z autonomousdatabases.database.oracle.com 2022-06-22T01:21:37Z -cdbs.database.oracle.com 2022-06-22T01:21:37Z <<<< CRD for CDB +cdbs.database.oracle.com 2022-06-22T01:21:37Z <<<< certificaterequests.cert-manager.io 2022-06-21T17:03:46Z certificates.cert-manager.io 2022-06-21T17:03:47Z challenges.acme.cert-manager.io 2022-06-21T17:03:47Z @@ -57,7 +57,7 @@ dbcssystems.database.oracle.com 2022-06-22T01:21:38Z issuers.cert-manager.io 2022-06-21T17:03:49Z oraclerestdataservices.database.oracle.com 2022-06-22T01:21:38Z orders.acme.cert-manager.io 2022-06-21T17:03:49Z -pdbs.database.oracle.com 2022-06-22T01:21:39Z <<<< CRD for PDB +pdbs.database.oracle.com 2022-06-22T01:21:39Z <<<< shardingdatabases.database.oracle.com 2022-06-22T01:21:39Z singleinstancedatabases.database.oracle.com 2022-06-22T01:21:40Z ``` @@ -179,11 +179,11 @@ To create a CDB CRD, a sample .yaml file is available here: [config/samples/onpr The Oracle Database Operator On-Prem Controller creates the PDB kind as a custom resource that models a PDB as a native Kubernetes object. There is a one-to-one mapping between the actual PDB and the Kubernetes PDB Custom Resource. You cannot have more than one Kubernetes resource for a target PDB. This PDB resource can be used to perform PDB-LM operations by specifying the action attribute in the PDB Specs. Each PDB resource follows the PDB CRD as defined here: [config/crd/bases/database.oracle.com_pdbs.yaml](../../../config/crd/bases/database.oracle.com_pdbs.yaml) -To create a PDB CRD Resource, a sample .yaml file is available here: [config/samples/onpremdb/pdb.yaml](../../../config/samples/onpremdb/pdb.yaml) +To create a PDB CRD Resource, a sample .yaml file is available here: [config/samples/onpremdb/pdb_create.yaml](../../config/samples/onpremdb/pdb_create.yaml) # Use Cases for PDB Lifecycle Management Operations using Oracle DB Operator On-Prem Controller -Using Oracle DB Operator On-Prem Controller, you can perform the following PDB-LM operations: CREATE, CLONE, MODIFY, DELETE, STATUS, PLUG, UNPLUG +Using Oracle DB Operator On-Prem Controller, you can perform the following PDB-LM operations: CREATE, CLONE, MODIFY, DELETE, UNPLUG, PLUG. 1. [Create PDB](./provisioning/create_pdb.md) 2. [Clone PDB](./provisioning/clone_pdb.md) diff --git a/docs/onpremdb/provisioning/clone_pdb.md b/docs/onpremdb/provisioning/clone_pdb.md index 225c35ea..b731d301 100644 --- a/docs/onpremdb/provisioning/clone_pdb.md +++ b/docs/onpremdb/provisioning/clone_pdb.md @@ -2,6 +2,8 @@ In this use case, a PDB is cloned using Oracle DB Operator On-Prem controller. +To clone a PDB CRD Resource, a sample .yaml file is available here: [config/samples/onpremdb/pdb_clone.yaml](../../../config/samples/onpremdb/pdb_clone.yaml) + **NOTE:** It is assumed that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-manage-pdb-life-cycle-using-oracle-db-operator-on-prem-database-controller) steps. This example uses `clone_pdb.yaml` to clone a PDB using Oracle DB Operator On-Prem Controller with: diff --git a/docs/onpremdb/provisioning/create_pdb.log b/docs/onpremdb/provisioning/create_pdb.log index 7ae2d6f0..ebba9e66 100644 --- a/docs/onpremdb/provisioning/create_pdb.log +++ b/docs/onpremdb/provisioning/create_pdb.log @@ -42,7 +42,7 @@ data: -- This is the .yaml file used to create a PDB CRD resource in this use case: -% cat pdb.yaml +% cat pdb_create.yaml # # Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. @@ -79,7 +79,7 @@ spec: % kubectl apply -f pdb_secret.yaml secret/pdb1-secret created -% kubectl apply -f pdb.yaml +% kubectl apply -f pdb_create.yaml pdb.database.oracle.com/pdb1 created diff --git a/docs/onpremdb/provisioning/create_pdb.md b/docs/onpremdb/provisioning/create_pdb.md index d6296baf..4f6e57aa 100644 --- a/docs/onpremdb/provisioning/create_pdb.md +++ b/docs/onpremdb/provisioning/create_pdb.md @@ -2,7 +2,7 @@ The Oracle Database Operator On-Prem Controller creates the PDB kind as a custom resource that models a PDB as a native Kubernetes object. There is a one-to-one mapping between the actual PDB and the Kubernetes PDB Custom Resource. Each PDB resource follows the PDB CRD as defined here: [config/crd/bases/database.oracle.com_pdbs.yaml](../../../config/crd/bases/database.oracle.com_pdbs.yaml) -To create a PDB CRD Resource, a sample .yaml file is available here: [config/samples/onpremdb/pdb.yaml](../../../config/samples/onpremdb/pdb.yaml) +To create a PDB CRD Resource, a sample .yaml file is available here: [config/samples/onpremdb/pdb_create.yaml](../../../config/samples/onpremdb/pdb_create.yaml) **NOTE:** It is assumed that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-manage-pdb-life-cycle-using-oracle-db-operator-on-prem-database-controller) steps. diff --git a/docs/onpremdb/provisioning/delete_pdb.md b/docs/onpremdb/provisioning/delete_pdb.md index 360793f4..e36bb473 100644 --- a/docs/onpremdb/provisioning/delete_pdb.md +++ b/docs/onpremdb/provisioning/delete_pdb.md @@ -2,6 +2,8 @@ In this use case, a PDB is deleted using Oracle DB Operator On-Prem controller. +To delete a PDB CRD Resource, a sample .yaml file is available here: [config/samples/onpremdb/pdb_delete.yaml](../../../config/samples/onpremdb/pdb_delete.yaml) + **NOTE:** It is assumed that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-manage-pdb-life-cycle-using-oracle-db-operator-on-prem-database-controller) steps. This example uses `delete_pdb.yaml` to delete a PDB using Oracle DB Operator On-Prem Controller with: diff --git a/docs/onpremdb/provisioning/modify_pdb.md b/docs/onpremdb/provisioning/modify_pdb.md index 7e22d420..a0432efb 100644 --- a/docs/onpremdb/provisioning/modify_pdb.md +++ b/docs/onpremdb/provisioning/modify_pdb.md @@ -2,6 +2,8 @@ In this use case, the state of an existing PDB is modified using Oracle DB Operator On-Prem controller. +To modify a PDB CRD Resource, a sample .yaml file is available here: [config/samples/onpremdb/pdb_modify.yaml](../../../config/samples/onpremdb/pdb_modify.yaml) + **NOTE:** It is assumed that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-manage-pdb-life-cycle-using-oracle-db-operator-on-prem-database-controller) steps. Subcase 1: This example uses `modify_pdb_close.yaml` to close a PDB using Oracle DB Operator On-Prem Controller with: @@ -44,7 +46,7 @@ Subcase 2: This example uses `modify_pdb_open.yaml` to open a PDB using Oracle D - Option to close the state (i.e. modify) as `READ WRITE` -**NOTE:** For the details of the parameters to be used in the .yaml file, please refer [config/crd/bases/database.oracle.com_pdbs.yaml](../../config/crd/bases/database.oracle.com_pdbs.yaml) +**NOTE:** For the details of the parameters to be used in the .yaml file, please refer [config/crd/bases/database.oracle.com_pdbs.yaml](../../../config/crd/bases/database.oracle.com_pdbs.yaml) Use the file: [modify_pdb_open.yaml](./modify_pdb_open.yaml) for this use case as below: diff --git a/docs/onpremdb/provisioning/plug_pdb.md b/docs/onpremdb/provisioning/plug_pdb.md index 059cc91d..02645f52 100644 --- a/docs/onpremdb/provisioning/plug_pdb.md +++ b/docs/onpremdb/provisioning/plug_pdb.md @@ -2,6 +2,8 @@ In this use case, a PDB is plugged in using Oracle DB Operator On-Prem controller using an existing .xml file which was generated when the PDB was unplugged from this target CDB or another CDB. +To plug in a PDB CRD Resource, a sample .yaml file is available here: [config/samples/onpremdb/pdb_plug.yaml](../../../config/samples/onpremdb/pdb_plug.yaml) + **NOTE:** It is assumed that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-manage-pdb-life-cycle-using-oracle-db-operator-on-prem-database-controller) steps. This example uses `plug_pdb.yaml` to plug in a PDB to a target CDB using Oracle DB Operator On-Prem Controller with: diff --git a/docs/onpremdb/provisioning/unplug_pdb.md b/docs/onpremdb/provisioning/unplug_pdb.md index eb317ef7..fb98fc8b 100644 --- a/docs/onpremdb/provisioning/unplug_pdb.md +++ b/docs/onpremdb/provisioning/unplug_pdb.md @@ -2,6 +2,8 @@ In this use case, a PDB is unplugged using Oracle DB Operator On-Prem controller. +To unplug a PDB CRD Resource, a sample .yaml file is available here: [config/samples/onpremdb/pdb_unplug.yaml](../../../config/samples/onpremdb/pdb_unplug.yaml) + **NOTE:** It is assumed that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-manage-pdb-life-cycle-using-oracle-db-operator-on-prem-database-controller) steps. This example uses `unplug_pdb.yaml` to unplug a PDB from a target CDB using Oracle DB Operator On-Prem Controller with: From be3e3d4d22dbf999f77f51b4214e6b85c0681a8c Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Tue, 28 Jun 2022 21:26:53 +0530 Subject: [PATCH 464/628] Update deployment yaml --- .gitlab-ci.yml | 28 ----- oracle-database-operator.yaml | 207 ++++++++++++++++++++-------------- 2 files changed, 124 insertions(+), 111 deletions(-) delete mode 100644 .gitlab-ci.yml diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml deleted file mode 100644 index 1c5aaf93..00000000 --- a/.gitlab-ci.yml +++ /dev/null @@ -1,28 +0,0 @@ -build-operator: - stage: build - variables: - IMAGE: "$DOCKER_REPO:$CI_COMMIT_BRANCH" - OP_YAML: oracle-database-operator.yaml - script: - - make docker-build IMG="$IMAGE" - - docker push "$IMAGE" - - newimage=$(docker inspect $IMAGE | python -c 'import json,sys; print json.load(sys.stdin)[0]["RepoDigests"][0]') - - echo $newimage - - docker rmi "$IMAGE" && docker system prune -f - - make operator-yaml IMG=$newimage - - if [ "$CI_COMMIT_BRANCH" != "master" ]; then sed -i "s/\(replicas.\) 3/\1 1/g" ./$OP_YAML; fi - - curl -s -n $ARTIFACTORY_REPO/$CI_COMMIT_BRANCH/$OP_YAML -T ./$OP_YAML - only: - variables: - - $CI_COMMIT_MESSAGE =~ /\#run-pipeline/ - - $CI_COMMIT_BRANCH =~ /master/ - - $CI_MERGE_REQUEST_ID != "" - except: - variables: - - $CI_COMMIT_MESSAGE =~ /\#skip-pipeline/ - -cleanup: - stage: .post - script: - - echo "Clean up downloaded binaries" - - rm -rf bin/ diff --git a/oracle-database-operator.yaml b/oracle-database-operator.yaml index c6e88988..5e516801 100644 --- a/oracle-database-operator.yaml +++ b/oracle-database-operator.yaml @@ -192,9 +192,6 @@ spec: status: description: AutonomousDatabaseBackupStatus defines the observed state of AutonomousDatabaseBackup properties: - autonomousDatabaseBackupOCID: - description: 'INSERT ADDITIONAL STATUS FIELD - define observed state of cluster Important: Run "make" to regenerate code after modifying this file' - type: string autonomousDatabaseOCID: type: string compartmentOCID: @@ -203,11 +200,10 @@ spec: type: string dbName: type: string - displayName: - type: string isAutomatic: type: boolean lifecycleState: + description: 'AutonomousDatabaseBackupLifecycleStateEnum Enum with underlying type: string' type: string timeEnded: type: string @@ -221,7 +217,6 @@ spec: - compartmentOCID - dbDisplayName - dbName - - displayName - isAutomatic - lifecycleState - type @@ -328,14 +323,13 @@ spec: status: description: AutonomousDatabaseRestoreStatus defines the observed state of AutonomousDatabaseRestore properties: - autonomousDatabaseOCID: - type: string dbName: type: string displayName: description: 'INSERT ADDITIONAL STATUS FIELD - define observed state of cluster Important: Run "make" to regenerate code after modifying this file' type: string status: + description: 'WorkRequestStatusEnum Enum with underlying type: string' type: string timeAccepted: type: string @@ -343,11 +337,13 @@ spec: type: string timeStarted: type: string + workRequestOCID: + type: string required: - - autonomousDatabaseOCID - dbName - displayName - status + - workRequestOCID type: object type: object served: true @@ -384,6 +380,9 @@ spec: - jsonPath: .spec.details.displayName name: Display Name type: string + - jsonPath: .spec.details.dbName + name: Db Name + type: string - jsonPath: .status.lifecycleState name: State type: string @@ -1080,10 +1079,13 @@ spec: name: Database type: string - jsonPath: .status.databaseApiUrl - name: 'Database Api & Apex Url ' + name: Database API URL type: string - jsonPath: .status.databaseActionsUrl - name: Database Actions Url + name: Database Actions URL + type: string + - jsonPath: .status.apexUrl + name: Apex URL type: string name: v1alpha1 schema: @@ -1107,12 +1109,12 @@ spec: keepSecret: type: boolean secretKey: + default: oracle_pwd type: string secretName: type: string required: - - keepSecret - - secretKey + - secretName type: object apexPassword: description: OracleRestDataServicePassword defines the secret containing Password mapped to secretKey @@ -1120,12 +1122,12 @@ spec: keepSecret: type: boolean secretKey: + default: oracle_pwd type: string secretName: type: string required: - - keepSecret - - secretKey + - secretName type: object databaseRef: type: string @@ -1155,12 +1157,12 @@ spec: keepSecret: type: boolean secretKey: + default: oracle_pwd type: string secretName: type: string required: - - keepSecret - - secretKey + - secretName type: object ordsUser: type: string @@ -1176,6 +1178,8 @@ spec: type: string storageClass: type: string + volumeName: + type: string type: object replicas: minimum: 1 @@ -1186,20 +1190,23 @@ spec: properties: enable: type: boolean - pdb: + pdbName: type: string - schema: + schemaName: type: string urlMapping: type: string required: - enable - - pdb - - schema + - schemaName type: object type: array serviceAccountName: type: string + serviceAnnotations: + additionalProperties: + type: string + type: object required: - adminPassword - databaseRef @@ -1210,7 +1217,7 @@ spec: properties: apexConfigured: type: boolean - clusterDbApiUrl: + apexUrl: type: string commonUsersCreated: type: boolean @@ -1236,8 +1243,6 @@ spec: type: string ordsInstalled: type: boolean - ordsSetupCompleted: - type: boolean replicas: type: integer serviceIP: @@ -1973,11 +1978,11 @@ spec: keepSecret: type: boolean secretKey: + default: oracle_pwd type: string secretName: type: string required: - - secretKey - secretName type: object archiveLog: @@ -1999,6 +2004,8 @@ spec: image: description: SingleInstanceDatabaseImage defines the Image source and pullSecrets for POD properties: + prebuiltDB: + type: boolean pullFrom: type: string pullSecrets: @@ -2020,8 +2027,6 @@ spec: sgaTarget: type: integer type: object - installApex: - type: boolean loadBalancer: type: boolean nodeSelector: @@ -2034,14 +2039,18 @@ spec: description: SingleInstanceDatabasePersistence defines the storage size and class for PVC properties: accessMode: + enum: + - ReadWriteOnce + - ReadWriteMany type: string size: type: string storageClass: type: string - required: - - size - - storageClass + volumeClaimAnnotation: + type: string + volumeName: + type: string type: object readinessCheckPeriod: type: integer @@ -2049,8 +2058,13 @@ spec: type: integer serviceAccountName: type: string + serviceAnnotations: + additionalProperties: + type: string + type: object sid: - description: SID can only have a-z , A-Z, 0-9 . It cant have any special characters + description: SID must be alphanumeric (no special characters, only a-z, A-Z, 0-9), and no longer than 12 characters. + maxLength: 12 pattern: ^[a-zA-Z0-9]+$ type: string required: @@ -2118,8 +2132,10 @@ spec: connectString: type: string datafilesCreated: + default: "false" type: string datafilesPatched: + default: "false" type: string edition: type: string @@ -2159,15 +2175,21 @@ spec: description: SingleInstanceDatabasePersistence defines the storage size and class for PVC properties: accessMode: + enum: + - ReadWriteOnce + - ReadWriteMany type: string size: type: string storageClass: type: string - required: - - size - - storageClass + volumeClaimAnnotation: + type: string + volumeName: + type: string type: object + prebuiltDB: + type: boolean releaseUpdate: type: string replicas: @@ -2184,7 +2206,6 @@ spec: type: string required: - persistence - - replicas type: object type: object served: true @@ -2746,53 +2767,6 @@ spec: selector: control-plane: controller-manager --- -apiVersion: apps/v1 -kind: Deployment -metadata: - labels: - control-plane: controller-manager - name: oracle-database-operator-controller-manager - namespace: oracle-database-operator-system -spec: - replicas: 3 - selector: - matchLabels: - control-plane: controller-manager - template: - metadata: - labels: - control-plane: controller-manager - spec: - containers: - - args: - - --enable-leader-election - command: - - /manager - image: phx.ocir.io/intsanjaysingh/db-operator/sharding/dbcs-operator:0.2.0 - imagePullPolicy: Always - name: manager - ports: - - containerPort: 9443 - name: webhook-server - protocol: TCP - resources: - limits: - cpu: 400m - memory: 400Mi - requests: - cpu: 400m - memory: 400Mi - volumeMounts: - - mountPath: /tmp/k8s-webhook-server/serving-certs - name: cert - readOnly: true - terminationGracePeriodSeconds: 10 - volumes: - - name: cert - secret: - defaultMode: 420 - secretName: webhook-server-cert ---- apiVersion: cert-manager.io/v1 kind: Certificate metadata: @@ -2954,6 +2928,26 @@ metadata: cert-manager.io/inject-ca-from: oracle-database-operator-system/oracle-database-operator-serving-cert name: oracle-database-operator-validating-webhook-configuration webhooks: +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /validate-database-oracle-com-v1alpha1-autonomouscontainerdatabase + failurePolicy: Fail + name: vautonomouscontainerdatabase.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + resources: + - autonomouscontainerdatabases + sideEffects: None - admissionReviewVersions: - v1 clientConfig: @@ -2971,7 +2965,6 @@ webhooks: operations: - CREATE - UPDATE - - DELETE resources: - autonomousdatabases sideEffects: None @@ -3100,3 +3093,51 @@ webhooks: resources: - singleinstancedatabases sideEffects: None +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + labels: + control-plane: controller-manager + name: oracle-database-operator-controller-manager + namespace: oracle-database-operator-system +spec: + replicas: 3 + selector: + matchLabels: + control-plane: controller-manager + template: + metadata: + labels: + control-plane: controller-manager + spec: + containers: + - args: + - --enable-leader-election + command: + - /manager + image: container-registry.oracle.com/database/operator:0.2.0 + imagePullPolicy: Always + name: manager + ports: + - containerPort: 9443 + name: webhook-server + protocol: TCP + resources: + limits: + cpu: 400m + memory: 400Mi + requests: + cpu: 400m + memory: 400Mi + volumeMounts: + - mountPath: /tmp/k8s-webhook-server/serving-certs + name: cert + readOnly: true + terminationGracePeriodSeconds: 10 + volumes: + - name: cert + secret: + defaultMode: 420 + secretName: webhook-server-cert +--- From b9c6544f1208a951f329ec51d4fed4bf023609de Mon Sep 17 00:00:00 2001 From: Ting-Lan Wang <61528376+harayuanwang@users.noreply.github.com> Date: Tue, 28 Jun 2022 09:26:31 -0700 Subject: [PATCH 465/628] Update README.md Added command for deleting on-premdb --- README.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index d8d63691..b5ec6729 100644 --- a/README.md +++ b/README.md @@ -108,14 +108,16 @@ YAML file templates are available under [`/config/samples`](./config/samples/). To delete all the CRD instances deployed to cluster by the operator, run the following commands, where is the namespace of the cluster object: ```sh - kubectl delete oraclerestdataservices.database.oracle.com --all -n + kubectl delete oraclerestdataservice.database.oracle.com --all -n kubectl delete singleinstancedatabase.database.oracle.com --all -n kubectl delete shardingdatabase.database.oracle.com --all -n + kubectl delete dbcssystem.database.oracle.com --all -n kubectl delete autonomousdatabase.database.oracle.com --all -n kubectl delete autonomousdatabasebackup.database.oracle.com --all -n kubectl delete autonomousdatabaserestore.database.oracle.com --all -n kubectl delete autonomouscontainerdatabase.database.oracle.com --all -n - kubectl delete dbcssystem.database.oracle.com --all -n + kubectl delete cdb.database.oracle.com --all -n + kubectl delete pdb.database.oracle.com --all -n ``` After all CRD instances are deleted, it is safe to remove the CRDs, APISerivces and operator deployment. From 78d9609db30ba30d69d4ff5f376fb4e7ee0f7ad0 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Tue, 28 Jun 2022 16:32:32 +0000 Subject: [PATCH 466/628] Updated README.md --- README.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index d8d63691..b5ec6729 100644 --- a/README.md +++ b/README.md @@ -108,14 +108,16 @@ YAML file templates are available under [`/config/samples`](./config/samples/). To delete all the CRD instances deployed to cluster by the operator, run the following commands, where is the namespace of the cluster object: ```sh - kubectl delete oraclerestdataservices.database.oracle.com --all -n + kubectl delete oraclerestdataservice.database.oracle.com --all -n kubectl delete singleinstancedatabase.database.oracle.com --all -n kubectl delete shardingdatabase.database.oracle.com --all -n + kubectl delete dbcssystem.database.oracle.com --all -n kubectl delete autonomousdatabase.database.oracle.com --all -n kubectl delete autonomousdatabasebackup.database.oracle.com --all -n kubectl delete autonomousdatabaserestore.database.oracle.com --all -n kubectl delete autonomouscontainerdatabase.database.oracle.com --all -n - kubectl delete dbcssystem.database.oracle.com --all -n + kubectl delete cdb.database.oracle.com --all -n + kubectl delete pdb.database.oracle.com --all -n ``` After all CRD instances are deleted, it is safe to remove the CRDs, APISerivces and operator deployment. From 1f20f432a94597b3b0a7b59f00ba302001ca40c6 Mon Sep 17 00:00:00 2001 From: Mohammed Yunus Date: Wed, 29 Jun 2022 08:42:36 +0000 Subject: [PATCH 467/628] Update .gitlab-ci.yml file --- .gitlab-ci.yml | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) create mode 100644 .gitlab-ci.yml diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml new file mode 100644 index 00000000..1c5aaf93 --- /dev/null +++ b/.gitlab-ci.yml @@ -0,0 +1,28 @@ +build-operator: + stage: build + variables: + IMAGE: "$DOCKER_REPO:$CI_COMMIT_BRANCH" + OP_YAML: oracle-database-operator.yaml + script: + - make docker-build IMG="$IMAGE" + - docker push "$IMAGE" + - newimage=$(docker inspect $IMAGE | python -c 'import json,sys; print json.load(sys.stdin)[0]["RepoDigests"][0]') + - echo $newimage + - docker rmi "$IMAGE" && docker system prune -f + - make operator-yaml IMG=$newimage + - if [ "$CI_COMMIT_BRANCH" != "master" ]; then sed -i "s/\(replicas.\) 3/\1 1/g" ./$OP_YAML; fi + - curl -s -n $ARTIFACTORY_REPO/$CI_COMMIT_BRANCH/$OP_YAML -T ./$OP_YAML + only: + variables: + - $CI_COMMIT_MESSAGE =~ /\#run-pipeline/ + - $CI_COMMIT_BRANCH =~ /master/ + - $CI_MERGE_REQUEST_ID != "" + except: + variables: + - $CI_COMMIT_MESSAGE =~ /\#skip-pipeline/ + +cleanup: + stage: .post + script: + - echo "Clean up downloaded binaries" + - rm -rf bin/ From 7c9ff46b1d4d3409594bcf69d7fb5e20e7edbe58 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Thu, 30 Jun 2022 20:09:57 +0530 Subject: [PATCH 468/628] Update the operator yaml apply command Signed-off-by: Yunus Qureshi --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index b5ec6729..18520443 100644 --- a/README.md +++ b/README.md @@ -65,7 +65,7 @@ Oracle strongly recommends that you ensure your system meets the following [Prer Run the following command ```sh - kubectl apply -f oracle-database-operator.yaml + kubectl apply -f https://raw.githubusercontent.com/oracle/oracle-database-operator/main/oracle-database-operator.yaml ``` Ensure that operator pods are up and running. Operator pod replicas are set to a default of 3 for High Availability, which can be scaled up and down. From 7d6b454c37046dff1c19f939d18925b619115984 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Thu, 30 Jun 2022 20:16:37 +0530 Subject: [PATCH 469/628] Fix typo Signed-off-by: Yunus Qureshi --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 18520443..7a6149df 100644 --- a/README.md +++ b/README.md @@ -84,7 +84,7 @@ Oracle strongly recommends that you ensure your system meets the following [Prer You should see that the operator is up and running, along with the shipped controllers. -For more details, see [Oracle Database Operator Installation Instrunctions](./docs/installation/OPERATOR_INSTALLATION_README.md). +For more details, see [Oracle Database Operator Installation Instructions](./docs/installation/OPERATOR_INSTALLATION_README.md). ## Getting Started with the Operator (Quickstart) From 50589c5dba5d25d271aed91bcb96ba20119066ba Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Fri, 1 Jul 2022 18:03:09 +0000 Subject: [PATCH 470/628] Update .gitlab-ci.yml file --- .gitlab-ci.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 1c5aaf93..9c978f45 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -20,6 +20,7 @@ build-operator: except: variables: - $CI_COMMIT_MESSAGE =~ /\#skip-pipeline/ + - $CI_COMMIT_TAG != "" cleanup: stage: .post From 22f1f1d3251159a633621ccc85cbd0bc3b730da7 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Fri, 1 Jul 2022 18:07:52 +0000 Subject: [PATCH 471/628] Revert "Update .gitlab-ci.yml file" This reverts commit 50589c5dba5d25d271aed91bcb96ba20119066ba --- .gitlab-ci.yml | 1 - 1 file changed, 1 deletion(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 9c978f45..1c5aaf93 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -20,7 +20,6 @@ build-operator: except: variables: - $CI_COMMIT_MESSAGE =~ /\#skip-pipeline/ - - $CI_COMMIT_TAG != "" cleanup: stage: .post From ee9bb97461ed7c53adbd10b286cee7a97009950a Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Fri, 1 Jul 2022 18:31:14 +0000 Subject: [PATCH 472/628] Tinglwan create tag skip ci --- .gitlab-ci.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 1c5aaf93..cbef6241 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -4,6 +4,7 @@ build-operator: IMAGE: "$DOCKER_REPO:$CI_COMMIT_BRANCH" OP_YAML: oracle-database-operator.yaml script: + - echo $CI_COMMIT_TAG - make docker-build IMG="$IMAGE" - docker push "$IMAGE" - newimage=$(docker inspect $IMAGE | python -c 'import json,sys; print json.load(sys.stdin)[0]["RepoDigests"][0]') @@ -20,6 +21,7 @@ build-operator: except: variables: - $CI_COMMIT_MESSAGE =~ /\#skip-pipeline/ + - $CI_COMMIT_TAG != null cleanup: stage: .post From b67e4347bf3e9baa16841972038fb91e8802857e Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Mon, 11 Jul 2022 14:27:25 +0530 Subject: [PATCH 473/628] Added env vars for tcps support Signed-off-by: abhisbyk --- .../v1alpha1/singleinstancedatabase_types.go | 2 + ...se.oracle.com_singleinstancedatabases.yaml | 4 ++ .../samples/sidb/singleinstancedatabase.yaml | 6 +++ .../singleinstancedatabase_controller.go | 46 +++++++++++++++++-- 4 files changed, 53 insertions(+), 5 deletions(-) diff --git a/apis/database/v1alpha1/singleinstancedatabase_types.go b/apis/database/v1alpha1/singleinstancedatabase_types.go index 11ee82db..12b8ee9d 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_types.go +++ b/apis/database/v1alpha1/singleinstancedatabase_types.go @@ -65,6 +65,8 @@ type SingleInstanceDatabaseSpec struct { FlashBack bool `json:"flashBack,omitempty"` ArchiveLog bool `json:"archiveLog,omitempty"` ForceLogging bool `json:"forceLog,omitempty"` + EnableTCPS bool `json:"enableTCPS,omitempty"` + TcpsPort int `json:"tcpsPort,omitempty"` CloneFrom string `json:"cloneFrom,omitempty"` ReadinessCheckPeriod int `json:"readinessCheckPeriod,omitempty"` diff --git a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml index a4124691..6e7c3ec4 100644 --- a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml +++ b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml @@ -87,6 +87,8 @@ spec: - enterprise - express type: string + enableTCPS: + type: boolean flashBack: type: boolean forceLog: @@ -160,6 +162,8 @@ spec: maxLength: 12 pattern: ^[a-zA-Z0-9]+$ type: string + tcpsPort: + type: integer required: - image type: object diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index f0502697..0d099255 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -44,6 +44,12 @@ spec: ## Enable/Disable ForceLogging forceLog: false + ## Enable TCPS + enableTCPS: false + + ## TCPS custom port + tcpsPort: 1522 + ## NA if cloning from a SourceDB (cloneFrom is set) ## Specify both sgaSize and pgaSize (in MB) or dont specify both ## Specify Non-Zero value to use diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 714c07c0..82351e2d 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -78,6 +78,9 @@ type SingleInstanceDatabaseReconciler struct { var requeueY ctrl.Result = ctrl.Result{Requeue: true, RequeueAfter: 15 * time.Second} var requeueN ctrl.Result = ctrl.Result{} +// Service Port Declaration +var svc_port string + const singleInstanceDatabaseFinalizer = "database.oracle.com/singleinstancedatabasefinalizer" //+kubebuilder:rbac:groups=database.oracle.com,resources=singleinstancedatabases,verbs=get;list;watch;create;update;patch;delete @@ -134,6 +137,14 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct r.Status().Update(ctx, singleInstanceDatabase) } + // Service Port Initialization + svc_port = func() string { + if singleInstanceDatabase.Spec.EnableTCPS { + return strconv.Itoa(singleInstanceDatabase.Spec.TcpsPort) + } + return "1521" + }() + // Manage SingleInstanceDatabase Deletion result, err = r.manageSingleInstanceDatabaseDeletion(req, ctx, singleInstanceDatabase) if result.Requeue { @@ -639,7 +650,16 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, }, }, - Ports: []corev1.ContainerPort{{ContainerPort: 1521}, {ContainerPort: 5500}}, + Ports: []corev1.ContainerPort{ + {ContainerPort: func() int32 { + if m.Spec.EnableTCPS { + return int32(m.Spec.TcpsPort) + } + return int32(1521) + }(), + }, + {ContainerPort: 5500}, + }, ReadinessProbe: &corev1.Probe{ ProbeHandler: corev1.ProbeHandler{ @@ -687,7 +707,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, { Name: "SVC_PORT", - Value: "1521", + Value: svc_port, }, { Name: "ORACLE_CHARACTERSET", @@ -697,6 +717,14 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns Name: "ORACLE_EDITION", Value: m.Spec.Edition, }, + { + Name: "ENABLE_TCPS", + Value: strconv.FormatBool(m.Spec.EnableTCPS), + }, + { + Name: "TCPS_PORT", + Value: strconv.Itoa(m.Spec.TcpsPort), + }, } } if m.Spec.CloneFrom == "" { @@ -708,7 +736,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, { Name: "SVC_PORT", - Value: "1521", + Value: svc_port, }, { Name: "CREATE_PDB", @@ -766,6 +794,14 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns Name: "SKIP_DATAPATCH", Value: "true", }, + { + Name: "ENABLE_TCPS", + Value: strconv.FormatBool(m.Spec.EnableTCPS), + }, + { + Name: "TCPS_PORT", + Value: strconv.Itoa(m.Spec.TcpsPort), + }, } } // For clone DB use case @@ -776,7 +812,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, { Name: "SVC_PORT", - Value: "1521", + Value: svc_port, }, { Name: "ORACLE_SID", @@ -790,7 +826,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns Name: "PRIMARY_DB_CONN_STR", Value: func() string { if dbcommons.IsSourceDatabaseOnCluster(m.Spec.CloneFrom) { - return n.Name + ":1521/" + n.Spec.Sid + return n.Name + ":" + svc_port + "/" + n.Spec.Sid } return m.Spec.CloneFrom }(), From 254487312053df46a56e7c71b8cf58e554d55773 Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Mon, 11 Jul 2022 15:05:00 +0530 Subject: [PATCH 474/628] Correcting port for the db service Signed-off-by: abhisbyk --- .../database/singleinstancedatabase_controller.go | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 82351e2d..c852e9ae 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -914,8 +914,13 @@ func (r *SingleInstanceDatabaseReconciler) instantiateSVCSpec(m *dbapi.SingleIns Spec: corev1.ServiceSpec{ Ports: []corev1.ServicePort{ { - Name: "listener", - Port: 1521, + Name: "listener", + Port: func() int32 { + if m.Spec.EnableTCPS { + return int32(m.Spec.TcpsPort) + } + return int32(1521) + }(), Protocol: corev1.ProtocolTCP, }, { From bc0c0524e4ed700f155a30af7c5a63a5d19094c6 Mon Sep 17 00:00:00 2001 From: Jan Leemans Date: Mon, 18 Jul 2022 11:01:29 +0200 Subject: [PATCH 475/628] Issue #26 Documentation NFS Volume Detailing the documentation to set up a Single Instance DB using an NFS Volume Signed-off-by: Jan Leemans --- docs/sidb/README.md | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index e55580ca..6c4b0fd6 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -307,7 +307,7 @@ nodeSelector: ``` ##### OCI NFS Volume Static Provisioning -Similar to the block volume static provisioning, you have to manually create a file system resource from the OCI console, and fetch its `OCID, Mount Target and Export Path`. Mention these values in the following YAML file to create the persistent volume: +Similar to the block volume static provisioning, you have to manually create a file system resource from the OCI console, and fetch its `OCID, Mount Target IP Address and Export Path`. Mention these values in the following YAML file to create the persistent volume: ```yaml apiVersion: v1 @@ -323,10 +323,17 @@ spec: persistentVolumeReclaimPolicy: Retain csi: driver: fss.csi.oraclecloud.com - volumeHandle: ":/" + volumeHandle: "::/" ``` -**Note:** Whenever a mount target is provisioned in OCI, its `Reported Size (GiB)` values are very large. This is visible on the mount target page when logged in to the OCI console. Some applications will fail to install if the results of a space requirements check show too much available disk space. So specify, in gibibytes (GiB), the maximum capacity reported by file systems exported through this mount target. This setting does not limit the actual amount of data you can store. +**Note:** +- Example volumeHandler in the above config file : + + `volumeHandle: "ocid1.filesystem.oc1.eu_frankfurt_1.aaaaaqe3bj...eaaa:10.0.10.156:/FileSystem-20220713-1036-02"` + +- Whenever a mount target is provisioned in OCI, its `Reported Size (GiB)` values are very large. This is visible on the mount target page when logged in to the OCI console. Some applications will fail to install if the results of a space requirements check show too much available disk space. So in the OCI Console, click the little "Pencil" icon besides the **Reported Size** parameter of the Mount Target to specify, in gigabytes (GiB), the maximum capacity reported by file systems exported through this mount target. This setting does not limit the actual amount of data you can store. + +- Make sure to open the required ports to access the NFS volume from the K8S cluster: add the required ports to the security list of the subnet where your K8S nodes are connected to; see **[here](https://docs.oracle.com/en-us/iaas/Content/File/Tasks/securitylistsfilestorage.htm)** for the details. ### Configuring a Database The `OraOperator` facilitates you to configure the database. Various database configuration options are explained in the following subsections: From 5e7f3b7493f96f8549eced57a33b1af6b275b1bc Mon Sep 17 00:00:00 2001 From: Jan Leemans Date: Mon, 18 Jul 2022 13:34:58 +0200 Subject: [PATCH 476/628] #26 type correction Correct line description in text (volumeHandle) --- docs/sidb/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 6c4b0fd6..91e52927 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -327,7 +327,7 @@ spec: ``` **Note:** -- Example volumeHandler in the above config file : +- Example volumeHandle in the above config file : `volumeHandle: "ocid1.filesystem.oc1.eu_frankfurt_1.aaaaaqe3bj...eaaa:10.0.10.156:/FileSystem-20220713-1036-02"` From 5085f826baf45b31da84b35551f1122f4c140570 Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Fri, 29 Jul 2022 10:50:33 +0530 Subject: [PATCH 477/628] Intermediate commit-1 Signed-off-by: abhisbyk --- apis/database/v1alpha1/singleinstancedatabase_types.go | 2 ++ 1 file changed, 2 insertions(+) diff --git a/apis/database/v1alpha1/singleinstancedatabase_types.go b/apis/database/v1alpha1/singleinstancedatabase_types.go index 12b8ee9d..ab20ae2a 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_types.go +++ b/apis/database/v1alpha1/singleinstancedatabase_types.go @@ -148,6 +148,8 @@ type SingleInstanceDatabaseStatus struct { PdbConnectString string `json:"pdbConnectString,omitempty"` ApexInstalled bool `json:"apexInstalled,omitempty"` PrebuiltDB bool `json:"prebuiltDB,omitempty"` + // +kubebuilder:default:=false + IsTcpsEnabled bool `json:"isTcpsEnabled"` // +patchMergeKey=type // +patchStrategy=merge From a42d805a2eb5097c69bfe475698578ca65416951 Mon Sep 17 00:00:00 2001 From: Abhishek Kumar Date: Fri, 29 Jul 2022 05:34:31 +0000 Subject: [PATCH 478/628] Showing FQDN of LB if it's ingress IP is not available --- .../database/oraclerestdataservice_controller.go | 13 +++++++++---- .../database/singleinstancedatabase_controller.go | 11 ++++++++--- 2 files changed, 17 insertions(+), 7 deletions(-) diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 9021a6ec..319b0d1e 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -855,13 +855,18 @@ func (r *OracleRestDataServiceReconciler) createSVC(ctx context.Context, req ctr m.Status.ServiceIP = "" if m.Spec.LoadBalancer { if len(svc.Status.LoadBalancer.Ingress) > 0 { - m.Status.DatabaseApiUrl = "https://" + svc.Status.LoadBalancer.Ingress[0].IP + ":" + + // 'lbAddress' will contain the Fully Qualified Hostname of the LB. If the hostname is not available it will contain the IP address of the LB + lbAddress := svc.Status.LoadBalancer.Ingress[0].Hostname + if lbAddress == "" { + lbAddress = svc.Status.LoadBalancer.Ingress[0].IP + } + m.Status.DatabaseApiUrl = "https://" + lbAddress + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/ords/" + n.Status.Pdbname + "/_/db-api/stable/" - m.Status.ServiceIP = svc.Status.LoadBalancer.Ingress[0].IP - m.Status.DatabaseActionsUrl = "https://" + svc.Status.LoadBalancer.Ingress[0].IP + ":" + + m.Status.ServiceIP = lbAddress + m.Status.DatabaseActionsUrl = "https://" + lbAddress + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/ords/sql-developer" if m.Status.ApexConfigured { - m.Status.ApxeUrl = "https://" + svc.Status.LoadBalancer.Ingress[0].IP + ":" + + m.Status.ApxeUrl = "https://" + lbAddress + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/ords/" + n.Status.Pdbname + "/apex" } } diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 714c07c0..a97e3446 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1090,9 +1090,14 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex if m.Spec.LoadBalancer { m.Status.ClusterConnectString = svc.Name + "." + svc.Namespace + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/" + strings.ToUpper(sid) if len(svc.Status.LoadBalancer.Ingress) > 0 { - m.Status.ConnectString = svc.Status.LoadBalancer.Ingress[0].IP + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/" + strings.ToUpper(sid) - m.Status.PdbConnectString = svc.Status.LoadBalancer.Ingress[0].IP + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/" + strings.ToUpper(pdbName) - m.Status.OemExpressUrl = "https://" + svc.Status.LoadBalancer.Ingress[0].IP + ":" + fmt.Sprint(svc.Spec.Ports[1].Port) + "/em" + // 'lbAddress' will contain the Fully Qualified Hostname of the LB. If the hostname is not available it will contain the IP address of the LB + lbAddress := svc.Status.LoadBalancer.Ingress[0].Hostname + if lbAddress == "" { + lbAddress = svc.Status.LoadBalancer.Ingress[0].IP + } + m.Status.ConnectString = lbAddress + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/" + strings.ToUpper(sid) + m.Status.PdbConnectString = lbAddress + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/" + strings.ToUpper(pdbName) + m.Status.OemExpressUrl = "https://" + lbAddress + ":" + fmt.Sprint(svc.Spec.Ports[1].Port) + "/em" } return requeueN, nil } From b48aadde5edb5d6ab36e5e12d4110813248fa725 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Fri, 29 Jul 2022 19:14:58 +0000 Subject: [PATCH 479/628] Tinglwan fix makefile --- Makefile | 56 +++++++++++++++++++++++++++++++------------------------- 1 file changed, 31 insertions(+), 25 deletions(-) diff --git a/Makefile b/Makefile index 78cf72ac..0d1163df 100644 --- a/Makefile +++ b/Makefile @@ -104,32 +104,38 @@ operator-yaml: manifests kustomize undeploy: ## Undeploy controller from the K8s cluster specified in ~/.kube/config. $(KUSTOMIZE) build config/default | kubectl delete -f - +##@ Build Dependencies + +## Location to install dependencies to +LOCALBIN ?= $(shell pwd)/bin +$(LOCALBIN): + mkdir -p $(LOCALBIN) + +## Tool Binaries +KUSTOMIZE ?= $(LOCALBIN)/kustomize +CONTROLLER_GEN ?= $(LOCALBIN)/controller-gen +ENVTEST ?= $(LOCALBIN)/setup-envtest + +## Tool Versions +KUSTOMIZE_VERSION ?= v3.8.7 +CONTROLLER_TOOLS_VERSION ?= v0.6.1 + +KUSTOMIZE_INSTALL_SCRIPT ?= "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh" +.PHONY: kustomize +kustomize: $(KUSTOMIZE) ## Download kustomize locally if necessary. +$(KUSTOMIZE): $(LOCALBIN) + curl -s $(KUSTOMIZE_INSTALL_SCRIPT) | bash -s -- $(subst v,,$(KUSTOMIZE_VERSION)) $(LOCALBIN) + +.PHONY: controller-gen +controller-gen: $(CONTROLLER_GEN) ## Download controller-gen locally if necessary. +$(CONTROLLER_GEN): $(LOCALBIN) + GOBIN=$(LOCALBIN) go install sigs.k8s.io/controller-tools/cmd/controller-gen@$(CONTROLLER_TOOLS_VERSION) + +.PHONY: envtest +envtest: $(ENVTEST) ## Download envtest-setup locally if necessary. +$(ENVTEST): $(LOCALBIN) + GOBIN=$(LOCALBIN) go install sigs.k8s.io/controller-runtime/tools/setup-envtest@latest -CONTROLLER_GEN = $(shell pwd)/bin/controller-gen -controller-gen: ## Download controller-gen locally if necessary. - $(call go-get-tool,$(CONTROLLER_GEN),sigs.k8s.io/controller-tools/cmd/controller-gen@v0.6.1) - -KUSTOMIZE = $(shell pwd)/bin/kustomize -kustomize: ## Download kustomize locally if necessary. - $(call go-get-tool,$(KUSTOMIZE),sigs.k8s.io/kustomize/kustomize/v3@v3.8.7) - -ENVTEST = $(shell pwd)/bin/setup-envtest -envtest: ## Download envtest-setup locally if necessary. - $(call go-get-tool,$(ENVTEST),sigs.k8s.io/controller-runtime/tools/setup-envtest@latest) - -# go-get-tool will 'go get' any package $2 and install it to $1. -PROJECT_DIR := $(shell dirname $(abspath $(lastword $(MAKEFILE_LIST)))) -define go-get-tool -@[ -f $(1) ] || { \ -set -e ;\ -TMP_DIR=$$(mktemp -d) ;\ -cd $$TMP_DIR ;\ -go mod init tmp ;\ -echo "Downloading $(2)" ;\ -GOBIN=$(PROJECT_DIR)/bin go get $(2) ;\ -rm -rf $$TMP_DIR ;\ -} -endef .PHONY: bundle bundle: manifests kustomize ## Generate bundle manifests and metadata, then validate generated files. From 4046b0bfce2b3cea70b6f60e9fd17f7fcca717b2 Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Tue, 2 Aug 2022 13:44:56 +0530 Subject: [PATCH 480/628] Added the LiveLab info, version upgrade comment Signed-off-by: abhisbyk --- README.md | 5 +++++ docs/sidb/README.md | 7 +++++++ 2 files changed, 12 insertions(+) diff --git a/README.md b/README.md index 7a6149df..2c37af2c 100644 --- a/README.md +++ b/README.md @@ -67,6 +67,11 @@ Oracle strongly recommends that you ensure your system meets the following [Prer ```sh kubectl apply -f https://raw.githubusercontent.com/oracle/oracle-database-operator/main/oracle-database-operator.yaml ``` + --- + **NOTE:** + The above command will also upgrade the existing v0.1.0 `OraOperator` installation to the latest version i.e. v0.2.0. + + --- Ensure that operator pods are up and running. Operator pod replicas are set to a default of 3 for High Availability, which can be scaled up and down. diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 91e52927..6d960194 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -32,6 +32,7 @@ Oracle Database Operator for Kubernetes (`OraOperator`) includes the Single Inst * [APEX Installation](#apex-installation) * [Delete ORDS](#delete-ords) * [Maintenance Operations](#maintenance-operations) + * [Additional Information](#additional-information) ## Prerequisites @@ -793,3 +794,9 @@ If you need to perform some maintenance operations (Database/ORDS) manually, the 3. Log In to `sqlplus` to perform manual operations by using the following command: sqlplus / as sysdba + +## Additional information +Detailed instructions for setting up Single Instance Database by OraOperator using OCI free trial account is available now in the LiveLab format. Please use the following link: + [https://oracle.github.io/cloudtestdrive/AppDev/database-operator/workshops/freetier/?lab=introduction](https://oracle.github.io/cloudtestdrive/AppDev/database-operator/workshops/freetier/?lab=introduction) + +Thanks, [Jan Leemans](https://github.com/janleemans), for this effort!! From 4028846913bb0de8659791bca4a449336783d7b4 Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Tue, 2 Aug 2022 14:24:45 +0530 Subject: [PATCH 481/628] Some additional doc fixes Signed-off-by: abhisbyk --- README.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 2c37af2c..7ab5a833 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ ## Make Oracle Database Kubernetes Native - Take 2 -As part of Oracle's resolution to make Oracle Database Kubernetes-native (that is, observable and operable by Kubernetes), Oracle released _Oracle Database Operator for Kubernetes_ (`OraOperator`). OraOperator extends the Kubernetes API with custom resources and controllers for automating Oracle Database lifecycle management. +As part of Oracle's resolution to make Oracle Database Kubernetes-native (that is, observable and operable by Kubernetes), Oracle released _Oracle Database Operator for Kubernetes_ (`OraOperator` or the operator). OraOperator extends the Kubernetes API with custom resources and controllers for automating Oracle Database lifecycle management. In this v0.2.0 release, `OraOperator` supports the following database configurations and infrastructure: @@ -32,7 +32,7 @@ The upcoming releases will support new configurations, operations and capabiliti ## Release Status -**CAUTION:** The current release of `OraOperator` (v0.2.0) is for development and test only. DO NOT USE IN PRODUCTION. +**CAUTION:** The current release of `OraOperator` (v0.2.0) is for development and testing only. DO NOT USE IN PRODUCTION. This release has been installed and tested on the following Kubernetes platforms: @@ -73,7 +73,7 @@ Oracle strongly recommends that you ensure your system meets the following [Prer --- - Ensure that operator pods are up and running. Operator pod replicas are set to a default of 3 for High Availability, which can be scaled up and down. + Ensure that the operator pods are up and running. For high availability, Operator pod replicas are set to a default of 3. You can scale this setting up or down. ```sh $ kubectl get pods -n oracle-database-operator-system @@ -106,7 +106,7 @@ YAML file templates are available under [`/config/samples`](./config/samples/). ## Uninstall the Operator - To uninstall the operator, the final step consists of deciding whether or not you want to delete the CRDs and APIServices that were introduced to the cluster by the operator. Choose one of the following options: + To uninstall the operator, the final step consists of deciding whether you want to delete the custom resource definitions (CRDs) and Kubernetes APIServices introduced into the cluster by the operator. Choose one of the following options: * ### Deleting the CRDs and APIServices @@ -125,13 +125,13 @@ YAML file templates are available under [`/config/samples`](./config/samples/). kubectl delete pdb.database.oracle.com --all -n ``` - After all CRD instances are deleted, it is safe to remove the CRDs, APISerivces and operator deployment. + After all CRD instances are deleted, it is safe to remove the CRDs, APISerivces and operator deployment. Use the following command: ```sh kubectl delete -f oracle-database-operator.yaml --ignore-not-found=true ``` - Note: If the CRD instances are not deleted, and the operator is deleted by using the preceding command, then operator deployment and instance objects (pods,services, PVCs, and so on) are deleted. However, the CRD deletion stops responding, because the CRD instances have properties that prevent its deletion and that can only be removed by the operator pod, which is deleted when the APIServices are deleted. + Note: If the CRD instances are not deleted, and the operator is deleted by using the preceding command, then operator deployment and instance objects (pods, services, PVCs, and so on) are deleted. However, if that happens, then the CRD deletion stops responding. This is because the CRD instances have properties that prevent their deletion, and that can only be removed by the operator pod, which is deleted when the APIServices are deleted. ## Docs of the supported Oracle Database configurations From 4e130468ecd9fd2a666f5fc658867d28af9eeb66 Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Sun, 7 Aug 2022 23:48:46 +0530 Subject: [PATCH 482/628] Added cert renewal logic, enable/disable TCPs, creation of clusterIP service for ORDS/APEX installation Signed-off-by: abhisbyk --- .../v1alpha1/oraclerestdataservice_webhook.go | 1 - .../v1alpha1/singleinstancedatabase_types.go | 4 +- commons/database/constants.go | 26 +- commons/database/utils.go | 16 + ...se.oracle.com_singleinstancedatabases.yaml | 8 + .../singleinstancedatabase_controller.go | 324 ++++++++++++------ 6 files changed, 269 insertions(+), 110 deletions(-) diff --git a/apis/database/v1alpha1/oraclerestdataservice_webhook.go b/apis/database/v1alpha1/oraclerestdataservice_webhook.go index 0676c7b3..9e63eee2 100644 --- a/apis/database/v1alpha1/oraclerestdataservice_webhook.go +++ b/apis/database/v1alpha1/oraclerestdataservice_webhook.go @@ -39,7 +39,6 @@ package v1alpha1 import ( - apierrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/runtime" "k8s.io/apimachinery/pkg/runtime/schema" diff --git a/apis/database/v1alpha1/singleinstancedatabase_types.go b/apis/database/v1alpha1/singleinstancedatabase_types.go index ab20ae2a..5f8efbc4 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_types.go +++ b/apis/database/v1alpha1/singleinstancedatabase_types.go @@ -149,7 +149,9 @@ type SingleInstanceDatabaseStatus struct { ApexInstalled bool `json:"apexInstalled,omitempty"` PrebuiltDB bool `json:"prebuiltDB,omitempty"` // +kubebuilder:default:=false - IsTcpsEnabled bool `json:"isTcpsEnabled"` + IsTcpsEnabled bool `json:"isTcpsEnabled"` + TcpsPort int `json:"tcpsPort,omitempty"` + CertCreationTimestamp string `json:"certCreationTimestamp,omitempty"` // +patchMergeKey=type // +patchStrategy=merge diff --git a/commons/database/constants.go b/commons/database/constants.go index 3c899e51..0e71742d 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -38,6 +38,8 @@ package commons +const DEFAULT_LISTENER_PORT int32 = 1521 + const ORACLE_UID int64 = 54321 const ORACLE_GUID int64 = 54321 @@ -318,12 +320,12 @@ const InitORDSCMD string = "if [ -f $ORDS_HOME/config/ords/defaults.xml ]; then "\numask 022" const GetSessionInfoSQL string = "select s.sid || ',' || s.serial# as Info FROM v\\$session s, v\\$process p " + - "WHERE (s.username = 'ORDS_PUBLIC_USER' or "+ - "s.username = 'APEX_PUBLIC_USER' or "+ - "s.username = 'APEX_REST_PUBLIC_USER' or "+ - "s.username = 'APEX_LISTENER' or "+ - "s.username = 'C##_DBAPI_CDB_ADMIN' or "+ - "s.username = 'C##_DBAPI_PDB_ADMIN' ) AND p.addr(+) = s.paddr;" + "WHERE (s.username = 'ORDS_PUBLIC_USER' or " + + "s.username = 'APEX_PUBLIC_USER' or " + + "s.username = 'APEX_REST_PUBLIC_USER' or " + + "s.username = 'APEX_LISTENER' or " + + "s.username = 'C##_DBAPI_CDB_ADMIN' or " + + "s.username = 'C##_DBAPI_PDB_ADMIN' ) AND p.addr(+) = s.paddr;" const KillSessionSQL string = "alter system kill session '%[1]s';" @@ -426,13 +428,12 @@ const ConfigureApexRest string = "if [ -f ${ORDS_HOME}/config/apex/apex_rest_con "echo -e \"%[1]s\n%[1]s\" | %[2]s ; else echo \"Apex Folder doesn't exist\" ; fi ;" const AlterApexUsers string = "\nALTER SESSION SET CONTAINER=%[2]s;" + - "\n ALTER USER APEX_PUBLIC_USER IDENTIFIED BY \\\"%[1]s\\\" ACCOUNT UNLOCK; "+ + "\n ALTER USER APEX_PUBLIC_USER IDENTIFIED BY \\\"%[1]s\\\" ACCOUNT UNLOCK; " + "\n ALTER USER APEX_REST_PUBLIC_USER IDENTIFIED BY \\\"%[1]s\\\" ACCOUNT UNLOCK;" + "\n ALTER USER APEX_LISTENER IDENTIFIED BY \\\"%[1]s\\\" ACCOUNT UNLOCK;" + "\nexec APEX_UTIL.set_workspace(p_workspace => 'INTERNAL');" + "\nexec APEX_UTIL.EDIT_USER(p_user_id => APEX_UTIL.GET_USER_ID('ADMIN'), p_user_name => 'ADMIN', p_web_password => '%[1]s', p_new_password => '%[1]s');\n" - const CopyApexImages string = " ( while true; do sleep 60; echo \"Copying Apex Images...\" ; done ) & mkdir -p /opt/oracle/oradata/${ORACLE_SID^^}_ORDS/apex/images && " + " cp -R /opt/oracle/oradata/${ORACLE_SID^^}/apex/images/* /opt/oracle/oradata/${ORACLE_SID^^}_ORDS/apex/images; chown -R oracle:oinstall /opt/oracle/oradata/${ORACLE_SID^^}_ORDS/apex; kill -9 $!;" @@ -480,3 +481,12 @@ const SetApexUsers string = "\numask 177" + // Get Sid, Pdbname, Edition for prebuilt db const GetSidPdbEditionCMD string = "echo $ORACLE_SID,$ORACLE_PDB,$ORACLE_EDITION,Edition;" + +// Command to enable TCPS as a formatted string. The parameter would be the port at which TCPS is enabled. +const EnableTcpsCMD string = "$ORACLE_BASE/$CONFIG_TCPS_FILE %d" + +// Command for TCPS certs renewal to prevent their expiry. It is same as the EnableTcpsCMD +const RenewCertsCMD string = EnableTcpsCMD + +// Command to disable TCPS +const DisableTcpsCMD string = "$ORACLE_BASE/$CONFIG_TCPS_FILE disable" diff --git a/commons/database/utils.go b/commons/database/utils.go index c5ebbefd..100b532c 100644 --- a/commons/database/utils.go +++ b/commons/database/utils.go @@ -50,6 +50,7 @@ import ( "unicode" corev1 "k8s.io/api/core/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" apierrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/types" @@ -673,3 +674,18 @@ func ApexPasswordValidator(pwd string) bool { return hasMinLen && hasUpper && hasLower && hasNumber && hasSpecial } + +// Function for patching the K8s service with the payload. +// Patch strategy used: Strategic Merge Patch +func PatchService(config *rest.Config, namespace string, ctx context.Context, req ctrl.Request, svcName string, payload string) error { + log := ctrllog.FromContext(ctx).WithValues("patchService", req.NamespacedName) + client, err := kubernetes.NewForConfig(config) + if err != nil { + log.Error(err, "config error") + } + + // Trying to patch the service resource using Strategic Merge strategy + log.Info("Patching the service", "Service", svcName) + _, err = client.CoreV1().Services(namespace).Patch(ctx, svcName, types.StrategicMergePatchType, []byte(payload), metav1.PatchOptions{}) + return err +} diff --git a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml index 6e7c3ec4..b6aa6352 100644 --- a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml +++ b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml @@ -175,6 +175,8 @@ spec: type: boolean archiveLog: type: string + certCreationTimestamp: + type: string charset: type: string cloneFrom: @@ -283,6 +285,9 @@ spec: type: integer initSgaSize: type: integer + isTcpsEnabled: + default: false + type: boolean nodes: items: type: string @@ -329,7 +334,10 @@ spec: type: object status: type: string + tcpsPort: + type: integer required: + - isTcpsEnabled - persistence type: object type: object diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index c852e9ae..d58aa76c 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -78,8 +78,9 @@ type SingleInstanceDatabaseReconciler struct { var requeueY ctrl.Result = ctrl.Result{Requeue: true, RequeueAfter: 15 * time.Second} var requeueN ctrl.Result = ctrl.Result{} -// Service Port Declaration -var svc_port string +// For scheduling reconcile to renew certs if TCPS is enabled +// Default value is requeueN (No reconcile) +var futureRequeue ctrl.Result = requeueN const singleInstanceDatabaseFinalizer = "database.oracle.com/singleinstancedatabasefinalizer" @@ -137,14 +138,6 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct r.Status().Update(ctx, singleInstanceDatabase) } - // Service Port Initialization - svc_port = func() string { - if singleInstanceDatabase.Spec.EnableTCPS { - return strconv.Itoa(singleInstanceDatabase.Spec.TcpsPort) - } - return "1521" - }() - // Manage SingleInstanceDatabase Deletion result, err = r.manageSingleInstanceDatabaseDeletion(req, ctx, singleInstanceDatabase) if result.Requeue { @@ -203,6 +196,13 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct } } + // Configure TCPS + result, err = r.configTcps(singleInstanceDatabase, readyPod, ctx, req) + if result.Requeue { + r.Log.Info("Reconcile queued") + return result, nil + } + // Update DB config result, err = r.updateDBConfig(singleInstanceDatabase, readyPod, ctx, req) if result.Requeue { @@ -241,6 +241,15 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct completed = true r.Log.Info("Reconcile completed") + + // Scheduling a reconcile for certificate renewal, if TCPS is enabled + if futureRequeue != requeueN { + r.Log.Info("Scheduling Reconcile for cert renewal", "Duration(Hours)", futureRequeue.RequeueAfter.Hours()) + copyFutureRequeue := futureRequeue + futureRequeue = requeueN + return copyFutureRequeue, nil + } + return requeueN, nil } @@ -650,16 +659,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, }, }, - Ports: []corev1.ContainerPort{ - {ContainerPort: func() int32 { - if m.Spec.EnableTCPS { - return int32(m.Spec.TcpsPort) - } - return int32(1521) - }(), - }, - {ContainerPort: 5500}, - }, + Ports: []corev1.ContainerPort{{ContainerPort: dbcommons.DEFAULT_LISTENER_PORT}, {ContainerPort: 5500}}, ReadinessProbe: &corev1.Probe{ ProbeHandler: corev1.ProbeHandler{ @@ -707,7 +707,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, { Name: "SVC_PORT", - Value: svc_port, + Value: strconv.Itoa(int(dbcommons.DEFAULT_LISTENER_PORT)), }, { Name: "ORACLE_CHARACTERSET", @@ -717,14 +717,6 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns Name: "ORACLE_EDITION", Value: m.Spec.Edition, }, - { - Name: "ENABLE_TCPS", - Value: strconv.FormatBool(m.Spec.EnableTCPS), - }, - { - Name: "TCPS_PORT", - Value: strconv.Itoa(m.Spec.TcpsPort), - }, } } if m.Spec.CloneFrom == "" { @@ -736,7 +728,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, { Name: "SVC_PORT", - Value: svc_port, + Value: strconv.Itoa(int(dbcommons.DEFAULT_LISTENER_PORT)), }, { Name: "CREATE_PDB", @@ -794,14 +786,6 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns Name: "SKIP_DATAPATCH", Value: "true", }, - { - Name: "ENABLE_TCPS", - Value: strconv.FormatBool(m.Spec.EnableTCPS), - }, - { - Name: "TCPS_PORT", - Value: strconv.Itoa(m.Spec.TcpsPort), - }, } } // For clone DB use case @@ -812,7 +796,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, { Name: "SVC_PORT", - Value: svc_port, + Value: strconv.Itoa(int(dbcommons.DEFAULT_LISTENER_PORT)), }, { Name: "ORACLE_SID", @@ -826,7 +810,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns Name: "PRIMARY_DB_CONN_STR", Value: func() string { if dbcommons.IsSourceDatabaseOnCluster(m.Spec.CloneFrom) { - return n.Name + ":" + svc_port + "/" + n.Spec.Sid + return n.Name + ":" + strconv.Itoa(int(dbcommons.DEFAULT_LISTENER_PORT)) + "/" + n.Spec.Sid } return m.Spec.CloneFrom }(), @@ -890,13 +874,14 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns //############################################################################# // Instantiate Service spec from SingleInstanceDatabase spec //############################################################################# -func (r *SingleInstanceDatabaseReconciler) instantiateSVCSpec(m *dbapi.SingleInstanceDatabase) *corev1.Service { +func (r *SingleInstanceDatabaseReconciler) instantiateSVCSpec(m *dbapi.SingleInstanceDatabase, + svcName string, listenerPort int32, svcType corev1.ServiceType) *corev1.Service { svc := &corev1.Service{ TypeMeta: metav1.TypeMeta{ Kind: "Service", }, ObjectMeta: metav1.ObjectMeta{ - Name: m.Name, + Name: svcName, Namespace: m.Namespace, Labels: map[string]string{ "app": m.Name, @@ -914,13 +899,8 @@ func (r *SingleInstanceDatabaseReconciler) instantiateSVCSpec(m *dbapi.SingleIns Spec: corev1.ServiceSpec{ Ports: []corev1.ServicePort{ { - Name: "listener", - Port: func() int32 { - if m.Spec.EnableTCPS { - return int32(m.Spec.TcpsPort) - } - return int32(1521) - }(), + Name: "listener", + Port: listenerPort, Protocol: corev1.ProtocolTCP, }, { @@ -932,12 +912,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiateSVCSpec(m *dbapi.SingleIns Selector: map[string]string{ "app": m.Name, }, - Type: corev1.ServiceType(func() string { - if m.Spec.LoadBalancer { - return "LoadBalancer" - } - return "NodePort" - }()), + Type: svcType, }, } // Set SingleInstanceDatabase instance as the owner and controller @@ -1073,49 +1048,121 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex log := r.Log.WithValues("createOrReplaceSVC", req.NamespacedName) - svcDeleted := false - // Check if the Service already exists, if not create a new one - svc := &corev1.Service{} - // Get retrieves an obj ( a struct pointer ) for the given object key from the Kubernetes Cluster. - err := r.Get(ctx, types.NamespacedName{Name: m.Name, Namespace: m.Namespace}, svc) - if err == nil { - svcType := corev1.ServiceType("NodePort") - if m.Spec.LoadBalancer { - svcType = corev1.ServiceType("LoadBalancer") + /** If TCPS is enabled, there will be two k8s services: + 1. One service exposing the TCPS port for the user to connect, + 2. One service exposing default listerner port inside the cluster only (for ORDS, APEX installation etc) + **/ + + defaultSvc := &corev1.Service{} + tcpsSvc := &corev1.Service{} + // userSvc would indicate the service which is used to connect to the Database by the database user + var userSvc *corev1.Service + + defaultSvcName := m.Name + tcpsSvcName := m.Name + "-tcps" + + // Querying for the K8s service resources + getDefaultSvcErr := r.Get(ctx, types.NamespacedName{Name: defaultSvcName, Namespace: m.Namespace}, defaultSvc) + getTcpsSvcErr := r.Get(ctx, types.NamespacedName{Name: tcpsSvcName, Namespace: m.Namespace}, tcpsSvc) + + // svcType defines the type of the service as specified in the singleinstancedatabase.yaml file + svcType := corev1.ServiceType("NodePort") + if m.Spec.LoadBalancer { + svcType = corev1.ServiceType("LoadBalancer") + } + + if m.Spec.EnableTCPS { + if getDefaultSvcErr != nil && apierrors.IsNotFound(getDefaultSvcErr) { + // Create a new cluster service + svc := r.instantiateSVCSpec(m, defaultSvcName, dbcommons.DEFAULT_LISTENER_PORT, corev1.ServiceType("ClusterIP")) + log.Info("Creating a new service", "Service.Namespace", svc.Namespace, "Service.Name", svc.Name) + err := r.Create(ctx, svc) + if err != nil { + log.Error(err, "Failed to create new service", "Service.Namespace", svc.Namespace, "Service.Name", svc.Name) + return requeueY, err + } + } else if defaultSvc.Spec.Type != corev1.ServiceType("ClusterIP") { + // Patch the service type + payload := "{\"spec\": {\"type\": \"ClusterIP\"}}" + //Attempt Service patching + log.Info("Patching the service", "Service.Name", defaultSvc.Name, "Target Type", "ClusterIP") + err := dbcommons.PatchService(r.Config, m.Namespace, ctx, req, defaultSvcName, payload) + if err != nil { + log.Error(err, "Failed to patch Service") + return requeueY, err + } + } - if svc.Spec.Type != svcType { - log.Info("Deleting service", "name", svc.Name) - err = r.Delete(ctx, svc) + // When TCPS is enabled userSvc would point to tcpsSvc + userSvc = tcpsSvc + if getTcpsSvcErr != nil && apierrors.IsNotFound(getTcpsSvcErr) { + // Reset connect strings whenever service is recreated /* + m.Status.ConnectString = dbcommons.ValueUnavailable + m.Status.PdbConnectString = dbcommons.ValueUnavailable + m.Status.OemExpressUrl = dbcommons.ValueUnavailable + // Create a new service + svc := r.instantiateSVCSpec(m, tcpsSvcName, int32(m.Spec.TcpsPort), svcType) + log.Info("Creating a new service", "Service.Namespace", svc.Namespace, "Service.Name", svc.Name) + err := r.Create(ctx, svc) if err != nil { - r.Log.Error(err, "Failed to delete service", "name", svc.Name) - return requeueN, err + log.Error(err, "Failed to create new service", "Service.Namespace", svc.Namespace, "Service.Name", svc.Name) + return requeueY, err + } + userSvc = svc + } else if tcpsSvc.Spec.Type != svcType { + // Patch the service type + payload := fmt.Sprintf("{\"spec\": {\"type\": \"%s\"}}", svcType) + //Attempt Service patching + log.Info("Patching the service", "Service.Name", defaultSvc.Name, "Target Type", svcType) + err := dbcommons.PatchService(r.Config, m.Namespace, ctx, req, tcpsSvcName, payload) + if err != nil { + log.Error(err, "Failed to patch Service") + return requeueY, err } - svcDeleted = true } - } - if svcDeleted || err != nil && apierrors.IsNotFound(err) { - // Define a new Service - svc = r.instantiateSVCSpec(m) - log.Info("Creating a new service", "Service.Namespace", svc.Namespace, "Service.Name", svc.Name) - err = r.Create(ctx, svc) - if err != nil { - log.Error(err, "Failed to create new service", "Service.Namespace", svc.Namespace, "Service.Name", svc.Name) - return requeueY, err + + } else { + // Only one service is required if TCPS is not enabled + // userSvc would point to the defaultSvc + userSvc = defaultSvc + if getDefaultSvcErr != nil && apierrors.IsNotFound(getDefaultSvcErr) { + // Reset connect strings whenever service is recreated /* + m.Status.ConnectString = dbcommons.ValueUnavailable + m.Status.PdbConnectString = dbcommons.ValueUnavailable + m.Status.OemExpressUrl = dbcommons.ValueUnavailable + // Create a new service with + svc := r.instantiateSVCSpec(m, defaultSvcName, dbcommons.DEFAULT_LISTENER_PORT, svcType) + log.Info("Creating a new service", "Service.Namespace", svc.Namespace, "Service.Name", svc.Name) + err := r.Create(ctx, svc) + if err != nil { + log.Error(err, "Failed to create new service", "Service.Namespace", svc.Namespace, "Service.Name", svc.Name) + return requeueY, err + } + userSvc = svc + } else if defaultSvc.Spec.Type != svcType { + // Patch the service type + payload := fmt.Sprintf("{\"spec\": {\"type\": \"%s\"}}", svcType) + //Attempt Service patching + log.Info("Patching the service", "Service.Name", defaultSvc.Name, "Target Type", svcType) + err := dbcommons.PatchService(r.Config, m.Namespace, ctx, req, defaultSvcName, payload) + if err != nil { + log.Error(err, "Failed to patch Service") + return requeueY, err + } } - eventReason := "Service creation" - eventMsg := "successfully created service type " + string(svc.Spec.Type) - r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) - log.Info(eventMsg) - // Reset connect strings whenever service is recreated /* - m.Status.ConnectString = dbcommons.ValueUnavailable - m.Status.PdbConnectString = dbcommons.ValueUnavailable - m.Status.OemExpressUrl = dbcommons.ValueUnavailable - } else if err != nil { - log.Error(err, "Failed to get Service") - return requeueY, err + + if getTcpsSvcErr == nil { + // Delete this tcps service + log.Info("Deleting service", "name", tcpsSvcName) + err := r.Delete(ctx, tcpsSvc) + if err != nil { + r.Log.Error(err, "Failed to delete service", "name", tcpsSvc.Name) + return requeueN, err + } + } + } - log.Info("Found Existing Service ", "Service Name ", svc.Name) pdbName := strings.ToUpper(m.Spec.Pdbname) sid := m.Spec.Sid @@ -1129,21 +1176,21 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex } if m.Spec.LoadBalancer { - m.Status.ClusterConnectString = svc.Name + "." + svc.Namespace + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/" + strings.ToUpper(sid) - if len(svc.Status.LoadBalancer.Ingress) > 0 { - m.Status.ConnectString = svc.Status.LoadBalancer.Ingress[0].IP + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/" + strings.ToUpper(sid) - m.Status.PdbConnectString = svc.Status.LoadBalancer.Ingress[0].IP + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/" + strings.ToUpper(pdbName) - m.Status.OemExpressUrl = "https://" + svc.Status.LoadBalancer.Ingress[0].IP + ":" + fmt.Sprint(svc.Spec.Ports[1].Port) + "/em" + m.Status.ClusterConnectString = userSvc.Name + "." + userSvc.Namespace + ":" + fmt.Sprint(userSvc.Spec.Ports[0].Port) + "/" + strings.ToUpper(sid) + if len(userSvc.Status.LoadBalancer.Ingress) > 0 { + m.Status.ConnectString = userSvc.Status.LoadBalancer.Ingress[0].IP + ":" + fmt.Sprint(userSvc.Spec.Ports[0].Port) + "/" + strings.ToUpper(sid) + m.Status.PdbConnectString = userSvc.Status.LoadBalancer.Ingress[0].IP + ":" + fmt.Sprint(userSvc.Spec.Ports[0].Port) + "/" + strings.ToUpper(pdbName) + m.Status.OemExpressUrl = "https://" + userSvc.Status.LoadBalancer.Ingress[0].IP + ":" + fmt.Sprint(userSvc.Spec.Ports[1].Port) + "/em" } return requeueN, nil } - m.Status.ClusterConnectString = svc.Name + "." + svc.Namespace + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/" + strings.ToUpper(sid) + m.Status.ClusterConnectString = userSvc.Name + "." + userSvc.Namespace + ":" + fmt.Sprint(userSvc.Spec.Ports[0].Port) + "/" + strings.ToUpper(sid) nodeip := dbcommons.GetNodeIp(r, ctx, req) if nodeip != "" { - m.Status.ConnectString = nodeip + ":" + fmt.Sprint(svc.Spec.Ports[0].NodePort) + "/" + strings.ToUpper(sid) - m.Status.PdbConnectString = nodeip + ":" + fmt.Sprint(svc.Spec.Ports[0].NodePort) + "/" + strings.ToUpper(pdbName) - m.Status.OemExpressUrl = "https://" + nodeip + ":" + fmt.Sprint(svc.Spec.Ports[1].NodePort) + "/em" + m.Status.ConnectString = nodeip + ":" + fmt.Sprint(userSvc.Spec.Ports[0].NodePort) + "/" + strings.ToUpper(sid) + m.Status.PdbConnectString = nodeip + ":" + fmt.Sprint(userSvc.Spec.Ports[0].NodePort) + "/" + strings.ToUpper(pdbName) + m.Status.OemExpressUrl = "https://" + nodeip + ":" + fmt.Sprint(userSvc.Spec.Ports[1].NodePort) + "/em" } return requeueN, nil @@ -1708,6 +1755,83 @@ func (r *SingleInstanceDatabaseReconciler) deleteWallet(m *dbapi.SingleInstanceD return requeueN, nil } +//############################################################################# +// Configuring TCPS +//############################################################################# +func (r *SingleInstanceDatabaseReconciler) configTcps(m *dbapi.SingleInstanceDatabase, + readyPod corev1.Pod, ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + eventReason := "Configuring TCPS" + if m.Spec.EnableTCPS && !m.Status.IsTcpsEnabled { + // Enable TCPS + eventMsg := "Enabling TCPS in the database..." + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + + _, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", + ctx, req, false, "bash", "-c", fmt.Sprintf(dbcommons.EnableTcpsCMD, m.Spec.TcpsPort)) + if err != nil { + r.Log.Error(err, err.Error()) + eventMsg = "Error encountered in enabling TCPS!" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + return requeueY, nil + } + // Updating the Status and publishing the event + m.Status.CertCreationTimestamp = time.Now().Format(time.RFC3339) + m.Status.IsTcpsEnabled = true + r.Status().Update(ctx, m) + + eventMsg = "TCPS Enabled." + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + + // 26040h = 1085 days + futureRequeue = ctrl.Result{Requeue: true, RequeueAfter: func() time.Duration { requeueDuration, _ := time.ParseDuration("26040h"); return requeueDuration }()} + + } else if !m.Spec.EnableTCPS && m.Status.IsTcpsEnabled { + // Disable TCPS + eventMsg := "Disabling TCPS in the database..." + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + + _, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", + ctx, req, false, "bash", "-c", dbcommons.DisableTcpsCMD) + if err != nil { + r.Log.Error(err, err.Error()) + return requeueY, nil + } + + // Updating the Status and publishing the event + m.Status.CertCreationTimestamp = "" + m.Status.IsTcpsEnabled = false + r.Status().Update(ctx, m) + + eventMsg = "TCPS Disabled." + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + + } else if m.Spec.EnableTCPS && m.Status.IsTcpsEnabled { + // Cert Renewal Logic + // Certificates are renewed when 10 days remain for certs expiry + certCreationTimestamp, _ := time.Parse(time.RFC3339, m.Status.CertCreationTimestamp) + duration := time.Since(certCreationTimestamp) + allowdDuration, _ := time.ParseDuration("26000h") + if duration > allowdDuration { + _, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", + ctx, req, false, "bash", "-c", fmt.Sprintf(dbcommons.EnableTcpsCMD, m.Spec.TcpsPort)) + if err != nil { + r.Log.Error(err, err.Error()) + return requeueY, nil + } + // Updating the Status and publishing the event + m.Status.CertCreationTimestamp = time.Now().Format(time.RFC3339) + r.Status().Update(ctx, m) + + eventMsg := "TCPS Certificates Renewed at time %s," + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg, time.Now().Format(time.RFC3339)) + + // 26040h = 1085 days + futureRequeue = ctrl.Result{Requeue: true, RequeueAfter: func() time.Duration { requeueDuration, _ := time.ParseDuration("26040h"); return requeueDuration }()} + } + } + return requeueN, nil +} + //############################################################################# // Execute Datapatch //############################################################################# From a22970dc64f4ffa4bfacbefa3aa5695e23a6e7ec Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Mon, 8 Aug 2022 12:32:18 +0530 Subject: [PATCH 483/628] Added port change validation, status to Updating while enabling/disabling tcps Signed-off-by: abhisbyk --- .../v1alpha1/singleinstancedatabase_webhook.go | 4 ++++ .../database/singleinstancedatabase_controller.go | 11 +++++++++++ 2 files changed, 15 insertions(+) diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index 2b527083..9dc171e4 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -297,6 +297,10 @@ func (r *SingleInstanceDatabase) ValidateUpdate(oldRuntimeObject runtime.Object) allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("persistence"), "uninstall ORDS to change Persistence")) } + if old.Status.IsTcpsEnabled && old.Status.TcpsPort != r.Spec.TcpsPort { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("tcpsPort"), "cannot change TCPS port, please disable TCPS first then enable it with newly desired port")) + } if len(allErrs) == 0 { return nil } diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index d58aa76c..d232d189 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1763,6 +1763,9 @@ func (r *SingleInstanceDatabaseReconciler) configTcps(m *dbapi.SingleInstanceDat eventReason := "Configuring TCPS" if m.Spec.EnableTCPS && !m.Status.IsTcpsEnabled { // Enable TCPS + m.Status.Status = dbcommons.StatusUpdating + r.Status().Update(ctx, m) + eventMsg := "Enabling TCPS in the database..." r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) @@ -1777,6 +1780,7 @@ func (r *SingleInstanceDatabaseReconciler) configTcps(m *dbapi.SingleInstanceDat // Updating the Status and publishing the event m.Status.CertCreationTimestamp = time.Now().Format(time.RFC3339) m.Status.IsTcpsEnabled = true + m.Status.TcpsPort = m.Spec.TcpsPort r.Status().Update(ctx, m) eventMsg = "TCPS Enabled." @@ -1787,6 +1791,9 @@ func (r *SingleInstanceDatabaseReconciler) configTcps(m *dbapi.SingleInstanceDat } else if !m.Spec.EnableTCPS && m.Status.IsTcpsEnabled { // Disable TCPS + m.Status.Status = dbcommons.StatusUpdating + r.Status().Update(ctx, m) + eventMsg := "Disabling TCPS in the database..." r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) @@ -1799,6 +1806,7 @@ func (r *SingleInstanceDatabaseReconciler) configTcps(m *dbapi.SingleInstanceDat // Updating the Status and publishing the event m.Status.CertCreationTimestamp = "" + m.Status.TcpsPort = 0 m.Status.IsTcpsEnabled = false r.Status().Update(ctx, m) @@ -1812,6 +1820,9 @@ func (r *SingleInstanceDatabaseReconciler) configTcps(m *dbapi.SingleInstanceDat duration := time.Since(certCreationTimestamp) allowdDuration, _ := time.ParseDuration("26000h") if duration > allowdDuration { + m.Status.Status = dbcommons.StatusUpdating + r.Status().Update(ctx, m) + _, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", fmt.Sprintf(dbcommons.EnableTcpsCMD, m.Spec.TcpsPort)) if err != nil { From 9886cb9e8cf7fadd6f4076f43aeb0ec25b2332b0 Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Mon, 8 Aug 2022 21:40:28 +0530 Subject: [PATCH 484/628] Updating the status properly for the LB Signed-off-by: abhisbyk --- .../singleinstancedatabase_controller.go | 27 ++++++++++--------- 1 file changed, 14 insertions(+), 13 deletions(-) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 267d36ce..128ae2ff 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1124,13 +1124,15 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex } else { // Only one service is required if TCPS is not enabled + + // Reset connect strings whenever service is recreated /* + m.Status.ConnectString = dbcommons.ValueUnavailable + m.Status.PdbConnectString = dbcommons.ValueUnavailable + m.Status.OemExpressUrl = dbcommons.ValueUnavailable + // userSvc would point to the defaultSvc userSvc = defaultSvc if getDefaultSvcErr != nil && apierrors.IsNotFound(getDefaultSvcErr) { - // Reset connect strings whenever service is recreated /* - m.Status.ConnectString = dbcommons.ValueUnavailable - m.Status.PdbConnectString = dbcommons.ValueUnavailable - m.Status.OemExpressUrl = dbcommons.ValueUnavailable // Create a new service with svc := r.instantiateSVCSpec(m, defaultSvcName, dbcommons.DEFAULT_LISTENER_PORT, svcType) log.Info("Creating a new service", "Service.Namespace", svc.Namespace, "Service.Name", svc.Name) @@ -1187,15 +1189,14 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex m.Status.PdbConnectString = lbAddress + ":" + fmt.Sprint(userSvc.Spec.Ports[0].Port) + "/" + strings.ToUpper(pdbName) m.Status.OemExpressUrl = "https://" + lbAddress + ":" + fmt.Sprint(userSvc.Spec.Ports[1].Port) + "/em" } - return requeueN, nil - } - - m.Status.ClusterConnectString = userSvc.Name + "." + userSvc.Namespace + ":" + fmt.Sprint(userSvc.Spec.Ports[0].Port) + "/" + strings.ToUpper(sid) - nodeip := dbcommons.GetNodeIp(r, ctx, req) - if nodeip != "" { - m.Status.ConnectString = nodeip + ":" + fmt.Sprint(userSvc.Spec.Ports[0].NodePort) + "/" + strings.ToUpper(sid) - m.Status.PdbConnectString = nodeip + ":" + fmt.Sprint(userSvc.Spec.Ports[0].NodePort) + "/" + strings.ToUpper(pdbName) - m.Status.OemExpressUrl = "https://" + nodeip + ":" + fmt.Sprint(userSvc.Spec.Ports[1].NodePort) + "/em" + } else { + m.Status.ClusterConnectString = userSvc.Name + "." + userSvc.Namespace + ":" + fmt.Sprint(userSvc.Spec.Ports[0].Port) + "/" + strings.ToUpper(sid) + nodeip := dbcommons.GetNodeIp(r, ctx, req) + if nodeip != "" { + m.Status.ConnectString = nodeip + ":" + fmt.Sprint(userSvc.Spec.Ports[0].NodePort) + "/" + strings.ToUpper(sid) + m.Status.PdbConnectString = nodeip + ":" + fmt.Sprint(userSvc.Spec.Ports[0].NodePort) + "/" + strings.ToUpper(pdbName) + m.Status.OemExpressUrl = "https://" + nodeip + ":" + fmt.Sprint(userSvc.Spec.Ports[1].NodePort) + "/em" + } } return requeueN, nil From 522ab6b2ad68523c39f9434068aa7ea8bd212abc Mon Sep 17 00:00:00 2001 From: Ruggero Citton Date: Tue, 16 Aug 2022 10:46:48 +0200 Subject: [PATCH 485/628] bug:33822886 --- apis/database/v1alpha1/pdb_webhook.go | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-) diff --git a/apis/database/v1alpha1/pdb_webhook.go b/apis/database/v1alpha1/pdb_webhook.go index b02b797d..573a8729 100644 --- a/apis/database/v1alpha1/pdb_webhook.go +++ b/apis/database/v1alpha1/pdb_webhook.go @@ -36,6 +36,11 @@ ** SOFTWARE. */ +/* MODIFIED (MM/DD/YY) +** rcitton 07/14/22 - 33822886 +*/ + + package v1alpha1 import ( @@ -157,6 +162,14 @@ func (r *PDB) validateAction(allErrs *field.ErrorList) { *allErrs = append(*allErrs, field.Required(field.NewPath("spec").Child("fileNameConversions"), "Please specify a value for fileNameConversions. Values can be a filename convert pattern or NONE")) } + if r.Spec.TotalSize == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("totalSize"), "When the storage is not UNLIMITED the Total Size must be specified")) + } + if r.Spec.TempSize == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("tempSize"), "When the storage is not UNLIMITED the Temp Size must be specified")) + } if *(r.Spec.TDEImport) { r.validateTDEInfo(allErrs) } @@ -168,11 +181,11 @@ func (r *PDB) validateAction(allErrs *field.ErrorList) { } if r.Spec.TotalSize == "" { *allErrs = append(*allErrs, - field.Required(field.NewPath("spec").Child("totalSize"), "Please specify size of the tablespace")) + field.Required(field.NewPath("spec").Child("totalSize"), "When the storage is not UNLIMITED the Total Size must be specified")) } if r.Spec.TempSize == "" { *allErrs = append(*allErrs, - field.Required(field.NewPath("spec").Child("tempSize"), "Please specify size of the temporary tablespace")) + field.Required(field.NewPath("spec").Child("tempSize"), "When the storage is not UNLIMITED the Temp Size must be specified")) } case "PLUG": if r.Spec.XMLFileName == "" { From 6a96a2b0afdb29255462b4fb73bf9e4b1a4444f6 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Wed, 17 Aug 2022 11:22:11 -0400 Subject: [PATCH 486/628] remove deprecated go get --- Makefile | 57 +++++++++++++++++++++++++++++++------------------------- 1 file changed, 32 insertions(+), 25 deletions(-) diff --git a/Makefile b/Makefile index 78cf72ac..4736ead9 100644 --- a/Makefile +++ b/Makefile @@ -104,32 +104,38 @@ operator-yaml: manifests kustomize undeploy: ## Undeploy controller from the K8s cluster specified in ~/.kube/config. $(KUSTOMIZE) build config/default | kubectl delete -f - +##@ Build Dependencies + +## Location to install dependencies to +LOCALBIN ?= $(shell pwd)/bin +$(LOCALBIN): + mkdir -p $(LOCALBIN) + +## Tool Binaries +KUSTOMIZE ?= $(LOCALBIN)/kustomize +CONTROLLER_GEN ?= $(LOCALBIN)/controller-gen +ENVTEST ?= $(LOCALBIN)/setup-envtest + +## Tool Versions +KUSTOMIZE_VERSION ?= v3.8.7 +CONTROLLER_TOOLS_VERSION ?= v0.6.1 + +KUSTOMIZE_INSTALL_SCRIPT ?= "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh" +.PHONY: kustomize +kustomize: $(KUSTOMIZE) ## Download kustomize locally if necessary. +$(KUSTOMIZE): $(LOCALBIN) + curl -s $(KUSTOMIZE_INSTALL_SCRIPT) | bash -s -- $(subst v,,$(KUSTOMIZE_VERSION)) $(LOCALBIN) + +.PHONY: controller-gen +controller-gen: $(CONTROLLER_GEN) ## Download controller-gen locally if necessary. +$(CONTROLLER_GEN): $(LOCALBIN) + GOBIN=$(LOCALBIN) go install sigs.k8s.io/controller-tools/cmd/controller-gen@$(CONTROLLER_TOOLS_VERSION) + +.PHONY: envtest +envtest: $(ENVTEST) ## Download envtest-setup locally if necessary. +$(ENVTEST): $(LOCALBIN) + GOBIN=$(LOCALBIN) go install sigs.k8s.io/controller-runtime/tools/setup-envtest@latest -CONTROLLER_GEN = $(shell pwd)/bin/controller-gen -controller-gen: ## Download controller-gen locally if necessary. - $(call go-get-tool,$(CONTROLLER_GEN),sigs.k8s.io/controller-tools/cmd/controller-gen@v0.6.1) - -KUSTOMIZE = $(shell pwd)/bin/kustomize -kustomize: ## Download kustomize locally if necessary. - $(call go-get-tool,$(KUSTOMIZE),sigs.k8s.io/kustomize/kustomize/v3@v3.8.7) - -ENVTEST = $(shell pwd)/bin/setup-envtest -envtest: ## Download envtest-setup locally if necessary. - $(call go-get-tool,$(ENVTEST),sigs.k8s.io/controller-runtime/tools/setup-envtest@latest) - -# go-get-tool will 'go get' any package $2 and install it to $1. -PROJECT_DIR := $(shell dirname $(abspath $(lastword $(MAKEFILE_LIST)))) -define go-get-tool -@[ -f $(1) ] || { \ -set -e ;\ -TMP_DIR=$$(mktemp -d) ;\ -cd $$TMP_DIR ;\ -go mod init tmp ;\ -echo "Downloading $(2)" ;\ -GOBIN=$(PROJECT_DIR)/bin go get $(2) ;\ -rm -rf $$TMP_DIR ;\ -} -endef .PHONY: bundle bundle: manifests kustomize ## Generate bundle manifests and metadata, then validate generated files. @@ -186,3 +192,4 @@ catalog-build: opm ## Build a catalog image. .PHONY: catalog-push catalog-push: ## Push a catalog image. $(MAKE) docker-push IMG=$(CATALOG_IMG) + From ea48a4ec09c7302a1895e779a04b0536033e3267 Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Mon, 22 Aug 2022 12:58:48 +0530 Subject: [PATCH 487/628] Two services: clusterIP for internal communication and external LB/NP for users Signed-off-by: abhisbyk --- .../v1alpha1/singleinstancedatabase_types.go | 5 +- .../singleinstancedatabase_webhook.go | 14 +- commons/database/constants.go | 9 +- ...se.oracle.com_singleinstancedatabases.yaml | 10 +- .../samples/sidb/singleinstancedatabase.yaml | 8 +- .../singleinstancedatabase_controller.go | 324 +++++++++++------- 6 files changed, 224 insertions(+), 146 deletions(-) diff --git a/apis/database/v1alpha1/singleinstancedatabase_types.go b/apis/database/v1alpha1/singleinstancedatabase_types.go index 5f8efbc4..d5b26e78 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_types.go +++ b/apis/database/v1alpha1/singleinstancedatabase_types.go @@ -61,12 +61,12 @@ type SingleInstanceDatabaseSpec struct { Charset string `json:"charset,omitempty"` Pdbname string `json:"pdbName,omitempty"` LoadBalancer bool `json:"loadBalancer,omitempty"` + ServicePort int `json:"servicePort,omitempty"` ServiceAnnotations map[string]string `json:"serviceAnnotations,omitempty"` FlashBack bool `json:"flashBack,omitempty"` ArchiveLog bool `json:"archiveLog,omitempty"` ForceLogging bool `json:"forceLog,omitempty"` EnableTCPS bool `json:"enableTCPS,omitempty"` - TcpsPort int `json:"tcpsPort,omitempty"` CloneFrom string `json:"cloneFrom,omitempty"` ReadinessCheckPeriod int `json:"readinessCheckPeriod,omitempty"` @@ -150,8 +150,9 @@ type SingleInstanceDatabaseStatus struct { PrebuiltDB bool `json:"prebuiltDB,omitempty"` // +kubebuilder:default:=false IsTcpsEnabled bool `json:"isTcpsEnabled"` - TcpsPort int `json:"tcpsPort,omitempty"` CertCreationTimestamp string `json:"certCreationTimestamp,omitempty"` + DbHostname string `json:"dbHostname,omitempty"` + DbPort int `json:"dbPort,omitempty"` // +patchMergeKey=type // +patchStrategy=merge diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index 9dc171e4..0d450871 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -234,6 +234,16 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { } } + // servicePort validation + if !r.Spec.LoadBalancer { + // NodePort service is expected. In this case servicePort should be in range 30000-32767 + if r.Spec.ServicePort != 0 && (r.Spec.ServicePort < 30000 || r.Spec.ServicePort > 32767) { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("servicePort"), r.Spec.ServicePort, + "servicePort should be in 30000-32767 range.")) + } + } + if len(allErrs) == 0 { return nil } @@ -297,10 +307,6 @@ func (r *SingleInstanceDatabase) ValidateUpdate(oldRuntimeObject runtime.Object) allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("persistence"), "uninstall ORDS to change Persistence")) } - if old.Status.IsTcpsEnabled && old.Status.TcpsPort != r.Spec.TcpsPort { - allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec").Child("tcpsPort"), "cannot change TCPS port, please disable TCPS first then enable it with newly desired port")) - } if len(allErrs) == 0 { return nil } diff --git a/commons/database/constants.go b/commons/database/constants.go index 0e71742d..3aeb66f7 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -38,7 +38,9 @@ package commons -const DEFAULT_LISTENER_PORT int32 = 1521 +const CONTAINER_LISTENER_PORT int32 = 1521 + +const CONTAINER_TCPS_PORT int32 = 1522 const ORACLE_UID int64 = 54321 @@ -483,10 +485,13 @@ const SetApexUsers string = "\numask 177" + const GetSidPdbEditionCMD string = "echo $ORACLE_SID,$ORACLE_PDB,$ORACLE_EDITION,Edition;" // Command to enable TCPS as a formatted string. The parameter would be the port at which TCPS is enabled. -const EnableTcpsCMD string = "$ORACLE_BASE/$CONFIG_TCPS_FILE %d" +const EnableTcpsCMD string = "$ORACLE_BASE/$CONFIG_TCPS_FILE" // Command for TCPS certs renewal to prevent their expiry. It is same as the EnableTcpsCMD const RenewCertsCMD string = EnableTcpsCMD // Command to disable TCPS const DisableTcpsCMD string = "$ORACLE_BASE/$CONFIG_TCPS_FILE disable" + +// TCPS clientWallet update command +const ClientWalletUpdate string = "sed -i -e 's/HOST.*$/HOST=%s)/g' -e 's/PORT.*$/PORT=%d)/g' ${ORACLE_BASE}/oradata/clientWallet/${ORACLE_SID}/tnsnames.ora" diff --git a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml index b6aa6352..b8383ef8 100644 --- a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml +++ b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml @@ -156,14 +156,14 @@ spec: additionalProperties: type: string type: object + servicePort: + type: integer sid: description: SID must be alphanumeric (no special characters, only a-z, A-Z, 0-9), and no longer than 12 characters. maxLength: 12 pattern: ^[a-zA-Z0-9]+$ type: string - tcpsPort: - type: integer required: - image type: object @@ -263,6 +263,10 @@ spec: datafilesPatched: default: "false" type: string + dbHostname: + type: string + dbPort: + type: integer edition: type: string flashBack: @@ -334,8 +338,6 @@ spec: type: object status: type: string - tcpsPort: - type: integer required: - isTcpsEnabled - persistence diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index 0d099255..9adc9601 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -47,9 +47,6 @@ spec: ## Enable TCPS enableTCPS: false - ## TCPS custom port - tcpsPort: 1522 - ## NA if cloning from a SourceDB (cloneFrom is set) ## Specify both sgaSize and pgaSize (in MB) or dont specify both ## Specify Non-Zero value to use @@ -85,6 +82,11 @@ spec: ## Type of service . Applicable on cloud enviroments only ## if loadBalService : false, service type = "NodePort" else "LoadBalancer" loadBalancer: false + + ## If loadBalancer is enabled, the servicePort is the load balancer port number + ## If loadBalancer is disabled, the servicePort is the NodePort(should be in range 30000-32767) + #servicePort: 30001 + ## Service Annotations (Cloud provider specific), for configuring the service (e.g. private LoadBalancer service) #serviceAnnotations: # service.beta.kubernetes.io/oci-load-balancer-internal: "true" diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 128ae2ff..09b54fdb 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -57,6 +57,7 @@ import ( metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/runtime" "k8s.io/apimachinery/pkg/types" + "k8s.io/apimachinery/pkg/util/intstr" "k8s.io/client-go/rest" "k8s.io/client-go/tools/record" ctrl "sigs.k8s.io/controller-runtime" @@ -659,7 +660,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, }, }, - Ports: []corev1.ContainerPort{{ContainerPort: dbcommons.DEFAULT_LISTENER_PORT}, {ContainerPort: 5500}}, + Ports: []corev1.ContainerPort{{ContainerPort: dbcommons.CONTAINER_LISTENER_PORT}, {ContainerPort: 5500}}, ReadinessProbe: &corev1.Probe{ ProbeHandler: corev1.ProbeHandler{ @@ -707,7 +708,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, { Name: "SVC_PORT", - Value: strconv.Itoa(int(dbcommons.DEFAULT_LISTENER_PORT)), + Value: strconv.Itoa(int(dbcommons.CONTAINER_LISTENER_PORT)), }, { Name: "ORACLE_CHARACTERSET", @@ -728,7 +729,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, { Name: "SVC_PORT", - Value: strconv.Itoa(int(dbcommons.DEFAULT_LISTENER_PORT)), + Value: strconv.Itoa(int(dbcommons.CONTAINER_LISTENER_PORT)), }, { Name: "CREATE_PDB", @@ -796,7 +797,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, { Name: "SVC_PORT", - Value: strconv.Itoa(int(dbcommons.DEFAULT_LISTENER_PORT)), + Value: strconv.Itoa(int(dbcommons.CONTAINER_LISTENER_PORT)), }, { Name: "ORACLE_SID", @@ -810,7 +811,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns Name: "PRIMARY_DB_CONN_STR", Value: func() string { if dbcommons.IsSourceDatabaseOnCluster(m.Spec.CloneFrom) { - return n.Name + ":" + strconv.Itoa(int(dbcommons.DEFAULT_LISTENER_PORT)) + "/" + n.Spec.Sid + return n.Name + ":" + strconv.Itoa(int(dbcommons.CONTAINER_LISTENER_PORT)) + "/" + n.Spec.Sid } return m.Spec.CloneFrom }(), @@ -875,7 +876,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns // Instantiate Service spec from SingleInstanceDatabase spec //############################################################################# func (r *SingleInstanceDatabaseReconciler) instantiateSVCSpec(m *dbapi.SingleInstanceDatabase, - svcName string, listenerPort int32, svcType corev1.ServiceType) *corev1.Service { + svcName string, ports []corev1.ServicePort, svcType corev1.ServiceType) *corev1.Service { svc := &corev1.Service{ TypeMeta: metav1.TypeMeta{ Kind: "Service", @@ -897,24 +898,14 @@ func (r *SingleInstanceDatabaseReconciler) instantiateSVCSpec(m *dbapi.SingleIns }(), }, Spec: corev1.ServiceSpec{ - Ports: []corev1.ServicePort{ - { - Name: "listener", - Port: listenerPort, - Protocol: corev1.ProtocolTCP, - }, - { - Name: "xmldb", - Port: 5500, - Protocol: corev1.ProtocolTCP, - }, - }, + Ports: []corev1.ServicePort{}, Selector: map[string]string{ "app": m.Name, }, Type: svcType, }, } + svc.Spec.Ports = ports // Set SingleInstanceDatabase instance as the owner and controller ctrl.SetControllerReference(m, svc, r.Scheme) return svc @@ -1041,129 +1032,144 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplacePVC(ctx context.Contex } //############################################################################# -// Create a Service for SingleInstanceDatabase +// Create Services for SingleInstanceDatabase //############################################################################# func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Context, req ctrl.Request, m *dbapi.SingleInstanceDatabase) (ctrl.Result, error) { log := r.Log.WithValues("createOrReplaceSVC", req.NamespacedName) - /** If TCPS is enabled, there will be two k8s services: - 1. One service exposing the TCPS port for the user to connect, - 2. One service exposing default listerner port inside the cluster only (for ORDS, APEX installation etc) + /** Two k8s services gets created: + 1. One service is ClusterIP service for cluster only communications on the listener port, + 2. One service is NodePort/LoadBalancer (according to the YAML specs) for users to connect **/ - defaultSvc := &corev1.Service{} - tcpsSvc := &corev1.Service{} - // userSvc would indicate the service which is used to connect to the Database by the database user - var userSvc *corev1.Service + // clusterSvc is the cluster-wide service and extSvc is the external service for the users to connect + clusterSvc := &corev1.Service{} + extSvc := &corev1.Service{} - defaultSvcName := m.Name - tcpsSvcName := m.Name + "-tcps" + clusterSvcName := m.Name + extSvcName := m.Name + "-ext" + + // svcPort is the intended port for extSvc taken from singleinstancedatabase YAML file + // If loadBalancer is true, it would be the listener port otherwise it would be node port + svcPort := func() int32 { + if m.Spec.ServicePort != 0 { + return int32(m.Spec.ServicePort) + } else { + if m.Spec.EnableTCPS { + return dbcommons.CONTAINER_TCPS_PORT + } else { + return dbcommons.CONTAINER_LISTENER_PORT + } + } + }() + + // extSvcTargetPort is used to check the target port of the extSvc when TCPS is enabled/disabled + extSvcTargetPort := dbcommons.CONTAINER_LISTENER_PORT + if m.Spec.EnableTCPS { + extSvcTargetPort = dbcommons.CONTAINER_TCPS_PORT + } // Querying for the K8s service resources - getDefaultSvcErr := r.Get(ctx, types.NamespacedName{Name: defaultSvcName, Namespace: m.Namespace}, defaultSvc) - getTcpsSvcErr := r.Get(ctx, types.NamespacedName{Name: tcpsSvcName, Namespace: m.Namespace}, tcpsSvc) + getClusterSvcErr := r.Get(ctx, types.NamespacedName{Name: clusterSvcName, Namespace: m.Namespace}, clusterSvc) + getExtSvcErr := r.Get(ctx, types.NamespacedName{Name: extSvcName, Namespace: m.Namespace}, extSvc) + + if getClusterSvcErr != nil && apierrors.IsNotFound(getClusterSvcErr) { + // Create a new ClusterIP service + ports := []corev1.ServicePort{{Name: "listener", Port: dbcommons.CONTAINER_LISTENER_PORT, Protocol: corev1.ProtocolTCP}} + svc := r.instantiateSVCSpec(m, clusterSvcName, ports, corev1.ServiceType("ClusterIP")) + log.Info("Creating a new service", "Service.Namespace", svc.Namespace, "Service.Name", svc.Name) + err := r.Create(ctx, svc) + if err != nil { + log.Error(err, "Failed to create new service", "Service.Namespace", svc.Namespace, "Service.Name", svc.Name) + return requeueY, err + } + } else if getClusterSvcErr != nil { + // Error encountered in obtaining the clusterSvc service resource + log.Error(getClusterSvcErr, "Error encountered in obtaining the service", clusterSvcName) + return requeueY, getClusterSvcErr + } - // svcType defines the type of the service as specified in the singleinstancedatabase.yaml file - svcType := corev1.ServiceType("NodePort") + // extSvcType defines the type of the service (LoadBalancer/NodePort) for extSvc as specified in the singleinstancedatabase.yaml file + extSvcType := corev1.ServiceType("NodePort") if m.Spec.LoadBalancer { - svcType = corev1.ServiceType("LoadBalancer") + extSvcType = corev1.ServiceType("LoadBalancer") } - if m.Spec.EnableTCPS { - if getDefaultSvcErr != nil && apierrors.IsNotFound(getDefaultSvcErr) { - // Create a new cluster service - svc := r.instantiateSVCSpec(m, defaultSvcName, dbcommons.DEFAULT_LISTENER_PORT, corev1.ServiceType("ClusterIP")) - log.Info("Creating a new service", "Service.Namespace", svc.Namespace, "Service.Name", svc.Name) - err := r.Create(ctx, svc) - if err != nil { - log.Error(err, "Failed to create new service", "Service.Namespace", svc.Namespace, "Service.Name", svc.Name) - return requeueY, err - } - } else if defaultSvc.Spec.Type != corev1.ServiceType("ClusterIP") { - // Patch the service type - payload := "{\"spec\": {\"type\": \"ClusterIP\"}}" - //Attempt Service patching - log.Info("Patching the service", "Service.Name", defaultSvc.Name, "Target Type", "ClusterIP") - err := dbcommons.PatchService(r.Config, m.Namespace, ctx, req, defaultSvcName, payload) - if err != nil { - log.Error(err, "Failed to patch Service") - return requeueY, err - } - - } + isExtSvcFound := true - // When TCPS is enabled userSvc would point to tcpsSvc - userSvc = tcpsSvc - if getTcpsSvcErr != nil && apierrors.IsNotFound(getTcpsSvcErr) { - // Reset connect strings whenever service is recreated /* - m.Status.ConnectString = dbcommons.ValueUnavailable - m.Status.PdbConnectString = dbcommons.ValueUnavailable - m.Status.OemExpressUrl = dbcommons.ValueUnavailable - // Create a new service - svc := r.instantiateSVCSpec(m, tcpsSvcName, int32(m.Spec.TcpsPort), svcType) - log.Info("Creating a new service", "Service.Namespace", svc.Namespace, "Service.Name", svc.Name) - err := r.Create(ctx, svc) - if err != nil { - log.Error(err, "Failed to create new service", "Service.Namespace", svc.Namespace, "Service.Name", svc.Name) - return requeueY, err - } - userSvc = svc - } else if tcpsSvc.Spec.Type != svcType { - // Patch the service type - payload := fmt.Sprintf("{\"spec\": {\"type\": \"%s\"}}", svcType) - //Attempt Service patching - log.Info("Patching the service", "Service.Name", defaultSvc.Name, "Target Type", svcType) - err := dbcommons.PatchService(r.Config, m.Namespace, ctx, req, tcpsSvcName, payload) + if getExtSvcErr != nil && apierrors.IsNotFound(getExtSvcErr) { + isExtSvcFound = false + } else if getExtSvcErr != nil { + // Error encountered in obtaining the extSvc service resource + log.Error(getExtSvcErr, "Error encountered in obtaining the service", extSvcName) + return requeueY, getExtSvcErr + } else { + // extSvc service found + var extSvcPort int32 + if extSvc.Spec.Type == corev1.ServiceType("LoadBalancer") { + extSvcPort = extSvc.Spec.Ports[1].Port + } else if extSvc.Spec.Type == corev1.ServiceType("NodePort") { + extSvcPort = extSvc.Spec.Ports[1].NodePort + } + + if extSvc.Spec.Type != extSvcType || extSvcPort != svcPort || extSvc.Spec.Ports[1].TargetPort.IntVal != extSvcTargetPort { + // Deleting th service + log.Info("Deleting service", "name", extSvcName) + err := r.Delete(ctx, extSvc) if err != nil { - log.Error(err, "Failed to patch Service") - return requeueY, err + r.Log.Error(err, "Failed to delete service", "name", extSvcName) + return requeueN, err } + isExtSvcFound = false } + } - } else { - // Only one service is required if TCPS is not enabled - - // Reset connect strings whenever service is recreated /* + if !isExtSvcFound { + // Reset connect strings whenever extSvc is recreated + m.Status.Status = dbcommons.StatusUpdating m.Status.ConnectString = dbcommons.ValueUnavailable m.Status.PdbConnectString = dbcommons.ValueUnavailable m.Status.OemExpressUrl = dbcommons.ValueUnavailable - // userSvc would point to the defaultSvc - userSvc = defaultSvc - if getDefaultSvcErr != nil && apierrors.IsNotFound(getDefaultSvcErr) { - // Create a new service with - svc := r.instantiateSVCSpec(m, defaultSvcName, dbcommons.DEFAULT_LISTENER_PORT, svcType) - log.Info("Creating a new service", "Service.Namespace", svc.Namespace, "Service.Name", svc.Name) - err := r.Create(ctx, svc) - if err != nil { - log.Error(err, "Failed to create new service", "Service.Namespace", svc.Namespace, "Service.Name", svc.Name) - return requeueY, err - } - userSvc = svc - } else if defaultSvc.Spec.Type != svcType { - // Patch the service type - payload := fmt.Sprintf("{\"spec\": {\"type\": \"%s\"}}", svcType) - //Attempt Service patching - log.Info("Patching the service", "Service.Name", defaultSvc.Name, "Target Type", svcType) - err := dbcommons.PatchService(r.Config, m.Namespace, ctx, req, defaultSvcName, payload) - if err != nil { - log.Error(err, "Failed to patch Service") - return requeueY, err - } + // New service has to be created + ports := []corev1.ServicePort{ + { + Name: "xmldb", + Port: 5500, + Protocol: corev1.ProtocolTCP, + }, } - if getTcpsSvcErr == nil { - // Delete this tcps service - log.Info("Deleting service", "name", tcpsSvcName) - err := r.Delete(ctx, tcpsSvc) - if err != nil { - r.Log.Error(err, "Failed to delete service", "name", tcpsSvc.Name) - return requeueN, err + if m.Spec.LoadBalancer { + ports = append(ports, corev1.ServicePort{ + Name: "listener", + Protocol: corev1.ProtocolTCP, + Port: svcPort, + TargetPort: intstr.FromInt(int(extSvcTargetPort)), + }) + } else { + ports = append(ports, corev1.ServicePort{ + Name: "listener", + Protocol: corev1.ProtocolTCP, + Port: extSvcTargetPort, + TargetPort: intstr.FromInt(int(extSvcTargetPort)), + }) + if m.Spec.ServicePort != 0 { + ports[1].NodePort = svcPort } } + // Create the service + svc := r.instantiateSVCSpec(m, extSvcName, ports, extSvcType) + log.Info("Creating a new service", "Service.Namespace", svc.Namespace, "Service.Name", svc.Name) + err := r.Create(ctx, svc) + if err != nil { + log.Error(err, "Failed to create new service", "Service.Namespace", svc.Namespace, "Service.Name", svc.Name) + return requeueY, err + } + extSvc = svc } pdbName := strings.ToUpper(m.Spec.Pdbname) @@ -1178,24 +1184,24 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex } if m.Spec.LoadBalancer { - m.Status.ClusterConnectString = userSvc.Name + "." + userSvc.Namespace + ":" + fmt.Sprint(userSvc.Spec.Ports[0].Port) + "/" + strings.ToUpper(sid) - if len(userSvc.Status.LoadBalancer.Ingress) > 0 { + m.Status.ClusterConnectString = extSvc.Name + "." + extSvc.Namespace + ":" + fmt.Sprint(extSvc.Spec.Ports[1].Port) + "/" + strings.ToUpper(sid) + if len(extSvc.Status.LoadBalancer.Ingress) > 0 { // 'lbAddress' will contain the Fully Qualified Hostname of the LB. If the hostname is not available it will contain the IP address of the LB - lbAddress := userSvc.Status.LoadBalancer.Ingress[0].Hostname + lbAddress := extSvc.Status.LoadBalancer.Ingress[0].Hostname if lbAddress == "" { - lbAddress = userSvc.Status.LoadBalancer.Ingress[0].IP + lbAddress = extSvc.Status.LoadBalancer.Ingress[0].IP } - m.Status.ConnectString = lbAddress + ":" + fmt.Sprint(userSvc.Spec.Ports[0].Port) + "/" + strings.ToUpper(sid) - m.Status.PdbConnectString = lbAddress + ":" + fmt.Sprint(userSvc.Spec.Ports[0].Port) + "/" + strings.ToUpper(pdbName) - m.Status.OemExpressUrl = "https://" + lbAddress + ":" + fmt.Sprint(userSvc.Spec.Ports[1].Port) + "/em" + m.Status.ConnectString = lbAddress + ":" + fmt.Sprint(extSvc.Spec.Ports[1].Port) + "/" + strings.ToUpper(sid) + m.Status.PdbConnectString = lbAddress + ":" + fmt.Sprint(extSvc.Spec.Ports[1].Port) + "/" + strings.ToUpper(pdbName) + m.Status.OemExpressUrl = "https://" + lbAddress + ":" + fmt.Sprint(extSvc.Spec.Ports[0].Port) + "/em" } } else { - m.Status.ClusterConnectString = userSvc.Name + "." + userSvc.Namespace + ":" + fmt.Sprint(userSvc.Spec.Ports[0].Port) + "/" + strings.ToUpper(sid) + m.Status.ClusterConnectString = extSvc.Name + "." + extSvc.Namespace + ":" + fmt.Sprint(extSvc.Spec.Ports[1].Port) + "/" + strings.ToUpper(sid) nodeip := dbcommons.GetNodeIp(r, ctx, req) if nodeip != "" { - m.Status.ConnectString = nodeip + ":" + fmt.Sprint(userSvc.Spec.Ports[0].NodePort) + "/" + strings.ToUpper(sid) - m.Status.PdbConnectString = nodeip + ":" + fmt.Sprint(userSvc.Spec.Ports[0].NodePort) + "/" + strings.ToUpper(pdbName) - m.Status.OemExpressUrl = "https://" + nodeip + ":" + fmt.Sprint(userSvc.Spec.Ports[1].NodePort) + "/em" + m.Status.ConnectString = nodeip + ":" + fmt.Sprint(extSvc.Spec.Ports[1].NodePort) + "/" + strings.ToUpper(sid) + m.Status.PdbConnectString = nodeip + ":" + fmt.Sprint(extSvc.Spec.Ports[1].NodePort) + "/" + strings.ToUpper(pdbName) + m.Status.OemExpressUrl = "https://" + nodeip + ":" + fmt.Sprint(extSvc.Spec.Ports[0].NodePort) + "/em" } } @@ -1761,6 +1767,50 @@ func (r *SingleInstanceDatabaseReconciler) deleteWallet(m *dbapi.SingleInstanceD return requeueN, nil } +//############################################################################# +// Updating clientWallet when TCPS is enabled +//############################################################################# +func (r *SingleInstanceDatabaseReconciler) updateClientWallet(m *dbapi.SingleInstanceDatabase, + readyPod corev1.Pod, ctx context.Context, req ctrl.Request) error { + // Updation of tnsnames.ora in clientWallet for HOST and PORT fields + extSvc := &corev1.Service{} + extSvcName := m.Name + "-ext" + getExtSvcErr := r.Get(ctx, types.NamespacedName{Name: extSvcName, Namespace: m.Namespace}, extSvc) + + if getExtSvcErr == nil { + var host string + var port int32 + if m.Spec.LoadBalancer { + if len(extSvc.Status.LoadBalancer.Ingress) > 0 { + host = extSvc.Status.LoadBalancer.Ingress[0].Hostname + if host == "" { + host = extSvc.Status.LoadBalancer.Ingress[0].IP + } + port = extSvc.Spec.Ports[1].Port + } + } else { + host = dbcommons.GetNodeIp(r, ctx, req) + if host != "" { + port = extSvc.Spec.Ports[1].NodePort + } + } + if host != "" && host != m.Status.DbHostname && port != int32(m.Status.DbPort) { + _, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", + ctx, req, false, "bash", "-c", fmt.Sprintf(dbcommons.ClientWalletUpdate, host, port)) + if err != nil { + r.Log.Error(err, err.Error()) + return err + } + m.Status.DbHostname = host + m.Status.DbPort = int(port) + } + } else { + r.Log.Info("Unable to get the service while updating the clientWallet", "Service.Namespace", extSvc.Namespace, "Service.Name", extSvcName) + return getExtSvcErr + } + return nil +} + //############################################################################# // Configuring TCPS //############################################################################# @@ -1775,18 +1825,18 @@ func (r *SingleInstanceDatabaseReconciler) configTcps(m *dbapi.SingleInstanceDat eventMsg := "Enabling TCPS in the database..." r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) - _, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", - ctx, req, false, "bash", "-c", fmt.Sprintf(dbcommons.EnableTcpsCMD, m.Spec.TcpsPort)) + out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", + ctx, req, false, "bash", "-c", dbcommons.EnableTcpsCMD) if err != nil { r.Log.Error(err, err.Error()) eventMsg = "Error encountered in enabling TCPS!" r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) return requeueY, nil } + r.Log.Info("enableTcps Output : \n" + out) // Updating the Status and publishing the event m.Status.CertCreationTimestamp = time.Now().Format(time.RFC3339) m.Status.IsTcpsEnabled = true - m.Status.TcpsPort = m.Spec.TcpsPort r.Status().Update(ctx, m) eventMsg = "TCPS Enabled." @@ -1795,6 +1845,12 @@ func (r *SingleInstanceDatabaseReconciler) configTcps(m *dbapi.SingleInstanceDat // 26040h = 1085 days futureRequeue = ctrl.Result{Requeue: true, RequeueAfter: func() time.Duration { requeueDuration, _ := time.ParseDuration("26040h"); return requeueDuration }()} + // update clientWallet + err = r.updateClientWallet(m, readyPod, ctx, req) + if err != nil { + r.Log.Error(err, "Error in updating tnsnames.ora in clientWallet...") + return requeueY, nil + } } else if !m.Spec.EnableTCPS && m.Status.IsTcpsEnabled { // Disable TCPS m.Status.Status = dbcommons.StatusUpdating @@ -1803,16 +1859,15 @@ func (r *SingleInstanceDatabaseReconciler) configTcps(m *dbapi.SingleInstanceDat eventMsg := "Disabling TCPS in the database..." r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) - _, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", + out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", dbcommons.DisableTcpsCMD) if err != nil { r.Log.Error(err, err.Error()) return requeueY, nil } - + r.Log.Info("disable TCPS Output : \n" + out) // Updating the Status and publishing the event m.Status.CertCreationTimestamp = "" - m.Status.TcpsPort = 0 m.Status.IsTcpsEnabled = false r.Status().Update(ctx, m) @@ -1829,12 +1884,13 @@ func (r *SingleInstanceDatabaseReconciler) configTcps(m *dbapi.SingleInstanceDat m.Status.Status = dbcommons.StatusUpdating r.Status().Update(ctx, m) - _, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", - ctx, req, false, "bash", "-c", fmt.Sprintf(dbcommons.EnableTcpsCMD, m.Spec.TcpsPort)) + out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", + ctx, req, false, "bash", "-c", fmt.Sprintf(dbcommons.EnableTcpsCMD)) if err != nil { r.Log.Error(err, err.Error()) return requeueY, nil } + r.Log.Info("Cert Renewal Output : \n" + out) // Updating the Status and publishing the event m.Status.CertCreationTimestamp = time.Now().Format(time.RFC3339) r.Status().Update(ctx, m) @@ -1845,6 +1901,12 @@ func (r *SingleInstanceDatabaseReconciler) configTcps(m *dbapi.SingleInstanceDat // 26040h = 1085 days futureRequeue = ctrl.Result{Requeue: true, RequeueAfter: func() time.Duration { requeueDuration, _ := time.ParseDuration("26040h"); return requeueDuration }()} } + // update clientWallet + err := r.updateClientWallet(m, readyPod, ctx, req) + if err != nil { + r.Log.Error(err, "Error in updating tnsnames.ora clientWallet...") + return requeueY, nil + } } return requeueN, nil } From 3a49ff3bef970eec64fd25423a1e6bf16435faa3 Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Tue, 23 Aug 2022 13:59:43 +0530 Subject: [PATCH 488/628] Readme changes for TCPS and custom ports Signed-off-by: abhisbyk --- docs/sidb/README.md | 58 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 58 insertions(+) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 6d960194..eb951bf6 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -21,6 +21,8 @@ Oracle Database Operator for Kubernetes (`OraOperator`) includes the Single Inst * [Advanced Database Configurations](#advanced-database-configurations) * [Run Database with Multiple Replicas](#run-database-with-multiple-replicas) * [Setup Database with LoadBalancer](#setup-database-with-loadbalancer) + * [Enabling TCPS Connections](#enabling-tcps-connections) + * [Specifying Custom Ports](#specifying-custom-ports) * [OracleRestDataService Resource](#oraclerestdataservice-resource) * [REST Enable a Database](#rest-enable-a-database) * [Provision ORDS](#provision-ords) @@ -504,6 +506,62 @@ $ kubectl --type=merge -p '{"spec":{"loadBalancer": true}}' patch singleinstance singleinstancedatabase.database.oracle.com/sidb-sample patched ``` +### Enabling TCPS Connections +You can enable TCPS connections in the database by setting the `enableTCPS` field to `true` in the [config/samples/sidb/singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml) file, and applying it using `kubectl apply` command. + +Alternatively, you can use the following command: +```bash +kubectl patch --type=merge singleinstancedatabases.database.oracle.com sidb-sample -p '{"spec": {"enableTCPS": true}}' +``` + +When TCPS connections are enabled, a Kubernetes event is published notifying the same. This event can be seen by any one of the following commands: +```bash +kubectl describe singleinstancedatabases.database.oracle.com sidb-sample + +kubectl get events +``` + +Once TCPS connections are enabled, the database connect string will change accordingly. The TCPS connections status can also be queried by the following command: +```bash +kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.isTcpsEnabled}" +true +``` + +The following steps are required to connect the Database using TCPS: +- You need to download the wallet from the Persistent Volume(PV) attached with the database pod. You can use the following command to get the list of pod: + ```bash + kubectl get po + NAME READY STATUS RESTARTS AGE + sidb-sample-gaqoe 1/1 Running 0 3d14h + ``` +- The location of the wallet inside the pod is as `/opt/oracle/oradata/clientWallet/$ORACLE_SID`. **Let us assume the `ORACLE_SID` is `ORCL1` for the upcoming example commands**. The sample command to download the wallet is as follows: + ```bash + kubectl cp sidb-sample-gaqoe:/opt/oracle/oradata/clientWallet/ORCL1 + ``` +- This wallet includes the sample `tnsnames.ora` and `sqlnet.ora` files. All the TNS entries for the database (corresponding to the CDB and PDB) resides in `tnsnames.ora` file. You need to go inside the downloaded wallet directory and set the `TNS_ADMIN` environment variable to point to the current directory as follows: + ```bash + # After going inside the downloaded wallet directory + export TNS_ADMIN=$(pwd) + ``` + After this, you can connect using SQL\*Plus using the following sample commands: + ```bash + sqlplus sys@ORCL1 as sysdba + + sqlplus system@ORCL1 + ``` +**NOTE:** +- Only database server authentication is supported (no mTLS). +- When TCPS is enabled, a self-signed certificate is generated and stored inside the wallets. For users' convenience, a client-side wallet is generated and stored at `/opt/oracle/oradata/clientWallet/$ORACLE_SID` location. +- The self-signed certificate used with TCPS has validity for 3 years. After the certificate is expired, it will be renewed by the `OraOperator` automatically. You need to download the wallet again after the auto-renewal. + +### Specifying Custom Ports +As mentioned in the section [Setup Database with LoadBalancer](#setup-database-with-loadbalancer), there are two kubernetes services possible for the database: NodePort and LoadBalancer. You can specify which port to use with these services by editing the `servicePort` field of the [config/samples/sidb/singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml) file. + +If the `LoadBalancer` is enabled, the `servicePort` will be the opened load balancer port for database connections. + +In case of `NodePort` service, the `servicePort` will be the opened port on the Kubernetes nodes for database connections. In this case, the allowed range for the `servicePort` is 30000-32767. + + ## OracleRestDataService Resource The Oracle Database Operator creates the `OracleRestDataService` as a custom resource. We will refer `OracleRestDataService` as ORDS from now onwards. Creating ORDS as a custom resource enables the RESTful API access to the Oracle Database in K8s and enables it to be managed as a native Kubernetes object. From 8e59edbd482b8ea529e3653b1d3d18ae2e032c90 Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Mon, 29 Aug 2022 13:01:26 +0530 Subject: [PATCH 489/628] Configurable renewCertDuration and Nodeport svc abrupt deletion fix Signed-off-by: abhisbyk --- .../v1alpha1/singleinstancedatabase_types.go | 4 +- .../singleinstancedatabase_webhook.go | 21 ++++++++ ...se.oracle.com_singleinstancedatabases.yaml | 8 +-- .../samples/sidb/singleinstancedatabase.yaml | 4 ++ .../singleinstancedatabase_controller.go | 50 +++++++++++-------- 5 files changed, 60 insertions(+), 27 deletions(-) diff --git a/apis/database/v1alpha1/singleinstancedatabase_types.go b/apis/database/v1alpha1/singleinstancedatabase_types.go index d5b26e78..a65ed9dd 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_types.go +++ b/apis/database/v1alpha1/singleinstancedatabase_types.go @@ -67,6 +67,7 @@ type SingleInstanceDatabaseSpec struct { ArchiveLog bool `json:"archiveLog,omitempty"` ForceLogging bool `json:"forceLog,omitempty"` EnableTCPS bool `json:"enableTCPS,omitempty"` + CertRenewDuration string `json:"certRenewDuration,omitempty"` CloneFrom string `json:"cloneFrom,omitempty"` ReadinessCheckPeriod int `json:"readinessCheckPeriod,omitempty"` @@ -151,8 +152,7 @@ type SingleInstanceDatabaseStatus struct { // +kubebuilder:default:=false IsTcpsEnabled bool `json:"isTcpsEnabled"` CertCreationTimestamp string `json:"certCreationTimestamp,omitempty"` - DbHostname string `json:"dbHostname,omitempty"` - DbPort int `json:"dbPort,omitempty"` + CertRenewDuration string `json:"certRenewDuration,omitempty"` // +patchMergeKey=type // +patchStrategy=merge diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index 0d450871..0d76ee74 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -40,6 +40,7 @@ package v1alpha1 import ( "strings" + "time" dbcommons "github.com/oracle/oracle-database-operator/commons/database" @@ -244,6 +245,26 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { } } + // Certificate Renew Duration Validation + if r.Spec.CertRenewDuration != "" { + duration, err := time.ParseDuration(r.Spec.CertRenewDuration) + if err != nil { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("certRenewDuration"), r.Spec.CertRenewDuration, + "Please provide valid string to parse the certRenewDuration.")) + } + maxLimit, _ := time.ParseDuration("26000h") + minLimit, _ := time.ParseDuration("1m") + if duration > maxLimit || duration < minLimit { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("certRenewDuration"), r.Spec.CertRenewDuration, + "Please specify certRenewDuration in the range: 1m to 26000h")) + } + } else { + // Setting the default value + r.Spec.CertRenewDuration = "26000h" + } + if len(allErrs) == 0 { return nil } diff --git a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml index b8383ef8..fbbe19ca 100644 --- a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml +++ b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml @@ -77,6 +77,8 @@ spec: type: object archiveLog: type: boolean + certRenewDuration: + type: string charset: type: string cloneFrom: @@ -177,6 +179,8 @@ spec: type: string certCreationTimestamp: type: string + certRenewDuration: + type: string charset: type: string cloneFrom: @@ -263,10 +267,6 @@ spec: datafilesPatched: default: "false" type: string - dbHostname: - type: string - dbPort: - type: integer edition: type: string flashBack: diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index 9adc9601..f861b31c 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -47,6 +47,10 @@ spec: ## Enable TCPS enableTCPS: false + ## Certificate Renewal Duration: The time after which certificates will be renewed if TCPS connections are enabled; can be in hours(h), minutes(m) and seconds(s) + ## Maximum value is 26000h + #certRenewDuration: 26000h + ## NA if cloning from a SourceDB (cloneFrom is set) ## Specify both sgaSize and pgaSize (in MB) or dont specify both ## Specify Non-Zero value to use diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 09b54fdb..67402b76 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -197,22 +197,22 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct } } - // Configure TCPS - result, err = r.configTcps(singleInstanceDatabase, readyPod, ctx, req) + // Update DB config + result, err = r.updateDBConfig(singleInstanceDatabase, readyPod, ctx, req) if result.Requeue { r.Log.Info("Reconcile queued") return result, nil } - // Update DB config - result, err = r.updateDBConfig(singleInstanceDatabase, readyPod, ctx, req) + // Update Init Parameters + result, err = r.updateInitParameters(singleInstanceDatabase, readyPod, ctx, req) if result.Requeue { r.Log.Info("Reconcile queued") return result, nil } - // Update Init Parameters - result, err = r.updateInitParameters(singleInstanceDatabase, readyPod, ctx, req) + // Configure TCPS + result, err = r.configTcps(singleInstanceDatabase, readyPod, ctx, req) if result.Requeue { r.Log.Info("Reconcile queued") return result, nil @@ -1114,7 +1114,7 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex extSvcPort = extSvc.Spec.Ports[1].NodePort } - if extSvc.Spec.Type != extSvcType || extSvcPort != svcPort || extSvc.Spec.Ports[1].TargetPort.IntVal != extSvcTargetPort { + if extSvc.Spec.Type != extSvcType || (m.Spec.ServicePort != 0 && extSvcPort != svcPort) || extSvc.Spec.Ports[1].TargetPort.IntVal != extSvcTargetPort { // Deleting th service log.Info("Deleting service", "name", extSvcName) err := r.Delete(ctx, extSvc) @@ -1794,16 +1794,15 @@ func (r *SingleInstanceDatabaseReconciler) updateClientWallet(m *dbapi.SingleIns port = extSvc.Spec.Ports[1].NodePort } } - if host != "" && host != m.Status.DbHostname && port != int32(m.Status.DbPort) { - _, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", - ctx, req, false, "bash", "-c", fmt.Sprintf(dbcommons.ClientWalletUpdate, host, port)) - if err != nil { - r.Log.Error(err, err.Error()) - return err - } - m.Status.DbHostname = host - m.Status.DbPort = int(port) + + r.Log.Info("Updating the client wallet...") + _, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", + ctx, req, false, "bash", "-c", fmt.Sprintf(dbcommons.ClientWalletUpdate, host, port)) + if err != nil { + r.Log.Error(err, err.Error()) + return err } + } else { r.Log.Info("Unable to get the service while updating the clientWallet", "Service.Namespace", extSvc.Namespace, "Service.Name", extSvcName) return getExtSvcErr @@ -1842,8 +1841,9 @@ func (r *SingleInstanceDatabaseReconciler) configTcps(m *dbapi.SingleInstanceDat eventMsg = "TCPS Enabled." r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) - // 26040h = 1085 days - futureRequeue = ctrl.Result{Requeue: true, RequeueAfter: func() time.Duration { requeueDuration, _ := time.ParseDuration("26040h"); return requeueDuration }()} + requeueDuration, _ := time.ParseDuration(m.Spec.CertRenewDuration) + requeueDuration += func() time.Duration { requeueDuration, _ := time.ParseDuration("1s"); return requeueDuration }() + futureRequeue = ctrl.Result{Requeue: true, RequeueAfter: requeueDuration} // update clientWallet err = r.updateClientWallet(m, readyPod, ctx, req) @@ -1879,7 +1879,7 @@ func (r *SingleInstanceDatabaseReconciler) configTcps(m *dbapi.SingleInstanceDat // Certificates are renewed when 10 days remain for certs expiry certCreationTimestamp, _ := time.Parse(time.RFC3339, m.Status.CertCreationTimestamp) duration := time.Since(certCreationTimestamp) - allowdDuration, _ := time.ParseDuration("26000h") + allowdDuration, _ := time.ParseDuration(m.Spec.CertRenewDuration) if duration > allowdDuration { m.Status.Status = dbcommons.StatusUpdating r.Status().Update(ctx, m) @@ -1898,8 +1898,16 @@ func (r *SingleInstanceDatabaseReconciler) configTcps(m *dbapi.SingleInstanceDat eventMsg := "TCPS Certificates Renewed at time %s," r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg, time.Now().Format(time.RFC3339)) - // 26040h = 1085 days - futureRequeue = ctrl.Result{Requeue: true, RequeueAfter: func() time.Duration { requeueDuration, _ := time.ParseDuration("26040h"); return requeueDuration }()} + requeueDuration, _ := time.ParseDuration(m.Spec.CertRenewDuration) + requeueDuration += func() time.Duration { requeueDuration, _ := time.ParseDuration("1s"); return requeueDuration }() + futureRequeue = ctrl.Result{Requeue: true, RequeueAfter: requeueDuration} + } + if m.Status.CertRenewDuration != m.Spec.CertRenewDuration { + requeueDuration, _ := time.ParseDuration(m.Spec.CertRenewDuration) + requeueDuration += func() time.Duration { requeueDuration, _ := time.ParseDuration("1s"); return requeueDuration }() + futureRequeue = ctrl.Result{Requeue: true, RequeueAfter: requeueDuration} + + m.Status.CertRenewDuration = m.Spec.CertRenewDuration } // update clientWallet err := r.updateClientWallet(m, readyPod, ctx, req) From 04765b2133cd52f4e0d189e3112c1dac6e9e2220 Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Mon, 29 Aug 2022 13:33:07 +0530 Subject: [PATCH 490/628] Cert renewal duration is changed to 26280h (3 yrs) Signed-off-by: abhisbyk --- apis/database/v1alpha1/singleinstancedatabase_webhook.go | 6 +++--- config/samples/sidb/singleinstancedatabase.yaml | 4 ++-- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index 0d76ee74..8dd0b250 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -253,16 +253,16 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { field.Invalid(field.NewPath("spec").Child("certRenewDuration"), r.Spec.CertRenewDuration, "Please provide valid string to parse the certRenewDuration.")) } - maxLimit, _ := time.ParseDuration("26000h") + maxLimit, _ := time.ParseDuration("26280h") minLimit, _ := time.ParseDuration("1m") if duration > maxLimit || duration < minLimit { allErrs = append(allErrs, field.Invalid(field.NewPath("spec").Child("certRenewDuration"), r.Spec.CertRenewDuration, - "Please specify certRenewDuration in the range: 1m to 26000h")) + "Please specify certRenewDuration in the range: 1m to 26280h")) } } else { // Setting the default value - r.Spec.CertRenewDuration = "26000h" + r.Spec.CertRenewDuration = "26280h" } if len(allErrs) == 0 { diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index f861b31c..739779d8 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -48,8 +48,8 @@ spec: enableTCPS: false ## Certificate Renewal Duration: The time after which certificates will be renewed if TCPS connections are enabled; can be in hours(h), minutes(m) and seconds(s) - ## Maximum value is 26000h - #certRenewDuration: 26000h + ## Maximum value is 26280h (3 years), Minimum value is 1m + #certRenewDuration: 26280h ## NA if cloning from a SourceDB (cloneFrom is set) ## Specify both sgaSize and pgaSize (in MB) or dont specify both From d45dc52b4a006a70623c79e48fce9729a1d54de1 Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Mon, 29 Aug 2022 14:11:39 +0530 Subject: [PATCH 491/628] Disabling the cert renewal logic if tcpsCertRenewInterval is not given in the yaml file, default 2 years Signed-off-by: abhisbyk --- .../v1alpha1/singleinstancedatabase_types.go | 22 +++--- .../singleinstancedatabase_webhook.go | 16 ++--- .../samples/sidb/singleinstancedatabase.yaml | 6 +- .../sidb/singleinstancedatabase_tcps.yaml | 68 +++++++++++++++++++ .../singleinstancedatabase_controller.go | 15 ++-- docs/sidb/README.md | 4 +- 6 files changed, 97 insertions(+), 34 deletions(-) create mode 100644 config/samples/sidb/singleinstancedatabase_tcps.yaml diff --git a/apis/database/v1alpha1/singleinstancedatabase_types.go b/apis/database/v1alpha1/singleinstancedatabase_types.go index a65ed9dd..3eb64d07 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_types.go +++ b/apis/database/v1alpha1/singleinstancedatabase_types.go @@ -57,17 +57,17 @@ type SingleInstanceDatabaseSpec struct { // +k8s:openapi-gen=true // +kubebuilder:validation:Pattern=`^[a-zA-Z0-9]+$` // +kubebuilder:validation:MaxLength:=12 - Sid string `json:"sid,omitempty"` - Charset string `json:"charset,omitempty"` - Pdbname string `json:"pdbName,omitempty"` - LoadBalancer bool `json:"loadBalancer,omitempty"` - ServicePort int `json:"servicePort,omitempty"` - ServiceAnnotations map[string]string `json:"serviceAnnotations,omitempty"` - FlashBack bool `json:"flashBack,omitempty"` - ArchiveLog bool `json:"archiveLog,omitempty"` - ForceLogging bool `json:"forceLog,omitempty"` - EnableTCPS bool `json:"enableTCPS,omitempty"` - CertRenewDuration string `json:"certRenewDuration,omitempty"` + Sid string `json:"sid,omitempty"` + Charset string `json:"charset,omitempty"` + Pdbname string `json:"pdbName,omitempty"` + LoadBalancer bool `json:"loadBalancer,omitempty"` + ServicePort int `json:"servicePort,omitempty"` + ServiceAnnotations map[string]string `json:"serviceAnnotations,omitempty"` + FlashBack bool `json:"flashBack,omitempty"` + ArchiveLog bool `json:"archiveLog,omitempty"` + ForceLogging bool `json:"forceLog,omitempty"` + EnableTCPS bool `json:"enableTCPS,omitempty"` + TcpsCertRenewInterval string `json:"tcpsCertRenewInterval,omitempty"` CloneFrom string `json:"cloneFrom,omitempty"` ReadinessCheckPeriod int `json:"readinessCheckPeriod,omitempty"` diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index 8dd0b250..8a269747 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -246,25 +246,21 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { } // Certificate Renew Duration Validation - if r.Spec.CertRenewDuration != "" { - duration, err := time.ParseDuration(r.Spec.CertRenewDuration) + if r.Spec.TcpsCertRenewInterval != "" { + duration, err := time.ParseDuration(r.Spec.TcpsCertRenewInterval) if err != nil { allErrs = append(allErrs, - field.Invalid(field.NewPath("spec").Child("certRenewDuration"), r.Spec.CertRenewDuration, - "Please provide valid string to parse the certRenewDuration.")) + field.Invalid(field.NewPath("spec").Child("tcpsCertRenewInterval"), r.Spec.TcpsCertRenewInterval, + "Please provide valid string to parse the tcpsCertRenewInterval.")) } maxLimit, _ := time.ParseDuration("26280h") minLimit, _ := time.ParseDuration("1m") if duration > maxLimit || duration < minLimit { allErrs = append(allErrs, - field.Invalid(field.NewPath("spec").Child("certRenewDuration"), r.Spec.CertRenewDuration, - "Please specify certRenewDuration in the range: 1m to 26280h")) + field.Invalid(field.NewPath("spec").Child("tcpsCertRenewInterval"), r.Spec.TcpsCertRenewInterval, + "Please specify tcpsCertRenewInterval in the range: 1m to 26280h")) } - } else { - // Setting the default value - r.Spec.CertRenewDuration = "26280h" } - if len(allErrs) == 0 { return nil } diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index 739779d8..edb56e0a 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -47,9 +47,9 @@ spec: ## Enable TCPS enableTCPS: false - ## Certificate Renewal Duration: The time after which certificates will be renewed if TCPS connections are enabled; can be in hours(h), minutes(m) and seconds(s) - ## Maximum value is 26280h (3 years), Minimum value is 1m - #certRenewDuration: 26280h + ## TCPS Certificate Renewal Interval: The time after which TCPS certificate will be renewed if TCPS connections are enabled; can be in hours(h), minutes(m) and seconds(s) + ## Maximum value is 26280h (3 years), Minimum value is 1m; Default value is 17520h (2 years) + certRenewDuration: 17520h ## NA if cloning from a SourceDB (cloneFrom is set) ## Specify both sgaSize and pgaSize (in MB) or dont specify both diff --git a/config/samples/sidb/singleinstancedatabase_tcps.yaml b/config/samples/sidb/singleinstancedatabase_tcps.yaml new file mode 100644 index 00000000..67e9a9bf --- /dev/null +++ b/config/samples/sidb/singleinstancedatabase_tcps.yaml @@ -0,0 +1,68 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +apiVersion: v1 +kind: Secret +metadata: + name: db-admin-secret + namespace: default +type: Opaque +stringData: + # Specify your DB password here + oracle_pwd: + +--- + +apiVersion: database.oracle.com/v1alpha1 +kind: SingleInstanceDatabase +metadata: + # Creates base sidb-sample. Use singleinstancedatabase_clone.yaml for cloning + # and singleinstancedatabase_patch.yaml for patching + name: sidb-sample + namespace: default +spec: + + ## Use only alphanumeric characters for sid + sid: ORCL1 + + ## DB edition. + edition: enterprise + + ## Secret containing SIDB password mapped to secretKey + adminPassword: + secretName: db-admin-secret + + ## DB character set + charset: AL32UTF8 + + ## PDB name + pdbName: orclpdb1 + + ## Enable/Disable ArchiveLog. Should be true to allow DB cloning + archiveLog: true + + ## Enable TCPS + enableTCPS: true + + ## TCPS Certificate Renewal Interval: The time after which TCPS certificate will be renewed if TCPS connections are enabled; can be in hours(h), minutes(m) and seconds(s) + ## Maximum value is 26280h (3 years), Minimum value is 1m; Default value is 17520h (2 years) + certRenewDuration: 17520h + + ## Database image details + image: + pullFrom: container-registry.oracle.com/database/enterprise:latest + pullSecrets: oracle-container-registry-secret + + ## size is the required minimum size of the persistent volume + ## storageClass is specified for automatic volume provisioning + ## accessMode can only accept one of ReadWriteOnce, ReadWriteMany + persistence: + size: 100Gi + ## oci-bv applies to OCI block volumes. Use "standard" storageClass for dynamic provisioning in Minikube. Update as appropriate for other cloud service providers + storageClass: "oci-bv" + accessMode: "ReadWriteOnce" + + ## Count of Database Pods. + replicas: 1 diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 67402b76..af89e906 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1841,7 +1841,7 @@ func (r *SingleInstanceDatabaseReconciler) configTcps(m *dbapi.SingleInstanceDat eventMsg = "TCPS Enabled." r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) - requeueDuration, _ := time.ParseDuration(m.Spec.CertRenewDuration) + requeueDuration, _ := time.ParseDuration(m.Spec.TcpsCertRenewInterval) requeueDuration += func() time.Duration { requeueDuration, _ := time.ParseDuration("1s"); return requeueDuration }() futureRequeue = ctrl.Result{Requeue: true, RequeueAfter: requeueDuration} @@ -1874,12 +1874,11 @@ func (r *SingleInstanceDatabaseReconciler) configTcps(m *dbapi.SingleInstanceDat eventMsg = "TCPS Disabled." r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) - } else if m.Spec.EnableTCPS && m.Status.IsTcpsEnabled { + } else if m.Spec.EnableTCPS && m.Status.IsTcpsEnabled && m.Spec.TcpsCertRenewInterval != "" { // Cert Renewal Logic - // Certificates are renewed when 10 days remain for certs expiry certCreationTimestamp, _ := time.Parse(time.RFC3339, m.Status.CertCreationTimestamp) duration := time.Since(certCreationTimestamp) - allowdDuration, _ := time.ParseDuration(m.Spec.CertRenewDuration) + allowdDuration, _ := time.ParseDuration(m.Spec.TcpsCertRenewInterval) if duration > allowdDuration { m.Status.Status = dbcommons.StatusUpdating r.Status().Update(ctx, m) @@ -1898,16 +1897,16 @@ func (r *SingleInstanceDatabaseReconciler) configTcps(m *dbapi.SingleInstanceDat eventMsg := "TCPS Certificates Renewed at time %s," r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg, time.Now().Format(time.RFC3339)) - requeueDuration, _ := time.ParseDuration(m.Spec.CertRenewDuration) + requeueDuration, _ := time.ParseDuration(m.Spec.TcpsCertRenewInterval) requeueDuration += func() time.Duration { requeueDuration, _ := time.ParseDuration("1s"); return requeueDuration }() futureRequeue = ctrl.Result{Requeue: true, RequeueAfter: requeueDuration} } - if m.Status.CertRenewDuration != m.Spec.CertRenewDuration { - requeueDuration, _ := time.ParseDuration(m.Spec.CertRenewDuration) + if m.Status.CertRenewDuration != m.Spec.TcpsCertRenewInterval { + requeueDuration, _ := time.ParseDuration(m.Spec.TcpsCertRenewInterval) requeueDuration += func() time.Duration { requeueDuration, _ := time.ParseDuration("1s"); return requeueDuration }() futureRequeue = ctrl.Result{Requeue: true, RequeueAfter: requeueDuration} - m.Status.CertRenewDuration = m.Spec.CertRenewDuration + m.Status.CertRenewDuration = m.Spec.TcpsCertRenewInterval } // update clientWallet err := r.updateClientWallet(m, readyPod, ctx, req) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index eb951bf6..f3c5af1d 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -551,8 +551,8 @@ The following steps are required to connect the Database using TCPS: ``` **NOTE:** - Only database server authentication is supported (no mTLS). -- When TCPS is enabled, a self-signed certificate is generated and stored inside the wallets. For users' convenience, a client-side wallet is generated and stored at `/opt/oracle/oradata/clientWallet/$ORACLE_SID` location. -- The self-signed certificate used with TCPS has validity for 3 years. After the certificate is expired, it will be renewed by the `OraOperator` automatically. You need to download the wallet again after the auto-renewal. +- When TCPS is enabled, a self-signed certificate is generated and stored inside the wallets. For users' convenience, a client-side wallet is generated and stored at `/opt/oracle/oradata/clientWallet/$ORACLE_SID` location in the pod. +- The self-signed certificate used with TCPS has validity for 2 years. After the certificate is expired, it will be renewed by the `OraOperator` automatically. You need to download the wallet again after the auto-renewal. ### Specifying Custom Ports As mentioned in the section [Setup Database with LoadBalancer](#setup-database-with-loadbalancer), there are two kubernetes services possible for the database: NodePort and LoadBalancer. You can specify which port to use with these services by editing the `servicePort` field of the [config/samples/sidb/singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml) file. From 5af166beac850a7a93ea7e1817379b22b24aff64 Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Mon, 29 Aug 2022 15:44:17 +0530 Subject: [PATCH 492/628] Updating attr name in the yaml file Signed-off-by: abhisbyk --- config/samples/sidb/singleinstancedatabase.yaml | 2 +- config/samples/sidb/singleinstancedatabase_tcps.yaml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index edb56e0a..efcbcee1 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -49,7 +49,7 @@ spec: ## TCPS Certificate Renewal Interval: The time after which TCPS certificate will be renewed if TCPS connections are enabled; can be in hours(h), minutes(m) and seconds(s) ## Maximum value is 26280h (3 years), Minimum value is 1m; Default value is 17520h (2 years) - certRenewDuration: 17520h + tcpsCertRenewInterval: 17520h ## NA if cloning from a SourceDB (cloneFrom is set) ## Specify both sgaSize and pgaSize (in MB) or dont specify both diff --git a/config/samples/sidb/singleinstancedatabase_tcps.yaml b/config/samples/sidb/singleinstancedatabase_tcps.yaml index 67e9a9bf..22d3695a 100644 --- a/config/samples/sidb/singleinstancedatabase_tcps.yaml +++ b/config/samples/sidb/singleinstancedatabase_tcps.yaml @@ -48,7 +48,7 @@ spec: ## TCPS Certificate Renewal Interval: The time after which TCPS certificate will be renewed if TCPS connections are enabled; can be in hours(h), minutes(m) and seconds(s) ## Maximum value is 26280h (3 years), Minimum value is 1m; Default value is 17520h (2 years) - certRenewDuration: 17520h + tcpsCertRenewInterval: 17520h ## Database image details image: From dbf8f5f5bb95dbfcf0868c9830844b0519040972 Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Mon, 29 Aug 2022 16:01:34 +0530 Subject: [PATCH 493/628] Yaml file comment corrections Signed-off-by: abhisbyk --- config/samples/sidb/singleinstancedatabase.yaml | 4 +++- config/samples/sidb/singleinstancedatabase_tcps.yaml | 4 +++- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index efcbcee1..2a3c9cae 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -47,8 +47,10 @@ spec: ## Enable TCPS enableTCPS: false - ## TCPS Certificate Renewal Interval: The time after which TCPS certificate will be renewed if TCPS connections are enabled; can be in hours(h), minutes(m) and seconds(s) + ## TCPS Certificate Renewal Interval: The time after which TCPS certificate will be renewed if TCPS connections are enabled. + ## tcpsCertRenewInterval can be in hours(h), minutes(m) and seconds(s); e.g. 17520h, 8760h etc. ## Maximum value is 26280h (3 years), Minimum value is 1m; Default value is 17520h (2 years) + ## If this field is commented out/removed from the yaml, it will disable the auto-renewal feature for TCPS certificate tcpsCertRenewInterval: 17520h ## NA if cloning from a SourceDB (cloneFrom is set) diff --git a/config/samples/sidb/singleinstancedatabase_tcps.yaml b/config/samples/sidb/singleinstancedatabase_tcps.yaml index 22d3695a..8089c85c 100644 --- a/config/samples/sidb/singleinstancedatabase_tcps.yaml +++ b/config/samples/sidb/singleinstancedatabase_tcps.yaml @@ -46,8 +46,10 @@ spec: ## Enable TCPS enableTCPS: true - ## TCPS Certificate Renewal Interval: The time after which TCPS certificate will be renewed if TCPS connections are enabled; can be in hours(h), minutes(m) and seconds(s) + ## TCPS Certificate Renewal Interval: The time after which TCPS certificate will be renewed if TCPS connections are enabled. + ## tcpsCertRenewInterval can be in hours(h), minutes(m) and seconds(s); e.g. 17520h, 8760h etc. ## Maximum value is 26280h (3 years), Minimum value is 1m; Default value is 17520h (2 years) + ## If this field is commented out/removed from the yaml, it will disable the auto-renewal feature for TCPS certificate tcpsCertRenewInterval: 17520h ## Database image details From 4655cf5cf09cafc833cf500674ee68af5dd3e7d8 Mon Sep 17 00:00:00 2001 From: Abhishek Kumar Date: Thu, 1 Sep 2022 12:45:49 +0000 Subject: [PATCH 494/628] Additional TCPS features --- .../v1alpha1/singleinstancedatabase_types.go | 42 +++-- .../singleinstancedatabase_webhook.go | 18 +- commons/database/constants.go | 3 + ...se.oracle.com_singleinstancedatabases.yaml | 25 ++- .../samples/sidb/singleinstancedatabase.yaml | 10 +- .../oraclerestdataservice_controller.go | 3 +- .../singleinstancedatabase_controller.go | 162 +++++++++++++----- docs/sidb/README.md | 13 +- 8 files changed, 202 insertions(+), 74 deletions(-) diff --git a/apis/database/v1alpha1/singleinstancedatabase_types.go b/apis/database/v1alpha1/singleinstancedatabase_types.go index 3eb64d07..7edd9af8 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_types.go +++ b/apis/database/v1alpha1/singleinstancedatabase_types.go @@ -61,7 +61,8 @@ type SingleInstanceDatabaseSpec struct { Charset string `json:"charset,omitempty"` Pdbname string `json:"pdbName,omitempty"` LoadBalancer bool `json:"loadBalancer,omitempty"` - ServicePort int `json:"servicePort,omitempty"` + ListenerPort int `json:"listenerPort,omitempty"` + TcpsListenerPort int `json:"tcpsListenerPort,omitempty"` ServiceAnnotations map[string]string `json:"serviceAnnotations,omitempty"` FlashBack bool `json:"flashBack,omitempty"` ArchiveLog bool `json:"archiveLog,omitempty"` @@ -131,28 +132,31 @@ type SingleInstanceDatabaseStatus struct { DatafilesPatched string `json:"datafilesPatched,omitempty"` ConnectString string `json:"connectString,omitempty"` ClusterConnectString string `json:"clusterConnectString,omitempty"` + TcpsConnectString string `json:"tcpsConnectString,omitempty"` StandbyDatabases map[string]string `json:"standbyDatabases,omitempty"` // +kubebuilder:default:="false" - DatafilesCreated string `json:"datafilesCreated,omitempty"` - Sid string `json:"sid,omitempty"` - Edition string `json:"edition,omitempty"` - Charset string `json:"charset,omitempty"` - Pdbname string `json:"pdbName,omitempty"` - InitSgaSize int `json:"initSgaSize,omitempty"` - InitPgaSize int `json:"initPgaSize,omitempty"` - CloneFrom string `json:"cloneFrom,omitempty"` - FlashBack string `json:"flashBack,omitempty"` - ArchiveLog string `json:"archiveLog,omitempty"` - ForceLogging string `json:"forceLog,omitempty"` - OemExpressUrl string `json:"oemExpressUrl,omitempty"` - OrdsReference string `json:"ordsReference,omitempty"` - PdbConnectString string `json:"pdbConnectString,omitempty"` - ApexInstalled bool `json:"apexInstalled,omitempty"` - PrebuiltDB bool `json:"prebuiltDB,omitempty"` + DatafilesCreated string `json:"datafilesCreated,omitempty"` + Sid string `json:"sid,omitempty"` + Edition string `json:"edition,omitempty"` + Charset string `json:"charset,omitempty"` + Pdbname string `json:"pdbName,omitempty"` + InitSgaSize int `json:"initSgaSize,omitempty"` + InitPgaSize int `json:"initPgaSize,omitempty"` + CloneFrom string `json:"cloneFrom,omitempty"` + FlashBack string `json:"flashBack,omitempty"` + ArchiveLog string `json:"archiveLog,omitempty"` + ForceLogging string `json:"forceLog,omitempty"` + OemExpressUrl string `json:"oemExpressUrl,omitempty"` + OrdsReference string `json:"ordsReference,omitempty"` + PdbConnectString string `json:"pdbConnectString,omitempty"` + TcpsPdbConnectString string `json:"tcpsPdbConnectString,omitempty"` + ApexInstalled bool `json:"apexInstalled,omitempty"` + PrebuiltDB bool `json:"prebuiltDB,omitempty"` // +kubebuilder:default:=false IsTcpsEnabled bool `json:"isTcpsEnabled"` CertCreationTimestamp string `json:"certCreationTimestamp,omitempty"` - CertRenewDuration string `json:"certRenewDuration,omitempty"` + CertRenewInterval string `json:"certRenewInterval,omitempty"` + ClientWalletLoc string `json:"clientWalletLoc,omitempty"` // +patchMergeKey=type // +patchStrategy=merge @@ -172,6 +176,8 @@ type SingleInstanceDatabaseStatus struct { // +kubebuilder:printcolumn:JSONPath=".status.role",name="Role",type="string",priority=1 // +kubebuilder:printcolumn:JSONPath=".status.releaseUpdate",name="Version",type="string" // +kubebuilder:printcolumn:JSONPath=".status.connectString",name="Connect Str",type="string" +// +kubebuilder:printcolumn:JSONPath=".status.tcpsConnectString",name="TCPS Connect Str",type="string" +// +kubebuilder:printcolumn:JSONPath=".status.tcpsPdbConnectString",name="TCPS Pdb Connect Str",type="string", priority=1 // +kubebuilder:printcolumn:JSONPath=".status.pdbConnectString",name="Pdb Connect Str",type="string",priority=1 // +kubebuilder:printcolumn:JSONPath=".status.oemExpressUrl",name="Oem Express Url",type="string" diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index 8a269747..af52efc2 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -235,14 +235,24 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { } } - // servicePort validation + // servicePort and tcpServicePort validation if !r.Spec.LoadBalancer { // NodePort service is expected. In this case servicePort should be in range 30000-32767 - if r.Spec.ServicePort != 0 && (r.Spec.ServicePort < 30000 || r.Spec.ServicePort > 32767) { + if r.Spec.ListenerPort != 0 && (r.Spec.ListenerPort < 30000 || r.Spec.ListenerPort > 32767) { allErrs = append(allErrs, - field.Invalid(field.NewPath("spec").Child("servicePort"), r.Spec.ServicePort, - "servicePort should be in 30000-32767 range.")) + field.Invalid(field.NewPath("spec").Child("listenerPort"), r.Spec.ListenerPort, + "listenerPort should be in 30000-32767 range.")) } + if r.Spec.TcpsListenerPort != 0 && (r.Spec.TcpsListenerPort < 30000 || r.Spec.TcpsListenerPort > 32767) { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("tcpsListenerPort"), r.Spec.TcpsListenerPort, + "tcpsListenerPort should be in 30000-32767 range.")) + } + } + if r.Spec.ListenerPort != 0 && r.Spec.TcpsListenerPort != 0 && r.Spec.ListenerPort == r.Spec.TcpsListenerPort { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("tcpsListenerPort"), r.Spec.TcpsListenerPort, + "listenerPort and tcpsListenerPort can not be equal.")) } // Certificate Renew Duration Validation diff --git a/commons/database/constants.go b/commons/database/constants.go index 3aeb66f7..9ce63500 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -495,3 +495,6 @@ const DisableTcpsCMD string = "$ORACLE_BASE/$CONFIG_TCPS_FILE disable" // TCPS clientWallet update command const ClientWalletUpdate string = "sed -i -e 's/HOST.*$/HOST=%s)/g' -e 's/PORT.*$/PORT=%d)/g' ${ORACLE_BASE}/oradata/clientWallet/${ORACLE_SID}/tnsnames.ora" + +// TCPS clientWallet location +const ClientWalletLocation string = "/opt/oracle/oradata/clientWallet/%s" diff --git a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml index fbbe19ca..5acf2b04 100644 --- a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml +++ b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml @@ -33,6 +33,13 @@ spec: - jsonPath: .status.connectString name: Connect Str type: string + - jsonPath: .status.tcpsConnectString + name: TCPS Connect Str + type: string + - jsonPath: .status.tcpsPdbConnectString + name: TCPS Pdb Connect Str + priority: 1 + type: string - jsonPath: .status.pdbConnectString name: Pdb Connect Str priority: 1 @@ -77,8 +84,6 @@ spec: type: object archiveLog: type: boolean - certRenewDuration: - type: string charset: type: string cloneFrom: @@ -122,6 +127,8 @@ spec: sgaTarget: type: integer type: object + listenerPort: + type: integer loadBalancer: type: boolean nodeSelector: @@ -158,14 +165,16 @@ spec: additionalProperties: type: string type: object - servicePort: - type: integer sid: description: SID must be alphanumeric (no special characters, only a-z, A-Z, 0-9), and no longer than 12 characters. maxLength: 12 pattern: ^[a-zA-Z0-9]+$ type: string + tcpsCertRenewInterval: + type: string + tcpsListenerPort: + type: integer required: - image type: object @@ -179,10 +188,12 @@ spec: type: string certCreationTimestamp: type: string - certRenewDuration: + certRenewInterval: type: string charset: type: string + clientWalletLoc: + type: string cloneFrom: type: string clusterConnectString: @@ -338,6 +349,10 @@ spec: type: object status: type: string + tcpsConnectString: + type: string + tcpsPdbConnectString: + type: string required: - isTcpsEnabled - persistence diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index 2a3c9cae..dbfbe8e9 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -89,9 +89,13 @@ spec: ## if loadBalService : false, service type = "NodePort" else "LoadBalancer" loadBalancer: false - ## If loadBalancer is enabled, the servicePort is the load balancer port number - ## If loadBalancer is disabled, the servicePort is the NodePort(should be in range 30000-32767) - #servicePort: 30001 + ## 'listenerPort' and 'tcpsListenerPort' fields customizes port cofigurations for normal and tcps database listeners + ## 'tcpsListenerPort' will come in effect only when 'enableTCPS' field is set + ## If loadBalancer is enabled, the listenerPort, tcpsListenerPort will be the load balancer ports + ## If loadBalancer is disabled, the listenerPort, tcpsListenerPort will be the node ports(should be in range 30000-32767) + ## If enableTCPS is set, and listenerPort is commented/not mentioned in the YAML file, only TCPS endpoint will be exposed + #listenerPort: 30001 + #tcpsListenerPort: 30002 ## Service Annotations (Cloud provider specific), for configuring the service (e.g. private LoadBalancer service) #serviceAnnotations: diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 319b0d1e..267759aa 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -582,7 +582,7 @@ func (r *OracleRestDataServiceReconciler) instantiatePodSpec(m *dbapi.OracleRest { Name: "init-permissions", Image: m.Spec.Image.PullFrom, - Command: []string{"/bin/sh", "-c", fmt.Sprintf("chown %d:%d /opt/oracle/ords/config/ords", int(dbcommons.ORACLE_UID), int(dbcommons.DBA_GUID))}, + Command: []string{"/bin/sh", "-c", fmt.Sprintf("chown %d:%d /opt/oracle/ords/config/ords || true", int(dbcommons.ORACLE_UID), int(dbcommons.DBA_GUID))}, SecurityContext: &corev1.SecurityContext{ // User ID 0 means, root user RunAsUser: func() *int64 { i := int64(0); return &i }(), @@ -728,6 +728,7 @@ func (r *OracleRestDataServiceReconciler) instantiatePodSpec(m *dbapi.OracleRest SecurityContext: &corev1.PodSecurityContext{ RunAsUser: func() *int64 { i := int64(dbcommons.ORACLE_UID); return &i }(), RunAsGroup: func() *int64 { i := int64(dbcommons.DBA_GUID); return &i }(), + FSGroup: func() *int64 { i := int64(dbcommons.DBA_GUID); return &i }(), }, ImagePullSecrets: []corev1.LocalObjectReference{ diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index af89e906..46210899 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -567,7 +567,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns initContainers = append(initContainers, corev1.Container{ Name: "init-permissions", Image: m.Spec.Image.PullFrom, - Command: []string{"/bin/sh", "-c", fmt.Sprintf("chown %d:%d /opt/oracle/oradata", int(dbcommons.ORACLE_UID), int(dbcommons.ORACLE_GUID))}, + Command: []string{"/bin/sh", "-c", fmt.Sprintf("chown %d:%d /opt/oracle/oradata || true", int(dbcommons.ORACLE_UID), int(dbcommons.ORACLE_GUID))}, SecurityContext: &corev1.SecurityContext{ // User ID 0 means, root user RunAsUser: func() *int64 { i := int64(0); return &i }(), @@ -857,6 +857,10 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns i := int64(dbcommons.ORACLE_GUID) return &i }(), + FSGroup: func() *int64 { + i := int64(dbcommons.ORACLE_GUID) + return &i + }(), }, ImagePullSecrets: []corev1.LocalObjectReference{ { @@ -1040,7 +1044,7 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex log := r.Log.WithValues("createOrReplaceSVC", req.NamespacedName) /** Two k8s services gets created: - 1. One service is ClusterIP service for cluster only communications on the listener port, + 1. One service is ClusterIP service for cluster only communications on the listener port 1521, 2. One service is NodePort/LoadBalancer (according to the YAML specs) for users to connect **/ @@ -1051,25 +1055,25 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex clusterSvcName := m.Name extSvcName := m.Name + "-ext" - // svcPort is the intended port for extSvc taken from singleinstancedatabase YAML file + // svcPort is the intended port for extSvc taken from singleinstancedatabase YAML file for normal database connection // If loadBalancer is true, it would be the listener port otherwise it would be node port svcPort := func() int32 { - if m.Spec.ServicePort != 0 { - return int32(m.Spec.ServicePort) + if m.Spec.ListenerPort != 0 { + return int32(m.Spec.ListenerPort) } else { - if m.Spec.EnableTCPS { - return dbcommons.CONTAINER_TCPS_PORT - } else { - return dbcommons.CONTAINER_LISTENER_PORT - } + return dbcommons.CONTAINER_LISTENER_PORT } }() - // extSvcTargetPort is used to check the target port of the extSvc when TCPS is enabled/disabled - extSvcTargetPort := dbcommons.CONTAINER_LISTENER_PORT - if m.Spec.EnableTCPS { - extSvcTargetPort = dbcommons.CONTAINER_TCPS_PORT - } + // tcpsSvcPort is the intended port for extSvc taken from singleinstancedatabase YAML file for TCPS connection + // If loadBalancer is true, it would be the listener port otherwise it would be node port + tcpsSvcPort := func() int32 { + if m.Spec.TcpsListenerPort != 0 { + return int32(m.Spec.TcpsListenerPort) + } else { + return dbcommons.CONTAINER_TCPS_PORT + } + }() // Querying for the K8s service resources getClusterSvcErr := r.Get(ctx, types.NamespacedName{Name: clusterSvcName, Namespace: m.Namespace}, clusterSvc) @@ -1106,15 +1110,39 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex log.Error(getExtSvcErr, "Error encountered in obtaining the service", extSvcName) return requeueY, getExtSvcErr } else { - // extSvc service found - var extSvcPort int32 - if extSvc.Spec.Type == corev1.ServiceType("LoadBalancer") { - extSvcPort = extSvc.Spec.Ports[1].Port - } else if extSvc.Spec.Type == corev1.ServiceType("NodePort") { - extSvcPort = extSvc.Spec.Ports[1].NodePort + // Counting required number of ports in extSvc + requiredPorts := 2 + if m.Spec.EnableTCPS && m.Spec.ListenerPort != 0 { + requiredPorts = 3 + } + + // Obtaining all ports of the extSvc k8s service + var targetPorts []int32 + for _, port := range extSvc.Spec.Ports { + if extSvc.Spec.Type == corev1.ServiceType("LoadBalancer") { + targetPorts = append(targetPorts, port.Port) + } else if extSvc.Spec.Type == corev1.ServiceType("NodePort") { + targetPorts = append(targetPorts, port.NodePort) + } + } + + deleteSvc := false + if extSvc.Spec.Type != extSvcType || len(extSvc.Spec.Ports) != requiredPorts { + deleteSvc = true + } else { + // Match for the ports + if (m.Spec.ListenerPort != 0 && svcPort != targetPorts[1]) || (m.Spec.TcpsListenerPort != 0 && tcpsSvcPort != targetPorts[len(targetPorts)-1]) { + deleteSvc = true + } + if m.Spec.ListenerPort == 0 && m.Spec.TcpsListenerPort == 0 { + if (m.Spec.EnableTCPS && extSvc.Spec.Ports[1].TargetPort.IntVal != dbcommons.CONTAINER_TCPS_PORT) || + (!m.Spec.EnableTCPS && extSvc.Spec.Ports[1].TargetPort.IntVal != dbcommons.CONTAINER_LISTENER_PORT) { + deleteSvc = true + } + } } - if extSvc.Spec.Type != extSvcType || (m.Spec.ServicePort != 0 && extSvcPort != svcPort) || extSvc.Spec.Ports[1].TargetPort.IntVal != extSvcTargetPort { + if deleteSvc { // Deleting th service log.Info("Deleting service", "name", extSvcName) err := r.Delete(ctx, extSvc) @@ -1132,6 +1160,8 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex m.Status.ConnectString = dbcommons.ValueUnavailable m.Status.PdbConnectString = dbcommons.ValueUnavailable m.Status.OemExpressUrl = dbcommons.ValueUnavailable + m.Status.TcpsConnectString = dbcommons.ValueUnavailable + m.Status.TcpsPdbConnectString = dbcommons.ValueUnavailable // New service has to be created ports := []corev1.ServicePort{ @@ -1143,21 +1173,56 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex } if m.Spec.LoadBalancer { - ports = append(ports, corev1.ServicePort{ - Name: "listener", - Protocol: corev1.ProtocolTCP, - Port: svcPort, - TargetPort: intstr.FromInt(int(extSvcTargetPort)), - }) + if m.Spec.EnableTCPS { + if m.Spec.ListenerPort != 0 { + ports = append(ports, corev1.ServicePort{ + Name: "listener", + Protocol: corev1.ProtocolTCP, + Port: svcPort, + TargetPort: intstr.FromInt(int(dbcommons.CONTAINER_LISTENER_PORT)), + }) + } + ports = append(ports, corev1.ServicePort{ + Name: "listener-tcps", + Protocol: corev1.ProtocolTCP, + Port: tcpsSvcPort, + TargetPort: intstr.FromInt(int(dbcommons.CONTAINER_TCPS_PORT)), + }) + } else { + ports = append(ports, corev1.ServicePort{ + Name: "listener", + Protocol: corev1.ProtocolTCP, + Port: svcPort, + TargetPort: intstr.FromInt(int(dbcommons.CONTAINER_LISTENER_PORT)), + }) + } } else { - ports = append(ports, corev1.ServicePort{ - Name: "listener", - Protocol: corev1.ProtocolTCP, - Port: extSvcTargetPort, - TargetPort: intstr.FromInt(int(extSvcTargetPort)), - }) - if m.Spec.ServicePort != 0 { - ports[1].NodePort = svcPort + if m.Spec.EnableTCPS { + if m.Spec.ListenerPort != 0 { + ports = append(ports, corev1.ServicePort{ + Name: "listener", + Protocol: corev1.ProtocolTCP, + Port: dbcommons.CONTAINER_LISTENER_PORT, + NodePort: svcPort, + }) + } + ports = append(ports, corev1.ServicePort{ + Name: "listener-tcps", + Protocol: corev1.ProtocolTCP, + Port: dbcommons.CONTAINER_TCPS_PORT, + }) + if m.Spec.TcpsListenerPort != 0 { + ports[len(ports)-1].NodePort = tcpsSvcPort + } + } else { + ports = append(ports, corev1.ServicePort{ + Name: "listener", + Protocol: corev1.ProtocolTCP, + Port: dbcommons.CONTAINER_LISTENER_PORT, + }) + if m.Spec.ListenerPort != 0 { + ports[len(ports)-1].NodePort = svcPort + } } } @@ -1194,6 +1259,10 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex m.Status.ConnectString = lbAddress + ":" + fmt.Sprint(extSvc.Spec.Ports[1].Port) + "/" + strings.ToUpper(sid) m.Status.PdbConnectString = lbAddress + ":" + fmt.Sprint(extSvc.Spec.Ports[1].Port) + "/" + strings.ToUpper(pdbName) m.Status.OemExpressUrl = "https://" + lbAddress + ":" + fmt.Sprint(extSvc.Spec.Ports[0].Port) + "/em" + if m.Spec.EnableTCPS { + m.Status.TcpsConnectString = lbAddress + ":" + fmt.Sprint(extSvc.Spec.Ports[len(extSvc.Spec.Ports)-1].Port) + "/" + strings.ToUpper(sid) + m.Status.TcpsPdbConnectString = lbAddress + ":" + fmt.Sprint(extSvc.Spec.Ports[len(extSvc.Spec.Ports)-1].Port) + "/" + strings.ToUpper(pdbName) + } } } else { m.Status.ClusterConnectString = extSvc.Name + "." + extSvc.Namespace + ":" + fmt.Sprint(extSvc.Spec.Ports[1].Port) + "/" + strings.ToUpper(sid) @@ -1202,6 +1271,10 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex m.Status.ConnectString = nodeip + ":" + fmt.Sprint(extSvc.Spec.Ports[1].NodePort) + "/" + strings.ToUpper(sid) m.Status.PdbConnectString = nodeip + ":" + fmt.Sprint(extSvc.Spec.Ports[1].NodePort) + "/" + strings.ToUpper(pdbName) m.Status.OemExpressUrl = "https://" + nodeip + ":" + fmt.Sprint(extSvc.Spec.Ports[0].NodePort) + "/em" + if m.Spec.EnableTCPS { + m.Status.TcpsConnectString = nodeip + ":" + fmt.Sprint(extSvc.Spec.Ports[len(extSvc.Spec.Ports)-1].NodePort) + "/" + strings.ToUpper(sid) + m.Status.TcpsPdbConnectString = nodeip + ":" + fmt.Sprint(extSvc.Spec.Ports[len(extSvc.Spec.Ports)-1].NodePort) + "/" + strings.ToUpper(pdbName) + } } } @@ -1786,12 +1859,12 @@ func (r *SingleInstanceDatabaseReconciler) updateClientWallet(m *dbapi.SingleIns if host == "" { host = extSvc.Status.LoadBalancer.Ingress[0].IP } - port = extSvc.Spec.Ports[1].Port + port = extSvc.Spec.Ports[len(extSvc.Spec.Ports)-1].Port } } else { host = dbcommons.GetNodeIp(r, ctx, req) if host != "" { - port = extSvc.Spec.Ports[1].NodePort + port = extSvc.Spec.Ports[len(extSvc.Spec.Ports)-1].NodePort } } @@ -1836,6 +1909,7 @@ func (r *SingleInstanceDatabaseReconciler) configTcps(m *dbapi.SingleInstanceDat // Updating the Status and publishing the event m.Status.CertCreationTimestamp = time.Now().Format(time.RFC3339) m.Status.IsTcpsEnabled = true + m.Status.ClientWalletLoc = fmt.Sprintf(dbcommons.ClientWalletLocation, m.Spec.Sid) r.Status().Update(ctx, m) eventMsg = "TCPS Enabled." @@ -1869,6 +1943,7 @@ func (r *SingleInstanceDatabaseReconciler) configTcps(m *dbapi.SingleInstanceDat // Updating the Status and publishing the event m.Status.CertCreationTimestamp = "" m.Status.IsTcpsEnabled = false + m.Status.ClientWalletLoc = "" r.Status().Update(ctx, m) eventMsg = "TCPS Disabled." @@ -1901,13 +1976,20 @@ func (r *SingleInstanceDatabaseReconciler) configTcps(m *dbapi.SingleInstanceDat requeueDuration += func() time.Duration { requeueDuration, _ := time.ParseDuration("1s"); return requeueDuration }() futureRequeue = ctrl.Result{Requeue: true, RequeueAfter: requeueDuration} } - if m.Status.CertRenewDuration != m.Spec.TcpsCertRenewInterval { + if m.Status.CertRenewInterval != m.Spec.TcpsCertRenewInterval { requeueDuration, _ := time.ParseDuration(m.Spec.TcpsCertRenewInterval) requeueDuration += func() time.Duration { requeueDuration, _ := time.ParseDuration("1s"); return requeueDuration }() futureRequeue = ctrl.Result{Requeue: true, RequeueAfter: requeueDuration} - m.Status.CertRenewDuration = m.Spec.TcpsCertRenewInterval + m.Status.CertRenewInterval = m.Spec.TcpsCertRenewInterval + } + // update clientWallet + err := r.updateClientWallet(m, readyPod, ctx, req) + if err != nil { + r.Log.Error(err, "Error in updating tnsnames.ora clientWallet...") + return requeueY, nil } + } else if m.Spec.EnableTCPS && m.Status.IsTcpsEnabled && m.Spec.TcpsCertRenewInterval == "" { // update clientWallet err := r.updateClientWallet(m, readyPod, ctx, req) if err != nil { diff --git a/docs/sidb/README.md b/docs/sidb/README.md index f3c5af1d..8ad98e4f 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -555,11 +555,18 @@ The following steps are required to connect the Database using TCPS: - The self-signed certificate used with TCPS has validity for 2 years. After the certificate is expired, it will be renewed by the `OraOperator` automatically. You need to download the wallet again after the auto-renewal. ### Specifying Custom Ports -As mentioned in the section [Setup Database with LoadBalancer](#setup-database-with-loadbalancer), there are two kubernetes services possible for the database: NodePort and LoadBalancer. You can specify which port to use with these services by editing the `servicePort` field of the [config/samples/sidb/singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml) file. +As mentioned in the section [Setup Database with LoadBalancer](#setup-database-with-loadbalancer), there are two kubernetes services possible for the database: NodePort and LoadBalancer. You can specify which port to use with these services by editing the `listenerPort` and `tcpsListenerPort` fields of the [config/samples/sidb/singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml) file. -If the `LoadBalancer` is enabled, the `servicePort` will be the opened load balancer port for database connections. +`listenerPort` is intended for normal database connections. Similarly, `tcpsListenerPort` is intended for TCPS database connections. -In case of `NodePort` service, the `servicePort` will be the opened port on the Kubernetes nodes for database connections. In this case, the allowed range for the `servicePort` is 30000-32767. +If the `LoadBalancer` is enabled, the `listenerPort`, and `tcpsListenerPort` will be the opened ports on the Load Balancer for normal and TCPS database connections respectively. + +In case of `NodePort` service, `listenerPort`, and `tcpsListenerPort` will be the opened ports on the Kubernetes nodes for for normal and TCPS database connections respectively. In this case, the allowed range for the `listenerPort`, and `tcpsListenerPort` is 30000-32767. + +**NOTE:** +- `listenerPort` and `tcpsListenerPort` can not have same values. +- `tcpsListenerPort` will come into effect only when TCPS connections are enabled (i.e. `enableTCPS` field is set in [config/samples/sidb/singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml) file). +- If TCPS connections are enabled, and `listenerPort` is commented/removed in the [config/samples/sidb/singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml) file, only TCPS endpoint will be exposed. ## OracleRestDataService Resource From aa37794a964ddd190d06b0630f687d27320ba49b Mon Sep 17 00:00:00 2001 From: Abhishek Kumar Date: Thu, 8 Sep 2022 06:01:25 +0000 Subject: [PATCH 495/628] Bugfixes related to TCPS changes --- apis/database/v1alpha1/pdb_webhook.go | 5 +- .../v1alpha1/singleinstancedatabase_types.go | 2 +- .../singleinstancedatabase_webhook.go | 5 + commons/database/constants.go | 19 ++++ commons/database/utils.go | 2 +- ...se.oracle.com_singleinstancedatabases.yaml | 8 +- .../singleinstancedatabase_controller.go | 95 +++++++++++++++++-- docs/sidb/README.md | 4 +- 8 files changed, 120 insertions(+), 20 deletions(-) diff --git a/apis/database/v1alpha1/pdb_webhook.go b/apis/database/v1alpha1/pdb_webhook.go index 573a8729..1fb2f4cf 100644 --- a/apis/database/v1alpha1/pdb_webhook.go +++ b/apis/database/v1alpha1/pdb_webhook.go @@ -38,9 +38,8 @@ /* MODIFIED (MM/DD/YY) ** rcitton 07/14/22 - 33822886 -*/ - - + */ + package v1alpha1 import ( diff --git a/apis/database/v1alpha1/singleinstancedatabase_types.go b/apis/database/v1alpha1/singleinstancedatabase_types.go index 7edd9af8..cfd22425 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_types.go +++ b/apis/database/v1alpha1/singleinstancedatabase_types.go @@ -177,8 +177,8 @@ type SingleInstanceDatabaseStatus struct { // +kubebuilder:printcolumn:JSONPath=".status.releaseUpdate",name="Version",type="string" // +kubebuilder:printcolumn:JSONPath=".status.connectString",name="Connect Str",type="string" // +kubebuilder:printcolumn:JSONPath=".status.tcpsConnectString",name="TCPS Connect Str",type="string" -// +kubebuilder:printcolumn:JSONPath=".status.tcpsPdbConnectString",name="TCPS Pdb Connect Str",type="string", priority=1 // +kubebuilder:printcolumn:JSONPath=".status.pdbConnectString",name="Pdb Connect Str",type="string",priority=1 +// +kubebuilder:printcolumn:JSONPath=".status.tcpsPdbConnectString",name="TCPS Pdb Connect Str",type="string", priority=1 // +kubebuilder:printcolumn:JSONPath=".status.oemExpressUrl",name="Oem Express Url",type="string" // SingleInstanceDatabase is the Schema for the singleinstancedatabases API diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index af52efc2..f2127e1e 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -254,6 +254,11 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { field.Invalid(field.NewPath("spec").Child("tcpsListenerPort"), r.Spec.TcpsListenerPort, "listenerPort and tcpsListenerPort can not be equal.")) } + if r.Spec.EnableTCPS && r.Spec.TcpsListenerPort == 0 && r.Spec.ListenerPort == 1522 { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("listenerPort"), r.Spec.ListenerPort, + "listenerPort can not be 1522 as the default port for tcpsListenerPort is 1522.")) + } // Certificate Renew Duration Validation if r.Spec.TcpsCertRenewInterval != "" { diff --git a/commons/database/constants.go b/commons/database/constants.go index 9ce63500..712f9819 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -498,3 +498,22 @@ const ClientWalletUpdate string = "sed -i -e 's/HOST.*$/HOST=%s)/g' -e 's/PORT.* // TCPS clientWallet location const ClientWalletLocation string = "/opt/oracle/oradata/clientWallet/%s" + +// Service Patch Payloads +// Three port payload: one OEM express, one TCP and one TCPS port +const ThreePortPayload string = "{\"spec\": { \"ports\": [{\"name\": \"xmldb\", \"port\": 5500, \"protocol\": \"TCP\"},{%s},{%s}]}}" + +// Two port payload: one OEM express, one TCP/TCPS port +const TwoPortPayload string = "{\"spec\": { \"ports\": [{\"name\": \"xmldb\", \"port\": 5500, \"protocol\": \"TCP\"},{%s}]}}" + +// Payload section for listener port +const LsnrPort string = "\"name\": \"listener\", \"protocol\": \"TCP\", \"port\": %d" + +// Payload section for listener node port +const LsnrNodePort string = "\"name\": \"listener\", \"protocol\": \"TCP\", \"port\": 1521, \"nodePort\": %d" + +// Payload section for TCPS port +const TcpsPort string = "\"name\": \"listener-tcps\", \"protocol\": \"TCP\", \"port\": %d" + +// Payload section for TCPS node port +const TcpsNodePort string = "\"name\": \"listener-tcps\", \"protocol\": \"TCP\", \"port\": 1522, \"nodePort\": %d" diff --git a/commons/database/utils.go b/commons/database/utils.go index 100b532c..91fdfb09 100644 --- a/commons/database/utils.go +++ b/commons/database/utils.go @@ -686,6 +686,6 @@ func PatchService(config *rest.Config, namespace string, ctx context.Context, re // Trying to patch the service resource using Strategic Merge strategy log.Info("Patching the service", "Service", svcName) - _, err = client.CoreV1().Services(namespace).Patch(ctx, svcName, types.StrategicMergePatchType, []byte(payload), metav1.PatchOptions{}) + _, err = client.CoreV1().Services(namespace).Patch(ctx, svcName, types.MergePatchType, []byte(payload), metav1.PatchOptions{}) return err } diff --git a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml index 5acf2b04..493800d6 100644 --- a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml +++ b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml @@ -36,14 +36,14 @@ spec: - jsonPath: .status.tcpsConnectString name: TCPS Connect Str type: string - - jsonPath: .status.tcpsPdbConnectString - name: TCPS Pdb Connect Str - priority: 1 - type: string - jsonPath: .status.pdbConnectString name: Pdb Connect Str priority: 1 type: string + - jsonPath: .status.tcpsPdbConnectString + name: TCPS Pdb Connect Str + priority: 1 + type: string - jsonPath: .status.oemExpressUrl name: Oem Express Url type: string diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 46210899..7e432188 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1127,17 +1127,38 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex } deleteSvc := false - if extSvc.Spec.Type != extSvcType || len(extSvc.Spec.Ports) != requiredPorts { + patchSvc := false + if extSvc.Spec.Type != extSvcType { deleteSvc = true } else { - // Match for the ports - if (m.Spec.ListenerPort != 0 && svcPort != targetPorts[1]) || (m.Spec.TcpsListenerPort != 0 && tcpsSvcPort != targetPorts[len(targetPorts)-1]) { - deleteSvc = true + // Conditions to determine whether to patch or not + if len(extSvc.Spec.Ports) != requiredPorts { + patchSvc = true } - if m.Spec.ListenerPort == 0 && m.Spec.TcpsListenerPort == 0 { - if (m.Spec.EnableTCPS && extSvc.Spec.Ports[1].TargetPort.IntVal != dbcommons.CONTAINER_TCPS_PORT) || - (!m.Spec.EnableTCPS && extSvc.Spec.Ports[1].TargetPort.IntVal != dbcommons.CONTAINER_LISTENER_PORT) { - deleteSvc = true + + if (m.Spec.ListenerPort != 0 && svcPort != targetPorts[1]) || (m.Spec.EnableTCPS && m.Spec.TcpsListenerPort != 0 && tcpsSvcPort != targetPorts[len(targetPorts)-1]) { + patchSvc = true + } + + if m.Spec.LoadBalancer { + if m.Spec.EnableTCPS { + if m.Spec.TcpsListenerPort == 0 && tcpsSvcPort != targetPorts[len(targetPorts)-1] { + patchSvc = true + } + } else { + if m.Spec.ListenerPort == 0 && svcPort != targetPorts[1] { + patchSvc = true + } + } + } else { + if m.Spec.EnableTCPS { + if m.Spec.TcpsListenerPort == 0 && tcpsSvcPort != extSvc.Spec.Ports[len(targetPorts)-1].TargetPort.IntVal { + patchSvc = true + } + } else { + if m.Spec.ListenerPort == 0 && svcPort != extSvc.Spec.Ports[1].TargetPort.IntVal { + patchSvc = true + } } } } @@ -1145,13 +1166,69 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex if deleteSvc { // Deleting th service log.Info("Deleting service", "name", extSvcName) - err := r.Delete(ctx, extSvc) + // Setting GracePeriodSeconds to 0 for instant deletion + delOpts := &client.DeleteOptions{} + var gracePeriod client.GracePeriodSeconds = 0 + gracePeriod.ApplyToDelete(delOpts) + + err := r.Delete(ctx, extSvc, delOpts) if err != nil { r.Log.Error(err, "Failed to delete service", "name", extSvcName) return requeueN, err } isExtSvcFound = false } + + if patchSvc { + // Reset connect strings whenever patching happens + m.Status.Status = dbcommons.StatusUpdating + m.Status.ConnectString = dbcommons.ValueUnavailable + m.Status.PdbConnectString = dbcommons.ValueUnavailable + m.Status.OemExpressUrl = dbcommons.ValueUnavailable + m.Status.TcpsConnectString = dbcommons.ValueUnavailable + m.Status.TcpsPdbConnectString = dbcommons.ValueUnavailable + + // Payload formation for patching the service + var payload string + if m.Spec.LoadBalancer { + if m.Spec.EnableTCPS { + if m.Spec.ListenerPort != 0 { + payload = fmt.Sprintf(dbcommons.ThreePortPayload, fmt.Sprintf(dbcommons.LsnrPort, svcPort), fmt.Sprintf(dbcommons.TcpsPort, tcpsSvcPort)) + } else { + payload = fmt.Sprintf(dbcommons.TwoPortPayload, fmt.Sprintf(dbcommons.TcpsPort, tcpsSvcPort)) + } + } else { + payload = fmt.Sprintf(dbcommons.TwoPortPayload, fmt.Sprintf(dbcommons.LsnrPort, svcPort)) + } + } else { + if m.Spec.EnableTCPS { + if m.Spec.ListenerPort != 0 && m.Spec.TcpsListenerPort != 0 { + payload = fmt.Sprintf(dbcommons.ThreePortPayload, fmt.Sprintf(dbcommons.LsnrNodePort, svcPort), fmt.Sprintf(dbcommons.TcpsNodePort, tcpsSvcPort)) + } else if m.Spec.ListenerPort != 0 { + payload = fmt.Sprintf(dbcommons.ThreePortPayload, fmt.Sprintf(dbcommons.LsnrNodePort, svcPort), fmt.Sprintf(dbcommons.TcpsPort, tcpsSvcPort)) + } else if m.Spec.TcpsListenerPort != 0 { + payload = fmt.Sprintf(dbcommons.TwoPortPayload, fmt.Sprintf(dbcommons.TcpsNodePort, tcpsSvcPort)) + } else { + payload = fmt.Sprintf(dbcommons.TwoPortPayload, fmt.Sprintf(dbcommons.TcpsPort, tcpsSvcPort)) + } + } else { + if m.Spec.ListenerPort != 0 { + payload = fmt.Sprintf(dbcommons.TwoPortPayload, fmt.Sprintf(dbcommons.LsnrNodePort, svcPort)) + } else { + payload = fmt.Sprintf(dbcommons.TwoPortPayload, fmt.Sprintf(dbcommons.LsnrPort, svcPort)) + } + } + } + + //Attemp Service Pathcing + log.Info("Patching the service", "Service.Name", extSvc.Name, "payload", payload) + err := dbcommons.PatchService(r.Config, m.Namespace, ctx, req, extSvcName, payload) + if err != nil { + log.Error(err, "Failed to patch Service") + } + //Requeue once after patching + return requeueY, err + } } if !isExtSvcFound { diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 8ad98e4f..29a1fe6b 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -63,8 +63,8 @@ To obtain a quick database status, use the following command as an example: ```sh $ kubectl get singleinstancedatabase sidb-sample -NAME EDITION STATUS VERSION CONNECT STR OEM EXPRESS URL -sidb-sample Enterprise Healthy 19.3.0.0.0 10.0.25.54:1521/ORCLCDB https://10.0.25.54:5500/em +NAME EDITION STATUS VERSION CONNECT STR TCPS CONNECT STR OEM EXPRESS URL +sidb-sample Enterprise Healthy 19.3.0.0.0 10.0.25.54:1521/ORCL1 Unavailable https://10.0.25.54:5500/em ``` #### Detailed Status From ea6244597e2af11d3e12e0a6183dbe662d080d66 Mon Sep 17 00:00:00 2001 From: Abhishek Kumar Date: Fri, 9 Sep 2022 11:00:24 +0000 Subject: [PATCH 496/628] Bugfixes for 34578825 and 34582087 --- .../v1alpha1/singleinstancedatabase_webhook.go | 18 ++++++++++-------- commons/database/constants.go | 4 ++-- 2 files changed, 12 insertions(+), 10 deletions(-) diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index f2127e1e..e2671082 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -243,25 +243,27 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { field.Invalid(field.NewPath("spec").Child("listenerPort"), r.Spec.ListenerPort, "listenerPort should be in 30000-32767 range.")) } - if r.Spec.TcpsListenerPort != 0 && (r.Spec.TcpsListenerPort < 30000 || r.Spec.TcpsListenerPort > 32767) { + if r.Spec.EnableTCPS && r.Spec.TcpsListenerPort != 0 && (r.Spec.TcpsListenerPort < 30000 || r.Spec.TcpsListenerPort > 32767) { allErrs = append(allErrs, field.Invalid(field.NewPath("spec").Child("tcpsListenerPort"), r.Spec.TcpsListenerPort, "tcpsListenerPort should be in 30000-32767 range.")) } + } else { + // LoadBalancer Service is expected. + if r.Spec.EnableTCPS && r.Spec.TcpsListenerPort == 0 && r.Spec.ListenerPort == 1522 { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("listenerPort"), r.Spec.ListenerPort, + "listenerPort can not be 1522 as the default port for tcpsListenerPort is 1522.")) + } } - if r.Spec.ListenerPort != 0 && r.Spec.TcpsListenerPort != 0 && r.Spec.ListenerPort == r.Spec.TcpsListenerPort { + if r.Spec.EnableTCPS && r.Spec.ListenerPort != 0 && r.Spec.TcpsListenerPort != 0 && r.Spec.ListenerPort == r.Spec.TcpsListenerPort { allErrs = append(allErrs, field.Invalid(field.NewPath("spec").Child("tcpsListenerPort"), r.Spec.TcpsListenerPort, "listenerPort and tcpsListenerPort can not be equal.")) } - if r.Spec.EnableTCPS && r.Spec.TcpsListenerPort == 0 && r.Spec.ListenerPort == 1522 { - allErrs = append(allErrs, - field.Invalid(field.NewPath("spec").Child("listenerPort"), r.Spec.ListenerPort, - "listenerPort can not be 1522 as the default port for tcpsListenerPort is 1522.")) - } // Certificate Renew Duration Validation - if r.Spec.TcpsCertRenewInterval != "" { + if r.Spec.EnableTCPS && r.Spec.TcpsCertRenewInterval != "" { duration, err := time.ParseDuration(r.Spec.TcpsCertRenewInterval) if err != nil { allErrs = append(allErrs, diff --git a/commons/database/constants.go b/commons/database/constants.go index 712f9819..9df367b0 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -507,13 +507,13 @@ const ThreePortPayload string = "{\"spec\": { \"ports\": [{\"name\": \"xmldb\", const TwoPortPayload string = "{\"spec\": { \"ports\": [{\"name\": \"xmldb\", \"port\": 5500, \"protocol\": \"TCP\"},{%s}]}}" // Payload section for listener port -const LsnrPort string = "\"name\": \"listener\", \"protocol\": \"TCP\", \"port\": %d" +const LsnrPort string = "\"name\": \"listener\", \"protocol\": \"TCP\", \"port\": %d, \"targetPort\": 1521" // Payload section for listener node port const LsnrNodePort string = "\"name\": \"listener\", \"protocol\": \"TCP\", \"port\": 1521, \"nodePort\": %d" // Payload section for TCPS port -const TcpsPort string = "\"name\": \"listener-tcps\", \"protocol\": \"TCP\", \"port\": %d" +const TcpsPort string = "\"name\": \"listener-tcps\", \"protocol\": \"TCP\", \"port\": %d, \"targetPort\": 1522" // Payload section for TCPS node port const TcpsNodePort string = "\"name\": \"listener-tcps\", \"protocol\": \"TCP\", \"port\": 1522, \"nodePort\": %d" From 6ba14c75f3df583b38bca2cf05888beb246da602 Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Tue, 13 Sep 2022 11:06:35 +0530 Subject: [PATCH 497/628] Sample YAML file changes Signed-off-by: abhisbyk --- .../sidb/oraclerestdataservice_apex.yaml | 29 ------------- .../sidb/oraclerestdataservice_create.yaml | 12 ------ .../sidb/oraclerestdataservice_secrets.yaml | 33 +++++++++++++++ .../sidb/singleinstancedatabase_create.yaml | 12 ------ .../sidb/singleinstancedatabase_express.yaml | 12 ------ .../singleinstancedatabase_prebuiltdb.yaml | 12 ------ .../sidb/singleinstancedatabase_secrets.yaml | 41 +++++++++++++++++++ docs/sidb/README.md | 33 +++++++++++++-- 8 files changed, 104 insertions(+), 80 deletions(-) create mode 100644 config/samples/sidb/oraclerestdataservice_secrets.yaml create mode 100644 config/samples/sidb/singleinstancedatabase_secrets.yaml diff --git a/config/samples/sidb/oraclerestdataservice_apex.yaml b/config/samples/sidb/oraclerestdataservice_apex.yaml index f56437e6..a619799d 100644 --- a/config/samples/sidb/oraclerestdataservice_apex.yaml +++ b/config/samples/sidb/oraclerestdataservice_apex.yaml @@ -3,35 +3,6 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # -apiVersion: v1 -kind: Secret -metadata: - name: ords-secret - namespace: default -type: Opaque -stringData: - ## Specify your ORDS password here - oracle_pwd: - ---- - -apiVersion: v1 -kind: Secret -metadata: - name: apex-secret - namespace: default -type: Opaque -stringData: - ## Specify your APEX password here - ## This password should complete the following requirements: - ## 1. Contain at least 6 characters. - ## 2. Contain at least one numeric character (0123456789). - ## 3. Contain at least one punctuation character (!"#$%&()``*+,-/:;?_). - ## 4. Contain at least one uppercase alphabetic character. - oracle_pwd: - ---- - apiVersion: database.oracle.com/v1alpha1 kind: OracleRestDataService metadata: diff --git a/config/samples/sidb/oraclerestdataservice_create.yaml b/config/samples/sidb/oraclerestdataservice_create.yaml index dfba1626..ed7493c1 100644 --- a/config/samples/sidb/oraclerestdataservice_create.yaml +++ b/config/samples/sidb/oraclerestdataservice_create.yaml @@ -3,18 +3,6 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # -apiVersion: v1 -kind: Secret -metadata: - name: ords-secret - namespace: default -type: Opaque -stringData: - ## Specify your ORDS password here - oracle_pwd: - ---- - apiVersion: database.oracle.com/v1alpha1 kind: OracleRestDataService metadata: diff --git a/config/samples/sidb/oraclerestdataservice_secrets.yaml b/config/samples/sidb/oraclerestdataservice_secrets.yaml new file mode 100644 index 00000000..75b9f93a --- /dev/null +++ b/config/samples/sidb/oraclerestdataservice_secrets.yaml @@ -0,0 +1,33 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +## ORDS password secret +apiVersion: v1 +kind: Secret +metadata: + name: ords-secret + namespace: default +type: Opaque +stringData: + ## Specify your ORDS password here + oracle_pwd: + +--- + +## APEX password secret +apiVersion: v1 +kind: Secret +metadata: + name: apex-secret + namespace: default +type: Opaque +stringData: + ## Specify your APEX password here + ## This password should complete the following requirements: + ## 1. Contain at least 6 characters. + ## 2. Contain at least one numeric character (0123456789). + ## 3. Contain at least one punctuation character (!"#$%&()``*+,-/:;?_). + ## 4. Contain at least one uppercase alphabetic character. + oracle_pwd: \ No newline at end of file diff --git a/config/samples/sidb/singleinstancedatabase_create.yaml b/config/samples/sidb/singleinstancedatabase_create.yaml index 6435d850..413c1e3e 100644 --- a/config/samples/sidb/singleinstancedatabase_create.yaml +++ b/config/samples/sidb/singleinstancedatabase_create.yaml @@ -3,18 +3,6 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # -apiVersion: v1 -kind: Secret -metadata: - name: db-admin-secret - namespace: default -type: Opaque -stringData: - # Specify your DB password here - oracle_pwd: - ---- - apiVersion: database.oracle.com/v1alpha1 kind: SingleInstanceDatabase metadata: diff --git a/config/samples/sidb/singleinstancedatabase_express.yaml b/config/samples/sidb/singleinstancedatabase_express.yaml index 61a0b404..4c79ba6d 100644 --- a/config/samples/sidb/singleinstancedatabase_express.yaml +++ b/config/samples/sidb/singleinstancedatabase_express.yaml @@ -3,18 +3,6 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # -apiVersion: v1 -kind: Secret -metadata: - name: xedb-admin-secret - namespace: default -type: Opaque -stringData: - ## Specify your DB password here - oracle_pwd: - ---- - apiVersion: database.oracle.com/v1alpha1 kind: SingleInstanceDatabase metadata: diff --git a/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml b/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml index ddd9b581..092a1897 100644 --- a/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml +++ b/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml @@ -3,18 +3,6 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # -apiVersion: v1 -kind: Secret -metadata: - name: prebuiltdb-admin-secret - namespace: default -type: Opaque -stringData: - ## Specify your DB password here - oracle_pwd: - ---- - apiVersion: database.oracle.com/v1alpha1 kind: SingleInstanceDatabase metadata: diff --git a/config/samples/sidb/singleinstancedatabase_secrets.yaml b/config/samples/sidb/singleinstancedatabase_secrets.yaml new file mode 100644 index 00000000..6504a6ac --- /dev/null +++ b/config/samples/sidb/singleinstancedatabase_secrets.yaml @@ -0,0 +1,41 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +## Database Admin Password Secret +apiVersion: v1 +kind: Secret +metadata: + name: db-admin-secret + namespace: default +type: Opaque +stringData: + ## Specify your DB password here + oracle_pwd: + +--- + +## Oracle Database XE Admin password secret +apiVersion: v1 +kind: Secret +metadata: + name: prebuiltdb-admin-secret + namespace: default +type: Opaque +stringData: + ## Specify your DB password here + oracle_pwd: + +--- + +## Prebuilt-Database Admin password secret +apiVersion: v1 +kind: Secret +metadata: + name: xedb-admin-secret + namespace: default +type: Opaque +stringData: + ## Specify your DB password here + oracle_pwd: diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 91e52927..f7542abe 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -134,9 +134,9 @@ The `adminPassword` field in the above `singleinstancedatabase.yaml` file refers Create this secret using the following command as an example: - kubectl create secret generic admin-secret --from-literal=oracle_pwd= + kubectl create secret generic db-admin-secret --from-literal=oracle_pwd= -This command creates a secret named `admin-secret`, with the key `oracle_pwd` mapped to the actual password specified in the command. +This command creates a secret named `db-admin-secret`, with the key `oracle_pwd` mapped to the actual password specified in the command. ### Create a Database @@ -198,6 +198,13 @@ This command pulls the XE image uploaded on the [Oracle Container Registry](http - For XE database, only single replica mode (i.e. `replicas: 1`) is supported. - For XE database, you **cannot change** the init parameters i.e. `cpuCount, processes, sgaTarget or pgaAggregateTarget`. +#### Additional Information +You are required to specify the database admin password secret in the corresponding YAML file. The default values mentioned in the `adminPassword.secretName` fields of [singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml), [singleinstancedatabase_prebuiltdb.yaml](../../config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml) and [singleinstancedatabase_express.yaml](../../config/samples/sidb/singleinstancedatabase_express.yaml) files are `db-admin-secret`, `prebuiltdb-admin-secret`, and `xedb-admin-secret` respectively. You can create these secrets manually by using the sample command mentioned in the [Template YAML](#template-yaml) section. Otherwise, you can create these secrets at once by filling the passwords in the **[singleinstancedatabase_secrets.yaml](../../config/samples/sidb/singleinstancedatabase_secrets.yaml)** file and applying it using the command below: + +```bash +kubectl apply -f singleinstancedatabase_secrets.yaml +``` + ### Connecting to Database Creating a new database instance takes a while. When the `status` column returns the response `Healthy`, the Database is open for connections. @@ -587,6 +594,20 @@ $ kubectl apply -f oraclerestdataservice_create.yaml ``` After this command completes, ORDS is installed in the container database (CDB) of the Single Instance Database. +##### NOTE: +You are required to specify the ORDS secret in the [oraclerestdataservice_create.yaml](../../config/samples/sidb/oraclerestdataservice_create.yaml) file. The default value mentioned in the `adminPassword.secretName` field is `ords-secret`. You can create this secret manually by using the following command: + +```bash +kubectl create secret generic ords-secret --from-literal=oracle_pwd= +``` + +Otherwise, you can create this secret together with the APEX secret by filling the passwords in the **[oraclerestdataservice_secrets.yaml](../../config/samples/sidb/oraclerestdataservice_secrets.yaml)** file and applying it using the command below: + +```bash +kubectl apply -f singleinstancedatabase_secrets.yaml +``` +The APEX secret created above, will be used while [installing APEX](#apex-installation). + #### Creation Status Creating a new ORDS instance takes a while. To check the status of the ORDS instance, use the following command: @@ -743,7 +764,13 @@ The `OraOperator` facilitates installation of APEX in the database and also conf kubectl apply -f oraclerestdataservice_apex.yaml -* The APEX Password is used as a common password for `APEX_PUBLIC_USER, APEX_REST_PUBLIC_USER, APEX_LISTENER` and Apex administrator (username: `ADMIN`) mapped to secretKey. +* The APEX Password is used as a common password for `APEX_PUBLIC_USER, APEX_REST_PUBLIC_USER, APEX_LISTENER` and Apex administrator (username: `ADMIN`) mapped to secretKey. You can create APEX secret using the following command: + + ```bash + kubectl create secret generic apex-secret --from-literal=oracle_pwd= + ``` + Please refer [this](#note) section for APEX secret creation using the **[oraclerestdataservice_secrets.yaml](../../config/samples/sidb/oraclerestdataservice_secrets.yaml)** file. + * The status of ORDS turns to `Updating` during APEX configuration, and changes to `Healthy` after successful configuration. You can also check status by using the following command: From f9a27db4c6329e7e2bba12d766576bfd51941942 Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Tue, 13 Sep 2022 12:01:04 +0530 Subject: [PATCH 498/628] Readme corrections Signed-off-by: abhisbyk --- config/samples/sidb/oraclerestdataservice_secrets.yaml | 2 +- docs/sidb/README.md | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/config/samples/sidb/oraclerestdataservice_secrets.yaml b/config/samples/sidb/oraclerestdataservice_secrets.yaml index 75b9f93a..f5092a42 100644 --- a/config/samples/sidb/oraclerestdataservice_secrets.yaml +++ b/config/samples/sidb/oraclerestdataservice_secrets.yaml @@ -30,4 +30,4 @@ stringData: ## 2. Contain at least one numeric character (0123456789). ## 3. Contain at least one punctuation character (!"#$%&()``*+,-/:;?_). ## 4. Contain at least one uppercase alphabetic character. - oracle_pwd: \ No newline at end of file + oracle_pwd: diff --git a/docs/sidb/README.md b/docs/sidb/README.md index f7542abe..875e64a5 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -199,7 +199,7 @@ This command pulls the XE image uploaded on the [Oracle Container Registry](http - For XE database, you **cannot change** the init parameters i.e. `cpuCount, processes, sgaTarget or pgaAggregateTarget`. #### Additional Information -You are required to specify the database admin password secret in the corresponding YAML file. The default values mentioned in the `adminPassword.secretName` fields of [singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml), [singleinstancedatabase_prebuiltdb.yaml](../../config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml) and [singleinstancedatabase_express.yaml](../../config/samples/sidb/singleinstancedatabase_express.yaml) files are `db-admin-secret`, `prebuiltdb-admin-secret`, and `xedb-admin-secret` respectively. You can create these secrets manually by using the sample command mentioned in the [Template YAML](#template-yaml) section. Otherwise, you can create these secrets at once by filling the passwords in the **[singleinstancedatabase_secrets.yaml](../../config/samples/sidb/singleinstancedatabase_secrets.yaml)** file and applying it using the command below: +You are required to specify the database admin password secret in the corresponding YAML file. The default values mentioned in the `adminPassword.secretName` fields of [singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml), [singleinstancedatabase_prebuiltdb.yaml](../../config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml) and [singleinstancedatabase_express.yaml](../../config/samples/sidb/singleinstancedatabase_express.yaml) files are `db-admin-secret`, `prebuiltdb-admin-secret`, and `xedb-admin-secret` respectively. You can create these secrets manually by using the sample command mentioned in the [Template YAML](#template-yaml) section. Alternatively, you can create these secrets by filling the passwords in the **[singleinstancedatabase_secrets.yaml](../../config/samples/sidb/singleinstancedatabase_secrets.yaml)** file and applying it using the command below: ```bash kubectl apply -f singleinstancedatabase_secrets.yaml @@ -601,7 +601,7 @@ You are required to specify the ORDS secret in the [oraclerestdataservice_create kubectl create secret generic ords-secret --from-literal=oracle_pwd= ``` -Otherwise, you can create this secret together with the APEX secret by filling the passwords in the **[oraclerestdataservice_secrets.yaml](../../config/samples/sidb/oraclerestdataservice_secrets.yaml)** file and applying it using the command below: +Alternatively, you can create this secret together with the APEX secret by filling the passwords in the **[oraclerestdataservice_secrets.yaml](../../config/samples/sidb/oraclerestdataservice_secrets.yaml)** file and applying it using the command below: ```bash kubectl apply -f singleinstancedatabase_secrets.yaml From 5c4917a2e757f0ce0b1f4252ed95f72552c9b733 Mon Sep 17 00:00:00 2001 From: abhisbyk Date: Tue, 13 Sep 2022 12:18:44 +0530 Subject: [PATCH 499/628] Readme corrections Signed-off-by: abhisbyk --- docs/sidb/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 875e64a5..d6e7e8fa 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -601,7 +601,7 @@ You are required to specify the ORDS secret in the [oraclerestdataservice_create kubectl create secret generic ords-secret --from-literal=oracle_pwd= ``` -Alternatively, you can create this secret together with the APEX secret by filling the passwords in the **[oraclerestdataservice_secrets.yaml](../../config/samples/sidb/oraclerestdataservice_secrets.yaml)** file and applying it using the command below: +Alternatively, you can create this secret and the APEX secret by filling the passwords in the **[oraclerestdataservice_secrets.yaml](../../config/samples/sidb/oraclerestdataservice_secrets.yaml)** file and applying it using the command below: ```bash kubectl apply -f singleinstancedatabase_secrets.yaml From f47cdc1027023c964220a9aed621af9d38bdb06e Mon Sep 17 00:00:00 2001 From: matteo malvezzi Date: Thu, 15 Sep 2022 15:30:30 +0200 Subject: [PATCH 500/628] ords 22.2 https implementation --- README.md | 6 +- apis/database/v1alpha1/cdb_types.go | 15 + apis/database/v1alpha1/cdb_webhook.go | 6 + apis/database/v1alpha1/pdb_types.go | 18 +- .../v1alpha1/zz_generated.deepcopy.go | 85 +++ .../{onpremdb => multitenant}/cdb.yaml | 0 .../{onpremdb => multitenant}/cdb_secret.yaml | 0 .../{onpremdb => multitenant}/pdb_clone.yaml | 0 .../{onpremdb => multitenant}/pdb_create.yaml | 0 .../{onpremdb => multitenant}/pdb_delete.yaml | 0 .../{onpremdb => multitenant}/pdb_modify.yaml | 0 .../{onpremdb => multitenant}/pdb_plug.yaml | 0 .../{onpremdb => multitenant}/pdb_secret.yaml | 0 .../{onpremdb => multitenant}/pdb_unplug.yaml | 0 controllers/database/cdb_controller.go | 45 +- controllers/database/pdb_controller.go | 89 +++- docs/{onpremdb => multitenant}/README.md | 100 ++-- .../images/K8S_SECURE1.png | Bin .../images/K8S_SECURE2.png | Bin .../images/K8S_SECURE3.png | Bin .../images/K8S_SECURE4.png | Bin docs/multitenant/openssl_schema.jpg | Bin 0 -> 54221 bytes .../provisioning/add_replica.log | 0 .../provisioning/add_replica.md | 0 .../provisioning/add_replica.yaml | 0 .../provisioning/cdb.log | 0 .../provisioning/cdb.yaml | 0 .../provisioning/cdb_crd_resource.md | 0 .../provisioning/cdb_secret.log | 0 .../provisioning/cdb_secret.yaml | 0 .../provisioning/clone_pdb.log | 0 .../provisioning/clone_pdb.md | 0 .../provisioning/clone_pdb.yaml | 0 .../provisioning/create_pdb.log | 0 .../provisioning/create_pdb.md | 0 .../provisioning/create_pdb.yaml | 0 .../provisioning/delete_pdb.log | 0 .../provisioning/delete_pdb.md | 0 .../provisioning/delete_pdb.yaml | 0 .../example_setup_using_oci_oke_cluster.md | 0 .../provisioning/known_issues.md | 0 .../provisioning/modify_pdb.log | 0 .../provisioning/modify_pdb.md | 0 .../provisioning/modify_pdb_close.yaml | 0 .../provisioning/modify_pdb_open.yaml | 0 docs/multitenant/provisioning/ords_image.log | 503 ++++++++++++++++++ docs/multitenant/provisioning/ords_image.md | 64 +++ .../provisioning/pdb.log | 0 .../provisioning/pdb.yaml | 0 .../provisioning/pdb_crd_resource.md | 0 .../provisioning/pdb_secret.log | 0 .../provisioning/pdb_secret.yaml | 0 .../provisioning/plug_pdb.log | 0 .../provisioning/plug_pdb.md | 0 .../provisioning/plug_pdb.yaml | 0 .../provisioning/unplug_pdb.log | 0 .../provisioning/unplug_pdb.md | 0 .../provisioning/unplug_pdb.yaml | 0 .../provisioning/validation_error.md | 0 docs/onpremdb/provisioning/ords_image.log | 263 --------- docs/onpremdb/provisioning/ords_image.md | 68 --- oracle-database-operator.yaml | 84 ++- ords/Dockerfile | 63 +-- ords/cdbadmin.properties.tmpl | 3 - ords/ords_params.properties.tmpl | 16 - ords/runOrds.sh | 75 --- ords/runOrdsSSL.sh | 193 +++++++ ords/setupwebuser.sh | 20 - ords/standalone.properties.tmpl | 8 - 69 files changed, 1170 insertions(+), 554 deletions(-) rename config/samples/{onpremdb => multitenant}/cdb.yaml (100%) rename config/samples/{onpremdb => multitenant}/cdb_secret.yaml (100%) rename config/samples/{onpremdb => multitenant}/pdb_clone.yaml (100%) rename config/samples/{onpremdb => multitenant}/pdb_create.yaml (100%) rename config/samples/{onpremdb => multitenant}/pdb_delete.yaml (100%) rename config/samples/{onpremdb => multitenant}/pdb_modify.yaml (100%) rename config/samples/{onpremdb => multitenant}/pdb_plug.yaml (100%) rename config/samples/{onpremdb => multitenant}/pdb_secret.yaml (100%) rename config/samples/{onpremdb => multitenant}/pdb_unplug.yaml (100%) rename docs/{onpremdb => multitenant}/README.md (59%) rename docs/{onpremdb => multitenant}/images/K8S_SECURE1.png (100%) rename docs/{onpremdb => multitenant}/images/K8S_SECURE2.png (100%) rename docs/{onpremdb => multitenant}/images/K8S_SECURE3.png (100%) rename docs/{onpremdb => multitenant}/images/K8S_SECURE4.png (100%) create mode 100644 docs/multitenant/openssl_schema.jpg rename docs/{onpremdb => multitenant}/provisioning/add_replica.log (100%) rename docs/{onpremdb => multitenant}/provisioning/add_replica.md (100%) rename docs/{onpremdb => multitenant}/provisioning/add_replica.yaml (100%) rename docs/{onpremdb => multitenant}/provisioning/cdb.log (100%) rename docs/{onpremdb => multitenant}/provisioning/cdb.yaml (100%) rename docs/{onpremdb => multitenant}/provisioning/cdb_crd_resource.md (100%) rename docs/{onpremdb => multitenant}/provisioning/cdb_secret.log (100%) rename docs/{onpremdb => multitenant}/provisioning/cdb_secret.yaml (100%) rename docs/{onpremdb => multitenant}/provisioning/clone_pdb.log (100%) rename docs/{onpremdb => multitenant}/provisioning/clone_pdb.md (100%) rename docs/{onpremdb => multitenant}/provisioning/clone_pdb.yaml (100%) rename docs/{onpremdb => multitenant}/provisioning/create_pdb.log (100%) rename docs/{onpremdb => multitenant}/provisioning/create_pdb.md (100%) rename docs/{onpremdb => multitenant}/provisioning/create_pdb.yaml (100%) rename docs/{onpremdb => multitenant}/provisioning/delete_pdb.log (100%) rename docs/{onpremdb => multitenant}/provisioning/delete_pdb.md (100%) rename docs/{onpremdb => multitenant}/provisioning/delete_pdb.yaml (100%) rename docs/{onpremdb => multitenant}/provisioning/example_setup_using_oci_oke_cluster.md (100%) rename docs/{onpremdb => multitenant}/provisioning/known_issues.md (100%) rename docs/{onpremdb => multitenant}/provisioning/modify_pdb.log (100%) rename docs/{onpremdb => multitenant}/provisioning/modify_pdb.md (100%) rename docs/{onpremdb => multitenant}/provisioning/modify_pdb_close.yaml (100%) rename docs/{onpremdb => multitenant}/provisioning/modify_pdb_open.yaml (100%) create mode 100644 docs/multitenant/provisioning/ords_image.log create mode 100644 docs/multitenant/provisioning/ords_image.md rename docs/{onpremdb => multitenant}/provisioning/pdb.log (100%) rename docs/{onpremdb => multitenant}/provisioning/pdb.yaml (100%) rename docs/{onpremdb => multitenant}/provisioning/pdb_crd_resource.md (100%) rename docs/{onpremdb => multitenant}/provisioning/pdb_secret.log (100%) rename docs/{onpremdb => multitenant}/provisioning/pdb_secret.yaml (100%) rename docs/{onpremdb => multitenant}/provisioning/plug_pdb.log (100%) rename docs/{onpremdb => multitenant}/provisioning/plug_pdb.md (100%) rename docs/{onpremdb => multitenant}/provisioning/plug_pdb.yaml (100%) rename docs/{onpremdb => multitenant}/provisioning/unplug_pdb.log (100%) rename docs/{onpremdb => multitenant}/provisioning/unplug_pdb.md (100%) rename docs/{onpremdb => multitenant}/provisioning/unplug_pdb.yaml (100%) rename docs/{onpremdb => multitenant}/provisioning/validation_error.md (100%) delete mode 100644 docs/onpremdb/provisioning/ords_image.log delete mode 100644 docs/onpremdb/provisioning/ords_image.md delete mode 100644 ords/cdbadmin.properties.tmpl delete mode 100644 ords/ords_params.properties.tmpl delete mode 100644 ords/runOrds.sh create mode 100644 ords/runOrdsSSL.sh delete mode 100644 ords/setupwebuser.sh delete mode 100644 ords/standalone.properties.tmpl diff --git a/README.md b/README.md index 7ab5a833..1c091927 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,7 @@ In this v0.2.0 release, `OraOperator` supports the following database configurat * Oracle Autonomous Database on dedicated Cloud infrastructure (ADB-D) * Containerized Single Instance databases (SIDB) deployed in the Oracle Kubernetes Engine (OKE) and any k8s where OraOperator is deployed * Containerized Sharded databases (SHARDED) deployed in OKE and any k8s where OraOperator is deployed -* Oracle On-Premises Databases (CDB/PDBs, Exadata) +* Oracle Multitenant Databases (CDB/PDBs) * Oracle Database Cloud Service (DBCS) (VMDB) * Oracle Autonomous Container Database (ACD) (infrastructure) the infrastructure for provisionning Autonomous Databases. @@ -25,7 +25,7 @@ This release of Oracle Database Operator for Kubernetes (the operator) supports * ACD: provision, bind, restart, terminate (soft/hard) * SIDB: Provision, clone, patch (in-place/out-of-place), update database initialization parameters, update database configuration (Flashback, archiving), Oracle Enterprise Manager (EM) Express (a basic observability console), Oracle REST Data Service (ORDS) to support REST based SQL, PDB management, SQL Developer Web, and Application Express (Apex) * SHARDED: Provision/deploy sharded databases and the shard topology, Add a new shard, Delete an existing shard -* On-Premises Database: Bind to a CDB, Create a  PDB, Plug a  PDB, Unplug a PDB, Delete a PDB, Clone a PDB, Open/Close a PDB +* Oracle Multitenant Database: Bind to a CDB, Create a  PDB, Plug a  PDB, Unplug a PDB, Delete a PDB, Clone a PDB, Open/Close a PDB * Database Cloud Service: Provision, Bind, Scale Up/Down, Liveness Probe, Manual Backup The upcoming releases will support new configurations, operations and capabilities. @@ -99,7 +99,7 @@ The quickstarts are designed for specific database configurations: * [Oracle Autonomous Container Database](./docs/acd/README.md) * [Containerized Oracle Single Instance Database](./docs/sidb/README.md) * [Containerized Oracle Sharded Database](./docs/sharding/README.md) -* [Oracle On-Premises Database](./docs/onpremdb/README.md) +* [Oracle Multitenant Database](./docs/multitenant/README.md) * [Oracle Database Cloud Service](./docs/dbcs/README.md) YAML file templates are available under [`/config/samples`](./config/samples/). You can copy and edit these template files to configure them for your use cases. diff --git a/apis/database/v1alpha1/cdb_types.go b/apis/database/v1alpha1/cdb_types.go index ccf5b0f5..0687cb7f 100644 --- a/apis/database/v1alpha1/cdb_types.go +++ b/apis/database/v1alpha1/cdb_types.go @@ -51,12 +51,19 @@ type CDBSpec struct { CDBName string `json:"cdbName,omitempty"` // Name of the CDB Service ServiceName string `json:"serviceName,omitempty"` + + TestVariable string `json:"TestVariable,omitempty"` + // Password for the CDB System Administrator SysAdminPwd CDBSysAdminPassword `json:"sysAdminPwd,omitempty"` // User in the root container with sysdba priviledges to manage PDB lifecycle CDBAdminUser CDBAdminUser `json:"cdbAdminUser,omitempty"` // Password for the CDB Administrator to manage PDB lifecycle CDBAdminPwd CDBAdminPassword `json:"cdbAdminPwd,omitempty"` + + CDBTlsKey CDBTLSKEY `json:"cdbTlsKey,omitempty"` + CDBTlsCrt CDBTLSCRT `json:"cdbTlsCrt,omitempty"` + // Password for user ORDS_PUBLIC_USER ORDSPwd ORDSPassword `json:"ordsPwd,omitempty"` // ORDS server port. For now, keep it as 8888. TO BE USED IN FUTURE RELEASE. @@ -120,6 +127,14 @@ type WebServerPassword struct { Secret CDBSecret `json:"secret"` } +type CDBTLSKEY struct { + Secret CDBSecret `json:"secret"` +} + +type CDBTLSCRT struct { + Secret CDBSecret `json:"secret"` +} + // CDBStatus defines the observed state of CDB type CDBStatus struct { // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster diff --git a/apis/database/v1alpha1/cdb_webhook.go b/apis/database/v1alpha1/cdb_webhook.go index 2d591223..5ca7f9b8 100644 --- a/apis/database/v1alpha1/cdb_webhook.go +++ b/apis/database/v1alpha1/cdb_webhook.go @@ -92,6 +92,12 @@ func (r *CDB) ValidateCreate() error { allErrs = append(allErrs, field.Required(field.NewPath("spec").Child("serviceName"), "Please specify CDB Service name")) } + + if r.Spec.TestVariable == "" { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("TestVariable"), "Please specify CDB testvarable")) + } + if r.Spec.SCANName == "" { allErrs = append(allErrs, field.Required(field.NewPath("spec").Child("scanName"), "Please specify SCAN Name for CDB")) diff --git a/apis/database/v1alpha1/pdb_types.go b/apis/database/v1alpha1/pdb_types.go index c5b95c81..a73d3dd8 100644 --- a/apis/database/v1alpha1/pdb_types.go +++ b/apis/database/v1alpha1/pdb_types.go @@ -47,6 +47,10 @@ type PDBSpec struct { // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster // Important: Run "make" to regenerate code after modifying this file + PDBTlsKey PDBTLSKEY `json:"pdbTlsKey,omitempty"` + PDBTlsCrt PDBTLSCRT `json:"pdbTlsCrt,omitempty"` + PDBTlsCat PDBTLSCAT `json:"pdbTlsCat,omitempty"` + // Name of the CDB Custom Resource that runs the ORDS container CDBResName string `json:"cdbResName,omitempty"` // Name of the CDB @@ -132,6 +136,18 @@ type PDBSecret struct { Key string `json:"key"` } +type PDBTLSKEY struct { + Secret PDBSecret `json:"secret"` +} + +type PDBTLSCRT struct { + Secret PDBSecret `json:"secret"` +} + +type PDBTLSCAT struct { + Secret PDBSecret `json:"secret"` +} + // PDBStatus defines the observed state of PDB type PDBStatus struct { // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster @@ -157,7 +173,7 @@ type PDBStatus struct { // +kubebuilder:object:root=true // +kubebuilder:subresource:status -// +kubebuilder:printcolumn:JSONPath=".status.connString",name="Connect String",type="string",description="The connect string to be used" +// +kubebuilder:printcolumn:JSONPath=".status.connString",name="Connect_String",type="string",description="The connect string to be used" // +kubebuilder:printcolumn:JSONPath=".spec.cdbName",name="CDB Name",type="string",description="Name of the CDB" // +kubebuilder:printcolumn:JSONPath=".spec.pdbName",name="PDB Name",type="string",description="Name of the PDB" // +kubebuilder:printcolumn:JSONPath=".status.openMode",name="PDB State",type="string",description="PDB Open Mode" diff --git a/apis/database/v1alpha1/zz_generated.deepcopy.go b/apis/database/v1alpha1/zz_generated.deepcopy.go index 17eb7052..72d93dc2 100644 --- a/apis/database/v1alpha1/zz_generated.deepcopy.go +++ b/apis/database/v1alpha1/zz_generated.deepcopy.go @@ -702,6 +702,8 @@ func (in *CDBSpec) DeepCopyInto(out *CDBSpec) { out.SysAdminPwd = in.SysAdminPwd out.CDBAdminUser = in.CDBAdminUser out.CDBAdminPwd = in.CDBAdminPwd + out.CDBTlsKey = in.CDBTlsKey + out.CDBTlsCrt = in.CDBTlsCrt out.ORDSPwd = in.ORDSPwd out.WebServerUser = in.WebServerUser out.WebServerPwd = in.WebServerPwd @@ -755,6 +757,38 @@ func (in *CDBSysAdminPassword) DeepCopy() *CDBSysAdminPassword { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CDBTLSCRT) DeepCopyInto(out *CDBTLSCRT) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDBTLSCRT. +func (in *CDBTLSCRT) DeepCopy() *CDBTLSCRT { + if in == nil { + return nil + } + out := new(CDBTLSCRT) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CDBTLSKEY) DeepCopyInto(out *CDBTLSKEY) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDBTLSKEY. +func (in *CDBTLSKEY) DeepCopy() *CDBTLSKEY { + if in == nil { + return nil + } + out := new(CDBTLSKEY) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *CatalogSpec) DeepCopyInto(out *CatalogSpec) { *out = *in @@ -1662,6 +1696,9 @@ func (in *PDBSecret) DeepCopy() *PDBSecret { // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *PDBSpec) DeepCopyInto(out *PDBSpec) { *out = *in + out.PDBTlsKey = in.PDBTlsKey + out.PDBTlsCrt = in.PDBTlsCrt + out.PDBTlsCat = in.PDBTlsCat out.AdminName = in.AdminName out.AdminPwd = in.AdminPwd if in.ReuseTempFile != nil { @@ -1723,6 +1760,54 @@ func (in *PDBStatus) DeepCopy() *PDBStatus { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PDBTLSCAT) DeepCopyInto(out *PDBTLSCAT) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBTLSCAT. +func (in *PDBTLSCAT) DeepCopy() *PDBTLSCAT { + if in == nil { + return nil + } + out := new(PDBTLSCAT) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PDBTLSCRT) DeepCopyInto(out *PDBTLSCRT) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBTLSCRT. +func (in *PDBTLSCRT) DeepCopy() *PDBTLSCRT { + if in == nil { + return nil + } + out := new(PDBTLSCRT) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PDBTLSKEY) DeepCopyInto(out *PDBTLSKEY) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBTLSKEY. +func (in *PDBTLSKEY) DeepCopy() *PDBTLSKEY { + if in == nil { + return nil + } + out := new(PDBTLSKEY) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *PITSpec) DeepCopyInto(out *PITSpec) { *out = *in diff --git a/config/samples/onpremdb/cdb.yaml b/config/samples/multitenant/cdb.yaml similarity index 100% rename from config/samples/onpremdb/cdb.yaml rename to config/samples/multitenant/cdb.yaml diff --git a/config/samples/onpremdb/cdb_secret.yaml b/config/samples/multitenant/cdb_secret.yaml similarity index 100% rename from config/samples/onpremdb/cdb_secret.yaml rename to config/samples/multitenant/cdb_secret.yaml diff --git a/config/samples/onpremdb/pdb_clone.yaml b/config/samples/multitenant/pdb_clone.yaml similarity index 100% rename from config/samples/onpremdb/pdb_clone.yaml rename to config/samples/multitenant/pdb_clone.yaml diff --git a/config/samples/onpremdb/pdb_create.yaml b/config/samples/multitenant/pdb_create.yaml similarity index 100% rename from config/samples/onpremdb/pdb_create.yaml rename to config/samples/multitenant/pdb_create.yaml diff --git a/config/samples/onpremdb/pdb_delete.yaml b/config/samples/multitenant/pdb_delete.yaml similarity index 100% rename from config/samples/onpremdb/pdb_delete.yaml rename to config/samples/multitenant/pdb_delete.yaml diff --git a/config/samples/onpremdb/pdb_modify.yaml b/config/samples/multitenant/pdb_modify.yaml similarity index 100% rename from config/samples/onpremdb/pdb_modify.yaml rename to config/samples/multitenant/pdb_modify.yaml diff --git a/config/samples/onpremdb/pdb_plug.yaml b/config/samples/multitenant/pdb_plug.yaml similarity index 100% rename from config/samples/onpremdb/pdb_plug.yaml rename to config/samples/multitenant/pdb_plug.yaml diff --git a/config/samples/onpremdb/pdb_secret.yaml b/config/samples/multitenant/pdb_secret.yaml similarity index 100% rename from config/samples/onpremdb/pdb_secret.yaml rename to config/samples/multitenant/pdb_secret.yaml diff --git a/config/samples/onpremdb/pdb_unplug.yaml b/config/samples/multitenant/pdb_unplug.yaml similarity index 100% rename from config/samples/onpremdb/pdb_unplug.yaml rename to config/samples/multitenant/pdb_unplug.yaml diff --git a/controllers/database/cdb_controller.go b/controllers/database/cdb_controller.go index 9be4477b..5c9bc3ee 100644 --- a/controllers/database/cdb_controller.go +++ b/controllers/database/cdb_controller.go @@ -283,7 +283,8 @@ func (r *CDBReconciler) validateORDSPods(ctx context.Context, req ctrl.Request, if pod.Status.Phase == corev1.PodRunning { // Get ORDS Status out, err := dbcommons.ExecCommand(r, r.Config, pod.Name, pod.Namespace, "", ctx, req, false, "bash", "-c", getORDSStatus) - if strings.Contains(out, "HTTP/1.1 200 OK") || strings.Contains(strings.ToUpper(err.Error()), "HTTP/1.1 200 OK") { + if strings.Contains(out, "HTTP/1.1 200 OK") || strings.Contains(strings.ToUpper(err.Error()), "HTTP/1.1 200 OK") || + strings.Contains(out, "HTTP/2") || strings.Contains(strings.ToUpper(err.Error()), " HTTP/2") { readyPods++ } else if strings.Contains(out, "HTTP/1.1 404 Not Found") || strings.Contains(strings.ToUpper(err.Error()), "HTTP/1.1 404 NOT FOUND") { // Check if DB connection parameters are correct @@ -348,6 +349,36 @@ func (r *CDBReconciler) createPodSpec(cdb *dbapi.CDB) corev1.PodSpec { }, }, }, + /***/ + { + Secret: &corev1.SecretProjection{ + LocalObjectReference: corev1.LocalObjectReference{ + Name: cdb.Spec.CDBTlsKey.Secret.SecretName, + }, + Items: []corev1.KeyToPath{ + { + Key: cdb.Spec.CDBTlsKey.Secret.Key, + Path: cdb.Spec.CDBTlsKey.Secret.Key, + }, + }, + }, + }, + + { + Secret: &corev1.SecretProjection{ + LocalObjectReference: corev1.LocalObjectReference{ + Name: cdb.Spec.CDBTlsCrt.Secret.SecretName, + }, + Items: []corev1.KeyToPath{ + { + Key: cdb.Spec.CDBTlsCrt.Secret.Key, + Path: cdb.Spec.CDBTlsCrt.Secret.Key, + }, + }, + }, + }, + + /***/ { Secret: &corev1.SecretProjection{ LocalObjectReference: corev1.LocalObjectReference{ @@ -418,6 +449,18 @@ func (r *CDBReconciler) createPodSpec(cdb *dbapi.CDB) corev1.PodSpec { Name: "ORACLE_HOST", Value: cdb.Spec.DBServer, }, + { + Name: "TESTVAR", + Value: cdb.Spec.TestVariable, + }, + { + Name: "TLSCRT", + Value: cdb.Spec.CDBTlsCrt.Secret.Key, + }, + { + Name: "TLSKEY", + Value: cdb.Spec.CDBTlsKey.Secret.Key, + }, { Name: "ORACLE_PORT", Value: strconv.Itoa(cdb.Spec.DBPort), diff --git a/controllers/database/pdb_controller.go b/controllers/database/pdb_controller.go index 42a21b51..099cb921 100644 --- a/controllers/database/pdb_controller.go +++ b/controllers/database/pdb_controller.go @@ -41,7 +41,10 @@ package controllers import ( "bytes" "context" + "crypto/tls" + "crypto/x509" "encoding/json" + //"encoding/pem" "errors" "fmt" "io/ioutil" @@ -401,7 +404,65 @@ func (r *PDBReconciler) callAPI(ctx context.Context, req ctrl.Request, pdb *dbap log := r.Log.WithValues("callAPI", req.NamespacedName) var err error - httpclient := &http.Client{} + + secret := &corev1.Secret{} + + err = r.Get(ctx, types.NamespacedName{Name: pdb.Spec.PDBTlsKey.Secret.SecretName, Namespace: pdb.Namespace}, secret) + + if err != nil { + if apierrors.IsNotFound(err) { + log.Info("Secret not found:" + pdb.Spec.PDBTlsKey.Secret.SecretName) + return "", err + } + log.Error(err, "Unable to get the secret.") + return "", err + } + rsaKeyPEM := secret.Data[pdb.Spec.PDBTlsKey.Secret.Key] + + err = r.Get(ctx, types.NamespacedName{Name: pdb.Spec.PDBTlsCrt.Secret.SecretName, Namespace: pdb.Namespace}, secret) + + if err != nil { + if apierrors.IsNotFound(err) { + log.Info("Secret not found:" + pdb.Spec.PDBTlsCrt.Secret.SecretName) + return "", err + } + log.Error(err, "Unable to get the secret.") + return "", err + } + + rsaCertPEM := secret.Data[pdb.Spec.PDBTlsCrt.Secret.Key] + + err = r.Get(ctx, types.NamespacedName{Name: pdb.Spec.PDBTlsCat.Secret.SecretName, Namespace: pdb.Namespace}, secret) + + if err != nil { + if apierrors.IsNotFound(err) { + log.Info("Secret not found:" + pdb.Spec.PDBTlsCat.Secret.SecretName) + return "", err + } + log.Error(err, "Unable to get the secret.") + return "", err + } + + caCert := secret.Data[pdb.Spec.PDBTlsCat.Secret.Key] + + r.Recorder.Eventf(pdb, corev1.EventTypeWarning, "ORDSINFO", string(rsaKeyPEM)) + r.Recorder.Eventf(pdb, corev1.EventTypeWarning, "ORDSINFO", string(rsaCertPEM)) + r.Recorder.Eventf(pdb, corev1.EventTypeWarning, "ORDSINFO", string(caCert)) + + certificate, err := tls.X509KeyPair([]byte(rsaCertPEM), []byte(rsaKeyPEM)) + if err != nil { + pdb.Status.Msg = "Error tls.X509KeyPair" + return "", err + } + + caCertPool := x509.NewCertPool() + caCertPool.AppendCertsFromPEM(caCert) + + tlsConf := &tls.Config{Certificates: []tls.Certificate{certificate}, RootCAs: caCertPool} + + tr := &http.Transport{TLSClientConfig: tlsConf} + + httpclient := &http.Client{Transport: tr} log.Info("Issuing REST call", "URL", url, "Action", action) @@ -411,7 +472,7 @@ func (r *PDBReconciler) callAPI(ctx context.Context, req ctrl.Request, pdb *dbap } // Get Web Server User - secret := &corev1.Secret{} + //secret := &corev1.Secret{} err = r.Get(ctx, types.NamespacedName{Name: cdb.Spec.WebServerUser.Secret.SecretName, Namespace: cdb.Namespace}, secret) if err != nil { if apierrors.IsNotFound(err) { @@ -440,7 +501,7 @@ func (r *PDBReconciler) callAPI(ctx context.Context, req ctrl.Request, pdb *dbap if action == "GET" { httpreq, err = http.NewRequest(action, url, nil) } else { - //fmt.Println("payload:", payload) + fmt.Println("payload:", payload) jsonValue, _ := json.Marshal(payload) httpreq, err = http.NewRequest(action, url, bytes.NewBuffer(jsonValue)) } @@ -456,12 +517,14 @@ func (r *PDBReconciler) callAPI(ctx context.Context, req ctrl.Request, pdb *dbap resp, err := httpclient.Do(httpreq) if err != nil { + errmsg := err.Error() log.Error(err, "Failed - Could not connect to ORDS Pod", "err", err.Error()) - pdb.Status.Msg = "Could not connect to ORDS Pod" - r.Recorder.Eventf(pdb, corev1.EventTypeWarning, "ORDSError", "Failed: Could not connect to ORDS Pod") + pdb.Status.Msg = "Error: Could not connect to ORDS Pod" + r.Recorder.Eventf(pdb, corev1.EventTypeWarning, "ORDSError", errmsg) return "", err } + r.Recorder.Eventf(pdb, corev1.EventTypeWarning, "Done", pdb.Spec.CDBResName) if resp.StatusCode != http.StatusOK { bb, _ := ioutil.ReadAll(resp.Body) @@ -563,7 +626,7 @@ func (r *PDBReconciler) createPDB(ctx context.Context, req ctrl.Request, pdb *db values["tdeSecret"] = tdeSecret } - url := "http://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + url := "https://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" pdb.Status.TotalSize = pdb.Spec.TotalSize pdb.Status.Phase = pdbPhaseCreate @@ -618,7 +681,7 @@ func (r *PDBReconciler) clonePDB(ctx context.Context, req ctrl.Request, pdb *dba values["tempSize"] = pdb.Spec.TempSize } - url := "http://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdb.Spec.SrcPDBName + "/" + url := "https://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdb.Spec.SrcPDBName + "/" pdb.Status.Phase = pdbPhaseClone pdb.Status.Msg = "Waiting for PDB to be cloned" @@ -685,7 +748,7 @@ func (r *PDBReconciler) plugPDB(ctx context.Context, req ctrl.Request, pdb *dbap values["asClone"] = strconv.FormatBool(*(pdb.Spec.AsClone)) } - url := "http://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + url := "https://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" pdb.Status.TotalSize = pdb.Spec.TotalSize pdb.Status.Phase = pdbPhasePlug @@ -741,7 +804,7 @@ func (r *PDBReconciler) unplugPDB(ctx context.Context, req ctrl.Request, pdb *db values["tdeExport"] = strconv.FormatBool(*(pdb.Spec.TDEExport)) } - url := "http://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdb.Spec.PDBName + "/" + url := "https://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdb.Spec.PDBName + "/" pdb.Status.Phase = pdbPhaseUnplug pdb.Status.Msg = "Waiting for PDB to be unplugged" @@ -806,7 +869,7 @@ func (r *PDBReconciler) modifyPDB(ctx context.Context, req ctrl.Request, pdb *db "getScript": strconv.FormatBool(*(pdb.Spec.GetScript))} pdbName := pdb.Spec.PDBName - url := "http://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdbName + "/status" + url := "https://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdbName + "/status" pdb.Status.Phase = pdbPhaseModify pdb.Status.ModifyOption = pdb.Spec.PDBState + "-" + pdb.Spec.ModifyOption @@ -842,7 +905,7 @@ func (r *PDBReconciler) getPDBState(ctx context.Context, req ctrl.Request, pdb * } pdbName := pdb.Spec.PDBName - url := "http://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdbName + "/status" + url := "https://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdbName + "/status" pdb.Status.Msg = "Getting PDB state" if err := r.Status().Update(ctx, pdb); err != nil { @@ -882,7 +945,7 @@ func (r *PDBReconciler) mapPDB(ctx context.Context, req ctrl.Request, pdb *dbapi } pdbName := pdb.Spec.PDBName - url := "http://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdbName + "/" + url := "https://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdbName + "/" pdb.Status.Msg = "Mapping PDB" if err := r.Status().Update(ctx, pdb); err != nil { @@ -1015,7 +1078,7 @@ func (r *PDBReconciler) deletePDBInstance(req ctrl.Request, ctx context.Context, } pdbName := pdb.Spec.PDBName - url := "http://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdbName + "/" + url := "https://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdbName + "/" pdb.Status.Phase = pdbPhaseDelete pdb.Status.Msg = "Waiting for PDB to be deleted" diff --git a/docs/onpremdb/README.md b/docs/multitenant/README.md similarity index 59% rename from docs/onpremdb/README.md rename to docs/multitenant/README.md index 3c77c1b4..be311094 100644 --- a/docs/onpremdb/README.md +++ b/docs/multitenant/README.md @@ -1,19 +1,23 @@ -# Oracle On-Prem Database Controller + -CDBs and PDBs are the part of the Oracle Database's [Multitenant Architecture](https://docs.oracle.com/en/database/oracle/oracle-database/21/multi/introduction-to-the-multitenant-architecture.html#GUID-AB84D6C9-4BBE-4D36-992F-2BB85739329F). The On-Prem Database Controller is a feature of Oracle DB Operator for Kubernetes (OraOperator) which helps to manage the life cycle of Pluggable Databases (PDBs) in an Oracle Container Database(CDB). -The target CDB (for which the PDB life cycle management is needed) can be running on an on-prem machine and to manage its PDBs, the Oracle DB Operator can run on an on-prem Kubernetes system (For Example: [Oracle Linux Cloud Native Environment or OLCNE](https://docs.oracle.com/en/operating-systems/olcne/)). + +# Oracle Multitenant Database Controller + +CDBs and PDBs are the part of the Oracle Database's [Multitenant Architecture](https://docs.oracle.com/en/database/oracle/oracle-database/21/multi/introduction-to-the-multitenant-architecture.html#GUID-AB84D6C9-4BBE-4D36-992F-2BB85739329F). The Multitenant Database Controller is a feature of Oracle DB Operator for Kubernetes (OraOperator) which helps to manage the life cycle of Pluggable Databases (PDBs) in an Oracle Container Database(CDB). + +The target CDB (for which the PDB life cycle management is needed) can be running on an multitenant machine and to manage its PDBs, the Oracle DB Operator can run on an multitenant Kubernetes system (For Example: [Oracle Linux Cloud Native Environment or OLCNE](https://docs.oracle.com/en/operating-systems/olcne/)). NOTE: The target CDB (for which the PDB life cycle management is needed) **can also** run in a Cloud environment as well (For Example: OCI's [Oracle Base Database Service](https://docs.oracle.com/en-us/iaas/dbcs/doc/bare-metal-and-virtual-machine-db-systems.html)) and to manage its PDBs, the Oracle DB Operator can run on a Kubernetes Cluster running in cloud (For Example: OCI's [Container Engine for Kubernetes or OKE](https://docs.oracle.com/en-us/iaas/Content/ContEng/Concepts/contengoverview.htm#Overview_of_Container_Engine_for_Kubernetes)) -# Oracle DB Operator On-Prem Database Controller Deployment +# Oracle DB Operator Multitenant Database Controller Deployment To deploy OraOperator, use this [Oracle Database Operator for Kubernetes](https://github.com/oracle/oracle-database-operator/blob/main/README.md) step-by-step procedure. -After the Oracle Database Operator is deployed, you can see the DB Operator Pods running in the Kubernetes Cluster. As part of the OraOperator deployment, the On-Prem Database Controller is deployed and we can see the CRDs (Custom Resource Definition) for CDB and PDB in the list of CRDs. The following output is an example of such a deployment: -``` +After the Oracle Database Operator is deployed, you can see the DB Operator Pods running in the Kubernetes Cluster. As part of the OraOperator deployment, the multitenant Database Controller is deployed and we can see the CRDs (Custom Resource Definition) for CDB and PDB in the list of CRDs. The following output is an example of such a deployment: +```bash [root@test-server oracle-database-operator]# kubectl get ns NAME STATUS AGE cert-manager Active 32h @@ -65,9 +69,9 @@ singleinstancedatabases.database.oracle.com 2022-06-22T01:21:40Z The following sections explain the setup and functionality of this controller. -# Prerequsites to manage PDB Life Cycle using Oracle DB Operator On-Prem Database Controller +# Prerequsites to manage PDB Life Cycle using Oracle DB Operator Multitenant Database Controller -Before you want to manage the life cycle of a PDB in a CDB using the Oracle DB Operator On-Prem Database Controller, complete the following steps. +Before you want to manage the life cycle of a PDB in a CDB using the Oracle DB Operator Multitenant Database Controller, complete the following steps. **CAUTION :** You must make the changes specified in this section before you proceed to the next section. @@ -86,90 +90,98 @@ You cannot have an ORDS enabled schema in the container database. To perform the Create the CDB administrator user and grant the required privileges. In this example, the user is `C##DBAPI_CDB_ADMIN`. However, any suitable common user name can be used. -Also, create a user named SQL_ADMIN. You can use the below set of SQL commands for this purpose: - -``` +```SQL SQL> conn /as sysdba -- Create the below users at the database level: -DROP USER SQL_ADMIN CASCADE; -CREATE USER SQL_ADMIN IDENTIFIED BY welcome1; ALTER SESSION SET "_oracle_script"=true; DROP USER C##DBAPI_CDB_ADMIN cascade; CREATE USER C##DBAPI_CDB_ADMIN IDENTIFIED BY CONTAINER=ALL ACCOUNT UNLOCK; GRANT SYSOPER TO C##DBAPI_CDB_ADMIN CONTAINER = ALL; GRANT SYSDBA TO C##DBAPI_CDB_ADMIN CONTAINER = ALL; GRANT CREATE SESSION TO C##DBAPI_CDB_ADMIN CONTAINER = ALL; -GRANT CREATE SESSION TO SQL_ADMIN; -GRANT SYSDBA TO SQL_ADMIN; - + -- Verify the account status of the below usernames. They should not be in locked status: col username for a30 col account_status for a30 -select username, account_status from dba_users where username in ('ORDS_PUBLIC_USER','SQL_ADMIN','C##DBAPI_CDB_ADMIN','APEX_PUBLIC_USER','APEX_REST_PUBLIC_USER'); +select username, account_status from dba_users where username in ('ORDS_PUBLIC_USER','C##DBAPI_CDB_ADMIN','APEX_PUBLIC_USER','APEX_REST_PUBLIC_USER'); ``` - - ### Reference Setup: Example of a setup using OCI OKE(Kubernetes Cluster) and a CDB in Cloud (OCI Exadata Database Cluster) Please refer [here](./provisioning/example_setup_using_oci_oke_cluster.md) for steps to configure a Kubernetes Cluster and a CDB. This example uses an OCI OKE Cluster as the Kubernetes Cluster and a CDB in OCI Exadata Database service. - + ## Oracle REST Data Service (ORDS) Image - Oracle DB Operator On-Prem Database controller requires the Oracle REST Data Services (ORDS) image for PDB Lifecycle Management in the target CDB. + Oracle DB Operator Multitenant Database controller requires the Oracle REST Data Services (ORDS) image for PDB Lifecycle Management in the target CDB. You can build this image by using the ORDS [Dockerfile](../../../ords/Dockerfile) - - > **_NOTE:_** Download the required binaries to build this image i.e. Oracle Rest Data Services 'ords-< version >.zip', from http://www.oracle.com/technetwork/developer-tools/rest-data-services/downloads/index.html - - > **_NOTE:_** The current version of Oracle DB Operator On-Prem Controller has been tested with `ORDS 21.4.3` version. + + > **_NOTE:_** The current version of Oracle DB Operator Multitenant Controller has been tested with **ords 22.2.1.202.1302** version. -Please refer [here](./provisioning/ords_image.md) for the steps to build ORDS Docker Image with `ORDS 21.4.3` version. +Please refer [here](./provisioning/ords_image.md) for the steps to build ORDS Docker Image with **ords 22.2.1.202.1302** version. + ## Kubernetes Secrets - Oracle DB Operator On-Prem Database Controller uses Kubernetes Secrets to store usernames and passwords to manage the life cycle operations of a PDB in the target CDB. + Oracle DB Operator Multitenant Database Controller uses Kubernetes Secrets to store usernames and passwords to manage the life cycle operations of a PDB in the target CDB. In addition to that ,in order to use https protocol, all certificates need to be stored using Kubernests Secret. ### Secrets for CDB CRD - Create a secret file as shown here: [config/samples/onpremdb/cdb_secret.yaml](../../config/samples/onpremdb/cdb_secret.yaml). Modify this file with the `base64` encoded values of the required passwords for CDB and use it to create the required secrets. + Create a secret file as shown here: [config/samples/multitenant/cdb_secret.yaml](../../config/samples/multitenant/cdb_secret.yaml). Modify this file with the `base64` encoded values of the required passwords for CDB and use it to create the required secrets. - ```sh - $ kubectl apply -f cdb_secret.yaml + ```bash + kubectl apply -f cdb_secret.yaml ``` **Note:** In order to get the base64 encoded value for a password, please use the below command like below at the command prompt. The value you get is the base64 encoded value for that password string. - ```sh + ```bash echo -n "" | base64 ``` - **Note:** On successful creation of the CDB Resource, the CDB secrets would be deleted from the Kubernetes system. + **Note:** On successful creation of the CDB Resource, the CDB secrets would be deleted from the Kubernetes . ### Secrets for PDB CRD - Create a secret file as shown here: [config/samples/onpremdb/pdb_secret.yaml](../../config/samples/onpremdb/pdb_secret.yaml). Modify this file with the `base64` encoded values of the required passwords for PDB and use it to create the required secrets. + Create a secret file as shown here: [config/samples/multitenant/pdb_secret.yaml](../../config/samples/multitenant/pdb_secret.yaml). Modify this file with the `base64` encoded values of the required passwords for PDB and use it to create the required secrets. - ```sh - $ kubectl apply -f pdb_secret.yaml + ```bash + kubectl apply -f pdb_secret.yaml ``` **NOTE:** Refer to command provided above to encode the password using base64. - **NOTE:** Don't leave plaintext files containing sensitive data on disk. After loading the Secret, remove the plaintext file or move it to secure storage. + **NOTE:** Don't leave plaintext files containing sensitive data on disk. After loading the Secret, remove the plaintext file or move it to secure storage. +### Secrets for CERTIFICATES + +Create certificates and key on your local host and use them to create Kubernet secret + +```bash +genrsa -out ca.key 2048 +openssl req -new -x509 -days 365 -key ca.key -subj "/C=CN/ST=GD/L=SZ/O=oracle, Inc./CN=oracle Root CA" -out ca.crt +openssl req -newkey rsa:2048 -nodes -keyout tls.key -subj "/C=CN/ST=GD/L=SZ/O=oracle, Inc./CN=cdb-dev-ords" -out server.csr +/usr/bin/echo "subjectAltName=DNS:cdb-dev-ords,DNS:www.example.com" > extfile.txt +openssl x509 -req -extfile extfile.txt -days 365 -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out tls.crt +``` + +```bash +kubectl create secret tls db-tls --key="tls.key" --cert="tls.crt" -n oracle-database-operator-system +kubectl create secret generic db-ca --from-file=ca.crt -n oracle-database-operator-system +``` + +image_not_found +**Note:** On successful creation of the certificates secret creation remove files or move to secure storage . + ## Kubernetes CRD for CDB -The Oracle Database Operator On-Prem Controller creates the CDB kind as a custom resource that models a target CDB as a native Kubernetes object. This is used only to create Pods to connect to the target CDB to perform PDB-LM operations. These CDB resources can be scaled up and down based on the expected load using replicas. Each CDB resource follows the CDB CRD as defined here: [config/crd/bases/database.oracle.com_cdbs.yaml](../../config/crd/bases/database.oracle.com_cdbs.yaml) +The Oracle Database Operator Multitenant Controller creates the CDB kind as a custom resource that models a target CDB as a native Kubernetes object. This is used only to create Pods to connect to the target CDB to perform PDB-LM operations. These CDB resources can be scaled up and down based on the expected load using replicas. Each CDB resource follows the CDB CRD as defined here: [config/crd/bases/database.oracle.com_cdbs.yaml](../../config/crd/bases/database.oracle.com_cdbs.yaml) -To create a CDB CRD, a sample .yaml file is available here: [config/samples/onpremdb/cdb.yaml](../../config/samples/onpremdb/cdb.yaml) +To create a CDB CRD, a sample .yaml file is available here: [config/samples/multitenant/cdb.yaml](../../config/samples/multitenant/cdb.yaml) -**Note:** The password and username fields in this `cdb.yaml` yaml are Kubernetes Secrets created earlier. Please see the section [Kubernetes Secrets](ORACLE_ONPREMDB_CONTROLLER_README.md#kubernetes-secrets) for more information. Please see [Kubernetes Private Registry Documenation]( https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/) for creating secrets for pulling images from docker private registry. +**Note:** The password and username fields in this *cdb.yaml* yaml are Kubernetes Secrets created earlier. Please see the section [Kubernetes Secrets](https://kubernetes.io/docs/concepts/configuration/secret/) for more information. Please see [Kubernetes Private Registry Documenation]( https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/) for creating secrets for pulling images from docker private registry. 1. [Use Case: Create a CDB CRD Resource](./provisioning/cdb_crd_resource.md) 2. [Use Case: Add another replica to an existing CDB CRD Resource](./provisioning/add_replica.md) @@ -177,13 +189,13 @@ To create a CDB CRD, a sample .yaml file is available here: [config/samples/onpr + ## Kubernetes CRD for PDB -The Oracle Database Operator On-Prem Controller creates the PDB kind as a custom resource that models a PDB as a native Kubernetes object. There is a one-to-one mapping between the actual PDB and the Kubernetes PDB Custom Resource. You cannot have more than one Kubernetes resource for a target PDB. This PDB resource can be used to perform PDB-LM operations by specifying the action attribute in the PDB Specs. Each PDB resource follows the PDB CRD as defined here: [config/crd/bases/database.oracle.com_pdbs.yaml](../../../config/crd/bases/database.oracle.com_pdbs.yaml) +The Oracle Database Operator Multitenant Controller creates the PDB kind as a custom resource that models a PDB as a native Kubernetes object. There is a one-to-one mapping between the actual PDB and the Kubernetes PDB Custom Resource. You cannot have more than one Kubernetes resource for a target PDB. This PDB resource can be used to perform PDB-LM operations by specifying the action attribute in the PDB Specs. Each PDB resource follows the PDB CRD as defined here: [config/crd/bases/database.oracle.com_pdbs.yaml](../../../config/crd/bases/database.oracle.com_pdbs.yaml) -To create a PDB CRD Resource, a sample .yaml file is available here: [config/samples/onpremdb/pdb_create.yaml](../../config/samples/onpremdb/pdb_create.yaml) +To create a PDB CRD Resource, a sample .yaml file is available here: [config/samples/multitenant/pdb_create.yaml](../../config/samples/multitenant/pdb_create.yaml) -# Use Cases for PDB Lifecycle Management Operations using Oracle DB Operator On-Prem Controller +# Use Cases for PDB Lifecycle Management Operations using Oracle DB Operator Multitenant Controller -Using Oracle DB Operator On-Prem Controller, you can perform the following PDB-LM operations: CREATE, CLONE, MODIFY, DELETE, UNPLUG, PLUG. +Using Oracle DB Operator Multitenant Controller, you can perform the following PDB-LM operations: CREATE, CLONE, MODIFY, DELETE, UNPLUG, PLUG. 1. [Create PDB](./provisioning/create_pdb.md) 2. [Clone PDB](./provisioning/clone_pdb.md) @@ -200,4 +212,4 @@ Please check [here](./provisioning/validation_error.md) for the details to look ## Known issues -Please refer [here](./provisioning/known_issues.md) for the known issues related to Oracle DB Operator On-Prem Controller. +Please refer [here](./provisioning/known_issues.md) for the known issues related to Oracle DB Operator Multitenant Controller. diff --git a/docs/onpremdb/images/K8S_SECURE1.png b/docs/multitenant/images/K8S_SECURE1.png similarity index 100% rename from docs/onpremdb/images/K8S_SECURE1.png rename to docs/multitenant/images/K8S_SECURE1.png diff --git a/docs/onpremdb/images/K8S_SECURE2.png b/docs/multitenant/images/K8S_SECURE2.png similarity index 100% rename from docs/onpremdb/images/K8S_SECURE2.png rename to docs/multitenant/images/K8S_SECURE2.png diff --git a/docs/onpremdb/images/K8S_SECURE3.png b/docs/multitenant/images/K8S_SECURE3.png similarity index 100% rename from docs/onpremdb/images/K8S_SECURE3.png rename to docs/multitenant/images/K8S_SECURE3.png diff --git a/docs/onpremdb/images/K8S_SECURE4.png b/docs/multitenant/images/K8S_SECURE4.png similarity index 100% rename from docs/onpremdb/images/K8S_SECURE4.png rename to docs/multitenant/images/K8S_SECURE4.png diff --git a/docs/multitenant/openssl_schema.jpg b/docs/multitenant/openssl_schema.jpg new file mode 100644 index 0000000000000000000000000000000000000000..4453d52f27eaa83c6891a9ba6c8bef418b89ec8e GIT binary patch literal 54221 zcmeFYXH-*L`!9+O6#=D7bwdc!0@Ax~DS?D0p$VZ#3j_#AZ-Q=>-U9?e-IN4El@LM| zlpd;t(2Gh_ssaK6f`|Q{_uTW|`{9mrzTETS{+~6*T5GH|pZQyJ&fgq!K4tx$`u&OS z78DEt(_OegM|a`;p!+>br$u+^;y?1w`O>*uroa4;T%)JILVx`l1H<)e*RL}$-ehFB z!F=QTbtV=j=9{-}vD{){WMyN$#ddyw>z_(4{G)m4^0jltTQ{!XIDh)TDZk&*v0S@& z^U{M$7li08vRt^ta^ZIu9sfU{dg;PHLicaEdWHVlQar7L3?;Q8yKluLxosRAb{iVxSFJ8EI z{#Kjiyb~{7xq8Ttzu1HvQm+=W17u;f??*pte&-Z6 zrS5QW$(uR)zC;%hJ31GC&(JYmI>%tS#6tIg?$q>u;{2bwmvH{pbefZBZ0>h(xBaf((-Y+Ap8iFrmU3CnmkDXsd@$M? zs#`XpCj+2dN+B$8S|jO5#Vi-1g@W1{hStZkEuSu39INO zb~D6tCI;=&_3KbNYQZVSc6{D6@L|7Qk+XK)eYgmPR{Z?p?%m(w+h1+AfTe+O0s;B~&x@{@Ym%})Zs}OGmMP9Ll#Z~hZtldxQ28|)#v-Tn@49iMQbwia4ZtoU|(b0L3|4;FQujmY( zK4rbW9e{QcE$QQtb;K{XC&OO-3bq3;lE<1wl#fvK}&nZ^>J` zC!t+%ak;K9kSIQ5;X?I5T_H$n{;K<~94O|=;*urFnvjF^3ddT)o5~c*ZJ*Khw5s}m zRVI&)jeWs$2C-st0HKGRv#E@j{DW9r2S6!)JxUCevl(9Cg7**Js#=ks-ZZR9MW@xW zJY%H5y;lzA-SMZP$WlnYBlajbH^>pdz3M0)SH~Y>N+=&vjj6Cq5-B@sN;s_3P>#(; zhOw7j4U~>6+ppm>7Ot$DELC|%;>g%{na5YQD;O1=g(d2I_?j+ zVQuSf3La&R=(H+}6?yI;(DVEtF~?jDwmhwn215SAk`z*7PWR-~W5plThj~GSPFTUu zk=S3?NKyMEkMn&%3VE|24_PQIgC#rR?6OV+HWt)2vF|l#*uR#qYL3m?&pPuH-3pk` zUYU;fH=nFh?mE`~O(z<)ztviNXH1gqUUFi2uW08k^6|2$agS+rW2v{L@x+#hStDLGv#t0h_3P{MPaewQK) zwP)Q0jrh%O&Y2%lio!C$a&l%ThNiJUu!9`~#BP(1vBN@Q$-#Izf!f=#?#HF&!K-Me zof_}OY2(oT_i=H}{f(y~WYc`PwO$8$B&3oeVUnGmjuVBY7YWG;NZz%o&23IPSY2qI zWc1@q@%bj`yj9OMv$O`J!jpjAZ}7AkYLO4K7(F=RVA$N^iY|1A!JW->!QL=GB?*;Utot&OD)dW^b&AbKze=Rgl<-L>46l2p!a4qa}%ynbRynDGdTYsrJHbILv7bpS_u zInL?^-f#y9$(pWNk-^|1P@+AX^sstlWj46X3K(x)7~8_j{F|E1xM7#607P?sX zl{=eLTYqYh>T+;gmQ}N8E3^ z2OYqA(8P6p#hg#>!rQ8OkIgkWFWtS-xmNTrZXCApxyR)?>N-T;P4oG;p9Np!F4U|= zyBYG2Z}N6ANF;xzth{Xi_s z;V|??$o)HqawdxI4C{5kWCy#aPABs#YuALy@4wIMdk!AE?3nxlznX4&5Un-6ce}9f zY2_}y_hMJMMD^l8rzdrdf{^9*VM#II!EchszwqUP5|YCcg2N7qkXlFC z>EXv|VPcKRg!~`ohQwz8<&s^ksI%3X9D9C@W^$=@ZNhpuS#N?r)zNaCJ-OE*o`Ekl zUO~0Kfzs7B20Z14ScwUzStmN%y#4rMJnY{1^ygzyGSOO1d`)Ioo#?ovafOQZR0)5w z2h~>(jR%rQAU+s^R3(yk#Ca@*`C>4?kzXm^!bMZ{cV(j0Yq_|*u=_K?OSi_F-+CdH zg{ihXLfebLm+WY?>U?oepAhPUcJ}-JE9j9-i~f4G?O8QP#m&LkuAdd<{p=pzPP_ND z?HsJmO9iD5y(iRP;w?a1pp#JQa0~vt+vqUSIR|kimRc;!E(F#1^5CgsTPZ))P`2@d zO_Yf3UuNPQZwF^|g}Mls7{nlfao8F{c)PU#4+>~>zia5R)Ya9_P)$RRkX*(4WYibg zXdzXdSP(FRaZcPI#I)!!@l^uZAsuVjxKSlW5er|{*e=GulHXzeyIW1)Ef852lx?X) z2wrN^;ZGgsN6vPsu-O|p-~Sefl;TZFbH)`8p~2DIAtH946A4?;v8URzjZ6s7{C;T#y<<%~a96vbKdz6JlS_#>x zeDU18B8GM-LNO~6V#5$-vu^SP(R&*ehK6II>Ud73fHLLr2#7O8*=)Be%iTX;ml*0P z=Dc_)tI=3;IOSQ4-ghLPrI``2$03O;WW{OzA@z7h#J8&lWTg~BR1ql_Wu=&;usUm4 z(h&0FyYB;8=R<(K_5XK^@9V(y(N@)2{^s%E$FH%|I5Ug9SP{wI96v;Q!Trm`hu?L9 zUdWI0w?vQYN}I7Ts=~tMgur7Nc7*S3a3=fiwfN)^w|2hYbWpvKvzRTL-*ms}ZtWVM zn#ld8^Q}E9JGFg=1cgTsE)#d@zSGk(d| zD%tAG!_ONWO2s|n&QT~zWaiD;qL9tM@$?;Pj4#7Edv_KXmQ@v=0mC&EdwTfvFBRLe*0mqmknta7Oiq$8|qjIcu0U zBtR5OsRIom_lytJqYbb3x_YlW-TqCt$^V<~W25L&j#s95Q$-VpF(fOwMEhrms>rNY zwmfDvYlB5@ANzuyP~bZHChLPxHz6Y^Jm`-_)3Gjj+*&fLp*{>#EKC}31Hh9if8vC- zZ7}#*yEh5EHohFWb2;#c)!1e3lLss%Q7-I5_w~Lhg7lPrVt>AFE+pYjO{$7x5R( z|AUJ+|ASHtW!EkLgNy3_lfCc6{|4Iq2N(Z?y>#U-GX8^$ul|F*7t8+(>fh7<|BoGr zsW9Bflw^nTZ*h4bHe9|bWjJM^r~MD*_O=;2te+*sQ|uAH>Fy*2U8Zqo9+v&!J zs~Lw`hQ7!==|Y-d5b63+zvxe7)U&9ez)GKmot*<+ck+PhEN%YnNYHyc&21YCbC4KoRCaNFBVf$6s7z6r zv^?|amIwfi4A$M(sr7WJMGdOfO>+r%P&Q75&EnUcw;M$B?Zs--cFvOOe$&~Y_of*R z&n|9O(4+w-jko`waFJc~a2uTYGNt2!z!%nB$11svz#IPsBa0`%5yxC0h_b%O;dn=XS zq-$fV4;s?^@H;$iK~KNCN}HK*1`KhCZ{;Gd0-5FCQz8u5D{1*g?1&V*LxTa^&ixTM z5u)ljH}{3VBJc}W*A`QYZtC?A#@Oo0(lis(E0Y2yKw-3&w9E$uoR_a=yn7E*j41+} z-()^tPGM3g`{1m&5>Qfp_~El-x;8koSa~*b8OM?;;RKM?=#EN4m-66hdPSco&=bog z9u@Yuk+m2VUuwBQBAju;I%k*IB(eeSQRUt7D5tyk(?mDL0)z}KEbW;HKF zI3ZAZQXMd@{3*s+w>BhobDFGG_Shn2sfm5d3E9G%bB$Vwm{zm0NGC2Uah5%`4HR0E zt0O>D7to98=~Z>X=_LfE?-wXmmTq&^0=lmZ$_MzXt5a_qH9z*N}K#xk_-_}mBa@IqoJuPxI=rB@rJbhGSxj=h^M%4_C*}2-x#{b(j;a$GRUo3f;w+5z+RqvWwNiX(f&~gc zT&BfaLqYc`QnEI9BL#Eja?>#{p(?S6OJYcH!>j_(W#~#%>AYpD&hqmM{a#-{71t}9 zs%ZNfuVz=iRrM^;GAaEr3jqp5hGHP4yzv-@+Qk`kD8UV{+%|wq+MZyKJbL(bz!fIlEQl}CB zCh`8|2@@gtvKruoJyNIHBC=#i9Flr2o6b)({`|KgbZNle74FrsV*VJMJ>dP8!%H_r z3MehRm>p0jk4*(J3%bD~#9(H+wW%rl|5Dj)aJAk_`!VzwZ2KC)lm1J)H7x{hYZ{B{ z21GzKX#*u0+@6S;zJ|rHt7ZRExQa2)^W?twdFV}|fKGgWGE=D?lnY2BH?T)hU7Kl! zGdYVczfi+LmQ9uHm` z^l3b~&sM5g87b7?@5Cacjr8tIIe9&cQbfzaCW@!fXdyYd2SP#u?w&8B0S?oT_vgcl zB~?9Dm&HWGK~3G_pNb!1SsMv=YmYXI9*|x*(}j3QZg^EJ&6++27tq6zq3T&9z=`~a znB@|Xh;z|{H+=^Jf74x9sjaIjjsI9x3D8WWq4G(Y=oytp5_u*>w66h1OD|~?!Le{* zQ*J)`SHo{gdZP3J3U(L4Wdl@ZOmYCwi^%S(eDm=;Kd0jM>e>oM)UD(Bq(`n)kxzm+E z|K~r|(b0uCSxpRO$TF&38==(rNDjJ`qH(>}DVeO9th&vNA4sAx2XfGuGx|UO&6)dh zBHGn4r;)Gmj@@xH`Qe8cJrte_C%Bs&0fn^@hzMzW>XX1KYFA-!N&(7)$&uU zCPHcyfUKNWj2txHS9bS7>}|#u|H3~%bxpZU;S7uA1DiAqB*ZkN5>(jWsKd|-#o5J~ zbf~35Do%|nJEpHemix*7Q2BpcZ`n;@^=C{%_*8l$R6+U;!!<6>Sb*bZOWq5)pVW)6-D z6kD&L-JmaDgK)L%yhtp4zr(!@9in8 z=GG5c8y#p?U*xs_Jd;ZeDNN6CM|nJ&`|+DjB-x?HYDXwqJYn!BhonxX+HGPi%S!h* z?Hv)VL16LJ9JmKMZPQ>Z#3}fqaM))MeZ0qAo_cSywwK4NJ62&A@N|85jld8uh}IvSKM3hfF$VT>%ffgq)&O?PNRvVc z%pzSB-!xBOsxuLn9p_mmRoU~Y>u;|_6Ax6?sGuI^TZBQAXjas9FxC1lUw)>+tfff8 z9MGHk}V`dwxSNg)kGwsht(E^qfJ`+TU}UoGuhGE7EA$`Fuk0e(NId)15PQ=G|qFx~-@;cGD~5d{uqUhqM$UbCj>Dhov3V>8v$zh7ft21;mVaCjOG|CU+pyHIFBUk#Fz)xz`hvY2wd7r9ul2O=-U@l zRzHx*+8>{J*3=wek5vQWJ^iOq)$V)*DAZs^&!o6Xkcpzit$EUHkZz!igjwG;CjEJp zt-UlfyK74=8Ni(BfNnC6DFWub7E6Zou&`3R^fg5UalOzC4jW-MG&hXG`y9H88K}Rb zMD2`=oApxHlS45}e#nsU7E)m0PW3+642y8O^XCOlT2tsNk@joP6-pW^YSbuKQgz-- zZdUS(gr(=NBYwDTLKPFwl4bRs(UupjvQ=(JJK?I7CmxiT0I043(4N{~EZ8?QmG$A$ zq$Y^hRKAy5Y|Lz;?y`~Ks=faqgp6rK_d2X_2i|>cE`BIaS$C6wkw^qgnswZJkbPX5 zPd97OgEtL(0!YNm@AY$)+t?x4QQfRzO zNSU(tyoEs*NTaGDkztsv0xRyB1MW8ih#YO7d;nD7F>@LdUh*}uIIZB#t-FFj9OU>E z%Xx))5RC>2OtSV?$=~isT+OauaJsfG*5yBE`WmV^bVI%@ra@WOe~$(GK%g=;i8`>R zkz)L^V=Cc`VRwOrUsb8oA`ac8UNlA1ZQ9mooUn>V1VjczFV|bvuP$bb1*Gw&kI0|N zhL90?1HN+&yK|QTB!LG4A7Qn~4CO1Nxj2qIwy#96hPb^2AW*ozo_VNVEUx$=TO8Vp zod3N)mT382FTZQdGut-5#M1fpgrSs|iGKE<{Vu)TGy0_*N9{jV?ume|iLLtzy`8Yw z>zWo66-w^KR}a&%3=hfUl1-iW6_iiLe^R9={Z_0yh}tzjxEWY=Sc9o-C*Hr+NHLSW z(;WtfcosoG{y8ckX_n))Yh_3mSj)hSDG=TwVCKF3!wD}J^Za>3+7Rj(?B*nsi_Vnk z^^hI(^$sgm zH=3Eo&n$J|S6!~9+kF*84U|i@v2Fwv|GIOtQ*1EZgI{y3N;Q>GD{E5+^oka7TQaq> zw4zma$d?UiDaUwGO?UHz()YXtjU^(Ay#$G$iarveUk<;F+xjXig?wThSPe0J*<_Nk zUeqU?4Vg&PMns~`ik9<|UAVnhFTMZk!_3lcThNvL<(wS7d)e;c3DIVYE0jg>MoZaX zUwWMhI^sG&7}KS#c|{;Fz&6>{?=7|*!@XgSy&IA|>?<~TD%pRbV--C4DC?8wA*(_* zq$9W)pE}CuoKEz$ntH_+0-t;s?;`TR&}A$U*gNa;ETORg*!_$QkdxxYvhV~^UcCJ3 z1<70t&JPYLk)-ZwC~A*V14OsfXysoDHKz@fR3=Lr(dZ_Ga|4FNYV4~C5tSBr4lnZw z(^p8fG@q}y9$3K@AG{B{W00AvJ)0Ewh&&K<8~J6a+7u~kBEQJ~c4l2LoRLn&m7!aq zRJ3o8VfIvK0J-JT+vkLa=2O*gSMx0%I7cDWKqg%8OUNZc0MX@{T!ZAWzV){SHM{DE z>>1@vJP56mHz~0J`$iNc>+09%=G1j>CwXpF@_g#hgM`e{Caf^|<4S#!%OlUPca`8i zj7OH!9sW5flWj9*cmdBtX~j86HoG0tN??}zX8A&C@24JzUWbWJO>87LeTgj93t4qgowvO*h{rbx4Mp?V^ zz1ujSIi7D_y=UGvy@bxw`GX4^Z}G2EVUL{-Vql-Ay94=NccuKgzvI>tZWR zWhrF)+vncuUdJcKX6EX9W?%RZV-IG~3w_n{SYiX`QX+7o5fF5pe)_q}LAHDUj?I>T z@LZJhNa-7Qo<;YnmE2f*;>Qx2n4uDPCZDb_4)U}aYT~2OI#3)XCqfZ>x*0ek*|&NK zEe}o_&H>;XR1Fli#*o(e)@eh!eFt0IMSZ&oPEeWFSGSu=@m)S&Rj$Nyrr5+a zY?);U@tF&DfX45{k2Jp8)56_46LviDaN)@(y0msd;I|(8>Jt(837a%5NC&PBlVY<=)0M29nNAHQxvJrP6{shkNYLmf9K6gA*=NI&EhHX6j<7Z<@ zY2QfrA!uOmIoTRlYL>cPYQGwwaBF96Xkyxch|cmU?uQ#TV-mjvlW{x(o(IrmCyU-y z@SiFT8l80vxd*~0OiBks8;naj39Y{*Jo9425T>W67!<^y<@zpcIs{h21DT+J2vTT6 zQ6#@_eeXm9NZxPF13DNCN;oOUDE}3&PHlLVIS5ecTnG-{LS3 zv*ug%8nYe80KkI1;;qk}MhN^E~`{W|HGPj>RY?3`$)eSr5}L35IUP z1`Ctm_e))HU7CSVh>zDyi2EmspoA$%INg0|(3P~rUS3LF3kGL8D*CI?|2xF!(i=Rx(w@ca zK8{ik&(q0dE;Zgt>Quk7{QH zY4>Qfptze-Bs6O7yd6mTEc#)?U~hGi9b$q;gTWAuzTu*aPQ0 zi!s;|+5oS_4P$bK`coiQyPjip;?heNkEF>|)Zc`K~;Ry<}&`STCQVqq3&Yj)^AHo8V%-;=7kU#u_B=*7aaKTOQe>}+!o zH?at_HmH1Q2{m2cFMuW02e1eDgVpvPsTSAX*f5km&LzFbKbr}$QM+!2e{XyiJ7RB*q*&Lq+Sc&(K4E$#DaDZ^Ye}VAgkW;#mPQOe+;>ceg0tNMeK3OP47(T!jWbPu9b(Jv|_EQe2U4Y`_!+#zE>=z zAzJlM?+qzaJh!%pv5$M}g$Fb3=W`XOzA82V0ojyHOas+MrEHM;4f|b=H_#x?(x=5@ zSG65Ut7cm(xX~yT6q1B0Fsa^rU1hv$w6=z0Q25q`MugpL1-k>)tuE6SHg8T1{ZT5L zPB88Tz-FZiLkO3>KoHQuANcnUXarSYxAKni zAg)+QhGUR4J(Cc0G00RVX!rPh%^7GtW9KQY%1OO%aFAua`agx{22cskJ-hay!rag%5bW^Wsh%w!YpK1LG! z;JHo$shpJl_&Y9rv4q1QR`y*qA}R4%4;x2=LBnTaLQ1u8UZ4N%3Ag=A|$yM3#zVfOhm02xF zv1;m}Ew+f_)fnB(0QU)64@!rGJ0oGLVF6c-UR=AMW&N1?aSpyZ1e;k^lS;YxXk{DsMN0gnE`Me&F2FdA2l?#G-cXz>5NFI0RxGKAbL=%mhK$#E zfp8*wF3kxID=Tfi4KLG7e@p@sZRPlA>WB%A)?SH?`A8K14}=j(H zHZKo;btGl@Fj}N&b+}YM6oaCdFH?kxsCe$iR;Vyo$L{Cwb2)$@Cwv4@VHRn!CU?F71XB1~$QH)lZRNpsAwiwL>6pWUnof9E z^JZu6oSf&v{J1n8|6w_GZ|jf#eG94372@{{i@pdpOwyb2(vT>UNZ8YaDHIe^IS*eG zF+&vqWKrr~cES$N{h#~VP5-9rSeQ7t6STekn{ME3!4`9-l=}1K_SY-zcQ(I`%=6x3 zVzXT*hP~w-EWbyC`mWllM-?luETl&_Kz5P1Z5ux85TN)R85VOW9jmo#1Fn(l=B4H_ zf7oC!a20Aan^6OvPiwI%KuPtucD6L7 z$f^>EX_4+)<#7VoR4l|MrZJNP;!KtU+>g)>6PUio>Urm?>!UImn-kNoz^w=hdFLOh}Q&o2uoSaVEpq2 zr`f&R$8^_va1a`IK(NQrTM*9NYOYT*Tk*Q9%adcLLlu4+7O#55>O_t)gdv{Ov<$VnZKvkdQprN>eEVTWg)bfLb6;mL-f{hN zLXR)hrkB5RK!);myL=H~TBx^H)mFhQoF3_8#hTV^knF<39bR(LwAkN7X5M`K+L#a` z(b~$s#krHr4A(IG#Rbq>#PbJ_-@vL+E@MW@)Bhf=73Af#w>hn71CF0moM6;kBfhq+ zocyMH{`FAlr2W^cV|)Ep|0e%uho1+&KkFIQD!;90oMzCuCY2$ya5eNB%P?RIt=2+u z_!jOae20rY{fPTp+e%YNV%d3w>man2>s}{*X(vD|%WTt;p49)!N#B;w--6SGeiJni&oZ~!-(U44eDfOp;S#eU64 z%V}am-LMfScn>(!5Q>BdA$2k+K0N&bA8vEB_Q*|cI4!}%3L%lI>z@Al27OUf1R)KNeK-KtPL`EwObbCQ(8&-WO(&kdV-tkP!av z|K>RNtvNRNO(&>f_?wRK{Fx@d+5d}Q{OQq=MTxt5hQpmeQUFi&U`0YuaOp?51siNk zf34q#%)i#UTZKey%Ht8%J}Kx=kw`gE8jfamMOP&$qyw{mh!|^W|@+*+gON6jU5&9Fe!K!D~xbG zit(0jj!v+qb~fgyH0uJj|B{*FtRZzZBD33AU?l?uqI; z@3?Bo*pF%9%A~V+$rs~)4dWe|-VdkXIyKGu*<`wLZCF-ZV|uSVfhjwsYQS?)UiSSx z-*xM7jg%F(X_9`wbw8%uHUS@(8na#1Cub2eO!U3JZ760~L#XS!uMJfr25=c?DlXP@ zPpA{J_9W80`KoR!Nnjs(H3;9uY~p}BumhnJ-TkYN?&(A0J&r4f(cvhDk6u(8$ku%t z2QlBPa77SoNVvWSGp|j3rgvFnz#!ntuL48aNMx65jkvqaPYc~sdL0c{&4=Yy_q^3j zi&|$mKjV7XcWtLjR5NhAiF;S<+(M_b=iqA*S|)~m%w%;ni0=kx|2SJKYr==Y1_2>^ zbNa;DpWXwpM~qdREuR#p8Tp`8$DNi6)$D;_^TQh5SRtq@@A4QE*^38r97b&VJLc!! zLoWnE2OCM4GFv-R2k-KbYNOH2iw-l|rZVi#b=G#m%^pu?MaBmzF#{*~s*E>J0ve2x zVa1=vrKw6t_$9GK2e-~@K~S!IZ%Xuaq)Da;!#XSH#{&5(h$tJx_J9m8(a zmG@5OJEDLOc?^W2p2BV4d7mE@tt5h^qMPEg!sk^GvC`{e!dWd3NAATNLpKIFOKGdiv9S#7J8@Hxm_!6J&3Ej(l1;=nKVaRK1c#3(vA@F`q#cB%Fk zeS6x5RPGXomCIkt`xC_P*(u%!!m%MK2f;PCcE4(u zGoWMJdv{2u@7S!&Ns%yDcX3$3Mf-xqZ|z?%+@^;n^77>6-RU6)hxO>1lC%H+`8WIexk3 zQ^SvX8TVRp`6n~|V*ST0ZBHjzwI76xsxvbSdP;2;8ntTlCpvkv0roL)9a2FSdMuR3;+mh-+U*@0^sli1iS zpZD!hk@KW=_vz~}9_O^xeC?ML#>|5102(7t^1lu%W3$iQ+4MG$siXpZTzuegBDhRSjrzv{}2}JkFC5uMkxHREgr9upCcpQ0RA2l;HX2I8;jY2$!lbj)wO<0}!z5z4f$-2GcV;<{u=8{rSZ>|h zt{H~`@l{@V<&68Ijs2_*!SXl|@Icxbn<`4Z=#etHs=S@KIDYJ1Xrxo`*uC7qf#pDd z?BRJ@!dCZK1;&e<#6+{dM%(OuyE~YkPN+Ao@|RXkb}-j@z1R@YxIEKpbOV_oRqtvx zZkA?MJT_U(BeS6axlLh1K z)dA1OdqsiVf~|-zpI>!__IHUnF*B>gDka$7ES4G?+BpWAUGOm zKCPL_CPa9$D?ROAjLY$*N}{af`n%2AR$Ok)jki_s@(SNeL}5xm^*0VLzHuxokLIID z;I)T9B0!!1t~LH3X)oKGhJCzhB9*egzu`BP`fMw-%0=*=P_F_-d}%v&YUF4Ou2zH@P1s=MqS z_f04rnyo`GO)?u@yN)mRT9UH!he5!$GRH@ruH=DQM@UP;*ht~WeW)fo5f&Sk$tHB) z9A*7iMvI;wMJrkU(-(e+Hq%WC7H&k<1SlUs=}OB9lp3T6(5;N zbrv^gmWY`oH4}cU#;LsFgW++L%jD$}oc*i&j|PW!fr_gX;WB^Q{6=J+wQ@XkR zL9yG`Y~bq0fT>8C%L69a)(cXb&TEA0Fjzq>rbm&Xb(Xa$URSDJ@Fi)vHpy0#b4_t` zBKGoP`mD#a`(h+Z8F^mqr0Y-N8QW5m>A^MnK0cnVOk}g z+brWZ-Q3zozOQWqVgdsPnWnA#!9SYqU#;w{pR@)ZzHpyj$Xo~E+uqT@72M4|4wg$M zo8m{Le#w$rDcZQ;<%S?=VYy8R1U2-beKR``MW6RT;KSX&AAhm1t`XYC%m1|Ek=zZD zBV<(tsuSM2QLh^+V&maVzYyGbO%y~suL z63}=G%SZv)LIN3x-5!#mI`V8)6cM0x#?KewC`Fu zSWozC#mp;%esgLZ#PE3apH@n1p1>XrPDWK#@0EDxD9xS-(kW**q zvhH^ez52L(Jn^qzFrB$r(o7hu*}>)#@1xwZrckpk)zUou>ZcA}PpxmY6_y+>J{L$X zrt#gl>&4r+;Qi1u{wlw_(BIRtM|yAwP!%)8a3Ap+%M;TepPHKADQ1bX47pGNg^QAJ z6@0{rNKgWBn~1bU681dJ%4PXE>pj*y;jxK(;g4v@(vr~)u9UeJ6t7sq;N=EIwMv~8 zesHwfsC0EErUgrfhEG-FUkxb<8f{vBDu%JV-21PZtbR);dCSgOstTiB`uklIcl#)dDzxSUd@qt2;~T9O+c>_4@e7hxZm zW;9^F)qbIewbx*^WXMnq31maU6ak-bM>jb$k*kJr8(xrzT(Lg?iG}e9!}>UnNzmF& zL)65hu9)F~g(SE2vK-7P=%qf2+$N0VJ&>!+4oHJ727dJ4-#1Eqe0|Swez~?VhL?YA0X@FO?+i4zWztSa$X`m zk_P%=EY_!Rhl^tW-h0j6aoh=I{;n(&jHwNBp!iS>K$AH>KLD{L_!=#8 z+R?xhDp#@;zTVxlV(fNs2r=P0<>oV>T=VG}aREXk?wjyRWIkF_wjDP#FRT)YRm||3 z(uD>c#!P!@G04gOyz_|sbMrY}m}IvTB$ENGZ$^tdEb75fSIHT@p-^xr<=ZV*eW*BQ_8^`MU8mVEeffPSX2-Y?_59{~6t-cjkmP1@M# z42lW-mGo9fp$ossZK^Q*Cf~S0(79TYK-{yoa7IeYO}Q*nl9|Mq>5nKswVB)0fZ4G)LJRR@RXeZ%T#;hDDhS$1o1LlDz9%ecN z5wB$tpRs$gz{Vne4;BV52~Z^&28SB7%cL@b6#xEY#qMS*^K0Sduts{OU`HT-nskx= z7P*vj%w019Ln233QsG#j2m;QMb)-bF_T5|$?Wy~P@{{Fc6E9LlJy!CxGU3Mct*x~5 z$pWVJ0D!4?AzMCu%9Rqib=NwIMkrw=y?}esw>t1`i_Q3%Tr2b=BUOlk=*Ym*@#WQz z2C_Ed^x1GKRc{F4$T?Oe?F0@~WA#KB#R6IH)-Bke@=JqL>Ytj%2$MHnryar~O&r$` z(uT}PTH}n6{XZSnWGNMm{mPp}h?0$;%Sr4*5Is(y3k?gJ^}YQo)B5R*Co`hNX(W6a zX3;*HeW0YsSG}zrri>Hw7QEW&zY8_E z5*oywY&YQ2*R&s6E#L8^Y%uoO#8US*+TZ~6sM8O(&N97ra7n{xSV!Wfu~3zmT?(%9J4WBL%N#XL=r_6c4e&^Kk~;ftmZY$y>NFewIqii z2-PTCmB|%P^STQ`w@h3ZbfKy#pEcO?G%+ttQhCfuybo@VddX`^cjWYJ?|6D}VOA7c zDaM*(zYv@c1c|BEzqAcpny7hkKQI<++`JXgP>-jLHC{K$9kV-=SB|Qpm@ek8s`oFP z(jxRhz{o=@v7|3ofSo4W^{J))m^Hvhr(Ir!1n!hbnJOIrP#k_Iol(X41 zJq2!7j7uBke3~7c6V`Vek?x^CI>OmDw$gaIWh6lTHXBNa@&7h4k|ww!hT)848zPgN zT?AA}UY^<|r2!n$vSSG&Qe%m3wHby35<_KGKF83dl!?vv?($JjYH2HDfRJ9>lwt)g zwsbv{Ge&YlqK|c2savMcg^+D!tn!|CsuyMEjgh5%L66Pr-TCy)8IP$L`q=03b{mS) z3>*_#qfXz<^+3h$0t%Oev~XiWO1HJu1V933V4Sqyo|IZhqMGoS!P(2R%g?xsv&6)c zA*|Z#Q58;0tkba|ucw(-pzgjn1Gc0poR(?4f*8HWAq40!Q17SzT{OvPuAkeirn zw7X)2x;!{Z%^-FBI_=K!(9e-T<=l5AUZW|rj*%j;pRp0rFZnAyJaTKR_pirTj1N7H z^KiKq4ow3V!7O}P#`K@fJ&i@Y7EP&oW!b%1?=V;(G2*Lnl`?YIS8^(N-FbQ}?7k#t z!nlAq=iEy3xwnt0aW85#tY_w?|C?8LJi=utZ(^)HeDojQRmzN=e*D<3CvFoVJY|^2 zscD}-3h^O_?0>ZSZW_BdHXw}M5f;y}EHhI&XaY$oZ&9AZg2^UM{s**2Mb^peG%b->+q-hCi7Anu>iXme`K6al?0RwJwRHJnouJN};5^ zBuu!UFw1C7YPuW;6o>Y(2Ed?lmrq85UMnN9+ggj1PF#IkkCQt zRZ*mcs-a1js&pyR;d%M3bKZ5Iv+nV%yY4;j{o~C)$x7DDWWKX!&z_mRKf7>U+r}sk z1(ez}DTBxWftqSU{k}lhQv2-}vq2d8eWAxu^%}+k7K$QET6QZ4k%3HTJRkbWxR4!I zO1ZY_xg4{ILdZ2epRe%_tbX#kWQ6?9@^q8>G)3?x%c8RC`C*G2T&~VHhxz&N z7OgO}RpycOZ4MC~1M3XeGhe?dk=A=rfRNs5_r#@p=X&KEL{L@<)`H-~_T~@xH2!bp z!tXwc8$_3czM3@CNtna-zZ@(?ju&Y!&u_9MLFBA9ecrt;_=K(7=SrC2!Ad_!^ddnh zOZPL=M6m-x&g-S)Kyv(M;9WefiB4miN3_Rh^|m>}FuXM+4HW%L#Y{wsJmyg*QY!6O zH>(IpJ48Q7(WU*XnQ1VxL* zsN<&~;ZGJg{jl^Mc z+Ug5*8iEP8l7tC}dd_e~Yh}`#+x>-nOP+o`m|~FbI?`@9jyfGskDRx7cyXqwOn*~H zgx<5!>)QxGxW|uI(n9^P0 zHTVoOB{?Oqv0wKMsE}@2ZZNR0xcRx5Uqsop{$x*AqB(p&f+J(R&5RFWc44O zr+RTdczC0&!uqAt{zM>JcACD?l3hzXyKs@ap62UfV8z(W#{PK6(?Je;=%G9eMi^YWIl!d@(N959`amt#N*nM>j^9G{Heo9aMp|8 zEuSXl=|Y5`U?)ZTDD%Caou4rCM*}3(qWFEO(!5cM^58W{WKvbi*=fB#%i+s8)GVR{ z!Hkbfq+^xrd-QQ8RYxo4VcwM&`Xy1k;y)hTBOsuf{)AO>y1b2g6k1@9i1ONBC#Mh( z+l!YLVV)&l`#j?u4B(B|M$hEk+imJi6i;C{_SfqPY&{+T%xBN>erU{e%E*)rQM>k< zZsGc~KV+U%R5Kq1xj#1_>Q%M8(%dLYVjH=nq@Kdq{Go;vSCINm(|Z`=6rC_y$S_eW zUa9lueui|rhwQ_vB?f`fR5VqZ-ut5lPah!_6hYENjP#3ppY9G+}to85XE99dZ9GDjWGyDt3Z8;I`Ol_p-7fuav@x#6xv14vigP-u1THSb$8k$bA{SwkFFfY^D zSLtyD-q2p_(r{i+n2Wv1Czhnz&rvt(CKk!Ew4usXKbF&8%Akla-ib9PF*&JsUW3p*7rkJykk*gSHypKr|Y98sB_pht83(3!D@dxzF zo+o-ovp1KsyPB7Dj$i(gKc$>(UcF-e@3(DC!Q@u)?WU1On=c=7G$?XXtaty2xRv9r z&t7gVq+D^8;GE+(2#TQB)^snVZOh&4K|dw9jA~p)?NtdY@a;{w? zUYII9ZM+%}s5#pJz;RVaYN*@QVxrbvgGV)A#Dv;|RD5f5Zw24B7`)As5RN07Y~xC= z%Clz-!b9{UKS5R423f27a@FxLc>H`a$Mr`)qsKJcr!pRIDnzLWuDV~4F} z|E_uw#aBOxN?&K;X|?aBOcMrY65+Y)ghh<}Z>K>8CgfIj>Nm#v}-8wuEg(336L1qp23)fhfe-1Vse zI-zsXg9StMvS5ZxpEzl#Ae2LT>Q?C2!|_CJ3y|wI0gaZjdH-TVrKFRlh!KvaYcPrPrWJ?YLeB{%y$G@oatDyQ#nw2k`!}!b2Yf{eDI6BT>VUctEDCh^wi@ zj)g7jr;fabb7bMUs%^J8=taY?&!K$QIY%zR(WR)b#4J; z@?0UJ8Hg2s8juYh)qe9v{)c8|ekB@Luir(*+~Wq0==-ZlNb{3qVxQ%SOBvMO}?P_ffd$gR#8G^+1KIPiWk+iX0xm4YrvkO-WGxy09?UhpX zr(X@a33;8JdsaBNpEeR8Is`qk*^QB7hq;U}a?qD)_dF+^v5BYR6 ze`;GsN+rBi_EE!lUL{~&TslxJKU&aXQs36BRpWUM18er1SSUy za=-0@PA=;Idh+3iI74e*z|(D-f-TdQ{g>|2`r3~Cjg5{Tj6$BNM!FN5B2nToX=fyX z^!80slE}p~D3Tt=IT@a~O?u~86YR(YYkvg?U<~f0F*M3=1DUvLy-(a%TgIqS1-90& zStiPf-SNO3iY7|7yfw|x5Ysbpo2#*?#3R*TBUrJ(e5iaPMN%)nf&)P>C*g8n1?$ud|6FiP8bz6slim;@f=Dtx{IVo{=9k4+ zU_)cmhD1Vd-u{l5CgTiz2sOc7_N zAY!8VVqFJ}PKiTzathl&CkFPVe7oLDlUZhgT(o~F>#Zh6AH6b8@}MB^goq1%iPo2M z!-)9928R5VOiE+Bx@^1NI<+k59x4&3mx^oaML-hi93le16x9Dt~8% z?(oDQ4kbh8+}?Z+HI{mz%+|9#Py#xWF~(VX)2%V!)m!|eF7+nhLC@2z%}TB&)p0cX zq4{9p)#XC&L2byyhd;jiyAGv2AZk_PsxRivm-{h(zuS~KqZAcjRC2~2yG1SMiZSY? z$bMnwOzt&Mc-)T&S%25n4HhzoJk4n%m41l9e;p8n1fW(Gn>um7NLB&uXZ2bM6J~E| zLg#Je0{qNz0|uR*E^-VFHOhnqs_*U@{!Wlm@;99tGY8H>vU;f`vhdZfqoK8osKEw{ zF}Ayj$`Yc!gybyq?G>l9KNq zcsp&HZcDY?BievYN(d` zf#bQ}7fDNx+`b5XTnBWnp$E9B6>5sz@s}>RRsBSkJI9t?4V=T;GdbS1%-!06VZJw8 z9MN2VR3?E86A=M&cp$Z+YKQAfeiS!!#5pDJ_O!!S@Qh1auh)1;OrRO^RJcK6tLrwZ z-AJ|Aph|^$_dz$whHMfC)&G{)WumrY86HjT<#%sqsU8qUuUTI&#c} zfcq@p@guXk`2lu@NMLmY|r{9Kli$nm0Lod@zKH2TvrZPS7&AhA}>%f_kyx#g0Jl;3Q487IBD=w{(@mRr{)s6zv^G~!t_ae(QQPYlZD$5={NU!Dol-*8shISQpZN1lcH2+ImxCmPXnDU|QUy$*Q zJ$y+}yFJ6modA9P=l5Wq3;pKL`!gbvC1naCJ2~GKkpzHMO$eqW(DT{p6loQgTUd)j zdJ&9>1W8I&u-r0tfY3JM==1Z6QCmE$tgEY!q z@5jH4Us(?Jc<6QERl4gn54fwi0Q!`7HTIX9Jfsk-xVp@Rgr!UvOFoP(Oub_&`A-GN zuJ4khR`?3EG>Ego1Kb_0jvo>*{I+WnNZ#L?hhIyj=Ji;Vdz>1K8{dsV$)aQ#*=@AYx0Mv+MOo9=Q3%eXCn>)I&l8a%Pjd zOA3dClYjCxewS0+EkniS&)`he3cLe*$sM;J zd>OrdXr^nSmvaVgQk~UO_!|U9jB{ls_}?j6`o)eV%@tsWLtzgcl26*)sm2T46$) zN#GN^S=NwWkoT9eKv83P6yJ;3r7{iCSQr-=-;=Z?OS=3;pr|>(XI{eEsgNx-KTb+0 zzsw(C*5xe<@t50%d{Ptg19q}F%hfE)uDSv7CbijT0lYLw6>f?SV zLV!E9S}BDmiz}v7G>9-t;=55Z3i#4P*U(K3&Rs@(?D_q+vJN07Ne&4y9@;12k z5xzg-@NYU-;zXd=`e{gg@bjI;%2^|zOPpPo(%3uI@QHCN6Mh(`eN^^AJ`Z>z?m1jj zU-YvUZ*>PtNk%avWiF zsxNMw*(f~S(QJn>d`RrZh*7KZl#-s$KXuva+0l69-94t)rW~WQ>p78Lw!pF#qF|g^ zTrOhrO4I+<3}3BoWGZ&cxca9aqP}D$iSy&1&fNuCu=F_tHIX05lGFkcTk)lukDbPE z=X!xmHSUD;Gem?sEiCm`ndtZ@F1wvPmkKskVBn%BM-)FGe0iZrQBGn(hRyQ<%y*9R zodtz5D7-k6Qr~$+$;CEB?sAI0_nrWMPM3N){83?qUBvuDkrC1Px5Uq1WVL4sJ51L% z>JLudtEVKR49aaBf^bal)95|E8+`{+w9Jq5Wfd6W*B!&4A9hAVD$54 zTp6YuLl4BHJKZ-!JbVu-QCWEQpaD|}X{SzyzO)iaXsRGRO(iz%d#~MiS2>$FDr^$(C~%)eJYJ(>wb7Kb z-*dT$H{Ojiirvff6&%#-Gc>@H07VhGP{rp}bIjrzte5IuSabt9wT}w#_}drMg$F!` z;0+nxFIU!^bgE}y``lcqQ@f*hq5Q$Gb=f-9%cZ;0D{hafb~O)!zFLBije$(u`!Nw# zk{TlM5@=0JUcVY#-IAZHwUMU?_nF9S3yg2XE@!g6ng>65FRv6bTGihfqk<|VzAB%; z{~b1b8wg{)xWwBI+g>D#h(S27CpnOqsYzU$ zg=coo?XGK-Xh(z=SMQA0fBWopoO!5eF{9!pnI~YVQ{(|J#)PRt{UqP`V9>^(@w}Hp zsZzg~Lg{w}lqu+PbMP+zrQIro91%5hK^bt1KD z!J=oxJ`QnOT7jn~ab&elplIXX0Q^zgR)^d}o-Uc##B*XvV0wN}7dvCwnIWn-!9J5+u$fS61Jdvi-zV=v zZ{>cC?71?1qfO+ejrE%w9iI=~iUjxp&hHPWywV`<@KSB$v%l${@3WspT?4~+pLO5w z=2}{_aN6A>v}>kX)r`tj5$Ac*`!;D$&v(vgxnT%2B^$2uANF4Ofgn$10MAc>k*~0T28^Ro;e-xIqm7?AGrGXtdmPOrCKLD-w%v;>h`w0#W z5Z+V!xvi?I^J_zI);RCfuPsTN4V?;#7YzSMdUxXPHy(#&|ELSBn`$5BdiGN8HJ+*m zxjWkH+RD{i;Kf?jWza29wA-o`+S4y+?Nl`GY;w!}Z#o@&p=)?0$kN)r!DTYbi*pdI zQnsKB(qi1~Ba?4amg-S9id}3?|UFVIs_Nic{AeUxl z8KA5^AZi-(_;VM8+$tYf=(-3MkWcCU>vrpL_&a#`GVJ=AUc~t<=C`F?2;G{3pwO73 zBE50}=}AhPa67@pKV8O}$GA{1>8C_bx8!2wobkr}o*)ElRgqNU*Xh8L%_;=1xJrvE zw2sm*b(29|s;HL`)H>)$;T&-XkFkB)^G?lwidz}#bsen5^D<)6UkQnfusqk(Txz%e z9Oj-o+M;7huD*%7)o>+kM#+%NU~^nFxL#q()c%<4WuH^ZpI^dD2mufVd@WL7b#|wW>1o6v21HM*U zOW1z!7WpE&vWBQua{une6{S>UBQzjU*CwK0H@?P)5ueL42UHl!G8aCGjCVhkA9)0L=Hs|6|8mOWg1znu4t)=a;{Ipyu1#=$zK z?|c6x=eyRC!Zz{P&IC%K%}rX;GexCMO2Sk%+D`jOBoAp5Wmr~rX7&`s14(*_Ik&vS zM_80z9*qdgJl@Cier*1;-D19~W_NeTuQF)cH3JoMkyQ`yb_E9ImA)#efEl)=AAJ076xBw+7ZHC}68-NHb zl?IPj5p4bj-j}lr;A`yK#Vpp_a+Y@GlgC7%^k{G+*(9({j69Y(3J8|l$ZUK65pphv zSc@tq?o{0`Tje}*Bh!&MXrimGvOlSA}FjFZ-Z_uGO3ozqoE_aaZ+B&>B-;H zSKT;rSS|e>8j3DWBHE%^sP!&91MX=A_Y&$gL9jkujA!TPW}uToIZ;$)OjP_6W4BEf%{CUaT2P$fT^FMJ=W;m3woIIgkhR9ipcoM zPsG;jy>v7@Z_p&^eahAFm6xvS_TXNq0EB(s`GWU3N_C0#la-Vx`8nKbUg!9+kSvQLCr$g-O->t}QGvE;l+$&sSK}l%NvebCiaWoEM0Qw%t|Ybr z6^!gpyrIw?1>x9~;fI{K{d&mJ+*Hwur7|@?yA)+m&F(AW^(qNFA!rLGEA(GU)96y? zhO5|Ju3Cqan8T+on9?SOk=fsVdN(p4(G=Yvn}C^0@dfZeECd^d^&NK~P6Wqx_R_1w ze*=G@W8JIXfI3O8fj}Pd(QwZv^q7hKDhPy*}Aio%RkR!a4WX@r4RCYw>d}9KpVqkvmuO=i3g82 zoT(F<<{*CGP-R`qZ<3lbN^eYJ2J{$-i91YKwAPmSPR1yP&%doyPe5Z(K)`-MWcL|O z3?v{`b%lI666ZA_M{S5rTY9*E9&k{}t5pB8W%PL&E0c2jh=FlMGZy7wAR@Y_U%PeZ z$m4TEfMrxmZ2SA5w-oCl{ErI6r~EM{=LNHt!9VS5`I85`f(85{cNzhC*#{m-#gJ|}`dovr=( z2dz6b!ytqr--2j;<@LKVOvC65>nOsf*Kewx zDtlOe{llz|Z8#Fg&#yd=@=FwSt^_htrizq2+u~lxB_8YT7q5;aexzo!=QtPO!Ybot zer-AHK98VSzq51zn-*ep19`p<=5sDmz*VVsqwfV1K|e@X{{HA4jxVW+oO)=)qMh0q z^G~aSK+ZY?1(G=K;_*?bvMfBH{iXH%?{zksug8(^E83#Ga)shr!v!8m1M_ZXt=5d0 zyj<0U@JsxO>h;$xh%rwy{_wVKRyx_zP7S9LtCcjaqb#4a+^0zkNGDghncRFXSgXTg zRSE$z=HCAt0YWTD)?&4n=IgJ+*KutOd6joYOOedL2LRBl&E#`GDexQ`lEo8SD|7T4 z1Pc}`iN-;^#$}}1w=D0<;sbBU|Asg%%@&$jvbkT)x*6Ogb+=Pga~W>@Zh`_GdaTv{L7&pfY7PQZC@$=E}MUICVMwG`mQKneS6xw9qwT)#-(0Wgp%ysm_6%75Q3O?P$l2*n!cInprrf{>jX0 z?dobEQ%=iZ({to)1bQCbE@3Hi-2>Wdl!YCXJU3iai83bds)`B-j5}>f%iu@D%T^2*mP(JC z(3gNiIYLuft(*Qo#$FA;0IyINBWz63dTUi%=_ctWb)46tZ#ee-PAN>z^m(UO-t1+~ zGJod+GgtsF&wnn3R3kc^8s=gtEJnsWGI7BizY=T>TU8?Cr3_zM&k=kYm6Z3R-gd9E zVmZ9Oi3~rkOa0+Ry@rv@HWWI%D>I78no&5w&IgVBMt}%H3h%oSjzV4dn3k845yos0EbPP^DkE16V>&c43-p& zlzAbJf_Hh|sg)7edF*4fA|yMwL_l0YL=*ePmZO&7N^h8mkkRw`;M+6TVtpUqErxXCx)jz zV{Y93Hh~zPQX>yKYocX1+aHdibGaW>sDzZ*5$%|tcXUW0d+{J60eXhN7d7= z2GT>VC%GZWYCsbBpV<`tF}8mORZHpc^!qkkzGO~RiMJj|GB=-Uk2X-IAI8(*k)9;> z6$j7iS}A_tQ{}*K63Wt!ecykP!eqsb3%p*NSrqZ)N%kggel5eIonLoi5tA@odziMX z&zY`O5U%I7{LR$l@~00IqL24&ac8ianCfqkW93mLQGG`Sj-3`dX1cpW-(UL4d9t3i zSbI}i?_uNNcK0Tv0$A-P7&*cOwL>ggjN`cy#v&U$@}2aksxUymw;bU3cXG2opDGYa zaIVXPi7AC!+&MDuOh%^~;HoO){-(2<&oK)~)pOU~)`*qJhtE>5XJ#&L;9^VS%Z>&I z@FnqePnSu~sJJZs`3AS8Q|Ya_pJ#C<)93dYb$-{iQ{z~Rg6CWMR-AASC(m*VO{YN$t{@0FLFp6TTgFcz9SxXooHd zi>bpX5oD%hvkK$<){L!eH9_!3I1_hj=@B1K1H}87rtP1&to> z*3fJSD@lM_5g<^L8vaS`r;ouQ*D6aMTmxqJIv_1*8#za&G!0a$^q~^X7l=U-=P?;B6DvPRF1T7jJ@2B$J&HtF_Tm6X zy(qsB9Cb1G>$_86tI;-0wX$EhQru)u6jVUv;|B)otEux8Tk2Ci{1~+jN%d_lKtN>O z=C;6{-wC!e&p=4aNdttVFZePI1BSL8g?7c)x9eea;E>pHbYNS)pM(KdjNhkuq^4%3 zh+#9|M{oMEWVM1P>oEHZqn`n0qZ(-z^`Ywm{co2W#5C4?c_jbBpB*fz{!N#9!}GyE zP0xQC+@ewyQR0d`pS%dmj11rjlUtgI@>Md4({KlmPx&gicPKmS=p^+`(YdQJ9U2lf zwC#Om(OCowx`Rz2J!^srO{h`&Afr2}W@F$z-Oq1+XN2Cj3?(@uZhUnPRL!3<>^5tI zik7CqHw9X~>k<4)(=T{-bY5vm8>+Om5~)}T{CZ|i)7q|a5xQA5-%~R1rk})Zmjs!K zD9!n(#H>UX4d>chNs>x{(&XE$dh^6yhz?IY;y1eLa(3bmiOHuV#jsiPDr2cK(49oi z+E!slr_BNyLBpd9Y_>H#vqQhb+vVpbzt7CayCo&ZW#BiYm(J=ctZyZ*zV!cns9^_6<{^9`InM^|Whm zzO2#b$VTd!Bv*8AcmQf2kY$l+zaF1NkT6Rs>#D9;9TwI@zsa9O};^fvOhWYDL^g`8aP?V4{ zAF2Jh06&=6?ZI5o9r=nKkAu3~sHjwKb-3iXManeM$8b*FE7elk%)|$oYr9|7RC3 zT>qQSrKfzt;=BInkT>qd=-w_@2{Iv3LRv^~1?n&w8ak{wK;S^YvMFE#pq@OAOYFq6 zG9=a$U;)dwnvdQat@2ppviy+}lms=qeXmR2=Bc&4ACvUt6LN+Q3>(=wn4Ee|`ZpPU z7$fcuEd2^hRIGk))EInm&e#>8FfTZ26TuniQ3l!RA!H((f1x1Em`!P!h*72Y73}tK zm`}Wg5PuYwmnU7^JkLX*s7394*`|u*FFmEQIn~VV?fRh8RIL%300T z4XeS_h;!8h3GcC<_H$}&!S6yt>TsVh`g_{*DPRjDLNIHj6<4&3b$^fZo5lyy_+@CVW+*2LxC%+@Q-4+c*DpljnFE<=p>k+-y0tFJ7EfXhP;g_D%~s=UlXNx=L3q!sMIo+d%=jQ+`b1( zO=K!Ke$qXPF*q;epcugpH3&ykx@|$ce=(I?@vK{#CYMX!IUKkw_L(D`#-T9zV**<( z$o_rbw>aR2(-uGP-*ms3sitoD$3r2=HF8Iy;`MBhU0lm6R0jmutBcXI$`&l`A-Eai zIhYIK9zha<=hB>RA5d_YU362AbF51p+DPgv{c4$c0u^ZNpq`p)Uk}F)m9g0GELGd6 z>XYB;1_yrE%Qo+%oeZ`fbTBq51l$dcvOAyV&g2*!NDY77yhz62}bN6~fYGjTKEq*hQSNbZo zK|!kld>>g0VtGYe7#1Z$ZkZ75V;iqnY{*Z`$Cv4+l>w`=#n+RA7ps|tD@nvu!_NDh zHU{y0I4h_4?xKv#zR}x+4vTBObp&rETMSe^suDtxBzF_%-$*@p_#3-Hq`LDh`n`-l z&54>GZbXvOanP5YBNGJdTm@QO!5v7n(piMEXllYqS^K0v zJOAG!+y9gIWPgKWqO9kyXYo(hADe3yR|L$L=RPYw4&L*j6Mg-EQuBX9S)grVGZ4ph zVm~+C_=?R~bT#8Z+;w{SZ#v$Si4P3_mZ#;vp_%{9D(EepH>gh9f74m+@2xLhGJ`e> zJ=NosKWy%34!D25=r^_Adm{NBa+aa`{%Xe3-tPMHgOUL5;9LWeZx3png@?z>a?5ri z*ar00`IJcnBM8@S;Fu8C$FlV$(x-|h{2lUT49Kbq@6^yHZpl)e?dfNS=`bI_7;s@f zZ@AKumuhMzYpa=Z-HqpH?G5+fyj9h3v<4m(m@r9N zPQYcjM?p=Z1dgaje|&7;YJUrs?FKh=j^N~z8*Qyy1>iS(+p4NcJDju2ir8^DdaJE@ zTN(RzZ_WRN0{wBV^CONdZ{6Jhy^Ailp7~arSGjkbA^A34`%LpC7CBJ>&#FvOFCs-O z+9Vfsd-953_`^{v&W@sCVj$NqrZN7I zJbrD>LS4r%`=wd13Ph&O&8cI^#bA!(KkWZ-RkWyacw(nPa5}jC47Vo|nTc_^PXcym zdE10C%(ZFHrc18Q;sCgsxpGow+o?lpsq%fC<6J5%-MBOA9J{f=Xh&Feni!CiL3tJ8 zt*C&^Xz*wl8=wm2Kn)WZk1sk^w!d>uefNUb;nh-TACOrj{cpNz!H4#%d)7$PQo>D` ze&CihmwWx0`}qqK3zP~@`RJ$WmsfoOergtvo?2<$mX2IqweFi0VAOhtsG$yVT^#TB zqj2dAK>^^d!RK-;hEc0*f3jdeYs{v_#L0f9f%GB2SXi2sL!h9n_Pvpw=-QieOq}Fj zHkN`l!29C`0$2bAK-EnB-_fzxd7aW68=h&e>%5xmD1L*|eBn3uFc7Kj{E{3wSk1w_ znuO27pE^*j^T&tv=GW{9#@xOKF@6B<*rhz%L zP5m=;mT=DrC+B28>~;Kzd4G!4ka3; zhw$+iM&)uc@XQVa(?`z%TwNfv58{B@B6$W$>-IU_WNM-HdJU91MAv$ahl#=&6A5DT zxfGF?8#T+?i`sC~ZNS-uFW$2*cf&SuV)wzw)3{%E-x*48%<*nsyBzU^=5be|q1-ZQ z^WpSpGaX$i@ITYX=lnOs2?U*(?F=*iMV=35qe`Es0js&4SC9$GP?KZja#+n^-sW=1)ai?pIfqD ztqcegb&|E57j9@P{D`ZP5Zevpv|YyM7X{ap0hdTs=!TmJ{qP^c?)Td_ZP8Bw z0hF#35t(IvHcefY3_TGCSf`(JcN3R97t~)Jhn{!U*)CK`V#5}3NCrbFV=AdcxV`;k3mdUae@ciN8v{V7A!iRjHn*K(kIND zOEBFVcW70@#`lUDGA7qQBlz-9mU4&vrTOLLW~ii1Bf=%R^3fXRj7XqLx__i!3hVOx zd1We2I`t4eT64ghcS?XMYS zUZqeT#-ace5y9ng}8=KwB6U1y<>bGfhVN}amu?1o>i0Rl``>ODF1pPgc~hh%mCYJbvhXm5@K+YMOETkP!6P~=ndDs6Htzuv_BHd6Y1SDzXSLG#7ex`R{c zvtWd4If2@#iR7pPTq_J?g#ucD(o)X6w+!06LGZ`a{9$cW;B+9c{EWR; zIl5PQ>m9JZ{arSBjL+^#2KpPjrIxlA zhiWE#$v-nwD474+HrumjXb*Mkq|s`nDVugZKcG-&YQmWLLb-5~CYoPB>E?erpa1JG zna;8X=xQ7H`2)jbXHp=A>;&d!{5A-a`!`+0mF6{O30azcfMo$*Cvx7SZSb-!c0{bC zMpFcT%`-59B~iBS`57VgjN6;?p$v(0y1k0_jmp)yD^I!9p)J81|(O#|GwNpS&ehgOlVLNzTMy~Ab;PshbZ=;w$ zQ>4!e#BI*%EW(LhQt?Lpv(6d2Tyaggm0IB?1}(qD?o?$r5C;*p1j`2GSUor?w_ub5 z&x{4EW6r>y^~~!shn{Sc$d6wVccTVG-n#$nOaD<@rfp-H51JOzr(V>o=XUOb+lDCWEhzsqOxuFfXc zl5JVE(R@rs%iPpkxtXOf&vkw1RZglPKhhC3cWf*}pDH^zr%xb9ewLe(; z$W%JBTs4uy$IfjuGe*a@yl%+7*X@jEGQ@zssRRCUs+{jzg1es0g)iypfA-w4C)V)j z;qc?V)K9TrmIe8=2sA6Gg2#vYH%Mn6zd;d3vAkuoOwCz*`C4;_XJmu(NZ|WYf7d=F zy4Pv5LL9nA`rM~45L=Vxrwggl1)Vv>meM5trkk7&t>rF;xrs*4GEM+7{O|QlNDq5z z2BC&0^Xl8x1BH{jYX&Ly_Cepl-|wuS_UOYDB3IZndk%J59obH7#=f92b^`;^Pl{wl z-bkyy>d4*c-Y{4#U&@CzJGnJKzZ(>Qh=(beA92|pGeZk2WBkk+W*O?Wvv}&oi?_4S zoQlp8$%A;y55|g+q*Rv@C1-toB+Hw3$`u`X8KNh+veT=fOOr$u*(J+WOu$bqvnw5g z#lLO9obJCWN(X(vv);{{>RY@EZrtbF%lCh@GeEH~ub%$MW^_*UXrNI%2t|AIb!_8X z#8k$p(||$p)Ao^P7tQ~;_!ZNsHkuDqiB;}1#?Ns9H)?Jaxn z3!>N&Zc}YaDOAR|pyyvNKf3?04ZiUI%g6HHHxh}XPL)l5EJd;rJh>)@zD05J6}>_S zowI33ozrtO$ZUr!T2-Tk2Ld#bD(oXP^rbE&ZzVv%IT4~-Xgm>=pLwA z!8EeQJrf=S^6iP;Ux!F)LdK`qnFejMa#z@;*MbDOJjL=+I`mH=8yNOQEb&=>&u5o; z=)<>3$cGA3Nb`_dflk!dEhmMubllp>HD{PSZF?3F=x@5ZfZm$jh?*nIPxV_9dCHX> z>87e|T8gc$bFm#6iaw?0oVXWO)A^}LAJ*@@XPWfkWATf!SMIRgg8PTR zDDLTCYe&5pV-cqNuIfQyA1nEf^d}&9Gv^-!aCQ*UhAtb88H|`7j}4w0s5q*aoKKINyd*5^RojEgi?wVO^va*t_|NYCC@BjXVc+1=tx|4}y!jIBK0X8TJ)kxCy#O4=vIm$E^}% znPAr=CMh1jUX2i6ZMCZp7jE;z z!oaLEGDh2XILkm;Ss4&xLn>3jYSm2DqI)jv2MQuoX}fOwfiA`s8c9s)p9@#>DrOkd z+otN1x33jCJqrv}gT+0COjL!E^I_uaLU;W_cYeXWzF-sU(C93aFkumEgh;dz;bv=1 ze0g?La+^|r1Z}Q_8=6)M9)UT*ztiW&mcEh@xlcVvb$Vb9dw_g5zz`f{lc-x5P>M}j zc-j`R!pmCY>_PA=JONj7CweLkv#;jF(El7CZcbxeXAUzM2r~}0^wID=*%jUn?x7E3 z9tzb1rg=ww@AN?#800Lc=N#2#I2qq34i*Ap7N;_WnMCwyOr}Ax<{BfCRCZP$S5{Fj zJ>ojdwk=U7;f#WeZqjLivw_+1;cfSaluDHMYsHi{6rO|&_pX)ozG*zsogGA*VXtY? zYu{ciV=~EVd&J>f^=4v^VWcQ}z-pZ5)BWR6yemHWs^-qwgHOe7OFm7?*VA%_tRC!| zx$I6`ICY^q$Yi5dNn&S@O`J>>+1M6rI|S^^w3VvL+Sf{tM6|lPCuurN5;|(*=_o+O zWp)eAHM3z>LUIReH>3XH=91~xI0ZL{uj&f9uIYbHplvr(+GF~Gx3myC!|8pEZ*Hwm zkM`kF^3*DV+e+gsa_P<3Pd((Prr1{&Dk@N@Su+bKjC$m(N{`cP#}A@sj9vG7D}t}~ z%s-{L34@L!Yf}#=&t}D>%J3*@@z!waG^1 zg*xFMV>|Taw78Ps2Gl#Ns{WAqfCIS$LX|wp7tcs6^_<-M_0}o&F^72gj9wYsN-rx} zk4arqHIgTiGpAaaB(3W>9ibq(>nbtrhX)az&jDxCx3=DH%B|C%10)c{&8vrBc55#8 z5Z$X0`(DIRMAO@IfYPpcqrtSo)QbS57jOTfEoClz3aIDtKvz?Sv8ikat!sWIEThu( zQQ91R;=O6*Ekb?eJ9=b~Yo^FM68iGp zT8W8kG1=^3xK?5`B_#!TlwxwiCfP0nrq6$-3j*S2{p1Y7S*8{J812siGz_OrEy5BN zUVi3{qeQGqPcl@ejpSEZrccWNy_nifvJPk}ewCS+=JG=_SZ&p9x6C=o+!E95a^;>( zv_WTaTqp&~#IyP~%s^Qlt`jZ6TY>Nlm`pt-{*C6rgE z!`^As=kmlSmQpfRO>4c@QI$Ol*~P9PzflC@$A9iQS@x(KEoT!kacUnu8p>L;MtqYdS9li()O^3kQnVYfnInC&C`UuT zo;}fsaU_=D+z*!5BfVEY{cs8xe6_0UCKV^SDD?)MI)6zis}Iueudf&(RaR}J0aDxS zR0|$eg!lxy)agQiVobqmC$@0s2TzJekZ$^_W^X2=%NHb=et?2^jIs$BaRq~~-J zJc1ER&EoOQCc4Yw#8GV(Tl~w}Xj)fRY}!_`MdmP8 za-)hNo^nr4%4^HeTsPd=#pr2Y8Zoj-s{eRRtq(hO{9PDX{dOj$!pdX z>Xj0m@6RTF2olI#s~ zlJ|?7bhL~6r(FkCan}s0J)BswwK>&$J?{@Q{6BH*{nPXO$;L}qQsk%5)a&fI;yHkK z@O@sa{mfoi@l4@0#DdXnnx}tq7=S5>7{&i#&PuY1X*kt=vC(o{?W8dCjl|g%7rvgI z++7M!tBQ!6j$!mr`+~vArEh{YgRVEQU$yAz-Dmty2YQ{Rs@)LIs}7?AwOs|*mwr|O z33cy*J}N|oAXSz#mO*x32Q_mPT%0drQZ)BD-t==22bosZ$Z%x(4K|S-fuYPDA>o-5 zDH4Z>BQ5W!@%vJiI|&0IOor7elEffc-0baDPs^9kdmx#1ZPsd7Fn4XzGitdwqC6gc z#s3a%OG@1G7+C#Mqd0sdfFx%I5dn(kg|Y|Hqtwra#`}ZN@OOss-^J+%?A9Myj;~;J z6ZP~xFU2QYhbv`^*ap(Ce>V2vj)7@Wg(`m>-<}>Yk^s_%(mmLZ{3smKqwCRC)ewS0?I`rh9ecll3B04xSj0T*)MHobcBW1czeQCeJF9y_lGgzOji_v6Uy*an zD<(Tg$dP=_MLpj~#rj??PO;po?`_pI#N+@Ido@AA!%1;z%7vv$_VruTZSpjFGXqQ= zfjajnLnET)O!8$aT;M%$RWbx^RNYajzM9Y7xd!ss4+mO8PF1shn?Fy$<{z(U5_Z(% zDP8-bVI~Az=RhaHqF;t?S+vQHNOpGd`Pj|O3eL1?_2G5F4rjt=_zLh5a!wZ#hX=-3 zXIs0GSJIOn{oc`q>ZRKny|pTY)LKh--1qE6cS!Aw0#H3LNx6B6LxA@j@Vb6Dv$AGQ zI_+26DN0jeJz=FC$%s9>V+WAE6ulK9SXLcMkzq zZ+m?Evu1bc_mSNT1%HgJ{%K?Vv$Orf_Fs6vHm}l|Hb1+44(P66E8sAaII~=>UcJC_ zq#t-$_wW1L`yR!c!iYgD;y&SZllq6#94;=uI9F+qNT0rV;=)xGI4A|k2$8cApLy4< zj`$t8NYP%_Gs5B3z~hpYFtH=%tJ}YPmoN0q=#y7gJPj#$O=b|pP`VNZI-T3AN*!{M6w)z=pp{yQoZt4=;$)b zw&F6QTBT{Cfb0S6Rk5N|5wEdYMmBqmk{?EW_$D%hcn_>{gw%a_^@p7!GhcJdo_K)e zIY3Huf|Mi`&Sy1IjjT4laoTR|oYtTAQ?7EIYnbV{grikV{tGRpo$mskyf8^GtBza= z{+N0$$8WQZ2ZB6p`h3ij`uvcXX<5E?6C-4FbN?_gj(|7x$!?`TrAF@n&jHnLgy0`$ z1ww|qzOVXN>w7mSqN~XB3Ns+Xv?d(Rs-l_xHHpsPw2vf&I&)?@^M}U9B`c>KSXf0n zm()>SJ4i#J?Vi+J$8eI1AGEPtPkJI0!<*bL+-BI%C4c8EE&VJ13sciC3)L5_jqx|Y zQ-Mbn*)LvHO~7VIf;xy$pXk%8Ggm*ChD&u=&DI3L9u*_Ai^%t=XTu?KBiTG)TU%QM zjA%)k$FRh|+`z*1fkqfdbE!J)o7nu8NLr&w$C#4PHI=GGkpTHs^V8+|<_GTQfMcLg z0Mlv)Z@HBY%XK^9J87_l2v8*Ldeiu_UA^0S;$cF&)x9nH9~8;!EDDu3q}t~0UJ%s0 zwE;aUr4Isq8W*f?$ig(qR_~kxe(9G>^C%$sQ9Ws!ny>|}yknRoD=B$p`j|pwmBMCT zk9GVH0+CFv@osUZ*V~b>r37W#5A*2k$-Fr#*QO`RW|RLMAfomy<}8%$Fm=(d{@t9pmDkK*j=(8AgDrL5(v>o_ zxM43%NNQlTDC_GmUe0A%s^t4H2sEACZkM_)uk~ZMO)M&7%C9g?ltjS1P_U4-tLqx|r1{t=%>+%fDH; zMk93%bN^x?+ap;VwxwC-Ne`@s$3T1nW+cXPkJq$Bg16x<1im;)Sob}UQ;0& zQ>7-JDGr~FiN?CK3G7o_@U9J?nHm#6f|_^qJ`}wE`y&qk{D(-1{{)oD1-(oJPOv`U zBkMUJ)?Qf`QsHvkHs*r6q_ACd?e{p5Sg+F4I5G4dB8s^2iiCEljGQkmOv9cAAGm+} zV>v{jMqiO!hRbs!3tRK7F9;Hrz3(cjIo>L8Q0Zpn)sQ4pIKe8VEUh6ue!rwa^Bc43 zp(iv@F%{K_LW9D|5#uBSE75^x@nZhp>g3J=IQ5=$K)*84ME<8K2|BjQX>$(v!FCQ1 zl-9j8b8Og%z`?nq+`rtL2kP?N&Brxpr+C`At&MIYrvKA2Y|MxMQCZ%o!Xy?k~$@pukZ;sgZaNA@G^wy@^nj?k2gjLj6 z#~iNMmgsX2^E8HDbufCPmQCD2Znl|c4ZG{-XTOwQf2~c4p7a_(?Y>it)SWp;^QcUZ8m5#+gRc1_fG%MByvqa6a ztDba0&aRYebZn`H`eQ8!5}5irHFx1TIi5(vRJpn{%H#Dh)vDnXW!F0kIJ?Qy7CT}X z^!jKws=G4NywFW$6O&>S{`GT1+-gqNqT71b zEu1!7J?;}%{DfR82H|X{bKc%-Im_iL0cTBr151!QH3tlC=uB1W^KPed9cxx}zXx2X zO8k4R{sGs1GQq99^zFnK!ey?`FJu z4k$}iI|uB=A6}=pkzMDVvb1r%u$rQ<)Ncb`0=w}X%_^hp;w;2vcjp{%`Sxcxm=U!! zeakgzxj)Z3Ix#nVS}qz+5&;rmw>G)$5?wRla~5_7LmYANmQ!EatqU!e+gQ_?nkxqW zK#raRa#-oI#&_|2!gz_urtmJ6@zrQ^~~|$(hy1;2{0)1Ef zZakd5wTrNx8sqO4zRI*+`g&x42tlDP^rq^JQ!}0vrr)KQq0ty~+I6aLTNTf51BHe} zBqR*5JV_=cS02|@)_c~+_QR2_CmIUx005e-!Qa~O#fR&c&mO}K&H)~Rh(j)=9wo7> zdhm_i1NEJpEt`iuqI*4fJ)Ck>{Hnp#8u7es{ka15;wQ=Yx)3#ad+nEN*6AL-t|9bI zK%aL;jjVYFdu8zXT4q(XJlK8=zVAwg2x%hWM;{$`;;izM;gp%JeRUft!Eft{%Kmn^ z+Y4{J?pf|MUZz1`Qymgrmn`_Lza0(nU)VqsY62(A{3EJoMy(|nBJ-ZDuUpYO>pTnD zTH;WN3(hp@wBSsn81Ugis=|~enFXIbs!GkMVTeEVGA$jGnJ|of_hCg)!dvH}NOQqP z>DIFA0!Qa|cm<#knTg-{Ka6ZKs zQ!5Z>L+-rvM2s8CB*i$4J!$ccSZWvBcxdoag*u&BO?T<(Ie_1}Y@^$>wGh@}QC!m0 zwc|6N*s;kD*I%6*`vw7tXpUb{{M9QMS1;Xg^=bKa0)MIc@B?&kaZ?j})-$l7YWzLf z{<;|*I3yW~(vg$v;+xlSsk|F4xHS+PL)RN1FzI$OqlcMmx}@F+|HgA1lwE}fx% zhno#27kr$gQimcXm8xET^4Cu9(t+6?m44i^-lDmQlb>=7YY6+S51r{~Hkn-0)z%n? zaiXm9HMij7hav&1p;mX_Iy++NHwHY}5#tt2*CKc68=DVvcgdRi?O}v!V+Qz~XrE!i zPROFwG%J0>O0M#Udf&v>R@YTol^=q3**PN=gFcYy+WvyE=jF4SGX&N*Boso~uw7kE zd`Fo5c=Lm$Q;Cn4?_44z??}aqnbUx=HvE~V(#39KbcY?e1*LK^qq zR?-#1i=FVs%@?>+*DOO4PC7)-0pD}#!46JVUz7)l<_Tp19^~U4ow<*m9FlhH4s# zlxcDEsz#BW9cs%qVE-b?-K2(9{mqFms%|~lqfAA;eD4oUSE`wvIKS^qx~p=BRF4NK zQhzNHZkgGvko35uF%#X7LZi7y&o;_)vNz8Gl+1ROGVj{F99JqqzOYo-#-bc&U8%00 z(igYIwyY`cusHrx$y9W*Ch=LX|02`!c!^0lf9Cz9T>98m9J1QXoi8n>*yY2dk#!n!QWX+dk(&5b~&+Z+JXUZIW-(Sv|+}kr8ag|_W{YQc`k=2j%K`I|6v);S=raQGac&vy>ke{y;>bmlT* z3dCqyO2D}t47jja(e;#@TVUeyIiLXzX@0WIlv!DPs{qdxYZVtwNbh3T=ehQNdG9W& zahRCjA{05=1LXb^_PdxqaxcEr z1y`kZuB@#Ht|JYu(UC(tBG*^MN@w#)R%pB<9ixift9(8qZD*!SACUWEEdI~;h5G#{ z(@t+{fbQWFQt!Oiw;I1BI!~u=&|%X>;T)nx;6#xUo**8br&Stl#Dj4uuBT!O(xZ$n z@WuFTt{SjEt;Fn?ONa@lvICu?>Phz#^{a9Xvh={j4{-e>t&^+zA9*Kn zgKw351n*DA)49+56rn5+KZ>T0<_ldj3F+xn^7`R1?3|SA2kq5jo=780NKBQFPPXe9 zCcD?OIE3$SSOEhv|jEu zshhGk<{qQK>CaQsxW%-NFczrs`Zl&mX^mvaugd78PRusu4`nHiiBx)>u91=9h6<)V6MDRpQM(rh2K!rgGzMlVAyL zRk$39RTixvSG{w$c&Lu-mf@D!&a-z3T`R`gt7#%^T<9&yRfUZ_;?=WQpHp&UydQJz z@_q^~?6_ni0posByS4i>EovS53gz>Pxb8p2%Cr;iA(M5 zVzEbXwVZM;7_GlLY{=akm1J)(reb(1;n^%EajcuFd`$JY2S44hy0KF1Y@~_PF2Wv*Gz(j}wof=|)K#Qk z18T=)j-!;fKnfp&@58ELeKJFGppTKPKLiEYrzg6?&2OFq=qnp+gNnW!JVgurdXzeC zjlOMaw9cCETQ?UiKw_0vx7*Nr1XAQQw|japzkI52cb`zr##$tYBuEtX?@dGoJ2WbY znaM}v5ZuS*9Wc=(=l4^Ta>pm7tq7@~r$3ew+EAxF0_yKql6-=)ak#Yix@ZEj`BP^= z0Ze3g#j;|x_coBmVJ6F#={KYB7wKcl z#v5bxTROIZl-L{uHP%H)GzlG;Xq~x~&85R%^C0d(CBIp~{phexV78{j!#_<)`;{<)Vj{ zHirWWatb{{+eH9?4@Iy4tdc1V*F7i7G;D+5W@)4OCphhxiBA@c#qKtxckQnPQM zR?;jTQUFU_MOxYo95-{H{{B#xW4;Lo`2i63mb)oR}>9zFEA0p3m*uPU^ zVt*1EP{xTzYbwXm7rk{3(It(xSg#@-Wu3ddq<$SI-vl*gMh;EZ(u|pmQhx6c5-_zP zapD^JCO9btk;MozzeBf?SD_|&*iNp61m^+VF$I`h(5P$3d#l?3z~S*<>*D{sFFULG ztEZ0f2cfNbo60H}3DryJC>SS!!gQ5MLFVXGAYR#|Gy0;IYYd9orcB31cLh1zjIYjw z62iW>UqDvv%+oyl+kN}xk5M#Ig3Ir`s-H;Sl}m_9TLbhg=J9un10fwF^BJwVCza1p z&IZ*MHur#Kl%2>>nA5@*NQi?$bB*b!xauhN-I#+sW)`26q>dF3a1g>K+W7O@q$ut7 zx$#YrVYNEhB~eaFsdjBo%qiGy6-z2vG+k5lT8bVpHOUOiJO>yOo;(`#u#c*}_n6DFu#MQ zRIH9fQ&z6EX3ex3yST$Pdpu_ML29EwD#VJx%FBSNEbImfZGg(IN)6Pav$qk;eJ`Er z`bs@YDd%S69P6`4(-fF~?jQ{JOI|H=g}?IWl3b<8SmS1wHEamSt@>`oZD_|apVYf} zM(voCp|`4ZiOis0r19H%I}LhkYWLwq`^L^5HdLsQpzQYeGme;=6=V04 zVg_9`HdU^!2rs)_=v__N1G54N_M?(x0w5zlY1$uEc_Y1tp|l^qWOwvAZWyf)#6&AShnh=Ho`a zf{Ceg#G5$rmwzIFID{fPxV!i0bUYz~vGB`ppY`0esVZrEYhc!D_tl0zubkuKNbo$* zsOVPxpmumx#f)ZNn!n%O8VmpA(n5yW3}t;-g50C<&Hz0MAvSmU?gH15BCcU}k@zWy z_G%!W#dAa)azK^JGf6#Wy~Yz(kl$xpi-9eP#M+7;E<+>N@Fz44bNxxMCC~86qJzks zYTX!5n&Iy?WVG=sf_bo5F`{n5c+Vjpe$GFl?_Cv(j56mJE?*vXr)>|+Bk|Kf%8!nD zBY`}oN{J}7vQ~}ivIu%pxBP;>a-NutMG^Oo(q#~kw{|k#R?z64Tmn}YxolQGImeKT zS5$Vk!+f^Z;TVp1Z-tW}Na0bE5-?bH^VS02-HDw70#x~O0obANWr~V)gQO&*DxNeXtJ$d(l8b4@b zgxWXP=Ur!B{_skz@blhivDlQ|A7?l-H37*c0cF@k7W!q>4{f7x9yU!5|H_MHsg!wT z?j92hBq|U6431o(6}}u5IDe=U*jYxT3u2+k+R7StD^zMos{(E8a1ioEvv2Q*QM!ZD zx=gFpcPBkA!NOL{W~&Q2L$_;VoS!AfCb`{?y`yHMx*W|;P|Grz6lI8iJHWr>iB>(4 zT3z3uhxF?>#aN&j;}SxQe_q#F7udh+qvXtysg)KlB*Hf1YUNVi&tF&|53@j7>#XCO zOS(+`4}k1h&`{mF5gwRRU>2$>SsNEag&b;N^%>h$> zc9|h`Llc|GjzV+BH?^(wd9ZK~$SBPlCfXQ+xs(+&S4wf~_MBf7%S_ZyxHyz`T&lKm zZD)`f9!ZcP_$g!Ht4=4{l%$Ux_u9={-EPL;FcW&1tSz0#Q8Ce(qp69MfG(!U^DIYa zx4!V$juq%_}uZeLQ9MX+0&6N#i&(5N$2dPe$kcY%yf<}rYFc=bZ=C}x}ZJy zL3^msV^=~LR|l^KeXDX2G#)%r$uo_O%z>baQeD3o#T3iiR{GF>lRr+&lc@7UKi6b> zCTLkp{)2Ef63l4W8O)|;tGAJ^l%Xn~&!cvogS~f@BNHgd)i5+!A#c4wXOXDa-@0;t z(oS>kS<-O98h={+1z~~QEQ#y%N&B~I*+jF&hfS$>B@0JhsgE{z#dU}ju0jMne%xxE zsVwc*T3fV>X}1zqT2AD_a_Lr6&DK$HP^9x`JBX!UE_>3ad^X|b+nQzh@`~^yZ*OE{ zj_udExLDo@w@lY}^J1v?ih^a|C3}iFUQOyR7Ask25H4ZrD;5jp4>kA;-RhabQTP>( zVlruHHbx}) zYmu_3@@R8T>R3!)YQfNoP2EbU8i-4k6AkfxbS50pM}5dF%|~+8X_u?JslA+*3*&6( z!8HSGbuu0Uk5u2Snh$ln1PA*qyh^i?MryqW|q`}9=&5vwNs=Z({%c-~L=yNcKy0Sf}bk|As*Av=Oi)4{o z`u3M6?P8m^eIGObNj{?ocDA0UkCb8<$Ph`Kz4sNgPH!RB$X(vHJR@GTqw=g38}kNe z>^v^1g*$$709-1Y8@lBtGTVae+#kO>Tn;rJ@lUQOhy$M)opMI-C7lC&Zquut1EwEG z$?pAR9eo&XVAk?1j66ogBJ1?_ZzWeld<+Q*{_vM#{BM20@cDu2{lbC5Dq;!rG_qCB z++4C?`|#)er&3!x%aXB@Jy~Y9!~I!JKvw|6;^m}I1Kj>t2KM(yrRFA#kXjQ6)OvA$ zrTrz!)&yP4po&6w?@ivtA+G`@BlSIBQs?_rt z=DKm9s9!yF`S1Um!?X~=5pI2EKCe)pMp_O10d4i~FaiH@E#16glWKwYf0p#Bhc(oI zxecw3?w**PB|DBX6@G=43E3yi zyDgFWeu8K_*SlIV^s1S%BAeyTTy0|7(KlQ=S6A_9T|pq-XSpwQ`fWtsTrrkY_1@py zO#hrQ%Hp@gynoI-^cM)w-#OD-V0ahRo$igkx)KXe#dNljO4!@!hBf*Z!WE@FWUsgK zA$v)?ga{f<|N)Y$%vtLOh;YQ$V$8oNy}So&YCK;0Dbx|j?Z;vgp9(fFiGYH#l} zY{%sSle^F%{mp&Gj5^a_4M~7GrZ2Oxb0Ol%^~rPSb<;y!!`+B1XSJ`e9=()e(_%!^;ii+s zBX2VwOPNTY&nqIEW8toqGO1dS6s2Iu$rSXFV5ot3>@k?BN=m{vq4q;KA{)*Dcj0ug zl4ca&Ges&n)E_zPy&xBSVi32`hY0z~5`DDccW?gjGHKzd27>x7VX&*zmab=Wt9L@S zv<|{2-A2_XCWWDLoDHo5=YX`A1L)OkhR=DPsLVM9cXa}el?Tqxuw;+SQ_r=8FsQ-U zIV$3#*?TP6*yfcKbDnEhb~EZ&<#Pt#_@)Lq%!{;P@4G@`S_cBhd#iyvOm;2>LksPu z(3>P-!ua{TkG)5A=EnvI`Y-1I7NJ3D9(=a`7A*qKy7bAn12Ye$4WuqfWf(eaNuv)WS2_NVA;dk{VUbb|Ow_55;T$s9&{(49L|KkX%*Q2RFWE6@*pE ze8s(QJ|>hq*VD<%>U`HzIGs^&K{UyHDd&&a%eXsvvf0ywm=d04Ntm8F2fPYE?<+=6 z*UQYhY&Mfv7f4W%C#**6jm3*IO6P#d$g@fTM!GY$l9TdN3(H?ViF`~}qZ>9=9V5U~ z-Zw&WaY^7+=XB@Zt^%_s2@7O%kK$j#a8o+qp(ENJLI^UXXQJr1R+X;8=N0dmTl+B|ACv^7q+NWT5q!a40v`V{IcS|1He=r?v z$?92zz~6+jebygrGz;&cYN`7 z!u-mZa-^*0!;s3FY`(1D*rmcvdXH5PVggjQqz+(>^5r8sE!b81wrv(0e6tiTUB-=<6c-1ut~bLpU!;KAJ#4b%)QWH7@XJI4 z`yd=RZb7zIZX!O4@NSqK+A!INViW-y#$UQOZw#(nj=e|!VWa}bjCqmllq9FscCyG7 z+|;^s;OM=y{H;7dRy<}dsi>HO5@@Nh;h4!D3dRgY4lZ4rHWh&SW{(LU?R~MScN|aTtg`~U(Ons!IK{{GzU6Nmwi%lfJ5+gCbt+R zXv(QD>NUf5wM`uxlBaR>T5E`H(6-!b@0S$pI63yWuf0rhSm4?~4g>r&S+8?BI^s zrfZ!ZHS)=3f+|blavVMEN_Nd`#sd-?wHYaGKuO6Q77;i`?DCB?@ODkMkeUC;&(SQk zgp_Ev4#?=UMY*`IL^D)P-8+b%JdyeXvq1ZWBIz6W4d5#yk#x3>2ej zt7`EQlg?bum&vN33bWw>?q0$A;#W&^N5>o0+r+U0k_iR!ba(Fbsvo)zyp3H;a;|F6 z)v?lYh9wpxlJY>)5>nAbx3R5&o(Bb1Y#K!iI*iHI;5u3VYo3b;X|tHcbyk~2a12LZ z)9}evm1SH>`w#(Kf2mz`TDyR^Yv}ocspmcUOspq3`_3hmO#w4wGOz6Y;9ie3p^w?4 z0&In@sfSz23G;TA-c<+kd`^o<%nM`{NJKUDBzny++5x-%Ij0pOzUvbKHd_`v`l8Bm zct^>OqybtyRqgFnuD((T#oHycX;Gh87Q8xjSI|FOHBP|}OPeZ-XXeP%rUmSY1w4z) z{2Ji#9lpg|*LFb+K;TXeS`t^I3}vDhNy~TtUHBF3_sQJ)JMCsH4BWHik!AOj5oEgMve&ZeiqZ zDvQ^8xl_H9vuHpb2yNKGNMbj?FEzJ9cR{$W3`xakS$6xU?ziim&dv1C* zB(kP)54N%5F;RTV25rH^Yxyf^malaX8YH+9!jxx-e5Y-_!-sN)`I zY{Zj4n8IZx9n^!GjZ!tF3cq9><k%8`TBZ1-(c=H{hM zlh{#tle(m{r1g33Zx@~p*pjnCm8UL^Pk0vDDx1_{@5UYVEuaP4jZoQCTAWsa`0B@q zYY}{GBw}-jh}=}~EoygZJMG5UFYLCZx>HBhQ!3gF$y~k~;;~fB?1kZ~V%gxg+J_h) z?B6x9{wq4ot9^^~sG-EDZe}NQWKdQN`W!$d{8}R!Hp|TQ_1C%l7gcPAdLJPIk#0M4 zNA2wwrzH3?_iZh 44b86e8925c4 +Step 2/10 : ENV ORDS_HOME=/opt/oracle/ords RUN_FILE="runOrdsSSL.sh" + ---> Running in e22c5a03b869 +Removing intermediate container e22c5a03b869 + ---> 1421497abef8 +Step 3/10 : COPY $RUN_FILE $ORDS_HOME/ + ---> d96ac1477d2d +Step 4/10 : RUN yum -y install yum-utils bind-utils tree hostname openssl net-tools zip unzip tar wget vim-minimal which sudo expect procps && yum-config-manager --add-repo=http://yum.oracle.com/repo/OracleLinux/OL8/oracle/software/x86_64 && yum -y install java-11-openjdk-devel && yum -y install ords && yum -y install iproute && yum clean all + ---> Running in c08b8dac80a5 +Oracle Linux 8 BaseOS Latest (x86_64) 72 MB/s | 49 MB 00:00 +Oracle Linux 8 Application Stream (x86_64) 88 MB/s | 37 MB 00:00 +Last metadata expiration check: 0:00:07 ago on Mon 12 Sep 2022 03:23:32 PM UTC. +Package yum-utils-4.0.21-11.0.1.el8.noarch is already installed. +Package vim-minimal-2:8.0.1763-19.0.1.el8_6.2.x86_64 is already installed. +Package procps-ng-3.3.15-6.0.1.el8.x86_64 is already installed. +Dependencies resolved. +================================================================================ + Package Arch Version Repository Size +================================================================================ +Installing: + bind-utils x86_64 32:9.11.36-3.el8 ol8_appstream 452 k + expect x86_64 5.45.4-5.el8 ol8_baseos_latest 266 k + hostname x86_64 3.20-6.el8 ol8_baseos_latest 32 k + net-tools x86_64 2.0-0.52.20160912git.el8 ol8_baseos_latest 322 k + openssl x86_64 1:1.1.1k-7.el8_6 ol8_baseos_latest 709 k + sudo x86_64 1.8.29-8.el8 ol8_baseos_latest 925 k + tar x86_64 2:1.30-5.el8 ol8_baseos_latest 838 k + tree x86_64 1.7.0-15.el8 ol8_baseos_latest 59 k + unzip x86_64 6.0-46.0.1.el8 ol8_baseos_latest 196 k + wget x86_64 1.19.5-10.0.1.el8 ol8_appstream 734 k + which x86_64 2.21-17.el8 ol8_baseos_latest 49 k + zip x86_64 3.0-23.el8 ol8_baseos_latest 270 k +Upgrading: + openssl-libs x86_64 1:1.1.1k-7.el8_6 ol8_baseos_latest 1.5 M + vim-minimal x86_64 2:8.0.1763-19.0.1.el8_6.4 ol8_baseos_latest 575 k +Installing dependencies: + bind-libs x86_64 32:9.11.36-3.el8 ol8_appstream 175 k + bind-libs-lite x86_64 32:9.11.36-3.el8 ol8_appstream 1.2 M + bind-license noarch 32:9.11.36-3.el8 ol8_appstream 103 k + fstrm x86_64 0.6.1-2.el8 ol8_appstream 29 k + libmaxminddb x86_64 1.2.0-10.el8 ol8_appstream 33 k + libmetalink x86_64 0.1.3-7.el8 ol8_baseos_latest 32 k + protobuf-c x86_64 1.3.0-6.el8 ol8_appstream 37 k + python3-bind noarch 32:9.11.36-3.el8 ol8_appstream 150 k + python3-ply noarch 3.9-9.el8 ol8_baseos_latest 111 k + tcl x86_64 1:8.6.8-2.el8 ol8_baseos_latest 1.1 M + +Transaction Summary +================================================================================ +Install 22 Packages +Upgrade 2 Packages + +Total download size: 9.7 M +Downloading Packages: +(1/24): expect-5.45.4-5.el8.x86_64.rpm 158 kB/s | 266 kB 00:01 +(2/24): hostname-3.20-6.el8.x86_64.rpm 18 kB/s | 32 kB 00:01 +(3/24): libmetalink-0.1.3-7.el8.x86_64.rpm 18 kB/s | 32 kB 00:01 +(4/24): net-tools-2.0-0.52.20160912git.el8.x86_ 2.3 MB/s | 322 kB 00:00 +(5/24): openssl-1.1.1k-7.el8_6.x86_64.rpm 4.0 MB/s | 709 kB 00:00 +(6/24): python3-ply-3.9-9.el8.noarch.rpm 538 kB/s | 111 kB 00:00 +(7/24): sudo-1.8.29-8.el8.x86_64.rpm 5.0 MB/s | 925 kB 00:00 +(8/24): tar-1.30-5.el8.x86_64.rpm 4.2 MB/s | 838 kB 00:00 +(9/24): unzip-6.0-46.0.1.el8.x86_64.rpm 3.6 MB/s | 196 kB 00:00 +(10/24): tcl-8.6.8-2.el8.x86_64.rpm 4.1 MB/s | 1.1 MB 00:00 +(11/24): which-2.21-17.el8.x86_64.rpm 613 kB/s | 49 kB 00:00 +(12/24): tree-1.7.0-15.el8.x86_64.rpm 208 kB/s | 59 kB 00:00 +(13/24): bind-libs-9.11.36-3.el8.x86_64.rpm 1.3 MB/s | 175 kB 00:00 +(14/24): bind-license-9.11.36-3.el8.noarch.rpm 2.6 MB/s | 103 kB 00:00 +(15/24): bind-libs-lite-9.11.36-3.el8.x86_64.rp 6.8 MB/s | 1.2 MB 00:00 +(16/24): bind-utils-9.11.36-3.el8.x86_64.rpm 3.6 MB/s | 452 kB 00:00 +(17/24): zip-3.0-23.el8.x86_64.rpm 804 kB/s | 270 kB 00:00 +(18/24): libmaxminddb-1.2.0-10.el8.x86_64.rpm 529 kB/s | 33 kB 00:00 +(19/24): fstrm-0.6.1-2.el8.x86_64.rpm 161 kB/s | 29 kB 00:00 +(20/24): python3-bind-9.11.36-3.el8.noarch.rpm 2.0 MB/s | 150 kB 00:00 +(21/24): protobuf-c-1.3.0-6.el8.x86_64.rpm 351 kB/s | 37 kB 00:00 +(22/24): vim-minimal-8.0.1763-19.0.1.el8_6.4.x8 6.4 MB/s | 575 kB 00:00 +(23/24): wget-1.19.5-10.0.1.el8.x86_64.rpm 3.3 MB/s | 734 kB 00:00 +(24/24): openssl-libs-1.1.1k-7.el8_6.x86_64.rpm 6.8 MB/s | 1.5 MB 00:00 +-------------------------------------------------------------------------------- +Total 3.3 MB/s | 9.7 MB 00:02 +Running transaction check +Transaction check succeeded. +Running transaction test +Transaction test succeeded. +Running transaction + Preparing : 1/1 + Upgrading : openssl-libs-1:1.1.1k-7.el8_6.x86_64 1/26 + Running scriptlet: openssl-libs-1:1.1.1k-7.el8_6.x86_64 1/26 + Installing : protobuf-c-1.3.0-6.el8.x86_64 2/26 + Installing : libmaxminddb-1.2.0-10.el8.x86_64 3/26 + Running scriptlet: libmaxminddb-1.2.0-10.el8.x86_64 3/26 + Installing : fstrm-0.6.1-2.el8.x86_64 4/26 + Installing : bind-license-32:9.11.36-3.el8.noarch 5/26 + Installing : bind-libs-lite-32:9.11.36-3.el8.x86_64 6/26 + Installing : bind-libs-32:9.11.36-3.el8.x86_64 7/26 + Upgrading : vim-minimal-2:8.0.1763-19.0.1.el8_6.4.x86_64 8/26 + Installing : unzip-6.0-46.0.1.el8.x86_64 9/26 + Installing : tcl-1:8.6.8-2.el8.x86_64 10/26 + Running scriptlet: tcl-1:8.6.8-2.el8.x86_64 10/26 + Installing : python3-ply-3.9-9.el8.noarch 11/26 + Installing : python3-bind-32:9.11.36-3.el8.noarch 12/26 + Installing : libmetalink-0.1.3-7.el8.x86_64 13/26 + Installing : wget-1.19.5-10.0.1.el8.x86_64 14/26 + Running scriptlet: wget-1.19.5-10.0.1.el8.x86_64 14/26 + Installing : bind-utils-32:9.11.36-3.el8.x86_64 15/26 + Installing : expect-5.45.4-5.el8.x86_64 16/26 + Installing : zip-3.0-23.el8.x86_64 17/26 + Installing : sudo-1.8.29-8.el8.x86_64 18/26 + Running scriptlet: sudo-1.8.29-8.el8.x86_64 18/26 + Installing : openssl-1:1.1.1k-7.el8_6.x86_64 19/26 + Installing : which-2.21-17.el8.x86_64 20/26 + Installing : tree-1.7.0-15.el8.x86_64 21/26 + Installing : tar-2:1.30-5.el8.x86_64 22/26 + Running scriptlet: tar-2:1.30-5.el8.x86_64 22/26 + Installing : net-tools-2.0-0.52.20160912git.el8.x86_64 23/26 + Running scriptlet: net-tools-2.0-0.52.20160912git.el8.x86_64 23/26 + Installing : hostname-3.20-6.el8.x86_64 24/26 + Running scriptlet: hostname-3.20-6.el8.x86_64 24/26 + Cleanup : vim-minimal-2:8.0.1763-19.0.1.el8_6.2.x86_64 25/26 + Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 26/26 + Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 26/26 + Verifying : expect-5.45.4-5.el8.x86_64 1/26 + Verifying : hostname-3.20-6.el8.x86_64 2/26 + Verifying : libmetalink-0.1.3-7.el8.x86_64 3/26 + Verifying : net-tools-2.0-0.52.20160912git.el8.x86_64 4/26 + Verifying : openssl-1:1.1.1k-7.el8_6.x86_64 5/26 + Verifying : python3-ply-3.9-9.el8.noarch 6/26 + Verifying : sudo-1.8.29-8.el8.x86_64 7/26 + Verifying : tar-2:1.30-5.el8.x86_64 8/26 + Verifying : tcl-1:8.6.8-2.el8.x86_64 9/26 + Verifying : tree-1.7.0-15.el8.x86_64 10/26 + Verifying : unzip-6.0-46.0.1.el8.x86_64 11/26 + Verifying : which-2.21-17.el8.x86_64 12/26 + Verifying : zip-3.0-23.el8.x86_64 13/26 + Verifying : bind-libs-32:9.11.36-3.el8.x86_64 14/26 + Verifying : bind-libs-lite-32:9.11.36-3.el8.x86_64 15/26 + Verifying : bind-license-32:9.11.36-3.el8.noarch 16/26 + Verifying : bind-utils-32:9.11.36-3.el8.x86_64 17/26 + Verifying : fstrm-0.6.1-2.el8.x86_64 18/26 + Verifying : libmaxminddb-1.2.0-10.el8.x86_64 19/26 + Verifying : protobuf-c-1.3.0-6.el8.x86_64 20/26 + Verifying : python3-bind-32:9.11.36-3.el8.noarch 21/26 + Verifying : wget-1.19.5-10.0.1.el8.x86_64 22/26 + Verifying : openssl-libs-1:1.1.1k-7.el8_6.x86_64 23/26 + Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 24/26 + Verifying : vim-minimal-2:8.0.1763-19.0.1.el8_6.4.x86_64 25/26 + Verifying : vim-minimal-2:8.0.1763-19.0.1.el8_6.2.x86_64 26/26 + +Upgraded: + openssl-libs-1:1.1.1k-7.el8_6.x86_64 + vim-minimal-2:8.0.1763-19.0.1.el8_6.4.x86_64 +Installed: + bind-libs-32:9.11.36-3.el8.x86_64 + bind-libs-lite-32:9.11.36-3.el8.x86_64 + bind-license-32:9.11.36-3.el8.noarch + bind-utils-32:9.11.36-3.el8.x86_64 + expect-5.45.4-5.el8.x86_64 + fstrm-0.6.1-2.el8.x86_64 + hostname-3.20-6.el8.x86_64 + libmaxminddb-1.2.0-10.el8.x86_64 + libmetalink-0.1.3-7.el8.x86_64 + net-tools-2.0-0.52.20160912git.el8.x86_64 + openssl-1:1.1.1k-7.el8_6.x86_64 + protobuf-c-1.3.0-6.el8.x86_64 + python3-bind-32:9.11.36-3.el8.noarch + python3-ply-3.9-9.el8.noarch + sudo-1.8.29-8.el8.x86_64 + tar-2:1.30-5.el8.x86_64 + tcl-1:8.6.8-2.el8.x86_64 + tree-1.7.0-15.el8.x86_64 + unzip-6.0-46.0.1.el8.x86_64 + wget-1.19.5-10.0.1.el8.x86_64 + which-2.21-17.el8.x86_64 + zip-3.0-23.el8.x86_64 + +Complete! +Adding repo from: http://yum.oracle.com/repo/OracleLinux/OL8/oracle/software/x86_64 +created by dnf config-manager from http://yum.o 221 kB/s | 45 kB 00:00 +Dependencies resolved. +============================================================================================= + Package Arch Version Repository Size +============================================================================================= +Installing: + java-11-openjdk-devel x86_64 1:11.0.16.1.1-1.el8_6 ol8_appstream 3.4 M +Installing dependencies: + alsa-lib x86_64 1.2.6.1-3.el8 ol8_appstream 491 k + avahi-libs x86_64 0.7-20.el8 ol8_baseos_latest 62 k + copy-jdk-configs noarch 4.0-2.el8 ol8_appstream 30 k + crypto-policies-scripts noarch 20211116-1.gitae470d6.el8 ol8_baseos_latest 83 k + cups-libs x86_64 1:2.2.6-45.el8_6.2 ol8_baseos_latest 434 k + giflib x86_64 5.1.4-3.el8 ol8_appstream 51 k + graphite2 x86_64 1.3.10-10.el8 ol8_appstream 122 k + harfbuzz x86_64 1.7.5-3.el8 ol8_appstream 295 k + java-11-openjdk x86_64 1:11.0.16.1.1-1.el8_6 ol8_appstream 272 k + java-11-openjdk-headless x86_64 1:11.0.16.1.1-1.el8_6 ol8_appstream 40 M + javapackages-filesystem noarch 5.3.0-1.module+el8+5136+7ff78f74 ol8_appstream 30 k + lcms2 x86_64 2.9-2.el8 ol8_appstream 164 k + libX11 x86_64 1.6.8-5.el8 ol8_appstream 611 k + libX11-common noarch 1.6.8-5.el8 ol8_appstream 158 k + libXau x86_64 1.0.9-3.el8 ol8_appstream 37 k + libXcomposite x86_64 0.4.4-14.el8 ol8_appstream 28 k + libXext x86_64 1.3.4-1.el8 ol8_appstream 45 k + libXi x86_64 1.7.10-1.el8 ol8_appstream 49 k + libXrender x86_64 0.9.10-7.el8 ol8_appstream 33 k + libXtst x86_64 1.2.3-7.el8 ol8_appstream 22 k + libfontenc x86_64 1.1.3-8.el8 ol8_appstream 37 k + libjpeg-turbo x86_64 1.5.3-12.el8 ol8_appstream 157 k + libpkgconf x86_64 1.4.2-1.el8 ol8_baseos_latest 35 k + libxcb x86_64 1.13.1-1.el8 ol8_appstream 231 k + lksctp-tools x86_64 1.0.18-3.el8 ol8_baseos_latest 100 k + lua x86_64 5.3.4-12.el8 ol8_appstream 192 k + nspr x86_64 4.32.0-1.el8_4 ol8_appstream 142 k + nss x86_64 3.67.0-7.el8_5 ol8_appstream 741 k + nss-softokn x86_64 3.67.0-7.el8_5 ol8_appstream 487 k + nss-softokn-freebl x86_64 3.67.0-7.el8_5 ol8_appstream 395 k + nss-sysinit x86_64 3.67.0-7.el8_5 ol8_appstream 73 k + nss-util x86_64 3.67.0-7.el8_5 ol8_appstream 137 k + pkgconf x86_64 1.4.2-1.el8 ol8_baseos_latest 38 k + pkgconf-m4 noarch 1.4.2-1.el8 ol8_baseos_latest 17 k + pkgconf-pkg-config x86_64 1.4.2-1.el8 ol8_baseos_latest 15 k + ttmkfdir x86_64 3.0.9-54.el8 ol8_appstream 62 k + tzdata-java noarch 2022c-1.el8 ol8_appstream 186 k + xorg-x11-font-utils x86_64 1:7.5-41.el8 ol8_appstream 104 k + xorg-x11-fonts-Type1 noarch 7.5-19.el8 ol8_appstream 522 k +Enabling module streams: + javapackages-runtime 201801 + +Transaction Summary +============================================================================================= +Install 40 Packages + +Total download size: 50 M +Installed size: 194 M +Downloading Packages: +(1/40): crypto-policies-scripts-20211116-1.gita 1.3 MB/s | 83 kB 00:00 +(2/40): avahi-libs-0.7-20.el8.x86_64.rpm 952 kB/s | 62 kB 00:00 +(3/40): libpkgconf-1.4.2-1.el8.x86_64.rpm 2.2 MB/s | 35 kB 00:00 +(4/40): cups-libs-2.2.6-45.el8_6.2.x86_64.rpm 4.9 MB/s | 434 kB 00:00 +(5/40): lksctp-tools-1.0.18-3.el8.x86_64.rpm 3.9 MB/s | 100 kB 00:00 +(6/40): pkgconf-1.4.2-1.el8.x86_64.rpm 2.3 MB/s | 38 kB 00:00 +(7/40): pkgconf-m4-1.4.2-1.el8.noarch.rpm 1.1 MB/s | 17 kB 00:00 +(8/40): pkgconf-pkg-config-1.4.2-1.el8.x86_64.r 1.1 MB/s | 15 kB 00:00 +(9/40): copy-jdk-configs-4.0-2.el8.noarch.rpm 1.8 MB/s | 30 kB 00:00 +(10/40): giflib-5.1.4-3.el8.x86_64.rpm 3.0 MB/s | 51 kB 00:00 +(11/40): alsa-lib-1.2.6.1-3.el8.x86_64.rpm 12 MB/s | 491 kB 00:00 +(12/40): graphite2-1.3.10-10.el8.x86_64.rpm 5.9 MB/s | 122 kB 00:00 +(13/40): harfbuzz-1.7.5-3.el8.x86_64.rpm 13 MB/s | 295 kB 00:00 +(14/40): java-11-openjdk-11.0.16.1.1-1.el8_6.x8 15 MB/s | 272 kB 00:00 +(15/40): javapackages-filesystem-5.3.0-1.module 2.1 MB/s | 30 kB 00:00 +(16/40): lcms2-2.9-2.el8.x86_64.rpm 9.5 MB/s | 164 kB 00:00 +(17/40): libX11-1.6.8-5.el8.x86_64.rpm 24 MB/s | 611 kB 00:00 +(18/40): java-11-openjdk-devel-11.0.16.1.1-1.el 40 MB/s | 3.4 MB 00:00 +(19/40): libX11-common-1.6.8-5.el8.noarch.rpm 8.6 MB/s | 158 kB 00:00 +(20/40): libXau-1.0.9-3.el8.x86_64.rpm 2.6 MB/s | 37 kB 00:00 +(21/40): libXcomposite-0.4.4-14.el8.x86_64.rpm 2.2 MB/s | 28 kB 00:00 +(22/40): libXext-1.3.4-1.el8.x86_64.rpm 2.7 MB/s | 45 kB 00:00 +(23/40): libXi-1.7.10-1.el8.x86_64.rpm 2.8 MB/s | 49 kB 00:00 +(24/40): libXrender-0.9.10-7.el8.x86_64.rpm 2.4 MB/s | 33 kB 00:00 +(25/40): libXtst-1.2.3-7.el8.x86_64.rpm 1.6 MB/s | 22 kB 00:00 +(26/40): libfontenc-1.1.3-8.el8.x86_64.rpm 2.7 MB/s | 37 kB 00:00 +(27/40): libjpeg-turbo-1.5.3-12.el8.x86_64.rpm 9.6 MB/s | 157 kB 00:00 +(28/40): libxcb-1.13.1-1.el8.x86_64.rpm 13 MB/s | 231 kB 00:00 +(29/40): lua-5.3.4-12.el8.x86_64.rpm 11 MB/s | 192 kB 00:00 +(30/40): nspr-4.32.0-1.el8_4.x86_64.rpm 9.2 MB/s | 142 kB 00:00 +(31/40): nss-3.67.0-7.el8_5.x86_64.rpm 31 MB/s | 741 kB 00:00 +(32/40): nss-softokn-3.67.0-7.el8_5.x86_64.rpm 24 MB/s | 487 kB 00:00 +(33/40): nss-softokn-freebl-3.67.0-7.el8_5.x86_ 18 MB/s | 395 kB 00:00 +(34/40): nss-sysinit-3.67.0-7.el8_5.x86_64.rpm 4.3 MB/s | 73 kB 00:00 +(35/40): nss-util-3.67.0-7.el8_5.x86_64.rpm 8.7 MB/s | 137 kB 00:00 +(36/40): ttmkfdir-3.0.9-54.el8.x86_64.rpm 4.0 MB/s | 62 kB 00:00 +(37/40): tzdata-java-2022c-1.el8.noarch.rpm 12 MB/s | 186 kB 00:00 +(38/40): xorg-x11-font-utils-7.5-41.el8.x86_64. 6.0 MB/s | 104 kB 00:00 +(39/40): xorg-x11-fonts-Type1-7.5-19.el8.noarch 23 MB/s | 522 kB 00:00 +(40/40): java-11-openjdk-headless-11.0.16.1.1-1 73 MB/s | 40 MB 00:00 +-------------------------------------------------------------------------------- +Total 71 MB/s | 50 MB 00:00 +Running transaction check +Transaction check succeeded. +Running transaction test +Transaction test succeeded. +Running transaction + Running scriptlet: copy-jdk-configs-4.0-2.el8.noarch 1/1 + Running scriptlet: java-11-openjdk-headless-1:11.0.16.1.1-1.el8_6.x86_6 1/1 + Preparing : 1/1 + Installing : nspr-4.32.0-1.el8_4.x86_64 1/40 + Running scriptlet: nspr-4.32.0-1.el8_4.x86_64 1/40 + Installing : nss-util-3.67.0-7.el8_5.x86_64 2/40 + Installing : libjpeg-turbo-1.5.3-12.el8.x86_64 3/40 + Installing : nss-softokn-freebl-3.67.0-7.el8_5.x86_64 4/40 + Installing : nss-softokn-3.67.0-7.el8_5.x86_64 5/40 + Installing : tzdata-java-2022c-1.el8.noarch 6/40 + Installing : ttmkfdir-3.0.9-54.el8.x86_64 7/40 + Installing : lua-5.3.4-12.el8.x86_64 8/40 + Installing : copy-jdk-configs-4.0-2.el8.noarch 9/40 + Installing : libfontenc-1.1.3-8.el8.x86_64 10/40 + Installing : libXau-1.0.9-3.el8.x86_64 11/40 + Installing : libxcb-1.13.1-1.el8.x86_64 12/40 + Installing : libX11-common-1.6.8-5.el8.noarch 13/40 + Installing : libX11-1.6.8-5.el8.x86_64 14/40 + Installing : libXext-1.3.4-1.el8.x86_64 15/40 + Installing : libXi-1.7.10-1.el8.x86_64 16/40 + Installing : libXtst-1.2.3-7.el8.x86_64 17/40 + Installing : libXcomposite-0.4.4-14.el8.x86_64 18/40 + Installing : libXrender-0.9.10-7.el8.x86_64 19/40 + Installing : lcms2-2.9-2.el8.x86_64 20/40 + Running scriptlet: lcms2-2.9-2.el8.x86_64 20/40 + Installing : javapackages-filesystem-5.3.0-1.module+el8+5136+7f 21/40 + Installing : graphite2-1.3.10-10.el8.x86_64 22/40 + Installing : harfbuzz-1.7.5-3.el8.x86_64 23/40 + Running scriptlet: harfbuzz-1.7.5-3.el8.x86_64 23/40 + Installing : giflib-5.1.4-3.el8.x86_64 24/40 + Installing : alsa-lib-1.2.6.1-3.el8.x86_64 25/40 + Running scriptlet: alsa-lib-1.2.6.1-3.el8.x86_64 25/40 + Installing : pkgconf-m4-1.4.2-1.el8.noarch 26/40 + Installing : lksctp-tools-1.0.18-3.el8.x86_64 27/40 + Running scriptlet: lksctp-tools-1.0.18-3.el8.x86_64 27/40 + Installing : libpkgconf-1.4.2-1.el8.x86_64 28/40 + Installing : pkgconf-1.4.2-1.el8.x86_64 29/40 + Installing : pkgconf-pkg-config-1.4.2-1.el8.x86_64 30/40 + Installing : xorg-x11-font-utils-1:7.5-41.el8.x86_64 31/40 + Installing : xorg-x11-fonts-Type1-7.5-19.el8.noarch 32/40 + Running scriptlet: xorg-x11-fonts-Type1-7.5-19.el8.noarch 32/40 + Installing : crypto-policies-scripts-20211116-1.gitae470d6.el8. 33/40 + Installing : nss-sysinit-3.67.0-7.el8_5.x86_64 34/40 + Installing : nss-3.67.0-7.el8_5.x86_64 35/40 + Installing : avahi-libs-0.7-20.el8.x86_64 36/40 + Installing : cups-libs-1:2.2.6-45.el8_6.2.x86_64 37/40 + Installing : java-11-openjdk-headless-1:11.0.16.1.1-1.el8_6.x86 38/40 + Running scriptlet: java-11-openjdk-headless-1:11.0.16.1.1-1.el8_6.x86 38/40 + Installing : java-11-openjdk-1:11.0.16.1.1-1.el8_6.x86_64 39/40 + Running scriptlet: java-11-openjdk-1:11.0.16.1.1-1.el8_6.x86_64 39/40 + Installing : java-11-openjdk-devel-1:11.0.16.1.1-1.el8_6.x86_64 40/40 + Running scriptlet: java-11-openjdk-devel-1:11.0.16.1.1-1.el8_6.x86_64 40/40 + Running scriptlet: copy-jdk-configs-4.0-2.el8.noarch 40/40 + Running scriptlet: crypto-policies-scripts-20211116-1.gitae470d6.el8. 40/40 + Running scriptlet: nss-3.67.0-7.el8_5.x86_64 40/40 + Running scriptlet: java-11-openjdk-headless-1:11.0.16.1.1-1.el8_6.x86 40/40 + Running scriptlet: java-11-openjdk-1:11.0.16.1.1-1.el8_6.x86_64 40/40 + Running scriptlet: java-11-openjdk-devel-1:11.0.16.1.1-1.el8_6.x86_64 40/40 + Verifying : avahi-libs-0.7-20.el8.x86_64 1/40 + Verifying : crypto-policies-scripts-20211116-1.gitae470d6.el8. 2/40 + Verifying : cups-libs-1:2.2.6-45.el8_6.2.x86_64 3/40 + Verifying : libpkgconf-1.4.2-1.el8.x86_64 4/40 + Verifying : lksctp-tools-1.0.18-3.el8.x86_64 5/40 + Verifying : pkgconf-1.4.2-1.el8.x86_64 6/40 + Verifying : pkgconf-m4-1.4.2-1.el8.noarch 7/40 + Verifying : pkgconf-pkg-config-1.4.2-1.el8.x86_64 8/40 + Verifying : alsa-lib-1.2.6.1-3.el8.x86_64 9/40 + Verifying : copy-jdk-configs-4.0-2.el8.noarch 10/40 + Verifying : giflib-5.1.4-3.el8.x86_64 11/40 + Verifying : graphite2-1.3.10-10.el8.x86_64 12/40 + Verifying : harfbuzz-1.7.5-3.el8.x86_64 13/40 + Verifying : java-11-openjdk-1:11.0.16.1.1-1.el8_6.x86_64 14/40 + Verifying : java-11-openjdk-devel-1:11.0.16.1.1-1.el8_6.x86_64 15/40 + Verifying : java-11-openjdk-headless-1:11.0.16.1.1-1.el8_6.x86 16/40 + Verifying : javapackages-filesystem-5.3.0-1.module+el8+5136+7f 17/40 + Verifying : lcms2-2.9-2.el8.x86_64 18/40 + Verifying : libX11-1.6.8-5.el8.x86_64 19/40 + Verifying : libX11-common-1.6.8-5.el8.noarch 20/40 + Verifying : libXau-1.0.9-3.el8.x86_64 21/40 + Verifying : libXcomposite-0.4.4-14.el8.x86_64 22/40 + Verifying : libXext-1.3.4-1.el8.x86_64 23/40 + Verifying : libXi-1.7.10-1.el8.x86_64 24/40 + Verifying : libXrender-0.9.10-7.el8.x86_64 25/40 + Verifying : libXtst-1.2.3-7.el8.x86_64 26/40 + Verifying : libfontenc-1.1.3-8.el8.x86_64 27/40 + Verifying : libjpeg-turbo-1.5.3-12.el8.x86_64 28/40 + Verifying : libxcb-1.13.1-1.el8.x86_64 29/40 + Verifying : lua-5.3.4-12.el8.x86_64 30/40 + Verifying : nspr-4.32.0-1.el8_4.x86_64 31/40 + Verifying : nss-3.67.0-7.el8_5.x86_64 32/40 + Verifying : nss-softokn-3.67.0-7.el8_5.x86_64 33/40 + Verifying : nss-softokn-freebl-3.67.0-7.el8_5.x86_64 34/40 + Verifying : nss-sysinit-3.67.0-7.el8_5.x86_64 35/40 + Verifying : nss-util-3.67.0-7.el8_5.x86_64 36/40 + Verifying : ttmkfdir-3.0.9-54.el8.x86_64 37/40 + Verifying : tzdata-java-2022c-1.el8.noarch 38/40 + Verifying : xorg-x11-font-utils-1:7.5-41.el8.x86_64 39/40 + Verifying : xorg-x11-fonts-Type1-7.5-19.el8.noarch 40/40 + +Installed: + alsa-lib-1.2.6.1-3.el8.x86_64 + avahi-libs-0.7-20.el8.x86_64 + copy-jdk-configs-4.0-2.el8.noarch + crypto-policies-scripts-20211116-1.gitae470d6.el8.noarch + cups-libs-1:2.2.6-45.el8_6.2.x86_64 + giflib-5.1.4-3.el8.x86_64 + graphite2-1.3.10-10.el8.x86_64 + harfbuzz-1.7.5-3.el8.x86_64 + java-11-openjdk-1:11.0.16.1.1-1.el8_6.x86_64 + java-11-openjdk-devel-1:11.0.16.1.1-1.el8_6.x86_64 + java-11-openjdk-headless-1:11.0.16.1.1-1.el8_6.x86_64 + javapackages-filesystem-5.3.0-1.module+el8+5136+7ff78f74.noarch + lcms2-2.9-2.el8.x86_64 + libX11-1.6.8-5.el8.x86_64 + libX11-common-1.6.8-5.el8.noarch + libXau-1.0.9-3.el8.x86_64 + libXcomposite-0.4.4-14.el8.x86_64 + libXext-1.3.4-1.el8.x86_64 + libXi-1.7.10-1.el8.x86_64 + libXrender-0.9.10-7.el8.x86_64 + libXtst-1.2.3-7.el8.x86_64 + libfontenc-1.1.3-8.el8.x86_64 + libjpeg-turbo-1.5.3-12.el8.x86_64 + libpkgconf-1.4.2-1.el8.x86_64 + libxcb-1.13.1-1.el8.x86_64 + lksctp-tools-1.0.18-3.el8.x86_64 + lua-5.3.4-12.el8.x86_64 + nspr-4.32.0-1.el8_4.x86_64 + nss-3.67.0-7.el8_5.x86_64 + nss-softokn-3.67.0-7.el8_5.x86_64 + nss-softokn-freebl-3.67.0-7.el8_5.x86_64 + nss-sysinit-3.67.0-7.el8_5.x86_64 + nss-util-3.67.0-7.el8_5.x86_64 + pkgconf-1.4.2-1.el8.x86_64 + pkgconf-m4-1.4.2-1.el8.noarch + pkgconf-pkg-config-1.4.2-1.el8.x86_64 + ttmkfdir-3.0.9-54.el8.x86_64 + tzdata-java-2022c-1.el8.noarch + xorg-x11-font-utils-1:7.5-41.el8.x86_64 + xorg-x11-fonts-Type1-7.5-19.el8.noarch + +Complete! +Last metadata expiration check: 0:00:10 ago on Mon 12 Sep 2022 03:23:49 PM UTC. +Dependencies resolved. +============================================================================================== + Package + Arch Version Repository Size +============================================================================================== +Installing: + ords noarch 22.2.1-2.el8 yum.oracle.com_repo_OracleLinux_OL8_oracle_software_x86_64 83 M +Installing dependencies: + lsof x86_64 4.93.2-1.el8 ol8_baseos_latest 253 k + +Transaction Summary +============================================================================================== +Install 2 Packages + +Total download size: 83 M +Installed size: 87 M +Downloading Packages: +(1/2): lsof-4.93.2-1.el8.x86_64.rpm 3.0 MB/s | 253 kB 00:00 +(2/2): ords-22.2.1-2.el8.noarch.rpm 56 MB/s | 83 MB 00:01 +-------------------------------------------------------------------------------- +Total 56 MB/s | 83 MB 00:01 +Running transaction check +Transaction check succeeded. +Running transaction test +Transaction test succeeded. +Running transaction + Preparing : 1/1 + Installing : lsof-4.93.2-1.el8.x86_64 1/2 + Running scriptlet: ords-22.2.1-2.el8.noarch 2/2 + Installing : ords-22.2.1-2.el8.noarch 2/2 + Running scriptlet: ords-22.2.1-2.el8.noarch 2/2 +INFO: Before starting ORDS service, run the below command as user oracle: + ords --config /etc/ords/config install + + Verifying : lsof-4.93.2-1.el8.x86_64 1/2 + Verifying : ords-22.2.1-2.el8.noarch 2/2 + +Installed: + lsof-4.93.2-1.el8.x86_64 ords-22.2.1-2.el8.noarch + +Complete! +Last metadata expiration check: 0:00:15 ago on Mon 12 Sep 2022 03:23:49 PM UTC. +Package iproute-5.15.0-4.el8.x86_64 is already installed. +Dependencies resolved. +Nothing to do. +Complete! +24 files removed +Removing intermediate container c08b8dac80a5 + ---> bb1a717f3e6e +Step 5/10 : RUN mkdir -p $ORDS_HOME/doc_root && mkdir -p $ORDS_HOME/error && mkdir -p $ORDS_HOME/secrets && chmod ug+x $ORDS_HOME/*.sh && groupadd -g 54322 dba && usermod -u 54321 -d /home/oracle -g dba -m -s /bin/bash oracle && chown -R oracle:dba $ORDS_HOME && echo "oracle ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers + ---> Running in 0103c070f4b6 +Removing intermediate container 0103c070f4b6 + ---> 089d06d9b198 +Step 6/10 : USER oracle + ---> Running in 51b1846c8c6f +Removing intermediate container 51b1846c8c6f + ---> 6c7b115954a4 +Step 7/10 : WORKDIR /home/oracle + ---> Running in 5862e2bc8df9 +Removing intermediate container 5862e2bc8df9 + ---> 28543543a88c +Step 8/10 : VOLUME ["$ORDS_HOME/config/ords"] + ---> Running in 465398d6f2bb +Removing intermediate container 465398d6f2bb + ---> 4037eb7f2f12 +Step 9/10 : EXPOSE 8888 + ---> Running in 2813ab5473f6 +Removing intermediate container 2813ab5473f6 + ---> 3410f1be2fff +Step 10/10 : CMD $ORDS_HOME/$RUN_FILE + ---> Running in 0a9a72408177 +Removing intermediate container 0a9a72408177 + ---> 2ef5dc95701b +Successfully built 2ef5dc95701b +Successfully tagged oracle/ords-dboper:22.2.1 + diff --git a/docs/multitenant/provisioning/ords_image.md b/docs/multitenant/provisioning/ords_image.md new file mode 100644 index 00000000..21abcbab --- /dev/null +++ b/docs/multitenant/provisioning/ords_image.md @@ -0,0 +1,64 @@ + + +# Build ORDS Docker Image + +In the below steps, we are building an ORDS Docker Image for ORDS Software. The image built can be later pushed to a local repository to be used later for a deployment. + +**NOTE:** It is assumed that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-manage-pdb-life-cycle-using-oracle-db-operator-on-prem-database-controller) steps. + +1. Clone the software using git: +```sh + git clone git@orahub.oci.oraclecorp.com:rac-docker-dev/oracle-database-operator.git + cd oracle-database-operator/ords/ +``` + +2. Login to the registry: container-registry.oracle.com + +**NOTE:** To login to this registry, you will need to the URL https://container-registry.oracle.com , Sign in, then click on "Java" and then accept the agreement. + +```bash +docker login container-registry.oracle.com +``` + +3. Login to a repo where you want to push your docker image (if needed) to pull during deployment in your environment. + +```bash +docker login +``` + +4. Build the docker image by using below command: + +```bash +docker build -t oracle/ords-dboper:ords-latest . +``` + +5. Check the docker image details using: + +```bash +docker images +``` + +6. Tag and push the image to your docker repository. + +NOTE: We have the repo as `phx.ocir.io//oracle/ords:latest`. Please change as per your environment. + +```bash +docker tag oracle/ords-dboper:ords-latest phx.ocir.io//oracle/ords:latest +docker push phx.ocir.io//oracle/ords:latest +``` + +7. Verify the image pushed to your docker repository. + +You can refer to below sample output for above steps as well. + +8. Create a Kubernetes Secret for your docker repository to pull the image during deployment using the below command: + +```bash +kubectl create secret generic container-registry-secret --from-file=.dockerconfigjson=./.docker/config.json --type=kubernetes.io/dockerconfigjson -n oracle-database-operator-system +``` + +This Kubernetes secret will be provided in the .yaml file against the parameter `ordsImagePullSecret` to pull the ORDS Docker Image from your docker repository (if its a private repository). + +## Sample Output + +[Here](./ords_image.log) is the sample output for docker image created for ORDS latest version diff --git a/docs/onpremdb/provisioning/pdb.log b/docs/multitenant/provisioning/pdb.log similarity index 100% rename from docs/onpremdb/provisioning/pdb.log rename to docs/multitenant/provisioning/pdb.log diff --git a/docs/onpremdb/provisioning/pdb.yaml b/docs/multitenant/provisioning/pdb.yaml similarity index 100% rename from docs/onpremdb/provisioning/pdb.yaml rename to docs/multitenant/provisioning/pdb.yaml diff --git a/docs/onpremdb/provisioning/pdb_crd_resource.md b/docs/multitenant/provisioning/pdb_crd_resource.md similarity index 100% rename from docs/onpremdb/provisioning/pdb_crd_resource.md rename to docs/multitenant/provisioning/pdb_crd_resource.md diff --git a/docs/onpremdb/provisioning/pdb_secret.log b/docs/multitenant/provisioning/pdb_secret.log similarity index 100% rename from docs/onpremdb/provisioning/pdb_secret.log rename to docs/multitenant/provisioning/pdb_secret.log diff --git a/docs/onpremdb/provisioning/pdb_secret.yaml b/docs/multitenant/provisioning/pdb_secret.yaml similarity index 100% rename from docs/onpremdb/provisioning/pdb_secret.yaml rename to docs/multitenant/provisioning/pdb_secret.yaml diff --git a/docs/onpremdb/provisioning/plug_pdb.log b/docs/multitenant/provisioning/plug_pdb.log similarity index 100% rename from docs/onpremdb/provisioning/plug_pdb.log rename to docs/multitenant/provisioning/plug_pdb.log diff --git a/docs/onpremdb/provisioning/plug_pdb.md b/docs/multitenant/provisioning/plug_pdb.md similarity index 100% rename from docs/onpremdb/provisioning/plug_pdb.md rename to docs/multitenant/provisioning/plug_pdb.md diff --git a/docs/onpremdb/provisioning/plug_pdb.yaml b/docs/multitenant/provisioning/plug_pdb.yaml similarity index 100% rename from docs/onpremdb/provisioning/plug_pdb.yaml rename to docs/multitenant/provisioning/plug_pdb.yaml diff --git a/docs/onpremdb/provisioning/unplug_pdb.log b/docs/multitenant/provisioning/unplug_pdb.log similarity index 100% rename from docs/onpremdb/provisioning/unplug_pdb.log rename to docs/multitenant/provisioning/unplug_pdb.log diff --git a/docs/onpremdb/provisioning/unplug_pdb.md b/docs/multitenant/provisioning/unplug_pdb.md similarity index 100% rename from docs/onpremdb/provisioning/unplug_pdb.md rename to docs/multitenant/provisioning/unplug_pdb.md diff --git a/docs/onpremdb/provisioning/unplug_pdb.yaml b/docs/multitenant/provisioning/unplug_pdb.yaml similarity index 100% rename from docs/onpremdb/provisioning/unplug_pdb.yaml rename to docs/multitenant/provisioning/unplug_pdb.yaml diff --git a/docs/onpremdb/provisioning/validation_error.md b/docs/multitenant/provisioning/validation_error.md similarity index 100% rename from docs/onpremdb/provisioning/validation_error.md rename to docs/multitenant/provisioning/validation_error.md diff --git a/docs/onpremdb/provisioning/ords_image.log b/docs/onpremdb/provisioning/ords_image.log deleted file mode 100644 index 7cce536a..00000000 --- a/docs/onpremdb/provisioning/ords_image.log +++ /dev/null @@ -1,263 +0,0 @@ --- Clone the software using git - -% git clone git@orahub.oci.oraclecorp.com:rac-docker-dev/oracle-database-operator.git -Cloning into 'oracle-database-operator'... -Use of the Oracle network and applications is intended solely for Oracle's authorized users. The use of these resources by Oracle employees and contractors is subject to company policies, including the Code of Conduct, Acceptable Use Policy and Information Protection Policy; access may be monitored and logged, to the extent permitted by law, in accordance with Oracle policies. Unauthorized use may result in termination of your access, disciplinary action and/or civil and criminal penalties. -remote: Enumerating objects: 11244, done. -remote: Counting objects: 100% (506/506), done. -remote: Compressing objects: 100% (205/205), done. -remote: Total 11244 (delta 305), reused 493 (delta 296), pack-reused 10738 -Receiving objects: 100% (11244/11244), 4.11 MiB | 397.00 KiB/s, done. -Resolving deltas: 100% (7510/7510), done. -% cd oracle-database-operator/ords/ - - --- Download the ORDS Version 21.4.3 software and copy that to the current location: - -% ls -rlt -total 79968 --rw-rw-rw- 1 root root 81861484 Jun 26 04:07 ords-21.4.3.117.0405.zip --rw-r--r-- 1 root root 266 Jun 26 04:13 standalone.properties.tmpl --rw-r--r-- 1 root root 639 Jun 26 04:13 setupwebuser.sh --rw-r--r-- 1 root root 3096 Jun 26 04:13 runOrds.sh --rw-r--r-- 1 root root 427 Jun 26 04:13 ords_params.properties.tmpl --rw-r--r-- 1 root root 2230 Jun 26 04:13 Dockerfile --rw-r--r-- 1 root root 123 Jun 26 04:13 cdbadmin.properties.tmpl - - --- Login to repository: login container-registry.oracle.com -% docker login container-registry.oracle.com -Username: -Password: -WARNING! Your password will be stored unencrypted in /root/.docker/config.json. -Configure a credential helper to remove this warning. See -https://docs.docker.com/engine/reference/commandline/login/#credentials-store - -Login Succeeded - - --- Login to a repository where you want to push the ORDS docker image: -% docker login phx.ocir.io -Username (/>): -Password: -WARNING! Your password will be stored unencrypted in /root/.docker/config.json. -Configure a credential helper to remove this warning. See -https://docs.docker.com/engine/reference/commandline/login/#credentials-store - -Login Succeeded - - --- Build the ORDS Docker Image: -[root@docker-test-server ords]# docker build -t oracle/ords-dboper:ords-21.4.3 . -Sending build context to Docker daemon 81.88MB -Step 1/10 : FROM container-registry.oracle.com/java/jdk:latest - ---> b94ea1fc0f64 -Step 2/10 : ENV ORDS_HOME=/opt/oracle/ords INSTALL_FILE=ords*.zip CONFIG_PROPS="ords_params.properties.tmpl" STANDALONE_PROPS="standalone.properties.tmpl" CDBADMIN_PROPS="cdbadmin.properties.tmpl" SETUP_WEBUSER="setupwebuser.sh" RUN_FILE="runOrds.sh" - ---> Running in baf28954867a -Removing intermediate container baf28954867a - ---> 40d169e44c21 -Step 3/10 : COPY $INSTALL_FILE $CONFIG_PROPS $STANDALONE_PROPS $CDBADMIN_PROPS $RUN_FILE $SETUP_WEBUSER $ORDS_HOME/ - ---> 1d6a27f6ce13 -Step 4/10 : RUN yum -y install bind-utils net-tools zip unzip tar wget vim-minimal which sudo expect procps && yum clean all - ---> Running in e0879fd25edc -Oracle Linux 8 BaseOS Latest (x86_64) 74 MB/s | 46 MB 00:00 -Oracle Linux 8 Application Stream (x86_64) 66 MB/s | 36 MB 00:00 -Last metadata expiration check: 0:00:15 ago on Mon 27 Jun 2022 04:34:12 PM UTC. -Package vim-minimal-2:8.0.1763-16.0.2.el8_5.13.x86_64 is already installed. -Package procps-ng-3.3.15-6.0.1.el8.x86_64 is already installed. -Dependencies resolved. -================================================================================ - Package Arch Version Repository Size -================================================================================ -Installing: - bind-utils x86_64 32:9.11.36-3.el8 ol8_appstream 452 k - expect x86_64 5.45.4-5.el8 ol8_baseos_latest 266 k - net-tools x86_64 2.0-0.52.20160912git.el8 ol8_baseos_latest 322 k - sudo x86_64 1.8.29-8.el8 ol8_baseos_latest 925 k - tar x86_64 2:1.30-5.el8 ol8_baseos_latest 838 k - unzip x86_64 6.0-46.el8 ol8_baseos_latest 196 k - wget x86_64 1.19.5-10.0.1.el8 ol8_appstream 734 k - which x86_64 2.21-17.el8 ol8_baseos_latest 49 k - zip x86_64 3.0-23.el8 ol8_baseos_latest 270 k -Installing dependencies: - bind-libs x86_64 32:9.11.36-3.el8 ol8_appstream 175 k - bind-libs-lite x86_64 32:9.11.36-3.el8 ol8_appstream 1.2 M - bind-license noarch 32:9.11.36-3.el8 ol8_appstream 103 k - fstrm x86_64 0.6.1-2.el8 ol8_appstream 29 k - libmaxminddb x86_64 1.2.0-10.el8 ol8_appstream 33 k - libmetalink x86_64 0.1.3-7.el8 ol8_baseos_latest 32 k - protobuf-c x86_64 1.3.0-6.el8 ol8_appstream 37 k - python3-bind noarch 32:9.11.36-3.el8 ol8_appstream 150 k - python3-ply noarch 3.9-9.el8 ol8_baseos_latest 111 k - tcl x86_64 1:8.6.8-2.el8 ol8_baseos_latest 1.1 M - -Transaction Summary -================================================================================ -Install 19 Packages - -Total download size: 6.9 M -Installed size: 21 M -Downloading Packages: -(1/19): libmetalink-0.1.3-7.el8.x86_64.rpm 683 kB/s | 32 kB 00:00 -(2/19): net-tools-2.0-0.52.20160912git.el8.x86_ 5.7 MB/s | 322 kB 00:00 -(3/19): expect-5.45.4-5.el8.x86_64.rpm 4.3 MB/s | 266 kB 00:00 -(4/19): python3-ply-3.9-9.el8.noarch.rpm 5.4 MB/s | 111 kB 00:00 -(5/19): sudo-1.8.29-8.el8.x86_64.rpm 23 MB/s | 925 kB 00:00 -(6/19): unzip-6.0-46.el8.x86_64.rpm 12 MB/s | 196 kB 00:00 -(7/19): tar-1.30-5.el8.x86_64.rpm 14 MB/s | 838 kB 00:00 -(8/19): which-2.21-17.el8.x86_64.rpm 3.9 MB/s | 49 kB 00:00 -(9/19): tcl-8.6.8-2.el8.x86_64.rpm 16 MB/s | 1.1 MB 00:00 -(10/19): zip-3.0-23.el8.x86_64.rpm 12 MB/s | 270 kB 00:00 -(11/19): bind-libs-9.11.36-3.el8.x86_64.rpm 7.2 MB/s | 175 kB 00:00 -(12/19): bind-license-9.11.36-3.el8.noarch.rpm 9.6 MB/s | 103 kB 00:00 -(13/19): fstrm-0.6.1-2.el8.x86_64.rpm 4.8 MB/s | 29 kB 00:00 -(14/19): libmaxminddb-1.2.0-10.el8.x86_64.rpm 4.6 MB/s | 33 kB 00:00 -(15/19): bind-utils-9.11.36-3.el8.x86_64.rpm 17 MB/s | 452 kB 00:00 -(16/19): protobuf-c-1.3.0-6.el8.x86_64.rpm 3.4 MB/s | 37 kB 00:00 -(17/19): python3-bind-9.11.36-3.el8.noarch.rpm 16 MB/s | 150 kB 00:00 -(18/19): bind-libs-lite-9.11.36-3.el8.x86_64.rp 19 MB/s | 1.2 MB 00:00 -(19/19): wget-1.19.5-10.0.1.el8.x86_64.rpm 24 MB/s | 734 kB 00:00 --------------------------------------------------------------------------------- -Total 31 MB/s | 6.9 MB 00:00 -Running transaction check -Transaction check succeeded. -Running transaction test -Transaction test succeeded. -Running transaction - Preparing : 1/1 - Installing : protobuf-c-1.3.0-6.el8.x86_64 1/19 - Installing : libmaxminddb-1.2.0-10.el8.x86_64 2/19 - Running scriptlet: libmaxminddb-1.2.0-10.el8.x86_64 2/19 - Installing : fstrm-0.6.1-2.el8.x86_64 3/19 - Installing : bind-license-32:9.11.36-3.el8.noarch 4/19 - Installing : bind-libs-lite-32:9.11.36-3.el8.x86_64 5/19 - Installing : bind-libs-32:9.11.36-3.el8.x86_64 6/19 - Installing : unzip-6.0-46.el8.x86_64 7/19 - Installing : tcl-1:8.6.8-2.el8.x86_64 8/19 - Running scriptlet: tcl-1:8.6.8-2.el8.x86_64 8/19 - Installing : python3-ply-3.9-9.el8.noarch 9/19 - Installing : python3-bind-32:9.11.36-3.el8.noarch 10/19 - Installing : libmetalink-0.1.3-7.el8.x86_64 11/19 - Installing : wget-1.19.5-10.0.1.el8.x86_64 12/19 - Running scriptlet: wget-1.19.5-10.0.1.el8.x86_64 12/19 - Installing : bind-utils-32:9.11.36-3.el8.x86_64 13/19 - Installing : expect-5.45.4-5.el8.x86_64 14/19 - Installing : zip-3.0-23.el8.x86_64 15/19 - Installing : which-2.21-17.el8.x86_64 16/19 - Installing : tar-2:1.30-5.el8.x86_64 17/19 - Running scriptlet: tar-2:1.30-5.el8.x86_64 17/19 - Installing : sudo-1.8.29-8.el8.x86_64 18/19 - Running scriptlet: sudo-1.8.29-8.el8.x86_64 18/19 - Installing : net-tools-2.0-0.52.20160912git.el8.x86_64 19/19 - Running scriptlet: net-tools-2.0-0.52.20160912git.el8.x86_64 19/19 - Verifying : expect-5.45.4-5.el8.x86_64 1/19 - Verifying : libmetalink-0.1.3-7.el8.x86_64 2/19 - Verifying : net-tools-2.0-0.52.20160912git.el8.x86_64 3/19 - Verifying : python3-ply-3.9-9.el8.noarch 4/19 - Verifying : sudo-1.8.29-8.el8.x86_64 5/19 - Verifying : tar-2:1.30-5.el8.x86_64 6/19 - Verifying : tcl-1:8.6.8-2.el8.x86_64 7/19 - Verifying : unzip-6.0-46.el8.x86_64 8/19 - Verifying : which-2.21-17.el8.x86_64 9/19 - Verifying : zip-3.0-23.el8.x86_64 10/19 - Verifying : bind-libs-32:9.11.36-3.el8.x86_64 11/19 - Verifying : bind-libs-lite-32:9.11.36-3.el8.x86_64 12/19 - Verifying : bind-license-32:9.11.36-3.el8.noarch 13/19 - Verifying : bind-utils-32:9.11.36-3.el8.x86_64 14/19 - Verifying : fstrm-0.6.1-2.el8.x86_64 15/19 - Verifying : libmaxminddb-1.2.0-10.el8.x86_64 16/19 - Verifying : protobuf-c-1.3.0-6.el8.x86_64 17/19 - Verifying : python3-bind-32:9.11.36-3.el8.noarch 18/19 - Verifying : wget-1.19.5-10.0.1.el8.x86_64 19/19 - -Installed: - bind-libs-32:9.11.36-3.el8.x86_64 - bind-libs-lite-32:9.11.36-3.el8.x86_64 - bind-license-32:9.11.36-3.el8.noarch - bind-utils-32:9.11.36-3.el8.x86_64 - expect-5.45.4-5.el8.x86_64 - fstrm-0.6.1-2.el8.x86_64 - libmaxminddb-1.2.0-10.el8.x86_64 - libmetalink-0.1.3-7.el8.x86_64 - net-tools-2.0-0.52.20160912git.el8.x86_64 - protobuf-c-1.3.0-6.el8.x86_64 - python3-bind-32:9.11.36-3.el8.noarch - python3-ply-3.9-9.el8.noarch - sudo-1.8.29-8.el8.x86_64 - tar-2:1.30-5.el8.x86_64 - tcl-1:8.6.8-2.el8.x86_64 - unzip-6.0-46.el8.x86_64 - wget-1.19.5-10.0.1.el8.x86_64 - which-2.21-17.el8.x86_64 - zip-3.0-23.el8.x86_64 - -Complete! -17 files removed -Removing intermediate container e0879fd25edc - ---> 824de67afe8c -Step 5/10 : RUN mkdir -p $ORDS_HOME/doc_root && chmod ug+x $ORDS_HOME/*.sh && groupadd -g 54322 dba && useradd -u 54321 -d /home/oracle -g dba -m -s /bin/bash oracle && cd $ORDS_HOME && jar -xf $INSTALL_FILE && rm $INSTALL_FILE && mkdir -p $ORDS_HOME/config/ords && mkdir -p $ORDS_HOME/secrets && java -jar $ORDS_HOME/ords.war configdir $ORDS_HOME/config && chown -R oracle:dba $ORDS_HOME - ---> Running in 623f0b8b31a0 -2022-06-27T16:34:43.462Z INFO Set config.dir to /opt/oracle/ords/config in: /opt/oracle/ords/ords.war -Removing intermediate container 623f0b8b31a0 - ---> b5b4693d5ecf -Step 6/10 : USER oracle - ---> Running in dd481ff2fc2d -Removing intermediate container dd481ff2fc2d - ---> e4d3603d79ec -Step 7/10 : WORKDIR /home/oracle - ---> Running in 5aba61a54f70 -Removing intermediate container 5aba61a54f70 - ---> bf0501caf083 -Step 8/10 : VOLUME ["$ORDS_HOME/config/ords"] - ---> Running in 26e4dbd9c1ed -Removing intermediate container 26e4dbd9c1ed - ---> dc4134ac42aa -Step 9/10 : EXPOSE 8888 - ---> Running in a5fbea15a1ee -Removing intermediate container a5fbea15a1ee - ---> cf615843f041 -Step 10/10 : CMD $ORDS_HOME/$RUN_FILE - ---> Running in 444eaf7568e8 -Removing intermediate container 444eaf7568e8 - ---> 7c95be92a842 -Successfully built 7c95be92a842 -Successfully tagged oracle/ords-dboper:ords-21.4.3 - - - --- Check the docker image details: - -docker images -REPOSITORY TAG IMAGE ID CREATED SIZE -oracle/ords-dboper ords-21.4.3 7c95be92a842 44 seconds ago 771M - - --- Tag and push the image to your docker repository --- NOTE: We have used the repo as "phx.ocir.io//oracle/ords" -% docker tag oracle/ords-dboper:ords-21.4.3 phx.ocir.io//oracle/ords:21.4.3 -% docker push phx.ocir.io//oracle/ords:21.4.3 -The push refers to repository [phx.ocir.io//oracle/ords] -98222b4c695a: Pushed -d8fd7c53f02f: Pushed -2023d4f36fb3: Pushed -a14158bad0d3: Layer already exists -0c1b321d1256: Layer already exists -74a1be7b5642: Layer already exists -21.4.3: digest: sha256:0f75cba13605944d2ea970c38330b65bd81509aa2cfc63d63f454769d61c1d5b size: 1589 - - --- Verify the image pushed to your docker repository. - -For example, in above example, you can check the image on the cloud console with signature "sha256:0f75cba13605944d2ea970c38330b65bd81509aa2cfc63d63f454769d61c1d5b" - - --- Create a Kubernetes Secret for the docker repository (if its private) - -% kubectl create secret generic container-registry-secret --from-file=.dockerconfigjson=./.docker/config.json --type=kubernetes.io/dockerconfigjson -n oracle-database-operator-system -secret/container-registry-secret created - -% kubectl get secret -n oracle-database-operator-system -NAME TYPE DATA AGE -container-registry-secret kubernetes.io/dockerconfigjson 1 26s -default-token-x86d6 kubernetes.io/service-account-token 3 5d16h -webhook-server-cert kubernetes.io/tls 3 5d16h diff --git a/docs/onpremdb/provisioning/ords_image.md b/docs/onpremdb/provisioning/ords_image.md deleted file mode 100644 index 4399f3fb..00000000 --- a/docs/onpremdb/provisioning/ords_image.md +++ /dev/null @@ -1,68 +0,0 @@ -# Build ORDS Docker Image - -In the below steps, we are building an ORDS Docker Image for ORDS Software. - -The image built can be later pushed to a local repository to be used later for a deployment. - -**NOTE:** It is assumed that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-manage-pdb-life-cycle-using-oracle-db-operator-on-prem-database-controller) steps. - -1. Clone the software using git: -```sh -[root@test-server oracle-database-operator]# git clone git@orahub.oci.oraclecorp.com:rac-docker-dev/oracle-database-operator.git -[root@test-server oracle-database-operator]# cd oracle-database-operator/ords/ -``` - -2. Download the ORDS Software for required ORDS version. For Example: For ORDS Version 21.4.3, use this [link](https://www.oracle.com/tools/ords/ords-downloads-2143.html) - -3. Copy the downloaded software .zip file to the current location. - -4. Login to the registry: container-registry.oracle.com - -**NOTE:** To login to this registry, you will need to the URL https://container-registry.oracle.com , Sign in, then click on "Java" and then accept the agreement. - -```sh -docker login container-registry.oracle.com -``` - -5. Login to a repo where you want to push your docker image (if needed) to pull during deployment in your environment. - -```sh -docker login -``` - -6. Build the docker image by using below command: - -```sh -docker build -t oracle/ords-dboper:ords-21.4.3 . -``` - -7. Check the docker image details using: - -```sh -docker images -``` - -8. Tag and push the image to your docker repository. - -NOTE: We have the repo as `phx.ocir.io//oracle/ords:21.4.3`. Please change as per your environment. - -```sh -docker tag oracle/ords-dboper:ords-21.4.3 phx.ocir.io//oracle/ords:21.4.3 -docker push phx.ocir.io//oracle/ords:21.4.3 -``` - -9. Verify the image pushed to your docker repository. - -You can refer to below sample output for above steps as well. - -10. Create a Kubernetes Secret for your docker repository to pull the image during deployment using the below command: - -```sh -kubectl create secret generic container-registry-secret --from-file=.dockerconfigjson=./.docker/config.json --type=kubernetes.io/dockerconfigjson -n oracle-database-operator-system -``` - -This Kubernetes secret will be provided in the .yaml file against the parameter `ordsImagePullSecret` to pull the ORDS Docker Image from your docker repository (if its a private repository). - -## Sample Output - -[Here](./ords_image.log) is the sample output for docker image created for ORDS version 21.4.3 diff --git a/oracle-database-operator.yaml b/oracle-database-operator.yaml index 5e516801..9180474e 100644 --- a/oracle-database-operator.yaml +++ b/oracle-database-operator.yaml @@ -651,6 +651,8 @@ spec: spec: description: CDBSpec defines the desired state of CDB properties: + TestVariable: + type: string cdbAdminPwd: description: Password for the CDB Administrator to manage PDB lifecycle properties: @@ -688,6 +690,38 @@ spec: cdbName: description: Name of the CDB type: string + cdbTlsCrt: + properties: + secret: + description: CDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + cdbTlsKey: + properties: + secret: + description: CDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object dbPort: description: DB server port type: integer @@ -1282,7 +1316,7 @@ spec: - additionalPrinterColumns: - description: The connect string to be used jsonPath: .status.connString - name: Connect String + name: Connect_String type: string - description: Name of the CDB jsonPath: .spec.cdbName @@ -1416,6 +1450,54 @@ spec: - OPEN - CLOSE type: string + pdbTlsCat: + properties: + secret: + description: PDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + pdbTlsCrt: + properties: + secret: + description: PDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + pdbTlsKey: + properties: + secret: + description: PDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object reuseTempFile: description: Whether to reuse temp file type: boolean diff --git a/ords/Dockerfile b/ords/Dockerfile index 1d137cab..14823269 100644 --- a/ords/Dockerfile +++ b/ords/Dockerfile @@ -1,58 +1,42 @@ -# LICENSE UPL 1.0 +#LICENSE UPL 1.0 # # Copyright (c) 1982-2017 Oracle and/or its affiliates. All rights reserved. # # ORACLE DOCKERFILES PROJECT # -------------------------- -# This is the Dockerfile for Oracle Rest Data Services +# This is the Dockerfile for Oracle Rest Data Services 22.2 # -# REQUIRED FILES TO BUILD THIS IMAGE -# ---------------------------------- -# (1) ords.3.0.10.165.06.53.zip -# Download Oracle Rest Data Services from -# http://www.oracle.com/technetwork/developer-tools/rest-data-services/downloads/index.html -# -# HOW TO BUILD THIS IMAGE -# ----------------------- -# Put the downloaded file in the same directory as this Dockerfile -# Run: -# $ docker build -t oracle/restdataservices:3.0.10 . -# -# Pull base image -# --------------- -FROM container-registry.oracle.com/java/jdk:latest +FROM container-registry.oracle.com/java/jdk:latest # Environment variables required for this build (do NOT change) # ------------------------------------------------------------- ENV ORDS_HOME=/opt/oracle/ords \ - INSTALL_FILE=ords*.zip \ - CONFIG_PROPS="ords_params.properties.tmpl" \ - STANDALONE_PROPS="standalone.properties.tmpl" \ - CDBADMIN_PROPS="cdbadmin.properties.tmpl" \ - SETUP_WEBUSER="setupwebuser.sh" \ - RUN_FILE="runOrds.sh" + RUN_FILE="runOrdsSSL.sh" + +#RUN_FILE_NOSSL="runOrdsNOSSL.sh" # Copy binaries # ------------- -COPY $INSTALL_FILE $CONFIG_PROPS $STANDALONE_PROPS $CDBADMIN_PROPS $RUN_FILE $SETUP_WEBUSER $ORDS_HOME/ +COPY $RUN_FILE $ORDS_HOME +#COPY $RUN_FILE_NOSSL $ORDS_HOME/ -RUN yum -y install bind-utils net-tools zip unzip tar wget vim-minimal which sudo expect procps && \ +RUN yum -y install yum-utils bind-utils tree hostname openssl net-tools zip unzip tar wget vim-minimal which sudo expect procps && \ + yum-config-manager --add-repo=http://yum.oracle.com/repo/OracleLinux/OL8/oracle/software/x86_64 && \ + yum -y install java-11-openjdk-devel && \ + yum -y install ords && \ + yum -y install iproute && \ yum clean all # Setup filesystem and oracle user -# Adjust file permissions, go to /opt/oracle as user 'oracle' to proceed with ORDS installation # ------------------------------------------------------------ -RUN mkdir -p $ORDS_HOME/doc_root && \ - chmod ug+x $ORDS_HOME/*.sh && \ - groupadd -g 54322 dba && \ - useradd -u 54321 -d /home/oracle -g dba -m -s /bin/bash oracle && \ - cd $ORDS_HOME && \ - jar -xf $INSTALL_FILE && \ - rm $INSTALL_FILE && \ - mkdir -p $ORDS_HOME/config/ords && \ - mkdir -p $ORDS_HOME/secrets && \ - java -jar $ORDS_HOME/ords.war configdir $ORDS_HOME/config && \ - chown -R oracle:dba $ORDS_HOME +RUN mkdir -p $ORDS_HOME/doc_root && \ + mkdir -p $ORDS_HOME/error && \ + mkdir -p $ORDS_HOME/secrets && \ + chmod ug+x $ORDS_HOME/*.sh && \ + groupadd -g 54322 dba && \ + usermod -u 54321 -d /home/oracle -g dba -m -s /bin/bash oracle && \ + chown -R oracle:dba $ORDS_HOME && \ + echo "oracle ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers # Finalize setup # ------------------- @@ -62,5 +46,8 @@ WORKDIR /home/oracle VOLUME ["$ORDS_HOME/config/ords"] EXPOSE 8888 -# Define default command to start Oracle Database. +# Define default command to start Ords Services CMD $ORDS_HOME/$RUN_FILE + +## ONLY FOR DEVELOPMENT STAGE +#CMD ["/usr/sbin/init"] diff --git a/ords/cdbadmin.properties.tmpl b/ords/cdbadmin.properties.tmpl deleted file mode 100644 index 34bc56fd..00000000 --- a/ords/cdbadmin.properties.tmpl +++ /dev/null @@ -1,3 +0,0 @@ -database.api.admin.enabled=true -db.cdb.adminUser=###CDBADMIN_USER### as SYSDBA -db.cdb.adminUser.password=###CDBADMIN_PWD### \ No newline at end of file diff --git a/ords/ords_params.properties.tmpl b/ords/ords_params.properties.tmpl deleted file mode 100644 index d2aaf33b..00000000 --- a/ords/ords_params.properties.tmpl +++ /dev/null @@ -1,16 +0,0 @@ -db.hostname=###ORACLE_HOST### -db.port=###ORACLE_PORT### -db.servicename=###ORACLE_SERVICE### -database.api.admin.enabled=true -database.api.enabled=true -feature.sdw=true -plsql.gateway.add=false -restEnabledSql.active=true -rest.services.apex.add=false -rest.services.ords.add=true -standalone.http.port=8888 -standalone.mode=false -standalone.use.https=true -sys.user=SYS -sys.password=###ORACLE_PWD### -user.public.password=###ORDS_PWD### \ No newline at end of file diff --git a/ords/runOrds.sh b/ords/runOrds.sh deleted file mode 100644 index 60bab949..00000000 --- a/ords/runOrds.sh +++ /dev/null @@ -1,75 +0,0 @@ -#!/bin/bash -# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. -# -# Since: June, 2017 -# Author: gerald.venzl@oracle.com -# Description: Setup and runs Oracle Rest Data Services. -# -# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS HEADER. -# -# Copyright (c) 2014-2017 Oracle and/or its affiliates. All rights reserved. -# -# MODIFIED (DD-Mon-YY) -# gdbhat 16-Aug-21 - Added CDB Admin properties. Commented out ORACLE_PWD check - -function setupOrds() { - - # Check whether the Oracle DB password has been specified - #if [ "$ORACLE_PWD" == "" ]; then - # echo "Error: No ORACLE_PWD specified!" - # echo "Please specify Oracle DB password using the ORACLE_PWD environment variable." - # exit 1; - #fi; - - # Defaults - ORACLE_SERVICE=${ORACLE_SERVICE:-"ORCLPDB1"} - ORACLE_HOST=${ORACLE_HOST:-"localhost"} - ORACLE_PORT=${ORACLE_PORT:-"1521"} - APEXI=${APEXI:-"$ORDS_HOME/doc_root/i"} - ORDS_PORT=${ORDS_PORT:-"8888"} - - ORDS_PWD=`cat $ORDS_HOME/secrets/$ORDS_PWD_KEY` - ORACLE_PWD=`cat $ORDS_HOME/secrets/$ORACLE_PWD_KEY` - CDBADMIN_USER=`cat $ORDS_HOME/secrets/$CDBADMIN_USER_KEY` - CDBADMIN_PWD=`cat $ORDS_HOME/secrets/$CDBADMIN_PWD_KEY` - - # Make standalone dir - mkdir -p $ORDS_HOME/config/ords/standalone - - # Copy template files - cp $ORDS_HOME/$CONFIG_PROPS $ORDS_HOME/params/ords_params.properties - cp $ORDS_HOME/$STANDALONE_PROPS $ORDS_HOME/config/ords/standalone/standalone.properties - cp $ORDS_HOME/$CDBADMIN_PROPS $ORDS_HOME/cdbadmin.properties - - # Replace DB related variables (ords_params.properties) - sed -i -e "s|###ORACLE_SERVICE###|$ORACLE_SERVICE|g" $ORDS_HOME/params/ords_params.properties - sed -i -e "s|###ORACLE_HOST###|$ORACLE_HOST|g" $ORDS_HOME/params/ords_params.properties - sed -i -e "s|###ORACLE_PORT###|$ORACLE_PORT|g" $ORDS_HOME/params/ords_params.properties - sed -i -e "s|###ORDS_PWD###|$ORDS_PWD|g" $ORDS_HOME/params/ords_params.properties - sed -i -e "s|###ORACLE_PWD###|$ORACLE_PWD|g" $ORDS_HOME/params/ords_params.properties - - # Replace standalone runtime variables (standalone.properties) - sed -i -e "s|###PORT###|$ORDS_PORT|g" $ORDS_HOME/config/ords/standalone/standalone.properties - sed -i -e "s|###DOC_ROOT###|$ORDS_HOME/doc_root|g" $ORDS_HOME/config/ords/standalone/standalone.properties - sed -i -e "s|###APEXI###|$APEXI|g" $ORDS_HOME/config/ords/standalone/standalone.properties - - # Replace CDB Admin runtime variables (cdbadmin.properties) - sed -i -e "s|###CDBADMIN_USER###|$CDBADMIN_USER|g" $ORDS_HOME/cdbadmin.properties - sed -i -e "s|###CDBADMIN_PWD###|$CDBADMIN_PWD|g" $ORDS_HOME/cdbadmin.properties - - # Start ORDS setup - java -jar $ORDS_HOME/ords.war install simple 2>&1 | tee /tmp/ords_install.log - - # Setup Web Server User - $ORDS_HOME/$SETUP_WEBUSER -} - -############# MAIN ################ - -# Check whether ords is already setup -if [ ! -f $ORDS_HOME/config/ords/standalone/standalone.properties ]; then - setupOrds; - java -jar $ORDS_HOME/ords.war set-properties --conf apex_pu $ORDS_HOME/cdbadmin.properties -fi; - -java -jar $ORDS_HOME/ords.war standalone diff --git a/ords/runOrdsSSL.sh b/ords/runOrdsSSL.sh new file mode 100644 index 00000000..ac357637 --- /dev/null +++ b/ords/runOrdsSSL.sh @@ -0,0 +1,193 @@ +#!/bin/bash +# +# Since: June, 2022 +# Author: matteo.malvezzi@oracle.com +# Description: Setup and runs Oracle Rest Data Services 22.2. +# +# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS HEADER. +# +# Copyright (c) 2014-2017 Oracle and/or its affiliates. All rights reserved. +# +# MODIFIED (DD-Mon-YY) +# mmalvezz 25-Jun-22 - Initial version + +export ORDS=/usr/local/bin/ords +export ETCFILE=/etc/ords.conf +export CONFIG=`cat /etc/ords.conf |grep -v ^#|grep ORDS_CONFIG| awk -F= '{print $2}'` +export export _JAVA_OPTIONS="-Xms1126M -Xmx1126M" +export ERRORFOLDER=/opt/oracle/ords/error +export KEYSTORE=~/keystore +export OPENSSL=/usr/bin/openssl +export PASSFILE=${KEYSTORE}/PASSWORD +export HN=`hostname` +#export KEY=${KEYSTORE}/${HN}-key.der +#export CERTIFICATE=${KEYSTORE}/${HN}.der +export KEY=$ORDS_HOME/secrets/$TLSKEY +export CERTIFICATE=$ORDS_HOME/secrets/$TLSCRT + +export CUSTOMURL="jdbc:oracle:thin:@(DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = racnode1)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = TESTORDS)))" +echo $CUSTOMURL + +function SetParameter() { + ##ords config info <--- Use this command to get the list + + $ORDS --config ${CONFIG} config set security.requestValidationFunction false + $ORDS --config ${CONFIG} config set jdbc.MaxLimit 100 + $ORDS --config ${CONFIG} config set jdbc.InitialLimit 50 + $ORDS --config ${CONFIG} config set error.externalPath ${ERRORFOLDER} + $ORDS --config ${CONFIG} config set standalone.access.log /home/oracle + $ORDS --config ${CONFIG} config set standalone.https.port 8888 + $ORDS --config ${CONFIG} config set standalone.https.cert ${CERTIFICATE} + $ORDS --config ${CONFIG} config set standalone.https.cert.key ${KEY} + $ORDS --config ${CONFIG} config set restEnabledSql.active true + $ORDS --config ${CONFIG} config set security.verifySSL true + $ORDS --config ${CONFIG} config set database.api.enabled true + $ORDS --config ${CONFIG} config set plsql.gateway.mode false + $ORDS --config ${CONFIG} config set database.api.management.services.disabled false + $ORDS --config ${CONFIG} config set misc.pagination.maxRows 1000 + $ORDS --config ${CONFIG} config set db.cdb.adminUser "${CDBADMIN_USER:-C##DBAPI_CDB_ADMIN} AS SYSDBA" + $ORDS --config ${CONFIG} config secret --password-stdin db.cdb.adminUser.password << EOF +${CDBADMIN_PWD:-WElcome_12##} +EOF + +## $ORDS --config ${CONFIG} config set db.username "SYS AS SYSDBA" +## $ORDS --config ${CONFIG} config secret --password-stdin db.password <$PASSFILE +welcome1 +EOF + +## $JAVA_HOME/bin/keytool -genkey -keyalg RSA -alias selfsigned -keystore keystore.jks \ +## -dname "CN=${HN}, OU=Example Department, O=Example Company, L=Birmingham, ST=West Midlands, C=GB" \ +## -storepass welcome1 -validity 3600 -keysize 2048 -keypass welcome1 +## +## +## $JAVA_HOME/bin/keytool -importkeystore -srckeystore keystore.jks -srcalias selfsigned -srcstorepass welcome1 \ +## -destkeystore keystore.p12 -deststoretype PKCS12 -deststorepass welcome1 -destkeypass welcome1 +## +## +## ${OPENSSL} pkcs12 -in ${KEYSTORE}/keystore.p12 -nodes -nocerts -out ${KEYSTORE}/${HN}-key.pem -passin file:${PASSFILE} +## ${OPENSSL} pkcs12 -in ${KEYSTORE}/keystore.p12 -nokeys -out ${KEYSTORE}/${HN}.pem -passin file:${PASSFILE} +## ${OPENSSL} pkcs8 -topk8 -inform PEM -outform DER -in ${HN}-key.pem -out ${HN}-key.der -nocrypt +## ${OPENSSL} x509 -inform PEM -outform DER -in ${HN}.pem -out ${HN}.der + + + + + + + + +rm $PASSFILE +ls -ltr $KEYSTORE + + + +} + + +function setupOrds() { + +echo "====================================================" +echo CONFIG=$CONFIG + +export ORDS_LOGS=/tmp + + [ -f $ORDS_HOME/secrets/$WEBSERVER_USER_KEY ] && + { + WEBSERVER_USER=`cat $ORDS_HOME/secrets/$WEBSERVER_USER_KEY` + } + + [ -f $ORDS_HOME/secrets/$WEBSERVER_PASSWORD_KEY ] && + { + WEBSERVER_PASSWORD=`cat $ORDS_HOME/secrets/$WEBSERVER_PASSWORD_KEY` + } + + [ -f $ORDS_HOME/secrets/$CDBADMIN_USER_KEY ] && + { + CDBADMIN_USER=`cat $ORDS_HOME/secrets/$CDBADMIN_USER_KEY` + } + + [ -f $ORDS_HOME/secrets/$CDBADMIN_PWD_KEY ] && + { + CDBADMIN_PWD=`cat $ORDS_HOME/secrets/$CDBADMIN_PWD_KEY` + } + + [ -f $ORDS_HOME/secrets/$ORACLE_PWD_KEY ] && + { + SYSDBA_PASSWORD=`cat $ORDS_HOME/secrets/$ORACLE_PWD_KEY` + } + + [ -f $ORDS_HOME/secrets/$ORACLE_PWD_KEY ] && + { + ORDS_PASSWORD=`cat $ORDS_HOME/secrets/$ORDS_PWD_KEY` + } + +setupHTTPS; + +SetParameter; + +$ORDS --config ${CONFIG} install \ + --admin-user ${SYSDBA_USER:-"SYS AS SYSDBA"} \ + --db-hostname ${ORACLE_HOST:-racnode1} \ + --db-port ${ORACLE_PORT:-1521} \ + --db-servicename ${ORACLE_SERVICE:-TESTORDS} \ + --feature-db-api true \ + --feature-rest-enabled-sql true \ + --log-folder ${ORDS_LOGS} \ + --proxy-user \ + --password-stdin < Date: Fri, 16 Sep 2022 10:29:26 +0000 Subject: [PATCH 501/628] Reducing number of events, explain about tcpsCertRenewInterval in readme --- .../sidb/singleinstancedatabase_tcps.yaml | 12 ---------- .../singleinstancedatabase_controller.go | 24 +++++++++---------- docs/sidb/README.md | 22 +++++++++-------- 3 files changed, 24 insertions(+), 34 deletions(-) diff --git a/config/samples/sidb/singleinstancedatabase_tcps.yaml b/config/samples/sidb/singleinstancedatabase_tcps.yaml index 8089c85c..85ceec73 100644 --- a/config/samples/sidb/singleinstancedatabase_tcps.yaml +++ b/config/samples/sidb/singleinstancedatabase_tcps.yaml @@ -3,18 +3,6 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # -apiVersion: v1 -kind: Secret -metadata: - name: db-admin-secret - namespace: default -type: Opaque -stringData: - # Specify your DB password here - oracle_pwd: - ---- - apiVersion: database.oracle.com/v1alpha1 kind: SingleInstanceDatabase metadata: diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 7e432188..77d40f90 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -674,7 +674,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns if m.Spec.ReadinessCheckPeriod > 0 { return int32(m.Spec.ReadinessCheckPeriod) } - return 30 + return 60 }(), }, @@ -1796,15 +1796,13 @@ func (r *SingleInstanceDatabaseReconciler) validateDBReadiness(m *dbapi.SingleIn return requeueY, readyPod, err } if readyPod.Name == "" { - eventReason := "Database Pending" - eventMsg := "status of database is not ready, retrying..." m.Status.Status = dbcommons.StatusPending if ok, _ := dbcommons.IsAnyPodWithStatus(available, corev1.PodFailed); ok { - eventReason = "Database Failed" - eventMsg = "pod creation failed" + eventReason := "Database Failed" + eventMsg := "pod creation failed" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) } else if ok, runningPod := dbcommons.IsAnyPodWithStatus(available, corev1.PodRunning); ok { - eventReason = "Database Creating" - eventMsg = "database creation in progress..." + r.Log.Info("Database Creating...", "Name", m.Name) m.Status.Status = dbcommons.StatusCreating if m.Spec.CloneFrom != "" { // Required since clone creates the datafiles under primary database SID folder @@ -1823,18 +1821,20 @@ func (r *SingleInstanceDatabaseReconciler) validateDBReadiness(m *dbapi.SingleIn r.Log.Info("GetCheckpointFileCMD Output : \n" + out) if out != "" { - eventReason = "Database Unhealthy" - eventMsg = "datafiles exists" + eventReason := "Database Unhealthy" + eventMsg := "datafiles exists" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) m.Status.DatafilesCreated = "true" m.Status.Status = dbcommons.StatusNotReady r.updateORDSStatus(m, ctx, req) } + } else { + r.Log.Info("Database Pending...", "Name", m.Name) } - r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) - r.Log.Info(eventMsg) + // As No pod is ready now , turn on mode when pod is ready . so requeue the request - return requeueY, readyPod, errors.New(eventMsg) + return requeueY, readyPod, errors.New("no pod is ready currently") } if m.Status.DatafilesPatched != "true" { eventReason := "Datapatch Pending" diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 3a7ba085..1535d14c 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -535,15 +535,9 @@ true ``` The following steps are required to connect the Database using TCPS: -- You need to download the wallet from the Persistent Volume(PV) attached with the database pod. You can use the following command to get the list of pod: +- You need to download the wallet from the Persistent Volume (PV) attached with the database pod. The location of the wallet inside the pod is as `/opt/oracle/oradata/clientWallet/$ORACLE_SID`. **Let us assume the `ORACLE_SID` is `ORCL1`, and singleinstance database resource name is `sidb-sample` for the upcoming example command**. You can copy the wallet to the destination directory by the following command: ```bash - kubectl get po - NAME READY STATUS RESTARTS AGE - sidb-sample-gaqoe 1/1 Running 0 3d14h - ``` -- The location of the wallet inside the pod is as `/opt/oracle/oradata/clientWallet/$ORACLE_SID`. **Let us assume the `ORACLE_SID` is `ORCL1` for the upcoming example commands**. The sample command to download the wallet is as follows: - ```bash - kubectl cp sidb-sample-gaqoe:/opt/oracle/oradata/clientWallet/ORCL1 + kubectl cp $(kubectl get pods -l app=sidb-sample -o=jsonpath='{.items[0].metadata.name}'):/opt/oracle/oradata/clientWallet/ORCL1 ``` - This wallet includes the sample `tnsnames.ora` and `sqlnet.ora` files. All the TNS entries for the database (corresponding to the CDB and PDB) resides in `tnsnames.ora` file. You need to go inside the downloaded wallet directory and set the `TNS_ADMIN` environment variable to point to the current directory as follows: ```bash @@ -553,13 +547,20 @@ The following steps are required to connect the Database using TCPS: After this, you can connect using SQL\*Plus using the following sample commands: ```bash sqlplus sys@ORCL1 as sysdba - - sqlplus system@ORCL1 + ``` +- Alternatively, you can use the following SQL\*Plus command to connect using TCPS without setting TNS_ADMIN environment variable: + ```bash + sqlplus sys@tcps://?wallet_location= + ``` + Here, TCPS connect string can be found by using the following command: + ```bash + kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.TcpsConnectString}" ``` **NOTE:** - Only database server authentication is supported (no mTLS). - When TCPS is enabled, a self-signed certificate is generated and stored inside the wallets. For users' convenience, a client-side wallet is generated and stored at `/opt/oracle/oradata/clientWallet/$ORACLE_SID` location in the pod. - The self-signed certificate used with TCPS has validity for 2 years. After the certificate is expired, it will be renewed by the `OraOperator` automatically. You need to download the wallet again after the auto-renewal. +- You can set the certificate renew interval with the help of `tcpsCertRenewInterval` field in the **[config/samples/sidb/singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml)** file. The minimum accepted value is 1m, and the maximum value is 26280h (3 years). The certificates used with TCPS will automatically be renewed after this interval. If this field is omitted/commented in the yaml file, the certificates will not be renewed automatically. ### Specifying Custom Ports As mentioned in the section [Setup Database with LoadBalancer](#setup-database-with-loadbalancer), there are two kubernetes services possible for the database: NodePort and LoadBalancer. You can specify which port to use with these services by editing the `listenerPort` and `tcpsListenerPort` fields of the [config/samples/sidb/singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml) file. @@ -574,6 +575,7 @@ In case of `NodePort` service, `listenerPort`, and `tcpsListenerPort` will be th - `listenerPort` and `tcpsListenerPort` can not have same values. - `tcpsListenerPort` will come into effect only when TCPS connections are enabled (i.e. `enableTCPS` field is set in [config/samples/sidb/singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml) file). - If TCPS connections are enabled, and `listenerPort` is commented/removed in the [config/samples/sidb/singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml) file, only TCPS endpoint will be exposed. +- If LoadBalancer is enabled, and either `listenerPort` or `tcpsListenerPort` is changed, then it takes some time to complete the work requests (drain existing backend sets and create new ones). SingleInstanceDatabase and LoadBalancer remains in the healthy state, but, you can check the progress of the work requests by logging into the OCI console and checking the corresponding LoadBalancer. ## OracleRestDataService Resource From 4d45c025a4af01de6721815856ce6d3ee0e2ad89 Mon Sep 17 00:00:00 2001 From: Abhishek Kumar Date: Fri, 16 Sep 2022 11:28:39 +0000 Subject: [PATCH 502/628] Set min tcpsCertRenewInterval to 5m and some readme changes --- .../singleinstancedatabase_webhook.go | 4 +- .../crd/bases/database.oracle.com_cdbs.yaml | 34 +++++++++++++ .../crd/bases/database.oracle.com_pdbs.yaml | 50 ++++++++++++++++++- .../samples/sidb/singleinstancedatabase.yaml | 2 +- .../sidb/singleinstancedatabase_tcps.yaml | 2 +- docs/sidb/README.md | 4 +- 6 files changed, 89 insertions(+), 7 deletions(-) diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index e2671082..cc72cbfc 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -271,11 +271,11 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { "Please provide valid string to parse the tcpsCertRenewInterval.")) } maxLimit, _ := time.ParseDuration("26280h") - minLimit, _ := time.ParseDuration("1m") + minLimit, _ := time.ParseDuration("5m") if duration > maxLimit || duration < minLimit { allErrs = append(allErrs, field.Invalid(field.NewPath("spec").Child("tcpsCertRenewInterval"), r.Spec.TcpsCertRenewInterval, - "Please specify tcpsCertRenewInterval in the range: 1m to 26280h")) + "Please specify tcpsCertRenewInterval in the range: 5m to 26280h")) } } if len(allErrs) == 0 { diff --git a/config/crd/bases/database.oracle.com_cdbs.yaml b/config/crd/bases/database.oracle.com_cdbs.yaml index e50888d0..bf64810b 100644 --- a/config/crd/bases/database.oracle.com_cdbs.yaml +++ b/config/crd/bases/database.oracle.com_cdbs.yaml @@ -65,6 +65,8 @@ spec: spec: description: CDBSpec defines the desired state of CDB properties: + TestVariable: + type: string cdbAdminPwd: description: Password for the CDB Administrator to manage PDB lifecycle properties: @@ -103,6 +105,38 @@ spec: cdbName: description: Name of the CDB type: string + cdbTlsCrt: + properties: + secret: + description: CDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + cdbTlsKey: + properties: + secret: + description: CDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object dbPort: description: DB server port type: integer diff --git a/config/crd/bases/database.oracle.com_pdbs.yaml b/config/crd/bases/database.oracle.com_pdbs.yaml index 113b2b4a..1bc8d905 100644 --- a/config/crd/bases/database.oracle.com_pdbs.yaml +++ b/config/crd/bases/database.oracle.com_pdbs.yaml @@ -19,7 +19,7 @@ spec: - additionalPrinterColumns: - description: The connect string to be used jsonPath: .status.connString - name: Connect String + name: Connect_String type: string - description: Name of the CDB jsonPath: .spec.cdbName @@ -166,6 +166,54 @@ spec: - OPEN - CLOSE type: string + pdbTlsCat: + properties: + secret: + description: PDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + pdbTlsCrt: + properties: + secret: + description: PDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + pdbTlsKey: + properties: + secret: + description: PDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object reuseTempFile: description: Whether to reuse temp file type: boolean diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index dbfbe8e9..80123b69 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -49,7 +49,7 @@ spec: ## TCPS Certificate Renewal Interval: The time after which TCPS certificate will be renewed if TCPS connections are enabled. ## tcpsCertRenewInterval can be in hours(h), minutes(m) and seconds(s); e.g. 17520h, 8760h etc. - ## Maximum value is 26280h (3 years), Minimum value is 1m; Default value is 17520h (2 years) + ## Maximum value is 26280h (3 years), Minimum value is 5m; Default value is 17520h (2 years) ## If this field is commented out/removed from the yaml, it will disable the auto-renewal feature for TCPS certificate tcpsCertRenewInterval: 17520h diff --git a/config/samples/sidb/singleinstancedatabase_tcps.yaml b/config/samples/sidb/singleinstancedatabase_tcps.yaml index 85ceec73..08ba9ee9 100644 --- a/config/samples/sidb/singleinstancedatabase_tcps.yaml +++ b/config/samples/sidb/singleinstancedatabase_tcps.yaml @@ -36,7 +36,7 @@ spec: ## TCPS Certificate Renewal Interval: The time after which TCPS certificate will be renewed if TCPS connections are enabled. ## tcpsCertRenewInterval can be in hours(h), minutes(m) and seconds(s); e.g. 17520h, 8760h etc. - ## Maximum value is 26280h (3 years), Minimum value is 1m; Default value is 17520h (2 years) + ## Maximum value is 26280h (3 years), Minimum value is 5m; Default value is 17520h (2 years) ## If this field is commented out/removed from the yaml, it will disable the auto-renewal feature for TCPS certificate tcpsCertRenewInterval: 17520h diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 1535d14c..043aec31 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -560,7 +560,7 @@ The following steps are required to connect the Database using TCPS: - Only database server authentication is supported (no mTLS). - When TCPS is enabled, a self-signed certificate is generated and stored inside the wallets. For users' convenience, a client-side wallet is generated and stored at `/opt/oracle/oradata/clientWallet/$ORACLE_SID` location in the pod. - The self-signed certificate used with TCPS has validity for 2 years. After the certificate is expired, it will be renewed by the `OraOperator` automatically. You need to download the wallet again after the auto-renewal. -- You can set the certificate renew interval with the help of `tcpsCertRenewInterval` field in the **[config/samples/sidb/singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml)** file. The minimum accepted value is 1m, and the maximum value is 26280h (3 years). The certificates used with TCPS will automatically be renewed after this interval. If this field is omitted/commented in the yaml file, the certificates will not be renewed automatically. +- You can set the certificate renew interval with the help of `tcpsCertRenewInterval` field in the **[config/samples/sidb/singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml)** file. The minimum accepted value is 5m, and the maximum value is 26280h (3 years). The certificates used with TCPS will automatically be renewed after this interval. If this field is omitted/commented in the yaml file, the certificates will not be renewed automatically. ### Specifying Custom Ports As mentioned in the section [Setup Database with LoadBalancer](#setup-database-with-loadbalancer), there are two kubernetes services possible for the database: NodePort and LoadBalancer. You can specify which port to use with these services by editing the `listenerPort` and `tcpsListenerPort` fields of the [config/samples/sidb/singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml) file. @@ -575,7 +575,7 @@ In case of `NodePort` service, `listenerPort`, and `tcpsListenerPort` will be th - `listenerPort` and `tcpsListenerPort` can not have same values. - `tcpsListenerPort` will come into effect only when TCPS connections are enabled (i.e. `enableTCPS` field is set in [config/samples/sidb/singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml) file). - If TCPS connections are enabled, and `listenerPort` is commented/removed in the [config/samples/sidb/singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml) file, only TCPS endpoint will be exposed. -- If LoadBalancer is enabled, and either `listenerPort` or `tcpsListenerPort` is changed, then it takes some time to complete the work requests (drain existing backend sets and create new ones). SingleInstanceDatabase and LoadBalancer remains in the healthy state, but, you can check the progress of the work requests by logging into the OCI console and checking the corresponding LoadBalancer. +- If LoadBalancer is enabled, and either `listenerPort` or `tcpsListenerPort` is changed, then it takes some time to complete the work requests (drain existing backend sets and create new ones). In this time, the database connectivity is broken. Although, SingleInstanceDatabase and LoadBalancer remain in the healthy state, you can check the progress of the work requests by logging into the cloud provider's console and checking the corresponding LoadBalancer. ## OracleRestDataService Resource From 2f70630d855f760cb8baf174f4c059b5d31fb712 Mon Sep 17 00:00:00 2001 From: matteo malvezzi Date: Mon, 19 Sep 2022 09:15:52 +0200 Subject: [PATCH 503/628] Correnct Dockerfile typo --- ords/Dockerfile | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/ords/Dockerfile b/ords/Dockerfile index 14823269..d4e16b6c 100644 --- a/ords/Dockerfile +++ b/ords/Dockerfile @@ -10,7 +10,7 @@ FROM container-registry.oracle.com/java/jdk:latest # Environment variables required for this build (do NOT change) # ------------------------------------------------------------- -ENV ORDS_HOME=/opt/oracle/ords \ +ENV ORDS_HOME=/opt/oracle/ords/ \ RUN_FILE="runOrdsSSL.sh" #RUN_FILE_NOSSL="runOrdsNOSSL.sh" @@ -18,7 +18,7 @@ ENV ORDS_HOME=/opt/oracle/ords \ # Copy binaries # ------------- COPY $RUN_FILE $ORDS_HOME -#COPY $RUN_FILE_NOSSL $ORDS_HOME/ +#COPY $RUN_FILE_NOSSL $ORDS_HOME RUN yum -y install yum-utils bind-utils tree hostname openssl net-tools zip unzip tar wget vim-minimal which sudo expect procps && \ yum-config-manager --add-repo=http://yum.oracle.com/repo/OracleLinux/OL8/oracle/software/x86_64 && \ From c5a4da5fb9e554425e10e16c2af43b74d77b87e8 Mon Sep 17 00:00:00 2001 From: Abhishek Kumar Date: Wed, 21 Sep 2022 12:23:13 +0000 Subject: [PATCH 504/628] TCPS connections bugfixes --- .../singleinstancedatabase_webhook.go | 8 +++---- commons/database/constants.go | 6 ++--- .../samples/sidb/singleinstancedatabase.yaml | 2 +- .../sidb/singleinstancedatabase_tcps.yaml | 2 +- docs/sidb/README.md | 24 +++++-------------- 5 files changed, 15 insertions(+), 27 deletions(-) diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index cc72cbfc..1006c68e 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -250,10 +250,10 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { } } else { // LoadBalancer Service is expected. - if r.Spec.EnableTCPS && r.Spec.TcpsListenerPort == 0 && r.Spec.ListenerPort == 1522 { + if r.Spec.EnableTCPS && r.Spec.TcpsListenerPort == 0 && r.Spec.ListenerPort == int(dbcommons.CONTAINER_TCPS_PORT) { allErrs = append(allErrs, field.Invalid(field.NewPath("spec").Child("listenerPort"), r.Spec.ListenerPort, - "listenerPort can not be 1522 as the default port for tcpsListenerPort is 1522.")) + "listenerPort can not be 2484 as the default port for tcpsListenerPort is 2484.")) } } if r.Spec.EnableTCPS && r.Spec.ListenerPort != 0 && r.Spec.TcpsListenerPort != 0 && r.Spec.ListenerPort == r.Spec.TcpsListenerPort { @@ -271,11 +271,11 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { "Please provide valid string to parse the tcpsCertRenewInterval.")) } maxLimit, _ := time.ParseDuration("26280h") - minLimit, _ := time.ParseDuration("5m") + minLimit, _ := time.ParseDuration("24h") if duration > maxLimit || duration < minLimit { allErrs = append(allErrs, field.Invalid(field.NewPath("spec").Child("tcpsCertRenewInterval"), r.Spec.TcpsCertRenewInterval, - "Please specify tcpsCertRenewInterval in the range: 5m to 26280h")) + "Please specify tcpsCertRenewInterval in the range: 24h to 26280h")) } } if len(allErrs) == 0 { diff --git a/commons/database/constants.go b/commons/database/constants.go index 9df367b0..a1d7bfc8 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -40,7 +40,7 @@ package commons const CONTAINER_LISTENER_PORT int32 = 1521 -const CONTAINER_TCPS_PORT int32 = 1522 +const CONTAINER_TCPS_PORT int32 = 2484 const ORACLE_UID int64 = 54321 @@ -513,7 +513,7 @@ const LsnrPort string = "\"name\": \"listener\", \"protocol\": \"TCP\", \"port\" const LsnrNodePort string = "\"name\": \"listener\", \"protocol\": \"TCP\", \"port\": 1521, \"nodePort\": %d" // Payload section for TCPS port -const TcpsPort string = "\"name\": \"listener-tcps\", \"protocol\": \"TCP\", \"port\": %d, \"targetPort\": 1522" +const TcpsPort string = "\"name\": \"listener-tcps\", \"protocol\": \"TCP\", \"port\": %d, \"targetPort\": 2484" // Payload section for TCPS node port -const TcpsNodePort string = "\"name\": \"listener-tcps\", \"protocol\": \"TCP\", \"port\": 1522, \"nodePort\": %d" +const TcpsNodePort string = "\"name\": \"listener-tcps\", \"protocol\": \"TCP\", \"port\": 2484, \"nodePort\": %d" diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index 80123b69..0fafd46e 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -49,7 +49,7 @@ spec: ## TCPS Certificate Renewal Interval: The time after which TCPS certificate will be renewed if TCPS connections are enabled. ## tcpsCertRenewInterval can be in hours(h), minutes(m) and seconds(s); e.g. 17520h, 8760h etc. - ## Maximum value is 26280h (3 years), Minimum value is 5m; Default value is 17520h (2 years) + ## Maximum value is 26280h (3 years), Minimum value is 24h; Default value is 17520h (2 years) ## If this field is commented out/removed from the yaml, it will disable the auto-renewal feature for TCPS certificate tcpsCertRenewInterval: 17520h diff --git a/config/samples/sidb/singleinstancedatabase_tcps.yaml b/config/samples/sidb/singleinstancedatabase_tcps.yaml index 08ba9ee9..b4a64ba8 100644 --- a/config/samples/sidb/singleinstancedatabase_tcps.yaml +++ b/config/samples/sidb/singleinstancedatabase_tcps.yaml @@ -36,7 +36,7 @@ spec: ## TCPS Certificate Renewal Interval: The time after which TCPS certificate will be renewed if TCPS connections are enabled. ## tcpsCertRenewInterval can be in hours(h), minutes(m) and seconds(s); e.g. 17520h, 8760h etc. - ## Maximum value is 26280h (3 years), Minimum value is 5m; Default value is 17520h (2 years) + ## Maximum value is 26280h (3 years), Minimum value is 24h; Default value is 17520h (2 years) ## If this field is commented out/removed from the yaml, it will disable the auto-renewal feature for TCPS certificate tcpsCertRenewInterval: 17520h diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 043aec31..1e9f02c3 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -520,14 +520,6 @@ Alternatively, you can use the following command: ```bash kubectl patch --type=merge singleinstancedatabases.database.oracle.com sidb-sample -p '{"spec": {"enableTCPS": true}}' ``` - -When TCPS connections are enabled, a Kubernetes event is published notifying the same. This event can be seen by any one of the following commands: -```bash -kubectl describe singleinstancedatabases.database.oracle.com sidb-sample - -kubectl get events -``` - Once TCPS connections are enabled, the database connect string will change accordingly. The TCPS connections status can also be queried by the following command: ```bash kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.isTcpsEnabled}" @@ -548,26 +540,22 @@ The following steps are required to connect the Database using TCPS: ```bash sqlplus sys@ORCL1 as sysdba ``` -- Alternatively, you can use the following SQL\*Plus command to connect using TCPS without setting TNS_ADMIN environment variable: - ```bash - sqlplus sys@tcps://?wallet_location= - ``` - Here, TCPS connect string can be found by using the following command: - ```bash - kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.TcpsConnectString}" - ``` **NOTE:** - Only database server authentication is supported (no mTLS). - When TCPS is enabled, a self-signed certificate is generated and stored inside the wallets. For users' convenience, a client-side wallet is generated and stored at `/opt/oracle/oradata/clientWallet/$ORACLE_SID` location in the pod. - The self-signed certificate used with TCPS has validity for 2 years. After the certificate is expired, it will be renewed by the `OraOperator` automatically. You need to download the wallet again after the auto-renewal. -- You can set the certificate renew interval with the help of `tcpsCertRenewInterval` field in the **[config/samples/sidb/singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml)** file. The minimum accepted value is 5m, and the maximum value is 26280h (3 years). The certificates used with TCPS will automatically be renewed after this interval. If this field is omitted/commented in the yaml file, the certificates will not be renewed automatically. +- You can set the certificate renew interval with the help of `tcpsCertRenewInterval` field in the **[config/samples/sidb/singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml)** file. The minimum accepted value is 24h, and the maximum value is 26280h (3 years). The certificates used with TCPS will automatically be renewed after this interval. If this field is omitted/commented in the yaml file, the certificates will not be renewed automatically. +- When the certificate gets created/renewed, the `.status.certCreationTimestamp` status variable gets updated accordingly. You can see this timestamp by using the following command: + ```bash + kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.certCreationTimestamp}" + ``` ### Specifying Custom Ports As mentioned in the section [Setup Database with LoadBalancer](#setup-database-with-loadbalancer), there are two kubernetes services possible for the database: NodePort and LoadBalancer. You can specify which port to use with these services by editing the `listenerPort` and `tcpsListenerPort` fields of the [config/samples/sidb/singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml) file. `listenerPort` is intended for normal database connections. Similarly, `tcpsListenerPort` is intended for TCPS database connections. -If the `LoadBalancer` is enabled, the `listenerPort`, and `tcpsListenerPort` will be the opened ports on the Load Balancer for normal and TCPS database connections respectively. +If the `LoadBalancer` is enabled, the `listenerPort`, and `tcpsListenerPort` will be the opened ports on the Load Balancer for normal and TCPS database connections respectively. The default values of `listenerPort` and `tcpsListenerPort` are 1521 and 2484 respectively when the `LoadBalancer` is enabled. In case of `NodePort` service, `listenerPort`, and `tcpsListenerPort` will be the opened ports on the Kubernetes nodes for for normal and TCPS database connections respectively. In this case, the allowed range for the `listenerPort`, and `tcpsListenerPort` is 30000-32767. From fe853c3c2f2ab07553b2f7b33d6054376949c507 Mon Sep 17 00:00:00 2001 From: matteo malvezzi Date: Mon, 26 Sep 2022 16:34:30 +0200 Subject: [PATCH 505/628] fix 34510825 + check of cert/key/ca input param --- apis/database/v1alpha1/cdb_types.go | 1 - apis/database/v1alpha1/cdb_webhook.go | 15 +- apis/database/v1alpha1/pdb_webhook.go | 15 + controllers/database/cdb_controller.go | 6 +- cscope.out | 2060 ++++++++++++++++++++++++ cscope.tmplst | 1 + 6 files changed, 2087 insertions(+), 11 deletions(-) create mode 100644 cscope.out create mode 100644 cscope.tmplst diff --git a/apis/database/v1alpha1/cdb_types.go b/apis/database/v1alpha1/cdb_types.go index 0687cb7f..9762c1cc 100644 --- a/apis/database/v1alpha1/cdb_types.go +++ b/apis/database/v1alpha1/cdb_types.go @@ -52,7 +52,6 @@ type CDBSpec struct { // Name of the CDB Service ServiceName string `json:"serviceName,omitempty"` - TestVariable string `json:"TestVariable,omitempty"` // Password for the CDB System Administrator SysAdminPwd CDBSysAdminPassword `json:"sysAdminPwd,omitempty"` diff --git a/apis/database/v1alpha1/cdb_webhook.go b/apis/database/v1alpha1/cdb_webhook.go index 5ca7f9b8..e38d037b 100644 --- a/apis/database/v1alpha1/cdb_webhook.go +++ b/apis/database/v1alpha1/cdb_webhook.go @@ -92,11 +92,16 @@ func (r *CDB) ValidateCreate() error { allErrs = append(allErrs, field.Required(field.NewPath("spec").Child("serviceName"), "Please specify CDB Service name")) } - - if r.Spec.TestVariable == "" { - allErrs = append(allErrs, - field.Required(field.NewPath("spec").Child("TestVariable"), "Please specify CDB testvarable")) - } + + if reflect.ValueOf(r.Spec.CDBTlsKey).IsZero() { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("cdbTlsKey"), "Please specify CDB Tls key(secret)")) + } + + if reflect.ValueOf(r.Spec.CDBTlsCrt).IsZero() { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("cdbTlsCrt"), "Please specify CDB Tls Certificate(secret)")) + } if r.Spec.SCANName == "" { allErrs = append(allErrs, diff --git a/apis/database/v1alpha1/pdb_webhook.go b/apis/database/v1alpha1/pdb_webhook.go index 1fb2f4cf..0e78790a 100644 --- a/apis/database/v1alpha1/pdb_webhook.go +++ b/apis/database/v1alpha1/pdb_webhook.go @@ -147,6 +147,21 @@ func (r *PDB) validateAction(allErrs *field.ErrorList) { pdblog.Info("Valdiating PDB Resource Action : " + action) + if reflect.ValueOf(r.Spec.PDBTlsKey).IsZero() { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("pdbTlsKey"), "Please specify PDB Tls Key(secret)")) + } + + if reflect.ValueOf(r.Spec.PDBTlsCrt).IsZero() { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("pdbTlsCrt"), "Please specify PDB Tls Certificate(secret)")) + } + + if reflect.ValueOf(r.Spec.PDBTlsCat).IsZero() { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("pdbTlsCat"), "Please specify PDB Tls Certificate Authority(secret)")) + } + switch action { case "CREATE": if reflect.ValueOf(r.Spec.AdminName).IsZero() { diff --git a/controllers/database/cdb_controller.go b/controllers/database/cdb_controller.go index 5c9bc3ee..02c04142 100644 --- a/controllers/database/cdb_controller.go +++ b/controllers/database/cdb_controller.go @@ -286,7 +286,7 @@ func (r *CDBReconciler) validateORDSPods(ctx context.Context, req ctrl.Request, if strings.Contains(out, "HTTP/1.1 200 OK") || strings.Contains(strings.ToUpper(err.Error()), "HTTP/1.1 200 OK") || strings.Contains(out, "HTTP/2") || strings.Contains(strings.ToUpper(err.Error()), " HTTP/2") { readyPods++ - } else if strings.Contains(out, "HTTP/1.1 404 Not Found") || strings.Contains(strings.ToUpper(err.Error()), "HTTP/1.1 404 NOT FOUND") { + } else if strings.Contains(out, "HTTP/1.1 404 Not Found") || strings.Contains(strings.ToUpper(err.Error()), "HTTP/1.1 404 NOT FOUND") || strings.Contains(strings.ToUpper(err.Error()), "HTTP/2 404") || strings.Contains(strings.ToUpper(err.Error()), "Failed to connect to localhost") { // Check if DB connection parameters are correct getORDSInstallStatus := " grep -q 'Failed to' /tmp/ords_install.log; echo $?;" out, _ := dbcommons.ExecCommand(r, r.Config, pod.Name, pod.Namespace, "", ctx, req, false, "bash", "-c", getORDSInstallStatus) @@ -449,10 +449,6 @@ func (r *CDBReconciler) createPodSpec(cdb *dbapi.CDB) corev1.PodSpec { Name: "ORACLE_HOST", Value: cdb.Spec.DBServer, }, - { - Name: "TESTVAR", - Value: cdb.Spec.TestVariable, - }, { Name: "TLSCRT", Value: cdb.Spec.CDBTlsCrt.Secret.Key, diff --git a/cscope.out b/cscope.out new file mode 100644 index 00000000..fbb75c7b --- /dev/null +++ b/cscope.out @@ -0,0 +1,2060 @@ +cscope 15 $HOME/WORKDIR/Ords22_operator/oracle-database-operator 0000008991 + @Makefile + +1 # +#CÝyrighˆ( +c +è2022, +O¿þe + +ªd +/ +Ü + +™s + +affžŸ‹s +. + +2 #Liûn£d +und” + +the + +Univ”§l + +P”missive + +Liûn£ + +v + 1.0 +as + +shown + +© + +h‰p +: + +4 #Cu¼’ˆ +O³¿tÜ + +v”siÚ + + +5 + gVERSION + ?= 0.0.1 + +6 #DeçuÉ +bundË + +image + +g + + +7 +BUNDLE_IMG + ?ð +cÚŒÞËr +- +bundË +: + $$ +( +VERSION +) + +9 + `iâeq + ( + `$ +( +Üigš + +CHANNELS +), +undefšed +) + +10 +BUNDLE_CHANNELS + :ð-- +chªÃls += + $$ +( +CHANNELS +) + +11 +’dif + + +12 + `iâeq + ( + `$ +( +Üigš + +DEFAULT_CHANNEL +), +undefšed +) + +13 +BUNDLE_DEFAULT_CHANNEL + :ð--- +chªÃl += + $$ +( +DEFAULT_CHANNEL +) + +14 +’dif + + +15 +BUNDLE_METADATA_OPTS + ?ð + `$ +( +BUNDLE_CHANNELS +è + $$ +( +BUNDLE_DEFAULT_CHANNEL +) + +17 #Imag +URL + +to + +u£ + +®l + +buždšg +/ +pushšg + +image + +rg‘s + + +18 +IMG + ?ð +cÚŒÞËr +: +Ï‹¡ + + +19 #Produû +CRDs + +th© + +wÜk + +back + +to + +Kub”Ës + 1.11 ( +no + +v”siÚ + +cÚv”siÚ +) + +20 #API +v”siÚ + +has + +to + +be + +v1 +Ø +u£ + + `deçuÉšg + ( +h‰ps +: + +21 +CRD_OPTIONS + ?= "crd:trivialVersions=true,preserveUnknownFields=false" + +22 #ENVTEST_K8S_VERSION +»ãrs + +to + +the + +v”siÚ + +of + +kubebužd” + +as£ts +Ø +be + +dowÆßded + +by + +’v‹¡ + +bš¬y +. + +23 +ENVTEST_K8S_VERSION + = 1.21 + +24 #O³¿tÜ +YAML + +fže + + +25 +OPERATOR_YAML += + `$$ +( +ba£Çme + $$( +pwd +)). +yaml + + +27 #G‘ +the + +cu¼’Žy + +u£d + +gÞªg + +š¡®l + + `·th + ( +š + +GOPATH +/ +bš +, +uÆess + +GOBIN + +is + +£t +) + +28 + `iãq + (, + $$ +( +sh–l + +go + +’v + +GOBIN +)) + +29 +GOBIN += + `$ +( +sh–l + +go + +’v + +GOPATH +)/ +bš + + +31 +GOBIN += + $$ +( +sh–l + +go + +’v + +GOBIN +) + +32 +’dif + + +34 #S‘tšg +SHELL + +to + +bash + +®lows + bash +commªds +Ø +be + +execu‹d + +by + +»ces +. + +35 #Thi  +is + +a + +»quœem’t + '£tup-’v‹¡.sh' +š + +the + +‹¡ + +rg‘ +. + +36 #O±iÚ  +¬e + +£t + +to + +ex™ + +wh’ + +a + +»ce + +lše + +ex™s + +nÚ +- +z”o + +Ü +‡ +ped + +commªd + +çžs +. + +37 +SHELL + = / +u¤ +/ +bš +/ +’v + +bash + - +o + +peçž + + +38 . +SHELLFLAGS + = - +ec + + +40 +®l +: +bužd + + +42 ##@ +Dev–Ým’t + + +44 +mªiã¡s +: +cÚŒÞËr +- +g’ + ## +G’”©e + +WebhookCÚfigu¿tiÚ +, +Clu¡”RÞe + +ªd + +Cu¡omResourûDefš™iÚ + +objeùs +. + +45 + `$ +( +CONTROLLER_GEN +è + $$ +( +CRD_OPTIONS +è +rbac +: +rÞeName += +mªag” +- +rÞe + +webhook + +·ths +="./..." +ouut +: +üd +: +¬tiçùs +: +cÚfig +=cÚfig/üd/ +ba£s + + +47 +g’”©e +: +cÚŒÞËr +- +g’ + ## +G’”©e + +code + +cÚššg + +D“pCÝy +, +D“pCÝyIÁo +, +ªd + +D“pCÝyObjeù + +m‘hod + +im¶em’tiÚs +. + +48 + $$ +( +CONTROLLER_GEN +è +objeù +: +h—d”Fže +="hack/bož”¶©e.go.txt" +·ths +="./..." + +50 +fmt +: ## +Run + +go + fmˆ +agaš¡ + +code +. + +51 +go + +fmt + ./... + +53 +v‘ +: ## +Run + +go + v‘ +agaš¡ + +code +. + +54 +go + +v‘ + ./... + +56 +TEST + ?ð./ +­is +/ +d©aba£ +/ +v1®pha1 + ./ +commÚs +/... ./ +cÚŒÞËrs +/... + +57 +‹¡ +: +mªiã¡s + +g’”©e + +fmt + +v‘ + +’v‹¡ + ## +Run + +un™ + +‹¡s +. + +58 +KUBEBUILDER_ASSETS +="$(sh–È$(ENVTESTèu£ $(ENVTEST_K8S_VERSIONè-°·th)" +go + +‹¡ + + `$ +( +TEST +è- +cov”´ofže + +cov” +. +out + + +60 +E2ETEST + ?ð./ +‹¡ +/ +e2e +/ + +61 +e2e +: +mªiã¡s + +g’”©e + +fmt + +v‘ + +’v‹¡ + ## +Run +ƒ2 +‹¡s +. + +62 +KUBEBUILDER_ASSETS +="$(sh–È$(ENVTESTèu£ $(ENVTEST_K8S_VERSIONè-°·th)" +gškgo + - +v + -- +timeout +=2 +h30m + -- +çž +- +ç¡ + + $$ +( +E2ETEST +) + +64 ##@ +Bužd + + +66 +bužd +: +g’”©e + +fmt + +v‘ + ## +Bužd + +mªag” + +bš¬y +. + +67 +go + +bužd + - +o + +bš +/ +mªag” + +maš +.go + +69 +run +: +mªiã¡s + +g’”©e + +fmt + +v‘ + ## +Run + +a + +cÚŒÞËr + +äom + +your + +ho¡ +. + +70 +go + +run + ./ +maš +.go + +72 +dock” +- +bužd +: +mªiã¡s + +g’”©e + +fmt + +v‘ + #‹¡ ## +Bužd + dock” +image + +w™h + +the + +mªag” +. +Di§bË +h +‹¡ + +but + +k“p +h +v®id©iÚs + +to + +çž + +ç¡ + + +73 +dock” + +bužd + -- +no +- +ÿche += +Œue + --bužd- +¬g + +h‰p_´oxy += +$ +{ +HTTP_PROXY +} --bužd-¬g h‰ps_´oxy=${HTTPS_PROXY} . -ˆ${IMG} + +75 #dock”- +bužd +- +´oxy +: +‹¡ + + +76 #dock” +bužd + --bužd- +¬g + +h‰p_´oxy += +$ +{h‰p_´oxy} --bužd-¬g +h‰ps_´oxy +=${h‰ps_´oxy} bužd . - +t + ${ +IMG +} + +78 +dock” +- +push +: ## +Push + dock” +image + +w™h + +the + +mªag” +. + +79 +dock” + +push + +$ +{ +IMG +} + +81 ##@ +D•loym’t + + +83 +š¡®l +: +mªiã¡s + +ku¡omize + ## +In¡®l + +CRDs + +što + +the + +K8s + +þu¡” + +¥ecif›d + +š + ~/. +kube +/ +cÚfig +. + +84 + $$ +( +KUSTOMIZE +è +bužd + +cÚfig +/ +üd + | +kubeùl + +­¶y + - +f + - + +86 +unš¡®l +: +mªiã¡s + +ku¡omize + ## +Unš¡®l + +CRDs + +äom + +the + +K8s + +þu¡” + +¥ecif›d + +š + ~/. +kube +/ +cÚfig +. + +87 + $$ +( +KUSTOMIZE +è +bužd + +cÚfig +/ +üd + | +kubeùl + +d–‘e + - +f + - + +89 +d•loy +: +mªiã¡s + +ku¡omize + ## +D•loy + +cÚŒÞËr + +to + +the + +K8s + +þu¡” + +¥ecif›d + +š + ~/. +kube +/ +cÚfig +. + +90 +cd + +cÚfig +/ +mªag” + && + $$ +( +KUSTOMIZE +è +ed™ + +£t + +image + +cÚŒÞËr += +$ +{ +IMG +} + +91 + $$ +( +KUSTOMIZE +è +bužd + +cÚfig +/ | +kubeùl + +­¶y + - +f + - + +94 #U£d +£d + +to + +»pos™iÚ + +the + +cÚŒÞËr +- +mªag” + +D•loym’t + +aá” +h +û¹ifiÿ‹ + +ü—tiÚ + +š +h +OPERATOR_YAML + + +95 +Ý”©Ü +- +yaml +: +mªiã¡s + +ku¡omize + + +96 +cd + +cÚfig +/ +mªag” + && + $$ +( +KUSTOMIZE +è +ed™ + +£t + +image + +cÚŒÞËr += +$ +{ +IMG +} + +97 + $$ +( +KUSTOMIZE +è +bužd + +cÚfig +/ > "${OPERATOR_YAML}" + +98 +£d + - +i +. +bak + - +e + '/^apiVersion:‡pps\/v1/,/---/d' "${OPERATOR_YAML}" + +99 ( +echo + --- && +£d + '/^apiVersion:‡pps\/v1/,/---/!d' "${OPERATOR_YAML}.bak") >> "${OPERATOR_YAML}" + +100 +rm + "${OPERATOR_YAML}.bak" + +102 +und•loy +: ## +Und•loy + +cÚŒÞËr + +äom + +the + +K8s + +þu¡” + +¥ecif›d + +š + ~/. +kube +/ +cÚfig +. + +103 + $$ +( +KUSTOMIZE +è +bužd + +cÚfig +/ | +kubeùl + +d–‘e + - +f + - + +105 ##@ +Bužd + +D•’d’c›s + + +107 ## +LoÿtiÚ + +to + +š¡®l + +d•’d’c›s +o + +108 +LOCALBIN + ?ð + `$ +( +sh–l + +pwd +)/ +bš + + +109 + $$ +( +LOCALBIN +): + +110 +mkdœ + - +p + + $$ +( +LOCALBIN +) + +112 ## +ToÞ + +Bš¬›s + + +113 +KUSTOMIZE + ?ð + `$ +( +LOCALBIN +)/ +ku¡omize + + +114 +CONTROLLER_GEN + ?ð + `$ +( +LOCALBIN +)/ +cÚŒÞËr +- +g’ + + +115 +ENVTEST + ?ð + `$ +( +LOCALBIN +)/ +£tup +- +’v‹¡ + + +117 ## +ToÞ + +V”siÚs + + +118 +KUSTOMIZE_VERSION + ?ð +v3 +.8.7 + +119 +CONTROLLER_TOOLS_VERSION + ?ð +v0 +.6.1 + +121 +KUSTOMIZE_INSTALL_SCRIPT + ?= "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh" + +122 . +PHONY +: +ku¡omize + + +123 +ku¡omize +: + $$ +( +KUSTOMIZE +è## +DowÆßd + +ku¡omize + +loÿÎy +  +Ãûs§ry +. + +124 + $$ +( +KUSTOMIZE +): + $$ +( +LOCALBIN +) + +125 +cu¾ + - +s + + `$ +( +KUSTOMIZE_INSTALL_SCRIPT +è| +bash + - -- $( +sub¡ + +v +,,$( +KUSTOMIZE_VERSION +)è$( +LOCALBIN +) + +127 . +PHONY +: +cÚŒÞËr +- +g’ + + +128 +cÚŒÞËr +- +g’ +: + $$ +( +CONTROLLER_GEN +è## +DowÆßd + +cÚŒÞËr +- +g’ + +loÿÎy +  +Ãûs§ry +. + +129 + $$ +( +CONTROLLER_GEN +): + $$ +( +LOCALBIN +) + +130 +GOBIN += + $$ +( +LOCALBIN +è +go + +š¡®l + +sigs +. +k8s +. +io +/ +cÚŒÞËr +- +toÞs +/ +cmd +/cÚŒÞËr- +g’ +@ + `$ +( +CONTROLLER_TOOLS_VERSION +) + +132 . +PHONY +: +’v‹¡ + + +133 +’v‹¡ +: + $$ +( +ENVTEST +è## +DowÆßd + +’v‹¡ +- +£tup + +loÿÎy +  +Ãûs§ry +. + +134 + $$ +( +ENVTEST +): + $$ +( +LOCALBIN +) + +135 +GOBIN += + $$ +( +LOCALBIN +è +go + +š¡®l + +sigs +. +k8s +. +io +/ +cÚŒÞËr +- +ruÁime +/ +toÞs +/ +£tup +- +’v‹¡ +@ +Ï‹¡ + + +138 . +PHONY +: +bundË + + +139 +bundË +: +mªiã¡s + +ku¡omize + ## +G’”©e + bundË mªiã¡  +ªd + +m‘ad©a +, +th’ + +v®id©e + +g’”©ed + +fžes +. + +140 +Ý”©Ü +- +sdk + +g’”©e + +ku¡omize + +mªiã¡s + - +q + + +141 +cd + +cÚfig +/ +mªag” + && + $$ +( +KUSTOMIZE +è +ed™ + +£t + +image + +cÚŒÞËr += + `$ +( +IMG +) + +142 + $$ +( +KUSTOMIZE +è +bužd + +cÚfig +/ +mªiã¡s + | +Ý”©Ü +- +sdk + +g’”©e + +bundË + - +q + -- +ov”wr™e + -- +v”siÚ + + `$ +( +VERSION +è + $$ +( +BUNDLE_METADATA_OPTS +) + +143 +Ý”©Ü +- +sdk + +bundË + +v®id©e + ./bundle + +145 . +PHONY +: +bundË +- +bužd + + +146 +bundË +- +bužd +: ## +Bužd + +the + bundË +image +. + +147 +dock” + +bužd + - +f + +bundË +. +Dock”fže + - +t + + `$ +( +BUNDLE_IMG +) . + +149 . +PHONY +: +bundË +- +push + + +150 +bundË +- +push +: ## +Push + +the + bundË +image +. + +151 + $$ +( +MAKE +è +dock” +- +push + +IMG += + `$ +( +BUNDLE_IMG +) + +153 . +PHONY +: +Ým + + +154 +OPM + = ./ +bš +/ +Ým + + +155 +Ým +: ## +DowÆßd + opm +loÿÎy +  +Ãûs§ry +. + +156 + `iãq + (, + `$ +( +wždÿrd + + $$ +( +OPM +))) + +157 + `iãq + (, + `$ +( +sh–l + +which + +Ým + 2>/ +dev +/ +nuÎ +)) + +159 +£t + - +e + ;\ + +160 +mkdœ + - +p + + `$ +( +dœ + $( +OPM +)) ;\ + +161 +OS += + `$ +( +sh–l + +go + +’v + +GOOS +è&& +ARCH +=$(sh–ÈgØ’v +GOARCH +) && \ + +162 +cu¾ + - +sSLo + + `$ +( +OPM +è +h‰ps +: + +163 +chmod + + +x + + `$ +( +OPM +) ;\ + +164 + } +} + +166 + gOPM + = + $$ +( +sh–l + +which + +Ým +) + +167 +’dif + + +168 +’dif + + +170 #A +comma +- +£·¿‹d + +li¡ + +of + +bundË + + `images + ( +e +. +g +. +make + +ÿlog +- +bužd + +BUNDLE_IMGS += +exam¶e +. +com +/ +Ý”©Ü +-bundË: +v0 +.1.0,example.com/operator-bundle:v0.2.0). + +171 #The£ +images + +MUST + +exi¡ + +š + +a + +»gi¡ry + +ªd + +be + +puÎ +- +abË +. + +172 +BUNDLE_IMGS + ?ð + $$ +( +BUNDLE_IMG +) + +174 #Th +image + +g + +giv’ + +to + +the + +»suÉšg + +ÿlog + + `image + ( +e +. +g +. +make + c©®og- +bužd + +CATALOG_IMG += +exam¶e +. +com +/ +Ý”©Ü +-ÿlog: +v0 +.2.0). + +175 +CATALOG_IMG + ?ð + `$ +( +IMAGE_TAG_BASE +)- +ÿlog +: + $v$ +( +VERSION +) + +177 #S‘ +CATALOG_BASE_IMG + +to + +ª + +exi¡šg + +ÿlog + +image + +g +Ø +add + +$BUNDLE_IMGS +Ø +th© + image. + +178 + `iâeq + ( + `$ +( +Üigš + +CATALOG_BASE_IMG +), +undefšed +) + +179 +FROM_INDEX_OPT + :ð-- +äom +- +šdex + + $$ +( +CATALOG_BASE_IMG +) + +180 +’dif + + +182 #Bužd +a + +ÿlog + +image + +by + +addšg + +bundË + +images + +to + +ª + +em±y + c©®og +usšg + +the + +Ý”©Ü + +·ckage + +mªag” + +toÞ +, 'opm'. + +183 #Thi  +»ce + +švokes + 'Ým' +š + '£mv”' +bundË + +add + +mode +. +FÜ + +mÜe + +šfÜm©iÚ + +Ú +‡dd +modes +, +£e +: + +185 . +PHONY +: +ÿlog +- +bužd + + +186 +ÿlog +- +bužd +: +Ým + ## +Bužd + +a + c©®og +image +. + +187 + $$ +( +OPM +è +šdex + +add + -- +cÚš” +- +toÞ + +dock” + -- +mode + +£mv” + -- +g + + `$ +( +CATALOG_IMG +è-- +bundËs + $( +BUNDLE_IMGS +è + $$ +( +FROM_INDEX_OPT +) + +189 #Push +the + +ÿlog + +image +. + +190 . +PHONY +: +ÿlog +- +push + + +191 +ÿlog +- +push +: ## +Push + +a + c©®og +image +. + +192 + $$ +( +MAKE +è +dock” +- +push + +IMG += + `$ +( +CATALOG_IMG +) + + @ +1 +. +1 +/usr/include +1 +9 +Makefile diff --git a/cscope.tmplst b/cscope.tmplst new file mode 100644 index 00000000..0937d485 --- /dev/null +++ b/cscope.tmplst @@ -0,0 +1 @@ +./Makefile From 8990cbf2057fc003a35286a37d187fd0f86d2060 Mon Sep 17 00:00:00 2001 From: Abhishek Kumar Date: Thu, 29 Sep 2022 16:13:28 +0000 Subject: [PATCH 506/628] Patching the svc while changing type from LB to NodePort, max cert validity 1yr --- .../singleinstancedatabase_webhook.go | 4 +- commons/database/constants.go | 4 +- .../crd/bases/database.oracle.com_cdbs.yaml | 2 - .../samples/sidb/singleinstancedatabase.yaml | 6 +- .../sidb/singleinstancedatabase_tcps.yaml | 6 +- .../singleinstancedatabase_controller.go | 84 +++++++------------ docs/sidb/README.md | 4 +- 7 files changed, 44 insertions(+), 66 deletions(-) diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index 1006c68e..6323550b 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -270,12 +270,12 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { field.Invalid(field.NewPath("spec").Child("tcpsCertRenewInterval"), r.Spec.TcpsCertRenewInterval, "Please provide valid string to parse the tcpsCertRenewInterval.")) } - maxLimit, _ := time.ParseDuration("26280h") + maxLimit, _ := time.ParseDuration("8760h") minLimit, _ := time.ParseDuration("24h") if duration > maxLimit || duration < minLimit { allErrs = append(allErrs, field.Invalid(field.NewPath("spec").Child("tcpsCertRenewInterval"), r.Spec.TcpsCertRenewInterval, - "Please specify tcpsCertRenewInterval in the range: 24h to 26280h")) + "Please specify tcpsCertRenewInterval in the range: 24h to 8760h")) } } if len(allErrs) == 0 { diff --git a/commons/database/constants.go b/commons/database/constants.go index a1d7bfc8..76750ffb 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -501,10 +501,10 @@ const ClientWalletLocation string = "/opt/oracle/oradata/clientWallet/%s" // Service Patch Payloads // Three port payload: one OEM express, one TCP and one TCPS port -const ThreePortPayload string = "{\"spec\": { \"ports\": [{\"name\": \"xmldb\", \"port\": 5500, \"protocol\": \"TCP\"},{%s},{%s}]}}" +const ThreePortPayload string = "{\"spec\": { \"type\": \"%s\", \"ports\": [{\"name\": \"xmldb\", \"port\": 5500, \"protocol\": \"TCP\"},{%s},{%s}]}}" // Two port payload: one OEM express, one TCP/TCPS port -const TwoPortPayload string = "{\"spec\": { \"ports\": [{\"name\": \"xmldb\", \"port\": 5500, \"protocol\": \"TCP\"},{%s}]}}" +const TwoPortPayload string = "{\"spec\": { \"type\": \"%s\", \"ports\": [{\"name\": \"xmldb\", \"port\": 5500, \"protocol\": \"TCP\"},{%s}]}}" // Payload section for listener port const LsnrPort string = "\"name\": \"listener\", \"protocol\": \"TCP\", \"port\": %d, \"targetPort\": 1521" diff --git a/config/crd/bases/database.oracle.com_cdbs.yaml b/config/crd/bases/database.oracle.com_cdbs.yaml index bf64810b..26eec406 100644 --- a/config/crd/bases/database.oracle.com_cdbs.yaml +++ b/config/crd/bases/database.oracle.com_cdbs.yaml @@ -65,8 +65,6 @@ spec: spec: description: CDBSpec defines the desired state of CDB properties: - TestVariable: - type: string cdbAdminPwd: description: Password for the CDB Administrator to manage PDB lifecycle properties: diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index 0fafd46e..f836f6dd 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -48,10 +48,10 @@ spec: enableTCPS: false ## TCPS Certificate Renewal Interval: The time after which TCPS certificate will be renewed if TCPS connections are enabled. - ## tcpsCertRenewInterval can be in hours(h), minutes(m) and seconds(s); e.g. 17520h, 8760h etc. - ## Maximum value is 26280h (3 years), Minimum value is 24h; Default value is 17520h (2 years) + ## tcpsCertRenewInterval can be in hours(h), minutes(m) and seconds(s); e.g. 4380h, 8760h etc. + ## Maximum value is 8760h (1 year), Minimum value is 24h; Default value is 8760h (1 year) ## If this field is commented out/removed from the yaml, it will disable the auto-renewal feature for TCPS certificate - tcpsCertRenewInterval: 17520h + tcpsCertRenewInterval: 8760h ## NA if cloning from a SourceDB (cloneFrom is set) ## Specify both sgaSize and pgaSize (in MB) or dont specify both diff --git a/config/samples/sidb/singleinstancedatabase_tcps.yaml b/config/samples/sidb/singleinstancedatabase_tcps.yaml index b4a64ba8..2f9e4994 100644 --- a/config/samples/sidb/singleinstancedatabase_tcps.yaml +++ b/config/samples/sidb/singleinstancedatabase_tcps.yaml @@ -35,10 +35,10 @@ spec: enableTCPS: true ## TCPS Certificate Renewal Interval: The time after which TCPS certificate will be renewed if TCPS connections are enabled. - ## tcpsCertRenewInterval can be in hours(h), minutes(m) and seconds(s); e.g. 17520h, 8760h etc. - ## Maximum value is 26280h (3 years), Minimum value is 24h; Default value is 17520h (2 years) + ## tcpsCertRenewInterval can be in hours(h), minutes(m) and seconds(s); e.g. 4380h, 8760h etc. + ## Maximum value is 8760h (1 year), Minimum value is 24h; Default value is 8760h (1 year) ## If this field is commented out/removed from the yaml, it will disable the auto-renewal feature for TCPS certificate - tcpsCertRenewInterval: 17520h + tcpsCertRenewInterval: 8760h ## Database image details image: diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 77d40f90..314b2c66 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1126,57 +1126,37 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex } } - deleteSvc := false patchSvc := false - if extSvc.Spec.Type != extSvcType { - deleteSvc = true - } else { - // Conditions to determine whether to patch or not - if len(extSvc.Spec.Ports) != requiredPorts { - patchSvc = true - } - if (m.Spec.ListenerPort != 0 && svcPort != targetPorts[1]) || (m.Spec.EnableTCPS && m.Spec.TcpsListenerPort != 0 && tcpsSvcPort != targetPorts[len(targetPorts)-1]) { - patchSvc = true - } + // Conditions to determine whether to patch or not + if extSvc.Spec.Type != extSvcType || len(extSvc.Spec.Ports) != requiredPorts { + patchSvc = true + } - if m.Spec.LoadBalancer { - if m.Spec.EnableTCPS { - if m.Spec.TcpsListenerPort == 0 && tcpsSvcPort != targetPorts[len(targetPorts)-1] { - patchSvc = true - } - } else { - if m.Spec.ListenerPort == 0 && svcPort != targetPorts[1] { - patchSvc = true - } + if (m.Spec.ListenerPort != 0 && svcPort != targetPorts[1]) || (m.Spec.EnableTCPS && m.Spec.TcpsListenerPort != 0 && tcpsSvcPort != targetPorts[len(targetPorts)-1]) { + patchSvc = true + } + + if m.Spec.LoadBalancer { + if m.Spec.EnableTCPS { + if m.Spec.TcpsListenerPort == 0 && tcpsSvcPort != targetPorts[len(targetPorts)-1] { + patchSvc = true } } else { - if m.Spec.EnableTCPS { - if m.Spec.TcpsListenerPort == 0 && tcpsSvcPort != extSvc.Spec.Ports[len(targetPorts)-1].TargetPort.IntVal { - patchSvc = true - } - } else { - if m.Spec.ListenerPort == 0 && svcPort != extSvc.Spec.Ports[1].TargetPort.IntVal { - patchSvc = true - } + if m.Spec.ListenerPort == 0 && svcPort != targetPorts[1] { + patchSvc = true } } - } - - if deleteSvc { - // Deleting th service - log.Info("Deleting service", "name", extSvcName) - // Setting GracePeriodSeconds to 0 for instant deletion - delOpts := &client.DeleteOptions{} - var gracePeriod client.GracePeriodSeconds = 0 - gracePeriod.ApplyToDelete(delOpts) - - err := r.Delete(ctx, extSvc, delOpts) - if err != nil { - r.Log.Error(err, "Failed to delete service", "name", extSvcName) - return requeueN, err + } else { + if m.Spec.EnableTCPS { + if m.Spec.TcpsListenerPort == 0 && tcpsSvcPort != extSvc.Spec.Ports[len(targetPorts)-1].TargetPort.IntVal { + patchSvc = true + } + } else { + if m.Spec.ListenerPort == 0 && svcPort != extSvc.Spec.Ports[1].TargetPort.IntVal { + patchSvc = true + } } - isExtSvcFound = false } if patchSvc { @@ -1193,29 +1173,29 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex if m.Spec.LoadBalancer { if m.Spec.EnableTCPS { if m.Spec.ListenerPort != 0 { - payload = fmt.Sprintf(dbcommons.ThreePortPayload, fmt.Sprintf(dbcommons.LsnrPort, svcPort), fmt.Sprintf(dbcommons.TcpsPort, tcpsSvcPort)) + payload = fmt.Sprintf(dbcommons.ThreePortPayload, extSvcType, fmt.Sprintf(dbcommons.LsnrPort, svcPort), fmt.Sprintf(dbcommons.TcpsPort, tcpsSvcPort)) } else { - payload = fmt.Sprintf(dbcommons.TwoPortPayload, fmt.Sprintf(dbcommons.TcpsPort, tcpsSvcPort)) + payload = fmt.Sprintf(dbcommons.TwoPortPayload, extSvcType, fmt.Sprintf(dbcommons.TcpsPort, tcpsSvcPort)) } } else { - payload = fmt.Sprintf(dbcommons.TwoPortPayload, fmt.Sprintf(dbcommons.LsnrPort, svcPort)) + payload = fmt.Sprintf(dbcommons.TwoPortPayload, extSvcType, fmt.Sprintf(dbcommons.LsnrPort, svcPort)) } } else { if m.Spec.EnableTCPS { if m.Spec.ListenerPort != 0 && m.Spec.TcpsListenerPort != 0 { - payload = fmt.Sprintf(dbcommons.ThreePortPayload, fmt.Sprintf(dbcommons.LsnrNodePort, svcPort), fmt.Sprintf(dbcommons.TcpsNodePort, tcpsSvcPort)) + payload = fmt.Sprintf(dbcommons.ThreePortPayload, extSvcType, fmt.Sprintf(dbcommons.LsnrNodePort, svcPort), fmt.Sprintf(dbcommons.TcpsNodePort, tcpsSvcPort)) } else if m.Spec.ListenerPort != 0 { - payload = fmt.Sprintf(dbcommons.ThreePortPayload, fmt.Sprintf(dbcommons.LsnrNodePort, svcPort), fmt.Sprintf(dbcommons.TcpsPort, tcpsSvcPort)) + payload = fmt.Sprintf(dbcommons.ThreePortPayload, extSvcType, fmt.Sprintf(dbcommons.LsnrNodePort, svcPort), fmt.Sprintf(dbcommons.TcpsPort, tcpsSvcPort)) } else if m.Spec.TcpsListenerPort != 0 { - payload = fmt.Sprintf(dbcommons.TwoPortPayload, fmt.Sprintf(dbcommons.TcpsNodePort, tcpsSvcPort)) + payload = fmt.Sprintf(dbcommons.TwoPortPayload, extSvcType, fmt.Sprintf(dbcommons.TcpsNodePort, tcpsSvcPort)) } else { - payload = fmt.Sprintf(dbcommons.TwoPortPayload, fmt.Sprintf(dbcommons.TcpsPort, tcpsSvcPort)) + payload = fmt.Sprintf(dbcommons.TwoPortPayload, extSvcType, fmt.Sprintf(dbcommons.TcpsPort, tcpsSvcPort)) } } else { if m.Spec.ListenerPort != 0 { - payload = fmt.Sprintf(dbcommons.TwoPortPayload, fmt.Sprintf(dbcommons.LsnrNodePort, svcPort)) + payload = fmt.Sprintf(dbcommons.TwoPortPayload, extSvcType, fmt.Sprintf(dbcommons.LsnrNodePort, svcPort)) } else { - payload = fmt.Sprintf(dbcommons.TwoPortPayload, fmt.Sprintf(dbcommons.LsnrPort, svcPort)) + payload = fmt.Sprintf(dbcommons.TwoPortPayload, extSvcType, fmt.Sprintf(dbcommons.LsnrPort, svcPort)) } } } diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 1e9f02c3..33257d24 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -543,8 +543,8 @@ The following steps are required to connect the Database using TCPS: **NOTE:** - Only database server authentication is supported (no mTLS). - When TCPS is enabled, a self-signed certificate is generated and stored inside the wallets. For users' convenience, a client-side wallet is generated and stored at `/opt/oracle/oradata/clientWallet/$ORACLE_SID` location in the pod. -- The self-signed certificate used with TCPS has validity for 2 years. After the certificate is expired, it will be renewed by the `OraOperator` automatically. You need to download the wallet again after the auto-renewal. -- You can set the certificate renew interval with the help of `tcpsCertRenewInterval` field in the **[config/samples/sidb/singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml)** file. The minimum accepted value is 24h, and the maximum value is 26280h (3 years). The certificates used with TCPS will automatically be renewed after this interval. If this field is omitted/commented in the yaml file, the certificates will not be renewed automatically. +- The self-signed certificate used with TCPS has validity for 1 year. After the certificate is expired, it will be renewed by the `OraOperator` automatically. You need to download the wallet again after the auto-renewal. +- You can set the certificate renew interval with the help of `tcpsCertRenewInterval` field in the **[config/samples/sidb/singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml)** file. The minimum accepted value is 24h, and the maximum value is 8760h (1 year). The certificates used with TCPS will automatically be renewed after this interval. If this field is omitted/commented in the yaml file, the certificates will not be renewed automatically. - When the certificate gets created/renewed, the `.status.certCreationTimestamp` status variable gets updated accordingly. You can see this timestamp by using the following command: ```bash kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.certCreationTimestamp}" From 0fc0c7448a3845141d5bfdecf37aed37250ab0b4 Mon Sep 17 00:00:00 2001 From: matteo malvezzi Date: Mon, 10 Oct 2022 09:06:03 +0200 Subject: [PATCH 507/628] bug 34677093 --- apis/database/v1alpha1/cdb_types.go | 3 - apis/database/v1alpha1/cdb_webhook.go | 5 +- .../crd/bases/database.oracle.com_cdbs.yaml | 7 - config/samples/multitenant/cdb.yaml | 3 +- controllers/DBserver | 33 + controllers/database/pdb_controller.go | 14 +- cscope.out | 2060 ----------------- cscope.tmplst | 1 - .../multitenant/provisioning/add_replica.yaml | 1 - docs/multitenant/provisioning/cdb.yaml | 1 - oracle-database-operator.yaml | 40 +- 11 files changed, 74 insertions(+), 2094 deletions(-) create mode 100644 controllers/DBserver delete mode 100644 cscope.out delete mode 100644 cscope.tmplst diff --git a/apis/database/v1alpha1/cdb_types.go b/apis/database/v1alpha1/cdb_types.go index 9762c1cc..a987718a 100644 --- a/apis/database/v1alpha1/cdb_types.go +++ b/apis/database/v1alpha1/cdb_types.go @@ -80,8 +80,6 @@ type CDBSpec struct { WebServerUser WebServerUser `json:"webServerUser,omitempty"` // Password for the Web Server User WebServerPwd WebServerPassword `json:"webServerPwd,omitempty"` - // SCAN Name - SCANName string `json:"scanName,omitempty"` // Name of the DB server DBServer string `json:"dbServer,omitempty"` // DB server port @@ -152,7 +150,6 @@ type CDBStatus struct { // +kubebuilder:printcolumn:JSONPath=".spec.cdbName",name="CDB Name",type="string",description="Name of the CDB" // +kubebuilder:printcolumn:JSONPath=".spec.dbServer",name="DB Server",type="string",description=" Name of the DB Server" // +kubebuilder:printcolumn:JSONPath=".spec.dbPort",name="DB Port",type="integer",description="DB server port" -// +kubebuilder:printcolumn:JSONPath=".spec.scanName",name="SCAN Name",type="string",description="SCAN Name" // +kubebuilder:printcolumn:JSONPath=".spec.replicas",name="Replicas",type="integer",description="Replicas" // +kubebuilder:printcolumn:JSONPath=".status.phase",name="Status",type="string",description="Status of the CDB Resource" // +kubebuilder:printcolumn:JSONPath=".status.msg",name="Message",type="string",description="Error message, if any" diff --git a/apis/database/v1alpha1/cdb_webhook.go b/apis/database/v1alpha1/cdb_webhook.go index e38d037b..c760875c 100644 --- a/apis/database/v1alpha1/cdb_webhook.go +++ b/apis/database/v1alpha1/cdb_webhook.go @@ -103,10 +103,11 @@ func (r *CDB) ValidateCreate() error { field.Required(field.NewPath("spec").Child("cdbTlsCrt"), "Please specify CDB Tls Certificate(secret)")) } - if r.Spec.SCANName == "" { + /*if r.Spec.SCANName == "" { allErrs = append(allErrs, field.Required(field.NewPath("spec").Child("scanName"), "Please specify SCAN Name for CDB")) - } + }*/ + if r.Spec.DBServer == "" { allErrs = append(allErrs, field.Required(field.NewPath("spec").Child("dbServer"), "Please specify Database Server Name or IP Address")) diff --git a/config/crd/bases/database.oracle.com_cdbs.yaml b/config/crd/bases/database.oracle.com_cdbs.yaml index 26eec406..ab2ab2ef 100644 --- a/config/crd/bases/database.oracle.com_cdbs.yaml +++ b/config/crd/bases/database.oracle.com_cdbs.yaml @@ -29,10 +29,6 @@ spec: jsonPath: .spec.dbPort name: DB Port type: integer - - description: SCAN Name - jsonPath: .spec.scanName - name: SCAN Name - type: string - description: Replicas jsonPath: .spec.replicas name: Replicas @@ -183,9 +179,6 @@ spec: replicas: description: Number of ORDS Containers to create type: integer - scanName: - description: SCAN Name - type: string serviceName: description: Name of the CDB Service type: string diff --git a/config/samples/multitenant/cdb.yaml b/config/samples/multitenant/cdb.yaml index 7b566cd7..e3513d12 100644 --- a/config/samples/multitenant/cdb.yaml +++ b/config/samples/multitenant/cdb.yaml @@ -9,7 +9,6 @@ metadata: namespace: oracle-database-operator-system spec: cdbName: "devcdb" - scanName: "devdb" dbServer: "172.17.0.4" dbPort: 1521 replicas: 1 @@ -41,4 +40,4 @@ spec: webServerPwd: secret: secretName: "cdb1-secret" - key: "webserver_pwd" \ No newline at end of file + key: "webserver_pwd" diff --git a/controllers/DBserver b/controllers/DBserver new file mode 100644 index 00000000..57c0f4fb --- /dev/null +++ b/controllers/DBserver @@ -0,0 +1,33 @@ +# This viminfo file was generated by Vim 7.4. +# You may edit it if you're careful! + +# Value of 'encoding' when this file was written +*encoding=utf-8 + + +# hlsearch on (H) or off (h): +~H +# Command Line History (newest to oldest): +:q! +:q + +# Search String History (newest to oldest): + +# Expression History (newest to oldest): + +# Input Line History (newest to oldest): + +# Input Line History (newest to oldest): + +# Registers: + +# File marks: +'0 1 0 ~/WORKDIR/Ords22_operator/oracle-database-operator/controllers/grep + +# Jumplist (newest first): +-' 1 0 ~/WORKDIR/Ords22_operator/oracle-database-operator/controllers/grep + +# History of marks within files (newest to oldest): + +> ~/WORKDIR/Ords22_operator/oracle-database-operator/controllers/grep + " 1 0 diff --git a/controllers/database/pdb_controller.go b/controllers/database/pdb_controller.go index 099cb921..07580725 100644 --- a/controllers/database/pdb_controller.go +++ b/controllers/database/pdb_controller.go @@ -444,10 +444,11 @@ func (r *PDBReconciler) callAPI(ctx context.Context, req ctrl.Request, pdb *dbap } caCert := secret.Data[pdb.Spec.PDBTlsCat.Secret.Key] - + /* r.Recorder.Eventf(pdb, corev1.EventTypeWarning, "ORDSINFO", string(rsaKeyPEM)) r.Recorder.Eventf(pdb, corev1.EventTypeWarning, "ORDSINFO", string(rsaCertPEM)) r.Recorder.Eventf(pdb, corev1.EventTypeWarning, "ORDSINFO", string(caCert)) + */ certificate, err := tls.X509KeyPair([]byte(rsaCertPEM), []byte(rsaKeyPEM)) if err != nil { @@ -501,7 +502,6 @@ func (r *PDBReconciler) callAPI(ctx context.Context, req ctrl.Request, pdb *dbap if action == "GET" { httpreq, err = http.NewRequest(action, url, nil) } else { - fmt.Println("payload:", payload) jsonValue, _ := json.Marshal(payload) httpreq, err = http.NewRequest(action, url, bytes.NewBuffer(jsonValue)) } @@ -641,7 +641,7 @@ func (r *PDBReconciler) createPDB(ctx context.Context, req ctrl.Request, pdb *db r.Recorder.Eventf(pdb, corev1.EventTypeNormal, "Created", "PDB '%s' created successfully", pdb.Spec.PDBName) - pdb.Status.ConnString = cdb.Spec.SCANName + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName + pdb.Status.ConnString = cdb.Spec.DBServer + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName log.Info("Created PDB Resource", "PDB Name", pdb.Spec.PDBName) r.getPDBState(ctx, req, pdb) return nil @@ -695,7 +695,7 @@ func (r *PDBReconciler) clonePDB(ctx context.Context, req ctrl.Request, pdb *dba r.Recorder.Eventf(pdb, corev1.EventTypeNormal, "Created", "PDB '%s' cloned successfully", pdb.Spec.PDBName) - pdb.Status.ConnString = cdb.Spec.SCANName + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName + pdb.Status.ConnString = cdb.Spec.DBServer + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName log.Info("Cloned PDB successfully", "Source PDB Name", pdb.Spec.SrcPDBName, "Clone PDB Name", pdb.Spec.PDBName) r.getPDBState(ctx, req, pdb) return nil @@ -763,7 +763,7 @@ func (r *PDBReconciler) plugPDB(ctx context.Context, req ctrl.Request, pdb *dbap r.Recorder.Eventf(pdb, corev1.EventTypeNormal, "Created", "PDB '%s' plugged successfully", pdb.Spec.PDBName) - pdb.Status.ConnString = cdb.Spec.SCANName + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName + pdb.Status.ConnString = cdb.Spec.DBServer + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName log.Info("Successfully plugged PDB", "PDB Name", pdb.Spec.PDBName) r.getPDBState(ctx, req, pdb) return nil @@ -883,7 +883,7 @@ func (r *PDBReconciler) modifyPDB(ctx context.Context, req ctrl.Request, pdb *db } r.Recorder.Eventf(pdb, corev1.EventTypeNormal, "Modified", "PDB '%s' modified successfully", pdb.Spec.PDBName) - pdb.Status.ConnString = cdb.Spec.SCANName + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName + pdb.Status.ConnString = cdb.Spec.DBServer + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName log.Info("Successfully modified PDB state", "PDB Name", pdb.Spec.PDBName) r.getPDBState(ctx, req, pdb) @@ -970,7 +970,7 @@ func (r *PDBReconciler) mapPDB(ctx context.Context, req ctrl.Request, pdb *dbapi pdb.Status.OpenMode = objmap["open_mode"].(string) pdb.Status.TotalSize = fmt.Sprintf("%.2f", totSizeInGB) + "G" - pdb.Status.ConnString = cdb.Spec.SCANName + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName + pdb.Status.ConnString = cdb.Spec.DBServer + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName log.Info("Successfully mapped PDB to Kubernetes resource", "PDB Name", pdb.Spec.PDBName) return nil diff --git a/cscope.out b/cscope.out deleted file mode 100644 index fbb75c7b..00000000 --- a/cscope.out +++ /dev/null @@ -1,2060 +0,0 @@ -cscope 15 $HOME/WORKDIR/Ords22_operator/oracle-database-operator 0000008991 - @Makefile - -1 # -#CÝyrighˆ( -c -è2022, -O¿þe - -ªd -/ -Ü - -™s - -affžŸ‹s -. - -2 #Liûn£d -und” - -the - -Univ”§l - -P”missive - -Liûn£ - -v - 1.0 -as - -shown - -© - -h‰p -: - -4 #Cu¼’ˆ -O³¿tÜ - -v”siÚ - - -5 - gVERSION - ?= 0.0.1 - -6 #DeçuÉ -bundË - -image - -g - - -7 -BUNDLE_IMG - ?ð -cÚŒÞËr -- -bundË -: - $$ -( -VERSION -) - -9 - `iâeq - ( - `$ -( -Üigš - -CHANNELS -), -undefšed -) - -10 -BUNDLE_CHANNELS - :ð-- -chªÃls -= - $$ -( -CHANNELS -) - -11 -’dif - - -12 - `iâeq - ( - `$ -( -Üigš - -DEFAULT_CHANNEL -), -undefšed -) - -13 -BUNDLE_DEFAULT_CHANNEL - :ð--- -chªÃl -= - $$ -( -DEFAULT_CHANNEL -) - -14 -’dif - - -15 -BUNDLE_METADATA_OPTS - ?ð - `$ -( -BUNDLE_CHANNELS -è - $$ -( -BUNDLE_DEFAULT_CHANNEL -) - -17 #Imag -URL - -to - -u£ - -®l - -buždšg -/ -pushšg - -image - -rg‘s - - -18 -IMG - ?ð -cÚŒÞËr -: -Ï‹¡ - - -19 #Produû -CRDs - -th© - -wÜk - -back - -to - -Kub”Ës - 1.11 ( -no - -v”siÚ - -cÚv”siÚ -) - -20 #API -v”siÚ - -has - -to - -be - -v1 -Ø -u£ - - `deçuÉšg - ( -h‰ps -: - -21 -CRD_OPTIONS - ?= "crd:trivialVersions=true,preserveUnknownFields=false" - -22 #ENVTEST_K8S_VERSION -»ãrs - -to - -the - -v”siÚ - -of - -kubebužd” - -as£ts -Ø -be - -dowÆßded - -by - -’v‹¡ - -bš¬y -. - -23 -ENVTEST_K8S_VERSION - = 1.21 - -24 #O³¿tÜ -YAML - -fže - - -25 -OPERATOR_YAML -= - `$$ -( -ba£Çme - $$( -pwd -)). -yaml - - -27 #G‘ -the - -cu¼’Žy - -u£d - -gÞªg - -š¡®l - - `·th - ( -š - -GOPATH -/ -bš -, -uÆess - -GOBIN - -is - -£t -) - -28 - `iãq - (, - $$ -( -sh–l - -go - -’v - -GOBIN -)) - -29 -GOBIN -= - `$ -( -sh–l - -go - -’v - -GOPATH -)/ -bš - - -31 -GOBIN -= - $$ -( -sh–l - -go - -’v - -GOBIN -) - -32 -’dif - - -34 #S‘tšg -SHELL - -to - -bash - -®lows - bash -commªds -Ø -be - -execu‹d - -by - -»ces -. - -35 #Thi  -is - -a - -»quœem’t - '£tup-’v‹¡.sh' -š - -the - -‹¡ - -rg‘ -. - -36 #O±iÚ  -¬e - -£t - -to - -ex™ - -wh’ - -a - -»ce - -lše - -ex™s - -nÚ -- -z”o - -Ü -‡ -ped - -commªd - -çžs -. - -37 -SHELL - = / -u¤ -/ -bš -/ -’v - -bash - - -o - -peçž - - -38 . -SHELLFLAGS - = - -ec - - -40 -®l -: -bužd - - -42 ##@ -Dev–Ým’t - - -44 -mªiã¡s -: -cÚŒÞËr -- -g’ - ## -G’”©e - -WebhookCÚfigu¿tiÚ -, -Clu¡”RÞe - -ªd - -Cu¡omResourûDefš™iÚ - -objeùs -. - -45 - `$ -( -CONTROLLER_GEN -è - $$ -( -CRD_OPTIONS -è -rbac -: -rÞeName -= -mªag” -- -rÞe - -webhook - -·ths -="./..." -ouut -: -üd -: -¬tiçùs -: -cÚfig -=cÚfig/üd/ -ba£s - - -47 -g’”©e -: -cÚŒÞËr -- -g’ - ## -G’”©e - -code - -cÚššg - -D“pCÝy -, -D“pCÝyIÁo -, -ªd - -D“pCÝyObjeù - -m‘hod - -im¶em’tiÚs -. - -48 - $$ -( -CONTROLLER_GEN -è -objeù -: -h—d”Fže -="hack/bož”¶©e.go.txt" -·ths -="./..." - -50 -fmt -: ## -Run - -go - fmˆ -agaš¡ - -code -. - -51 -go - -fmt - ./... - -53 -v‘ -: ## -Run - -go - v‘ -agaš¡ - -code -. - -54 -go - -v‘ - ./... - -56 -TEST - ?ð./ -­is -/ -d©aba£ -/ -v1®pha1 - ./ -commÚs -/... ./ -cÚŒÞËrs -/... - -57 -‹¡ -: -mªiã¡s - -g’”©e - -fmt - -v‘ - -’v‹¡ - ## -Run - -un™ - -‹¡s -. - -58 -KUBEBUILDER_ASSETS -="$(sh–È$(ENVTESTèu£ $(ENVTEST_K8S_VERSIONè-°·th)" -go - -‹¡ - - `$ -( -TEST -è- -cov”´ofže - -cov” -. -out - - -60 -E2ETEST - ?ð./ -‹¡ -/ -e2e -/ - -61 -e2e -: -mªiã¡s - -g’”©e - -fmt - -v‘ - -’v‹¡ - ## -Run -ƒ2 -‹¡s -. - -62 -KUBEBUILDER_ASSETS -="$(sh–È$(ENVTESTèu£ $(ENVTEST_K8S_VERSIONè-°·th)" -gškgo - - -v - -- -timeout -=2 -h30m - -- -çž -- -ç¡ - - $$ -( -E2ETEST -) - -64 ##@ -Bužd - - -66 -bužd -: -g’”©e - -fmt - -v‘ - ## -Bužd - -mªag” - -bš¬y -. - -67 -go - -bužd - - -o - -bš -/ -mªag” - -maš -.go - -69 -run -: -mªiã¡s - -g’”©e - -fmt - -v‘ - ## -Run - -a - -cÚŒÞËr - -äom - -your - -ho¡ -. - -70 -go - -run - ./ -maš -.go - -72 -dock” -- -bužd -: -mªiã¡s - -g’”©e - -fmt - -v‘ - #‹¡ ## -Bužd - dock” -image - -w™h - -the - -mªag” -. -Di§bË -h -‹¡ - -but - -k“p -h -v®id©iÚs - -to - -çž - -ç¡ - - -73 -dock” - -bužd - -- -no -- -ÿche -= -Œue - --bužd- -¬g - -h‰p_´oxy -= -$ -{ -HTTP_PROXY -} --bužd-¬g h‰ps_´oxy=${HTTPS_PROXY} . -ˆ${IMG} - -75 #dock”- -bužd -- -´oxy -: -‹¡ - - -76 #dock” -bužd - --bužd- -¬g - -h‰p_´oxy -= -$ -{h‰p_´oxy} --bužd-¬g -h‰ps_´oxy -=${h‰ps_´oxy} bužd . - -t - ${ -IMG -} - -78 -dock” -- -push -: ## -Push - dock” -image - -w™h - -the - -mªag” -. - -79 -dock” - -push - -$ -{ -IMG -} - -81 ##@ -D•loym’t - - -83 -š¡®l -: -mªiã¡s - -ku¡omize - ## -In¡®l - -CRDs - -što - -the - -K8s - -þu¡” - -¥ecif›d - -š - ~/. -kube -/ -cÚfig -. - -84 - $$ -( -KUSTOMIZE -è -bužd - -cÚfig -/ -üd - | -kubeùl - -­¶y - - -f - - - -86 -unš¡®l -: -mªiã¡s - -ku¡omize - ## -Unš¡®l - -CRDs - -äom - -the - -K8s - -þu¡” - -¥ecif›d - -š - ~/. -kube -/ -cÚfig -. - -87 - $$ -( -KUSTOMIZE -è -bužd - -cÚfig -/ -üd - | -kubeùl - -d–‘e - - -f - - - -89 -d•loy -: -mªiã¡s - -ku¡omize - ## -D•loy - -cÚŒÞËr - -to - -the - -K8s - -þu¡” - -¥ecif›d - -š - ~/. -kube -/ -cÚfig -. - -90 -cd - -cÚfig -/ -mªag” - && - $$ -( -KUSTOMIZE -è -ed™ - -£t - -image - -cÚŒÞËr -= -$ -{ -IMG -} - -91 - $$ -( -KUSTOMIZE -è -bužd - -cÚfig -/ | -kubeùl - -­¶y - - -f - - - -94 #U£d -£d - -to - -»pos™iÚ - -the - -cÚŒÞËr -- -mªag” - -D•loym’t - -aá” -h -û¹ifiÿ‹ - -ü—tiÚ - -š -h -OPERATOR_YAML - - -95 -Ý”©Ü -- -yaml -: -mªiã¡s - -ku¡omize - - -96 -cd - -cÚfig -/ -mªag” - && - $$ -( -KUSTOMIZE -è -ed™ - -£t - -image - -cÚŒÞËr -= -$ -{ -IMG -} - -97 - $$ -( -KUSTOMIZE -è -bužd - -cÚfig -/ > "${OPERATOR_YAML}" - -98 -£d - - -i -. -bak - - -e - '/^apiVersion:‡pps\/v1/,/---/d' "${OPERATOR_YAML}" - -99 ( -echo - --- && -£d - '/^apiVersion:‡pps\/v1/,/---/!d' "${OPERATOR_YAML}.bak") >> "${OPERATOR_YAML}" - -100 -rm - "${OPERATOR_YAML}.bak" - -102 -und•loy -: ## -Und•loy - -cÚŒÞËr - -äom - -the - -K8s - -þu¡” - -¥ecif›d - -š - ~/. -kube -/ -cÚfig -. - -103 - $$ -( -KUSTOMIZE -è -bužd - -cÚfig -/ | -kubeùl - -d–‘e - - -f - - - -105 ##@ -Bužd - -D•’d’c›s - - -107 ## -LoÿtiÚ - -to - -š¡®l - -d•’d’c›s -o - -108 -LOCALBIN - ?ð - `$ -( -sh–l - -pwd -)/ -bš - - -109 - $$ -( -LOCALBIN -): - -110 -mkdœ - - -p - - $$ -( -LOCALBIN -) - -112 ## -ToÞ - -Bš¬›s - - -113 -KUSTOMIZE - ?ð - `$ -( -LOCALBIN -)/ -ku¡omize - - -114 -CONTROLLER_GEN - ?ð - `$ -( -LOCALBIN -)/ -cÚŒÞËr -- -g’ - - -115 -ENVTEST - ?ð - `$ -( -LOCALBIN -)/ -£tup -- -’v‹¡ - - -117 ## -ToÞ - -V”siÚs - - -118 -KUSTOMIZE_VERSION - ?ð -v3 -.8.7 - -119 -CONTROLLER_TOOLS_VERSION - ?ð -v0 -.6.1 - -121 -KUSTOMIZE_INSTALL_SCRIPT - ?= "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh" - -122 . -PHONY -: -ku¡omize - - -123 -ku¡omize -: - $$ -( -KUSTOMIZE -è## -DowÆßd - -ku¡omize - -loÿÎy -  -Ãûs§ry -. - -124 - $$ -( -KUSTOMIZE -): - $$ -( -LOCALBIN -) - -125 -cu¾ - - -s - - `$ -( -KUSTOMIZE_INSTALL_SCRIPT -è| -bash - - -- $( -sub¡ - -v -,,$( -KUSTOMIZE_VERSION -)è$( -LOCALBIN -) - -127 . -PHONY -: -cÚŒÞËr -- -g’ - - -128 -cÚŒÞËr -- -g’ -: - $$ -( -CONTROLLER_GEN -è## -DowÆßd - -cÚŒÞËr -- -g’ - -loÿÎy -  -Ãûs§ry -. - -129 - $$ -( -CONTROLLER_GEN -): - $$ -( -LOCALBIN -) - -130 -GOBIN -= - $$ -( -LOCALBIN -è -go - -š¡®l - -sigs -. -k8s -. -io -/ -cÚŒÞËr -- -toÞs -/ -cmd -/cÚŒÞËr- -g’ -@ - `$ -( -CONTROLLER_TOOLS_VERSION -) - -132 . -PHONY -: -’v‹¡ - - -133 -’v‹¡ -: - $$ -( -ENVTEST -è## -DowÆßd - -’v‹¡ -- -£tup - -loÿÎy -  -Ãûs§ry -. - -134 - $$ -( -ENVTEST -): - $$ -( -LOCALBIN -) - -135 -GOBIN -= - $$ -( -LOCALBIN -è -go - -š¡®l - -sigs -. -k8s -. -io -/ -cÚŒÞËr -- -ruÁime -/ -toÞs -/ -£tup -- -’v‹¡ -@ -Ï‹¡ - - -138 . -PHONY -: -bundË - - -139 -bundË -: -mªiã¡s - -ku¡omize - ## -G’”©e - bundË mªiã¡  -ªd - -m‘ad©a -, -th’ - -v®id©e - -g’”©ed - -fžes -. - -140 -Ý”©Ü -- -sdk - -g’”©e - -ku¡omize - -mªiã¡s - - -q - - -141 -cd - -cÚfig -/ -mªag” - && - $$ -( -KUSTOMIZE -è -ed™ - -£t - -image - -cÚŒÞËr -= - `$ -( -IMG -) - -142 - $$ -( -KUSTOMIZE -è -bužd - -cÚfig -/ -mªiã¡s - | -Ý”©Ü -- -sdk - -g’”©e - -bundË - - -q - -- -ov”wr™e - -- -v”siÚ - - `$ -( -VERSION -è - $$ -( -BUNDLE_METADATA_OPTS -) - -143 -Ý”©Ü -- -sdk - -bundË - -v®id©e - ./bundle - -145 . -PHONY -: -bundË -- -bužd - - -146 -bundË -- -bužd -: ## -Bužd - -the - bundË -image -. - -147 -dock” - -bužd - - -f - -bundË -. -Dock”fže - - -t - - `$ -( -BUNDLE_IMG -) . - -149 . -PHONY -: -bundË -- -push - - -150 -bundË -- -push -: ## -Push - -the - bundË -image -. - -151 - $$ -( -MAKE -è -dock” -- -push - -IMG -= - `$ -( -BUNDLE_IMG -) - -153 . -PHONY -: -Ým - - -154 -OPM - = ./ -bš -/ -Ým - - -155 -Ým -: ## -DowÆßd - opm -loÿÎy -  -Ãûs§ry -. - -156 - `iãq - (, - `$ -( -wždÿrd - - $$ -( -OPM -))) - -157 - `iãq - (, - `$ -( -sh–l - -which - -Ým - 2>/ -dev -/ -nuÎ -)) - -159 -£t - - -e - ;\ - -160 -mkdœ - - -p - - `$ -( -dœ - $( -OPM -)) ;\ - -161 -OS -= - `$ -( -sh–l - -go - -’v - -GOOS -è&& -ARCH -=$(sh–ÈgØ’v -GOARCH -) && \ - -162 -cu¾ - - -sSLo - - `$ -( -OPM -è -h‰ps -: - -163 -chmod - + -x - - `$ -( -OPM -) ;\ - -164 - } -} - -166 - gOPM - = - $$ -( -sh–l - -which - -Ým -) - -167 -’dif - - -168 -’dif - - -170 #A -comma -- -£·¿‹d - -li¡ - -of - -bundË - - `images - ( -e -. -g -. -make - -ÿlog -- -bužd - -BUNDLE_IMGS -= -exam¶e -. -com -/ -Ý”©Ü --bundË: -v0 -.1.0,example.com/operator-bundle:v0.2.0). - -171 #The£ -images - -MUST - -exi¡ - -š - -a - -»gi¡ry - -ªd - -be - -puÎ -- -abË -. - -172 -BUNDLE_IMGS - ?ð - $$ -( -BUNDLE_IMG -) - -174 #Th -image - -g - -giv’ - -to - -the - -»suÉšg - -ÿlog - - `image - ( -e -. -g -. -make - c©®og- -bužd - -CATALOG_IMG -= -exam¶e -. -com -/ -Ý”©Ü --ÿlog: -v0 -.2.0). - -175 -CATALOG_IMG - ?ð - `$ -( -IMAGE_TAG_BASE -)- -ÿlog -: - $v$ -( -VERSION -) - -177 #S‘ -CATALOG_BASE_IMG - -to - -ª - -exi¡šg - -ÿlog - -image - -g -Ø -add - -$BUNDLE_IMGS -Ø -th© - image. - -178 - `iâeq - ( - `$ -( -Üigš - -CATALOG_BASE_IMG -), -undefšed -) - -179 -FROM_INDEX_OPT - :ð-- -äom -- -šdex - - $$ -( -CATALOG_BASE_IMG -) - -180 -’dif - - -182 #Bužd -a - -ÿlog - -image - -by - -addšg - -bundË - -images - -to - -ª - -em±y - c©®og -usšg - -the - -Ý”©Ü - -·ckage - -mªag” - -toÞ -, 'opm'. - -183 #Thi  -»ce - -švokes - 'Ým' -š - '£mv”' -bundË - -add - -mode -. -FÜ - -mÜe - -šfÜm©iÚ - -Ú -‡dd -modes -, -£e -: - -185 . -PHONY -: -ÿlog -- -bužd - - -186 -ÿlog -- -bužd -: -Ým - ## -Bužd - -a - c©®og -image -. - -187 - $$ -( -OPM -è -šdex - -add - -- -cÚš” -- -toÞ - -dock” - -- -mode - -£mv” - -- -g - - `$ -( -CATALOG_IMG -è-- -bundËs - $( -BUNDLE_IMGS -è - $$ -( -FROM_INDEX_OPT -) - -189 #Push -the - -ÿlog - -image -. - -190 . -PHONY -: -ÿlog -- -push - - -191 -ÿlog -- -push -: ## -Push - -a - c©®og -image -. - -192 - $$ -( -MAKE -è -dock” -- -push - -IMG -= - `$ -( -CATALOG_IMG -) - - @ -1 -. -1 -/usr/include -1 -9 -Makefile diff --git a/cscope.tmplst b/cscope.tmplst deleted file mode 100644 index 0937d485..00000000 --- a/cscope.tmplst +++ /dev/null @@ -1 +0,0 @@ -./Makefile diff --git a/docs/multitenant/provisioning/add_replica.yaml b/docs/multitenant/provisioning/add_replica.yaml index 2bc54495..fac2d7ba 100644 --- a/docs/multitenant/provisioning/add_replica.yaml +++ b/docs/multitenant/provisioning/add_replica.yaml @@ -9,7 +9,6 @@ metadata: namespace: oracle-database-operator-system spec: cdbName: "goldcdb" - scanName: "goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com" dbServer: "goldhost1.lbsub52b3b1cae.okecluster.oraclevcn.com" ordsImage: phx.ocir.io//oracle/ords:21.4.3 dbPort: 1521 diff --git a/docs/multitenant/provisioning/cdb.yaml b/docs/multitenant/provisioning/cdb.yaml index 4282040e..8e25a763 100644 --- a/docs/multitenant/provisioning/cdb.yaml +++ b/docs/multitenant/provisioning/cdb.yaml @@ -9,7 +9,6 @@ metadata: namespace: oracle-database-operator-system spec: cdbName: "goldcdb" - scanName: "goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com" dbServer: "goldhost1.lbsub52b3b1cae.okecluster.oraclevcn.com" ordsImage: phx.ocir.io//oracle/ords:21.4.3 ordsImagePullSecret: "container-registry-secret" diff --git a/oracle-database-operator.yaml b/oracle-database-operator.yaml index 9180474e..95786f0b 100644 --- a/oracle-database-operator.yaml +++ b/oracle-database-operator.yaml @@ -619,10 +619,6 @@ spec: jsonPath: .spec.dbPort name: DB Port type: integer - - description: SCAN Name - jsonPath: .spec.scanName - name: SCAN Name - type: string - description: Replicas jsonPath: .spec.replicas name: Replicas @@ -651,8 +647,6 @@ spec: spec: description: CDBSpec defines the desired state of CDB properties: - TestVariable: - type: string cdbAdminPwd: description: Password for the CDB Administrator to manage PDB lifecycle properties: @@ -768,9 +762,6 @@ spec: replicas: description: Number of ORDS Containers to create type: integer - scanName: - description: SCAN Name - type: string serviceName: description: Name of the CDB Service type: string @@ -2031,10 +2022,17 @@ spec: - jsonPath: .status.connectString name: Connect Str type: string + - jsonPath: .status.tcpsConnectString + name: TCPS Connect Str + type: string - jsonPath: .status.pdbConnectString name: Pdb Connect Str priority: 1 type: string + - jsonPath: .status.tcpsPdbConnectString + name: TCPS Pdb Connect Str + priority: 1 + type: string - jsonPath: .status.oemExpressUrl name: Oem Express Url type: string @@ -2079,6 +2077,8 @@ spec: - enterprise - express type: string + enableTCPS: + type: boolean flashBack: type: boolean forceLog: @@ -2109,6 +2109,8 @@ spec: sgaTarget: type: integer type: object + listenerPort: + type: integer loadBalancer: type: boolean nodeSelector: @@ -2149,6 +2151,10 @@ spec: maxLength: 12 pattern: ^[a-zA-Z0-9]+$ type: string + tcpsCertRenewInterval: + type: string + tcpsListenerPort: + type: integer required: - image type: object @@ -2159,8 +2165,14 @@ spec: type: boolean archiveLog: type: string + certCreationTimestamp: + type: string + certRenewInterval: + type: string charset: type: string + clientWalletLoc: + type: string cloneFrom: type: string clusterConnectString: @@ -2241,6 +2253,9 @@ spec: type: integer initSgaSize: type: integer + isTcpsEnabled: + default: false + type: boolean nodes: items: type: string @@ -2286,7 +2301,12 @@ spec: type: object status: type: string + tcpsConnectString: + type: string + tcpsPdbConnectString: + type: string required: + - isTcpsEnabled - persistence type: object type: object @@ -3198,7 +3218,7 @@ spec: - --enable-leader-election command: - /manager - image: container-registry.oracle.com/database/operator:0.2.0 + image: lin.ocir.io/intsanjaysingh/mmalvezz/testppr/myoperator:latest imagePullPolicy: Always name: manager ports: From b4365d002eeef2244f6dbab9ca9421a5014ce675 Mon Sep 17 00:00:00 2001 From: mmalvezz Date: Wed, 19 Oct 2022 17:50:48 +0200 Subject: [PATCH 508/628] bug 34699326 + usecase01 --- apis/database/v1alpha1/cdb_types.go | 2 + apis/database/v1alpha1/cdb_webhook.go | 16 +- controllers/DBserver | 33 -- controllers/database/cdb_controller.go | 4 + controllers/database/pdb_controller.go | 36 +- docs/multitenant/usecase01/BuildImage.log | 487 +++++++++++++++ docs/multitenant/usecase01/ImagePush.log | 11 + docs/multitenant/usecase01/README.md | 555 ++++++++++++++++++ docs/multitenant/usecase01/cdb.log | 372 ++++++++++++ docs/multitenant/usecase01/cdb.yaml | 46 ++ .../usecase01/openssl_execution.log | 22 + docs/multitenant/usecase01/ordsconfig.log | 35 ++ docs/multitenant/usecase01/sync.sh | 5 + docs/multitenant/usecase01/testapi.log | 49 ++ oracle-database-operator.yaml | 6 + ords/runOrdsSSL.sh | 32 +- 16 files changed, 1662 insertions(+), 49 deletions(-) delete mode 100644 controllers/DBserver create mode 100644 docs/multitenant/usecase01/BuildImage.log create mode 100644 docs/multitenant/usecase01/ImagePush.log create mode 100644 docs/multitenant/usecase01/README.md create mode 100644 docs/multitenant/usecase01/cdb.log create mode 100644 docs/multitenant/usecase01/cdb.yaml create mode 100644 docs/multitenant/usecase01/openssl_execution.log create mode 100644 docs/multitenant/usecase01/ordsconfig.log create mode 100644 docs/multitenant/usecase01/sync.sh create mode 100644 docs/multitenant/usecase01/testapi.log diff --git a/apis/database/v1alpha1/cdb_types.go b/apis/database/v1alpha1/cdb_types.go index a987718a..b5f39707 100644 --- a/apis/database/v1alpha1/cdb_types.go +++ b/apis/database/v1alpha1/cdb_types.go @@ -86,6 +86,7 @@ type CDBSpec struct { DBPort int `json:"dbPort,omitempty"` // Node Selector for running the Pod NodeSelector map[string]string `json:"nodeSelector,omitempty"` + DBTnsurl string `json:"dbTnsurl,omitempty"` } // CDBSecret defines the secretName @@ -150,6 +151,7 @@ type CDBStatus struct { // +kubebuilder:printcolumn:JSONPath=".spec.cdbName",name="CDB Name",type="string",description="Name of the CDB" // +kubebuilder:printcolumn:JSONPath=".spec.dbServer",name="DB Server",type="string",description=" Name of the DB Server" // +kubebuilder:printcolumn:JSONPath=".spec.dbPort",name="DB Port",type="integer",description="DB server port" +// +kubebuilder:printcolumn:JSONPath=".spec.dbTnsurl",name="TNS STRING",type="string",description=" string of the tnsalias" // +kubebuilder:printcolumn:JSONPath=".spec.replicas",name="Replicas",type="integer",description="Replicas" // +kubebuilder:printcolumn:JSONPath=".status.phase",name="Status",type="string",description="Status of the CDB Resource" // +kubebuilder:printcolumn:JSONPath=".status.msg",name="Message",type="string",description="Error message, if any" diff --git a/apis/database/v1alpha1/cdb_webhook.go b/apis/database/v1alpha1/cdb_webhook.go index c760875c..25975092 100644 --- a/apis/database/v1alpha1/cdb_webhook.go +++ b/apis/database/v1alpha1/cdb_webhook.go @@ -88,7 +88,7 @@ func (r *CDB) ValidateCreate() error { var allErrs field.ErrorList - if r.Spec.ServiceName == "" { + if r.Spec.ServiceName == "" && r.Spec.DBServer != "" { allErrs = append(allErrs, field.Required(field.NewPath("spec").Child("serviceName"), "Please specify CDB Service name")) } @@ -108,15 +108,21 @@ func (r *CDB) ValidateCreate() error { field.Required(field.NewPath("spec").Child("scanName"), "Please specify SCAN Name for CDB")) }*/ - if r.Spec.DBServer == "" { + if ((r.Spec.DBServer == "" && r.Spec.DBTnsurl == "") || (r.Spec.DBServer != "" && r.Spec.DBTnsurl != "")) { allErrs = append(allErrs, - field.Required(field.NewPath("spec").Child("dbServer"), "Please specify Database Server Name or IP Address")) + field.Required(field.NewPath("spec").Child("dbServer"), "Please specify Database Server Name/IP Address or tnsalias string")) } - if r.Spec.DBPort == 0 { + + if r.Spec.DBTnsurl != "" && ( r.Spec.DBServer != "" || r.Spec.DBPort != 0 || r.Spec.ServiceName != "" ) { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("dbServer"), "DBtnsurl is orthogonal to (DBServer,DBport,Services)")) + } + + if r.Spec.DBPort == 0 && r.Spec.DBServer != "" { allErrs = append(allErrs, field.Required(field.NewPath("spec").Child("dbPort"), "Please specify DB Server Port")) } - if r.Spec.DBPort < 0 { + if r.Spec.DBPort < 0 && r.Spec.DBServer != "" { allErrs = append(allErrs, field.Required(field.NewPath("spec").Child("dbPort"), "Please specify a valid DB Server Port")) } diff --git a/controllers/DBserver b/controllers/DBserver deleted file mode 100644 index 57c0f4fb..00000000 --- a/controllers/DBserver +++ /dev/null @@ -1,33 +0,0 @@ -# This viminfo file was generated by Vim 7.4. -# You may edit it if you're careful! - -# Value of 'encoding' when this file was written -*encoding=utf-8 - - -# hlsearch on (H) or off (h): -~H -# Command Line History (newest to oldest): -:q! -:q - -# Search String History (newest to oldest): - -# Expression History (newest to oldest): - -# Input Line History (newest to oldest): - -# Input Line History (newest to oldest): - -# Registers: - -# File marks: -'0 1 0 ~/WORKDIR/Ords22_operator/oracle-database-operator/controllers/grep - -# Jumplist (newest first): --' 1 0 ~/WORKDIR/Ords22_operator/oracle-database-operator/controllers/grep - -# History of marks within files (newest to oldest): - -> ~/WORKDIR/Ords22_operator/oracle-database-operator/controllers/grep - " 1 0 diff --git a/controllers/database/cdb_controller.go b/controllers/database/cdb_controller.go index 02c04142..58eebf51 100644 --- a/controllers/database/cdb_controller.go +++ b/controllers/database/cdb_controller.go @@ -449,6 +449,10 @@ func (r *CDBReconciler) createPodSpec(cdb *dbapi.CDB) corev1.PodSpec { Name: "ORACLE_HOST", Value: cdb.Spec.DBServer, }, + { + Name: "DBTNSURL", + Value: cdb.Spec.DBTnsurl, + }, { Name: "TLSCRT", Value: cdb.Spec.CDBTlsCrt.Secret.Key, diff --git a/controllers/database/pdb_controller.go b/controllers/database/pdb_controller.go index 07580725..f9231a66 100644 --- a/controllers/database/pdb_controller.go +++ b/controllers/database/pdb_controller.go @@ -641,7 +641,12 @@ func (r *PDBReconciler) createPDB(ctx context.Context, req ctrl.Request, pdb *db r.Recorder.Eventf(pdb, corev1.EventTypeNormal, "Created", "PDB '%s' created successfully", pdb.Spec.PDBName) + if cdb.Spec.DBServer != "" { pdb.Status.ConnString = cdb.Spec.DBServer + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName + } else { + pdb.Status.ConnString = cdb.Spec.DBTnsurl + } + log.Info("Created PDB Resource", "PDB Name", pdb.Spec.PDBName) r.getPDBState(ctx, req, pdb) return nil @@ -695,7 +700,13 @@ func (r *PDBReconciler) clonePDB(ctx context.Context, req ctrl.Request, pdb *dba r.Recorder.Eventf(pdb, corev1.EventTypeNormal, "Created", "PDB '%s' cloned successfully", pdb.Spec.PDBName) - pdb.Status.ConnString = cdb.Spec.DBServer + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName + if cdb.Spec.DBServer != "" { + pdb.Status.ConnString = cdb.Spec.DBServer + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName + } else { + pdb.Status.ConnString = cdb.Spec.DBTnsurl + } + + log.Info("Cloned PDB successfully", "Source PDB Name", pdb.Spec.SrcPDBName, "Clone PDB Name", pdb.Spec.PDBName) r.getPDBState(ctx, req, pdb) return nil @@ -763,7 +774,12 @@ func (r *PDBReconciler) plugPDB(ctx context.Context, req ctrl.Request, pdb *dbap r.Recorder.Eventf(pdb, corev1.EventTypeNormal, "Created", "PDB '%s' plugged successfully", pdb.Spec.PDBName) - pdb.Status.ConnString = cdb.Spec.DBServer + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName + if cdb.Spec.DBServer != "" { + pdb.Status.ConnString = cdb.Spec.DBServer + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName + } else { + pdb.Status.ConnString = cdb.Spec.DBTnsurl + } + log.Info("Successfully plugged PDB", "PDB Name", pdb.Spec.PDBName) r.getPDBState(ctx, req, pdb) return nil @@ -883,7 +899,13 @@ func (r *PDBReconciler) modifyPDB(ctx context.Context, req ctrl.Request, pdb *db } r.Recorder.Eventf(pdb, corev1.EventTypeNormal, "Modified", "PDB '%s' modified successfully", pdb.Spec.PDBName) - pdb.Status.ConnString = cdb.Spec.DBServer + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName + + if cdb.Spec.DBServer != "" { + pdb.Status.ConnString = cdb.Spec.DBServer + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName + } else { + pdb.Status.ConnString = cdb.Spec.DBTnsurl + } + log.Info("Successfully modified PDB state", "PDB Name", pdb.Spec.PDBName) r.getPDBState(ctx, req, pdb) @@ -970,7 +992,13 @@ func (r *PDBReconciler) mapPDB(ctx context.Context, req ctrl.Request, pdb *dbapi pdb.Status.OpenMode = objmap["open_mode"].(string) pdb.Status.TotalSize = fmt.Sprintf("%.2f", totSizeInGB) + "G" - pdb.Status.ConnString = cdb.Spec.DBServer + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName + + if cdb.Spec.DBServer != "" { + pdb.Status.ConnString = cdb.Spec.DBServer + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName + } else { + pdb.Status.ConnString = cdb.Spec.DBTnsurl + } + log.Info("Successfully mapped PDB to Kubernetes resource", "PDB Name", pdb.Spec.PDBName) return nil diff --git a/docs/multitenant/usecase01/BuildImage.log b/docs/multitenant/usecase01/BuildImage.log new file mode 100644 index 00000000..4ee2fa05 --- /dev/null +++ b/docs/multitenant/usecase01/BuildImage.log @@ -0,0 +1,487 @@ +/usr/bin/docker build -t oracle/ords-dboper:latest . +Sending build context to Docker daemon 92.38MB +Step 1/10 : FROM container-registry.oracle.com/java/jdk:latest +Trying to pull repository container-registry.oracle.com/java/jdk ... +latest: Pulling from container-registry.oracle.com/java/jdk +7cb069903b8a: Pull complete +a98ca67f4239: Pull complete +1b4060d1d804: Pull complete +Digest: sha256:8e7161bbd6a3a3beb77ee6f2d80c17ae4c80d88e0f5af667a19a0271c33f1b5e +Status: Downloaded newer image for container-registry.oracle.com/java/jdk:latest + ---> ad9ff1bbe92a +Step 2/10 : ENV ORDS_HOME=/opt/oracle/ords/ RUN_FILE="runOrdsSSL.sh" + ---> Running in e6f76deab66e +Removing intermediate container e6f76deab66e + ---> 0b26c489e4fd +Step 3/10 : COPY $RUN_FILE $ORDS_HOME + ---> ee472155adab +Step 4/10 : RUN yum -y install yum-utils bind-utils tree hostname openssl net-tools zip unzip tar wget vim-minimal which sudo expect procps && yum-config-manager --add-repo=http://yum.oracle.com/repo/OracleLinux/OL8/oracle/software/x86_64 && yum -y install java-11-openjdk-devel && yum -y install ords && yum -y install iproute && yum clean all + ---> Running in d38a69d2cc70 +Oracle Linux 8 BaseOS Latest (x86_64) 105 MB/s | 50 MB 00:00 +Oracle Linux 8 Application Stream (x86_64) 90 MB/s | 38 MB 00:00 +Last metadata expiration check: 0:00:07 ago on Mon 10 Oct 2022 04:06:15 PM UTC. +Package yum-utils-4.0.21-11.0.1.el8.noarch is already installed. +Package tar-2:1.30-5.el8.x86_64 is already installed. +Package vim-minimal-2:8.0.1763-19.0.1.el8_6.4.x86_64 is already installed. +Package procps-ng-3.3.15-6.0.1.el8.x86_64 is already installed. +Dependencies resolved. +================================================================================ + Package Arch Version Repository Size +================================================================================ +Installing: + bind-utils x86_64 32:9.11.36-3.el8_6.1 ol8_appstream 452 k + expect x86_64 5.45.4-5.el8 ol8_baseos_latest 266 k + hostname x86_64 3.20-6.el8 ol8_baseos_latest 32 k + net-tools x86_64 2.0-0.52.20160912git.el8 ol8_baseos_latest 322 k + openssl x86_64 1:1.1.1k-7.el8_6 ol8_baseos_latest 709 k + sudo x86_64 1.8.29-8.el8 ol8_baseos_latest 925 k + tree x86_64 1.7.0-15.el8 ol8_baseos_latest 59 k + unzip x86_64 6.0-46.0.1.el8 ol8_baseos_latest 196 k + wget x86_64 1.19.5-10.0.1.el8 ol8_appstream 734 k + which x86_64 2.21-17.el8 ol8_baseos_latest 49 k + zip x86_64 3.0-23.el8 ol8_baseos_latest 270 k +Installing dependencies: + bind-libs x86_64 32:9.11.36-3.el8_6.1 ol8_appstream 175 k + bind-libs-lite x86_64 32:9.11.36-3.el8_6.1 ol8_appstream 1.2 M + bind-license noarch 32:9.11.36-3.el8_6.1 ol8_appstream 103 k + fstrm x86_64 0.6.1-2.el8 ol8_appstream 29 k + libmaxminddb x86_64 1.2.0-10.el8 ol8_appstream 33 k + libmetalink x86_64 0.1.3-7.el8 ol8_baseos_latest 32 k + protobuf-c x86_64 1.3.0-6.el8 ol8_appstream 37 k + python3-bind noarch 32:9.11.36-3.el8_6.1 ol8_appstream 150 k + python3-ply noarch 3.9-9.el8 ol8_baseos_latest 111 k + tcl x86_64 1:8.6.8-2.el8 ol8_baseos_latest 1.1 M + +Transaction Summary +================================================================================ +Install 21 Packages + +Total download size: 6.9 M +Installed size: 20 M +Downloading Packages: +(1/21): hostname-3.20-6.el8.x86_64.rpm 555 kB/s | 32 kB 00:00 +(2/21): libmetalink-0.1.3-7.el8.x86_64.rpm 492 kB/s | 32 kB 00:00 +(3/21): expect-5.45.4-5.el8.x86_64.rpm 3.2 MB/s | 266 kB 00:00 +(4/21): python3-ply-3.9-9.el8.noarch.rpm 5.5 MB/s | 111 kB 00:00 +(5/21): net-tools-2.0-0.52.20160912git.el8.x86_ 6.7 MB/s | 322 kB 00:00 +(6/21): openssl-1.1.1k-7.el8_6.x86_64.rpm 12 MB/s | 709 kB 00:00 +(7/21): tree-1.7.0-15.el8.x86_64.rpm 4.1 MB/s | 59 kB 00:00 +(8/21): sudo-1.8.29-8.el8.x86_64.rpm 19 MB/s | 925 kB 00:00 +(9/21): which-2.21-17.el8.x86_64.rpm 2.5 MB/s | 49 kB 00:00 +(10/21): unzip-6.0-46.0.1.el8.x86_64.rpm 5.9 MB/s | 196 kB 00:00 +(11/21): tcl-8.6.8-2.el8.x86_64.rpm 15 MB/s | 1.1 MB 00:00 +(12/21): zip-3.0-23.el8.x86_64.rpm 15 MB/s | 270 kB 00:00 +(13/21): bind-libs-9.11.36-3.el8_6.1.x86_64.rpm 7.9 MB/s | 175 kB 00:00 +(14/21): bind-license-9.11.36-3.el8_6.1.noarch. 4.9 MB/s | 103 kB 00:00 +(15/21): bind-utils-9.11.36-3.el8_6.1.x86_64.rp 21 MB/s | 452 kB 00:00 +(16/21): bind-libs-lite-9.11.36-3.el8_6.1.x86_6 28 MB/s | 1.2 MB 00:00 +(17/21): libmaxminddb-1.2.0-10.el8.x86_64.rpm 1.8 MB/s | 33 kB 00:00 +(18/21): fstrm-0.6.1-2.el8.x86_64.rpm 1.0 MB/s | 29 kB 00:00 +(19/21): protobuf-c-1.3.0-6.el8.x86_64.rpm 1.4 MB/s | 37 kB 00:00 +(20/21): python3-bind-9.11.36-3.el8_6.1.noarch. 9.2 MB/s | 150 kB 00:00 +(21/21): wget-1.19.5-10.0.1.el8.x86_64.rpm 7.5 MB/s | 734 kB 00:00 +-------------------------------------------------------------------------------- +Total 20 MB/s | 6.9 MB 00:00 +Running transaction check +Transaction check succeeded. +Running transaction test +Transaction test succeeded. +Running transaction + Preparing : 1/1 + Installing : protobuf-c-1.3.0-6.el8.x86_64 1/21 + Installing : libmaxminddb-1.2.0-10.el8.x86_64 2/21 + Running scriptlet: libmaxminddb-1.2.0-10.el8.x86_64 2/21 + Installing : fstrm-0.6.1-2.el8.x86_64 3/21 + Installing : bind-license-32:9.11.36-3.el8_6.1.noarch 4/21 + Installing : bind-libs-lite-32:9.11.36-3.el8_6.1.x86_64 5/21 + Installing : bind-libs-32:9.11.36-3.el8_6.1.x86_64 6/21 + Installing : unzip-6.0-46.0.1.el8.x86_64 7/21 + Installing : tcl-1:8.6.8-2.el8.x86_64 8/21 + Running scriptlet: tcl-1:8.6.8-2.el8.x86_64 8/21 + Installing : python3-ply-3.9-9.el8.noarch 9/21 + Installing : python3-bind-32:9.11.36-3.el8_6.1.noarch 10/21 + Installing : libmetalink-0.1.3-7.el8.x86_64 11/21 + Installing : wget-1.19.5-10.0.1.el8.x86_64 12/21 + Running scriptlet: wget-1.19.5-10.0.1.el8.x86_64 12/21 + Installing : bind-utils-32:9.11.36-3.el8_6.1.x86_64 13/21 + Installing : expect-5.45.4-5.el8.x86_64 14/21 + Installing : zip-3.0-23.el8.x86_64 15/21 + Installing : which-2.21-17.el8.x86_64 16/21 + Installing : tree-1.7.0-15.el8.x86_64 17/21 + Installing : sudo-1.8.29-8.el8.x86_64 18/21 + Running scriptlet: sudo-1.8.29-8.el8.x86_64 18/21 + Installing : openssl-1:1.1.1k-7.el8_6.x86_64 19/21 + Installing : net-tools-2.0-0.52.20160912git.el8.x86_64 20/21 + Running scriptlet: net-tools-2.0-0.52.20160912git.el8.x86_64 20/21 + Installing : hostname-3.20-6.el8.x86_64 21/21 + Running scriptlet: hostname-3.20-6.el8.x86_64 21/21 + Verifying : expect-5.45.4-5.el8.x86_64 1/21 + Verifying : hostname-3.20-6.el8.x86_64 2/21 + Verifying : libmetalink-0.1.3-7.el8.x86_64 3/21 + Verifying : net-tools-2.0-0.52.20160912git.el8.x86_64 4/21 + Verifying : openssl-1:1.1.1k-7.el8_6.x86_64 5/21 + Verifying : python3-ply-3.9-9.el8.noarch 6/21 + Verifying : sudo-1.8.29-8.el8.x86_64 7/21 + Verifying : tcl-1:8.6.8-2.el8.x86_64 8/21 + Verifying : tree-1.7.0-15.el8.x86_64 9/21 + Verifying : unzip-6.0-46.0.1.el8.x86_64 10/21 + Verifying : which-2.21-17.el8.x86_64 11/21 + Verifying : zip-3.0-23.el8.x86_64 12/21 + Verifying : bind-libs-32:9.11.36-3.el8_6.1.x86_64 13/21 + Verifying : bind-libs-lite-32:9.11.36-3.el8_6.1.x86_64 14/21 + Verifying : bind-license-32:9.11.36-3.el8_6.1.noarch 15/21 + Verifying : bind-utils-32:9.11.36-3.el8_6.1.x86_64 16/21 + Verifying : fstrm-0.6.1-2.el8.x86_64 17/21 + Verifying : libmaxminddb-1.2.0-10.el8.x86_64 18/21 + Verifying : protobuf-c-1.3.0-6.el8.x86_64 19/21 + Verifying : python3-bind-32:9.11.36-3.el8_6.1.noarch 20/21 + Verifying : wget-1.19.5-10.0.1.el8.x86_64 21/21 + +Installed: + bind-libs-32:9.11.36-3.el8_6.1.x86_64 + bind-libs-lite-32:9.11.36-3.el8_6.1.x86_64 + bind-license-32:9.11.36-3.el8_6.1.noarch + bind-utils-32:9.11.36-3.el8_6.1.x86_64 + expect-5.45.4-5.el8.x86_64 + fstrm-0.6.1-2.el8.x86_64 + hostname-3.20-6.el8.x86_64 + libmaxminddb-1.2.0-10.el8.x86_64 + libmetalink-0.1.3-7.el8.x86_64 + net-tools-2.0-0.52.20160912git.el8.x86_64 + openssl-1:1.1.1k-7.el8_6.x86_64 + protobuf-c-1.3.0-6.el8.x86_64 + python3-bind-32:9.11.36-3.el8_6.1.noarch + python3-ply-3.9-9.el8.noarch + sudo-1.8.29-8.el8.x86_64 + tcl-1:8.6.8-2.el8.x86_64 + tree-1.7.0-15.el8.x86_64 + unzip-6.0-46.0.1.el8.x86_64 + wget-1.19.5-10.0.1.el8.x86_64 + which-2.21-17.el8.x86_64 + zip-3.0-23.el8.x86_64 + +Complete! +Adding repo from: http://yum.oracle.com/repo/OracleLinux/OL8/oracle/software/x86_64 +created by dnf config-manager from http://yum.o 194 kB/s | 49 kB 00:00 +Dependencies resolved. +============================================================================================= + Package Arch Version Repository Size +============================================================================================= +Installing: + java-11-openjdk-devel x86_64 1:11.0.16.1.1-1.el8_6 ol8_appstream 3.4 M +Installing dependencies: + alsa-lib x86_64 1.2.6.1-3.el8 ol8_appstream 491 k + avahi-libs x86_64 0.7-20.el8 ol8_baseos_latest 62 k + copy-jdk-configs noarch 4.0-2.el8 ol8_appstream 30 k + crypto-policies-scripts noarch 20211116-1.gitae470d6.el8 ol8_baseos_latest 83 k + cups-libs x86_64 1:2.2.6-45.el8_6.2 ol8_baseos_latest 434 k + giflib x86_64 5.1.4-3.el8 ol8_appstream 51 k + graphite2 x86_64 1.3.10-10.el8 ol8_appstream 122 k + harfbuzz x86_64 1.7.5-3.el8 ol8_appstream 295 k + java-11-openjdk x86_64 1:11.0.16.1.1-1.el8_6 ol8_appstream 272 k + java-11-openjdk-headless x86_64 1:11.0.16.1.1-1.el8_6 ol8_appstream 40 M + javapackages-filesystem noarch 5.3.0-1.module+el8+5136+7ff78f74 ol8_appstream 30 k + lcms2 x86_64 2.9-2.el8 ol8_appstream 164 k + libX11 x86_64 1.6.8-5.el8 ol8_appstream 611 k + libX11-common noarch 1.6.8-5.el8 ol8_appstream 158 k + libXau x86_64 1.0.9-3.el8 ol8_appstream 37 k + libXcomposite x86_64 0.4.4-14.el8 ol8_appstream 28 k + libXext x86_64 1.3.4-1.el8 ol8_appstream 45 k + libXi x86_64 1.7.10-1.el8 ol8_appstream 49 k + libXrender x86_64 0.9.10-7.el8 ol8_appstream 33 k + libXtst x86_64 1.2.3-7.el8 ol8_appstream 22 k + libfontenc x86_64 1.1.3-8.el8 ol8_appstream 37 k + libjpeg-turbo x86_64 1.5.3-12.el8 ol8_appstream 157 k + libpkgconf x86_64 1.4.2-1.el8 ol8_baseos_latest 35 k + libxcb x86_64 1.13.1-1.el8 ol8_appstream 231 k + lksctp-tools x86_64 1.0.18-3.el8 ol8_baseos_latest 100 k + lua x86_64 5.3.4-12.el8 ol8_appstream 192 k + nspr x86_64 4.34.0-3.el8_6 ol8_appstream 143 k + nss x86_64 3.79.0-10.el8_6 ol8_appstream 747 k + nss-softokn x86_64 3.79.0-10.el8_6 ol8_appstream 1.2 M + nss-softokn-freebl x86_64 3.79.0-10.el8_6 ol8_appstream 398 k + nss-sysinit x86_64 3.79.0-10.el8_6 ol8_appstream 74 k + nss-util x86_64 3.79.0-10.el8_6 ol8_appstream 138 k + pkgconf x86_64 1.4.2-1.el8 ol8_baseos_latest 38 k + pkgconf-m4 noarch 1.4.2-1.el8 ol8_baseos_latest 17 k + pkgconf-pkg-config x86_64 1.4.2-1.el8 ol8_baseos_latest 15 k + ttmkfdir x86_64 3.0.9-54.el8 ol8_appstream 62 k + tzdata-java noarch 2022d-1.el8 ol8_appstream 186 k + xorg-x11-font-utils x86_64 1:7.5-41.el8 ol8_appstream 104 k + xorg-x11-fonts-Type1 noarch 7.5-19.el8 ol8_appstream 522 k +Enabling module streams: + javapackages-runtime 201801 + +Transaction Summary +============================================================================================= +Install 40 Packages + +Total download size: 50 M +Installed size: 196 M +Downloading Packages: +(1/40): crypto-policies-scripts-20211116-1.gita 1.3 MB/s | 83 kB 00:00 +(2/40): avahi-libs-0.7-20.el8.x86_64.rpm 879 kB/s | 62 kB 00:00 +(3/40): libpkgconf-1.4.2-1.el8.x86_64.rpm 2.0 MB/s | 35 kB 00:00 +(4/40): cups-libs-2.2.6-45.el8_6.2.x86_64.rpm 4.5 MB/s | 434 kB 00:00 +(5/40): lksctp-tools-1.0.18-3.el8.x86_64.rpm 3.7 MB/s | 100 kB 00:00 +(6/40): pkgconf-1.4.2-1.el8.x86_64.rpm 2.2 MB/s | 38 kB 00:00 +(7/40): pkgconf-m4-1.4.2-1.el8.noarch.rpm 1.2 MB/s | 17 kB 00:00 +(8/40): pkgconf-pkg-config-1.4.2-1.el8.x86_64.r 929 kB/s | 15 kB 00:00 +(9/40): copy-jdk-configs-4.0-2.el8.noarch.rpm 2.2 MB/s | 30 kB 00:00 +(10/40): giflib-5.1.4-3.el8.x86_64.rpm 3.3 MB/s | 51 kB 00:00 +(11/40): graphite2-1.3.10-10.el8.x86_64.rpm 7.7 MB/s | 122 kB 00:00 +(12/40): alsa-lib-1.2.6.1-3.el8.x86_64.rpm 12 MB/s | 491 kB 00:00 +(13/40): java-11-openjdk-11.0.16.1.1-1.el8_6.x8 14 MB/s | 272 kB 00:00 +(14/40): harfbuzz-1.7.5-3.el8.x86_64.rpm 8.7 MB/s | 295 kB 00:00 +(15/40): javapackages-filesystem-5.3.0-1.module 2.0 MB/s | 30 kB 00:00 +(16/40): lcms2-2.9-2.el8.x86_64.rpm 6.7 MB/s | 164 kB 00:00 +(17/40): java-11-openjdk-devel-11.0.16.1.1-1.el 46 MB/s | 3.4 MB 00:00 +(18/40): libX11-common-1.6.8-5.el8.noarch.rpm 8.4 MB/s | 158 kB 00:00 +(19/40): libX11-1.6.8-5.el8.x86_64.rpm 17 MB/s | 611 kB 00:00 +(20/40): libXau-1.0.9-3.el8.x86_64.rpm 2.6 MB/s | 37 kB 00:00 +(21/40): libXcomposite-0.4.4-14.el8.x86_64.rpm 2.0 MB/s | 28 kB 00:00 +(22/40): libXi-1.7.10-1.el8.x86_64.rpm 2.2 MB/s | 49 kB 00:00 +(23/40): libXext-1.3.4-1.el8.x86_64.rpm 1.6 MB/s | 45 kB 00:00 +(24/40): libXtst-1.2.3-7.el8.x86_64.rpm 1.1 MB/s | 22 kB 00:00 +(25/40): libXrender-0.9.10-7.el8.x86_64.rpm 1.3 MB/s | 33 kB 00:00 +(26/40): libfontenc-1.1.3-8.el8.x86_64.rpm 2.2 MB/s | 37 kB 00:00 +(27/40): libjpeg-turbo-1.5.3-12.el8.x86_64.rpm 8.6 MB/s | 157 kB 00:00 +(28/40): libxcb-1.13.1-1.el8.x86_64.rpm 13 MB/s | 231 kB 00:00 +(29/40): lua-5.3.4-12.el8.x86_64.rpm 11 MB/s | 192 kB 00:00 +(30/40): nspr-4.34.0-3.el8_6.x86_64.rpm 7.8 MB/s | 143 kB 00:00 +(31/40): nss-3.79.0-10.el8_6.x86_64.rpm 23 MB/s | 747 kB 00:00 +(32/40): nss-softokn-3.79.0-10.el8_6.x86_64.rpm 42 MB/s | 1.2 MB 00:00 +(33/40): nss-softokn-freebl-3.79.0-10.el8_6.x86 19 MB/s | 398 kB 00:00 +(34/40): nss-sysinit-3.79.0-10.el8_6.x86_64.rpm 5.3 MB/s | 74 kB 00:00 +(35/40): nss-util-3.79.0-10.el8_6.x86_64.rpm 8.7 MB/s | 138 kB 00:00 +(36/40): ttmkfdir-3.0.9-54.el8.x86_64.rpm 4.2 MB/s | 62 kB 00:00 +(37/40): tzdata-java-2022d-1.el8.noarch.rpm 11 MB/s | 186 kB 00:00 +(38/40): xorg-x11-font-utils-7.5-41.el8.x86_64. 6.7 MB/s | 104 kB 00:00 +(39/40): xorg-x11-fonts-Type1-7.5-19.el8.noarch 24 MB/s | 522 kB 00:00 +(40/40): java-11-openjdk-headless-11.0.16.1.1-1 77 MB/s | 40 MB 00:00 +-------------------------------------------------------------------------------- +Total 74 MB/s | 50 MB 00:00 +Running transaction check +Transaction check succeeded. +Running transaction test +Transaction test succeeded. +Running transaction + Running scriptlet: copy-jdk-configs-4.0-2.el8.noarch 1/1 + Running scriptlet: java-11-openjdk-headless-1:11.0.16.1.1-1.el8_6.x86_6 1/1 + Preparing : 1/1 + Installing : nspr-4.34.0-3.el8_6.x86_64 1/40 + Running scriptlet: nspr-4.34.0-3.el8_6.x86_64 1/40 + Installing : nss-util-3.79.0-10.el8_6.x86_64 2/40 + Installing : libjpeg-turbo-1.5.3-12.el8.x86_64 3/40 + Installing : nss-softokn-freebl-3.79.0-10.el8_6.x86_64 4/40 + Installing : nss-softokn-3.79.0-10.el8_6.x86_64 5/40 + Installing : tzdata-java-2022d-1.el8.noarch 6/40 + Installing : ttmkfdir-3.0.9-54.el8.x86_64 7/40 + Installing : lua-5.3.4-12.el8.x86_64 8/40 + Installing : copy-jdk-configs-4.0-2.el8.noarch 9/40 + Installing : libfontenc-1.1.3-8.el8.x86_64 10/40 + Installing : libXau-1.0.9-3.el8.x86_64 11/40 + Installing : libxcb-1.13.1-1.el8.x86_64 12/40 + Installing : libX11-common-1.6.8-5.el8.noarch 13/40 + Installing : libX11-1.6.8-5.el8.x86_64 14/40 + Installing : libXext-1.3.4-1.el8.x86_64 15/40 + Installing : libXi-1.7.10-1.el8.x86_64 16/40 + Installing : libXtst-1.2.3-7.el8.x86_64 17/40 + Installing : libXcomposite-0.4.4-14.el8.x86_64 18/40 + Installing : libXrender-0.9.10-7.el8.x86_64 19/40 + Installing : lcms2-2.9-2.el8.x86_64 20/40 + Running scriptlet: lcms2-2.9-2.el8.x86_64 20/40 + Installing : javapackages-filesystem-5.3.0-1.module+el8+5136+7f 21/40 + Installing : graphite2-1.3.10-10.el8.x86_64 22/40 + Installing : harfbuzz-1.7.5-3.el8.x86_64 23/40 + Running scriptlet: harfbuzz-1.7.5-3.el8.x86_64 23/40 + Installing : giflib-5.1.4-3.el8.x86_64 24/40 + Installing : alsa-lib-1.2.6.1-3.el8.x86_64 25/40 + Running scriptlet: alsa-lib-1.2.6.1-3.el8.x86_64 25/40 + Installing : pkgconf-m4-1.4.2-1.el8.noarch 26/40 + Installing : lksctp-tools-1.0.18-3.el8.x86_64 27/40 + Running scriptlet: lksctp-tools-1.0.18-3.el8.x86_64 27/40 + Installing : libpkgconf-1.4.2-1.el8.x86_64 28/40 + Installing : pkgconf-1.4.2-1.el8.x86_64 29/40 + Installing : pkgconf-pkg-config-1.4.2-1.el8.x86_64 30/40 + Installing : xorg-x11-font-utils-1:7.5-41.el8.x86_64 31/40 + Installing : xorg-x11-fonts-Type1-7.5-19.el8.noarch 32/40 + Running scriptlet: xorg-x11-fonts-Type1-7.5-19.el8.noarch 32/40 + Installing : crypto-policies-scripts-20211116-1.gitae470d6.el8. 33/40 + Installing : nss-sysinit-3.79.0-10.el8_6.x86_64 34/40 + Installing : nss-3.79.0-10.el8_6.x86_64 35/40 + Installing : avahi-libs-0.7-20.el8.x86_64 36/40 + Installing : cups-libs-1:2.2.6-45.el8_6.2.x86_64 37/40 + Installing : java-11-openjdk-headless-1:11.0.16.1.1-1.el8_6.x86 38/40 + Running scriptlet: java-11-openjdk-headless-1:11.0.16.1.1-1.el8_6.x86 38/40 + Installing : java-11-openjdk-1:11.0.16.1.1-1.el8_6.x86_64 39/40 + Running scriptlet: java-11-openjdk-1:11.0.16.1.1-1.el8_6.x86_64 39/40 + Installing : java-11-openjdk-devel-1:11.0.16.1.1-1.el8_6.x86_64 40/40 + Running scriptlet: java-11-openjdk-devel-1:11.0.16.1.1-1.el8_6.x86_64 40/40 + Running scriptlet: copy-jdk-configs-4.0-2.el8.noarch 40/40 + Running scriptlet: crypto-policies-scripts-20211116-1.gitae470d6.el8. 40/40 + Running scriptlet: nss-3.79.0-10.el8_6.x86_64 40/40 + Running scriptlet: java-11-openjdk-headless-1:11.0.16.1.1-1.el8_6.x86 40/40 + Running scriptlet: java-11-openjdk-1:11.0.16.1.1-1.el8_6.x86_64 40/40 + Running scriptlet: java-11-openjdk-devel-1:11.0.16.1.1-1.el8_6.x86_64 40/40 + Verifying : avahi-libs-0.7-20.el8.x86_64 1/40 + Verifying : crypto-policies-scripts-20211116-1.gitae470d6.el8. 2/40 + Verifying : cups-libs-1:2.2.6-45.el8_6.2.x86_64 3/40 + Verifying : libpkgconf-1.4.2-1.el8.x86_64 4/40 + Verifying : lksctp-tools-1.0.18-3.el8.x86_64 5/40 + Verifying : pkgconf-1.4.2-1.el8.x86_64 6/40 + Verifying : pkgconf-m4-1.4.2-1.el8.noarch 7/40 + Verifying : pkgconf-pkg-config-1.4.2-1.el8.x86_64 8/40 + Verifying : alsa-lib-1.2.6.1-3.el8.x86_64 9/40 + Verifying : copy-jdk-configs-4.0-2.el8.noarch 10/40 + Verifying : giflib-5.1.4-3.el8.x86_64 11/40 + Verifying : graphite2-1.3.10-10.el8.x86_64 12/40 + Verifying : harfbuzz-1.7.5-3.el8.x86_64 13/40 + Verifying : java-11-openjdk-1:11.0.16.1.1-1.el8_6.x86_64 14/40 + Verifying : java-11-openjdk-devel-1:11.0.16.1.1-1.el8_6.x86_64 15/40 + Verifying : java-11-openjdk-headless-1:11.0.16.1.1-1.el8_6.x86 16/40 + Verifying : javapackages-filesystem-5.3.0-1.module+el8+5136+7f 17/40 + Verifying : lcms2-2.9-2.el8.x86_64 18/40 + Verifying : libX11-1.6.8-5.el8.x86_64 19/40 + Verifying : libX11-common-1.6.8-5.el8.noarch 20/40 + Verifying : libXau-1.0.9-3.el8.x86_64 21/40 + Verifying : libXcomposite-0.4.4-14.el8.x86_64 22/40 + Verifying : libXext-1.3.4-1.el8.x86_64 23/40 + Verifying : libXi-1.7.10-1.el8.x86_64 24/40 + Verifying : libXrender-0.9.10-7.el8.x86_64 25/40 + Verifying : libXtst-1.2.3-7.el8.x86_64 26/40 + Verifying : libfontenc-1.1.3-8.el8.x86_64 27/40 + Verifying : libjpeg-turbo-1.5.3-12.el8.x86_64 28/40 + Verifying : libxcb-1.13.1-1.el8.x86_64 29/40 + Verifying : lua-5.3.4-12.el8.x86_64 30/40 + Verifying : nspr-4.34.0-3.el8_6.x86_64 31/40 + Verifying : nss-3.79.0-10.el8_6.x86_64 32/40 + Verifying : nss-softokn-3.79.0-10.el8_6.x86_64 33/40 + Verifying : nss-softokn-freebl-3.79.0-10.el8_6.x86_64 34/40 + Verifying : nss-sysinit-3.79.0-10.el8_6.x86_64 35/40 + Verifying : nss-util-3.79.0-10.el8_6.x86_64 36/40 + Verifying : ttmkfdir-3.0.9-54.el8.x86_64 37/40 + Verifying : tzdata-java-2022d-1.el8.noarch 38/40 + Verifying : xorg-x11-font-utils-1:7.5-41.el8.x86_64 39/40 + Verifying : xorg-x11-fonts-Type1-7.5-19.el8.noarch 40/40 + +Installed: + alsa-lib-1.2.6.1-3.el8.x86_64 + avahi-libs-0.7-20.el8.x86_64 + copy-jdk-configs-4.0-2.el8.noarch + crypto-policies-scripts-20211116-1.gitae470d6.el8.noarch + cups-libs-1:2.2.6-45.el8_6.2.x86_64 + giflib-5.1.4-3.el8.x86_64 + graphite2-1.3.10-10.el8.x86_64 + harfbuzz-1.7.5-3.el8.x86_64 + java-11-openjdk-1:11.0.16.1.1-1.el8_6.x86_64 + java-11-openjdk-devel-1:11.0.16.1.1-1.el8_6.x86_64 + java-11-openjdk-headless-1:11.0.16.1.1-1.el8_6.x86_64 + javapackages-filesystem-5.3.0-1.module+el8+5136+7ff78f74.noarch + lcms2-2.9-2.el8.x86_64 + libX11-1.6.8-5.el8.x86_64 + libX11-common-1.6.8-5.el8.noarch + libXau-1.0.9-3.el8.x86_64 + libXcomposite-0.4.4-14.el8.x86_64 + libXext-1.3.4-1.el8.x86_64 + libXi-1.7.10-1.el8.x86_64 + libXrender-0.9.10-7.el8.x86_64 + libXtst-1.2.3-7.el8.x86_64 + libfontenc-1.1.3-8.el8.x86_64 + libjpeg-turbo-1.5.3-12.el8.x86_64 + libpkgconf-1.4.2-1.el8.x86_64 + libxcb-1.13.1-1.el8.x86_64 + lksctp-tools-1.0.18-3.el8.x86_64 + lua-5.3.4-12.el8.x86_64 + nspr-4.34.0-3.el8_6.x86_64 + nss-3.79.0-10.el8_6.x86_64 + nss-softokn-3.79.0-10.el8_6.x86_64 + nss-softokn-freebl-3.79.0-10.el8_6.x86_64 + nss-sysinit-3.79.0-10.el8_6.x86_64 + nss-util-3.79.0-10.el8_6.x86_64 + pkgconf-1.4.2-1.el8.x86_64 + pkgconf-m4-1.4.2-1.el8.noarch + pkgconf-pkg-config-1.4.2-1.el8.x86_64 + ttmkfdir-3.0.9-54.el8.x86_64 + tzdata-java-2022d-1.el8.noarch + xorg-x11-font-utils-1:7.5-41.el8.x86_64 + xorg-x11-fonts-Type1-7.5-19.el8.noarch + +Complete! +Last metadata expiration check: 0:00:10 ago on Mon 10 Oct 2022 04:06:28 PM UTC. +Dependencies resolved. +============================================================================================== + Package + Arch Version Repository Size +============================================================================================== +Installing: + ords noarch 22.3.0-7.el8 yum.oracle.com_repo_OracleLinux_OL8_oracle_software_x86_64 87 M +Installing dependencies: + lsof x86_64 4.93.2-1.el8 ol8_baseos_latest 253 k + +Transaction Summary +============================================================================================== +Install 2 Packages + +Total download size: 87 M +Installed size: 92 M +Downloading Packages: +(1/2): lsof-4.93.2-1.el8.x86_64.rpm 3.0 MB/s | 253 kB 00:00 +(2/2): ords-22.3.0-7.el8.noarch.rpm 66 MB/s | 87 MB 00:01 +-------------------------------------------------------------------------------- +Total 66 MB/s | 87 MB 00:01 +Running transaction check +Transaction check succeeded. +Running transaction test +Transaction test succeeded. +Running transaction + Preparing : 1/1 + Installing : lsof-4.93.2-1.el8.x86_64 1/2 + Running scriptlet: ords-22.3.0-7.el8.noarch 2/2 + Installing : ords-22.3.0-7.el8.noarch 2/2 + Running scriptlet: ords-22.3.0-7.el8.noarch 2/2 +INFO: Before starting ORDS service, run the below command as user oracle: + ords --config /etc/ords/config install + + Verifying : lsof-4.93.2-1.el8.x86_64 1/2 + Verifying : ords-22.3.0-7.el8.noarch 2/2 + +Installed: + lsof-4.93.2-1.el8.x86_64 ords-22.3.0-7.el8.noarch + +Complete! +Last metadata expiration check: 0:00:15 ago on Mon 10 Oct 2022 04:06:28 PM UTC. +Package iproute-5.15.0-4.el8_6.1.x86_64 is already installed. +Dependencies resolved. +Nothing to do. +Complete! +24 files removed +Removing intermediate container d38a69d2cc70 + ---> 3a7b8edb327e +Step 5/10 : RUN mkdir -p $ORDS_HOME/doc_root && mkdir -p $ORDS_HOME/error && mkdir -p $ORDS_HOME/secrets && chmod ug+x $ORDS_HOME/*.sh && groupadd -g 54322 dba && usermod -u 54321 -d /home/oracle -g dba -m -s /bin/bash oracle && chown -R oracle:dba $ORDS_HOME && echo "oracle ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers + ---> Running in 1d05951f8252 +Removing intermediate container 1d05951f8252 + ---> 265cb7ab4f2c +Step 6/10 : USER oracle + ---> Running in 180d432ae42d +Removing intermediate container 180d432ae42d + ---> a9caee3d9426 +Step 7/10 : WORKDIR /home/oracle + ---> Running in bf8ac95c724a +Removing intermediate container bf8ac95c724a + ---> 4623d696e603 +Step 8/10 : VOLUME ["$ORDS_HOME/config/ords"] + ---> Running in 3afce627e4c0 +Removing intermediate container 3afce627e4c0 + ---> 914d4ee42ede +Step 9/10 : EXPOSE 8888 + ---> Running in 13460b132c52 +Removing intermediate container 13460b132c52 + ---> 4c9edba5aade +Step 10/10 : CMD $ORDS_HOME/$RUN_FILE + ---> Running in f97b17d8cea4 +Removing intermediate container f97b17d8cea4 + ---> c8e95aadf5e3 +Successfully built c8e95aadf5e3 +Successfully tagged oracle/ords-dboper:latest + diff --git a/docs/multitenant/usecase01/ImagePush.log b/docs/multitenant/usecase01/ImagePush.log new file mode 100644 index 00000000..9b8df426 --- /dev/null +++ b/docs/multitenant/usecase01/ImagePush.log @@ -0,0 +1,11 @@ +/usr/bin/docker tag oracle/ords-dboper:latest /ords-dboper:latest +/usr/bin/docker push /ords-dboper:latest +The push refers to repository [/ords-dboper] +aef18205865c: Pushing [=============================> ] 56.55MB/95.45MB +2564d855e579: Pushing [=======> ] 57.08MB/357.6MB +a70a4f9a73c3: Pushed +f283c83ba6ac: Pushed +8c6709989678: Pushing [=======> ] 52.58MB/332.7MB +5bfd57d8f58a: Pushing [========> ] 37.47MB/229.2MB + + diff --git a/docs/multitenant/usecase01/README.md b/docs/multitenant/usecase01/README.md new file mode 100644 index 00000000..428a99f5 --- /dev/null +++ b/docs/multitenant/usecase01/README.md @@ -0,0 +1,555 @@ + + + +# STEP BY STEP USE CASE + +- [INTRODUCTION](#introduction) +- [OPERATION STEPS ](#operation-steps) +- [Download latest version from orahub ](#download-latest-version-from-orahub-a-namedownloada) +- [Upload webhook certificates](#upload-webhook-certificates-a-namewebhooka) +- [Create the dboperator](#create-the-dboperator-a-namedboperatora) +- [Create Secret for container registry](#create-secret-for-container-registry) +- [Build ords immage ](#build-ords-immage-a-nameordsimagea) +- [Database Configuration](#database-configuration) +- [Create CDB secret ](#create-cdb-secret) +- [Create Certificates](#create-certificates) +- [Apply cdb.yaml](#apply-cdbyaml) +- [Logs and throuble shutting](#cdb---logs-and-throuble-shutting) +- [Create PDB secret](#create-pdb-secret) +- [Other action ](#other-actions) + + + +###### INTRODUCTION + +This readme is a step by step guide used to implement database multi tenant operator. It assumes that a kubernets cluster and a database server are already available (no matter if single instance or RAC). kubectl must be configured in order to reach k8s cluster. + +The following table reports the parameters required to configure and use oracle multi tenant controller for pluggable database lifecycle management. + +| yaml file parameters | value | description /ords parameter | +|-------------- |--------------------------- |-------------------------------------------------| +| dbserver | or | [--db-hostname][1] | +| dbTnsurl | | [--db-custom-url/db.customURL][dbtnsurl] | +| port | | [--db-port][2] | +| cdbName | | Container Name | +| name | | Ords podname prefix in cdb.yaml | +| name | | pdb resource in pdb.yaml | +| ordsImage | /ords-dboper:latest|My public container registry | +| pdbName | | Pluggable database name | +| servicename | | [--db-servicename][3] | +| sysadmin_user | | [--admin-user][adminuser] | +| sysadmin_pwd | | [--password-stdin][pwdstdin] | +| cdbadmin_user | | [db.cdb.adminUser][1] | +| cdbadmin_pwd | | [db.cdb.adminUser.password][cdbadminpwd] | +| webserver_user| | [https user][http] NOT A DB USER | +| webserver_pwd | | [http user password][http] | +| ords_pwd | | [ORDS_PUBLIC_USER password][public_user] | +| pdbTlsKey | | [standalone.https.cert.key][key] | +| pdbTlsCrt | | [standalone.https.cert][cr] | +| pdbTlsCat | | certificate authority | + + + +### OPERATIONAL STEPS +---- + +##### Download latest version from github + + +```bash +git clone https://github.com/oracle/oracle-database-operator.git +``` + +If golang compiler is installed on your environment and you've got a public container registry then you can compile the operator, upload to the registry and use it + +```bash + +cd oracle-database-operator +make generate +make manifests +make install +make docker-build IMG=/operator:latest + +make operator-yaml IMG=operator:latest +``` + +> **NOTE:** The last make executions recreates the **oracle-database-operator.yaml** with the **image:** parameter pointing to your public container registry. If you don't have a golang compilation environment you can use the **oracle-database-operator.yaml** provided in the github distribution. Check [operator installation documentation](../installation/OPERATOR_INSTALLATION_README.md ) for more details. + +> **NOTE:** If you are using oracle-container-registry make sure to accept the license agreement otherwise the operator image pull fails. +---- +##### Upload webhook certificates + +```bash +kubectl apply -f https://github.com/jetstack/cert-manager/releases/latest/download/cert-manager.yaml +``` + +##### Create the dboperator + +```bash +cd oracle-database-operator +/usr/bin/kubectl apply -f oracle-database-operator.yaml +``` ++ Check the status of the operator + +```bash +/usr/bin/kubectl get pods -n oracle-database-operator-system +NAME READY STATUS RESTARTS AGE +oracle-database-operator-controller-manager-557ff6c659-g7t66 1/1 Running 0 10s +oracle-database-operator-controller-manager-557ff6c659-rssmj 1/1 Running 0 10s +oracle-database-operator-controller-manager-557ff6c659-xpswv 1/1 Running 0 10s + +``` +---- +##### Create secret for container registry + ++ Make sure to login to your container registry and then create the secret for you container registry. + +```bash +docker login **** +/usr/bin/kubectl create secret generic container-registry-secret --from-file=.dockerconfigjson=/home/oracle/.docker/config.json --type=kubernetes.io/dockerconfigjson -n oracle-database-operator-system +``` + ++ Check secret + +```bash +kubectl get secret -n oracle-database-operator-system +NAME TYPE DATA AGE +container-registry-secret kubernetes.io/dockerconfigjson 1 19s +webhook-server-cert kubernetes.io/tls +``` +---- +##### Build ords immage + ++ Build the ords image, downloading ords software is no longer needed; just build the image and push it to your repository + +```bash +cd oracle-database-operator/ords +docker build -t oracle/ords-dboper:latest . +``` + +[example of execution](./BuildImage.log) ++ Login to your container registry and push the ords image. + +```bash +docker tag /ords-dboper:latest +docker push /ords-dboper:latest +``` +[example of execution](./ImagePush.log) + +---- +##### Database Configuration + ++ Configure Database + +Connect as sysdba and execute the following script in order to create the required ords accounts. + +```sql +ALTER SESSION SET "_oracle_script"=true; +DROP USER cascade; +CREATE USER IDENTIFIED BY CONTAINER=ALL ACCOUNT UNLOCK; +GRANT SYSOPER TO CONTAINER = ALL; +GRANT SYSDBA TO CONTAINER = ALL; +GRANT CREATE SESSION TO CONTAINER = ALL; +``` +---- +##### Create CDB secret + ++ Create secret for CDB connection + +```bash +kubectl apply -f cdb_secret.yaml -n oracle-database-operator-system + +``` +Exmaple: **cdb_secret.yaml** + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: cdb1-secret + namespace: oracle-database-operator-system +type: Opaque +data: + ords_pwd: "encoded value" + sysadmin_pwd: "encoded value" + cdbadmin_user: "encoded value" + cdbadmin_pwd: "encoded value" + webserver_user: "encoded value" + webserver_pwd: "encoded value" + +``` +Use **base64** command to encode/decode username and password in the secret file as shown in the following example + +- encode +```bash +echo "ThisIsMyPassword" |base64 -i +VGhpc0lzTXlQYXNzd29yZAo= +``` +- decode +```bash + echo "VGhpc0lzTXlQYXNzd29yZAo=" | base64 --decode +ThisIsMyPassword + +``` + + +>Note that we do not have to create webuser on the database. + ++ Check secret: + +```bash +kubectl get secret -n oracle-database-operator-system +NAME TYPE DATA AGE +cdb1-secret Opaque 6 7s <--- +container-registry-secret kubernetes.io/dockerconfigjson 1 2m17s +webhook-server-cert kubernetes.io/tls 3 4m55s +``` + +>**TIPS:** Use the following commands to analyze contents of an existing secret ```bash kubectl get secret -o yaml -n ``` +---- +##### Create Certificates + ++ Create certificates: At this stage we need to create certificates on our local machine and upload into kubernetes cluster by creating new secrets. + + + +```text + + +-----------+ + | openssl | + +-----------+ + | + | + +-----------+ + | tls.key | + | tls.crt +------------+ + | ca.crt | | + +-----------+ | + | + +------------------------|---------------------------+ + |KUBERNETES +------+--------+ | + |CLUSTER +---|kubernet secret|---+ | + | | +---------------+ | | + | | | | + | +----------+---+ https +--+----------+ | + | |ORDS CONTAINER|<-------------->| PDB/POD | | + | +----------+---+ +-------------+ | + | cdb.yaml | pdb.yaml | + +-------------|--------------------------------------+ + | + | + +-----------+ + | DB SERVER | + +-----------+ + +``` + +```bash + +genrsa -out 2048 +openssl req -new -x509 -days 365 -key -subj "/C=CN/ST=GD/L=SZ/O=oracle, Inc./CN=oracle Root CA" -out +openssl req -newkey rsa:2048 -nodes -keyout -subj "/C=CN/ST=GD/L=SZ/O=oracle, Inc./CN=-ords" -out server.csr +/usr/bin/echo "subjectAltName=DNS:-ords,DNS:www.example.com" > extfile.txt +openssl x509 -req -extfile extfile.txt -days 365 -in server.csr -CA -CAkey -CAcreateserial -out + +kubectl create secret tls db-tls --key="" --cert="" -n oracle-database-operator-system +kubectl create secret generic db-ca --from-file= -n oracle-database-operator-system + +``` + +[example of execution:](./openssl_execution.log) + + +---- +###### Apply cdb.yaml + ++ Create ords container + +```bash +/usr/bin/kubectl apply -f cdb.yaml -n oracle-database-operator-system +``` +Example: **cdb.yaml** + +```yaml +apiVersion: database.oracle.com/v1alpha1 +kind: CDB +metadata: + name: + namespace: oracle-database-operator-system +spec: + cdbName: "" + dbServer: "" or + dbPort: + ordsImage: "/ords-dboper:.latest" + ordsImagePullPolicy: "Always" + serviceName: + replicas: 1 + sysAdminPwd: + secret: + secretName: "cdb1-secret" + key: "sysadmin_pwd" + ordsPwd: + secret: + secretName: "cdb1-secret" + key: "ords_pwd" + cdbAdminUser: + secret: + secretName: "cdb1-secret" + key: "cdbadmin_user" + cdbAdminPwd: + secret: + secretName: "cdb1-secret" + key: "cdbadmin_pwd" + webServerUser: + secret: + secretName: "cdb1-secret" + key: "webserver_user" + webServerPwd: + secret: + secretName: "cdb1-secret" + key: "webserver_pwd" + cdbTlsKey: + secret: + secretName: "db-tls" + key: "" + cdbTlsCrt: + secret: + secretName: "db-tls" + key: ":" + +``` +> **Note** if you are working in dataguard environment with multiple sites (AC/DR) specifying the host name (dbServer/dbPort/serviceName) may not be the suitable solution for this kind of configuration, use **dbTnsurl** instead. Specify the whole tns string which includes the hosts/scan list. + +``` + +----------+ + ____| standbyB | + | | scanB | (DESCRIPTION= + +----------+ | +----------+ (CONNECT_TIMEOUT=90) + | primary |_______| (RETRY_COUNT=30)(RETRY_DELAY=10)(TRANSPORT_CONNECT_TIMEOUT=70) + | scanA | | +----------+ (TRANSPORT_CONNECT_TIMEOUT=10)(LOAD_BALLANCE=ON) + +----------+ |___| stanbyC | (ADDRESS=(PROTOCOL=TCP)(HOST=scanA.testrac.com)(PORT=1521)(IP=V4_ONLY)) + | scanC | (ADDRESS=(PROTOCOL=TCP)(HOST=scanB.testrac.com)(PORT=1521)(IP=V4_ONLY)) + +----------+ (ADDRESS=(PROTOCOL=TCP)(HOST=scanC.testrac.com)(PORT=1521)(IP=V4_ONLY)) + (CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=TESTORDS))) + + + dbtnsurl:((DESCRIPTION=(CONNECT_TIMEOUT=90)(RETRY_COUNT=30)(RETRY_DELAY=10)(TRANSPORT_CONNECT_TIMEOUT=70)(TRANS...... +``` + +[example of cdb.yaml](./cdb.yaml) + +---- +###### CDB - Logs and throuble shutting + ++ Check the status of ords container + +```bash +/usr/bin/kubectl get pods -n oracle-database-operator-system +NAME READY STATUS RESTARTS AGE +cdb-dev-ords-rs-m9ggp 0/1 ContainerCreating 0 67s <----- +oracle-database-operator-controller-manager-557ff6c659-g7t66 1/1 Running 0 11m +oracle-database-operator-controller-manager-557ff6c659-rssmj 1/1 Running 0 11m +oracle-database-operator-controller-manager-557ff6c659-xpswv 1/1 Running 0 11m +``` ++ Make sure that the cdb container is running + +```bash +/usr/bin/kubectl get pods -n oracle-database-operator-system +NAME READY STATUS RESTARTS AGE +cdb-dev-ords-rs-dnshz 1/1 Running 0 31s +oracle-database-operator-controller-manager-557ff6c659-9bjfl 1/1 Running 0 2m42s +oracle-database-operator-controller-manager-557ff6c659-cx8hd 1/1 Running 0 2m42s +oracle-database-operator-controller-manager-557ff6c659-rq9xs 1/1 Running 0 2m42s +``` ++ Check the status of the services + +```bash +kubectl get cdb -n oracle-database-operator-system +NAME CDB NAME DB SERVER DB PORT REPLICAS STATUS MESSAGE +[.....................................................] Ready +``` ++ Use log file to trouble shutting + +```bash +/usr/bin/kubectl logs `/usr/bin/kubectl get pods -n oracle-database-operator-system|grep ords|cut -d ' ' -f 1` -n oracle-database-operator-system +``` +[example of execution](./cdb.log) + ++ Test REST API from the pod. By querying the metadata catalog you can verify the status of https setting + +```bash + /usr/bin/kubectl exec -it `/usr/bin/kubectl get pods -n oracle-database-operator-system|grep ords|cut -d ' ' -f 1` -n oracle-database-operator-system -i -t -- /usr/bin/curl -sSkv -k -X GET https://localhost:8888/ords/_/db-api/stable/metadata-catalog/ +``` +[example of execution](./testapi.log) + ++ Verify the pod environment varaibles + ```bash + kubectl set env pods --all --list -n oracle-database-operator-system + ``` + ++ Connect to cdb pod + +```bash + kubectl exec -it `kubectl get pods -n oracle-database-operator-system|grep ords|cut -d ' ' -f 1` -n oracle-database-operator-system bash +``` ++ Dump ords server configuration + +```bash +/usr/bin/kubectl exec -it `/usr/bin/kubectl get pods -n oracle-database-operator-system|grep ords|cut -d ' ' -f 1` -n oracle-database-operator-system -i -t -- /usr/local/bin/ords --config /etc/ords/config config list +``` +[Example of executions](./ordsconfig.log) + +----- +###### Create PDB secret + + +```bash +/usr/bin/kubectl apply -f pdb.yaml -n oracle-database-operator-system +``` +Exmaple: **pdb_secret.yaml** + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: pdb1-secret + namespace: oracle-database-operator-system +type: Opaque +data: + sysadmin_user: "encoded value" + sysadmin_pwd: "encoded value" +``` + ++ Check secret creation + +```bash +kubectl get secret -n oracle-database-operator-system +NAME TYPE DATA AGE +cdb1-secret Opaque 6 79m +container-registry-secret kubernetes.io/dockerconfigjson 1 79m +db-ca Opaque 1 78m +db-tls kubernetes.io/tls 2 78m +pdb1-secret Opaque 2 79m <--- +webhook-server-cert kubernetes.io/tls 3 79m +``` +--- +###### Apply pdb yaml file to create pdb + +```bash +/usr/bin/kubectl apply -f pdb.yaml -n oracle-database-operator-system +``` + +Example: **pdb.yaml** + +```yaml +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: + namespace: oracle-database-operator-system + labels: + cdb: +spec: + cdbResName: "" + cdbName: "" + pdbName: + adminName: + secret: + secretName: "pdb1-secret" + key: "sysadmin_user" + adminPwd: + secret: + secretName: "pdb1-secret" + key: "sysadmin_pwd" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "" + fileNameConversions: "NONE" + totalSize: "1G" + tempSize: "100M" + action: "Create" +``` + ++ Monitor the pdb creation status until message is success + +```bash +kubectl get pdbs --all-namespaces=true + + +-----------------------------------------+ +-----------------------------------------+ + | STATUS MESSAGE |______\ | STATUS MESSAGE | + | Creating Waiting for PDB to be created | / | Ready Success | + +-----------------------------------------+ +-----------------------------------------+ + +NAMESPACE NAME DBSERVER CDB NAME PDB NAME PDB STATE PDB SIZE STATUS MESSAGE +oracle-database-operator-system 1G Creating Waiting for PDB to be created + +[wait sometimes] + +kubectl get pdbs --all-namespaces=true +NAMESPACE NAME DBSERVER CDB NAME PDB NAME PDB STATE PDB SIZE STATUS MESSAGE +oracle-database-operator-system pdb1 READ WRITE 1G Ready Success +``` + +Connect to the hosts and verify the PDB creation. + +```text +[oracle@racnode1 ~]$ sqlplus '/as sysdba' +[...] +Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production +Version 19.15.0.0.0 + + +SQL> show pdbs + + CON_ID CON_NAME OPEN MODE RESTRICTED +---------- ------------------------------ ---------- ---------- + 2 PDB$SEED READ ONLY NO + 3 PDBDEV READ WRITE NO + +``` +Check controller log to debug pluggable database life cycle actions in case of problem + +```bash +kubectl logs -f $(kubectl get pods -n oracle-database-operator-system|grep oracle-database-operator-controller|head -1|cut -d ' ' -f 1) -n oracle-database-operator-system +``` + +--- +###### Other actions + +Configure and use other yaml files to perform pluggable database life cycle managment action **modify_pdb_open.yaml** **modify_pdb_close.yaml** + +> **Note** sql command *"alter pluggable database open instances=all;"* acts only on closed databases, so you want get any oracle error in case of execution against an pluggable database already opened + + + + +[1]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/installing-and-configuring-oracle-rest-data-services.html#GUID-E9625FAB-9BC8-468B-9FF9-443C88D76FA1:~:text=Table%202%2D2%20Command%20Options%20for%20Command%2DLine%20Interface%20Installation + +[2]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/installing-and-configuring-oracle-rest-data-services.html#GUID-E9625FAB-9BC8-468B-9FF9-443C88D76FA1:~:text=Table%202%2D2%20Command%20Options%20for%20Command%2DLine%20Interface%20Installation + +[3]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/installing-and-configuring-oracle-rest-data-services.html#GUID-DAA027FA-A4A6-43E1-B8DD-C92B330C2341:~:text=%2D%2Ddb%2Dservicename%20%3Cstring%3E + +[adminuser]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/installing-and-configuring-oracle-rest-data-services.html#GUID-A9AED253-4EEC-4E13-A0C4-B7CE82EC1C22:~:text=Table%202%2D6%20Command%20Options%20for%20Uninstall%20CLI + +[public_user]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/using-multitenant-architecture-oracle-rest-data-services.html#GUID-E64A141A-A71F-4979-8D33-C5F8496D3C19:~:text=Preinstallation%20Tasks%20for%20Oracle%20REST%20Data%20Services%20CDB%20Installation + +[key]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/about-REST-configuration-files.html#GUID-006F916B-8594-4A78-B500-BB85F35C12A0:~:text=standalone.https.cert.key + +[cr]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/about-REST-configuration-files.html#GUID-006F916B-8594-4A78-B500-BB85F35C12A0 + +[cdbadminpwd]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/about-REST-configuration-files.html#GUID-006F916B-8594-4A78-B500-BB85F35C12A0:~:text=Table%20C%2D1%20Oracle%20REST%20Data%20Services%20Configuration%20Settings + +[pwdstdin]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/installing-and-configuring-oracle-rest-data-services.html#GUID-88479C84-CAC1-4133-A33E-7995A645EC05:~:text=default%20database%20pool.-,2.1.4.1%20Understanding%20Command%20Options%20for%20Command%2DLine%20Interface%20Installation,-Table%202%2D2 + +[http]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/installing-and-configuring-oracle-rest-data-services.html#GUID-BEECC057-A8F5-4EAB-B88E-9828C2809CD8:~:text=Example%3A%20delete%20%5B%2D%2Dglobal%5D-,user%20add,-Add%20a%20user + +[dbtnsurl]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/installing-and-configuring-oracle-rest-data-services.html#GUID-A9AED253-4EEC-4E13-A0C4-B7CE82EC1C22 \ No newline at end of file diff --git a/docs/multitenant/usecase01/cdb.log b/docs/multitenant/usecase01/cdb.log new file mode 100644 index 00000000..c75e9bf8 --- /dev/null +++ b/docs/multitenant/usecase01/cdb.log @@ -0,0 +1,372 @@ +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M +NOT_INSTALLED=2 + SETUP +==================================================== +CONFIG=/etc/ords/config ++ export ORDS_LOGS=/tmp ++ ORDS_LOGS=/tmp ++ '[' -f /opt/oracle/ords//secrets/webserver_user ']' +++ cat /opt/oracle/ords//secrets/webserver_user ++ WEBSERVER_USER=.... ++ '[' -f /opt/oracle/ords//secrets/webserver_pwd ']' +++ cat /opt/oracle/ords//secrets/webserver_pwd ++ WEBSERVER_PASSWORD=.... ++ '[' -f /opt/oracle/ords//secrets/cdbadmin_user ']' +++ cat /opt/oracle/ords//secrets/cdbadmin_user ++ CDBADMIN_USER=.... ++ '[' -f /opt/oracle/ords//secrets/cdbadmin_pwd ']' +++ cat /opt/oracle/ords//secrets/cdbadmin_pwd ++ CDBADMIN_PWD=.... ++ '[' -f /opt/oracle/ords//secrets/sysadmin_pwd ']' +++ cat /opt/oracle/ords//secrets/sysadmin_pwd ++ SYSDBA_PASSWORD=..... ++ '[' -f /opt/oracle/ords//secrets/sysadmin_pwd ']' +++ cat /opt/oracle/ords//secrets/ords_pwd ++ ORDS_PASSWORD=.... ++ setupHTTPS ++ rm -rf /home/oracle/keystore ++ '[' '!' -d /home/oracle/keystore ']' ++ mkdir /home/oracle/keystore ++ cd /home/oracle/keystore ++ cat ++ rm /home/oracle/keystore/PASSWORD ++ ls -ltr /home/oracle/keystore +total 0 ++ SetParameter ++ /usr/local/bin/ords --config /etc/ords/config config set security.requestValidationFunction false +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 22.3 Production on Tue Oct 11 07:45:22 2022 + +Copyright (c) 2010, 2022, Oracle. + +Configuration: + /etc/ords/config/ + +The setting named: security.requestValidationFunction was set to: false in configuration: default ++ /usr/local/bin/ords --config /etc/ords/config config set jdbc.MaxLimit 100 +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 22.3 Production on Tue Oct 11 07:45:23 2022 + +Copyright (c) 2010, 2022, Oracle. + +Configuration: + /etc/ords/config/ + +The setting named: jdbc.MaxLimit was set to: 100 in configuration: default ++ /usr/local/bin/ords --config /etc/ords/config config set jdbc.InitialLimit 50 +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 22.3 Production on Tue Oct 11 07:45:24 2022 + +Copyright (c) 2010, 2022, Oracle. + +Configuration: + /etc/ords/config/ + +The setting named: jdbc.InitialLimit was set to: 50 in configuration: default ++ /usr/local/bin/ords --config /etc/ords/config config set error.externalPath /opt/oracle/ords/error +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 22.3 Production on Tue Oct 11 07:45:26 2022 + +Copyright (c) 2010, 2022, Oracle. + +Configuration: + /etc/ords/config/ + +The global setting named: error.externalPath was set to: /opt/oracle/ords/error ++ /usr/local/bin/ords --config /etc/ords/config config set standalone.access.log /home/oracle +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 22.3 Production on Tue Oct 11 07:45:27 2022 + +Copyright (c) 2010, 2022, Oracle. + +Configuration: + /etc/ords/config/ + +The global setting named: standalone.access.log was set to: /home/oracle ++ /usr/local/bin/ords --config /etc/ords/config config set standalone.https.port 8888 +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 22.3 Production on Tue Oct 11 07:45:28 2022 + +Copyright (c) 2010, 2022, Oracle. + +Configuration: + /etc/ords/config/ + +The global setting named: standalone.https.port was set to: 8888 ++ /usr/local/bin/ords --config /etc/ords/config config set standalone.https.cert /opt/oracle/ords//secrets/tls.crt +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 22.3 Production on Tue Oct 11 07:45:29 2022 + +Copyright (c) 2010, 2022, Oracle. + +Configuration: + /etc/ords/config/ + +The global setting named: standalone.https.cert was set to: /opt/oracle/ords//secrets/tls.crt ++ /usr/local/bin/ords --config /etc/ords/config config set standalone.https.cert.key /opt/oracle/ords//secrets/tls.key +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 22.3 Production on Tue Oct 11 07:45:31 2022 + +Copyright (c) 2010, 2022, Oracle. + +Configuration: + /etc/ords/config/ + +The global setting named: standalone.https.cert.key was set to: /opt/oracle/ords//secrets/tls.key ++ /usr/local/bin/ords --config /etc/ords/config config set restEnabledSql.active true +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 22.3 Production on Tue Oct 11 07:45:32 2022 + +Copyright (c) 2010, 2022, Oracle. + +Configuration: + /etc/ords/config/ + +The setting named: restEnabledSql.active was set to: true in configuration: default ++ /usr/local/bin/ords --config /etc/ords/config config set security.verifySSL true +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 22.3 Production on Tue Oct 11 07:45:33 2022 + +Copyright (c) 2010, 2022, Oracle. + +Configuration: + /etc/ords/config/ + +The global setting named: security.verifySSL was set to: true ++ /usr/local/bin/ords --config /etc/ords/config config set database.api.enabled true +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 22.3 Production on Tue Oct 11 07:45:34 2022 + +Copyright (c) 2010, 2022, Oracle. + +Configuration: + /etc/ords/config/ + +The global setting named: database.api.enabled was set to: true ++ /usr/local/bin/ords --config /etc/ords/config config set plsql.gateway.mode false +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 22.3 Production on Tue Oct 11 07:45:35 2022 + +Copyright (c) 2010, 2022, Oracle. + +Configuration: + /etc/ords/config/ + +Invalid VALUE argument false for KEY plsql.gateway.mode. ++ /usr/local/bin/ords --config /etc/ords/config config set database.api.management.services.disabled false +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 22.3 Production on Tue Oct 11 07:45:37 2022 + +Copyright (c) 2010, 2022, Oracle. + +Configuration: + /etc/ords/config/ + +The global setting named: database.api.management.services.disabled was set to: false ++ /usr/local/bin/ords --config /etc/ords/config config set misc.pagination.maxRows 1000 +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 22.3 Production on Tue Oct 11 07:45:38 2022 + +Copyright (c) 2010, 2022, Oracle. + +Configuration: + /etc/ords/config/ + +The setting named: misc.pagination.maxRows was set to: 1000 in configuration: default ++ /usr/local/bin/ords --config /etc/ords/config config set db.cdb.adminUser 'C##DBAPI_CDB_ADMIN AS SYSDBA' +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 22.3 Production on Tue Oct 11 07:45:39 2022 + +Copyright (c) 2010, 2022, Oracle. + +Configuration: + /etc/ords/config/ + +The setting named: db.cdb.adminUser was set to: C##DBAPI_CDB_ADMIN AS SYSDBA in configuration: default ++ /usr/local/bin/ords --config /etc/ords/config config secret --password-stdin db.cdb.adminUser.password +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 22.3 Production on Tue Oct 11 07:45:40 2022 + +Copyright (c) 2010, 2022, Oracle. + +Configuration: + /etc/ords/config/ + +The setting named: db.cdb.adminUser.password was set to: ****** in configuration: default ++ /usr/local/bin/ords --config /etc/ords/config config user add --password-stdin sql_admin 'SQL Administrator, System Administrator' +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 22.3 Production on Tue Oct 11 07:45:42 2022 + +Copyright (c) 2010, 2022, Oracle. + +Configuration: + /etc/ords/config/ + +Created user sql_admin in file /etc/ords/config/global/credentials ++ /usr/local/bin/ords --config /etc/ords/config install --admin-user 'SYS AS SYSDBA' --db-hostname racnode1.testrac.com --db-port 1521 --db-servicename TESTORDS --feature-db-api true --feature-rest-enabled-sql true --log-folder /tmp --proxy-user --password-stdin +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 22.3 Production on Tue Oct 11 07:45:43 2022 + +Copyright (c) 2010, 2022, Oracle. + +Configuration: + /etc/ords/config/ + +Oracle REST Data Services - Non-Interactive Install +Connecting to database user: SYS AS SYSDBA url: jdbc:oracle:thin:@//racnode1.testrac.com:1521/TESTORDS + +Retrieving information.. +Your database connection is to a CDB. ORDS common user ORDS_PUBLIC_USER will be created in the CDB. ORDS schema will be installed in the PDBs. +Root CDB$ROOT - create ORDS common user +PDB PDB$SEED - install ORDS 22.3.0.r2781755 +PDB PDB$SEED - configure PL/SQL gateway user APEX_PUBLIC_USER in ORDS version 22.3.0.r2781755 + +The setting named: db.connectionType was set to: basic in configuration: default +The setting named: db.hostname was set to: racnode1.testrac.com in configuration: default +The setting named: db.port was set to: 1521 in configuration: default +The setting named: db.servicename was set to: TESTORDS in configuration: default +The setting named: db.serviceNameSuffix was set to: in configuration: default +The setting named: plsql.gateway.mode was set to: proxied in configuration: default +The setting named: db.username was set to: ORDS_PUBLIC_USER in configuration: default +The setting named: db.password was set to: ****** in configuration: default +The setting named: security.requestValidationFunction was set to: wwv_flow_epg_include_modules.authorize in configuration: default +2022-10-11T07:45:45.885Z INFO Installing Oracle REST Data Services version 22.3.0.r2781755 in CDB$ROOT +2022-10-11T07:45:46.703Z INFO ... Verified database prerequisites +2022-10-11T07:45:46.946Z INFO ... Created Oracle REST Data Services proxy user +2022-10-11T07:45:46.979Z INFO Completed installation for Oracle REST Data Services version 22.3.0.r2781755. Elapsed time: 00:00:01.71 + +2022-10-11T07:45:46.986Z INFO Installing Oracle REST Data Services version 22.3.0.r2781755 in PDB$SEED +2022-10-11T07:45:47.078Z INFO ... Verified database prerequisites +2022-10-11T07:45:47.290Z INFO ... Created Oracle REST Data Services proxy user +2022-10-11T07:45:47.741Z INFO ... Created Oracle REST Data Services schema +2022-10-11T07:45:48.097Z INFO ... Granted privileges to Oracle REST Data Services +2022-10-11T07:45:51.848Z INFO ... Created Oracle REST Data Services database objects +2022-10-11T07:46:00.829Z INFO Completed installation for Oracle REST Data Services version 22.3.0.r2781755. Elapsed time: 00:00:13.841 + +2022-10-11T07:46:00.898Z INFO Completed configuring PL/SQL gateway user for Oracle REST Data Services version 22.3.0.r2781755. Elapsed time: 00:00:00.68 + +2022-10-11T07:46:00.898Z INFO Completed CDB installation for Oracle REST Data Services version 22.3.0.r2781755. Total elapsed time: 00:00:15.17 + +2022-10-11T07:46:00.898Z INFO Log file written to /tmp/ords_cdb_install_2022-10-11_074545_78000.log +2022-10-11T07:46:00.901Z INFO To run in standalone mode, use the ords serve command: +2022-10-11T07:46:00.901Z INFO ords --config /etc/ords/config serve +2022-10-11T07:46:00.901Z INFO Visit the ORDS Documentation to access tutorials, developer guides and more to help you get started with the new ORDS Command Line Interface (http://oracle.com/rest). ++ '[' 0 -ne 0 ']' ++ StartUp ++ /usr/local/bin/ords --config /etc/ords/config serve --port 8888 --secure +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 22.3 Production on Tue Oct 11 07:46:02 2022 + +Copyright (c) 2010, 2022, Oracle. + +Configuration: + /etc/ords/config/ + +2022-10-11T07:46:02.286Z INFO HTTPS and HTTPS/2 listening on host: 0.0.0.0 port: 8888 +2022-10-11T07:46:02.302Z INFO Disabling document root because the specified folder does not exist: /etc/ords/config/global/doc_root +2022-10-11T07:46:04.636Z INFO Configuration properties for: |default|lo| +db.servicename=TESTORDS +db.serviceNameSuffix= +java.specification.version=19 +conf.use.wallet=true +database.api.management.services.disabled=false +sun.jnu.encoding=UTF-8 +user.region=US +java.class.path=/opt/oracle/ords/ords.war +java.vm.vendor=Oracle Corporation +standalone.https.cert.key=/opt/oracle/ords//secrets/tls.key +sun.arch.data.model=64 +nashorn.args=--no-deprecation-warning +java.vendor.url=https://java.oracle.com/ +resource.templates.enabled=false +user.timezone=UTC +db.port=1521 +java.vm.specification.version=19 +os.name=Linux +sun.java.launcher=SUN_STANDARD +user.country=US +sun.boot.library.path=/usr/java/jdk-19/lib +sun.java.command=/opt/oracle/ords/ords.war --config /etc/ords/config serve --port 8888 --secure +jdk.debug=release +sun.cpu.endian=little +user.home=/home/oracle +oracle.dbtools.launcher.executable.jar.path=/opt/oracle/ords/ords.war +user.language=en +db.cdb.adminUser.password=****** +java.specification.vendor=Oracle Corporation +java.version.date=2022-09-20 +database.api.enabled=true +java.home=/usr/java/jdk-19 +db.username=ORDS_PUBLIC_USER +file.separator=/ +java.vm.compressedOopsMode=32-bit +line.separator= + +restEnabledSql.active=true +java.specification.name=Java Platform API Specification +java.vm.specification.vendor=Oracle Corporation +java.awt.headless=true +standalone.https.cert=/opt/oracle/ords//secrets/tls.crt +db.hostname=racnode1.testrac.com +db.password=****** +sun.management.compiler=HotSpot 64-Bit Tiered Compilers +security.requestValidationFunction=wwv_flow_epg_include_modules.authorize +misc.pagination.maxRows=1000 +java.runtime.version=19+36-2238 +user.name=oracle +error.externalPath=/opt/oracle/ords/error +stdout.encoding=UTF-8 +path.separator=: +db.cdb.adminUser=C##DBAPI_CDB_ADMIN AS SYSDBA +os.version=5.4.17-2136.308.9.el7uek.x86_64 +java.runtime.name=Java(TM) SE Runtime Environment +file.encoding=UTF-8 +plsql.gateway.mode=proxied +security.verifySSL=true +standalone.https.port=8888 +java.vm.name=Java HotSpot(TM) 64-Bit Server VM +java.vendor.url.bug=https://bugreport.java.com/bugreport/ +java.io.tmpdir=/tmp +oracle.dbtools.cmdline.ShellCommand=ords +java.version=19 +user.dir=/home/oracle/keystore +os.arch=amd64 +java.vm.specification.name=Java Virtual Machine Specification +jdbc.MaxLimit=100 +oracle.dbtools.cmdline.home=/opt/oracle/ords +native.encoding=UTF-8 +java.library.path=/usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib +java.vendor=Oracle Corporation +java.vm.info=mixed mode, sharing +stderr.encoding=UTF-8 +java.vm.version=19+36-2238 +sun.io.unicode.encoding=UnicodeLittle +jdbc.InitialLimit=50 +db.connectionType=basic +java.class.version=63.0 +standalone.access.log=/home/oracle + +2022-10-11T07:46:06.669Z INFO Oracle REST Data Services initialized +Oracle REST Data Services version : 22.3.0.r2781755 +Oracle REST Data Services server info: jetty/10.0.11 +Oracle REST Data Services java info: Java HotSpot(TM) 64-Bit Server VM 19+36-2238 + diff --git a/docs/multitenant/usecase01/cdb.yaml b/docs/multitenant/usecase01/cdb.yaml new file mode 100644 index 00000000..25488168 --- /dev/null +++ b/docs/multitenant/usecase01/cdb.yaml @@ -0,0 +1,46 @@ +apiVersion: database.oracle.com/v1alpha1 +kind: CDB +metadata: + name: cdb-dev + namespace: oracle-database-operator-system +spec: + cdbName: "DB12" + dbServer: "racnode1.testrac.com" + dbPort: 1521 + ordsImage: "/ords-dboper:latest" + ordsImagePullPolicy: "Always" + serviceName: "TESTORDS" + replicas: 1 + sysAdminPwd: + secret: + secretName: "cdb1-secret" + key: "sysadmin_pwd" + ordsPwd: + secret: + secretName: "cdb1-secret" + key: "ords_pwd" + cdbAdminUser: + secret: + secretName: "cdb1-secret" + key: "cdbadmin_user" + cdbAdminPwd: + secret: + secretName: "cdb1-secret" + key: "cdbadmin_pwd" + webServerUser: + secret: + secretName: "cdb1-secret" + key: "webserver_user" + webServerPwd: + secret: + secretName: "cdb1-secret" + key: "webserver_pwd" + cdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + cdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + diff --git a/docs/multitenant/usecase01/openssl_execution.log b/docs/multitenant/usecase01/openssl_execution.log new file mode 100644 index 00000000..30a1c5d4 --- /dev/null +++ b/docs/multitenant/usecase01/openssl_execution.log @@ -0,0 +1,22 @@ +/usr/bin/openssl genrsa -out ca.key 2048 +Generating RSA private key, 2048 bit long modulus +......................................................................................................................................................................................+++ +...................................+++ +e is 65537 (0x10001) +/usr/bin/openssl req -new -x509 -days 365 -key ca.key -subj "/C=CN/ST=GD/L=SZ/O=oracle, Inc./CN=oracle Root CA" -out ca.crt +/usr/bin/openssl req -newkey rsa:2048 -nodes -keyout tls.key -subj "/C=CN/ST=GD/L=SZ/O=oracle, Inc./CN=cdb-dev-ords" -out server.csr +Generating a 2048 bit RSA private key +...................................+++ +........................................+++ +writing new private key to 'tls.key' +----- +/usr/bin/echo "subjectAltName=DNS:cdb-dev-ords,DNS:www.example.com" > extfile.txt +/usr/bin/openssl x509 -req -extfile extfile.txt -days 365 -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out tls.crt +Signature ok +subject=/C=CN/ST=GD/L=SZ/O=oracle, Inc./CN=cdb-dev-ords +Getting CA Private Key +/usr/bin/kubectl create secret tls db-tls --key="tls.key" --cert="tls.crt" -n oracle-database-operator-system +secret/db-tls created +/usr/bin/kubectl create secret generic db-ca --from-file="ca.crt" -n oracle-database-operator-system +secret/db-ca created + diff --git a/docs/multitenant/usecase01/ordsconfig.log b/docs/multitenant/usecase01/ordsconfig.log new file mode 100644 index 00000000..ad5e7bab --- /dev/null +++ b/docs/multitenant/usecase01/ordsconfig.log @@ -0,0 +1,35 @@ +: Release 22.3 Production on Tue Oct 11 12:51:50 2022 + +Copyright (c) 2010, 2022, Oracle. + +Configuration: + /etc/ords/config/ + +Database pool: default + +Setting Value Source +----------------------------------------- -------------------------------------- ----------- +database.api.enabled true Global +database.api.management.services.disabled false Global +db.cdb.adminUser C##DBAPI_CDB_ADMIN AS SYSDBA Pool +db.cdb.adminUser.password ****** Pool Wallet +db.connectionType basic Pool +db.hostname racnode1.testrac.com Pool +db.password ****** Pool Wallet +db.port 1521 Pool +db.serviceNameSuffix Pool +db.servicename TESTORDS Pool +db.username ORDS_PUBLIC_USER Pool +error.externalPath /opt/oracle/ords/error Global +jdbc.InitialLimit 50 Pool +jdbc.MaxLimit 100 Pool +misc.pagination.maxRows 1000 Pool +plsql.gateway.mode proxied Pool +restEnabledSql.active true Pool +security.requestValidationFunction wwv_flow_epg_include_modules.authorize Pool +security.verifySSL true Global +standalone.access.log /home/oracle Global +standalone.https.cert /opt/oracle/ords//secrets/tls.crt Global +standalone.https.cert.key /opt/oracle/ords//secrets/tls.key Global +standalone.https.port 8888 Global + diff --git a/docs/multitenant/usecase01/sync.sh b/docs/multitenant/usecase01/sync.sh new file mode 100644 index 00000000..d982e796 --- /dev/null +++ b/docs/multitenant/usecase01/sync.sh @@ -0,0 +1,5 @@ +#!/bin/bash + + +sudo cp README.md /home/oracle/WORKDIR/Ords22_operator/oracle-database-operator/docs/ords +sudo chown oracle:dba /home/oracle/WORKDIR/Ords22_operator/oracle-database-operator/docs/ords/README.md diff --git a/docs/multitenant/usecase01/testapi.log b/docs/multitenant/usecase01/testapi.log new file mode 100644 index 00000000..4c95b457 --- /dev/null +++ b/docs/multitenant/usecase01/testapi.log @@ -0,0 +1,49 @@ +* Trying 127.0.0.1... +* TCP_NODELAY set +* Connected to localhost (127.0.0.1) port 8888 (#0) +* ALPN, offering h2 +* ALPN, offering http/1.1 +* successfully set certificate verify locations: +* CAfile: /etc/pki/tls/certs/ca-bundle.crt + CApath: none +* TLSv1.3 (OUT), TLS handshake, Client hello (1): +* TLSv1.3 (IN), TLS handshake, Server hello (2): +* TLSv1.3 (IN), TLS handshake, [no content] (0): +* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8): +* TLSv1.3 (IN), TLS handshake, Certificate (11): +* TLSv1.3 (IN), TLS handshake, CERT verify (15): +* TLSv1.3 (IN), TLS handshake, Finished (20): +* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1): +* TLSv1.3 (OUT), TLS handshake, [no content] (0): +* TLSv1.3 (OUT), TLS handshake, Finished (20): +* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 +* ALPN, server accepted to use h2 +* Server certificate: +* subject: C=CN; ST=GD; L=SZ; O=oracle, Inc.; CN=cdb-dev-ords +* start date: Oct 11 07:44:38 2022 GMT +* expire date: Oct 11 07:44:38 2023 GMT +* issuer: C=CN; ST=GD; L=SZ; O=oracle, Inc.; CN=oracle Root CA +* SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway. +* Using HTTP2, server supports multi-use +* Connection state changed (HTTP/2 confirmed) +* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0 +* TLSv1.3 (OUT), TLS app data, [no content] (0): +* TLSv1.3 (OUT), TLS app data, [no content] (0): +* TLSv1.3 (OUT), TLS app data, [no content] (0): +* Using Stream ID: 1 (easy handle 0x564be7b0b970) +* TLSv1.3 (OUT), TLS app data, [no content] (0): +> GET /ords/_/db-api/stable/metadata-catalog/ HTTP/2 +> Host: localhost:8888 +> User-Agent: curl/7.61.1 +> Accept: */* +> +* TLSv1.3 (IN), TLS handshake, [no content] (0): +* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4): +* TLSv1.3 (IN), TLS handshake, [no content] (0): +* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4): +* TLSv1.3 (IN), TLS app data, [no content] (0): +* Connection state changed (MAX_CONCURRENT_STREAMS == 128)! +* TLSv1.3 (OUT), TLS app data, [no content] (0): +* TLSv1.3 (IN), TLS app data, [no content] (0): +* TLSv1.3 (IN), TLS app data, [no content] (0): + diff --git a/oracle-database-operator.yaml b/oracle-database-operator.yaml index 95786f0b..309cbcb0 100644 --- a/oracle-database-operator.yaml +++ b/oracle-database-operator.yaml @@ -619,6 +619,10 @@ spec: jsonPath: .spec.dbPort name: DB Port type: integer + - description: ' string of the tnsalias' + jsonPath: .spec.dbTnsurl + name: TNS STRING + type: string - description: Replicas jsonPath: .spec.replicas name: Replicas @@ -722,6 +726,8 @@ spec: dbServer: description: Name of the DB server type: string + dbTnsurl: + type: string nodeSelector: additionalProperties: type: string diff --git a/ords/runOrdsSSL.sh b/ords/runOrdsSSL.sh index ac357637..23b99f1e 100644 --- a/ords/runOrdsSSL.sh +++ b/ords/runOrdsSSL.sh @@ -10,6 +10,7 @@ # # MODIFIED (DD-Mon-YY) # mmalvezz 25-Jun-22 - Initial version +# mmalvezz 17-Oct-22 - db.customURL utilization export ORDS=/usr/local/bin/ords export ETCFILE=/etc/ords.conf @@ -24,13 +25,32 @@ export HN=`hostname` #export CERTIFICATE=${KEYSTORE}/${HN}.der export KEY=$ORDS_HOME/secrets/$TLSKEY export CERTIFICATE=$ORDS_HOME/secrets/$TLSCRT +export TNS_ADMIN=/opt/oracle/ords/ +export TNSNAME=${TNS_ADMIN}/tnsnames.ora +export TNSALIAS=ordstns +echo "${TNSALIAS}=${DBTNSURL}" >$TNSNAME + + -export CUSTOMURL="jdbc:oracle:thin:@(DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = racnode1)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = TESTORDS)))" -echo $CUSTOMURL function SetParameter() { ##ords config info <--- Use this command to get the list +[[ ! -z "${ORACLE_HOST}" && -z "${DBTNSURL}" ]] && { + $ORDS --config ${CONFIG} config set db.hostname ${ORACLE_HOST:-racnode1} + $ORDS --config ${CONFIG} config set db.port ${ORACLE_PORT:-1521} + $ORDS --config ${CONFIG} config set db.servicename ${ORACLE_SERVICE:-TESTORDS} +} + +[[ -z "${ORACLE_HOST}" && ! -z "${DBTNSURL}" ]] && { + #$ORDS --config ${CONFIG} config set db.tnsAliasName ${TNSALIAS} + #$ORDS --config ${CONFIG} config set db.tnsDirectory ${TNS_ADMIN} + #$ORDS --config ${CONFIG} config set db.connectionType tns + + $ORDS --config ${CONFIG} config set db.connectionType customurl + $ORDS --config ${CONFIG} config set db.customURL jdbc:oracle:thin:@${DBTNSURL} +} + $ORDS --config ${CONFIG} config set security.requestValidationFunction false $ORDS --config ${CONFIG} config set jdbc.MaxLimit 100 $ORDS --config ${CONFIG} config set jdbc.InitialLimit 50 @@ -42,7 +62,7 @@ function SetParameter() { $ORDS --config ${CONFIG} config set restEnabledSql.active true $ORDS --config ${CONFIG} config set security.verifySSL true $ORDS --config ${CONFIG} config set database.api.enabled true - $ORDS --config ${CONFIG} config set plsql.gateway.mode false + $ORDS --config ${CONFIG} config set plsql.gateway.mode disabled $ORDS --config ${CONFIG} config set database.api.management.services.disabled false $ORDS --config ${CONFIG} config set misc.pagination.maxRows 1000 $ORDS --config ${CONFIG} config set db.cdb.adminUser "${CDBADMIN_USER:-C##DBAPI_CDB_ADMIN} AS SYSDBA" @@ -146,12 +166,8 @@ export ORDS_LOGS=/tmp setupHTTPS; SetParameter; - $ORDS --config ${CONFIG} install \ --admin-user ${SYSDBA_USER:-"SYS AS SYSDBA"} \ - --db-hostname ${ORACLE_HOST:-racnode1} \ - --db-port ${ORACLE_PORT:-1521} \ - --db-servicename ${ORACLE_SERVICE:-TESTORDS} \ --feature-db-api true \ --feature-rest-enabled-sql true \ --log-folder ${ORDS_LOGS} \ @@ -161,6 +177,8 @@ ${SYSDBA_PASSWORD:-WElcome_12##} ${ORDS_PASSWORD:-WElcome_12##} EOF + + if [ $? -ne 0 ] then echo "Installation error" From ed1770589ff0dd06ed5d5225784e817d21596875 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Fri, 18 Nov 2022 10:53:15 -0500 Subject: [PATCH 509/628] update third-party dependencies --- go.mod | 80 ++++++++++-------- go.sum | 251 +++++++++++++++++++++++++++++++++------------------------ 2 files changed, 191 insertions(+), 140 deletions(-) diff --git a/go.mod b/go.mod index 4e950301..7eeb98a0 100644 --- a/go.mod +++ b/go.mod @@ -4,65 +4,79 @@ go 1.17 require ( github.com/go-logr/logr v1.2.3 - github.com/onsi/ginkgo/v2 v2.1.3 - github.com/onsi/gomega v1.19.0 - github.com/oracle/oci-go-sdk/v64 v64.0.0 + github.com/onsi/ginkgo/v2 v2.5.0 + github.com/onsi/gomega v1.24.1 + go.uber.org/zap v1.21.0 gopkg.in/yaml.v2 v2.4.0 - k8s.io/api v0.23.6 - k8s.io/apimachinery v0.23.6 - k8s.io/client-go v0.23.6 - sigs.k8s.io/controller-runtime v0.11.2 + k8s.io/api v0.25.4 + k8s.io/apimachinery v0.25.4 + k8s.io/client-go v0.25.4 + sigs.k8s.io/controller-runtime v0.13.1 sigs.k8s.io/yaml v1.3.0 ) require ( - cloud.google.com/go/compute v1.2.0 // indirect + github.com/PuerkitoBio/purell v1.1.1 // indirect + github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578 // indirect github.com/beorn7/perks v1.0.1 // indirect github.com/cespare/xxhash/v2 v2.1.2 // indirect github.com/davecgh/go-spew v1.1.1 // indirect - github.com/evanphx/json-patch v5.6.0+incompatible // indirect - github.com/fsnotify/fsnotify v1.5.1 // indirect - github.com/go-logr/zapr v1.2.2 // indirect + github.com/emicklei/go-restful/v3 v3.8.0 // indirect + github.com/evanphx/json-patch/v5 v5.6.0 // indirect + github.com/fsnotify/fsnotify v1.5.4 // indirect + github.com/go-logr/zapr v1.2.3 // indirect + github.com/go-openapi/jsonpointer v0.19.5 // indirect + github.com/go-openapi/jsonreference v0.19.5 // indirect + github.com/go-openapi/swag v0.19.14 // indirect github.com/gofrs/flock v0.8.1 // indirect github.com/gogo/protobuf v1.3.2 // indirect github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect github.com/golang/protobuf v1.5.2 // indirect - github.com/google/go-cmp v0.5.7 // indirect - github.com/google/gofuzz v1.2.0 // indirect - github.com/google/uuid v1.3.0 // indirect - github.com/googleapis/gnostic v0.5.5 // indirect + github.com/google/gnostic v0.5.7-v3refs // indirect + github.com/google/go-cmp v0.5.9 // indirect github.com/imdario/mergo v0.3.12 // indirect + github.com/josharian/intern v1.0.0 // indirect github.com/json-iterator/go v1.1.12 // indirect + github.com/mailru/easyjson v0.7.6 // indirect github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 // indirect github.com/moby/spdystream v0.2.0 // indirect github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect github.com/modern-go/reflect2 v1.0.2 // indirect + github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect github.com/pkg/errors v0.9.1 // indirect - github.com/prometheus/client_golang v1.12.1 // indirect + github.com/prometheus/client_golang v1.12.2 // indirect github.com/prometheus/client_model v0.2.0 // indirect github.com/prometheus/common v0.32.1 // indirect github.com/prometheus/procfs v0.7.3 // indirect github.com/sony/gobreaker v0.5.0 // indirect github.com/spf13/pflag v1.0.5 // indirect - go.uber.org/atomic v1.9.0 // indirect - go.uber.org/multierr v1.7.0 // indirect - go.uber.org/zap v1.21.0 // indirect - golang.org/x/net v0.0.0-20220225172249-27dd8689420f // indirect + github.com/stretchr/objx v0.4.0 // indirect + golang.org/x/net v0.2.0 // indirect golang.org/x/oauth2 v0.0.0-20211104180415-d3ed0bb246c8 // indirect - golang.org/x/sys v0.0.0-20220209214540-3681064d5158 // indirect - golang.org/x/term v0.0.0-20210927222741-03fcf44c2211 // indirect - golang.org/x/text v0.3.7 // indirect - golang.org/x/time v0.0.0-20211116232009-f0f3c7e86c11 // indirect + golang.org/x/sys v0.2.0 // indirect + golang.org/x/term v0.2.0 // indirect + golang.org/x/text v0.4.0 // indirect + golang.org/x/time v0.0.0-20220609170525-579cf78fd858 // indirect gomodules.xyz/jsonpatch/v2 v2.2.0 // indirect google.golang.org/appengine v1.6.7 // indirect - google.golang.org/protobuf v1.27.1 // indirect + google.golang.org/protobuf v1.28.0 // indirect gopkg.in/inf.v0 v0.9.1 // indirect - gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b // indirect - k8s.io/apiextensions-apiserver v0.23.5 // indirect - k8s.io/component-base v0.23.5 // indirect - k8s.io/klog/v2 v2.40.1 // indirect - k8s.io/kube-openapi v0.0.0-20220124234850-424119656bbf // indirect - k8s.io/utils v0.0.0-20220127004650-9b3446523e65 // indirect - sigs.k8s.io/json v0.0.0-20211208200746-9f7c6b3444d2 // indirect - sigs.k8s.io/structured-merge-diff/v4 v4.2.1 // indirect + gopkg.in/yaml.v3 v3.0.1 // indirect + k8s.io/apiextensions-apiserver v0.25.0 // indirect + k8s.io/component-base v0.25.0 // indirect + k8s.io/klog/v2 v2.70.1 // indirect + k8s.io/kube-openapi v0.0.0-20220803162953-67bda5d908f1 // indirect + k8s.io/utils v0.0.0-20220728103510-ee6ede2d64ed // indirect + sigs.k8s.io/json v0.0.0-20220713155537-f223a00ba0e2 // indirect + sigs.k8s.io/structured-merge-diff/v4 v4.2.3 // indirect +) + +require ( + cloud.google.com/go/compute v1.2.0 // indirect + github.com/evanphx/json-patch v5.6.0+incompatible // indirect + github.com/google/gofuzz v1.2.0 // indirect + github.com/google/uuid v1.3.0 // indirect + github.com/oracle/oci-go-sdk/v64 v64.0.0 + go.uber.org/atomic v1.9.0 // indirect + go.uber.org/multierr v1.7.0 // indirect ) diff --git a/go.sum b/go.sum index 62d4fc56..f694646d 100644 --- a/go.sum +++ b/go.sum @@ -50,13 +50,14 @@ cloud.google.com/go/storage v1.6.0/go.mod h1:N7U0C8pVQ/+NIKOBQyamJIeKQKkZ+mxpohl cloud.google.com/go/storage v1.8.0/go.mod h1:Wv1Oy7z6Yz3DshWRJFhqM/UCfaWIRTdp0RXyy7KQOVs= cloud.google.com/go/storage v1.10.0/go.mod h1:FLPqc6j+Ki4BU591ie1oL6qBQGu2Bl/tZ9ullr3+Kg0= dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU= -github.com/Azure/go-ansiterm v0.0.0-20210608223527-2377c96fe795/go.mod h1:LmzpDX56iTiv29bbRTIsUNlaFfuhWRQBWjQdVyAevI8= github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1/go.mod h1:xomTg63KZ2rFqZQzSB4Vz2SUXa1BpHTVz9L5PTmPC4E= github.com/Azure/go-autorest v14.2.0+incompatible/go.mod h1:r+4oMnoxhatjLLJ6zxSWATqVooLgysK6ZNox3g/xq24= -github.com/Azure/go-autorest/autorest v0.11.18/go.mod h1:dSiJPy22c3u0OtOKDNttNgqpNFY/GeWa7GH/Pz56QRA= -github.com/Azure/go-autorest/autorest/adal v0.9.13/go.mod h1:W/MM4U6nLxnIskrw4UwWzlHfGjwUS50aOsc/I3yuU8M= +github.com/Azure/go-autorest/autorest v0.11.27/go.mod h1:7l8ybrIdUmGqZMTD0sRtAr8NvbHjfofbf8RSP2q7w7U= +github.com/Azure/go-autorest/autorest/adal v0.9.18/go.mod h1:XVVeme+LZwABT8K5Lc3hA4nAe8LDBVle26gTrguhhPQ= +github.com/Azure/go-autorest/autorest/adal v0.9.20/go.mod h1:XVVeme+LZwABT8K5Lc3hA4nAe8LDBVle26gTrguhhPQ= github.com/Azure/go-autorest/autorest/date v0.3.0/go.mod h1:BI0uouVdmngYNUzGWeSYnokU+TrmwEsOqdt8Y6sso74= github.com/Azure/go-autorest/autorest/mocks v0.4.1/go.mod h1:LTp+uSrOhSkaKrUy935gNZuuIPPVsHlr9DSOxSayd+k= +github.com/Azure/go-autorest/autorest/mocks v0.4.2/go.mod h1:Vy7OitM9Kei0i1Oj+LvyAWMXJHeKH1MVlzFugfVrmyU= github.com/Azure/go-autorest/logger v0.2.1/go.mod h1:T9E3cAhj2VqvPOtCYAvby9aBXkZmbF5NWuPV8+WeEW8= github.com/Azure/go-autorest/tracing v0.6.0/go.mod h1:+vhtPC754Xsa23ID7GlGsrdKBpUA79WCAKPPZVC2DeU= github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU= @@ -64,7 +65,9 @@ github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym github.com/NYTimes/gziphandler v0.0.0-20170623195520-56545f4a5d46/go.mod h1:3wb06e3pkSAbeQ52E9H9iFoQsEEwGN64994WTCIhntQ= github.com/NYTimes/gziphandler v1.1.1/go.mod h1:n/CVRwUEOgIxrgPvAQhUUr9oeUtvrhMomdKFjzJNB0c= github.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAElWljhcU= +github.com/PuerkitoBio/purell v1.1.1 h1:WEQqlqaGbrPkxLJWfBwQmfEAE1Z7ONdDLqrN38tNFfI= github.com/PuerkitoBio/purell v1.1.1/go.mod h1:c11w/QuzBsJSee3cPx9rAFu61PvFxuPbtSwDGJws/X0= +github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578 h1:d+Bc7a5rLufV/sSk/8dngufqelfh6jnri85riMAaF/M= github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578/go.mod h1:uGdkoq3SwY9Y+13GIhn11/XLaGBb4BfwItxLd5jeuXE= github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc= github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc= @@ -72,10 +75,12 @@ github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRF github.com/alecthomas/units v0.0.0-20190717042225-c3de453c63f4/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0= github.com/alecthomas/units v0.0.0-20190924025748-f65c72e2690d/go.mod h1:rBZYJk541a8SKzHPHnH3zbiI+7dagKZ0cgpgrD7Fyho= github.com/antihax/optional v1.0.0/go.mod h1:uupD/76wgC+ih3iEmQUL+0Ugr19nfwCT1kdvxnR2qWY= -github.com/antlr/antlr4/runtime/Go/antlr v0.0.0-20210826220005-b48c857c3a0e/go.mod h1:F7bn7fEU90QkQ3tnmaTx3LTKLEDqnwWODIYppRQ5hnY= +github.com/antlr/antlr4/runtime/Go/antlr v0.0.0-20220418222510-f25a4f6275ed/go.mod h1:F7bn7fEU90QkQ3tnmaTx3LTKLEDqnwWODIYppRQ5hnY= github.com/armon/circbuf v0.0.0-20150827004946-bbbad097214e/go.mod h1:3U/XgcO3hCbHZ8TKRvWD2dDTCfh9M9ya+I9JpbB7O8o= github.com/armon/go-metrics v0.0.0-20180917152333-f0300d1749da/go.mod h1:Q73ZrmVTwzkszR9V5SSuryQ31EELlFMUz1kKyl939pY= github.com/armon/go-radix v0.0.0-20180808171621-7fddfc383310/go.mod h1:ufUuZ+zHj4x4TnLV4JWEpy2hxWSpsRywHrMgIH9cCH8= +github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 h1:0CwZNZbxp69SHPdPJAN/hZIm0C4OItdklCFmMRWYpio= +github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5/go.mod h1:wHh0iHkYZB8zMSxRWpUBQtwG5a7fFgvEO+odwuTv2gs= github.com/asaskevich/govalidator v0.0.0-20190424111038-f61b66f89f4a/go.mod h1:lB+ZfQJz7igIIfQNfa7Ml4HSf2uFQQRzpGGRXenZAgY= github.com/benbjohnson/clock v1.0.3/go.mod h1:bGMdMPoPVvcYyt1gHDf4J2KE153Yf9BuiUKYMaxlTDM= github.com/benbjohnson/clock v1.1.0 h1:Q92kusRqC1XV2MjkWETPvjJVqKetz1OzxZB7mHJLju8= @@ -86,8 +91,7 @@ github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= github.com/bgentry/speakeasy v0.1.0/go.mod h1:+zsyZBPWlz7T6j88CTgSN5bM796AkVf0kBD4zp0CCIs= github.com/bketelsen/crypt v0.0.3-0.20200106085610-5cbc8cc4026c/go.mod h1:MKsuJmJgSg28kpZDP6UIiPt0e0Oz0kqKNGyRaWEPv84= -github.com/bketelsen/crypt v0.0.4/go.mod h1:aI6NrJ0pMGgvZKL1iVgXLnfIFJtfV+bKCoqOes/6LfM= -github.com/blang/semver v3.5.1+incompatible/go.mod h1:kRBLl5iJ+tD4TcOOxsy/0fnwebNt5EWlYSAyrTnjyyk= +github.com/blang/semver/v4 v4.0.0/go.mod h1:IbckMUScFkM3pff0VJDNKRiT6TG/YpiHIM2yvyW5YoQ= github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU= github.com/certifi/gocertifi v0.0.0-20191021191039-0944d244cd40/go.mod h1:sGbDF6GwGcLpkNXPUTkMRoywsNa/ol15pxFe6ERfguA= github.com/certifi/gocertifi v0.0.0-20200922220541-2c3bb06c6054/go.mod h1:sGbDF6GwGcLpkNXPUTkMRoywsNa/ol15pxFe6ERfguA= @@ -103,7 +107,11 @@ github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDk github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc= github.com/cncf/udpa/go v0.0.0-20200629203442-efcf912fb354/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk= github.com/cncf/udpa/go v0.0.0-20201120205902-5459f2c99403/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk= +github.com/cncf/udpa/go v0.0.0-20210930031921-04548b0d99d4/go.mod h1:6pvJx4me5XPnfI9Z40ddWsdw2W/uZgQLFXToKeRcDiI= github.com/cncf/xds/go v0.0.0-20210312221358-fbca930ec8ed/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs= +github.com/cncf/xds/go v0.0.0-20210922020428-25de7278fc84/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs= +github.com/cncf/xds/go v0.0.0-20211001041855-01bcc9b48dfe/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs= +github.com/cncf/xds/go v0.0.0-20211011173535-cb28da3451f1/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs= github.com/cockroachdb/datadriven v0.0.0-20200714090401-bf6692d28da5/go.mod h1:h6jFvWxBdQXxjopDMZyH2UVceIRfR84bdzbkoKrsWNo= github.com/cockroachdb/errors v1.2.4/go.mod h1:rQD95gz6FARkaKkQXUksEje/d9a6wBJoCr5oaCLELYA= github.com/cockroachdb/logtags v0.0.0-20190617123548-eb05cc24525f/go.mod h1:i/u985jwjWRlyHXQbwatDASoW0RMlZ/3i9yJHE2xLkI= @@ -115,6 +123,7 @@ github.com/coreos/go-systemd v0.0.0-20190321100706-95778dfbb74e/go.mod h1:F5haX7 github.com/coreos/go-systemd/v22 v22.3.2/go.mod h1:Y58oyj3AT4RCenI/lSvhwexgC+NSVTIJ3seZv2GcEnc= github.com/coreos/pkg v0.0.0-20180928190104-399ea9e2e55f/go.mod h1:E3G3o1h8I7cfcXa63jLwjI0eiQQMgzzUDFVpN/nH/eA= github.com/cpuguy83/go-md2man/v2 v2.0.0/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU= +github.com/cpuguy83/go-md2man/v2 v2.0.1/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o= github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= github.com/creack/pty v1.1.11/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= @@ -126,8 +135,8 @@ github.com/docopt/docopt-go v0.0.0-20180111231733-ee0de3bc6815/go.mod h1:WwZ+bS3 github.com/dustin/go-humanize v1.0.0/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk= github.com/elazarl/goproxy v0.0.0-20180725130230-947c36da3153 h1:yUdfgN0XgIJw7foRItutHYUIhlcKzcSf5vDpdhQAKTc= github.com/elazarl/goproxy v0.0.0-20180725130230-947c36da3153/go.mod h1:/Zj4wYkgs4iZTTu3o/KG3Itv/qCCa8VVMlb3i9OVuzc= -github.com/emicklei/go-restful v0.0.0-20170410110728-ff4f55a20633/go.mod h1:otzb+WCGbkyDHkqmQmT5YD2WR4BBwUdeQoFo8l/7tVs= -github.com/emicklei/go-restful v2.9.5+incompatible/go.mod h1:otzb+WCGbkyDHkqmQmT5YD2WR4BBwUdeQoFo8l/7tVs= +github.com/emicklei/go-restful/v3 v3.8.0 h1:eCZ8ulSerjdAiaNpF7GxXIE7ZCMo1moN1qX+S609eVw= +github.com/emicklei/go-restful/v3 v3.8.0/go.mod h1:6n3XBCmQQb25CM2LCACGz8ukIrRry+4bhvbpWn3mrbc= github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4= github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4= github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98= @@ -135,19 +144,21 @@ github.com/envoyproxy/go-control-plane v0.9.7/go.mod h1:cwu0lG7PUMfa9snN8LXBig5y github.com/envoyproxy/go-control-plane v0.9.9-0.20201210154907-fd9021fe5dad/go.mod h1:cXg6YxExXjJnVBQHBLXeUAgxn2UodCpnH306RInaBQk= github.com/envoyproxy/go-control-plane v0.9.9-0.20210217033140-668b12f5399d/go.mod h1:cXg6YxExXjJnVBQHBLXeUAgxn2UodCpnH306RInaBQk= github.com/envoyproxy/go-control-plane v0.9.9-0.20210512163311-63b5d3c536b0/go.mod h1:hliV/p42l8fGbc6Y9bQ70uLwIvmJyVE5k4iMKlh8wCQ= +github.com/envoyproxy/go-control-plane v0.10.2-0.20220325020618-49ff273808a1/go.mod h1:KJwIaB5Mv44NWtYuAOFCVOjcI94vtpEz2JU/D2v6IjE= github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c= github.com/evanphx/json-patch v0.5.2/go.mod h1:ZWS5hhDbVDyob71nXKNL0+PWn6ToqBHMikGIFbs31qQ= github.com/evanphx/json-patch v4.12.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk= github.com/evanphx/json-patch v5.6.0+incompatible h1:jBYDEEiFBPxA0v50tFdvOzQQTCvpL6mnFh5mB2/l16U= github.com/evanphx/json-patch v5.6.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk= +github.com/evanphx/json-patch/v5 v5.6.0 h1:b91NhWfaz02IuVxO9faSllyAtNXHMPkC5J8sJCLunww= +github.com/evanphx/json-patch/v5 v5.6.0/go.mod h1:G79N1coSVB93tBe7j6PhzjmR3/2VvlbKOFpnXhI9Bw4= github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4= github.com/felixge/httpsnoop v1.0.1/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U= -github.com/form3tech-oss/jwt-go v3.2.2+incompatible/go.mod h1:pbq4aXjuKjdthFRnoDwaVPLA+WlJuPGy+QneDUgJi2k= github.com/form3tech-oss/jwt-go v3.2.3+incompatible/go.mod h1:pbq4aXjuKjdthFRnoDwaVPLA+WlJuPGy+QneDUgJi2k= github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo= github.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ= -github.com/fsnotify/fsnotify v1.5.1 h1:mZcQUHVQUQWoPXXtuf9yuEXKudkV2sx1E06UadKWpgI= -github.com/fsnotify/fsnotify v1.5.1/go.mod h1:T3375wBYaZdLLcVNkcVbzGHY7f1l/uK5T5Ai1i3InKU= +github.com/fsnotify/fsnotify v1.5.4 h1:jRbGcIw6P2Meqdwuo0H1p6JVLbL5DHKAKlYndzMwVZI= +github.com/fsnotify/fsnotify v1.5.4/go.mod h1:OVB6XrOHzAwXMpEM7uPOzcehqUV2UqJxmVXmkdnm1bU= github.com/getkin/kin-openapi v0.76.0/go.mod h1:660oXbgy5JFMKreazJaQTw7o+X00qeSyhcnluiMv+Xg= github.com/getsentry/raven-go v0.2.0/go.mod h1:KungGk8q33+aIAZUIVWZDr2OfAEBsO49PX4NzFV5kcQ= github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04= @@ -166,14 +177,16 @@ github.com/go-logr/logr v1.2.0/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbV github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A= github.com/go-logr/logr v1.2.3 h1:2DntVwHkVopvECVRSlL5PSo9eG+cAkDCuckLubN+rq0= github.com/go-logr/logr v1.2.3/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A= -github.com/go-logr/zapr v1.2.0/go.mod h1:Qa4Bsj2Vb+FAVeAKsLD8RLQ+YRJB8YDmOAKxaBQf7Ro= -github.com/go-logr/zapr v1.2.2 h1:5YNlIL6oZLydaV4dOFjL8YpgXF/tPeTbnpatnu3cq6o= -github.com/go-logr/zapr v1.2.2/go.mod h1:eIauM6P8qSvTw5o2ez6UEAfGjQKrxQTl5EoK+Qa2oG4= +github.com/go-logr/zapr v1.2.3 h1:a9vnzlIBPQBBkeaR9IuMUfmVOrQlkoC4YfPoFkX3T7A= +github.com/go-logr/zapr v1.2.3/go.mod h1:eIauM6P8qSvTw5o2ez6UEAfGjQKrxQTl5EoK+Qa2oG4= github.com/go-openapi/jsonpointer v0.19.3/go.mod h1:Pl9vOtqEWErmShwVjC8pYs9cog34VGT37dQOVbmoatg= +github.com/go-openapi/jsonpointer v0.19.5 h1:gZr+CIYByUqjcgeLXnQu2gHYQC9o73G2XUeOFYEICuY= github.com/go-openapi/jsonpointer v0.19.5/go.mod h1:Pl9vOtqEWErmShwVjC8pYs9cog34VGT37dQOVbmoatg= github.com/go-openapi/jsonreference v0.19.3/go.mod h1:rjx6GuL8TTa9VaixXglHmQmIL98+wF9xc8zWvFonSJ8= +github.com/go-openapi/jsonreference v0.19.5 h1:1WJP/wi4OjB4iV8KVbH73rQaoialJrqv8gitZLxGLtM= github.com/go-openapi/jsonreference v0.19.5/go.mod h1:RdybgQwPxbL4UEjuAruzK1x3nE69AqPYEJeo/TWfEeg= github.com/go-openapi/swag v0.19.5/go.mod h1:POnQmlKehdgb5mhVOsnJFsivZCEZ/vjK9gh66Z9tfKk= +github.com/go-openapi/swag v0.19.14 h1:gm3vOOXfiuw5i9p5N9xJvfjvuofpyvLA9Wr6QfK5Fng= github.com/go-openapi/swag v0.19.14/go.mod h1:QYRuS/SOXUCsnplDa677K7+DxSOj6IPNl/eQntq43wQ= github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY= github.com/go-task/slim-sprig v0.0.0-20210107165309-348f09dbbbc0/go.mod h1:fyg7847qk6SyHyPtNmDHnmrv/HOrqktSC+C9fM+CJOE= @@ -185,6 +198,8 @@ github.com/gogo/protobuf v1.2.1/go.mod h1:hp+jE20tsWTFYpLwKvXlhS1hjn+gTNwPg2I6zV github.com/gogo/protobuf v1.3.1/go.mod h1:SlYgWuQ5SjCEi6WLHjHCa1yvBfUnHcTbrrZtXPKa29o= github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q= github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q= +github.com/golang-jwt/jwt/v4 v4.0.0/go.mod h1:/xlHOz8bRuivTWchD4jCa+NbatV+wEUSzwAxVc6locg= +github.com/golang-jwt/jwt/v4 v4.2.0/go.mod h1:/xlHOz8bRuivTWchD4jCa+NbatV+wEUSzwAxVc6locg= github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q= github.com/golang/glog v1.0.0/go.mod h1:EWib/APOK0SL3dFbYqvxE3UYd8E6s1ouQ7iEp/0LWV4= github.com/golang/groupcache v0.0.0-20190129154638-5b532d6fd5ef/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= @@ -224,8 +239,9 @@ github.com/golang/snappy v0.0.3/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEW github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= github.com/google/btree v1.0.1/go.mod h1:xXMiIv4Fb/0kKde4SpL7qlzvu5cMJDRkFDxJfI9uaxA= -github.com/google/cel-go v0.9.0/go.mod h1:U7ayypeSkw23szu4GaQTPJGx66c20mx8JklMSxrmI1w= -github.com/google/cel-spec v0.6.0/go.mod h1:Nwjgxy5CbjlPrtCWjeDjUyKMl8w41YBYGjsyDdqk0xA= +github.com/google/cel-go v0.12.4/go.mod h1:Av7CU6r6X3YmcHR9GXqVDaEJYfEtSxl6wvIjUQTriCw= +github.com/google/gnostic v0.5.7-v3refs h1:FhTMOKj2VhjpouxvWJAV1TL304uMlb9zcDqkl6cEI54= +github.com/google/gnostic v0.5.7-v3refs/go.mod h1:73MKFl6jIHelAJNaBGFzt3SPtZULs9dYrGFt8OiIsHQ= github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M= github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= @@ -238,8 +254,10 @@ github.com/google/go-cmp v0.5.3/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/ github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.6/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= -github.com/google/go-cmp v0.5.7 h1:81/ik6ipDQS2aGcBfIN5dHDB36BwrStyeAQquSYCV4o= github.com/google/go-cmp v0.5.7/go.mod h1:n+brtR0CgQNWTVd5ZUFpTBC8YFBDLK/h/bpaJ8/DtOE= +github.com/google/go-cmp v0.5.8/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= +github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38= +github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= github.com/google/gofuzz v1.1.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= github.com/google/gofuzz v1.2.0 h1:xRy4A+RhZaiKjJ1bPfwQ8sedCA+YS2YcCHW6ec7JMi0= @@ -271,9 +289,6 @@ github.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+ github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk= github.com/googleapis/gax-go/v2 v2.1.0/go.mod h1:Q3nei7sK6ybPYH7twZdmQpAd1MKb7pfu6SK+H1/DsU0= github.com/googleapis/gax-go/v2 v2.1.1/go.mod h1:hddJymUZASv3XPyGkUpKj8pPO47Rmb0eJc8R6ouapiM= -github.com/googleapis/gnostic v0.5.1/go.mod h1:6U4PtQXGIEt/Z3h5MAT7FNofLnw9vXk2cUuW7uA/OeU= -github.com/googleapis/gnostic v0.5.5 h1:9fHAtK0uDfpveeqqo1hkEZJcFvYXAiCN3UutL8F9xHw= -github.com/googleapis/gnostic v0.5.5/go.mod h1:7+EbHbldMins07ALC74bsA81Ovc97DwqyJO1AENw9kA= github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY= github.com/gorilla/mux v1.8.0/go.mod h1:DVbg23sWSpFRCP0SfiEN6jmj59UnW/n46BH5rLB71So= github.com/gorilla/websocket v1.4.2/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE= @@ -306,13 +321,14 @@ github.com/hashicorp/serf v0.8.2/go.mod h1:6hOLApaqBFA1NXqRQAsxw9QxuDEvNxSQRwA/J github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU= github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc= github.com/ianlancetaylor/demangle v0.0.0-20200824232613-28f6c0f3b639/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc= -github.com/imdario/mergo v0.3.5/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA= +github.com/imdario/mergo v0.3.6/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA= github.com/imdario/mergo v0.3.12 h1:b6R2BslTbIEToALKP7LxUvijTsNI9TAe80pLWN2g/HU= github.com/imdario/mergo v0.3.12/go.mod h1:jmQim1M+e3UYxmgPu/WyfjB3N3VflVyUjjjwH0dnCYA= github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8= github.com/jessevdk/go-flags v1.4.0/go.mod h1:4FA24M0QyGHXBuZZK/XkWh8h0e1EYbRYJSGM75WSRxI= github.com/jonboulle/clockwork v0.1.0/go.mod h1:Ii8DK3G1RaLaWxj9trq07+26W01tbo22gdxWY5EU2bo= github.com/jonboulle/clockwork v0.2.2/go.mod h1:Pkfl5aHPm1nk2H9h0bjmnJD/BcgbGXUBGnn1kMkgxc8= +github.com/josharian/intern v1.0.0 h1:vlS4z54oSdjm0bgjRigI+G1HpF+tI+9rE5LLzOg8HmY= github.com/josharian/intern v1.0.0/go.mod h1:5DoeVV0s6jJacbCEi61lwdGj/aVlrQvzHFFd8Hwg//Y= github.com/jpillora/backoff v1.0.0/go.mod h1:J/6gKK9jxlEcS3zixgDgUAsiuZ7yrSoa/FX5e0EB2j4= github.com/json-iterator/go v1.1.6/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU= @@ -340,9 +356,9 @@ github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI= github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= github.com/magiconair/properties v1.8.1/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ= -github.com/magiconair/properties v1.8.5/go.mod h1:y3VJvCyxH9uVvJTWEGAELF3aiYNyPKd5NZ3oSwXrF60= github.com/mailru/easyjson v0.0.0-20190614124828-94de47d64c63/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc= github.com/mailru/easyjson v0.0.0-20190626092158-b2ccc519800e/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc= +github.com/mailru/easyjson v0.7.6 h1:8yTIVnZgCoiM1TgqoeTl+LfU5Jg6/xL3QhGQnimLYnA= github.com/mailru/easyjson v0.7.6/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc= github.com/mattn/go-colorable v0.0.9/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaOChaDxuIBZU= github.com/mattn/go-isatty v0.0.3/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4= @@ -361,7 +377,7 @@ github.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh github.com/mitchellh/mapstructure v1.4.1/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo= github.com/moby/spdystream v0.2.0 h1:cjW1zVyyoiM0T7b6UoySUFqzXMoqRckQtXwGPiBhOM8= github.com/moby/spdystream v0.2.0/go.mod h1:f7i0iNDQJ059oMTcWxx8MA/zKFIuD/lY+0GqbN2Wy8c= -github.com/moby/term v0.0.0-20210610120745-9d4ed1856297/go.mod h1:vgPCkQMyxTZ7IDy8SXRufE172gr8+K/JE/7hHFxHW3A= +github.com/moby/term v0.0.0-20210619224110-3f7ff695adc6/go.mod h1:E2VnQOmVuvZB6UYnnDB0qG5Nq/1tD9acaOpo6xmt0Kw= github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg= github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= @@ -370,6 +386,7 @@ github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3Rllmb github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M= github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk= github.com/munnerz/goautoneg v0.0.0-20120707110453-a547fc61f48d/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ= +github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA= github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ= github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U= github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U= @@ -380,27 +397,33 @@ github.com/nxadm/tail v1.4.4/go.mod h1:kenIhsEOeOJmVchQTgglprH7qJGnHDVpk1VPCcaMI github.com/nxadm/tail v1.4.8 h1:nPr65rt6Y5JFSKQO7qToXr7pePgD6Gwiw05lkbyAQTE= github.com/nxadm/tail v1.4.8/go.mod h1:+ncqLTQzXmGhMZNUePPaPqPvBxHAIsmXswZKocGu+AU= github.com/oklog/ulid v1.3.1/go.mod h1:CirwcVhetQ6Lv90oh/F+FBtV6XMibvdAFo93nm5qn4U= -github.com/onsi/ginkgo v0.0.0-20170829012221-11459a886d9c/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE= github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE= github.com/onsi/ginkgo v1.12.1/go.mod h1:zj2OWP4+oCPe1qIXoGWkgMRwljMUYCdkwsT2108oapk= -github.com/onsi/ginkgo v1.14.0/go.mod h1:iSB4RoI2tjJc9BBv4NKIKWKya62Rps+oPG/Lv9klQyY= github.com/onsi/ginkgo v1.16.4/go.mod h1:dX+/inL/fNMqNlz0e9LfyB9TswhZpCVdJM/Z6Vvnwo0= github.com/onsi/ginkgo v1.16.5 h1:8xi0RTUf59SOSfEtZMvwTvXYMzG4gV23XVHOZiXNtnE= github.com/onsi/ginkgo v1.16.5/go.mod h1:+E8gABHa3K6zRBolWtd+ROzc/U5bkGt0FwiG042wbpU= -github.com/onsi/ginkgo/v2 v2.1.3 h1:e/3Cwtogj0HA+25nMP1jCMDIf8RtRYbGwGGuBIFztkc= github.com/onsi/ginkgo/v2 v2.1.3/go.mod h1:vw5CSIxN1JObi/U8gcbwft7ZxR2dgaR70JSE3/PpL4c= -github.com/onsi/gomega v0.0.0-20170829124025-dcabb60a477c/go.mod h1:C1qb7wdrVGGVU+Z6iS04AVkA3Q65CEZX59MT0QO5uiA= +github.com/onsi/ginkgo/v2 v2.1.4/go.mod h1:um6tUpWM/cxCK3/FK8BXqEiUMUwRgSM4JXG47RKZmLU= +github.com/onsi/ginkgo/v2 v2.1.6/go.mod h1:MEH45j8TBi6u9BMogfbp0stKC5cdGjumZj5Y7AG4VIk= +github.com/onsi/ginkgo/v2 v2.3.0/go.mod h1:Eew0uilEqZmIEZr8JrvYlvOM7Rr6xzTmMV8AyFNU9d0= +github.com/onsi/ginkgo/v2 v2.4.0/go.mod h1:iHkDK1fKGcBoEHT5W7YBq4RFWaQulw+caOMkAt4OrFo= +github.com/onsi/ginkgo/v2 v2.5.0 h1:TRtrvv2vdQqzkwrQ1ke6vtXf7IK34RBUJafIy1wMwls= +github.com/onsi/ginkgo/v2 v2.5.0/go.mod h1:Luc4sArBICYCS8THh8v3i3i5CuSZO+RaQRaJoeNwomw= github.com/onsi/gomega v1.7.1/go.mod h1:XdKZgCCFLUoM/7CFJVPcG8C1xQ1AJ0vpAezJrB7JYyY= github.com/onsi/gomega v1.10.1/go.mod h1:iN09h71vgCQne3DLsj+A5owkum+a2tYe+TOCB1ybHNo= github.com/onsi/gomega v1.17.0/go.mod h1:HnhC7FXeEQY45zxNK3PPoIUhzk/80Xly9PcubAlGdZY= -github.com/onsi/gomega v1.19.0 h1:4ieX6qQjPP/BfC3mpsAtIGGlxTWPeA3Inl/7DtXw1tw= github.com/onsi/gomega v1.19.0/go.mod h1:LY+I3pBVzYsTBU1AnDwOSxaYi9WoWiqgwooUqq9yPro= +github.com/onsi/gomega v1.20.1/go.mod h1:DtrZpjmvpn2mPm4YWQa0/ALMDj9v4YxLgojwPeREyVo= +github.com/onsi/gomega v1.21.1/go.mod h1:iYAIXgPSaDHak0LCMA+AWBpIKBr8WZicMxnE8luStNc= +github.com/onsi/gomega v1.22.1/go.mod h1:x6n7VNe4hw0vkyYUM4mjIXx3JbLiPaBPNgB7PRQ1tuM= +github.com/onsi/gomega v1.24.0/go.mod h1:Z/NWtiqwBrwUt4/2loMmHL63EDLnYHmVbuBpDr2vQAg= +github.com/onsi/gomega v1.24.1 h1:KORJXNNTzJXzu4ScJWssJfJMnJ+2QJqhoQSRwNlze9E= +github.com/onsi/gomega v1.24.1/go.mod h1:3AOiACssS3/MajrniINInwbfOOtfZvplPzuRSmvt1jM= github.com/opentracing/opentracing-go v1.1.0/go.mod h1:UkNAQd3GIcIGf0SeVgPpRdFStlNbqXla1AfSYxPUl2o= github.com/oracle/oci-go-sdk/v64 v64.0.0 h1:ALQDxoKLeNfxLbbOkjnNxMf3/AyWhnHhMJxc8tZXRgI= github.com/oracle/oci-go-sdk/v64 v64.0.0/go.mod h1:eLqTg9MSXE/9naair0yFKqiP1RlYghuzBDvEgZvwjYs= github.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc= github.com/pelletier/go-toml v1.2.0/go.mod h1:5z9KED0ma1S8pY6P1sdut58dfprrGBbd/94hg7ilaic= -github.com/pelletier/go-toml v1.9.3/go.mod h1:u1nR/EPcESfeI/szUZKdtJ0xRNbUoANCkoOuaOx1Y+c= github.com/peterbourgon/diskv v2.0.1+incompatible/go.mod h1:uqqh8zWWbv1HBMNONnaR/tNboyR3/BZd58JJSHlUSCU= github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= @@ -410,14 +433,16 @@ github.com/pkg/sftp v1.10.1/go.mod h1:lYOWFsE0bwd1+KfKJaKeuokY15vzFx25BLbzYYoAxZ github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/posener/complete v1.1.1/go.mod h1:em0nMJCgc9GFtwrmVmEMR/ZL6WyhyjMBndrE9hABlRI= -github.com/pquerna/cachecontrol v0.0.0-20171018203845-0dec1b30a021/go.mod h1:prYjPmNq4d1NPVmpShWobRqXY3q7Vp+80DqgxxUrUIA= +github.com/pquerna/cachecontrol v0.1.0/go.mod h1:NrUG3Z7Rdu85UNR3vm7SOsl1nFIeSiQnrHV5K9mBcUI= github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw= github.com/prometheus/client_golang v0.9.3/go.mod h1:/TN21ttK/J9q6uSwhBd54HahCDft0ttaMvbicHlPoso= github.com/prometheus/client_golang v1.0.0/go.mod h1:db9x61etRT2tGnBNRi70OPL5FsnadC4Ky3P0J6CfImo= github.com/prometheus/client_golang v1.7.1/go.mod h1:PY5Wy2awLA44sXw4AOSfFBetzPP4j5+D6mVACh+pe2M= github.com/prometheus/client_golang v1.11.0/go.mod h1:Z6t4BnS23TR94PD6BsDNk8yVqroYurpAkEiz0P2BEV0= -github.com/prometheus/client_golang v1.12.1 h1:ZiaPsmm9uiBeaSMRznKsCDNtPCS0T3JVDGF+06gjBzk= +github.com/prometheus/client_golang v1.11.1/go.mod h1:Z6t4BnS23TR94PD6BsDNk8yVqroYurpAkEiz0P2BEV0= github.com/prometheus/client_golang v1.12.1/go.mod h1:3Z9XVyYiZYEO+YQWt3RD2R3jrbd179Rt297l4aS6nDY= +github.com/prometheus/client_golang v1.12.2 h1:51L9cDoUHVrXx4zWYlcLQIZ+d+VXHgqnYKkIuq4g/34= +github.com/prometheus/client_golang v1.12.2/go.mod h1:3Z9XVyYiZYEO+YQWt3RD2R3jrbd179Rt297l4aS6nDY= github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo= github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= @@ -428,7 +453,6 @@ github.com/prometheus/common v0.4.0/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y8 github.com/prometheus/common v0.4.1/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4= github.com/prometheus/common v0.10.0/go.mod h1:Tlit/dnDKsSWFlCLTWaA1cyBgKHSMdTB80sz/V91rCo= github.com/prometheus/common v0.26.0/go.mod h1:M7rCNAaPfAosfx8veZJCuw84e35h3Cfd9VFqTh1DIvc= -github.com/prometheus/common v0.28.0/go.mod h1:vu+V0TpY+O6vW9J44gczi3Ap/oXXR10b+M/gUGO4Hls= github.com/prometheus/common v0.32.1 h1:hWIdL3N2HoUx3B8j3YN9mWor0qhY/NlEKZEaXxuIRh4= github.com/prometheus/common v0.32.1/go.mod h1:vu+V0TpY+O6vW9J44gczi3Ap/oXXR10b+M/gUGO4Hls= github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk= @@ -443,6 +467,7 @@ github.com/rogpeppe/fastuuid v0.0.0-20150106093220-6724a57986af/go.mod h1:XWv6So github.com/rogpeppe/fastuuid v1.2.0/go.mod h1:jVj6XXZzXRy/MSR5jhDC/2q6DgLz+nrA6LYCDYWNEvQ= github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4= github.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM= +github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM= github.com/ryanuber/columnize v0.0.0-20160712163229-9b3edd62028f/go.mod h1:sm1tb6uqfes/u+d4ooFouqFdy9/2g9QGwK3SQygK0Ts= github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529/go.mod h1:DxrIzT+xaE7yg65j358z/aeFdxmN0P9QXhEzd20vsDc= github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc= @@ -462,27 +487,27 @@ github.com/spf13/afero v1.1.2/go.mod h1:j4pytiNVoe2o6bmDsKpLACNPDBIoEAkihy7loJ1B github.com/spf13/afero v1.2.2/go.mod h1:9ZxEEn6pIJ8Rxe320qSDBk6AsU0r9pR7Q4OcevTdifk= github.com/spf13/afero v1.6.0/go.mod h1:Ai8FlHk4v/PARR026UzYexafAt9roJ7LcLMAmO6Z93I= github.com/spf13/cast v1.3.0/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE= -github.com/spf13/cast v1.3.1/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE= github.com/spf13/cobra v1.1.3/go.mod h1:pGADOWyqRD/YMrPZigI/zbliZ2wVD/23d+is3pSWzOo= -github.com/spf13/cobra v1.2.1/go.mod h1:ExllRjgxM/piMAM+3tAZvg8fsklGAf3tPfi+i8t68Nk= +github.com/spf13/cobra v1.4.0/go.mod h1:Wo4iy3BUC+X2Fybo0PDqwJIv3dNRiZLHQymsfxlB84g= github.com/spf13/jwalterweatherman v1.0.0/go.mod h1:cQK4TGJAtQXfYWX+Ddv3mKDzgVb68N+wFjFa4jdeBTo= -github.com/spf13/jwalterweatherman v1.1.0/go.mod h1:aNWZUN0dPAAO/Ljvb5BEdw96iTZ0EXowPYD95IqWIGo= github.com/spf13/pflag v1.0.3/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4= github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA= github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= github.com/spf13/viper v1.7.0/go.mod h1:8WkrPz2fc9jxqZNCJI/76HCieCp4Q8HaLFoCha5qpdg= -github.com/spf13/viper v1.8.1/go.mod h1:o0Pch8wJ9BVSWGQMbra6iw0oQ5oktSIBaujf1rJH9Ns= github.com/stoewer/go-strcase v1.2.0/go.mod h1:IBiWB2sKIp3wVVQ3Y035++gc+knqhUQag1KpM8ahLw8= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= -github.com/stretchr/objx v0.1.1 h1:2vfRuCMp5sSVIDSqO8oNnWJq7mPa6KVP3iPIwFBuy8A= github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= +github.com/stretchr/objx v0.4.0 h1:M2gUjqZET1qApGOWNSnZ49BAIMX4F/1plDv3+l31EJ4= +github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw= github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs= github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4= github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA= github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= -github.com/stretchr/testify v1.7.0 h1:nwc3DEeHmmLAfoZucVR881uASk0Mfjw8xYJ99tb5CcY= github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= +github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= +github.com/stretchr/testify v1.8.0 h1:pSgiaMZlXftHpm5L7V1+rVB+AZJydKsMxsQBIJw4PKk= +github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU= github.com/subosito/gotenv v1.2.0/go.mod h1:N0PQaV/YGNqwC0u51sEeR/aUtSLEXKX9iv69rRypqCw= github.com/tmc/grpc-websocket-proxy v0.0.0-20190109142713-0ad062ec5ee5/go.mod h1:ncp9v5uamzpCO7NfCPTXjqaC+bZgJeR0sMTm6dMHP7U= github.com/tmc/grpc-websocket-proxy v0.0.0-20201229170055-e5319fda7802/go.mod h1:ncp9v5uamzpCO7NfCPTXjqaC+bZgJeR0sMTm6dMHP7U= @@ -492,16 +517,17 @@ github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9de github.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.3.5/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k= -github.com/yuin/goldmark v1.4.0/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k= +github.com/yuin/goldmark v1.4.1/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k= +github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY= go.etcd.io/bbolt v1.3.2/go.mod h1:IbVyRI1SCnLcuJnV2u8VeU0CEYM7e686BmAb1XKL+uU= go.etcd.io/bbolt v1.3.6/go.mod h1:qXsaaIqmgQH0T+OPdb99Bf+PKfBBQVAdyD6TY9G8XM4= -go.etcd.io/etcd/api/v3 v3.5.0/go.mod h1:cbVKeC6lCfl7j/8jBhAK6aIYO9XOjdptoxU/nLQcPvs= -go.etcd.io/etcd/client/pkg/v3 v3.5.0/go.mod h1:IJHfcCEKxYu1Os13ZdwCwIUTUVGYTSAM3YSwc9/Ac1g= -go.etcd.io/etcd/client/v2 v2.305.0/go.mod h1:h9puh54ZTgAKtEbut2oe9P4L/oqKCVB6xsXlzd7alYQ= -go.etcd.io/etcd/client/v3 v3.5.0/go.mod h1:AIKXXVX/DQXtfTEqBryiLTUXwON+GuvO6Z7lLS/oTh0= -go.etcd.io/etcd/pkg/v3 v3.5.0/go.mod h1:UzJGatBQ1lXChBkQF0AuAtkRQMYnHubxAEYIrC3MSsE= -go.etcd.io/etcd/raft/v3 v3.5.0/go.mod h1:UFOHSIvO/nKwd4lhkwabrTD3cqW5yVyYYf/KlD00Szc= -go.etcd.io/etcd/server/v3 v3.5.0/go.mod h1:3Ah5ruV+M+7RZr0+Y/5mNLwC+eQlni+mQmOVdCRJoS4= +go.etcd.io/etcd/api/v3 v3.5.4/go.mod h1:5GB2vv4A4AOn3yk7MftYGHkUfGtDHnEraIjym4dYz5A= +go.etcd.io/etcd/client/pkg/v3 v3.5.4/go.mod h1:IJHfcCEKxYu1Os13ZdwCwIUTUVGYTSAM3YSwc9/Ac1g= +go.etcd.io/etcd/client/v2 v2.305.4/go.mod h1:Ud+VUwIi9/uQHOMA+4ekToJ12lTxlv0zB/+DHwTGEbU= +go.etcd.io/etcd/client/v3 v3.5.4/go.mod h1:ZaRkVgBZC+L+dLCjTcF1hRXpgZXQPOvnA/Ak/gq3kiY= +go.etcd.io/etcd/pkg/v3 v3.5.4/go.mod h1:OI+TtO+Aa3nhQSppMbwE4ld3uF1/fqqwbpfndbbrEe0= +go.etcd.io/etcd/raft/v3 v3.5.4/go.mod h1:SCuunjYvZFC0fBX0vxMSPjuZmpcSk+XaAcMrD6Do03w= +go.etcd.io/etcd/server/v3 v3.5.4/go.mod h1:S5/YTU15KxymM5l3T6b09sNOHPXqGYIZStpuuGbb65c= go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU= go.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8= go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= @@ -526,7 +552,6 @@ go.uber.org/atomic v1.7.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc= go.uber.org/atomic v1.9.0 h1:ECmE8Bn/WFTYwEW/bpKD3M8VtR/zQVbavAoalC1PYyE= go.uber.org/atomic v1.9.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc= go.uber.org/goleak v1.1.10/go.mod h1:8a7PlsEVH3e/a/GLqe5IIrQx6GzcnRmZEufDUTk4A7A= -go.uber.org/goleak v1.1.11-0.20210813005559-691160354723/go.mod h1:cwTWslyiVhfpKIDGSZEM2HlOvcqm+tG4zioyIeLoqMQ= go.uber.org/goleak v1.1.11/go.mod h1:cwTWslyiVhfpKIDGSZEM2HlOvcqm+tG4zioyIeLoqMQ= go.uber.org/goleak v1.1.12 h1:gZAh5/EyT/HQwlpkCy6wTpqfH9H8Lz8zbm3dZh+OyzA= go.uber.org/goleak v1.1.12/go.mod h1:cwTWslyiVhfpKIDGSZEM2HlOvcqm+tG4zioyIeLoqMQ= @@ -537,7 +562,6 @@ go.uber.org/multierr v1.7.0/go.mod h1:7EAYxJLBy9rStEaz58O2t4Uvip6FSURkq8/ppBp95a go.uber.org/zap v1.10.0/go.mod h1:vwi/ZaCAaUcBkycHslxD9B2zi4UTXhF60s6SWpuDF0Q= go.uber.org/zap v1.17.0/go.mod h1:MXVU+bhUf/A7Xi2HNOnopQOrmycQ5Ih87HtOu4q5SSo= go.uber.org/zap v1.19.0/go.mod h1:xg/QME4nWcxGxrpdeYfq7UvYrLh66cuVKdrbD1XF/NI= -go.uber.org/zap v1.19.1/go.mod h1:j3DNczoxDZroyBnOT1L/Q79cfUMGZxlv/9dzN7SM1rI= go.uber.org/zap v1.21.0 h1:WefMeulhovoZ2sYXz7st6K0sLj7bBhpiFaud4r4zST8= go.uber.org/zap v1.21.0/go.mod h1:wjWOCqI0f2ZZrJF/UufIOkiC8ii6tm1iqIsLo76RfJw= golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= @@ -548,8 +572,11 @@ golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8U golang.org/x/crypto v0.0.0-20190820162420-60c769a6c586/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= -golang.org/x/crypto v0.0.0-20201002170205-7f63de1d35b0/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= -golang.org/x/crypto v0.0.0-20210817164053-32db794688a5/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc= +golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc= +golang.org/x/crypto v0.0.0-20211215153901-e495a2d5b3d3/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4= +golang.org/x/crypto v0.0.0-20220131195533-30dcbda58838/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4= +golang.org/x/crypto v0.0.0-20220315160706-3147a52a75dd/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4= +golang.org/x/crypto v0.1.0/go.mod h1:RecgLatLF4+eUMCP1PoPZQb+cVrJcOPbHkTkbkB9sbw= golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8= @@ -585,6 +612,9 @@ golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.4.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.4.1/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= +golang.org/x/mod v0.6.0-dev.0.20220106191415-9b9b3d81d5e3/go.mod h1:3p9vT2HGsQu2K1YbXdKPJLVgG5VJdoTa1poYQBtP1AY= +golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4= +golang.org/x/mod v0.6.0/go.mod h1:4mET923SAdbXp2ki8ey+zGs1SLqsuM2Y0uvdZR/fUNI= golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= @@ -631,11 +661,14 @@ golang.org/x/net v0.0.0-20210405180319-a5a99cb37ef4/go.mod h1:p54w0d4576C0XHj96b golang.org/x/net v0.0.0-20210428140749-89ef3d95e781/go.mod h1:OJAsFXCWl8Ukc7SiCT/9KSuxbyM7479/AVlXFRxuMCk= golang.org/x/net v0.0.0-20210503060351-7fd8e65b6420/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= golang.org/x/net v0.0.0-20210525063256-abc453219eb5/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= -golang.org/x/net v0.0.0-20210805182204-aaa1db679c0d/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= -golang.org/x/net v0.0.0-20210825183410-e898025ed96a/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= -golang.org/x/net v0.0.0-20211209124913-491a49abca63/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= -golang.org/x/net v0.0.0-20220225172249-27dd8689420f h1:oA4XRj0qtSt8Yo1Zms0CUlsT3KG69V2UGQWPBxujDmc= +golang.org/x/net v0.0.0-20211015210444-4f30a5c0130f/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= +golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= golang.org/x/net v0.0.0-20220225172249-27dd8689420f/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk= +golang.org/x/net v0.0.0-20220425223048-2871e0cb64e4/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk= +golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c= +golang.org/x/net v0.1.0/go.mod h1:Cx3nUiGt4eDBEyega/BKRp+/AlGL8hYe7U9odMt2Cco= +golang.org/x/net v0.2.0 h1:sZfSu1wtKLGlWI4ZZayP0ck9Y73K1ynO6gqzTdBVdPU= +golang.org/x/net v0.2.0/go.mod h1:KqCZLdyyvdV855qA2rE3GC2aiw5xGR5TEjj8smXukLY= golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U= golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= @@ -647,7 +680,6 @@ golang.org/x/oauth2 v0.0.0-20201208152858-08078c50e5b5/go.mod h1:KelEdhl1UZF7XfJ golang.org/x/oauth2 v0.0.0-20210218202405-ba52d332ba99/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= golang.org/x/oauth2 v0.0.0-20210220000619-9bb904979d93/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= golang.org/x/oauth2 v0.0.0-20210313182246-cd4f82c27b84/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= -golang.org/x/oauth2 v0.0.0-20210402161424-2e8d93401602/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= golang.org/x/oauth2 v0.0.0-20210514164344-f6687ab2804c/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= golang.org/x/oauth2 v0.0.0-20210628180205-a41e5a781914/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= golang.org/x/oauth2 v0.0.0-20210805134026-6f1e6394065a/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= @@ -665,6 +697,7 @@ golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208/go.mod h1:RxMgew5VJxzue5/jJ golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= @@ -700,12 +733,10 @@ golang.org/x/sys v0.0.0-20200331124033-c3d80250170d/go.mod h1:h1NjWce9XRLGQEsW7w golang.org/x/sys v0.0.0-20200501052902-10377860bb8e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200511232937-7e40ca221e25/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200515095857-1151b9dac4a9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200519105757-fe76b779f299/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200523222454-059865788121/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200615200032-f1bc736245b1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200625212154-ddb9806d33ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200803210538-64077c9b5642/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200831180312-196b9ba8737a/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200905004654-be1d3432aa8f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200923182605-d9f96fdee20d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= @@ -730,21 +761,27 @@ golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBc golang.org/x/sys v0.0.0-20210616094352-59db8d763f22/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20210806184541-e5e7981a1069/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20210809222454-d867a43fc93e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20210823070655-63515b42dcdf/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20210831042530-f4d43177bf5e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20210908233432-aa78b53d3365/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20211029165221-6e7872819dc8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20211019181941-9d821ace8654/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20211124211545-fe61309f8881/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20211210111614-af8b64212486/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20220114195835-da31bd327af9/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20220209214540-3681064d5158 h1:rm+CHSpPEEW2IsXUib1ThaHIjuBVZjxNgSKmBLFfD4c= golang.org/x/sys v0.0.0-20220209214540-3681064d5158/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20220319134239-a9b59b0215f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20220412211240-33da011f77ad/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20220422013727-9388b58f7150/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.2.0 h1:ljd4t30dBnAvMZaQCevtY0xLLD0A+bRZXbgLMLU1F/A= +golang.org/x/sys v0.2.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= -golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8= -golang.org/x/term v0.0.0-20210927222741-03fcf44c2211 h1:JGgROgKl9N8DuW20oFS5gxc+lE67/N3FcwmBPMe7ArY= golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8= +golang.org/x/term v0.1.0/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8= +golang.org/x/term v0.2.0 h1:z85xZCsEl7bi/KwbNADeBYoOP0++7W1ipu+aGnpwzRM= +golang.org/x/term v0.2.0/go.mod h1:TVmDHMZPmdnySmBfhjOoOdhjzdE1h4u1VwSiw2l1Nuc= golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= @@ -753,15 +790,16 @@ golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.3.5/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= -golang.org/x/text v0.3.7 h1:olpwvP2KacW1ZWvsR7uQhoyTYvKAupfQrRGBFM352Gk= golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ= +golang.org/x/text v0.4.0 h1:BrVqGRd7+k1DiOgtnFvAkoQEWQvBc25ouMJM6429SFg= +golang.org/x/text v0.4.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8= golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/time v0.0.0-20210220033141-f8bda1e9f3ba/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= -golang.org/x/time v0.0.0-20210723032227-1f47c861a9ac/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= -golang.org/x/time v0.0.0-20211116232009-f0f3c7e86c11 h1:GZokNIeuVkl3aZHJchRrr13WCsols02MLUcz1U9is6M= -golang.org/x/time v0.0.0-20211116232009-f0f3c7e86c11/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= +golang.org/x/time v0.0.0-20220210224613-90d013bbcef8/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= +golang.org/x/time v0.0.0-20220609170525-579cf78fd858 h1:Dpdu/EMxGMFgq0CeYMh4fazTD2vtlZRYE7wyynxJb9U= +golang.org/x/time v0.0.0-20220609170525-579cf78fd858/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/tools v0.0.0-20180221164845-07fd8470d635/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20181030221726-6c7e314b6563/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= @@ -823,11 +861,12 @@ golang.org/x/tools v0.1.2/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= golang.org/x/tools v0.1.3/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= golang.org/x/tools v0.1.4/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= golang.org/x/tools v0.1.5/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= -golang.org/x/tools v0.1.6-0.20210820212750-d4cc65f0b2ff/go.mod h1:YD9qOF0M9xpSpdWTBbzEl5e/RnCefISl8E5Noe10jFM= +golang.org/x/tools v0.1.10/go.mod h1:Uh6Zz+xoGYZom868N8YTex3t7RhtHDBrE8Gzo9bV56E= +golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc= +golang.org/x/tools v0.2.0/go.mod h1:y4OqIKeOV/fWJetJ8bXPU1sEVniLMIyDAZWeHdV+NTA= golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= -golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 h1:go1bK/D/BFZV2I8cIQd1NKEZ+0owSTG1fDTci4IqFcE= golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= gomodules.xyz/jsonpatch/v2 v2.2.0 h1:4pT439QV83L+G9FkcCriY6EkpcK6r6bK+A5FBUMI7qY= gomodules.xyz/jsonpatch/v2 v2.2.0/go.mod h1:WXp+iVDkoLQqPudfQ9GBlwB2eZ5DKOnjQZCYdOS8GPY= @@ -852,7 +891,6 @@ google.golang.org/api v0.36.0/go.mod h1:+z5ficQTmoYpPn8LCUNVpK5I7hwkpjbcgqA7I34q google.golang.org/api v0.40.0/go.mod h1:fYKFpnQN0DsDSKRVRcQSDQNtqWPfM9i+zNPxepjRCQ8= google.golang.org/api v0.41.0/go.mod h1:RkxM5lITDfTzmyKFPt+wGrCJbVfniCr2ool8kTBzRTU= google.golang.org/api v0.43.0/go.mod h1:nQsDGjRXMo4lvh5hP0TKqF244gqhGcr/YSIykhUk/94= -google.golang.org/api v0.44.0/go.mod h1:EBOGZqzyhtvMDoxwS97ctnh0zUmYY6CxqXsc1AvkYD8= google.golang.org/api v0.47.0/go.mod h1:Wbvgpq1HddcWVtzsVLyfLp8lDg6AA241LmgIL59tHXo= google.golang.org/api v0.48.0/go.mod h1:71Pr1vy+TAZRPkPs/xlCf5SsU8WjuAWv1Pfjbtukyy4= google.golang.org/api v0.50.0/go.mod h1:4bNT5pAuq5ji4SRZm+5QIkjny9JAyVD/3gaSihNefaw= @@ -905,7 +943,6 @@ google.golang.org/genproto v0.0.0-20200804131852-c06518451d9c/go.mod h1:FWY/as6D google.golang.org/genproto v0.0.0-20200825200019-8632dd797987/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= google.golang.org/genproto v0.0.0-20200904004341-0bd0a958aa1d/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= google.golang.org/genproto v0.0.0-20201019141844-1ed22bb0c154/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= -google.golang.org/genproto v0.0.0-20201102152239-715cce707fb0/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= google.golang.org/genproto v0.0.0-20201109203340-2640f1f9cdfb/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= google.golang.org/genproto v0.0.0-20201201144952-b05cb90ed32e/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= google.golang.org/genproto v0.0.0-20201210142538-e3217bee35cc/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= @@ -937,6 +974,7 @@ google.golang.org/genproto v0.0.0-20211208223120-3a66f561d7aa/go.mod h1:5CzLGKJ6 google.golang.org/genproto v0.0.0-20211221195035-429b39de9b1c/go.mod h1:5CzLGKJ67TSI2B9POpiiyGha0AjJvZIUgRMt1dSmuhc= google.golang.org/genproto v0.0.0-20220114231437-d2e6a121cae0/go.mod h1:5CzLGKJ67TSI2B9POpiiyGha0AjJvZIUgRMt1dSmuhc= google.golang.org/genproto v0.0.0-20220201184016-50beb8ab5c44/go.mod h1:5CzLGKJ67TSI2B9POpiiyGha0AjJvZIUgRMt1dSmuhc= +google.golang.org/genproto v0.0.0-20220502173005-c8bf987b8c21/go.mod h1:RAyBrSAP7Fh3Nc84ghnVLDPuV51xc9agzmm4Ph6i0Q4= google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c= google.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38= google.golang.org/grpc v1.21.1/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM= @@ -963,6 +1001,8 @@ google.golang.org/grpc v1.39.0/go.mod h1:PImNr+rS9TWYb2O4/emRugxiyHZ5JyHW5F+RPnD google.golang.org/grpc v1.39.1/go.mod h1:PImNr+rS9TWYb2O4/emRugxiyHZ5JyHW5F+RPnDzfrE= google.golang.org/grpc v1.40.0/go.mod h1:ogyxbiOoUXAkP+4+xa6PZSE9DZgIHtSpzjDTB9KAK34= google.golang.org/grpc v1.40.1/go.mod h1:ogyxbiOoUXAkP+4+xa6PZSE9DZgIHtSpzjDTB9KAK34= +google.golang.org/grpc v1.46.0/go.mod h1:vN9eftEi1UMyUsIF80+uQXhHjbXYbm0uXoFCACuMGWk= +google.golang.org/grpc v1.47.0/go.mod h1:vN9eftEi1UMyUsIF80+uQXhHjbXYbm0uXoFCACuMGWk= google.golang.org/grpc/cmd/protoc-gen-go-grpc v1.1.0/go.mod h1:6Kw0yEErY5E/yWrBtf03jp27GLLJujG4z/JK95pnjjw= google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8= google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0= @@ -976,8 +1016,9 @@ google.golang.org/protobuf v1.24.0/go.mod h1:r/3tXBNzIEhYS9I1OUVjXDlt8tc493IdKGj google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c= google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw= google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc= -google.golang.org/protobuf v1.27.1 h1:SnqbnDw1V7RiZcXPx5MEeqPv2s79L9i7BJUlG/+RurQ= google.golang.org/protobuf v1.27.1/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc= +google.golang.org/protobuf v1.28.0 h1:w43yiav+6bVFTBQFZX0r7ipe9JQ1QsbMgHwbBziscLw= +google.golang.org/protobuf v1.28.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I= gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= @@ -989,7 +1030,6 @@ gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMy gopkg.in/inf.v0 v0.9.1 h1:73M5CoZyi3ZLMOyDlQh031Cx6N9NDJ2Vvfl76EDAgDc= gopkg.in/inf.v0 v0.9.1/go.mod h1:cWUDdTG/fYaXco+Dcufb5Vnc6Gp2YChqWtbxRZE0mXw= gopkg.in/ini.v1 v1.51.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k= -gopkg.in/ini.v1 v1.62.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k= gopkg.in/natefinch/lumberjack.v2 v2.0.0/go.mod h1:l0ndWWf7gzL7RNwBG7wST/UCcT4T24xpD6X8LsfU/+k= gopkg.in/resty.v1 v1.12.0/go.mod h1:mDo4pnntr5jdWRML875a/NmxYqAlA73dVijT2AXvQQo= gopkg.in/square/go-jose.v2 v2.2.2/go.mod h1:M9dMgbHiYLoDGQrXy7OpJDJWiKiU//h+vD76mk0e1AI= @@ -1007,8 +1047,9 @@ gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY= gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ= gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.0-20200615113413-eeeca48fe776/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= -gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b h1:h8qDotaEPuJATrMmW04NCwg7v22aHH28wwpauUhK9Oo= gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= +gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= +gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gotest.tools/v3 v3.0.2/go.mod h1:3SzNCllyD9/Y+b5r9JIKQ474KzkZyqLqEfYqMsX94Bk= gotest.tools/v3 v3.0.3/go.mod h1:Z7Lb0S5l+klDB31fvDQX8ss/FlKDxtlFlw3Oa8Ymbl8= honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= @@ -1018,46 +1059,42 @@ honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWh honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg= honnef.co/go/tools v0.0.1-2020.1.3/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k= honnef.co/go/tools v0.0.1-2020.1.4/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k= -k8s.io/api v0.23.5/go.mod h1:Na4XuKng8PXJ2JsploYYrivXrINeTaycCGcYgF91Xm8= -k8s.io/api v0.23.6 h1:yOK34wbYECH4RsJbQ9sfkFK3O7f/DUHRlzFehkqZyVw= -k8s.io/api v0.23.6/go.mod h1:1kFaYxGCFHYp3qd6a85DAj/yW8aVD6XLZMqJclkoi9g= -k8s.io/apiextensions-apiserver v0.23.5 h1:5SKzdXyvIJKu+zbfPc3kCbWpbxi+O+zdmAJBm26UJqI= -k8s.io/apiextensions-apiserver v0.23.5/go.mod h1:ntcPWNXS8ZPKN+zTXuzYMeg731CP0heCTl6gYBxLcuQ= -k8s.io/apimachinery v0.23.5/go.mod h1:BEuFMMBaIbcOqVIJqNZJXGFTP4W6AycEpb5+m/97hrM= -k8s.io/apimachinery v0.23.6 h1:RH1UweWJkWNTlFx0D8uxOpaU1tjIOvVVWV/bu5b3/NQ= -k8s.io/apimachinery v0.23.6/go.mod h1:BEuFMMBaIbcOqVIJqNZJXGFTP4W6AycEpb5+m/97hrM= -k8s.io/apiserver v0.23.5/go.mod h1:7wvMtGJ42VRxzgVI7jkbKvMbuCbVbgsWFT7RyXiRNTw= -k8s.io/client-go v0.23.5/go.mod h1:flkeinTO1CirYgzMPRWxUCnV0G4Fbu2vLhYCObnt/r4= -k8s.io/client-go v0.23.6 h1:7h4SctDVQAQbkHQnR4Kzi7EyUyvla5G1pFWf4+Od7hQ= -k8s.io/client-go v0.23.6/go.mod h1:Umt5icFOMLV/+qbtZ3PR0D+JA6lvvb3syzodv4irpK4= -k8s.io/code-generator v0.23.5/go.mod h1:S0Q1JVA+kSzTI1oUvbKAxZY/DYbA/ZUb4Uknog12ETk= -k8s.io/component-base v0.23.5 h1:8qgP5R6jG1BBSXmRYW+dsmitIrpk8F/fPEvgDenMCCE= -k8s.io/component-base v0.23.5/go.mod h1:c5Nq44KZyt1aLl0IpHX82fhsn84Sb0jjzwjpcA42bY0= +k8s.io/api v0.25.0/go.mod h1:ttceV1GyV1i1rnmvzT3BST08N6nGt+dudGrquzVQWPk= +k8s.io/api v0.25.4 h1:3YO8J4RtmG7elEgaWMb4HgmpS2CfY1QlaOz9nwB+ZSs= +k8s.io/api v0.25.4/go.mod h1:IG2+RzyPQLllQxnhzD8KQNEu4c4YvyDTpSMztf4A0OQ= +k8s.io/apiextensions-apiserver v0.25.0 h1:CJ9zlyXAbq0FIW8CD7HHyozCMBpDSiH7EdrSTCZcZFY= +k8s.io/apiextensions-apiserver v0.25.0/go.mod h1:3pAjZiN4zw7R8aZC5gR0y3/vCkGlAjCazcg1me8iB/E= +k8s.io/apimachinery v0.25.0/go.mod h1:qMx9eAk0sZQGsXGu86fab8tZdffHbwUfsvzqKn4mfB0= +k8s.io/apimachinery v0.25.4 h1:CtXsuaitMESSu339tfhVXhQrPET+EiWnIY1rcurKnAc= +k8s.io/apimachinery v0.25.4/go.mod h1:jaF9C/iPNM1FuLl7Zuy5b9v+n35HGSh6AQ4HYRkCqwo= +k8s.io/apiserver v0.25.0/go.mod h1:BKwsE+PTC+aZK+6OJQDPr0v6uS91/HWxX7evElAH6xo= +k8s.io/client-go v0.25.0/go.mod h1:lxykvypVfKilxhTklov0wz1FoaUZ8X4EwbhS6rpRfN8= +k8s.io/client-go v0.25.4 h1:3RNRDffAkNU56M/a7gUfXaEzdhZlYhoW8dgViGy5fn8= +k8s.io/client-go v0.25.4/go.mod h1:8trHCAC83XKY0wsBIpbirZU4NTUpbuhc2JnI7OruGZw= +k8s.io/code-generator v0.25.0/go.mod h1:B6jZgI3DvDFAualltPitbYMQ74NjaCFxum3YeKZZ+3w= +k8s.io/component-base v0.25.0 h1:haVKlLkPCFZhkcqB6WCvpVxftrg6+FK5x1ZuaIDaQ5Y= +k8s.io/component-base v0.25.0/go.mod h1:F2Sumv9CnbBlqrpdf7rKZTmmd2meJq0HizeyY/yAFxk= k8s.io/gengo v0.0.0-20210813121822-485abfe95c7c/go.mod h1:FiNAH4ZV3gBg2Kwh89tzAEV2be7d5xI0vBa/VySYy3E= +k8s.io/gengo v0.0.0-20211129171323-c02415ce4185/go.mod h1:FiNAH4ZV3gBg2Kwh89tzAEV2be7d5xI0vBa/VySYy3E= k8s.io/klog/v2 v2.0.0/go.mod h1:PBfzABfn139FHAV07az/IF9Wp1bkk3vpT2XSJ76fSDE= k8s.io/klog/v2 v2.2.0/go.mod h1:Od+F08eJP+W3HUb4pSrPpgp9DGU4GzlpG/TmITuYh/Y= -k8s.io/klog/v2 v2.30.0/go.mod h1:y1WjHnz7Dj687irZUWR/WLkLc5N1YHtjLdmgWjndZn0= -k8s.io/klog/v2 v2.40.1 h1:P4RRucWk/lFOlDdkAr3mc7iWFkgKrZY9qZMAgek06S4= -k8s.io/klog/v2 v2.40.1/go.mod h1:y1WjHnz7Dj687irZUWR/WLkLc5N1YHtjLdmgWjndZn0= -k8s.io/kube-openapi v0.0.0-20211115234752-e816edb12b65/go.mod h1:sX9MT8g7NVZM5lVL/j8QyCCJe8YSMW30QvGZWaCIDIk= -k8s.io/kube-openapi v0.0.0-20220124234850-424119656bbf h1:M9XBsiMslw2lb2ZzglC0TOkBPK5NQi0/noUrdnoFwUg= -k8s.io/kube-openapi v0.0.0-20220124234850-424119656bbf/go.mod h1:sX9MT8g7NVZM5lVL/j8QyCCJe8YSMW30QvGZWaCIDIk= +k8s.io/klog/v2 v2.70.1 h1:7aaoSdahviPmR+XkS7FyxlkkXs6tHISSG03RxleQAVQ= +k8s.io/klog/v2 v2.70.1/go.mod h1:y1WjHnz7Dj687irZUWR/WLkLc5N1YHtjLdmgWjndZn0= +k8s.io/kube-openapi v0.0.0-20220803162953-67bda5d908f1 h1:MQ8BAZPZlWk3S9K4a9NCkIFQtZShWqoha7snGixVgEA= +k8s.io/kube-openapi v0.0.0-20220803162953-67bda5d908f1/go.mod h1:C/N6wCaBHeBHkHUesQOQy2/MZqGgMAFPqGsGQLdbZBU= k8s.io/utils v0.0.0-20210802155522-efc7438f0176/go.mod h1:jPW/WVKK9YHAvNhRxK0md/EJ228hCsBRufyofKtW8HA= -k8s.io/utils v0.0.0-20211116205334-6203023598ed/go.mod h1:jPW/WVKK9YHAvNhRxK0md/EJ228hCsBRufyofKtW8HA= -k8s.io/utils v0.0.0-20220127004650-9b3446523e65 h1:ONWS0Wgdg5wRiQIAui7L/023aC9+IxrIrydY7l8llsE= -k8s.io/utils v0.0.0-20220127004650-9b3446523e65/go.mod h1:jPW/WVKK9YHAvNhRxK0md/EJ228hCsBRufyofKtW8HA= +k8s.io/utils v0.0.0-20220728103510-ee6ede2d64ed h1:jAne/RjBTyawwAy0utX5eqigAwz/lQhTmy+Hr/Cpue4= +k8s.io/utils v0.0.0-20220728103510-ee6ede2d64ed/go.mod h1:jPW/WVKK9YHAvNhRxK0md/EJ228hCsBRufyofKtW8HA= rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8= rsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0= rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA= -sigs.k8s.io/apiserver-network-proxy/konnectivity-client v0.0.30/go.mod h1:fEO7lRTdivWO2qYVCVG7dEADOMo/MLDCVr8So2g88Uw= -sigs.k8s.io/controller-runtime v0.11.2 h1:H5GTxQl0Mc9UjRJhORusqfJCIjBO8UtUxGggCwL1rLA= -sigs.k8s.io/controller-runtime v0.11.2/go.mod h1:P6QCzrEjLaZGqHsfd+os7JQ+WFZhvB8MRFsn4dWF7O4= -sigs.k8s.io/json v0.0.0-20211020170558-c049b76a60c6/go.mod h1:p4QtZmO4uMYipTQNzagwnNoseA6OxSUutVw05NhYDRs= -sigs.k8s.io/json v0.0.0-20211208200746-9f7c6b3444d2 h1:kDi4JBNAsJWfz1aEXhO8Jg87JJaPNLh5tIzYHgStQ9Y= -sigs.k8s.io/json v0.0.0-20211208200746-9f7c6b3444d2/go.mod h1:B+TnT182UBxE84DiCz4CVE26eOSDAeYCpfDnC2kdKMY= -sigs.k8s.io/structured-merge-diff/v4 v4.0.2/go.mod h1:bJZC9H9iH24zzfZ/41RGcq60oK1F7G282QMXDPYydCw= -sigs.k8s.io/structured-merge-diff/v4 v4.2.1 h1:bKCqE9GvQ5tiVHn5rfn1r+yao3aLQEaLzkkmAkf+A6Y= -sigs.k8s.io/structured-merge-diff/v4 v4.2.1/go.mod h1:j/nl6xW8vLS49O8YvXW1ocPhZawJtm+Yrr7PPRQ0Vg4= +sigs.k8s.io/apiserver-network-proxy/konnectivity-client v0.0.32/go.mod h1:fEO7lRTdivWO2qYVCVG7dEADOMo/MLDCVr8So2g88Uw= +sigs.k8s.io/controller-runtime v0.13.1 h1:tUsRCSJVM1QQOOeViGeX3GMT3dQF1eePPw6sEE3xSlg= +sigs.k8s.io/controller-runtime v0.13.1/go.mod h1:Zbz+el8Yg31jubvAEyglRZGdLAjplZl+PgtYNI6WNTI= +sigs.k8s.io/json v0.0.0-20220713155537-f223a00ba0e2 h1:iXTIw73aPyC+oRdyqqvVJuloN1p0AC/kzH07hu3NE+k= +sigs.k8s.io/json v0.0.0-20220713155537-f223a00ba0e2/go.mod h1:B8JuhiUyNFVKdsE8h686QcCxMaH6HrOAZj4vswFpcB0= +sigs.k8s.io/structured-merge-diff/v4 v4.2.3 h1:PRbqxJClWWYMNV1dhaG4NsibJbArud9kFxnAMREiWFE= +sigs.k8s.io/structured-merge-diff/v4 v4.2.3/go.mod h1:qjx8mGObPmV2aSZepjQjbmb2ihdVs8cGKBraizNC69E= sigs.k8s.io/yaml v1.2.0/go.mod h1:yfXDCHCao9+ENCvLSE62v9VSji2MKu5jeNfTrofGhJc= sigs.k8s.io/yaml v1.3.0 h1:a2VclLzOGrwOHDiV8EfBGhvjHvP46CtW5j6POvhYGGo= sigs.k8s.io/yaml v1.3.0/go.mod h1:GeOyir5tyXNByN85N/dRIT9es5UQNerPYEKK56eTBm8= From 4e369edec55eb00c18c6ba42cae2e11176149fbb Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Fri, 18 Nov 2022 12:03:11 -0500 Subject: [PATCH 510/628] update oci-go-sdk --- apis/database/v1alpha1/adbfamily_common_utils.go | 6 +++--- .../v1alpha1/autonomouscontainerdatabase_types.go | 2 +- .../v1alpha1/autonomouscontainerdatabase_webhook_test.go | 4 ++-- apis/database/v1alpha1/autonomousdatabase_types.go | 2 +- apis/database/v1alpha1/autonomousdatabase_webhook.go | 4 ++-- apis/database/v1alpha1/autonomousdatabase_webhook_test.go | 4 ++-- apis/database/v1alpha1/autonomousdatabasebackup_types.go | 4 ++-- .../v1alpha1/autonomousdatabasebackup_webhook_test.go | 2 +- apis/database/v1alpha1/autonomousdatabaserestore_types.go | 6 +++--- .../v1alpha1/autonomousdatabaserestore_webhook_test.go | 2 +- commons/dbcssystem/dbcs_reconciler.go | 8 ++++---- commons/dbcssystem/dcommon.go | 6 +++--- commons/k8s/create.go | 4 ++-- commons/oci/containerdatabase.go | 4 ++-- commons/oci/database.go | 4 ++-- commons/oci/provider.go | 4 ++-- commons/oci/vault.go | 4 ++-- commons/oci/workrequest.go | 4 ++-- commons/sharding/scommon.go | 4 ++-- .../database/autonomouscontainerdatabase_controller.go | 2 +- controllers/database/autonomousdatabase_controller.go | 4 ++-- .../database/autonomousdatabaserestore_controller.go | 2 +- controllers/database/dbcssystem_controller.go | 6 +++--- controllers/database/shardingdatabase_controller.go | 4 ++-- go.mod | 6 +++--- go.sum | 4 ++-- test/e2e/autonomouscontainerdatabase_test.go | 4 ++-- test/e2e/autonomousdatabase_controller_bind_test.go | 6 +++--- test/e2e/autonomousdatabase_controller_create_test.go | 4 ++-- test/e2e/behavior/shared_behaviors.go | 6 +++--- test/e2e/suite_test.go | 4 ++-- test/e2e/util/oci_acd_request.go | 4 ++-- test/e2e/util/oci_config_util.go | 2 +- test/e2e/util/oci_db_request.go | 4 ++-- test/e2e/util/oci_work_request.go | 4 ++-- 35 files changed, 72 insertions(+), 72 deletions(-) diff --git a/apis/database/v1alpha1/adbfamily_common_utils.go b/apis/database/v1alpha1/adbfamily_common_utils.go index 8dcb13d8..e6691a12 100644 --- a/apis/database/v1alpha1/adbfamily_common_utils.go +++ b/apis/database/v1alpha1/adbfamily_common_utils.go @@ -46,9 +46,9 @@ import ( metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "sigs.k8s.io/controller-runtime/pkg/client" - "github.com/oracle/oci-go-sdk/v64/common" - "github.com/oracle/oci-go-sdk/v64/database" - "github.com/oracle/oci-go-sdk/v64/workrequests" + "github.com/oracle/oci-go-sdk/v65/common" + "github.com/oracle/oci-go-sdk/v65/database" + "github.com/oracle/oci-go-sdk/v65/workrequests" ) // LastSuccessfulSpec is an annotation key which maps to the value of last successful spec diff --git a/apis/database/v1alpha1/autonomouscontainerdatabase_types.go b/apis/database/v1alpha1/autonomouscontainerdatabase_types.go index 28f36445..abf0ee0d 100644 --- a/apis/database/v1alpha1/autonomouscontainerdatabase_types.go +++ b/apis/database/v1alpha1/autonomouscontainerdatabase_types.go @@ -42,7 +42,7 @@ import ( "encoding/json" "reflect" - "github.com/oracle/oci-go-sdk/v64/database" + "github.com/oracle/oci-go-sdk/v65/database" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" ) diff --git a/apis/database/v1alpha1/autonomouscontainerdatabase_webhook_test.go b/apis/database/v1alpha1/autonomouscontainerdatabase_webhook_test.go index 11a8d5b2..668a575d 100644 --- a/apis/database/v1alpha1/autonomouscontainerdatabase_webhook_test.go +++ b/apis/database/v1alpha1/autonomouscontainerdatabase_webhook_test.go @@ -42,8 +42,8 @@ import ( "encoding/json" "time" - "github.com/oracle/oci-go-sdk/v64/common" - "github.com/oracle/oci-go-sdk/v64/database" + "github.com/oracle/oci-go-sdk/v65/common" + "github.com/oracle/oci-go-sdk/v65/database" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/types" // +kubebuilder:scaffold:imports diff --git a/apis/database/v1alpha1/autonomousdatabase_types.go b/apis/database/v1alpha1/autonomousdatabase_types.go index f39d10bf..413c16de 100644 --- a/apis/database/v1alpha1/autonomousdatabase_types.go +++ b/apis/database/v1alpha1/autonomousdatabase_types.go @@ -42,7 +42,7 @@ import ( "encoding/json" "reflect" - "github.com/oracle/oci-go-sdk/v64/database" + "github.com/oracle/oci-go-sdk/v65/database" metaV1 "k8s.io/apimachinery/pkg/apis/meta/v1" ) diff --git a/apis/database/v1alpha1/autonomousdatabase_webhook.go b/apis/database/v1alpha1/autonomousdatabase_webhook.go index 8aeaa00d..9c01ad95 100644 --- a/apis/database/v1alpha1/autonomousdatabase_webhook.go +++ b/apis/database/v1alpha1/autonomousdatabase_webhook.go @@ -41,8 +41,8 @@ package v1alpha1 import ( "fmt" - "github.com/oracle/oci-go-sdk/v64/common" - "github.com/oracle/oci-go-sdk/v64/database" + "github.com/oracle/oci-go-sdk/v65/common" + "github.com/oracle/oci-go-sdk/v65/database" apierrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/runtime" "k8s.io/apimachinery/pkg/runtime/schema" diff --git a/apis/database/v1alpha1/autonomousdatabase_webhook_test.go b/apis/database/v1alpha1/autonomousdatabase_webhook_test.go index 639507ef..ee26021f 100644 --- a/apis/database/v1alpha1/autonomousdatabase_webhook_test.go +++ b/apis/database/v1alpha1/autonomousdatabase_webhook_test.go @@ -42,8 +42,8 @@ import ( "encoding/json" "time" - "github.com/oracle/oci-go-sdk/v64/common" - "github.com/oracle/oci-go-sdk/v64/database" + "github.com/oracle/oci-go-sdk/v65/common" + "github.com/oracle/oci-go-sdk/v65/database" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/types" // +kubebuilder:scaffold:imports diff --git a/apis/database/v1alpha1/autonomousdatabasebackup_types.go b/apis/database/v1alpha1/autonomousdatabasebackup_types.go index 27531fd5..d64e549b 100644 --- a/apis/database/v1alpha1/autonomousdatabasebackup_types.go +++ b/apis/database/v1alpha1/autonomousdatabasebackup_types.go @@ -41,8 +41,8 @@ package v1alpha1 import ( metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" - "github.com/oracle/oci-go-sdk/v64/common" - "github.com/oracle/oci-go-sdk/v64/database" + "github.com/oracle/oci-go-sdk/v65/common" + "github.com/oracle/oci-go-sdk/v65/database" ) // EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! diff --git a/apis/database/v1alpha1/autonomousdatabasebackup_webhook_test.go b/apis/database/v1alpha1/autonomousdatabasebackup_webhook_test.go index 481f4ac7..497c3f28 100644 --- a/apis/database/v1alpha1/autonomousdatabasebackup_webhook_test.go +++ b/apis/database/v1alpha1/autonomousdatabasebackup_webhook_test.go @@ -41,7 +41,7 @@ import ( "context" "time" - "github.com/oracle/oci-go-sdk/v64/common" + "github.com/oracle/oci-go-sdk/v65/common" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/types" // +kubebuilder:scaffold:imports diff --git a/apis/database/v1alpha1/autonomousdatabaserestore_types.go b/apis/database/v1alpha1/autonomousdatabaserestore_types.go index bbc4e1c0..4bea0043 100644 --- a/apis/database/v1alpha1/autonomousdatabaserestore_types.go +++ b/apis/database/v1alpha1/autonomousdatabaserestore_types.go @@ -43,9 +43,9 @@ import ( metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" - "github.com/oracle/oci-go-sdk/v64/common" - "github.com/oracle/oci-go-sdk/v64/database" - "github.com/oracle/oci-go-sdk/v64/workrequests" + "github.com/oracle/oci-go-sdk/v65/common" + "github.com/oracle/oci-go-sdk/v65/database" + "github.com/oracle/oci-go-sdk/v65/workrequests" ) // EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! diff --git a/apis/database/v1alpha1/autonomousdatabaserestore_webhook_test.go b/apis/database/v1alpha1/autonomousdatabaserestore_webhook_test.go index cd92779f..aed87d71 100644 --- a/apis/database/v1alpha1/autonomousdatabaserestore_webhook_test.go +++ b/apis/database/v1alpha1/autonomousdatabaserestore_webhook_test.go @@ -38,7 +38,7 @@ package v1alpha1 import ( - "github.com/oracle/oci-go-sdk/v64/common" + "github.com/oracle/oci-go-sdk/v65/common" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" // +kubebuilder:scaffold:imports ) diff --git a/commons/dbcssystem/dbcs_reconciler.go b/commons/dbcssystem/dbcs_reconciler.go index f923ec86..d4df1ab4 100644 --- a/commons/dbcssystem/dbcs_reconciler.go +++ b/commons/dbcssystem/dbcs_reconciler.go @@ -46,10 +46,10 @@ import ( "time" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v64/common" - "github.com/oracle/oci-go-sdk/v64/core" - "github.com/oracle/oci-go-sdk/v64/database" - "github.com/oracle/oci-go-sdk/v64/workrequests" + "github.com/oracle/oci-go-sdk/v65/common" + "github.com/oracle/oci-go-sdk/v65/core" + "github.com/oracle/oci-go-sdk/v65/database" + "github.com/oracle/oci-go-sdk/v65/workrequests" "k8s.io/client-go/tools/record" "k8s.io/client-go/util/retry" diff --git a/commons/dbcssystem/dcommon.go b/commons/dbcssystem/dcommon.go index 9ba66079..3af3a6b4 100644 --- a/commons/dbcssystem/dcommon.go +++ b/commons/dbcssystem/dcommon.go @@ -44,9 +44,9 @@ import ( "strconv" "strings" - "github.com/oracle/oci-go-sdk/v64/common" - "github.com/oracle/oci-go-sdk/v64/database" - "github.com/oracle/oci-go-sdk/v64/workrequests" + "github.com/oracle/oci-go-sdk/v65/common" + "github.com/oracle/oci-go-sdk/v65/database" + "github.com/oracle/oci-go-sdk/v65/workrequests" "sigs.k8s.io/controller-runtime/pkg/client" databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" diff --git a/commons/k8s/create.go b/commons/k8s/create.go index 3c058156..5055bc0e 100644 --- a/commons/k8s/create.go +++ b/commons/k8s/create.go @@ -43,8 +43,8 @@ import ( dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" - "github.com/oracle/oci-go-sdk/v64/common" - "github.com/oracle/oci-go-sdk/v64/database" + "github.com/oracle/oci-go-sdk/v65/common" + "github.com/oracle/oci-go-sdk/v65/database" corev1 "k8s.io/api/core/v1" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "sigs.k8s.io/controller-runtime/pkg/client" diff --git a/commons/oci/containerdatabase.go b/commons/oci/containerdatabase.go index 9fa9ec91..a5313d41 100644 --- a/commons/oci/containerdatabase.go +++ b/commons/oci/containerdatabase.go @@ -41,8 +41,8 @@ package oci import ( "context" - "github.com/oracle/oci-go-sdk/v64/common" - "github.com/oracle/oci-go-sdk/v64/database" + "github.com/oracle/oci-go-sdk/v65/common" + "github.com/oracle/oci-go-sdk/v65/database" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" ) diff --git a/commons/oci/database.go b/commons/oci/database.go index 1e08cc66..10d6c8ff 100644 --- a/commons/oci/database.go +++ b/commons/oci/database.go @@ -43,8 +43,8 @@ import ( "fmt" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v64/common" - "github.com/oracle/oci-go-sdk/v64/database" + "github.com/oracle/oci-go-sdk/v65/common" + "github.com/oracle/oci-go-sdk/v65/database" "sigs.k8s.io/controller-runtime/pkg/client" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" diff --git a/commons/oci/provider.go b/commons/oci/provider.go index a5ded3cc..e875969c 100644 --- a/commons/oci/provider.go +++ b/commons/oci/provider.go @@ -41,8 +41,8 @@ package oci import ( "errors" - "github.com/oracle/oci-go-sdk/v64/common" - "github.com/oracle/oci-go-sdk/v64/common/auth" + "github.com/oracle/oci-go-sdk/v65/common" + "github.com/oracle/oci-go-sdk/v65/common/auth" "sigs.k8s.io/controller-runtime/pkg/client" diff --git a/commons/oci/vault.go b/commons/oci/vault.go index 7a83e4f9..d7069a88 100644 --- a/commons/oci/vault.go +++ b/commons/oci/vault.go @@ -43,8 +43,8 @@ import ( "encoding/base64" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v64/common" - "github.com/oracle/oci-go-sdk/v64/secrets" + "github.com/oracle/oci-go-sdk/v65/common" + "github.com/oracle/oci-go-sdk/v65/secrets" ) type VaultService interface { diff --git a/commons/oci/workrequest.go b/commons/oci/workrequest.go index 4eb0b2d0..f68d2766 100644 --- a/commons/oci/workrequest.go +++ b/commons/oci/workrequest.go @@ -44,8 +44,8 @@ import ( "github.com/go-logr/logr" "sigs.k8s.io/controller-runtime/pkg/client" - "github.com/oracle/oci-go-sdk/v64/common" - "github.com/oracle/oci-go-sdk/v64/workrequests" + "github.com/oracle/oci-go-sdk/v65/common" + "github.com/oracle/oci-go-sdk/v65/workrequests" ) type WorkRequestService interface { diff --git a/commons/sharding/scommon.go b/commons/sharding/scommon.go index 7f6d5da2..5a25a78d 100644 --- a/commons/sharding/scommon.go +++ b/commons/sharding/scommon.go @@ -49,8 +49,8 @@ import ( "strings" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v64/common" - "github.com/oracle/oci-go-sdk/v64/ons" + "github.com/oracle/oci-go-sdk/v65/common" + "github.com/oracle/oci-go-sdk/v65/ons" appsv1 "k8s.io/api/apps/v1" corev1 "k8s.io/api/core/v1" "k8s.io/apimachinery/pkg/api/errors" diff --git a/controllers/database/autonomouscontainerdatabase_controller.go b/controllers/database/autonomouscontainerdatabase_controller.go index 660bf0f8..3845a03f 100644 --- a/controllers/database/autonomouscontainerdatabase_controller.go +++ b/controllers/database/autonomouscontainerdatabase_controller.go @@ -45,7 +45,7 @@ import ( "reflect" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v64/database" + "github.com/oracle/oci-go-sdk/v65/database" corev1 "k8s.io/api/core/v1" apiErrors "k8s.io/apimachinery/pkg/api/errors" diff --git a/controllers/database/autonomousdatabase_controller.go b/controllers/database/autonomousdatabase_controller.go index b3ca889d..379ab86b 100644 --- a/controllers/database/autonomousdatabase_controller.go +++ b/controllers/database/autonomousdatabase_controller.go @@ -49,8 +49,8 @@ import ( "time" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v64/common" - "github.com/oracle/oci-go-sdk/v64/database" + "github.com/oracle/oci-go-sdk/v65/common" + "github.com/oracle/oci-go-sdk/v65/database" corev1 "k8s.io/api/core/v1" apiErrors "k8s.io/apimachinery/pkg/api/errors" diff --git a/controllers/database/autonomousdatabaserestore_controller.go b/controllers/database/autonomousdatabaserestore_controller.go index 5a25759c..254731bb 100644 --- a/controllers/database/autonomousdatabaserestore_controller.go +++ b/controllers/database/autonomousdatabaserestore_controller.go @@ -53,7 +53,7 @@ import ( "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" "sigs.k8s.io/controller-runtime/pkg/predicate" - "github.com/oracle/oci-go-sdk/v64/common" + "github.com/oracle/oci-go-sdk/v65/common" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" "github.com/oracle/oracle-database-operator/commons/adb_family" "github.com/oracle/oracle-database-operator/commons/k8s" diff --git a/controllers/database/dbcssystem_controller.go b/controllers/database/dbcssystem_controller.go index d279bf84..003be62a 100644 --- a/controllers/database/dbcssystem_controller.go +++ b/controllers/database/dbcssystem_controller.go @@ -48,9 +48,9 @@ import ( "github.com/oracle/oracle-database-operator/commons/oci" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v64/core" - "github.com/oracle/oci-go-sdk/v64/database" - "github.com/oracle/oci-go-sdk/v64/workrequests" + "github.com/oracle/oci-go-sdk/v65/core" + "github.com/oracle/oci-go-sdk/v65/database" + "github.com/oracle/oci-go-sdk/v65/workrequests" "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/runtime" "k8s.io/client-go/tools/record" diff --git a/controllers/database/shardingdatabase_controller.go b/controllers/database/shardingdatabase_controller.go index a67401d0..c76b7f45 100644 --- a/controllers/database/shardingdatabase_controller.go +++ b/controllers/database/shardingdatabase_controller.go @@ -47,8 +47,8 @@ import ( "time" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v64/common" - "github.com/oracle/oci-go-sdk/v64/ons" + "github.com/oracle/oci-go-sdk/v65/common" + "github.com/oracle/oci-go-sdk/v65/ons" appsv1 "k8s.io/api/apps/v1" corev1 "k8s.io/api/core/v1" "k8s.io/apimachinery/pkg/api/errors" diff --git a/go.mod b/go.mod index 7eeb98a0..9e37f7a3 100644 --- a/go.mod +++ b/go.mod @@ -1,6 +1,6 @@ module github.com/oracle/oracle-database-operator -go 1.17 +go 1.19 require ( github.com/go-logr/logr v1.2.3 @@ -13,6 +13,7 @@ require ( k8s.io/client-go v0.25.4 sigs.k8s.io/controller-runtime v0.13.1 sigs.k8s.io/yaml v1.3.0 + github.com/oracle/oci-go-sdk/v65 v65.26.1 ) require ( @@ -76,7 +77,6 @@ require ( github.com/evanphx/json-patch v5.6.0+incompatible // indirect github.com/google/gofuzz v1.2.0 // indirect github.com/google/uuid v1.3.0 // indirect - github.com/oracle/oci-go-sdk/v64 v64.0.0 go.uber.org/atomic v1.9.0 // indirect go.uber.org/multierr v1.7.0 // indirect -) +) \ No newline at end of file diff --git a/go.sum b/go.sum index f694646d..fc9f98e0 100644 --- a/go.sum +++ b/go.sum @@ -420,8 +420,8 @@ github.com/onsi/gomega v1.24.0/go.mod h1:Z/NWtiqwBrwUt4/2loMmHL63EDLnYHmVbuBpDr2 github.com/onsi/gomega v1.24.1 h1:KORJXNNTzJXzu4ScJWssJfJMnJ+2QJqhoQSRwNlze9E= github.com/onsi/gomega v1.24.1/go.mod h1:3AOiACssS3/MajrniINInwbfOOtfZvplPzuRSmvt1jM= github.com/opentracing/opentracing-go v1.1.0/go.mod h1:UkNAQd3GIcIGf0SeVgPpRdFStlNbqXla1AfSYxPUl2o= -github.com/oracle/oci-go-sdk/v64 v64.0.0 h1:ALQDxoKLeNfxLbbOkjnNxMf3/AyWhnHhMJxc8tZXRgI= -github.com/oracle/oci-go-sdk/v64 v64.0.0/go.mod h1:eLqTg9MSXE/9naair0yFKqiP1RlYghuzBDvEgZvwjYs= +github.com/oracle/oci-go-sdk/v65 v65.26.1 h1:Ms20RSRj+CuvQmw5ET1TkmzxLBI+bmLjZ6NYANA3gkk= +github.com/oracle/oci-go-sdk/v65 v65.26.1/go.mod h1:oyMrMa1vOzzKTmPN+kqrTR9y9kPA2tU1igN3NUSNTIE= github.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc= github.com/pelletier/go-toml v1.2.0/go.mod h1:5z9KED0ma1S8pY6P1sdut58dfprrGBbd/94hg7ilaic= github.com/peterbourgon/diskv v2.0.1+incompatible/go.mod h1:uqqh8zWWbv1HBMNONnaR/tNboyR3/BZd58JJSHlUSCU= diff --git a/test/e2e/autonomouscontainerdatabase_test.go b/test/e2e/autonomouscontainerdatabase_test.go index 7e8d7191..f27d8a6d 100644 --- a/test/e2e/autonomouscontainerdatabase_test.go +++ b/test/e2e/autonomouscontainerdatabase_test.go @@ -42,8 +42,8 @@ import ( "context" "time" - "github.com/oracle/oci-go-sdk/v64/common" - "github.com/oracle/oci-go-sdk/v64/database" + "github.com/oracle/oci-go-sdk/v65/common" + "github.com/oracle/oci-go-sdk/v65/database" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/types" diff --git a/test/e2e/autonomousdatabase_controller_bind_test.go b/test/e2e/autonomousdatabase_controller_bind_test.go index ce5de55a..58de3356 100644 --- a/test/e2e/autonomousdatabase_controller_bind_test.go +++ b/test/e2e/autonomousdatabase_controller_bind_test.go @@ -42,9 +42,9 @@ import ( "context" "time" - "github.com/oracle/oci-go-sdk/v64/common" - "github.com/oracle/oci-go-sdk/v64/database" - "github.com/oracle/oci-go-sdk/v64/workrequests" + "github.com/oracle/oci-go-sdk/v65/common" + "github.com/oracle/oci-go-sdk/v65/database" + "github.com/oracle/oci-go-sdk/v65/workrequests" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/types" diff --git a/test/e2e/autonomousdatabase_controller_create_test.go b/test/e2e/autonomousdatabase_controller_create_test.go index 789d124e..9cf0e7e7 100644 --- a/test/e2e/autonomousdatabase_controller_create_test.go +++ b/test/e2e/autonomousdatabase_controller_create_test.go @@ -42,8 +42,8 @@ import ( "context" "time" - "github.com/oracle/oci-go-sdk/v64/common" - "github.com/oracle/oci-go-sdk/v64/database" + "github.com/oracle/oci-go-sdk/v65/common" + "github.com/oracle/oci-go-sdk/v65/database" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/types" diff --git a/test/e2e/behavior/shared_behaviors.go b/test/e2e/behavior/shared_behaviors.go index 244f07ba..00fc02c9 100644 --- a/test/e2e/behavior/shared_behaviors.go +++ b/test/e2e/behavior/shared_behaviors.go @@ -47,9 +47,9 @@ import ( "github.com/onsi/ginkgo/v2" "github.com/onsi/gomega" - "github.com/oracle/oci-go-sdk/v64/common" - "github.com/oracle/oci-go-sdk/v64/database" - "github.com/oracle/oci-go-sdk/v64/workrequests" + "github.com/oracle/oci-go-sdk/v65/common" + "github.com/oracle/oci-go-sdk/v65/database" + "github.com/oracle/oci-go-sdk/v65/workrequests" corev1 "k8s.io/api/core/v1" k8sErrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/types" diff --git a/test/e2e/suite_test.go b/test/e2e/suite_test.go index d0e195ed..a9fa06e5 100644 --- a/test/e2e/suite_test.go +++ b/test/e2e/suite_test.go @@ -49,8 +49,8 @@ import ( "github.com/onsi/ginkgo/v2" "github.com/onsi/gomega" "github.com/onsi/gomega/gexec" - "github.com/oracle/oci-go-sdk/v64/common" - "github.com/oracle/oci-go-sdk/v64/database" + "github.com/oracle/oci-go-sdk/v65/common" + "github.com/oracle/oci-go-sdk/v65/database" corev1 "k8s.io/api/core/v1" "k8s.io/client-go/kubernetes/scheme" "k8s.io/client-go/rest" diff --git a/test/e2e/util/oci_acd_request.go b/test/e2e/util/oci_acd_request.go index 1bd8f345..f0a3a841 100644 --- a/test/e2e/util/oci_acd_request.go +++ b/test/e2e/util/oci_acd_request.go @@ -41,8 +41,8 @@ package e2eutil import ( "context" - "github.com/oracle/oci-go-sdk/v64/common" - "github.com/oracle/oci-go-sdk/v64/database" + "github.com/oracle/oci-go-sdk/v65/common" + "github.com/oracle/oci-go-sdk/v65/database" // "io" // "io/ioutil" // "time" diff --git a/test/e2e/util/oci_config_util.go b/test/e2e/util/oci_config_util.go index 5fb5527f..e66e029b 100644 --- a/test/e2e/util/oci_config_util.go +++ b/test/e2e/util/oci_config_util.go @@ -48,7 +48,7 @@ import ( "regexp" "strings" - "github.com/oracle/oci-go-sdk/v64/common" + "github.com/oracle/oci-go-sdk/v65/common" corev1 "k8s.io/api/core/v1" ) diff --git a/test/e2e/util/oci_db_request.go b/test/e2e/util/oci_db_request.go index 5a835803..09cb508a 100644 --- a/test/e2e/util/oci_db_request.go +++ b/test/e2e/util/oci_db_request.go @@ -41,8 +41,8 @@ package e2eutil import ( "context" - "github.com/oracle/oci-go-sdk/v64/common" - "github.com/oracle/oci-go-sdk/v64/database" + "github.com/oracle/oci-go-sdk/v65/common" + "github.com/oracle/oci-go-sdk/v65/database" "io" "io/ioutil" "time" diff --git a/test/e2e/util/oci_work_request.go b/test/e2e/util/oci_work_request.go index a58ce3cf..9751ef63 100644 --- a/test/e2e/util/oci_work_request.go +++ b/test/e2e/util/oci_work_request.go @@ -41,8 +41,8 @@ package e2eutil import ( "context" - "github.com/oracle/oci-go-sdk/v64/common" - "github.com/oracle/oci-go-sdk/v64/workrequests" + "github.com/oracle/oci-go-sdk/v65/common" + "github.com/oracle/oci-go-sdk/v65/workrequests" "time" ) From 6668f6be5a3473c21da8fcdcf3536cf7bb87de75 Mon Sep 17 00:00:00 2001 From: Matteo Malvezzi Date: Tue, 22 Nov 2022 15:54:28 +0000 Subject: [PATCH 511/628] fix for bug 34817258 UNPLUGPDB FUNCTION : VARIABLE TDESECRET AND TDEPASSWORD HAVE WRONG SCOPE DEFINITION --- controllers/database/pdb_controller.go | 35 +++++++++++++++++++++----- 1 file changed, 29 insertions(+), 6 deletions(-) diff --git a/controllers/database/pdb_controller.go b/controllers/database/pdb_controller.go index f9231a66..05e550c3 100644 --- a/controllers/database/pdb_controller.go +++ b/controllers/database/pdb_controller.go @@ -119,6 +119,10 @@ var ( const PDBFinalizer = "database.oracle.com/PDBfinalizer" +var tdePassword string +var tdeSecret string + + //+kubebuilder:rbac:groups=database.oracle.com,resources=pdbs,verbs=get;list;watch;create;update;patch;delete //+kubebuilder:rbac:groups=database.oracle.com,resources=pdbs/status,verbs=get;update;patch //+kubebuilder:rbac:groups=database.oracle.com,resources=pdbs/finalizers,verbs=get;create;update;patch;delete @@ -585,6 +589,9 @@ func (r *PDBReconciler) createPDB(ctx context.Context, req ctrl.Request, pdb *db log := r.Log.WithValues("createPDB", req.NamespacedName) var err error + var tdePassword string + var tdeSecret string + cdb, err := r.getCDBResource(ctx, req, pdb) if err != nil { @@ -613,14 +620,17 @@ func (r *PDBReconciler) createPDB(ctx context.Context, req ctrl.Request, pdb *db "getScript": strconv.FormatBool(*(pdb.Spec.GetScript))} if *(pdb.Spec.TDEImport) { - tdePassword, err := r.getSecret(ctx, req, pdb, pdb.Spec.TDEPassword.Secret.SecretName, pdb.Spec.TDEPassword.Secret.Key) + tdePassword, err = r.getSecret(ctx, req, pdb, pdb.Spec.TDEPassword.Secret.SecretName, pdb.Spec.TDEPassword.Secret.Key) if err != nil { return err } - tdeSecret, err := r.getSecret(ctx, req, pdb, pdb.Spec.TDESecret.Secret.SecretName, pdb.Spec.TDESecret.Secret.Key) + tdeSecret, err = r.getSecret(ctx, req, pdb, pdb.Spec.TDESecret.Secret.SecretName, pdb.Spec.TDESecret.Secret.Key) if err != nil { return err } + + tdeSecret = tdeSecret[:len(tdeSecret)-1] + tdePassword = tdeSecret[:len(tdePassword)-1] values["tdePassword"] = tdePassword values["tdeKeystorePath"] = pdb.Spec.TDEKeystorePath values["tdeSecret"] = tdeSecret @@ -636,6 +646,7 @@ func (r *PDBReconciler) createPDB(ctx context.Context, req ctrl.Request, pdb *db } _, err = r.callAPI(ctx, req, pdb, url, values, "POST") if err != nil { + log.Error(err, "callAPI error", err.Error()) return err } @@ -720,6 +731,8 @@ func (r *PDBReconciler) plugPDB(ctx context.Context, req ctrl.Request, pdb *dbap log := r.Log.WithValues("plugPDB", req.NamespacedName) var err error + var tdePassword string + var tdeSecret string cdb, err := r.getCDBResource(ctx, req, pdb) if err != nil { @@ -742,14 +755,17 @@ func (r *PDBReconciler) plugPDB(ctx context.Context, req ctrl.Request, pdb *dbap "getScript": strconv.FormatBool(*(pdb.Spec.GetScript))} if *(pdb.Spec.TDEImport) { - tdePassword, err := r.getSecret(ctx, req, pdb, pdb.Spec.TDEPassword.Secret.SecretName, pdb.Spec.TDEPassword.Secret.Key) + tdePassword, err = r.getSecret(ctx, req, pdb, pdb.Spec.TDEPassword.Secret.SecretName, pdb.Spec.TDEPassword.Secret.Key) if err != nil { return err } - tdeSecret, err := r.getSecret(ctx, req, pdb, pdb.Spec.TDESecret.Secret.SecretName, pdb.Spec.TDESecret.Secret.Key) + tdeSecret, err = r.getSecret(ctx, req, pdb, pdb.Spec.TDESecret.Secret.SecretName, pdb.Spec.TDESecret.Secret.Key) if err != nil { return err } + + tdeSecret = tdeSecret[:len(tdeSecret)-1] + tdePassword = tdeSecret[:len(tdePassword)-1] values["tdePassword"] = tdePassword values["tdeKeystorePath"] = pdb.Spec.TDEKeystorePath values["tdeSecret"] = tdeSecret @@ -793,6 +809,9 @@ func (r *PDBReconciler) unplugPDB(ctx context.Context, req ctrl.Request, pdb *db log := r.Log.WithValues("unplugPDB", req.NamespacedName) var err error + var tdePassword string + var tdeSecret string + cdb, err := r.getCDBResource(ctx, req, pdb) if err != nil { @@ -806,14 +825,17 @@ func (r *PDBReconciler) unplugPDB(ctx context.Context, req ctrl.Request, pdb *db if *(pdb.Spec.TDEExport) { // Get the TDE Password - tdePassword, err := r.getSecret(ctx, req, pdb, pdb.Spec.TDEPassword.Secret.SecretName, pdb.Spec.TDEPassword.Secret.Key) + tdePassword, err = r.getSecret(ctx, req, pdb, pdb.Spec.TDEPassword.Secret.SecretName, pdb.Spec.TDEPassword.Secret.Key) if err != nil { return err } - tdeSecret, err := r.getSecret(ctx, req, pdb, pdb.Spec.TDESecret.Secret.SecretName, pdb.Spec.TDESecret.Secret.Key) + tdeSecret, err = r.getSecret(ctx, req, pdb, pdb.Spec.TDESecret.Secret.SecretName, pdb.Spec.TDESecret.Secret.Key) if err != nil { return err } + + tdeSecret = tdeSecret[:len(tdeSecret)-1] + tdePassword = tdeSecret[:len(tdePassword)-1] values["tdePassword"] = tdePassword values["tdeKeystorePath"] = pdb.Spec.TDEKeystorePath values["tdeSecret"] = tdeSecret @@ -821,6 +843,7 @@ func (r *PDBReconciler) unplugPDB(ctx context.Context, req ctrl.Request, pdb *db } url := "https://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdb.Spec.PDBName + "/" + log.Info("CallAPI(url)", "url", url) pdb.Status.Phase = pdbPhaseUnplug pdb.Status.Msg = "Waiting for PDB to be unplugged" From f3d13a8837dbb6a305072d9b35e50eb31ad6e606 Mon Sep 17 00:00:00 2001 From: matteo malvezzi Date: Fri, 2 Dec 2022 10:55:26 +0100 Subject: [PATCH 512/628] Bug 34855542 - WRONG DELETE PAYLOAD DEFINITION IN THE PDB CONTROLLER --- controllers/database/pdb_controller.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/controllers/database/pdb_controller.go b/controllers/database/pdb_controller.go index 05e550c3..0b52fd88 100644 --- a/controllers/database/pdb_controller.go +++ b/controllers/database/pdb_controller.go @@ -1121,7 +1121,7 @@ func (r *PDBReconciler) deletePDBInstance(req ctrl.Request, ctx context.Context, } values := map[string]string{ - "method": "DELETE", + "action": "KEEP", "getScript": strconv.FormatBool(*(pdb.Spec.GetScript))} if pdb.Spec.DropAction != "" { From 4a72573b0ab8e30764c95b7f81088ebf425cfffc Mon Sep 17 00:00:00 2001 From: Param Saini Date: Wed, 7 Dec 2022 21:55:05 -0800 Subject: [PATCH 513/628] Added Oracle Database Operator 0.2.1 --- .gitlab-ci.yml | 30 ------------------------------ oracle-database-operator.yaml | 2 +- 2 files changed, 1 insertion(+), 31 deletions(-) delete mode 100644 .gitlab-ci.yml diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml deleted file mode 100644 index cbef6241..00000000 --- a/.gitlab-ci.yml +++ /dev/null @@ -1,30 +0,0 @@ -build-operator: - stage: build - variables: - IMAGE: "$DOCKER_REPO:$CI_COMMIT_BRANCH" - OP_YAML: oracle-database-operator.yaml - script: - - echo $CI_COMMIT_TAG - - make docker-build IMG="$IMAGE" - - docker push "$IMAGE" - - newimage=$(docker inspect $IMAGE | python -c 'import json,sys; print json.load(sys.stdin)[0]["RepoDigests"][0]') - - echo $newimage - - docker rmi "$IMAGE" && docker system prune -f - - make operator-yaml IMG=$newimage - - if [ "$CI_COMMIT_BRANCH" != "master" ]; then sed -i "s/\(replicas.\) 3/\1 1/g" ./$OP_YAML; fi - - curl -s -n $ARTIFACTORY_REPO/$CI_COMMIT_BRANCH/$OP_YAML -T ./$OP_YAML - only: - variables: - - $CI_COMMIT_MESSAGE =~ /\#run-pipeline/ - - $CI_COMMIT_BRANCH =~ /master/ - - $CI_MERGE_REQUEST_ID != "" - except: - variables: - - $CI_COMMIT_MESSAGE =~ /\#skip-pipeline/ - - $CI_COMMIT_TAG != null - -cleanup: - stage: .post - script: - - echo "Clean up downloaded binaries" - - rm -rf bin/ diff --git a/oracle-database-operator.yaml b/oracle-database-operator.yaml index 309cbcb0..e92f842d 100644 --- a/oracle-database-operator.yaml +++ b/oracle-database-operator.yaml @@ -3224,7 +3224,7 @@ spec: - --enable-leader-election command: - /manager - image: lin.ocir.io/intsanjaysingh/mmalvezz/testppr/myoperator:latest + image: container-registry.oracle.com/database/operator:0.2.1 imagePullPolicy: Always name: manager ports: From 2847800b6caa13617f722f59353fe38f2829568b Mon Sep 17 00:00:00 2001 From: Param Saini Date: Wed, 7 Dec 2022 22:07:52 -0800 Subject: [PATCH 514/628] Added Oracle Database Operator 0.2.1 --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 1c091927..2a9595cf 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,7 @@ As part of Oracle's resolution to make Oracle Database Kubernetes-native (that is, observable and operable by Kubernetes), Oracle released _Oracle Database Operator for Kubernetes_ (`OraOperator` or the operator). OraOperator extends the Kubernetes API with custom resources and controllers for automating Oracle Database lifecycle management. -In this v0.2.0 release, `OraOperator` supports the following database configurations and infrastructure: +In this v0.2.1 release, `OraOperator` supports the following database configurations and infrastructure: * Oracle Autonomous Database on shared Oracle Cloud Infrastructure (OCI) (ADB-S) * Oracle Autonomous Database on dedicated Cloud infrastructure (ADB-D) @@ -32,7 +32,7 @@ The upcoming releases will support new configurations, operations and capabiliti ## Release Status -**CAUTION:** The current release of `OraOperator` (v0.2.0) is for development and testing only. DO NOT USE IN PRODUCTION. +**CAUTION:** The current release of `OraOperator` (v0.2.1) is for development and testing only. DO NOT USE IN PRODUCTION. This release has been installed and tested on the following Kubernetes platforms: @@ -69,7 +69,7 @@ Oracle strongly recommends that you ensure your system meets the following [Prer ``` --- **NOTE:** - The above command will also upgrade the existing v0.1.0 `OraOperator` installation to the latest version i.e. v0.2.0. + The above command will also upgrade the existing v0.2.0 `OraOperator` installation to the latest version i.e. v0.2.1. --- From 7c6ca30aac83542e97c2003a5508dfad179049f3 Mon Sep 17 00:00:00 2001 From: matteo malvezzi Date: Wed, 14 Dec 2022 16:57:30 +0100 Subject: [PATCH 515/628] bug 34836177 --- controllers/database/cdb_controller.go | 10 +- controllers/database/pdb_controller.go | 129 ++++++++++++++----------- 2 files changed, 76 insertions(+), 63 deletions(-) diff --git a/controllers/database/cdb_controller.go b/controllers/database/cdb_controller.go index 58eebf51..50778ccd 100644 --- a/controllers/database/cdb_controller.go +++ b/controllers/database/cdb_controller.go @@ -108,7 +108,7 @@ const CDBFinalizer = "database.oracle.com/CDBfinalizer" // - https://pkg.go.dev/sigs.k8s.io/controller-runtime@v0.9.2/pkg/reconcile func (r *CDBReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { - log := r.Log.WithValues("onpremdboperator", req.NamespacedName) + log := r.Log.WithValues("multitenantoperator", req.NamespacedName) log.Info("Reconcile requested") reconcilePeriod := r.Interval * time.Second @@ -449,10 +449,10 @@ func (r *CDBReconciler) createPodSpec(cdb *dbapi.CDB) corev1.PodSpec { Name: "ORACLE_HOST", Value: cdb.Spec.DBServer, }, - { - Name: "DBTNSURL", - Value: cdb.Spec.DBTnsurl, - }, + { + Name: "DBTNSURL", + Value: cdb.Spec.DBTnsurl, + }, { Name: "TLSCRT", Value: cdb.Spec.CDBTlsCrt.Secret.Key, diff --git a/controllers/database/pdb_controller.go b/controllers/database/pdb_controller.go index 0b52fd88..d40bd7ec 100644 --- a/controllers/database/pdb_controller.go +++ b/controllers/database/pdb_controller.go @@ -121,7 +121,7 @@ const PDBFinalizer = "database.oracle.com/PDBfinalizer" var tdePassword string var tdeSecret string - +var floodcontrol bool = false //+kubebuilder:rbac:groups=database.oracle.com,resources=pdbs,verbs=get;list;watch;create;update;patch;delete //+kubebuilder:rbac:groups=database.oracle.com,resources=pdbs/status,verbs=get;update;patch @@ -137,7 +137,7 @@ var tdeSecret string // For more details, check Reconcile and its Result here: // - https://pkg.go.dev/sigs.k8s.io/controller-runtime@v0.9.2/pkg/reconcile func (r *PDBReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { - log := r.Log.WithValues("onpremdboperator", req.NamespacedName) + log := r.Log.WithValues("multitenantoperator", req.NamespacedName) log.Info("Reconcile requested") reconcilePeriod := r.Interval * time.Second @@ -229,6 +229,8 @@ func (r *PDBReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.R err = r.getPDBState(ctx, req, pdb) case pdbPhaseMap: err = r.mapPDB(ctx, req, pdb) + case pdbPhaseFail: + err = r.mapPDB(ctx, req, pdb) default: log.Info("DEFAULT:", "Name", pdb.Name, "Phase", phase, "Status", strconv.FormatBool(pdb.Status.Status)) return requeueN, nil @@ -243,7 +245,7 @@ func (r *PDBReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.R } log.Info("Reconcile completed") - return requeueN, nil + return requeueY, nil } /************************************************* @@ -448,11 +450,11 @@ func (r *PDBReconciler) callAPI(ctx context.Context, req ctrl.Request, pdb *dbap } caCert := secret.Data[pdb.Spec.PDBTlsCat.Secret.Key] - /* - r.Recorder.Eventf(pdb, corev1.EventTypeWarning, "ORDSINFO", string(rsaKeyPEM)) - r.Recorder.Eventf(pdb, corev1.EventTypeWarning, "ORDSINFO", string(rsaCertPEM)) - r.Recorder.Eventf(pdb, corev1.EventTypeWarning, "ORDSINFO", string(caCert)) - */ + /* + r.Recorder.Eventf(pdb, corev1.EventTypeWarning, "ORDSINFO", string(rsaKeyPEM)) + r.Recorder.Eventf(pdb, corev1.EventTypeWarning, "ORDSINFO", string(rsaCertPEM)) + r.Recorder.Eventf(pdb, corev1.EventTypeWarning, "ORDSINFO", string(caCert)) + */ certificate, err := tls.X509KeyPair([]byte(rsaCertPEM), []byte(rsaKeyPEM)) if err != nil { @@ -535,18 +537,28 @@ func (r *PDBReconciler) callAPI(ctx context.Context, req ctrl.Request, pdb *dbap if resp.StatusCode == 404 { pdb.Status.ConnString = "" pdb.Status.Msg = pdb.Spec.PDBName + " not found" + } else { - pdb.Status.Msg = "ORDS Error - HTTP Status Code:" + strconv.Itoa(resp.StatusCode) + if floodcontrol == false { + pdb.Status.Msg = "ORDS Error - HTTP Status Code:" + strconv.Itoa(resp.StatusCode) + } + } + + if floodcontrol == false { + log.Info("ORDS Error - HTTP Status Code :"+strconv.Itoa(resp.StatusCode), "Err", string(bb)) } - log.Info("ORDS Error - HTTP Status Code :"+strconv.Itoa(resp.StatusCode), "Err", string(bb)) var apiErr ORDSError json.Unmarshal([]byte(bb), &apiErr) - r.Recorder.Eventf(pdb, corev1.EventTypeWarning, "ORDSError", "Failed: %s", apiErr.Message) + if floodcontrol == false { + r.Recorder.Eventf(pdb, corev1.EventTypeWarning, "ORDSError", "Failed: %s", apiErr.Message) + } //fmt.Printf("%+v", apiErr) //fmt.Println(string(bb)) + floodcontrol = true return "", errors.New("ORDS Error") } + floodcontrol = false defer resp.Body.Close() @@ -589,9 +601,8 @@ func (r *PDBReconciler) createPDB(ctx context.Context, req ctrl.Request, pdb *db log := r.Log.WithValues("createPDB", req.NamespacedName) var err error - var tdePassword string - var tdeSecret string - + var tdePassword string + var tdeSecret string cdb, err := r.getCDBResource(ctx, req, pdb) if err != nil { @@ -620,17 +631,17 @@ func (r *PDBReconciler) createPDB(ctx context.Context, req ctrl.Request, pdb *db "getScript": strconv.FormatBool(*(pdb.Spec.GetScript))} if *(pdb.Spec.TDEImport) { - tdePassword, err = r.getSecret(ctx, req, pdb, pdb.Spec.TDEPassword.Secret.SecretName, pdb.Spec.TDEPassword.Secret.Key) + tdePassword, err = r.getSecret(ctx, req, pdb, pdb.Spec.TDEPassword.Secret.SecretName, pdb.Spec.TDEPassword.Secret.Key) if err != nil { return err } - tdeSecret, err = r.getSecret(ctx, req, pdb, pdb.Spec.TDESecret.Secret.SecretName, pdb.Spec.TDESecret.Secret.Key) + tdeSecret, err = r.getSecret(ctx, req, pdb, pdb.Spec.TDESecret.Secret.SecretName, pdb.Spec.TDESecret.Secret.Key) if err != nil { return err } - tdeSecret = tdeSecret[:len(tdeSecret)-1] - tdePassword = tdeSecret[:len(tdePassword)-1] + tdeSecret = tdeSecret[:len(tdeSecret)-1] + tdePassword = tdeSecret[:len(tdePassword)-1] values["tdePassword"] = tdePassword values["tdeKeystorePath"] = pdb.Spec.TDEKeystorePath values["tdeSecret"] = tdeSecret @@ -652,12 +663,12 @@ func (r *PDBReconciler) createPDB(ctx context.Context, req ctrl.Request, pdb *db r.Recorder.Eventf(pdb, corev1.EventTypeNormal, "Created", "PDB '%s' created successfully", pdb.Spec.PDBName) - if cdb.Spec.DBServer != "" { - pdb.Status.ConnString = cdb.Spec.DBServer + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName - } else { - pdb.Status.ConnString = cdb.Spec.DBTnsurl - } - + if cdb.Spec.DBServer != "" { + pdb.Status.ConnString = cdb.Spec.DBServer + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName + } else { + pdb.Status.ConnString = cdb.Spec.DBTnsurl + } + log.Info("Created PDB Resource", "PDB Name", pdb.Spec.PDBName) r.getPDBState(ctx, req, pdb) return nil @@ -711,12 +722,11 @@ func (r *PDBReconciler) clonePDB(ctx context.Context, req ctrl.Request, pdb *dba r.Recorder.Eventf(pdb, corev1.EventTypeNormal, "Created", "PDB '%s' cloned successfully", pdb.Spec.PDBName) - if cdb.Spec.DBServer != "" { - pdb.Status.ConnString = cdb.Spec.DBServer + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName - } else { - pdb.Status.ConnString = cdb.Spec.DBTnsurl - } - + if cdb.Spec.DBServer != "" { + pdb.Status.ConnString = cdb.Spec.DBServer + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName + } else { + pdb.Status.ConnString = cdb.Spec.DBTnsurl + } log.Info("Cloned PDB successfully", "Source PDB Name", pdb.Spec.SrcPDBName, "Clone PDB Name", pdb.Spec.PDBName) r.getPDBState(ctx, req, pdb) @@ -731,8 +741,8 @@ func (r *PDBReconciler) plugPDB(ctx context.Context, req ctrl.Request, pdb *dbap log := r.Log.WithValues("plugPDB", req.NamespacedName) var err error - var tdePassword string - var tdeSecret string + var tdePassword string + var tdeSecret string cdb, err := r.getCDBResource(ctx, req, pdb) if err != nil { @@ -763,9 +773,9 @@ func (r *PDBReconciler) plugPDB(ctx context.Context, req ctrl.Request, pdb *dbap if err != nil { return err } - - tdeSecret = tdeSecret[:len(tdeSecret)-1] - tdePassword = tdeSecret[:len(tdePassword)-1] + + tdeSecret = tdeSecret[:len(tdeSecret)-1] + tdePassword = tdeSecret[:len(tdePassword)-1] values["tdePassword"] = tdePassword values["tdeKeystorePath"] = pdb.Spec.TDEKeystorePath values["tdeSecret"] = tdeSecret @@ -790,11 +800,11 @@ func (r *PDBReconciler) plugPDB(ctx context.Context, req ctrl.Request, pdb *dbap r.Recorder.Eventf(pdb, corev1.EventTypeNormal, "Created", "PDB '%s' plugged successfully", pdb.Spec.PDBName) - if cdb.Spec.DBServer != "" { - pdb.Status.ConnString = cdb.Spec.DBServer + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName - } else { - pdb.Status.ConnString = cdb.Spec.DBTnsurl - } + if cdb.Spec.DBServer != "" { + pdb.Status.ConnString = cdb.Spec.DBServer + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName + } else { + pdb.Status.ConnString = cdb.Spec.DBTnsurl + } log.Info("Successfully plugged PDB", "PDB Name", pdb.Spec.PDBName) r.getPDBState(ctx, req, pdb) @@ -809,9 +819,8 @@ func (r *PDBReconciler) unplugPDB(ctx context.Context, req ctrl.Request, pdb *db log := r.Log.WithValues("unplugPDB", req.NamespacedName) var err error - var tdePassword string - var tdeSecret string - + var tdePassword string + var tdeSecret string cdb, err := r.getCDBResource(ctx, req, pdb) if err != nil { @@ -834,8 +843,8 @@ func (r *PDBReconciler) unplugPDB(ctx context.Context, req ctrl.Request, pdb *db return err } - tdeSecret = tdeSecret[:len(tdeSecret)-1] - tdePassword = tdeSecret[:len(tdePassword)-1] + tdeSecret = tdeSecret[:len(tdeSecret)-1] + tdePassword = tdeSecret[:len(tdePassword)-1] values["tdePassword"] = tdePassword values["tdeKeystorePath"] = pdb.Spec.TDEKeystorePath values["tdeSecret"] = tdeSecret @@ -843,7 +852,7 @@ func (r *PDBReconciler) unplugPDB(ctx context.Context, req ctrl.Request, pdb *db } url := "https://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdb.Spec.PDBName + "/" - log.Info("CallAPI(url)", "url", url) + log.Info("CallAPI(url)", "url", url) pdb.Status.Phase = pdbPhaseUnplug pdb.Status.Msg = "Waiting for PDB to be unplugged" @@ -888,6 +897,10 @@ func (r *PDBReconciler) modifyPDB(ctx context.Context, req ctrl.Request, pdb *db err = r.getPDBState(ctx, req, pdb) if err != nil { + if apierrors.IsNotFound(err) { + log.Info("Warning PDB does not exist", "PDB Name", pdb.Spec.PDBName) + return nil + } return err } @@ -923,12 +936,11 @@ func (r *PDBReconciler) modifyPDB(ctx context.Context, req ctrl.Request, pdb *db r.Recorder.Eventf(pdb, corev1.EventTypeNormal, "Modified", "PDB '%s' modified successfully", pdb.Spec.PDBName) - if cdb.Spec.DBServer != "" { - pdb.Status.ConnString = cdb.Spec.DBServer + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName - } else { - pdb.Status.ConnString = cdb.Spec.DBTnsurl - } - + if cdb.Spec.DBServer != "" { + pdb.Status.ConnString = cdb.Spec.DBServer + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName + } else { + pdb.Status.ConnString = cdb.Spec.DBTnsurl + } log.Info("Successfully modified PDB state", "PDB Name", pdb.Spec.PDBName) r.getPDBState(ctx, req, pdb) @@ -961,6 +973,8 @@ func (r *PDBReconciler) getPDBState(ctx context.Context, req ctrl.Request, pdb * if err != nil { pdb.Status.OpenMode = "UNKNOWN" + pdb.Status.Msg = "CHECK PDB STATUS" + pdb.Status.Status = false return err } @@ -1016,12 +1030,11 @@ func (r *PDBReconciler) mapPDB(ctx context.Context, req ctrl.Request, pdb *dbapi pdb.Status.OpenMode = objmap["open_mode"].(string) pdb.Status.TotalSize = fmt.Sprintf("%.2f", totSizeInGB) + "G" - if cdb.Spec.DBServer != "" { - pdb.Status.ConnString = cdb.Spec.DBServer + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName - } else { - pdb.Status.ConnString = cdb.Spec.DBTnsurl - } - + if cdb.Spec.DBServer != "" { + pdb.Status.ConnString = cdb.Spec.DBServer + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName + } else { + pdb.Status.ConnString = cdb.Spec.DBTnsurl + } log.Info("Successfully mapped PDB to Kubernetes resource", "PDB Name", pdb.Spec.PDBName) return nil From 2370f0b68e5869ffee3d822639173ce8b31ea566 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Fri, 16 Dec 2022 09:04:47 -0500 Subject: [PATCH 516/628] update libs --- go.mod | 8 +- go.sum | 295 +-------------------------------------------------------- 2 files changed, 6 insertions(+), 297 deletions(-) diff --git a/go.mod b/go.mod index 9e37f7a3..f33b48a9 100644 --- a/go.mod +++ b/go.mod @@ -6,14 +6,14 @@ require ( github.com/go-logr/logr v1.2.3 github.com/onsi/ginkgo/v2 v2.5.0 github.com/onsi/gomega v1.24.1 - go.uber.org/zap v1.21.0 + github.com/oracle/oci-go-sdk/v65 v65.26.1 + go.uber.org/zap v1.23.0 gopkg.in/yaml.v2 v2.4.0 k8s.io/api v0.25.4 k8s.io/apimachinery v0.25.4 k8s.io/client-go v0.25.4 sigs.k8s.io/controller-runtime v0.13.1 sigs.k8s.io/yaml v1.3.0 - github.com/oracle/oci-go-sdk/v65 v65.26.1 ) require ( @@ -25,7 +25,6 @@ require ( github.com/emicklei/go-restful/v3 v3.8.0 // indirect github.com/evanphx/json-patch/v5 v5.6.0 // indirect github.com/fsnotify/fsnotify v1.5.4 // indirect - github.com/go-logr/zapr v1.2.3 // indirect github.com/go-openapi/jsonpointer v0.19.5 // indirect github.com/go-openapi/jsonreference v0.19.5 // indirect github.com/go-openapi/swag v0.19.14 // indirect @@ -75,8 +74,9 @@ require ( require ( cloud.google.com/go/compute v1.2.0 // indirect github.com/evanphx/json-patch v5.6.0+incompatible // indirect + github.com/go-logr/zapr v1.2.3 // indirect github.com/google/gofuzz v1.2.0 // indirect github.com/google/uuid v1.3.0 // indirect go.uber.org/atomic v1.9.0 // indirect go.uber.org/multierr v1.7.0 // indirect -) \ No newline at end of file +) diff --git a/go.sum b/go.sum index fc9f98e0..38d62bf3 100644 --- a/go.sum +++ b/go.sum @@ -26,7 +26,6 @@ cloud.google.com/go v0.93.3/go.mod h1:8utlLll2EF5XMAV15woO4lSbWQlk8rer9aLOfLh7+Y cloud.google.com/go v0.94.1/go.mod h1:qAlAugsXlC+JWO+Bke5vCtc9ONxjQT3drlTTnAplMW4= cloud.google.com/go v0.97.0/go.mod h1:GF7l59pYBVlXQIBLx3a761cZ41F9bBH3JUlihCt2Udc= cloud.google.com/go v0.99.0/go.mod h1:w0Xx2nLzqWJPuozYQX+hFfCSI8WioryfRDzkoI/Y2ZA= -cloud.google.com/go v0.100.2 h1:t9Iw5QH5v4XtlEQaCtUY7x6sCABps8sW0acw7e2WQ6Y= cloud.google.com/go v0.100.2/go.mod h1:4Xra9TjzAeYHrl5+oeLlzbM2k3mjVhZh4UqTZ//w99A= cloud.google.com/go/bigquery v1.0.1/go.mod h1:i/xbL2UlR5RvWAURpBYZTtm/cXjCha9lbfbpx4poX+o= cloud.google.com/go/bigquery v1.3.0/go.mod h1:PjpwJnslEMmckchkHFfq+HTD2DmtT67aNFKH1/VBDHE= @@ -39,7 +38,6 @@ cloud.google.com/go/compute v1.2.0 h1:EKki8sSdvDU0OO9mAXGwPXOTOgPz2l08R0/IutDH11 cloud.google.com/go/compute v1.2.0/go.mod h1:xlogom/6gr8RJGBe7nT2eGsQYAFUbbv8dbC29qE3Xmw= cloud.google.com/go/datastore v1.0.0/go.mod h1:LXYbyblFSglQ5pkeyhO+Qmw7ukd3C+pD7TKLgZqpHYE= cloud.google.com/go/datastore v1.1.0/go.mod h1:umbIZjpQpHh4hmRpGhH4tLFup+FVzqBi1b3c64qFpCk= -cloud.google.com/go/firestore v1.1.0/go.mod h1:ulACoGHTpvq5r8rxGJ4ddJZBZqakUQqClKRT5SZwBmk= cloud.google.com/go/pubsub v1.0.1/go.mod h1:R0Gpsv3s54REJCy4fxDixWD93lHJMoZTyQ2kNxGRt3I= cloud.google.com/go/pubsub v1.1.0/go.mod h1:EwwdRX2sKPjnvnqCa270oGRyludottCI76h+R3AArQw= cloud.google.com/go/pubsub v1.2.0/go.mod h1:jhfEVHT8odbXTkndysNHCcx0awwzvfOlguIAii9o8iA= @@ -50,20 +48,8 @@ cloud.google.com/go/storage v1.6.0/go.mod h1:N7U0C8pVQ/+NIKOBQyamJIeKQKkZ+mxpohl cloud.google.com/go/storage v1.8.0/go.mod h1:Wv1Oy7z6Yz3DshWRJFhqM/UCfaWIRTdp0RXyy7KQOVs= cloud.google.com/go/storage v1.10.0/go.mod h1:FLPqc6j+Ki4BU591ie1oL6qBQGu2Bl/tZ9ullr3+Kg0= dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU= -github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1/go.mod h1:xomTg63KZ2rFqZQzSB4Vz2SUXa1BpHTVz9L5PTmPC4E= -github.com/Azure/go-autorest v14.2.0+incompatible/go.mod h1:r+4oMnoxhatjLLJ6zxSWATqVooLgysK6ZNox3g/xq24= -github.com/Azure/go-autorest/autorest v0.11.27/go.mod h1:7l8ybrIdUmGqZMTD0sRtAr8NvbHjfofbf8RSP2q7w7U= -github.com/Azure/go-autorest/autorest/adal v0.9.18/go.mod h1:XVVeme+LZwABT8K5Lc3hA4nAe8LDBVle26gTrguhhPQ= -github.com/Azure/go-autorest/autorest/adal v0.9.20/go.mod h1:XVVeme+LZwABT8K5Lc3hA4nAe8LDBVle26gTrguhhPQ= -github.com/Azure/go-autorest/autorest/date v0.3.0/go.mod h1:BI0uouVdmngYNUzGWeSYnokU+TrmwEsOqdt8Y6sso74= -github.com/Azure/go-autorest/autorest/mocks v0.4.1/go.mod h1:LTp+uSrOhSkaKrUy935gNZuuIPPVsHlr9DSOxSayd+k= -github.com/Azure/go-autorest/autorest/mocks v0.4.2/go.mod h1:Vy7OitM9Kei0i1Oj+LvyAWMXJHeKH1MVlzFugfVrmyU= -github.com/Azure/go-autorest/logger v0.2.1/go.mod h1:T9E3cAhj2VqvPOtCYAvby9aBXkZmbF5NWuPV8+WeEW8= -github.com/Azure/go-autorest/tracing v0.6.0/go.mod h1:+vhtPC754Xsa23ID7GlGsrdKBpUA79WCAKPPZVC2DeU= github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU= github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo= -github.com/NYTimes/gziphandler v0.0.0-20170623195520-56545f4a5d46/go.mod h1:3wb06e3pkSAbeQ52E9H9iFoQsEEwGN64994WTCIhntQ= -github.com/NYTimes/gziphandler v1.1.1/go.mod h1:n/CVRwUEOgIxrgPvAQhUUr9oeUtvrhMomdKFjzJNB0c= github.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAElWljhcU= github.com/PuerkitoBio/purell v1.1.1 h1:WEQqlqaGbrPkxLJWfBwQmfEAE1Z7ONdDLqrN38tNFfI= github.com/PuerkitoBio/purell v1.1.1/go.mod h1:c11w/QuzBsJSee3cPx9rAFu61PvFxuPbtSwDGJws/X0= @@ -75,27 +61,14 @@ github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRF github.com/alecthomas/units v0.0.0-20190717042225-c3de453c63f4/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0= github.com/alecthomas/units v0.0.0-20190924025748-f65c72e2690d/go.mod h1:rBZYJk541a8SKzHPHnH3zbiI+7dagKZ0cgpgrD7Fyho= github.com/antihax/optional v1.0.0/go.mod h1:uupD/76wgC+ih3iEmQUL+0Ugr19nfwCT1kdvxnR2qWY= -github.com/antlr/antlr4/runtime/Go/antlr v0.0.0-20220418222510-f25a4f6275ed/go.mod h1:F7bn7fEU90QkQ3tnmaTx3LTKLEDqnwWODIYppRQ5hnY= -github.com/armon/circbuf v0.0.0-20150827004946-bbbad097214e/go.mod h1:3U/XgcO3hCbHZ8TKRvWD2dDTCfh9M9ya+I9JpbB7O8o= -github.com/armon/go-metrics v0.0.0-20180917152333-f0300d1749da/go.mod h1:Q73ZrmVTwzkszR9V5SSuryQ31EELlFMUz1kKyl939pY= -github.com/armon/go-radix v0.0.0-20180808171621-7fddfc383310/go.mod h1:ufUuZ+zHj4x4TnLV4JWEpy2hxWSpsRywHrMgIH9cCH8= github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 h1:0CwZNZbxp69SHPdPJAN/hZIm0C4OItdklCFmMRWYpio= -github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5/go.mod h1:wHh0iHkYZB8zMSxRWpUBQtwG5a7fFgvEO+odwuTv2gs= -github.com/asaskevich/govalidator v0.0.0-20190424111038-f61b66f89f4a/go.mod h1:lB+ZfQJz7igIIfQNfa7Ml4HSf2uFQQRzpGGRXenZAgY= -github.com/benbjohnson/clock v1.0.3/go.mod h1:bGMdMPoPVvcYyt1gHDf4J2KE153Yf9BuiUKYMaxlTDM= github.com/benbjohnson/clock v1.1.0 h1:Q92kusRqC1XV2MjkWETPvjJVqKetz1OzxZB7mHJLju8= github.com/benbjohnson/clock v1.1.0/go.mod h1:J11/hYXuz8f4ySSvYwY0FKfm+ezbsZBKZxNJlLklBHA= github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q= github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8= github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= -github.com/bgentry/speakeasy v0.1.0/go.mod h1:+zsyZBPWlz7T6j88CTgSN5bM796AkVf0kBD4zp0CCIs= -github.com/bketelsen/crypt v0.0.3-0.20200106085610-5cbc8cc4026c/go.mod h1:MKsuJmJgSg28kpZDP6UIiPt0e0Oz0kqKNGyRaWEPv84= -github.com/blang/semver/v4 v4.0.0/go.mod h1:IbckMUScFkM3pff0VJDNKRiT6TG/YpiHIM2yvyW5YoQ= github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU= -github.com/certifi/gocertifi v0.0.0-20191021191039-0944d244cd40/go.mod h1:sGbDF6GwGcLpkNXPUTkMRoywsNa/ol15pxFe6ERfguA= -github.com/certifi/gocertifi v0.0.0-20200922220541-2c3bb06c6054/go.mod h1:sGbDF6GwGcLpkNXPUTkMRoywsNa/ol15pxFe6ERfguA= -github.com/cespare/xxhash v1.1.0 h1:a6HrQnmkObjyL+Gs60czilIUGqrzKutQD6XZog3p+ko= github.com/cespare/xxhash v1.1.0/go.mod h1:XrSqR1VqqWfGrhpAt58auRo0WTKS1nRRg3ghfAqPWnc= github.com/cespare/xxhash/v2 v2.1.1/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= github.com/cespare/xxhash/v2 v2.1.2 h1:YRXhKfTDauu4ajMg1TPgFO5jnlC2HCbmLXMcTG5cbYE= @@ -107,34 +80,13 @@ github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDk github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc= github.com/cncf/udpa/go v0.0.0-20200629203442-efcf912fb354/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk= github.com/cncf/udpa/go v0.0.0-20201120205902-5459f2c99403/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk= -github.com/cncf/udpa/go v0.0.0-20210930031921-04548b0d99d4/go.mod h1:6pvJx4me5XPnfI9Z40ddWsdw2W/uZgQLFXToKeRcDiI= github.com/cncf/xds/go v0.0.0-20210312221358-fbca930ec8ed/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs= -github.com/cncf/xds/go v0.0.0-20210922020428-25de7278fc84/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs= -github.com/cncf/xds/go v0.0.0-20211001041855-01bcc9b48dfe/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs= -github.com/cncf/xds/go v0.0.0-20211011173535-cb28da3451f1/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs= -github.com/cockroachdb/datadriven v0.0.0-20200714090401-bf6692d28da5/go.mod h1:h6jFvWxBdQXxjopDMZyH2UVceIRfR84bdzbkoKrsWNo= -github.com/cockroachdb/errors v1.2.4/go.mod h1:rQD95gz6FARkaKkQXUksEje/d9a6wBJoCr5oaCLELYA= -github.com/cockroachdb/logtags v0.0.0-20190617123548-eb05cc24525f/go.mod h1:i/u985jwjWRlyHXQbwatDASoW0RMlZ/3i9yJHE2xLkI= -github.com/coreos/bbolt v1.3.2/go.mod h1:iRUV2dpdMOn7Bo10OQBFzIJO9kkE559Wcmn+qkEiiKk= -github.com/coreos/etcd v3.3.13+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE= -github.com/coreos/go-oidc v2.1.0+incompatible/go.mod h1:CgnwVTmzoESiwO9qyAFEMiHoZ1nMCKZlZ9V6mm3/LKc= -github.com/coreos/go-semver v0.3.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk= -github.com/coreos/go-systemd v0.0.0-20190321100706-95778dfbb74e/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4= -github.com/coreos/go-systemd/v22 v22.3.2/go.mod h1:Y58oyj3AT4RCenI/lSvhwexgC+NSVTIJ3seZv2GcEnc= -github.com/coreos/pkg v0.0.0-20180928190104-399ea9e2e55f/go.mod h1:E3G3o1h8I7cfcXa63jLwjI0eiQQMgzzUDFVpN/nH/eA= -github.com/cpuguy83/go-md2man/v2 v2.0.0/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU= -github.com/cpuguy83/go-md2man/v2 v2.0.1/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o= github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= -github.com/creack/pty v1.1.11/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= -github.com/dgrijalva/jwt-go v3.2.0+incompatible/go.mod h1:E3ru+11k8xSBh+hMPgOLZmtrrCbhqsmaPHjLKYnJCaQ= -github.com/dgryski/go-sip13 v0.0.0-20181026042036-e10d5fee7954/go.mod h1:vAd38F8PWV+bWy6jNmig1y/TA+kYO4g3RSRF0IAv0no= github.com/docopt/docopt-go v0.0.0-20180111231733-ee0de3bc6815/go.mod h1:WwZ+bS3ebgob9U8Nd0kOddGdZWjyMGR8Wziv+TBNwSE= -github.com/dustin/go-humanize v1.0.0/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk= github.com/elazarl/goproxy v0.0.0-20180725130230-947c36da3153 h1:yUdfgN0XgIJw7foRItutHYUIhlcKzcSf5vDpdhQAKTc= -github.com/elazarl/goproxy v0.0.0-20180725130230-947c36da3153/go.mod h1:/Zj4wYkgs4iZTTu3o/KG3Itv/qCCa8VVMlb3i9OVuzc= github.com/emicklei/go-restful/v3 v3.8.0 h1:eCZ8ulSerjdAiaNpF7GxXIE7ZCMo1moN1qX+S609eVw= github.com/emicklei/go-restful/v3 v3.8.0/go.mod h1:6n3XBCmQQb25CM2LCACGz8ukIrRry+4bhvbpWn3mrbc= github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4= @@ -144,23 +96,14 @@ github.com/envoyproxy/go-control-plane v0.9.7/go.mod h1:cwu0lG7PUMfa9snN8LXBig5y github.com/envoyproxy/go-control-plane v0.9.9-0.20201210154907-fd9021fe5dad/go.mod h1:cXg6YxExXjJnVBQHBLXeUAgxn2UodCpnH306RInaBQk= github.com/envoyproxy/go-control-plane v0.9.9-0.20210217033140-668b12f5399d/go.mod h1:cXg6YxExXjJnVBQHBLXeUAgxn2UodCpnH306RInaBQk= github.com/envoyproxy/go-control-plane v0.9.9-0.20210512163311-63b5d3c536b0/go.mod h1:hliV/p42l8fGbc6Y9bQ70uLwIvmJyVE5k4iMKlh8wCQ= -github.com/envoyproxy/go-control-plane v0.10.2-0.20220325020618-49ff273808a1/go.mod h1:KJwIaB5Mv44NWtYuAOFCVOjcI94vtpEz2JU/D2v6IjE= github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c= github.com/evanphx/json-patch v0.5.2/go.mod h1:ZWS5hhDbVDyob71nXKNL0+PWn6ToqBHMikGIFbs31qQ= -github.com/evanphx/json-patch v4.12.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk= github.com/evanphx/json-patch v5.6.0+incompatible h1:jBYDEEiFBPxA0v50tFdvOzQQTCvpL6mnFh5mB2/l16U= github.com/evanphx/json-patch v5.6.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk= github.com/evanphx/json-patch/v5 v5.6.0 h1:b91NhWfaz02IuVxO9faSllyAtNXHMPkC5J8sJCLunww= github.com/evanphx/json-patch/v5 v5.6.0/go.mod h1:G79N1coSVB93tBe7j6PhzjmR3/2VvlbKOFpnXhI9Bw4= -github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4= -github.com/felixge/httpsnoop v1.0.1/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U= -github.com/form3tech-oss/jwt-go v3.2.3+incompatible/go.mod h1:pbq4aXjuKjdthFRnoDwaVPLA+WlJuPGy+QneDUgJi2k= -github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo= -github.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ= github.com/fsnotify/fsnotify v1.5.4 h1:jRbGcIw6P2Meqdwuo0H1p6JVLbL5DHKAKlYndzMwVZI= github.com/fsnotify/fsnotify v1.5.4/go.mod h1:OVB6XrOHzAwXMpEM7uPOzcehqUV2UqJxmVXmkdnm1bU= -github.com/getkin/kin-openapi v0.76.0/go.mod h1:660oXbgy5JFMKreazJaQTw7o+X00qeSyhcnluiMv+Xg= -github.com/getsentry/raven-go v0.2.0/go.mod h1:KungGk8q33+aIAZUIVWZDr2OfAEBsO49PX4NzFV5kcQ= github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04= github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU= github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8= @@ -172,7 +115,6 @@ github.com/go-logfmt/logfmt v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9 github.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V4qmtdjCk= github.com/go-logfmt/logfmt v0.5.0/go.mod h1:wCYkCAKZfumFQihp8CzCvQ3paCTfi41vtzG1KdI/P7A= github.com/go-logr/logr v0.1.0/go.mod h1:ixOQHD9gLJUVQQ2ZOR7zLEifBX6tGkNJF4QyIY7sIas= -github.com/go-logr/logr v0.2.0/go.mod h1:z6/tIYblkpsD+a4lm/fGIIU9mZ+XfAiaFtq7xTgseGU= github.com/go-logr/logr v1.2.0/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A= github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A= github.com/go-logr/logr v1.2.3 h1:2DntVwHkVopvECVRSlL5PSo9eG+cAkDCuckLubN+rq0= @@ -182,27 +124,18 @@ github.com/go-logr/zapr v1.2.3/go.mod h1:eIauM6P8qSvTw5o2ez6UEAfGjQKrxQTl5EoK+Qa github.com/go-openapi/jsonpointer v0.19.3/go.mod h1:Pl9vOtqEWErmShwVjC8pYs9cog34VGT37dQOVbmoatg= github.com/go-openapi/jsonpointer v0.19.5 h1:gZr+CIYByUqjcgeLXnQu2gHYQC9o73G2XUeOFYEICuY= github.com/go-openapi/jsonpointer v0.19.5/go.mod h1:Pl9vOtqEWErmShwVjC8pYs9cog34VGT37dQOVbmoatg= -github.com/go-openapi/jsonreference v0.19.3/go.mod h1:rjx6GuL8TTa9VaixXglHmQmIL98+wF9xc8zWvFonSJ8= github.com/go-openapi/jsonreference v0.19.5 h1:1WJP/wi4OjB4iV8KVbH73rQaoialJrqv8gitZLxGLtM= github.com/go-openapi/jsonreference v0.19.5/go.mod h1:RdybgQwPxbL4UEjuAruzK1x3nE69AqPYEJeo/TWfEeg= github.com/go-openapi/swag v0.19.5/go.mod h1:POnQmlKehdgb5mhVOsnJFsivZCEZ/vjK9gh66Z9tfKk= github.com/go-openapi/swag v0.19.14 h1:gm3vOOXfiuw5i9p5N9xJvfjvuofpyvLA9Wr6QfK5Fng= github.com/go-openapi/swag v0.19.14/go.mod h1:QYRuS/SOXUCsnplDa677K7+DxSOj6IPNl/eQntq43wQ= github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY= -github.com/go-task/slim-sprig v0.0.0-20210107165309-348f09dbbbc0/go.mod h1:fyg7847qk6SyHyPtNmDHnmrv/HOrqktSC+C9fM+CJOE= -github.com/godbus/dbus/v5 v5.0.4/go.mod h1:xhWf0FNVPg57R7Z0UbKHbJfkEywrmjJnf7w5xrFpKfA= github.com/gofrs/flock v0.8.1 h1:+gYjHKf32LDeiEEFhQaotPbLuUXjY5ZqxKgXy7n59aw= github.com/gofrs/flock v0.8.1/go.mod h1:F1TvTiK9OcQqauNUHlbJvyl9Qa1QvF/gOUDKA14jxHU= github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ= -github.com/gogo/protobuf v1.2.1/go.mod h1:hp+jE20tsWTFYpLwKvXlhS1hjn+gTNwPg2I6zVXpSg4= -github.com/gogo/protobuf v1.3.1/go.mod h1:SlYgWuQ5SjCEi6WLHjHCa1yvBfUnHcTbrrZtXPKa29o= github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q= github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q= -github.com/golang-jwt/jwt/v4 v4.0.0/go.mod h1:/xlHOz8bRuivTWchD4jCa+NbatV+wEUSzwAxVc6locg= -github.com/golang-jwt/jwt/v4 v4.2.0/go.mod h1:/xlHOz8bRuivTWchD4jCa+NbatV+wEUSzwAxVc6locg= github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q= -github.com/golang/glog v1.0.0/go.mod h1:EWib/APOK0SL3dFbYqvxE3UYd8E6s1ouQ7iEp/0LWV4= -github.com/golang/groupcache v0.0.0-20190129154638-5b532d6fd5ef/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= @@ -238,8 +171,6 @@ github.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiu github.com/golang/snappy v0.0.3/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q= github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= -github.com/google/btree v1.0.1/go.mod h1:xXMiIv4Fb/0kKde4SpL7qlzvu5cMJDRkFDxJfI9uaxA= -github.com/google/cel-go v0.12.4/go.mod h1:Av7CU6r6X3YmcHR9GXqVDaEJYfEtSxl6wvIjUQTriCw= github.com/google/gnostic v0.5.7-v3refs h1:FhTMOKj2VhjpouxvWJAV1TL304uMlb9zcDqkl6cEI54= github.com/google/gnostic v0.5.7-v3refs/go.mod h1:73MKFl6jIHelAJNaBGFzt3SPtZULs9dYrGFt8OiIsHQ= github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M= @@ -255,11 +186,9 @@ github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/ github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.6/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.7/go.mod h1:n+brtR0CgQNWTVd5ZUFpTBC8YFBDLK/h/bpaJ8/DtOE= -github.com/google/go-cmp v0.5.8/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38= github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= -github.com/google/gofuzz v1.1.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= github.com/google/gofuzz v1.2.0 h1:xRy4A+RhZaiKjJ1bPfwQ8sedCA+YS2YcCHW6ec7JMi0= github.com/google/gofuzz v1.2.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs= @@ -277,7 +206,6 @@ github.com/google/pprof v0.0.0-20201023163331-3e6fc7fc9c4c/go.mod h1:kpwsk12EmLe github.com/google/pprof v0.0.0-20201203190320-1bf35d6f28c2/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= github.com/google/pprof v0.0.0-20210122040257-d980be63207e/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= github.com/google/pprof v0.0.0-20210226084205-cbba55b83ad5/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= -github.com/google/pprof v0.0.0-20210407192527-94a9f03dee38/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= github.com/google/pprof v0.0.0-20210601050228-01bbb1931b22/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= github.com/google/pprof v0.0.0-20210609004039-a478d1d731e9/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= github.com/google/pprof v0.0.0-20210720184732-4bb14d4b1be1/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= @@ -289,45 +217,15 @@ github.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+ github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk= github.com/googleapis/gax-go/v2 v2.1.0/go.mod h1:Q3nei7sK6ybPYH7twZdmQpAd1MKb7pfu6SK+H1/DsU0= github.com/googleapis/gax-go/v2 v2.1.1/go.mod h1:hddJymUZASv3XPyGkUpKj8pPO47Rmb0eJc8R6ouapiM= -github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY= -github.com/gorilla/mux v1.8.0/go.mod h1:DVbg23sWSpFRCP0SfiEN6jmj59UnW/n46BH5rLB71So= github.com/gorilla/websocket v1.4.2/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE= -github.com/gregjones/httpcache v0.0.0-20180305231024-9cad4c3443a7/go.mod h1:FecbI9+v66THATjSRHfNgh1IVFe/9kFxbXtjV0ctIMA= -github.com/grpc-ecosystem/go-grpc-middleware v1.0.0/go.mod h1:FiyG127CGDf3tlThmgyCl78X/SZQqEOJBCDaAfeWzPs= -github.com/grpc-ecosystem/go-grpc-middleware v1.3.0/go.mod h1:z0ButlSOZa5vEBq9m2m2hlwIgKw+rp3sdCBRoJY+30Y= -github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0/go.mod h1:8NvIoxWQoOIhqOTXgfV/d3M/q6VIi02HzZEHgUlZvzk= -github.com/grpc-ecosystem/grpc-gateway v1.9.0/go.mod h1:vNeuVxBJEsws4ogUvrchl83t/GYV9WGTSLVdBhOQFDY= github.com/grpc-ecosystem/grpc-gateway v1.16.0/go.mod h1:BDjrQk3hbvj6Nolgz8mAMFbcEtjT1g+wF4CSlocrBnw= -github.com/hashicorp/consul/api v1.1.0/go.mod h1:VmuI/Lkw1nC05EYQWNKwWGbkg+FbDBtguAZLlVdkD9Q= -github.com/hashicorp/consul/sdk v0.1.1/go.mod h1:VKf9jXwCTEY1QZP2MOLRhb5i/I/ssyNV1vwHyQBF0x8= -github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4= -github.com/hashicorp/go-cleanhttp v0.5.1/go.mod h1:JpRdi6/HCYpAwUzNwuwqhbovhLtngrth3wmdIIUrZ80= -github.com/hashicorp/go-immutable-radix v1.0.0/go.mod h1:0y9vanUI8NX6FsYoO3zeMjhV/C5i9g4Q3DwcSNZ4P60= -github.com/hashicorp/go-msgpack v0.5.3/go.mod h1:ahLV/dePpqEmjfWmKiqvPkv/twdG7iPBM1vqhUKIvfM= -github.com/hashicorp/go-multierror v1.0.0/go.mod h1:dHtQlpGsu+cZNNAkkCN/P3hoUDHhCYQXV3UM06sGGrk= -github.com/hashicorp/go-rootcerts v1.0.0/go.mod h1:K6zTfqpRlCUIjkwsN4Z+hiSfzSTQa6eBIzfwKfwNnHU= -github.com/hashicorp/go-sockaddr v1.0.0/go.mod h1:7Xibr9yA9JjQq1JpNB2Vw7kxv8xerXegt+ozgdvDeDU= -github.com/hashicorp/go-syslog v1.0.0/go.mod h1:qPfqrKkXGihmCqbJM2mZgkZGvKG1dFdvsLplgctolz4= -github.com/hashicorp/go-uuid v1.0.0/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro= -github.com/hashicorp/go-uuid v1.0.1/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro= -github.com/hashicorp/go.net v0.0.1/go.mod h1:hjKkEWcCURg++eb33jQU7oqQcI9XDCnUzHA0oac0k90= github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8= github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8= -github.com/hashicorp/hcl v1.0.0/go.mod h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ= -github.com/hashicorp/logutils v1.0.0/go.mod h1:QIAnNjmIWmVIIkWDTG1z5v++HQmx9WQRO+LraFDTW64= -github.com/hashicorp/mdns v1.0.0/go.mod h1:tL+uN++7HEJ6SQLQ2/p+z2pH24WQKWjBPkE0mNTz8vQ= -github.com/hashicorp/memberlist v0.1.3/go.mod h1:ajVTdAv/9Im8oMAAj5G31PhhMCZJV2pPBoIllUwCN7I= -github.com/hashicorp/serf v0.8.2/go.mod h1:6hOLApaqBFA1NXqRQAsxw9QxuDEvNxSQRwA/JwenrHc= -github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU= github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc= github.com/ianlancetaylor/demangle v0.0.0-20200824232613-28f6c0f3b639/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc= -github.com/imdario/mergo v0.3.6/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA= github.com/imdario/mergo v0.3.12 h1:b6R2BslTbIEToALKP7LxUvijTsNI9TAe80pLWN2g/HU= github.com/imdario/mergo v0.3.12/go.mod h1:jmQim1M+e3UYxmgPu/WyfjB3N3VflVyUjjjwH0dnCYA= -github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8= github.com/jessevdk/go-flags v1.4.0/go.mod h1:4FA24M0QyGHXBuZZK/XkWh8h0e1EYbRYJSGM75WSRxI= -github.com/jonboulle/clockwork v0.1.0/go.mod h1:Ii8DK3G1RaLaWxj9trq07+26W01tbo22gdxWY5EU2bo= -github.com/jonboulle/clockwork v0.2.2/go.mod h1:Pkfl5aHPm1nk2H9h0bjmnJD/BcgbGXUBGnn1kMkgxc8= github.com/josharian/intern v1.0.0 h1:vlS4z54oSdjm0bgjRigI+G1HpF+tI+9rE5LLzOg8HmY= github.com/josharian/intern v1.0.0/go.mod h1:5DoeVV0s6jJacbCEi61lwdGj/aVlrQvzHFFd8Hwg//Y= github.com/jpillora/backoff v1.0.0/go.mod h1:J/6gKK9jxlEcS3zixgDgUAsiuZ7yrSoa/FX5e0EB2j4= @@ -338,16 +236,12 @@ github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnr github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo= github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU= github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk= -github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU= github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w= github.com/julienschmidt/httprouter v1.3.0/go.mod h1:JR6WtHb+2LUe8TCKY3cZOxFyyO8IZAc4RVcycCCAKdM= -github.com/kisielk/errcheck v1.1.0/go.mod h1:EZBBE59ingxPouuu3KfxchcWSUPOHkagtvWXihfKN4Q= -github.com/kisielk/errcheck v1.2.0/go.mod h1:/BMXB+zMLi60iA8Vv6Ksmxu/1UDYcXs4uQLJ+jE2L00= github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8= github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck= github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= github.com/konsorten/go-windows-terminal-sequences v1.0.3/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= -github.com/kr/fs v0.1.0/go.mod h1:FFnZGqtBN9Gxj7eW1uZ42v5BccTP0vu6NEaFoC2HwRg= github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc= github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo= github.com/kr/pretty v0.2.0/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI= @@ -355,29 +249,15 @@ github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ= github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI= github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= -github.com/magiconair/properties v1.8.1/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ= github.com/mailru/easyjson v0.0.0-20190614124828-94de47d64c63/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc= github.com/mailru/easyjson v0.0.0-20190626092158-b2ccc519800e/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc= github.com/mailru/easyjson v0.7.6 h1:8yTIVnZgCoiM1TgqoeTl+LfU5Jg6/xL3QhGQnimLYnA= github.com/mailru/easyjson v0.7.6/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc= -github.com/mattn/go-colorable v0.0.9/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaOChaDxuIBZU= -github.com/mattn/go-isatty v0.0.3/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4= github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0= github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 h1:I0XW9+e1XWDxdcEniV4rQAIOPUGDq67JSCiRCgGCZLI= github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369/go.mod h1:BSXmuO+STAnVfrANrmjBb36TMTDstsz7MSK+HVaYKv4= -github.com/miekg/dns v1.0.14/go.mod h1:W1PPwlIAgtquWBMBEV9nkV9Cazfe8ScdGz/Lj7v3Nrg= -github.com/mitchellh/cli v1.0.0/go.mod h1:hNIlj7HEI86fIcpObd7a0FcrxTWetlwJDGcceTlRvqc= -github.com/mitchellh/go-homedir v1.0.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0= -github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0= -github.com/mitchellh/go-testing-interface v1.0.0/go.mod h1:kRemZodwjscx+RGhAo8eIhFbs2+BFgRtFPeD/KE+zxI= -github.com/mitchellh/gox v0.4.0/go.mod h1:Sd9lOJ0+aimLBi73mGofS1ycjY8lL3uZM3JPS42BGNg= -github.com/mitchellh/iochan v1.0.0/go.mod h1:JwYml1nuB7xOzsp52dPpHFffvOCDupsG0QubkSMEySY= -github.com/mitchellh/mapstructure v0.0.0-20160808181253-ca63d7c062ee/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y= -github.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y= -github.com/mitchellh/mapstructure v1.4.1/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo= github.com/moby/spdystream v0.2.0 h1:cjW1zVyyoiM0T7b6UoySUFqzXMoqRckQtXwGPiBhOM8= github.com/moby/spdystream v0.2.0/go.mod h1:f7i0iNDQJ059oMTcWxx8MA/zKFIuD/lY+0GqbN2Wy8c= -github.com/moby/term v0.0.0-20210619224110-3f7ff695adc6/go.mod h1:E2VnQOmVuvZB6UYnnDB0qG5Nq/1tD9acaOpo6xmt0Kw= github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg= github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= @@ -385,62 +265,30 @@ github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lN github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0= github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M= github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk= -github.com/munnerz/goautoneg v0.0.0-20120707110453-a547fc61f48d/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ= github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA= github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ= github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U= github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U= -github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f/go.mod h1:ZdcZmHo+o7JKHSa8/e818NopupXU1YMK5fe1lsApnBw= github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e h1:fD57ERR4JtEqsWbfPhv4DMiApHyliiK5xCTNVSPiaAs= github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e/go.mod h1:zD1mROLANZcx1PVRCS0qkT7pwLkGfwJo4zjcN/Tysno= -github.com/nxadm/tail v1.4.4/go.mod h1:kenIhsEOeOJmVchQTgglprH7qJGnHDVpk1VPCcaMI8A= github.com/nxadm/tail v1.4.8 h1:nPr65rt6Y5JFSKQO7qToXr7pePgD6Gwiw05lkbyAQTE= -github.com/nxadm/tail v1.4.8/go.mod h1:+ncqLTQzXmGhMZNUePPaPqPvBxHAIsmXswZKocGu+AU= -github.com/oklog/ulid v1.3.1/go.mod h1:CirwcVhetQ6Lv90oh/F+FBtV6XMibvdAFo93nm5qn4U= -github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE= -github.com/onsi/ginkgo v1.12.1/go.mod h1:zj2OWP4+oCPe1qIXoGWkgMRwljMUYCdkwsT2108oapk= -github.com/onsi/ginkgo v1.16.4/go.mod h1:dX+/inL/fNMqNlz0e9LfyB9TswhZpCVdJM/Z6Vvnwo0= github.com/onsi/ginkgo v1.16.5 h1:8xi0RTUf59SOSfEtZMvwTvXYMzG4gV23XVHOZiXNtnE= -github.com/onsi/ginkgo v1.16.5/go.mod h1:+E8gABHa3K6zRBolWtd+ROzc/U5bkGt0FwiG042wbpU= -github.com/onsi/ginkgo/v2 v2.1.3/go.mod h1:vw5CSIxN1JObi/U8gcbwft7ZxR2dgaR70JSE3/PpL4c= -github.com/onsi/ginkgo/v2 v2.1.4/go.mod h1:um6tUpWM/cxCK3/FK8BXqEiUMUwRgSM4JXG47RKZmLU= -github.com/onsi/ginkgo/v2 v2.1.6/go.mod h1:MEH45j8TBi6u9BMogfbp0stKC5cdGjumZj5Y7AG4VIk= -github.com/onsi/ginkgo/v2 v2.3.0/go.mod h1:Eew0uilEqZmIEZr8JrvYlvOM7Rr6xzTmMV8AyFNU9d0= -github.com/onsi/ginkgo/v2 v2.4.0/go.mod h1:iHkDK1fKGcBoEHT5W7YBq4RFWaQulw+caOMkAt4OrFo= github.com/onsi/ginkgo/v2 v2.5.0 h1:TRtrvv2vdQqzkwrQ1ke6vtXf7IK34RBUJafIy1wMwls= github.com/onsi/ginkgo/v2 v2.5.0/go.mod h1:Luc4sArBICYCS8THh8v3i3i5CuSZO+RaQRaJoeNwomw= -github.com/onsi/gomega v1.7.1/go.mod h1:XdKZgCCFLUoM/7CFJVPcG8C1xQ1AJ0vpAezJrB7JYyY= -github.com/onsi/gomega v1.10.1/go.mod h1:iN09h71vgCQne3DLsj+A5owkum+a2tYe+TOCB1ybHNo= -github.com/onsi/gomega v1.17.0/go.mod h1:HnhC7FXeEQY45zxNK3PPoIUhzk/80Xly9PcubAlGdZY= -github.com/onsi/gomega v1.19.0/go.mod h1:LY+I3pBVzYsTBU1AnDwOSxaYi9WoWiqgwooUqq9yPro= -github.com/onsi/gomega v1.20.1/go.mod h1:DtrZpjmvpn2mPm4YWQa0/ALMDj9v4YxLgojwPeREyVo= -github.com/onsi/gomega v1.21.1/go.mod h1:iYAIXgPSaDHak0LCMA+AWBpIKBr8WZicMxnE8luStNc= -github.com/onsi/gomega v1.22.1/go.mod h1:x6n7VNe4hw0vkyYUM4mjIXx3JbLiPaBPNgB7PRQ1tuM= -github.com/onsi/gomega v1.24.0/go.mod h1:Z/NWtiqwBrwUt4/2loMmHL63EDLnYHmVbuBpDr2vQAg= github.com/onsi/gomega v1.24.1 h1:KORJXNNTzJXzu4ScJWssJfJMnJ+2QJqhoQSRwNlze9E= github.com/onsi/gomega v1.24.1/go.mod h1:3AOiACssS3/MajrniINInwbfOOtfZvplPzuRSmvt1jM= -github.com/opentracing/opentracing-go v1.1.0/go.mod h1:UkNAQd3GIcIGf0SeVgPpRdFStlNbqXla1AfSYxPUl2o= github.com/oracle/oci-go-sdk/v65 v65.26.1 h1:Ms20RSRj+CuvQmw5ET1TkmzxLBI+bmLjZ6NYANA3gkk= github.com/oracle/oci-go-sdk/v65 v65.26.1/go.mod h1:oyMrMa1vOzzKTmPN+kqrTR9y9kPA2tU1igN3NUSNTIE= -github.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc= -github.com/pelletier/go-toml v1.2.0/go.mod h1:5z9KED0ma1S8pY6P1sdut58dfprrGBbd/94hg7ilaic= -github.com/peterbourgon/diskv v2.0.1+incompatible/go.mod h1:uqqh8zWWbv1HBMNONnaR/tNboyR3/BZd58JJSHlUSCU= github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4= github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= -github.com/pkg/sftp v1.10.1/go.mod h1:lYOWFsE0bwd1+KfKJaKeuokY15vzFx25BLbzYYoAxZI= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= -github.com/posener/complete v1.1.1/go.mod h1:em0nMJCgc9GFtwrmVmEMR/ZL6WyhyjMBndrE9hABlRI= -github.com/pquerna/cachecontrol v0.1.0/go.mod h1:NrUG3Z7Rdu85UNR3vm7SOsl1nFIeSiQnrHV5K9mBcUI= github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw= -github.com/prometheus/client_golang v0.9.3/go.mod h1:/TN21ttK/J9q6uSwhBd54HahCDft0ttaMvbicHlPoso= github.com/prometheus/client_golang v1.0.0/go.mod h1:db9x61etRT2tGnBNRi70OPL5FsnadC4Ky3P0J6CfImo= github.com/prometheus/client_golang v1.7.1/go.mod h1:PY5Wy2awLA44sXw4AOSfFBetzPP4j5+D6mVACh+pe2M= github.com/prometheus/client_golang v1.11.0/go.mod h1:Z6t4BnS23TR94PD6BsDNk8yVqroYurpAkEiz0P2BEV0= -github.com/prometheus/client_golang v1.11.1/go.mod h1:Z6t4BnS23TR94PD6BsDNk8yVqroYurpAkEiz0P2BEV0= -github.com/prometheus/client_golang v1.12.1/go.mod h1:3Z9XVyYiZYEO+YQWt3RD2R3jrbd179Rt297l4aS6nDY= github.com/prometheus/client_golang v1.12.2 h1:51L9cDoUHVrXx4zWYlcLQIZ+d+VXHgqnYKkIuq4g/34= github.com/prometheus/client_golang v1.12.2/go.mod h1:3Z9XVyYiZYEO+YQWt3RD2R3jrbd179Rt297l4aS6nDY= github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo= @@ -448,52 +296,28 @@ github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1: github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= github.com/prometheus/client_model v0.2.0 h1:uq5h0d+GuxiXLJLNABMgp2qUWDPiLvgCzz2dUR+/W/M= github.com/prometheus/client_model v0.2.0/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= -github.com/prometheus/common v0.0.0-20181113130724-41aa239b4cce/go.mod h1:daVV7qP5qjZbuso7PdcryaAu0sAZbrN9i7WWcTMWvro= -github.com/prometheus/common v0.4.0/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4= github.com/prometheus/common v0.4.1/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4= github.com/prometheus/common v0.10.0/go.mod h1:Tlit/dnDKsSWFlCLTWaA1cyBgKHSMdTB80sz/V91rCo= github.com/prometheus/common v0.26.0/go.mod h1:M7rCNAaPfAosfx8veZJCuw84e35h3Cfd9VFqTh1DIvc= github.com/prometheus/common v0.32.1 h1:hWIdL3N2HoUx3B8j3YN9mWor0qhY/NlEKZEaXxuIRh4= github.com/prometheus/common v0.32.1/go.mod h1:vu+V0TpY+O6vW9J44gczi3Ap/oXXR10b+M/gUGO4Hls= github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk= -github.com/prometheus/procfs v0.0.0-20190507164030-5867b95ac084/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA= github.com/prometheus/procfs v0.0.2/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA= github.com/prometheus/procfs v0.1.3/go.mod h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4OA4YeYWdaU= github.com/prometheus/procfs v0.6.0/go.mod h1:cz+aTbrPOrUb4q7XlbU9ygM+/jj0fzG6c1xBZuNvfVA= github.com/prometheus/procfs v0.7.3 h1:4jVXhlkAyzOScmCkXBTOLRLTz8EeU+eyjrwB/EPq0VU= github.com/prometheus/procfs v0.7.3/go.mod h1:cz+aTbrPOrUb4q7XlbU9ygM+/jj0fzG6c1xBZuNvfVA= -github.com/prometheus/tsdb v0.7.1/go.mod h1:qhTCs0VvXwvX/y3TZrWD7rabWM+ijKTux40TwIPHuXU= -github.com/rogpeppe/fastuuid v0.0.0-20150106093220-6724a57986af/go.mod h1:XWv6SoW27p1b0cqNHllgS5HIMJraePCO15w5zCzIWYg= github.com/rogpeppe/fastuuid v1.2.0/go.mod h1:jVj6XXZzXRy/MSR5jhDC/2q6DgLz+nrA6LYCDYWNEvQ= github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4= -github.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM= -github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM= -github.com/ryanuber/columnize v0.0.0-20160712163229-9b3edd62028f/go.mod h1:sm1tb6uqfes/u+d4ooFouqFdy9/2g9QGwK3SQygK0Ts= -github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529/go.mod h1:DxrIzT+xaE7yg65j358z/aeFdxmN0P9QXhEzd20vsDc= -github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc= github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo= github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE= github.com/sirupsen/logrus v1.6.0/go.mod h1:7uNnSEd1DgxDLC74fIahvMZmmYsHGZGEOFrfsX/uA88= -github.com/sirupsen/logrus v1.7.0/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0= -github.com/sirupsen/logrus v1.8.1/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0= -github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1:OnSkiWE9lh6wB0YB77sQom3nweQdgAjqCqsofrRNTgc= -github.com/smartystreets/goconvey v1.6.4/go.mod h1:syvi0/a8iFYH4r/RixwvyeAJjdLS9QV7WQ/tjFTllLA= -github.com/soheilhy/cmux v0.1.4/go.mod h1:IM3LyeVVIOuxMH7sFAkER9+bJ4dT7Ms6E4xg4kGIyLM= -github.com/soheilhy/cmux v0.1.5/go.mod h1:T7TcVDs9LWfQgPlPsdngu6I6QIoyIFZDDC6sNE1GqG0= github.com/sony/gobreaker v0.5.0 h1:dRCvqm0P490vZPmy7ppEk2qCnCieBooFJ+YoXGYB+yg= github.com/sony/gobreaker v0.5.0/go.mod h1:ZKptC7FHNvhBz7dN2LGjPVBz2sZJmc0/PkyDJOjmxWY= github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA= -github.com/spf13/afero v1.1.2/go.mod h1:j4pytiNVoe2o6bmDsKpLACNPDBIoEAkihy7loJ1B0CQ= github.com/spf13/afero v1.2.2/go.mod h1:9ZxEEn6pIJ8Rxe320qSDBk6AsU0r9pR7Q4OcevTdifk= -github.com/spf13/afero v1.6.0/go.mod h1:Ai8FlHk4v/PARR026UzYexafAt9roJ7LcLMAmO6Z93I= -github.com/spf13/cast v1.3.0/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE= -github.com/spf13/cobra v1.1.3/go.mod h1:pGADOWyqRD/YMrPZigI/zbliZ2wVD/23d+is3pSWzOo= -github.com/spf13/cobra v1.4.0/go.mod h1:Wo4iy3BUC+X2Fybo0PDqwJIv3dNRiZLHQymsfxlB84g= -github.com/spf13/jwalterweatherman v1.0.0/go.mod h1:cQK4TGJAtQXfYWX+Ddv3mKDzgVb68N+wFjFa4jdeBTo= -github.com/spf13/pflag v1.0.3/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4= github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA= github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= -github.com/spf13/viper v1.7.0/go.mod h1:8WkrPz2fc9jxqZNCJI/76HCieCp4Q8HaLFoCha5qpdg= github.com/stoewer/go-strcase v1.2.0/go.mod h1:IBiWB2sKIp3wVVQ3Y035++gc+knqhUQag1KpM8ahLw8= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= @@ -507,27 +331,11 @@ github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/ github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= github.com/stretchr/testify v1.8.0 h1:pSgiaMZlXftHpm5L7V1+rVB+AZJydKsMxsQBIJw4PKk= -github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU= -github.com/subosito/gotenv v1.2.0/go.mod h1:N0PQaV/YGNqwC0u51sEeR/aUtSLEXKX9iv69rRypqCw= -github.com/tmc/grpc-websocket-proxy v0.0.0-20190109142713-0ad062ec5ee5/go.mod h1:ncp9v5uamzpCO7NfCPTXjqaC+bZgJeR0sMTm6dMHP7U= -github.com/tmc/grpc-websocket-proxy v0.0.0-20201229170055-e5319fda7802/go.mod h1:ncp9v5uamzpCO7NfCPTXjqaC+bZgJeR0sMTm6dMHP7U= -github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2/go.mod h1:UETIi67q53MR2AWcXfiuqkDkRtnGDLqkBTpCHuJHxtU= github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.3.5/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k= -github.com/yuin/goldmark v1.4.1/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k= -github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY= -go.etcd.io/bbolt v1.3.2/go.mod h1:IbVyRI1SCnLcuJnV2u8VeU0CEYM7e686BmAb1XKL+uU= -go.etcd.io/bbolt v1.3.6/go.mod h1:qXsaaIqmgQH0T+OPdb99Bf+PKfBBQVAdyD6TY9G8XM4= -go.etcd.io/etcd/api/v3 v3.5.4/go.mod h1:5GB2vv4A4AOn3yk7MftYGHkUfGtDHnEraIjym4dYz5A= -go.etcd.io/etcd/client/pkg/v3 v3.5.4/go.mod h1:IJHfcCEKxYu1Os13ZdwCwIUTUVGYTSAM3YSwc9/Ac1g= -go.etcd.io/etcd/client/v2 v2.305.4/go.mod h1:Ud+VUwIi9/uQHOMA+4ekToJ12lTxlv0zB/+DHwTGEbU= -go.etcd.io/etcd/client/v3 v3.5.4/go.mod h1:ZaRkVgBZC+L+dLCjTcF1hRXpgZXQPOvnA/Ak/gq3kiY= -go.etcd.io/etcd/pkg/v3 v3.5.4/go.mod h1:OI+TtO+Aa3nhQSppMbwE4ld3uF1/fqqwbpfndbbrEe0= -go.etcd.io/etcd/raft/v3 v3.5.4/go.mod h1:SCuunjYvZFC0fBX0vxMSPjuZmpcSk+XaAcMrD6Do03w= -go.etcd.io/etcd/server/v3 v3.5.4/go.mod h1:S5/YTU15KxymM5l3T6b09sNOHPXqGYIZStpuuGbb65c= go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU= go.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8= go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= @@ -535,48 +343,24 @@ go.opencensus.io v0.22.3/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= go.opencensus.io v0.22.4/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= go.opencensus.io v0.22.5/go.mod h1:5pWMHQbX5EPX2/62yrJeAkowc+lfs/XD7Uxpq3pI6kk= go.opencensus.io v0.23.0/go.mod h1:XItmlyltB5F7CS4xOC1DcqMoFqwtC6OG2xF7mCv7P7E= -go.opentelemetry.io/contrib v0.20.0/go.mod h1:G/EtFaa6qaN7+LxqfIAT3GiZa7Wv5DTBUzl5H4LY0Kc= -go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.20.0/go.mod h1:oVGt1LRbBOBq1A5BQLlUg9UaU/54aiHw8cgjV3aWZ/E= -go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.20.0/go.mod h1:2AboqHi0CiIZU0qwhtUfCYD1GeUzvvIXWNkhDt7ZMG4= -go.opentelemetry.io/otel v0.20.0/go.mod h1:Y3ugLH2oa81t5QO+Lty+zXf8zC9L26ax4Nzoxm/dooo= -go.opentelemetry.io/otel/exporters/otlp v0.20.0/go.mod h1:YIieizyaN77rtLJra0buKiNBOm9XQfkPEKBeuhoMwAM= -go.opentelemetry.io/otel/metric v0.20.0/go.mod h1:598I5tYlH1vzBjn+BTuhzTCSb/9debfNp6R3s7Pr1eU= -go.opentelemetry.io/otel/oteltest v0.20.0/go.mod h1:L7bgKf9ZB7qCwT9Up7i9/pn0PWIa9FqQ2IQ8LoxiGnw= -go.opentelemetry.io/otel/sdk v0.20.0/go.mod h1:g/IcepuwNsoiX5Byy2nNV0ySUF1em498m7hBWC279Yc= -go.opentelemetry.io/otel/sdk/export/metric v0.20.0/go.mod h1:h7RBNMsDJ5pmI1zExLi+bJK+Dr8NQCh0qGhm1KDnNlE= -go.opentelemetry.io/otel/sdk/metric v0.20.0/go.mod h1:knxiS8Xd4E/N+ZqKmUPf3gTTZ4/0TjTXukfxjzSTpHE= -go.opentelemetry.io/otel/trace v0.20.0/go.mod h1:6GjCW8zgDjwGHGa6GkyeB8+/5vjT16gUEi0Nf1iBdgw= go.opentelemetry.io/proto/otlp v0.7.0/go.mod h1:PqfVotwruBrMGOCsRd/89rSnXhoiJIqeYNgFYFoEGnI= -go.uber.org/atomic v1.4.0/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE= go.uber.org/atomic v1.7.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc= go.uber.org/atomic v1.9.0 h1:ECmE8Bn/WFTYwEW/bpKD3M8VtR/zQVbavAoalC1PYyE= go.uber.org/atomic v1.9.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc= go.uber.org/goleak v1.1.10/go.mod h1:8a7PlsEVH3e/a/GLqe5IIrQx6GzcnRmZEufDUTk4A7A= -go.uber.org/goleak v1.1.11/go.mod h1:cwTWslyiVhfpKIDGSZEM2HlOvcqm+tG4zioyIeLoqMQ= go.uber.org/goleak v1.1.12 h1:gZAh5/EyT/HQwlpkCy6wTpqfH9H8Lz8zbm3dZh+OyzA= -go.uber.org/goleak v1.1.12/go.mod h1:cwTWslyiVhfpKIDGSZEM2HlOvcqm+tG4zioyIeLoqMQ= -go.uber.org/multierr v1.1.0/go.mod h1:wR5kodmAFQ0UK8QlbwjlSNy0Z68gJhDJUG5sjR94q/0= go.uber.org/multierr v1.6.0/go.mod h1:cdWPpRnG4AhwMwsgIHip0KRBQjJy5kYEpYjJxpXp9iU= go.uber.org/multierr v1.7.0 h1:zaiO/rmgFjbmCXdSYJWQcdvOCsthmdaHfr3Gm2Kx4Ec= go.uber.org/multierr v1.7.0/go.mod h1:7EAYxJLBy9rStEaz58O2t4Uvip6FSURkq8/ppBp95ak= -go.uber.org/zap v1.10.0/go.mod h1:vwi/ZaCAaUcBkycHslxD9B2zi4UTXhF60s6SWpuDF0Q= -go.uber.org/zap v1.17.0/go.mod h1:MXVU+bhUf/A7Xi2HNOnopQOrmycQ5Ih87HtOu4q5SSo= go.uber.org/zap v1.19.0/go.mod h1:xg/QME4nWcxGxrpdeYfq7UvYrLh66cuVKdrbD1XF/NI= -go.uber.org/zap v1.21.0 h1:WefMeulhovoZ2sYXz7st6K0sLj7bBhpiFaud4r4zST8= -go.uber.org/zap v1.21.0/go.mod h1:wjWOCqI0f2ZZrJF/UufIOkiC8ii6tm1iqIsLo76RfJw= +go.uber.org/zap v1.23.0 h1:OjGQ5KQDEUawVHxNwQgPpiypGHOxo2mNZsOqTak4fFY= +go.uber.org/zap v1.23.0/go.mod h1:D+nX8jyLsMHMYrln8A0rJjFt/T/9/bGgIhAqxv5URuY= golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= -golang.org/x/crypto v0.0.0-20181029021203-45a5f77698d3/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= -golang.org/x/crypto v0.0.0-20190820162420-60c769a6c586/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= -golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc= -golang.org/x/crypto v0.0.0-20211215153901-e495a2d5b3d3/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4= -golang.org/x/crypto v0.0.0-20220131195533-30dcbda58838/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4= -golang.org/x/crypto v0.0.0-20220315160706-3147a52a75dd/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4= -golang.org/x/crypto v0.1.0/go.mod h1:RecgLatLF4+eUMCP1PoPZQb+cVrJcOPbHkTkbkB9sbw= golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8= @@ -612,16 +396,9 @@ golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.4.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.4.1/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= -golang.org/x/mod v0.6.0-dev.0.20220106191415-9b9b3d81d5e3/go.mod h1:3p9vT2HGsQu2K1YbXdKPJLVgG5VJdoTa1poYQBtP1AY= -golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4= -golang.org/x/mod v0.6.0/go.mod h1:4mET923SAdbXp2ki8ey+zGs1SLqsuM2Y0uvdZR/fUNI= golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= -golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= -golang.org/x/net v0.0.0-20181023162649-9b4f9f5ad519/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= -golang.org/x/net v0.0.0-20181201002055-351d144fa1fc/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= -golang.org/x/net v0.0.0-20181220203305-927f97764cc3/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= @@ -644,7 +421,6 @@ golang.org/x/net v0.0.0-20200324143707-d3edc9973b7e/go.mod h1:qpuaurCH72eLCgpAm/ golang.org/x/net v0.0.0-20200501053045-e0ff5e5a1de5/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= golang.org/x/net v0.0.0-20200506145744-7e3656a0809f/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= golang.org/x/net v0.0.0-20200513185701-a91f0712d120/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= -golang.org/x/net v0.0.0-20200520004742-59133d7f0dd7/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= golang.org/x/net v0.0.0-20200520182314-0ba52f642ac2/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= golang.org/x/net v0.0.0-20200625001655-4c5254603344/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= golang.org/x/net v0.0.0-20200707034311-ab3426394381/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= @@ -652,21 +428,13 @@ golang.org/x/net v0.0.0-20200822124328-c89045814202/go.mod h1:/O7V0waA8r7cgGh81R golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= golang.org/x/net v0.0.0-20201031054903-ff519b6c9102/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= golang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= -golang.org/x/net v0.0.0-20201202161906-c7110b5ffcbb/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= golang.org/x/net v0.0.0-20201209123823-ac852fbbde11/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= golang.org/x/net v0.0.0-20210119194325-5f4716e94777/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= golang.org/x/net v0.0.0-20210316092652-d523dce5a7f4/go.mod h1:RBQZq4jEuRlivfhVLdyRGr576XBO4/greRjx4P4O3yc= golang.org/x/net v0.0.0-20210405180319-a5a99cb37ef4/go.mod h1:p54w0d4576C0XHj96bSt6lcn1PtDYWL6XObtHCRCNQM= -golang.org/x/net v0.0.0-20210428140749-89ef3d95e781/go.mod h1:OJAsFXCWl8Ukc7SiCT/9KSuxbyM7479/AVlXFRxuMCk= golang.org/x/net v0.0.0-20210503060351-7fd8e65b6420/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= golang.org/x/net v0.0.0-20210525063256-abc453219eb5/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= -golang.org/x/net v0.0.0-20211015210444-4f30a5c0130f/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= -golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= -golang.org/x/net v0.0.0-20220225172249-27dd8689420f/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk= -golang.org/x/net v0.0.0-20220425223048-2871e0cb64e4/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk= -golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c= -golang.org/x/net v0.1.0/go.mod h1:Cx3nUiGt4eDBEyega/BKRp+/AlGL8hYe7U9odMt2Cco= golang.org/x/net v0.2.0 h1:sZfSu1wtKLGlWI4ZZayP0ck9Y73K1ynO6gqzTdBVdPU= golang.org/x/net v0.2.0/go.mod h1:KqCZLdyyvdV855qA2rE3GC2aiw5xGR5TEjj8smXukLY= golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U= @@ -697,13 +465,8 @@ golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208/go.mod h1:RxMgew5VJxzue5/jJ golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= -golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= -golang.org/x/sys v0.0.0-20181026203630-95b1ffbd15a5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= -golang.org/x/sys v0.0.0-20181107165924-66b7b1311ac8/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20190312061237-fead79001313/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= @@ -714,11 +477,7 @@ golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7w golang.org/x/sys v0.0.0-20190606165138-5da285871e9c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190624142023-c5567b49c5d0/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20190904154756-749cb33beabd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20191001151750-bb3f8db39f24/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20191005200804-aed5e4c7ecf9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20191120155948-bd437916bb0e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20191228213918-04cbcbbfeed8/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200106162015-b016eb3dc98e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= @@ -738,12 +497,10 @@ golang.org/x/sys v0.0.0-20200615200032-f1bc736245b1/go.mod h1:h1NjWce9XRLGQEsW7w golang.org/x/sys v0.0.0-20200625212154-ddb9806d33ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200803210538-64077c9b5642/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200905004654-be1d3432aa8f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200923182605-d9f96fdee20d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20201201145000-ef89a241ccb3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210104204734-6f8348627aad/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20210112080510-489259a85091/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210119212857-b64e53b001e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210220050731-9a76102bfb43/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= @@ -751,35 +508,24 @@ golang.org/x/sys v0.0.0-20210305230114-8fe3ee5dd75b/go.mod h1:h1NjWce9XRLGQEsW7w golang.org/x/sys v0.0.0-20210315160823-c6e025ad8005/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210320140829-1e4c9ba3b0c4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20210403161142-5e06dd20ab57/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210510120138-977fb7262007/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20210514084401-e8d321eab015/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20210603081109-ebe580a85c40/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20210603125802-9665404d3644/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20210616094352-59db8d763f22/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20210806184541-e5e7981a1069/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20210823070655-63515b42dcdf/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20210908233432-aa78b53d3365/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20211019181941-9d821ace8654/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20211124211545-fe61309f8881/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20211210111614-af8b64212486/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20220114195835-da31bd327af9/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20220209214540-3681064d5158/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20220319134239-a9b59b0215f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20220412211240-33da011f77ad/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20220422013727-9388b58f7150/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.2.0 h1:ljd4t30dBnAvMZaQCevtY0xLLD0A+bRZXbgLMLU1F/A= golang.org/x/sys v0.2.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= -golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8= -golang.org/x/term v0.1.0/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8= golang.org/x/term v0.2.0 h1:z85xZCsEl7bi/KwbNADeBYoOP0++7W1ipu+aGnpwzRM= golang.org/x/term v0.2.0/go.mod h1:TVmDHMZPmdnySmBfhjOoOdhjzdE1h4u1VwSiw2l1Nuc= golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= @@ -790,37 +536,29 @@ golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.3.5/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= -golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ= golang.org/x/text v0.4.0 h1:BrVqGRd7+k1DiOgtnFvAkoQEWQvBc25ouMJM6429SFg= golang.org/x/text v0.4.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8= golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= -golang.org/x/time v0.0.0-20210220033141-f8bda1e9f3ba/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= -golang.org/x/time v0.0.0-20220210224613-90d013bbcef8/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/time v0.0.0-20220609170525-579cf78fd858 h1:Dpdu/EMxGMFgq0CeYMh4fazTD2vtlZRYE7wyynxJb9U= golang.org/x/time v0.0.0-20220609170525-579cf78fd858/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= -golang.org/x/tools v0.0.0-20180221164845-07fd8470d635/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= -golang.org/x/tools v0.0.0-20181030221726-6c7e314b6563/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY= golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= golang.org/x/tools v0.0.0-20190312151545-0bb0c0a6e846/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= golang.org/x/tools v0.0.0-20190312170243-e65039ee4138/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= -golang.org/x/tools v0.0.0-20190328211700-ab21143f2384/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= golang.org/x/tools v0.0.0-20190425150028-36563e24a262/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= golang.org/x/tools v0.0.0-20190506145303-2d16b83fe98c/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= golang.org/x/tools v0.0.0-20190606124116-d0a3d012864b/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= golang.org/x/tools v0.0.0-20190621195816-6e04913cbbac/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= -golang.org/x/tools v0.0.0-20190624222133-a101b041ded4/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= golang.org/x/tools v0.0.0-20190628153133-6cdbf07be9d0/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= golang.org/x/tools v0.0.0-20190816200558-6889da9d5479/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= golang.org/x/tools v0.0.0-20190911174233-4f2ddba30aff/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= golang.org/x/tools v0.0.0-20191012152004-8de300cfc20a/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= golang.org/x/tools v0.0.0-20191108193012-7d206e10da11/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= -golang.org/x/tools v0.0.0-20191112195655-aa38f8e97acc/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= golang.org/x/tools v0.0.0-20191113191852-77e3bb0ad9e7/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= golang.org/x/tools v0.0.0-20191115202509-3a792d9c32b2/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= @@ -840,7 +578,6 @@ golang.org/x/tools v0.0.0-20200304193943-95d2e580d8eb/go.mod h1:o4KQGtdN14AW+yjs golang.org/x/tools v0.0.0-20200312045724-11d5b4c81c7d/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw= golang.org/x/tools v0.0.0-20200331025713-a30bf2db82d4/go.mod h1:Sl4aGygMT6LrqrWclx+PTx3U+LnKx/seiNR+3G19Ar8= golang.org/x/tools v0.0.0-20200501065659-ab2804fb9c9d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= -golang.org/x/tools v0.0.0-20200505023115-26f46d2f7ef8/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= golang.org/x/tools v0.0.0-20200512131952-2bc93b1c0c88/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= golang.org/x/tools v0.0.0-20200515010526-7d3b6ebf133d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= golang.org/x/tools v0.0.0-20200618134242-20370b0cb4b2/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= @@ -852,7 +589,6 @@ golang.org/x/tools v0.0.0-20200904185747-39188db58858/go.mod h1:Cj7w3i3Rnn0Xh82u golang.org/x/tools v0.0.0-20201110124207-079ba7bd75cd/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= golang.org/x/tools v0.0.0-20201201161351-ac6f37ff4c2a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= golang.org/x/tools v0.0.0-20201208233053-a543418bbed2/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= -golang.org/x/tools v0.0.0-20201224043029-2b0845dc783e/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= golang.org/x/tools v0.0.0-20210105154028-b0ab187a4818/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= golang.org/x/tools v0.1.0/go.mod h1:xkSsbof2nBLbhDlRMhhhyNLN/zl3eTqcnHD5viDpcZ0= @@ -861,9 +597,6 @@ golang.org/x/tools v0.1.2/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= golang.org/x/tools v0.1.3/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= golang.org/x/tools v0.1.4/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= golang.org/x/tools v0.1.5/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= -golang.org/x/tools v0.1.10/go.mod h1:Uh6Zz+xoGYZom868N8YTex3t7RhtHDBrE8Gzo9bV56E= -golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc= -golang.org/x/tools v0.2.0/go.mod h1:y4OqIKeOV/fWJetJ8bXPU1sEVniLMIyDAZWeHdV+NTA= golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= @@ -931,7 +664,6 @@ google.golang.org/genproto v0.0.0-20200228133532-8c2c7df3a383/go.mod h1:55QSHmfG google.golang.org/genproto v0.0.0-20200305110556-506484158171/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= google.golang.org/genproto v0.0.0-20200312145019-da6875a35672/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= google.golang.org/genproto v0.0.0-20200331122359-1ee6d9798940/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= -google.golang.org/genproto v0.0.0-20200423170343-7949de9c1215/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= google.golang.org/genproto v0.0.0-20200430143042-b979b6f78d84/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= google.golang.org/genproto v0.0.0-20200511104702-f5ebc3bea380/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= google.golang.org/genproto v0.0.0-20200513103714-09dca8ec2884/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= @@ -974,7 +706,6 @@ google.golang.org/genproto v0.0.0-20211208223120-3a66f561d7aa/go.mod h1:5CzLGKJ6 google.golang.org/genproto v0.0.0-20211221195035-429b39de9b1c/go.mod h1:5CzLGKJ67TSI2B9POpiiyGha0AjJvZIUgRMt1dSmuhc= google.golang.org/genproto v0.0.0-20220114231437-d2e6a121cae0/go.mod h1:5CzLGKJ67TSI2B9POpiiyGha0AjJvZIUgRMt1dSmuhc= google.golang.org/genproto v0.0.0-20220201184016-50beb8ab5c44/go.mod h1:5CzLGKJ67TSI2B9POpiiyGha0AjJvZIUgRMt1dSmuhc= -google.golang.org/genproto v0.0.0-20220502173005-c8bf987b8c21/go.mod h1:RAyBrSAP7Fh3Nc84ghnVLDPuV51xc9agzmm4Ph6i0Q4= google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c= google.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38= google.golang.org/grpc v1.21.1/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM= @@ -1001,8 +732,6 @@ google.golang.org/grpc v1.39.0/go.mod h1:PImNr+rS9TWYb2O4/emRugxiyHZ5JyHW5F+RPnD google.golang.org/grpc v1.39.1/go.mod h1:PImNr+rS9TWYb2O4/emRugxiyHZ5JyHW5F+RPnDzfrE= google.golang.org/grpc v1.40.0/go.mod h1:ogyxbiOoUXAkP+4+xa6PZSE9DZgIHtSpzjDTB9KAK34= google.golang.org/grpc v1.40.1/go.mod h1:ogyxbiOoUXAkP+4+xa6PZSE9DZgIHtSpzjDTB9KAK34= -google.golang.org/grpc v1.46.0/go.mod h1:vN9eftEi1UMyUsIF80+uQXhHjbXYbm0uXoFCACuMGWk= -google.golang.org/grpc v1.47.0/go.mod h1:vN9eftEi1UMyUsIF80+uQXhHjbXYbm0uXoFCACuMGWk= google.golang.org/grpc/cmd/protoc-gen-go-grpc v1.1.0/go.mod h1:6Kw0yEErY5E/yWrBtf03jp27GLLJujG4z/JK95pnjjw= google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8= google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0= @@ -1026,16 +755,9 @@ gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8 gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f h1:BLraFXnmrev5lT+xlilqcH8XK9/i0At2xKjWk4p6zsU= gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI= -gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys= gopkg.in/inf.v0 v0.9.1 h1:73M5CoZyi3ZLMOyDlQh031Cx6N9NDJ2Vvfl76EDAgDc= gopkg.in/inf.v0 v0.9.1/go.mod h1:cWUDdTG/fYaXco+Dcufb5Vnc6Gp2YChqWtbxRZE0mXw= -gopkg.in/ini.v1 v1.51.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k= -gopkg.in/natefinch/lumberjack.v2 v2.0.0/go.mod h1:l0ndWWf7gzL7RNwBG7wST/UCcT4T24xpD6X8LsfU/+k= -gopkg.in/resty.v1 v1.12.0/go.mod h1:mDo4pnntr5jdWRML875a/NmxYqAlA73dVijT2AXvQQo= -gopkg.in/square/go-jose.v2 v2.2.2/go.mod h1:M9dMgbHiYLoDGQrXy7OpJDJWiKiU//h+vD76mk0e1AI= gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 h1:uRGJdciOHaEIrze2W8Q3AKkepLTh2hOroT7a+7czfdQ= -gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWDmTeBkI65Dw0HsyUHuEVlX15mw= -gopkg.in/yaml.v2 v2.0.0-20170812160011-eb3733d160e7/go.mod h1:JAlM8MvJe8wmxCU4Bli9HhUf9+ttbYbLASfIpnQbh74= gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.2.3/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= @@ -1050,8 +772,6 @@ gopkg.in/yaml.v3 v3.0.0-20200615113413-eeeca48fe776/go.mod h1:K4uyk7z7BCEPqu6E+C gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= -gotest.tools/v3 v3.0.2/go.mod h1:3SzNCllyD9/Y+b5r9JIKQ474KzkZyqLqEfYqMsX94Bk= -gotest.tools/v3 v3.0.3/go.mod h1:Z7Lb0S5l+klDB31fvDQX8ss/FlKDxtlFlw3Oa8Ymbl8= honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= @@ -1059,42 +779,31 @@ honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWh honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg= honnef.co/go/tools v0.0.1-2020.1.3/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k= honnef.co/go/tools v0.0.1-2020.1.4/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k= -k8s.io/api v0.25.0/go.mod h1:ttceV1GyV1i1rnmvzT3BST08N6nGt+dudGrquzVQWPk= k8s.io/api v0.25.4 h1:3YO8J4RtmG7elEgaWMb4HgmpS2CfY1QlaOz9nwB+ZSs= k8s.io/api v0.25.4/go.mod h1:IG2+RzyPQLllQxnhzD8KQNEu4c4YvyDTpSMztf4A0OQ= k8s.io/apiextensions-apiserver v0.25.0 h1:CJ9zlyXAbq0FIW8CD7HHyozCMBpDSiH7EdrSTCZcZFY= k8s.io/apiextensions-apiserver v0.25.0/go.mod h1:3pAjZiN4zw7R8aZC5gR0y3/vCkGlAjCazcg1me8iB/E= -k8s.io/apimachinery v0.25.0/go.mod h1:qMx9eAk0sZQGsXGu86fab8tZdffHbwUfsvzqKn4mfB0= k8s.io/apimachinery v0.25.4 h1:CtXsuaitMESSu339tfhVXhQrPET+EiWnIY1rcurKnAc= k8s.io/apimachinery v0.25.4/go.mod h1:jaF9C/iPNM1FuLl7Zuy5b9v+n35HGSh6AQ4HYRkCqwo= -k8s.io/apiserver v0.25.0/go.mod h1:BKwsE+PTC+aZK+6OJQDPr0v6uS91/HWxX7evElAH6xo= -k8s.io/client-go v0.25.0/go.mod h1:lxykvypVfKilxhTklov0wz1FoaUZ8X4EwbhS6rpRfN8= k8s.io/client-go v0.25.4 h1:3RNRDffAkNU56M/a7gUfXaEzdhZlYhoW8dgViGy5fn8= k8s.io/client-go v0.25.4/go.mod h1:8trHCAC83XKY0wsBIpbirZU4NTUpbuhc2JnI7OruGZw= -k8s.io/code-generator v0.25.0/go.mod h1:B6jZgI3DvDFAualltPitbYMQ74NjaCFxum3YeKZZ+3w= k8s.io/component-base v0.25.0 h1:haVKlLkPCFZhkcqB6WCvpVxftrg6+FK5x1ZuaIDaQ5Y= k8s.io/component-base v0.25.0/go.mod h1:F2Sumv9CnbBlqrpdf7rKZTmmd2meJq0HizeyY/yAFxk= -k8s.io/gengo v0.0.0-20210813121822-485abfe95c7c/go.mod h1:FiNAH4ZV3gBg2Kwh89tzAEV2be7d5xI0vBa/VySYy3E= -k8s.io/gengo v0.0.0-20211129171323-c02415ce4185/go.mod h1:FiNAH4ZV3gBg2Kwh89tzAEV2be7d5xI0vBa/VySYy3E= k8s.io/klog/v2 v2.0.0/go.mod h1:PBfzABfn139FHAV07az/IF9Wp1bkk3vpT2XSJ76fSDE= -k8s.io/klog/v2 v2.2.0/go.mod h1:Od+F08eJP+W3HUb4pSrPpgp9DGU4GzlpG/TmITuYh/Y= k8s.io/klog/v2 v2.70.1 h1:7aaoSdahviPmR+XkS7FyxlkkXs6tHISSG03RxleQAVQ= k8s.io/klog/v2 v2.70.1/go.mod h1:y1WjHnz7Dj687irZUWR/WLkLc5N1YHtjLdmgWjndZn0= k8s.io/kube-openapi v0.0.0-20220803162953-67bda5d908f1 h1:MQ8BAZPZlWk3S9K4a9NCkIFQtZShWqoha7snGixVgEA= k8s.io/kube-openapi v0.0.0-20220803162953-67bda5d908f1/go.mod h1:C/N6wCaBHeBHkHUesQOQy2/MZqGgMAFPqGsGQLdbZBU= -k8s.io/utils v0.0.0-20210802155522-efc7438f0176/go.mod h1:jPW/WVKK9YHAvNhRxK0md/EJ228hCsBRufyofKtW8HA= k8s.io/utils v0.0.0-20220728103510-ee6ede2d64ed h1:jAne/RjBTyawwAy0utX5eqigAwz/lQhTmy+Hr/Cpue4= k8s.io/utils v0.0.0-20220728103510-ee6ede2d64ed/go.mod h1:jPW/WVKK9YHAvNhRxK0md/EJ228hCsBRufyofKtW8HA= rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8= rsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0= rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA= -sigs.k8s.io/apiserver-network-proxy/konnectivity-client v0.0.32/go.mod h1:fEO7lRTdivWO2qYVCVG7dEADOMo/MLDCVr8So2g88Uw= sigs.k8s.io/controller-runtime v0.13.1 h1:tUsRCSJVM1QQOOeViGeX3GMT3dQF1eePPw6sEE3xSlg= sigs.k8s.io/controller-runtime v0.13.1/go.mod h1:Zbz+el8Yg31jubvAEyglRZGdLAjplZl+PgtYNI6WNTI= sigs.k8s.io/json v0.0.0-20220713155537-f223a00ba0e2 h1:iXTIw73aPyC+oRdyqqvVJuloN1p0AC/kzH07hu3NE+k= sigs.k8s.io/json v0.0.0-20220713155537-f223a00ba0e2/go.mod h1:B8JuhiUyNFVKdsE8h686QcCxMaH6HrOAZj4vswFpcB0= sigs.k8s.io/structured-merge-diff/v4 v4.2.3 h1:PRbqxJClWWYMNV1dhaG4NsibJbArud9kFxnAMREiWFE= sigs.k8s.io/structured-merge-diff/v4 v4.2.3/go.mod h1:qjx8mGObPmV2aSZepjQjbmb2ihdVs8cGKBraizNC69E= -sigs.k8s.io/yaml v1.2.0/go.mod h1:yfXDCHCao9+ENCvLSE62v9VSji2MKu5jeNfTrofGhJc= sigs.k8s.io/yaml v1.3.0 h1:a2VclLzOGrwOHDiV8EfBGhvjHvP46CtW5j6POvhYGGo= sigs.k8s.io/yaml v1.3.0/go.mod h1:GeOyir5tyXNByN85N/dRIT9es5UQNerPYEKK56eTBm8= From cf0cdafc69ae125c9b8d88aad68e34d1f114b6c0 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Fri, 16 Dec 2022 14:13:35 +0000 Subject: [PATCH 517/628] Update .gitlab-ci.yml file --- .gitlab-ci.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index cbef6241..f5150ed7 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -4,6 +4,7 @@ build-operator: IMAGE: "$DOCKER_REPO:$CI_COMMIT_BRANCH" OP_YAML: oracle-database-operator.yaml script: + - echo go version - echo $CI_COMMIT_TAG - make docker-build IMG="$IMAGE" - docker push "$IMAGE" From a4131c18541c64407486c82011315f8d3d7b543a Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Fri, 16 Dec 2022 14:14:25 +0000 Subject: [PATCH 518/628] Update .gitlab-ci.yml file --- .gitlab-ci.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index f5150ed7..daf3c9e9 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -4,7 +4,7 @@ build-operator: IMAGE: "$DOCKER_REPO:$CI_COMMIT_BRANCH" OP_YAML: oracle-database-operator.yaml script: - - echo go version + - go version - echo $CI_COMMIT_TAG - make docker-build IMG="$IMAGE" - docker push "$IMAGE" From 67c98d74aacf3d65736257ef9642aa844287cca3 Mon Sep 17 00:00:00 2001 From: Mohammed Yunus Date: Wed, 21 Dec 2022 11:39:19 +0000 Subject: [PATCH 519/628] Update .gitlab-ci.yml file --- .gitlab-ci.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index cbef6241..8f1824a2 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -7,7 +7,7 @@ build-operator: - echo $CI_COMMIT_TAG - make docker-build IMG="$IMAGE" - docker push "$IMAGE" - - newimage=$(docker inspect $IMAGE | python -c 'import json,sys; print json.load(sys.stdin)[0]["RepoDigests"][0]') + - newimage=$(docker inspect $IMAGE | python -c 'import json,sys; print(json.load(sys.stdin)[0]["RepoDigests"][0])') - echo $newimage - docker rmi "$IMAGE" && docker system prune -f - make operator-yaml IMG=$newimage From b171222549af5110a676c29dfb99ee2f9f385896 Mon Sep 17 00:00:00 2001 From: Mohammed Yunus Date: Mon, 2 Jan 2023 14:09:00 +0000 Subject: [PATCH 520/628] Fix image digest determination in .gitlab-ci.yml --- .gitlab-ci.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 8f1824a2..c7734479 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -7,7 +7,7 @@ build-operator: - echo $CI_COMMIT_TAG - make docker-build IMG="$IMAGE" - docker push "$IMAGE" - - newimage=$(docker inspect $IMAGE | python -c 'import json,sys; print(json.load(sys.stdin)[0]["RepoDigests"][0])') + - newimage=$DOCKER_REPO@$(skopeo inspect docker://$IMAGE | jq -r .Digest) - echo $newimage - docker rmi "$IMAGE" && docker system prune -f - make operator-yaml IMG=$newimage From 3fefa26d6efdb9627d9b65c1a322472418dd255c Mon Sep 17 00:00:00 2001 From: matteo malvezzi Date: Wed, 4 Jan 2023 15:47:26 +0100 Subject: [PATCH 521/628] Add changes to pdb controller + usecase02 --- controllers/database/pdb_controller.go | 39 +- docs/multitenant/README.md | 2 + docs/multitenant/usecase02/README.md | 618 ++++++++++++++++++ docs/multitenant/usecase02/pdb_clone.yaml | 42 ++ docs/multitenant/usecase02/pdb_plug.yaml | 37 ++ docs/multitenant/usecase02/pdb_plugtde.yaml | 56 ++ docs/multitenant/usecase02/pdb_unplug.yaml | 31 + docs/multitenant/usecase02/pdb_unplugtde.yaml | 54 ++ docs/multitenant/usecase02/tde_secret.yaml | 15 + 9 files changed, 887 insertions(+), 7 deletions(-) create mode 100644 docs/multitenant/usecase02/README.md create mode 100644 docs/multitenant/usecase02/pdb_clone.yaml create mode 100644 docs/multitenant/usecase02/pdb_plug.yaml create mode 100644 docs/multitenant/usecase02/pdb_plugtde.yaml create mode 100644 docs/multitenant/usecase02/pdb_unplug.yaml create mode 100644 docs/multitenant/usecase02/pdb_unplugtde.yaml create mode 100644 docs/multitenant/usecase02/tde_secret.yaml diff --git a/controllers/database/pdb_controller.go b/controllers/database/pdb_controller.go index d40bd7ec..91d7c440 100644 --- a/controllers/database/pdb_controller.go +++ b/controllers/database/pdb_controller.go @@ -229,8 +229,8 @@ func (r *PDBReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.R err = r.getPDBState(ctx, req, pdb) case pdbPhaseMap: err = r.mapPDB(ctx, req, pdb) - case pdbPhaseFail: - err = r.mapPDB(ctx, req, pdb) + case pdbPhaseFail: + err = r.mapPDB(ctx, req, pdb) default: log.Info("DEFAULT:", "Name", pdb.Name, "Phase", phase, "Status", strconv.FormatBool(pdb.Status.Status)) return requeueN, nil @@ -904,12 +904,24 @@ func (r *PDBReconciler) modifyPDB(ctx context.Context, req ctrl.Request, pdb *db return err } - // To prevent Reconcile from Modifying again whenever the Operator gets re-started - modOption := pdb.Spec.PDBState + "-" + pdb.Spec.ModifyOption - if pdb.Status.ModifyOption == modOption { + if pdb.Status.OpenMode == "READ WRITE" && pdb.Spec.PDBState == "OPEN" && pdb.Spec.ModifyOption == "READ WRITE" { + /* Database is already open no action required */ + return nil + } + + if pdb.Status.OpenMode == "MOUNTED" && pdb.Spec.PDBState == "CLOSE" && pdb.Spec.ModifyOption == "IMMEDIATE" { + /* Database is already close no action required */ return nil } + // To prevent Reconcile from Modifying again whenever the Operator gets re-started + /* + modOption := pdb.Spec.PDBState + "-" + pdb.Spec.ModifyOption + if pdb.Status.ModifyOption == modOption { + return nil + } + */ + cdb, err := r.getCDBResource(ctx, req, pdb) if err != nil { return err @@ -919,6 +931,8 @@ func (r *PDBReconciler) modifyPDB(ctx context.Context, req ctrl.Request, pdb *db "state": pdb.Spec.PDBState, "modifyOption": pdb.Spec.ModifyOption, "getScript": strconv.FormatBool(*(pdb.Spec.GetScript))} + log.Info("MODIFY PDB", "pdb.Spec.PDBState=", pdb.Spec.PDBState, "pdb.Spec.ModifyOption=", pdb.Spec.ModifyOption) + log.Info("PDB STATUS OPENMODE", "pdb.Status.OpenMode=", pdb.Status.OpenMode) pdbName := pdb.Spec.PDBName url := "https://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdbName + "/status" @@ -973,8 +987,8 @@ func (r *PDBReconciler) getPDBState(ctx context.Context, req ctrl.Request, pdb * if err != nil { pdb.Status.OpenMode = "UNKNOWN" - pdb.Status.Msg = "CHECK PDB STATUS" - pdb.Status.Status = false + pdb.Status.Msg = "CHECK PDB STATUS" + pdb.Status.Status = false return err } @@ -985,6 +999,13 @@ func (r *PDBReconciler) getPDBState(ctx context.Context, req ctrl.Request, pdb * pdb.Status.OpenMode = objmap["open_mode"].(string) + if pdb.Status.OpenMode == "READ WRITE" { + err := r.mapPDB(ctx, req, pdb) + if err != nil { + log.Info("Fail to Map resource getting PDB state") + } + } + log.Info("Successfully obtained PDB state", "PDB Name", pdb.Spec.PDBName, "State", objmap["open_mode"].(string)) return nil } @@ -1036,6 +1057,10 @@ func (r *PDBReconciler) mapPDB(ctx context.Context, req ctrl.Request, pdb *dbapi pdb.Status.ConnString = cdb.Spec.DBTnsurl } + if err := r.Status().Update(ctx, pdb); err != nil { + log.Error(err, "Failed to update status for :"+pdb.Name, "err", err.Error()) + } + log.Info("Successfully mapped PDB to Kubernetes resource", "PDB Name", pdb.Spec.PDBName) return nil } diff --git a/docs/multitenant/README.md b/docs/multitenant/README.md index be311094..e9452497 100644 --- a/docs/multitenant/README.md +++ b/docs/multitenant/README.md @@ -4,6 +4,8 @@ # Oracle Multitenant Database Controller +> WARNING: Examples with https are located in the usecases directories + CDBs and PDBs are the part of the Oracle Database's [Multitenant Architecture](https://docs.oracle.com/en/database/oracle/oracle-database/21/multi/introduction-to-the-multitenant-architecture.html#GUID-AB84D6C9-4BBE-4D36-992F-2BB85739329F). The Multitenant Database Controller is a feature of Oracle DB Operator for Kubernetes (OraOperator) which helps to manage the life cycle of Pluggable Databases (PDBs) in an Oracle Container Database(CDB). The target CDB (for which the PDB life cycle management is needed) can be running on an multitenant machine and to manage its PDBs, the Oracle DB Operator can run on an multitenant Kubernetes system (For Example: [Oracle Linux Cloud Native Environment or OLCNE](https://docs.oracle.com/en/operating-systems/olcne/)). diff --git a/docs/multitenant/usecase02/README.md b/docs/multitenant/usecase02/README.md new file mode 100644 index 00000000..f43d1f26 --- /dev/null +++ b/docs/multitenant/usecase02/README.md @@ -0,0 +1,618 @@ + + + +# UNPLUG - PLUG - CLONE + +- [UNPLUG - PLUG - CLONE](#unplug---plug---clone) + - [INTRODUCTION](#introduction) + - [UNPLUG DATABASE](#unplug-database) + - [PLUG DATABASE](#plug-database) + - [CLONE PDB](#clone-pdb) + - [UNPLUG AND PLUG WITH TDE](#unplug-and-plug-with-tde) + +### INTRODUCTION + +This page explains how to plug and unplug database a pdb; it assumes that you have already configured a pluggable database (see usecase01) +The following table reports the parameters required to configure and use oracle multi tenant controller for pluggable database lifecycle management. + +| yaml file parameters | value | description /ords parameter | +|-------------- |--------------------------- |-------------------------------------------------| +| dbserver | or | [--db-hostname][1] | +| dbTnsurl | | [--db-custom-url/db.customURL][dbtnsurl] | +| port | | [--db-port][2] | +| cdbName | | Container Name | +| name | | Ords podname prefix in cdb.yaml | +| name | | pdb resource in pdb.yaml | +| ordsImage | /ords-dboper:latest|My public container registry | +| pdbName | | Pluggable database name | +| servicename | | [--db-servicename][3] | +| sysadmin_user | | [--admin-user][adminuser] | +| sysadmin_pwd | | [--password-stdin][pwdstdin] | +| cdbadmin_user | | [db.cdb.adminUser][1] | +| cdbadmin_pwd | | [db.cdb.adminUser.password][cdbadminpwd] | +| webserver_user| | [https user][http] NOT A DB USER | +| webserver_pwd | | [http user password][http] | +| ords_pwd | | [ORDS_PUBLIC_USER password][public_user] | +| pdbTlsKey | | [standalone.https.cert.key][key] | +| pdbTlsCrt | | [standalone.https.cert][cr] | +| pdbTlsCat | | certificate authority | +| xmlFileName | | path for the unplug and plug operation | +| srcPdbName | | name of the database to be cloned | +| fileNameConversions | | used for database cloning | +| tdeKeystorePath | | [tdeKeystorePath][tdeKeystorePath] | +| tdeExport | | [tdeExport] | +| tdeSecret | | [tdeSecret][tdeSecret] | +| tdePassword | | [tdeSecret][tdeSecret] | + +```text + + + +--------------------------------+ + UNPLUG PDB PLUG PDB | CLONE PDB | + | | + +-----------+ +-----------+ | +-----------+ +----------+ | + | PDB | | PDB | | | PDB | |CLONED PDB| | + +----+------+ +----+------+ | +----+------+ +----------+ | + | | | | | | ++----> UNPLUG -----+ +--> PLUG | CLONE ---------+ | +| | | | | | | | +| +----+------+ | | +----+------+ | +----+------+ | +| | Container | | | | Container | | | Container | | +| | | | | | | | | | | +| +-----------+ | | +-----------+ | +-----------+ | +| | | | | +| +------+----+ | | kubectk apply -f pdb_clone.yaml| +| | | | | | +| +------|-----------|--------+ | +--------------------------------+ +| | +----+----+ +--+------+ | | +| | |xml file | |DB FILES | |--+ +| | +---------+ +---------+ | | +| +---------------------------+ | +| | +| | ++- kubectl apply -f pdb_unplug.yaml | + | + kubectl apply -f pdb_plug.yaml-----+ +``` + +### UNPLUG DATABASE + +Use the following command to check kubernets pdb resources. Note that the output of the commands can be tailored to fit to your needs. Just check the structure of pdb resource **kubectl get pdbs -n oracle-database-operator-system -o=json** and modify the script accordingly. For the sake of simplicity put this command in a single script **checkpdbs.sh**. + +```bash +kubectl get pdbs -n oracle-database-operator-system -o=jsonpath='{range .items[*]} +{"\n==================================================================\n"} +{"CDB="}{.metadata.labels.cdb} +{"K8SNAME="}{.metadata.name} +{"PDBNAME="}{.spec.pdbName} +{"OPENMODE="}{.status.openMode} +{"ACTION="}{.status.action} +{"MSG="}{.status.msg} +{"\n"}{end}' +``` + +We assume that the pluggable database pdbdev is already configured on opened in read write mode + +```bash +./checkpdbs.sh +================================================================== +CDB=cdb-dev +K8SNAME=pdb1 +PDBNAME=pdbdev +OPENMODE=READ WRITE +ACTION=CREATE +MSG=Success + +``` + +Prepare a new yaml file **pdb_unplug.yaml** to unplug the pdbdev database. Make sure that the path of the xml file is correct and check the existence of all the required secrets. + +```yaml +#pdb_unplug.yaml +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbName: "DB12" + pdbName: "pdbdev" + xmlFileName: "/tmp/pdbunplug.xml" + action: "Unplug" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" +``` + +Close the pluggable database by applying the following yaml file **pdb_close.yaml** + +```yaml +#pdb_close.yaml +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbName: "DB12" + pdbName: "pdbdev" + adminName: + secret: + secretName: "pdb1-secret" + key: "sysadmin_user" + adminPwd: + secret: + secretName: "pdb1-secret" + key: "sysadmin_pwd" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + pdbState: "CLOSE" + modifyOption: "IMMEDIATE" + action: "Modify" +``` + +```bash +kubectl apply -f pdb_close.yaml +pdb.database.oracle.com/pdb1 configured + +sh checkpdbs.sh +================================================================== +CDB=cdb-dev +K8SNAME=pdb1 +PDBNAME=pdbdev +OPENMODE=MOUNTED +ACTION=MODIFY +MSG=Success +``` +After that apply the unplug file **pdb_unplug.yaml** ; The resource is no longer available once the unplug operation is completed. + +```bash +kubectl apply -f pdb_unplug.yaml +pdb.database.oracle.com/pdb1 configured + +sh checkpdbs.sh +================================================================== +CDB=cdb-dev +K8SNAME=pdb1 +PDBNAME=pdbdev +OPENMODE=MOUNTED +ACTION=MODIFY +MSG=Waiting for PDB to be unplugged +``` + +Check kubernets log files and the database alert log + +```text +/usr/bin/kubectl logs -f pod/`/usr/bin/kubectl get pods -n oracle-database-operator-system|grep oracle-database-operator-controller|head -1|cut -d ' ' -f 1` -n oracle-database-operator-system +[...] +base-oracle-com-v1alpha1-pdb", "UID": "6f469423-85e5-4287-94d5-3d91a04b621e", "kind": "database.oracle.com/v1alpha1, Kind=PDB", "resource": {"group":"database.oracle.com","version":"v1alpha1","resource":"pdbs"}} +2023-01-03T14:04:05Z INFO pdb-webhook ValidateUpdate-Validating PDB spec for : pdb1 +2023-01-03T14:04:05Z INFO pdb-webhook validateCommon {"name": "pdb1"} +2023-01-03T14:04:05Z INFO pdb-webhook Valdiating PDB Resource Action : UNPLUG +2023-01-03T14:04:05Z DEBUG controller-runtime.webhook.webhooks wrote response {"webhook": "/validate-database-oracle-com-v1alpha1-pdb", "code": 200, "reason": "", "UID": "6f469423-85e5-4287-94d5-3d91a04b621e", "allowed": true} + + +[database alert log] +Domain Action Reconfiguration complete (total time 0.0 secs) +Completed: ALTER PLUGGABLE DATABASE "pdbdev" UNPLUG INTO '/tmp/pdbunplug.xml' +DROP PLUGGABLE DATABASE "pdbdev" KEEP DATAFILES +2023-01-03T14:04:05.518845+00:00 +Deleted Oracle managed file +DATA/DB12/F146D9482AA0260FE0531514000AB1BC/TEMPFILE/temp.266.1125061101 +2023-01-03T14:04:05.547820+00:00 +Stopped service pdbdev +Completed: DROP PLUGGABLE DATABASE "pdbdev" KEEP DATAFILES + +``` + + +login to the server and check xml file existence. Get the datafile path on the ASM filesystem. + +```bash +ls -ltr /tmp/pdbunplug.xml +-rw-r--r--. 1 oracle asmadmin 8007 Jan 3 14:04 /tmp/pdbunplug.xml +[..] +cat /tmp/pdbunplug.xml |grep path + +DATA/DB12/F146D9482AA0260FE0531514000AB1BC/DATAFILE/system.353.1125061021 + +DATA/DB12/F146D9482AA0260FE0531514000AB1BC/DATAFILE/sysaux.328.1125061021 + +DATA/DB12/F146D9482AA0260FE0531514000AB1BC/DATAFILE/undotbs1.347.1125061021 + +DATA/DB12/F146D9482AA0260FE0531514000AB1BC/TEMPFILE/temp.266.1125061101 + +DATA/DB12/F146D9482AA0260FE0531514000AB1BC/DATAFILE/undo_2.318.1125061021 +[..] +asmcmd ls -l +DATA/DB12/F146D9482AA0260FE0531514000AB1BC/DATAFILE/system.353.1125061021 +Type Redund Striped Time Sys Name +DATAFILE UNPROT COARSE JAN 03 14:00:00 Y system.353.1125061021 +``` + +### PLUG DATABASE + +Prepare a new yaml file **pdb_plug.yaml** to plug the database back into the container. + +```yaml + +# pdb_plug.yaml +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbName: "DB12" + pdbName: "pdbdev" + xmlFileName: "/tmp/pdbunplug.xml" + fileNameConversions: "NONE" + sourceFileNameConversions: "NONE" + copyAction: "MOVE" + totalSize: "1G" + tempSize: "100M" + action: "Plug" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + +``` +Apply **pdb_plug.yaml** + +```bash +kubectl apply -f pdb_plug.yaml +[...] +sh checkpdbs.sh +================================================================== +CDB=cdb-dev +K8SNAME=pdb1 +PDBNAME=pdbdev +OPENMODE= +ACTION= +MSG=Waiting for PDB to be plugged +[...] +sh checkpdbs.sh +================================================================== +CDB=cdb-dev +K8SNAME=pdb1 +PDBNAME=pdbdev +OPENMODE=READ WRITE +ACTION=PLUG +MSG=Success +``` + +Check kubernets log files and the database alert log + +```text +/usr/bin/kubectl logs -f pod/`/usr/bin/kubectl get pods -n oracle-database-operator-system|grep oracle-database-operator-controller|head -1|cut -d ' ' -f 1` -n oracle-database-operator-system + +2023-01-03T14:33:51Z INFO pdb-webhook ValidateCreate-Validating PDB spec for : pdb1 +2023-01-03T14:33:51Z INFO pdb-webhook validateCommon {"name": "pdb1"} +2023-01-03T14:33:51Z INFO pdb-webhook Valdiating PDB Resource Action : PLUG +2023-01-03T14:33:51Z INFO pdb-webhook PDB Resource : pdb1 successfully validated for Action : PLUG +2023-01-03T14:33:51Z DEBUG controller-runtime.webhook.webhooks wrote response {"webhook": "/validate-database-oracle-com-v1alpha1-pdb", "code": 200, "reason": "", "UID": "fccac7ba-7540-42ff-93b2-46675506a098", "allowed": true} +2023-01-03T14:34:16Z DEBUG controller-runtime.webhook.webhooks received request {"webhook": "/mutate-database-oracle-com-v1alpha1-pdb", "UID": "766dadcc-aeea-4a80-bc17-e957b4a44d3c", "kind": "database.oracle.com/v1alpha1, Kind=PDB", "resource": {"group":"database.oracle.com","version":"v1alpha1","resource":"pdbs"}} +2023-01-03T14:34:16Z INFO pdb-webhook Setting default values in PDB spec for : pdb1 +2023-01-03T14:34:16Z DEBUG controller-runtime.webhook.webhooks wrote response {"webhook": "/mutate-database-oracle-com-v1alpha1-pdb", "code": 200, "reason": "", "UID": "766dadcc-aeea-4a80-bc17-e957b4a44d3c", "allowed": true} + +[database alert log] +... +All grantable enqueues granted +freeing rdom 3 +freeing the fusion rht of pdb 3 +freeing the pdb enqueue rht +Domain Action Reconfiguration complete (total time 0.0 secs) +Completed: CREATE PLUGGABLE DATABASE "pdbdev" + USING '/tmp/pdbunplug.xml' + SOURCE_FILE_NAME_CONVERT=NONE + MOVE + FILE_NAME_CONVERT=NONE + STORAGE UNLIMITED TEMPFILE REUSE + +2023-01-03T14:35:41.500186+00:00 +ALTER PLUGGABLE DATABASE "pdbdev" OPEN READ WRITE INSTANCES=ALL +2023-01-03T14:35:41.503482+00:00 +PDBDEV(3):Pluggable database PDBDEV opening in read write +PDBDEV(3):SUPLOG: Initialize PDB SUPLOG SGA, old value 0x0, new value 0x18 +PDBDEV(3):Autotune of undo retention is turned on +... +``` +### CLONE PDB + +Prepare and apply a new yaml file **pdb_clone.yaml** to clone the existing pluggable database. + +```yaml +#pdb_clone.yaml +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb2 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbName: "DB12" + pdbName: "pdb2-clone" + srcPdbName: "pdbdev" + fileNameConversions: "NONE" + totalSize: "UNLIMITED" + tempSize: "UNLIMITED" + adminName: + secret: + secretName: "pdb1-secret" + key: "sysadmin_user" + adminPwd: + secret: + secretName: "pdb1-secret" + key: "sysadmin_pwd" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + action: "Clone" + +``` +```bash +kubectl apply -f pdb_clone.yaml +pdb.database.oracle.com/pdb2 created +[oracle@mitk01 https.ords.22]$ sh checkpdbs.sh +================================================================== +CDB=cdb-dev +K8SNAME=pdb1 +PDBNAME=pdbdev +OPENMODE=READ WRITE +ACTION=PLUG +MSG=Success +================================================================== +CDB=cdb-dev +K8SNAME=pdb2 +PDBNAME=pdb2-clone +OPENMODE= +ACTION= +MSG=Waiting for PDB to be cloned +[...] +[.wait sometimes..] + sh checkpdbs.sh +================================================================== +CDB=cdb-dev +K8SNAME=pdb1 +PDBNAME=pdbdev +OPENMODE=READ WRITE +ACTION=PLUG +MSG=Success +================================================================== +CDB=cdb-dev +K8SNAME=pdb2 +PDBNAME=pdb2-clone +OPENMODE=READ WRITE +ACTION=CLONE +MSG=Success +``` +log info + +```text +[kubernets log] +2023-01-03T15:13:31Z INFO pdb-webhook - asClone : false +2023-01-03T15:13:31Z INFO pdb-webhook - getScript : false +2023-01-03T15:13:31Z DEBUG controller-runtime.webhook.webhooks wrote response {"webhook": "/mutate-database-oracle-com-v1alpha1-pdb", "code": 200, "reason": "", "UID": "7c17a715-7e4e-47d4-ad42-dcb37526bb3e", "allowed": true} +2023-01-03T15:13:31Z DEBUG controller-runtime.webhook.webhooks received request {"webhook": "/validate-database-oracle-com-v1alpha1-pdb", "UID": "11e0d49c-afaa-47ac-a301-f1fdd1e70173", "kind": "database.oracle.com/v1alpha1, Kind=PDB", "resource": {"group":"database.oracle.com","version":"v1alpha1","resource":"pdbs"}} +2023-01-03T15:13:31Z INFO pdb-webhook ValidateCreate-Validating PDB spec for : pdb2 +2023-01-03T15:13:31Z INFO pdb-webhook validateCommon {"name": "pdb2"} +2023-01-03T15:13:31Z INFO pdb-webhook Valdiating PDB Resource Action : CLONE +2023-01-03T15:13:31Z INFO pdb-webhook PDB Resource : pdb2 successfully validated for Action : CLONE +2023-01-03T15:13:31Z DEBUG controller-runtime.webhook.webhooks wrote response {"webhook": "/validate-database-oracle-com-v1alpha1-pdb", "code": 200, "reason": "", "UID": "11e0d49c-afaa-47ac-a301-f1fdd1e70173", "allowed": true} + +[database alert log] +Domain Action Reconfiguration complete (total time 0.0 secs) +2023-01-03T15:15:00.670436+00:00 +Completed: CREATE PLUGGABLE DATABASE "pdb2-clone" FROM "pdbdev" + STORAGE UNLIMITED + TEMPFILE REUSE + FILE_NAME_CONVERT=NONE +ALTER PLUGGABLE DATABASE "pdbdev" CLOSE IMMEDIATE INSTANCES=ALL +2023-01-03T15:15:00.684271+00:00 +PDBDEV(3):Pluggable database PDBDEV closing +PDBDEV(3):JIT: pid 8235 requesting stop +PDBDEV(3):Buffer Cache flush started: 3 +PDBDEV(3):Buffer Cache flush finished: 3 + +``` +### UNPLUG AND PLUG WITH TDE + +You can use unplug and plug database with TDE; in order to do that you have to specify a key store path and create new kubernets secret for TDE using the following yaml file. **tde_secrete.yaml**. The procedure to unplug and plug database does not change apply the same file. + +```yaml +#tde_secret +apiVersion: v1 +kind: Secret +metadata: + name: tde1-secret + namespace: oracle-database-operator-system +type: Opaque +data: + tdepassword: "d2VsY29tZTEK" + tdesecret: "bW1hbHZlenoK" +``` + +```bash +kubectl apply -f tde_secret.yaml +``` + +The file to unplug and plug database with TDE are the following + + +```yaml +#pdb_unplugtde.yaml +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbName: "DB12" + pdbName: "pdbdev" + adminName: + secret: + secretName: pdb1-secret + key: "sysadmin_user" + adminPwd: + secret: + secretName: pdb1-secret + key: "sysadmin_pwd" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + tdePassword: + secret: + secretName: "tde1-secret" + key: "tdepassword" + tdeSecret: + secret: + secretName: "tde1-secret" + key: "tdesecret" + totalSize: 1G + tempSize: 1G + unlimitedStorage: true + reuseTempFile: true + fileNameConversions: NONE + action: "Unplug" + xmlFileName: "/home/oracle/unplugpdb.xml" + tdeExport: true +``` + +```yaml +#pdb_plugtde.ymal +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbName: "DB12" + pdbName: "pdbdev" + adminName: + secret: + secretName: pdb1-secret + key: "sysadmin_user" + adminPwd: + secret: + secretName: pdb1-secret + key: "sysadmin_pwd" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + tdePassword: + secret: + secretName: "tde1-secret" + key: "tdepassword" + tdeSecret: + secret: + secretName: "tde1-secret" + key: "tdesecret" + totalSize: 1G + tempSize: "100M" + unlimitedStorage: true + reuseTempFile: true + fileNameConversions: NONE + sourceFileNameConversions: "NONE" + copyAction: "MOVE" + action: "Plug" + xmlFileName: /home/oracle/unplugpdb.xml + tdeImport: true + tdeKeystorePath: /home/oracle/keystore + +``` + + + + + + +[1]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/installing-and-configuring-oracle-rest-data-services.html#GUID-E9625FAB-9BC8-468B-9FF9-443C88D76FA1:~:text=Table%202%2D2%20Command%20Options%20for%20Command%2DLine%20Interface%20Installation + +[2]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/installing-and-configuring-oracle-rest-data-services.html#GUID-E9625FAB-9BC8-468B-9FF9-443C88D76FA1:~:text=Table%202%2D2%20Command%20Options%20for%20Command%2DLine%20Interface%20Installation + +[3]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/installing-and-configuring-oracle-rest-data-services.html#GUID-DAA027FA-A4A6-43E1-B8DD-C92B330C2341:~:text=%2D%2Ddb%2Dservicename%20%3Cstring%3E + +[adminuser]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/installing-and-configuring-oracle-rest-data-services.html#GUID-A9AED253-4EEC-4E13-A0C4-B7CE82EC1C22:~:text=Table%202%2D6%20Command%20Options%20for%20Uninstall%20CLI + +[public_user]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/using-multitenant-architecture-oracle-rest-data-services.html#GUID-E64A141A-A71F-4979-8D33-C5F8496D3C19:~:text=Preinstallation%20Tasks%20for%20Oracle%20REST%20Data%20Services%20CDB%20Installation + +[key]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/about-REST-configuration-files.html#GUID-006F916B-8594-4A78-B500-BB85F35C12A0:~:text=standalone.https.cert.key + +[cr]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/about-REST-configuration-files.html#GUID-006F916B-8594-4A78-B500-BB85F35C12A0 + +[cdbadminpwd]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/about-REST-configuration-files.html#GUID-006F916B-8594-4A78-B500-BB85F35C12A0:~:text=Table%20C%2D1%20Oracle%20REST%20Data%20Services%20Configuration%20Settings + + +[pwdstdin]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/installing-and-configuring-oracle-rest-data-services.html#GUID-88479C84-CAC1-4133-A33E-7995A645EC05:~:text=default%20database%20pool.-,2.1.4.1%20Understanding%20Command%20Options%20for%20Command%2DLine%20Interface%20Installation,-Table%202%2D2 + +[http]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/installing-and-configuring-oracle-rest-data-services.html#GUID-BEECC057-A8F5-4EAB-B88E-9828C2809CD8:~:text=Example%3A%20delete%20%5B%2D%2Dglobal%5D-,user%20add,-Add%20a%20user + +[dbtnsurl]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/installing-and-configuring-oracle-rest-data-services.html#GUID-A9AED253-4EEC-4E13-A0C4-B7CE82EC1C22 + +[tdeKeystorePath]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/21.4/orrst/op-database-pdbs-pdb_name-post.html + +[tdeSecret]:https://docs.oracle.com/en/database/oracle/oracle-database/19/sqlrf/ADMINISTER-KEY-MANAGEMENT.html#GUID-E5B2746F-19DC-4E94-83EC-A6A5C84A3EA9 diff --git a/docs/multitenant/usecase02/pdb_clone.yaml b/docs/multitenant/usecase02/pdb_clone.yaml new file mode 100644 index 00000000..befae41e --- /dev/null +++ b/docs/multitenant/usecase02/pdb_clone.yaml @@ -0,0 +1,42 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb2 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbName: "DB12" + pdbName: "pdb2-clone" + srcPdbName: "pdbdev" + fileNameConversions: "NONE" + totalSize: "UNLIMITED" + tempSize: "UNLIMITED" + adminName: + secret: + secretName: "pdb1-secret" + key: "sysadmin_user" + adminPwd: + secret: + secretName: "pdb1-secret" + key: "sysadmin_pwd" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + action: "Clone" + diff --git a/docs/multitenant/usecase02/pdb_plug.yaml b/docs/multitenant/usecase02/pdb_plug.yaml new file mode 100644 index 00000000..ac035623 --- /dev/null +++ b/docs/multitenant/usecase02/pdb_plug.yaml @@ -0,0 +1,37 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbName: "DB12" + pdbName: "pdbdev" + xmlFileName: "/tmp/pdbunplug.xml" + fileNameConversions: "NONE" + sourceFileNameConversions: "NONE" + copyAction: "MOVE" + totalSize: "1G" + tempSize: "100M" + action: "Plug" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + + diff --git a/docs/multitenant/usecase02/pdb_plugtde.yaml b/docs/multitenant/usecase02/pdb_plugtde.yaml new file mode 100644 index 00000000..17d84346 --- /dev/null +++ b/docs/multitenant/usecase02/pdb_plugtde.yaml @@ -0,0 +1,56 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbName: "DB12" + pdbName: "pdbdev" + adminName: + secret: + secretName: pdb1-secret + key: "sysadmin_user" + adminPwd: + secret: + secretName: pdb1-secret + key: "sysadmin_pwd" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + tdePassword: + secret: + secretName: "tde1-secret" + key: "tdepassword" + tdeSecret: + secret: + secretName: "tde1-secret" + key: "tdesecret" + totalSize: 1G + tempSize: "100M" + unlimitedStorage: true + reuseTempFile: true + fileNameConversions: NONE + sourceFileNameConversions: "NONE" + copyAction: "MOVE" + action: "Plug" + xmlFileName: /home/oracle/unplugpdb.xml + tdeImport: true + tdeKeystorePath: /home/oracle/keystore + diff --git a/docs/multitenant/usecase02/pdb_unplug.yaml b/docs/multitenant/usecase02/pdb_unplug.yaml new file mode 100644 index 00000000..6d995ac8 --- /dev/null +++ b/docs/multitenant/usecase02/pdb_unplug.yaml @@ -0,0 +1,31 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbName: "DB12" + pdbName: "pdbdev" + xmlFileName: "/tmp/pdbunplug.xml" + action: "Unplug" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + diff --git a/docs/multitenant/usecase02/pdb_unplugtde.yaml b/docs/multitenant/usecase02/pdb_unplugtde.yaml new file mode 100644 index 00000000..4c26bffe --- /dev/null +++ b/docs/multitenant/usecase02/pdb_unplugtde.yaml @@ -0,0 +1,54 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +apiVersion: database.oracle.com/v1alpha1 +Kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbName: "DB12" + pdbName: "pdbdev" + adminName: + secret: + secretName: pdb1-secret + key: "sysadmin_user" + adminPwd: + secret: + secretName: pdb1-secret + key: "sysadmin_pwd" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + tdePassword: + secret: + secretName: "tde1-secret" + key: "tdepassword" + tdeSecret: + secret: + secretName: "tde1-secret" + key: "tdesecret" + totalSize: 1G + tempSize: 1G + unlimitedStorage: true + reuseTempFile: true + fileNameConversions: NONE + action: "Unplug" + xmlFileName: "/home/oracle/unplugpdb.xml" + tdeExport: true + tdeKeystorePath: "/home/oracle/keystore" + diff --git a/docs/multitenant/usecase02/tde_secret.yaml b/docs/multitenant/usecase02/tde_secret.yaml new file mode 100644 index 00000000..5975e27e --- /dev/null +++ b/docs/multitenant/usecase02/tde_secret.yaml @@ -0,0 +1,15 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +apiVersion: v1 +kind: Secret +metadata: + name: tde1-secret + namespace: oracle-database-operator-system +type: Opaque +data: + tdepassword: "Y2hhbmdlbWUK" + tdesecret: "Y2hhbmdlbWUK" + From 62b33ee8fc497d8ce485662c81757697ccf1523b Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Fri, 6 Jan 2023 23:10:00 +0000 Subject: [PATCH 522/628] Update .gitlab-ci.yml file --- .gitlab-ci.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index daf3c9e9..318e03f7 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -1,4 +1,4 @@ -build-operator: +build-operator: stage: build variables: IMAGE: "$DOCKER_REPO:$CI_COMMIT_BRANCH" From 2182d59f1cb939d52eabb5f1ed0776979a18aa22 Mon Sep 17 00:00:00 2001 From: nebojsa Date: Tue, 10 Jan 2023 10:22:35 +0100 Subject: [PATCH 523/628] :hammer: fix : swapped comments referencing prebuilt and xe MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Hi, the comments above the prebuilt and xe secrets seem to have been accidentaly swapped. This proposal is a change to make it right. Please accept my Pull-Request 🀠--- config/samples/sidb/singleinstancedatabase_secrets.yaml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/config/samples/sidb/singleinstancedatabase_secrets.yaml b/config/samples/sidb/singleinstancedatabase_secrets.yaml index 6504a6ac..2633973e 100644 --- a/config/samples/sidb/singleinstancedatabase_secrets.yaml +++ b/config/samples/sidb/singleinstancedatabase_secrets.yaml @@ -16,7 +16,7 @@ stringData: --- -## Oracle Database XE Admin password secret +## Prebuilt-Database Admin password secret apiVersion: v1 kind: Secret metadata: @@ -29,7 +29,7 @@ stringData: --- -## Prebuilt-Database Admin password secret +## Oracle Database XE Admin password secret apiVersion: v1 kind: Secret metadata: From d686e19fd63448e6ba24d355b84654437ff119b2 Mon Sep 17 00:00:00 2001 From: matteo malvezzi Date: Wed, 11 Jan 2023 12:44:15 +0100 Subject: [PATCH 524/628] Correct clone/create defect --- controllers/database/pdb_controller.go | 28 ++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/controllers/database/pdb_controller.go b/controllers/database/pdb_controller.go index 91d7c440..da4405a8 100644 --- a/controllers/database/pdb_controller.go +++ b/controllers/database/pdb_controller.go @@ -618,6 +618,18 @@ func (r *PDBReconciler) createPDB(ctx context.Context, req ctrl.Request, pdb *db return err } + /* Prevent creating an existing pdb */ + err = r.getPDBState(ctx, req, pdb) + if err != nil { + if apierrors.IsNotFound(err) { + log.Info("Check PDB not existence completed", "PDB Name", pdb.Spec.PDBName) + } + + } else { + log.Info("Database already exists ", "PDB Name", pdb.Spec.PDBName) + return nil + } + values := map[string]string{ "method": "CREATE", "pdb_name": pdb.Spec.PDBName, @@ -679,6 +691,10 @@ func (r *PDBReconciler) createPDB(ctx context.Context, req ctrl.Request, pdb *db /************************************************/ func (r *PDBReconciler) clonePDB(ctx context.Context, req ctrl.Request, pdb *dbapi.PDB) error { + if pdb.Spec.PDBName == pdb.Spec.SrcPDBName { + return nil + } + log := r.Log.WithValues("clonePDB", req.NamespacedName) var err error @@ -688,6 +704,18 @@ func (r *PDBReconciler) clonePDB(ctx context.Context, req ctrl.Request, pdb *dba return err } + /* Prevent cloning an existing pdb */ + err = r.getPDBState(ctx, req, pdb) + if err != nil { + if apierrors.IsNotFound(err) { + log.Info("Check PDB not existence completed", "PDB Name", pdb.Spec.PDBName) + } + + } else { + log.Info("Database already exists ", "PDB Name", pdb.Spec.PDBName) + return nil + } + values := map[string]string{ "method": "CLONE", "clonePDBName": pdb.Spec.PDBName, From 2241adbc03ca28eaef0f8b628da3de7d6f8c2d63 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Fri, 13 Jan 2023 15:15:47 +0000 Subject: [PATCH 525/628] Update Dockerfile --- Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Dockerfile b/Dockerfile index 01c2bf7b..ee625e1d 100644 --- a/Dockerfile +++ b/Dockerfile @@ -3,7 +3,7 @@ # # Build the manager binary -FROM golang:1.17 as builder +FROM golang:1.19 as builder WORKDIR /workspace # Copy the Go Modules manifests From 02f90864a4ff0ef5a63900b5a43c31867e5de8eb Mon Sep 17 00:00:00 2001 From: Mohammed Yunus Date: Wed, 21 Dec 2022 11:39:19 +0000 Subject: [PATCH 526/628] Update .gitlab-ci.yml file --- .gitlab-ci.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 318e03f7..84e9af6b 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -8,7 +8,7 @@ build-operator: - echo $CI_COMMIT_TAG - make docker-build IMG="$IMAGE" - docker push "$IMAGE" - - newimage=$(docker inspect $IMAGE | python -c 'import json,sys; print json.load(sys.stdin)[0]["RepoDigests"][0]') + - newimage=$(docker inspect $IMAGE | python -c 'import json,sys; print(json.load(sys.stdin)[0]["RepoDigests"][0])') - echo $newimage - docker rmi "$IMAGE" && docker system prune -f - make operator-yaml IMG=$newimage From e6103ce64756346447e57e2265ce04bd10c1f103 Mon Sep 17 00:00:00 2001 From: Mohammed Yunus Date: Mon, 2 Jan 2023 14:09:00 +0000 Subject: [PATCH 527/628] Fix image digest determination in .gitlab-ci.yml --- .gitlab-ci.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 84e9af6b..f472d414 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -8,7 +8,7 @@ build-operator: - echo $CI_COMMIT_TAG - make docker-build IMG="$IMAGE" - docker push "$IMAGE" - - newimage=$(docker inspect $IMAGE | python -c 'import json,sys; print(json.load(sys.stdin)[0]["RepoDigests"][0])') + - newimage=$DOCKER_REPO@$(skopeo inspect docker://$IMAGE | jq -r .Digest) - echo $newimage - docker rmi "$IMAGE" && docker system prune -f - make operator-yaml IMG=$newimage From 29a5a147bdd5ad3ab6e462842385815ae3286542 Mon Sep 17 00:00:00 2001 From: matteo malvezzi Date: Wed, 4 Jan 2023 15:47:26 +0100 Subject: [PATCH 528/628] Add changes to pdb controller + usecase02 --- controllers/database/pdb_controller.go | 39 +- docs/multitenant/README.md | 2 + docs/multitenant/usecase02/README.md | 618 ++++++++++++++++++ docs/multitenant/usecase02/pdb_clone.yaml | 42 ++ docs/multitenant/usecase02/pdb_plug.yaml | 37 ++ docs/multitenant/usecase02/pdb_plugtde.yaml | 56 ++ docs/multitenant/usecase02/pdb_unplug.yaml | 31 + docs/multitenant/usecase02/pdb_unplugtde.yaml | 54 ++ docs/multitenant/usecase02/tde_secret.yaml | 15 + 9 files changed, 887 insertions(+), 7 deletions(-) create mode 100644 docs/multitenant/usecase02/README.md create mode 100644 docs/multitenant/usecase02/pdb_clone.yaml create mode 100644 docs/multitenant/usecase02/pdb_plug.yaml create mode 100644 docs/multitenant/usecase02/pdb_plugtde.yaml create mode 100644 docs/multitenant/usecase02/pdb_unplug.yaml create mode 100644 docs/multitenant/usecase02/pdb_unplugtde.yaml create mode 100644 docs/multitenant/usecase02/tde_secret.yaml diff --git a/controllers/database/pdb_controller.go b/controllers/database/pdb_controller.go index d40bd7ec..91d7c440 100644 --- a/controllers/database/pdb_controller.go +++ b/controllers/database/pdb_controller.go @@ -229,8 +229,8 @@ func (r *PDBReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.R err = r.getPDBState(ctx, req, pdb) case pdbPhaseMap: err = r.mapPDB(ctx, req, pdb) - case pdbPhaseFail: - err = r.mapPDB(ctx, req, pdb) + case pdbPhaseFail: + err = r.mapPDB(ctx, req, pdb) default: log.Info("DEFAULT:", "Name", pdb.Name, "Phase", phase, "Status", strconv.FormatBool(pdb.Status.Status)) return requeueN, nil @@ -904,12 +904,24 @@ func (r *PDBReconciler) modifyPDB(ctx context.Context, req ctrl.Request, pdb *db return err } - // To prevent Reconcile from Modifying again whenever the Operator gets re-started - modOption := pdb.Spec.PDBState + "-" + pdb.Spec.ModifyOption - if pdb.Status.ModifyOption == modOption { + if pdb.Status.OpenMode == "READ WRITE" && pdb.Spec.PDBState == "OPEN" && pdb.Spec.ModifyOption == "READ WRITE" { + /* Database is already open no action required */ + return nil + } + + if pdb.Status.OpenMode == "MOUNTED" && pdb.Spec.PDBState == "CLOSE" && pdb.Spec.ModifyOption == "IMMEDIATE" { + /* Database is already close no action required */ return nil } + // To prevent Reconcile from Modifying again whenever the Operator gets re-started + /* + modOption := pdb.Spec.PDBState + "-" + pdb.Spec.ModifyOption + if pdb.Status.ModifyOption == modOption { + return nil + } + */ + cdb, err := r.getCDBResource(ctx, req, pdb) if err != nil { return err @@ -919,6 +931,8 @@ func (r *PDBReconciler) modifyPDB(ctx context.Context, req ctrl.Request, pdb *db "state": pdb.Spec.PDBState, "modifyOption": pdb.Spec.ModifyOption, "getScript": strconv.FormatBool(*(pdb.Spec.GetScript))} + log.Info("MODIFY PDB", "pdb.Spec.PDBState=", pdb.Spec.PDBState, "pdb.Spec.ModifyOption=", pdb.Spec.ModifyOption) + log.Info("PDB STATUS OPENMODE", "pdb.Status.OpenMode=", pdb.Status.OpenMode) pdbName := pdb.Spec.PDBName url := "https://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdbName + "/status" @@ -973,8 +987,8 @@ func (r *PDBReconciler) getPDBState(ctx context.Context, req ctrl.Request, pdb * if err != nil { pdb.Status.OpenMode = "UNKNOWN" - pdb.Status.Msg = "CHECK PDB STATUS" - pdb.Status.Status = false + pdb.Status.Msg = "CHECK PDB STATUS" + pdb.Status.Status = false return err } @@ -985,6 +999,13 @@ func (r *PDBReconciler) getPDBState(ctx context.Context, req ctrl.Request, pdb * pdb.Status.OpenMode = objmap["open_mode"].(string) + if pdb.Status.OpenMode == "READ WRITE" { + err := r.mapPDB(ctx, req, pdb) + if err != nil { + log.Info("Fail to Map resource getting PDB state") + } + } + log.Info("Successfully obtained PDB state", "PDB Name", pdb.Spec.PDBName, "State", objmap["open_mode"].(string)) return nil } @@ -1036,6 +1057,10 @@ func (r *PDBReconciler) mapPDB(ctx context.Context, req ctrl.Request, pdb *dbapi pdb.Status.ConnString = cdb.Spec.DBTnsurl } + if err := r.Status().Update(ctx, pdb); err != nil { + log.Error(err, "Failed to update status for :"+pdb.Name, "err", err.Error()) + } + log.Info("Successfully mapped PDB to Kubernetes resource", "PDB Name", pdb.Spec.PDBName) return nil } diff --git a/docs/multitenant/README.md b/docs/multitenant/README.md index be311094..e9452497 100644 --- a/docs/multitenant/README.md +++ b/docs/multitenant/README.md @@ -4,6 +4,8 @@ # Oracle Multitenant Database Controller +> WARNING: Examples with https are located in the usecases directories + CDBs and PDBs are the part of the Oracle Database's [Multitenant Architecture](https://docs.oracle.com/en/database/oracle/oracle-database/21/multi/introduction-to-the-multitenant-architecture.html#GUID-AB84D6C9-4BBE-4D36-992F-2BB85739329F). The Multitenant Database Controller is a feature of Oracle DB Operator for Kubernetes (OraOperator) which helps to manage the life cycle of Pluggable Databases (PDBs) in an Oracle Container Database(CDB). The target CDB (for which the PDB life cycle management is needed) can be running on an multitenant machine and to manage its PDBs, the Oracle DB Operator can run on an multitenant Kubernetes system (For Example: [Oracle Linux Cloud Native Environment or OLCNE](https://docs.oracle.com/en/operating-systems/olcne/)). diff --git a/docs/multitenant/usecase02/README.md b/docs/multitenant/usecase02/README.md new file mode 100644 index 00000000..f43d1f26 --- /dev/null +++ b/docs/multitenant/usecase02/README.md @@ -0,0 +1,618 @@ + + + +# UNPLUG - PLUG - CLONE + +- [UNPLUG - PLUG - CLONE](#unplug---plug---clone) + - [INTRODUCTION](#introduction) + - [UNPLUG DATABASE](#unplug-database) + - [PLUG DATABASE](#plug-database) + - [CLONE PDB](#clone-pdb) + - [UNPLUG AND PLUG WITH TDE](#unplug-and-plug-with-tde) + +### INTRODUCTION + +This page explains how to plug and unplug database a pdb; it assumes that you have already configured a pluggable database (see usecase01) +The following table reports the parameters required to configure and use oracle multi tenant controller for pluggable database lifecycle management. + +| yaml file parameters | value | description /ords parameter | +|-------------- |--------------------------- |-------------------------------------------------| +| dbserver | or | [--db-hostname][1] | +| dbTnsurl | | [--db-custom-url/db.customURL][dbtnsurl] | +| port | | [--db-port][2] | +| cdbName | | Container Name | +| name | | Ords podname prefix in cdb.yaml | +| name | | pdb resource in pdb.yaml | +| ordsImage | /ords-dboper:latest|My public container registry | +| pdbName | | Pluggable database name | +| servicename | | [--db-servicename][3] | +| sysadmin_user | | [--admin-user][adminuser] | +| sysadmin_pwd | | [--password-stdin][pwdstdin] | +| cdbadmin_user | | [db.cdb.adminUser][1] | +| cdbadmin_pwd | | [db.cdb.adminUser.password][cdbadminpwd] | +| webserver_user| | [https user][http] NOT A DB USER | +| webserver_pwd | | [http user password][http] | +| ords_pwd | | [ORDS_PUBLIC_USER password][public_user] | +| pdbTlsKey | | [standalone.https.cert.key][key] | +| pdbTlsCrt | | [standalone.https.cert][cr] | +| pdbTlsCat | | certificate authority | +| xmlFileName | | path for the unplug and plug operation | +| srcPdbName | | name of the database to be cloned | +| fileNameConversions | | used for database cloning | +| tdeKeystorePath | | [tdeKeystorePath][tdeKeystorePath] | +| tdeExport | | [tdeExport] | +| tdeSecret | | [tdeSecret][tdeSecret] | +| tdePassword | | [tdeSecret][tdeSecret] | + +```text + + + +--------------------------------+ + UNPLUG PDB PLUG PDB | CLONE PDB | + | | + +-----------+ +-----------+ | +-----------+ +----------+ | + | PDB | | PDB | | | PDB | |CLONED PDB| | + +----+------+ +----+------+ | +----+------+ +----------+ | + | | | | | | ++----> UNPLUG -----+ +--> PLUG | CLONE ---------+ | +| | | | | | | | +| +----+------+ | | +----+------+ | +----+------+ | +| | Container | | | | Container | | | Container | | +| | | | | | | | | | | +| +-----------+ | | +-----------+ | +-----------+ | +| | | | | +| +------+----+ | | kubectk apply -f pdb_clone.yaml| +| | | | | | +| +------|-----------|--------+ | +--------------------------------+ +| | +----+----+ +--+------+ | | +| | |xml file | |DB FILES | |--+ +| | +---------+ +---------+ | | +| +---------------------------+ | +| | +| | ++- kubectl apply -f pdb_unplug.yaml | + | + kubectl apply -f pdb_plug.yaml-----+ +``` + +### UNPLUG DATABASE + +Use the following command to check kubernets pdb resources. Note that the output of the commands can be tailored to fit to your needs. Just check the structure of pdb resource **kubectl get pdbs -n oracle-database-operator-system -o=json** and modify the script accordingly. For the sake of simplicity put this command in a single script **checkpdbs.sh**. + +```bash +kubectl get pdbs -n oracle-database-operator-system -o=jsonpath='{range .items[*]} +{"\n==================================================================\n"} +{"CDB="}{.metadata.labels.cdb} +{"K8SNAME="}{.metadata.name} +{"PDBNAME="}{.spec.pdbName} +{"OPENMODE="}{.status.openMode} +{"ACTION="}{.status.action} +{"MSG="}{.status.msg} +{"\n"}{end}' +``` + +We assume that the pluggable database pdbdev is already configured on opened in read write mode + +```bash +./checkpdbs.sh +================================================================== +CDB=cdb-dev +K8SNAME=pdb1 +PDBNAME=pdbdev +OPENMODE=READ WRITE +ACTION=CREATE +MSG=Success + +``` + +Prepare a new yaml file **pdb_unplug.yaml** to unplug the pdbdev database. Make sure that the path of the xml file is correct and check the existence of all the required secrets. + +```yaml +#pdb_unplug.yaml +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbName: "DB12" + pdbName: "pdbdev" + xmlFileName: "/tmp/pdbunplug.xml" + action: "Unplug" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" +``` + +Close the pluggable database by applying the following yaml file **pdb_close.yaml** + +```yaml +#pdb_close.yaml +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbName: "DB12" + pdbName: "pdbdev" + adminName: + secret: + secretName: "pdb1-secret" + key: "sysadmin_user" + adminPwd: + secret: + secretName: "pdb1-secret" + key: "sysadmin_pwd" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + pdbState: "CLOSE" + modifyOption: "IMMEDIATE" + action: "Modify" +``` + +```bash +kubectl apply -f pdb_close.yaml +pdb.database.oracle.com/pdb1 configured + +sh checkpdbs.sh +================================================================== +CDB=cdb-dev +K8SNAME=pdb1 +PDBNAME=pdbdev +OPENMODE=MOUNTED +ACTION=MODIFY +MSG=Success +``` +After that apply the unplug file **pdb_unplug.yaml** ; The resource is no longer available once the unplug operation is completed. + +```bash +kubectl apply -f pdb_unplug.yaml +pdb.database.oracle.com/pdb1 configured + +sh checkpdbs.sh +================================================================== +CDB=cdb-dev +K8SNAME=pdb1 +PDBNAME=pdbdev +OPENMODE=MOUNTED +ACTION=MODIFY +MSG=Waiting for PDB to be unplugged +``` + +Check kubernets log files and the database alert log + +```text +/usr/bin/kubectl logs -f pod/`/usr/bin/kubectl get pods -n oracle-database-operator-system|grep oracle-database-operator-controller|head -1|cut -d ' ' -f 1` -n oracle-database-operator-system +[...] +base-oracle-com-v1alpha1-pdb", "UID": "6f469423-85e5-4287-94d5-3d91a04b621e", "kind": "database.oracle.com/v1alpha1, Kind=PDB", "resource": {"group":"database.oracle.com","version":"v1alpha1","resource":"pdbs"}} +2023-01-03T14:04:05Z INFO pdb-webhook ValidateUpdate-Validating PDB spec for : pdb1 +2023-01-03T14:04:05Z INFO pdb-webhook validateCommon {"name": "pdb1"} +2023-01-03T14:04:05Z INFO pdb-webhook Valdiating PDB Resource Action : UNPLUG +2023-01-03T14:04:05Z DEBUG controller-runtime.webhook.webhooks wrote response {"webhook": "/validate-database-oracle-com-v1alpha1-pdb", "code": 200, "reason": "", "UID": "6f469423-85e5-4287-94d5-3d91a04b621e", "allowed": true} + + +[database alert log] +Domain Action Reconfiguration complete (total time 0.0 secs) +Completed: ALTER PLUGGABLE DATABASE "pdbdev" UNPLUG INTO '/tmp/pdbunplug.xml' +DROP PLUGGABLE DATABASE "pdbdev" KEEP DATAFILES +2023-01-03T14:04:05.518845+00:00 +Deleted Oracle managed file +DATA/DB12/F146D9482AA0260FE0531514000AB1BC/TEMPFILE/temp.266.1125061101 +2023-01-03T14:04:05.547820+00:00 +Stopped service pdbdev +Completed: DROP PLUGGABLE DATABASE "pdbdev" KEEP DATAFILES + +``` + + +login to the server and check xml file existence. Get the datafile path on the ASM filesystem. + +```bash +ls -ltr /tmp/pdbunplug.xml +-rw-r--r--. 1 oracle asmadmin 8007 Jan 3 14:04 /tmp/pdbunplug.xml +[..] +cat /tmp/pdbunplug.xml |grep path + +DATA/DB12/F146D9482AA0260FE0531514000AB1BC/DATAFILE/system.353.1125061021 + +DATA/DB12/F146D9482AA0260FE0531514000AB1BC/DATAFILE/sysaux.328.1125061021 + +DATA/DB12/F146D9482AA0260FE0531514000AB1BC/DATAFILE/undotbs1.347.1125061021 + +DATA/DB12/F146D9482AA0260FE0531514000AB1BC/TEMPFILE/temp.266.1125061101 + +DATA/DB12/F146D9482AA0260FE0531514000AB1BC/DATAFILE/undo_2.318.1125061021 +[..] +asmcmd ls -l +DATA/DB12/F146D9482AA0260FE0531514000AB1BC/DATAFILE/system.353.1125061021 +Type Redund Striped Time Sys Name +DATAFILE UNPROT COARSE JAN 03 14:00:00 Y system.353.1125061021 +``` + +### PLUG DATABASE + +Prepare a new yaml file **pdb_plug.yaml** to plug the database back into the container. + +```yaml + +# pdb_plug.yaml +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbName: "DB12" + pdbName: "pdbdev" + xmlFileName: "/tmp/pdbunplug.xml" + fileNameConversions: "NONE" + sourceFileNameConversions: "NONE" + copyAction: "MOVE" + totalSize: "1G" + tempSize: "100M" + action: "Plug" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + +``` +Apply **pdb_plug.yaml** + +```bash +kubectl apply -f pdb_plug.yaml +[...] +sh checkpdbs.sh +================================================================== +CDB=cdb-dev +K8SNAME=pdb1 +PDBNAME=pdbdev +OPENMODE= +ACTION= +MSG=Waiting for PDB to be plugged +[...] +sh checkpdbs.sh +================================================================== +CDB=cdb-dev +K8SNAME=pdb1 +PDBNAME=pdbdev +OPENMODE=READ WRITE +ACTION=PLUG +MSG=Success +``` + +Check kubernets log files and the database alert log + +```text +/usr/bin/kubectl logs -f pod/`/usr/bin/kubectl get pods -n oracle-database-operator-system|grep oracle-database-operator-controller|head -1|cut -d ' ' -f 1` -n oracle-database-operator-system + +2023-01-03T14:33:51Z INFO pdb-webhook ValidateCreate-Validating PDB spec for : pdb1 +2023-01-03T14:33:51Z INFO pdb-webhook validateCommon {"name": "pdb1"} +2023-01-03T14:33:51Z INFO pdb-webhook Valdiating PDB Resource Action : PLUG +2023-01-03T14:33:51Z INFO pdb-webhook PDB Resource : pdb1 successfully validated for Action : PLUG +2023-01-03T14:33:51Z DEBUG controller-runtime.webhook.webhooks wrote response {"webhook": "/validate-database-oracle-com-v1alpha1-pdb", "code": 200, "reason": "", "UID": "fccac7ba-7540-42ff-93b2-46675506a098", "allowed": true} +2023-01-03T14:34:16Z DEBUG controller-runtime.webhook.webhooks received request {"webhook": "/mutate-database-oracle-com-v1alpha1-pdb", "UID": "766dadcc-aeea-4a80-bc17-e957b4a44d3c", "kind": "database.oracle.com/v1alpha1, Kind=PDB", "resource": {"group":"database.oracle.com","version":"v1alpha1","resource":"pdbs"}} +2023-01-03T14:34:16Z INFO pdb-webhook Setting default values in PDB spec for : pdb1 +2023-01-03T14:34:16Z DEBUG controller-runtime.webhook.webhooks wrote response {"webhook": "/mutate-database-oracle-com-v1alpha1-pdb", "code": 200, "reason": "", "UID": "766dadcc-aeea-4a80-bc17-e957b4a44d3c", "allowed": true} + +[database alert log] +... +All grantable enqueues granted +freeing rdom 3 +freeing the fusion rht of pdb 3 +freeing the pdb enqueue rht +Domain Action Reconfiguration complete (total time 0.0 secs) +Completed: CREATE PLUGGABLE DATABASE "pdbdev" + USING '/tmp/pdbunplug.xml' + SOURCE_FILE_NAME_CONVERT=NONE + MOVE + FILE_NAME_CONVERT=NONE + STORAGE UNLIMITED TEMPFILE REUSE + +2023-01-03T14:35:41.500186+00:00 +ALTER PLUGGABLE DATABASE "pdbdev" OPEN READ WRITE INSTANCES=ALL +2023-01-03T14:35:41.503482+00:00 +PDBDEV(3):Pluggable database PDBDEV opening in read write +PDBDEV(3):SUPLOG: Initialize PDB SUPLOG SGA, old value 0x0, new value 0x18 +PDBDEV(3):Autotune of undo retention is turned on +... +``` +### CLONE PDB + +Prepare and apply a new yaml file **pdb_clone.yaml** to clone the existing pluggable database. + +```yaml +#pdb_clone.yaml +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb2 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbName: "DB12" + pdbName: "pdb2-clone" + srcPdbName: "pdbdev" + fileNameConversions: "NONE" + totalSize: "UNLIMITED" + tempSize: "UNLIMITED" + adminName: + secret: + secretName: "pdb1-secret" + key: "sysadmin_user" + adminPwd: + secret: + secretName: "pdb1-secret" + key: "sysadmin_pwd" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + action: "Clone" + +``` +```bash +kubectl apply -f pdb_clone.yaml +pdb.database.oracle.com/pdb2 created +[oracle@mitk01 https.ords.22]$ sh checkpdbs.sh +================================================================== +CDB=cdb-dev +K8SNAME=pdb1 +PDBNAME=pdbdev +OPENMODE=READ WRITE +ACTION=PLUG +MSG=Success +================================================================== +CDB=cdb-dev +K8SNAME=pdb2 +PDBNAME=pdb2-clone +OPENMODE= +ACTION= +MSG=Waiting for PDB to be cloned +[...] +[.wait sometimes..] + sh checkpdbs.sh +================================================================== +CDB=cdb-dev +K8SNAME=pdb1 +PDBNAME=pdbdev +OPENMODE=READ WRITE +ACTION=PLUG +MSG=Success +================================================================== +CDB=cdb-dev +K8SNAME=pdb2 +PDBNAME=pdb2-clone +OPENMODE=READ WRITE +ACTION=CLONE +MSG=Success +``` +log info + +```text +[kubernets log] +2023-01-03T15:13:31Z INFO pdb-webhook - asClone : false +2023-01-03T15:13:31Z INFO pdb-webhook - getScript : false +2023-01-03T15:13:31Z DEBUG controller-runtime.webhook.webhooks wrote response {"webhook": "/mutate-database-oracle-com-v1alpha1-pdb", "code": 200, "reason": "", "UID": "7c17a715-7e4e-47d4-ad42-dcb37526bb3e", "allowed": true} +2023-01-03T15:13:31Z DEBUG controller-runtime.webhook.webhooks received request {"webhook": "/validate-database-oracle-com-v1alpha1-pdb", "UID": "11e0d49c-afaa-47ac-a301-f1fdd1e70173", "kind": "database.oracle.com/v1alpha1, Kind=PDB", "resource": {"group":"database.oracle.com","version":"v1alpha1","resource":"pdbs"}} +2023-01-03T15:13:31Z INFO pdb-webhook ValidateCreate-Validating PDB spec for : pdb2 +2023-01-03T15:13:31Z INFO pdb-webhook validateCommon {"name": "pdb2"} +2023-01-03T15:13:31Z INFO pdb-webhook Valdiating PDB Resource Action : CLONE +2023-01-03T15:13:31Z INFO pdb-webhook PDB Resource : pdb2 successfully validated for Action : CLONE +2023-01-03T15:13:31Z DEBUG controller-runtime.webhook.webhooks wrote response {"webhook": "/validate-database-oracle-com-v1alpha1-pdb", "code": 200, "reason": "", "UID": "11e0d49c-afaa-47ac-a301-f1fdd1e70173", "allowed": true} + +[database alert log] +Domain Action Reconfiguration complete (total time 0.0 secs) +2023-01-03T15:15:00.670436+00:00 +Completed: CREATE PLUGGABLE DATABASE "pdb2-clone" FROM "pdbdev" + STORAGE UNLIMITED + TEMPFILE REUSE + FILE_NAME_CONVERT=NONE +ALTER PLUGGABLE DATABASE "pdbdev" CLOSE IMMEDIATE INSTANCES=ALL +2023-01-03T15:15:00.684271+00:00 +PDBDEV(3):Pluggable database PDBDEV closing +PDBDEV(3):JIT: pid 8235 requesting stop +PDBDEV(3):Buffer Cache flush started: 3 +PDBDEV(3):Buffer Cache flush finished: 3 + +``` +### UNPLUG AND PLUG WITH TDE + +You can use unplug and plug database with TDE; in order to do that you have to specify a key store path and create new kubernets secret for TDE using the following yaml file. **tde_secrete.yaml**. The procedure to unplug and plug database does not change apply the same file. + +```yaml +#tde_secret +apiVersion: v1 +kind: Secret +metadata: + name: tde1-secret + namespace: oracle-database-operator-system +type: Opaque +data: + tdepassword: "d2VsY29tZTEK" + tdesecret: "bW1hbHZlenoK" +``` + +```bash +kubectl apply -f tde_secret.yaml +``` + +The file to unplug and plug database with TDE are the following + + +```yaml +#pdb_unplugtde.yaml +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbName: "DB12" + pdbName: "pdbdev" + adminName: + secret: + secretName: pdb1-secret + key: "sysadmin_user" + adminPwd: + secret: + secretName: pdb1-secret + key: "sysadmin_pwd" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + tdePassword: + secret: + secretName: "tde1-secret" + key: "tdepassword" + tdeSecret: + secret: + secretName: "tde1-secret" + key: "tdesecret" + totalSize: 1G + tempSize: 1G + unlimitedStorage: true + reuseTempFile: true + fileNameConversions: NONE + action: "Unplug" + xmlFileName: "/home/oracle/unplugpdb.xml" + tdeExport: true +``` + +```yaml +#pdb_plugtde.ymal +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbName: "DB12" + pdbName: "pdbdev" + adminName: + secret: + secretName: pdb1-secret + key: "sysadmin_user" + adminPwd: + secret: + secretName: pdb1-secret + key: "sysadmin_pwd" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + tdePassword: + secret: + secretName: "tde1-secret" + key: "tdepassword" + tdeSecret: + secret: + secretName: "tde1-secret" + key: "tdesecret" + totalSize: 1G + tempSize: "100M" + unlimitedStorage: true + reuseTempFile: true + fileNameConversions: NONE + sourceFileNameConversions: "NONE" + copyAction: "MOVE" + action: "Plug" + xmlFileName: /home/oracle/unplugpdb.xml + tdeImport: true + tdeKeystorePath: /home/oracle/keystore + +``` + + + + + + +[1]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/installing-and-configuring-oracle-rest-data-services.html#GUID-E9625FAB-9BC8-468B-9FF9-443C88D76FA1:~:text=Table%202%2D2%20Command%20Options%20for%20Command%2DLine%20Interface%20Installation + +[2]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/installing-and-configuring-oracle-rest-data-services.html#GUID-E9625FAB-9BC8-468B-9FF9-443C88D76FA1:~:text=Table%202%2D2%20Command%20Options%20for%20Command%2DLine%20Interface%20Installation + +[3]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/installing-and-configuring-oracle-rest-data-services.html#GUID-DAA027FA-A4A6-43E1-B8DD-C92B330C2341:~:text=%2D%2Ddb%2Dservicename%20%3Cstring%3E + +[adminuser]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/installing-and-configuring-oracle-rest-data-services.html#GUID-A9AED253-4EEC-4E13-A0C4-B7CE82EC1C22:~:text=Table%202%2D6%20Command%20Options%20for%20Uninstall%20CLI + +[public_user]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/using-multitenant-architecture-oracle-rest-data-services.html#GUID-E64A141A-A71F-4979-8D33-C5F8496D3C19:~:text=Preinstallation%20Tasks%20for%20Oracle%20REST%20Data%20Services%20CDB%20Installation + +[key]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/about-REST-configuration-files.html#GUID-006F916B-8594-4A78-B500-BB85F35C12A0:~:text=standalone.https.cert.key + +[cr]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/about-REST-configuration-files.html#GUID-006F916B-8594-4A78-B500-BB85F35C12A0 + +[cdbadminpwd]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/about-REST-configuration-files.html#GUID-006F916B-8594-4A78-B500-BB85F35C12A0:~:text=Table%20C%2D1%20Oracle%20REST%20Data%20Services%20Configuration%20Settings + + +[pwdstdin]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/installing-and-configuring-oracle-rest-data-services.html#GUID-88479C84-CAC1-4133-A33E-7995A645EC05:~:text=default%20database%20pool.-,2.1.4.1%20Understanding%20Command%20Options%20for%20Command%2DLine%20Interface%20Installation,-Table%202%2D2 + +[http]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/installing-and-configuring-oracle-rest-data-services.html#GUID-BEECC057-A8F5-4EAB-B88E-9828C2809CD8:~:text=Example%3A%20delete%20%5B%2D%2Dglobal%5D-,user%20add,-Add%20a%20user + +[dbtnsurl]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/installing-and-configuring-oracle-rest-data-services.html#GUID-A9AED253-4EEC-4E13-A0C4-B7CE82EC1C22 + +[tdeKeystorePath]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/21.4/orrst/op-database-pdbs-pdb_name-post.html + +[tdeSecret]:https://docs.oracle.com/en/database/oracle/oracle-database/19/sqlrf/ADMINISTER-KEY-MANAGEMENT.html#GUID-E5B2746F-19DC-4E94-83EC-A6A5C84A3EA9 diff --git a/docs/multitenant/usecase02/pdb_clone.yaml b/docs/multitenant/usecase02/pdb_clone.yaml new file mode 100644 index 00000000..befae41e --- /dev/null +++ b/docs/multitenant/usecase02/pdb_clone.yaml @@ -0,0 +1,42 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb2 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbName: "DB12" + pdbName: "pdb2-clone" + srcPdbName: "pdbdev" + fileNameConversions: "NONE" + totalSize: "UNLIMITED" + tempSize: "UNLIMITED" + adminName: + secret: + secretName: "pdb1-secret" + key: "sysadmin_user" + adminPwd: + secret: + secretName: "pdb1-secret" + key: "sysadmin_pwd" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + action: "Clone" + diff --git a/docs/multitenant/usecase02/pdb_plug.yaml b/docs/multitenant/usecase02/pdb_plug.yaml new file mode 100644 index 00000000..ac035623 --- /dev/null +++ b/docs/multitenant/usecase02/pdb_plug.yaml @@ -0,0 +1,37 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbName: "DB12" + pdbName: "pdbdev" + xmlFileName: "/tmp/pdbunplug.xml" + fileNameConversions: "NONE" + sourceFileNameConversions: "NONE" + copyAction: "MOVE" + totalSize: "1G" + tempSize: "100M" + action: "Plug" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + + diff --git a/docs/multitenant/usecase02/pdb_plugtde.yaml b/docs/multitenant/usecase02/pdb_plugtde.yaml new file mode 100644 index 00000000..17d84346 --- /dev/null +++ b/docs/multitenant/usecase02/pdb_plugtde.yaml @@ -0,0 +1,56 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbName: "DB12" + pdbName: "pdbdev" + adminName: + secret: + secretName: pdb1-secret + key: "sysadmin_user" + adminPwd: + secret: + secretName: pdb1-secret + key: "sysadmin_pwd" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + tdePassword: + secret: + secretName: "tde1-secret" + key: "tdepassword" + tdeSecret: + secret: + secretName: "tde1-secret" + key: "tdesecret" + totalSize: 1G + tempSize: "100M" + unlimitedStorage: true + reuseTempFile: true + fileNameConversions: NONE + sourceFileNameConversions: "NONE" + copyAction: "MOVE" + action: "Plug" + xmlFileName: /home/oracle/unplugpdb.xml + tdeImport: true + tdeKeystorePath: /home/oracle/keystore + diff --git a/docs/multitenant/usecase02/pdb_unplug.yaml b/docs/multitenant/usecase02/pdb_unplug.yaml new file mode 100644 index 00000000..6d995ac8 --- /dev/null +++ b/docs/multitenant/usecase02/pdb_unplug.yaml @@ -0,0 +1,31 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbName: "DB12" + pdbName: "pdbdev" + xmlFileName: "/tmp/pdbunplug.xml" + action: "Unplug" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + diff --git a/docs/multitenant/usecase02/pdb_unplugtde.yaml b/docs/multitenant/usecase02/pdb_unplugtde.yaml new file mode 100644 index 00000000..4c26bffe --- /dev/null +++ b/docs/multitenant/usecase02/pdb_unplugtde.yaml @@ -0,0 +1,54 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +apiVersion: database.oracle.com/v1alpha1 +Kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbName: "DB12" + pdbName: "pdbdev" + adminName: + secret: + secretName: pdb1-secret + key: "sysadmin_user" + adminPwd: + secret: + secretName: pdb1-secret + key: "sysadmin_pwd" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + tdePassword: + secret: + secretName: "tde1-secret" + key: "tdepassword" + tdeSecret: + secret: + secretName: "tde1-secret" + key: "tdesecret" + totalSize: 1G + tempSize: 1G + unlimitedStorage: true + reuseTempFile: true + fileNameConversions: NONE + action: "Unplug" + xmlFileName: "/home/oracle/unplugpdb.xml" + tdeExport: true + tdeKeystorePath: "/home/oracle/keystore" + diff --git a/docs/multitenant/usecase02/tde_secret.yaml b/docs/multitenant/usecase02/tde_secret.yaml new file mode 100644 index 00000000..5975e27e --- /dev/null +++ b/docs/multitenant/usecase02/tde_secret.yaml @@ -0,0 +1,15 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +apiVersion: v1 +kind: Secret +metadata: + name: tde1-secret + namespace: oracle-database-operator-system +type: Opaque +data: + tdepassword: "Y2hhbmdlbWUK" + tdesecret: "Y2hhbmdlbWUK" + From d0ad6d4bb87cc20611a562e76c8e40f5439a2475 Mon Sep 17 00:00:00 2001 From: matteo malvezzi Date: Wed, 11 Jan 2023 12:44:15 +0100 Subject: [PATCH 529/628] Correct clone/create defect --- controllers/database/pdb_controller.go | 28 ++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/controllers/database/pdb_controller.go b/controllers/database/pdb_controller.go index 91d7c440..da4405a8 100644 --- a/controllers/database/pdb_controller.go +++ b/controllers/database/pdb_controller.go @@ -618,6 +618,18 @@ func (r *PDBReconciler) createPDB(ctx context.Context, req ctrl.Request, pdb *db return err } + /* Prevent creating an existing pdb */ + err = r.getPDBState(ctx, req, pdb) + if err != nil { + if apierrors.IsNotFound(err) { + log.Info("Check PDB not existence completed", "PDB Name", pdb.Spec.PDBName) + } + + } else { + log.Info("Database already exists ", "PDB Name", pdb.Spec.PDBName) + return nil + } + values := map[string]string{ "method": "CREATE", "pdb_name": pdb.Spec.PDBName, @@ -679,6 +691,10 @@ func (r *PDBReconciler) createPDB(ctx context.Context, req ctrl.Request, pdb *db /************************************************/ func (r *PDBReconciler) clonePDB(ctx context.Context, req ctrl.Request, pdb *dbapi.PDB) error { + if pdb.Spec.PDBName == pdb.Spec.SrcPDBName { + return nil + } + log := r.Log.WithValues("clonePDB", req.NamespacedName) var err error @@ -688,6 +704,18 @@ func (r *PDBReconciler) clonePDB(ctx context.Context, req ctrl.Request, pdb *dba return err } + /* Prevent cloning an existing pdb */ + err = r.getPDBState(ctx, req, pdb) + if err != nil { + if apierrors.IsNotFound(err) { + log.Info("Check PDB not existence completed", "PDB Name", pdb.Spec.PDBName) + } + + } else { + log.Info("Database already exists ", "PDB Name", pdb.Spec.PDBName) + return nil + } + values := map[string]string{ "method": "CLONE", "clonePDBName": pdb.Spec.PDBName, From dd86d12b6395069dbad4ce23de7caa0592bcfe25 Mon Sep 17 00:00:00 2001 From: Abhishek Kumar Date: Tue, 7 Feb 2023 13:54:33 +0000 Subject: [PATCH 530/628] Unification of standby controller in singleinstancedatabase controller --- PROJECT | 13 + .../v1alpha1/dataguardbroker_types.go | 127 ++ .../v1alpha1/dataguardbroker_webhook.go | 139 +++ .../v1alpha1/oraclerestdataservice_types.go | 2 +- .../v1alpha1/oraclerestdataservice_webhook.go | 2 +- .../v1alpha1/singleinstancedatabase_types.go | 12 +- .../singleinstancedatabase_webhook.go | 7 +- .../v1alpha1/zz_generated.deepcopy.go | 153 +++ commons/database/constants.go | 95 +- commons/database/utils.go | 7 + .../crd/bases/database.oracle.com_cdbs.yaml | 6 + .../database.oracle.com_dataguardbrokers.yaml | 144 +++ ...se.oracle.com_singleinstancedatabases.yaml | 26 +- config/crd/kustomization.yaml | 3 + .../cainjection_in_dataguardbrokers.yaml | 8 + .../patches/webhook_in_dataguardbrokers.yaml | 17 + config/rbac/dataguardbroker_editor_role.yaml | 24 + config/rbac/dataguardbroker_viewer_role.yaml | 20 + config/rbac/role.yaml | 26 + .../database_v1alpha1_dataguardbroker.yaml | 6 + config/samples/kustomization.yaml | 1 + config/samples/sidb/dataguardbroker.yaml | 43 + config/samples/sidb/openshift_rbac.yaml | 2 +- .../samples/sidb/oraclerestdataservice.yaml | 2 +- .../sidb/oraclerestdataservice_apex.yaml | 2 +- .../sidb/oraclerestdataservice_create.yaml | 2 +- .../sidb/oraclerestdataservice_secrets.yaml | 2 +- .../samples/sidb/singleinstancedatabase.yaml | 10 +- .../sidb/singleinstancedatabase_clone.yaml | 2 +- .../sidb/singleinstancedatabase_create.yaml | 2 +- .../sidb/singleinstancedatabase_express.yaml | 2 +- .../sidb/singleinstancedatabase_patch.yaml | 2 +- .../singleinstancedatabase_prebuiltdb.yaml | 2 +- .../sidb/singleinstancedatabase_secrets.yaml | 2 +- .../sidb/singleinstancedatabase_standby.yaml | 47 + .../sidb/singleinstancedatabase_tcps.yaml | 2 +- config/webhook/manifests.yaml | 42 + .../database/dataguardbroker_controller.go | 1088 +++++++++++++++++ .../oraclerestdataservice_controller.go | 130 +- .../singleinstancedatabase_controller.go | 863 ++++++++++--- docs/sidb/README.md | 194 +++ main.go | 14 + 42 files changed, 3021 insertions(+), 272 deletions(-) create mode 100644 apis/database/v1alpha1/dataguardbroker_types.go create mode 100644 apis/database/v1alpha1/dataguardbroker_webhook.go create mode 100644 config/crd/bases/database.oracle.com_dataguardbrokers.yaml create mode 100644 config/crd/patches/cainjection_in_dataguardbrokers.yaml create mode 100644 config/crd/patches/webhook_in_dataguardbrokers.yaml create mode 100644 config/rbac/dataguardbroker_editor_role.yaml create mode 100644 config/rbac/dataguardbroker_viewer_role.yaml create mode 100644 config/samples/database_v1alpha1_dataguardbroker.yaml create mode 100644 config/samples/sidb/dataguardbroker.yaml create mode 100644 config/samples/sidb/singleinstancedatabase_standby.yaml create mode 100644 controllers/database/dataguardbroker_controller.go diff --git a/PROJECT b/PROJECT index 67db6526..c402ecfd 100644 --- a/PROJECT +++ b/PROJECT @@ -123,4 +123,17 @@ resources: kind: DbcsSystem path: github.com/oracle/oracle-database-operator/apis/database/v1alpha1 version: v1alpha1 +- api: + crdVersion: v1beta1 + namespaced: true + controller: true + domain: oracle.com + group: database + kind: DataguardBroker + path: github.com/oracle/oracle-database-operator/apis/database/v1alpha1 + version: v1alpha1 + webhooks: + defaulting: true + validation: true + webhookVersion: v1beta1 version: "3" diff --git a/apis/database/v1alpha1/dataguardbroker_types.go b/apis/database/v1alpha1/dataguardbroker_types.go new file mode 100644 index 00000000..12c69998 --- /dev/null +++ b/apis/database/v1alpha1/dataguardbroker_types.go @@ -0,0 +1,127 @@ +/* +** Copyright (c) 2023 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v1alpha1 + +import ( + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! +// NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized. + +// DataguardBrokerSpec defines the desired state of DataguardBroker +type DataguardBrokerSpec struct { + // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster + // Important: Run "make" to regenerate code after modifying this file + + PrimaryDatabaseRef string `json:"primaryDatabaseRef"` + StandbyDatabaseRefs []string `json:"standbyDatabaseRefs"` + SetAsPrimaryDatabase string `json:"setAsPrimaryDatabase,omitempty"` + LoadBalancer bool `json:"loadBalancer,omitempty"` + + // +kubebuilder:validation:Enum=MaxPerformance;MaxAvailability + ProtectionMode string `json:"protectionMode"` + NodeSelector map[string]string `json:"nodeSelector,omitempty"` + FastStartFailOver DataguardBrokerFastStartFailOver `json:"fastStartFailOver,omitempty"` + AdminPassword DataguardBrokerPassword `json:"adminPassword"` +} + +// DataguardBrokerPassword defines the secret containing Password mapped to secretKey +type DataguardBrokerPassword struct { + SecretName string `json:"secretName,omitempty"` + SecretKey string `json:"secretKey"` + KeepSecret bool `json:"keepSecret"` +} + +type DataguardBrokerFastStartFailOver struct { + Enable bool `json:"enable,omitempty"` + Strategy []DataguardBrokerStrategy `json:"strategy,omitempty"` +} + +// FSFO strategy +type DataguardBrokerStrategy struct { + SourceDatabaseRef string `json:"sourceDatabaseRef,omitempty"` + TargetDatabaseRefs string `json:"targetDatabaseRefs,omitempty"` +} + +// DataguardBrokerStatus defines the observed state of DataguardBroker +type DataguardBrokerStatus struct { + // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster + // Important: Run "make" to regenerate code after modifying this file + + PrimaryDatabaseRef string `json:"primaryDatabaseRef,omitempty"` + ProtectionMode string `json:"protectionMode,omitempty"` + PrimaryDatabase string `json:"primaryDatabase,omitempty"` + StandbyDatabases string `json:"standbyDatabases,omitempty"` + ExternalConnectString string `json:"externalConnectString,omitempty"` + ClusterConnectString string `json:"clusterConnectString,omitempty"` + Status string `json:"status,omitempty"` +} + +//+kubebuilder:object:root=true +//+kubebuilder:subresource:status +// +kubebuilder:printcolumn:JSONPath=".status.primaryDatabase",name="Primary",type="string" +// +kubebuilder:printcolumn:JSONPath=".status.standbyDatabases",name="Standbys",type="string" +// +kubebuilder:printcolumn:JSONPath=".spec.protectionMode",name="Protection Mode",type="string" +// +kubebuilder:printcolumn:JSONPath=".status.clusterConnectString",name="Cluster Connect Str",type="string",priority=1 +// +kubebuilder:printcolumn:JSONPath=".status.externalConnectString",name="Connect Str",type="string" +// +kubebuilder:printcolumn:JSONPath=".spec.primaryDatabaseRef",name="Primary Database",type="string" +// +kubebuilder:printcolumn:JSONPath=".status.status",name="Status",type="string" + +// DataguardBroker is the Schema for the dataguardbrokers API +type DataguardBroker struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + Spec DataguardBrokerSpec `json:"spec,omitempty"` + Status DataguardBrokerStatus `json:"status,omitempty"` +} + +//+kubebuilder:object:root=true + +// DataguardBrokerList contains a list of DataguardBroker +type DataguardBrokerList struct { + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata,omitempty"` + Items []DataguardBroker `json:"items"` +} + +func init() { + SchemeBuilder.Register(&DataguardBroker{}, &DataguardBrokerList{}) +} diff --git a/apis/database/v1alpha1/dataguardbroker_webhook.go b/apis/database/v1alpha1/dataguardbroker_webhook.go new file mode 100644 index 00000000..0c838600 --- /dev/null +++ b/apis/database/v1alpha1/dataguardbroker_webhook.go @@ -0,0 +1,139 @@ +/* +** Copyright (c) 2023 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v1alpha1 + +import ( + "strings" + + apierrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/runtime/schema" + "k8s.io/apimachinery/pkg/util/validation/field" + ctrl "sigs.k8s.io/controller-runtime" + logf "sigs.k8s.io/controller-runtime/pkg/log" + "sigs.k8s.io/controller-runtime/pkg/webhook" +) + +// log is for logging in this package. +var dataguardbrokerlog = logf.Log.WithName("dataguardbroker-resource") + +func (r *DataguardBroker) SetupWebhookWithManager(mgr ctrl.Manager) error { + return ctrl.NewWebhookManagedBy(mgr). + For(r). + Complete() +} + +// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! + +//+kubebuilder:webhook:path=/mutate-database-oracle-com-v1alpha1-dataguardbroker,mutating=true,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=dataguardbrokers,verbs=create;update,versions=v1alpha1,name=mdataguardbroker.kb.io,admissionReviewVersions={v1,v1beta1} + +var _ webhook.Defaulter = &DataguardBroker{} + +// Default implements webhook.Defaulter so a webhook will be registered for the type +func (r *DataguardBroker) Default() { + dataguardbrokerlog.Info("default", "name", r.Name) + + // TODO(user): fill in your defaulting logic. +} + +// TODO(user): change verbs to "verbs=create;update;delete" if you want to enable deletion validation. +//+kubebuilder:webhook:verbs=create;update,path=/validate-database-oracle-com-v1alpha1-dataguardbroker,mutating=false,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=dataguardbrokers,versions=v1alpha1,name=vdataguardbroker.kb.io,admissionReviewVersions={v1,v1beta1} + +var _ webhook.Validator = &DataguardBroker{} + +// ValidateCreate implements webhook.Validator so a webhook will be registered for the type +func (r *DataguardBroker) ValidateCreate() error { + dataguardbrokerlog.Info("validate create", "name", r.Name) + + // TODO(user): fill in your validation logic upon object creation. + return nil +} + +// ValidateUpdate implements webhook.Validator so a webhook will be registered for the type +func (r *DataguardBroker) ValidateUpdate(old runtime.Object) error { + dataguardbrokerlog.Info("validate update", "name", r.Name) + + dataguardbrokerlog.Info("validate update", "name", r.Name) + var allErrs field.ErrorList + + // check creation validations first + err := r.ValidateCreate() + if err != nil { + return err + } + + // Validate Deletion + if r.GetDeletionTimestamp() != nil { + err := r.ValidateDelete() + if err != nil { + return err + } + } + + // Now check for updation errors + oldObj, ok := old.(*DataguardBroker) + if !ok { + return nil + } + + if oldObj.Status.ProtectionMode != "" && !strings.EqualFold(r.Spec.ProtectionMode, oldObj.Status.ProtectionMode) { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("protectionMode"), "cannot be changed")) + } + if oldObj.Status.PrimaryDatabaseRef != "" && !strings.EqualFold(oldObj.Status.PrimaryDatabaseRef, r.Spec.PrimaryDatabaseRef) { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("primaryDatabaseRef"), "cannot be changed")) + } + + if len(allErrs) == 0 { + return nil + } + return apierrors.NewInvalid( + schema.GroupKind{Group: "database.oracle.com", Kind: "DataguardBroker"}, + r.Name, allErrs) + +} + +// ValidateDelete implements webhook.Validator so a webhook will be registered for the type +func (r *DataguardBroker) ValidateDelete() error { + dataguardbrokerlog.Info("validate delete", "name", r.Name) + + // TODO(user): fill in your validation logic upon object deletion. + return nil +} diff --git a/apis/database/v1alpha1/oraclerestdataservice_types.go b/apis/database/v1alpha1/oraclerestdataservice_types.go index dad990ee..37f61a8a 100644 --- a/apis/database/v1alpha1/oraclerestdataservice_types.go +++ b/apis/database/v1alpha1/oraclerestdataservice_types.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2022 Oracle and/or its affiliates. +** Copyright (c) 2023 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/apis/database/v1alpha1/oraclerestdataservice_webhook.go b/apis/database/v1alpha1/oraclerestdataservice_webhook.go index 9e63eee2..285a4978 100644 --- a/apis/database/v1alpha1/oraclerestdataservice_webhook.go +++ b/apis/database/v1alpha1/oraclerestdataservice_webhook.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2022 Oracle and/or its affiliates. +** Copyright (c) 2023 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** diff --git a/apis/database/v1alpha1/singleinstancedatabase_types.go b/apis/database/v1alpha1/singleinstancedatabase_types.go index cfd22425..4c87b7b8 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_types.go +++ b/apis/database/v1alpha1/singleinstancedatabase_types.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2022 Oracle and/or its affiliates. +** Copyright (c) 2023 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** @@ -69,8 +69,11 @@ type SingleInstanceDatabaseSpec struct { ForceLogging bool `json:"forceLog,omitempty"` EnableTCPS bool `json:"enableTCPS,omitempty"` TcpsCertRenewInterval string `json:"tcpsCertRenewInterval,omitempty"` + DgBrokerConfigured bool `json:"dgBrokerConfigured,omitempty"` CloneFrom string `json:"cloneFrom,omitempty"` + PrimaryDatabaseRef string `json:"primaryDatabaseRef,omitempty"` + CreateAsStandby bool `json:"createAsStandby,omitempty"` ReadinessCheckPeriod int `json:"readinessCheckPeriod,omitempty"` ServiceAccountName string `json:"serviceAccountName,omitempty"` @@ -157,6 +160,8 @@ type SingleInstanceDatabaseStatus struct { CertCreationTimestamp string `json:"certCreationTimestamp,omitempty"` CertRenewInterval string `json:"certRenewInterval,omitempty"` ClientWalletLoc string `json:"clientWalletLoc,omitempty"` + PrimaryDatabase string `json:"primaryDatabase,omitempty"` + DgBrokerConfigured bool `json:"dgBrokerConfigured,omitempty"` // +patchMergeKey=type // +patchStrategy=merge @@ -172,12 +177,13 @@ type SingleInstanceDatabaseStatus struct { //+kubebuilder:subresource:status // +kubebuilder:subresource:scale:specpath=.spec.replicas,statuspath=.status.replicas // +kubebuilder:printcolumn:JSONPath=".status.edition",name="Edition",type="string" +// +kubebuilder:printcolumn:JSONPath=".status.sid",name="Sid",type="string,priority=1" // +kubebuilder:printcolumn:JSONPath=".status.status",name="Status",type="string" -// +kubebuilder:printcolumn:JSONPath=".status.role",name="Role",type="string",priority=1 +// +kubebuilder:printcolumn:JSONPath=".status.role",name="Role",type="string" // +kubebuilder:printcolumn:JSONPath=".status.releaseUpdate",name="Version",type="string" // +kubebuilder:printcolumn:JSONPath=".status.connectString",name="Connect Str",type="string" -// +kubebuilder:printcolumn:JSONPath=".status.tcpsConnectString",name="TCPS Connect Str",type="string" // +kubebuilder:printcolumn:JSONPath=".status.pdbConnectString",name="Pdb Connect Str",type="string",priority=1 +// +kubebuilder:printcolumn:JSONPath=".status.tcpsConnectString",name="TCPS Connect Str",type="string" // +kubebuilder:printcolumn:JSONPath=".status.tcpsPdbConnectString",name="TCPS Pdb Connect Str",type="string", priority=1 // +kubebuilder:printcolumn:JSONPath=".status.oemExpressUrl",name="Oem Express Url",type="string" diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index 6323550b..1dc343bd 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2022 Oracle and/or its affiliates. +** Copyright (c) 2023 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** @@ -159,6 +159,11 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { field.Invalid(field.NewPath("spec").Child("cloneFrom"), r.Spec.CloneFrom, "Cloning not supported for Express edition")) } + if r.Spec.CreateAsStandby { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("createAsStandby"), r.Spec.CreateAsStandby, + "Physical Standby Database creation is not supported for Express edition")) + } if strings.ToUpper(r.Spec.Sid) != "XE" { allErrs = append(allErrs, field.Invalid(field.NewPath("spec").Child("sid"), r.Spec.Sid, diff --git a/apis/database/v1alpha1/zz_generated.deepcopy.go b/apis/database/v1alpha1/zz_generated.deepcopy.go index 72d93dc2..0585bc7b 100644 --- a/apis/database/v1alpha1/zz_generated.deepcopy.go +++ b/apis/database/v1alpha1/zz_generated.deepcopy.go @@ -875,6 +875,159 @@ func (in *ConnectionStringSpec) DeepCopy() *ConnectionStringSpec { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DataguardBroker) DeepCopyInto(out *DataguardBroker) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + in.Spec.DeepCopyInto(&out.Spec) + out.Status = in.Status +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DataguardBroker. +func (in *DataguardBroker) DeepCopy() *DataguardBroker { + if in == nil { + return nil + } + out := new(DataguardBroker) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *DataguardBroker) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DataguardBrokerFastStartFailOver) DeepCopyInto(out *DataguardBrokerFastStartFailOver) { + *out = *in + if in.Strategy != nil { + in, out := &in.Strategy, &out.Strategy + *out = make([]DataguardBrokerStrategy, len(*in)) + copy(*out, *in) + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DataguardBrokerFastStartFailOver. +func (in *DataguardBrokerFastStartFailOver) DeepCopy() *DataguardBrokerFastStartFailOver { + if in == nil { + return nil + } + out := new(DataguardBrokerFastStartFailOver) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DataguardBrokerList) DeepCopyInto(out *DataguardBrokerList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]DataguardBroker, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DataguardBrokerList. +func (in *DataguardBrokerList) DeepCopy() *DataguardBrokerList { + if in == nil { + return nil + } + out := new(DataguardBrokerList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *DataguardBrokerList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DataguardBrokerPassword) DeepCopyInto(out *DataguardBrokerPassword) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DataguardBrokerPassword. +func (in *DataguardBrokerPassword) DeepCopy() *DataguardBrokerPassword { + if in == nil { + return nil + } + out := new(DataguardBrokerPassword) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DataguardBrokerSpec) DeepCopyInto(out *DataguardBrokerSpec) { + *out = *in + if in.StandbyDatabaseRefs != nil { + in, out := &in.StandbyDatabaseRefs, &out.StandbyDatabaseRefs + *out = make([]string, len(*in)) + copy(*out, *in) + } + if in.NodeSelector != nil { + in, out := &in.NodeSelector, &out.NodeSelector + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + in.FastStartFailOver.DeepCopyInto(&out.FastStartFailOver) + out.AdminPassword = in.AdminPassword +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DataguardBrokerSpec. +func (in *DataguardBrokerSpec) DeepCopy() *DataguardBrokerSpec { + if in == nil { + return nil + } + out := new(DataguardBrokerSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DataguardBrokerStatus) DeepCopyInto(out *DataguardBrokerStatus) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DataguardBrokerStatus. +func (in *DataguardBrokerStatus) DeepCopy() *DataguardBrokerStatus { + if in == nil { + return nil + } + out := new(DataguardBrokerStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DataguardBrokerStrategy) DeepCopyInto(out *DataguardBrokerStrategy) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DataguardBrokerStrategy. +func (in *DataguardBrokerStrategy) DeepCopy() *DataguardBrokerStrategy { + if in == nil { + return nil + } + out := new(DataguardBrokerStrategy) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *DbStatus) DeepCopyInto(out *DbStatus) { *out = *in diff --git a/commons/database/constants.go b/commons/database/constants.go index 76750ffb..b48af253 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -97,70 +97,83 @@ const ArchiveLogFalseCMD string = CreateChkFileCMD + " && " + const StandbyDatabasePrerequisitesSQL string = "ALTER SYSTEM SET db_create_file_dest='/opt/oracle/oradata/';" + "\nALTER SYSTEM SET db_create_online_log_dest_1='/opt/oracle/oradata/';" + "\nALTER SYSTEM SWITCH LOGFILE;" + - "\nALTER DATABASE ADD STANDBY LOGFILE THREAD 1 GROUP 10 SIZE 200M;" + - "\nALTER DATABASE ADD STANDBY LOGFILE THREAD 1 GROUP 11 SIZE 200M;" + - "\nALTER DATABASE ADD STANDBY LOGFILE THREAD 1 GROUP 12 SIZE 200M;" + - "\nALTER DATABASE ADD STANDBY LOGFILE THREAD 1 GROUP 13 SIZE 200M;" + + "\nALTER DATABASE ADD STANDBY LOGFILE THREAD 1 SIZE 200M;" + + "\nALTER DATABASE ADD STANDBY LOGFILE THREAD 1 SIZE 200M;" + + "\nALTER DATABASE ADD STANDBY LOGFILE THREAD 1 SIZE 200M;" + + "\nALTER DATABASE ADD STANDBY LOGFILE THREAD 1 SIZE 200M;" + "\nALTER SYSTEM SET STANDBY_FILE_MANAGEMENT=AUTO;" + "\nALTER SYSTEM SET dg_broker_start=TRUE;" const StandbyTnsnamesEntry string = ` ##STANDBYDATABASE_SID## = (DESCRIPTION = -(ADDRESS = (PROTOCOL = TCP)(HOST = ##STANDBYDATABASE_SERVICE_EXPOSED## )(PORT = 1521)) -(CONNECT_DATA = -(SERVER = DEDICATED) -(SERVICE_NAME = ##STANDBYDATABASE_SID##) -) + (ADDRESS = (PROTOCOL = TCP)(HOST = ##STANDBYDATABASE_SERVICE_EXPOSED## )(PORT = 1521)) + (CONNECT_DATA = + (SERVER = DEDICATED) + (SERVICE_NAME = ##STANDBYDATABASE_SID##) + ) ) ` const PDBTnsnamesEntry string = ` ##PDB_NAME## = (DESCRIPTION = -(ADDRESS = (PROTOCOL = TCP)(HOST = 0.0.0.0 )(PORT = 1521)) -(CONNECT_DATA = -(SERVER = DEDICATED) -(SERVICE_NAME = ##PDB_NAME##) -) + (ADDRESS = (PROTOCOL = TCP)(HOST = 0.0.0.0 )(PORT = 1521)) + (CONNECT_DATA = + (SERVER = DEDICATED) + (SERVICE_NAME = ##PDB_NAME##) + ) ) ` const PrimaryTnsnamesEntry string = ` ${PRIMARY_SID} = -(DESCRIPTION = -(ADDRESS = (PROTOCOL = TCP)(HOST = ${PRIMARY_IP})(PORT = 1521 )) -(CONNECT_DATA = - (SERVER = DEDICATED) - (SERVICE_NAME = ${PRIMARY_SID}) -) -) -` + (DESCRIPTION = + (ADDRESS = (PROTOCOL = TCP)(HOST = ${PRIMARY_IP})(PORT = 1521 )) + (CONNECT_DATA = + (SERVER = DEDICATED) + (SERVICE_NAME = ${PRIMARY_SID}) + ) + ) + ` const ListenerEntry string = `LISTENER = (DESCRIPTION_LIST = -(DESCRIPTION = -(ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC1)) -(ADDRESS = (PROTOCOL = TCP)(HOST = 0.0.0.0)(PORT = 1521)) -) + (DESCRIPTION = + (ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC1)) + (ADDRESS = (PROTOCOL = TCP)(HOST = 0.0.0.0)(PORT = 1521)) + ) ) SID_LIST_LISTENER = -(SID_LIST = -(SID_DESC = - (GLOBAL_DBNAME = ${ORACLE_SID^^}) - (SID_NAME = ${ORACLE_SID^^}) - (ORACLE_HOME = ${ORACLE_HOME}) -) -(SID_DESC = - (GLOBAL_DBNAME = ${ORACLE_SID^^}_DGMGRL) - (SID_NAME = ${ORACLE_SID^^}) - (ORACLE_HOME = ${ORACLE_HOME}) - (ENVS="TNS_ADMIN=/opt/oracle/oradata/dbconfig/${ORACLE_SID^^}") -) -) + (SID_LIST = + (SID_DESC = + (GLOBAL_DBNAME = ${ORACLE_SID^^}) + (SID_NAME = ${ORACLE_SID^^}) + (ORACLE_HOME = ${ORACLE_HOME}) + ) + (SID_DESC = + (GLOBAL_DBNAME = DATAGUARD) + (SID_NAME = ${ORACLE_SID^^}) + (ORACLE_HOME = ${ORACLE_HOME}) + ) + (SID_DESC = + (GLOBAL_DBNAME = ${ORACLE_SID^^}_DGMGRL) + (SID_NAME = ${ORACLE_SID^^}) + (ORACLE_HOME = ${ORACLE_HOME}) + (ENVS="TNS_ADMIN=/opt/oracle/oradata/dbconfig/${ORACLE_SID^^}") + ) + ) DEDICATED_THROUGH_BROKER_LISTENER=ON ` +const CreateAdminPasswordFile string = "umask 177\n cat > admin.pwd < dgmgrl.cmd\n umask 022" + +const RemoveAdminPasswordFile string = "rm -rf admin.pwd" + +const RemoveDGMGRLScriptFile string = "rm -rf dgmgrl.cmd" + const DataguardBrokerMaxPerformanceCMD string = "CREATE CONFIGURATION dg_config AS PRIMARY DATABASE IS ${PRIMARY_SID} CONNECT IDENTIFIER IS ${PRIMARY_DB_CONN_STR};" + "\nADD DATABASE ${ORACLE_SID} AS CONNECT IDENTIFIER IS ${SVC_HOST}:1521/${ORACLE_SID} MAINTAINED AS PHYSICAL;" + "\nEDIT DATABASE ${PRIMARY_SID} SET PROPERTY LogXptMode='ASYNC';" + @@ -201,6 +214,10 @@ const DataguardBrokerGetDatabaseCMD string = "SELECT DATABASE || ':' || DATAGUAR const EnableFSFOCMD string = "ENABLE FAST_START FAILOVER;" +const RemoveDataguardConfiguration string = "DISABLE FAST_START FAILOVER;" + + "\nEDIT CONFIGURATION SET PROTECTION MODE AS MAXPERFORMANCE;" + + "\nREMOVE CONFIGURATION;" + const GetDatabaseRoleCMD string = "SELECT DATABASE_ROLE FROM V\\$DATABASE; " const RunDatapatchCMD string = " ( while true; do sleep 60; echo \"Installing patches...\" ; done ) & if ! $ORACLE_HOME/OPatch/datapatch -skip_upgrade_check;" + diff --git a/commons/database/utils.go b/commons/database/utils.go index 91fdfb09..4d42350d 100644 --- a/commons/database/utils.go +++ b/commons/database/utils.go @@ -675,6 +675,13 @@ func ApexPasswordValidator(pwd string) bool { return hasMinLen && hasUpper && hasLower && hasNumber && hasSpecial } +func GetSqlClient(edition string) string { + if edition == "express" { + return "su -p oracle -c \"sqlplus -s / as sysdba\"" + } + return "sqlplus -s / as sysdba" +} + // Function for patching the K8s service with the payload. // Patch strategy used: Strategic Merge Patch func PatchService(config *rest.Config, namespace string, ctx context.Context, req ctrl.Request, svcName string, payload string) error { diff --git a/config/crd/bases/database.oracle.com_cdbs.yaml b/config/crd/bases/database.oracle.com_cdbs.yaml index ab2ab2ef..6b1c350c 100644 --- a/config/crd/bases/database.oracle.com_cdbs.yaml +++ b/config/crd/bases/database.oracle.com_cdbs.yaml @@ -29,6 +29,10 @@ spec: jsonPath: .spec.dbPort name: DB Port type: integer + - description: ' string of the tnsalias' + jsonPath: .spec.dbTnsurl + name: TNS STRING + type: string - description: Replicas jsonPath: .spec.replicas name: Replicas @@ -137,6 +141,8 @@ spec: dbServer: description: Name of the DB server type: string + dbTnsurl: + type: string nodeSelector: additionalProperties: type: string diff --git a/config/crd/bases/database.oracle.com_dataguardbrokers.yaml b/config/crd/bases/database.oracle.com_dataguardbrokers.yaml new file mode 100644 index 00000000..98e7fc1a --- /dev/null +++ b/config/crd/bases/database.oracle.com_dataguardbrokers.yaml @@ -0,0 +1,144 @@ + +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.6.1 + creationTimestamp: null + name: dataguardbrokers.database.oracle.com +spec: + group: database.oracle.com + names: + kind: DataguardBroker + listKind: DataguardBrokerList + plural: dataguardbrokers + singular: dataguardbroker + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .status.primaryDatabase + name: Primary + type: string + - jsonPath: .status.standbyDatabases + name: Standbys + type: string + - jsonPath: .spec.protectionMode + name: Protection Mode + type: string + - jsonPath: .status.clusterConnectString + name: Cluster Connect Str + priority: 1 + type: string + - jsonPath: .status.externalConnectString + name: Connect Str + type: string + - jsonPath: .spec.primaryDatabaseRef + name: Primary Database + type: string + - jsonPath: .status.status + name: Status + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + description: DataguardBroker is the Schema for the dataguardbrokers API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: DataguardBrokerSpec defines the desired state of DataguardBroker + properties: + adminPassword: + description: DataguardBrokerPassword defines the secret containing + Password mapped to secretKey + properties: + keepSecret: + type: boolean + secretKey: + type: string + secretName: + type: string + required: + - keepSecret + - secretKey + type: object + fastStartFailOver: + properties: + enable: + type: boolean + strategy: + items: + description: FSFO strategy + properties: + sourceDatabaseRef: + type: string + targetDatabaseRefs: + type: string + type: object + type: array + type: object + loadBalancer: + type: boolean + nodeSelector: + additionalProperties: + type: string + type: object + primaryDatabaseRef: + type: string + protectionMode: + enum: + - MaxPerformance + - MaxAvailability + type: string + setAsPrimaryDatabase: + type: string + standbyDatabaseRefs: + items: + type: string + type: array + required: + - adminPassword + - primaryDatabaseRef + - protectionMode + - standbyDatabaseRefs + type: object + status: + description: DataguardBrokerStatus defines the observed state of DataguardBroker + properties: + clusterConnectString: + type: string + externalConnectString: + type: string + primaryDatabase: + type: string + primaryDatabaseRef: + type: string + protectionMode: + type: string + standbyDatabases: + type: string + status: + type: string + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] diff --git a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml index 493800d6..2a24a483 100644 --- a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml +++ b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml @@ -25,7 +25,6 @@ spec: type: string - jsonPath: .status.role name: Role - priority: 1 type: string - jsonPath: .status.releaseUpdate name: Version @@ -88,6 +87,10 @@ spec: type: string cloneFrom: type: string + createAsStandby: + type: boolean + dgBrokerConfigured: + type: boolean edition: enum: - standard @@ -155,6 +158,8 @@ spec: volumeName: type: string type: object + primaryDatabaseRef: + type: string readinessCheckPeriod: type: integer replicas: @@ -203,13 +208,14 @@ spec: description: "Condition contains details for one aspect of the current state of this API Resource. --- This struct is intended for direct use as an array at the field path .status.conditions. For example, - type FooStatus struct{ // Represents the observations of a - foo's current state. // Known .status.conditions.type are: - \"Available\", \"Progressing\", and \"Degraded\" // +patchMergeKey=type - \ // +patchStrategy=merge // +listType=map // +listMapKey=type - \ Conditions []metav1.Condition `json:\"conditions,omitempty\" - patchStrategy:\"merge\" patchMergeKey:\"type\" protobuf:\"bytes,1,rep,name=conditions\"` - \n // other fields }" + \n \ttype FooStatus struct{ \t // Represents the observations + of a foo's current state. \t // Known .status.conditions.type + are: \"Available\", \"Progressing\", and \"Degraded\" \t // + +patchMergeKey=type \t // +patchStrategy=merge \t // +listType=map + \t // +listMapKey=type \t Conditions []metav1.Condition + `json:\"conditions,omitempty\" patchStrategy:\"merge\" patchMergeKey:\"type\" + protobuf:\"bytes,1,rep,name=conditions\"` \n \t // other fields + \t}" properties: lastTransitionTime: description: lastTransitionTime is the last time the condition @@ -278,6 +284,8 @@ spec: datafilesPatched: default: "false" type: string + dgBrokerConfigured: + type: boolean edition: type: string flashBack: @@ -335,6 +343,8 @@ spec: type: object prebuiltDB: type: boolean + primaryDatabase: + type: string releaseUpdate: type: string replicas: diff --git a/config/crd/kustomization.yaml b/config/crd/kustomization.yaml index 1cee363a..c411e3ed 100644 --- a/config/crd/kustomization.yaml +++ b/config/crd/kustomization.yaml @@ -16,6 +16,7 @@ resources: - bases/database.oracle.com_oraclerestdataservices.yaml - bases/database.oracle.com_autonomouscontainerdatabases.yaml - bases/database.oracle.com_dbcssystems.yaml +- bases/database.oracle.com_dataguardbrokers.yaml # +kubebuilder:scaffold:crdkustomizeresource patchesStrategicMerge: @@ -30,6 +31,7 @@ patchesStrategicMerge: #- patches/webhook_in_oraclerestdataservices.yaml #- patches/webhook_in_autonomouscontainerdatabases.yaml #- patches/webhook_in_dbcssystems.yaml +#- patches/webhook_in_dataguardbrokers.yaml # +kubebuilder:scaffold:crdkustomizewebhookpatch # [CERTMANAGER] To enable webhook, uncomment all the sections with [CERTMANAGER] prefix. @@ -43,6 +45,7 @@ patchesStrategicMerge: #- patches/cainjection_in_oraclerestdataservices.yaml #- patches/cainjection_in_autonomouscontainerdatabases.yaml #- patches/cainjection_in_dbcssystems.yaml +#- patches/cainjection_in_dataguardbrokers.yaml # +kubebuilder:scaffold:crdkustomizecainjectionpatch # the following config is for teaching kustomize how to do kustomization for CRDs. diff --git a/config/crd/patches/cainjection_in_dataguardbrokers.yaml b/config/crd/patches/cainjection_in_dataguardbrokers.yaml new file mode 100644 index 00000000..6409f54c --- /dev/null +++ b/config/crd/patches/cainjection_in_dataguardbrokers.yaml @@ -0,0 +1,8 @@ +# The following patch adds a directive for certmanager to inject CA into the CRD +# CRD conversion requires k8s 1.13 or later. +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + annotations: + cert-manager.io/inject-ca-from: $(CERTIFICATE_NAMESPACE)/$(CERTIFICATE_NAME) + name: dataguardbrokers.database.oracle.com diff --git a/config/crd/patches/webhook_in_dataguardbrokers.yaml b/config/crd/patches/webhook_in_dataguardbrokers.yaml new file mode 100644 index 00000000..10f62234 --- /dev/null +++ b/config/crd/patches/webhook_in_dataguardbrokers.yaml @@ -0,0 +1,17 @@ +# The following patch enables conversion webhook for CRD +# CRD conversion requires k8s 1.13 or later. +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: dataguardbrokers.database.oracle.com +spec: + conversion: + strategy: Webhook + webhookClientConfig: + # this is "\n" used as a placeholder, otherwise it will be rejected by the apiserver for being blank, + # but we're going to set it later using the cert-manager (or potentially a patch if not using cert-manager) + caBundle: Cg== + service: + namespace: system + name: webhook-service + path: /convert diff --git a/config/rbac/dataguardbroker_editor_role.yaml b/config/rbac/dataguardbroker_editor_role.yaml new file mode 100644 index 00000000..d9ad534f --- /dev/null +++ b/config/rbac/dataguardbroker_editor_role.yaml @@ -0,0 +1,24 @@ +# permissions for end users to edit dataguardbrokers. +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: dataguardbroker-editor-role +rules: +- apiGroups: + - database.oracle.com + resources: + - dataguardbrokers + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - database.oracle.com + resources: + - dataguardbrokers/status + verbs: + - get diff --git a/config/rbac/dataguardbroker_viewer_role.yaml b/config/rbac/dataguardbroker_viewer_role.yaml new file mode 100644 index 00000000..4fe41628 --- /dev/null +++ b/config/rbac/dataguardbroker_viewer_role.yaml @@ -0,0 +1,20 @@ +# permissions for end users to view dataguardbrokers. +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: dataguardbroker-viewer-role +rules: +- apiGroups: + - database.oracle.com + resources: + - dataguardbrokers + verbs: + - get + - list + - watch +- apiGroups: + - database.oracle.com + resources: + - dataguardbrokers/status + verbs: + - get diff --git a/config/rbac/role.yaml b/config/rbac/role.yaml index e6d35b96..89b630dd 100644 --- a/config/rbac/role.yaml +++ b/config/rbac/role.yaml @@ -260,6 +260,32 @@ rules: - get - patch - update +- apiGroups: + - database.oracle.com + resources: + - dataguardbrokers + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - database.oracle.com + resources: + - dataguardbrokers/finalizers + verbs: + - update +- apiGroups: + - database.oracle.com + resources: + - dataguardbrokers/status + verbs: + - get + - patch + - update - apiGroups: - database.oracle.com resources: diff --git a/config/samples/database_v1alpha1_dataguardbroker.yaml b/config/samples/database_v1alpha1_dataguardbroker.yaml new file mode 100644 index 00000000..8b46e319 --- /dev/null +++ b/config/samples/database_v1alpha1_dataguardbroker.yaml @@ -0,0 +1,6 @@ +apiVersion: database.oracle.com/v1alpha1 +kind: DataguardBroker +metadata: + name: dataguardbroker-sample +spec: + # TODO(user): Add fields here diff --git a/config/samples/kustomization.yaml b/config/samples/kustomization.yaml index fd4781bf..0ad50fcf 100644 --- a/config/samples/kustomization.yaml +++ b/config/samples/kustomization.yaml @@ -16,4 +16,5 @@ resources: - sharding/shardingdatabase.yaml - sharding/sharding_v1alpha1_provshard.yaml - dbcs/database_v1alpha1_dbcssystem.yaml +- database_v1alpha1_dataguardbroker.yaml # +kubebuilder:scaffold:manifestskustomizesamples diff --git a/config/samples/sidb/dataguardbroker.yaml b/config/samples/sidb/dataguardbroker.yaml new file mode 100644 index 00000000..1632b01f --- /dev/null +++ b/config/samples/sidb/dataguardbroker.yaml @@ -0,0 +1,43 @@ +# +# Copyright (c) 2023, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +apiVersion: database.oracle.com/v1alpha1 +kind: DataguardBroker +metadata: + name: dataguardbroker-sample + namespace: default +spec: + + ## Primary DB ref. This can be of kind SingleInstanceDatabase or CloneDB + primaryDatabaseRef: "sidb-sample" + + ## Standby DB pod CRD Metadata Name to add this DB to DG config + standbyDatabaseRefs: + # - standbyDatabase-sample1 + # - standbyDatabase-sample2 + + ## Secret containing databaseRef password mapped to secretKey. + ## This secret will be deleted after Dataguard setup unless keepSecret set to true. + adminPassword: + secretName: + secretKey: + keepSecret: false + + ## Deploy only on nodes having required labels . Format label_name : label_value . + ## Leave empty if there is no such requirement . + ## Uncomment to use + # nodeSelector: + # failure-domain.beta.kubernetes.io/zone: bVCG:PHX-AD-1 + # pool: sidb + + ## Type of service . Applicable on cloud enviroments only + ## if loadBalService : false , service type = "NodePort" . else "LoadBalancer" + loadBalancer: false + + ## Protection Mode for dg configuration . MaxAvailability or MaxPerformance + protectionMode: MaxAvailability + + ## Manual Switchover to this database to make it primary(if not already) . + setAsPrimaryDatabase: "" \ No newline at end of file diff --git a/config/samples/sidb/openshift_rbac.yaml b/config/samples/sidb/openshift_rbac.yaml index ded5ef69..1de921d4 100644 --- a/config/samples/sidb/openshift_rbac.yaml +++ b/config/samples/sidb/openshift_rbac.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2022, Oracle and/or its affiliates. +# Copyright (c) 2023, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # diff --git a/config/samples/sidb/oraclerestdataservice.yaml b/config/samples/sidb/oraclerestdataservice.yaml index f2ab93ab..cef0276f 100644 --- a/config/samples/sidb/oraclerestdataservice.yaml +++ b/config/samples/sidb/oraclerestdataservice.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Copyright (c) 2023, Oracle and/or its affiliates. All rights reserved. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: database.oracle.com/v1alpha1 diff --git a/config/samples/sidb/oraclerestdataservice_apex.yaml b/config/samples/sidb/oraclerestdataservice_apex.yaml index a619799d..6bdc9fb5 100644 --- a/config/samples/sidb/oraclerestdataservice_apex.yaml +++ b/config/samples/sidb/oraclerestdataservice_apex.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Copyright (c) 2023, Oracle and/or its affiliates. All rights reserved. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # diff --git a/config/samples/sidb/oraclerestdataservice_create.yaml b/config/samples/sidb/oraclerestdataservice_create.yaml index ed7493c1..454abf37 100644 --- a/config/samples/sidb/oraclerestdataservice_create.yaml +++ b/config/samples/sidb/oraclerestdataservice_create.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Copyright (c) 2023, Oracle and/or its affiliates. All rights reserved. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # diff --git a/config/samples/sidb/oraclerestdataservice_secrets.yaml b/config/samples/sidb/oraclerestdataservice_secrets.yaml index f5092a42..9e587960 100644 --- a/config/samples/sidb/oraclerestdataservice_secrets.yaml +++ b/config/samples/sidb/oraclerestdataservice_secrets.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Copyright (c) 2023, Oracle and/or its affiliates. All rights reserved. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index f836f6dd..407223fa 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2022, Oracle and/or its affiliates. +# Copyright (c) 2023, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: database.oracle.com/v1alpha1 @@ -17,6 +17,14 @@ spec: ## specify connect string as `:/` instead of source database ref cloneFrom: "" + ## Reference to a source primary database from which + ## Format: 1. Intra-cluster: you can give name of the primary database or the database connect string in `:/` format + ## 2. Inter-cluster: Database connect string in `:/` format + primaryDatabaseRef: "" + + ## Enable this flag for creating Physical Standby Database + createAsStandby: false + ## DB edition. N/A if cloning from a Source DB in current K8s cluster (if cloneFrom is set to a database ref) ## Valid values for edition are enterprise, standard or express edition: enterprise diff --git a/config/samples/sidb/singleinstancedatabase_clone.yaml b/config/samples/sidb/singleinstancedatabase_clone.yaml index de575050..c0e1eace 100644 --- a/config/samples/sidb/singleinstancedatabase_clone.yaml +++ b/config/samples/sidb/singleinstancedatabase_clone.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2022, Oracle and/or its affiliates. +# Copyright (c) 2023, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # diff --git a/config/samples/sidb/singleinstancedatabase_create.yaml b/config/samples/sidb/singleinstancedatabase_create.yaml index 413c1e3e..d09d9c26 100644 --- a/config/samples/sidb/singleinstancedatabase_create.yaml +++ b/config/samples/sidb/singleinstancedatabase_create.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2022, Oracle and/or its affiliates. +# Copyright (c) 2023, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # diff --git a/config/samples/sidb/singleinstancedatabase_express.yaml b/config/samples/sidb/singleinstancedatabase_express.yaml index 4c79ba6d..64f2e351 100644 --- a/config/samples/sidb/singleinstancedatabase_express.yaml +++ b/config/samples/sidb/singleinstancedatabase_express.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2022, Oracle and/or its affiliates. +# Copyright (c) 2023, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # diff --git a/config/samples/sidb/singleinstancedatabase_patch.yaml b/config/samples/sidb/singleinstancedatabase_patch.yaml index be2de1cf..9a211cdc 100644 --- a/config/samples/sidb/singleinstancedatabase_patch.yaml +++ b/config/samples/sidb/singleinstancedatabase_patch.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2022, Oracle and/or its affiliates. +# Copyright (c) 2023, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # diff --git a/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml b/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml index 092a1897..d59b6a9c 100644 --- a/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml +++ b/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2022, Oracle and/or its affiliates. +# Copyright (c) 2023, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # diff --git a/config/samples/sidb/singleinstancedatabase_secrets.yaml b/config/samples/sidb/singleinstancedatabase_secrets.yaml index 6504a6ac..19fd7670 100644 --- a/config/samples/sidb/singleinstancedatabase_secrets.yaml +++ b/config/samples/sidb/singleinstancedatabase_secrets.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2022, Oracle and/or its affiliates. +# Copyright (c) 2023, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # diff --git a/config/samples/sidb/singleinstancedatabase_standby.yaml b/config/samples/sidb/singleinstancedatabase_standby.yaml new file mode 100644 index 00000000..19309014 --- /dev/null +++ b/config/samples/sidb/singleinstancedatabase_standby.yaml @@ -0,0 +1,47 @@ +# +# Copyright (c) 2023, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +apiVersion: database.oracle.com/v1alpha1 +kind: SingleInstanceDatabase +metadata: + # Creates base sidb-sample. Use singleinstancedatabase_clone.yaml for cloning + # and singleinstancedatabase_patch.yaml for patching + name: sidb-sample + namespace: default +spec: + + ## Use only alphanumeric characters for sid + sid: ORCL1 + + ## Reference to a source primary database from which + ## Format: 1. Intra-cluster: you can give name of the primary database or the database connect string in `:/` format + ## 2. Inter-cluster: Database connect string in `:/` format + primaryDatabaseRef: "sidb-sample" + + ## Enable this flag for creating Physical Standby Database + createAsStandby: true + + ## Secret containing SIDB password mapped to secretKey + adminPassword: + secretName: db-admin-secret + + ## Database image details + image: + pullFrom: container-registry.oracle.com/database/enterprise:latest + pullSecrets: oracle-container-registry-secret + + ## size is the required minimum size of the persistent volume + ## storageClass is specified for automatic volume provisioning + ## accessMode can only accept one of ReadWriteOnce, ReadWriteMany + persistence: + size: 100Gi + ## oci-bv applies to OCI block volumes. Use "standard" storageClass for dynamic provisioning in Minikube. Update as appropriate for other cloud service providers + storageClass: "oci-bv" + accessMode: "ReadWriteOnce" + + ## Type of service . Applicable on cloud enviroments only + ## if loadBalService : false, service type = "NodePort" else "LoadBalancer" + loadBalancer: false + diff --git a/config/samples/sidb/singleinstancedatabase_tcps.yaml b/config/samples/sidb/singleinstancedatabase_tcps.yaml index 2f9e4994..d11a0d2a 100644 --- a/config/samples/sidb/singleinstancedatabase_tcps.yaml +++ b/config/samples/sidb/singleinstancedatabase_tcps.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2022, Oracle and/or its affiliates. +# Copyright (c) 2023, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # diff --git a/config/webhook/manifests.yaml b/config/webhook/manifests.yaml index 61d66981..28f50b79 100644 --- a/config/webhook/manifests.yaml +++ b/config/webhook/manifests.yaml @@ -67,6 +67,27 @@ webhooks: resources: - cdbs sideEffects: None +- admissionReviewVersions: + - v1 + - v1beta1 + clientConfig: + service: + name: webhook-service + namespace: system + path: /mutate-database-oracle-com-v1alpha1-dataguardbroker + failurePolicy: Fail + name: mdataguardbroker.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + resources: + - dataguardbrokers + sideEffects: None - admissionReviewVersions: - v1 - v1beta1 @@ -239,6 +260,27 @@ webhooks: resources: - cdbs sideEffects: None +- admissionReviewVersions: + - v1 + - v1beta1 + clientConfig: + service: + name: webhook-service + namespace: system + path: /validate-database-oracle-com-v1alpha1-dataguardbroker + failurePolicy: Fail + name: vdataguardbroker.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + resources: + - dataguardbrokers + sideEffects: None - admissionReviewVersions: - v1 - v1beta1 diff --git a/controllers/database/dataguardbroker_controller.go b/controllers/database/dataguardbroker_controller.go new file mode 100644 index 00000000..bd0d2c74 --- /dev/null +++ b/controllers/database/dataguardbroker_controller.go @@ -0,0 +1,1088 @@ +/* +** Copyright (c) 2023 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package controllers + +import ( + "context" + "errors" + "fmt" + "strings" + "time" + + "github.com/go-logr/logr" + corev1 "k8s.io/api/core/v1" + apierrors "k8s.io/apimachinery/pkg/api/errors" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/types" + "k8s.io/client-go/rest" + "k8s.io/client-go/tools/record" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/controller" + "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" + + dbapi "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + dbcommons "github.com/oracle/oracle-database-operator/commons/database" +) + +// DataguardBrokerReconciler reconciles a DataguardBroker object +type DataguardBrokerReconciler struct { + client.Client + Log logr.Logger + Scheme *runtime.Scheme + Config *rest.Config + Recorder record.EventRecorder +} + +const dataguardBrokerFinalizer = "database.oracle.com/dataguardbrokerfinalizer" + +//+kubebuilder:rbac:groups=database.oracle.com,resources=dataguardbrokers,verbs=get;list;watch;create;update;patch;delete +//+kubebuilder:rbac:groups=database.oracle.com,resources=dataguardbrokers/status,verbs=get;update;patch +//+kubebuilder:rbac:groups=database.oracle.com,resources=dataguardbrokers/finalizers,verbs=update +//+kubebuilder:rbac:groups="",resources=pods;pods/log;pods/exec;persistentvolumeclaims;services;nodes;events,verbs=create;delete;get;list;patch;update;watch + +// Reconcile is part of the main kubernetes reconciliation loop which aims to +// move the current state of the cluster closer to the desired state. +// TODO(user): Modify the Reconcile function to compare the state specified by +// the DataguardBroker object against the actual cluster state, and then +// perform operations to make the cluster state reflect the state specified by +// the user. +// +// For more details, check Reconcile and its Result here: +// - https://pkg.go.dev/sigs.k8s.io/controller-runtime@v0.8.3/pkg/reconcile +func (r *DataguardBrokerReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + + r.Log.Info("Reconcile requested") + + dataguardBroker := &dbapi.DataguardBroker{} + err := r.Get(ctx, types.NamespacedName{Namespace: req.Namespace, Name: req.Name}, dataguardBroker) + if err != nil { + if apierrors.IsNotFound(err) { + r.Log.Info("Resource deleted") + return requeueN, nil + } + return requeueN, err + } + + // Manage SingleInstanceDatabase Deletion + result, err := r.manageDataguardBrokerDeletion(req, ctx, dataguardBroker) + if result.Requeue { + r.Log.Info("Reconcile queued") + return result, nil + } + + // Fetch Primary Database Reference + singleInstanceDatabase := &dbapi.SingleInstanceDatabase{} + err = r.Get(ctx, types.NamespacedName{Namespace: req.Namespace, Name: dataguardBroker.Spec.PrimaryDatabaseRef}, singleInstanceDatabase) + if err != nil { + if apierrors.IsNotFound(err) { + r.Log.Info("Resource deleted") + return requeueN, nil + } + return requeueN, err + } + + /* Initialize Status */ + if dataguardBroker.Status.Status == "" { + dataguardBroker.Status.Status = dbcommons.StatusCreating + dataguardBroker.Status.ExternalConnectString = dbcommons.ValueUnavailable + dataguardBroker.Status.ClusterConnectString = dbcommons.ValueUnavailable + r.Status().Update(ctx, dataguardBroker) + } + + // Always refresh status before a reconcile + defer r.Status().Update(ctx, dataguardBroker) + + // Create Service to point to primary database always + result = r.createSVC(ctx, req, dataguardBroker) + if result.Requeue { + r.Log.Info("Reconcile queued") + return result, nil + } + + // Validate if Primary Database Reference is ready + result, sidbReadyPod, adminPassword := r.validateSidbReadiness(dataguardBroker, singleInstanceDatabase, ctx, req) + if result.Requeue { + r.Log.Info("Reconcile queued") + return result, nil + } + + // Setup the DG Configuration + result = r.setupDataguardBrokerConfiguration(dataguardBroker, singleInstanceDatabase, sidbReadyPod, adminPassword, ctx, req) + if result.Requeue { + r.Log.Info("Reconcile queued") + return result, nil + } + + // If LoadBalancer = true , ensure Connect String is updated + if dataguardBroker.Status.ExternalConnectString == dbcommons.ValueUnavailable { + return requeueY, nil + } + + // Set a particular database as primary + result = r.SetAsPrimaryDatabase(singleInstanceDatabase.Spec.Sid, dataguardBroker.Spec.SetAsPrimaryDatabase, dataguardBroker, + singleInstanceDatabase, adminPassword, ctx, req) + if result.Requeue { + r.Log.Info("Reconcile queued") + return result, nil + } + + dataguardBroker.Status.Status = dbcommons.StatusReady + + r.Log.Info("Reconcile completed") + return ctrl.Result{}, nil + +} + +// ##################################################################################################### +// +// Validate Readiness of the primary DB specified +// +// ##################################################################################################### +func (r *DataguardBrokerReconciler) validateSidbReadiness(m *dbapi.DataguardBroker, + n *dbapi.SingleInstanceDatabase, ctx context.Context, req ctrl.Request) (ctrl.Result, corev1.Pod, string) { + + log := r.Log.WithValues("validateSidbReadiness", req.NamespacedName) + adminPassword := "" + // ## FETCH THE SIDB REPLICAS . + sidbReadyPod, _, _, _, err := dbcommons.FindPods(r, n.Spec.Image.Version, + n.Spec.Image.PullFrom, n.Name, n.Namespace, ctx, req) + if err != nil { + log.Error(err, err.Error()) + return requeueY, sidbReadyPod, adminPassword + } + + if n.Status.Status != dbcommons.StatusReady { + + eventReason := "Waiting" + eventMsg := "Waiting for " + n.Name + " to be Ready" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + return requeueY, sidbReadyPod, adminPassword + } + + // Validate databaseRef Admin Password + adminPasswordSecret := &corev1.Secret{} + err = r.Get(ctx, types.NamespacedName{Name: m.Spec.AdminPassword.SecretName, Namespace: m.Namespace}, adminPasswordSecret) + if err != nil { + if apierrors.IsNotFound(err) { + //m.Status.Status = dbcommons.StatusError + eventReason := "Waiting" + eventMsg := "waiting for secret : " + m.Spec.AdminPassword.SecretName + " to get created" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + r.Log.Info("Secret " + m.Spec.AdminPassword.SecretName + " Not Found") + return requeueY, sidbReadyPod, adminPassword + } + log.Error(err, err.Error()) + return requeueY, sidbReadyPod, adminPassword + } + adminPassword = string(adminPasswordSecret.Data[m.Spec.AdminPassword.SecretKey]) + + out, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | %s", fmt.Sprintf(dbcommons.ValidateAdminPassword, adminPassword), dbcommons.GetSqlClient(n.Spec.Edition))) + if err != nil { + log.Error(err, err.Error()) + return requeueY, sidbReadyPod, adminPassword + } + if strings.Contains(out, "USER is \"SYS\"") { + log.Info("validated Admin password successfully") + } else if strings.Contains(out, "ORA-01017") { + //m.Status.Status = dbcommons.StatusError + eventReason := "Logon denied" + eventMsg := "invalid databaseRef admin password. secret: " + m.Spec.AdminPassword.SecretName + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) + return requeueY, sidbReadyPod, adminPassword + } else { + return requeueY, sidbReadyPod, adminPassword + } + + return requeueN, sidbReadyPod, adminPassword +} + +// ############################################################################# +// +// Instantiate Service spec from StandbyDatabase spec +// +// ############################################################################# +func (r *DataguardBrokerReconciler) instantiateSVCSpec(m *dbapi.DataguardBroker) *corev1.Service { + svc := &corev1.Service{ + TypeMeta: metav1.TypeMeta{ + Kind: "Service", + }, + ObjectMeta: metav1.ObjectMeta{ + Name: m.Name, + Namespace: m.Namespace, + Labels: map[string]string{ + "app": m.Name, + }, + }, + Spec: corev1.ServiceSpec{ + Ports: []corev1.ServicePort{ + { + Name: "listener", + Port: 1521, + Protocol: corev1.ProtocolTCP, + }, + { + Name: "xmldb", + Port: 5500, + Protocol: corev1.ProtocolTCP, + }, + }, + Selector: map[string]string{ + "app": m.Name, + }, + Type: corev1.ServiceType(func() string { + if m.Spec.LoadBalancer { + return "LoadBalancer" + } + return "NodePort" + }()), + }, + } + // Set StandbyDatabase instance as the owner and controller + ctrl.SetControllerReference(m, svc, r.Scheme) + return svc +} + +// ############################################################################# +// +// Create a Service for StandbyDatabase +// +// ############################################################################# +func (r *DataguardBrokerReconciler) createSVC(ctx context.Context, req ctrl.Request, + m *dbapi.DataguardBroker) ctrl.Result { + + log := r.Log.WithValues("createSVC", req.NamespacedName) + // Check if the Service already exists, if not create a new one + svc := &corev1.Service{} + // Get retrieves an obj for the given object key from the Kubernetes Cluster. + // obj must be a struct pointer so that obj can be updated with the response returned by the Server. + // Here foundsvc is the struct pointer to corev1.Service{} + err := r.Get(ctx, types.NamespacedName{Name: m.Name, Namespace: m.Namespace}, svc) + if err != nil && apierrors.IsNotFound(err) { + // Define a new Service + svc = r.instantiateSVCSpec(m) + log.Info("Creating a new Service", "Service.Namespace", svc.Namespace, "Service.Name", svc.Name) + err = r.Create(ctx, svc) + //err = r.Update(ctx, svc) + if err != nil { + log.Error(err, "Failed to create new Service", "Service.Namespace", svc.Namespace, "Service.Name", svc.Name) + return requeueY + } else { + timeout := 30 + // Waiting for Service to get created as sometimes it takes some time to create a service . 30 seconds TImeout + err = dbcommons.WaitForStatusChange(r, svc.Name, m.Namespace, ctx, req, time.Duration(timeout)*time.Second, "svc", "creation") + if err != nil { + log.Error(err, "Error in Waiting for svc status for Creation", "svc.Namespace", svc.Namespace, "SVC.Name", svc.Name) + return requeueY + } + log.Info("Succesfully Created New Service ", "Service.Name : ", svc.Name) + } + time.Sleep(10 * time.Second) + + } else if err != nil { + log.Error(err, "Failed to get Service") + return requeueY + } else if err == nil { + log.Info(" ", "Found Existing Service ", svc.Name) + } + + // update service status + + m.Status.ClusterConnectString = svc.Name + "." + svc.Namespace + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/DATAGUARD" + if m.Spec.LoadBalancer { + if len(svc.Status.LoadBalancer.Ingress) > 0 { + lbAddress := svc.Status.LoadBalancer.Ingress[0].Hostname + if lbAddress == "" { + lbAddress = svc.Status.LoadBalancer.Ingress[0].IP + } + m.Status.ExternalConnectString = lbAddress + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/DATAGUARD" + } + } else { + nodeip := dbcommons.GetNodeIp(r, ctx, req) + if nodeip != "" { + m.Status.ExternalConnectString = nodeip + ":" + fmt.Sprint(svc.Spec.Ports[0].NodePort) + "/DATAGUARD" + } + } + + return requeueN +} + +// ############################################################################# +// +// Setup the requested DG Configuration +// +// ############################################################################# +func (r *DataguardBrokerReconciler) setupDataguardBrokerConfiguration(m *dbapi.DataguardBroker, n *dbapi.SingleInstanceDatabase, + sidbReadyPod corev1.Pod, adminPassword string, ctx context.Context, req ctrl.Request) ctrl.Result { + log := r.Log.WithValues("setupDataguardBrokerConfiguration", req.NamespacedName) + + for i := 0; i < len(m.Spec.StandbyDatabaseRefs); i++ { + + standbyDatabase := &dbapi.SingleInstanceDatabase{} + err := r.Get(ctx, types.NamespacedName{Namespace: req.Namespace, Name: m.Spec.StandbyDatabaseRefs[i]}, standbyDatabase) + if err != nil { + if apierrors.IsNotFound(err) { + eventReason := "Warning" + eventMsg := m.Spec.StandbyDatabaseRefs[i] + "not found" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + continue + } + log.Error(err, err.Error()) + return requeueY + } + + // Check if dataguard broker is already configured for the standby database + if standbyDatabase.Status.DgBrokerConfigured { + log.Info("Dataguard broker for standbyDatabase : " + standbyDatabase.Name + " is already configured") + continue + } + + m.Status.Status = dbcommons.StatusCreating + r.Status().Update(ctx, m) + + // ## FETCH THE STANDBY REPLICAS . + standbyDatabaseReadyPod, _, _, _, err := dbcommons.FindPods(r, n.Spec.Image.Version, + n.Spec.Image.PullFrom, standbyDatabase.Name, standbyDatabase.Namespace, ctx, req) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + + if standbyDatabase.Status.Status != dbcommons.StatusReady { + + eventReason := "Waiting" + eventMsg := "Waiting for " + standbyDatabase.Name + " to be Ready" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + return requeueY + + } + + result := r.setupDataguardBrokerConfigurationForGivenDB(m, n, standbyDatabase, standbyDatabaseReadyPod, sidbReadyPod, ctx, req, adminPassword) + if result.Requeue { + return result + } + + // Update Databases + r.updateReconcileStatus(m, sidbReadyPod, ctx, req) + } + + eventReason := "DG Configuration up to date" + eventMsg := "" + + // Patch DataguardBroker Service to point selector to Current Primary Name + result := r.patchService(m, sidbReadyPod, n, ctx, req) + if result.Requeue { + return result + } + + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + + return requeueN +} + +// ############################################################################# +// +// Patch DataguardBroker Service to point selector to Current Primary Name +// +// ############################################################################# +func (r *DataguardBrokerReconciler) patchService(m *dbapi.DataguardBroker, sidbReadyPod corev1.Pod, n *dbapi.SingleInstanceDatabase, + ctx context.Context, req ctrl.Request) ctrl.Result { + log := r.Log.WithValues("patchService", req.NamespacedName) + databases, out, err := dbcommons.GetDatabasesInDgConfig(sidbReadyPod, r, r.Config, ctx, req) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + if !strings.Contains(out, "ORA-") { + primarySid := strings.ToUpper(dbcommons.GetPrimaryDatabase(databases)) + primaryName := n.Name + if primarySid != n.Spec.Sid { + primaryName = n.Status.StandbyDatabases[primarySid] + } + + // Patch DataguardBroker Service to point selector to Current Primary Name + svc := &corev1.Service{} + err = r.Get(ctx, types.NamespacedName{Name: req.Name, Namespace: req.Namespace}, svc) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + svc.Spec.Selector["app"] = primaryName + err = r.Update(ctx, svc) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + + m.Status.ClusterConnectString = svc.Name + "." + svc.Namespace + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/DATAGUARD" + if m.Spec.LoadBalancer { + if len(svc.Status.LoadBalancer.Ingress) > 0 { + lbAddress := svc.Status.LoadBalancer.Ingress[0].Hostname + if lbAddress == "" { + lbAddress = svc.Status.LoadBalancer.Ingress[0].IP + } + m.Status.ExternalConnectString = lbAddress + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/DATAGUARD" + } + } else { + nodeip := dbcommons.GetNodeIp(r, ctx, req) + if nodeip != "" { + m.Status.ExternalConnectString = nodeip + ":" + fmt.Sprint(svc.Spec.Ports[0].NodePort) + "/DATAGUARD" + } + } + } + return requeueN +} + +// ############################################################################# +// +// Set up DG Configuration for a given StandbyDatabase +// +// ############################################################################# +func (r *DataguardBrokerReconciler) setupDataguardBrokerConfigurationForGivenDB(m *dbapi.DataguardBroker, n *dbapi.SingleInstanceDatabase, standbyDatabase *dbapi.SingleInstanceDatabase, + standbyDatabaseReadyPod corev1.Pod, sidbReadyPod corev1.Pod, ctx context.Context, req ctrl.Request, adminPassword string) ctrl.Result { + + log := r.Log.WithValues("setupDataguardBrokerConfigurationForGivenDB", req.NamespacedName) + + if standbyDatabaseReadyPod.Name == "" || sidbReadyPod.Name == "" { + return requeueY + } + + // ## CHECK IF DG CONFIGURATION AVAILABLE IN PRIMARY DATABSE## + out, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | dgmgrl / as sysdba ", dbcommons.DBShowConfigCMD)) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + log.Info("ShowConfiguration Output") + log.Info(out) + + if strings.Contains(out, "ORA-16525") { + log.Info("ORA-16525: The Oracle Data Guard broker is not yet available on Primary") + return requeueY + } + + _, err = dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, true, "bash", "-c", + fmt.Sprintf(dbcommons.CreateAdminPasswordFile, adminPassword)) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + log.Info("DB Admin pwd file created") + + if strings.Contains(out, "ORA-16532") { + // ORA-16532: Oracle Data Guard broker configuration does not exist , so create one + if m.Spec.ProtectionMode == "MaxPerformance" { + // Construct the password file and dgbroker command file + out, err := dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf(dbcommons.CreateDGMGRLScriptFile, dbcommons.DataguardBrokerMaxPerformanceCMD)) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + log.Info("DGMGRL command file creation output") + log.Info(out) + + // ## DG CONFIGURATION FOR PRIMARY DB || MODE : MAXPERFORMANCE ## + out, err = dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + "dgmgrl sys@${PRIMARY_DB_CONN_STR} @dgmgrl.cmd < admin.pwd && rm -rf dgmgrl.cmd") + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + log.Info("DgConfigurationMaxPerformance Output") + log.Info(out) + + } else if m.Spec.ProtectionMode == "MaxAvailability" { + // ## DG CONFIGURATION FOR PRIMARY DB || MODE : MAX AVAILABILITY ## + out, err := dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf(dbcommons.CreateDGMGRLScriptFile, dbcommons.DataguardBrokerMaxAvailabilityCMD)) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + log.Info("DGMGRL command file creation output") + log.Info(out) + + // ## DG CONFIGURATION FOR PRIMARY DB || MODE : MAXPERFORMANCE ## + out, err = dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + "dgmgrl sys@${PRIMARY_DB_CONN_STR} @dgmgrl.cmd < admin.pwd && rm -rf dgmgrl.cmd") + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + log.Info("DgConfigurationMaxAvailability Output") + log.Info(out) + + } else { + log.Info("SPECIFY correct Protection Mode . Either MaxAvailability or MaxPerformance") + return requeueY + } + + // ## SHOW CONFIGURATION DG + out, err := dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | dgmgrl / as sysdba ", dbcommons.DBShowConfigCMD)) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } else { + log.Info("ShowConfiguration Output") + log.Info(out) + } + // Set DG Configured status to true for this standbyDatabase. so that in next reconcilation, we dont configure this again + standbyDatabase.Status.DgBrokerConfigured = true + r.Status().Update(ctx, standbyDatabase) + + // Remove admin pwd file + _, err = dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, true, "bash", "-c", + dbcommons.RemoveAdminPasswordFile) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + log.Info("DB Admin pwd file removed") + + return requeueN + } + + // DG Configuration Exists . So add the standbyDatabase to the existing DG Configuration + databases, _, err := dbcommons.GetDatabasesInDgConfig(sidbReadyPod, r, r.Config, ctx, req) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + + // ## ADD DATABASE TO DG CONFIG , IF NOT PRESENT + found, _ := dbcommons.IsDatabaseFound(standbyDatabase.Spec.Sid, databases, "") + if found { + return requeueN + } + primarySid := dbcommons.GetPrimaryDatabase(databases) + + // If user adds a new standby to a dg config when failover happened to one ot the standbys, we need to have current primary connect string + primaryConnectString := n.Name + ":1521/" + primarySid + if !strings.EqualFold(primarySid, n.Spec.Sid) { + primaryConnectString = n.Status.StandbyDatabases[primarySid] + ":1521/" + primarySid + } + + if m.Spec.ProtectionMode == "MaxPerformance" { + // ## DG CONFIGURATION FOR PRIMARY DB || MODE : MAXPERFORMANCE ## + out, err := dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf(dbcommons.CreateDGMGRLScriptFile, dbcommons.DataguardBrokerAddDBMaxPerformanceCMD)) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + log.Info("DGMGRL command file creation output") + log.Info(out) + + out, err = dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("dgmgrl sys@%s @dgmgrl.cmd < admin.pwd && rm -rf dgmgrl.cmd ", primaryConnectString)) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + log.Info("DgConfigurationMaxPerformance Output") + log.Info(out) + + } else if m.Spec.ProtectionMode == "MaxAvailability" { + // ## DG CONFIGURATION FOR PRIMARY DB || MODE : MAX AVAILABILITY ## + out, err := dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf(dbcommons.CreateDGMGRLScriptFile, dbcommons.DataguardBrokerAddDBMaxAvailabilityCMD)) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + log.Info("DGMGRL command file creation output") + log.Info(out) + + out, err = dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("dgmgrl sys@%s @dgmgrl.cmd < admin.pwd && rm -rf dgmgrl.cmd ", primaryConnectString)) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + log.Info("DgConfigurationMaxAvailability Output") + log.Info(out) + + } else { + log.Info("SPECIFY correct Protection Mode . Either MaxAvailability or MaxPerformance") + log.Error(err, err.Error()) + return requeueY + } + + databases, _, err = dbcommons.GetDatabasesInDgConfig(sidbReadyPod, r, r.Config, ctx, req) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + + // ## SET PROPERTY FASTSTARTFAILOVERTARGET FOR EACH DATABASE TO ALL OTHER DATABASES IN DG CONFIG . + for i := 0; i < len(databases); i++ { + out, err = dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("dgmgrl sys@%s \"EDIT DATABASE %s SET PROPERTY FASTSTARTFAILOVERTARGET=%s\"< admin.pwd", primaryConnectString, + strings.Split(databases[i], ":")[0], getFSFOTargets(i, databases))) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + log.Info("SETTING FSFO TARGET OUTPUT") + log.Info(out) + + out, err = dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("dgmgrl sys@%s \"SHOW DATABASE %s FASTSTARTFAILOVERTARGET\" < admin.pwd", primaryConnectString, strings.Split(databases[i], ":")[0])) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + log.Info("FSFO TARGETS OF " + databases[i]) + log.Info(out) + + } + // Remove admin pwd file + _, err = dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, true, "bash", "-c", + dbcommons.RemoveAdminPasswordFile) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + log.Info("DB Admin pwd file removed") + + // Set DG Configured status to true for this standbyDatabase. so that in next reconcilation, we dont configure this again + standbyDatabase.Status.DgBrokerConfigured = true + r.Status().Update(ctx, standbyDatabase) + + return requeueN +} + +// ############################################################################# +// +// Return FSFO targets of each StandbyDatabase +// Concatenation of all strings in databases slice expecting that of index 1 +// +// ############################################################################# +func getFSFOTargets(index int, databases []string) string { + fsfotargets := "" + for i := 0; i < len(databases); i++ { + if i != index { + splitstr := strings.Split(databases[i], ":") + if fsfotargets == "" { + fsfotargets = splitstr[0] + } else { + fsfotargets = fsfotargets + "," + splitstr[0] + } + } + } + return fsfotargets +} + +// ##################################################################################################### +// +// Switchovers to 'sid' db to make 'sid' db primary +// +// ##################################################################################################### +func (r *DataguardBrokerReconciler) SetAsPrimaryDatabase(sidbSid string, targetSid string, m *dbapi.DataguardBroker, n *dbapi.SingleInstanceDatabase, + adminPassword string, ctx context.Context, req ctrl.Request) ctrl.Result { + + log := r.Log.WithValues("SetAsPrimaryDatabase", req.NamespacedName) + if targetSid == "" { + log.Info("Specified sid is nil") + return requeueN + } + + // Fetch the SIDB Ready Pod + sidbReadyPod, _, _, _, err := dbcommons.FindPods(r, n.Spec.Image.Version, + (n.Spec.Image.PullFrom), n.Name, n.Namespace, ctx, req) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + + // Fetch databases in dataguard broker configuration + databases, _, err := dbcommons.GetDatabasesInDgConfig(sidbReadyPod, r, r.Config, ctx, req) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + + // Fetch the current Primary database + primarySid := dbcommons.GetPrimaryDatabase(databases) + if strings.EqualFold(primarySid, targetSid) { + log.Info(targetSid + " is already Primary") + return requeueN + } + + m.Status.Status = dbcommons.StatusUpdating + r.Status().Update(ctx, m) + + found, _ := dbcommons.IsDatabaseFound(targetSid, databases, "") + if !found { + log.Info(targetSid + " not yet set in DG config") + return requeueY + } + + // Fetch the PrimarySid Ready Pod to create chk file + var primaryReq ctrl.Request + var primaryReadyPod corev1.Pod + if !strings.EqualFold(primarySid, sidbSid) { + primaryReq = ctrl.Request{ + NamespacedName: types.NamespacedName{ + Namespace: req.Namespace, + Name: n.Status.StandbyDatabases[strings.ToUpper(primarySid)], + }, + } + primaryReadyPod, _, _, _, err = dbcommons.FindPods(r, "", "", primaryReq.Name, primaryReq.Namespace, ctx, req) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + } else { + primaryReadyPod = sidbReadyPod + } + + // Fetch the targetSid Ready Pod to create chk file + var targetReq ctrl.Request + var targetReadyPod corev1.Pod + if !strings.EqualFold(targetSid, sidbSid) { + targetReq = ctrl.Request{ + NamespacedName: types.NamespacedName{ + Namespace: req.Namespace, + Name: n.Status.StandbyDatabases[strings.ToUpper(targetSid)], + }, + } + targetReadyPod, _, _, _, err = dbcommons.FindPods(r, "", "", targetReq.Name, targetReq.Namespace, ctx, req) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + } else { + targetReadyPod = sidbReadyPod + } + + // Create a chk File so that no other pods take the lock during Switchover . + out, err := dbcommons.ExecCommand(r, r.Config, primaryReadyPod.Name, primaryReadyPod.Namespace, "", ctx, req, false, "bash", "-c", dbcommons.CreateChkFileCMD) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + log.Info("Successfully Created chk file " + out) + out, err = dbcommons.ExecCommand(r, r.Config, targetReadyPod.Name, targetReadyPod.Namespace, "", ctx, req, false, "bash", "-c", dbcommons.CreateChkFileCMD) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + log.Info("Successfully Created chk file " + out) + + eventReason := "Waiting" + eventMsg := "Switchover In Progress" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + + // Connect to 'primarySid' db using dgmgrl and switchover to 'targetSid' db to make 'targetSid' db primary + _, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", + fmt.Sprintf(dbcommons.CreateAdminPasswordFile, adminPassword)) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + log.Info("DB Admin pwd file created") + + out, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("dgmgrl sys@%s \"SWITCHOVER TO %s\" < admin.pwd", primarySid, targetSid)) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + log.Info("SWITCHOVER TO " + targetSid + " Output") + log.Info(out) + + //Delete pwd file + _, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", + dbcommons.RemoveAdminPasswordFile) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + log.Info("DB Admin pwd file removed") + + eventReason = "Success" + eventMsg = "Switchover Completed Successfully" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + + // Remove the chk File . + _, err = dbcommons.ExecCommand(r, r.Config, primaryReadyPod.Name, primaryReadyPod.Namespace, "", ctx, req, false, "bash", "-c", dbcommons.RemoveChkFileCMD) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + out, err = dbcommons.ExecCommand(r, r.Config, targetReadyPod.Name, targetReadyPod.Namespace, "", ctx, req, false, "bash", "-c", dbcommons.RemoveChkFileCMD) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + log.Info("Successfully Removed chk file " + out) + + // Update Databases + r.updateReconcileStatus(m, sidbReadyPod, ctx, req) + + // Update status of Primary true/false on 'primary' db (From which switchover initiated) + if !strings.EqualFold(primarySid, sidbSid) { + + standbyDatabase := &dbapi.SingleInstanceDatabase{} + err = r.Get(ctx, primaryReq.NamespacedName, standbyDatabase) + if err != nil { + return requeueN + } + out, err := dbcommons.GetDatabaseRole(primaryReadyPod, r, r.Config, ctx, primaryReq, n.Spec.Edition) + if err == nil { + standbyDatabase.Status.Role = strings.ToUpper(out) + } + r.Status().Update(ctx, standbyDatabase) + + } else { + sidbReq := ctrl.Request{ + NamespacedName: types.NamespacedName{ + Namespace: req.Namespace, + Name: n.Name, + }, + } + out, err := dbcommons.GetDatabaseRole(sidbReadyPod, r, r.Config, ctx, sidbReq, n.Spec.Edition) + if err == nil { + n.Status.Role = strings.ToUpper(out) + } + r.Status().Update(ctx, n) + } + + // Update status of Primary true/false on 'sid' db (To which switchover initiated) + if !strings.EqualFold(targetSid, sidbSid) { + + standbyDatabase := &dbapi.SingleInstanceDatabase{} + err = r.Get(ctx, targetReq.NamespacedName, standbyDatabase) + if err != nil { + return requeueN + } + out, err := dbcommons.GetDatabaseRole(targetReadyPod, r, r.Config, ctx, targetReq, n.Spec.Edition) + if err == nil { + standbyDatabase.Status.Role = strings.ToUpper(out) + } + r.Status().Update(ctx, standbyDatabase) + + } else { + sidbReq := ctrl.Request{ + NamespacedName: types.NamespacedName{ + Namespace: req.Namespace, + Name: n.Name, + }, + } + out, err := dbcommons.GetDatabaseRole(sidbReadyPod, r, r.Config, ctx, sidbReq, n.Spec.Edition) + if err == nil { + n.Status.Role = strings.ToUpper(out) + } + r.Status().Update(ctx, n) + } + + // Patch DataguardBroker Service to point selector to Current Primary Name and updates client db connection strings on dataguardBroker + result := r.patchService(m, sidbReadyPod, n, ctx, req) + if result.Requeue { + return result + } + + return requeueN +} + +// ############################################################################# +// +// Update Reconcile Status +// +// ############################################################################# +func (r *DataguardBrokerReconciler) updateReconcileStatus(m *dbapi.DataguardBroker, sidbReadyPod corev1.Pod, + ctx context.Context, req ctrl.Request) (err error) { + + // ConnectStrings updated in PatchService() + var databases []string + databases, _, err = dbcommons.GetDatabasesInDgConfig(sidbReadyPod, r, r.Config, ctx, req) + if err == nil { + primaryDatabase := "" + standbyDatabases := "" + for i := 0; i < len(databases); i++ { + splitstr := strings.Split(databases[i], ":") + if strings.ToUpper(splitstr[1]) == "PRIMARY" { + primaryDatabase = strings.ToUpper(splitstr[0]) + } + if strings.ToUpper(splitstr[1]) == "PHYSICAL_STANDBY" { + if standbyDatabases != "" { + standbyDatabases += "," + strings.ToUpper(splitstr[0]) + } else { + standbyDatabases = strings.ToUpper(splitstr[0]) + } + } + } + m.Status.PrimaryDatabase = primaryDatabase + m.Status.StandbyDatabases = standbyDatabases + } + + m.Status.PrimaryDatabaseRef = m.Spec.PrimaryDatabaseRef + m.Status.ProtectionMode = m.Spec.ProtectionMode + return +} + +// ############################################################################# +// +// Manage Finalizer to cleanup before deletion of DataguardBroker +// +// ############################################################################# +func (r *DataguardBrokerReconciler) manageDataguardBrokerDeletion(req ctrl.Request, ctx context.Context, m *dbapi.DataguardBroker) (ctrl.Result, error) { + log := r.Log.WithValues("manageDataguardBrokerDeletion", req.NamespacedName) + + // Check if the DataguardBroker instance is marked to be deleted, which is + // indicated by the deletion timestamp being set. + isDataguardBrokerMarkedToBeDeleted := m.GetDeletionTimestamp() != nil + if isDataguardBrokerMarkedToBeDeleted { + if controllerutil.ContainsFinalizer(m, dataguardBrokerFinalizer) { + // Run finalization logic for dataguardBrokerFinalizer. If the + // finalization logic fails, don't remove the finalizer so + // that we can retry during the next reconciliation. + result, err := r.cleanupDataguardBroker(req, ctx, m) + if result.Requeue { + return result, err + } + + // Remove dataguardBrokerFinalizer. Once all finalizers have been + // removed, the object will be deleted. + controllerutil.RemoveFinalizer(m, dataguardBrokerFinalizer) + err = r.Update(ctx, m) + if err != nil { + log.Error(err, err.Error()) + return requeueY, err + } + } + return requeueY, errors.New("deletion pending") + } + + // Add finalizer for this CR + if !controllerutil.ContainsFinalizer(m, dataguardBrokerFinalizer) { + controllerutil.AddFinalizer(m, dataguardBrokerFinalizer) + err := r.Update(ctx, m) + if err != nil { + log.Error(err, err.Error()) + return requeueY, err + } + } + return requeueN, nil +} + +// ############################################################################# +// +// Finalization logic for DataguardBrokerFinalizer +// +// ############################################################################# +func (r *DataguardBrokerReconciler) cleanupDataguardBroker(req ctrl.Request, ctx context.Context, m *dbapi.DataguardBroker) (ctrl.Result, error) { + log := r.Log.WithValues("cleanupDataguardBroker", req.NamespacedName) + + // Fetch Primary Database Reference + singleInstanceDatabase := &dbapi.SingleInstanceDatabase{} + err := r.Get(ctx, types.NamespacedName{Namespace: req.Namespace, Name: m.Spec.PrimaryDatabaseRef}, singleInstanceDatabase) + if err != nil { + if apierrors.IsNotFound(err) { + r.Log.Info("Resource deleted. No need to remove dataguard configuration") + return requeueN, nil + } + return requeueY, err + } + + // Validate if Primary Database Reference is ready + result, sidbReadyPod, _ := r.validateSidbReadiness(m, singleInstanceDatabase, ctx, req) + if result.Requeue { + r.Log.Info("Reconcile queued") + return result, nil + } + + // Get Primary database to remove dataguard configuration + _, out, err := dbcommons.GetDatabasesInDgConfig(sidbReadyPod, r, r.Config, ctx, req) + if err != nil { + log.Error(err, err.Error()) + return requeueY, err + } + + //primarySid := dbcommons.GetPrimaryDatabase(databases) + + out, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | dgmgrl / as sysdba ", dbcommons.RemoveDataguardConfiguration)) + if err != nil { + log.Error(err, err.Error()) + return requeueY, err + } + log.Info("RemoveDataguardConfiguration Output") + log.Info(out) + + for i := 0; i < len(m.Spec.StandbyDatabaseRefs); i++ { + + standbyDatabase := &dbapi.SingleInstanceDatabase{} + err := r.Get(ctx, types.NamespacedName{Namespace: req.Namespace, Name: m.Spec.StandbyDatabaseRefs[i]}, standbyDatabase) + if err != nil { + if apierrors.IsNotFound(err) { + continue + } + log.Error(err, err.Error()) + return requeueY, err + } + + // Set DgBrokerConfigured to false + standbyDatabase.Status.DgBrokerConfigured = false + r.Status().Update(ctx, standbyDatabase) + } + + log.Info("Successfully cleaned up Dataguard Broker") + return requeueN, nil +} + +// ############################################################################# +// +// SetupWithManager sets up the controller with the Manager +// +// ############################################################################# +func (r *DataguardBrokerReconciler) SetupWithManager(mgr ctrl.Manager) error { + return ctrl.NewControllerManagedBy(mgr). + For(&dbapi.DataguardBroker{}). + Owns(&corev1.Pod{}). //Watch for deleted pods of DataguardBroker Owner + WithEventFilter(dbcommons.ResourceEventHandler()). + WithOptions(controller.Options{MaxConcurrentReconciles: 100}). //ReconcileHandler is never invoked concurrently with the same object. + Complete(r) +} diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 267759aa..26784b5b 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2022 Oracle and/or its affiliates. +** Copyright (c) 2023 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** @@ -219,9 +219,11 @@ func (r *OracleRestDataServiceReconciler) Reconcile(ctx context.Context, req ctr return ctrl.Result{}, nil } -//############################################################################# -// Validate the CRD specs -//############################################################################# +// ############################################################################# +// +// Validate the CRD specs +// +// ############################################################################# func (r *OracleRestDataServiceReconciler) validate(m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase, ctx context.Context) (ctrl.Result, error) { @@ -302,9 +304,11 @@ func (r *OracleRestDataServiceReconciler) validate(m *dbapi.OracleRestDataServic return requeueN, err } -//##################################################################################################### -// Validate Readiness of the primary DB specified -//##################################################################################################### +// ##################################################################################################### +// +// Validate Readiness of the primary DB specified +// +// ##################################################################################################### func (r *OracleRestDataServiceReconciler) validateSIDBReadiness(m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase, ctx context.Context, req ctrl.Request) (ctrl.Result, corev1.Pod) { @@ -383,9 +387,11 @@ func (r *OracleRestDataServiceReconciler) validateSIDBReadiness(m *dbapi.OracleR return requeueN, sidbReadyPod } -//##################################################################################################### -// Check ORDS Health Status -//##################################################################################################### +// ##################################################################################################### +// +// Check ORDS Health Status +// +// ##################################################################################################### func (r *OracleRestDataServiceReconciler) checkHealthStatus(m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase, sidbReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) (ctrl.Result, corev1.Pod) { log := r.Log.WithValues("checkHealthStatus", req.NamespacedName) @@ -444,9 +450,11 @@ func (r *OracleRestDataServiceReconciler) checkHealthStatus(m *dbapi.OracleRestD return requeueN, readyPod } -//############################################################################# -// Instantiate Service spec from OracleRestDataService spec -//############################################################################# +// ############################################################################# +// +// Instantiate Service spec from OracleRestDataService spec +// +// ############################################################################# func (r *OracleRestDataServiceReconciler) instantiateSVCSpec(m *dbapi.OracleRestDataService) *corev1.Service { svc := &corev1.Service{ TypeMeta: metav1.TypeMeta{ @@ -492,9 +500,11 @@ func (r *OracleRestDataServiceReconciler) instantiateSVCSpec(m *dbapi.OracleRest return svc } -//############################################################################# -// Instantiate POD spec from OracleRestDataService spec -//############################################################################# +// ############################################################################# +// +// Instantiate POD spec from OracleRestDataService spec +// +// ############################################################################# func (r *OracleRestDataServiceReconciler) instantiatePodSpec(m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase) (*corev1.Pod, *corev1.Secret) { @@ -749,9 +759,11 @@ func (r *OracleRestDataServiceReconciler) instantiatePodSpec(m *dbapi.OracleRest // Instantiate POD spec from OracleRestDataService spec //############################################################################# -//############################################################################# -// Instantiate Persistent Volume Claim spec from SingleInstanceDatabase spec -//############################################################################# +// ############################################################################# +// +// Instantiate Persistent Volume Claim spec from SingleInstanceDatabase spec +// +// ############################################################################# func (r *OracleRestDataServiceReconciler) instantiatePVCSpec(m *dbapi.OracleRestDataService) *corev1.PersistentVolumeClaim { pvc := &corev1.PersistentVolumeClaim{ @@ -802,9 +814,11 @@ func (r *OracleRestDataServiceReconciler) instantiatePVCSpec(m *dbapi.OracleRest return pvc } -//############################################################################# -// Create a Service for OracleRestDataService -//############################################################################# +// ############################################################################# +// +// Create a Service for OracleRestDataService +// +// ############################################################################# func (r *OracleRestDataServiceReconciler) createSVC(ctx context.Context, req ctrl.Request, m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase) ctrl.Result { @@ -888,9 +902,11 @@ func (r *OracleRestDataServiceReconciler) createSVC(ctx context.Context, req ctr return requeueN } -//############################################################################# -// Stake a claim for Persistent Volume -//############################################################################# +// ############################################################################# +// +// Stake a claim for Persistent Volume +// +// ############################################################################# func (r *OracleRestDataServiceReconciler) createPVC(ctx context.Context, req ctrl.Request, m *dbapi.OracleRestDataService) (ctrl.Result, error) { @@ -922,9 +938,11 @@ func (r *OracleRestDataServiceReconciler) createPVC(ctx context.Context, req ctr return requeueN, nil } -//############################################################################# -// Create the requested POD replicas -//############################################################################# +// ############################################################################# +// +// Create the requested POD replicas +// +// ############################################################################# func (r *OracleRestDataServiceReconciler) createPods(m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase, ctx context.Context, req ctrl.Request) ctrl.Result { @@ -1008,9 +1026,11 @@ func (r *OracleRestDataServiceReconciler) createPods(m *dbapi.OracleRestDataServ return requeueN } -//############################################################################# -// Manage Finalizer to cleanup before deletion of OracleRestDataService -//############################################################################# +// ############################################################################# +// +// Manage Finalizer to cleanup before deletion of OracleRestDataService +// +// ############################################################################# func (r *OracleRestDataServiceReconciler) manageOracleRestDataServiceDeletion(req ctrl.Request, ctx context.Context, m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase) ctrl.Result { log := r.Log.WithValues("manageOracleRestDataServiceDeletion", req.NamespacedName) @@ -1065,9 +1085,11 @@ func (r *OracleRestDataServiceReconciler) manageOracleRestDataServiceDeletion(re return requeueN } -//############################################################################# -// Finalization logic for OracleRestDataServiceFinalizer -//############################################################################# +// ############################################################################# +// +// Finalization logic for OracleRestDataServiceFinalizer +// +// ############################################################################# func (r *OracleRestDataServiceReconciler) cleanupOracleRestDataService(req ctrl.Request, ctx context.Context, m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase) error { log := r.Log.WithValues("cleanupOracleRestDataService", req.NamespacedName) @@ -1209,9 +1231,11 @@ func (r *OracleRestDataServiceReconciler) cleanupOracleRestDataService(req ctrl. return nil } -//############################################################################# -// Configure APEX -//############################################################################# +// ############################################################################# +// +// Configure APEX +// +// ############################################################################# func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase, sidbReadyPod corev1.Pod, ordsReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) ctrl.Result { log := r.Log.WithValues("configureApex", req.NamespacedName) @@ -1295,9 +1319,11 @@ func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataS return requeueY } -//############################################################################# -// Install APEX in SIDB -//############################################################################# +// ############################################################################# +// +// Install APEX in SIDB +// +// ############################################################################# func (r *OracleRestDataServiceReconciler) installApex(m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase, ordsReadyPod corev1.Pod, apexPassword string, ctx context.Context, req ctrl.Request) ctrl.Result { log := r.Log.WithValues("installApex", req.NamespacedName) @@ -1361,9 +1387,11 @@ func (r *OracleRestDataServiceReconciler) installApex(m *dbapi.OracleRestDataSer return requeueN } -//############################################################################# -// Delete Secrets -//############################################################################# +// ############################################################################# +// +// Delete Secrets +// +// ############################################################################# func (r *OracleRestDataServiceReconciler) deleteSecrets(m *dbapi.OracleRestDataService, ctx context.Context, req ctrl.Request) { log := r.Log.WithValues("deleteSecrets", req.NamespacedName) @@ -1408,9 +1436,11 @@ func (r *OracleRestDataServiceReconciler) deleteSecrets(m *dbapi.OracleRestDataS } -//############################################################################# -// Rest Enable/Disable Schemas -//############################################################################# +// ############################################################################# +// +// Rest Enable/Disable Schemas +// +// ############################################################################# func (r *OracleRestDataServiceReconciler) restEnableSchemas(m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase, sidbReadyPod corev1.Pod, ordsReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) ctrl.Result { @@ -1542,9 +1572,11 @@ func (r *OracleRestDataServiceReconciler) restEnableSchemas(m *dbapi.OracleRestD return requeueN } -//############################################################################# -// SetupWithManager sets up the controller with the Manager. -//############################################################################# +// ############################################################################# +// +// SetupWithManager sets up the controller with the Manager. +// +// ############################################################################# func (r *OracleRestDataServiceReconciler) SetupWithManager(mgr ctrl.Manager) error { return ctrl.NewControllerManagedBy(mgr). For(&dbapi.OracleRestDataService{}). diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 314b2c66..54bde566 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2022 Oracle and/or its affiliates. +** Copyright (c) 2023 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** @@ -109,6 +109,7 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct singleInstanceDatabase := &dbapi.SingleInstanceDatabase{} cloneFromDatabase := &dbapi.SingleInstanceDatabase{} + referredPrimaryDatabase := &dbapi.SingleInstanceDatabase{} // Execute for every reconcile defer r.updateReconcileStatus(singleInstanceDatabase, ctx, &result, &err, &blocked, &completed) @@ -134,6 +135,7 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct singleInstanceDatabase.Status.Role = dbcommons.ValueUnavailable singleInstanceDatabase.Status.ConnectString = dbcommons.ValueUnavailable singleInstanceDatabase.Status.PdbConnectString = dbcommons.ValueUnavailable + singleInstanceDatabase.Status.TcpsConnectString = dbcommons.ValueUnavailable singleInstanceDatabase.Status.OemExpressUrl = dbcommons.ValueUnavailable singleInstanceDatabase.Status.ReleaseUpdate = dbcommons.ValueUnavailable r.Status().Update(ctx, singleInstanceDatabase) @@ -147,7 +149,7 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct } // First validate - result, err = r.validate(singleInstanceDatabase, cloneFromDatabase, ctx, req) + result, err = r.validate(singleInstanceDatabase, cloneFromDatabase, referredPrimaryDatabase, ctx, req) if result.Requeue { r.Log.Info("Spec validation failed, Reconcile queued") return result, nil @@ -172,7 +174,7 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct } // POD creation - result, err = r.createOrReplacePods(singleInstanceDatabase, cloneFromDatabase, ctx, req) + result, err = r.createOrReplacePods(singleInstanceDatabase, cloneFromDatabase, referredPrimaryDatabase, ctx, req) if result.Requeue { r.Log.Info("Reconcile queued") return result, nil @@ -197,29 +199,47 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct } } - // Update DB config - result, err = r.updateDBConfig(singleInstanceDatabase, readyPod, ctx, req) - if result.Requeue { - r.Log.Info("Reconcile queued") - return result, nil - } + if strings.ToUpper(singleInstanceDatabase.Status.Role) == "PRIMARY" { + // Update DB config + result, err = r.updateDBConfig(singleInstanceDatabase, readyPod, ctx, req) + if result.Requeue { + r.Log.Info("Reconcile queued") + return result, nil + } - // Update Init Parameters - result, err = r.updateInitParameters(singleInstanceDatabase, readyPod, ctx, req) - if result.Requeue { - r.Log.Info("Reconcile queued") - return result, nil - } + // Update Init Parameters + result, err = r.updateInitParameters(singleInstanceDatabase, readyPod, ctx, req) + if result.Requeue { + r.Log.Info("Reconcile queued") + return result, nil + } + + // Configure TCPS + result, err = r.configTcps(singleInstanceDatabase, readyPod, ctx, req) + if result.Requeue { + r.Log.Info("Reconcile queued") + return result, nil + } + + } else { + // Database is in role of standby + err = SetupStandbyDatabase(r, singleInstanceDatabase, referredPrimaryDatabase, ctx, req) + if err != nil { + return requeueY, err + } + + singleInstanceDatabase.Status.PrimaryDatabase = referredPrimaryDatabase.Name + // Store all standbyDatabase sid:name in a map to use it during manual switchover. + if len(referredPrimaryDatabase.Status.StandbyDatabases) == 0 { + referredPrimaryDatabase.Status.StandbyDatabases = make(map[string]string) + } + referredPrimaryDatabase.Status.StandbyDatabases[strings.ToUpper(singleInstanceDatabase.Spec.Sid)] = singleInstanceDatabase.Name + r.Status().Update(ctx, referredPrimaryDatabase) - // Configure TCPS - result, err = r.configTcps(singleInstanceDatabase, readyPod, ctx, req) - if result.Requeue { - r.Log.Info("Reconcile queued") - return result, nil } // Run Datapatch - if singleInstanceDatabase.Status.DatafilesPatched != "true" { + if strings.ToUpper(singleInstanceDatabase.Status.Role) == "PRIMARY" && singleInstanceDatabase.Status.DatafilesPatched != "true" { // add a blocking reconcile condition err = errors.New("processing datapatch execution") blocked = true @@ -254,9 +274,11 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct return requeueN, nil } -//############################################################################# -// Update each reconcile condtion/status -//############################################################################# +// ############################################################################# +// +// Update each reconcile condtion/status +// +// ############################################################################# func (r *SingleInstanceDatabaseReconciler) updateReconcileStatus(m *dbapi.SingleInstanceDatabase, ctx context.Context, result *ctrl.Result, err *error, blocked *bool, completed *bool) { @@ -315,13 +337,15 @@ func (r *SingleInstanceDatabaseReconciler) updateReconcileStatus(m *dbapi.Single meta.SetStatusCondition(&m.Status.Conditions, condition) } -//############################################################################# -// Validate the CRD specs -// m = SingleInstanceDatabase -// n = CloneFromDatabase -//############################################################################# +// ############################################################################# +// +// Validate the CRD specs +// m = SingleInstanceDatabase +// n = CloneFromDatabase +// +// ############################################################################# func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatabase, - n *dbapi.SingleInstanceDatabase, ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + n *dbapi.SingleInstanceDatabase, rp *dbapi.SingleInstanceDatabase, ctx context.Context, req ctrl.Request) (ctrl.Result, error) { var err error eventReason := "Spec Error" var eventMsgs []string @@ -373,6 +397,9 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab if m.Spec.Edition == "express" && m.Spec.CloneFrom != "" { eventMsgs = append(eventMsgs, "cloning not supported for express edition") } + if m.Spec.Edition == "express" && m.Spec.PrimaryDatabaseRef != "" && m.Spec.CreateAsStandby { + eventMsgs = append(eventMsgs, "Standby database creation is not supported for express edition") + } if m.Status.OrdsReference != "" && m.Status.Persistence.Size != "" && m.Status.Persistence != m.Spec.Persistence { eventMsgs = append(eventMsgs, "uninstall ORDS to change Peristence") } @@ -453,15 +480,48 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab } + if m.Spec.PrimaryDatabaseRef != "" && m.Spec.CreateAsStandby { + + // Fetch the Primary database reference, required for all iterations + err = r.Get(ctx, types.NamespacedName{Namespace: req.Namespace, Name: m.Spec.PrimaryDatabaseRef}, rp) + if err != nil { + if apierrors.IsNotFound(err) { + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, err.Error()) + r.Log.Info(err.Error()) + return requeueN, err + } + return requeueY, err + } + + if m.Status.DatafilesCreated == "true" || + !dbcommons.IsSourceDatabaseOnCluster(m.Spec.PrimaryDatabaseRef) { + return requeueN, nil + } + m.Status.Edition = rp.Status.Edition + + err = ValidatePrimaryDatabaseForStandbyCreation(r, m, rp, ctx, req) + if err != nil { + return requeueY, err + } + + r.Log.Info("Settingup Primary Database for standby creation...") + err = SetupPrimaryDatabase(r, m, rp, ctx, req) + if err != nil { + return requeueY, err + } + + } r.Log.Info("Completed reconcile validation") return requeueN, nil } -//############################################################################# -// Instantiate POD spec from SingleInstanceDatabase spec -//############################################################################# -func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleInstanceDatabase, n *dbapi.SingleInstanceDatabase, +// ############################################################################# +// +// Instantiate POD spec from SingleInstanceDatabase spec +// +// ############################################################################# +func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleInstanceDatabase, n *dbapi.SingleInstanceDatabase, rp *dbapi.SingleInstanceDatabase, requiredAffinity bool) *corev1.Pod { // POD spec @@ -720,8 +780,8 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, } } - if m.Spec.CloneFrom == "" { - // For new DB use case + if m.Spec.CloneFrom != "" { + // Clone DB use-case return []corev1.EnvVar{ { Name: "SVC_HOST", @@ -731,65 +791,105 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns Name: "SVC_PORT", Value: strconv.Itoa(int(dbcommons.CONTAINER_LISTENER_PORT)), }, - { - Name: "CREATE_PDB", - Value: func() string { - if m.Spec.Pdbname != "" { - return "true" - } - return "false" - }(), - }, { Name: "ORACLE_SID", Value: strings.ToUpper(m.Spec.Sid), }, { - Name: "WALLET_DIR", + Name: "WALLET_DIR", + Value: "/opt/oracle/oradata/dbconfig/$(ORACLE_SID)/.wallet", + }, + { + Name: "PRIMARY_DB_CONN_STR", Value: func() string { - if m.Spec.Image.PrebuiltDB { - return "" // No wallets for prebuilt DB + if dbcommons.IsSourceDatabaseOnCluster(m.Spec.CloneFrom) { + return n.Name + ":" + strconv.Itoa(int(dbcommons.CONTAINER_LISTENER_PORT)) + "/" + n.Spec.Sid } - return "/opt/oracle/oradata/dbconfig/$(ORACLE_SID)/.wallet" + return m.Spec.CloneFrom }(), }, { - Name: "ORACLE_PDB", - Value: m.Spec.Pdbname, + Name: "ORACLE_HOSTNAME", + ValueFrom: &corev1.EnvVarSource{ + FieldRef: &corev1.ObjectFieldSelector{ + FieldPath: "status.podIP", + }, + }, }, { - Name: "ORACLE_CHARACTERSET", - Value: m.Spec.Charset, + Name: "CLONE_DB", + Value: "true", }, { - Name: "ORACLE_EDITION", - Value: m.Spec.Edition, + Name: "SKIP_DATAPATCH", + Value: "true", + }, + } + + } else if m.Spec.PrimaryDatabaseRef != "" && m.Spec.CreateAsStandby { + //Standby DB Usecase + return []corev1.EnvVar{ + { + Name: "SVC_HOST", + Value: m.Name, + }, + { + Name: "SVC_PORT", + Value: strconv.Itoa(int(dbcommons.CONTAINER_LISTENER_PORT)), + }, + { + Name: "ORACLE_SID", + Value: strings.ToUpper(m.Spec.Sid), }, { - Name: "INIT_SGA_SIZE", + Name: "WALLET_DIR", + Value: "/opt/oracle/oradata/dbconfig/$(ORACLE_SID)/.wallet", + }, + { + Name: "PRIMARY_DB_CONN_STR", Value: func() string { - if m.Spec.InitParams.SgaTarget > 0 && m.Spec.InitParams.PgaAggregateTarget > 0 { - return strconv.Itoa(m.Spec.InitParams.SgaTarget) + if dbcommons.IsSourceDatabaseOnCluster(m.Spec.PrimaryDatabaseRef) { + return rp.Name + ":" + strconv.Itoa(int(dbcommons.CONTAINER_LISTENER_PORT)) + "/" + rp.Spec.Sid } - return "" + return m.Spec.PrimaryDatabaseRef }(), }, { - Name: "INIT_PGA_SIZE", + Name: "PRIMARY_SID", + Value: strings.ToUpper(rp.Spec.Sid), + }, + { + Name: "PRIMARY_IP", + Value: rp.Name, + }, + { + Name: "CREATE_PDB", Value: func() string { - if m.Spec.InitParams.SgaTarget > 0 && m.Spec.InitParams.PgaAggregateTarget > 0 { - return strconv.Itoa(m.Spec.InitParams.SgaTarget) + if rp.Spec.Pdbname != "" { + return "true" } - return "" + return "false" }(), }, + { + Name: "ORACLE_HOSTNAME", + ValueFrom: &corev1.EnvVarSource{ + FieldRef: &corev1.ObjectFieldSelector{ + FieldPath: "status.podIP", + }, + }, + }, + { + Name: "STANDBY_DB", + Value: "true", + }, { Name: "SKIP_DATAPATCH", Value: "true", }, } } - // For clone DB use case + return []corev1.EnvVar{ { Name: "SVC_HOST", @@ -800,39 +900,63 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns Value: strconv.Itoa(int(dbcommons.CONTAINER_LISTENER_PORT)), }, { - Name: "ORACLE_SID", - Value: strings.ToUpper(m.Spec.Sid), + Name: "CREATE_PDB", + Value: func() string { + if m.Spec.Pdbname != "" { + return "true" + } + return "false" + }(), }, { - Name: "WALLET_DIR", - Value: "/opt/oracle/oradata/dbconfig/$(ORACLE_SID)/.wallet", + Name: "ORACLE_SID", + Value: strings.ToUpper(m.Spec.Sid), }, { - Name: "PRIMARY_DB_CONN_STR", + Name: "WALLET_DIR", Value: func() string { - if dbcommons.IsSourceDatabaseOnCluster(m.Spec.CloneFrom) { - return n.Name + ":" + strconv.Itoa(int(dbcommons.CONTAINER_LISTENER_PORT)) + "/" + n.Spec.Sid + if m.Spec.Image.PrebuiltDB { + return "" // No wallets for prebuilt DB } - return m.Spec.CloneFrom + return "/opt/oracle/oradata/dbconfig/$(ORACLE_SID)/.wallet" }(), }, { - Name: "ORACLE_HOSTNAME", - ValueFrom: &corev1.EnvVarSource{ - FieldRef: &corev1.ObjectFieldSelector{ - FieldPath: "status.podIP", - }, - }, + Name: "ORACLE_PDB", + Value: m.Spec.Pdbname, }, { - Name: "CLONE_DB", - Value: "true", + Name: "ORACLE_CHARACTERSET", + Value: m.Spec.Charset, + }, + { + Name: "ORACLE_EDITION", + Value: m.Spec.Edition, + }, + { + Name: "INIT_SGA_SIZE", + Value: func() string { + if m.Spec.InitParams.SgaTarget > 0 && m.Spec.InitParams.PgaAggregateTarget > 0 { + return strconv.Itoa(m.Spec.InitParams.SgaTarget) + } + return "" + }(), + }, + { + Name: "INIT_PGA_SIZE", + Value: func() string { + if m.Spec.InitParams.SgaTarget > 0 && m.Spec.InitParams.PgaAggregateTarget > 0 { + return strconv.Itoa(m.Spec.InitParams.SgaTarget) + } + return "" + }(), }, { Name: "SKIP_DATAPATCH", Value: "true", }, } + }(), }}, @@ -876,9 +1000,11 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns } -//############################################################################# -// Instantiate Service spec from SingleInstanceDatabase spec -//############################################################################# +// ############################################################################# +// +// Instantiate Service spec from SingleInstanceDatabase spec +// +// ############################################################################# func (r *SingleInstanceDatabaseReconciler) instantiateSVCSpec(m *dbapi.SingleInstanceDatabase, svcName string, ports []corev1.ServicePort, svcType corev1.ServiceType) *corev1.Service { svc := &corev1.Service{ @@ -915,9 +1041,11 @@ func (r *SingleInstanceDatabaseReconciler) instantiateSVCSpec(m *dbapi.SingleIns return svc } -//############################################################################# -// Instantiate Persistent Volume Claim spec from SingleInstanceDatabase spec -//############################################################################# +// ############################################################################# +// +// Instantiate Persistent Volume Claim spec from SingleInstanceDatabase spec +// +// ############################################################################# func (r *SingleInstanceDatabaseReconciler) instantiatePVCSpec(m *dbapi.SingleInstanceDatabase) *corev1.PersistentVolumeClaim { pvc := &corev1.PersistentVolumeClaim{ @@ -977,9 +1105,11 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePVCSpec(m *dbapi.SingleIns return pvc } -//############################################################################# -// Stake a claim for Persistent Volume -//############################################################################# +// ############################################################################# +// +// Stake a claim for Persistent Volume +// +// ############################################################################# func (r *SingleInstanceDatabaseReconciler) createOrReplacePVC(ctx context.Context, req ctrl.Request, m *dbapi.SingleInstanceDatabase) (ctrl.Result, error) { @@ -1035,18 +1165,20 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplacePVC(ctx context.Contex return requeueN, nil } -//############################################################################# -// Create Services for SingleInstanceDatabase -//############################################################################# +// ############################################################################# +// +// Create Services for SingleInstanceDatabase +// +// ############################################################################# func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Context, req ctrl.Request, m *dbapi.SingleInstanceDatabase) (ctrl.Result, error) { log := r.Log.WithValues("createOrReplaceSVC", req.NamespacedName) /** Two k8s services gets created: - 1. One service is ClusterIP service for cluster only communications on the listener port 1521, - 2. One service is NodePort/LoadBalancer (according to the YAML specs) for users to connect - **/ + 1. One service is ClusterIP service for cluster only communications on the listener port 1521, + 2. One service is NodePort/LoadBalancer (according to the YAML specs) for users to connect + **/ // clusterSvc is the cluster-wide service and extSvc is the external service for the users to connect clusterSvc := &corev1.Service{} @@ -1338,12 +1470,14 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex return requeueN, nil } -//############################################################################# -// Create new Pods or delete old/extra pods -// m = SingleInstanceDatabase -// n = CloneFromDatabase -//############################################################################# -func (r *SingleInstanceDatabaseReconciler) createOrReplacePods(m *dbapi.SingleInstanceDatabase, n *dbapi.SingleInstanceDatabase, +// ############################################################################# +// +// Create new Pods or delete old/extra pods +// m = SingleInstanceDatabase +// n = CloneFromDatabase +// +// ############################################################################# +func (r *SingleInstanceDatabaseReconciler) createOrReplacePods(m *dbapi.SingleInstanceDatabase, n *dbapi.SingleInstanceDatabase, rp *dbapi.SingleInstanceDatabase, ctx context.Context, req ctrl.Request) (ctrl.Result, error) { log := r.Log.WithValues("createOrReplacePods", req.NamespacedName) @@ -1442,7 +1576,7 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplacePods(m *dbapi.SingleIn r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) } // If version is same , call createPods() with the same version , and no of Replicas required - return r.createPods(m, n, ctx, req, replicasFound, false) + return r.createPods(m, n, rp, ctx, req, replicasFound, false) } // Version/Image changed @@ -1515,7 +1649,7 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplacePods(m *dbapi.SingleIn // create new Pods with the new Version and no.of Replicas required // if m.Status.Replicas > 1, then it is replica based patching - result, err := r.createPods(m, n, ctx, req, newReplicasFound, m.Status.Replicas > 1) + result, err := r.createPods(m, n, rp, ctx, req, newReplicasFound, m.Status.Replicas > 1) if result.Requeue { return result, err } @@ -1526,9 +1660,11 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplacePods(m *dbapi.SingleIn // PATCHING END } -//############################################################################# -// Function for creating Oracle Wallet -//############################################################################# +// ############################################################################# +// +// Function for creating Oracle Wallet +// +// ############################################################################# func (r *SingleInstanceDatabaseReconciler) createWallet(m *dbapi.SingleInstanceDatabase, ctx context.Context, req ctrl.Request) (ctrl.Result, error) { // Wallet not supported for XE Database @@ -1642,13 +1778,15 @@ func (r *SingleInstanceDatabaseReconciler) createWallet(m *dbapi.SingleInstanceD return requeueN, nil } -//############################################################################## -// Create the requested POD replicas -// m = SingleInstanceDatabase -// n = CloneFromDatabase -// patching = Boolean variable to differentiate normal usecase with patching -//############################################################################## -func (r *SingleInstanceDatabaseReconciler) createPods(m *dbapi.SingleInstanceDatabase, n *dbapi.SingleInstanceDatabase, +// ############################################################################## +// +// Create the requested POD replicas +// m = SingleInstanceDatabase +// n = CloneFromDatabase +// patching = Boolean variable to differentiate normal usecase with patching +// +// ############################################################################## +func (r *SingleInstanceDatabaseReconciler) createPods(m *dbapi.SingleInstanceDatabase, n *dbapi.SingleInstanceDatabase, rp *dbapi.SingleInstanceDatabase, ctx context.Context, req ctrl.Request, replicasFound int, replicaPatching bool) (ctrl.Result, error) { log := r.Log.WithValues("createPods", req.NamespacedName) @@ -1670,7 +1808,7 @@ func (r *SingleInstanceDatabaseReconciler) createPods(m *dbapi.SingleInstanceDat // if Found < Required, create new pods, name of pods are generated randomly for i := replicasFound; i < replicasReq; i++ { // mandatory pod affinity if it is replica based patching or not the first pod - pod := r.instantiatePodSpec(m, n, replicaPatching || !firstPod) + pod := r.instantiatePodSpec(m, n, rp, replicaPatching || !firstPod) log.Info("Creating a new "+m.Name+" POD", "POD.Namespace", pod.Namespace, "POD.Name", pod.Name) err := r.Create(ctx, pod) if err != nil { @@ -1702,11 +1840,13 @@ func (r *SingleInstanceDatabaseReconciler) createPods(m *dbapi.SingleInstanceDat return requeueN, nil } -//############################################################################# -// Create the requested POD replicas -// m = SingleInstanceDatabase -// n = CloneFromDatabase -//############################################################################# +// ############################################################################# +// +// Create the requested POD replicas +// m = SingleInstanceDatabase +// n = CloneFromDatabase +// +// ############################################################################# func (r *SingleInstanceDatabaseReconciler) deletePods(ctx context.Context, req ctrl.Request, m *dbapi.SingleInstanceDatabase, available []corev1.Pod, readyPod corev1.Pod, replicasFound int, replicasRequired int) (ctrl.Result, error) { log := r.Log.WithValues("deletePods", req.NamespacedName) @@ -1763,9 +1903,11 @@ func (r *SingleInstanceDatabaseReconciler) deletePods(ctx context.Context, req c return requeueN, nil } -//############################################################################# -// ValidateDBReadiness and return the ready POD -//############################################################################# +// ############################################################################# +// +// ValidateDBReadiness and return the ready POD +// +// ############################################################################# func (r *SingleInstanceDatabaseReconciler) validateDBReadiness(m *dbapi.SingleInstanceDatabase, ctx context.Context, req ctrl.Request) (ctrl.Result, corev1.Pod, error) { @@ -1816,11 +1958,7 @@ func (r *SingleInstanceDatabaseReconciler) validateDBReadiness(m *dbapi.SingleIn // As No pod is ready now , turn on mode when pod is ready . so requeue the request return requeueY, readyPod, errors.New("no pod is ready currently") } - if m.Status.DatafilesPatched != "true" { - eventReason := "Datapatch Pending" - eventMsg := "datapatch execution pending" - r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) - } + available = append(available, readyPod) podNamesFinal := dbcommons.GetPodNames(available) r.Log.Info("Final "+m.Name+" Pods After Deleting (or) Adding Extra Pods ( Including The Ready Pod ) ", "Pod Names", podNamesFinal) @@ -1845,13 +1983,21 @@ func (r *SingleInstanceDatabaseReconciler) validateDBReadiness(m *dbapi.SingleIn } } + if strings.ToUpper(m.Status.Role) == "PRIMARY" && m.Status.DatafilesPatched != "true" { + eventReason := "Datapatch Pending" + eventMsg := "datapatch execution pending" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + } + return requeueN, readyPod, nil } -//############################################################################# -// Function for deleting the Oracle Wallet -//############################################################################# +// ############################################################################# +// +// Function for deleting the Oracle Wallet +// +// ############################################################################# func (r *SingleInstanceDatabaseReconciler) deleteWallet(m *dbapi.SingleInstanceDatabase, ctx context.Context, req ctrl.Request) (ctrl.Result, error) { // Wallet not supported for XE Database @@ -1897,9 +2043,11 @@ func (r *SingleInstanceDatabaseReconciler) deleteWallet(m *dbapi.SingleInstanceD return requeueN, nil } -//############################################################################# -// Updating clientWallet when TCPS is enabled -//############################################################################# +// ############################################################################# +// +// Updating clientWallet when TCPS is enabled +// +// ############################################################################# func (r *SingleInstanceDatabaseReconciler) updateClientWallet(m *dbapi.SingleInstanceDatabase, readyPod corev1.Pod, ctx context.Context, req ctrl.Request) error { // Updation of tnsnames.ora in clientWallet for HOST and PORT fields @@ -1940,9 +2088,11 @@ func (r *SingleInstanceDatabaseReconciler) updateClientWallet(m *dbapi.SingleIns return nil } -//############################################################################# -// Configuring TCPS -//############################################################################# +// ############################################################################# +// +// Configuring TCPS +// +// ############################################################################# func (r *SingleInstanceDatabaseReconciler) configTcps(m *dbapi.SingleInstanceDatabase, readyPod corev1.Pod, ctx context.Context, req ctrl.Request) (ctrl.Result, error) { eventReason := "Configuring TCPS" @@ -2057,9 +2207,11 @@ func (r *SingleInstanceDatabaseReconciler) configTcps(m *dbapi.SingleInstanceDat return requeueN, nil } -//############################################################################# -// Execute Datapatch -//############################################################################# +// ############################################################################# +// +// Execute Datapatch +// +// ############################################################################# func (r *SingleInstanceDatabaseReconciler) runDatapatch(m *dbapi.SingleInstanceDatabase, readyPod corev1.Pod, ctx context.Context, req ctrl.Request) (ctrl.Result, error) { @@ -2120,9 +2272,11 @@ func (r *SingleInstanceDatabaseReconciler) runDatapatch(m *dbapi.SingleInstanceD return requeueN, nil } -//############################################################################# -// Update Init Parameters -//############################################################################# +// ############################################################################# +// +// Update Init Parameters +// +// ############################################################################# func (r *SingleInstanceDatabaseReconciler) updateInitParameters(m *dbapi.SingleInstanceDatabase, readyPod corev1.Pod, ctx context.Context, req ctrl.Request) (ctrl.Result, error) { log := r.Log.WithValues("updateInitParameters", req.NamespacedName) @@ -2170,9 +2324,11 @@ func (r *SingleInstanceDatabaseReconciler) updateInitParameters(m *dbapi.SingleI return requeueN, nil } -//############################################################################# -// Update DB config params like FLASHBACK , FORCELOGGING , ARCHIVELOG -//############################################################################# +// ############################################################################# +// +// Update DB config params like FLASHBACK , FORCELOGGING , ARCHIVELOG +// +// ############################################################################# func (r *SingleInstanceDatabaseReconciler) updateDBConfig(m *dbapi.SingleInstanceDatabase, readyPod corev1.Pod, ctx context.Context, req ctrl.Request) (ctrl.Result, error) { @@ -2370,9 +2526,11 @@ func (r *SingleInstanceDatabaseReconciler) updateDBConfig(m *dbapi.SingleInstanc return requeueN, nil } -//############################################################################# -// Update ORDS Status -//############################################################################# +// ############################################################################# +// +// Update ORDS Status +// +// ############################################################################# func (r *SingleInstanceDatabaseReconciler) updateORDSStatus(m *dbapi.SingleInstanceDatabase, ctx context.Context, req ctrl.Request) { if m.Status.OrdsReference == "" { @@ -2392,9 +2550,11 @@ func (r *SingleInstanceDatabaseReconciler) updateORDSStatus(m *dbapi.SingleInsta } } -//############################################################################# -// Manage Finalizer to cleanup before deletion of SingleInstanceDatabase -//############################################################################# +// ############################################################################# +// +// Manage Finalizer to cleanup before deletion of SingleInstanceDatabase +// +// ############################################################################# func (r *SingleInstanceDatabaseReconciler) manageSingleInstanceDatabaseDeletion(req ctrl.Request, ctx context.Context, m *dbapi.SingleInstanceDatabase) (ctrl.Result, error) { log := r.Log.WithValues("manageSingleInstanceDatabaseDeletion", req.NamespacedName) @@ -2437,9 +2597,11 @@ func (r *SingleInstanceDatabaseReconciler) manageSingleInstanceDatabaseDeletion( return requeueN, nil } -//############################################################################# -// Finalization logic for singleInstanceDatabaseFinalizer -//############################################################################# +// ############################################################################# +// +// Finalization logic for singleInstanceDatabaseFinalizer +// +// ############################################################################# func (r *SingleInstanceDatabaseReconciler) cleanupSingleInstanceDatabase(req ctrl.Request, ctx context.Context, m *dbapi.SingleInstanceDatabase) (ctrl.Result, error) { log := r.Log.WithValues("cleanupSingleInstanceDatabase", req.NamespacedName) @@ -2480,9 +2642,11 @@ func (r *SingleInstanceDatabaseReconciler) cleanupSingleInstanceDatabase(req ctr return requeueN, nil } -//############################################################################# -// SetupWithManager sets up the controller with the Manager -//############################################################################# +// ############################################################################# +// +// SetupWithManager sets up the controller with the Manager +// +// ############################################################################# func (r *SingleInstanceDatabaseReconciler) SetupWithManager(mgr ctrl.Manager) error { return ctrl.NewControllerManagedBy(mgr). For(&dbapi.SingleInstanceDatabase{}). @@ -2491,3 +2655,388 @@ func (r *SingleInstanceDatabaseReconciler) SetupWithManager(mgr ctrl.Manager) er WithOptions(controller.Options{MaxConcurrentReconciles: 100}). //ReconcileHandler is never invoked concurrently with the same object. Complete(r) } + +// StandbyDatabase functions +// Primary DB Validation +func CheckPrimaryDatabaseStatus(p *dbapi.SingleInstanceDatabase) error { + + if p.Status.Status != dbcommons.StatusReady { + return fmt.Errorf("referred primary database %v is NOT READY", p.Name) + } + return nil +} + +func CheckDatabaseRoleAsPrimary(p *dbapi.SingleInstanceDatabase) error { + + if strings.ToUpper(p.Status.Role) != "PRIMARY" { + return fmt.Errorf("referred database %v is not in PRIMARY role", p.Name) + } + return nil +} + +func GetDatabaseReadyPod(r client.Reader, d *dbapi.SingleInstanceDatabase, ctx context.Context, req ctrl.Request) (corev1.Pod, error) { + + dbReadyPod, _, _, _, err := dbcommons.FindPods(r, d.Spec.Image.Version, + d.Spec.Image.PullFrom, d.Name, d.Namespace, ctx, req) + + return dbReadyPod, err +} + +func GetDatabaseAdminPassword(r client.Reader, d *dbapi.SingleInstanceDatabase, ctx context.Context) (string, error) { + + adminPasswordSecret := &corev1.Secret{} + adminPassword := "" + err := r.Get(ctx, types.NamespacedName{Name: d.Spec.AdminPassword.SecretName, Namespace: d.Namespace}, adminPasswordSecret) + if err != nil { + return adminPassword, err + } + adminPassword = string(adminPasswordSecret.Data[d.Spec.AdminPassword.SecretKey]) + return adminPassword, nil +} + +func ValidatePrimaryDatabaseAdminPassword(r *SingleInstanceDatabaseReconciler, p *dbapi.SingleInstanceDatabase, + adminPassword string, ctx context.Context, req ctrl.Request) error { + + dbReadyPod, err := GetDatabaseReadyPod(r, p, ctx, req) + if err != nil { + return err + } + + out, err := dbcommons.ExecCommand(r, r.Config, dbReadyPod.Name, dbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | %s", fmt.Sprintf(dbcommons.ValidateAdminPassword, adminPassword), dbcommons.GetSqlClient(p.Spec.Edition))) + if err != nil { + return err + } + + if strings.Contains(out, "USER is \"SYS\"") { + r.Log.Info("validated Admin password successfully") + } else { + if strings.Contains(out, "ORA-01017") { + r.Log.Info("Invalid primary database password, Logon denied") + } + return fmt.Errorf("primary database admin password validation failed") + } + + return nil +} + +func ValidateDatabaseConfiguration(p *dbapi.SingleInstanceDatabase) error { + if p.Status.ArchiveLog == "false" || p.Status.FlashBack == "false" || p.Status.ForceLogging == "false" { + return fmt.Errorf("all of Archivelog, Flashback and ForceLogging modes are not enabled in the primary database %v", p.Name) + } + return nil +} + +func ValidatePrimaryDatabaseForStandbyCreation(r *SingleInstanceDatabaseReconciler, stdby *dbapi.SingleInstanceDatabase, + primary *dbapi.SingleInstanceDatabase, ctx context.Context, req ctrl.Request) error { + + log := r.Log.WithValues("ValidatePrimaryDatabase", req.NamespacedName) + + if stdby.Status.DatafilesCreated == "true" { + return nil + } + + log.Info(fmt.Sprintf("Checking primary database %s status...", primary.Name)) + err := CheckPrimaryDatabaseStatus(primary) + if err != nil { + stdby.Status.Status = dbcommons.StatusPending + return err + } + + log.Info("Checking for referred database role...") + err = CheckDatabaseRoleAsPrimary(primary) + if err != nil { + stdby.Status.Status = dbcommons.StatusError + return err + } + + r.Recorder.Eventf(stdby, corev1.EventTypeNormal, "Validation", "Primary database is ready") + + adminPassword, err := GetDatabaseAdminPassword(r, stdby, ctx) + if err != nil { + stdby.Status.Status = dbcommons.StatusError + return err + } + + log.Info(fmt.Sprintf("Validating admin password for the primary Database %s...", primary.Name)) + err = ValidatePrimaryDatabaseAdminPassword(r, primary, adminPassword, ctx, req) + if err != nil { + stdby.Status.Status = dbcommons.StatusError + return err + } + + log.Info(fmt.Sprintf("Validating primary database %s configuration...", primary.Name)) + err = ValidateDatabaseConfiguration(primary) + if err != nil { + stdby.Status.Status = dbcommons.StatusError + return err + } + + r.Recorder.Eventf(stdby, corev1.EventTypeNormal, "Validation", "Successfully validated the primary database admin password and configuration") + + return nil +} + +// Primary DB Setup +func GetTotalDatabasePods(r client.Reader, d *dbapi.SingleInstanceDatabase, ctx context.Context, req ctrl.Request) (int, error) { + _, totalPods, _, _, err := dbcommons.FindPods(r, d.Spec.Image.Version, + d.Spec.Image.PullFrom, d.Name, d.Namespace, ctx, req) + + return totalPods, err +} + +func SetupTnsNamesPrimaryForDG(r *SingleInstanceDatabaseReconciler, p *dbapi.SingleInstanceDatabase, s *dbapi.SingleInstanceDatabase, + primaryReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) error { + + out, err := dbcommons.ExecCommand(r, r.Config, primaryReadyPod.Name, primaryReadyPod.Namespace, "", + ctx, req, false, "bash", "-c", fmt.Sprintf("cat /opt/oracle/oradata/dbconfig/%s/tnsnames.ora", strings.ToUpper(p.Spec.Sid))) + if err != nil { + return fmt.Errorf("error obtaining the contents of tnsnames.ora in the primary database %v", p.Name) + } + r.Log.Info("tnsnames.ora content is as follows:") + r.Log.Info(out) + + if strings.Contains(out, "(SERVICE_NAME = "+strings.ToUpper(s.Spec.Sid)+")") { + r.Log.Info("TNS ENTRY OF " + s.Spec.Sid + " ALREADY EXISTS ON PRIMARY Database ") + } else { + tnsnamesEntry := dbcommons.StandbyTnsnamesEntry + tnsnamesEntry = strings.ReplaceAll(tnsnamesEntry, "##STANDBYDATABASE_SID##", s.Spec.Sid) + tnsnamesEntry = strings.ReplaceAll(tnsnamesEntry, "##STANDBYDATABASE_SERVICE_EXPOSED##", s.Name) + + out, err = dbcommons.ExecCommand(r, r.Config, primaryReadyPod.Name, primaryReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | cat >> /opt/oracle/oradata/dbconfig/%s/tnsnames.ora ", tnsnamesEntry, strings.ToUpper(p.Spec.Sid))) + if err != nil { + return fmt.Errorf("unable to set tnsnames.ora in the primary database %v", p.Name) + } + r.Log.Info("Modifying tnsnames.ora Output") + r.Log.Info(out) + + } + return nil +} + +func RestartListenerInDatabase(r *SingleInstanceDatabaseReconciler, primaryReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) error { + r.Log.Info("Restarting listener in the database through pod", "primary database pod name", primaryReadyPod.Name) + out, err := dbcommons.ExecCommand(r, r.Config, primaryReadyPod.Name, primaryReadyPod.Namespace, "", + ctx, req, false, "bash", "-c", "lsnrctl stop && lsnrctl start") + if err != nil { + return fmt.Errorf("unable to restart listener in the database through pod %v", primaryReadyPod.Name) + } + r.Log.Info("Listener restart output") + r.Log.Info(out) + return nil +} + +func SetupListenerPrimaryForDG(r *SingleInstanceDatabaseReconciler, p *dbapi.SingleInstanceDatabase, s *dbapi.SingleInstanceDatabase, + primaryReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) error { + + out, err := dbcommons.ExecCommand(r, r.Config, primaryReadyPod.Name, primaryReadyPod.Namespace, "", + ctx, req, false, "bash", "-c", fmt.Sprintf("cat /opt/oracle/oradata/dbconfig/%s/listener.ora ", strings.ToUpper(p.Spec.Sid))) + if err != nil { + return fmt.Errorf("unable to obtain contents of listener.ora in primary database %v", p.Name) + } + r.Log.Info("listener.ora Output") + r.Log.Info(out) + + if strings.Contains(out, strings.ToUpper(p.Spec.Sid)+"_DGMGRL") { + r.Log.Info("LISTENER.ORA ALREADY HAS " + p.Spec.Sid + "_DGMGRL ENTRY IN SID_LIST_LISTENER ") + } else { + out, err = dbcommons.ExecCommand(r, r.Config, primaryReadyPod.Name, primaryReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | cat > /opt/oracle/oradata/dbconfig/%s/listener.ora ", dbcommons.ListenerEntry, strings.ToUpper(p.Spec.Sid))) + if err != nil { + return fmt.Errorf("unable to modify listener.ora in the primary database %v", p.Name) + } + r.Log.Info("Modifying listener.ora Output") + r.Log.Info(out) + + err = RestartListenerInDatabase(r, primaryReadyPod, ctx, req) + if err != nil { + return err + } + + } + return nil +} + +func SetupInitParamsPrimaryForDG(r *SingleInstanceDatabaseReconciler, primaryReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) error { + r.Log.Info("Running StandbyDatabasePrerequisitesSQL in the primary database") + out, err := dbcommons.ExecCommand(r, r.Config, primaryReadyPod.Name, primaryReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | sqlplus -s / as sysdba", dbcommons.StandbyDatabasePrerequisitesSQL)) + if err != nil { + return fmt.Errorf("unable to run StandbyDatabasePrerequisitesSQL in primary database") + } + r.Log.Info("StandbyDatabasePrerequisites Output") + r.Log.Info(out) + return nil +} + +func SetupPrimaryDatabase(r *SingleInstanceDatabaseReconciler, stdby *dbapi.SingleInstanceDatabase, + primary *dbapi.SingleInstanceDatabase, ctx context.Context, req ctrl.Request) error { + + log := r.Log.WithValues("SetupPrimaryDatabase", req.NamespacedName) + + totalStandbyPods, err := GetTotalDatabasePods(r, stdby, ctx, req) + if err != nil { + return err + } + // NO need to setup primary database if standby database pods are initialized + if totalStandbyPods > 0 { + return nil + } + + primaryDbReadyPod, err := GetDatabaseReadyPod(r, primary, ctx, req) + if err != nil { + return err + } + + log.Info("Setting up tnsnames.ora in primary database", "primaryDatabase", primary.Name) + err = SetupTnsNamesPrimaryForDG(r, primary, stdby, primaryDbReadyPod, ctx, req) + if err != nil { + return err + } + + log.Info("Setting up listener.ora in primary database", "primaryDatabase", primary.Name) + err = SetupListenerPrimaryForDG(r, primary, stdby, primaryDbReadyPod, ctx, req) + if err != nil { + return err + } + + log.Info("Setting up some InitParams for DG in primary database", "primaryDatabase", primary.Name) + err = SetupInitParamsPrimaryForDG(r, primaryDbReadyPod, ctx, req) + if err != nil { + return err + } + + return nil + +} + +// Standby DB Setup +func GetAllPdbInDatabase(r *SingleInstanceDatabaseReconciler, dbReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) ([]string, error) { + var pdbs []string + out, err := dbcommons.ExecCommand(r, r.Config, dbReadyPod.Name, dbReadyPod.Namespace, "", + ctx, req, false, "bash", "-c", fmt.Sprintf("echo -e \"%s\" | sqlplus -s / as sysdba", dbcommons.GetPdbsSQL)) + if err != nil { + return pdbs, err + } + r.Log.Info("GetPdbsSQL Output") + r.Log.Info(out) + + pdbs, _ = dbcommons.StringToLines(out) + return pdbs, nil +} + +func SetupTnsNamesForPDBListInDatabase(r *SingleInstanceDatabaseReconciler, d *dbapi.SingleInstanceDatabase, + dbReadyPod corev1.Pod, ctx context.Context, req ctrl.Request, pdbList []string) error { + for _, pdb := range pdbList { + if pdb == "" { + continue + } + + // Get the Tnsnames.ora entries + out, err := dbcommons.ExecCommand(r, r.Config, dbReadyPod.Name, dbReadyPod.Namespace, "", + ctx, req, false, "bash", "-c", fmt.Sprintf("cat /opt/oracle/oradata/dbconfig/%s/tnsnames.ora", strings.ToUpper(d.Spec.Sid))) + if err != nil { + return err + } + r.Log.Info("tnsnames.ora Output") + r.Log.Info(out) + + if strings.Contains(out, "(SERVICE_NAME = "+strings.ToUpper(pdb)+")") { + r.Log.Info("TNS ENTRY OF " + strings.ToUpper(pdb) + " ALREADY EXISTS ON SIDB ") + } else { + tnsnamesEntry := dbcommons.PDBTnsnamesEntry + tnsnamesEntry = strings.ReplaceAll(tnsnamesEntry, "##PDB_NAME##", strings.ToUpper(pdb)) + + // Add Tnsnames.ora For pdb on Standby Database + out, err = dbcommons.ExecCommand(r, r.Config, dbReadyPod.Name, dbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | cat >> /opt/oracle/oradata/dbconfig/%s/tnsnames.ora ", tnsnamesEntry, strings.ToUpper(d.Spec.Sid))) + if err != nil { + return err + } + r.Log.Info("Modifying tnsnames.ora for Pdb Output") + r.Log.Info(out) + + } + } + + return nil +} + +func SetupPrimaryDBTnsNamesInStandby(r *SingleInstanceDatabaseReconciler, s *dbapi.SingleInstanceDatabase, + dbReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) error { + + out, err := dbcommons.ExecCommand(r, r.Config, dbReadyPod.Name, dbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | cat >> /opt/oracle/oradata/dbconfig/%s/tnsnames.ora ", dbcommons.PrimaryTnsnamesEntry, strings.ToUpper(s.Spec.Sid))) + if err != nil { + return err + } + r.Log.Info("Modifying tnsnames.ora Output") + r.Log.Info(out) + + return nil +} + +func EnableFlashbackInDatabase(r *SingleInstanceDatabaseReconciler, dbReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) error { + out, err := dbcommons.ExecCommand(r, r.Config, dbReadyPod.Name, dbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | %s", dbcommons.FlashBackTrueSQL, dbcommons.GetSqlClient("enterprise"))) + if err != nil { + return err + } + r.Log.Info("FlashBackTrue Output") + r.Log.Info(out) + return nil +} + +func SetupStandbyDatabase(r *SingleInstanceDatabaseReconciler, stdby *dbapi.SingleInstanceDatabase, + primary *dbapi.SingleInstanceDatabase, ctx context.Context, req ctrl.Request) error { + + primaryReadyPod, err := GetDatabaseReadyPod(r, primary, ctx, req) + if err != nil { + return err + } + r.Log.Info("Primary DB Name: " + primaryReadyPod.Name) + + stdbyReadyPod, err := GetDatabaseReadyPod(r, stdby, ctx, req) + if err != nil { + return err + } + + r.Log.Info("Getting the list of all pdbs in primary database") + pdbListPrimary, err := GetAllPdbInDatabase(r, primaryReadyPod, ctx, req) + if err != nil { + return err + } + + r.Log.Info("Setting up tnsnames in standby database for the pdbs of primary database") + err = SetupTnsNamesForPDBListInDatabase(r, stdby, stdbyReadyPod, ctx, req, pdbListPrimary) + if err != nil { + return err + } + + r.Log.Info("Setting up tnsnames entry for primary database in standby database") + err = SetupPrimaryDBTnsNamesInStandby(r, stdby, stdbyReadyPod, ctx, req) + if err != nil { + return err + } + + r.Log.Info("Setting up listener in the standby database") + err = SetupListenerPrimaryForDG(r, stdby, primary, stdbyReadyPod, ctx, req) + if err != nil { + return err + } + + flashBackStatus, _, _, result := dbcommons.CheckDBConfig(stdbyReadyPod, r, r.Config, ctx, req, stdby.Spec.Edition) + if result.Requeue { + return fmt.Errorf("error in obtaining the Database Config status") + } + if !flashBackStatus { + r.Log.Info("Setting up flashback mode in the standby database") + err = EnableFlashbackInDatabase(r, stdbyReadyPod, ctx, req) + if err != nil { + return err + } + } + + return nil +} diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 33257d24..39a80373 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -16,6 +16,7 @@ Oracle Database Operator for Kubernetes (`OraOperator`) includes the Single Inst * [Configure ArchiveLog, Flashback and ForceLog](#configure-archivelog-flashback-and-forcelog) * [Change Init Parameters](#change-init-parameters) * [Clone a Database](#clone-a-database) + * [Create a Standby Database](#create-a-standby-database) * [Patch a Database](#patch-a-database) * [Delete a Database](#delete-a-database) * [Advanced Database Configurations](#advanced-database-configurations) @@ -33,6 +34,7 @@ Oracle Database Operator for Kubernetes (`OraOperator`) includes the Single Inst * [Database Actions](#database-actions) * [APEX Installation](#apex-installation) * [Delete ORDS](#delete-ords) + * [DataguardBroker Resource](#dataguardbroker-resource) * [Maintenance Operations](#maintenance-operations) * [Additional Information](#additional-information) @@ -426,6 +428,39 @@ $ kubectl apply -f singleinstancedatabase_clone.yaml **Note:** The clone database can specify a database image that is different from the source database. In such cases, cloning is supported only between databases of the same major release. +### Create a Standby Database + +#### Prerequisites +Before creating a standby, ArchiveLog, FlashBack, and ForceLog on primary Single Instance Database(`.spec.primaryDatabaseRef`) should be turned on. + +#### Template YAML +To create a standby database, edit and apply the sample yaml file [config/samples/sidb/singleinstancedatabase_standby.yaml](../../config/samples/sidb/singleinstancedatabase_standby.yaml). + +**NOTE:** +- The `adminPassword` field of the above [config/samples/sidb/singleinstancedatabase_standby.yaml](../../config/samples/sidb/singleinstancedatabase_standby.yaml) contains an admin password secret of the primary database ref for Standby Database creation. This secret gets deleted after the database pod becomes ready for security reasons. +- Mention referred primary database in `.spec.primaryDatabaseRef` in the yaml file. +- `.spec.createAsStandby` field of the yaml file should be true. +- Database configuration like `Archivelog`, `FlashBack`, `ForceLog`, `TCPS connections` are not supported for standby database. + +#### List Standby Databases + +```sh +NAME EDITION STATUS ROLE VERSION CONNECT STR TCPS CONNECT STR OEM EXPRESS URL +sidb-19 Enterprise Healthy PRIMARY 19.3.0.0.0 10.25.0.26:1521/ORCL1 Unavailable https://10.25.0.26:5500/em +stdby-1 Enterprise Healthy PHYSICAL_STANDBY 19.3.0.0.0 10.25.0.27:32392/ORCLS1 Unavailable https://10.25.0.27:30329/em + +``` + +#### Creation Status + + Creating a new standby database instance takes a while. When the 'status' status returns the response "Healthy", the Database is open for connections. + + ```sh +$ kubectl get singleinstancedatabase stdby-1 -o "jsonpath={.status.status}" + + Healthy +``` + ### Patch a Database Databases running in your cluster and managed by the Oracle Database operator can be patched or rolled back between release updates of the same major release. To patch databases, specify an image of the higher release update. To roll back databases, specify an image of the lower release update. @@ -864,6 +899,165 @@ password: `.spec.apexPassword` - You cannot delete the referred Database before deleting its ORDS resource. - APEX, if installed, also gets uninstalled from the database when ORDS gets deleted. +## DataguardBroker Resource + +The Oracle Database Operator creates the DataguardBroker kind as a custom resource that enables Oracle Database to be managed as a native Kubernetes object + +### Resource Details + +#### DataguardBroker List + +To list the DataguardBroker resources, use the following command: + +```sh + $ kubectl get dataguardbroker -o name + + dataguardbroker.database.oracle.com/dataguardbroker-sample + +``` + +#### Quick Status + +```sh + $ kubectl get dataguardbroker dataguardbroker-sample + + NAME PRIMARY STANDBYS PROTECTION MODE CONNECT STR PRIMARY DATABASE + dataguardbroker-sample ORCL ORCLS1,ORCLS2 MaxAvailability 10.0.25.85:31555/DATAGUARD sidb-19c-1 + +``` + + +#### Detailed Status + +```sh + $ kubectl describe dataguardbroker dataguardbroker-sample + + Name: dataguardbroker-sample + Namespace: default + Labels: + Annotations: + API Version: database.oracle.com/v1alpha1 + Kind: DataguardBroker + Metadata: + Creation Timestamp: 2023-01-23T04:29:04Z + Finalizers: + database.oracle.com/dataguardbrokerfinalizer + Generation: 3 + Managed Fields: + API Version: database.oracle.com/v1alpha1 + Fields Type: FieldsV1 + fieldsV1: + ... + Manager: manager + Operation: Update + Time: 2023-01-23T04:30:20Z + API Version: database.oracle.com/v1alpha1 + Fields Type: FieldsV1 + fieldsV1: + ... + Manager: kubectl-client-side-apply + Operation: Update + Time: 2023-01-23T04:44:40Z + Resource Version: 75178376 + UID: c04a3d88-2018-4f7f-b232-b74d6c3d9479 + Spec: + Admin Password: + Keep Secret: true + Secret Key: oracle_pwd + Secret Name: db-secret + Fast Start Fail Over: + Enable: true + Primary Database Ref: sidb-sample + Protection Mode: MaxAvailability + Set As Primary Database: + Standby Database Refs: + standby-sample-1 + standby-sample-2 + Status: + Cluster Connect String: dataguardbroker-sample.default:1521/DATAGUARD + External Connect String: 10.0.25.85:31167/DATAGUARD + Primary Database: OR19E3 + Standby Databases: OR19E3S1,OR19E3S2 + Events: + Type Reason Age From Message + ---- ------ ---- ---- ------- + Normal SUCCESS 42m DataguardBroker + Normal DG Configuration up to date 24m (x13 over 56m) DataguardBroker +``` + +### Template YAML + +After you have created standbys, now configure dataguard broker for standbys and primary databases by mentioning the dataguard sample yaml. + +For the use cases detailed below a sample .yaml file is available at +[config/samples/sidb/dataguardbroker.yaml](./../../config/samples/sidb/dataguardbroker.yaml) + +**Note:** The `adminPassword` field of the above `dataguardbroker.yaml` yaml contains a Admin Password secret of primary database ref for Dataguard configuration and observer creation. This secret gets deleted after the database pod becomes ready for security reasons. + + +### Setup DataguardBroker Configuration for a Single Instance Database + +Provision a new DataguardBroker custom resource for a single instance database(`.spec.primaryDatabaseRef`) by specifying appropriate values for the attributes in the example `.yaml` file, and running the following command: + +```sh +$ kubectl create -f dataguardbroker.yaml + + dataguardbroker.database.oracle.com/dataguardbroker-sample created +``` + +### DataguardBroker Status + +Configuring DataguardBroker takes a while. When the 'status' status returns the response "Healthy", the Database is open for connections. + +```sh +$ kubectl get dataguardbroker dataguardbroker-sample -o "jsonpath={.status.status}" + +Healthy +``` + +### Connection Information for Primary Database + + External and internal (running in Kubernetes pods) clients can connect to the primary database using `.status.connectString` and `.status.clusterConnectString` + respectively in the following command + + ```sh + $ kubectl get dataguardbroker dataguardbroker-sample -o "jsonpath={.status.externalConnectString}" + + 144.25.10.119:1521/DATAGUARD + ``` + +### Set any database as Primary Database (Switchover) + +Mention SID of the any databases (SID of one of `.spec.primaryDatabaseRef` , `.spec.standbyDatabaseRefs[]`) to be set primary in the `.spec.setAsPrimaryDatabase` of [dataguardbroker.yaml](./../../config/samples/sidb/dataguardbroker.yaml) and apply the yaml file. + +The database will be set to primary. Ignored if the database is already primary. + +```sh +$ kubectl apply -f dataguardbroker.yaml + + dataguardbroker.database.oracle.com/dataguardbroker-sample apply + +``` + +### Patch Attributes + +The following attributes cannot be patched post DataguardBroker resource Creation : `primaryDatabaseRef, standbyDatabaseRefs, loadBalancer`. + +```sh +$ kubectl --type=merge -p '{"spec":{"primaryDatabaseRef":"ORCL"}}' patch dataguardbroker dataguardbroker-sample + + The DataguardBroker "dataguardbroker-sample" is invalid: spec.sid: Forbidden: cannot be changed + ``` + +### Delete DataguardBroker Resource + +```sh +$ kubectl delete dataguardbroker dgbroker-sample + + dataguardbroker.database.oracle.com/dgbroker-sample deleted +``` + +**NOTE :** You can only delete DataGuard broker when role of `.spec.primaryDatabaseRef` is PRIMARY ## Maintenance Operations If you need to perform some maintenance operations (Database/ORDS) manually, then the procedure is as follows: diff --git a/main.go b/main.go index 2b09f7f0..3665faa0 100644 --- a/main.go +++ b/main.go @@ -248,6 +248,20 @@ func main() { setupLog.Error(err, "unable to create controller", "controller", "CDB") os.Exit(1) } + if err = (&databasecontroller.DataguardBrokerReconciler{ + Client: mgr.GetClient(), + Log: ctrl.Log.WithName("controllers").WithName("database").WithName("DataguardBroker"), + Scheme: mgr.GetScheme(), + Config: mgr.GetConfig(), + Recorder: mgr.GetEventRecorderFor("DataguardBroker"), + }).SetupWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create controller", "controller", "DataguardBroker") + os.Exit(1) + } + if err = (&databasev1alpha1.DataguardBroker{}).SetupWebhookWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create webhook", "webhook", "DataguardBroker") + os.Exit(1) + } // +kubebuilder:scaffold:builder // Add index for PDB CR to enable mgr to cache PDBs From 0bb532f29dce54e9285ee2c324d4a7b6596bbcdb Mon Sep 17 00:00:00 2001 From: Mohammed Yunus Date: Wed, 8 Feb 2023 14:02:47 +0000 Subject: [PATCH 531/628] Fix status output column annotation for Sid --- apis/database/v1alpha1/singleinstancedatabase_types.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/apis/database/v1alpha1/singleinstancedatabase_types.go b/apis/database/v1alpha1/singleinstancedatabase_types.go index 4c87b7b8..4a55ee5e 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_types.go +++ b/apis/database/v1alpha1/singleinstancedatabase_types.go @@ -177,7 +177,7 @@ type SingleInstanceDatabaseStatus struct { //+kubebuilder:subresource:status // +kubebuilder:subresource:scale:specpath=.spec.replicas,statuspath=.status.replicas // +kubebuilder:printcolumn:JSONPath=".status.edition",name="Edition",type="string" -// +kubebuilder:printcolumn:JSONPath=".status.sid",name="Sid",type="string,priority=1" +// +kubebuilder:printcolumn:JSONPath=".status.sid",name="Sid",type="string",priority=1 // +kubebuilder:printcolumn:JSONPath=".status.status",name="Status",type="string" // +kubebuilder:printcolumn:JSONPath=".status.role",name="Role",type="string" // +kubebuilder:printcolumn:JSONPath=".status.releaseUpdate",name="Version",type="string" From 999d92a0d44db6168d6539a14eb94c54bba471dd Mon Sep 17 00:00:00 2001 From: Abhishek Kumar Date: Mon, 13 Feb 2023 15:12:21 +0000 Subject: [PATCH 532/628] Added some fixes --- .../v1alpha1/dataguardbroker_types.go | 5 +-- .../v1alpha1/dataguardbroker_webhook.go | 5 ++- .../singleinstancedatabase_webhook.go | 6 ++++ .../v1alpha1/zz_generated.deepcopy.go | 7 +++- commons/database/utils.go | 35 +++++++++---------- .../database.oracle.com_dataguardbrokers.yaml | 4 +-- ...database.oracle.com_shardingdatabases.yaml | 15 ++++---- ...se.oracle.com_singleinstancedatabases.yaml | 10 ++++-- .../database/dataguardbroker_controller.go | 17 ++++----- .../singleinstancedatabase_controller.go | 4 +-- main.go | 9 ++--- 11 files changed, 67 insertions(+), 50 deletions(-) diff --git a/apis/database/v1alpha1/dataguardbroker_types.go b/apis/database/v1alpha1/dataguardbroker_types.go index 12c69998..0c3f2764 100644 --- a/apis/database/v1alpha1/dataguardbroker_types.go +++ b/apis/database/v1alpha1/dataguardbroker_types.go @@ -65,8 +65,9 @@ type DataguardBrokerSpec struct { // DataguardBrokerPassword defines the secret containing Password mapped to secretKey type DataguardBrokerPassword struct { SecretName string `json:"secretName,omitempty"` - SecretKey string `json:"secretKey"` - KeepSecret bool `json:"keepSecret"` + // +kubebuilder:default:="oracle_pwd" + SecretKey string `json:"secretKey,omitempty"` + KeepSecret *bool `json:"keepSecret,omitempty"` } type DataguardBrokerFastStartFailOver struct { diff --git a/apis/database/v1alpha1/dataguardbroker_webhook.go b/apis/database/v1alpha1/dataguardbroker_webhook.go index 0c838600..5a49e8ea 100644 --- a/apis/database/v1alpha1/dataguardbroker_webhook.go +++ b/apis/database/v1alpha1/dataguardbroker_webhook.go @@ -69,7 +69,10 @@ var _ webhook.Defaulter = &DataguardBroker{} func (r *DataguardBroker) Default() { dataguardbrokerlog.Info("default", "name", r.Name) - // TODO(user): fill in your defaulting logic. + if r.Spec.AdminPassword.KeepSecret == nil { + keepSecret := true + r.Spec.AdminPassword.KeepSecret = &keepSecret + } } // TODO(user): change verbs to "verbs=create;update;delete" if you want to enable deletion validation. diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index 1dc343bd..b4fc3dd0 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -105,6 +105,12 @@ func (r *SingleInstanceDatabase) Default() { r.Spec.Replicas = 1 } } + + if r.Spec.PrimaryDatabaseRef != "" && r.Spec.CreateAsStandby { + if r.Spec.Replicas == 0 { + r.Spec.Replicas = 1 + } + } } // TODO(user): change verbs to "verbs=create;update;delete" if you want to enable deletion validation. diff --git a/apis/database/v1alpha1/zz_generated.deepcopy.go b/apis/database/v1alpha1/zz_generated.deepcopy.go index 0585bc7b..7be4d181 100644 --- a/apis/database/v1alpha1/zz_generated.deepcopy.go +++ b/apis/database/v1alpha1/zz_generated.deepcopy.go @@ -957,6 +957,11 @@ func (in *DataguardBrokerList) DeepCopyObject() runtime.Object { // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *DataguardBrokerPassword) DeepCopyInto(out *DataguardBrokerPassword) { *out = *in + if in.KeepSecret != nil { + in, out := &in.KeepSecret, &out.KeepSecret + *out = new(bool) + **out = **in + } } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DataguardBrokerPassword. @@ -985,7 +990,7 @@ func (in *DataguardBrokerSpec) DeepCopyInto(out *DataguardBrokerSpec) { } } in.FastStartFailOver.DeepCopyInto(&out.FastStartFailOver) - out.AdminPassword = in.AdminPassword + in.AdminPassword.DeepCopyInto(&out.AdminPassword) } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DataguardBrokerSpec. diff --git a/commons/database/utils.go b/commons/database/utils.go index 4d42350d..9c2d8363 100644 --- a/commons/database/utils.go +++ b/commons/database/utils.go @@ -540,35 +540,32 @@ func GetNodeIp(r client.Reader, ctx context.Context, req ctrl.Request) string { log := ctrllog.FromContext(ctx).WithValues("GetNodeIp", req.NamespacedName) - readyPod, _, available, _, err := FindPods(r, "", "", req.Name, req.Namespace, ctx, req) + //new workflow + nl := &corev1.NodeList{} + err := r.List(ctx, nl) + nodeip := "" if err != nil { log.Error(err, err.Error()) - return "" - } - if readyPod.Name != "" { - available = append(available, readyPod) + return nodeip } - nodeip := "" - for _, pod := range available { - if nodeip == "" { - nodeip = pod.Status.HostIP - } - if pod.Status.HostIP < nodeip { - nodeip = pod.Status.HostIP + + for _, address := range nl.Items[0].Status.Addresses { + if address.Type == "ExternalIP" { + nodeip = address.Address + break } } - - node := &corev1.Node{} - err = r.Get(ctx, types.NamespacedName{Name: nodeip, Namespace: req.Namespace}, node) - - if err == nil { - for _, address := range node.Status.Addresses { - if address.Type == "ExternalIP" { + if nodeip == "" { + for _, address := range nl.Items[0].Status.Addresses { + if address.Type == "InternalIP" { nodeip = address.Address + break } } } + log.Info("Node IP obtained ! ", "nodeip: ", nodeip) + return nodeip } diff --git a/config/crd/bases/database.oracle.com_dataguardbrokers.yaml b/config/crd/bases/database.oracle.com_dataguardbrokers.yaml index 98e7fc1a..53aa4c3d 100644 --- a/config/crd/bases/database.oracle.com_dataguardbrokers.yaml +++ b/config/crd/bases/database.oracle.com_dataguardbrokers.yaml @@ -66,12 +66,10 @@ spec: keepSecret: type: boolean secretKey: + default: oracle_pwd type: string secretName: type: string - required: - - keepSecret - - secretKey type: object fastStartFailOver: properties: diff --git a/config/crd/bases/database.oracle.com_shardingdatabases.yaml b/config/crd/bases/database.oracle.com_shardingdatabases.yaml index 17dd4e23..47432123 100644 --- a/config/crd/bases/database.oracle.com_shardingdatabases.yaml +++ b/config/crd/bases/database.oracle.com_shardingdatabases.yaml @@ -342,13 +342,14 @@ spec: description: "Condition contains details for one aspect of the current state of this API Resource. --- This struct is intended for direct use as an array at the field path .status.conditions. For example, - type FooStatus struct{ // Represents the observations of a - foo's current state. // Known .status.conditions.type are: - \"Available\", \"Progressing\", and \"Degraded\" // +patchMergeKey=type - \ // +patchStrategy=merge // +listType=map // +listMapKey=type - \ Conditions []metav1.Condition `json:\"conditions,omitempty\" - patchStrategy:\"merge\" patchMergeKey:\"type\" protobuf:\"bytes,1,rep,name=conditions\"` - \n // other fields }" + \n \ttype FooStatus struct{ \t // Represents the observations + of a foo's current state. \t // Known .status.conditions.type + are: \"Available\", \"Progressing\", and \"Degraded\" \t // + +patchMergeKey=type \t // +patchStrategy=merge \t // +listType=map + \t // +listMapKey=type \t Conditions []metav1.Condition + `json:\"conditions,omitempty\" patchStrategy:\"merge\" patchMergeKey:\"type\" + protobuf:\"bytes,1,rep,name=conditions\"` \n \t // other fields + \t}" properties: lastTransitionTime: description: lastTransitionTime is the last time the condition diff --git a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml index 2a24a483..83f6b3cf 100644 --- a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml +++ b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml @@ -20,6 +20,10 @@ spec: - jsonPath: .status.edition name: Edition type: string + - jsonPath: .status.sid + name: Sid + priority: 1 + type: string - jsonPath: .status.status name: Status type: string @@ -32,13 +36,13 @@ spec: - jsonPath: .status.connectString name: Connect Str type: string - - jsonPath: .status.tcpsConnectString - name: TCPS Connect Str - type: string - jsonPath: .status.pdbConnectString name: Pdb Connect Str priority: 1 type: string + - jsonPath: .status.tcpsConnectString + name: TCPS Connect Str + type: string - jsonPath: .status.tcpsPdbConnectString name: TCPS Pdb Connect Str priority: 1 diff --git a/controllers/database/dataguardbroker_controller.go b/controllers/database/dataguardbroker_controller.go index bd0d2c74..76bce6b6 100644 --- a/controllers/database/dataguardbroker_controller.go +++ b/controllers/database/dataguardbroker_controller.go @@ -103,9 +103,9 @@ func (r *DataguardBrokerReconciler) Reconcile(ctx context.Context, req ctrl.Requ // Manage SingleInstanceDatabase Deletion result, err := r.manageDataguardBrokerDeletion(req, ctx, dataguardBroker) - if result.Requeue { + if result.Requeue || err != nil { r.Log.Info("Reconcile queued") - return result, nil + return result, err } // Fetch Primary Database Reference @@ -151,11 +151,6 @@ func (r *DataguardBrokerReconciler) Reconcile(ctx context.Context, req ctrl.Requ return result, nil } - // If LoadBalancer = true , ensure Connect String is updated - if dataguardBroker.Status.ExternalConnectString == dbcommons.ValueUnavailable { - return requeueY, nil - } - // Set a particular database as primary result = r.SetAsPrimaryDatabase(singleInstanceDatabase.Spec.Sid, dataguardBroker.Spec.SetAsPrimaryDatabase, dataguardBroker, singleInstanceDatabase, adminPassword, ctx, req) @@ -164,6 +159,11 @@ func (r *DataguardBrokerReconciler) Reconcile(ctx context.Context, req ctrl.Requ return result, nil } + // If LoadBalancer = true , ensure Connect String is updated + if dataguardBroker.Status.ExternalConnectString == dbcommons.ValueUnavailable { + return requeueY, nil + } + dataguardBroker.Status.Status = dbcommons.StatusReady r.Log.Info("Reconcile completed") @@ -325,7 +325,7 @@ func (r *DataguardBrokerReconciler) createSVC(ctx context.Context, req ctrl.Requ } // update service status - + log.Info("Updating the service status...") m.Status.ClusterConnectString = svc.Name + "." + svc.Namespace + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/DATAGUARD" if m.Spec.LoadBalancer { if len(svc.Status.LoadBalancer.Ingress) > 0 { @@ -341,6 +341,7 @@ func (r *DataguardBrokerReconciler) createSVC(ctx context.Context, req ctrl.Requ m.Status.ExternalConnectString = nodeip + ":" + fmt.Sprint(svc.Spec.Ports[0].NodePort) + "/DATAGUARD" } } + r.Status().Update(ctx, m) return requeueN } diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 54bde566..939c3e7a 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -380,7 +380,7 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab if m.Status.Sid != "" && !strings.EqualFold(m.Spec.Sid, m.Status.Sid) { eventMsgs = append(eventMsgs, "sid cannot be updated") } - if m.Spec.CloneFrom == "" && m.Status.Edition != "" && !strings.EqualFold(m.Status.Edition, m.Spec.Edition) { + if m.Spec.CloneFrom == "" && m.Spec.Edition != "" && m.Status.Edition != "" && !strings.EqualFold(m.Status.Edition, m.Spec.Edition) { eventMsgs = append(eventMsgs, "edition cannot be updated") } if m.Status.Charset != "" && !strings.EqualFold(m.Status.Charset, m.Spec.Charset) { @@ -2012,7 +2012,7 @@ func (r *SingleInstanceDatabaseReconciler) deleteWallet(m *dbapi.SingleInstanceD // Deleting the secret and then deleting the wallet // If the secret is not found it means that the secret and wallet both are deleted, hence no need to requeue - if !*m.Spec.AdminPassword.KeepSecret { + if m.Spec.AdminPassword.KeepSecret != nil && !*m.Spec.AdminPassword.KeepSecret { r.Log.Info("Querying the database secret ...") secret := &corev1.Secret{} err := r.Get(ctx, types.NamespacedName{Name: m.Spec.AdminPassword.SecretName, Namespace: m.Namespace}, secret) diff --git a/main.go b/main.go index 3665faa0..6dbf2305 100644 --- a/main.go +++ b/main.go @@ -222,6 +222,10 @@ func main() { setupLog.Error(err, "unable to create webhook", "webhook", "AutonomousContainerDatabase") os.Exit(1) } + if err = (&databasev1alpha1.DataguardBroker{}).SetupWebhookWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create webhook", "webhook", "DataguardBroker") + os.Exit(1) + } } // PDB Reconciler @@ -258,10 +262,7 @@ func main() { setupLog.Error(err, "unable to create controller", "controller", "DataguardBroker") os.Exit(1) } - if err = (&databasev1alpha1.DataguardBroker{}).SetupWebhookWithManager(mgr); err != nil { - setupLog.Error(err, "unable to create webhook", "webhook", "DataguardBroker") - os.Exit(1) - } + // +kubebuilder:scaffold:builder // Add index for PDB CR to enable mgr to cache PDBs From 72ad2b2bd50da939f4591190fc76edf265c5b180 Mon Sep 17 00:00:00 2001 From: Abhishek Kumar Date: Wed, 15 Feb 2023 14:04:12 +0000 Subject: [PATCH 533/628] Added some DG changes --- .../v1alpha1/dataguardbroker_types.go | 11 +- .../v1alpha1/dataguardbroker_webhook.go | 4 - .../v1alpha1/zz_generated.deepcopy.go | 21 - .../database.oracle.com_dataguardbrokers.yaml | 14 +- .../database_v1alpha1_dataguardbroker.yaml | 6 - config/samples/sidb/dataguardbroker.yaml | 14 - .../database/dataguardbroker_controller.go | 10 +- .../singleinstancedatabase_controller.go | 28 ++ docs/sidb/README.md | 377 +++++++++--------- 9 files changed, 217 insertions(+), 268 deletions(-) delete mode 100644 config/samples/database_v1alpha1_dataguardbroker.yaml diff --git a/apis/database/v1alpha1/dataguardbroker_types.go b/apis/database/v1alpha1/dataguardbroker_types.go index 0c3f2764..8f03a113 100644 --- a/apis/database/v1alpha1/dataguardbroker_types.go +++ b/apis/database/v1alpha1/dataguardbroker_types.go @@ -59,15 +59,6 @@ type DataguardBrokerSpec struct { ProtectionMode string `json:"protectionMode"` NodeSelector map[string]string `json:"nodeSelector,omitempty"` FastStartFailOver DataguardBrokerFastStartFailOver `json:"fastStartFailOver,omitempty"` - AdminPassword DataguardBrokerPassword `json:"adminPassword"` -} - -// DataguardBrokerPassword defines the secret containing Password mapped to secretKey -type DataguardBrokerPassword struct { - SecretName string `json:"secretName,omitempty"` - // +kubebuilder:default:="oracle_pwd" - SecretKey string `json:"secretKey,omitempty"` - KeepSecret *bool `json:"keepSecret,omitempty"` } type DataguardBrokerFastStartFailOver struct { @@ -102,7 +93,7 @@ type DataguardBrokerStatus struct { // +kubebuilder:printcolumn:JSONPath=".spec.protectionMode",name="Protection Mode",type="string" // +kubebuilder:printcolumn:JSONPath=".status.clusterConnectString",name="Cluster Connect Str",type="string",priority=1 // +kubebuilder:printcolumn:JSONPath=".status.externalConnectString",name="Connect Str",type="string" -// +kubebuilder:printcolumn:JSONPath=".spec.primaryDatabaseRef",name="Primary Database",type="string" +// +kubebuilder:printcolumn:JSONPath=".spec.primaryDatabaseRef",name="Primary Database",type="string", priority=1 // +kubebuilder:printcolumn:JSONPath=".status.status",name="Status",type="string" // DataguardBroker is the Schema for the dataguardbrokers API diff --git a/apis/database/v1alpha1/dataguardbroker_webhook.go b/apis/database/v1alpha1/dataguardbroker_webhook.go index 5a49e8ea..4db987d9 100644 --- a/apis/database/v1alpha1/dataguardbroker_webhook.go +++ b/apis/database/v1alpha1/dataguardbroker_webhook.go @@ -69,10 +69,6 @@ var _ webhook.Defaulter = &DataguardBroker{} func (r *DataguardBroker) Default() { dataguardbrokerlog.Info("default", "name", r.Name) - if r.Spec.AdminPassword.KeepSecret == nil { - keepSecret := true - r.Spec.AdminPassword.KeepSecret = &keepSecret - } } // TODO(user): change verbs to "verbs=create;update;delete" if you want to enable deletion validation. diff --git a/apis/database/v1alpha1/zz_generated.deepcopy.go b/apis/database/v1alpha1/zz_generated.deepcopy.go index 7be4d181..b6601b6b 100644 --- a/apis/database/v1alpha1/zz_generated.deepcopy.go +++ b/apis/database/v1alpha1/zz_generated.deepcopy.go @@ -954,26 +954,6 @@ func (in *DataguardBrokerList) DeepCopyObject() runtime.Object { return nil } -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *DataguardBrokerPassword) DeepCopyInto(out *DataguardBrokerPassword) { - *out = *in - if in.KeepSecret != nil { - in, out := &in.KeepSecret, &out.KeepSecret - *out = new(bool) - **out = **in - } -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DataguardBrokerPassword. -func (in *DataguardBrokerPassword) DeepCopy() *DataguardBrokerPassword { - if in == nil { - return nil - } - out := new(DataguardBrokerPassword) - in.DeepCopyInto(out) - return out -} - // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *DataguardBrokerSpec) DeepCopyInto(out *DataguardBrokerSpec) { *out = *in @@ -990,7 +970,6 @@ func (in *DataguardBrokerSpec) DeepCopyInto(out *DataguardBrokerSpec) { } } in.FastStartFailOver.DeepCopyInto(&out.FastStartFailOver) - in.AdminPassword.DeepCopyInto(&out.AdminPassword) } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DataguardBrokerSpec. diff --git a/config/crd/bases/database.oracle.com_dataguardbrokers.yaml b/config/crd/bases/database.oracle.com_dataguardbrokers.yaml index 53aa4c3d..b5fc87c4 100644 --- a/config/crd/bases/database.oracle.com_dataguardbrokers.yaml +++ b/config/crd/bases/database.oracle.com_dataguardbrokers.yaml @@ -35,6 +35,7 @@ spec: type: string - jsonPath: .spec.primaryDatabaseRef name: Primary Database + priority: 1 type: string - jsonPath: .status.status name: Status @@ -59,18 +60,6 @@ spec: spec: description: DataguardBrokerSpec defines the desired state of DataguardBroker properties: - adminPassword: - description: DataguardBrokerPassword defines the secret containing - Password mapped to secretKey - properties: - keepSecret: - type: boolean - secretKey: - default: oracle_pwd - type: string - secretName: - type: string - type: object fastStartFailOver: properties: enable: @@ -106,7 +95,6 @@ spec: type: string type: array required: - - adminPassword - primaryDatabaseRef - protectionMode - standbyDatabaseRefs diff --git a/config/samples/database_v1alpha1_dataguardbroker.yaml b/config/samples/database_v1alpha1_dataguardbroker.yaml deleted file mode 100644 index 8b46e319..00000000 --- a/config/samples/database_v1alpha1_dataguardbroker.yaml +++ /dev/null @@ -1,6 +0,0 @@ -apiVersion: database.oracle.com/v1alpha1 -kind: DataguardBroker -metadata: - name: dataguardbroker-sample -spec: - # TODO(user): Add fields here diff --git a/config/samples/sidb/dataguardbroker.yaml b/config/samples/sidb/dataguardbroker.yaml index 1632b01f..e95460fe 100644 --- a/config/samples/sidb/dataguardbroker.yaml +++ b/config/samples/sidb/dataguardbroker.yaml @@ -18,20 +18,6 @@ spec: # - standbyDatabase-sample1 # - standbyDatabase-sample2 - ## Secret containing databaseRef password mapped to secretKey. - ## This secret will be deleted after Dataguard setup unless keepSecret set to true. - adminPassword: - secretName: - secretKey: - keepSecret: false - - ## Deploy only on nodes having required labels . Format label_name : label_value . - ## Leave empty if there is no such requirement . - ## Uncomment to use - # nodeSelector: - # failure-domain.beta.kubernetes.io/zone: bVCG:PHX-AD-1 - # pool: sidb - ## Type of service . Applicable on cloud enviroments only ## if loadBalService : false , service type = "NodePort" . else "LoadBalancer" loadBalancer: false diff --git a/controllers/database/dataguardbroker_controller.go b/controllers/database/dataguardbroker_controller.go index 76bce6b6..7eeb3dc1 100644 --- a/controllers/database/dataguardbroker_controller.go +++ b/controllers/database/dataguardbroker_controller.go @@ -199,20 +199,20 @@ func (r *DataguardBrokerReconciler) validateSidbReadiness(m *dbapi.DataguardBrok // Validate databaseRef Admin Password adminPasswordSecret := &corev1.Secret{} - err = r.Get(ctx, types.NamespacedName{Name: m.Spec.AdminPassword.SecretName, Namespace: m.Namespace}, adminPasswordSecret) + err = r.Get(ctx, types.NamespacedName{Name: n.Spec.AdminPassword.SecretName, Namespace: n.Namespace}, adminPasswordSecret) if err != nil { if apierrors.IsNotFound(err) { //m.Status.Status = dbcommons.StatusError eventReason := "Waiting" - eventMsg := "waiting for secret : " + m.Spec.AdminPassword.SecretName + " to get created" + eventMsg := "waiting for secret : " + n.Spec.AdminPassword.SecretName + " to get created" r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) - r.Log.Info("Secret " + m.Spec.AdminPassword.SecretName + " Not Found") + r.Log.Info("Secret " + n.Spec.AdminPassword.SecretName + " Not Found") return requeueY, sidbReadyPod, adminPassword } log.Error(err, err.Error()) return requeueY, sidbReadyPod, adminPassword } - adminPassword = string(adminPasswordSecret.Data[m.Spec.AdminPassword.SecretKey]) + adminPassword = string(adminPasswordSecret.Data[n.Spec.AdminPassword.SecretKey]) out, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", fmt.Sprintf("echo -e \"%s\" | %s", fmt.Sprintf(dbcommons.ValidateAdminPassword, adminPassword), dbcommons.GetSqlClient(n.Spec.Edition))) @@ -225,7 +225,7 @@ func (r *DataguardBrokerReconciler) validateSidbReadiness(m *dbapi.DataguardBrok } else if strings.Contains(out, "ORA-01017") { //m.Status.Status = dbcommons.StatusError eventReason := "Logon denied" - eventMsg := "invalid databaseRef admin password. secret: " + m.Spec.AdminPassword.SecretName + eventMsg := "invalid databaseRef admin password. secret: " + n.Spec.AdminPassword.SecretName r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) return requeueY, sidbReadyPod, adminPassword } else { diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 939c3e7a..368b4cc3 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -994,6 +994,34 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, } + // Adding pod anti-affinity for standby cases + if m.Spec.PrimaryDatabaseRef != "" && m.Spec.CreateAsStandby { + weightedPodAffinityTerm := corev1.WeightedPodAffinityTerm{ + Weight: 100, + PodAffinityTerm: corev1.PodAffinityTerm{ + LabelSelector: &metav1.LabelSelector{ + MatchExpressions: []metav1.LabelSelectorRequirement{{ + Key: "app", + Operator: metav1.LabelSelectorOpIn, + Values: []string{rp.Name}, + }}, + }, + TopologyKey: "kubernetes.io/hostname", + }, + } + if m.Spec.Persistence.AccessMode == "ReadWriteOnce" { + pod.Spec.Affinity.PodAntiAffinity = &corev1.PodAntiAffinity{ + PreferredDuringSchedulingIgnoredDuringExecution: []corev1.WeightedPodAffinityTerm{ + weightedPodAffinityTerm, + }, + } + } else { + pod.Spec.Affinity.PodAntiAffinity.PreferredDuringSchedulingIgnoredDuringExecution = + append(pod.Spec.Affinity.PodAntiAffinity.PreferredDuringSchedulingIgnoredDuringExecution, weightedPodAffinityTerm) + } + + } + // Set SingleInstanceDatabase instance as the owner and controller ctrl.SetControllerReference(m, pod, r.Scheme) return pod diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 39a80373..2b91132b 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -16,7 +16,6 @@ Oracle Database Operator for Kubernetes (`OraOperator`) includes the Single Inst * [Configure ArchiveLog, Flashback and ForceLog](#configure-archivelog-flashback-and-forcelog) * [Change Init Parameters](#change-init-parameters) * [Clone a Database](#clone-a-database) - * [Create a Standby Database](#create-a-standby-database) * [Patch a Database](#patch-a-database) * [Delete a Database](#delete-a-database) * [Advanced Database Configurations](#advanced-database-configurations) @@ -24,6 +23,9 @@ Oracle Database Operator for Kubernetes (`OraOperator`) includes the Single Inst * [Setup Database with LoadBalancer](#setup-database-with-loadbalancer) * [Enabling TCPS Connections](#enabling-tcps-connections) * [Specifying Custom Ports](#specifying-custom-ports) + * [Creating an Oracle Data Guard Configuration](#creating-an-oracle-data-guard-configuration) + * [Creating a Standby Database](#creating-a-standby-database) + * [Setup DataGuardBroker Configuration for a Single Instance Database](#setup-dataguardbroker-configuration-for-a-single-instance-database) * [OracleRestDataService Resource](#oraclerestdataservice-resource) * [REST Enable a Database](#rest-enable-a-database) * [Provision ORDS](#provision-ords) @@ -34,7 +36,6 @@ Oracle Database Operator for Kubernetes (`OraOperator`) includes the Single Inst * [Database Actions](#database-actions) * [APEX Installation](#apex-installation) * [Delete ORDS](#delete-ords) - * [DataguardBroker Resource](#dataguardbroker-resource) * [Maintenance Operations](#maintenance-operations) * [Additional Information](#additional-information) @@ -427,39 +428,6 @@ $ kubectl apply -f singleinstancedatabase_clone.yaml ``` **Note:** The clone database can specify a database image that is different from the source database. In such cases, cloning is supported only between databases of the same major release. - -### Create a Standby Database - -#### Prerequisites -Before creating a standby, ArchiveLog, FlashBack, and ForceLog on primary Single Instance Database(`.spec.primaryDatabaseRef`) should be turned on. - -#### Template YAML -To create a standby database, edit and apply the sample yaml file [config/samples/sidb/singleinstancedatabase_standby.yaml](../../config/samples/sidb/singleinstancedatabase_standby.yaml). - -**NOTE:** -- The `adminPassword` field of the above [config/samples/sidb/singleinstancedatabase_standby.yaml](../../config/samples/sidb/singleinstancedatabase_standby.yaml) contains an admin password secret of the primary database ref for Standby Database creation. This secret gets deleted after the database pod becomes ready for security reasons. -- Mention referred primary database in `.spec.primaryDatabaseRef` in the yaml file. -- `.spec.createAsStandby` field of the yaml file should be true. -- Database configuration like `Archivelog`, `FlashBack`, `ForceLog`, `TCPS connections` are not supported for standby database. - -#### List Standby Databases - -```sh -NAME EDITION STATUS ROLE VERSION CONNECT STR TCPS CONNECT STR OEM EXPRESS URL -sidb-19 Enterprise Healthy PRIMARY 19.3.0.0.0 10.25.0.26:1521/ORCL1 Unavailable https://10.25.0.26:5500/em -stdby-1 Enterprise Healthy PHYSICAL_STANDBY 19.3.0.0.0 10.25.0.27:32392/ORCLS1 Unavailable https://10.25.0.27:30329/em - -``` - -#### Creation Status - - Creating a new standby database instance takes a while. When the 'status' status returns the response "Healthy", the Database is open for connections. - - ```sh -$ kubectl get singleinstancedatabase stdby-1 -o "jsonpath={.status.status}" - - Healthy -``` ### Patch a Database @@ -600,6 +568,185 @@ In case of `NodePort` service, `listenerPort`, and `tcpsListenerPort` will be th - If TCPS connections are enabled, and `listenerPort` is commented/removed in the [config/samples/sidb/singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml) file, only TCPS endpoint will be exposed. - If LoadBalancer is enabled, and either `listenerPort` or `tcpsListenerPort` is changed, then it takes some time to complete the work requests (drain existing backend sets and create new ones). In this time, the database connectivity is broken. Although, SingleInstanceDatabase and LoadBalancer remain in the healthy state, you can check the progress of the work requests by logging into the cloud provider's console and checking the corresponding LoadBalancer. +### Creating an Oracle Data Guard Configuration + +### Creating a Standby Database + +#### Prerequisites +Before creating a standby, ArchiveLog, FlashBack, and ForceLog on primary Single Instance Database(`.spec.primaryDatabaseRef`) should be turned on. + +#### Template YAML +To create a standby database, edit and apply the sample yaml file [config/samples/sidb/singleinstancedatabase_standby.yaml](../../config/samples/sidb/singleinstancedatabase_standby.yaml). + +**NOTE:** +- The `adminPassword` field of the above [config/samples/sidb/singleinstancedatabase_standby.yaml](../../config/samples/sidb/singleinstancedatabase_standby.yaml) contains an admin password secret of the primary database ref for Standby Database creation. This secret gets deleted after the database pod becomes ready for security reasons. +- Mention referred primary database in `.spec.primaryDatabaseRef` in the yaml file. +- `.spec.createAsStandby` field of the yaml file should be true. +- Database configuration like `Archivelog`, `FlashBack`, `ForceLog`, `TCPS connections` are not supported for standby database. + +#### List Standby Databases + +```sh +NAME EDITION STATUS ROLE VERSION CONNECT STR TCPS CONNECT STR OEM EXPRESS URL +sidb-19 Enterprise Healthy PRIMARY 19.3.0.0.0 10.25.0.26:1521/ORCL1 Unavailable https://10.25.0.26:5500/em +stdby-1 Enterprise Healthy PHYSICAL_STANDBY 19.3.0.0.0 10.25.0.27:32392/ORCLS1 Unavailable https://10.25.0.27:30329/em + +``` + +#### Creation Status + + Creating a new standby database instance takes a while. When the 'status' status returns the response "Healthy", the Database is open for connections. + + ```sh +$ kubectl get singleinstancedatabase stdby-1 -o "jsonpath={.status.status}" + + Healthy +``` + +### Setup DataGuardBroker Configuration for a Single Instance Database + +#### Template YAML +After you have created standbys, now configure dataguard broker for standbys and primary databases by mentioning the dataguard sample yaml. + +For the use cases detailed below a sample .yaml file is available at +[config/samples/sidb/dataguardbroker.yaml](./../../config/samples/sidb/dataguardbroker.yaml) + +**Note:** The `adminPassword` field of the above `dataguardbroker.yaml` yaml contains a Admin Password secret of primary database ref for Dataguard configuration and observer creation. This secret gets deleted after the database pod becomes ready for security reasons. + +#### Setup DataGuardBroker Resource + +Provision a new DataguardBroker custom resource for a single instance database(`.spec.primaryDatabaseRef`) by specifying appropriate values for the attributes in the example `.yaml` file, and running the following command: + +```sh +$ kubectl create -f dataguardbroker.yaml + + dataguardbroker.database.oracle.com/dataguardbroker-sample created +``` + +#### DataguardBroker List + +To list the DataguardBroker resources, use the following command: + +```sh + $ kubectl get dataguardbroker -o name + + dataguardbroker.database.oracle.com/dataguardbroker-sample + +``` + +#### Quick Status + +```sh + $ kubectl get dataguardbroker dataguardbroker-sample + + NAME PRIMARY STANDBYS PROTECTION MODE CONNECT STR STATUS + dataguardbroker-sample ORCL ORCLS1,ORCLS2 MaxAvailability 10.0.25.85:31555/DATAGUARD Healthy + +``` + + +#### Detailed Status + +```sh + $ kubectl describe dataguardbroker dataguardbroker-sample + + Name: dataguardbroker-sample + Namespace: default + Labels: + Annotations: + API Version: database.oracle.com/v1alpha1 + Kind: DataguardBroker + Metadata: + Creation Timestamp: 2023-01-23T04:29:04Z + Finalizers: + database.oracle.com/dataguardbrokerfinalizer + Generation: 3 + Managed Fields: + API Version: database.oracle.com/v1alpha1 + Fields Type: FieldsV1 + fieldsV1: + ... + Manager: manager + Operation: Update + Time: 2023-01-23T04:30:20Z + API Version: database.oracle.com/v1alpha1 + Fields Type: FieldsV1 + fieldsV1: + ... + Manager: kubectl-client-side-apply + Operation: Update + Time: 2023-01-23T04:44:40Z + Resource Version: 75178376 + UID: c04a3d88-2018-4f7f-b232-b74d6c3d9479 + Spec: + Admin Password: + Keep Secret: true + Secret Key: oracle_pwd + Secret Name: db-secret + Fast Start Fail Over: + Enable: true + Primary Database Ref: sidb-sample + Protection Mode: MaxAvailability + Set As Primary Database: + Standby Database Refs: + standby-sample-1 + standby-sample-2 + Status: + Cluster Connect String: dataguardbroker-sample.default:1521/DATAGUARD + External Connect String: 10.0.25.85:31167/DATAGUARD + Primary Database: OR19E3 + Standby Databases: OR19E3S1,OR19E3S2 + Status: Healthy + Events: + Type Reason Age From Message + ---- ------ ---- ---- ------- + Normal SUCCESS 42m DataguardBroker + Normal DG Configuration up to date 24m (x13 over 56m) DataguardBroker +``` + +#### Connection Information for Primary Database + + External and internal (running in Kubernetes pods) clients can connect to the primary database using `.status.connectString` and `.status.clusterConnectString` + respectively in the following command + + ```sh + $ kubectl get dataguardbroker dataguardbroker-sample -o "jsonpath={.status.externalConnectString}" + + 10.0.25.87:1521/DATAGUARD + ``` + +#### Set any database as Primary Database (Switchover) + +Mention SID of the any databases (SID of one of `.spec.primaryDatabaseRef` , `.spec.standbyDatabaseRefs[]`) to be set primary in the `.spec.setAsPrimaryDatabase` of [dataguardbroker.yaml](./../../config/samples/sidb/dataguardbroker.yaml) and apply the yaml file. + +The database will be set to primary. Ignored if the database is already primary. + +```sh +$ kubectl apply -f dataguardbroker.yaml + + dataguardbroker.database.oracle.com/dataguardbroker-sample apply + +``` + +#### Patching DataGuardBroker Attributes + +The following attributes cannot be patched post DataguardBroker resource Creation : `primaryDatabaseRef, standbyDatabaseRefs, loadBalancer`. + +```sh +$ kubectl --type=merge -p '{"spec":{"primaryDatabaseRef":"ORCL"}}' patch dataguardbroker dataguardbroker-sample + + The DataguardBroker "dataguardbroker-sample" is invalid: spec.sid: Forbidden: cannot be changed + ``` + +#### Delete DataguardBroker Resource + +```sh +$ kubectl delete dataguardbroker dgbroker-sample + + dataguardbroker.database.oracle.com/dgbroker-sample deleted +``` + +**NOTE :** You can only delete DataGuard broker when role of `.spec.primaryDatabaseRef` is PRIMARY ## OracleRestDataService Resource @@ -899,166 +1046,6 @@ password: `.spec.apexPassword` - You cannot delete the referred Database before deleting its ORDS resource. - APEX, if installed, also gets uninstalled from the database when ORDS gets deleted. -## DataguardBroker Resource - -The Oracle Database Operator creates the DataguardBroker kind as a custom resource that enables Oracle Database to be managed as a native Kubernetes object - -### Resource Details - -#### DataguardBroker List - -To list the DataguardBroker resources, use the following command: - -```sh - $ kubectl get dataguardbroker -o name - - dataguardbroker.database.oracle.com/dataguardbroker-sample - -``` - -#### Quick Status - -```sh - $ kubectl get dataguardbroker dataguardbroker-sample - - NAME PRIMARY STANDBYS PROTECTION MODE CONNECT STR PRIMARY DATABASE - dataguardbroker-sample ORCL ORCLS1,ORCLS2 MaxAvailability 10.0.25.85:31555/DATAGUARD sidb-19c-1 - -``` - - -#### Detailed Status - -```sh - $ kubectl describe dataguardbroker dataguardbroker-sample - - Name: dataguardbroker-sample - Namespace: default - Labels: - Annotations: - API Version: database.oracle.com/v1alpha1 - Kind: DataguardBroker - Metadata: - Creation Timestamp: 2023-01-23T04:29:04Z - Finalizers: - database.oracle.com/dataguardbrokerfinalizer - Generation: 3 - Managed Fields: - API Version: database.oracle.com/v1alpha1 - Fields Type: FieldsV1 - fieldsV1: - ... - Manager: manager - Operation: Update - Time: 2023-01-23T04:30:20Z - API Version: database.oracle.com/v1alpha1 - Fields Type: FieldsV1 - fieldsV1: - ... - Manager: kubectl-client-side-apply - Operation: Update - Time: 2023-01-23T04:44:40Z - Resource Version: 75178376 - UID: c04a3d88-2018-4f7f-b232-b74d6c3d9479 - Spec: - Admin Password: - Keep Secret: true - Secret Key: oracle_pwd - Secret Name: db-secret - Fast Start Fail Over: - Enable: true - Primary Database Ref: sidb-sample - Protection Mode: MaxAvailability - Set As Primary Database: - Standby Database Refs: - standby-sample-1 - standby-sample-2 - Status: - Cluster Connect String: dataguardbroker-sample.default:1521/DATAGUARD - External Connect String: 10.0.25.85:31167/DATAGUARD - Primary Database: OR19E3 - Standby Databases: OR19E3S1,OR19E3S2 - Events: - Type Reason Age From Message - ---- ------ ---- ---- ------- - Normal SUCCESS 42m DataguardBroker - Normal DG Configuration up to date 24m (x13 over 56m) DataguardBroker -``` - -### Template YAML - -After you have created standbys, now configure dataguard broker for standbys and primary databases by mentioning the dataguard sample yaml. - -For the use cases detailed below a sample .yaml file is available at -[config/samples/sidb/dataguardbroker.yaml](./../../config/samples/sidb/dataguardbroker.yaml) - -**Note:** The `adminPassword` field of the above `dataguardbroker.yaml` yaml contains a Admin Password secret of primary database ref for Dataguard configuration and observer creation. This secret gets deleted after the database pod becomes ready for security reasons. - - -### Setup DataguardBroker Configuration for a Single Instance Database - -Provision a new DataguardBroker custom resource for a single instance database(`.spec.primaryDatabaseRef`) by specifying appropriate values for the attributes in the example `.yaml` file, and running the following command: - -```sh -$ kubectl create -f dataguardbroker.yaml - - dataguardbroker.database.oracle.com/dataguardbroker-sample created -``` - -### DataguardBroker Status - -Configuring DataguardBroker takes a while. When the 'status' status returns the response "Healthy", the Database is open for connections. - -```sh -$ kubectl get dataguardbroker dataguardbroker-sample -o "jsonpath={.status.status}" - -Healthy -``` - -### Connection Information for Primary Database - - External and internal (running in Kubernetes pods) clients can connect to the primary database using `.status.connectString` and `.status.clusterConnectString` - respectively in the following command - - ```sh - $ kubectl get dataguardbroker dataguardbroker-sample -o "jsonpath={.status.externalConnectString}" - - 144.25.10.119:1521/DATAGUARD - ``` - -### Set any database as Primary Database (Switchover) - -Mention SID of the any databases (SID of one of `.spec.primaryDatabaseRef` , `.spec.standbyDatabaseRefs[]`) to be set primary in the `.spec.setAsPrimaryDatabase` of [dataguardbroker.yaml](./../../config/samples/sidb/dataguardbroker.yaml) and apply the yaml file. - -The database will be set to primary. Ignored if the database is already primary. - -```sh -$ kubectl apply -f dataguardbroker.yaml - - dataguardbroker.database.oracle.com/dataguardbroker-sample apply - -``` - -### Patch Attributes - -The following attributes cannot be patched post DataguardBroker resource Creation : `primaryDatabaseRef, standbyDatabaseRefs, loadBalancer`. - -```sh -$ kubectl --type=merge -p '{"spec":{"primaryDatabaseRef":"ORCL"}}' patch dataguardbroker dataguardbroker-sample - - The DataguardBroker "dataguardbroker-sample" is invalid: spec.sid: Forbidden: cannot be changed - ``` - -### Delete DataguardBroker Resource - -```sh -$ kubectl delete dataguardbroker dgbroker-sample - - dataguardbroker.database.oracle.com/dgbroker-sample deleted -``` - -**NOTE :** You can only delete DataGuard broker when role of `.spec.primaryDatabaseRef` is PRIMARY - ## Maintenance Operations If you need to perform some maintenance operations (Database/ORDS) manually, then the procedure is as follows: 1. Use `kubectl exec` to access the pod where you want to perform the manual operation, a command similar to the following: From 89a15ffab4772156705db6a3fe74c922b6251355 Mon Sep 17 00:00:00 2001 From: Abhishek Kumar Date: Fri, 17 Feb 2023 10:19:14 +0000 Subject: [PATCH 534/628] Readme changes, and standby SID validation --- .../singleinstancedatabase_controller.go | 6 ++++++ docs/sidb/README.md | 16 ++++++++++++---- 2 files changed, 18 insertions(+), 4 deletions(-) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 368b4cc3..3db50f99 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -493,6 +493,12 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab return requeueY, err } + if m.Spec.Sid == rp.Spec.Sid { + r.Log.Info("Standby database SID can not be same as the Primary database SID") + r.Recorder.Eventf(m, corev1.EventTypeWarning, "Spec Error", "Standby and Primary database SID can not be same") + return requeueY, err + } + if m.Status.DatafilesCreated == "true" || !dbcommons.IsSourceDatabaseOnCluster(m.Spec.PrimaryDatabaseRef) { return requeueN, nil diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 2b91132b..33013dfd 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -587,10 +587,20 @@ To create a standby database, edit and apply the sample yaml file [config/sample #### List Standby Databases ```sh +kubectl get singleinstancedatabase + NAME EDITION STATUS ROLE VERSION CONNECT STR TCPS CONNECT STR OEM EXPRESS URL -sidb-19 Enterprise Healthy PRIMARY 19.3.0.0.0 10.25.0.26:1521/ORCL1 Unavailable https://10.25.0.26:5500/em -stdby-1 Enterprise Healthy PHYSICAL_STANDBY 19.3.0.0.0 10.25.0.27:32392/ORCLS1 Unavailable https://10.25.0.27:30329/em +sidb-19 Enterprise Healthy PRIMARY 19.3.0.0.0 10.25.0.26:1521/ORCL1 Unavailable https://10.25.0.26:5500/em +stdby-1 Enterprise Healthy PHYSICAL_STANDBY 19.3.0.0.0 10.25.0.27:32392/ORCLS1 Unavailable https://10.25.0.27:30329/em + +``` +### Query Primary Database Reference +You can query the corresponding primary database for every standby database. + +```sh +kubectl get singleinstancedatabase stdby-1 -o "jsonpath={.status.primaryDatabase}" +sidb-19 ``` #### Creation Status @@ -611,8 +621,6 @@ After you have created standbys, now configure dataguard broker for standbys and For the use cases detailed below a sample .yaml file is available at [config/samples/sidb/dataguardbroker.yaml](./../../config/samples/sidb/dataguardbroker.yaml) -**Note:** The `adminPassword` field of the above `dataguardbroker.yaml` yaml contains a Admin Password secret of primary database ref for Dataguard configuration and observer creation. This secret gets deleted after the database pod becomes ready for security reasons. - #### Setup DataGuardBroker Resource Provision a new DataguardBroker custom resource for a single instance database(`.spec.primaryDatabaseRef`) by specifying appropriate values for the attributes in the example `.yaml` file, and running the following command: From 7ddc1788e50fdeda588e3da26dc45810411f6b64 Mon Sep 17 00:00:00 2001 From: Oleksandra Pavlusieva Date: Mon, 20 Feb 2023 16:53:14 +0200 Subject: [PATCH 535/628] Update SECURITY.md --- SECURITY.md | 34 +++++++++++++++++++--------------- 1 file changed, 19 insertions(+), 15 deletions(-) diff --git a/SECURITY.md b/SECURITY.md index 5acefa28..fb238413 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -1,32 +1,36 @@ -# Reporting Security Vulnerabilities +# Reporting security vulnerabilities -Oracle values the independent security research community, and believes that -responsible disclosure of security vulnerabilities helps us to ensure the security -and privacy of all of our users. +Oracle values the independent security research community and believes that +responsible disclosure of security vulnerabilities helps us ensure the security +and privacy of all our users. Please do NOT raise a GitHub Issue to report a security vulnerability. If you -believe you have found a security vulnerability, then please submit a report to +believe you have found a security vulnerability, please submit a report to [secalert_us@oracle.com][1] preferably with a proof of concept. Please review some additional information on [how to report security vulnerabilities to Oracle][2]. -Oracle encourages anyone who contacts Oracle Security to use email encryption, using +We encourage people who contact Oracle Security to use email encryption using [our encryption key][3]. -Please do not use other channels, or contact the project maintainers +We ask that you do not use other channels or contact the project maintainers directly. -For non-vulnerability related security issues, including ideas for new or improved -security features, you are welcome to post these as GitHub Issues. +Non-vulnerability related security issues including ideas for new or improved +security features are welcome on GitHub Issues. -## Security Updates, Alerts and Bulletins +## Security updates, alerts and bulletins -Oracle issues security updates on a regular cadence. Many of our projects typically include release security fixes in conjunction with the [Oracle Critical Patch Update][3] program. Security updates are released on the -Tuesday closest to the 17th day of January, April, July and October. A pre-release announcement will be published on the Thursday preceding each release. Additional information, including past advisories, is available on our [security alerts][4] +Security updates will be released on a regular cadence. Many of our projects +will typically release security fixes in conjunction with the +[Oracle Critical Patch Update][3] program. Additional +information, including past advisories, is available on our [security alerts][4] page. -## Security-Related Information +## Security-related information -Oracle will provide security-related information in our documentation. The information can be a threat model, best practices for secure use, or any known security issues. Please note -that labs and example code are intended to demonstrate a concept. These examples should not be used for production use without ensuring that the code is hardened, and in compliance with common security practices. +We will provide security related information such as a threat model, considerations +for secure use, or any known security issues in our documentation. Please note +that labs and sample code are intended to demonstrate a concept and may not be +sufficiently hardened for production use. [1]: mailto:secalert_us@oracle.com [2]: https://www.oracle.com/corporate/security-practices/assurance/vulnerability/reporting.html From c1768f94175ac8c764b0ba33bb46ea551f99380d Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Thu, 23 Feb 2023 09:59:58 +0000 Subject: [PATCH 536/628] Bump golang.org/x/text from 0.3.7 to 0.3.8 Bumps [golang.org/x/text](https://github.com/golang/text) from 0.3.7 to 0.3.8. - [Release notes](https://github.com/golang/text/releases) - [Commits](https://github.com/golang/text/compare/v0.3.7...v0.3.8) --- updated-dependencies: - dependency-name: golang.org/x/text dependency-type: indirect ... Signed-off-by: dependabot[bot] --- go.mod | 6 +++--- go.sum | 14 ++++++++++++-- 2 files changed, 15 insertions(+), 5 deletions(-) diff --git a/go.mod b/go.mod index 4e950301..e73b8fbd 100644 --- a/go.mod +++ b/go.mod @@ -7,6 +7,7 @@ require ( github.com/onsi/ginkgo/v2 v2.1.3 github.com/onsi/gomega v1.19.0 github.com/oracle/oci-go-sdk/v64 v64.0.0 + go.uber.org/zap v1.21.0 gopkg.in/yaml.v2 v2.4.0 k8s.io/api v0.23.6 k8s.io/apimachinery v0.23.6 @@ -46,12 +47,11 @@ require ( github.com/spf13/pflag v1.0.5 // indirect go.uber.org/atomic v1.9.0 // indirect go.uber.org/multierr v1.7.0 // indirect - go.uber.org/zap v1.21.0 // indirect golang.org/x/net v0.0.0-20220225172249-27dd8689420f // indirect golang.org/x/oauth2 v0.0.0-20211104180415-d3ed0bb246c8 // indirect - golang.org/x/sys v0.0.0-20220209214540-3681064d5158 // indirect + golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f // indirect golang.org/x/term v0.0.0-20210927222741-03fcf44c2211 // indirect - golang.org/x/text v0.3.7 // indirect + golang.org/x/text v0.3.8 // indirect golang.org/x/time v0.0.0-20211116232009-f0f3c7e86c11 // indirect gomodules.xyz/jsonpatch/v2 v2.2.0 // indirect google.golang.org/appengine v1.6.7 // indirect diff --git a/go.sum b/go.sum index 62d4fc56..630b64c4 100644 --- a/go.sum +++ b/go.sum @@ -493,6 +493,7 @@ github.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9de github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.3.5/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k= github.com/yuin/goldmark v1.4.0/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k= +github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY= go.etcd.io/bbolt v1.3.2/go.mod h1:IbVyRI1SCnLcuJnV2u8VeU0CEYM7e686BmAb1XKL+uU= go.etcd.io/bbolt v1.3.6/go.mod h1:qXsaaIqmgQH0T+OPdb99Bf+PKfBBQVAdyD6TY9G8XM4= go.etcd.io/etcd/api/v3 v3.5.0/go.mod h1:cbVKeC6lCfl7j/8jBhAK6aIYO9XOjdptoxU/nLQcPvs= @@ -550,6 +551,7 @@ golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8U golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= golang.org/x/crypto v0.0.0-20201002170205-7f63de1d35b0/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= golang.org/x/crypto v0.0.0-20210817164053-32db794688a5/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc= +golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc= golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8= @@ -585,6 +587,7 @@ golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.4.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.4.1/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= +golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4= golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= @@ -636,6 +639,8 @@ golang.org/x/net v0.0.0-20210825183410-e898025ed96a/go.mod h1:9nx3DQGgdP8bBQD5qx golang.org/x/net v0.0.0-20211209124913-491a49abca63/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= golang.org/x/net v0.0.0-20220225172249-27dd8689420f h1:oA4XRj0qtSt8Yo1Zms0CUlsT3KG69V2UGQWPBxujDmc= golang.org/x/net v0.0.0-20220225172249-27dd8689420f/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk= +golang.org/x/net v0.0.0-20220722155237-a158d28d115b h1:PxfKdU9lEEDYjdIzOtC4qFWgkU2rGHdKlKowJSMN9h0= +golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c= golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U= golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= @@ -665,6 +670,7 @@ golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208/go.mod h1:RxMgew5VJxzue5/jJ golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= @@ -739,8 +745,10 @@ golang.org/x/sys v0.0.0-20211124211545-fe61309f8881/go.mod h1:oPkhp1MJrh7nUepCBc golang.org/x/sys v0.0.0-20211210111614-af8b64212486/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20220114195835-da31bd327af9/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20220209214540-3681064d5158 h1:rm+CHSpPEEW2IsXUib1ThaHIjuBVZjxNgSKmBLFfD4c= golang.org/x/sys v0.0.0-20220209214540-3681064d5158/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f h1:v4INt8xihDGvnrfjMDVXGxw9wrfxYyCjk0KbXjhR55s= +golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8= golang.org/x/term v0.0.0-20210927222741-03fcf44c2211 h1:JGgROgKl9N8DuW20oFS5gxc+lE67/N3FcwmBPMe7ArY= @@ -753,8 +761,9 @@ golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.3.5/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= -golang.org/x/text v0.3.7 h1:olpwvP2KacW1ZWvsR7uQhoyTYvKAupfQrRGBFM352Gk= golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ= +golang.org/x/text v0.3.8 h1:nAL+RVCQ9uMn3vJZbV+MRnydTJFPf8qqY42YiA6MrqY= +golang.org/x/text v0.3.8/go.mod h1:E6s5w1FMmriuDzIBO73fBruAKo1PCIq6d2Q6DHfQ8WQ= golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= @@ -824,6 +833,7 @@ golang.org/x/tools v0.1.3/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= golang.org/x/tools v0.1.4/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= golang.org/x/tools v0.1.5/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= golang.org/x/tools v0.1.6-0.20210820212750-d4cc65f0b2ff/go.mod h1:YD9qOF0M9xpSpdWTBbzEl5e/RnCefISl8E5Noe10jFM= +golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc= golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= From fcce30c01570edf381fb56779ffe474418dee079 Mon Sep 17 00:00:00 2001 From: lokesh-sreedhara Date: Sun, 5 Mar 2023 01:10:08 -0500 Subject: [PATCH 537/628] Assigned a security context constraint (SCC) to the service account that grants the requested access --- config/samples/sidb/openshift_rbac.yaml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/config/samples/sidb/openshift_rbac.yaml b/config/samples/sidb/openshift_rbac.yaml index ded5ef69..a4ca25f0 100644 --- a/config/samples/sidb/openshift_rbac.yaml +++ b/config/samples/sidb/openshift_rbac.yaml @@ -12,6 +12,9 @@ name: sidb-scc namespace: default allowPrivilegedContainer: false + users: + - system:serviceaccount:default:sidb-sa + - system:serviceaccount:default:oracle-database-operator runAsUser: type: MustRunAsRange uidRangeMin: 0 From 77d4b29f5d03e952b49a5238062538145fa47a98 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 6 Mar 2023 23:00:20 +0000 Subject: [PATCH 538/628] Bump golang.org/x/net from 0.0.0-20220225172249-27dd8689420f to 0.7.0 Bumps [golang.org/x/net](https://github.com/golang/net) from 0.0.0-20220225172249-27dd8689420f to 0.7.0. - [Release notes](https://github.com/golang/net/releases) - [Commits](https://github.com/golang/net/commits/v0.7.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: indirect ... Signed-off-by: dependabot[bot] --- go.mod | 8 ++++---- go.sum | 14 ++++++++------ 2 files changed, 12 insertions(+), 10 deletions(-) diff --git a/go.mod b/go.mod index e73b8fbd..b4a985a3 100644 --- a/go.mod +++ b/go.mod @@ -47,11 +47,11 @@ require ( github.com/spf13/pflag v1.0.5 // indirect go.uber.org/atomic v1.9.0 // indirect go.uber.org/multierr v1.7.0 // indirect - golang.org/x/net v0.0.0-20220225172249-27dd8689420f // indirect + golang.org/x/net v0.7.0 // indirect golang.org/x/oauth2 v0.0.0-20211104180415-d3ed0bb246c8 // indirect - golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f // indirect - golang.org/x/term v0.0.0-20210927222741-03fcf44c2211 // indirect - golang.org/x/text v0.3.8 // indirect + golang.org/x/sys v0.5.0 // indirect + golang.org/x/term v0.5.0 // indirect + golang.org/x/text v0.7.0 // indirect golang.org/x/time v0.0.0-20211116232009-f0f3c7e86c11 // indirect gomodules.xyz/jsonpatch/v2 v2.2.0 // indirect google.golang.org/appengine v1.6.7 // indirect diff --git a/go.sum b/go.sum index 630b64c4..2439bd99 100644 --- a/go.sum +++ b/go.sum @@ -637,10 +637,10 @@ golang.org/x/net v0.0.0-20210525063256-abc453219eb5/go.mod h1:9nx3DQGgdP8bBQD5qx golang.org/x/net v0.0.0-20210805182204-aaa1db679c0d/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= golang.org/x/net v0.0.0-20210825183410-e898025ed96a/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= golang.org/x/net v0.0.0-20211209124913-491a49abca63/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= -golang.org/x/net v0.0.0-20220225172249-27dd8689420f h1:oA4XRj0qtSt8Yo1Zms0CUlsT3KG69V2UGQWPBxujDmc= golang.org/x/net v0.0.0-20220225172249-27dd8689420f/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk= -golang.org/x/net v0.0.0-20220722155237-a158d28d115b h1:PxfKdU9lEEDYjdIzOtC4qFWgkU2rGHdKlKowJSMN9h0= golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c= +golang.org/x/net v0.7.0 h1:rJrUqqhjsgNp7KqAIc25s9pZnjU7TUcSY7HcVZjdn1g= +golang.org/x/net v0.7.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs= golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U= golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= @@ -747,12 +747,14 @@ golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e/go.mod h1:oPkhp1MJrh7nUepCBc golang.org/x/sys v0.0.0-20220114195835-da31bd327af9/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20220209214540-3681064d5158/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f h1:v4INt8xihDGvnrfjMDVXGxw9wrfxYyCjk0KbXjhR55s= golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.5.0 h1:MUK/U/4lj1t1oPg0HfuXDN/Z1wv31ZJ/YcPiGccS4DU= +golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8= -golang.org/x/term v0.0.0-20210927222741-03fcf44c2211 h1:JGgROgKl9N8DuW20oFS5gxc+lE67/N3FcwmBPMe7ArY= golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8= +golang.org/x/term v0.5.0 h1:n2a8QNdAb0sZNpU9R1ALUXBbY+w51fCQDN+7EdxNBsY= +golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k= golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= @@ -762,8 +764,8 @@ golang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.3.5/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ= -golang.org/x/text v0.3.8 h1:nAL+RVCQ9uMn3vJZbV+MRnydTJFPf8qqY42YiA6MrqY= -golang.org/x/text v0.3.8/go.mod h1:E6s5w1FMmriuDzIBO73fBruAKo1PCIq6d2Q6DHfQ8WQ= +golang.org/x/text v0.7.0 h1:4BRB4x83lYWy72KwLD/qYDuTu7q9PjSagHvijDw7cLo= +golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8= golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= From f596f1d7967215d4f3b9f234e4c093bf1d124ffb Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Thu, 23 Mar 2023 17:42:04 +0000 Subject: [PATCH 539/628] Update docs/adb/README.md --- docs/adb/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/adb/README.md b/docs/adb/README.md index d682250f..1f68fe82 100644 --- a/docs/adb/README.md +++ b/docs/adb/README.md @@ -21,7 +21,7 @@ After you create the resource, you can use the operator to perform the following * [Scale the OCPU core count or storage](#scale-the-ocpu-core-count-or-storage) an Autonomous Database * [Rename](#rename) an Autonomous Database -* [Manage ADMIN database user password](#manage-admin-passsword) of an Autonomous Database +* [Manage ADMIN database user password](#manage-admin-password) of an Autonomous Database * [Download instance credentials (wallets)](#download-wallets) of an Autonomous Database * [Stop/Start/Terminate](#stopstartterminate) an Autonomous Database * [Delete the resource](#delete-the-resource) from the cluster @@ -234,7 +234,7 @@ You can rename the database by changing the values of the `dbName` and `displayN autonomousdatabase.database.oracle.com/autonomousdatabase-sample configured ``` -## Manage Admin Passsword +## Manage Admin Password > Note: this operation requires an `AutonomousDatabase` object to be in your cluster. This example assumes the provision operation or the bind operation has been completed, and the operator is authorized with API Key Authentication. From e3a2c4dc2a6095a91e357a90101518779fd30437 Mon Sep 17 00:00:00 2001 From: matteo malvezzi Date: Thu, 13 Apr 2023 12:09:59 +0200 Subject: [PATCH 540/628] fix bug 35287543 --- controllers/database/pdb_controller.go | 3 + oracle-database-operator.yaml | 218 ++++++++++++++++++++++++- 2 files changed, 215 insertions(+), 6 deletions(-) diff --git a/controllers/database/pdb_controller.go b/controllers/database/pdb_controller.go index da4405a8..179142b0 100644 --- a/controllers/database/pdb_controller.go +++ b/controllers/database/pdb_controller.go @@ -490,6 +490,7 @@ func (r *PDBReconciler) callAPI(ctx context.Context, req ctrl.Request, pdb *dbap return "", err } webUser := string(secret.Data[cdb.Spec.WebServerUser.Secret.Key]) + webUser = strings.TrimSpace(webUser) // Get Web Server User Password secret = &corev1.Secret{} @@ -503,6 +504,8 @@ func (r *PDBReconciler) callAPI(ctx context.Context, req ctrl.Request, pdb *dbap return "", err } webUserPwd := string(secret.Data[cdb.Spec.WebServerPwd.Secret.Key]) + webUserPwd = strings.TrimSpace(webUserPwd) + var httpreq *http.Request if action == "GET" { diff --git a/oracle-database-operator.yaml b/oracle-database-operator.yaml index 309cbcb0..a50924b3 100644 --- a/oracle-database-operator.yaml +++ b/oracle-database-operator.yaml @@ -853,6 +853,131 @@ status: --- apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.6.1 + creationTimestamp: null + name: dataguardbrokers.database.oracle.com +spec: + group: database.oracle.com + names: + kind: DataguardBroker + listKind: DataguardBrokerList + plural: dataguardbrokers + singular: dataguardbroker + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .status.primaryDatabase + name: Primary + type: string + - jsonPath: .status.standbyDatabases + name: Standbys + type: string + - jsonPath: .spec.protectionMode + name: Protection Mode + type: string + - jsonPath: .status.clusterConnectString + name: Cluster Connect Str + priority: 1 + type: string + - jsonPath: .status.externalConnectString + name: Connect Str + type: string + - jsonPath: .spec.primaryDatabaseRef + name: Primary Database + priority: 1 + type: string + - jsonPath: .status.status + name: Status + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + description: DataguardBroker is the Schema for the dataguardbrokers API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: DataguardBrokerSpec defines the desired state of DataguardBroker + properties: + fastStartFailOver: + properties: + enable: + type: boolean + strategy: + items: + description: FSFO strategy + properties: + sourceDatabaseRef: + type: string + targetDatabaseRefs: + type: string + type: object + type: array + type: object + loadBalancer: + type: boolean + nodeSelector: + additionalProperties: + type: string + type: object + primaryDatabaseRef: + type: string + protectionMode: + enum: + - MaxPerformance + - MaxAvailability + type: string + setAsPrimaryDatabase: + type: string + standbyDatabaseRefs: + items: + type: string + type: array + required: + - primaryDatabaseRef + - protectionMode + - standbyDatabaseRefs + type: object + status: + description: DataguardBrokerStatus defines the observed state of DataguardBroker + properties: + clusterConnectString: + type: string + externalConnectString: + type: string + primaryDatabase: + type: string + primaryDatabaseRef: + type: string + protectionMode: + type: string + standbyDatabases: + type: string + status: + type: string + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition metadata: annotations: controller-gen.kubebuilder.io/version: v0.6.1 @@ -1915,7 +2040,7 @@ spec: type: object conditions: items: - description: "Condition contains details for one aspect of the current state of this API Resource. --- This struct is intended for direct use as an array at the field path .status.conditions. For example, type FooStatus struct{ // Represents the observations of a foo's current state. // Known .status.conditions.type are: \"Available\", \"Progressing\", and \"Degraded\" // +patchMergeKey=type // +patchStrategy=merge // +listType=map // +listMapKey=type Conditions []metav1.Condition `json:\"conditions,omitempty\" patchStrategy:\"merge\" patchMergeKey:\"type\" protobuf:\"bytes,1,rep,name=conditions\"` \n // other fields }" + description: "Condition contains details for one aspect of the current state of this API Resource. --- This struct is intended for direct use as an array at the field path .status.conditions. For example, \n \ttype FooStatus struct{ \t // Represents the observations of a foo's current state. \t // Known .status.conditions.type are: \"Available\", \"Progressing\", and \"Degraded\" \t // +patchMergeKey=type \t // +patchStrategy=merge \t // +listType=map \t // +listMapKey=type \t Conditions []metav1.Condition `json:\"conditions,omitempty\" patchStrategy:\"merge\" patchMergeKey:\"type\" protobuf:\"bytes,1,rep,name=conditions\"` \n \t // other fields \t}" properties: lastTransitionTime: description: lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. @@ -2015,12 +2140,15 @@ spec: - jsonPath: .status.edition name: Edition type: string + - jsonPath: .status.sid + name: Sid + priority: 1 + type: string - jsonPath: .status.status name: Status type: string - jsonPath: .status.role name: Role - priority: 1 type: string - jsonPath: .status.releaseUpdate name: Version @@ -2028,13 +2156,13 @@ spec: - jsonPath: .status.connectString name: Connect Str type: string - - jsonPath: .status.tcpsConnectString - name: TCPS Connect Str - type: string - jsonPath: .status.pdbConnectString name: Pdb Connect Str priority: 1 type: string + - jsonPath: .status.tcpsConnectString + name: TCPS Connect Str + type: string - jsonPath: .status.tcpsPdbConnectString name: TCPS Pdb Connect Str priority: 1 @@ -2077,6 +2205,10 @@ spec: type: string cloneFrom: type: string + createAsStandby: + type: boolean + dgBrokerConfigured: + type: boolean edition: enum: - standard @@ -2142,6 +2274,8 @@ spec: volumeName: type: string type: object + primaryDatabaseRef: + type: string readinessCheckPeriod: type: integer replicas: @@ -2185,7 +2319,7 @@ spec: type: string conditions: items: - description: "Condition contains details for one aspect of the current state of this API Resource. --- This struct is intended for direct use as an array at the field path .status.conditions. For example, type FooStatus struct{ // Represents the observations of a foo's current state. // Known .status.conditions.type are: \"Available\", \"Progressing\", and \"Degraded\" // +patchMergeKey=type // +patchStrategy=merge // +listType=map // +listMapKey=type Conditions []metav1.Condition `json:\"conditions,omitempty\" patchStrategy:\"merge\" patchMergeKey:\"type\" protobuf:\"bytes,1,rep,name=conditions\"` \n // other fields }" + description: "Condition contains details for one aspect of the current state of this API Resource. --- This struct is intended for direct use as an array at the field path .status.conditions. For example, \n \ttype FooStatus struct{ \t // Represents the observations of a foo's current state. \t // Known .status.conditions.type are: \"Available\", \"Progressing\", and \"Degraded\" \t // +patchMergeKey=type \t // +patchStrategy=merge \t // +listType=map \t // +listMapKey=type \t Conditions []metav1.Condition `json:\"conditions,omitempty\" patchStrategy:\"merge\" patchMergeKey:\"type\" protobuf:\"bytes,1,rep,name=conditions\"` \n \t // other fields \t}" properties: lastTransitionTime: description: lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. @@ -2237,6 +2371,8 @@ spec: datafilesPatched: default: "false" type: string + dgBrokerConfigured: + type: boolean edition: type: string flashBack: @@ -2293,6 +2429,8 @@ spec: type: object prebuiltDB: type: boolean + primaryDatabase: + type: string releaseUpdate: type: string replicas: @@ -2636,6 +2774,32 @@ rules: - get - patch - update +- apiGroups: + - database.oracle.com + resources: + - dataguardbrokers + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - database.oracle.com + resources: + - dataguardbrokers/finalizers + verbs: + - update +- apiGroups: + - database.oracle.com + resources: + - dataguardbrokers/status + verbs: + - get + - patch + - update - apiGroups: - database.oracle.com resources: @@ -2965,6 +3129,27 @@ webhooks: resources: - cdbs sideEffects: None +- admissionReviewVersions: + - v1 + - v1beta1 + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /mutate-database-oracle-com-v1alpha1-dataguardbroker + failurePolicy: Fail + name: mdataguardbroker.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + resources: + - dataguardbrokers + sideEffects: None - admissionReviewVersions: - v1 - v1beta1 @@ -3137,6 +3322,27 @@ webhooks: resources: - cdbs sideEffects: None +- admissionReviewVersions: + - v1 + - v1beta1 + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /validate-database-oracle-com-v1alpha1-dataguardbroker + failurePolicy: Fail + name: vdataguardbroker.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + resources: + - dataguardbrokers + sideEffects: None - admissionReviewVersions: - v1 - v1beta1 From 44c5ef72757fbca8c35f5d7923d1f2dd40d35640 Mon Sep 17 00:00:00 2001 From: Mohammed Yunus Date: Thu, 20 Apr 2023 10:20:38 +0000 Subject: [PATCH 541/628] Update .gitlab-ci.yml --- .gitlab-ci.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index f472d414..b567f938 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -13,7 +13,7 @@ build-operator: - docker rmi "$IMAGE" && docker system prune -f - make operator-yaml IMG=$newimage - if [ "$CI_COMMIT_BRANCH" != "master" ]; then sed -i "s/\(replicas.\) 3/\1 1/g" ./$OP_YAML; fi - - curl -s -n $ARTIFACTORY_REPO/$CI_COMMIT_BRANCH/$OP_YAML -T ./$OP_YAML + - curl -s --netrc-file $HOME/.netrc_gitlab $ARTIFACTORY_REPO/$CI_COMMIT_BRANCH/$OP_YAML -T ./$OP_YAML only: variables: - $CI_COMMIT_MESSAGE =~ /\#run-pipeline/ From f0115a57fc244a48ee0b567dc0c2398b2a740352 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Fri, 21 Apr 2023 10:18:41 -0400 Subject: [PATCH 542/628] merge ACD doc to ADB --- README.md | 23 ++++++++++++----------- docs/{acd/README.md => adb/ACD.md} | 0 2 files changed, 12 insertions(+), 11 deletions(-) rename docs/{acd/README.md => adb/ACD.md} (100%) diff --git a/README.md b/README.md index 1c091927..deeab140 100644 --- a/README.md +++ b/README.md @@ -6,13 +6,14 @@ As part of Oracle's resolution to make Oracle Database Kubernetes-native (that i In this v0.2.0 release, `OraOperator` supports the following database configurations and infrastructure: -* Oracle Autonomous Database on shared Oracle Cloud Infrastructure (OCI) (ADB-S) -* Oracle Autonomous Database on dedicated Cloud infrastructure (ADB-D) +* Oracle Autonomous Database: + * Oracle Autonomous Database shared Oracle Cloud Infrastructure (OCI) (ADB-S) + * Oracle Autonomous Database on dedicated Cloud infrastructure (ADB-D) + * Oracle Autonomous Container Database (ACD) (infrastructure) the infrastructure for provisionning Autonomous Databases. * Containerized Single Instance databases (SIDB) deployed in the Oracle Kubernetes Engine (OKE) and any k8s where OraOperator is deployed * Containerized Sharded databases (SHARDED) deployed in OKE and any k8s where OraOperator is deployed * Oracle Multitenant Databases (CDB/PDBs) * Oracle Database Cloud Service (DBCS) (VMDB) -* Oracle Autonomous Container Database (ACD) (infrastructure) the infrastructure for provisionning Autonomous Databases. Oracle will continue to extent OraOperator to support additional Oracle Database configurations. @@ -20,8 +21,7 @@ Oracle will continue to extent OraOperator to support additional Oracle Databas This release of Oracle Database Operator for Kubernetes (the operator) supports the following lifecycle operations: -* ADB-S: Provision, Bind, Start, Stop, terminate (soft/hard), scale (up/down), Manual Backup, Manual Restore -* ADB-D: provision, bind, start, stop, terminate (soft/hard), scale (up/down), Manual Backup, Manual Restore +* ADB-S/ADB-D: Provision, Bind, Start, Stop, terminate (soft/hard), scale (up/down), Manual Backup, Manual Restore * ACD: provision, bind, restart, terminate (soft/hard) * SIDB: Provision, clone, patch (in-place/out-of-place), update database initialization parameters, update database configuration (Flashback, archiving), Oracle Enterprise Manager (EM) Express (a basic observability console), Oracle REST Data Service (ORDS) to support REST based SQL, PDB management, SQL Developer Web, and Application Express (Apex) * SHARDED: Provision/deploy sharded databases and the shard topology, Add a new shard, Delete an existing shard @@ -67,6 +67,7 @@ Oracle strongly recommends that you ensure your system meets the following [Prer ```sh kubectl apply -f https://raw.githubusercontent.com/oracle/oracle-database-operator/main/oracle-database-operator.yaml ``` + --- **NOTE:** The above command will also upgrade the existing v0.1.0 `OraOperator` installation to the latest version i.e. v0.2.0. @@ -96,13 +97,13 @@ For more details, see [Oracle Database Operator Installation Instructions](./doc The quickstarts are designed for specific database configurations: * [Oracle Autonomous Database](./docs/adb/README.md) -* [Oracle Autonomous Container Database](./docs/acd/README.md) +* [Oracle Autonomous Container Database](./docs/adb/ACD.md) * [Containerized Oracle Single Instance Database](./docs/sidb/README.md) * [Containerized Oracle Sharded Database](./docs/sharding/README.md) * [Oracle Multitenant Database](./docs/multitenant/README.md) * [Oracle Database Cloud Service](./docs/dbcs/README.md) -YAML file templates are available under [`/config/samples`](./config/samples/). You can copy and edit these template files to configure them for your use cases. +YAML file templates are available under [`/config/samples`](./config/samples/). You can copy and edit these template files to configure them for your use cases. ## Uninstall the Operator @@ -133,7 +134,6 @@ YAML file templates are available under [`/config/samples`](./config/samples/). Note: If the CRD instances are not deleted, and the operator is deleted by using the preceding command, then operator deployment and instance objects (pods, services, PVCs, and so on) are deleted. However, if that happens, then the CRD deletion stops responding. This is because the CRD instances have properties that prevent their deletion, and that can only be removed by the operator pod, which is deleted when the APIServices are deleted. - ## Docs of the supported Oracle Database configurations * [Oracle Autonomous Database](https://docs.oracle.com/en-us/iaas/Content/Database/Concepts/adboverview.htm) @@ -155,21 +155,22 @@ You can submit a GitHub issue, or you can also file an [Oracle Support service]( Secure platforms are an important basis for general system security. Ensure that your deployment is in compliance with common security practices. ### Managing Sensitive Data + Kubernetes secrets are the usual means for storing credentials or passwords input for access. The operator reads the Secrets programmatically, which limits exposure of sensitive data. However, to protect your sensitive data, Oracle strongly recommends that you set and get sensitive data from Oracle Cloud Infrastructure Vault, or from third-party Vaults. The following is an example of a YAML file fragment for specifying Oracle Cloud Infrastructure Vault as the repository for the admin password. - ``` + +```yaml adminPassword: ociSecretOCID: ocid1.vaultsecret.oc1... ``` + Examples in this repository where passwords are entered on the command line are for demonstration purposes only. ### Reporting a Security Issue See [Reporting security vulnerabilities](./SECURITY.md) - - ## License Copyright (c) 2022 Oracle and/or its affiliates. diff --git a/docs/acd/README.md b/docs/adb/ACD.md similarity index 100% rename from docs/acd/README.md rename to docs/adb/ACD.md From 23d99b9af9d76373b218401bf8138eeba04d31d8 Mon Sep 17 00:00:00 2001 From: ishaan_desai Date: Thu, 4 May 2023 11:42:32 +0000 Subject: [PATCH 543/628] Idesai 23c free support sidb operator --- .../v1alpha1/singleinstancedatabase_types.go | 2 +- .../singleinstancedatabase_webhook.go | 45 +++++++++---- .../sidb/singleinstancedatabase_free.yaml | 38 +++++++++++ .../singleinstancedatabase_prebuiltdb.yaml | 4 +- .../sidb/singleinstancedatabase_secrets.yaml | 13 ++++ .../singleinstancedatabase_controller.go | 64 +++++++++++++------ 6 files changed, 132 insertions(+), 34 deletions(-) create mode 100644 config/samples/sidb/singleinstancedatabase_free.yaml diff --git a/apis/database/v1alpha1/singleinstancedatabase_types.go b/apis/database/v1alpha1/singleinstancedatabase_types.go index 4a55ee5e..81290735 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_types.go +++ b/apis/database/v1alpha1/singleinstancedatabase_types.go @@ -50,7 +50,7 @@ type SingleInstanceDatabaseSpec struct { // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster // Important: Run "make" to regenerate code after modifying this file - // +kubebuilder:validation:Enum=standard;enterprise;express + // +kubebuilder:validation:Enum=standard;enterprise;express;free Edition string `json:"edition,omitempty"` // SID must be alphanumeric (no special characters, only a-z, A-Z, 0-9), and no longer than 12 characters. diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index b4fc3dd0..d45f941c 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -86,6 +86,8 @@ func (r *SingleInstanceDatabase) Default() { if r.Spec.Sid == "" { if r.Spec.Edition == "express" { r.Spec.Sid = "XE" + } else if r.Spec.Edition == "free" { + r.Spec.Sid = "FREE" } else { r.Spec.Sid = "ORCLCDB" } @@ -94,12 +96,14 @@ func (r *SingleInstanceDatabase) Default() { if r.Spec.Pdbname == "" { if r.Spec.Edition == "express" { r.Spec.Pdbname = "XEPDB1" + } else if r.Spec.Edition == "free" { + r.Spec.Pdbname = "FREEPDB1" } else { r.Spec.Pdbname = "ORCLPDB1" } } - if r.Spec.Edition == "express" { + if r.Spec.Edition == "express" || r.Spec.Edition == "free" { if r.Status.Replicas == 1 { // default the replicas for XE r.Spec.Replicas = 1 @@ -147,8 +151,8 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { // Replica validation if r.Spec.Replicas > 1 { valMsg := "" - if r.Spec.Edition == "express" { - valMsg = "should be 1 for express edition" + if r.Spec.Edition == "express" || r.Spec.Edition == "free" { + valMsg = "should be 1 for " + r.Spec.Edition + " edition" } if r.Spec.Persistence.Size == "" { valMsg = "should be 1 if no persistence is specified" @@ -158,47 +162,57 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { field.Invalid(field.NewPath("spec").Child("replicas"), r.Spec.Replicas, valMsg)) } } - - if r.Spec.Edition == "express" { + + if r.Spec.Edition == "express" || r.Spec.Edition == "free" { if r.Spec.CloneFrom != "" { allErrs = append(allErrs, field.Invalid(field.NewPath("spec").Child("cloneFrom"), r.Spec.CloneFrom, - "Cloning not supported for Express edition")) + "Cloning not supported for " + r.Spec.Edition + " edition")) } if r.Spec.CreateAsStandby { allErrs = append(allErrs, field.Invalid(field.NewPath("spec").Child("createAsStandby"), r.Spec.CreateAsStandby, - "Physical Standby Database creation is not supported for Express edition")) + "Physical Standby Database creation is not supported for " + r.Spec.Edition + " edition")) } - if strings.ToUpper(r.Spec.Sid) != "XE" { + if r.Spec.Edition == "express" && strings.ToUpper(r.Spec.Sid) != "XE" { allErrs = append(allErrs, field.Invalid(field.NewPath("spec").Child("sid"), r.Spec.Sid, "Express edition SID must only be XE")) } - if strings.ToUpper(r.Spec.Pdbname) != "XEPDB1" { + if r.Spec.Edition == "free" && strings.ToUpper(r.Spec.Sid) != "FREE" { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("sid"), r.Spec.Sid, + "Free edition SID must only be FREE")) + } + if r.Spec.Edition == "express" && strings.ToUpper(r.Spec.Pdbname) != "XEPDB1" { allErrs = append(allErrs, field.Invalid(field.NewPath("spec").Child("pdbName"), r.Spec.Pdbname, "Express edition PDB must be XEPDB1")) } + if r.Spec.Edition == "free" && strings.ToUpper(r.Spec.Pdbname) != "FREEPDB1" { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("pdbName"), r.Spec.Pdbname, + "Free edition PDB must be FREEPDB1")) + } if r.Spec.InitParams.CpuCount != 0 { allErrs = append(allErrs, field.Invalid(field.NewPath("spec").Child("initParams").Child("cpuCount"), r.Spec.InitParams.CpuCount, - "Express edition does not support changing init parameter cpuCount.")) + r.Spec.Edition + " edition does not support changing init parameter cpuCount.")) } if r.Spec.InitParams.Processes != 0 { allErrs = append(allErrs, field.Invalid(field.NewPath("spec").Child("initParams").Child("processes"), r.Spec.InitParams.Processes, - "Express edition does not support changing init parameter process.")) + r.Spec.Edition + " edition does not support changing init parameter process.")) } if r.Spec.InitParams.SgaTarget != 0 { allErrs = append(allErrs, field.Invalid(field.NewPath("spec").Child("initParams").Child("sgaTarget"), r.Spec.InitParams.SgaTarget, - "Express edition does not support changing init parameter sgaTarget.")) + r.Spec.Edition + " edition does not support changing init parameter sgaTarget.")) } if r.Spec.InitParams.PgaAggregateTarget != 0 { allErrs = append(allErrs, field.Invalid(field.NewPath("spec").Child("initParams").Child("pgaAggregateTarget"), r.Spec.InitParams.PgaAggregateTarget, - "Express edition does not support changing init parameter pgaAggregateTarget.")) + r.Spec.Edition + " edition does not support changing init parameter pgaAggregateTarget.")) } } else { if r.Spec.Sid == "XE" { @@ -206,6 +220,11 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { field.Invalid(field.NewPath("spec").Child("sid"), r.Spec.Sid, "XE is reserved as the SID for Express edition of the database")) } + if r.Spec.Sid == "FREE" { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("sid"), r.Spec.Sid, + "FREE is reserved as the SID for FREE edition of the database")) + } } if r.Spec.CloneFrom != "" { diff --git a/config/samples/sidb/singleinstancedatabase_free.yaml b/config/samples/sidb/singleinstancedatabase_free.yaml new file mode 100644 index 00000000..30ba1d16 --- /dev/null +++ b/config/samples/sidb/singleinstancedatabase_free.yaml @@ -0,0 +1,38 @@ +# +# Copyright (c) 2023, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +apiVersion: database.oracle.com/v1alpha1 +kind: SingleInstanceDatabase +metadata: + name: freedb-sample + namespace: default +spec: + + ## Use only alphanumeric characters for sid + sid: FREE + + ## DB edition + edition: free + + ## Secret containing SIDB password mapped to secretKey + adminPassword: + secretName: freedb-admin-secret + + ## Database image details + image: + pullFrom: container-registry.oracle.com/database/free:latest + prebuiltDB: true + + ## size is the required minimum size of the persistent volume + ## storageClass is specified for automatic volume provisioning + ## accessMode can only accept one of ReadWriteOnce, ReadWriteMany + persistence: + size: 50Gi + ## oci-bv applies to OCI block volumes. Use "standard" storageClass for dynamic provisioning in Minikube. Update as appropriate for other cloud service providers + storageClass: "oci-bv" + accessMode: "ReadWriteOnce" + + ## Count of Database Pods. Should be 1 for express edition. + replicas: 1 \ No newline at end of file diff --git a/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml b/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml index d59b6a9c..5e4d0a4f 100644 --- a/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml +++ b/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml @@ -11,7 +11,7 @@ metadata: spec: ## DB edition - edition: express + edition: free ## Secret containing SIDB password mapped to secretKey adminPassword: @@ -19,7 +19,7 @@ spec: ## Database Image image: - pullFrom: container-registry.oracle.com/database/express:latest + pullFrom: container-registry.oracle.com/database/free:latest prebuiltDB: true ## Persistence is optional for prebuilt DB image diff --git a/config/samples/sidb/singleinstancedatabase_secrets.yaml b/config/samples/sidb/singleinstancedatabase_secrets.yaml index 19fd7670..50fbff91 100644 --- a/config/samples/sidb/singleinstancedatabase_secrets.yaml +++ b/config/samples/sidb/singleinstancedatabase_secrets.yaml @@ -39,3 +39,16 @@ type: Opaque stringData: ## Specify your DB password here oracle_pwd: + +--- + +## Prebuilt-Database Admin password secret +apiVersion: v1 +kind: Secret +metadata: + name: freedb-admin-secret + namespace: default +type: Opaque +stringData: + ## Specify your DB password here + oracle_pwd: diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 3db50f99..7372461c 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -85,6 +85,8 @@ var futureRequeue ctrl.Result = requeueN const singleInstanceDatabaseFinalizer = "database.oracle.com/singleinstancedatabasefinalizer" +var oemExpressUrl string + //+kubebuilder:rbac:groups=database.oracle.com,resources=singleinstancedatabases,verbs=get;list;watch;create;update;patch;delete //+kubebuilder:rbac:groups=database.oracle.com,resources=singleinstancedatabases/status,verbs=get;update;patch //+kubebuilder:rbac:groups=database.oracle.com,resources=singleinstancedatabases/finalizers,verbs=update @@ -369,9 +371,9 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab } } - // If Express Edition, ensure Replicas=1 - if m.Spec.Edition == "express" && m.Spec.Replicas > 1 { - eventMsgs = append(eventMsgs, "express edition supports only one replica") + // If Express/Free Edition, ensure Replicas=1 + if (m.Spec.Edition == "express" || m.Spec.Edition == "free") && m.Spec.Replicas > 1 { + eventMsgs = append(eventMsgs, m.Spec.Edition + " edition supports only one replica") } // If no persistence, ensure Replicas=1 if m.Spec.Persistence.Size == "" && m.Spec.Replicas > 1 { @@ -394,11 +396,11 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab m.Status.CloneFrom != dbcommons.NoCloneRef && m.Status.CloneFrom != m.Spec.CloneFrom) { eventMsgs = append(eventMsgs, "cloneFrom cannot be updated") } - if m.Spec.Edition == "express" && m.Spec.CloneFrom != "" { - eventMsgs = append(eventMsgs, "cloning not supported for express edition") + if (m.Spec.Edition == "express" || m.Spec.Edition == "free") && m.Spec.CloneFrom != "" { + eventMsgs = append(eventMsgs, "cloning not supported for " + m.Spec.Edition + " edition") } - if m.Spec.Edition == "express" && m.Spec.PrimaryDatabaseRef != "" && m.Spec.CreateAsStandby { - eventMsgs = append(eventMsgs, "Standby database creation is not supported for express edition") + if (m.Spec.Edition == "express" || m.Spec.Edition == "free") && m.Spec.PrimaryDatabaseRef != "" && m.Spec.CreateAsStandby { + eventMsgs = append(eventMsgs, "Standby database creation is not supported for " + m.Spec.Edition + " edition") } if m.Status.OrdsReference != "" && m.Status.Persistence.Size != "" && m.Status.Persistence != m.Spec.Persistence { eventMsgs = append(eventMsgs, "uninstall ORDS to change Peristence") @@ -665,8 +667,8 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }) } } - /* Wallet only for non-express edition, non-prebuiltDB */ - if m.Spec.Edition != "express" && !m.Spec.Image.PrebuiltDB { + /* Wallet only for edition barring express and free editions, non-prebuiltDB */ + if (m.Spec.Edition != "express" && m.Spec.Edition != "free") && !m.Spec.Image.PrebuiltDB { initContainers = append(initContainers, corev1.Container{ Name: "init-wallet", Image: m.Spec.Image.PullFrom, @@ -766,7 +768,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }(), Env: func() []corev1.EnvVar { // adding XE support, useful for dev/test/CI-CD - if m.Spec.Edition == "express" { + if m.Spec.Edition == "express" || m.Spec.Edition == "free" { return []corev1.EnvVar{ { Name: "SVC_HOST", @@ -1481,7 +1483,7 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex } m.Status.ConnectString = lbAddress + ":" + fmt.Sprint(extSvc.Spec.Ports[1].Port) + "/" + strings.ToUpper(sid) m.Status.PdbConnectString = lbAddress + ":" + fmt.Sprint(extSvc.Spec.Ports[1].Port) + "/" + strings.ToUpper(pdbName) - m.Status.OemExpressUrl = "https://" + lbAddress + ":" + fmt.Sprint(extSvc.Spec.Ports[0].Port) + "/em" + oemExpressUrl = "https://" + lbAddress + ":" + fmt.Sprint(extSvc.Spec.Ports[0].Port) + "/em" if m.Spec.EnableTCPS { m.Status.TcpsConnectString = lbAddress + ":" + fmt.Sprint(extSvc.Spec.Ports[len(extSvc.Spec.Ports)-1].Port) + "/" + strings.ToUpper(sid) m.Status.TcpsPdbConnectString = lbAddress + ":" + fmt.Sprint(extSvc.Spec.Ports[len(extSvc.Spec.Ports)-1].Port) + "/" + strings.ToUpper(pdbName) @@ -1493,7 +1495,7 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex if nodeip != "" { m.Status.ConnectString = nodeip + ":" + fmt.Sprint(extSvc.Spec.Ports[1].NodePort) + "/" + strings.ToUpper(sid) m.Status.PdbConnectString = nodeip + ":" + fmt.Sprint(extSvc.Spec.Ports[1].NodePort) + "/" + strings.ToUpper(pdbName) - m.Status.OemExpressUrl = "https://" + nodeip + ":" + fmt.Sprint(extSvc.Spec.Ports[0].NodePort) + "/em" + oemExpressUrl = "https://" + nodeip + ":" + fmt.Sprint(extSvc.Spec.Ports[0].NodePort) + "/em" if m.Spec.EnableTCPS { m.Status.TcpsConnectString = nodeip + ":" + fmt.Sprint(extSvc.Spec.Ports[len(extSvc.Spec.Ports)-1].NodePort) + "/" + strings.ToUpper(sid) m.Status.TcpsPdbConnectString = nodeip + ":" + fmt.Sprint(extSvc.Spec.Ports[len(extSvc.Spec.Ports)-1].NodePort) + "/" + strings.ToUpper(pdbName) @@ -1701,8 +1703,8 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplacePods(m *dbapi.SingleIn // ############################################################################# func (r *SingleInstanceDatabaseReconciler) createWallet(m *dbapi.SingleInstanceDatabase, ctx context.Context, req ctrl.Request) (ctrl.Result, error) { - // Wallet not supported for XE Database - if m.Spec.Edition == "express" { + // Wallet not supported for Express/Free Database + if m.Spec.Edition == "express" || m.Spec.Edition == "free" { return requeueN, nil } @@ -2016,6 +2018,16 @@ func (r *SingleInstanceDatabaseReconciler) validateDBReadiness(m *dbapi.SingleIn m.Status.ReleaseUpdate = version } } + oemSupport, err := isOEMSupported(r,m.Status.ReleaseUpdate) + if err != nil { + r.Log.Error(err, err.Error()) + return requeueY, readyPod, err + } + if oemSupport { + m.Status.OemExpressUrl = oemExpressUrl + } else { + m.Status.OemExpressUrl = dbcommons.ValueUnavailable + } if strings.ToUpper(m.Status.Role) == "PRIMARY" && m.Status.DatafilesPatched != "true" { eventReason := "Datapatch Pending" @@ -2034,8 +2046,8 @@ func (r *SingleInstanceDatabaseReconciler) validateDBReadiness(m *dbapi.SingleIn // ############################################################################# func (r *SingleInstanceDatabaseReconciler) deleteWallet(m *dbapi.SingleInstanceDatabase, ctx context.Context, req ctrl.Request) (ctrl.Result, error) { - // Wallet not supported for XE Database - if m.Spec.Edition == "express" { + // Wallet not supported for Express/Free Database + if m.Spec.Edition == "express" || m.Spec.Edition == "free" { return requeueN, nil } @@ -2250,9 +2262,9 @@ func (r *SingleInstanceDatabaseReconciler) runDatapatch(m *dbapi.SingleInstanceD readyPod corev1.Pod, ctx context.Context, req ctrl.Request) (ctrl.Result, error) { // Datapatch not supported for XE Database - if m.Spec.Edition == "express" { + if m.Spec.Edition == "express" || m.Spec.Edition == "free" { eventReason := "Datapatch Check" - eventMsg := "datapatch not supported for express edition" + eventMsg := "datapatch not supported for " + m.Spec.Edition + " edition" r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) r.Log.Info(eventMsg) return requeueN, nil @@ -3074,3 +3086,19 @@ func SetupStandbyDatabase(r *SingleInstanceDatabaseReconciler, stdby *dbapi.Sing return nil } + + +func isOEMSupported (r *SingleInstanceDatabaseReconciler,version string) (bool,error) { + majorVersion, err := strconv.Atoi(strings.Split(version,".")[0]) + r.Log.Info("majorVersion of database is " + strconv.Itoa(majorVersion)) + if err != nil { + return false,err + } + if majorVersion > 21{ + r.Log.Info("major Version " + strconv.Itoa(majorVersion) + " is greater that 21 so OEM Express is not supported") + return false,nil + } else { + r.Log.Info("major Version " + strconv.Itoa(majorVersion) + " is lesser than equal to 21 so OEM Express is supported") + return true,nil + } +} \ No newline at end of file From 49eb449435f8bfaf996b7b25904a2dcc34396560 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Tue, 9 May 2023 14:56:55 +0530 Subject: [PATCH 544/628] go mod tidy --- go.mod | 5 +---- go.sum | 30 +----------------------------- 2 files changed, 2 insertions(+), 33 deletions(-) diff --git a/go.mod b/go.mod index 433fedd8..81c4d435 100644 --- a/go.mod +++ b/go.mod @@ -51,7 +51,6 @@ require ( github.com/sony/gobreaker v0.5.0 // indirect github.com/spf13/pflag v1.0.5 // indirect github.com/stretchr/objx v0.4.0 // indirect - golang.org/x/net v0.2.0 // indirect go.uber.org/atomic v1.9.0 // indirect go.uber.org/multierr v1.7.0 // indirect golang.org/x/net v0.7.0 // indirect @@ -59,7 +58,7 @@ require ( golang.org/x/sys v0.5.0 // indirect golang.org/x/term v0.5.0 // indirect golang.org/x/text v0.7.0 // indirect - golang.org/x/time v0.0.0-20211116232009-f0f3c7e86c11 // indirect + golang.org/x/time v0.0.0-20220609170525-579cf78fd858 // indirect gomodules.xyz/jsonpatch/v2 v2.2.0 // indirect google.golang.org/appengine v1.6.7 // indirect google.golang.org/protobuf v1.28.0 // indirect @@ -80,6 +79,4 @@ require ( github.com/go-logr/zapr v1.2.3 // indirect github.com/google/gofuzz v1.2.0 // indirect github.com/google/uuid v1.3.0 // indirect - go.uber.org/atomic v1.9.0 // indirect - go.uber.org/multierr v1.7.0 // indirect ) diff --git a/go.sum b/go.sum index 5e151f5a..aacb116a 100644 --- a/go.sum +++ b/go.sum @@ -336,17 +336,6 @@ github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9de github.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.3.5/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k= -github.com/yuin/goldmark v1.4.0/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k= -github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY= -go.etcd.io/bbolt v1.3.2/go.mod h1:IbVyRI1SCnLcuJnV2u8VeU0CEYM7e686BmAb1XKL+uU= -go.etcd.io/bbolt v1.3.6/go.mod h1:qXsaaIqmgQH0T+OPdb99Bf+PKfBBQVAdyD6TY9G8XM4= -go.etcd.io/etcd/api/v3 v3.5.0/go.mod h1:cbVKeC6lCfl7j/8jBhAK6aIYO9XOjdptoxU/nLQcPvs= -go.etcd.io/etcd/client/pkg/v3 v3.5.0/go.mod h1:IJHfcCEKxYu1Os13ZdwCwIUTUVGYTSAM3YSwc9/Ac1g= -go.etcd.io/etcd/client/v2 v2.305.0/go.mod h1:h9puh54ZTgAKtEbut2oe9P4L/oqKCVB6xsXlzd7alYQ= -go.etcd.io/etcd/client/v3 v3.5.0/go.mod h1:AIKXXVX/DQXtfTEqBryiLTUXwON+GuvO6Z7lLS/oTh0= -go.etcd.io/etcd/pkg/v3 v3.5.0/go.mod h1:UzJGatBQ1lXChBkQF0AuAtkRQMYnHubxAEYIrC3MSsE= -go.etcd.io/etcd/raft/v3 v3.5.0/go.mod h1:UFOHSIvO/nKwd4lhkwabrTD3cqW5yVyYYf/KlD00Szc= -go.etcd.io/etcd/server/v3 v3.5.0/go.mod h1:3Ah5ruV+M+7RZr0+Y/5mNLwC+eQlni+mQmOVdCRJoS4= go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU= go.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8= go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= @@ -372,9 +361,6 @@ golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8U golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= -golang.org/x/crypto v0.0.0-20201002170205-7f63de1d35b0/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= -golang.org/x/crypto v0.0.0-20210817164053-32db794688a5/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc= -golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc= golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8= @@ -410,7 +396,6 @@ golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.4.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.4.1/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= -golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4= golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= @@ -450,11 +435,6 @@ golang.org/x/net v0.0.0-20210316092652-d523dce5a7f4/go.mod h1:RBQZq4jEuRlivfhVLd golang.org/x/net v0.0.0-20210405180319-a5a99cb37ef4/go.mod h1:p54w0d4576C0XHj96bSt6lcn1PtDYWL6XObtHCRCNQM= golang.org/x/net v0.0.0-20210503060351-7fd8e65b6420/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= golang.org/x/net v0.0.0-20210525063256-abc453219eb5/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= -golang.org/x/net v0.0.0-20210805182204-aaa1db679c0d/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= -golang.org/x/net v0.0.0-20210825183410-e898025ed96a/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= -golang.org/x/net v0.0.0-20211209124913-491a49abca63/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= -golang.org/x/net v0.0.0-20220225172249-27dd8689420f/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk= -golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c= golang.org/x/net v0.7.0 h1:rJrUqqhjsgNp7KqAIc25s9pZnjU7TUcSY7HcVZjdn1g= golang.org/x/net v0.7.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs= golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U= @@ -485,8 +465,6 @@ golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208/go.mod h1:RxMgew5VJxzue5/jJ golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= @@ -544,13 +522,10 @@ golang.org/x/sys v0.0.0-20211124211545-fe61309f8881/go.mod h1:oPkhp1MJrh7nUepCBc golang.org/x/sys v0.0.0-20211210111614-af8b64212486/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20220114195835-da31bd327af9/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20220209214540-3681064d5158/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20220412211240-33da011f77ad/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.5.0 h1:MUK/U/4lj1t1oPg0HfuXDN/Z1wv31ZJ/YcPiGccS4DU= golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= -golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8= -golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8= golang.org/x/term v0.5.0 h1:n2a8QNdAb0sZNpU9R1ALUXBbY+w51fCQDN+7EdxNBsY= golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k= golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= @@ -561,7 +536,6 @@ golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.3.5/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= -golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ= golang.org/x/text v0.7.0 h1:4BRB4x83lYWy72KwLD/qYDuTu7q9PjSagHvijDw7cLo= golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8= golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= @@ -623,8 +597,6 @@ golang.org/x/tools v0.1.2/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= golang.org/x/tools v0.1.3/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= golang.org/x/tools v0.1.4/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= golang.org/x/tools v0.1.5/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= -golang.org/x/tools v0.1.6-0.20210820212750-d4cc65f0b2ff/go.mod h1:YD9qOF0M9xpSpdWTBbzEl5e/RnCefISl8E5Noe10jFM= -golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc= golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= From 3fa80db94b06338529db150a0b1ac1cb337137d4 Mon Sep 17 00:00:00 2001 From: matteo malvezzi Date: Mon, 15 May 2023 16:01:10 +0200 Subject: [PATCH 545/628] bug 35357707 --- config/samples/multitenant/cdb_secret.yaml | 12 +- config/samples/multitenant/pdb_secret.yaml | 4 +- controllers/database/pdb_controller.go | 44 +++- docs/multitenant/README.md | 3 +- docs/multitenant/provisioning/cdb_secret.log | 0 docs/multitenant/provisioning/cdb_secret.yaml | 12 +- docs/multitenant/provisioning/pdb.log | 0 docs/multitenant/provisioning/pdb_secret.log | 0 docs/multitenant/provisioning/pdb_secret.yaml | 4 +- docs/multitenant/usecase01/README.md | 33 +-- docs/multitenant/usecase01/cdb.yaml | 46 ---- .../usecase01/{ => logfiles}/BuildImage.log | 0 .../usecase01/{ => logfiles}/ImagePush.log | 0 .../usecase01/{ => logfiles}/cdb.log | 0 .../{ => logfiles}/openssl_execution.log | 0 .../usecase01/{ => logfiles}/ordsconfig.log | 0 .../usecase01/{ => logfiles}/testapi.log | 0 docs/multitenant/usecase01/makefile | 247 ++++++++++++++++++ docs/multitenant/usecase01/sync.sh | 5 - docs/multitenant/usecase02/tde_secret.yaml | 4 +- 20 files changed, 324 insertions(+), 90 deletions(-) delete mode 100644 docs/multitenant/provisioning/cdb_secret.log delete mode 100644 docs/multitenant/provisioning/pdb.log delete mode 100644 docs/multitenant/provisioning/pdb_secret.log delete mode 100644 docs/multitenant/usecase01/cdb.yaml rename docs/multitenant/usecase01/{ => logfiles}/BuildImage.log (100%) rename docs/multitenant/usecase01/{ => logfiles}/ImagePush.log (100%) rename docs/multitenant/usecase01/{ => logfiles}/cdb.log (100%) rename docs/multitenant/usecase01/{ => logfiles}/openssl_execution.log (100%) rename docs/multitenant/usecase01/{ => logfiles}/ordsconfig.log (100%) rename docs/multitenant/usecase01/{ => logfiles}/testapi.log (100%) create mode 100644 docs/multitenant/usecase01/makefile delete mode 100644 docs/multitenant/usecase01/sync.sh diff --git a/config/samples/multitenant/cdb_secret.yaml b/config/samples/multitenant/cdb_secret.yaml index 5cc8b351..e270100d 100644 --- a/config/samples/multitenant/cdb_secret.yaml +++ b/config/samples/multitenant/cdb_secret.yaml @@ -9,9 +9,9 @@ metadata: namespace: oracle-database-operator-system type: Opaque data: - ords_pwd: "d2VsY29tZTE=" - sysadmin_pwd: "V0VsY29tZV8xMiMj" - cdbadmin_user: "QyMjREJBUElfQ0RCX0FETUlO" - cdbadmin_pwd: "V0VsY29tZV8xMiMj" - webserver_user: "c3FsX2FkbWlu" - webserver_pwd: "d2VsY29tZTE=" \ No newline at end of file + ords_pwd: "[base64 encode value]" + sysadmin_pwd: "[base64 encode value]" + cdbadmin_user: "[base64 encode value]" + cdbadmin_pwd: "[base64 encode value]" + webserver_user: "[base64 encode values]" + webserver_pwd: "[base64 encode values]" diff --git a/config/samples/multitenant/pdb_secret.yaml b/config/samples/multitenant/pdb_secret.yaml index ca0d21dd..8a3202d9 100644 --- a/config/samples/multitenant/pdb_secret.yaml +++ b/config/samples/multitenant/pdb_secret.yaml @@ -9,5 +9,5 @@ metadata: namespace: oracle-database-operator-system type: Opaque data: - sysadmin_user: "cGRiYWRtaW4=" - sysadmin_pwd: "V0VsY29tZV8xMiMj" \ No newline at end of file + sysadmin_user: "[ base64 encode value]" + sysadmin_pwd: "[ base64 encode value]" diff --git a/controllers/database/pdb_controller.go b/controllers/database/pdb_controller.go index 179142b0..735deaf7 100644 --- a/controllers/database/pdb_controller.go +++ b/controllers/database/pdb_controller.go @@ -49,6 +49,7 @@ import ( "fmt" "io/ioutil" "net/http" + "regexp" "strconv" "strings" "time" @@ -490,7 +491,7 @@ func (r *PDBReconciler) callAPI(ctx context.Context, req ctrl.Request, pdb *dbap return "", err } webUser := string(secret.Data[cdb.Spec.WebServerUser.Secret.Key]) - webUser = strings.TrimSpace(webUser) + webUser = strings.TrimSpace(webUser) // Get Web Server User Password secret = &corev1.Secret{} @@ -504,8 +505,7 @@ func (r *PDBReconciler) callAPI(ctx context.Context, req ctrl.Request, pdb *dbap return "", err } webUserPwd := string(secret.Data[cdb.Spec.WebServerPwd.Secret.Key]) - webUserPwd = strings.TrimSpace(webUserPwd) - + webUserPwd = strings.TrimSpace(webUserPwd) var httpreq *http.Request if action == "GET" { @@ -681,9 +681,11 @@ func (r *PDBReconciler) createPDB(ctx context.Context, req ctrl.Request, pdb *db if cdb.Spec.DBServer != "" { pdb.Status.ConnString = cdb.Spec.DBServer + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName } else { + ParseTnsAlias(&(cdb.Spec.DBTnsurl), &(pdb.Spec.PDBName)) pdb.Status.ConnString = cdb.Spec.DBTnsurl } + log.Info("New connect strinng", "tnsurl", cdb.Spec.DBTnsurl) log.Info("Created PDB Resource", "PDB Name", pdb.Spec.PDBName) r.getPDBState(ctx, req, pdb) return nil @@ -756,6 +758,7 @@ func (r *PDBReconciler) clonePDB(ctx context.Context, req ctrl.Request, pdb *dba if cdb.Spec.DBServer != "" { pdb.Status.ConnString = cdb.Spec.DBServer + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName } else { + ParseTnsAlias(&(cdb.Spec.DBTnsurl), &(pdb.Spec.PDBName)) pdb.Status.ConnString = cdb.Spec.DBTnsurl } @@ -887,6 +890,14 @@ func (r *PDBReconciler) unplugPDB(ctx context.Context, req ctrl.Request, pdb *db pdb.Status.Phase = pdbPhaseUnplug pdb.Status.Msg = "Waiting for PDB to be unplugged" + + if cdb.Spec.DBServer != "" { + pdb.Status.ConnString = cdb.Spec.DBServer + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName + } else { + ParseTnsAlias(&(cdb.Spec.DBTnsurl), &(pdb.Spec.PDBName)) + pdb.Status.ConnString = cdb.Spec.DBTnsurl + } + if err := r.Status().Update(ctx, pdb); err != nil { log.Error(err, "Failed to update status for :"+pdb.Name, "err", err.Error()) } @@ -1085,6 +1096,7 @@ func (r *PDBReconciler) mapPDB(ctx context.Context, req ctrl.Request, pdb *dbapi if cdb.Spec.DBServer != "" { pdb.Status.ConnString = cdb.Spec.DBServer + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName } else { + ParseTnsAlias(&(cdb.Spec.DBTnsurl), &(pdb.Spec.PDBName)) pdb.Status.ConnString = cdb.Spec.DBTnsurl } @@ -1235,3 +1247,29 @@ func (r *PDBReconciler) SetupWithManager(mgr ctrl.Manager) error { WithOptions(controller.Options{MaxConcurrentReconciles: 100}). Complete(r) } + +/************************************************************* +Enh 35357707 - PROVIDE THE PDB TNSALIAS INFORMATION +**************************************************************/ + +func ParseTnsAlias(tns *string, pdbsrv *string) { + fmt.Printf("Analyzing string [%s]\n", *tns) + fmt.Printf("Relacing srv [%s]\n", *pdbsrv) + + if strings.Contains(strings.ToUpper(*tns), "SERVICE_NAME") == false { + fmt.Print("Cannot generate tns alias for pdb") + return + } + + if strings.Contains(strings.ToUpper(*tns), "ORACLE_SID") == true { + fmt.Print("Cannot generate tns alias for pdb") + return + } + + *pdbsrv = fmt.Sprintf("SERVICE_NAME=%s", *pdbsrv) + tnsreg := regexp.MustCompile(`SERVICE_NAME=\w+`) + *tns = tnsreg.ReplaceAllString(*tns, *pdbsrv) + + fmt.Printf("Newstring [%s]\n", *tns) + +} diff --git a/docs/multitenant/README.md b/docs/multitenant/README.md index e9452497..ccf6957c 100644 --- a/docs/multitenant/README.md +++ b/docs/multitenant/README.md @@ -122,9 +122,8 @@ Please refer [here](./provisioning/example_setup_using_oci_oke_cluster.md) for s You can build this image by using the ORDS [Dockerfile](../../../ords/Dockerfile) - > **_NOTE:_** The current version of Oracle DB Operator Multitenant Controller has been tested with **ords 22.2.1.202.1302** version. -Please refer [here](./provisioning/ords_image.md) for the steps to build ORDS Docker Image with **ords 22.2.1.202.1302** version. +Please refer [here](./provisioning/ords_image.md) for the steps to build ORDS Docker Image + ## Kubernetes Secrets diff --git a/docs/multitenant/provisioning/cdb_secret.log b/docs/multitenant/provisioning/cdb_secret.log deleted file mode 100644 index e69de29b..00000000 diff --git a/docs/multitenant/provisioning/cdb_secret.yaml b/docs/multitenant/provisioning/cdb_secret.yaml index f6c5031f..4d03499e 100644 --- a/docs/multitenant/provisioning/cdb_secret.yaml +++ b/docs/multitenant/provisioning/cdb_secret.yaml @@ -9,9 +9,9 @@ metadata: namespace: oracle-database-operator-system type: Opaque data: - ords_pwd: "V0VsY29tZV8yMSMj" - sysadmin_pwd: "V0VsY29tZV8yMSMj" - cdbadmin_user: "QyMjREJBUElfQ0RCX0FETUlO" - cdbadmin_pwd: "V0VsY29tZV8yMSMj" - webserver_user: "c3FsX2FkbWlu" - webserver_pwd: "d2VsY29tZTE=" + ords_pwd: "[ base64 encode values ]" + sysadmin_pwd: "[ base64 encode values ]" + cdbadmin_user: "[ base64 encode values ]" + cdbadmin_pwd: "[ base64 encode values ]" + webserver_user: "[ base64 encode values ]" + webserver_pwd: "[base64 encode values ]" diff --git a/docs/multitenant/provisioning/pdb.log b/docs/multitenant/provisioning/pdb.log deleted file mode 100644 index e69de29b..00000000 diff --git a/docs/multitenant/provisioning/pdb_secret.log b/docs/multitenant/provisioning/pdb_secret.log deleted file mode 100644 index e69de29b..00000000 diff --git a/docs/multitenant/provisioning/pdb_secret.yaml b/docs/multitenant/provisioning/pdb_secret.yaml index 22a84dc9..cf385dda 100644 --- a/docs/multitenant/provisioning/pdb_secret.yaml +++ b/docs/multitenant/provisioning/pdb_secret.yaml @@ -9,5 +9,5 @@ metadata: namespace: oracle-database-operator-system type: Opaque data: - sysadmin_user: "cGRiYWRtaW4=" - sysadmin_pwd: "V0VsY29tZV8yMSMj" + sysadmin_user: "[ base64 encode values ]" + sysadmin_pwd: "[base64 encode values ]" diff --git a/docs/multitenant/usecase01/README.md b/docs/multitenant/usecase01/README.md index 428a99f5..490c4607 100644 --- a/docs/multitenant/usecase01/README.md +++ b/docs/multitenant/usecase01/README.md @@ -5,7 +5,7 @@ - [INTRODUCTION](#introduction) - [OPERATION STEPS ](#operation-steps) -- [Download latest version from orahub ](#download-latest-version-from-orahub-a-namedownloada) +- [Download latest version from github ](#download-latest-version-from-orahub-a-namedownloada) - [Upload webhook certificates](#upload-webhook-certificates-a-namewebhooka) - [Create the dboperator](#create-the-dboperator-a-namedboperatora) - [Create Secret for container registry](#create-secret-for-container-registry) @@ -20,7 +20,7 @@ -###### INTRODUCTION +##### INTRODUCTION This readme is a step by step guide used to implement database multi tenant operator. It assumes that a kubernets cluster and a database server are already available (no matter if single instance or RAC). kubectl must be configured in order to reach k8s cluster. @@ -48,12 +48,13 @@ The following table reports the parameters required to configure and use oracle | pdbTlsCrt | | [standalone.https.cert][cr] | | pdbTlsCat | | certificate authority | - +> A [makfile](./makefile) is available to sped up the command execution for the multitenant setup and test. See the comments in the header of file ### OPERATIONAL STEPS ---- -##### Download latest version from github + +#### Download latest version from github ```bash @@ -77,13 +78,13 @@ make operator-yaml IMG=operator:latest > **NOTE:** If you are using oracle-container-registry make sure to accept the license agreement otherwise the operator image pull fails. ---- -##### Upload webhook certificates +#### Upload webhook certificates ```bash kubectl apply -f https://github.com/jetstack/cert-manager/releases/latest/download/cert-manager.yaml ``` -##### Create the dboperator +#### Create the dboperator ```bash cd oracle-database-operator @@ -100,7 +101,7 @@ oracle-database-operator-controller-manager-557ff6c659-xpswv 1/1 Running ``` ---- -##### Create secret for container registry +#### Create secret for container registry + Make sure to login to your container registry and then create the secret for you container registry. @@ -118,7 +119,7 @@ container-registry-secret kubernetes.io/dockerconfigjson 1 19s webhook-server-cert kubernetes.io/tls ``` ---- -##### Build ords immage +#### Build ords immage + Build the ords image, downloading ords software is no longer needed; just build the image and push it to your repository @@ -137,7 +138,7 @@ docker push /ords-dboper:latest [example of execution](./ImagePush.log) ---- -##### Database Configuration +#### Database Configuration + Configure Database @@ -152,7 +153,7 @@ GRANT SYSDBA TO CONTAINER = ALL; GRANT CREATE SESSION TO CONTAINER = ALL; ``` ---- -##### Create CDB secret +#### Create CDB secret + Create secret for CDB connection @@ -207,7 +208,7 @@ webhook-server-cert kubernetes.io/tls 3 4m55s >**TIPS:** Use the following commands to analyze contents of an existing secret ```bash kubectl get secret -o yaml -n ``` ---- -##### Create Certificates +#### Create Certificates + Create certificates: At this stage we need to create certificates on our local machine and upload into kubernetes cluster by creating new secrets. @@ -261,7 +262,7 @@ kubectl create secret generic db-ca --from-file= -n oracle-database-op ---- -###### Apply cdb.yaml +#### Apply cdb.yaml + Create ords container @@ -339,7 +340,7 @@ spec: [example of cdb.yaml](./cdb.yaml) ---- -###### CDB - Logs and throuble shutting +#### CDB - Logs and throuble shutting + Check the status of ords container @@ -400,7 +401,7 @@ NAME CDB NAME DB SERVER DB PORT REPLICAS STATUS MESSAG [Example of executions](./ordsconfig.log) ----- -###### Create PDB secret +#### Create PDB secret ```bash @@ -433,7 +434,7 @@ pdb1-secret Opaque 2 79m <--- webhook-server-cert kubernetes.io/tls 3 79m ``` --- -###### Apply pdb yaml file to create pdb +#### Apply pdb yaml file to create pdb ```bash /usr/bin/kubectl apply -f pdb.yaml -n oracle-database-operator-system @@ -523,7 +524,7 @@ kubectl logs -f $(kubectl get pods -n oracle-database-operator-system|grep oracl ``` --- -###### Other actions +#### Other actions Configure and use other yaml files to perform pluggable database life cycle managment action **modify_pdb_open.yaml** **modify_pdb_close.yaml** diff --git a/docs/multitenant/usecase01/cdb.yaml b/docs/multitenant/usecase01/cdb.yaml deleted file mode 100644 index 25488168..00000000 --- a/docs/multitenant/usecase01/cdb.yaml +++ /dev/null @@ -1,46 +0,0 @@ -apiVersion: database.oracle.com/v1alpha1 -kind: CDB -metadata: - name: cdb-dev - namespace: oracle-database-operator-system -spec: - cdbName: "DB12" - dbServer: "racnode1.testrac.com" - dbPort: 1521 - ordsImage: "/ords-dboper:latest" - ordsImagePullPolicy: "Always" - serviceName: "TESTORDS" - replicas: 1 - sysAdminPwd: - secret: - secretName: "cdb1-secret" - key: "sysadmin_pwd" - ordsPwd: - secret: - secretName: "cdb1-secret" - key: "ords_pwd" - cdbAdminUser: - secret: - secretName: "cdb1-secret" - key: "cdbadmin_user" - cdbAdminPwd: - secret: - secretName: "cdb1-secret" - key: "cdbadmin_pwd" - webServerUser: - secret: - secretName: "cdb1-secret" - key: "webserver_user" - webServerPwd: - secret: - secretName: "cdb1-secret" - key: "webserver_pwd" - cdbTlsKey: - secret: - secretName: "db-tls" - key: "tls.key" - cdbTlsCrt: - secret: - secretName: "db-tls" - key: "tls.crt" - diff --git a/docs/multitenant/usecase01/BuildImage.log b/docs/multitenant/usecase01/logfiles/BuildImage.log similarity index 100% rename from docs/multitenant/usecase01/BuildImage.log rename to docs/multitenant/usecase01/logfiles/BuildImage.log diff --git a/docs/multitenant/usecase01/ImagePush.log b/docs/multitenant/usecase01/logfiles/ImagePush.log similarity index 100% rename from docs/multitenant/usecase01/ImagePush.log rename to docs/multitenant/usecase01/logfiles/ImagePush.log diff --git a/docs/multitenant/usecase01/cdb.log b/docs/multitenant/usecase01/logfiles/cdb.log similarity index 100% rename from docs/multitenant/usecase01/cdb.log rename to docs/multitenant/usecase01/logfiles/cdb.log diff --git a/docs/multitenant/usecase01/openssl_execution.log b/docs/multitenant/usecase01/logfiles/openssl_execution.log similarity index 100% rename from docs/multitenant/usecase01/openssl_execution.log rename to docs/multitenant/usecase01/logfiles/openssl_execution.log diff --git a/docs/multitenant/usecase01/ordsconfig.log b/docs/multitenant/usecase01/logfiles/ordsconfig.log similarity index 100% rename from docs/multitenant/usecase01/ordsconfig.log rename to docs/multitenant/usecase01/logfiles/ordsconfig.log diff --git a/docs/multitenant/usecase01/testapi.log b/docs/multitenant/usecase01/logfiles/testapi.log similarity index 100% rename from docs/multitenant/usecase01/testapi.log rename to docs/multitenant/usecase01/logfiles/testapi.log diff --git a/docs/multitenant/usecase01/makefile b/docs/multitenant/usecase01/makefile new file mode 100644 index 00000000..0a96d1e7 --- /dev/null +++ b/docs/multitenant/usecase01/makefile @@ -0,0 +1,247 @@ +# __ __ _ __ _ _ +# | \/ | __ _| | _____ / _(_) | ___ +# | |\/| |/ _` | |/ / _ \ |_| | |/ _ \ +# | | | | (_| | < __/ _| | | __/ +# |_| |_|\__,_|_|\_\___|_| |_|_|\___| +# +# ___ +# / _ \ _ __ _ __ _ __ ___ _ __ ___ +# | | | | '_ \| '_ \| '__/ _ \ '_ ` _ \ +# | |_| | | | | |_) | | | __/ | | | | | +# \___/|_| |_| .__/|_| \___|_| |_| |_| +# |_| +# ____ _ _ _ +# / ___|___ _ __ | |_ _ __ ___ | | | ___ _ __ +# | | / _ \| '_ \| __| '__/ _ \| | |/ _ \ '__| +# | |__| (_) | | | | |_| | | (_) | | | __/ | +# \____\___/|_| |_|\__|_| \___/|_|_|\___|_| +# +# +# This makefile helps to speed up the kubectl commands executions to deploy and test +# the OnPremises operator. Although it has few functionality you can adapt to your needs +# by adding much more targets. +# +# Quick start: +# ~~~~~~~~~~~ +# +# - Copy files of tab.1 in the makefile directory. +# - Edit the secret files and other yaml files with the correct credential as +# specified in the documentation. +# - Edit makefile updating variables of tab.2 +# - Execute commands of tab.3 "make step1" "make step2" "make step3".... +# +# Tab.1 - List of required files +# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +# +# +-----------------------------+---------------------------------------------+ +# |oracle-database-operator.yaml| Opertaor yaml file | +# +-----------------------------+---------------------------------------------+ +# |cdb_secret.yaml | Secret file for the rest server pod | +# +-----------------------------+---------------------------------------------+ +# |pdb_secret.yaml | Secret file for the pdb creation | +# +-----------------------------+---------------------------------------------+ +# |tde_secret.yaml | Secret file for the tablepsace enc. | +# +-----------------------------+---------------------------------------------+ +# |cdb.yaml | Rest server pod creation | +# +-----------------------------+---------------------------------------------+ +# |pdb.yaml | Pluggable database creation | +# +-----------------------------+---------------------------------------------+ +# |oracle-database-operator.yaml| Database operator | +# +-----------------------------+---------------------------------------------+ +# |Dockerfiles | Dockerfile for CBD | +# +-----------------------------+---------------------------------------------+ +# |runOrdsSSL.sh | Init script executed by Dockerfile | +# +-----------------------------+---------------------------------------------+ +# +# Tab.2 - List of variables +# ~~~~~~~~~~~~~~~~~~~~~~~~~ +# +# +-----------------------------+---------------------------------------------+ +# |OCIR | Your image registry | +# +-----------------------------+---------------------------------------------+ +# |OCIRPATH | Path of the image in your registry | +# +-----------------------------+---------------------------------------------+ +# +# Tab.3 - Execution steps +# ~~~~~~~~~~~~~~~~~~~~~~~ +# +# +-----------------------------+---------------------------------------------+ +# | MAKEFILE TARGETS LIST | +# | ----- ooo ----- | +# | - TARGET - - DESCRIPTION - | +# +-----------------------------+-------------------------------------+-------+ +# |step1 | Build rest server images | | +# +-----------------------------+-------------------------------------+ REST | +# |step2 | Tag the immages | SRV | +# +-----------------------------+-------------------------------------+ IMG | +# |step3 | Push the image into the repository | | +# +-----------------------------+-------------------------------------+-------+ +# |step4 | Load webhook certmanager | DB | +# +-----------------------------+-------------------------------------+ OPER | +# |step5 | Create the db operator | | +# +-----------------------------+-------------------------------------+-------+ +# |step6 | Create tls certificates | T | +# +-----------------------------+-------------------------------------+ L | +# |step7 | Create tls secret | S | +# +-----------------------------+---------------------------------------------+ +# |step8 | Create database secrets | +# +-----------------------------+---------------------------------------------+ +# |step9 | Create restserver pod | +# | | +---------------------------------------------+ +# | +---> checkstep9 | Monitor the executions | +# +-----------------------------+---------------------------------------------+ +# |step10 | Create pluggable database | +# | | +---------------------------------------------+ +# | +---> checkpdb | Monitor PDB status | +# +-----------------------------+---------------------------------------------+ +# | DIAGNOSTIC TARGETS | +# +-----------------------------+---------------------------------------------+ +# | dump | Dump pods info into a file | +# +-----------------------------+---------------------------------------------+ +# | reloadop | Reload the db operator | +# +-----------------------------+---------------------------------------------+ +# | login | Login into cdb pod | +# +-----------------------------+---------------------------------------------+ + + +################ TAB 2 VARIABLES ############ +OCIR=[...........YOUR REGISTRY...........] +OCIRPATH=[...PATH IN YOUR REGISTRY.....]/$(REST_SERVER)-dboper:$(ORDSVERSION) +############################################# +REST_SERVER=ords +ORDSVERSION=latest +DOCKER=/usr/bin/docker +KUBECTL=/usr/bin/kubectl +ORDS=/usr/local/bin/ords +CONFIG=/etc/ords/config +IMAGE=oracle/$(REST_SERVER)-dboper:$(ORDSVERSION) +DBOPERATOR=oracle-database-operator.yaml +URLPATH=/_/db-api/stable/database/pdbs/ +OPENSSL=/usr/bin/openssl +ORDSPORT=8888 +MAKE=/usr/bin/make +DOCKERFILE=Dockerfile +RM=/usr/bin/rm +ECHO=/usr/bin/echo +NAMESPACE=oracle-database-operator-system +CERTMANAGER=https://github.com/jetstack/cert-manager/releases/latest/download/cert-manager.yaml +CDB_SECRET=cdb_secret.yaml +PDB_SECRET=pdb_secret.yaml +TDE_SECRET=tde_secret.yaml +CDB=cdb.yaml +PDB=pdb.yaml +SKEY=tls.key +SCRT=tls.crt +CART=ca.crt +COMPANY=oracle + +step1: createimage +step2: tagimage +step3: push +step4: certmanager +step5: dboperator +step6: tlscert +step7: tlssecret +step8: dbsecret +step9: cdb +step10: pdb + +checkstep9: checkcdb + + +createimage: + @echo "BUILDING CDB IMAGES" + @if [[ ! -f ./Dockerfile ]]; \ + then\ + echo "DOCKERFILE DOES NOT EXISTS";\ + exit 1; \ + fi; + @if [[ ! -f ../runOrdsSSL.sh ]]; \ + then\ + echo "DOCKERFILE DOES NOT EXISTS";\ + exit 1; \ + fi; + $(DOCKER) build -t $(IMAGE) . + +tagimage: + @echo "TAG IMAGE" + $(DOCKER) tag $(IMAGE) $(OCIR)$(OCIRPATH) + +push: + @echo "PUSH IMAGE INTO THE REGISTRY" + $(DOCKER) push $(OCIR)$(OCIRPATH) + +certmanager: + @echo "WEBHOOK CERT MANAGER" + $(KUBECTL) apply -f $(CERTMANAGER) + +dboperator: + @echo "ORACLE DATABASE OPERATOR" + $(KUBECTL) apply -f $(DBOPERATOR) + +tlscert: + @echo "CREATING TLS CERTIFICATES" + $(OPENSSL) genrsa -out ca.key 2048 + $(OPENSSL) req -new -x509 -days 365 -key ca.key -subj "/C=CN/ST=GD/L=SZ/O=$(COMPANY), Inc./CN=$(COMPANY) Root CA" -out ca.crt + $(OPENSSL) req -newkey rsa:2048 -nodes -keyout $(SKEY) -subj "/C=CN/ST=GD/L=SZ/O=$(COMPANY), Inc./CN=cdb-dev-$(REST_SERVER)" -out server.csr + $(ECHO) "subjectAltName=DNS:cdb-dev-$(REST_SERVER),DNS:www.example.com" > extfile.txt + $(OPENSSL) x509 -req -extfile extfile.txt -days 365 -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out $(SCRT) + +tlssecret: + @echo "CREATING TLS SECRETS" + $(KUBECTL) create secret tls db-tls --key="$(SKEY)" --cert="$(SCRT)" -n $(NAMESPACE) + $(KUBECTL) create secret generic db-ca --from-file="$(CART)" -n $(NAMESPACE) + +dbsecret: + @echo "CREATING DB SECRETS" + $(KUBECTL) apply -f $(CDB_SECRET) -n $(NAMESPACE) + $(KUBECTL) apply -f $(PDB_SECRET) -n $(NAMESPACE) + $(KUBECTL) apply -f $(TDE_SECRET) -n $(NAMESPACE) + +cdb: + @echo "CREATING REST SRV POD" + $(KUBECTL) apply -f $(CDB) + +checkcdb: + $(KUBECTL) logs -f `$(KUBECTL) get pods -n $(NAMESPACE)|grep $(REST_SERVER)|cut -d ' ' -f 1` -n $(NAMESPACE) + +pdb: + $(KUBECTL) apply -f $(PDB) + +checkpdb: + $(KUBECTL) get pdbs -n $(NAMESPACE) + +dump: + @$(eval TMPSP := $(shell date "+%y%m%d%H%M%S" )) + @$(eval DIAGFILE := ./opdmp.$(TMPSP)) + @>$(DIAGFILE) + @echo "OPERATOR DUMP" >> $(DIAGFILE) + @echo "~~~~~~~~~~~~~" >> $(DIAGFILE) + $(KUBECTL) logs pod/`$(KUBECTL) get pods -n $(NAMESPACE)|grep oracle-database-operator-controller|head -1|cut -d ' ' -f 1` -n $(NAMESPACE) >>$(DIAGFILE) + $(KUBECTL) logs pod/`$(KUBECTL) get pods -n $(NAMESPACE)|grep oracle-database-operator-controller|head -2|tail -1 | cut -d ' ' -f 1` -n $(NAMESPACE) >>$(DIAGFILE) + $(KUBECTL) logs pod/`$(KUBECTL) get pods -n $(NAMESPACE)|grep oracle-database-operator-controller|tail -1|cut -d ' ' -f 1` -n $(NAMESPACE) >>$(DIAGFILE) + @echo "CDB LOG DUMP" >> $(DIAGFILE) + @echo "~~~~~~~~" >> $(DIAGFILE) + $(KUBECTL) logs `$(KUBECTL) get pods -n $(NAMESPACE)|grep $(REST_SERVER)| cut -d ' ' -f 1` -n $(NAMESPACE) >>$(DIAGFILE) + @echo "SECRET DMP" >>$(DIAGFILE) + @echo "~~~~~~~~" >> $(DIAGFILE) + $(KUBECTL) get secrets -o yaml -n $(NAMESPACE) >> $(DIAGFILE) + @echo "CDB/PDB DMP" >> $(DIAGFILE) + $(KUBECTL) get pdbs -o yaml -n $(NAMESPACE) >> $(DIAGFILE) + $(KUBECTL) get cdb -o yaml -n $(NAMESPACE) >> $(DIAGFILE) + @echo "CLUSTER INFO" >> $(DIAGFILE) + $(KUBECTL) get nodes -o wide + $(KUBECTL) get svc --namespace=kube-system + +reloadop: + echo "RESTARTING OPERATOR" + $(eval OP1 := $(shell $(KUBECTL) get pods -n $(NAMESPACE)|grep oracle-database-operator-controller|head -1|cut -d ' ' -f 1 )) + $(eval OP2 := $(shell $(KUBECTL) get pods -n $(NAMESPACE)|grep oracle-database-operator-controller|head -2|tail -1|cut -d ' ' -f 1 )) + $(eval OP3 := $(shell $(KUBECTL) get pods -n $(NAMESPACE)|grep oracle-database-operator-controller|tail -1|cut -d ' ' -f 1 )) + $(KUBECTL) get pod $(OP1) -n $(NAMESPACE) -o yaml | kubectl replace --force -f - + $(KUBECTL) get pod $(OP2) -n $(NAMESPACE) -o yaml | kubectl replace --force -f - + $(KUBECTL) get pod $(OP3) -n $(NAMESPACE) -o yaml | kubectl replace --force -f - + +login: + $(KUBECTL) exec -it `$(KUBECTL) get pods -n $(NAMESPACE)|grep $(REST_SERVER)|cut -d ' ' -f 1` -n $(NAMESPACE) bash + diff --git a/docs/multitenant/usecase01/sync.sh b/docs/multitenant/usecase01/sync.sh deleted file mode 100644 index d982e796..00000000 --- a/docs/multitenant/usecase01/sync.sh +++ /dev/null @@ -1,5 +0,0 @@ -#!/bin/bash - - -sudo cp README.md /home/oracle/WORKDIR/Ords22_operator/oracle-database-operator/docs/ords -sudo chown oracle:dba /home/oracle/WORKDIR/Ords22_operator/oracle-database-operator/docs/ords/README.md diff --git a/docs/multitenant/usecase02/tde_secret.yaml b/docs/multitenant/usecase02/tde_secret.yaml index 5975e27e..d0186ff2 100644 --- a/docs/multitenant/usecase02/tde_secret.yaml +++ b/docs/multitenant/usecase02/tde_secret.yaml @@ -10,6 +10,6 @@ metadata: namespace: oracle-database-operator-system type: Opaque data: - tdepassword: "Y2hhbmdlbWUK" - tdesecret: "Y2hhbmdlbWUK" + tdepassword: "[base64 encode value]" + tdesecret: "[base64 encode value]" From c763f2e5e476a96730ad8a20a7e87bf305f26e36 Mon Sep 17 00:00:00 2001 From: Rahul Vats Date: Tue, 23 May 2023 05:58:18 +0000 Subject: [PATCH 546/628] Ords Github bugs --- apis/database/v1alpha1/oraclerestdataservice_webhook.go | 8 ++++++++ commons/database/constants.go | 2 +- controllers/database/oraclerestdataservice_controller.go | 2 ++ 3 files changed, 11 insertions(+), 1 deletion(-) diff --git a/apis/database/v1alpha1/oraclerestdataservice_webhook.go b/apis/database/v1alpha1/oraclerestdataservice_webhook.go index 285a4978..be83fffc 100644 --- a/apis/database/v1alpha1/oraclerestdataservice_webhook.go +++ b/apis/database/v1alpha1/oraclerestdataservice_webhook.go @@ -112,6 +112,14 @@ func (r *OracleRestDataService) ValidateCreate() error { } } + // Validating databaseRef and ORDS kind name not to be same + if r.Spec.DatabaseRef == r.Name { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("Name"), + "cannot be same as DatabaseRef: " + r.Spec.DatabaseRef)) + + } + if len(allErrs) == 0 { return nil } diff --git a/commons/database/constants.go b/commons/database/constants.go index b48af253..ef5162f6 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -364,7 +364,7 @@ const UninstallORDSCMD string = "\numask 177" + const GetORDSStatus string = "curl -sSkv -k -X GET https://localhost:8443/ords/_/db-api/stable/metadata-catalog/" -const ValidateAdminPassword string = "conn sys/%s@${ORACLE_SID} as sysdba\nshow user" +const ValidateAdminPassword string = "conn sys/\\\"%s\\\"@${ORACLE_SID} as sysdba\nshow user" const ReconcileError string = "ReconcileError" diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 26784b5b..1a3359f7 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -370,6 +370,8 @@ func (r *OracleRestDataServiceReconciler) validateSIDBReadiness(m *dbapi.OracleR log.Info(eventMsg) return requeueY, sidbReadyPod } else { + eventMsg := "login attempt failed for database admin password in secret " + m.Spec.AdminPassword.SecretName + log.Info(eventMsg) return requeueY, sidbReadyPod } From 481d608ea05cbecb528b4c2fc49c78a4820ef45a Mon Sep 17 00:00:00 2001 From: ishaan_desai Date: Tue, 23 May 2023 07:39:59 +0000 Subject: [PATCH 547/628] Dataguard Bugs Resolution --- commons/database/constants.go | 7 + commons/database/utils.go | 20 +++ config/samples/sidb/dataguardbroker.yaml | 2 +- .../database/dataguardbroker_controller.go | 120 +++++++++++++++--- .../singleinstancedatabase_controller.go | 19 +++ docs/sidb/README.md | 12 +- 6 files changed, 154 insertions(+), 26 deletions(-) diff --git a/commons/database/constants.go b/commons/database/constants.go index ef5162f6..33dce251 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -104,6 +104,10 @@ const StandbyDatabasePrerequisitesSQL string = "ALTER SYSTEM SET db_create_file_ "\nALTER SYSTEM SET STANDBY_FILE_MANAGEMENT=AUTO;" + "\nALTER SYSTEM SET dg_broker_start=TRUE;" +const GetDBOpenMode string = "select open_mode from v\\$database;" + +const ModifyStdbyDBOpenMode string = "alter database recover managed standby database disconnect;" + const StandbyTnsnamesEntry string = ` ##STANDBYDATABASE_SID## = (DESCRIPTION = @@ -208,6 +212,9 @@ const DataguardBrokerAddDBMaxAvailabilityCMD string = "ADD DATABASE ${ORACLE_SID "(CONNECT_DATA=(SERVICE_NAME=${ORACLE_SID}_DGMGRL)(INSTANCE_NAME=${ORACLE_SID})(SERVER=DEDICATED)))';" + "\nENABLE CONFIGURATION;" +const RemoveStandbyDBFromDGConfgCMD string = "DISABLE DATABASE ${ORACLE_SID};" + + "\nREMOVE DATABASE ${ORACLE_SID};" + const DBShowConfigCMD string = "SHOW CONFIGURATION;" const DataguardBrokerGetDatabaseCMD string = "SELECT DATABASE || ':' || DATAGUARD_ROLE AS DATABASE FROM V\\$DG_BROKER_CONFIG;" diff --git a/commons/database/utils.go b/commons/database/utils.go index 9c2d8363..c1ecd680 100644 --- a/commons/database/utils.go +++ b/commons/database/utils.go @@ -505,6 +505,26 @@ func GetDatabaseRole(readyPod corev1.Pod, r client.Reader, return "", errors.New("database role is nil") } +func GetDatabaseOpenMode(readyPod corev1.Pod, r client.Reader, + config *rest.Config, ctx context.Context, req ctrl.Request, edition string) (string, error) { + log := ctrllog.FromContext(ctx).WithValues("GetDatabaseOpenMode",req.NamespacedName) + + out,err := ExecCommand(r, config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | %s",GetDBOpenMode,SQLPlusCLI)) + if err != nil { + return "",err + } + log.Info(out) + if !strings.Contains(out, "no rows selected") && !strings.Contains(out, "ORA-") { + out1 := strings.Replace(out, " ", "_", -1) + // filtering output and storing databse_role in "database_role" + databaseOpenMode := strings.Fields(out1)[2] + // first 2 values in the slice will be column name(DATABASE_ROLE) and a seperator(--------------) . + return databaseOpenMode, nil + } + return "", errors.New("database open mode is nil") + } + // Returns true if any of the pod in 'pods' is with pod.Status.Phase == phase func IsAnyPodWithStatus(pods []corev1.Pod, phase corev1.PodPhase) (bool, corev1.Pod) { anyPodWithPhase := false diff --git a/config/samples/sidb/dataguardbroker.yaml b/config/samples/sidb/dataguardbroker.yaml index e95460fe..47c9ee28 100644 --- a/config/samples/sidb/dataguardbroker.yaml +++ b/config/samples/sidb/dataguardbroker.yaml @@ -25,5 +25,5 @@ spec: ## Protection Mode for dg configuration . MaxAvailability or MaxPerformance protectionMode: MaxAvailability - ## Manual Switchover to this database to make it primary(if not already) . + ## Manual Switchover to this database to make it primary(if not already), requires target Database SID . setAsPrimaryDatabase: "" \ No newline at end of file diff --git a/controllers/database/dataguardbroker_controller.go b/controllers/database/dataguardbroker_controller.go index 7eeb3dc1..1d76fe48 100644 --- a/controllers/database/dataguardbroker_controller.go +++ b/controllers/database/dataguardbroker_controller.go @@ -403,7 +403,7 @@ func (r *DataguardBrokerReconciler) setupDataguardBrokerConfiguration(m *dbapi.D // Update Databases r.updateReconcileStatus(m, sidbReadyPod, ctx, req) - } + } eventReason := "DG Configuration up to date" eventMsg := "" @@ -657,26 +657,28 @@ func (r *DataguardBrokerReconciler) setupDataguardBrokerConfigurationForGivenDB( } // ## SET PROPERTY FASTSTARTFAILOVERTARGET FOR EACH DATABASE TO ALL OTHER DATABASES IN DG CONFIG . - for i := 0; i < len(databases); i++ { - out, err = dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf("dgmgrl sys@%s \"EDIT DATABASE %s SET PROPERTY FASTSTARTFAILOVERTARGET=%s\"< admin.pwd", primaryConnectString, - strings.Split(databases[i], ":")[0], getFSFOTargets(i, databases))) - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - log.Info("SETTING FSFO TARGET OUTPUT") - log.Info(out) + if (m.Spec.FastStartFailOver.Enable == true){ + for i := 0; i < len(databases); i++ { + out, err = dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("dgmgrl sys@%s \"EDIT DATABASE %s SET PROPERTY FASTSTARTFAILOVERTARGET=%s\"< admin.pwd", primaryConnectString, + strings.Split(databases[i], ":")[0], getFSFOTargets(i, databases))) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + log.Info("SETTING FSFO TARGET OUTPUT") + log.Info(out) - out, err = dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf("dgmgrl sys@%s \"SHOW DATABASE %s FASTSTARTFAILOVERTARGET\" < admin.pwd", primaryConnectString, strings.Split(databases[i], ":")[0])) - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - log.Info("FSFO TARGETS OF " + databases[i]) - log.Info(out) + out, err = dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("dgmgrl sys@%s \"SHOW DATABASE %s FASTSTARTFAILOVERTARGET\" < admin.pwd", primaryConnectString, strings.Split(databases[i], ":")[0])) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + log.Info("FSFO TARGETS OF " + databases[i]) + log.Info(out) + } } // Remove admin pwd file _, err = dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, true, "bash", "-c", @@ -694,6 +696,59 @@ func (r *DataguardBrokerReconciler) setupDataguardBrokerConfigurationForGivenDB( return requeueN } +// ############################################################################# +// +// Remove up DG Configuration for a given StandbyDatabase - To be Tested +// +// ############################################################################# +func (r *DataguardBrokerReconciler) removeDataguardBrokerConfigurationForGivenDB( m *dbapi.DataguardBroker,n *dbapi.SingleInstanceDatabase,standbyDatabase *dbapi.SingleInstanceDatabase, standbyDatabaseReadyPod corev1.Pod, sidbReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) ctrl.Result { + log := r.Log.WithValues("removeDataguardBrokerConfigurationForGivenDB", req.NamespacedName) + + if standbyDatabaseReadyPod.Name == "" || sidbReadyPod.Name == ""{ + return requeueY + } + + // ## CHECK IF DG CONFIGURATION IS AVAILABLE IN PRIMARY DATABASE ## + out, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | dgmgrl / as sysdba", dbcommons.DBShowConfigCMD)) + + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + log.Info("Showconfiguration Output") + log.Info(out) + + if strings.Contains(out, "ORA-16525") { + log.Info("ORA-16525: The Oracle Data Guard broker is not yet available on Primary") + return requeueY + } + + // ## REMOVING STANDBY DATABASE FROM DG CONFIGURATION ## + out, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf(dbcommons.CreateDGMGRLScriptFile, dbcommons.RemoveStandbyDBFromDGConfgCMD)) + + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + + // ## SHOW CONFIGURATION + out, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | dgmgrl / as sysdba", dbcommons.DBShowConfigCMD)) + if err != nil { + log.Error(err, err.Error()) + return requeueY + } + log.Info("Showconfiguration Output") + log.Info(out) + // Set DG Configured status to false for this standbyDatabase. so that in next reconcilation, we dont configure this again + standbyDatabase.Status.DgBrokerConfigured = false + r.Status().Update(ctx, standbyDatabase) + + return requeueN +} + // ############################################################################# // // Return FSFO targets of each StandbyDatabase @@ -976,6 +1031,33 @@ func (r *DataguardBrokerReconciler) manageDataguardBrokerDeletion(req ctrl.Reque // indicated by the deletion timestamp being set. isDataguardBrokerMarkedToBeDeleted := m.GetDeletionTimestamp() != nil if isDataguardBrokerMarkedToBeDeleted { + + // Make a singleinstancedatabase with empty + singleInstanceDatabase := &dbapi.SingleInstanceDatabase{} + err := r.Get(ctx, types.NamespacedName{Namespace: req.Namespace, Name: m.Spec.PrimaryDatabaseRef}, singleInstanceDatabase) + if err != nil { + log.Error(err,err.Error()) + return requeueY,err + } + // Get its POD + // Validate if Primary Database Reference is ready + result, sidbReadyPod, _ := r.validateSidbReadiness(m, singleInstanceDatabase, ctx, req) + if result.Requeue { + log.Info("Reconcile queued") + return result, nil + } + // Get its Role + out, err := dbcommons.GetDatabaseRole(sidbReadyPod, r, r.Config, ctx, req, singleInstanceDatabase.Spec.Edition) + // check if its PRIMARY + if strings.ToUpper(out) != "PRIMARY" { + eventReason := "Deletion" + eventMsg := "DataGuard Broker cannot be deleted since primaryDatabaseRef is not in PRIMARY role" + log.Info("DataGuard Broker cannot be deleted since primaryDatabaseRef is not in PRIMARY role") + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) + return requeueN, nil + } + // if not PRIMARY throw error and log it + if controllerutil.ContainsFinalizer(m, dataguardBrokerFinalizer) { // Run finalization logic for dataguardBrokerFinalizer. If the // finalization logic fails, don't remove the finalizer so diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 7372461c..7ad7a56a 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -229,6 +229,24 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct if err != nil { return requeueY, err } + + databaseOpenMode,err := dbcommons.GetDatabaseOpenMode(readyPod, r, r.Config, ctx, req, singleInstanceDatabase.Spec.Edition) + + if err != nil { + r.Log.Error(err, err.Error()) + return requeueY, err + } + r.Log.Info("DB openMode Output") + r.Log.Info(databaseOpenMode) + if databaseOpenMode == "READ_ONLY" { + out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c",fmt.Sprintf("echo -e \"%s\" | %s",dbcommons.ModifyStdbyDBOpenMode,dbcommons.SQLPlusCLI)) + if err != nil { + r.Log.Error(err, err.Error()) + return requeueY, err + } + r.Log.Info("Standby DB open mode modified") + r.Log.Info(out) + } singleInstanceDatabase.Status.PrimaryDatabase = referredPrimaryDatabase.Name // Store all standbyDatabase sid:name in a map to use it during manual switchover. @@ -498,6 +516,7 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab if m.Spec.Sid == rp.Spec.Sid { r.Log.Info("Standby database SID can not be same as the Primary database SID") r.Recorder.Eventf(m, corev1.EventTypeWarning, "Spec Error", "Standby and Primary database SID can not be same") + m.Status.Status = dbcommons.StatusError return requeueY, err } diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 33013dfd..9bbea8d9 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -735,16 +735,16 @@ $ kubectl apply -f dataguardbroker.yaml dataguardbroker.database.oracle.com/dataguardbroker-sample apply ``` +Or use the patch command -#### Patching DataGuardBroker Attributes +```sh +$ kubectl --type=merge -p '{"spec":{"setAsPrimaryDatabase":"ORCLS1"}}' patch dataguardbroker dataguardbroker-sample -The following attributes cannot be patched post DataguardBroker resource Creation : `primaryDatabaseRef, standbyDatabaseRefs, loadBalancer`. + dataguardbroker.database.oracle.com/dataguardbroker-sample patched +``` -```sh -$ kubectl --type=merge -p '{"spec":{"primaryDatabaseRef":"ORCL"}}' patch dataguardbroker dataguardbroker-sample +**NOTE :** The following attributes cannot be patched post DataguardBroker resource Creation : `primaryDatabaseRef, protectionMode`. - The DataguardBroker "dataguardbroker-sample" is invalid: spec.sid: Forbidden: cannot be changed - ``` #### Delete DataguardBroker Resource From 4876e144076bf0a807b4ae2fc3a25317ee00071e Mon Sep 17 00:00:00 2001 From: ishaan_desai Date: Thu, 25 May 2023 06:24:20 +0000 Subject: [PATCH 548/628] 23c Free support and Dataguard Controller enhancements --- .../sidb/singleinstancedatabase_free.yaml | 1 + .../sidb/singleinstancedatabase_secrets.yaml | 2 +- .../database/dataguardbroker_controller.go | 21 ++++++++++ .../singleinstancedatabase_controller.go | 41 ++++++++++--------- docs/sidb/README.md | 19 +++++++++ 5 files changed, 63 insertions(+), 21 deletions(-) diff --git a/config/samples/sidb/singleinstancedatabase_free.yaml b/config/samples/sidb/singleinstancedatabase_free.yaml index 30ba1d16..558bd8f4 100644 --- a/config/samples/sidb/singleinstancedatabase_free.yaml +++ b/config/samples/sidb/singleinstancedatabase_free.yaml @@ -22,6 +22,7 @@ spec: ## Database image details image: + ## Oracle Database Free is only supported from DB version 23.2 onwards pullFrom: container-registry.oracle.com/database/free:latest prebuiltDB: true diff --git a/config/samples/sidb/singleinstancedatabase_secrets.yaml b/config/samples/sidb/singleinstancedatabase_secrets.yaml index 1be55cb4..d98432ee 100644 --- a/config/samples/sidb/singleinstancedatabase_secrets.yaml +++ b/config/samples/sidb/singleinstancedatabase_secrets.yaml @@ -42,7 +42,7 @@ stringData: --- -## Prebuilt-Database Admin password secret +## Oracle Database Free Admin password secret apiVersion: v1 kind: Secret metadata: diff --git a/controllers/database/dataguardbroker_controller.go b/controllers/database/dataguardbroker_controller.go index 1d76fe48..861211f6 100644 --- a/controllers/database/dataguardbroker_controller.go +++ b/controllers/database/dataguardbroker_controller.go @@ -355,6 +355,21 @@ func (r *DataguardBrokerReconciler) setupDataguardBrokerConfiguration(m *dbapi.D sidbReadyPod corev1.Pod, adminPassword string, ctx context.Context, req ctrl.Request) ctrl.Result { log := r.Log.WithValues("setupDataguardBrokerConfiguration", req.NamespacedName) + databases, _, err := dbcommons.GetDatabasesInDgConfig(sidbReadyPod, r, r.Config, ctx, req) + dbSet := make(map[string]struct{}) + if err != nil { + if err.Error() != "databases in DG config is nil" { + return requeueY + } + } + if len(databases) > 0 { + log.Info("Databases in DG config are :") + for i := 0; i < len(databases); i++ { + log.Info(strings.Split(databases[i], ":")[0]) + dbSet[strings.ToUpper(strings.Split(databases[i], ":")[0])] = struct{}{} + } + } + for i := 0; i < len(m.Spec.StandbyDatabaseRefs); i++ { standbyDatabase := &dbapi.SingleInstanceDatabase{} @@ -369,6 +384,12 @@ func (r *DataguardBrokerReconciler) setupDataguardBrokerConfiguration(m *dbapi.D log.Error(err, err.Error()) return requeueY } + _, ok := dbSet[standbyDatabase.Status.Sid] + if ok { + log.Info("A database with the same SID is already configured in the DG") + r.Recorder.Eventf(m, corev1.EventTypeWarning, "Spec Error", "A database with the same SID " + standbyDatabase.Status.Sid + " is already configured in the DG") + continue + } // Check if dataguard broker is already configured for the standby database if standbyDatabase.Status.DgBrokerConfigured { diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 7ad7a56a..af7d17f1 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -2037,17 +2037,27 @@ func (r *SingleInstanceDatabaseReconciler) validateDBReadiness(m *dbapi.SingleIn m.Status.ReleaseUpdate = version } } - oemSupport, err := isOEMSupported(r,m.Status.ReleaseUpdate) + dbMajorVersion, err := strconv.Atoi(strings.Split(m.Status.ReleaseUpdate,".")[0]) if err != nil { r.Log.Error(err, err.Error()) return requeueY, readyPod, err } - if oemSupport { - m.Status.OemExpressUrl = oemExpressUrl - } else { + r.Log.Info("DB Major Version is " + strconv.Itoa(dbMajorVersion)) + // Validating that free edition of the database is only supported from database 23c onwards + if (m.Spec.Edition == "free" && dbMajorVersion < 23){ + r.Log.Info("Oracle Database Free is only supported from version 23c onwards") + r.Recorder.Eventf(m, corev1.EventTypeWarning, "Spec Error", "Oracle Database Free is only supported from version 23c onwards") + m.Status.Status = dbcommons.StatusError + return requeueN, readyPod, errors.New("Oracle Database Free is only supported from version 23c onwards") + } + // Checking if OEM is supported in the provided Database version + if (dbMajorVersion >= 23 ) { m.Status.OemExpressUrl = dbcommons.ValueUnavailable + } else { + m.Status.OemExpressUrl = oemExpressUrl } + if strings.ToUpper(m.Status.Role) == "PRIMARY" && m.Status.DatafilesPatched != "true" { eventReason := "Datapatch Pending" eventMsg := "datapatch execution pending" @@ -2623,6 +2633,13 @@ func (r *SingleInstanceDatabaseReconciler) updateORDSStatus(m *dbapi.SingleInsta func (r *SingleInstanceDatabaseReconciler) manageSingleInstanceDatabaseDeletion(req ctrl.Request, ctx context.Context, m *dbapi.SingleInstanceDatabase) (ctrl.Result, error) { log := r.Log.WithValues("manageSingleInstanceDatabaseDeletion", req.NamespacedName) + + log.Info("DG broker is configured with the DB " + m.Status.Sid + " : " + strconv.FormatBool(m.Status.DgBrokerConfigured)) + if (m.Status.DgBrokerConfigured){ + log.Info("Database cannot be deleted as it is present in a DataGuard Broker configuration") + r.Recorder.Eventf(m, corev1.EventTypeWarning, "Deleting", "Database cannot be deleted as it is present in a DataGuard Broker configuration") + return requeueN,nil + } // Check if the SingleInstanceDatabase instance is marked to be deleted, which is // indicated by the deletion timestamp being set. @@ -3105,19 +3122,3 @@ func SetupStandbyDatabase(r *SingleInstanceDatabaseReconciler, stdby *dbapi.Sing return nil } - - -func isOEMSupported (r *SingleInstanceDatabaseReconciler,version string) (bool,error) { - majorVersion, err := strconv.Atoi(strings.Split(version,".")[0]) - r.Log.Info("majorVersion of database is " + strconv.Itoa(majorVersion)) - if err != nil { - return false,err - } - if majorVersion > 21{ - r.Log.Info("major Version " + strconv.Itoa(majorVersion) + " is greater that 21 so OEM Express is not supported") - return false,nil - } else { - r.Log.Info("major Version " + strconv.Itoa(majorVersion) + " is lesser than equal to 21 so OEM Express is supported") - return true,nil - } -} \ No newline at end of file diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 9bbea8d9..ccf3419e 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -26,6 +26,7 @@ Oracle Database Operator for Kubernetes (`OraOperator`) includes the Single Inst * [Creating an Oracle Data Guard Configuration](#creating-an-oracle-data-guard-configuration) * [Creating a Standby Database](#creating-a-standby-database) * [Setup DataGuardBroker Configuration for a Single Instance Database](#setup-dataguardbroker-configuration-for-a-single-instance-database) + * [Delete a standby database with dataguard broker configured](#delete-a-standby-database-with-dataguard-broker-configured) * [OracleRestDataService Resource](#oraclerestdataservice-resource) * [REST Enable a Database](#rest-enable-a-database) * [Provision ORDS](#provision-ords) @@ -756,6 +757,24 @@ $ kubectl delete dataguardbroker dgbroker-sample **NOTE :** You can only delete DataGuard broker when role of `.spec.primaryDatabaseRef` is PRIMARY +### Delete a standby database with dataguard broker configured + +To delete a standby database in a dataguard configuration, delete the dataguardbroker resource first followed by the standby database + +#### Delete DataguardBroker Resource +```sh +$ kubectl delete dataguardbroker dgbroker-sample + + dataguardbroker.database.oracle.com/dgbroker-sample deleted +``` + +#### Delete Standby Database +```sh +$ kubectl delete singleinstancedatabase stdby-1 + + singleinstancedatabase.database.oracle.com "stdby-1" deleted +``` + ## OracleRestDataService Resource The Oracle Database Operator creates the `OracleRestDataService` as a custom resource. We will refer `OracleRestDataService` as ORDS from now onwards. Creating ORDS as a custom resource enables the RESTful API access to the Oracle Database in K8s and enables it to be managed as a native Kubernetes object. From 3ad2f3c4b8ab05558b61dd63bed1e53b898b5f3b Mon Sep 17 00:00:00 2001 From: ishaan_desai Date: Tue, 30 May 2023 13:58:27 +0000 Subject: [PATCH 549/628] DG and SIDB bug fixes --- .../v1alpha1/dataguardbroker_types.go | 16 ++--- .../v1alpha1/dataguardbroker_webhook.go | 18 ++++++ .../singleinstancedatabase_webhook.go | 19 ++++++ config/samples/sidb/dataguardbroker.yaml | 6 +- .../sidb/singleinstancedatabase_standby.yaml | 10 ++-- .../database/dataguardbroker_controller.go | 58 +++++++++---------- .../singleinstancedatabase_controller.go | 19 +++--- 7 files changed, 93 insertions(+), 53 deletions(-) diff --git a/apis/database/v1alpha1/dataguardbroker_types.go b/apis/database/v1alpha1/dataguardbroker_types.go index 8f03a113..138e2bdb 100644 --- a/apis/database/v1alpha1/dataguardbroker_types.go +++ b/apis/database/v1alpha1/dataguardbroker_types.go @@ -50,15 +50,15 @@ type DataguardBrokerSpec struct { // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster // Important: Run "make" to regenerate code after modifying this file - PrimaryDatabaseRef string `json:"primaryDatabaseRef"` - StandbyDatabaseRefs []string `json:"standbyDatabaseRefs"` - SetAsPrimaryDatabase string `json:"setAsPrimaryDatabase,omitempty"` - LoadBalancer bool `json:"loadBalancer,omitempty"` - + PrimaryDatabaseRef string `json:"primaryDatabaseRef"` + StandbyDatabaseRefs []string `json:"standbyDatabaseRefs"` + SetAsPrimaryDatabase string `json:"setAsPrimaryDatabase,omitempty"` + LoadBalancer bool `json:"loadBalancer,omitempty"` + ServiceAnnotations map[string]string `json:"serviceAnnotations,omitempty"` // +kubebuilder:validation:Enum=MaxPerformance;MaxAvailability - ProtectionMode string `json:"protectionMode"` - NodeSelector map[string]string `json:"nodeSelector,omitempty"` - FastStartFailOver DataguardBrokerFastStartFailOver `json:"fastStartFailOver,omitempty"` + ProtectionMode string `json:"protectionMode"` + NodeSelector map[string]string `json:"nodeSelector,omitempty"` + FastStartFailOver DataguardBrokerFastStartFailOver `json:"fastStartFailOver,omitempty"` } type DataguardBrokerFastStartFailOver struct { diff --git a/apis/database/v1alpha1/dataguardbroker_webhook.go b/apis/database/v1alpha1/dataguardbroker_webhook.go index 4db987d9..c947f7a3 100644 --- a/apis/database/v1alpha1/dataguardbroker_webhook.go +++ b/apis/database/v1alpha1/dataguardbroker_webhook.go @@ -69,6 +69,24 @@ var _ webhook.Defaulter = &DataguardBroker{} func (r *DataguardBroker) Default() { dataguardbrokerlog.Info("default", "name", r.Name) + if (r.Spec.LoadBalancer) { + if r.Spec.ServiceAnnotations == nil { + r.Spec.ServiceAnnotations= make(map[string]string) + } + // Annotations required for a flexible load balancer on oci + _, ok := r.Spec.ServiceAnnotations["service.beta.kubernetes.io/oci-load-balancer-shape"] + if(!ok) { + r.Spec.ServiceAnnotations["service.beta.kubernetes.io/oci-load-balancer-shape"] = "flexible" + } + _,ok = r.Spec.ServiceAnnotations["service.beta.kubernetes.io/oci-load-balancer-shape-flex-min"] + if(!ok) { + r.Spec.ServiceAnnotations["service.beta.kubernetes.io/oci-load-balancer-shape-flex-min"] = "10" + } + _,ok = r.Spec.ServiceAnnotations["service.beta.kubernetes.io/oci-load-balancer-shape-flex-max"] + if(!ok) { + r.Spec.ServiceAnnotations["service.beta.kubernetes.io/oci-load-balancer-shape-flex-max"] = "100" + } + } } // TODO(user): change verbs to "verbs=create;update;delete" if you want to enable deletion validation. diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index d45f941c..0afadde8 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -72,6 +72,25 @@ var _ webhook.Defaulter = &SingleInstanceDatabase{} func (r *SingleInstanceDatabase) Default() { singleinstancedatabaselog.Info("default", "name", r.Name) + if r.Spec.LoadBalancer { + // Annotations required for a flexible load balancer on oci + if r.Spec.ServiceAnnotations == nil { + r.Spec.ServiceAnnotations= make(map[string]string) + } + _, ok := r.Spec.ServiceAnnotations["service.beta.kubernetes.io/oci-load-balancer-shape"] + if(!ok) { + r.Spec.ServiceAnnotations["service.beta.kubernetes.io/oci-load-balancer-shape"] = "flexible" + } + _,ok = r.Spec.ServiceAnnotations["service.beta.kubernetes.io/oci-load-balancer-shape-flex-min"] + if(!ok) { + r.Spec.ServiceAnnotations["service.beta.kubernetes.io/oci-load-balancer-shape-flex-min"] = "10" + } + _,ok = r.Spec.ServiceAnnotations["service.beta.kubernetes.io/oci-load-balancer-shape-flex-max"] + if(!ok) { + r.Spec.ServiceAnnotations["service.beta.kubernetes.io/oci-load-balancer-shape-flex-max"] = "100" + } + } + if r.Spec.AdminPassword.KeepSecret == nil { keepSecret := true r.Spec.AdminPassword.KeepSecret = &keepSecret diff --git a/config/samples/sidb/dataguardbroker.yaml b/config/samples/sidb/dataguardbroker.yaml index 47c9ee28..26ab9d79 100644 --- a/config/samples/sidb/dataguardbroker.yaml +++ b/config/samples/sidb/dataguardbroker.yaml @@ -15,8 +15,8 @@ spec: ## Standby DB pod CRD Metadata Name to add this DB to DG config standbyDatabaseRefs: - # - standbyDatabase-sample1 - # - standbyDatabase-sample2 + # - standbydatabase-sample + # - standbydatabase-sample1 ## Type of service . Applicable on cloud enviroments only ## if loadBalService : false , service type = "NodePort" . else "LoadBalancer" @@ -26,4 +26,4 @@ spec: protectionMode: MaxAvailability ## Manual Switchover to this database to make it primary(if not already), requires target Database SID . - setAsPrimaryDatabase: "" \ No newline at end of file + setAsPrimaryDatabase: "" diff --git a/config/samples/sidb/singleinstancedatabase_standby.yaml b/config/samples/sidb/singleinstancedatabase_standby.yaml index 19309014..e93cc5e4 100644 --- a/config/samples/sidb/singleinstancedatabase_standby.yaml +++ b/config/samples/sidb/singleinstancedatabase_standby.yaml @@ -6,18 +6,16 @@ apiVersion: database.oracle.com/v1alpha1 kind: SingleInstanceDatabase metadata: - # Creates base sidb-sample. Use singleinstancedatabase_clone.yaml for cloning + # Creates base standbydatabase-sample. Use singleinstancedatabase_clone.yaml for cloning # and singleinstancedatabase_patch.yaml for patching - name: sidb-sample + name: standbydatabase-sample namespace: default spec: ## Use only alphanumeric characters for sid - sid: ORCL1 + sid: ORCLS - ## Reference to a source primary database from which - ## Format: 1. Intra-cluster: you can give name of the primary database or the database connect string in `:/` format - ## 2. Inter-cluster: Database connect string in `:/` format + ## Reference to a source primary database name primaryDatabaseRef: "sidb-sample" ## Enable this flag for creating Physical Standby Database diff --git a/controllers/database/dataguardbroker_controller.go b/controllers/database/dataguardbroker_controller.go index 861211f6..808311eb 100644 --- a/controllers/database/dataguardbroker_controller.go +++ b/controllers/database/dataguardbroker_controller.go @@ -103,10 +103,14 @@ func (r *DataguardBrokerReconciler) Reconcile(ctx context.Context, req ctrl.Requ // Manage SingleInstanceDatabase Deletion result, err := r.manageDataguardBrokerDeletion(req, ctx, dataguardBroker) - if result.Requeue || err != nil { + if result.Requeue { r.Log.Info("Reconcile queued") return result, err } + if err != nil { + r.Log.Error(err,err.Error()) + return result,err + } // Fetch Primary Database Reference singleInstanceDatabase := &dbapi.SingleInstanceDatabase{} @@ -251,6 +255,15 @@ func (r *DataguardBrokerReconciler) instantiateSVCSpec(m *dbapi.DataguardBroker) Labels: map[string]string{ "app": m.Name, }, + Annotations : func() map[string]string { + annotations := make(map[string]string) + if len(m.Spec.ServiceAnnotations) != 0 { + for key, value := range m.Spec.ServiceAnnotations { + annotations[key] = value + } + } + return annotations + }(), }, Spec: corev1.ServiceSpec{ Ports: []corev1.ServicePort{ @@ -1052,33 +1065,6 @@ func (r *DataguardBrokerReconciler) manageDataguardBrokerDeletion(req ctrl.Reque // indicated by the deletion timestamp being set. isDataguardBrokerMarkedToBeDeleted := m.GetDeletionTimestamp() != nil if isDataguardBrokerMarkedToBeDeleted { - - // Make a singleinstancedatabase with empty - singleInstanceDatabase := &dbapi.SingleInstanceDatabase{} - err := r.Get(ctx, types.NamespacedName{Namespace: req.Namespace, Name: m.Spec.PrimaryDatabaseRef}, singleInstanceDatabase) - if err != nil { - log.Error(err,err.Error()) - return requeueY,err - } - // Get its POD - // Validate if Primary Database Reference is ready - result, sidbReadyPod, _ := r.validateSidbReadiness(m, singleInstanceDatabase, ctx, req) - if result.Requeue { - log.Info("Reconcile queued") - return result, nil - } - // Get its Role - out, err := dbcommons.GetDatabaseRole(sidbReadyPod, r, r.Config, ctx, req, singleInstanceDatabase.Spec.Edition) - // check if its PRIMARY - if strings.ToUpper(out) != "PRIMARY" { - eventReason := "Deletion" - eventMsg := "DataGuard Broker cannot be deleted since primaryDatabaseRef is not in PRIMARY role" - log.Info("DataGuard Broker cannot be deleted since primaryDatabaseRef is not in PRIMARY role") - r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) - return requeueN, nil - } - // if not PRIMARY throw error and log it - if controllerutil.ContainsFinalizer(m, dataguardBrokerFinalizer) { // Run finalization logic for dataguardBrokerFinalizer. If the // finalization logic fails, don't remove the finalizer so @@ -1138,8 +1124,22 @@ func (r *DataguardBrokerReconciler) cleanupDataguardBroker(req ctrl.Request, ctx return result, nil } + // Get its Role + out, err := dbcommons.GetDatabaseRole(sidbReadyPod, r, r.Config, ctx, req, singleInstanceDatabase.Spec.Edition) + if err != nil { + log.Error(err, err.Error()) + return requeueY, err + } + // check if its PRIMARY database + if strings.ToUpper(out) != "PRIMARY" { + eventReason := "Deletion" + eventMsg := "DataGuard Broker cannot be deleted since primaryDatabaseRef is not in PRIMARY role" + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) + return requeueY, errors.New(eventMsg) + } + // Get Primary database to remove dataguard configuration - _, out, err := dbcommons.GetDatabasesInDgConfig(sidbReadyPod, r, r.Config, ctx, req) + _, out, err = dbcommons.GetDatabasesInDgConfig(sidbReadyPod, r, r.Config, ctx, req) if err != nil { log.Error(err, err.Error()) return requeueY, err diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index af7d17f1..462a5381 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -149,6 +149,10 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct r.Log.Info("Reconcile queued") return result, nil } + if err != nil { + r.Log.Error(err,err.Error()) + return result, err + } // First validate result, err = r.validate(singleInstanceDatabase, cloneFromDatabase, referredPrimaryDatabase, ctx, req) @@ -2633,13 +2637,7 @@ func (r *SingleInstanceDatabaseReconciler) updateORDSStatus(m *dbapi.SingleInsta func (r *SingleInstanceDatabaseReconciler) manageSingleInstanceDatabaseDeletion(req ctrl.Request, ctx context.Context, m *dbapi.SingleInstanceDatabase) (ctrl.Result, error) { log := r.Log.WithValues("manageSingleInstanceDatabaseDeletion", req.NamespacedName) - - log.Info("DG broker is configured with the DB " + m.Status.Sid + " : " + strconv.FormatBool(m.Status.DgBrokerConfigured)) - if (m.Status.DgBrokerConfigured){ - log.Info("Database cannot be deleted as it is present in a DataGuard Broker configuration") - r.Recorder.Eventf(m, corev1.EventTypeWarning, "Deleting", "Database cannot be deleted as it is present in a DataGuard Broker configuration") - return requeueN,nil - } + // Check if the SingleInstanceDatabase instance is marked to be deleted, which is // indicated by the deletion timestamp being set. @@ -2697,6 +2695,13 @@ func (r *SingleInstanceDatabaseReconciler) cleanupSingleInstanceDatabase(req ctr return requeueY, nil } + if (m.Status.DgBrokerConfigured){ + eventReason := "Cannot Delete" + eventMsg := "Database cannot be deleted as it is present in a DataGuard Broker configuration" + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventReason) + return requeueY,errors.New(eventMsg) + } + // call deletePods() with zero pods in avaiable and nil readyPod to delete all pods result, err := r.deletePods(ctx, req, m, []corev1.Pod{}, corev1.Pod{}, 0, 0) if result.Requeue { From 04c0c0ffa9be8caa45f0d4dc6c62126cbe99ef99 Mon Sep 17 00:00:00 2001 From: ishaan_desai Date: Mon, 5 Jun 2023 14:10:00 +0000 Subject: [PATCH 550/628] Bug Fixes DG and SIDB controllers --- .../singleinstancedatabase_webhook.go | 40 ++++++++++++++++++- .../singleinstancedatabase_controller.go | 8 +++- docs/sidb/README.md | 31 ++++++++++++-- 3 files changed, 73 insertions(+), 6 deletions(-) diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index 0afadde8..864171d9 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -40,7 +40,8 @@ package v1alpha1 import ( "strings" - "time" + "time" + "strconv" dbcommons "github.com/oracle/oracle-database-operator/commons/database" @@ -360,6 +361,43 @@ func (r *SingleInstanceDatabase) ValidateUpdate(oldRuntimeObject runtime.Object) if !ok { return nil } + + if (old.Status.Role != dbcommons.ValueUnavailable && old.Status.Role != "PRIMARY") { + // Restriciting Patching of secondary databases archiveLog, forceLog, flashBack + statusArchiveLog, _ := strconv.ParseBool(old.Status.ArchiveLog) + if statusArchiveLog != r.Spec.ArchiveLog { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("archiveLog"), "cannot be changed")) + } + statusFlashBack, _ := strconv.ParseBool(old.Status.FlashBack) + if statusFlashBack != r.Spec.FlashBack { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("flashBack"), "cannot be changed")) + } + statusForceLogging, _ := strconv.ParseBool(old.Status.ForceLogging) + if statusForceLogging != r.Spec.ForceLogging { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("forceLog"), "cannot be changed")) + } + // Restriciting Patching of secondary databases InitParams + if old.Status.InitParams.SgaTarget != r.Spec.InitParams.SgaTarget { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("InitParams").Child("sgaTarget"), "cannot be changed")) + } + if old.Status.InitParams.PgaAggregateTarget != r.Spec.InitParams.PgaAggregateTarget { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("InitParams").Child("pgaAggregateTarget"), "cannot be changed")) + } + if old.Status.InitParams.CpuCount != r.Spec.InitParams.CpuCount { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("InitParams").Child("cpuCount"), "cannot be changed")) + } + if old.Status.InitParams.Processes != r.Spec.InitParams.Processes { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("InitParams").Child("processes"), "cannot be changed")) + } + } + if old.Status.DatafilesCreated == "true" && (old.Status.PrebuiltDB != r.Spec.Image.PrebuiltDB) { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("image").Child("prebuiltDB"), "cannot be changed")) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 462a5381..97548e48 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -205,7 +205,10 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct } } + + if strings.ToUpper(singleInstanceDatabase.Status.Role) == "PRIMARY" { + // Update DB config result, err = r.updateDBConfig(singleInstanceDatabase, readyPod, ctx, req) if result.Requeue { @@ -644,7 +647,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns VolumeSource: corev1.VolumeSource{ Secret: &corev1.SecretVolumeSource{ SecretName: m.Spec.AdminPassword.SecretName, - Optional: func() *bool { i := (m.Spec.Edition != "express"); return &i }(), + Optional: func() *bool { i := (m.Spec.Edition != "express" && m.Spec.Edition != "free"); return &i }(), Items: []corev1.KeyToPath{{ Key: m.Spec.AdminPassword.SecretKey, Path: "oracle_pwd", @@ -777,7 +780,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns Name: "datamount", }) } - if m.Spec.Edition == "express" || m.Spec.Image.PrebuiltDB { + if m.Spec.Edition == "express" || m.Spec.Edition == "free" || m.Spec.Image.PrebuiltDB { // mounts pwd as secrets for express edition // or prebuilt db mounts = append(mounts, corev1.VolumeMount{ @@ -2855,6 +2858,7 @@ func ValidatePrimaryDatabaseForStandbyCreation(r *SingleInstanceDatabaseReconcil log.Info(fmt.Sprintf("Validating primary database %s configuration...", primary.Name)) err = ValidateDatabaseConfiguration(primary) if err != nil { + r.Recorder.Eventf(stdby,corev1.EventTypeWarning,"Spec Error", "all of Archivelog, Flashback and ForceLogging modes are not enabled in the primary database " + primary.Name) stdby.Status.Status = dbcommons.StatusError return err } diff --git a/docs/sidb/README.md b/docs/sidb/README.md index ccf3419e..b07b32d2 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -8,6 +8,7 @@ Oracle Database Operator for Kubernetes (`OraOperator`) includes the Single Inst * [New Database](#new-database) * [Pre-built Database](#pre-built-database) * [XE Database](#xe-database) + * [Free Database](#free-database) * [Connecting to Database](#connecting-to-database) * [Database Persistence (Storage) Configuration Options](#database-persistence-storage-configuration-options) * [Dynamic Persistence](#dynamic-persistence) @@ -205,8 +206,21 @@ This command pulls the XE image uploaded on the [Oracle Container Registry](http - For XE database, only single replica mode (i.e. `replicas: 1`) is supported. - For XE database, you **cannot change** the init parameters i.e. `cpuCount, processes, sgaTarget or pgaAggregateTarget`. +#### Free Database +To provision new Oracle Database Free database, use the sample **[config/samples/sidb/singleinstancedatabase_free.yaml](../../config/samples/sidb/singleinstancedatabase_free.yaml)** file. For example: + + kubectl apply -f singleinstancedatabase_free.yaml + +This command pulls the Free image uploaded on the [Oracle Container Registry](https://container-registry.oracle.com/). + +**NOTE:** +- Provisioning Oracle Database Free is supported for release 23c (23.2.0) and later releases. +- For Free database, only single replica mode (i.e. `replicas: 1`) is supported. +- For Free database, you **cannot change** the init parameters i.e. `cpuCount, processes, sgaTarget or pgaAggregateTarget`. +- Oracle Enterprise Manager is not supported from release 23c and later release. + #### Additional Information -You are required to specify the database admin password secret in the corresponding YAML file. The default values mentioned in the `adminPassword.secretName` fields of [singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml), [singleinstancedatabase_prebuiltdb.yaml](../../config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml) and [singleinstancedatabase_express.yaml](../../config/samples/sidb/singleinstancedatabase_express.yaml) files are `db-admin-secret`, `prebuiltdb-admin-secret`, and `xedb-admin-secret` respectively. You can create these secrets manually by using the sample command mentioned in the [Template YAML](#template-yaml) section. Alternatively, you can create these secrets by filling the passwords in the **[singleinstancedatabase_secrets.yaml](../../config/samples/sidb/singleinstancedatabase_secrets.yaml)** file and applying it using the command below: +You are required to specify the database admin password secret in the corresponding YAML file. The default values mentioned in the `adminPassword.secretName` fields of [singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml), [singleinstancedatabase_prebuiltdb.yaml](../../config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml), [singleinstancedatabase_express.yaml](../../config/samples/sidb/singleinstancedatabase_express.yaml) and [singleinstancedatabse_free.yaml](../../config/samples/sidb/singleinstancedatabase_free.yaml) files are `db-admin-secret`, `prebuiltdb-admin-secret`, `xedb-admin-secret` and `free-admin-secret` respectively. You can create these secrets manually by using the sample command mentioned in the [Template YAML](#template-yaml) section. Alternatively, you can create these secrets by filling the passwords in the **[singleinstancedatabase_secrets.yaml](../../config/samples/sidb/singleinstancedatabase_secrets.yaml)** file and applying it using the command below: ```bash kubectl apply -f singleinstancedatabase_secrets.yaml @@ -580,7 +594,7 @@ Before creating a standby, ArchiveLog, FlashBack, and ForceLog on primary Single To create a standby database, edit and apply the sample yaml file [config/samples/sidb/singleinstancedatabase_standby.yaml](../../config/samples/sidb/singleinstancedatabase_standby.yaml). **NOTE:** -- The `adminPassword` field of the above [config/samples/sidb/singleinstancedatabase_standby.yaml](../../config/samples/sidb/singleinstancedatabase_standby.yaml) contains an admin password secret of the primary database ref for Standby Database creation. This secret gets deleted after the database pod becomes ready for security reasons. +- The `adminPassword` field of the above [config/samples/sidb/singleinstancedatabase_standby.yaml](../../config/samples/sidb/singleinstancedatabase_standby.yaml) contains an admin password secret of the primary database ref for Standby Database creation. This secret will get deleted after the database pod becomes ready if the `keepSecret` attribute of `adminPassword` field is set to `false`. By default `keepSecret` is set to `true`. - Mention referred primary database in `.spec.primaryDatabaseRef` in the yaml file. - `.spec.createAsStandby` field of the yaml file should be true. - Database configuration like `Archivelog`, `FlashBack`, `ForceLog`, `TCPS connections` are not supported for standby database. @@ -757,7 +771,18 @@ $ kubectl delete dataguardbroker dgbroker-sample **NOTE :** You can only delete DataGuard broker when role of `.spec.primaryDatabaseRef` is PRIMARY -### Delete a standby database with dataguard broker configured +### Patch primary and standby databases in dataguard configuration + +Databases (both primary and standby) running in you cluster and managed by the Oracle Database operator can be patched or rolled back between release updates of the same major release. While patching databases configured with the dataguard broker you need to first patch the Primary database followed by seconday/standby databases in any order. + +To patch an existing database, edit and apply the **[config/samples/sidb/singleinstancedatabase_patch.yaml](../../config/samples/sidb/singleinstancedatabase_patch.yaml)** file of the database resource/object either by specifying a new release update for image attributes, or by running the following command: + +```sh +kubectl --type=merge -p '{"spec":{"image":{"pullFrom":"patched-image:tag","pullSecrets":"pull-secret"}}}' patch singleinstancedatabase + +``` + +### Delete a database in dataguard configuration To delete a standby database in a dataguard configuration, delete the dataguardbroker resource first followed by the standby database From 13751d94d7f00caea09baeb8087bbdb8b25f168b Mon Sep 17 00:00:00 2001 From: ishaan_desai Date: Thu, 8 Jun 2023 09:34:29 +0000 Subject: [PATCH 551/628] Sidb Bug Fixes --- .../v1alpha1/singleinstancedatabase_webhook.go | 6 ++++++ controllers/database/dataguardbroker_controller.go | 13 +++++++------ .../database/singleinstancedatabase_controller.go | 6 ++++++ 3 files changed, 19 insertions(+), 6 deletions(-) diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index 864171d9..5425d99c 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -398,6 +398,12 @@ func (r *SingleInstanceDatabase) ValidateUpdate(oldRuntimeObject runtime.Object) } } + // if Db is in a dataguard configuration. Restrict enabling Tcps on the Primary DB + if old.Status.DgBrokerConfigured && r.Spec.EnableTCPS { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("enableTCPS"), "cannot enable tcps as database is in a dataguard configuration")) + } + if old.Status.DatafilesCreated == "true" && (old.Status.PrebuiltDB != r.Spec.Image.PrebuiltDB) { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("image").Child("prebuiltDB"), "cannot be changed")) diff --git a/controllers/database/dataguardbroker_controller.go b/controllers/database/dataguardbroker_controller.go index 808311eb..96c3df0c 100644 --- a/controllers/database/dataguardbroker_controller.go +++ b/controllers/database/dataguardbroker_controller.go @@ -44,7 +44,6 @@ import ( "fmt" "strings" "time" - "github.com/go-logr/logr" corev1 "k8s.io/api/core/v1" apierrors "k8s.io/apimachinery/pkg/api/errors" @@ -543,8 +542,8 @@ func (r *DataguardBrokerReconciler) setupDataguardBrokerConfigurationForGivenDB( } log.Info("DB Admin pwd file created") + // ORA-16532: Oracle Data Guard broker configuration does not exist , so create one if strings.Contains(out, "ORA-16532") { - // ORA-16532: Oracle Data Guard broker configuration does not exist , so create one if m.Spec.ProtectionMode == "MaxPerformance" { // Construct the password file and dgbroker command file out, err := dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, false, "bash", "-c", @@ -565,7 +564,6 @@ func (r *DataguardBrokerReconciler) setupDataguardBrokerConfigurationForGivenDB( } log.Info("DgConfigurationMaxPerformance Output") log.Info(out) - } else if m.Spec.ProtectionMode == "MaxAvailability" { // ## DG CONFIGURATION FOR PRIMARY DB || MODE : MAX AVAILABILITY ## out, err := dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, false, "bash", "-c", @@ -586,7 +584,6 @@ func (r *DataguardBrokerReconciler) setupDataguardBrokerConfigurationForGivenDB( } log.Info("DgConfigurationMaxAvailability Output") log.Info(out) - } else { log.Info("SPECIFY correct Protection Mode . Either MaxAvailability or MaxPerformance") return requeueY @@ -602,10 +599,11 @@ func (r *DataguardBrokerReconciler) setupDataguardBrokerConfigurationForGivenDB( log.Info("ShowConfiguration Output") log.Info(out) } - // Set DG Configured status to true for this standbyDatabase. so that in next reconcilation, we dont configure this again + // Set DG Configured status to true for this standbyDatabase and primary Database. so that in next reconcilation, we dont configure this again + n.Status.DgBrokerConfigured = true standbyDatabase.Status.DgBrokerConfigured = true r.Status().Update(ctx, standbyDatabase) - + r.Status().Update(ctx, n) // Remove admin pwd file _, err = dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, true, "bash", "-c", dbcommons.RemoveAdminPasswordFile) @@ -1173,6 +1171,9 @@ func (r *DataguardBrokerReconciler) cleanupDataguardBroker(req ctrl.Request, ctx r.Status().Update(ctx, standbyDatabase) } + singleInstanceDatabase.Status.DgBrokerConfigured = false + r.Status().Update(ctx, singleInstanceDatabase) + log.Info("Successfully cleaned up Dataguard Broker") return requeueN, nil } diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 97548e48..ce56b5fa 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -527,6 +527,12 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab return requeueY, err } + if rp.Status.IsTcpsEnabled { + r.Recorder.Eventf(m, corev1.EventTypeWarning, "Cannot Create", "Standby for TCPS enabled Primary Database is not supported ") + m.Status.Status = dbcommons.StatusError + return requeueY, nil + } + if m.Status.DatafilesCreated == "true" || !dbcommons.IsSourceDatabaseOnCluster(m.Spec.PrimaryDatabaseRef) { return requeueN, nil From 693dd110d73c895b441305b528b358fafb90e6df Mon Sep 17 00:00:00 2001 From: Mohammed Yunus Date: Fri, 16 Jun 2023 05:39:27 +0000 Subject: [PATCH 552/628] Fix lint errors --- .../database/dataguardbroker_controller.go | 37 +++++++------- .../singleinstancedatabase_controller.go | 48 +++++++++---------- 2 files changed, 43 insertions(+), 42 deletions(-) diff --git a/controllers/database/dataguardbroker_controller.go b/controllers/database/dataguardbroker_controller.go index 96c3df0c..183a1445 100644 --- a/controllers/database/dataguardbroker_controller.go +++ b/controllers/database/dataguardbroker_controller.go @@ -44,6 +44,7 @@ import ( "fmt" "strings" "time" + "github.com/go-logr/logr" corev1 "k8s.io/api/core/v1" apierrors "k8s.io/apimachinery/pkg/api/errors" @@ -100,15 +101,15 @@ func (r *DataguardBrokerReconciler) Reconcile(ctx context.Context, req ctrl.Requ return requeueN, err } - // Manage SingleInstanceDatabase Deletion + // Manage DataguardBroker Deletion result, err := r.manageDataguardBrokerDeletion(req, ctx, dataguardBroker) if result.Requeue { r.Log.Info("Reconcile queued") return result, err } if err != nil { - r.Log.Error(err,err.Error()) - return result,err + r.Log.Error(err, err.Error()) + return result, err } // Fetch Primary Database Reference @@ -254,7 +255,7 @@ func (r *DataguardBrokerReconciler) instantiateSVCSpec(m *dbapi.DataguardBroker) Labels: map[string]string{ "app": m.Name, }, - Annotations : func() map[string]string { + Annotations: func() map[string]string { annotations := make(map[string]string) if len(m.Spec.ServiceAnnotations) != 0 { for key, value := range m.Spec.ServiceAnnotations { @@ -399,7 +400,7 @@ func (r *DataguardBrokerReconciler) setupDataguardBrokerConfiguration(m *dbapi.D _, ok := dbSet[standbyDatabase.Status.Sid] if ok { log.Info("A database with the same SID is already configured in the DG") - r.Recorder.Eventf(m, corev1.EventTypeWarning, "Spec Error", "A database with the same SID " + standbyDatabase.Status.Sid + " is already configured in the DG") + r.Recorder.Eventf(m, corev1.EventTypeWarning, "Spec Error", "A database with the same SID "+standbyDatabase.Status.Sid+" is already configured in the DG") continue } @@ -436,7 +437,7 @@ func (r *DataguardBrokerReconciler) setupDataguardBrokerConfiguration(m *dbapi.D // Update Databases r.updateReconcileStatus(m, sidbReadyPod, ctx, req) - } + } eventReason := "DG Configuration up to date" eventMsg := "" @@ -689,7 +690,7 @@ func (r *DataguardBrokerReconciler) setupDataguardBrokerConfigurationForGivenDB( } // ## SET PROPERTY FASTSTARTFAILOVERTARGET FOR EACH DATABASE TO ALL OTHER DATABASES IN DG CONFIG . - if (m.Spec.FastStartFailOver.Enable == true){ + if m.Spec.FastStartFailOver.Enable { for i := 0; i < len(databases); i++ { out, err = dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, false, "bash", "-c", fmt.Sprintf("dgmgrl sys@%s \"EDIT DATABASE %s SET PROPERTY FASTSTARTFAILOVERTARGET=%s\"< admin.pwd", primaryConnectString, @@ -730,20 +731,22 @@ func (r *DataguardBrokerReconciler) setupDataguardBrokerConfigurationForGivenDB( // ############################################################################# // -// Remove up DG Configuration for a given StandbyDatabase - To be Tested +// Remove a Database from DG Configuration // // ############################################################################# -func (r *DataguardBrokerReconciler) removeDataguardBrokerConfigurationForGivenDB( m *dbapi.DataguardBroker,n *dbapi.SingleInstanceDatabase,standbyDatabase *dbapi.SingleInstanceDatabase, standbyDatabaseReadyPod corev1.Pod, sidbReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) ctrl.Result { +// +//lint:ignore U1000 deferred for next release +func (r *DataguardBrokerReconciler) removeDatabaseFromDGConfig(m *dbapi.DataguardBroker, n *dbapi.SingleInstanceDatabase, standbyDatabase *dbapi.SingleInstanceDatabase, standbyDatabaseReadyPod corev1.Pod, sidbReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) ctrl.Result { log := r.Log.WithValues("removeDataguardBrokerConfigurationForGivenDB", req.NamespacedName) - if standbyDatabaseReadyPod.Name == "" || sidbReadyPod.Name == ""{ + if standbyDatabaseReadyPod.Name == "" || sidbReadyPod.Name == "" { return requeueY } // ## CHECK IF DG CONFIGURATION IS AVAILABLE IN PRIMARY DATABASE ## - out, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + out, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", fmt.Sprintf("echo -e \"%s\" | dgmgrl / as sysdba", dbcommons.DBShowConfigCMD)) - + if err != nil { log.Error(err, err.Error()) return requeueY @@ -757,7 +760,7 @@ func (r *DataguardBrokerReconciler) removeDataguardBrokerConfigurationForGivenDB } // ## REMOVING STANDBY DATABASE FROM DG CONFIGURATION ## - out, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + _, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", fmt.Sprintf(dbcommons.CreateDGMGRLScriptFile, dbcommons.RemoveStandbyDBFromDGConfgCMD)) if err != nil { @@ -765,8 +768,8 @@ func (r *DataguardBrokerReconciler) removeDataguardBrokerConfigurationForGivenDB return requeueY } - // ## SHOW CONFIGURATION - out, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + // ## SHOW CONFIGURATION + _, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", fmt.Sprintf("echo -e \"%s\" | dgmgrl / as sysdba", dbcommons.DBShowConfigCMD)) if err != nil { log.Error(err, err.Error()) @@ -1122,7 +1125,7 @@ func (r *DataguardBrokerReconciler) cleanupDataguardBroker(req ctrl.Request, ctx return result, nil } - // Get its Role + // Get its Role out, err := dbcommons.GetDatabaseRole(sidbReadyPod, r, r.Config, ctx, req, singleInstanceDatabase.Spec.Edition) if err != nil { log.Error(err, err.Error()) @@ -1137,7 +1140,7 @@ func (r *DataguardBrokerReconciler) cleanupDataguardBroker(req ctrl.Request, ctx } // Get Primary database to remove dataguard configuration - _, out, err = dbcommons.GetDatabasesInDgConfig(sidbReadyPod, r, r.Config, ctx, req) + _, _, err = dbcommons.GetDatabasesInDgConfig(sidbReadyPod, r, r.Config, ctx, req) if err != nil { log.Error(err, err.Error()) return requeueY, err diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index ce56b5fa..ff36d22e 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -48,6 +48,8 @@ import ( dbapi "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" dbcommons "github.com/oracle/oracle-database-operator/commons/database" + "golang.org/x/text/cases" + "golang.org/x/text/language" "github.com/go-logr/logr" corev1 "k8s.io/api/core/v1" @@ -130,7 +132,7 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct if singleInstanceDatabase.Status.Status == "" { singleInstanceDatabase.Status.Status = dbcommons.StatusPending if singleInstanceDatabase.Spec.Edition != "" { - singleInstanceDatabase.Status.Edition = strings.Title(singleInstanceDatabase.Spec.Edition) + singleInstanceDatabase.Status.Edition = cases.Title(language.English).String(singleInstanceDatabase.Spec.Edition) } else { singleInstanceDatabase.Status.Edition = dbcommons.ValueUnavailable } @@ -150,7 +152,7 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct return result, nil } if err != nil { - r.Log.Error(err,err.Error()) + r.Log.Error(err, err.Error()) return result, err } @@ -205,8 +207,6 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct } } - - if strings.ToUpper(singleInstanceDatabase.Status.Role) == "PRIMARY" { // Update DB config @@ -236,9 +236,9 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct if err != nil { return requeueY, err } - - databaseOpenMode,err := dbcommons.GetDatabaseOpenMode(readyPod, r, r.Config, ctx, req, singleInstanceDatabase.Spec.Edition) - + + databaseOpenMode, err := dbcommons.GetDatabaseOpenMode(readyPod, r, r.Config, ctx, req, singleInstanceDatabase.Spec.Edition) + if err != nil { r.Log.Error(err, err.Error()) return requeueY, err @@ -246,7 +246,7 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct r.Log.Info("DB openMode Output") r.Log.Info(databaseOpenMode) if databaseOpenMode == "READ_ONLY" { - out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c",fmt.Sprintf("echo -e \"%s\" | %s",dbcommons.ModifyStdbyDBOpenMode,dbcommons.SQLPlusCLI)) + out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", fmt.Sprintf("echo -e \"%s\" | %s", dbcommons.ModifyStdbyDBOpenMode, dbcommons.SQLPlusCLI)) if err != nil { r.Log.Error(err, err.Error()) return requeueY, err @@ -398,7 +398,7 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab // If Express/Free Edition, ensure Replicas=1 if (m.Spec.Edition == "express" || m.Spec.Edition == "free") && m.Spec.Replicas > 1 { - eventMsgs = append(eventMsgs, m.Spec.Edition + " edition supports only one replica") + eventMsgs = append(eventMsgs, m.Spec.Edition+" edition supports only one replica") } // If no persistence, ensure Replicas=1 if m.Spec.Persistence.Size == "" && m.Spec.Replicas > 1 { @@ -422,10 +422,10 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab eventMsgs = append(eventMsgs, "cloneFrom cannot be updated") } if (m.Spec.Edition == "express" || m.Spec.Edition == "free") && m.Spec.CloneFrom != "" { - eventMsgs = append(eventMsgs, "cloning not supported for " + m.Spec.Edition + " edition") + eventMsgs = append(eventMsgs, "cloning not supported for "+m.Spec.Edition+" edition") } if (m.Spec.Edition == "express" || m.Spec.Edition == "free") && m.Spec.PrimaryDatabaseRef != "" && m.Spec.CreateAsStandby { - eventMsgs = append(eventMsgs, "Standby database creation is not supported for " + m.Spec.Edition + " edition") + eventMsgs = append(eventMsgs, "Standby database creation is not supported for "+m.Spec.Edition+" edition") } if m.Status.OrdsReference != "" && m.Status.Persistence.Size != "" && m.Status.Persistence != m.Spec.Persistence { eventMsgs = append(eventMsgs, "uninstall ORDS to change Peristence") @@ -1502,7 +1502,7 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex if sid == "" || pdbName == "" || edition == "" { return requeueN, nil } - m.Status.Edition = strings.Title(edition) + m.Status.Edition = cases.Title(language.English).String(edition) } if m.Spec.LoadBalancer { @@ -2050,27 +2050,26 @@ func (r *SingleInstanceDatabaseReconciler) validateDBReadiness(m *dbapi.SingleIn m.Status.ReleaseUpdate = version } } - dbMajorVersion, err := strconv.Atoi(strings.Split(m.Status.ReleaseUpdate,".")[0]) + dbMajorVersion, err := strconv.Atoi(strings.Split(m.Status.ReleaseUpdate, ".")[0]) if err != nil { r.Log.Error(err, err.Error()) return requeueY, readyPod, err } r.Log.Info("DB Major Version is " + strconv.Itoa(dbMajorVersion)) // Validating that free edition of the database is only supported from database 23c onwards - if (m.Spec.Edition == "free" && dbMajorVersion < 23){ - r.Log.Info("Oracle Database Free is only supported from version 23c onwards") - r.Recorder.Eventf(m, corev1.EventTypeWarning, "Spec Error", "Oracle Database Free is only supported from version 23c onwards") + if m.Spec.Edition == "free" && dbMajorVersion < 23 { + errMsg := "the Oracle Database Free is only available from version 23c onwards" + r.Recorder.Eventf(m, corev1.EventTypeWarning, "Spec Error", errMsg) m.Status.Status = dbcommons.StatusError - return requeueN, readyPod, errors.New("Oracle Database Free is only supported from version 23c onwards") + return requeueN, readyPod, errors.New(errMsg) } - // Checking if OEM is supported in the provided Database version - if (dbMajorVersion >= 23 ) { + // Checking if OEM is supported in the provided Database version + if dbMajorVersion >= 23 { m.Status.OemExpressUrl = dbcommons.ValueUnavailable } else { m.Status.OemExpressUrl = oemExpressUrl } - if strings.ToUpper(m.Status.Role) == "PRIMARY" && m.Status.DatafilesPatched != "true" { eventReason := "Datapatch Pending" eventMsg := "datapatch execution pending" @@ -2647,7 +2646,6 @@ func (r *SingleInstanceDatabaseReconciler) manageSingleInstanceDatabaseDeletion( m *dbapi.SingleInstanceDatabase) (ctrl.Result, error) { log := r.Log.WithValues("manageSingleInstanceDatabaseDeletion", req.NamespacedName) - // Check if the SingleInstanceDatabase instance is marked to be deleted, which is // indicated by the deletion timestamp being set. isSingleInstanceDatabaseMarkedToBeDeleted := m.GetDeletionTimestamp() != nil @@ -2704,11 +2702,11 @@ func (r *SingleInstanceDatabaseReconciler) cleanupSingleInstanceDatabase(req ctr return requeueY, nil } - if (m.Status.DgBrokerConfigured){ + if m.Status.DgBrokerConfigured { eventReason := "Cannot Delete" - eventMsg := "Database cannot be deleted as it is present in a DataGuard Broker configuration" + eventMsg := "database cannot be deleted as it is present in a DataGuard Broker configuration" r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventReason) - return requeueY,errors.New(eventMsg) + return requeueY, errors.New(eventMsg) } // call deletePods() with zero pods in avaiable and nil readyPod to delete all pods @@ -2864,7 +2862,7 @@ func ValidatePrimaryDatabaseForStandbyCreation(r *SingleInstanceDatabaseReconcil log.Info(fmt.Sprintf("Validating primary database %s configuration...", primary.Name)) err = ValidateDatabaseConfiguration(primary) if err != nil { - r.Recorder.Eventf(stdby,corev1.EventTypeWarning,"Spec Error", "all of Archivelog, Flashback and ForceLogging modes are not enabled in the primary database " + primary.Name) + r.Recorder.Eventf(stdby, corev1.EventTypeWarning, "Spec Error", "all of Archivelog, Flashback and ForceLogging modes are not enabled in the primary database "+primary.Name) stdby.Status.Status = dbcommons.StatusError return err } From abf244da1f94fb697cb20fb88164c8930442b35e Mon Sep 17 00:00:00 2001 From: Mohammed Yunus Date: Fri, 16 Jun 2023 06:24:36 +0000 Subject: [PATCH 553/628] Add commit env --- Dockerfile | 3 +++ Makefile | 6 ++---- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/Dockerfile b/Dockerfile index ee625e1d..fc44c58b 100644 --- a/Dockerfile +++ b/Dockerfile @@ -26,6 +26,9 @@ RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 GO111MODULE=on go build -a -o manager # Use oraclelinux:8-slim as base image to package the manager binary FROM oraclelinux:8-slim +ARG CI_COMMIT_SHA CI_COMMIT_BRANCH +ENV COMMIT_SHA=${CI_COMMIT_SHA} \ + COMMIT_BRANCH=${CI_COMMIT_BRANCH} WORKDIR / COPY --from=builder /workspace/manager . RUN useradd -u 1002 nonroot diff --git a/Makefile b/Makefile index 4736ead9..e904262b 100644 --- a/Makefile +++ b/Makefile @@ -72,10 +72,8 @@ run: manifests generate fmt vet ## Run a controller from your host. go run ./main.go docker-build: manifests generate fmt vet #test ## Build docker image with the manager. Disable the test but keep the validations to fail fast - docker build --no-cache=true --build-arg http_proxy=${HTTP_PROXY} --build-arg https_proxy=${HTTPS_PROXY} . -t ${IMG} - -#docker-build-proxy: test -# docker build --build-arg http_proxy=${http_proxy} --build-arg https_proxy=${https_proxy} build . -t ${IMG} + docker build --no-cache=true --build-arg http_proxy=${HTTP_PROXY} --build-arg https_proxy=${HTTPS_PROXY} \ + --build-arg CI_COMMIT_SHA=${CI_COMMIT_SHA} --build-arg CI_COMMIT_BRANCH=${CI_COMMIT_BRANCH} . -t ${IMG} docker-push: ## Push docker image with the manager. docker push ${IMG} From c7c3242933bc8fb46243a83f54a7c470765d088d Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Fri, 16 Jun 2023 16:50:16 +0530 Subject: [PATCH 554/628] Fix log messaging --- .gitlab-ci.yml | 2 +- controllers/database/singleinstancedatabase_controller.go | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index b567f938..d5080ff9 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -5,7 +5,7 @@ build-operator: OP_YAML: oracle-database-operator.yaml script: - go version - - echo $CI_COMMIT_TAG + - echo $CI_COMMIT_SHORT_SHA - make docker-build IMG="$IMAGE" - docker push "$IMAGE" - newimage=$DOCKER_REPO@$(skopeo inspect docker://$IMAGE | jq -r .Digest) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index ff36d22e..a4baf0a9 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -2705,7 +2705,7 @@ func (r *SingleInstanceDatabaseReconciler) cleanupSingleInstanceDatabase(req ctr if m.Status.DgBrokerConfigured { eventReason := "Cannot Delete" eventMsg := "database cannot be deleted as it is present in a DataGuard Broker configuration" - r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventReason) + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) return requeueY, errors.New(eventMsg) } From ba834c07fd056713d5a958fb0835be03547ccae6 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Fri, 16 Jun 2023 14:26:03 +0000 Subject: [PATCH 555/628] Update docs/adb/ADB_MANUAL_BACKUP.md, docs/adb/README.md --- docs/adb/ADB_MANUAL_BACKUP.md | 2 +- docs/adb/README.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/adb/ADB_MANUAL_BACKUP.md b/docs/adb/ADB_MANUAL_BACKUP.md index ecf75fea..2adec8e4 100644 --- a/docs/adb/ADB_MANUAL_BACKUP.md +++ b/docs/adb/ADB_MANUAL_BACKUP.md @@ -2,7 +2,7 @@ To create manual backups of Autonomous Databases, use this procedure. -Oracle Cloud Infrastructure (OCI) automatically backs up your Autonomous Databases, and retains these backups for 60 days. You can restore and recover your database to any point-in-time in this retention period. Automatic backups are full backups taken every 60 days, with daily incremental backups. You can also create manual backups to supplement your automatic backups. Manual backups are stored in an Object Storage bucket that you create, and are retained for 60 days. For more information, please visit [this page](https://docs.oracle.com/en-us/iaas/Content/Database/Tasks/adbbackingup.htm). +Oracle Cloud Infrastructure (OCI) automatically backs up your Autonomous Databases, and retains these backups for 60 days. You can restore and recover your database to any point-in-time in this retention period. Automatic backups are full backups taken every 60 days, with daily incremental backups. You can also create manual backups for your database. Manual backups are stored in an Object Storage bucket that you create, and are retained for 60 days. Note that Oracle doesn not recommend create or use manual backups. For more information, please visit [Backing Up and Restoring Autonomous Database](https://docs.oracle.com/en-us/iaas/Content/Database/Tasks/adbbackingup.htm) and [Backup and Restore Notes](https://docs.oracle.com/en-us/iaas/autonomous-database-shared/doc/backup-restore-notes.html). ## Prerequisites diff --git a/docs/adb/README.md b/docs/adb/README.md index 1f68fe82..6f5ea7b0 100644 --- a/docs/adb/README.md +++ b/docs/adb/README.md @@ -58,7 +58,7 @@ Follow the steps to provision an Autonomous Database that will map objects in yo ![acd-id-3](/images/adb/acd-id-2.png) -3. Create a Kubernetes Secret to hold the password of the ADMIN user. +3. Create a Kubernetes Secret to hold the password of the ADMIN user. **The key and the name of the secret must be the same.** You can create this secret with the following command (as an example): From 7ed75ca11c868b42851f25f20613ce771a25a374 Mon Sep 17 00:00:00 2001 From: Mohammed Yunus Date: Mon, 19 Jun 2023 14:40:09 +0000 Subject: [PATCH 556/628] Update Dockerfile --- Dockerfile | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/Dockerfile b/Dockerfile index fc44c58b..3c249825 100644 --- a/Dockerfile +++ b/Dockerfile @@ -26,7 +26,8 @@ RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 GO111MODULE=on go build -a -o manager # Use oraclelinux:8-slim as base image to package the manager binary FROM oraclelinux:8-slim -ARG CI_COMMIT_SHA CI_COMMIT_BRANCH +ARG CI_COMMIT_SHA +ARG CI_COMMIT_BRANCH ENV COMMIT_SHA=${CI_COMMIT_SHA} \ COMMIT_BRANCH=${CI_COMMIT_BRANCH} WORKDIR / From e217d047d860165b8fcda080d7395984c0d47375 Mon Sep 17 00:00:00 2001 From: douglas_williams Date: Tue, 20 Jun 2023 18:48:33 +0000 Subject: [PATCH 557/628] Minor doc style/spelling updates, and update copyright to 2022, 2023. --- README.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index 389cd8b0..df504ba2 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ ## Make Oracle Database Kubernetes Native - Take 2 -As part of Oracle's resolution to make Oracle Database Kubernetes-native (that is, observable and operable by Kubernetes), Oracle released _Oracle Database Operator for Kubernetes_ (`OraOperator` or the operator). OraOperator extends the Kubernetes API with custom resources and controllers for automating Oracle Database lifecycle management. +As part of Oracle's resolution to make Oracle Database Kubernetes native (that is, observable and operable by Kubernetes), Oracle released _Oracle Database Operator for Kubernetes_ (`OraOperator` or the operator). OraOperator extends the Kubernetes API with custom resources and controllers for automating Oracle Database lifecycle management. In this v0.2.1 release, `OraOperator` supports the following database configurations and infrastructure: @@ -15,7 +15,7 @@ In this v0.2.1 release, `OraOperator` supports the following database configurat * Oracle Multitenant Databases (CDB/PDBs) * Oracle Database Cloud Service (DBCS) (VMDB) -Oracle will continue to extent OraOperator to support additional Oracle Database configurations. +Oracle will continue to extend `OraOperator` to support additional Oracle Database configurations. ## Features Summary @@ -50,7 +50,7 @@ Oracle strongly recommends that you ensure your system meets the following [Prer * ### Install cert-manager - The operator uses webhooks for validating user input before persisting it in Etcd. Webhooks require TLS certificates that are generated and managed by a certificate manager. + The operator uses webhooks for validating user input before persisting it in etcd. Webhooks require TLS certificates that are generated and managed by a certificate manager. Install the certificate manager with the following command: @@ -70,7 +70,7 @@ Oracle strongly recommends that you ensure your system meets the following [Prer --- **NOTE:** - The above command will also upgrade the existing v0.2.0 `OraOperator` installation to the latest version i.e. v0.2.1. + The above command will also upgrade the existing v0.2.0 `OraOperator` installation to the latest version; for example, v0.2.1. --- @@ -126,7 +126,7 @@ YAML file templates are available under [`/config/samples`](./config/samples/). kubectl delete pdb.database.oracle.com --all -n ``` - After all CRD instances are deleted, it is safe to remove the CRDs, APISerivces and operator deployment. Use the following command: + After all CRD instances are deleted, it is safe to remove the CRDs, APIServices and operator deployment. To remove these files, use the following command: ```sh kubectl delete -f oracle-database-operator.yaml --ignore-not-found=true @@ -148,7 +148,7 @@ See [Contributing to this Repository](./CONTRIBUTING.md) ## Support -You can submit a GitHub issue, or you can also file an [Oracle Support service](https://support.oracle.com/portal/) request, using the product id: 14430. +You can submit a GitHub issue, or you can also file an [Oracle Support service](https://support.oracle.com/portal/) request, using this product ID: 14430. ## Security @@ -173,5 +173,5 @@ See [Reporting security vulnerabilities](./SECURITY.md) ## License -Copyright (c) 2022 Oracle and/or its affiliates. +Copyright (c) 2022, 2023 Oracle and/or its affiliates. Released under the Universal Permissive License v1.0 as shown at [https://oss.oracle.com/licenses/upl/](https://oss.oracle.com/licenses/upl/) From dbaf4aa9b6303a3fea4c08b3b9c8b0da931a0bdb Mon Sep 17 00:00:00 2001 From: douglas_williams Date: Fri, 23 Jun 2023 02:11:01 +0000 Subject: [PATCH 558/628] Update THIRD_PARTY_LICENSES.txt --- THIRD_PARTY_LICENSES.txt | 902 +++++++++++++++++++++++++++++++++++++-- 1 file changed, 858 insertions(+), 44 deletions(-) diff --git a/THIRD_PARTY_LICENSES.txt b/THIRD_PARTY_LICENSES.txt index 792cde69..74a065b6 100644 --- a/THIRD_PARTY_LICENSES.txt +++ b/THIRD_PARTY_LICENSES.txt @@ -1,7 +1,811 @@ -Operator SDK +------------------------------------- +Operator SDK 1.26.1 https://github.com/operator-framework/operator-sdk +Apache 2.0 + +------------------------------------- + +Apache License: + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + + ------------------------------ + GO lang + https://github.com/golang + + + Copyright (c) 2009 The Go Authors. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are +met: + + * Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above +copyright notice, this list of conditions and the following disclaimer +in the documentation and/or other materials provided with the +distribution. + * Neither the name of Google Inc. nor the names of its +contributors may be used to endorse or promote products derived from +this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +------------------------- +apimachinery 0.27.2 +https://github.com/kubernetes/apimachinery/tr +Apache 2.0 + +------------------------- +controller-runtime 0.15.0 +https://github.com/kubernetes-sigs/controller-runtime/tree/v0.14.6 +Apache 2.0 + +------------------------- +golang 1.20.2 +https://github.com/golang/go/archive/refs/tags/go1.20.2.zip +BSD 2-clause or 3-clause + + +BSD 2-clause or 3-clause License: + +Copyright (c) 2009 The Go Authors. All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are +met: + + * Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above +copyright notice, this list of conditions and the following disclaimer +in the documentation and/or other materials provided with the +distribution. + * Neither the name of Google Inc. nor the names of its +contributors may be used to endorse or promote products derived from +this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +------------------------------- Copyright notices -------------------------- +-- various source files +Copyright 2021 The Go Authors. All rights reserved. +-- various go source files under src/cmd/compile/internal/ and src/cmd/link/internal/ +// Copyright © 1994-1999 Lucent Technologies Inc. All rights reserved. +// Portions Copyright © 1995-1997 C H Forsyth (forsyth@terzarima.net) +// Portions Copyright © 1997-1999 Vita Nuova Limited +// Portions Copyright © 2000-2007 Vita Nuova Holdings Limited (www.vitanuova.c\om) +// Portions Copyright © 2004,2006 Bruce Ellis +// Portions Copyright © 2005-2007 C H Forsyth (forsyth@terzarima.net) +// Revisions Copyright © 2000-2007 Lucent Technologies Inc. and others +// Portions Copyright © 2009 The Go Authors. All rights reserved. +// +// Permission is hereby granted, free of charge, to any person obtaining a copy +// of this software and associated documentation files (the "Software"), to dea\l +// in the Software without restriction, including without limitation the rights +// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +// copies of the Software, and to permit persons to whom the Software is +// furnished to do so, subject to the following conditions: +// +// The above copyright notice and this permission notice shall be included in +// all copies or substantial portions of the Software. +// +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM\, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +// THE SOFTWARE. +---------------------------- Fourth-party information ---------------------- +== NAME OF DEPENDENCY 1 +github.com/google/pprof +== License Type +Apache 2.0 +== Copyright Notices +--------------------------------(separator)--------------------------------- +== NAME OF DEPENDENCY 2 +github.com/chzyer/readline +== License Type +MIT +== Copyright Notices +The MIT License (MIT) + +Copyright (c) 2015 Chzyer + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. +--------------------------------(separator)--------------------------------- +== NAME OF DEPENDENCY 3 +github.com/chzyer/test +== License Type +MIT +== Copyright Notices +The MIT License (MIT) + +Copyright (c) 2016 chzyer + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. +--------------------------------(separator)--------------------------------- +== NAME OF DEPENDENCY 4 +github.com/chzyer/logex +== License Type +MIT +== Copyright Notices +The MIT License (MIT) + +Copyright (c) 2015 Chzyer + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. +--------------------------------(separator)--------------------------------- +== NAME OF DEPENDENCY 5 +golang.org/x/sys +== License Type +BSD 3-clause +== Copyright Notices +Copyright (c) 2009 The Go Authors. All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are +met: + + * Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above +copyright notice, this list of conditions and the following disclaimer +in the documentation and/or other materials provided with the +distribution. + * Neither the name of Google Inc. nor the names of its +contributors may be used to endorse or promote products derived from +this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +--------------------------------(separator)--------------------------------- +== NAME OF DEPENDENCY 6 +github.com/ianlancetaylor/demangle +== License Type +BSD 3-clause +== Copyright Notices +Copyright (c) 2015 The Go Authors. All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are +met: + + * Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above +copyright notice, this list of conditions and the following disclaimer +in the documentation and/or other materials provided with the +distribution. + * Neither the name of Google Inc. nor the names of its +contributors may be used to endorse or promote products derived from +this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +--------------------------------(separator)--------------------------------- +== NAME OF DEPENDENCY 7 +golang.org/x/arch +== License Type +BSD 3-clause +== Copyright Notices +Copyright (c) 2015 The Go Authors. All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are +met: + + * Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above +copyright notice, this list of conditions and the following disclaimer +in the documentation and/or other materials provided with the +distribution. + * Neither the name of Google Inc. nor the names of its +contributors may be used to endorse or promote products derived from +this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +--------------------------------(separator)--------------------------------- +== NAME OF DEPENDENCY 8 +golang.org/x/mod +== License Type +BSD 3-clause +== Copyright Notices +Copyright (c) 2009 The Go Authors. All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are +met: + + * Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above +copyright notice, this list of conditions and the following disclaimer +in the documentation and/or other materials provided with the +distribution. + * Neither the name of Google Inc. nor the names of its +contributors may be used to endorse or promote products derived from +this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +--------------------------------(separator)--------------------------------- +== NAME OF DEPENDENCY 9 +golang.org/x/sync +== License Type +BSD 3-clause +== Copyright Notices +Copyright (c) 2009 The Go Authors. All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are +met: + + * Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above +copyright notice, this list of conditions and the following disclaimer +in the documentation and/or other materials provided with the +distribution. + * Neither the name of Google Inc. nor the names of its +contributors may be used to endorse or promote products derived from +this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +--------------------------------(separator)--------------------------------- +== NAME OF DEPENDENCY 10 +golang.org/x/sys +== License Type +BSD 3-clause +== Copyright Notices +Copyright (c) 2009 The Go Authors. All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are +met: + + * Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above +copyright notice, this list of conditions and the following disclaimer +in the documentation and/or other materials provided with the +distribution. + * Neither the name of Google Inc. nor the names of its +contributors may be used to endorse or promote products derived from +this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +--------------------------------(separator)--------------------------------- +== NAME OF DEPENDENCY 11 +golang.org/x/term +== License Type +BSD 3-clause +== Copyright Notices +Copyright (c) 2009 The Go Authors. All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are +met: + + * Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above +copyright notice, this list of conditions and the following disclaimer +in the documentation and/or other materials provided with the +distribution. + * Neither the name of Google Inc. nor the names of its +contributors may be used to endorse or promote products derived from +this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +--------------------------------(separator)--------------------------------- +== NAME OF DEPENDENCY 12 +golang.org/x/tools +== License Type +BSD 3-clause +== Copyright Notices +Copyright (c) 2009 The Go Authors. All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are +met: + + * Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above +copyright notice, this list of conditions and the following disclaimer +in the documentation and/or other materials provided with the +distribution. + * Neither the name of Google Inc. nor the names of its +contributors may be used to endorse or promote products derived from +this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +--------------------------------(separator)--------------------------------- +== NAME OF DEPENDENCY 13 +golang.org/x/crypto +== License Type +BSD 3-clause +== Copyright Notices +Copyright (c) 2009 The Go Authors. All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are +met: + + * Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above +copyright notice, this list of conditions and the following disclaimer +in the documentation and/or other materials provided with the +distribution. + * Neither the name of Google Inc. nor the names of its +contributors may be used to endorse or promote products derived from +this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +--------------------------------(separator)--------------------------------- +== NAME OF DEPENDENCY 14 +github.com/mmcloughlin/avo +== License Type +BSD 3-clause +== Copyright Notices +BSD 3-Clause License + +Copyright (c) 2018, Michael McLoughlin +All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + +* Redistributions of source code must retain the above copyright notice, this + list of conditions and the following disclaimer. + +* Redistributions in binary form must reproduce the above copyright notice, + this list of conditions and the following disclaimer in the documentation + and/or other materials provided with the distribution. + +* Neither the name of the copyright holder nor the names of its + contributors may be used to endorse or promote products derived from + this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR +SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER +CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, +OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +--------------------------------(separator)--------------------------------- +== NAME OF DEPENDENCY 15 +golang.org/x/net +== License Type +BSD 3-clause +== Copyright Notices +Copyright (c) 2009 The Go Authors. All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are +met: + + * Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above +copyright notice, this list of conditions and the following disclaimer +in the documentation and/or other materials provided with the +distribution. + * Neither the name of Google Inc. nor the names of its +contributors may be used to endorse or promote products derived from +this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +--------------------------------(separator)--------------------------------- +== NAME OF DEPENDENCY 16 +golang.org/x/text +== License Type +BSD 3-clause +== Copyright Notices +Copyright (c) 2009 The Go Authors. All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are +met: + + * Redistributions of source code must retain the above copyright +notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above +copyright notice, this list of conditions and the following disclaimer +in the documentation and/or other materials provided with the +distribution. + * Neither the name of Google Inc. nor the names of its +contributors may be used to endorse or promote products derived from +this software without specific prior written permission. -Apache License +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +--------------------------------(separator)--------------------------------- +== LICENSES +. +== Text of license (Apache 2.0) + + Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ @@ -202,42 +1006,11 @@ Apache License WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. - - ------------------------------ - GO lang - https://github.com/golang - - Copyright (c) 2009 The Go Authors. - -Redistribution and use in source and binary forms, with or without -modification, are permitted provided that the following conditions are -met: - - * Redistributions of source code must retain the above copyright -notice, this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above -copyright notice, this list of conditions and the following disclaimer -in the documentation and/or other materials provided with the -distribution. - * Neither the name of Google Inc. nor the names of its -contributors may be used to endorse or promote products derived from -this software without specific prior written permission. - -THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS -"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT -LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR -A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT -OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, -SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT -LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, -DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY -THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT -(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE -OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. -------------------------- Logr https://pkg.go.dev/github.com/go-logr/logr +https://github.com/go-logr/logr/tree/v1.2.4 Apache 2.0 License ------------------------- @@ -246,6 +1019,8 @@ github.com/oracle/oci-go-sdk/v43 Dual-License: UPL + Apache 2.0 +UPL license: + Copyright (c) 2016, 2018, 2020, Oracle and/or its affiliates. This software is dual-licensed to you under the Universal Permissive License (UPL) 1.0 as shown at https://oss.oracle.com/licenses/upl @@ -287,6 +1062,23 @@ The Universal Permissive License (UPL), Version 1.0 LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + + +------------------------- +ginkgo 2.9.7 +https://github.com/onsi/ginkgo/tree/v2.9.7 +MIT +------------------------------------ +Gomega +github.com/onsi/gomega +MIT License +Copyright (c) 2013-2014 Onsi Fakhouri + +---------------------------- +gomega 1.27.7 +http://onsi.github.io/gomega/ +MIT + ------------------------- Kubernetes api https://pkg.go.dev/k8s.io/api @@ -308,6 +1100,31 @@ https://pkg.go.dev/sigs.k8s.io/controller-runtime Apache 2.0 ------------------------------------ +kubernetes-sigs/yaml 1.3.0 +https://github.com/kubernetes-sigs/yaml/tree/v1.3.0 +MIT + +------------------------- +OCI SDK for Go 65.37.1 +https://github.com/oracle/oci-go-sdk +Multiple Licenses: Apache 2.0, UPL + +------------------------------ +Operator Lifecycle Manager (OLM) +github.com/operator-framework/operator-lifecycle-manager +Apache 2.0 + + +------------------------------------ +Prometheus Operator 0.65.2 +https://github.com/prometheus-operator/prometheus-operator +Apache 2.0 + +------------------------------------ +yaml 3.0.1 +https://github.com/go-yaml/yaml/releases/tag/v3.0.1 +Dual license: Apache 2.0, MIT + YAML support for the Go language https://pkg.go.dev/gopkg.in/yaml.v2 Apache 2.0 @@ -315,8 +1132,14 @@ Apache 2.0 ------------------------------------ YAML marshaling and unmarshaling support for Go https://pkg.go.dev/sigs.k8s.io/yaml -DUal license: BSD-3-Clause, MIT +Dual license: BSD-3-Clause, MIT +------------------------------------ +zap 1.24.0 +https://github.com/uber-go/zap/tree/v1.24.0 +MIT + +------------------------------------ The MIT License (MIT) Copyright (c) 2014 Sam Ghods @@ -393,13 +1216,4 @@ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. --------------------------- -Gomega -github.com/onsi/gomega -MIT License -Copyright (c) 2013-2014 Onsi Fakhouri - ------------------------------------- -Operator Lifecycle Manager (OLM) -github.com/operator-framework/operator-lifecycle-manager -Apache 2.0 From 83f5f1574fa20621e107f701375da5f8606c3ad8 Mon Sep 17 00:00:00 2001 From: douglas_williams Date: Fri, 23 Jun 2023 11:44:55 +0000 Subject: [PATCH 559/628] Update docs/adb/README.md to reflect consistency, spelling and style changes,... --- docs/adb/README.md | 38 +++++++++++++++++++------------------- 1 file changed, 19 insertions(+), 19 deletions(-) diff --git a/docs/adb/README.md b/docs/adb/README.md index 6f5ea7b0..144f9d41 100644 --- a/docs/adb/README.md +++ b/docs/adb/README.md @@ -6,7 +6,7 @@ As indicated in the prerequisites (see above), to interact with OCI services, ei ## Required Permissions -The opeartor must be given the required type of access in a policy written by an administrator to manage the Autonomous Databases. See [Let database and fleet admins manage Autonomous Databases](https://docs.oracle.com/en-us/iaas/Content/Identity/Concepts/commonpolicies.htm#db-admins-manage-adb) for sample Autonomous Database policies. +The operator must be given the required type of access in a policy written by an administrator to manage the Autonomous Databases. See [Let database and fleet admins manage Autonomous Databases](https://docs.oracle.com/en-us/iaas/Content/Identity/Concepts/commonpolicies.htm#db-admins-manage-adb) for sample Autonomous Database policies. The permission to view the workrequests is also required, so that the operator will update the resources when the work is done. See [Viewing Work Requests](https://docs.oracle.com/en-us/iaas/Content/ContEng/Tasks/contengviewingworkrequests.htm#contengviewingworkrequests) for sample work request policies. @@ -26,15 +26,15 @@ After you create the resource, you can use the operator to perform the following * [Stop/Start/Terminate](#stopstartterminate) an Autonomous Database * [Delete the resource](#delete-the-resource) from the cluster -To debug the Oracle Autonomous Databases with Oracle Database Operator, see [Debugging and troubleshooting](#debugging-and-troubleshooting) +To debug the Oracle Autonomous Databases with Oracle Database operator, see [Debugging and troubleshooting](#debugging-and-troubleshooting) ## Provision an Autonomous Database -Follow the steps to provision an Autonomous Database that will map objects in your cluster. +Follow these steps to provision an Autonomous Database that will map objects in your cluster. 1. Get the `Compartment OCID`. - Login Cloud Console and click `Compartment`. + Log in to the Cloud Console and click `Compartment`. ![compartment-1](/images/adb/compartment-1.png) @@ -42,7 +42,7 @@ Follow the steps to provision an Autonomous Database that will map objects in yo ![compartment-2](/images/adb/compartment-2.png) -2. To create an Autonomous Database on Dedicated Exadata Infrastructure (ADB-D), the OCID of Oracle Autonomous Container Database is required. +2. To create an Autonomous Database on Dedicated Exadata Infrastructure (ADB-D), the OCID of the Oracle Autonomous Container Database is required. You can skip this step if you want to create a Autonomous Database on Shared Exadata Infrastructure (ADB-S). @@ -54,13 +54,13 @@ Follow the steps to provision an Autonomous Database that will map objects in yo ![acd-id-2](/images/adb/acd-id-1.png) - Click on the name of Autonomous Container Database and copy the `Autonomous Container Database OCID` from Cloud Console. + Click on the name of Autonomous Container Database and copy the `Autonomous Container Database OCID` from the Cloud Console. ![acd-id-3](/images/adb/acd-id-2.png) 3. Create a Kubernetes Secret to hold the password of the ADMIN user. **The key and the name of the secret must be the same.** - You can create this secret with the following command (as an example): + You can create this secret by using a command similar to the following example: ```sh kubectl create secret generic admin-password --from-literal=admin-password='password_here' @@ -111,9 +111,9 @@ Follow the steps to provision an Autonomous Database that will map objects in yo 5. Choose the type of network access (optional): - By default, the network access type is set to PUBLIC, which allows secure connections from anywhere. Uncomment the code block if you want configure the netowrk acess. See [Configuring Network Access of Autonomous Database](./NETWORK_ACCESS_OPTIONS.md) for more information. + By default, the network access type is set to PUBLIC, which allows secure connections from anywhere. Uncomment the code block if you want configure the network acess. See [Configuring Network Access of Autonomous Database](./NETWORK_ACCESS_OPTIONS.md) for more information. -6. Apply the yaml: +6. Apply the YAML: ```sh kubectl apply -f config/samples/adb/autonomousdatabase_create.yaml @@ -170,9 +170,9 @@ The operator also generates the `AutonomousBackup` custom resources if a databas ## Scale the OCPU core count or storage -> Note: this operation requires an `AutonomousDatabase` object to be in your cluster. This example assumes either the provision operation or the bind operation has been done by the users and the operator is authorized with API Key Authentication. +> Note: this operation requires an `AutonomousDatabase` object to be in your cluster. To use this example, either the provision operation or the bind operation must be done, and the operator is authorized with API Key Authentication. -Users can scale up or scale down the Oracle Autonomous Database OCPU core count or storage by updating the `cpuCoreCount` and `dataStorageSizeInTBs` parameters. The `isAutoScalingEnabled` indicates whether auto scaling is enabled. Here is an example of scaling the CPU count and storage size (TB) up to 2 and turning off the auto-scaling by updating the `autonomousdatabase-sample` custom resource. +You can scale up or scale down the Oracle Autonomous Database OCPU core count or storage by updating the `cpuCoreCount` and `dataStorageSizeInTBs` parameters. The `isAutoScalingEnabled` indicates whether auto scaling is enabled. In this example, the CPU count and storage size (TB) are scaled up to 2 and the auto-scaling is turned off by updating the `autonomousdatabase-sample` custom resource. 1. An example YAML file is available here: [config/samples/adb/autonomousdatabase_scale.yaml](./../../config/samples/adb/autonomousdatabase_scale.yaml) @@ -339,14 +339,14 @@ To use the secret in a deployment, refer to [Using Secrets](https://kubernetes.i > Note: this operation requires an `AutonomousDatabase` object to be in your cluster. This example assumes the provision operation or the bind operation has been done by the users and the operator is authorized with API Key Authentication. -Users can start/stop/terminate a database using the `lifecycleState` attribute. +To start, stop, or terminate a database, use the `lifecycleState` attribute. Here's a list of the values you can set for `lifecycleState`: * `AVAILABLE`: to start the database * `STOPPED`: to stop the database * `TERMINATED`: to terminate the database -1. A sample .yaml file is available here: [config/samples/adb/autonomousdatabase_stop_start_terminate.yaml](./../../config/samples/adb/autonomousdatabase_stop_start_terminate.yaml) +1. An example .yaml file is available here: [config/samples/adb/autonomousdatabase_stop_start_terminate.yaml](./../../config/samples/adb/autonomousdatabase_stop_start_terminate.yaml) ```yaml --- @@ -376,9 +376,9 @@ Here's a list of the values you can set for `lifecycleState`: The `hardLink` defines the behavior when the resource is deleted from the cluster. If the `hardLink` is set to true, the Operator terminates the Autonomous Database in OCI when the resource is removed; otherwise, the database remains unchanged. By default the value is `false` if it is not explicitly specified. -Follow the steps to delete the resource and terminate the Autonomous Database. +To delete the resource and terminate the Autonomous Database, complete these steps: -1. Use the example [autonomousdatabase_delete_resource.yaml](./../../config/samples/adb/autonomousdatabase_delete_resource.yaml) which sets the attribute `hardLink` to true. +1. Use the example [autonomousdatabase_delete_resource.yaml](./../../config/samples/adb/autonomousdatabase_delete_resource.yaml), which sets the attribute `hardLink` to true. ```yaml --- @@ -415,17 +415,17 @@ Now, you can verify that the database is in TERMINATING state on the Cloud Conso ### Show the details of the resource -If you edit and re-apply the `.yaml` file, the Autonomous Database controller will only update the parameters that the file contains. The parameters which are not in the file will not be impacted. To get the verbose output of the current spec, use below command: +If you edit and reapply the `.yaml` file, then the Autonomous Database controller will only update the parameters that the file contains. The parameters that are not in the file will not be updated. To obtain the verbose output of the current spec, use the following command: ```sh kubectl describe adb/autonomousdatabase-sample ``` -If any error occurs during the reconciliation loop, the Operator reports the error using the resource's event stream, which shows up in kubectl describe output. +If any error occurs during the reconciliation loop, then the operator reports the error using the resource's event stream, which shows up in kubectl describe output. ### Check the logs of the pod where the operator deploys -Follow the steps to check the logs. +To check the logs, use these steps: 1. List the pod replicas @@ -433,7 +433,7 @@ Follow the steps to check the logs. kubectl get pods -n oracle-database-operator-system ``` -2. Use the below command to check the logs of the Pod which has a failure +2. Use the following command to check the logs of the Pod that has a failure ```sh kubectl logs -f pod/oracle-database-operator-controller-manager-78666fdddb-s4xcm -n oracle-database-operator-system From 0d0985c68a2e9872527ca52ac9acb80cd58aa302 Mon Sep 17 00:00:00 2001 From: norman_japheth_aberin Date: Fri, 23 Jun 2023 15:49:44 +0000 Subject: [PATCH 560/628] Database Observer - Adding Observability controller changes --- PROJECT | 9 + .../v1alpha1/databaseobserver_types.go | 138 +++++ .../v1alpha1/groupversion_info.go | 58 ++ .../v1alpha1/zz_generated.deepcopy.go | 266 +++++++++ commons/observability/constants.go | 92 ++++ commons/observability/dashboard.go | 276 ++++++++++ commons/observability/panels.go | 328 ++++++++++++ commons/observability/utils.go | 198 +++++++ ...vability.oracle.com_databaseobservers.yaml | 225 ++++++++ config/crd/kustomization.yaml | 3 + .../cainjection_in_databaseobservers.yaml | 8 + .../patches/webhook_in_databaseobservers.yaml | 17 + config/rbac/databaseobserver_editor_role.yaml | 24 + config/rbac/databaseobserver_viewer_role.yaml | 20 + config/rbac/role.yaml | 71 +++ config/samples/kustomization.yaml | 1 + ...servability_v1alpha1_databaseobserver.yaml | 52 ++ .../databaseobserver_controller.go | 277 ++++++++++ .../databaseobserver_resource.go | 505 ++++++++++++++++++ controllers/observability/suite_test.go | 103 ++++ docs/observability/README.md | 194 +++++++ docs/observability/REQUIREMENTS.md | 56 ++ go.mod | 12 +- go.sum | 44 +- main.go | 22 +- 25 files changed, 2979 insertions(+), 20 deletions(-) create mode 100644 apis/observability/v1alpha1/databaseobserver_types.go create mode 100644 apis/observability/v1alpha1/groupversion_info.go create mode 100644 apis/observability/v1alpha1/zz_generated.deepcopy.go create mode 100644 commons/observability/constants.go create mode 100644 commons/observability/dashboard.go create mode 100644 commons/observability/panels.go create mode 100644 commons/observability/utils.go create mode 100644 config/crd/bases/observability.oracle.com_databaseobservers.yaml create mode 100644 config/crd/patches/cainjection_in_databaseobservers.yaml create mode 100644 config/crd/patches/webhook_in_databaseobservers.yaml create mode 100644 config/rbac/databaseobserver_editor_role.yaml create mode 100644 config/rbac/databaseobserver_viewer_role.yaml create mode 100644 config/samples/observability/observability_v1alpha1_databaseobserver.yaml create mode 100644 controllers/observability/databaseobserver_controller.go create mode 100644 controllers/observability/databaseobserver_resource.go create mode 100644 controllers/observability/suite_test.go create mode 100644 docs/observability/README.md create mode 100644 docs/observability/REQUIREMENTS.md diff --git a/PROJECT b/PROJECT index c402ecfd..5cbb036c 100644 --- a/PROJECT +++ b/PROJECT @@ -136,4 +136,13 @@ resources: defaulting: true validation: true webhookVersion: v1beta1 +- api: + crdVersion: v1beta1 + namespaced: true + controller: true + domain: oracle.com + group: observability + kind: DatabaseObserver + path: github.com/oracle/oracle-database-operator/apis/observability/v1alpha1 + version: v1alpha1 version: "3" diff --git a/apis/observability/v1alpha1/databaseobserver_types.go b/apis/observability/v1alpha1/databaseobserver_types.go new file mode 100644 index 00000000..a16d6043 --- /dev/null +++ b/apis/observability/v1alpha1/databaseobserver_types.go @@ -0,0 +1,138 @@ +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v1alpha1 + +import ( + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +type StatusEnum string + +// DatabaseObserverSpec defines the desired state of DatabaseObserver +type DatabaseObserverSpec struct { + // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster + // Important: Run "make" to regenerate code after modifying this file + + Database DatabaseObserverDatabase `json:"database,omitempty"` + Exporter DatabaseObserverExporterConfig `json:"exporter,omitempty"` + Prometheus PrometheusConfig `json:"prometheus,omitempty"` + Replicas int32 `json:"replicas,omitempty"` +} + +// DatabaseObserverDatabase defines the database details used for DatabaseObserver +type DatabaseObserverDatabase struct { + DBName string `json:"dbName,omitempty"` + //DBUser DBSecret `json:"dbUser,omitempty"` + //DBServiceName DBSecret `json:"dbServiceName,omitempty"` + DBPassword DBSecret `json:"dbPassword,omitempty"` + DBWallet DBSecret `json:"dbWallet,omitempty"` + DBConnectionString DBSecret `json:"dbConnectionString,omitempty"` +} + +// DatabaseObserverExporterConfig defines the configuration details related to the exporters of DatabaseObserver +type DatabaseObserverExporterConfig struct { + ExporterImage string `json:"image,omitempty"` + ExporterConfig DatabaseObserverConfigMap `json:"configuration,omitempty"` + Service DatabaseObserverService `json:"service,omitempty"` +} + +// DatabaseObserverService defines the exporter service component of DatabaseObserver +type DatabaseObserverService struct { + Port int32 `json:"port,omitempty"` +} + +// PrometheusConfig defines the generated resources for Prometheus +type PrometheusConfig struct { + Labels map[string]string `json:"labels,omitempty"` + Port string `json:"port,omitempty"` +} + +type DBSecret struct { + Key string `json:"key,omitempty"` + SecretName string `json:"secret,omitempty"` + VaultOCID string `json:"vaultOCID,omitempty"` + VaultSecretName string `json:"vaultSecretName,omitempty"` +} + +type DatabaseObserverConfigMap struct { + Configmap ConfigMapDetails `json:"configmap,omitempty"` +} + +// ConfigMapDetails defines the configmap name +type ConfigMapDetails struct { + Key string `json:"key,omitempty"` + Name string `json:"configmapName,omitempty"` +} + +// DatabaseObserverStatus defines the observed state of DatabaseObserver +type DatabaseObserverStatus struct { + // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster + // Important: Run "make" to regenerate code after modifying this file + Conditions []metav1.Condition `json:"conditions"` + Status string `json:"status,omitempty"` + ExporterConfig string `json:"exporterConfig"` + Replicas int `json:"replicas,omitempty"` +} + +//+kubebuilder:object:root=true +//+kubebuilder:subresource:status + +// DatabaseObserver is the Schema for the databaseobservers API +// +kubebuilder:printcolumn:JSONPath=".status.exporterConfig",name="ExporterConfig",type=string +// +kubebuilder:printcolumn:JSONPath=".status.status",name="Status",type=string +type DatabaseObserver struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + Spec DatabaseObserverSpec `json:"spec,omitempty"` + Status DatabaseObserverStatus `json:"status,omitempty"` +} + +//+kubebuilder:object:root=true + +// DatabaseObserverList contains a list of DatabaseObserver +type DatabaseObserverList struct { + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata,omitempty"` + Items []DatabaseObserver `json:"items"` +} + +func init() { + SchemeBuilder.Register(&DatabaseObserver{}, &DatabaseObserverList{}) +} diff --git a/apis/observability/v1alpha1/groupversion_info.go b/apis/observability/v1alpha1/groupversion_info.go new file mode 100644 index 00000000..304840f4 --- /dev/null +++ b/apis/observability/v1alpha1/groupversion_info.go @@ -0,0 +1,58 @@ +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +// Package v1alpha1 contains API Schema definitions for the observability v1alpha1 API group +// +kubebuilder:object:generate=true +// +groupName=observability.oracle.com +package v1alpha1 + +import ( + "k8s.io/apimachinery/pkg/runtime/schema" + "sigs.k8s.io/controller-runtime/pkg/scheme" +) + +var ( + // GroupVersion is group version used to register these objects + GroupVersion = schema.GroupVersion{Group: "observability.oracle.com", Version: "v1alpha1"} + + // SchemeBuilder is used to add go types to the GroupVersionKind scheme + SchemeBuilder = &scheme.Builder{GroupVersion: GroupVersion} + + // AddToScheme adds the types in this group-version to the given scheme. + AddToScheme = SchemeBuilder.AddToScheme +) diff --git a/apis/observability/v1alpha1/zz_generated.deepcopy.go b/apis/observability/v1alpha1/zz_generated.deepcopy.go new file mode 100644 index 00000000..3c298bdd --- /dev/null +++ b/apis/observability/v1alpha1/zz_generated.deepcopy.go @@ -0,0 +1,266 @@ +//go:build !ignore_autogenerated +// +build !ignore_autogenerated + +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +// Code generated by controller-gen. DO NOT EDIT. + +package v1alpha1 + +import ( + "k8s.io/apimachinery/pkg/apis/meta/v1" + runtime "k8s.io/apimachinery/pkg/runtime" +) + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *ConfigMapDetails) DeepCopyInto(out *ConfigMapDetails) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ConfigMapDetails. +func (in *ConfigMapDetails) DeepCopy() *ConfigMapDetails { + if in == nil { + return nil + } + out := new(ConfigMapDetails) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DBSecret) DeepCopyInto(out *DBSecret) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DBSecret. +func (in *DBSecret) DeepCopy() *DBSecret { + if in == nil { + return nil + } + out := new(DBSecret) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DatabaseObserver) DeepCopyInto(out *DatabaseObserver) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + in.Spec.DeepCopyInto(&out.Spec) + in.Status.DeepCopyInto(&out.Status) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserver. +func (in *DatabaseObserver) DeepCopy() *DatabaseObserver { + if in == nil { + return nil + } + out := new(DatabaseObserver) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *DatabaseObserver) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DatabaseObserverConfigMap) DeepCopyInto(out *DatabaseObserverConfigMap) { + *out = *in + out.Configmap = in.Configmap +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserverConfigMap. +func (in *DatabaseObserverConfigMap) DeepCopy() *DatabaseObserverConfigMap { + if in == nil { + return nil + } + out := new(DatabaseObserverConfigMap) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DatabaseObserverDatabase) DeepCopyInto(out *DatabaseObserverDatabase) { + *out = *in + out.DBPassword = in.DBPassword + out.DBWallet = in.DBWallet + out.DBConnectionString = in.DBConnectionString +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserverDatabase. +func (in *DatabaseObserverDatabase) DeepCopy() *DatabaseObserverDatabase { + if in == nil { + return nil + } + out := new(DatabaseObserverDatabase) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DatabaseObserverExporterConfig) DeepCopyInto(out *DatabaseObserverExporterConfig) { + *out = *in + out.ExporterConfig = in.ExporterConfig + out.Service = in.Service +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserverExporterConfig. +func (in *DatabaseObserverExporterConfig) DeepCopy() *DatabaseObserverExporterConfig { + if in == nil { + return nil + } + out := new(DatabaseObserverExporterConfig) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DatabaseObserverList) DeepCopyInto(out *DatabaseObserverList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]DatabaseObserver, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserverList. +func (in *DatabaseObserverList) DeepCopy() *DatabaseObserverList { + if in == nil { + return nil + } + out := new(DatabaseObserverList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *DatabaseObserverList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DatabaseObserverService) DeepCopyInto(out *DatabaseObserverService) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserverService. +func (in *DatabaseObserverService) DeepCopy() *DatabaseObserverService { + if in == nil { + return nil + } + out := new(DatabaseObserverService) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DatabaseObserverSpec) DeepCopyInto(out *DatabaseObserverSpec) { + *out = *in + out.Database = in.Database + out.Exporter = in.Exporter + in.Prometheus.DeepCopyInto(&out.Prometheus) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserverSpec. +func (in *DatabaseObserverSpec) DeepCopy() *DatabaseObserverSpec { + if in == nil { + return nil + } + out := new(DatabaseObserverSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DatabaseObserverStatus) DeepCopyInto(out *DatabaseObserverStatus) { + *out = *in + if in.Conditions != nil { + in, out := &in.Conditions, &out.Conditions + *out = make([]v1.Condition, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserverStatus. +func (in *DatabaseObserverStatus) DeepCopy() *DatabaseObserverStatus { + if in == nil { + return nil + } + out := new(DatabaseObserverStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PrometheusConfig) DeepCopyInto(out *PrometheusConfig) { + *out = *in + if in.Labels != nil { + in, out := &in.Labels, &out.Labels + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PrometheusConfig. +func (in *PrometheusConfig) DeepCopy() *PrometheusConfig { + if in == nil { + return nil + } + out := new(PrometheusConfig) + in.DeepCopyInto(out) + return out +} diff --git a/commons/observability/constants.go b/commons/observability/constants.go new file mode 100644 index 00000000..42e484a8 --- /dev/null +++ b/commons/observability/constants.go @@ -0,0 +1,92 @@ +package observability + +import "github.com/oracle/oracle-database-operator/apis/observability/v1alpha1" + +const UnknownValue = "UNKNOWN" + +// Observability Status +const ( + StatusObservabilityPending v1alpha1.StatusEnum = "PENDING" + StatusObservabilityError v1alpha1.StatusEnum = "ERROR" + StatusObservabilityReady v1alpha1.StatusEnum = "READY" + StatusObservabilityTerminating v1alpha1.StatusEnum = "TERMINATING" +) + +// ConditionTypes +const ( + PhaseReconcile = "Reconcile" + PhaseControllerValidation = "Validation" + PhaseExportersDeploy = "CreateExportersDeployment" + PhaseExportersDeployValidation = "ValidateExportersDeployment" + PhaseExportersDeployUpdate = "UpdateExportersDeployment" + PhaseExportersSVC = "CreateExporterService" + PhaseExportersServiceMonitor = "CreateExporterServiceMonitor" + PhaseExportersGrafanaConfigMap = "CreateGrafanaConfigMap" + PhaseExportersConfigMap = "CreateExporterConfigurationConfigMap" +) + +// Reason +const ( + ReasonInitStart = "InitializationStarted" + ReasonInitSuccess = "InitializationSucceeded" + ReasonValidationFailure = "ValidationFailed" + ReasonSetFailure = "SetResourceFailed" + ReasonCreateFailure = "CreateResourceFailed" + ReasonCreateSuccess = "CreateResourceSucceeded" + ReasonUpdateSuccess = "UpdateResourceSucceeded" + ReasonUpdateFailure = "UpdateResourceFailed" + ReasonRetrieveFailure = "RetrieveResourceFailed" + ReasonRetrieveSuccess = "RetrieveResourceSucceeded" +) + +// Log Names +const ( + LogReconcile = "LogReconcile" + LogControllerValidation = "LogValidation" + LogExportersDeploy = "LogCreateExportersDeployment" + LogExportersSVC = "LogCreateExporterService" + LogExportersServiceMonitor = "LogCreateExporterServiceMonitor" + LogExportersGrafanaConfigMap = "LogCreateGrafanaConfigMap" + LogExportersConfigMap = "LogCreateExporterConfigurationConfigMap" +) +const ( + DefaultDbUserKey = "user" + DefaultDBServiceNameKey = "sn" + DefaultDBPasswordKey = "password" + DefaultDBConnectionStringKey = "access" + DefaultDBWalletMountPath = "/creds" +) +const ( + DefaultExporterImage = "container-registry.oracle.com/database/observability-exporter:0.1.0" + DefaultServicePort = 9161 + DefaultPrometheusPort = "metrics" + DefaultReplicaCount = 1 + DefaultConfig = "[[metric]]\ncontext = \"sessions\"\nlabels = [\"inst_id\", \"status\", \"type\"]\nmetricsdesc = { value = \"Gauge metric with count of sessions by status and type.\" }\nrequest = '''\nSELECT\n inst_id,\n status,\n type,\n COUNT(*) AS value\nFROM\n gv$session\nGROUP BY\n status,\n type,\n inst_id\n'''\n\n" + + DefaultExporterConfigMountRootPath = "/observability" + DefaultConfigurationConfigmapKey = "config.toml" + DefaultExporterConfigmapPath = "config.toml" + DefaultExporterConfigMountPathFull = "/observability/config.toml" + DefaultGrafanaConfigmapKey = "dashboard.json" + DefaultExporterConfigmapPrefix = "obs-cm-" + DefaultServicemonitorPrefix = "obs-servicemonitor-" +) + +const ( + DefaultLabelKey = "app" + DefaultLabelPrefix = "obs-" + DefaultGrafanaConfigMapNamePrefix = "obs-json-dash-" + DefaultExporterDeploymentPrefix = "obs-deploy-" + DefaultExporterContainerName = "observability-exporter" +) + +// Known environment variables +const ( + EnvVarDataSourceServiceName = "DATA_SOURCE_SERVICENAME" + EnvVarDataSourceUser = "DATA_SOURCE_USER" + EnvVarDataSourcePassword = "DATA_SOURCE_PASSWORD" + EnvVarDataSourceName = "DATA_SOURCE_NAME" + EnvVarDataSourcePwdVaultSecretName = "vault_secret_name" + EnvVarDataSourcePwdVaultId = "vault_id" + EnvVarCustomConfigmap = "CUSTOM_METRICS" +) diff --git a/commons/observability/dashboard.go b/commons/observability/dashboard.go new file mode 100644 index 00000000..4e138bc1 --- /dev/null +++ b/commons/observability/dashboard.go @@ -0,0 +1,276 @@ +package observability + +type Annotation struct { + List []List `json:"list"` +} + +type List struct { + BuiltIn int `json:"builtIn"` + Datasource string `json:"datasource"` + Enable bool `json:"enable"` + Hide bool `json:"hide"` + IconColor string `json:"iconColor"` + Limit int `json:"limit"` + Name string `json:"name"` + Type string `json:"type"` +} + +type Legend struct { + AlignAsTable bool `json:"alignAsTable,omitempty"` + Avg bool `json:"avg,omitempty"` + Current bool `json:"current,omitempty"` + Max bool `json:"max,omitempty"` + Min bool `json:"min,omitempty"` + Show bool `json:"show,omitempty"` + Total bool `json:"total,omitempty"` + Values bool `json:"values,omitempty"` + Calcs []string `json:"calcs,omitempty"` + DisplayMode string `json:"displayMode,omitempty"` + Placement string `json:"placement,omitempty"` +} + +type ReduceOptions struct { + Calcs []string `json:"calcs"` + Fields string `json:"fields"` + Values bool `json:"values"` +} + +type Options struct { + Num0 struct { + Text string `json:"text,omitempty"` + } `json:"0,omitempty"` + Num1 struct { + Text string `json:"text"` + } `json:"1,omitempty"` + ColorMode string `json:"colorMode,omitempty"` + GraphMode string `json:"graphMode,omitempty"` + JustifyMode string `json:"justifyMode,omitempty"` + Orientation string `json:"orientation,omitempty"` + ReduceOptions ReduceOptions `json:"reduceOptions,omitempty"` + Text struct{} `json:"text"` + TextMode string `json:"textMode,omitempty"` + AlertThreshold bool `json:"alertThreshold,omitempty"` + Legend Legend `json:"legend,omitempty"` + Tooltip Tooltip `json:"tooltip,omitempty"` +} + +type GridPos struct { + H int `json:"h"` + W int `json:"w"` + X int `json:"x"` + Y int `json:"y"` +} + +type Color struct { + Mode string `json:"mode"` +} + +type Mappings struct { + Options Options `json:"options"` + Type string `json:"type"` +} + +type Thresholds struct { + Mode string `json:"mode"` + Steps []Steps `json:"steps"` +} + +type Steps struct { + Color string `json:"color"` + Value interface{} `json:"value,omitempty"` +} +type Defaults struct { + Color Color `json:"color"` + Mappings []Mappings `json:"mappings"` + Custom Custom `json:"custom"` + Thresholds Thresholds `json:"thresholds"` + Unit string `json:"unit"` +} + +type HideFrom struct { + Legend bool `json:"legend"` + Tooltip bool `json:"tooltip"` + Viz bool `json:"viz"` +} + +type ScaleDistribution struct { + Type string `json:"type"` +} + +type Stacking struct { + Group string `json:"group"` + Mode string `json:"mode"` +} + +type ThresholdStyle struct { + Mode string `json:"mode"` +} + +type Custom struct { + AxisLabel string `json:"axisLabel"` + AxisPlacement string `json:"axisPlacement"` + BarAlignment int `json:"barAlignment"` + DrawStyle string `json:"drawStyle"` + FillOpacity int `json:"fillOpacity"` + GradientMode string `json:"gradientMode"` + HideFrom HideFrom `json:"hideFrom"` + LineInterpolation string `json:"lineInterpolation"` + LineWidth int `json:"lineWidth"` + PointSize int `json:"pointSize"` + ScaleDistribution ScaleDistribution `json:"scaleDistribution"` + ShowPoints string `json:"showPoints"` + SpanNulls bool `json:"spanNulls"` + Stacking Stacking `json:"stacking"` + ThresholdsStyle ThresholdStyle `json:"thresholdsStyle"` +} + +type FieldConfig struct { + Defaults Defaults `json:"defaults"` + Overrides []interface{} `json:"overrides"` +} + +type Targets struct { + Exemplar bool `json:"exemplar"` + Expr string `json:"expr"` + Interval string `json:"interval"` + IntervalFactor int `json:"intervalFactor,omitempty"` + LegendFormat string `json:"legendFormat"` + RefID string `json:"refId"` +} + +type Tooltip struct { + Shared bool `json:"shared,omitempty"` + Sort int `json:"sort,omitempty"` + ValueType string `json:"value_type,omitempty"` + Mode string `json:"mode,omitempty"` +} + +type XAxis struct { + Buckets interface{} `json:"buckets"` + Mode string `json:"mode"` + Name interface{} `json:"name"` + Show bool `json:"show"` + Values []interface{} `json:"values"` +} + +type YAxis struct { + Align bool `json:"align"` + AlignLevel interface{} `json:"alignLevel"` +} + +type YAxes struct { + Format string `json:"format"` + Label interface{} `json:"label"` + LogBase int `json:"logBase"` + Max interface{} `json:"max"` + Min interface{} `json:"min"` + Show bool `json:"show"` +} + +type Panels struct { + Collapsed bool `json:"collapsed,omitempty"` + Datasource interface{} `json:"datasource"` + GridPos GridPos `json:"gridPos"` + ID int `json:"id"` + Panels []interface{} `json:"panels,omitempty"` + Title string `json:"title"` + Type string `json:"type"` + CacheTimeout interface{} `json:"cacheTimeout,omitempty"` + FieldConfig FieldConfig `json:"fieldConfig,omitempty"` + Interval interface{} `json:"interval,omitempty"` + Links []interface{} `json:"links,omitempty"` + MaxDataPoints int `json:"maxDataPoints,omitempty"` + Options Options `json:"options,omitempty"` + PluginVersion string `json:"pluginVersion,omitempty"` + Targets []Targets `json:"targets,omitempty"` + TimeFrom interface{} `json:"timeFrom,omitempty"` + TimeShift interface{} `json:"timeShift,omitempty"` + Description string `json:"description,omitempty"` + AliasColors struct { + } `json:"aliasColors,omitempty"` + Bars bool `json:"bars,omitempty"` + DashLength int `json:"dashLength,omitempty"` + Dashes bool `json:"dashes,omitempty"` + Fill int `json:"fill,omitempty"` + FillGradient int `json:"fillGradient,omitempty"` + HiddenSeries bool `json:"hiddenSeries,omitempty"` + Legend Legend `json:"legend,omitempty"` + Lines bool `json:"lines,omitempty"` + Linewidth int `json:"linewidth,omitempty"` + NullPointMode string `json:"nullPointMode,omitempty"` + Percentage bool `json:"percentage,omitempty"` + Pointradius int `json:"pointradius,omitempty"` + Points bool `json:"points,omitempty"` + Renderer string `json:"renderer,omitempty"` + SeriesOverrides []interface{} `json:"seriesOverrides,omitempty"` + SpaceLength int `json:"spaceLength,omitempty"` + Stack bool `json:"stack,omitempty"` + SteppedLine bool `json:"steppedLine,omitempty"` + Thresholds []interface{} `json:"thresholds,omitempty"` + TimeRegions []interface{} `json:"timeRegions,omitempty"` + Tooltip Tooltip `json:"tooltip,omitempty"` + Xaxis XAxis `json:"xaxis,omitempty"` + Yaxes []YAxes `json:"yaxes,omitempty"` + Yaxis YAxis `json:"yaxis,omitempty"` + HideTimeOverride bool `json:"hideTimeOverride,omitempty"` + Alert struct { + AlertRuleTags struct { + } `json:"alertRuleTags"` + Conditions []struct { + Evaluator struct { + Params []float64 `json:"params"` + Type string `json:"type"` + } `json:"evaluator"` + Operator struct { + Type string `json:"type"` + } `json:"operator"` + Query struct { + Params []string `json:"params"` + } `json:"query"` + Reducer struct { + Params []interface{} `json:"params"` + Type string `json:"type"` + } `json:"reducer"` + Type string `json:"type"` + } `json:"conditions"` + ExecutionErrorState string `json:"executionErrorState"` + For string `json:"for"` + Frequency string `json:"frequency"` + Handler int `json:"handler"` + Message string `json:"message"` + Name string `json:"name"` + NoDataState string `json:"noDataState"` + Notifications []interface{} `json:"notifications"` + } `json:"alert,omitempty"` + Transformations []interface{} `json:"transformations,omitempty"` +} + +type Dashboard struct { + Annotations Annotation `json:"annotations"` + Description string `json:"description"` + Editable bool `json:"editable"` + GnetID int `json:"gnetId"` + GraphTooltip int `json:"graphTooltip"` + ID int `json:"id"` + Links []interface{} `json:"links"` + Panels []Panels `json:"panels"` + Refresh bool `json:"refresh"` + SchemaVersion int `json:"schemaVersion"` + Style string `json:"style"` + Tags []interface{} `json:"tags"` + Templating struct { + List []interface{} `json:"list"` + } `json:"templating"` + Time struct { + From string `json:"from"` + To string `json:"to"` + } `json:"time"` + Timepicker struct { + RefreshIntervals []string `json:"refresh_intervals"` + TimeOptions []string `json:"time_options"` + } `json:"timepicker"` + Timezone string `json:"timezone"` + Title string `json:"title"` + UID string `json:"uid"` + Version int `json:"version"` +} diff --git a/commons/observability/panels.go b/commons/observability/panels.go new file mode 100644 index 00000000..70e5626c --- /dev/null +++ b/commons/observability/panels.go @@ -0,0 +1,328 @@ +package observability + +func generateEventMesh() Panels { + return Panels{ + Title: "Event Mesh (Message Propagation)", + Type: "row", + Panels: nil, + ID: 84, + Collapsed: true, + Datasource: nil, + GridPos: GridPos{H: 1, W: 24, X: 0, Y: 75}, + } +} + +func generateCPULoad() Panels { + return Panels{ + Title: "CPU Load", + Datasource: "Prometheus", + AliasColors: struct{}{}, + Bars: false, + DashLength: 10, + Dashes: false, + Description: "", + Fill: 1, + FillGradient: 0, + GridPos: GridPos{H: 6, W: 7, X: 0, Y: 57}, + HiddenSeries: false, + ID: 64, + Legend: Legend{ + AlignAsTable: true, + Avg: true, + Current: true, + Max: true, + Min: true, + Show: true, + Total: false, + Values: true, + }, + Lines: true, + Linewidth: 1, + Links: nil, + NullPointMode: "null", + Options: Options{ + AlertThreshold: true, + }, + Percentage: true, + PluginVersion: "8.1.5", + Pointradius: 5, + Points: false, + Renderer: "flot", + SeriesOverrides: nil, + SpaceLength: 10, + Stack: false, + SteppedLine: false, + + Targets: []Targets{{ + Exemplar: true, + Expr: "base_cpu_systemLoadAverage", + Interval: "", + IntervalFactor: 2, + LegendFormat: "CPU Load", + RefID: "A", + }}, + Thresholds: nil, + TimeFrom: nil, + TimeRegions: nil, + TimeShift: nil, + Tooltip: Tooltip{ + Shared: true, + Sort: 0, + ValueType: "individual", + }, + Xaxis: XAxis{ + Buckets: nil, + Mode: "time", + Name: nil, + Show: true, + Values: nil, + }, + Yaxes: []YAxes{{ + Format: "percentunit", + Label: nil, + LogBase: 1, + Max: nil, + Min: nil, + Show: true, + }, { + Format: "short", + Label: nil, + LogBase: 1, + Max: nil, + Min: nil, + Show: false, + }}, + Yaxis: YAxis{ + Align: false, + AlignLevel: nil, + }, + } +} + +func generateDBStatus(dbName string) Panels { + return Panels{ + CacheTimeout: nil, + Datasource: "Prometheus", + FieldConfig: FieldConfig{ + Defaults: Defaults{ + Color: Color{ + Mode: "thresholds", + }, + Mappings: []Mappings{ + { + Options: Options{ + Num0: struct { + Text string `json:"text,omitempty"` + }{Text: "DOWN"}, + Num1: struct { + Text string `json:"text"` + }{Text: "UP"}, + }, + Type: "value", + }, + }, + Thresholds: Thresholds{ + Mode: "absolute", + Steps: []Steps{ + { + Color: "#d44a3a", + Value: nil, + }, + { + Color: "#37872D", + Value: 1, + }, + { + Color: "#FADE2A", + }, + }}, + Unit: "none", + }, + Overrides: nil, + }, + GridPos: GridPos{H: 2, W: 4, X: 12, Y: 49}, + ID: 59, + Interval: nil, + Links: nil, + MaxDataPoints: 100, + Options: Options{ + ColorMode: "background", + GraphMode: "none", + JustifyMode: "auto", + Orientation: "horizontal", + ReduceOptions: ReduceOptions{ + Calcs: []string{"lastNotNull"}, + Fields: "", + Values: false, + }, + Text: struct{}{}, + TextMode: "auto", + }, + PluginVersion: "8.1.5", + Targets: []Targets{ + { + Exemplar: true, + Expr: "oracledb_up{job=\"db-metrics-exporter-" + dbName + "\"}", + Interval: "", + LegendFormat: "", + RefID: "A", + }, + }, + TimeFrom: nil, + TimeShift: nil, + Title: "DB Status", + Transformations: nil, + Type: "stat", + } +} + +func generateDBSessions(dbName string) Panels { + return Panels{ + Title: "DB Sessions", + Type: "timeseries", + Targets: []Targets{{ + Exemplar: true, + Expr: "oracledb_" + dbName + "_sessions_value", + Interval: "", + LegendFormat: "", + RefID: "A", + }}, + ID: 35, + Options: Options{ + Legend: Legend{ + Calcs: nil, + DisplayMode: "list", + Placement: "bottom", + }, + Tooltip: Tooltip{Mode: "single"}, + }, + GridPos: GridPos{H: 8, W: 12, X: 12, Y: 27}, + Datasource: nil, + FieldConfig: FieldConfig{ + Defaults: Defaults{ + Color: Color{Mode: "palette-classic"}, + Custom: Custom{ + AxisLabel: "", + AxisPlacement: "auto", + BarAlignment: 0, + DrawStyle: "line", + FillOpacity: 0, + GradientMode: "none", + HideFrom: HideFrom{ + Legend: false, + Tooltip: false, + Viz: false, + }, + LineInterpolation: "linear", + LineWidth: 1, + PointSize: 5, + ScaleDistribution: ScaleDistribution{ + Type: "linear", + }, + ShowPoints: "auto", + SpanNulls: false, + Stacking: Stacking{ + Group: "A", + Mode: "none", + }, + ThresholdsStyle: ThresholdStyle{ + Mode: "off", + }, + }, + Mappings: nil, + Thresholds: Thresholds{ + Mode: "absolute", + Steps: []Steps{{ + Color: "green", + Value: nil, + }, { + Color: "red", + Value: 80, + }}, + }, + }, + Overrides: nil, + }, + } +} + +func GenerateDashboard(dbName string) *Dashboard { + pEventMesh := generateEventMesh() + pCPULoad := generateCPULoad() + var panels = []Panels{pEventMesh, pCPULoad} + + if dbName != "" { + pDBStatus := generateDBStatus(dbName) + pDBSessions := generateDBSessions(dbName) + panels = append(panels, pDBSessions) + panels = append(panels, pDBStatus) + } + + return &Dashboard{ + Annotations: Annotation{ + List: []List{{ + BuiltIn: 1, + Datasource: "Prometheus", + Enable: true, + Hide: true, + IconColor: "rgba(0, 211, 255, 1)", + Limit: 100, + Name: "Annotations & Alerts", + Type: "dashboard", + }}, + }, + Description: "Monitoring", + Editable: true, + GnetID: 0, + GraphTooltip: 0, + ID: 0, + Links: nil, + Panels: panels, + Refresh: false, + SchemaVersion: 30, + Style: "dark", + Tags: nil, + Templating: struct { + List []interface{} `json:"list"` + }{}, + Time: struct { + From string `json:"from"` + To string `json:"to"` + }{ + From: "now-1m", + To: "now", + }, + Timepicker: struct { + RefreshIntervals []string `json:"refresh_intervals"` + TimeOptions []string `json:"time_options"` + }{ + RefreshIntervals: []string{ + "5s", + "10s", + "30s", + "1m", + "5m", + "15m", + "30m", + "1h", + "2h", + "1d", + }, + TimeOptions: []string{ + "5m", + "15m", + "1h", + "6h", + "12h", + "24h", + "2d", + "7d", + "30d", + }, + }, + Timezone: "browser", + Title: "Dashboard", + UID: "", + Version: 1, + } +} diff --git a/commons/observability/utils.go b/commons/observability/utils.go new file mode 100644 index 00000000..921ea7f1 --- /dev/null +++ b/commons/observability/utils.go @@ -0,0 +1,198 @@ +package observability + +import ( + "encoding/json" + apiv1 "github.com/oracle/oracle-database-operator/apis/observability/v1alpha1" + corev1 "k8s.io/api/core/v1" +) + +func GetGrafanaJSONData(api *apiv1.DatabaseObserver) ([]byte, error) { + dbName := api.Spec.Database.DBName + dash := GenerateDashboard(dbName) + j, err := json.Marshal(dash) + if err != nil { + return nil, err + } + return j, nil + +} + +func GetExporterLabels(api *apiv1.DatabaseObserver) map[string]string { + var l = make(map[string]string) + if labels := api.Spec.Prometheus.Labels; labels != nil && len(labels) > 0 { + for k, v := range labels { + l[k] = v + } + l["release"] = "stable" + return l + } + return map[string]string{ + DefaultLabelKey: DefaultLabelPrefix + api.Name, + "release": "stable", + } + +} + +func GetExporterServicePort(api *apiv1.DatabaseObserver) int32 { + if rPort := api.Spec.Exporter.Service.Port; rPort != 0 { + return rPort + } + return int32(DefaultServicePort) +} + +func GetExporterServiceMonitorPort(api *apiv1.DatabaseObserver) string { + if rPort := api.Spec.Prometheus.Port; rPort != "" { + return rPort + } + return DefaultPrometheusPort + +} +func GetExporterDeploymentVolumeMounts(api *apiv1.DatabaseObserver) []corev1.VolumeMount { + + volM := make([]corev1.VolumeMount, 0) + rConfigMountPath := DefaultExporterConfigMountRootPath + + volM = append(volM, corev1.VolumeMount{ + Name: "config-volume", + MountPath: rConfigMountPath, + }) + + // api.Spec.Database.DBWallet.SecretName optional + // if null, consider the database NON-ADB and connect as such + if secretName := api.Spec.Database.DBWallet.SecretName; secretName != "" { + volM = append(volM, corev1.VolumeMount{ + Name: "creds", + MountPath: DefaultDBWalletMountPath, + }) + } + return volM +} + +func GetExporterDeploymentVolumes(api *apiv1.DatabaseObserver) []corev1.Volume { + + vol := make([]corev1.Volume, 0) + cVolumeSourceName := api.Spec.Exporter.ExporterConfig.Configmap.Name + cVolumeSourceKey := api.Spec.Exporter.ExporterConfig.Configmap.Key + rVolumeSourceName := DefaultExporterConfigmapPrefix + api.Name + + // config-volume Volume + var nameToUse = rVolumeSourceName + var keyToUse = DefaultConfigurationConfigmapKey + + if cVolumeSourceName != "" { // override + nameToUse = cVolumeSourceName + } + if cVolumeSourceKey != "" { // override + keyToUse = cVolumeSourceKey + } + cMSource := &corev1.ConfigMapVolumeSource{ + LocalObjectReference: corev1.LocalObjectReference{ + Name: nameToUse, + }, + Items: []corev1.KeyToPath{{ + Key: keyToUse, + Path: DefaultExporterConfigmapPath, + }}, + } + vol = append(vol, corev1.Volume{Name: "config-volume", VolumeSource: corev1.VolumeSource{ConfigMap: cMSource}}) + + // creds Volume + // api.Spec.Database.DBWallet.SecretName optional + // if null, consider the database NON-ADB and connect as such + if secretName := api.Spec.Database.DBWallet.SecretName; secretName != "" { + + vol = append(vol, corev1.Volume{ + Name: "creds", + VolumeSource: corev1.VolumeSource{ + Secret: &corev1.SecretVolumeSource{ + SecretName: secretName, + }, + }, + }) + } + return vol +} + +func GetExporterSelector(api *apiv1.DatabaseObserver) map[string]string { + var s = make(map[string]string) + if labels := api.Spec.Prometheus.Labels; labels != nil && len(labels) > 0 { + for k, v := range labels { + s[k] = v + } + return s + + } + return map[string]string{DefaultLabelKey: DefaultLabelPrefix + api.Name} + +} + +func GetExporterEnvs(api *apiv1.DatabaseObserver) []corev1.EnvVar { + var e = make([]corev1.EnvVar, 0) + + rDBPasswordKey := api.Spec.Database.DBPassword.Key + rDBPasswordName := api.Spec.Database.DBPassword.SecretName + optional := true + + // DEFAULT_METRICS environment variable + e = append(e, corev1.EnvVar{ + Name: "DEFAULT_METRICS", + Value: DefaultExporterConfigMountPathFull, + }) + + // DATA_SOURCE_SERVICENAME environment variable + // DATA_SOURCE_USER environment variable + // DATA_SOURCE_NAME environment variable + + rDBConnStringSKey := api.Spec.Database.DBConnectionString.Key + rDBConnStringSName := api.Spec.Database.DBConnectionString.SecretName + + if rDBConnStringSKey == "" { // overwrite + rDBConnStringSKey = DefaultDBConnectionStringKey + } + dbConnectionString := &corev1.EnvVarSource{ + SecretKeyRef: &corev1.SecretKeySelector{ + Key: rDBConnStringSKey, + LocalObjectReference: corev1.LocalObjectReference{Name: rDBConnStringSName}, + Optional: &optional, + }} + e = append(e, corev1.EnvVar{Name: EnvVarDataSourceName, ValueFrom: dbConnectionString}) + + // DB PASSWORD environment variable + // DATA_SOURCE_PASSWORD environment variable + if rDBPasswordKey == "" { // overwrite + rDBPasswordKey = DefaultDBPasswordKey + } + dbPassword := &corev1.EnvVarSource{ + SecretKeyRef: &corev1.SecretKeySelector{ + Key: rDBPasswordKey, + LocalObjectReference: corev1.LocalObjectReference{Name: rDBPasswordName}, + Optional: &optional, + }} + + e = append(e, corev1.EnvVar{Name: EnvVarDataSourcePassword, ValueFrom: dbPassword}) + + // TNS_ADMIN environment variable + if secretName := api.Spec.Database.DBWallet.SecretName; secretName != "" { + e = append(e, corev1.EnvVar{ + Name: "TNS_ADMIN", + Value: DefaultDBWalletMountPath, + }) + } + + return e +} + +func GetExporterReplicas(api *apiv1.DatabaseObserver) int32 { + if rc := api.Spec.Replicas; rc != 0 { + return rc + } + return int32(DefaultReplicaCount) +} + +func GetExporterImage(api *apiv1.DatabaseObserver) string { + if img := api.Spec.Exporter.ExporterImage; img != "" { + return img + } + return DefaultExporterImage + +} diff --git a/config/crd/bases/observability.oracle.com_databaseobservers.yaml b/config/crd/bases/observability.oracle.com_databaseobservers.yaml new file mode 100644 index 00000000..bb663594 --- /dev/null +++ b/config/crd/bases/observability.oracle.com_databaseobservers.yaml @@ -0,0 +1,225 @@ + +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.6.1 + creationTimestamp: null + name: databaseobservers.observability.oracle.com +spec: + group: observability.oracle.com + names: + kind: DatabaseObserver + listKind: DatabaseObserverList + plural: databaseobservers + singular: databaseobserver + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .status.exporterConfig + name: ExporterConfig + type: string + - jsonPath: .status.status + name: Status + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + description: DatabaseObserver is the Schema for the databaseobservers API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: DatabaseObserverSpec defines the desired state of DatabaseObserver + properties: + database: + description: DatabaseObserverDatabase defines the database details + used for DatabaseObserver + properties: + dbConnectionString: + properties: + key: + type: string + secret: + type: string + type: object + dbName: + type: string + dbPassword: + properties: + key: + type: string + secret: + type: string + type: object + dbServiceName: + properties: + key: + type: string + secret: + type: string + type: object + dbUser: + properties: + key: + type: string + secret: + type: string + type: object + dbWallet: + properties: + key: + type: string + secret: + type: string + type: object + type: object + exporter: + description: DatabaseObserverExporterConfig defines the configuration + details related to the exporters of DatabaseObserver + properties: + configuration: + properties: + configmap: + description: ConfigMapDetails defines the configmap name + properties: + configmapName: + type: string + key: + type: string + type: object + type: object + image: + type: string + service: + description: DatabaseObserverService defines the exporter service + component of DatabaseObserver + properties: + port: + format: int32 + type: integer + type: object + type: object + prometheus: + description: PrometheusConfig defines the generated resources for + Prometheus + properties: + labels: + additionalProperties: + type: string + type: object + port: + type: string + type: object + replicas: + format: int32 + type: integer + type: object + status: + description: DatabaseObserverStatus defines the observed state of DatabaseObserver + properties: + conditions: + description: 'INSERT ADDITIONAL STATUS FIELD - define observed state + of cluster Important: Run "make" to regenerate code after modifying + this file' + items: + description: "Condition contains details for one aspect of the current + state of this API Resource. --- This struct is intended for direct + use as an array at the field path .status.conditions. For example, + \n \ttype FooStatus struct{ \t // Represents the observations + of a foo's current state. \t // Known .status.conditions.type + are: \"Available\", \"Progressing\", and \"Degraded\" \t // + +patchMergeKey=type \t // +patchStrategy=merge \t // +listType=map + \t // +listMapKey=type \t Conditions []metav1.Condition + `json:\"conditions,omitempty\" patchStrategy:\"merge\" patchMergeKey:\"type\" + protobuf:\"bytes,1,rep,name=conditions\"` \n \t // other fields + \t}" + properties: + lastTransitionTime: + description: lastTransitionTime is the last time the condition + transitioned from one status to another. This should be when + the underlying condition changed. If that is not known, then + using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: message is a human readable message indicating + details about the transition. This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: observedGeneration represents the .metadata.generation + that the condition was set based upon. For instance, if .metadata.generation + is currently 12, but the .status.conditions[x].observedGeneration + is 9, the condition is out of date with respect to the current + state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: reason contains a programmatic identifier indicating + the reason for the condition's last transition. Producers + of specific condition types may define expected values and + meanings for this field, and whether the values are considered + a guaranteed API. The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + --- Many .condition.type values are consistent across resources + like Available, but because arbitrary conditions can be useful + (see .node.status.conditions), the ability to deconflict is + important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + exporterConfig: + type: string + replicas: + type: integer + status: + type: string + required: + - conditions + - exporterConfig + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] diff --git a/config/crd/kustomization.yaml b/config/crd/kustomization.yaml index c411e3ed..b9f3aa8c 100644 --- a/config/crd/kustomization.yaml +++ b/config/crd/kustomization.yaml @@ -17,6 +17,7 @@ resources: - bases/database.oracle.com_autonomouscontainerdatabases.yaml - bases/database.oracle.com_dbcssystems.yaml - bases/database.oracle.com_dataguardbrokers.yaml +- bases/observability.oracle.com_databaseobservers.yaml # +kubebuilder:scaffold:crdkustomizeresource patchesStrategicMerge: @@ -32,6 +33,7 @@ patchesStrategicMerge: #- patches/webhook_in_autonomouscontainerdatabases.yaml #- patches/webhook_in_dbcssystems.yaml #- patches/webhook_in_dataguardbrokers.yaml +#- patches/webhook_in_databaseobservers.yaml # +kubebuilder:scaffold:crdkustomizewebhookpatch # [CERTMANAGER] To enable webhook, uncomment all the sections with [CERTMANAGER] prefix. @@ -46,6 +48,7 @@ patchesStrategicMerge: #- patches/cainjection_in_autonomouscontainerdatabases.yaml #- patches/cainjection_in_dbcssystems.yaml #- patches/cainjection_in_dataguardbrokers.yaml +#- patches/cainjection_in_databaseobservers.yaml # +kubebuilder:scaffold:crdkustomizecainjectionpatch # the following config is for teaching kustomize how to do kustomization for CRDs. diff --git a/config/crd/patches/cainjection_in_databaseobservers.yaml b/config/crd/patches/cainjection_in_databaseobservers.yaml new file mode 100644 index 00000000..bef0b6c0 --- /dev/null +++ b/config/crd/patches/cainjection_in_databaseobservers.yaml @@ -0,0 +1,8 @@ +# The following patch adds a directive for certmanager to inject CA into the CRD +# CRD conversion requires k8s 1.13 or later. +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + annotations: + cert-manager.io/inject-ca-from: $(CERTIFICATE_NAMESPACE)/$(CERTIFICATE_NAME) + name: databaseobservers.observability.oracle.com diff --git a/config/crd/patches/webhook_in_databaseobservers.yaml b/config/crd/patches/webhook_in_databaseobservers.yaml new file mode 100644 index 00000000..40fa3960 --- /dev/null +++ b/config/crd/patches/webhook_in_databaseobservers.yaml @@ -0,0 +1,17 @@ +# The following patch enables conversion webhook for CRD +# CRD conversion requires k8s 1.13 or later. +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: databaseobservers.observability.oracle.com +spec: + conversion: + strategy: Webhook + webhookClientConfig: + # this is "\n" used as a placeholder, otherwise it will be rejected by the apiserver for being blank, + # but we're going to set it later using the cert-manager (or potentially a patch if not using cert-manager) + caBundle: Cg== + service: + namespace: system + name: webhook-service + path: /convert diff --git a/config/rbac/databaseobserver_editor_role.yaml b/config/rbac/databaseobserver_editor_role.yaml new file mode 100644 index 00000000..b6451f85 --- /dev/null +++ b/config/rbac/databaseobserver_editor_role.yaml @@ -0,0 +1,24 @@ +# permissions for end users to edit databaseobservers. +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: databaseobserver-editor-role +rules: +- apiGroups: + - observability.oracle.com + resources: + - databaseobservers + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - observability.oracle.com + resources: + - databaseobservers/status + verbs: + - get diff --git a/config/rbac/databaseobserver_viewer_role.yaml b/config/rbac/databaseobserver_viewer_role.yaml new file mode 100644 index 00000000..e3ba538d --- /dev/null +++ b/config/rbac/databaseobserver_viewer_role.yaml @@ -0,0 +1,20 @@ +# permissions for end users to view databaseobservers. +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: databaseobserver-viewer-role +rules: +- apiGroups: + - observability.oracle.com + resources: + - databaseobservers + verbs: + - get + - list + - watch +- apiGroups: + - observability.oracle.com + resources: + - databaseobservers/status + verbs: + - get diff --git a/config/rbac/role.yaml b/config/rbac/role.yaml index 89b630dd..ab77777b 100644 --- a/config/rbac/role.yaml +++ b/config/rbac/role.yaml @@ -6,6 +6,23 @@ metadata: creationTimestamp: null name: manager-role rules: +- apiGroups: + - "" + resources: + - configmaps + - deployments + - events + - pods + - secrets + - services + verbs: + - create + - delete + - get + - list + - patch + - update + - watch - apiGroups: - "" resources: @@ -81,6 +98,21 @@ rules: - patch - update - watch +- apiGroups: + - apps + resources: + - configmaps + - deployments + - pods + - services + verbs: + - create + - delete + - get + - list + - patch + - update + - watch - apiGroups: - apps resources: @@ -428,3 +460,42 @@ rules: - get - patch - update +- apiGroups: + - monitoring.coreos.com + resources: + - prometheusrules + - servicemonitors + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - observability.oracle.com + resources: + - databaseobservers + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - observability.oracle.com + resources: + - databaseobservers/finalizers + verbs: + - update +- apiGroups: + - observability.oracle.com + resources: + - databaseobservers/status + verbs: + - get + - patch + - update diff --git a/config/samples/kustomization.yaml b/config/samples/kustomization.yaml index 0ad50fcf..93687749 100644 --- a/config/samples/kustomization.yaml +++ b/config/samples/kustomization.yaml @@ -17,4 +17,5 @@ resources: - sharding/sharding_v1alpha1_provshard.yaml - dbcs/database_v1alpha1_dbcssystem.yaml - database_v1alpha1_dataguardbroker.yaml +- observability_v1alpha1_databaseobserver.yaml # +kubebuilder:scaffold:manifestskustomizesamples diff --git a/config/samples/observability/observability_v1alpha1_databaseobserver.yaml b/config/samples/observability/observability_v1alpha1_databaseobserver.yaml new file mode 100644 index 00000000..7412794f --- /dev/null +++ b/config/samples/observability/observability_v1alpha1_databaseobserver.yaml @@ -0,0 +1,52 @@ +apiVersion: observability.oracle.com/v1alpha1 +kind: DatabaseObserver +metadata: + name: database-observer-sample +spec: + database: + + # Required + dbName: "" + # Required + dbPassword: + key: "password" + secret: db-password + dbConnectionString: + key: "access" + secret: db-connect-string + + # Optional + # This secret contains the wallet files and must be provided if you + # are connecting to your database with a wallet + dbWallet: + secret: instance-wallet + + + # Configuration + exporter: + # Optional + # image: "" + + # Optional + # configuration: + # configmap: + # key: "" + # configmapName: "" + + # Provides configuration to the port used by the exporter service + # Optional + service: + port: 9161 + + prometheus: + # Provides configuration to the port used in the service monitor + # Optional + port: metrics + # Used by servicemonitors, deployment and service + # Optional + labels: + app: app-sample-label + + # Provides configuration to the exporter replicas created + # Optional + replicas: 1 diff --git a/controllers/observability/databaseobserver_controller.go b/controllers/observability/databaseobserver_controller.go new file mode 100644 index 00000000..5777a5a3 --- /dev/null +++ b/controllers/observability/databaseobserver_controller.go @@ -0,0 +1,277 @@ +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package controllers + +import ( + "context" + "errors" + appsv1 "k8s.io/api/apps/v1" + corev1 "k8s.io/api/core/v1" + "k8s.io/apimachinery/pkg/api/meta" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/types" + "k8s.io/client-go/tools/record" + "time" + + "github.com/go-logr/logr" + "k8s.io/apimachinery/pkg/runtime" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" + + apiv1 "github.com/oracle/oracle-database-operator/apis/observability/v1alpha1" + constants "github.com/oracle/oracle-database-operator/commons/observability" +) + +// DatabaseObserverReconciler reconciles a DatabaseObserver object +type DatabaseObserverReconciler struct { + client.Client + Log logr.Logger + Scheme *runtime.Scheme + Recorder record.EventRecorder +} + +//+kubebuilder:rbac:groups=observability.oracle.com,resources=databaseobservers,verbs=get;list;watch;create;update;patch;delete +//+kubebuilder:rbac:groups=observability.oracle.com,resources=databaseobservers/status,verbs=get;update;patch +//+kubebuilder:rbac:groups=observability.oracle.com,resources=databaseobservers/finalizers,verbs=update +//+kubebuilder:rbac:groups=apps,resources=pods;deployments;services;configmaps,verbs=get;list;watch;create;update;patch;delete +//+kubebuilder:rbac:groups="",resources=pods;deployments;services;secrets;configmaps;events,verbs=get;list;watch;create;update;patch;delete +//+kubebuilder:rbac:groups=monitoring.coreos.com,resources=prometheusrules;servicemonitors,verbs=get;list;watch;create;update;patch;delete + +// Reconcile is part of the main kubernetes reconciliation loop which aims to +// move the current state of the cluster closer to the desired state. +// TODO(user): Modify the Reconcile function to compare the state specified by +// the DatabaseObserver object against the actual cluster state, and then +// perform operations to make the cluster state reflect the state specified by +// the user. +// +// For more details, check Reconcile and its Result here: +// - https://pkg.go.dev/sigs.k8s.io/controller-runtime@v0.6.4/pkg/reconcile +func (r *DatabaseObserverReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + log := r.Log.WithValues(constants.LogReconcile, req.NamespacedName) + log.Info("Begin Reconcile", "NamespacedName", req.NamespacedName) + + cr := &apiv1.DatabaseObserver{} // Fetch CR + if err := r.Get(context.TODO(), req.NamespacedName, cr); err != nil { + log.Error(err, "Failed to fetch Observability CRD") + return ctrl.Result{}, err + } + + defer r.validateReadiness(cr, ctx, req) + + if err := r.initialize(ctx, cr, req); err != nil { + return ctrl.Result{}, err + } + + if err := r.validateSpecs(cr); err != nil { + meta.SetStatusCondition(&cr.Status.Conditions, metav1.Condition{ + Type: constants.PhaseControllerValidation, + Status: metav1.ConditionFalse, + Reason: constants.ReasonValidationFailure, + Message: err.Error(), + }) + r.updateStatusAndRetrieve(cr, ctx, req) + return ctrl.Result{}, err + } + + if res, err := r.manageConfigmap(cr, ctx, req); err != nil { + return res, err + } + + exporterDeployment := &ObservabilityDeploymentResource{} + if res, err := r.manageResource(exporterDeployment, cr, ctx, req); err != nil { + return res, err + } + if err := r.compareOrUpdateDeployment(exporterDeployment, cr, ctx, req); err != nil { + return ctrl.Result{}, err + } + + exporterService := &ObservabilityServiceResource{} + if res, err := r.manageResource(exporterService, cr, ctx, req); err != nil { + return res, err + } + + exporterServiceMonitor := &ObservabilityServiceMonitorResource{} + if res, err := r.manageResource(exporterServiceMonitor, cr, ctx, req); err != nil { + return res, err + } + + exporterGrafanaOutputCM := &ObservabilityGrafanaJsonResource{} + if res, err := r.manageResource(exporterGrafanaOutputCM, cr, ctx, req); err != nil { + return res, err + } + + if !r.isExporterDeploymentReady(cr, ctx, req) { + return ctrl.Result{Requeue: true, RequeueAfter: 15 * time.Second}, nil + } + return ctrl.Result{}, nil +} + +func (r *DatabaseObserverReconciler) initialize(ctx context.Context, cr *apiv1.DatabaseObserver, req ctrl.Request) error { + + if cr.Status.Conditions == nil || len(cr.Status.Conditions) == 0 { + + meta.SetStatusCondition(&cr.Status.Conditions, metav1.Condition{ + Type: string(constants.StatusObservabilityReady), + Status: metav1.ConditionUnknown, + Reason: constants.ReasonInitStart, + Message: "Starting reconciliation", + }) + + cr.Status.Status = string(constants.StatusObservabilityPending) + cr.Status.ExporterConfig = constants.UnknownValue + + r.updateStatusAndRetrieve(cr, ctx, req) + r.Recorder.Event(cr, corev1.EventTypeNormal, constants.ReasonInitSuccess, "Initialization of observability resource completed") + } + + return nil +} + +func (r *DatabaseObserverReconciler) validateSpecs(api *apiv1.DatabaseObserver) error { + + // Does DB Password Secret Name have a value + // if all secretName, vaultOCID and vaultSecretName are empty, then error out + if api.Spec.Database.DBPassword.SecretName == "" && + api.Spec.Database.DBPassword.VaultOCID == "" && + api.Spec.Database.DBPassword.VaultSecretName == "" { + return errors.New("spec validation failed due to required field dbPassword not having a value") + } + + // Does DB Password Secret Name actually exist if vault fields are not used? + if api.Spec.Database.DBPassword.VaultOCID == "" || api.Spec.Database.DBPassword.VaultSecretName == "" { + dbSecret := &corev1.Secret{} + if err := r.Get(context.TODO(), types.NamespacedName{Name: api.Spec.Database.DBPassword.SecretName, Namespace: api.Namespace}, dbSecret); err != nil { + r.Log.Error(err, "Failed to find secret", "Name", api.Spec.Database.DBPassword.SecretName, "Namespace", api.Namespace) + return err + } + } + + //sn := api.Spec.Database.DBServiceName.SecretName + //user := api.Spec.Database.DBUser.SecretName + connStr := api.Spec.Database.DBConnectionString.SecretName + + // Does ConnectionString have a value + //var snAndUserCannotBeUsed = sn == "" || user == "" + + if connStr == "" { + + return errors.New("spec validation failed due to required fields dbConnectionString or dbServiceName and dbUser not having a value") + + } else { + + // Does DB Connection String Secret Name actually exist + dbConnectSecret := &corev1.Secret{} + if err := r.Get(context.TODO(), types.NamespacedName{Name: api.Spec.Database.DBConnectionString.SecretName, Namespace: api.Namespace}, dbConnectSecret); err != nil { + r.Log.Error(err, "Failed to find secret", "Name", api.Spec.Database.DBConnectionString.SecretName, "Namespace", api.Namespace) + return err + } + + } + + // Did the user provide a custom configuration configmap + // If so, does it actually exist + configurationCMName := api.Spec.Exporter.ExporterConfig.Configmap.Name + if configurationCMName != "" { + + configurationCM := &corev1.ConfigMap{} + if err := r.Get(context.TODO(), types.NamespacedName{Name: configurationCMName, Namespace: api.Namespace}, configurationCM); err != nil { + r.Log.Error(err, "Failed to find configuration config map", "Name", configurationCMName, "Namespace", api.Namespace) + return err + } + } + + meta.RemoveStatusCondition(&api.Status.Conditions, constants.PhaseControllerValidation) // Clear any validation conditions + r.Log.Info("Completed reconcile validation") + return nil // valid, did not encounter any errors +} + +func (r *DatabaseObserverReconciler) validateReadiness(api *apiv1.DatabaseObserver, ctx context.Context, req ctrl.Request) { + + // get latest object + if err := r.Get(context.TODO(), req.NamespacedName, api); err != nil { + r.Log.Error(err, "Failed to fetch updated CR") + } + + // evaluate if there exists conditions present that invalidates readiness + if api.Status.Conditions != nil && len(api.Status.Conditions) > 1 { + meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + Type: string(constants.StatusObservabilityReady), + Status: metav1.ConditionFalse, + Reason: "DeploymentUnsuccessful", + Message: "Observability exporter deployment failure", + }) + api.Status.Status = string(constants.StatusObservabilityError) + } else { + meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + Type: string(constants.StatusObservabilityReady), + Status: metav1.ConditionTrue, + Reason: "ResourceReady", + Message: "Observability exporter deployed successfully", + }) + api.Status.Status = string(constants.StatusObservabilityReady) + } + + if err := r.Status().Update(ctx, api); err != nil { + r.Log.Error(err, "Failed to update resource status") + } + +} + +func (r *DatabaseObserverReconciler) updateStatusAndRetrieve(api *apiv1.DatabaseObserver, ctx context.Context, req ctrl.Request) { + + // make update + if err := r.Status().Update(ctx, api); err != nil { + r.Log.Error(err, "Failed to update resource status") + } + + // refresh cr before changes + if err := r.Get(context.TODO(), req.NamespacedName, api); err != nil { + r.Log.Error(err, "Failed to fetch updated CR") + } + +} + +// SetupWithManager sets up the controller with the Manager. +func (r *DatabaseObserverReconciler) SetupWithManager(mgr ctrl.Manager) error { + return ctrl.NewControllerManagedBy(mgr). + For(&apiv1.DatabaseObserver{}). + Owns(&appsv1.Deployment{}). + Owns(&corev1.Service{}). + Complete(r) +} diff --git a/controllers/observability/databaseobserver_resource.go b/controllers/observability/databaseobserver_resource.go new file mode 100644 index 00000000..a3d3a7a3 --- /dev/null +++ b/controllers/observability/databaseobserver_resource.go @@ -0,0 +1,505 @@ +package controllers + +import ( + "context" + apiv1 "github.com/oracle/oracle-database-operator/apis/observability/v1alpha1" + "github.com/oracle/oracle-database-operator/commons/observability" + monitorv1 "github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring/v1" + appsv1 "k8s.io/api/apps/v1" + corev1 "k8s.io/api/core/v1" + "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/api/meta" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/runtime/schema" + "k8s.io/apimachinery/pkg/types" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" +) + +/* +This handler file contains all the methods that +retrieve/find and create all related resources +on Kubernetes. +*/ + +type ObservabilityConfigResource struct{} +type ObservabilityDeploymentResource struct{} +type ObservabilityServiceResource struct{} +type ObservabilityGrafanaJsonResource struct{} +type ObservabilityServiceMonitorResource struct{} + +type ObservabilityResource interface { + generate(*apiv1.DatabaseObserver, *runtime.Scheme) (*unstructured.Unstructured, error) + identify() (string, string, schema.GroupVersionKind) +} + +func (r *DatabaseObserverReconciler) isExporterDeploymentReady(api *apiv1.DatabaseObserver, ctx context.Context, req ctrl.Request) bool { + + // get latest deployment + dep := &appsv1.Deployment{} + rName := observability.DefaultExporterDeploymentPrefix + api.Name + + // defer update for status changes below + defer r.updateStatusAndRetrieve(api, ctx, req) + if err := r.Get(context.TODO(), types.NamespacedName{Name: rName, Namespace: api.Namespace}, dep); err != nil { + r.Log.Error(err, "Failed to fetch deployment for validating readiness") + meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + Type: observability.PhaseExportersDeployValidation, + Status: metav1.ConditionFalse, + Reason: observability.ReasonRetrieveFailure, + Message: "Failed to retrieve deployment for validation of observability exporter deployment readiness", + }) + return false + } + + labels := dep.Spec.Template.Labels + cLabels := client.MatchingLabels{} + for k, v := range labels { + cLabels[k] = v + } + + pods := &corev1.PodList{} + if err := r.List(context.TODO(), pods, []client.ListOption{cLabels}...); err != nil { + r.Log.Error(err, "Failed to fetch list of observability exporter pods") + meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + Type: observability.PhaseExportersDeployValidation, + Status: metav1.ConditionFalse, + Reason: observability.ReasonRetrieveFailure, + Message: "Failed to retrieve pods for validation of observability exporter deployment readiness", + }) + return false + } + + for _, pod := range pods.Items { + if pod.Status.Phase != corev1.PodRunning { + r.Log.Info("Observability exporter pod(s) found but not ready") + meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + Type: observability.PhaseExportersDeployValidation, + Status: metav1.ConditionFalse, + Reason: observability.ReasonValidationFailure, + Message: "Validation of observability exporter deployment readiness failed", + }) + return false + } + } + + meta.RemoveStatusCondition(&api.Status.Conditions, observability.PhaseExportersDeployValidation) + return true +} + +func (r *DatabaseObserverReconciler) compareOrUpdateDeployment(or ObservabilityResource, api *apiv1.DatabaseObserver, ctx context.Context, req ctrl.Request) error { + var updated bool + + // get desired + desiredObj, genErr := or.generate(api, r.Scheme) + if genErr != nil { + r.Log.WithName(observability.LogExportersDeploy).Error(genErr, "Failed to generate deployment") + return genErr + } + desiredDeployment := &appsv1.Deployment{} + if err := r.Scheme.Convert(desiredObj, desiredDeployment, nil); err != nil { + r.Log.WithName(observability.LogExportersDeploy).Error(genErr, "Failed to convert generated deployment") + return err + } + + // defer for updates to status below + defer r.updateStatusAndRetrieve(api, ctx, req) + // retrieve existing deployment + foundDeployment := &appsv1.Deployment{} + if err := r.Get(context.TODO(), types.NamespacedName{Name: desiredObj.GetName(), Namespace: req.Namespace}, foundDeployment); err != nil { + r.Log.WithName(observability.LogExportersDeploy).Error(genErr, "Failed to retrieve deployment") + meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + Type: observability.PhaseExportersDeployUpdate, + Status: metav1.ConditionFalse, + Reason: observability.ReasonRetrieveFailure, + Message: "Failed to retrieve deployment: " + desiredObj.GetName(), + }) + return err + } + + // check containerImage + foundContainerImage := foundDeployment.Spec.Template.Spec.Containers[0].Image + desiredContainerImage := desiredDeployment.Spec.Template.Spec.Containers[0].Image + if foundContainerImage != desiredContainerImage { + r.Log.WithName(observability.LogExportersDeploy).Info("Updating deployment container image") + foundDeployment.Spec.Template.Spec.Containers[0].Image = desiredContainerImage + updated = true + } + + // check config-volume + var foundConfigmapName, desiredConfigmapName string + desiredVolumes := desiredDeployment.Spec.Template.Spec.Volumes + for _, v := range desiredVolumes { + if v.Name == "config-volume" { + desiredConfigmapName = v.ConfigMap.Name + } + } + + foundVolumes := foundDeployment.Spec.Template.Spec.Volumes + for _, v := range foundVolumes { + if v.Name == "config-volume" { + foundConfigmapName = v.ConfigMap.Name + } + } + + if desiredConfigmapName != foundConfigmapName { + r.Log.WithName(observability.LogExportersDeploy).Info("Updating deployment volumes with new configuration configmap") + foundDeployment.Spec.Template.Spec.Volumes = observability.GetExporterDeploymentVolumes(api) + updated = true + } + + // make the update + if updated { + if err := r.Update(context.TODO(), foundDeployment); err != nil { + r.Log.WithName(observability.LogExportersDeploy).Error(err, "Failed to update deployment", "name", desiredObj.GetName()) + r.Recorder.Event(api, corev1.EventTypeWarning, observability.ReasonUpdateFailure, "Failed to update deployment: "+desiredObj.GetName()) + return err + } + + if desiredConfigmapName != foundConfigmapName { // if configmap was updated + api.Status.ExporterConfig = desiredConfigmapName + } + r.Recorder.Event(api, corev1.EventTypeNormal, observability.ReasonUpdateSuccess, "Succeeded updating deployment: "+desiredObj.GetName()) + } + + meta.RemoveStatusCondition(&api.Status.Conditions, observability.PhaseExportersDeployUpdate) // Clear any conditions + return nil +} + +func (r *DatabaseObserverReconciler) manageConfigmap(api *apiv1.DatabaseObserver, ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + + // retrieve spec information related to resource, + cName := api.Spec.Exporter.ExporterConfig.Configmap.Name + rName := observability.DefaultExporterConfigmapPrefix + api.Name + rKey := observability.DefaultConfigurationConfigmapKey + metricsConfigData := observability.DefaultConfig + + var nameToFind string + if cName != "" { + nameToFind = cName + } else { + nameToFind = rName + } + + // generate + crOwnedConfigmap := &corev1.ConfigMap{ + ObjectMeta: metav1.ObjectMeta{ + Name: rName, + Namespace: api.Namespace, + }, + Data: map[string]string{ + rKey: metricsConfigData, + }, + } + + // defer for updates to status below + defer r.updateStatusAndRetrieve(api, ctx, req) + + // set CR as owner + if err := controllerutil.SetControllerReference(api, crOwnedConfigmap, r.Scheme); err != nil { + r.Log.WithName(observability.LogExportersConfigMap).Error(err, "Failed to set controller reference for configmap", "name", nameToFind) + meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + Type: observability.PhaseExportersConfigMap, + Status: metav1.ConditionFalse, + Reason: observability.ReasonSetFailure, + Message: "Failed to set owner reference for configmap", + }) + return ctrl.Result{}, err + } + + // find resource + foundConfigmap := &corev1.ConfigMap{} + getErr := r.Get(ctx, types.NamespacedName{Name: nameToFind, Namespace: req.Namespace}, foundConfigmap) + + // if resource not found and custom configmap was not provided, create resource + if getErr != nil && errors.IsNotFound(getErr) && cName == "" { + r.Log.WithName(observability.LogExportersConfigMap).Info("Creating exporter configuration configmap resource", "name", nameToFind) + if err := r.Create(context.TODO(), crOwnedConfigmap); err != nil { // create + r.Log.WithName(observability.LogExportersConfigMap).Error(err, "Failed to create configmap", "name", nameToFind) + r.Recorder.Event(api, corev1.EventTypeWarning, observability.ReasonCreateFailure, "Failed creating configmap: "+rName) + + meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + Type: observability.PhaseExportersConfigMap, + Status: metav1.ConditionFalse, + Reason: observability.ReasonCreateFailure, + Message: "Failed to create configmap " + nameToFind, + }) + return ctrl.Result{}, err + } + r.Recorder.Event(api, corev1.EventTypeNormal, observability.ReasonCreateSuccess, "Succeeded creating configmap: "+rName) + + } else if getErr != nil && errors.IsNotFound(getErr) && cName != "" { // if custom configmap was not found + r.Log.WithName(observability.LogExportersConfigMap).Error(getErr, "Failed to retrieve custom configmap", "name", cName) + meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + Type: observability.PhaseExportersConfigMap, + Status: metav1.ConditionFalse, + Reason: observability.ReasonRetrieveFailure, + Message: "Failed to retrieve custom configmap " + nameToFind, + }) + return ctrl.Result{}, getErr + + } else if getErr != nil { // if an error occurred + r.Log.WithName(observability.LogExportersConfigMap).Error(getErr, "Failed to retrieve due to some error", "name", nameToFind) + meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + Type: observability.PhaseExportersConfigMap, + Status: metav1.ConditionFalse, + Reason: observability.ReasonRetrieveFailure, + Message: "Failed to retrieve configmap: " + nameToFind, + }) + return ctrl.Result{}, getErr + + } + + // refresh cr before changes + if err := r.Get(context.TODO(), req.NamespacedName, api); err != nil { + r.Log.Error(err, "Failed to fetch updated CR") + } + + // create event if exporter config needs to be updated + if getErr == nil && cName != "" && api.Status.ExporterConfig != nameToFind { + r.Recorder.Event(api, corev1.EventTypeNormal, observability.ReasonRetrieveSuccess, "Succeeded retrieving custom configmap: "+nameToFind) + } + + api.Status.ExporterConfig = nameToFind + meta.RemoveStatusCondition(&api.Status.Conditions, observability.PhaseExportersConfigMap) // Clear any conditions + return ctrl.Result{}, nil + +} +func (r *DatabaseObserverReconciler) manageResource(or ObservabilityResource, api *apiv1.DatabaseObserver, ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + resourceType, logName, gvk := or.identify() + + // generate desired object based on api.Spec + desiredObj, genErr := or.generate(api, r.Scheme) + if genErr != nil { + r.Log.WithName(logName).Error(genErr, "Failed to generate "+gvk.Kind) + meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + Type: resourceType, + Status: metav1.ConditionFalse, + Reason: observability.ReasonSetFailure, + Message: "Failed to generate " + gvk.Kind, + }) + return ctrl.Result{}, genErr + } + + // retrieve object if it exists + foundObj := &unstructured.Unstructured{} + foundObj.SetGroupVersionKind(gvk) + getErr := r.Get(context.TODO(), types.NamespacedName{Name: desiredObj.GetName(), Namespace: req.Namespace}, foundObj) + + // defer for updates to status below + defer r.updateStatusAndRetrieve(api, ctx, req) + + // if resource not found, create resource then return + if getErr != nil && errors.IsNotFound(getErr) { + r.Log.WithName(logName).Info("Creating resource "+gvk.Kind, "name", desiredObj.GetName()) + + if err := r.Create(context.TODO(), desiredObj); err != nil { // create + r.Log.WithName(logName).Error(err, "Failed to create "+gvk.Kind, "name", desiredObj.GetName()) + r.Recorder.Event(api, corev1.EventTypeWarning, observability.ReasonCreateFailure, "Failed creating resource "+gvk.Kind+": "+desiredObj.GetName()) + meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + Type: resourceType, + Status: metav1.ConditionFalse, + Reason: observability.ReasonCreateFailure, + Message: "Failed to create " + gvk.Kind + ": " + desiredObj.GetName(), + }) + return ctrl.Result{}, err + } + + r.Recorder.Event(api, corev1.EventTypeNormal, observability.ReasonCreateSuccess, "Succeeded creating "+gvk.Kind+": "+desiredObj.GetName()) + + } else if getErr != nil { // if an error occurred + r.Log.WithName(logName).Error(getErr, "Failed to retrieve "+desiredObj.GetKind(), "name", desiredObj.GetName()) + meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + Type: resourceType, + Status: metav1.ConditionFalse, + Reason: observability.ReasonRetrieveFailure, + Message: "Failed to retrieve " + gvk.Kind + ": " + desiredObj.GetName(), + }) + return ctrl.Result{}, getErr + } + + // if it does exist, if it's not the same, update with details from desired object + meta.RemoveStatusCondition(&api.Status.Conditions, resourceType) // Clear any conditions + return ctrl.Result{}, nil +} + +func (resource *ObservabilityDeploymentResource) generate(api *apiv1.DatabaseObserver, scheme *runtime.Scheme) (*unstructured.Unstructured, error) { + rName := observability.DefaultExporterDeploymentPrefix + api.Name + rContainerName := observability.DefaultExporterContainerName + rContainerImage := observability.GetExporterImage(api) + rVolumes := observability.GetExporterDeploymentVolumes(api) + rVolumeMounts := observability.GetExporterDeploymentVolumeMounts(api) + rSelectors := observability.GetExporterSelector(api) + rReplicas := observability.GetExporterReplicas(api) + rEnvs := observability.GetExporterEnvs(api) + + rPort := []corev1.ContainerPort{ + {ContainerPort: 8080}, + } + + obj := &appsv1.Deployment{ + ObjectMeta: metav1.ObjectMeta{ + Name: rName, + Namespace: api.Namespace, + }, + Spec: appsv1.DeploymentSpec{ + Replicas: &rReplicas, + Selector: &metav1.LabelSelector{ + MatchLabels: rSelectors, + }, + Template: corev1.PodTemplateSpec{ + ObjectMeta: metav1.ObjectMeta{ + Labels: rSelectors, + }, + Spec: corev1.PodSpec{ + Containers: []corev1.Container{{ + Image: rContainerImage, + ImagePullPolicy: corev1.PullAlways, + Name: rContainerName, + Env: rEnvs, + VolumeMounts: rVolumeMounts, + Ports: rPort, + }}, + RestartPolicy: corev1.RestartPolicyAlways, + Volumes: rVolumes, + }, + }, + }, + } + + if err := controllerutil.SetControllerReference(api, obj, scheme); err != nil { + return nil, err + } + + var u = &unstructured.Unstructured{} + if err := scheme.Convert(obj, u, nil); err != nil { + return nil, err + } + return u, nil +} +func (resource *ObservabilityServiceResource) generate(api *apiv1.DatabaseObserver, scheme *runtime.Scheme) (*unstructured.Unstructured, error) { + rServiceName := "obs-svc-" + api.Name + rLabels := observability.GetExporterLabels(api) + rPort := observability.GetExporterServicePort(api) + rSelector := observability.GetExporterSelector(api) + + obj := &corev1.Service{ + ObjectMeta: metav1.ObjectMeta{ + Name: rServiceName, + Labels: rLabels, + Namespace: api.Namespace, + }, + Spec: corev1.ServiceSpec{ + Type: "NodePort", + Selector: rSelector, + Ports: []corev1.ServicePort{ + { + Name: "metrics", + Port: rPort, + }, + }, + }, + } + + if err := controllerutil.SetControllerReference(api, obj, scheme); err != nil { + return nil, err + } + + var u = &unstructured.Unstructured{} + if err := scheme.Convert(obj, u, nil); err != nil { + return nil, err + } + return u, nil +} +func (resource *ObservabilityServiceMonitorResource) generate(api *apiv1.DatabaseObserver, scheme *runtime.Scheme) (*unstructured.Unstructured, error) { + rName := observability.DefaultServicemonitorPrefix + api.Name + rLabels := observability.GetExporterLabels(api) + rSelector := observability.GetExporterSelector(api) + rPort := observability.GetExporterServiceMonitorPort(api) + rInterval := "20s" + + obj := &monitorv1.ServiceMonitor{ + ObjectMeta: metav1.ObjectMeta{ + Name: rName, + Labels: rLabels, + Namespace: api.Namespace, + }, + Spec: monitorv1.ServiceMonitorSpec{ + Endpoints: []monitorv1.Endpoint{{ + Interval: monitorv1.Duration(rInterval), + Port: rPort, + }}, + Selector: metav1.LabelSelector{ + MatchLabels: rSelector, + }, + }, + } + + if err := controllerutil.SetControllerReference(api, obj, scheme); err != nil { + return nil, err + } + + var u = &unstructured.Unstructured{} + if err := scheme.Convert(obj, u, nil); err != nil { + return nil, err + } + return u, nil +} +func (resource *ObservabilityGrafanaJsonResource) generate(api *apiv1.DatabaseObserver, scheme *runtime.Scheme) (*unstructured.Unstructured, error) { + rName := observability.DefaultGrafanaConfigMapNamePrefix + api.Name + grafanaJSON, err := observability.GetGrafanaJSONData(api) + if err != nil { + return nil, err + } + + obj := &corev1.ConfigMap{ + ObjectMeta: metav1.ObjectMeta{ + Name: rName, + Namespace: api.Namespace, + }, + Data: map[string]string{observability.DefaultGrafanaConfigmapKey: string(grafanaJSON)}, + } + + if err := controllerutil.SetControllerReference(api, obj, scheme); err != nil { + return nil, err + } + + var u = &unstructured.Unstructured{} + if err := scheme.Convert(obj, u, nil); err != nil { + return nil, err + } + return u, nil + +} + +func (resource *ObservabilityDeploymentResource) identify() (string, string, schema.GroupVersionKind) { + return observability.PhaseExportersDeploy, observability.LogExportersDeploy, schema.GroupVersionKind{ + Group: "apps", + Version: "v1", + Kind: "Deployment", + } +} +func (resource *ObservabilityServiceResource) identify() (string, string, schema.GroupVersionKind) { + return observability.PhaseExportersSVC, observability.LogExportersSVC, schema.GroupVersionKind{ + Group: "", + Version: "v1", + Kind: "Service", + } +} +func (resource *ObservabilityServiceMonitorResource) identify() (string, string, schema.GroupVersionKind) { + return observability.PhaseExportersServiceMonitor, observability.LogExportersServiceMonitor, schema.GroupVersionKind{ + Group: "monitoring.coreos.com", + Version: "v1", + Kind: "ServiceMonitor", + } +} +func (resource *ObservabilityGrafanaJsonResource) identify() (string, string, schema.GroupVersionKind) { + return observability.PhaseExportersGrafanaConfigMap, observability.LogExportersGrafanaConfigMap, schema.GroupVersionKind{ + Group: "", + Version: "v1", + Kind: "ConfigMap", + } +} diff --git a/controllers/observability/suite_test.go b/controllers/observability/suite_test.go new file mode 100644 index 00000000..12931e52 --- /dev/null +++ b/controllers/observability/suite_test.go @@ -0,0 +1,103 @@ +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package controllers + +import ( + "path/filepath" + "testing" + + . "github.com/onsi/ginkgo" + . "github.com/onsi/gomega" + "k8s.io/client-go/kubernetes/scheme" + "k8s.io/client-go/rest" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/envtest" + "sigs.k8s.io/controller-runtime/pkg/envtest/printer" + logf "sigs.k8s.io/controller-runtime/pkg/log" + "sigs.k8s.io/controller-runtime/pkg/log/zap" + + observabilityv1alpha1 "github.com/oracle/oracle-database-operator/apis/observability/v1alpha1" + //+kubebuilder:scaffold:imports +) + +// These tests use Ginkgo (BDD-style Go testing framework). Refer to +// http://onsi.github.io/ginkgo/ to learn more about Ginkgo. + +var cfg *rest.Config +var k8sClient client.Client +var testEnv *envtest.Environment + +func TestAPIs(t *testing.T) { + RegisterFailHandler(Fail) + + RunSpecsWithDefaultAndCustomReporters(t, + "Controller Suite", + []Reporter{printer.NewlineReporter{}}) +} + +var _ = BeforeSuite(func(done Done) { + logf.SetLogger(zap.New(zap.WriteTo(GinkgoWriter), zap.UseDevMode(true))) + + By("bootstrapping test environment") + testEnv = &envtest.Environment{ + CRDDirectoryPaths: []string{filepath.Join("..", "..", "config", "crd", "bases")}, + } + + var err error + cfg, err = testEnv.Start() + Expect(err).ToNot(HaveOccurred()) + Expect(cfg).ToNot(BeNil()) + + err = observabilityv1alpha1.AddToScheme(scheme.Scheme) + Expect(err).NotTo(HaveOccurred()) + + //+kubebuilder:scaffold:scheme + + k8sClient, err = client.New(cfg, client.Options{Scheme: scheme.Scheme}) + Expect(err).ToNot(HaveOccurred()) + Expect(k8sClient).ToNot(BeNil()) + + close(done) +}, 60) + +var _ = AfterSuite(func() { + By("tearing down the test environment") + err := testEnv.Stop() + Expect(err).ToNot(HaveOccurred()) +}) diff --git a/docs/observability/README.md b/docs/observability/README.md new file mode 100644 index 00000000..42f3e01b --- /dev/null +++ b/docs/observability/README.md @@ -0,0 +1,194 @@ +# Managing Observability on Kubernetes for Oracle Databases + +Oracle Database Operator for Kubernetes (`OraOperator`) includes the +Observability Controller for Oracle Databases, which automates the +deployment of the database observability exporter, creates Service Monitors +for Prometheus and provides a configmap containing a JSON for a sample dashboard +in Grafana. The following sections explain the setup and functionality +of the operator + +- [Requirements](#requirements) + - [The Observability Custom Resource](#observability-custom-resource) + - [Create](#create) + - [Delete](#delete) + - [Patch](#patch) + +## Requirements +Oracle recommends that you follow the [requirements](./REQUIREMENTS.md). + +## DatabaseObserver Custom Resource +The Oracle Database Operator __v1.0.0__ includes the Observability controller. The Observability controller automates +the deployment and setting up of the Oracle Database exporter and related resources to make databases observable. + +### Resource Details +#### Observability List +To list the Observability custom resources, use the following command as an example: +```sh +$ kubectl get databaseobserver +``` + +#### Quick Status +To obtain a quick status, use the following command as an example: + +> Note: The databaseobserver custom resource is named `obs-sample`. +> We will use this name only in the following examples + +```sh +$ kubectl get databaseobserver obs-sample +NAME EXPORTERCONFIG STATUS +obs-sample obs-cm-obs-sample READY +``` + +#### Detailed Status +To obtain a detailed database status, use the following command as an example: + +```sh +$ kubectl describe databaseobserver obs-sample +Name: obs-sample +Namespace: ... +Labels: ... +Annotations: ... +API Version: observability.oracle.com/v1alpha1 +Kind: DatabaseObserver +Metadata: ... +Spec: ... +Status: + Conditions: + Last Transition Time: 2023-05-08T15:23:05Z + Message: Observability exporter deployed successfully + Reason: ResourceReady + Status: True + Type: READY + Exporter Config: obs-cm-obs-sample + Status: READY +Events: + Type Reason Age From Message + ---- ------ ---- ---- ------- + Normal InitializationSucceeded 16s Observability Initialization of observability resource completed + Normal CreateResourceSucceeded 16s Observability Succeeded creating configmap: obs-cm-obs-sample + Normal CreateResourceSucceeded 16s Observability Succeeded creating Deployment: obs-deploy-obs-sample + Normal CreateResourceSucceeded 16s Observability Succeeded creating Service: obs-svc-obs-sample + Normal CreateResourceSucceeded 16s Observability Succeeded creating ServiceMonitor: obs-servicemonitor-obs-sample + Normal CreateResourceSucceeded 16s Observability Succeeded creating ConfigMap: obs-json-dash-obs-sample +``` + + +## Create +To provision a new Observability custom resource on the Kubernetes cluster, use the example **[config/samples/observability/observability_create.yaml](../../config/samples/observability/observability_create.yaml)**. + +In the example YAML file, the Observability custom resource checks the following properties: + +| Attribute | Type | Default | Required? | Example | +|-------------------------------------------------------|--------|-----------------|-------------|-----------------------------------------------------------------------| +| `spec.database.dbName` | string | - | Yes | _oradevdb_ | +| `spec.database.dbPassword.key` | string | password | Optional | _admin-password_ | +| `spec.database.dbPassword.secret` | string | - | Required | _devsec-dbpassword_ | +| `spec.database.dbWallet.secret` | string | - | Conditional | _devsec-oradevdb-wallet_ | +| `spec.database.dbConnectionString.key` | string | access | Optional | _connection_ | +| `spec.database.dbConnectionString.secret` | string | - | Required | _devsec-dbconnectionstring_ | +| `spec.exporter.image` | string | - | No | _container-registry.oracle.com/database/observability-exporter:1.0.2_ | +| `spec.exporter.configuration.configmap.key` | string | config.toml | No | _config.toml_ | +| `spec.exporter.configuration.configmap.configmapName` | string | - | No | _devcm-oradevdb-config_ | +| `spec.exporter.service.port` | number | 9161 | No | _9161_ | +| `spec.prometheus.port` | string | metrics | No | _metrics_ | +| `spec.prometheus.labels` | map | app: obs-{name} | No | _app: oradevdb-apps_ | +| `spec.replicas` | number | 1 | No | _1_ | + +To provide the remaining references above, follow the guide provided below. + +To enable the exporter to make a connection and observe your database, whether it is in the cloud or it is +a containerized database, you need to provide a working set of database credentials. + + +1. Create a Kubernetes secret for the Database Connection String. + ```bash + kubectl create secret generic db-connect-string --from-literal=access=admin/password@sampledb_tp + ``` + +2. Create a Kubernetes secret for the Database Password. + ```bash + kubectl create secret generic db-password --from-literal=password='password' + ``` + +3. (Conditional) If your database is an Autonomous Database, create a Kubernetes secret for the Autonomous Database Wallet + + If you have an Autonomous database, you can create a secret for the wallet by following this [step](../adb/README.md#download-wallets). + + +Once the secrets are created, to create the Observability resource, apply the YAML +```bash + kubectl apply -f observability_create.yaml +``` + +### Configurations + +The Observability controller provides multiple other fields for configuration but +are optional: + +- `spec.exporter.image` - You can update this field to override the exporter image used + by the controller, and is useful for using the latest exporter version ahead of what the + Observability controller supports. +- `spec.epxorter.configuration.configmap.configmapName` and `spec.epxorter.configuration.configmap.key` - You can + set this field to the name of a custom configmap that you want to provide the controller to use + instead of the default configmap. This field can be updated and the deployment will replace the configmap being used. +- `spec.exporter.service.port` - You can change the port used by the exporter service created by the controller. +- `spec.prometheus.port` - You can change the port used by the service monitor created by the controller. +- `spec.prometheus.labels` - You can set the labels that is specific to your usage. This field defines the labels + and selector labels used by the resources. This field cannot be updated after the custom resource is created. +- `spec.replicas` - You can set this number to the replica count of the exporter deployment. + +## Delete + +To delete the DatabaseObserver custom resource and all related resources: + +```bash +kubectl delete databaseobserver obs-sample +``` + +## Patch + +The Observability controller currently supports updates for only the following fields: + +- `spec.exporter.image` - If there is a specific release of the observability exporter you require, you can update + the _image_ field and the controller will attempt to update the exporter deployment image. The command below + shows how to patch the field to the latest image of the exporter. + ```bash + kubectl --type=merge -p '{"spec":{"exporter":{"image":"container-registry.oracle.com/database/observability-exporter:latest"}}}' patch databaseobserver obs-sample + ``` + +- `spec.exporter.configuration.configmap.configmapName` - The exporter configuration `.toml` file defines the metrics + are scraped by the exporter. This field provides the ability for you to supply your own configmap containing + the custom configurations that you require. The command below shows how to patch the field with your own configmap. + ```bash + kubectl --type=merge -p '{"spec":{"exporter":{"configuration":{"configmap": {"configmapName": "my-custom-configmap", "key": "config.toml"}}}' patch databaseobserver obs-sample + ``` + + +## Debugging and troubleshooting + +### Show the details of the resource + +To get the verbose output of the current spec, use the command below: + +```sh +kubectl describe databaseobserver/database-observer-sample +``` + +If any error occurs during the reconciliation loop, the Operator either reports +the error using the resource's event stream, or will show the error under conditions. + +### Check the logs of the pod where the operator deploys + +Follow the steps to check the logs. + +1. List the pod replicas + + ```sh + kubectl get pods -n oracle-database-operator-system + ``` + +2. Use the below command to check the logs of the deployment + + ```sh + kubectl logs deployment.apps/oracle-database-operator-controller-manager -n oracle-database-operator-system + ``` diff --git a/docs/observability/REQUIREMENTS.md b/docs/observability/REQUIREMENTS.md new file mode 100644 index 00000000..439356f7 --- /dev/null +++ b/docs/observability/REQUIREMENTS.md @@ -0,0 +1,56 @@ +# Observability Controller Requirements +To properly deploy an Observability custom resource, complete the following steps below. + +## Install and configure Prometheus + +The Observability controller creates multiple resources that include the Prometheus `servicemonitor`. +These `servicemonitors` are used to define monitoring for a set of services by scraping +metrics from within Kubernetes. In order for the controller to create ServiceMonitors, the +ServiceMonitor custom resource must exist. + +### Checking if the Custom Resource Exists +To check if the custom resource already exists, run: + +```bash +kubectl api-resources | grep monitoring +``` +You should see the below entry listed among others included with Prometheus +```bash +# result +... +servicemonitors smon monitoring.coreos.com/v1 true ServiceMonitor +... +``` + +### Installing Prometheus +If you do not have Prometheus installed, you can install Prometheus using different ways. Below +shows two ways of installing Prometheus + +#### Using the Prometheus Operator +Using the Prometheus Operator for an example, run the following command which installs the Prometheus Operator into +the Kubernetes cluster. This bundle includes all Prometheus CRDs (including the required _servicemonitor_). +```bash +kubectl create -f https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/master/bundle.yaml +``` + +#### Using Helm +Alternatively, you can install Prometheus using helm. [Helm](https://helm.sh/docs/) must be installed beforehand. +Using helm, run the following command to add the Prometheus repository as follows: +```bash +helm repo add prometheus-community https://prometheus-community.github.io/helm-charts +``` +Run the update command to retrieve the latest from the Prometheus chart repository. +```bash +helm repo update +``` +Run the installation command to install prometheus without a custom template. +```bash +helm install prometheus prometheus-community/prometheus +``` + + +## Provision or set up your Database + +The Observability Controller does not include the provisioning of databases by itself. Before creating +the Observability custom resource, you will need to prepare your Oracle Database and the required secrets containing +database credentials. The database user must have access to Database Performance (V$) views. \ No newline at end of file diff --git a/go.mod b/go.mod index 81c4d435..7c5016df 100644 --- a/go.mod +++ b/go.mod @@ -4,9 +4,11 @@ go 1.19 require ( github.com/go-logr/logr v1.2.3 + github.com/onsi/ginkgo v1.16.5 github.com/onsi/ginkgo/v2 v2.5.0 github.com/onsi/gomega v1.24.1 github.com/oracle/oci-go-sdk/v65 v65.26.1 + github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring v0.62.0 go.uber.org/zap v1.23.0 gopkg.in/yaml.v2 v2.4.0 k8s.io/api v0.25.4 @@ -64,11 +66,11 @@ require ( google.golang.org/protobuf v1.28.0 // indirect gopkg.in/inf.v0 v0.9.1 // indirect gopkg.in/yaml.v3 v3.0.1 // indirect - k8s.io/apiextensions-apiserver v0.25.0 // indirect - k8s.io/component-base v0.25.0 // indirect - k8s.io/klog/v2 v2.70.1 // indirect + k8s.io/apiextensions-apiserver v0.25.4 // indirect + k8s.io/component-base v0.25.4 // indirect + k8s.io/klog/v2 v2.80.1 // indirect k8s.io/kube-openapi v0.0.0-20220803162953-67bda5d908f1 // indirect - k8s.io/utils v0.0.0-20220728103510-ee6ede2d64ed // indirect + k8s.io/utils v0.0.0-20221108210102-8e77b1f39fe2 // indirect sigs.k8s.io/json v0.0.0-20220713155537-f223a00ba0e2 // indirect sigs.k8s.io/structured-merge-diff/v4 v4.2.3 // indirect ) @@ -79,4 +81,6 @@ require ( github.com/go-logr/zapr v1.2.3 // indirect github.com/google/gofuzz v1.2.0 // indirect github.com/google/uuid v1.3.0 // indirect + github.com/nxadm/tail v1.4.8 // indirect + gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 // indirect ) diff --git a/go.sum b/go.sum index aacb116a..7ca15dba 100644 --- a/go.sum +++ b/go.sum @@ -102,6 +102,8 @@ github.com/evanphx/json-patch v5.6.0+incompatible h1:jBYDEEiFBPxA0v50tFdvOzQQTCv github.com/evanphx/json-patch v5.6.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk= github.com/evanphx/json-patch/v5 v5.6.0 h1:b91NhWfaz02IuVxO9faSllyAtNXHMPkC5J8sJCLunww= github.com/evanphx/json-patch/v5 v5.6.0/go.mod h1:G79N1coSVB93tBe7j6PhzjmR3/2VvlbKOFpnXhI9Bw4= +github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo= +github.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ= github.com/fsnotify/fsnotify v1.5.4 h1:jRbGcIw6P2Meqdwuo0H1p6JVLbL5DHKAKlYndzMwVZI= github.com/fsnotify/fsnotify v1.5.4/go.mod h1:OVB6XrOHzAwXMpEM7uPOzcehqUV2UqJxmVXmkdnm1bU= github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04= @@ -114,7 +116,6 @@ github.com/go-kit/log v0.1.0/go.mod h1:zbhenjAZHb184qTLMA9ZjW7ThYL0H2mk7Q6pNt4vb github.com/go-logfmt/logfmt v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9GBnD5lWE= github.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V4qmtdjCk= github.com/go-logfmt/logfmt v0.5.0/go.mod h1:wCYkCAKZfumFQihp8CzCvQ3paCTfi41vtzG1KdI/P7A= -github.com/go-logr/logr v0.1.0/go.mod h1:ixOQHD9gLJUVQQ2ZOR7zLEifBX6tGkNJF4QyIY7sIas= github.com/go-logr/logr v1.2.0/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A= github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A= github.com/go-logr/logr v1.2.3 h1:2DntVwHkVopvECVRSlL5PSo9eG+cAkDCuckLubN+rq0= @@ -130,6 +131,7 @@ github.com/go-openapi/swag v0.19.5/go.mod h1:POnQmlKehdgb5mhVOsnJFsivZCEZ/vjK9gh github.com/go-openapi/swag v0.19.14 h1:gm3vOOXfiuw5i9p5N9xJvfjvuofpyvLA9Wr6QfK5Fng= github.com/go-openapi/swag v0.19.14/go.mod h1:QYRuS/SOXUCsnplDa677K7+DxSOj6IPNl/eQntq43wQ= github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY= +github.com/go-task/slim-sprig v0.0.0-20210107165309-348f09dbbbc0/go.mod h1:fyg7847qk6SyHyPtNmDHnmrv/HOrqktSC+C9fM+CJOE= github.com/gofrs/flock v0.8.1 h1:+gYjHKf32LDeiEEFhQaotPbLuUXjY5ZqxKgXy7n59aw= github.com/gofrs/flock v0.8.1/go.mod h1:F1TvTiK9OcQqauNUHlbJvyl9Qa1QvF/gOUDKA14jxHU= github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ= @@ -221,6 +223,7 @@ github.com/gorilla/websocket v1.4.2/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/ad github.com/grpc-ecosystem/grpc-gateway v1.16.0/go.mod h1:BDjrQk3hbvj6Nolgz8mAMFbcEtjT1g+wF4CSlocrBnw= github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8= github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8= +github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU= github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc= github.com/ianlancetaylor/demangle v0.0.0-20200824232613-28f6c0f3b639/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc= github.com/imdario/mergo v0.3.12 h1:b6R2BslTbIEToALKP7LxUvijTsNI9TAe80pLWN2g/HU= @@ -271,10 +274,17 @@ github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRW github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U= github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e h1:fD57ERR4JtEqsWbfPhv4DMiApHyliiK5xCTNVSPiaAs= github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e/go.mod h1:zD1mROLANZcx1PVRCS0qkT7pwLkGfwJo4zjcN/Tysno= +github.com/nxadm/tail v1.4.4/go.mod h1:kenIhsEOeOJmVchQTgglprH7qJGnHDVpk1VPCcaMI8A= github.com/nxadm/tail v1.4.8 h1:nPr65rt6Y5JFSKQO7qToXr7pePgD6Gwiw05lkbyAQTE= +github.com/nxadm/tail v1.4.8/go.mod h1:+ncqLTQzXmGhMZNUePPaPqPvBxHAIsmXswZKocGu+AU= +github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE= +github.com/onsi/ginkgo v1.12.1/go.mod h1:zj2OWP4+oCPe1qIXoGWkgMRwljMUYCdkwsT2108oapk= github.com/onsi/ginkgo v1.16.5 h1:8xi0RTUf59SOSfEtZMvwTvXYMzG4gV23XVHOZiXNtnE= +github.com/onsi/ginkgo v1.16.5/go.mod h1:+E8gABHa3K6zRBolWtd+ROzc/U5bkGt0FwiG042wbpU= github.com/onsi/ginkgo/v2 v2.5.0 h1:TRtrvv2vdQqzkwrQ1ke6vtXf7IK34RBUJafIy1wMwls= github.com/onsi/ginkgo/v2 v2.5.0/go.mod h1:Luc4sArBICYCS8THh8v3i3i5CuSZO+RaQRaJoeNwomw= +github.com/onsi/gomega v1.7.1/go.mod h1:XdKZgCCFLUoM/7CFJVPcG8C1xQ1AJ0vpAezJrB7JYyY= +github.com/onsi/gomega v1.10.1/go.mod h1:iN09h71vgCQne3DLsj+A5owkum+a2tYe+TOCB1ybHNo= github.com/onsi/gomega v1.24.1 h1:KORJXNNTzJXzu4ScJWssJfJMnJ+2QJqhoQSRwNlze9E= github.com/onsi/gomega v1.24.1/go.mod h1:3AOiACssS3/MajrniINInwbfOOtfZvplPzuRSmvt1jM= github.com/oracle/oci-go-sdk/v65 v65.26.1 h1:Ms20RSRj+CuvQmw5ET1TkmzxLBI+bmLjZ6NYANA3gkk= @@ -285,6 +295,8 @@ github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4= github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring v0.62.0 h1:55138zTXw/yRYizPxZ672I/aDD7Yte3uYRAfUjWUu2M= +github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring v0.62.0/go.mod h1:j51242bf6LQwvJ1JPKWApzTnifmCwcQq0i1p29ylWiM= github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw= github.com/prometheus/client_golang v1.0.0/go.mod h1:db9x61etRT2tGnBNRi70OPL5FsnadC4Ky3P0J6CfImo= github.com/prometheus/client_golang v1.7.1/go.mod h1:PY5Wy2awLA44sXw4AOSfFBetzPP4j5+D6mVACh+pe2M= @@ -315,7 +327,6 @@ github.com/sirupsen/logrus v1.6.0/go.mod h1:7uNnSEd1DgxDLC74fIahvMZmmYsHGZGEOFrf github.com/sony/gobreaker v0.5.0 h1:dRCvqm0P490vZPmy7ppEk2qCnCieBooFJ+YoXGYB+yg= github.com/sony/gobreaker v0.5.0/go.mod h1:ZKptC7FHNvhBz7dN2LGjPVBz2sZJmc0/PkyDJOjmxWY= github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA= -github.com/spf13/afero v1.2.2/go.mod h1:9ZxEEn6pIJ8Rxe320qSDBk6AsU0r9pR7Q4OcevTdifk= github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA= github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= github.com/stoewer/go-strcase v1.2.0/go.mod h1:IBiWB2sKIp3wVVQ3Y035++gc+knqhUQag1KpM8ahLw8= @@ -348,7 +359,7 @@ go.uber.org/atomic v1.7.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc= go.uber.org/atomic v1.9.0 h1:ECmE8Bn/WFTYwEW/bpKD3M8VtR/zQVbavAoalC1PYyE= go.uber.org/atomic v1.9.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc= go.uber.org/goleak v1.1.10/go.mod h1:8a7PlsEVH3e/a/GLqe5IIrQx6GzcnRmZEufDUTk4A7A= -go.uber.org/goleak v1.1.12 h1:gZAh5/EyT/HQwlpkCy6wTpqfH9H8Lz8zbm3dZh+OyzA= +go.uber.org/goleak v1.2.0 h1:xqgm/S+aQvhWFTtR0XK3Jvg7z8kGV8P4X14IzwN3Eqk= go.uber.org/multierr v1.6.0/go.mod h1:cdWPpRnG4AhwMwsgIHip0KRBQjJy5kYEpYjJxpXp9iU= go.uber.org/multierr v1.7.0 h1:zaiO/rmgFjbmCXdSYJWQcdvOCsthmdaHfr3Gm2Kx4Ec= go.uber.org/multierr v1.7.0/go.mod h1:7EAYxJLBy9rStEaz58O2t4Uvip6FSURkq8/ppBp95ak= @@ -398,6 +409,7 @@ golang.org/x/mod v0.4.1/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= +golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= @@ -421,6 +433,7 @@ golang.org/x/net v0.0.0-20200324143707-d3edc9973b7e/go.mod h1:qpuaurCH72eLCgpAm/ golang.org/x/net v0.0.0-20200501053045-e0ff5e5a1de5/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= golang.org/x/net v0.0.0-20200506145744-7e3656a0809f/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= golang.org/x/net v0.0.0-20200513185701-a91f0712d120/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= +golang.org/x/net v0.0.0-20200520004742-59133d7f0dd7/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= golang.org/x/net v0.0.0-20200520182314-0ba52f642ac2/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= golang.org/x/net v0.0.0-20200625001655-4c5254603344/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= golang.org/x/net v0.0.0-20200707034311-ab3426394381/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= @@ -467,6 +480,7 @@ golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJ golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20190312061237-fead79001313/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= @@ -477,7 +491,10 @@ golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7w golang.org/x/sys v0.0.0-20190606165138-5da285871e9c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190624142023-c5567b49c5d0/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20190904154756-749cb33beabd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20191001151750-bb3f8db39f24/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20191005200804-aed5e4c7ecf9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20191120155948-bd437916bb0e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20191228213918-04cbcbbfeed8/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200106162015-b016eb3dc98e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= @@ -501,6 +518,7 @@ golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7w golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20201201145000-ef89a241ccb3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210104204734-6f8348627aad/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20210112080510-489259a85091/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210119212857-b64e53b001e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210220050731-9a76102bfb43/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= @@ -589,6 +607,7 @@ golang.org/x/tools v0.0.0-20200904185747-39188db58858/go.mod h1:Cj7w3i3Rnn0Xh82u golang.org/x/tools v0.0.0-20201110124207-079ba7bd75cd/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= golang.org/x/tools v0.0.0-20201201161351-ac6f37ff4c2a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= golang.org/x/tools v0.0.0-20201208233053-a543418bbed2/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= +golang.org/x/tools v0.0.0-20201224043029-2b0845dc783e/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= golang.org/x/tools v0.0.0-20210105154028-b0ab187a4818/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= golang.org/x/tools v0.1.0/go.mod h1:xkSsbof2nBLbhDlRMhhhyNLN/zl3eTqcnHD5viDpcZ0= @@ -755,9 +774,11 @@ gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8 gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f h1:BLraFXnmrev5lT+xlilqcH8XK9/i0At2xKjWk4p6zsU= gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI= +gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys= gopkg.in/inf.v0 v0.9.1 h1:73M5CoZyi3ZLMOyDlQh031Cx6N9NDJ2Vvfl76EDAgDc= gopkg.in/inf.v0 v0.9.1/go.mod h1:cWUDdTG/fYaXco+Dcufb5Vnc6Gp2YChqWtbxRZE0mXw= gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 h1:uRGJdciOHaEIrze2W8Q3AKkepLTh2hOroT7a+7czfdQ= +gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWDmTeBkI65Dw0HsyUHuEVlX15mw= gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.2.3/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= @@ -781,21 +802,20 @@ honnef.co/go/tools v0.0.1-2020.1.3/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9 honnef.co/go/tools v0.0.1-2020.1.4/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k= k8s.io/api v0.25.4 h1:3YO8J4RtmG7elEgaWMb4HgmpS2CfY1QlaOz9nwB+ZSs= k8s.io/api v0.25.4/go.mod h1:IG2+RzyPQLllQxnhzD8KQNEu4c4YvyDTpSMztf4A0OQ= -k8s.io/apiextensions-apiserver v0.25.0 h1:CJ9zlyXAbq0FIW8CD7HHyozCMBpDSiH7EdrSTCZcZFY= -k8s.io/apiextensions-apiserver v0.25.0/go.mod h1:3pAjZiN4zw7R8aZC5gR0y3/vCkGlAjCazcg1me8iB/E= +k8s.io/apiextensions-apiserver v0.25.4 h1:7hu9pF+xikxQuQZ7/30z/qxIPZc2J1lFElPtr7f+B6U= +k8s.io/apiextensions-apiserver v0.25.4/go.mod h1:bkSGki5YBoZWdn5pWtNIdGvDrrsRWlmnvl9a+tAw5vQ= k8s.io/apimachinery v0.25.4 h1:CtXsuaitMESSu339tfhVXhQrPET+EiWnIY1rcurKnAc= k8s.io/apimachinery v0.25.4/go.mod h1:jaF9C/iPNM1FuLl7Zuy5b9v+n35HGSh6AQ4HYRkCqwo= k8s.io/client-go v0.25.4 h1:3RNRDffAkNU56M/a7gUfXaEzdhZlYhoW8dgViGy5fn8= k8s.io/client-go v0.25.4/go.mod h1:8trHCAC83XKY0wsBIpbirZU4NTUpbuhc2JnI7OruGZw= -k8s.io/component-base v0.25.0 h1:haVKlLkPCFZhkcqB6WCvpVxftrg6+FK5x1ZuaIDaQ5Y= -k8s.io/component-base v0.25.0/go.mod h1:F2Sumv9CnbBlqrpdf7rKZTmmd2meJq0HizeyY/yAFxk= -k8s.io/klog/v2 v2.0.0/go.mod h1:PBfzABfn139FHAV07az/IF9Wp1bkk3vpT2XSJ76fSDE= -k8s.io/klog/v2 v2.70.1 h1:7aaoSdahviPmR+XkS7FyxlkkXs6tHISSG03RxleQAVQ= -k8s.io/klog/v2 v2.70.1/go.mod h1:y1WjHnz7Dj687irZUWR/WLkLc5N1YHtjLdmgWjndZn0= +k8s.io/component-base v0.25.4 h1:n1bjg9Yt+G1C0WnIDJmg2fo6wbEU1UGMRiQSjmj7hNQ= +k8s.io/component-base v0.25.4/go.mod h1:nnZJU8OP13PJEm6/p5V2ztgX2oyteIaAGKGMYb2L2cY= +k8s.io/klog/v2 v2.80.1 h1:atnLQ121W371wYYFawwYx1aEY2eUfs4l3J72wtgAwV4= +k8s.io/klog/v2 v2.80.1/go.mod h1:y1WjHnz7Dj687irZUWR/WLkLc5N1YHtjLdmgWjndZn0= k8s.io/kube-openapi v0.0.0-20220803162953-67bda5d908f1 h1:MQ8BAZPZlWk3S9K4a9NCkIFQtZShWqoha7snGixVgEA= k8s.io/kube-openapi v0.0.0-20220803162953-67bda5d908f1/go.mod h1:C/N6wCaBHeBHkHUesQOQy2/MZqGgMAFPqGsGQLdbZBU= -k8s.io/utils v0.0.0-20220728103510-ee6ede2d64ed h1:jAne/RjBTyawwAy0utX5eqigAwz/lQhTmy+Hr/Cpue4= -k8s.io/utils v0.0.0-20220728103510-ee6ede2d64ed/go.mod h1:jPW/WVKK9YHAvNhRxK0md/EJ228hCsBRufyofKtW8HA= +k8s.io/utils v0.0.0-20221108210102-8e77b1f39fe2 h1:GfD9OzL11kvZN5iArC6oTS7RTj7oJOIfnislxYlqTj8= +k8s.io/utils v0.0.0-20221108210102-8e77b1f39fe2/go.mod h1:OLgZIPagt7ERELqWJFomSt595RzquPNLL48iOWgYOg0= rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8= rsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0= rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA= diff --git a/main.go b/main.go index 6dbf2305..2b6b7e8c 100644 --- a/main.go +++ b/main.go @@ -41,10 +41,15 @@ package main import ( "context" "flag" + monitorv1 "github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring/v1" "os" "strconv" "time" + databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + observabilityv1alpha1 "github.com/oracle/oracle-database-operator/apis/observability/v1alpha1" + databasecontroller "github.com/oracle/oracle-database-operator/controllers/database" + observabilitycontroller "github.com/oracle/oracle-database-operator/controllers/observability" "go.uber.org/zap/zapcore" "k8s.io/apimachinery/pkg/runtime" utilruntime "k8s.io/apimachinery/pkg/util/runtime" @@ -53,9 +58,6 @@ import ( ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/controller-runtime/pkg/log/zap" - - databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" - databasecontroller "github.com/oracle/oracle-database-operator/controllers/database" // +kubebuilder:scaffold:imports ) @@ -66,8 +68,10 @@ var ( func init() { utilruntime.Must(clientgoscheme.AddToScheme(scheme)) - utilruntime.Must(databasev1alpha1.AddToScheme(scheme)) + utilruntime.Must(observabilityv1alpha1.AddToScheme(scheme)) + utilruntime.Must(monitorv1.AddToScheme(scheme)) + // +kubebuilder:scaffold:scheme } @@ -263,6 +267,16 @@ func main() { os.Exit(1) } + // Observability DatabaseObserver Reconciler + if err = (&observabilitycontroller.DatabaseObserverReconciler{ + Client: mgr.GetClient(), + Log: ctrl.Log.WithName("controllers").WithName("observability").WithName("DatabaseObserver"), + Scheme: mgr.GetScheme(), + Recorder: mgr.GetEventRecorderFor("DatabaseObserver"), + }).SetupWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create controller", "controller", "DatabaseObserver") + os.Exit(1) + } // +kubebuilder:scaffold:builder // Add index for PDB CR to enable mgr to cache PDBs From 08f6d2e7f4f2e9974f98d386da4eee5af12062df Mon Sep 17 00:00:00 2001 From: psaini Date: Sat, 24 Jun 2023 05:39:04 +0000 Subject: [PATCH 561/628] Added Observability Controller --- README.md | 21 ++++++++------------- 1 file changed, 8 insertions(+), 13 deletions(-) diff --git a/README.md b/README.md index df504ba2..f8212e15 100644 --- a/README.md +++ b/README.md @@ -1,10 +1,10 @@ # Oracle Database Operator for Kubernetes -## Make Oracle Database Kubernetes Native - Take 2 +## Make Oracle Database Kubernetes Native As part of Oracle's resolution to make Oracle Database Kubernetes native (that is, observable and operable by Kubernetes), Oracle released _Oracle Database Operator for Kubernetes_ (`OraOperator` or the operator). OraOperator extends the Kubernetes API with custom resources and controllers for automating Oracle Database lifecycle management. -In this v0.2.1 release, `OraOperator` supports the following database configurations and infrastructure: +In this v1.0.0 release, `OraOperator` supports the following database configurations and infrastructure: * Oracle Autonomous Database: * Oracle Autonomous Database shared Oracle Cloud Infrastructure (OCI) (ADB-S) @@ -14,6 +14,7 @@ In this v0.2.1 release, `OraOperator` supports the following database configurat * Containerized Sharded databases (SHARDED) deployed in OKE and any k8s where OraOperator is deployed * Oracle Multitenant Databases (CDB/PDBs) * Oracle Database Cloud Service (DBCS) (VMDB) +* Oracle Database Observer Oracle will continue to extend `OraOperator` to support additional Oracle Database configurations. @@ -27,18 +28,17 @@ This release of Oracle Database Operator for Kubernetes (the operator) supports * SHARDED: Provision/deploy sharded databases and the shard topology, Add a new shard, Delete an existing shard * Oracle Multitenant Database: Bind to a CDB, Create a  PDB, Plug a  PDB, Unplug a PDB, Delete a PDB, Clone a PDB, Open/Close a PDB * Database Cloud Service: Provision, Bind, Scale Up/Down, Liveness Probe, Manual Backup +* Orace Database Observer: Provide Quick/Detailed Database Status The upcoming releases will support new configurations, operations and capabilities. ## Release Status -**CAUTION:** The current release of `OraOperator` (v0.2.1) is for development and testing only. DO NOT USE IN PRODUCTION. - This release has been installed and tested on the following Kubernetes platforms: -* [Oracle Container Engine for Kubernetes (OKE)](https://www.oracle.com/cloud-native/container-engine-kubernetes/) with Kubernetes 1.17 or later -* [Oracle Linux Cloud Native Environment(OLCNE)](https://docs.oracle.com/en/operating-systems/olcne/) 1.3 or later -* [Minikube](https://minikube.sigs.k8s.io/docs/) with version v1.21.0 or later +* [Oracle Container Engine for Kubernetes (OKE)](https://www.oracle.com/cloud-native/container-engine-kubernetes/) with Kubernetes 1.24 +* [Oracle Linux Cloud Native Environment(OLCNE)](https://docs.oracle.com/en/operating-systems/olcne/) 1.6 +* [Minikube](https://minikube.sigs.k8s.io/docs/) with version v1.21.0 * [Azure Kubernetes Service](https://azure.microsoft.com/en-us/services/kubernetes-service/) * [Amazon Elastic Kubernetes Service](https://aws.amazon.com/eks/) * [Red Hat OKD](https://www.okd.io/) @@ -68,12 +68,6 @@ Oracle strongly recommends that you ensure your system meets the following [Prer kubectl apply -f https://raw.githubusercontent.com/oracle/oracle-database-operator/main/oracle-database-operator.yaml ``` - --- - **NOTE:** - The above command will also upgrade the existing v0.2.0 `OraOperator` installation to the latest version; for example, v0.2.1. - - --- - Ensure that the operator pods are up and running. For high availability, Operator pod replicas are set to a default of 3. You can scale this setting up or down. ```sh @@ -102,6 +96,7 @@ The quickstarts are designed for specific database configurations: * [Containerized Oracle Sharded Database](./docs/sharding/README.md) * [Oracle Multitenant Database](./docs/multitenant/README.md) * [Oracle Database Cloud Service](./docs/dbcs/README.md) +* [Oracle Database Observer](./doc/observability/README.md) YAML file templates are available under [`/config/samples`](./config/samples/). You can copy and edit these template files to configure them for your use cases. From e819fda52494b289b5be7b5fb1625dac63fa2c2d Mon Sep 17 00:00:00 2001 From: psaini Date: Sat, 24 Jun 2023 05:43:23 +0000 Subject: [PATCH 562/628] Added Observability Controller --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index f8212e15..d583ed35 100644 --- a/README.md +++ b/README.md @@ -96,7 +96,7 @@ The quickstarts are designed for specific database configurations: * [Containerized Oracle Sharded Database](./docs/sharding/README.md) * [Oracle Multitenant Database](./docs/multitenant/README.md) * [Oracle Database Cloud Service](./docs/dbcs/README.md) -* [Oracle Database Observer](./doc/observability/README.md) +* [Oracle Database Observer](./docs/observability/README.md) YAML file templates are available under [`/config/samples`](./config/samples/). You can copy and edit these template files to configure them for your use cases. From 828bff6059559a480687fd0da620acae227637ca Mon Sep 17 00:00:00 2001 From: psaini Date: Sun, 25 Jun 2023 11:25:54 +0000 Subject: [PATCH 563/628] Updated README.md --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index d583ed35..a472c942 100644 --- a/README.md +++ b/README.md @@ -14,7 +14,7 @@ In this v1.0.0 release, `OraOperator` supports the following database configurat * Containerized Sharded databases (SHARDED) deployed in OKE and any k8s where OraOperator is deployed * Oracle Multitenant Databases (CDB/PDBs) * Oracle Database Cloud Service (DBCS) (VMDB) -* Oracle Database Observer +* Cloud Native Database Observer Oracle will continue to extend `OraOperator` to support additional Oracle Database configurations. @@ -28,7 +28,7 @@ This release of Oracle Database Operator for Kubernetes (the operator) supports * SHARDED: Provision/deploy sharded databases and the shard topology, Add a new shard, Delete an existing shard * Oracle Multitenant Database: Bind to a CDB, Create a  PDB, Plug a  PDB, Unplug a PDB, Delete a PDB, Clone a PDB, Open/Close a PDB * Database Cloud Service: Provision, Bind, Scale Up/Down, Liveness Probe, Manual Backup -* Orace Database Observer: Provide Quick/Detailed Database Status +* Cloud Native Database Observer: Provide Quick/Detailed Database Status The upcoming releases will support new configurations, operations and capabilities. @@ -96,7 +96,7 @@ The quickstarts are designed for specific database configurations: * [Containerized Oracle Sharded Database](./docs/sharding/README.md) * [Oracle Multitenant Database](./docs/multitenant/README.md) * [Oracle Database Cloud Service](./docs/dbcs/README.md) -* [Oracle Database Observer](./docs/observability/README.md) +* [Cloud Native Database Observer](./docs/observability/README.md) YAML file templates are available under [`/config/samples`](./config/samples/). You can copy and edit these template files to configure them for your use cases. From 5a558c954d0060f99b1be5a7f6bf556f6768fd86 Mon Sep 17 00:00:00 2001 From: psaini Date: Tue, 27 Jun 2023 07:34:57 +0000 Subject: [PATCH 564/628] Added README.md Fixes --- README.md | 6 +++--- docs/dbcs/README.md | 34 ++++++++++++++++++---------------- 2 files changed, 21 insertions(+), 19 deletions(-) diff --git a/README.md b/README.md index a472c942..bc05e261 100644 --- a/README.md +++ b/README.md @@ -13,7 +13,7 @@ In this v1.0.0 release, `OraOperator` supports the following database configurat * Containerized Single Instance databases (SIDB) deployed in the Oracle Kubernetes Engine (OKE) and any k8s where OraOperator is deployed * Containerized Sharded databases (SHARDED) deployed in OKE and any k8s where OraOperator is deployed * Oracle Multitenant Databases (CDB/PDBs) -* Oracle Database Cloud Service (DBCS) (VMDB) +* Oracle Base Database Cloud Service (BDBCS) * Cloud Native Database Observer Oracle will continue to extend `OraOperator` to support additional Oracle Database configurations. @@ -27,7 +27,7 @@ This release of Oracle Database Operator for Kubernetes (the operator) supports * SIDB: Provision, clone, patch (in-place/out-of-place), update database initialization parameters, update database configuration (Flashback, archiving), Oracle Enterprise Manager (EM) Express (a basic observability console), Oracle REST Data Service (ORDS) to support REST based SQL, PDB management, SQL Developer Web, and Application Express (Apex) * SHARDED: Provision/deploy sharded databases and the shard topology, Add a new shard, Delete an existing shard * Oracle Multitenant Database: Bind to a CDB, Create a  PDB, Plug a  PDB, Unplug a PDB, Delete a PDB, Clone a PDB, Open/Close a PDB -* Database Cloud Service: Provision, Bind, Scale Up/Down, Liveness Probe, Manual Backup +* Oracle Base Database Cloud Service (BDBCS): Provision, Bind, Scale Up/Down, Liveness Probe, Manual Backup * Cloud Native Database Observer: Provide Quick/Detailed Database Status The upcoming releases will support new configurations, operations and capabilities. @@ -95,7 +95,7 @@ The quickstarts are designed for specific database configurations: * [Containerized Oracle Single Instance Database](./docs/sidb/README.md) * [Containerized Oracle Sharded Database](./docs/sharding/README.md) * [Oracle Multitenant Database](./docs/multitenant/README.md) -* [Oracle Database Cloud Service](./docs/dbcs/README.md) +* [Oracle Base Database Cloud Service (BDBCS)](./docs/dbcs/README.md) * [Cloud Native Database Observer](./docs/observability/README.md) YAML file templates are available under [`/config/samples`](./config/samples/). You can copy and edit these template files to configure them for your use cases. diff --git a/docs/dbcs/README.md b/docs/dbcs/README.md index b73bcc21..c8b8d5d9 100644 --- a/docs/dbcs/README.md +++ b/docs/dbcs/README.md @@ -1,10 +1,12 @@ # Using the DB Operator DBCS Controller -Oracle Cloud Infastructure (OCI) Database Cloud Service (DBCS) provides single-node Database (DB) systems, deployed on virtual machines, and provides two-node Oracle Real Appliation Clusters (Oracle RAC) database systems on virtual machines. +Oracle Cloud Infastructure (OCI) Oracle Base Database Cloud Service (BDBCS) provides single-node Database (DB) systems, deployed on virtual machines, and provides two-node Oracle Real Appliation Clusters (Oracle RAC) database systems on virtual machines. The single-node DB systems and Oracle RAC systems on virtual machines are [co-managed Oracle Database cloud solutions](https://docs.oracle.com/en-us/iaas/Content/Database/Concepts/overview.htm). To manage the lifecycle of an OCI DBCS system, you can use the OCI Console, the REST API, or the Oracle Cloud Infrastructure command-line interface (CLI). At the granular level, you can use the Oracle Database CLI (DBCLI), Oracle Enterprise Manager, or Oracle SQL Developer. -The Oracle DB Operator DBCS Controller is a feature of the Oracle DB Operator for Kubernetes (OraOperator) which uses OCI's DBCS service to support lifecycle management of the database systems. +The Oracle DB Operator DBCS Controller is a feature of the Oracle DB Operator for Kubernetes (OraOperator) which uses OCI's BDBCS service to support lifecycle management of the database systems. + +Note: Oracle Base Database Cloud Service (BDBCS) was previously known as Database Cloud Service (DBCS). # Supported Database Editions and Versions @@ -113,7 +115,7 @@ kubectl create secret generic oci-privatekey --from-file=privatekey=/root/.oci/o ``` -## 3. Create a Kubernetes secret named `admin-password`; This passward must meet the minimum passward requirements for the OCI DBCS Service. +## 3. Create a Kubernetes secret named `admin-password`; This passward must meet the minimum passward requirements for the OCI BDBCS Service. For example: ``` @@ -123,7 +125,7 @@ kubectl create secret generic admin-password --from-file=./admin-password -n def ``` -## 4. Create a Kubernetes secret named `tde-password`; this passward must meet the minimum passward requirements for the OCI DBCS Service. +## 4. Create a Kubernetes secret named `tde-password`; this passward must meet the minimum passward requirements for the OCI BDBCS Service. For example: ``` @@ -165,22 +167,22 @@ The key's randomart image is: # Use Cases to manage the lifecycle of an OCI DBCS System with Oracle DB Operator DBCS Controller -For more informatoin about the multiple use cases available to you to deploy and manage the OCI DBCS Service-based database using the Oracle DB Operator DBCS Controller, review this list: +For more informatoin about the multiple use cases available to you to deploy and manage the OCI BDBCS Service-based database using the Oracle DB Operator DBCS Controller, review this list: -[1. Deploy a DB System using OCI DBCS Service with minimal parameters](./provisioning/dbcs_service_with_minimal_parameters.md) -[2. Binding to an existing DBCS System already deployed in OCI DBCS Service](./provisioning/bind_to_existing_dbcs_system.md) -[3. Scale UP the shape of an existing DBCS System](./provisioning/scale_up_dbcs_system_shape.md) -[4. Scale DOWN the shape of an existing DBCS System](./provisioning/scale_down_dbcs_system_shape.md) -[5. Scale UP the storage of an existing DBCS System](./provisioning/scale_up_storage.md) -[6. Update License type of an existing DBCS System](./provisioning/update_license.md) -[7. Terminate an existing DBCS System](./provisioning/terminate_dbcs_system.md) -[8. Create DBCS with All Parameters with Storage Management as LVM](./provisioning/dbcs_service_with_all_parameters_lvm.md) -[9. Create DBCS with All Parameters with Storage Management as ASM](./provisioning/dbcs_service_with_all_parameters_asm.md) -[10. Deploy a 2 Node RAC DB System using OCI DBCS Service](./provisioning/dbcs_service_with_2_node_rac.md) +[1. Deploy a DB System using OCI BDBCS Service with minimal parameters](./provisioning/dbcs_service_with_minimal_parameters.md) +[2. Binding to an existing DBCS System already deployed in OCI BDBCS Service](./provisioning/bind_to_existing_dbcs_system.md) +[3. Scale UP the shape of an existing BDBCS System](./provisioning/scale_up_dbcs_system_shape.md) +[4. Scale DOWN the shape of an existing BDBCS System](./provisioning/scale_down_dbcs_system_shape.md) +[5. Scale UP the storage of an existing BDBCS System](./provisioning/scale_up_storage.md) +[6. Update License type of an existing BDBCS System](./provisioning/update_license.md) +[7. Terminate an existing BDBCS System](./provisioning/terminate_dbcs_system.md) +[8. Create BDBCS with All Parameters with Storage Management as LVM](./provisioning/dbcs_service_with_all_parameters_lvm.md) +[9. Create BDBCS with All Parameters with Storage Management as ASM](./provisioning/dbcs_service_with_all_parameters_asm.md) +[10. Deploy a 2 Node RAC DB System using OCI BDBCS Service](./provisioning/dbcs_service_with_2_node_rac.md) ## Connecting to OCI DBCS database deployed using Oracle DB Operator DBCS Controller -After you have deployed the OCI DBCS database with the Oracle DB Operator DBCS Controller, you can connect to the database. To see how to connect and use the database, refer to the steps in [Database Connectivity](./provisioning/database_connection.md). +After you have deployed the OCI BDBCS database with the Oracle DB Operator DBCS Controller, you can connect to the database. To see how to connect and use the database, refer to the steps in [Database Connectivity](./provisioning/database_connection.md). ## Known Issues From 093fb95bad866176a231e00a34461a7f9ead3d7e Mon Sep 17 00:00:00 2001 From: kuassi_mensah Date: Tue, 27 Jun 2023 15:02:43 +0000 Subject: [PATCH 565/628] Update the master Readme. --- README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index bc05e261..1cc6efed 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,7 @@ As part of Oracle's resolution to make Oracle Database Kubernetes native (that is, observable and operable by Kubernetes), Oracle released _Oracle Database Operator for Kubernetes_ (`OraOperator` or the operator). OraOperator extends the Kubernetes API with custom resources and controllers for automating Oracle Database lifecycle management. -In this v1.0.0 release, `OraOperator` supports the following database configurations and infrastructure: +In this v1.0.0 production release, `OraOperator` supports the following database configurations and infrastructure: * Oracle Autonomous Database: * Oracle Autonomous Database shared Oracle Cloud Infrastructure (OCI) (ADB-S) @@ -14,7 +14,7 @@ In this v1.0.0 release, `OraOperator` supports the following database configurat * Containerized Sharded databases (SHARDED) deployed in OKE and any k8s where OraOperator is deployed * Oracle Multitenant Databases (CDB/PDBs) * Oracle Base Database Cloud Service (BDBCS) -* Cloud Native Database Observer +* Cloud Native Database Observer (Preview status) Oracle will continue to extend `OraOperator` to support additional Oracle Database configurations. @@ -34,7 +34,7 @@ The upcoming releases will support new configurations, operations and capabiliti ## Release Status -This release has been installed and tested on the following Kubernetes platforms: +This production release has been installed and tested on the following Kubernetes platforms: * [Oracle Container Engine for Kubernetes (OKE)](https://www.oracle.com/cloud-native/container-engine-kubernetes/) with Kubernetes 1.24 * [Oracle Linux Cloud Native Environment(OLCNE)](https://docs.oracle.com/en/operating-systems/olcne/) 1.6 @@ -143,7 +143,7 @@ See [Contributing to this Repository](./CONTRIBUTING.md) ## Support -You can submit a GitHub issue, or you can also file an [Oracle Support service](https://support.oracle.com/portal/) request, using this product ID: 14430. +You can submit a GitHub issue, and/or you file an [Oracle Support service](https://support.oracle.com/portal/) request, using this product ID: 14430. ## Security From cb16c5fa67ac5000bae1a70dc228c6646438ec9c Mon Sep 17 00:00:00 2001 From: kuassi_mensah Date: Tue, 27 Jun 2023 15:20:12 +0000 Subject: [PATCH 566/628] Update observability/Readme.md --- docs/observability/README.md | 388 +++++++++++++++++------------------ 1 file changed, 194 insertions(+), 194 deletions(-) diff --git a/docs/observability/README.md b/docs/observability/README.md index 42f3e01b..f369a779 100644 --- a/docs/observability/README.md +++ b/docs/observability/README.md @@ -1,194 +1,194 @@ -# Managing Observability on Kubernetes for Oracle Databases - -Oracle Database Operator for Kubernetes (`OraOperator`) includes the -Observability Controller for Oracle Databases, which automates the -deployment of the database observability exporter, creates Service Monitors -for Prometheus and provides a configmap containing a JSON for a sample dashboard -in Grafana. The following sections explain the setup and functionality -of the operator - -- [Requirements](#requirements) - - [The Observability Custom Resource](#observability-custom-resource) - - [Create](#create) - - [Delete](#delete) - - [Patch](#patch) - -## Requirements -Oracle recommends that you follow the [requirements](./REQUIREMENTS.md). - -## DatabaseObserver Custom Resource -The Oracle Database Operator __v1.0.0__ includes the Observability controller. The Observability controller automates -the deployment and setting up of the Oracle Database exporter and related resources to make databases observable. - -### Resource Details -#### Observability List -To list the Observability custom resources, use the following command as an example: -```sh -$ kubectl get databaseobserver -``` - -#### Quick Status -To obtain a quick status, use the following command as an example: - -> Note: The databaseobserver custom resource is named `obs-sample`. -> We will use this name only in the following examples - -```sh -$ kubectl get databaseobserver obs-sample -NAME EXPORTERCONFIG STATUS -obs-sample obs-cm-obs-sample READY -``` - -#### Detailed Status -To obtain a detailed database status, use the following command as an example: - -```sh -$ kubectl describe databaseobserver obs-sample -Name: obs-sample -Namespace: ... -Labels: ... -Annotations: ... -API Version: observability.oracle.com/v1alpha1 -Kind: DatabaseObserver -Metadata: ... -Spec: ... -Status: - Conditions: - Last Transition Time: 2023-05-08T15:23:05Z - Message: Observability exporter deployed successfully - Reason: ResourceReady - Status: True - Type: READY - Exporter Config: obs-cm-obs-sample - Status: READY -Events: - Type Reason Age From Message - ---- ------ ---- ---- ------- - Normal InitializationSucceeded 16s Observability Initialization of observability resource completed - Normal CreateResourceSucceeded 16s Observability Succeeded creating configmap: obs-cm-obs-sample - Normal CreateResourceSucceeded 16s Observability Succeeded creating Deployment: obs-deploy-obs-sample - Normal CreateResourceSucceeded 16s Observability Succeeded creating Service: obs-svc-obs-sample - Normal CreateResourceSucceeded 16s Observability Succeeded creating ServiceMonitor: obs-servicemonitor-obs-sample - Normal CreateResourceSucceeded 16s Observability Succeeded creating ConfigMap: obs-json-dash-obs-sample -``` - - -## Create -To provision a new Observability custom resource on the Kubernetes cluster, use the example **[config/samples/observability/observability_create.yaml](../../config/samples/observability/observability_create.yaml)**. - -In the example YAML file, the Observability custom resource checks the following properties: - -| Attribute | Type | Default | Required? | Example | -|-------------------------------------------------------|--------|-----------------|-------------|-----------------------------------------------------------------------| -| `spec.database.dbName` | string | - | Yes | _oradevdb_ | -| `spec.database.dbPassword.key` | string | password | Optional | _admin-password_ | -| `spec.database.dbPassword.secret` | string | - | Required | _devsec-dbpassword_ | -| `spec.database.dbWallet.secret` | string | - | Conditional | _devsec-oradevdb-wallet_ | -| `spec.database.dbConnectionString.key` | string | access | Optional | _connection_ | -| `spec.database.dbConnectionString.secret` | string | - | Required | _devsec-dbconnectionstring_ | -| `spec.exporter.image` | string | - | No | _container-registry.oracle.com/database/observability-exporter:1.0.2_ | -| `spec.exporter.configuration.configmap.key` | string | config.toml | No | _config.toml_ | -| `spec.exporter.configuration.configmap.configmapName` | string | - | No | _devcm-oradevdb-config_ | -| `spec.exporter.service.port` | number | 9161 | No | _9161_ | -| `spec.prometheus.port` | string | metrics | No | _metrics_ | -| `spec.prometheus.labels` | map | app: obs-{name} | No | _app: oradevdb-apps_ | -| `spec.replicas` | number | 1 | No | _1_ | - -To provide the remaining references above, follow the guide provided below. - -To enable the exporter to make a connection and observe your database, whether it is in the cloud or it is -a containerized database, you need to provide a working set of database credentials. - - -1. Create a Kubernetes secret for the Database Connection String. - ```bash - kubectl create secret generic db-connect-string --from-literal=access=admin/password@sampledb_tp - ``` - -2. Create a Kubernetes secret for the Database Password. - ```bash - kubectl create secret generic db-password --from-literal=password='password' - ``` - -3. (Conditional) If your database is an Autonomous Database, create a Kubernetes secret for the Autonomous Database Wallet - - If you have an Autonomous database, you can create a secret for the wallet by following this [step](../adb/README.md#download-wallets). - - -Once the secrets are created, to create the Observability resource, apply the YAML -```bash - kubectl apply -f observability_create.yaml -``` - -### Configurations - -The Observability controller provides multiple other fields for configuration but -are optional: - -- `spec.exporter.image` - You can update this field to override the exporter image used - by the controller, and is useful for using the latest exporter version ahead of what the - Observability controller supports. -- `spec.epxorter.configuration.configmap.configmapName` and `spec.epxorter.configuration.configmap.key` - You can - set this field to the name of a custom configmap that you want to provide the controller to use - instead of the default configmap. This field can be updated and the deployment will replace the configmap being used. -- `spec.exporter.service.port` - You can change the port used by the exporter service created by the controller. -- `spec.prometheus.port` - You can change the port used by the service monitor created by the controller. -- `spec.prometheus.labels` - You can set the labels that is specific to your usage. This field defines the labels - and selector labels used by the resources. This field cannot be updated after the custom resource is created. -- `spec.replicas` - You can set this number to the replica count of the exporter deployment. - -## Delete - -To delete the DatabaseObserver custom resource and all related resources: - -```bash -kubectl delete databaseobserver obs-sample -``` - -## Patch - -The Observability controller currently supports updates for only the following fields: - -- `spec.exporter.image` - If there is a specific release of the observability exporter you require, you can update - the _image_ field and the controller will attempt to update the exporter deployment image. The command below - shows how to patch the field to the latest image of the exporter. - ```bash - kubectl --type=merge -p '{"spec":{"exporter":{"image":"container-registry.oracle.com/database/observability-exporter:latest"}}}' patch databaseobserver obs-sample - ``` - -- `spec.exporter.configuration.configmap.configmapName` - The exporter configuration `.toml` file defines the metrics - are scraped by the exporter. This field provides the ability for you to supply your own configmap containing - the custom configurations that you require. The command below shows how to patch the field with your own configmap. - ```bash - kubectl --type=merge -p '{"spec":{"exporter":{"configuration":{"configmap": {"configmapName": "my-custom-configmap", "key": "config.toml"}}}' patch databaseobserver obs-sample - ``` - - -## Debugging and troubleshooting - -### Show the details of the resource - -To get the verbose output of the current spec, use the command below: - -```sh -kubectl describe databaseobserver/database-observer-sample -``` - -If any error occurs during the reconciliation loop, the Operator either reports -the error using the resource's event stream, or will show the error under conditions. - -### Check the logs of the pod where the operator deploys - -Follow the steps to check the logs. - -1. List the pod replicas - - ```sh - kubectl get pods -n oracle-database-operator-system - ``` - -2. Use the below command to check the logs of the deployment - - ```sh - kubectl logs deployment.apps/oracle-database-operator-controller-manager -n oracle-database-operator-system - ``` +# Cloud Native Observability on Kubernetes for Oracle Databases + +Oracle Database Operator for Kubernetes (`OraOperator`) includes, as a preview, the +Cloud Native Observability Controller for Oracle Databases, which automates the +deployment of the database observability exporter, creates Service Monitors +for Prometheus and provides a configmap containing a JSON for a sample dashboard +in Grafana. The following sections explain the setup and functionality +of the operator + +- [Requirements](#requirements) + - [The Observability Custom Resource](#observability-custom-resource) + - [Create](#create) + - [Delete](#delete) + - [Patch](#patch) + +## Requirements +Oracle recommends that you follow the [requirements](./REQUIREMENTS.md). + +## DatabaseObserver Custom Resource +The Oracle Database Operator __v1.0.0__ includes the Observability controller, as a Preview (i.e., not a production quality feature). The Observability controller automates +the deployment and setting up of the Oracle Database exporter and related resources to make databases observable. + +### Resource Details +#### Observability List +To list the Observability custom resources, use the following command as an example: +```sh +$ kubectl get databaseobserver +``` + +#### Quick Status +To obtain a quick status, use the following command as an example: + +> Note: The databaseobserver custom resource is named `obs-sample`. +> We will use this name only in the following examples + +```sh +$ kubectl get databaseobserver obs-sample +NAME EXPORTERCONFIG STATUS +obs-sample obs-cm-obs-sample READY +``` + +#### Detailed Status +To obtain a detailed database status, use the following command as an example: + +```sh +$ kubectl describe databaseobserver obs-sample +Name: obs-sample +Namespace: ... +Labels: ... +Annotations: ... +API Version: observability.oracle.com/v1alpha1 +Kind: DatabaseObserver +Metadata: ... +Spec: ... +Status: + Conditions: + Last Transition Time: 2023-05-08T15:23:05Z + Message: Observability exporter deployed successfully + Reason: ResourceReady + Status: True + Type: READY + Exporter Config: obs-cm-obs-sample + Status: READY +Events: + Type Reason Age From Message + ---- ------ ---- ---- ------- + Normal InitializationSucceeded 16s Observability Initialization of observability resource completed + Normal CreateResourceSucceeded 16s Observability Succeeded creating configmap: obs-cm-obs-sample + Normal CreateResourceSucceeded 16s Observability Succeeded creating Deployment: obs-deploy-obs-sample + Normal CreateResourceSucceeded 16s Observability Succeeded creating Service: obs-svc-obs-sample + Normal CreateResourceSucceeded 16s Observability Succeeded creating ServiceMonitor: obs-servicemonitor-obs-sample + Normal CreateResourceSucceeded 16s Observability Succeeded creating ConfigMap: obs-json-dash-obs-sample +``` + + +## Create +To provision a new Observability custom resource on the Kubernetes cluster, use the example **[config/samples/observability/observability_create.yaml](../../config/samples/observability/observability_create.yaml)**. + +In the example YAML file, the Observability custom resource checks the following properties: + +| Attribute | Type | Default | Required? | Example | +|-------------------------------------------------------|--------|-----------------|-------------|-----------------------------------------------------------------------| +| `spec.database.dbName` | string | - | Yes | _oradevdb_ | +| `spec.database.dbPassword.key` | string | password | Optional | _admin-password_ | +| `spec.database.dbPassword.secret` | string | - | Required | _devsec-dbpassword_ | +| `spec.database.dbWallet.secret` | string | - | Conditional | _devsec-oradevdb-wallet_ | +| `spec.database.dbConnectionString.key` | string | access | Optional | _connection_ | +| `spec.database.dbConnectionString.secret` | string | - | Required | _devsec-dbconnectionstring_ | +| `spec.exporter.image` | string | - | No | _container-registry.oracle.com/database/observability-exporter:1.0.2_ | +| `spec.exporter.configuration.configmap.key` | string | config.toml | No | _config.toml_ | +| `spec.exporter.configuration.configmap.configmapName` | string | - | No | _devcm-oradevdb-config_ | +| `spec.exporter.service.port` | number | 9161 | No | _9161_ | +| `spec.prometheus.port` | string | metrics | No | _metrics_ | +| `spec.prometheus.labels` | map | app: obs-{name} | No | _app: oradevdb-apps_ | +| `spec.replicas` | number | 1 | No | _1_ | + +To provide the remaining references above, follow the guide provided below. + +To enable the exporter to make a connection and observe your database, whether it is in the cloud or it is +a containerized database, you need to provide a working set of database credentials. + + +1. Create a Kubernetes secret for the Database Connection String. + ```bash + kubectl create secret generic db-connect-string --from-literal=access=admin/password@sampledb_tp + ``` + +2. Create a Kubernetes secret for the Database Password. + ```bash + kubectl create secret generic db-password --from-literal=password='password' + ``` + +3. (Conditional) If your database is an Autonomous Database, create a Kubernetes secret for the Autonomous Database Wallet + + If you have an Autonomous database, you can create a secret for the wallet by following this [step](../adb/README.md#download-wallets). + + +Once the secrets are created, to create the Observability resource, apply the YAML +```bash + kubectl apply -f observability_create.yaml +``` + +### Configurations + +The Observability controller provides multiple other fields for configuration but +are optional: + +- `spec.exporter.image` - You can update this field to override the exporter image used + by the controller, and is useful for using the latest exporter version ahead of what the + Observability controller supports. +- `spec.epxorter.configuration.configmap.configmapName` and `spec.epxorter.configuration.configmap.key` - You can + set this field to the name of a custom configmap that you want to provide the controller to use + instead of the default configmap. This field can be updated and the deployment will replace the configmap being used. +- `spec.exporter.service.port` - You can change the port used by the exporter service created by the controller. +- `spec.prometheus.port` - You can change the port used by the service monitor created by the controller. +- `spec.prometheus.labels` - You can set the labels that is specific to your usage. This field defines the labels + and selector labels used by the resources. This field cannot be updated after the custom resource is created. +- `spec.replicas` - You can set this number to the replica count of the exporter deployment. + +## Delete + +To delete the DatabaseObserver custom resource and all related resources: + +```bash +kubectl delete databaseobserver obs-sample +``` + +## Patch + +The Observability controller currently supports updates for only the following fields: + +- `spec.exporter.image` - If there is a specific release of the observability exporter you require, you can update + the _image_ field and the controller will attempt to update the exporter deployment image. The command below + shows how to patch the field to the latest image of the exporter. + ```bash + kubectl --type=merge -p '{"spec":{"exporter":{"image":"container-registry.oracle.com/database/observability-exporter:latest"}}}' patch databaseobserver obs-sample + ``` + +- `spec.exporter.configuration.configmap.configmapName` - The exporter configuration `.toml` file defines the metrics + are scraped by the exporter. This field provides the ability for you to supply your own configmap containing + the custom configurations that you require. The command below shows how to patch the field with your own configmap. + ```bash + kubectl --type=merge -p '{"spec":{"exporter":{"configuration":{"configmap": {"configmapName": "my-custom-configmap", "key": "config.toml"}}}' patch databaseobserver obs-sample + ``` + + +## Debugging and troubleshooting + +### Show the details of the resource + +To get the verbose output of the current spec, use the command below: + +```sh +kubectl describe databaseobserver/database-observer-sample +``` + +If any error occurs during the reconciliation loop, the Operator either reports +the error using the resource's event stream, or will show the error under conditions. + +### Check the logs of the pod where the operator deploys + +Follow the steps to check the logs. + +1. List the pod replicas + + ```sh + kubectl get pods -n oracle-database-operator-system + ``` + +2. Use the below command to check the logs of the deployment + + ```sh + kubectl logs deployment.apps/oracle-database-operator-controller-manager -n oracle-database-operator-system + ``` From 484289adef167362905743b4125b3b89740abefb Mon Sep 17 00:00:00 2001 From: ruggero_citton Date: Tue, 27 Jun 2023 15:20:49 +0000 Subject: [PATCH 567/628] the below -> the following --- docs/multitenant/README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/multitenant/README.md b/docs/multitenant/README.md index ccf6957c..8b1407a6 100644 --- a/docs/multitenant/README.md +++ b/docs/multitenant/README.md @@ -88,14 +88,14 @@ Before you want to manage the life cycle of a PDB in a CDB using the Oracle DB O Pluggable Database management operation is performed in the Container Database (CDB) and it includes create, clone, plug, unplug, delete, modify and map operations. -You cannot have an ORDS enabled schema in the container database. To perform the PDB lifecycle management operations, the default CDB administrator credentials must be defined by performing the below steps on the target CDB(s): +You cannot have an ORDS enabled schema in the container database. To perform the PDB lifecycle management operations, the default CDB administrator credentials must be defined by performing following steps on the target CDB(s): Create the CDB administrator user and grant the required privileges. In this example, the user is `C##DBAPI_CDB_ADMIN`. However, any suitable common user name can be used. ```SQL SQL> conn /as sysdba --- Create the below users at the database level: +-- Create following users at the database level: ALTER SESSION SET "_oracle_script"=true; DROP USER C##DBAPI_CDB_ADMIN cascade; @@ -105,7 +105,7 @@ GRANT SYSDBA TO C##DBAPI_CDB_ADMIN CONTAINER = ALL; GRANT CREATE SESSION TO C##DBAPI_CDB_ADMIN CONTAINER = ALL; --- Verify the account status of the below usernames. They should not be in locked status: +-- Verify the account status of the following usernames. They should not be in locked status: col username for a30 col account_status for a30 @@ -138,7 +138,7 @@ Please refer [here](./provisioning/ords_image.md) for the steps to build ORDS Do kubectl apply -f cdb_secret.yaml ``` - **Note:** In order to get the base64 encoded value for a password, please use the below command like below at the command prompt. The value you get is the base64 encoded value for that password string. + **Note:** In order to get the base64 encoded value for a password, please use the following command like below at the command prompt. The value you get is the base64 encoded value for that password string. ```bash echo -n "" | base64 From 9e49dcf2eac03a03a05a66b04e1afc3e620c4816 Mon Sep 17 00:00:00 2001 From: harayuanwang Date: Tue, 27 Jun 2023 15:32:06 +0000 Subject: [PATCH 568/628] Replace acd-id-2.png --- images/adb/acd-id-2.png | Bin 46632 -> 40885 bytes 1 file changed, 0 insertions(+), 0 deletions(-) diff --git a/images/adb/acd-id-2.png b/images/adb/acd-id-2.png index 300bcd791ac27ada39522013ae924f2a61c676bd..1680d34de2d356295edfc71826b3fb863c97565c 100644 GIT binary patch delta 36226 zcmZs?RahKd(6);O2_6XU?(QB)a0%`ff(-5sjk_g-JHZoz2X}XOhr!)l_PpQ!{rlPn z`(zG!R(G#e-Ce8dsk=&!;r_M5Rm!2m&V%LAzrypeae%YXkHOCvSn!;J++a*hDsU^t zM|d6{4)81nJvb7B2W*PTMMptFA>s8lJGnhVODoduPSiPo)vyP*CKM6A88@1?=pya$hYNW)*&IQfN>dGGSPq`>ytKqekg>U|=vQDABIn^>#Hpn&65kab zD;-)*P4wE9M}5uEnH_UGMy2%1Ul1ho3OBdnS#>qi(AsX<53V<0dn{Q%yAgYZD`q1h z9aT5E@`fn#opH9g!P@iTzssXYN6--stG}qKR8~|~i(6{4vEH+g=Iu~_3#_z*CVFO` z2c$vg&$!-SB+9=8V*Y|-qy6lfq}wTo?Hd07-sRMx-Jpou5i=mQ*But(HoYmheU)!S z$+#6PD;26xar^ok1Erk`7*HV;q$Wfl?R0vv)1}L>N=vf&0{;Mgo0T{%_#N zKj-IWn&?FG!Cll1AO9G91ThnpSmpi-B$}5{ie0ldsC?h4gRZfom8z_2;VP?5*xAce z_ETq4;(xExWz})#WRD)##tJ@q&`O=sK=><&uJI_qTRiN{$f0?dYcMLcIFQ*#$w)zlWgdef3E{4jVMn;sSijP`*9SZdp~k?9@E z!4y|MRWpH{D=0D7p*`%+$vp4;?JluDOb0ZwCGS8-l~^y%m(z!p$Z{gR;yVSOhSRNmkgsp2(tUcu@M9xPXFeYgvW?D}%%cW))aDt3_6#Xi)I z`)n&bR`|0=oN>H_rjuqz@nf(9hupjXnG@*23ad`%UVteMq($eS)W1NA(~-Hu1-|;g3EKgt#W8{9 z1&85C!G?i{aEM|5f_HJ8V9~)kxJC>D8?WT5M)Q3(69Xre<~u9?E9j7l++Hnh%P}fQ1x9{gsLhK3mVwfyapsfW zE8rybuhzE~Ra7R-_?VYGhvXR{H>B+2FN~?7oc!W}*ePXI)2b4R!@>%K_2B6Jdglyf zc-tH!eR-L(AVYehk{MHM>xw=3-+Nmh^ET4oF<$yO@($P9T9>u*n40&i5vhTv;_#fI^6)hX?!a@DT=`5u_KJnD$Ymq4xYBVjkH^(0f4tw|h+3Q!^Z z17NZV?AxWQ<*LaW39*c?IhmoVWL?cNZ63H7((rteFK*^C-GBIk+rVV+@hMw~hvbkE zk3ANy6a24~H_D@H(R7%H<_MQR9wR!L5z~LtEdih)vCHFVOSyh+(q-a9hA-T@m# zp)GznTm4c2>5&WFt0k(*Zi$yaX%%=%`_LT|C*S+3tn%0{d{wA)J(xPA9aIl5@^EKM z#%9Mfa0?n_ESpDUQm->gVe4IXGGM18Y;4;tnuz`9;=cMT@TSO)*sao4S86Gm8krMb46@{p!fxgc+#N>cqh(~jo7MRYK zN%V;T0#Yv?61`iY?k5E@Z5uYjxV{gVEPjh-{XVV4m-PDbG+fnB#tghYt?%sR09yHt zG1LL_$h;E4a*&4u`kbMTPe8`d7kon+1$=f483f8){Jc}*truIKvV7^-pR-!}O4(k! zgsaL&LVRS7k*Y5C?YV8zu}zY>#`qWnkYKkCT1?2b%|ABKMd zu#3gWfM+LN;05-PgkwGSQj3fBuLkQcK2&`Wf8$c9n22#vcPSH%Bd}Uf7ZkhF_IRl9 zuzIHaT3EeUvrkU!R6^kodJzux9vA50@rx6oS0 z*eJAp{Bhd#4)yzUJmlgoUkIx{Dvd*}O9NF#2h#sm(wij?7#)6_cE4&F{3fuV@6c>G zCjKUJu==My)Aw>v2wX*gLufbknXTnqQr9e&_dCi`B{7rbc8alo=Q=KUm4FhM6PRv1 zpu;J~=QrLz5h?}sWoBC=Al=^ZuMgBM$0j&=-UF?pe9pUsmIHJd_DrNp^+>}^zN03k zioN@%(5Zu&%Nrz({2$C09ok>Ef5?#k;BBs8^eSdHHJ|03Ub;Nkdm9ibOB(c^W?N2UheaKu`o$O0W-9vWXooHEg*m$xCUDr^1x3%|qck=M|s4Myl&;>{cd zwPE;d<_y61eRkp{j}o6`a9~A08bt~Xsd}Z2|JW>u#NlLmsQ=mg8g^v9Y*#OZm&v;C zm^I!%%|)cdCyguR_1J~r6eRf0!jvagN-{UB`uF(^{pN#l-nBeIPCGz7=nKd6!}=j4 z5)r%%_{X$&>bE+ACD{BPo_#^r$)-}Fmo9ZQ9S}e|0Z~4sfn<=cL)6@Y zLl6|+fG4fX!cLLfpg;+3^Y4R{)}cylk4n^8KRde?=x$=(*q&?Q4cB_v49OCl2G}kz zSdqbE?c`4Kw5AJ0S7|-oE;}gXtod<>E~?DGlMB9q#FS054u%l3#Bi3+H1a8YD0oJT zs1}*QpDrOE8UeW3(20ci6kk;F4r9E^Z8fKm?c(n7Y^Q&&E=v0zUldtF1!dy<7Wp=6~uWGdoSB9A$=!>$w)DgZ|b|VoF-qZAm3rs!C zIzg+=x6{A*68C8lcRil?6AlC0xXI7*9+j?}tEb^CNtlT`-P2)=*x^`(VbZ%W$Nr83r7j8Dbl z*e%y30m-cTOfg~;0riz8n{d1^N*r!IAk`rb^C68iL6T4fN*Q8?0!jY5_BA+Q@cm;Gab_F=irt(q)3B*?ujGYZaQbb6*T0 z7kWmZPSGu?kgSibMIKqE5%Iv!LWN3H$21m6QjV`ksCXmu&Kz0lu4rwiI&Ep(mWPkH z!?SgU`k+{6wN)YcJ=ZKm{@=!xCO_OwG=PLyj&?=&M@Se=+%rC#LBY|#(R5z*5nXOe zAkV*u(b!(cdb&u7qQ)ic*ik8iPsw!AC{@KW_j%C>4x7P)dIq(kW_RSLm&@KDiES=u zsjRftepN4Gn9tU82KA;6>%?}m%WDXH0?zPGm3iUd7^_>uswFnXQ6|WRcSl}Oj_5}J zH1pA%TX1pLZDm{^bc|#1(b2sC|NG5w=QXdrL1p2Sf(*+xQeiEqTNm@Xu97i-I;l^2dumtM z(8=g9VTEh0-x;LONvL1<(wA=M)!Q9Sr`NapS9pCiPfpB#n+Uy+o=ld4u96qr4cM0O zTf&de@Q)Kxn1gQfhV=6WF0r;~hAVz|#yjh8{s!6pkD!CqW`013;Ar6W>5RVltS-YH zQJ~t}>{{3vGRiNfa6Dk(GRo^_Nnf0Duu`|+%PqJSL9Iz%*L|tGe~P^yNBmv7ol}pm z@9{`zA))$+O?Y@r_=Jj8|KH5xx8=OPp+}< z%_nu>VYX&PBVXWxR`BSUd#74~4QyApDoy5ht?IQ;jyZV6F2-(CK1rHY+UDHhfC&@3 zsxhu2dV@RX{#Imuv}mSuFk6f%t-d_e_H@deCFJ%#D`{46ey}9??e(txvKQZM8>}Lh z>^85fjc5WM(9*YO^vPR`>_6{9Wgl3*Tlt&cdb&C{;|}=GYHEXs55-La+?N*EUT-RZ z!BPGo5$O# zrTNPaIwxzIO7-hb{**VVwUIt$8F;+79g;Y-L);(ET*mXj_juPWRb}%IqO~Xa-xn%J z)t)etc>zsX8{c75qy0&S@ZF)X8n}IqEw%F)id>nrPAX4COr_WtR-@X~0&Wg@!sFlh z)?D$Tx2Z*W9_igk`QeJ!@zooNad=&Bk4O{SC3a#+_8MAdxdFI%an!*#SgxR1VXLL9aq$WPI-H^y zE`rhTCSRor?kNZ?nfZiT6Hb!a*)aY;BI`)&nT6tz}aJFBZ(O)rL z3?Qs@x~Q=_h|IlucOv*o+Z%Yj%HhU0a?IXg|;@^Ur?UflGwKi?j~D057^{4IM` z5#7PH)5cP8Ly{s-F7~RiNQ1`^;g|*_-Z)MQz{*1%X(fs?sNKZ*GB#pDC}|aK&s@RE zD#)1OiMo|0fyu#-dC&Rf1ceW)&2Gcbz-_>~#Znyzr^{^b+itS{YuVk&a^0k}UERa3 zsn~;APr;4p&D}};x&eS(2uOy^>$}C$42bDW^lpu~NS(ZO z?(pLpxnl<`VZIL}2bxy4zrCDkrud6OJ<1z~lTP^$pOYlQs+U_+B6I0I`3FKkw9d|z z zoVZvGKHA4C&aS{3Z>lR=G`{Yqv-XMuzsIt_)zdnrQA)9_!bp}mZpphqJVBKoqPop* z{;g09`?HQtR&!Es?;9cJ-3<=bJv6fIhP`YbIl3CFyy=5~^G%mkI1>$W(`e|M1#)1n z)YnlH+^oi{F=0)NQG#Um%=56=d@lb(0C7 zvPO&DKqRsO#nf|ki^mw?Aos~-w{b6f7>($dkVcGE1GFe;4wZj$Esj{>}V_W7pWZf1FN>=J;$olcA)=$=og*xt2y zH_`&<7emaioK-*DZCAfqC#g7-s7Sd-?Hstmw4xkASZEx;PZKR_w(7r zDZ%}50{dJ!_sbLJfa~d$U=4ZZTR*v9W~WAjjZA>()3FP#2T*J0K4*pCoGn$+8ZvI7 zRJe2CIV(UuFbs@2@HX|HW14)sfLQIy{o#H7Hk2S!Wc^rD;mE5hJ<;(2EJpmSiWH9i z!RnY3Jvge;wHdFJ#K-F3`*9wrxPJ>5206?7n(w7V&Cr*uH2;@sVYl;qWLl2@qa3+2 zH->NO9f_bCXsaP`3er6!zui4{4t30+SCbeK4%$+XD?sAvm}o8_ZAXX7ZI^i0o64EK zo+tG*VwvOr+w*yIKiV@jY9uCf(sV+V{kvr@w>WfL`52au9>+ppi_kwwHD|}=>N#oB zhoSChipOV&5XYPEoN$*wDP_XI*@ZHcdbF|o4w1PHL&-y(b1FBOuJsV(lt{^v45Z(| zCL80z>FTNy*~JLf?T8ppJE*)@I(w>2Gis z2DLfUVMWhgI}uovo)nf2tWW<5_*1_0W&QiN`BTxkuDGFc=jD_7P?k;G?H}RfbN{%? z%{U!JRV%B4^cVB64)^&AU1)>=Y8$>CQ zUVMCK9-Ne6u6ZsZu1uD4u(bl|`Wm_#iDnSo!=TL1Qh98k60F)Uonlcvr)R@BvONgC zKIDT$ls{O(4cXj`x_vKukhzJsMt@ideC`W(U@sB z7FXK4^+%lA=%dV@-*!sM7WK9!I$GfitrVO1fwD*v$!|rEf;Ffs)<&+<7d0N+9j_4GzZ;6s*@Mmn|ClYHbWF5Dmd=0(y)0NUu!8T3DOvSqU>c@~1qo->BiC?NX z;PZ8!?=c5^%zYzU9E*rL(a2XbCgoTkJOLO9I1i71w$t)q7-US*7w?&P=N6R0jmqqI zt#m?9t~feeF>-t_BY&EZhU^GW(ur)sQhDuK9HIX^C=P{UHXXjj1&t4tC9_CzTn>D-eekXm(-vomfvkg@KR~I4Gp63 z6)T3V|CASbE(8{6yRQ(``qIR?cM8PUITmwBoj$RPQu53{$qh_(l^*}60Dsx^`4ys? z51Quy@&{x~Jyq}bTdQN0T;X@fc$^xIR-oKc5^CBVG1nqxnzOza1qzbs85aH7>vD_qH zn0+WSd7Iy|S~aPBEbcf}ko4 zfEjFwR@{y4cs5~;El9{sYegfqo3Cf>G>U9YMaHJ)P z)M_;w{9=jObj`*43M~xkZ-gx}WHR~3sj5r`dPdrG;_EgW3t$VzA1jO9L^U!1;VMv)>@xRg1d=W-Cr>rw}ah>l@X*opXnI% z{Y5qYeD1}J_zs!WiNKTog;f-Hc%X)hube z>8SkAbRjjzzgng>@-%{4+d(%1T+S55)KnyXp1HiJS~1y>3ePL}qd`pEL20s0hJK`T zNMPWuYsJB6byXU*t7Ec-Hn<~`aypnUmqJx9tk{vR92UX3|Ndv{q$*g>&2S(D=D4v_zL6w7h7Mn zqr=r%hDjPe!sc-C4ZJ@r0vLW2jmLnPGIG@0emMlFT#Y0B5S_|$Uy-wK*hKWg%4f^g zv<`>wHlSC9AY~hbR~4tl1tF<>8O7!kBGNFjD)TQAf4%>;JFP6NbQS$FD7Jw?U7Xqy zi6W9Q+v?*nx(H3wU4O^!L`_9~LXQ<5HVeyk>~UBq2`t|bZHB_aX>_rbNqp^W(oA$0k}A z!@ajmAGH3S7b5ZaP6slkusgjT4p$_89nTiMKJ*v)CF_X6n{Kf|7qN=(G)k+9Ac@e^ ze+R*?Y^mSbxbGS2E&e^!OAC;%7eo5}1i@QkapWpp7gs}_fG+lhXnyY(JcOAbV^8r@ z82Kk!v21T;#!I`8KF3wfTUS`8HTxG_6A|Y(T64taMC9&ma>1LE{bXQ#V^3AcV*^!u z=Nv(0ukren#GlSDe#ee##UNs|Y_D6wlRy{s_rVZ|l@*+S6eArvGbSgufS$c;gVhh8 zDh&a5xDNX8s&dQNh)mzXxfXQ7ig;L z`Klz!1GUKL66^lc+DLo!xSxJmVYRI%U>;z#96%&nRDNy_7gXpAgvw z`NZuy2Iy426DAR?icMABDb~3^WNX8gYL^c{YeNs2_H%_h)^u?}T)*=e;p`8|pf{mS z1jgdm@2Oh7js(@kR$q`lpC&vbADJoUm-&tt0mNCFgQyFxjuW4dOd*{or>&uhf+l#p zG!|t}ugbe48Tns*Me*R*U%R-@|Eaq=&%EnaBN*7{p`Y$nEB)CeY@vzDXIMn2wI(48 z=0#-8z=cU;$`>FZ45frwvL60h`BV|z=CoT{oIP{gM=+@;s9Q8&)#t`&G}y=Cye&Ih z3alZBtBUy>@n!Fj&U)a{MWmTUWAVK&BX66F4J743tV5bZJ<7Ow6BAPAU-t#?hQ@Li zqy$QB3+q^h5(`J{EhRd^GTm3^OZoeLT!kk5lcUwpP7kte(-KA3qtF~HZQztl6xBhy*=ne zEArwtyS9s#XMWrAEqeRGcj6P181)7amUw@;jmu&76lh9sh(9%2b^g)FAkT6^|1DP_|#$FJxNFEdUR*Er8@kv8S`b&9cjV_C2p3p6Cwvi5+m?UL&FE)!q@LO(k4 ztLMkuq}q0muGrhKNJx_V-(DO#DZjOx+SJ8$?T4^6Y{1caLlf1_*4SrBxcQvSZ&K;$ z>AbS5=R<|%hAX?MZ5rwNy?bWiR5I>REwJ1GIUV276}#@lfg#8D8ATcm(q*>?%eYEf zY+yA%;;3)2o&G6JG5OrjF0d8g)pdz6M=(Bu1--id39Pl7mrm#3NkKPA^V+F%Q)V>w zJ}`gp-lHlrti-=sK+JR>*Lzka7gRC&O(rC;0)m>>;=fVL9?Hsl2kFcDt zCa=yN=W)OH^}{!A46SiJ9QgW_7q?JlhOKLKN+WiO?4c1-p(y+obhVS^u{L=|?`-zb z1I>l#iIK$6YV~()OwZA>PD(lQ0xe3Kq2uFSlUTZ9>;g8@blSf+m@4N|s|`Q`-uB=Y z4Z#>^;;i#RZm?)x=hFkg;k5LZnon;+?wsW##+PBJYViliFtdY4`s!3<$+VVn7j|g~ zf28EG{-O;EBsCNG91VdJ0q@SWHuQR3jn*ghzdjJGMK?5WpyNk%^Rzi8O9uw-4V~*1 zRw9iWad}8_u^1lnRirX?F`M}J@wtTta+_0-`E-XvNbs{rjVgcw;ltEBsF0;676i@? z!hhmg@qaoI%Ian0Co*rDm6&eS%ywlFfV6)ew=(qHeD|nJ(FXq$a6NR*7GNpc@Oar_ zYZ<;0PbGW#y8dOrLwf5k0~0E2!{4=HJCSb{1WGFCW6BpL!C|>$jW;=+V^s%A+u<^F z1h$VeMMi@1d-FiAc;hrkQr_tV+!tk|(I!sD`{)m`$oJ#e7<7LjW~n%(hL)o!mu8xe)D(R&I3p1)jb z%4B96T0W;m-#5a|^z;0(12D!lwT%j!zc{o#Seyi09Ipa%-Su;~jsuk83?l2ALw8nO z4Yx7eaXKpbW$2qG2its$t~hQ*?N6(NLHv9B3l2lFsDATA8vWuJxNQ%-Q5PgeXztUx4{ zuH^MDJqGfwzyhwW#|O@+x5Wg37joo+&MLkKPo9mq#Ow_(Y3e+_91kDNl$E&7vqkMr zIeLhdTyap1RA`=mxoRIfE`#>6u89p?9apTCi$9wUGLLgPO?Ttz+s9?aWzBLltg#pn zNbB=()l@D9hBtbhMm1BOaI>yti+S7RXF%%a6C0gEfBRCvd(b zWq7VzASb{)kRnvmw!gBE?5PouMY;8Z4INs$8@S?V_=g0J7V+|rQD2?mG!7VH6u0Rg zDdx6VMTdLE6R;Y$yEeeT0x4$lV4FLX0GD_BlTza_&P4%P=;aS;#?up=R$(URWFl4^hyLmV?Tl@2bwBD*g@~rMW?o=? zd$ZwcP6>SbWm=#nuF=(gGiGE4{@=_mTPn@@b?k zM^eij9ygja^?lp=ioT5?V?|8c0Im#thaJrYU)NUPxWzO2CS~SMo~ps8t}#mulOcVX zTl)!^?=eTXvUZ{f>2uAg;N=-U`OU83X5oP<<8hXTwQf4o+)wg!`TAo3FV;}U6hOYvDK zLi$6Cd45QSXaBgWNNoEv^Gu$mW7QMp9Cg{k%EneSN;dV|;POZkryK{tExK>6uXPX7 z!myp!{G9g^SGjFdA?1k{wLnZ6<)JKL<7Fd*I#ma!9IT#GisXIhWi&*sJ#Q+xisqSM zBemf9eKc&G+)#WDJw0oz^Ld~Yald8QCsL9uB%=+@zgc_O!iHR4?<6NS9s)5LWC+nL zR<0vsM1U|eOdfs~oed08;WE;xN=5ZlL)O*uhpAxSAtjtoW|G~%VF6GkUm=SIMXanG ztX0oWL@!gQc@bHr99?JGB=<{VPkvFLp=yoBi;sAAsH6EOvZI~~4>#LS%YU?t$@~pJ z|MOEdia8xYng~q8DNtlV-6?`xPaFE2FI5@CO#d)^5$XBMD!9Z325H?@c@Ieq(~|~; zV2js(sIW5Worf9aumpZx+etGhyCW2fjv^cxpklrw*L$7jR_ts6{R^*0ZnIF-5uH7M z5n>L~!V`fR3s#4SboLAK&>^bC{45wRX$uiT#8lMF~bEC$RPvfLq8>|uFIUiGa&V+ zj6ib+F-6+!vcSiJ`j%=(m>^|Kj_yrqM@M^mL#jsKq?IT{CZZB^^ld*!n>2?|UoH3~ zp@GA1?_!+jMudVpKbsS(+Zc5&l3CNv5N%QwDh;qd-nbn_N$kH z2Tp-P!$C7itMUVIZuZFxhdS!oRBh%{i1qDUagGwJSlurOzVVc6!URq+BWp=8y4VJbo6wC zsfjYb6Dd*X7{|}Ol-|qnL`{CO+oB(|^OAc;FWN}@Tpa?{aOzARAWS6`YM;|tj8jyZ z{6jL=iV9Vc#hF|*!X)pZa*`k8eMsz(`g<|2d1U`Y+1jtPYxqo3lbJSDYM zpZ%gHxx#C^BQUeXjXrpyRnmUhiQGXp369?_{tLkO@>5=N!cK>j*&4wmll6nn-Yr2M21obx|3#?8Av4#j<>)TH_pb)z@$SGC;tjdMyW?|A=uoI^O4mh z6(5lPc5Zh0?dNZ5J^BBc8Kpc8s{ma9x1LC|2FgeD#9Y|AR9PP;*k(Ar|HISHR3e%r zx`O2@qo?e+7>oFIaGVn%Qhp@jMr9(9`~j zQ)Z4BQx`5*B~i9TE5d(U+%$HKDCC=bzV)Z+R2e^vu%@O*_3nZ(l^ZLM&^ zI*e8KJ!m5TzyCrRWCNZ6ZE^4EAB)%qa|cOm(nKp$jk6-sna@rj-tk6%)v{r(dD^o_ z4OGx|0%AR#>(}{bn_p-J7wu|s&ey8OLq_E!O{QGYWf*a15lSnvX`%7jx_MrQlJ^%H zDaOCqf%d_};@J=y>3RdbVfgl41y_9I2}Y;nDR$SNL02e=c;b=8 zIVth&ow!5v236HWqu(k(2W;%m-&}E%1rY`407e9MI?3lH78{}mYxVMPuIemrzUGS2 z#xXG=gZ=F~&yQ%@Wf)gUc{K{KA+8PmJLkt7f0qnJs>mThy+NmUF$&CIj-k$paLbB*QfKlaepfS{W=k(k#$x0Ip{k(=d`AOdN z-UH<0TcPLZaoSzd!(ZP6VxYv}GswNu6!n`V7u0mnn5A$-IA&#iCsM^|k_&e$huLm* z%Dmb+#40K1?CPIqDRedOH;KuclmE{(Si;(&H&QB+1CrLXSUSX4$ln%`A@idkV3_;n7u2YC{RdGbuE6g$lC z3$eL#s`O9q)K6bciFR;}aHwTrJw5Ec7kTj{$u8XKu76f57gG^8VnOe|FlUN5f+cR- z{CozmihAG3S|zWh{I`M&29>w9d(P4R3y7F6Pc~+bqrVaE_^DqigNpe02Dm)T_=6E# z<)p&KK^t9yZde5h4VFB`*FilL-i{)*&yb-%X-$3O%sRCB|FvUiS)q5-X*{J%6cvqi zew;!}wXB3|Km%IvW`oZ`QyfJoZm~Jiz^zb+rrSlo@xNZt429G(Y!!jC>`4|AnBdRh z}YVo?9WKaz>}DW1Wz7toz zg!L`yu09V;q-nq4@PbqA(jVQ!985@lo&ROIGi`g9f4*pi#}ROGk@neB-1ClT2Z$e3 zX`m3V!hrW~yr=v%74fIL;j)2eBOQ&VTmJ=r04G7K&OdVz)|aY{3Q@f3Ubd{yxYtn@ zJCnqDNIcqDgVY-#v39c3RSC^C*DD#@pq1y{tWT_90EKulM(Z*1za5X^xOxu_icKHK zL_Vm5!HzQnG?&ZHj|Zg?9}*14O*4S#+yE z3%Tx7oGkrQm#O@I@uBpR#297UhP+mXS0Xf)NYG8KgjON>m+e1xBO{EB(Xid&9*i$} zmAK?><}cf*EFT7T5fo}{lr6UDFv1Y02}jMDTo7pYlJC*l(4?o!NA4gk^$D0c56}*l z64e4zmXA4?_qC97kguUI62SB7LMQdGMURn#n_~J9Fs@W%F{&XW)s_1jMl?J2C`4vhcF7Ol5S3R?Phyu82L)Sjgeiq)l4Wb&qGA=MkY0>P8k z5v6Fu9!dc%)gJ*1FGaIVSK=-K<$F+&Vj9;MA&=8C^DXVE$S-vmqRUsj-qO)Ob%`K& zKQCFww>d~!k>(Nid!M25-zBuJBlmQBQm9{VkGNINq>~UL5PJp%+s&71ONHW3XvV*o zni_Z+=++ugl2-VBM%5_tmNBrzaNKDfc8*L}aTFhiZ2*L%5E+C;F#85!dlJOc5U=5V zJ>rQu=-;qpxRHBs5u&~4zbc;>mv^&#HXx&Vln`7br94Y{MwV2hU~u@Jfu{SYn`sR~ zETf=_h+veb-SVrm!--iWLN0&q)}IZ``;n}oJc%nf!8?Iwyh9ZbzcJ9X&M~f_p_6+a zW#Sxa{u$7N2~tGDublV!vV|FdtNad4-SG1}SC|k}4kB*{XWb^rjsAenR9-Wao||E7 zlVoLyt|sH{FqrhzE3+LQtX}5>Xq*yTz2}l?=*ikJ z+Y^}B&uHOThqN8zn}2;*U8?2#<0R>|wb?QAULb%FNw5zXB{Sna4zdv#Us+D)Z-IE)lNCU~}=J6#b+gWXe1K2JqB$`->? zk(k=fNFwDcBZ8jibMub!Txf8Rq?UGhSd6DDVZ#OZhOQsD?VSxhR^6P37>fs>C=inj z8Q_&TWcI-4t!98__+GcMBK(fe&gL@QnS-3#IRES@@b^X{`_9d9{vAo+J8#eD3t+gs z`BfS!$x1fv|3dwkEkvnele^t7A%;5Xlf&uP#D_Ht0$;!D)>^3|OGT^rz7hgM96+ZpXl6l4Tupii;IX;24b6x@ zNQa_ELoJP08x#f=f#${e6f8UGVt&#ky2_KziT#&6h@)Sk@tDMRD2?4$Bl_j3RFsia z#|{|4gVNtzty#ub8eMq(=0txoZSM?oA2Cw|H(O!hf7aDs~36p9(xn15V z;~r0(Ru(rDd)UU|y>0cr`B~KQ^x90my`LYS43*flWGwqhd}4r1*D1Nnt8uXT-)vFy zYhs51zGZvr6eI7nkt}}g$MR|I`6nnkPK`wBnyVO2O$ep4(8%~?%i9jE(|B$0wsoYm z4DB>YJ0~8Vp_pqDSXo4FEClU8GFV8js|?yC(-h`;ZyG+_PY%RXyC$e=$Njt-Z+bwjq2Pt#|JInD6@6 zy!B>o+IxmcI&0F2Emhpu?Z@8@`Lh3l3ML{3W%yD3T4akw81f7BURC7t%ZDuA0~%$2 z0+NL??OT!+HIpP}G!&n}xL9gLqV@lv1v20BPtPyt)Qx_wJq1#-9WEOUCa-WG!^Z6R z0O>DK)3mHWVOQsc&G5mZ3|po*-(yZF48FJBbX1W7-W(8ndv-aPugG!M_c;-(zFT`5 zKqB|0P!W45C_>i({1*M{SGQN8T|31I?*BpGv?&Hu|CK?|NCdu`rki|s-AU3e1-ERf z06)hJwURubs5faR6#8c2UvUfuNL)rZtD!vPP%5W!mtYPBv>vPY8azoy%0)_Qeov(A zf=$w89qvHu(9yW&vF_&##mk3tKLvk>a>EHoH)Z(Rx$4gfSr1e z;l}HVg(0v3Gaw}GCi<-hIka&4KZ10py!rHJt8`o%(4rz&D^EczW+!g%2rr*q=#R8rySNqZZ(2OIlXf`tpU zayUEFb#}yF7oqs(`LX=&2Mby5P&WJk*;+H^HzGPo2sW^?I^=OP+x&Z4|IfT0*W=B+ zTkSke?$x4YPN!2AO>vfYdU9a1WFwt4QhhI;{!k|pdBJnTpXg1v#Jx<(PpWMx6i44S$F1(R;U|=HsSlWkDm~i{8uSm%9|GGi6%s zO|{p`uk^bZZaAVQa9VC-ef?N}wcGZJ)&Tu5W@G`^Do;o$8Qn$_cHEOpKEN}1)E^)Tk#iJ4N{W!ynL~-gspPj%m#xLi=Yp9fx@6BQ|mL)n4*Tb z-drQnu~=$^xP`u0;$n;nhxIltLg_S9azJfC?fJ!d>KsMnX~q{HYkmX zIR&K{-0Y8T^i}2zYQno*=Bc_l1nPL3nzMqEp*>`ZikqgS@0F8Go9*hsZgYm(^;Y$E z%UY!H(}Gl65jNXK)H_fFvZMC!N-}%pMNWnkKTc#zHmRR)pMe}^X_gOYz)DdMTsa&C z%^9`&Fbc+X2uu}{tj;+)^V?4m>^wue;2ta$wQrNfUk;=diP=;Q#qN%4qGWXE(WR@p z>zC9XZ#<}%W}C|Cg7buxqsRIOQakyISOW8R(n)T{K=QAgU;Eh*lJE_JdWH)B*MS(q zTg!aN3^S))%qRd_MD9<092XbqW1VPp9br;>Sk!!jrW)!&(}-cKgzNEQkFr_%5EF#Y zQp1d*L=TWSaV>uJg_hFguUj$f<`6Gp#HreeUtXvaE_cam$lc#o(^KJ(q+`PR(Fq7A zbnyt>g-oF(SLs*<_SVmQOIFm4Z*5PT67xN{N0C$Q>o9=Sc%$=Zj43B6L(m{Oe``2f zNY_N?A8`MwTBhK)jWEn;?Y!bo;e8$dvLvIEY1BS6_`FE@W zgowIKA1RsDzXs|pihagn&+|8{WS0A{ha4*Ne1=Dnrln+-q zTjN$rzzR%SP2bzMci*P>xm5yCqP^)Tek_E#{y)Y&y&lR+|KYYJRn&G4=>mytji+)N z8hIVihC(cGxfk4LV%!hV8G|A|6TL?pcVn;%RNLOv`Ur2fyfjn=yhX8RBnC@q^3o@K6sQ`eI`+&iNW=s%4qU%W zg`>^>D^gbx)pTF?^T4evVZ z{0R~=nG8FVJ-1w2wgGzkaA^c}{ufi$>SXBw*7At}ox8~N=mj&q%yX*A+H~E15HRR; zTlXMy<$h*8LE;0tk2p!hio=A!5s2cAe$Fd~RDP3ecsOZtCROlpy2@qK=sdpdXfADu z=}e!lO8#av?;Z?Qjzw**)3@L3b4sJ%(xL)w{HEghGv1-=a%bE~)}M{sN9I`^yA}gC z)g%{0bo^gf7Cgdcha3~CZMe6%xdYQ_55As?CXfiIJvr z4<&==J?N)&rjO1hv$?20EN$Ckmav(qYnS^3OjM#H!MroGrpGIGZ=p_gs0Q^KfDm6h znNjxKNyvie$ZfsYIO~(LyNEREGzTd+s9Zd0qFdUrD+EFVF;{yN@RWHy9$$UfkITCv zQz5{I=f?O=3_br?PL1uV3aDg3X`$8%i7YZHA~6tyoEfMcH8tNa6kYUFM(JNQ`q}K$ zP(X!#gleZbJ)xw9ere?s%XS9r-96-+`pw7>uIlU&@*!os#KH!?-!FOgAs_4H(rQy} zD5syFkk*nr*6l1<`m6E<7y!=`xvwR(^X!-!b%B{v_3rQMj4>3^LKS|tfzTth9a^I+3ViB!%CvykRy8k)06FjP`{Y5j8tHJj?^ymOfHvxoE zOckVbzbP&jbclw0e`n=L2A7te!oR=Dl7er9km{HI8{}}fqUL~9oChF{E_M>um`z|6 z2s8_cN6A*d!zgW0GG#nf*)u@Vh0f&IV;{d6&$55A!vrA!JzpmznyU^aVrOxI1Bra9 z7zIIxd}?QjU5Vx43=ZN(+ekF#<8jsW$0gk%G%f*Jg}KHL6Ny@`u`0Z?7-+C6Qs+hP z%l+(0lfQ)jlwlBR?1df>GKt&Vq}&Y&$tv*=nT~`hK9@FY8x& z0!JX=Z9Mq%hTg4FK?L((xn0Op{UmOu&6klGa>O8~0)fk6j!rto4^FB66tww=;X;@D zj8-|hnRjr)O|uac)5LZJ!=i_1-icy~fj-5bzK>r`*_Kk zU*D>g=%)w@_YnoM@3#t4H&SC*wLyH`D5W!ULE_hkUqkR=6ml+%)$&={oIX&zL1+RK zS`9Ao?7|yoQRuWw|Ap3D@0yVk-uw6vYhSg4U!)t`Sz#MFj!_V~nx~)c|I~Q@$-tNg zV~IZ!YY##qxunLK7i2R=6A~CDLl$z+v)2|_1+^P<3Mh;z^EQ9Wp0$Qb8l;ShblGaH za*bGC4cBpkLXA88+GAu7xl6Y9vLh2uEM3&zjendn`?z8j_aE#{44$!wW`ES+?PC&YbD@8n%_!tJn?YuS=r+k`Z@Va zAt~DmkscU2j`4-pf9_|EZpA;Mki8#-zu($xekbvFB#prSY3|zlVkM&ngmFb zX;xz=rY0KIzST%OMT^XK8eVwc|}Z2MY`+Dh*7F>Iw$fQe|jGk zPTY~DR+jyNrAwnS-9JA45Lxhy$||^Y%SJ`HniI}@;!LT%8m8niw4bpMde5GuD%gj* z*&PyBj=rF(*+H;qHiOSY?v2+gDw`@Gov*+bSEsH9`B0&B)MqbT$gH`>r3fA&C)H8m zmi&S*oS_G{KyPLFUgL(Rwh_M&2T31O+ z+};a7gls{m-(PtKak254zy?!TjuoL>2f}jBSZ3(9cvaIXsNz_S>j&rTIoh!p+L06Mo^w#MKGu_e7Eje*! z?RG6OA3bdIhX^Ix*Kd8n6qW4Mxj(e&U^DvbhS!185KMt$V+V74*u;2NU$kJ{kqYJX&a+Tuo;pW)j2P^x~exi%cO4D+UdG&+y8 zMS~f{yfsPlddd_)Vj4??X1B$NyK6^m0XY+$5~pz};_8T!i#;-TC0vieBGSev7oQ1w zl0JvHu&Hb~DWZ(u=+bwQ(S=(G+oz_`F>`IwsdeadxnIT?+!mn-8>K^kjF^ z9d<-R{OH~ZM_SRr>>s78;xBgMD9B%(3Qdxi`fA#Y<-Yw-Osx|)Wp`%1=eOsKx>~TD z%&6LPW+x{2OxgheM8l2J#1fd&frlla=S}e_#*Pz~iI5bp7A!pUgBT`J1u2E?7M0aI z0lrU^xoRNT?rt(7V|~%<%A)DmWW(9w!*1Zz`9w!Fk22wRWJIs^mUVT|DcV>m1?%U; zoi44`%-yL9ux=3^lif3?8%IQD(p;&-8)@29#SJgu*_@;CGB>CvXuE~cR{?M1P6fMf zbrR@Tm{Jh+GGu}SGch2{dW{oB*5C{2*bZ2lfc-4{S@x>mQ0qX>$1W-p`_35&O-(|2 zSd9`x?nLo5$j;ILmVp_LEL6OR&XO4D`d)R>0ad1ywz;dDG6UXG@sT_8Gr}(q10k94 z%ujs32l$N=eKFM0jCf5G7n;w6`U{hi%GZp)so}$3Eg5;tNeHkrnnbvd0$fQ&1a)ET z^6OxjgTdv;;-0O>L<**tYFJId!nOpc1bwh4lS7B-*#37l=;SLe6V?EJbRQbE*x_$=#dh#55Y-K5R%xRx^OW`Wot8ELKHN}VCLIp za$t+xX!K(AO_2}lssAjiP<#5LyH|~58}oW-tOXG6|GX8dMm|_0=)rOYlfq=5cq{zG zkgAiv3%=TT(Yf?Yp?auj$7gb>#E|_ERHo6TPr5gbdlUMc+^>!sqn+#@z|u1mF^of5husHL0t1lKi^x0O zJphIuk}1`@yWc0BMuACq2kd~_7S=fgt*rSz}NEv?kPinf$ldHIvi-=sJ3y7 zy4`pkhf}_uB53O&2$Lz)dK3Z=D-!R`2ab>cS>ylzG2Rcdsr)TWK~o4#_~cSL^Z~g0 zec#w6V4h;c;Q$`08&+hd8oIpP#zQs17){hJ%n74LNbBxb_SdPEIQLh!T0Wg8UA?r& zw?T$L$zXDc(2l{*_z*lS5$97;99foUEAn z=m0(uOCd}zB8fq~{Fb91?G~)+hmIW!2_y5}j_`*WP_dX@D4t@LB@gT|mPtU2fAOD#-?Wd>TFa83vulp1?&G$B0{Mm zUvi-~rJ2CnX(hknKfZF*tSB=OBApulXorTe~InL95w>IB7FERl#5LWRVXoka3NBlW-tg!ra( zk-LadhZj|Rr*?L(Uw$@o1qP8YnF=4j)gAU-ee&($J7!fv(6GUGtX3q5RfM0ku4It$ zI?t}G$z}}#d8w29M0Uq;e@Vw|Hy49y15>sX-DIMHagS+D4pS3QUjX5Pr;Ivr@p$4$ zk-vnmVVT88|7ZA9;uKIjqsbv8G=NQ*^h0%@R?zb*8okI#G2fU9ErY!25rF9!FV)*y zXsc;CbM5~tNvpP+`3^~*Ef*CBB=tmD=*5R{8%ho9nIG36gH8mEU{}n0a;cnbD?jVG zlErF!a+xj5m|+;4Yu+_)UkXMmn%%r@9cKln|M zsZ1Z;2O9j^p7stTK?S%Llk7^|AkM+NA(l2NU@rsobax<DW0V$EVSG?8BPosbgzzho>Q`jl%4 zc+93!bb0*78U24&)~N>RJhr6*k7tyxq^ZHJOq!s^IiQZTeN;6TlO-Osw-@?(95%kU z&*2J($M-Z+ZQh6G_xz{LOp7*6C0F~)lDgj8#^$r7q7n(6(m?Z%E@OS9Dd>QkH>5cWeJXE5HO~*08{9Z_Q1o!vGDBqP&s-n+m_?7a}V)O;w{V+-6@dd}uqZTzv4)n!#&9oGqUvsyD0Oy)Vpt;G{qQpR8I6 z_tG#pN@xP0@eJc*I(J(rba9#hGF!v_f#nuJ7*U~rH40uWm5T+4Wm#cI&IEWrH_51| z1dui^)A<~?(jCVVv}_vwTL)+sD*Mw7apTgDcZtG7{HHCH7*9_W`}0-#M2v51Y^F2~ zYFx==EzwCh6+tdXw2q>u!V1DnucpBg;4|8J-wxVBJby82>`3@Hp>T3L$(p}4A9tIq z*SYOGG*TlNg2-{tf0afj1*rg<@_*(l#VHpQR`B5>#N6n*2&lC3Q4{*0O{x3@XzbPY zMLzOfmgc53!DJ1cc2mCLRY^|TN)8*k&RbEPV7WZM!dL+Grt9nm@C**jg5G*ZVcr2a zl#PIDf8eiN`vZ-*V3aRR--n&fhBbF65_z95g#_o2$*(#WwrlY@n(I?%Q!)`d zUV|d17=)LX0E0X{5r?*nxF3v50;3y%&EqN zZi}^Feb?Uo{A{GB-`WlXxten_%ne`jfVNKWmN_pas4jwMMP0Yg%wQTf+h5`u(D`v9 zVJlv{It>qHkRJlOsOc5LBtYX6S;e= z8k=StPURq0wh%s4$u~*76Ze|@IGQP}_gXDCUo0&d?!BVRS4-)o>jB=3YG{lRbZAVZ zFZhPN{hp^SNcH3fU!fGvqi}}_tiRGx9@qRZu51T3zSt~az)<}uNh4Fu((fQVo;OxK zj-2U!=|YB$8*P_2i}Yzky1bay2oo0)i*MW%`_eDGUY>59tB1{ha@|MU0<4&SO6LI4 zC}(@(@3;Qh9BJ&HyN%1SnXK|+i}Am-$=1EQ!x`BZ;L8ync7P@*mu;990lM4I4>|Zq zOUAP^EJ8?aSZ%g(K0rN#m$qLjs8iho* z0)87Jbr3}84BXxWgX-C=nfb2VpT_5iWU-2|B>%O`~^Tt*4l#XQ5M? z=9|Wwt$6Z$PUAF>J!*X*V;z;*^1~30U$kx0k!U*)YzF8b4wD1<{Ya;6`O5JCcBdsB z@_YwM7>qdtJ|K#-gSs&IwP(xB1~PqCYRTEe{^^)P1PO1ac=?Xj#AiC}y~5H!MC-<# zhOIC$dA?&6!rLbd0;Y}%adOusmo)i-@6)cV7J?VSlt#m|hDK|>@N5>|asb$Zn(uxu z+bLe9VDyf$3aT_#%mS(u^+S9=TgYeyAW14a5Qz&i_ECAc9_c_I%UYg^NP4Vz^Q0&1 zP8T3{WJsOob=h@Frgs=l0-Ci}06_uW38+#g0;~qTJUJ-!tgdHCx_9q>Cuo;;SMTLM zsoX2w`QLnWb_8RusnPRAQj<~tT2>{1+$q5RBXE+^FGQHdUf}a6aBe#NBN@(_0OHLah22 zCG+z9w7nT6l$d81-#u?tHB!ydTibUt><|t>OGt^G2Wre*HbzC#3>TeFS>NY!&3yln zr99hhH6fN9``I2glN|q*P0&u~lBEX09>ka+A=i~ZW1zD+TP`%GQEl-ZY1bo)7GkU| zx;c6A{_ET)&h}vc7W*48;eAIIE2EG7Mt`KTBD5lr!){H)9bW#I1O&iSBt!|{sjr)H z63i`HzcCcw9B}4*o=5H}MIm@hM&7w{x`ap7M!Nb3@G&i&?T)OmlPH_Y*nDAEaSCfx zP|fn5cI7AqK1lijRpu+S5FngAN2Pl&?03>l0u@!Mp?bFoEb}NOQqLEX2~y{2rOA8+ za;y3~CDJ~8bv0rEw`ig!aSTBfAwR$0j@kCh20^6LJ^*wDvW{>QBg5Q+Gg?w1+hG!U zT)T)kWv$l!_av`#!d*FY$*ilURW$e&)?~+wBpsH>q%t<$G0J~R1+gq`pw2V*BQ5rx zko7U6;Jo18STau1*#ZbV2_Km2Me1wo0pf>r6q%qAz-5?&2??Hya=QG)x;1`Q-D;%{CmzU#oJc{*JYSNpEBg7lyU#uoYnS>0G*P$ zE-4{qhethKX0pG%mNxA@TMr`N-%Co1O#GvxW>CGG1DQEGqsf;oPD=OxSnSbV)OC;& z6VIEPyC&JcZ%qMiV+2H!?x2uOCE611Gnp)Db;LHX`8jFNL7Nc95RguDpRlxEi<6|= zju?!8tLG|)6go?zJ=B+3@2izkLq;gLc@qa)RA;LuQF=lCQ*!bG;~(bqW9_(XYt5okDU#%C_AA<5DUK| zS^1ENb?T>L4>{q?1B!0q6p-})TgsD5EAX*d`O`q;MZ}6owdLh+S1UwQJSRz8%rju{QYxri&KVSFDOPsE> zYy(Dm$Njb~*VC8cG+mE*BJC!}I8`v-ED=DCn>sC_V-e2H~sFkYw3!$?mS^%9xVg zP3)hvy*z^$I_QSs2ofATQi!#rk{B$9$-QAdHl6EUs@@8UaXt9#%p+sBn*hjqrO&vz z^jt2mK-=Z??_g}9PF%a~r%yG33Y{9gHVBe3Nol$*FpcyBclYo0wW8vE&Uq{fRlujCd;z)T{bZ2g443J{tbUbfx&N zsEiiJ6n5`xU#TDI+sx}#>oh-Uw-C^HY zW6l1nktxl5db7*QHC_Xv#BLqO?b+IMIgbrEFl#UfJp7Tan$Bzg$K~U5Fy>H!lA1n3 z7te;;enGqu-+{?IOmbP#CerSo&e(o>;~z#R96`?;6_0K$+vV%688zK^QSSgnM+x$i zJlw5e!We<%K8b+qx-VSBkk{&B{6hUpv`9ud3JArqTz){OCEWoP{%VL2X(u9pQnTrF z!GkIru!9#m)I-rktb)>Yk3u+0-;ZVp%G_NPAn+R&8zx2>uQMyVsMJ#r%8!29JA7^L z{?)EE=(Ix^CJUSiaPAjNc}}7u}S`xwuUvn`z&e1p33V8xyP`{G%dk&0t`H6d?>Y{{#~&#;nQkaRw5rY6^Qyu zt>!;z767&X{?0@Jd$QplWmXn_i3B&pUNA+bzE{L(EI${YAb=3mT3cFH z>@mFu*qnc|KTb}5elQk&i#F%me)lm%!W{13#PF{Pd;69^iB&g_DD?KXrw#wa@>g7; zt@KE$ouHMk(*!R7^;O(2H9EboW6s3tI$fkZVMZb#Pr? zB7@H`#c7w$rKFNlRQy}^eUR{@DBydSPGxbHqms}``sp2+k>z7GbZm)RrDw&`Vvv%< zyHYc^^HnC=4W4021y#AGq7IBxn0dA*WK*7?=3agxb#bZBj;-eVW~fi2?}x@ct%M;d z3We}Hs|K~lknk{_!;mE9xsB!0iy!BcbH6kH!Mi4`qhe*7a1Z!w z%ai^kv+M}E%E01M-WXD9&Eg!(B}a7>A4oKWPD;qi;;m4{#YV;F-!>?LZ`knKJ~`n1 z{&yBeC$1WPVQ7vLjhKX$-=ju!RBaMgCXRM6sxWNALAlF(6NpGfWY*00_ArP}GQ+@- zck_y)JaJ0KHMv4`)i90UO- z{Vyou9-;?wfNi8?kp+aj5qEDQ-OI%%O1=16R8lTd8YtzE z7x`dUrISuc(Scrw|AvVUG*1#T2u2rY2PkV8aMO~Wt}z9G$n8~L&glJlL-tj>M&LXx z0e(e^eD^i@*N--|kJ#tfFrtn>=a=K(!fYM7b-e)T9|+Bg2MpbA=%Iq%fre)iiCp8^Z2(f! z7^SwSEKG?siWPOq={FH1W9;)Yz?=WjcMBwuy%YCLFf%4L?lC0A)5SImuE_0ple*L$ znc{YvT79S1oOC6R@W2S}bp{spgn4U-=p+cc&oqLuXLE*82by?#dJ?8_Mw&A|BKS-9 zqt{fC9^_rf!7Mu)y;&uYnwlN^D>2O@*5*+nXNK-@)UAQ?l-UtU_BjTTZalw)X+f*0=RedlKTD>TuT<&W{NW((T68? zO|VN1%vFDtS(=>P8tL+UnGw_BWodYbJ5wfo$5flTvkVy2cW%27kUm&?da z`=^r9LxK|D5S;^b(;$B-Zi%L&EVnguw%-J>^8G!b9{xZ%>5L^OnRZ|ctCgH4*Mv@zw}xWD?_wKeltYPYQE!;SdvzOmIOVrQrXWTl`E2oqkgf@gZNm(Z zHi4cX7MvG7+ashGvC`KDk45(vSCJ$u6ov+CS2vtbw9U<0k;-W9ybEl)Z6bIu>)u0&Ea-inu!}>TMk?_V@S1#AWG!}?Acju*E9*J}m zHJ!K_Q#mUq3-TsAJ9ld5p4Rxc_5ovzB($OwS%xn~o&ciTAiaG{kjn;@WLD12Bi^{Z&0akR5K6(a{LD{I^(v~{;__h$8WrM2$my2U2h^}!|c zlyOyObkz&O@s%^Zd*(EKBSI8qj!JQLw(HH0PaDp2m4kJxXa?WQ4PBd1 z?odqW8yz+n{DHp`o?K*Ye?j?~<~q?p`;s8nS3d)J!?G1Wscnkvi;bDm9)kA{?+c&3 z-=*Z%-7@4e2F|a>;Kf4dD3i*O)0zbphzRO#9ivU-LkE@=Tu{es z)Zq{)^_O=}$5Q3cXaU^tGn0v;F`(~+018=D;^-?hGKhPksIfzXI4=LwSPwET`8Xu~ z7_eCjpzMH8T;ZXGCYvnHLs#z(zMuuP1hV|oqJY=|4wTkY<7NO zkhDMa(Cm5aytkBJQ_2+BL*u#lu06C!X`zu`ifAxpeX7aQ;X07O70w})oKv}0seNU0XN2ypDFJD`o4j0hVt&RvG zbB6giMMf=dNPESaVNO4DbL_6zB-0F|giB$=i3NQFiU%vXf*N>@;5QbBlkwJF0`uxj z31A%5Z3f%*!>~d8Q7rBdts#$~ED(5D!dAVVM;bOhcpZs+JDG|gLHCY;oQ(adZLbq6 zJO8H-i$+Dq1&V~OTLGDzl;*f?+n))Db*7He4d2`ELrPd=?_7W-9s0}GJEcejHdCo( zL<7%Y>lOMPJCC6a&Sq32u5sTN8?RIe`^Aw~_TcR=0-r%WQ(FLv(Zo9lo@8J@bT{I} zZo=hK~2t2e8=KlK()VW93Pm^YTyE4tlUjAii%BNn&QuS_+fYuYVLx z-TaEVWtW5l(i})@^oTc`R;vu-o1q8;vl2qKfCcYabJT|ma#`UN<)~wpk zX_`scVu4k&Yi=hr!vAs3q-;GwP=8sr#`t-UU~o3vEt@YFO8VTp0Z;n<5xT`ukh{26 zGiswVJ*1qjiZTG-S~lR~!GsaABt%VJP0^wrn49@=6b6L$bD7B_9wemRvQBMvzZ;kvOpSo4bXQ?D)sj*bmDPrk}({FU&^h zoF-4aZ=%b;%1`%K{)&@`m}tz=o5q|A!e3=wa{ZP<9I37e{f7#3DcqMuj^t>(e*NE=Bz3r$nbW?5Yv!hRn03a?ZkdX=>a#kfP`X14_9v(5H1lbW%-N}{u@ zAJSKheAvbWAKJ}N2LO3%b$9lIshy9(^gV@6d zV~YccN9 z4Gl%p@Q6oP7vwb$?Fl10t?aZ-l!Tnhbh>+n;DGOSGG(r%EC&*OS)rv%btu5C&l=PY#p%c0zYceQ0Xb58 zy`BWu*$?vWIBwh0%s0H@JzX=Aw$N{`Z*X-C+@ju^Zy2xZ@6~1!4?G~8`V~$Pzd3*~ ze-qvI_|C+p*z@LwU{y#pK-w$2M`R%|yGC%qEw0uehHFeGJ|GkRzkP$U{gV8{EN}Dh z(z=eIpVcJu+W%pYv;j+8H9C9DaGf@i4T=$m{}Z%ozA@H5Y}qQ0DCH&?sX>Zj@me$N zfM-}h(!p>RF;o$RN#=F>r!w|Oz>^6=_7u}Z&MC7i+f_XpK<_Yc0Ih4^&0 ze`j9f#ncib_<&v5+>$<060bzZX3!Q!7c3e2{H?A!*ZAirHr96p zcB94-l%YL5+`J=j-F44$9j5zpabCE&1pA*-O4KVlUlSEOfSu0?SeU6P!nQu{yZm)L zK&*u+Y6sS29>A!TTcfjT(FoDPAx}%sQ3G)i_0Ty6BMlq_DhW3P)$%Cj;*5Sa|@`^c;JyhCgX@xxBPh9XD@$vYj@ z{TTL`%S6*SPXL+ygB{0sZh3FO5hgB#vk zKbPKhqr#)6)@Gi03QrbtlP~=Am+ydTPq zP#lWlf$>Yr96CrJ2Vo2uWKg% z@Gv!;aq3;i_mJ*5ysvlnt>bYF^6cmcw1=xoEX!K{y~eSY+K#ywQ@>zL;zMHpYG2vn zr@&)?#?!BQ+PH5SYX>e5UtrFT_xmsH@9j(GEmO+8j!_)}OlMW|W5Y%$!$^=Ym^KOp zsI$8`0{!ol{0xuKCeyJDPMY*Hx>6ZmKNgxyF#8KX(9EHGU-sWritWNzB;EhLfqFB2 z#vP5);;KyY->y+a(46&@?Icca%g^T(1Afulk+pZQzfC+KxXJ2ry_WizVkyRY}cfOaF- zYs{_TX5nyTglBd5M&wKA60TJA)<*XIok_=xTJ#PA4*I z{KR9@{3`7-*(^1cP)}XE9!Natv=taWZq;>8H_sqj)!@9QU$byBpLS@Ap_@c>q% zv#5HoX`0|_#zmY;rgbpQqdL>5I$ln$&oVkJS(T@3QyXTgTN&i3UOO2`C@Y~_9!m-M zfbUY;bbjeM{rnG7XVVVK^tvI`U7$#1D~pcg*-Ov{U_2*O78%fLNU_!bMsGMZ{5@5k ze6*mW>Y;76)6%*1Pa8jq3|9R(M?t--&VtDN$wC}o{W>G-l_!cr{;O#5VR`-&qteC$ zddFTu)8s?izso@EO~8Tw^zl+-^L-!rn6#QrS)UU%#0J#C^h|734yS{`za${(N(-Pf z;sg@v2dLA=xoUd^|slxi_-X&PQp~`*S>znsI;z6W+I&z^=;k`)o zE!82bTCU-oTi=gUi-Z^5UuBrz9R70&jZL-KOhU_f=e%4IkS4)8nW`{b=MA2Z8Zpm@ znlLx+eSXgWZwwHgFFP>2qton~givA$Ng$eX@86G2k~$u-?c}9Bhh^%eRjq(b?uN=L zmqKVV%xW=z5jwqjgQ5Mpy83TS($cB*T5nk;F{HeYFS|T#2805dztVIRYv(#w_8cx& zCzm6yZ1*Q6hj)c;`5e2-y#I4GB$ME28Ri-ju+0D72J2~K)3R>BRmk)P==a=6q!6c% ztJ44z3fwrc^}Y(|R+J&n+Ni5S0Rv++XF^vbAhso%8q6MBch|9 zDnWmhs5*gIZbBYu=2n_2`u(`5Nn8XE#W1_$18LnQC#}yj^JZ^KlxR4tt`m{qj))%{AbM$KiG$a2@)!(jxqsXuQ15Ua6 z(MF1_)*;E}>ki{l z9Z7gZYd^dpbViIi+W)CXUF(4glH5Q+WM7Czu+GxT2`F};-HV%5D*FBkVci|2MyO_0JHW4I1pTq+SN z-Dc-^7hiuJ+d*DpF~;E{L0>GpKv<{1d`)%$9Gj<5S8ea|1R8Yuy?lH~n;SYem?8Le z7-}ub#(}hkf8M1Ry9=@KG_eTgxdGB|HNlAap`8h`5=3}ff)%P*Eplf{PO2m zr^bJ@oATH)N`O*aHQC8iM*g6LNv1t*oANOZZduNuL)_kxuGsk1t?ReJ3C>j;_uRiB ztg)sTZk}Y{oLhe8)+M1yd1>#r`p_5X#oA>^_l|wuTQ!*csS>SP-hA_TV-e#$&3TNC z_a)|;^M6<0mp&rN={%c}elClG1N7CJhhHR9RPyTuUTTMv*XTmF-BBj1QR~`&`nZn$ zN`q1-2p~G-_~3#}Hizz^ZmLoK8{SQXAirRR1-J*x@MCm5PvicM2{_&aZsJ*`bse;` zFa;5|R!_7bo}XP5K7(deID~_kTZ)CXUIolWbVs%!qtfYf!Ft~KW(f)gs#1mBipZ^q zp4#>M_cn`QYYEj zq9MTP_4TZ>+5e_gdKbPHjF3drvZK_apQd#v*ty$QR&tvtDYMCHNS1f3w1C1U(5>&; z(NTU(8K?^z|3KQZ=12gN_;OS0eP(5ZzJm~S*qTrk-_V?SAz*O-aCqiDy{1|7vQcDn z7(nxw0ohOCx+A?qL+$P!7v1R_@Ta6&=m%)1-`Yu=I*%e45BBk zAnDymvzw4m?ISz1L?!dC?!dN*Xp9+%T$75s#M=4H1={cBh%*B|3dA z&OEI)LD(2R!%quFmV_n?c4p*7x=Fo*dG``+>IpxgtW!^WYlK(8K9W-?n+QX2ga;k6 zND_NKhDL3dW43 zK1E)SaveRCp=;w)=l|7g?(s~he;n`p)QA}+ZKs(fr=gNKZaD|xXi3D8wqh<5xy_|x zr^#loQARH7P*Rb5av!0YTT5-!?>OYLnV4H86I-@%rscOTzvtiQ_Ilp0*YkP4-_Q5; zyuofU?A#5Vql*nseR$%pF$oCecpDNQ(v9R68~Wa{KDyo4{Jg4y$9L_wLV;$~eBY#V zS9e6;gjkYQ@G=`0^xa zi;Pp$i{3;AL!ZgDU$PQa~Ke*Th90QAHC8ep6k|3`#>IT%OiI) zacHp1A^qet?;k!A#CD3#wAOv>Q8yxGafH;7=76(r5FgxtzmGg*XP!VB@6Dcv2 zJswca9+FxUSPP*#8+(kTcAb5}+t1mgAqWfMsayz#>cDGUbqYy<5pE&~nZ}=CKk2u< zhAb|DyI`PzjXL419|q`<71TtIPFZ^&nZo_Ea%$I#+=~kSr8)ZR`~mF^F5Qd@b@nr=)D4?q{M5wCqX;e3(N(HX{++{JRUy4eT!|0 zDejy=tRJ_f(JeC^xSqj9S6#aYJr6n&a)$}oPMEcJc_`+kgsg667i!YBn9olxs%s1s zVNqkGW~Rq{rN`LBtgDu&%$-kkH!a64GPR6wp@jFb%_G)$`!m#_meZ~1{f7uFhVa$$ z;alpBywGoKC()ZPLWj471_3RC|ktM6T3Y4E#DufOxB& zCHrgsC@Y>Vyu_$A_)RgA=Vp1T3wL*4UuZ zf78;%1_0sOgT#ZxbZH20{jK*qH8CbX@gKq{RzYKNl+yQ6DSd!-Z6Sv)?&9sn;h(Xm z#u_rCgsO3izKFtUQIP+eL!m`Y%=U4+PN@GqsU5r~Au_s$SU;=*Z-F=T;0b+N&=q+d zSNI7mXsXL2k6s~3(VeG__|B4(dJZrJ8#NA+3f^4~_bXC(*_&SymA;@Q$EvLBNA(=| zti4$)!jt%GXg_npx_6pc6f*j%P_Q=rqHnn3R==dI%Tn|zC#F{M(dz69WBTPokQBb| zHNISb(GM$Dyn+h3C8E#?de-guNcb*lXd&=LGTE$cri6%l;d?7E-S2r!Bg+>gcu|+c zhrQ9ySbTu~wWW6YQV7T};La&#jTs6ZpuytyGZf2L+VN1bP9h3SHzzD;uu%PHr0t*O zW2zKd>&&VjAB$Ikm#a|Sf%@@TD&5sIk9h;(iw>b;-XsAkC#D1%`;yL-808*{OYaVSy zJ_0dyYn(xrV4Wpr1e!{d2EP^?5Xa)tRpp>Sj)BP6Zc!wPy(z|j8vWN%eLY?Pm|J#~ zQJ9dL9E$d@7ito&@@OpXIFgL1ZVsfnI*JPWF1gCge~9D?1}Sxq$@8`o804M`9XACVh@;r^Wm04WkTX01Opb_o!(Y{JC)sGJ~Gbj zE}QxS4MZ}KSsmw6kk!Mb=FFI()aRKVUD2~WY_~3u)=}8whVodR^379nJj`Q4{DE;y z$(Pm1%W{>Cfwk#ZR<^fWs+)7N10VNhN7eZpZHu2o^sfwEC@JQgbi^a+N1bd6{#zo9 zGyZcc=Z=|kh>r1z3=^d+T(`F_Iee|Hw7B1N>W6pl# z%}X-}Cem0C=kscmJe5DH(C?9$Pz#2v{Z1s#YU@aCuC)<-_uu#ps~-@$F+Hfc>{_}Z)F zh#P;wmO_>vk_MBck&mvF^fLPP?)+8h=YETgFu=M}VQe0NJL9!;P7#eOHka zKwvIqc5o~p)m@MZo!@<;?MphM>?}DATtH4Kt1CYF|F(qw@u)n-EVFd@umb4o8*Sf6}9{`SaAdYfo$WN*<5|?WQ6u+Q5g5{r_6%~?bMw$&-!=P_ITdP zY|Oi?#k3>dcJy=!>6{CKP3>4YuiH6y_I6=RT&OEGKjL?|?bP;?CPf>8)o)=~-9&mC zp_{_R;XidWmz9Hz6ix4(vyqtki)MZSA!mp2jIeDkPGpsDVCLD@u1pIkJ3i&!o*y0oN5zEYajm`p`we3TLXHU)q?X}s@sHW@s0KYO* pdeP61os7fWtoT3P0(Iag$F!@X(DzAsur12w=IZ%dgL6>I{{V!aqz(W8 delta 42101 zcmX`SWk4KF7p;p0f+x5WLV~*vgNGo&-Q5Z9J`d?bGJzGA$0cB3gkXOC6jpYF zJ<<7)McOwfV6SF~fQ1#K|LcMVPxFggbXH$Ti{})%#U$K^kOnuY=z59h?N6(W5>Zi6 z2-|1(@9pfKcjIdfu9YYMI9y{~U0wDT(~|kkoEn4?4IZz;HpTONlz%Wl}mEkx`T_+gD^w zS^!DJ>yV9ayJEEsp3?FQ9G`5l*DjkgE1&m^Qbi;0AlcvBNi4y*lJe7|>qzmH(+Yg@ z&#Xs}^V3thRYX~P&j-@vulmb8lc=htFBAf4RD%5VK7)4)!_uPBd?Pj+-|G2dWok!< z>vDWT5b222i#f9p9rNcqLsZ-zsu9tNfAw@zHsY3Q<66-F_q;hizZZm@!`Dl6s!+Pr z(K!aoS<-Vx527mAd_z^#zgPd?s}z~*2Tc7r-&rLh@z$bRxGrbOFaD_MO;mJaUm}+X zQDiBZ6>ds#beqfWILeByz`;GKIfe;dcSz->>Qq?_0hca`skj7uFwE~`E^y*pLi`;o zg+)z<#Z-(D^B{)!+Xi8Pc386SdoiQ8ZWFF)+RQd_dP-~*79n_f-awh#c}$Q_F8jIG zSv?rU^Jw9$zVHBv z{l&BU=T^?WfEPywUgnMD7CuA|1yoPEulQ>_GtNvoMnC2%q>DQ#&ypmvvDS;@qZWf# z8irI?WNh}Qc4js$67ckm!L}QYWGV8CqB#~93VE9xod^j?-~m87&naNSnBQt|LXX|y zGq6aC-I)e)h}Fbqh3$~q!liPq^pW)YTkaR4X*j0{##T~RC!w2uAR9(RU)#+i?h@@l67Th*7)7O2 zp}?-G;cr67@-I8;sibpJp#4?z{WFcf+F(=P_5c--7OSR2E(lo~tLAqAIHh?oC{?)t zke0KF{RgY2#+FPHMslzu9w;#)2$n5ONQiXF-XFXOZQI+4^~;2bi>muEE3EnGc!SfV zuuMzGfQS3D?I!-%)9&-Ds?ChWpVJvjsl#BEs^g#V@j!+jr?=X_Znay#;%K*h5d~pZ zcqAvtgx4ifm7P3TIGHuNJE_+?j`-Q`qUx%-y!fKf!F`Fupm$dHKMPS+gavDYNFOfY z%nj-zcltu{CVG%960kHrU(g!Oop*geYLihyQI;H%3W6;0+E{HBJGN_n=ku-z^{0|J zbW)>2kq!#2dHRFZ`!;C5NwWO)9##>9;`;<)b{<-)=$PII-( zc!d^6)q1UUE=B@lWHGNuQOIwecuQ-GEJdWc{KKOy?8B25&c;!PEj4gk>DV&lGv_}( z%J5VGJizD)jqrWPO{38+nn9yRU9;ATVl0(2(PpiAh_19O2j}2`PPU?itvFUR6o05J z`CmTr%~FN;GXc9MZlsv&-N{mo+ohq!QcZrX%^ISfgPtyqjmqyOoay|Xyxeq-L+WI9 zvkDH|DQ(sB_GXUXs~^Dv6?-eHswq#OEeTGxL)zgYow;q-`CcujasuXs&&ZyhD8Qm2 zA&9EV=m$*0i0e#}j1s%-@N+rvtmCA%JUlY>c$nm#T*7xf%HEtl?`*}liIuyz3KI4XC-V+CY1*gJux-0F9$U8@ zma}t-!+lMOr>lzC8&59<&tm$$eU%BQ4pKso>U0Ljrl4>+JRg-ZLv~~3qTgTk1&4o6@ZUwDb1#3)jiO@1@4s)mUyldB z?2tb-T=u69ZoJ)P-5f7qQnu#*4zrYO$BjPA2|zXJj>NvW&jo&J2uWs9>{dnV5(d&kNs0W(d9RM>_1+jE{8orvq7p<8#+>6Czuh$hm6I(_VIflj>=%*@Na0C~mKmny>t34Njo ztSIoG&@0DR_iX@SSpCp|XJ%`pZ529E6NQ_jv?FWgg+N3XL>qRo)g4u4o4DaM>j?e> zsSLu6vfK9~8gYn;fOudmwM=)>?uO*q6LCeNb;BBlhFyJDkFx-|+=w~H5`h95Wib&e#{gL$z(ek$8WfzYQ#dH8i0S9VAc}M~R=(JoY!L%+c-%CQ5Vo)8}AJn_RG1 zcr_Kfc!mE=jw5l?t3=b*6R~l5-Mpfylm!fYE0a$1AO`Qrm$jBkDsgh}xYozLREK?1 z*J06v563;pBIJ?GXFQOZk6^sP87`fl!af`_4g{h=_N{*X832cWL62v$=ZUcn_8cD7cbLYoXAG-R*A)`7PZXx| zehnK?vc~^2nCfFS_W(JN_}5K-w6s~ykYI5=9~aHC3cFNfKgkF8Wmq`Z!h_R3LBIUw zv|%Fw6>}Z|vP2!R8lZkr~bAf_jTr${9n~2MP zJozb?VV3Y(dx=QGsFr72t4c=7#g^FmB*Q7#7NST9=LP(2SM(#cQGu_ga2*=NzVoZ< zFRrdQW825@5FfJ0lhNG(-RN+l>NddFq;O+OYQuQXUP zTd&mNPUpef?G0b%Rckcr`v6fQ44b+F`FKS8j!Q~i-~sxV9lv&{G!q`L_m9Os5t-P) zS*+BD+p+ty7pTTOl!8?)>l-hH+!bjt-MM^ZQ-9!(7<@AuPotfwCJZ(?nyk)s>NLeW z>N>2;X3(j}X@N-`F`R{kcdoMAf@4IQ;Sn9!JdS{*a#|)-$W?2fE;qz-(NNEo=Q_zQ zSlH!01)$#nrAJNBjVTY|Zc%$lztf%)~;D~N2o zW`h_DH-|I038w*%PKd*qr{6#MIB<)#T>w4Ltda$7+eiBlQ%uKFf2DF+`!`zk1d6xG z+lEe#<1waG@+}E86$iP`V#4oO9fu3GuySqr!vgh#{+#*)RzLmJQQ(S=wbm^>;ph1) zmtkegW9_-DoDc4*jVS7vFW0gZ4?A97H~L(INvZn5;K$$S+6Uvr1TmS-#2@S!Tuh9a zzomS_a}^dIoRx!WeJWNC;&L#?Z)@iG`io1-IHq?ofJC*08D5eWBo2cVHA7UXMP2gy zc<#fE89f=m9@A9xL?v0C52-SsuAXDGq6(ZJ(m=vD?)mnQ^BA*CPqB6PXVN9}op6)Q zPFROo1@WOt3K5FjWo_^Av}{MQ{U)=__#krkpX(mSjE_g9l_sU1wT7#*dyLQ9?)&?F zZs**lCZoXnLF6Es{dCuiliC^CX7{}n`<>l&&+}Mua!=*@4Up$!$9oGxL>(>Kyy2v$ zR)*IVvioy-gUP6tk@4HfW6hkZmgM^ze3iYC+>*y}x$|sO*jya7yx>p*1tQ@ujx0?6 zn<#D9HD`x0&V|AL$-Xt$ca^u-^8v_VVP-}`8a0N(Y1U!boF|_G2(r{R%ltfV%?Cg?XqNY zwG9qs9?~$&q)pUuNVfWS%5`;+8#yfU?k14|zI4|xy3II{E@wn~xr3Qpl(~JGRgyEX zzPHt^UvHa_5_o+w;HWDeG+fi2O}=pb zwW@<8)ZibbyP+|+UihwhPpko#kC%+lt-D?3XRH5N)eec(K-0lU`|}BX>3wGk83})B zYO~2o`%wu!WW|1C$az7paQyO)*Cz2@;GP8d*H(Cfc*Ns%L1aZhF%WZuq3&E?1FRHK z+uuO47eE&7U|wMbp#OtG$@F33Bb-MZA%m~k#ia7eMogcS1Q0$quB!rALUp}p6lp=^ zQOM)P;xYL1oo2-M11cK_QrN?yx&^RY1o-$AGJhJaD`)Z=0%1B-}8xM$iNkA>%(?v3b^YL-&o*b({D7Z%TMG1`fAA`n*PbtkmTp{ z4nb+XT(gPMbCBytV<3$*H17xWVwRu@GJZC22mfY6vWxXf;KgE7QWGRW@Yp-Z#kx*A)w4t^Bsy8Zx_Q`hqU}ELa8^8%^V$C%iNsE* z|6u*0P|)HDx*9#_^7Z82pB6eqND?e(bv;eQY&yJ?#FD3)6hUCINNZwJ@c{STXmNHs zLn3lkL;uoq#PA)&3fZr_iOjb|0V>qE~}cp`{}X*=bVU~ zFh%&gO$jJwhKCdvl z&9aO^FJ!;5i?`=ofFCbMfcnfTjnxAUpP|82Ihi^x{{Q3??=lcyGZi2l>l zv2B)G*ks!3KU0%~ELp%7f{g8YYq9Q9vOaI}Uk3vvr9Gc#@<9%pZIaGX8YFmL&lBx* z_r2tho?|sV@BG^_U=|r`^O%#KHRqTaG1yq&y~>ZxFIpRMrqb@AjlEKcd}kn1ZQuIV z4jLtC$qF{=yfw_CUNrFDbi6M6l5N``#oQHSM-h#ob!ipLJ8vy7q5$u1Xx;JXA6R&H zoo1Jz6Quq3OL1~D7TmMt&-xPr&m?_o6lZRfbp996w}6Nup1E(1RVB2Vd_tgj4f6{2(l zWST%iPjhPNq^z(Wq|j~#oq~Rw>Aaz%B6H=#s#Ryjy~0e;bkJ%TJoz5Kc(`H;z$WYF zp{fS!v5KAVaaSot5P=d?`qytFFewr#i3y>PLh4YrG%>`R)un9D^<FwiNI+Vb&2p`6q;4M0=CTkC{YoKyXU5hGfI8@Psl!B6Nr`yAFsRcvfscxo(4<8<@g#+52fyX}N?bCB*+-b#9l^{lyW zkZv2&v5wXQy(;t>lldR}Y?yosk@U#=%O(>Wf~SgKA) zZljsf1^q5bxNXu3j8YV1e^CqYX8ais_z-&-%G^F!Yr7$AU+&k9L1{vFRN1@Vfg(e` z5s5#t5fs`JDRpiC;znYnof6sO#>%2V^4<=N?WPt6uQfZ*aVIave~#DdyHa~F_?1<)tT{DE ztjac3UCe(14Me(aLD$4<>a66cA_SWxyBmO{5xW=UfsqvNCv)uusm2~mpJTd?=e2VV z&@m&p;ef((^sAeCH#nk+;n3HR6OpF$}P$qS8%$Ygv6=*&HSB*J7t*B(7{9u zn76_i%Vdr5b8y>CGP}o>9SKO}Zt@;w^}u=H{eXRh>DC_3^mhU7ITFx=L5O#kdq-mN z;?Hg&qE)mW8QCN@(Ur4`i(VsAR&cy*x^c^C78qfAdh_o#78c0QB?cxs4YKomX<2qKN>XgLe{hJae~pv^1@CQ7@U}&PRy4JwiUtcy;%+Z%5sPwNK~DcS#C{nrT#gVoqls=?e$Gk3rZGy-~!5HC%~GQRf&qGN{v5pqQjr*ea~ zFA6EcPJ+wp=RI!*o!Na%Y=UK2UyQPRAoPumBUk&hf+LaGAe^$lA6kg~j6MbuXsd+2 zT%uE7b1=eOF=>>0{tdq32gjfhZ=SMwh2y(EPG!Y@a_)WN&>73@5OOBW@;~h5Ca|$D zG&0s8c&H~j1) z?utg$9u;%;sliRT`AHm0l7O;xqglyJwoX?kc=F??i(NwbKx=WELSmOi)xVYyb|7Ei z6gjb&0JOO3x@j(3wpQuu!T8ozv(ujip|r0>?RT#Y;0+*j*Z?0QXE&|O{Yb!*HIakr zy>uuW*x$p5WLg5jgrZ`SeWBgr=keB7IoLFMc+U8H0nsG^gNgBYnQ1?k4f5RHM0fn^ z`3c0<77ta3qanT8=!+vYc>v`)za=zhJO2sfrI$40HstGW2|j6|xRq4S&5V`tTUcdS z1AQT1#E3TAyR%!0e!nrRe%9AKjEJC=$Eaz5Z1Ra1CgvuXk=o@Zs|02V2Z(|Q!M!=M z=+sG}_X{B+AREQA$l&Q&T2vIU-$po``WHk3ez{ChJ3yK#<@utH{S$M!O>YJb4>k;G zt2!Hw7j>hvV%+S*F*SvN>-cTgpam-_Sb7rcLcGrfqQJs@lq(u5(m}xS!wN>hE+(`a zIL1Ob_!kcBM!;tTfRFf-tA-dsM^Ln`UsZH}XLP#jhbcIT@dF7!;Aq$rh@rACp*~0c zh}fMZ&9_5vl;BJ_C`Ou^5w7M|=RojzWRw32Kx8stg?5B^Y}EWbCFDlRWH9ANz zBYn`TJs@N+#9py_RJQe{x}Zq^#B1ua5gkyAY|I`kNMe6?x*|tPGH+ge?V3eNUaRHu zm7sfvTX^Z}I9u{N9Z2XKab*9GS{AGT7H4G{Gs$2)vF)v#Vnmw-b5QoP;$_aolqUo~ z_2ApDK{`n!(M`u7ZQZI?!cWZq^#mMRsxl4$bMig>We>(}RdT{k{q z0uZo9oP(hU3t_M~Oa4NkEji)+7@%ua&?ED8&Q2r#LfB@dpgcn~-OCNi!o!g6dT9-2TAs(QT6&i)mwyLWmVEa zPlQKViHtipNq)2aAGCqLuXYJ}R=8|9fJR^s8j|mT@!y?p>lO@XE64-nLDmNomKa>k zgRJ7@xnA=+t~%*Vq<@R^ZQQ`dhLbhl7YyWkwb={|$yO)TgXpH8zI<-q9>Xl77L@-+ z?CzgSLf#pPgqSkZR=_>dZyGV#rIDf|1LqhhjhXVt_ooG6Mt5?f*tfyF(X{>^&16sT zDZz9#y;4Q{6P#CQ;ks;3-Lw@* zyek~m?50}GkAdcx{xh81&4IK(!xV@@CJk}tj4&mL%Buye8QThNg{H{yo^p&a-%&sR z;SEbOyrQDIuu`4K;O8`3&~cqA=1QQKq(|iBIAe$;^yvQFp|BI!9O$@Yl2`(Ebfx`O>(D$e$Fn9M%SVCl{uyFAIf-ve|7M`8Ye38>e_OcPFFhrdld<@ z4YFKUt11OWY--tAOI$-H+}vjkB4U7kmec97%1!~2Z26M1D~v8=n^!A-1fL2neXExA z19DJh6U<52#VFZE@lYa$eo!$OZNrM)>$F?ul8_^v)sKfms-aZYCvbS@`qQqzh1H6- zpyeOOz_T%dS!~Ra2AgnZ{;aK34CyAXzt?JdpnQLi`?Gwt`<>6CzK%sx9Pr%2C+)$O zOvPV6ef=WPP9FIO(#D81K&{FoprXRW^!WTBKq`1Pl8kc?ErzfnGSx7^;~~N4gp5oz zU$XFCx}WWlN*FJ~Pkqisu}1^gOrW{_Q`Xk?;~n15wM%{tA@L)Enm5n80cT~g~q zcPP=q0IoY)eg7ZEx2FU|O?0Lsqo7!z%!>ZS=)Vbb%(PTR0upj@5$p>kX;uq$_u%BN zDTmyizR#2S+!aOVLLG>M`64m8v*F+^1e3~n&?IejBXu;UwcjV(QQC$-wDp)EwzrP}t zIqX<`DG!<6_RLWR=ZKoaJl&p?YX@aMt$$>YkVu_AagnW_ zt!qgn!<#CI81@tGpBtaOG@##8R`GmhV1N{>}aejc$*pcCh^pNG|26I+Ok|DSQIwW_tbBzYS(x_&)spl z9^|Tdxx$=sOHT-7bE47c-=d9Z|rSh9pykK2u9E|VJTF!?9vfQUc@eNkq0Bp{DhT6-jFC4M@D{;Zv77_yVi$E}QG0R?L=r7`*7>6Bpry|E zWHlSD{kW_AMBM^?s1zHw>NF$Mob7wA{z8`e?O~b0<6} z(pS(|ZdLmMmcr`H?x_H-@|9tl*7k7c!;n8ZF;Pn&3c;1fxkIzTr1mW78qb)Y=o3Q_ z-;g*u6Kp1gE;wOd$w)HetUT&)xNb%QJxIHr~B{c$o65@6>w->><{nXOdEm*QSH^RN+ z?Y0s^2{)J^T?2~H%?{NquY);r+| zN$UDfkejh@SAm@MQGBiw2(0tclT02s{x>1@G0r;E`%c0AkyUHuFVL{}qdW3U0^TCk zgRiwqAOKxrXAA?~ZzfZCPgiE0=fEndYy9`J_1l)G03qOeg|K+aQhz&)r7qfTDQwM7 zK8&x(3>_t8mle^bZr?@{5d^5`T;9JtxX^Zns5EQc?4{byV-QTNgRS`uEv+$wNF0K@ zuXXahBB}f=CjdLaw;aREA2AYSZDM7(70f)YhNzL(({jOl9^GS#e5iv}#Oovft#P9g zknY{5`DvXD7 zsMGVVwyEcnEyt7$gBySJMdC;SsTx^#};6W{s z9$V|{9j`favRU0=tgT;Rs~hTHv}GsxyR-x%@KKA;u!t^4h}$hegs}AI)?9@`UW;8Z z!v^zXuJYNfI`b*TyC2ELyt(`cFOLK8*SnD6+>+t9s_`zRk)Uf2XwPoO!l;#5p)ktb z)EXs?rBVxQQ=Jd041I8P0{PaZk!yMtmTP zXI%A4VuWY$Fhs5GSLJ$}v+_Wt0@?tD<2)rPh8fC1S)-@h;3h@Uvkb+1!Nc2%*fP_B zxkRYaJ^UrEC(3BEQI@WF#Hz4|;aXzzSB8%IPD*5w!wjEunwq!6L1j4_%9XnxX@n3C z>Iq`71L=${s{c+Rd86zqJaK`#oRw{pbDA|%yt*%94u25Bq~?euy^`jO*9RG?1`;oU zms(i!YqZ6KzRvSxpA@N?uo@7`M+C9oA1&Mn3hlR*amiJQn1ML5b1W?3c62oQw_Yhz zm|?Kuzrx1TctkdCx|~a09=yV3`jS3>`Bb<7K9IMX^>O``-P-iQ6vP#t8(Fq;Z$iCB zNKj{=iyY2lQjo^6XS%2nC_gkvDTupbMmX5#FfB0!!`K^02>Kf2L?{vYsI;jq@dud0 zAeMUdTLXtkuz<4VzJh_Al;@K0<*KP?CZ!}Iv?LgjSKr!%)3jA>4OUp(dev!bE49@L z{KSAcA>yE!;yf<#nuQH|u=jLDJy>&x{5mwx7Xs_NrWJ*Uc2Yv5X@F@qffJ zeb`oN$$7Dz>f`J~wp0Ia5vJUN-IhasGWHy@30ph4o@biyR3|#FtQ@{l(1b-kO zB$Cf)^y`7Oll9jlE*?BdY#h0uoUA4hWD`D!D`Eo84+sEawsw|E@=j2?G#g)`_-A!p z!;{NLun~+l9%2)^zo89~ncJvWb&`6xsPpC-Bp0x>Xg2Fy=OaDFrBeC!`v2(##hu|>a!iY#?m>L0Tf@G5^MCcuj~0FF z)$-fp*_qL8 zTu_1>5CNGIWb<8jsxzb#HpO5Z{PS=nOg`=8qpW8N@_Z8_8HzydOQBhkulo!grtN31 zv7{=GlFeslRfKW|H<&b}!ew{gNJn#7xP6*`x7jZ+M!1FY?#3et*|X}O5kg^p?V`M- zIZQhmhZ06=qVtK6nM@8B9|yOpGN+u-jnRvAmZ4>Q+rEZ)8?LA#z_G@B0uRT1_c1)w zfDFoGL@*~uo)35@N%CpH-~O_7H`#(gNFUb^>F`L`gp01sA>TtZ_`EsK zvVrng_8&kt*M~x|Bc0@o==XP+tzf>@_aFbPL=kJZsAOsa82RD*A3Rzwepo53s@jL! zpXj2Y3g!q2LPhH(bmjWr!J7lV{ZwK&hdp^kejn#f%H;%8HNC7v2bkOQ4!>km%PMKg zBMmGG3r?{)R*)$sk>$m&k!_m?_I{=fw4)#rXlr!C0%^W|-^7OxX=*E>U@E6mev7Mp zc%*GfGek-FCZhRLn>1Ryu~2Hd2{Z1WQ^mW9vEvC&nBn*Ds}4E@tH_=7Ctci~g@Ma<-H=bfQ#IVsb|FtWa9Wj9IvYiyce2i^K8jhX63*iCn#Ib z+r3@=`HuilT9tykB)E&Z{CC%Va3175v9e%0qp_L+B|f|2lD3xFNSREKx&F`GpYxgU zD>GeIwg--!%O4N+eBML>7v77FZp9DPSG(h4&7ZYz{)T>C_O6B_@T&dPJCV5GL)GOE zl`03EF8#}L9-eV>O)xBiK9d8%t=}#P^rb?Eu}404B9f2EX@QYKY=7(;!4)8DEVUwh z5u_hxg32aSiu0pw)$Hyl_N;Dx|DKOJnJ?yvOe|i)I&_8vq2mduZBH5s$*-*pVm6~e zL!syJkevNhLQwQ3#sWMOD!DQrmgLM?xNbxgD>rnSub_55axQXd43Iz;93NE5*>-fX z7o^wRV_^n|^==OA_I*~=~YGXV(Jw$ z12~icg5?;;L>+d>^q&Ga6A!6 z^4szDDl4q*c+K*o%E$kI6go&x)LBa2b~t$Qk?HtG5}y*?=5V!yoKp6!vG!6bT)w?8s!$ZPPixgO7`8KD3IQ^hUE6(#&#B9Q>hOx$yEUJn$;Mz-NlQ-HuP+B(0 z=S=b9R#KOEGy=A*d9s+J_&ddzsA|2b=9(-;TRPFgHLMw8g{VdB@aVzP?!wqBh@8lQ zvPGQfP#N%$DAiDZ88aNM8f0;4sfiC-ni)<(%yfwJ!)y(^pLXQ-h0pKRknEWJg)&Yp zule#V1EeePhNwi7Y1cRWnuY-UPIW`hp{cuZIp-#F-FH-#gg`BZKWSfR2GFtOq50JcnG`jYyMB3XI9gCY$#%m9b_Nu%wgJu z`{9gE3xEfHJ(vWGItXa8i2mK}r68$_K`5@%lp7ix-n7j6^s9#=jhxvZKhgFf|Kk3T zpgmJnl8+I@+8V;BCD7<~*w1TCNcT`M)ZCJwgd`9_&Rmc4-w|&kh#si}Du9`B=&s%5 z-0YlrXW%j4wDwOKXgY6+A8-dhR9|@7J(W~|9Rk8xidQSLi1c$yYkiu_TO~TaC`Uz)0ZY{cCp!q1JZ1DS{5hx+WQI4U? z^&&~CY;A9`BBT<+84JhrD%a5@LlylKLQ=ll2-lw<|J8-!r3lTwHrBlC;9=^jKc15b z%ovjBr1!BS3gd6)&)iP}CZ!V(3Tu5sM2UEPQq7eXFSu)65KyQ$1Lh~UL%rV>r6{7%$(O?Qf==PQM3RJux6+5%F-i}qP zMGJV?m(=2ZXdeZw8q+HW0!VrwbvQ2o&HT-0FGgKD5HzGd^bb`0Z=y7g(5>^G2m9FK zPvOuL8N~vd`|nLm_~?* zC_3J60s1>v=o=5q-Xvw2rWXmO_mFL3(rsc!snC`u76CayvLG{y#|y@exYCK4NPMXt zJFjGvZI8}CTfmg36gX0sA1lS={(n!vH$2n94U&d-DjPA$!}oir(=ze<{&|v94bM!l zKjf8eCHg1ASlwW`HCG}VC+t|lA^T%}RIX9|fnHUNGu3$YyP)Fo8bx*Je=z9AOqebao1(+rSuhDmpKhYS#7>dstMKz1ZU| zW*n#lonNN5kZfP?HgUUKBDd)SsbUybfAoW2C(;ew}U#Y-_Sv1U7KF39!nH1c|vM09or2GOGzGiHnWjMcy~T0Iim11@oKnX&~=&O$nu zf+Fs01v}?}%GHkA+rx|=oG1A6!0^SuwYQ;u$q+JTa~zFn6|5SK@4cg&+%~HcP@aZH zsQ@e;oiCLrIS@gb@BMst^f7@+gASc4Z%maP=p$uuIZDMSi+{ie%W14VvbG2B*H^Y-v#`b{ibT^uU`S@ zjJrK%AiqHLIhtvfgb+D*uVwoOj`oUlcp~sSPi(f=C3cWp$VjzU=YE0v^Skxg)s>wT-c> zo7?sp7Yu1#=A6CCfshs|A>AdEe_*}>R@vroeG=+74h1NXkex&ugPIZg2 zc|UQa54gna=Tt!#a|MHDMO1{EK+Xp`r8de6cBICub(5x_W7LhA`!L$+U<&v+(hQ`X zT^Pf`OlkNh*!GqI^rTpDa4sQ&A&RePn2M!3+%|d|Q#@)oALvdl$ zsXv;+%Ch}r!pR~<{l#F7*+B$GX7sVzfhgFJXP1PGA_hW15lrXZv*186v0z5P{WXx_ z%cY@P_9)^AJ&eOa&___9$j4i(&HW42_A?%hd?o7a*wWI(pvgCTQgB39+YrLIOaL-M zncA5%6kxP#%G5Vyb zleS4b(AiF9!?;`b6TfQ{1!7_gn?OSr?lH^fV`UK0@6&bnIs1agV5bmQX%$Mi?BzpxX-Y&+elZcn=daQLnP|ff7~)u(bR>0H2Y{!6`7J1$eCca-s+wr;5l!_c&&Z(p^X|b0X($*orwA@jprY3rCcQV7No`sAufF0woRn7;19%{m3@grN(2YRI=T)WcI5)@U zd1p0P-Rz`Qml~me;PnP&6o>y(r_q@G2(+wd5+h|a_zw)y5#c+T?AhIk6cp0{4j<5q zk38<{Whqt33BK0W{+^Qrx4&FTHC;#WvN|lQyghHc4`+AqDvvxs4LE~?(JLeWCiv)e zTAY>49QQQLhAEK9zuI=Z);pc8YCPSZ7E+2v%x55-1rRusdK_e54bxZBcy)kYmF0H& z!(-d-)=_8v&N%0}bqXU1brg<3F&;)}l2OqtTMlJ@6iiQfIcoviLXIpe63QFaHT%7< z%eC$M%ylgP^ApvQ)rBaSIi;cEGsx>0(>G$3{^yJgK@6D@3Qb`2)8Oh9_Y$$kfot^w zQR@N!#*!lC@Lr_Gzb+e3if_?h$A9f-Wk;uJ1fx$fzHsqwNho7)6NB~UQ zRq_GZj;VeM;3=pTf%<*}+(F{8m|b&7l{zd)?kSz&ea6#t_H+#S2B?8jYuZou=asAT znx&a1ZTA9KP&dn9teilx1^j}Zp38#9Av`d?p?z-;o@Sk%5iZ#>wA)+fL6%Y{W zxIB9O!4xR62xV_gdV|Y%(E$t3XDxD2c94aT&#mgDdVuU|6CQK$Z`P3Aw*p43 z`rou^!?kvBB(&oBP@XN93hG)?`$2to(Sv#Xv@6Sd>K}bJ&SpAFpwk2OG|6)6e#-=b zpNi3-XsKpTc0~wAH!KnfjK)Aog?I!>ag|d_q`*_rs`G+Q%|ekv4J1Kp2!r?ROQhgi zqilXgu)|NEwl%Yj`6rRP|MaRMUpN`?w?7~wU#U6yJv z_j&<2xj0Utv?Bfp;os4@L{_-K);Gh^q;2%58y_tt%C#Eio!(mCmXD8C0bhCFeNpqU z_Z%9QOo`LC`3PkV@d=)wo(`#o3LO)+f~T`i8jDcI`l$U(uz-_Y!`mYsA(vMYA5x(N zl6cS{ZJ()3DqW__>GE(Wb1Mz8b!)ZMz*#LcGfRrgud$R6@tL!T*|~!0H|A82D`aiQ z1T2S>7W5XTbZ7pIn`L?<4mjtexg1$&BA%f@QV2*_@wBXHp(`osYNvuJ5LG_Iahyk- z^kX5qZSx~l6W(|LrIsw`w7tXb;PGnH^{zVL>2_fQhO(S~Hu?GJ;OcC%8#Or)!r&8p$l zr3Ol^AK{%s|M6lYeW3VH68rFF%Qz_ail3zFtveXoB6wJ|)BQP!Jo{=9krV9r9W~FV6Fblqi9S*l&6$-8eTlD0YjO=H4 zt6tI^H+Rvv^#13cfVKJA0_|a#d~_PHnEF6K$iCPgmv2|A9h%A+ zrQQS;Gyc;&@-2l(j(W>SbEk zj{0lCQx}b}l<>e2_#Rp6Q79-_WPg~SI4DMJ{q26kRS;SaB^mMyBi@jkAAot(Dw0_l zvf)Kp9(M}vCj;aOfNsuLjaq^1;%KWE2A7Bj7L(bmcI7d-gX?Vzv(DX9YTCt0$(Dxv zY}lI)KZj&^h_WKY?2pt2u>sw|C4u=IX+DZ`6~Px4Egl8V=Oc? z^|QLzx4i;<7q?3*9Xc^74)8KZ`^tG$TY)|4$$$Qdv!`74twE??;=N_>v~g$r2yF#n zCY~hQjwtd0n$)Azpjg9JBViv}y$&bkC*2uehYNKWKOphG*3DbInr=KTfVe*z8-Agj z01=JgODvV#^cZb!mG15423Z_5WNOrRe>@x@AD|BlhjX7HpkxGBXtCNY&ta0#vPL!V zu*U`+Qh6pJIyr_o*+6}JVk|n9yEK=pjuTxQO>9<+@`r!_S+#lG!lA1XnPvVa17AJ1eagZr{=X?ha_WzntCX-5>hVAHmS+w-Xm4!6 zbm011zL%W@xR8>V%PFaXGu`8;xCR6WfTmxQ!5$f3KtyQTX_-0|M^zJIYehXse3 zXJ*aw-Fts_&4NXjo2m?H`GS*9?2+x5@!;RqwQTyM^4b`m>q6bBttB+O>F@?!#^-l- z_hUs(CeEbohU?*P=uze_8*9%)&qe#SLC<{SvcJjbejg>!9dN~Go}>;y20%0NuOm;i zf_%2<0TqWo*t>3jqw#kOvU=3s4UI}3L*vcX+Ylp3MT?fysMp?2pd!;X;Ra*f)C{pr zU5inC;@?d*O+<*rNL{suOk1%Tz>!lCVLR9udzucD$|cx(v9;gjO+x-Vu2r9F-ctXa zSjn06ai{qIW|Q_V^anfYDssUuzjtPDG9#5YPQdcHCLHzTNUgP=8*RfH7FrW=Eb0oy22xhs*FJCXhN`yh=S3E1 zRvHj-uuRaPC0d7)yH7tdRkq@V*2gL>c)kB68EP@vq)Tr-S1RnuZ@{ZV0>&+l>z7V0 z4s9z3X<~)GPaI)27iI^!KBzG`+fQDD_=x1YU6klq>QJ&g8qiE6%)4wnzzv~m-?Z!O z4woBQE0A0Uz$@LlIa&!BgDA`tDHK-HMm3goQ4T7eIAphsXk85%>{cz~TL15W{iMeH z_#~b9`MYTN_e0;rtW8sc;ry*tv+on1KDTKE=i`$&lNiPbdAb_SmZwdk9%*I@dc+aT zdm4W|qhiM7B1av|cao-#W}jc(YxLbl(I$lF*at?P8*;?2T0eU>9+e8l=u*I`9g zWkXMtdwtA(+J0%cI94vB&_e6WMuq}lMR-KzxNa=Zx8UOF^HmKKQB!fR)rWw)pL>?O zoWLj4${IAleH)o<$udNU-f-GszC|7dzh^xVUO${#mk}1 ze?1rFm5${pXZneBd{n%RiR8Axj~xBFG@(o;ChZu!-|}11u;0et8eL_TB?qMx`oz=L zec#4*a%KjEtkt*MrqAuCeYJMQ9|}Ycph@GkU+Pzurl1p^+b==Op)Q)BYMpo=dG`7p z%b&k(XP%3;OyhP-yfeA19C>7m0s}l=^caFgh zvY`)qRtcn0u5~&UwZk;F)Wn^SHRnT87}~l)4@OTeNCD)WE|1BEE1I@U(||6N&_1f< zrP-Y}(Hhm&$dKB&WxX5=$UbOJj(*V0gc(%-t!;BI$<5J2%#acHKdgAL$uB0q>_`rh zs<1j__81hyqafz{E?H?HO=u8OYbJooATWpfZC!RAM}J`DXOWdP++*HGzQ6T({*}!^ z)?&m1XNm?-AI28Av#iEiRMhDrTC$oU=oMGU;?yqwB0y)C@J?GDh^$gR?mHJP zW!2f_QSup~B-dBtm8K&U%`)_{!D?qyN%ABxToMt9?jfJoV_fG9TDz(Jp&q6t+uvzn zs_q{lhjMoL!)d^aGs#aS3sngt3WhYdSQWg!0bsK_Izr*1tfO)785`bWcH8>f7w0PW zP43V4^HbD2mVO>`zLJ0kE=1qR`F6Y>TRo?PDXmo~ypFPHrqRnL5W0 z#p=!s_!U@>s~f5(=P3QX5xN;mb*SabMD5M`+w9MHJUI-}vROkSS}j8FGf5SNn@+Ks zzH4cTmYZDfwcyG(J9|`#Czo$+?L;;+ua6TR2qMhccz?14<8Q*l09Ts|DR0W)pr6|r zU}B(?{KRT_UM##>*2}x-g-@Iv9VZ-`7 zTG3w5R&KV=6{MG#9A(7>sA?_4$!Bu>d>Nk1Jt*)x>k}u$eI}t<`~V_(cL%<tf!8IDNA7zX@hrE5E;r$S(*CpuX*UP%e1 zse$Sh`h@K-+vnhIUEGVQLv|KF-$~rT%~7!yOCEk?%UA~5Xae4E*=j_cocxQd27_Ot zH>45d7*&4|!alU)%hpBa;uQy~SoLlYcFh@0ZYC&lT)cjaUB{95;9E0XT?=qnUChc9xlmv7&Su0Pa4 zFL!afo)_JD5@~Ev6^t;J@=w~q=-H%)(e1q59CMyQXrCwtn4a$snOIlMqK8b=lc*{6 z+FkQ4@7yWd79Co&Ju?@hWaG8@->Pr+>c;R9IVN_aJB>71lcd=)R)oA&G&}T<5d3J* zymy9rct&RC4zV*dW!A`9f9OXtpz}zgX8$Wru`H1D zoGbee(`UtY868S*I4D8*HhO;`P5zhgV6?kSWt7Xnx*zXOhI;p=8E2OWM4l3v=e+??G^atDp4{w1+o2;UgU|ctFYwyt ziQrUY=vDjCm98P_b54Er)ywK&GPOFk(;ui3sI*EBj|-z5|Mf1L+O)5r{i*e{u6+2M zByWQaGnK)Oi%f(U;zxt_@0v-pB89wXQ-g*Hchhpz5<#Tv%Ub+0hURSF1d{A~)Sl-U+f8Z|)s^Lv6kOl>P(iB>~ z&VPUQ3|${waRw-37w8*_?|?rOSCzqMs7FKVeQBDX@50S|vizWr#CUOW|MbI8thj5A zkG2f|t}Vh&KK_>8WY|bAQJ8L^uj!9J=x!P6yanBnKZ)B9d}c?fbh1D>ksel&OK|?RsVgi zx11t9ph9{+{{O!%1hr!o9iPby>pQ3HFDlF6yR!U#S>Vy|O)Y2yza%(5wmLhOzF8~o z@f7hA>MQpCdA%@*Oh%V}lQldQ^p;%o^MX9=^Q5YZ0f&&pBfAQQ20@WR-4C%aN-3(& za;uNU^dpfUxsG7I?-N-Xm=P)a^v)*b@is%_Can?^j&Mi;@(BhbI3)=U0?}=aTU_7u zT9coFQ*{O3%{Hd`&_tRLwF+RC#8d04bt-4J^2JV!^6^{@Y%v<#EZx{N%3<68aZX6& zQTq|0)^?)kxbw5YzEw3>*`jEf(6acwb+5pHt&UOgEnjLQLKI{^d_zgVM?L>vc3dmx zL?VtG(IyZYS1MG>L}#D*wwtXM!#pq8=HE}GzLmnOi6{-sJCk&9qYRIlt;tZ3KsVyt zt&@;faGN27WOVMUQ!704ja`zuaVv{a#dL1T{|c#;&A|<`cC8WIO%>01MX4M(ab*I) zJ@kCv>p-l5q$v>yx1Hr+0khr@HW zs5ej%_?TwY9r*ybcL8J*&OaOs+8!RK5^1D7Nju^`@oul0PpNi4awbX^LC%LJj7gri zLA)JV<|n7geleZ5kDcy85SvtWwWGRC7=IiqQIDIN9Rw%#g)a(HNqmQ zHc&-Iiu%DrAmD;DQ|Cp4(6xh;Cn(TiI<g4a6_yl4 z`AT(SmOpjd%~gxQ&t=q%*FgS!Lv9bB2Ed?cKr$qsE7#hok9t=qqZ^X+)^JHV4#J91 zUjvfARG4-`dq&KVB~WSj6Bt*Ldb^`m|6!X%pZN`T#|){;$|q&P&?g%yMHuUBR^(FWej3xH347NoWIUipIhbny*5zwcgaZ7^1RMH6 z&&YTOAC^o=-81U#SNibob!fT#dK*b9cf-^ai6)7;EOLe~3vK5r)T3-T><(zHe*pLQ zmfGKIJS4SCiWkv#apuGCK%s8ivL80y&66jfIkCdKl$LZ9udj9Q&7cD5ox46cl-0EiR=&E(-Ng?)3whmbNN*2wjV3aPenCZC1Sh#W z$F*Z|#$6$Y`w{QP@5c9+z1Mu5H^b>es-nnTt~i{R&->(Go{y{_1Sw8iS(Y4Heq9|c zeKGLcF}Gf87v9fgxqaQBKBU2ko$%9)T(B8~BzntbOt`og$dBe#Y5q4FCi6LifeNJS z!FX;U#A+5EN#qc}Uob}W+7gK7(DA*h?lU_kG~J0er(*yo(Fco#k<2|-o)_B;|H~@- z7hQJ4o4&oCGh`DyZLP&1kAUb8Cn%az0(`Nr@n?`YmyZ#8EpobUdSa&Fv4vjU)61r? z>YKwl5jlZXpF~s5y4YWED;9#{y`qePvsrHll<7_JLmI1hu&2UtXJT4H5paJoSpDC2 z9*=uH4QpVO*kVr(PbqS2^Kuo1EkeK5vs%yhn#Xw~n2_<}pEywVt3JK;23Rb!VYlWs z{B}UMU{m(1ZcoVLL{DIE*#(@7J1F0QTnlxG#{sh|9?-rsb}{OG+Aj$JGGILF7gD5{ z{bIec*56Fzsq{c3?0CgY$biuJe`8Z$hW-f!9?MS1(6zRUr8NpF`IHuAs!X_O9CFok zquvCmyJk@X`j$>DhZr-itinnf}YsCBB5=(m1-t zs8*KKD-4+g*#yL7nw7j!T;R2uR&pEFMnF%LbqZwAcmF8Ewg-dvL9!k?8=DE5N}E{M zSHRSE-%0-hz|Z71f+Wi??Q-9oNItY+`OCoL3fb8hUuSyHJaNNv&T0`W|2#KC8hSsI zg0ht`6PUcqggQMO?2C(jNYQgHA-IW`XN@aSEiUEoA{h-EwFij+$#Wb!M7}*%xyrt0 zFyvsG_J5R^ljkGDw__MB`qsGVyf8r&+!&F=4?xq)^M5DUo) zac`MjVwx)KfmJl^H(R1O!h9L}P(ZC$Wfb7^bUim&;VYOHc%Z6S$r-I0aa7S#L83*X zN}>aKX6ubf0{sKnj-s7m5XnO*{4A64D}lFnptiKXHc2PHQb&14j3Nyol!ncQJZ^y}u3WygaRr?-o(=cB1QFpXE3R?ED8}I*zQ;rn+sesMmCCe--9$ zf&z}rP`+x?k4o#QqKM0$z3ehcV{L7UKXxvA!|Gn`_q(KNd|$J6r>aN9Gas*}Xx&io zV@X{s@-S4{b-2&(cCqVItW}Fke42WI^@M;;V)iA$XD7J~XQ2?~Gm9?11mBwl{;0!fnS)oqkwbxA!^u=zaP9i}_!AK^c2jT*}&oLIGkG6=_ zuUP2vI4>VhROkzW(@FT9lH#oC2^hHE>Flc%ZyNqQecr$p*v27_iHp5{_6Fq+#R%1S z?ivhkZj7Mt5DOxfiSfyO=V)LDq#GqS7;l4$uIxR^s8QSV)6S~*NgOGAp*vL(_L$HX zh8vN9pm6@j%heX#h3$a^bCAU1?_|IT3hTMce*gXbr|)i$@U8uL^9Aw=Omhim<~@Sv zYzIV4%$2Cc=o$Lmo0AKZx$he{`+bss#<;3~>W`~AfgBCoW%mL@xyUby;%IwtfY2T0f%X${;FNsdGP0pN0l(;;Q-Tg zW>9=2fc6DA0*`+W?5DM#;mKn)e(6|y=fM9X7@GU~>{>k39XS8}JvVq@31uyJiE#$Q zL;&hiA2{yD6j&U@^O4$R1DfPRu!!G-!IWSK_25=k_;CX6;yeDskZ97uu>?P<6@Zoy z>d_9#rE~POD;{dpB^>L}AZoG?ig~+)NsfDzpbDg^ZpaLzkjLQq8BRmW7V+E-46_+IicJG57sc&Y!MUPSK0Bpg+N=6R7{}@m@G_R!v?# zSW*8??I(0&xrpu!O=A_jo~sVGkw4RT-3L`_Br}-V8li5{h9eI~mJ_A2}Q4W=`iBOVCqv#&{>Kp2Il$n&;oaQ=jm z(bop*D9mX3xrsYV8vGVUqchWYsZ&!U*zAtv6S`|4^J@lH-SAm2=D58VWfP=u1|+!J zzzE>K2#f4II+Z5)Gd@ zs*PP~H!?#bef$0&Fq>n)+m^KfOGg}62kr`xG!qCp+sl4ogLa5G4-}?!Hu1!j%nw(u zxGxzs|3%}USyAJMd_7DcwQro|BRU5G7+9k_RMrcng@NNQCI+HzbIxeWdX&M-tmlsSSqJ(tf1Gi52|&4;BByvgQQtb%;5t)V%Aoj!*T zHS%mziinx_nuY+~KMc`Ml_Yj>43e#I>&@bkqYaQa#M+`(T&de&{c9@Q6@$xSRIRV` zdI%Ix#0-y0-rQSneyIU0vUwy2eu;;o?q9Nxxb5q)N*`qky3N*$@z+bX@rsFbGd-nCOo4SiGw#4%Fo?zrzaJRoxW@71oh%6ls)lpNRQaBmNyx!XtTeX{7E(L z$UYoIt=no={Rq60?i(+P*40!sBlLHQFzF>_`#@eXZSvaAs$sf<_KE5JDSwXfeN`+)T_=*r45fK* zL(nT=jG}9*#w~O5=hUwWf7EuPGx&l)DxEvbRd1QU3&5SC@QzU_u6~t4Eps%|u-X|j z$y@VuLoRvJvO`g zyQDcPut!#I_^dHy*x3xqqWVQgW!c7zD%TjTp!aVOCCYi$z}uc2cS;s>&Cav&GQMU4 zHP*izAcIZpe{K283mM!(BB)rx45@uV$hSk9eQT^$aV{D)$L+45fxN;xUg>7W-Sy(s zJ|t`=4v|iT&YC0V%2jc04c7R)blpODn)WEXZS*Jaf7Omc{&820dp8hgP^GQN1g7pi z{w{u!_H%6pIcR@I9?Kv}7>8R4-xBkkt=w{4C;^>q-46@hB@=(#jL^ zrBl1t0YO?mCBxwNoi?=8%S)xPUR<)ztw_7pk|7a-`?kZLWlskUey(Jm%iy0K1_-s) zop|f#1<#{!EIx(e9t2x;&-m+1@RAgE+Z9IykBP!(G2ZWJ)>S1{$&*Mp*K^k=Msp%j zkW)nRUYFn+amx%whUn$=V7G8#zEMYPydSorHqW_{J_QmH^JoMwwkeCdkub<|q<-{$ z=zVR`@|PcTR^p)A!Rk(c{7wLh3OKb(N*ggt^?4iHoDv=@>}Yq!5ZNn<3jVE2_~%3+ zqN^yhP00u`MfU*rYn$6#f~KL@TR@25Fr7d; z{E2;YM=B-Gao^Bnn)?+O+~@VG0m&iLBr!M)yRo<`GzgkYwc>z&8`K#C-bi4pqt(gr zqwTM@WQu`(@b=`+o@@+{vqW7?=uH!C``u-CSfgEt5jIM6l)7W8-8hs;{6czpe(ADD$U)M1XaG@u7k)hN$EWE1fMRL6HGib zX>Zvh#&X~!tM3nnCS!ZI)cq%du9yINV*s8TDX_NNm-xBOPP(6zbCh;y9( zBFDL&@Ylcs<-R{DCCSDZoGGT*81^z19tmS&X3NoIhfoFG`5^9*8gxY$?aAt!QR)FZ zQMz%#5bh9}T_;MkYeyChoZ_5HQPKE7vKG*MILNf~kBqr=+NfUvyCN?3E}?AB9g9Z7?m2qaGz+%h0resKVDp z;W6%cUu`%t_r3TBqeWMIUiT4%bo5UxXbZQ**)zEfLYq%8_xds$gue90V9zy`o8nD| zM{hSrD>_QmZn!qW0t1{&OeI2tYJ3C9lU=9v1b21lGl4{MExt^a5a=@baAqb(VushY z8GE^Iiofp$SsyGgAAy+-|Q7}=Z5?*hDBrQM{0H|X~WN4BoXAW!I^ z8hp-!+dBzi5Saq6UIOyqAISs@Z0LAa9Rx_Dp)%ZZ=+!Y6NKrA7M?BmQy%}DvuLwNH zeiJ~hf6r;_?yN6jo(f#|oUOe)YZlW96wU!{z_T-$Hy#+%;>w%Yqs0Dw>0@Z?3b!xT zRCJFfC9piu&t=J+a3a)Gq3%v=C1b7jauzDu5AM2LA&E%I0W>>SXOZM1TWat4Qhmar zxzzA>a<@#^N4tahrDxF;iACTJ9B&j8sEHv6_SlWiuHBirt`r4bqgWnl@x0R37{>^J zT@@?@a=7u~r~Dr4Ar621RuLV=%X4(x>|7ZAG1n z8f*7i1`t~tFu~(t=1@1$i?1!Ip$P^0vEHy~O!AxvFb9kE|G}rFtyDg{#De zDCtiWX^10$u4H=GlSs`ZbzjYmC=)0jKDEh&>8?}Tqsb+$PAa7_O-pLbUCyMa_*|u{6Yx&|D(&=YubN`{`)KEBxOI zT-q=*=o1OZLeq#LOe8&^aa_9*t7`%tb0wkUV4;k#eh0|A-7;R*+4{@K)^?||zE1Mf z*n9Jw-gWMsG`!x6ZID}bPb5Gfzb4{a?ckPFA?Y+2-Fr61_o6-8v+xIfW15I}$%jjr z(DSgQxOAh~h4?9Sq8UArMtHD?%e{}nyZpErJyx+bpXN8Xo`eL*rP&)D`bKaurl0uM zVZVDpw(h4X&jNpLEoqHE9O^%Jat$DLr&RwU@eZ8e2q{$EqC(jhID>^U3i4%^zouz7 zJICXX(XR14FxfH9eC1HSK?zUJVZHi2261Tgpf|&zS-F_ciz=JO21(IW#NU+SSgG!G z+<+Wc+FjEqXr{%3jog?8=Ad_^X10TEvglCDRC63w)hcN6kbHd?IsuHEe$XTR@3D5Q zy5&kr;Y&=yin|Lb;NI+U$|?wd^Keu#cM0f`?&88bY;UBK!b^KhUrS;O7tmNLimHq) z3&;{dY#@pd>E!=J5<|@o+HE{${@|e^jx|F*$37?ClZBZ1_=r-#{fxaid8Z6c>Hjk_ z)_(lhopu6-a8RsPb-|aUaW2AfUYC6%rFy?ff5!9u)7Bo4n?$g-9oF_w7{G?o`hzoY zOpt_a6klO|E(?wa1@l-MdH9QT<*w5CXwZm&G589yo(Y+kJ<*rAQzNB$uHE;MzpJfY zNTK$ovCHn7@q+=#6hCR+#4K9*^RKvH3>1S`tL^~rlHHq|#Bh&IS$a8XpWkUxK{Uaj zFKZ40kl4jAyeAPQ_|5WdN%-4U_r5*XvFkAwe5Cbz1rDcemq}UrfsO$9%xX<5{8X7n zoNQ9nSp$#1j(z&_Q5Nps1ML@wKWAL{pZ;uA464;Ee)>^|o=3Br_B6 zoz=d;i;=1lrjmRx?K?&}uG-$8miQy$yWPN_GK;La6xN;jyBPWnt$qhvjXkCVDmlz+ z)inp{#_m6|VV)Bs)(UYstIyC>ioxa}@kXTi7(@G?QS=4a6_gY z%cAp)1Em~|Vnp~d95Ll<;e7PD0_W(Dtk}DbFRHW=_(_ma4@Y#!H~LcoYE)Sd>o^8S zvR-yl3@fI}|1SN;=3@pN=RQb#a{F{!HgWsMv=IkRMHlp0f10HEA~$wTbC@_9KIIE# z+)gAqP|9Y0QydolrKvyAm_th2^{ilTs50#tJ2KCFcz-{M`vAq0PShZy!HRVxo!7;D z1_DDN;!nix&U}`YdIMce9mHjsA~60a?{$Fk%OHHP!xgr>Ugq=n ziDZXlM!I-me6FNm`SnEr{wojcJ@K@l^v1E-xK|uOCV_PF_E>!ncd|VCD>8e&aq6i% zX?~A(mt2UP;_J%~$gzs-U*V)KkF~1XwNl9LC)46iZi!wyr)ROT7*Q;_>hgDK924c% z^1PJvnZp!YN|#3fN*FOrSV|VjE!ZHVNc|r5ukrgrA0A8b%vIy9EORIOA~qAi9vAgQEDf~Bs~LQib(cuX-W&KvT(YEnCeE?TypNsW z`JKd~nP6U?5*sI)8b5qM#E2((LzTASd_`W<*~RXdj|O=6a9b&Rl8AKi(8!RuS`cgL zJR;RcG%{0pB92BoZI0vqWN=?l7)2^DAE}csGBsWgCL>Bz{W0pD^ajLmM(B2aZG1*r z@tVi$jYXOj5GB$fsxSJLaabY$hxvn=l12kVuv`8EmFQzJ0yAH+Kf~*2E^-3Q;)&el z?v4G>3l885IcroUMYz965|N)NytX)>d@^%)`V38~;vHlW-%v&lhQ=m8)63-K8!P#W0~<};1;ojR+UB{VzqfU6J~{4_MN|6C8&4wH+c1zr zRnYyu4|r^jZwx=a+*H6TMQlD|3#AK3ATkW*Gt>iC_~>~ih;Ly8Em|f~9QS?cCivkj za(TVo@$+~f%872a8*LuP6#ng7Wd?rkB&(5kGGi?-pl%=xwY%*{M^1mxBtPWdePh z&BN7S8t?Dq>tXYRwFByhqj^CG+vFu&778ha`z#HMlNhURRnw8CRoMcZ zd@2^c4>=8e5?0!ntyaHTgKh5BX43s6)M(*E(ON&W@odras>IC(qgG%pds3@iWkzmw zzlq#i|Hu_SV2yc~SbgX181OzxBklufE^R?V`+X5+D;%9P^61ipMIKAZ(6Gw(_)Gm6AXC@W{0dGc(L`3nA zD(S8#_B*-XoJ3p`a=!k_QyvBkKRVq-BIWGG)2{9AT6kd+T5i^H-Et#F*aeIQNZ+0a zZ@+b6GJeleEE`vkw#q2!Ik`^ii-Ki+Mv6i$`#Nm9!Aq64&xu-7&4S-#Z>$ zfv{Gnt$^NC(;dcC%9V+1KIWCQ8cd%l^zztAKb3$~PlR8Eh#X7hTUeVo)RM)l;PUst z%VY!h{}yv)ry}G zN>EMq+mFx_>;1R;k9l1xjX6+6&-8=M#yW1pf!6{UhUQzYqBwcoDE{UROm&Ezj585- zIQ3jm_Dhmje*T5nGt&DAJE+3#4h+uj!XDYqB3Xn-Lu7$QK1t^Cy|Wjk)LOUeW4K`g z$`pV8rA`>Kq!=MU`N7%Hf9xZ|Wc--u35kh*>Gj}BX#a{N2o%mel)sxnebXG*l)PQ z?u_<3Z>-ZBK;i9R3A{0{T42x)9d0>3z$Y3Jik=W;iBm*yw=<3rh$)Xg`0IA3BV?h9 zNCLg&knn1(D2o8xRl5YHBqo+mcu(0dluSP0I|*$eLK)>jjp&t(k?W zY^l9Y^5ZV3@i1qcpO?-*XPuQee;FV;@SHi3`R0c5^Lv>F3VlhOY0krQDCnJ+slg{M zeT`S(e#v9>>kq}=nS}q&WYbJpx|uzF+m3EH=6y1qo1;5+%=aTst#4e+hWI2Um3Fk+ zB-M!r^8ou%oFrXcl3lsL_rWdp_zeMUHJ@4EuaG!Vz3(5f6)um31N42CCriHFC8zFk zVFT&(189+D5Mx5hi*pUKy4{421e3b{H)9eI^g04ki1Rltes1P8n0nD4`^2P;UK48o z2VJdexb87w-J4xjY>WFB(I2_TEi?HBNA~^c1&TWG_EJ5#|H5izHdl3VQ$BKyT|+KchKO-; zk#~bw?NpgmFfhN*g`5z#O9J-c9U4j)hxUCdQBG?4;YKI}QT*;=@It$tA1wj{ZOJPk z8EzEAS9V;I@npu|5%OwFRYDknpGq7T-%K8?{Akl2(JJPAjPbYn3h(Zm#QM=Y5l`iT z9O=!ndj!!CEB2wb+grkwZhtWjZLb9rf0G&=1koJDfTm5e%X0~TiFlv0Dj6IRPeUp- zlmI2qjzW8ol)_NTmO;s&LG_-A*bn9P=XhIj(+|7Zmhoe%kL+reH&O_D_=P4r2WHj7 za83k9(WqsnPW;*irrK0YUA>I4as>mxVAZ(I34`2e5-R5M=O(|oXu?B$-@VPobE~LM zsGOr7n)SK94kp1DJzG7}#`A&)weMS#6@2YX@_{`}j9qfBjzb;M^o8oEG4g^e?eb-l zjZA^0;NGFfY156NtAMk`F=p7odD9cmr<1>yKMl51g-pLTqau&0WK!SbZ+r*HY+brP zcB90~ES-98xX`%A)un;}PtTxMk}=rv|XXw5zf$#ZpR^&8@v{nnBee#_R;4UP#Y@et;rH^W>gAo>ymMavsbl3yQ}z>HbcE zx{#JT2u>Zm?WY8@H=t@zj0318Oo%nj*8?y%D*3jb*@TU2ug)XjhQzj|DMph5yl}%3 zUdh;Xy)o!8xl&&>Hb25idEul0)`z7-WgjS% zkv=&pqk@+8eT#z)0{^!AztZ-WS%F=lgqcWhVzj*&nao<;Z}NW>bOPRJu#T^;J5|{@ zvto!j5u(Y+I_RXI`CXH9GvtH7F4xLp zZ?Yd9x0Z*=kQgyU6PTk&?81!iYu@Je=l90RvX@@MwyqYoE*|@joygB81&giRK`Uv~ zUalF#saY%Ru`<~uRU|!S8d?AoyMH_PSNc0|zkFPe1~A$ab57$TZ(W>nkyEi91I6z2^Paqp3Uy>u7Q@MKf z!BlYum~rjm=Hts4;qA&ent0U(!FKP33ZDh0fW3B{hJFIbzwpWID2H-?gQKdF1rQ&q z01}yfA)UyXchHv+j>^`HhMf#)9F*4#G8TQw7O_zGQBxBXhiJ|=fXQL$ z3a|@;i}b?0ml^o#_z7A35sW2MI=d!*Yh1J`1H$vw1cx(3$6EgzZP~SLeJ%rg885CK zCMGg6A@RXd`rb-wU{6^_G~cRL`xRJJ;?oZI)KIJl9#NKf$atH#M_iXieCx_vzw^zW zyd^?IKjYaRrZ&)^!~&0|`2uG5Q$deN8W?t7E^ixMEc%+=4!4Dc@X_kQc9>Dy!x}(e zG%HNf;CyZCYd z8bjq5j!S}JF4F6z{(tR564^f8UR%xh@UK^=-ZQs=V{*MiE^~_(@B5vU5oAg~qh^o4 z7QB+HVB9v~VU}837>(r@vy)df@XGS?x%4%o16ICZE3uXJ%*%McjL7YC`K?QFraMW? zRo)-n7x%KUlrwn(w$Xu^a8+LJ50BOq6_F`-o{TNA(t7u`XX(JLu*Y6I*nP2^<>TDi z|69xYRq{DISl~ReJae&=XR);jYOt$(oe$@9RTlvLv&b6R8JLcBtSSX;R^IG1-c90Zzs5<=$Ra$ z8EdByud6x#D};E^1@;7;$pszCw-+FN->ip1!|VRCjXnei-`aVCF1lT?bMz(L*E$~& zonV`_{mvwSxG{g`6nwF5njtBZvkCIA4iK*Lh9ssr@b&kkt}@+LSO>L7$7|&6-Y88}-0OVM@uVr=x$9<$E%fzQ2T?tbHT#BTu*)mqXu3Lo{Vz(> zEhWdhx%v{{lcJ30cl<vClL6%h-Nr@J*tpt(VS3VJOfGM%IMGTPNaJ zsal4IrLR>K2Z%9H+5mDT9&~x83$S4#O>p~%WC4H&Ruzplfp!?)TQGFLJ1X*@ovbMh z5uZCWtP5)#TQRY2o;?tjl2Wt9##{wsIyv>b72NOw#WR!-=QP&d+;88@=nD3OXog-3 zfr|%cuPbXucL+ESRdBf4cbSl79UaO1kLP)JXLsZ4$#RopRecu?$EK}OsDyM^Z=`k# zz%QE8-h1FrG!M>W{~0a!qZa~P<+eG1=hp9>8Wi6O^)Td2!Qx(93Ac4EA1fJv_4Lmmx<3cqT z_j5xj21f!+??Bpsi|FG9wt1DfcbVkFGb;Tydp_>oIkt-pu<%j<~2A0QyE*K8Fke#9%Lvzm@m+4X@XQ*#Kp103|uy(D`~G_GqT_ zV2Pi;nSSHTGqQ7WmV0T3f%nR930nqN<+hy&cgY7=M{grXHySfz9BvC9}~1JYgoK}{~8kq`2Hy^ek4z%AJFG(`L5*|ws+_r~1L%2gO17?bJBR(+k=jFYHT4rbQt5@&2{`Ih% zGCSq}3af zIwUfNV)U{W5Qc*F^2Cv;0KE&m#RPg%e%jh8#Xz&Pgu3xCPmbEvY`P(FJ%^)eacu46 zM2u%op!gdBi^2D+`XVr(>i%U?(9ys1#}1KeT?i)#kQfJd*q0VyK~R~V^@&}fGycG~@{h&r<7|43Ym07^E-&E9GnEzO`V19Z52}n+6G77$vakVto}t`n9`8Ra90=de2<e=$s zy!=n}QQYo2W9_l`U$1eEklZ`a&ekAsj!d5}P2u(NYTIqT%6DW{@-|s`h^NZzKj7&U zSo1-4Lxkt7wc2@dUS$-U?o(XorvLMVhA~ zMMDqzkIkJI7Dm2)>(9u;EPQM4hI$pc);q5DR|Az%EAuP-t|s~{#$yEK7M?DkF2mUs z_xa2vc(1s~U3!TvQCP|`Qo$ZDU>NGyiCRNm#gIHo?7rY-dD%TrA2m;S3s$#PcYJL= zD5^P*fc-c#Z&KoAH3m+-*WP4AQwXEU_uGcER5Wfn*B)481dbje!OP|9t7#c5R1enb z`vyK7B5!+@MzSuKu3z{TE0Q~{85}Yl{f#={`Y|iJD3CI9e+!lerFLNfg46#{AS$!v zeJa@EKJv_KYi@IyyI|cVmo%AI&@_{EiZNynVNs`y%v1RfmFw}z!9)U4xr91bCYU98 zf{uCcg~wqbHkK>1va@uNH(WJdt%@_{c3N>Z!I^2S!0cL?3C*U+$ax?6S9cQ7QW8*= z=MoEcabfdVW;qq52?Ad$?x(V!%|-?OBjYm4updrcd!CIFxm|{vCHk8N;wlV1tEWt) zd&eukh;sd1vBvuYwgC?tNZxQ3Lm%E;E|$%D89W|Ej_gnT!7HA@{qG|PQEG?9M|K=x zOO;Kc*1()%jk?2d_pL5 zqen08)xVb$2w?Mu<~*|N@|?Ewt(=bcAgvEQX@sP7SfY?2NiFOi6lZZ3XMHQxw*B?H z-1PT*T>d9q!HK9J-uvUZQsf-u?1B;(ra72zh75{f z>8=tHxe^GZ^@)b;WqFTAK`ik-dBY`pj|Z;$C|1OgDQWK$ud`ZCIkfx|g!@h$>rE-W zC7^|~@WWTb<|+CX>|OX|SU;Lg0PjFdUYz1?-~Ngt51tmRRdrVy9E(W{5C_kZYL8B~ zgkw$xhz`~pJ;M>}QTR?z?CKIdR@sGuZsWg^?5Uq4^@DS#1&hb)u&dJi$WwIPF%*s3 zjJ$JLx5bwdMBr}+|6Y??ua)OxEK6D_c~Z#>3$FXakDNV8$)s_9IfA~@yzEovNiMex z)k+16;Lo}sr^(&1Q~8CLA5U8^HrMyh9}W0{XfeNyCB@WCq#K@~O(aqWDoqUw4Kw#2 z%T$IQ`aZXV=ssex8efz4u-RgCL1cB6=4+>L3$B3`Xx|3Zj=F2+<`XM(<=KqD33MMTsCt zev|k6z2Exof3wy(cina7InT52xqCkw=nVf-8TfgHmTC5bvs?;4AJq3By}iqkLfehT zMdz{1q?7Dnm!?`U4Kq92>Bb=QNTWYPQhAn#W?ZtXKyo)Ahrs^D1=&_*g;27q(en7e z9wuSgD`Eya##7A|ZRaspPS6`exY=*3gmWl#$>jZ$IE-2*&q#j!X#%=aeXaBxzT{r$ zw58)74-6V_68+`dYZSDg{`X>HtdGfor;%=_n!QAlDOp36zevL4*)lh|vNr7f1mrpy zyO@JjOC42kGc0r!fT(NaQPWCnZ3`9(@N)5)G3HAGr1P9`E0fYtm%k&8<1+u82<{&c zT8B}I03@O)*~rJ4H~nGc6>V?=M>IZ9k3>I){L;CE)4lS`er0KDrt%vY{QR@_D4|{x z2>Sv*XWn(Q&IGSXI--k#O{dvY)#cYTJ2%iAF_dVbF*6H$&$r!g2JbW3xzw`>ZJ+q3ug@?YA|GB2tF^F$k03!NdT3SdpSr-L1EKuj)XMX_ z-IuRS=KZV_6c%x^ISZ_y;O%u>7K>%)RAHwmutn70d%kj?iG^)gYHVywCinH@z3Jxa z3j)iloX)S~^MfcE=;}RC@lr}TRRJ+RK}MNN;#0Hdl*^yEb+}&Y7Ff%YN?)8Bk}{Z@ zod3tcuM?!N`F*{KiCs{o>Cd%KPoaz}C7OCEE7j-x@HP)ErlyVU7a?bxdNjwKID!c% zdIh;CuYNIPT!s*{)cg6fL^Zf1CAAs^R}EEol~Sa=Ic_c?447QaWOWju{}Kv04e(A5p(UD;+m0>y?re2M`=8j| z&+X%V4FQjD8L^yB56<_4pO7EfA*Xi=e%f-SLq8hTu zjn`IgrEZ1%-mXZ>(??5Wycmj%g#89|b304Yju(H*qdYbCywf1u#!GuKz>%{b_sj8d zk9vWK%gf~(QH>@Nc>Ho^$W}36Y|uG1!_{f74gU9!zyAlF*+;&1ENdpeBYaxrF%Gcy zcGkuY_yD0TS+Hbx^3Rnx&f9=f!aE2va2PzuFmSqzAf6&?v{CLos60cGTz=u90%Qll z1oIA68%2yN7Q;ry0*)K=EhJ1ep3!MKsgvTC!HNR2JB~yT+0I|IQ`P9KSnTloRo*CR zq35J?(NpDk+$Ex9Q+S^-l8j}@_jW@Q#TTV*_)yt3>ILK2^!VC3HIaG$A7gkjW8tgw zybEStAJ)A9QcTKBPC(Gm$H;$tBDOnDFc8ENZN}_Aa7K_9=mJuv%nd|ta)!s(De@YG zy61v55AvnFKYjmY`D&@#AQz&0aM4pbK#kK~9nQGaw+vr1Fvck6eCp4wK6w&sTObP; zizS>D5RAX=jx(J%b-tzgemn)!uNT^>WyD|2wBJ#gsoM-sx$b3u_X(=0HlN_+$1tM* z{qUO^8joxt;dum;DcdA}Ch_~On9)Ye!JU`aU%!z9jCwy4gc&}XPj0&*ZGYi8SkoHn zF2-^*I}<;#xracvQzHj9^~p0Z?#S_&{fV%YWGNuw8wi{74nH zr1fhep@*)xzvTgy<>$1qR`?K4qXi%;qKVhYAdy;OqS5o#WY1>!Z$eug5n223{^DpP z$@=Qu=$91kGkGC_hdUC*RJs*6A;zq0$WPBvgB5Pl-(pW&9+Zg<`XBE-WJf`^E`~GI zEjedAz?mDB@OdA=VCCP=%Y(iL*9FIr4BSR#R;a^5s}$Bmn3jZj>f&`~^yX2Px9 z4q||zojLJZ?2yUs&wJhXsn#R30|OgB{fNzq$@Y}EC+Mk?PC(hF9y(fD$?QU@XQdicQDqNEE4_qYu7epbh%nhJ`WiQDWi zZ>LijOM{2^XQ~ikxcpG{fhUDOL`PTKO!ch(sY0dz03Nl%>-PGooaUEqZ$GL}9o%3a zp|C9}WAk$=3YgojNDSolff+!y8u6(l(jOWxUrhsHh3T*(_zTl*WR*(3EIaE6BA?~# z6fBA-DoADb47F*Ab}q_i4vvT$3A5GQC}79D`O&%SJ3C)Sy;kn3*&f{HrZP(1eir|@ zOQy}~qjOR1dBq6k!o7n}SvHZgOjC@XfqN!wH(uM2?ysNPKUXvd#M=Mx}9 zt4GEBaE&jKo1UoCHU_re@yl9fkk?j2h?(Cj<%BCNe}3?83ZPuBUd$AeR&U*K@P1c^ zeR=U|L}E#o%e2QAjMI+I<9%+n8HW+{HpDbqJ0q+DJ|U-`evg5KEWk+*D`KQ%feBX% zrktNUGE1{|a1w;ER&vm}WQ&dH$~oHJ967^j?jkU8?lz-tuY6<6F>#|8n}unQW}J_2 zNxS>PyEuq@1_^Sm-xa`AUiDNurQd7uLA~+UW8Pwd0SQU8sc#A5fURtMlxug$)*yK5 z377>Q6if_#IU3PsJ2VU8OrPdr7x3M>HG8PDNq9$*{ZZU_wT3q~V9H}~spu)MJNcY3 z`X{Vq@9`nSCV_PMTo$$m-ZpL1P1}4EU85i);gIj&edcO@S9KSj{FqKtty5k&|wCb4iU`~uxSy967?@fR|M;`|x?bV@E0QtsIT745e zZO~}HV+261%!~qgf@kL`A^y@_H8l5iw{ayKCUr(Ir{8PEZ5T2~`2Rr3W^`u1ld89M z8_(lTtx_EZ4jha+Hkxm4qy?iw`)z=u%5WdrI! z7?|F@u)8+CJVLVa)y2~)?>F$V>RU#yWO}i^>lshz*jGKiu&OVJ#8g3#nKb(sGIR>k zo4||cFAh|LWI=c$^j;8nbw8H;IV-w*9DC%(v9(;t8hq$XSPKjvY>qm}jUoO{;hh|9 z_&^%m1Z`(-=fj_eY^;E9SXL=^*#N=ToA1I4!gy(vGbtw2M4N&cY>Wqw(n)$i75FCJ zKW?Mho<_>se6oyj(ucR?fFEnz)viY+$_Wm5o-%qH$J+#1+!_OjwmIm>B8^|@`J{J3 zxqK|4N1Vx9+ws-gZpZy6ss+3!FsdS|{O_!O2H1eTH^e{wPR7FRwcD5nUeAU={~D37 ziuOE7?gh2gbsL@0usJP}Udrm%*9iYuIiWNIP$*y0@tpw_)(w8&X3Jfsj(LA%$f6ED zx~JrUz1D@@rDj@L?k9-5MMvKJB^(VyyVjnBsF1l+h|NbC-IMa}D>?Z<_^<$_hF6w7 zeaYIgBo#;kJZFQ$wzxO=*n(GhnFbs`Z9}NTf?s5<3|GIEW3mg*U`zRdb5&jG37SUK zLAmhSf`u;RPM|arYDiD)OO??22=*Om?KGs;P3L}Fp2pkb)Cy54{kAgXf=}X{Krbq( zANDU7QTM1HTm#(d9VmRP{`456KTC8S$||T)fvoimkk7%Nd&Gfo#8z+df4*M+nCQkW z^>0ku_{j%zu{92EDfBu4(|+ta>g*`E;PpsoYY*!IXGvowpQz)be#<>{UF?+pb?Ua?l8c9515 z?h?)}#GIR$caIOH8c(mQ*+F0yC+GYOp|2?_%CCbA%-8$UPmPWT>!so&6%XEy6U;HL z59oe5YwovDd-ToZ3K3lVyN#Yan>6L-^^KO$&uuSB$ZjLfG#+3>0R<_@lq8zGy1Z54 zzO$om70>-3j~j)54Yl0UPFR;UbyR!)lSLUt6Z#AJS&bF78ra`w^lQJh!b$FfDTmUs z?A`Z#@E!y|p1{l|`F(RFTXlW55G3nP=`g-C)3$X}&?t-}e-rKslXls<)i^%4t5b^i zC>Zz&YMBY6YV7G^cd?4g2d)=FJR3qTP#D)BpfwLXW2>EC zzcU^b{PZX5XnXFF-1{+{n*vJH7qtR}?g(xPQ(c&jdmae2EX6IIMn$`j` zw)yGL)szN1+8LeVfD0p6-+$Ce!N)4AZIy1$S;dEJI4MmP1ocChdG=|jLrgNpPY6eZ z$Se78*7QbxqSTNU)W>_U<6}LMCXC%XivfP~RU<4mJ0Z9?f~0ow<%^t16>8*c4YhBy zugaG(3;#q|jhJX4nd^WC$-5^`XYoo}ZZ;+K`v5?Htg$xMwX^jwv3cAJ{_Mf!`~$b{ zJnrqp=zXpLnZ`{D7miuXfL(7?$-BS!X<0QOwq*jVNkDZ-rNUi&OZ1%K%1|MR7{{75 zVq$*#>CK-{RMuqnPX315qeYHjoH-YMt=BM*!1JE56w?)03(1wrD-_4g;FxDkk5sIRm2qOWEhgF% zyV}8wX%iLD2L^&Rcc~5ughF?Xf}Z3Tj~Qd8#9#$iUIXP#9XWi47*@O7N2kdWT0W*X zXgfRe*<9w3EpF}5^qzIcaj~JSzi5#``O=mru09{~)Bz)QCSA5kmJS4XypuLN)H8Qg(%B@)Xgbk_-oAxkj+&v01Q# zJ5MZW3>;qoKwKj;#Hn%(z3*P~y#396yJZApannj&*PAp9Do{X^vwd=3NtiQ(Xv4+* z{07=F#58~6>BaAh^DufUzbGvF{-keo3Hw5W^GlpN*@>x`+7DqWz(b_099Pb-O&cq) z=oAx;$2C=-MhPnN*fLZ!?~ur_S^9^aYE9>~F5K&*nZ3EqSsJU5b1+J^LNr;y#W6>|YqZ?8nEUzyYBB7Mu7tcgF(DPu~jM?UZeiEAsw_)xUlst7rJ zM6Ytj>9dp{qKwN4l~uId1_-y8cZn~}n2z;48hLl0zy9rkr>h~tSV;s6Cpgi@c2yc=kW_f@LHE5QaIZb5 z@t$L(7xx6+Yc6d#%p#8S;OtW;m6H<&ej-Ix=?{$(T@YJEnHhSE~7UpVyXA6GFJal!|E} zMX>MriU_ujbYOkEUsrz{^;A;A-m-%&Y+YBGMG~;I?jRn95m?36tp@waB!pPX-{tyX zmnz<+XWDXx;kZ^CLNXN{5wTe(#5I0f)M!{~43Tae>Li_CfKz z?m~d2Ck%gD_|!tko(c7RdN}uc`Gd>{D5@5Y1Ui8eN4%8IotlQR&o7>K*j^P;6CyOt zUKINrx&ZbRSoyKq21Qjdq!0)-hCEsykL;{Kw8L80D}%NU0t9zQ4RK;19sZ>-iHK49 zyVn*`|Av=?V9zc-fPEp_Jh5HuD^xFDd?{VF5^_Q>f4wMRn2tm~?#$n%kQr3CCa1idqad+ZyP4(4O_~RhF zQ*bV0Ox?x0tzCgHM3|-z>MD}3dqcEymGpCR~3hFs70S7l_a_4cHiNseG8j&o|tGiaZMnFhy`hjkL77^8iUx# zzx+{TW^?0Z)}=g=3L*)bX=bJ%UO1iVVCFoXz&uh^FkZ!SHI*&l!*0;?!MY8#wM-yT z>bdA8IZ#YI=z^ZY@TkG*xg*ifN)*6Lfz1p>KU!ebeR5Gf9k`|%hk)UL&hTm1^D#mf z$c|^$+P>DT1S7SShr_2rh^GC)K0hd{c|yC_b4B|Ewf{eiUs(_fbtUZpgD+<{xL~DE zZy_7|&vHOdy_b{2s_XC0!5;cunLaQQ4HMZ&K`gM0M$Fobhsa`{`5S5CEoHO#8t=MA zjuPjP!DXnB&O|R(uH?oKuu5L&_4HxK|C2I#t-7uq1H#EM&tI&M@v4f%RYz3&<5Be^Mifq`ILxjezV}>zDtuKbliZ;1rROKi`e3%K{M@3H+D30Wz~_!2(|heh;4&V-KKpE2y5`h-z>N?6kvS(_05EQEj4PHu+=)~sG49`dO>d$jJ#T%yfB69 zvn#+GiGc{ZX^QiKw zXE#UqS+lyFZ|ghVZ*_V>CGdJVcP`N1@|pi*T@hU^s#QKlEQwf5p?U0dYGH3dxwmEF z1K}$Z1wKM2+7y#>WL?IQ{%jrcDkWlRZltJrWJ39&UR;|(I%%O4R4((-7~znpexe@( z{!{LsV;lx`3DNu|O;q+_X_DL9V-P>PXYzm-nS}4p!7!>*&|oT91tqkELEZjhudi$6 zJ4cd^iXenPj%4Xy>3DUeKSbD^gv`hGFLR2+Q#ijAGr%p(`~N51ZS(&p-L1+ZOI8TO zlD;>b4^MqW^L|RII5Ihp4kT>l(Q3tTI*@J7_qJ6ujyqlpqWFbnX}vG_A;&Ro7_G!S zf{Mi_q}_c5^Txl{?Ay9k>$J5%&O(Z3q;(6*8fWA*)&h3Kkze{#!^i~-;F4s00S`Tq za>dPfrtJdrVxB04V)VFp?V#roujyNd0Conwk%q?4(; zA;KUaJ|cJiRj7yZ>sxkfI1A#7(HouNuR;hpk+@@Ht@Pa2eP=pgmkw2M8H4;nNwkhl zt7R;2@RNeKET{d(Vm|}OXLMX2rYYA$7~FYLPW-=u)+Jfh?*z;)@Z7eQ-g%T(J)~|j zgaq+8+dsT4I%{xsvr00;n}}o-yZ;x6xkd6&t=0dXc4?!94>&$weIqR|_5{iQzd+3` z65Vun<;(N|)^!QtHAFWU?E2rSDPqfN$8PTawhg0_Ht30#M~9_Bhg0$;7RF|aJB3d< zZ+v$7%VLY8S-p(7ozZlz(Xf78cPAYAJrPGjAp+-85^}@7yk+jma03OU@1?~_!taa3 zJ5Z|4Z0W5q4bkfev8r=93SZp_{#?5X=oN_yq>rlC>xmH-TwFaTA{*Wy6rt7aud*hy zSCN~U4++Qxd5x&oD`hCysMx2n3+vAh{oy*I)JT2lzfNXEhR8)}b0gj^#G;(vo7VyF zV4ip-R(xGtttii&esa1|X#OV%e(u>^r`Y*bt(IZiyDiLd(`TkdeDVIaA-)$Y&PnNJ z{eKpvgRDbK;38TK{SVCkPx?j!s9b=)AwqoyasRTp3<2y4HQe1;PBh5;-{tH&c8naw ziqn>7Z~q4d|C@QAagfb6L$@KC`@hpHIm8bC=GRi}+@Q?I-5^}L_l@qo)pU&df7suu A2mk;8 From 837a70547a6d9c7ccfc0cca7131c93a58105a3ab Mon Sep 17 00:00:00 2001 From: Mohammed Yunus Date: Tue, 27 Jun 2023 16:20:41 +0000 Subject: [PATCH 569/628] Readme changes --- README.md | 8 +++++-- docs/sidb/README.md | 58 +++++++++++++++++++++++---------------------- 2 files changed, 36 insertions(+), 30 deletions(-) diff --git a/README.md b/README.md index 1cc6efed..950c76b3 100644 --- a/README.md +++ b/README.md @@ -14,6 +14,7 @@ In this v1.0.0 production release, `OraOperator` supports the following database * Containerized Sharded databases (SHARDED) deployed in OKE and any k8s where OraOperator is deployed * Oracle Multitenant Databases (CDB/PDBs) * Oracle Base Database Cloud Service (BDBCS) +* Oracle Data Guard (Preview status) * Cloud Native Database Observer (Preview status) Oracle will continue to extend `OraOperator` to support additional Oracle Database configurations. @@ -28,6 +29,7 @@ This release of Oracle Database Operator for Kubernetes (the operator) supports * SHARDED: Provision/deploy sharded databases and the shard topology, Add a new shard, Delete an existing shard * Oracle Multitenant Database: Bind to a CDB, Create a  PDB, Plug a  PDB, Unplug a PDB, Delete a PDB, Clone a PDB, Open/Close a PDB * Oracle Base Database Cloud Service (BDBCS): Provision, Bind, Scale Up/Down, Liveness Probe, Manual Backup +* Oracle Data Guard: Provision a Standby for the SIDB resource and setup a Data Guard configuration to enable manual switch over * Cloud Native Database Observer: Provide Quick/Detailed Database Status The upcoming releases will support new configurations, operations and capabilities. @@ -38,7 +40,7 @@ This production release has been installed and tested on the following Kubernete * [Oracle Container Engine for Kubernetes (OKE)](https://www.oracle.com/cloud-native/container-engine-kubernetes/) with Kubernetes 1.24 * [Oracle Linux Cloud Native Environment(OLCNE)](https://docs.oracle.com/en/operating-systems/olcne/) 1.6 -* [Minikube](https://minikube.sigs.k8s.io/docs/) with version v1.21.0 +* [Minikube](https://minikube.sigs.k8s.io/docs/) with version v1.29.0 * [Azure Kubernetes Service](https://azure.microsoft.com/en-us/services/kubernetes-service/) * [Amazon Elastic Kubernetes Service](https://aws.amazon.com/eks/) * [Red Hat OKD](https://www.okd.io/) @@ -92,7 +94,7 @@ The quickstarts are designed for specific database configurations: * [Oracle Autonomous Database](./docs/adb/README.md) * [Oracle Autonomous Container Database](./docs/adb/ACD.md) -* [Containerized Oracle Single Instance Database](./docs/sidb/README.md) +* [Containerized Oracle Single Instance Database and Data Guard](./docs/sidb/README.md) * [Containerized Oracle Sharded Database](./docs/sharding/README.md) * [Oracle Multitenant Database](./docs/multitenant/README.md) * [Oracle Base Database Cloud Service (BDBCS)](./docs/dbcs/README.md) @@ -119,6 +121,8 @@ YAML file templates are available under [`/config/samples`](./config/samples/). kubectl delete autonomouscontainerdatabase.database.oracle.com --all -n kubectl delete cdb.database.oracle.com --all -n kubectl delete pdb.database.oracle.com --all -n + kubectl delete dataguardbrokers.database.oracle.com --all -n + kubectl delete databaseobservers.observability.oracle.com --all -n ``` After all CRD instances are deleted, it is safe to remove the CRDs, APIServices and operator deployment. To remove these files, use the following command: diff --git a/docs/sidb/README.md b/docs/sidb/README.md index b07b32d2..492eb6b4 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -14,8 +14,8 @@ Oracle Database Operator for Kubernetes (`OraOperator`) includes the Single Inst * [Dynamic Persistence](#dynamic-persistence) * [Static Persistence](#static-persistence) * [Configuring a Database](#configuring-a-database) - * [Configure ArchiveLog, Flashback and ForceLog](#configure-archivelog-flashback-and-forcelog) - * [Change Init Parameters](#change-init-parameters) + * [Switching Database Modes](#switching-database-modes) + * [Changing Init Parameters](#changing-init-parameters) * [Clone a Database](#clone-a-database) * [Patch a Database](#patch-a-database) * [Delete a Database](#delete-a-database) @@ -24,10 +24,10 @@ Oracle Database Operator for Kubernetes (`OraOperator`) includes the Single Inst * [Setup Database with LoadBalancer](#setup-database-with-loadbalancer) * [Enabling TCPS Connections](#enabling-tcps-connections) * [Specifying Custom Ports](#specifying-custom-ports) - * [Creating an Oracle Data Guard Configuration](#creating-an-oracle-data-guard-configuration) - * [Creating a Standby Database](#creating-a-standby-database) - * [Setup DataGuardBroker Configuration for a Single Instance Database](#setup-dataguardbroker-configuration-for-a-single-instance-database) - * [Delete a standby database with dataguard broker configured](#delete-a-standby-database-with-dataguard-broker-configured) + * [Setup Data Guard Configuration for a Single Instance Database](#setup-data-guard-configuration-for-a-single-instance-database) + * [Create a Standby Database](#create-a-standby-database) + * [Add the Databases in Data Guard Configuration](#add-the-databases-in-data-guard-configuration) + * [Delete a database configured for Data Guard](#delete-a-database-configured-for-data-guard) * [OracleRestDataService Resource](#oraclerestdataservice-resource) * [REST Enable a Database](#rest-enable-a-database) * [Provision ORDS](#provision-ords) @@ -161,10 +161,15 @@ To provision a new database instance on the Kubernetes cluster, use the example secret/oracle-container-registry-secret created ``` - This secret can also be created using the docker config file after a successful docker login + This secret can also be created from the docker config.json or from podman auth.json after a successful login ```sh - $ docker login container-registry.oracle.com - $ kubectl create secret generic oracle-container-registry-secret --from-file=.dockerconfigjson=.docker/config.json --type=kubernetes.io/dockerconfigjson + docker login container-registry.oracle.com + kubectl create secret generic oracle-container-registry-secret --from-file=.dockerconfigjson=.docker/config.json --type=kubernetes.io/dockerconfigjson + ``` + or + ```sh + podman login container-registry.oracle.com + podman create secret generic oracle-container-registry-secret --from-file=.dockerconfigjson=${XDG_RUNTIME_DIR}/containers/auth.json --type=kubernetes.io/dockerconfigjson ``` 3. Provision a new database instance on the cluster by using the following command: @@ -202,7 +207,7 @@ To provision new Oracle Database Express Edition (XE) database, use the sample * This command pulls the XE image uploaded on the [Oracle Container Registry](https://container-registry.oracle.com/). **NOTE:** -- Provisioning Oracle Database express edition is supported for release 21c (21.3.0) and later releases. +- Provisioning Oracle Database express edition is supported for release 21c (21.3.0) only. - For XE database, only single replica mode (i.e. `replicas: 1`) is supported. - For XE database, you **cannot change** the init parameters i.e. `cpuCount, processes, sgaTarget or pgaAggregateTarget`. @@ -366,17 +371,17 @@ spec: ### Configuring a Database The `OraOperator` facilitates you to configure the database. Various database configuration options are explained in the following subsections: -#### Configure ArchiveLog, Flashback, and ForceLog -The following database parameters can be updated after the database is created: +#### Switching Database Modes +The following database modes can be updated after the database is created: - `flashBack` - `archiveLog` - `forceLog` -To change these parameters, change their attribute values, and apply the change by using the +To change these modes, change their attribute values, and apply the change by using the `kubectl apply` or `kubectl edit/patch` commands. -**Caution**: Enable `archiveLog` mode before setting `flashback` to `ON`, and set `flashback` to `OFF` before disabling `archiveLog` mode. +**Caution**: Enable `archiveLog` mode before setting `flashBack` to `ON`, and set `flashBack` to `OFF` before disabling `archiveLog` mode. For example: @@ -393,7 +398,7 @@ $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath=[{.status.archiveL [true, true, true] ``` -#### Change Init Parameters +#### Changing Init Parameters The following database initialization parameters can be updated after the database is created: @@ -559,7 +564,6 @@ The following steps are required to connect the Database using TCPS: sqlplus sys@ORCL1 as sysdba ``` **NOTE:** -- Only database server authentication is supported (no mTLS). - When TCPS is enabled, a self-signed certificate is generated and stored inside the wallets. For users' convenience, a client-side wallet is generated and stored at `/opt/oracle/oradata/clientWallet/$ORACLE_SID` location in the pod. - The self-signed certificate used with TCPS has validity for 1 year. After the certificate is expired, it will be renewed by the `OraOperator` automatically. You need to download the wallet again after the auto-renewal. - You can set the certificate renew interval with the help of `tcpsCertRenewInterval` field in the **[config/samples/sidb/singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml)** file. The minimum accepted value is 24h, and the maximum value is 8760h (1 year). The certificates used with TCPS will automatically be renewed after this interval. If this field is omitted/commented in the yaml file, the certificates will not be renewed automatically. @@ -583,9 +587,9 @@ In case of `NodePort` service, `listenerPort`, and `tcpsListenerPort` will be th - If TCPS connections are enabled, and `listenerPort` is commented/removed in the [config/samples/sidb/singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml) file, only TCPS endpoint will be exposed. - If LoadBalancer is enabled, and either `listenerPort` or `tcpsListenerPort` is changed, then it takes some time to complete the work requests (drain existing backend sets and create new ones). In this time, the database connectivity is broken. Although, SingleInstanceDatabase and LoadBalancer remain in the healthy state, you can check the progress of the work requests by logging into the cloud provider's console and checking the corresponding LoadBalancer. -### Creating an Oracle Data Guard Configuration +### Setup Data Guard Configuration for a Single Instance Database -### Creating a Standby Database +### Create a Standby Database #### Prerequisites Before creating a standby, ArchiveLog, FlashBack, and ForceLog on primary Single Instance Database(`.spec.primaryDatabaseRef`) should be turned on. @@ -628,15 +632,14 @@ $ kubectl get singleinstancedatabase stdby-1 -o "jsonpath={.status.status}" Healthy ``` -### Setup DataGuardBroker Configuration for a Single Instance Database +### Add the Databases in Data Guard Configuration #### Template YAML -After you have created standbys, now configure dataguard broker for standbys and primary databases by mentioning the dataguard sample yaml. -For the use cases detailed below a sample .yaml file is available at +After creating standbys, setup a dataguard configuration with protection mode and switch over capability using the following sample yaml. [config/samples/sidb/dataguardbroker.yaml](./../../config/samples/sidb/dataguardbroker.yaml) -#### Setup DataGuardBroker Resource +#### Create DataGuardBroker Resource Provision a new DataguardBroker custom resource for a single instance database(`.spec.primaryDatabaseRef`) by specifying appropriate values for the attributes in the example `.yaml` file, and running the following command: @@ -667,7 +670,6 @@ To list the DataguardBroker resources, use the following command: ``` - #### Detailed Status ```sh @@ -738,9 +740,9 @@ To list the DataguardBroker resources, use the following command: 10.0.25.87:1521/DATAGUARD ``` -#### Set any database as Primary Database (Switchover) +#### Performing a Switchover -Mention SID of the any databases (SID of one of `.spec.primaryDatabaseRef` , `.spec.standbyDatabaseRefs[]`) to be set primary in the `.spec.setAsPrimaryDatabase` of [dataguardbroker.yaml](./../../config/samples/sidb/dataguardbroker.yaml) and apply the yaml file. +Specify the approppriate SID (SID of one of `.spec.primaryDatabaseRef` , `.spec.standbyDatabaseRefs[]`) to be set primary in the `.spec.setAsPrimaryDatabase` of [dataguardbroker.yaml](./../../config/samples/sidb/dataguardbroker.yaml) and apply the yaml file. The database will be set to primary. Ignored if the database is already primary. @@ -771,7 +773,7 @@ $ kubectl delete dataguardbroker dgbroker-sample **NOTE :** You can only delete DataGuard broker when role of `.spec.primaryDatabaseRef` is PRIMARY -### Patch primary and standby databases in dataguard configuration +### Patch primary and standby databases in Data Guard configuration Databases (both primary and standby) running in you cluster and managed by the Oracle Database operator can be patched or rolled back between release updates of the same major release. While patching databases configured with the dataguard broker you need to first patch the Primary database followed by seconday/standby databases in any order. @@ -782,9 +784,9 @@ kubectl --type=merge -p '{"spec":{"image":{"pullFrom":"patched-image:tag","pullS ``` -### Delete a database in dataguard configuration +### Delete a database configured for Data Guard -To delete a standby database in a dataguard configuration, delete the dataguardbroker resource first followed by the standby database +To delete a standby or primary database configured for Data Guard, delete the dataguardbroker resource first followed by the standby databases and finally the primary database #### Delete DataguardBroker Resource ```sh From 2e7b98ccc5b1df83520705241118f026e1a14205 Mon Sep 17 00:00:00 2001 From: Mohammed Yunus Date: Wed, 28 Jun 2023 10:34:13 +0000 Subject: [PATCH 570/628] Update PREREQUISITES.md --- docs/sidb/PREREQUISITES.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/sidb/PREREQUISITES.md b/docs/sidb/PREREQUISITES.md index d442c11a..501fd632 100644 --- a/docs/sidb/PREREQUISITES.md +++ b/docs/sidb/PREREQUISITES.md @@ -6,7 +6,7 @@ To deploy Oracle Database Operator for Kubernetes on Oracle Docker images, compl Build Single Instance Database Docker Images from source, following the instructions at [https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance), or use the pre-built images available at [https://container-registry.oracle.com](https://container-registry.oracle.com) by signing in and accepting the required license agreement. - Oracle Database Releases Supported: Oracle Database 19c Enterprise Edition or Standard Edition, and later releases. Oracle Database 21.3 Express Edition. + Oracle Database Releases Supported: Enterprise and Standard Edition for Oracle Database 19c, and later releases. Express Edition for Oracle Database 21.3 only. Oracle Database Free 23.2.0 and later Free releases Build Oracle REST Data Service Docker Images from source following the instructions at [https://github.com/oracle/docker-images/tree/main/OracleRestDataServices](https://github.com/oracle/docker-images/tree/main/OracleRestDataServices). Supported Oracle REST Data Service version is 21.4.2 @@ -19,9 +19,9 @@ To deploy Oracle Database Operator for Kubernetes on Oracle Docker images, compl * ### Minikube Cluster Environment - By default, Minikube creates a node with 2GB RAM, 2 CPUs, and 20GB disk space when a cluster is created using `minikube start` command. However, these resources (particularly disk space and RAM) may not be sufficient for running and managing Oracle Database using the OraOperator. It is recommended to have larger RAM and disk space for better performance. For example, the following command creates a Minikube cluster with 6GB RAM and 50GB disk space for the Minikube VM: + By default, Minikube creates a node with 2GB RAM, 2 CPUs, and 20GB disk space when a cluster is created using `minikube start` command. However, these resources (particularly disk space and RAM) may not be sufficient for running and managing Oracle Database using the OraOperator. It is recommended to have larger RAM and disk space for better performance. For example, the following command creates a Minikube cluster with 8GB RAM and 100GB disk space for the Minikube VM: ``` - minikube start --memory=6g --disk-size=50g + minikube start --memory=8g --disk-size=100g ``` From ad16432360d8a6892c7dfd8439c3e74dcf29506b Mon Sep 17 00:00:00 2001 From: Mohammed Yunus Date: Wed, 28 Jun 2023 10:51:27 +0000 Subject: [PATCH 571/628] Update PREREQUISITES.md --- docs/sidb/PREREQUISITES.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/sidb/PREREQUISITES.md b/docs/sidb/PREREQUISITES.md index 501fd632..6315d308 100644 --- a/docs/sidb/PREREQUISITES.md +++ b/docs/sidb/PREREQUISITES.md @@ -6,7 +6,7 @@ To deploy Oracle Database Operator for Kubernetes on Oracle Docker images, compl Build Single Instance Database Docker Images from source, following the instructions at [https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance), or use the pre-built images available at [https://container-registry.oracle.com](https://container-registry.oracle.com) by signing in and accepting the required license agreement. - Oracle Database Releases Supported: Enterprise and Standard Edition for Oracle Database 19c, and later releases. Express Edition for Oracle Database 21.3 only. Oracle Database Free 23.2.0 and later Free releases + Oracle Database Releases Supported: Enterprise and Standard Edition for Oracle Database 19c, and later releases. Express Edition for Oracle Database 21.3.0 only. Oracle Database Free 23.2.0 and later Free releases Build Oracle REST Data Service Docker Images from source following the instructions at [https://github.com/oracle/docker-images/tree/main/OracleRestDataServices](https://github.com/oracle/docker-images/tree/main/OracleRestDataServices). Supported Oracle REST Data Service version is 21.4.2 From a70268059b306f5285b546bd9ef0107db6349bf3 Mon Sep 17 00:00:00 2001 From: kuassi_mensah Date: Wed, 28 Jun 2023 14:02:43 +0000 Subject: [PATCH 572/628] Replaced operator with controller. --- docs/observability/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/observability/README.md b/docs/observability/README.md index f369a779..fd7916d5 100644 --- a/docs/observability/README.md +++ b/docs/observability/README.md @@ -5,7 +5,7 @@ Cloud Native Observability Controller for Oracle Databases, which automates the deployment of the database observability exporter, creates Service Monitors for Prometheus and provides a configmap containing a JSON for a sample dashboard in Grafana. The following sections explain the setup and functionality -of the operator +of the controller - [Requirements](#requirements) - [The Observability Custom Resource](#observability-custom-resource) From 0bddb69c10393a45b68cc4887078bfcef9a9e059 Mon Sep 17 00:00:00 2001 From: norman_japheth_aberin Date: Wed, 28 Jun 2023 14:36:00 +0000 Subject: [PATCH 573/628] Updated controller lifecycle operations --- README.md | 2 +- docs/observability/README.md | 12 ++++++++---- 2 files changed, 9 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 950c76b3..58962a00 100644 --- a/README.md +++ b/README.md @@ -30,7 +30,7 @@ This release of Oracle Database Operator for Kubernetes (the operator) supports * Oracle Multitenant Database: Bind to a CDB, Create a  PDB, Plug a  PDB, Unplug a PDB, Delete a PDB, Clone a PDB, Open/Close a PDB * Oracle Base Database Cloud Service (BDBCS): Provision, Bind, Scale Up/Down, Liveness Probe, Manual Backup * Oracle Data Guard: Provision a Standby for the SIDB resource and setup a Data Guard configuration to enable manual switch over -* Cloud Native Database Observer: Provide Quick/Detailed Database Status +* Cloud Native Database Observer: Deploy DB exporter resources, Undeploy DB exporter resources, Patch DB exporter configuration The upcoming releases will support new configurations, operations and capabilities. diff --git a/docs/observability/README.md b/docs/observability/README.md index fd7916d5..a327d64e 100644 --- a/docs/observability/README.md +++ b/docs/observability/README.md @@ -8,10 +8,14 @@ in Grafana. The following sections explain the setup and functionality of the controller - [Requirements](#requirements) - - [The Observability Custom Resource](#observability-custom-resource) - - [Create](#create) - - [Delete](#delete) - - [Patch](#patch) +- [The Observability Custom Resource](#observability-custom-resource) + - [List](#observability-list) + - [Quick Status](#quick-status) + - [Detailed Status](#detailed-status) +- [Create](#create) +- [Delete](#delete) +- [Patch](#patch) +- [Troubleshooting](#debugging-and-troubleshooting) ## Requirements Oracle recommends that you follow the [requirements](./REQUIREMENTS.md). From 2f245eb0dc54a584940100debe0240f389e2e30b Mon Sep 17 00:00:00 2001 From: Mohammed Yunus Date: Wed, 28 Jun 2023 15:24:31 +0000 Subject: [PATCH 574/628] Update README.md --- docs/sidb/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 492eb6b4..be389c9f 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -24,7 +24,7 @@ Oracle Database Operator for Kubernetes (`OraOperator`) includes the Single Inst * [Setup Database with LoadBalancer](#setup-database-with-loadbalancer) * [Enabling TCPS Connections](#enabling-tcps-connections) * [Specifying Custom Ports](#specifying-custom-ports) - * [Setup Data Guard Configuration for a Single Instance Database](#setup-data-guard-configuration-for-a-single-instance-database) + * [Setup Data Guard Configuration for a Single Instance Database (Preview status)](#setup-data-guard-configuration-for-a-single-instance-database-preview-status) * [Create a Standby Database](#create-a-standby-database) * [Add the Databases in Data Guard Configuration](#add-the-databases-in-data-guard-configuration) * [Delete a database configured for Data Guard](#delete-a-database-configured-for-data-guard) @@ -587,7 +587,7 @@ In case of `NodePort` service, `listenerPort`, and `tcpsListenerPort` will be th - If TCPS connections are enabled, and `listenerPort` is commented/removed in the [config/samples/sidb/singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml) file, only TCPS endpoint will be exposed. - If LoadBalancer is enabled, and either `listenerPort` or `tcpsListenerPort` is changed, then it takes some time to complete the work requests (drain existing backend sets and create new ones). In this time, the database connectivity is broken. Although, SingleInstanceDatabase and LoadBalancer remain in the healthy state, you can check the progress of the work requests by logging into the cloud provider's console and checking the corresponding LoadBalancer. -### Setup Data Guard Configuration for a Single Instance Database +### Setup Data Guard Configuration for a Single Instance Database (Preview status) ### Create a Standby Database From 9f9ba873cfddb6bced0de310da1df37449eda312 Mon Sep 17 00:00:00 2001 From: douglas_williams Date: Tue, 4 Jul 2023 05:59:16 +0000 Subject: [PATCH 575/628] Updates for style and format updates as requested 29-Jun --- docs/multitenant/README.md | 51 +++++++++++++++++++------------------- 1 file changed, 25 insertions(+), 26 deletions(-) diff --git a/docs/multitenant/README.md b/docs/multitenant/README.md index 8b1407a6..724c39e6 100644 --- a/docs/multitenant/README.md +++ b/docs/multitenant/README.md @@ -4,13 +4,13 @@ # Oracle Multitenant Database Controller -> WARNING: Examples with https are located in the usecases directories +> WARNING: Examples with https are located in the use case directories -CDBs and PDBs are the part of the Oracle Database's [Multitenant Architecture](https://docs.oracle.com/en/database/oracle/oracle-database/21/multi/introduction-to-the-multitenant-architecture.html#GUID-AB84D6C9-4BBE-4D36-992F-2BB85739329F). The Multitenant Database Controller is a feature of Oracle DB Operator for Kubernetes (OraOperator) which helps to manage the life cycle of Pluggable Databases (PDBs) in an Oracle Container Database(CDB). +CDBs and PDBs are part of the Oracle Database [Multitenant Architecture](https://docs.oracle.com/en/database/oracle/oracle-database/21/multi/introduction-to-the-multitenant-architecture.html#GUID-AB84D6C9-4BBE-4D36-992F-2BB85739329F). The Multitenant Database Controller is a feature of Oracle DB Operator for Kubernetes (`OraOperator`), which helps to manage the lifecycle of Pluggable Databases (PDBs) in an Oracle Container Database (CDB). -The target CDB (for which the PDB life cycle management is needed) can be running on an multitenant machine and to manage its PDBs, the Oracle DB Operator can run on an multitenant Kubernetes system (For Example: [Oracle Linux Cloud Native Environment or OLCNE](https://docs.oracle.com/en/operating-systems/olcne/)). +The target CDB for which PDB lifecycle management is needed can be running on a machine on-premises. To manage the PDBs of that target CDB, you can run the Oracle DB Operator on a Kubernetes system on-premises (For Example: [Oracle Linux Cloud Native Environment or OLCNE](https://docs.oracle.com/en/operating-systems/olcne/)). -NOTE: The target CDB (for which the PDB life cycle management is needed) **can also** run in a Cloud environment as well (For Example: OCI's [Oracle Base Database Service](https://docs.oracle.com/en-us/iaas/dbcs/doc/bare-metal-and-virtual-machine-db-systems.html)) and to manage its PDBs, the Oracle DB Operator can run on a Kubernetes Cluster running in cloud (For Example: OCI's [Container Engine for Kubernetes or OKE](https://docs.oracle.com/en-us/iaas/Content/ContEng/Concepts/contengoverview.htm#Overview_of_Container_Engine_for_Kubernetes)) +NOTE: The target CDB can also run in a Cloud environment, such as an OCI [Oracle Base Database Service](https://docs.oracle.com/en-us/iaas/dbcs/doc/bare-metal-and-virtual-machine-db-systems.html)). To manage PDBs on the target CDB, the Oracle DB Operator can run on a Kubernetes Cluster running in the cloud, such as OCI's [Container Engine for Kubernetes or OKE](https://docs.oracle.com/en-us/iaas/Content/ContEng/Concepts/contengoverview.htm#Overview_of_Container_Engine_for_Kubernetes)) @@ -18,7 +18,7 @@ NOTE: The target CDB (for which the PDB life cycle management is needed) **can a To deploy OraOperator, use this [Oracle Database Operator for Kubernetes](https://github.com/oracle/oracle-database-operator/blob/main/README.md) step-by-step procedure. -After the Oracle Database Operator is deployed, you can see the DB Operator Pods running in the Kubernetes Cluster. As part of the OraOperator deployment, the multitenant Database Controller is deployed and we can see the CRDs (Custom Resource Definition) for CDB and PDB in the list of CRDs. The following output is an example of such a deployment: +After the Oracle Database Operator is deployed, you can see the DB Operator Pods running in the Kubernetes Cluster. As part of the `OraOperator` deployment, the multitenant Database Controller is deployed. You can see the CRDs (Custom Resource Definition) for the CDB and PDBs in the list of CRDs. The following output is an example of such a deployment: ```bash [root@test-server oracle-database-operator]# kubectl get ns NAME STATUS AGE @@ -73,9 +73,7 @@ The following sections explain the setup and functionality of this controller. # Prerequsites to manage PDB Life Cycle using Oracle DB Operator Multitenant Database Controller -Before you want to manage the life cycle of a PDB in a CDB using the Oracle DB Operator Multitenant Database Controller, complete the following steps. - -**CAUTION :** You must make the changes specified in this section before you proceed to the next section. +**CAUTION :** You must complete the following steps before managing the lifecycle of a PDB in a CDB using the Oracle DB Operator Multitenant Database Controller. * [Prepare CDB for PDB Lifecycle Management or PDB-LM](#prepare-cdb-for-pdb-lifecycle-management-pdb-lm) * [Oracle REST Data Service or ORDS Image](#oracle-rest-data-service-ords-image) @@ -86,11 +84,11 @@ Before you want to manage the life cycle of a PDB in a CDB using the Oracle DB O + ## Prepare CDB for PDB Lifecycle Management (PDB-LM) -Pluggable Database management operation is performed in the Container Database (CDB) and it includes create, clone, plug, unplug, delete, modify and map operations. +Pluggable Database (PDB) management operations are performed in the Container Database (CDB). These operations include create, clone, plug, unplug, delete, modify and map operations. -You cannot have an ORDS enabled schema in the container database. To perform the PDB lifecycle management operations, the default CDB administrator credentials must be defined by performing following steps on the target CDB(s): +You cannot have an ORDS-enabled schema in the container database. To perform the PDB lifecycle management operations, you must first use the following steps to define the default CDB administrator credentials on target CDBs: -Create the CDB administrator user and grant the required privileges. In this example, the user is `C##DBAPI_CDB_ADMIN`. However, any suitable common user name can be used. +Create the CDB administrator user, and grant the required privileges. In this example, the user is `C##DBAPI_CDB_ADMIN`. However, any suitable common user name can be used. ```SQL SQL> conn /as sysdba @@ -114,37 +112,38 @@ select username, account_status from dba_users where username in ('ORDS_PUBLIC_U ### Reference Setup: Example of a setup using OCI OKE(Kubernetes Cluster) and a CDB in Cloud (OCI Exadata Database Cluster) -Please refer [here](./provisioning/example_setup_using_oci_oke_cluster.md) for steps to configure a Kubernetes Cluster and a CDB. This example uses an OCI OKE Cluster as the Kubernetes Cluster and a CDB in OCI Exadata Database service. +See this [provisioning example setup](./provisioning/example_setup_using_oci_oke_cluster.md) for steps to configure a Kubernetes Cluster and a CDB. This example uses an OCI OKE Cluster as the Kubernetes Cluster and a CDB in OCI Exadata Database service. + ## Oracle REST Data Service (ORDS) Image - Oracle DB Operator Multitenant Database controller requires the Oracle REST Data Services (ORDS) image for PDB Lifecycle Management in the target CDB. + Oracle DB Operator Multitenant Database controller requires that the Oracle REST Data Services (ORDS) image for PDB Lifecycle Management is present in the target CDB. You can build this image by using the ORDS [Dockerfile](../../../ords/Dockerfile) -Please refer [here](./provisioning/ords_image.md) for the steps to build ORDS Docker Image + For the steps to build the ORDS Docker image, see [ORDS_image](./provisioning/ords_image.md) + ## Kubernetes Secrets - Oracle DB Operator Multitenant Database Controller uses Kubernetes Secrets to store usernames and passwords to manage the life cycle operations of a PDB in the target CDB. In addition to that ,in order to use https protocol, all certificates need to be stored using Kubernests Secret. + Oracle DB Operator Multitenant Database Controller uses Kubernetes Secrets to store usernames and passwords that you must have to manage the lifecycle operations of a PDB in the target CDB. In addition, to use https protocol, all certificates need to be stored using Kubernetes Secret. ### Secrets for CDB CRD - Create a secret file as shown here: [config/samples/multitenant/cdb_secret.yaml](../../config/samples/multitenant/cdb_secret.yaml). Modify this file with the `base64` encoded values of the required passwords for CDB and use it to create the required secrets. + Create a secret file as shown here: [config/samples/multitenant/cdb_secret.yaml](../../config/samples/multitenant/cdb_secret.yaml). Modify this file with the `base64` encoded values of the required passwords for CDB, and use this file to create the required secrets. ```bash kubectl apply -f cdb_secret.yaml ``` - **Note:** In order to get the base64 encoded value for a password, please use the following command like below at the command prompt. The value you get is the base64 encoded value for that password string. + **Note:** To obtain the `base64` encoded value for a password, use the following command: ```bash echo -n "" | base64 ``` + The value that is returned is the base64-encoded value for that password string. - **Note:** On successful creation of the CDB Resource, the CDB secrets would be deleted from the Kubernetes . + **Note:** After successful creation of the CDB Resource, the CDB secrets are deleted from the Kubernetes system . ### Secrets for PDB CRD Create a secret file as shown here: [config/samples/multitenant/pdb_secret.yaml](../../config/samples/multitenant/pdb_secret.yaml). Modify this file with the `base64` encoded values of the required passwords for PDB and use it to create the required secrets. @@ -152,13 +151,13 @@ Please refer [here](./provisioning/ords_image.md) for the steps to build ORDS Do ```bash kubectl apply -f pdb_secret.yaml ``` - **NOTE:** Refer to command provided above to encode the password using base64. + **NOTE:** To encode the password using `base64`, see the command example in the preceding **Secrets for CDB CRD** section. **NOTE:** Don't leave plaintext files containing sensitive data on disk. After loading the Secret, remove the plaintext file or move it to secure storage. ### Secrets for CERTIFICATES -Create certificates and key on your local host and use them to create Kubernet secret +Create the certificates and key on your local host, and use them to create the Kubernetes secret. ```bash genrsa -out ca.key 2048 @@ -178,11 +177,11 @@ kubectl create secret generic db-ca --from-file=ca.crt -n oracle-database-operat + ## Kubernetes CRD for CDB -The Oracle Database Operator Multitenant Controller creates the CDB kind as a custom resource that models a target CDB as a native Kubernetes object. This is used only to create Pods to connect to the target CDB to perform PDB-LM operations. These CDB resources can be scaled up and down based on the expected load using replicas. Each CDB resource follows the CDB CRD as defined here: [config/crd/bases/database.oracle.com_cdbs.yaml](../../config/crd/bases/database.oracle.com_cdbs.yaml) +The Oracle Database Operator Multitenant Controller creates the CDB kind as a custom resource that models a target CDB as a native Kubernetes object. This kind is used only to create Pods to connect to the target CDB to perform PDB-LM operations. These CDB resources can be scaled, based on the expected load, using replicas. Each CDB resource follows the CDB CRD as defined here: [config/crd/bases/database.oracle.com_cdbs.yaml](../../config/crd/bases/database.oracle.com_cdbs.yaml) -To create a CDB CRD, a sample .yaml file is available here: [config/samples/multitenant/cdb.yaml](../../config/samples/multitenant/cdb.yaml) +To create a CDB CRD, see this example `.yaml` file: [config/samples/multitenant/cdb.yaml](../../config/samples/multitenant/cdb.yaml) -**Note:** The password and username fields in this *cdb.yaml* yaml are Kubernetes Secrets created earlier. Please see the section [Kubernetes Secrets](https://kubernetes.io/docs/concepts/configuration/secret/) for more information. Please see [Kubernetes Private Registry Documenation]( https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/) for creating secrets for pulling images from docker private registry. +**Note:** The password and username fields in this *cdb.yaml* Yaml are the Kubernetes Secrets created earlier in this procedure. For more information, see the section [Kubernetes Secrets](https://kubernetes.io/docs/concepts/configuration/secret/). To understand more about creating secrets for pulling images from a Docker private registry, see [Kubernetes Private Registry Documenation]( https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/). 1. [Use Case: Create a CDB CRD Resource](./provisioning/cdb_crd_resource.md) 2. [Use Case: Add another replica to an existing CDB CRD Resource](./provisioning/add_replica.md) @@ -196,7 +195,7 @@ To create a PDB CRD Resource, a sample .yaml file is available here: [config/sam # Use Cases for PDB Lifecycle Management Operations using Oracle DB Operator Multitenant Controller -Using Oracle DB Operator Multitenant Controller, you can perform the following PDB-LM operations: CREATE, CLONE, MODIFY, DELETE, UNPLUG, PLUG. +Using the Oracle DB Operator Multitenant Controller, you can perform the following PDB-LM operations: CREATE, CLONE, MODIFY, DELETE, UNPLUG, PLUG. 1. [Create PDB](./provisioning/create_pdb.md) 2. [Clone PDB](./provisioning/clone_pdb.md) @@ -208,9 +207,9 @@ Using Oracle DB Operator Multitenant Controller, you can perform the following P ## Validation and Errors -Please check [here](./provisioning/validation_error.md) for the details to look for any validation error. +To see how to look for any validation errors, see [validation_error](./provisioning/validation_error.md). ## Known issues -Please refer [here](./provisioning/known_issues.md) for the known issues related to Oracle DB Operator Multitenant Controller. +To find out about known issue related to Oracle DB Operator Multitenant Controller, see [known_issues](./provisioning/known_issues.md). From 9ff2abdf4df7c71f3e73e7a1a3650e751b883977 Mon Sep 17 00:00:00 2001 From: racpack Date: Tue, 4 Jul 2023 17:18:35 +0000 Subject: [PATCH 576/628] Revert observability controller branch --- PROJECT | 9 - README.md | 4 - .../v1alpha1/databaseobserver_types.go | 138 ----- .../v1alpha1/groupversion_info.go | 58 -- .../v1alpha1/zz_generated.deepcopy.go | 266 --------- commons/observability/constants.go | 92 ---- commons/observability/dashboard.go | 276 ---------- commons/observability/panels.go | 328 ------------ commons/observability/utils.go | 198 ------- ...vability.oracle.com_databaseobservers.yaml | 225 -------- config/crd/kustomization.yaml | 3 - .../cainjection_in_databaseobservers.yaml | 8 - .../patches/webhook_in_databaseobservers.yaml | 17 - config/rbac/databaseobserver_editor_role.yaml | 24 - config/rbac/databaseobserver_viewer_role.yaml | 20 - config/rbac/role.yaml | 71 --- config/samples/kustomization.yaml | 1 - ...servability_v1alpha1_databaseobserver.yaml | 52 -- .../databaseobserver_controller.go | 277 ---------- .../databaseobserver_resource.go | 505 ------------------ controllers/observability/suite_test.go | 103 ---- docs/observability/README.md | 198 ------- docs/observability/REQUIREMENTS.md | 56 -- go.mod | 12 +- go.sum | 44 +- main.go | 22 +- 26 files changed, 20 insertions(+), 2987 deletions(-) delete mode 100644 apis/observability/v1alpha1/databaseobserver_types.go delete mode 100644 apis/observability/v1alpha1/groupversion_info.go delete mode 100644 apis/observability/v1alpha1/zz_generated.deepcopy.go delete mode 100644 commons/observability/constants.go delete mode 100644 commons/observability/dashboard.go delete mode 100644 commons/observability/panels.go delete mode 100644 commons/observability/utils.go delete mode 100644 config/crd/bases/observability.oracle.com_databaseobservers.yaml delete mode 100644 config/crd/patches/cainjection_in_databaseobservers.yaml delete mode 100644 config/crd/patches/webhook_in_databaseobservers.yaml delete mode 100644 config/rbac/databaseobserver_editor_role.yaml delete mode 100644 config/rbac/databaseobserver_viewer_role.yaml delete mode 100644 config/samples/observability/observability_v1alpha1_databaseobserver.yaml delete mode 100644 controllers/observability/databaseobserver_controller.go delete mode 100644 controllers/observability/databaseobserver_resource.go delete mode 100644 controllers/observability/suite_test.go delete mode 100644 docs/observability/README.md delete mode 100644 docs/observability/REQUIREMENTS.md diff --git a/PROJECT b/PROJECT index 5cbb036c..c402ecfd 100644 --- a/PROJECT +++ b/PROJECT @@ -136,13 +136,4 @@ resources: defaulting: true validation: true webhookVersion: v1beta1 -- api: - crdVersion: v1beta1 - namespaced: true - controller: true - domain: oracle.com - group: observability - kind: DatabaseObserver - path: github.com/oracle/oracle-database-operator/apis/observability/v1alpha1 - version: v1alpha1 version: "3" diff --git a/README.md b/README.md index 58962a00..fb4874cf 100644 --- a/README.md +++ b/README.md @@ -15,7 +15,6 @@ In this v1.0.0 production release, `OraOperator` supports the following database * Oracle Multitenant Databases (CDB/PDBs) * Oracle Base Database Cloud Service (BDBCS) * Oracle Data Guard (Preview status) -* Cloud Native Database Observer (Preview status) Oracle will continue to extend `OraOperator` to support additional Oracle Database configurations. @@ -30,7 +29,6 @@ This release of Oracle Database Operator for Kubernetes (the operator) supports * Oracle Multitenant Database: Bind to a CDB, Create a  PDB, Plug a  PDB, Unplug a PDB, Delete a PDB, Clone a PDB, Open/Close a PDB * Oracle Base Database Cloud Service (BDBCS): Provision, Bind, Scale Up/Down, Liveness Probe, Manual Backup * Oracle Data Guard: Provision a Standby for the SIDB resource and setup a Data Guard configuration to enable manual switch over -* Cloud Native Database Observer: Deploy DB exporter resources, Undeploy DB exporter resources, Patch DB exporter configuration The upcoming releases will support new configurations, operations and capabilities. @@ -98,7 +96,6 @@ The quickstarts are designed for specific database configurations: * [Containerized Oracle Sharded Database](./docs/sharding/README.md) * [Oracle Multitenant Database](./docs/multitenant/README.md) * [Oracle Base Database Cloud Service (BDBCS)](./docs/dbcs/README.md) -* [Cloud Native Database Observer](./docs/observability/README.md) YAML file templates are available under [`/config/samples`](./config/samples/). You can copy and edit these template files to configure them for your use cases. @@ -122,7 +119,6 @@ YAML file templates are available under [`/config/samples`](./config/samples/). kubectl delete cdb.database.oracle.com --all -n kubectl delete pdb.database.oracle.com --all -n kubectl delete dataguardbrokers.database.oracle.com --all -n - kubectl delete databaseobservers.observability.oracle.com --all -n ``` After all CRD instances are deleted, it is safe to remove the CRDs, APIServices and operator deployment. To remove these files, use the following command: diff --git a/apis/observability/v1alpha1/databaseobserver_types.go b/apis/observability/v1alpha1/databaseobserver_types.go deleted file mode 100644 index a16d6043..00000000 --- a/apis/observability/v1alpha1/databaseobserver_types.go +++ /dev/null @@ -1,138 +0,0 @@ -/* -** Copyright (c) 2022 Oracle and/or its affiliates. -** -** The Universal Permissive License (UPL), Version 1.0 -** -** Subject to the condition set forth below, permission is hereby granted to any -** person obtaining a copy of this software, associated documentation and/or data -** (collectively the "Software"), free of charge and under any and all copyright -** rights in the Software, and any and all patent rights owned or freely -** licensable by each licensor hereunder covering either (i) the unmodified -** Software as contributed to or provided by such licensor, or (ii) the Larger -** Works (as defined below), to deal in both -** -** (a) the Software, and -** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if -** one is included with the Software (each a "Larger Work" to which the Software -** is contributed by such licensors), -** -** without restriction, including without limitation the rights to copy, create -** derivative works of, display, perform, and distribute the Software and make, -** use, sell, offer for sale, import, export, have made, and have sold the -** Software and the Larger Work(s), and to sublicense the foregoing rights on -** either these or other terms. -** -** This license is subject to the following condition: -** The above copyright notice and either this complete permission notice or at -** a minimum a reference to the UPL must be included in all copies or -** substantial portions of the Software. -** -** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -** SOFTWARE. - */ - -package v1alpha1 - -import ( - metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" -) - -type StatusEnum string - -// DatabaseObserverSpec defines the desired state of DatabaseObserver -type DatabaseObserverSpec struct { - // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster - // Important: Run "make" to regenerate code after modifying this file - - Database DatabaseObserverDatabase `json:"database,omitempty"` - Exporter DatabaseObserverExporterConfig `json:"exporter,omitempty"` - Prometheus PrometheusConfig `json:"prometheus,omitempty"` - Replicas int32 `json:"replicas,omitempty"` -} - -// DatabaseObserverDatabase defines the database details used for DatabaseObserver -type DatabaseObserverDatabase struct { - DBName string `json:"dbName,omitempty"` - //DBUser DBSecret `json:"dbUser,omitempty"` - //DBServiceName DBSecret `json:"dbServiceName,omitempty"` - DBPassword DBSecret `json:"dbPassword,omitempty"` - DBWallet DBSecret `json:"dbWallet,omitempty"` - DBConnectionString DBSecret `json:"dbConnectionString,omitempty"` -} - -// DatabaseObserverExporterConfig defines the configuration details related to the exporters of DatabaseObserver -type DatabaseObserverExporterConfig struct { - ExporterImage string `json:"image,omitempty"` - ExporterConfig DatabaseObserverConfigMap `json:"configuration,omitempty"` - Service DatabaseObserverService `json:"service,omitempty"` -} - -// DatabaseObserverService defines the exporter service component of DatabaseObserver -type DatabaseObserverService struct { - Port int32 `json:"port,omitempty"` -} - -// PrometheusConfig defines the generated resources for Prometheus -type PrometheusConfig struct { - Labels map[string]string `json:"labels,omitempty"` - Port string `json:"port,omitempty"` -} - -type DBSecret struct { - Key string `json:"key,omitempty"` - SecretName string `json:"secret,omitempty"` - VaultOCID string `json:"vaultOCID,omitempty"` - VaultSecretName string `json:"vaultSecretName,omitempty"` -} - -type DatabaseObserverConfigMap struct { - Configmap ConfigMapDetails `json:"configmap,omitempty"` -} - -// ConfigMapDetails defines the configmap name -type ConfigMapDetails struct { - Key string `json:"key,omitempty"` - Name string `json:"configmapName,omitempty"` -} - -// DatabaseObserverStatus defines the observed state of DatabaseObserver -type DatabaseObserverStatus struct { - // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster - // Important: Run "make" to regenerate code after modifying this file - Conditions []metav1.Condition `json:"conditions"` - Status string `json:"status,omitempty"` - ExporterConfig string `json:"exporterConfig"` - Replicas int `json:"replicas,omitempty"` -} - -//+kubebuilder:object:root=true -//+kubebuilder:subresource:status - -// DatabaseObserver is the Schema for the databaseobservers API -// +kubebuilder:printcolumn:JSONPath=".status.exporterConfig",name="ExporterConfig",type=string -// +kubebuilder:printcolumn:JSONPath=".status.status",name="Status",type=string -type DatabaseObserver struct { - metav1.TypeMeta `json:",inline"` - metav1.ObjectMeta `json:"metadata,omitempty"` - - Spec DatabaseObserverSpec `json:"spec,omitempty"` - Status DatabaseObserverStatus `json:"status,omitempty"` -} - -//+kubebuilder:object:root=true - -// DatabaseObserverList contains a list of DatabaseObserver -type DatabaseObserverList struct { - metav1.TypeMeta `json:",inline"` - metav1.ListMeta `json:"metadata,omitempty"` - Items []DatabaseObserver `json:"items"` -} - -func init() { - SchemeBuilder.Register(&DatabaseObserver{}, &DatabaseObserverList{}) -} diff --git a/apis/observability/v1alpha1/groupversion_info.go b/apis/observability/v1alpha1/groupversion_info.go deleted file mode 100644 index 304840f4..00000000 --- a/apis/observability/v1alpha1/groupversion_info.go +++ /dev/null @@ -1,58 +0,0 @@ -/* -** Copyright (c) 2022 Oracle and/or its affiliates. -** -** The Universal Permissive License (UPL), Version 1.0 -** -** Subject to the condition set forth below, permission is hereby granted to any -** person obtaining a copy of this software, associated documentation and/or data -** (collectively the "Software"), free of charge and under any and all copyright -** rights in the Software, and any and all patent rights owned or freely -** licensable by each licensor hereunder covering either (i) the unmodified -** Software as contributed to or provided by such licensor, or (ii) the Larger -** Works (as defined below), to deal in both -** -** (a) the Software, and -** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if -** one is included with the Software (each a "Larger Work" to which the Software -** is contributed by such licensors), -** -** without restriction, including without limitation the rights to copy, create -** derivative works of, display, perform, and distribute the Software and make, -** use, sell, offer for sale, import, export, have made, and have sold the -** Software and the Larger Work(s), and to sublicense the foregoing rights on -** either these or other terms. -** -** This license is subject to the following condition: -** The above copyright notice and either this complete permission notice or at -** a minimum a reference to the UPL must be included in all copies or -** substantial portions of the Software. -** -** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -** SOFTWARE. - */ - -// Package v1alpha1 contains API Schema definitions for the observability v1alpha1 API group -// +kubebuilder:object:generate=true -// +groupName=observability.oracle.com -package v1alpha1 - -import ( - "k8s.io/apimachinery/pkg/runtime/schema" - "sigs.k8s.io/controller-runtime/pkg/scheme" -) - -var ( - // GroupVersion is group version used to register these objects - GroupVersion = schema.GroupVersion{Group: "observability.oracle.com", Version: "v1alpha1"} - - // SchemeBuilder is used to add go types to the GroupVersionKind scheme - SchemeBuilder = &scheme.Builder{GroupVersion: GroupVersion} - - // AddToScheme adds the types in this group-version to the given scheme. - AddToScheme = SchemeBuilder.AddToScheme -) diff --git a/apis/observability/v1alpha1/zz_generated.deepcopy.go b/apis/observability/v1alpha1/zz_generated.deepcopy.go deleted file mode 100644 index 3c298bdd..00000000 --- a/apis/observability/v1alpha1/zz_generated.deepcopy.go +++ /dev/null @@ -1,266 +0,0 @@ -//go:build !ignore_autogenerated -// +build !ignore_autogenerated - -/* -** Copyright (c) 2022 Oracle and/or its affiliates. -** -** The Universal Permissive License (UPL), Version 1.0 -** -** Subject to the condition set forth below, permission is hereby granted to any -** person obtaining a copy of this software, associated documentation and/or data -** (collectively the "Software"), free of charge and under any and all copyright -** rights in the Software, and any and all patent rights owned or freely -** licensable by each licensor hereunder covering either (i) the unmodified -** Software as contributed to or provided by such licensor, or (ii) the Larger -** Works (as defined below), to deal in both -** -** (a) the Software, and -** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if -** one is included with the Software (each a "Larger Work" to which the Software -** is contributed by such licensors), -** -** without restriction, including without limitation the rights to copy, create -** derivative works of, display, perform, and distribute the Software and make, -** use, sell, offer for sale, import, export, have made, and have sold the -** Software and the Larger Work(s), and to sublicense the foregoing rights on -** either these or other terms. -** -** This license is subject to the following condition: -** The above copyright notice and either this complete permission notice or at -** a minimum a reference to the UPL must be included in all copies or -** substantial portions of the Software. -** -** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -** SOFTWARE. - */ - -// Code generated by controller-gen. DO NOT EDIT. - -package v1alpha1 - -import ( - "k8s.io/apimachinery/pkg/apis/meta/v1" - runtime "k8s.io/apimachinery/pkg/runtime" -) - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *ConfigMapDetails) DeepCopyInto(out *ConfigMapDetails) { - *out = *in -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ConfigMapDetails. -func (in *ConfigMapDetails) DeepCopy() *ConfigMapDetails { - if in == nil { - return nil - } - out := new(ConfigMapDetails) - in.DeepCopyInto(out) - return out -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *DBSecret) DeepCopyInto(out *DBSecret) { - *out = *in -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DBSecret. -func (in *DBSecret) DeepCopy() *DBSecret { - if in == nil { - return nil - } - out := new(DBSecret) - in.DeepCopyInto(out) - return out -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *DatabaseObserver) DeepCopyInto(out *DatabaseObserver) { - *out = *in - out.TypeMeta = in.TypeMeta - in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) - in.Spec.DeepCopyInto(&out.Spec) - in.Status.DeepCopyInto(&out.Status) -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserver. -func (in *DatabaseObserver) DeepCopy() *DatabaseObserver { - if in == nil { - return nil - } - out := new(DatabaseObserver) - in.DeepCopyInto(out) - return out -} - -// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. -func (in *DatabaseObserver) DeepCopyObject() runtime.Object { - if c := in.DeepCopy(); c != nil { - return c - } - return nil -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *DatabaseObserverConfigMap) DeepCopyInto(out *DatabaseObserverConfigMap) { - *out = *in - out.Configmap = in.Configmap -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserverConfigMap. -func (in *DatabaseObserverConfigMap) DeepCopy() *DatabaseObserverConfigMap { - if in == nil { - return nil - } - out := new(DatabaseObserverConfigMap) - in.DeepCopyInto(out) - return out -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *DatabaseObserverDatabase) DeepCopyInto(out *DatabaseObserverDatabase) { - *out = *in - out.DBPassword = in.DBPassword - out.DBWallet = in.DBWallet - out.DBConnectionString = in.DBConnectionString -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserverDatabase. -func (in *DatabaseObserverDatabase) DeepCopy() *DatabaseObserverDatabase { - if in == nil { - return nil - } - out := new(DatabaseObserverDatabase) - in.DeepCopyInto(out) - return out -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *DatabaseObserverExporterConfig) DeepCopyInto(out *DatabaseObserverExporterConfig) { - *out = *in - out.ExporterConfig = in.ExporterConfig - out.Service = in.Service -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserverExporterConfig. -func (in *DatabaseObserverExporterConfig) DeepCopy() *DatabaseObserverExporterConfig { - if in == nil { - return nil - } - out := new(DatabaseObserverExporterConfig) - in.DeepCopyInto(out) - return out -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *DatabaseObserverList) DeepCopyInto(out *DatabaseObserverList) { - *out = *in - out.TypeMeta = in.TypeMeta - in.ListMeta.DeepCopyInto(&out.ListMeta) - if in.Items != nil { - in, out := &in.Items, &out.Items - *out = make([]DatabaseObserver, len(*in)) - for i := range *in { - (*in)[i].DeepCopyInto(&(*out)[i]) - } - } -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserverList. -func (in *DatabaseObserverList) DeepCopy() *DatabaseObserverList { - if in == nil { - return nil - } - out := new(DatabaseObserverList) - in.DeepCopyInto(out) - return out -} - -// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. -func (in *DatabaseObserverList) DeepCopyObject() runtime.Object { - if c := in.DeepCopy(); c != nil { - return c - } - return nil -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *DatabaseObserverService) DeepCopyInto(out *DatabaseObserverService) { - *out = *in -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserverService. -func (in *DatabaseObserverService) DeepCopy() *DatabaseObserverService { - if in == nil { - return nil - } - out := new(DatabaseObserverService) - in.DeepCopyInto(out) - return out -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *DatabaseObserverSpec) DeepCopyInto(out *DatabaseObserverSpec) { - *out = *in - out.Database = in.Database - out.Exporter = in.Exporter - in.Prometheus.DeepCopyInto(&out.Prometheus) -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserverSpec. -func (in *DatabaseObserverSpec) DeepCopy() *DatabaseObserverSpec { - if in == nil { - return nil - } - out := new(DatabaseObserverSpec) - in.DeepCopyInto(out) - return out -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *DatabaseObserverStatus) DeepCopyInto(out *DatabaseObserverStatus) { - *out = *in - if in.Conditions != nil { - in, out := &in.Conditions, &out.Conditions - *out = make([]v1.Condition, len(*in)) - for i := range *in { - (*in)[i].DeepCopyInto(&(*out)[i]) - } - } -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserverStatus. -func (in *DatabaseObserverStatus) DeepCopy() *DatabaseObserverStatus { - if in == nil { - return nil - } - out := new(DatabaseObserverStatus) - in.DeepCopyInto(out) - return out -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *PrometheusConfig) DeepCopyInto(out *PrometheusConfig) { - *out = *in - if in.Labels != nil { - in, out := &in.Labels, &out.Labels - *out = make(map[string]string, len(*in)) - for key, val := range *in { - (*out)[key] = val - } - } -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PrometheusConfig. -func (in *PrometheusConfig) DeepCopy() *PrometheusConfig { - if in == nil { - return nil - } - out := new(PrometheusConfig) - in.DeepCopyInto(out) - return out -} diff --git a/commons/observability/constants.go b/commons/observability/constants.go deleted file mode 100644 index 42e484a8..00000000 --- a/commons/observability/constants.go +++ /dev/null @@ -1,92 +0,0 @@ -package observability - -import "github.com/oracle/oracle-database-operator/apis/observability/v1alpha1" - -const UnknownValue = "UNKNOWN" - -// Observability Status -const ( - StatusObservabilityPending v1alpha1.StatusEnum = "PENDING" - StatusObservabilityError v1alpha1.StatusEnum = "ERROR" - StatusObservabilityReady v1alpha1.StatusEnum = "READY" - StatusObservabilityTerminating v1alpha1.StatusEnum = "TERMINATING" -) - -// ConditionTypes -const ( - PhaseReconcile = "Reconcile" - PhaseControllerValidation = "Validation" - PhaseExportersDeploy = "CreateExportersDeployment" - PhaseExportersDeployValidation = "ValidateExportersDeployment" - PhaseExportersDeployUpdate = "UpdateExportersDeployment" - PhaseExportersSVC = "CreateExporterService" - PhaseExportersServiceMonitor = "CreateExporterServiceMonitor" - PhaseExportersGrafanaConfigMap = "CreateGrafanaConfigMap" - PhaseExportersConfigMap = "CreateExporterConfigurationConfigMap" -) - -// Reason -const ( - ReasonInitStart = "InitializationStarted" - ReasonInitSuccess = "InitializationSucceeded" - ReasonValidationFailure = "ValidationFailed" - ReasonSetFailure = "SetResourceFailed" - ReasonCreateFailure = "CreateResourceFailed" - ReasonCreateSuccess = "CreateResourceSucceeded" - ReasonUpdateSuccess = "UpdateResourceSucceeded" - ReasonUpdateFailure = "UpdateResourceFailed" - ReasonRetrieveFailure = "RetrieveResourceFailed" - ReasonRetrieveSuccess = "RetrieveResourceSucceeded" -) - -// Log Names -const ( - LogReconcile = "LogReconcile" - LogControllerValidation = "LogValidation" - LogExportersDeploy = "LogCreateExportersDeployment" - LogExportersSVC = "LogCreateExporterService" - LogExportersServiceMonitor = "LogCreateExporterServiceMonitor" - LogExportersGrafanaConfigMap = "LogCreateGrafanaConfigMap" - LogExportersConfigMap = "LogCreateExporterConfigurationConfigMap" -) -const ( - DefaultDbUserKey = "user" - DefaultDBServiceNameKey = "sn" - DefaultDBPasswordKey = "password" - DefaultDBConnectionStringKey = "access" - DefaultDBWalletMountPath = "/creds" -) -const ( - DefaultExporterImage = "container-registry.oracle.com/database/observability-exporter:0.1.0" - DefaultServicePort = 9161 - DefaultPrometheusPort = "metrics" - DefaultReplicaCount = 1 - DefaultConfig = "[[metric]]\ncontext = \"sessions\"\nlabels = [\"inst_id\", \"status\", \"type\"]\nmetricsdesc = { value = \"Gauge metric with count of sessions by status and type.\" }\nrequest = '''\nSELECT\n inst_id,\n status,\n type,\n COUNT(*) AS value\nFROM\n gv$session\nGROUP BY\n status,\n type,\n inst_id\n'''\n\n" - - DefaultExporterConfigMountRootPath = "/observability" - DefaultConfigurationConfigmapKey = "config.toml" - DefaultExporterConfigmapPath = "config.toml" - DefaultExporterConfigMountPathFull = "/observability/config.toml" - DefaultGrafanaConfigmapKey = "dashboard.json" - DefaultExporterConfigmapPrefix = "obs-cm-" - DefaultServicemonitorPrefix = "obs-servicemonitor-" -) - -const ( - DefaultLabelKey = "app" - DefaultLabelPrefix = "obs-" - DefaultGrafanaConfigMapNamePrefix = "obs-json-dash-" - DefaultExporterDeploymentPrefix = "obs-deploy-" - DefaultExporterContainerName = "observability-exporter" -) - -// Known environment variables -const ( - EnvVarDataSourceServiceName = "DATA_SOURCE_SERVICENAME" - EnvVarDataSourceUser = "DATA_SOURCE_USER" - EnvVarDataSourcePassword = "DATA_SOURCE_PASSWORD" - EnvVarDataSourceName = "DATA_SOURCE_NAME" - EnvVarDataSourcePwdVaultSecretName = "vault_secret_name" - EnvVarDataSourcePwdVaultId = "vault_id" - EnvVarCustomConfigmap = "CUSTOM_METRICS" -) diff --git a/commons/observability/dashboard.go b/commons/observability/dashboard.go deleted file mode 100644 index 4e138bc1..00000000 --- a/commons/observability/dashboard.go +++ /dev/null @@ -1,276 +0,0 @@ -package observability - -type Annotation struct { - List []List `json:"list"` -} - -type List struct { - BuiltIn int `json:"builtIn"` - Datasource string `json:"datasource"` - Enable bool `json:"enable"` - Hide bool `json:"hide"` - IconColor string `json:"iconColor"` - Limit int `json:"limit"` - Name string `json:"name"` - Type string `json:"type"` -} - -type Legend struct { - AlignAsTable bool `json:"alignAsTable,omitempty"` - Avg bool `json:"avg,omitempty"` - Current bool `json:"current,omitempty"` - Max bool `json:"max,omitempty"` - Min bool `json:"min,omitempty"` - Show bool `json:"show,omitempty"` - Total bool `json:"total,omitempty"` - Values bool `json:"values,omitempty"` - Calcs []string `json:"calcs,omitempty"` - DisplayMode string `json:"displayMode,omitempty"` - Placement string `json:"placement,omitempty"` -} - -type ReduceOptions struct { - Calcs []string `json:"calcs"` - Fields string `json:"fields"` - Values bool `json:"values"` -} - -type Options struct { - Num0 struct { - Text string `json:"text,omitempty"` - } `json:"0,omitempty"` - Num1 struct { - Text string `json:"text"` - } `json:"1,omitempty"` - ColorMode string `json:"colorMode,omitempty"` - GraphMode string `json:"graphMode,omitempty"` - JustifyMode string `json:"justifyMode,omitempty"` - Orientation string `json:"orientation,omitempty"` - ReduceOptions ReduceOptions `json:"reduceOptions,omitempty"` - Text struct{} `json:"text"` - TextMode string `json:"textMode,omitempty"` - AlertThreshold bool `json:"alertThreshold,omitempty"` - Legend Legend `json:"legend,omitempty"` - Tooltip Tooltip `json:"tooltip,omitempty"` -} - -type GridPos struct { - H int `json:"h"` - W int `json:"w"` - X int `json:"x"` - Y int `json:"y"` -} - -type Color struct { - Mode string `json:"mode"` -} - -type Mappings struct { - Options Options `json:"options"` - Type string `json:"type"` -} - -type Thresholds struct { - Mode string `json:"mode"` - Steps []Steps `json:"steps"` -} - -type Steps struct { - Color string `json:"color"` - Value interface{} `json:"value,omitempty"` -} -type Defaults struct { - Color Color `json:"color"` - Mappings []Mappings `json:"mappings"` - Custom Custom `json:"custom"` - Thresholds Thresholds `json:"thresholds"` - Unit string `json:"unit"` -} - -type HideFrom struct { - Legend bool `json:"legend"` - Tooltip bool `json:"tooltip"` - Viz bool `json:"viz"` -} - -type ScaleDistribution struct { - Type string `json:"type"` -} - -type Stacking struct { - Group string `json:"group"` - Mode string `json:"mode"` -} - -type ThresholdStyle struct { - Mode string `json:"mode"` -} - -type Custom struct { - AxisLabel string `json:"axisLabel"` - AxisPlacement string `json:"axisPlacement"` - BarAlignment int `json:"barAlignment"` - DrawStyle string `json:"drawStyle"` - FillOpacity int `json:"fillOpacity"` - GradientMode string `json:"gradientMode"` - HideFrom HideFrom `json:"hideFrom"` - LineInterpolation string `json:"lineInterpolation"` - LineWidth int `json:"lineWidth"` - PointSize int `json:"pointSize"` - ScaleDistribution ScaleDistribution `json:"scaleDistribution"` - ShowPoints string `json:"showPoints"` - SpanNulls bool `json:"spanNulls"` - Stacking Stacking `json:"stacking"` - ThresholdsStyle ThresholdStyle `json:"thresholdsStyle"` -} - -type FieldConfig struct { - Defaults Defaults `json:"defaults"` - Overrides []interface{} `json:"overrides"` -} - -type Targets struct { - Exemplar bool `json:"exemplar"` - Expr string `json:"expr"` - Interval string `json:"interval"` - IntervalFactor int `json:"intervalFactor,omitempty"` - LegendFormat string `json:"legendFormat"` - RefID string `json:"refId"` -} - -type Tooltip struct { - Shared bool `json:"shared,omitempty"` - Sort int `json:"sort,omitempty"` - ValueType string `json:"value_type,omitempty"` - Mode string `json:"mode,omitempty"` -} - -type XAxis struct { - Buckets interface{} `json:"buckets"` - Mode string `json:"mode"` - Name interface{} `json:"name"` - Show bool `json:"show"` - Values []interface{} `json:"values"` -} - -type YAxis struct { - Align bool `json:"align"` - AlignLevel interface{} `json:"alignLevel"` -} - -type YAxes struct { - Format string `json:"format"` - Label interface{} `json:"label"` - LogBase int `json:"logBase"` - Max interface{} `json:"max"` - Min interface{} `json:"min"` - Show bool `json:"show"` -} - -type Panels struct { - Collapsed bool `json:"collapsed,omitempty"` - Datasource interface{} `json:"datasource"` - GridPos GridPos `json:"gridPos"` - ID int `json:"id"` - Panels []interface{} `json:"panels,omitempty"` - Title string `json:"title"` - Type string `json:"type"` - CacheTimeout interface{} `json:"cacheTimeout,omitempty"` - FieldConfig FieldConfig `json:"fieldConfig,omitempty"` - Interval interface{} `json:"interval,omitempty"` - Links []interface{} `json:"links,omitempty"` - MaxDataPoints int `json:"maxDataPoints,omitempty"` - Options Options `json:"options,omitempty"` - PluginVersion string `json:"pluginVersion,omitempty"` - Targets []Targets `json:"targets,omitempty"` - TimeFrom interface{} `json:"timeFrom,omitempty"` - TimeShift interface{} `json:"timeShift,omitempty"` - Description string `json:"description,omitempty"` - AliasColors struct { - } `json:"aliasColors,omitempty"` - Bars bool `json:"bars,omitempty"` - DashLength int `json:"dashLength,omitempty"` - Dashes bool `json:"dashes,omitempty"` - Fill int `json:"fill,omitempty"` - FillGradient int `json:"fillGradient,omitempty"` - HiddenSeries bool `json:"hiddenSeries,omitempty"` - Legend Legend `json:"legend,omitempty"` - Lines bool `json:"lines,omitempty"` - Linewidth int `json:"linewidth,omitempty"` - NullPointMode string `json:"nullPointMode,omitempty"` - Percentage bool `json:"percentage,omitempty"` - Pointradius int `json:"pointradius,omitempty"` - Points bool `json:"points,omitempty"` - Renderer string `json:"renderer,omitempty"` - SeriesOverrides []interface{} `json:"seriesOverrides,omitempty"` - SpaceLength int `json:"spaceLength,omitempty"` - Stack bool `json:"stack,omitempty"` - SteppedLine bool `json:"steppedLine,omitempty"` - Thresholds []interface{} `json:"thresholds,omitempty"` - TimeRegions []interface{} `json:"timeRegions,omitempty"` - Tooltip Tooltip `json:"tooltip,omitempty"` - Xaxis XAxis `json:"xaxis,omitempty"` - Yaxes []YAxes `json:"yaxes,omitempty"` - Yaxis YAxis `json:"yaxis,omitempty"` - HideTimeOverride bool `json:"hideTimeOverride,omitempty"` - Alert struct { - AlertRuleTags struct { - } `json:"alertRuleTags"` - Conditions []struct { - Evaluator struct { - Params []float64 `json:"params"` - Type string `json:"type"` - } `json:"evaluator"` - Operator struct { - Type string `json:"type"` - } `json:"operator"` - Query struct { - Params []string `json:"params"` - } `json:"query"` - Reducer struct { - Params []interface{} `json:"params"` - Type string `json:"type"` - } `json:"reducer"` - Type string `json:"type"` - } `json:"conditions"` - ExecutionErrorState string `json:"executionErrorState"` - For string `json:"for"` - Frequency string `json:"frequency"` - Handler int `json:"handler"` - Message string `json:"message"` - Name string `json:"name"` - NoDataState string `json:"noDataState"` - Notifications []interface{} `json:"notifications"` - } `json:"alert,omitempty"` - Transformations []interface{} `json:"transformations,omitempty"` -} - -type Dashboard struct { - Annotations Annotation `json:"annotations"` - Description string `json:"description"` - Editable bool `json:"editable"` - GnetID int `json:"gnetId"` - GraphTooltip int `json:"graphTooltip"` - ID int `json:"id"` - Links []interface{} `json:"links"` - Panels []Panels `json:"panels"` - Refresh bool `json:"refresh"` - SchemaVersion int `json:"schemaVersion"` - Style string `json:"style"` - Tags []interface{} `json:"tags"` - Templating struct { - List []interface{} `json:"list"` - } `json:"templating"` - Time struct { - From string `json:"from"` - To string `json:"to"` - } `json:"time"` - Timepicker struct { - RefreshIntervals []string `json:"refresh_intervals"` - TimeOptions []string `json:"time_options"` - } `json:"timepicker"` - Timezone string `json:"timezone"` - Title string `json:"title"` - UID string `json:"uid"` - Version int `json:"version"` -} diff --git a/commons/observability/panels.go b/commons/observability/panels.go deleted file mode 100644 index 70e5626c..00000000 --- a/commons/observability/panels.go +++ /dev/null @@ -1,328 +0,0 @@ -package observability - -func generateEventMesh() Panels { - return Panels{ - Title: "Event Mesh (Message Propagation)", - Type: "row", - Panels: nil, - ID: 84, - Collapsed: true, - Datasource: nil, - GridPos: GridPos{H: 1, W: 24, X: 0, Y: 75}, - } -} - -func generateCPULoad() Panels { - return Panels{ - Title: "CPU Load", - Datasource: "Prometheus", - AliasColors: struct{}{}, - Bars: false, - DashLength: 10, - Dashes: false, - Description: "", - Fill: 1, - FillGradient: 0, - GridPos: GridPos{H: 6, W: 7, X: 0, Y: 57}, - HiddenSeries: false, - ID: 64, - Legend: Legend{ - AlignAsTable: true, - Avg: true, - Current: true, - Max: true, - Min: true, - Show: true, - Total: false, - Values: true, - }, - Lines: true, - Linewidth: 1, - Links: nil, - NullPointMode: "null", - Options: Options{ - AlertThreshold: true, - }, - Percentage: true, - PluginVersion: "8.1.5", - Pointradius: 5, - Points: false, - Renderer: "flot", - SeriesOverrides: nil, - SpaceLength: 10, - Stack: false, - SteppedLine: false, - - Targets: []Targets{{ - Exemplar: true, - Expr: "base_cpu_systemLoadAverage", - Interval: "", - IntervalFactor: 2, - LegendFormat: "CPU Load", - RefID: "A", - }}, - Thresholds: nil, - TimeFrom: nil, - TimeRegions: nil, - TimeShift: nil, - Tooltip: Tooltip{ - Shared: true, - Sort: 0, - ValueType: "individual", - }, - Xaxis: XAxis{ - Buckets: nil, - Mode: "time", - Name: nil, - Show: true, - Values: nil, - }, - Yaxes: []YAxes{{ - Format: "percentunit", - Label: nil, - LogBase: 1, - Max: nil, - Min: nil, - Show: true, - }, { - Format: "short", - Label: nil, - LogBase: 1, - Max: nil, - Min: nil, - Show: false, - }}, - Yaxis: YAxis{ - Align: false, - AlignLevel: nil, - }, - } -} - -func generateDBStatus(dbName string) Panels { - return Panels{ - CacheTimeout: nil, - Datasource: "Prometheus", - FieldConfig: FieldConfig{ - Defaults: Defaults{ - Color: Color{ - Mode: "thresholds", - }, - Mappings: []Mappings{ - { - Options: Options{ - Num0: struct { - Text string `json:"text,omitempty"` - }{Text: "DOWN"}, - Num1: struct { - Text string `json:"text"` - }{Text: "UP"}, - }, - Type: "value", - }, - }, - Thresholds: Thresholds{ - Mode: "absolute", - Steps: []Steps{ - { - Color: "#d44a3a", - Value: nil, - }, - { - Color: "#37872D", - Value: 1, - }, - { - Color: "#FADE2A", - }, - }}, - Unit: "none", - }, - Overrides: nil, - }, - GridPos: GridPos{H: 2, W: 4, X: 12, Y: 49}, - ID: 59, - Interval: nil, - Links: nil, - MaxDataPoints: 100, - Options: Options{ - ColorMode: "background", - GraphMode: "none", - JustifyMode: "auto", - Orientation: "horizontal", - ReduceOptions: ReduceOptions{ - Calcs: []string{"lastNotNull"}, - Fields: "", - Values: false, - }, - Text: struct{}{}, - TextMode: "auto", - }, - PluginVersion: "8.1.5", - Targets: []Targets{ - { - Exemplar: true, - Expr: "oracledb_up{job=\"db-metrics-exporter-" + dbName + "\"}", - Interval: "", - LegendFormat: "", - RefID: "A", - }, - }, - TimeFrom: nil, - TimeShift: nil, - Title: "DB Status", - Transformations: nil, - Type: "stat", - } -} - -func generateDBSessions(dbName string) Panels { - return Panels{ - Title: "DB Sessions", - Type: "timeseries", - Targets: []Targets{{ - Exemplar: true, - Expr: "oracledb_" + dbName + "_sessions_value", - Interval: "", - LegendFormat: "", - RefID: "A", - }}, - ID: 35, - Options: Options{ - Legend: Legend{ - Calcs: nil, - DisplayMode: "list", - Placement: "bottom", - }, - Tooltip: Tooltip{Mode: "single"}, - }, - GridPos: GridPos{H: 8, W: 12, X: 12, Y: 27}, - Datasource: nil, - FieldConfig: FieldConfig{ - Defaults: Defaults{ - Color: Color{Mode: "palette-classic"}, - Custom: Custom{ - AxisLabel: "", - AxisPlacement: "auto", - BarAlignment: 0, - DrawStyle: "line", - FillOpacity: 0, - GradientMode: "none", - HideFrom: HideFrom{ - Legend: false, - Tooltip: false, - Viz: false, - }, - LineInterpolation: "linear", - LineWidth: 1, - PointSize: 5, - ScaleDistribution: ScaleDistribution{ - Type: "linear", - }, - ShowPoints: "auto", - SpanNulls: false, - Stacking: Stacking{ - Group: "A", - Mode: "none", - }, - ThresholdsStyle: ThresholdStyle{ - Mode: "off", - }, - }, - Mappings: nil, - Thresholds: Thresholds{ - Mode: "absolute", - Steps: []Steps{{ - Color: "green", - Value: nil, - }, { - Color: "red", - Value: 80, - }}, - }, - }, - Overrides: nil, - }, - } -} - -func GenerateDashboard(dbName string) *Dashboard { - pEventMesh := generateEventMesh() - pCPULoad := generateCPULoad() - var panels = []Panels{pEventMesh, pCPULoad} - - if dbName != "" { - pDBStatus := generateDBStatus(dbName) - pDBSessions := generateDBSessions(dbName) - panels = append(panels, pDBSessions) - panels = append(panels, pDBStatus) - } - - return &Dashboard{ - Annotations: Annotation{ - List: []List{{ - BuiltIn: 1, - Datasource: "Prometheus", - Enable: true, - Hide: true, - IconColor: "rgba(0, 211, 255, 1)", - Limit: 100, - Name: "Annotations & Alerts", - Type: "dashboard", - }}, - }, - Description: "Monitoring", - Editable: true, - GnetID: 0, - GraphTooltip: 0, - ID: 0, - Links: nil, - Panels: panels, - Refresh: false, - SchemaVersion: 30, - Style: "dark", - Tags: nil, - Templating: struct { - List []interface{} `json:"list"` - }{}, - Time: struct { - From string `json:"from"` - To string `json:"to"` - }{ - From: "now-1m", - To: "now", - }, - Timepicker: struct { - RefreshIntervals []string `json:"refresh_intervals"` - TimeOptions []string `json:"time_options"` - }{ - RefreshIntervals: []string{ - "5s", - "10s", - "30s", - "1m", - "5m", - "15m", - "30m", - "1h", - "2h", - "1d", - }, - TimeOptions: []string{ - "5m", - "15m", - "1h", - "6h", - "12h", - "24h", - "2d", - "7d", - "30d", - }, - }, - Timezone: "browser", - Title: "Dashboard", - UID: "", - Version: 1, - } -} diff --git a/commons/observability/utils.go b/commons/observability/utils.go deleted file mode 100644 index 921ea7f1..00000000 --- a/commons/observability/utils.go +++ /dev/null @@ -1,198 +0,0 @@ -package observability - -import ( - "encoding/json" - apiv1 "github.com/oracle/oracle-database-operator/apis/observability/v1alpha1" - corev1 "k8s.io/api/core/v1" -) - -func GetGrafanaJSONData(api *apiv1.DatabaseObserver) ([]byte, error) { - dbName := api.Spec.Database.DBName - dash := GenerateDashboard(dbName) - j, err := json.Marshal(dash) - if err != nil { - return nil, err - } - return j, nil - -} - -func GetExporterLabels(api *apiv1.DatabaseObserver) map[string]string { - var l = make(map[string]string) - if labels := api.Spec.Prometheus.Labels; labels != nil && len(labels) > 0 { - for k, v := range labels { - l[k] = v - } - l["release"] = "stable" - return l - } - return map[string]string{ - DefaultLabelKey: DefaultLabelPrefix + api.Name, - "release": "stable", - } - -} - -func GetExporterServicePort(api *apiv1.DatabaseObserver) int32 { - if rPort := api.Spec.Exporter.Service.Port; rPort != 0 { - return rPort - } - return int32(DefaultServicePort) -} - -func GetExporterServiceMonitorPort(api *apiv1.DatabaseObserver) string { - if rPort := api.Spec.Prometheus.Port; rPort != "" { - return rPort - } - return DefaultPrometheusPort - -} -func GetExporterDeploymentVolumeMounts(api *apiv1.DatabaseObserver) []corev1.VolumeMount { - - volM := make([]corev1.VolumeMount, 0) - rConfigMountPath := DefaultExporterConfigMountRootPath - - volM = append(volM, corev1.VolumeMount{ - Name: "config-volume", - MountPath: rConfigMountPath, - }) - - // api.Spec.Database.DBWallet.SecretName optional - // if null, consider the database NON-ADB and connect as such - if secretName := api.Spec.Database.DBWallet.SecretName; secretName != "" { - volM = append(volM, corev1.VolumeMount{ - Name: "creds", - MountPath: DefaultDBWalletMountPath, - }) - } - return volM -} - -func GetExporterDeploymentVolumes(api *apiv1.DatabaseObserver) []corev1.Volume { - - vol := make([]corev1.Volume, 0) - cVolumeSourceName := api.Spec.Exporter.ExporterConfig.Configmap.Name - cVolumeSourceKey := api.Spec.Exporter.ExporterConfig.Configmap.Key - rVolumeSourceName := DefaultExporterConfigmapPrefix + api.Name - - // config-volume Volume - var nameToUse = rVolumeSourceName - var keyToUse = DefaultConfigurationConfigmapKey - - if cVolumeSourceName != "" { // override - nameToUse = cVolumeSourceName - } - if cVolumeSourceKey != "" { // override - keyToUse = cVolumeSourceKey - } - cMSource := &corev1.ConfigMapVolumeSource{ - LocalObjectReference: corev1.LocalObjectReference{ - Name: nameToUse, - }, - Items: []corev1.KeyToPath{{ - Key: keyToUse, - Path: DefaultExporterConfigmapPath, - }}, - } - vol = append(vol, corev1.Volume{Name: "config-volume", VolumeSource: corev1.VolumeSource{ConfigMap: cMSource}}) - - // creds Volume - // api.Spec.Database.DBWallet.SecretName optional - // if null, consider the database NON-ADB and connect as such - if secretName := api.Spec.Database.DBWallet.SecretName; secretName != "" { - - vol = append(vol, corev1.Volume{ - Name: "creds", - VolumeSource: corev1.VolumeSource{ - Secret: &corev1.SecretVolumeSource{ - SecretName: secretName, - }, - }, - }) - } - return vol -} - -func GetExporterSelector(api *apiv1.DatabaseObserver) map[string]string { - var s = make(map[string]string) - if labels := api.Spec.Prometheus.Labels; labels != nil && len(labels) > 0 { - for k, v := range labels { - s[k] = v - } - return s - - } - return map[string]string{DefaultLabelKey: DefaultLabelPrefix + api.Name} - -} - -func GetExporterEnvs(api *apiv1.DatabaseObserver) []corev1.EnvVar { - var e = make([]corev1.EnvVar, 0) - - rDBPasswordKey := api.Spec.Database.DBPassword.Key - rDBPasswordName := api.Spec.Database.DBPassword.SecretName - optional := true - - // DEFAULT_METRICS environment variable - e = append(e, corev1.EnvVar{ - Name: "DEFAULT_METRICS", - Value: DefaultExporterConfigMountPathFull, - }) - - // DATA_SOURCE_SERVICENAME environment variable - // DATA_SOURCE_USER environment variable - // DATA_SOURCE_NAME environment variable - - rDBConnStringSKey := api.Spec.Database.DBConnectionString.Key - rDBConnStringSName := api.Spec.Database.DBConnectionString.SecretName - - if rDBConnStringSKey == "" { // overwrite - rDBConnStringSKey = DefaultDBConnectionStringKey - } - dbConnectionString := &corev1.EnvVarSource{ - SecretKeyRef: &corev1.SecretKeySelector{ - Key: rDBConnStringSKey, - LocalObjectReference: corev1.LocalObjectReference{Name: rDBConnStringSName}, - Optional: &optional, - }} - e = append(e, corev1.EnvVar{Name: EnvVarDataSourceName, ValueFrom: dbConnectionString}) - - // DB PASSWORD environment variable - // DATA_SOURCE_PASSWORD environment variable - if rDBPasswordKey == "" { // overwrite - rDBPasswordKey = DefaultDBPasswordKey - } - dbPassword := &corev1.EnvVarSource{ - SecretKeyRef: &corev1.SecretKeySelector{ - Key: rDBPasswordKey, - LocalObjectReference: corev1.LocalObjectReference{Name: rDBPasswordName}, - Optional: &optional, - }} - - e = append(e, corev1.EnvVar{Name: EnvVarDataSourcePassword, ValueFrom: dbPassword}) - - // TNS_ADMIN environment variable - if secretName := api.Spec.Database.DBWallet.SecretName; secretName != "" { - e = append(e, corev1.EnvVar{ - Name: "TNS_ADMIN", - Value: DefaultDBWalletMountPath, - }) - } - - return e -} - -func GetExporterReplicas(api *apiv1.DatabaseObserver) int32 { - if rc := api.Spec.Replicas; rc != 0 { - return rc - } - return int32(DefaultReplicaCount) -} - -func GetExporterImage(api *apiv1.DatabaseObserver) string { - if img := api.Spec.Exporter.ExporterImage; img != "" { - return img - } - return DefaultExporterImage - -} diff --git a/config/crd/bases/observability.oracle.com_databaseobservers.yaml b/config/crd/bases/observability.oracle.com_databaseobservers.yaml deleted file mode 100644 index bb663594..00000000 --- a/config/crd/bases/observability.oracle.com_databaseobservers.yaml +++ /dev/null @@ -1,225 +0,0 @@ - ---- -apiVersion: apiextensions.k8s.io/v1 -kind: CustomResourceDefinition -metadata: - annotations: - controller-gen.kubebuilder.io/version: v0.6.1 - creationTimestamp: null - name: databaseobservers.observability.oracle.com -spec: - group: observability.oracle.com - names: - kind: DatabaseObserver - listKind: DatabaseObserverList - plural: databaseobservers - singular: databaseobserver - scope: Namespaced - versions: - - additionalPrinterColumns: - - jsonPath: .status.exporterConfig - name: ExporterConfig - type: string - - jsonPath: .status.status - name: Status - type: string - name: v1alpha1 - schema: - openAPIV3Schema: - description: DatabaseObserver is the Schema for the databaseobservers API - properties: - apiVersion: - description: 'APIVersion defines the versioned schema of this representation - of an object. Servers should convert recognized schemas to the latest - internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' - type: string - kind: - description: 'Kind is a string value representing the REST resource this - object represents. Servers may infer this from the endpoint the client - submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' - type: string - metadata: - type: object - spec: - description: DatabaseObserverSpec defines the desired state of DatabaseObserver - properties: - database: - description: DatabaseObserverDatabase defines the database details - used for DatabaseObserver - properties: - dbConnectionString: - properties: - key: - type: string - secret: - type: string - type: object - dbName: - type: string - dbPassword: - properties: - key: - type: string - secret: - type: string - type: object - dbServiceName: - properties: - key: - type: string - secret: - type: string - type: object - dbUser: - properties: - key: - type: string - secret: - type: string - type: object - dbWallet: - properties: - key: - type: string - secret: - type: string - type: object - type: object - exporter: - description: DatabaseObserverExporterConfig defines the configuration - details related to the exporters of DatabaseObserver - properties: - configuration: - properties: - configmap: - description: ConfigMapDetails defines the configmap name - properties: - configmapName: - type: string - key: - type: string - type: object - type: object - image: - type: string - service: - description: DatabaseObserverService defines the exporter service - component of DatabaseObserver - properties: - port: - format: int32 - type: integer - type: object - type: object - prometheus: - description: PrometheusConfig defines the generated resources for - Prometheus - properties: - labels: - additionalProperties: - type: string - type: object - port: - type: string - type: object - replicas: - format: int32 - type: integer - type: object - status: - description: DatabaseObserverStatus defines the observed state of DatabaseObserver - properties: - conditions: - description: 'INSERT ADDITIONAL STATUS FIELD - define observed state - of cluster Important: Run "make" to regenerate code after modifying - this file' - items: - description: "Condition contains details for one aspect of the current - state of this API Resource. --- This struct is intended for direct - use as an array at the field path .status.conditions. For example, - \n \ttype FooStatus struct{ \t // Represents the observations - of a foo's current state. \t // Known .status.conditions.type - are: \"Available\", \"Progressing\", and \"Degraded\" \t // - +patchMergeKey=type \t // +patchStrategy=merge \t // +listType=map - \t // +listMapKey=type \t Conditions []metav1.Condition - `json:\"conditions,omitempty\" patchStrategy:\"merge\" patchMergeKey:\"type\" - protobuf:\"bytes,1,rep,name=conditions\"` \n \t // other fields - \t}" - properties: - lastTransitionTime: - description: lastTransitionTime is the last time the condition - transitioned from one status to another. This should be when - the underlying condition changed. If that is not known, then - using the time when the API field changed is acceptable. - format: date-time - type: string - message: - description: message is a human readable message indicating - details about the transition. This may be an empty string. - maxLength: 32768 - type: string - observedGeneration: - description: observedGeneration represents the .metadata.generation - that the condition was set based upon. For instance, if .metadata.generation - is currently 12, but the .status.conditions[x].observedGeneration - is 9, the condition is out of date with respect to the current - state of the instance. - format: int64 - minimum: 0 - type: integer - reason: - description: reason contains a programmatic identifier indicating - the reason for the condition's last transition. Producers - of specific condition types may define expected values and - meanings for this field, and whether the values are considered - a guaranteed API. The value should be a CamelCase string. - This field may not be empty. - maxLength: 1024 - minLength: 1 - pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ - type: string - status: - description: status of the condition, one of True, False, Unknown. - enum: - - "True" - - "False" - - Unknown - type: string - type: - description: type of condition in CamelCase or in foo.example.com/CamelCase. - --- Many .condition.type values are consistent across resources - like Available, but because arbitrary conditions can be useful - (see .node.status.conditions), the ability to deconflict is - important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) - maxLength: 316 - pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ - type: string - required: - - lastTransitionTime - - message - - reason - - status - - type - type: object - type: array - exporterConfig: - type: string - replicas: - type: integer - status: - type: string - required: - - conditions - - exporterConfig - type: object - type: object - served: true - storage: true - subresources: - status: {} -status: - acceptedNames: - kind: "" - plural: "" - conditions: [] - storedVersions: [] diff --git a/config/crd/kustomization.yaml b/config/crd/kustomization.yaml index b9f3aa8c..c411e3ed 100644 --- a/config/crd/kustomization.yaml +++ b/config/crd/kustomization.yaml @@ -17,7 +17,6 @@ resources: - bases/database.oracle.com_autonomouscontainerdatabases.yaml - bases/database.oracle.com_dbcssystems.yaml - bases/database.oracle.com_dataguardbrokers.yaml -- bases/observability.oracle.com_databaseobservers.yaml # +kubebuilder:scaffold:crdkustomizeresource patchesStrategicMerge: @@ -33,7 +32,6 @@ patchesStrategicMerge: #- patches/webhook_in_autonomouscontainerdatabases.yaml #- patches/webhook_in_dbcssystems.yaml #- patches/webhook_in_dataguardbrokers.yaml -#- patches/webhook_in_databaseobservers.yaml # +kubebuilder:scaffold:crdkustomizewebhookpatch # [CERTMANAGER] To enable webhook, uncomment all the sections with [CERTMANAGER] prefix. @@ -48,7 +46,6 @@ patchesStrategicMerge: #- patches/cainjection_in_autonomouscontainerdatabases.yaml #- patches/cainjection_in_dbcssystems.yaml #- patches/cainjection_in_dataguardbrokers.yaml -#- patches/cainjection_in_databaseobservers.yaml # +kubebuilder:scaffold:crdkustomizecainjectionpatch # the following config is for teaching kustomize how to do kustomization for CRDs. diff --git a/config/crd/patches/cainjection_in_databaseobservers.yaml b/config/crd/patches/cainjection_in_databaseobservers.yaml deleted file mode 100644 index bef0b6c0..00000000 --- a/config/crd/patches/cainjection_in_databaseobservers.yaml +++ /dev/null @@ -1,8 +0,0 @@ -# The following patch adds a directive for certmanager to inject CA into the CRD -# CRD conversion requires k8s 1.13 or later. -apiVersion: apiextensions.k8s.io/v1beta1 -kind: CustomResourceDefinition -metadata: - annotations: - cert-manager.io/inject-ca-from: $(CERTIFICATE_NAMESPACE)/$(CERTIFICATE_NAME) - name: databaseobservers.observability.oracle.com diff --git a/config/crd/patches/webhook_in_databaseobservers.yaml b/config/crd/patches/webhook_in_databaseobservers.yaml deleted file mode 100644 index 40fa3960..00000000 --- a/config/crd/patches/webhook_in_databaseobservers.yaml +++ /dev/null @@ -1,17 +0,0 @@ -# The following patch enables conversion webhook for CRD -# CRD conversion requires k8s 1.13 or later. -apiVersion: apiextensions.k8s.io/v1beta1 -kind: CustomResourceDefinition -metadata: - name: databaseobservers.observability.oracle.com -spec: - conversion: - strategy: Webhook - webhookClientConfig: - # this is "\n" used as a placeholder, otherwise it will be rejected by the apiserver for being blank, - # but we're going to set it later using the cert-manager (or potentially a patch if not using cert-manager) - caBundle: Cg== - service: - namespace: system - name: webhook-service - path: /convert diff --git a/config/rbac/databaseobserver_editor_role.yaml b/config/rbac/databaseobserver_editor_role.yaml deleted file mode 100644 index b6451f85..00000000 --- a/config/rbac/databaseobserver_editor_role.yaml +++ /dev/null @@ -1,24 +0,0 @@ -# permissions for end users to edit databaseobservers. -apiVersion: rbac.authorization.k8s.io/v1 -kind: ClusterRole -metadata: - name: databaseobserver-editor-role -rules: -- apiGroups: - - observability.oracle.com - resources: - - databaseobservers - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - observability.oracle.com - resources: - - databaseobservers/status - verbs: - - get diff --git a/config/rbac/databaseobserver_viewer_role.yaml b/config/rbac/databaseobserver_viewer_role.yaml deleted file mode 100644 index e3ba538d..00000000 --- a/config/rbac/databaseobserver_viewer_role.yaml +++ /dev/null @@ -1,20 +0,0 @@ -# permissions for end users to view databaseobservers. -apiVersion: rbac.authorization.k8s.io/v1 -kind: ClusterRole -metadata: - name: databaseobserver-viewer-role -rules: -- apiGroups: - - observability.oracle.com - resources: - - databaseobservers - verbs: - - get - - list - - watch -- apiGroups: - - observability.oracle.com - resources: - - databaseobservers/status - verbs: - - get diff --git a/config/rbac/role.yaml b/config/rbac/role.yaml index ab77777b..89b630dd 100644 --- a/config/rbac/role.yaml +++ b/config/rbac/role.yaml @@ -6,23 +6,6 @@ metadata: creationTimestamp: null name: manager-role rules: -- apiGroups: - - "" - resources: - - configmaps - - deployments - - events - - pods - - secrets - - services - verbs: - - create - - delete - - get - - list - - patch - - update - - watch - apiGroups: - "" resources: @@ -98,21 +81,6 @@ rules: - patch - update - watch -- apiGroups: - - apps - resources: - - configmaps - - deployments - - pods - - services - verbs: - - create - - delete - - get - - list - - patch - - update - - watch - apiGroups: - apps resources: @@ -460,42 +428,3 @@ rules: - get - patch - update -- apiGroups: - - monitoring.coreos.com - resources: - - prometheusrules - - servicemonitors - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - observability.oracle.com - resources: - - databaseobservers - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - observability.oracle.com - resources: - - databaseobservers/finalizers - verbs: - - update -- apiGroups: - - observability.oracle.com - resources: - - databaseobservers/status - verbs: - - get - - patch - - update diff --git a/config/samples/kustomization.yaml b/config/samples/kustomization.yaml index 93687749..0ad50fcf 100644 --- a/config/samples/kustomization.yaml +++ b/config/samples/kustomization.yaml @@ -17,5 +17,4 @@ resources: - sharding/sharding_v1alpha1_provshard.yaml - dbcs/database_v1alpha1_dbcssystem.yaml - database_v1alpha1_dataguardbroker.yaml -- observability_v1alpha1_databaseobserver.yaml # +kubebuilder:scaffold:manifestskustomizesamples diff --git a/config/samples/observability/observability_v1alpha1_databaseobserver.yaml b/config/samples/observability/observability_v1alpha1_databaseobserver.yaml deleted file mode 100644 index 7412794f..00000000 --- a/config/samples/observability/observability_v1alpha1_databaseobserver.yaml +++ /dev/null @@ -1,52 +0,0 @@ -apiVersion: observability.oracle.com/v1alpha1 -kind: DatabaseObserver -metadata: - name: database-observer-sample -spec: - database: - - # Required - dbName: "" - # Required - dbPassword: - key: "password" - secret: db-password - dbConnectionString: - key: "access" - secret: db-connect-string - - # Optional - # This secret contains the wallet files and must be provided if you - # are connecting to your database with a wallet - dbWallet: - secret: instance-wallet - - - # Configuration - exporter: - # Optional - # image: "" - - # Optional - # configuration: - # configmap: - # key: "" - # configmapName: "" - - # Provides configuration to the port used by the exporter service - # Optional - service: - port: 9161 - - prometheus: - # Provides configuration to the port used in the service monitor - # Optional - port: metrics - # Used by servicemonitors, deployment and service - # Optional - labels: - app: app-sample-label - - # Provides configuration to the exporter replicas created - # Optional - replicas: 1 diff --git a/controllers/observability/databaseobserver_controller.go b/controllers/observability/databaseobserver_controller.go deleted file mode 100644 index 5777a5a3..00000000 --- a/controllers/observability/databaseobserver_controller.go +++ /dev/null @@ -1,277 +0,0 @@ -/* -** Copyright (c) 2022 Oracle and/or its affiliates. -** -** The Universal Permissive License (UPL), Version 1.0 -** -** Subject to the condition set forth below, permission is hereby granted to any -** person obtaining a copy of this software, associated documentation and/or data -** (collectively the "Software"), free of charge and under any and all copyright -** rights in the Software, and any and all patent rights owned or freely -** licensable by each licensor hereunder covering either (i) the unmodified -** Software as contributed to or provided by such licensor, or (ii) the Larger -** Works (as defined below), to deal in both -** -** (a) the Software, and -** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if -** one is included with the Software (each a "Larger Work" to which the Software -** is contributed by such licensors), -** -** without restriction, including without limitation the rights to copy, create -** derivative works of, display, perform, and distribute the Software and make, -** use, sell, offer for sale, import, export, have made, and have sold the -** Software and the Larger Work(s), and to sublicense the foregoing rights on -** either these or other terms. -** -** This license is subject to the following condition: -** The above copyright notice and either this complete permission notice or at -** a minimum a reference to the UPL must be included in all copies or -** substantial portions of the Software. -** -** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -** SOFTWARE. - */ - -package controllers - -import ( - "context" - "errors" - appsv1 "k8s.io/api/apps/v1" - corev1 "k8s.io/api/core/v1" - "k8s.io/apimachinery/pkg/api/meta" - metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" - "k8s.io/apimachinery/pkg/types" - "k8s.io/client-go/tools/record" - "time" - - "github.com/go-logr/logr" - "k8s.io/apimachinery/pkg/runtime" - ctrl "sigs.k8s.io/controller-runtime" - "sigs.k8s.io/controller-runtime/pkg/client" - - apiv1 "github.com/oracle/oracle-database-operator/apis/observability/v1alpha1" - constants "github.com/oracle/oracle-database-operator/commons/observability" -) - -// DatabaseObserverReconciler reconciles a DatabaseObserver object -type DatabaseObserverReconciler struct { - client.Client - Log logr.Logger - Scheme *runtime.Scheme - Recorder record.EventRecorder -} - -//+kubebuilder:rbac:groups=observability.oracle.com,resources=databaseobservers,verbs=get;list;watch;create;update;patch;delete -//+kubebuilder:rbac:groups=observability.oracle.com,resources=databaseobservers/status,verbs=get;update;patch -//+kubebuilder:rbac:groups=observability.oracle.com,resources=databaseobservers/finalizers,verbs=update -//+kubebuilder:rbac:groups=apps,resources=pods;deployments;services;configmaps,verbs=get;list;watch;create;update;patch;delete -//+kubebuilder:rbac:groups="",resources=pods;deployments;services;secrets;configmaps;events,verbs=get;list;watch;create;update;patch;delete -//+kubebuilder:rbac:groups=monitoring.coreos.com,resources=prometheusrules;servicemonitors,verbs=get;list;watch;create;update;patch;delete - -// Reconcile is part of the main kubernetes reconciliation loop which aims to -// move the current state of the cluster closer to the desired state. -// TODO(user): Modify the Reconcile function to compare the state specified by -// the DatabaseObserver object against the actual cluster state, and then -// perform operations to make the cluster state reflect the state specified by -// the user. -// -// For more details, check Reconcile and its Result here: -// - https://pkg.go.dev/sigs.k8s.io/controller-runtime@v0.6.4/pkg/reconcile -func (r *DatabaseObserverReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { - log := r.Log.WithValues(constants.LogReconcile, req.NamespacedName) - log.Info("Begin Reconcile", "NamespacedName", req.NamespacedName) - - cr := &apiv1.DatabaseObserver{} // Fetch CR - if err := r.Get(context.TODO(), req.NamespacedName, cr); err != nil { - log.Error(err, "Failed to fetch Observability CRD") - return ctrl.Result{}, err - } - - defer r.validateReadiness(cr, ctx, req) - - if err := r.initialize(ctx, cr, req); err != nil { - return ctrl.Result{}, err - } - - if err := r.validateSpecs(cr); err != nil { - meta.SetStatusCondition(&cr.Status.Conditions, metav1.Condition{ - Type: constants.PhaseControllerValidation, - Status: metav1.ConditionFalse, - Reason: constants.ReasonValidationFailure, - Message: err.Error(), - }) - r.updateStatusAndRetrieve(cr, ctx, req) - return ctrl.Result{}, err - } - - if res, err := r.manageConfigmap(cr, ctx, req); err != nil { - return res, err - } - - exporterDeployment := &ObservabilityDeploymentResource{} - if res, err := r.manageResource(exporterDeployment, cr, ctx, req); err != nil { - return res, err - } - if err := r.compareOrUpdateDeployment(exporterDeployment, cr, ctx, req); err != nil { - return ctrl.Result{}, err - } - - exporterService := &ObservabilityServiceResource{} - if res, err := r.manageResource(exporterService, cr, ctx, req); err != nil { - return res, err - } - - exporterServiceMonitor := &ObservabilityServiceMonitorResource{} - if res, err := r.manageResource(exporterServiceMonitor, cr, ctx, req); err != nil { - return res, err - } - - exporterGrafanaOutputCM := &ObservabilityGrafanaJsonResource{} - if res, err := r.manageResource(exporterGrafanaOutputCM, cr, ctx, req); err != nil { - return res, err - } - - if !r.isExporterDeploymentReady(cr, ctx, req) { - return ctrl.Result{Requeue: true, RequeueAfter: 15 * time.Second}, nil - } - return ctrl.Result{}, nil -} - -func (r *DatabaseObserverReconciler) initialize(ctx context.Context, cr *apiv1.DatabaseObserver, req ctrl.Request) error { - - if cr.Status.Conditions == nil || len(cr.Status.Conditions) == 0 { - - meta.SetStatusCondition(&cr.Status.Conditions, metav1.Condition{ - Type: string(constants.StatusObservabilityReady), - Status: metav1.ConditionUnknown, - Reason: constants.ReasonInitStart, - Message: "Starting reconciliation", - }) - - cr.Status.Status = string(constants.StatusObservabilityPending) - cr.Status.ExporterConfig = constants.UnknownValue - - r.updateStatusAndRetrieve(cr, ctx, req) - r.Recorder.Event(cr, corev1.EventTypeNormal, constants.ReasonInitSuccess, "Initialization of observability resource completed") - } - - return nil -} - -func (r *DatabaseObserverReconciler) validateSpecs(api *apiv1.DatabaseObserver) error { - - // Does DB Password Secret Name have a value - // if all secretName, vaultOCID and vaultSecretName are empty, then error out - if api.Spec.Database.DBPassword.SecretName == "" && - api.Spec.Database.DBPassword.VaultOCID == "" && - api.Spec.Database.DBPassword.VaultSecretName == "" { - return errors.New("spec validation failed due to required field dbPassword not having a value") - } - - // Does DB Password Secret Name actually exist if vault fields are not used? - if api.Spec.Database.DBPassword.VaultOCID == "" || api.Spec.Database.DBPassword.VaultSecretName == "" { - dbSecret := &corev1.Secret{} - if err := r.Get(context.TODO(), types.NamespacedName{Name: api.Spec.Database.DBPassword.SecretName, Namespace: api.Namespace}, dbSecret); err != nil { - r.Log.Error(err, "Failed to find secret", "Name", api.Spec.Database.DBPassword.SecretName, "Namespace", api.Namespace) - return err - } - } - - //sn := api.Spec.Database.DBServiceName.SecretName - //user := api.Spec.Database.DBUser.SecretName - connStr := api.Spec.Database.DBConnectionString.SecretName - - // Does ConnectionString have a value - //var snAndUserCannotBeUsed = sn == "" || user == "" - - if connStr == "" { - - return errors.New("spec validation failed due to required fields dbConnectionString or dbServiceName and dbUser not having a value") - - } else { - - // Does DB Connection String Secret Name actually exist - dbConnectSecret := &corev1.Secret{} - if err := r.Get(context.TODO(), types.NamespacedName{Name: api.Spec.Database.DBConnectionString.SecretName, Namespace: api.Namespace}, dbConnectSecret); err != nil { - r.Log.Error(err, "Failed to find secret", "Name", api.Spec.Database.DBConnectionString.SecretName, "Namespace", api.Namespace) - return err - } - - } - - // Did the user provide a custom configuration configmap - // If so, does it actually exist - configurationCMName := api.Spec.Exporter.ExporterConfig.Configmap.Name - if configurationCMName != "" { - - configurationCM := &corev1.ConfigMap{} - if err := r.Get(context.TODO(), types.NamespacedName{Name: configurationCMName, Namespace: api.Namespace}, configurationCM); err != nil { - r.Log.Error(err, "Failed to find configuration config map", "Name", configurationCMName, "Namespace", api.Namespace) - return err - } - } - - meta.RemoveStatusCondition(&api.Status.Conditions, constants.PhaseControllerValidation) // Clear any validation conditions - r.Log.Info("Completed reconcile validation") - return nil // valid, did not encounter any errors -} - -func (r *DatabaseObserverReconciler) validateReadiness(api *apiv1.DatabaseObserver, ctx context.Context, req ctrl.Request) { - - // get latest object - if err := r.Get(context.TODO(), req.NamespacedName, api); err != nil { - r.Log.Error(err, "Failed to fetch updated CR") - } - - // evaluate if there exists conditions present that invalidates readiness - if api.Status.Conditions != nil && len(api.Status.Conditions) > 1 { - meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ - Type: string(constants.StatusObservabilityReady), - Status: metav1.ConditionFalse, - Reason: "DeploymentUnsuccessful", - Message: "Observability exporter deployment failure", - }) - api.Status.Status = string(constants.StatusObservabilityError) - } else { - meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ - Type: string(constants.StatusObservabilityReady), - Status: metav1.ConditionTrue, - Reason: "ResourceReady", - Message: "Observability exporter deployed successfully", - }) - api.Status.Status = string(constants.StatusObservabilityReady) - } - - if err := r.Status().Update(ctx, api); err != nil { - r.Log.Error(err, "Failed to update resource status") - } - -} - -func (r *DatabaseObserverReconciler) updateStatusAndRetrieve(api *apiv1.DatabaseObserver, ctx context.Context, req ctrl.Request) { - - // make update - if err := r.Status().Update(ctx, api); err != nil { - r.Log.Error(err, "Failed to update resource status") - } - - // refresh cr before changes - if err := r.Get(context.TODO(), req.NamespacedName, api); err != nil { - r.Log.Error(err, "Failed to fetch updated CR") - } - -} - -// SetupWithManager sets up the controller with the Manager. -func (r *DatabaseObserverReconciler) SetupWithManager(mgr ctrl.Manager) error { - return ctrl.NewControllerManagedBy(mgr). - For(&apiv1.DatabaseObserver{}). - Owns(&appsv1.Deployment{}). - Owns(&corev1.Service{}). - Complete(r) -} diff --git a/controllers/observability/databaseobserver_resource.go b/controllers/observability/databaseobserver_resource.go deleted file mode 100644 index a3d3a7a3..00000000 --- a/controllers/observability/databaseobserver_resource.go +++ /dev/null @@ -1,505 +0,0 @@ -package controllers - -import ( - "context" - apiv1 "github.com/oracle/oracle-database-operator/apis/observability/v1alpha1" - "github.com/oracle/oracle-database-operator/commons/observability" - monitorv1 "github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring/v1" - appsv1 "k8s.io/api/apps/v1" - corev1 "k8s.io/api/core/v1" - "k8s.io/apimachinery/pkg/api/errors" - "k8s.io/apimachinery/pkg/api/meta" - metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" - "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" - "k8s.io/apimachinery/pkg/runtime" - "k8s.io/apimachinery/pkg/runtime/schema" - "k8s.io/apimachinery/pkg/types" - ctrl "sigs.k8s.io/controller-runtime" - "sigs.k8s.io/controller-runtime/pkg/client" - "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" -) - -/* -This handler file contains all the methods that -retrieve/find and create all related resources -on Kubernetes. -*/ - -type ObservabilityConfigResource struct{} -type ObservabilityDeploymentResource struct{} -type ObservabilityServiceResource struct{} -type ObservabilityGrafanaJsonResource struct{} -type ObservabilityServiceMonitorResource struct{} - -type ObservabilityResource interface { - generate(*apiv1.DatabaseObserver, *runtime.Scheme) (*unstructured.Unstructured, error) - identify() (string, string, schema.GroupVersionKind) -} - -func (r *DatabaseObserverReconciler) isExporterDeploymentReady(api *apiv1.DatabaseObserver, ctx context.Context, req ctrl.Request) bool { - - // get latest deployment - dep := &appsv1.Deployment{} - rName := observability.DefaultExporterDeploymentPrefix + api.Name - - // defer update for status changes below - defer r.updateStatusAndRetrieve(api, ctx, req) - if err := r.Get(context.TODO(), types.NamespacedName{Name: rName, Namespace: api.Namespace}, dep); err != nil { - r.Log.Error(err, "Failed to fetch deployment for validating readiness") - meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ - Type: observability.PhaseExportersDeployValidation, - Status: metav1.ConditionFalse, - Reason: observability.ReasonRetrieveFailure, - Message: "Failed to retrieve deployment for validation of observability exporter deployment readiness", - }) - return false - } - - labels := dep.Spec.Template.Labels - cLabels := client.MatchingLabels{} - for k, v := range labels { - cLabels[k] = v - } - - pods := &corev1.PodList{} - if err := r.List(context.TODO(), pods, []client.ListOption{cLabels}...); err != nil { - r.Log.Error(err, "Failed to fetch list of observability exporter pods") - meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ - Type: observability.PhaseExportersDeployValidation, - Status: metav1.ConditionFalse, - Reason: observability.ReasonRetrieveFailure, - Message: "Failed to retrieve pods for validation of observability exporter deployment readiness", - }) - return false - } - - for _, pod := range pods.Items { - if pod.Status.Phase != corev1.PodRunning { - r.Log.Info("Observability exporter pod(s) found but not ready") - meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ - Type: observability.PhaseExportersDeployValidation, - Status: metav1.ConditionFalse, - Reason: observability.ReasonValidationFailure, - Message: "Validation of observability exporter deployment readiness failed", - }) - return false - } - } - - meta.RemoveStatusCondition(&api.Status.Conditions, observability.PhaseExportersDeployValidation) - return true -} - -func (r *DatabaseObserverReconciler) compareOrUpdateDeployment(or ObservabilityResource, api *apiv1.DatabaseObserver, ctx context.Context, req ctrl.Request) error { - var updated bool - - // get desired - desiredObj, genErr := or.generate(api, r.Scheme) - if genErr != nil { - r.Log.WithName(observability.LogExportersDeploy).Error(genErr, "Failed to generate deployment") - return genErr - } - desiredDeployment := &appsv1.Deployment{} - if err := r.Scheme.Convert(desiredObj, desiredDeployment, nil); err != nil { - r.Log.WithName(observability.LogExportersDeploy).Error(genErr, "Failed to convert generated deployment") - return err - } - - // defer for updates to status below - defer r.updateStatusAndRetrieve(api, ctx, req) - // retrieve existing deployment - foundDeployment := &appsv1.Deployment{} - if err := r.Get(context.TODO(), types.NamespacedName{Name: desiredObj.GetName(), Namespace: req.Namespace}, foundDeployment); err != nil { - r.Log.WithName(observability.LogExportersDeploy).Error(genErr, "Failed to retrieve deployment") - meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ - Type: observability.PhaseExportersDeployUpdate, - Status: metav1.ConditionFalse, - Reason: observability.ReasonRetrieveFailure, - Message: "Failed to retrieve deployment: " + desiredObj.GetName(), - }) - return err - } - - // check containerImage - foundContainerImage := foundDeployment.Spec.Template.Spec.Containers[0].Image - desiredContainerImage := desiredDeployment.Spec.Template.Spec.Containers[0].Image - if foundContainerImage != desiredContainerImage { - r.Log.WithName(observability.LogExportersDeploy).Info("Updating deployment container image") - foundDeployment.Spec.Template.Spec.Containers[0].Image = desiredContainerImage - updated = true - } - - // check config-volume - var foundConfigmapName, desiredConfigmapName string - desiredVolumes := desiredDeployment.Spec.Template.Spec.Volumes - for _, v := range desiredVolumes { - if v.Name == "config-volume" { - desiredConfigmapName = v.ConfigMap.Name - } - } - - foundVolumes := foundDeployment.Spec.Template.Spec.Volumes - for _, v := range foundVolumes { - if v.Name == "config-volume" { - foundConfigmapName = v.ConfigMap.Name - } - } - - if desiredConfigmapName != foundConfigmapName { - r.Log.WithName(observability.LogExportersDeploy).Info("Updating deployment volumes with new configuration configmap") - foundDeployment.Spec.Template.Spec.Volumes = observability.GetExporterDeploymentVolumes(api) - updated = true - } - - // make the update - if updated { - if err := r.Update(context.TODO(), foundDeployment); err != nil { - r.Log.WithName(observability.LogExportersDeploy).Error(err, "Failed to update deployment", "name", desiredObj.GetName()) - r.Recorder.Event(api, corev1.EventTypeWarning, observability.ReasonUpdateFailure, "Failed to update deployment: "+desiredObj.GetName()) - return err - } - - if desiredConfigmapName != foundConfigmapName { // if configmap was updated - api.Status.ExporterConfig = desiredConfigmapName - } - r.Recorder.Event(api, corev1.EventTypeNormal, observability.ReasonUpdateSuccess, "Succeeded updating deployment: "+desiredObj.GetName()) - } - - meta.RemoveStatusCondition(&api.Status.Conditions, observability.PhaseExportersDeployUpdate) // Clear any conditions - return nil -} - -func (r *DatabaseObserverReconciler) manageConfigmap(api *apiv1.DatabaseObserver, ctx context.Context, req ctrl.Request) (ctrl.Result, error) { - - // retrieve spec information related to resource, - cName := api.Spec.Exporter.ExporterConfig.Configmap.Name - rName := observability.DefaultExporterConfigmapPrefix + api.Name - rKey := observability.DefaultConfigurationConfigmapKey - metricsConfigData := observability.DefaultConfig - - var nameToFind string - if cName != "" { - nameToFind = cName - } else { - nameToFind = rName - } - - // generate - crOwnedConfigmap := &corev1.ConfigMap{ - ObjectMeta: metav1.ObjectMeta{ - Name: rName, - Namespace: api.Namespace, - }, - Data: map[string]string{ - rKey: metricsConfigData, - }, - } - - // defer for updates to status below - defer r.updateStatusAndRetrieve(api, ctx, req) - - // set CR as owner - if err := controllerutil.SetControllerReference(api, crOwnedConfigmap, r.Scheme); err != nil { - r.Log.WithName(observability.LogExportersConfigMap).Error(err, "Failed to set controller reference for configmap", "name", nameToFind) - meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ - Type: observability.PhaseExportersConfigMap, - Status: metav1.ConditionFalse, - Reason: observability.ReasonSetFailure, - Message: "Failed to set owner reference for configmap", - }) - return ctrl.Result{}, err - } - - // find resource - foundConfigmap := &corev1.ConfigMap{} - getErr := r.Get(ctx, types.NamespacedName{Name: nameToFind, Namespace: req.Namespace}, foundConfigmap) - - // if resource not found and custom configmap was not provided, create resource - if getErr != nil && errors.IsNotFound(getErr) && cName == "" { - r.Log.WithName(observability.LogExportersConfigMap).Info("Creating exporter configuration configmap resource", "name", nameToFind) - if err := r.Create(context.TODO(), crOwnedConfigmap); err != nil { // create - r.Log.WithName(observability.LogExportersConfigMap).Error(err, "Failed to create configmap", "name", nameToFind) - r.Recorder.Event(api, corev1.EventTypeWarning, observability.ReasonCreateFailure, "Failed creating configmap: "+rName) - - meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ - Type: observability.PhaseExportersConfigMap, - Status: metav1.ConditionFalse, - Reason: observability.ReasonCreateFailure, - Message: "Failed to create configmap " + nameToFind, - }) - return ctrl.Result{}, err - } - r.Recorder.Event(api, corev1.EventTypeNormal, observability.ReasonCreateSuccess, "Succeeded creating configmap: "+rName) - - } else if getErr != nil && errors.IsNotFound(getErr) && cName != "" { // if custom configmap was not found - r.Log.WithName(observability.LogExportersConfigMap).Error(getErr, "Failed to retrieve custom configmap", "name", cName) - meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ - Type: observability.PhaseExportersConfigMap, - Status: metav1.ConditionFalse, - Reason: observability.ReasonRetrieveFailure, - Message: "Failed to retrieve custom configmap " + nameToFind, - }) - return ctrl.Result{}, getErr - - } else if getErr != nil { // if an error occurred - r.Log.WithName(observability.LogExportersConfigMap).Error(getErr, "Failed to retrieve due to some error", "name", nameToFind) - meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ - Type: observability.PhaseExportersConfigMap, - Status: metav1.ConditionFalse, - Reason: observability.ReasonRetrieveFailure, - Message: "Failed to retrieve configmap: " + nameToFind, - }) - return ctrl.Result{}, getErr - - } - - // refresh cr before changes - if err := r.Get(context.TODO(), req.NamespacedName, api); err != nil { - r.Log.Error(err, "Failed to fetch updated CR") - } - - // create event if exporter config needs to be updated - if getErr == nil && cName != "" && api.Status.ExporterConfig != nameToFind { - r.Recorder.Event(api, corev1.EventTypeNormal, observability.ReasonRetrieveSuccess, "Succeeded retrieving custom configmap: "+nameToFind) - } - - api.Status.ExporterConfig = nameToFind - meta.RemoveStatusCondition(&api.Status.Conditions, observability.PhaseExportersConfigMap) // Clear any conditions - return ctrl.Result{}, nil - -} -func (r *DatabaseObserverReconciler) manageResource(or ObservabilityResource, api *apiv1.DatabaseObserver, ctx context.Context, req ctrl.Request) (ctrl.Result, error) { - resourceType, logName, gvk := or.identify() - - // generate desired object based on api.Spec - desiredObj, genErr := or.generate(api, r.Scheme) - if genErr != nil { - r.Log.WithName(logName).Error(genErr, "Failed to generate "+gvk.Kind) - meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ - Type: resourceType, - Status: metav1.ConditionFalse, - Reason: observability.ReasonSetFailure, - Message: "Failed to generate " + gvk.Kind, - }) - return ctrl.Result{}, genErr - } - - // retrieve object if it exists - foundObj := &unstructured.Unstructured{} - foundObj.SetGroupVersionKind(gvk) - getErr := r.Get(context.TODO(), types.NamespacedName{Name: desiredObj.GetName(), Namespace: req.Namespace}, foundObj) - - // defer for updates to status below - defer r.updateStatusAndRetrieve(api, ctx, req) - - // if resource not found, create resource then return - if getErr != nil && errors.IsNotFound(getErr) { - r.Log.WithName(logName).Info("Creating resource "+gvk.Kind, "name", desiredObj.GetName()) - - if err := r.Create(context.TODO(), desiredObj); err != nil { // create - r.Log.WithName(logName).Error(err, "Failed to create "+gvk.Kind, "name", desiredObj.GetName()) - r.Recorder.Event(api, corev1.EventTypeWarning, observability.ReasonCreateFailure, "Failed creating resource "+gvk.Kind+": "+desiredObj.GetName()) - meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ - Type: resourceType, - Status: metav1.ConditionFalse, - Reason: observability.ReasonCreateFailure, - Message: "Failed to create " + gvk.Kind + ": " + desiredObj.GetName(), - }) - return ctrl.Result{}, err - } - - r.Recorder.Event(api, corev1.EventTypeNormal, observability.ReasonCreateSuccess, "Succeeded creating "+gvk.Kind+": "+desiredObj.GetName()) - - } else if getErr != nil { // if an error occurred - r.Log.WithName(logName).Error(getErr, "Failed to retrieve "+desiredObj.GetKind(), "name", desiredObj.GetName()) - meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ - Type: resourceType, - Status: metav1.ConditionFalse, - Reason: observability.ReasonRetrieveFailure, - Message: "Failed to retrieve " + gvk.Kind + ": " + desiredObj.GetName(), - }) - return ctrl.Result{}, getErr - } - - // if it does exist, if it's not the same, update with details from desired object - meta.RemoveStatusCondition(&api.Status.Conditions, resourceType) // Clear any conditions - return ctrl.Result{}, nil -} - -func (resource *ObservabilityDeploymentResource) generate(api *apiv1.DatabaseObserver, scheme *runtime.Scheme) (*unstructured.Unstructured, error) { - rName := observability.DefaultExporterDeploymentPrefix + api.Name - rContainerName := observability.DefaultExporterContainerName - rContainerImage := observability.GetExporterImage(api) - rVolumes := observability.GetExporterDeploymentVolumes(api) - rVolumeMounts := observability.GetExporterDeploymentVolumeMounts(api) - rSelectors := observability.GetExporterSelector(api) - rReplicas := observability.GetExporterReplicas(api) - rEnvs := observability.GetExporterEnvs(api) - - rPort := []corev1.ContainerPort{ - {ContainerPort: 8080}, - } - - obj := &appsv1.Deployment{ - ObjectMeta: metav1.ObjectMeta{ - Name: rName, - Namespace: api.Namespace, - }, - Spec: appsv1.DeploymentSpec{ - Replicas: &rReplicas, - Selector: &metav1.LabelSelector{ - MatchLabels: rSelectors, - }, - Template: corev1.PodTemplateSpec{ - ObjectMeta: metav1.ObjectMeta{ - Labels: rSelectors, - }, - Spec: corev1.PodSpec{ - Containers: []corev1.Container{{ - Image: rContainerImage, - ImagePullPolicy: corev1.PullAlways, - Name: rContainerName, - Env: rEnvs, - VolumeMounts: rVolumeMounts, - Ports: rPort, - }}, - RestartPolicy: corev1.RestartPolicyAlways, - Volumes: rVolumes, - }, - }, - }, - } - - if err := controllerutil.SetControllerReference(api, obj, scheme); err != nil { - return nil, err - } - - var u = &unstructured.Unstructured{} - if err := scheme.Convert(obj, u, nil); err != nil { - return nil, err - } - return u, nil -} -func (resource *ObservabilityServiceResource) generate(api *apiv1.DatabaseObserver, scheme *runtime.Scheme) (*unstructured.Unstructured, error) { - rServiceName := "obs-svc-" + api.Name - rLabels := observability.GetExporterLabels(api) - rPort := observability.GetExporterServicePort(api) - rSelector := observability.GetExporterSelector(api) - - obj := &corev1.Service{ - ObjectMeta: metav1.ObjectMeta{ - Name: rServiceName, - Labels: rLabels, - Namespace: api.Namespace, - }, - Spec: corev1.ServiceSpec{ - Type: "NodePort", - Selector: rSelector, - Ports: []corev1.ServicePort{ - { - Name: "metrics", - Port: rPort, - }, - }, - }, - } - - if err := controllerutil.SetControllerReference(api, obj, scheme); err != nil { - return nil, err - } - - var u = &unstructured.Unstructured{} - if err := scheme.Convert(obj, u, nil); err != nil { - return nil, err - } - return u, nil -} -func (resource *ObservabilityServiceMonitorResource) generate(api *apiv1.DatabaseObserver, scheme *runtime.Scheme) (*unstructured.Unstructured, error) { - rName := observability.DefaultServicemonitorPrefix + api.Name - rLabels := observability.GetExporterLabels(api) - rSelector := observability.GetExporterSelector(api) - rPort := observability.GetExporterServiceMonitorPort(api) - rInterval := "20s" - - obj := &monitorv1.ServiceMonitor{ - ObjectMeta: metav1.ObjectMeta{ - Name: rName, - Labels: rLabels, - Namespace: api.Namespace, - }, - Spec: monitorv1.ServiceMonitorSpec{ - Endpoints: []monitorv1.Endpoint{{ - Interval: monitorv1.Duration(rInterval), - Port: rPort, - }}, - Selector: metav1.LabelSelector{ - MatchLabels: rSelector, - }, - }, - } - - if err := controllerutil.SetControllerReference(api, obj, scheme); err != nil { - return nil, err - } - - var u = &unstructured.Unstructured{} - if err := scheme.Convert(obj, u, nil); err != nil { - return nil, err - } - return u, nil -} -func (resource *ObservabilityGrafanaJsonResource) generate(api *apiv1.DatabaseObserver, scheme *runtime.Scheme) (*unstructured.Unstructured, error) { - rName := observability.DefaultGrafanaConfigMapNamePrefix + api.Name - grafanaJSON, err := observability.GetGrafanaJSONData(api) - if err != nil { - return nil, err - } - - obj := &corev1.ConfigMap{ - ObjectMeta: metav1.ObjectMeta{ - Name: rName, - Namespace: api.Namespace, - }, - Data: map[string]string{observability.DefaultGrafanaConfigmapKey: string(grafanaJSON)}, - } - - if err := controllerutil.SetControllerReference(api, obj, scheme); err != nil { - return nil, err - } - - var u = &unstructured.Unstructured{} - if err := scheme.Convert(obj, u, nil); err != nil { - return nil, err - } - return u, nil - -} - -func (resource *ObservabilityDeploymentResource) identify() (string, string, schema.GroupVersionKind) { - return observability.PhaseExportersDeploy, observability.LogExportersDeploy, schema.GroupVersionKind{ - Group: "apps", - Version: "v1", - Kind: "Deployment", - } -} -func (resource *ObservabilityServiceResource) identify() (string, string, schema.GroupVersionKind) { - return observability.PhaseExportersSVC, observability.LogExportersSVC, schema.GroupVersionKind{ - Group: "", - Version: "v1", - Kind: "Service", - } -} -func (resource *ObservabilityServiceMonitorResource) identify() (string, string, schema.GroupVersionKind) { - return observability.PhaseExportersServiceMonitor, observability.LogExportersServiceMonitor, schema.GroupVersionKind{ - Group: "monitoring.coreos.com", - Version: "v1", - Kind: "ServiceMonitor", - } -} -func (resource *ObservabilityGrafanaJsonResource) identify() (string, string, schema.GroupVersionKind) { - return observability.PhaseExportersGrafanaConfigMap, observability.LogExportersGrafanaConfigMap, schema.GroupVersionKind{ - Group: "", - Version: "v1", - Kind: "ConfigMap", - } -} diff --git a/controllers/observability/suite_test.go b/controllers/observability/suite_test.go deleted file mode 100644 index 12931e52..00000000 --- a/controllers/observability/suite_test.go +++ /dev/null @@ -1,103 +0,0 @@ -/* -** Copyright (c) 2022 Oracle and/or its affiliates. -** -** The Universal Permissive License (UPL), Version 1.0 -** -** Subject to the condition set forth below, permission is hereby granted to any -** person obtaining a copy of this software, associated documentation and/or data -** (collectively the "Software"), free of charge and under any and all copyright -** rights in the Software, and any and all patent rights owned or freely -** licensable by each licensor hereunder covering either (i) the unmodified -** Software as contributed to or provided by such licensor, or (ii) the Larger -** Works (as defined below), to deal in both -** -** (a) the Software, and -** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if -** one is included with the Software (each a "Larger Work" to which the Software -** is contributed by such licensors), -** -** without restriction, including without limitation the rights to copy, create -** derivative works of, display, perform, and distribute the Software and make, -** use, sell, offer for sale, import, export, have made, and have sold the -** Software and the Larger Work(s), and to sublicense the foregoing rights on -** either these or other terms. -** -** This license is subject to the following condition: -** The above copyright notice and either this complete permission notice or at -** a minimum a reference to the UPL must be included in all copies or -** substantial portions of the Software. -** -** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -** SOFTWARE. - */ - -package controllers - -import ( - "path/filepath" - "testing" - - . "github.com/onsi/ginkgo" - . "github.com/onsi/gomega" - "k8s.io/client-go/kubernetes/scheme" - "k8s.io/client-go/rest" - "sigs.k8s.io/controller-runtime/pkg/client" - "sigs.k8s.io/controller-runtime/pkg/envtest" - "sigs.k8s.io/controller-runtime/pkg/envtest/printer" - logf "sigs.k8s.io/controller-runtime/pkg/log" - "sigs.k8s.io/controller-runtime/pkg/log/zap" - - observabilityv1alpha1 "github.com/oracle/oracle-database-operator/apis/observability/v1alpha1" - //+kubebuilder:scaffold:imports -) - -// These tests use Ginkgo (BDD-style Go testing framework). Refer to -// http://onsi.github.io/ginkgo/ to learn more about Ginkgo. - -var cfg *rest.Config -var k8sClient client.Client -var testEnv *envtest.Environment - -func TestAPIs(t *testing.T) { - RegisterFailHandler(Fail) - - RunSpecsWithDefaultAndCustomReporters(t, - "Controller Suite", - []Reporter{printer.NewlineReporter{}}) -} - -var _ = BeforeSuite(func(done Done) { - logf.SetLogger(zap.New(zap.WriteTo(GinkgoWriter), zap.UseDevMode(true))) - - By("bootstrapping test environment") - testEnv = &envtest.Environment{ - CRDDirectoryPaths: []string{filepath.Join("..", "..", "config", "crd", "bases")}, - } - - var err error - cfg, err = testEnv.Start() - Expect(err).ToNot(HaveOccurred()) - Expect(cfg).ToNot(BeNil()) - - err = observabilityv1alpha1.AddToScheme(scheme.Scheme) - Expect(err).NotTo(HaveOccurred()) - - //+kubebuilder:scaffold:scheme - - k8sClient, err = client.New(cfg, client.Options{Scheme: scheme.Scheme}) - Expect(err).ToNot(HaveOccurred()) - Expect(k8sClient).ToNot(BeNil()) - - close(done) -}, 60) - -var _ = AfterSuite(func() { - By("tearing down the test environment") - err := testEnv.Stop() - Expect(err).ToNot(HaveOccurred()) -}) diff --git a/docs/observability/README.md b/docs/observability/README.md deleted file mode 100644 index a327d64e..00000000 --- a/docs/observability/README.md +++ /dev/null @@ -1,198 +0,0 @@ -# Cloud Native Observability on Kubernetes for Oracle Databases - -Oracle Database Operator for Kubernetes (`OraOperator`) includes, as a preview, the -Cloud Native Observability Controller for Oracle Databases, which automates the -deployment of the database observability exporter, creates Service Monitors -for Prometheus and provides a configmap containing a JSON for a sample dashboard -in Grafana. The following sections explain the setup and functionality -of the controller - -- [Requirements](#requirements) -- [The Observability Custom Resource](#observability-custom-resource) - - [List](#observability-list) - - [Quick Status](#quick-status) - - [Detailed Status](#detailed-status) -- [Create](#create) -- [Delete](#delete) -- [Patch](#patch) -- [Troubleshooting](#debugging-and-troubleshooting) - -## Requirements -Oracle recommends that you follow the [requirements](./REQUIREMENTS.md). - -## DatabaseObserver Custom Resource -The Oracle Database Operator __v1.0.0__ includes the Observability controller, as a Preview (i.e., not a production quality feature). The Observability controller automates -the deployment and setting up of the Oracle Database exporter and related resources to make databases observable. - -### Resource Details -#### Observability List -To list the Observability custom resources, use the following command as an example: -```sh -$ kubectl get databaseobserver -``` - -#### Quick Status -To obtain a quick status, use the following command as an example: - -> Note: The databaseobserver custom resource is named `obs-sample`. -> We will use this name only in the following examples - -```sh -$ kubectl get databaseobserver obs-sample -NAME EXPORTERCONFIG STATUS -obs-sample obs-cm-obs-sample READY -``` - -#### Detailed Status -To obtain a detailed database status, use the following command as an example: - -```sh -$ kubectl describe databaseobserver obs-sample -Name: obs-sample -Namespace: ... -Labels: ... -Annotations: ... -API Version: observability.oracle.com/v1alpha1 -Kind: DatabaseObserver -Metadata: ... -Spec: ... -Status: - Conditions: - Last Transition Time: 2023-05-08T15:23:05Z - Message: Observability exporter deployed successfully - Reason: ResourceReady - Status: True - Type: READY - Exporter Config: obs-cm-obs-sample - Status: READY -Events: - Type Reason Age From Message - ---- ------ ---- ---- ------- - Normal InitializationSucceeded 16s Observability Initialization of observability resource completed - Normal CreateResourceSucceeded 16s Observability Succeeded creating configmap: obs-cm-obs-sample - Normal CreateResourceSucceeded 16s Observability Succeeded creating Deployment: obs-deploy-obs-sample - Normal CreateResourceSucceeded 16s Observability Succeeded creating Service: obs-svc-obs-sample - Normal CreateResourceSucceeded 16s Observability Succeeded creating ServiceMonitor: obs-servicemonitor-obs-sample - Normal CreateResourceSucceeded 16s Observability Succeeded creating ConfigMap: obs-json-dash-obs-sample -``` - - -## Create -To provision a new Observability custom resource on the Kubernetes cluster, use the example **[config/samples/observability/observability_create.yaml](../../config/samples/observability/observability_create.yaml)**. - -In the example YAML file, the Observability custom resource checks the following properties: - -| Attribute | Type | Default | Required? | Example | -|-------------------------------------------------------|--------|-----------------|-------------|-----------------------------------------------------------------------| -| `spec.database.dbName` | string | - | Yes | _oradevdb_ | -| `spec.database.dbPassword.key` | string | password | Optional | _admin-password_ | -| `spec.database.dbPassword.secret` | string | - | Required | _devsec-dbpassword_ | -| `spec.database.dbWallet.secret` | string | - | Conditional | _devsec-oradevdb-wallet_ | -| `spec.database.dbConnectionString.key` | string | access | Optional | _connection_ | -| `spec.database.dbConnectionString.secret` | string | - | Required | _devsec-dbconnectionstring_ | -| `spec.exporter.image` | string | - | No | _container-registry.oracle.com/database/observability-exporter:1.0.2_ | -| `spec.exporter.configuration.configmap.key` | string | config.toml | No | _config.toml_ | -| `spec.exporter.configuration.configmap.configmapName` | string | - | No | _devcm-oradevdb-config_ | -| `spec.exporter.service.port` | number | 9161 | No | _9161_ | -| `spec.prometheus.port` | string | metrics | No | _metrics_ | -| `spec.prometheus.labels` | map | app: obs-{name} | No | _app: oradevdb-apps_ | -| `spec.replicas` | number | 1 | No | _1_ | - -To provide the remaining references above, follow the guide provided below. - -To enable the exporter to make a connection and observe your database, whether it is in the cloud or it is -a containerized database, you need to provide a working set of database credentials. - - -1. Create a Kubernetes secret for the Database Connection String. - ```bash - kubectl create secret generic db-connect-string --from-literal=access=admin/password@sampledb_tp - ``` - -2. Create a Kubernetes secret for the Database Password. - ```bash - kubectl create secret generic db-password --from-literal=password='password' - ``` - -3. (Conditional) If your database is an Autonomous Database, create a Kubernetes secret for the Autonomous Database Wallet - - If you have an Autonomous database, you can create a secret for the wallet by following this [step](../adb/README.md#download-wallets). - - -Once the secrets are created, to create the Observability resource, apply the YAML -```bash - kubectl apply -f observability_create.yaml -``` - -### Configurations - -The Observability controller provides multiple other fields for configuration but -are optional: - -- `spec.exporter.image` - You can update this field to override the exporter image used - by the controller, and is useful for using the latest exporter version ahead of what the - Observability controller supports. -- `spec.epxorter.configuration.configmap.configmapName` and `spec.epxorter.configuration.configmap.key` - You can - set this field to the name of a custom configmap that you want to provide the controller to use - instead of the default configmap. This field can be updated and the deployment will replace the configmap being used. -- `spec.exporter.service.port` - You can change the port used by the exporter service created by the controller. -- `spec.prometheus.port` - You can change the port used by the service monitor created by the controller. -- `spec.prometheus.labels` - You can set the labels that is specific to your usage. This field defines the labels - and selector labels used by the resources. This field cannot be updated after the custom resource is created. -- `spec.replicas` - You can set this number to the replica count of the exporter deployment. - -## Delete - -To delete the DatabaseObserver custom resource and all related resources: - -```bash -kubectl delete databaseobserver obs-sample -``` - -## Patch - -The Observability controller currently supports updates for only the following fields: - -- `spec.exporter.image` - If there is a specific release of the observability exporter you require, you can update - the _image_ field and the controller will attempt to update the exporter deployment image. The command below - shows how to patch the field to the latest image of the exporter. - ```bash - kubectl --type=merge -p '{"spec":{"exporter":{"image":"container-registry.oracle.com/database/observability-exporter:latest"}}}' patch databaseobserver obs-sample - ``` - -- `spec.exporter.configuration.configmap.configmapName` - The exporter configuration `.toml` file defines the metrics - are scraped by the exporter. This field provides the ability for you to supply your own configmap containing - the custom configurations that you require. The command below shows how to patch the field with your own configmap. - ```bash - kubectl --type=merge -p '{"spec":{"exporter":{"configuration":{"configmap": {"configmapName": "my-custom-configmap", "key": "config.toml"}}}' patch databaseobserver obs-sample - ``` - - -## Debugging and troubleshooting - -### Show the details of the resource - -To get the verbose output of the current spec, use the command below: - -```sh -kubectl describe databaseobserver/database-observer-sample -``` - -If any error occurs during the reconciliation loop, the Operator either reports -the error using the resource's event stream, or will show the error under conditions. - -### Check the logs of the pod where the operator deploys - -Follow the steps to check the logs. - -1. List the pod replicas - - ```sh - kubectl get pods -n oracle-database-operator-system - ``` - -2. Use the below command to check the logs of the deployment - - ```sh - kubectl logs deployment.apps/oracle-database-operator-controller-manager -n oracle-database-operator-system - ``` diff --git a/docs/observability/REQUIREMENTS.md b/docs/observability/REQUIREMENTS.md deleted file mode 100644 index 439356f7..00000000 --- a/docs/observability/REQUIREMENTS.md +++ /dev/null @@ -1,56 +0,0 @@ -# Observability Controller Requirements -To properly deploy an Observability custom resource, complete the following steps below. - -## Install and configure Prometheus - -The Observability controller creates multiple resources that include the Prometheus `servicemonitor`. -These `servicemonitors` are used to define monitoring for a set of services by scraping -metrics from within Kubernetes. In order for the controller to create ServiceMonitors, the -ServiceMonitor custom resource must exist. - -### Checking if the Custom Resource Exists -To check if the custom resource already exists, run: - -```bash -kubectl api-resources | grep monitoring -``` -You should see the below entry listed among others included with Prometheus -```bash -# result -... -servicemonitors smon monitoring.coreos.com/v1 true ServiceMonitor -... -``` - -### Installing Prometheus -If you do not have Prometheus installed, you can install Prometheus using different ways. Below -shows two ways of installing Prometheus - -#### Using the Prometheus Operator -Using the Prometheus Operator for an example, run the following command which installs the Prometheus Operator into -the Kubernetes cluster. This bundle includes all Prometheus CRDs (including the required _servicemonitor_). -```bash -kubectl create -f https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/master/bundle.yaml -``` - -#### Using Helm -Alternatively, you can install Prometheus using helm. [Helm](https://helm.sh/docs/) must be installed beforehand. -Using helm, run the following command to add the Prometheus repository as follows: -```bash -helm repo add prometheus-community https://prometheus-community.github.io/helm-charts -``` -Run the update command to retrieve the latest from the Prometheus chart repository. -```bash -helm repo update -``` -Run the installation command to install prometheus without a custom template. -```bash -helm install prometheus prometheus-community/prometheus -``` - - -## Provision or set up your Database - -The Observability Controller does not include the provisioning of databases by itself. Before creating -the Observability custom resource, you will need to prepare your Oracle Database and the required secrets containing -database credentials. The database user must have access to Database Performance (V$) views. \ No newline at end of file diff --git a/go.mod b/go.mod index 7c5016df..81c4d435 100644 --- a/go.mod +++ b/go.mod @@ -4,11 +4,9 @@ go 1.19 require ( github.com/go-logr/logr v1.2.3 - github.com/onsi/ginkgo v1.16.5 github.com/onsi/ginkgo/v2 v2.5.0 github.com/onsi/gomega v1.24.1 github.com/oracle/oci-go-sdk/v65 v65.26.1 - github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring v0.62.0 go.uber.org/zap v1.23.0 gopkg.in/yaml.v2 v2.4.0 k8s.io/api v0.25.4 @@ -66,11 +64,11 @@ require ( google.golang.org/protobuf v1.28.0 // indirect gopkg.in/inf.v0 v0.9.1 // indirect gopkg.in/yaml.v3 v3.0.1 // indirect - k8s.io/apiextensions-apiserver v0.25.4 // indirect - k8s.io/component-base v0.25.4 // indirect - k8s.io/klog/v2 v2.80.1 // indirect + k8s.io/apiextensions-apiserver v0.25.0 // indirect + k8s.io/component-base v0.25.0 // indirect + k8s.io/klog/v2 v2.70.1 // indirect k8s.io/kube-openapi v0.0.0-20220803162953-67bda5d908f1 // indirect - k8s.io/utils v0.0.0-20221108210102-8e77b1f39fe2 // indirect + k8s.io/utils v0.0.0-20220728103510-ee6ede2d64ed // indirect sigs.k8s.io/json v0.0.0-20220713155537-f223a00ba0e2 // indirect sigs.k8s.io/structured-merge-diff/v4 v4.2.3 // indirect ) @@ -81,6 +79,4 @@ require ( github.com/go-logr/zapr v1.2.3 // indirect github.com/google/gofuzz v1.2.0 // indirect github.com/google/uuid v1.3.0 // indirect - github.com/nxadm/tail v1.4.8 // indirect - gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 // indirect ) diff --git a/go.sum b/go.sum index 7ca15dba..aacb116a 100644 --- a/go.sum +++ b/go.sum @@ -102,8 +102,6 @@ github.com/evanphx/json-patch v5.6.0+incompatible h1:jBYDEEiFBPxA0v50tFdvOzQQTCv github.com/evanphx/json-patch v5.6.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk= github.com/evanphx/json-patch/v5 v5.6.0 h1:b91NhWfaz02IuVxO9faSllyAtNXHMPkC5J8sJCLunww= github.com/evanphx/json-patch/v5 v5.6.0/go.mod h1:G79N1coSVB93tBe7j6PhzjmR3/2VvlbKOFpnXhI9Bw4= -github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo= -github.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ= github.com/fsnotify/fsnotify v1.5.4 h1:jRbGcIw6P2Meqdwuo0H1p6JVLbL5DHKAKlYndzMwVZI= github.com/fsnotify/fsnotify v1.5.4/go.mod h1:OVB6XrOHzAwXMpEM7uPOzcehqUV2UqJxmVXmkdnm1bU= github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04= @@ -116,6 +114,7 @@ github.com/go-kit/log v0.1.0/go.mod h1:zbhenjAZHb184qTLMA9ZjW7ThYL0H2mk7Q6pNt4vb github.com/go-logfmt/logfmt v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9GBnD5lWE= github.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V4qmtdjCk= github.com/go-logfmt/logfmt v0.5.0/go.mod h1:wCYkCAKZfumFQihp8CzCvQ3paCTfi41vtzG1KdI/P7A= +github.com/go-logr/logr v0.1.0/go.mod h1:ixOQHD9gLJUVQQ2ZOR7zLEifBX6tGkNJF4QyIY7sIas= github.com/go-logr/logr v1.2.0/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A= github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A= github.com/go-logr/logr v1.2.3 h1:2DntVwHkVopvECVRSlL5PSo9eG+cAkDCuckLubN+rq0= @@ -131,7 +130,6 @@ github.com/go-openapi/swag v0.19.5/go.mod h1:POnQmlKehdgb5mhVOsnJFsivZCEZ/vjK9gh github.com/go-openapi/swag v0.19.14 h1:gm3vOOXfiuw5i9p5N9xJvfjvuofpyvLA9Wr6QfK5Fng= github.com/go-openapi/swag v0.19.14/go.mod h1:QYRuS/SOXUCsnplDa677K7+DxSOj6IPNl/eQntq43wQ= github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY= -github.com/go-task/slim-sprig v0.0.0-20210107165309-348f09dbbbc0/go.mod h1:fyg7847qk6SyHyPtNmDHnmrv/HOrqktSC+C9fM+CJOE= github.com/gofrs/flock v0.8.1 h1:+gYjHKf32LDeiEEFhQaotPbLuUXjY5ZqxKgXy7n59aw= github.com/gofrs/flock v0.8.1/go.mod h1:F1TvTiK9OcQqauNUHlbJvyl9Qa1QvF/gOUDKA14jxHU= github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ= @@ -223,7 +221,6 @@ github.com/gorilla/websocket v1.4.2/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/ad github.com/grpc-ecosystem/grpc-gateway v1.16.0/go.mod h1:BDjrQk3hbvj6Nolgz8mAMFbcEtjT1g+wF4CSlocrBnw= github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8= github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8= -github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU= github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc= github.com/ianlancetaylor/demangle v0.0.0-20200824232613-28f6c0f3b639/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc= github.com/imdario/mergo v0.3.12 h1:b6R2BslTbIEToALKP7LxUvijTsNI9TAe80pLWN2g/HU= @@ -274,17 +271,10 @@ github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRW github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U= github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e h1:fD57ERR4JtEqsWbfPhv4DMiApHyliiK5xCTNVSPiaAs= github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e/go.mod h1:zD1mROLANZcx1PVRCS0qkT7pwLkGfwJo4zjcN/Tysno= -github.com/nxadm/tail v1.4.4/go.mod h1:kenIhsEOeOJmVchQTgglprH7qJGnHDVpk1VPCcaMI8A= github.com/nxadm/tail v1.4.8 h1:nPr65rt6Y5JFSKQO7qToXr7pePgD6Gwiw05lkbyAQTE= -github.com/nxadm/tail v1.4.8/go.mod h1:+ncqLTQzXmGhMZNUePPaPqPvBxHAIsmXswZKocGu+AU= -github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE= -github.com/onsi/ginkgo v1.12.1/go.mod h1:zj2OWP4+oCPe1qIXoGWkgMRwljMUYCdkwsT2108oapk= github.com/onsi/ginkgo v1.16.5 h1:8xi0RTUf59SOSfEtZMvwTvXYMzG4gV23XVHOZiXNtnE= -github.com/onsi/ginkgo v1.16.5/go.mod h1:+E8gABHa3K6zRBolWtd+ROzc/U5bkGt0FwiG042wbpU= github.com/onsi/ginkgo/v2 v2.5.0 h1:TRtrvv2vdQqzkwrQ1ke6vtXf7IK34RBUJafIy1wMwls= github.com/onsi/ginkgo/v2 v2.5.0/go.mod h1:Luc4sArBICYCS8THh8v3i3i5CuSZO+RaQRaJoeNwomw= -github.com/onsi/gomega v1.7.1/go.mod h1:XdKZgCCFLUoM/7CFJVPcG8C1xQ1AJ0vpAezJrB7JYyY= -github.com/onsi/gomega v1.10.1/go.mod h1:iN09h71vgCQne3DLsj+A5owkum+a2tYe+TOCB1ybHNo= github.com/onsi/gomega v1.24.1 h1:KORJXNNTzJXzu4ScJWssJfJMnJ+2QJqhoQSRwNlze9E= github.com/onsi/gomega v1.24.1/go.mod h1:3AOiACssS3/MajrniINInwbfOOtfZvplPzuRSmvt1jM= github.com/oracle/oci-go-sdk/v65 v65.26.1 h1:Ms20RSRj+CuvQmw5ET1TkmzxLBI+bmLjZ6NYANA3gkk= @@ -295,8 +285,6 @@ github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4= github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= -github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring v0.62.0 h1:55138zTXw/yRYizPxZ672I/aDD7Yte3uYRAfUjWUu2M= -github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring v0.62.0/go.mod h1:j51242bf6LQwvJ1JPKWApzTnifmCwcQq0i1p29ylWiM= github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw= github.com/prometheus/client_golang v1.0.0/go.mod h1:db9x61etRT2tGnBNRi70OPL5FsnadC4Ky3P0J6CfImo= github.com/prometheus/client_golang v1.7.1/go.mod h1:PY5Wy2awLA44sXw4AOSfFBetzPP4j5+D6mVACh+pe2M= @@ -327,6 +315,7 @@ github.com/sirupsen/logrus v1.6.0/go.mod h1:7uNnSEd1DgxDLC74fIahvMZmmYsHGZGEOFrf github.com/sony/gobreaker v0.5.0 h1:dRCvqm0P490vZPmy7ppEk2qCnCieBooFJ+YoXGYB+yg= github.com/sony/gobreaker v0.5.0/go.mod h1:ZKptC7FHNvhBz7dN2LGjPVBz2sZJmc0/PkyDJOjmxWY= github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA= +github.com/spf13/afero v1.2.2/go.mod h1:9ZxEEn6pIJ8Rxe320qSDBk6AsU0r9pR7Q4OcevTdifk= github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA= github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= github.com/stoewer/go-strcase v1.2.0/go.mod h1:IBiWB2sKIp3wVVQ3Y035++gc+knqhUQag1KpM8ahLw8= @@ -359,7 +348,7 @@ go.uber.org/atomic v1.7.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc= go.uber.org/atomic v1.9.0 h1:ECmE8Bn/WFTYwEW/bpKD3M8VtR/zQVbavAoalC1PYyE= go.uber.org/atomic v1.9.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc= go.uber.org/goleak v1.1.10/go.mod h1:8a7PlsEVH3e/a/GLqe5IIrQx6GzcnRmZEufDUTk4A7A= -go.uber.org/goleak v1.2.0 h1:xqgm/S+aQvhWFTtR0XK3Jvg7z8kGV8P4X14IzwN3Eqk= +go.uber.org/goleak v1.1.12 h1:gZAh5/EyT/HQwlpkCy6wTpqfH9H8Lz8zbm3dZh+OyzA= go.uber.org/multierr v1.6.0/go.mod h1:cdWPpRnG4AhwMwsgIHip0KRBQjJy5kYEpYjJxpXp9iU= go.uber.org/multierr v1.7.0 h1:zaiO/rmgFjbmCXdSYJWQcdvOCsthmdaHfr3Gm2Kx4Ec= go.uber.org/multierr v1.7.0/go.mod h1:7EAYxJLBy9rStEaz58O2t4Uvip6FSURkq8/ppBp95ak= @@ -409,7 +398,6 @@ golang.org/x/mod v0.4.1/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= -golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= @@ -433,7 +421,6 @@ golang.org/x/net v0.0.0-20200324143707-d3edc9973b7e/go.mod h1:qpuaurCH72eLCgpAm/ golang.org/x/net v0.0.0-20200501053045-e0ff5e5a1de5/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= golang.org/x/net v0.0.0-20200506145744-7e3656a0809f/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= golang.org/x/net v0.0.0-20200513185701-a91f0712d120/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= -golang.org/x/net v0.0.0-20200520004742-59133d7f0dd7/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= golang.org/x/net v0.0.0-20200520182314-0ba52f642ac2/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= golang.org/x/net v0.0.0-20200625001655-4c5254603344/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= golang.org/x/net v0.0.0-20200707034311-ab3426394381/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= @@ -480,7 +467,6 @@ golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJ golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= -golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20190312061237-fead79001313/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= @@ -491,10 +477,7 @@ golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7w golang.org/x/sys v0.0.0-20190606165138-5da285871e9c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190624142023-c5567b49c5d0/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20190904154756-749cb33beabd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20191001151750-bb3f8db39f24/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20191005200804-aed5e4c7ecf9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20191120155948-bd437916bb0e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20191228213918-04cbcbbfeed8/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200106162015-b016eb3dc98e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= @@ -518,7 +501,6 @@ golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7w golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20201201145000-ef89a241ccb3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210104204734-6f8348627aad/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20210112080510-489259a85091/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210119212857-b64e53b001e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210220050731-9a76102bfb43/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= @@ -607,7 +589,6 @@ golang.org/x/tools v0.0.0-20200904185747-39188db58858/go.mod h1:Cj7w3i3Rnn0Xh82u golang.org/x/tools v0.0.0-20201110124207-079ba7bd75cd/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= golang.org/x/tools v0.0.0-20201201161351-ac6f37ff4c2a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= golang.org/x/tools v0.0.0-20201208233053-a543418bbed2/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= -golang.org/x/tools v0.0.0-20201224043029-2b0845dc783e/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= golang.org/x/tools v0.0.0-20210105154028-b0ab187a4818/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= golang.org/x/tools v0.1.0/go.mod h1:xkSsbof2nBLbhDlRMhhhyNLN/zl3eTqcnHD5viDpcZ0= @@ -774,11 +755,9 @@ gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8 gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f h1:BLraFXnmrev5lT+xlilqcH8XK9/i0At2xKjWk4p6zsU= gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI= -gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys= gopkg.in/inf.v0 v0.9.1 h1:73M5CoZyi3ZLMOyDlQh031Cx6N9NDJ2Vvfl76EDAgDc= gopkg.in/inf.v0 v0.9.1/go.mod h1:cWUDdTG/fYaXco+Dcufb5Vnc6Gp2YChqWtbxRZE0mXw= gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 h1:uRGJdciOHaEIrze2W8Q3AKkepLTh2hOroT7a+7czfdQ= -gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWDmTeBkI65Dw0HsyUHuEVlX15mw= gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.2.3/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= @@ -802,20 +781,21 @@ honnef.co/go/tools v0.0.1-2020.1.3/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9 honnef.co/go/tools v0.0.1-2020.1.4/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k= k8s.io/api v0.25.4 h1:3YO8J4RtmG7elEgaWMb4HgmpS2CfY1QlaOz9nwB+ZSs= k8s.io/api v0.25.4/go.mod h1:IG2+RzyPQLllQxnhzD8KQNEu4c4YvyDTpSMztf4A0OQ= -k8s.io/apiextensions-apiserver v0.25.4 h1:7hu9pF+xikxQuQZ7/30z/qxIPZc2J1lFElPtr7f+B6U= -k8s.io/apiextensions-apiserver v0.25.4/go.mod h1:bkSGki5YBoZWdn5pWtNIdGvDrrsRWlmnvl9a+tAw5vQ= +k8s.io/apiextensions-apiserver v0.25.0 h1:CJ9zlyXAbq0FIW8CD7HHyozCMBpDSiH7EdrSTCZcZFY= +k8s.io/apiextensions-apiserver v0.25.0/go.mod h1:3pAjZiN4zw7R8aZC5gR0y3/vCkGlAjCazcg1me8iB/E= k8s.io/apimachinery v0.25.4 h1:CtXsuaitMESSu339tfhVXhQrPET+EiWnIY1rcurKnAc= k8s.io/apimachinery v0.25.4/go.mod h1:jaF9C/iPNM1FuLl7Zuy5b9v+n35HGSh6AQ4HYRkCqwo= k8s.io/client-go v0.25.4 h1:3RNRDffAkNU56M/a7gUfXaEzdhZlYhoW8dgViGy5fn8= k8s.io/client-go v0.25.4/go.mod h1:8trHCAC83XKY0wsBIpbirZU4NTUpbuhc2JnI7OruGZw= -k8s.io/component-base v0.25.4 h1:n1bjg9Yt+G1C0WnIDJmg2fo6wbEU1UGMRiQSjmj7hNQ= -k8s.io/component-base v0.25.4/go.mod h1:nnZJU8OP13PJEm6/p5V2ztgX2oyteIaAGKGMYb2L2cY= -k8s.io/klog/v2 v2.80.1 h1:atnLQ121W371wYYFawwYx1aEY2eUfs4l3J72wtgAwV4= -k8s.io/klog/v2 v2.80.1/go.mod h1:y1WjHnz7Dj687irZUWR/WLkLc5N1YHtjLdmgWjndZn0= +k8s.io/component-base v0.25.0 h1:haVKlLkPCFZhkcqB6WCvpVxftrg6+FK5x1ZuaIDaQ5Y= +k8s.io/component-base v0.25.0/go.mod h1:F2Sumv9CnbBlqrpdf7rKZTmmd2meJq0HizeyY/yAFxk= +k8s.io/klog/v2 v2.0.0/go.mod h1:PBfzABfn139FHAV07az/IF9Wp1bkk3vpT2XSJ76fSDE= +k8s.io/klog/v2 v2.70.1 h1:7aaoSdahviPmR+XkS7FyxlkkXs6tHISSG03RxleQAVQ= +k8s.io/klog/v2 v2.70.1/go.mod h1:y1WjHnz7Dj687irZUWR/WLkLc5N1YHtjLdmgWjndZn0= k8s.io/kube-openapi v0.0.0-20220803162953-67bda5d908f1 h1:MQ8BAZPZlWk3S9K4a9NCkIFQtZShWqoha7snGixVgEA= k8s.io/kube-openapi v0.0.0-20220803162953-67bda5d908f1/go.mod h1:C/N6wCaBHeBHkHUesQOQy2/MZqGgMAFPqGsGQLdbZBU= -k8s.io/utils v0.0.0-20221108210102-8e77b1f39fe2 h1:GfD9OzL11kvZN5iArC6oTS7RTj7oJOIfnislxYlqTj8= -k8s.io/utils v0.0.0-20221108210102-8e77b1f39fe2/go.mod h1:OLgZIPagt7ERELqWJFomSt595RzquPNLL48iOWgYOg0= +k8s.io/utils v0.0.0-20220728103510-ee6ede2d64ed h1:jAne/RjBTyawwAy0utX5eqigAwz/lQhTmy+Hr/Cpue4= +k8s.io/utils v0.0.0-20220728103510-ee6ede2d64ed/go.mod h1:jPW/WVKK9YHAvNhRxK0md/EJ228hCsBRufyofKtW8HA= rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8= rsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0= rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA= diff --git a/main.go b/main.go index 2b6b7e8c..6dbf2305 100644 --- a/main.go +++ b/main.go @@ -41,15 +41,10 @@ package main import ( "context" "flag" - monitorv1 "github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring/v1" "os" "strconv" "time" - databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" - observabilityv1alpha1 "github.com/oracle/oracle-database-operator/apis/observability/v1alpha1" - databasecontroller "github.com/oracle/oracle-database-operator/controllers/database" - observabilitycontroller "github.com/oracle/oracle-database-operator/controllers/observability" "go.uber.org/zap/zapcore" "k8s.io/apimachinery/pkg/runtime" utilruntime "k8s.io/apimachinery/pkg/util/runtime" @@ -58,6 +53,9 @@ import ( ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/controller-runtime/pkg/log/zap" + + databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + databasecontroller "github.com/oracle/oracle-database-operator/controllers/database" // +kubebuilder:scaffold:imports ) @@ -68,10 +66,8 @@ var ( func init() { utilruntime.Must(clientgoscheme.AddToScheme(scheme)) - utilruntime.Must(databasev1alpha1.AddToScheme(scheme)) - utilruntime.Must(observabilityv1alpha1.AddToScheme(scheme)) - utilruntime.Must(monitorv1.AddToScheme(scheme)) + utilruntime.Must(databasev1alpha1.AddToScheme(scheme)) // +kubebuilder:scaffold:scheme } @@ -267,16 +263,6 @@ func main() { os.Exit(1) } - // Observability DatabaseObserver Reconciler - if err = (&observabilitycontroller.DatabaseObserverReconciler{ - Client: mgr.GetClient(), - Log: ctrl.Log.WithName("controllers").WithName("observability").WithName("DatabaseObserver"), - Scheme: mgr.GetScheme(), - Recorder: mgr.GetEventRecorderFor("DatabaseObserver"), - }).SetupWithManager(mgr); err != nil { - setupLog.Error(err, "unable to create controller", "controller", "DatabaseObserver") - os.Exit(1) - } // +kubebuilder:scaffold:builder // Add index for PDB CR to enable mgr to cache PDBs From 9fd9401ff52cd35df99ac3d810edab5acf8ab5eb Mon Sep 17 00:00:00 2001 From: Mohammed Yunus Date: Wed, 5 Jul 2023 07:29:04 +0000 Subject: [PATCH 577/628] Sidb readme fixes --- README.md | 2 +- docs/sidb/README.md | 68 ++++++++++++++++++++------------------------- 2 files changed, 31 insertions(+), 39 deletions(-) diff --git a/README.md b/README.md index fb4874cf..dfbdd5a7 100644 --- a/README.md +++ b/README.md @@ -28,7 +28,7 @@ This release of Oracle Database Operator for Kubernetes (the operator) supports * SHARDED: Provision/deploy sharded databases and the shard topology, Add a new shard, Delete an existing shard * Oracle Multitenant Database: Bind to a CDB, Create a  PDB, Plug a  PDB, Unplug a PDB, Delete a PDB, Clone a PDB, Open/Close a PDB * Oracle Base Database Cloud Service (BDBCS): Provision, Bind, Scale Up/Down, Liveness Probe, Manual Backup -* Oracle Data Guard: Provision a Standby for the SIDB resource and setup a Data Guard configuration to enable manual switch over +* Oracle Data Guard: Provision a Standby for the SIDB resource, Create a Data Guard Configuration, Perform a Switchover, Patch Primary and Standby databases in Data Guard Configuration The upcoming releases will support new configurations, operations and capabilities. diff --git a/docs/sidb/README.md b/docs/sidb/README.md index be389c9f..eb5116a4 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -26,8 +26,10 @@ Oracle Database Operator for Kubernetes (`OraOperator`) includes the Single Inst * [Specifying Custom Ports](#specifying-custom-ports) * [Setup Data Guard Configuration for a Single Instance Database (Preview status)](#setup-data-guard-configuration-for-a-single-instance-database-preview-status) * [Create a Standby Database](#create-a-standby-database) - * [Add the Databases in Data Guard Configuration](#add-the-databases-in-data-guard-configuration) - * [Delete a database configured for Data Guard](#delete-a-database-configured-for-data-guard) + * [Create a Data Guard Configuration](#create-a-data-guard-configuration) + * [Perform a Switchover](#perform-a-switchover) + * [Patch Primary and Standby databases in Data Guard configuration](#patch-primary-and-standby-databases-in-data-guard-configuration) + * [Delete the Data Guard Configuration](#delete-the-data-guard-configuration) * [OracleRestDataService Resource](#oraclerestdataservice-resource) * [REST Enable a Database](#rest-enable-a-database) * [Provision ORDS](#provision-ords) @@ -179,7 +181,7 @@ To provision a new database instance on the Kubernetes cluster, use the example singleinstancedatabase.database.oracle.com/sidb-sample created ``` -**NOTE:** +**Note:** - For ease of use, the storage class **oci-bv** is specified in the **[singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml)**. This storage class facilitates dynamic provisioning of the OCI block volumes on the Oracle OKE for persistent storage of the database. The supported access mode for this class is `ReadWriteOnce`. For other cloud providers, you can similarly use their dynamic provisioning storage classes. - It is beneficial to have the database replica pods more than or equal to the number of available nodes if `ReadWriteMany` access mode is used with the OCI NFS volume. By doing so, the pods get distributed on different nodes and the database image is downloaded on all those nodes. This helps in reducing time for the database fail-over if the active database pod dies. - Supports Oracle Database Enterprise Edition (19.3.0), and later releases. @@ -206,7 +208,7 @@ To provision new Oracle Database Express Edition (XE) database, use the sample * This command pulls the XE image uploaded on the [Oracle Container Registry](https://container-registry.oracle.com/). -**NOTE:** +**Note:** - Provisioning Oracle Database express edition is supported for release 21c (21.3.0) only. - For XE database, only single replica mode (i.e. `replicas: 1`) is supported. - For XE database, you **cannot change** the init parameters i.e. `cpuCount, processes, sgaTarget or pgaAggregateTarget`. @@ -218,7 +220,7 @@ To provision new Oracle Database Free database, use the sample **[config/samples This command pulls the Free image uploaded on the [Oracle Container Registry](https://container-registry.oracle.com/). -**NOTE:** +**Note:** - Provisioning Oracle Database Free is supported for release 23c (23.2.0) and later releases. - For Free database, only single replica mode (i.e. `replicas: 1`) is supported. - For Free database, you **cannot change** the init parameters i.e. `cpuCount, processes, sgaTarget or pgaAggregateTarget`. @@ -272,13 +274,14 @@ SQL> ``` **Note:** The `<.spec.adminPassword>` above refers to the database password for SYS, SYSTEM and PDBADMIN users, which in turn represented by `spec` section's `adminPassword` field of the **[config/samples/sidb/singleinstancedatabase.yaml](../config/samples/sidb/../../../../config/samples/sidb/singleinstancedatabase.yaml)** file. -The Oracle Database inside the container also has Oracle Enterprise Manager Express (OEM Express) configured. To access OEM Express, start the browser, and paste in a URL similar to the following example: +The Oracle Database inside the container also has Oracle Enterprise Manager Express (OEM Express) as a basic observability console. To access OEM Express, start the browser, and paste in a URL similar to the following example: ```sh $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.oemExpressUrl}" https://10.0.25.54:5500/em ``` +**Note:** OEM Express is not available for 23c and later releases ### Database Persistence (Storage) Configuration Options The database persistence can be achieved in the following two ways: @@ -409,7 +412,7 @@ The following database initialization parameters can be updated after the databa Change their attribute values and apply using `kubectl apply` or `kubectl edit/patch` commands. -**NOTE:** +**Note:** The value for the initialization parameter `sgaTarget` that you provide should be within the range set by [sga_min_size, sga_max_size]. If the value you provide is not in that range, then `sga_target` is not updated to the value you specify for `sgaTarget`. #### Immutable YAML Attributes @@ -563,7 +566,7 @@ The following steps are required to connect the Database using TCPS: ```bash sqlplus sys@ORCL1 as sysdba ``` -**NOTE:** +**Note:** - When TCPS is enabled, a self-signed certificate is generated and stored inside the wallets. For users' convenience, a client-side wallet is generated and stored at `/opt/oracle/oradata/clientWallet/$ORACLE_SID` location in the pod. - The self-signed certificate used with TCPS has validity for 1 year. After the certificate is expired, it will be renewed by the `OraOperator` automatically. You need to download the wallet again after the auto-renewal. - You can set the certificate renew interval with the help of `tcpsCertRenewInterval` field in the **[config/samples/sidb/singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml)** file. The minimum accepted value is 24h, and the maximum value is 8760h (1 year). The certificates used with TCPS will automatically be renewed after this interval. If this field is omitted/commented in the yaml file, the certificates will not be renewed automatically. @@ -581,7 +584,7 @@ If the `LoadBalancer` is enabled, the `listenerPort`, and `tcpsListenerPort` wil In case of `NodePort` service, `listenerPort`, and `tcpsListenerPort` will be the opened ports on the Kubernetes nodes for for normal and TCPS database connections respectively. In this case, the allowed range for the `listenerPort`, and `tcpsListenerPort` is 30000-32767. -**NOTE:** +**Note:** - `listenerPort` and `tcpsListenerPort` can not have same values. - `tcpsListenerPort` will come into effect only when TCPS connections are enabled (i.e. `enableTCPS` field is set in [config/samples/sidb/singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml) file). - If TCPS connections are enabled, and `listenerPort` is commented/removed in the [config/samples/sidb/singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml) file, only TCPS endpoint will be exposed. @@ -597,7 +600,7 @@ Before creating a standby, ArchiveLog, FlashBack, and ForceLog on primary Single #### Template YAML To create a standby database, edit and apply the sample yaml file [config/samples/sidb/singleinstancedatabase_standby.yaml](../../config/samples/sidb/singleinstancedatabase_standby.yaml). -**NOTE:** +**Note:** - The `adminPassword` field of the above [config/samples/sidb/singleinstancedatabase_standby.yaml](../../config/samples/sidb/singleinstancedatabase_standby.yaml) contains an admin password secret of the primary database ref for Standby Database creation. This secret will get deleted after the database pod becomes ready if the `keepSecret` attribute of `adminPassword` field is set to `false`. By default `keepSecret` is set to `true`. - Mention referred primary database in `.spec.primaryDatabaseRef` in the yaml file. - `.spec.createAsStandby` field of the yaml file should be true. @@ -632,7 +635,7 @@ $ kubectl get singleinstancedatabase stdby-1 -o "jsonpath={.status.status}" Healthy ``` -### Add the Databases in Data Guard Configuration +### Create a Data Guard Configuration #### Template YAML @@ -641,13 +644,14 @@ After creating standbys, setup a dataguard configuration with protection mode an #### Create DataGuardBroker Resource -Provision a new DataguardBroker custom resource for a single instance database(`.spec.primaryDatabaseRef`) by specifying appropriate values for the attributes in the example `.yaml` file, and running the following command: +Provision a new DataguardBroker custom resource for a single instance database(`.spec.primaryDatabaseRef`) by specifying appropriate values for the primary and standby databases in the example `.yaml` file, and running the following command: ```sh $ kubectl create -f dataguardbroker.yaml dataguardbroker.database.oracle.com/dataguardbroker-sample created ``` +**Note:** The following attributes cannot be patched post DataguardBroker resource creation : `primaryDatabaseRef, protectionMode` #### DataguardBroker List @@ -729,18 +733,7 @@ To list the DataguardBroker resources, use the following command: Normal DG Configuration up to date 24m (x13 over 56m) DataguardBroker ``` -#### Connection Information for Primary Database - - External and internal (running in Kubernetes pods) clients can connect to the primary database using `.status.connectString` and `.status.clusterConnectString` - respectively in the following command - - ```sh - $ kubectl get dataguardbroker dataguardbroker-sample -o "jsonpath={.status.externalConnectString}" - - 10.0.25.87:1521/DATAGUARD - ``` - -#### Performing a Switchover +### Perform a Switchover Specify the approppriate SID (SID of one of `.spec.primaryDatabaseRef` , `.spec.standbyDatabaseRefs[]`) to be set primary in the `.spec.setAsPrimaryDatabase` of [dataguardbroker.yaml](./../../config/samples/sidb/dataguardbroker.yaml) and apply the yaml file. @@ -760,20 +753,18 @@ $ kubectl --type=merge -p '{"spec":{"setAsPrimaryDatabase":"ORCLS1"}}' patch dat dataguardbroker.database.oracle.com/dataguardbroker-sample patched ``` -**NOTE :** The following attributes cannot be patched post DataguardBroker resource Creation : `primaryDatabaseRef, protectionMode`. - +#### Static Primary Database Connection String -#### Delete DataguardBroker Resource + External and internal (running in Kubernetes pods) clients can connect to the primary database using `.status.connectString` and `.status.clusterConnectString` of the DataguardBroker resource respectively. These connection strings are fixed for the DataguardBroker resource and will not change on switchover. They can be queried using the following command -```sh -$ kubectl delete dataguardbroker dgbroker-sample - - dataguardbroker.database.oracle.com/dgbroker-sample deleted -``` + ```sh + $ kubectl get dataguardbroker dataguardbroker-sample -o "jsonpath={.status.externalConnectString}" -**NOTE :** You can only delete DataGuard broker when role of `.spec.primaryDatabaseRef` is PRIMARY + 10.0.25.87:1521/DATAGUARD + ``` + The above connection string will always automatically route to the Primary database not requiring clients to change the connection string after switchover -### Patch primary and standby databases in Data Guard configuration +### Patch Primary and Standby databases in Data Guard configuration Databases (both primary and standby) running in you cluster and managed by the Oracle Database operator can be patched or rolled back between release updates of the same major release. While patching databases configured with the dataguard broker you need to first patch the Primary database followed by seconday/standby databases in any order. @@ -784,7 +775,7 @@ kubectl --type=merge -p '{"spec":{"image":{"pullFrom":"patched-image:tag","pullS ``` -### Delete a database configured for Data Guard +### Delete the Data Guard Configuration To delete a standby or primary database configured for Data Guard, delete the dataguardbroker resource first followed by the standby databases and finally the primary database @@ -794,6 +785,7 @@ $ kubectl delete dataguardbroker dgbroker-sample dataguardbroker.database.oracle.com/dgbroker-sample deleted ``` +**Note:** Deleting of DataGuardBroker resource is allowed only when role of `.spec.primaryDatabaseRef` is PRIMARY #### Delete Standby Database ```sh @@ -886,7 +878,7 @@ $ kubectl apply -f oraclerestdataservice_create.yaml ``` After this command completes, ORDS is installed in the container database (CDB) of the Single Instance Database. -##### NOTE: +##### Note: You are required to specify the ORDS secret in the [oraclerestdataservice_create.yaml](../../config/samples/sidb/oraclerestdataservice_create.yaml) file. The default value mentioned in the `adminPassword.secretName` field is `ords-secret`. You can create this secret manually by using the following command: ```bash @@ -1011,7 +1003,7 @@ Fetch all entries from 'DEPT' table by calling the following API -d $'select * from dept;' | python -m json.tool ``` -**NOTE:** `.spec.restEnableSchema[].urlMapping` is optional and is defaulted to `.spec.restEnableSchemas[].schemaName` +**Note:** `.spec.restEnableSchema[].urlMapping` is optional and is defaulted to `.spec.restEnableSchemas[].schemaName` ##### Database Actions @@ -1089,7 +1081,7 @@ password: `.spec.apexPassword` ![application-express-admin-home](/images/sidb/application-express-admin-home.png) -**NOTE:** +**Note:** - By default, the full development environment is initialized in APEX. After deployment, you can change it manually to the runtime environment. To change environments, run the script `apxdevrm.sql` after connecting to the primary database from the ORDS pod as the `SYS` user with `SYSDBA` privilege. For detailed instructions, see: [Converting a Full Development Environment to a Runtime Environment](https://docs.oracle.com/en/database/oracle/application-express/21.2/htmig/converting-between-runtime-and-full-development-environments.html#GUID-B0621B40-3441-44ED-9D86-29B058E26BE9). ### Delete ORDS From bb81b8e5b1c5660ec7f0b4842310e997d1798afe Mon Sep 17 00:00:00 2001 From: psaini Date: Thu, 6 Jul 2023 15:07:06 +0000 Subject: [PATCH 578/628] Added Doc fixes --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index dfbdd5a7..66cd62ec 100644 --- a/README.md +++ b/README.md @@ -27,7 +27,7 @@ This release of Oracle Database Operator for Kubernetes (the operator) supports * SIDB: Provision, clone, patch (in-place/out-of-place), update database initialization parameters, update database configuration (Flashback, archiving), Oracle Enterprise Manager (EM) Express (a basic observability console), Oracle REST Data Service (ORDS) to support REST based SQL, PDB management, SQL Developer Web, and Application Express (Apex) * SHARDED: Provision/deploy sharded databases and the shard topology, Add a new shard, Delete an existing shard * Oracle Multitenant Database: Bind to a CDB, Create a  PDB, Plug a  PDB, Unplug a PDB, Delete a PDB, Clone a PDB, Open/Close a PDB -* Oracle Base Database Cloud Service (BDBCS): Provision, Bind, Scale Up/Down, Liveness Probe, Manual Backup +* Oracle Base Database Cloud Service (BDBCS): provision, bind, scale shape Up/Down, Scale Storage Up, Terminate and Update License * Oracle Data Guard: Provision a Standby for the SIDB resource, Create a Data Guard Configuration, Perform a Switchover, Patch Primary and Standby databases in Data Guard Configuration The upcoming releases will support new configurations, operations and capabilities. From 7312e4ac8b3db89590a1cb39d44407e97d7a147c Mon Sep 17 00:00:00 2001 From: psaini Date: Thu, 6 Jul 2023 15:20:57 +0000 Subject: [PATCH 579/628] Add production version --- .gitignore | 3 ++- oracle-database-operator.yaml | 2 +- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/.gitignore b/.gitignore index 0711d342..618e3efb 100644 --- a/.gitignore +++ b/.gitignore @@ -5,4 +5,5 @@ testbin/* onpremtest/* ords/*zip .gitattributes -.vscode \ No newline at end of file +.vscode +.gitlab-ci.yml diff --git a/oracle-database-operator.yaml b/oracle-database-operator.yaml index 52220c42..624a2c6c 100644 --- a/oracle-database-operator.yaml +++ b/oracle-database-operator.yaml @@ -3430,7 +3430,7 @@ spec: - --enable-leader-election command: - /manager - image: container-registry.oracle.com/database/operator:0.2.1 + image: container-registry.oracle.com/database/operator:1.0.0 imagePullPolicy: Always name: manager ports: From d7c14d92b193ca6ca2577990593cf316f81b4516 Mon Sep 17 00:00:00 2001 From: Yunus Qureshi Date: Thu, 13 Jul 2023 18:22:59 +0530 Subject: [PATCH 580/628] Operator yaml fix --- oracle-database-operator.yaml | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/oracle-database-operator.yaml b/oracle-database-operator.yaml index 52220c42..70f026c1 100644 --- a/oracle-database-operator.yaml +++ b/oracle-database-operator.yaml @@ -935,6 +935,10 @@ spec: - MaxPerformance - MaxAvailability type: string + serviceAnnotations: + additionalProperties: + type: string + type: object setAsPrimaryDatabase: type: string standbyDatabaseRefs: @@ -2214,6 +2218,7 @@ spec: - standard - enterprise - express + - free type: string enableTCPS: type: boolean @@ -3430,7 +3435,7 @@ spec: - --enable-leader-election command: - /manager - image: container-registry.oracle.com/database/operator:0.2.1 + image: container-registry.oracle.com/database/operator:1.0.0 imagePullPolicy: Always name: manager ports: From 236fb40aaf694c610243cffe7c761714eeb7550d Mon Sep 17 00:00:00 2001 From: Ting-Lan Wang Date: Fri, 25 Aug 2023 11:06:16 -0400 Subject: [PATCH 581/628] Update cert-manager command in README.md --- README.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 66cd62ec..5a4614ae 100644 --- a/README.md +++ b/README.md @@ -55,7 +55,11 @@ Oracle strongly recommends that you ensure your system meets the following [Prer Install the certificate manager with the following command: ```sh - kubectl apply -f https://github.com/jetstack/cert-manager/releases/latest/download/cert-manager.yaml + # Kubernetes 1.16+ + kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.1.1/cert-manager.yaml + + # Kubernetes <1.16 + kubectl apply --validate=false -f https://github.com/cert-manager/cert-manager/releases/download/v1.1.1/cert-manager-legacy.yaml ``` ## Quick Install of the Operator From 1c4bb5db8a04939383f4697423d941026b8ab967 Mon Sep 17 00:00:00 2001 From: Ting-Lan Wang Date: Mon, 28 Aug 2023 14:35:21 -0400 Subject: [PATCH 582/628] Update latest cert-manager command in README.md --- README.md | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/README.md b/README.md index 5a4614ae..614f6c65 100644 --- a/README.md +++ b/README.md @@ -55,11 +55,7 @@ Oracle strongly recommends that you ensure your system meets the following [Prer Install the certificate manager with the following command: ```sh - # Kubernetes 1.16+ - kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.1.1/cert-manager.yaml - - # Kubernetes <1.16 - kubectl apply --validate=false -f https://github.com/cert-manager/cert-manager/releases/download/v1.1.1/cert-manager-legacy.yaml + kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.12.0/cert-manager.yaml ``` ## Quick Install of the Operator From ec3f0ad5dc1911f7c7c7907e2bcc8e219fe2c2af Mon Sep 17 00:00:00 2001 From: Oleksandra Pavlusieva Date: Tue, 29 Aug 2023 12:31:17 +0300 Subject: [PATCH 583/628] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 614f6c65..a0ff79c9 100644 --- a/README.md +++ b/README.md @@ -86,7 +86,7 @@ You should see that the operator is up and running, along with the shipped contr For more details, see [Oracle Database Operator Installation Instructions](./docs/installation/OPERATOR_INSTALLATION_README.md). -## Getting Started with the Operator (Quickstart) +## Getting Started The quickstarts are designed for specific database configurations: From 4e2f86247a8cbf595b760f3ad57eaa8e8c6beffc Mon Sep 17 00:00:00 2001 From: Oleksandra Pavlusieva Date: Tue, 29 Aug 2023 12:31:58 +0300 Subject: [PATCH 584/628] Update SECURITY.md --- SECURITY.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/SECURITY.md b/SECURITY.md index fb238413..2ca81027 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -21,7 +21,7 @@ security features are welcome on GitHub Issues. Security updates will be released on a regular cadence. Many of our projects will typically release security fixes in conjunction with the -[Oracle Critical Patch Update][3] program. Additional +Oracle Critical Patch Update program. Additional information, including past advisories, is available on our [security alerts][4] page. From b6fb409659a31e12bccdb6b5e364361ec65993a1 Mon Sep 17 00:00:00 2001 From: Ishaan Date: Wed, 6 Sep 2023 18:41:46 +0530 Subject: [PATCH 585/628] Fixing getSidPdbEdition function --- controllers/database/singleinstancedatabase_controller.go | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index a4baf0a9..49f0c8a4 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1498,9 +1498,9 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex sid := m.Spec.Sid if m.Spec.Image.PrebuiltDB { edition := "" - sid, pdbName, edition = dbcommons.GetSidPdbEdition(r, r.Config, ctx, req) - if sid == "" || pdbName == "" || edition == "" { - return requeueN, nil + _, _, edition, err := dbcommons.GetSidPdbEdition(r, r.Config, ctx, ctrl.Request{NamespacedName: types.NamespacedName{Namespace: m.Namespace, Name: m.Name}}) + if err != nil { + return requeueY, err } m.Status.Edition = cases.Title(language.English).String(edition) } From 41a613ceb10de23ee5dc5f7ec4e64b40736eca7f Mon Sep 17 00:00:00 2001 From: Ishaan Date: Wed, 6 Sep 2023 18:44:05 +0530 Subject: [PATCH 586/628] Fixing getSidPdbEdition function-2 --- commons/database/constants.go | 6 ++--- commons/database/utils.go | 50 +++++++++++++++++------------------ 2 files changed, 28 insertions(+), 28 deletions(-) diff --git a/commons/database/constants.go b/commons/database/constants.go index 33dce251..de07c0d4 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -212,8 +212,8 @@ const DataguardBrokerAddDBMaxAvailabilityCMD string = "ADD DATABASE ${ORACLE_SID "(CONNECT_DATA=(SERVICE_NAME=${ORACLE_SID}_DGMGRL)(INSTANCE_NAME=${ORACLE_SID})(SERVER=DEDICATED)))';" + "\nENABLE CONFIGURATION;" -const RemoveStandbyDBFromDGConfgCMD string = "DISABLE DATABASE ${ORACLE_SID};" + - "\nREMOVE DATABASE ${ORACLE_SID};" +const RemoveStandbyDBFromDGConfgCMD string = "DISABLE DATABASE ${ORACLE_SID};" + + "\nREMOVE DATABASE ${ORACLE_SID};" const DBShowConfigCMD string = "SHOW CONFIGURATION;" @@ -506,7 +506,7 @@ const SetApexUsers string = "\numask 177" + "\numask 022" // Get Sid, Pdbname, Edition for prebuilt db -const GetSidPdbEditionCMD string = "echo $ORACLE_SID,$ORACLE_PDB,$ORACLE_EDITION,Edition;" +const GetSidPdbEditionCMD string = "echo $ORACLE_SID,$ORACLE_PDB,$ORACLE_EDITION;" // Command to enable TCPS as a formatted string. The parameter would be the port at which TCPS is enabled. const EnableTcpsCMD string = "$ORACLE_BASE/$CONFIG_TCPS_FILE" diff --git a/commons/database/utils.go b/commons/database/utils.go index c1ecd680..1dc46f18 100644 --- a/commons/database/utils.go +++ b/commons/database/utils.go @@ -507,23 +507,23 @@ func GetDatabaseRole(readyPod corev1.Pod, r client.Reader, func GetDatabaseOpenMode(readyPod corev1.Pod, r client.Reader, config *rest.Config, ctx context.Context, req ctrl.Request, edition string) (string, error) { - log := ctrllog.FromContext(ctx).WithValues("GetDatabaseOpenMode",req.NamespacedName) + log := ctrllog.FromContext(ctx).WithValues("GetDatabaseOpenMode", req.NamespacedName) - out,err := ExecCommand(r, config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf("echo -e \"%s\" | %s",GetDBOpenMode,SQLPlusCLI)) - if err != nil { - return "",err - } - log.Info(out) - if !strings.Contains(out, "no rows selected") && !strings.Contains(out, "ORA-") { - out1 := strings.Replace(out, " ", "_", -1) - // filtering output and storing databse_role in "database_role" - databaseOpenMode := strings.Fields(out1)[2] - // first 2 values in the slice will be column name(DATABASE_ROLE) and a seperator(--------------) . - return databaseOpenMode, nil - } - return "", errors.New("database open mode is nil") + out, err := ExecCommand(r, config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | %s", GetDBOpenMode, SQLPlusCLI)) + if err != nil { + return "", err } + log.Info(out) + if !strings.Contains(out, "no rows selected") && !strings.Contains(out, "ORA-") { + out1 := strings.Replace(out, " ", "_", -1) + // filtering output and storing databse_role in "database_role" + databaseOpenMode := strings.Fields(out1)[2] + // first 2 values in the slice will be column name(DATABASE_ROLE) and a seperator(--------------) . + return databaseOpenMode, nil + } + return "", errors.New("database open mode is nil") +} // Returns true if any of the pod in 'pods' is with pod.Status.Phase == phase func IsAnyPodWithStatus(pods []corev1.Pod, phase corev1.PodPhase) (bool, corev1.Pod) { @@ -590,29 +590,29 @@ func GetNodeIp(r client.Reader, ctx context.Context, req ctrl.Request) string { } // GetSidPdbEdition to display sid, pdbname, edition in ConnectionString -func GetSidPdbEdition(r client.Reader, config *rest.Config, ctx context.Context, req ctrl.Request) (string, string, string) { +func GetSidPdbEdition(r client.Reader, config *rest.Config, ctx context.Context, req ctrl.Request) (string, string, string, error) { - log := ctrllog.FromContext(ctx).WithValues("GetNodeIp", req.NamespacedName) + log := ctrllog.FromContext(ctx).WithValues("GetSidbPdbEdition", req.NamespacedName) - readyPod, _, _, _, err := FindPods(r, "", "", req.Name, req.Namespace, ctx, req) + sidbReadyPod, _, _, _, err := FindPods(r, "", "", req.Name, req.Namespace, ctx, req) if err != nil { log.Error(err, err.Error()) - return "", "", "" + return "", "", "", errors.New("error while fetching sidb ready pod for sidb " + req.Name) } - if readyPod.Name != "" { - out, err := ExecCommand(r, config, readyPod.Name, readyPod.Namespace, "", + if sidbReadyPod.Name != "" { + out, err := ExecCommand(r, config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", GetSidPdbEditionCMD) if err != nil { log.Error(err, err.Error()) - return "", "", "" + return "", "", "", errors.New("error while execing GetSidPdbEditionCMD on sidb " + req.Name) } splitstr := strings.Split(out, ",") - if len(splitstr) == 4 { - return splitstr[0], splitstr[1], splitstr[2] + if len(splitstr) == 3 { + return splitstr[0], splitstr[1], splitstr[2], nil } } - return "", "", "" + return "", "", "", errors.New("error while sidb ready pod name is nil") } // Get Datapatch Status From a8df0bb706821d236c72c6e56bb02caa334ef557 Mon Sep 17 00:00:00 2001 From: Ishaan Date: Thu, 7 Sep 2023 11:05:37 +0530 Subject: [PATCH 587/628] Refactor- svc creation after pod creation --- .../database/singleinstancedatabase_controller.go | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 49f0c8a4..0bdf5054 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -167,22 +167,22 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct return result, nil } - // Service creation - result, err = r.createOrReplaceSVC(ctx, req, singleInstanceDatabase) + // PVC Creation + result, err = r.createOrReplacePVC(ctx, req, singleInstanceDatabase) if result.Requeue { r.Log.Info("Reconcile queued") return result, nil } - // PVC Creation - result, err = r.createOrReplacePVC(ctx, req, singleInstanceDatabase) + // POD creation + result, err = r.createOrReplacePods(singleInstanceDatabase, cloneFromDatabase, referredPrimaryDatabase, ctx, req) if result.Requeue { r.Log.Info("Reconcile queued") return result, nil } - // POD creation - result, err = r.createOrReplacePods(singleInstanceDatabase, cloneFromDatabase, referredPrimaryDatabase, ctx, req) + // Service creation + result, err = r.createOrReplaceSVC(ctx, req, singleInstanceDatabase) if result.Requeue { r.Log.Info("Reconcile queued") return result, nil From a5772a8f85a28249bdcbb2046e9fc88b6f82708b Mon Sep 17 00:00:00 2001 From: Ishaan Date: Thu, 7 Sep 2023 12:16:56 +0530 Subject: [PATCH 588/628] adding logs to debug --- controllers/database/singleinstancedatabase_controller.go | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 0bdf5054..63cbb8cf 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1498,10 +1498,14 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex sid := m.Spec.Sid if m.Spec.Image.PrebuiltDB { edition := "" - _, _, edition, err := dbcommons.GetSidPdbEdition(r, r.Config, ctx, ctrl.Request{NamespacedName: types.NamespacedName{Namespace: m.Namespace, Name: m.Name}}) + sid, pdb, edition, err := dbcommons.GetSidPdbEdition(r, r.Config, ctx, ctrl.Request{NamespacedName: types.NamespacedName{Namespace: m.Namespace, Name: m.Name}}) if err != nil { return requeueY, err } + r.Log.Info("Sid for the database is " + sid) + r.Log.Info("Pdb for the database is " + pdb) + r.Log.Info("edition for the database is " + edition) + r.Log.Info("editon for the dataase is " + cases.Title(language.English).String(edition)) m.Status.Edition = cases.Title(language.English).String(edition) } From bf0698ba1478e20dddb1a9c2dd75a85bc0479b0c Mon Sep 17 00:00:00 2001 From: Ishaan Date: Thu, 7 Sep 2023 12:43:08 +0530 Subject: [PATCH 589/628] adding logs to debug-1 --- commons/database/utils.go | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/commons/database/utils.go b/commons/database/utils.go index 1dc46f18..ee9ff64c 100644 --- a/commons/database/utils.go +++ b/commons/database/utils.go @@ -594,11 +594,13 @@ func GetSidPdbEdition(r client.Reader, config *rest.Config, ctx context.Context, log := ctrllog.FromContext(ctx).WithValues("GetSidbPdbEdition", req.NamespacedName) + log.Info("Finding Pods for the database " + req.Name) sidbReadyPod, _, _, _, err := FindPods(r, "", "", req.Name, req.Namespace, ctx, req) if err != nil { log.Error(err, err.Error()) return "", "", "", errors.New("error while fetching sidb ready pod for sidb " + req.Name) } + log.Info("Sidb Ready Pod name is " + sidbReadyPod.Name) if sidbReadyPod.Name != "" { out, err := ExecCommand(r, config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", GetSidPdbEditionCMD) @@ -606,10 +608,9 @@ func GetSidPdbEdition(r client.Reader, config *rest.Config, ctx context.Context, log.Error(err, err.Error()) return "", "", "", errors.New("error while execing GetSidPdbEditionCMD on sidb " + req.Name) } + log.Info(out) splitstr := strings.Split(out, ",") - if len(splitstr) == 3 { - return splitstr[0], splitstr[1], splitstr[2], nil - } + return splitstr[0], splitstr[1], splitstr[2], nil } return "", "", "", errors.New("error while sidb ready pod name is nil") From 09f677ad876aa8086e5455279a14fcab8a41c55e Mon Sep 17 00:00:00 2001 From: Ishaan Date: Thu, 7 Sep 2023 13:23:14 +0530 Subject: [PATCH 590/628] adding logs to debug-3 --- controllers/database/singleinstancedatabase_controller.go | 2 ++ 1 file changed, 2 insertions(+) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 63cbb8cf..bb1e20cd 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -280,6 +280,7 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct // If LoadBalancer = true , ensure Connect String is updated if singleInstanceDatabase.Status.ConnectString == dbcommons.ValueUnavailable { + r.Log.Info("Connect String Not available for the database " + singleInstanceDatabase.Name) return requeueY, nil } @@ -1497,6 +1498,7 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex pdbName := strings.ToUpper(m.Spec.Pdbname) sid := m.Spec.Sid if m.Spec.Image.PrebuiltDB { + r.Log.Info("Initiliazing free database sid, pdb, edition") edition := "" sid, pdb, edition, err := dbcommons.GetSidPdbEdition(r, r.Config, ctx, ctrl.Request{NamespacedName: types.NamespacedName{Namespace: m.Namespace, Name: m.Name}}) if err != nil { From c93be49f1f2b55fcf9bc0f23bf5f13cd46c12e29 Mon Sep 17 00:00:00 2001 From: Ishaan Date: Thu, 7 Sep 2023 13:27:29 +0530 Subject: [PATCH 591/628] adding logs to debug-4 --- controllers/database/singleinstancedatabase_controller.go | 2 ++ 1 file changed, 2 insertions(+) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index bb1e20cd..86f2891b 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1511,6 +1511,8 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex m.Status.Edition = cases.Title(language.English).String(edition) } + r.Log.Info("Setting connect string statues") + if m.Spec.LoadBalancer { m.Status.ClusterConnectString = extSvc.Name + "." + extSvc.Namespace + ":" + fmt.Sprint(extSvc.Spec.Ports[1].Port) + "/" + strings.ToUpper(sid) if len(extSvc.Status.LoadBalancer.Ingress) > 0 { From d1ca212aed3641c5afe2ca04364110df5b11195a Mon Sep 17 00:00:00 2001 From: Ishaan Date: Thu, 7 Sep 2023 15:23:06 +0530 Subject: [PATCH 592/628] trimming the edition string before setting edition status --- controllers/database/singleinstancedatabase_controller.go | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 86f2891b..accc0960 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -131,11 +131,7 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct /* Initialize Status */ if singleInstanceDatabase.Status.Status == "" { singleInstanceDatabase.Status.Status = dbcommons.StatusPending - if singleInstanceDatabase.Spec.Edition != "" { - singleInstanceDatabase.Status.Edition = cases.Title(language.English).String(singleInstanceDatabase.Spec.Edition) - } else { - singleInstanceDatabase.Status.Edition = dbcommons.ValueUnavailable - } + singleInstanceDatabase.Status.Edition = dbcommons.ValueUnavailable singleInstanceDatabase.Status.Role = dbcommons.ValueUnavailable singleInstanceDatabase.Status.ConnectString = dbcommons.ValueUnavailable singleInstanceDatabase.Status.PdbConnectString = dbcommons.ValueUnavailable @@ -1508,7 +1504,7 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex r.Log.Info("Pdb for the database is " + pdb) r.Log.Info("edition for the database is " + edition) r.Log.Info("editon for the dataase is " + cases.Title(language.English).String(edition)) - m.Status.Edition = cases.Title(language.English).String(edition) + m.Status.Edition = strings.TrimSpace(cases.Title(language.English).String(edition)) } r.Log.Info("Setting connect string statues") From ea069ffaf4017efe004fd371e5e3cf55e99b613c Mon Sep 17 00:00:00 2001 From: Ishaan Date: Thu, 7 Sep 2023 15:46:12 +0530 Subject: [PATCH 593/628] Fixing reconcile --- controllers/database/singleinstancedatabase_controller.go | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index accc0960..515f761f 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -131,7 +131,11 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct /* Initialize Status */ if singleInstanceDatabase.Status.Status == "" { singleInstanceDatabase.Status.Status = dbcommons.StatusPending - singleInstanceDatabase.Status.Edition = dbcommons.ValueUnavailable + if singleInstanceDatabase.Spec.Edition != "" { + singleInstanceDatabase.Status.Edition = cases.Title(language.English).String(singleInstanceDatabase.Spec.Edition) + } else { + singleInstanceDatabase.Status.Edition = dbcommons.ValueUnavailable + } singleInstanceDatabase.Status.Role = dbcommons.ValueUnavailable singleInstanceDatabase.Status.ConnectString = dbcommons.ValueUnavailable singleInstanceDatabase.Status.PdbConnectString = dbcommons.ValueUnavailable From e135cfb1fb6dbd67003d9de88365acaf3044f84b Mon Sep 17 00:00:00 2001 From: Ishaan Date: Thu, 7 Sep 2023 17:20:58 +0530 Subject: [PATCH 594/628] removing logs and cleaning code --- commons/database/utils.go | 4 +--- controllers/database/singleinstancedatabase_controller.go | 8 ++------ 2 files changed, 3 insertions(+), 9 deletions(-) diff --git a/commons/database/utils.go b/commons/database/utils.go index ee9ff64c..c5599631 100644 --- a/commons/database/utils.go +++ b/commons/database/utils.go @@ -594,13 +594,11 @@ func GetSidPdbEdition(r client.Reader, config *rest.Config, ctx context.Context, log := ctrllog.FromContext(ctx).WithValues("GetSidbPdbEdition", req.NamespacedName) - log.Info("Finding Pods for the database " + req.Name) sidbReadyPod, _, _, _, err := FindPods(r, "", "", req.Name, req.Namespace, ctx, req) if err != nil { log.Error(err, err.Error()) return "", "", "", errors.New("error while fetching sidb ready pod for sidb " + req.Name) } - log.Info("Sidb Ready Pod name is " + sidbReadyPod.Name) if sidbReadyPod.Name != "" { out, err := ExecCommand(r, config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", GetSidPdbEditionCMD) @@ -609,7 +607,7 @@ func GetSidPdbEdition(r client.Reader, config *rest.Config, ctx context.Context, return "", "", "", errors.New("error while execing GetSidPdbEditionCMD on sidb " + req.Name) } log.Info(out) - splitstr := strings.Split(out, ",") + splitstr := strings.Split(strings.TrimSpace(out), ",") return splitstr[0], splitstr[1], splitstr[2], nil } diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 515f761f..61e2dccf 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1500,15 +1500,11 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex if m.Spec.Image.PrebuiltDB { r.Log.Info("Initiliazing free database sid, pdb, edition") edition := "" - sid, pdb, edition, err := dbcommons.GetSidPdbEdition(r, r.Config, ctx, ctrl.Request{NamespacedName: types.NamespacedName{Namespace: m.Namespace, Name: m.Name}}) + _, _, edition, err := dbcommons.GetSidPdbEdition(r, r.Config, ctx, ctrl.Request{NamespacedName: types.NamespacedName{Namespace: m.Namespace, Name: m.Name}}) if err != nil { return requeueY, err } - r.Log.Info("Sid for the database is " + sid) - r.Log.Info("Pdb for the database is " + pdb) - r.Log.Info("edition for the database is " + edition) - r.Log.Info("editon for the dataase is " + cases.Title(language.English).String(edition)) - m.Status.Edition = strings.TrimSpace(cases.Title(language.English).String(edition)) + m.Status.Edition = cases.Title(language.English).String(edition) } r.Log.Info("Setting connect string statues") From b02d97267180935c5a6788d2ac24cd722ba9e542 Mon Sep 17 00:00:00 2001 From: Ishaan Date: Fri, 8 Sep 2023 11:47:02 +0530 Subject: [PATCH 595/628] Enhancing logging and errors --- commons/database/utils.go | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/commons/database/utils.go b/commons/database/utils.go index c5599631..89ad7948 100644 --- a/commons/database/utils.go +++ b/commons/database/utils.go @@ -594,24 +594,25 @@ func GetSidPdbEdition(r client.Reader, config *rest.Config, ctx context.Context, log := ctrllog.FromContext(ctx).WithValues("GetSidbPdbEdition", req.NamespacedName) - sidbReadyPod, _, _, _, err := FindPods(r, "", "", req.Name, req.Namespace, ctx, req) + readyPod, _, _, _, err := FindPods(r, "", "", req.Name, req.Namespace, ctx, req) if err != nil { log.Error(err, err.Error()) - return "", "", "", errors.New("error while fetching sidb ready pod for sidb " + req.Name) + return "", "", "", fmt.Errorf("error while fetching ready pod %s : \n %s", readyPod.Name, err.Error()) } - if sidbReadyPod.Name != "" { - out, err := ExecCommand(r, config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", + if readyPod.Name != "" { + out, err := ExecCommand(r, config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", GetSidPdbEditionCMD) if err != nil { log.Error(err, err.Error()) - return "", "", "", errors.New("error while execing GetSidPdbEditionCMD on sidb " + req.Name) + return "", "", "", err } log.Info(out) splitstr := strings.Split(strings.TrimSpace(out), ",") return splitstr[0], splitstr[1], splitstr[2], nil } - - return "", "", "", errors.New("error while sidb ready pod name is nil") + err = errors.New("ready pod name is nil") + log.Error(err, err.Error()) + return "", "", "", err } // Get Datapatch Status From 7bcd33f1661d66e23ae6258b22866d0f8de295bf Mon Sep 17 00:00:00 2001 From: Ishaan Date: Fri, 8 Sep 2023 11:48:54 +0530 Subject: [PATCH 596/628] Removing title case from log --- controllers/database/singleinstancedatabase_controller.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 61e2dccf..c7d2f456 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -280,7 +280,7 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct // If LoadBalancer = true , ensure Connect String is updated if singleInstanceDatabase.Status.ConnectString == dbcommons.ValueUnavailable { - r.Log.Info("Connect String Not available for the database " + singleInstanceDatabase.Name) + r.Log.Info("Connect string not available for the database " + singleInstanceDatabase.Name) return requeueY, nil } From c5e5f25f1944c31f4a508ff6a5c188aceb925461 Mon Sep 17 00:00:00 2001 From: Ishaan Date: Fri, 8 Sep 2023 12:09:40 +0530 Subject: [PATCH 597/628] Enhancing logic for edition,sid,pdb fetching and logs --- commons/database/utils.go | 4 +-- .../singleinstancedatabase_controller.go | 25 +++++++++++-------- 2 files changed, 17 insertions(+), 12 deletions(-) diff --git a/commons/database/utils.go b/commons/database/utils.go index 89ad7948..6e6f6d02 100644 --- a/commons/database/utils.go +++ b/commons/database/utils.go @@ -606,8 +606,8 @@ func GetSidPdbEdition(r client.Reader, config *rest.Config, ctx context.Context, log.Error(err, err.Error()) return "", "", "", err } - log.Info(out) - splitstr := strings.Split(strings.TrimSpace(out), ",") + log.Info("GetSidPdbEditionCMD output \n" + out) + splitstr := strings.Split((strings.TrimSpace(out)), ",") return splitstr[0], splitstr[1], splitstr[2], nil } err = errors.New("ready pod name is nil") diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index c7d2f456..27c4a6e9 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1495,20 +1495,25 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex extSvc = svc } - pdbName := strings.ToUpper(m.Spec.Pdbname) - sid := m.Spec.Sid + var sid, pdbName string + var getSidPdbEditionErr error if m.Spec.Image.PrebuiltDB { - r.Log.Info("Initiliazing free database sid, pdb, edition") - edition := "" - _, _, edition, err := dbcommons.GetSidPdbEdition(r, r.Config, ctx, ctrl.Request{NamespacedName: types.NamespacedName{Namespace: m.Namespace, Name: m.Name}}) - if err != nil { - return requeueY, err + r.Log.Info("Initiliazing database sid, pdb, edition for prebuilt database") + var edition string + sid, pdbName, edition, getSidPdbEditionErr = dbcommons.GetSidPdbEdition(r, r.Config, ctx, ctrl.Request{NamespacedName: types.NamespacedName{Namespace: m.Namespace, Name: m.Name}}) + if getSidPdbEditionErr != nil { + return requeueY, getSidPdbEditionErr } + r.Log.Info("Prebuilt database: %s has SID : %s, PDB : %s, EDITION: %s", m.Name, sid, pdbName, edition) m.Status.Edition = cases.Title(language.English).String(edition) } - - r.Log.Info("Setting connect string statues") - + r.Log.Info("Prebuilt database: %s has SID : %s, PDB : %s, EDITION: %s", m.Name, sid, pdbName) + if sid == "" { + sid = strings.ToUpper(m.Spec.Sid) + } + if pdbName == "" { + pdbName = strings.ToUpper(m.Spec.Pdbname) + } if m.Spec.LoadBalancer { m.Status.ClusterConnectString = extSvc.Name + "." + extSvc.Namespace + ":" + fmt.Sprint(extSvc.Spec.Ports[1].Port) + "/" + strings.ToUpper(sid) if len(extSvc.Status.LoadBalancer.Ingress) > 0 { From 602626d383aee905f6bf8c4b084f245ddf3fc742 Mon Sep 17 00:00:00 2001 From: Ishaan Date: Fri, 8 Sep 2023 12:49:20 +0530 Subject: [PATCH 598/628] Fixing reconcile --- controllers/database/singleinstancedatabase_controller.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 27c4a6e9..ccf16c74 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1507,7 +1507,7 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex r.Log.Info("Prebuilt database: %s has SID : %s, PDB : %s, EDITION: %s", m.Name, sid, pdbName, edition) m.Status.Edition = cases.Title(language.English).String(edition) } - r.Log.Info("Prebuilt database: %s has SID : %s, PDB : %s, EDITION: %s", m.Name, sid, pdbName) + r.Log.Info("Prebuilt database: %s has SID : %s, PDB : %s", m.Name, sid, pdbName) if sid == "" { sid = strings.ToUpper(m.Spec.Sid) } From 8c07e6a9bd9d5749bd4780f6d75f4cbbc6b7bdc6 Mon Sep 17 00:00:00 2001 From: Ishaan Date: Fri, 8 Sep 2023 13:02:41 +0530 Subject: [PATCH 599/628] Fixing reconcile --- controllers/database/singleinstancedatabase_controller.go | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index ccf16c74..8ee7d8df 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1504,10 +1504,10 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex if getSidPdbEditionErr != nil { return requeueY, getSidPdbEditionErr } - r.Log.Info("Prebuilt database: %s has SID : %s, PDB : %s, EDITION: %s", m.Name, sid, pdbName, edition) + r.Log.Info(fmt.Sprintf("Prebuilt database: %s has SID : %s, PDB : %s, EDITION: %s", m.Name, sid, pdbName, edition)) m.Status.Edition = cases.Title(language.English).String(edition) } - r.Log.Info("Prebuilt database: %s has SID : %s, PDB : %s", m.Name, sid, pdbName) + r.Log.Info(fmt.Sprintf("Prebuilt database: %s has SID : %s, PDB : %s", m.Name, sid, pdbName)) if sid == "" { sid = strings.ToUpper(m.Spec.Sid) } From 8e49ed7725270bbae888b0b915689f9366cfabc4 Mon Sep 17 00:00:00 2001 From: Ishaan Date: Fri, 8 Sep 2023 13:09:49 +0530 Subject: [PATCH 600/628] Removing unecessary logs --- controllers/database/singleinstancedatabase_controller.go | 1 - 1 file changed, 1 deletion(-) diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index 8ee7d8df..e468214c 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1507,7 +1507,6 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex r.Log.Info(fmt.Sprintf("Prebuilt database: %s has SID : %s, PDB : %s, EDITION: %s", m.Name, sid, pdbName, edition)) m.Status.Edition = cases.Title(language.English).String(edition) } - r.Log.Info(fmt.Sprintf("Prebuilt database: %s has SID : %s, PDB : %s", m.Name, sid, pdbName)) if sid == "" { sid = strings.ToUpper(m.Spec.Sid) } From 2e9295afc0b3b0c776d40a25ac46b0a0d8ade2d5 Mon Sep 17 00:00:00 2001 From: ting-lan-wang Date: Tue, 26 Sep 2023 12:07:55 -0400 Subject: [PATCH 601/628] fix adb webhook and reconcile issue --- Makefile | 2 +- .../v1alpha1/autonomousdatabase_webhook.go | 4 +- .../database/autonomousdatabase_controller.go | 44 +++++++++++-------- 3 files changed, 29 insertions(+), 21 deletions(-) diff --git a/Makefile b/Makefile index e904262b..033b8a36 100644 --- a/Makefile +++ b/Makefile @@ -61,7 +61,7 @@ test: manifests generate fmt vet envtest ## Run unit tests. E2ETEST ?= ./test/e2e/ e2e: manifests generate fmt vet envtest ## Run e2e tests. - KUBEBUILDER_ASSETS="$(shell $(ENVTEST) use $(ENVTEST_K8S_VERSION) -p path)" ginkgo -v --timeout=2h30m --fail-fast $(E2ETEST) + KUBEBUILDER_ASSETS="$(shell $(ENVTEST) use $(ENVTEST_K8S_VERSION) -p path)" go test $(E2ETEST) -test.timeout 0 -test.v --ginkgo.fail-fast ##@ Build diff --git a/apis/database/v1alpha1/autonomousdatabase_webhook.go b/apis/database/v1alpha1/autonomousdatabase_webhook.go index 9c01ad95..bac49c70 100644 --- a/apis/database/v1alpha1/autonomousdatabase_webhook.go +++ b/apis/database/v1alpha1/autonomousdatabase_webhook.go @@ -164,7 +164,9 @@ func (r *AutonomousDatabase) ValidateUpdate(old runtime.Object) error { // cannot change lifecycleState with other fields together (except the oci config) var lifecycleChanged, otherFieldsChanged bool - lifecycleChanged = oldADB.Spec.Details.LifecycleState != "" && oldADB.Spec.Details.LifecycleState != r.Spec.Details.LifecycleState + lifecycleChanged = oldADB.Spec.Details.LifecycleState != "" && + r.Spec.Details.LifecycleState != "" && + oldADB.Spec.Details.LifecycleState != r.Spec.Details.LifecycleState var copiedADB *AutonomousDatabaseSpec = r.Spec.DeepCopy() copiedADB.Details.LifecycleState = oldADB.Spec.Details.LifecycleState copiedADB.OCIConfig = oldADB.Spec.OCIConfig diff --git a/controllers/database/autonomousdatabase_controller.go b/controllers/database/autonomousdatabase_controller.go index 379ab86b..770a3fa4 100644 --- a/controllers/database/autonomousdatabase_controller.go +++ b/controllers/database/autonomousdatabase_controller.go @@ -157,10 +157,15 @@ func (r *AutonomousDatabaseReconciler) eventFilterPredicate() predicate.Predicat if adbOk { oldADB := e.ObjectOld.(*dbv1alpha1.AutonomousDatabase) - if !reflect.DeepEqual(oldADB.Status, desiredADB.Status) || - (controllerutil.ContainsFinalizer(oldADB, dbv1alpha1.LastSuccessfulSpec) != controllerutil.ContainsFinalizer(desiredADB, dbv1alpha1.LastSuccessfulSpec)) || + statusChanged := !reflect.DeepEqual(oldADB.Status, desiredADB.Status) + + oldLastSucSpec := oldADB.GetAnnotations()[dbv1alpha1.LastSuccessfulSpec] + desiredLastSucSpec := desiredADB.GetAnnotations()[dbv1alpha1.LastSuccessfulSpec] + lastSucSpecChanged := oldLastSucSpec != desiredLastSucSpec + + if statusChanged || lastSucSpecChanged || (controllerutil.ContainsFinalizer(oldADB, dbv1alpha1.ADBFinalizer) != controllerutil.ContainsFinalizer(desiredADB, dbv1alpha1.ADBFinalizer)) { - // Don't enqueue if the status, lastSucSpec, or the finalizler changes + // Don't enqueue if the status, lastSucSpec, or the finalizler changes the first time return false } @@ -458,8 +463,6 @@ func (r *AutonomousDatabaseReconciler) validateOperation( } else { l.Info("No operation specified; sync the resource") - testOldADB := adb.DeepCopy() - // The user doesn't change the spec and the controller should pull the spec from the OCI. specChanged, err := r.getADB(logger, adb) if err != nil { @@ -470,12 +473,12 @@ func (r *AutonomousDatabaseReconciler) validateOperation( l.Info("The local spec doesn't match the oci's spec; update the CR") // Erase the status.lifecycleState temporarily to avoid the webhook error. - oldADB := adb.DeepCopy() + tmpADB := adb.DeepCopy() adb.Status.LifecycleState = "" - r.KubeClient.Status().Update(context.TODO(), adb) - adb.Spec = oldADB.Spec - - adb.DeepCopy().RemoveUnchangedDetails(testOldADB.Spec) + if err := r.KubeClient.Status().Update(context.TODO(), adb); err != nil { + return false, emptyResult, err + } + adb.Spec = tmpADB.Spec if err := r.updateCR(adb); err != nil { return false, emptyResult, err @@ -580,7 +583,9 @@ func (r *AutonomousDatabaseReconciler) validateFinalizer(logger logr.Logger, adb // updateCR updates the lastSucSpec and the CR func (r *AutonomousDatabaseReconciler) updateCR(adb *dbv1alpha1.AutonomousDatabase) error { // Update the lastSucSpec - if err := adb.UpdateLastSuccessfulSpec(); err != nil { + // Should patch the lastSuccessfulSpec first, otherwise, the update event will be + // filtered out by predicate since the lastSuccessfulSpec is changed. + if err := r.patchLastSuccessfulSpec(adb); err != nil { return err } @@ -933,22 +938,23 @@ func (r *AutonomousDatabaseReconciler) validateDesiredLifecycleState( } // The logic of updating the network access configurations is as follows: -// 1. Shared databases: -// If the network access type changes -// a. to PUBLIC: +// +// 1. Shared databases: +// If the network access type changes +// a. to PUBLIC: // was RESTRICTED: re-enable IsMTLSConnectionRequired if its not. Then set WhitelistedIps to an array with a single empty string entry. // was PRIVATE: re-enable IsMTLSConnectionRequired if its not. Then set PrivateEndpointLabel to an emtpy string. -// b. to RESTRICTED: +// b. to RESTRICTED: // was PUBLIC: set WhitelistedIps to desired IPs/CIDR blocks/VCN OCID. Configure the IsMTLSConnectionRequired settings if it is set to disabled. // was PRIVATE: re-enable IsMTLSConnectionRequired if its not. Set the type to PUBLIC first, and then configure the WhitelistedIps. Finally resume the IsMTLSConnectionRequired settings if it was, or is configured as disabled. -// c. to PRIVATE: +// c. to PRIVATE: // was PUBLIC: set subnetOCID and nsgOCIDs. Configure the IsMTLSConnectionRequired settings if it is set. // was RESTRICTED: set subnetOCID and nsgOCIDs. Configure the IsMTLSConnectionRequired settings if it is set. // -// Otherwise, if the network access type remains the same, apply the network configuration, and then set the IsMTLSConnectionRequired. +// Otherwise, if the network access type remains the same, apply the network configuration, and then set the IsMTLSConnectionRequired. // -// 2. Dedicated databases: -// Apply the configs directly +// 2. Dedicated databases: +// Apply the configs directly func (r *AutonomousDatabaseReconciler) validateGeneralNetworkAccess( logger logr.Logger, adb *dbv1alpha1.AutonomousDatabase, From 1096f1f93e4468d08715adf33802aedf18487050 Mon Sep 17 00:00:00 2001 From: ting-lan-wang Date: Tue, 3 Oct 2023 17:40:56 -0400 Subject: [PATCH 602/628] fix adb reconcile --- .../database/autonomousdatabase_controller.go | 45 ++++++++++--------- 1 file changed, 24 insertions(+), 21 deletions(-) diff --git a/controllers/database/autonomousdatabase_controller.go b/controllers/database/autonomousdatabase_controller.go index 770a3fa4..54ff3431 100644 --- a/controllers/database/autonomousdatabase_controller.go +++ b/controllers/database/autonomousdatabase_controller.go @@ -157,15 +157,17 @@ func (r *AutonomousDatabaseReconciler) eventFilterPredicate() predicate.Predicat if adbOk { oldADB := e.ObjectOld.(*dbv1alpha1.AutonomousDatabase) + specChanged := !reflect.DeepEqual(oldADB.Spec, desiredADB.Spec) statusChanged := !reflect.DeepEqual(oldADB.Status, desiredADB.Status) oldLastSucSpec := oldADB.GetAnnotations()[dbv1alpha1.LastSuccessfulSpec] desiredLastSucSpec := desiredADB.GetAnnotations()[dbv1alpha1.LastSuccessfulSpec] lastSucSpecChanged := oldLastSucSpec != desiredLastSucSpec - if statusChanged || lastSucSpecChanged || + if (!specChanged && statusChanged) || lastSucSpecChanged || (controllerutil.ContainsFinalizer(oldADB, dbv1alpha1.ADBFinalizer) != controllerutil.ContainsFinalizer(desiredADB, dbv1alpha1.ADBFinalizer)) { - // Don't enqueue if the status, lastSucSpec, or the finalizler changes the first time + // Don't enqueue in the folowing condition: + // 1. only status changes 2. lastSucSpec changes 3. ADBFinalizer changes return false } @@ -276,31 +278,36 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R } /****************************************************************** - * Requeue if it's in an intermediate state. Update the status right before - * exiting the reconcile, otherwise the modifiedADB will be overwritten - * by the object returned from the cluster. + * Update the resource if the spec has been changed. + * Requeue if it's in an intermediate state. Update the status first + * , otherwise the modifiedADB will be overwritten by the object + * returned from the cluster. ******************************************************************/ - if dbv1alpha1.IsADBIntermediateState(modifiedADB.Status.LifecycleState) { - logger.WithName("IsADBIntermediateState").Info("LifecycleState is " + string(modifiedADB.Status.LifecycleState) + "; reconcile queued") - - if err := r.KubeClient.Status().Update(context.TODO(), modifiedADB); err != nil { + if !reflect.DeepEqual(modifiedADB.Spec, desiredADB.Spec) { + if err := r.KubeClient.Update(context.TODO(), modifiedADB); err != nil { return r.manageError(logger.WithName("IsADBIntermediateState"), modifiedADB, err) } + return emptyResult, nil + } + copiedADB := modifiedADB.DeepCopy() + if err := r.KubeClient.Status().Update(context.TODO(), modifiedADB); err != nil { + return r.manageError(logger.WithName("Status().Update"), modifiedADB, err) + } + modifiedADB.Spec = copiedADB.Spec + + if dbv1alpha1.IsADBIntermediateState(modifiedADB.Status.LifecycleState) { + logger.WithName("IsADBIntermediateState").Info("LifecycleState is " + string(modifiedADB.Status.LifecycleState) + "; reconcile queued") return requeueResult, nil } /****************************************************************** - * Update the lastSucSpec and the status, and then finish the reconcile. - * Requeue if it's in an intermediate state or the modifiedADB - * doesn't match the desiredADB. + * Update the lastSucSpec, and then finish the reconcile. + * Requeue if the ADB is terminated, but the finalizer is not yet + * removed. ******************************************************************/ - // Do the comparison before updating the status to avoid being overwritten - var requeue bool = false - if !reflect.DeepEqual(modifiedADB.Spec, desiredADB.Spec) { - requeue = true - } + var requeue bool = false if modifiedADB.GetDeletionTimestamp() != nil && controllerutil.ContainsFinalizer(modifiedADB, dbv1alpha1.ADBFinalizer) && modifiedADB.Status.LifecycleState == database.AutonomousDatabaseLifecycleStateTerminated { @@ -312,10 +319,6 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R return r.manageError(logger.WithName("patchLastSuccessfulSpec"), modifiedADB, err) } - if err := r.KubeClient.Status().Update(context.TODO(), modifiedADB); err != nil { - return r.manageError(logger.WithName("Status().Update"), modifiedADB, err) - } - if requeue { logger.Info("Reconcile queued") return requeueResult, nil From 1683616f1a6283c5015fd034b45a817f714188e0 Mon Sep 17 00:00:00 2001 From: Paramdeep Saini Date: Fri, 10 May 2024 05:21:45 +0000 Subject: [PATCH 603/628] Added support for v1.1.0 --- .gitlab-ci.yml | 31 - Dockerfile | 13 +- Makefile | 31 +- PROJECT | 13 + README.md | 109 +- SECURITY.md | 2 +- THIRD_PARTY_LICENSES.txt | 46 +- .../autonomouscontainerdatabase_webhook.go | 17 +- .../v1alpha1/autonomousdatabase_types.go | 20 +- .../v1alpha1/autonomousdatabase_webhook.go | 45 +- .../autonomousdatabasebackup_types.go | 3 +- .../autonomousdatabasebackup_webhook.go | 31 +- .../autonomousdatabaserestore_webhook.go | 31 +- apis/database/v1alpha1/cdb_types.go | 4 +- apis/database/v1alpha1/cdb_webhook.go | 47 +- .../v1alpha1/dataguardbroker_webhook.go | 59 +- apis/database/v1alpha1/dbcssystem_types.go | 5 +- .../v1alpha1/oraclerestdataservice_webhook.go | 35 +- apis/database/v1alpha1/pdb_types.go | 19 + apis/database/v1alpha1/pdb_webhook.go | 62 +- .../v1alpha1/shardingdatabase_types.go | 136 +- .../v1alpha1/singleinstancedatabase_types.go | 23 +- .../singleinstancedatabase_webhook.go | 261 +-- apis/database/v1alpha1/webhook_suite_test.go | 14 +- .../v1alpha1/zz_generated.deepcopy.go | 188 ++- .../v1alpha1/databaseobserver_types.go | 144 ++ .../v1alpha1/databaseobserver_webhook.go | 185 +++ .../v1alpha1/groupversion_info.go | 58 + .../v1alpha1/zz_generated.deepcopy.go | 298 ++++ commons/database/constants.go | 19 +- commons/database/utils.go | 126 +- commons/dbcssystem/dbcs_reconciler.go | 6 + commons/observability/constants.go | 172 ++ commons/observability/utils.go | 402 +++++ commons/oci/database.go | 4 +- commons/oci/provider.go | 32 +- commons/oci/wallet.go | 20 +- commons/sharding/catalog.go | 74 +- commons/sharding/exec.go | 73 + commons/sharding/gsm.go | 46 +- commons/sharding/provstatus.go | 8 +- commons/sharding/scommon.go | 270 +++- commons/sharding/shard.go | 74 +- .../bases/database.oracle.com_DbcsSystem.yaml | 240 +++ ....oracle.com_autonomousdatabasebackups.yaml | 4 + ...tabase.oracle.com_autonomousdatabases.yaml | 75 + .../database.oracle.com_dataguardbrokers.yaml | 4 + .../crd/bases/database.oracle.com_pdbs.yaml | 38 + ...database.oracle.com_shardingdatabases.yaml | 262 ++- ...se.oracle.com_singleinstancedatabases.yaml | 35 +- ...vability.oracle.com_databaseobservers.yaml | 227 +++ config/crd/kustomization.yaml | 3 + .../cainjenction_in_databaseobservers.yaml | 8 + .../patches/webhook_in_databaseobservers.yaml | 17 + config/manager/kustomization.yaml | 4 +- config/manager/manager.yaml | 3 + config/rbac/auth_proxy_role_binding.yaml | 4 +- config/rbac/databaseobserver_editor_role.yaml | 24 + config/rbac/databaseobserver_viewer_role.yaml | 20 + config/rbac/role.yaml | 119 +- config/rbac/role_binding.yaml | 2 +- .../adb/autonomousdatabase_backup.yaml | 4 +- .../adb/autonomousdatabase_create.yaml | 7 +- ...onomousdatabase_update_network_access.yaml | 2 +- config/samples/kustomization.yaml | 3 +- .../observability/databaseobserver.yaml | 44 + .../databaseobserver_custom_config.yaml | 28 + .../databaseobserver_minimal.yaml | 22 + .../observability/databaseobserver_vault.yaml | 25 + .../observability/sample-dashboard.json | 1414 +++++++++++++++++ .../samples/observability/sample_config.toml | 29 + config/samples/sidb/dataguardbroker.yaml | 2 +- .../samples/sidb/oraclerestdataservice.yaml | 5 +- .../samples/sidb/singleinstancedatabase.yaml | 62 +- .../sidb/singleinstancedatabase_clone.yaml | 7 +- .../sidb/singleinstancedatabase_free.yaml | 2 +- .../sidb/singleinstancedatabase_standby.yaml | 8 +- .../sidb/singleinstancedatabase_tcps.yaml | 7 +- config/webhook/manifests.yaml | 40 + .../database/autonomousdatabase_controller.go | 162 +- controllers/database/cdb_controller.go | 119 +- .../database/dataguardbroker_controller.go | 55 +- .../oraclerestdataservice_controller.go | 5 +- controllers/database/pdb_controller.go | 230 +-- .../database/shardingdatabase_controller.go | 295 +++- .../singleinstancedatabase_controller.go | 981 +++++++++--- .../databaseobserver_controller.go | 547 +++++++ .../databaseobserver_resource.go | 181 +++ controllers/observability/suite_test.go | 100 ++ docs/adb/ACD.md | 4 +- ...NUAL_BACKUP.md => ADB_LONG_TERM_BACKUP.md} | 18 +- docs/adb/ADB_PREREQUISITES.md | 36 +- docs/adb/ADB_RESTORE.md | 2 +- docs/adb/README.md | 14 +- docs/dbcs/provisioning/database_connection.md | 53 + .../dbcs_service_with_minimal_parameters.md | 13 +- .../dbcs_service_with_minimal_parameters.yaml | 22 +- ..._with_minimal_parameters_sample_output.log | 302 ++-- docs/dbcs/usecase01/README.md | 199 +++ docs/multitenant/NamespaceSeg.md | 14 + docs/multitenant/README.md | 19 +- docs/multitenant/images/K8S_NAMESPACE_SEG.png | Bin 0 -> 269014 bytes docs/multitenant/usecase01/README.md | 5 + docs/multitenant/usecase01/cdb_create.yaml | 44 + docs/multitenant/usecase01/cdb_secret.yaml | 17 + docs/multitenant/usecase01/makefile | 62 +- .../usecase01/oracle-database-operator.yaml | 1 + docs/multitenant/usecase01/pdb_close.yaml | 44 + docs/multitenant/usecase01/pdb_create.yaml | 46 + docs/multitenant/usecase01/pdb_map.yaml | 44 + docs/multitenant/usecase01/pdb_open.yaml | 43 + docs/multitenant/usecase01/pdb_secret.yaml | 16 + docs/multitenant/usecase01/tde_secret.yaml | 17 + docs/multitenant/usecase02/README.md | 35 +- docs/multitenant/usecase02/pdb_clone.yaml | 13 +- docs/multitenant/usecase02/pdb_plug.yaml | 12 +- docs/multitenant/usecase02/pdb_unplug.yaml | 12 +- docs/multitenant/usecase03/Dockerfile | 53 + .../usecase03/NamespaceSegregation.png | Bin 0 -> 270813 bytes docs/multitenant/usecase03/README.md | 266 ++++ docs/multitenant/usecase03/cdb_create.yaml | 44 + .../usecase03/cdb_creation_log.txt | 336 ++++ docs/multitenant/usecase03/cdb_secret.yaml | 17 + docs/multitenant/usecase03/gentlscert.sh | 23 + docs/multitenant/usecase03/makefile | 291 ++++ .../usecase03/ns_namespace_cdb.yaml | 7 + .../usecase03/ns_namespace_pdb.yaml | 7 + .../usecase03/operator_creation_log.txt | 27 + docs/multitenant/usecase03/pdb_create.yaml | 46 + .../usecase03/pdb_creation_log.txt | 6 + docs/multitenant/usecase03/pdb_secret.yaml | 16 + docs/observability/README.md | 256 +++ docs/sharding/README.md | 69 +- .../create_kubernetes_secret_for_db_user.md | 49 +- docs/sharding/provisioning/oraclesi.yaml | 27 +- .../provisioning/oraclesi_pvc_commented.yaml | 58 +- ..._persistent_volume_having_db_gold_image.md | 2 +- docs/sharding/provisioning/shard_prov.yaml | 46 - .../provisioning/shard_prov_clone.yaml | 64 - .../shard_prov_clone_across_ads.yaml | 72 - .../shard_prov_send_notification.yaml | 75 - ..._cloning_db_from_gold_image_across_ads.md} | 25 +- ...ng_by_cloning_db_gold_image_in_same_ad.md} | 25 +- ...provisioning_with_control_on_resources.md} | 20 +- ...th_notification_using_oci_notification.md} | 26 +- ...ing_provisioning_without_db_gold_image.md} | 20 +- ...ding_scale_in_delete_an_existing_shard.md} | 20 +- .../ssharding_scale_out_add_shards.md} | 24 +- .../system_sharding/ssharding_shard_prov.yaml | 58 + .../ssharding_shard_prov_clone.yaml | 82 + ...ssharding_shard_prov_clone_across_ads.yaml | 82 + .../ssharding_shard_prov_delshard.yaml | 68 + .../ssharding_shard_prov_extshard.yaml | 67 + .../ssharding_shard_prov_memory_cpu.yaml} | 51 +- ...sharding_shard_prov_send_notification.yaml | 85 + ...y_cloning_db_from_gold_image_across_ads.md | 53 + ...ing_by_cloning_db_gold_image_in_same_ad.md | 49 + ..._provisioning_with_control_on_resources.md | 42 + ...ith_notification_using_oci_notification.md | 82 + ...ding_provisioning_without_db_gold_image.md | 37 + ...rding_scale_in_delete_an_existing_shard.md | 47 + .../udsharding_scale_out_add_shards.md | 34 + .../udsharding_shard_prov.yaml | 59 + .../udsharding_shard_prov_clone.yaml | 83 + ...dsharding_shard_prov_clone_across_ads.yaml | 83 + .../udsharding_shard_prov_delshard.yaml} | 50 +- .../udsharding_shard_prov_extshard.yaml} | 49 +- .../udsharding_shard_prov_memory_cpu.yaml | 90 ++ ...sharding_shard_prov_send_notification.yaml | 85 + docs/sidb/PREREQUISITES.md | 16 +- docs/sidb/README.md | 174 +- go.mod | 136 +- go.sum | 799 ++-------- image.png | Bin 0 -> 6913 bytes main.go | 86 +- oracle-database-operator.yaml | 648 +++++++- rbac/cluster-role-binding.yaml | 15 + rbac/default-ns-role-binding.yaml | 15 + rbac/node-rbac.yaml | 27 + rbac/persistent-volume-rbac.yaml | 28 + rbac/storage-class-rbac.yaml | 28 + test/e2e/resource/test_config.yaml | 2 +- test/e2e/util/util.go | 2 +- 183 files changed, 12605 insertions(+), 2562 deletions(-) delete mode 100644 .gitlab-ci.yml create mode 100644 apis/observability/v1alpha1/databaseobserver_types.go create mode 100644 apis/observability/v1alpha1/databaseobserver_webhook.go create mode 100644 apis/observability/v1alpha1/groupversion_info.go create mode 100644 apis/observability/v1alpha1/zz_generated.deepcopy.go create mode 100644 commons/observability/constants.go create mode 100644 commons/observability/utils.go create mode 100644 config/crd/bases/database.oracle.com_DbcsSystem.yaml create mode 100644 config/crd/bases/observability.oracle.com_databaseobservers.yaml create mode 100644 config/crd/patches/cainjenction_in_databaseobservers.yaml create mode 100644 config/crd/patches/webhook_in_databaseobservers.yaml create mode 100644 config/rbac/databaseobserver_editor_role.yaml create mode 100644 config/rbac/databaseobserver_viewer_role.yaml create mode 100644 config/samples/observability/databaseobserver.yaml create mode 100644 config/samples/observability/databaseobserver_custom_config.yaml create mode 100644 config/samples/observability/databaseobserver_minimal.yaml create mode 100644 config/samples/observability/databaseobserver_vault.yaml create mode 100644 config/samples/observability/sample-dashboard.json create mode 100644 config/samples/observability/sample_config.toml create mode 100644 controllers/observability/databaseobserver_controller.go create mode 100644 controllers/observability/databaseobserver_resource.go create mode 100644 controllers/observability/suite_test.go rename docs/adb/{ADB_MANUAL_BACKUP.md => ADB_LONG_TERM_BACKUP.md} (70%) create mode 100644 docs/dbcs/usecase01/README.md create mode 100644 docs/multitenant/NamespaceSeg.md create mode 100644 docs/multitenant/images/K8S_NAMESPACE_SEG.png create mode 100644 docs/multitenant/usecase01/cdb_create.yaml create mode 100644 docs/multitenant/usecase01/cdb_secret.yaml create mode 120000 docs/multitenant/usecase01/oracle-database-operator.yaml create mode 100644 docs/multitenant/usecase01/pdb_close.yaml create mode 100644 docs/multitenant/usecase01/pdb_create.yaml create mode 100644 docs/multitenant/usecase01/pdb_map.yaml create mode 100644 docs/multitenant/usecase01/pdb_open.yaml create mode 100644 docs/multitenant/usecase01/pdb_secret.yaml create mode 100644 docs/multitenant/usecase01/tde_secret.yaml create mode 100644 docs/multitenant/usecase03/Dockerfile create mode 100644 docs/multitenant/usecase03/NamespaceSegregation.png create mode 100644 docs/multitenant/usecase03/README.md create mode 100644 docs/multitenant/usecase03/cdb_create.yaml create mode 100644 docs/multitenant/usecase03/cdb_creation_log.txt create mode 100644 docs/multitenant/usecase03/cdb_secret.yaml create mode 100644 docs/multitenant/usecase03/gentlscert.sh create mode 100644 docs/multitenant/usecase03/makefile create mode 100644 docs/multitenant/usecase03/ns_namespace_cdb.yaml create mode 100644 docs/multitenant/usecase03/ns_namespace_pdb.yaml create mode 100644 docs/multitenant/usecase03/operator_creation_log.txt create mode 100644 docs/multitenant/usecase03/pdb_create.yaml create mode 100644 docs/multitenant/usecase03/pdb_creation_log.txt create mode 100644 docs/multitenant/usecase03/pdb_secret.yaml create mode 100644 docs/observability/README.md delete mode 100644 docs/sharding/provisioning/shard_prov.yaml delete mode 100644 docs/sharding/provisioning/shard_prov_clone.yaml delete mode 100644 docs/sharding/provisioning/shard_prov_clone_across_ads.yaml delete mode 100644 docs/sharding/provisioning/shard_prov_send_notification.yaml rename docs/sharding/provisioning/{provisioning_by_cloning_db_from_gold_image_across_ads.md => system_sharding/ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md} (57%) rename docs/sharding/provisioning/{provisioning_by_cloning_db_gold_image_in_same_ad.md => system_sharding/ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md} (54%) rename docs/sharding/provisioning/{provisioning_with_control_on_resources.md => system_sharding/ssharding_provisioning_with_control_on_resources.md} (58%) rename docs/sharding/provisioning/{provisioning_with_notification_using_oci_notification.md => system_sharding/ssharding_provisioning_with_notification_using_oci_notification.md} (71%) rename docs/sharding/provisioning/{provisioning_without_db_gold_image.md => system_sharding/ssharding_provisioning_without_db_gold_image.md} (59%) rename docs/sharding/provisioning/{scale_in_delete_an_existing_shard.md => system_sharding/ssharding_scale_in_delete_an_existing_shard.md} (56%) rename docs/sharding/provisioning/{scale_out_add_shards.md => system_sharding/ssharding_scale_out_add_shards.md} (51%) create mode 100644 docs/sharding/provisioning/system_sharding/ssharding_shard_prov.yaml create mode 100644 docs/sharding/provisioning/system_sharding/ssharding_shard_prov_clone.yaml create mode 100644 docs/sharding/provisioning/system_sharding/ssharding_shard_prov_clone_across_ads.yaml create mode 100644 docs/sharding/provisioning/system_sharding/ssharding_shard_prov_delshard.yaml create mode 100644 docs/sharding/provisioning/system_sharding/ssharding_shard_prov_extshard.yaml rename docs/sharding/provisioning/{shard_prov_memory_cpu.yaml => system_sharding/ssharding_shard_prov_memory_cpu.yaml} (60%) create mode 100644 docs/sharding/provisioning/system_sharding/ssharding_shard_prov_send_notification.yaml create mode 100644 docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_from_gold_image_across_ads.md create mode 100644 docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_gold_image_in_same_ad.md create mode 100644 docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_with_control_on_resources.md create mode 100644 docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_with_notification_using_oci_notification.md create mode 100644 docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_without_db_gold_image.md create mode 100644 docs/sharding/provisioning/user-defined-sharding/udsharding_scale_in_delete_an_existing_shard.md create mode 100644 docs/sharding/provisioning/user-defined-sharding/udsharding_scale_out_add_shards.md create mode 100644 docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov.yaml create mode 100644 docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_clone.yaml create mode 100644 docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_clone_across_ads.yaml rename docs/sharding/provisioning/{shard_prov_extshard.yaml => user-defined-sharding/udsharding_shard_prov_delshard.yaml} (51%) rename docs/sharding/provisioning/{shard_prov_delshard.yaml => user-defined-sharding/udsharding_shard_prov_extshard.yaml} (50%) create mode 100644 docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_memory_cpu.yaml create mode 100644 docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_send_notification.yaml create mode 100644 image.png create mode 100644 rbac/cluster-role-binding.yaml create mode 100644 rbac/default-ns-role-binding.yaml create mode 100644 rbac/node-rbac.yaml create mode 100644 rbac/persistent-volume-rbac.yaml create mode 100644 rbac/storage-class-rbac.yaml diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml deleted file mode 100644 index d5080ff9..00000000 --- a/.gitlab-ci.yml +++ /dev/null @@ -1,31 +0,0 @@ -build-operator: - stage: build - variables: - IMAGE: "$DOCKER_REPO:$CI_COMMIT_BRANCH" - OP_YAML: oracle-database-operator.yaml - script: - - go version - - echo $CI_COMMIT_SHORT_SHA - - make docker-build IMG="$IMAGE" - - docker push "$IMAGE" - - newimage=$DOCKER_REPO@$(skopeo inspect docker://$IMAGE | jq -r .Digest) - - echo $newimage - - docker rmi "$IMAGE" && docker system prune -f - - make operator-yaml IMG=$newimage - - if [ "$CI_COMMIT_BRANCH" != "master" ]; then sed -i "s/\(replicas.\) 3/\1 1/g" ./$OP_YAML; fi - - curl -s --netrc-file $HOME/.netrc_gitlab $ARTIFACTORY_REPO/$CI_COMMIT_BRANCH/$OP_YAML -T ./$OP_YAML - only: - variables: - - $CI_COMMIT_MESSAGE =~ /\#run-pipeline/ - - $CI_COMMIT_BRANCH =~ /master/ - - $CI_MERGE_REQUEST_ID != "" - except: - variables: - - $CI_COMMIT_MESSAGE =~ /\#skip-pipeline/ - - $CI_COMMIT_TAG != null - -cleanup: - stage: .post - script: - - echo "Clean up downloaded binaries" - - rm -rf bin/ diff --git a/Dockerfile b/Dockerfile index 3c249825..11a56962 100644 --- a/Dockerfile +++ b/Dockerfile @@ -3,7 +3,18 @@ # # Build the manager binary -FROM golang:1.19 as builder +ARG BUILDER_IMG +FROM ${BUILDER_IMG} as builder + +# Download golang if BUILD_INTERNAL is set to true +ARG INSTALL_GO +ARG GOLANG_VERSION +RUN if [ "$INSTALL_GO" = "true" ]; then \ + curl -LJO https://go.dev/dl/go${GOLANG_VERSION}.linux-amd64.tar.gz &&\ + rm -rf /usr/local/go && tar -C /usr/local -xzf go${GOLANG_VERSION}.linux-amd64.tar.gz &&\ + rm go${GOLANG_VERSION}.linux-amd64.tar.gz; \ + fi +ENV PATH=${GOLANG_VERSION:+"${PATH}:/usr/local/go/bin"} WORKDIR /workspace # Copy the Go Modules manifests diff --git a/Makefile b/Makefile index 033b8a36..88e14843 100644 --- a/Makefile +++ b/Makefile @@ -71,12 +71,23 @@ build: generate fmt vet ## Build manager binary. run: manifests generate fmt vet ## Run a controller from your host. go run ./main.go -docker-build: manifests generate fmt vet #test ## Build docker image with the manager. Disable the test but keep the validations to fail fast - docker build --no-cache=true --build-arg http_proxy=${HTTP_PROXY} --build-arg https_proxy=${HTTPS_PROXY} \ - --build-arg CI_COMMIT_SHA=${CI_COMMIT_SHA} --build-arg CI_COMMIT_BRANCH=${CI_COMMIT_BRANCH} . -t ${IMG} +GOLANG_VERSION ?= 1.21.7 +## Download golang in the Dockerfile if BUILD_INTERNAL is set to true. +## Otherwise, use golang image from docker hub as the builder. +ifeq ($(BUILD_INTERNAL), true) +BUILDER_IMG = oraclelinux:8 +BUILD_ARGS = --build-arg BUILDER_IMG=$(BUILDER_IMG) --build-arg GOLANG_VERSION=$(GOLANG_VERSION) --build-arg INSTALL_GO=true +else +BUILDER_IMG = golang:$(GOLANG_VERSION) +BUILD_ARGS = --build-arg BUILDER_IMG=$(BUILDER_IMG) --build-arg INSTALL_GO=false +endif +docker-build: #manifests generate fmt vet #test ## Build docker image with the manager. Disable the test but keep the validations to fail fast + docker build --no-cache=true --build-arg http_proxy=$(HTTP_PROXY) --build-arg https_proxy=$(HTTPS_PROXY) \ + --build-arg CI_COMMIT_SHA=$(CI_COMMIT_SHA) --build-arg CI_COMMIT_BRANCH=$(CI_COMMIT_BRANCH) \ + $(BUILD_ARGS) . -t $(IMG) docker-push: ## Push docker image with the manager. - docker push ${IMG} + docker push $(IMG) ##@ Deployment @@ -87,17 +98,17 @@ uninstall: manifests kustomize ## Uninstall CRDs from the K8s cluster specified $(KUSTOMIZE) build config/crd | kubectl delete -f - deploy: manifests kustomize ## Deploy controller to the K8s cluster specified in ~/.kube/config. - cd config/manager && $(KUSTOMIZE) edit set image controller=${IMG} + cd config/manager && $(KUSTOMIZE) edit set image controller=$(IMG) $(KUSTOMIZE) build config/default | kubectl apply -f - # Bug:34265574 # Used sed to reposition the controller-manager Deployment after the certificate creation in the OPERATOR_YAML operator-yaml: manifests kustomize - cd config/manager && $(KUSTOMIZE) edit set image controller=${IMG} - $(KUSTOMIZE) build config/default > "${OPERATOR_YAML}" - sed -i.bak -e '/^apiVersion: apps\/v1/,/---/d' "${OPERATOR_YAML}" - (echo --- && sed '/^apiVersion: apps\/v1/,/---/!d' "${OPERATOR_YAML}.bak") >> "${OPERATOR_YAML}" - rm "${OPERATOR_YAML}.bak" + cd config/manager && $(KUSTOMIZE) edit set image controller=$(IMG) + $(KUSTOMIZE) build config/default > "$(OPERATOR_YAML)" + sed -i.bak -e '/^apiVersion: apps\/v1/,/---/d' "$(OPERATOR_YAML)" + (echo --- && sed '/^apiVersion: apps\/v1/,/---/!d' "$(OPERATOR_YAML).bak") >> "$(OPERATOR_YAML)" + rm "$(OPERATOR_YAML).bak" undeploy: ## Undeploy controller from the K8s cluster specified in ~/.kube/config. $(KUSTOMIZE) build config/default | kubectl delete -f - diff --git a/PROJECT b/PROJECT index c402ecfd..7b3ec718 100644 --- a/PROJECT +++ b/PROJECT @@ -136,4 +136,17 @@ resources: defaulting: true validation: true webhookVersion: v1beta1 +- api: + crdVersion: v1beta1 + namespaced: true + controller: true + domain: oracle.com + group: observability + kind: DatabaseObserver + path: github.com/oracle/oracle-database-operator/apis/observability/v1alpha1 + version: v1alpha1 + webhooks: + defaulting: true + validation: true + webhookVersion: v1beta1 version: "3" diff --git a/README.md b/README.md index a0ff79c9..ed684a8a 100644 --- a/README.md +++ b/README.md @@ -4,31 +4,48 @@ As part of Oracle's resolution to make Oracle Database Kubernetes native (that is, observable and operable by Kubernetes), Oracle released _Oracle Database Operator for Kubernetes_ (`OraOperator` or the operator). OraOperator extends the Kubernetes API with custom resources and controllers for automating Oracle Database lifecycle management. -In this v1.0.0 production release, `OraOperator` supports the following database configurations and infrastructure: +In this v1.1.0 production release, `OraOperator` supports the following database configurations and infrastructure: * Oracle Autonomous Database: * Oracle Autonomous Database shared Oracle Cloud Infrastructure (OCI) (ADB-S) * Oracle Autonomous Database on dedicated Cloud infrastructure (ADB-D) - * Oracle Autonomous Container Database (ACD) (infrastructure) the infrastructure for provisionning Autonomous Databases. + * Oracle Autonomous Container Database (ACD) (infrastructure) the infrastructure for provisioning Autonomous Databases. * Containerized Single Instance databases (SIDB) deployed in the Oracle Kubernetes Engine (OKE) and any k8s where OraOperator is deployed * Containerized Sharded databases (SHARDED) deployed in OKE and any k8s where OraOperator is deployed * Oracle Multitenant Databases (CDB/PDBs) * Oracle Base Database Cloud Service (BDBCS) * Oracle Data Guard (Preview status) +* Oracle Database Observability (Preview status) Oracle will continue to extend `OraOperator` to support additional Oracle Database configurations. +## New in V1.1.0 Release +* Namespace scope deployment option +* Support for Oracle Database 23ai Free (with SIDB) +* Automatic Storage Expansion for SIDB and Sharded DB +* User-Defined Sharding +* TCPS support customer provided certs +* Execute custom scripts during DB setup/startup +* Patching for SIDB Primary/Standby in Data Guard +* Long-term backup for Autonomous Databases (ADB): Moves to long-term backup and removes the deprecated mandatory backup +* Wallet expiry date for ADB: A user-freindly enhancement to display wallet expiry date in the status of assiciated ADB +* Wait-for-Completion option for ADB: Supports `kubectl wait` command that allows user to wait a specific condition on ADB +* OKE workload Identify: Supports OKE workload indentity authentication method. For more details, refer to [Oracle Autonomous Database (ADB) Prerequisites](docs/adb/ADB_PREREQUISITES.md#authorized-with-oke-workload-identity) +* Database Observability (Preview - Metrics) + ## Features Summary This release of Oracle Database Operator for Kubernetes (the operator) supports the following lifecycle operations: -* ADB-S/ADB-D: Provision, Bind, Start, Stop, terminate (soft/hard), scale (up/down), Manual Backup, Manual Restore +* ADB-S/ADB-D: Provision, bind, start, stop, terminate (soft/hard), scale (up/down), long-term backup, manual restore * ACD: provision, bind, restart, terminate (soft/hard) * SIDB: Provision, clone, patch (in-place/out-of-place), update database initialization parameters, update database configuration (Flashback, archiving), Oracle Enterprise Manager (EM) Express (a basic observability console), Oracle REST Data Service (ORDS) to support REST based SQL, PDB management, SQL Developer Web, and Application Express (Apex) * SHARDED: Provision/deploy sharded databases and the shard topology, Add a new shard, Delete an existing shard * Oracle Multitenant Database: Bind to a CDB, Create a  PDB, Plug a  PDB, Unplug a PDB, Delete a PDB, Clone a PDB, Open/Close a PDB * Oracle Base Database Cloud Service (BDBCS): provision, bind, scale shape Up/Down, Scale Storage Up, Terminate and Update License * Oracle Data Guard: Provision a Standby for the SIDB resource, Create a Data Guard Configuration, Perform a Switchover, Patch Primary and Standby databases in Data Guard Configuration +* Oracle Database Observability: create, patch, delete databaseObserver resources +* Watch over a set of namespaces or all the namespaces in the cluster using the "WATCH_NAMESPACE" env variable of the operator deployment The upcoming releases will support new configurations, operations and capabilities. @@ -55,19 +72,70 @@ Oracle strongly recommends that you ensure your system meets the following [Prer Install the certificate manager with the following command: ```sh - kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.12.0/cert-manager.yaml + kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.14.4/cert-manager.yaml ``` -## Quick Install of the Operator +* ### Create Role Bindings for Access Management + + OraOperator supports the following two modes of deployment: + ##### 1. Cluster Scoped Deployment + + This is the default mode, in which OraOperator is deployed to operate in a cluster, and to monitor all the namespaces in the cluster. + + - Grant the `serviceaccount:oracle-database-operator-system:default` cluster wide access for the resources by applying [cluster-role-binding.yaml](./rbac/cluster-role-binding.yaml) + + ```sh + kubectl apply -f rbac/cluster-role-binding.yaml + ``` + + - Next, apply the [oracle-database-operator.yaml](./oracle-database-operator.yaml) to deploy the Operator + + ```sh + kubectl apply -f oracle-database-operator.yaml + ``` + + ##### 2. Namespace Scoped Deployment - To install the operator in the cluster quickly, you can use a single [oracle-database-operator.yaml](https://github.com/oracle/oracle-database-operator/blob/main/oracle-database-operator.yaml) file. + In this mode, OraOperator can be deployed to operate in a namespace, and to monitor one or many namespaces. - Run the following command + - Grant `serviceaccount:oracle-database-operator-system:default` service account with resource access in the required namespaces. For example, to monitor only the default namespace, apply the [default-ns-role-binding.yaml](./rbac/default-ns-role-binding.yaml) + + ```sh + kubectl apply -f rbac/default-ns-role-binding.yaml + ``` + To watch additional namespaces, create different role binding files for each namespace, using [default-ns-role-binding.yaml](./rbac/default-ns-role-binding.yaml) as a template, and changing the `metadata.name` and `metadata.namespace` fields + + - Next, edit the [oracle-database-operator.yaml](./oracle-database-operator.yaml) to add the required namespaces under `WATCH_NAMESPACE`. Use comma-delimited values for multiple namespaces. + + ```sh + - name: WATCH_NAMESPACE + value: "default" + ``` + - Finally, apply the edited [oracle-database-operator.yaml](./oracle-database-operator.yaml) to deploy the Operator + + ```sh + kubectl apply -f oracle-database-operator.yaml + ``` + + +* ### ClusterRole and ClusterRoleBinding for NodePort services + + To expose services on each node's IP and port (the NodePort) apply the [node-rbac.yaml](./rbac/node-rbac.yaml). Note that this step is not required for LoadBalancer services. ```sh - kubectl apply -f https://raw.githubusercontent.com/oracle/oracle-database-operator/main/oracle-database-operator.yaml + kubectl apply -f rbac/node-rbac.yaml ``` +## Install Oracle DB Operator + + After you have completed the preceding prerequisite changes, you can install the operator. To install the operator in the cluster quickly, you can apply the modified `oracle-database-operator.yaml` file from the preceding step. + + Run the following command + + ```sh + kubectl apply -f oracle-database-operator.yaml + ``` + Ensure that the operator pods are up and running. For high availability, Operator pod replicas are set to a default of 3. You can scale this setting up or down. ```sh @@ -86,9 +154,9 @@ You should see that the operator is up and running, along with the shipped contr For more details, see [Oracle Database Operator Installation Instructions](./docs/installation/OPERATOR_INSTALLATION_README.md). -## Getting Started +## Getting Started with the Operator (Quickstart) -The quickstarts are designed for specific database configurations: +The following quickstarts are designed for specific database configurations: * [Oracle Autonomous Database](./docs/adb/README.md) * [Oracle Autonomous Container Database](./docs/adb/ACD.md) @@ -97,13 +165,17 @@ The quickstarts are designed for specific database configurations: * [Oracle Multitenant Database](./docs/multitenant/README.md) * [Oracle Base Database Cloud Service (BDBCS)](./docs/dbcs/README.md) + +The following quickstart is designed for non-database configurations: +* [Oracle Database Observability](./docs/observability/README.md) + YAML file templates are available under [`/config/samples`](./config/samples/). You can copy and edit these template files to configure them for your use cases. ## Uninstall the Operator To uninstall the operator, the final step consists of deciding whether you want to delete the custom resource definitions (CRDs) and Kubernetes APIServices introduced into the cluster by the operator. Choose one of the following options: -* ### Deleting the CRDs and APIServices +* ### Delete the CRDs and APIServices To delete all the CRD instances deployed to cluster by the operator, run the following commands, where is the namespace of the cluster object: @@ -119,8 +191,17 @@ YAML file templates are available under [`/config/samples`](./config/samples/). kubectl delete cdb.database.oracle.com --all -n kubectl delete pdb.database.oracle.com --all -n kubectl delete dataguardbrokers.database.oracle.com --all -n + kubectl delete databaseobserver.observability.oracle.com --all -n + ``` + +* ### Delete the RBACs + + ```sh + cat rbac/* | kubectl delete -f - ``` +* ### Delete the Deployment + After all CRD instances are deleted, it is safe to remove the CRDs, APIServices and operator deployment. To remove these files, use the following command: ```sh @@ -129,7 +210,7 @@ YAML file templates are available under [`/config/samples`](./config/samples/). Note: If the CRD instances are not deleted, and the operator is deleted by using the preceding command, then operator deployment and instance objects (pods, services, PVCs, and so on) are deleted. However, if that happens, then the CRD deletion stops responding. This is because the CRD instances have properties that prevent their deletion, and that can only be removed by the operator pod, which is deleted when the APIServices are deleted. -## Docs of the supported Oracle Database configurations +## Documentation for the supported Oracle Database configurations * [Oracle Autonomous Database](https://docs.oracle.com/en-us/iaas/Content/Database/Concepts/adboverview.htm) * [Components of Dedicated Autonomous Database](https://docs.oracle.com/en-us/iaas/autonomous-database/doc/components.html) @@ -143,7 +224,7 @@ See [Contributing to this Repository](./CONTRIBUTING.md) ## Support -You can submit a GitHub issue, and/or you file an [Oracle Support service](https://support.oracle.com/portal/) request, using this product ID: 14430. +You can submit a GitHub issue, oir submit an issue and then file an [Oracle Support service](https://support.oracle.com/portal/) request. To file an issue or a service request, use the following product ID: 14430. ## Security @@ -168,5 +249,5 @@ See [Reporting security vulnerabilities](./SECURITY.md) ## License -Copyright (c) 2022, 2023 Oracle and/or its affiliates. +Copyright (c) 2022, 2024 Oracle and/or its affiliates. Released under the Universal Permissive License v1.0 as shown at [https://oss.oracle.com/licenses/upl/](https://oss.oracle.com/licenses/upl/) diff --git a/SECURITY.md b/SECURITY.md index 2ca81027..fb238413 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -21,7 +21,7 @@ security features are welcome on GitHub Issues. Security updates will be released on a regular cadence. Many of our projects will typically release security fixes in conjunction with the -Oracle Critical Patch Update program. Additional +[Oracle Critical Patch Update][3] program. Additional information, including past advisories, is available on our [security alerts][4] page. diff --git a/THIRD_PARTY_LICENSES.txt b/THIRD_PARTY_LICENSES.txt index 74a065b6..d75f3946 100644 --- a/THIRD_PARTY_LICENSES.txt +++ b/THIRD_PARTY_LICENSES.txt @@ -1,5 +1,5 @@ ------------------------------------- -Operator SDK 1.26.1 +Operator SDK 1.32.0 https://github.com/operator-framework/operator-sdk Apache 2.0 @@ -208,7 +208,7 @@ Apache License: limitations under the License. ------------------------------ - GO lang + GO lang 1.21.4 https://github.com/golang @@ -241,18 +241,18 @@ THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. ------------------------- -apimachinery 0.27.2 +apimachinery 0.28.4 https://github.com/kubernetes/apimachinery/tr Apache 2.0 ------------------------- -controller-runtime 0.15.0 -https://github.com/kubernetes-sigs/controller-runtime/tree/v0.14.6 +controller-runtime 0.16.3 +https://github.com/kubernetes-sigs/controller-runtime/releases/tag/v0.16.3 Apache 2.0 ------------------------- -golang 1.20.2 -https://github.com/golang/go/archive/refs/tags/go1.20.2.zip +golang 1.21.4 +https://github.com/golang/go/releases/tag/go1.21.4 BSD 2-clause or 3-clause @@ -1008,14 +1008,14 @@ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. limitations under the License. -------------------------- -Logr +Logr 1.3.0 https://pkg.go.dev/github.com/go-logr/logr -https://github.com/go-logr/logr/tree/v1.2.4 +https://github.com/go-logr/logr/tree/v1.3.0 Apache 2.0 License ------------------------- -OCI Go SDK -github.com/oracle/oci-go-sdk/v43 +OCI Go SDK 65.53.0 +https://github.com/oracle/oci-go-sdk/releases/tag/v65.53.0 Dual-License: UPL + Apache 2.0 @@ -1065,8 +1065,8 @@ The Universal Permissive License (UPL), Version 1.0 ------------------------- -ginkgo 2.9.7 -https://github.com/onsi/ginkgo/tree/v2.9.7 +ginkgo 2.13.1 +https://github.com/onsi/ginkgo/releases/tag/v2.13.1 MIT ------------------------------------ Gomega @@ -1075,37 +1075,37 @@ MIT License Copyright (c) 2013-2014 Onsi Fakhouri ---------------------------- -gomega 1.27.7 +gomega 1.30.0 http://onsi.github.io/gomega/ MIT ------------------------- -Kubernetes api +Kubernetes api 0.28.4 https://pkg.go.dev/k8s.io/api Apache 2.0 ---------------------------------- -Kubernetes apimachinery +Kubernetes apimachinery 0.28.4 https://pkg.go.dev/k8s.io/apimachinery Apache 2.0 ----------------------------------- -Kubernetes client-go +Kubernetes client-go 0.28.4 https://pkg.go.dev/k8s.io/client-go Apache 2.0 ------------------------------------- -Kubernetes controller-runtime project +Kubernetes controller-runtime project 0.16.3 https://pkg.go.dev/sigs.k8s.io/controller-runtime Apache 2.0 ------------------------------------ -kubernetes-sigs/yaml 1.3.0 +kubernetes-sigs/yaml 1.4.0 https://github.com/kubernetes-sigs/yaml/tree/v1.3.0 MIT ------------------------- -OCI SDK for Go 65.37.1 +OCI SDK for Go 65.53.0 https://github.com/oracle/oci-go-sdk Multiple Licenses: Apache 2.0, UPL @@ -1135,8 +1135,8 @@ https://pkg.go.dev/sigs.k8s.io/yaml Dual license: BSD-3-Clause, MIT ------------------------------------ -zap 1.24.0 -https://github.com/uber-go/zap/tree/v1.24.0 +zap 1.26.0 +https://github.com/uber-go/zap/releases/tag/v1.26.0 MIT ------------------------------------ @@ -1192,7 +1192,7 @@ THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. ------------------------ -Ginkgo +Ginkgo 2.13.1 github.com/onsi/ginkgo MIT License Copyright (c) 2013-2014 Onsi Fakhouri diff --git a/apis/database/v1alpha1/autonomouscontainerdatabase_webhook.go b/apis/database/v1alpha1/autonomouscontainerdatabase_webhook.go index 3e9f5e1f..37e1d819 100644 --- a/apis/database/v1alpha1/autonomouscontainerdatabase_webhook.go +++ b/apis/database/v1alpha1/autonomouscontainerdatabase_webhook.go @@ -46,6 +46,7 @@ import ( ctrl "sigs.k8s.io/controller-runtime" logf "sigs.k8s.io/controller-runtime/pkg/log" "sigs.k8s.io/controller-runtime/pkg/webhook" + "sigs.k8s.io/controller-runtime/pkg/webhook/admission" ) // log is for logging in this package. @@ -62,15 +63,15 @@ func (r *AutonomousContainerDatabase) SetupWebhookWithManager(mgr ctrl.Manager) var _ webhook.Validator = &AutonomousContainerDatabase{} // ValidateCreate implements webhook.Validator so a webhook will be registered for the type -func (r *AutonomousContainerDatabase) ValidateCreate() error { +func (r *AutonomousContainerDatabase) ValidateCreate() (admission.Warnings, error) { autonomouscontainerdatabaselog.Info("validate create", "name", r.Name) // TODO(user): fill in your validation logic upon object creation. - return nil + return nil, nil } // ValidateUpdate implements webhook.Validator so a webhook will be registered for the type -func (r *AutonomousContainerDatabase) ValidateUpdate(old runtime.Object) error { +func (r *AutonomousContainerDatabase) ValidateUpdate(old runtime.Object) (admission.Warnings, error) { var allErrs field.ErrorList var oldACD *AutonomousContainerDatabase = old.(*AutonomousContainerDatabase) @@ -78,7 +79,7 @@ func (r *AutonomousContainerDatabase) ValidateUpdate(old runtime.Object) error { // skip the update of adding ADB OCID or binding if oldACD.Status.LifecycleState == "" { - return nil + return nil, nil } // cannot update when the old state is in intermediate state, except for the terminate operatrion @@ -95,17 +96,17 @@ func (r *AutonomousContainerDatabase) ValidateUpdate(old runtime.Object) error { } if len(allErrs) == 0 { - return nil + return nil, nil } - return apierrors.NewInvalid( + return nil, apierrors.NewInvalid( schema.GroupKind{Group: "database.oracle.com", Kind: "AutonomousContainerDatabase"}, r.Name, allErrs) } // ValidateDelete implements webhook.Validator so a webhook will be registered for the type -func (r *AutonomousContainerDatabase) ValidateDelete() error { +func (r *AutonomousContainerDatabase) ValidateDelete() (admission.Warnings, error) { autonomouscontainerdatabaselog.Info("validate delete", "name", r.Name) // TODO(user): fill in your validation logic upon object deletion. - return nil + return nil, nil } diff --git a/apis/database/v1alpha1/autonomousdatabase_types.go b/apis/database/v1alpha1/autonomousdatabase_types.go index 413c16de..cd23b3f3 100644 --- a/apis/database/v1alpha1/autonomousdatabase_types.go +++ b/apis/database/v1alpha1/autonomousdatabase_types.go @@ -43,14 +43,14 @@ import ( "reflect" "github.com/oracle/oci-go-sdk/v65/database" - metaV1 "k8s.io/apimachinery/pkg/apis/meta/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" ) // EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! // NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized. // name of our custom finalizer -const ADBFinalizer = "database.oracle.com/adb-finalizer" +const ADB_FINALIZER = "database.oracle.com/adb-finalizer" // AutonomousDatabaseSpec defines the desired state of AutonomousDatabase // Important: Run "make" to regenerate code after modifying this file @@ -159,7 +159,13 @@ type AutonomousDatabaseStatus struct { // Important: Run "make" to regenerate code after modifying this file LifecycleState database.AutonomousDatabaseLifecycleStateEnum `json:"lifecycleState,omitempty"` TimeCreated string `json:"timeCreated,omitempty"` + WalletExpiringDate string `json:"walletExpiringDate,omitempty"` AllConnectionStrings []ConnectionStringProfile `json:"allConnectionStrings,omitempty"` + // +patchMergeKey=type + // +patchStrategy=merge + // +listType=map + // +listMapKey=type + Conditions []metav1.Condition `json:"conditions,omitempty"` } type TLSAuthenticationEnum string @@ -192,8 +198,8 @@ type ConnectionStringSpec struct { // +kubebuilder:printcolumn:JSONPath=".spec.details.dbWorkload",name="Workload Type",type=string // +kubebuilder:printcolumn:JSONPath=".status.timeCreated",name="Created",type=string type AutonomousDatabase struct { - metaV1.TypeMeta `json:",inline"` - metaV1.ObjectMeta `json:"metadata,omitempty"` + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` Spec AutonomousDatabaseSpec `json:"spec,omitempty"` Status AutonomousDatabaseStatus `json:"status,omitempty"` @@ -203,8 +209,8 @@ type AutonomousDatabase struct { // AutonomousDatabaseList contains a list of AutonomousDatabase type AutonomousDatabaseList struct { - metaV1.TypeMeta `json:",inline"` - metaV1.ListMeta `json:"metadata,omitempty"` + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata,omitempty"` Items []AutonomousDatabase `json:"items"` } @@ -329,7 +335,7 @@ func (adb *AutonomousDatabase) UpdateFromOCIADB(ociObj database.AutonomousDataba if *ociObj.IsDedicated { adb.Spec.Details.NetworkAccess.AccessType = NetworkAccessTypePrivate } else { - if ociObj.NsgIds != nil { + if ociObj.NsgIds != nil || ociObj.PrivateEndpoint != nil || ociObj.PrivateEndpointIp != nil || ociObj.PrivateEndpointLabel != nil { adb.Spec.Details.NetworkAccess.AccessType = NetworkAccessTypePrivate } else if ociObj.WhitelistedIps != nil { adb.Spec.Details.NetworkAccess.AccessType = NetworkAccessTypeRestricted diff --git a/apis/database/v1alpha1/autonomousdatabase_webhook.go b/apis/database/v1alpha1/autonomousdatabase_webhook.go index bac49c70..b25e8104 100644 --- a/apis/database/v1alpha1/autonomousdatabase_webhook.go +++ b/apis/database/v1alpha1/autonomousdatabase_webhook.go @@ -43,6 +43,7 @@ import ( "github.com/oracle/oci-go-sdk/v65/common" "github.com/oracle/oci-go-sdk/v65/database" + dbcommons "github.com/oracle/oracle-database-operator/commons/database" apierrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/runtime" "k8s.io/apimachinery/pkg/runtime/schema" @@ -50,6 +51,7 @@ import ( ctrl "sigs.k8s.io/controller-runtime" logf "sigs.k8s.io/controller-runtime/pkg/log" "sigs.k8s.io/controller-runtime/pkg/webhook" + "sigs.k8s.io/controller-runtime/pkg/webhook/admission" ) // log is for logging in this package. @@ -97,12 +99,24 @@ var _ webhook.Validator = &AutonomousDatabase{} // ValidateCreate implements webhook.Validator so a webhook will be registered for the type // ValidateCreate checks if the spec is valid for a provisioning or a binding operation -func (r *AutonomousDatabase) ValidateCreate() error { - +func (r *AutonomousDatabase) ValidateCreate() (admission.Warnings, error) { var allErrs field.ErrorList autonomousdatabaselog.Info("validate create", "name", r.Name) + namespaces := dbcommons.GetWatchNamespaces() + _, hasEmptyString := namespaces[""] + isClusterScoped := len(namespaces) == 1 && hasEmptyString + if !isClusterScoped { + _, containsNamespace := namespaces[r.Namespace] + // Check if the allowed namespaces maps contains the required namespace + if len(namespaces) != 0 && !containsNamespace { + allErrs = append(allErrs, + field.Invalid(field.NewPath("metadata").Child("namespace"), r.Namespace, + "Oracle database operator doesn't watch over this namespace")) + } + } + if r.Spec.Details.AutonomousDatabaseOCID == nil { // provisioning operation allErrs = validateCommon(r, allErrs) allErrs = validateNetworkAccess(r, allErrs) @@ -115,15 +129,15 @@ func (r *AutonomousDatabase) ValidateCreate() error { } if len(allErrs) == 0 { - return nil + return nil, nil } - return apierrors.NewInvalid( + return nil, apierrors.NewInvalid( schema.GroupKind{Group: "database.oracle.com", Kind: "AutonomousDatabase"}, r.Name, allErrs) } // ValidateUpdate implements webhook.Validator so a webhook will be registered for the type -func (r *AutonomousDatabase) ValidateUpdate(old runtime.Object) error { +func (r *AutonomousDatabase) ValidateUpdate(old runtime.Object) (admission.Warnings, error) { var allErrs field.ErrorList var oldADB *AutonomousDatabase = old.(*AutonomousDatabase) @@ -131,7 +145,7 @@ func (r *AutonomousDatabase) ValidateUpdate(old runtime.Object) error { // skip the update of adding ADB OCID or binding if oldADB.Status.LifecycleState == "" { - return nil + return nil, nil } // cannot update when the old state is in intermediate, except for the change to the hardLink or the terminate operatrion during valid lifecycleState @@ -187,9 +201,9 @@ func (r *AutonomousDatabase) ValidateUpdate(old runtime.Object) error { allErrs = validateNetworkAccess(r, allErrs) if len(allErrs) == 0 { - return nil + return nil, nil } - return apierrors.NewInvalid( + return nil, apierrors.NewInvalid( schema.GroupKind{Group: "database.oracle.com", Kind: "AutonomousDatabase"}, r.Name, allErrs) } @@ -226,12 +240,13 @@ func validateNetworkAccess(adb *AutonomousDatabase, allErrs field.ErrorList) fie field.Forbidden(field.NewPath("spec").Child("details").Child("networkAccess").Child("privateEndpoint").Child("subnetOCID"), fmt.Sprintf("subnetOCID cannot be empty when the network access type is %s", NetworkAccessTypePrivate))) } + } - if adb.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs == nil { - allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec").Child("details").Child("networkAccess").Child("privateEndpoint").Child("nsgOCIDs"), - fmt.Sprintf("nsgOCIDs cannot be empty when the network access type is %s", NetworkAccessTypePrivate))) - } + // NsgOCIDs only applies to PRIVATE accessType + if adb.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs != nil && adb.Spec.Details.NetworkAccess.AccessType != NetworkAccessTypePrivate { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("details").Child("networkAccess").Child("privateEndpoint").Child("nsgOCIDs"), + fmt.Sprintf("NsgOCIDs cannot only be applied when network access type is %s.", NetworkAccessTypePrivate))) } // IsAccessControlEnabled is not applicable to a shared database @@ -263,11 +278,11 @@ func validateNetworkAccess(adb *AutonomousDatabase, allErrs field.ErrorList) fie } // ValidateDelete implements webhook.Validator so a webhook will be registered for the type -func (r *AutonomousDatabase) ValidateDelete() error { +func (r *AutonomousDatabase) ValidateDelete() (admission.Warnings, error) { autonomousdatabaselog.Info("validate delete", "name", r.Name) // TODO(user): fill in your validation logic upon object deletion. - return nil + return nil, nil } // Returns true if AutonomousContainerDatabaseOCID has value. diff --git a/apis/database/v1alpha1/autonomousdatabasebackup_types.go b/apis/database/v1alpha1/autonomousdatabasebackup_types.go index d64e549b..876cb811 100644 --- a/apis/database/v1alpha1/autonomousdatabasebackup_types.go +++ b/apis/database/v1alpha1/autonomousdatabasebackup_types.go @@ -55,7 +55,8 @@ type AutonomousDatabaseBackupSpec struct { Target TargetSpec `json:"target,omitempty"` DisplayName *string `json:"displayName,omitempty"` AutonomousDatabaseBackupOCID *string `json:"autonomousDatabaseBackupOCID,omitempty"` - + IsLongTermBackup *bool `json:"isLongTermBackup,omitempty"` + RetentionPeriodInDays *int `json:"retentionPeriodInDays,omitempty"` OCIConfig OCIConfigSpec `json:"ociConfig,omitempty"` } diff --git a/apis/database/v1alpha1/autonomousdatabasebackup_webhook.go b/apis/database/v1alpha1/autonomousdatabasebackup_webhook.go index c08c8c58..99bf3815 100644 --- a/apis/database/v1alpha1/autonomousdatabasebackup_webhook.go +++ b/apis/database/v1alpha1/autonomousdatabasebackup_webhook.go @@ -39,6 +39,7 @@ package v1alpha1 import ( + dbcommons "github.com/oracle/oracle-database-operator/commons/database" apierrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/runtime" "k8s.io/apimachinery/pkg/runtime/schema" @@ -46,6 +47,7 @@ import ( ctrl "sigs.k8s.io/controller-runtime" logf "sigs.k8s.io/controller-runtime/pkg/log" "sigs.k8s.io/controller-runtime/pkg/webhook" + "sigs.k8s.io/controller-runtime/pkg/webhook/admission" ) // log is for logging in this package. @@ -76,11 +78,24 @@ func (r *AutonomousDatabaseBackup) Default() { var _ webhook.Validator = &AutonomousDatabaseBackup{} // ValidateCreate implements webhook.Validator so a webhook will be registered for the type -func (r *AutonomousDatabaseBackup) ValidateCreate() error { +func (r *AutonomousDatabaseBackup) ValidateCreate() (admission.Warnings, error) { autonomousdatabasebackuplog.Info("validate create", "name", r.Name) var allErrs field.ErrorList + namespaces := dbcommons.GetWatchNamespaces() + _, hasEmptyString := namespaces[""] + isClusterScoped := len(namespaces) == 1 && hasEmptyString + if !isClusterScoped { + _, containsNamespace := namespaces[r.Namespace] + // Check if the allowed namespaces maps contains the required namespace + if len(namespaces) != 0 && !containsNamespace { + allErrs = append(allErrs, + field.Invalid(field.NewPath("metadata").Child("namespace"), r.Namespace, + "Oracle database operator doesn't watch over this namespace")) + } + } + if r.Spec.Target.K8sADB.Name == nil && r.Spec.Target.OCIADB.OCID == nil { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("target"), "target ADB is empty")) @@ -92,15 +107,15 @@ func (r *AutonomousDatabaseBackup) ValidateCreate() error { } if len(allErrs) == 0 { - return nil + return nil, nil } - return apierrors.NewInvalid( + return nil, apierrors.NewInvalid( schema.GroupKind{Group: "database.oracle.com", Kind: "AutonomousDatabaseBackup"}, r.Name, allErrs) } // ValidateUpdate implements webhook.Validator so a webhook will be registered for the type -func (r *AutonomousDatabaseBackup) ValidateUpdate(old runtime.Object) error { +func (r *AutonomousDatabaseBackup) ValidateUpdate(old runtime.Object) (admission.Warnings, error) { autonomousdatabasebackuplog.Info("validate update", "name", r.Name) var allErrs field.ErrorList @@ -132,17 +147,17 @@ func (r *AutonomousDatabaseBackup) ValidateUpdate(old runtime.Object) error { } if len(allErrs) == 0 { - return nil + return nil, nil } - return apierrors.NewInvalid( + return nil, apierrors.NewInvalid( schema.GroupKind{Group: "database.oracle.com", Kind: "AutonomousDatabaseBackup"}, r.Name, allErrs) } // ValidateDelete implements webhook.Validator so a webhook will be registered for the type -func (r *AutonomousDatabaseBackup) ValidateDelete() error { +func (r *AutonomousDatabaseBackup) ValidateDelete() (admission.Warnings, error) { autonomousdatabasebackuplog.Info("validate delete", "name", r.Name) // TODO(user): fill in your validation logic upon object deletion. - return nil + return nil, nil } diff --git a/apis/database/v1alpha1/autonomousdatabaserestore_webhook.go b/apis/database/v1alpha1/autonomousdatabaserestore_webhook.go index 8b4dd974..4f96bd7b 100644 --- a/apis/database/v1alpha1/autonomousdatabaserestore_webhook.go +++ b/apis/database/v1alpha1/autonomousdatabaserestore_webhook.go @@ -39,6 +39,7 @@ package v1alpha1 import ( + dbcommons "github.com/oracle/oracle-database-operator/commons/database" apierrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/runtime" "k8s.io/apimachinery/pkg/runtime/schema" @@ -46,6 +47,7 @@ import ( ctrl "sigs.k8s.io/controller-runtime" logf "sigs.k8s.io/controller-runtime/pkg/log" "sigs.k8s.io/controller-runtime/pkg/webhook" + "sigs.k8s.io/controller-runtime/pkg/webhook/admission" ) // log is for logging in this package. @@ -65,11 +67,24 @@ func (r *AutonomousDatabaseRestore) SetupWebhookWithManager(mgr ctrl.Manager) er var _ webhook.Validator = &AutonomousDatabaseRestore{} // ValidateCreate implements webhook.Validator so a webhook will be registered for the type -func (r *AutonomousDatabaseRestore) ValidateCreate() error { +func (r *AutonomousDatabaseRestore) ValidateCreate() (admission.Warnings, error) { autonomousdatabaserestorelog.Info("validate create", "name", r.Name) var allErrs field.ErrorList + namespaces := dbcommons.GetWatchNamespaces() + _, hasEmptyString := namespaces[""] + isClusterScoped := len(namespaces) == 1 && hasEmptyString + if !isClusterScoped { + _, containsNamespace := namespaces[r.Namespace] + // Check if the allowed namespaces maps contains the required namespace + if len(namespaces) != 0 && !containsNamespace { + allErrs = append(allErrs, + field.Invalid(field.NewPath("metadata").Child("namespace"), r.Namespace, + "Oracle database operator doesn't watch over this namespace")) + } + } + // Validate the target ADB if r.Spec.Target.K8sADB.Name == nil && r.Spec.Target.OCIADB.OCID == nil { allErrs = append(allErrs, @@ -104,31 +119,31 @@ func (r *AutonomousDatabaseRestore) ValidateCreate() error { } if len(allErrs) == 0 { - return nil + return nil, nil } - return apierrors.NewInvalid( + return nil, apierrors.NewInvalid( schema.GroupKind{Group: "database.oracle.com", Kind: "AutonomousDatabaseRestore"}, r.Name, allErrs) } // ValidateUpdate implements webhook.Validator so a webhook will be registered for the type -func (r *AutonomousDatabaseRestore) ValidateUpdate(old runtime.Object) error { +func (r *AutonomousDatabaseRestore) ValidateUpdate(old runtime.Object) (admission.Warnings, error) { autonomousdatabaserestorelog.Info("validate update", "name", r.Name) var allErrs field.ErrorList if len(allErrs) == 0 { - return nil + return nil, nil } - return apierrors.NewInvalid( + return nil, apierrors.NewInvalid( schema.GroupKind{Group: "database.oracle.com", Kind: "AutonomousDatabaseRestore"}, r.Name, allErrs) } // ValidateDelete implements webhook.Validator so a webhook will be registered for the type -func (r *AutonomousDatabaseRestore) ValidateDelete() error { +func (r *AutonomousDatabaseRestore) ValidateDelete() (admission.Warnings, error) { autonomousdatabaserestorelog.Info("validate delete", "name", r.Name) // TODO(user): fill in your validation logic upon object deletion. - return nil + return nil, nil } diff --git a/apis/database/v1alpha1/cdb_types.go b/apis/database/v1alpha1/cdb_types.go index b5f39707..206781b2 100644 --- a/apis/database/v1alpha1/cdb_types.go +++ b/apis/database/v1alpha1/cdb_types.go @@ -52,7 +52,6 @@ type CDBSpec struct { // Name of the CDB Service ServiceName string `json:"serviceName,omitempty"` - // Password for the CDB System Administrator SysAdminPwd CDBSysAdminPassword `json:"sysAdminPwd,omitempty"` // User in the root container with sysdba priviledges to manage PDB lifecycle @@ -86,7 +85,7 @@ type CDBSpec struct { DBPort int `json:"dbPort,omitempty"` // Node Selector for running the Pod NodeSelector map[string]string `json:"nodeSelector,omitempty"` - DBTnsurl string `json:"dbTnsurl,omitempty"` + DBTnsurl string `json:"dbTnsurl,omitempty"` } // CDBSecret defines the secretName @@ -155,6 +154,7 @@ type CDBStatus struct { // +kubebuilder:printcolumn:JSONPath=".spec.replicas",name="Replicas",type="integer",description="Replicas" // +kubebuilder:printcolumn:JSONPath=".status.phase",name="Status",type="string",description="Status of the CDB Resource" // +kubebuilder:printcolumn:JSONPath=".status.msg",name="Message",type="string",description="Error message, if any" +// +kubebuilder:resource:path=cdbs,scope=Namespaced // CDB is the Schema for the cdbs API type CDB struct { diff --git a/apis/database/v1alpha1/cdb_webhook.go b/apis/database/v1alpha1/cdb_webhook.go index 25975092..345b6f75 100644 --- a/apis/database/v1alpha1/cdb_webhook.go +++ b/apis/database/v1alpha1/cdb_webhook.go @@ -49,6 +49,7 @@ import ( ctrl "sigs.k8s.io/controller-runtime" logf "sigs.k8s.io/controller-runtime/pkg/log" "sigs.k8s.io/controller-runtime/pkg/webhook" + "sigs.k8s.io/controller-runtime/pkg/webhook/admission" ) // log is for logging in this package. @@ -83,7 +84,7 @@ func (r *CDB) Default() { var _ webhook.Validator = &CDB{} // ValidateCreate implements webhook.Validator so a webhook will be registered for the type -func (r *CDB) ValidateCreate() error { +func (r *CDB) ValidateCreate() (admission.Warnings, error) { cdblog.Info("ValidateCreate", "name", r.Name) var allErrs field.ErrorList @@ -92,32 +93,32 @@ func (r *CDB) ValidateCreate() error { allErrs = append(allErrs, field.Required(field.NewPath("spec").Child("serviceName"), "Please specify CDB Service name")) } - - if reflect.ValueOf(r.Spec.CDBTlsKey).IsZero() { - allErrs = append(allErrs, - field.Required(field.NewPath("spec").Child("cdbTlsKey"), "Please specify CDB Tls key(secret)")) - } - if reflect.ValueOf(r.Spec.CDBTlsCrt).IsZero() { - allErrs = append(allErrs, - field.Required(field.NewPath("spec").Child("cdbTlsCrt"), "Please specify CDB Tls Certificate(secret)")) - } + if reflect.ValueOf(r.Spec.CDBTlsKey).IsZero() { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("cdbTlsKey"), "Please specify CDB Tls key(secret)")) + } + + if reflect.ValueOf(r.Spec.CDBTlsCrt).IsZero() { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("cdbTlsCrt"), "Please specify CDB Tls Certificate(secret)")) + } /*if r.Spec.SCANName == "" { allErrs = append(allErrs, field.Required(field.NewPath("spec").Child("scanName"), "Please specify SCAN Name for CDB")) }*/ - if ((r.Spec.DBServer == "" && r.Spec.DBTnsurl == "") || (r.Spec.DBServer != "" && r.Spec.DBTnsurl != "")) { + if (r.Spec.DBServer == "" && r.Spec.DBTnsurl == "") || (r.Spec.DBServer != "" && r.Spec.DBTnsurl != "") { allErrs = append(allErrs, field.Required(field.NewPath("spec").Child("dbServer"), "Please specify Database Server Name/IP Address or tnsalias string")) } - if r.Spec.DBTnsurl != "" && ( r.Spec.DBServer != "" || r.Spec.DBPort != 0 || r.Spec.ServiceName != "" ) { + if r.Spec.DBTnsurl != "" && (r.Spec.DBServer != "" || r.Spec.DBPort != 0 || r.Spec.ServiceName != "") { allErrs = append(allErrs, field.Required(field.NewPath("spec").Child("dbServer"), "DBtnsurl is orthogonal to (DBServer,DBport,Services)")) - } - + } + if r.Spec.DBPort == 0 && r.Spec.DBServer != "" { allErrs = append(allErrs, field.Required(field.NewPath("spec").Child("dbPort"), "Please specify DB Server Port")) @@ -159,20 +160,20 @@ func (r *CDB) ValidateCreate() error { field.Required(field.NewPath("spec").Child("webServerPwd"), "Please specify password for the Web Server User having SQL Administrator role")) } if len(allErrs) == 0 { - return nil + return nil, nil } - return apierrors.NewInvalid( + return nil, apierrors.NewInvalid( schema.GroupKind{Group: "database.oracle.com", Kind: "CDB"}, r.Name, allErrs) } // ValidateUpdate implements webhook.Validator so a webhook will be registered for the type -func (r *CDB) ValidateUpdate(old runtime.Object) error { +func (r *CDB) ValidateUpdate(old runtime.Object) (admission.Warnings, error) { cdblog.Info("validate update", "name", r.Name) isCDBMarkedToBeDeleted := r.GetDeletionTimestamp() != nil if isCDBMarkedToBeDeleted { - return nil + return nil, nil } var allErrs field.ErrorList @@ -180,7 +181,7 @@ func (r *CDB) ValidateUpdate(old runtime.Object) error { // Check for updation errors oldCDB, ok := old.(*CDB) if !ok { - return nil + return nil, nil } if r.Spec.DBPort < 0 { @@ -201,18 +202,18 @@ func (r *CDB) ValidateUpdate(old runtime.Object) error { } if len(allErrs) == 0 { - return nil + return nil, nil } - return apierrors.NewInvalid( + return nil, apierrors.NewInvalid( schema.GroupKind{Group: "database.oracle.com", Kind: "CDB"}, r.Name, allErrs) } // ValidateDelete implements webhook.Validator so a webhook will be registered for the type -func (r *CDB) ValidateDelete() error { +func (r *CDB) ValidateDelete() (admission.Warnings, error) { cdblog.Info("validate delete", "name", r.Name) // TODO(user): fill in your validation logic upon object deletion. - return nil + return nil, nil } diff --git a/apis/database/v1alpha1/dataguardbroker_webhook.go b/apis/database/v1alpha1/dataguardbroker_webhook.go index c947f7a3..a9d59286 100644 --- a/apis/database/v1alpha1/dataguardbroker_webhook.go +++ b/apis/database/v1alpha1/dataguardbroker_webhook.go @@ -41,6 +41,7 @@ package v1alpha1 import ( "strings" + dbcommons "github.com/oracle/oracle-database-operator/commons/database" apierrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/runtime" "k8s.io/apimachinery/pkg/runtime/schema" @@ -48,6 +49,7 @@ import ( ctrl "sigs.k8s.io/controller-runtime" logf "sigs.k8s.io/controller-runtime/pkg/log" "sigs.k8s.io/controller-runtime/pkg/webhook" + "sigs.k8s.io/controller-runtime/pkg/webhook/admission" ) // log is for logging in this package. @@ -69,21 +71,21 @@ var _ webhook.Defaulter = &DataguardBroker{} func (r *DataguardBroker) Default() { dataguardbrokerlog.Info("default", "name", r.Name) - if (r.Spec.LoadBalancer) { + if r.Spec.LoadBalancer { if r.Spec.ServiceAnnotations == nil { - r.Spec.ServiceAnnotations= make(map[string]string) + r.Spec.ServiceAnnotations = make(map[string]string) } - // Annotations required for a flexible load balancer on oci + // Annotations required for a flexible load balancer on oci _, ok := r.Spec.ServiceAnnotations["service.beta.kubernetes.io/oci-load-balancer-shape"] - if(!ok) { + if !ok { r.Spec.ServiceAnnotations["service.beta.kubernetes.io/oci-load-balancer-shape"] = "flexible" } - _,ok = r.Spec.ServiceAnnotations["service.beta.kubernetes.io/oci-load-balancer-shape-flex-min"] - if(!ok) { + _, ok = r.Spec.ServiceAnnotations["service.beta.kubernetes.io/oci-load-balancer-shape-flex-min"] + if !ok { r.Spec.ServiceAnnotations["service.beta.kubernetes.io/oci-load-balancer-shape-flex-min"] = "10" } - _,ok = r.Spec.ServiceAnnotations["service.beta.kubernetes.io/oci-load-balancer-shape-flex-max"] - if(!ok) { + _, ok = r.Spec.ServiceAnnotations["service.beta.kubernetes.io/oci-load-balancer-shape-flex-max"] + if !ok { r.Spec.ServiceAnnotations["service.beta.kubernetes.io/oci-load-balancer-shape-flex-max"] = "100" } } @@ -95,38 +97,53 @@ func (r *DataguardBroker) Default() { var _ webhook.Validator = &DataguardBroker{} // ValidateCreate implements webhook.Validator so a webhook will be registered for the type -func (r *DataguardBroker) ValidateCreate() error { +func (r *DataguardBroker) ValidateCreate() (admission.Warnings, error) { + dataguardbrokerlog.Info("validate create", "name", r.Name) + var allErrs field.ErrorList + namespaces := dbcommons.GetWatchNamespaces() + _, containsNamespace := namespaces[r.Namespace] + // Check if the allowed namespaces maps contains the required namespace + if len(namespaces) != 0 && !containsNamespace { + allErrs = append(allErrs, + field.Invalid(field.NewPath("metadata").Child("namespace"), r.Namespace, + "Oracle database operator doesn't watch over this namespace")) + } - // TODO(user): fill in your validation logic upon object creation. - return nil + if len(allErrs) == 0 { + return nil, nil + } + + return nil, apierrors.NewInvalid( + schema.GroupKind{Group: "database.oracle.com", Kind: "Dataguard"}, + r.Name, allErrs) } // ValidateUpdate implements webhook.Validator so a webhook will be registered for the type -func (r *DataguardBroker) ValidateUpdate(old runtime.Object) error { +func (r *DataguardBroker) ValidateUpdate(old runtime.Object) (admission.Warnings, error) { dataguardbrokerlog.Info("validate update", "name", r.Name) dataguardbrokerlog.Info("validate update", "name", r.Name) var allErrs field.ErrorList // check creation validations first - err := r.ValidateCreate() + _, err := r.ValidateCreate() if err != nil { - return err + return nil, err } // Validate Deletion if r.GetDeletionTimestamp() != nil { - err := r.ValidateDelete() + warnings, err := r.ValidateDelete() if err != nil { - return err + return warnings, err } } // Now check for updation errors oldObj, ok := old.(*DataguardBroker) if !ok { - return nil + return nil, nil } if oldObj.Status.ProtectionMode != "" && !strings.EqualFold(r.Spec.ProtectionMode, oldObj.Status.ProtectionMode) { @@ -139,18 +156,18 @@ func (r *DataguardBroker) ValidateUpdate(old runtime.Object) error { } if len(allErrs) == 0 { - return nil + return nil, nil } - return apierrors.NewInvalid( + return nil, apierrors.NewInvalid( schema.GroupKind{Group: "database.oracle.com", Kind: "DataguardBroker"}, r.Name, allErrs) } // ValidateDelete implements webhook.Validator so a webhook will be registered for the type -func (r *DataguardBroker) ValidateDelete() error { +func (r *DataguardBroker) ValidateDelete() (admission.Warnings, error) { dataguardbrokerlog.Info("validate delete", "name", r.Name) // TODO(user): fill in your validation logic upon object deletion. - return nil + return nil, nil } diff --git a/apis/database/v1alpha1/dbcssystem_types.go b/apis/database/v1alpha1/dbcssystem_types.go index 28773fb2..37e80a6b 100644 --- a/apis/database/v1alpha1/dbcssystem_types.go +++ b/apis/database/v1alpha1/dbcssystem_types.go @@ -156,8 +156,9 @@ type VmNetworkDetails struct { NetworkSG string `json:"networkSG,omitempty"` } -//+kubebuilder:object:root=true -//+kubebuilder:subresource:status +// +kubebuilder:object:root=true +// +kubebuilder:subresource:status +// +kubebuilder:resource:path=DbcsSystem,scope=Namespaced // DbcsSystem is the Schema for the dbcssystems API type DbcsSystem struct { diff --git a/apis/database/v1alpha1/oraclerestdataservice_webhook.go b/apis/database/v1alpha1/oraclerestdataservice_webhook.go index be83fffc..bfe3208c 100644 --- a/apis/database/v1alpha1/oraclerestdataservice_webhook.go +++ b/apis/database/v1alpha1/oraclerestdataservice_webhook.go @@ -39,6 +39,7 @@ package v1alpha1 import ( + dbcommons "github.com/oracle/oracle-database-operator/commons/database" apierrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/runtime" "k8s.io/apimachinery/pkg/runtime/schema" @@ -46,6 +47,7 @@ import ( ctrl "sigs.k8s.io/controller-runtime" logf "sigs.k8s.io/controller-runtime/pkg/log" "sigs.k8s.io/controller-runtime/pkg/webhook" + "sigs.k8s.io/controller-runtime/pkg/webhook/admission" ) // log is for logging in this package. @@ -86,11 +88,20 @@ func (r *OracleRestDataService) Default() { var _ webhook.Validator = &OracleRestDataService{} // ValidateCreate implements webhook.Validator so a webhook will be registered for the type -func (r *OracleRestDataService) ValidateCreate() error { +func (r *OracleRestDataService) ValidateCreate() (admission.Warnings, error) { oraclerestdataservicelog.Info("validate create", "name", r.Name) var allErrs field.ErrorList + namespaces := dbcommons.GetWatchNamespaces() + _, containsNamespace := namespaces[r.Namespace] + // Check if the allowed namespaces maps contains the required namespace + if len(namespaces) != 0 && !containsNamespace { + allErrs = append(allErrs, + field.Invalid(field.NewPath("metadata").Child("namespace"), r.Namespace, + "Oracle database operator doesn't watch over this namespace")) + } + // Persistence spec validation if r.Spec.Persistence.Size == "" && (r.Spec.Persistence.AccessMode != "" || r.Spec.Persistence.StorageClass != "" || r.Spec.Persistence.VolumeName != "") { @@ -116,34 +127,34 @@ func (r *OracleRestDataService) ValidateCreate() error { if r.Spec.DatabaseRef == r.Name { allErrs = append(allErrs, field.Forbidden(field.NewPath("Name"), - "cannot be same as DatabaseRef: " + r.Spec.DatabaseRef)) + "cannot be same as DatabaseRef: "+r.Spec.DatabaseRef)) } if len(allErrs) == 0 { - return nil + return nil, nil } - return apierrors.NewInvalid( + return nil, apierrors.NewInvalid( schema.GroupKind{Group: "database.oracle.com", Kind: "OracleRestDataService"}, r.Name, allErrs) } // ValidateUpdate implements webhook.Validator so a webhook will be registered for the type -func (r *OracleRestDataService) ValidateUpdate(oldRuntimeObject runtime.Object) error { +func (r *OracleRestDataService) ValidateUpdate(oldRuntimeObject runtime.Object) (admission.Warnings, error) { oraclerestdataservicelog.Info("validate update", "name", r.Name) var allErrs field.ErrorList // check creation validations first - err := r.ValidateCreate() + warnings, err := r.ValidateCreate() if err != nil { - return err + return warnings, err } // Now check for updation errors old, ok := oldRuntimeObject.(*OracleRestDataService) if !ok { - return nil + return nil, nil } if old.Status.DatabaseRef != "" && old.Status.DatabaseRef != r.Spec.DatabaseRef { @@ -156,18 +167,18 @@ func (r *OracleRestDataService) ValidateUpdate(oldRuntimeObject runtime.Object) } if len(allErrs) == 0 { - return nil + return nil, nil } - return apierrors.NewInvalid( + return nil, apierrors.NewInvalid( schema.GroupKind{Group: "database.oracle.com", Kind: "OracleRestDataService"}, r.Name, allErrs) } // ValidateDelete implements webhook.Validator so a webhook will be registered for the type -func (r *OracleRestDataService) ValidateDelete() error { +func (r *OracleRestDataService) ValidateDelete() (admission.Warnings, error) { oraclerestdataservicelog.Info("validate delete", "name", r.Name) // TODO(user): fill in your validation logic upon object deletion. - return nil + return nil, nil } diff --git a/apis/database/v1alpha1/pdb_types.go b/apis/database/v1alpha1/pdb_types.go index a73d3dd8..1e4da0a1 100644 --- a/apis/database/v1alpha1/pdb_types.go +++ b/apis/database/v1alpha1/pdb_types.go @@ -51,6 +51,8 @@ type PDBSpec struct { PDBTlsCrt PDBTLSCRT `json:"pdbTlsCrt,omitempty"` PDBTlsCat PDBTLSCAT `json:"pdbTlsCat,omitempty"` + // CDB Namespace + CDBNamespace string `json:"cdbNamespace,omitempty"` // Name of the CDB Custom Resource that runs the ORDS container CDBResName string `json:"cdbResName,omitempty"` // Name of the CDB @@ -63,6 +65,10 @@ type PDBSpec struct { AdminName PDBAdminName `json:"adminName,omitempty"` // The administrator password for the new PDB. This property is required when the Action property is Create. AdminPwd PDBAdminPassword `json:"adminPwd,omitempty"` + // Web Server User with SQL Administrator role to allow us to authenticate to the PDB Lifecycle Management REST endpoints + WebServerUsr WebServerUserPDB `json:"webServerUser,omitempty"` + // Password for the Web ServerPDB User + WebServerPwd WebServerPasswordPDB `json:"webServerPwd,omitempty"` // Relevant for Create and Plug operations. As defined in the Oracle Multitenant Database documentation. Values can be a filename convert pattern or NONE. FileNameConversions string `json:"fileNameConversions,omitempty"` // This property is required when the Action property is Plug. As defined in the Oracle Multitenant Database documentation. Values can be a source filename convert pattern or NONE. @@ -130,6 +136,17 @@ type TDESecret struct { Secret PDBSecret `json:"secret"` } +// WebServerUser defines the secret containing Web Server User mapped to key 'webServerUser' to manage PDB lifecycle + +type WebServerUserPDB struct { + Secret PDBSecret `json:"secret"` +} + +// WebServerPassword defines the secret containing password for Web Server User mapped to key 'webServerPwd' to manage PDB lifecycle +type WebServerPasswordPDB struct { + Secret PDBSecret `json:"secret"` +} + // PDBSecret defines the secretName type PDBSecret struct { SecretName string `json:"secretName"` @@ -180,6 +197,8 @@ type PDBStatus struct { // +kubebuilder:printcolumn:JSONPath=".status.totalSize",name="PDB Size",type="string",description="Total Size of the PDB" // +kubebuilder:printcolumn:JSONPath=".status.phase",name="Status",type="string",description="Status of the PDB Resource" // +kubebuilder:printcolumn:JSONPath=".status.msg",name="Message",type="string",description="Error message, if any" +// +kubebuilder:resource:path=pdbs,scope=Namespaced + // PDB is the Schema for the pdbs API type PDB struct { metav1.TypeMeta `json:",inline"` diff --git a/apis/database/v1alpha1/pdb_webhook.go b/apis/database/v1alpha1/pdb_webhook.go index 0e78790a..1577198e 100644 --- a/apis/database/v1alpha1/pdb_webhook.go +++ b/apis/database/v1alpha1/pdb_webhook.go @@ -54,6 +54,7 @@ import ( ctrl "sigs.k8s.io/controller-runtime" logf "sigs.k8s.io/controller-runtime/pkg/log" "sigs.k8s.io/controller-runtime/pkg/webhook" + "sigs.k8s.io/controller-runtime/pkg/webhook/admission" ) // log is for logging in this package. @@ -77,8 +78,8 @@ func (r *PDB) Default() { if action == "DELETE" { if r.Spec.DropAction == "" { - r.Spec.DropAction = "KEEP" - pdblog.Info(" - dropAction : KEEP") + r.Spec.DropAction = "INCLUDING" + pdblog.Info(" - dropAction : INCLUDING") } } else if action != "MODIFY" && action != "STATUS" { if r.Spec.ReuseTempFile == nil { @@ -121,7 +122,7 @@ func (r *PDB) Default() { var _ webhook.Validator = &PDB{} // ValidateCreate implements webhook.Validator so a webhook will be registered for the type -func (r *PDB) ValidateCreate() error { +func (r *PDB) ValidateCreate() (admission.Warnings, error) { pdblog.Info("ValidateCreate-Validating PDB spec for : " + r.Name) var allErrs field.ErrorList @@ -134,9 +135,9 @@ func (r *PDB) ValidateCreate() error { if len(allErrs) == 0 { pdblog.Info("PDB Resource : " + r.Name + " successfully validated for Action : " + action) - return nil + return nil, nil } - return apierrors.NewInvalid( + return nil, apierrors.NewInvalid( schema.GroupKind{Group: "database.oracle.com", Kind: "PDB"}, r.Name, allErrs) } @@ -147,21 +148,21 @@ func (r *PDB) validateAction(allErrs *field.ErrorList) { pdblog.Info("Valdiating PDB Resource Action : " + action) - if reflect.ValueOf(r.Spec.PDBTlsKey).IsZero() { - *allErrs = append(*allErrs, - field.Required(field.NewPath("spec").Child("pdbTlsKey"), "Please specify PDB Tls Key(secret)")) - } - - if reflect.ValueOf(r.Spec.PDBTlsCrt).IsZero() { - *allErrs = append(*allErrs, - field.Required(field.NewPath("spec").Child("pdbTlsCrt"), "Please specify PDB Tls Certificate(secret)")) - } - - if reflect.ValueOf(r.Spec.PDBTlsCat).IsZero() { - *allErrs = append(*allErrs, - field.Required(field.NewPath("spec").Child("pdbTlsCat"), "Please specify PDB Tls Certificate Authority(secret)")) - } - + if reflect.ValueOf(r.Spec.PDBTlsKey).IsZero() { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("pdbTlsKey"), "Please specify PDB Tls Key(secret)")) + } + + if reflect.ValueOf(r.Spec.PDBTlsCrt).IsZero() { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("pdbTlsCrt"), "Please specify PDB Tls Certificate(secret)")) + } + + if reflect.ValueOf(r.Spec.PDBTlsCat).IsZero() { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("pdbTlsCat"), "Please specify PDB Tls Certificate Authority(secret)")) + } + switch action { case "CREATE": if reflect.ValueOf(r.Spec.AdminName).IsZero() { @@ -172,6 +173,15 @@ func (r *PDB) validateAction(allErrs *field.ErrorList) { *allErrs = append(*allErrs, field.Required(field.NewPath("spec").Child("adminPwd"), "Please specify PDB System Administrator Password")) } + if reflect.ValueOf(r.Spec.WebServerUsr).IsZero() { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("WebServerUser"), "Please specify the http webServerUser")) + } + if reflect.ValueOf(r.Spec.WebServerPwd).IsZero() { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("webServerPwd"), "Please specify the http webserverPassword")) + } + if r.Spec.FileNameConversions == "" { *allErrs = append(*allErrs, field.Required(field.NewPath("spec").Child("fileNameConversions"), "Please specify a value for fileNameConversions. Values can be a filename convert pattern or NONE")) @@ -242,12 +252,12 @@ func (r *PDB) validateAction(allErrs *field.ErrorList) { } // ValidateUpdate implements webhook.Validator so a webhook will be registered for the type -func (r *PDB) ValidateUpdate(old runtime.Object) error { +func (r *PDB) ValidateUpdate(old runtime.Object) (admission.Warnings, error) { pdblog.Info("ValidateUpdate-Validating PDB spec for : " + r.Name) isPDBMarkedToBeDeleted := r.GetDeletionTimestamp() != nil if isPDBMarkedToBeDeleted { - return nil + return nil, nil } var allErrs field.ErrorList @@ -272,19 +282,19 @@ func (r *PDB) ValidateUpdate(old runtime.Object) error { } if len(allErrs) == 0 { - return nil + return nil, nil } - return apierrors.NewInvalid( + return nil, apierrors.NewInvalid( schema.GroupKind{Group: "database.oracle.com", Kind: "PDB"}, r.Name, allErrs) } // ValidateDelete implements webhook.Validator so a webhook will be registered for the type -func (r *PDB) ValidateDelete() error { +func (r *PDB) ValidateDelete() (admission.Warnings, error) { pdblog.Info("ValidateDelete-Validating PDB spec for : " + r.Name) // TODO(user): fill in your validation logic upon object deletion. - return nil + return nil, nil } // Validate common specs needed for all PDB Actions diff --git a/apis/database/v1alpha1/shardingdatabase_types.go b/apis/database/v1alpha1/shardingdatabase_types.go index b2f7142e..3b4f9c17 100644 --- a/apis/database/v1alpha1/shardingdatabase_types.go +++ b/apis/database/v1alpha1/shardingdatabase_types.go @@ -58,26 +58,39 @@ import ( type ShardingDatabaseSpec struct { // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster // Important: Run "make" to regenerate code after modifying this file - Shard []ShardSpec `json:"shard"` - Catalog []CatalogSpec `json:"catalog"` // The catalogSpes accept all the catalog parameters - Gsm []GsmSpec `json:"gsm"` // The GsmSpec will accept all the Gsm parameter - StorageClass string `json:"storageClass,omitempty"` // Optional Accept storage class name - DbImage string `json:"dbImage"` // Accept DB Image name - DbImagePullSecret string `json:"dbImagePullSecret,omitempty"` // Optional The name of an image pull secret in case of a private docker repository. - GsmImage string `json:"gsmImage"` // Acccept the GSM image name - GsmImagePullSecret string `json:"gsmImagePullSecret,omitempty"` // Optional The name of an image pull secret in case of a private docker repository. - Secret string `json:"secret"` // Secret Name to be used with Shard - StagePvcName string `json:"stagePvcName,omitempty"` // the Stagepvc for the backup of cluster - PortMappings []PortMapping `json:"portMappings,omitempty"` // Port mappings for the service that is created. The service is created if there is at least - Namespace string `json:"namespace,omitempty"` // Target namespace of the application. - IsDebug bool `json:"isDebug,omitempty"` // Optional parameter to enable logining - IsExternalSvc bool `json:"isExternalSvc,omitempty"` - IsClone bool `json:"isClone,omitempty"` - IsDataGuard bool `json:"isDataGuard,omitempty"` - ScriptsLocation string `json:"scriptsLocation,omitempty"` - NsConfigMap string `json:"nsConfigMap,omitempty"` - NsSecret string `json:"nsSecret,omitempty"` - IsDeleteOraPvc bool `json:"isDeleteOraPvc,omitempty"` + Shard []ShardSpec `json:"shard"` + Catalog []CatalogSpec `json:"catalog"` // The catalogSpes accept all the catalog parameters + Gsm []GsmSpec `json:"gsm"` // The GsmSpec will accept all the Gsm parameter + StorageClass string `json:"storageClass,omitempty"` // Optional Accept storage class name + DbImage string `json:"dbImage"` // Accept DB Image name + DbImagePullSecret string `json:"dbImagePullSecret,omitempty"` // Optional The name of an image pull secret in case of a private docker repository. + GsmImage string `json:"gsmImage"` // Acccept the GSM image name + GsmImagePullSecret string `json:"gsmImagePullSecret,omitempty"` // Optional The name of an image pull secret in case of a private docker repository. + StagePvcName string `json:"stagePvcName,omitempty"` // the Stagepvc for the backup of cluster + PortMappings []PortMapping `json:"portMappings,omitempty"` // Port mappings for the service that is created. The service is created if there is at least + Namespace string `json:"namespace,omitempty"` // Target namespace of the application. + IsDebug bool `json:"isDebug,omitempty"` // Optional parameter to enable logining + IsExternalSvc bool `json:"isExternalSvc,omitempty"` + IsClone bool `json:"isClone,omitempty"` + IsDataGuard bool `json:"isDataGuard,omitempty"` + ScriptsLocation string `json:"scriptsLocation,omitempty"` + IsDeleteOraPvc bool `json:"isDeleteOraPvc,omitempty"` + ReadinessCheckPeriod int `json:"readinessCheckPeriod,omitempty"` + LivenessCheckPeriod int `json:"liveinessCheckPeriod,omitempty"` + ReplicationType string `json:"replicationType,omitempty"` + IsDownloadScripts bool `json:"isDownloadScripts,omitempty"` + InvitedNodeSubnetFlag string `json:"invitedNodeSubnetFlag,omitempty"` + InvitedNodeSubnet string `json:"InvitedNodeSubnet,omitempty"` + ShardingType string `json:"shardingType,omitempty"` + GsmShardSpace []GsmShardSpaceSpec `json:"gsmShardSpace,omitempty"` + GsmShardGroup []GsmShardGroupSpec `json:"gsmShardGroup,omitempty"` + ShardRegion []string `json:"shardRegion,omitempty"` + ShardBuddyRegion string `json:"shardBuddyRegion,omitempty"` + GsmService []GsmServiceSpec `json:"gsmService,omitempty"` + ShardConfigName string `json:"shardConfigName,omitempty"` + GsmDevMode string `json:"gsmDevMode,omitempty"` + DbSecret *SecretDetails `json:"dbSecret,omitempty"` // Secret Name to be used with Shard + IsTdeWallet bool `json:"isTdeWallet,omitempty"` } // To understand Metav1.Condition, please refer the link https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1 @@ -88,7 +101,8 @@ type ShardingDatabaseStatus struct { Shard map[string]string `json:"shards,omitempty"` Catalog map[string]string `json:"catalogs,omitempty"` - Gsm GsmStatus `json:"gsm,omitempty"` + + Gsm GsmStatus `json:"gsm,omitempty"` // +patchMergeKey=type // +patchStrategy=merge @@ -106,6 +120,12 @@ type GsmStatus struct { Services string `json:"services,omitempty"` } +type GsmShardDetails struct { + Name string `json:"name,omitempty"` + Available string `json:"available,omitempty"` + State string `json:"State,omitempty"` +} + type GsmStatusDetails struct { Name string `json:"name,omitempty"` K8sInternalSvc string `json:"k8sInternalSvc,omitempty"` @@ -118,8 +138,12 @@ type GsmStatusDetails struct { //+kubebuilder:object:root=true //+kubebuilder:subresource:status +//+kubebuilder:printcolumn:JSONPath=".status.gsm.state",name="Gsm State",type=string +//+kubebuilder:printcolumn:JSONPath=".status.gsm.services",name="Services",type=string +//+kubebuilder:printcolumn:JSONPath=".status.gsm.shards",name="shards",type=string,priority=1 // ShardingDatabase is the Schema for the shardingdatabases API +// +kubebuilder:resource:path=shardingdatabases,scope=Namespaced type ShardingDatabase struct { metav1.TypeMeta `json:",inline"` metav1.ObjectMeta `json:"metadata,omitempty"` @@ -151,6 +175,10 @@ type ShardSpec struct { PvAnnotations map[string]string `json:"pvAnnotations,omitempty"` PvMatchLabels map[string]string `json:"pvMatchLabels,omitempty"` ImagePulllPolicy *corev1.PullPolicy `json:"imagePullPolicy,omitempty"` + ShardSpace string `json:"shardSpace,omitempty"` + ShardGroup string `json:"shardGroup,omitempty"` + ShardRegion string `json:"shardRegion,omitempty"` + DeployAs string `json:"deployAs,omitempty"` } // CatalogSpec defines the desired state of CatalogSpec @@ -174,7 +202,7 @@ type CatalogSpec struct { type GsmSpec struct { Name string `json:"name"` // Gsm name that will be used deploy StatefulSet - Replicas int32 `json:"replicas,omitempty"` // Gsm Replicas. If you set OraGsmPvcName then it is set default to 1. + //Replicas int32 `json:"replicas,omitempty"` // Gsm Replicas. If you set OraGsmPvcName then it is set default to 1. EnvVars []EnvironmentVariable `json:"envVars,omitempty"` //Optional Env variables for GSM StorageSizeInGb int32 `json:"storageSizeInGb,omitempty"` // This parameter will not be used if you use OraGsmPvcName Resources *corev1.ResourceRequirements `json:"resources,omitempty" protobuf:"bytes,1,opt,name=resources"` // Optional resource requirement for the container. @@ -184,6 +212,70 @@ type GsmSpec struct { NodeSelector map[string]string `json:"nodeSelector,omitempty"` PvMatchLabels map[string]string `json:"pvMatchLabels,omitempty"` ImagePulllPolicy *corev1.PullPolicy `json:"imagePullPolicy,omitempty"` + Region string `json:"region,omitempty"` + DirectorName string `json:"directorName,omitempty"` +} + +// ShardGroupSpec Specification + +type GsmShardGroupSpec struct { + Name string `json:"name"` // Name of the shardgroup. + Region string `json:"region,omitempty"` + DeployAs string `json:"deployAs,omitempty"` +} + +// ShardSpace Specs +type GsmShardSpaceSpec struct { + Name string `json:"name"` // Name of the shardSpace. + Chunks int `json:"chunks,omitempty"` //chunks is optional + ProtectionMode string `json:"protectionMode,omitempty"` // Data guard protection mode + ShardGroup string `json:"shardGroup,omitempty"` +} + +// Service Definition +type GsmServiceSpec struct { + Name string `json:"name"` // Name of the shardSpace. + Available string `json:"available,omitempty"` + ClbGoal string `json:"clbGoal,omitempty"` + CommitOutcome string `json:"commitOutcome,omitempty"` + DrainTimeout string `json:"drainTimeout,omitempty"` + Dtp string `json:"dtp,omitempty"` + Edition string `json:"edition,omitempty"` + FailoverPrimary string `json:"failoverPrimary,omitempty"` + FailoverRestore string `json:"failoverRestore,omitempty"` + FailoverDelay string `json:"failoverDelay,omitempty"` + FailoverMethod string `json:"failoverMethod,omitempty"` + FailoverRetry string `json:"failoverRetry,omitempty"` + FailoverType string `json:"failoverType,omitempty"` + GdsPool string `json:"gdsPool,omitempty"` + Role string `json:"role,omitempty"` + SessionState string `json:"sessionState,omitempty"` + Lag int `json:"lag,omitempty"` + Locality string `json:"locality,omitempty"` + Notification string `json:"notification,omitempty"` + PdbName string `json:"pdbName,omitempty"` + Policy string `json:"policy,omitempty"` + Preferrred string `json:"preferred,omitempty"` + PreferredAll string `json:"prferredAll,omitempty"` + RegionFailover string `json:"regionFailover,omitempty"` + StopOption string `json:"stopOption,omitempty"` + SqlTrasactionProfile string `json:"sqlTransactionProfile,omitempty"` + TableFamily string `json:"tableFamily,omitempty"` + Retention string `json:"retention,omitempty"` + TfaPolicy string `json:"tfaPolicy,omitempty"` +} + +// Secret Details +type SecretDetails struct { + Name string `json:"name"` // Name of the secret. + KeyFileName string `json:"keyFileName,omitempty"` // Name of the key. + NsConfigMap string `json:"nsConfigMap,omitempty"` + NsSecret string `json:"nsSecret,omitempty"` + PwdFileName string `json:"pwdFileName"` + PwdFileMountLocation string `json:"pwdFileMountLocation,omitempty"` + KeyFileMountLocation string `json:"keyFileMountLocation,omitempty"` + KeySecretName string `json:"keySecretName,omitempty"` + EncryptionType string `json:"encryptionType,omitempty"` } // EnvironmentVariable represents a named variable accessible for containers. diff --git a/apis/database/v1alpha1/singleinstancedatabase_types.go b/apis/database/v1alpha1/singleinstancedatabase_types.go index 81290735..0d49aa6b 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_types.go +++ b/apis/database/v1alpha1/singleinstancedatabase_types.go @@ -64,16 +64,17 @@ type SingleInstanceDatabaseSpec struct { ListenerPort int `json:"listenerPort,omitempty"` TcpsListenerPort int `json:"tcpsListenerPort,omitempty"` ServiceAnnotations map[string]string `json:"serviceAnnotations,omitempty"` - FlashBack bool `json:"flashBack,omitempty"` - ArchiveLog bool `json:"archiveLog,omitempty"` - ForceLogging bool `json:"forceLog,omitempty"` + FlashBack *bool `json:"flashBack,omitempty"` + ArchiveLog *bool `json:"archiveLog,omitempty"` + ForceLogging *bool `json:"forceLog,omitempty"` EnableTCPS bool `json:"enableTCPS,omitempty"` TcpsCertRenewInterval string `json:"tcpsCertRenewInterval,omitempty"` + TcpsTlsSecret string `json:"tcpsTlsSecret,omitempty"` DgBrokerConfigured bool `json:"dgBrokerConfigured,omitempty"` - CloneFrom string `json:"cloneFrom,omitempty"` - PrimaryDatabaseRef string `json:"primaryDatabaseRef,omitempty"` - CreateAsStandby bool `json:"createAsStandby,omitempty"` + PrimaryDatabaseRef string `json:"primaryDatabaseRef,omitempty"` + // +kubebuilder:validation:Enum=primary;standby;clone + CreateAs string `json:"createAs,omitempty"` ReadinessCheckPeriod int `json:"readinessCheckPeriod,omitempty"` ServiceAccountName string `json:"serviceAccountName,omitempty"` @@ -84,7 +85,7 @@ type SingleInstanceDatabaseSpec struct { AdminPassword SingleInstanceDatabaseAdminPassword `json:"adminPassword,omitempty"` Image SingleInstanceDatabaseImage `json:"image"` Persistence SingleInstanceDatabasePersistence `json:"persistence,omitempty"` - InitParams SingleInstanceDatabaseInitParams `json:"initParams,omitempty"` + InitParams *SingleInstanceDatabaseInitParams `json:"initParams,omitempty"` } // SingleInstanceDatabasePersistence defines the storage size and class for PVC @@ -93,8 +94,10 @@ type SingleInstanceDatabasePersistence struct { StorageClass string `json:"storageClass,omitempty"` // +kubebuilder:validation:Enum=ReadWriteOnce;ReadWriteMany AccessMode string `json:"accessMode,omitempty"` - VolumeName string `json:"volumeName,omitempty"` + DatafilesVolumeName string `json:"datafilesVolumeName,omitempty"` + ScriptsVolumeName string `json:"scriptsVolumeName,omitempty"` VolumeClaimAnnotation string `json:"volumeClaimAnnotation,omitempty"` + SetWritePermissions *bool `json:"setWritePermissions,omitempty"` } // SingleInstanceDatabaseInitParams defines the Init Parameters @@ -145,7 +148,7 @@ type SingleInstanceDatabaseStatus struct { Pdbname string `json:"pdbName,omitempty"` InitSgaSize int `json:"initSgaSize,omitempty"` InitPgaSize int `json:"initPgaSize,omitempty"` - CloneFrom string `json:"cloneFrom,omitempty"` + CreatedAs string `json:"createdAs,omitempty"` FlashBack string `json:"flashBack,omitempty"` ArchiveLog string `json:"archiveLog,omitempty"` ForceLogging string `json:"forceLog,omitempty"` @@ -162,6 +165,8 @@ type SingleInstanceDatabaseStatus struct { ClientWalletLoc string `json:"clientWalletLoc,omitempty"` PrimaryDatabase string `json:"primaryDatabase,omitempty"` DgBrokerConfigured bool `json:"dgBrokerConfigured,omitempty"` + // +kubebuilder:default:="" + TcpsTlsSecret string `json:"tcpsTlsSecret"` // +patchMergeKey=type // +patchStrategy=merge diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index 5425d99c..1a47207d 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -39,9 +39,9 @@ package v1alpha1 import ( - "strings" - "time" "strconv" + "strings" + "time" dbcommons "github.com/oracle/oracle-database-operator/commons/database" @@ -52,6 +52,7 @@ import ( ctrl "sigs.k8s.io/controller-runtime" logf "sigs.k8s.io/controller-runtime/pkg/log" "sigs.k8s.io/controller-runtime/pkg/webhook" + "sigs.k8s.io/controller-runtime/pkg/webhook/admission" ) // log is for logging in this package. @@ -74,20 +75,20 @@ func (r *SingleInstanceDatabase) Default() { singleinstancedatabaselog.Info("default", "name", r.Name) if r.Spec.LoadBalancer { - // Annotations required for a flexible load balancer on oci + // Annotations required for a flexible load balancer on oci if r.Spec.ServiceAnnotations == nil { - r.Spec.ServiceAnnotations= make(map[string]string) + r.Spec.ServiceAnnotations = make(map[string]string) } _, ok := r.Spec.ServiceAnnotations["service.beta.kubernetes.io/oci-load-balancer-shape"] - if(!ok) { + if !ok { r.Spec.ServiceAnnotations["service.beta.kubernetes.io/oci-load-balancer-shape"] = "flexible" } - _,ok = r.Spec.ServiceAnnotations["service.beta.kubernetes.io/oci-load-balancer-shape-flex-min"] - if(!ok) { + _, ok = r.Spec.ServiceAnnotations["service.beta.kubernetes.io/oci-load-balancer-shape-flex-min"] + if !ok { r.Spec.ServiceAnnotations["service.beta.kubernetes.io/oci-load-balancer-shape-flex-min"] = "10" } - _,ok = r.Spec.ServiceAnnotations["service.beta.kubernetes.io/oci-load-balancer-shape-flex-max"] - if(!ok) { + _, ok = r.Spec.ServiceAnnotations["service.beta.kubernetes.io/oci-load-balancer-shape-flex-max"] + if !ok { r.Spec.ServiceAnnotations["service.beta.kubernetes.io/oci-load-balancer-shape-flex-max"] = "100" } } @@ -98,11 +99,15 @@ func (r *SingleInstanceDatabase) Default() { } if r.Spec.Edition == "" { - if r.Spec.CloneFrom == "" && !r.Spec.Image.PrebuiltDB { + if r.Spec.CreateAs == "clone" && !r.Spec.Image.PrebuiltDB { r.Spec.Edition = "enterprise" } } + if r.Spec.CreateAs == "" { + r.Spec.CreateAs = "primary" + } + if r.Spec.Sid == "" { if r.Spec.Edition == "express" { r.Spec.Sid = "XE" @@ -124,14 +129,9 @@ func (r *SingleInstanceDatabase) Default() { } if r.Spec.Edition == "express" || r.Spec.Edition == "free" { - if r.Status.Replicas == 1 { - // default the replicas for XE - r.Spec.Replicas = 1 - } - } - - if r.Spec.PrimaryDatabaseRef != "" && r.Spec.CreateAsStandby { - if r.Spec.Replicas == 0 { + // Allow zero replicas as a means to bounce the DB + if r.Status.Replicas == 1 && r.Spec.Replicas > 1 { + // If not zero, default the replicas to 1 r.Spec.Replicas = 1 } } @@ -143,13 +143,22 @@ func (r *SingleInstanceDatabase) Default() { var _ webhook.Validator = &SingleInstanceDatabase{} // ValidateCreate implements webhook.Validator so a webhook will be registered for the type -func (r *SingleInstanceDatabase) ValidateCreate() error { +func (r *SingleInstanceDatabase) ValidateCreate() (admission.Warnings, error) { singleinstancedatabaselog.Info("validate create", "name", r.Name) var allErrs field.ErrorList + namespaces := dbcommons.GetWatchNamespaces() + _, containsNamespace := namespaces[r.Namespace] + // Check if the allowed namespaces maps contains the required namespace + if len(namespaces) != 0 && !containsNamespace { + allErrs = append(allErrs, + field.Invalid(field.NewPath("metadata").Child("namespace"), r.Namespace, + "Oracle database operator doesn't watch over this namespace")) + } + // Persistence spec validation if r.Spec.Persistence.Size == "" && (r.Spec.Persistence.AccessMode != "" || - r.Spec.Persistence.StorageClass != "" || r.Spec.Persistence.VolumeName != "") { + r.Spec.Persistence.StorageClass != "" || r.Spec.Persistence.DatafilesVolumeName != "") { allErrs = append(allErrs, field.Invalid(field.NewPath("spec").Child("persistence").Child("size"), r.Spec.Persistence, "invalid persistence specification, specify required size")) @@ -168,6 +177,40 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { } } + if r.Spec.CreateAs == "standby" { + if r.Spec.ArchiveLog != nil { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("archiveLog"), + r.Spec.ArchiveLog, "archiveLog cannot be specified for standby databases")) + } + if r.Spec.FlashBack != nil { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("flashBack"), + r.Spec.FlashBack, "flashBack cannot be specified for standby databases")) + } + if r.Spec.ForceLogging != nil { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("forceLog"), + r.Spec.ForceLogging, "forceLog cannot be specified for standby databases")) + } + if r.Spec.InitParams != nil { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("initParams"), + r.Spec.InitParams, "initParams cannot be specified for standby databases")) + } + if r.Spec.Persistence.ScriptsVolumeName != "" { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("persistence").Child("scriptsVolumeName"), + r.Spec.Persistence.ScriptsVolumeName, "scriptsVolumeName cannot be specified for standby databases")) + } + if r.Spec.EnableTCPS { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("enableTCPS"), + r.Spec.EnableTCPS, "enableTCPS cannot be specified for standby databases")) + } + + } + // Replica validation if r.Spec.Replicas > 1 { valMsg := "" @@ -182,17 +225,22 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { field.Invalid(field.NewPath("spec").Child("replicas"), r.Spec.Replicas, valMsg)) } } - + + if (r.Spec.CreateAs == "clone" || r.Spec.CreateAs == "standby") && r.Spec.PrimaryDatabaseRef == "" { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("primaryDatabaseRef"), r.Spec.PrimaryDatabaseRef, "Primary Database reference cannot be null for a secondary database")) + } + if r.Spec.Edition == "express" || r.Spec.Edition == "free" { - if r.Spec.CloneFrom != "" { + if r.Spec.CreateAs == "clone" { allErrs = append(allErrs, - field.Invalid(field.NewPath("spec").Child("cloneFrom"), r.Spec.CloneFrom, - "Cloning not supported for " + r.Spec.Edition + " edition")) + field.Invalid(field.NewPath("spec").Child("createAs"), r.Spec.CreateAs, + "Cloning not supported for "+r.Spec.Edition+" edition")) } - if r.Spec.CreateAsStandby { + if r.Spec.CreateAs == "standby" { allErrs = append(allErrs, - field.Invalid(field.NewPath("spec").Child("createAsStandby"), r.Spec.CreateAsStandby, - "Physical Standby Database creation is not supported for " + r.Spec.Edition + " edition")) + field.Invalid(field.NewPath("spec").Child("createAs"), r.Spec.CreateAs, + "Physical Standby Database creation is not supported for "+r.Spec.Edition+" edition")) } if r.Spec.Edition == "express" && strings.ToUpper(r.Spec.Sid) != "XE" { allErrs = append(allErrs, @@ -214,25 +262,10 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { field.Invalid(field.NewPath("spec").Child("pdbName"), r.Spec.Pdbname, "Free edition PDB must be FREEPDB1")) } - if r.Spec.InitParams.CpuCount != 0 { - allErrs = append(allErrs, - field.Invalid(field.NewPath("spec").Child("initParams").Child("cpuCount"), r.Spec.InitParams.CpuCount, - r.Spec.Edition + " edition does not support changing init parameter cpuCount.")) - } - if r.Spec.InitParams.Processes != 0 { + if r.Spec.InitParams != nil { allErrs = append(allErrs, - field.Invalid(field.NewPath("spec").Child("initParams").Child("processes"), r.Spec.InitParams.Processes, - r.Spec.Edition + " edition does not support changing init parameter process.")) - } - if r.Spec.InitParams.SgaTarget != 0 { - allErrs = append(allErrs, - field.Invalid(field.NewPath("spec").Child("initParams").Child("sgaTarget"), r.Spec.InitParams.SgaTarget, - r.Spec.Edition + " edition does not support changing init parameter sgaTarget.")) - } - if r.Spec.InitParams.PgaAggregateTarget != 0 { - allErrs = append(allErrs, - field.Invalid(field.NewPath("spec").Child("initParams").Child("pgaAggregateTarget"), r.Spec.InitParams.PgaAggregateTarget, - r.Spec.Edition + " edition does not support changing init parameter pgaAggregateTarget.")) + field.Invalid(field.NewPath("spec").Child("initParams"), *r.Spec.InitParams, + r.Spec.Edition+" edition does not support changing init parameters")) } } else { if r.Spec.Sid == "XE" { @@ -247,29 +280,29 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { } } - if r.Spec.CloneFrom != "" { + if r.Spec.CreateAs == "clone" { if r.Spec.Image.PrebuiltDB { allErrs = append(allErrs, - field.Invalid(field.NewPath("spec").Child("cloneFrom"), r.Spec.CloneFrom, + field.Invalid(field.NewPath("spec").Child("createAs"), r.Spec.CreateAs, "cannot clone to create a prebuilt db")) - } else if strings.Contains(r.Spec.CloneFrom, ":") && strings.Contains(r.Spec.CloneFrom, "/") && r.Spec.Edition == "" { + } else if strings.Contains(r.Spec.PrimaryDatabaseRef, ":") && strings.Contains(r.Spec.PrimaryDatabaseRef, "/") && r.Spec.Edition == "" { //Edition must be passed when cloning from a source database other than same k8s cluster allErrs = append(allErrs, - field.Invalid(field.NewPath("spec").Child("edition"), r.Spec.CloneFrom, + field.Invalid(field.NewPath("spec").Child("edition"), r.Spec.CreateAs, "Edition must be passed when cloning from a source database other than same k8s cluster")) } } - if r.Status.FlashBack == "true" && r.Spec.FlashBack { - if !r.Spec.ArchiveLog { + if r.Status.FlashBack == "true" && r.Spec.FlashBack != nil && *r.Spec.FlashBack { + if r.Spec.ArchiveLog != nil && !*r.Spec.ArchiveLog { allErrs = append(allErrs, field.Invalid(field.NewPath("spec").Child("archiveLog"), r.Spec.ArchiveLog, "Cannot disable Archivelog. Please disable Flashback first.")) } } - if r.Status.ArchiveLog == "false" && !r.Spec.ArchiveLog { - if r.Spec.FlashBack { + if r.Status.ArchiveLog == "false" && r.Spec.ArchiveLog != nil && !*r.Spec.ArchiveLog { + if *r.Spec.FlashBack { allErrs = append(allErrs, field.Invalid(field.NewPath("spec").Child("flashBack"), r.Spec.FlashBack, "Cannot enable Flashback. Please enable Archivelog first.")) @@ -306,6 +339,7 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { "listenerPort can not be 2484 as the default port for tcpsListenerPort is 2484.")) } } + if r.Spec.EnableTCPS && r.Spec.ListenerPort != 0 && r.Spec.TcpsListenerPort != 0 && r.Spec.ListenerPort == r.Spec.TcpsListenerPort { allErrs = append(allErrs, field.Invalid(field.NewPath("spec").Child("tcpsListenerPort"), r.Spec.TcpsListenerPort, @@ -328,87 +362,128 @@ func (r *SingleInstanceDatabase) ValidateCreate() error { "Please specify tcpsCertRenewInterval in the range: 24h to 8760h")) } } + + // tcpsTlsSecret validations + if !r.Spec.EnableTCPS && r.Spec.TcpsTlsSecret != "" { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("tcpsTlsSecret"), + " is allowed only if enableTCPS is true")) + } + if r.Spec.TcpsTlsSecret != "" && r.Spec.TcpsCertRenewInterval != "" { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("tcpsCertRenewInterval"), + " is applicable only for self signed certs")) + } + + if r.Spec.InitParams != nil { + if (r.Spec.InitParams.PgaAggregateTarget != 0 && r.Spec.InitParams.SgaTarget == 0) || (r.Spec.InitParams.PgaAggregateTarget == 0 && r.Spec.InitParams.SgaTarget != 0) { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("initParams"), + r.Spec.InitParams, "initParams value invalid : Provide values for both pgaAggregateTarget and SgaTarget")) + } + } + if len(allErrs) == 0 { - return nil + return nil, nil } - return apierrors.NewInvalid( + return nil, apierrors.NewInvalid( schema.GroupKind{Group: "database.oracle.com", Kind: "SingleInstanceDatabase"}, r.Name, allErrs) } // ValidateUpdate implements webhook.Validator so a webhook will be registered for the type -func (r *SingleInstanceDatabase) ValidateUpdate(oldRuntimeObject runtime.Object) error { +func (r *SingleInstanceDatabase) ValidateUpdate(oldRuntimeObject runtime.Object) (admission.Warnings, error) { singleinstancedatabaselog.Info("validate update", "name", r.Name) var allErrs field.ErrorList // check creation validations first - err := r.ValidateCreate() + warnings, err := r.ValidateCreate() if err != nil { - return err + return warnings, err } // Validate Deletion if r.GetDeletionTimestamp() != nil { - err := r.ValidateDelete() + warnings, err := r.ValidateDelete() if err != nil { - return err + return warnings, err } } // Now check for updation errors old, ok := oldRuntimeObject.(*SingleInstanceDatabase) if !ok { - return nil + return nil, nil + } + + if old.Status.CreatedAs == "clone" { + if r.Spec.Edition != "" && old.Status.Edition != "" && !strings.EqualFold(old.Status.Edition, r.Spec.Edition) { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("edition"), "Edition of a cloned singleinstancedatabase cannot be changed post creation")) + } + + if !strings.EqualFold(old.Status.PrimaryDatabase, r.Spec.PrimaryDatabaseRef) { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("primaryDatabaseRef"), "Primary database of a cloned singleinstancedatabase cannot be changed post creation")) + } } - if (old.Status.Role != dbcommons.ValueUnavailable && old.Status.Role != "PRIMARY") { + if old.Status.Role != dbcommons.ValueUnavailable && old.Status.Role != "PRIMARY" { // Restriciting Patching of secondary databases archiveLog, forceLog, flashBack statusArchiveLog, _ := strconv.ParseBool(old.Status.ArchiveLog) - if statusArchiveLog != r.Spec.ArchiveLog { + if r.Spec.ArchiveLog != nil && (statusArchiveLog != *r.Spec.ArchiveLog) { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("archiveLog"), "cannot be changed")) } statusFlashBack, _ := strconv.ParseBool(old.Status.FlashBack) - if statusFlashBack != r.Spec.FlashBack { + if r.Spec.FlashBack != nil && (statusFlashBack != *r.Spec.FlashBack) { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("flashBack"), "cannot be changed")) } statusForceLogging, _ := strconv.ParseBool(old.Status.ForceLogging) - if statusForceLogging != r.Spec.ForceLogging { + if r.Spec.ForceLogging != nil && (statusForceLogging != *r.Spec.ForceLogging) { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("forceLog"), "cannot be changed")) } + // Restriciting Patching of secondary databases InitParams - if old.Status.InitParams.SgaTarget != r.Spec.InitParams.SgaTarget { - allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec").Child("InitParams").Child("sgaTarget"), "cannot be changed")) + if r.Spec.InitParams != nil { + if old.Status.InitParams.SgaTarget != r.Spec.InitParams.SgaTarget { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("initParams").Child("sgaTarget"), "cannot be changed")) + } + if old.Status.InitParams.PgaAggregateTarget != r.Spec.InitParams.PgaAggregateTarget { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("initParams").Child("pgaAggregateTarget"), "cannot be changed")) + } + if old.Status.InitParams.CpuCount != r.Spec.InitParams.CpuCount { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("initParams").Child("cpuCount"), "cannot be changed")) + } + if old.Status.InitParams.Processes != r.Spec.InitParams.Processes { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("initParams").Child("processes"), "cannot be changed")) + } } - if old.Status.InitParams.PgaAggregateTarget != r.Spec.InitParams.PgaAggregateTarget { - allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec").Child("InitParams").Child("pgaAggregateTarget"), "cannot be changed")) - } - if old.Status.InitParams.CpuCount != r.Spec.InitParams.CpuCount { + } + + // if Db is in a dataguard configuration or referred by Standby databases then Restrict enabling Tcps on the Primary DB + if r.Spec.EnableTCPS { + if old.Status.DgBrokerConfigured { allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec").Child("InitParams").Child("cpuCount"), "cannot be changed")) - } - if old.Status.InitParams.Processes != r.Spec.InitParams.Processes { + field.Forbidden(field.NewPath("spec").Child("enableTCPS"), "cannot enable tcps as database is in a dataguard configuration")) + } else if len(old.Status.StandbyDatabases) != 0 { allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec").Child("InitParams").Child("processes"), "cannot be changed")) + field.Forbidden(field.NewPath("spec").Child("enableTCPS"), "cannot enable tcps as database is referred by one or more standby databases")) } } - // if Db is in a dataguard configuration. Restrict enabling Tcps on the Primary DB - if old.Status.DgBrokerConfigured && r.Spec.EnableTCPS { - allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec").Child("enableTCPS"), "cannot enable tcps as database is in a dataguard configuration")) - } - if old.Status.DatafilesCreated == "true" && (old.Status.PrebuiltDB != r.Spec.Image.PrebuiltDB) { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("image").Child("prebuiltDB"), "cannot be changed")) } - if r.Spec.CloneFrom == "" && old.Status.Edition != "" && !strings.EqualFold(old.Status.Edition, r.Spec.Edition) { + if r.Spec.Edition != "" && old.Status.Edition != "" && !strings.EqualFold(old.Status.Edition, r.Spec.Edition) { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("edition"), "cannot be changed")) } @@ -424,27 +499,27 @@ func (r *SingleInstanceDatabase) ValidateUpdate(oldRuntimeObject runtime.Object) allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("pdbname"), "cannot be changed")) } - if old.Status.CloneFrom != "" && - (old.Status.CloneFrom == dbcommons.NoCloneRef && r.Spec.CloneFrom != "" || - old.Status.CloneFrom != dbcommons.NoCloneRef && old.Status.CloneFrom != r.Spec.CloneFrom) { + if old.Status.CreatedAs == "clone" && + (old.Status.PrimaryDatabase == dbcommons.ValueUnavailable && r.Spec.PrimaryDatabaseRef != "" || + old.Status.PrimaryDatabase != dbcommons.ValueUnavailable && old.Status.PrimaryDatabase != r.Spec.PrimaryDatabaseRef) { allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec").Child("cloneFrom"), "cannot be changed")) + field.Forbidden(field.NewPath("spec").Child("primaryDatabaseRef"), "cannot be changed")) } if old.Status.OrdsReference != "" && r.Status.Persistence != r.Spec.Persistence { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("persistence"), "uninstall ORDS to change Persistence")) } if len(allErrs) == 0 { - return nil + return nil, nil } - return apierrors.NewInvalid( + return nil, apierrors.NewInvalid( schema.GroupKind{Group: "database.oracle.com", Kind: "SingleInstanceDatabase"}, r.Name, allErrs) } // ValidateDelete implements webhook.Validator so a webhook will be registered for the type -func (r *SingleInstanceDatabase) ValidateDelete() error { +func (r *SingleInstanceDatabase) ValidateDelete() (admission.Warnings, error) { singleinstancedatabaselog.Info("validate delete", "name", r.Name) var allErrs field.ErrorList if r.Status.OrdsReference != "" { @@ -452,9 +527,9 @@ func (r *SingleInstanceDatabase) ValidateDelete() error { field.Forbidden(field.NewPath("status").Child("ordsReference"), "delete "+r.Status.OrdsReference+" to cleanup this SIDB")) } if len(allErrs) == 0 { - return nil + return nil, nil } - return apierrors.NewInvalid( + return nil, apierrors.NewInvalid( schema.GroupKind{Group: "database.oracle.com", Kind: "SingleInstanceDatabase"}, r.Name, allErrs) } diff --git a/apis/database/v1alpha1/webhook_suite_test.go b/apis/database/v1alpha1/webhook_suite_test.go index 343f279c..3f740a41 100644 --- a/apis/database/v1alpha1/webhook_suite_test.go +++ b/apis/database/v1alpha1/webhook_suite_test.go @@ -61,6 +61,8 @@ import ( "sigs.k8s.io/controller-runtime/pkg/envtest" logf "sigs.k8s.io/controller-runtime/pkg/log" "sigs.k8s.io/controller-runtime/pkg/log/zap" + "sigs.k8s.io/controller-runtime/pkg/webhook" + metricsserver "sigs.k8s.io/controller-runtime/pkg/metrics/server" ) // To avoid dot import @@ -136,11 +138,15 @@ var _ = BeforeSuite(func() { webhookInstallOptions := &testEnv.WebhookInstallOptions mgr, err := ctrl.NewManager(cfg, ctrl.Options{ Scheme: scheme, - Host: webhookInstallOptions.LocalServingHost, - Port: webhookInstallOptions.LocalServingPort, - CertDir: webhookInstallOptions.LocalServingCertDir, + WebhookServer: webhook.NewServer(webhook.Options{ + Port: webhookInstallOptions.LocalServingPort, + Host: webhookInstallOptions.LocalServingHost, + CertDir: webhookInstallOptions.LocalServingCertDir, + }), LeaderElection: false, - MetricsBindAddress: "0", + Metrics: metricsserver.Options{ + BindAddress: "0", + }, }) Expect(err).NotTo(HaveOccurred()) diff --git a/apis/database/v1alpha1/zz_generated.deepcopy.go b/apis/database/v1alpha1/zz_generated.deepcopy.go index b6601b6b..f1eaf503 100644 --- a/apis/database/v1alpha1/zz_generated.deepcopy.go +++ b/apis/database/v1alpha1/zz_generated.deepcopy.go @@ -288,6 +288,16 @@ func (in *AutonomousDatabaseBackupSpec) DeepCopyInto(out *AutonomousDatabaseBack *out = new(string) **out = **in } + if in.IsLongTermBackup != nil { + in, out := &in.IsLongTermBackup, &out.IsLongTermBackup + *out = new(bool) + **out = **in + } + if in.RetentionPeriodInDays != nil { + in, out := &in.RetentionPeriodInDays, &out.RetentionPeriodInDays + *out = new(int) + **out = **in + } in.OCIConfig.DeepCopyInto(&out.OCIConfig) } @@ -543,6 +553,13 @@ func (in *AutonomousDatabaseStatus) DeepCopyInto(out *AutonomousDatabaseStatus) (*in)[i].DeepCopyInto(&(*out)[i]) } } + if in.Conditions != nil { + in, out := &in.Conditions, &out.Conditions + *out = make([]v1.Condition, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseStatus. @@ -962,6 +979,13 @@ func (in *DataguardBrokerSpec) DeepCopyInto(out *DataguardBrokerSpec) { *out = make([]string, len(*in)) copy(*out, *in) } + if in.ServiceAnnotations != nil { + in, out := &in.ServiceAnnotations, &out.ServiceAnnotations + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } if in.NodeSelector != nil { in, out := &in.NodeSelector, &out.NodeSelector *out = make(map[string]string, len(*in)) @@ -1245,6 +1269,66 @@ func (in *EnvironmentVariable) DeepCopy() *EnvironmentVariable { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *GsmServiceSpec) DeepCopyInto(out *GsmServiceSpec) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new GsmServiceSpec. +func (in *GsmServiceSpec) DeepCopy() *GsmServiceSpec { + if in == nil { + return nil + } + out := new(GsmServiceSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *GsmShardDetails) DeepCopyInto(out *GsmShardDetails) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new GsmShardDetails. +func (in *GsmShardDetails) DeepCopy() *GsmShardDetails { + if in == nil { + return nil + } + out := new(GsmShardDetails) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *GsmShardGroupSpec) DeepCopyInto(out *GsmShardGroupSpec) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new GsmShardGroupSpec. +func (in *GsmShardGroupSpec) DeepCopy() *GsmShardGroupSpec { + if in == nil { + return nil + } + out := new(GsmShardGroupSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *GsmShardSpaceSpec) DeepCopyInto(out *GsmShardSpaceSpec) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new GsmShardSpaceSpec. +func (in *GsmShardSpaceSpec) DeepCopy() *GsmShardSpaceSpec { + if in == nil { + return nil + } + out := new(GsmShardSpaceSpec) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *GsmSpec) DeepCopyInto(out *GsmSpec) { *out = *in @@ -1838,6 +1922,8 @@ func (in *PDBSpec) DeepCopyInto(out *PDBSpec) { out.PDBTlsCat = in.PDBTlsCat out.AdminName = in.AdminName out.AdminPwd = in.AdminPwd + out.WebServerUsr = in.WebServerUsr + out.WebServerPwd = in.WebServerPwd if in.ReuseTempFile != nil { in, out := &in.ReuseTempFile, &out.ReuseTempFile *out = new(bool) @@ -2027,6 +2113,21 @@ func (in *PrivateEndpointSpec) DeepCopy() *PrivateEndpointSpec { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *SecretDetails) DeepCopyInto(out *SecretDetails) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SecretDetails. +func (in *SecretDetails) DeepCopy() *SecretDetails { + if in == nil { + return nil + } + out := new(SecretDetails) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *ShardSpec) DeepCopyInto(out *ShardSpec) { *out = *in @@ -2166,6 +2267,31 @@ func (in *ShardingDatabaseSpec) DeepCopyInto(out *ShardingDatabaseSpec) { *out = make([]PortMapping, len(*in)) copy(*out, *in) } + if in.GsmShardSpace != nil { + in, out := &in.GsmShardSpace, &out.GsmShardSpace + *out = make([]GsmShardSpaceSpec, len(*in)) + copy(*out, *in) + } + if in.GsmShardGroup != nil { + in, out := &in.GsmShardGroup, &out.GsmShardGroup + *out = make([]GsmShardGroupSpec, len(*in)) + copy(*out, *in) + } + if in.ShardRegion != nil { + in, out := &in.ShardRegion, &out.ShardRegion + *out = make([]string, len(*in)) + copy(*out, *in) + } + if in.GsmService != nil { + in, out := &in.GsmService, &out.GsmService + *out = make([]GsmServiceSpec, len(*in)) + copy(*out, *in) + } + if in.DbSecret != nil { + in, out := &in.DbSecret, &out.DbSecret + *out = new(SecretDetails) + **out = **in + } } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ShardingDatabaseSpec. @@ -2327,6 +2453,11 @@ func (in *SingleInstanceDatabaseList) DeepCopyObject() runtime.Object { // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *SingleInstanceDatabasePersistence) DeepCopyInto(out *SingleInstanceDatabasePersistence) { *out = *in + if in.SetWritePermissions != nil { + in, out := &in.SetWritePermissions, &out.SetWritePermissions + *out = new(bool) + **out = **in + } } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SingleInstanceDatabasePersistence. @@ -2349,6 +2480,21 @@ func (in *SingleInstanceDatabaseSpec) DeepCopyInto(out *SingleInstanceDatabaseSp (*out)[key] = val } } + if in.FlashBack != nil { + in, out := &in.FlashBack, &out.FlashBack + *out = new(bool) + **out = **in + } + if in.ArchiveLog != nil { + in, out := &in.ArchiveLog, &out.ArchiveLog + *out = new(bool) + **out = **in + } + if in.ForceLogging != nil { + in, out := &in.ForceLogging, &out.ForceLogging + *out = new(bool) + **out = **in + } if in.NodeSelector != nil { in, out := &in.NodeSelector, &out.NodeSelector *out = make(map[string]string, len(*in)) @@ -2358,8 +2504,12 @@ func (in *SingleInstanceDatabaseSpec) DeepCopyInto(out *SingleInstanceDatabaseSp } in.AdminPassword.DeepCopyInto(&out.AdminPassword) out.Image = in.Image - out.Persistence = in.Persistence - out.InitParams = in.InitParams + in.Persistence.DeepCopyInto(&out.Persistence) + if in.InitParams != nil { + in, out := &in.InitParams, &out.InitParams + *out = new(SingleInstanceDatabaseInitParams) + **out = **in + } } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SingleInstanceDatabaseSpec. @@ -2395,7 +2545,7 @@ func (in *SingleInstanceDatabaseStatus) DeepCopyInto(out *SingleInstanceDatabase } } out.InitParams = in.InitParams - out.Persistence = in.Persistence + in.Persistence.DeepCopyInto(&out.Persistence) } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SingleInstanceDatabaseStatus. @@ -2546,6 +2696,22 @@ func (in *WebServerPassword) DeepCopy() *WebServerPassword { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *WebServerPasswordPDB) DeepCopyInto(out *WebServerPasswordPDB) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new WebServerPasswordPDB. +func (in *WebServerPasswordPDB) DeepCopy() *WebServerPasswordPDB { + if in == nil { + return nil + } + out := new(WebServerPasswordPDB) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *WebServerUser) DeepCopyInto(out *WebServerUser) { *out = *in @@ -2561,3 +2727,19 @@ func (in *WebServerUser) DeepCopy() *WebServerUser { in.DeepCopyInto(out) return out } + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *WebServerUserPDB) DeepCopyInto(out *WebServerUserPDB) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new WebServerUserPDB. +func (in *WebServerUserPDB) DeepCopy() *WebServerUserPDB { + if in == nil { + return nil + } + out := new(WebServerUserPDB) + in.DeepCopyInto(out) + return out +} diff --git a/apis/observability/v1alpha1/databaseobserver_types.go b/apis/observability/v1alpha1/databaseobserver_types.go new file mode 100644 index 00000000..97827d17 --- /dev/null +++ b/apis/observability/v1alpha1/databaseobserver_types.go @@ -0,0 +1,144 @@ +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v1alpha1 + +import ( + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +type StatusEnum string + +// DatabaseObserverSpec defines the desired state of DatabaseObserver +type DatabaseObserverSpec struct { + Database DatabaseObserverDatabase `json:"database,omitempty"` + Exporter DatabaseObserverExporterConfig `json:"exporter,omitempty"` + Prometheus PrometheusConfig `json:"prometheus,omitempty"` + OCIConfig OCIConfigSpec `json:"ociConfig,omitempty"` + Replicas int32 `json:"replicas,omitempty"` +} + +// DatabaseObserverDatabase defines the database details used for DatabaseObserver +type DatabaseObserverDatabase struct { + DBUser DBSecret `json:"dbUser,omitempty"` + DBPassword DBSecretWithVault `json:"dbPassword,omitempty"` + DBWallet DBSecret `json:"dbWallet,omitempty"` + DBConnectionString DBSecret `json:"dbConnectionString,omitempty"` +} + +// DatabaseObserverExporterConfig defines the configuration details related to the exporters of DatabaseObserver +type DatabaseObserverExporterConfig struct { + ExporterImage string `json:"image,omitempty"` + ExporterConfig DatabaseObserverConfigMap `json:"configuration,omitempty"` + Service DatabaseObserverService `json:"service,omitempty"` +} + +// DatabaseObserverService defines the exporter service component of DatabaseObserver +type DatabaseObserverService struct { + Port int32 `json:"port,omitempty"` +} + +// PrometheusConfig defines the generated resources for Prometheus +type PrometheusConfig struct { + Labels map[string]string `json:"labels,omitempty"` + Port string `json:"port,omitempty"` +} + +type DBSecret struct { + Key string `json:"key,omitempty"` + SecretName string `json:"secret,omitempty"` +} + +type DBSecretWithVault struct { + Key string `json:"key,omitempty"` + SecretName string `json:"secret,omitempty"` + VaultOCID string `json:"vaultOCID,omitempty"` + VaultSecretName string `json:"vaultSecretName,omitempty"` +} + +type DatabaseObserverConfigMap struct { + Configmap ConfigMapDetails `json:"configmap,omitempty"` +} + +// ConfigMapDetails defines the configmap name +type ConfigMapDetails struct { + Key string `json:"key,omitempty"` + Name string `json:"configmapName,omitempty"` +} + +type OCIConfigSpec struct { + ConfigMapName string `json:"configMapName,omitempty"` + SecretName string `json:"secretName,omitempty"` +} + +// DatabaseObserverStatus defines the observed state of DatabaseObserver +type DatabaseObserverStatus struct { + // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster + // Important: Run "make" to regenerate code after modifying this file + Conditions []metav1.Condition `json:"conditions"` + Status string `json:"status,omitempty"` + ExporterConfig string `json:"exporterConfig"` + Replicas int `json:"replicas,omitempty"` +} + +//+kubebuilder:object:root=true +//+kubebuilder:subresource:status + +// DatabaseObserver is the Schema for the databaseobservers API +// +kubebuilder:printcolumn:JSONPath=".status.exporterConfig",name="ExporterConfig",type=string +// +kubebuilder:printcolumn:JSONPath=".status.status",name="Status",type=string +type DatabaseObserver struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + Spec DatabaseObserverSpec `json:"spec,omitempty"` + Status DatabaseObserverStatus `json:"status,omitempty"` +} + +//+kubebuilder:object:root=true + +// DatabaseObserverList contains a list of DatabaseObserver +type DatabaseObserverList struct { + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata,omitempty"` + Items []DatabaseObserver `json:"items"` +} + +func init() { + SchemeBuilder.Register(&DatabaseObserver{}, &DatabaseObserverList{}) +} diff --git a/apis/observability/v1alpha1/databaseobserver_webhook.go b/apis/observability/v1alpha1/databaseobserver_webhook.go new file mode 100644 index 00000000..2ab9b732 --- /dev/null +++ b/apis/observability/v1alpha1/databaseobserver_webhook.go @@ -0,0 +1,185 @@ +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v1alpha1 + +import ( + dbcommons "github.com/oracle/oracle-database-operator/commons/database" + apierrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/runtime/schema" + "k8s.io/apimachinery/pkg/util/validation/field" + ctrl "sigs.k8s.io/controller-runtime" + logf "sigs.k8s.io/controller-runtime/pkg/log" + "sigs.k8s.io/controller-runtime/pkg/webhook" + "sigs.k8s.io/controller-runtime/pkg/webhook/admission" + "strings" +) + +// log is for logging in this package. +var databaseobserverlog = logf.Log.WithName("databaseobserver-resource") + +const ( + AllowedExporterImage = "container-registry.oracle.com/database/observability-exporter" + ErrorSpecValidationMissingConnString = "a required field for database connection string secret is missing or does not have a value" + ErrorSpecValidationMissingDBUser = "a required field for database user secret is missing or does not have a value" + ErrorSpecValidationMissingDBVaultField = "a field for the OCI vault has a value but the other required field is missing or does not have a value" + ErrorSpecValidationMissingOCIConfig = "a field(s) for the OCI Config is missing or does not have a value when fields for the OCI vault has values" + ErrorSpecValidationMissingDBPasswordSecret = "a required field for the database password secret is missing or does not have a value" + ErrorSpecExporterImageNotAllowed = "a different exporter image was found, only official database exporter container images are currently supported" +) + +func (r *DatabaseObserver) SetupWebhookWithManager(mgr ctrl.Manager) error { + return ctrl.NewWebhookManagedBy(mgr). + For(r). + Complete() +} + +// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! + +//+kubebuilder:webhook:path=/mutate-observability-oracle-com-v1alpha1-databaseobserver,mutating=true,sideEffects=none,failurePolicy=fail,groups=observability.oracle.com,resources=databaseobservers,verbs=create;update,versions=v1alpha1,name=mdatabaseobserver.kb.io,admissionReviewVersions=v1 + +var _ webhook.Defaulter = &DatabaseObserver{} + +// Default implements webhook.Defaulter so a webhook will be registered for the type +func (r *DatabaseObserver) Default() { + databaseobserverlog.Info("default", "name", r.Name) + + // TODO(user): fill in your defaulting logic. +} + +// TODO(user): change verbs to "verbs=create;update;delete" if you want to enable deletion validation. +//+kubebuilder:webhook:verbs=create;update,path=/validate-observability-oracle-com-v1alpha1-databaseobserver,mutating=false,sideEffects=none,failurePolicy=fail,groups=observability.oracle.com,resources=databaseobservers,versions=v1alpha1,name=vdatabaseobserver.kb.io,admissionReviewVersions=v1 + +var _ webhook.Validator = &DatabaseObserver{} + +// ValidateCreate implements webhook.Validator so a webhook will be registered for the type +func (r *DatabaseObserver) ValidateCreate() (admission.Warnings, error) { + databaseobserverlog.Info("validate create", "name", r.Name) + + var e field.ErrorList + ns := dbcommons.GetWatchNamespaces() + + // Check for namespace/cluster scope access + if _, isDesiredNamespaceWithinScope := ns[r.Namespace]; !isDesiredNamespaceWithinScope && len(ns) > 0 { + e = append(e, + field.Invalid(field.NewPath("metadata").Child("namespace"), r.Namespace, + "Oracle database operator doesn't watch over this namespace")) + } + + // Check required secret for db user has value + if r.Spec.Database.DBUser.SecretName == "" { + e = append(e, + field.Invalid(field.NewPath("spec").Child("database").Child("dbUser").Child("secret"), r.Spec.Database.DBUser.SecretName, + ErrorSpecValidationMissingDBUser)) + } + + // Check required secret for db connection string has value + if r.Spec.Database.DBConnectionString.SecretName == "" { + e = append(e, + field.Invalid(field.NewPath("spec").Child("database").Child("dbConnectionString").Child("secret"), r.Spec.Database.DBConnectionString.SecretName, + ErrorSpecValidationMissingConnString)) + } + + // The other vault field must have value if one does + if (r.Spec.Database.DBPassword.VaultOCID != "" && r.Spec.Database.DBPassword.VaultSecretName == "") || + (r.Spec.Database.DBPassword.VaultSecretName != "" && r.Spec.Database.DBPassword.VaultOCID == "") { + + e = append(e, + field.Invalid(field.NewPath("spec").Child("database").Child("dbPassword"), r.Spec.Database.DBPassword, + ErrorSpecValidationMissingDBVaultField)) + } + + // if vault fields have value, ociConfig must have values + if r.Spec.Database.DBPassword.VaultOCID != "" && r.Spec.Database.DBPassword.VaultSecretName != "" && + (r.Spec.OCIConfig.SecretName == "" || r.Spec.OCIConfig.ConfigMapName == "") { + + e = append(e, + field.Invalid(field.NewPath("spec").Child("ociConfig"), r.Spec.OCIConfig, + ErrorSpecValidationMissingOCIConfig)) + } + + // If all of {DB Password Secret Name and vaultOCID+vaultSecretName} have no value, then error out + if r.Spec.Database.DBPassword.SecretName == "" && + r.Spec.Database.DBPassword.VaultOCID == "" && + r.Spec.Database.DBPassword.VaultSecretName == "" { + + e = append(e, + field.Invalid(field.NewPath("spec").Child("database").Child("dbPassword").Child("secret"), r.Spec.Database.DBPassword.SecretName, + ErrorSpecValidationMissingDBPasswordSecret)) + } + + // disallow usage of any other image than the observability-exporter + if r.Spec.Exporter.ExporterImage != "" && !strings.HasPrefix(r.Spec.Exporter.ExporterImage, AllowedExporterImage) { + e = append(e, + field.Invalid(field.NewPath("spec").Child("exporter").Child("image"), r.Spec.Exporter.ExporterImage, + ErrorSpecExporterImageNotAllowed)) + } + + // Return if any errors + if len(e) > 0 { + return nil, apierrors.NewInvalid(schema.GroupKind{Group: "observability.oracle.com", Kind: "DatabaseObserver"}, r.Name, e) + } + return nil, nil + +} + +// ValidateUpdate implements webhook.Validator so a webhook will be registered for the type +func (r *DatabaseObserver) ValidateUpdate(old runtime.Object) (admission.Warnings, error) { + databaseobserverlog.Info("validate update", "name", r.Name) + var e field.ErrorList + + // disallow usage of any other image than the observability-exporter + if r.Spec.Exporter.ExporterImage != "" && !strings.HasPrefix(r.Spec.Exporter.ExporterImage, AllowedExporterImage) { + e = append(e, + field.Invalid(field.NewPath("spec").Child("exporter").Child("image"), r.Spec.Exporter.ExporterImage, + ErrorSpecExporterImageNotAllowed)) + } + // Return if any errors + if len(e) > 0 { + return nil, apierrors.NewInvalid(schema.GroupKind{Group: "observability.oracle.com", Kind: "DatabaseObserver"}, r.Name, e) + } + return nil, nil +} + +// ValidateDelete implements webhook.Validator so a webhook will be registered for the type +func (r *DatabaseObserver) ValidateDelete() (admission.Warnings, error) { + databaseobserverlog.Info("validate delete", "name", r.Name) + + return nil, nil +} diff --git a/apis/observability/v1alpha1/groupversion_info.go b/apis/observability/v1alpha1/groupversion_info.go new file mode 100644 index 00000000..304840f4 --- /dev/null +++ b/apis/observability/v1alpha1/groupversion_info.go @@ -0,0 +1,58 @@ +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +// Package v1alpha1 contains API Schema definitions for the observability v1alpha1 API group +// +kubebuilder:object:generate=true +// +groupName=observability.oracle.com +package v1alpha1 + +import ( + "k8s.io/apimachinery/pkg/runtime/schema" + "sigs.k8s.io/controller-runtime/pkg/scheme" +) + +var ( + // GroupVersion is group version used to register these objects + GroupVersion = schema.GroupVersion{Group: "observability.oracle.com", Version: "v1alpha1"} + + // SchemeBuilder is used to add go types to the GroupVersionKind scheme + SchemeBuilder = &scheme.Builder{GroupVersion: GroupVersion} + + // AddToScheme adds the types in this group-version to the given scheme. + AddToScheme = SchemeBuilder.AddToScheme +) diff --git a/apis/observability/v1alpha1/zz_generated.deepcopy.go b/apis/observability/v1alpha1/zz_generated.deepcopy.go new file mode 100644 index 00000000..39b438eb --- /dev/null +++ b/apis/observability/v1alpha1/zz_generated.deepcopy.go @@ -0,0 +1,298 @@ +//go:build !ignore_autogenerated +// +build !ignore_autogenerated + +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +// Code generated by controller-gen. DO NOT EDIT. + +package v1alpha1 + +import ( + "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/runtime" +) + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *ConfigMapDetails) DeepCopyInto(out *ConfigMapDetails) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ConfigMapDetails. +func (in *ConfigMapDetails) DeepCopy() *ConfigMapDetails { + if in == nil { + return nil + } + out := new(ConfigMapDetails) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DBSecret) DeepCopyInto(out *DBSecret) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DBSecret. +func (in *DBSecret) DeepCopy() *DBSecret { + if in == nil { + return nil + } + out := new(DBSecret) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DBSecretWithVault) DeepCopyInto(out *DBSecretWithVault) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DBSecretWithVault. +func (in *DBSecretWithVault) DeepCopy() *DBSecretWithVault { + if in == nil { + return nil + } + out := new(DBSecretWithVault) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DatabaseObserver) DeepCopyInto(out *DatabaseObserver) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + in.Spec.DeepCopyInto(&out.Spec) + in.Status.DeepCopyInto(&out.Status) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserver. +func (in *DatabaseObserver) DeepCopy() *DatabaseObserver { + if in == nil { + return nil + } + out := new(DatabaseObserver) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *DatabaseObserver) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DatabaseObserverConfigMap) DeepCopyInto(out *DatabaseObserverConfigMap) { + *out = *in + out.Configmap = in.Configmap +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserverConfigMap. +func (in *DatabaseObserverConfigMap) DeepCopy() *DatabaseObserverConfigMap { + if in == nil { + return nil + } + out := new(DatabaseObserverConfigMap) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DatabaseObserverDatabase) DeepCopyInto(out *DatabaseObserverDatabase) { + *out = *in + out.DBUser = in.DBUser + out.DBPassword = in.DBPassword + out.DBWallet = in.DBWallet + out.DBConnectionString = in.DBConnectionString +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserverDatabase. +func (in *DatabaseObserverDatabase) DeepCopy() *DatabaseObserverDatabase { + if in == nil { + return nil + } + out := new(DatabaseObserverDatabase) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DatabaseObserverExporterConfig) DeepCopyInto(out *DatabaseObserverExporterConfig) { + *out = *in + out.ExporterConfig = in.ExporterConfig + out.Service = in.Service +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserverExporterConfig. +func (in *DatabaseObserverExporterConfig) DeepCopy() *DatabaseObserverExporterConfig { + if in == nil { + return nil + } + out := new(DatabaseObserverExporterConfig) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DatabaseObserverList) DeepCopyInto(out *DatabaseObserverList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]DatabaseObserver, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserverList. +func (in *DatabaseObserverList) DeepCopy() *DatabaseObserverList { + if in == nil { + return nil + } + out := new(DatabaseObserverList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *DatabaseObserverList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DatabaseObserverService) DeepCopyInto(out *DatabaseObserverService) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserverService. +func (in *DatabaseObserverService) DeepCopy() *DatabaseObserverService { + if in == nil { + return nil + } + out := new(DatabaseObserverService) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DatabaseObserverSpec) DeepCopyInto(out *DatabaseObserverSpec) { + *out = *in + out.Database = in.Database + out.Exporter = in.Exporter + in.Prometheus.DeepCopyInto(&out.Prometheus) + out.OCIConfig = in.OCIConfig +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserverSpec. +func (in *DatabaseObserverSpec) DeepCopy() *DatabaseObserverSpec { + if in == nil { + return nil + } + out := new(DatabaseObserverSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DatabaseObserverStatus) DeepCopyInto(out *DatabaseObserverStatus) { + *out = *in + if in.Conditions != nil { + in, out := &in.Conditions, &out.Conditions + *out = make([]v1.Condition, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserverStatus. +func (in *DatabaseObserverStatus) DeepCopy() *DatabaseObserverStatus { + if in == nil { + return nil + } + out := new(DatabaseObserverStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *OCIConfigSpec) DeepCopyInto(out *OCIConfigSpec) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new OCIConfigSpec. +func (in *OCIConfigSpec) DeepCopy() *OCIConfigSpec { + if in == nil { + return nil + } + out := new(OCIConfigSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PrometheusConfig) DeepCopyInto(out *PrometheusConfig) { + *out = *in + if in.Labels != nil { + in, out := &in.Labels, &out.Labels + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PrometheusConfig. +func (in *PrometheusConfig) DeepCopy() *PrometheusConfig { + if in == nil { + return nil + } + out := new(PrometheusConfig) + in.DeepCopyInto(out) + return out +} diff --git a/commons/database/constants.go b/commons/database/constants.go index de07c0d4..6f27750d 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -50,8 +50,6 @@ const DBA_GUID int64 = 54322 const SQLPlusCLI string = "sqlplus -s / as sysdba" -const NoCloneRef string = "Unavailable" - const GetVersionSQL string = "SELECT VERSION_FULL FROM V\\$INSTANCE;" const CheckModesSQL string = "SELECT 'log_mode:' || log_mode AS log_mode ,'flashback_on:' || flashback_on AS flashback_on ,'force_logging:' || force_logging AS force_logging FROM v\\$database;" @@ -102,6 +100,8 @@ const StandbyDatabasePrerequisitesSQL string = "ALTER SYSTEM SET db_create_file_ "\nALTER DATABASE ADD STANDBY LOGFILE THREAD 1 SIZE 200M;" + "\nALTER DATABASE ADD STANDBY LOGFILE THREAD 1 SIZE 200M;" + "\nALTER SYSTEM SET STANDBY_FILE_MANAGEMENT=AUTO;" + + "\nALTER SYSTEM SET dg_broker_config_file1='/opt/oracle/oradata/dbconfig/dr1${ORACLE_SID}.dat' scope=both;" + + "\nALTER SYSTEM SET dg_broker_config_file2='/opt/oracle/oradata/dbconfig/dr2${ORACLE_SID}.dat';" + "\nALTER SYSTEM SET dg_broker_start=TRUE;" const GetDBOpenMode string = "select open_mode from v\\$database;" @@ -422,13 +422,16 @@ const InitWalletCMD string = "if [ ! -f $ORACLE_BASE/oradata/.${ORACLE_SID}${CHE const InitPrebuiltDbCMD string = "if [ ! -d /mnt/oradata/${ORACLE_SID} -a -d $ORACLE_BASE/oradata/${ORACLE_SID} ]; then cp -v $ORACLE_BASE/oradata/.${ORACLE_SID}$CHECKPOINT_FILE_EXTN /mnt/oradata && " + " cp -vr $ORACLE_BASE/oradata/${ORACLE_SID} /mnt/oradata && cp -vr $ORACLE_BASE/oradata/dbconfig /mnt/oradata; fi " -const AlterSgaPgaCpuCMD string = "echo -e \"alter system set sga_target=%dM scope=both; \n alter system set pga_aggregate_target=%dM scope=both; \n alter system set cpu_count=%d; \" | %s " - +const AlterSgaPgaCMD string = "echo -e \"alter system set sga_target=%dM scope=both; \n alter system set pga_aggregate_target=%dM scope=both; \" | %s " +const AlterCpuCountCMD string = "echo -e \"alter system set cpu_count=%d; \" | %s" const AlterProcessesCMD string = "echo -e \"alter system set processes=%d scope=spfile; \" | %s && " + CreateChkFileCMD + " && " + "echo -e \"SHUTDOWN IMMEDIATE; \n STARTUP MOUNT; \n ALTER DATABASE OPEN; \n ALTER PLUGGABLE DATABASE ALL OPEN; \n ALTER SYSTEM REGISTER;\" | %s && " + RemoveChkFileCMD -const GetInitParamsSQL string = "echo -e \"select name,display_value from v\\$parameter where name in ('sga_target','pga_aggregate_target','cpu_count','processes') order by name asc;\" | %s" +const GetInitParamsSQL string = "column name format a20;" + + "\ncolumn display_value format a20;" + + "\nset linesize 100 pagesize 50;" + + "\nselect name,display_value from v\\$parameter where name in ('sga_target','pga_aggregate_target','cpu_count','processes') order by name asc;" const UnzipApexOnSIDBPod string = "if [ -f /opt/oracle/oradata/apex-latest.zip ]; then unzip -o /opt/oracle/oradata/apex-latest.zip -d /opt/oracle/oradata/${ORACLE_SID^^}; else echo \"apex-latest.zip not found\"; fi;" @@ -517,6 +520,12 @@ const RenewCertsCMD string = EnableTcpsCMD // Command to disable TCPS const DisableTcpsCMD string = "$ORACLE_BASE/$CONFIG_TCPS_FILE disable" +// Location of tls certs +const TlsCertsLocation string = "/run/secrets/tls_secret" + +// Check Mount in pods +const PodMountsCmd string = "awk '$2 == \"%s\" {print}' /proc/mounts" + // TCPS clientWallet update command const ClientWalletUpdate string = "sed -i -e 's/HOST.*$/HOST=%s)/g' -e 's/PORT.*$/PORT=%d)/g' ${ORACLE_BASE}/oradata/clientWallet/${ORACLE_SID}/tnsnames.ora" diff --git a/commons/database/utils.go b/commons/database/utils.go index 6e6f6d02..1723bc90 100644 --- a/commons/database/utils.go +++ b/commons/database/utils.go @@ -45,6 +45,8 @@ import ( "errors" "fmt" "math/rand" + "os" + "strconv" "strings" "time" "unicode" @@ -70,6 +72,8 @@ import ( var requeueY ctrl.Result = ctrl.Result{Requeue: true, RequeueAfter: 15 * time.Second} var requeueN ctrl.Result = ctrl.Result{} +var ErrNoReadyPod = errors.New("SingleInstanceDatabase has no ready pod currently") + // Filter events that trigger reconcilation func ResourceEventHandler() predicate.Predicate { return predicate.Funcs{ @@ -400,6 +404,74 @@ func CheckDBConfig(readyPod corev1.Pod, r client.Reader, config *rest.Config, return flashBackStatus, archiveLogStatus, forceLoggingStatus, requeueN } +func CheckDBInitParams(sidbReadyPod corev1.Pod, r client.Reader, config *rest.Config, + ctx context.Context, req ctrl.Request) (int, int, int, int, error) { + log := ctrllog.FromContext(ctx).WithValues("CheckDBParams", req.NamespacedName) + + if sidbReadyPod.Name == "" { + log.Info("No Pod is Ready") + // As No pod is ready now , turn on mode when pod is ready . so requeue the request + return -1, -1, -1, -1, fmt.Errorf("no pod is ready") + } + + log.Info("Check database init params") + + out, err := ExecCommand(r, config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", + ctx, req, false, "bash", "-c", fmt.Sprintf("echo -e \"%s\" | sqlplus -s / as sysdba", GetInitParamsSQL)) + if err != nil { + log.Error(err, err.Error()) + return -1, -1, -1, -1, err + } + if strings.Contains(out, "no rows selected") { + return -1, -1, -1, -1, errors.New("cannot fetch values for database init params") + } + if strings.Contains(out, "ORA-") { + return -1, -1, -1, -1, fmt.Errorf("error while getting database init params\n%s", out) + } + log.Info(fmt.Sprintf("Database initParams are \n%s", out)) + initParams := strings.Split(out, "\n") + initParams = initParams[3:] + log.Info(fmt.Sprintf("%v", initParams)) + log.Info(fmt.Sprintf("length of initParams is %v", len(initParams))) + log.Info("After parsing init param are " + strings.Join(initParams, ",")) + + log.Info("Parsing cpuCount") + log.Info(strings.Fields(initParams[0])[1]) + cpu_count, err := strconv.Atoi(strings.Fields(initParams[0])[1]) + if err != nil { + return -1, -1, -1, -1, err + } + log.Info("After parsing cpuCount", "cpuCount", cpu_count) + + log.Info("Parsing pga_aggregate_target_value") + log.Info(strings.Fields(initParams[1])[1]) + pga_aggregate_target_value := strings.Fields(initParams[1])[1] + pga_aggregate_target, err := strconv.Atoi(pga_aggregate_target_value[0 : len(pga_aggregate_target_value)-1]) + if err != nil { + return -1, -1, -1, -1, err + } + log.Info("After parsing pga_aggregate_target_value", "pga_aggregate_target_value", pga_aggregate_target) + + log.Info("Parsing processes") + log.Info(strings.Fields(initParams[2])[1]) + processes, err := strconv.Atoi(strings.Fields(initParams[2])[1]) + if err != nil { + return -1, -1, -1, -1, err + } + log.Info("After parsing processes", "processes", processes) + + log.Info("parsing sga_target_value") + log.Info(strings.Fields(initParams[3])[1]) + sga_target_value := strings.Fields(initParams[3])[1] + sga_target, err := strconv.Atoi(sga_target_value[0 : len(sga_target_value)-1]) + if err != nil { + return -1, -1, -1, -1, err + } + log.Info("After parsing sgaTarget", "sgaTarget", sga_target) + + return cpu_count, pga_aggregate_target, processes, sga_target, nil +} + // CHECKS IF SID IN DATABASES SLICE , AND ITS DGROLE func IsDatabaseFound(sid string, databases []string, dgrole string) (bool, bool) { found := false @@ -458,34 +530,37 @@ func GetDatabasesInDgConfig(readyPod corev1.Pod, r client.Reader, // Returns Database version func GetDatabaseVersion(readyPod corev1.Pod, r client.Reader, - config *rest.Config, ctx context.Context, req ctrl.Request, edition string) (string, string, error) { + config *rest.Config, ctx context.Context, req ctrl.Request) (string, error) { + log := ctrllog.FromContext(ctx).WithValues("GetDatabaseVersion", req.NamespacedName) // ## FIND DATABASES PRESENT IN DG CONFIGURATION out, err := ExecCommand(r, config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", fmt.Sprintf("echo -e \"%s\" | %s", GetVersionSQL, SQLPlusCLI)) if err != nil { - return "", "", err + return "", err } log.Info("GetDatabaseVersion Output") log.Info(out) - - if !strings.Contains(out, "no rows selected") && !strings.Contains(out, "ORA-") { - out1 := strings.Replace(out, " ", "_", -1) - // filtering output and storing databses in dg configuration in "databases" slice - out2 := strings.Fields(out1) - - // first 2 values in the slice will be column name(VERSION) and a seperator(--------------) . so the version would be out2[2] - version := out2[2] - return version, out, nil + if strings.Contains(out, "no rows selected") { + return "", errors.New("cannot fetch database version") + } + if strings.Contains(out, "ORA-") { + return "", errors.New("error while trying to get the database version " + out) } - return "", out, errors.New("database version is nil") + out1 := strings.Replace(out, " ", "_", -1) + // filtering output and storing databses in dg configuration in "databases" slice + out2 := strings.Fields(out1) + // first 2 values in the slice will be column name(VERSION) and a seperator(--------------) . so the version would be out2[2] + version := out2[2] + return version, nil } // Fetch role by quering the DB func GetDatabaseRole(readyPod corev1.Pod, r client.Reader, - config *rest.Config, ctx context.Context, req ctrl.Request, edition string) (string, error) { + config *rest.Config, ctx context.Context, req ctrl.Request) (string, error) { + log := ctrllog.FromContext(ctx).WithValues("GetDatabaseRole", req.NamespacedName) out, err := ExecCommand(r, config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", @@ -497,7 +572,7 @@ func GetDatabaseRole(readyPod corev1.Pod, r client.Reader, if !strings.Contains(out, "no rows selected") && !strings.Contains(out, "ORA-") { out = strings.Replace(out, " ", "_", -1) // filtering output and storing databse_role in "database_role" - databaseRole := strings.Fields(out)[2] + databaseRole := strings.ToUpper(strings.Fields(out)[2]) // first 2 values in the slice will be column name(DATABASE_ROLE) and a seperator(--------------) . return databaseRole, nil @@ -610,9 +685,9 @@ func GetSidPdbEdition(r client.Reader, config *rest.Config, ctx context.Context, splitstr := strings.Split((strings.TrimSpace(out)), ",") return splitstr[0], splitstr[1], splitstr[2], nil } - err = errors.New("ready pod name is nil") - log.Error(err, err.Error()) - return "", "", "", err + // err = errors.New("ready pod name is nil") + log.Error(err, ErrNoReadyPod.Error()) + return "", "", "", ErrNoReadyPod } // Get Datapatch Status @@ -713,3 +788,20 @@ func PatchService(config *rest.Config, namespace string, ctx context.Context, re _, err = client.CoreV1().Services(namespace).Patch(ctx, svcName, types.MergePatchType, []byte(payload), metav1.PatchOptions{}) return err } + +func GetWatchNamespaces() map[string]bool { + // Fetching the allowed namespaces from env variables + var watchNamespaceEnvVar = "WATCH_NAMESPACE" + ns, _ := os.LookupEnv(watchNamespaceEnvVar) + ns = strings.TrimSpace(ns) + namespaces := make(map[string]bool) + if len(ns) == 0 { + return namespaces + } + namespacesArr := strings.Split(ns, ",") + // put slice values into map + for _, s := range namespacesArr { + namespaces[s] = true + } + return namespaces +} diff --git a/commons/dbcssystem/dbcs_reconciler.go b/commons/dbcssystem/dbcs_reconciler.go index d4df1ab4..70e50294 100644 --- a/commons/dbcssystem/dbcs_reconciler.go +++ b/commons/dbcssystem/dbcs_reconciler.go @@ -94,6 +94,7 @@ func CreateAndGetDbcsId(logger logr.Logger, kubeClient client.Client, dbClient d dbcsDetails.CompartmentId = common.String(dbcs.Spec.DbSystem.CompartmentId) dbcsDetails.SubnetId = common.String(dbcs.Spec.DbSystem.SubnetId) dbcsDetails.Shape = common.String(dbcs.Spec.DbSystem.Shape) + dbcsDetails.Domain = common.String(dbcs.Spec.DbSystem.Domain) if dbcs.Spec.DbSystem.DisplayName != "" { dbcsDetails.DisplayName = common.String(dbcs.Spec.DbSystem.DisplayName) } @@ -535,6 +536,11 @@ func GetResourceState(logger logr.Logger, dbClient database.DatabaseClient, Id s func SetDBCSStatus(dbClient database.DatabaseClient, dbcs *databasev1alpha1.DbcsSystem, nwClient core.VirtualNetworkClient, wrClient workrequests.WorkRequestClient) error { + if dbcs.Spec.Id == nil { + dbcs.Status.State = "FAILED" + return nil + } + dbcsId := *dbcs.Spec.Id dbcsReq := database.GetDbSystemRequest{ diff --git a/commons/observability/constants.go b/commons/observability/constants.go new file mode 100644 index 00000000..a4d85b71 --- /dev/null +++ b/commons/observability/constants.go @@ -0,0 +1,172 @@ +package observability + +import "github.com/oracle/oracle-database-operator/apis/observability/v1alpha1" + +const ( + UnknownValue = "UNKNOWN" + DefaultValue = "DEFAULT" +) + +// Observability Status +const ( + StatusObservabilityPending v1alpha1.StatusEnum = "PENDING" + StatusObservabilityError v1alpha1.StatusEnum = "ERROR" + StatusObservabilityReady v1alpha1.StatusEnum = "READY" +) + +// Log Names +const ( + LogReconcile = "ObservabilityExporterLogger" + LogExportersDeploy = "ObservabilityExporterDeploymentLogger" + LogExportersSVC = "ObservabilityExporterServiceLogger" + LogExportersServiceMonitor = "ObservabilityExporterServiceMonitorLogger" +) + +// Defaults +const ( + DefaultDbUserKey = "username" + DefaultDBPasswordKey = "password" + DefaultDBConnectionStringKey = "connection" + DefaultLabelKey = "app" + DefaultConfigVolumeString = "config-volume" + DefaultWalletVolumeString = "creds" + DefaultOCIPrivateKeyVolumeString = "ocikey" + DefaultOCIConfigFingerprintKey = "fingerprint" + DefaultOCIConfigRegionKey = "region" + DefaultOCIConfigTenancyKey = "tenancy" + DefaultOCIConfigUserKey = "user" + + DefaultExporterImage = "container-registry.oracle.com/database/observability-exporter:1.1.0" + DefaultServicePort = 9161 + DefaultPrometheusPort = "metrics" + DefaultReplicaCount = 1 + DefaultExporterConfigMountRootPath = "/oracle/observability" + DefaultOracleHome = "/lib/oracle/21/client64/lib" + DefaultOracleTNSAdmin = DefaultOracleHome + "/network/admin" + DefaultExporterConfigmapFilename = "config.toml" + DefaultVaultPrivateKeyRootPath = "/oracle/config" + DefaultPrivateKeyFileKey = "privatekey" + DefaultPrivateKeyFileName = "private.pem" + DefaultVaultPrivateKeyAbsolutePath = DefaultVaultPrivateKeyRootPath + "/" + DefaultPrivateKeyFileName + DefaultExporterConfigmapAbsolutePath = DefaultExporterConfigMountRootPath + "/" + DefaultExporterConfigmapFilename +) + +// default resource prefixes +const ( + DefaultServiceMonitorPrefix = "obs-servicemonitor-" + DefaultLabelPrefix = "obs-" + DefaultExporterDeploymentPrefix = "obs-deploy-" + DefaultExporterContainerName = "observability-exporter" +) + +// Known environment variables +const ( + EnvVarOracleHome = "ORACLE_HOME" + EnvVarDataSourceUser = "DB_USERNAME" + EnvVarDataSourcePassword = "DB_PASSWORD" + EnvVarDataSourceConnectString = "DB_CONNECT_STRING" + EnvVarDataSourcePwdVaultSecretName = "VAULT_SECRET_NAME" + EnvVarDataSourcePwdVaultId = "VAULT_ID" + EnvVarCustomConfigmap = "CUSTOM_METRICS" + EnvVarTNSAdmin = "TNS_ADMIN" + EnvVarVaultTenancyOCID = "vault_tenancy_ocid" + EnvVarVaultUserOCID = "vault_user_ocid" + EnvVarVaultFingerprint = "vault_fingerprint" + EnvVarVaultPrivateKeyPath = "vault_private_key_path" + EnvVarVaultRegion = "vault_region" +) + +// Positive ConditionTypes +const ( + IsCRAvailable = "ExporterReady" + IsExporterDeploymentReady = "DeploymentReady" + IsExporterServiceReady = "ServiceReady" + IsExporterServiceMonitorReady = "ServiceMonitorReady" +) + +// Reason +const ( + ReasonInitStart = "InitializationStarted" + ReasonReadyValidated = "ReadinessValidated" + ReasonValidationInProgress = "ReadinessValidationInProgress" + ReasonReadyFailed = "ReadinessValidationFailed" + ReasonDeploymentSpecValidationFailed = "SpecValidationFailed" + + ReasonDeploymentSuccessful = "ResourceDeployed" + ReasonDeploymentUpdated = "ResourceDeploymentUpdated" + ReasonDeploymentUpdateFailed = "ResourceDeploymentUpdateFailed" + ReasonDeploymentFailed = "ResourceDeploymentFailed" + ReasonDeploymentPending = "ResourceDeploymentInProgress" + + ReasonGeneralResourceGenerationFailed = "ResourceGenerationFailed" + ReasonGeneralResourceCreated = "ResourceCreated" + ReasonGeneralResourceCreationFailed = "ResourceCreationFailed" + ReasonGeneralResourceValidationCompleted = "ResourceDeployed" + ReasonGeneralResourceValidationFailureDueToError = "ResourceCouldNotBeValidated" +) + +// Log Errors +const ( + ErrorCRRetrieve = "an error occurred with retrieving the cr" + ErrorStatusUpdate = "an error occurred with updating the cr status" + ErrorSpecValidationFailedDueToAnError = "an error occurred with validating the exporter deployment spec" + ErrorDeploymentPodsFailure = "an error occurred with deploying exporter deployment pods" + ErrorDeploymentUpdate = "an error occurred with updating exporter deployment" + ErrorResourceCreationFailure = "an error occurred with creating databaseobserver resource" + ErrorResourceRetrievalFailureDueToAnError = "an error occurred with retrieving databaseobserver resource" +) + +// Log Infos +const ( + LogCRStart = "Started DatabaseObserver instance reconciliation" + LogCREnd = "Ended DatabaseObserver instance reconciliation, resource must have been deleted." + LogResourceCreated = "Created DatabaseObserver resource successfully" + LogResourceUpdated = "Updated DatabaseObserver resource successfully" + LogResourceFound = "Validated DatabaseObserver resource readiness" +) + +// Messages +const ( + MessageCRInitializationStarted = "Started initialization of custom resource" + MessageCRValidated = "Completed validation of custom resource readiness successfully" + MessageCRValidationFailed = "Failed to validate readiness of custom resource due to an error" + MessageCRValidationWaiting = "Waiting for other resources to be ready to fully validate readiness" + + MessageResourceCreated = "Completed creation of resource successfully" + MessageResourceCreationFailed = "Failed to create resource due to an error" + MessageResourceReadinessValidated = "Completed validation of resource readiness" + MessageResourceReadinessValidationFailed = "Failed to validate resource due to an error retrieving resource" + MessageResourceGenerationFailed = "Failed to generate resource due to an error" + + MessageExporterDeploymentSpecValidationFailed = "Failed to validate export deployment spec due to an error with the spec" + MessageExporterDeploymentImageUpdated = "Completed updating exporter deployment image successfully" + MessageExporterDeploymentEnvironmentUpdated = "Completed updating exporter deployment environment values successfully" + MessageExporterDeploymentReplicaUpdated = "Completed updating exporter deployment replicaCount successfully" + MessageExporterDeploymentVolumesUpdated = "Completed updating exporter deployment volumes successfully" + MessageExporterDeploymentUpdateFailed = "Failed to update exporter deployment due to an error" + MessageExporterDeploymentValidationFailed = "Failed to validate exporter deployment due to an error retrieving resource" + MessageExporterDeploymentSuccessful = "Completed validation of exporter deployment readiness" + MessageExporterDeploymentFailed = "Failed to deploy exporter deployment due to PodFailure" + MessageExporterDeploymentListingFailed = "Failed to list exporter deployment pods" + MessageExporterDeploymentPending = "Waiting for exporter deployment pods to be ready" +) + +// Event Recorder Outputs +const ( + EventReasonFailedCRRetrieval = "ExporterRetrievalFailed" + EventMessageFailedCRRetrieval = "Encountered error retrieving databaseObserver instance" + + EventReasonSpecError = "DeploymentSpecValidationFailed" + EventMessageSpecErrorDBPasswordMissing = "Spec validation failed due to missing dbPassword field values" + EventMessageSpecErrorDBPasswordSecretMissing = "Spec validation failed due to required dbPassword secret not found" + EventMessageSpecErrorDBConnectionStringSecretMissing = "Spec validation failed due to required dbConnectionString secret not found" + EventMessageSpecErrorDBPUserSecretMissing = "Spec validation failed due to dbUser secret not found" + EventMessageSpecErrorConfigmapMissing = "Spec validation failed due to custom config configmap not found" + EventMessageSpecErrorDBWalletSecretMissing = "Spec validation failed due to provided dbWallet secret not found" + + EventReasonUpdateSucceeded = "ExporterDeploymentUpdated" + EventMessageUpdatedImageSucceeded = "Exporter deployment image updated successfully" + EventMessageUpdatedEnvironmentSucceeded = "Exporter deployment environment values updated successfully" + EventMessageUpdatedVolumesSucceeded = "Exporter deployment volumes updated successfully" + EventMessageUpdatedReplicaSucceeded = "Exporter deployment replicaCount updated successfully" +) diff --git a/commons/observability/utils.go b/commons/observability/utils.go new file mode 100644 index 00000000..f396b95a --- /dev/null +++ b/commons/observability/utils.go @@ -0,0 +1,402 @@ +package observability + +import ( + apiv1 "github.com/oracle/oracle-database-operator/apis/observability/v1alpha1" + appsv1 "k8s.io/api/apps/v1" + corev1 "k8s.io/api/core/v1" +) + +// GetExporterLabels function retrieves exporter labels from api or provides default +func GetExporterLabels(api *apiv1.DatabaseObserver) map[string]string { + var l = make(map[string]string) + + if labels := api.Spec.Prometheus.Labels; labels != nil && len(labels) > 0 { + for k, v := range labels { + l[k] = v + } + l["release"] = "stable" + return l + } + return map[string]string{ + DefaultLabelKey: DefaultLabelPrefix + api.Name, + "release": "stable", + } + +} + +// GetExporterServicePort function retrieves exporter service port from api or provides default +func GetExporterServicePort(api *apiv1.DatabaseObserver) int32 { + if rPort := api.Spec.Exporter.Service.Port; rPort != 0 { + return rPort + } + return int32(DefaultServicePort) +} + +// GetExporterServiceMonitorPort function retrieves exporter service monitor port from api or provides default +func GetExporterServiceMonitorPort(api *apiv1.DatabaseObserver) string { + if rPort := api.Spec.Prometheus.Port; rPort != "" { + return rPort + } + return DefaultPrometheusPort + +} + +// GetExporterDeploymentVolumeMounts function retrieves volume mounts from api or provides default +func GetExporterDeploymentVolumeMounts(api *apiv1.DatabaseObserver) []corev1.VolumeMount { + + volM := make([]corev1.VolumeMount, 0) + + if cVolumeSourceName := api.Spec.Exporter.ExporterConfig.Configmap.Name; cVolumeSourceName != "" { + volM = append(volM, corev1.VolumeMount{ + Name: DefaultConfigVolumeString, + MountPath: DefaultExporterConfigMountRootPath, + }) + } + + // api.Spec.Database.DBWallet.SecretName optional + // if null, consider the database NON-ADB and connect as such + if secretName := api.Spec.Database.DBWallet.SecretName; secretName != "" { + volM = append(volM, corev1.VolumeMount{ + Name: DefaultWalletVolumeString, + MountPath: DefaultOracleTNSAdmin, + }) + } + + // api.Spec.OCIConfig.SecretName required if vault is used + if secretName := api.Spec.OCIConfig.SecretName; secretName != "" { + volM = append(volM, corev1.VolumeMount{ + Name: DefaultOCIPrivateKeyVolumeString, + MountPath: DefaultVaultPrivateKeyRootPath, + }) + } + return volM +} + +// GetExporterDeploymentVolumes function retrieves volumes from api or provides default +func GetExporterDeploymentVolumes(api *apiv1.DatabaseObserver) []corev1.Volume { + + vol := make([]corev1.Volume, 0) + + // config-volume Volume + // if null, the exporter uses the default built-in config + if cVolumeSourceName := api.Spec.Exporter.ExporterConfig.Configmap.Name; cVolumeSourceName != "" { + + cVolumeSourceKey := api.Spec.Exporter.ExporterConfig.Configmap.Key + cMSource := &corev1.ConfigMapVolumeSource{ + LocalObjectReference: corev1.LocalObjectReference{ + Name: cVolumeSourceName, + }, + Items: []corev1.KeyToPath{{ + Key: cVolumeSourceKey, + Path: DefaultExporterConfigmapFilename, + }}, + } + + vol = append(vol, corev1.Volume{Name: DefaultConfigVolumeString, VolumeSource: corev1.VolumeSource{ConfigMap: cMSource}}) + } + + // creds Volume + // api.Spec.Database.DBWallet.SecretName optional + // if null, consider the database NON-ADB and connect as such + if secretName := api.Spec.Database.DBWallet.SecretName; secretName != "" { + + vol = append(vol, corev1.Volume{ + Name: DefaultWalletVolumeString, + VolumeSource: corev1.VolumeSource{ + Secret: &corev1.SecretVolumeSource{ + SecretName: secretName, + }, + }, + }) + } + + // ocikey Volume + // api.Spec.Database.DBWallet.SecretName optional + if secretName := api.Spec.OCIConfig.SecretName; secretName != "" { + + OCIConfigSource := &corev1.SecretVolumeSource{ + SecretName: secretName, + Items: []corev1.KeyToPath{{ + Key: DefaultPrivateKeyFileKey, + Path: DefaultPrivateKeyFileName, + }}, + } + + vol = append(vol, corev1.Volume{ + Name: DefaultOCIPrivateKeyVolumeString, + VolumeSource: corev1.VolumeSource{Secret: OCIConfigSource}, + }) + } + return vol +} + +// GetExporterSelector function retrieves labels from api or provides default +func GetExporterSelector(api *apiv1.DatabaseObserver) map[string]string { + var s = make(map[string]string) + if labels := api.Spec.Prometheus.Labels; labels != nil && len(labels) > 0 { + for k, v := range labels { + s[k] = v + } + return s + + } + return map[string]string{DefaultLabelKey: DefaultLabelPrefix + api.Name} + +} + +// GetExporterEnvs function retrieves env from api or provides default +func GetExporterEnvs(api *apiv1.DatabaseObserver) []corev1.EnvVar { + + optional := true + rDBPasswordKey := api.Spec.Database.DBPassword.Key + rDBPasswordName := api.Spec.Database.DBPassword.SecretName + rDBConnectStrKey := api.Spec.Database.DBConnectionString.Key + rDBConnectStrName := api.Spec.Database.DBConnectionString.SecretName + rDBVaultSecretName := api.Spec.Database.DBPassword.VaultSecretName + rDBVaultOCID := api.Spec.Database.DBPassword.VaultOCID + rDBUserSKey := api.Spec.Database.DBUser.Key + rDBUserSName := api.Spec.Database.DBUser.SecretName + rOCIConfigCMName := api.Spec.OCIConfig.ConfigMapName + + var env = make([]corev1.EnvVar, 0) + + // DB_USERNAME environment variable + if rDBUserSKey == "" { // overwrite + rDBUserSKey = DefaultDbUserKey + } + envUser := &corev1.EnvVarSource{ + SecretKeyRef: &corev1.SecretKeySelector{ + Key: rDBUserSKey, + LocalObjectReference: corev1.LocalObjectReference{Name: rDBUserSName}, + Optional: &optional, + }} + env = append(env, corev1.EnvVar{Name: EnvVarDataSourceUser, ValueFrom: envUser}) + + // DB_CONNECT_STRING environment variable + if rDBConnectStrKey == "" { + rDBConnectStrKey = DefaultDBConnectionStringKey + } + envConnectStr := &corev1.EnvVarSource{ + SecretKeyRef: &corev1.SecretKeySelector{ + Key: rDBConnectStrKey, + LocalObjectReference: corev1.LocalObjectReference{Name: rDBConnectStrName}, + Optional: &optional, + }} + env = append(env, corev1.EnvVar{Name: EnvVarDataSourceConnectString, ValueFrom: envConnectStr}) + + // DB_PASSWORD environment variable + // if useVault, add environment variables for Vault ID and Vault Secret Name + useVault := rDBVaultSecretName != "" && rDBVaultOCID != "" + if useVault { + + env = append(env, corev1.EnvVar{Name: EnvVarDataSourcePwdVaultSecretName, Value: rDBVaultSecretName}) + env = append(env, corev1.EnvVar{Name: EnvVarDataSourcePwdVaultId, Value: rDBVaultOCID}) + + // Configuring the configProvider prefixed with vault_ + // https://github.com/oracle/oracle-db-appdev-monitoring/blob/main/vault/vault.go + configSourceFingerprintValue := &corev1.EnvVarSource{ + ConfigMapKeyRef: &corev1.ConfigMapKeySelector{ + Key: DefaultOCIConfigFingerprintKey, + LocalObjectReference: corev1.LocalObjectReference{Name: rOCIConfigCMName}, + Optional: &optional, + }, + } + configSourceRegionValue := &corev1.EnvVarSource{ + ConfigMapKeyRef: &corev1.ConfigMapKeySelector{ + Key: DefaultOCIConfigRegionKey, + LocalObjectReference: corev1.LocalObjectReference{Name: rOCIConfigCMName}, + Optional: &optional, + }, + } + configSourceTenancyValue := &corev1.EnvVarSource{ + ConfigMapKeyRef: &corev1.ConfigMapKeySelector{ + Key: DefaultOCIConfigTenancyKey, + LocalObjectReference: corev1.LocalObjectReference{Name: rOCIConfigCMName}, + Optional: &optional, + }, + } + configSourceUserValue := &corev1.EnvVarSource{ + ConfigMapKeyRef: &corev1.ConfigMapKeySelector{ + Key: DefaultOCIConfigUserKey, + LocalObjectReference: corev1.LocalObjectReference{Name: rOCIConfigCMName}, + Optional: &optional, + }, + } + + env = append(env, corev1.EnvVar{Name: EnvVarVaultFingerprint, ValueFrom: configSourceFingerprintValue}) + env = append(env, corev1.EnvVar{Name: EnvVarVaultUserOCID, ValueFrom: configSourceUserValue}) + env = append(env, corev1.EnvVar{Name: EnvVarVaultTenancyOCID, ValueFrom: configSourceTenancyValue}) + env = append(env, corev1.EnvVar{Name: EnvVarVaultRegion, ValueFrom: configSourceRegionValue}) + env = append(env, corev1.EnvVar{Name: EnvVarVaultPrivateKeyPath, Value: DefaultVaultPrivateKeyAbsolutePath}) + + } else { + + if rDBPasswordKey == "" { // overwrite + rDBPasswordKey = DefaultDBPasswordKey + } + dbPassword := &corev1.EnvVarSource{ + SecretKeyRef: &corev1.SecretKeySelector{ + Key: rDBPasswordKey, + LocalObjectReference: corev1.LocalObjectReference{Name: rDBPasswordName}, + Optional: &optional, + }} + + env = append(env, corev1.EnvVar{Name: EnvVarDataSourcePassword, ValueFrom: dbPassword}) + + } + + // CUSTOM_METRICS environment variable + if customMetricsName := api.Spec.Exporter.ExporterConfig.Configmap.Name; customMetricsName != "" { + customMetrics := DefaultExporterConfigmapAbsolutePath + env = append(env, corev1.EnvVar{Name: EnvVarCustomConfigmap, Value: customMetrics}) + } + + env = append(env, corev1.EnvVar{Name: EnvVarOracleHome, Value: DefaultOracleHome}) + env = append(env, corev1.EnvVar{Name: EnvVarTNSAdmin, Value: DefaultOracleTNSAdmin}) + return env +} + +// GetExporterReplicas function retrieves replicaCount from api or provides default +func GetExporterReplicas(api *apiv1.DatabaseObserver) int32 { + if rc := api.Spec.Replicas; rc != 0 { + return rc + } + return int32(DefaultReplicaCount) +} + +// GetExporterImage function retrieves image from api or provides default +func GetExporterImage(api *apiv1.DatabaseObserver) string { + if img := api.Spec.Exporter.ExporterImage; img != "" { + return img + } + return DefaultExporterImage + +} + +func IsUpdateRequiredForContainerImage(desired *appsv1.Deployment, found *appsv1.Deployment) bool { + foundImage := found.Spec.Template.Spec.Containers[0].Image + desiredImage := desired.Spec.Template.Spec.Containers[0].Image + + return foundImage != desiredImage +} + +func IsUpdateRequiredForEnvironmentVars(desired *appsv1.Deployment, found *appsv1.Deployment) bool { + var updateEnvsRequired bool + desiredEnvValues := make(map[string]string) + + foundEnvs := found.Spec.Template.Spec.Containers[0].Env + desiredEnvs := desired.Spec.Template.Spec.Containers[0].Env + if len(foundEnvs) != len(desiredEnvs) { + updateEnvsRequired = true + } else { + for _, v := range desiredEnvs { + + if v.Name == EnvVarDataSourceUser || + v.Name == EnvVarDataSourceConnectString || + v.Name == EnvVarDataSourcePassword { + + ref := *(*v.ValueFrom).SecretKeyRef + desiredEnvValues[v.Name] = ref.Key + "-" + ref.Name + + } else if v.Name == EnvVarVaultFingerprint || + v.Name == EnvVarVaultRegion || + v.Name == EnvVarVaultTenancyOCID || + v.Name == EnvVarVaultUserOCID { + + ref := *(*v.ValueFrom).ConfigMapKeyRef + desiredEnvValues[v.Name] = ref.Key + "-" + ref.Name + + } else if v.Name == EnvVarDataSourcePwdVaultId || + v.Name == EnvVarDataSourcePwdVaultSecretName || + v.Name == EnvVarCustomConfigmap { + + desiredEnvValues[v.Name] = v.Value + } + } + + for _, v := range foundEnvs { + var foundValue string + + if v.Name == EnvVarDataSourceUser || + v.Name == EnvVarDataSourceConnectString || + v.Name == EnvVarDataSourcePassword { + + ref := *(*v.ValueFrom).SecretKeyRef + foundValue = ref.Key + "-" + ref.Name + + } else if v.Name == EnvVarVaultFingerprint || + v.Name == EnvVarVaultRegion || + v.Name == EnvVarVaultTenancyOCID || + v.Name == EnvVarVaultUserOCID { + + ref := *(*v.ValueFrom).ConfigMapKeyRef + foundValue = ref.Key + "-" + ref.Name + + } else if v.Name == EnvVarDataSourcePwdVaultId || + v.Name == EnvVarDataSourcePwdVaultSecretName || + v.Name == EnvVarCustomConfigmap { + + foundValue = v.Value + } + + if desiredEnvValues[v.Name] != foundValue { + updateEnvsRequired = true + } + } + } + return updateEnvsRequired +} + +func IsUpdateRequiredForVolumes(desired *appsv1.Deployment, found *appsv1.Deployment) bool { + var updateVolumesRequired bool + var foundConfigmap, desiredConfigmap string + var foundWalletSecret, desiredWalletSecret string + var foundOCIConfig, desiredOCIConfig string + + desiredVolumes := desired.Spec.Template.Spec.Volumes + foundVolumes := found.Spec.Template.Spec.Volumes + + if len(desiredVolumes) != len(foundVolumes) { + updateVolumesRequired = true + } else { + for _, v := range desiredVolumes { + if v.Name == DefaultConfigVolumeString { + desiredConfigmap = v.ConfigMap.Name + for _, key := range v.ConfigMap.Items { + desiredConfigmap += key.Key + } + } else if v.Name == DefaultWalletVolumeString { + desiredWalletSecret = v.VolumeSource.Secret.SecretName + + } else if v.Name == DefaultOCIPrivateKeyVolumeString { + desiredOCIConfig = v.VolumeSource.Secret.SecretName + } + } + + for _, v := range foundVolumes { + if v.Name == DefaultConfigVolumeString { + foundConfigmap = v.ConfigMap.Name + for _, key := range v.ConfigMap.Items { + foundConfigmap += key.Key + } + } else if v.Name == DefaultWalletVolumeString { + foundWalletSecret = v.VolumeSource.Secret.SecretName + + } else if v.Name == DefaultOCIPrivateKeyVolumeString { + foundOCIConfig = v.VolumeSource.Secret.SecretName + } + } + } + + return updateVolumesRequired || + desiredConfigmap != foundConfigmap || + desiredWalletSecret != foundWalletSecret || + desiredOCIConfig != foundOCIConfig +} + +func IsUpdateRequiredForReplicas(desired *appsv1.Deployment, found *appsv1.Deployment) bool { + foundReplicas := *found.Spec.Replicas + desiredReplicas := *desired.Spec.Replicas + + return desiredReplicas != foundReplicas +} diff --git a/commons/oci/database.go b/commons/oci/database.go index 10d6c8ff..9c3cd4d8 100644 --- a/commons/oci/database.go +++ b/commons/oci/database.go @@ -407,7 +407,9 @@ func (d *databaseService) ListAutonomousDatabaseBackups(adbOCID string) (databas func (d *databaseService) CreateAutonomousDatabaseBackup(adbBackup *dbv1alpha1.AutonomousDatabaseBackup, adbOCID string) (database.CreateAutonomousDatabaseBackupResponse, error) { createBackupRequest := database.CreateAutonomousDatabaseBackupRequest{ CreateAutonomousDatabaseBackupDetails: database.CreateAutonomousDatabaseBackupDetails{ - AutonomousDatabaseId: common.String(adbOCID), + AutonomousDatabaseId: common.String(adbOCID), + IsLongTermBackup: adbBackup.Spec.IsLongTermBackup, + RetentionPeriodInDays: adbBackup.Spec.RetentionPeriodInDays, }, } diff --git a/commons/oci/provider.go b/commons/oci/provider.go index e875969c..152f1efd 100644 --- a/commons/oci/provider.go +++ b/commons/oci/provider.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2022 Oracle and/or its affiliates. +** Copyright (c) 2022, 2024 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** @@ -40,6 +40,8 @@ package oci import ( "errors" + "fmt" + "os" "github.com/oracle/oci-go-sdk/v65/common" "github.com/oracle/oci-go-sdk/v65/common/auth" @@ -65,7 +67,9 @@ type APIKeyAuth struct { } func GetOCIProvider(kubeClient client.Client, authData APIKeyAuth) (common.ConfigurationProvider, error) { - if authData.ConfigMapName != nil && authData.SecretName != nil { + if authData.ConfigMapName != nil && authData.SecretName == nil { + return getWorkloadIdentityProvider(kubeClient, authData) + } else if authData.ConfigMapName != nil && authData.SecretName != nil { provider, err := getProviderWithAPIKey(kubeClient, authData) if err != nil { return nil, err @@ -80,6 +84,30 @@ func GetOCIProvider(kubeClient client.Client, authData APIKeyAuth) (common.Confi } } +func getWorkloadIdentityProvider(kubeClient client.Client, authData APIKeyAuth) (common.ConfigurationProvider, error) { + ociConfigMap, err := k8s.FetchConfigMap(kubeClient, authData.Namespace, *authData.ConfigMapName) + if err != nil { + return nil, err + } + // Ensure configmap is set with proper data + if len(ociConfigMap.Data) == 0 { + return nil, fmt.Errorf("OCI ConfigMap %s has no data", ociConfigMap.Name) + } + region, ok := ociConfigMap.Data[regionKey] + if !ok || len(region) == 0 { + return nil, fmt.Errorf("OCI Region Key %s missing from OCI ConfigMap %s", regionKey, ociConfigMap.Name) + } + // OCI SDK requires specific, dynamic environment variables for workload identity. + if err = os.Setenv(auth.ResourcePrincipalVersionEnvVar, auth.ResourcePrincipalVersion2_2); err != nil { + + return nil, fmt.Errorf("unable to set OCI SDK environment variable %s: %v", auth.ResourcePrincipalVersionEnvVar, err) + } + if err = os.Setenv(auth.ResourcePrincipalRegionEnvVar, region); err != nil { + return nil, fmt.Errorf("unable to set OCI SDK environment variable %s: %v", auth.ResourcePrincipalRegionEnvVar, err) + } + return auth.OkeWorkloadIdentityConfigurationProvider() +} + func getProviderWithAPIKey(kubeClient client.Client, authData APIKeyAuth) (common.ConfigurationProvider, error) { var region, fingerprint, user, tenancy, passphrase, privatekeyValue string diff --git a/commons/oci/wallet.go b/commons/oci/wallet.go index 27ebb150..a5f9235e 100644 --- a/commons/oci/wallet.go +++ b/commons/oci/wallet.go @@ -42,6 +42,7 @@ import ( "archive/zip" "io" "io/ioutil" + "strings" ) // ExtractWallet extracts the wallet and returns a map object which holds the byte values of the unzipped files. @@ -76,27 +77,34 @@ func saveWalletZip(content io.ReadCloser) (string, error) { } func unzipWallet(path string) (map[string][]byte, error) { - data := map[string][]byte{} + files := map[string][]byte{} reader, err := zip.OpenReader(path) if err != nil { - return data, err + return files, err } defer reader.Close() for _, file := range reader.File { reader, err := file.Open() if err != nil { - return data, err + return files, err } content, err := ioutil.ReadAll(reader) if err != nil { - return data, err + return files, err } - data[file.Name] = content + files[file.Name] = content } - return data, nil + return files, nil } + +func WalletExpiringDate(files map[string][]byte) string { + data := string(files["README"]) + + line := data[strings.Index(data, "this wallet will expire on"):strings.Index(data, ".\nIn order to avoid")] + return strings.TrimSpace(strings.TrimPrefix(line, "this wallet will expire on")) +} \ No newline at end of file diff --git a/commons/sharding/catalog.go b/commons/sharding/catalog.go index f87efea8..8a0019dd 100644 --- a/commons/sharding/catalog.go +++ b/commons/sharding/catalog.go @@ -141,10 +141,14 @@ func buildPodSpecForCatalog(instance *databasev1alpha1.ShardingDatabase, OraCata RunAsUser: &user, FSGroup: &group, }, - InitContainers: buildInitContainerSpecForCatalog(instance, OraCatalogSpex), - Containers: buildContainerSpecForCatalog(instance, OraCatalogSpex), - Volumes: buildVolumeSpecForCatalog(instance, OraCatalogSpex), + Containers: buildContainerSpecForCatalog(instance, OraCatalogSpex), + Volumes: buildVolumeSpecForCatalog(instance, OraCatalogSpex), } + + if (instance.Spec.IsDownloadScripts) && (instance.Spec.ScriptsLocation != "") { + spec.InitContainers = buildInitContainerSpecForCatalog(instance, OraCatalogSpex) + } + if len(instance.Spec.DbImagePullSecret) > 0 { spec.ImagePullSecrets = []corev1.LocalObjectReference{ { @@ -170,16 +174,10 @@ func buildVolumeSpecForCatalog(instance *databasev1alpha1.ShardingDatabase, OraC Name: OraCatalogSpex.Name + "secretmap-vol3", VolumeSource: corev1.VolumeSource{ Secret: &corev1.SecretVolumeSource{ - SecretName: instance.Spec.Secret, + SecretName: instance.Spec.DbSecret.Name, }, }, }, - { - Name: OraCatalogSpex.Name + "orascript-vol5", - VolumeSource: corev1.VolumeSource{ - EmptyDir: &corev1.EmptyDirVolumeSource{}, - }, - }, { Name: OraCatalogSpex.Name + "oradshm-vol6", VolumeSource: corev1.VolumeSource{ @@ -196,6 +194,10 @@ func buildVolumeSpecForCatalog(instance *databasev1alpha1.ShardingDatabase, OraC result = append(result, corev1.Volume{Name: OraCatalogSpex.Name + "orastage-vol7", VolumeSource: corev1.VolumeSource{PersistentVolumeClaim: &corev1.PersistentVolumeClaimVolumeSource{ClaimName: instance.Spec.StagePvcName}}}) } + if instance.Spec.IsDownloadScripts { + result = append(result, corev1.Volume{Name: OraCatalogSpex.Name + "orascript-vol5", VolumeSource: corev1.VolumeSource{EmptyDir: &corev1.EmptyDirVolumeSource{}}}) + } + return result } @@ -217,29 +219,49 @@ func buildContainerSpecForCatalog(instance *databasev1alpha1.ShardingDatabase, O VolumeMounts: buildVolumeMountSpecForCatalog(instance, OraCatalogSpex), LivenessProbe: &corev1.Probe{ // TODO: Investigate if it's ok to call status every 10 seconds - FailureThreshold: int32(30), - PeriodSeconds: int32(240), - InitialDelaySeconds: int32(300), - TimeoutSeconds: int32(60), + FailureThreshold: int32(3), + InitialDelaySeconds: int32(30), + PeriodSeconds: func() int32 { + if instance.Spec.LivenessCheckPeriod > 0 { + return int32(instance.Spec.LivenessCheckPeriod) + } + return 60 + }(), + TimeoutSeconds: int32(30), ProbeHandler: corev1.ProbeHandler{ Exec: &corev1.ExecAction{ - Command: getLivenessCmd("CATALOG"), + Command: []string{"/bin/sh", "-c", "if [ -f $ORACLE_BASE/checkDBLockStatus.sh ]; then $ORACLE_BASE/checkDBLockStatus.sh ; else $ORACLE_BASE/checkDBStatus.sh; fi "}, }, }, }, /** - // Disabling this because the pod is not reachable till the time startup probe completes and without network pod configuration cannot be completed. - StartupProbe: &corev1.Probe{ - // Initial delay should be big, because shard setup takes time - FailureThreshold: int32(30), - PeriodSeconds: int32(120), - Handler: corev1.Handler{ + ReadinessProbe: &corev1.Probe{ + ProbeHandler: corev1.ProbeHandler{ Exec: &corev1.ExecAction{ - Command: getLivenessCmd("CATALOG"), + //Command: getReadinessCmd("CATALOG"), + Command: []string{"/bin/sh", "-c", "if [ -f $ORACLE_BASE/checkDBLockStatus.sh ]; then $ORACLE_BASE/checkDBLockStatus.sh ; else $ORACLE_BASE/checkDBStatus.sh; fi "}, }, }, + InitialDelaySeconds: 20, + TimeoutSeconds: 20, + PeriodSeconds: func() int32 { + if instance.Spec.ReadinessCheckPeriod > 0 { + return int32(instance.Spec.ReadinessCheckPeriod) + } + return 60 + }(), }, **/ + StartupProbe: &corev1.Probe{ + FailureThreshold: int32(120), + PeriodSeconds: int32(40), + InitialDelaySeconds: int32(30), + ProbeHandler: corev1.ProbeHandler{ + Exec: &corev1.ExecAction{ + Command: []string{"/bin/sh", "-c", "if [ -f $ORACLE_BASE/checkDBLockStatus.sh ]; then $ORACLE_BASE/checkDBLockStatus.sh ; else $ORACLE_BASE/checkDBStatus.sh; fi "}, + }, + }, + }, Env: buildEnvVarsSpec(instance, OraCatalogSpex.EnvVars, OraCatalogSpex.Name, "CATALOG", false, ""), } if instance.Spec.IsClone { @@ -256,7 +278,7 @@ func buildContainerSpecForCatalog(instance *databasev1alpha1.ShardingDatabase, O return result } -//Function to build the init Container Spec +// Function to build the init Container Spec func buildInitContainerSpecForCatalog(instance *databasev1alpha1.ShardingDatabase, OraCatalogSpex databasev1alpha1.CatalogSpec) []corev1.Container { var result []corev1.Container // building the init Container Spec @@ -297,7 +319,9 @@ func buildVolumeMountSpecForCatalog(instance *databasev1alpha1.ShardingDatabase, var result []corev1.VolumeMount result = append(result, corev1.VolumeMount{Name: OraCatalogSpex.Name + "secretmap-vol3", MountPath: oraSecretMount, ReadOnly: true}) result = append(result, corev1.VolumeMount{Name: OraCatalogSpex.Name + "-oradata-vol4", MountPath: oraDataMount}) - result = append(result, corev1.VolumeMount{Name: OraCatalogSpex.Name + "orascript-vol5", MountPath: oraScriptMount}) + if instance.Spec.IsDownloadScripts { + result = append(result, corev1.VolumeMount{Name: OraCatalogSpex.Name + "orascript-vol5", MountPath: oraDbScriptMount}) + } result = append(result, corev1.VolumeMount{Name: OraCatalogSpex.Name + "oradshm-vol6", MountPath: oraShm}) if len(instance.Spec.StagePvcName) != 0 { @@ -328,7 +352,7 @@ func volumeClaimTemplatesForCatalog(instance *databasev1alpha1.ShardingDatabase, corev1.ReadWriteOnce, }, StorageClassName: &instance.Spec.StorageClass, - Resources: corev1.ResourceRequirements{ + Resources: corev1.VolumeResourceRequirements{ Requests: corev1.ResourceList{ corev1.ResourceStorage: resource.MustParse(strconv.FormatInt(int64(OraCatalogSpex.StorageSizeInGb), 10) + "Gi"), }, diff --git a/commons/sharding/exec.go b/commons/sharding/exec.go index 48fc18cf..c1921018 100644 --- a/commons/sharding/exec.go +++ b/commons/sharding/exec.go @@ -40,16 +40,24 @@ package commons import ( "bytes" + "fmt" "net/http" + "time" databasealphav1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" "github.com/go-logr/logr" corev1 "k8s.io/api/core/v1" + "k8s.io/apimachinery/pkg/runtime/schema" + "k8s.io/apimachinery/pkg/runtime/serializer" + "k8s.io/cli-runtime/pkg/genericclioptions" "k8s.io/client-go/kubernetes" "k8s.io/client-go/kubernetes/scheme" + "k8s.io/client-go/rest" "k8s.io/client-go/tools/clientcmd" "k8s.io/client-go/tools/remotecommand" + "k8s.io/kubectl/pkg/cmd/cp" + "k8s.io/kubectl/pkg/cmd/util" ) // ExecCMDInContainer execute command in first container of a pod @@ -106,3 +114,68 @@ func ExecCommand(podName string, cmd []string, kubeClient kubernetes.Interface, return execOut.String(), execErr.String(), nil } + +func GetPodCopyConfig(kubeClient kubernetes.Interface, kubeConfig clientcmd.ClientConfig, instance *databasealphav1.ShardingDatabase, logger logr.Logger) (*rest.Config, *kubernetes.Clientset, error) { + + var clientSet *kubernetes.Clientset + config, err := kubeConfig.ClientConfig() + if err != nil { + return config, clientSet, err + } + clientSet, err = kubernetes.NewForConfig(config) + config.APIPath = "/api" + config.GroupVersion = &schema.GroupVersion{Version: "v1"} + config.NegotiatedSerializer = serializer.WithoutConversionCodecFactory{CodecFactory: scheme.Codecs} + + return config, clientSet, err + +} + +func KctlCopyFile(kubeClient kubernetes.Interface, kubeConfig clientcmd.ClientConfig, instance *databasealphav1.ShardingDatabase, restConfig *rest.Config, kclientset *kubernetes.Clientset, logger logr.Logger, src string, dst string, containername string) (*bytes.Buffer, *bytes.Buffer, *bytes.Buffer, error) { + + var in, out, errOut *bytes.Buffer + var ioStreams genericclioptions.IOStreams + for count := 0; ; count++ { + ioStreams, in, out, errOut = genericclioptions.NewTestIOStreams() + copyOptions := cp.NewCopyOptions(ioStreams) + copyOptions.ClientConfig = restConfig + if len(containername) != 0 { + copyOptions.Container = containername + } + configFlags := genericclioptions.NewConfigFlags(false) + f := util.NewFactory(configFlags) + cmd := cp.NewCmdCp(f, ioStreams) + err := copyOptions.Complete(f, cmd, []string{src, dst}) + if err != nil { + return nil, nil, nil, err + } + + c := rest.CopyConfig(restConfig) + cs, err := kubernetes.NewForConfig(c) + if err != nil { + return nil, nil, nil, err + } + + copyOptions.ClientConfig = c + copyOptions.Clientset = cs + + err = copyOptions.Run() + if err != nil { + if !shouldRetry(count, err) { + return nil, nil, nil, fmt.Errorf("could not run copy operation: %v. Stdout: %v, Stderr: %v", err, out.String(), errOut.String()) + } + time.Sleep(10 * time.Second) + continue + } + break + } + return in, out, errOut, nil + +} + +func shouldRetry(count int, err error) bool { + if count < connectFailureMaxTries { + return err.Error() == errorDialingBackendEOF + } + return false +} diff --git a/commons/sharding/gsm.go b/commons/sharding/gsm.go index b6c1f0a8..6ac1414f 100644 --- a/commons/sharding/gsm.go +++ b/commons/sharding/gsm.go @@ -121,6 +121,7 @@ func buildStatefulSpecForGsm(instance *databasev1alpha1.ShardingDatabase, OraGsm }, VolumeClaimTemplates: volumeClaimTemplatesForGsm(instance, OraGsmSpex), } + /** if OraGsmSpex.Replicas == 0 { OraGsmSpex.Replicas = 1 sfsetspec.Replicas = &OraGsmSpex.Replicas @@ -128,6 +129,7 @@ func buildStatefulSpecForGsm(instance *databasev1alpha1.ShardingDatabase, OraGsm OraGsmSpex.Replicas = 1 sfsetspec.Replicas = &OraGsmSpex.Replicas } + **/ return sfsetspec } @@ -143,10 +145,14 @@ func buildPodSpecForGsm(instance *databasev1alpha1.ShardingDatabase, OraGsmSpex RunAsUser: &user, FSGroup: &group, }, - InitContainers: buildInitContainerSpecForGsm(instance, OraGsmSpex), - Containers: buildContainerSpecForGsm(instance, OraGsmSpex), - Volumes: buildVolumeSpecForGsm(instance, OraGsmSpex), + Containers: buildContainerSpecForGsm(instance, OraGsmSpex), + Volumes: buildVolumeSpecForGsm(instance, OraGsmSpex), + } + + if (instance.Spec.IsDownloadScripts) && (instance.Spec.ScriptsLocation != "") { + spec.InitContainers = buildInitContainerSpecForGsm(instance, OraGsmSpex) } + if len(instance.Spec.GsmImagePullSecret) > 0 { spec.ImagePullSecrets = []corev1.LocalObjectReference{ { @@ -171,16 +177,10 @@ func buildVolumeSpecForGsm(instance *databasev1alpha1.ShardingDatabase, OraGsmSp Name: OraGsmSpex.Name + "secretmap-vol3", VolumeSource: corev1.VolumeSource{ Secret: &corev1.SecretVolumeSource{ - SecretName: instance.Spec.Secret, + SecretName: instance.Spec.DbSecret.Name, }, }, }, - { - Name: OraGsmSpex.Name + "orascript-vol5", - VolumeSource: corev1.VolumeSource{ - EmptyDir: &corev1.EmptyDirVolumeSource{}, - }, - }, { Name: OraGsmSpex.Name + "oradshm-vol6", VolumeSource: corev1.VolumeSource{ @@ -197,6 +197,9 @@ func buildVolumeSpecForGsm(instance *databasev1alpha1.ShardingDatabase, OraGsmSp result = append(result, corev1.Volume{Name: OraGsmSpex.Name + "orastage-vol7", VolumeSource: corev1.VolumeSource{PersistentVolumeClaim: &corev1.PersistentVolumeClaimVolumeSource{ClaimName: instance.Spec.StagePvcName}}}) } + if instance.Spec.IsDownloadScripts { + result = append(result, corev1.Volume{Name: OraGsmSpex.Name + "orascript-vol5", VolumeSource: corev1.VolumeSource{EmptyDir: &corev1.EmptyDirVolumeSource{}}}) + } return result } @@ -230,10 +233,15 @@ func buildContainerSpecForGsm(instance *databasev1alpha1.ShardingDatabase, OraGs VolumeMounts: buildVolumeMountSpecForGsm(instance, OraGsmSpex), LivenessProbe: &corev1.Probe{ // TODO: Investigate if it's ok to call status every 10 seconds - FailureThreshold: int32(30), - PeriodSeconds: int32(240), - InitialDelaySeconds: int32(300), - TimeoutSeconds: int32(60), + FailureThreshold: int32(3), + InitialDelaySeconds: int32(30), + PeriodSeconds: func() int32 { + if instance.Spec.LivenessCheckPeriod > 0 { + return int32(instance.Spec.LivenessCheckPeriod) + } + return 60 + }(), + TimeoutSeconds: int32(20), ProbeHandler: corev1.ProbeHandler{ Exec: &corev1.ExecAction{ Command: getLivenessCmd("GSM"), @@ -263,7 +271,7 @@ func buildContainerSpecForGsm(instance *databasev1alpha1.ShardingDatabase, OraGs return result } -//Function to build the init Container Spec +// Function to build the init Container Spec func buildInitContainerSpecForGsm(instance *databasev1alpha1.ShardingDatabase, OraGsmSpex databasev1alpha1.GsmSpec) []corev1.Container { var result []corev1.Container // building the init Container Spec @@ -305,7 +313,9 @@ func buildVolumeMountSpecForGsm(instance *databasev1alpha1.ShardingDatabase, Ora var result []corev1.VolumeMount result = append(result, corev1.VolumeMount{Name: OraGsmSpex.Name + "secretmap-vol3", MountPath: oraSecretMount, ReadOnly: true}) result = append(result, corev1.VolumeMount{Name: OraGsmSpex.Name + "-oradata-vol4", MountPath: oraGsmDataMount}) - result = append(result, corev1.VolumeMount{Name: OraGsmSpex.Name + "orascript-vol5", MountPath: oraScriptMount}) + if instance.Spec.IsDownloadScripts { + result = append(result, corev1.VolumeMount{Name: OraGsmSpex.Name + "orascript-vol5", MountPath: oraScriptMount}) + } result = append(result, corev1.VolumeMount{Name: OraGsmSpex.Name + "oradshm-vol6", MountPath: oraShm}) if len(instance.Spec.StagePvcName) != 0 { @@ -336,7 +346,7 @@ func volumeClaimTemplatesForGsm(instance *databasev1alpha1.ShardingDatabase, Ora corev1.ReadWriteOnce, }, StorageClassName: &instance.Spec.StorageClass, - Resources: corev1.ResourceRequirements{ + Resources: corev1.VolumeResourceRequirements{ Requests: corev1.ResourceList{ corev1.ResourceStorage: resource.MustParse(strconv.FormatInt(int64(OraGsmSpex.StorageSizeInGb), 10) + "Gi"), }, @@ -443,7 +453,7 @@ func UpdateProvForGsm(instance *databasev1alpha1.ShardingDatabase, // Ensure deployment replicas match the desired state if sfSet.Spec.Replicas != nil { if *sfSet.Spec.Replicas != size { - msg = "Current StatefulSet replicas do not match configured Shard Replicas. Gsm is configured with only 1 but current replicas is set with " + strconv.FormatInt(int64(*sfSet.Spec.Replicas), 10) + msg = "Current StatefulSet replicas do not match configured GSM Replicas. Gsm is configured with only 1 but current replicas is set with " + strconv.FormatInt(int64(*sfSet.Spec.Replicas), 10) LogMessages("DEBUG", msg, nil, instance, logger) isUpdate = true } diff --git a/commons/sharding/provstatus.go b/commons/sharding/provstatus.go index 6c34b007..87796553 100644 --- a/commons/sharding/provstatus.go +++ b/commons/sharding/provstatus.go @@ -78,7 +78,7 @@ func UpdateGsmStatusData(instance *databasealphav1.ShardingDatabase, Specidx int K8sInternalSvcName := svcName + "." + getInstanceNs(instance) + ".svc.cluster.local" _, K8sInternalSvcIP, _ := GetSvcIp(instance.Spec.Gsm[Specidx].Name+"-0", K8sInternalSvcName, instance, kubeClient, kubeConfig, logger) _, K8sExternalSvcIP, _ := GetSvcIp(instance.Spec.Gsm[Specidx].Name+"-0", k8sExternalSvcName, instance, kubeClient, kubeConfig, logger) - DbPasswordSecret := instance.Spec.Secret + DbPasswordSecret := instance.Spec.DbSecret.Name instance.Status.Gsm.Services = GetGsmServices(instance.Spec.Gsm[Specidx].Name+"-0", instance, kubeClient, kubeConfig, logger) // externIp := strings.Replace(K8sInternalSvcIP, "/r/n", "", -1) @@ -125,7 +125,7 @@ func UpdateCatalogStatusData(instance *databasealphav1.ShardingDatabase, Specidx K8sInternalSvcName := svcName + "." + getInstanceNs(instance) + ".svc.cluster.local" _, K8sInternalSvcIP, _ := GetSvcIp(instance.Spec.Catalog[Specidx].Name+"-0", K8sInternalSvcName, instance, kubeClient, kubeConfig, logger) _, K8sExternalSvcIP, _ := GetSvcIp(instance.Spec.Catalog[Specidx].Name+"-0", k8sExternalSvcName, instance, kubeClient, kubeConfig, logger) - DbPasswordSecret := instance.Spec.Secret + DbPasswordSecret := instance.Spec.DbSecret.Name oracleSid := GetSidName(instance.Spec.Catalog[Specidx].EnvVars, instance.Spec.Catalog[Specidx].Name) oraclePdb := GetPdbName(instance.Spec.Catalog[Specidx].EnvVars, instance.Spec.Catalog[Specidx].Name) role := GetDbRole(instance.Spec.Catalog[Specidx].Name+"-0", instance, kubeClient, kubeConfig, logger) @@ -185,7 +185,7 @@ func UpdateShardStatusData(instance *databasealphav1.ShardingDatabase, Specidx i K8sInternalSvcName := svcName + "." + getInstanceNs(instance) + ".svc.cluster.local" _, K8sInternalSvcIP, _ := GetSvcIp(instance.Spec.Shard[Specidx].Name+"-0", K8sInternalSvcName, instance, kubeClient, kubeConfig, logger) _, K8sExternalSvcIP, _ := GetSvcIp(instance.Spec.Shard[Specidx].Name+"-0", k8sExternalSvcName, instance, kubeClient, kubeConfig, logger) - DbPasswordSecret := instance.Spec.Secret + DbPasswordSecret := instance.Spec.DbSecret.Name oracleSid := GetSidName(instance.Spec.Shard[Specidx].EnvVars, instance.Spec.Shard[Specidx].Name) oraclePdb := GetPdbName(instance.Spec.Shard[Specidx].EnvVars, instance.Spec.Shard[Specidx].Name) role := GetDbRole(instance.Spec.Shard[Specidx].Name+"-0", instance, kubeClient, kubeConfig, logger) @@ -347,7 +347,7 @@ func CheckGsmStatus(gname string, instance *databasealphav1.ShardingDatabase, ku return nil } -//============ Functiont o check the status of the Shard and catalog ========= +// ============ Functiont o check the status of the Shard and catalog ========= // ================================ Validate shard =========================== func ValidateDbSetup(podName string, instance *databasealphav1.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, ) error { diff --git a/commons/sharding/scommon.go b/commons/sharding/scommon.go index 5a25a78d..d5d438b1 100644 --- a/commons/sharding/scommon.go +++ b/commons/sharding/scommon.go @@ -41,13 +41,17 @@ package commons import ( "context" "fmt" + "slices" databasealphav1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" "regexp" "strconv" "strings" + "os" + "github.com/go-logr/logr" "github.com/oracle/oci-go-sdk/v65/common" "github.com/oracle/oci-go-sdk/v65/ons" @@ -77,6 +81,7 @@ const ( oraRunAsUser = int64(54321) oraFsGroup = int64(54321) oraScriptMount = "/opt/oracle/scripts/sharding/scripts" + oraDbScriptMount = "/opt/oracle/scripts/sharding" oraDataMount = "/opt/oracle/oradata" oraGsmDataMount = "/opt/oracle/gsmdata" oraConfigMapMount = "/mnt/config-map" @@ -91,6 +96,9 @@ const ( oraLocalOnsPort = 6123 oraAgentPort = 8080 ShardingDatabaseFinalizer = "Shardingdb.oracle.com" + TmpLoc = "/var/tmp" + connectFailureMaxTries = 5 + errorDialingBackendEOF = "error dialing backend: EOF" ) // Function to build the env var specification @@ -98,13 +106,10 @@ func buildEnvVarsSpec(instance *databasealphav1.ShardingDatabase, variables []da var result []corev1.EnvVar var varinfo string var sidFlag bool = false - var secretFlag bool = false - var pwdFileFLag bool = false - var pwdKeyFlag bool = false var pdbFlag bool = false var sDirectParam bool = false var sGroup1Params bool = false - var sGroup2Params bool = false + //var sGroup2Params bool = false var catalogParams bool = false var oldPdbFlag bool = false var oldSidFlag bool = false @@ -115,15 +120,6 @@ func buildEnvVarsSpec(instance *databasealphav1.ShardingDatabase, variables []da if variable.Name == "ORACLE_SID" { sidFlag = true } - if variable.Name == "SECRET_VOLUME" { - secretFlag = true - } - if variable.Name == "COMMON_OS_PWD_FILE" { - pwdFileFLag = true - } - if variable.Name == "PWD_KEY" { - pwdKeyFlag = true - } if variable.Name == "ORACLE_PDB" { pdbFlag = true } @@ -133,9 +129,6 @@ func buildEnvVarsSpec(instance *databasealphav1.ShardingDatabase, variables []da if variable.Name == "SHARD1_GROUP_PARAMS" { sGroup1Params = true } - if variable.Name == "SHARD2_GROUP_PARAMS" { - sGroup2Params = true - } if variable.Name == "CATALOG_PARAMS" { catalogParams = true } @@ -153,6 +146,7 @@ func buildEnvVarsSpec(instance *databasealphav1.ShardingDatabase, variables []da } result = append(result, corev1.EnvVar{Name: variable.Name, Value: variable.Value}) } + if !shardSetupFlag { if restype == "SHARD" { result = append(result, corev1.EnvVar{Name: "SHARD_SETUP", Value: "true"}) @@ -172,7 +166,6 @@ func buildEnvVarsSpec(instance *databasealphav1.ShardingDatabase, variables []da result = append(result, corev1.EnvVar{Name: "ENABLE_ARCHIVELOG", Value: "true"}) } } - if !sidFlag { if restype == "SHARD" { result = append(result, corev1.EnvVar{Name: "ORACLE_SID", Value: strings.ToUpper(name)}) @@ -189,29 +182,84 @@ func buildEnvVarsSpec(instance *databasealphav1.ShardingDatabase, variables []da result = append(result, corev1.EnvVar{Name: "ORACLE_PDB", Value: strings.ToUpper(name) + "PDB"}) } } - if !secretFlag { - result = append(result, corev1.EnvVar{Name: "SECRET_VOLUME", Value: "/mnt/secrets"}) + // Secret Settings + + if strings.ToLower(instance.Spec.DbSecret.EncryptionType) != "base64" { + result = append(result, corev1.EnvVar{Name: "PWD_KEY", Value: instance.Spec.DbSecret.KeyFileName}) + result = append(result, corev1.EnvVar{Name: "COMMON_OS_PWD_FILE", Value: instance.Spec.DbSecret.PwdFileName}) + } else { + result = append(result, corev1.EnvVar{Name: "PASSWORD_FILE", Value: instance.Spec.DbSecret.PwdFileName}) } - if !pwdFileFLag { - result = append(result, corev1.EnvVar{Name: "COMMON_OS_PWD_FILE", Value: "common_os_pwdfile.enc"}) + if len(instance.Spec.DbSecret.PwdFileMountLocation) != 0 { + result = append(result, corev1.EnvVar{Name: "SECRET_VOLUME", Value: instance.Spec.DbSecret.PwdFileMountLocation}) + } else { + result = append(result, corev1.EnvVar{Name: "SECRET_VOLUME", Value: oraSecretMount}) } - if !pwdKeyFlag { - result = append(result, corev1.EnvVar{Name: "PWD_KEY", Value: "pwd.key"}) + if len(instance.Spec.DbSecret.KeyFileMountLocation) != 0 { + result = append(result, corev1.EnvVar{Name: "KEY_SECRET_VOLUME", Value: instance.Spec.DbSecret.KeyFileMountLocation}) + } else { + result = append(result, corev1.EnvVar{Name: "KEY_SECRET_VOLUME", Value: oraSecretMount}) } + if restype == "GSM" { if !sDirectParam { - // varinfo = "director_name=sharddirector" + sDirectorCounter + ";director_region=primary;director_port=1521" + //varinfo = "director_name=sharddirector" + sDirectorCounter + ";director_region=primary;director_port=1521" varinfo = directorParams result = append(result, corev1.EnvVar{Name: "SHARD_DIRECTOR_PARAMS", Value: varinfo}) } - if !sGroup1Params { - varinfo = "group_name=shardgroup1;deploy_as=primary;group_region=primary" - result = append(result, corev1.EnvVar{Name: "SHARD1_GROUP_PARAMS", Value: varinfo}) + if strings.ToUpper(instance.Spec.ShardingType) != "USER" { + if !sGroup1Params { + if len(instance.Spec.GsmShardGroup) > 0 { + for i := 0; i < len(instance.Spec.GsmShardGroup); i++ { + if strings.ToUpper(instance.Spec.GsmShardGroup[i].DeployAs) == "PRIMARY" { + group_name := instance.Spec.GsmShardGroup[i].Name + //deploy_as := instance.Spec.ShardGroup[i].DeployAs + region := instance.Spec.GsmShardGroup[i].Region + varinfo = "group_name=" + group_name + ";" + "deploy_as=primary;" + "group_region=" + region + result = append(result, corev1.EnvVar{Name: "SHARD1_GROUP_PARAMS", Value: varinfo}) + } + if strings.ToUpper(instance.Spec.GsmShardGroup[i].DeployAs) == "STANDBY" { + group_name := instance.Spec.GsmShardGroup[i].Name + //deploy_as := instance.Spec.ShardGroup[i].DeployAs + region := instance.Spec.GsmShardGroup[i].Region + varinfo = "group_name=" + group_name + ";" + "deploy_as=standby;" + "group_region=" + region + result = append(result, corev1.EnvVar{Name: "SHARD2_GROUP_PARAMS", Value: varinfo}) + } + } + } + } else { + varinfo = "group_name=shardgroup1;deploy_as=primary;group_region=primary" + result = append(result, corev1.EnvVar{Name: "SHARD1_GROUP_PARAMS", Value: varinfo}) + } + } + + if strings.ToUpper(instance.Spec.ShardingType) == "USER" { + result = append(result, corev1.EnvVar{Name: "SHARDING_TYPE", Value: "USER"}) } - if instance.Spec.IsDataGuard { - if !sGroup2Params { - varinfo = "group_name=shardgroup2;deploy_as=standby;group_region=standby" - result = append(result, corev1.EnvVar{Name: "SHARD2_GROUP_PARAMS", Value: varinfo}) + // SERVICE Params setting + var svc string + if len(instance.Spec.GsmService) > 0 { + svc = "" + for i := 0; i < len(instance.Spec.GsmService); i++ { + svc = svc + "service_name=" + instance.Spec.GsmService[i].Name + ";" + if len(instance.Spec.GsmService[i].Role) != 0 { + svc = svc + "service_role=" + instance.Spec.GsmService[i].Role + } else { + svc = svc + "service_role=primary" + } + result = append(result, corev1.EnvVar{Name: "SERVICE" + fmt.Sprint(i) + "_PARAMS", Value: svc}) + svc = "" + } + } + + if strings.ToUpper(instance.Spec.GsmDevMode) != "FALSE" { + result = append(result, corev1.EnvVar{Name: "DEV_MODE", Value: "TRUE"}) + } + + if strings.ToUpper(instance.Spec.InvitedNodeSubnetFlag) != "FALSE" { + result = append(result, corev1.EnvVar{Name: "INVITED_NODE_SUBNET_FLAG", Value: "TRUE"}) + if instance.Spec.InvitedNodeSubnet != "" { + result = append(result, corev1.EnvVar{Name: "INVITED_NODE_SUBNET", Value: instance.Spec.InvitedNodeSubnet}) } } if !catalogParams { @@ -625,17 +673,59 @@ func buildCatalogParams(instance *databasealphav1.ShardingDatabase) string { var sidFlag bool = false var pdbFlag bool = false var portFlag bool = false - var regionFlag bool = false var cnameFlag bool = false var chunksFlag bool = false var sidName string var pdbName string var cport string - var cregion string var cname string var catchunks string + var catalog_region, shard_space string result = "catalog_host=" + instance.Spec.Catalog[0].Name + "-0" + "." + instance.Spec.Catalog[0].Name + ";" + + //Checking if replcia type set to native + var sspace_arr []string + if strings.ToUpper(instance.Spec.ShardingType) == "USER" { + shard_space = "" + result = result + "sharding_type=user;" + for i := 0; i < len(instance.Spec.Shard); i++ { + sspace_arr = append(sspace_arr, instance.Spec.Shard[i].ShardSpace) + } + slices.Sort(sspace_arr) + sspace_arr = slices.Compact(sspace_arr) //[a b c d] + for i := 0; i < len(sspace_arr); i++ { + shard_space = shard_space + sspace_arr[i] + "," + } + shard_space = strings.TrimSuffix(shard_space, ",") + result = result + "shard_space=" + shard_space + ";" + } else if strings.ToUpper(instance.Spec.ReplicationType) == "NATIVE" { + result = result + "repl_type=native;" + } else { + fmt.Fprintln(os.Stdout, []any{""}...) + } + + var region_arr []string + for i := 0; i < len(instance.Spec.Shard); i++ { + region_arr = append(region_arr, instance.Spec.Shard[i].ShardRegion) + } + + for i := 0; i < len(instance.Spec.Gsm); i++ { + region_arr = append(region_arr, instance.Spec.Gsm[i].Region) + } + + slices.Sort(region_arr) + region_arr = slices.Compact(region_arr) //[a b c d] + for i := 0; i < len(region_arr); i++ { + catalog_region = catalog_region + region_arr[i] + "," + } + catalog_region = strings.TrimSuffix(catalog_region, ",") + result = result + "catalog_region=" + catalog_region + ";" + + if len(instance.Spec.ShardConfigName) != 0 { + result = result + "shard_configname=" + instance.Spec.ShardConfigName + ";" + } + for _, variable := range variables { if variable.Name == "ORACLE_SID" { sidFlag = true @@ -649,10 +739,6 @@ func buildCatalogParams(instance *databasealphav1.ShardingDatabase) string { portFlag = true cport = variable.Value } - if variable.Name == "CATALOG_REGION" { - regionFlag = true - cregion = variable.Value - } if variable.Name == "CATALOG_NAME" { cnameFlag = true cname = variable.Value @@ -661,6 +747,7 @@ func buildCatalogParams(instance *databasealphav1.ShardingDatabase) string { chunksFlag = true catchunks = variable.Value } + } if !sidFlag { @@ -697,15 +784,7 @@ func buildCatalogParams(instance *databasealphav1.ShardingDatabase) string { if chunksFlag { result = result + "catalog_chunks=" + catchunks + ";" } - - if !regionFlag { - varinfo = "catalog_region=primary,standby" - result = result + varinfo - } else { - varinfo = "catalog_region=" + cregion - result = result + varinfo - } - + result = strings.TrimSuffix(result, ";") return result } @@ -731,31 +810,40 @@ func buildDirectorParams(instance *databasealphav1.ShardingDatabase, oraGsmSpex result = result + varinfo } - switch idx { - case 0: - varinfo = "director_region=primary;" + if oraGsmSpex.Region != "" { + varinfo = "director_region=" + oraGsmSpex.Region + ";" result = result + varinfo - case 1: - varinfo = "director_region=standby;" + } else { + switch idx { + case 0: + varinfo = "director_region=primary;" + result = result + varinfo + case 1: + varinfo = "director_region=standby;" + result = result + varinfo + default: + // Do nothing + } result = result + varinfo - default: - // Do nothing } if !dportFlag { varinfo = "director_port=1522" result = result + varinfo } - + result = strings.TrimSuffix(result, ";") return result } -func BuildShardParams(sfSet *appsv1.StatefulSet) string { +func BuildShardParams(instance *databasealphav1.ShardingDatabase, sfSet *appsv1.StatefulSet, OraShardSpex databasev1alpha1.ShardSpec) string { var variables []corev1.EnvVar = sfSet.Spec.Template.Spec.Containers[0].Env var result string var varinfo string var isShardPort bool = false - var isShardGrp bool = false + //var isShardGrp bool = false + //var i int32 + //var isShardSpace bool = false + //var isShardRegion bool = false result = "shard_host=" + sfSet.Name + "-0" + "." + sfSet.Name + ";" for _, variable := range variables { @@ -772,23 +860,32 @@ func BuildShardParams(sfSet *appsv1.StatefulSet) string { result = result + varinfo isShardPort = true } - if variable.Name == "SHARD_GROUP" { - varinfo = "shard_group=" + variable.Value + ";" - result = result + varinfo - isShardGrp = true - } + + } + if OraShardSpex.ShardGroup != "" { + varinfo = "shard_group=" + OraShardSpex.ShardGroup + ";" + result = result + varinfo } - if !isShardPort { - varinfo = "shard_port=" + "1521" + ";" + if OraShardSpex.ShardSpace != "" { + varinfo = "shard_space=" + OraShardSpex.ShardSpace + ";" + result = result + varinfo + } + if OraShardSpex.ShardRegion != "" { + varinfo = "shard_region=" + OraShardSpex.ShardRegion + ";" result = result + varinfo } - if !isShardGrp { - varinfo = "shard_group=" + "shardgroup1" + if OraShardSpex.DeployAs != "" { + varinfo = "deploy_as=" + OraShardSpex.DeployAs + ";" result = result + varinfo } + if !isShardPort { + varinfo = "shard_port=" + "1521" + ";" + result = result + varinfo + } + result = strings.TrimSuffix(result, ";") return result } @@ -858,7 +955,7 @@ func getNoChunksCmd(sparamStr string) []string { func shardValidationCmd() []string { - var oraShardValidateCmd = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", "--checkliveness=true ", "--optype=primaryshard"} + var oraShardValidateCmd = []string{oraDbScriptMount + "/cmdExec", "/bin/python", oraDbScriptMount + "/main.py ", "--checkliveness=true ", "--optype=primaryshard"} return oraShardValidateCmd } @@ -884,25 +981,47 @@ func getShardDelCmd(sparams string) []string { func getLivenessCmd(resType string) []string { var livenessCmd []string if resType == "SHARD" { - livenessCmd = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", "--checkliveness=true", "--optype=primaryshard"} + livenessCmd = []string{oraDbScriptMount + "/cmdExec", "/bin/python", oraDbScriptMount + "/main.py ", "--checkliveness=true", "--optype=primaryshard"} } if resType == "CATALOG" { - livenessCmd = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", "--checkliveness=true", "--optype=catalog"} + livenessCmd = []string{oraDbScriptMount + "/cmdExec", "/bin/python", oraDbScriptMount + "/main.py ", "--checkliveness=true", "--optype=catalog"} } if resType == "GSM" { livenessCmd = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", "--checkliveness=true", "--optype=gsm"} } if resType == "STANDBY" { - livenessCmd = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", "--checkliveness=true", "--optype=standbyshard"} + livenessCmd = []string{oraDbScriptMount + "/cmdExec", "/bin/python", oraDbScriptMount + "/main.py ", "--checkliveness=true", "--optype=standbyshard"} } return livenessCmd } +func getReadinessCmd(resType string) []string { + var readynessCmd []string + if resType == "SHARD" { + readynessCmd = []string{oraDbScriptMount + "/cmdExec", "/bin/python", oraDbScriptMount + "/main.py ", "--checkreadyness=true", "--optype=primaryshard"} + } + if resType == "CATALOG" { + readynessCmd = []string{oraDbScriptMount + "/cmdExec", "/bin/python", oraDbScriptMount + "/main.py ", "--checkreadyness=true", "--optype=catalog"} + } + if resType == "GSM" { + readynessCmd = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", "--checkreadyness=true", "--optype=gsm"} + } + if resType == "STANDBY" { + readynessCmd = []string{oraDbScriptMount + "/cmdExec", "/bin/python", oraDbScriptMount + "/main.py ", "--checkreadyness=true", "--optype=standbyshard"} + } + return readynessCmd +} + func getGsmShardValidateCmd(shardName string) []string { var validateCmd []string = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", "--validateshard=" + strconv.Quote(shardName), "--optype=gsm"} return validateCmd } +func GetTdeKeyLocCmd() []string { + var tdeKeyCmd []string = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", "--gettdekey=true", "--optype=gsm"} + return tdeKeyCmd +} + func getOnlineShardCmd(sparamStr string) []string { var onlineCmd []string = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", "--checkonlineshard=" + strconv.Quote(sparamStr), "--optype=gsm"} return onlineCmd @@ -927,9 +1046,9 @@ func getInitContainerCmd(resType string, name string, ) string { var initCmd string if resType == "WEB" { - initCmd = "chown -R 54321:54321 " + oraScriptMount + ";chmod 755 " + oraScriptMount + "/*;chown -R 54321:54321 /opt/oracle/oradata;chmod 750 /opt/oracle/oradata" + initCmd = "chown -R 54321:54321 " + oraDbScriptMount + ";chmod 755 " + oraDbScriptMount + "/*;chown -R 54321:54321 /opt/oracle/oradata;chmod 750 /opt/oracle/oradata" } else { - initCmd = resType + ";chown -R 54321:54321 " + oraScriptMount + ";chmod 755 " + oraScriptMount + "/*;chown -R 54321:54321 /opt/oracle/oradata;chmod 750 /opt/oracle/oradata" + initCmd = resType + ";chown -R 54321:54321 " + oraDbScriptMount + ";chmod 755 " + oraDbScriptMount + "/*;chown -R 54321:54321 /opt/oracle/oradata;chmod 750 /opt/oracle/oradata" } return initCmd } @@ -945,6 +1064,11 @@ func getGsmInitContainerCmd(resType string, name string, return initCmd } +func getResetPasswdCmd(sparamStr string) []string { + var resetPasswdCmd []string = []string{oraScriptMount + "/cmdExec", "/bin/python", oraScriptMount + "/main.py ", "--resetpassword=true"} + return resetPasswdCmd +} + func GetFmtStr(pstr string, ) string { return "[" + pstr + "]" @@ -1249,3 +1373,7 @@ func SendNotification(title string, body string, instance *databasealphav1.Shard LogMessages("DEBUG", msg, nil, instance, logger) } } + +func GetSecretMount() string { + return oraSecretMount +} diff --git a/commons/sharding/shard.go b/commons/sharding/shard.go index ecebf606..6dd7a6ea 100644 --- a/commons/sharding/shard.go +++ b/commons/sharding/shard.go @@ -141,10 +141,14 @@ func buildPodSpecForShard(instance *databasev1alpha1.ShardingDatabase, OraShardS RunAsUser: &user, FSGroup: &group, }, - InitContainers: buildInitContainerSpecForShard(instance, OraShardSpex), - Containers: buildContainerSpecForShard(instance, OraShardSpex), - Volumes: buildVolumeSpecForShard(instance, OraShardSpex), + Containers: buildContainerSpecForShard(instance, OraShardSpex), + Volumes: buildVolumeSpecForShard(instance, OraShardSpex), } + + if (instance.Spec.IsDownloadScripts) && (instance.Spec.ScriptsLocation != "") { + spec.InitContainers = buildInitContainerSpecForShard(instance, OraShardSpex) + } + if len(instance.Spec.DbImagePullSecret) > 0 { spec.ImagePullSecrets = []corev1.LocalObjectReference{ { @@ -171,16 +175,10 @@ func buildVolumeSpecForShard(instance *databasev1alpha1.ShardingDatabase, OraSha Name: OraShardSpex.Name + "secretmap-vol3", VolumeSource: corev1.VolumeSource{ Secret: &corev1.SecretVolumeSource{ - SecretName: instance.Spec.Secret, + SecretName: instance.Spec.DbSecret.Name, }, }, }, - { - Name: OraShardSpex.Name + "orascript-vol5", - VolumeSource: corev1.VolumeSource{ - EmptyDir: &corev1.EmptyDirVolumeSource{}, - }, - }, { Name: OraShardSpex.Name + "oradshm-vol6", VolumeSource: corev1.VolumeSource{ @@ -196,7 +194,9 @@ func buildVolumeSpecForShard(instance *databasev1alpha1.ShardingDatabase, OraSha if len(instance.Spec.StagePvcName) != 0 { result = append(result, corev1.Volume{Name: OraShardSpex.Name + "orastage-vol7", VolumeSource: corev1.VolumeSource{PersistentVolumeClaim: &corev1.PersistentVolumeClaimVolumeSource{ClaimName: instance.Spec.StagePvcName}}}) } - + if instance.Spec.IsDownloadScripts { + result = append(result, corev1.Volume{Name: OraShardSpex.Name + "orascript-vol5", VolumeSource: corev1.VolumeSource{EmptyDir: &corev1.EmptyDirVolumeSource{}}}) + } return result } @@ -218,28 +218,50 @@ func buildContainerSpecForShard(instance *databasev1alpha1.ShardingDatabase, Ora VolumeMounts: buildVolumeMountSpecForShard(instance, OraShardSpex), LivenessProbe: &corev1.Probe{ // TODO: Investigate if it's ok to call status every 10 seconds - FailureThreshold: int32(30), - PeriodSeconds: int32(240), - InitialDelaySeconds: int32(300), - TimeoutSeconds: int32(120), + FailureThreshold: int32(3), + InitialDelaySeconds: int32(30), + PeriodSeconds: func() int32 { + if instance.Spec.LivenessCheckPeriod > 0 { + return int32(instance.Spec.LivenessCheckPeriod) + } + return 60 + }(), + TimeoutSeconds: int32(30), ProbeHandler: corev1.ProbeHandler{ Exec: &corev1.ExecAction{ - Command: getLivenessCmd("SHARD"), + Command: []string{"/bin/sh", "-c", "if [ -f $ORACLE_BASE/checkDBLockStatus.sh ]; then $ORACLE_BASE/checkDBLockStatus.sh ; else $ORACLE_BASE/checkDBStatus.sh; fi "}, }, }, }, /** + ReadinessProbe: &corev1.Probe{ + ProbeHandler: corev1.ProbeHandler{ + Exec: &corev1.ExecAction{ + //Command: getReadinessCmd("SHARD"), + Command: []string{"/bin/sh", "-c", "if [ -f $ORACLE_BASE/checkDBLockStatus.sh ]; then $ORACLE_BASE/checkDBLockStatus.sh ; else $ORACLE_BASE/checkDBStatus.sh; fi "}, + }, + }, + InitialDelaySeconds: 20, + TimeoutSeconds: 20, + PeriodSeconds: func() int32 { + if instance.Spec.ReadinessCheckPeriod > 0 { + return int32(instance.Spec.ReadinessCheckPeriod) + } + return 60 + }(), + }, + **/ // Disabling this because ping stop working and sharding topologu never gets configured. StartupProbe: &corev1.Probe{ - FailureThreshold: int32(30), - PeriodSeconds: int32(180), - Handler: corev1.Handler{ + FailureThreshold: int32(30), + PeriodSeconds: int32(180), + InitialDelaySeconds: int32(30), + ProbeHandler: corev1.ProbeHandler{ Exec: &corev1.ExecAction{ - Command: getLivenessCmd("SHARD"), + Command: []string{"/bin/sh", "-c", "if [ -f $ORACLE_BASE/checkDBLockStatus.sh ]; then $ORACLE_BASE/checkDBLockStatus.sh ; else $ORACLE_BASE/checkDBStatus.sh; fi "}, }, }, }, - **/ Env: buildEnvVarsSpec(instance, OraShardSpex.EnvVars, OraShardSpex.Name, "SHARD", false, "NONE"), } @@ -257,10 +279,10 @@ func buildContainerSpecForShard(instance *databasev1alpha1.ShardingDatabase, Ora return result } -//Function to build the init Container Spec +// Function to build the init Container Spec func buildInitContainerSpecForShard(instance *databasev1alpha1.ShardingDatabase, OraShardSpex databasev1alpha1.ShardSpec) []corev1.Container { var result []corev1.Container - privFlag := true + privFlag := false var uid int64 = 0 // building the init Container Spec @@ -300,7 +322,9 @@ func buildVolumeMountSpecForShard(instance *databasev1alpha1.ShardingDatabase, O var result []corev1.VolumeMount result = append(result, corev1.VolumeMount{Name: OraShardSpex.Name + "secretmap-vol3", MountPath: oraSecretMount, ReadOnly: true}) result = append(result, corev1.VolumeMount{Name: OraShardSpex.Name + "-oradata-vol4", MountPath: oraDataMount}) - result = append(result, corev1.VolumeMount{Name: OraShardSpex.Name + "orascript-vol5", MountPath: oraScriptMount}) + if instance.Spec.IsDownloadScripts { + result = append(result, corev1.VolumeMount{Name: OraShardSpex.Name + "orascript-vol5", MountPath: oraDbScriptMount}) + } result = append(result, corev1.VolumeMount{Name: OraShardSpex.Name + "oradshm-vol6", MountPath: oraShm}) if len(instance.Spec.StagePvcName) != 0 { @@ -331,7 +355,7 @@ func volumeClaimTemplatesForShard(instance *databasev1alpha1.ShardingDatabase, O corev1.ReadWriteOnce, }, StorageClassName: &instance.Spec.StorageClass, - Resources: corev1.ResourceRequirements{ + Resources: corev1.VolumeResourceRequirements{ Requests: corev1.ResourceList{ corev1.ResourceStorage: resource.MustParse(strconv.FormatInt(int64(OraShardSpex.StorageSizeInGb), 10) + "Gi"), }, diff --git a/config/crd/bases/database.oracle.com_DbcsSystem.yaml b/config/crd/bases/database.oracle.com_DbcsSystem.yaml new file mode 100644 index 00000000..e933d5a4 --- /dev/null +++ b/config/crd/bases/database.oracle.com_DbcsSystem.yaml @@ -0,0 +1,240 @@ + +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.6.1 + creationTimestamp: null + name: DbcsSystem.database.oracle.com +spec: + group: database.oracle.com + names: + kind: DbcsSystem + listKind: DbcsSystemList + plural: DbcsSystem + singular: dbcssystem + scope: Namespaced + versions: + - name: v1alpha1 + schema: + openAPIV3Schema: + description: DbcsSystem is the Schema for the dbcssystems API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: DbcsSystemSpec defines the desired state of DbcsSystem + properties: + dbSystem: + properties: + availabilityDomain: + type: string + backupSubnetId: + type: string + clusterName: + type: string + compartmentId: + type: string + cpuCoreCount: + type: integer + dbAdminPaswordSecret: + type: string + dbBackupConfig: + description: DB Backup COnfig Network Struct + properties: + autoBackupEnabled: + type: boolean + autoBackupWindow: + type: string + backupDestinationDetails: + type: string + recoveryWindowsInDays: + type: integer + type: object + dbDomain: + type: string + dbEdition: + type: string + dbName: + type: string + dbUniqueName: + type: string + dbVersion: + type: string + dbWorkload: + type: string + diskRedundancy: + type: string + displayName: + type: string + domain: + type: string + faultDomains: + items: + type: string + type: array + hostName: + type: string + initialDataStorageSizeInGB: + type: integer + kmsKeyId: + type: string + kmsKeyVersionId: + type: string + licenseModel: + type: string + nodeCount: + type: integer + pdbName: + type: string + privateIp: + type: string + shape: + type: string + sshPublicKeys: + items: + type: string + type: array + storageManagement: + type: string + subnetId: + type: string + tags: + additionalProperties: + type: string + type: object + tdeWalletPasswordSecret: + type: string + timeZone: + type: string + required: + - availabilityDomain + - compartmentId + - dbAdminPaswordSecret + - hostName + - shape + - sshPublicKeys + - subnetId + type: object + hardLink: + type: boolean + id: + type: string + ociConfigMap: + type: string + ociSecret: + type: string + required: + - ociConfigMap + type: object + status: + description: DbcsSystemStatus defines the observed state of DbcsSystem + properties: + availabilityDomain: + type: string + cpuCoreCount: + type: integer + dataStoragePercentage: + type: integer + dataStorageSizeInGBs: + type: integer + dbEdition: + type: string + dbInfo: + items: + description: DbcsSystemStatus defines the observed state of DbcsSystem + properties: + dbHomeId: + type: string + dbName: + type: string + dbUniqueName: + type: string + dbWorkload: + type: string + id: + type: string + type: object + type: array + displayName: + type: string + id: + type: string + licenseModel: + type: string + network: + properties: + clientSubnet: + type: string + domainName: + type: string + hostName: + type: string + listenerPort: + type: integer + networkSG: + type: string + scanDnsName: + type: string + vcnName: + type: string + type: object + nodeCount: + type: integer + recoStorageSizeInGB: + type: integer + shape: + type: string + state: + type: string + storageManagement: + type: string + subnetId: + type: string + timeZone: + type: string + workRequests: + items: + properties: + operationId: + type: string + operationType: + type: string + percentComplete: + type: string + timeAccepted: + type: string + timeFinished: + type: string + timeStarted: + type: string + required: + - operationId + - operationType + type: object + type: array + required: + - state + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] diff --git a/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml b/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml index 0541b985..a5c37507 100644 --- a/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml +++ b/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml @@ -61,6 +61,8 @@ spec: type: string displayName: type: string + isLongTermBackup: + type: boolean ociConfig: description: "*********************** *\tOCI config ***********************" properties: @@ -69,6 +71,8 @@ spec: secretName: type: string type: object + retentionPeriodInDays: + type: integer target: description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster Important: Run "make" to regenerate code after modifying this file' diff --git a/config/crd/bases/database.oracle.com_autonomousdatabases.yaml b/config/crd/bases/database.oracle.com_autonomousdatabases.yaml index c70b9dd6..f77407f3 100644 --- a/config/crd/bases/database.oracle.com_autonomousdatabases.yaml +++ b/config/crd/bases/database.oracle.com_autonomousdatabases.yaml @@ -228,6 +228,79 @@ spec: - connectionStrings type: object type: array + conditions: + items: + description: "Condition contains details for one aspect of the current + state of this API Resource. --- This struct is intended for direct + use as an array at the field path .status.conditions. For example, + \n \ttype FooStatus struct{ \t // Represents the observations + of a foo's current state. \t // Known .status.conditions.type + are: \"Available\", \"Progressing\", and \"Degraded\" \t // + +patchMergeKey=type \t // +patchStrategy=merge \t // +listType=map + \t // +listMapKey=type \t Conditions []metav1.Condition + `json:\"conditions,omitempty\" patchStrategy:\"merge\" patchMergeKey:\"type\" + protobuf:\"bytes,1,rep,name=conditions\"` \n \t // other fields + \t}" + properties: + lastTransitionTime: + description: lastTransitionTime is the last time the condition + transitioned from one status to another. This should be when + the underlying condition changed. If that is not known, then + using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: message is a human readable message indicating + details about the transition. This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: observedGeneration represents the .metadata.generation + that the condition was set based upon. For instance, if .metadata.generation + is currently 12, but the .status.conditions[x].observedGeneration + is 9, the condition is out of date with respect to the current + state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: reason contains a programmatic identifier indicating + the reason for the condition's last transition. Producers + of specific condition types may define expected values and + meanings for this field, and whether the values are considered + a guaranteed API. The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + --- Many .condition.type values are consistent across resources + like Available, but because arbitrary conditions can be useful + (see .node.status.conditions), the ability to deconflict is + important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map lifecycleState: description: 'INSERT ADDITIONAL STATUS FIELD - define observed state of cluster Important: Run "make" to regenerate code after modifying @@ -235,6 +308,8 @@ spec: type: string timeCreated: type: string + walletExpiringDate: + type: string type: object type: object served: true diff --git a/config/crd/bases/database.oracle.com_dataguardbrokers.yaml b/config/crd/bases/database.oracle.com_dataguardbrokers.yaml index b5fc87c4..f19a3e22 100644 --- a/config/crd/bases/database.oracle.com_dataguardbrokers.yaml +++ b/config/crd/bases/database.oracle.com_dataguardbrokers.yaml @@ -88,6 +88,10 @@ spec: - MaxPerformance - MaxAvailability type: string + serviceAnnotations: + additionalProperties: + type: string + type: object setAsPrimaryDatabase: type: string standbyDatabaseRefs: diff --git a/config/crd/bases/database.oracle.com_pdbs.yaml b/config/crd/bases/database.oracle.com_pdbs.yaml index 1bc8d905..8e6cb94f 100644 --- a/config/crd/bases/database.oracle.com_pdbs.yaml +++ b/config/crd/bases/database.oracle.com_pdbs.yaml @@ -122,6 +122,9 @@ spec: cdbName: description: Name of the CDB type: string + cdbNamespace: + description: CDB Namespace + type: string cdbResName: description: Name of the CDB Custom Resource that runs the ORDS container type: string @@ -290,6 +293,41 @@ spec: storage. Even when set to true, totalSize and tempSize MUST be specified in the request if Action is Create. type: boolean + webServerPwd: + description: Password for the Web ServerPDB User + properties: + secret: + description: PDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + webServerUser: + description: Web Server User with SQL Administrator role to allow + us to authenticate to the PDB Lifecycle Management REST endpoints + properties: + secret: + description: PDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object xmlFileName: description: XML metadata filename to be used for Plug or Unplug operations type: string diff --git a/config/crd/bases/database.oracle.com_shardingdatabases.yaml b/config/crd/bases/database.oracle.com_shardingdatabases.yaml index 47432123..554ed506 100644 --- a/config/crd/bases/database.oracle.com_shardingdatabases.yaml +++ b/config/crd/bases/database.oracle.com_shardingdatabases.yaml @@ -16,7 +16,18 @@ spec: singular: shardingdatabase scope: Namespaced versions: - - name: v1alpha1 + - additionalPrinterColumns: + - jsonPath: .status.gsm.state + name: Gsm State + type: string + - jsonPath: .status.gsm.services + name: Services + type: string + - jsonPath: .status.gsm.shards + name: shards + priority: 1 + type: string + name: v1alpha1 schema: openAPIV3Schema: description: ShardingDatabase is the Schema for the shardingdatabases API @@ -36,6 +47,8 @@ spec: spec: description: ShardingDatabaseSpec defines the desired state of ShardingDatabase properties: + InvitedNodeSubnet: + type: string catalog: items: description: CatalogSpec defines the desired state of CatalogSpec @@ -82,6 +95,28 @@ spec: description: ResourceRequirements describes the compute resource requirements. properties: + claims: + description: "Claims lists the names of resources, defined + in spec.resourceClaims, that are used by this container. + \n This is an alpha field and requires enabling the DynamicResourceAllocation + feature gate. \n This field is immutable. It can only + be set for containers." + items: + description: ResourceClaim references one entry in PodSpec.ResourceClaims. + properties: + name: + description: Name must match the name of one entry + in pod.spec.resourceClaims of the Pod where this + field is used. It makes that resource available + inside a container. + type: string + required: + - name + type: object + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map limits: additionalProperties: anyOf: @@ -102,8 +137,8 @@ spec: description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, - otherwise to an implementation-defined value. More info: - https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + otherwise to an implementation-defined value. Requests + cannot exceed Limits. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' type: object type: object storageSizeInGb: @@ -117,11 +152,41 @@ spec: type: string dbImagePullSecret: type: string + dbSecret: + description: Secret Details + properties: + encryptionType: + type: string + keyFileMountLocation: + type: string + keyFileName: + type: string + keySecretName: + type: string + name: + type: string + nsConfigMap: + type: string + nsSecret: + type: string + pwdFileMountLocation: + type: string + pwdFileName: + type: string + required: + - name + - pwdFileName + type: object gsm: items: description: GsmSpec defines the desired state of GsmSpec properties: + directorName: + type: string envVars: + description: Replicas int32 `json:"replicas,omitempty"` // + Gsm Replicas. If you set OraGsmPvcName then it is set default + to 1. items: description: EnvironmentVariable represents a named variable accessible for containers. @@ -155,13 +220,34 @@ spec: type: object pvcName: type: string - replicas: - format: int32 - type: integer + region: + type: string resources: description: ResourceRequirements describes the compute resource requirements. properties: + claims: + description: "Claims lists the names of resources, defined + in spec.resourceClaims, that are used by this container. + \n This is an alpha field and requires enabling the DynamicResourceAllocation + feature gate. \n This field is immutable. It can only + be set for containers." + items: + description: ResourceClaim references one entry in PodSpec.ResourceClaims. + properties: + name: + description: Name must match the name of one entry + in pod.spec.resourceClaims of the Pod where this + field is used. It makes that resource available + inside a container. + type: string + required: + - name + type: object + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map limits: additionalProperties: anyOf: @@ -182,8 +268,8 @@ spec: description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, - otherwise to an implementation-defined value. More info: - https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + otherwise to an implementation-defined value. Requests + cannot exceed Limits. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' type: object type: object storageSizeInGb: @@ -193,10 +279,109 @@ spec: - name type: object type: array + gsmDevMode: + type: string gsmImage: type: string gsmImagePullSecret: type: string + gsmService: + items: + description: Service Definition + properties: + available: + type: string + clbGoal: + type: string + commitOutcome: + type: string + drainTimeout: + type: string + dtp: + type: string + edition: + type: string + failoverDelay: + type: string + failoverMethod: + type: string + failoverPrimary: + type: string + failoverRestore: + type: string + failoverRetry: + type: string + failoverType: + type: string + gdsPool: + type: string + lag: + type: integer + locality: + type: string + name: + type: string + notification: + type: string + pdbName: + type: string + policy: + type: string + preferred: + type: string + prferredAll: + type: string + regionFailover: + type: string + retention: + type: string + role: + type: string + sessionState: + type: string + sqlTransactionProfile: + type: string + stopOption: + type: string + tableFamily: + type: string + tfaPolicy: + type: string + required: + - name + type: object + type: array + gsmShardGroup: + items: + properties: + deployAs: + type: string + name: + type: string + region: + type: string + required: + - name + type: object + type: array + gsmShardSpace: + items: + description: ShardSpace Specs + properties: + chunks: + type: integer + name: + type: string + protectionMode: + type: string + shardGroup: + type: string + required: + - name + type: object + type: array + invitedNodeSubnetFlag: + type: string isClone: type: boolean isDataGuard: @@ -205,14 +390,16 @@ spec: type: boolean isDeleteOraPvc: type: boolean + isDownloadScripts: + type: boolean isExternalSvc: type: boolean + isTdeWallet: + type: boolean + liveinessCheckPeriod: + type: integer namespace: type: string - nsConfigMap: - type: string - nsSecret: - type: string portMappings: items: description: PortMapping is a specification of port mapping for @@ -233,9 +420,11 @@ spec: - targetPort type: object type: array - scriptsLocation: + readinessCheckPeriod: + type: integer + replicationType: type: string - secret: + scriptsLocation: type: string shard: description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster @@ -244,6 +433,8 @@ spec: description: ShardSpec is a specification of Shards for an application deployment. properties: + deployAs: + type: string envVars: items: description: EnvironmentVariable represents a named variable @@ -286,6 +477,28 @@ spec: description: ResourceRequirements describes the compute resource requirements. properties: + claims: + description: "Claims lists the names of resources, defined + in spec.resourceClaims, that are used by this container. + \n This is an alpha field and requires enabling the DynamicResourceAllocation + feature gate. \n This field is immutable. It can only + be set for containers." + items: + description: ResourceClaim references one entry in PodSpec.ResourceClaims. + properties: + name: + description: Name must match the name of one entry + in pod.spec.resourceClaims of the Pod where this + field is used. It makes that resource available + inside a container. + type: string + required: + - name + type: object + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map limits: additionalProperties: anyOf: @@ -306,10 +519,16 @@ spec: description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, - otherwise to an implementation-defined value. More info: - https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + otherwise to an implementation-defined value. Requests + cannot exceed Limits. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' type: object type: object + shardGroup: + type: string + shardRegion: + type: string + shardSpace: + type: string storageSizeInGb: format: int32 type: integer @@ -317,6 +536,16 @@ spec: - name type: object type: array + shardBuddyRegion: + type: string + shardConfigName: + type: string + shardRegion: + items: + type: string + type: array + shardingType: + type: string stagePvcName: type: string storageClass: @@ -326,7 +555,6 @@ spec: - dbImage - gsm - gsmImage - - secret - shard type: object status: diff --git a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml index 83f6b3cf..acfa2bea 100644 --- a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml +++ b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml @@ -89,10 +89,12 @@ spec: type: boolean charset: type: string - cloneFrom: + createAs: + enum: + - primary + - standby + - clone type: string - createAsStandby: - type: boolean dgBrokerConfigured: type: boolean edition: @@ -100,6 +102,7 @@ spec: - standard - enterprise - express + - free type: string enableTCPS: type: boolean @@ -153,14 +156,18 @@ spec: - ReadWriteOnce - ReadWriteMany type: string + datafilesVolumeName: + type: string + scriptsVolumeName: + type: string + setWritePermissions: + type: boolean size: type: string storageClass: type: string volumeClaimAnnotation: type: string - volumeName: - type: string type: object primaryDatabaseRef: type: string @@ -184,6 +191,8 @@ spec: type: string tcpsListenerPort: type: integer + tcpsTlsSecret: + type: string required: - image type: object @@ -203,8 +212,6 @@ spec: type: string clientWalletLoc: type: string - cloneFrom: - type: string clusterConnectString: type: string conditions: @@ -282,6 +289,8 @@ spec: x-kubernetes-list-type: map connectString: type: string + createdAs: + type: string datafilesCreated: default: "false" type: string @@ -336,14 +345,18 @@ spec: - ReadWriteOnce - ReadWriteMany type: string + datafilesVolumeName: + type: string + scriptsVolumeName: + type: string + setWritePermissions: + type: boolean size: type: string storageClass: type: string volumeClaimAnnotation: type: string - volumeName: - type: string type: object prebuiltDB: type: boolean @@ -367,9 +380,13 @@ spec: type: string tcpsPdbConnectString: type: string + tcpsTlsSecret: + default: "" + type: string required: - isTcpsEnabled - persistence + - tcpsTlsSecret type: object type: object served: true diff --git a/config/crd/bases/observability.oracle.com_databaseobservers.yaml b/config/crd/bases/observability.oracle.com_databaseobservers.yaml new file mode 100644 index 00000000..b0801738 --- /dev/null +++ b/config/crd/bases/observability.oracle.com_databaseobservers.yaml @@ -0,0 +1,227 @@ + +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.6.1 + creationTimestamp: null + name: databaseobservers.observability.oracle.com +spec: + group: observability.oracle.com + names: + kind: DatabaseObserver + listKind: DatabaseObserverList + plural: databaseobservers + singular: databaseobserver + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .status.exporterConfig + name: ExporterConfig + type: string + - jsonPath: .status.status + name: Status + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + description: DatabaseObserver is the Schema for the databaseobservers API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: DatabaseObserverSpec defines the desired state of DatabaseObserver + properties: + database: + description: DatabaseObserverDatabase defines the database details + used for DatabaseObserver + properties: + dbConnectionString: + properties: + key: + type: string + secret: + type: string + type: object + dbPassword: + properties: + key: + type: string + secret: + type: string + vaultOCID: + type: string + vaultSecretName: + type: string + type: object + dbUser: + properties: + key: + type: string + secret: + type: string + type: object + dbWallet: + properties: + key: + type: string + secret: + type: string + type: object + type: object + exporter: + description: DatabaseObserverExporterConfig defines the configuration + details related to the exporters of DatabaseObserver + properties: + configuration: + properties: + configmap: + description: ConfigMapDetails defines the configmap name + properties: + configmapName: + type: string + key: + type: string + type: object + type: object + image: + type: string + service: + description: DatabaseObserverService defines the exporter service + component of DatabaseObserver + properties: + port: + format: int32 + type: integer + type: object + type: object + ociConfig: + properties: + configMapName: + type: string + secretName: + type: string + type: object + prometheus: + description: PrometheusConfig defines the generated resources for + Prometheus + properties: + labels: + additionalProperties: + type: string + type: object + port: + type: string + type: object + replicas: + format: int32 + type: integer + type: object + status: + description: DatabaseObserverStatus defines the observed state of DatabaseObserver + properties: + conditions: + description: 'INSERT ADDITIONAL STATUS FIELD - define observed state + of cluster Important: Run "make" to regenerate code after modifying + this file' + items: + description: "Condition contains details for one aspect of the current + state of this API Resource. --- This struct is intended for direct + use as an array at the field path .status.conditions. For example, + \n \ttype FooStatus struct{ \t // Represents the observations + of a foo's current state. \t // Known .status.conditions.type + are: \"Available\", \"Progressing\", and \"Degraded\" \t // + +patchMergeKey=type \t // +patchStrategy=merge \t // +listType=map + \t // +listMapKey=type \t Conditions []metav1.Condition + `json:\"conditions,omitempty\" patchStrategy:\"merge\" patchMergeKey:\"type\" + protobuf:\"bytes,1,rep,name=conditions\"` \n \t // other fields + \t}" + properties: + lastTransitionTime: + description: lastTransitionTime is the last time the condition + transitioned from one status to another. This should be when + the underlying condition changed. If that is not known, then + using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: message is a human readable message indicating + details about the transition. This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: observedGeneration represents the .metadata.generation + that the condition was set based upon. For instance, if .metadata.generation + is currently 12, but the .status.conditions[x].observedGeneration + is 9, the condition is out of date with respect to the current + state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: reason contains a programmatic identifier indicating + the reason for the condition's last transition. Producers + of specific condition types may define expected values and + meanings for this field, and whether the values are considered + a guaranteed API. The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + --- Many .condition.type values are consistent across resources + like Available, but because arbitrary conditions can be useful + (see .node.status.conditions), the ability to deconflict is + important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + exporterConfig: + type: string + replicas: + type: integer + status: + type: string + required: + - conditions + - exporterConfig + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] diff --git a/config/crd/kustomization.yaml b/config/crd/kustomization.yaml index c411e3ed..b9f3aa8c 100644 --- a/config/crd/kustomization.yaml +++ b/config/crd/kustomization.yaml @@ -17,6 +17,7 @@ resources: - bases/database.oracle.com_autonomouscontainerdatabases.yaml - bases/database.oracle.com_dbcssystems.yaml - bases/database.oracle.com_dataguardbrokers.yaml +- bases/observability.oracle.com_databaseobservers.yaml # +kubebuilder:scaffold:crdkustomizeresource patchesStrategicMerge: @@ -32,6 +33,7 @@ patchesStrategicMerge: #- patches/webhook_in_autonomouscontainerdatabases.yaml #- patches/webhook_in_dbcssystems.yaml #- patches/webhook_in_dataguardbrokers.yaml +#- patches/webhook_in_databaseobservers.yaml # +kubebuilder:scaffold:crdkustomizewebhookpatch # [CERTMANAGER] To enable webhook, uncomment all the sections with [CERTMANAGER] prefix. @@ -46,6 +48,7 @@ patchesStrategicMerge: #- patches/cainjection_in_autonomouscontainerdatabases.yaml #- patches/cainjection_in_dbcssystems.yaml #- patches/cainjection_in_dataguardbrokers.yaml +#- patches/cainjection_in_databaseobservers.yaml # +kubebuilder:scaffold:crdkustomizecainjectionpatch # the following config is for teaching kustomize how to do kustomization for CRDs. diff --git a/config/crd/patches/cainjenction_in_databaseobservers.yaml b/config/crd/patches/cainjenction_in_databaseobservers.yaml new file mode 100644 index 00000000..278ea7fa --- /dev/null +++ b/config/crd/patches/cainjenction_in_databaseobservers.yaml @@ -0,0 +1,8 @@ +# The following patch adds a directive for certmanager to inject CA into the CRD +# CRD conversion requires k8s 1.13 or later. +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + annotations: + cert-manager.io/inject-ca-from: $(CERTIFICATE_NAMESPACE)/$(CERTIFICATE_NAME) + name: databaseobservers.observability.oracle.com \ No newline at end of file diff --git a/config/crd/patches/webhook_in_databaseobservers.yaml b/config/crd/patches/webhook_in_databaseobservers.yaml new file mode 100644 index 00000000..e61411df --- /dev/null +++ b/config/crd/patches/webhook_in_databaseobservers.yaml @@ -0,0 +1,17 @@ +# The following patch enables conversion webhook for CRD +# CRD conversion requires k8s 1.13 or later. +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: databaseobservers.observability.oracle.com +spec: + conversion: + strategy: Webhook + webhookClientConfig: + # this is "\n" used as a placeholder, otherwise it will be rejected by the apiserver for being blank, + # but we're going to set it later using the cert-manager (or potentially a patch if not using cert-manager) + caBundle: Cg== + service: + namespace: system + name: webhook-service + path: /convert \ No newline at end of file diff --git a/config/manager/kustomization.yaml b/config/manager/kustomization.yaml index b8ec1f58..e7ed68ce 100644 --- a/config/manager/kustomization.yaml +++ b/config/manager/kustomization.yaml @@ -8,5 +8,5 @@ apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization images: - name: controller - newName: container-registry.oracle.com/database/operator - newTag: 0.1.0 + newName: phx.ocir.io/intsanjaysingh/db-repo/oracle/database + newTag: sharding-operator diff --git a/config/manager/manager.yaml b/config/manager/manager.yaml index 90df4158..54340faf 100644 --- a/config/manager/manager.yaml +++ b/config/manager/manager.yaml @@ -41,4 +41,7 @@ spec: requests: cpu: 400m memory: 400Mi + env: + - name : WATCH_NAMESPACE + value : "" terminationGracePeriodSeconds: 10 diff --git a/config/rbac/auth_proxy_role_binding.yaml b/config/rbac/auth_proxy_role_binding.yaml index 55665e32..6a702753 100644 --- a/config/rbac/auth_proxy_role_binding.yaml +++ b/config/rbac/auth_proxy_role_binding.yaml @@ -5,11 +5,11 @@ apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: - name: oracle-database-operator-proxy-rolebinding + name: proxy-rolebinding roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole - name: proxy-role + name: oracle-database-operator-proxy-role subjects: - kind: ServiceAccount name: default diff --git a/config/rbac/databaseobserver_editor_role.yaml b/config/rbac/databaseobserver_editor_role.yaml new file mode 100644 index 00000000..900c4b88 --- /dev/null +++ b/config/rbac/databaseobserver_editor_role.yaml @@ -0,0 +1,24 @@ +# permissions for end users to edit databaseobservers. +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: databaseobserver-editor-role +rules: + - apiGroups: + - observability.oracle.com + resources: + - databaseobservers + verbs: + - create + - delete + - get + - list + - patch + - update + - watch + - apiGroups: + - observability.oracle.com + resources: + - databaseobservers/status + verbs: + - get \ No newline at end of file diff --git a/config/rbac/databaseobserver_viewer_role.yaml b/config/rbac/databaseobserver_viewer_role.yaml new file mode 100644 index 00000000..ef447b21 --- /dev/null +++ b/config/rbac/databaseobserver_viewer_role.yaml @@ -0,0 +1,20 @@ +# permissions for end users to view databaseobservers. +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: databaseobserver-viewer-role +rules: + - apiGroups: + - observability.oracle.com + resources: + - databaseobservers + verbs: + - get + - list + - watch + - apiGroups: + - observability.oracle.com + resources: + - databaseobservers/status + verbs: + - get \ No newline at end of file diff --git a/config/rbac/role.yaml b/config/rbac/role.yaml index 89b630dd..2f33c915 100644 --- a/config/rbac/role.yaml +++ b/config/rbac/role.yaml @@ -54,8 +54,28 @@ rules: - apiGroups: - "" resources: + - deployments - events - - nodes + - pods + - services + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - "" + resources: + - events + verbs: + - create + - patch +- apiGroups: + - "" + resources: - persistentvolumeclaims - pods - pods/exec @@ -69,6 +89,14 @@ rules: - patch - update - watch +- apiGroups: + - "" + resources: + - persistentvolumes + verbs: + - get + - list + - watch - apiGroups: - '''''' resources: @@ -81,6 +109,27 @@ rules: - patch - update - watch +- apiGroups: + - apps + resources: + - configmaps + verbs: + - get + - list +- apiGroups: + - apps + resources: + - deployments + - pods + - services + verbs: + - create + - delete + - get + - list + - patch + - update + - watch - apiGroups: - apps resources: @@ -118,9 +167,9 @@ rules: - "" resources: - configmaps + - containers - events - namespaces - - nodes - persistentvolumeclaims - pods - pods/exec @@ -135,6 +184,26 @@ rules: - patch - update - watch +- apiGroups: + - "" + resources: + - configmaps + - containers + - events + - namespaces + - pods + - pods/exec + - pods/log + - secrets + - services + verbs: + - create + - delete + - get + - list + - patch + - update + - watch - apiGroups: - "" resources: @@ -428,3 +497,49 @@ rules: - get - patch - update +- apiGroups: + - monitoring.coreos.com + resources: + - servicemonitors + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - observability.oracle.com + resources: + - databaseobservers + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - observability.oracle.com + resources: + - databaseobservers/finalizers + verbs: + - update +- apiGroups: + - observability.oracle.com + resources: + - databaseobservers/status + verbs: + - get + - patch + - update +- apiGroups: + - storage.k8s.io + resources: + - storageclasses + verbs: + - get + - list + - watch diff --git a/config/rbac/role_binding.yaml b/config/rbac/role_binding.yaml index 1abe0ec7..f2ccb566 100644 --- a/config/rbac/role_binding.yaml +++ b/config/rbac/role_binding.yaml @@ -3,7 +3,7 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: rbac.authorization.k8s.io/v1 -kind: ClusterRoleBinding +kind: RoleBinding metadata: name: oracle-database-operator-manager-rolebinding roleRef: diff --git a/config/samples/adb/autonomousdatabase_backup.yaml b/config/samples/adb/autonomousdatabase_backup.yaml index ac3fb779..0099a347 100644 --- a/config/samples/adb/autonomousdatabase_backup.yaml +++ b/config/samples/adb/autonomousdatabase_backup.yaml @@ -7,7 +7,7 @@ kind: AutonomousDatabaseBackup metadata: name: autonomousdatabasebackup-sample spec: - # Before you can create manual backups, you must have an Object Storage bucket and your database must be configured to connect to it. This is a one-time operation. + # Before you can create on-demand backups, you must have an Object Storage bucket and your database must be configured to connect to it. This is a one-time operation. # See https://docs.oracle.com/en-us/iaas/Content/Database/Tasks/adbbackingup.htm#creatingbucket target: k8sADB: @@ -16,6 +16,8 @@ spec: # ociADB: # ocid: ocid1.autonomousdatabase... displayName: autonomousdatabasebackup-sample + isLongTermBackup: true + retentionPeriodInDays: 90 # minimum retention period is 90 days # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. ociConfig: diff --git a/config/samples/adb/autonomousdatabase_create.yaml b/config/samples/adb/autonomousdatabase_create.yaml index 3ecf966f..aa77a94c 100644 --- a/config/samples/adb/autonomousdatabase_create.yaml +++ b/config/samples/adb/autonomousdatabase_create.yaml @@ -1,5 +1,5 @@ # -# Copyright (c) 2022, Oracle and/or its affiliates. +# Copyright (c) 2022, 2024, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: database.oracle.com/v1alpha1 @@ -30,7 +30,7 @@ spec: # # Uncomment this block to configure the network access type with the RESTRICTED option. # # This option lets you restrict access by defining access control rules in an Access Control List (ACL). - # # By specifying an ACL, the database will be accessible from a whitelisted set of IP addresses, CIDR (Classless Inter-Domain Routing) blocks, or VCNs. + # # By specifying an ACL, the database will be accessible from a whitelisted set of IP addresses, CIDR (Classless Inter-Domain Routing) blocks, or VCNs. # # Use a semicolon (;) as a deliminator between the VCN-specific subnets or IPs. # accessType: RESTRICTED # accessControlList: @@ -42,7 +42,7 @@ spec: # isMTLSConnectionRequired: true # # Uncomment this block to configure the network access type with the PRIVATE option. - # # This option assigns a private endpoint, private IP, and hostname to your database. + # # This option assigns a private endpoint, private IP, and hostname to your database. # # Specifying this option allows traffic only from the VCN you specify. # # This allows you to define security rules, ingress/egress, at the Network Security Group (NSG) level and to control traffic to your Autonomous Database. # accessType: PRIVATE @@ -61,4 +61,5 @@ spec: # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. ociConfig: configMapName: oci-cred + # Comment out secretName if using OKE workload identity secretName: oci-privatekey \ No newline at end of file diff --git a/config/samples/adb/autonomousdatabase_update_network_access.yaml b/config/samples/adb/autonomousdatabase_update_network_access.yaml index 0762efcb..f0e98806 100644 --- a/config/samples/adb/autonomousdatabase_update_network_access.yaml +++ b/config/samples/adb/autonomousdatabase_update_network_access.yaml @@ -32,7 +32,7 @@ spec: # accessType: PRIVATE # privateEndpoint: # subnetOCID: ocid1.subnet... - # nsgOCIDs: + # nsgOCIDs: # Optional # - ocid1.networksecuritygroup... # # Uncomment this block to configure the network access of an dedicated Autonomous Database (ADB-D) with an access control list. diff --git a/config/samples/kustomization.yaml b/config/samples/kustomization.yaml index 0ad50fcf..3bf0eab5 100644 --- a/config/samples/kustomization.yaml +++ b/config/samples/kustomization.yaml @@ -16,5 +16,6 @@ resources: - sharding/shardingdatabase.yaml - sharding/sharding_v1alpha1_provshard.yaml - dbcs/database_v1alpha1_dbcssystem.yaml -- database_v1alpha1_dataguardbroker.yaml + - database_v1alpha1_dataguardbroker.yaml + - observability/databaseobserver.yaml # +kubebuilder:scaffold:manifestskustomizesamples diff --git a/config/samples/observability/databaseobserver.yaml b/config/samples/observability/databaseobserver.yaml new file mode 100644 index 00000000..b3140549 --- /dev/null +++ b/config/samples/observability/databaseobserver.yaml @@ -0,0 +1,44 @@ +# example +apiVersion: observability.oracle.com/v1alpha1 +kind: DatabaseObserver +metadata: + name: obs-sample +spec: + database: + dbUser: + key: "username" + secret: db-secret + + dbPassword: + key: "password" + secret: db-secret + + dbConnectionString: + key: "connection" + secret: db-secret + + dbWallet: + secret: instance-wallet + + exporter: + image: "container-registry.oracle.com/database/observability-exporter:latest" + configuration: + configmap: + key: "config.toml" + configmapName: "devcm-oradevdb-config" + + service: + port: 9161 + + prometheus: + port: metrics + labels: + app: app-sample-label + + replicas: 1 + + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey + + diff --git a/config/samples/observability/databaseobserver_custom_config.yaml b/config/samples/observability/databaseobserver_custom_config.yaml new file mode 100644 index 00000000..1e9fff47 --- /dev/null +++ b/config/samples/observability/databaseobserver_custom_config.yaml @@ -0,0 +1,28 @@ +# example +apiVersion: observability.oracle.com/v1alpha1 +kind: DatabaseObserver +metadata: + name: obs-sample + namespace: observer +spec: + database: + dbUser: + key: "username" + secret: db-secret + + dbPassword: + key: "password" + secret: db-secret + + dbConnectionString: + key: "connection" + secret: db-secret + + dbWallet: + secret: instance-wallet + + exporter: + configuration: + configmap: + key: "config.toml" + configmapName: "devcm-oradevdb-config" \ No newline at end of file diff --git a/config/samples/observability/databaseobserver_minimal.yaml b/config/samples/observability/databaseobserver_minimal.yaml new file mode 100644 index 00000000..2eeaf3ab --- /dev/null +++ b/config/samples/observability/databaseobserver_minimal.yaml @@ -0,0 +1,22 @@ +# example +apiVersion: observability.oracle.com/v1alpha1 +kind: DatabaseObserver +metadata: + name: obs-sample + namespace: observer +spec: + database: + dbUser: + key: "username" + secret: db-secret + + dbPassword: + key: "password" + secret: db-secret + + dbConnectionString: + key: "connection" + secret: db-secret + + dbWallet: + secret: instance-wallets \ No newline at end of file diff --git a/config/samples/observability/databaseobserver_vault.yaml b/config/samples/observability/databaseobserver_vault.yaml new file mode 100644 index 00000000..fa2e09d4 --- /dev/null +++ b/config/samples/observability/databaseobserver_vault.yaml @@ -0,0 +1,25 @@ +# example +apiVersion: observability.oracle.com/v1alpha1 +kind: DatabaseObserver +metadata: + name: obs-sample +spec: + database: + dbUser: + key: "username" + secret: db-secret + + dbPassword: + vaultSecretName: sample_secret + vaultOCID: ocid1.vault.oc1.. + + dbConnectionString: + key: "connection" + secret: db-secret + + dbWallet: + secret: instance-wallet + + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey \ No newline at end of file diff --git a/config/samples/observability/sample-dashboard.json b/config/samples/observability/sample-dashboard.json new file mode 100644 index 00000000..5b05b05c --- /dev/null +++ b/config/samples/observability/sample-dashboard.json @@ -0,0 +1,1414 @@ +{ + "__inputs": [ + { + "name": "Prometheus", + "label": "prometheus", + "description": "", + "type": "datasource", + "pluginId": "prometheus", + "pluginName": "Prometheus" + } + ], + "__requires": [ + { + "type": "grafana", + "id": "grafana", + "name": "Grafana", + "version": "4.5.1" + }, + { + "type": "panel", + "id": "graph", + "name": "Graph", + "version": "" + }, + { + "type": "datasource", + "id": "prometheus", + "name": "Prometheus", + "version": "1.0.0" + }, + { + "type": "panel", + "id": "singlestat", + "name": "Singlestat", + "version": "" + } + ], + "annotations": { + "list": [ + { + "builtIn": 1, + "datasource": { + "type": "grafana", + "uid": "-- Grafana --" + }, + "enable": true, + "hide": true, + "iconColor": "rgba(0, 211, 255, 1)", + "name": "Annotations & Alerts", + "type": "dashboard" + } + ] + }, + "editable": true, + "fiscalYearStartMonth": 0, + "graphTooltip": 0, + "id": 5, + "links": [], + "liveNow": false, + "panels": [ + { + "collapsed": false, + "gridPos": { + "h": 1, + "w": 24, + "x": 0, + "y": 0 + }, + "id": 3, + "panels": [], + "title": "Oracle Database Details", + "type": "row" + }, + { + "datasource": "Prometheus", + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "mappings": [ + { + "options": { + "0": { + "index": 1, + "text": "DEAD" + }, + "1": { + "index": 0, + "text": "ALIVE" + } + }, + "type": "value" + } + ], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 80 + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 6, + "w": 4, + "x": 0, + "y": 1 + }, + "id": 10, + "options": { + "colorMode": "value", + "graphMode": "area", + "justifyMode": "auto", + "orientation": "auto", + "reduceOptions": { + "calcs": [ + "lastNotNull" + ], + "fields": "", + "values": false + }, + "textMode": "auto" + }, + "pluginVersion": "9.5.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "PBFA97CFB590B2093" + }, + "editorMode": "builder", + "exemplar": false, + "expr": "oracledb_up", + "format": "table", + "instant": true, + "legendFormat": "__auto", + "range": false, + "refId": "A" + } + ], + "title": "Database Status", + "type": "stat" + }, + { + "datasource": "Prometheus", + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "decbytes" + }, + "overrides": [] + }, + "gridPos": { + "h": 6, + "w": 3, + "x": 4, + "y": 1 + }, + "id": 5, + "options": { + "colorMode": "value", + "graphMode": "area", + "justifyMode": "auto", + "orientation": "auto", + "reduceOptions": { + "calcs": [ + "lastNotNull" + ], + "fields": "", + "values": false + }, + "textMode": "auto" + }, + "pluginVersion": "9.5.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "PBFA97CFB590B2093" + }, + "editorMode": "builder", + "exemplar": false, + "expr": "oracledb_obaas_db_system_value{name=\"sga_max_size\"}", + "format": "table", + "instant": true, + "legendFormat": "__auto", + "range": false, + "refId": "A" + } + ], + "title": "SGA Max Size", + "type": "stat" + }, + { + "datasource": "Prometheus", + "description": "", + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "decbytes" + }, + "overrides": [] + }, + "gridPos": { + "h": 6, + "w": 3, + "x": 7, + "y": 1 + }, + "id": 6, + "options": { + "colorMode": "value", + "graphMode": "area", + "justifyMode": "auto", + "orientation": "auto", + "reduceOptions": { + "calcs": [ + "lastNotNull" + ], + "fields": "", + "values": false + }, + "textMode": "auto" + }, + "pluginVersion": "9.5.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "PBFA97CFB590B2093" + }, + "editorMode": "builder", + "exemplar": false, + "expr": "oracledb_obaas_db_system_value{name=\"pga_aggregate_limit\"}", + "format": "table", + "instant": true, + "legendFormat": "__auto", + "range": false, + "refId": "A" + } + ], + "title": "PGA Aggregate Limit", + "type": "stat" + }, + { + "datasource": "Prometheus", + "description": "", + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 80 + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 6, + "w": 5, + "x": 10, + "y": 1 + }, + "id": 11, + "options": { + "colorMode": "value", + "graphMode": "area", + "justifyMode": "auto", + "orientation": "auto", + "reduceOptions": { + "calcs": [ + "lastNotNull" + ], + "fields": "", + "values": false + }, + "textMode": "auto" + }, + "pluginVersion": "9.5.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "PBFA97CFB590B2093" + }, + "editorMode": "builder", + "expr": "oracledb_sessions_value{status=\"ACTIVE\"}", + "legendFormat": "__auto", + "range": true, + "refId": "A" + } + ], + "title": "Active Sessions", + "type": "stat" + }, + { + "datasource": "Prometheus", + "description": "", + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 80 + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 6, + "w": 3, + "x": 15, + "y": 1 + }, + "id": 12, + "options": { + "colorMode": "value", + "graphMode": "area", + "justifyMode": "auto", + "orientation": "auto", + "reduceOptions": { + "calcs": [ + "lastNotNull" + ], + "fields": "", + "values": false + }, + "textMode": "auto" + }, + "pluginVersion": "9.5.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "PBFA97CFB590B2093" + }, + "editorMode": "builder", + "expr": "oracledb_activity_user_commits", + "legendFormat": "__auto", + "range": true, + "refId": "A" + } + ], + "title": "User commits", + "type": "stat" + }, + { + "datasource": "Prometheus", + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "none" + }, + "overrides": [] + }, + "gridPos": { + "h": 6, + "w": 3, + "x": 18, + "y": 1 + }, + "id": 13, + "options": { + "colorMode": "value", + "graphMode": "area", + "justifyMode": "auto", + "orientation": "auto", + "reduceOptions": { + "calcs": [ + "lastNotNull" + ], + "fields": "", + "values": false + }, + "textMode": "auto" + }, + "pluginVersion": "9.5.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "PBFA97CFB590B2093" + }, + "editorMode": "builder", + "expr": "oracledb_activity_execute_count", + "legendFormat": "__auto", + "range": true, + "refId": "A" + } + ], + "title": "Execute count", + "type": "stat" + }, + { + "datasource": "Prometheus", + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "custom": { + "align": "auto", + "cellOptions": { + "type": "auto" + }, + "inspect": false + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 80 + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 3, + "w": 3, + "x": 21, + "y": 1 + }, + "id": 7, + "options": { + "cellHeight": "sm", + "footer": { + "countRows": false, + "fields": "", + "reducer": [ + "sum" + ], + "show": false + }, + "showHeader": false + }, + "pluginVersion": "9.5.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "PBFA97CFB590B2093" + }, + "editorMode": "code", + "exemplar": false, + "expr": "max(oracledb_obaas_db_platform_value) by (platform_name)", + "format": "table", + "instant": true, + "legendFormat": "{{platform_name}}", + "range": false, + "refId": "A" + } + ], + "title": "Database Platform", + "transformations": [ + { + "id": "organize", + "options": { + "excludeByName": { + "Time": true, + "Value": true + }, + "indexByName": {}, + "renameByName": {} + } + } + ], + "type": "table" + }, + { + "datasource": "Prometheus", + "description": "", + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 80 + } + ] + } + }, + "overrides": [] + }, + "gridPos": { + "h": 3, + "w": 3, + "x": 21, + "y": 4 + }, + "id": 2, + "options": { + "colorMode": "value", + "graphMode": "area", + "justifyMode": "auto", + "orientation": "auto", + "reduceOptions": { + "calcs": [ + "lastNotNull" + ], + "fields": "", + "values": false + }, + "textMode": "auto" + }, + "pluginVersion": "9.5.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "PBFA97CFB590B2093" + }, + "editorMode": "builder", + "exemplar": false, + "expr": "oracledb_obaas_db_system_value{name=\"cpu_count\"}", + "format": "table", + "instant": true, + "legendFormat": "__auto", + "range": false, + "refId": "A" + } + ], + "title": "CPU Count", + "type": "stat" + }, + { + "collapsed": false, + "gridPos": { + "h": 1, + "w": 24, + "x": 0, + "y": 7 + }, + "id": 8, + "panels": [], + "title": "Top SQL", + "type": "row" + }, + { + "datasource": "Prometheus", + "description": "", + "fieldConfig": { + "defaults": { + "color": { + "mode": "thresholds" + }, + "custom": { + "align": "auto", + "cellOptions": { + "type": "auto" + }, + "inspect": false + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 80 + } + ] + } + }, + "overrides": [ + { + "matcher": { + "id": "byName", + "options": "SQL ID" + }, + "properties": [ + { + "id": "custom.width", + "value": 226 + } + ] + }, + { + "matcher": { + "id": "byName", + "options": "SQL Text (extract)" + }, + "properties": [ + { + "id": "custom.width", + "value": 1311 + } + ] + } + ] + }, + "gridPos": { + "h": 9, + "w": 24, + "x": 0, + "y": 8 + }, + "id": 9, + "options": { + "cellHeight": "sm", + "footer": { + "countRows": false, + "fields": "", + "reducer": [ + "sum" + ], + "show": false + }, + "showHeader": true, + "sortBy": [] + }, + "pluginVersion": "9.5.2", + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "PBFA97CFB590B2093" + }, + "editorMode": "code", + "exemplar": false, + "expr": "max(oracledb_obaas_top_sql_elapsed) by (sql_id, sql_text)", + "format": "table", + "instant": true, + "legendFormat": "__auto", + "range": false, + "refId": "A" + } + ], + "title": "Top SQL by elapsed time running", + "transformations": [ + { + "id": "organize", + "options": { + "excludeByName": { + "Time": true + }, + "indexByName": {}, + "renameByName": { + "Value": "Elapsed Time", + "sql_id": "SQL ID", + "sql_text": "SQL Text (extract)" + } + } + }, + { + "id": "sortBy", + "options": { + "fields": {}, + "sort": [ + { + "desc": true, + "field": "Elapsed Time" + } + ] + } + } + ], + "type": "table" + }, + { + "collapsed": false, + "gridPos": { + "h": 1, + "w": 24, + "x": 0, + "y": 17 + }, + "id": 4, + "panels": [], + "title": "System Wait Classes", + "type": "row" + }, + { + "datasource": "Prometheus", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 19, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "µs" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 0, + "y": 18 + }, + "id": 14, + "options": { + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "bottom", + "showLegend": false + }, + "tooltip": { + "mode": "single", + "sort": "none" + } + }, + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "PBFA97CFB590B2093" + }, + "editorMode": "builder", + "expr": "oracledb_wait_time_concurrency", + "interval": "$interval", + "legendFormat": "", + "range": true, + "refId": "A" + } + ], + "title": "Wait time - Concurrency", + "type": "timeseries" + }, + { + "datasource": "Prometheus", + "description": "", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 20, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green", + "value": null + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "µs" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 12, + "y": 18 + }, + "id": 15, + "options": { + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "bottom", + "showLegend": false + }, + "tooltip": { + "mode": "single", + "sort": "none" + } + }, + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "PBFA97CFB590B2093" + }, + "editorMode": "builder", + "expr": "oracledb_wait_time_commit", + "interval": "$interval", + "legendFormat": "", + "range": true, + "refId": "A" + } + ], + "title": "Wait time - Commit", + "type": "timeseries" + }, + { + "datasource": "Prometheus", + "description": "", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 20, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "µs" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 0, + "y": 26 + }, + "id": 16, + "options": { + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "bottom", + "showLegend": false + }, + "tooltip": { + "mode": "single", + "sort": "none" + } + }, + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "PBFA97CFB590B2093" + }, + "editorMode": "builder", + "expr": "oracledb_wait_time_system_io", + "interval": "$interval", + "legendFormat": "", + "range": true, + "refId": "A" + } + ], + "title": "Wait time - System I/O", + "type": "timeseries" + }, + { + "datasource": "Prometheus", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 20, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "µs" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 12, + "y": 26 + }, + "id": 17, + "options": { + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "bottom", + "showLegend": false + }, + "tooltip": { + "mode": "single", + "sort": "none" + } + }, + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "PBFA97CFB590B2093" + }, + "editorMode": "builder", + "expr": "oracledb_wait_time_user_io", + "interval": "$interval", + "legendFormat": "", + "range": true, + "refId": "A" + } + ], + "title": "Wait time - User I/O", + "type": "timeseries" + }, + { + "datasource": "Prometheus", + "description": "", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 20, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "µs" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 0, + "y": 34 + }, + "id": 18, + "options": { + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "bottom", + "showLegend": false + }, + "tooltip": { + "mode": "single", + "sort": "none" + } + }, + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "PBFA97CFB590B2093" + }, + "editorMode": "builder", + "expr": "oracledb_wait_time_application", + "interval": "$interval", + "legendFormat": "", + "range": true, + "refId": "A" + } + ], + "title": "Wait time - Application", + "type": "timeseries" + }, + { + "datasource": "Prometheus", + "description": "", + "fieldConfig": { + "defaults": { + "color": { + "mode": "palette-classic" + }, + "custom": { + "axisCenteredZero": false, + "axisColorMode": "text", + "axisLabel": "", + "axisPlacement": "auto", + "barAlignment": 0, + "drawStyle": "line", + "fillOpacity": 20, + "gradientMode": "none", + "hideFrom": { + "legend": false, + "tooltip": false, + "viz": false + }, + "lineInterpolation": "linear", + "lineWidth": 1, + "pointSize": 5, + "scaleDistribution": { + "type": "linear" + }, + "showPoints": "auto", + "spanNulls": false, + "stacking": { + "group": "A", + "mode": "none" + }, + "thresholdsStyle": { + "mode": "off" + } + }, + "mappings": [], + "thresholds": { + "mode": "absolute", + "steps": [ + { + "color": "green" + }, + { + "color": "red", + "value": 80 + } + ] + }, + "unit": "µs" + }, + "overrides": [] + }, + "gridPos": { + "h": 8, + "w": 12, + "x": 12, + "y": 34 + }, + "id": 19, + "options": { + "legend": { + "calcs": [], + "displayMode": "list", + "placement": "bottom", + "showLegend": false + }, + "tooltip": { + "mode": "single", + "sort": "none" + } + }, + "targets": [ + { + "datasource": { + "type": "prometheus", + "uid": "PBFA97CFB590B2093" + }, + "editorMode": "builder", + "expr": "oracledb_wait_time_network", + "interval": "$interval", + "legendFormat": "", + "range": true, + "refId": "A" + } + ], + "title": "Wait time - Network", + "type": "timeseries" + } + ], + "refresh": "", + "schemaVersion": 38, + "style": "dark", + "tags": [], + "templating": { + "list": [ + { + "auto": true, + "auto_count": 200, + "auto_min": "10s", + "current": { + "selected": false, + "text": "auto", + "value": "$__auto_interval_interval" + }, + "hide": 0, + "label": "Interval", + "name": "interval", + "options": [ + { + "selected": true, + "text": "auto", + "value": "$__auto_interval_interval" + }, + { + "selected": false, + "text": "1m", + "value": "1m" + }, + { + "selected": false, + "text": "10m", + "value": "10m" + }, + { + "selected": false, + "text": "30m", + "value": "30m" + }, + { + "selected": false, + "text": "1h", + "value": "1h" + }, + { + "selected": false, + "text": "6h", + "value": "6h" + }, + { + "selected": false, + "text": "12h", + "value": "12h" + }, + { + "selected": false, + "text": "1d", + "value": "1d" + }, + { + "selected": false, + "text": "7d", + "value": "7d" + }, + { + "selected": false, + "text": "14d", + "value": "14d" + }, + { + "selected": false, + "text": "30d", + "value": "30d" + } + ], + "query": "1m,10m,30m,1h,6h,12h,1d,7d,14d,30d", + "refresh": 2, + "skipUrlSync": false, + "type": "interval" + } + ] + }, + "time": { + "from": "now-6h", + "to": "now" + }, + "timepicker": {}, + "timezone": "", + "title": "Oracle Dashboard", + "uid": "obaas_oracle_dashboard", + "version": 20, + "weekStart": "" +} \ No newline at end of file diff --git a/config/samples/observability/sample_config.toml b/config/samples/observability/sample_config.toml new file mode 100644 index 00000000..0989d769 --- /dev/null +++ b/config/samples/observability/sample_config.toml @@ -0,0 +1,29 @@ +[[metric]] +context = "obaas_db_system" +labels = [ "name" ] +metricsdesc = { value = "Database system resources metric" } +request = ''' +select name, value +from v$parameter +where name in ('cpu_count', 'sga_max_size', 'pga_aggregate_limit') +''' + +[[metric]] +context = "obaas_db_platform" +labels = [ "platform_name" ] +metricsdesc = { value = "Database platform" } +request = ''' +SELECT platform_name, 1 as value FROM v$database +''' + +[[metric]] +context = "obaas_top_sql" +labels = [ "sql_id", "sql_text" ] +metricsdesc = { elapsed = "SQL statement elapsed time running" } +request = ''' +select * from ( +select sql_id, elapsed_time / 1000000 as elapsed, SUBSTRB(REPLACE(sql_text,'',' '),1,55) as sql_text +from V$SQLSTATS +order by elapsed_time desc +) where ROWNUM <= 15 +''' diff --git a/config/samples/sidb/dataguardbroker.yaml b/config/samples/sidb/dataguardbroker.yaml index 26ab9d79..5425afb7 100644 --- a/config/samples/sidb/dataguardbroker.yaml +++ b/config/samples/sidb/dataguardbroker.yaml @@ -10,7 +10,7 @@ metadata: namespace: default spec: - ## Primary DB ref. This can be of kind SingleInstanceDatabase or CloneDB + ## Primary DB ref. This is of kind SingleInstanceDatabase primaryDatabaseRef: "sidb-sample" ## Standby DB pod CRD Metadata Name to add this DB to DG config diff --git a/config/samples/sidb/oraclerestdataservice.yaml b/config/samples/sidb/oraclerestdataservice.yaml index cef0276f..911a9b1e 100644 --- a/config/samples/sidb/oraclerestdataservice.yaml +++ b/config/samples/sidb/oraclerestdataservice.yaml @@ -41,10 +41,9 @@ spec: keepSecret: true ## ORDS image details - ## Build the ORDS image following instructions at - ## https://github.com/oracle/docker-images/tree/main/OracleRestDataServices + ## Supported ORDS image is container-registry.oracle.com/database/ords:21.4.2-gh image: - pullFrom: + pullFrom: container-registry.oracle.com/database/ords:21.4.2-gh pullSecrets: ## Dedicated persistent storage is optional. If not specified, ORDS will use persistent storage from .spec.databaseRef diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index 407223fa..3fcf7d97 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -11,36 +11,33 @@ spec: ## Use only alphanumeric characters for sid up to a maximum of 8 characters sid: ORCL1 - - ## Specify a source database ref to copy/clone from any SIDB in current K8s cluster instead of creating a fresh one - ## If cloning from an external containerized DB which could be either standalone or in any K8s cluster, - ## specify connect string as `:/` instead of source database ref - cloneFrom: "" - - ## Reference to a source primary database from which - ## Format: 1. Intra-cluster: you can give name of the primary database or the database connect string in `:/` format - ## 2. Inter-cluster: Database connect string in `:/` format - primaryDatabaseRef: "" - ## Enable this flag for creating Physical Standby Database - createAsStandby: false - - ## DB edition. N/A if cloning from a Source DB in current K8s cluster (if cloneFrom is set to a database ref) - ## Valid values for edition are enterprise, standard or express + ## DB edition. N/A for createAs clone or standby + ## Valid values for edition are enterprise, standard, express or free edition: enterprise + + ## Type of database. + ## Valid values for createAs are primary, clone or standby + ## Valid only for enterprise and standard editions + createAs: primary + + ## Reference to a source primary database. + ## Valid only for createAs clone or standby + ## The name of a source primary database resource from the same namespace + primaryDatabaseRef: "" ## Secret containing SIDB password mapped to secretKey. secretKey defaults to oracle_pwd - ## Should refer to adminPassword of Source DB if cloning from a Source DB (i.e if cloneFrom is set) + ## Should refer to adminPassword of Source DB if createAs is clone or standby ## This secret will be deleted after creation of the database unless keepSecret is set to true which is the default adminPassword: secretName: secretKey: keepSecret: true - ## DB character set. N/A if cloning from a Source DB (if cloneFrom is set) + ## DB character set. N/A for createAs clone or standby charset: AL32UTF8 - ## PDB name. N/A if cloning from a Source DB (if cloneFrom is set) + ## PDB name. N/A for createAs clone or standby pdbName: orclpdb1 ## Enable/Disable Flashback @@ -53,15 +50,20 @@ spec: forceLog: false ## Enable TCPS - enableTCPS: false + enableTCPS: - ## TCPS Certificate Renewal Interval: The time after which TCPS certificate will be renewed if TCPS connections are enabled. + ## User specified TLS-Cert Secret + ## The following specified TLS certs will be used instead of self-signed + tcpsTlsSecret: + + ## TCPS Certificate Renewal Interval: (Valid for Self-Signed Certificates) + ## The time after which TCPS certificate will be renewed if TCPS connections are enabled. ## tcpsCertRenewInterval can be in hours(h), minutes(m) and seconds(s); e.g. 4380h, 8760h etc. ## Maximum value is 8760h (1 year), Minimum value is 24h; Default value is 8760h (1 year) ## If this field is commented out/removed from the yaml, it will disable the auto-renewal feature for TCPS certificate tcpsCertRenewInterval: 8760h - ## NA if cloning from a SourceDB (cloneFrom is set) + ## N/A for createAs clone or standby ## Specify both sgaSize and pgaSize (in MB) or dont specify both ## Specify Non-Zero value to use ## You cannot change these initParams for Oracle Database Express (XE) edition @@ -82,16 +84,24 @@ spec: pullSecrets: prebuiltDB: false + + ## Database storage details ## size is the required minimum size of the persistent volume - ## storageClass is specified for automatic volume provisioning - ## accessMode can only accept one of ReadWriteOnce, ReadWriteMany - ## volumeName is optional. Specify for binding to a specific PV and set storageClass to an empty string to disable automatic volume provisioning + ## storageClass is specified for dynamic volume provisioning and datafilesVolumeName for static provisioning persistence: + ## if the storageClass supports volume expansion, patch the size attribute to expand the volume + ## Shrinking volumes is not allowed size: 100Gi - ## oci-bv applies to OCI block volumes. Use "standard" storageClass for dynamic provisioning in Minikube. Update as appropriate for other cloud service providers + ## set ownership/permissions for writing to datafiles volume. This is usually needed for NFS volumes. + setWritePermissions: true + ## oci-bv applies to OCI block volumes. Use "standard" storageClass for dynamic provisioning in Minikube. Update as appropriate for other cloud services storageClass: "oci-bv" + ## accessMode can only accept one of ReadWriteOnce, ReadWriteMany accessMode: "ReadWriteOnce" - volumeName: "" + ## datafilesVolumeName is optional. Specify for binding to a specific PV and set storageClass to an empty string to disable automatic volume provisioning + datafilesVolumeName: "" + ## Optionally specify a volume containing scripts in 'setup' and 'startup' folders to be executed during database setup and startup respectively. + scriptsVolumeName: "" ## Type of service . Applicable on cloud enviroments only ## if loadBalService : false, service type = "NodePort" else "LoadBalancer" diff --git a/config/samples/sidb/singleinstancedatabase_clone.yaml b/config/samples/sidb/singleinstancedatabase_clone.yaml index c0e1eace..f25484d9 100644 --- a/config/samples/sidb/singleinstancedatabase_clone.yaml +++ b/config/samples/sidb/singleinstancedatabase_clone.yaml @@ -13,9 +13,12 @@ spec: ## Use only alphanumeric characters for sid sid: ORCL2 - ## A source database ref to clone from. + ## The name of a source primary database resource to clone from the same namespace ## Make sure the source database has been created by applying singeinstancedatabase_create.yaml - cloneFrom: sidb-sample + primaryDatabaseRef: sidb-sample + + ## Intended type of database. + createAs: clone ## Should refer to SourceDB secret ## Secret containing SIDB password mapped to secretKey diff --git a/config/samples/sidb/singleinstancedatabase_free.yaml b/config/samples/sidb/singleinstancedatabase_free.yaml index 558bd8f4..6dd0aa39 100644 --- a/config/samples/sidb/singleinstancedatabase_free.yaml +++ b/config/samples/sidb/singleinstancedatabase_free.yaml @@ -35,5 +35,5 @@ spec: storageClass: "oci-bv" accessMode: "ReadWriteOnce" - ## Count of Database Pods. Should be 1 for express edition. + ## Count of Database Pods. Should be 1 for free edition. replicas: 1 \ No newline at end of file diff --git a/config/samples/sidb/singleinstancedatabase_standby.yaml b/config/samples/sidb/singleinstancedatabase_standby.yaml index e93cc5e4..644438b4 100644 --- a/config/samples/sidb/singleinstancedatabase_standby.yaml +++ b/config/samples/sidb/singleinstancedatabase_standby.yaml @@ -15,11 +15,11 @@ spec: ## Use only alphanumeric characters for sid sid: ORCLS - ## Reference to a source primary database name + ## The name of a source primary database resource from the same namespace primaryDatabaseRef: "sidb-sample" - ## Enable this flag for creating Physical Standby Database - createAsStandby: true + ## Intended type of database. + createAs: standby ## Secret containing SIDB password mapped to secretKey adminPassword: @@ -43,3 +43,5 @@ spec: ## if loadBalService : false, service type = "NodePort" else "LoadBalancer" loadBalancer: false + replicas: 1 + \ No newline at end of file diff --git a/config/samples/sidb/singleinstancedatabase_tcps.yaml b/config/samples/sidb/singleinstancedatabase_tcps.yaml index d11a0d2a..06389f96 100644 --- a/config/samples/sidb/singleinstancedatabase_tcps.yaml +++ b/config/samples/sidb/singleinstancedatabase_tcps.yaml @@ -34,7 +34,12 @@ spec: ## Enable TCPS enableTCPS: true - ## TCPS Certificate Renewal Interval: The time after which TCPS certificate will be renewed if TCPS connections are enabled. + ## User specified TLS-Cert Secret + ## The following specified TLS certs will be used instead of self-signed + tcpsTlsSecret: my-tls-secret + + ## TCPS Certificate Renewal Interval: (Valid for Self-Signed Certificates) + ## The time after which TCPS certificate will be renewed if TCPS connections are enabled. ## tcpsCertRenewInterval can be in hours(h), minutes(m) and seconds(s); e.g. 4380h, 8760h etc. ## Maximum value is 8760h (1 year), Minimum value is 24h; Default value is 8760h (1 year) ## If this field is commented out/removed from the yaml, it will disable the auto-renewal feature for TCPS certificate diff --git a/config/webhook/manifests.yaml b/config/webhook/manifests.yaml index 28f50b79..175abd47 100644 --- a/config/webhook/manifests.yaml +++ b/config/webhook/manifests.yaml @@ -151,6 +151,26 @@ webhooks: resources: - singleinstancedatabases sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: webhook-service + namespace: system + path: /mutate-observability-oracle-com-v1alpha1-databaseobserver + failurePolicy: Fail + name: mdatabaseobserver.kb.io + rules: + - apiGroups: + - observability.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + resources: + - databaseobservers + sideEffects: None --- apiVersion: admissionregistration.k8s.io/v1 @@ -345,3 +365,23 @@ webhooks: resources: - singleinstancedatabases sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: webhook-service + namespace: system + path: /validate-observability-oracle-com-v1alpha1-databaseobserver + failurePolicy: Fail + name: vdatabaseobserver.kb.io + rules: + - apiGroups: + - observability.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + resources: + - databaseobservers + sideEffects: None diff --git a/controllers/database/autonomousdatabase_controller.go b/controllers/database/autonomousdatabase_controller.go index 54ff3431..bf56bfe0 100644 --- a/controllers/database/autonomousdatabase_controller.go +++ b/controllers/database/autonomousdatabase_controller.go @@ -52,8 +52,9 @@ import ( "github.com/oracle/oci-go-sdk/v65/common" "github.com/oracle/oci-go-sdk/v65/database" - corev1 "k8s.io/api/core/v1" apiErrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/api/meta" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/runtime" "k8s.io/apimachinery/pkg/types" @@ -66,7 +67,6 @@ import ( "sigs.k8s.io/controller-runtime/pkg/handler" "sigs.k8s.io/controller-runtime/pkg/predicate" "sigs.k8s.io/controller-runtime/pkg/reconcile" - "sigs.k8s.io/controller-runtime/pkg/source" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" "github.com/oracle/oracle-database-operator/commons/annotations" @@ -92,33 +92,30 @@ func (r *AutonomousDatabaseReconciler) SetupWithManager(mgr ctrl.Manager) error return ctrl.NewControllerManagedBy(mgr). For(&dbv1alpha1.AutonomousDatabase{}). Watches( - &source.Kind{Type: &dbv1alpha1.AutonomousDatabaseBackup{}}, - handler.EnqueueRequestsFromMapFunc(r.enqueueMapFn()), + &dbv1alpha1.AutonomousDatabaseBackup{}, + handler.EnqueueRequestsFromMapFunc(r.enqueueMapFn), ). Watches( - &source.Kind{Type: &dbv1alpha1.AutonomousDatabaseRestore{}}, - handler.EnqueueRequestsFromMapFunc(r.enqueueMapFn()), + &dbv1alpha1.AutonomousDatabaseRestore{}, + handler.EnqueueRequestsFromMapFunc(r.enqueueMapFn), ). WithEventFilter(predicate.And(r.eventFilterPredicate(), r.watchPredicate())). WithOptions(controller.Options{MaxConcurrentReconciles: 50}). // ReconcileHandler is never invoked concurrently with the same object. Complete(r) } +func (r *AutonomousDatabaseReconciler) enqueueMapFn(ctx context.Context, o client.Object) []reconcile.Request { + reqs := make([]reconcile.Request, len(o.GetOwnerReferences())) -func (r *AutonomousDatabaseReconciler) enqueueMapFn() handler.MapFunc { - return func(o client.Object) []reconcile.Request { - reqs := make([]reconcile.Request, len(o.GetOwnerReferences())) - - for _, owner := range o.GetOwnerReferences() { - reqs = append(reqs, reconcile.Request{ - NamespacedName: types.NamespacedName{ - Name: owner.Name, - Namespace: o.GetNamespace(), - }, - }) - } - - return reqs + for _, owner := range o.GetOwnerReferences() { + reqs = append(reqs, reconcile.Request{ + NamespacedName: types.NamespacedName{ + Name: owner.Name, + Namespace: o.GetNamespace(), + }, + }) } + + return reqs } func (r *AutonomousDatabaseReconciler) watchPredicate() predicate.Predicate { @@ -165,9 +162,9 @@ func (r *AutonomousDatabaseReconciler) eventFilterPredicate() predicate.Predicat lastSucSpecChanged := oldLastSucSpec != desiredLastSucSpec if (!specChanged && statusChanged) || lastSucSpecChanged || - (controllerutil.ContainsFinalizer(oldADB, dbv1alpha1.ADBFinalizer) != controllerutil.ContainsFinalizer(desiredADB, dbv1alpha1.ADBFinalizer)) { + (controllerutil.ContainsFinalizer(oldADB, dbv1alpha1.ADB_FINALIZER) != controllerutil.ContainsFinalizer(desiredADB, dbv1alpha1.ADB_FINALIZER)) { // Don't enqueue in the folowing condition: - // 1. only status changes 2. lastSucSpec changes 3. ADBFinalizer changes + // 1. only status changes 2. lastSucSpec changes 3. ADB_FINALIZER changes return false } @@ -190,6 +187,7 @@ func (r *AutonomousDatabaseReconciler) eventFilterPredicate() predicate.Predicat // +kubebuilder:rbac:groups=database.oracle.com,resources=autonomouscontainerdatabases,verbs=get;list // +kubebuilder:rbac:groups=coordination.k8s.io,resources=leases,verbs=create;get;list;update // +kubebuilder:rbac:groups="",resources=configmaps;secrets,verbs=get;list;watch;create;update;patch;delete +// +kubebuilder:rbac:groups="",resources=events,verbs=create;patch // Reconcile is the funtion that the operator calls every time when the reconciliation loop is triggered. // It go to the beggining of the reconcile if an error is returned. We won't return a error if it is related @@ -279,18 +277,22 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R /****************************************************************** * Update the resource if the spec has been changed. - * Requeue if it's in an intermediate state. Update the status first - * , otherwise the modifiedADB will be overwritten by the object - * returned from the cluster. + * This will trigger another reconcile, so returns with an empty + * result. ******************************************************************/ if !reflect.DeepEqual(modifiedADB.Spec, desiredADB.Spec) { if err := r.KubeClient.Update(context.TODO(), modifiedADB); err != nil { - return r.manageError(logger.WithName("IsADBIntermediateState"), modifiedADB, err) + return r.manageError(logger.WithName("updateSpec"), modifiedADB, err) } return emptyResult, nil } + /****************************************************************** + * Update the status at the end of every reconcile. + ******************************************************************/ copiedADB := modifiedADB.DeepCopy() + + updateCondition(modifiedADB, nil) if err := r.KubeClient.Status().Update(context.TODO(), modifiedADB); err != nil { return r.manageError(logger.WithName("Status().Update"), modifiedADB, err) } @@ -309,7 +311,7 @@ func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.R var requeue bool = false if modifiedADB.GetDeletionTimestamp() != nil && - controllerutil.ContainsFinalizer(modifiedADB, dbv1alpha1.ADBFinalizer) && + controllerutil.ContainsFinalizer(modifiedADB, dbv1alpha1.ADB_FINALIZER) && modifiedADB.Status.LifecycleState == database.AutonomousDatabaseLifecycleStateTerminated { logger.Info("The ADB is TERMINATED. The CR is to be deleted but finalizer is not yet removed; reconcile queued") requeue = true @@ -351,20 +353,24 @@ func (r *AutonomousDatabaseReconciler) setupOCIClients(logger logr.Logger, adb * return nil } -func (r *AutonomousDatabaseReconciler) manageError(logger logr.Logger, adb *dbv1alpha1.AutonomousDatabase, issue error) (ctrl.Result, error) { +func (r *AutonomousDatabaseReconciler) manageError(logger logr.Logger, adb *dbv1alpha1.AutonomousDatabase, err error) (ctrl.Result, error) { l := logger.WithName("manageError") - // Has synced at least once - if adb.Status.LifecycleState != "" { - // Send event - r.Recorder.Event(adb, corev1.EventTypeWarning, "UpdateFailed", issue.Error()) + if adb.Status.LifecycleState == "" { + // First time entering reconcile + updateCondition(adb, err) - var finalIssue = issue + l.Error(err, "CreateFailed") + + return emptyResult, nil + } else { + // Has synced at least once + var finalError = err // Roll back ociADB := adb.DeepCopy() specChanged, err := r.getADB(l, ociADB) if err != nil { - finalIssue = k8s.CombineErrors(finalIssue, err) + finalError = k8s.CombineErrors(finalError, err) } // Will exit the Reconcile anyway after the manageError is called. @@ -372,25 +378,72 @@ func (r *AutonomousDatabaseReconciler) manageError(logger logr.Logger, adb *dbv1 // Clear the lifecycleState first to avoid the webhook error when update during an intermediate state adb.Status.LifecycleState = "" if err := r.KubeClient.Status().Update(context.TODO(), adb); err != nil { - finalIssue = k8s.CombineErrors(finalIssue, err) + finalError = k8s.CombineErrors(finalError, err) } adb.Spec = ociADB.Spec if err := r.KubeClient.Update(context.TODO(), adb); err != nil { - finalIssue = k8s.CombineErrors(finalIssue, err) + finalError = k8s.CombineErrors(finalError, err) } } - l.Error(finalIssue, "UpdateFailed") + updateCondition(adb, err) + + l.Error(finalError, "UpdateFailed") return emptyResult, nil + } +} + +const CONDITION_TYPE_COMPLETE = "Complete" +const CONDITION_REASON_COMPLETE = "ReconcileComplete" + +func updateCondition(adb *dbv1alpha1.AutonomousDatabase, err error) { + var condition metav1.Condition + + errMsg := func() string { + if err != nil { + return err.Error() + } + return "no reconcile errors" + }() + + // If error occurs, ReconcileComplete will be marked as true and the error message will still be listed + // If the ADB lifecycleState is intermediate, then ReconcileComplete will be marked as false + if err != nil { + condition = metav1.Condition{ + Type: CONDITION_TYPE_COMPLETE, + LastTransitionTime: metav1.Now(), + ObservedGeneration: adb.GetGeneration(), + Reason: CONDITION_REASON_COMPLETE, + Message: errMsg, + Status: metav1.ConditionTrue, + } + } else if dbv1alpha1.IsADBIntermediateState(adb.Status.LifecycleState) { + condition = metav1.Condition{ + Type: CONDITION_TYPE_COMPLETE, + LastTransitionTime: metav1.Now(), + ObservedGeneration: adb.GetGeneration(), + Reason: CONDITION_REASON_COMPLETE, + Message: errMsg, + Status: metav1.ConditionFalse, + } } else { - // Send event - r.Recorder.Event(adb, corev1.EventTypeWarning, "CreateFailed", issue.Error()) + condition = metav1.Condition{ + Type: CONDITION_TYPE_COMPLETE, + LastTransitionTime: metav1.Now(), + ObservedGeneration: adb.GetGeneration(), + Reason: CONDITION_REASON_COMPLETE, + Message: errMsg, + Status: metav1.ConditionTrue, + } + } - return emptyResult, issue + if len(adb.Status.Conditions) > 0 { + meta.RemoveStatusCondition(&adb.Status.Conditions, condition.Type) } + meta.SetStatusCondition(&adb.Status.Conditions, condition) } func (r *AutonomousDatabaseReconciler) validateOperation( @@ -398,7 +451,7 @@ func (r *AutonomousDatabaseReconciler) validateOperation( adb *dbv1alpha1.AutonomousDatabase, ociADB *dbv1alpha1.AutonomousDatabase) (exit bool, result ctrl.Result, err error) { - lastSpec, err := adb.GetLastSuccessfulSpec() + lastSucSpec, err := adb.GetLastSuccessfulSpec() if err != nil { return false, emptyResult, err } @@ -406,7 +459,7 @@ func (r *AutonomousDatabaseReconciler) validateOperation( l := logger.WithName("validateOperation") // If lastSucSpec is nil, then it's CREATE or BIND opertaion - if lastSpec == nil { + if lastSucSpec == nil { if adb.Spec.Details.AutonomousDatabaseOCID == nil { l.Info("Create operation") err := r.createADB(logger, adb) @@ -444,7 +497,7 @@ func (r *AutonomousDatabaseReconciler) validateOperation( // the user updates the spec (UPDATE operation), otherwise it's a SYNC operation. lastDifADB := adb.DeepCopy() - lastDetailsChanged, err := lastDifADB.RemoveUnchangedDetails(*lastSpec) + lastDetailsChanged, err := lastDifADB.RemoveUnchangedDetails(*lastSucSpec) if err != nil { return false, emptyResult, err } @@ -502,7 +555,7 @@ func (r *AutonomousDatabaseReconciler) validateCleanup(logger logr.Logger, adb * return false, nil } - if controllerutil.ContainsFinalizer(adb, dbv1alpha1.ADBFinalizer) { + if controllerutil.ContainsFinalizer(adb, dbv1alpha1.ADB_FINALIZER) { if adb.Status.LifecycleState == database.AutonomousDatabaseLifecycleStateTerminating { // Delete in progress, continue with the reconcile logic return false, nil @@ -512,7 +565,7 @@ func (r *AutonomousDatabaseReconciler) validateCleanup(logger logr.Logger, adb * // The adb has been deleted. Remove the finalizer and exit the reconcile. // Once all finalizers have been removed, the object will be deleted. l.Info("Resource is in TERMINATED state; remove the finalizer") - if err := k8s.RemoveFinalizerAndPatch(r.KubeClient, adb, dbv1alpha1.ADBFinalizer); err != nil { + if err := k8s.RemoveFinalizerAndPatch(r.KubeClient, adb, dbv1alpha1.ADB_FINALIZER); err != nil { return false, err } return true, nil @@ -521,7 +574,7 @@ func (r *AutonomousDatabaseReconciler) validateCleanup(logger logr.Logger, adb * if adb.Spec.Details.AutonomousDatabaseOCID == nil { l.Info("Missing AutonomousDatabaseOCID to terminate Autonomous Database; remove the finalizer anyway", "Name", adb.Name, "Namespace", adb.Namespace) // Remove finalizer anyway. - if err := k8s.RemoveFinalizerAndPatch(r.KubeClient, adb, dbv1alpha1.ADBFinalizer); err != nil { + if err := k8s.RemoveFinalizerAndPatch(r.KubeClient, adb, dbv1alpha1.ADB_FINALIZER); err != nil { return false, err } return true, nil @@ -553,18 +606,18 @@ func (r *AutonomousDatabaseReconciler) validateFinalizer(logger logr.Logger, adb // Delete is not schduled. Update the finalizer for this CR if hardLink is present var finalizerChanged = false if adb.Spec.HardLink != nil { - if *adb.Spec.HardLink && !controllerutil.ContainsFinalizer(adb, dbv1alpha1.ADBFinalizer) { + if *adb.Spec.HardLink && !controllerutil.ContainsFinalizer(adb, dbv1alpha1.ADB_FINALIZER) { l.Info("Finalizer added") - if err := k8s.AddFinalizerAndPatch(r.KubeClient, adb, dbv1alpha1.ADBFinalizer); err != nil { + if err := k8s.AddFinalizerAndPatch(r.KubeClient, adb, dbv1alpha1.ADB_FINALIZER); err != nil { return false, err } finalizerChanged = true - } else if !*adb.Spec.HardLink && controllerutil.ContainsFinalizer(adb, dbv1alpha1.ADBFinalizer) { + } else if !*adb.Spec.HardLink && controllerutil.ContainsFinalizer(adb, dbv1alpha1.ADB_FINALIZER) { l.Info("Finalizer removed") - if err := k8s.RemoveFinalizerAndPatch(r.KubeClient, adb, dbv1alpha1.ADBFinalizer); err != nil { + if err := k8s.RemoveFinalizerAndPatch(r.KubeClient, adb, dbv1alpha1.ADB_FINALIZER); err != nil { return false, err } @@ -953,6 +1006,7 @@ func (r *AutonomousDatabaseReconciler) validateDesiredLifecycleState( // c. to PRIVATE: // was PUBLIC: set subnetOCID and nsgOCIDs. Configure the IsMTLSConnectionRequired settings if it is set. // was RESTRICTED: set subnetOCID and nsgOCIDs. Configure the IsMTLSConnectionRequired settings if it is set. +// *Note: OCI requires nsgOCIDs to be an empty string rather than nil when we don't want the adb to be included in any network security group. // // Otherwise, if the network access type remains the same, apply the network configuration, and then set the IsMTLSConnectionRequired. // @@ -1167,6 +1221,12 @@ func (r *AutonomousDatabaseReconciler) validateNetworkAccess( l.Info("Sending request to OCI to configure network access options") + // When the network access type is set to PRIVATE, any nil type of nsgOCIDs needs to be set to an empty string, otherwise, OCI SDK returns a 400 error + if difADB.Spec.Details.NetworkAccess.AccessType == dbv1alpha1.NetworkAccessTypePrivate && + difADB.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs == nil { + difADB.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs = []string{} + } + resp, err := r.dbService.UpdateNetworkAccess(*adb.Spec.Details.AutonomousDatabaseOCID, difADB) if err != nil { return false, err @@ -1222,6 +1282,8 @@ func (r *AutonomousDatabaseReconciler) validateWallet(logger logr.Logger, adb *d return err } + adb.Status.WalletExpiringDate = oci.WalletExpiringDate(data) + label := map[string]string{"app": adb.GetName()} if err := k8s.CreateSecret(r.KubeClient, adb.Namespace, walletName, data, adb, label); err != nil { diff --git a/controllers/database/cdb_controller.go b/controllers/database/cdb_controller.go index 50778ccd..5e6c0aca 100644 --- a/controllers/database/cdb_controller.go +++ b/controllers/database/cdb_controller.go @@ -41,6 +41,7 @@ package controllers import ( "context" "errors" + //"fmt" "strconv" "strings" @@ -220,9 +221,11 @@ func (r *CDBReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.R return requeueN, nil } -/********************************************************** - * Create a ReplicaSet for pods based on the ORDS container - /********************************************************/ +/* +********************************************************* + - Create a ReplicaSet for pods based on the ORDS container + /******************************************************* +*/ func (r *CDBReconciler) createORDSInstances(ctx context.Context, req ctrl.Request, cdb *dbapi.CDB) error { log := r.Log.WithValues("createORDSInstances", req.NamespacedName) @@ -251,9 +254,11 @@ func (r *CDBReconciler) createORDSInstances(ctx context.Context, req ctrl.Reques return nil } -/************************************************* - * Validate ORDS Pod. Check if there are any errors - /************************************************/ +/* +************************************************ + - Validate ORDS Pod. Check if there are any errors + /*********************************************** +*/ func (r *CDBReconciler) validateORDSPods(ctx context.Context, req ctrl.Request, cdb *dbapi.CDB) error { log := r.Log.WithValues("validateORDSPod", req.NamespacedName) @@ -277,7 +282,8 @@ func (r *CDBReconciler) validateORDSPods(ctx context.Context, req ctrl.Request, return errors.New("Waiting for ORDS pods to start") } - getORDSStatus := " curl -sSkv -k -X GET https://localhost:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/stable/metadata-catalog/ || curl -sSkv -X GET http://localhost:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/stable/metadata-catalog/ " + /* /opt/oracle/ords/secrets/$TLSKEY /opt/oracle/ords/secrets/$TLSCRT */ + getORDSStatus := " curl --cert /opt/oracle/ords/secrets/tls.crt --key /opt/oracle/ords/secrets/tls.key -sSkv -k -X GET https://localhost:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/stable/metadata-catalog/ || curl --cert /opt/oracle/ords/secrets/tls.crt --key /opt/oracle/ords/secrets/tls.key -sSkv -X GET http://localhost:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/stable/metadata-catalog/ " readyPods := 0 for _, pod := range podList.Items { if pod.Status.Phase == corev1.PodRunning { @@ -311,9 +317,12 @@ func (r *CDBReconciler) validateORDSPods(ctx context.Context, req ctrl.Request, return nil } -/************************ - * Create Pod spec -/************************/ +/* +*********************** + - Create Pod spec + +/*********************** +*/ func (r *CDBReconciler) createPodSpec(cdb *dbapi.CDB) corev1.PodSpec { podSpec := corev1.PodSpec{ @@ -531,9 +540,12 @@ func (r *CDBReconciler) createPodSpec(cdb *dbapi.CDB) corev1.PodSpec { return podSpec } -/************************ - * Create ReplicaSet spec -/************************/ +/* +*********************** + - Create ReplicaSet spec + +/*********************** +*/ func (r *CDBReconciler) createReplicaSetSpec(cdb *dbapi.CDB) *appsv1.ReplicaSet { replicas := int32(cdb.Spec.Replicas) @@ -570,9 +582,11 @@ func (r *CDBReconciler) createReplicaSetSpec(cdb *dbapi.CDB) *appsv1.ReplicaSet return replicaSet } -/********************************************************** - * Evaluate change in Spec post creation and instantiation - /********************************************************/ +/* +********************************************************* + - Evaluate change in Spec post creation and instantiation + /******************************************************* +*/ func (r *CDBReconciler) deleteReplicaSet(ctx context.Context, req ctrl.Request, cdb *dbapi.CDB) error { log := r.Log.WithValues("deleteReplicaSet", req.NamespacedName) @@ -596,9 +610,11 @@ func (r *CDBReconciler) deleteReplicaSet(ctx context.Context, req ctrl.Request, return nil } -/********************************************************** - * Evaluate change in Spec post creation and instantiation - /********************************************************/ +/* +********************************************************* + - Evaluate change in Spec post creation and instantiation + /******************************************************* +*/ func (r *CDBReconciler) evaluateSpecChange(ctx context.Context, req ctrl.Request, cdb *dbapi.CDB) error { log := r.Log.WithValues("evaluateSpecChange", req.NamespacedName) @@ -673,9 +689,11 @@ func (r *CDBReconciler) evaluateSpecChange(ctx context.Context, req ctrl.Request return nil } -/************************************************* - * Create a Cluster Service for ORDS CDB Pod - /************************************************/ +/* +************************************************ + - Create a Cluster Service for ORDS CDB Pod + /*********************************************** +*/ func (r *CDBReconciler) createORDSSVC(ctx context.Context, req ctrl.Request, cdb *dbapi.CDB) error { log := r.Log.WithValues("createORDSSVC", req.NamespacedName) @@ -701,9 +719,11 @@ func (r *CDBReconciler) createORDSSVC(ctx context.Context, req ctrl.Request, cdb return nil } -/************************ - * Create Service spec - /************************/ +/* +*********************** + - Create Service spec + /*********************** +*/ func (r *CDBReconciler) createSvcSpec(cdb *dbapi.CDB) *corev1.Service { svc := &corev1.Service{ @@ -726,9 +746,11 @@ func (r *CDBReconciler) createSvcSpec(cdb *dbapi.CDB) *corev1.Service { return svc } -/************************************************* - * Check CDB deletion - /************************************************/ +/* +************************************************ + - Check CDB deletion + /*********************************************** +*/ func (r *CDBReconciler) manageCDBDeletion(ctx context.Context, req ctrl.Request, cdb *dbapi.CDB) error { log := r.Log.WithValues("manageCDBDeletion", req.NamespacedName) @@ -781,9 +803,12 @@ func (r *CDBReconciler) manageCDBDeletion(ctx context.Context, req ctrl.Request, return nil } -/************************************************* - * Delete CDB Resource -/************************************************/ +/* +************************************************ + - Delete CDB Resource + +/*********************************************** +*/ func (r *CDBReconciler) deleteCDBInstance(ctx context.Context, req ctrl.Request, cdb *dbapi.CDB) error { log := r.Log.WithValues("deleteCDBInstance", req.NamespacedName) @@ -824,9 +849,11 @@ func (r *CDBReconciler) deleteCDBInstance(ctx context.Context, req ctrl.Request, return nil } -/************************************************* - * Get Secret Key for a Secret Name - /************************************************/ +/* +************************************************ + - Get Secret Key for a Secret Name + /*********************************************** +*/ func (r *CDBReconciler) verifySecrets(ctx context.Context, req ctrl.Request, cdb *dbapi.CDB) error { log := r.Log.WithValues("verifySecrets", req.NamespacedName) @@ -855,9 +882,11 @@ func (r *CDBReconciler) verifySecrets(ctx context.Context, req ctrl.Request, cdb return nil } -/************************************************* - * Get Secret Key for a Secret Name - /************************************************/ +/* +************************************************ + - Get Secret Key for a Secret Name + /*********************************************** +*/ func (r *CDBReconciler) checkSecret(ctx context.Context, req ctrl.Request, cdb *dbapi.CDB, secretName string) error { log := r.Log.WithValues("checkSecret", req.NamespacedName) @@ -877,9 +906,11 @@ func (r *CDBReconciler) checkSecret(ctx context.Context, req ctrl.Request, cdb * return nil } -/************************************************* - * Delete Secrets - /************************************************/ +/* +************************************************ + - Delete Secrets + /*********************************************** +*/ func (r *CDBReconciler) deleteSecrets(ctx context.Context, req ctrl.Request, cdb *dbapi.CDB) { log := r.Log.WithValues("deleteSecrets", req.NamespacedName) @@ -935,9 +966,11 @@ func (r *CDBReconciler) deleteSecrets(ctx context.Context, req ctrl.Request, cdb } } -/************************************************************** - * SetupWithManager sets up the controller with the Manager. - /*************************************************************/ +/* +************************************************************* + - SetupWithManager sets up the controller with the Manager. + /************************************************************ +*/ func (r *CDBReconciler) SetupWithManager(mgr ctrl.Manager) error { return ctrl.NewControllerManagedBy(mgr). For(&dbapi.CDB{}). diff --git a/controllers/database/dataguardbroker_controller.go b/controllers/database/dataguardbroker_controller.go index 183a1445..9faaefd2 100644 --- a/controllers/database/dataguardbroker_controller.go +++ b/controllers/database/dataguardbroker_controller.go @@ -76,7 +76,8 @@ const dataguardBrokerFinalizer = "database.oracle.com/dataguardbrokerfinalizer" //+kubebuilder:rbac:groups=database.oracle.com,resources=dataguardbrokers,verbs=get;list;watch;create;update;patch;delete //+kubebuilder:rbac:groups=database.oracle.com,resources=dataguardbrokers/status,verbs=get;update;patch //+kubebuilder:rbac:groups=database.oracle.com,resources=dataguardbrokers/finalizers,verbs=update -//+kubebuilder:rbac:groups="",resources=pods;pods/log;pods/exec;persistentvolumeclaims;services;nodes;events,verbs=create;delete;get;list;patch;update;watch +//+kubebuilder:rbac:groups="",resources=pods;pods/log;pods/exec;persistentvolumeclaims;services,verbs=create;delete;get;list;patch;update;watch +//+kubebuilder:rbac:groups="",resources=events,verbs=create;patch // Reconcile is part of the main kubernetes reconciliation loop which aims to // move the current state of the cluster closer to the desired state. @@ -397,18 +398,18 @@ func (r *DataguardBrokerReconciler) setupDataguardBrokerConfiguration(m *dbapi.D log.Error(err, err.Error()) return requeueY } - _, ok := dbSet[standbyDatabase.Status.Sid] - if ok { - log.Info("A database with the same SID is already configured in the DG") - r.Recorder.Eventf(m, corev1.EventTypeWarning, "Spec Error", "A database with the same SID "+standbyDatabase.Status.Sid+" is already configured in the DG") - continue - } // Check if dataguard broker is already configured for the standby database if standbyDatabase.Status.DgBrokerConfigured { log.Info("Dataguard broker for standbyDatabase : " + standbyDatabase.Name + " is already configured") continue } + _, ok := dbSet[standbyDatabase.Status.Sid] + if ok { + log.Info("A database with the same SID is already configured in the DG") + r.Recorder.Eventf(m, corev1.EventTypeWarning, "Spec Error", "A database with the same SID "+standbyDatabase.Status.Sid+" is already configured in the DG") + continue + } m.Status.Status = dbcommons.StatusCreating r.Status().Update(ctx, m) @@ -834,6 +835,22 @@ func (r *DataguardBrokerReconciler) SetAsPrimaryDatabase(sidbSid string, targetS return requeueY } + dbInDgConfig := false + for i := 0; i < len(databases); i++ { + splitstr := strings.Split(databases[i], ":") + if strings.ToUpper(splitstr[0]) == strings.ToUpper(targetSid) { + dbInDgConfig = true + break + } + } + + if !dbInDgConfig { + eventReason := "Cannot Switchover" + eventMsg := fmt.Sprintf("Database %s not a part of the dataguard configuration", targetSid) + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) + return requeueN + } + // Fetch the current Primary database primarySid := dbcommons.GetPrimaryDatabase(databases) if strings.EqualFold(primarySid, targetSid) { @@ -961,7 +978,7 @@ func (r *DataguardBrokerReconciler) SetAsPrimaryDatabase(sidbSid string, targetS if err != nil { return requeueN } - out, err := dbcommons.GetDatabaseRole(primaryReadyPod, r, r.Config, ctx, primaryReq, n.Spec.Edition) + out, err := dbcommons.GetDatabaseRole(primaryReadyPod, r, r.Config, ctx, primaryReq) if err == nil { standbyDatabase.Status.Role = strings.ToUpper(out) } @@ -974,7 +991,7 @@ func (r *DataguardBrokerReconciler) SetAsPrimaryDatabase(sidbSid string, targetS Name: n.Name, }, } - out, err := dbcommons.GetDatabaseRole(sidbReadyPod, r, r.Config, ctx, sidbReq, n.Spec.Edition) + out, err := dbcommons.GetDatabaseRole(sidbReadyPod, r, r.Config, ctx, sidbReq) if err == nil { n.Status.Role = strings.ToUpper(out) } @@ -989,7 +1006,7 @@ func (r *DataguardBrokerReconciler) SetAsPrimaryDatabase(sidbSid string, targetS if err != nil { return requeueN } - out, err := dbcommons.GetDatabaseRole(targetReadyPod, r, r.Config, ctx, targetReq, n.Spec.Edition) + out, err := dbcommons.GetDatabaseRole(targetReadyPod, r, r.Config, ctx, targetReq) if err == nil { standbyDatabase.Status.Role = strings.ToUpper(out) } @@ -1002,7 +1019,7 @@ func (r *DataguardBrokerReconciler) SetAsPrimaryDatabase(sidbSid string, targetS Name: n.Name, }, } - out, err := dbcommons.GetDatabaseRole(sidbReadyPod, r, r.Config, ctx, sidbReq, n.Spec.Edition) + out, err := dbcommons.GetDatabaseRole(sidbReadyPod, r, r.Config, ctx, sidbReq) if err == nil { n.Status.Role = strings.ToUpper(out) } @@ -1125,20 +1142,6 @@ func (r *DataguardBrokerReconciler) cleanupDataguardBroker(req ctrl.Request, ctx return result, nil } - // Get its Role - out, err := dbcommons.GetDatabaseRole(sidbReadyPod, r, r.Config, ctx, req, singleInstanceDatabase.Spec.Edition) - if err != nil { - log.Error(err, err.Error()) - return requeueY, err - } - // check if its PRIMARY database - if strings.ToUpper(out) != "PRIMARY" { - eventReason := "Deletion" - eventMsg := "DataGuard Broker cannot be deleted since primaryDatabaseRef is not in PRIMARY role" - r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) - return requeueY, errors.New(eventMsg) - } - // Get Primary database to remove dataguard configuration _, _, err = dbcommons.GetDatabasesInDgConfig(sidbReadyPod, r, r.Config, ctx, req) if err != nil { @@ -1148,7 +1151,7 @@ func (r *DataguardBrokerReconciler) cleanupDataguardBroker(req ctrl.Request, ctx //primarySid := dbcommons.GetPrimaryDatabase(databases) - out, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + out, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", fmt.Sprintf("echo -e \"%s\" | dgmgrl / as sysdba ", dbcommons.RemoveDataguardConfiguration)) if err != nil { log.Error(err, err.Error()) diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 1a3359f7..783ae70c 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -80,7 +80,8 @@ type OracleRestDataServiceReconciler struct { //+kubebuilder:rbac:groups=database.oracle.com,resources=oraclerestdataservices,verbs=get;list;watch;create;update;patch;delete //+kubebuilder:rbac:groups=database.oracle.com,resources=oraclerestdataservices/status,verbs=get;update;patch //+kubebuilder:rbac:groups=database.oracle.com,resources=oraclerestdataservices/finalizers,verbs=update -//+kubebuilder:rbac:groups="",resources=pods;pods/log;pods/exec;persistentvolumeclaims;services;nodes;events,verbs=create;delete;get;list;patch;update;watch +//+kubebuilder:rbac:groups="",resources=pods;pods/log;pods/exec;persistentvolumeclaims;services,verbs=create;delete;get;list;patch;update;watch +//+kubebuilder:rbac:groups="",resources=events,verbs=create;patch // Reconcile is part of the main kubernetes reconciliation loop which aims to // move the current state of the cluster closer to the desired state. @@ -785,7 +786,7 @@ func (r *OracleRestDataServiceReconciler) instantiatePVCSpec(m *dbapi.OracleRest accessMode = append(accessMode, corev1.PersistentVolumeAccessMode(m.Spec.Persistence.AccessMode)) return accessMode }(), - Resources: corev1.ResourceRequirements{ + Resources: corev1.VolumeResourceRequirements{ Requests: map[corev1.ResourceName]resource.Quantity{ // Requests describes the minimum amount of compute resources required "storage": resource.MustParse(m.Spec.Persistence.Size), diff --git a/controllers/database/pdb_controller.go b/controllers/database/pdb_controller.go index 735deaf7..5c173740 100644 --- a/controllers/database/pdb_controller.go +++ b/controllers/database/pdb_controller.go @@ -128,6 +128,12 @@ var floodcontrol bool = false //+kubebuilder:rbac:groups=database.oracle.com,resources=pdbs/status,verbs=get;update;patch //+kubebuilder:rbac:groups=database.oracle.com,resources=pdbs/finalizers,verbs=get;create;update;patch;delete +// +kubebuilder:rbac:groups=core,resources=pods;pods/log;pods/exec;secrets;containers;services;events;configmaps;namespaces,verbs=get;list;watch;create;update;patch;delete +// +kubebuilder:rbac:groups=core,resources=pods/exec,verbs=create +// +kubebuilder:rbac:groups=apps,resources=statefulsets,verbs=get;list;watch;create;update;patch;delete +// +kubebuilder:rbac:groups='',resources=statefulsets/finalizers,verbs=get;list;watch;create;update;patch;delete + + // Reconcile is part of the main kubernetes reconciliation loop which aims to // move the current state of the cluster closer to the desired state. // TODO(user): Modify the Reconcile function to compare the state specified by @@ -137,6 +143,7 @@ var floodcontrol bool = false // // For more details, check Reconcile and its Result here: // - https://pkg.go.dev/sigs.k8s.io/controller-runtime@v0.9.2/pkg/reconcile + func (r *PDBReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { log := r.Log.WithValues("multitenantoperator", req.NamespacedName) log.Info("Reconcile requested") @@ -249,9 +256,11 @@ func (r *PDBReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.R return requeueY, nil } -/************************************************* - * Validate the PDB Spec - /************************************************/ +/* +************************************************ + - Validate the PDB Spec + /*********************************************** +*/ func (r *PDBReconciler) validatePhase(ctx context.Context, req ctrl.Request, pdb *dbapi.PDB) { log := r.Log.WithValues("validatePhase", req.NamespacedName) @@ -282,28 +291,27 @@ func (r *PDBReconciler) validatePhase(ctx context.Context, req ctrl.Request, pdb log.Info("Validation complete") } -/**************************************************************** - * Check for Duplicate PDB. Same PDB name on the same CDB resource. - /***************************************************************/ +/* +*************************************************************** + - Check for Duplicate PDB. Same PDB name on the same CDB resource. + /************************************************************** +*/ func (r *PDBReconciler) checkDuplicatePDB(ctx context.Context, req ctrl.Request, pdb *dbapi.PDB) error { - log := r.Log.WithValues("checkDuplicatePDB", req.NamespacedName) - // Name of the CDB CR that holds the ORDS container cdbResName := pdb.Spec.CDBResName - //cdbame := pdb.Spec.CDBName - // Name of the PDB resource + log := r.Log.WithValues("checkDuplicatePDB", pdb.Spec.CDBNamespace) pdbResName := pdb.Spec.PDBName pdbList := &dbapi.PDBList{} - listOpts := []client.ListOption{client.InNamespace(req.Namespace), client.MatchingFields{"spec.pdbName": pdbResName}} + listOpts := []client.ListOption{client.InNamespace(pdb.Spec.CDBNamespace), client.MatchingFields{"spec.pdbName": pdbResName}} // List retrieves list of objects for a given namespace and list options. err := r.List(ctx, pdbList, listOpts...) if err != nil { - log.Info("Failed to list pdbs", "Namespace", req.Namespace, "Error", err) + log.Info("Failed to list pdbs", "Namespace", pdb.Spec.CDBNamespace, "Error", err) return err } @@ -325,9 +333,11 @@ func (r *PDBReconciler) checkDuplicatePDB(ctx context.Context, req ctrl.Request, return nil } -/**************************************************************** - * Get the Custom Resource for the CDB mentioned in the PDB Spec - /***************************************************************/ +/* +*************************************************************** + - Get the Custom Resource for the CDB mentioned in the PDB Spec + /************************************************************** +*/ func (r *PDBReconciler) getCDBResource(ctx context.Context, req ctrl.Request, pdb *dbapi.PDB) (dbapi.CDB, error) { log := r.Log.WithValues("getCDBResource", req.NamespacedName) @@ -336,15 +346,16 @@ func (r *PDBReconciler) getCDBResource(ctx context.Context, req ctrl.Request, pd // Name of the CDB CR that holds the ORDS container cdbResName := pdb.Spec.CDBResName + cdbNamespace := pdb.Spec.CDBNamespace // Get CDB CR corresponding to the CDB name specified in the PDB spec err := r.Get(context.Background(), client.ObjectKey{ - Namespace: req.Namespace, + Namespace: cdbNamespace, Name: cdbResName, }, &cdb) if err != nil { - log.Info("Failed to get CRD for CDB", "Name", cdbResName, "Namespace", req.Namespace, "Error", err.Error()) + log.Info("Failed to get CRD for CDB", "Name", cdbResName, "Namespace", pdb.Spec.CDBNamespace, "Error", err.Error()) pdb.Status.Msg = "Unable to get CRD for CDB : " + cdbResName r.Status().Update(ctx, pdb) return cdb, err @@ -354,9 +365,11 @@ func (r *PDBReconciler) getCDBResource(ctx context.Context, req ctrl.Request, pd return cdb, nil } -/**************************************************************** - * Get the ORDS Pod for the CDB mentioned in the PDB Spec - /***************************************************************/ +/* +*************************************************************** + - Get the ORDS Pod for the CDB mentioned in the PDB Spec + /************************************************************** +*/ func (r *PDBReconciler) getORDSPod(ctx context.Context, req ctrl.Request, pdb *dbapi.PDB) (corev1.Pod, error) { log := r.Log.WithValues("getORDSPod", req.NamespacedName) @@ -368,12 +381,12 @@ func (r *PDBReconciler) getORDSPod(ctx context.Context, req ctrl.Request, pdb *d // Get ORDS Pod associated with the CDB Name specified in the PDB Spec err := r.Get(context.Background(), client.ObjectKey{ - Namespace: req.Namespace, + Namespace: pdb.Spec.CDBNamespace, Name: cdbResName + "-ords", }, &cdbPod) if err != nil { - log.Info("Failed to get Pod for CDB", "Name", cdbResName, "Namespace", req.Namespace, "Error", err.Error()) + log.Info("Failed to get Pod for CDB", "Name", cdbResName, "Namespace", pdb.Spec.CDBNamespace, "Error", err.Error()) pdb.Status.Msg = "Unable to get ORDS Pod for CDB : " + cdbResName return cdbPod, err } @@ -382,9 +395,11 @@ func (r *PDBReconciler) getORDSPod(ctx context.Context, req ctrl.Request, pdb *d return cdbPod, nil } -/************************************************* - * Get Secret Key for a Secret Name - /************************************************/ +/* +************************************************ + - Get Secret Key for a Secret Name + /*********************************************** +*/ func (r *PDBReconciler) getSecret(ctx context.Context, req ctrl.Request, pdb *dbapi.PDB, secretName string, keyName string) (string, error) { log := r.Log.WithValues("getSecret", req.NamespacedName) @@ -404,9 +419,11 @@ func (r *PDBReconciler) getSecret(ctx context.Context, req ctrl.Request, pdb *db return string(secret.Data[keyName]), nil } -/************************************************* - * Issue a REST API Call to the ORDS container - /************************************************/ +/* +************************************************ + - Issue a REST API Call to the ORDS container + /*********************************************** +*/ func (r *PDBReconciler) callAPI(ctx context.Context, req ctrl.Request, pdb *dbapi.PDB, url string, payload map[string]string, action string) (string, error) { log := r.Log.WithValues("callAPI", req.NamespacedName) @@ -474,37 +491,37 @@ func (r *PDBReconciler) callAPI(ctx context.Context, req ctrl.Request, pdb *dbap log.Info("Issuing REST call", "URL", url, "Action", action) - cdb, err := r.getCDBResource(ctx, req, pdb) - if err != nil { - return "", err - } + /* + cdb, err := r.getCDBResource(ctx, req, pdb) + if err != nil { + return "", err + } + */ - // Get Web Server User - //secret := &corev1.Secret{} - err = r.Get(ctx, types.NamespacedName{Name: cdb.Spec.WebServerUser.Secret.SecretName, Namespace: cdb.Namespace}, secret) + err = r.Get(ctx, types.NamespacedName{Name: pdb.Spec.WebServerUsr.Secret.SecretName, Namespace: pdb.Namespace}, secret) if err != nil { if apierrors.IsNotFound(err) { - log.Info("Secret not found:" + cdb.Spec.WebServerUser.Secret.SecretName) + log.Info("Secret not found:" + pdb.Spec.WebServerUsr.Secret.SecretName) return "", err } log.Error(err, "Unable to get the secret.") return "", err } - webUser := string(secret.Data[cdb.Spec.WebServerUser.Secret.Key]) + + webUser := string(secret.Data[pdb.Spec.WebServerUsr.Secret.Key]) webUser = strings.TrimSpace(webUser) - // Get Web Server User Password secret = &corev1.Secret{} - err = r.Get(ctx, types.NamespacedName{Name: cdb.Spec.WebServerPwd.Secret.SecretName, Namespace: cdb.Namespace}, secret) + err = r.Get(ctx, types.NamespacedName{Name: pdb.Spec.WebServerPwd.Secret.SecretName, Namespace: pdb.Namespace}, secret) if err != nil { if apierrors.IsNotFound(err) { - log.Info("Secret not found:" + cdb.Spec.WebServerPwd.Secret.SecretName) + log.Info("Secret not found:" + pdb.Spec.WebServerPwd.Secret.SecretName) return "", err } log.Error(err, "Unable to get the secret.") return "", err } - webUserPwd := string(secret.Data[cdb.Spec.WebServerPwd.Secret.Key]) + webUserPwd := string(secret.Data[pdb.Spec.WebServerPwd.Secret.Key]) webUserPwd = strings.TrimSpace(webUserPwd) var httpreq *http.Request @@ -596,9 +613,11 @@ func (r *PDBReconciler) callAPI(ctx context.Context, req ctrl.Request, pdb *dbap return respData, nil } -/************************************************* - * Create a PDB - /************************************************/ +/* +************************************************ + - Create a PDB + /*********************************************** +*/ func (r *PDBReconciler) createPDB(ctx context.Context, req ctrl.Request, pdb *dbapi.PDB) error { log := r.Log.WithValues("createPDB", req.NamespacedName) @@ -662,7 +681,8 @@ func (r *PDBReconciler) createPDB(ctx context.Context, req ctrl.Request, pdb *db values["tdeSecret"] = tdeSecret } - url := "https://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + //url := "https://"+ pdb.Spec.CDBNamespace + "." + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + url := "https://" + pdb.Spec.CDBResName + "-ords." + pdb.Spec.CDBNamespace + ":" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" pdb.Status.TotalSize = pdb.Spec.TotalSize pdb.Status.Phase = pdbPhaseCreate @@ -672,7 +692,7 @@ func (r *PDBReconciler) createPDB(ctx context.Context, req ctrl.Request, pdb *db } _, err = r.callAPI(ctx, req, pdb, url, values, "POST") if err != nil { - log.Error(err, "callAPI error", err.Error()) + log.Error(err, "callAPI error", "err", err.Error()) return err } @@ -681,8 +701,8 @@ func (r *PDBReconciler) createPDB(ctx context.Context, req ctrl.Request, pdb *db if cdb.Spec.DBServer != "" { pdb.Status.ConnString = cdb.Spec.DBServer + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName } else { - ParseTnsAlias(&(cdb.Spec.DBTnsurl), &(pdb.Spec.PDBName)) pdb.Status.ConnString = cdb.Spec.DBTnsurl + ParseTnsAlias(&(pdb.Status.ConnString), &(pdb.Spec.PDBName)) } log.Info("New connect strinng", "tnsurl", cdb.Spec.DBTnsurl) @@ -691,9 +711,11 @@ func (r *PDBReconciler) createPDB(ctx context.Context, req ctrl.Request, pdb *db return nil } -/************************************************* - * Clone a PDB - /************************************************/ +/* +************************************************ + - Clone a PDB + /*********************************************** +*/ func (r *PDBReconciler) clonePDB(ctx context.Context, req ctrl.Request, pdb *dbapi.PDB) error { if pdb.Spec.PDBName == pdb.Spec.SrcPDBName { @@ -741,7 +763,9 @@ func (r *PDBReconciler) clonePDB(ctx context.Context, req ctrl.Request, pdb *dba values["tempSize"] = pdb.Spec.TempSize } - url := "https://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdb.Spec.SrcPDBName + "/" + //url := "https://"+ pdb.Spec.CDBNamespace + "." + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdb.Spec.SrcPDBName + "/" + //url := "https://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdb.Spec.SrcPDBName + "/" + url := "https://" + pdb.Spec.CDBResName + "-ords." + pdb.Spec.CDBNamespace + ":" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdb.Spec.SrcPDBName + "/" pdb.Status.Phase = pdbPhaseClone pdb.Status.Msg = "Waiting for PDB to be cloned" @@ -758,8 +782,8 @@ func (r *PDBReconciler) clonePDB(ctx context.Context, req ctrl.Request, pdb *dba if cdb.Spec.DBServer != "" { pdb.Status.ConnString = cdb.Spec.DBServer + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName } else { - ParseTnsAlias(&(cdb.Spec.DBTnsurl), &(pdb.Spec.PDBName)) pdb.Status.ConnString = cdb.Spec.DBTnsurl + ParseTnsAlias(&(pdb.Status.ConnString), &(pdb.Spec.PDBName)) } log.Info("Cloned PDB successfully", "Source PDB Name", pdb.Spec.SrcPDBName, "Clone PDB Name", pdb.Spec.PDBName) @@ -767,9 +791,11 @@ func (r *PDBReconciler) clonePDB(ctx context.Context, req ctrl.Request, pdb *dba return nil } -/************************************************* - * Plug a PDB - /************************************************/ +/* +************************************************ + - Plug a PDB + /*********************************************** +*/ func (r *PDBReconciler) plugPDB(ctx context.Context, req ctrl.Request, pdb *dbapi.PDB) error { log := r.Log.WithValues("plugPDB", req.NamespacedName) @@ -819,7 +845,9 @@ func (r *PDBReconciler) plugPDB(ctx context.Context, req ctrl.Request, pdb *dbap values["asClone"] = strconv.FormatBool(*(pdb.Spec.AsClone)) } - url := "https://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + //url := "https://"+ pdb.Spec.CDBNamespace + "." + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + //url := "https://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + url := "https://" + pdb.Spec.CDBResName + "-ords." + pdb.Spec.CDBNamespace + ":" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" pdb.Status.TotalSize = pdb.Spec.TotalSize pdb.Status.Phase = pdbPhasePlug @@ -845,9 +873,11 @@ func (r *PDBReconciler) plugPDB(ctx context.Context, req ctrl.Request, pdb *dbap return nil } -/************************************************* - * Unplug a PDB - /************************************************/ +/* +************************************************ + - Unplug a PDB + /*********************************************** +*/ func (r *PDBReconciler) unplugPDB(ctx context.Context, req ctrl.Request, pdb *dbapi.PDB) error { log := r.Log.WithValues("unplugPDB", req.NamespacedName) @@ -885,7 +915,9 @@ func (r *PDBReconciler) unplugPDB(ctx context.Context, req ctrl.Request, pdb *db values["tdeExport"] = strconv.FormatBool(*(pdb.Spec.TDEExport)) } - url := "https://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdb.Spec.PDBName + "/" + //url := "https://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdb.Spec.PDBName + "/" + url := "https://" + pdb.Spec.CDBResName + "-ords." + pdb.Spec.CDBNamespace + ":" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdb.Spec.PDBName + "/" + log.Info("CallAPI(url)", "url", url) pdb.Status.Phase = pdbPhaseUnplug @@ -894,8 +926,8 @@ func (r *PDBReconciler) unplugPDB(ctx context.Context, req ctrl.Request, pdb *db if cdb.Spec.DBServer != "" { pdb.Status.ConnString = cdb.Spec.DBServer + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName } else { - ParseTnsAlias(&(cdb.Spec.DBTnsurl), &(pdb.Spec.PDBName)) pdb.Status.ConnString = cdb.Spec.DBTnsurl + ParseTnsAlias(&(pdb.Status.ConnString), &(pdb.Spec.PDBName)) } if err := r.Status().Update(ctx, pdb); err != nil { @@ -928,9 +960,11 @@ func (r *PDBReconciler) unplugPDB(ctx context.Context, req ctrl.Request, pdb *db return nil } -/************************************************* - * Modify a PDB state - /************************************************/ +/* +************************************************ + - Modify a PDB state + /*********************************************** +*/ func (r *PDBReconciler) modifyPDB(ctx context.Context, req ctrl.Request, pdb *dbapi.PDB) error { log := r.Log.WithValues("modifyPDB", req.NamespacedName) @@ -977,7 +1011,8 @@ func (r *PDBReconciler) modifyPDB(ctx context.Context, req ctrl.Request, pdb *db log.Info("PDB STATUS OPENMODE", "pdb.Status.OpenMode=", pdb.Status.OpenMode) pdbName := pdb.Spec.PDBName - url := "https://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdbName + "/status" + //url := "https://" + pdb.Spec.CDBNamespace + "." + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdbName + "/status" + url := "https://" + pdb.Spec.CDBResName + "-ords." + pdb.Spec.CDBNamespace + ":" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdbName + "/status" pdb.Status.Phase = pdbPhaseModify pdb.Status.ModifyOption = pdb.Spec.PDBState + "-" + pdb.Spec.ModifyOption @@ -1003,9 +1038,11 @@ func (r *PDBReconciler) modifyPDB(ctx context.Context, req ctrl.Request, pdb *db return nil } -/************************************************* - * Get PDB State - /************************************************/ +/* +************************************************ + - Get PDB State + /*********************************************** +*/ func (r *PDBReconciler) getPDBState(ctx context.Context, req ctrl.Request, pdb *dbapi.PDB) error { log := r.Log.WithValues("getPDBState", req.NamespacedName) @@ -1018,7 +1055,9 @@ func (r *PDBReconciler) getPDBState(ctx context.Context, req ctrl.Request, pdb * } pdbName := pdb.Spec.PDBName - url := "https://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdbName + "/status" + //url := "https://"+ pdb.Spec.CDBNamespace + "." + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdbName + "/status" + //url := "https://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdbName + "/status" + url := "https://" + pdb.Spec.CDBResName + "-ords." + pdb.Spec.CDBNamespace + ":" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdbName + "/status" pdb.Status.Msg = "Getting PDB state" if err := r.Status().Update(ctx, pdb); err != nil { @@ -1052,9 +1091,11 @@ func (r *PDBReconciler) getPDBState(ctx context.Context, req ctrl.Request, pdb * return nil } -/************************************************* - * Map Database PDB to Kubernetes PDB CR - /************************************************/ +/* +************************************************ + - Map Database PDB to Kubernetes PDB CR + /*********************************************** +*/ func (r *PDBReconciler) mapPDB(ctx context.Context, req ctrl.Request, pdb *dbapi.PDB) error { log := r.Log.WithValues("mapPDB", req.NamespacedName) @@ -1067,7 +1108,9 @@ func (r *PDBReconciler) mapPDB(ctx context.Context, req ctrl.Request, pdb *dbapi } pdbName := pdb.Spec.PDBName - url := "https://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdbName + "/" + //url := "https://"+ pdb.Spec.CDBNamespace + "." + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdbName + "/" + //url := "https://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdbName + "/" + url := "https://" + pdb.Spec.CDBResName + "-ords." + pdb.Spec.CDBNamespace + ":" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdbName + "/" pdb.Status.Msg = "Mapping PDB" if err := r.Status().Update(ctx, pdb); err != nil { @@ -1096,8 +1139,8 @@ func (r *PDBReconciler) mapPDB(ctx context.Context, req ctrl.Request, pdb *dbapi if cdb.Spec.DBServer != "" { pdb.Status.ConnString = cdb.Spec.DBServer + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName } else { - ParseTnsAlias(&(cdb.Spec.DBTnsurl), &(pdb.Spec.PDBName)) pdb.Status.ConnString = cdb.Spec.DBTnsurl + ParseTnsAlias(&(pdb.Status.ConnString), &(pdb.Spec.PDBName)) } if err := r.Status().Update(ctx, pdb); err != nil { @@ -1108,9 +1151,11 @@ func (r *PDBReconciler) mapPDB(ctx context.Context, req ctrl.Request, pdb *dbapi return nil } -/************************************************* - * Delete a PDB - /************************************************/ +/* +************************************************ + - Delete a PDB + /*********************************************** +*/ func (r *PDBReconciler) deletePDB(ctx context.Context, req ctrl.Request, pdb *dbapi.PDB) error { log := r.Log.WithValues("deletePDB", req.NamespacedName) @@ -1143,9 +1188,11 @@ func (r *PDBReconciler) deletePDB(ctx context.Context, req ctrl.Request, pdb *db return nil } -/************************************************* - * Check PDB deletion - /************************************************/ +/* +************************************************ + - Check PDB deletion + /*********************************************** +*/ func (r *PDBReconciler) managePDBDeletion(ctx context.Context, req ctrl.Request, pdb *dbapi.PDB) error { log := r.Log.WithValues("managePDBDeletion", req.NamespacedName) @@ -1187,9 +1234,11 @@ func (r *PDBReconciler) managePDBDeletion(ctx context.Context, req ctrl.Request, return nil } -/************************************************* - * Finalization logic for PDBFinalizer - /************************************************/ +/* +************************************************ + - Finalization logic for PDBFinalizer + /*********************************************** +*/ func (r *PDBReconciler) deletePDBInstance(req ctrl.Request, ctx context.Context, pdb *dbapi.PDB) error { log := r.Log.WithValues("deletePDBInstance", req.NamespacedName) @@ -1210,7 +1259,7 @@ func (r *PDBReconciler) deletePDBInstance(req ctrl.Request, ctx context.Context, } pdbName := pdb.Spec.PDBName - url := "https://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdbName + "/" + url := "https://" + pdb.Spec.CDBResName + "-ords." + pdb.Spec.CDBNamespace + ":" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdbName + "/" pdb.Status.Phase = pdbPhaseDelete pdb.Status.Msg = "Waiting for PDB to be deleted" @@ -1227,9 +1276,11 @@ func (r *PDBReconciler) deletePDBInstance(req ctrl.Request, ctx context.Context, return nil } -/************************************************************** - * SetupWithManager sets up the controller with the Manager. - /*************************************************************/ +/* +************************************************************* + - SetupWithManager sets up the controller with the Manager. + /************************************************************ +*/ func (r *PDBReconciler) SetupWithManager(mgr ctrl.Manager) error { return ctrl.NewControllerManagedBy(mgr). For(&dbapi.PDB{}). @@ -1253,6 +1304,7 @@ Enh 35357707 - PROVIDE THE PDB TNSALIAS INFORMATION **************************************************************/ func ParseTnsAlias(tns *string, pdbsrv *string) { + var swaptns string fmt.Printf("Analyzing string [%s]\n", *tns) fmt.Printf("Relacing srv [%s]\n", *pdbsrv) @@ -1266,9 +1318,9 @@ func ParseTnsAlias(tns *string, pdbsrv *string) { return } - *pdbsrv = fmt.Sprintf("SERVICE_NAME=%s", *pdbsrv) + swaptns = fmt.Sprintf("SERVICE_NAME=%s", *pdbsrv) tnsreg := regexp.MustCompile(`SERVICE_NAME=\w+`) - *tns = tnsreg.ReplaceAllString(*tns, *pdbsrv) + *tns = tnsreg.ReplaceAllString(*tns, swaptns) fmt.Printf("Newstring [%s]\n", *tns) diff --git a/controllers/database/shardingdatabase_controller.go b/controllers/database/shardingdatabase_controller.go index c76b7f45..88823afe 100644 --- a/controllers/database/shardingdatabase_controller.go +++ b/controllers/database/shardingdatabase_controller.go @@ -44,6 +44,7 @@ import ( "fmt" "reflect" "strconv" + "strings" "time" "github.com/go-logr/logr" @@ -70,7 +71,7 @@ import ( shardingv1 "github.com/oracle/oracle-database-operator/commons/sharding" ) -//Sharding Topology +// Sharding Topology type ShardingTopology struct { topicid string Instance *databasev1alpha1.ShardingDatabase @@ -89,12 +90,14 @@ type ShardingDatabaseReconciler struct { kubeConfig clientcmd.ClientConfig Recorder record.EventRecorder osh []*ShardingTopology + InCluster bool + Namespace string } // +kubebuilder:rbac:groups=database.oracle.com,resources=shardingdatabases,verbs=get;list;watch;create;update;patch;delete // +kubebuilder:rbac:groups=database.oracle.com,resources=shardingdatabases/status,verbs=get;update;patch // +kubebuilder:rbac:groups=database.oracle.com,resources=shardingdatabases/finalizers,verbs=get;create;update;patch;delete -// +kubebuilder:rbac:groups=core,resources=pods;pods/log;pods/exec;secrets;services;events;nodes;configmaps;persistentvolumeclaims;namespaces,verbs=get;list;watch;create;update;patch;delete +// +kubebuilder:rbac:groups=core,resources=pods;pods/log;pods/exec;secrets;containers;services;events;configmaps;persistentvolumeclaims;namespaces,verbs=get;list;watch;create;update;patch;delete // +kubebuilder:rbac:groups=core,resources=pods/exec,verbs=create // +kubebuilder:rbac:groups=apps,resources=statefulsets,verbs=get;list;watch;create;update;patch;delete // +kubebuilder:rbac:groups='',resources=statefulsets/finalizers,verbs=get;list;watch;create;update;patch;delete @@ -184,21 +187,21 @@ func (r *ShardingDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.Req if !instFlag { //r.setCrdLifeCycleState(instance, &result, &err, stateType) result = resultNq - return result, fmt.Errorf("DId not fid the instance in checkProvInstance") + return result, fmt.Errorf("DId not find the instance in checkProvInstance") } // ================================ OCI Notification Provider =========== r.getOnsConfigProvider(instance, idx) // =============================== Checking Namespace ============== - if instance.Spec.Namespace != "" { - err = shardingv1.AddNamespace(instance, r.Client, r.Log) - if err != nil { - //r.setCrdLifeCycleState(instance, &result, &err, stateType) - result = resultNq - return result, err - } - } else { + if instance.Spec.Namespace == "" { + ///err = shardingv1.AddNamespace(instance, r.Client, r.Log) + //if err != nil { + // //r.setCrdLifeCycleState(instance, &result, &err, stateType) + // result = resultNq + // return result, err + // } + // } else { instance.Spec.Namespace = "default" } @@ -441,6 +444,7 @@ func (r *ShardingDatabaseReconciler) SetupWithManager(mgr ctrl.Manager) error { Owns(&appsv1.StatefulSet{}). Owns(&corev1.Service{}). Owns(&corev1.Pod{}). + Owns(&corev1.Secret{}). WithEventFilter(r.eventFilterPredicate()). WithOptions(controller.Options{MaxConcurrentReconciles: 50}). //MaxConcurrentReconciles is the maximum number of concurrent Reconciles which can be run. Defaults to 1 Complete(r) @@ -454,6 +458,22 @@ func (r *ShardingDatabaseReconciler) eventFilterPredicate() predicate.Predicate return true }, UpdateFunc: func(e event.UpdateEvent) bool { + if old, ok := e.ObjectOld.(*corev1.Secret); ok { + if new, ok := e.ObjectNew.(*corev1.Secret); ok { + for i := 0; i < len(r.osh); i++ { + oshInst := r.osh[i] + if (new.Name == oshInst.Instance.Spec.DbSecret.Name) && (new.Name == old.Name) { + _, ok := old.Data[oshInst.Instance.Spec.DbSecret.PwdFileName] + if ok { + if !reflect.DeepEqual(old.Data[oshInst.Instance.Spec.DbSecret.PwdFileName], new.Data[oshInst.Instance.Spec.DbSecret.PwdFileName]) { + shardingv1.LogMessages("INFO", "Secret Changed", nil, oshInst.Instance, r.Log) + } + } + shardingv1.LogMessages("INFO", "Secret update block", nil, oshInst.Instance, r.Log) + } + } + } + } return true }, DeleteFunc: func(e event.DeleteEvent) bool { @@ -495,13 +515,32 @@ func (r *ShardingDatabaseReconciler) eventFilterPredicate() predicate.Predicate } } +// ================ Function to check secret update============= +func (r *ShardingDatabaseReconciler) UpdateSecret(instance *databasev1alpha1.ShardingDatabase, kClient client.Client, logger logr.Logger) (ctrl.Result, error) { + + sc := &corev1.Secret{} + //var err error + + // Reading a Secret + var err error = kClient.Get(context.TODO(), types.NamespacedName{ + Name: instance.Spec.DbSecret.Name, + Namespace: instance.Spec.Namespace, + }, sc) + + if err != nil { + return ctrl.Result{}, nil + } + + return ctrl.Result{}, nil +} + // ================== Function to get the Notification controller ============== func (r *ShardingDatabaseReconciler) getOnsConfigProvider(instance *databasev1alpha1.ShardingDatabase, idx int, ) { var err error - if instance.Spec.NsConfigMap != "" && instance.Spec.NsSecret != "" && r.osh[idx].onsProviderFlag != true { - cmName := instance.Spec.NsConfigMap - secName := instance.Spec.NsSecret + if instance.Spec.DbSecret.NsConfigMap != "" && instance.Spec.DbSecret.NsSecret != "" && r.osh[idx].onsProviderFlag != true { + cmName := instance.Spec.DbSecret.NsConfigMap + secName := instance.Spec.DbSecret.NsSecret shardingv1.LogMessages("DEBUG", "Received parameters are "+shardingv1.GetFmtStr(cmName)+","+shardingv1.GetFmtStr(secName), nil, instance, r.Log) region, user, tenancy, passphrase, fingerprint, topicid := shardingv1.ReadConfigMap(cmName, instance, r.Client, r.Log) privatekey := shardingv1.ReadSecret(secName, instance, r.Client, r.Log) @@ -831,6 +870,7 @@ func (r *ShardingDatabaseReconciler) validateSpex(instance *databasev1alpha1.Sha var eventMsg string var eventErr string = "Spec Error" + var i int32 lastSuccSpec, err := instance.GetLastSuccessfulSpec() if err != nil { @@ -841,6 +881,53 @@ func (r *ShardingDatabaseReconciler) validateSpex(instance *databasev1alpha1.Sha if lastSuccSpec == nil { // Logic to check if inital Spec is good or not + err = r.checkShardingType(instance, idx) + if err != nil { + return err + } + + if len(instance.Spec.Shard) > 0 { + for i = 0; i < int32(len(instance.Spec.Shard)); i++ { + OraShardSpex := instance.Spec.Shard[i] + if OraShardSpex.IsDelete != true { + err = r.checkShardSpace(instance, OraShardSpex) + if err != nil { + return err + } + err = r.checkShardGroup(instance, OraShardSpex) + if err != nil { + return err + } + } + } + } + + // Check Secret configuration + if instance.Spec.DbSecret == nil { + return fmt.Errorf("Secret specification cannot be null, you need to set secret details") + } else { + if len(instance.Spec.DbSecret.Name) == 0 { + return fmt.Errorf("instance.Spec.DbSecret.Name cannot be empty") + } + if len(instance.Spec.DbSecret.PwdFileName) == 0 { + return fmt.Errorf("instance.Spec.DbSecret.PwdFileName cannot be empty") + } + if strings.ToLower(instance.Spec.DbSecret.EncryptionType) != "base64" { + if strings.ToLower(instance.Spec.DbSecret.KeyFileName) == "" { + return fmt.Errorf("instance.Spec.DbSecret.KeyFileName cannot be empty") + } + } + if len(instance.Spec.DbSecret.PwdFileMountLocation) == 0 { + msg := "instance.Spec.DbSecret.PwdFileMountLocation is not set. Setting it to default " + shardingv1.GetSecretMount() + shardingv1.LogMessages("INFO", msg, nil, instance, r.Log) + } + + if len(instance.Spec.DbSecret.KeyFileMountLocation) == 0 { + msg := "instance.Spec.DbSecret.KeyFileMountLocation is not set. Setting it to default " + shardingv1.GetSecretMount() + shardingv1.LogMessages("INFO", msg, nil, instance, r.Log) + } + } + // Once the initial Spec is been validated then update the last Sucessful Spec err = instance.UpdateLastSuccessfulSpec(r.Client) if err != nil { @@ -890,6 +977,65 @@ func (r *ShardingDatabaseReconciler) validateSpex(instance *databasev1alpha1.Sha return nil } +func (r *ShardingDatabaseReconciler) checkShardingType(instance *databasev1alpha1.ShardingDatabase, idx int) error { + var i, k int32 + var regionFlag bool + + for k = 0; k < int32(len(instance.Spec.Gsm)); k++ { + regionFlag = false + for i = 0; i < int32(len(instance.Spec.Shard)); i++ { + if instance.Spec.Gsm[k].Region == instance.Spec.Shard[i].ShardRegion { + regionFlag = true + } + } + if !regionFlag { + msg := instance.Spec.Gsm[k].Region + " does not match with any region with Shard region. Region will be created during shard director provisioning" + shardingv1.LogMessages("INFO", msg, nil, instance, r.Log) + } + } + + return nil +} + +// Check the ShardGroups/ Shard Space and Shard group Name +// checkShrdGSR is Shardgroup/ShardSpace/ShardRegion + +func (r *ShardingDatabaseReconciler) checkShardSpace(instance *databasev1alpha1.ShardingDatabase, OraShardSpex databasev1alpha1.ShardSpec) error { + + if instance.Spec.ShardingType != "" { + // Check for the Sharding Type and if it is USER do following + if strings.TrimSpace(strings.ToUpper(instance.Spec.ShardingType)) == "USER" { + if len(OraShardSpex.ShardRegion) == 0 { + return fmt.Errorf("Shard region cannot be empty! ") + } + if len(OraShardSpex.ShardSpace) == 0 { + return fmt.Errorf("Shard Space in " + OraShardSpex.Name + " cannot be empty") + } + } + } + return nil +} + +// Check the ShardGroups/ Shard Space and Shard group Name +// checkShrdGSR is Shardgroup/ShardSpace/ShardRegion + +func (r *ShardingDatabaseReconciler) checkShardGroup(instance *databasev1alpha1.ShardingDatabase, OraShardSpex databasev1alpha1.ShardSpec) error { + + // We need to check Shard Region and Shard Group for ShardingType='SYSTEM' and 'NATIVE' + if strings.TrimSpace(strings.ToUpper(instance.Spec.ShardingType)) != "USER" { + if len(OraShardSpex.ShardRegion) == 0 { + return fmt.Errorf("Shard region cannot be empty! in " + OraShardSpex.Name) + } + if len(OraShardSpex.ShardGroup) == 0 { + return fmt.Errorf("Shard group in " + OraShardSpex.Name + " cannot be empty") + } + + // + + } + return nil +} + // Compare GSM Env Variables func (r *ShardingDatabaseReconciler) comapreGsmEnvVariables(instance *databasev1alpha1.ShardingDatabase, lastSuccSpec *databasev1alpha1.ShardingDatabaseSpec) bool { @@ -1029,6 +1175,7 @@ func (r *ShardingDatabaseReconciler) validateGsm(instance *databasev1alpha1.Shar if availableFlag != true { gsmSfSet = gsmSfSet1 gsmPod = gsmPod1 + // availableFlag = true availableFlag = true } } @@ -1102,6 +1249,7 @@ func (r *ShardingDatabaseReconciler) validateCatalog(instance *databasev1alpha1. if availlableFlag != true { catalogSfSet = catalogSfSet1 catalogPod = catalogPod1 + // availlableFlag = true availlableFlag = true } } @@ -1202,7 +1350,6 @@ func (r *ShardingDatabaseReconciler) validateShard(instance *databasev1alpha1.Sh } // This function updates the shard topology over all -// func (r *ShardingDatabaseReconciler) updateShardTopologyStatus(instance *databasev1alpha1.ShardingDatabase) error { //shardPod := &corev1.Pod{} //gsmSfSet := &appsv1.StatefulSet{} @@ -1237,7 +1384,7 @@ func (r *ShardingDatabaseReconciler) updateShardTopologyShardsInGsm(instance *da if err != nil { continue } else { - _ = r.verifyShards(instance, gsmPod, shardSfSet) + _ = r.verifyShards(instance, gsmPod, shardSfSet, OraShardSpex) } } @@ -1369,7 +1516,7 @@ func (r *ShardingDatabaseReconciler) addPrimaryShards(instance *databasev1alpha1 //gsmSfSet := &appsv1.StatefulSet{} gsmPod := &corev1.Pod{} var sparams1 string - var deployFlag = false + var deployFlag = true var errStr = false //var msg string @@ -1383,7 +1530,8 @@ func (r *ShardingDatabaseReconciler) addPrimaryShards(instance *databasev1alpha1 for i = 0; i < int32(len(instance.Spec.Shard)); i++ { OraShardSpex := instance.Spec.Shard[i] // stateStr := shardingv1.GetGsmShardStatus(instance, OraShardSpex.Name) - // !strings.Contains(stateStr, "DELETE") + // strings.Contains(stateStr, "DELETE") + if OraShardSpex.IsDelete != true { if setLifeCycleFlag != true { setLifeCycleFlag = true @@ -1395,52 +1543,88 @@ func (r *ShardingDatabaseReconciler) addPrimaryShards(instance *databasev1alpha1 shardSfSet, _, err = r.validateShard(instance, OraShardSpex, int(i)) if err != nil { errStr = true + deployFlag = false continue } // 2nd Step is to check if GSM is in good state if not then just return because you can't do anything _, gsmPod, err = r.validateGsm(instance) if err != nil { + deployFlag = false return err } // 3rd step to check if shard is in GSM if not then continue - sparams := shardingv1.BuildShardParams(shardSfSet) + sparams := shardingv1.BuildShardParams(instance, shardSfSet, OraShardSpex) sparams1 = sparams err = shardingv1.CheckShardInGsm(gsmPod.Name, sparams, instance, r.kubeClient, r.kubeConfig, r.Log) if err == nil { // if you are in this block then it means that shard already exist in the GSM and we do not need to anything continue } + + // Copy file from pod to FS + // configrest, kclientset, err := shardingv1.GetPodCopyConfig(r.kubeClient, r.kubeConfig, instance, r.Log) + // if err != nil { + // return fmt.Errorf("Error occurred in getting KubeConfig, cannot perform copy operation from the pod") + // } + + // _, _, err = shardingv1.ExecCommand(gsmPod.Name, shardingv1.GetTdeKeyLocCmd(), r.kubeClient, r.kubeConfig, instance, r.Log) + // if err != nil { + // fmt.Printf("Error occurred during the while getting the TDE key from the pod " + gsmPod.Name) + // //return err + // } + // fileName := "/tmp/tde_key" + // last := fileName[strings.LastIndex(fileName, "/")+1:] + // fileName1 := last + // fsLoc := shardingv1.TmpLoc + "/" + fileName1 + // _, _, _, err = shardingv1.KctlCopyFile(r.kubeClient, r.kubeConfig, instance, configrest, kclientset, r.Log, fmt.Sprintf("%s/%s:/%s", instance.Spec.Namespace, gsmPod.Name, fileName), fsLoc, "") + // if err != nil { + // fmt.Printf("failed to copy file") + // //return err + // } + + // Copying it to Shard Pod + // _, _, _, err = shardingv1.KctlCopyFile(r.kubeClient, r.kubeConfig, instance, configrest, kclientset, r.Log, fsLoc, fmt.Sprintf("%s/%s:/%s", instance.Spec.Namespace, OraShardSpex.Name+"-0", fsLoc), "") + // if err != nil { + // fmt.Printf("failed to copy file") + // //return err + /// } + // If the shard doesn't exist in GSM then just add the shard statefulset and update GSM shard status // ADD Shard in GSM + r.updateGsmShardStatus(instance, OraShardSpex.Name, string(databasev1alpha1.AddingShardState)) - err := shardingv1.AddShardInGsm(gsmPod.Name, sparams, instance, r.kubeClient, r.kubeConfig, r.Log) + err = shardingv1.AddShardInGsm(gsmPod.Name, sparams, instance, r.kubeClient, r.kubeConfig, r.Log) if err != nil { r.updateGsmShardStatus(instance, OraShardSpex.Name, string(databasev1alpha1.AddingShardErrorState)) title = "Shard Addition Failure" message = "Error occurred during shard " + shardingv1.GetFmtStr(OraShardSpex.Name) + " addition." r.sendMessage(instance, title, message) - } else { - deployFlag = true + deployFlag = false } } } + if errStr == true { + shardingv1.LogMessages("INFO", "Some shards are still pending for addition. Requeue the reconcile loop.", nil, instance, r.Log) + return fmt.Errorf("shards are not ready for addition.") + } + // ======= Deploy Shard Logic ========= if deployFlag == true { _ = shardingv1.DeployShardInGsm(gsmPod.Name, sparams1, instance, r.kubeClient, r.kubeConfig, r.Log) r.updateShardTopologyShardsInGsm(instance, gsmPod) + } else { + shardingv1.LogMessages("INFO", "Shards are not added in GSM. Deploy operation will happen after shard addition. Requeue the reconcile loop.", nil, instance, r.Log) + return fmt.Errorf("shards addition are pending.") } } - if errStr == true { - shardingv1.LogMessages("INFO", "Some shards are still pending for addition. Requeue the reconcile loop.", nil, instance, r.Log) - return fmt.Errorf("Shard Addition is pending.") - } + shardingv1.LogMessages("INFO", "Completed the shard addition operation. For details, check the CRD resource status for GSM and Shards.", nil, instance, r.Log) return nil } // This function Check the online shard -func (r *ShardingDatabaseReconciler) verifyShards(instance *databasev1alpha1.ShardingDatabase, gsmPod *corev1.Pod, shardSfSet *appsv1.StatefulSet) error { +func (r *ShardingDatabaseReconciler) verifyShards(instance *databasev1alpha1.ShardingDatabase, gsmPod *corev1.Pod, shardSfSet *appsv1.StatefulSet, OraShardSpex databasev1alpha1.ShardSpec) error { //var result ctrl.Result //var i int32 var err error @@ -1448,13 +1632,15 @@ func (r *ShardingDatabaseReconciler) verifyShards(instance *databasev1alpha1.Sha var message string // ================================ Check Shards ================== //veryify shard make shard state online and it must be executed to check shard state after every CRUD operation - sparams := shardingv1.BuildShardParams(shardSfSet) + sparams := shardingv1.BuildShardParams(instance, shardSfSet, OraShardSpex) err = shardingv1.CheckOnlineShardInGsm(gsmPod.Name, sparams, instance, r.kubeClient, r.kubeConfig, r.Log) if err != nil { // If the shard doesn't exist in GSM then just delete the shard statefulset and update GSM shard status /// Terminate state means we will remove teh shard entry from GSM shard status r.updateGsmShardStatus(instance, shardSfSet.Name, string(databasev1alpha1.ShardOnlineErrorState)) - shardingv1.CancelChunksInGsm(gsmPod.Name, sparams, instance, r.kubeClient, r.kubeConfig, r.Log) + if strings.ToUpper(instance.Spec.ReplicationType) != "NATIVE" { + shardingv1.CancelChunksInGsm(gsmPod.Name, sparams, instance, r.kubeClient, r.kubeConfig, r.Log) + } return err } oldStateStr := shardingv1.GetGsmShardStatus(instance, shardSfSet.Name) @@ -1490,6 +1676,7 @@ func (r *ShardingDatabaseReconciler) delGsmShard(instance *databasev1alpha1.Shar shardingv1.LogMessages("DEBUG", "Starting shard deletion operation.", nil, instance, r.Log) // ================================ Shard Delete Logic =================== + if len(instance.Spec.Shard) > 0 { for i = 0; i < int32(len(instance.Spec.Shard)); i++ { OraShardSpex := instance.Spec.Shard[i] @@ -1518,7 +1705,7 @@ func (r *ShardingDatabaseReconciler) delGsmShard(instance *databasev1alpha1.Shar continue } // 3rd step to check if shard is in GSM if not then continue - sparams := shardingv1.BuildShardParams(shardSfSet) + sparams := shardingv1.BuildShardParams(instance, shardSfSet, OraShardSpex) err = shardingv1.CheckShardInGsm(gsmPod.Name, sparams, instance, r.kubeClient, r.kubeConfig, r.Log) if err != nil { // If the shard doesn't exist in GSM then just delete the shard statefulset and update GSM shard status @@ -1541,28 +1728,33 @@ func (r *ShardingDatabaseReconciler) delGsmShard(instance *databasev1alpha1.Shar // 5th Step // Move the chunks before performing any Delete // If you are in this block then it means that shard is ONline and can be deleted - err = shardingv1.MoveChunks(gsmPod.Name, sparams, instance, r.kubeClient, r.kubeConfig, r.Log) - if err != nil { - r.updateGsmShardStatus(instance, OraShardSpex.Name, string(databasev1alpha1.ChunkMoveError)) - title = "Chunk Movement Failure" - message = "Error occurred during chunk movement in shard " + shardingv1.GetFmtStr(OraShardSpex.Name) + " deletion." - r.sendMessage(instance, title, message) - continue - } - // 6th Step - // Check if Chunks has moved before performing actual delete - // This is a loop and will check unless there is a error or chunks has moved - // Validate if the chunks has moved before performing shard deletion - for { - err = shardingv1.VerifyChunks(gsmPod.Name, sparams, instance, r.kubeClient, r.kubeConfig, r.Log) - if err == nil { - break - } else { - msg = "Sleeping for 120 seconds and will check status again of chunks movement in gsm for shard: " + shardingv1.GetFmtStr(OraShardSpex.Name) - shardingv1.LogMessages("INFO", msg, nil, instance, r.Log) - time.Sleep(120 * time.Second) + if strings.ToUpper(instance.Spec.ReplicationType) != "NATIVE" { + if len(instance.Spec.ReplicationType) == 0 { + err = shardingv1.MoveChunks(gsmPod.Name, sparams, instance, r.kubeClient, r.kubeConfig, r.Log) + if err != nil { + r.updateGsmShardStatus(instance, OraShardSpex.Name, string(databasev1alpha1.ChunkMoveError)) + title = "Chunk Movement Failure" + message = "Error occurred during chunk movement in shard " + shardingv1.GetFmtStr(OraShardSpex.Name) + " deletion." + r.sendMessage(instance, title, message) + continue + } + // 6th Step + // Check if Chunks has moved before performing actual delete + // This is a loop and will check unless there is a error or chunks has moved + // Validate if the chunks has moved before performing shard deletion + for { + err = shardingv1.VerifyChunks(gsmPod.Name, sparams, instance, r.kubeClient, r.kubeConfig, r.Log) + if err == nil { + break + } else { + msg = "Sleeping for 120 seconds and will check status again of chunks movement in gsm for shard: " + shardingv1.GetFmtStr(OraShardSpex.Name) + shardingv1.LogMessages("INFO", msg, nil, instance, r.Log) + time.Sleep(120 * time.Second) + } + } } } + // 7th Step remove the shards from the GSM // This steps will delete the shard entry from the GSM // It will delete CDB from catalog @@ -1574,6 +1766,7 @@ func (r *ShardingDatabaseReconciler) delGsmShard(instance *databasev1alpha1.Shar r.updateShardStatus(instance, int(i), string(databasev1alpha1.ShardRemoveError)) continue } + // 8th Step // Delete the Statefulset as all the chunks has moved and Shard can be phyiscally deleted r.delShard(instance, shardSfSet.Name, shardSfSet, shardPod, int(i)) @@ -1645,7 +1838,7 @@ func (r *ShardingDatabaseReconciler) delShard(instance *databasev1alpha1.Shardin } } -//======== GSM Invited Node ========== +// ======== GSM Invited Node ========== // Remove and add GSM invited node func (r *ShardingDatabaseReconciler) gsmInvitedNodeOp(instance *databasev1alpha1.ShardingDatabase, objName string, ) { diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index e468214c..fa01ae7e 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -53,6 +53,7 @@ import ( "github.com/go-logr/logr" corev1 "k8s.io/api/core/v1" + storagev1 "k8s.io/api/storage/v1" apierrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/api/meta" "k8s.io/apimachinery/pkg/api/resource" @@ -92,7 +93,10 @@ var oemExpressUrl string //+kubebuilder:rbac:groups=database.oracle.com,resources=singleinstancedatabases,verbs=get;list;watch;create;update;patch;delete //+kubebuilder:rbac:groups=database.oracle.com,resources=singleinstancedatabases/status,verbs=get;update;patch //+kubebuilder:rbac:groups=database.oracle.com,resources=singleinstancedatabases/finalizers,verbs=update -//+kubebuilder:rbac:groups="",resources=pods;pods/log;pods/exec;persistentvolumeclaims;services;nodes;events,verbs=create;delete;get;list;patch;update;watch +//+kubebuilder:rbac:groups="",resources=pods;pods/log;pods/exec;persistentvolumeclaims;services,verbs=create;delete;get;list;patch;update;watch +//+kubebuilder:rbac:groups="",resources=persistentvolumes,verbs=get;list;watch +//+kubebuilder:rbac:groups="",resources=events,verbs=create;patch +//+kubebuilder:rbac:groups=storage.k8s.io,resources=storageclasses,verbs=get;list;watch // Reconcile is part of the main kubernetes reconciliation loop which aims to // move the current state of the cluster closer to the desired state. @@ -167,8 +171,15 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct return result, nil } - // PVC Creation - result, err = r.createOrReplacePVC(ctx, req, singleInstanceDatabase) + // PVC Creation for Datafiles Volume + result, err = r.createOrReplacePVCforDatafilesVol(ctx, req, singleInstanceDatabase) + if result.Requeue { + r.Log.Info("Reconcile queued") + return result, nil + } + + // PVC Creation for customScripts Volume + result, err = r.createOrReplacePVCforCustomScriptsVol(ctx, req, singleInstanceDatabase) if result.Requeue { r.Log.Info("Reconcile queued") return result, nil @@ -189,8 +200,7 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct } // Validate readiness - var readyPod corev1.Pod - result, readyPod, err = r.validateDBReadiness(singleInstanceDatabase, ctx, req) + result, readyPod, err := r.validateDBReadiness(singleInstanceDatabase, ctx, req) if result.Requeue { r.Log.Info("Reconcile queued") return result, nil @@ -207,7 +217,9 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct } } - if strings.ToUpper(singleInstanceDatabase.Status.Role) == "PRIMARY" { + sidbRole, err := dbcommons.GetDatabaseRole(readyPod, r, r.Config, ctx, req) + + if sidbRole == "PRIMARY" { // Update DB config result, err = r.updateDBConfig(singleInstanceDatabase, readyPod, ctx, req) @@ -232,9 +244,11 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct } else { // Database is in role of standby - err = SetupStandbyDatabase(r, singleInstanceDatabase, referredPrimaryDatabase, ctx, req) - if err != nil { - return requeueY, err + if !singleInstanceDatabase.Status.DgBrokerConfigured { + err = SetupStandbyDatabase(r, singleInstanceDatabase, referredPrimaryDatabase, ctx, req) + if err != nil { + return requeueY, err + } } databaseOpenMode, err := dbcommons.GetDatabaseOpenMode(readyPod, r, r.Config, ctx, req, singleInstanceDatabase.Spec.Edition) @@ -245,7 +259,8 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct } r.Log.Info("DB openMode Output") r.Log.Info(databaseOpenMode) - if databaseOpenMode == "READ_ONLY" { + if databaseOpenMode == "READ_ONLY" || databaseOpenMode == "MOUNTED" { + // Changing the open mode for sidb to "READ ONLY WITH APPLY" out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", fmt.Sprintf("echo -e \"%s\" | %s", dbcommons.ModifyStdbyDBOpenMode, dbcommons.SQLPlusCLI)) if err != nil { r.Log.Error(err, err.Error()) @@ -284,8 +299,11 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct return requeueY, nil } - // update status to Ready after all operations succeed - singleInstanceDatabase.Status.Status = dbcommons.StatusReady + // updating singleinstancedatabase Status + err = r.updateSidbStatus(singleInstanceDatabase, readyPod, ctx, req) + if err != nil { + return requeueY, err + } r.updateORDSStatus(singleInstanceDatabase, ctx, req) completed = true @@ -408,26 +426,12 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab if m.Status.Sid != "" && !strings.EqualFold(m.Spec.Sid, m.Status.Sid) { eventMsgs = append(eventMsgs, "sid cannot be updated") } - if m.Spec.CloneFrom == "" && m.Spec.Edition != "" && m.Status.Edition != "" && !strings.EqualFold(m.Status.Edition, m.Spec.Edition) { - eventMsgs = append(eventMsgs, "edition cannot be updated") - } if m.Status.Charset != "" && !strings.EqualFold(m.Status.Charset, m.Spec.Charset) { eventMsgs = append(eventMsgs, "charset cannot be updated") } if m.Status.Pdbname != "" && !strings.EqualFold(m.Status.Pdbname, m.Spec.Pdbname) { eventMsgs = append(eventMsgs, "pdbName cannot be updated") } - if m.Status.CloneFrom != "" && - (m.Status.CloneFrom == dbcommons.NoCloneRef && m.Spec.CloneFrom != "" || - m.Status.CloneFrom != dbcommons.NoCloneRef && m.Status.CloneFrom != m.Spec.CloneFrom) { - eventMsgs = append(eventMsgs, "cloneFrom cannot be updated") - } - if (m.Spec.Edition == "express" || m.Spec.Edition == "free") && m.Spec.CloneFrom != "" { - eventMsgs = append(eventMsgs, "cloning not supported for "+m.Spec.Edition+" edition") - } - if (m.Spec.Edition == "express" || m.Spec.Edition == "free") && m.Spec.PrimaryDatabaseRef != "" && m.Spec.CreateAsStandby { - eventMsgs = append(eventMsgs, "Standby database creation is not supported for "+m.Spec.Edition+" edition") - } if m.Status.OrdsReference != "" && m.Status.Persistence.Size != "" && m.Status.Persistence != m.Spec.Persistence { eventMsgs = append(eventMsgs, "uninstall ORDS to change Peristence") } @@ -462,20 +466,16 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab m.Status.Pdbname = m.Spec.Pdbname m.Status.Persistence = m.Spec.Persistence m.Status.PrebuiltDB = m.Spec.Image.PrebuiltDB - if m.Spec.CloneFrom == "" { - m.Status.CloneFrom = dbcommons.NoCloneRef - } else { - m.Status.CloneFrom = m.Spec.CloneFrom - } - if m.Spec.CloneFrom != "" { + + if m.Spec.CreateAs == "clone" { // Once a clone database has created , it has no link with its reference if m.Status.DatafilesCreated == "true" || - !dbcommons.IsSourceDatabaseOnCluster(m.Spec.CloneFrom) { + !dbcommons.IsSourceDatabaseOnCluster(m.Spec.PrimaryDatabaseRef) { return requeueN, nil } // Fetch the Clone database reference - err = r.Get(ctx, types.NamespacedName{Namespace: req.Namespace, Name: m.Spec.CloneFrom}, n) + err = r.Get(ctx, types.NamespacedName{Namespace: req.Namespace, Name: m.Spec.PrimaryDatabaseRef}, n) if err != nil { if apierrors.IsNotFound(err) { r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, err.Error()) @@ -488,13 +488,13 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab if n.Status.Status != dbcommons.StatusReady { m.Status.Status = dbcommons.StatusPending eventReason := "Source Database Pending" - eventMsg := "status of database " + m.Spec.CloneFrom + " is not ready, retrying..." + eventMsg := "status of database " + n.Name + " is not ready, retrying..." r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) err = errors.New(eventMsg) return requeueY, err } - if !n.Spec.ArchiveLog { + if !*n.Spec.ArchiveLog { m.Status.Status = dbcommons.StatusPending eventReason := "Source Database Check" eventMsg := "enable ArchiveLog for database " + n.Name @@ -505,10 +505,10 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab } m.Status.Edition = n.Status.Edition - + m.Status.PrimaryDatabase = n.Name } - if m.Spec.PrimaryDatabaseRef != "" && m.Spec.CreateAsStandby { + if m.Spec.CreateAs == "standby" && m.Status.Role != "PRIMARY" { // Fetch the Primary database reference, required for all iterations err = r.Get(ctx, types.NamespacedName{Namespace: req.Namespace, Name: m.Spec.PrimaryDatabaseRef}, rp) @@ -545,7 +545,7 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab return requeueY, err } - r.Log.Info("Settingup Primary Database for standby creation...") + r.Log.Info("Setting up Primary Database for standby creation...") err = SetupPrimaryDatabase(r, m, rp, ctx, req) if err != nil { return requeueY, err @@ -636,7 +636,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns } }(), Volumes: []corev1.Volume{{ - Name: "datamount", + Name: "datafiles-vol", VolumeSource: func() corev1.VolumeSource { if m.Spec.Persistence.Size == "" { return corev1.VolumeSource{EmptyDir: &corev1.EmptyDirVolumeSource{}} @@ -661,10 +661,48 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }}, }, }, + }, { + Name: "tls-secret-vol", + VolumeSource: func() corev1.VolumeSource { + if m.Spec.TcpsTlsSecret == "" { + return corev1.VolumeSource{EmptyDir: &corev1.EmptyDirVolumeSource{}} + } + /* tls-secret is specified */ + return corev1.VolumeSource{ + Secret: &corev1.SecretVolumeSource{ + SecretName: m.Spec.TcpsTlsSecret, + Optional: func() *bool { i := true; return &i }(), + Items: []corev1.KeyToPath{ + { + Key: "tls.crt", // Mount the certificate + Path: "cert.crt", // Mount path inside the container + }, + { + Key: "tls.key", // Mount the private key + Path: "client.key", // Mount path inside the container + }, + }, + }, + } + }(), + }, { + Name: "custom-scripts-vol", + VolumeSource: func() corev1.VolumeSource { + if m.Spec.Persistence.ScriptsVolumeName == "" || m.Spec.Persistence.ScriptsVolumeName == m.Spec.Persistence.DatafilesVolumeName { + return corev1.VolumeSource{EmptyDir: &corev1.EmptyDirVolumeSource{}} + } + /* Persistence.ScriptsVolumeName is specified */ + return corev1.VolumeSource{ + PersistentVolumeClaim: &corev1.PersistentVolumeClaimVolumeSource{ + ClaimName: m.Name + "-" + m.Spec.Persistence.ScriptsVolumeName, + ReadOnly: false, + }, + } + }(), }}, InitContainers: func() []corev1.Container { initContainers := []corev1.Container{} - if m.Spec.Persistence.Size != "" { + if m.Spec.Persistence.Size != "" && m.Spec.Persistence.SetWritePermissions != nil && *m.Spec.Persistence.SetWritePermissions { initContainers = append(initContainers, corev1.Container{ Name: "init-permissions", Image: m.Spec.Image.PullFrom, @@ -675,30 +713,30 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, VolumeMounts: []corev1.VolumeMount{{ MountPath: "/opt/oracle/oradata", - Name: "datamount", + Name: "datafiles-vol", }}, }) - if m.Spec.Image.PrebuiltDB { - initContainers = append(initContainers, corev1.Container{ - Name: "init-prebuiltdb", - Image: m.Spec.Image.PullFrom, - Command: []string{"/bin/sh", "-c", dbcommons.InitPrebuiltDbCMD}, - SecurityContext: &corev1.SecurityContext{ - RunAsUser: func() *int64 { i := int64(dbcommons.ORACLE_UID); return &i }(), - RunAsGroup: func() *int64 { i := int64(dbcommons.ORACLE_GUID); return &i }(), - }, - VolumeMounts: []corev1.VolumeMount{{ - MountPath: "/mnt/oradata", - Name: "datamount", - }}, - Env: []corev1.EnvVar{ - { - Name: "ORACLE_SID", - Value: strings.ToUpper(m.Spec.Sid), - }, + } + if m.Spec.Image.PrebuiltDB { + initContainers = append(initContainers, corev1.Container{ + Name: "init-prebuiltdb", + Image: m.Spec.Image.PullFrom, + Command: []string{"/bin/sh", "-c", dbcommons.InitPrebuiltDbCMD}, + SecurityContext: &corev1.SecurityContext{ + RunAsUser: func() *int64 { i := int64(dbcommons.ORACLE_UID); return &i }(), + RunAsGroup: func() *int64 { i := int64(dbcommons.ORACLE_GUID); return &i }(), + }, + VolumeMounts: []corev1.VolumeMount{{ + MountPath: "/mnt/oradata", + Name: "datafiles-vol", + }}, + Env: []corev1.EnvVar{ + { + Name: "ORACLE_SID", + Value: strings.ToUpper(m.Spec.Sid), }, - }) - } + }, + }) } /* Wallet only for edition barring express and free editions, non-prebuiltDB */ if (m.Spec.Edition != "express" && m.Spec.Edition != "free") && !m.Spec.Image.PrebuiltDB { @@ -716,19 +754,19 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, { Name: "WALLET_DIR", - Value: "/opt/oracle/oradata/dbconfig/$(ORACLE_SID)/.wallet", + Value: "/opt/oracle/oradata/dbconfig/${ORACLE_SID}/.wallet", }, }, Command: []string{"/bin/sh"}, Args: func() []string { edition := "" - if m.Spec.CloneFrom == "" { + if m.Spec.CreateAs != "clone" { edition = m.Spec.Edition if m.Spec.Edition == "" { edition = "enterprise" } } else { - if !dbcommons.IsSourceDatabaseOnCluster(m.Spec.CloneFrom) { + if !dbcommons.IsSourceDatabaseOnCluster(m.Spec.PrimaryDatabaseRef) { edition = m.Spec.Edition } else { edition = n.Spec.Edition @@ -745,7 +783,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, VolumeMounts: []corev1.VolumeMount{{ MountPath: "/opt/oracle/oradata", - Name: "datamount", + Name: "datafiles-vol", }}, }) } @@ -754,42 +792,75 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns Containers: []corev1.Container{{ Name: m.Name, Image: m.Spec.Image.PullFrom, + SecurityContext: &corev1.SecurityContext{ + Capabilities: &corev1.Capabilities{ + // Allow priority elevation for DB processes + Add: []corev1.Capability{"SYS_NICE"}, + }, + }, Lifecycle: &corev1.Lifecycle{ PreStop: &corev1.LifecycleHandler{ Exec: &corev1.ExecAction{ - Command: []string{"/bin/sh", "-c", "/bin/echo -en 'shutdown abort;\n' | env ORACLE_SID=${ORACLE_SID^^} sqlplus -S / as sysdba"}, + Command: func() []string { + // For patching use cases shutdown immediate is needed especially for standby databases + shutdown_mode := "immediate" + if m.Spec.Edition == "express" || m.Spec.Edition == "free" { + // express/free do not support patching + // To terminate any zombie instances left over due to forced termination + shutdown_mode = "abort" + } + return []string{"/bin/sh", "-c", "/bin/echo -en 'shutdown " + shutdown_mode + ";\n' | env ORACLE_SID=${ORACLE_SID^^} sqlplus -S / as sysdba"} + }(), }, }, }, Ports: []corev1.ContainerPort{{ContainerPort: dbcommons.CONTAINER_LISTENER_PORT}, {ContainerPort: 5500}}, - ReadinessProbe: &corev1.Probe{ - ProbeHandler: corev1.ProbeHandler{ - Exec: &corev1.ExecAction{ - Command: []string{"/bin/sh", "-c", "if [ -f $ORACLE_BASE/checkDBLockStatus.sh ]; then $ORACLE_BASE/checkDBLockStatus.sh ; else $ORACLE_BASE/checkDBStatus.sh; fi "}, - }, - }, - InitialDelaySeconds: 20, - TimeoutSeconds: 20, - PeriodSeconds: func() int32 { - if m.Spec.ReadinessCheckPeriod > 0 { - return int32(m.Spec.ReadinessCheckPeriod) + ReadinessProbe: func() *corev1.Probe { + if m.Spec.CreateAs == "primary" { + return &corev1.Probe{ + ProbeHandler: corev1.ProbeHandler{ + Exec: &corev1.ExecAction{ + Command: []string{"/bin/sh", "-c", "if [ -f $ORACLE_BASE/checkDBLockStatus.sh ]; then $ORACLE_BASE/checkDBLockStatus.sh ; else $ORACLE_BASE/checkDBStatus.sh; fi "}, + }, + }, + InitialDelaySeconds: 20, + TimeoutSeconds: 20, + PeriodSeconds: func() int32 { + if m.Spec.ReadinessCheckPeriod > 0 { + return int32(m.Spec.ReadinessCheckPeriod) + } + return 60 + }(), } - return 60 - }(), - }, - + } else { + return &corev1.Probe{ + ProbeHandler: corev1.ProbeHandler{ + Exec: &corev1.ExecAction{ + Command: []string{"/bin/sh", "-c", "if [ -f $ORACLE_BASE/oradata/.$ORACLE_SID$CHECKPOINT_FILE_EXTN ]; then if [ -f $ORACLE_BASE/checkDBLockStatus.sh ]; then $ORACLE_BASE/checkDBLockStatus.sh ; else $ORACLE_BASE/checkDBStatus.sh; fi else true; fi "}, + }, + }, + InitialDelaySeconds: 0, + TimeoutSeconds: 20, + PeriodSeconds: func() int32 { + if m.Spec.ReadinessCheckPeriod > 0 { + return int32(m.Spec.ReadinessCheckPeriod) + } + return 60 + }(), + } + } + }(), VolumeMounts: func() []corev1.VolumeMount { mounts := []corev1.VolumeMount{} if m.Spec.Persistence.Size != "" { mounts = append(mounts, corev1.VolumeMount{ MountPath: "/opt/oracle/oradata", - Name: "datamount", + Name: "datafiles-vol", }) } if m.Spec.Edition == "express" || m.Spec.Edition == "free" || m.Spec.Image.PrebuiltDB { - // mounts pwd as secrets for express edition - // or prebuilt db + // mounts pwd as secrets for express edition or prebuilt db mounts = append(mounts, corev1.VolumeMount{ MountPath: "/run/secrets/oracle_pwd", ReadOnly: true, @@ -797,6 +868,39 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns SubPath: "oracle_pwd", }) } + if m.Spec.TcpsTlsSecret != "" { + mounts = append(mounts, corev1.VolumeMount{ + MountPath: dbcommons.TlsCertsLocation, + ReadOnly: true, + Name: "tls-secret-vol", + }) + } + if m.Spec.Persistence.ScriptsVolumeName != "" { + mounts = append(mounts, corev1.VolumeMount{ + MountPath: "/opt/oracle/scripts/startup/", + ReadOnly: true, + Name: func() string { + if m.Spec.Persistence.ScriptsVolumeName != m.Spec.Persistence.DatafilesVolumeName { + return "custom-scripts-vol" + } else { + return "datafiles-vol" + } + }(), + SubPath: "startup", + }) + mounts = append(mounts, corev1.VolumeMount{ + MountPath: "/opt/oracle/scripts/setup/", + ReadOnly: true, + Name: func() string { + if m.Spec.Persistence.ScriptsVolumeName != m.Spec.Persistence.DatafilesVolumeName { + return "custom-scripts-vol" + } else { + return "datafiles-vol" + } + }(), + SubPath: "setup", + }) + } return mounts }(), Env: func() []corev1.EnvVar { @@ -821,7 +925,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, } } - if m.Spec.CloneFrom != "" { + if m.Spec.CreateAs == "clone" { // Clone DB use-case return []corev1.EnvVar{ { @@ -838,25 +942,18 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, { Name: "WALLET_DIR", - Value: "/opt/oracle/oradata/dbconfig/$(ORACLE_SID)/.wallet", + Value: "/opt/oracle/oradata/dbconfig/${ORACLE_SID}/.wallet", }, { Name: "PRIMARY_DB_CONN_STR", Value: func() string { - if dbcommons.IsSourceDatabaseOnCluster(m.Spec.CloneFrom) { + if dbcommons.IsSourceDatabaseOnCluster(m.Spec.PrimaryDatabaseRef) { return n.Name + ":" + strconv.Itoa(int(dbcommons.CONTAINER_LISTENER_PORT)) + "/" + n.Spec.Sid } - return m.Spec.CloneFrom + return m.Spec.PrimaryDatabaseRef }(), }, - { - Name: "ORACLE_HOSTNAME", - ValueFrom: &corev1.EnvVarSource{ - FieldRef: &corev1.ObjectFieldSelector{ - FieldPath: "status.podIP", - }, - }, - }, + CreateOracleHostnameEnvVarObj(m, n), { Name: "CLONE_DB", Value: "true", @@ -867,7 +964,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, } - } else if m.Spec.PrimaryDatabaseRef != "" && m.Spec.CreateAsStandby { + } else if m.Spec.CreateAs == "standby" { //Standby DB Usecase return []corev1.EnvVar{ { @@ -884,7 +981,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }, { Name: "WALLET_DIR", - Value: "/opt/oracle/oradata/dbconfig/$(ORACLE_SID)/.wallet", + Value: "/opt/oracle/oradata/dbconfig/${ORACLE_SID}/.wallet", }, { Name: "PRIMARY_DB_CONN_STR", @@ -959,7 +1056,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns if m.Spec.Image.PrebuiltDB { return "" // No wallets for prebuilt DB } - return "/opt/oracle/oradata/dbconfig/$(ORACLE_SID)/.wallet" + return "/opt/oracle/oradata/dbconfig/${ORACLE_SID}/.wallet" }(), }, { @@ -977,7 +1074,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns { Name: "INIT_SGA_SIZE", Value: func() string { - if m.Spec.InitParams.SgaTarget > 0 && m.Spec.InitParams.PgaAggregateTarget > 0 { + if m.Spec.InitParams != nil && m.Spec.InitParams.SgaTarget > 0 && m.Spec.InitParams.PgaAggregateTarget > 0 { return strconv.Itoa(m.Spec.InitParams.SgaTarget) } return "" @@ -986,7 +1083,7 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns { Name: "INIT_PGA_SIZE", Value: func() string { - if m.Spec.InitParams.SgaTarget > 0 && m.Spec.InitParams.PgaAggregateTarget > 0 { + if m.Spec.InitParams != nil && m.Spec.InitParams.SgaTarget > 0 && m.Spec.InitParams.PgaAggregateTarget > 0 { return strconv.Itoa(m.Spec.InitParams.SgaTarget) } return "" @@ -1032,11 +1129,12 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns Name: m.Spec.Image.PullSecrets, }, }, + ServiceAccountName: m.Spec.ServiceAccountName, }, } // Adding pod anti-affinity for standby cases - if m.Spec.PrimaryDatabaseRef != "" && m.Spec.CreateAsStandby { + if m.Spec.CreateAs == "standby" { weightedPodAffinityTerm := corev1.WeightedPodAffinityTerm{ Weight: 100, PodAffinityTerm: corev1.PodAffinityTerm{ @@ -1143,14 +1241,14 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePVCSpec(m *dbapi.SingleIns accessMode = append(accessMode, corev1.PersistentVolumeAccessMode(m.Spec.Persistence.AccessMode)) return accessMode }(), - Resources: corev1.ResourceRequirements{ + Resources: corev1.VolumeResourceRequirements{ Requests: map[corev1.ResourceName]resource.Quantity{ // Requests describes the minimum amount of compute resources required "storage": resource.MustParse(m.Spec.Persistence.Size), }, }, StorageClassName: &m.Spec.Persistence.StorageClass, - VolumeName: m.Spec.Persistence.VolumeName, + VolumeName: m.Spec.Persistence.DatafilesVolumeName, Selector: func() *metav1.LabelSelector { if m.Spec.Persistence.StorageClass != "oci" { return nil @@ -1176,34 +1274,152 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePVCSpec(m *dbapi.SingleIns // ############################################################################# // -// Stake a claim for Persistent Volume +// Stake a claim for Persistent Volume for customScript Volume +// +// ############################################################################# + +func (r *SingleInstanceDatabaseReconciler) createOrReplacePVCforCustomScriptsVol(ctx context.Context, req ctrl.Request, + m *dbapi.SingleInstanceDatabase) (ctrl.Result, error) { + + log := r.Log.WithValues("createPVC CustomScripts Vol", req.NamespacedName) + + // if customScriptsVolumeName is not present or it is same than DatafilesVolumeName + if m.Spec.Persistence.ScriptsVolumeName == "" || m.Spec.Persistence.ScriptsVolumeName == m.Spec.Persistence.DatafilesVolumeName { + return requeueN, nil + } + + pvcDeleted := false + pvcName := string(m.Name) + "-" + string(m.Spec.Persistence.ScriptsVolumeName) + // Check if the PVC already exists using r.Get, if not create a new one using r.Create + pvc := &corev1.PersistentVolumeClaim{} + // Get retrieves an obj ( a struct pointer ) for the given object key from the Kubernetes Cluster. + err := r.Get(ctx, types.NamespacedName{Name: pvcName, Namespace: m.Namespace}, pvc) + + if err == nil { + if m.Spec.Persistence.ScriptsVolumeName != "" && pvc.Spec.VolumeName != m.Spec.Persistence.ScriptsVolumeName { + // call deletePods() with zero pods in avaiable and nil readyPod to delete all pods + result, err := r.deletePods(ctx, req, m, []corev1.Pod{}, corev1.Pod{}, 0, 0) + if result.Requeue { + return result, err + } + + log.Info("Deleting PVC", " name ", pvc.Name) + err = r.Delete(ctx, pvc) + if err != nil { + r.Log.Error(err, "Failed to delete Pvc", "Pvc.Name", pvc.Name) + return requeueN, err + } + pvcDeleted = true + } else { + log.Info("Found Existing PVC", "Name", pvc.Name) + return requeueN, nil + } + } + if pvcDeleted || err != nil && apierrors.IsNotFound(err) { + // Define a new PVC + + // get accessMode and storage of pv mentioned to be used in pvc spec + pv := &corev1.PersistentVolume{} + pvName := m.Spec.Persistence.ScriptsVolumeName + // Get retrieves an obj ( a struct pointer ) for the given object key from the Kubernetes Cluster. + pvErr := r.Get(ctx, types.NamespacedName{Name: pvName, Namespace: m.Namespace}, pv) + if pvErr != nil { + log.Error(pvErr, "Failed to get PV") + return requeueY, pvErr + } + + volumeQty := pv.Spec.Capacity[corev1.ResourceStorage] + + AccessMode := pv.Spec.AccessModes[0] + Storage := int(volumeQty.Value()) + StorageClass := "" + + log.Info(fmt.Sprintf("PV storage: %v\n", Storage)) + log.Info(fmt.Sprintf("PV AccessMode: %v\n", AccessMode)) + + pvc := &corev1.PersistentVolumeClaim{ + TypeMeta: metav1.TypeMeta{ + Kind: "PersistentVolumeClaim", + }, + ObjectMeta: metav1.ObjectMeta{ + Name: pvcName, + Namespace: m.Namespace, + Labels: map[string]string{ + "app": m.Name, + }, + }, + Spec: corev1.PersistentVolumeClaimSpec{ + AccessModes: func() []corev1.PersistentVolumeAccessMode { + var accessMode []corev1.PersistentVolumeAccessMode + accessMode = append(accessMode, corev1.PersistentVolumeAccessMode(AccessMode)) + return accessMode + }(), + Resources: corev1.VolumeResourceRequirements{ + Requests: map[corev1.ResourceName]resource.Quantity{ + // Requests describes the minimum amount of compute resources required + "storage": *resource.NewQuantity(int64(Storage), resource.BinarySI), + }, + }, + StorageClassName: &StorageClass, + VolumeName: pvName, + }, + } + + // Set SingleInstanceDatabase instance as the owner and controller + ctrl.SetControllerReference(m, pvc, r.Scheme) + + log.Info("Creating a new PVC", "PVC.Namespace", pvc.Namespace, "PVC.Name", pvc.Name) + err = r.Create(ctx, pvc) + if err != nil { + log.Error(err, "Failed to create new PVC", "PVC.Namespace", pvc.Namespace, "PVC.Name", pvc.Name) + return requeueY, err + } + return requeueN, nil + } else if err != nil { + log.Error(err, "Failed to get PVC") + return requeueY, err + } + + return requeueN, nil +} + +// ############################################################################# +// +// Stake a claim for Persistent Volume for Datafiles Volume // // ############################################################################# -func (r *SingleInstanceDatabaseReconciler) createOrReplacePVC(ctx context.Context, req ctrl.Request, +func (r *SingleInstanceDatabaseReconciler) createOrReplacePVCforDatafilesVol(ctx context.Context, req ctrl.Request, m *dbapi.SingleInstanceDatabase) (ctrl.Result, error) { + log := r.Log.WithValues("createPVC Datafiles-Vol", req.NamespacedName) + // Don't create PVC if persistence is not chosen if m.Spec.Persistence.Size == "" { return requeueN, nil } - log := r.Log.WithValues("createPVC", req.NamespacedName) pvcDeleted := false // Check if the PVC already exists using r.Get, if not create a new one using r.Create pvc := &corev1.PersistentVolumeClaim{} // Get retrieves an obj ( a struct pointer ) for the given object key from the Kubernetes Cluster. err := r.Get(ctx, types.NamespacedName{Name: m.Name, Namespace: m.Namespace}, pvc) + if err == nil { if *pvc.Spec.StorageClassName != m.Spec.Persistence.StorageClass || - pvc.Spec.Resources.Requests["storage"] != resource.MustParse(m.Spec.Persistence.Size) || - (m.Spec.Persistence.VolumeName != "" && pvc.Spec.VolumeName != m.Spec.Persistence.VolumeName) || + (m.Spec.Persistence.DatafilesVolumeName != "" && pvc.Spec.VolumeName != m.Spec.Persistence.DatafilesVolumeName) || pvc.Spec.AccessModes[0] != corev1.PersistentVolumeAccessMode(m.Spec.Persistence.AccessMode) { - // call deletePods() with zero pods in avaiable and nil readyPod to delete all pods + // PV change use cases which would trigger recreation of SIDB pods are :- + // 1. Change in storage class + // 2. Change in volume name + // 3. Change in volume access mode + + // deleting singleinstancedatabase resource result, err := r.deletePods(ctx, req, m, []corev1.Pod{}, corev1.Pod{}, 0, 0) if result.Requeue { return result, err } + // deleting persistent volume claim log.Info("Deleting PVC", " name ", pvc.Name) err = r.Delete(ctx, pvc) if err != nil { @@ -1211,11 +1427,52 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplacePVC(ctx context.Contex return requeueN, err } pvcDeleted = true + + } else if pvc.Spec.Resources.Requests["storage"] != resource.MustParse(m.Spec.Persistence.Size) { + // check the storage class of the pvc + // if the storage class doesn't support resize the throw an error event and try expanding via deleting and recreating the pv and pods + if pvc.Spec.StorageClassName == nil || *pvc.Spec.StorageClassName == "" { + r.Recorder.Eventf(m, corev1.EventTypeWarning, "PVC not resizable", "Cannot resize pvc as storage class is either nil or default") + return requeueN, fmt.Errorf("cannot resize pvc as storage class is either nil or default") + } + + storageClassName := *pvc.Spec.StorageClassName + storageClass := &storagev1.StorageClass{} + err := r.Get(ctx, types.NamespacedName{Name: storageClassName}, storageClass) + if err != nil { + return requeueY, fmt.Errorf("error while fetching the storage class") + } + + if storageClass.AllowVolumeExpansion == nil || !*storageClass.AllowVolumeExpansion { + r.Recorder.Eventf(m, corev1.EventTypeWarning, "PVC not resizable", "The storage class doesn't support volume expansion") + return requeueN, fmt.Errorf("the storage class %s doesn't support volume expansion", storageClassName) + } + + newPVCSize := resource.MustParse(m.Spec.Persistence.Size) + newPVCSizeAdd := &newPVCSize + if newPVCSizeAdd.Cmp(pvc.Spec.Resources.Requests["storage"]) < 0 { + r.Recorder.Eventf(m, corev1.EventTypeWarning, "Cannot Resize PVC", "Forbidden: field can not be less than previous value") + return requeueN, fmt.Errorf("Resizing PVC to lower size volume not allowed") + } + + // Expanding the persistent volume claim + pvc.Spec.Resources.Requests["storage"] = resource.MustParse(m.Spec.Persistence.Size) + log.Info("Updating PVC", "pvc", pvc.Name, "volume", pvc.Spec.VolumeName) + r.Recorder.Eventf(m, corev1.EventTypeNormal, "Updating PVC - volume expansion", "Resizing the pvc for storage expansion") + err = r.Update(ctx, pvc) + if err != nil { + log.Error(err, "Error while updating the PVCs") + return requeueY, fmt.Errorf("error while updating the PVCs") + } + } else { + log.Info("Found Existing PVC", "Name", pvc.Name) return requeueN, nil + } } + if pvcDeleted || err != nil && apierrors.IsNotFound(err) { // Define a new PVC pvc = r.instantiatePVCSpec(m) @@ -1501,6 +1758,9 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex r.Log.Info("Initiliazing database sid, pdb, edition for prebuilt database") var edition string sid, pdbName, edition, getSidPdbEditionErr = dbcommons.GetSidPdbEdition(r, r.Config, ctx, ctrl.Request{NamespacedName: types.NamespacedName{Namespace: m.Namespace, Name: m.Name}}) + if errors.Is(getSidPdbEditionErr, dbcommons.ErrNoReadyPod) { + return requeueN, nil + } if getSidPdbEditionErr != nil { return requeueY, getSidPdbEditionErr } @@ -1800,7 +2060,7 @@ func (r *SingleInstanceDatabaseReconciler) createWallet(m *dbapi.SingleInstanceD return requeueY, nil } - if m.Spec.CloneFrom == "" && m.Spec.Edition != "express" { + if m.Spec.CreateAs != "clone" && m.Spec.Edition != "express" { //Check if Edition of m.Spec.Sid is same as m.Spec.Edition getEditionFile := dbcommons.GetEnterpriseEditionFileCMD eventReason := m.Spec.Sid + " is a enterprise edition" @@ -1984,107 +2244,96 @@ func (r *SingleInstanceDatabaseReconciler) deletePods(ctx context.Context, req c // ValidateDBReadiness and return the ready POD // // ############################################################################# -func (r *SingleInstanceDatabaseReconciler) validateDBReadiness(m *dbapi.SingleInstanceDatabase, +func (r *SingleInstanceDatabaseReconciler) validateDBReadiness(sidb *dbapi.SingleInstanceDatabase, ctx context.Context, req ctrl.Request) (ctrl.Result, corev1.Pod, error) { - readyPod, _, available, _, err := dbcommons.FindPods(r, m.Spec.Image.Version, - m.Spec.Image.PullFrom, m.Name, m.Namespace, ctx, req) + log := r.Log.WithValues("validateDBReadiness", req.NamespacedName) + + log.Info("Validating readiness for database") + + sidbReadyPod, _, available, _, err := dbcommons.FindPods(r, sidb.Spec.Image.Version, + sidb.Spec.Image.PullFrom, sidb.Name, sidb.Namespace, ctx, req) if err != nil { r.Log.Error(err, err.Error()) - return requeueY, readyPod, err + return requeueY, sidbReadyPod, err } - if readyPod.Name == "" { - m.Status.Status = dbcommons.StatusPending + + if sidbReadyPod.Name == "" { + sidb.Status.Status = dbcommons.StatusPending + log.Info("no pod currently in ready state") if ok, _ := dbcommons.IsAnyPodWithStatus(available, corev1.PodFailed); ok { eventReason := "Database Failed" eventMsg := "pod creation failed" - r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) - } else if ok, runningPod := dbcommons.IsAnyPodWithStatus(available, corev1.PodRunning); ok { - r.Log.Info("Database Creating...", "Name", m.Name) - m.Status.Status = dbcommons.StatusCreating - if m.Spec.CloneFrom != "" { - // Required since clone creates the datafiles under primary database SID folder - r.Log.Info("Creating the SID directory link for clone database", "name", m.Spec.Sid) - _, err := dbcommons.ExecCommand(r, r.Config, runningPod.Name, runningPod.Namespace, "", - ctx, req, false, "bash", "-c", dbcommons.CreateSIDlinkCMD) - if err != nil { - r.Log.Info(err.Error()) - } - } - out, err := dbcommons.ExecCommand(r, r.Config, runningPod.Name, runningPod.Namespace, "", + r.Recorder.Eventf(sidb, corev1.EventTypeNormal, eventReason, eventMsg) + } else if ok, _ := dbcommons.IsAnyPodWithStatus(available, corev1.PodRunning); ok { + + out, err := dbcommons.ExecCommand(r, r.Config, available[0].Name, sidb.Namespace, "", ctx, req, false, "bash", "-c", dbcommons.GetCheckpointFileCMD) if err != nil { r.Log.Info(err.Error()) } - r.Log.Info("GetCheckpointFileCMD Output : \n" + out) if out != "" { + log.Info("Database initialzied") eventReason := "Database Unhealthy" eventMsg := "datafiles exists" - r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) - m.Status.DatafilesCreated = "true" - m.Status.Status = dbcommons.StatusNotReady - r.updateORDSStatus(m, ctx, req) + r.Recorder.Eventf(sidb, corev1.EventTypeNormal, eventReason, eventMsg) + sidb.Status.DatafilesCreated = "true" + sidb.Status.Status = dbcommons.StatusNotReady + r.updateORDSStatus(sidb, ctx, req) + } else { + log.Info("Database Creating....", "Name", sidb.Name) + sidb.Status.Status = dbcommons.StatusCreating } } else { - r.Log.Info("Database Pending...", "Name", m.Name) + log.Info("Database Pending....", "Name", sidb.Name) } - - // As No pod is ready now , turn on mode when pod is ready . so requeue the request - return requeueY, readyPod, errors.New("no pod is ready currently") + log.Info("no pod currently in ready state") + return requeueY, sidbReadyPod, nil } - available = append(available, readyPod) - podNamesFinal := dbcommons.GetPodNames(available) - r.Log.Info("Final "+m.Name+" Pods After Deleting (or) Adding Extra Pods ( Including The Ready Pod ) ", "Pod Names", podNamesFinal) - r.Log.Info(m.Name+" Replicas Available", "Count", len(podNamesFinal)) - r.Log.Info(m.Name+" Replicas Required", "Count", m.Spec.Replicas) - - eventReason := "Database Ready" - eventMsg := "database open on pod " + readyPod.Name + " scheduled on node " + readyPod.Status.HostIP - r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) - - m.Status.DatafilesCreated = "true" - - // DB is ready, fetch and update other info - out, err := dbcommons.GetDatabaseRole(readyPod, r, r.Config, ctx, req, m.Spec.Edition) - if err == nil { - m.Status.Role = strings.ToUpper(out) - } - version, out, err := dbcommons.GetDatabaseVersion(readyPod, r, r.Config, ctx, req, m.Spec.Edition) - if err == nil { - if !strings.Contains(out, "ORA-") { - m.Status.ReleaseUpdate = version + if sidb.Spec.CreateAs == "clone" { + // Required since clone creates the datafiles under primary database SID folder + r.Log.Info("Creating the SID directory link for clone database", "name", sidb.Spec.Sid) + _, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", + ctx, req, false, "bash", "-c", dbcommons.CreateSIDlinkCMD) + if err != nil { + r.Log.Info(err.Error()) } } - dbMajorVersion, err := strconv.Atoi(strings.Split(m.Status.ReleaseUpdate, ".")[0]) + + version, err := dbcommons.GetDatabaseVersion(sidbReadyPod, r, r.Config, ctx, req) + if err != nil { + return requeueY, sidbReadyPod, err + } + dbMajorVersion, err := strconv.Atoi(strings.Split(version, ".")[0]) if err != nil { r.Log.Error(err, err.Error()) - return requeueY, readyPod, err + return requeueY, sidbReadyPod, err } r.Log.Info("DB Major Version is " + strconv.Itoa(dbMajorVersion)) // Validating that free edition of the database is only supported from database 23c onwards - if m.Spec.Edition == "free" && dbMajorVersion < 23 { + if sidb.Spec.Edition == "free" && dbMajorVersion < 23 { errMsg := "the Oracle Database Free is only available from version 23c onwards" - r.Recorder.Eventf(m, corev1.EventTypeWarning, "Spec Error", errMsg) - m.Status.Status = dbcommons.StatusError - return requeueN, readyPod, errors.New(errMsg) - } - // Checking if OEM is supported in the provided Database version - if dbMajorVersion >= 23 { - m.Status.OemExpressUrl = dbcommons.ValueUnavailable - } else { - m.Status.OemExpressUrl = oemExpressUrl + r.Recorder.Eventf(sidb, corev1.EventTypeWarning, "Spec Error", errMsg) + sidb.Status.Status = dbcommons.StatusError + return requeueY, sidbReadyPod, errors.New(errMsg) } - if strings.ToUpper(m.Status.Role) == "PRIMARY" && m.Status.DatafilesPatched != "true" { - eventReason := "Datapatch Pending" - eventMsg := "datapatch execution pending" - r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) - } + available = append(available, sidbReadyPod) + podNamesFinal := dbcommons.GetPodNames(available) + r.Log.Info("Final "+sidb.Name+" Pods After Deleting (or) Adding Extra Pods ( Including The Ready Pod ) ", "Pod Names", podNamesFinal) + r.Log.Info(sidb.Name+" Replicas Available", "Count", len(podNamesFinal)) + r.Log.Info(sidb.Name+" Replicas Required", "Count", sidb.Spec.Replicas) + + eventReason := "Database Ready" + eventMsg := "database open on pod " + sidbReadyPod.Name + " scheduled on node " + sidbReadyPod.Status.HostIP + r.Recorder.Eventf(sidb, corev1.EventTypeNormal, eventReason, eventMsg) - return requeueN, readyPod, nil + sidb.Status.CreatedAs = sidb.Spec.CreateAs + + return requeueN, sidbReadyPod, nil } @@ -2191,20 +2440,56 @@ func (r *SingleInstanceDatabaseReconciler) updateClientWallet(m *dbapi.SingleIns func (r *SingleInstanceDatabaseReconciler) configTcps(m *dbapi.SingleInstanceDatabase, readyPod corev1.Pod, ctx context.Context, req ctrl.Request) (ctrl.Result, error) { eventReason := "Configuring TCPS" - if m.Spec.EnableTCPS && !m.Status.IsTcpsEnabled { - // Enable TCPS - m.Status.Status = dbcommons.StatusUpdating + + if (m.Spec.EnableTCPS) && + ((!m.Status.IsTcpsEnabled) || // TCPS Enabled from a TCP state + (m.Spec.TcpsTlsSecret != "" && m.Status.TcpsTlsSecret == "") || // TCPS Secret is added in spec + (m.Spec.TcpsTlsSecret == "" && m.Status.TcpsTlsSecret != "") || // TCPS Secret is removed in spec + (m.Spec.TcpsTlsSecret != "" && m.Status.TcpsTlsSecret != "" && m.Spec.TcpsTlsSecret != m.Status.TcpsTlsSecret)) { //TCPS secret is changed + + // Set status to Updating, except when an error has been thrown from configTCPS script + if m.Status.Status != dbcommons.StatusError { + m.Status.Status = dbcommons.StatusUpdating + } r.Status().Update(ctx, m) eventMsg := "Enabling TCPS in the database..." r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + var TcpsCommand = dbcommons.EnableTcpsCMD + if m.Spec.TcpsTlsSecret != "" { // case when tls secret is either added or changed + TcpsCommand = "export TCPS_CERTS_LOCATION=" + dbcommons.TlsCertsLocation + " && " + dbcommons.EnableTcpsCMD + + // Checking for tls-secret mount in pods + out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", + ctx, req, false, "bash", "-c", fmt.Sprintf(dbcommons.PodMountsCmd, dbcommons.TlsCertsLocation)) + r.Log.Info("Mount Check Output") + r.Log.Info(out) + if err != nil { + r.Log.Error(err, err.Error()) + return requeueY, nil + } + + if (m.Status.TcpsTlsSecret != "") || // case when TCPS Secret is changed + (!strings.Contains(out, dbcommons.TlsCertsLocation)) { // if mount is not there in pod + // call deletePods() with zero pods in avaiable and nil readyPod to delete all pods + result, err := r.deletePods(ctx, req, m, []corev1.Pod{}, corev1.Pod{}, 0, 0) + if result.Requeue { + return result, err + } + m.Status.TcpsTlsSecret = "" // to avoid reconciled pod deletions, in case of TCPS secret change and it fails + } + } + + // Enable TCPS out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", - ctx, req, false, "bash", "-c", dbcommons.EnableTcpsCMD) + ctx, req, false, "bash", "-c", TcpsCommand) if err != nil { r.Log.Error(err, err.Error()) eventMsg = "Error encountered in enabling TCPS!" r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + m.Status.Status = dbcommons.StatusError + r.Status().Update(ctx, m) return requeueY, nil } r.Log.Info("enableTcps Output : \n" + out) @@ -2212,6 +2497,14 @@ func (r *SingleInstanceDatabaseReconciler) configTcps(m *dbapi.SingleInstanceDat m.Status.CertCreationTimestamp = time.Now().Format(time.RFC3339) m.Status.IsTcpsEnabled = true m.Status.ClientWalletLoc = fmt.Sprintf(dbcommons.ClientWalletLocation, m.Spec.Sid) + // m.Spec.TcpsTlsSecret can be empty or non-empty + // Store secret name in case of tls-secret addition or change, otherwise would be "" + if m.Spec.TcpsTlsSecret != "" { + m.Status.TcpsTlsSecret = m.Spec.TcpsTlsSecret + } else { + m.Status.TcpsTlsSecret = "" + } + r.Status().Update(ctx, m) eventMsg = "TCPS Enabled." @@ -2246,6 +2539,8 @@ func (r *SingleInstanceDatabaseReconciler) configTcps(m *dbapi.SingleInstanceDat m.Status.CertCreationTimestamp = "" m.Status.IsTcpsEnabled = false m.Status.ClientWalletLoc = "" + m.Status.TcpsTlsSecret = "" + r.Status().Update(ctx, m) eventMsg = "TCPS Disabled." @@ -2376,26 +2671,49 @@ func (r *SingleInstanceDatabaseReconciler) updateInitParameters(m *dbapi.SingleI readyPod corev1.Pod, ctx context.Context, req ctrl.Request) (ctrl.Result, error) { log := r.Log.WithValues("updateInitParameters", req.NamespacedName) - if m.Status.InitParams == m.Spec.InitParams { + if m.Spec.InitParams == nil { + return requeueN, nil + } + if m.Status.InitParams == *m.Spec.InitParams { return requeueN, nil } - out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", - ctx, req, false, "bash", "-c", fmt.Sprintf(dbcommons.AlterSgaPgaCpuCMD, m.Spec.InitParams.SgaTarget, - m.Spec.InitParams.PgaAggregateTarget, m.Spec.InitParams.CpuCount, dbcommons.SQLPlusCLI)) - if err != nil { - log.Error(err, err.Error()) - return requeueY, err + if (m.Spec.InitParams.PgaAggregateTarget != 0 && (m.Spec.InitParams.PgaAggregateTarget != m.Status.InitParams.PgaAggregateTarget)) || (m.Spec.InitParams.SgaTarget != 0 && (m.Spec.InitParams.SgaTarget != m.Status.InitParams.SgaTarget)) { + log.Info("Executing alter sga pga command", "pga_size", m.Spec.InitParams.PgaAggregateTarget, "sga_size", m.Spec.InitParams.SgaTarget) + out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", + ctx, req, false, "bash", "-c", fmt.Sprintf(dbcommons.AlterSgaPgaCMD, m.Spec.InitParams.SgaTarget, + m.Spec.InitParams.PgaAggregateTarget, dbcommons.SQLPlusCLI)) + if err != nil { + log.Error(err, err.Error()) + return requeueY, err + } + // Notify the user about unsucessfull init-parameter value change + if strings.Contains(out, "ORA-") { + eventReason := "Invalid init-param value" + eventMsg := "Unable to change the init-param as specified. Error log: \n" + out + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) + } + log.Info("AlterSgaPgaCpuCMD Output:" + out) } - // Notify the user about unsucessfull init-parameter value change - if strings.Contains(out, "ORA-") { - eventReason := "Invalid init-param value" - eventMsg := "Unable to change the init-param as specified. Error log: \n" + out - r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) + + if (m.Spec.InitParams.CpuCount != 0) && (m.Status.InitParams.CpuCount != m.Spec.InitParams.CpuCount) { + log.Info("Executing alter cpu count command", "cpuCount", m.Spec.InitParams.CpuCount) + out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, + "bash", "-c", fmt.Sprintf(dbcommons.AlterCpuCountCMD, m.Spec.InitParams.CpuCount, dbcommons.SQLPlusCLI)) + if err != nil { + log.Error(err, err.Error()) + return requeueY, err + } + if strings.Contains(out, "ORA-") { + eventReason := "Invalid init-param value" + eventMsg := "Unable to change the init-param as specified. Error log: \n" + out + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) + } + log.Info("AlterCpuCountCMD Output:" + out) } - log.Info("AlterSgaPgaCpuCMD Output:" + out) - if m.Status.InitParams.Processes != m.Spec.InitParams.Processes { + if (m.Spec.InitParams.Processes != 0) && (m.Status.InitParams.Processes != m.Spec.InitParams.Processes) { + log.Info("Executing alter processes command", "processes", m.Spec.InitParams.Processes) // Altering 'Processes' needs database to be restarted out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", fmt.Sprintf(dbcommons.AlterProcessesCMD, m.Spec.InitParams.Processes, dbcommons.SQLPlusCLI, @@ -2406,16 +2724,6 @@ func (r *SingleInstanceDatabaseReconciler) updateInitParameters(m *dbapi.SingleI } log.Info("AlterProcessesCMD Output:" + out) } - - out, err = dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", - ctx, req, false, "bash", "-c", fmt.Sprintf(dbcommons.GetInitParamsSQL, dbcommons.SQLPlusCLI)) - if err != nil { - log.Error(err, err.Error()) - return requeueY, err - } - log.Info("GetInitParamsSQL Output:" + out) - - m.Status.InitParams = m.Spec.InitParams return requeueN, nil } @@ -2445,19 +2753,12 @@ func (r *SingleInstanceDatabaseReconciler) updateDBConfig(m *dbapi.SingleInstanc m.Status.Status = dbcommons.StatusNotReady return result, nil } - m.Status.ArchiveLog = strconv.FormatBool(archiveLogStatus) - m.Status.ForceLogging = strconv.FormatBool(forceLoggingStatus) - m.Status.FlashBack = strconv.FormatBool(flashBackStatus) - - log.Info("Flashback", "Status :", flashBackStatus) - log.Info("ArchiveLog", "Status :", archiveLogStatus) - log.Info("ForceLog", "Status :", forceLoggingStatus) //################################################################################################# // TURNING FLASHBACK , ARCHIVELOG , FORCELOGGING TO TRUE //################################################################################################# - if m.Spec.ArchiveLog && !archiveLogStatus { + if m.Spec.ArchiveLog != nil && *m.Spec.ArchiveLog && !archiveLogStatus { out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", dbcommons.CreateDBRecoveryDestCMD) @@ -2488,7 +2789,7 @@ func (r *SingleInstanceDatabaseReconciler) updateDBConfig(m *dbapi.SingleInstanc } - if m.Spec.ForceLogging && !forceLoggingStatus { + if m.Spec.ForceLogging != nil && *m.Spec.ForceLogging && !forceLoggingStatus { out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", fmt.Sprintf("echo -e \"%s\" | %s", dbcommons.ForceLoggingTrueSQL, dbcommons.SQLPlusCLI)) if err != nil { @@ -2499,7 +2800,7 @@ func (r *SingleInstanceDatabaseReconciler) updateDBConfig(m *dbapi.SingleInstanc log.Info(out) } - if m.Spec.FlashBack && !flashBackStatus { + if m.Spec.FlashBack != nil && *m.Spec.FlashBack && !flashBackStatus { _, archiveLogStatus, _, result := dbcommons.CheckDBConfig(readyPod, r, r.Config, ctx, req, m.Spec.Edition) if result.Requeue { m.Status.Status = dbcommons.StatusNotReady @@ -2531,7 +2832,7 @@ func (r *SingleInstanceDatabaseReconciler) updateDBConfig(m *dbapi.SingleInstanc // TURNING FLASHBACK , ARCHIVELOG , FORCELOGGING TO FALSE //################################################################################################# - if !m.Spec.FlashBack && flashBackStatus { + if m.Spec.FlashBack != nil && !*m.Spec.FlashBack && flashBackStatus { out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", fmt.Sprintf("echo -e \"%s\" | %s", dbcommons.FlashBackFalseSQL, dbcommons.SQLPlusCLI)) if err != nil { @@ -2541,7 +2842,7 @@ func (r *SingleInstanceDatabaseReconciler) updateDBConfig(m *dbapi.SingleInstanc log.Info("FlashBackFalse Output") log.Info(out) } - if !m.Spec.ArchiveLog && archiveLogStatus { + if m.Spec.ArchiveLog != nil && !*m.Spec.ArchiveLog && archiveLogStatus { flashBackStatus, _, _, result := dbcommons.CheckDBConfig(readyPod, r, r.Config, ctx, req, m.Spec.Edition) if result.Requeue { m.Status.Status = dbcommons.StatusNotReady @@ -2569,7 +2870,7 @@ func (r *SingleInstanceDatabaseReconciler) updateDBConfig(m *dbapi.SingleInstanc changeArchiveLog = true } } - if !m.Spec.ForceLogging && forceLoggingStatus { + if m.Spec.ForceLogging != nil && !*m.Spec.ForceLogging && forceLoggingStatus { out, err := dbcommons.ExecCommand(r, r.Config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", fmt.Sprintf("echo -e \"%s\" | %s", dbcommons.ForceLoggingFalseSQL, dbcommons.SQLPlusCLI)) if err != nil { @@ -2601,26 +2902,112 @@ func (r *SingleInstanceDatabaseReconciler) updateDBConfig(m *dbapi.SingleInstanc // Needs to restart the Non Ready Pods ( Delete old ones and create new ones ) if m.Status.FlashBack == strconv.FormatBool(false) && flashBackStatus { - // call FindPods() to fetch pods all version/images of the same SIDB kind + // // call FindPods() to fetch pods all version/images of the same SIDB kind readyPod, replicasFound, available, _, err := dbcommons.FindPods(r, "", "", m.Name, m.Namespace, ctx, req) if err != nil { log.Error(err, err.Error()) return requeueY, err } - // delete non ready Pods as flashback needs restart of pods + // delete non ready Pods as flashback needs restart of pods to make sure failover works in sidbs with multiple replicas _, err = r.deletePods(ctx, req, m, available, readyPod, replicasFound, 1) - return requeueY, err + if err != nil { + log.Error(err, err.Error()) + return requeueY, err + } + return requeueN, err } m.Status.FlashBack = strconv.FormatBool(flashBackStatus) - if !changeArchiveLog && (flashBackStatus != m.Spec.FlashBack || - archiveLogStatus != m.Spec.ArchiveLog || forceLoggingStatus != m.Spec.ForceLogging) { + if !changeArchiveLog && ((m.Spec.FlashBack != nil && (flashBackStatus != *m.Spec.FlashBack)) || + (m.Spec.ArchiveLog != nil && (archiveLogStatus != *m.Spec.ArchiveLog)) || (m.Spec.ForceLogging != nil && (forceLoggingStatus != *m.Spec.ForceLogging))) { return requeueY, nil } return requeueN, nil } +// ############################################################################# +// +// # Update Single instance database resource status +// +// ############################################################################# +func (r *SingleInstanceDatabaseReconciler) updateSidbStatus(sidb *dbapi.SingleInstanceDatabase, sidbReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) error { + + log := r.Log.WithValues("updateSidbStatus", req.NamespacedName) + + flashBackStatus, archiveLogStatus, forceLoggingStatus, result := dbcommons.CheckDBConfig(sidbReadyPod, r, r.Config, ctx, req, sidb.Spec.Edition) + if result.Requeue { + sidb.Status.Status = dbcommons.StatusNotReady + return fmt.Errorf("could not check the database conifg of %s", sidb.Name) + } + + log.Info("flashBack", "Status :", flashBackStatus, "Reconcile Step : ", "updateSidbStatus") + log.Info("ArchiveLog", "Status :", archiveLogStatus, "Reconcile Step : ", "updateSidbStatus") + log.Info("forceLogging", "Status :", forceLoggingStatus, "Reconcile Step : ", "updateSidbStatus") + + sidb.Status.ArchiveLog = strconv.FormatBool(archiveLogStatus) + sidb.Status.ForceLogging = strconv.FormatBool(forceLoggingStatus) + sidb.Status.FlashBack = strconv.FormatBool(flashBackStatus) + + cpu_count, pga_aggregate_target, processes, sga_target, err := dbcommons.CheckDBInitParams(sidbReadyPod, r, r.Config, ctx, req) + if err != nil { + return err + } + sidbInitParams := dbapi.SingleInstanceDatabaseInitParams{ + SgaTarget: sga_target, + PgaAggregateTarget: pga_aggregate_target, + Processes: processes, + CpuCount: cpu_count, + } + // log.Info("GetInitParamsSQL Output:" + out) + + sidb.Status.InitParams = sidbInitParams + // sidb.Status.InitParams = sidb.Spec.InitParams + + // Get database role and update the status + sidbRole, err := dbcommons.GetDatabaseRole(sidbReadyPod, r, r.Config, ctx, req) + if err != nil { + return err + } + log.Info("Database "+sidb.Name, "Database Role : ", sidbRole) + sidb.Status.Role = sidbRole + + // Get database version and update the status + version, err := dbcommons.GetDatabaseVersion(sidbReadyPod, r, r.Config, ctx, req) + if err != nil { + return err + } + log.Info("Database "+sidb.Name, "Database Version : ", version) + sidb.Status.ReleaseUpdate = version + + dbMajorVersion, err := strconv.Atoi(strings.Split(sidb.Status.ReleaseUpdate, ".")[0]) + if err != nil { + r.Log.Error(err, err.Error()) + return err + } + log.Info("Database "+sidb.Name, "Database Major Version : ", dbMajorVersion) + + // Checking if OEM is supported in the provided Database version + if dbMajorVersion >= 23 { + sidb.Status.OemExpressUrl = dbcommons.ValueUnavailable + } else { + sidb.Status.OemExpressUrl = oemExpressUrl + } + + if sidb.Status.Role == "PRIMARY" && sidb.Status.DatafilesPatched != "true" { + eventReason := "Datapatch Pending" + eventMsg := "datapatch execution pending" + r.Recorder.Eventf(sidb, corev1.EventTypeNormal, eventReason, eventMsg) + } + + // update status to Ready after all operations succeed + sidb.Status.Status = dbcommons.StatusReady + + r.Status().Update(ctx, sidb) + + return nil +} + // ############################################################################# // // Update ORDS Status @@ -2758,8 +3145,11 @@ func (r *SingleInstanceDatabaseReconciler) SetupWithManager(mgr ctrl.Manager) er Complete(r) } -// StandbyDatabase functions -// Primary DB Validation +// ############################################################################# +// +// Check primary database status +// +// ############################################################################# func CheckPrimaryDatabaseStatus(p *dbapi.SingleInstanceDatabase) error { if p.Status.Status != dbcommons.StatusReady { @@ -2768,6 +3158,11 @@ func CheckPrimaryDatabaseStatus(p *dbapi.SingleInstanceDatabase) error { return nil } +// ############################################################################# +// +// Check if refered database is the primary database +// +// ############################################################################# func CheckDatabaseRoleAsPrimary(p *dbapi.SingleInstanceDatabase) error { if strings.ToUpper(p.Status.Role) != "PRIMARY" { @@ -2776,6 +3171,11 @@ func CheckDatabaseRoleAsPrimary(p *dbapi.SingleInstanceDatabase) error { return nil } +// ############################################################################# +// +// Get ready pod for the singleinstancedatabase resource +// +// ############################################################################# func GetDatabaseReadyPod(r client.Reader, d *dbapi.SingleInstanceDatabase, ctx context.Context, req ctrl.Request) (corev1.Pod, error) { dbReadyPod, _, _, _, err := dbcommons.FindPods(r, d.Spec.Image.Version, @@ -2784,6 +3184,11 @@ func GetDatabaseReadyPod(r client.Reader, d *dbapi.SingleInstanceDatabase, ctx c return dbReadyPod, err } +// ############################################################################# +// +// Get admin password for singleinstancedatabase +// +// ############################################################################# func GetDatabaseAdminPassword(r client.Reader, d *dbapi.SingleInstanceDatabase, ctx context.Context) (string, error) { adminPasswordSecret := &corev1.Secret{} @@ -2796,6 +3201,11 @@ func GetDatabaseAdminPassword(r client.Reader, d *dbapi.SingleInstanceDatabase, return adminPassword, nil } +// ############################################################################# +// +// Validate primary singleinstancedatabase admin password +// +// ############################################################################# func ValidatePrimaryDatabaseAdminPassword(r *SingleInstanceDatabaseReconciler, p *dbapi.SingleInstanceDatabase, adminPassword string, ctx context.Context, req ctrl.Request) error { @@ -2822,13 +3232,33 @@ func ValidatePrimaryDatabaseAdminPassword(r *SingleInstanceDatabaseReconciler, p return nil } +// ############################################################################# +// +// Validate refered primary database db params are all enabled +// +// ############################################################################# func ValidateDatabaseConfiguration(p *dbapi.SingleInstanceDatabase) error { + var missingModes []string + if p.Status.ArchiveLog == "false" { + missingModes = append(missingModes, "ArchiveLog") + } + if p.Status.FlashBack == "false" { + missingModes = append(missingModes, "FlashBack") + } + if p.Status.ForceLogging == "false" { + missingModes = append(missingModes, "ForceLogging") + } if p.Status.ArchiveLog == "false" || p.Status.FlashBack == "false" || p.Status.ForceLogging == "false" { - return fmt.Errorf("all of Archivelog, Flashback and ForceLogging modes are not enabled in the primary database %v", p.Name) + return fmt.Errorf("%v modes are not enabled in the primary database %v", strings.Join(missingModes, ","), p.Name) } return nil } +// ############################################################################# +// +// Validate refered primary database for standby sidb creation +// +// ############################################################################# func ValidatePrimaryDatabaseForStandbyCreation(r *SingleInstanceDatabaseReconciler, stdby *dbapi.SingleInstanceDatabase, primary *dbapi.SingleInstanceDatabase, ctx context.Context, req ctrl.Request) error { @@ -2870,7 +3300,7 @@ func ValidatePrimaryDatabaseForStandbyCreation(r *SingleInstanceDatabaseReconcil log.Info(fmt.Sprintf("Validating primary database %s configuration...", primary.Name)) err = ValidateDatabaseConfiguration(primary) if err != nil { - r.Recorder.Eventf(stdby, corev1.EventTypeWarning, "Spec Error", "all of Archivelog, Flashback and ForceLogging modes are not enabled in the primary database "+primary.Name) + r.Recorder.Eventf(stdby, corev1.EventTypeWarning, "Spec Error", err.Error()) stdby.Status.Status = dbcommons.StatusError return err } @@ -2880,7 +3310,11 @@ func ValidatePrimaryDatabaseForStandbyCreation(r *SingleInstanceDatabaseReconcil return nil } -// Primary DB Setup +// ############################################################################# +// +// Get total database pods for singleinstancedatabase +// +// ############################################################################# func GetTotalDatabasePods(r client.Reader, d *dbapi.SingleInstanceDatabase, ctx context.Context, req ctrl.Request) (int, error) { _, totalPods, _, _, err := dbcommons.FindPods(r, d.Spec.Image.Version, d.Spec.Image.PullFrom, d.Name, d.Namespace, ctx, req) @@ -2888,6 +3322,11 @@ func GetTotalDatabasePods(r client.Reader, d *dbapi.SingleInstanceDatabase, ctx return totalPods, err } +// ############################################################################# +// +// Set tns names for primary database for dataguard configuraion +// +// ############################################################################# func SetupTnsNamesPrimaryForDG(r *SingleInstanceDatabaseReconciler, p *dbapi.SingleInstanceDatabase, s *dbapi.SingleInstanceDatabase, primaryReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) error { @@ -2918,6 +3357,11 @@ func SetupTnsNamesPrimaryForDG(r *SingleInstanceDatabaseReconciler, p *dbapi.Sin return nil } +// ############################################################################# +// +// Restarting listners in database +// +// ############################################################################# func RestartListenerInDatabase(r *SingleInstanceDatabaseReconciler, primaryReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) error { r.Log.Info("Restarting listener in the database through pod", "primary database pod name", primaryReadyPod.Name) out, err := dbcommons.ExecCommand(r, r.Config, primaryReadyPod.Name, primaryReadyPod.Namespace, "", @@ -2930,6 +3374,11 @@ func RestartListenerInDatabase(r *SingleInstanceDatabaseReconciler, primaryReady return nil } +// ############################################################################# +// +// Setup primary listener for dataguard configuration +// +// ############################################################################# func SetupListenerPrimaryForDG(r *SingleInstanceDatabaseReconciler, p *dbapi.SingleInstanceDatabase, s *dbapi.SingleInstanceDatabase, primaryReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) error { @@ -2961,10 +3410,15 @@ func SetupListenerPrimaryForDG(r *SingleInstanceDatabaseReconciler, p *dbapi.Sin return nil } +// ############################################################################# +// +// Setup init parameters of primary database for dataguard configuration +// +// ############################################################################# func SetupInitParamsPrimaryForDG(r *SingleInstanceDatabaseReconciler, primaryReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) error { r.Log.Info("Running StandbyDatabasePrerequisitesSQL in the primary database") out, err := dbcommons.ExecCommand(r, r.Config, primaryReadyPod.Name, primaryReadyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf("echo -e \"%s\" | sqlplus -s / as sysdba", dbcommons.StandbyDatabasePrerequisitesSQL)) + fmt.Sprintf("echo -e \"%s\" | %s", dbcommons.StandbyDatabasePrerequisitesSQL, dbcommons.SQLPlusCLI)) if err != nil { return fmt.Errorf("unable to run StandbyDatabasePrerequisitesSQL in primary database") } @@ -2973,6 +3427,11 @@ func SetupInitParamsPrimaryForDG(r *SingleInstanceDatabaseReconciler, primaryRea return nil } +// ############################################################################# +// +// Setup primary database for standby singleinstancedatabase +// +// ############################################################################# func SetupPrimaryDatabase(r *SingleInstanceDatabaseReconciler, stdby *dbapi.SingleInstanceDatabase, primary *dbapi.SingleInstanceDatabase, ctx context.Context, req ctrl.Request) error { @@ -3014,12 +3473,17 @@ func SetupPrimaryDatabase(r *SingleInstanceDatabaseReconciler, stdby *dbapi.Sing } -// Standby DB Setup +// ############################################################################# +// +// Get all pdbs in a singleinstancedatabase +// +// ############################################################################# func GetAllPdbInDatabase(r *SingleInstanceDatabaseReconciler, dbReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) ([]string, error) { var pdbs []string out, err := dbcommons.ExecCommand(r, r.Config, dbReadyPod.Name, dbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", fmt.Sprintf("echo -e \"%s\" | sqlplus -s / as sysdba", dbcommons.GetPdbsSQL)) if err != nil { + r.Log.Error(err, err.Error()) return pdbs, err } r.Log.Info("GetPdbsSQL Output") @@ -3029,6 +3493,11 @@ func GetAllPdbInDatabase(r *SingleInstanceDatabaseReconciler, dbReadyPod corev1. return pdbs, nil } +// ############################################################################# +// +// Setup tnsnames.ora for all the pdb list in the singleinstancedatabase +// +// ############################################################################# func SetupTnsNamesForPDBListInDatabase(r *SingleInstanceDatabaseReconciler, d *dbapi.SingleInstanceDatabase, dbReadyPod corev1.Pod, ctx context.Context, req ctrl.Request, pdbList []string) error { for _, pdb := range pdbList { @@ -3066,6 +3535,11 @@ func SetupTnsNamesForPDBListInDatabase(r *SingleInstanceDatabaseReconciler, d *d return nil } +// ############################################################################# +// +// Setup tnsnames.ora in standby database for primary singleinstancedatabase +// +// ############################################################################# func SetupPrimaryDBTnsNamesInStandby(r *SingleInstanceDatabaseReconciler, s *dbapi.SingleInstanceDatabase, dbReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) error { @@ -3080,6 +3554,11 @@ func SetupPrimaryDBTnsNamesInStandby(r *SingleInstanceDatabaseReconciler, s *dba return nil } +// ############################################################################# +// +// Enabling flashback in singleinstancedatabase +// +// ############################################################################# func EnableFlashbackInDatabase(r *SingleInstanceDatabaseReconciler, dbReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) error { out, err := dbcommons.ExecCommand(r, r.Config, dbReadyPod.Name, dbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", fmt.Sprintf("echo -e \"%s\" | %s", dbcommons.FlashBackTrueSQL, dbcommons.GetSqlClient("enterprise"))) @@ -3091,6 +3570,11 @@ func EnableFlashbackInDatabase(r *SingleInstanceDatabaseReconciler, dbReadyPod c return nil } +// ############################################################################# +// +// setup standby database +// +// ############################################################################# func SetupStandbyDatabase(r *SingleInstanceDatabaseReconciler, stdby *dbapi.SingleInstanceDatabase, primary *dbapi.SingleInstanceDatabase, ctx context.Context, req ctrl.Request) error { @@ -3143,3 +3627,34 @@ func SetupStandbyDatabase(r *SingleInstanceDatabaseReconciler, stdby *dbapi.Sing return nil } + +// ############################################################################# +// +// Create oracle hostname environment variable object to be passed to sidb +// +// ############################################################################# +func CreateOracleHostnameEnvVarObj(sidb *dbapi.SingleInstanceDatabase, referedPrimaryDatabase *dbapi.SingleInstanceDatabase) corev1.EnvVar { + dbMajorVersion, err := strconv.Atoi(strings.Split(referedPrimaryDatabase.Status.ReleaseUpdate, ".")[0]) + if err != nil { + // r.Log.Error(err, err.Error()) + return corev1.EnvVar{ + Name: "ORACLE_HOSTNAME", + Value: "", + } + } + if dbMajorVersion >= 23 { + return corev1.EnvVar{ + Name: "ORACLE_HOSTNAME", + Value: sidb.Name, + } + } else { + return corev1.EnvVar{ + Name: "ORACLE_HOSTNAME", + ValueFrom: &corev1.EnvVarSource{ + FieldRef: &corev1.ObjectFieldSelector{ + FieldPath: "status.podIP", + }, + }, + } + } +} diff --git a/controllers/observability/databaseobserver_controller.go b/controllers/observability/databaseobserver_controller.go new file mode 100644 index 00000000..bd58e71e --- /dev/null +++ b/controllers/observability/databaseobserver_controller.go @@ -0,0 +1,547 @@ +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package controllers + +import ( + "context" + "errors" + "github.com/go-logr/logr" + appsv1 "k8s.io/api/apps/v1" + corev1 "k8s.io/api/core/v1" + apiError "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/api/meta" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/types" + "k8s.io/client-go/tools/record" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" + "time" + + apiv1 "github.com/oracle/oracle-database-operator/apis/observability/v1alpha1" + constants "github.com/oracle/oracle-database-operator/commons/observability" +) + +// DatabaseObserverReconciler reconciles a DatabaseObserver object +type DatabaseObserverReconciler struct { + client.Client + Log logr.Logger + Scheme *runtime.Scheme + Recorder record.EventRecorder +} + +//+kubebuilder:rbac:groups=observability.oracle.com,resources=databaseobservers,verbs=get;list;watch;create;update;patch;delete +//+kubebuilder:rbac:groups=observability.oracle.com,resources=databaseobservers/status,verbs=get;update;patch +//+kubebuilder:rbac:groups=observability.oracle.com,resources=databaseobservers/finalizers,verbs=update +//+kubebuilder:rbac:groups=apps,resources=pods;deployments;services,verbs=get;list;watch;create;update;patch;delete +//+kubebuilder:rbac:groups=apps,resources=configmaps,verbs=get;list +//+kubebuilder:rbac:groups="",resources=pods;deployments;services;events,verbs=get;list;watch;create;update;patch;delete +//+kubebuilder:rbac:groups="",resources=secrets;configmaps,verbs=get;list +//+kubebuilder:rbac:groups=monitoring.coreos.com,resources=servicemonitors,verbs=get;list;watch;create;update;patch;delete + +// Reconcile is part of the main kubernetes reconciliation loop which aims to +// move the current state of the cluster closer to the desired state. +// TODO(user): Modify the Reconcile function to compare the state specified by +// the DatabaseObserver object against the actual cluster state, and then +// perform operations to make the cluster state reflect the state specified by +// the user. +// +// For more details, check Reconcile and its Result here: +// - https://pkg.go.dev/sigs.k8s.io/controller-runtime@v0.6.4/pkg/reconcile +func (r *DatabaseObserverReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + + r.Log.WithName(constants.LogReconcile).Info(constants.LogCRStart, "NamespacedName", req.NamespacedName) + + // fetch databaseObserver + api := &apiv1.DatabaseObserver{} + if e := r.Get(context.TODO(), req.NamespacedName, api); e != nil { + + // if CR is not found or does not exist then + // consider either CR has been deleted + if apiError.IsNotFound(e) { + r.Log.WithName(constants.LogReconcile).Info(constants.LogCREnd) + return ctrl.Result{}, nil + } + + r.Log.WithName(constants.LogReconcile).Error(e, constants.ErrorCRRetrieve) + r.Recorder.Event(api, corev1.EventTypeWarning, constants.EventReasonFailedCRRetrieval, constants.EventMessageFailedCRRetrieval) + return ctrl.Result{}, e + + } + + // evaluate overall custom resource readiness at the end of the stack + defer r.validateCustomResourceReadiness(ctx, req) + + // initialize databaseObserver custom resource + if e := r.initialize(ctx, api, req); e != nil { + return ctrl.Result{}, e + } + + // validate specs + if e := r.validateSpecs(api); e != nil { + meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + Type: constants.IsExporterDeploymentReady, + Status: metav1.ConditionFalse, + Reason: constants.ReasonDeploymentSpecValidationFailed, + Message: constants.MessageExporterDeploymentSpecValidationFailed, + }) + if e := r.Status().Update(ctx, api); e != nil { + r.Log.WithName(constants.LogReconcile).Error(e, constants.ErrorStatusUpdate) + } + r.Log.WithName(constants.LogExportersDeploy).Error(e, constants.ErrorSpecValidationFailedDueToAnError) + return ctrl.Result{}, e + } + + // create resource if they do not exist + exporterDeployment := &ObservabilityDeploymentResource{} + if res, e := r.createResourceIfNotExists(exporterDeployment, api, ctx, req); e != nil { + return res, e + } + + if res, e := r.checkDeploymentForUpdates(exporterDeployment, api, ctx, req); e != nil { + meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + Type: constants.IsExporterDeploymentReady, + Status: metav1.ConditionFalse, + Reason: constants.ReasonDeploymentUpdateFailed, + Message: constants.MessageExporterDeploymentUpdateFailed, + }) + return res, e + } + + exporterService := &ObservabilityServiceResource{} + if res, e := r.createResourceIfNotExists(exporterService, api, ctx, req); e != nil { + return res, e + } + + exporterServiceMonitor := &ObservabilityServiceMonitorResource{} + if res, e := r.createResourceIfNotExists(exporterServiceMonitor, api, ctx, req); e != nil { + return res, e + } + + // check if deployment pods are ready + return r.validateDeploymentReadiness(api, ctx, req) +} + +// initialize method sets the initial status to PENDING, exporterConfig and sets the base condition +func (r *DatabaseObserverReconciler) initialize(ctx context.Context, api *apiv1.DatabaseObserver, req ctrl.Request) error { + + if api.Status.Conditions == nil || len(api.Status.Conditions) == 0 { + + // set condition + meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + Type: constants.IsCRAvailable, + Status: metav1.ConditionFalse, + Reason: constants.ReasonInitStart, + Message: constants.MessageCRInitializationStarted, + }) + + api.Status.Status = string(constants.StatusObservabilityPending) + api.Status.ExporterConfig = constants.UnknownValue + if e := r.Status().Update(ctx, api); e != nil { + r.Log.WithName(constants.LogReconcile).Error(e, constants.ErrorStatusUpdate) + return e + } + + } + + return nil +} + +// validateSpecs method checks the values and secrets passed in the spec +func (r *DatabaseObserverReconciler) validateSpecs(api *apiv1.DatabaseObserver) error { + + // If either Vault Fields are empty, then assume a DBPassword secret is supplied. If the DBPassword secret not found, then error out + if api.Spec.Database.DBPassword.VaultOCID == "" || api.Spec.Database.DBPassword.VaultSecretName == "" { + dbSecret := &corev1.Secret{} + if e := r.Get(context.TODO(), types.NamespacedName{Name: api.Spec.Database.DBPassword.SecretName, Namespace: api.Namespace}, dbSecret); e != nil { + r.Recorder.Event(api, corev1.EventTypeWarning, constants.EventReasonSpecError, constants.EventMessageSpecErrorDBPasswordSecretMissing) + return e + } + } + + // Does DB Connection String Secret Name actually exist + dbConnectSecret := &corev1.Secret{} + if e := r.Get(context.TODO(), types.NamespacedName{Name: api.Spec.Database.DBConnectionString.SecretName, Namespace: api.Namespace}, dbConnectSecret); e != nil { + r.Recorder.Event(api, corev1.EventTypeWarning, constants.EventReasonSpecError, constants.EventMessageSpecErrorDBConnectionStringSecretMissing) + return e + } + + // Does DB User String Secret Name actually exist + dbUserSecret := &corev1.Secret{} + if e := r.Get(context.TODO(), types.NamespacedName{Name: api.Spec.Database.DBUser.SecretName, Namespace: api.Namespace}, dbUserSecret); e != nil { + r.Recorder.Event(api, corev1.EventTypeWarning, constants.EventReasonSpecError, constants.EventMessageSpecErrorDBPUserSecretMissing) + return e + } + + // Does a custom configuration configmap actually exist, if provided + if configurationCMName := api.Spec.Exporter.ExporterConfig.Configmap.Name; configurationCMName != "" { + configurationCM := &corev1.ConfigMap{} + if e := r.Get(context.TODO(), types.NamespacedName{Name: configurationCMName, Namespace: api.Namespace}, configurationCM); e != nil { + r.Recorder.Event(api, corev1.EventTypeWarning, constants.EventReasonSpecError, constants.EventMessageSpecErrorConfigmapMissing) + return e + } + } + + // Does DBWallet actually exist, if provided + if dbWalletSecretName := api.Spec.Database.DBWallet.SecretName; dbWalletSecretName != "" { + dbWalletSecret := &corev1.Secret{} + if e := r.Get(context.TODO(), types.NamespacedName{Name: dbWalletSecretName, Namespace: api.Namespace}, dbWalletSecret); e != nil { + r.Recorder.Event(api, corev1.EventTypeWarning, constants.EventReasonSpecError, constants.EventMessageSpecErrorDBWalletSecretMissing) + return e + } + } + + return nil // valid, did not encounter any errors +} + +// createResourceIfNotExists method creates an ObserverResource if they have not yet been created +func (r *DatabaseObserverReconciler) createResourceIfNotExists(or ObserverResource, api *apiv1.DatabaseObserver, ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + + conditionType, logger, groupVersionKind := or.identify() + + // update after + defer r.Status().Update(ctx, api) + + // generate desired object based on api.Spec + desiredObj, genErr := or.generate(api, r.Scheme) + if genErr != nil { + meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + Type: conditionType, + Status: metav1.ConditionFalse, + Reason: constants.ReasonGeneralResourceGenerationFailed, + Message: constants.MessageResourceGenerationFailed, + }) + return ctrl.Result{}, genErr + } + + // if resource exists, retrieve the resource + foundObj := &unstructured.Unstructured{} + foundObj.SetGroupVersionKind(groupVersionKind) + getErr := r.Get(context.TODO(), types.NamespacedName{Name: desiredObj.GetName(), Namespace: req.Namespace}, foundObj) + + // if resource not found, create resource then return + if getErr != nil && apiError.IsNotFound(getErr) { + + if e := r.Create(context.TODO(), desiredObj); e != nil { // create + meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + Type: conditionType, + Status: metav1.ConditionFalse, + Reason: constants.ReasonGeneralResourceCreationFailed, + Message: constants.MessageResourceCreationFailed, + }) + r.Log.WithName(logger).Error(e, constants.ErrorResourceCreationFailure, "ResourceName", desiredObj.GetName(), "Kind", groupVersionKind, "Namespace", req.Namespace) + return ctrl.Result{}, e + } + + // mark ready if created + meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + Type: conditionType, + Status: metav1.ConditionTrue, + Reason: constants.ReasonGeneralResourceCreated, + Message: constants.MessageResourceCreated, + }) + r.Log.WithName(logger).Info(constants.LogResourceCreated, "ResourceName", desiredObj.GetName(), "Kind", groupVersionKind, "Namespace", req.Namespace) + + } else if getErr != nil { // if an error occurred + meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + Type: conditionType, + Status: metav1.ConditionFalse, + Reason: constants.ReasonGeneralResourceValidationFailureDueToError, + Message: constants.MessageResourceReadinessValidationFailed, + }) + r.Log.WithName(logger).Error(getErr, constants.ErrorResourceRetrievalFailureDueToAnError, "ResourceName", desiredObj.GetName(), "Kind", groupVersionKind, "Namespace", req.Namespace) + return ctrl.Result{}, getErr + + } else if getErr == nil && conditionType != constants.IsExporterDeploymentReady { // exclude deployment + meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + Type: conditionType, + Status: metav1.ConditionTrue, + Reason: constants.ReasonGeneralResourceValidationCompleted, + Message: constants.MessageResourceReadinessValidated, + }) + r.Log.WithName(logger).Info(constants.LogResourceFound, "ResourceName", desiredObj.GetName(), "Kind", groupVersionKind, "Namespace", req.Namespace) + + } + + // if no other error and resource, other than Deployments, have already been created before, end validation and return + return ctrl.Result{}, nil +} + +// checkDeploymentForUpdates method checks the deployment if it needs to be updated +func (r *DatabaseObserverReconciler) checkDeploymentForUpdates(or ObserverResource, api *apiv1.DatabaseObserver, ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + + // declare + foundDeployment := &appsv1.Deployment{} + + // generate object + desiredObj, genErr := or.generate(api, r.Scheme) + if genErr != nil { + return ctrl.Result{}, genErr + } + + // convert + desiredDeployment := &appsv1.Deployment{} + if e := r.Scheme.Convert(desiredObj, desiredDeployment, nil); e != nil { + return ctrl.Result{}, e + } + + // retrieve latest deployment + if e := r.Get(context.TODO(), types.NamespacedName{Name: desiredObj.GetName(), Namespace: req.Namespace}, foundDeployment); e != nil { + return ctrl.Result{}, e + } + // check for containerImage + if constants.IsUpdateRequiredForContainerImage(desiredDeployment, foundDeployment) { + foundDeployment.Spec.Template.Spec.Containers[0].Image = constants.GetExporterImage(api) + + if e := r.updateDeployment(api, ctx, req, foundDeployment, constants.MessageExporterDeploymentImageUpdated, constants.EventMessageUpdatedImageSucceeded); e != nil { + return ctrl.Result{}, e + } + } + + // retrieve latest deployment + if e := r.Get(context.TODO(), types.NamespacedName{Name: desiredObj.GetName(), Namespace: req.Namespace}, foundDeployment); e != nil { + return ctrl.Result{}, e + } + // check environment variables + if constants.IsUpdateRequiredForEnvironmentVars(desiredDeployment, foundDeployment) { + foundDeployment.Spec.Template.Spec.Containers[0].Env = constants.GetExporterEnvs(api) + + if e := r.updateDeployment(api, ctx, req, foundDeployment, constants.MessageExporterDeploymentEnvironmentUpdated, constants.EventMessageUpdatedEnvironmentSucceeded); e != nil { + return ctrl.Result{}, e + } + } + + // retrieve latest deployment + foundDeployment = &appsv1.Deployment{} + if e := r.Get(context.TODO(), types.NamespacedName{Name: desiredObj.GetName(), Namespace: req.Namespace}, foundDeployment); e != nil { + return ctrl.Result{}, e + } + // check config-volume, creds and ocikey + if constants.IsUpdateRequiredForVolumes(desiredDeployment, foundDeployment) { + foundDeployment.Spec.Template.Spec.Volumes = constants.GetExporterDeploymentVolumes(api) + foundDeployment.Spec.Template.Spec.Containers[0].VolumeMounts = constants.GetExporterDeploymentVolumeMounts(api) + + if e := r.updateDeployment(api, ctx, req, foundDeployment, constants.MessageExporterDeploymentVolumesUpdated, constants.EventMessageUpdatedVolumesSucceeded); e != nil { + return ctrl.Result{}, e + } + } + + // update status for exporter config + var setConfigmapNameStatus string + for _, v := range desiredDeployment.Spec.Template.Spec.Volumes { + if v.Name == constants.DefaultConfigVolumeString { + setConfigmapNameStatus = v.ConfigMap.Name + api.Status.ExporterConfig = setConfigmapNameStatus + } + } + if api.Status.ExporterConfig != setConfigmapNameStatus { + api.Status.ExporterConfig = constants.DefaultValue + } + r.Status().Update(ctx, api) + + // retrieve latest deployment + foundDeployment = &appsv1.Deployment{} + if e := r.Get(context.TODO(), types.NamespacedName{Name: desiredObj.GetName(), Namespace: req.Namespace}, foundDeployment); e != nil { + return ctrl.Result{}, e + } + // check replicateCount + if constants.IsUpdateRequiredForReplicas(desiredDeployment, foundDeployment) { + desiredReplicaCount := constants.GetExporterReplicas(api) + foundDeployment.Spec.Replicas = &desiredReplicaCount + + if e := r.updateDeployment(api, ctx, req, foundDeployment, constants.MessageExporterDeploymentReplicaUpdated, constants.EventMessageUpdatedReplicaSucceeded); e != nil { + return ctrl.Result{}, e + } + } + + return ctrl.Result{}, nil +} + +// updateDeployment method updates the deployment and sets the condition +func (r *DatabaseObserverReconciler) updateDeployment(api *apiv1.DatabaseObserver, ctx context.Context, req ctrl.Request, d *appsv1.Deployment, updateMessage string, recorderMessage string) error { + + // make update + defer r.Status().Update(ctx, api) + + if e := r.Update(context.TODO(), d); e != nil { + meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + Type: constants.IsExporterDeploymentReady, + Status: metav1.ConditionFalse, + Reason: constants.ReasonDeploymentUpdateFailed, + Message: constants.MessageExporterDeploymentUpdateFailed, + }) + r.Log.WithName(constants.LogExportersDeploy).Error(e, constants.ErrorDeploymentUpdate, "ResourceName", d.GetName(), "Kind", "Deployment", "Namespace", req.Namespace) + return e + } + + // update completed, however the pods needs to be validated for readiness + meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + Type: constants.IsExporterDeploymentReady, + Status: metav1.ConditionFalse, + Reason: constants.ReasonDeploymentUpdated, + Message: updateMessage, + }) + r.Log.WithName(constants.LogExportersDeploy).Info(constants.LogResourceUpdated, "ResourceName", d.GetName(), "Kind", "Deployment", "Namespace", req.Namespace) + r.Recorder.Event(api, corev1.EventTypeNormal, constants.EventReasonUpdateSucceeded, recorderMessage) + + return nil +} + +// validateDeploymentReadiness method evaluates deployment readiness by checking the status of all deployment pods +func (r *DatabaseObserverReconciler) validateDeploymentReadiness(api *apiv1.DatabaseObserver, ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + + d := &appsv1.Deployment{} + rName := constants.DefaultExporterDeploymentPrefix + api.Name + + // update after + defer r.Status().Update(ctx, api) + + // get latest deployment + if e := r.Get(context.TODO(), types.NamespacedName{Name: rName, Namespace: api.Namespace}, d); e != nil { + meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + Type: constants.IsExporterDeploymentReady, + Status: metav1.ConditionFalse, + Reason: constants.ReasonGeneralResourceValidationFailureDueToError, + Message: constants.MessageExporterDeploymentValidationFailed, + }) + return ctrl.Result{}, e + } + + // get deployment labels + labels := d.Spec.Template.Labels + cLabels := client.MatchingLabels{} + for k, v := range labels { + cLabels[k] = v + } + + // list pods + pods := &corev1.PodList{} + if e := r.List(context.TODO(), pods, []client.ListOption{cLabels}...); e != nil { + meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + Type: constants.IsExporterDeploymentReady, + Status: metav1.ConditionFalse, + Reason: constants.ReasonDeploymentFailed, + Message: constants.MessageExporterDeploymentListingFailed, + }) + return ctrl.Result{}, e + } + + // check each pod phase + for _, pod := range pods.Items { + if pod.Status.Phase == corev1.PodFailed { + meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + Type: constants.IsExporterDeploymentReady, + Status: metav1.ConditionFalse, + Reason: constants.ReasonDeploymentFailed, + Message: constants.MessageExporterDeploymentFailed, + }) + return ctrl.Result{}, errors.New(constants.ErrorDeploymentPodsFailure) + + } else if pod.Status.Phase != corev1.PodRunning { // pod could be creating, + meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + Type: constants.IsExporterDeploymentReady, + Status: metav1.ConditionUnknown, + Reason: constants.ReasonDeploymentPending, + Message: constants.MessageExporterDeploymentPending, + }) + return ctrl.Result{Requeue: true, RequeueAfter: 15 * time.Second}, nil + } + } + + // once all pods are found to be running, mark deployment as ready and the exporter as ready + meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + Type: constants.IsExporterDeploymentReady, + Status: metav1.ConditionTrue, + Reason: constants.ReasonDeploymentSuccessful, + Message: constants.MessageExporterDeploymentSuccessful, + }) + return ctrl.Result{}, nil +} + +// validateCustomResourceReadiness method evaluates CR readiness by cycling through all conditions and checking for any condition with False Status +func (r *DatabaseObserverReconciler) validateCustomResourceReadiness(ctx context.Context, req ctrl.Request) { + + // get latest object + api := &apiv1.DatabaseObserver{} + if e := r.Get(context.TODO(), req.NamespacedName, api); e != nil { + r.Log.WithName(constants.LogReconcile).Error(e, constants.ErrorCRRetrieve) + return + } + + // make update + defer r.Status().Update(ctx, api) + + if meta.IsStatusConditionPresentAndEqual(api.Status.Conditions, constants.IsExporterDeploymentReady, metav1.ConditionUnknown) { + meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + Type: constants.IsCRAvailable, + Status: metav1.ConditionFalse, + Reason: constants.ReasonValidationInProgress, + Message: constants.MessageCRValidationWaiting, + }) + api.Status.Status = string(constants.StatusObservabilityPending) + } else if meta.IsStatusConditionFalse(api.Status.Conditions, constants.IsExporterDeploymentReady) || + meta.IsStatusConditionFalse(api.Status.Conditions, constants.IsExporterServiceReady) || + meta.IsStatusConditionFalse(api.Status.Conditions, constants.IsExporterServiceMonitorReady) { + meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + Type: constants.IsCRAvailable, + Status: metav1.ConditionFalse, + Reason: constants.ReasonReadyFailed, + Message: constants.MessageCRValidationFailed, + }) + api.Status.Status = string(constants.StatusObservabilityError) + } else { + meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + Type: constants.IsCRAvailable, + Status: metav1.ConditionTrue, + Reason: constants.ReasonReadyValidated, + Message: constants.MessageCRValidated, + }) + api.Status.Status = string(constants.StatusObservabilityReady) + } +} + +// SetupWithManager sets up the controller with the Manager. +func (r *DatabaseObserverReconciler) SetupWithManager(mgr ctrl.Manager) error { + return ctrl.NewControllerManagedBy(mgr). + For(&apiv1.DatabaseObserver{}). + Owns(&appsv1.Deployment{}). + Owns(&corev1.Service{}). + Complete(r) +} diff --git a/controllers/observability/databaseobserver_resource.go b/controllers/observability/databaseobserver_resource.go new file mode 100644 index 00000000..75e05330 --- /dev/null +++ b/controllers/observability/databaseobserver_resource.go @@ -0,0 +1,181 @@ +package controllers + +import ( + apiv1 "github.com/oracle/oracle-database-operator/apis/observability/v1alpha1" + constants "github.com/oracle/oracle-database-operator/commons/observability" + monitorv1 "github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring/v1" + appsv1 "k8s.io/api/apps/v1" + corev1 "k8s.io/api/core/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/runtime/schema" + "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" +) + +/* +This handler file contains all the methods that +retrieve/find and create all related resources +on Kubernetes. +*/ + +type ObservabilityDeploymentResource struct{} +type ObservabilityServiceResource struct{} +type ObservabilityServiceMonitorResource struct{} + +type ObserverResource interface { + generate(*apiv1.DatabaseObserver, *runtime.Scheme) (*unstructured.Unstructured, error) + identify() (string, string, schema.GroupVersionKind) +} + +func (resource *ObservabilityDeploymentResource) generate(api *apiv1.DatabaseObserver, scheme *runtime.Scheme) (*unstructured.Unstructured, error) { + rName := constants.DefaultExporterDeploymentPrefix + api.Name + rContainerName := constants.DefaultExporterContainerName + rContainerImage := constants.GetExporterImage(api) + rVolumes := constants.GetExporterDeploymentVolumes(api) + rVolumeMounts := constants.GetExporterDeploymentVolumeMounts(api) + rSelectors := constants.GetExporterSelector(api) + rReplicas := constants.GetExporterReplicas(api) + rEnvs := constants.GetExporterEnvs(api) + + rPort := []corev1.ContainerPort{ + {ContainerPort: 8080}, + } + + obj := &appsv1.Deployment{ + ObjectMeta: metav1.ObjectMeta{ + Name: rName, + Namespace: api.Namespace, + }, + Spec: appsv1.DeploymentSpec{ + Replicas: &rReplicas, + Selector: &metav1.LabelSelector{ + MatchLabels: rSelectors, + }, + Template: corev1.PodTemplateSpec{ + ObjectMeta: metav1.ObjectMeta{ + Labels: rSelectors, + }, + Spec: corev1.PodSpec{ + Containers: []corev1.Container{{ + Image: rContainerImage, + ImagePullPolicy: corev1.PullAlways, + Name: rContainerName, + Env: rEnvs, + VolumeMounts: rVolumeMounts, + Ports: rPort, + }}, + RestartPolicy: corev1.RestartPolicyAlways, + Volumes: rVolumes, + }, + }, + }, + } + + if err := controllerutil.SetControllerReference(api, obj, scheme); err != nil { + return nil, err + } + + var u = &unstructured.Unstructured{} + if err := scheme.Convert(obj, u, nil); err != nil { + return nil, err + } + return u, nil +} + +func (resource *ObservabilityServiceResource) generate(api *apiv1.DatabaseObserver, scheme *runtime.Scheme) (*unstructured.Unstructured, error) { + rServiceName := "obs-svc-" + api.Name + rLabels := constants.GetExporterLabels(api) + rPort := constants.GetExporterServicePort(api) + rSelector := constants.GetExporterSelector(api) + + obj := &corev1.Service{ + ObjectMeta: metav1.ObjectMeta{ + Name: rServiceName, + Labels: rLabels, + Namespace: api.Namespace, + }, + Spec: corev1.ServiceSpec{ + Type: "ClusterIP", + Selector: rSelector, + Ports: []corev1.ServicePort{ + { + Name: "metrics", + Port: rPort, + }, + }, + }, + } + + if err := controllerutil.SetControllerReference(api, obj, scheme); err != nil { + return nil, err + } + + var u = &unstructured.Unstructured{} + if err := scheme.Convert(obj, u, nil); err != nil { + return nil, err + } + return u, nil +} + +func (resource *ObservabilityServiceMonitorResource) generate(api *apiv1.DatabaseObserver, scheme *runtime.Scheme) (*unstructured.Unstructured, error) { + rName := constants.DefaultServiceMonitorPrefix + api.Name + rLabels := constants.GetExporterLabels(api) + rSelector := constants.GetExporterSelector(api) + rPort := constants.GetExporterServiceMonitorPort(api) + rInterval := "20s" + + obj := &monitorv1.ServiceMonitor{ + ObjectMeta: metav1.ObjectMeta{ + Name: rName, + Labels: rLabels, + Namespace: api.Namespace, + }, + Spec: monitorv1.ServiceMonitorSpec{ + Endpoints: []monitorv1.Endpoint{{ + Interval: monitorv1.Duration(rInterval), + Port: rPort, + }}, + Selector: metav1.LabelSelector{ + MatchLabels: rSelector, + }, + }, + } + + // set reference + if e := controllerutil.SetControllerReference(api, obj, scheme); e != nil { + return nil, e + } + + // convert + var u = &unstructured.Unstructured{} + if e := scheme.Convert(obj, u, nil); e != nil { + return nil, e + } + + return u, nil +} + +func (resource *ObservabilityDeploymentResource) identify() (string, string, schema.GroupVersionKind) { + return constants.IsExporterDeploymentReady, constants.LogExportersDeploy, schema.GroupVersionKind{ + Group: "apps", + Version: "v1", + Kind: "Deployment", + } +} + +func (resource *ObservabilityServiceResource) identify() (string, string, schema.GroupVersionKind) { + return constants.IsExporterServiceReady, constants.LogExportersSVC, schema.GroupVersionKind{ + Group: "", + Version: "v1", + Kind: "Service", + } +} + +func (resource *ObservabilityServiceMonitorResource) identify() (string, string, schema.GroupVersionKind) { + return constants.IsExporterServiceMonitorReady, constants.LogExportersServiceMonitor, schema.GroupVersionKind{ + Group: "monitoring.coreos.com", + Version: "v1", + Kind: "ServiceMonitor", + } +} diff --git a/controllers/observability/suite_test.go b/controllers/observability/suite_test.go new file mode 100644 index 00000000..4500ff5a --- /dev/null +++ b/controllers/observability/suite_test.go @@ -0,0 +1,100 @@ +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package controllers + +import ( + "path/filepath" + "testing" + + . "github.com/onsi/ginkgo/v2" + . "github.com/onsi/gomega" + "k8s.io/client-go/kubernetes/scheme" + "k8s.io/client-go/rest" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/envtest" + logf "sigs.k8s.io/controller-runtime/pkg/log" + "sigs.k8s.io/controller-runtime/pkg/log/zap" + + observabilityv1alpha1 "github.com/oracle/oracle-database-operator/apis/observability/v1alpha1" + //+kubebuilder:scaffold:imports +) + +// These tests use Ginkgo (BDD-style Go testing framework). Refer to +// http://onsi.github.io/ginkgo/ to learn more about Ginkgo. + +var cfg *rest.Config +var k8sClient client.Client +var testEnv *envtest.Environment + +func TestAPIs(t *testing.T) { + RegisterFailHandler(Fail) + + RunSpecs(t, "Controller Suite") +} + +var _ = BeforeSuite(func(done Done) { + logf.SetLogger(zap.New(zap.WriteTo(GinkgoWriter), zap.UseDevMode(true))) + + By("bootstrapping test environment") + testEnv = &envtest.Environment{ + CRDDirectoryPaths: []string{filepath.Join("..", "..", "config", "crd", "bases")}, + } + + var err error + cfg, err = testEnv.Start() + Expect(err).ToNot(HaveOccurred()) + Expect(cfg).ToNot(BeNil()) + + err = observabilityv1alpha1.AddToScheme(scheme.Scheme) + Expect(err).NotTo(HaveOccurred()) + + //+kubebuilder:scaffold:scheme + + k8sClient, err = client.New(cfg, client.Options{Scheme: scheme.Scheme}) + Expect(err).ToNot(HaveOccurred()) + Expect(k8sClient).ToNot(BeNil()) + + close(done) +}, 60) + +var _ = AfterSuite(func() { + By("tearing down the test environment") + err := testEnv.Stop() + Expect(err).ToNot(HaveOccurred()) +}) diff --git a/docs/adb/ACD.md b/docs/adb/ACD.md index ffd09116..81ee1a65 100644 --- a/docs/adb/ACD.md +++ b/docs/adb/ACD.md @@ -8,7 +8,7 @@ As indicated in the prerequisites (see above), to interact with OCI services, ei ## Required Permissions -The opeartor must be given the required type of access in a policy written by an administrator to manage the Autonomous Container Databases. See [Create an Autonomous Container Database](https://docs.oracle.com/en-us/iaas/autonomous-database/doc/create-acd.html) for the required policies. +The operator must be given the required type of access in a policy written by an administrator to manage the Autonomous Container Databases. See [Create an Autonomous Container Database](https://docs.oracle.com/en-us/iaas/autonomous-database/doc/create-acd.html) for the required policies. The permission to view the workrequests is also required, so that the operator will update the resources when the work is done. See [Viewing Work Requests](https://docs.oracle.com/en-us/iaas/Content/ContEng/Tasks/contengviewingworkrequests.htm#contengviewingworkrequests) for sample work request policies. @@ -172,7 +172,7 @@ Here's a list of the values you can set for `action`: * `RESTART`: to restart the database * `TERMINATE`: to terminate the database -* `SYNC`: to sync the database, will describe in the next section +* `SYNC`: to sync the local database with the remote one 1. A sample .yaml file is available here: [config/samples/acd/autonomouscontainerdatabase_restart_terminate.yaml](./../../config/samples/acd/autonomouscontainerdatabase_restart_terminate.yaml) diff --git a/docs/adb/ADB_MANUAL_BACKUP.md b/docs/adb/ADB_LONG_TERM_BACKUP.md similarity index 70% rename from docs/adb/ADB_MANUAL_BACKUP.md rename to docs/adb/ADB_LONG_TERM_BACKUP.md index 2adec8e4..4720697d 100644 --- a/docs/adb/ADB_MANUAL_BACKUP.md +++ b/docs/adb/ADB_LONG_TERM_BACKUP.md @@ -1,23 +1,21 @@ -# Backing up an Oracle Autonomous Database Manually +# Creating Long-Term Backups of an Oracle Autonomous Database -To create manual backups of Autonomous Databases, use this procedure. +To create long-term backups of Autonomous Databases, use this procedure. -Oracle Cloud Infrastructure (OCI) automatically backs up your Autonomous Databases, and retains these backups for 60 days. You can restore and recover your database to any point-in-time in this retention period. Automatic backups are full backups taken every 60 days, with daily incremental backups. You can also create manual backups for your database. Manual backups are stored in an Object Storage bucket that you create, and are retained for 60 days. Note that Oracle doesn not recommend create or use manual backups. For more information, please visit [Backing Up and Restoring Autonomous Database](https://docs.oracle.com/en-us/iaas/Content/Database/Tasks/adbbackingup.htm) and [Backup and Restore Notes](https://docs.oracle.com/en-us/iaas/autonomous-database-shared/doc/backup-restore-notes.html). +Oracle Cloud Infrastructure (OCI) automatically backs up your Autonomous Databases, and retains these backups for 60 days. You can restore and recover your database to any point-in-time in this retention period. Automatic backups are full backups taken every 60 days, with daily incremental backups. You can also create long-term backups for your database with a retention period between 3 months and up to 10 years. For more information, please visit [Create Long-Term Backups on Autonomous Database](https://docs.oracle.com/en/cloud/paas/autonomous-database/serverless/adbsb/backup-long-term.html) and [Backup and Restore Notes](https://docs.oracle.com/en-us/iaas/autonomous-database-serverless/doc/backup-restore-notes.html). -## Prerequisites - -To hold your Autonomous Database manual backups, you must create an Oracle Cloud Infrastructure Object Storage bucket, and configure your database to connect to it. To finish setting up manual backup storage, follow the steps in this page: [Setting Up a Bucket to Store Manual Backups](https://docs.oracle.com/en-us/iaas/Content/Database/Tasks/adbbackingup.htm#creatingbucket). Creating an Autonomous Database manual backup object storage bucket is a one-time operation. - -## Create Manual Backup +## Create Long-Term Backup To back up an Autonomous Database, complete this procedure. 1. Add the following fields to the AutonomousDatabaseBackup resource definition. An example `.yaml` file is available here: [`config/samples/adb/autonomousdatabase_backup.yaml`](./../../config/samples/adb/autonomousdatabase_backup.yaml) | Attribute | Type | Description | Required? | |----|----|----|----| + | `spec.displayName` | string | The user-friendly name for the backup. This name does not have to be unique. | Yes | + | `spec.isLongTermBackup` | boolean | Indicates whether the backup is long-term. | Yes | + | `spec.retentionPeriodInDays` | string | Retention period, in days, for long-term backups. Minimum retention period is 90 days. | Yes | | `spec.target.k8sADB.name` | string | The name of custom resource of the target Autonomous Database. Choose either the `spec.target.k8sADB.name` or the `spec.target.ociADB.ocid`, but not both. | Conditional | | `spec.target.ociADB.ocid` | string | The [OCID](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm) of the target AutonomousDatabase. Choose either the `spec.target.k8sADB.name` or the `spec.target.ociADB.ocid`, but not both. | Conditional | - | `spec.displayName` | string | The user-friendly name for the backup. This name does not have to be unique. | Yes | | `spec.ociConfig` | dictionary | Not required when the Operator is authorized with [Instance Principal](./ADB_PREREQUISITES.md#authorized-with-instance-principal). Otherwise, you will need the values from this section: [Authorized with API Key Authentication](./ADB_PREREQUISITES.md#authorized-with-api-key-authentication). | Conditional | | `spec.ociConfig.configMapName` | string | Name of the ConfigMap that holds the local OCI configuration | Conditional | | `spec.ociConfig.secretName`| string | Name of the Kubernetes (K8s) Secret that holds the private key value | Conditional | @@ -36,6 +34,8 @@ To back up an Autonomous Database, complete this procedure. # ociADB: # ocid: ocid1.autonomousdatabase... displayName: autonomousdatabasebackup-sample + isLongTermBackup: true + retentionPeriodInDays: 90 ociConfig: configMapName: oci-cred secretName: oci-privatekey diff --git a/docs/adb/ADB_PREREQUISITES.md b/docs/adb/ADB_PREREQUISITES.md index 3eaf5b4a..f8c04c4b 100644 --- a/docs/adb/ADB_PREREQUISITES.md +++ b/docs/adb/ADB_PREREQUISITES.md @@ -12,7 +12,7 @@ To provide access, choose **one of the following approaches**: ## Authorized with API Key Authentication -API keys are supplied by users to authenticate the operator accessing Oracle Cloud Infrastructure (OCI) services. The operator reads the credintials of the OCI user from a ConfigMap and a Secret. If you're using Oracle Container Engine for Kubernetes (OKE), you may alternatively use [Instance Principal](#authorized-with-instance-principal) to avoid the need to configure user credentails or a configuration file. If the operator is deployed in a third-party Kubernetes cluster, then the credentials or a configuration file are needed, since Instance principal authorization applies only to instances that are running in the OCI. +API keys are supplied by users to authenticate the operator accessing Oracle Cloud Infrastructure (OCI) services. The operator reads the credentials of the OCI user from a ConfigMap and a Secret. If you're using Oracle Container Engine for Kubernetes (OKE), you may alternatively use [Instance Principal](#authorized-with-instance-principal) to avoid the need to configure user credentials or a configuration file. If the operator is deployed in a third-party Kubernetes cluster, then the credentials or a configuration file are needed, since Instance principal authorization applies only to instances that are running in the OCI. Oracle recommends using the helper script `set_ocicredentials.sh` in the root directory of the repository; this script will generate a ConfigMap and a Secret with the OCI credentials. By default, the script parses the **DEFAULT** profile in `~/.oci/config`. The default names of the ConfigMap and the Secret are, respectively: `oci-cred` and `oci-privatekey`. @@ -46,9 +46,9 @@ After creating the ConfigMap and the Secret, use their names as the values of `o ## Authorized with Instance Principal -Instance principal authorization enables the operator to make API calls from an instance (that is, a node) without requiring the `ociConfigMap`, and `ociSecret` attributes in the `.yaml` file. This approach applies only to instances that are running in the Oracle Cloud Infrastructure (OCI). In addition, this approach grants permissions to the nodes that matche the rules, which means that all the pods in the nodes can make the service calls. +Instance principal authorization enables the operator to make API calls from an instance (that is, a node) without requiring the `ociConfigMap`, and `ociSecret` attributes in the `.yaml` file. This approach applies only to instances that are running in the Oracle Cloud Infrastructure (OCI). In addition, this approach grants permissions to the nodes that match the rules, which means that all the pods in the nodes can make the service calls. -To set up the instance princials, you will have to: +To set up the instance principals, you will have to: * [Define dynamic group that includes the nodes in which the operator runs](#define-dynamic-group) * [Define policies that grant to the dynamic group the required permissions for the operator to its OCI interactions](#define-policies) @@ -111,7 +111,7 @@ To set up the instance princials, you will have to: Allow dynamic-group to manage all-resources in tenancy ``` - Example 3: enable a particular resouce access for the dynamic group to manage Oracle Autonomous Database in a given compartment + Example 3: enable a particular resource access for the dynamic group to manage Oracle Autonomous Database in a given compartment ```sh Allow dynamic-group to manage autonomous-database-family in compartment @@ -120,3 +120,31 @@ To set up the instance princials, you will have to: 3. To apply the policy, click Create. At this stage, the instances where the operator deploys have been granted sufficient permissions to call OCI services. You can now proceed to the installation. + +### Authorized with OKE Workload Identity + +OKE Workload Identity grants the operator pods policy-driven access to OCI resources using OCI Identity and Access Management (IAM). +When using OKE Workload Identity, only the region must be specified in the ConfigMap corresponding to the `ociConfigMap` attribute. The `ociSecret` attribute should not be specified in the `.yaml` file. + +To set up the OKE Workload Identity, you will have to: + +### Configure Cluster Region + +The operator reads the OCI region from a ConfigMap. + +```sh +kubectl create configmap oci-cred \ +--from-literal=region= +``` + +### Define Policies + +1. Get the compartment name where the database resides/will be created. +2. Get the OCID of the OKE Cluster where the Oracle Database Operator is running. +3. Create the following policy in OCI IAM, supplying your compartment name and OKE Cluster OCID: + +``` +Allow any-user to manage all-resources in compartment where all {request.principal.namespace='oracle-database-operator-system',request.principal.type='workload',request.principal.cluster_id='',request.principal.service_account='default'} +``` + +After creating the policy, operator pods will be granted sufficient permissions to call OCI services. You can now proceed to the installation. \ No newline at end of file diff --git a/docs/adb/ADB_RESTORE.md b/docs/adb/ADB_RESTORE.md index a7f67888..7a80090a 100644 --- a/docs/adb/ADB_RESTORE.md +++ b/docs/adb/ADB_RESTORE.md @@ -2,7 +2,7 @@ To restore an Autonomous Database from a backup, use this document. -You can either use any existing manual or automatic backup to restore your database, or you can restore and recover your database to any point in time in the 60-day retention period of your automatic backups. For point-in-time restores, you specify a timestamp. Your Autonomous Database identifies which backup to use for the fastest restore. +You can either use any existing on-demand or automatic backup to restore your database, or you can restore and recover your database to any point in time in the 60-day retention period of your automatic backups. For point-in-time restores, you specify a timestamp. Your Autonomous Database identifies which backup to use for the fastest restore. ## Restore an Autonomous Database diff --git a/docs/adb/README.md b/docs/adb/README.md index 144f9d41..1b59c4d6 100644 --- a/docs/adb/README.md +++ b/docs/adb/README.md @@ -111,7 +111,7 @@ Follow these steps to provision an Autonomous Database that will map objects in 5. Choose the type of network access (optional): - By default, the network access type is set to PUBLIC, which allows secure connections from anywhere. Uncomment the code block if you want configure the network acess. See [Configuring Network Access of Autonomous Database](./NETWORK_ACCESS_OPTIONS.md) for more information. + By default, the network access type is set to PUBLIC, which allows secure connections from anywhere. Uncomment the code block if you want configure the network access. See [Configuring Network Access of Autonomous Database](./NETWORK_ACCESS_OPTIONS.md) for more information. 6. Apply the YAML: @@ -411,6 +411,18 @@ To delete the resource and terminate the Autonomous Database, complete these ste Now, you can verify that the database is in TERMINATING state on the Cloud Console. +## Roles and Privileges requirements for Oracle Autonomous Database Controller + +Autonomous Database controller uses Kubernetes objects such as: + + | Resources | Verbs | + | --- | --- | + | Configmaps | get list watch create update patch delete | + | Secrets | get list watch create update patch delete | + | Events | create patch | + +The defintion of all the Kubernetes Objects, which are to be used by the Oracle Autonomous Database Controller, comes from the `oracle-database-operator.yaml` file which is applied to deploy the **Oracle Database Operator**. + ## Debugging and troubleshooting ### Show the details of the resource diff --git a/docs/dbcs/provisioning/database_connection.md b/docs/dbcs/provisioning/database_connection.md index e69de29b..66ac2b5d 100644 --- a/docs/dbcs/provisioning/database_connection.md +++ b/docs/dbcs/provisioning/database_connection.md @@ -0,0 +1,53 @@ +## Database connection + +In order to retrieve the database connection use the kubectl describe command + +```sh +kubectl describe dbcssystems.database.oracle.com dbcssystem-create +``` + +You can use the following script (tnsalias.awk) to get a simple tnsalias + +```awk +!#/usr/bin/awk +( $0 ~ / Db Unique Name:/ ) { DB_UNIQUE_NAME=$4 } +( $0 ~ /Domain Name:/ ) { DB_DOMAIN=$3 } +( $0 ~ /Host Name:/ ) { HOSTNAME=$3 } +( $0 ~ /Listener Port:/ ) { PORT=$3 } + +END { + printf ("db_unique_name=%s\n",DB_UNIQUE_NAME); + printf ("db_domain=%s\n",DB_DOMAIN); + printf ("hostname=%s\n",HOSTNAME); + printf ("port=%s\n",PORT); + printf ("====== TNSALIAS ======\n"); + printf ("(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=%s)(PORT=%s))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=%s.%s)))\n", + HOSTNAME,PORT,DB_UNIQUE_NAME,DB_DOMAIN); +``` + +```text +kubectl describe dbcssystems.database.oracle.com dbcssystem-create |awk -f tnsalias.awk +db_unique_name=testdb_fg4_lin +db_domain=vcndns.oraclevcn.com +hostname=host1205 +port=1521 +====== TNSALIAS ====== +(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=host1205)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=testdb_fg4_lin.vcndns.oraclevcn.com))) + +sqlplus scott@"(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=host1205)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=testdb_fg4_lin.vcndns.oraclevcn.com)))" + + +SQL*Plus: Release 19.0.0.0.0 - Production on Fri Dec 15 14:16:42 2023 +Version 19.15.0.0.0 + +Copyright (c) 1982, 2022, Oracle. All rights reserved. + +Enter password: +Last Successful login time: Fri Dec 15 2023 14:14:07 +00:00 + +Connected to: +Oracle Database 19c EE High Perf Release 19.0.0.0.0 - Production +Version 19.18.0.0.0 + +SQL> +``` diff --git a/docs/dbcs/provisioning/dbcs_service_with_minimal_parameters.md b/docs/dbcs/provisioning/dbcs_service_with_minimal_parameters.md index f91370ca..f6bc8185 100644 --- a/docs/dbcs/provisioning/dbcs_service_with_minimal_parameters.md +++ b/docs/dbcs/provisioning/dbcs_service_with_minimal_parameters.md @@ -8,16 +8,17 @@ This example uses `dbcs_service_with_minimal_parameters.yaml` to deploy a Single - OCI Configmap as `oci-cred` - OCI Secret as `oci-privatekey` -- Availability Domain for the DBCS VMDB as `OLou:PHX-AD-1` -- Compartment OCID as `ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya` +- Availability Domain for the DBCS VMDB as `OLou:EU-MILAN-1-AD-1` +- Compartment OCID as `ocid1.compartment.oc1..aaaaaaaaks5baeqlvv4kyj2jiwnrbxgzm3gsumcfy4c6ntj2ro5i3a5gzhhq` - Database Admin Credential as `admin-password` -- Database Name as `db0130` -- Oracle Database Software Image Version as `21c` +- Database Name as `dbsystem0130` +- Oracle Database Software Image Version as `19c` - Database Workload Type as Transaction Processing i.e. `OLTP` -- Database Hostname Prefix as `host0130` +- Database Hostname Prefix as `host1205` - Oracle VMDB Shape as `VM.Standard2.1` - SSH Public key for the DBCS system being deployed as `oci-publickey` -- OCID of the Subnet as `ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a` +- domain `vcndns.oraclevcn.com` +- OCID of the Subnet as `ocid1.subnet.oc1.eu-milan-1.aaaaaaaaeiy3tvcsnyg6upfp3ydtu7jmfnmoyifq2ax6y45b5qpdbpide5xa` **NOTE:** For the details of the parameters to be used in the .yaml file, please refer [here](./dbcs_controller_parameters.md). diff --git a/docs/dbcs/provisioning/dbcs_service_with_minimal_parameters.yaml b/docs/dbcs/provisioning/dbcs_service_with_minimal_parameters.yaml index 9e4a5f3d..dc02cfa3 100644 --- a/docs/dbcs/provisioning/dbcs_service_with_minimal_parameters.yaml +++ b/docs/dbcs/provisioning/dbcs_service_with_minimal_parameters.yaml @@ -6,14 +6,20 @@ spec: ociConfigMap: "oci-cred" ociSecret: "oci-privatekey" dbSystem: - availabilityDomain: "OLou:PHX-AD-1" - compartmentId: "ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya" + availabilityDomain: "OLou:EU-MILAN-1-AD-1" + compartmentId: "ocid1.compartment.oc1..aaaaaaaaks5baeqlvv4kyj2jiwnrbxgzm3gsumcfy4c6ntj2ro5i3a5gzhhq" dbAdminPaswordSecret: "admin-password" - dbName: "db0130" - dbVersion: "21c" + dbEdition: "ENTERPRISE_EDITION_HIGH_PERFORMANCE" + dbName: "testdb" + displayName: "dbsystem0130" + licenseModel: "BRING_YOUR_OWN_LICENSE" + dbVersion: "19c" dbWorkload: "OLTP" - hostName: "host0130" + hostName: "host1205" shape: "VM.Standard2.1" - sshPublicKeys: - - "oci-publickey" - subnetId: "ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a" + domain: "vcndns.oraclevcn.com" + sshPublicKeys: + - "oci-publickey" + subnetId: "ocid1.subnet.oc1.eu-milan-1.aaaaaaaaeiy3tvcsnyg6upfp3ydtu7jmfnmoyifq2ax6y45b5qpdbpide5xa" + + diff --git a/docs/dbcs/provisioning/dbcs_service_with_minimal_parameters_sample_output.log b/docs/dbcs/provisioning/dbcs_service_with_minimal_parameters_sample_output.log index ddf1d81a..133f109c 100644 --- a/docs/dbcs/provisioning/dbcs_service_with_minimal_parameters_sample_output.log +++ b/docs/dbcs/provisioning/dbcs_service_with_minimal_parameters_sample_output.log @@ -1,257 +1,135 @@ -[root@docker-test-server test]# cat dbcs_service_with_minimal_parameters.yaml -apiVersion: database.oracle.com/v1alpha1 -kind: DbcsSystem -metadata: - name: dbcssystem-create -spec: - ociConfigMap: "oci-cred" - ociSecret: "oci-privatekey" - dbSystem: - availabilityDomain: "OLou:PHX-AD-1" - compartmentId: "ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya" - dbAdminPaswordSecret: "admin-password" - dbName: "db0130" - dbVersion: "21c" - dbWorkload: "OLTP" - hostName: "host0130" - shape: "VM.Standard2.1" - sshPublicKeys: - - "oci-publickey" - subnetId: "ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a" -[root@docker-test-server test]# -[root@docker-test-server test]# kubectl apply -f dbcs_service_with_minimal_parameters.yaml -dbcssystem.database.oracle.com/dbcssystem-create created - - -[root@docker-test-server test]# kubectl get ns - -kubectl get allNAME STATUS AGE -cert-manager Active 13d -default Active 139d -kube-node-lease Active 139d -kube-public Active 139d -kube-system Active 139d -oracle-database-operator-system Active 13d -shns Active 88d -[root@docker-test-server test]# -[root@docker-test-server test]# kubectl get all -n oracle-database-operator-system -NAME READY STATUS RESTARTS AGE -pod/oracle-database-operator-controller-manager-665874bd57-dlhls 1/1 Running 3 13d -pod/oracle-database-operator-controller-manager-665874bd57-g2cgw 1/1 Running 3 13d -pod/oracle-database-operator-controller-manager-665874bd57-q42f8 1/1 Running 4 13d - -NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE -service/oracle-database-operator-controller-manager-metrics-service ClusterIP 10.96.130.124 8443/TCP 13d -service/oracle-database-operator-webhook-service ClusterIP 10.96.4.104 443/TCP 13d - -NAME READY UP-TO-DATE AVAILABLE AGE -deployment.apps/oracle-database-operator-controller-manager 3/3 3 3 13d - -NAME DESIRED CURRENT READY AGE -replicaset.apps/oracle-database-operator-controller-manager-665874bd57 3 3 3 13d -[root@docker-test-server test]# -[root@docker-test-server test]# -[root@docker-test-server test]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-dlhls -n oracle-database-operator-system -. -. -2022-03-08T22:12:57.414Z INFO controller-runtime.manager.controller.dbcssystem OCI provider configured succesfully {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:12:58.013Z INFO controller-runtime.manager.controller.dbcssystem DbcsSystem DBSystem provisioning {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:13:05.772Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:14:06.499Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:15:07.256Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:16:07.877Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:17:08.237Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:18:08.852Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:19:09.184Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:20:10.253Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:21:10.576Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:22:10.948Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:23:11.443Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:24:11.872Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:25:12.206Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:26:12.543Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:27:13.053Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:28:13.582Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:29:13.927Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:30:14.415Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:31:14.712Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:32:15.236Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:33:16.113Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:34:16.397Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:35:16.723Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:36:17.178Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:37:17.454Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:38:17.849Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:39:18.305Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:40:18.724Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:41:19.050Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:42:19.570Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:43:19.836Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:44:20.258Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:45:20.592Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:46:20.917Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:47:21.225Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:48:21.587Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:49:21.841Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:50:22.214Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:51:22.519Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:52:22.875Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:53:23.324Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:54:23.755Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:55:24.273Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:56:24.672Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:57:25.518Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:58:26.113Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T22:59:26.373Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T23:00:26.650Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T23:01:26.895Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T23:02:27.437Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T23:03:27.835Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T23:04:28.231Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T23:05:28.653Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T23:06:29.286Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T23:07:29.637Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T23:08:30.061Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T23:09:30.327Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T23:10:30.838Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T23:11:31.312Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T23:12:31.755Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T23:13:32.075Z INFO controller-runtime.manager.controller.dbcssystem DbcsSystem system provisioned succesfully {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} -2022-03-08T23:13:34.641Z INFO controller-runtime.manager.controller.dbcssystem OCI provider configured succesfully {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} -2022-03-08T23:13:45.117Z INFO controller-runtime.manager.controller.dbcssystem OCI provider configured succesfully {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-create", "namespace": "default"} - - -[root@docker-test-server test]# kubectl describe dbcssystems.database.oracle.com dbcssystem-create +/usr/bin/kubectl describe dbcssystems.database.oracle.com dbcssystem-create Name: dbcssystem-create Namespace: default Labels: -Annotations: lastSuccessfulSpec: - {"dbSystem":{"compartmentId":"ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya","availabilityDomain":"O... +Annotations: kubectl.kubernetes.io/last-applied-configuration: + {"apiVersion":"database.oracle.com/v1alpha1","kind":"DbcsSystem","metadata":{"annotations":{},"name":"dbcssystem-create","namespace":"defa... API Version: database.oracle.com/v1alpha1 Kind: DbcsSystem Metadata: - Creation Timestamp: 2022-03-08T22:12:57Z + Creation Timestamp: 2023-12-12T12:59:58Z Generation: 1 Managed Fields: API Version: database.oracle.com/v1alpha1 Fields Type: FieldsV1 - fieldsV1: - f:metadata: - f:annotations: + Fields V 1: + F : Metadata: + F : Annotations: .: - f:kubectl.kubernetes.io/last-applied-configuration: - f:spec: + F : Kubectl . Kubernetes . Io / Last - Applied - Configuration: + F : Spec: .: - f:dbSystem: + F : Db System: .: - f:availabilityDomain: - f:compartmentId: - f:dbAdminPaswordSecret: - f:dbName: - f:dbVersion: - f:dbWorkload: - f:hostName: - f:shape: - f:sshPublicKeys: - f:subnetId: - f:ociConfigMap: - f:ociSecret: - Manager: kubectl-client-side-apply + F : Availability Domain: + F : Compartment Id: + F : Db Admin Pasword Secret: + F : Db Edition: + F : Db Name: + F : Db Version: + F : Db Workload: + F : Display Name: + F : Domain: + F : Host Name: + F : License Model: + F : Shape: + F : Ssh Public Keys: + F : Subnet Id: + F : Oci Config Map: + F : Oci Secret: + Manager: kubectl Operation: Update - Time: 2022-03-08T22:12:57Z + Time: 2023-12-12T12:59:58Z API Version: database.oracle.com/v1alpha1 Fields Type: FieldsV1 - fieldsV1: - f:metadata: - f:annotations: - f:lastSuccessfulSpec: - f:spec: - f:dbSystem: - f:dbBackupConfig: - f:id: - f:status: + Fields V 1: + F : Status: .: - f:availabilityDomain: - f:cpuCoreCount: - f:dataStoragePercentage: - f:dataStorageSizeInGBs: - f:dbEdition: - f:dbInfo: - f:displayName: - f:id: - f:licenseModel: - f:network: + F : Availability Domain: + F : Cpu Core Count: + F : Data Storage Percentage: + F : Data Storage Size In G Bs: + F : Db Edition: + F : Db Info: + F : Display Name: + F : Id: + F : License Model: + F : Network: .: - f:clientSubnet: - f:domainName: - f:hostName: - f:listenerPort: - f:scanDnsName: - f:vcnName: - f:nodeCount: - f:recoStorageSizeInGB: - f:shape: - f:state: - f:storageManagement: - f:subnetId: - f:timeZone: - f:workRequests: + F : Client Subnet: + F : Domain Name: + F : Host Name: + F : Listener Port: + F : Vcn Name: + F : Node Count: + F : Reco Storage Size In GB: + F : Shape: + F : State: + F : Storage Management: + F : Subnet Id: + F : Time Zone: + F : Work Requests: Manager: manager Operation: Update - Time: 2022-03-08T23:13:34Z - Resource Version: 55187354 - UID: 0cdbebc0-3aeb-43b1-ae7f-eb36e6c56000 + Subresource: status + Time: 2023-12-12T14:12:36Z + Resource Version: 1571919 + UID: e11353f3-1334-4ca8-af31-4b638442f429 Spec: Db System: - Availability Domain: OLou:PHX-AD-1 - Compartment Id: ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya + Availability Domain: OLou:EU-MILAN-1-AD-1 + Compartment Id: ocid1.compartment.oc1..aaaaaaaaks5baeqlvv4kyj2jiwnrbxgzm3gsumcfy4c6ntj2ro5i3a5gzhhq Db Admin Pasword Secret: admin-password - Db Name: db0130 - Db Version: 21c -apiVersion: database.oracle.com/v1alpha1 + Db Edition: ENTERPRISE_EDITION_HIGH_PERFORMANCE + Db Name: testdb + Db Version: 19c Db Workload: OLTP - Host Name: host0130 + Display Name: dbsystem0130 + Domain: vcndns.oraclevcn.com + Host Name: host1205 + License Model: BRING_YOUR_OWN_LICENSE Shape: VM.Standard2.1 Ssh Public Keys: oci-publickey - Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a + Subnet Id: ocid1.subnet.oc1.eu-milan-1.aaaaaaaaeiy3tvcsnyg6upfp3ydtu7jmfnmoyifq2ax6y45b5qpdbpide5xa Oci Config Map: oci-cred Oci Secret: oci-privatekey Status: - Availability Domain: OLou:PHX-AD-1 + Availability Domain: OLou:EU-MILAN-1-AD-1 Cpu Core Count: 1 Data Storage Percentage: 80 Data Storage Size In G Bs: 256 - Db Edition: ENTERPRISE_EDITION + Db Edition: ENTERPRISE_EDITION_HIGH_PERFORMANCE Db Info: - Db Home Id: ocid1.dbhome.oc1.phx.anyhqljr5gy3jhqat52milqwt3gq6lwohhacwg5yi4mtzq7c7hag53lrkugq - Db Name: db0130 - Db Unique Name: db0130_phx1zn + Db Home Id: ocid1.dbhome.oc1.eu-milan-1.anwgsljrx2vveliazumnbyvudq3rwkbc4brtamgqzyrjuwfbtx5k7hlqwx2a + Db Name: testdb + Db Unique Name: testdb_fg4_lin Db Workload: OLTP - Id: ocid1.database.oc1.phx.anyhqljrabf7htyackgmsaqjfexoqgrzuuk33ju2q25z2al43tnd5mhhvkra - Display Name: dbsystem20220308221302 - Id: ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa - License Model: LICENSE_INCLUDED + Id: ocid1.database.oc1.eu-milan-1.anwgsljrabf7htyasoe7b7mtfecc7tdfkp6w5knvvufxmk3phztxfktf6naq + Display Name: dbsystem0130 + Id: ocid1.dbsystem.oc1.eu-milan-1.anwgsljrabf7htyat3fsgcftfilt45bgbrfgawroa2oasamavsluwqyr5aya + License Model: BRING_YOUR_OWN_LICENSE Network: - Client Subnet: k8test-pubvcn - Domain Name: k8testpubvcn.k8test.oraclevcn.com - Host Name: host0130 + Client Subnet: vcnsbn + Domain Name: vcndns.oraclevcn.com + Host Name: host1205 Listener Port: 1521 - Scan Dns Name: host0130-scan.k8testpubvcn.k8test.oraclevcn.com - Vcn Name: k8test + Vcn Name: vcnnet Node Count: 1 Reco Storage Size In GB: 256 Shape: VM.Standard2.1 - State: AVAILABLE + State: PROVISIONING Storage Management: ASM - Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a + Subnet Id: ocid1.subnet.oc1.eu-milan-1.aaaaaaaaeiy3tvcsnyg6upfp3ydtu7jmfnmoyifq2ax6y45b5qpdbpide5xa Time Zone: UTC Work Requests: - Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrxivzvgzel47zuoyke5yk36o7mrgjl27vscd5z3bqptmyh3rxwbqq + Operation Id: ocid1.coreservicesworkrequest.oc1.eu-milan-1.abwgsljrkv6jwqtepnxyhnxgtzolw74bqfh5oqlwskq72dqgjpfs5rxu66wa Operation Type: Create DB System - Percent Complete: 100 - Time Accepted: 2022-03-08 22:13:02.999 +0000 UTC - Time Finished: 2022-03-08 23:11:50.46 +0000 UTC - Time Started: 2022-03-08 22:13:16.995 +0000 UTC + Percent Complete: 0 + Time Accepted: 2023-12-12 13:00:03.156 +0000 UTC + Operation Id: ocid1.coreservicesworkrequest.oc1.eu-milan-1.abwgsljrxudjz2qivun6ypupqhytam4axv4wago7nuauceqyapbceysjukfq + Operation Type: Create DB System + Percent Complete: 0 + Time Accepted: 2023-12-12 14:12:36.047 +0000 UTC Events: -[root@docker-test-server test]# + diff --git a/docs/dbcs/usecase01/README.md b/docs/dbcs/usecase01/README.md new file mode 100644 index 00000000..4349e211 --- /dev/null +++ b/docs/dbcs/usecase01/README.md @@ -0,0 +1,199 @@ + +# Makefile for the dbcs automation creation + +This [makefile](#makefile) helps to speed up the **DBCS** creation. Edit all the credentials related to your tenancy in the configmap target section and update the **NAMESPACE** variable. Specify the oci pem key that you have created during ocicli configuration **OCIPEM** + +```makefile +[...] +ONAMESPACE=oracle-database-operator-system +NAMESPACE=[MY_NAMESPACE] +OCIPEN=[PATH_TO_OCI_API_KEY_PEM] +[...] +configmap: + $(KUBECTL) create configmap oci-cred \ + --from-literal=tenancy=[MY_TENANCY_ID] + --from-literal=user=[MY_USER_ID] \ + --from-literal=fingerprint=[MY_FINGER_PRINT] \ + --from-literal=region=[MY_REGION] -n $(NAMESPACE) + +[...] +``` +Specify the admin password and the tde password in adminpass and tdepass + +```makefile +adminpass: + echo "[SPECIFY_PASSWORD_HERE]" > ./admin-password + $(KUBECTL) create secret generic admin-password --from-file=./admin-password -n $(NAMESPACE) + $(RM) ./admin-password + +tdepass: + echo "[SPECIFY_PASSWORD_HERE]" > ./tde-password + $(KUBECTL) create secret generic tde-password --from-file=./tde-password -n $(NAMESPACE) + $(RM) ./tde-password +``` + +Execute the following targets step1 step2 step3 step4 step5 to setup secrets and certificates. + +```bash +make step1 +make step2 +make step3 +make step4 +make step5 +``` + +Create the file **dbcs_service_with_minimal_parameters.yaml** + +```yaml +apiVersion: database.oracle.com/v1alpha1 +kind: DbcsSystem +metadata: + name: dbcssystem-create + namespace: [MY_NAMESPACE] +spec: + ociConfigMap: "oci-cred" + ociSecret: "oci-privatekey" + dbSystem: + availabilityDomain: "OLou:EU-MILAN-1-AD-1" + compartmentId: "[MY_COMPARTMENT_ID]" + dbAdminPaswordSecret: "admin-password" + dbEdition: "ENTERPRISE_EDITION_HIGH_PERFORMANCE" + dbName: "testdb" + displayName: "dbsystem_example" + licenseModel: "BRING_YOUR_OWN_LICENSE" + dbVersion: "19c" + dbWorkload: "OLTP" + hostName: "host_example_1205" + shape: "VM.Standard2.1" + domain: "example.com" + sshPublicKeys: + - "oci-publickey" + subnetId: "[MY_SUBNET_ID]" + +``` + +Execute the target make file create **make create** or apply directly the above yaml file **kubectl apply -f dbcs_service_with_minimal_parameters.yaml** to create DBCS . Verify the DBCS creation by executing **kubectl get DbcsSystem -n [MY_NAMESPACE]** + +``` +kubectl get DbcsSystem -n [MY_NAMESPACE] +NAME AGE +dbcssystem-create 52m +``` +Use the describe command to verify the status and the attributes of the dbcs system created + +```bash +kubectl describe DbcsSystem dbcssystem-create -n [...] +``` +```text +Name: dbcssystem-create +Namespace: pdbnamespace +Labels: +Annotations: kubectl.kubernetes.io/last-applied-configuration: + {"apiVersion":"database.oracle.com/v1alpha1","kind":"DbcsSystem","metadata":{"annotations":{},"name":"dbcssystem-create","namespace":"pdbn...}} + +API Version: database.oracle.com/v1alpha1 +Kind: DbcsSystem +Metadata: + Creation Timestamp: 2024-03-15T14:53:02Z + + Db System: + Availability Domain: OLou:EU-MILAN-1-AD-1 + Compartment Id: [MY_COMPARTMENT_ID] + Db Admin Pasword Secret: admin-password + Db Edition: ENTERPRISE_EDITION_HIGH_PERFORMANCE + Db Name: testdb + Db Version: 19c + Db Workload: OLTP + Display Name: "dbsystem_example" + Domain: example.com + Host Name: host_example_1205 + License Model: BRING_YOUR_OWN_LICENSE + Shape: VM.Standard2.1 + Ssh Public Keys: + oci-publickey + Subnet Id: [MY_SUBNET_ID] + Oci Config Map: oci-cred + Oci Secret: oci-privatekey +Status: + Availability Domain: OLou:EU-MILAN-1-AD-1 + Cpu Core Count: 1 +``` +## makefile + +```Makefile +ONAMESPACE=oracle-database-operator-system +NAMESPACE=[MY_NAMESPACE] +OCIPEN=[PATH_TO_OCI_API_KEY_PEM] +KUBECTL=/usr/bin/kubectl +CERTMANAGER=https://github.com/jetstack/cert-manager/releases/latest/download/cert-manager.yaml +RM=/usr/bin/rm + +certmanager: + $(KUBECTL) apply -f $(CERTMANAGER) + +prereq: step1 step2 step3 step4 step5 + +step1: configmap +step2: ociprivkey +step3: adminpass +step4: tdepass +step5: ocipubkey + + +configmap: + $(KUBECTL) create configmap oci-cred \ + --from-literal=tenancy=[MY_TENANCY_ID] + --from-literal=user=[MY_USER_ID] \ + --from-literal=fingerprint=[MY_FINGER_PRINT] \ + --from-literal=region=[MY_REGION] -n $(NAMESPACE) + +ociprivkey: + $(KUBECTL) create secret generic oci-privatekey --from-file=privatekey=[PATH_TO_OCI_API_KEY_PEM] -n $(NAMESPACE) + +adminpass: + echo "WElcome_12##" > ./admin-password + $(KUBECTL) create secret generic admin-password --from-file=./admin-password -n $(NAMESPACE) + $(RM) ./admin-password + +tdepass: + echo "WElcome_12##" > ./tde-password + $(KUBECTL) create secret generic tde-password --from-file=./tde-password -n $(NAMESPACE) + $(RM) ./tde-password + +ocipubkey: + #ssh-keygen -N "" -C "DBCS_System"-`date +%Y%m` -P "" + $(KUBECTL) create secret generic oci-publickey --from-file=publickey=/home/oracle/.ssh/id_rsa.pub -n $(NAMESPACE) + +clean: delprivkey delpubkey deladminpass delconfigmap deltdepass + +delconfigmap: + $(KUBECTL) delete configmap oci-cred -n $(NAMESPACE) +delprivkey: + $(KUBECTL) delete secret oci-privatekey -n $(NAMESPACE) +delpubkey: + $(KUBECTL) delete secret oci-publickey -n $(NAMESPACE) +deltdepass: + $(KUBECTL) delete secret tde-password -n $(NAMESPACE) +deladminpass: + $(KUBECTL) delete secret admin-password -n $(NAMESPACE) +checkmap: + $(KUBECTL) get configmaps oci-cred -o yaml -n $(NAMESPACE) |grep -A 5 -B 2 "^data:" +checkdbcs: + $(KUBECTL) describe dbcssystems.database.oracle.com dbcssystem-create -n $(NAMESPACE) +getall: + $(KUBECTL) get all -n $(NAMESPACE) +getmaps: + $(KUBECTL) get configmaps oci-cred -n $(NAMESPACE) -o yaml +descdbcss: + $(KUBECTL) describe dbcssystems.database.oracle.com dbcssystem-create -n $(NAMESPACE) +getdbcs: + $(KUBECTL) get DbcsSystem -n $(NAMESPACE) +create: + $(KUBECTL) apply -f dbcs_service_with_minimal_parameters.yaml -n $(NAMESPACE) +xlog1: + $(KUBECTL) logs -f pod/`$(KUBECTL) get pods -n $(ONAMESPACE)|grep oracle-database-operator-controller|head -1|cut -d ' ' -f 1` -n $(ONAMESPACE) +xlog2: + $(KUBECTL) logs -f pod/`$(KUBECTL) get pods -n $(ONAMESPACE)|grep oracle-database-operator-controller|head -2|tail -1|cut -d ' ' -f 1` -n $(ONAMESPACE) +xlog3: + $(KUBECTL) logs -f pod/`$(KUBECTL) get pods -n $(ONAMESPACE)|grep oracle-database-operator-controller|tail -1|cut -d ' ' -f 1` -n $(ONAMESPACE) +``` \ No newline at end of file diff --git a/docs/multitenant/NamespaceSeg.md b/docs/multitenant/NamespaceSeg.md new file mode 100644 index 00000000..6738fe56 --- /dev/null +++ b/docs/multitenant/NamespaceSeg.md @@ -0,0 +1,14 @@ + + +# Namespace segregation + +With the namespace segregation pdb controller and cdb controller run in different namespaces. The new functionality introduces a new parameter (the cdb namespace) in pdb crd definition. In case you don't need the namespace segregation you have to sepcify the namespace name that you are using for yours crd and pods anyway. Refer to usercase01 and usecase02 to see single namespace configuration. Refer to usecase03 to see examples of namespace segregation. + +# Secrets + +In order to use multiple namespace we need to create approriate secrets in each namespace. Tls certificate secrets must be created in all namespaces (db-ca db-tls). + +![general_schema](./images/K8S_NAMESPACE_SEG.png) + + + diff --git a/docs/multitenant/README.md b/docs/multitenant/README.md index 724c39e6..b20eaa4a 100644 --- a/docs/multitenant/README.md +++ b/docs/multitenant/README.md @@ -6,6 +6,13 @@ > WARNING: Examples with https are located in the use case directories +Detailed examples can be found here + +- [Usecase01](./usecase01) pdb crd and cdb pod are running in the same namesaoce +- [Usecase02](./usecase02) unplug and plug operation examples +- [Usecase03](./usecase03) multiple namespace example cdb pod ,pdb crd and pod operator are running in different namespaces + + CDBs and PDBs are part of the Oracle Database [Multitenant Architecture](https://docs.oracle.com/en/database/oracle/oracle-database/21/multi/introduction-to-the-multitenant-architecture.html#GUID-AB84D6C9-4BBE-4D36-992F-2BB85739329F). The Multitenant Database Controller is a feature of Oracle DB Operator for Kubernetes (`OraOperator`), which helps to manage the lifecycle of Pluggable Databases (PDBs) in an Oracle Container Database (CDB). The target CDB for which PDB lifecycle management is needed can be running on a machine on-premises. To manage the PDBs of that target CDB, you can run the Oracle DB Operator on a Kubernetes system on-premises (For Example: [Oracle Linux Cloud Native Environment or OLCNE](https://docs.oracle.com/en/operating-systems/olcne/)). @@ -128,6 +135,8 @@ See this [provisioning example setup](./provisioning/example_setup_using_oci_oke Oracle DB Operator Multitenant Database Controller uses Kubernetes Secrets to store usernames and passwords that you must have to manage the lifecycle operations of a PDB in the target CDB. In addition, to use https protocol, all certificates need to be stored using Kubernetes Secret. + **Note** In multi namespace enviroment you have to create specific secrets for each namespaces + ### Secrets for CDB CRD Create a secret file as shown here: [config/samples/multitenant/cdb_secret.yaml](../../config/samples/multitenant/cdb_secret.yaml). Modify this file with the `base64` encoded values of the required passwords for CDB, and use this file to create the required secrets. @@ -159,11 +168,11 @@ See this [provisioning example setup](./provisioning/example_setup_using_oci_oke Create the certificates and key on your local host, and use them to create the Kubernetes secret. -```bash -genrsa -out ca.key 2048 -openssl req -new -x509 -days 365 -key ca.key -subj "/C=CN/ST=GD/L=SZ/O=oracle, Inc./CN=oracle Root CA" -out ca.crt -openssl req -newkey rsa:2048 -nodes -keyout tls.key -subj "/C=CN/ST=GD/L=SZ/O=oracle, Inc./CN=cdb-dev-ords" -out server.csr -/usr/bin/echo "subjectAltName=DNS:cdb-dev-ords,DNS:www.example.com" > extfile.txt +```bash +openssl genrsa -out ca.key 2048 +openssl req -new -x509 -days 365 -key ca.key -subj "/C=US/ST=California/L=SanFrancisco/O=oracle /CN=cdb-dev-ords /CN=localhost Root CA " -out ca.crt +openssl req -newkey rsa:2048 -nodes -keyout tls.key -subj "/C=US/ST=California/L=SanFrancisco/O=oracle /CN=cdb-dev-ords /CN=localhost" -out server.csr +echo "subjectAltName=DNS:cdb-dev-ords,DNS:www.example.com" > extfile.txt openssl x509 -req -extfile extfile.txt -days 365 -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out tls.crt ``` diff --git a/docs/multitenant/images/K8S_NAMESPACE_SEG.png b/docs/multitenant/images/K8S_NAMESPACE_SEG.png new file mode 100644 index 0000000000000000000000000000000000000000..594471ed54a6c980c9e63ab2f538c40a32a5dcb8 GIT binary patch literal 269014 zcmeD^2_V$j|IS1k38^SuNUj;n5aY-&%#2*)s$>(XqFT_x}wuqHJyZYjBYyOj;Vc zZ@Rvj6~dmxEGqlaD-pGXrteMsHr|N4O zY`1}-TrBsYwqcyj_S$T-1*0I>If6iAY{2|X%@(z9@&l}^17f<=0&&E~91KK)AEwF= z*#@4Ge=ubW8#61knH{*XHvlqYs()XJ0sjJ25s%6azUW5TX!(~_)V101jK_yP*Idz>&tWfamLjU zfV8n_S1{|8*QKb4O2Ka!6*HVaVv7W|j`b`|k(8uNuo(qIt!COLlgFQdBB(<_f-jGs z%wa#lm3aXK`#O9uwg{{_nFs(QnGK4Wc_)`ofekYkY_nm-F9DN~m61{f^R9#eT;3Kz zg(7%jIrRrTaX{JFW2fduN{L@e2_go8NC74c5t9bvREF?N0Xd;O^#DK>6oXltQz|!a z=~NS+)|C5$hGnMO6qB23YF3Bj>QkN0DwB|%>hEJ+Fg}Ge;l;`6Kv@h+LHOSx+W;md0ntJ@ ze=G)9R^-Rv->KIRQ46%TX{ze#Y*SVRZ|JJ3?NL?RrnggDi&_teubHZ|%BFj#Hc1}h zEd2n8ZE{z0r3OXz&lWM1A}V)9A}qdw$}EM)OHdj5EFKS~^8MLNo=PM?4wI)yoxc&= zwfAgO)=(8y*`~KmX`8O9sJ4#ko^5*Cdqj1OboEpbvPnfU+S7S%|Fc=aSZ;k)ZS zU$=<=(BJsWCI$#R^&`8@KX>>;p)zDg*ldkw9;;ZEQsoC%c>aE?7$`FT0(SrM_fT+n z8k0Z&|NOWu4IRIp|Loxe#_TWb{%O3KvVR7m23e z6Q$e=1k)X+CDosaflELrITREhGP5D4|B7sK>6H|2Y5qz>JNDGIf&P-^J-#{n-z>V1ive%ZJ`yp4-Nn4=LObV=qhvsvEX zP7kv%o@xxt=4X;WT+YKZYyVR8cV1VgrR5Yi`rZKn6J1s1J*s-T|BEj5P0W`X7n`wy zKMSB)T28q~KjXSEX_y=gq!9j#T$jpHDO{I&I|~P)RPfG0LJD{OahUFO=;2F9_%ZVT zF)H=(>YLr{AoPv}f%mBt2D0W3QAQ&`93N{#PWPl_F90w57c&sP3D5ooRz%55q+$gH zN#y1c%|8i}0A^0+=d(n!H5TgtN&v5?0B^d+M`V%sIe|2zEd7L}{3jHsZvx7v4EbkN zk0q#j?$56uOUxGp`g&;e@6Z>3NAZ`w0NCF=zxlm-64VT>pI>44YDngz4EU%fe7yQ* zg+T}DWMwtw50QX;boCuV17%?<-cdDJ?Nisp8j&L{c^9(C^dhe$WZ;wvnb%_aJNbROy<#4Lg;sm6Hv1u zD4a~1%Asb#aYj(h=44Q6aOlTjWni)Sr|yW)ybr%v*-y*LzCZ3u%}uC~{&{)WSB2{FSumZ*oO_Em&Y~j*tTR-}8iMzYr|={)k3i#7tI?sZ5q?_0MLq zRF!fzlcjo!e;^+Fxq(InHf3qzKZ$928b3~4>c>Pi87f-+e5fY%CydM=AdiMZ5LD8h zM~2M140A@Q`f+F*23QoCOU(-Be5OBr6L+3%)&C=TG&3mk{WcMy!jl4D>g{ZDqH^ik zocjk(4VN^GkpJe2+xf59Rv!*_~|jvh%+I|ET%{g}+d5e?E3DNnw;_A@^|Dl0`u*^sXu>D1e@P}`-$~uY8>bbdm{h) z`ZFcwO%-pFltIs;cBnMPkv^N;e?a%M)O_^*W2nzy^(|#_%JlBa(4sC69YB2w2$lUk=FpjG8P(&BiiA72Gn1zos@xVC<`NJ*yl>QS&=3jPU&6n8y zxg@2M#P^e#R9--pnbiAP;Q3jOfgqo+Kvs#qBs70I_(*)ZFWt`ppul}}Zf z?)u|H@DgZd&c?pe2Mv*%Gq*S-3JxZsih!@aML9-f`t%}7Pt-wD8~L`Ru;~v3;~)=e zy7`ZpgmagEPxm*UnEE(wi~|Dfv^|rJ`0>F!(*QcvnmTfc@;Q{9mQ|FXfZjeXQ~MJW zib7jpcG^vyT{O2Xq>a58%G?G$eJGXK$3K6y105$MIJg99V-7gm+^)7Eoy}Y^-`K9j zAw7GH6PkQ((WmWgR8iGc-m_C@>Wl!G%0^}FsnJuR{Kc6?BOJgG$zdAu7y*0#6LW}h z#cZ>%v$5B4o<{XQ@d{{n4qktHPgPXGAihIupSJ$#{d8~hVbo0b;9`oN0p>wPsoSGW z5A74Qp4s*L9~ihT;xl9dz%U~N!^8nVUS4-7(#T>Q;K4}e7(4bJfTZ|N>S5?IW^^nMX6pOeEMl>obE zJ}V!10>Ek0vo@EYYSZM(qLyZMHb}D3Vu!LvQJn@K#loEP??4CBjZK?g)6TS6n^w$w zh8oK!^J`--{{@jtk`IeC)j+x7OJaNU058I+wTn?QoPig&hWUFKh1oj$K; zaK3E@D0BeGXL`$`>5UKfsD9xrMzZ(!bA+9B^77nmF~7%Y6Z0MQ{L#rwKI3lce8lO~ zb43A&=G&R&`@8i;scvG5Umr@@6noAd*7G{{sf)P&mDJ&Rkz>A{3#O4{nje1i?pS}_ z3+w>gQ2-&Qj|-h~I+J~ZpLiuc`e>=n$GJz0QX8Rs!My8&$y?UWJ)diu62KrnHvhl4 zfae)El6>Ctv^VlkeC4NK{pmf>{H7h7;9`F7`8Rr@_aHFyxeY%y|53Dk9Y)|Sa%0m{ zkqV|y8u$`?EG?(x-fZzcN1h%6(?NA&&IhoW=Tv0)T*7}cnlHMNDW5Yv;h;&%yMMt+ z4-jm=Sq8<^5s+^g)s#YQ$LFGPX*cx6v@2d=Wtd31#5f?jQ<}j@t+5Z-)@M5 zZ~JA4&px#Ei#}#R;9q?H-3;+*u>3N_XU@|7;;=pQjR`=}ty z2(mBfvtOI_uRWiKSj^4Dz8xqtUg7^29{wL*7=3gL{{@|52A03{il5xW5B9P_kac`; zP6deeV1ToOlF;&z%V=q9seZfzS6{c!uoj4&b`B_tgBn=*NX*P>7hkl5uqdn zB?pv^Jr>}Z3RoXNULpWi3P8XqkWwnsO9p_ShI$W*$koxnQ)~Y{4*mN#oxW0J>>>L( zDUQ8)y}mLt2L}G&e&4@yQ-Jw0sxj7P7AR+cGh{#Pv@_Vi#sX^%N?@4x|uZNfPtAwkZHcSXfbNO+^C!!pW7hP=xCFpH2GI zGBg5dhP62|yE6&Me4hS4GXydx0&@sXo7N$-VsJxQ@gW77%cG!>Pcd|QVkobXxfzgi z{Qpy*($vih$lwCg4TVThf=blS1E^(%!2X3kVai~1vyRmM@;E8Al@UnfSH6fmb5qqM zAU0&Y`XeNvhU>nxuHpMo(+p)ylMwlqOdC_w^##hHQsGC+o1!Qx;mu2hee_v#MDE=8wXNOd|7zf@evt*&BX- z58Z?u9^4MNCa@jJ!6P7Lq`)PF(@|qR$gnctRodwH}fTyY#Qwqh5 z=J9p@2TpIKXdz-!5|U(AIm2zmAmGquO2HrKB2$%h5NI0^$|fhDPw6sqSn3}l#aDF_ zCMu6>i;e!V^76eGWzNC!d3fsF6rb7YX`kDBe@G<2i}k(E8kdz8lbO52 zJrqJ+sX9w*mym>sQ8m`tnmg=Mfog?70is8PY)>*1`#R?R4f^PO*;?em{3w#~AGO+N z4#uEpKu`!(U!MitpTum3&X;QSzd-hziFHP)`FDs_ifX9N08@s-zh(hbLQ-0cO2M;9 zRZ@bY6H#^FAAnTnbHse^dY>-&i`jIxPKOZYSPek`z`Q1NfvLmnL0UXSl$@B1LYtW* z5u)T(*N4o210SY>azLQXuqd=BSUZVD*v%IU__v(j|IhH_G=K~AKe6U8LY(cgtd3B_yIj}bV1Va3^ zZvy(ucZa^Tg`D;0$o6^3v}!=!Gh<%WVCDsIF(-5SuVh_=^ym4Ktf!lr+dMFuX0mo? z#(<^{;s+Juzr?7W1MrI3&z~hYuNa+yEfur?N4}h*{Z&f&&q^d<`F-`{!8CqRm-+O$ zv;`D051;r^w447dEdTS`#b1+z|CDr}VLLM-N7jG1)~=Jow2Q{P&FEb0xRxR z`61KZb1>kkA2kURZRTLDiLyYsv4hLALoOCq~>UA`=0!|s3X z&Q-nhqPOn$j!r&kaLhsF*o4AhR+c4284aE2#z;*4kgHdzt7Q){rp-eu>5(5B--!zg zfA^mC+q!+p6MmmqcF$KERJJmF{Nb0J!B_dNp<6fWw?btro#L^m8#m^(;LVVeM9aQp z*6+z0x}~HQ%^I4jvl`sWu%XXs@tileE?_@{FI{7Gc+>1&)~)eLww*nctqc;sdoy@0 zkki#=pVi9}T4vkPI=(yDo2O*hcl#viy>0c?`Df4nkccdUHilDyku;V8QA~xk0>gOY{A?o||bmKJ?w4 z?dJ68_-`?89YVFv4IXOYQqDcblGZP1PAwBv3KKhU+o4fOFVT=)8$P_DCe4;RwA7zV z__sFG?(X{y1yOPs2>1$KzVditYcb4=N9yo-v)h)%E2Eq$jV!gKY%7c28>Dv%E;aEG zn*zPV2J$R(P0ey2_bZrH$>SUGF>c54x#NRx`rkhnkA!Zl*`Ibegulns^M@jmn^hbuOX^%wHBHe@xX6OJx= zk!S?V8tus>HTv=OO^o(7-FYqs8$W)hEuY&&r7Gz_Ak8&V@vOe18U5i3?ngo|vRZ1( zx;zgRePk0KePVmy%Eq`z`-cF8{Z9pB3CGyr5E%tdk{X}KUG{PPggw_-#B#^bF!-eB zV6|0K5v){_?gBb>b$@d#d?lBC7lv|HNZj4zrmpj5FP&rkcL7R zPV)f6TYgh!982L~_Zyy+)#*n|_hOSOb!qk(>!#o7M{)WTu5ylICjrFSyiELHlro{W zKAVSe(Xt{lf!N<1LOfcYg|Y^+^=^_knfw@BLd&JgsR&6yDz5-_Xn|(%0Xkw&~4lePuOR7?9)}@6=K&d$qrzWlL4uu#T zgdV9#FenWu7UbviB&~fGB5XCr)iZK(Xl2DPnL!X#0b4ov;X$hi9V`272Ds{a)j>c3 ze&(m;1cX(s0I;^>_q^J@1|VJaLX0{ABNeL@l+exj{)$>q$=bdQx{_DC(*DNvoDfal zLlMGt#@AX8-%s0PS8dKb=G0jk!NbaeWvO-;bck9hbvRs(L0VmhDOHg7nTh9c{)eY` zHzW~=0>|m(43dnEFmInpm%Ahu$f1+&wgtdSAV(wCH5^7f50y%oY<@Zn@$gIN$L-1E z!om43;@bilVpVSUw=<1D3>m1;c5iA5Qj|Vo*uG7JD0DE(#Sostf=j9|jaB3u9qfh? zTVHdV;8Ie2J>&6gPUEuJh?_iQ?%Kx-D6urdm}41j+d{*m7i_pWBRIp1Q|_VrZsGdG zb4U*tz7&jcq2qkY+01#zSk!emN($ncjBTtu z8yjwm_2VQ>@EH!OTz<;4ytNi16%*Jl z&G|v&By;GoQl9Y6q|B1v(eP@5E=}BB!$6xIQL&S&8GB4z0Fjm5(2c(+N8&vtq*2)+ zSAP0+a)?Z6BwHNeIME=Hj!a83GOIO+X68AbO$C8v@!EWMZORKs!hkAH-hy*aaruQ> zTj@*;PAvR=YY~HpPibI6ZS&KLcL0rr)d?~-FBwd{#-&2V4je{T3;D;owyyZ_uI$n8 zZY;a}HCQmK4dS(;dFi~yNc~U3#A8!}Zfw!mpHkyP&!iC>q%au%w82tQgEj^I;8xSd zb3!LF)~zmCopi?_LBIHF%k5R0u?sUWZPJr=w+Mczf=gB6CLx-?A35mS5@Kq?NveH3 zN@@zSiY@`bH@=}b_}aXf1>RQJBs4M32uEFCjqBSMy!pc$RBlB4n$R8b;cdkXJO=Of z-?gT-jiuFf(%RCmvOa5A(pzQ$C#jM2cvIi-Rc{P|1@8KiyZNacX3NObEZm^X0+7N| zWZZ|)dij)MIsNZxMoLHQ#n^XPh>cE+k75Fz-3@3jkGEpR`NNZHU==5L_KX|!9c#%N z>Z&gPFbrlo($kfQFV4<(8;}r!a0ZOYpajB057luxzz)m0r$>a`*ueQhqtnc3MSXxq zq;&CzRqCv8Mmd=~5=^-}&aNvs*cu({@Q}Ingn%(C9HrqxpXxSnIQ3|$0^{9BCWK(N zLPB})C1t<2Ntr#*mXh+&ebo&aZ85f|oBEMiLhoLutUvMFilw32iFC2|9DQSmcCT+= z+R1*%jaAo4R$QIHbWl;BOTbUu$VreaZ(`P5#mKyYDbd&_f<=uQ|SaCFQamro4NOXpuw~SDtp|WM=Vx~d?^-#u^P4M!wNB5Rp zy4}XXYTt8>Ej;v*zl_M!r&xj3XMV&mr(;Wz!J@H{<2NrqiWJb7f6GCi;Lj?vDA+e- zPd?WezEh1(5eiK>2n0aZwyksxJf4*zs%zFVj4{H?#nP3YV}E18RT^2&LDn;mZM~9lHcaVEq_v<%xT$ovgx5rjdr#iv#87VHgN{mp{gXr^A%Q4` z+ILU)A}1b=jB#cqR+Ze8$m<}aq7v@mWYaaCHfBZ8msX6juG+lZu{3JZRF0&d?KQY_ z|C8TkR5;&d|ZNlyQO0qyliD-W!8B<{vbl{jx z*UjS*p0Cyi2_~YPczJwodGP(-tr8u=&Jch%xox>^yTihalV&GeFJR=VheJdDgZlnLijQvG1AWo#=@UdwBJR+#1h z6HT_vM`#B@o~A5MY(w9w;}LNOb`s)mG+EqRc!q2y9@FDEgV#=Ds)gn-G0ET6b@|p_GB2e3Wd)RKlU4+c%(V)HeYMTFKc?R#i@alt;5x@*BdMDAL!i;my&;{*H?C_OZO1g0|395=;3$#;KA3Yf;_{N|W z2ewCV1Rv*2ZjbNAZGl9*NncZGdUl(s$jGn(ufdHi3x+c2!%S))zc%{txZrh$?fv22 zb81{_H{xt z$ipu{nOxsDOA|>Zqm}P4Re?{U<(&=p2gxOH=_kK&P-(b-po&#ovHs22c#PT!LPVo^ z(H3|tWL>AB7d-zI*S26x$J>FmY&`mnBxL=A#R?JUy|+we>?}Pfw6cf!FVRR#}0&xSk#VuruAZ`EgUw z;`U_5Q=xB9!?iCCXAD5?Y9U+TU78iwpMzK)5G&yzJVJXFD@=`=p*x%a{QBNJvk(ADWf%!H#Oc&10*Yf*F_u}n!B3@#Za z_Y@{{t!tKqf7p_{)hDKuraXsku#2%HX2BLBjl)AnIXlsVoYzejxpa75dh$Gz=MHEQ?-$CSFU(QMI3v?w zb{;-Rv(hAkzJjL*8XL%?!;&ku(R3m*HcVlHMy&k~SIf<=(vhx*`|n-q$(QdhQ&%$^ z&N`YtT0CGBWs)5)S(#MiN8H?n&scRYGS_R=kzHfeZr;7x4X14Rs*fiNF3T_4o$ufl zTxadk7AoRC7VY#f!rf7`LGyL?dYQP~gS@$7BHr|)=*IBFqYjOBZyd1s^m0g19TMd9 zjT|=ax(8hR&-s3{Wm6op+ItW;_I|e^Q60`H}>9tSlzMf}!g5e5{VjG#q0$64C z2QL!pyWjiTmt*BYOO@7JtKRs$eOH=W_13Q)b;WXkBLJ`Xpn3Q94Y$UKr~nt+r5#b8 zEIGsPG+}-6!SR=rtzsAEB&!Q<=_f?D=9=J|4q&^-WlJ{SdcdA99>5zkVpaLLYse&x zzfW8?o^_v;oS-ebOunK$7ubFy|_yy*PRcP#V42~`*7A{?AaUq7gI;E!>) zi;9q0VQ^Dt8S3=plJ5QY@#$eqH7jAu=||q(73eOzUcVx}z2iCL$V)xk`|2tZA<*hY zbJXVRdvenuilnIz0ZosJQDjH=nX>?D5R)o`zYq3&%_+F7)+VrB9YwofXtx zjXbz{k)3@OQ)d_$!0DR?mP;ZY-|dPnoH;tFEAQ=u=TajU9?L@`j~WVyoed?T49jXoY|hJ$jTii4@1m_~&+uK)>cJ143lghovf#9`0}Q4mh9-)ax>6nqo>wr#tpme8I`E|M zEW<>9e7ADQKKFiN6=TZ85H9`ZkY$uHDv14+?nqeAcI=RIzr)~p>5yNM=CO)hq4L)Do9p@Z(WzTTDU+>FYp>SIP|r>m>eHLUmH?#axMc5unNu-;@;ixz8F#T2;a8e|+< za^KpmqT4c`_|&gQf-Y3atKJV4-cxGpESz(&Z~fr7TxlYUiI;&%7BLU2r`YepE9*jY z$|_K%u)@BYT~TyDZPKgv3oiBQKI(5eX9DTSnAm81-Sjcn7}JA;sF)2ZWA^SWkISHQ%OQ-QLA{dQS+m*6Xo$iru^~VIT{6VVrCwenhwK2tP-2zedFiB* zQ{0)MP~lhH4LKDH4U?WUK&;nGCp|qC_8)q z^WxXPL&tYrdR+zx$~*0-NozWNdOve4WC(RP^61#MWvIHMC?@v*|wJ|NYz zyP=ru>oxfM{*&WxTUxiktYU4JpJJts#j8$a~Hr+Am(9R|4WiG#5Vaf)SO4>_K#md7bo za}@CJ%VYD(c5Bhiq>B}%3Y9#zkrZBhkSO|BUNh&?hL{X$NA$Le+=h%oHt#PHtB7rSNOT+5Y7}LaF z+c8XJ4V}8^(tb>U-?~4{w`j3y-a5Zj2G6G~Ob&aKi;a?mjk}h-=iVH^79;ATn#!_* z=Wui`PX$lcw#F;05ltfdMSV)oU+RU628fAInD22>s)n^?Uk)G8hr;3yJ9*sLIT1+y zoSvF<_@Q0RzA;uie7vfZwIlXw#MwmczH*^`ZEwneh88J`?cnpX$4W~#1{CDT(l}`; za_Q%6!DMyD?54l9x!wj{nSs8Li$1pgeT68oYvCCFhQqbU_O-}kF?=TnCXo`8bys&E zxNF}ydX|lol+CmlR`|Z1p7=iE4WG3zvFp~dU4&EjPP-|PeX~w;$n42_=g3zp?^zl+L2cAtE4o4|UrQ${cTcG;wq|(tXLU=$DV1AHoDwD zin8KziR%*YazBs3LxoBrUYe%~u71EVOdPh?fJec3h6efkTZAViC%r6-gQ4vM@mo(I zOLHe!dvJX_iF#JUT_>-jqk12(v;{jipht(WP9i?YU&aOeBeHlUDhVMUW5zfNUrpEXCdjxk;DfW zld@lh3B0n}*jFNmulLCDpk+I)!< zu0p3cHqa5G4g9G!+&vPmLk*sK%fWvP-1QH*oc4EbRw4v-xk^^vDzeglIF9Li+J9Yw zsjM+ibL0N%Fk%iy-TwToEWXI97O6uX)y5{Q6PJ$Rb*n>=#G9Ll4Xv9IVZtG~W*@v+GRDaI>OjRxY}N19%?wyjKgTsawG zpwUt6p1j3Oztc9#i%ZO=@=3Ei_f53)&Aqn*_^fj5PAg#F=~P-8$kXK>NDxTx@z*zk z9dCXI3i_rfR%2binc^k=zD0ZG zn1Fd!el_W~V+rr6QTd#xq3GdBQ}g$R!d!xw77$yuFXu0cHYRlN)p%**sbtd#W;ZZQ}s(Ks`|_iCd?+g1`H`|*!T5vYZ&j$;!%axaEnt&3794ICWcIIX?G{U}L| zuAxA+(?@w7GtP^lNVK<)V7dMby+NO!lh84bO!SpPo@!xIw>xTklCfWZv7hQ~H&@Tv zv7%hdRN-=DN$!zM_Nw@pR30ypO{NSJvFh*5^u)D;UYtbUp*6VLy-&xDHS#grh zGRJ6(tuiF&xdGvJ#wEf{oUD6mPVbv$cSLD%>q(k7jmkNTpCX$%p2nDlg)!tX9Tj`X zsmwCg9~^(hqwV1G*9i&L8#i~Kauv{iU9+(x*1V*tK3ZbEdaYD?^McfrSGin5#tV78 zVooj+>kSxI6e+mL$QK!SNh90E^6_#FclYp}dw~J9`!zVT01@ zqbFtbuVlR2m&?~WDIibq5!;Si(6js9OIyf6H==^b_no-eo|6(36)lPA48nWFC`YqF3W$EZM@fvHeul?Lh}8 z_)@+Wf?5uZlZp-A-WsesPmH>XFN6X()nR^suxNu zmF;4QYNkV(-Tu5uuS*ysjJbm&cf)GN7J~4*Lu;4pAz;kkowT~2S8~7||LQC=pT~*Z zs-(ip5lsP0hPZDxjNdIn#@JyRhh183FaF+X{gD0wJ=QoZWZ?RqVF}rQN0xka2FbY>#@tSC+H4{BZdm5F zd+F-brKN!xd8FHt*7xhRmzOx#;nJRFTZxUG2#maq?qP~UTHL-VQl_``Z6H0d?5TIl zPE^J{J45zf6SKmbmz4z)Yhjje1?B8z_nCM(S8kfkQH?KXBwclho&mwiWMhxsOqeIvRlxxy8AP#Vv^tk@MTV$o(`(_=H*^(_*RNQ$=qsJgq_0|Cv%dDI`7wH289)^?W0eq>|1M^ zt$yS}ao(GW;$v=MJ(A|oLeIj`Etch?#HrDh!lD$~oL-*}>lHCaCf1?3C&uNIZ**W< z=p!{-Vodj9t%LSjO1`>wjSI)daNxMmiCf2V9m@8kt{o`k&)G~0QgmSoU-xKPx>7Oc znFvgduTkFWEx#qh$8{<B+B4GKcwd)JQJ0P_hyDmN5;-+rOSX0y? z30#P4tyhhiN)hQxhmwg0?_q^fxxJ6vNi;9CXY);yI`1Oi-gPhT_`93Fd{&E{hl_`r zj%h0vo`|(1q;KZ7$`QQj9HS;W(uwhZ$pS z_A4;u(9~~$*qvxM$=aUowf43|;AFVIzu|S;7$LbVp;7aSeYy|tK0=80y`#Tx)$)#s zgS0($JY(}!?ZlLaw+Ku%%3hAw4$>*!6DORZi>*s}uQ{|0U)4Z+Z5K_QJZ-c*X~CTi z0RFiuhow3u|UKr4RH!!;r4P? zKCkj8fjSnJnh+UcpO~wn-bsluFLgJ-k$x*Iu4wl3^c1BNAR=DoTbU|+}oXeHI zs&iwqE)ecbdUb4}`4+{ac+GpEUia;aI4W1ecbewbo`3FXByrd5O?2ytqoRhaIluvE zyMs=#d>x2@z0tt`aWxY0-%S}%Q%#E!}hAzPCkjaX`Ja#$=dCu#WPhBZF1J05iGpOXuGjtruwc&Vt52_Yae zWWzZSU|rXK6E!EDc!vat^KPK8av&!G&invJT}nE6H6e$bLOJtW^ArSfgMTxAbds7y zKu$Q>x(>vX1;|#6!#(fyY_V}K{A@{vTc*jM}`-9jo10@;O%mJ4N`9H zbA~O?i|$_r5>O>KJNE>DwB)POdk#pCyv;wgJJAqI&L9;Q%N^+u?yAe=4m;>}p+i>0)lX5%2A%`ojSmo)OVw|(7%i1u+}siI3yM%#;YNAtVZ zW$S|DknUpo#(SQ?#6a??8>R>3W}a2_@+9&nr{4|KW!k>Y+wm!n`osccZ92Pit)m*b zZVbq&wXRGw;#9CKxny2=I=_(53+XBDddx1r1>}n!TJ~hu`Heo8RZm%#9GBv#T4HKE z0kTC6@7u4cOg0Gu!&gu;N*wgWPp&u3Zvo%EI|t{wg;)eT z$y3CyfU3F6Ol&Ky>b<5uwib4J1D-JjBwm))p63H!5JSqwzHvH(_9^A^BWG1ES|);D zVHM>AQtmPX&vvoeI&!w^mAB}I91j68kVL(I{Ger@QmXmXhj5T#SvCsS??@Y;sds;B1>tRm?2{B109H% zq-)B?9YZg-wIv8wkyj4*4QE^2K#1$*?FwCgZe8DSc#3UR($fpU1?;p{OP0&nN9|O@ zm^TTia`p)77%R&&*A#t;N23 zv6gs!x9Aq1-B-3poIee6^9hNmB@cgRU$2vDsj1s@3y^B71$>;a-(#5mMYj78Z|v|8 z-1@~Dj9O7xWLIrQIZp46#m?ZB`C$+BKxXS$U}W?#dR-D{@dc^pSt2I_TUzv!4|*KW z!?|xDrrB@^>x%5V`{Y&nsx2-MkT3t9cd)NJM8wyrKcAatvQt&K=Oj)3HFsPi3M5*q z7adyI!2Hlz^h0~OVCOBoz6ke59U8DO;O1}uw6GxTv?CiMkD^t}mBvSYk$e0Pw<~XT zY=@ipj*eeT^H(1l)>t;YaDyHpcepjO6~SwqXRt`yw5K+MoG9<`a;IQ~40{e(yV2@Y zn%G%ntigiQmuZY#+2ApBKv#VbWZ0`++ZA#LZd-A4Z{?b{826;7Z<@o{Ou9?%!{6Io z$+Eew8-JR839>hfs8z{A^cbqMs^5vb zD_6)THkR<<>bDRxgB>?}j9@v20uY{p4lS-A6PYyHn@e7d5FWP5h%UB)C(Fj`-6g-G zbv|5eJr{1cCz|Ckc@8}=OD;-C&N*Cv<5D`dbYPphh!0=a&=6Srpld9EKk#btSTPQ) zh~RjU>HI+BI$m&lPWUm`y@wl_bvEX{eMTiQhI16Vb283^d@l8WSf|7%f~ zbKM3ywi`E~pNq=D7xDUD_SU9tEsnH;9~y5447`TqE8K;;$8{B)d<6a)iz10;woWvU@!*bBB2-;lEcBe5o zOiw1~iNA36;hwb+#4}DH6Kl?&Huscmc(JSBv+96z?W}~=d^uvim!XcxgP=sJ{pi4 z2ADopn65l`cOh5btzHw9f z($WyO1V#>$#^U9&4i}B0ttvteWFEN$vY-k6S&1ETBje4m3OvJH9M_XqjAjU<<+O`mm`u*+MxQLo|#9wi_CEeY1Un8;l>lMcD6UewtSoMGB8> zGRBsv#%Z(Q>d!-by@gac$K8r7C-?1md^T(C)#c6m6)I|XdZ&~G_Hzhosl2&o7kE}g z2j5G7Z|mATL6pd7`jPG$5g$dcpe5L?kjo~nqO2|Jh5<&97GBomoc{J@^u(pT6Ol+! z;ad$RadH5AJLQx2w_eA`%eJtr#l&wktmxt}WmzQi1gaS0YK7A)RU6c(Xe`+5Z|U#S zS_B(E!Sik`7neAO$FqYtLj+%`UUYBHXh~ugCSSpUW3{dw>2<@iQdWV!!XQ2rx;rB7 z(S?^_nH8*EuBfmrQs)+SUC&zbx}O^_9^nXBx6}lWQdO@F!)T5TlV<*Umm|5tjQK4# zgHu~9 z$z9mgx95BF>^Wgaul&GnI~ujRRtj;#zD%WjTQnRqn*`{6-m*-d@jYzgb> znIV%gM2s}`cp|d@UeWo>A4>Cv6sx@aa=o~s!8*9`rEU{+u@7^=q*^*fYa1>X z$znGj;FV75!tkF|I%Xq<1YrXi{0pt5T3EEz!{f|xmzlbo8^-Kf)P)U4G!sa23MG+i zN4D>!`*6UNop1shC$(pT&Ux_z4_=*5YT=L;4s}f}m0?Z2DpuZ?o)?N^SNBi@QJ74XY-YThq{hT?-x8z=>N zcHX*Q#I~rj6s8uFa6}`Ld5DpttAA}^v;Ssw;|2O=j)o(C6Cq(07sMz)3!@v9jh-iEM9SUY~pKExltg12HX=uW$?--?}tC%4WtL zzAQku0WWwUK$r#7W7HT(h<2{GAsCL@gx`52kv5iME+s4t+yyK+c#HWy-ef4CndM+<4^+j1dKVGvU zMA$%UBMm&n%y^TO;HB^i*9GjR{Ogu&++k7PivV_4uf~Qm0S(VJnp(s0Z3{{hk{&#} z8g?C+Qt8LG7M@&BP~`bV@jOlEgbYPa-JE(r({wJMHa+ZVdjaRp9w(samRnSq#BmTd z!DZqTB*GQ6<=6FPY&-ATq;DhRuH1QDg~P?GQ7a2#$D@yed+*{WUfzG|t+np{;rf%U zt2HOJ6$Z`=IX!uQSxd(Ljxkt?Sj}^(bu}H(GDz#r^Mh58SA~|pNH$R{3`7{eUvQKs zzGA8N*8WhJf~PBea>HquVbNuR4-Xwephxy*XsCoEm_)p#6x!Z%`~x|jv>*$Y_DDhdA}7M z$5|F0nmBlxS8k7^4bw6jW?baOT2I5P;&-=>3iTwT^fZ(j7*8WA*5wR&N19bgobrAv z;W2(AT|==!sQzWCRoT9x^yX1#Jq|^QjC4kL0`2X)Z?cE#d0Zt|HOkV)H8|vZ{vUgP z`4;sTwvWOvf|MYQNJ>fwN;gO=-8F!SfOI2DBZ7dGh)8$0bPp|3(hbAVJ@nAe;(Pz@ zeLQ==+JC@)>w)XDX05BQbDh^nV9HCq9Y*~3IM)L1xJi7p)&2Ca4(mihs)fTyJfjg?MHUWtNFmq<07ys!ec-O*fH>mMuePDf68=wwIq0 z9skuZTK&-($0yG}x16qJ9sVyv=98J)cc1h+Wga9Sm8$b*aR;g6;Gwo-g52FAJYR zAw>WkeDx1|Yr-0cQeJ37nWTiy_Gn%dVg$2Pqp2Q-!}S7)&$qsV<(|e)hOtQo-=o$1~Gz@SaC+A)O2uGHkrfh^t$V6%JQ{>qhPC_D&IAM$jcX} z5@;0VO3G0pqMGxj&hjQMm^wVlOCrsa{9Pb=^QPM2wBO(yKnrAKwq~VYFEb!z9 z7HCt@pF~dVr!zhpe-((+`3&j0`u7WXNX|N?yzjoaxr&cEt^IR;D&Oy#(~|3$BQayC zdrEPsd%9cfG*)7WmL*bT;#HL#>(hkbdC^H0AG)*^XF?V~{Bg#i>HO?3=2KiGto!l# z$wg`8pg*C)_J@B&-?o_AKoN)JfzZUBTD_mnKP@&y3E8Kc*-gt4&l{tqht&bZbOs*M{DHVUDESJCT1MRw?6b0~JMl=ZDOTC(_pBh`G*@OMOM&qFRE-d&04 zgsQfFjuDHg8%l}n>oiE5NnIvK74S>)D)^aQ=pGRlsYeJ058U)s=5286KL0RBU$ZZ~ zG8@$B!fnYpP2<5>diYQRF(&`NSb*hjA6cx6c9+zGQE~81maUr`Qt(ZUI?u{5MW;oB zH_I$j>YIiAn-9Htntjg9`fu&k#yBo7Xq4xxGwP~(OdyN2dD*n%y90io*E)vwO>ub1 zF9muo^g5Q}p_PxnsL=YJ;41hoec|z)p1EeZ_3i#{Y+&keGyb7pMK(HFZb2LQB0td; z!=Z6k-~pY)Ts^^f>@o&vd?cg0Pe@qbSp8FNAgvB#_7-i6?5svrRG&^nH<+$BE;OMt z%Vf2%GOauUa!&m>j(jv0;b?OFf*6=Ju;kV^=Z6zpEpd7KtqnsupEz0<{Nf~_CVu7A zijgmFV_w&+Wq#JY%N)f%bmbf=H&0|CMp}Agx{T_my)K)vBzY34eXa?*m5kO|6Q$-L zrctkx-H8Oz4wP8*@oEb~)(1@Md$K1{dW&ZD%0CBJ7!_~lWNw-_3pn!(2-|2~4VOb^ zR`{~?zWgdY6;e`a{=xwC@N`BUD$+Mv)Xc`U`%wbgbnvQ%#JZr3GqmPMt}oW^xt2ms zFep;m)P+#<&zVa%HlEgBsAV|nZu_0Cj=EO2v2Nh~y7Fm^5|qL5Zb63CN7@V$IyJES z2L-yBHy7k3-4$ml5RU-id>{0Mwzq$odout`Z&F+cSQnn)X=lVLOmS4wj^ z$d?Y4;}?K*9Q+tPu>3$ijO%^3`u6{Ab+6p-lt1~XssvVMEp|)cm$8aJU{;1h*?X(J z_ZsjlGq^b^gr0oE2~+xT$*M8I#zi70;N7a)lQeH2%Yi%*eIH+3*#i#f;0LU>hmB1^ z?$c;$5{5*8Q&!2mc3L)?^72bBLztYhR&M&c|+i^;KBPf>xeyz?eCxOZ31=7m@U&mkEh+rlr0 zzq$q5wAy9ABnwCW#mHOjN#2@u2{$@6jqkrlE%lB`70ifPs!wUcfsOSh38RBOJ6o;+izH0 zm4TgAF}W6&1APa!?*zxAz|sl7hTiwstReGmAxJL6)C*7ZAJsu;VnUpSj(;xi z0Q)?r4JUPM<9z+j{z{#D?)T2W{WphYM0v|R=e&x8U+;eIw$q?S2K$6QW1R|ov{c{D zB~pLwc;!Ev^1=RYYY7hGY+SMX6<}-q3_txW3zZojFLHDwMLMapeh#IlYfuBr)laZG zR@Gtp9x4y)#OPPvTr3Uxg)$X}MXuuOj^LbS7Wbbfx>k--{B-3^*#Ti1)4=hzl5+wy zZ8}s?@J=LH&O|hTay|=ijM8e4LiF)J?M^XaR;+?q?J>I2k0Th<3(*b^l*XpR`eb8N zqI}8Xdrx9hJn<|wG7pi$j@1*!{)-#YBevSnoM9|M=k7 z`+!B@0-xV%3>X7bF^G}@u3q5;EL``%k9jJz7DlUI_|b9*7l&=S$?^!jin3v$310#_ zx}VN47x8~C@|)4Fcf~8XXvYj*GdP=r--r>)$gM5~U=)&JW!gO4b#2ujwuvJ{y>Kta zlODdE-IMusJ8NRv+mDS%<^2xR3u=r3PrQ_x+PS|kEO$uW|CXaECB}h@*&-rIu|X4c zMHe(z?Zv9(=TLtWa` zOFn(^&b=>jL^qCLP3JB~QL+?bby;NoX=pX=vN-f1Wn0(*fq3Xo?ta7ObN>XXJozr= z$12(6ySPB@7>QK+*Ta;&pv=f!=hrhD-r@ps60PYwe5Gq2Y@noFxc4b$KrYeqnBObJ zpt^QcXd;_-6-KSdZ-t%D?$1k2G9HfL(RepMUxwh}zrmq!jKT2DkErlq-<4$qX5m+& z?dL3ZuHA~L|04{;)E2B~^R1GlnShI-h&15YcNiVNRU=rsGUm^g(Z{o4PhS3>J11IN z=ayP9`AHi(bLI&gDHF0I2c^8FK?`+%IG#y_x}d4rSE5OH3;_xx&i?A#GKfqfUuxCJ zjtlFJc~x`8ENi@b8ZgAIcB}Bq152~nv@u)LD;b0wSO0$@hlqZY9P#GB{gQ##131(X zUfsWV=?dlk-qsF|?!R&K9UCrq0$Np%_iE%<`}k4cBi<+7%Fjs!IDWWiU;d&x;i;k8 zl;px@AsR99>9m4ZHiF0S?WPxqYff75z6s{$c-WhWWt-*AwX#?FKXm2N(n%sPq&l#! z@x{{eQxT$xFG$mP%lQ?YjG@9?>JAe|dx0=8nFI0Ld9?V}PeFuxYTdiIH;$yb^M7nP z$ucWQN$VOathy|=nu&KlL^HHn4@kcpwa+9&k)9Sm>Jfk6fBkUsSoGJ$b^tl}z}~O8 zBV7=or3k(fzaV)-=3gT-F}vWFg#BqwnXgjs?E>j9Y_HJ@H6OE__({_E&DApf5*=vw z>%r-xyNF)-40w%Gv8Q(i;dK;ZtNo-yFuOna)qG#pr>=igv6`0p_wkDh(KgqWyDTow zISs=>97n7)cMw+FS)o#!N(TNjc`Zy&2hA&eS)BdnfX_i>?-7=X4m(G4Pw35c`UZ>XcI-lE(L% zr|dC;5dJJ3@ys`PtBnc-hogUn4jdXHEDOP(;+LSJ-0wWB?3F~tEU?5}1;?TwPmxGt zTbNsqeP^`d24$JOc^5$eHdh__uKur}Dvtn_wU)h$&)?x>@|Cc@l?!!X8W(yWt2ifc zjtoa+d4@;o<CJdFwXM8{!iq|mpJb=)u{YB zx5W2bm%ynnU}bQsH`V`-&$Esy$7m5(Efo!O%%l9;!pQ8qz)O3_B{*#IXZd1G@-a+m>n@=_$!9uI{ej6qs7O!b{K`zy z?m~cZC&fm2esiX&E~rIIYCNh4=P8%*E-fW1gi?S zUBiH~2yf?0U7Bnn^jdAq4RPPtEaB6G27iu_du8loj)Eel7;ixW_8?NlnR0YBJC>RO z{{-EU#M=ohdw5c2hxymd+Tm}G`pB6cG%K==C6^`*F9S@ zND=S@{M6hzOdcr=F&X+*fK)nfnR&jFuq|rOvIX|5AJoZqR`$28?-?Xb{+SPAs_G!; z!2mT7HapY%3!WTYyz3%rwF%&iHwF9}Pk~J|FPVtJHKW6pqYBcc+>+Ir(h<+8KSt zK*#~Si>FIipMDu<3dO?!?H&uU8rGvZkknM^pjX9kC+v;un=ljFLnpQyA4Bb{m!;3T)2=KZ?aT;o zN*{4+7p^CSD$w6M7QlyR)+A?ofjXA~n=t8V^TM>JEaiYniIP8uj~(faeMo5(o3WLA z8Lse%U#5UQ>sP5TpnzICX!zU^T83!tJ5i=>_{}$-k1b?OM*sK;(Juk3NvGTRdG)!Y zO(-D53iBHE+y4HhUa1D;ph>MK4OQ>{Wh|K1@ox0hpxQYi3s@k%K-l|PO6rs+bn=`T zh@3>Y7$f3K(|W*`3nnrmyf;53{MnNw`6?ehTfv-Iw{tQG8bG2m-N~M|782S7lG~HK zDtV(we1ih936jt1IozM)mb2_r9+Ta3d^9FYZvfPIIvXTy4O@JwOc0Q2WFL&29D&2- zBLO${{{5ytH<98hg>L0bTl3cm2W7vOZ{M=}wQ9g!S5qrY3$#s0xi@CN_AOhT7WxAf zqg`54muAp!X^k%>8sI+%tH)rrfOHp_8UdcrJ`gj=f(WAD zGP~}Zck{;A6+%TFD2tWHU<7e3tk#H3it0SsHSrU1fFj0DH?xwti@SK@+KQ3Ixy!S? zfr{DVc8{A^HI0I}5TphI_e;S>-taL{$&7oP!CnsAOh^TLAF3 z>sDdWiP%h^!QazbfEtFX$X`luzB#*IBlNF;xyc-&BhA;Ql^t`aYz`=42^;zp9izc79)!ge20@U`zzzw$X7u%e>> zDe9{4Pjp(wH>_OdKHPFb0YiE2K!pS?xC{nefR5#pW=}M)5|pB_TCRUm9iFS@cgNcv zFdNx0L^}X6+FW}v%>mg|y1T3OySvKS$7JXmjrC8dQx1Z$KEplDu|4f544{jA1D-4V zgwct(Jq5IHiKzsCrHTNFWjJWEWxebDp08iQP^euHXfs|2ux+@MLbFr3K#(Gm2E$T- z>G$y1^tIH)0IoCV(t_cV1O&8&XdXeb#W}VqbIKIqFso-*4LU~vdqndmmQ^j&IP1oq zSjZDe@3A)*3OLuy?s!GF%Ua6;10*S%CdSG3q(YW_L_fZi2tT+oM^Fu`e7zPE*7t72 zV^SYv42t6-RWU=x1o8Z!bE*)nVE!??uIeJpf=n_FlBV;q&iMntI0-qyrT7%x?82~% z^(!Dvv=p5B{+0uY834BRR6SiPtT@y`ABdO<$k~(U+>W7frbR*Q#JSk zm+*&66Dn`Hvc^an?{KQ>UBe88_xzUr!v>c1jvSxfl%tQ+LH}7Y?I?Q#T&s&z(FbXC zB&NWPJ#enV0t5F&SsouMlOTejx2L;~UTv&?HzoRQbHB#2sM)FGsJzbhx&ZVii&DO) z;VYURfg}okFE(qj!d!vEz$2RtcqEoEB|XzZXR5rqZ&VrEA|R7dD8!A%4p0m0o%wSH zXzP~aO?{nHmfuA?jyD1_*E@J)r1+_RvL_-m-=LBOASRYRip3y!2-V9qYua8z4;4}c z@9Bo;Jg{V1$KZ&yUOn5Ju2eus5X1P+uRt^B#rZ|;qOaG?>RL~jzB46n&JyD4RF{OQ>j@BjWRM~IU_77NP;;=CLlY4uS<(!38tr@ziNdwo#*>fL3r z25n3bBPr{$$i3iMf~3LU8g!}Yq;l_$2WK2N&#_$qFcejPsyJ1SQoKNY%cgoRH`%yI z;Ku~+0z_ZolzwZAWi`}K zYE_KjnPm=Bnlw(u;r|25e(KR|J7Pe=@(e?O0{rWsrTqkunZI}}!g`M^LL~%i^>t6% z2YOVY_eCXt>Z_3IIV>cNWi=K2%u=Zi&D@FfO(2ivHkFuMUE$!0YLWo>r>_a`^q1o5 zl<)Uxe_Q`hC zlpB>=+Hh*0N;R*Sj#LcfU+&ENrJT)!x&2O_M2EuCth)Y>FW9(7`RWkQjhrd5nn=?U z`2vPMf)F;Wd3SgT5Y2v>H1Wa*WT78Q{-{}0N4|-dE8Z`2+9FOGYZg;2jRrVQVJbQw zF-pnnDvYFBlxE1-NQhJ~v7FP4HUGb@OeKw0llk)C%CH{L#eMMxhBi6Ys|Q_kOCSR; zI}@TpV;6LwxoS_>%C(&nawEe>fD!&bG0E$Pn@$SY>MIYa1RemZ;O^59NkS4S!1XzT z@==lMlto@tfL@&RtccumC#1PP;Cw7?&Lw^)q^8oQ4;7{Zk{sz@ii(B#m{AebEC!$Qf%i^&pH|{7X6jq3} zkIA<@70Z^PVT8G*Jdp`b2Q{nddeYP0MAyAqoL>LEDhjp0WuL>8Z53O@fw*917|BsGt`6I;igudeC=MJ4jl#UffvusLrXGDeuaBZl z{0$i{dnEFr?lin-waf0VFAkb<%px`xTP61Zy6s2a2pgAA(mZGZh9|uHRtz*^btNiB zGhk>l_Y-5h#(IOVYJ*;s=8}P!s8~#H}wRb8^M77g`rwsR2 z{|8Ks<$NW|LP4=VMU-NW)UN|ykZBQy5t(17Ob>OZp+M*Ev4Fy%-Y#>H#ZZM`1yg_W zYRwx^h01}}%A@DN5%Q=?>2D-|Rmz)x?saN+#09y{WARROLBd}m7jWBa?dr<*&w&yLplP8{XEo0=5lE}mQoD-hsv+YfhLDC+4Z;pxDM6&@9mGL&vPw&8| zI^V#o)!lYF#H|NND5oLuvtJ_XONkgQv19X-@ib2=>}Lg#_=$6bX@7I;`g6iG<3sb35{d4PKzCblOG!KXt-FFnO}TudDx8(L0u&R!J6xRX-bXjnPztf3IryXxku(WRaP}3!!RgAu@U@gR zUIE2HM&8%8j9?EAg}BH&z$aiERzT5D$+x(<{@8I(^-scR8Lf2Hf?h0kzQ z+hk%&&MgAv$@~k~F#mXv`UN{|vx0Ow7nI5Mu;rehyN5;&3br15~puD)enq%i(V z5Mrc6FK6e_1yuFUn325Wm5F+?5C_1$oIhc9c3i!KK2Td=3H(j_}q-ysa*p$wHBt-W$|a97G(Vg+`{p0 zkHF9F75hd>jB+>(zO?Us7qBmX;jzs7gyRd16|6YlRc9yv3Wj8caU1#H0KSDIZw}NV zbje8EQIKrvHw9{k2fl0S!+|3GXBCeXe1qL|f`{~Qg=Z6Iyl|YmSCtH@DtvHUwl3;0 zS0VmN-{VR%HcXw`R3lgbG&p{{4^g@)MVO&ISr0Fd#s1|P8RPy z{)v_9nC#QPsukc9;@?Yg?(kaTl4`2q;d^7|%ln-f;eY$U8?c~AkJypA2j4y~9C-^S zYH^>4J8MGOxjsX8iBYF55GK&r8quIs1ovq=E<)U-IpqXVul7WeSu4F$hr5#b{F8pu z?g&HuZ23Wkd)4z8&FaT}+N=90SE}c$Zx;(a^F216|63z@v&)r!{zB@=JiwLxUHC`a zejJJm-TrCZ^^=?taDj$xesu)uNADL0x>pth+VBIyzJPbmhlQ|{@ycf$dO}9oA;c)3 z^?P*JILW?zP6r1t%sBt=n8d^sl6;})cc*!(qi$MIL)S{Gw8%0zoe&1!FSO@67M1M@H@UN)C_tHNj++Xbma!PZ&O9|HsP-PYMv1R~5 zHX5r3y-#@3I=re?mg?$*<%D^=SLTB24K!?I)4v~ib7!MIX}{HzdRM2$FP?_nTEnJE{Qq|O6!6Y2D|DFzWo&bCc$Uy3;eh$CmCH%P!b@*P zvN2%O?;iKficyYLqI%SAC!XHI(Tx?BAj;*NQ;i`cEy+K@h0p8R^IS^x64_IhS(YBO zlE}WAl9wMBBVGvwIMOt4`|lTY9Z90wdR(NwMfQe#9_YIRRZ9bO@S3|w`O85*f$iQz zgr@T*{4AQH0r1EK#`E{2W6UujQ%Z5^pAf+hsU+`_J3=H{FrFF{KB}gEbcFYO2f7$M z0)sB@UItK>1XQ%H^nYBe!g~LrOHVFDno0X|$<(y)6}wk)3=6dGfe;#5YgUp-4 z*ueTt*qv7hoKtbOHpacvb7{n~WI)LJr9ig;CstGAh+bG!(V{Lpn%?Gm=~>RI)0gDi z{X4+y^wH#{06LG)>YA2}>8jr@nuzySpLy#Rlj&w}?axcynQM+Sl-{tJBPeIQke>dV zM0~5#S~YLjWyP*p&Mp!c22o-EV$!xg`oQ70aVuWk1ylOFmx}{9j+PSdS7(3hyisBe z0B-v`?4d`NEzX_n@W3y7K#vKzJ)mO@kZtpPRE>~!6@k8=V?=-61*C}-W+>E9rQr~v z_`)sNdpOT@LWEP?k+zpEqt~|1@$4H#pVc&o^s&->%K6sLZyFi5}L6t>;RLOh=C)cY*)qhta?PLXINHa?5V~(QQ#@qZy_Inrz39<6*TF7r^icZRS5p1d_ zOv1}8h~u4yS)YW^{!W%V=?Y zaX3zO6?mm6)H4za;5Vx3%C{hw+}!VgQ7Xc*4mJgkTDyKkNp>O9$j!GWo+YfG@TClE zD8xw(jM^YZbf4X&ZFN>lECFRUG=^;7E(29#2}WKiKII~dwC3ekUPDc!nIjZ7>%%i`tB@@2(&*`R)p~hS(q`kxF@w8cNcDtsWH)J3#x* zK`$Ng(z(@aoA#>AtJSmrU;Qik=PvIhe~1^pfig(O0-KJ|gh=9`$H>cg8h|a2Zb+Z} zDlh>ZEB9)Q8^2Tw!$Cam=HLYxd5gj_UI&wVUT)I0vyNfH#y$V}vftclC2%Wf{@O3~ z|KdUfPi58%FORzgPMkrIt}-ryFlh5DUqsU`aVK~zP%otAlj3}ocsZ?rvLDua^l!iI z*V-Bo(c{0V|J~jT<5Ws7&Hoc)cHe&s>ujQqPuN<%b3d?7M&vAX0bM+ewjkL{P?#_9 zS${8`_s}i!Cdm1PK}s(~l7${eqSwK@rV-%+QluxZ>5S>y$O|stk)S>-o47SOaIv%q zK0*9*_^OtN>D&1#502Jlft%iFr`VO#M~#RUCPv)`yfcJ_@B3^L>bl2rLX1m=C@L+L zBdmo4v9WK*cR~HAhcC)axVJQV^)+^=tB0c79^GDI4%&*uYTyjzw2=qEA^ zs&SWQ!BLOGoXx3Z#dNWBRTZywm;1wKeeLJ`An=bqaWfd&(dA`xF%{*#CaR)PdAyz(pNJdFLU!8w${@TIF z+%glU2c2Ps!WGP*c~>3ksCYgsw5mw1)iACu80Hgvl_?_S@!eb#uE>~$(^wd*?Jo0{r$Htn_hq3kUA zY^U;chMzOsYwFKSI3D+xc$34fqL!d|lP9~5l98Aad3H3v1NX~#&xgI#F!e6Bk5dMi3rP3+T40UL%j>#!c-Y6Bq_Fl5sisf!{ zlfu0pkIPM|9EimtM2@1JdS8!6g!-9*t1L%^T2sy;@lE7?U7<|@c}a?DC?B4VL~61o zwmB+!S_6V6*sWD5lWY%lUPKMwQ;6wr_g|ZSZ>znqod}8BmCn}m3Qd2NFRksRy24-E9OaM70wt+tnOHvo)avgAgs%!b!(ELRD|d4=v72eRU}8%a5&iT zV{c01vuFvqMy9?rsIe1bMShq&^3y&z zXneXH;8XRfQ(GjaEM0*s>332(w)Z@TSF$G?2dYKA1EngAjkA<<;J|e6-#013j|G$r z0t3S2C_8y+*<2j&H+t3M16Aw6vl~OfA2jzI$^Y%E?B~6Xs!o?7D@&K8;!~=4OB>FQ zaWnkg;{9$P$oa)9XHL#quFX3cHQBdhkFk>%x*_l z&b7VUo&MhTURHVhkv$>SaQpj+oo5$+208a!B!OdJ*lRFXNlk9SORJd_z6vX2fGl#a z?4E50rJZ`kawscE1qBK6K5aU_zn*@~(8OoOA_u?a)$_vUB3Q#smQF0;$rnr2%<9iQS_jTDY_k_SRd#E{WG=DWe&Gdn zX{^|0FNwTPZ;J?4ubgpqn@{Cu=IqnU7HCNWxTi(D-5EENpFpUCf{@=c%)h>BVW4k1 zCi68)5hX_@|Aq(|gd&yTj0+M+gNxb^d2M_P;j`^%RGi&5F-;wcM8th&K=MmxCL%if z{!Aq!wFD(bX#FsRuyN*};&8h&vhRM305Y=4A8c~6t`Fai)Ntc*VuiGG2biv;+-kOP zuN6cmeAcm1?{03?5xY*zh}EH%&_!0De>lUhP5HbAi{sx^&h38oMHV3A0(X3vW5ykD!zUn%m%Rl6{7~14C*mOECv?25BIO?$Wi?B^m zPRg>CgXa;@v6_ewp+KJj2Mm=91v9~v0p~&~FfvGQ!K zVPsp+^VgBSFYUvOPiEZ6_F{F(Q(o`xr+;g_O(%2o$>rJk;a6DlH~MJw2j-SYRmEwJ zVBA;k(}}0-R|}rZD{5=~B|GQM2hBD^QaK(28~O)I)+a@dMe*p8uYn(Vw_W$oOT=SG z5j+Hb=Ds=d#i98`mt8aaD;*b$kBHQp^}!TA!2JK1=N&e9?HvwEX%6-L1Q1hCwx+6` z^7}fZCXe|H2$bD9k-`?+&z&(%ElNHXmdt2}>tAr}wFIbI3>9>GFg>%dBToUMR)9L& z;1rp9XY{TcqhSN}!tlibNrRi>5Fe(5?skPu6k!s{Ij(xk7gy_{liGi4T-Q2RKIf(O z8=f)5Rr`evuXob=W21DRgb%Kru$c&PQ4U8P`Ux22tj+n1?h?#BKUV@0=U%qsjIiM0 zKoOn+3WQ8RXuz^h?ZV7~0jkAZe^bR9EneacI1uv2wuI>5P)p(oTVZrflfa@)uvD^ z!E>0;muSA2$4g;#I7cH3_agW>LSwkf+U^1>ps$K_OI|l{*GIEL;@EXE0qcbk?j%ko zY+Z<>T`95>QKHhW?(Pq>yn#4oMmr29INEl1 zkXb{a@i7$!t5yY?IU9LtzTba&&8xPhi+A4W7U@RR*iC=`_m}`8nGqwvfz53XVHp>H zNx#!{jPZ_6#F-Q6-t$zX4OZw+7_g6FVKNy8Q}dA!Iv6R14gGvZl_Ku-Gd?Sne}>TC zkygT6$jl3b8(>SLTx8zT^!-i8I4+ZoKsz3Gf|#eFtbXL>f3hmHPul<(8BKlU0Lh0? ziBWANyKb=}7%!I#p7GCA&^gT2#iaV2PG<|C?a$Um9j^4sfObw6pnVaoVV=G(afOs2 zyAMWBik#%EfO$^YU>=z!=*^C=tx*fRsOKstJOxbx?Mzvmt>LVY*X{*bznGYU;y-=F zX>A1K7B>;$+`288CV&O*$JSUumdGbC)~!IV%%tkWzDc5h{bPRX5!sMF=6U&>t8;6> z(x45_2yX^Pn*nBA7R8T)NB+^E{rl5RG6wHU`~>~jfRb%AO?WLu%sKd@>;%@Gu6vLFKvtDt-b*>Y0S{Y^TD*l*#6Es!QA_m=oQkhIZqIk;;gzM~ycz&`$<$ft_GYBU@u5*B53 zjuHLls2^Yim?qeKT(o|p$20MZjjSs39)fC{YjCQC(95U!pY=B#Th8T6s?6hv0%?op zQGVdv`9^0b$?L03PW^KK=r4%=1l}ONK=8~^$FI8(rNrI*L%OOMTK^pk0DU;q;BFg6 z=3Tti5e#M>KCSpHk?MW;%6+%(GgjPGTLK#RNA1?7{ik3sou&u;ODUK*XafaQ99f{_ zVML4~G<*ukZE^we1a$g~_D}t`5ZF+PXzfMw`{B&L0xQ2i7`AgFQ{Jw3pO_r-9KNMI z5dF3eCQdRVPXHfL6bBN_mwQ<|=Yl7G{7^+0R-~uaj0P{# zZz2wlLh`s*$-<2qNxs5vu~>12E7qRR*Q;X z>ev?VmFZVVhvEe-tPCW@(=*XuQE)-?+pz&kLmEIQGN~M!G^+V6`yX9^E8x;^OA~g@ zy?E=da{v@-#3J5UzS3$a7fjG(`GQY4KzK3yM1+tSI{-}K)BEu!cq$my-G@bR5hzYb zi>d`LmBSVQJ;(`hV4&|CL4p`*Y+@5GB+pv~#q>SEev}#XKs7Z2+Z&taD?yX`UDM_V zvl4kvEm|xPBTa5@EfSRHM_D0NAak+g0IPt(-)>F#)?>GNlAQ~V_p)ixNkl;UAK*ty zZ&7_>hwmW<_=WzGBM&8S&mTVNdvMhXW{}!^_5=(!L(_?gFooQw-U%=zVIem#vGZvX zXq&P(xp`wfIpGoX*wM#-hS`V%1XfEIt%!rDK&1fNo0mo$u(*7yhyz(U}9Hwn?x zQgFbZ;y>n7FWNerbB$sSxZTW4E2MZxO3i!==zR*tRiStie>$>g^7XIUG^83ila7ch8 zph#(03Z@!##uvM{$iU1WaQlZbN*&lKok!Gvt=^6=40fo$Ze3)0 zPbG9p4EetpO_QOJD#fLe2c0B^DZoeQ?79cQqRyyJJLz{UfkocR`3NT29`F0zCV@$_ z5w3t3wX8K4@J8jgF0WE}w*a!4Wn6D3qN|9eon>Z9)9Fl^Sk2`a!y;*hepkPz{xkAY(;%tYo{JFK_~`kr8pVfZ(NwQ=oza_4QfL7QaLfD+ltx#cAaa zQMCpcgw9M78{LH+3Eo|US1g27TO^c^Fnqxo>UjSiC+1*6@&uvGS32aAo$MY+n6Nmt zKbL-(On#>whgWvno(HNbu-rAZ0-P(rj#K!&{*Mxi?r??a{U#Y~1Aw;rp|9}L^MN-d zL~_9qMZOKf`W9Dd2gdyx;BPwEPUPMP*X36_mgkRq0F@88Revg&CWwf`2%e z7;cX@8U9bcsjG*rmBa$-0K(jy1OTY^?2qsP55hqAyg){gUi&@+ZH9>c_~k}c)YxrB zL!{cd{%(h*&ezYN6Iu`_9E3y9$s7Yy;JgAxv{)CL$Tmtxs`$70IM?sQNaSO?8Jg#B zV3d~&nkXXFe;)eRZIy)6p=4R6X6Kyt60kEcX%0YXY(J52s0G* zqhlMtyaEwtuL4{_daYaylY!dHygZbK`o3WFeTt&B%d>qw?n#sg8amy+pq=*}ncBGD zpP*3{Cb!`IdC0g|Sr4noJ>QBHB{zwc@0&n+i_o!6E2 zK7hMP`I0LuDPDEYD!J?~ro|sD^j92MV;R`bvi65Qn{E0_2)#@nN{BIRK zk$-tVsbao;lRoA7TzT=GW`uCV`Jl(4LkO@56@#htus|ebap`ibm_(KgpDe5V&4+96S1WS#NXeV5h08UUd%CU4ZF4-bdn{@zF@kS2XY(*EV`k-b95 z2HXO3`eV!0q0-40v?bwk!S9y51_4p{Ur;0d6tqpx(^c#;BL+V|^s4wB8c(sp(UWVM z$J`IzPkJ~&_nEnuOW@0LqQ7|vA}t0Ib$nFXEl~igq-r08DcGi$?Izb|LRtLNycNIQ zlF%0`fZ*l{6rkn*!Rt8$rJ#U>)s@HT`&=` zE;zJ-ra$BKFu+mr_NzdNzVfx#ZF5Yo!xDIZ7EK0IdSW(7k=^nE`xg_4Sl6$mC&Ozg zQUyGbZ>mRVr}K#Ct(q2_$|G#)+LcOqW1Eccgt!<@7gN^8>Dl$3*JI&N)F+QFJ}elr zT@gK+swZ=Qe@1T%cu0$Wd*HDKI8L6*o&foNTcqQg$izfE@cGL334{d)Xa0z_X$$~N zcAv~Mx9=gz@T~=7KF}uEmA45jIPU42TNatEiAPmp+O=v}uZ>Idd z(Qb^1zHrg((8K6{xUGWeYLQ!-BFbW~5RECpY661b8M%8O&3=J0)`)n#d_MK>b+qNX z=hg~ac_AwL9PZE1d&?7R6c17eA z!y7xWw2b&)DPnWW!IS>-HhO?fdQ|Y_w3r7BS*JH(NaVc;eYW?65cEH2ZF+0%;?jO= z{u1kEfwD;lPIBUKe;f!goxtY4mR@M~m4i2ilvMRoqpRMP5dXY>;@a?@cZdlT!AMPP zT)o%uakvxI)QQ(X;Rw_+^o>ImA*}UG{@6~$sD`E0yJAA|;b;k&w_O7j-+RJcfHGWd zqLcImtkvpL9LEJTZ{w7)gxv60x8(`uLhbJf*T-aefiyt7$PKx-_#!_qIiaXs z3Zkzc|1TEcAWs*-MdbiXm5%UL2|2ZhAV`*_BX9ngx$2g=YjXEDxdl?K+tT29&%X#$ zh5cryPd$u zpu*SyljBvj;LFF*3(SFVcK9GX5?Xj7V5}{1eckJmHkj-62pUi{M zSnn!JgQCGy94LLv&fYQ@C1zgrT}T-;{#z?gwO|xl@uD7t&0Avz$>iZ=zNAeUZu<)< zk#v%(Le0cQ9x*zu$z0)N3dyg-8F!s$ICQZNL{-ao{$t6l%Y^J_iHkHGSZ+b0Epf)o(NX{Vr4_ z!u9s%`t_qX4*)t>r41mYB~}yN>r(S^WK>!798gYve>E5lOZO^5YbvCAB`kigtoXE@+*4vzV0KGKOI?XVTY2 z)az3a6OVkO6FaZ&IKtMD)_C0LRulCezQGwego=cb~(*7X?x`zyz zjY{!ccllEyfRJ8{`Ht)84L9z$C}EGna4m*X#pC{P8~n~Drk4vN&1mttEcEF4MwxY? z%pr^Au6b#Q_9E(EDhMRlhtV`Xll|NHo(N0hwITpltJ3^!r9UBduWp&30T9sXcn0jf z{SKor`NWER;@PG_tsG~F9)Fi}gCX|1?F~*#?dzV8hy2*lc~Qr2FtnSj%XSVAwhD4L4w8LPgCX0CpTNBVY@h{l;pA+9csYSrXJZ&l50p}f zqSZMg^fqW(OSIK|V1(g2^MQ80Y0M{Tc_bCs84LRIjzWLLw%6V@UUW zGZ@yhbq1nx8NdVTS9o7~jFB1+VrsPmQQgr+?<18&Nxznh4`R_3njze1UspqfdRlYT z(o9Oa&6&9wkO%PYYWJ-%jY(Rtsw z8=%;02!hb%`M%}!+>|;8>@PpC4XWP9cxu(kUgNl(a~fh%^i^gn)D-NDK%_2o92x&o%7*`^R&< zAKnk|`(f+e`#9#Fd#+ek?^&?r%=2 znT`W(yn_P*92thTc2=k#`ev@{op06Gvf(c`3O0X#h5ChES}>{~{+fA0i%BUYTY$IL zj(rGLW#}%>BF(NM%DD5P11{_zGn(neF9HduFshxnwhw5?T9b)Yr8nG0fwJPwHmg;V z4^$*4D(yeL$`L;62NEQ*vT5j7FN$LTlCZmw`-qm>5`0mP+)~AEr=j0BWOoq}UAJVR|BEdaU zN+joFwW{vZ0vVH~EweW(_sX~YXmtStG37wmF_7adG9& zqhb3iRL^DrTeZgDKdY0qRXGc872qCb`~`}-%Ka&K^T6L_Xu7Zq1JYxI)vnSNAf;OI zor2G9m<<%M2$yVTBQ3`z4&NiqNMqtd5`+0?T~H&Hc|!+}*zBvo<8{x`cu zLd#&syT`AaoS|oc5vwvEv+#23E<1YnK8_?saAk(m)sf!NiF$&~JkATn`Y(@A#eq$t z4ZYqqVEKfB_Q~#FdiJ})@skSB7hUtLtx{x4p;XuW9A3fXHZ`EIu4Pt2bHJfhePk}k zG5U=_a3ShoF!5C7O9h^WheBHT5a z7vuwZ3}GaC9HYElBz{m!1uwj6aNWC)3VW%9jv3+a0cxZNpsjTL=P?}UAAf&`;gL@i z9%67BC+SM#`j1Ui@8iAC>R8as168E-e%%Yf07=XD{;P8LI-^yr-@UGLh!q_s?sq#Q z`;d1!MBaL+fO_RJWYcxYgxSR*d?xv2d66Cm`ehWew+mudiGq>wYiLdiQkgqax#zlW z*WAzkRIMmn=ti%Lfr0EXk+zE^Ij`j+U*ifHg>8%@b?-$mg+$m-^J)f5Ul3V`F`H{4 zwPeoogDe^} zvR@sgceudb(bj^aKI8nDqdq*0pf#JaJjbWImh)+e z(j=6`Dv?`NGMEu;zDS|d_qynwW7QXXgCb>Azg1Goz0WjjLfSkww3DR>EJVJs#q;?e zM@R7C_eO_563a9s#oIjvw(5x;aVx{}BKmXu$@$5F4y_aO@x7-vYoe;LkW^s{%)TiZ z>>!jN*O&S3c@@JYQ25E1enqrm;=Yhfa1pXCl5^ltygkHyQOXQV^tXV2hVpLoVZ{^V z8_c--j1m-vrkZvOQ=VWiy}mz2-~|Z*53+Pg&S61|4P{GbD5$J=YAdUCsL|g^UyMMH zor%TL2Sd+LgC()OEI65vfLot&A`?EXzr(J4`ZfUu@52v3Zg@L;cl4tMYyP>OPq0Z5 z_DQ7dp&*jTi9O%_xYX4jnWgtP%D}Db4Qt^O)laZ*jn9jsfA^FWM5)65aw61)w*E9T zLc_9B1I%|6bN;RWhSvRen1mnM`^$B?z%)4UcgvABK&>XaMLFB$v1l{gHt&3fuye_+ z6rlx1%lm%^Fu^?GC&7(W{?W)I9Gi_$cJS6=)?1>#SyX&p38er;-Eu?3QuhxOrC?!= z*S?Vzg?}Re9~9c7&V-S1a2(M$U^6kxldigqZxmI?|=QdJo+g*cdFe6u?bTB=UO*{fGd z!nzlhETR|91S7$w)1D|$qoNUfN=WeDg8+{*H2Pl04uPnGw0oabw(G6fu9fGRlgGl$ zbySV*Ze~i-0cFb!71u#eV5Zy>dyk>p^lcE{9J|{~J;GrmfzVke-hZaYdmK%#PKs=Q zASwmBLGE_vZ3XLrp=<*&vAq$(yGoo>JK7M6VyXhoM`fMS|Bp|BEk zl<5{}(o@PltRtozXHw#q2NVJrT)-WAFB}&~ZiSt|+?2lqgQQcKgTAhbW-k{CE?r$Wnoq;EK>#_XOC*P{o$;1tHYvz-wvVfv>cwQG!p#!4JmwQ zp(O)$e|J=w@i8zLB>;cPp1E}$zKJF13S!Dv<3(CUAWYdnD$hcha4;&|ly{9_#78LY zkd%ra$Gbn0ICTndqN5W~%T6FDl|$Wloyvk9MEeiv;q!?NYpQ++#jrR->`&2VrMjH8 z?spVnkxJ%E9pN=L{girTN6rHC_u;C*Uw5S%n=z8s<=-N8BpVPBC9gdc%0BpV$(&(RwI~p~`y8)$5|*#U2dB1U|m@ z_qD+n|Y__HLdb$e`-

@)3Wiw9MahSO?t~GEIUMM};q1J!X-0-pYS9hZt+EoM8vwD^1 zp|HOIzbk6qX;SNWA!PMT{yDTCAWU4QZeJ)IfmZCqT{c8!1`y_r{n)rh)0ZY_JLkNO ze-=kCF$O9%YNkydxyCLn_t6$l;6rKp{>&!sKwwb_n1|9QWt1TFpL3t4)5F(TD|qrg zJXP-vzNS>YPt;FdmOFFeDguOvi6y6Cs$(p?C2`OoSqKpCuieJ ztA;_uszRrodmS-CGM-xS_EH)SXoBdH8gpD;RafPQ1?5-|XU~7`{mF#Jd`l%M+(?n@ z-N77+eMs7V`XfE)s2Zdg=|HN{BSG>_SB;Z@op8axb|~{<;Q0a7WQDPQg=n9Jfw)Pe zKCNh%!^T*_J}3q00C2U$u=;7P$=`$MRAjS7XM~W~&RkfW`4bh?ULZ(E@HkxxAOzU? zh1;NO2Nc7ZmXM;*voNLEaufyM_B(b&O{gCKR#Us0--|EL+yUPMi_uIpA~yIR?Bxb>sS!G3E;{v{faHCojXlLt}J;s)D6JQfLCRtlbc zufUIToca7U#xrOGJ5bE;*tLVy9d@keFR^5PQG)rUvuv2hecIYsD{$Uv_=_Oi91y2G zmcv!*W`}x51!aglsPD0u^du#79uA}4-l8OPj+-rYX|@mj?gV0ZrEjr^FZRYXb{b~t zsH<%TRY{MhT;Cdn1eHPGWl|tF2lc|j%6P5Se>8=MWn2S4K+o;!0DU{~*c;n?&P=V_ ziR`y_jcVxzcK98I><#|02qtp@9QdElAY&H-n$lseQ3)#0$9K5aDamdJuYosSX8F$> zuL9Iq9RMjr!gd47fDh9AH>I0S;+_`FJO&F`n*{SM-fWseV-;@)MCVXgDlv9azv%-| zD)u1K&7q|X{=ElLISRpojbu>(GS4xEK17|2tfPMq&X%#*D4-f|3NSkc$luy!;NJ)E zkt9#6RA7{qUkm5Vyi~BQcS;uQLQ!Hq zP0rl@EFe4;vDV4=EG#vE`8T|c1wG%XXG96%VTDDMW3jODG(ULH4M4>(@N7mq0VN_$ zZ)!vuCU$#@3tbTU8VSJ$thp8T8^JkXjE4~h-8c}fQ-8-FJX>R*ci{tWlrET+!39S& zNTKde4}nUgxSNEb#D|{Xa!N4Upg?57n^l85mCsD`u2Ib+4!`Cn;O92ogTV{qiqYEM zB^l4g7~WU~Ge(TyxZRXP3PD8wysbvc_#X05aB7DE?dO8RCbu8m6QDV^)Udj6ssvPr zgg{oj;GUt08?Y}5gmdLVrSt7xRpQk}@2@fSaE6G2`=rMfY4+3{ z&miEu?@k%JjDkM19GDp|P8(>5+@F0`ISO981mF}CAklRIGc+&;8G{{Qs@5$)KVKJ! zK+}L>2Tlu!v`_Yb!5oKuFm>cHD9CRB#H-c(3BBb?la0sAF&l|m(z&2MUz{2QF8tA4twH-Grd*-c1FFx( zU`mUS^PK2`_MgqN$}u26pX5&~;;hRO&>c%H^kP%@<;CBh!>_GCk@}||y6y797vNR& z+w;x(uWeU)-{@AtrAW!{!5szo$jScX09&!dZX$P-S_@ol$!QU-q6pvor#BBW01DmX zNKY3;K?}i_fC2}z_cm3}=_d;thJZqI&k};+EnsS}z+$nMJ3ml(z?O$828Vw(#|`}U z$s#N@#4O-%IR4E$vRM*dg%4g4kAgx~wq-PP>8`9X&`PD3dYM!T_#wc}b3|?goS&48 zFhQRqR8R%W87{54zh*ZryiwTfo4Ytc?kFU-{EjR1 z0H8w&(ig{82fsmrRRAnSW_G}d{ef5czyrTL2KXIxM{r2$YaYYTBY^U<4yNbm%lm=n zZ;Bwy0hpmY+5RF2&_cvc0e|UZw~*U%7hoL0Wzk=-%<$3dKi;bGJ~q3qfNJJOi!Hng zOYKl2J*G?Bm;;rtp?gV`5bPdc04q9$f*D)B3V&`SsoL4qLo@qT;Zfmo{{hUeTuia} zombeR`F#d?nI`k_b8wCeK#Wx!crhG!k*Bv;^)f!hH5-J6q+x%-7?%geb&g_QVsIFG z6>CY@px{-@o6mul0Y=qNU!SN39l~hBLM~VWwVvTJZoj6vAFY`!St9?EzqR+$CUN`k zj3BVQE~+M&+91)Fz4vHfd)rHMl%@%;@g=siA6_{RJuG^p3T;TxSE%<_wK5U+#Bhk- z_RoQ|JQEx^O_pI()$w zc9Oy`&MkQmh)7~a>C6!)bh!JkZ&i3(hW@;4jw)PiuY;_D`4Zm=9lGZ-fe~czKVuGN zpf$pBi?lxOiCcNP8DJR*fjFUQHN{w$P+H+0WVav{UK%EAZRM~J!C=x8ZGnLl9Pywu zFIHa4YI55M6^L%X!Bq$ftuXXA{s>oU$I>s|3l9@-mjsa3ftf?Y#T4fN&n-R2iEQQr zuKFZ8NTbQ%Af5xA*FzfB-V=jsdBOaP^n3j4=3^ofFTcAW$DXz;5f-o%6fR;xoEd}M z9}rxY3y6&sP+PK(XJHUJmiH3eL8xa!@;aZ7{Uk^{pX7&_YI&ur&(B3EEe_U&3iZd~ z1|%R8s-QRl%;tZH2udhP_1+f0<6%p<$FYF-DXN>dEk{|lKEajTIMK(Rg6iK#i16*& zzL$bm9+Xd8hBP&4np?`!JaP(pahq#4?*&`C^=NM8J1o@wI>* z6CLSc@uH&XLxRkJ8!afoS*#QzbZEmA@18S(;$3k~5kWBjo;Fp)+$`Xtt?J;_b5i(1 zg4=#%idEEEXeyv$Y;l7534q6U4dTYJSYw|Sb8|?A`V~R@4nl&iLp*G8Va&M}+WC0; zFn&zgYq%M!9!9U9DcJQoA`MW_J+EFO=)Go2EO!HRopOcFXQtKP_vKPJ6^2CS#-Sai z;Ceo{i7&0~xlD+azrA0Lshyo}1i}xU6j3*lQ{l_MY|KDTnA)vs79#tIbmT6Z=yF&l z8j~hmx!s?latt%piYj!XuClAY86~|xDeDz1yhZ~Rl}h4ui=Q28B<+V_WYRGM_T70X z&Mpl4ppKYsXCnV0`3pN81Pk|Z`&u*wbMUyYEvQjRz1&Ar5P}HD&y0Sgb0|EOA!Pfp zKE_# z@T&cZ5e3AN*&O6HWg>roV;lpNKUL6~ZerpH>(~P@?c+BrijEd{lK-+|F&67p&@e+d z;2YoT`J@0Lr1doDE23tTs6(wx1-Cw%7EyCzqH_JoJVL$uI{8O%Ld2eK@nd@fR~4xw zdo(zr2A{2J0VAy@zyLNbV64-LUH`?Oih%=KX2ZTpAo9hYuZ@6QdW>!c@cQfkRF6SQ z7L{Fk>ML$-0jKzlyabReLiNy_Ah(&HbKCQwMo^#oDu8?QDEN&V9uAxZwKh1i?e?we z9Q=X;;759h#{aG)Y8BfKXV(BBk9v*NLn)>AVAc-C;5m%EmrIvol`Jx-vh)BDqp#z9 z`25VS)8gwF=0WH&Y;Wv_2_ksYS9+1}#7K{9+7Rd&(_mrj7XwU;X7Tf>FJPl|td#+c z6&>Im$1^K2scG(mS9XBT*gaHoE1N7BB)2i2ZHF~GUy*2$r%U-Bg0BpnSVNQ>R-X{( zf8Xyv6G(gOkx$2{#{Foa2~_jv#y zVReO5voOx+CP-X@Nbx|5;k!o;x-+2b5EQ+%R%?_G^SzfD4l%sd&CIYujf@kEk6ZpP zzp6EdlEP;d2j))T=&_yW?IuEhjdl!B{cL}-OsaSg7~j_HPcIW-5{_c3eTp#f8(`>o zaa$u@A%=>t5ECgP$Q{;u2|y$T>Q#k$6-Hl(`rFYk-}t05=+f6D&cs~SIF~xpkKTA8 z=rOgL^X+~DxclLYz|o_RUZlqh7I&raaP0pv@*?_?B9X2VxZHbSW}SZ7o;C!VDF}?6 zac<&V62pUkun_>x_n~(hp+kG?Z9_N)P@1THrmv429aPhgwH_%3>vNwQ>zd;ih#XP? zrwO+a50NS9BLLWz0v2$;ekN5#D^g_lEO1d{U}9E}17U}n-Z*o&cdV>|_r|^Q>=z)K z@_kz3!3Krg!k(A!xLN#3JPdIZS(Qqr34!IsBaZ3X)}R2+T(V{_X%XyDtF|?Rw9f2t zOx=J4e|m~5dl&~2^HJa_cv)naS{-d~!AQZBF#=Hgic*})H24lU!D(ci)LUzy0K+N!sRgCF2o) z`6E%KpzBeSCm?1V0VNv@{80XRH6OPc{l~>REESu@a^O)|YI~M(56)tkBYk_8G_~O) z*vQ>pjWzZ^Zh+aTUofhm@&H6vWNZB-(Tor`Is@u38O#JkE}}K3`uV5o*}hAwKJho< z*JHB4iVWI1PylB|?6_*{cYYLF~3@ zs{R6J{sC}!$3QpTq|zLi+*V5NL|U11*H~CJN4ll%BA?k7nknsXG)S%oanoL#sxY3s zZZt4{f7;{$*qM90Qw!E`@r=QmN=kTBc)7&(%fWs?&n(sHd&5k0z#~#jpG?i3 ztWTmq(y(c(!T1@mD}C^C{Q*-`JBp0m4rEem*Y2>wASSZNXYeL|h`lyUg*+H+Jjkjt zbXDMBL?(nYhZGRbS9PL>1@Fug;FK_-Zt#KF^a1djxnNf32yi_ZghxOmB!~$X9>j=9 z!h=EPl(2GrFo8BeJ4eAVuwqcaclP4kHkOrzwBjb@M>C;>apb;ExgGv8tc@bG5CZ)Q zd9N4)-fd1*Yzg~~wKAdbapaZ?xKS#3xymTuJw5=LqBkH}Q~-2~BJdBne8?Pclg~z7 zOF->!@EbwM&o@$FCdWjH4SJjc_Q<%!FsU~fTsUeKFO{+u1|T*idXFSMpZ!tgNoE6k zibgp4`FJ48%iDf(HLwb?|279X2t7gZeDAKt_%!6530|9a+s~B|s0F={5xK9Nr)W|59ssUlTQavS>l`cLRI2 z_2?!=e`2usTxMeWMDJ6l8n74U(Xt7jaiGd+;%V<}XoHz{1w7FmI^TDgsQBfWdJz+N zRR9`Q)C&q#9v;b8d}X)%Av8>;hZ1^Tc-f~^v1wgIXK4MiPV>f_x&Z|w0bGfdmG1ty zR+obdDLOs1Dvvz-rNrV`1`~<{2ShCyIjUsyv6jK14KAh}ub}=iQmQ9UgM%^^bs}Ozru^u=@bTF5Fr+%Hj4G^I``r4OIOiJic^Ij@lY`9qOY{WJ0FTO zM+%@Dq!TkU-RjO`%N$~e`VX65@4}135wf$Wh#+UUJ8cfHF94RaX2e|uX%1$v*?+yD z2-~T-^wK0fX~IhOG?A07XPW-nzm83yY?>BepTvn%h!(hS!8fd3sLB3ROi5ORkDj)x z2D=BxtfyfPMpWe)H8OoqXDq@P>e^<3r#zqTW1lLBGv~7r{wdX#)0Tv&$fQ z7!E*}2tv_9vT@80NHI50Cpdnvx`9&>UtsR$V50PwPs9uo51x5t#v1o<5=$Ir>)o(X zR6SB^EO|0dr*%P-d3rmLz&#{WXDy^?gQaiiB|a2diKgTLeY+ zY{~~9TTPNTh0yI7dX)c>={*HDuL#dl@sxdXH=%M60iTt{&1{+-1It!cu;gQ-?@b!M z2?ob&_*V2}o2B&_d>TF2981$7B*+DYTeMDDB5S{*ZfozC8mmRl+18&mP6n;tJ${7@ zNReMwzISY>eVuReo&cNJ5IZcd>8Z{v9zG78H+4(_TIkLK`wtaL>^OwFCyz56_+aBf z{GfzeVcl%TuHsH*;xl*|mU&u5Qr`Q7gJv&uYw79rUFsR5I_qQJP5g`vdIDF9mlN&t zQTkHHgxPMZp1YDnAsoH8AlOG3BUll@wZXs3XY^^)PMfvVjFW(NLDVv9QULR}W=0`InDlf2*ltV^k$ z6xFR(-;?2ZnhibM89#(6)Xs;zJMQiCGje;?YZlL+rDt_1O4UbhxT?Z^VB^Se7HL|s zxx$3tSNZ3${Q1|EpC-p(%0BOaIQeQAGxziudQd+$rimCn(L`z@axtrK&kMQPBe2}6@( zb0cEwv_v<5IzC*qkVEqFS-FvBM|{z-&I}294gtZFLXIj-J5)o?6JOzUrh4jQ@gt$U zS;v^0X;osoyWe~N-mu$_Bj==f&{CuAV43Ii!BJ|8ClP@4fl#((n;M&r$CqmlUlLQ{ zfJ+w>ixx6zWyzy}WU`=|kjf6~(*q)QKMz@%8> zm|g-5otRoBimw&AVkkh#fw2MP#-e7x`*ZY7-CGa!cNj0rS}sDmT>j3SSXylS^^W@m zE`Z9uk*?^8t9A9S3y0rtKb~yE$i050f`3` z9yLFUkjj!W@?pRafRyMbk(%h{s=AulZDte)SQxVG^c6A7vJ5C-0%ZK~n62BwX#%a8HMagIi%m*j|Lx*-R)LoR9ct zHN-`#LUGkIOnzl#hqL&Sbj^>+Ah6glY33{0!@%!XGpNIDUY^v-ihShpaj>GT|& zZ?DNPM1Ji;DMK!R8iO6^y$Kk8v!8Pv2UcbH**X`DUB-lT#US9wy9~0*~!Sz9#j~b%>v)Bx=FOTZ47NU+PUb|nos|)s5tPGRmN}7LCUHCD-wTgm(_(9uu=AV>0L(rBF8nBw z$eb)_fBf_9!%FksRs=hpM!?WYUl;V<(Bn36s>{(rRh184uaNS;E;8{>lU!af*(gQ! zz}^}%@9Usgxk-d~vO#Sh6Yvi5sPN6gkgxymlBI6G3aQg}^JGE}#_~H4meyp@;M{hb z(1LFod`Yz}apgCJi0&nk^Cf=zzZa~POyB^){wxlSS~gN zFK}0KawlNyy&|NeOD87bQ{`F|@&>+WCVno~D0}VARM?_7b!3wDnE7pasam15Hw~wt z7b>bj3-aEgjZw@!43y6?WprEaKQC-?Rg_eq%0$n9VWssa=K)6|G1xJVCDv;3V5viA z9wFpN!;=AwhVkGOcFb*m|AgyY?IuusimE8KK)2;GeQ2 zj`1Dq-xd-rdE zV|Bl$DD)|(NSyn!%`X!ItCiI%Z`YnDZl=ULrZM$?Km5#rhm1@ zMHWsMJ_u|xAFoJ7n(J(b>?qzbzWr9C7LGIgz7q5)>VRZ1wlC3zs?Hj088A1z%i3lf zIC87SI&e7pzEO68_`K%q=P21nJ`Uv9>n7<5&a7pUnPP~DbN`ef7EAa|;cVCZ|4vTt zw<|M__vPr1U7Y_sx^Zliy8nzM7!0V^S>;MD8FeCBPcN(d?6C{7Q9IB9`Acv2F11UQ z_Gd(_GhEbx)Ukw;3VUhY$J+xfK(VC` zDy9-%>fZ#k-%FqVs>Tp?`T&~OaRS%Hlx|ZKU~w<+owo0ETF1Gs+LZSk4~{=fTr8Y< zE}ddq(yl8URn+8TkFNQ0f4&N-C2XwRRk{w|A_(lt_rMXgnrFXBU%uCI;cH5kmXtL= z`?R3Tum4nyB@i?|V@h7a)Gs{n{y+ybeW;~OJ06o{im;i!>wY|hhjbthRA<3!=YJvJW*A@LZjB%ah>CEteybscluYk#{P@t=3=w$+|--3 zEx<`GfVaGC1XZTnjR2*No6XT_WO)HmU%ul7?`GkVq?p=}4-(YU_)+jttm=E5Rd6LY zH<^Sgigr*nNy=i>f1U{uZ`}fQ)xH?U zt`9M(qu-#)q3fIr@tgy6FG5qR1n)>5EEI))vwG)4lk^ufl)MPwYA2n$Ci6`h@FI5H}!vO22k&N0xviEs z!#BHPb&P?@7;>c74E_tyo)VzL%bDJ;y1g-8gm)Panj$pctoch#XJ|`Kco4V)#v4Pu zYm$f;;`7kWxCoFG?eR(;MB^b&m-@i@Sdk+5Y7CMBj>Xyw?yB7mRrOk^-}$%a4sW(J zzN&%N80imAq;c+9)Gpfs;dvGmWF0y#f04B!A0>&n`L<7@pnlDf5B(3Jt}UEjuJQ7K z79P=0Me!Wh(>q2#p#?n0!i%y;g8Vs5d}#FGvoCG|j35RyINe|uVFx&NGsASo+tlK# z{v2Piy{G#Wdf?6gfM3x2M{P1#wQ14j$s7RW0hTe;uYRtp+;g>1p(_f*^;(XnnuhX9AI?IuAyv$)!D{?h%8iBRDsP@R3hcdes_W6wr5=Hv*LhkWhn! z3*+O(Pci*gj`S=HA)AGV-Cqt0Z@S!U!WH8H+|X93OQHu(I-&p^jz*GP+xiv%hFc48 zOQZ#!nk)eZ2F=qENWj!VS&4kA9{#eR;peej{K@tDsrd0xH9nn7&(*9Koby$Ao5Mh2 zCol9#<%hfUIeFtV&%1L{s%=YpYeb5XJ%XI)ZytDj(tdGL+gS}HZQ7qtKUw!cqN9jQuL)t)V`i450OsA^z{mcm3Xth}Eg7cb zp=Y6!9}BI%s6ECA=cK8p*-sW9ty^dN0fmu)tW?ih@1V~W#w7uZljRgnF5~w;CgZNP zDzo<>RFwOY^p{&v%D=~!`nkvX_z<}t`y0GOo;W=x7@ev1XfRPUlCXaJD$jEvISNMx z>~aVfU0B|s_cMIu&W`@~l^}SUV31+0ZTurtj;N7{`KOOfj)Fwux1E~APGBrhxyZn% zUDPcuW_!->29X^rtEJ@D@@c*IP>vr-zFV2+U%NTly%YN51Y7TCwd0H|_icaqBKMUK zA6()KcrvC(Q-=H}Q-l2WWZr9co{j;jpgZ3qCilm?wAOWZ5T5@1^(Q7Ss&Jf+sMOIP zlyUgVVwHqVgO5KV2xE{S43Z+j{DEkg#4uR=x$v62dmCIxo;bY_+^+B^cY1k(%Z`qx zXbuyWJmRu3tIkwyLkpcBjwJ4%ZXb0WRBsMCjE=r7upj)ySu*uNqOt8VU07x&RkiJy z&}n1tmFM)c-a)VN8f9nIw~9U7wy(PdrRAx?IoVdoWCxh#8qH_ zJNgEQsvGd5-V@wdDz5zwf_5STzRtQ3KFRwT7$-zTwC8ZuqV@t}Drr4;O7+CapJ2Ph zx0-Cupk3xo8_dZU{ZKXV7nFkWk}GQ{GkgkhBAlg>6Ra9Xqk=1)R)YAWGJ{{UmRvuP zQ|TCRm!8iPuY%!XXpzBpjqq`k$7cMDK5t&XucUmNNMrnkogWDhrHK-_XpGTQ&(^ZN zEa{vd)F34p72Tv<0e$OInoGs56Zy&Qz;HO?m}M7CWiOlL6#S1DjiB?`4brGdjs2z7 zR=b@K@%A8;Udvrsrz%DwHflC&Hym`FjS9|moH)0dlK1zH8$ATH0=R^plToBgg-;#P zEBds(Qx3OPN1(>c;B-ZwkKa9NSV&s7ph^KLxUMM8(_oj5b8=yPt?;5uADWb(1>>jT zn2KtXxBvdoC2Ikn{_?qFL~M=wKB0Irw^sE3tpp>UD5jR4vT7`vs!HS^7nj1Ne4})l z#d=Z<-XL1RARNvm?M7mXI3<~#Ym9G_9*=X=IdAPTp{r;u)VO(U@f9DY zyA;GD-tl;gP8itSB^v`hba>H}Sf3&4RdVX#)A-&inlMxGGInBg0&RZz+@Y(!xc0{n z=|9&ckB{lg4h`mITx|x!0v@f0)|a5lIg^%90kr~`&rWSO907IaAo#nTR?Bz0=RaFZ zU&jn(0@)OLa&r|%ygB5pyQIEiveqP|}-NRmGFKG7j& z^JR^PUV2y`<>kXdVUzh3XJD__xsN-VR$!yvs1Q@Gh5rzCIM0+vQaZtEYfIK2O zKb9g;ZXAlO_e=x!A7q#%^?s;XTK(*Zp{U1j>bgSZm3$r$0mATl$iWdF^KXduVfgg~Q9($4|p8O(3z0`@mVPXJUDqPLA z7_|t6Fb5U1+4?RYcE9LIF0xoHFueibH;gC`7el>TC{!K(_npkS!Ho$uM2;`KjkP zfhog#L&3t^H8nsGMac8lx3B~CawIU|$9T#b`?p(!Ujep{Pg*&~fWw_#9qtZDt_qQ8 zI%2Bb7+nX3mxnuA1!Yr(Mk2xaA9eWY$@deyRgn-v|Nhc`d{4c}tzjz0W0 zH@HjPWBj>G7HjtMu7o8p)!?@M=HK=!>uwWP>ue7S9}d(+BauTD#SfZqY=AsN)f*4A z5le7m=`&j#u$zdckNCf?Fek%!K5N`o^n?M;ZJ|s-5F7|F--^(?5+d3HznNd3kAz+` z@Ry7diN~hX`b5-B6*QJ{Dg%G5PaPx>H4x%|o0Bn8PaTidcpXjH$1(B7KX3SWt!6Y)n~?M)H17J0+qxJ%S_hok1t&*YlGOX=q0hyFRFy~*4nPd;j= z(acXFwGA|D8bRv}*ynE)#2t@w@CtK%6UD$wiW|1an{W?n6It(8;U#)sMRz2aR#Ckm z>#k5!WPX4v48;Z{nEJqeT9LQ*c%%0p)?iz28j zAtBYZJIvfJeG%i{6j6d60io)D(lHJWHc|JluorCC{MpU}vzaXr6(e6@qT(cqvk{%U z_E=|Yn469xV^5V#;ICqkn*pdX3gCy3^Mk&k!Rvv>XOtwP(C}4|Wv>nsXi}aGUWJj; z9F-!&l$BRu?E&FGw<1^$iFrP%<1K|OB*1+A$5+{ZpT0=$MeN7{-@`;lTeAN|M=T7# zGy@y9eEt829g#Nh$9;Vj)5;hic3%p~{V16}Zv3AiFx$KB>wgQcLg3J^jC@KIS4#@@ z<3orw_(NV7$)$1r69B(*f{4y1fqVkR#kRseSCdEhJp!4jxS*d~~%c*9#}rG3!+F07amcMmF-n@HPe| zyDM*Pav!YKcqC{NNPR_Il%rQ(`0}-p_Lt53occR{nGvjY2D0Xtt@AZ6XUL31ZE2pA z8NF%p+_76KZ7cI`*+dl}>j^`8D8lm?t_{7H04EC*+1kj`#cTX~PE(;3$X5-QR+slM zS#k5bhyI2CxkZkKE6>R=?$TUY6x0mK27?U?-Gq1$9>6wAzL13$rn>*UGTf&B81C}u z8-A*ESCR_iUwPr$Ib#1wee<7vY~{)=00&*(%1IB^)o(W*o+r#95H#6208XH~9tIo? zf}Go`0GsjX=_GK!`z2ka-s>~XGRYr}G92c=@!#3I^W*r&|1v<45C7j9pvC;Zruzd4 zyJXK@A`@rtk-LTQdH8@iPVdsu0zhNn%42qHsx`O@<(|?Qx3gjLxUcwrOC?o8>UN^G zU?*+l`3Woj*-48hk`v7vlrNo5^dmIW-ep09dyGt!34R7<78|`6nE@CP#^8*G?L70q z)e$W{RZQdNOQbmN(6N=F#QF~xAkY5Yi1(Fu36s5L)DC{JkTdCV=F?6YV~nq4Cc}hG z6Ebku&6$6Fi&#}?{M~=u#J?Bff4T{z?SH!osGru{LW5?*RkrZI?gRhU^A=8@C}9q` zYGhohpo@|jh@kbsnH32Yfg4lnWxsq~ZG4UiT7k&`b7VCJ9Q2)46@Yvv&LYo+w}q2| zSYI+o4_nYL_fDYPT7wxS6L%?t8I#hI+%g{^yE5MHlw+BMhatiU9tQ_q1;mPg7OC?f z2$S6j6jy_?IS_tjT4pYGc5pGl#2!ky|0+m)-&^FLxC+Ps$)5ky(8aArOd7e(YSm3~ z6?#Pf{~daSFfsHUZYwkf7%13|7vz!S;*alSI2UDk)v&9k#<8?nyd8duco&z%rEDxA zsiw@^!n1=gJ5lB9JN<$n*rekI;W>>|XmslV4TaBd4I>F}N%~fVYV^wu68iUKE(rO? zi~4uG;%+AAtp}q;8{VUDJ^cr(F?m%swk%jyG=L~od!LXW|77giccR7TKhom z*VxlnJs}nmP=`5zC%R;8yPtmSDBqbTiMf&m1+ulX3#FZ6!EFasJ4qKKa$MpzLiC)b z*;B-6x29h)4GFr;E2Mcj=bbSQ=bztq7;i7V{GcP|UP2Y=jdMAcJN!7iQo3NfVs?Cn zkh%nDz=lzU-tR0t-l%swP04s`J?b_t#@Zn#nl9P*Pr~(qFB-7UHALGpwLVj1-_C3e zqv+@x4@2J4mV-!FM@7#irL?r&+}hXO7=4 z=@goNpLEev9m;9Vb8lhL;Un<2k4*B$Y13|AOWIk%JNbJ+@}7qCNF+xmBJ^}JrhR6A zqrzj$ho0O;6;vl(Dahnr;!(xj*7^rWxbA~%=1n7sxvYPnyRBoGAijdtZW$Y=Y2{fc z1`n{C?DZ-+4qH-1U7mH*sRr&tO&88BfY>Mg?6*A%HF!!alJEY$PhUNnRa`fhX8YrG z!r^48a*_LL)kKb2vO376jscPLpWxljPYX1RgU{1U|pDdTPmF>hUKm`g}3qciz5lwO(eSbogD&wZS6j2LK`$9WIGF zCYGkY4qia}G~V$a;AjZQ?lwm@n#hPN+$bYXMF>CoGH4~}2y|qVk&8hTr!l-ALKqPb zW&;1F>5*i75N=sJKhbIOaE*UXc_F&ZEuP3;#v07nc*~5?{leuQM%q~5yfsuW-Y+#S zg=brU=Ii}o44NZ^Nv0HnfeRbfxx#WHfN-W(Xy^E+KVv(DptA_bvp_T?6)e$$bmA&9 z8rfPl*F7GRhUVCg+UlMZ?1X952QG|BF3o*6Z69tp;^R6Q3fypQRNbF$qb)lm7|76M zS8kKtI*rZ0raz_rt%#8V?18goCkBML8P}cjV#!X`?~HhK^a_b+0)ec(do~Z)2-hF( zm@H>Q=V3xu=iwTH#&`ExMCML^QSBEtUl!;eBU zrPj}Q%8ANiSE-r$zT&+*h8|%>9pj+xbP8Ie2G;n*r_W0OX`~Vu6Z#{m!avK(m)C;> zSXY)`p6w0!zzbt{~}}PIT$G1adHK)ak2$v+VFZ z6Wea%X3SXw(nJ%+(yrUDxwSg`)S3f;R5Gunc8(Md=2!+1sOe#m693xx5fxnM3ZjK1 zivN$D5kP==;)!x<|H~2;+K%FZ{)2eYa>>c~e3`qTyZeUM#-!r^0w`nRTpo@8=Y>Lh zf8>^!V1j=J`AvHa#rX=QlsurX%fcjg*VOBoes-zhjasNtED{P{rCw0K#bRWW6~<+& zO(mj`q5x`@Tb$(sQVsOlth6gvOZ7ZUbm`7?+U^uDxDA+4>mY^1#P>%Z=01GPi9}*J zAGYyW80`2eiVeHwd*W^R{>o3<;4YHai+m*i=K_l-U{X%*pvW7|!fL1_OjG+wXWFl!7iso4_5 z(Q{S>oZ$jq)25Mk_pJQtrm}W`xYf(Bj>leW-)r_Kd3xE z*b?0X{=Ee`9xsiFAZNsZ&XqElnZ}WP z=P?U!ij?nB(b$t`Iu*3Eplg4B1Dvyf3sNVes&^+xJJj{5fMQB-O2cI@s{^zqP=` zqZ0pG#$1P7_jFdIpOsRXHt~NlH}yZ7)9uzLeaI~u0T(RmG6%fXb+f+U`&%Qjv;V=XamLg5(Q8>RIm)>y2L7w^v^m>oB_CR( zo+eMhrTZR;(Xo|pDTaTt24x$tVt%Lmum7`Rf8Bct`WvoN9El@PJKvhBuwU+q@;rS} zZ~+v2ioJd>aMg*ADjDCx`oAQ>ngzIbgrh~g3{jV7a0x}=1^Syo>6V+Vb> zJ2dM$TR<8|1_;0!OCMx=)^|Mi)LCwl>lodGJ%0)B&hTC;*{U(04n`-I4?5$@O2*U_lAR8 z6fYRC&m(zo?SPxQP#n z1fYeEEG8Fptff93&aQYNBManP>`};I-9mM0N~>65dC@*+)o;{Y!5B%JLO|8GTV<~Y zXmS))_UAlYiO@3?XDjcLPgJFa_A)Doynft%1lRz5K!u66yz0**(D`1fU&%n5(FJ3G z@|ZLv05RVVhl2}DbJ^eHHx zg3lXB13lIQU{%@iAD&6!^I(TwzU*ZmF`z&q}>le1^ZCG3ran&1o*X4B#!ydG@QMUlK zCMC;~PP?TN;+vt<5jm0+aHFFlS$sc2Mnf!09)tIK4&?NHHuStD_FOok-|)-2L07eH zq8LXAyFwQbCsM`qeU+eZ+tdTU^9{$BR3y%_z!hlA>;na|>y*#AuaRdP$#F`)I9ucN zj;ch1>q$hi{xEK$3V8`+4|FAOrjpC$xdoX5itG~+JJP=oU#ej7b`sgoHMn6~SslRN zSV~FWP(YvOI6LYf;A8vv%Zg~JI36XqDb7O|ZlGjx37Wloq32zNP%Uz+n-Me?OZJb} zK%lqy$>N$6XyJ0O;_eBd9S71;8y}l@-n!Dqk*|E#I8KUm6wo(uQ?%=nVlr)cdH>+o zOybj_%ngB#p$w`x@t%e-TMpzD#1*5y1M!g!)+BB9rh$}u6Xi4L~_T$%kzeX~Zg z>`*ru(66D}^6~}aNd0ty>jVQ_u4FJd@AwZvjt8sb5F5PvBn{9A2dYJgc_m-$`Ph?? z3OYO^J?`aAI5GJs=zX#U;1DMP?23X%#8~i*&R%G9K6|i0z+%;`VYP~pl)|VqA z17uNSCnIJJVt}Np0wB*60IT4Tqe%|(3cICZPb%}7_jq()xO`9cHzp(e=D!HRyW`Bf zOTV*v*Y&(9IzK%sBIf`(BVt92V5oeI1)Y_K+MYaCi1xp>HC?IiGxCOmZUeHn+OG__ zKcu6-w>?BRiA0zptyanwTKy%QfU-7!+iG8O9<6gjutxT&cc>#q*xqSY_nc$Z1rnCm zz6;S2i+nsP_5rVf8h-?3gd?Zm%dQH^tn13)!BGvWy zb%qVla@B2cwGe4`EpS>Y=Kl0@wm6Comk&d@c340(mKj*EXX-R9Sl{55)SwiTo=!vR zpfP$2n+d9%Q?QZS=r^6T+{JX&8wR2B4tO~253p{8FlB>K8_}3dnihlQzQvmD4cc+n z{ts909ZvNh{*Tvj%AqndDr6>ml$AX)GEna% z_V0eZ-|x@o`?-GC)zu&MIPmYaMeBO@j^rV&bOuEK{E%LmU}fk7YKEQ4+U z#XIcu7}OOOkUmgvyDBvHk;wPDlzA(LYKPl7O4fz)LYyNSsn9GN8+YlItj?%^Az4H* zlXof1SA`sn#(p3_!IyBZ7+CPdEv&t&Lrf_os3#{p^?Gmw$c7nMi~Xf{)>KiG--?@gQ2*Bkr{>(;Aad02hLZhBQ5|b6_Xa-csbx;`VV^>P>m=_aIzA!ZbL>kf=T8zj4#e% zp}r>(ilkq>ldL&6?L}lfBVC-|G)A8|Km#<#z)c4ysf46oVcxOBNnVC*c{-;Ey+38Y#~&8L>AGICO(92x-Ag=++WT+8(lS+{TDa_Lk887!(S; zCDBis&nw?eDh$G&D4UbOPX!@`^VQHJ_ldyO%Zk&_8h6aN`|}tZsZX|!SZXG=SiRq#(ttY8By1EuRVa0?E0GzUJ&a{_b3`?$s_9&C zhi{WrJW<;!b$<22&)`(qZMh4E+^FzRN?H5xYDA(J$#H5^E!K26m#slvyXz*;DA0JO z_aK3FQS32mGRGZ7BAeAyga^voB&+4D+hRB2R=U5Cn(n>rl|LWEhDvCR;dpaxJb^tv zTnkvTqsmI9a4GSSlc2(XdktzL85czyi#)FPKVyVTpsofY+eJ+Fs5)1;pZ&L&A-2|0 zbqFh8F`70Nd$9$d!O^60ziQKFX*^`1F@Ec z1SN)qdwz$ToIKvqjX-f5Bo6*!1xH}8{1PH^*@VEg?FXc(MG<}(2qO$mE9FgFi3-{Jb@`|`NnVFFW;?@w;LEb3&+=oyi$v7FfS zG7Z!m6wp6POafNDMxZ3{tzPNNh!*}fYm-y{+_%CF>wi5}MM`7ZPFxKQy@a*N@(P7= zzG9M6xvYIgY7E;i;#4v$V~-<$d+?osYgV1PI<;qLcv!iema@LTH*1lJ)jUll z0c)3gX@*VcO|Pm(d7I)?wy3jQrhkq7h@e~l9Ml9xlX^{E|7ITLf}ZQm))duYcv27Q z9dBjvJlc(?Kbraiv4_bXFaMmiUONdoF1%?;cD@h}ZGe4qC`3VHa&ufa|HUS_PKirr zP6?ABq(Wog^_yPC7Z++|g}uN8EqSGbx)s&Nh0uAn%3Vtq+$*epsre}o`@VPh9SlQ` z#&PSIG03w)J9_J-Nmv+#!fIe5U71gi*MLyb9!MPw-fHhUm-BBF|LAs_cl1iZ-b>AwJ;Gf`gPtPZxsY5Vx}de4{$+y z9=Z4Th8m9Yj<}C`MTfV-AlkUiI(FJva9xG`lz(`#4@Urt6CP*wxeI*k!b@-}O$;`a z7}e;Qg|6^%lIgM{Gh;3&x`xyP&)>&VPSJ#O+e~T!`h9y^3*r(wNHp z&waoES78G&z9L%5!lzEkGXBL(Q3?sm&onGN31}B%Oo*|`A-r`0xs9w0Szjz%xe|XX z=%42t)Lni6aS>A8mW*8hk$YllRzf@oWL$mpvlf*IcmC&vpw^2SXN0|XkqWuVlmrU= z)q8T67cV`kSrLYf7>8znTmSQfMjkO7VfQMY@D!|?BmUx62&P|zKv{`XP; z4FVZj0u<=$+XGcSU>L~M(D5Xc?BYM5+&GOO={C{7b3hveBW3aW)>-7Ei!sZS4cVaG zcnewCpogDam~Ng5k732ifI=Dv6xao;BW0|2$RP3CsTPrm98wfmAY*bm$0hFrFRt{% z?Pv;cIew9Mr1D)xgitb-c}y0bw%;!POAX)jbB3o59xZc%T(8zsU=S)n16dN8h)~mt zQxI?FL(y`E^6vX^W1O>lSR7<4gfX^K#!z0><%CYBROfkGDm$~j7 zP>AcxlJps1?D|iT6XZ+?J^BR8WEdGy(q!MS1|cKDN3K`xT0q>j2n!fWdwaTy^Z4K% zI5I5!wjZhkd$!Uz?p&z`A=w2uS^oABFWw@19|yy;pZ0|c89{S2H8jJ3IQe^p~>Zv!PJ(rl|G^?OT!vD%?*|E@h`p;hbXIEj5ZPwHp-#s6dK6w7gvV)eb zg?*FesX*s>PPvfNw<6@yZ|w?RFH|ni<-4e55_CR~nOV?A`KtU&>15^brS9WD#$Is~ z&f5qerl6Jd{s59OHp2b~_+cUKl!(A|kSi4;J4sG<@-hnQ;y=I! z(f9VQt_lx?k>o||6zQ2=_WPUg5xite1;*Dz%2Vxs|8|JN_5bhB)k`Od4$9X12jQKF zlmicb{HlSTMb4;L4S9-xE>jkUcl#ndd$9!Gf*cEW;?s-}oN;BP7VVMI7 zhHH_g>d!?>2khLin7ilUrjg7g{x@aNoX;CUOl~~&Z(oi)*EsbB_;j+%X#98;xCP6g z4irHeWM!pI5#OlUX*=`+exUHrA2~Lsg`BWaXq2?_=2a~57CB&S%(rAv(l0%?oWQQ@ zu>|hC1Asypwy54U^=){MR-%*r3t28bVih2G*BbFz@Y)DQd5V23+OC#4z7-^KrwC|V zCPDhC24JVewBS%)6fzO<9cWe@rI)*qFx=J%GxbUOSG;@?R2ML_{PX`XE~@ZcvwR%_ zFq{95UHEzIY^-<>D6LzyUk>oit?qNV$!G}!#q{cX4BT{GCQxsrxmnH*!aIpHu1oM) zGr=dsv7n6d5fK5_;7&>Zq5yYIw>Gq(!m_gPgV+y zG!b65FDNK*w*J(uej1rX{;rF!FsvNdUg_iU@WgxneGS%nu+Dni4Cnt@)bPfj z0p%fpf4+H}5~~L{zN}i#WC&9#cZDBh1r6VuVkvjt@?H>BoVW_R;xy`#u)UM5V8HJ0 zRKf8))tihP@Bo%xJxO?kLPR4dM!iFNP%NoWz(0fsjzq9$J%9#I@4st^tBWcyTEhgl zci-(GgQ?TP#1n5mvnwY0tuyhzGp7P$%#?pGhyweI5izCuIyxzm(5mN)fN6~Oe@pEN zBfKV>)rc8J+z|cv@#E!4uHyHArH(=si$#~HKHM$d6%8#~&J}_JM}LeVL97w7VZ}wt z)L_G&z=FXaoesOqtB9;8t!cD|*zfa~gdQeEF5Sbcb({b@Q2+B3ykZs8d;ddkC?Sx_tiW9= z2}A}$rkl(Yq`^R^%!@Z8_8CkQY&gBNQ!qo!@LTq5-b8|yp|vS{<@LPY+D z3xKR)>bmP}R>x-k;!nFjG6b0k>3Abqd|Ex1UqY+#Odq8To`{YT9 zqA+)9VWNrUOJZ*_ej;*R9{vUnv(14noEtJ|ijYR|IvqJUICS786)rq&BM*Zin04y6 zEW@;lpKNv}K6AzaqVsjB^CNu!*){WivrM?8~alN1H!O@z!%Sal_srKLOQr_qg|t55`L zvApe)dE+Wxu43mVsgOH2BuDnz3q>SOrELJQkG3zAi%-u1WwA zuQbPBKaC8+)nplPc#)ab+MCo?S$v|5Yz*{0$vbF;7vYVtJ!7i&!M}=(MBbD2-2d3B zMC;m%5A4a|+IPG0LjSGl|5i3H?A+hY`59o6Z!ya-ENDvYwgR}tDT7=V{{c-*Ow27L zBxKuOg4Lx%jxmdptecEXMBD}+I*O}ih%=zS0f2ylOzm%E!PbV$@T0dU0|GqywgHwV znR{%+f-+-)btRnS9S{Gjp%H83yZ#wwU&LNG6L)@b{({m<*8LZuZ zBhrNIMo-j09j5gf7Lc*dt6yTPq>PX1%VK86hi+S#B!T#9u%#D#2Gq9sTkqEE$8` z!8i=-RV+00YeF2J7q7m*N2dCPFZ$~C4jfBk5J-3JFi!BGtiZmSoYt!bV`s{aMWpAG zZT~q#!ez+NIwi&qF_%Otz=WQ5{>V#?#laMlk}65wL_ZB}R2P}Bq@zPjgO!yP+t(@~ zEo50>l4Zb|vhUATaR;4?uszHKJIV^q8j~yctM$-NGi0DmpFIexZiB`es|Y5>82JYH z^M4kd9!q>?CQpf9k$U7ZvW1bcHivQ8x&;df%^LV-@`G*SV3Gcv+<$*6$0&1f*q^8R z4zxCypaqD+i)FaFt4GZvT&ACFxXda_bw#Jr(dd$tuV;U>ApsR~LnlhcWPY2}Bk*{` z_P=>U*>@0}1jfYhUlc^~M-=a6+dF15 zWDQP2kYNJn4W4MFudn~3^3i06$7_bCu`tQ~>Yq+OKZ}5Jj{v$2<+98JL;*2Y zK4?$@|0VUx#BXbG_?iawXGtuc3eUmeOlm@aZ1V&eGqZ4!C1LV#0@R*=RFDd1L&Vru zdy#8nX4d+@1D!vzPHa9Og-A))Kt%hRd0d@Q7tPlmd8Z_$45k8Eh{Oy#KG};t@4-W8Ia_(u2~7~)};h<(*fF^T^${W)zJW}!|pDt?Gm4Nw;%qGxhL^&g|hz7fcb(D+>y13!UTYRl{vk0OiB2s0P^mM)+ zK~kgUTaPI?kt@J;V;f8p8ty<&BsxP$eHf}I$b!b`#>Jk4xhWMOz5V`)Ya3)sO8fvA ze9O;p>gSif5VXHwcUfCnA9}7Q(@u%bIT%4Sa@zHMq4AW8(8L!pJ)(o$XFIBVbpwDn zn}Z$Ica~9UYb$~le0j}2Xk1--3;e)&5HU_>cFup_s z9B&6Gk+Yc?A-$`X!hYmdnCevf-q|npk169D`9zvMeHwAoEe>EhLRpm5|IhKKkZ{6y zA35qH_8~FN6)D+ntqbEW##hD}4KvY#xhBQ11=NWVa77%UWu?0>EjO1%3as94jhM5P z*Z_)-FJDT#yVY_8t`YgxohH(RubIxT$~_5X4&P3bGT0N+Imu|pngo%MA}{PTa7TgB z9F1FGwMcp1A9!mjRZ&?w&Ijh&Svuq*_*DNJAv0o7P^0U9?;N{Y8d>Y$7qB8t9hl@Wj zgw$HD)eCGY0$=Rn7ahqlTqid!{~?eR&85IMIrralR)Yt{vX}V2Ht|{c_~V1^9lMb- zt6hfNnnYy8`ZelKGuWX4wc*+^`w2bEYT(Q=|5}??uJo^~8Il24RYkOY-DL1t7C^!w zFh1Ol_9;1~h-*IVBqd0arYuF$E2*R)n<`m$hf~0{Bd9pC&8)>s6CVcI1g!+340J+ zB(asKE||fnJ-hw|c7>Mn=~n$-m{Vah!OH)22m&#pC>1^&2a%DMmMPeC(#2#cS8PgSmMR*jIax^sz6CIx{oFrAKh?>wZZJITOOq`{~!ym)R5Ef8K2a*J2dh z0|#h6bT3>epyV>FjUW&ytaz*St;ECua1{u}4%Q&Iwdf@R5;w6!&y15jYXCo4fd zLHsS|2-rd+AR}o9M(zCyNO-$XxD9>EE-m|6W~KVc90x+9x-@r=!Bed=!p^*_YMk$u zfem<;6n95c^7pbuejuXQHap=c50Q|(4~whrT}=FBTF8jLO(s?H5k)B_(SgGAdoi{& zU|Up<0Dz@?@=Crc6DN0L$5#DT?GbAKL7*^}NE#J=rB40W0)rMn#k6p;+&zn!~ zir1V#oJ65VKgSQx@*M)}qmvEi0w@SDVy=_uAN?^o=DatR$aM(ViZg>RHEkAQHvOsQ zC(x;2=E%~@YVjToHW)mp?DuyHv0ktgw|K$D2v&wi9|3^5h!!(;ni=W1K0DUQiPOJ2 z)dXVfbw1rJaS}5mm7&i<+RH;;QvZ^E)Snx%pTE{{{1b9sHX_g_O?qZ|cjuGKeD(VI zomH1LkXv3d#{u1Nfz0I01T_&V*a7L{$7}f1U>7VxYIaT**(^NX!uN?~7N(UK8%%>Y zex_E|wG3guJ1Bl8sZR0Tuj>0e6?aLbhsb9mr92Qvg$It7i}>mLG|Lqi&zw^b*LT%R zB2(>cQq}17Va`fh0rabypcWG_v+Z}vAKWBy)2{TB&3?2c2KHoAhX|gf>Vd?r!~C2* z@8GQ4Bud)NqYRI4w4uR6*E0`LGp&GtZgk~gc@({x4~X&Je;NrZNFuGv?W+k|xkX)h z6;hB)R}imCg#{-Tcx(IsUv|C*^`Zg~@U07p&hwpIC9a3Nb4{A6 zs^#*ek^}SyaOOCs8>5Zr^ijFuEL$959|xNuSd^Ba+e)InLdJyEWX3d$z2d;_I{p^d zK{YI|jS;awpQ6@&UV4WTYf8lWp^=pONihVe;nJ}3)c7+*Ts+=p$HCLq#V zdFKE;+7SRS?mw6VB+uxINLk4go{2Kb!||+{W*Me-5fVYpry&p7X$53F1q_qQ38xzg zX-_81OQs*pkk)h~t~|68c+(LScPCnEWW;)hHh+j*iEi4F9-4Zo63fae_JppxXiI*- z^(WEpeJXtrFdB-YPf*T2e(o23fM}r#oX)+i5CR9zUH-yEOV&mWVjC`y?97gCiNmL{ z;aFnz=0Y@lL%}^^L9ySFTYsTnbiM;u4k89z+jnKQl|i5BHE!NF%~{W$Jp>K>0?OH} z)U51%N<4NiIww7zgPF_&cGIk5r=?R7W#cu)y?y|1uK!SvvD=4#_TIDf{L5@s^j zAGW-AnexJ!crUW0UVUOo`rN^SuyN-^u{8Gbn_g2gw(ml|mG95Iyn<>%JfP60#Sp27 zi@|CW5MdL%I*yh*kN3oJbSpnlSNWRVh^Vb+vS(SI#~h3~)_lz~Qke{)x?;sp7g#jj%R~ZVI;XwNz ztSjy0EjWan(fb7kMk%Y*`;94ziP+ct$Cp*bIbM?Q8=qRS`kJ^qtmNw}pGRli=BK!Z z5(}XH6N9%q0eB6+X68ifho=O;-Bw7TJpg2D6g-`A7ts@A&~wWpKE;Tj*#&dKoM~DJ zEggm>e|z_lO&GR;aOrzZDj944&5dYFhur!rTfzk?5CYS}hx=dU&~)N2i7UpQFp+>f zplj*-G}|qhs#j=(iMw=IvQtvde?3Qai}a?OtuVc^fs`zoEDqPFB{43+!p3GK z?T;myy-Vu0A4g$2ow2b#QAvZTM8S#qUHEbTJ4TA_Ik*e{` zW(xlz;aeC#z1(AE8a3(MuiWo6SJk(_G!ib;{p=;FxYL~l|?fr5jH$geB}rc=1XNpvs3jOXr*-?ZCz%qlkgE-C#tKX zdBKHV*wA^z91p0b&J>rX6RIvxKr{(@jI`t>&f?MyKEsMApF5{U7T{#sLt^sgAQPoyyfZ%|1UPA|>3UKyroU%X zO+AhQdMwr!Abc~Z@Zvh+R3@`meOMgo?YcFE9QP0hR}0!bubw%%anr}@XV>HG;ur+>=_nmcvxZs^d(~UPbV%z`RPYfYhUD(Ejz1*~OH!fBMMZ|@(cv=x1DULdChww8ptB!&0LFKW{ zUe@#32QkLpyOz>44`L7I-M->SyN(Kf7&j56BLr`0BAk$`gW=nXLjU_AKAvHgG?`__a zJtW%iH`PO$O-!IyZVE7bXSriIDn3A7d>E=7iU5UBtm+19K1Jf|y#hMSA3@o?`c#t& zd*=D`WOP@#FCzUyD)kUtYMdD@PR(a*A3g#()f+iK|8r!GSaAhus#M0*RKub+J#?uP z-Iddy+3PHsy>QU~%_2QLy}0A$GYlTW_!19!^esjl&@)F6Ic;G9Bti>iS9%>1CL@NH zlB)u8W>4(TI-ju%HqXtVL5Q=lkKA1k@G5r7L3>@C&l6Z@nT_o7v%o+pkdAy3Lo3ic5yqamkMN-&cENbnVtkh(J$n(X0({!T62m|V2*5P{mdoUx$$m*rn?+xrv%JIq^L;RWqU2}#ctqAreJn$$)`B+9hhU0M{7c|nKD zCc9wXYa3iU@Xm1PkRBVYM8_vN&3~{1{3oXwTI}(#McfEL_Ot4x4#%J@ygQM^%6a7 zcD)?|i)_n0JdhL%(#};?y#TS_x^Mm*z9CvOzMD%Li^2HHJ;c>d0x?!&ijp`8$EQ9u zJJ@ca=IbYV36;H6)DlMO5yy5Nb33*&t6on6XsS(=guWlqAl4D}Xq8tcbWCq-V0B8qOP8236-wXH(=$+DhkN9qs zA7aI%mCD{fM>8|xzP!>9=6~1(UCBGW7RBKVlcON;!hQe#eL}Kz>#}nz`KmD-R{C{{ zXQigEd(QqDD4-meBFBe&xf4PjzziBue}H_y0VE$MruEHn(B5u7Sn@bjX_68zYKhx$ z(eeZ)ov_F1{T=Sfxl2I}UM`U>4Kl>KYT7+<5Pf`ce@lQ zx?>Q34Os@vGW~qIRT`yahOa;@wt;rJ7Tf)4&?fM|^ zxR3bj5k8#N^}rM8iCVKF9j$gN6qiu5sC=TBLf>e4DOF6`swRwMUqMBI;xkaa1M{GE`G6lA`$-wWl z(lSqEA>ls5#<@9eh5P4tJKL_jYmCd6W2~h_kU9Vke$unP3?y3Yg0np?{tPy0JFE0u zr!dDc+m%dy>e?L$;^f~LNAf=AN{mXQRW}!1Z;da-Dym&M){!WewLMf?nzxJx9^-3O zDkY1t3V^bCGwIgzn%ZI%JEbH=JmIaI%KMZ57X}SO@X))Dk8AmCL1vQ*P36I+TfS7hE`jiY1mvi7Ec6F?;PuxT_#_pW~1DoKGhv+W|m2S-e%j(HvdA<~Qx*ARj>S+4$2*m(%wAB=^o_Gd5;a%f zhn)HDfSXdgB)~kj^kXnq_6;RXjMtgL3eOr%R0A4&*)Bxp&9YLrcmPi7&dX`}9ec@VU=gUWzs|1vvD<_mHFShGrC$t3Yq32d>JXP5nt0UTJpmaU~utorSJ*s|66 zGUoD8pnEUYw_Mfu?9m0i(atGR*K|IQU~c_xcO=-c?y`D8+Q0Vq+*~zEf0^26Qodde zWJ7gh;>S%5ChPUL=Q3>F=QG8G-R(mQea9LCV(H!Wu!l!Jkz0$2w%Gw$)-?CyZ3=bs zk6MBzbL5M843ZoNhN^ex?$_XFBn->141)V*DKe2{x7LvTFxxN1IzKz>k>9?xp~15e z=mawy&y>Esm~X3-cPq!p$Dvs0QP%uk&ENh`Sh-#mp3c=1u6JV9v$M>6mTy0pZC`WR z*;q?tw;7I;em#6!iAt};ctodsu7}&X+&XWpJN-h1m1-m;SWS&;=Bg(`*6LM1Ryc6! zX9$0CRn*ZG%HO4gv3`qp#FSgY2O}qvB}0R6NNUQ{rQug z)La3ZncPB;hcao?v8#_^T!1+|GZ!dc@{MZy!SkG_h|QZGx>*(k;{S%%LAX zT(2xw?GJ}ebioML4IfoMPsnU;HWH+7LytfAZ#TW%=hAnDEtON~eYm6I>4hKOef>2n)3I_Q#bRWH zL)c6%l>V`D6ED+^cDg&JLnhgF7HyWJuhlt(ZAh3tazJ7=+5Y(O%tYN!mBda+1QZ`m9_NilQWXIIKGtn5 z%SuzCv}I%St7CP+Q$za?j+r(sVKz5TN;!_lGLugh8x{Yt=+8ZC)7u&$K5qWPs;jeN ze>DWM$70(XU5gd2w$EZNCOfz-;VTLE%?Muoc;~WGHA`&Y7|2~{B0ahH_lx@(hJbDK zzBm&kObhx;5*zt46N({M7cL8yS3s0QkfRpe$%K2A$wp%KHP#XIu*P?}CPN$B?bfV} zS&RgXcc2_tv_x{gb2#C(8}Oc>a$=d97%SLO2mNWb0dgD;CubS=8yOi{m5QW&x-EJ) zNA!W#>&-^ZY~Taw@_q1G{qp#U&-#pm`)o`yWckng%%!gs_s)Iwh_sxfJ@{GYQe@U5 zks)?%g29XRJA?PtF-bx!@XNe@)cM4|Ge~|2omR?ZC&r0IVsBXxUmj+!;4zVG z#if%w(i}1t&Ek{o|7?j`eEIDA+38*eYPpBskW94E&n1vy)@a<}mgUzEXm*r218O)1^oX;(oP7$?eJDi4)DPom@B*KevoCSTFpv8cpnzH%e2Non&&< z)84nZ>1&^rYTo~qxViE>^*dxzx!GT=FR|brRWS|W-jo<+JRCC8{c^*j=c8Tm7dJvz zqVnC{`Sa}E>)kGTI}5Koh!1k)%U9}l`k#mS_PwwN=1bZBF0`Wl^cj=oe{qN6V(ZJ9 z=xZAeO9N*!gn1?^9Mwy^)ye^hU1_`I5{333s}Mh~2hR}7Ek+>KQht2E-d-XZ?zwDz zlHX^$=Vt7qO=_j1<%pkrvEEM*b{zqExC^X6XDO_p3HpFMJg_lNF{KIuvT-QjGP^4= z2`=IN@SfXB^!Jj8VuQ)AFXf`k*tJf7Lh>5-?yx7`S9wmD z9y`oPb`pEJfZN$~R*cRE5;xWHm&}*NP9T{aF|6jy>6zcQo;wjO`Sq%dJ}9gBeaK80 z-I#|McNAb$Gk`@NKRE`HpsU}wyJ5=j-_B(2AJ1Ld4jY7~es9FKYWrx6(`>euR-&8c zaZAIc;`i@M66)5tGDTa>n|;418fsg7{KAN?ZIqYuSy(G%t6@sSs_r&t( z#jmg^=e@tpWZNf#fB8E1>_RmoM zC`;V3)02_x#`af7ONF+=&gP*G!|_WqIB~a5H|L$;-S>K)(3sLVGU3_;|!Y+6GNVJJg&-FN3PQPwDUwpdeBppR4A)%)}tLOUR(XvY4WM3LX z3ff;yQ`0pjDp;!siis2%O6fJFY!}ba7|cyIrAV3w{~mgGX`+eE=}|&jrDtOv?SW?+ z+lU@7prHqkw=Q5a$niGMokNWte_p~y)|K}V-@`Ua;T*fNxvOg&T00qJH?=>7)G`pYK1t+WSpBo5~1DD-Ozg%8L9CxmUJ^ zc=nqJdHh!Is~T2xSHRc(A}k!wjrd*h(5m2gCm^NTJmmM^KQ{UkHDj^tSx&ocoD;5z zhW5WcIy$7cgo&T5w-~Q#xanKtUK|&#lK5Ow{ECKa6ty*&&8y($=t3ane3=hxDn9q{+`$DS;Ul&xn$4HCd)Yx=Kt7&}V$>=u{LL z2mZcR3FUY*edS}L=C&MT4)MXQA)|Tp>h?(a@%mGyqBEqTjqYk`Q~o;UXOB7(QUW%< zq<~YHGT6{S+2^@Mj{hVwZu)6uRDWsoqoVYuOmV|ip`j{*YToX-`D~9XJ70S}!racp zN_pouUcNEV|I?>81Z_on(K0zh*iBl~XZ>^Lw9IGDNAKTsNY9?_O1m-GpRZn?^v*Em z7UL(F=+zjJXY4o^qVLmd86t+it-o@Ys8Du6ZsTBdmRjs2-jd|v`M#_d{&ocS71n7L z4xzdzx`-NZy&8v!>Orgbc!;fw%Gz^k=|49ff-A~TB*;_-$0SRau4WvPGn&qT#wV(L zZr7nea-MU=$a^vG#up?7(hJyyQq;$zZ>8u8+e;A4XitH5OjC_Z-j8Er!jrz~U-S=O*uXEn; zj%ffyWK&1FNjn>OMzdDm-u*8s5gG#qcUbGnBQ`_xoFz@0re z_LpCZaP={b{q@@9b^(gO3hx`EvCp$K1MmJw%(o7@I0;p%(!8R;ggKviI4~@9`uNSO z=eiO>vLdL35=Arnx?}H_nw?uo#NltB5M6SrCVSwCh23a_s@|( zA|$%4*RcJ|*e!XpZ+}*P@@SbUsX!uzZUEFHDuEZ#g6YkfE}ho|#>6s2-5zXHlzA%9 z4M^=Foj(G%0XN#P2Bc#q=1&xGJOa6y7mU(S!sVDBx7J=J+A zTk0mChp-vXk%26B)7=G=!YxdvbrVh5uv)h8ceUT^beDs-yY*>kR?zxs^m2dCm}S2Y6=lftmULjoK-BXS;3#7u~;Pr=YP-stnH=3 z6anX#d5**Huieqe*Wg{v;=fFN#5TooO?ch5|u7`N;+X{8K0A{COAn_ciP^Gdlsr%Kc)Ci&-+w{1cL zf=eNmc==PEo_3Jd7q%#?>_!r;#?ru^%Y~tQbekM+oPGBxQhO3Y>}$XHjiu|SpUBsA zx50Z}#V3*t>?vu5IKNx>YIrDSxoX$PFw}j&okx8UIka=8Bgc)_?_yXVpEsZXa#H}m zGWyBoYO!>D{Jt*d5p3!)D-5I{+@EA`tOh?j$6gbkjuh<(`|Y`k1KIdF zZVigbx=D;s($@Kkp&?N+XtY1%pJEs!M}PY7J8*Gxe}F7d$pdU0AtK}8KKNkQUKFaj zo{RWHs@$DrCTo4Ry{0#rfK;OIXfykG7#a*WbUgrf9`aP>P)0Wbl(C7A5HnOvD3qZ( zUmp!RiTXm3H7{D2?p#i|>0g5=YA^Tc5HI&}=Tp<3wOQnfC^0eX*>;=JB7O1ClJE4( zC2BS`cse_S^brDnv*)Ar_#A#IgE(TW%({F9lf=vBNBR}^>0Qs{ZSicXDa+c;6WPv4 zsrA0O(m^?aOw~n)UmXc;n`DQJxGGIY@IC8VIw;NPC zIs_4tx<`4VrBFxTlrDc{i4|==eU{b~$VYDCCmUsZC2}T%NN&7Y(Ai`AfH7?ThWQTi4`q`|r)HS|)4E=>Qc)B*w;=X=z8v75 zbF{b}a|E++=!mx$-%#dh>)fmX4nBjoS*X-jY5*v=r$y5Tpk*YPyFGb)G-T|*YlX0C?15FiYSX&oa(UYS!i5Vr!*5aGp`Ghf zNK<~rUsx>o-tQjTqfpIbl4Tu4XoI-;b$6z=ILwUw%@~SL^|5E?LdrtiV$Bxtk};0_ zUrKvh%J_NJDfWZ&c7e=fVkpl&v7|8EeAE~v9tx7(#|}LA$&WEuClR6pC(6N1^H}7~ zv+$6?gKWVcC?OdFz;DIT>hB%aeK>@@xm10dRM7|^oq>#p9+SrZk1U$`iSYMtWhx{cqxH5Ha!kaZPxn~lu=)XbZUzw@uP2GPt0%*KdNhn$<9 z-l;tdKL%QO^21`Ze?hpbY%L{3=cGxeN28kBxq9Ueu!Iawg!YdBi${Xf6@+EBzS%mz z=~MgQ8#9gj>=?|XM4bGm+&^rmvp=7j`KkL4f$Q_4G;mT~(B^;v_^YV}MU!Fs zh*}dHcSLTHob<_&`nAEnNg5M>dChbU@~*#XkBs5l4>P}II?FLCU;{u5M4V4DfNWr( zSrp1$j2@25*uEh8B3qTXb^4#&0Zs92$Zu|w`f>Wko6BpT=CgYCd)L4p_P2%J2)TSAVU&z3 zhV@A!U&H_;m%(#3;oHEw4ENeuZi-HSnqS5*eqVtsE)pXbOX=#K&h!PBNU@SgC+Ab1UK(%8 zZMVB^v*0l?MTUSys3dp^yz*Zlfj|5BFAT?K~SHG|DYvZfPB;qSsnpE1^ zWPf~M)+@0Yed&tvO*taW)-HtDT|H!~og`->1^5VB)B{op*~0*vyq(WOA#INa)wE ztJ7yE;qhle!6f2Q5_PgliU(hoIG#>WRk&PXH#qsG}h+N*xpCEIN;6DA} zJTX&SsW!^9hCC9ZSn_+qxBT}80{RhYe4-FS=N*r;o0{7=Nk^vlW?Vu$Ohawdo9LJpOIB)X)1Cenj z`Gg%3EMOo;H6Z>o-uqf3bBP2s9l_pp;!|Lq93#e)?AkH+1 zpP^$5QHr_WG;`AiC-UmIQtnmbnqP`_W{LD}^h!&U8#ldZ*D}Vgm9WV9NK9)E8kf9f zKjYi%SF4o`r#GQPA@6x?Jd9gdu(}8o>ZP%LXl%dqPcAJdjA7}P3#HCA{{zp|C#2X0 zV1rz`W*zNAid&_GeBBY?edS6FUacX&5bkKtE2VL%ux-S%sHB}D6~LcIH?_psOxhvT zQ8OtZe?}J4`t}|TP;qhlZCI-6f9%xOeOH#09pEngv@wnzn_cGq&dP21!7fCQyXw#M zni7<*Ow8-&qU)vO>E^cN{K$5nf6XB$UbwgN`H$v8_A^Xyh|JkD2S^^;FV0)s#8>*U z;|fw<*wxpy7q2`bf67EVbkDy%bx_!d4@D)*m~%?1v=V`{_BU?6NW7Fr%5?FvTILNVW=~V6M(;S=&{p4EjWS1O+MjAPo`K>AZ;zU0{#uf#YbQY|V38LPDIOeBo|u zSH4u2ETi}sHXknWNWK@6lNq#L9J z6+vnUX=Dgtkb2kr@B4X<GxtLa9ZgIoLV8>_&-wf6w?WUizL2+8ERfmKtOOhV)+e=6){e{VCBJ{AY{ z!};MpYY%Px4mdNtR$5-Z-KR+nc|eUtO)65pxHr8un8LvX&d`vKw+qA}+~+>%PL%JY z7I$-@9}w)9y}f;Y=erU7IN3g&fJByD!ob;mrnZmhT#>ha|6B^#zqmG~fyr+^h!c(Z zm_o@%2yB(bT0BZP? ztB7A74VB;({j;}ozXFs+|GT#4%s#Ro+gg8=Y;-EuO~GCR$L%#?pZ@{3B99cx9**Tg z;mAt)QWM(iLwG7x`+JseleinckeZXdrTyRK=>XlOv2X=ofE@@+zEwMV`f|fVfmn~8 z`5YLu7V>?FKdDU?EQVaobD@hKRa(qk6z`vf6Bc3v=w@EH}yC_u-*P!VNc;K68tf!vMZ7Z4CXxD~E1AWP`gHudS5@ybrFambr`y6Ad&uD=~X z&(Q&;J0 z!M`(%={HMHWj|`A-FCwZRlEf0l*M__zJO1${X|TuJlCi>a5&d^S0hioWH*Y!OR{K*J( z)tQBb?jW=;XuJHkt#m|jp*`*8aOMddctVz`&#f(Q?lHKxPPAi}iGkG5i}2-OG@7Y^ zeT;&PFzNzGuIzobzI)z*BI~$3gw5-b(`Qq#Q22?r^JX+hqQ@O*t4NsQv1(!Bi+CTf zEWZoc#E_V%afx1eI#p+Xz5Ow;5A{-gcytYQgB(jGa?c01fOz01l@tZzI@iC#OA2;d zO=z&CtQ-!D-`^QN9s%f`ryUZ|VBj_0iJ;W021#7gjn0R>ay!*Md5YyP_C7W)Y&D3) z+?fMMdA`ZOnUXI-gOtoG)dG8c^oqCehtx!CE>Lub&;w#|92m5P(e&PAm4 z6fOqZo)c~F8(gKV~R|-J~lQs6G~mcBhxaj6A=PZyttbdi2Yv5N-^nfP&y+^O0*gJ3Oa!}10)Zy z6DQY0Sjd2aH1{z*|NHMLQ}020M(Cq)kunXCpw9qT$Xm+wQj!m5pWYuja$w*sz4H$y z68VC`rQl033VlY05nJSO@D51Vj4MgYt%Ld$zM?-_HsJ=LDAU`fmnTLZf{}Nykedx3 zAGb`&7xv*@{l1@U>^Xjj4upgN!J1GwWDZGi31aq-fP0Ly@6tTlI{H^0npD*(>_$Uk zB2d#*Oi;@`ExP%66@yo^`_j^3a#Y3lQZt_4tasJ10d99ZDO ztawh@*TH(BxZ(&<8u0S5_UND|#s>#IJb$BZJO*k!HIQjIOQHNrx%KikmjA%9y>+!F zibq~w`;F0c$wf)Pyta;~5>Lq-{vRXR^>}AS51r@%Hx>(LJapi}uz1U!?RBgM!bI=B%hIV39Tf$Ln033CRRx#<0M1 zYoV7DgzmYAu`3bD^7M3?AlBkxSuT93(mlLnb?(g}sqsk3NDFOD?(f47+1F=%LVzfE zv8*HSxhb20y9XC=a#7M`s;Im@?CXmO055;aT3kZ-yY#i=n=|de!#$?0ulW_IWAonj z$0c;`O}vge3-N{WoQ!rEEn`iur5jmrP*mA?qV#D=2_1=^fyC7COKwjMEaZ~Bf8`Ev zTY~yO}CP|)@AY-LWzje6d{G4b*y0zfR~uv z=5s+dCpfq$?MjBGRGK&vBw`B&3Xk$5E+;gfqlvD1X?>DkgY=yuT?lw;c=T|~D0&16 zyLE9?eYuGqYDZR}h%#olTlF=PCS1SL+;*1*7?=GzkAa18rY}6nuVDRwAQ_nyp*0v{ zY73agjX>bC-u1@$I-7i104AKbQprV5bVq=`dqNHs@`bm&Np7Zh@`Wrj_(bMm)S^dc zI1gG+&m|PG|1a+jtHgkIGu=(|zwxW!@fS`X>hV1m{wqX8h&isKMRyc%pv3iIA?~PT z4^?neZujo6!lS8iuW@f8rACqlUCI}fN!4dd-1iw$yY8V7mLurY?E?xpRga+v%4qU8 zFc{liu&SH{#rh+>=HKn#uuT>8G@_KB`x`y!yQ4f*Lb<`Ef;OZ|YeE{>)n2;`!k>A; z_5h4&Me;xai7NQ~edq#AdcT^?`A2$7mn$BVj!|NTAWeyaFN7VX%!J$j#o)xl`wCGG zOn76oi(i>;@Gk8;5oU7Y?}$ypqE3H@AUb~qN*e5tePEabBgu8scL2U& zIs>OGfFVjr8#Qtjh3as^JpcN@HqhqTAFe0Fj1NhG#<1nR$a2J#*Y+A?<)^vb7!5BeN^qIBcZgw#B{VovX1&kp! z6g`B(gDMS$prNDvUC~8;ofs_d?Gp+Oa$gmWS#$7r;y-{owc1HdH2RwO)f=34*9v^l zE6>{PKmG?E03tuWhn#q|5p3t63Zmn5oi{D!ars^8Vo?tDc#o^ZD5j+jAj`;@pSuQO zahH4aZOHZ|Id{d&8(l}?o$Kdaz&2n}h!blJ$g+Sn{A6xIf{CIn)Ngh9s?q(nK{QbB%vDC2SNzFu0|vDyZGw2~{&2l$l7!d=*wylra6dZxYp8J>zkfG}e*$F2PyEuF zDrn;mMx!%I)NIsjxcu?1HaWPeC?_qSk<)NWeYpvT#eI2Xg}`0Ueflq&a1o+&Rx5}Y z5r)mIHA4*@ctSC*wX=#N(b(^O2AHfOXM&F)xX zvYGu0Y8&30gJ7rs-O6Q8U!*vNN63i2X;#Cv9e|lh48m!io{F|I_o@N8+_S9455QuJ zp`?Qk-Wh4&v!q2}hkonCbDNizD%zc;uNQ!E5N^*hU&{AB3M-bw8KWl*NOl8>4<%%#k;xhw$M5Q$V}_O-AXfk zaWt|i3Yfa_of3ts=T~}0h1O)p?3h!z6S;q306~|Ggr*vyDRVXdSl4p09s|j}J-CSS ze4rm!TcCoT0JZd=T0YiVt&Dy`;hLykGxnT!=zD#t`slPEYrwz}x_JPC@&;g3rDifY zu&i2!T!AFjFD?#PXU#zJ_1F{MDh%`fXEPy!i^fH4!aUI*!1gH#kfO!}egIwaCz*!I zKBmW>_|@5ID0hc9$aO6kZ^x1NGMK=sUt#Fg?gJ9_My0}LXIc}Sf^$}~m z1(9}(-do%)O{?g)f)tA60&o3f(4Q6i1KX<^I-Yh00aTSSj*$lTRiHJ>YXV5vhCt%M z_)z~pKs0wI8&*hm-;3l-vh}xrC!m0RE;%EgEa=og))5RB0vBv!uT%vmYFyd&zp!Q{ zU`ZvbaH z`tvs&H!4O2T=``mjn$5_2hY0hYdNU}c!X#|Jbtex4tSh3bNeVVTw^p>%I|6n#VN-F z{1|iB$qSA*&GxL?3Zc4NU4XO?^gxfW3eI z6VQQYY=_N90#8?qC%T5)BKm!T=ZwORw+AIt;?G4s?abjqMEH+wKT_V|c zh&Gp)p$L`&e>5+3*m^{d;dvX7<=tnxX;JgKGwRNhVHn$Ikjm$Wa{IE7bvTikH^7hw z65eESp}3aU{P5qrI~aNrnAFy6qx%E4tlogOJE^L-K>#yY_n}4x^7|I{k|(t?dk=ps zzUJE*X#2Hb^eFD2=hrK!6Xxw0H6{442TYrhezltDmhgGZfdm*%Y00jdL2x}N%_zN> zqGlSa=Q>kcb^%=RUQP?JNYZ+d_us3F%aEsKWiAUATOJJ;J`3z$Bma0;rBhd~BWqrK zlEv5D#a~EDHIvvS`q>l~p!XzxWoaGcZ!u4}3kt|&T9@rp*`6i*giY`cg#*YvUI$2~ zPi$~S{Lioa5LoBlkwcq4abUXJY29V)E~|-Vr3~}4&I$qWjh`gK+C}6{lW#oV#@=UG z79hGmC1DWwl2wB(c-`A!(Z1kunfT_Kh!2PwqVDi+YEZKxAYX z2P8(DRT;8K^D1jAdyjSOIo$HD>n{mEqUPfn!T#Ctu{-NuH1~COX;L40jHl@%Hrt6+ zTw&+9LZaNo&OKl;8fbpmx!#fmrP@g+d}_*c*W}kow$c)jy1%;N{gA7WD`QR!?+XxY ze3$4SQ3JPa=Xkx_1?G7U`uDHnCV=S{JRq`Q_VLR8HJl62)4E1G!Kt8rJYpkxKOy}q z%0M31vllvJ_=!GemnCcYhGRyu=N($XkiH38KXfXC$KpL-CLB6(#-KZe#{_*F<((+5 zaGZ-qk)j6=Xt?Q*SusTXzXRj0PN=Xb1Jx5SU&6T&HShALN3Yv;lO+(Q|9hrycYi@> zFOR7KIy;TW-AhA;fLs9gv=jrAKXvsIL>(%ulxkFFQ~H? z=xqvZ?~s@&U20$Z^-TNI=1)QMSSX^E3!|+@ZAkN9;XZwyR@sF|Wh_dDg~xuSd8c;x zOPi~iul93rj^p4f&jUU*-e3iSb?%5;HroE&>y6|q_Wm4sx9`7q0zKXA$5wLRBVKt) zp8!L>58GYNH;U-A8Pahepl-{bc5r&d^i|I87>dD)1qY@8VIlF(}Pl#;RX z{+IAxq+iY9ZkgY)@^eU8bibtUr+2jRD#XRQj&cTihGf?R2OWP_n2KllhtZ4d7-3(2 zvp#uJo^SLJpZgVEzj{9syA;*3XX)M;zGwT78K5og2K+UhQQrjr4zPUGJ^3S4pP6%a zI#wf3oO~x9qeIP!OsYFe+7-AP%YDk@W>wTol72gvDTpG{n&Cp6+aqif2NU@G1Q2kE zb8Q!hCN^j4oJ9Pe>yqFKAjLzlAGZ+oShdpqF7>s7`uh6&EahjatU{oEG|(q8GWB14J(-H93y_Z!dT#KY}+EWL*M7l&=jx zOfN!?E{HrX+N3>=yn9^!Bu}%}GG>5F65S5JC}S3Cy_4kTP0Q#!quZjjO2X?zbX7KnEA74lsS^7BI^ zky|hSZ6*ued5^t;^o6$8&k1~`m6arXidw6!mzas5s_TJp`@Y~gk3pa0NJ;vJPEXhb ze_zH$bjZ=H3XpwZ!N1!rBO|!MJ0Vd3rhx2g}u95sFo@NG(l*g8aU9RQ2 z=S7!$I@7?%yUeTDB4!@6gKEc|0-M>sMcuVd72QE{OPf5yJeJq$!(b<0XM8CA(Zn8l z53ajtIZ7*BR-fgV!xS;OLv+%D&tc$N_u)Znjfah>M9AfN2@4x$|1uEBcA4`CN>g5i z*9Gj{{c2(_DkVXEhxtU&+te(_h1%i75ef$H?sZQ1NxD5a^({YMV z8N99I>HobGdNMIwhVALOd1)Ze!7gL69Z(z>h-o8dYHjfkhF|#)2OxqBR8yM%DJGQO zikS3JSY}buxh{QKNI<}B7r{-y_+<4^qk>u=&ZLRv2n+wjZ6VwR9FX?*GYq1(5rLe{ z=X>pEW0Cw_>i`ENbbwx&aR+!p^ErP=;Za!x)<2HzQrnq|bm6?XhH~YHC`=-&tZc}U z21P)yWOCHQULqG&*3(y0Qm_)(!1k(VjLN~u^j}7&FtOlc9FbQX9C>z*eD<29{~3l| zBSw>TN%ik8*g=I&pF+xB9NS`Im^60a_f&Fpl-0lkBB7bkrG<%}MZc(2s`t81GA>IO z0Lw3i$d1bbJjDDsynQ;dXdN__A~vj9X3){)+!h36a%@sK{72k_lY>+5T|jzqc=%V1 zJWZyuiLh|}dhI~2P*XMWSAhBP8fUcwe-gNl9ALryqqdPA4E-7;h=vE9seKeA@}MMf zeZ)C};z8OvNbX6Tjlp+#TUT=CR9dzZ5?6;mGzad+^H}BlT6|ye;BCMqM1zEhaQvQe zKQD^BV-Z76xqJY^B$*^>Q%5;C1J4ifJC@{=bNcp_mJpeU^GuR?qCDU14d9-7VDDc_PbR-(*-RFcN z#yO~2ier=283}6#0vY~G{B_xTqwjN4voJC`mWMIH=y^XZsmRIhW1|Br9hQ20{Qm)= zj7pYoE&&NK)82V#2SQtW_Az)B#cqp|<$bi?&lu5#C{Q~obP}>I5a^YqO~y4LZYTz9 zR!7HhAd*S&vn;$~hAHm1wt=}oZf9Rk&{}Hj(dqKQr5%*w$DO{PB1M;^T0woA$U%q7 zhb-z1HkG}BLZ7TArSAi;ak^{GR24M1Vl;Y(NEXl*f~R+*_X%GR2FlE7Gs+N}KZ$5d z?kQBeX6acwtNIxyLw`U6Wat{7W#tucU9ujiSVa;M22b;x)!Z^#A4&;XjPpEnDbggA zc?J|n51W36hwnC+ICuRDk1DN~)?D28Fo|{5ZT#`-q z1JkBM`#aEUd0r4z=P?X%wI)SL%W)i)4q2;UP?g}lKqGqm3po+MJP>Eukx1D}^xFmvcFG0{ zu*+|M(!@DT;n}vf7pHD2=adD?DU= z8T|ax6751c6nl;HB{?eT?OKNhlTUiwJijhFP=Wg|O;jL~01=fOjkk)?0RoG;ZoECI z*$@k$d$LQbxVH7dL3N_9q@F@g1(tet! zV8Ek)E!o#w7q}2CoTQL%$Vr^?C-NV89rN4M8-0xn$Zx=gA(w@?6-6)e?}sgJ-xL_r z4kc5J5E4BG*bY_UNCa@~@ZJywdD{ssl#b;FY)LWU;G4g_7RMS@@p8I?aw7g6)Qz7P)uLPX@)SX3d&wf%R8q=vRMIo9}~O<>7r zs26-P*^aVW1AIVdMyW4JKR!F|C$A?@@3gE*xfS`Z8dgL_6U$iO;1~M%EcoJ|j7FZ> z4(_yWlM&@#C$ui2MTRh+13=eE!10rZ>#s$kPy9lnF%Z_$Hd`u-C=vBJzPLzIl zCb)Hc`$3d_ixM_)c~M(ziT@T0z&KIum59(L^mHKC4lwYxA?kBNiKGCE?DCe0bNJ2qd8GOD5u&1s)>R+hic42>vrY~ z6QsPo;W7?OHrM=P@RI%h*j}TxbZTXahmqINO6Y-)_346sjt<|4+T6eq(SauMrxAs% zplj9jJWb*!_C}wqTgQA*)_At~K##a91U$_%NGB}b;LGnz3@{Lc8;sRAsZP7xxRNlv(IG{7U`Jw?~(bD zZQ>^cMeXvdEPWQ6leVbIpp#B{BvQaIOJ-1pb4piZeyCOxrLO&D{Sx){};5sKp zJNOE8J>uf1n_ZkE8eb&bXuHM2k*3cR#{q^0Tr6;VJAMWw-}Hprh(d+lEy{BJOeUki zbGG(GG7+PB%YT|ivjEpP)7SIBAo?a(Nz5mXo_NqcnkEd(qLSy{lCd8amig~Gj#;aT zCWuzMgog?M0jzzHL4S&R8NVjOmS#u5<7--QjLhl|gqSkc`>S;5u#sKStB^kq11J*p zzFY-_1tG&fThznJMAf2hPQWg@69^IInH?}bumn3|t9PLU2iXHO;g>J{dIF0ox4M=w z+>0O!dclm0W$`2}yufwJyc_OY-ylO2I8XAe(5}di05slu;m9*&A?6}fCA#&{GKHT) z2>_HPWSabME^=G43DiabEY+Ycf5x*@x1)AhA}2OO2GYRk&9&2o#AA>)xZ97cwB$S= zwKTH{ImYBEBC!Fif!KheJxVK7i1nwZLO&#ry9pMgzk39yRCf<=W!3t8aG3MSDigGD z&dhll0Ce_l9l=6S8K5>3sCz7{9k?ey+_J@|eSuM_o11$qhm}!@GVAX|;WWL!b+T;< zxbv(x0qr1L{)NeLI`1DRUWH5Nfcej+Zb{6E4qLSjnE}89@$ZMRk;k-6rufrzs=T0v zveEo!*+600^{(7(OLou1tq_svIWC{M`1#Xs5*D}2LU+20GxL9G`et8PLQEFQvdqS& zYXCWQtsocSTj3B7Od<*w_5?sp@+X|k?6P^?>|X>?X21YbrnqgySu>!l%AD^AKL*~$ z0=j7ZC&!%s^S)>u5)`9i6~k~J*w>gJ)t9$HluMSL;R>dVx%p*}crk9Rd+?M@Q(`0R zYG21ovxr!S&X%cp0E5;CHlX%bHxQTMDYGv?4eg z@TtMr74xws&&-2)zf4i#jMkrNpBMZ0w=V2;;x9GyIH<78&QOFgNf zg2Ao2JDy~Ae}5oT(!Ia|j02c4EH_(oZ8YubxN;?+v9R7$iTZV%z#wRMab&m7{vjHjk|v)yR25Az@_zr07UaUj)3Mg4w!fIb?~cbd*4`7WF8RO2<82F z2C_a&)9SC>aa6!t2=&NU2y9XyFH}5Eci4)x+LroYWmCM{uo#>fvX-4GAJT5Bgen8F zU@GS4$&@`ATT?y-2fJ3tC1%rRBSHQOGanjM$HjKl{9jXJE7vE&t=6-_|Av55#T#^= z5tauea_hj2BfIcoSP(Kz$20O}d9@1Y<=V=|bAH>NHnV_VKHC%AUXGjZE-rF=JpAh!EfRe*7(uVQF&p* z$XNb7`%9+SBMs)7k(RxNoo4UHESEbrG#spjb>qj+$d0+Z9q$sIADs7ZS%R4T1}@^W z+4$jEkIPB$9UNZ2mvwrVBjx@_9o3l69uH6-6(0VhPhdUS!e0CpxTqPh z*dSd9>UI|c!)@xb#RVe+4-e+LyNS*tiSs)b8v^Pc-951wcAuLTgwpYyXCF+kK~BX{c{9o_`pmL057%@R_Ue$bq8>iyo{nJL$C=* zQ1Fqn`~g7Chd{iu7HpJC$bw4xc@7Hp4L}h=zaE3BU!mewT?29kzx|8eQ$h~R%!9X< zZJ%2d?yuz0H&DD+3A+JB46%?O_+w%|Xfzo4Kf#15Ttbjf=k$X{_c5l-Q_#ew(z2M+p8+(-*KCBzGD= z{_#Q3b#pSG`J9MSph}lPn?<}|?COwZy=xs2a_n)Njg_VK3aC|PRr?mhLHEmj%)SO7 z#f|U3slG3u%Ib-8-GEonyvd_DuIr>W4xKRYdXFJopWfbf_npBQo_Va;3pN1ZX+4a# zk5x?33B#`{y)6}{ZP;1pdetLJ{P!6S! zy_vm$nG-gt?(ExGIVH0v6V_%$pmz2x>lICBlmqC*WuI3cITiXuJu;MZ_l+`>3C;QW z>BFH5a57~r=X6{+()C1{cfmDw=))J^BplW|fqu(rdBM{eVj61DTU+hYKRtCC{=CIE z+_JCkXXThBl!>$A?edy8K#$OM*um9mP_#Gl_`;isa84#Bvpr#R9b+8cnx5$UOT*}w zCmX%(duytqrk=*?Ib^X9z35lSV7Eo4&{XKzIaMgncuv~O9dwQbRdhXIACi<_I^QtQ#S!4>Uzb2c=WY^F>0w+g!fPwuLootBG+KM)Q>42khew zJ{6x-xAk*{#qtH${P~$@RYCI4b}w2B$4~QyEw>-<{cGFaxd<#gIXN$gUWsuM65C|t z!e&e4z7UR8q-HCEp9|#dxU~I{CTh>+XIMQ4-ZiR^vYY@>c>J=vRS7o=rhbhB0a@r^ z-0|}ro-ye%L9^)Z<-a~Q=Hz8rie*T;OLFPv<~(j(czR4iOP;#nVMQKdXvVj)c83#@ z(jUIq!GKsUp?o5@ucLlF|20&hYv?&OQOp$8!hS2>U2{1^(?sRUZ=VHpb#J|qeQUNQ z(8w;&Y?s*>$2iFkJuu>bG9GtC&qCfRGphNzo)bwD5VKsd$@FUE{h@9*X}4GX4j zhN(BIj?>hY8`rM`kd_3{AeX9kbPF1uvi7;9Ytju&tFOkrrP6X&Esb`5WPj+LLe7Py zIU^es#=52_$&_W>QJh`GW z!B7)-HLlm@!+86@HJ0RXVUsm0hQPouHup@`$1Zo~o$WY8p8}7~85}vrzjp_QDCnP) zMGQN0v#nZhJlF7OsWi6{yWzDYqXT_gG7H>Tk!ZU+QUGl|@$N1IF#|HECc|!tW7v4F5mgpWEe(D2br@5A8R|L*ENJWO3s{6HblnYGJ zcCHlFB$?}vSHPAnI``ie_v6t}}FznULtZ@r~f6)y}r_(Cal%A2knP zY>j`zk$)oR3D**A`Ig|whLnAG#}*ngM-jC2cB9UO_^gM6briayZY)(njlY#@bMtb3 zFPNMDQOO}WveJDPoRqx`+1n+;zY9vhZ@12TQsjS@J3q?$F20Y{eD&E>Map{i%FO5M zlsCRqkLoHh(62Aq=q^e6TF(`jD$o!;2rFDZG1_fkhDRSBeq1Ij_y_0d(wFZaY7L_} zw=dcJSXGg+!J5SSt>C>sxhi{Qve^6KiDVN+NJ?bA+q~KP`4YV6{1n{*x9U548bKca zGB)(#G{?PdmnEoz85=o_*5GEWeNkb=h;sjglz}Nw4!<+tgwo)xvL>6r?uwD+L7R-J z=~Ta%!jd^3F?U7va^!Zg$UKG?-V1h?OW9P6IAtO+uC(Bdp!OP*n_i>tYRn+bhq5#i zw?H@4X5Muwg{e_+vUnwQh!cy^`-|?~>My#-K}tul0wFW^nI7(-9#Fw}riZ%N%{wxf zf0-VtIYT{S;=GHL;+6|b3XvFh1d+T$-gIwGGQR9sq3VP5`4+BI5>ml&u@-T;Iyr{c z{z|-r_sIg%%r7?BPUm;C|Ni%tSQeq&n&zu8>yonqKre|Ngg5_|V{cbInn0^SHnaL* zDbGtTlFuJfBCB7@Ex%9E;B#F>lC)c;wd7)05N_C5Ih zu)Qa?Ml76T@}jUEFtdv|I$soK3YcJseoo%9zmQ-`e45$~Ll}p)FKIo(OW)sF*|*bn;he^9m@77U_iF zP8+McsvTP!IntgPduz6KW$*!O3imU5g(mb4*~61l3e8lNoOWt)8Jfa=t{HjekeKROTyK<&9>`xxSkv@rqH zG{Tkwnf|wmdEK+Rce8b;F1YJhxUfU2hGz#{kb9UQ89}#joh_eR?x3Zc;sABr(g9qU!R$mEo;GwbM`Pgt0%k&)zTjf*}SCmIfuA zx&eEq%7^k3?%=+M`Pe8ddqef9XN!`uY6&NlSzZbrTym)TG$vl;3!{|RM2KhH-YbcPZ@uo+ZzDfcw#%O_* zpjlR|#cNPqJJYK5S2vNXL8ElIf0^Xfc+@`P+)Qfb^rTN^jzQOTsyM0=`L5-x$F9e% zXXO!>8V?R&LLK2<7kBe>OjKD>Pr*npi`g|w{~$-ow!MdWRZs=0w@(Y(m03xli<{7a zK*B$kzTC4h1VYZx>&bNo1wCWjd*)OQe;Hj1lHVT4^sjRiav_Bdn)Ox=3r_SAi?E~$%2BJZQ@8Pomx4nqPzWF(S8F_%lR%4)h3v2x&wL8a#+~+m%U~sZ?grSYeQ;ZR?wN;LJs=2z0EKigx;VwA7k73xph{Cd%a_zO2+-xi{{O{(G4F ziUO%Ho|q`e3&X~v4;RJ*$?+&UD3K)8gAV{+%>L;y7_`96^^xZFDpKfrc+;$clr8 zIBd*`q)-7migjTvOdKagTm`uIA!y%K1YJ2)iteK2b52YV{7(9>tZ%+Jb@1L|i+c5C zIE&%^z$7a+1Pogr@lQuz#VX$QyUq8}j4Q+Miu-)03GzA@?U-in_Y4 z(_yoj^aekX>R=lE`39D)2~DG#%yX?RemQ5NXBlN%w#WW7q@le9678gCYR-n8peH@2 zNExuL&I|Clw9^d0@nQ8O3zTWxJ!cM$qJ!yZqv+Y3U zd6l!|S6oHef{Z;eesYSNO@B%d@bM6muD9a3w`ZWpm$<>3RmMAm_mwnB(O$UvyS{PGJ57O0lI8kb{=c*_Y@9tjP?deirwhdd<$Sw(hb_v!cqIeP zVOUm45>!K5(wxAZa`xSu3Z)4`p`T7F@cU3mOen>LE56c%BL!+r=c><8)K?$U4z3V#yG(0p} zLQg%d7vrdeKl=$o!B7%`Fd2o%g5Xigkw_Z#JEytq`ilr%qjb{EJ%AxjM#Et7Z<(uf z;cqpe>1bole=l6s6yebfAPezSDd)aUfRZLc>flRn^a$_Rd+@6;--8AM=RW&p40YGKG=mJcOcp}(y04ko z>jaISu@W}AQHpb*oSGQ{=)qJcPk$A?m3VX=Ll^tY?d6C4SQlb$?8aZuO{E;W`NdRG zqg@qvG!3|@vt+y6qEK&)hTAOm=*Yck)k0@6Psnh{*hrYen3_BUJ2Di##p|r zb|qepKD~L*cLua>W!Lb>#E@^ALmI|nSZz)n%{h^C{givT z_dbsqh7}5x{=G}}5@e}`C&CK@(L5>!TG5~SF6Sx=_Cp5bqYl}=<7y4o6FZz(bXOq% zseJ2tMrLL#Cqc>hnb4k+^f&8Sor90Ls!{FLf4GJ+wkb~ERF`}sVP&RaYLp2_+nJ6! z!S~VxLOu3sP8@y$rg1;9 zp{(}~03QDlLazGZ3VsBSp@dcNpj<)Ut^8$^oslu=6%smpH_{Q7nET{@l|z{l4``eu zu5U{%G{x|VhBVmh^`D;4ORoES_?pz%MmFtYihqB9v`na5_86hq!$9PSM9UPyE54gB z?&I~dM`|+oK8>eCQvA(c!+%MZ)|I%0bi+0!l(QFXn^eAoDp1!SO)Ij4Gk|QhZL5e1 zPmT2*%pRQd)72UMe$Rc#@0OfMhOrD0PnTGMbZtaQj15Wd=JJI#vH`+3;l%zyHIZ$- z>3UwkTJ|JY2EMHC4(+c@dlyxb={KAZl`?}tQS&zdoldQ-QS!Z^PaCr$xC-rZjLU=} zpe$m0F65pSKc*p{Mk$ajFw;i2{N%?Ci2aAR!;&7G`a>fQo2g5u)lH?dBDkNYS?h5Y z^1u_Cdx^p}kWJ;HhY9m^#1sq{N$Qwg|7l-0Fx4viX(Nc$7?XG^lsTi-mH1n;0}3!% zzK7X~$A#mWa!uci0XRfCss&d&`txI4m#DqaNn;@M7WkgI3DkF0;e+Z>kh)RMN3z;3 z670V$Rh|;)N?Ik@QOCsG8E^-5*y-3yKHnHzWP7ZcFlLaMZr z{r+Gq&f_=qLRACWn3Q9WJqrcu82Et7=aklJCLlr_e|Z_wNhyEZywU}w&W=5A12YDd zM^fH5!I8e)(-4Q;?8xLg7wHzFyg&eo-#JlWWbP_2x)EbHQsrt%goqPr@MfE6=(~rB zsTN-pIClw08$t>qEjSCG*vrcqFD3L!wEP`gz7{7#jT&2fnKbL)IrIO#N%QPpzz`Nb zOiXMAQ>oc-k64<%tK4he+`Ww}O6fUU)u6Jo>Ryac9n53vb$}p!S5zt6X7$}an5a9J zVJv$p`peY+p2Osrhy6)s)hTG;UeD9UsF}Og?|?>%Jz&gRYJN5xNV=S`jrqg8%hB-~ z9*I=0oCx$n4C($3XqA;upj!v&D*Ke%D*rTf0@`k|Kk=sQZ2^)zZyJ8^w>XMz@a!4J z*$;;k!*a);*Eh2pBw40k=8clFH11>DFj*Am&iWDU%>Llv5eDD*d;JzFRvBcJh@n*b zF-p`MrQUa5P8`^j)=&`Ov)46jd}kjCoqSD>ePgl*T;OF>J@2J1;f< z&BNLsdsMSpi-!GKoU!G?3cN6>V3n#(h~!db*rbLkq=GA#ioujR2DUOF+*a7uyZ)aK zL?x3=)hH7Zz+v-^;#||RGlm?wj^6-A1vev7;m=p4sfvW1mXyM;u^-kL*^nbkPEzbC zHxtp^;pjAr*g1kI8a@*6PBl>#nMc89`_IwMBVj}#w!XJ^!XOY`F@ZRK%f^C!@_*IE2BfrE9>PxF&uxK#m-2RoF-OG|=^ zj$i_5&!vd~6O)7+VkCAxYeM_X;_VD|F`%ANYid1hk5;|HV@>cUe!b@_=>%nQx7YLs z293z)EzKLDRGhS?#6$h*EZk9$O_EY-DUOoEsgPIp_7p~*n?(Ax4vlEg8xiJ6(E<_d zKK;DhID;2k{{SOK?(?ZmU^Zp_$&O|uO}uB%fUDeb4zSo4jBpv$dGLRC9w!?Yk2A>S zMz@klxj$=s_6w1Y4h-NsdZoybd_7FRL|(QM%Fk}98g3)zg8U{!Bi+2;eD7M!$t;-i zMhkJMk*vZ-oWyNjmpz20Q@C2XyrXP;Sx^{&TaEGq<_Hf!gQt)I242;a`PNWMHmqts zTjBJbRA(9BeJ4jnVc4aVhV@Wzdf)h|U{M1o%aef;wYXA!O);+aCs@iA9muY$g11Dc_#cXZ7!C;`dFb%C*{f`aT-j}i zj3Zh}B%0}dtn~xyq@UANpWx@3)DuwFITz#%k;BjD0KIi^8P1F6V~}i{YOFeT3w6i0 zY;zqX zydLnkC6nBCr5Rf$29U%@2!ZHVk>)jy^aA%V;;!m{gO62MLHy1c&G}blZOASRqpavE zPiEPZJpyiqFz(Ki|Dy$ot8uq zGc*K%VF{X)Y&`-H58gWaOf%*qQeOR1wibIG)nPD9mv^Omq>EzBekHsjxrmxVpWerO zdLk5U7(ouvM-P9wU$fRf7yj5S8bQo{Fqs@>%d`L8P+!YNEhksdhn;q_IXeO5DkB9} zZL(^tcHrq1));<}a)fJ!qE#3_9@45Z#K0cd9c_FMsqyV1zb}w4*p*}t-yMM~TLQt< zIgd7GeI%RX!HXNmN}6mxj!3%(NKl1<>%{!-Nbj$I~B-mmKiKXvHE4WH+@dXsthrWQ2J+Pa4Inp!qrn%yefUH z>Zm8;7e@`G3V0*Y+zf@la~88h6eUT0-$Xxc)#VOWpweUW^%Q9;&)RhZcRwK$iTy~o zD=_Af&yWdG5cu8u7D$4K5cuS$eB*LP8FZ=Dsebzv357I%+CQ zd&RviQ&AsKH=ml7vRkw zExS$1In&^8@qEsaKCI9M*~DBaHPuIw#rh|1@rj97U0}E0FRm;LgE#&hsZ5-Yn)XC! zK&GJzURDl{ijxVn| zl8bq)%a0#TcFt%@Pw`6{|OoXqF+ILBPp+SfO|K&II?$Nm2_1E@6)Wju86w#4xlfqT9k5hsyYL)SE z1`Jn*mQ-{!_1SNqCoIS;??D*Lfkg^u`Hqd;BMu_%YPcQt115%m2{5wPRI+6CZ=dxa zX%pe0fy*(p*;(U*Gb~(5SQTf0JB)VKAeSr0r&NU? zt#DphAfrnE4iU1^PBDD&n`gqKymvO~aHqQ)I8~8edIZ z6}|T#KF3HLzL}x}HnkRK@pn)Yb|3D>SfYrraPQ$LfjybexqZKsbU>;0SG4uUz|QIJv`6^Pin zyFGU=qh$IhQ~#L*DF85)R%Y^L;vOB^#qEXuI5iH}P}y(PfmE%3A^jo`9Fw~HDs$so zPXo`{3!Z3>9S^@Oj(I()H!PT1?kLA!Fo;NQw7DgC2VFu7`{UoO1eM7+t~aEe_tY#3 zHI+@D)%QQy?QndFnv_2M%oyucVU-X!4Pka}%&K#^Rxg{U_wWzD`aJ`mN>fWF@x6gd z13G6}hRdq?=xg;Sy6Nt;DM_i4t~t1e0rGG4o5$f8*316XB|fZ=7(<^I7;u&9Z*sF3 z$R+NMa%QA3Go(U1+JyV&ukq^86Oj2@PsgbU+np=!)p2gC!XP(w)8%Yg2826X#*=4j zGPJG1H0^sps-nC3)tajhWqA~B6n}oUU1glNwOf=6tFIL;ynk`a-)kh}8{7Q0@#i&b z$HvWi2m$}NwsvNK&ix4a#j6CPUn@^1R(97|H5&0Np5sZ+ysY6H*;;CpNXMeO!TDg4f-+I zFAqsu?N?v__#FP_nfLY+qylY&=ERVCGi-rl(SDUQgH+%5Pzf25y5+D0}*-Ktl? z&*{8>z-AuX@gbLyT8Whn9+Q)7yZ+sK4u1Joxl}foFs#=iPhq8p9bGwki#__{J)>LV zk^42N>3?4Y88>6YijF^FmIy|-nJkMQyGhXJuMimsOjL=-F03|#J+I(kTirPbU$gR~ z*;GXDBcsqNP6t&;XFHzAeY%AIb39m7%GW|;!zR%@gV(;5o%3Y+b~Apv0A@B6}CUQhtG(5^eS=+>cul=_WuN#O(p6qeMCJvXqfU%f2F&7{}Zd3 zu;T5e&?Q+er@FTa-g7No5pX|;ZmZ_MDGC>16Up2;JY6_-eX?_x|B2{fsvSIzeo-SN z&(z_0tnt1yJG| zDl@I)OFm-5IO!NT!+e3+>U!jG?PkU44b!#h>}`Cv*euu@78k#9Z%b4U*jco@C3WVe zNeiUNJy{>GI;R%{2BIA>!Xc`LGB7Z}Gd160Qa}~0RE=sor=j;iLW{Gq?AiruaoroA z7IbBvy!a9#LEt;^V04f)qPMVAH^m&VyVA*E zS*G(cX-IDw!b%Q&Xxf?R`B^zA2NCslgJ)`Hyk_cWk4t3_avWb)VfJ{@vin_HaQ<$l zQHK;|iv!VOpWF7^uw zHcIgu{q?oAw{K@@GZqL&3z>NcUB>nGD?PZ)!mE5@efDD*xhEQ}*~X=zI#JoUYN|u) z<(7CkS4w~K+O4dt({7lDR^NHn-kW{O(p`1{)v{E>g52dep=uWLwVLQapvX5o`U{V2 z-UD17jlymFHQTp0ZaSsUtuPz_zu0s7jqWFnmcx&-sU9@InX%pnw7cwPYcY$)KGcBs zJr9#J4rKCUuKYt24P}1QF3HR}->UBRhW$f{3ELVBN17`9g%LK6^O1|S*LYkE^EZBG zNX39jgtwy7ms1~h7yZJ8&;PpOo}xsITputpvov&^G`FX$i zJG>~;o9{gZVwF|5o`)}v+*gLe{D1eUJT_^~B4(tz2KZk}j+_Dag{=Xw>lZC^YMSba z+z+T6t`Dd@SriD74!zx?(u%eZ`=o@D?y_h`dSX0EWR=&)fCnt|8hlbnPzB9_URMlm z_8)3#?1!Mql{%$g(Xs=gpGHkEzOa$ZITc8$@~H(fB6n(sI<%HKWx)a0j3`5O(EAhnzm<~4NSBU!^zjDEmsE8lz8s70zxhg0 zS|WAplLuL~MeL^L?j4(7=}gh@Oz|nh?jF?n+!~XBNZD@iTpc;|d#S}CY(8dsgYRwA zkEhAn27*!XTDSi?KusbhnNw6G&9UpLk9*Se`?Q$z)d2&pBnz4SR|?f@CI`Ry>AC!2}X6?m@4Iolu*LED#vcC+7|C)!m>!pg}>qVK7awpaZaIoMj6ZoTlHLlY&J1P21}-z zEw;j2nngH1r*BCAz^k{Y!+PHth1&vb{oAM^yFhW#n;RUc7WZ3<9ln2;@)VDhEXAFe zIvn+k#-5R54>zGuylLlnIZg$f>)U)8RgQt-?R0G%1J?l)CvbpQ0aumS1vW}Zl@ib| z)VuQcMa$k}-d0^ICiP6;KLz#DnWPBZmZV(bWcGc1*TU6rXI@@mwP)PZgnHhRzPYAa zDsvmu=5LDI@?Fj3*t3L^(pc18`YY<* z-&cSB4qbhh&qj{#vR=tBv|j>)n_JMmnQ>@{&&4Sj%bpG_QN?%&PGA511oA_9lh-|V zelO~-i>;+eO&Bh#aQU6*I@W*X_)d130Ce@$}*;y1lp1)uW0yq8?|Co^MIrio|Ma657{2pDkH=%Lz zN!Av@$@sf6S)Z!2J3@TO%yH?KlC;U6$8HK_qZ={mozEBF-iUKleK$Pi7*phT;X;M} z5B%K8;#L*RAe5>m0R1p9(KAFQFa}7bpd2rFdM^0XYKQjP*SMSL{sbo=slq0kaiQp3 z7yKs4HR4+K+~moa0$Q%y-@bi2alzLyoHPQEhM>VRw*YG}r$8N@B@SiPe@#-M`{*kJ z55T>vJ@@j(#){o&TFLa*h$!eFmkZhC&{iV7y%M49H)nj8HU6U+}hKxbK~{ zdM_=z@$^s#gLa#BppTFdILEiKkB)tN)ykGe=^m_6-I>CGK3TQGNX;@2rXw1D+Rsa_ znT!4;Bdos+RZgzdQQN{`L+o^G(+afejFo|t*HBp8md#*WJ+++9&ylMc-Tf1+8|mZT zceo@^;ePoZRKSesz6|NYCm0u@V$gpcIW?VL{Umem74yw_1BH4|W39~iZ)5Rfnp$Vi zJQIv#G3$xfkErY=+6p!4iJuKgXyelkB2mbW0F}LK)Zb$riut?j3hA zOgBHan3O8A)%PP$O(3*qp?N{Bf2|p8qD_(wn$J(XQZIT$Cr*{D{Tgk zvBVqlbFl^;5E8r^x4Tx8Q!glL@T+^t*kN0;J@)lXjs5k^$5yR0gQ|zDP6qb-Fy(<{ z+Nh>f&k_|Dr`>^tM#iM7G)M8m`RCGSe@RSUm4Wv?nXVng+}l6;JV!-`zFAX;J42fn zrPX-9@1OoJ*eBEbPSTJ?p}NX-YQspN1hNlna1MO*A)7pDH00x)R;~Lm{9x&~a$oC6 zemg~4m*(>i#vz_iX!jNHm!q!(Don5WA}Qjbbyk^dez4cMy7;{h>MTWD2ya0(nsEB=n-rLMMFYPMB1e z#{FE*nWPK#y}HhiNsO7?{B=hD`YE1=ozHJEtt0J#Ng#;qtt2y(=P?mkZWm5z-y<)9 z#W1;)nMu=%M`6nsWG&P`x^dn3mlm?dJx2~}nuP8wzA%u3h`D{+KjdS5^$g1}ywP1e z1uj)KOw9>KrYzn32=d(T`17_nRkWV=mpiL#Zex9_oVx&rUiwf$oeq5#TJx8R{_^E{ z?N9-;6j$nFgu-~u!q`(?nsSF8l|C*j43S|`Uyk}1hacw1KJX~V?Q*k4Fl3(dWSXWG5TFZS^Gd0Whlm@ZF6SujMmQQj;p zMz=+4m-gNgw!Ht$_sq6`$X73v7PI!ob!R|%v`(#+-m^tv^?}B_38h|B`+P^n1U|0n z8#HS!`p4mqK z<_A!;-uO-RvoestOw5+R@5xs?e92Hw3P9Uy;k2w4QwMV{Sl>rqpNj zCTOvXvgY6E;>4nD)ofAf2zbS29&&<&@FDLOfCubzyxh)F1~{qPAM(8xr>2k@SRpwr zQ%}!AjzYjETRY(^O*Xg9_oAbz>23!Bdqpv$MCr#I?--aN(c8 zH&@O-)?aMYHpGT0%`#ccz2q&4)5aQ{=(j^b614On}2w55BXtMSaRky{XRs$i=+OsL~g~XEH$phP6~uBJbkKrYkg`WtR^{M z=I+h)w|}qd=y-2PE=0i>tb7!fs^P2kO9nUtq|9Tj8hIRvS=L^j?eB;DUz=Q6(VN$E zyV3yd`m2cGi(*nzH%y=D_aRn~^Zrob%z$;S0-#&+Dk>^vpvagZGDxA%{DNU{CD84^ z{JORQK%ipe?vwx&o>xmQ`cyhKG4@@Z>da6AeT^cLK3@-k^WVp47&!C5leuSMr1q_X z=R57b8k%a{Yq0;i9wBb`#9Q_14|dgz`x}S^*EVo_n42su7JY)lc!JdR0pbcBuZ>^z z*GT*Z?oICd8$f}+M!pwjT==i5^DaYbj@_@@ZyKKJCs%MniaD1Vs%VEw0=kAK zk-F7C;NOPZvUE=MeAdL$rPAK|%A>D9#Z7Pv0a5I0FXoWdOF&;{rJq_)9gv)J)Mw-) zJ11v{{_?P}vn1l^<=hi=Ty>tnEp3}*uk1ZkMSkWzY&8?(;wf2o__|@7YoHx13qNm2 zgos6{Pmrk!Z&R7p5SJ6 zJ|{T_@VZj0W5%@B!6e{@@>QukKt4F7@2}aP_D(7Us9wqiurN*(OG=N=+!KHCX zG|!iFOWp>Pq*&vcEeh)@kHVzl8q!n&xGEw9S$@8}#{m@&e%zRG*01IrUuzIXeP|93 z4R}p2ng>Rx-LIqD@l&q&fd{q{!7~uC%`5LcBoqAk;NP?Rt6af|`*r+-o&^4o`;(h* zBr3a#qdZbQutYzuM5h_yoX+(hwYifoJS{J-u3bY9WYhw_!6B>R1n(>i=!dYg<}u8I zIp`qV*7o+N5*g;jc%HzUl+=Oc{`#yt@u#wL;8*mWFJK0N?OQ6(NLXjB4F_|y={f*1(D~tDiEsYtK`58HFX}IBFioUT z!BqqhB(luMuG{yjbPh-}aH01h(WM90+eO54-afX9O(!JXWa#G;wFH8jvW;ChMM{=Ty({KVbL1;W9b}NW^Nc)_aI#C*G;DB0=s!2?;O6r zRA-UIoWc>Mvj_wmNvfnbdcTrZ)p%yZpzyA~rQ;IuC3u!k9x~FlZx_Lq*XJ*+y09xV z;rzDd?J0YbyD1PZV)7N6chRngschHQ^sYSzrfDB z%!1sU;(K%-CH)vNjJBRcROAOB1zBZS8-7Dl%Xm@aCBpNoFD^jE&5xwF&eTpdvvdsJoE#2LUCj zjjPl|9VM{j7m>3Ei)RAK9ZQ&?z=OC&j7L{XTDm-^*_M6V)59*&o!7cF359c^LDYVn zg2rB?A!VLwu|WCkBjOe@MGM=@4v}yWD+;Od0|a_FCJM71 zzBubMs;ebI9;6eVVnt)f*ZOEqhoZTsxlK?gz9R~9xy!UGszCJ#`Hb!;^8({l8FH&lj`lCtm4E)<&ubf$8B7Peo&+=!g@KM=9@`_1xMF(9JcQb`@;t zk~Hp#Ezv75iHhr4;p3)p%|c7Cp3Snu?|KdoOs>Vz3eeLMGP|02$P?9WDHy9PsZ4)T z0XqkX}72`07Mia5T$TKWjoVp&*OF@vtSPGIcVIAnN<$iw%rWOK{oMVDpL*9lw zEQi9-l3A?N5=n|?h}i5M8B>rsBk9z5iqjTzo**w7OS-E}522d1!sjz+1y)&1M3g9W zdwF%=Ab#`q3o=S`KgLJE9N4Q$gvMc&ri=9sO9%A@*n}Xcn9wB`n5eOKZ2k5u%@NHY zdHzI|ex?)|C1OT;p+}TQ1~1c6KCPt<8qfm`P~?joHzO2;QF59%_Z8D+_3XKXcITdnx?6wt z<&}y$HvGWT!mS>@-&uXMH{cNE`Wux45yq*m|Hy7)*D_e>!L{0pT zaa!~4s#yRStR(T$&2#pGCboza($V2qtt@0Y_;PJom4Tw8jVgg~-a{}@q@~G~K&}MQ z#oZx}$53foS4%$?%@&pW5(?FDiy&8uMU{pbRJl9T!CDsd&DS2ScQNYWv{72EB$l?t zSg&PjSx-LK5lojcu}E<}ECgRkgFD#FNtKi_>4hh)YwVgXe@Lix>h?em0mASIr=JdQ z(ReL$rx4W18dN!@2|(jcwRzz&o39Dx28|ARnFpfZl^=<(4qQAvT;{sN+*`~^MQ@0Q z5$OwI+3(G^8KB;rI7|I`po9(H+`PV8(4ZBus6z#=cU~G*?cRInvInk$_6+{C`jtZ0 z4QpdPhUl#pOd@tyq1j}y>*6xjykIpx|_cY(NJ10rx{tWNWopis9jU_H^5P% z4lo4tbY+G%?Tyw-$gXrP0Uskn`Lyssoc8;(=-5OJ#nbFxJRwykU##^3bPjM)Xg{mx zi-+Vz)@%MucK%^5>7-+Zdy2j#$5$mtVubcD=DhXiyZl9)jQ|BC$dhX>eazxrU z9z-`6v~vEj0qQJ6G^*x(5q5X3zlERm5{l~2Hs_^}SGSV2<3Ja$<&69XCGb;j9hG(8 z4h&3!Gkg0CP$@b3U%*cDi=<=S35Mu{F2vJlefo&FNZNS{t2FH>v?kUjnJ4q-bi(h9 z$lX?T;A>`}T+4oKc2HunVk!6YLp$j;P<6#6`x)?Xskm%yv5^j3yeB8MzlAgs^Iw`8 zy8HVlDs?ZPSUNI1b&@C`L#+JIgD{Fc5S}e+q4?P#;k+r2=wtzQs4J0dmXbcW$86%coUf#B<_jgmdLBWG6 z(sf5-XE3`i85*ChwT1&p*j+TLNYKvKDC5TL-Df@yHEa7jHH*XVc5G2{{P3K)wc0~s z;P3E6;_81sX^3?efEL9{WdmsejI9Iw{mo&7vLr+@$3pt$oL~hn17-Q*(9_oS#ZkKl zo&Hj3U4E99nFob}e2isf?Iw9gwLYD`V51P{_yhomi1PE|1L0ks&sI)6&P{&w46M?- zq%Qe*A5WEqsIjatY4O?WTe4tSdcxq#>({z#G$3*AcZ??nB(}+-hJ!xbeoLQ zKhp0lJ1CE*<|NlRx6AgSHGMBZ6QKZsHCaEZ9CLRbK$BvVSI0awG^QX!CmoRy%fgX- z!T4+i$xIEBSW16*gG=7@wFmUp90wWtAy?p3hdMy&?}9O5RY-1!+aoBxQ&c~Dt83G1 z);RU?IQQ*tmjNETYJPos=H}Vo8i~VOq&W=C0rWa|+ z7f%)kfSD=7SD}kvUS9R$W^TI-MbmjRM@L8RC-pOj>?wG|*9Uav@MREm7sJaC903`u zh~|a;B}c)#`$F>IBia9gE_n8(B=go6h>!~SZ12&%4JPE(P||e=(y?I?+IMgTDT`?X zXjfmHy59&_jySR?L5im$A`KwI|IuzXD4(rnr<4x9Tp^U z!Y_d@p~(h%C~j#S~Sd+v5P9 z+^(#8?ELhSX$Fib+`nGTYVx}aZKdY6^H;8bSdv9$n&he1`(`j_QhpIso2y?3Z=Hk% z{8OH^g6w(YFZnRUb6*;ZBp9)1P4I+o48~rLpBtsU+W;u$2($F zsI%Cn4fB+=e*=W2Jxs6NO#L?Z#nD!*Hs=54<6bfIqEoNH_xC5#OK;h^kCdKjE)C9DG zl!oN#OY`_2$G%XYxW-jtbDQx$y0R~ERf9!`K{Hj-lf%S}nZgwpWgVB!SlNWZ6#7oO zZfi)!)_s3(Ik-9)8{sc$I#@IP;^$%V=4k3%5Id#89man;ftIB=ZyEL;q0S0iPc_0h zqb5dN(}4@aTKL5BGim5&&Pj)a5>$XwJ@O6X-$}z8oq-o?{Tzd1GD}n{uLO-aBj0~>Pb;G z9P<1)Fo|$2`a3G+#b`~{J#q})E$C6(?>zL#jCq7cKB&#E*HH#gJwHLIaW0_W!SOgn zf9$vbXp3U*l{rjNi8~W!CPy6&H^vrC8;iubMHSFq$hN+%FjwGC{%-qH#o{o4?Az0sc-VZwX9 z5;L$boLZ}Y^*U~1oS-_UR^(GRN4nz(y>$K2K!*Em{FVBhB5$)WJ;dbZjtl|mKJ9_! zbLlW#hgS{pRx`@r=QF)wOI#{a;)N=vfT4voY=i!I|8xRd*alTP(>39(r(a8rJTO$0 zf?G+&$BRYU3en91Hzzu#=*+J$k`XlW(Ck~2KTXp_@0_q#X~uJAut4=MMQE z!`}5*R7eQRbTqt9cVr&nxc2+JTSw5Eedr%F3wr|pPu*sQ-iL@VVfviSlUhR}e|*%R zYJMVY2nvW-ekH_~7(2`ekup&&?`L}B&)YqF|MKdwZcVQz$%V%^=$Q*GE`c_=jVWr; z?$N|0)gb6v+6*Xq1sw*w<+IH{1O71}Q_W#UiPd~`iOo5M^SvK^a|mVU@1pxCEyc$5 zIYwO^16ou6`sOEy!qGhE)#KXJIMI=v8v0(dnk_n&uoZFJbc6njgN}@`CYk+P)tBBd z3QJP#N;ZKux|5GeLZLBpJKr?kQzS`u7>&e%I~f*LIK6thmmp5gV3O8QjX_{Oqx9WR z$E)v5<}$tvZsLH>&lm*mNj?mw-tyX^^)nkZmHA7)7ZQV+VlIs}ML z>!(AV4JXD-v865!Lf$Bk_zmQW{oFz-#8abQ)#J?8`ay~@HE@{8{Cr;_j1 zzrfyB(DdiNECwZrY+4B03ZgdW|G3U;fLlhhbJ@um801WAUK~^`3tr30TQZ@K=pnU7 z66L#3h7pSjAS-WEWSns3y&SQ2Yr?pV|2=bXxq{mBbG#XYvY%K7CLLbz zW_2dcf#!HA(uh7HtZv?(J4*WyI->5aox;bei)~R+g2$ZL&R4<_T>}=vPj9rT@v($` z(R*k+>QM2)E9L4Vr=3qz?369|6Q}ZkO?xEO3Q>1$Z9*-8L3G$(#ljQ)-C|KCykH|R zt;b}o+S>RzXr@r)spZyzObcjbdym5TowqO5-)Ys4K1iiW*kX@A4F}ML;c7Q-flwt^{8nS+%*@Oo-iy91dx0MyM=@agdv2!! z7!bJ-%~+UaU>C@5RA{99^PPL;mUJa7riEbQq;)hace>(V1Wu0(C>Ao6qq-c?Z24-w zF!Yx!xbm?2Z9{INs!eImYtu~cFnSxrfG|sfcDpQ(HRl+9TH!%W1XLMTxF3juosBH| zAmf`W(sxkx@9Ws2a2&* zfCGDjH1!VVL?(RYJoC&*{`JvQC?V@B*szkc;NajgKz_4c#L9CsP!b+VN>;X~SO+d()-lX)y-H2=w z)OT-Qcn=F&?r=z)G$s_3oKixrso@3^>Q)eQR3d*t!s%!BK%add|MOWpnB-pL}4k>uz)F^g+W_t2hm1=p9ugaH>(@?>_!HSOfw-%O5`iICG zuA!E_t7;7U%5pkf%+jg(OiF`S)MRPNN4zgp#uPnm#tkRFRG!r``D3K2hZ2CX`l(x{ zFec*N>OoANgjzCddKA(RZ$Z{f3LAHy>qQmbce5uGZX=5sIP+c11-fMI69{ow&9(vnQz~boXxYZ`n>>7pLIw9AdAO{*}ks@k}J+whnUlK$bHjkD~dH zBpk!*A&TyFuD9oYkg71w3A#tO3X3E)-pv9hVn za)r^!n7%#OTNRO0k-VMC+1)XiHu-^$f##5uDhz&bBoy8PWuK&UYru`y!m2=BlY)S@ zx34jW9>Q(*E{xr}Z!Lc`-19L- z$CJ|S&VB&|(EhoJPi*vKtLK)FD?=pZXE$y>U9adAr_w=|aL- znzx;k)0_velecB~POc(?adpXBQ3KptWO#t(YsY&p*mZQr1|*#Vhu!4%PNlBwAiB5i zZ)clAYE?EqT9;xgzznIcw2ST+jU<{`d6HtdnQ*zwWO-$~^jMyZjw5wYS_=eD-0zt( z8o)HCYy2L%->&)2fs=zHiJGWrfXl{U1Y0Wi8hvg-3w6`6iR&8DBd`^t4^v%6rUDS% z&<*c!xh?zdP!IW z>9tY0{Ey&5TK>6^ZzB~j50!hr7;ays4RG0wljb-i*`Mfxq6viiS`w9F84P~0| z;HME#BHP1msh0zm-=8Stofe?~FA(be^Tmke^|=8(g0d0s<=Dn}qh+8_+>nD{o9_Z6 zN=-@WMNhr2fvz%1JY0WSjGg$2tF&%qi^-x|1A<$7fO#z8Rbq$*fb6~c`a8WDkfa?2 zrR34UGuLhhLowT4iyJvQRS>Fij1`TPlhm@uRJ&}Da~LuHN!C7zgF%e0IKaVb<`;VaZmFgzbb71~sm zyYtzFJ^BvvpgrZwrj8wgmU+J`^QdmF#NjRB3%S3|j1ZGJXWnQjA4E{WhfHN6yo0d| zGVw6bWNpgR&FJ@-Jf$_y)ORe)}sKkq9LOPN3K-iNHrDj|f?V6xBI{5dJ>YE{C8&h)+F^tEZphOjUZjaRm z&}fTKWIc4hJ;4bApkNmOF|c&XfqC5~_wX=JfF4vahjDy$(8PT|>fd+2BxQkPQ`NO1 z@ATx!+1;_}StPG6^`NZo#SZO-`Rwtl3#AvJ_{Rnu7=8B6HeUT_2#*`)u+$URa0_@p(ScAl2K-wwkZh)(wyw5q*`58yFoYX(jl z}W$AB0&l%wfqM zSs|m7xl+)8{figMQHMN1W)r)@414!k>#BJ}iTDc`@QD z?s>-%S;6Wx@4NQSlj5^+lXmOu^BWw|$l@2U`LIiymz?xb6H(g)B3naxej%$2N_=w7 z2dq)Ds0~@WvX85pbnQpe@gHUlO;!d*O9>?y?4my(x%HJUJ2%$@&ZZoHO2d(4N4hC` ziOr{U{`^;+7tRyAGzazGgX5-i|3hY_3;!HMMb8pU)T~g%cvz`-dh^ni+)?6DAjr*9 zil3*p;#(*jc$t0HuuY@UXROXra0?R2=?#`Ix9BA6NrBhp0@ah}0h|$(+MqSW?oz0isp=cxjv_JurY9p0Gcd;TWF(WgG@gND?83){?OzQ`v6B#< zZjj{)j@F*(sy`0P-j7<&mNdJkOgOX)E~?4rP=~@w6g`*6(VA+sk_6H;X1quxpsd2C zBCD=z;7i1g&%mHHV^#zN+gZDFQ8--SCRVstggVg`R+guB58~HQ zUzV?rq*S}Zr=KZBBKsN@2tZ+$bP2Arf}k|GD;N4)%U#mo9-=@&6f#F)7Yx_XP_1*t zO-4_0asE1XvGnWnyT{ZXaICO!>@eg(U0jm-b6C$LUicTHbViHqo$xX?SElFQu}6QzpRlnRW@leGX6!dm z+?IOqsr*ceTM0{qxUeex(2R$<#R-%ZqHMkX>cQo2=^Aj= zY(^SKqTrfXCQIf?9F+e&sOVsLB=k|`<>FNQu!AkJ9@datLrR9=fCtw~&?U%~$WlFy z(>{+fXR!kj5b9D}5Lg z{MbnB1w5%lr-Dz6su1pEiC#qGFdN)k%fzFQ+Kibtyk0yySR!N6+Gi7{3>#G72o1AX zuD~*yloDk&^)FYRa*#wt!qn_cw`h?9g3CH~WS}N~255hoj-sS?mE}6V3{|%!ek*dT zRiFovD5L%kxn!hqoSxt>f`jyh=g(Kj%5X+qT5kQ^433~1~;frXCYrudbJ3HG2;-EdG>L!WrPC@Ayy!Pj3B@dOS zkwEyf?J0r3p5Sr{j+VnN5F_eVRmQIs(P;nQ0`8@h4+MbEJniI{N1*cu4F%UgciQEJ z`3jMnrdl@N7lELkQ!oCE6dXtZuzuBT*4?Zuto4Z#xNtqee>BD8R|idsFzp|XtM%?e zbYRw;(!eqRDz`xMsqrI78R)~k%CViI{q^!Z!Sgn#*R0dIY3elRoHUd2HT#{GOd&v3pb8oRVvaAB9qmAxA8155Bes$&bUGqY9 zdrlWN(c{-E;Xgiksd2vS*0g4@CgNMXz&`X}Er4*pJq(DHV*Ozw?LnqdA;I>|uEqM& z#Mxijzt=xlFpygy@wpdih*2#dQ0Dlpb$x+Bf^4B!JW>Rw6#=Q_OyaE7z{G$b-F()U zE{kD1{v#V@JYgbm)JON=-`!cQTL3uIv$|7KR5T-0OOH_K9t?zOsgwa)s~ds;OuBk# zmEBksn*t{a!`+xFJsjRmzYJeRdINIlZo$^)upPho362?f5@`xlOnvG=Zs3!cr}?l4 z<|pa{%Il}I%s~ChW)8r14-CwC8m9A#{r`(|^k4<3TkwHJZRF1A#7*cyN7TU<$p04< z+W5?Q#9X9!H+42aAVQ|H`OgC~W9%C4r~VU_bXiqlRXYBlJN4T$x8j2OZz=WT?C0Sy zE(nFrqq_h}StWZ=cDzn#A4fcO>U<$=#5`ZFJq!j>bW7~RqQr*e`}GIxyDs;EvqUdI zI6Xc2@^J<7EEr#c2?*I=R5-kNa$%ymN#$!Sn>nDqu3uk$BVDd0d4773qaFY$(dpcZ z-g)X?seh#-w^JIwJoj-3fP~vyz^2$NhknnobG3XW-yu&=e!-yMpFK52oin);a*JSB&)7N{Gv zmlwzC*CPG|84FmBf(pnuee|hI0%;gV>fYKsK5e@{W4MfrjxyAX3ZAvGVTXqBxN*^^ zf&kvV>Pyj+7B>~T!+;u^Ns(?Q8KBiz_4Di<^@05Em(2_o{2}Rm&;c69(uWZ6Va+L! z8Uv(50k~!BbDBWs9*x-ph#GXzt}+4GFu^U%E@GU^$&c}CaM1aS*aWW!;0^Zv4e+V zeeJ-tM6Em(`u#gPF|k8z8xTLg=9cMv?f5}20@3eppR9iGA~9_Ozq}x$J%aBeMe^`@ zNZwekNp!L7pvqVxg&V66Y%bG&k<=%B#y!2P=xV)ywW)MEv)>~{(a+^i;7pp9sB*9= zq}3W;%k0_zm-b$&IHTt3^7Ji2dj}XklSaT-RjNjl;pcDfmVeZ>Gf6nDw$6h<+{fqM zjp%jZ`i%u~bDnoI>tmG?2ci9y%5*ryK=bU!ZW=8)At<5Ymi8DfRsW$-ttXfmz3C$Z z@F5fu_N4VVDwZva5u7-v=TpHD><5S`ss*IYmdVcnX#Z}wJX}g)hE_*L7;Jbp?DxrCT#0cjqkYX{NV6Vt@AzD`r zPTySQuGg7`MNzG98%}fO(LawHe?9XHi8?BlM$nXSkl23^ zzq@{{NPm7lrk&`361dgtIi(ZB5^W$y3=3LgxKsVJjJ`N+t zE=opXoAnn!g|mlm-YQ=W0|y`9C=e!X%Xm&CYd=T2;Zb*W28f?7@+-cH2W0i~9*w6_D6?`mwsDJ60li5e_`AERUlws=1}kM6xhG;#w4IS-jXcZQCVG$drAGo;(HC zo&i#-Azdn{B3VHmojf~J10Hw4KDAqpK)_tw{WvvMVEfnufCDc z^HtXWo_U$|vQdKHn#3Gib;idzP%SIa@m(QxT_*VsjDbBReT9d2k8p}S_UdCbmdY9m zXLHF*#vrsgzq4`10O?C&i;f*ZtxCi`-9nAlJi{2j&Tt-sA%+YdRSnt0XjI}P9nE|5 zLmqXsTG_vjj4pu_(||m%sJ|1|m}b z(fDvtK~0cWCr`48d4f5@uWC{KN37&-oHo$`QIyO?)Xm>s)cY66#5h8u@aUZwfIhTH zSMon~N?{|xK66XYP4msUwYV6{ZY#L8x;SkZfOu7&_l2kizqvtgiK@m%!(4dTojW(j zLQ^<}VZ`#b>=Y?eb1Nce&J*-TTvYh;8EKqxN{5tu9cKhk59N-auw%M7;0z*K2HF1C zY1*+aGr=Dz;VBMpAS9~1inj2r^0ae{;Yu20-?f>?b0i`H8MTi=MM81wdiNekd~!G&x_ETOO?2tN+hZ*X4|X&GGsOGtdrs+ECHIOJ z*+pEm={!?=(e}PCW&6{ruEwc!HFI!VI2IAHA7SujS7G0rKLrFzeNX8 za?94YlwL#vpVSBv(?)xUW8tKd?bvFs;REnm?vleQ(pYtIO!(uBD<`UTvFSa0ys zD4Wg&GmnXvS04^nor|#v+^`0V(y$KQX%l&dAGSnFb62?#X#D@u%$%A>G&2j1v(wYY!q&-VmIvp-@hj`K73R_ij1!bx zlx#0m&NZ~DyMNz!wX9Jw@M3q1(Scd!r#4;5kX7?~0ZFy+TGM0r1sN0R;7kN`}yZxh+A=>;$ zC*#hFi0ywoV$b;Ya*Qj-@m?)bST7^Pb&60UX6f2Fvh>smBzvgvY%SB+(^*jIAv)(C zo&a|VmKrZ+^6V*bdig@Q00W-%=0YmyUCJqJ7OkKo&H^f8clum~1`T601$Gw%`)3}I zut)vIU)EAEUx?NJcKW2Fqa=s*zsi?a|5Lt{Q5~^8=F@tZ^45h1-61}tYfs+($16<1 zxtIf=gUrsgHhK^D==^n{BKr+x8Du6I}8n@O9Ljv}r{0K}OXtDty^z3Cd& zCyb@iOzUlYOOvHHH(yb+bZwJd0y#<8acHH8CgsO(>&{?)N7zqB6=hFmpN&UtKm&KZ z9n;nT<0MuSBBx#O0}vFI{}2qdEjd}&_ux!4x$3|vwQ%T!PaVj^K|7)8j$}tX&W)qV ztFAf-#V0AkY?+&lLG@Q|aQvize;0lr{$6D{1;XL}UwtR#m61{e&7y_{Gp}u5`PB>& zh2R^YlVqz#54>Qirla(3SgJW_Wg0{A?V=*2)Z@pY9pl9Qn`*ESTZa0^9E=R0!Z3XK z`WcvfnGb+T2JD6Mhy&B?)xNhzgA%LGUj1b2FAYP*XoK0sdi_zzQmXV3bIrT|r&no0 zvid}qRxo91%u3Zz73we0cGq_jCa*0FL00I8j_L2;xr@I-$;BGfupI3ueYV-LEBF}{ zAYr2_^^gVWc9*=B4`P_LKtTN7YOKktu}kLR78GCqg`?>$6&V|5tktgg(DtE71Iy&u z3ae7jRzQTJU}R>OeX`^XGF?amr(O#&YVhqz_5WhPQ<8_bN?&)R_MlV<)68PY*4NU$ z@P6}5QXr6we;-pCCrk;4WH_QEreL%uIV|q}WpSR^o4S^yOI=c#o?X|Tt&kdEWJ4j7 zCv%kMDKZf5CQ7l*%<79iQo`@w^K(J-Qii-oALBR{^w+d07*0z4e#9RQ0cbyf&|f8E zHyVm674{Utk@fix!hd`tueD}`#9ple1OmnASSl|{S;rk6uIfo zDX{asY>*Gynz<>o^&iB^dM2B%Yh)oFQueD6j*eLehQEfWXBM_q*DFzzlkd z6-(-#+@xJF=+ilBKBQT_eT9%&PG4pOD`ird<8dkxZ^Nu|XZU1-LEgBXa{X?P3%@nB zLwGIUJL|Tf2(|5oHzaI+OUZDnA1eY(>4DUvUj=^+R%0Sgyx|`ZQGhty=-cgpn1CW% z2%f-DWYx{CD;QX!k}@N8*{A)cBZbZm{y?k`i2{Z#XNbLfSEiZ;0*7SV_W^MkVib0_ zYQ}BAEkh3@rbNZxs#*6{0ZZL*)>FQ^c2(m3(Gdhzc9x0KIe*+VL(Z2e19|jQtxt*F z<2Fb>=ocz!*IVFo+ro{s5_i9y%WwWT69^O`(oPr=KD}mB9iSN#sqcwhUOv??PAA-I zpp`+b8y#0mn7^yNpFo|U*3dUWZO~!WHciP z-yM^$<{177KDkr+!m8?z-3wXp44!?6aDb!3=IX*JBCES8N1BKd`2yaxC~)-JqF(-? zfKMIB02U5Ia!jzd8a%h(KD2tVP`%m`C1&ukJ-qX-JSmTG5I#w~zs7CG{KLr1PylQI zC+H8b45+eD1MKe_?C7ZgN5(ZJwx-pYwhYKO>Pi;5=IJQ+p0Tl1r`VhZ|FYprVM?bT zzs7~ksC+KRR~N4bmmI?|KZ45@yrZ^9KOW2=5`uRJYh7al%~J~d4T1q$lk$JVfhkA3l&pGRL+(S(Vc-z)EsKQLkvE=|TWc=tWsfmLrttRgXT&|zx6 zv6u(_1sWm(OIp7~-6r;H8jg$!_l)`OWNeuAvSW4Lyo}}gxSfu;9t}EcO+1%r4ssb# zJiG8ZiA%;|SNBBXKf7Gt#l{dCmM=4H)w$VQ_81DN%q9 zA+$Vr;AW4Pr^`IK`&vh#HlL#es6-@p>pUynkb;t`e}h2>yMx(@!iP5-RB{FB=qjV3 z8}&B%?5T|(wFGnBkT$lxESw_T(Q~H0J3DLGuyx!bo-FWu6B{X2ISbo(vLzU@o(B-q zdgkw3;_yVSr@b3+2KjNHsIIC<`Dovz@r#PuSQ;^*`gA+GZsmt9NdStcJ=FD#6>Zbx z$x}^}mH+i7DbpM@HGTY)W?$oEk1($Ae0W1JDFr!dOrP`#Y-QUQzE-EOSPJzt$uk=L z2k0DdkZhV~fc-@KkMOHsjyE4$VHbIR_M+kwzYnmX>ml-lM}eAg(|$qrtAka7x;4W^ z$wD2-*Qy>_)&5Xd_gf7Xf_)cbPi=#LD+u~^=6m)PItcrJL$4y2mu7{rG5k6*x%M+f zHZLEi{0f?}CshHo$dk!yAnGWw|GFwuHX@6C&?u{QYLf56w@JQB?+jT>P$*;M*d>rwyho4#wp6`ck<;IX)40$E=+ zSwA_E(qDwZ@u5H%bxx!4`+U6)`H|ZFRtl{_zZC<=!ZlVjP5wP_Jp7Vl$!{`nfpWuN z9Ie@O%4hFKSBl}9=pc(-woW0&zmk^)`nZ*8D!x@4aA-OA^_v2ezy+#I+AGvb%M$D0 z@-)0axpB-tGDyvzt$zs0hC-9G5NZD9+b|13rj+8u-s$wE964amNaaDG>6Ml5Caa zDJ$0v}*X8O^{ypyrGiE+B4rPT*DDktEb5y#@t*)GFcj`2tPz4s!9`us{hARW<8>_J=l4Y8db{Sc8p@MS6Lggm1X}{+}QLtR! zKEf%|xV^ctfWH{2jzZTfU%S;_0rz)y2QqWcK>nQjl}k!FV6LQb7@y|ZvHuB{3yY2( zzMsZo^y3lu^*uP(AiZV^A|@xkHLAGMt#&ls*N& zh6w79CFxR7-%E~+7JvhXlz!q(dNimJO#F=ak0^HM1%a{eWpoe%ZM5o6ij^Wlh@E_2 z$v#-HYOSQ4zHzFYgH6E%9KcB}Bic|0X*P7EfQdD0xh3&cI1W>8EW>4rzCIvvzVu=@ zSj|}D?oILS#kr9@reZVrxO`%Wd4BC1n?|9PY^9+N_B@6^2CoKZRLyCg4ab}WY;!XE zfZePB14lX6oPo~sOu2@$8FK7VHwMlweK{lGG%mQA2%K>!<*AWUzWM4uv(gCKBf&BB zTS+j`8LcCth7uVYz4D_BPztD29?w7DRTyIQt{ua%ussN+s`1w2K0BT+<))9rJNjm8 zf*!<`+2R>^10}v2pqgRJ8Os|_hfdU8F@Y4cgdd26ngg_3d5!sje3D^KRj^U=9}6ni z5(pmWj};&nLXKRB=GA{@l+en4buk?8uDBS3vO=eh&mkHovV5Bp$h068FI<=rX3!53 z8*m$+PFU#Y*1`C0Eeo^xEvN8oPoZ<}x}Li}S893_4%Ut3X(!c;A2NuCG$?tL!P2{9 z(I0ZfEc_A81DqZzHvb%LyC8Cf26r?j?XDad(+JvAfeuDmE(Ulx4-z;WJ zzY7Or-lj;J2j9&)K&jHbuo`JLY+lL=DL|$cT~mpUxN6m3r2H0Tmw#@iXP+)45k?6k zLqxL1&u zRXg?zR-z)3NWiPtxDDNfv4J`NwaC~ZlAQH{2}^L0Plp97W8xmRYV$P1IWJ}RfY)KP zGs66de8&5Qo&D7>1J9V?d-3~zLjQpVJ@faI@r~>x#}fuFZgm8$QnBWwN0f3C*iV|S zk64u-xDJBIJ%*B9wml;=B^eBE8CA|PK52|9f{5LrEh?2OM-tPryPJHjs%g|G@xfyw z;IY7<@85#f`~McSx`;WT>%n|}`}f4VIhcG>d-qGR`G@apFRUIfXOTW(`0=()jU`-z zNr?7SM7!YQn&^`eQQg>@05*LF=2?3>ULoBl1NxcT5`sazt13i#Gn&~KP^Z1$)C9FS z5%Xgfu-fPK7z$3Jql2X*;^k`BKdjt~%KPx@{fGA-UhaSQkZzgU>r?k1!Yu^C>dR-Z!|O15^$_EMK}Tj6{DUw>9RYm*m?*3CBtuijk7QhGfRd#bB3s>wpRZ%Y(uRP2d5L3 z)?@GmAH~JQ7^R{foi?-;?b0~HK4a*6uxOs%j<+4>w+zq^lXdy-_;PS8cK0>%k{)m4I#$z=y_nP8Z ze&tbOq{W?uAVu;RWJL@LH20`rK=jICVBvyKUq9hBw1Ro*0P zbizFJ-D)elt>f0!ntv9!eYby$y+50L%}L_EAWijzyJ1uHC9m`7Ow3ARd6~^%VSD}C zfv2iM3UPmbzWNH1O*wf5uywLBHG}8T?xcb0w0;3Un5iH_>@=jq&4C20MX>@tz{N(@ z4rc7--FRQ6MXXKCZJNF#_@Qp7+*A?=s!#)=EYC#!d;O@ygkzZKp3+Y3CVWMljMsXS z#u32n|7V9lT2&AOX5Pyf@ULvzpL7#;<=esiNsGUb8WLA3of_RATZ^kEq)o!0@=YWz zOpT>8oq0AdxO9Or@FR@a|1r_&<%^%)zScXF&-@%SD~}4~+p%a=f=f#C;TMF4)56VW z<1a#~j0DMK1BCZ~ja`lkw#NtBiGOvh+^sn=8i2Q1(jm@42}rGbQgHwl!zcnfO0BXH zFVd*@$D}nZRmtnk4)OXUjH;|1Zj-cpxj553>VSikI*F}y3$~uF`5o#Uw^r6Q7k+tp z`q40c{ebqZaU)Wv)--hKodU3tBi)&=>Non*E}#<+{JR!dHgSj`O$imqyQ6;=8uN?3 z{JW@@S2?2FU>g+E;%zpzT1_i4k!WE}>#E!T&CW9%Zy(|Rm`O}K#YK7t*IP!xcQ@zs z-wnQtjKG~A#co=;aZu~vb*pmANXlID1Yx_>n# z@UOS^jP{X2M7EsBQ&qn6jn`B*P3GOHlIQVL%c8=P<6d^9Su@SfbTJBSs!`WF-e~Q0 zSP3nG2A?_mcr@s%dS{Ssyv=&}@ywy&?QBLgnbVNYii3>qlxPxFrdR)>=|b0A`pS)> ztz3{cCiWClo%p5v_2E$w52+LnnODMsD`)ZwP$LByM@znbCakv8QUjd_BJaqLSlr43 z^-CRMpP&7Z`1Bs74%U(e!_a6647m0iCpWt7wY>?LE?vpUztxOmiFx+y_IRzgFJ&Z! zV#_uVGMKKUav$JH@}CMNu9D8N*e@?mEN!HM4SrrnR@!6Eh)}nLN>E z;xHN?*r$-nZ(%YG^=jIco8CI23I#xSWQy`eAox0!Dn1)DZ3M5Xi*0zsjeAXQiHT5D z1!6+D$ICz#-Hyh1Q$3`5Zi=120J0FbQH>QhDMG7>yZHLwsD!4z={mQEpwg3pe`^5< zCuE!GZoYoHn{E9A7+~hVOJawp(I}X@N5o&j;B@&6f^r5Ve|Eqjq5)B`?o++slCVd_ z9JaQvc?;gW97|tgjNfQnOK%{cdQo&Eub5rs)rVo+Kw9!;tBii@PpK_bz1}i)+D#5l zDHaMv-^mw+wz}GN9<4sM)PKz!DC7D4w>HNseClQF-Oy1Fqr~%oYD{)AP*ym{6Q4P~ z(pt3)uk0)eS|0BnpIz~oviNfMJem%n3N^(qQr*(RHlrVb)NSw`Qvrs8>G_o6Y`j*B zB<01}eU2Mr34Z0MkAA!Ei_bLs_2%Fa%JS((OCT0E`8Pb|pQP9oL`7zYYbAwqd%LcR zB<1PJp6z4MKR`CY-}m*elA1N7VOu)!6>3SuMyHy1Leak?Ks@-9gt32w^}l7;udbbePNJPCEM%&;!xJodW$j{`MY%4o%* zQh$z4DsLNi_TV@Lyyzz$xp)jjZv+>!<)M>a9^p8%J!hYsLs4&U`5v6^} z9ig{l52T>?^Z6`lQQi_?!si;g4-%HpzdFAtoLzV7Eh;$aEv8aU6`LnFpx7u?V?B>| z+SIgzlTYQ7ZIN*TEtwebj^B`pa+( z$hB3jZJhRo_9Z`koGI1h5^kDwW0j#rb1Pw>fB(~+pJ|v0P|Zm3c5F2yiUw~A5C z;Uo0{d<^B*XV=K6sHg~~K5%%5N#%`=T}@cUmuiW#vj5Ca8W6f@!o-XA@wU%Q%rHSZ2ze2e6eN)y6yTA2{xe9?!&rQ-LM zeN-}`yvETIOp^MWUaW^{nEn}6#tjFDOH)%ssXBh9if?662BfOr6!Z{L9G~1paXvRF z!Rw!3+)8+WBQW$NUAJTaX;24MA*-6`rvsj4f=wI>c$ z-^rp@*)OPM^U@&ceS4m>VknCp|E_B#n@x;?99qbIy+HB4X7XT8uBftdxc~F-el2Ro z8(Ct>2ITiy{R!YVR8HuqJm;lBPyl-v{uVx18)A{vw-n;jM!WOxt2e znL>4ih+pomQPe&Ar%g)7GJb=ri zId&=Fx7T}kj<)WJ3O8;jCO>lobBzSz9vXYxyE%>Z>grjEV>p;W^=@Rlac zeV}9bWAmUPdmxU?NhOZA6zTA8Mz(vyf?MqQ7;(_97jPf8d;T1Dh?~I=Ouo6HBn{($ zT?MXvmLO=R2tHeXcGqr?`Z}0~iV#l*KnDb`HcRxz+?|3}HGft9!C{x>(I6tBGN4?7 zu7@0=zkBwN#*mdVw7;GzEaV3Ba)y7*3@N(psn+UBWU!6kYeW;yR!Oe?$tU#*x4XPWx^z6e$lSgX@qhKLY(OgM;fJnYE z{#W<-ytyv8%26;h7VmS=aokQlSMdJ*m;`%eg{>t|&%dt1XM4jcN_`Ny3O2=WlA?dF zg3(;{Xvbz9m|x$AM@9^Vu1d`pXWdWdSOYVQB(y?5c!TocoQ6i=@M0Fo@DhdwB)0)5 zEa_Lz<-apE2aLYA4&zm?JzuMUq=>Tj9xRy^NJ<_nSWsmqd6~p-Izl0@T4B>h<KKOn>DP(4(T4^?++~Hs!|AFaDjFn1w8nAEQQsK+keKU4P)NFo*EUsRBXT z+BZ){5g;J#H>;Q=M0-4&;dy{=a|z|?dFQ-i`~sZ*g05I?&SgeFj;Ihv0Cd30!~ zKb__ET^OCXyt($H%Qo>=@UnD{UKjp&qgd^!K^t)IthpZG10%Fag7yH)O}1D}_(zBN z2G05258$c9awuGUzun}Pdcrlvm7^Od)6O#*Ur-t(B;y35UBS&;GzZW+N{1MQbU4nK z-$xev(S%^mq|C0C0lGd?~y-IyxM1K(U^Ny(uDEG+!qbA(lxW4fA z6v`W^d~0iZb{{4HQsLaWb&PI}6HWyCW%ya1jZ6B|(0&a?g9F-lJ5-_`G28Y>5Yn0Z z6Tb*+sdjw#Dv)`N;red+&zXU|0*X|StvTw=r8{ndhV$I5TX_efT#Y%vDvS4(T7z>W zZBXAVitg-Inw3&)R|`?lBgjk9Z#_T+zebwokyx9a*(N9Dj_>SdvR5nZdFa_&tHAis znTI!{ndOEZa_tyUVx28*bbf;LOw_S~UrCRv_uD{wh0Y-zu&~P#c=X)El{krJ<;T?r zW-KK1>M_Xqhg$H$%>Y=vMs#S~AHw=If27j8FS4P&($#sN_Co&4J9ocegC8Vv3KAN} ztAw4WqrqXM*Emr5dR523G-Hxdbj7P{&B2c6#?_I7;m08cmvFMT1* z%y%CyDm+pa*n!gXh-U;=Z)+ItTIR%L>S48EO%(4RmaponKlM_j>{c0)l24ozdLW9i0U4JvVkT!=GZz2;-SGRuX3WONL_b zNuRyg<0kt=MtX5`LTbB_9O+Hwf?~<|3Qa@}&~a~+LiO10UWk!m5-U4wMmZ(cqxz8W zd}tBsJr&B_4YH9t;AouLel~xwdgpFNMrS9JdzOliO#13fwUA#=-7dbOTT}8Zv@3NU zwTY%Y6Nyq?iFpJ{Rq3L}5eomNzQEtPm=MGynD@a}5wA^)p>5QAqbYf3zBfxiiNdYu z1e2KTqvQ3Q<0rf(WLg~$nwi^K3T*X)`?zIxs=wXOfDzz}Rm@Sc*frd=y2i=hQb+)4 z&^e8lln6trv$L}sCZv364s%VUzf5p2di1+IEGf4Zhgm zxXd}P`u!}VTTlOxZEGxh+CpaMOBTW}coJPFuydW-!oM`k5kMeAyfdqGxQ&FtQnH2` zi@>R4*?h*YQj!_a$YptXK64)w4jso&@xB@p$4-D4bZ+jtl<=C)uUaWDM+7yck?Z<% zxZTRjD3fKk8YzslI2{s!-SanvWx^S|!heA6#?_Hx)W$Xu+o>R{cQ50W>$C%%eC&;H zL;JAL?|wSkpCBg(TrC#5?rc6+%)aHJuZV zBnWN^1d^Tzt-@{E7WZk2$nNQZ0bT4W zd(z4>|A!FPSG1V6%AWIIz=_sXIda`2W;kp#OfcTV?}@_`O@c(};J6Axv3k(Ci&R{KBl$)kP2BshdYbKjIXtvFcv&xIOW@*!~ zB|#msdFG$}a3}RB3bN|kQ@<%#9vkDgK8(auHrHp7Nni9Vj1v6Bjzmtk@`h|Ocy2{#~M{yunJZcM+mLmM@M@hmJU z9bgD8l8)xSA)GT~8k5YxeuftlV*twEo&13w4-)vX?>xje&%$ws56|8xl;Pb>DTncH z;7*S9ILHf1Mc(T(+Y1SKTE?ry=5@7pfLU|B0GtB?eJC7jj8fw%D($RRu^K4}s&UT{NzGPP zX$#+j!n6Ra(?0aTe$);VN@%F_Y8Nn?@go6Bi^K+IT<#{xldQ{HQ;bw(=ugh=izRAO z$E}}v(zS*D{uGiQltnY`@+hvl$<8jXGaXscKX2jSl|FChgDNI8zw=KU0YH2=_buNozw{I zYd2Z(DG#&L<^l~>(N>HnrkxMtSCd-Bc!05tUo{>t0yzlrYcBZ(Ovu8%*uwdr5I^Ln z1&8Ih7>0C99L8KyBRTWiqT`Z-r3TgF-`ahTp+%X6&At5$mBQ#h36Yi%_Q@)Ri)goL z$J-2qF|wi4OWq~nsli+Bmr%x`wI2JFZIFq*OloKK4b`-<1e51F;ZefCy}jE-RaI#RCFj^}-g`}cVy7duB&{2dl)j8xl8tqI_~u{l zjkPJir2T+i44Z(@eKPNarrd(!1_ng)%y@EHG7DV~HD_X(UNH4;IR8uqMc!>4;)}65 zqAls^^q!lm)4(*7HcuMFJp*^}pkIiM#6Uo$bfndqMxf|9BkY#Gb9>#8Jh7PSD_k^* zb?iX|V-Sqz87a)9;&-qaHrCDPAW}*V5?xWLv}kHh#Y^S;RSR<0>P33j?+~kDyfoV4 z8PS^BD+#%4#x|cMFo6?^mn{k-Yh5JGa$7ryiwH|@^FDQC<9E0J9deMccT$b_#*T!U zS>OlYD;i+mjDL;qj4NA`l!{Y7cP>y^eDdwj@5LrI7m*XFzsy&4Q~i?^8!DO`s9{}f znhUyVomjHXw)Opjdh0+(QujwEgeJ(YBWie-+LtEI<>4w9U>L zA~fe~mD$Xoo0-1$($3#p+dJ1oHCwYJ2Q?BpNC>{>KvC&Qa`4-$Uc{Y)!z}`S{8GrU zv~2qO{^Tsp8%w>-^F}9F=;=A4B6g+7&LYkF=Bb^6m^VT0;hr20T3GE#mqp&v4r{|3 zbRyOIybe3)Mj-=h!nHmn39!m!*k@f$;w0aHM$TQwhu!geRMynxdz}xsZBh8PAo{fc z3ak@tKC{)BL_eVj$~yvTW#!t%-TNIpd;T7D<`gHMF9lj95eeN25@8t}1}ug}UW;FF zNE_lnyQfRz8sx*LFSn+%B@nO4o7BS6co+MLKdni1Ylq(fL&wI?vHjJK+k(^fS#o56 zv^A}tXt+TO4N#pEFI#PSJ@%ZU@8cbFGTQTYrNwYGwykre5nLo;H>b&dFdZosY-|{^ zK$M!iG!hI|JVHiO{dUK^Zv*;rouzPT{J2;*)z&g!1} z>~f!>_Gk{F?=Jkj#qXGsh!#jjE-M)cpm5wZw! zX$$5d2z_sK6|d?{E9*D$Mo-N|9GR$${+8skOMC5b-lzijjc&xND*x)DpQptTLYt@A zHN`Vpm+5zVJ1&l{QbxgezDMB4@}=?H zOD_%5aG{@H9gK{#OkHpb3Rcg>sMcr7P^3et{J$#cBMQ!t32X%zLaa}W*n`PQu4jOMn6Jo<^G{tFJHat0U?E0-!1dt zAc}a4RvfGI_LB^nc3#V9l%`EJIUYk5iu)? zf9jVQD1#Rf6^+iNm_mGw?Uw0h#y{iRbfr!!I#*LiQd;;$YmB|UE0jdyC@lma@);{$ z&E)T$E{;w-*3RY-D+F=l>LGJX0Q%)~|;Sx|)OjH@~2dT=FN;IVp3X$}X(9o7&_c zU|2*JWFq~FmQNSc=yk`U+4IIC*DHilUeea5h~nYhd0`zYP1%x7QP=t%b5eEh5 zEcOCvl~b0{US&P_k>TNXS>zm*W99hCzFWIrg@#>Rwe8E}J&JwD(ww*EpB_d=60-V9 zsp8bes+>f6p4q?DoxgIBv}!8Y6?B(y2FGqkoMSr()9Le3@69hv*T&!Gw{>pxE_GcR zEh})0%3>S4c+{t$KqbPArYv4aw3jxiux#^_ee(A^!4^i$4x%VKy5pq zp}d{6= zt#i@5SIU>-{q;{<+=Xt(YG_|OlG$zleV|}yCmMBa<0(VWAQ279NJDmw$uG`=Rz10C z72JJ^9o~b7w{XF-~>$3gUI`UtIxA2Js;XNLj7fO=yXi*zSXQ*B0qbKB7c%V^n zsxgYl!e5ir-@0BM9&oc^buZ7qCpy<%ZEdG}_&>hSy+9wLf*WFyz8yW)2iPHzKg)Z2 zJ{FCiF8AF%Z~5s1qmYd9N?G$Vi#wx0aP9?{)m^3^O~rkhJ>Xe$753RUUg*V*YY*I=~lM4L<46VQx~tnG&wa#>93OK z&m$PRPBoe=%Uyz7@?i0c0>V(w`T5yLU}sAif%f32{tVh{OanCTN{|W3HCzk}nN@DX zHkC+gS^Cx0y0`ZXYAs(d+XUt20aeMD7f!(Q%t@At5iAzI{!`dp1|7`<2c3`phd#%v zeGTOO#~MOj+MA8%sN}whOu&0Xy?r<|P#l0LUHrCr#aeh%`O#*r^8N_{`k&@fMGV65 zZ551Rqc->xUf5Blg0PBSMZ-i{8D*Spz-%YThgxi7aCC1?aDO`X#tW&g+W?V^wju4L z)ia}MeRkra6^@^8SoL!2>e61k{`Pt0_21)_i_rrLIMU`mU6%W|7NC3!cTt#w-{Y}1 zi9-_rl*I0bjf(QxuBasX5$vhP%5a~DSCuF=|K@mF>0*K#SvvHZ>xG631dz0j!n`5Q z=2+w>q+C0F*8g-Wpoarc?AB*qR8q|3^Dv^6I-tq4s*!jePv zMxS2n=EQT9nbD_Cigjw?bZMVYJfSrhfjpxz1W(YzgtN;BE21g#)KzjhZ zfU9ggQ<1T!seD@iOOU?1KA!>cCeX3^l9WGx8SPF3rL4jO)6K*!Z7fkxpO^R=18NBM zGfbjR|I6Gcl_+aaUtO7FsKvKpkKOhJzJl)$&A8jA^@cVeoWL^mx(?PA5WMz1_zygV z(3Ok|xB`oaug=+Oaxs)J4U#6H8%^t~!{;h(_+?K1a*)fm%N>#EXHv#aRl8U^4kM0} zU!DzcqjQ!Y%20l!JLp&as_4p$)Z_z`ABc2R>+}-E)hFBXNG^dEJsRZ9Y6Y%K9p!<@ z)6(~h?*wmPmeifX3Ys5aKN58E$RulwvlEHl6r{o-nL!J)2^%mp`7+_Z|I57-*69w^ zq3OZ{V`+`+!=<)Wfy6nnOY-kHA*knTO|Xx#4ptcXr?ak zNI&USQxuuPuiiwuio-A0h7{2%HON5#VRNUoUGGc1g}#X0LEN>Tbm8cNTlJtuFO)xH zkumcC4OHF9xEj2Ax#YJ7RuT~$8}IwCWc6U4xY!)NMETGop3Z{%0NF!sT{(b3!dXel zUZr8y(eun%6l(wYC{EHKCpxYqOCw-fgjUy@n>(-njLeWwglPrQ=g&p40q*neI)aIq>mv!hx4 z*o>#Gi;)FmN((u#u*!r=nNzGmjQ7HfJr(*LE-afVky|~8ImJCnxK8p~cAjZfX!ft;2ph<9oejGZ*IgdwKb0NcIiTjX*Tkc{KM$6QRNS!(T3gF}aD z*hed1`KM={gbA$(EBEy{^D%Aujx}cpT*6U|rYgMdKKPxzv|o`&((Qn3vKzlXKSs1{ zdmJZG-jV7~P9CNfUp3!+{C9otw$ekAgJM2#txK5FgQsh1gp99^R^=3AJ>!RaToh`Gu1Xr4_qBjXw|n$4;;>C*SpA3 z>ZAS2>SFkW)&!~4p8e+p7Rj4rAusBL?CDUXRp^|j5uKf9ccU4}`6V2+)Hhyy(Mt~& zU_FD;zt^y5-J(%Ds=V|qsQDB!xIMN`f|8wRGB2;=g5v;w^F1P7>;S~pr`=UCN6pk~qO5*`>bN-zS{4|ev|Fd1 z@6T<4JH9PcVG^JA2y`2>$GyDQ1g$(}GhquWQ9jSEF>somqfkZP^?8Q+Y3-3IIOU}( z{jSk^6Kv%deFefn{TVlF$`GWZQkAUCNw{!GSnuybKqqrg{5az<;0Wp=CfC}7y`mr9)&0Qyzv!kQB?x26gtFLjI+LnJZpc+ocT zJ8ISD(NxXvX*B9{77qhjk{ajdcj)N#J^TNFz{|ID)1w}4)*rcipW1%J`Xp0*mgFS{ zO-TUq6rw8Uz&Q{$Ukr{5%MKa^v_}oDX)U_wN$+sEDw0WT+h+BNY8^<=G6)abP06+IIn!`NDyPLhcmi!FfXU zywhxcQb)*fy*^5vq&SA|Z1CGnARB-I&$b_aQq(9Tbv zKDj%=2$=50Ips*>5(?G*qy)P)4FKz|&yl{EcM+9wTW{2{gAY&2;ntZTACwuVj8m#>`4I+UamFKEEb;bUlcm)`+(r*xht@}t{~fM?Y6OAHt)YC3DRyo*|i|A5sCFyfNdBDR5lSUoFgzvY%xFc)J$@YcD6YzUnc zf=&GD`{(18e<>*EsQ+&TCCM*>FRM6u!sjZk%V0Gs&2H#Z5xCSRP>jSet^?N@>c#aR)2aT~p zwvUcXve-ATwOhY-3;tN#Znik3$<2up7NMsYCv1(u&;P2(W6mDJ43Ef3zkU0G-_dD| zW$2@?P1DE{d!rp`+z+|rKk3A5F1JTOxK2vle=i)a=tf%sJM#Yi16BGqqp^y4DouOS zHha|<`TCDw{VjrzLvza^M8EauX#Y>#Z~y_HAOmjEX-GeFHlOI8Q~Nyx;~X*3=B$UJ zxofAChOOw^jo+Yww%PxC{~+@hj}Cn?MwQ+pc@G9B%I;41+emt-wV{{SV2yo7tq z!b})7_vufW-A51OL~BnxtSA24J8jYaTj3@N>Kwg}d9#t%;f*feISivE%34hLiO+BK zQkeXMo5*NI;P1gElgt^7dgYH9NXG3AR|*zq2Sy<-Lt`tB2~jl&kB~ADT%IJbCyx9;Rm9l>JFqsvN}Jlzh1lSXOtHNm9yLTs zR3)!32XlWRxzYBG0nvpMw~3#sTVeQ^+X1s1M2mI4&|XliTB2FJq4K&`6v^crJOjb- zH!UO-`}}uVPETIsA}L=h-}Bti<^ko9ro88I?04*YmVQKBCHuYZ^meK>Tmm*Ay~(^w&Y`Rd%j@7Py=bwt03+59JN z0EV4VMmdZWg8SGk&4ksIW3#kV4}Fe;t0r^pgmo-cM5U z9A2kwH$eYeRpRu&RV7)VDiQL#U4K7f4P{iU-KO9O*R~}1(~Tjp1=-oV?xLVarZMFQV)s=VZy3zFUX>H`TkT&eK40`Mz#94zRm zNx^_-Y)1aUhRux1e~2Cdu@wjBFQc>bh9kWj+%*u)cn;V}-C))>stmi8nc>`wK*2|& z*0zC~W{6}u@}Ci9OF9BYU~(Ptx!_f+P~&>OaktSf1N%F=+cN-vuC~4h=;cdDPU3^F zm#YmB=m}|1k!m9;swe{@VtKLJ{$7qw_VA;4v)}thz>lgQ=~=tO-4hEgw-@mp5D9l1 z!drUi64^Y1Oz0(HiEFJ9tUWFW%|YkP*?{xI9^8QMCL!(5!6zBhSPvCT2wr-XMbbKi z<^b%Hr?l(ir`RBw*7ch>=WE`0FL?faAFjP&c0`mo2!Emn#9O|mmX_jK_8ply0J3)% znDkqZc?Tly9wai2jEt-sqEodze!rX7{VmPuk>fqbs)hQ6~|^cL)?jzF^DVLj_Z{7dQXWjRjqM6^fnS3l%sekp=zly5(wF}!xrHsz5L zslSa)!HbfTCpOY)EXof`5d-@VKj!PMfpF?E?S{(3`SiP4I)~oRWj1;cRB+IGPuXz= z!3(?{@|H{JKJzUIDUw+I-jJ7Iq?Tm_kznL=Fm5^raTDE8`h(yC;5u)ECPDgU4?U#r zmPq=Z1U;_By?r9L;sS|RY|DD{c;t4*9l{K+^c~0=h?LlpB#F~gXrXn@6og2WCSXc( z72^*^%ImO(^Z(_yxDI_pN@gwOzS8vjq~m%1r@}=(-)Bmv#{tQGwOxFcrra1y5?F{p zNQ%JLMvS7mbqtbG0sd7$3p1b$wtRr9EY(pOSR2w;V7u~YZz!j;a0Y&XBob_+olmIo z5C{vQ8$*HHa4wPp^V4I6=?FfE661U3`v>83(Vuv;3T_3ad}_CQ?9}B<7)AL}s)n0f zid6tI`8D=Anp5eEG6WZvkHf;*4f$7sqmmu(V1U5S=PaP;XgF^v$QZo}=dZQQzY<1! zI?i6s$9QUP)$1$D_y9tqbLV0#VCHWW76JdMPMfXe^HQ2 zUh=$)$RH?Sy&?1v@x-R-f(nS+I%XBExK(6ff#EEyz-@a;&ZY&-CItQlLgG6R`TVep zzoYL5*}6ZD+f5-|w(#S|>O%iOBrs-fRPlM7tMWO53Qf&aaDaUgO1|5vrH zYRAGl*$iQct3?lf4 zB_-XC-)j{TzlQH5S`-nVKt*Lx5mnOoU5n@}QxLT9iEYI3+$839UzX*P*=cw1-~KR} z249@)kMPI;Kbk~`m9*4zX;M1NdcwUAL8&@>e0Jp{wr*Z_bF;_D>DTGqiy5-5;G#}l z(SR-vMH)ykPomNgo3P-(7Y`z`K$ymY!98-E63unvOh{U{9Kn>{hdLlbcJTA{y_=2_ z;l1jO$iGH@&Ctds>7q;}mW zrQk&p$B_W-L?=G_VK6AYlZ>SEI}GdgMsI*e!=>yd;4&!T-Q(fg8N=}V+t4}Nl^dp( zu?4ysjZ@gd!Li)Px97}=z81}RKiWMuPAcZ&Np<(zD{l95e{92e`qS3qXqB$M8qHii zZI`GK7hW)vqByYTd$a#~XKX0D;SYH;=&7{3-??}1t?0NSp&ikiB#irC{p6?A zb>suty;cIAQ>Yp@DL||8{I6bW7Kw~VqeQ2Ro|$FN65?lf?@|dYNqQ$3977nGgf9?p zp+rsJxO`cluK93(TMqgHMGDZ+`D9F)RSeS(A>CH3l(r-jRg2{oSzzUCxg!`yd0i{p zuoQ`N=Iu>MI>iNi`?yzvc2o)PE>P zZm(3es6PFd;BM@$DUPTyG(L%Fo*K<^(a=uqIs>+u1{dKzM4^g44yIO(rwXBQ8L!Gz zqMVA;T|po(72wnIX2EvZPmr3qVSmavH$Dw?8!h6Crj_bV?HHR8>OrGPf)MCzuc68b zdNjYL%+*^u)49P^iO%sjV8`Nivl>+yZcgQ@pI_br&8zl7!mivKDj?BoI2+}o)oA`X z!>O%^)V;=@o1;27Oux{J$e=NaEeu-KFUEZk!s`dXR5Q%tEbLc=dOGFO2$W{6}9}f$={b9m1pWO56%g=)^Qjb(z`tyBzMNmVET}5E~_p zYo|$Qi*IbprxcOF$XIv?HF5(bZim-KA#9FQNMthMdR%ni<#EGyB4ImADg$YX|L zaj~_1_CRm^e=&C6@l^l+-?!peIYzRwM25<=ow6%xrFhmd)Yy(*$1A;}(D zN7*Wb9FY;38A)8vcc1b5-hRK|?Ygf2`lsWJ_xt^Nj>qHvz{qIPH$kC}EpIL$|E>0n zLhtHho>B^)pK;&0UA8*nm)3jL)25WeF3=Wt!C^IH@&4seGHQW;dX`FX5WFc`xkhF7 z7&u+WsPFJnJS!7)HLq@6&#e&?#49}3=O1Y{xH?EYnXskuET16q@`&E)C;4wSWj>kf zGXYlNbb~;ZThZmf%8rs*6GZJ!Wc_-8?LA$?Ul&&N>BQFi+C1`C@6C;fPHWSI#pjZl zqNJ19Ysl;p<^{LG+?g1Ak|ThiF36tefRY!J!QWJk;`=D)qH&Q&(gv`QVe+Dvl>*rR zZ|Xws{{Nva{IN8CNm#*%H@y$-W!uW1jQjFHE7HJwg7vrX*44LNHkr-6bUl8l;>di- zWhL==((siw7CXA;*&|oU*AiG{{;2EXvXi)tm{gRl`{txcJm9i-d2;|X{Q6{3%LhO= zcIMLP*m<+1tKuK?b@SQPgZ$qdPDxhV)(}D-bn9`L?YNNiDH(ffFW#xT;m(45U0qgk z{h0%IzY7Il7txX1s+$vy*M4Y!a<%guAXDj3Y6Yq`n7;l`>r93B56^OcioK-BRlbIe zB~;1k*nBopLlO4C+xkd_2wHyY#3^EwR;$3al}JHuS^xVxm5VIjEN^qGIgvG`rxVYd zGE$J31NqI|ZM}{(88SYh?e|oCzrsN6Hyar3LLxQi2us;{ABIoK zYztaE9_PRE=G0e6Cp_$1boxyzC+9#9JjzEAZ`}3RW|G5l%6&Nc8EQ6-QoC632w{qh)Xm zBW=d%DJCEdlo@Q?`S7B^F{-qX9B*aS{r25Ehp#M}v{#NFKR*5}_2^lv=lZ0D2S#7J zr`ac$y-uWScKVRor84x!plE36YgKw2DAOR6N?r3)#$2iT)SDXiCy!G?6zV_1$S1IW z%#+K^pjcn)q0jtgHPVoc)fV|_M`Mg(#Gqc}R%Bzuj|z_^3Phe#F+Mqn2m!0l6$s-k z%0E-wQVXM`PtpmX81cXQh+vy9($P)FZm28(dsUyu<8Lh4s~KMz-~)FawZ?$sFv@~_ z<4iw}8g=u?)$)g(fnPw#W>?U0P!YP9mcV_lAJLc|gj|T6^|pYB19g5)RSMtT2TPqv zUNYOx9}rXLSSIvz%}_wS;i{MgYmbtOH7lBZj}TyR?7`DZ_IlrBgdmvD{4 zyNB3b$s*z}y0}3rcG6f0raCyDo=Xq%0q6MRn!flyT7XbzxZHRJGXcQS(Mq`nd?rR?aRwAEtQh7L2ZF;PkV0wP+BKT1lSKB zQRKxBCbMVwSoM|EE-81mj+O>$XC~be zCNt+=R|IT&((#^K0^7v9EGQ8^VSX0rJil*G+|nNB&7OdtQg)jp{5fOPMdv=lBsh@AAeh1 z(FJ@|SH5`9l*(L^2P$l8NGyy=MzIsQ#b488|!d>Zx{+UHd~6! zJkoFHeGSps<43<)Bz?z%aU^9~J z9&dghH4J0QOa?PpQ2lkJm2siKxl6@mpgx674ZfS5lLy3DMAQeiUk|JWk1FBVKzXHq zc=0c3Akv~4&R*mw=##qRaQ^xTo5-k=?Dib5I#7f?T<`m`%-%UO*!kf&0txf*CLs@q zc1e2kbDK$XpTJZh>m1g@L+}$17)H`>G*?Dg$3>^gP8*K3-%H37Ywqo+Eg2F=O_Mh3 z?z~a)2M4m~CHrrt`Tq@8qmvwql>C%=l!hxUeW-A8kN{>qI!dz!62LUtyWGlNj~_sN zLf3ysNo$wyRVvZ+%$@TQJ*BXOm7e(7bNiI69v^c-cG?LF&LRq~!@SiL##W3JUmB{$=z>os3a~$vx*Ngor5)~*|GMDO>r22< z@?%#~K3$S$Z+c3bl(ht1%yD0!?KqC?DP(k{!z!k&HPTZ?Son;U;SVI2 zaQmOGz0hwtNXCvnUNsWIJL;AdhKcBWij2Kx05Hz7W^#q8&5-3t*;QL=wUPsJnxVw} ziyeWXtBLf$j>i&RsZJ5(#c1)KuR+t_;OfaLV`L=929yfn!}uRy_tIZy-s`5(Rt-=R zN#?J6C|W_raD5HPA zXg-igrdc@+KUFEM~E`gMe9t$kz4;^0YL%#8C zHq&VX?;`^yl)oSnh9$20R@kJh%#Ja=TIOvIpD{=_KW7_LcjuwSrvQN~Po6!KZQO2? z6jU*8=j^2Eh&*L~MBg#@y816HxNz8ITCF_0aM#%5j%0Ex-NTJahRDHaSx` z(`xc~um4BGm7FU=_iZYNTv0kIO@TC2i|(I#U6(%-r0Z z$J^$dh$a6ttDAR;QEIte3gEYlkTQLdI zfV{d$?{KYNyzVAv5$BKXfK{oxLQP>gelm(%k+i8ZH}mzlZ@+O5!=%5N zTikg18#2PB<55$^Q86DVj??FIk=4k{JAKm166qdw5kkfGui8^*U$~z9{DYu$03mk@ zD!!-n0*v)U@&-Ah`(3K~-8_US8lDf83}O9dPvt z@0O7a?{`_pF7)UXvk<=d_m?aNDQCYo-6N|ODRC%{EoX>#u6oRvu3~G?bxl6BlYYQp z1+xRto$v$yO6FhxBdLkZMcRdwIl+M2-pls^O{OjJ>Nh|tA}-f#fxO=>d}G@}7Zh+N zAGs~~LUoS#e-!m>R}^WgaGqM891kFP;oZ7IA82BfTg`ruY*6 zj;YYLcQ76MV@724)Fw@D=#c&C`nraoDjO^QyKZno42PQ=dUtK@W(L>r^+u*$qUuxS zBqCDh`fQ!jEr1EtL3rpcOyOUn3NR}kTQ$(q98iun>7O(m%Pj8pI7Y#98K^x+904UOh9=Y8)j@tCZfqn$0a+ zm2>6Yb#i*OBSRiHRlr5Pz9~IG+iL=URm5ik%6qN@J>~6Fgae< zwYtGOqOGm{B{uAk3_W8a)fHy?Nu7%cuVf|^V$7^3NzVkqaH2oV>by_*0^tbx^Cmsgq{5Ng4xajJ4?hZ)9GlDXQ zvJ=y{nv(VrddzN9laoS{wWq=k^C{Q`QI0U%Jk1c?p07HnF5Pc`0HMt^H>fMu$nnE( zsSXRRM^lxESYld)?1cWxxSNo+3ewjMKOZC90AtsroEJ}? zFiKuYrBk`}P4Hc?IPDh}jG}t2_}em3JxyBbC&DqIdmd2AI8Fp^r7V?xtckJT1ExvR zBfV5%Q3=g3C9kf$qz+w;wW|h=|o*4*m=Qy;o6yW!zk4O7FAb~BF*~ZI7m__ z?BY3Tc`{E=-fR+#l&FL#BUz*Y5;j*z-!A1cBo~$X05S=W19tM9OBP4WA^x|kto%4LjW7c~;73Au8XfYvUVGk@ z5N3i*ycF^LPYb1uc8k4&5!BI4kSw|?c=H3u(KQTTcZ(v=UFV@^&==;C)w3az(zPU_ zmQVjLjcmSr=V`={wjTth(mO2qmn*p6z>Z1QE78H9AaA|-cu54dxsCcWNAj-Vo(6es zgH`q3{*)z#m76T#S`%!{_zUuSo!@9Ua(AoskZ`?r(TZnK+r|_(JI`n^3vagU#eScP zOIe?Q>Zg3bw&PkJ6FeyXy!CSwcV?K4K%IOZxh;<2YJ#2ri19Z4ZV{nP;Qd!z^64LO z31M2OS$9u!l7an3rhD<_Mw>bxz837vlQEGN_pEF9&_Ycfeum~*S?mdlnqRQ-*2hZr zU~b#MIF>y~n94kG5*t#`PFavEu4qigqBeP87qpU50gqjzMr>oPYrGQ6v^FhnMJbxa zHy@**H^#G_2s180dqmaIS2pJ~DKOssdI=KCvBpNJ28M>rLA+qKy)O!_(?$m25h=*EcbwugE07Y_0axnNGk}V+bClLl2Jun;J=rBc)nP-``u_R{$uN zRK9M)XIev)8WVG!KeZEOFvvj9eTO{INZm~VEe5`hZ?tzAO62=#oFIwykzvtUYKz;b zy`sFJV^e}(;yy41>_vE~wA9cl_3A2Wc~AU{tr00@e63X7n|8(4;){#kvzJrIh~@O$gk zABa;6{;+E`s?Dj6?D(5Vsq_l>FE;L^VE0B+UHh67V5k!9EZW{Sl$JuK zh->ti(wX0l^dhj5dH!5&hwahJH%+f{0oy0TjnzX~mw$v})}8rcvS>dxqiW_kx1>*N@z-#j9-r9|Mp%O@5uNF`%TnK#1^ zGU=$}!OVxPz0MY-0IS={e_9pkMZcV%J%%YRYZurKsDw#vqG%a^h!;UYb5d4u;)~C$ zm%uhq!z5ok{$W$ZwEN@BGhGG}vJ*xt^`Asu+V$K9QyaO-q02aonNo0CcmpIPafzGnZ}n%eJ~?_&c7aoX%XcQRbEJK&^K7xO}{K>yg`IeCIB3W=hcvkUz6uMUcLK)zP;v87v{3NB8;< zl3g20$fybp*C*Aq&YaXhBqoA-4~{$sZ4P^satolHrp^^bh;fvk`!;}2Eq@x3g;~v{=}S%6IeryFUmcNNGohyL`slHQL+1 zK@l0q*K(RUESt?|z~Z;b9scUCxX0X*HxS05ynMGnOS`K;3Z}$hCg)Qb=u&mFr1?v+(pc9fsP9+SU;ehR03tM=Udm71-Xq{3xLIHGf**YzUJmsB+8MDtE_*mB5PuaPc>Q7L@B}DF&$&C2=hKw` z@N()ijA9ol2i29noh3s+dkxy;0@P#E_-{t( zQqA{o;b62-xVwVW3 z{y&`MraIhBwyjRghrM9R3C|-R*I+~)fvZ>R_B*epyZrboIFrkVcdCiQI(;ce(&Yb8 zUkXAe0Y=t!CGv={+f8(kUlCf)_*9t5%TrdveZ>5)9sJ68nEhw`U1ouOGaSzV=hK$Wc(@GIEPq1m@rOFd zS53OxAv#;LP?BtGM*lS>-LUB7L9mJIROa>P zw0XCXrx4Mr*jhai_ z$ba7Re|!-v57hrdW6`SU^NKOb{mi#e7E%+W`=v;ByB7oh=xwWgI#htdQXv0j{**Ci zXaTeJ$x8lGWSqJ21RmsYR<5n3GWr0p;_LDAev@$f{6WYi`|wvV>M`A)mx#tBA@7Pw zY5e#ua$!$4>eZ8hphJ%ec_3jfz-yA^EL1jhoiTm{_j=Mwa$KamfnJp9HeQqAG1Z=d$dm@I-FMkt-rsxib1pxjz?~#` ze=V(1&O~+fR80ivu)T!coSmf*)54?6&417E@cq)$Z#w@z$E*zU)Z&EMf9Alu3}Aw< z02eYp6OD=fPqy%d=%#N=%@7;BXR}3k7AU7;;|G*bIpB=ceDBGWf5+w3rx7Z$?L{3y z9Mnj^_OSY51qzfXe`&U=|4XyA-FS_Z^8r(UTT}xH`KI}|8!Vw+$n#QGX~CjA3KP{Z z@t-KVESN~>&+JwV8Z)mn{G8$R1npB6#DEU5u&|`_?5#x300J+Lnvv z-+U{IfVvftF5ee^tnhFYp8bri2-O%0=N#2`#@)ByVH+aue*6)Wi z-!@b`9w&?3{#@88#D(Q>erTTH?ijt4r+%3Ge*)-e{x<-ffIlMJb;v?8XMf%?*ZN33 zEsFo~tfb~GvVKDehyT93zmtY0KrqE13}wgh^@tDMNai{)eAV&)yPC^4_>YB)JtyCh+%^L#LWUxNl8Tnsb@q|)eeQgr6{$6kFI}+i)gPKbAv?*v$N^n;j`6x z%LCj&9tne9thH?>TZb@ZYHy>Pypz~>&re0Vb|mmfCeyTUgEl`y9bkzivqH3uV6VzR zOI@+Wkss8GJebmXT0B5$1vd4f%*sogTkL?Pvn~<&NY4-75`a@1fp+;BSsuE(2ku$< zM4{ALIi{$+7EjAo;!o;1buQ(~0d$T`kFo|{Yz7ETsd->DUOj$kseJ!b?#%&xk?)TZ zbR+k6*L$-dyyBfV`fT}!_1RvNB(F(`?r} zJTj_)%~myaurnRT&%tT>C^1Ih%{CfIjI}2(hP``BndvXA5bKI`PFfFcN2(%D&D8zg z>=UNKYNhfwdcN-AkQs)4DT=Yd@|eCAV(7hoZQUxl~?Li8M0&ab9L}A<>m$RvwY5fR}}Kn*jsA;XztmU<`CdGDpBF0Gzm_m8^e;o&hoi|y}}7>zAMsxCnxv_*US zGzHE`CtLhvcxEHJ$Phdv;B@Lv5;E+9Y1ApFo?f8NO#Zk_nHZ%mN4I2Lkpzd3%=7ae<-}O@i%-}~2Li|?$4|&(;8~@6?`nzRa$^em* z1S(klk`~96pX$#9up@j63gF>T@9bkHgjQWf0=swferep0oSK~4+W5M@dXMP2m0w2! zw)6EDcWnF{kZ0!QB`@yFML?-Gt28vwJb%Hpo5P%Nq3kQi2kZjex`^}BtD@Q?GsUN4i0$naYb0jE)qiVZA2Y1KT6n~Zc2{lt*dAUsg zz0oeU*2DAk@+1jrJCFbW7<8r$7g>5AHoxwHyguJY*dNE6XZ;9kVH=2MEjMV#C2_Ut zz!ol(TKI$!osvXB-i277?W!Xe)wiJD1L)l6yi~SJGg&4#RGW^re{s;4X8NRYy_0Zk z6}W3hZ7pGg7Eos1?f6zvh?a_IDQd?FvvY1IhVr5^kY%eL^?TceT(N&9+qWP;)k-V9 zGk_T#RIyMe#urQQMBJ=VX51qL=vYPaG|upS*bL+l3+werJAO%+nVg`;?p*%8=gWmL z$8;+xVPVqihG5+J^)Np&TPuoP;O3zMbt8*RVs;u}QrdL!4p?{Wz$=Nd2!IYt5nSu(m0V2Ww&v zwM66ct;kYDpUb1*e5)=RQ-UmP!)}`g;--GFdWt1=-)LQQNDZxbTyc5-#>bh=6Wl&A`)gqKXbb!TKG`&!O!mPQuJ-&4aa9Yi zRQ-|2M~=cr#hqk>gvreZCW%tMYgh|2#|+j$k&m8Mil?z`BFxm$Z_OPY=ec^Zs!!O} z-}MB<#~m^3t`bLUtjMwcV|(i*su-H2I6&v&^pkRK{?K1;r?4&KK%DF{^De-e?@}`3 zX!%5Fdw5V%1*z>kfOw1GBcQ()vIn5{sCWW+=?!>4S3cuQI2f+Z(Tu<+yCKJDdLAd9 z?;&E|7W#Nx65#M{{;~0^b%j_c9{bF@Qrn!H1+_L&K9B8CQlK}Jxn9BK73%$v}RO^i8(#B zwB)N%a|OI58MUDiTsPyi4ldDQzd#m-O9UM|{~DXjwvrO@9!aS1wI#vrmI!+0OHy3a zeRWe6Ar);p#dJY+&%&yhV-Jk)g1JXqrqzICs;!>n+6hiQ$H$~>5QAu>knVMV!YPGB zpMHZpl_=0y*6pykS9KwvwqPapA1%PY?6#LUrrj>p7(Z5R-Yf7W{-aW@Mc)s4ej_i? z=w(h+U<(^~%rvUSH>i8+Or+Wh_3Zi{PEEz+=4jd!NC7lb>JC~U`K^~m;B_S9S`a3;v!{hc6?T0Ny=WVziPYm>D4nIW=}d{4!pN8qmkNC zeA^V8UBYz4qDsSn^J*F1UKQ;*ty_!+aZ#~tX;Nq_6{qaqQY!NYR}%F#q>s>bE{#5% z;N4wvru+xf_BXyC2=1vLkf(^C8vHy~c2&2vJYfpe+&g6w6qWhf}Pa`IR*d%qFC}xB z(8Xybr7viJ!w(^m(>`j0uM+h>Naw!J$ctznyT?}M4YwV;KP z4X})TcUL;rQ zxKy`iuwG$@&Mv5(Z-|-lKq#6i044n_2HWlGWU9AZcslD8Ym~gC*aIVyxu-Owx|z7s zz*QA4*ESP#<<)KtjRjR!*q%^!S|@4n;6)046W$N4N})T)gjvu&X4evrmVb%)E%oY> zSM=A}?6i`~)h{6aPdY?qcBixArA(K{<5jyaA;l3NECh3Zh=k5D2k3-Q*91|CVLip$q626lgTX<&E}6+2rtp^~y(oWO&l~?W4F7&ERVrWMgO) zdvix|NDv?14n$gV(}+mxzY*N@`_-|Pe+X_QijOmc+^h*Vgu)(sUYD6}o$JrPGsbi4 z{n@o(nH{v7n#jAJ{Y$T|3gtbRD7qM(a<-j=>egAj-!;>&pNXnTaUyuS$uXV!lv@U# z++y6#MYm}){Yyg2v`5v?26+9lg=0AH7c(IGN_j+%z`K=z2SMaQ(7bapaA7ChomV=8 zwD8j6rPCSDB6zjZ6KByZYYZH6dNL!R-ReTmUBG?hsY3$akVQY=o3O zn2v+@{cbnpR3*zF;rxBx*~|wBqNk21+&Tw_kG2HXRa8*TjlL;L1R7_s686jM8{wb) zS(1gxmkA)HQp;&96Nuooziz9q%pSn?&aTty2D3jR;X2DxV$)+IFkeMYux4XmrXh2Z zCo{AYK1T4CvJWA?tidcZ0=DKdV=P^9MHHRd3|)@O?bIpEO8;cXQvsOX#H$?nY<~>8 zD4x$n7j+Ta8r&X;*DwtEyNHB~hRr3qyDz!AFiWAF4H9L&{v^sSQjE}H*)B>~%=KVo z6AE+fEY!aMI6gI#L{D|?uo;^7H{Pc`YLG2C2?9h7ABamR7`p~Q*zeER907jGnC*Vz zrTRL6$$Y|3hOqkunW+NUH=*`OkNtnNIk#^)>E9h$8Z_j~PPiKn(V_q9%Wm?b&*2iZ z=;(j)_91Gmxpq-CYp6BRSEo4Oxw#?;Uu4fZ zgDEqXFEd;(SbYsK8%m>s{>JU);@St!y>$Ithf!tJSf@7D6+mYmyOQXbb@Tw)HW< z<)w@1564ee%(YG3HlFTN1ENns?K6% zoVUP*)pW)t?4Tv%3KAp1C%moUM)Agw^|zp5anJp3#o#{O9vD7|H=|N*S5N9lqJY}* zgfT~Gr_DG=`K~-$&PD3c3#>Qe+gIR(lAk+7y*O~=a#J1~r~zDR zL!f}=xRA2`7`jL8*LB2vGq0nU$mB)>K|HLNq{RD=p2HEfY<^}t^8BO0!jfU2JGRQE z=o8WW#P{6nSCq&a0O1|x6siEd&et&fNZR8*Z}j&YCyBFFM}`*_n3EzRC{4?1mvGYO zFJ6qvYKBN+yZDq zvA0bTD+_{MGgL$(w)c^?A#Qz-A9SXzU_B(Co;UxD=quV~*KhWUEs<=*DzpjHSxaOeOt3X0tge_T*ll zlH9QE2C0fXR=G32W1*_0DhK^@yzJEj4eB5^_lNN(P9WuBWY*Fn_*HbE*Wn>U1;Hoa z?Wl&d)bMagtExVt`!0+-VpJ((N^**3cH7%u_yGQvo>VyxM;z}e zU-I~S{I;QO5!&8N2Ydn=tj<*O7?n zm=v4xH=Gs3+K}5vCqxwL6pCp+y_)l33B@9leDRz5i?bD-bbbQp@=0teUl9ngG+RY^ z?}R7k%*~40^$=O?b5e}j_A)!zonrzVH z$6*SMbB>iL5N`I}OYV+LXJC`Ul!mXE#u~PuqqRG21*Cl!%R0g<);w*>u{)jKm-M4= zO=vhl)RDy@8ZYVs47T!T7Uo4lYAyunT_=C=89?hwGD;Uo?c(GQY0_VreHVYtmS5`( ziBStn`6W5gD`YLVzTER1zctjE#;?&$c20z#CZ=p_yFeos*0}Wowv?E<1~2pohL;+b zCjm0}muo#Dt9InF;lRmy>o1`v;SBM8%OVX^!|`lNH|`5YEN} zxSm}OFW=@g3b=70=GCX{xDLJEj6CEE=>2f%NgU?pM$flfZLk4pP~-89LCzza&oX2t z4qp}cCg*Qx2=s^DlA%cRJ6cnxe53<6jhoR~$@BR_&9_yI7C7vnof6K}$)PwaZ)RI& zD*WJ-(`aG6=`fF&6VvJb=>p$h;tO}2xJc=a(QJ)$$c=U?o3gdE&`QOoeemjn1(fIQ zL~;60zs*CfMZ}mnGLR>l3$dyo#L<<6gG|91kV9yl+vlsl*TT4M=b(*310?@=ylZ9qQ{eV+Hbmj z=|veVb_WIq?yH|vZa@qC6T2klJlLMTG|tJ~!132I2$7%&BliqKMhMk}->Y|fa1s~c zLGtQ6q_6l;@bBESmAV017|OQ?=(VW6z0OwNMgmV{HY=y9>rqBb&foNSSj98OvxkuA zz-iiN9H2Zp&0dqcbqd+Rh17OH16#z z+!@h$&#nK1s4^%!lH~F_zF>BrS07^RK$OhM@hf7Z(F6hSo@-3=dBdG60Ee_y1Q8E zJ3MVX|19Yh8*^Oh&%j`Sce{E3MUf>{epsO;q)kYItkT1j-$Y{ELi8$s7Lw*j10_s` zre$xkJVlb=wgu;lTysFE{Mn{9p@PymLd|g|iX*V_;M|QZqib2VtxVb9s(vOhyUk;Tqpe}1I+h_dW9%drn1KfX;W0YwL*cPk>l zaMJNULW6ib`XKBqg$y#4`LD6|)`PN&_t&!&-4pyD#$hsbg|@vW7JxJBWfeDc^bSsl zCMx~tcK{SDw(6$0lJfy?x z$bg%8dO?g_xMDOS-PGp?LK{J;1~tO`5XL5ZzhSWv6UUYrm^GPq!n3g1-$H(SGWbdO zJPkBB^KSFz$$wRp*AAWVy71v+fpNZyD7=0te4M{tf1hu=lJHTPpN?JT6hZlOyArzZ zJHu^?>q8($;r0Z*RTe^-fa!m{G;9$|c6IuavJOTYeS`YXtqtr7#=$! z(U>K^zua1tc-n=(a|S14y7YMh6z?89`qf_m@5~V}JIKtyqZ-snl%?iH_Gp8BeNZh- zUREFHJA{1SxZEX98Vb2Ep6Jnnu=5nSD9mT3-2iurFFGqzlUn}na@*J+e4Brbv~$=4 z)P4nT_K;gW{dTn_OBjt2Z^ij(94J8H3?z^7k4Mw&h&3~vJtKFN4RiLagk2D?RqSz% zGyPFy3QvVuF{F>I3NE(sJVFUq9Q_^qS$ormtuRe?wyStURkbO{vuAPryQfn_{pRb2 zJGX?}syL!CGN^lmUWV+3#^C)XUVO6uXF;C>TlLrbl5zh{?zIA>3`+ zacym_D}2GP;PEAix)n`NxMcO2b)rrVsoo%l{J6%a6VD~; z=B`lTd7UFyr5G>yi&Nr%HfhY@l!)ofqq$lGKIh6}(8)5U-JzQq* zkaio|jhT;^+*5?3#c4|I)V${KqLvkp`Lq&z$xoW{u98z7WEuS`VO>MGH35Ss3*PW| zL{GIC$}a9=mvFUstjvp?z{}Er*7umdpyGM!x0%s+oui=GxoKsp~)YB^!6`Al}>+ z5GF?BHMbJp92lgu8mj?|r@HP=CmDIxj5#W3jd$S?OX%V$UNPz*Z{w8DDg5{>WCz^s zJ5;M6%$n6@Q*UtK{;{LmLN3D%yK9E)up`fe`BGMlM2#L{56(47_FoW;odtR!s?X%f z2g-kdmrW7zJ;Q#C=I7pfSH2++Z{ycgH@V{ueYXSb@N#5rRJBs+5UIyScHs1&k zWt#IK^EU%jJP7KLtJc_;y*|F+mV`n)3=A zEmzlr_qT!{HtSvuqf}>VcIpVZLy?qS?(i-XvV&r3jbW|-wFV*O%q`59^xQhgt;%+E z=j#S&o!>nqUvT_DT#z)ao@auePwl<~k^D>`j@id@gGHOS4Xz^->HB^smih3Z%ZkgP_XW;T z++={mm7RAR2Jrrk&VC)A5qXT_thR0nqP{9Z8`a6j@DD+C zzWeL}>fNdF7Z~-};TmW`@7h5QaFXjNO_Uam}E&f#-;#EPb;hc8grX~I`; zBsH%Qq{B;zyKL@L;3*zD8i9zLw-J!i6_lF#L8s?wixohEDo1_$8I8G$99bM(8^^ej z|1$Bg&7Dk9%5x@IH9&TW;X6=cWDq@qudlCe{R!^h0Ej8~g$RMKvp$Q5C2hu#tkNY0 znTXp3`xtxY9Z2lrzqEyftSa&~EN^>Ie z>k|%Np0%0(-dCWXoFAc^EMFt74%&3P`pmpMcMgT?b9XPnTO-B2P;1Wh*}R30Zc5bN zootB)tw%llKI)Ec_OB9VS`jiM;nGpS@t45a)s_vWz}!CxlLZ5|vHp-S`8g_*xIA3e zf|x9fvX#z$ZjYf*|2P{`)5+;xcb3(&nPMM$&%?NV*jQzN`ojPPJB~aa$qJVfLEk?; zEWA%ts`mcyG_osj>wCelSLRg52*5f;gD{-Zb>r{nb(D-Y5JqVg zS!f-{Ay1R7`*ElEWlxes?e1tsv$|JQq2_^`u6olBY1WQN=}|3wY8CS1Cf|ZoF;|oZq10Gm`-i$fLSZRq{OP$c0g+^q%ht4`d9A!CMevC{o$P!hW zS3{||@xP{jrb9A?s#ABSe6o%U9JVq**ODRUsyFq?<8-UiSW5PRAk1(goT}q9@T6zO zL5`7wWFVQKd}^`K_9G5?8SON2#v7Z)d2R<4+;dsZ_?5$LQI&vM1x{GM#mlZqgOsjj zkTyL~zC{MMqUF7SIhC5&kOVNO9bTh&~J)ala8zoo}+a*WAdafEyV+-tX6$nedZk&Pg zS#WwA$zVkIWtfNLzLDXYJM&HcgQpcU__QL@K{JvGV@CZg8}M?FER}&6M>CTzy1#uL z{?jD@LA`%|NpU9f?iAlP__`{q!8p@7-DK+KSEOi8M5+l zB9~j#`@KTDu=3c_Y<9_7c(LBh_9Qlu4%XgwmxW%5Z?=m(oKt_`zDAf4Ag@kCC4IWy zZ^H-a8*OfdqN&YgG$a8zhx^R2p1?`Vt5eMa|mgfYkm2uqgg zPr~wQjASPH>#>J`i%c2QXwv-$NktStO7RVpbF4UROfI$F2iaa#0qpnG{XI7=&@^Ff zsiUqF7ovS1C5EgOk@E`#C^-(yp%2V5lHHL+4h9Y(F2_OkxegKNL(U8 zI_Bz~U1t8Kmp@*comO=(Vo~c$X9E?L&r7t1Wms!FJE3q`=%_{cG-JbUVuXa^t9Ygc$)x4Ab@yhx_%Kg&s1p?yX zxzWJvQn87aA;>17P|B9hSczuRRpqHzSwT<3(leN6J-sN9-Km(rJm?W7mGu1JVc5Y= z-^&RDe)nDOysaxDWN)Z}_Ak&G>ZJod-;Mk@kq;*MLff{Za4F&01H46>n8>=nywhKS zXVJBttz}6SHoeqz5kRiIVcgZ*BJ~g%J_Nob|IO`(AQ~3?D zAh(V7lA>@bIDr^`v=4=|b<)xDVUHT)E>~W+_(x~(gg)eqY$SdS5dQD%BCCPfBC7WL zhbb^yhR;c2;z)*URq@{K!{rW!qIiczwEs-jqDn8VWx$FfhzxQ$S+78~s_d6bntSr9 zT;K2Yu8}uRtTyl6OSM}0qZS*E@0?~0evy~=iap>3`HLy!aYaC#dX|3{_rm1NM*%c> z^YVvHV=39kD7}hB$y~&h9a83(O)@n_{hm=E?h5LVjS=9E2Ns6SqDfg zZ%4T)-94Bw=RBqVURz|Wz!KN{kW}V$ZmN;e2AOzLQ?@pPV8$;*px{j!F|!BcM%LT> zg32G6^Qv2H2H4R7pC!*>`Vtj2dJ@$q#|Ggh58*s$;fuAxj`dHD?$DG_kLX^l0H%`X zm%Ad-TvYR{jCYBid3e9}cB^uw{4+x}>nqB%B<-zak^FW}eBSLrSFC(Mks}eZ`1N(4 z4sTn1KauWh*!G}YyYZJ--^n7O?#L#9JaNh3JIdm213p z<%;y+vmr*O`!%;*8dw_<@}( zK}bsVFkvh3v8FG4O+h~MmmDaO@m{m_#05rz6!ZfbBs_s6kTbu<8A|LlOgwZ)-D@WQ zh(hoUsr%*z;2D0+7{#8d~WMWURn&v@h4vL%M1Q;-*q)qWF0`ZyvI@o*16t~>7jBhZcL}qXr zD;v)bg?1s0&OzL=*H_*!lTrWIoVG51K1`ou9YB2_HZQ}x&lhfsNBG;j+p7o+;k^!T zJ6ody*;!m(wIolVQyO{&Vu^7Eo&r~M>d=T*xC_qKYo7zRtX=BD%gAzfIw@+K!4OC* zch+FW(r>9BRbbz+8|z_!3JPivJzMAF?OjfvJ9aLPBjcfADj{M1M(yn#uZy!_xkkPd z>O!pvkW9qg>VIHCeOju5kzld5`wOfYOaT|w8-8@lgktC@8cR!^ZLtvCeXLGr1-c+H zmDlgDoP<_Jb7ej?)2K>+WvZ+;ej|6gSkm&x(5%qM^?_U2o~rZdFTCPL;}#0Ti45H0 z-P6%rGcIfPtt+I??$73iE@z#Kz2^xelq>3)#vBrJEUBL1nCCW}9Yh%L${m_bckBJP zGC`waS<`V-Vzm)01diW7y|Rl%LpWlqRjB9U#{9>(y>C;GUz^+eMpyo8_W`rSfU$BD z(_tH*2VNIWE>%qx)_f3gn@ zm>QoDS~NeA+S6dgtkR8)#P5k+?5X=XlXaG_)w%>w4WY)ZT|&$5D$Ijr=_M5uY;>@L zk`gLDz5$M`N6E+E-a$sIYnBD8;aM*phjkB--y_PFNe7T9&t4ADSYIbs*L>>v*f>W# z@YIh=7Sd&bkbw9cdDgMjaHd_Kz5)BdyBN9&cN>B2G;8k+v|ag({MGm9VZIMY#qaWN zX}tCic-MmD{0kdvlu|hR8$%2i(!|NywMI4Fn+(_CL2CC_)i{L4TPGPqzE@)QJQIO3 zc0mWkzEsyxGvvj2i*m=VGnRE{N$aaz$hgmx?DdWXY#VAkMMiOqH-H7A@}8G`ua7>=dn zwol>I%w51qy~3gBCQh3S6{-cRxA^s{#sjphY^=fk?aXcscDXC!DmVV=kDoog@PBxF z?|7>J|9?0fdylNFbF7q2_MS(C(x8w{A!KvN-s7NQ%ZQX@lu@FCjIzl{*`Y{gPElQt zSD)|q`}ju{<@+xb%BX8I_h3*-VVjX+QxB3qGShlG`PRO$79$LnP-=S#%advd5}95>$b2pc zUwgH7J+-9an6{$jOuKbI zWOS)pLoM!qU-C!c3^p?!3YbL(W$rbQ^hQ%sV9Q?K54Mon*3?ZF71* z%IuVb&=ZB&4xFynBaJ`0$f@H1YDBJZV{3F_H~*~`gd^}2I0AP;Y+eYcNyK{W8=B-u zsJM+T<}J)1zG_SQKvb;2UiZ-Y3b$0aFG%;_k(NPQ>GE}~lB6EXbe66Y;&0=gN9yCZ3~;=#(-G=I)zwQq!VtL1##U z!~D}Jr%k9QkkdV=LBhIMr4p_j2KUXM5_KKFz;B%_?o|A~{ZY(CfD> z8m-jh`zHphq6W)Ke!n@vYRHSsFuW-v9eaLnbsH~h%v|OaCT{fIL$1Y5E<&S1j?;6o z7tgpt{MMj1D|N~ro)*l|Zg-^C)Nf+?diu@@qc|1Ia-9I)DC1g>6Hy*YInF&|nV1A6 z@-n=4-RR**_C?b|H7l(0(<}5ETwf3yFi>+@(s*R_Yb^{fA!bPBPQ95f?;l;*zD1=$ zkXfBW_pf8_TijYLj(4gM;3mwz@PCYS=h%1;wDd%Z{J0a9#=coieS3?Hs2v4``1Q$R zyZy-5H*@tVQMcf-!}ymFg#MdyASzdQzAsqPRq;TbtMnQw;_W$$mLy~P4JoH6^roO; zntU>c6#FaX?R+@Ik|d`omI|UwdIG%fsxosFV9kyeJ?^=gR;Df~cF3#g! z!n$W_k8^nliFmdBZopp*ABc)J(VV(*(x`Srb@lh~AB!t?4^Cqvdpvd&BW?L5)FFe% zmo6p*3WaT(`A*S~mRaaArhug6W=eKh)VA2i)NIY=Byzg_L<>`xzLZI4-*+LYWOXXy z<(00XRe-3j1FH(utbWTYgU!)-)in|Di!Pe8iyGu$Ko#{d)YtDTZhA($cp5sRkV_gykCnp@wZHZVI~Uj<&3iBd9r z1K=+i(Ysxv|AkJL!()0uLkrDvtLFy3*MW8vf(nmlZ%)NIzvGTd-;li$74_lV`^U$? z#NOpQam(l3&3q*&;_g=Hd>0fam0>!xeHAWYAaR`d=AIo!Tbm(Dyep*qVA}vE?H0mV zBgi6Iv*zTg)A1K7zjWF7Nn-J6`XzpzuXT5k7=ZvHs@~=;2Mf6Y)6Ew`CEAE*2Oih( z;HR{H^4p94u#m#X1}Ts06-vp<=gY@}CWx15K({-=OBSU$-MWeYvZ;FXF|SM0)w9CH zKYvL_R1>10lA8U_Su(%8!qq8&{^P8HrVczYtvRVLrVMPoelTonJwb0PU&NXIZ;IU0GA zv{%5~(z0*&>xFZYXB#)rSqVt9aGUOr4+PCQWQp?ovT^;6K1}RzU`23o!~450&7=nu zpJ_CD_V~~XOu85pnfbE%Jy4Pc^ZpK#JcTp)T;7tH==4%lWQFWqCxz&uK1!YPf!MJf z)pP65C(7Dlt`rnQpK`rxFU$rZxwxCJK~9-13=%Zz=i4a-tPWYieq}>#%p>=AOTPCe z`_v<|nzpbgG;#9rytB#B118>ZKw{|%2a&XcSqJ%_PJ0G9NBqQ!1o z9k0iq@R?^~%Rd8Ha1cMyXeNy+Di?fqqJkLCwT#w7Sg-Eb4wF7cq{P`M@C@eIDlc`0VN&w;2WU)s-3?matvPUqKXDpT896E ztw`dEvW>A}3a331npb+b?iedxpwJCAb+i}CO}Og8b-_pZAITyJNET{tUur9nr+o)c z%d=G8dG!zG?5cvevN z-Hv(2lO1w`W_9Sk@WJy@q;K{^ks^l`)`;->f5>nN+@$Iq@2&_q&S5YQh;h*GLimz zx<|BNH4-(OtjbwV>UZ^!mWd=y5UrGO-YTUTfa4@D{g$zqgc0vquEX<+Gb5BX%v1(2 zu!;kZn{+ljiy-6aOp4W5P$O0H*v1Cw%aO63Gm~w z`#jf&F~SOUTZgtjRW6&0`X=f7yrk4jI4p7E>Me=qTI+115)J1o&t;|{(iSTa?)q^* zP^6sl1VHoyOth&8h7LVz2=^kJ#8QkQQxEF{RhkLNRFRq@c5?dlcw^YXk7Zzl9$iS6 z&sbNajrW{xFnYx@d*kYuI{#n0sy zQ~g1jGIjl+?^xnc;?X?EuGq(1=ZrUfPW!#4GL&bh2i}7U(u)g%;c?gYN}=4XTusOh zqomSM6?WS6&6p$8OuiqK^lo%(z5<*?AAa>$kNwwgZeGevou-K4aJQv8(aDjjM*oAT zSf`DZ+17F2qAh-qEZ(hK4R^hT1$cl-4}Ht;Q&o74YG}jp_XCyrPkQAMs~{@1bDbc0 zJg)UC_*Dq1^$A)3$eUWSR%f_u-;4V@T00K!z04}n3usgqlhp{Q*-wAaodk<$|(W^>C7FJ z_xZhyyAf0N4CW!aRt>aOI?`-OTs{;z($Samms$1$Of-5UEWYo+a0z>hp+|aZ6WTgs zien4Lc%$x5^oT*uDa0Trln#x5r9}1uhLtA0oApXs?NP%=GTD#ee;s^pn?>~NF3=Wa z?isRr3Cj(v)lS-_I8Ab?Iu+T(|EEyxsS}jBY2ck9LCej4O#WC(9OLiPl1+1@|Hy{I z88o#ufHLg0c^1AP+-8C#I%i5V>eh<#i)9#ed)l!$eNqO`42f=Yk0L*h7QzN4Ci3%8N{ zx{H%Kg9SJHSK;SZit%cR>ohcJSu&TNg!Ft2oNaCDeyTzA(JdE6BC9yyalgk}4at-p zjYLsvOnGC)mTsGj6_|&>?FUHZI1-V5W2)uGDI%giNAx-JYo)K*r|8~PyDjeQ z?3{s~Q)ra&i}221El)H9-^tT;OKgh^)*YtW@=<1kOwTr)K6s*~n+&+!;gV<3U|TN% zk^vc=7`sR*W!mSeJ>*FP6T(Q+2A7{fw$+{OimwhSj>uf943$m_#&fx={C9+tuIN0_2#3 z{#jJ(PQz%ReAM7pPjCaselt1Qc@)}HPeI-+HGdb)#W9f@Z=_AIM3>Gt7E7vF{mr{~ zRPiB7GwvFs5gS}bMJ2xm1!7t=hTQaUp6n>&Jj`+5#k=fqzMRXvrY(BSKMpr%Z`Bp= z`S9BS=M5vqvnU*RXX0kfif6z%6TZW+59?Q%uB8#i7mr@D zMssU5P+Z$)t{&DeGY|k@y9T`}1*a>w?WS2JFpITZxTNso*BwsQUmGot6h&JLGOc;_ z{ivc}?>R2L5S~~R{>!N0?9}BqKhEkhZz8@!zhx~{24WB>PTDV|1cP5gI8cUp*lu-? zrXkjIe$g&WW>;Hq9j9I-j8O@3MQQph=%M%7gwfqUfng%Ps7>ZZt?{7?MrotF>r--@ zACbjdBjIZWH>kUdVz_qNoY2=%DX&?x^*bN{F(gIPr~2>B)!(6r``Sz)Kd%i9a`KTf z?Xe_j-N4}MQT)l$J?gw)sbqf+EY?4x5HYi-~QgSNte&*L5#u@nPnetHPKIzl45$%4E;A_;J# zUyoJWToK`uQ1JoFghW1d>(XRws@6K|?c;>T702E;lWxDyaA;K|4?r#>x_}(9yxYR0 zjZ?+EA5QHbv(+o;sQ)bQg@}+ka*eXIqkw1yu_?)?3rt&^$VuC5>~t3sgjseq%d9@wOj%N zk}Lx>)}51MNh5o1hfkg#86*bn835e91?11v^N%l{Syw5Dr1%Tg?FEwo<6z{vL5Vdp z{|{-?1pV~;8aJtx$tGk9@mSO)s>rqMLrUV0u`kDhk3_b&JCt8~nkkzJ^c-QKws%%w zJ$*)7JJRZzTL#v$8i+~0Aj_i3hIgQM9mSG32fUyU;NoY_tZ>E% zKd0676H?d>5b6*MZv~^q3M3$1po#YBNQAwxU+R`!rP+P<*%}>~`Mmb3V?d*kRtl9@ zxFm`KC)D1%NETw$Ia}Mpb&-3*v2G~N%gW44;PZrnWC9s-zVO>`&|wAY(O1_GSl-p+8GN-Zz%>q5GJBwnls zqjk_u{4zgYYk<_zt4?!dLtg2^{MC+KDH)6gpcegK+mqy8+w3?XZ!ep1SM7GC?b}hl zkWrKk`fXaV?tF)_hi6BA-~A&^%Px-ZWYZ4gcy5p24?jUUMwiMzl_OHwc|)06UZ_@_ z2H&3ftfWgkla*)UK@r`o{k$wOf3PHyp$U%IKr!Sap_BPq-**Ewo(C~%vS8?9G`6^R zME|uEvX9@Hd{JUGK=_5=CDPk6`mh!{7R- zV$EpZ&qIDL@$E|i{lISvW#h@npL0MHsU07So*?Ce>JDUD3F#|ffT8#9)|B%-E^cce zqE8`is~)*$Ufsh}HJNO3k@?&7DBqL1dPDNXDh7Rkpi@rkUns0V+_-RIH`yXR5Goe@MO^h>AzNfNd{cf_lMs9nbI%b% zqxabggT}xz6|nWYue}D2P<}o+v}u4Jo~9@Ymr#gKKcGLC4yge4nxH5|0~za8`r0>s!@Lm?T6*X!;4*z8NMe#zTbuq&P43rR)!hl`vO zGQPJ577PtpY=JZDnmSW9Ma`?};haA~aMz`KP*x~Du5?9fP5AwZb8}Qb<0A>3~D6rQ~o!wl8Yt{ry(V~tlZk{$8pknm!ooI6g;)P9^>oxdk1Ha?PHCXR`OWh zBqNb-^RI9^~t^22Yb`^@E0dk6{`f#!aVmpGQlDdf}tbn{V!tRDuq0s zgVx^78C_u0X}XwDcknO79b?D_Y<%T^KaUwHJWnPnDC7|aF9i2-7RuagUjzB5^F-%R zek$H#xTiW1k834n7_Iq`mdBqYy#}XAcy7e|cOjJU_WzWh(KlHcc<`Y2LcI%RU#eo4 zSH()fG1G5YKy$@LpSxemx_2*dEcRNvKOA~_P|#ftH)VYS^?cs+jh7P7$*{xk-K58x zeO)0~W+j_XJtStGis#!C;0_D-zt$kmk?G#dT>BKV_GAPvZaw5{x3%zQWGICX6;8i# zXW8>it~|?zU($bb0hGV8ua>m94)P&xMMhq+VBtAhW3Ssbvn;a+Q`eRA5OSkLGTL!8 zQXW~eGq1lozlug6AJu4oSE1SuhH|<8!{lrH^uL&VV7gT{qMaQIrdut{jD*=*$9OK) zfT|oNpOThT4(jl#K@$Hgr1%zJj-vhpc`;I`zn}_}MNUa3AbM+R+PbNSwy@v^O%X|h zISK=x4^F97hQJW|JMGnW3RqRApet3-Ph!jH3x*ERDGA&}|DPFq(qRbU7B1dabmYZX zA^tlc92iChR6J@yL(kkN2JFtLL+nRAScmw6W)>T0;{ajU%i#Jg59s^7>X-TZ)3bjr z+D217g#kvj?xh)6Zt?*^_pos**BroWZDc10=>7nO&;+hX>tigzaA@X)ZZsQKQ=e_w z9WnkFr99DRB0vb@1;e zjWaBVs{$qpA7&_*S^!Co8bnk#=PCX|bdfwVsv_{#K*KvCoZU?3`-1t>qw~FN?Cf3o zn^VGe;(vclw4ofke24WpdV@57tqNKxc}&&4K?;NAXh$k`Yd;-Z5EK938GHvwrTG8C z;gf@pI`uy+J_>T)8C@=D#9MlNS2f8sHG=tI0 zyQ8e%-2pOQw5TM5^CeObsrIO@hon74=E3nxI<4qErwDX!y4rREQ&-#JXDJt$bLqrb z9npI9Pn@;WV_Ouup{!hPk|V1VjvmMRlLpfW@h0j6hcaeJdlHSu%gT;2SzchPw!6XI&Z>%SVZ{qNGvl+BEu3d3(c3=Mz5mgFXefC0C^zb!5Nro0wMWTj9uN4 zIX+dtRe>_D5>8LGzYtm&dp}wi810VXY|4C5w?K58hq#NifQ8P@4Y=2YX-Oxv!oM>R1KAs<&wq$+$&>R0aev0;~FIz-B9aBRpKT!AH7g>_GhHTg(@r-ZGW^1#T%#E0dEn1m4RMnGPXgrl*~3!87n=lSYgt z8vzlF^g%V@Oow+~(nY;gV)9?6T}LkjRN=jQfWp`&X%)(TPFxG>J6D9({{7v~Ytl$^ z3=w0bqw@xXuTono6mOGLT7e?y_9wcipFfePR|wJ;{kt-ouK6bjo5-N5*{3+Uy)Ju^ zy(}l?Y-yUDSDTG>+)%?elF_zX3)Pm8hN7JHYX)JPBQz1#I8(D-T7-_pWG4HZNioXA z7izW7vrJA@a@#zl4*CuJiR6&4LhdAY9>3=tIRx%srM7W>I#s=2KM3fa8J}>m2P?!h z!}7?D+}c_td)Fhfr|Js}=()9K{z%SZidA(%qz{!G}0J`rbGAE0)mH{AR-s2o?-Y*VuZembgj_<#ht zrCFD5u4Lgu@deAZ-chsuJ$v+@4B!HorKZ0e zO2xIrWrRZUO32IJn~Ml4@af{pY!bWA2oA;F$Yne@h+!*jYr1g89q zWpV2!bM^~N4nlI`G5BJLwgKI}uwyy}|2Cshxxx{hejc5-D}zN+$-K)HHeqGN-BC08 z@iIH(S4U2fT1P|79hW0uCl9plg6uEHtM0p+l-m{|Z%eB$_=aVSK8Cou8cY%Nqjxz$@z=YL{e|i6gCQ z#5_h;cXIacW5tb2?~)k*%*{m?GWzaDdtB+}p4xbK>?A`i58$xuw|6kMYJ>jH3hd>)M#CIbTVY5RG~?F@<@GYJ{I%z^TpZ$bJ9#)wrxx zd}10Y6Z zg$qasE59-1DBfV!C7z$G1L|=3&^FEE@2ni)Q&d%OSFRFHp%1XBh*5wkc&cVB3Vgy~ zm~k7d=LRRUjoDrQ#j@GDlDy8ggjlw9@y>v?eZdb0O7;6Zs^N!&M#`&DT*oz;T}^+_ ze|P%bsiL{_K)y`HCB$HecJC-f`Ng-G4n6iipZWasHp}7plFEr~Aq1ySXn!ZganHoA zE&RabWxxmG+brL_C~b20O$%?MS9)t*W3v4*$Pu~5k)}SILdDB2CYVhLmH`XDF zDMqKW4}C>l!_)zEp%t0K5f*Io=Tzd)Z@1^V>F)5YUP7@55MKszF!O)w&r4VG`RLt2 z0#d-lE8l(|ePPYn2y&rZ=a0TF{U=;*w8)%#dE*fwaOpy&S9`qQ)XjT;;13Tx8@u$e-80YRhp;e<9?gFwuwbfbxJj z5=EuaNNvO^J(u7qu(kcJt?5zszTlXOi@v}#(g-c2z+~@0gMhf1vw2Q6tg`veyL(V; z{OM{yQ>2sH{7|A$X59I_`%(Yv;hSUhd@~B|}{VgF{SbBnRrjRj*5wlfKozq8W@hg-gAtH5%NUer`F% zd5mswbD4>28x)y_t&`d%8WF_*HZnizxr2UDclu)))x%Par;;5(~pfnUr7)gQnt`X8Px?fjH72rQuf z+nLmgFrW;WsGVVb*@gSwbd-R)>UsZ@&YzwGV1ROXg(<>?f@Ova#R64k+x3#FEMm^r z*Xz{AfW?R6firTxD~O zeI&J)IqI-km6l|IvL8p4x^0VGOb_X2}%+p@7LV4QHC0L^eF-r%6RK4J@0=9P9CFhJ84_as;1rnKT>O*MCVq;!TjT?m zU0<5K>cVdBJY%8+cWy>s9T)C-q=5)7->=fYl)8ZB(9P^cYc&)Rg$UpzTA@#Q=QQe^ z-lm0enfF8;ioaPfCc7#adWa=&>>gM&>s8l=?%(*5{gyQ1)43ir!WA6)cAMdu->K)? zikE-&44qLJzi*-Z{oO)D+v5w5UBJ7OrdHQLgF5jOAh3R;&aTg)ImO&@V%F)t(xtyY z9!W;|mcbpYSz_ko*ljE1#JAtFnks2;+)HqZ>{DON@4`5KetUR5533ap2=`BLo1tE{3JA4PaOezs(16Xz5738T$BJC2|oH@ni{!Tiw@ zU%P3)Pe#pd!b;_M z%J#F0nzC(EH$wtE3%Dgxpy$PZMg4Gy*&8mjhx=MER~SlkL}j6$F=u6 zG8n{_)5v*~rP<9DItHsrR9WuMtvsTCmi2L3*}GITmgUg0C`jED4j(nJGQgRgiDwYu zr6NLxX-c`E;6#P>BjCM)v4#2Gd(H)Pg(q4$yqbouuIVWpE6iXFhQL&(QN~ zRgP5zQ1DMD$Z>7W9XZxz2?{uvc=7{(?T&j5Ey;L5Jo=&fnheIpU8PJ6flYqj=(wUM)7`YGR@K3|zps@=ZN{_){rl3R3-nmWqt1$NgBCc*y%S zlnKRe#0sgy^SU*muhm8xouTI`2i(I2bj~GPhSP(@-nn!h<#TD2C-YNo-uX`{(9Lxt zBXkBbJ5)6FcRwHvlmY@4(qDd6bBW12aW2a~U*Q`TeWUTl0qe~VKI~JO??qTW<#t!c zeWU~zQ)c+AC|UT7iNn6S4^LirB*gXAE{A5ypGxvgT_siB{OSjE5jIJnf>$+wjpfhh zdsp8Mddf)VT>D4Y`vZ#nH+NFS704TyX5H9(V8n`eGi72f)8NIdoe^$cyh-5q8JjZQ z1}07(OzJ7Ce@H!-AYp^-`*f9+T&4DXiioWmX@|3&I*C7BGxWA($w%Bar@q`bSLC2> ztsmy$yesBwXYR3J zExKfL?C6|IYL!Kd&qSV#xwCN2N%SzEoH8dI>jCn>&n;N?9&99DatjD}v#0G!kp6N& zcmAZ)RH9i^tq4fGOa!zY?8lpY9L1_1ghzO&??>{)|Dq7OAaVH_{-+RL-JZIilrdhY ztNX97gm-njO#H(u`*=X9^M=}$xjw?!C)zfDC8S?e>9QbGcw}#bofSKeN3%oGoA@Ii zI?wC9*tD<0QGpqSa^^MaM*7^O&>{CGxHxh9HRFMAnMfs>aL)+3aku3G=|3oiv&G(b z+H;Hbw_ZjG{^-K>muaFQyy{ZHDEzAY^)YOItUQZhgz1R*?J2Hzda$_6dw$H5dR}Pw z@CLE1c6$BI;6d|(%dEV#D-URkUT(aA#GD3~CiX)B$xK|bL#hpt8Z0Am$i9q`goCP; zS1kvZ>rnhAkV21t%2XwpG~>Q`Rib#DNwdK(rq>@bc{98tHwp_HYjf_^wNVDaS*?Kfjq5;IJLe~NTpk$ExWRJjxS#wu~)_oQD2aLIXo7(81+ zvk=|kC~K;Ac2ORNH1cx4dEK)d-Z@o|5l$p8LVc}#Uvgbkv zn%D3`f)}po{!q>RxqAuSc&-Q26}F=Cq)F+4h5Tjii`|A-i7wl6V9Aoas%!Uq2$_oy zHd_XsRW^I)mFukaW6Lm&J-kEo!*6gZS z)9e8s&0d92f<87L{4F-DY(=(TR-!bS$5_<5&0xsE@4s(IvCxL@Vz3)uR4@T$b)lCRvgcOWN2}c zhtD!%WfAm)h0T08c+hC`3r|A2WMTWj+5=X8jtZ;Ux5)VGCF)u0nPn!Z^cUG-z@~&g z4?|bwWxmZv@ow`wd^}q!W!k5={JKfv_d7ROuI@tc#D2H@lha4DQ~3Y>0*L>Ub@1mq z%fLgBq@kBz zNIJGu+vH?_5tvlTx!6I*!Lwi@eg?wbblXczgNuvUq0RWi{T3Iw}W zRGJMrhoqx;R5~tP=E-4~Lj7D5jOKbNr{Sh`fhynCROL(AkrCXM_a{5eUw#XV3-W7M zkPu4Ty@>I~ULI0v7F@y@2)94WeaH;WWw4LzaaA~TSKum?ceGQ^@;u;wXDm7jW+m@j zRaHsxA)vq00C{zppu~uYC_z+v7jVd2gdotP=|Z`FoGCvLy2d2jovmTJuP`W-&xUg; znC)-*wQ&&4izcGEC2O<2U%Wh1q+ydQnUV4f=F6Q7EIe)x_4hC5+f(I#dUWaOjl1Gj z4mGQt5ro#Oh;B!lV_!=;AiT?za%$Ooaefk<1*sFAJ%GhO|$9^?J1`SVSNnNEI5&Kj$jcR1eynxiLr zYVA}Y?W5zAQY%2-h7zl*t+cfw^DecCpY0h}OehJrRtf^?T#kW8=#dzxjw1dgd@dvU zW~r9Vs21{;1%$nan`6o-EOJ9#E!c)0eEVa_FHRT*28Xcmn`yR+@#lk&sO}?E}*l~0slLCu6{vwq|CUvH=&<%kd z)2Q?mitS!g$pSmHlSPH;9U-PAE(bcaD~YzjJqlx{Fsy8jEf!6C2a5NkXJnV;TRxzl z3*|?V=_CuwqB7StgtB8vjE-X!_vyE~8Mvv%Yy!WGy(J};`UBT2wmc_W{#vH?fV@K| z)eiVl#Pc$C>)a;rWX?8tUQwQP*$g&QA?kb$grJxGUY+)-JM1Q3@N2hMH)o_??PiKs zP38zF$5~O9-K+h*_w*JLEOMtjOK1Yy=A!>u&0p&w+e*l4UWV1wXt>4;7|*lz z#~k~lq2?6WrWt&B;k5T)3L-5EO9&h-KNCAt zXMUdqL$Lhv@dLO|0125p<3XKP2_xM~#^DCWIYnqNSzoq5Xc~0z>N?$ zgNYEKs;;K2@@!0C4jsk#Gv@qVslV7E~}JioZR{B(^AP{8RRRcR!*`rxQ^Dl^t!So}g3Yn~6WLtL?a*ObxEWXJk( z@+2EUG;bEF$KEd1$CsYoEyzq{6@B}=_ITYkrmL5@{&%=Zw~laEpH%gD&%=?UcjxM} zi{)hYdp78}am$%M>s19)B4b%Qi+GE%gJ#d@!e4B3f1kJz^?jdfZ0jVcGGZOhQ$VzS z;mOF1yIwKWl=X&zK}hB9{j*K9m8IrvUuaQ9CbCmEsXl?zk6x>eg9wzlOuoEzTLeOo zz?e9jhqsz6r;`!^yfRd+Hetr32O!t4jJH~wR(g{*q}c0CTVSFz3-iLj&WhW=6m~b= z)(ZBn)jDm~7Q1Y9z)qcIAa&^9pa}{5%06-w3ff`0%#-Z$eMf0o*duyUsdoR0FEvpH z1Yp!PVN{YQBTP)3FLy=U+~Sa7{yg!N&9LCoo(m#{iF`Q&cyG##I)fQ_z`3w1)j+#*z`@ z-7K0CCHV|FPBz2+c}Y)|Zvai7NW&e}qg+N`NtYAS<0Y*fy{hfj%=h$Q?(D7q<^q^O zr6=3u^celICr}{2$FOH3K5Nxm&B~SLMlfT71Pi0=UliDx#J8aboZk3!qwy8yel)|* z>pd>wXVVQfmH8jL1_&P431DwG;7<2XpuR(he<*mhggtU?p4(AayC>qfgMqPWLxDb3 zn!mUR)$%fMg>T=|?QG`2hCZWf^*_{R*N>FkarR$RNG?oBHf^Gu1ywu+%U}sjB17c) zgb-SGbN{bn%)P9%`EG-ZsG<*4uKCai)M3Y{kSKJ{LWY$`DnyRA!77Ys{>udIv-G!! z_yvx9#tRqw=GAO;>A%2K{mvc5nKm4@n4k()%?_NeGBE5fCD=!)L-3e6GNhok%k)d= zMF2yyjnq94=(ILfq0@5ACRkFpzG5TS@Hl&JljX$ZNT;yC#i%y#c^#`Sb$9dwqN=E@ zsOwmZNt31XeC(Y28aQcDKpD$}{-EegedRt2qKNrrhxi#r{bOyCb^qYF660;cDQnns!C#2xp0S0$`te%Zk@UKya23V*}K2&P4>Zl7l9y>C8 z;b|vHrqfZ~KqRPe2zxU*tXL-}{a-u=F289X}5 z7I_ehU?1{}m)qp+$+mfxrUC>|eEF@XwK#AZn3b6?=Mb?seQ(nDtOO z^;MR@SRpF?H%A?h!sYF`+n4@O`y%IQGeH7j`mTrO_$4M> z-9G&>1+`@vH~`9Ld0u&RA!pr|*bqq>LeTwUVEGh$=OmEVI6UMv-O#VL-6N<|c4;e@vkzbUx~*M{YL5~m2l=GWu*O@HpEG>`(oWd1%6O}OyOROXxR+= zyOFOs<74ac)__xw=ym>@GsF;6;@84~YgMwgLiPd%5|JxsZ2V;R9#?$N6Ng`ZmcIGm zz8a|!6E2`I<9>P4+?6WCGic9!KIPa*YUMG7A)FK@F$`jGbl(P*ep&=+@`+?J@Z7?W zOpGQ1WBJf8>d^$^1HOx}%YSpOsXB2Ybbd(>CKD4utcbR}CxuETy?mhi%c5|0pZC^D1GUnG|@t*nepp~27&~qix|g00$8{Wdf2N)aaD!_+EI*> z7e~1A3F%H`XTrF)--F8l0rIB1=y;UOtv}(PegMgXm5CioB<{A&0uWRZpRv(iS7`wT zQmG4!f*sW9Bb_I>T${juDch>vC6Si*$LQE?nGGZ%Za`4ho{YCT;2StX$n{*CI^$Zw zf6|>DC4M@eIcYAMME$zJo@&54K9=;Hb&p1q0M{}n`;m{TN<)+9beXYiunQA06)Q*~ zV1WEM&Jy?fKWPcjqcmZKDOgwgdnH+W&Li2=D0yyzXf&g?@Q%sv4-klW4bXK*XnQ0y zMcuug(iT^A)7?C~RKpZO= zrueRYyrEiU@zmheF;!AM5s&*F%(x_1tMyHlZOotWxk$>iFK1bk=IW)77zU6~y?8r4 zqae=m`s0r_`z_suB6kq1CnAW%*`-}IgmQ2kwz(WlTWj+%sYK^p7ffW-(wwPIvx5314Lnn?7RA8M zO8l;vq$3s}Olx9)lFQ)_vM@JO4;ZYj)A$aLmomq9e!$*F#QIYy+^0@U>i*zfFkR;R zq?yU%q{@b>@6al){w7pJeQebmgsW-b$V83qK9l~5T1b!J&j8Kf%cE>d7ojS1(ijsk z4>T{z>1^)e72n85zYgjqZ zoXWU^I1Vh3MrWN7I_9>KSG~@U+l*X&gZ=>AxI10xyD?veileon{o?5qG>GH1qEH}u zY8K3*>p&I6?%Dc0G8~GsK49@3pQyg{v`}eZ>qnx=q*lBj5m8L@)V_(sS*X6A6nhrC zx#LCJ=h?V}bC`~U}L2{NE%Q)VoP zKgGq*wvC-RH+J>&k+fHGWu3@B;YAE^_Wc?H5a+Un8vKc-Qf%A;A&pUb;hY z#szR~zIm+p0U!K~D!b1=U6n!ACCn*$Ol}^pOz*~z&mSkXLUu8#{I-X3fqi02i<($k zE$^CJL}X#w2_kZTSd&i92!#{pDRr4g0p&CSvy4v3cOTr3va_F#RsW#DZCmrku%7YJ zTB3Kqp}-I8EKf~bGIhhRd#d zGBes6)^q}Y52xALhie&J$^Mwx(_g&e*I4S^%Qc*v4o!m3ObK5#6_FGcyER*c*GU$6 zg%s-QmzaC5!6%DK|F%q=mya+edB24Z%ZRvhph{h{E^Ct%zbb`VheI#>SMxt_qz zsU!cxuD5%(EC~Hjwx0T%efVqpzuCUh>mej*`IudQ_^DGx!`l<$qxb0Cq%(M=`|kR) zjnN7-#s4w7?WXBNvZb|?EME9stCftNiy!GVbk~3|f<&V+i!fj|<7Q0@7}PGJG6jh1 zA{Kc^*N6GOJz@2C?zo)wOFp1HqgKIdMs4$=b=LPMIzlUU ztvvd2(?Ef09#rmzxjy8#q@IG!?BVw>Vs`WZ)2>&34>s!yY(h^ z(?Z;MGctmrh|&6kUn#-xT{>c-n?8(lk7UBVH2Ik(|C7{c6XdKhV}BxfG4=gA)zJUmNI_K zd9m85+#Av)g-kD!uZ?JnL~BK%W)+G~*tRsur?h{jcQCSROF4OL&kjYOC-4y$sORhK zcxf(GaRy%y36DEH)Y+RuSPm(s3Wu?Fnf*9eLLUF#d~H;Oi(@vA=?}QU7_nFY>Zy7n z+H5`eQ@#F-%B)_F|j~Xqwnaq~IEvrLDj3 zh&_pbf?YAEWP2aX(P^B^Mq*9Z^E2Xa@|b!vUBu)sv6dzv)=F3p5Jz@?LwH_HUn=Ev zm~repPOUyk(Qea{PX~6e=EF=U^Yti0uLY3yMAfhH_CHT^mvf;~nV?sAnjr7R!$@#t z_60QaZHb7|I6kAXF7z-Wh*#+L+%HN-?L0`MH*oxEI7Y?p#I*q zAuD@~#|0FaLDtag<_ySJ*8D53FnV|m=)?J}z0U_Kzoj3&zT&e!R{w55QvG|s&T(#d z&S+o0ee%ePnKZizxUxAnUp#4E{IFi*_P;K<6L7X&MvA73L3<0fNYTVk`T;N|U%0T| z@>bivy6jXA!~PWPCi)@UFh*D6Tv-5~_Fnl2;b#3dyx&O;AFG}HK5 zCZFQR{=neollp$)QRP?y*CEh3-d9=GcS&Dl*AZu#RObW$uM!|=mG}}Ul684Pt`zVH}uqT!bp2C@kzWtC4MO2Fzs2W|2LA@eZOZA*O2Y$S-0IVM zMeEoG&@~yzN2UKM@KEQvxu(^voMFAT5UDim7)`C=!tS;!Q(rS&X(1{HoibF5QB^C# zNQgn%+-TNO(6(B@xjD$T{j0Sa0RbxTtU0}8jA{ax{mM=q^53L4GK@DXKA9$o#dj)Cr8JIKt zJU&_4JnkdC-yc?-{PfaO+tU>i*>e^`DL0Bt_r^|`%N_Y4ZUQEq=T`R|WRN<|;Qp>O zvLx)VB(F~KtK`ZHAdki@Nd64#AkdA!?jc7lyLC~JVe3sWeVtj1uk)hH#k6P?1tn*X z`imf{D&aNaoUer6CJ#6ODk4eUKKD8x(T>7qt~OT^+;7D>j~~Vg_*Q$e^>Wt!S~_JN z;Q^IGhuYql#r7<=+P|d`4EmdLoQkz2NbC!?iHfSs75O)luzqn2yeF3az+{50Zjj&F zPo9mW5jN0wg$2G)O^xFq&A$#N{At`=X*4*ongZjRFk|f@&&E&7wW><8`EIF3AW+ho zcX`cm^p@uCkS@u$Yb}cGl^Jx4=ukgMx#4?G&-+cCSXX)=b2Zpw$ZKo=*}amDd1x7h5$JI`9+f^TWyOYlG(jES)FnjVEX$lT zbQg-Il%1|OEx7hfnQ)yxYOp=Jj}i436esBcmrYx%A%wX?z3~Pxk8kuh&OE{3bx^1O zM?7U88%_t_eRb-+KX|9{Oy069@{O1wF$R8(k2O!;2Jn#^$?w)^E2<@>KJ0Tr-WCj>7Lv3;ElJvZ^`H6k!h`?FNTLS=B*Qxnr;Rp!ERF*e#(ihf zI$+8)ns+Pbz;N`O#=P=wHXQE34c zq=eoCDHa3~sRAkrNL2z7iuA4m(os(rhUkA zVXXdE-m8Fzc9Dkh&ht0M(fC1gds;us$jjAAePT4Bi8iR+KFkhORc$XRDgJVT9%jF& zwS5ZlA>k3j6NXL9B=7iw|)j+Y&^C-XwV#s_Pw3nlcc$q1%D4dxKD_{X?Q0AIzp; z7JFqimM_dDxrC6UV~8j6p;8hGS%XASd`$wzAQG4NQCSSyRUq6<)1!+w01+?tq!~?f z*!~VZ% z+s`Nme~s1Ktrj@9Kqf*$V#dtc9M=XhvE+}qz}NZ;5IFtzAsJPqH#KR`db+7Ul|;H5 z)PLM@r3hWT{XH|bp7uZ>lvW-HJ1IAH2J!BEU3|{RgiIrgc7j98d8u6wS{d#}eAH4w z_>D!&?_Eq%Q~0?5kmX@SETx9_>Ft)ZgJ+KKCaOL1nrv&InRL2@G{1g2f`PsTfk^6S zjjmpl^sn}rDJMuPGXfc|rGs-9qT&1x$FP;okKX5rz*XFKLLz&`EjXp5P0a19a7Vu| z)il~hTAw!8mT7x;KH=rHk#oC(MP)|h2PsmGu3DcVN3zL2&(tV_A{@}2w07Vtd-wc2 z;g0$el=G?g(p`iUoKv;K;szs0!NTxIDM9ZvjU>FJb0 z5*D4d|4+w2%p+DqU;^mpA3|Guw6#qDfhD4Zu1fPCHeXX}NSi7mx;HY%G@KfnLa7Tp zY}v5V1FlL!w$_*D7~LLU`)(?2g?;gXqxd)0jK9j@a>`U~Lq7AEsB%J>~{7Xt%* zRVa)_-{+1q(FY)mHydIu?KXgwCEWO29Tfs@Wsn*|ETDz>z3B7hkMmy{?CraOpM97b zo;SJvQ!MGXdMIxyB6zODOpg%o4qg?%r!NFIye0Kt>pf`dv_8 zm1PsH3SDfF6Acd_AHcls`u~-L}-gd9wUa0P)khq72G+_=el*C4GKku@BY4N!l_+2;aq0kMZu?56s zOHPHZ8%&6Blo;=rPaYW@)6Qw>TrWe}p+8fq&D@{ugwzE6d>=Y)QK_WIU&%<`IsNmwWb^~e z=g^IF(lyeUXKcit<9&eR8##4-R!_vQZk;AFhrS*mH-?nlw{uq*Lxl_e+JYWO@zglA z$nIN~nca~L`;G?e3uSm(DX0PlY#_cLjdB&>?F8X!)6H>2FXfot32TYwh5nR}K%dpZ-ROVYwpeI-s^V+#fc9q` zp2Ft7NAgS2WBSwQ@ING}4Y9xhC?>>X%Lk$NrNi z;?sf_tpqUTi*W-$z@?Ss=0wy^iPZoh^DB^tD6l%uF80f?PHW>0Gb8O45hBMO+pB&j z2cJv)G<$55AepRtoo;?!*5LJ0%XPV%aUvK*-D>MR!e=6fnA!K2K54Zh;YMy1`m8AG zfOue+>7f)eh4+{b(Eh0V^4KnfbdT;u^B?z1Pkv>8Kg5B;`UYR;;3M%&TWHS5)2J!P^)TP?#-LaHVaq^#yOrv602+V24@rbB$gKBOd5k&T*7X}TB3~Wj;5>k0 zq;$^39zlBcZu2v6svj-2bFpYBpm{1asZ+25je;=;Xsb#0NbL$a0RsH$mT;*rdO{GQ zY$S10mvxliM-4&8oYXX3MqKif<+S^}^kbd9O@cVOJ)FNKbBVlf^zC9Jf$oNp2jb+l zK&z$R&Df-fX)E4jG+yD%)*Uh(n~5&Z6*c+WD)-h!V@)dzr|x{#r7@6iG%)QnrSbKe zcT)py_fXL`CgmyNPx)N&BNm#W&GkcXNTb6qFNs#~PYU$(f(N-Rbzc4F<{QLq{dV7@ z)<#e03IZq|IrUVDjhT2DLoi{Jrzm}Uy?pUAy_UIXhSYBF*Rpca<3E7ZQmdAX zGL#8EelE}Ls?awxm|SRXJ~!VQD$D=OA?g~q=|j{!H!ss=aqk&A`A+83OKf~&&j|Ju z`nadFFguvgMVHcJiY6*DmrEYrzycY>QyroAJrgk|b&6q&ZGUD0rx2&u`%2x~HQ4!( zM)rP_N27*3=wM}DyJ9}!B8)|ry{BvQmwjAWZYV*nBUybN-%=mt$UB5O?+&pFIk5;E z6lnL#oV%$K^b47Z7wn@f~ zhsC_Rg@00H`TXYR-)>@p7o`<@oeRECQR6>pQDQkC5;kb*ob;4#{HM3OF&WZ4;wgcn7|gEeBf=M;SqmM8KNuW4Eh@=1e&?6nc4JH`72`d~ervd(@%TMub#g@BziStfRO^w- z!3%x2Xr{RlrcE*)ZmDgSsAMCszGs{s&?)TWVpmuRrq&SBs*252TIQbN;>Y{^5(O7>`WHk zm{C8ngAfPp_2iEd;eOB}E{i&X+Od=fzqm|~-zOubCNkV*DMs1rVw0#Zau|~z6-h&3 zdq$X)bauy&zC%&$%hAsySDy!fGkHA9bT8J@zr7y8x8e_sB}~keN1HaKh6i+5ch-Im z#Gaz4ydr)2qg&r>CAgXvkrC?WfOWX#VhD4ny}aMN)shY^Z)G4*Zp(~NFWI;}8{l-d zm}wCaI!!BRBD~S@p!7!bn$|x=kn4MVbNo_`1Fi_qO&{`vu9A4}k#zjZ+TTJ4G?VTa zZN{ay;a<12|62>N{RNP%U}x2Bt?QQxiF>j(4>0G@hpy2e{Job~krID0RiV1lfb4~u zfRiXeB^rgrZ^j@-LS^aROS=WNTrtn66PWJ3toW$)taI<`A&+kiZV>`}Ujw zlL)goP&#JwYIg>KCWw#Y!+~x-SYe8hwRIK7aQFP_-GX@agtjqK$+;fKIpnUz@ZI*^ z!Vy*}(T$NaCv6kOZXX@H_b|25;MoPJR7$D{(V^TA)qn#^5Ph*_9Dgfn5$e$%p+g<< zR{EUhY&6GUQF7yB(Yz0&$+$3%_62!RAs>#sYyiRKh*{Wr$qUq2pm%}jaU2zXE7*7` z`zO*>Eo1~Df4O2IOIZuj<*ih4CNDpHe2{NYsLjb+tIozRrgo1kDnq1z27e>z_Rxz5 z0rjcTI%QVoUL=y0SY8_}wGZE|hIr}#qt){r4c-a36fz;UF;0l}s@4iKqr{bqO&Pgu z_L7E~f+26S?Q=gagtz@<7N4<$M^S0t&)3MkSjP?c`*0JPFgu3#n!l;{8O;kP$2mUF z=*$at@5xa=85j{H8UGNH)lxbW+jkb_*&Gpv;0-M1);q!@jq8J;vV3Qus`uw1Dht>^ z1@{+x<|m15Ahg*MLK{prI9N#@+<|XMIWW)2h_#@~%x`&P=B6VyT{&b(W31i>#z0y^c?A zDL*`>xRK+FF>vsJ*W@2zaTds>ADqlT15`p8t$_ltcgY zJ#f?WY{*U!ZpM8ypwX_rHliZjDlKNEtocK@a`kM!MC`~09lJHe3Q4{e7-ikQ$BNuamC44>WVBfD}DU4w;TK%qmbJg%G&nLR7XF8|&}at0#D8JLk*=M*22!5OV~;7dU=Fk7*4WDMY$(DXXhI`i&=zOXK0KR zo1`$%yw7{&vRJrMM#I&FRvIzx8%!Bqe$q4hn?os&U3n7aT-_BKNt^PzpLJ!rC+(a( zr)~`wBZG4uicyMFwNls#dRjw610P#j}PzMwIeGJ8X z6=So#ce*+5jN&2=s;raJ5}d2!dTD6c?zL2G;(%_obzrUQ#o7949O7sbjTRJjuUbMs! zX*y%V|1HJjSIKUqSLj0k{hc_NzQ}i^2PdZ0Au%!a6to9t+tf zCS6eS2ZNVe?ft?ZQhJEBsLR3g6Qq%*TH*AK&rW7cxIYkOw7kX29J}eOFpN3tEOPjQ z58DUNd(FN>qFg+~ z(Gng)si((epO8*L*{Ex-+m+1s!>;Fd&TgjP+rYxdu7%sHHdo5*)7gu1ioCn4{eKmm zaNy$ii_zK;$)ewMNW)HSFIlqp^f-R9PfHI)@0|AQH40Di!F%JgHg zVZV&QAkG=K0R&gr24T+N()c4+AO4aqzSutrmBq^um4UxrrP=YyX}O$%i8uVN;l7n9$p{1V5`QCekes&%%3X?Lq}=OvL!Xh z+3vorhSxDDF=$kM{HZ)L1Qq;y_3q_^RNVTSR2t1Hx?fl+S*5U)^gE$meHPXx@Mf&4 z*%>ZkpSTdw?(L4RKP!gF0blSKZTk2k3w&;V_RyEpNM$4{__l9UktmgjxjYry2F>Kd z2j%JTCZ~T+ku23CV@;}hpUs~OB0^LL;7ZI`vrOL$z~_cOoh^Ebc1FID=_lrl0bIX5 zTpeSZk-@n4xg)d{NnT}MD2xbgS%h2 zC;@(4BQh0nsR{2K;Z5srH;Ue(@5mAvMD?vAmcKtYxd8voT8g(;|D;KWvq(@4oq!)% zy>0eNj)F31jDf-^GAaZ;WuPfQh-ZW!Ta0|&-I2oMT({vG9NcED;7utwH4iiV7HcCo z_)ztUyRuAhu}vTQ%HT~HqlW(tnZnp`p!u5Fad>e+gkOkCBuRtv2Kk7Bt|qEH1x{z_ zmwC_@3kO8xKXQWKHq|(Q#Za<#_`zF4qhUq2PQnW#)>#Tx400ST9H*@dC)3S-S~z8< zNo6OhgVGh!qhjbaf(d1ZTW1RgJbG{EFGmS4#3`(+Fhoz7Zx+}3qBA0xoA74-qT>;B zDmy35e~x6^I1u{j5?t;7+#zvIxK${jpPV8XQ~3D!{MlE>QQGij8mqR&w8u$Tm|%V$ z8p7Xto+>Bce)+(1NRz!aQj3C_Por=i(MB~P(~|Y4$TC`U21X<6V!vq86XGfVwRoaK z;I_b=^?iR)okjsK*g}eJqBNC}k5|}zImn7uhc`d^m+?puU*M3b6M?@u8fOo~N&dSU z*V%+%SyCnWGP0s2Vc7HcvvcB9`I16J!7W)!YVp5Q_TLpF0c-OuEE?mq`h^Qr$P0?{ zYr<6HaEArEgU>wE6-WLB3F$bA#L9)P6Sj#sBk<*e97(E3nh& z`j`i1@PGfo5lIa_uVpPF;P(FSKOtY3{ihY5GT`R?@4qB3xJ1PgmSNnff`R9s=FWEOhxk@9o!A#ZC)*Q@y7#L<(bJr&a|JspOFy{3e+&_^G?0?Mu z-yw0>@$18hKmFiOaoE4=Tatg?{yX#tYS;Z&=ptYP<=F7q z(tn4pB16yHPusyz6XLl@XW4%bf!}UN ztp(?qhp%fMZN8lQk7aFefj^H3ft7ATcK3Pp;@_~5fh{-uyDv_0fW^(4yLQC#uf+u? z0;Yjj$_`s_Aj`9$zb9Y*|M)QaQA0f`l(87Zy!-%X*UKJT0S=Ahk*3XEzbi{vv0ho&#ldUc@82DexEH2hM*Pl zXT9;4Ff)Ecg(L9K+Q3ziL|ZxvG+rO$(eDr?YG5aM1Ltn z)S-DEIv$prK?RQ`H)!zkv*$4_u0SDtJ1^oppb=N=|NPGUkfGQU-Um_F3%iTDz9;&-1L_z1pJ&2d4VJwJ8Sx+M zH=$6OL9FfRFvOKkfMXu18Qt6Lxplt7{A-oRq6h)h1Ajxv^tIo{<0>FqD`UOC0blyO zyR%~E`eE?Wx7THR*Pz4EEgPI+F!SWpqjJ) zi0|2!upW1<#nD<@x8-dBEWQGBRtwb{SI#gHtA5x{uJ7CK-o;s6Z+9G6Ox4c6awESm%Y#5Z(Cw=q8 zczSq`x|6+@V1hT6a&PUv={b*Dj0qqhqFn)tH?Cd&QuKX&d`JUiWp~Ex2ul$flAj}7 z-8abNyUYb2({|MC8z=;wkeu`gXlr+?SZ9EWcJBKMV+|ZL%(bA#dIL4f>iQ{j6~FnJ z#dB?vL5C*(%mjuowX(8Akk zZfuWB9Mq!YcZLXh2@?ev$yZ>NBuzV@;`omB=>-YBoP&%gbg`cNs}ONrdOf;JouGd3nr%GGUeIxpQ|S?=9&B&0L9lMa`_- z0C$gBX@l)96m4uk)2Rc8khx&sJdW!J2TJkR43x&`o~IEYgwF%YWGf|FHySXxPv^Wv zNzL-{$*<=<;LGvMn9HCbka${BgJ_q8V|$?|YC3Tkv6HV&H@ z-6p+A#5uL?Fj6}0Z$4^aH({v!{m!YHI?o^6J3ycJ9O=7lS9JG6eFFl`-EIb#>ugK; zDN!5n{t%$2O3xJa_tC2BL+ikUWPgz!x^)NX6={9~xp4tC=r?>-blv247D<_88)DHk z_((&xdR`$DAM}>jFHV(#3B7+C9;#wkg7~9A8fL|{rmU+x4LR4unnO5LK(~{Q4;!qP zIZ!eG3Ti$iCuXliTz5drrZ6$LSZOT~**93x&l?N^@Z0khvVNE?Eh7mst7*W|18C=P zS5nzjb9_~FsCn~>$-J8OK8v0|QiMAh3M^{FJ9ckr@S!X()_`$!CkJ8rg(h}?+1H^- z`HwAf*as*Gka+<4G3Z9H^s|7aI9E`)~)I zJGxFsEnFpj8s|`t(l@5J&SGxDo)SoZqC#d_L(JR93d^_S#mnQxsUkAC88Q>#yu zLG0bGHmP}`w3*qvnVp(8tk8S9TXqY1!Xsuc@gIUXH6EpXh|D?{@FFL8>(cs5I`NDK z118pcI$9>Uk=TLQrx$~{qR!zRqqu4;0}f^Aw^d+YJhUNAjx@zWs}$Fk%-;6c-djyw zl4uOMR75CDJAf`gaJbmf(W75`pF!tkIkT;bP-YAB*v5M3-caVH<7UrGJH@z?wf%7i;RuXp&>14AewB zpdDv2=NiKWhZ5HCt*Nt-JE-7tTvL}erj2&HLD2+@Z5x5Dg!j$3OurcIXNYQ_AfdIx_^(Ja^9lML-9Hfxt}da|99liw>3hvj3Tn&rs5FLb%>QeT_isd*;-#bqZ-VSQ z;c|LtRkSBVxGK%!Zm)qLe{`cOk21Ms()x#vE}x{pFwoS0(V=xTE;e9UN_?UvM(e;i z%enEH+RI>DDqsiwMU$kl)WW_X!`!62%=_>u!w%^j;bF!H&7CC&IR-%#K8s#7M#kiq z=rw90|Ms>nmBt0zRP~qG+X0bbO;!wQ6L7>JF0@n|`^gn6k0vl^OTCq6C#DG`o1Ry3 zVtpWs*$5AZ7H~#}ru5Jy%$oDxMxf`BGfNv{l@b8QAsP_qT;5I&?D$O(1Q3gf?kU!01 zUAhO-n4NHK47C|{zaveV)}C0=6$^po(=@@Dm4ry1=0t8T?I$y$$JK~ESO&Ep*i+LW zn^nQAHiz|#y5^Y9S_eS>TIox-M7Ln7dLbt<4O6TJ^y@B{#kt;@W{{J);HRr5Fb9O? zZCni#n-}-dC@MDoE?wG+<)&$)oEACC8j;ofm7(NBz`ni0Mb^~fV|hKPOAazjWfQBs zx0R@JZ;F!H?KfV8EOriyqS>Ibo&Pn@YV1n8KIvc#|o;ZDNytV6_xX*Cpnon*_we;pM#`lid% zUbB=o#8+Gk%S9{QhN|$vdSV-}g3(*XRP|CGN3y}}Vf3l;xi$iXt&49{GZ{DH|UtOWeQgj6jbvdSZ$h*Tf2N!kwSam z7otw1*l;^=2|C=;+EaVvRaVjLCuE1h?tnV0)ku=wYOg87!1ROTPf&sc&MCIs`^EU^ zh3Jr}$@l6{<@7PCEHpV)r#HpeHUe0zrP>$tzX?*)PfA-EYwn3`k=;Zk&|X@aV$ZG= zv3M6z=(D&EiAh}LNnpcRaLm%1c;_&bzB`{&<@xNz>jwE!Ju&<4JQV^i~pZgU5> z`U%)d>AGp|A3-5iYUwzWqdAjc%?P1a*oV8dZ!&2OI(##iSD2C1J(hm}82+8aM)7Be zJ-*|H9n2L6Ys@x}sqnEm$)dfOMqf30Yt}2s9!neC65@*+oBT5E`n78w<8V(4eXeFV zrNL}1zHkKzYL5%uq0%VMzWc;BfRTP0?fA(%nfO(jDnpZknAq5=y?0)=t5iO){Gyqo5c67~FdJQ2x;>=t zJVGwCQO{j}hg(;f90FvZU$iI@0`a#9%9!a!e<2aWqAg;@*zg(uX`pwQ;-38(pF`jX z?XhQ&JPii+jx;(_lfJd`7@N(^zP2DJ_KWCo2rVB9vXcwQKTMor3U zuuT~Vd4YH)zpJ|!H%EDNN`1(m(V@)j=NdzG$wA9Q?~?v}nzT*pDn4%}u7(QrXEORS zNQ`_g6DA`cW?6n$ZunZtv#?)&2Wr|2)^k}BzW-_MujQz;zQ|~N-Kv!QO6+*B@Ata- z-+IFVJ>_RL)|_p60%8@+OKp&gX3YzP8*@j3XwE=rQ8n`4m?@(;Vo9oKrcqTT z3){Z+dAhjogcd24c(1m4h?1JP2GQb{AifSuJHJ*eLKTfJ9n3gSDdxiP4qUnU?HbXS zgNj+TQ$u^1xOeqKTv@dfL%dm2u2BNlaq_`9DX{VLt~kKg3vND>5au{B(G^(M3j3=(>?WPq4w0AUQ#Ym|y7Ec1B+=`U zQk^4LjwN=z5F=}yHzK(;J6GBLvPG;57wdOyB!>%|(dG8n4uRhWhnjV%a^1__AHDCk5!-$S<%Xc1|IH`h*huM!@){ zx}a%FZ_*_N@=Fo2(QHDgw24oTwude1JIu3kA%ZvX*mAU(pd(eh%#cY4xL*)&l=T?Drg zb+E9E{)4bsXp8wyCX1AyMq&IWh47C}?6}cBM{@Lm_)%8@Giln87&NdsO9EAhSI;@- ziDg#BJ8r~!zhCEg)o;F>R!Rq^vL(fwZiwaet~FwJAQw%e?u>Gz!B@l$U#-fHME66h z%fJ=+6&*KIsOJ3>q^B0FK)f~LJ1_(!WWN>_n{K7ErmOf~F6)auI{-bN0fVghmv=I) z`$=ya-ZUvTzLo>_)rY~@!&s?VX~=m=p0G8Bhn$@rdpdPPKE-d_KLcn{;p6kWT09#cETt+N|FN}(q-)p(vO zZOB|Q*bvTF!0v0cuq?J>ttu&&5t6HGIIE2{{)sG{SwfuS< z0>7LxJ=V&RGyrl~vWsDE#)M0hhKcqAuxf@Mz0`RkMa}5nH)=u_}vE_H_di za@vu59IbaXq}-W3_x`MJU$X~KdKVy|>58=QR#XJPJ9&=1Fm>ViS;muRo8sLDfc1e8 zL~p}canBx_o_}Nb{k}LX-7rtLXeAlBS}+BxT$(!v^Os>LE2Je{z4t72Sii@D)ME_dSyR4>FEyTNU|frww2l3J{E&K^^h2 zkSwS#GF0;Zv;z!jBgnXzJiqMUp%G-LdhaxX98AIMRK-5lzn38_0x8ad?$f7Wvs;Et z{tKe;-$PEp$GnTGzgU6LJk>E_G6Z^o+cOUm2`raViueHk$@z9%57l=fq;|Hmq7=x( zdg?W=MF4qq;r7wD7ozBszOR8&ZWSOvx(Wj>Hr_)ldr!x?j{?S132?XQ$^^6~+S^=vbFq8hlk?ZeoszlkP?uN*+>YHi zHIR*$?FW!XF;msQ#CLIYq;?BJ$(%oc_2{<^!;$i{8_@zipe;~Qy;wJl0AbH;sut>9 z9>Vop1k#6?E_2)iGR*syI<&WrXWR#N+&k|_?vz~>l(!EF+Wem8_u&ih;mRipcYAWY6b>Js%O983d47jb`h$*W`SPv1}P&{%ULm z%?3VOtFy9mn@J)aZ+&J{N?9)4oE7J~l4WqypDF4@=FvADd%&V+J19Ic_xb6h$ihb* zmRG<6bJwzduI`a$h*vwd@ySE^ZnUvILHa_?;C&^J1;0tW)9Zv)ZW`Gyy^)&0Q8 z@+PC9y*UB%fb0(fhVfXOh{G*}C4B+c`d;5bg`a2#+Ch7pV8deq<~`ng=d>wzj&65+ zHpNT?$bG~$7X3aSqf-kQHCXNUOfMw#1s!v}-av`nGiaODZE5Azh?@pjNVnVR8*KXT zfAr+$o+dekv+WV!1_cJ8_w2Jk{7=+hMcmlt{s*dD&n%>Y8yfMP5WKfX8TY_QZ5JMm zbn>6`Cr8AIb|)YD3c|D6CxTHSA_P}8FcCfos9OM5=Nh>eJBVXhAfaufoqw z+*2cLc-mp~9rB06>DhP2T26ry$xyfQWZbQ_FM!JjwUHnVuu>|dYmIEM&4nqdC~`lG zfZ8DXOFpR)md+n_KG()jntsqKQrZbU`9Oc)HsGM~*r|X?$6AHH`hP9HkMsn!MHXT739%`v5W^ zNI#v50kU1_b+))xv&8P9=B}G;4@8lPjjqLXRDz;hu7(Bc-mFRc=T6h_Xpi?bf+?{^OnRU zxDTLq^R&YFl{j3l1v$mcCFT)$coOEl)X>R$o3x{8jJ|X2KfnV zVOWn6-I_k#DK0gSvpVV5+sOOgcc_h7PTDb{Nn{wPB-+)VVY!K}M}}UlZF^!G@|^wo z`h(^BH!`wyF&pjt{#m@)0rRY15wS$MniEHZLzlXkPhju0^g3aM$n`Fg ztVgRmM|;G-82)?7fxYVseYzc^sYZn+{H8S^bG*V-Br0(bUqD{iJOH-Y2-iYI*!s$2 zA;|MvOS_!;2m(^~#LlMS0#5Lw*gkK*LDIPc1|eVNsK$}gsApJ zVbjcq{TG9jW;rZ6yfXVI!E3JwfAU~pnK&Rv{T8s?{u@t1mKdqlx*h|kF!inC`MY}y zvHYzo8l;~JZ*a8`>AW2pMDFy-e0PB)cFp1E`<>ra8t|O|0;~30hGi%?HK_f}_1Fe@ z9;Y*|7@3|)PwC#+Y5Jx#ROWm=82peF{(Jpnd*Eo-#k(e@%ifKxpxcIg*>uK}18Oha zaXjw)JV7VQSyBmcSEUq%n-?L;IhFD9sRtwsMQoM|oAM9$2xltQbiS?uMwuI#_StsP z!5V3&v&H|lUZ5CHYZvx}DllSd4pfIy?2`=v_QBZeTz4x(6H@5QdV(z&wtpyH>T9V; zzUM6b=|BMnXI19dsy((fsIn^$kcRRe*^c!8jwjt!!3M4k7FCSajY4Hzt62u5My>s(vc&z5j1_S5R)_YU3Sa zOjLrGgUxB@^{DA(e{O;O2|^xKl+?6?z(Y2tXb7HDwbye*!w%W@%-+6o?qnM-bqG_3 z6z}c!JKBCfK(z3twsJ~Ca$J;k$s^;mx9i(*Pz6-jZkF-+gQ)FBJ^Pnw;3WQyGKjl@ z1(mhd+F2T$GKn%4=+!&%dpq7QjLif3OWyq8@nBq&+?Fd)e{0W|q|8K2^wQRnpKN>q zmDb*^ayn#AQ7+erA*0Girv)UG6-6(E>znP*E_e4IIl9uv=hEX?&JNjov>17kWLp$ zEY3yWnf>&*`iX&*uqH;N^Brvbw&vft3fQwUa~mK5TrGJ_`|} zdv&`@7$ZaGz7=;2rTQn!gd})#CSN~*b|Ss2BK$B);9{FpMsjD;4)|CB{mvz4L8XC( z?kjNI%#*@c{D^gsp?NDLuDCMO`(Wht2E3@~@_{W>Ajs28|`{Yx>uyaCoY zO<*5#1-hqQE|TtAKJ(}JpTe(~NMNX|%#GeS>Kv7Kzkl@V9Cs1!Nhd{Ie&|NLtXi6L zF>`q%Z9T#c$}7DYG2>~XMojDtn<>Prt*_GcTr;{0nj(z}$`k0HKSUQ z=tWJRYC`?e;RJA5E*TUaBrlD!FE`WzHG9%3UPVZi9z`YCvk6vuR^WO~+Z|`vjc9e1 z91egKXZh92p|)FX58eleo2sQ>6ge6QU`sz}w0D$!I}~+{WkwhTAHxmI<$2OOc@$kt z#6HKBD!zm0(zb)v7eaZQ6x$b9Xxsrmw`xcY1jwbz|Dl`KhSS<1ZvL8}&-(4n2n zqxR`DO^?b&xMoNeDT?a4R*m=#&vQOyAfOI9C}7WG^~J3*9gs|T$!m+fK6D!Efh~%w zfbOKxg}Uu)O6TZn9SPz$!Lq3S+p@q;#3g%v=G3>AM!E?3Fz9pC?l}7un?Xo32KGpEMl8iv-Or`> za=1tI4hzLKE`2+c)366V5qp2+l_hDMw^pE4S2q*4A_I*~8I_<@bSaOnqj~z%JQ-qo zyH1xob$i>r(=>R+AndMm&NTjfW=8mg>^F9=#HBOPjWbba`}IX4UG7JzHn|9HaVE=P zX6>Yj+T2eUGL*UQG{_z3dyBxImsru+4eig`PkTb{#yeqMX7Fwph9BS8B7KU=3{RC6 zinRm%z`g2|kB(Qa_1hS1Nt_`Hj=X{Hy8OtWn9D>%3wjnx6uv2E!@EVPNg>Kc^88~6 zq>}ZX1t{W{L$QQ7BgY=rCL&#KE_XfD5c&1E1`FE~8(eGJZ{B35C)5-Nv_6PWn#qAB zks)k+I(OzuLzpi*yo1&9^v9I0FtbeDSNf45$5{yS4rH%F-@7$XSp*d;=1>!RyeZ%7g4bA7jtVwP z?hk@Z@_4M~3<^JfG{7QSUEdi|)LnSm62{}68oeGh=){oE`d{bj7xhH&5(isiYzN6{ zf3=NiFhEzlOz6TwoKRFoFtiSc#PX+(TG2Vv5NEIil)9o(+7Y*WeC30HC8(B;+HY$Z8$H1O&~$sY<20z(;lh6E))!vf5@ zb`5Hoj1>=ykEwuL93+Ew$~?mKkIUooZlu(%427PL0n<4``P#Yj|FUK#M_?b2qmgWKe^ zexcqVMtvc_Op_)z&~6=P+{eule(N7Uo7wvF2`O3!@iWo}2%IO8)Ux}HuXz^nc)B()?!SG__G>`y42xu<+HTyjF)I!N9gv@+tj|kWZ9rgliR~(z_!*9?iMLB%vKgisH&uQ50qKxN^?r zL%?Xj(ToqGI}!W=>9+aIYH3|}Pz5)4cQ!2{rhB2ZRv>_-*&%$?qTBB-|Im|zmd00y zKuhx=DI?nbP!|PGf0I`yxm4yq&ZFtSXY#+!uE@-)yYW*>=&E<{E%#GF{zRs%9+Kt%l3yo_- z9`&Z#D#|!IFNKrFX=L21U%k!kt?iae#>O!Bc0y`GLW)h?aHdcm#?@vP}d(j?#Ij=?0 z3ov31ki#)1Z3lnfyl5mdZy%#x_qrtW$%Wo^F>bcbhu2S#9*3!JH>KX?5`8Mh*g1)b zh-w5NiF2zT`nHRwbECRHs=9p?8e2z@?|sY0K4)flACIHvOiEC{cO)Gy0Bk^Dxe<5Q z{gCrdiD-KOe$=h8w>q9CnUL&m&OXYC$ilafuMU)CuY-2T8X#d;F;D6ctm!z!Z+n1a ztc4%m;lBfNzk!@KY$Hk7C|xb2q*9VB%>5p!jJTFC$tTzrxrfA{7|{bV)TNRoO@S?u zx977E!1u-HmT2~?uuVcaq?Dcaha;5$eD|2gPVckO5*}jg4657P4eWW+X|VI;LBjHw zULd(sHfA7UYsJCXvf~@KGxZ3FCVX4r65B=^{?(f$qlt!S-2!f`xnZfU>XY{XLHvqN zfVkhuQ{)nc{;uJ?&JB?EorjocxkZ;FML2osPG5l@zK zO?k-e4JWtV^oPu8rxbhym!jS(yZ9*98uCej_1s+0TfOYWZ7g>CxTWA$?5-Wi@bM;y zt!n3SwuFS!Wf}UkTA|ynlusX;F^i*47iPwpO8wbd^QLC;kUxI|@6wY~JA{E9h`RAZ zMJQMHLmMc2GV|Iec+2ngS^1R))hVdBj8*e!^_x6wm%h%Ik%42r(cZ?{CLKMDWmlW_ z{2pX{>ywAr4aPRSQwbnWyV8<@bXk@6&z_E2s$A}b5!CixWMy!{y@-m zGZ0E~uPZrr5Z=AQs6`nn1S&i-9nX0g%~XQa5XRmObw;u!i#z%}?mQYHM) z-T54>T2yo&ndC$eKFK=oc|w9r-OS;&14WIm9XP_{8mRGc!fo>O$OT$GqgA$~0?3VYO4ZzYp~i5IH2W67miuwIF%grg@IRB21IP1 zJv7Tf%H{Hr;wx9L!o@|XD7>Xa+aujW$4oKWO!P`@aQ$%`*Ah@@kCso>sa(=sBsEo9 zP}P^V_|$;OsYLuH$=my19HH6?p=S2}o=grq1Cwrl!1>*>ZOc%ConsjssrkPt84fd!2W{aFtosuoj8~|69GE zbQJ-gmQJoDw^x{0Q3+EABw@6pZnRL3OA^Tq*oIV&fT;8mN$JSY^@K#}3+sb47orRj zTB#$_oxbKKTA7m{E(f+8{PYn7bk0oq6JBE0ALj!x)1bH~rv#-=~yZ>Pv} zp9+NtbY?h#ZIX)gwH8kY36V;n!i9XV~Gv4fLTtgN>+WF%@RD zjE2R*vImI+d7kl59nRxm6VUab`3&}bSBiWBrgBxWKniBEBpHgS{<#bJ=c4DDYjJrQ zZ}QYvj|rz*{=HoNygv1SOJRxFJHVm|zhk`Q zFMocL^7`V}g~siOb?jtXQy^8?-P{{2!h?w8*zqS_TwFH^*S$z@AbOgue@DN-V$r{rFz8 zInUSqHp;Y<6ytH|#;~?H?UxO4_(&G{Um~9+>O&j7o9HE9bJYV;QnIad`f*8jiGDPF z32{)uIhrNFz3~(sBZK08unWI$i8_E;BI+7 z8UL_842fohrPFsiYjd1Mx_Tt7)n4o1VL#9>5vcuOa$o=R6syNkS+mf5d>)rknZkAN>mzQ z=!T&izI*HYuJ8Muv)1|Ztn zXmdFqaAPqd)$=0F+6Ilve%pDX1TDDTZ}HtkgK>KwLNQoK?4yRE3h|otvs}emQ>k>T z#PH}FMF!-lII^y5c?3L4d}ad8JzBvYYd5)N_L08(zVPZ@boZ@80h>Lar)Xv>$LfHb zna7SIdrzs%)Jdhs33>u8V#I5;L$RiJ4N|<;Jnytb+^lOyPFV%2ZZS}B=Gb4TUKl!- z+jn3#-A9h~qbHTAl&2$?65_-*Px3WikhddD*i$r;ysM2a@fS4uOd)dEQ8n+acocaH z^NUcb!bg&ogn~9tZ^cf=173|{lTKnJ)`3+4CJKup93A-vC<-nrluNyqKbzSlb}pfP zFFs!4d3NU5LC)rLu5#`oZue+*+nP}#xG13^dDm~U-MFx8pu#!g0{7W}qj&v(KlN01 zq)@({D<@S7zC%=4xEtGK2!n(B?tk$IiBSj^u=Pa8>7O?q9C+hmx; z@eG~xwrMbOGgTWn<*CKt2xHbyW_D))kkMg70GXC?oEE&>PfyiHH3hij?&VIN;qHon z8{&&&hE?e}Vd!m@XpT2)ylZvxVQl5Zp4Yc+qSw8XjJHoEUz2ZS6HdF9d-?`i zfT@C9W~**M4*xvw?|5^{39s2$=X)&%k60!q1ANW6sqxRTbQsJ=0(xWq%b~bD@DBkI zp4S~tI*L!XG|`Jh?*f(;E$am9+S|uaLTBeF{BEiK)^A&B3&Qb=(PF~K4g7L?Qk8${ zCaVLBOOQYi``L3wD2f+NnXozwa*?Xk-$-N08+8QL=_WVf1)*u`5W^!pHUSjp*(@W$ zdvHSHVxIG@r0~5a47ekR&*vL+ zIzha$CX;x6q^!WUZFSEmD#)fY=(^P8GH8M9IH<4TSP3P(bSO1)@Rcx_6MI|#7Aq~e3a+3Ul7lN7 z=8WSwq-ZNSgtqBf>Ry=Cc@3MuE-!e`U+q6?YYR@`|0D)8B}@Ek#oi-Y>v!Xs;AaRL zT=@Q{agOAF9>Rqm?|EcvgIx9f1^Aw_DJ%9LFYLW2@|!C-fArzTEQMZ$R?JWrzDK&T zPtc8-J$RKS49jE_qJO4KT6hQF!@n3%uFKjUC|?0}^>sw>zG?mYA@U>`C-^{f7iIT8 z9fY)D!tl`ho_WQ85;EnxH z^_s_jWX)=b;P~5$1|m6!Xa7faj*^7e$=%5v;{MAG`0uRZ|Kq=mOuLb*8oY9z((2mX zP2RWKoa@vAQ#v^728AR%iz7hJ1$8(w#-Y$_;mW@KwtZ)Q_R%;#PZ=e(ZVe}qM`ka z9RBLqh41WP*N`4Qt)TY-MeAK?;O{c&|8FH67N|0%N8srFwB~UAW&m)Vw|F{d*#rc- zzr$E0iB}4Mo^9|{zKNmrk0JxfIEK;nv3SX_Uf{DIDqcJhDd+X6Qw>=y_mfw!UYkeY6UWg z478qM@Xj=rT*|%XxkzP&A!iWnNO--#bFu|+g&}<0MOno|>DOJ9K~E0_>&qyZX_Y|X zVkK(U#c0(JlW9wcb=ZoDgtB-Trbd?F_MI?BZs^{2{)3}CtSg--Rxm^^2E(2`7?WJ- zC8~fzqIk^hU=SK0D;NQ9I7?$|@idAp20wqev@;Q?VK{f^VufR#<@1q&zCSj8mPSDq zLnbstq`5DtxHLzBX7DxsRq=WRm*q^|m~}I~#&$t>W`|kgU~_`BC6kEslkb|5Ob)rh zDojr_KD~)_pQ0EpJXk5-&TF}3l^Wk6YLWZ)=s>%yE#iooAymA%?`T%mbo#s2G#*** z?Sm!bX^3h$97=S1{$#eRsOVSf@K@>W-1SgtQyS%vLg6tNE0&jb zhRih#u?81`&=B<~BPy$AigMU`-1oSGw_@da?Dn7jb%#H_-_@fH7|10%`lJtYr!Di8 z>i2i;C)b2-Y9@Zh*n6NFfhp7<1|Skj((}&gLj-D$#X254`)^LqU1QnW656 z=MGSv!_c$TL6|@3T5vSOjb{c!7y(KapMC6j!!kl5f~xe2(d9a_PxEj_cN`G*j9Ujs zsMG{|te58+sMC(xy%W>2m7LP(Wq(;zqoyWOtL&$SfX>tDuWCg%{!|RVVV1UlD^X*uawD@N(ftJ%cH{Z)T>DplrWHy? z3c81e%o6f~Er%aRmfLs2XyuGIq1NchQR2g;;eJt_mjPJ};y=HKGMUpHZ02oosC>-Q zHK)lgYwnZTO|o1X>g8pz3)70^<=?-kV;`C2x~`frP*IY_a=BHM>81a}7lS`V?UtW2 z>~9yng?S-i{l|X21C_@9Ec4jy;clR4SWE*0>U$`|Y)!+dw4X_yk$Y<@+U`v>ri=aB zzT*u?rCZnDaAvBr%RDjbR?|#O*-Y)~yp3RmgR9RWVvExA2Z@W%ED%o2!G!-C+om`4 zix3XgR5eD8FI-QBqbemUa;mZCz924E3Ro%)a9aw|ZjFR`iS5In<*j4!UX6)Yc`oxI zRQTonuc4uI>(0n7|6YBtB1u~>-%2um5y8%e;g(&kuYnBmWB0a&h_P!}AmI*p5;4!! zCuoK7VZR^eQ2Ua{2{HPo&NeXU3O#tfHMw<TH!VScTQSfz{QRP}h;#8+&4TD~DMjA)T1%%!cA>C& zpQawHZKi8R8Ts8Z{FVLsPG3$R-u7cMWdLb&aNtAPFr@b>m*Byp8ECTF`1v+yd4Qs(`Dmccket&jlX0SDs9oZ zn8S^nCpzMQh5PT1f0vaZW+-9@x>;vEb=c95JwxwsM;!p0(m~xBO7ufm2>TmeUT{p; z3X*`yu_SO?g2xC_&Qhqio=H9jmfNYF3DU#Yu1rKC=LZ+z*R5-LpFW{Iw+0C+{W8b9 z#cq6LHxMVQ+wRSbUguUNn2+;WT&I{1^WiQmw&B3C2ijBYGK5O~6un~g0&c&8haj;{aX0?J zuQ5}xnkkk1)HAQ>8I?v#mg@q~@IZEar1SiF@}s(^thb&T`;iq#DRn)}IEO21<;OG< zmi%Dwl|FfAoulf5L`4_g1Y5QGea=O7kQW(2)n|SSN-Cdtxu*m&DH{m84Uv@t{$z&K zuGNkAZ(Sv*kN0|Y0%W|9Mhw+kD#wYnOZx6dTgYZw-r@iZB!m!mWPa-wj7X(-{{l#C z#{qsSoP>FUZTR+v1TtG`x)sGGp$fY{0N~d7D+tqL35s((ujTsgjEm7E$iW;Tg{Yp$ zliaM{->4n#F>~YQpa9g;z!*2>Ze3m;$hbvGubE1E)oZ51Q2!)4y*9^sqm!AvU}pM* z<*#JFa_;IS=heYxD)mB=e*KeBBKH-Y_KZi}Rnv~0pI-G0S}%MnhS{r>M)i6f(eVE9 zktK-Ei3!}MMJ+QSvZwgXoxDW%;_p7ahgM<@~gBgvB$$F zmv;XSk;c<4p(P|xzAQEe<(>IcQBhlx|HN|ZaPMRt!<`OS`qG8>xiW1M4{B&V5ig^2 zlu$Msa>JF*0wcWCSf!*!hx@qlE)8eI24HvBy~@t-tyJvehKV=z-r(mCD*1he_L1O z#A;E=&rw)D5#nQ$pnKw}5|6!LgdzU)xWF{na-_89fcvg);QAfF1lrx!d>@XwyANNt za88ub82^4zT7!vA=S4H=PKwoBUuV^LWOU@-!D^>5wq)pgUoi8LI_dFyp5mD=AEX+& zfO#(rPWJjk=O(702wHcA(fK;@44l!+C|`v;XCK8Eor))Oa15wSz$D*5RR9 zqU(&!g3ls;GCxYI-g;MBAp!edE{(%F2V~j@P(nzJGIg)NxSLFevTEdIF9Ihv<_2{` z+Os}92bjsdR6P{Ju;UunoM2o)7#q7UL{Xznfwaz&JUT2nGLkq35t3s61eeMZQZ{SlMtK}7fjLM_Q}-o* z84y$+4S==5mi+C97+YB%aj%l4cjr%*QbZgwTS^c3C6*i-_$6+~?yr6yvklGSQe%<- zQ~0fAAT<#3*WTWcxo{@O%fXJ6);<57O1%Y@`|CX@6K?}&r8j%T;q z!F0mw@3gH^y5+xL&1ktV7ZvBZI=5&XdlDTU3b#icQ3blgZFT(7{CKc;)}2klXa0Sx zXs7XEzw~a>^uvr9$?e9B?&b)!x7|bz#0wG6^%p8f;Cv)jxaLF!%4MU$*P}bKEAR!k zp2gWH!o2yiRyj+4S6qJVtC60*TZwx6k$2B1vXbEtyPQD~3dA*}`zN!hxmq6`=E$@; z2}b(WS!R%dTTrN!?m&lXu<~2KlMN}JzYa2J4Yhle+S4iJp50gX!O$t_{d?NT%ST_7SU?w3LnkZ^echGAPL#wPGpXMfs-w(gah5tv{VG>>-p3o7W&BwkPi9eU#0}CdZ7wt`+993{=*g6Y{Qha;iNV zub+_J((O1X~U4L#rPh27uu&yiz3c%5>sK< zmO%KjC5v*A>EOI~Pc`@yaX#9NbBct7Pz!WFQU?NU<*oHMQv@jet~~uwt3*csvchiO z2x)dWp8(Y@A8NI9^?QqNG9^#j+&ugAT|A#=ZY@7pLa5)P<+T4K-|*(lhDEQxOFmu8 z*=8o~ws(yD<$}l82Z{|F2q95nb_t~M7aU~PMrDKcfHgJ*+UsXP0=Fl^TV*Ew<535N zxc<^syDtM=3km`m!I1ufufT&{%iQo>8ZOC!9%4ZsxJfdDPsdy~Ztv_Od72cS>ZXso zW-H;H{Q&~^742%5vPP402@#il{ITE)mMzXUu8vg)qGxARrS)L=0ECiZD|V7aD%lC+WZ9 ztkG%^Pd231BQ!MsT3Hx6>JF4V`_;S+{PIO`vEWn$c700lm?JJrHlpGAK_ZpsGmAEj zOss7&4^***wv=*LX)z>ITvB%RN`a)mQ^NAz#%#sxF?gS>Yn1{k46C2850UJR)^vEB z!7w=t)A!b@ncxi=mF-~0)Xg$cYfsgwTZIBR`5`qMZ0f?CkdGOfq*=Wkpmj6+8V9#k z|3(bQ!rlPY#|Aqw7RTe;QUVhm=s2)W=x?yWUV!XCL3V6FXs{ZmJ@A6dagy!{;=<&| zY%)Am{|zbqM7{Kd9%h%uelTg7!d(H?#pdL1ZUW=PPL*ly7sQDiw-yJCLyHhWL&Gaa zJuUMMpp_2){&ru?{G}Ui;#C(Qu|Pi$XZ6Sedh=p9Eu&R?O%Exn+);NYUF$!dGDXLH zlk?9fBp^g6(op#*r}qKu;0#VYff?z^5c%VlR`Dnpa5TY!A0&t1qS8}U=YkIS#v);t zCXm1UD>d{8?!%Qq{TjEbmi7FwEZ~Dscy_l?3kchcgKtTM)b6v_4=9OUUU+iTBx}^T z9d+Wn<+c_BgR&2FOk%j2*5B+Yo-IU^*4%#a* z)88_XtB4s|YnQWWFsJBx)nT~Ll*2ME?zn>cPqO6~l4snaj4z|-eD;q`5Wd?!p=CO* ztfmQrHWdJeTerFNc{E1cKrK+s8mnssI>FTYvb#*N+rb6bNsSl?Te}}TqK2vVC0wwSNou~>Qv9KD zGNLT(*$W@}TwAhV`^b3AZ=TV0vxhmQfuEmRu^Ks~O!%TX^D{C9Dqvc5>4a@q30fd^ z_yv4QA+FCeZ-yRdK4vS{mS{o5Py zpye<#NKLx|+v782uhobvk4B#4FZUJMsW+ML9l<907A!(*((vu8|2?FE0JP~V_nshr zr>tSWa8E!sk_&j9!bdc4h>6z{lz7`C)Fz3e(1#Y}8zPmW z>?_Vb36Lcmj)BAR-XraSt`A3tuoSPfu3n!|su*Mz-9HjBZYBkuOk!Pt^kEs%uCC2n7dU56V_>;vYjt zkOcr_f^H#f{SHkukLU)kt6qRgmVnKADX--<*TV79fiv!fv zW*%@th9f@h!)c|CLe#nwd4PDWEDr+*))HWkaK)>f+UzlB68NRQ`*&)R9|OR#L@zA9 zgIMu)6odJAe%0Z6`(Fz%SRPNG$NtwbEN}=`qQUbYbc`UUIeDiSC>utcwc|qoyS~Kp zni8cJUKK9z5DS?cDz;oqkfb!Pf{YYIiL7HD8eb}h3LqoL&yev#rEm^#R@cD7V!+lJ60o- ztctmmz+-#1W|3821F4Ns*oT8Sx0{j0uLgq@m6@Ke3$H+?MhMF_b zO_s}M6AEHP%!UxTm;z2udkYyktHMwe=zXLaY*aHVor}ftdIstz>Q~cKFH^vO54Cq*W5EURh zOJ2lvj4in-b+kN2@Av6R4hHG^Z`-P(@nQhtkOwxP&wpBkMQzOSYmCWh*?O>Dv>X}2h;%jp(9jmYOcFaZ7oUeFFNKxm?plM z;svOqUX6SCx8NNCy`?l75iU3n64Ru({+H99x(C$or`i^IN>ihBl#T~L4zmQ!rym0; zNR~_BqS6B9Se3NnZnWx!05p>Vr9V+fHSGxT`J`YJy74yp>yL7K^TZ}OlwmF#c3zTX zS*fxiY=6?eY;ap)mfrD3U-8xW2R&JCp5J(OiO z`uc0F*j6u>^>RWj`LS#%3!9whT=V@~OG{uzrW?kpLlNy#84#~jNmNg%ewqE0faK`Y zN*DQPRXgpgWS?+Ck3XwMh?<+_YDS0B5v)qHZ-dETJ^-v`VNW!YKZB)<;f#k2F$r_? z+GGo;Hmq~6qpdgydP2>pu94;(K|yYY7!T$M$o)3_!MYUlOdPTzg9#emBegs@@`XLX zOtzWiT|V?uNHj+A-s0ZSb92OM$w`0;vDLA;blMExMwCWsH|SBb7((G++rjNtcAJbb zlZ~l08?j~VK7!ftl8E6|8a>yoUVkFl*WrBKGTqwz0_Bml-Ct9c?SZ6s-4|hl>#c}d z4h=kfq&uA;sCYS<7Ny#FibX~yyD^UvFQi3E12f4sb9G`RprnOE0|X+oh>wt$`Xy=i zs7qQCDN)vVmEn#nptjp*6wtYYKJeFTJb+bm_SgI&%l#F{G~!-jd^gx8kI$dN25p3M zcS$`yH7?#bcj1|*`USFaWydo-bg~_stT$jpon?KV(Nh71V|dz8jJUm4uGahJ0Ip@y z7<$6z7%L`nmU=&y&RA-f?g`z!M$7wxq;s!U+Nz>0>*yY|x!`uBQRFM@8%-JJvSWp2 z=wt^UM0c{^5)`~avAc9h3H8`*lDvkP1lD ze{3qd`GJ2+kY1|RajJ~DSrK}*iJ!9^?U^_VjNUjo95$<{TMmTtJ)ivCqqXn@U$Jxx zvwIk$u3s_)*7on>n9rRNRW8(4FJ#&JLUDwtc1izA*&xz`j<3E4S)b#lk={vfp* zyORcn3s>s@`i0&?7$4-Cjh6y%28g#b__Z(!i2I&glbamm=8DtRM314Aj+Hk-I9&fq zvo27b@uJO={J>3%mT^OsN-kcPr&=fw`5-8=T?6oTRr)V zJ98Sb*LwCU$yrz$S14m#17&#%IKs_?*5XN8$9n% zP$0;fNl3w*sU(Lfc#6ge5j4xfm1Me$7V9{LDdV1cOHlNi#)e2%D)|yK*T^Xj|3TvH z+sQ{uqYeDxs0;fvlObMrYrEJ{Y^0|g)D&)gGvUH#6_Vd|n62X)M4; z!`KmU)r0mAH{PMW8N*U1DsgT%jX>woKS>PYy_yTl7F@X#%U zT`@2HdH)~j<1_NmW19sdI3oJ%tlaoA!$SG};;P{TDesQH7#EC~8jSg*-{AaH4kexR zH*)9Af?v18n65rJ^U(Nh@Tr5Uh#CxFt?N=WgO9K0x16nlns>*d^DvfA-}rQ3vuZhQ z1-V;sv1G&opJ$?KRdPAhz0^c-q!UT$<}Z{zsi@&Q-#c(r`Jd=dUk>v$@Hh5#C=Ya%s z0#i^YLd{AelbXg!+T}X48hc^McZy$O2$3LP1je+g75=v}9mfC>LBv`cCRY$7EK$Yu z_PZ`jr1?N@a$p!=$%yd$72EMsQsJC&UgEx-#+x|AZPyhZg4gZNN<`yZMQKhNj&Bsc zK_ykrJH=o$;N*@h*9plVaLp+r!@ooip2{b)l~Q$^V`%&h9E(HE<5d9cf}v1kkD-qV z{e~A15j}plO$wJ2ZY>V_bKNZNW*N0LyAy%Z^0cd|1NDwv-u8kCs!1velF(hta-IJ_ zTUiiy1|lj$Ekg=N6URe91UT3v97Ed}1Jk${jl+0l?)GCWxh6ShEs-|RbF}bT#A#K< zCr}Ppg4&B;gCfbqd>QP7O}89)dbtFP1q;0tX0#Ny?y4|N$A}U?Wc3z^c}gMbXwjx{ zYg^l=8`0G%pChCf!QDhC`lfW`bcCp6ummn`%2Mq3%EZQ(mMCYD<0} zQHuc#ZVdpSZoGzPe4gcI<2IbhksNg=8Fe*11=K*P`i@eBdg3+;ZNt%P)Cp8;>FvVN za>b`znvaTyUzj6KM1vedA||cqS0sW7HfYjwmoEq>#4Lkh*onvcT6Nj`x_1HpEkHq4 zBLJ?)v_1LJA{St%$w#z&QoD{lUTAb9;$E;kYVsYK^8=@3%N;&}aZ$s1{F5C(VhtdF z$06^Wwm*gGU-JT6bw*`E_J=Gp?$_CL3>3lX2Gv6kLfpXD_{U(vG9vhib$`ubqPNAo zo3tIkC{sYVix9t&Ib$DJm?OJ>ovPgE+SqtebL60%YaoBmGjv?Z-Efy+8f34VVa&es zCVi)Qi$w?du}(XJdh}z|qp~0n-8SQ>S?jR)G|Dm05&>wBS^7ofjGaY_V5w5bUjb9& zP&200HE4?{D0TrP4R)NsQdTEtcA;&+#j2gLL}ZL~;$)!$3igP& z5++}c(L_Rfbi_X4a}9^x1}%>DcZ@9Y3Gn>GxEfCzUPT=uLgxlZUeVY*ZS(@nLmcpM zyn_bmDia%5I2wtM##igSi4L3`Wnd8bh;|o-g?TU;Zee!8%AYVTUP2wPnFKl`X4az6 zvUj{b_b~P|e!ebdlJzRI!yL*yxGZM)Y)|)s%}CX&RNj#VDzLzq=&6YW;&9%MoIa`- zztZkcyF0qiZu&<>;R)?{jMrc|vCp54w~SdnAX;vBG748J0mHyIOiy$_{|Np-(<1CyHgE&Bohsai{$+(VUjo8fwfwzo+`4@N+xgKu5}EIK((F) zmtpmy%3G-z5>@@fl~u{^c&R z0&CJG_`fqIaHcntKS1wh>ersuf6wlzPyJfr5&lag9B5^;=rJ#D84_Ug$XUjF2zEL7 z_~dkdIWlA?_f*rMQetfZh--HD)SV54O$Y{_@YCbaj4JjpXM_}&)7@+ISJ?IprbOa@ zj-FSdrto2djGHsPa_=8T1{p3ndH>Je3Sm^ZiAQ~^rqrC}?4G2~>9oa{9v8Om6n6`F z(j3?o^h+M(7VPY6Xj?uUX!qN96#n_{)?1%PzC4Jh)R!TB-!gT>kS`~yi1f{jW~Pvx z<>uEh#Nymm{PR7!#3d&6pT$XGSFW}SGP-u&V(13M-iup&a*th{Ui?C`p(fE*Si$%mZ}10JcLfQq9}NqV%koxh?Z`m4I1n7CHF$n*}-8G!Pj;{k>k z87p<4IjMHn_D|z2zP*@jZG5IKsPuf$7XU5YbenN@Y9}jTNLrDrj8C;)YXfk6$)`}6 zA%xzwO0LN1D`bPv?m(1goanM15brn<;HCyzucI9hNxrmUWJrm~l#yJmEa;3A|8<7E zxd%pmtH#Mm(;$f}o(z$EA>s+nJaR1egKjsjY4#k!x?}xaAwiWWKM}s})yfj8H-IBsGu8ehsd7_e2P<|N z!MVYLR0eXNA?jlAlnhW|5it$&%f16{CC^zl#73;IO#wG5v=~6Cy(J|()rXi5Gmg#C z4YetupIYu1YDM*}L9knnWyNx_sLg~Tl!@`7>dj}BIYayrL?5O3;{#nIlpL#kFe4J_ zrGaHs_^%?KJyiq^5II#1{Z8`7d?9RU3GG->1ZI zJ^1`t{tM%m3nVlFgc8(W(l~E?e2}8(O&D^Gs`K&%6~E46zmu)9_*X|C{*0P08!lUB zy>eY$d}-;Nvn8{-;j%ibyXZXDz2me3=Tn70LU8u?|$_ zH8q)Ca#*cDhEGm;&$0JT?0^xx?Jv4*Pi>B_(xt0DpT1`pt)!rQanL<$aE#KF6-6mD1RFX?nedh z51dveYQk?YduDohUhBa0a>BOA5bL#Q1{ikvW4K41*mnb39`KRiu!ZTMd_uqrRf(3x z1ltxD!ROn7Psw!sVbXmv;XO3g1C0=ZPC}z3Kl-2jfKjP9L=+HZWIxG;o#u_uhd_xL ztaRGY8J2nW^K0T(e%42-j)~>E%SkA82aTn-mUc5z^NJGgf9@pWuM#Y0VK&?NvAwCX??wrIItB#(-51nO*SAht4dlw>*HFjtdFhiRrK_^B;=d#D z{6XO|B^p=i$>} zL$y<$cK%Cdkh^OE)ZOeRHQLjccM<0Z-8HZ%Tk%xG;Q3$Gva?$<5o4>ysS{|`Z@nBg zo21oDj#xF`V`lVx%;=@={v)TLti-H4L2a{)D8|O_n2v#s`1tBeN*?lysJRW}Ex%gN z?XT*IE3vlp`XKNw=Ge86GNt0mxKo(ezs|86CTGI9#AeWNDMKw&E}WRFl>jlmQxg5D zT+(Kfjo;APkGRyuIud86@X^nROisi&O$2bexO}s~&tIy*aN8EaC^}qPZi$#JBJ?+@ zk-EY6=kJ7f19nPLZWBTe0~O>@oV2*f9J{scw492>!52+}e%)wgENG)xNAH7p|M12NB_3s;<9NZZhp29Vgn` zB~u0b&<=_%#cEuisr;Gm9J!D&ZrY3|r>{Fr^$5S!A> ziti(L312}KJN*)_Cm#1@*BP*_G{`>QH2=#5@S)REZtX?b*Hq)7|6ll6&^NrsQq@@?={rCd z5Fw@YvE8-FW$>TNh21vudOyufCs8+_v;@$Y><(mUU22_W z6BFx)i1J5i{GdK)NpRiPqBvAY{vrk!<|(-9J`%x2sRRmi}S;kgG&n;B3YKh7VT|943fq9Rr}8ghTTtehVA2io zv}JGlONWJTgWzo%qfBZy^sG1YVTOhAZcgRBo^zrdd4>(en%;zEQSp`zCr6RXStY!h zFR};MTFkUhj$@P`2~?M+Gg-@pRPE>yd|DQ$xn!m9t~<)?VjZ|#W){*Juie9?o^2v~ zqSJLwtSc|pYG$=eruqC$-hpf65A#>6FIqwR?vN9}h+XjoheaY?X)~*1mL)pS_q^=` zrO_W7m9Gyo&Q{3k0K1mJc`E$I_Am8mNalLE^oBvWDpWJ`n7aQ%Iln6`fa>~V=2Hr+ zx@gpuI)p$Lf9u-oP%sGFc_!sW&VxAhEg@8MX6e_OPyPOwyc0p#1a`YLEaaQo!mk}t45GyPvK09QGb8*mBaoWJBP3mgpct8?AJ z5JQlFK7b=X&?2#A%sSRJaMDMZ9WglWEdBUrb7c3VPl1zW$qL9|kP6>~LAV+r*YTvr z-D7uQT|i_82zd`$u(azx@E!1Pg93WZCQH9&bW8-o+qehmaGIe_*!V!|mRTJ&HEw$R zgT>``?siGdQRhm{i5vN@(O|p4qe2P}UP(jXM`VdQSe%D@u(C5Q#*oG3&ghGZthe@a zVNY5ZBbHZ(OY=q?DC0j~v0eTeD;z(^uX(3VIb_60g*j%~=cB>+yPNrto20`aU7jER zN-s*+{^8dutb!85mGpdF`(BOqj?1xv$?wiES2lUSWtL7qm&fy&<({-r<5>oY!hBtq zH4Vu_EaT#ziu5AO*eA3)Vbh+(IQ6AAu^gPoN!~BP54U(oa`izjt$yZQ4#g?E6`f8H zF2&VEMLq$yzR1H9>DLy)gNZEem(2c@KP-siD}e6$QpJgzSJF#S*Vk`c*Od06k9Y!P z?JdDH+bc1_tv}6IPJ#{n*o%D41=hJ`@TC^PfrifY=!t*j!s$r38YM{JEIVZj(Kl-B$cy2r}i0xYX0E5+o0VtcF_+cWRtqvBX1CKRA)7G zx#i=P_HE2WQNQTm^oK`9RBEl;9^)})yiTzuz-Gz8tU9f?b%OFw976qiGc~ew;85EE z{waRFRs3W-k6$<4N$`}A*^`v}sn698TSCVe3uM)$AGYBI`!Nq$_%THpkGboQR-aVI zU9n17mRHC;e&3i_E-H=Oxau|LzZF>H(I~z;zqC)&3F5Y2&4%y-Rj{z2OMjR)Fi9T_ z+Rt%6Y6qTz)MSyzZ-t5Q5!PNhXi8jx*vQ{|p>={{A%*e~Y2Y87$u3{ut(m??E^Tr% z>XMDu3AhuMOMQ9qr#yVcVj#Ef2|o;|^xR3#5#gMf{K97RVW zuEp=r5~>YjjJIi~M-uV&RtYFv^e(35tT9__D3VFcCAAKRafW|uG3rdzTqeYK-<;!9 zt7XHY_q44#?Yu&RW_llq4XR1TP`TrJ%-t?F>2uy5dA`^eSy0fGJ|mA?hl}q%=4u<{ zkQc&~K|g3Oe0Wjg{SN7I$yn5@{3`nlCDBC%tQOAIsW3){tGRDGV@uljs|wSZUg;Dc z?VF1`Z}q;_$Qml9uvw1cGu;^}XbO?w9X>0NjrqyqMQEasDUmCDr)WXctEQd5Ozn5c z_`}XG3itwf``4ViW%YZ$s&(eRIdkG(E633AhE1QhRCev~TfeUys1>kRxc`-b$>(>& zx)4|d@f?EwBR}3i%`|~WSq;agrF8{pPV*FJWbVbWQmLEn!DucY@F~Y;2Sjnm@S=2F zz}5)(xCwRcFlswi@qej9POeH?IEdU=#4<-|`Zd*|@`lc7iYkvmHVO0ID2TJM;H@tM|Uf-Ht6D?8{NXy?jLeH)|L;lz?#Fq1bl){YTMIlWj z<{E~r+T>eXS3$prvm6$jreCd+*O55m4hVJ}K*po)-0_sy8nGkmB)Iu)ei-~K#*wT` zWoc^SK3iv|j{lRR`Mc?3P6_-c^2$t3)aM*c9>j`S6)!DW?5HDKPBQ;9LY6Z!uQ&Zz zvQ27^%Wr#w@$Iy9;ApjL6qBo6@gHX^8d+wMj~I>yHser2mUUT&B~w6Fxw-0Umh#j2 z?dh*PFIXI#?aY{?cN{9)9^xf1#o`6aOzn4wlt={V7rk{TZgVSl$a>BsX*#UR{lux( zrdvAFa_$FTU-^~&z30`qwa@Wl4pdmOsEK)vwq|$6+O>989=h639&Qv^jFjGJ9z#*P z9olH$iO_eiRpP>bi9;<+Z=7nZ*#^KDDI(0ml;s2Fy3>Gqg#&XomD(1{KKO4 z2=e8aOJwt2++YdEw^0|3ro^{dzXNo&!~BT}sz40Khceilj_f#A)IAif&n}&>twFnq zUM|>+In#rTCTheq^aBr)qTXXTiVPe0=&>@c!dbc%mZ@_=^aab*@p$ANTT~{d9cfQR zOieVEmBqP#KVA92pJg!S7O2R5@g5j(M`)4v?fylY z>sWCs_t8p{Y*+n)ua%3~LAqO5&(4J4_g~)~t=|;ypC~A=y<)P07oc{uUsh~wNYKAa z&K&pbWXHXXUx_zO79)pIkVD6%c)J8R8|Urc*U?`n--P=>*3~s9{IsW?JnMUWS=~5V zX|?1%W$Fl8^%aq>s7NEjygJHg#@M|SkN?ZljaA-ui zDLaKN0zZil8q00R(s+|GN$ln&0N2xyep{i0_Y&^BBKDKm;Mub*RqCks=b^QD5>UKy z9^a=nL{1LhT_q_Y$@%4DGGVrD+&JrcV*YSqwOG|3e7|qOl(bGA9rj!q#d!*c;t<~> zjSJSo>(60Tj2Mz$h=aBnS`^N6@R%Y;`Zn^IDDqe*$3p18$KLb8t9>EKIY$eJN{swD zT<)~f|C-}F4shrnnEJ{i;CCen5j5!SIU9>fK+danDf=LPWYXmTpx!R`*-X2T+Npt7 zfcBNcWDy4NEhpYSBSw+7ylAKgp@yy=>~;+m4Jn<{ysc${U%Ll_k-hfV+7F-Pc}SYY zk34NiL6GJKC%(g)<*sV9mm%Wk!y)ud`_7jeG<%%4xkeG2G?heQB5){sBsm*rTgj~O z@W$+M_f@XQ$rs;xtUB(qa61E?whGQav4Cl*oF$*@r*YZXLs)JtO_Jjcc0;TMKcw}Zu46K+^>Bh zD9~Q7fjw=n)&r4WG%d02QH!^mcSx#zOmR>w6go+p4U3F?0{r!2;9yt(UgMeD@Dm3b zff%C9$#mM7Nie_^_{oXY8mvt;eHpE>EA>RVNjPd5Q{&DD%UvLriLtL{1?~20Nzlig zC}7)`?q@y+Z~uffgxJlFpIHA;r`!I4TRi0QJgTIr6WH^9$5NO2@U6$)@UupQU18Zo z%#O8Ibhoj4@wZOEilXv)uFJibiJuitJGCt%h*ac9#GC(m_!T$5THAc;rlp1coSHvB zNZ66B^A$Vj3RHlyvi1FQh@yO=EItc$Ifr-X8lzxy6rc8647Xzne4)E@1t1-{pUCM@ z-#&uWfK&uZQ$u|c*)H;w;3A8XPTZ{1sK`58ri; zd7)%V!Dcp`KZ}(rP>MS(B=!R)qUijmJ!p3VF&{dZcx>(62F*2Iv2I8WL#MOTaRkXB z(;3ze1iNCt!FGFJ%*(*zBPAj-6xH0i$QQV^*RX&??;_*9{t*ztEq|1o(b@xZFVAmIF2{L-rpg6@-^4F$3=jE+qc4Sx&UrxD;?qUuyw z7+kv!A?;TBR2U7JCmiyli0~GHlF`>0R6DgP#4=MEFr-t%AZF*Q5$Yss&oT=zfBw+DvO?-yQL~ z#jBVFZtlYrhXxTy%5c?V0r^u5{O5aA!;Ifm#vGz#Y<(U|kW{j@a9mjZQ&A$+!rHR5 zkp?%5iOJp*Y)|?wQ;}wLWquvX}~1s>JZ{ac(;hVhh3LKU6RuvIC3s zi=Tkf6UAC=Y%Ai++ElB~M6JIYhIjo_%*1w~6Xp{~Cr|r{_XP_VGnedEN2R&y%o;@R z{vP(sbSmJjJ3ZFB19A^Lg z`~&b?DcZ>F$`j)d^g5NlkDftX>Jv98U1U!GSaF9UaQI9@6;oW1dX}#uLyL3s{-I$} z_^;0cDss7=n{!`dU#y=w>7i5H38eaRb^VdXW!S9(-+UP5gOy+bSeRakuletULRkbG zThb-y!vt;?+6%zv>VOaXm?WQC2WB3L{@QR zpCgk^qlI_=o*CHy{jvZcgr88JoWLNIC5}(w2aqoe+PYQVi~i zJ2*Zb3p0moOiP`>UU~IN%g0P6+B$WO?lgTk1WpoUmuLItK%xjF7Pou=3}4&#Ac0zt znSS=uIUWLE69rag+y@>=iXUVUbrY|`>T@p&;|^K~2WWfF;jBWlY604fb&;KK`SJaV z*vZbIi5h+{eU~M>FYwWQ^Mvi=7~mXZSR;$fnl5-;YZj33u0aqXL?nUOxWt29i{8{* zc8N8MrGY%GOOt&oTN?YCvI~`UB+}z46dBuClEk1wmXSP# zlr?R#lw^x2q0Lmbij1X<66byQe1F&PT;Fr9^Vd23HC-di?e6}(-mmRlb{DrmhXvEW zxOBclI&m#_O-l59@&o~B!7UB_{SFHp?#^Bxa+KsyZdazqzk~bs?NEm$p?VSWVS^U20H+RCr;XuE>N3>FY9eN$bJJT|cfmw0BODa54-sEtUpffO@fO!Ii}wnZbPn3V^D3}8X#pv`TV2^(v{gES<>V~6*> z0A$ZQplP$jB%Ex5Y|_{eS^BG;7^#zNDJeH*_ZFeK44mJR#nVU?N` zk**VA_XuY0FW#TppSw)Ja!c710L=PH4p)GGMBE;RxZd906I9=u-aFewyf}2W{BXzK z&50uZ%FcNg%!!xnAS(1{^c5RMtl!Vfc6()*+j{Ahx)*hmmK+r--L50y28t z*fR#ElaOrVrS#3syPvWe0onfC9E9}TSLp4`8)-?iE)Y!ys1>s<3(t{TtvP+n*6d(A zW=p)EZDc8B$&UCPyN;xTIdu7l9S?(5!YkIp6DBJ`Ro~VYb=bzp)b4BrMZeW$7$MRB zg6gqm=*GE=rnmDY;8>hU&^mv`_FCRSIj5M|yf0|_pzO5jvec*LU{J<4!Y1|UUdLX} z*nXU;_%B70{5NFX7`9yWexvbEnyoB(b0&<#vOS&Lg%rK}{>3lrkAJe>On_{eyyP!r zg~T7)FuNr?Q_~XJ)z^$uu)f>Rv6p0q(h-W&4lkNRJEHC9&bMO{X*IQ+=%B1zs{bNlh0Z?Lk9@h=xZ~*4B+?_)dWsLA5tFvUQReUlgK8Nsv z6P|Gu;)v$jeN4b`5}TiZpcYuJG&Sx`mg&$gq+H=5-ldsY8=;P;2Zi-cuH*xN>Vx%O zpDTpV3i!5?Q--G7AkHuAN&9@JU7jPS5QfiJx4(s5Fb1Wmgw`0&?j^w05uAz^68QX- zNdD**V`jv0%SJY)qPFJmc?ix+U9Hy_NhGhF;qE+#$c7((-q^mX+#9=S@}vhw8w`khcHxA8f>)?fo&e1K^%*b<2rk>tfbtD^F zU&KZO1)OgrZMwc7)E-w6;r!_~tFq{`wZqZ)TBtLil}Q8 z74r_ao&<*Ep0%vtVpV|Jc!ew}0{^1#G&%uR1~x;)-y8Ce=UxW|&WVU$^^|f?LUypd zfK51|=v$m#I{m@b7Oo{fk$7gnVT7%-e;MAgXq|I>Y)B09M%+GcGw?@|4glCLFb-qx z)JTyP69{tPJI>txrikzz2+KmxoJ88U$+uwWDV;wzz;Jas8!=iX6a$7d!_QStSIiv* zP3Hg`cn{t;JFTS>25#n#H;3}&2;MM0fB*dA>5Zo0>xh{AzBaPq2sPUT*;QBs=idJ1 z{#b{@kR>l}Ij%vHfB-@{N?n!16DAnM7*!A2tEY#@rTx8!;DH7Ix%zI5l>(pcBN_>t z{WkQg#gP^dq`zNk*U1_gkVfARK-~ZHMcUdfO9l)jS#I|pen^utZoitVBI?dFNdiMz zx+>on!SI5eM=!mDWdWD+@&e(oWX5pov z4D_jAYM*iS`P7)&`-X3;8j$8vKi`i13tt#y+`0>!`(DJkQnq zrKw`Jk$4QasG?jpSKBt<;ho6CAID*J!Jz@Cm;4*h=XV6zh?jS|AxHmRWL~`kJw5D1 z3yIkGmAwOLP%24mg#|OE9i1O>>(q6t#3-mvimv1GPkSpL_!;okojzlRESZ9co5WW% zKeAxU0!(F1$hcDqHt_#_wtTKP4`TCw;22iwvOt8Ux&Ov z@O`U#+3(3i|Md549l@I@ua8Ll81~{|B@P)=eQ9mo)D!Be^5SM3@%$B12^^Z`;eV{fG!NGSY;Fj3xH4*}p2V%RA->XKIt2Oa^4m6OUEbX<(2)LkT zhlmaZjA2n^MFDBXCQ}?262Hz``~er9^~I`5G@r51z2=~0lbD))b;ilL{;l&uICQ!r zl`$|q94;&@B$*T*K<6mRgQ9dQKasOl5f9IhmOF5A{m+NinTmR4VHt{_Ef#i}HUjie zlN8#LIrk0n8^;hw?L-a?Qt4r{aDFJvi{vwq8!W0sUFbaH+=IJPN_j_rinQv}-T<&0 zEK?vt#ns8lkJ0Ehe1chY&X^mlXL=)V;n-E$Z_pJ50Qgtm7k(J5WjwgF(luPv<9%VA z31)iD^Mt{qL2`sl>0E-!_GGvkO^$4j?Ae#DLjQ#)I4bzI1jM7k5d1TXJr<=^VQrh{ zdGAp{EP27J9yN1BRik6Vd0FBul5@|k!IvrTh_+j@o>mORlqO7zs%{5HhsK>Sm&b_8 zyeK}M38<@tNTo+RjFiKn(F5623`xQ>a?uQcm*Kcew*!7C$&J%!3ZEy)E{GWdsU6sZ zdsojMInEQ`)CXP&i~O(n-}t&r^bWCh@X?V z;7a5yPI^Z!u1S!2H-(s12h^yJ9`qW6>$gP|uc)MhSJmO#jPf=0_9HUW zi(r=fxJOOgB!^m;jsIdl{eX>4?1G)8nOiOLn@?ZM)R3_gX^@_Fsh0eOLCzz>B9@Yi z>*w@HWMzJA!e=^cz`YAoZpS!`$ZXY@5tLGepcI?G9-o!!HtW7?t}^t=;tPs8D5i4# zZ@*$6cAkJLOjp6T>0ps}`Ry5pOHih|{*3*K{)^`^-akP06a z1yy2R7IM3Z>3LQ*PxqY^{wRfo2S};|M~uMbpmnX7B{FfU16TQkz*^z{ZB!H+3nZ8& zJ(eb6Gm~A9%VlIM5ObF5sEH&Kj|7bLk#Ja3+iIA<_4-B~;aFsm^o+e$Cgmm^Os!}087i-H{YknSrG-ek z=4pPfHNLNvFJ5Q=o7+dl#yL7mu{vwE&+tFc*J`#YC)1;|&oPLyuD7=XE;xaZ6oL?~ zHox%vu6dWi&6IxaB56W4_83MFeJ0NhA8Kp!QKIlUVrO^sPntU(z&9y#K7vL_#hT}` zmaMlW{UtSQ-)?sJ;xAg2>VV<;;?Mm4S{r&f#|QG&Z7wo(D7l79v)P0l|f-3R{`8^cwj$o|^1Z6yw zE5=_qwa}UMEkXR7CRL70Gn5~oRHr!{^S&DYQYXNdV1a|VECp|*$N+$=i}n2UAI4=K z@YiSx!r9D$7IEg?V@Wy!(M;hl|95SEA$ysICus_zwRb=Any4%TIlQV;xLy*-v8+1) zV^%-1X4)unme#PG{E1chQ&PlIWj?6|O!o1m#=);6c1)UVt~o+Z(m!OvWBv?3d-acI zaEsqycYQ2Bk{P(y@KOF{9_bkSMU7C2$zT}J%0?H`N!;|WMNofhvuT~M@m!X3qVkUE zRKOQ`v`zZ-O_7q&f{WG(Ksa!?-$n=r9dJ(i+ENkb68PlcfksDbDm^dHwQ_mAFcD6h zdffqos}2~_cP?Zzp%rEO_O5$#kcSAW81vNGjs3D`NgiZ_sh#YI>*MNnHx(huK+EQr z8?~8@zm*wQC7D3bt_`+q-OE(2z7zS`1u1FCiw`wHqv?`Z<3Tr@Ygy=>`gfl~)Vc*o zg0_JW-1=nPz=$pZj}}Ts9)IqJRj2Lr&QUGl`Bk`||DRknC{dM*wQ|-uM7}^MiqT~O zda4z}g~GI+fJ4H3)}qc0X=K4F18)(kV~#qnp;;)>U6p!vdwZ($%UJ~VHuucpOr1Up zUF{Z9cdZ?FQ z7lbxcO`qZdiHnY^SW)I_Sd#qPU8NR)T5B_9?Sv+&OU2$P=Ohr)t+_%Xm&#q#HV~*T zCMabbT_+~?7}K=qHH?$^&5b`uWj<|95$R>m0694&jR87|w;$$Po&O=Bz@K8;ue_f;(Wj&Jz` z+o<$`_oHK}rl;}nta1E4WFcVK3C_K#iY?`RVhkAR1Hkvwl2B-VdadCql~_BY!~)nJ zq4R8FEjQKJHR_^fFEX{ZT{vgXl-e0OX~wOX@MCV==Yt-8k~V_G167O7|GG!~J+-mE zrF@T)`=xh;`v@yomb_{{23YwhTuAzUy{SE4seBrBN_#Fvi;fPI?D};ZjU}(%BNm!s zEI|u+Xs?zM0Nqp>*sS4#A8v%y0$efLn!YH=E_`=5T;+j_%+~%M5725RVD+*!k>r-H z6rEJjC~rFAL9t;nA=hSv3q>rEl}8xoSa6)dWDnUTy%4?&wI;Y^ncZASho)hs<# zDk_ABEoQ|g7K$RnBQXDsEKAKJJ)qP^jD56e1NEQZn}) ziP3<+{YRS&NKq$Y6xwx`$ng+1cN3S9ZgsU?E`&?V`e4Fz)$_Yx4JcN49lM%WNU2Gl>PmvYjSOub&ijMVD=18@k$9 zj`rS-$f}XX`ROJkx96|*&zP0<1e2C)x)fNHS6Hj^ znG<#@ObOa8xHoT=j^0f*{fS3BZH3c(C!lw)^usq;=_-tFWuXH4T=Y6MG&^P%$muzK zv}44tqG1^IbOv{V;57lEvc$-|D9c?BPJa&_BwHLL6i{M>Gfm%f1lVnpT-tM~By0tm z_!Sy*XG3~^VI;d}k-*NIXJdYiTD}3|r8pMh`bS^dm3_cCHrlp7QcZo`Mt1?zzBUP+3~P} zQL?!M}m30dex@t zeKX__sN^||(}V>uJS|N{Zgl1Dk_fFZV(+kNrscEIBxd%=x>#CkY5fqvMqb$2VUv;k zHob;m&3tT+sz6H!1aj_Amr6@Xe3ovDd#);s$Ge+1w~I5SETumSnt0W>cT>u>wsbr$|-2IFTBl*t$+RKJ*U@_LOY4C)PnjKzuai0^J98=X^ESTM+I%5et{!k-*ZV% zxGwFwvK`z~R5{yS)-IpG#~QVTq*L5d6lTSIHkC=)_AkZbZ-s9{w?P4QskZe*hGnXz zS%rFcRCk31DkroqvINxHL&?&)IE8Zp^@k%gmgLAfk8rSid62oW?92MYE(h?G_|V> zGm=!^jUO-*o_>Wb&m?on7!_srWaW(qD_z;a|0wB|_&8Q7nJ^ia79=f>PVwW;MX?j{ z3c`-e$Gm(S3Rn(>TTi&rm`pyoys8w;m6_md&ekAZZA-3uqD8qp;!OQ{3#^w?47B!@ zVCwA?P4EXBCT;K+by*75nM~9CL50%QBYnH*jz*sqC&IDW4241e3ET~KTQ(l(Az53; zlxhYs@6bo(Sv{M@H_ia)xxkaoU=gicutH$CCJt4`gn_1lB6v z{AaW#v4-}<=4nROZra1i)FS>}lRGahTAe#waGu(2K8B5jhj(lOz6i|+!~XY7p(T-q z1@h~h+ol-LlQVI&z@I%w>oKG_v#iI}Y-T(~$DynpCzweZ(!0Syi~IZRtHt~<@891; z_wfWgRujkHNJ`<3#o<|V*@=l3{W<%O597G*`Ew@r?~W&1sK=#FCd(~J>j^68<1qnI z)Mo82$3&av3nSQd`@hfNME-8TSf;Zj=H=mC|6Cd#Qj&6n@#$J_9!`(%n3lr9{=F-U zih{OS?AX#hOL>$+L69RayCGe*2J?6I*0XFLtb(Vsxar%N_>E;Rw8DTe*m# zcCc%Y?R~X1Tq485>G@QCr>B>{Ym_N-&+w&NkRZXs7C6S4(L0ibfG6J|E>1zc@!p}H zY}0byMbLuVYDX*xG=BIXVzSsvZooM9&_B68Y`OoK7$UTpcT^xT02&^cLt%8qWgL7; zSjH>U;pkm4q75I8FpRwLX5OPMUsB*}{DXLcl&Lqa{VQIAG!4umiC$>~zgNYr@EJQl zR=xVed79&U5CZZ9cU3I%I~Rxdge)q#{`1~N=`c`|6(mA8 z0v4pDH^e_vcdV+(e(~C{v3{#ooN`Yr?>R@iZ@4&h_o&6Eh-Jmm~9T0{5gy}yRMwSFNl=)oeV5%<*@n&!LRQ3lRbGip%%(I>R2e* zsdBH!3HS;3WL@z_5c)kFT*d2>D!y4~NQ{g zMZ3N-f($4p$|8il_M*k}*5xditzrpT_cRbP{ zu(7`E_<%v>gGY30G*q)GCusUMxLv#njgkBPQ!ltAU9!)yBCu3FYM2@KDEBPdd&s9? ztNKDmzL75jDP3`|@f-r-Xlg`_L*d0LRKC9k!lVYiq+a}b^x$kzP*9B0SSp{`&z+!z z-3w+>kLPM%f4=|l;cLP?)g9j;f$=^}jba~|ae}m4GzezkW8tMLk3?>JL+9=4F^Dyv za;S9KqqMJ@OLyDc|EF|X1=JzHv;(L7jT00PxMVd zSH{Ecg?gs)7XFuq7jb2mRWH)2OwTV$mqHT7N7Qg+E=71>U9x~tH+UL?;da1W<&45QW?#tIt9$eB7 zdf$UGZy`LO`6?P+NG3ku$K7%5crtfai0hi=*BWNc|^< z1^lf}vQ?#~BV^l@iR(3odZ_)~r=LhtVTwPKX9??R#=6k8^3O!(#dtKb+Qsl2+P65_k!Gc3HSt+jGz3Y1CW%_c z_ZMjYQRG8l2|K? zmHw`fH=DizfUHzIjbUL)C?e-+D;h$w{nt5>vPv{vO;?qZ7S(WShzoaSR_KUd4HjJ( zO2wrcD2gR_ntwd?I)u3S8wk;CTc7eb5a@=NZxA7L5G?(r^Jm5&=`+6RaS2WD2ZH`v z=;Jej*^$B~(AX){`h{(p_MMG`Vvc=z#e&n{jDIHk@V6jFhuOW8#Ivjn^@mIV?i52 zB-q}7I)z;t)Ha2o(TDJ4+s=m`&SF5K>3vt)!)<#n8?=Vi1zlA~)9Kv?)Mt&f5R8)4 zL^*IVj#~?-BKWJGfyY!KAOH9seCuJ*d$FOPex$BSti1R4T`mh3j?WL_0W?<}F#x9X zUDg#mF+{X43n;)w;AxJn3VF>FEYb`NF-T2-h8GXnD9DaU&=?;dm-XaKvz!5%Gtxz{ z<4hfA(KZPa7Hx!h)Fg7&X)vt8s2ulNi8#;fkgYvmEj;q@x zQQeWuokB)Utg0~1Ag0%cSk{L#aYq)~He2_Mcs}dB(Ge_%z2Wsb@<59h%;m;!BjLe} zs!BF}4IZaPV&6M)v@nYI$|SMqHJHoO3S|4U8R6p%Pn7h;CG;kh*2clXF?*r}>7S9@ zaRfW=cBV2kuRMeVNX4slFFlbylZ<9n*977zOnKSWW;-r5iTM4}}2TVusZ4^kh#94V^ z-&&vDPcc1@a%^Nj-w(VMWR=R6B+5qNCe=8V&4eRS7fOXd_l@TU$*QYC7Mke){D*Ih z#?+mDtowM{h1#a&8IPu;*{^fr3YOP}o@1Xk=v%=L?gLI6wbIbNhU_(^gY3$8{@K4{nfztK(|;7sMn zRVp@-QCE($&CWO{LdHl4pnbO?SZF&2;=Rwdz`gUIcyBT4Q!*pMh}VmiTY!nTd;a3K zIX(`-Hgdj4D -n oracle-database-op ---- #### Apply cdb.yaml + +**note:** + Before creating the CDB pod make sure that all the pluggable databases in the container DB are open. + + + Create ords container ```bash diff --git a/docs/multitenant/usecase01/cdb_create.yaml b/docs/multitenant/usecase01/cdb_create.yaml new file mode 100644 index 00000000..01fc0a18 --- /dev/null +++ b/docs/multitenant/usecase01/cdb_create.yaml @@ -0,0 +1,44 @@ +apiVersion: database.oracle.com/v1alpha1 +kind: CDB +metadata: + name: cdb-dev + namespace: oracle-database-operator-system +spec: + cdbName: "DB12" + ordsImage: ".............your registry............./ords-dboper:latest" + ordsImagePullPolicy: "Always" + dbTnsurl : "...Container tns alias....." + replicas: 1 + sysAdminPwd: + secret: + secretName: "cdb1-secret" + key: "sysadmin_pwd" + ordsPwd: + secret: + secretName: "cdb1-secret" + key: "ords_pwd" + cdbAdminUser: + secret: + secretName: "cdb1-secret" + key: "cdbadmin_user" + cdbAdminPwd: + secret: + secretName: "cdb1-secret" + key: "cdbadmin_pwd" + webServerUser: + secret: + secretName: "cdb1-secret" + key: "webserver_user" + webServerPwd: + secret: + secretName: "cdb1-secret" + key: "webserver_pwd" + cdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + cdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + diff --git a/docs/multitenant/usecase01/cdb_secret.yaml b/docs/multitenant/usecase01/cdb_secret.yaml new file mode 100644 index 00000000..567b90a4 --- /dev/null +++ b/docs/multitenant/usecase01/cdb_secret.yaml @@ -0,0 +1,17 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: v1 +kind: Secret +metadata: + name: cdb1-secret + namespace: oracle-database-operator-system +type: Opaque +data: + ords_pwd: ".....base64 encoded password...." + sysadmin_pwd: ".....base64 encoded password...." + cdbadmin_user: ".....base64 encoded password...." + cdbadmin_pwd: ".....base64 encoded password...." + webserver_user: ".....base64 encoded password...." + webserver_pwd: ".....base64 encoded password...." diff --git a/docs/multitenant/usecase01/makefile b/docs/multitenant/usecase01/makefile index 0a96d1e7..e468ef62 100644 --- a/docs/multitenant/usecase01/makefile +++ b/docs/multitenant/usecase01/makefile @@ -42,10 +42,16 @@ # +-----------------------------+---------------------------------------------+ # |tde_secret.yaml | Secret file for the tablepsace enc. | # +-----------------------------+---------------------------------------------+ -# |cdb.yaml | Rest server pod creation | +# |cdb_create.yaml | Rest server pod creation | # +-----------------------------+---------------------------------------------+ -# |pdb.yaml | Pluggable database creation | +# |pdb_create.yaml | Pluggable database creation | # +-----------------------------+---------------------------------------------+ +# |pdb_close.yaml | Close pluggable database | +# +-----------------------------+---------------------------------------------+ +# |pdb_open.yaml | Open pluggable database | +# +-----------------------------+---------------------------------------------+ +# |pdb_map.yaml | Map an existing pdb | +# +-----------------------------+---------------------------------------------+ # |oracle-database-operator.yaml| Database operator | # +-----------------------------+---------------------------------------------+ # |Dockerfiles | Dockerfile for CBD | @@ -94,6 +100,15 @@ # | | +---------------------------------------------+ # | +---> checkpdb | Monitor PDB status | # +-----------------------------+---------------------------------------------+ +# |step11 | Close pluggable database | +# +-----------------------------+---------------------------------------------+ +# |step12 | Open pluggable database | +# +-----------------------------+---------------------------------------------+ +# |step13 | Map pluggable database | +# +-----------------------------+---------------------------------------------+ +# | Before testing step13 delete the crd: | +# | kubectl delete pdb pdb1 -n oracle-database-operator-system | +# +---------------------------------------------------------------------------+ # | DIAGNOSTIC TARGETS | # +-----------------------------+---------------------------------------------+ # | dump | Dump pods info into a file | @@ -120,20 +135,27 @@ URLPATH=/_/db-api/stable/database/pdbs/ OPENSSL=/usr/bin/openssl ORDSPORT=8888 MAKE=/usr/bin/make -DOCKERFILE=Dockerfile +DOCKERFILE=../../../ords/Dockerfile +RUNSCRIPT=../../../ords/runOrdsSSL.sh RM=/usr/bin/rm +CP=/usr/bin/cp ECHO=/usr/bin/echo NAMESPACE=oracle-database-operator-system CERTMANAGER=https://github.com/jetstack/cert-manager/releases/latest/download/cert-manager.yaml CDB_SECRET=cdb_secret.yaml PDB_SECRET=pdb_secret.yaml TDE_SECRET=tde_secret.yaml -CDB=cdb.yaml -PDB=pdb.yaml +CDB=cdb_create.yaml +PDB=pdb_create.yaml +PDB_CLOSE=pdb_close.yaml +PDB_OPEN=pdb_open.yaml +PDB_MAP=pdb_map.yaml SKEY=tls.key SCRT=tls.crt CART=ca.crt COMPANY=oracle +LOCALHOST=localhost +RESTPREFIX=cdb-dev step1: createimage step2: tagimage @@ -145,23 +167,29 @@ step7: tlssecret step8: dbsecret step9: cdb step10: pdb +step11: close +step12: open +step13: map checkstep9: checkcdb createimage: + $(CP) $(DOCKERFILE) . + $(CP) $(RUNSCRIPT) . @echo "BUILDING CDB IMAGES" @if [[ ! -f ./Dockerfile ]]; \ then\ echo "DOCKERFILE DOES NOT EXISTS";\ exit 1; \ fi; - @if [[ ! -f ../runOrdsSSL.sh ]]; \ + @if [[ ! -f ./runOrdsSSL.sh ]]; \ then\ echo "DOCKERFILE DOES NOT EXISTS";\ exit 1; \ fi; $(DOCKER) build -t $(IMAGE) . + $(RM) ./Dockerfile ./runOrdsSSL.sh tagimage: @echo "TAG IMAGE" @@ -179,11 +207,18 @@ dboperator: @echo "ORACLE DATABASE OPERATOR" $(KUBECTL) apply -f $(DBOPERATOR) + +#C: Country +#ST: State +#L: locality (city) +#O: Organization Name Organization Unit +#CN: Common Name + tlscert: - @echo "CREATING TLS CERTIFICATES" + @echo "CREATING TLS CERTIFICATES" $(OPENSSL) genrsa -out ca.key 2048 - $(OPENSSL) req -new -x509 -days 365 -key ca.key -subj "/C=CN/ST=GD/L=SZ/O=$(COMPANY), Inc./CN=$(COMPANY) Root CA" -out ca.crt - $(OPENSSL) req -newkey rsa:2048 -nodes -keyout $(SKEY) -subj "/C=CN/ST=GD/L=SZ/O=$(COMPANY), Inc./CN=cdb-dev-$(REST_SERVER)" -out server.csr + $(OPENSSL) req -new -x509 -days 365 -key ca.key -subj "/C=US/ST=California/L=SanFrancisco/O=$(COMPANY) /CN=$(RESTPREFIX)-$(REST_SERVER) /CN=$(LOCALHOST) Root CA " -out ca.crt + $(OPENSSL) req -newkey rsa:2048 -nodes -keyout $(SKEY) -subj "/C=US/ST=California/L=SanFrancisco/O=$(COMPANY) /CN=$(RESTPREFIX)-$(REST_SERVER) /CN=$(LOCALHOST)" -out server.csr $(ECHO) "subjectAltName=DNS:cdb-dev-$(REST_SERVER),DNS:www.example.com" > extfile.txt $(OPENSSL) x509 -req -extfile extfile.txt -days 365 -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out $(SCRT) @@ -208,6 +243,15 @@ checkcdb: pdb: $(KUBECTL) apply -f $(PDB) +close: + $(KUBECTL) apply -f $(PDB_CLOSE) + +open: + $(KUBECTL) apply -f $(PDB_OPEN) + +map: + $(KUBECTL) apply -f $(PDB_MAP) + checkpdb: $(KUBECTL) get pdbs -n $(NAMESPACE) diff --git a/docs/multitenant/usecase01/oracle-database-operator.yaml b/docs/multitenant/usecase01/oracle-database-operator.yaml new file mode 120000 index 00000000..d5bae7bc --- /dev/null +++ b/docs/multitenant/usecase01/oracle-database-operator.yaml @@ -0,0 +1 @@ +../../../oracle-database-operator.yaml \ No newline at end of file diff --git a/docs/multitenant/usecase01/pdb_close.yaml b/docs/multitenant/usecase01/pdb_close.yaml new file mode 100644 index 00000000..5917d33a --- /dev/null +++ b/docs/multitenant/usecase01/pdb_close.yaml @@ -0,0 +1,44 @@ +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "oracle-database-operator-system" + cdbName: "DB12" + pdbName: "pdbdev" + adminName: + secret: + secretName: "pdb1-secret" + key: "sysadmin_user" + adminPwd: + secret: + secretName: "pdb1-secret" + key: "sysadmin_pwd" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "pdb1-secret" + key: "webserver_user" + webServerPwd: + secret: + secretName: "pdb1-secret" + key: "webserver_pwd" + pdbState: "CLOSE" + modifyOption: "IMMEDIATE" + action: "Modify" + diff --git a/docs/multitenant/usecase01/pdb_create.yaml b/docs/multitenant/usecase01/pdb_create.yaml new file mode 100644 index 00000000..7953118f --- /dev/null +++ b/docs/multitenant/usecase01/pdb_create.yaml @@ -0,0 +1,46 @@ +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "oracle-database-operator-system" + cdbName: "DB12" + pdbName: "pdbdev" + adminName: + secret: + secretName: "pdb1-secret" + key: "sysadmin_user" + adminPwd: + secret: + secretName: "pdb1-secret" + key: "sysadmin_pwd" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "pdb1-secret" + key: "webserver_user" + webServerPwd: + secret: + secretName: "pdb1-secret" + key: "webserver_pwd" + fileNameConversions: "NONE" + tdeImport: false + totalSize: "1G" + tempSize: "100M" + action: "Create" + diff --git a/docs/multitenant/usecase01/pdb_map.yaml b/docs/multitenant/usecase01/pdb_map.yaml new file mode 100644 index 00000000..cd6d4ffb --- /dev/null +++ b/docs/multitenant/usecase01/pdb_map.yaml @@ -0,0 +1,44 @@ +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "oracle-database-operator-system" + cdbName: "DB12" + pdbName: "pdbdev" + adminName: + secret: + secretName: "pdb1-secret" + key: "sysadmin_user" + adminPwd: + secret: + secretName: "pdb1-secret" + key: "sysadmin_pwd" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "pdb1-secret" + key: "webserver_user" + webServerPwd: + secret: + secretName: "pdb1-secret" + key: "webserver_pwd" + fileNameConversions: "NONE" + totalSize: "1G" + tempSize: "100M" + action: "Map" diff --git a/docs/multitenant/usecase01/pdb_open.yaml b/docs/multitenant/usecase01/pdb_open.yaml new file mode 100644 index 00000000..25fdccc4 --- /dev/null +++ b/docs/multitenant/usecase01/pdb_open.yaml @@ -0,0 +1,43 @@ +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "oracle-database-operator-system" + cdbName: "DB12" + pdbName: "pdbdev" + adminName: + secret: + secretName: "pdb1-secret" + key: "sysadmin_user" + adminPwd: + secret: + secretName: "pdb1-secret" + key: "sysadmin_pwd" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "pdb1-secret" + key: "webserver_user" + webServerPwd: + secret: + secretName: "pdb1-secret" + key: "webserver_pwd" + action: "Modify" + pdbState: "OPEN" + modifyOption: "READ WRITE" diff --git a/docs/multitenant/usecase01/pdb_secret.yaml b/docs/multitenant/usecase01/pdb_secret.yaml new file mode 100644 index 00000000..60d95d76 --- /dev/null +++ b/docs/multitenant/usecase01/pdb_secret.yaml @@ -0,0 +1,16 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: v1 +kind: Secret +metadata: + name: pdb1-secret + namespace: oracle-database-operator-system +type: Opaque +data: + sysadmin_user: ".....base64 encoded password...." + sysadmin_pwd: ".....base64 encoded password...." + webserver_user: ".....base64 encoded password...." + webserver_pwd: ".....base64 encoded password...." + diff --git a/docs/multitenant/usecase01/tde_secret.yaml b/docs/multitenant/usecase01/tde_secret.yaml new file mode 100644 index 00000000..7cf66c03 --- /dev/null +++ b/docs/multitenant/usecase01/tde_secret.yaml @@ -0,0 +1,17 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: v1 +kind: Secret +metadata: + name: tde1-secret + namespace: oracle-database-operator-system +type: Opaque +data: + tdepassword: "bW1hbHZlenoK" + tdesecret: "bW1hbHZlenoK" + + + + diff --git a/docs/multitenant/usecase02/README.md b/docs/multitenant/usecase02/README.md index f43d1f26..0b060df7 100644 --- a/docs/multitenant/usecase02/README.md +++ b/docs/multitenant/usecase02/README.md @@ -8,10 +8,11 @@ - [UNPLUG DATABASE](#unplug-database) - [PLUG DATABASE](#plug-database) - [CLONE PDB](#clone-pdb) - - [UNPLUG AND PLUG WITH TDE](#unplug-and-plug-with-tde) ### INTRODUCTION +> ☞ The examples of this folder are based on single namespace **oracle-database-operator-system** + This page explains how to plug and unplug database a pdb; it assumes that you have already configured a pluggable database (see usecase01) The following table reports the parameters required to configure and use oracle multi tenant controller for pluggable database lifecycle management. @@ -108,6 +109,9 @@ MSG=Success Prepare a new yaml file **pdb_unplug.yaml** to unplug the pdbdev database. Make sure that the path of the xml file is correct and check the existence of all the required secrets. ```yaml +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# #pdb_unplug.yaml apiVersion: database.oracle.com/v1alpha1 kind: PDB @@ -118,6 +122,7 @@ metadata: cdb: cdb-dev spec: cdbResName: "cdb-dev" + cdbNamespace: "oracle-database-operator-system" cdbName: "DB12" pdbName: "pdbdev" xmlFileName: "/tmp/pdbunplug.xml" @@ -134,11 +139,23 @@ spec: secret: secretName: "db-ca" key: "ca.crt" + webServerUser: + secret: + secretName: "pdb1-secret" + key: "webserver_user" + webServerPwd: + secret: + secretName: "pdb1-secret" + key: "webserver_pwd" + ``` Close the pluggable database by applying the following yaml file **pdb_close.yaml** ```yaml +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# #pdb_close.yaml apiVersion: database.oracle.com/v1alpha1 kind: PDB @@ -149,6 +166,7 @@ metadata: cdb: cdb-dev spec: cdbResName: "cdb-dev" + cdbNamespace: "oracle-database-operator-system" cdbName: "DB12" pdbName: "pdbdev" adminName: @@ -253,7 +271,9 @@ DATAFILE UNPROT COARSE JAN 03 14:00:00 Y system.353.1125061021 Prepare a new yaml file **pdb_plug.yaml** to plug the database back into the container. ```yaml - +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# # pdb_plug.yaml apiVersion: database.oracle.com/v1alpha1 kind: PDB @@ -264,6 +284,7 @@ metadata: cdb: cdb-dev spec: cdbResName: "cdb-dev" + cdbNamespace: "oracle-database-operator-system" cdbName: "DB12" pdbName: "pdbdev" xmlFileName: "/tmp/pdbunplug.xml" @@ -362,6 +383,7 @@ metadata: cdb: cdb-dev spec: cdbResName: "cdb-dev" + cdbNamespace: "oracle-database-operator-system" cdbName: "DB12" pdbName: "pdb2-clone" srcPdbName: "pdbdev" @@ -458,6 +480,13 @@ PDBDEV(3):Buffer Cache flush finished: 3 ``` ### UNPLUG AND PLUG WITH TDE + + + +> ⚠ __WARNING FOR THE TDE USERS__ ⚠ According to the [ords documentation](https://docs.oracle.com/en/database/oracle/oracle-database/21/dbrst/op-database-pdbs-pdb_name-post.html) the plug and unplug operation with tde is supported only if ords runs on the same host of the database which is not the case of operator where ords runs on an isolated pods. Do not use pdb controller for unplug and plug operation with tde in production environments. + + + You can use unplug and plug database with TDE; in order to do that you have to specify a key store path and create new kubernets secret for TDE using the following yaml file. **tde_secrete.yaml**. The procedure to unplug and plug database does not change apply the same file. ```yaml @@ -491,6 +520,7 @@ metadata: cdb: cdb-dev spec: cdbResName: "cdb-dev" + cdbNamespace: "oracle-database-operator-system" cdbName: "DB12" pdbName: "pdbdev" adminName: @@ -541,6 +571,7 @@ metadata: cdb: cdb-dev spec: cdbResName: "cdb-dev" + cdbNamespace: "oracle-database-operator-system" cdbName: "DB12" pdbName: "pdbdev" adminName: diff --git a/docs/multitenant/usecase02/pdb_clone.yaml b/docs/multitenant/usecase02/pdb_clone.yaml index befae41e..be22020b 100644 --- a/docs/multitenant/usecase02/pdb_clone.yaml +++ b/docs/multitenant/usecase02/pdb_clone.yaml @@ -2,7 +2,6 @@ # Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # - apiVersion: database.oracle.com/v1alpha1 kind: PDB metadata: @@ -12,8 +11,9 @@ metadata: cdb: cdb-dev spec: cdbResName: "cdb-dev" + cdbNamespace: "oracle-database-operator-system" cdbName: "DB12" - pdbName: "pdb2-clone" + pdbName: "pdb2_clone" srcPdbName: "pdbdev" fileNameConversions: "NONE" totalSize: "UNLIMITED" @@ -38,5 +38,12 @@ spec: secret: secretName: "db-ca" key: "ca.crt" + webServerUser: + secret: + secretName: "pdb1-secret" + key: "webserver_user" + webServerPwd: + secret: + secretName: "pdb1-secret" + key: "webserver_pwd" action: "Clone" - diff --git a/docs/multitenant/usecase02/pdb_plug.yaml b/docs/multitenant/usecase02/pdb_plug.yaml index ac035623..0d0d3b37 100644 --- a/docs/multitenant/usecase02/pdb_plug.yaml +++ b/docs/multitenant/usecase02/pdb_plug.yaml @@ -2,7 +2,6 @@ # Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # - apiVersion: database.oracle.com/v1alpha1 kind: PDB metadata: @@ -12,9 +11,10 @@ metadata: cdb: cdb-dev spec: cdbResName: "cdb-dev" + cdbNamespace: "oracle-database-operator-system" cdbName: "DB12" pdbName: "pdbdev" - xmlFileName: "/tmp/pdbunplug.xml" + xmlFileName: "/tmp/pdb.xml" fileNameConversions: "NONE" sourceFileNameConversions: "NONE" copyAction: "MOVE" @@ -33,5 +33,13 @@ spec: secret: secretName: "db-ca" key: "ca.crt" + webServerUser: + secret: + secretName: "pdb1-secret" + key: "webserver_user" + webServerPwd: + secret: + secretName: "pdb1-secret" + key: "webserver_pwd" diff --git a/docs/multitenant/usecase02/pdb_unplug.yaml b/docs/multitenant/usecase02/pdb_unplug.yaml index 6d995ac8..085d337e 100644 --- a/docs/multitenant/usecase02/pdb_unplug.yaml +++ b/docs/multitenant/usecase02/pdb_unplug.yaml @@ -2,7 +2,6 @@ # Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # - apiVersion: database.oracle.com/v1alpha1 kind: PDB metadata: @@ -12,9 +11,10 @@ metadata: cdb: cdb-dev spec: cdbResName: "cdb-dev" + cdbNamespace: "oracle-database-operator-system" cdbName: "DB12" pdbName: "pdbdev" - xmlFileName: "/tmp/pdbunplug.xml" + xmlFileName: "/tmp/pdb.xml" action: "Unplug" pdbTlsKey: secret: @@ -28,4 +28,12 @@ spec: secret: secretName: "db-ca" key: "ca.crt" + webServerUser: + secret: + secretName: "pdb1-secret" + key: "webserver_user" + webServerPwd: + secret: + secretName: "pdb1-secret" + key: "webserver_pwd" diff --git a/docs/multitenant/usecase03/Dockerfile b/docs/multitenant/usecase03/Dockerfile new file mode 100644 index 00000000..5c27f11b --- /dev/null +++ b/docs/multitenant/usecase03/Dockerfile @@ -0,0 +1,53 @@ +#LICENSE UPL 1.0 +# +# Copyright (c) 1982-2017 Oracle and/or its affiliates. All rights reserved. +# +# ORACLE DOCKERFILES PROJECT +# -------------------------- +# This is the Dockerfile for Oracle Rest Data Services 22.2 +# +FROM container-registry.oracle.com/java/jdk:latest + +# Environment variables required for this build (do NOT change) +# ------------------------------------------------------------- +ENV ORDS_HOME=/opt/oracle/ords/ \ + RUN_FILE="runOrdsSSL.sh" + +#RUN_FILE_NOSSL="runOrdsNOSSL.sh" + +# Copy binaries +# ------------- +COPY $RUN_FILE $ORDS_HOME +#COPY $RUN_FILE_NOSSL $ORDS_HOME + +RUN yum -y install yum-utils bind-utils tree hostname openssl net-tools zip unzip tar wget vim-minimal which sudo expect procps && \ + yum-config-manager --add-repo=http://yum.oracle.com/repo/OracleLinux/OL8/oracle/software/x86_64 && \ + yum -y install java-11-openjdk-devel && \ + yum -y install ords && \ + yum -y install iproute && \ + yum clean all + +# Setup filesystem and oracle user +# ------------------------------------------------------------ +RUN mkdir -p $ORDS_HOME/doc_root && \ + mkdir -p $ORDS_HOME/error && \ + mkdir -p $ORDS_HOME/secrets && \ + chmod ug+x $ORDS_HOME/*.sh && \ + groupadd -g 54322 dba && \ + usermod -u 54321 -d /home/oracle -g dba -m -s /bin/bash oracle && \ + chown -R oracle:dba $ORDS_HOME && \ + echo "oracle ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers + +# Finalize setup +# ------------------- +USER oracle +WORKDIR /home/oracle + +#VOLUME ["$ORDS_HOME/config/ords"] +EXPOSE 8888 + +# Define default command to start Ords Services +CMD $ORDS_HOME/$RUN_FILE + +## ONLY FOR DEVELOPMENT STAGE +#CMD ["/usr/sbin/init"] diff --git a/docs/multitenant/usecase03/NamespaceSegregation.png b/docs/multitenant/usecase03/NamespaceSegregation.png new file mode 100644 index 0000000000000000000000000000000000000000..bcb0ae77818e87ed64bf248bc0c173a4aac28c73 GIT binary patch literal 270813 zcmeD^2_V$j|IS1k38^SuNX{|NWQ0(;Dsq%UjB^-fMy_#HvWaxLa&=M>Ns3&fLWd&> zkz?e_O%ulbf5VI@+qL`KRsXEb%=>=d`+nd1+}C?2c4?_CW?IQaMMbrE$M$V|sHkX& zR8$MDEnEOvc6wx1fj`vfJ!(o+iN)M)R8)Bl7!?DIor{Gv0#3yXRi669D=ub*L}PfN z+jzyrRSPySX@$9OqLg_C?N&@fyju+$ja`U zy5H0iZciRa6X|A+K$!B1t4fN9fuS}UKp?zOCGbhZ+TO_p{B@C$GM17M0G|{c91w7Q z_#t&`4C#KTvOa{u6Dm{d zNLfu35(&ne0koNyDL2`=57^=`!FS2xTM!i2~p4k@nywC391>)pU3ADgi!?u?Fl! z!}Jgwu?vZ|#!PYTLr4q;uv4g#DZ<*4#Kz44;lTkWPxnwHU%((qXb-r@Bwx340leG6 zO2pRb5FBL>$H37d)`0hcI~-6uCO>R*K$`y>m=Bfy1oLG+$9xK#oQ3#g0Wgp7p8Q5% zVobmBoJIXaP45+!A% z6UrK}DRD6^xbw$m@X3;N8T_5>eu&bbrKqW*vrAD~1@zESQQM=Url`9^TZ?iZkYLld z&S;x@I%Pg3w{pVld@)X^LjP2rc=r<6GB$&zOq4Q3? zp}hGlHvi9}yY?PMWepYKZHl^zN{Tuv!rHr3_9*IV?-AB9)X`PZoPpy14B7*A2?`Yh zKLB}V;{K!>1(z~21MR;F04TLzR*=7>;7gDVSPHkN==(GEdx}Ju=lDNq>U?5<&BA{( zvM~kll+5oZ5or>^W;FeMv7o$7X@+EwGPNVAxb}z8q(A=)A|cISZX&S||4k%7j!dG) zj}r-00%(RLIsFAHnf8p#b1DJTAkE~jrNo?60{AenTxSx_-=~sYv#DfKv--z0CP3L@ za2YWfna@SXAu%`<4xFmLMF$BLew->KrGFi&04|r$sDjimLoxqtBF}ph*$nlsxP~NU ze-+n|6j}TGYiaPeO=OCV@Q>>~U$=?>(&zZwDh3EV`AhPhf9~~%h)a`vVI&4f(Re7< zKkwTXvcN<~erH6?B|s z^!{xI4vg8~*!|P^GiAR8yP-b^=l+IxttE)4JCJqv&jaD)tO>Hq8QezJ+y21-xwtI( zEy;pX0;KCq3eFjVQi9@Sp24<=_j*9Xg6lVs#F- zOOfU?*)FBq{NcLNUkyur^f7)6OMU$G-F|iueaC><`(zdaNp%-fM!`WSA7f2Q`XnbX z05|&=lMucO&;A2eM9xm6K%0yt6t~3s*|BU+abjHljuOLf*)f@h==?lQ6_*-9?+rKxrTTh&# z?EU--!&e>FA7#KtHR0o@?^YOgA)G8NCw(O1Vjq2dhfqLSn9O?os6SBDhmY4NEx(9D z@hKMl#2Urtj^STMqxcj`KeI~l4L;5vRw?Ex8X^_4>@vlGS`1QE25|U^kuqfdI~AUy zpOe`(1WJ~{Gqgh}g$idHdh^e{MydRp+2f=4nHj1{)Kqb-^>~&Rb7(!KUuU%Q7n`hggp)1nSg{)!iDn(l9U>--?9xv zh9+xrf6CI??~rYO6ZfS)!F`ISb_VYM970XztM2=M%_p=?Nmvk(n7>5>y1t5w!ol=+WOqGzh6^ekPZtu&NoD_LBqwWWK6C{hO$!t1*8fmGczW zq~}o8e->Q(hET>l$gLtLBvCY|Ii$#(#ktdJ)jZOm=87$l_|^>Xz;fpwZ4kdc5Q8Sih~^FY0UfFjd_4^ z6rG10Ii)zU$dR@g_&A*~HPm!6F?=!&j zvz!k?I@*E+j#)x;R!rfC_fkSpQnhTY?Ll2V>9C?1+bVr5rvj(1n3{t<(kE}3di2MO zpc5Eo)@H&}CmfN6Gc!LV3{FI%u$Zs*A|GEeb!-y(NtBnQ4Dx+}ys2C?@E)X>ni~FN zuG;Ks^Qq^}#RfmVH`)OXw)dXScl&r@j%5H3)tWr(i98+h?#|nkNEzuLE91UlL6InP z^bWhpLz8BYg|N04L7G{krcT2W`S|mr57^~|00*WZtjz%B%zl(2!r9an{hi~QAJVl) zJE2GiCVe{I#%(G(%6oR~nmm~Rx^1Jf_T<}Bp!~&^MZq1wE0TC9=^X*D_=Yt^yP_4% z?X2y0IZvVbH*^8x&cf?YzmpSJzY!fi?ft#qr=B(!MuCxL9{>R`4}rLF*#W>SW#N7ZQ@dB*Y-T z*O2&{e)6^JbBI3h5+bDL?-qIhEXpWwa@en>yUZuBn1SW}B3M2rhreq1tom%~M&Jnm z$6U`SbD(I`B$t4&g{hr2f@HMVA?=YAr@=?D@M(G5)L@jVHcOkzIqbl%u0(x&|f5-rSjnN=5IQVg((htGI~Jf+W2Bl*D)n7JAXX;5Rzw%1ll>!gX~28Uf1!Ddx9N+OA7Gx zDL)m-(j+s{Ii1H-EO*kmH{IuBm#IO>Gng}>p2SgRp9?niIPenxj2jp%DtMn+#x{a< z*7TG|@*8IPY41knIe9cOr2(4;6ZZ%Z2CG_Hp&5QQ_rKI6-As=Zup`*nLM4T1qVIIGx-5WJwULz zRv9!;ML|AUqc6VIq+e!ww0{7n=B<~E#2bDNcl8fg^r?2ZuUn7RE zU7v$k%uGeTA1KqF;r|z2{&_EqKDvf~K&P07C-?G$z4qZG9UmNz0U|zV;O-zN zwS43+zocyh zfuBZM44g(OxsCJ-RU#o0z|Vi;utCxy|5uJ*Ns5Q=OP*ip>4OUAz3&%z_C9m&ks4;i z9zHr$ApnkZxm;)XcxN8_F<0I$+RD@%=?u2#Ci!Bg9K!n6<`^r`GUq9I(*cfu!>bRW zoRUESt|U3Fd4^km&V+0VQp_i}?Kf1R!R^fzQK(69m?2EjXlsy_3V>pY!hGI51I8Z_ zf&@gki#5gofRBV6q|b&V-~z&*`nHXf=}x(A8);o9zu03?uB1-WUuN_oUHjN=@*AZe zX@_?>$h@3<*eq0z!$R9dgh67}mK#~$_A}t`?^kwe#k-w_Jksw-%0{KN0wl4JvTbGsiiFK5@${0V#EvMP@x2ezn3$>KI|IV!RoKJY@e<~gSRSRB$GO`-;k^1XV9G5gm@M-#U-Rfq{uWp)7>T^PUd#=h*&AQ+bm-J+!a67 zLe72q-t0PCr$cZvj0Qk?U}%GXrxJERs=SymDOnkbGBrcMg-MmzhfIM(AG!_c07sc( zkSJkLPli+xoj|ngms<#6lmFv$k|Ncle-dS-l*jzz51a2yiX@a!_P|0$ zPi-0MjR?>|eLc&kaV*;TEP&lXI9E+a-l@B}xAQ^Y0hlUgTeiM*}h>;UgtlDMiUW zm=Ud%pzJL&Gq?q%G|r#k)>Q7^C;LQ@CilNeCI2@7Bu+`ABSQ!}2Y}|>$U}k>x|}}% zfn2`0Nzu z$*bnprj{sEJ91}o2T~c?REsYIPbn4E8mb+@%yBu`8O8Wow~;HqXFxu9;Iz_ZbL%*v zTgP-ml-?pWeec}6xFI-I=#tRxotG~QS+n||y>nIfyzs5NJtGqj>K(I@+14RYsHH`5 zVS0TBsv!bhH|XkB;%d=Nh;H@JN_^zU!gKt>!au!d{BBiU@`TqXhSl@chHYEvKK|mC zoW@o9uAx~s<9EfjtuzWpBX8W8HG(%?b|N+Fk{Q2G*3c{^u4vNGRGBg0R=N$nPK#&t z+`54EG_GWgrOl?9Pg%FdC&_l^t8Aqc|I?e!bAham4(p7kETLw^9-sw1A#4V>O{m)| zZk-f|KQrm7P9?U+cyHrg3eCb4k14dHK%4QZ=5u-iAlXP91uY&ULQK zA&2NT?Y((ST_k zYea1>FHAw!XXh_i{U|4hTXt!lKgV-Z?S_ZGyR+P!9v%B5+O1un#<|`@Elkq6+epId z1=Y!A!b+hc2W~qwDCs5`uxi7GHdLqDa)y-na|r&?YSPtpzdk=w7A*#5!Oc?^M`$U6 zdT~kGoHxB~QM5ABslw1gOVYNY@V$Oo2mexI50OdGJ8U4WGRNd9_j101S{6UP5f|-t z43{%D@TTwmbI}OM#_Ij4Ho?5T&Rx~;ZHTg1&A^K-6C;a86^|}hi>%9RaT$+Lih4NS z5`oOOyI*TT*Mak)O|~iDG}>3d(^8+=kcL09=tY7dG;^dogV^B5(>p%W(|G5(2z2b& zoz^@~<876R2Lh?C35#a-9!c*DlXpKHa*^3WTgK&ii0~upxTxc*fh!wgBkUgn5cWOg zkHH^hfr&}WvlG>LJnpiN>BaB4#w3z6ih{x>JO`>Q8w;T&5;PakeCi#?j}9Q}vuj5K zr4_u!hxqGU8W*qcefqBHoOW^hV6|%9V>OnfNQR_x2h4xw}tjbt~( z2=Bj#FV{Dgj43;nGcnG^9ntktT~u;nVr;NJf-56ueE2ywKf=#8F?~U1vP^3vx-e}^ zNWrcb=)xG!;itrt!ms+-2zfWyQ~;vm9s$yj@5E{zpnJ<}!iZ%m80dP#mApFbNXcGI zVucRX9wVK!JAFuYpMq7+k*q|3IGdM=9*k7R_ta%^(Jxw7Xv!D!heNPO^RwXV`%-Tj z(#QVUv`Hdn-7Ya$FOJq?Z_K*0@o(jDkB<)2YZ6v zH$RU^H=wD%<9F{pK{RJ{X+4$@$V$vSlh*ri5jsrT^}RXz3ZeHAOXC@})ezVUsKp=l z>CGVt5k9-H38!3Alv*+`EkpoH%?>{~VR?5jSpOj8aCy9bNk9=lFOMg2?XzG(%TbQ* z;S+-^%ZEq|0Ugwdj8ag17yJ7wYC*+od(&x(UvW$M8`ZIkY5E=t7pyh9)?#x%b&p+@8RMu^M@2Xn zGZTiX%3;7Ea;2nAm@J)?`YwhPe(q<+o5!O*Uwj{SPc8=*#(NRh8b}wj?RH-q!`Q>%{<OB8slx_siW&rggPAS{uw*7|VqHm$0?)`m7nIQQn$sAYob2lvhhuRXlfi`Ff3yn?`&0HN_B%!@7S)R< zB4n$LqF92DdX4p0a+N--!3;dgPG47?AID&1ZRsgXzkVbg5MAFWp(MhK_{zIY`ch}Z zIq%US0+-_GP-~1IJ8_)HU|`$jr(DZhYS5C=fo)RkA2d!dh8!*73hPMBDE<=#tHSG0 z#ojdtwB8;WGqIY!+t>vVS?LX(xQntx?vnx<745QRr(P!oOP55j#Nv+;^b=@Ev?MLP zT7zI}mhIV?A6Ocv&2!hfET1R{sN%#e80RFHU#PW}##sOO!auhb(h2#L1jg4iJuQC+ z&{$9%FKzvj&e&^AGDPHn4XR4OKgP9X#fNvLkN$LH+Uc*sgkG&5rxnFb<26d`dlD)d zlN@woi^l%sY9Cq#jhG<$fv~6b777~F$*2dn8aJL5IG(<4b@A%NJNof@MOT||uiA`R zn2v6hny|Zt_eO!Lpf2VI+kO^n%zHIGM#jX{=C#Q^w5Hxve5n-wv^S_>Kl z#>ePk$m^@Iy^7~He|Uq;36EP7vOO-WwTO;O|K0w(R@AmJ)H+UDTl%)G&m5BQmR`V4 zY#=_~)H`(58;xgzxxVCVdMb2?c2>o zM#jfR&;ifx2DFvMSu$e%VTsky^5a~4#`JrSHfIiYR+W7i0xKQi=}N#AWo5bbiwlUc z2aHN1`NBdD)v`N4ZDicj!h>&YV1J>}Vd}J^EuWw)4!FtGzS;t96R2|Q7P(hD_&rj6Q ziJv5IqSsvHz-p%%o@4iUyrh?tsF7!~X`znQ#&>Bnnx)0BmfNQAOCLe`MoLp}35hR~ zu5maDImLybrO);%U%C2G#DdtvyI$=%kIIV?5QAD2ib*fl*xOfc-l5QOf{p)ps$UG6 zH$(5%zSs@C`WXU`(^<=(xw2TWAQvb}#xCwtJu1y&lxpd=RKG91y*My7_vSl;@*@e0 zlkfUPpwj)kr3DfUlr1V2GZgTthtM}~f|Z>)vbXfo?N&Bs`|fKjVIhzFrG=h8#qhN} z^CN^h9bJkzCmbVo?B?Z15qx@bZ`o+${h6f~o%0RelgBZN>rkUnfI#980s)YzxRs`! z%dUJNSe}f%pXkH3Pa0khUa2?Hl{ejCyM2IB;bMqV(S@4w_Zs<6RLDN z!irxb%tWe7+-p4Ay*qbed@v{BL3;(?{t1Gi0AHkh&AX?25#x`BN7*wIDvNK5=eFZh zkn#7hGHDu58#2RbOUg%>S8ZPISQ0s5B1_cE@*3E&|H+@y+t}Y_=Hl5-;5Kd6(0Qv# ztKU3?S#Pik8~r?v2dnh(nUqmcoP5po!W{H9`K3J7>>7B&0osOkJk;)_fHHPvZiG0h zxuwyeGKkJoNl7ot-+Q6UF`y%=ao@5ApQjZ-m!qIjdNjdsR^#tGf3Fe+Nq%FD_sE2__l?js)p*hCDci**L9MaGdK3KAj_OX_9ylm<#hnQBwV*WY85;2ihcX} z&9o9IW}2vjT=a}sq%r$aV%S|Cd_jQ76|M6)=TJG`-rD9MU4!*LF-+|%b?CT(h1j!XyAvCfnsGF&6#nYOQernM zjxIZ6^bC{jhF%@T17`l-%HX|L(cG{<#J9$W2l^mj^KP2@1Kp(Y6!w?~(s^#WZmYrS z01V4)a{0NVNc{=tgC&I@@BDV)q$5cZGL3kmg8MrQHi*$`^KSgbun1dDG#3eHF9gNXW4oY+2N-|PDInj6g+V} zmu%SsGkTc-Ij&Ip=5`qs}b7* z>(u;!?Q}KrD(7`Evr>3dPrsB%eLZA+K@D`d!R_T64=|uuw%_aYE3@KcuzP}}T%KHm9s#QRM(Z7l;0k0*0NSctq!<0)Lw zbMJ%qhsRC1Agj{`8SzJkaSV^V*CJ^=Vi*$B>0Hu{@5ztrSXD0x`>-Wvt50+ZRarL8 zKqq~B^nxt}Du;)TvUb7;*{>Tfa%uOx_zcp})ev`>NSDH2$RXf^lrw@3<95+_75QUb zy6Rq9bf=?ed7rJ`R;wqsJEvcZaKAwAd_lH+`f2HU)AO(as+Gp+wB=mgkeEQOT}(M5 z8%@R|VnXG|sYKfDa5UfSEE(>6xc}az?mW4^Qgt=cq0A#`BSrm&k;YkZ5*3MseuT}P zxb#)`B67S&99cD1?dIOAU4PP+r|MV||FXQo-FXgf=W4AyT0?}~N28n`hPyjz)@#1b zS}z@&bC5enM97V{jR1Z04V?b7y0Pp0glcbd>%xpQ%slr3WxW+$oh zZ|TEFwd5FM8xLT*#$<{&-+I8BCmO&VG;CS%xO31rmA6+^CXRWZq%6NJs#Qv6QGam) z>)|^g-d)Zmw}`})B(aGR-j-NzX@&+wv*we5RdNug3ZueEXnkSux`BWT{8tPc-CAmN zinC~*>y1+T7kP1(NUS>_hr>VpmW=?f{Df@{v}S%TM(}ocUEx> z5n7d#d_?h0Y?~T0tj+E`?kcLh3-L;+PVEv=@F7gIuCat$|jsN31a z+Cg;v2Vq!K-(dCPs3ZWkFYy+Z%yu4Twm6HB`G-^Zne zGE}dGE~g!ScbBiL^m^Tjw6^x=VuxSqV&7L)67hkSFPb7ZU*D6Hnto!~?O8`lY@*HB z^TMLe*PJ}cGBL+8R6PwcYZi{0j9=*OvrL;PwLHVGy&7?F^CCO@Ooo)~ecpnic6>fO z4e=+XuPN4>XISz$=*gMXD&RErsTPG2kBu{k(r~^aFt)$s^%2}p(+H!{LsId<7$oH6 z_Dis|(IaY^|8QZH$=WQ|QPf#d3}Xu$es+e&YfSZGwXabYRqe>0F_o*p$(h)@*9>R^lR3e~lOHV$Y(jsn2kokg9DB#jXRdeWd?M!5O;ozPK*s;C=3Wgi8A4@j-0b z%|VMuBV-WkE8mkmN=0br#sWj%+a6}CjmCQUB3&x#gi32}67)K&!ntCCgpa@9JJ=Y@ zKT$N?nWi(?Fzmw>w1g;kuPOhm%rmYKT?k2QNBZ#^O^Uh6H; z9&&7#b-zRBdFhZ}q2|%@ogs2o_M7W?_R*+Ya4MbaxVKjOAU>ru&jE9dHmx}v;eNsB zer=&?L!9sKmb9@nr$3Q3y9T&l$1D;(EO4|d;{C+|U%BL#%29(cbr=R~&UQ00yRw%N zuAQc?PE)_$hqF5)UCO~F_riMP5iM$rT_r={nrmWXh~oQJZslDTd4#8a-QqMMN?vt- z$gu7bTW7)SgT3nq#$-zpn2f#jjWY?k7+r-v7j79Bs*{$1(go%ARjdla`>7LOy|_Vn?MM%PUqbBr=PIEakiu&qppI%6UUS;%(m9M}Cv3S9S1%1UC^8fRy& zwM)&%w~Oez(&|5(Fi6c`s9a~#8{1IJ8r0~oe&Isd=uqeD^hJ4En>p=7A5gIwyT8P) z_Sf#u4~^K{QJ)izkkJ|?LJ>8YU*QtU@?>k9XCH1n$?>yPJ)V5{md+h}?(8rbnjY=3ni_MVq0s|q0&6jU zW|irp!+~_~`|bxy-}}&7-9K@G-$*#A`?%Sr5XpO1^a`VUSM9-jz2|)~Zh!GjDh{_A zmoL2ajhwfLKJBV~Lv%3xrnsyH38Rayd(b<0%Hzj4@2C)tC;1#1@_1P{L!t5i{jjZZkW7oeJH_`DV%s+ll|u8iIIvIWm8xZG)bG-*A@7C(MtZn*vJ zCugN)FXjJ1|2DLKTq%%{wC&OndWW)dyBCS&^cIpahtAjyMrU2G#@+Xy7<=2?qH5Q4 zI=dys0mkSYc#ObF4|83$KRGgefva*6|1vDi1|OdBld@NFUTzzDpNN^%K)1%DgE}7@ z;|XP(^@-aE$g&jr@v{QBfde?-`&5=O4IL+GyhgN4+IC*lw#2t`$kKbpF?efR(5@e* ztsJB~u0Uhtk*tkZ7+mI{A&YO?gd^U_=4Zq{v%K~$(nUyY;09uUkO(!qz#lagoGv5l zFXpyoXNzRv20!=|?c~2h2h%xzFlHfEp)~X%+q2bjSfwhqeBOPzEM8e|%{m!0F@jaf z_5iY-Z|b_#PbAa940q#us-w9EGuznu*o&M`y*7F2QrGDraY`P}Q$1LPfW65O z=4>@iq2(Gcj8Mks3~t*oI&4R6RN=9RZ;Rp5(Are`RFT(qbmN(WCoj6R9p&S->I?NP zT&$A2&M$?|^C=U9!``GK!$d)&&L!_THwUmp3;U>~FsVU89l86;g|@Xl$^9By#7LHd&(9n!DcKm1pDjb>q@}>2m%Rm@*%7^)_SWV)Yg9!# z>Ov0c==%5N!i3I+qqrLm*CN{1B92D$oamoGh)>jB-F@J$eZ$BZ7ItD5!(wQ``!-s_ z`|vkBR)U1iTg!IhQ;cC%uQL%LhVX-_7V>&tgC|%6$7P9~emA!1)N#|aH2S@L1W9*p zs8c)s_7-D|Nl&(i1^r0rh+Kq39Pqsb2i@>iXwinmir;F##E-ok)>U!J&e10ICSTUg zTFpVzC+nRfUah=mq3;B-R>R)R{jfy+anhUHtX*=05(Nq4_M1dT>T6q|6Bf-U7^)x1 zFLthbMXf#9Wj`Q&tntzKk)a6pB|D>DK5lvl<&U<*y{IhAjLjjeOSsGVJQ@cPC=P#V zmdd~S0oxE^$X){$3F8_Z;PG!3oRFCCvM4$SY3q;MdK^)bGtS(N?cG7pwH)d^aUB)e z^MI-KoI~Yy8L!=V0wMm*-N#&wabf2-`n%U{*@16Ot2srCIZcx@aT2GKM9Z_7Vdcoc zXp@x>O-@xH?lL?(*kOVj8;#L$#n)ci8p!4m)93CxG4`fDdvr`71uIKrV5TXMrZ2wU zov}iuo93dX4!++9<}=Z0WxVr6m_W9~N`H4I;x1XDXg_^o)~isySC$)li}`VN9@!q$ zEQc3#x1YD`c*vNIiR8le`8Vr$SSyhfuGSony@oYH?hn(p(I|}ew+Cwje`+;nx47$I zy{GPS@IM`A-2)D%{au@t@Ijrf5*4=!E%hFbp?jb9T^DC4ZOGN!xW6itkd0QiKYuHe zC!(@h@{mWBkumf5rK32V%3w8Lwt*`iJ(@0e%v>X1I!IU7X5=j#P4D4UEXvR5Yb<=Y zQO8vw3~GFF*F9Wo(J`;aXE5}Ka6SnSbBNIR<-P-!YJ3$;P@TOim*dr1pD0tiA$&=4>co^iq*a*Kpxx!@?fd!rPdXD@cS=^ zaH*y5E4<6>uP0b^bmBm=(TWtq{#f_n#+NOvE0Z5rOoZ!ev=_N2Z86pBu#NQM5V5X! z(j>=u6D4(X@2vnH%WS(-@|btKDlGKnXmSq3^QCqB>ls3A91qf-tA3QP%cP!JTb8~) z+AS?($Q{JFEXr7sniBCZd$3}{S8r9U$)$spS)GEVo=6{ZwIrHs8R)caxj_}9u}kJG zi@v46O@YkxcB8x7*J2gy64KKb<`t|w*9y!5eT$T~9{V>RD8E0{m zD>AL;*y8(+zdhR$2r(PLN+9d$YmUAcTwT9lYuzg$!|bIQo+Isl2Hx!?bcHUsz_sDz zD*a{wY!z|Carccv)X#0bwP|6IkT-=f8teK@6)x%ZF4`;00L-(ptBJQAi@8sZ$YnL zo`b;V+)+5=s3|RvSU_87-gCX}cpk;ECkn^di4}Ym5ba^T%qr$pG~;hHV%Dqp#XyW> z1PM8w7lbd=yXJ>3CA@rtlUATM4(zi#eC@O+@i5-Mduv7uF0_7_Zd|{fnzNRvrj+#^ zYO%v!XMB$n4hi+HtRpaD?N2pp(n!~%`sho1Pe3aKQV!h}Jg?oDO z7VA&b>i7CN2^{svKwT-|suCo2xg%8*jr{tG{8Vncxq8-&7Uo!_2$ms=a}H;)R>noA zaCr%BGNBufQGaiy8*`-;e*H^yU&7zS z%DA^=_q=IxhnEz!oS=Hspq#zsgkIzGN? zBd~$dn z5moGh?%nTR+KL@?BghMly_c_fk!Ab>5-u*$n|_8bE{037%7wY)8s1#;iUF!uU{o;L z_0o;k4?Q>Ph9C8E3p^KKHNe?tq|%}jwucv6?nzVTJn8}7SF6;RaAd%uIZE2?4Doi% zE@@0$wDYy9cMm*EF)94`PIcttBPt3JnuVE=l;w1%R&?*PY=04#($bJ>xtbh2`mFFHb6CKIf<;4AoG|_ItAfz2aD2^OcT#uC< zi^A5^y7bHAyK}S0XhZ`EjHA*Fv}zpm#alQwwwcL3TguanSIeey+O~@Zn8yN( z2QTWKtKF#Ae$mkIel@Es@xXKZRvW6&#B8x^`}$F~B05 zNqSM-GO2Q8nz6~apGp8OXEeKruhvg?jQ-^COXr7GDPBmC6qbv{Dj5!?cKdTDye^>) z(Pj>gob{{eoAHA04y|3X2ah&;cf#_1Zt($k+^aK;JRZk$DiaGXhc^Z+8RWcOKX$he z5p9QV7;alF9PMw9ud`zU7o5X%P1^rWp%TCm$`?6#~&bE zhoH;$+RLGJ)9E?kw%rJGN;|iKO58KY;5$2=kJFf(cvawb7{cLb;YG+|ZP3tXv=pR= z>=*0H*JX~yi1lCJGbAn(@W_IPMn5U%!l>J+O`FYS-wjFMb}w0-va}>HJ(qY}!s>pV z_VQxqT5RgmEK8BmUSCL^+WJ# zz~!;!5HcPsI6heS?oIDbpOQ?^_yf*fT0T4msmDi7*mqDloa?&KkL%JOLG^^2l}KDt z;UT^!j7e7wLl;?U_qE$22A8GT=y%UjeL77^+)PqnqaiQpL&EtJd*OgWAEIILi+(Q=H zz3;A)x>a=2I*C*5ilB4Y;v{xrVaNUIjG)lMwY{|Q6}@XsveXY>D9U{^UUbwgv|GXq zQs7w-vc;lIm@xTvCD2IuR;SmeLVJYF5DB#?&hat1q#Nz%X4(kN=4g|>7^|SY780+n zUE{#A&>c7?aQxQM9EZ|9DQo)+c(XSXgA`mC!qz=nmZnt1emWeT?Q593ddnY4u(4eg znN3a_##3uy(N;jh&aCZgEpLp|37%mjEurwN|S}O(Y4lC4)(X zgZD53DV*Mi?>^?rGVY^u5qVs^*dj5Af! zyw=_p51a_o^EbF|8!aH4DKKJIzE9`j-A8cY-gmV3Et}slun|>L#?m)m)lNu$cni-^ zt?cD^?I4Z9JyHB=nwZ+;_nL!>xXOC!YdfiG<*1|Nhzstt;|Y~#BpcmpX^foC^gx03 zyNOXR<5HSM`(lL@y79vF>}OlpiN@LVx=8xFNczl|^_3;L%D5fE)UgjRHf)A?hL%id z<#xK%q?Wx4OGjRdA#l$3>ttE z1mP@;<#hJ!3|8>Y?{dc9C187A_mu!Ga+%!l#Tz1zUU7L^FpdV-bBYC9ZhR1)P3Y8w zYtmnks`a3W5fCjIO^S4NbFd)PveJ8+FvE0Cr15)mU|p{CR-PS|aS`L(q+81pl4o8t zg44Vg;&tDykgZ}hY==os&H3k^hT?Zk-$b<>KO$_voDCd+w%chG%GQAh*gMU8rO>h! z>+maS+)9IOjT#JxS{nN72EM~M;_#}Q~NChu}8W+wU0GGo~i|Ah0|pa6RZ*V{}N;TCl_md zuMXrkSW;GpX;gg(owRsT3Irx_}c}EiUm0ME;V+dP4~LnsXesa;@c}W1aD1z zG;E=@$zd_ytfb)+8`k*5Y=6+ce^xH=SrUkz;v^$KCWL^@kPT-+fOTElP2{X};_c!f z&bxuO(t(r&IQQdr~tMS>S6w2w}HBUhx_uL;wk4{js2uKMhTi1bjGXL4c;bk-P zhkZ7%ZMXF)Svv04rg=7L`C@(=)`c5IoKBEaRJPhMrX9pjfaJK1MrmTF6vkbYZQi}O zL>%r&N_HQ#ziVH+5+tM9E>oOvK0LI@Ypm99J9nq!YmjnlpFL!8UU>g9kbo+&*||Fa zq$OXK+H*i^_-)?F-3bN|QUM9H8!559M1C zbiej+#9MK^``X^Y_f?iL^o?h8LhY*WLh5bx%6fdV>S^N=>ovOk99^Fa-@k}yd%1D% zP4)+vw##Xlr#-2*^y44_Eiec+ywH81GnY5i=m?j6TJDOx>8&r)9L?@pm99G%i*OgwGurb6Dgu&E z-O$}2H}i~wmnVTYDeZ2c4uh(qx8qYT_3;IWnlx7D8b>wKz0n}2)~X`GkX_!Q_>x({ zsk{OnFNCM4>ruPBW{@v>XxWpU=QsLXRyk==d`yz9a*2u2ILHzWzRMB z(Y4T18*uc=An~%a<~$FWK{PQ7^Tz2k%BO_GkCaupXqgahg=M4?;tS}zD zsDPGSm&AS4*#qmAoW+63dzZjQ3b!1;yIR|VMkIE9U*9JB>p)ZXmiW%tv26kJ$pZd`4jdExmBt0~gEi;wIi`dBa>Sc^nvQLc=`X>I z0OVy+O;7HQXJP1{EL7ca01x#W0+UzqV_xM@Bw2Ztc74@Gn%!(B?S{EW=&!&;_wr|s zzJG3^GXfHRPiRGLzV_^oKtUQt-qZGuPwSt9ywS@5SFFJDC{*#;D z<|-4`{@!wN^FIZg^m07MtnAAB)}5B3TkYD!suOb8IL9;bewDwv-pxb4Yb05p=+Xof z9A#jM&5J%u%D5gqct-zsD1FIVr#r$C8=`2 zZz#+B23%A(cW21@v+H_?!jf$(6Q5o-t?O7MeQUw*aZOn8U{K`#pweUu3xta>on}!mM7bL8}#pMs(Jsmtl49 znD02ZGB5O@F34;h4UC8yLaj?=FS;Q4JX7d+U~{ux(m{`7xmfoNgj8!bK^>uecb~jU zTeZbS4CKqd=N{dWKgn&?mw>^?!2cg-E!fCP!w>V=0E)-yhI7XHvy#@}%Z zrzga@aTgUR47fQI04c~1J>|$k&!u45e5K)$U&J1Nn>OXGj%_ew-;uFvss8GNLmJD5 z7H-hR=M1$(w7|KIa`hK!n{?NtlM>|}Uhd!zmuAfdwHqx?B?%pcMjA|5J?VysmGvHj z2Xxd2K!&~AwVlCtV7BEq_g1WFjdo9b`lczA#ki~ZKJ2~Sl}zjFI&r61mmqpF30f6Q z1dqX5%eoynRma;NBZpIRZCeV%Td+*7ObhBZ+}`ttbiRW^-qjuFdO;@o$_S$L^9n#6 zZ=Z|iNFWU|C&ouK$1IZWl_*Sb8)N%?1yO$(cKPk~Vh`kI8<)~BCH;!(LOwj5gM*;ci`3H(IPCUh+un>;ru}3I*wm8JM5_I zUYiEST^n=WKNDAx&j0cN)%x>Plxn@fXap>0T=af5r}E zV)gk`W}Y(jFLw5MRvvI}c+imuRXQU58r@Qu3!ZFqDCA+xy7HyO4CfxHzWfMcz2`{L zdHKDIG`+Q@U7>j@i=?lf*(6fyoVt%j;dPHF=R?52M*?y}0n>-BltNf#PrN#`{`3-` z5}x2a*LLPvoKY)z+b*x)Db!H37a4m`to*pb+wse;wGW(Hif#x&tFNfB5LO5Z_bz&3 zqmkSYyTfy+_Xd^zSo)C?Y%i#bIdX$4;r&C;G@B=|jhoVzmIS-S)3XsZ7B81^xM&z< zSsr{Kus9cctwoC_ zpr4fCPbxuCQg~hTfVX#OMg7%G$3|69cC%I;KS3&bdCC=ohe%b0?gQES4jTKL7F`9B z@C?St|M}JRK{mLpK0Om7au$VQ+%(3HD>$leGsF!KzDy6lnizL%AvPAh3-8!$P_&x5 zrErfCp>H#nejAZi6H*(i86m`~U0TSqZe01&lVE;9g?g{6^ahM&t}C^!9Q&k;$K6dwh5L;Kn(BcBLa-e*N*yf8ZMDqY(ORZ3R2`k9vm(@ps^OeL z76g1b>v6tc?y1UMT)i=@f431|@%kG9&mR|Ua+Q}80bZ~$XfcLOa)b_4TbRGJN;?uG z=WL&1dP<>M$@L~K7x$tj~%&%-|uC1idzRs42x(q!j!7SYBOQ$&Wrha3#hP< zxfNPW?A!kMOy=6F%bWJgm)Gp@PA(4YW8>G__U4{l;2EJ^xE|VjTi53DBZWrN4tG@x z`6z&bmUC_e9M-YrrLCDa^wIp(u+m27w6`y##xLz1k3a|u-l{i_l?B+_A(ynjrL z!t<9ul;jR7RC@X4cyUC5I=HZP4N9~%`1r3HZz8UY$>DW|I;OaZM?aW=^>(Qr4T68kKr!qRPg>(clrixF#cnU7`zj{iV_AQQUVuptm1 zctHyE)FyYomqbu`&bQcw!G3z)F5X0v_VNf1CY%R>uOS{AjlhYqB}-=8^52jjfQ z9(p2!NiV^`^zELU&{D`@4LlTiUesc@jTVOaMQI%`PP05%P+w~!6)f1)Xp z1*|5#>y~ZYZeG>{2XA3VDndL5WjX-BsfoLZ%6v_LT9 z9rGf%??RU#DlgG=;~kD|^7Xd)>rM;NoLY#@e)p$rOafy(FaS+raG*ZNcWh)Z;{2jG zuEw(h20|xqPF|pDJexw41_&$~A2 zSxdVscU<4b=Hk_$l__S&rH6!h@8l(1-hc9~mCpX5x)Uv{H7B&?`_BtFJ$ZjwOWOX9 z5vWA0;=0tbng(bY#C7L+K^5dxf#olfj1>w3;YRNl9N~&9U#h*eFT^GP=}MoRQ0y6= z`r0eY!oAniD7+Wjyv~sBUJvZpZuDV%k?fn?31$TQuw4hcVw|eV?u64x+4uIfFLaRX zA*svnO0Vcx9#hqlq0WOpWR=@Y731j`9OcXM>Sd7kTj4RRMZux*gQvJ<_c&TJETdw? zMqI4%G`K2yck75icM?)pL#dwr6uf+0_Mmr!X}S1G@3-O}V>i+?6zT=)UY1yv?kh}d z8gbTTQxH#2qyInl-YPE2uKn{!(Y>DZvN4e*$j!seCl`19b3@q%Vx)^1J_jRh4o|_Fmh3!LVoruWOKDUE1$;2zr*#`_aCOgMx8A-*;R$?+&`c56&$xVrle{UYE|Kx<{Q|Mn%McclP z_>(Q;X`=eoCkv*;gW{u7aoQ~FB6Aoz)O1LGb$duJlebIp$&YUXKbG648Ikv%J0_A` z^qq(~&XB8g%-xl&WvkZmG19;xpPP zs7CXe_1v$u`jTpwZtb!@^s3%^oGeKNlVBtU${5h$F5#Q(=+1!T zA%$V3%FIC|H9BDyM#X69dugKBzLW3eUvIE%>_1rUZTZP?mI6B2jTW=c(8}EFQ!YogZ6(-$>g4F- zHHPI=c$0Q|t`<*w>L`=G!*V21C2u*I6Eg}9x~a<=2mf(c?*Y2FcEqGrIf8l4f`ta; zC|*kS8L?vP!CB>GK9l2{?yITyZ{!^WJN#7m{t=42B2OCmG-Ka-e)b#t89oZ$^Z5Maq9S_8pGbcD;~(P4Eyhle z#38*eG_j}JT~N8FIx!B(>;jEqlY zl5?MSTbRO{>vg&`agClw#56nR9L6KB2%y@)cu6444%=?IDwi-QfPNB95KOPR@7 zeuysSm*kcAGr7<{A}Lji68=1R-CtX{!KMA;;~ZVXzVOOyaEmk7dyZ*p_oo$y4uj#v)6K ze(L5E8s7i6>6s?rR{Otp7M+W1ECzK{o+qLkjQ=(+-a==V$s6I78ij-pIdtDS@KIYt zV7%fN#KLZXr**tLKb+WV|5mu)(LAi>$=#_|;k|*Vf&p*N*MT2#g6sb9gam<@EH)27I14U-tdyU1R z>x0Hky?K*pokf!-rNp6?rwTW7GS_XJ#T+A>T_*g%lOrLKuJ? zp3bO2Mf%4|+gLewKS@Aa4_-HrS`~M4gf%1-_~PuIYslw+21!b5Sm+b}IWy^&meZyS z)ocgtZNJmiF_-#ImW_MguY6i!1ZD8N+fm_7(bhvmj?HZT!NI(~)t>6$Y+gf84QN4+ zhRRO^jzpJzDwmsGfq#TEOshLiD!uu&6HSGUFzogcH=*qTqkY>Sm-jx(wFT1t#^1hI zYH*bMr~cDHQ4|3aN|c}Nvr&0W8UK?L43SPENBvlK&h)k_5HB4nB`pB%I7k>fc>nR? z2)_62>D&Le)4dA5QYP|I)d{T3TI^OJF5?vv;FkKsd3&q8cbf0L&*tKw5TcL73s?Mj z$)Y~N%1Qc4z`H}GH+5c5mK}8>`XQ;TwigiSAOYUk%gQSM%5y9u6-y$(F}HkPGcylE zY5AolIlOI11KK<(7}lWVl>Fg`{AuPFk6O`)57_O1j$hqQg@6N=;dIYi3%d$uW=Op( zyleDbcFHVPyV0>pZ)vfxoYUqLaPKZ0Z42>8o926`a7w1IryHK6B0P^~TngEo{c+9gq6GA;ve7@`SD@~|`jzl_ z3}`yxH_*H8n+@dN?SyHS*g6rJ{$pC0jEu;$u<;<|sBq%u1bEMLng}w7PL4O`c2{a# zb2U4E_TL>=5*IG>obxISeYySIZO0+?Y_w(ozC?-P@gjRyYP6$rM^G3YZL=yku4#hBp$>-ad#E(H6Q^5yeX%s; z7sgl;9=%GSJ&JdhQ#Np#;!-N$rf z;fITgUCmcZDZV)IUg^&OCqx?{O4_)CUcLsu)7c z08=l20uF}V_hXt0YlPG2mVA0YjE~1U-TM9rvyQTPp_M=aI<}w1Fcs|dC zA=))_uMN=VpawZg8NJml50pY0yi${g3)WfxahoJM%nScwJoSO;?4Hc`n^_~{z5!fh z2Jct6PH;;c81YhU{lWEhVYy54Ze;#jDKU0*+!irux;2KV3#PCkxs>z|$_{Y)()SAR zkoaw3(SwKm(n<4rKM!-*f$-#S&)tfqOJY-L;eSPQT}`w!r0gMl<_OwUHD*eTE_)1z zD%gAAPtN@141Bq(IL$57D@*rZOYyOf_5XIOYXSk4i=8XRPgQy?4lQHTh=FP7;~t~R zMXA&EJ3TjOkwga1POt{r4@Tsg_O+VCM9CqJ4j7GhI<*oM$~{B83+_rB(T*covbf67 zl*}bK-4;2&nmdfUEe?IiSQmDHBOaE>HK5;i?w>4GDA%p@SUK*0!C zP$tx_)0-J}Z*hTF5*=APd=+aSt)XPy_;)F0KrGSwnBOZ@ufB0iXd;hh75+ql-x9Zo z&7b!P>39T^N8NmWz7om9e~m}s5QpVk6jklcwk!J-sD;oV(RA|HBL;Pb^r@ z<~t-SasU=XDOtetuW(v^%NB5SZQQRdgHLB89=!a$w??!A=9*DF>8S~wIr9L9lo8dH zk5<}Jr-8aX7|$U_Ur^WWD^e#vh5!K)XL}vF3_O$Qml_SS-a!D-F0DSN|86Lqxar70Kq{-SWXVgLqG(yn4R#(w4mXebYEJw*Su6 z_wPtCJt$Q@-fNIs?dL~-jrx#!BR3}%;PCN|T@`nI@-uzaDanP+5)2ZO(`k9HJS304 z>82O(YfjqlMG6+=yW1Iw<(U-Db+Fa>KXBpH&`KrLr#i4|@Ws*aQx>95D$Z0l<@f?9 zqc3mzq|4~3oj^EPn}e)z8Y{c;lNaHdTK6vN`$k&-^S`>BbeRQ&q_E~{%WjLUHj zu?!tngVHa@>~hG_WT$11dc{8s{ChBYEc*RoJK!OBz~1+7N7}$cOBZ}CenI+<+`mC) zVs^nb71wi4iLX}2bb;(UuGd(#s*g#2(j-~Z=4z#GxfZnN&CvAGtw*nP2CT-Z*t1)O z@FoVi_2Z;VFmK@V>-qj%&+b1|@o(Sj-X$n2!Ps0^>bAHz=g^M;ejJJ7+(CHb&uZny z407?M?#ft z0l8dQ+jbjBRLV2;wY)Q(pQ9)ZnNcERZ~B{A2vf>qhS zb2#>E_`tq7>U|0LDM2|p#_iU^%3euS&jL-%S$r%C;uMKY)`hwCcyoglS19xBwRsc; zc)9v$^QPZ|Dm(&IRvLEBKEFrO9*Gb4~fMbqS360!}uET5HpP zZJt$36;`{rN`+{+Lk_jjr+X8&5{U11d^O}Y0LGV zanA3mcBi)di+{EaZsJ$ZbCec2oaO^H6Qqw3gig$MQB06iK*U|AQFTxVJs<_#w5|U7 zN=5Af1A-42w=voAOx&L!^rMuNIRTZ@cQHWsiK~db!6JDB{}1Qlxh095;ewyuj^h`Q zV**Q3&P*Fp{U5{hC9r{zJb(!6mL`gZ)(k|FAhx}_KOTfDqzvDLxz0#59s?qN?3XnU z#m(cz1dLRBX6{u$q6U3mV##0W%A5tOTNlUk_DXXF1&sv%5(pz%r3gj9ckt3?KMLYy z>9T<(I7R*cU-tjs*{vS^f6I-<;r(Y6dq3}uHJF^N-6;ZQzq$Tdv_O5tuYHEtM8Hy3 zn*x;SOIT^q{Yu>++!)q;0jhkA@lqmwCU%QHxVdXP@Kl$Epd0F`XUDguRV6AsM>l(} z?`KK1EFC8#ydQ?zG$&TT4RDR_55~Z&cl1C51h2j)YGxq$_2e}jbty!a*!(>QOl z|4q{H6Ql5f`00q1e+rk{x(%pLiWpB+60jMF6zGVZUYkhTUI;whNw-#--<)ZM1-ENR zjmMPYJ>xXorJ-bjP`;vjj@>SBXz`c7s1M^du!xstApRloXOrXt|BfJ##I3t?QIq+T zcFrQ4yx%NAnnQ0~gkkm4`l)e&QO?5mr1-$JpJ`wE&)LsRry|P{9r5GOj`iFM7x1vZ z^5^vs2gCw$O7Mj%Vuz*T0;6J|k{aJ|YgG7{b&cbKeGtMkzmh)06y3-YIgU|ivjzLm zY2aT~3`T$bqx!SEK*Ekx4Z0?q53BRp+<;?|ID$%+V~W$qUHyQ{C~v1rZR$K?%tlS@ z4RPQ2T;bD$W`Fk3JC$tY4uT@aSf(HVyC1FWL^-ya7ypC-kqC4BAcD8eY#*@is^u@c za>>>qXfhci;?5eNi~O2u$%-oPscQ^pI$HU@bSfTNWjlCX)%!ETd~hvj?UkPPwwo26 zt2LBk!QQPMH!lrj*j71XF&Pk)w_Zdc%{NBrDI8CA=|$ej`gNAaicojYPj}k?Jr`YU z2XU5C2-Qu`@Tau#Kx$GhZ|$v5CKnTB_FLM|{ngX^duGdrC;}3|rxwg%^GIQd$NTi|_lgMYG~RsQbme-6o%d+vjrsyoQPKllVNo2~KP1rK&k-gObx z#$-U_O##1_Q=k*g$|vIRO=xjnQGMPqViw^X!7?RYS+Wfm;G~S1d@yX3Y`GX2F5RMdQ&ufQ53-U5b zMfcS9e@9!n)bgLuBP3%7DuC}l7Xp)qWYeG>AyIIH%Go%SM5{K#C*|5ToAj5Plch>Z zA+BCW#sknAQD#|X8?ql+rEdZlB%u-OJsD=Ljy{{;cg9{b5U~U6;^7?LuUpBHPVpc> zv)4kbf#qlp1T}S9n00Yn$$R6vModI@(24Dq$56YNhTGIud~?#U(@ahMrLr;E)j0b~ z|AAf^L#EH5U*6lgwM`FVM{*!trE~OVvUIt((v|C@nG@wr=_79C%t=qA46WI*05&|Q zAuY!X&*XTFBrpkH~JgUKiH#-nIV0E*$0@*VM-HPg-#5_jv|~-qmn8zd%-IgPh>}V zZ+a&G+LI;yA{RSb&6EP$IT->4AkmqgG!GjK2~9%D?MYtc!Z8$qUa{B&X^>hzSJ1cR zT)Xtg`n4P<@eSn&SQEVNduzMNA(*R#wc1@6WUfHSO)T+?LBH*~Wq51~E@Ke*J&gs(VW^1w zrUdlO=hbeiOakHIyfyXAuXTy_eugumD)wKy`nH0ccfem4-Y7satt7qK;L0#N>rgHm zO1p5v-3Y$+bm5Ih#8n{AWfl5$|510okG2Sp#O;lFPr%N0z=`R_R!=jj--z_|W4%2H zA>U~ubN43N3r#yg%@X$cT1{k~<}-v}!MSC$MyjKiQ(g3$WR$T;ltBeRS=T9fqg46d zEK_Vatj;DiHCizo2&#l&6SM&qC&g9*`FUX!?g+^|IZlqYxB`z?vb}9 zuGVkwDt{lFp?@siKOLqN3}<c&*te7n!rxM*0K_r^ z6xnjk`@R`W=|J_nyKnlwe~Sf7Xa1!H!zC$j zXiG5MgJsL|ZPMqIC?epN&##)bjzI4b^{;pq)f~gzYdaDl4-}pI-dq^KTr;`F72U4v zr~(L(WUOzoPPQlIbLFB22&6>#!Ie3HY*_7^wYczp^HKLnT@W!SjEmI84I2_B@q@~# ze5}0Lr#x8QMYsjIsi*Bn2H-G~T{Y*gB>8tGrG;8^eI@#)Q1bl8^uUCyU>8xo+p&_rJMM*q(f&@;ck=2CYB2lybezUodnCCDDjG@wtoDX7Utz z?s=@hA~A<6>KKsj|04KqMnyf**?;pc>vc^Xm-2)-5L)`#PmBo5wngw*WBL z+xEt2@l)M24`f)8UM(}|m{|HG7Kh{^QmcB~(D?>>D4!vCM>`__{(HuCEcSS-)w8|n zT6v@d37qfz3KVm=&o3GmeZ6K@*LuTsohaFb(~~$LMv?vI)yk(rw-k+eG3N7hac{mQ zewQ@LG9E+6X%k?b2%4U3jq5f!GbM891;(q0;DJ6N3G2T_CbfkFq8x9bcT*3Frf0c7 z{7$SwijzYYODYGydD%bG=zjvq^gaxm{xaL<^-=YUcelkFv?W=Lw6fcx;DTofl8G?Y z>sHmu;MyJkoPFFj$9e&pp{NEj#HsQXlLVUDH`NNb$j3zjKP7V&Bl}CHbUWJL7N==h zH|$Da6&&k6pW;3+5-$~`9(XgW&e5Yd_HSK7=@wBH2Ku3(mSsquS*CEsNy7|0{$C*N zrxwe)BL)PlP(Kt1z@IJ}8c)!f`JKliyzj^&OhT|xSNrtGV6O`FuBc?9t}>a9{X**B z+}7g6T;=+(oSkUj`N+gVFRDISvkk}V8|B+PLw#UWSJ>^0mGkw3me|HJF*0TWT(k)-}@6H}>(1$~^tRJ1-~l#>7Iuu>b*Z$tl%hD!C3 zymFke;{UUiqp03tG+*_(HoO;9ak<~YF(&`^=|I<9lgYtkXL3wf{DKy=K=s*Lm8Mg2 zL3G3@P{RM?le~Gb=_rq@w(@{V;6Bg_Za&S?q@>XToIz2Pk4jahEDB=+biPT?ioAN^ zh%(a!n2!~01tgCJI3h;n3w!(Hb_Pb+d#_rYxYmZ|{khF_a^DZmz*;WnO$x8~TK021 z4v-%}|A`#K?_k&y@%Mb7e>4Zl6IH3!ZSh!?3x5nG3aiC_{FQ5eCYC2d{S@w+PA~I0 z3%~?hRUZgsg!Pgnb@m*^jC<@)a zIXwX2d6z(g+Q6?8kxiPUvG+<{6vnOjFr}odT9ZQL=-oU%aYiPHG1d9#v2`CtZPP&_ zxoETd?pDb(EMH%qF2TH~r2dn`Xi*=k~FAE$8g~Gy?!L8w$)?Qs(uTP?_{LR_Ud!%xrZq&SIjmvH> z+y`xVCQ%!U9g=&XyDgz`l$Fybbsm%eBa+P{%Ya0zuEfN>4H(`m@O+xN5S@G#!8!i) z&_dVe;`k-#TUE=E4bx0D>-lH^_*P99+Xk`nT(|bz-)w*_aUf!$TrtZkN?K0#2>>Cb z;A0!;&(*x!G4k|(_mPS48* z1rQZYY=v<-iFUD2t`YSkngDLI9z=qMO5aqth{{h3o=W^1-2}KQ^Z81Qg}lN*x+ujQ znO_%yAmbt&E4m1#L29QbQ7oK);dN?z#0j}6WcE&RMj>9J7PaHyA5&UeE^41L!iYg%_M$wfY|mSGr%gQL zT*fLIBb9^qJ0-s;V%?h}hh*OW+7=hHiD3;|RK`0&J-v&7>U;ycQG46*5Wfj@LOBkL zpM4iuUrNDhkN-PAnM6%rZ8s}`B1oAd%KTjb8z`uRDXo6{j;?Ndk`z{$nnHX_0_|;u zH93hJ@~*#GlSRdha0+8_fYe&R*-n)*#_pxklds{eo8pQ>bOLK1gVnB~HdaL|Djlfm z=6SkfWu%e>4hA$@JxLn4J+^NpZJ_%0SgO>rC~uY7^6SwE3*VhoI)kj<854?k zyf#Hg7Ge*umy0G$&hCoQ3!^!@&bQvaLps2t|H1Y010>x1YeM|@T>OP?0_K|h4?_Yb zdhZBkATMu5`U5N}&MTO%{X%|7O^*3L?3HV6uQFvv4aC5aQBv1p@Fwzr%V8}1wLY4$ z4hothH&ME@^6S%;^Wg<{6^sf|O7thvKUQmT?rx$4&(tvNq;KHLkj2e!;)31z?AoYBLrk%2akaKQ)wvjByuTdSt)4F% zVqqtmh*CHBS7kWCI3~WCqMES=flveP3unqI3)&YaBv~HUe^x^ujJBe z_97h%#AB}^nNC$Ef-g*%9Dk_8r`Ez1yDfh0(}1Y|fJ->Z^%3~&KCwtj614r{=S#aj z^MHN13-@JSdiD@%OL$q4i`Guj6&%F`=Q8lW26zic-t6c_=#qiBgCO}-4F!6K2eE7H z!;U8VW0{2W`5L$Nfv^Oz*UpW zDe2He7`w|%?YldF&hKTp5M=t0p(iu%&#AM+>~J33Ia$2(I1wkqA({-5wUCyj?ZmA=)02em%h)*^HcoY}f1Jn5HC;6k`0`)un zV1#N%9~K9DRu%&~2?D}Hz?}183H)Td_Bp$bkU?H33EF4<4(&g@G+#c)g9AABH~+8L zBqS%Bj3AD}i|&^S7-o5`l$+|2&1Eq_f(F-=zt|W@fgBCvS_BbaY6)b@Fw7B>cA?;R zD|sqot{PB%ms+aK=t=~w5EkEej2BuKKhNcJ2+7c05AQrpf;jMDx8Jac;$&I^JmBnR zDMcz_w4W^w%?+V0DrUb=su%*#Jqfs24O~YUJBy*-?V=hk))8Ygd-paRN&d7wtg>}M zJ-O;D_f{pGlE%B*z4XtH@K?QtoKhd}QX(`0R9HlPtQY{3jm7Ig?-HGKjI3%@X1Ms^ zIAY)IRXXE)0|^`5^dCT7-|8q&O;a5ybC@c>cq$f;9-@VnQ-c#_1)nDM|K06V%x?=d zDHM8t51=EkG3ZQoK$l6;hSnE{XUV-1_SkeN801@~sKwm; zyr;8pbZvyS1V5=kH~Td2ev)LSC;e+YIBoj|N9_Ak3}HV`a5<)8{EuwUoKKxCvwzz#JAE{HDS*l2v%01s zW4!9Oiy`8@)o<3Z#b~_Q*O+*zJ@b$K46QR_;sDYa+|tv(Q%P==JL=~3yDiz?RYpyI{;Rf4Mk_=kQ+Q!|H6p&O1ex0ARMi!XLQj+Th*t4iDn8 z7u1+g+k;w$ptEhBkE#XoLs_6d@h|Z&^MFi|l5F_~s!Tj$G+%@TTQA3%R;X~g8_MR= zdF-E!Qxe-oX^^Tqu`W)gPZi(V`8RKVWS3pdYBkvw*Ykt?(HAFK_Ig>Do5D8xILOD- zKXrm663QnCLbb-%d!yjWndi36)t;y)-Z$aG+0H1PeXP${xZY$l;GpvZRoI0@cJaRC&M&}PwnMg()fen<%hD>(_J$G?K@gwfXs zc23gPz|0B?r@J%dumQ0`kmQ{QoiP>uzr+Ns<*JDsLk~ zn$iu3B{%oG;FJmooP$llqmJ%`7|CvA=0mgXiRa1dCw%E6>hj;D2FI+CquS4J!#0@Z z5_3SMHMKr#9?SW@)G4zuPNFNxQv*tzlM(QM6*=qvrBVpsUA(!w)cO*y*IBUOIJ{Y|~s-dUbdV z{Aqek_rm#uWP*6fJ1B!>JkaULw-8A@%s4q24?Xb8V;j;Z-;0ev#mcQ7>)J2FLVxHU zSKH?W899rRN?v=TJ6^8Rjk6Bn!iK&7v1K(}s^thvXwlktwg32qNS@kUZZG%S15TWP zlde22g4FBut6Id+EO#S(EYKvR;gjxklyW&OkG31pdGu$$^ZVKwVA12>8UMLGZo>>p zuebl>WA@xNh5u}Ql9ar)d~1H-KcAvzp$nMenKZ>|UV_4WfzJo}XuXGTP}jju+j*p16yT|$iqG*HjJ>P;U0X{&g6!ULV9{%Pz3g~e;)VMiJ7l5mxVRI(&uuIV~) zkoatd& z$)gA$oE+$~Yjkg{cjzI2ELt_+w*Hn#vfz4%bEd!B=#lu3X$uAu)vi2p^Ehs{(@Gtg9u=)UA@GE62mI!HJ!306ixnSUTdc)*H= zniTB=BBH)PthhAZo2K&1>cyJHy#bYyjADu2r&Z$fumC&&A3;uII6Vo%uf$@5Unn@e;Ay?+HBK zlg4T{De~4($7+|*unNh%h8Zw6da78#Nf!gD76HSo?Ec@r;eyT~p>kS~lizSb zT70n8Mq3F=jMDgF3E|?ksFN7YL>1Mo3qX8V@@F44k7rtdUZ2REZvh6xL=32My8Zcj zU%k#os+TIQWT1|)b2nF95+85L;)VC*aPIL_u74uN45*uAJuWQYH9M+eWNpl!M4HO6 zC6PwPOWnRDF7?xT_VEs#jhQX{#o6%^(p|f2A;YtGK5twu5-$EpCziRE{U)euJ5qF| zHy~vDXRY%_|C;txFWR%|JmKaM+E-b`HP1Oy>fwZc54Ds?~BPIMB(2T1D%aL{0qU+td`b)as%_aoj{0{I$ z42vF77dt&&y*xA%$>gOdrxuOiaac^_Kz)=v^wT^@4t@4-q^r*Nud7HZONSz7Y7G_O zf)<@qp@X;DDDmvir|Hf_BtH~)=N0b0=S zAHQepS6^KPKY>q?Y;GS^dfByeJN7=Ln0AYi*x+7ibSsX17G!>#qQSD+{7`UoX@=l~ z;&3)Mw(qLDGV6StOr7W9q${h>UAD<1R;C>tRYR#)EYW$b#(vUNpOxQo5r-5;q7%UO^y!s-C3v-EUsm1iL<2<@zSQ_}9iC1vE^!vz<8| zFq;zu%Jn-X4fC+t;z|ZJI*NU z6EGcb)%6EY^SEs(!EH=LU)U=dzp3whF<-ObrZ%`LD^^_oS}Vr+%7K!)#O0pdw!|+D zhqDNHk}Y*cbDZlC5hkIuv`60`?w^dR#6(7(_#MSV&+mDw=wX(X$$BuMh#fMmhH2S8 zOZ1kk)?JRcYkK^itvl(acV+flD&j_Wv>s8WKUWiabzkB=b+%~g<$WEy5K~lF8zvXY zww!XIbwSO*O^IJ8b+387JxOV=+%{-{=dn+5#E(F?RDy*_hmkQvatK^Ny7CXOmfwm zZB#KEAuT(4%HIpI))JpB>}mMC`8#(!3EJ}LVOF}Tn5va(W?l~ILe9Npp0P>?pDgX8 z>OOblT%54cg8aFN^^yGhl&{MXf|ZKBY)0}_I8i!w$Yw~UlDdiTt3{vqn(Nd4MRv5+ zsM=6wl3Ft6a53yUck)_s+J)Zbp@@2WxPJHI)uDojvkx`<*!xX=V)V$;@=rx}fUZ^9 zQP3C9G6vF^n_$AWW`*6k`Vx1gKPd)m#8WKSug?+PgLNff`jP?B(BP!|MlCJnu}M@l zkU}1CGaL+VioR7bdPg;Bv>>-!+_-S`PRpnk#ZW2vab%18qv+1w7vV9~IRf1kUYBO% zd-3`Y)8A0jW*y_4j-;?2X?;#VK(tJbJS6F{aw4|IE!$0txPmcq zJsYDBr2^v&iS098f@9~l!U_z8_!2+=flQ{=#qnmCXehJ~KLBK< zztmXtzt>%HAwvJ?PvB&1HGjow5J_w_19{pn#{oVecT^owDC!+s($H*ZF?^moSj!eP z6f&Tv1yfQ$$eViCdD_kVb18dtSU42}DbRpb-jYO-YLXqtHL zs0qWQxapZXN-G$7bqAE7+4(D^<(JfKW{cIsQ zS7UMSWN%(Hn*a{>Y!-ld{1LFvvG~JCPNLoGx&VM701bJW|2kgIhxHSx6UXXL*2@G| z4d(hLVZi4J+-7ob0DZ{WwIjjoe*Rm-8Fdg?{ZRD!gxApm7DTeje7^s?5#A5HVhw;} zp(zj04G2$mXOjRXoGjRNl?4C?773FSVNn3CVia&%ky0{`PF;YwGy#=>CReGd*Z~%p zmSR})m8QVwihB7$kSpq>#Q6Cw?HdU7IwyxQ%k_5v;m>~4uo!t(2l|0bJ{Z>}JK0~% zyj?a)eiRkeV$f*M)GDRRn(1>&=>t9j5AW(81?L}C$$zq-8mGh18x~keiz?K`mWFs- z0R$3jUKfA^LT02Z}Yxb#JiE8e<)sJOI+ClI0x6g zIpGu$H#WEZ1;cY~Fg^KIwB=;N0HA`Xlwte+(zSiO*x}b<2Ql`bym@FQ$hCY{WJv%% zVXW#E!X<^^qb+g3KH-r2v1vl4dl^DCO-Q|5!R$mrlUX@M7VOPHB58`^dqsQc{{*V)@@Jrh10QpM# zap$L_wPHy#tt&7ks7kLtQDLYEcJ}2`i@N9OcJ`J?*na1?NhA^7SO(kkbS7y;-IcFCWg?+nvT&`FMNqYneu3ZMu;7hwuf3SF zK7WN!Y>4j9un2|wErn1deE7GK8&lz^SLSGjOa|i17J%G=WL|@I> zt;x#FDmV@;T@kTL5K*m&+txEq#J!hgi;f}!(tiXT7RuUH=xqtyThYjf-w#oTVB;FS zd2+xFm~uX`4)VCZI*+%DVn)TW;u|xg-Vzdu10;^$1puN2kL&s3ONF+9C|b#d&xu#_ z!T$|TM!tRdpuq+)R+R@nRU?&;O6&co!Do`YMCe;)8Y@&S9dG&-sP*ZKY1uE(5uoxf zGSnf4MqJbT1(w7$jsDBtE=qbffCm-EiaJ30dqB(lbLYS=S!=C-x9c_;QMcU$C&+-n z+!dWN-E3_^6Lu`QXDXBjH$;6z51$e7L_Oxu0UZ|8RaCD?^*5;Oy6kbiaY@Upks+8| zLSJ|rashpIA31nXyx60x%o_83>h=YtHERp%V0J$rD=L{FY$O;2`F|6TEIb_BFtZQYB~8RP*cY=R#jinZ@o?pJy*fFB)f>3K8WN>ElOO zzMu@P>0L8I!pex^2}YZ(bjhVTn%|c&Vs`xTLi$13Ly&Gp9Uj`w9w4ce;c`cXA0Qn- zYqKeF9)Atfa@|S7+$WOO&XU4pnouXAxnunK_TGJa@x`RA9F5`P5U%Sb2(1L|j8|Z# z1>$l0M#!5wa_^s;%i9Le{`Nkz3{L8atw~Ml@f@#AycGt6&=K+qghGx6r%n)TEZ}XD2`*8~^>}2NqExi7Mjh z!)q{>uf`tyrM{PqPJ6oAnnGV$-h)73w;d=;9oyf$ErBtW#JGa~zvO@27x-s#$3X#Q zK#u+*DTgLTNh@ePQ-1#;{(10d*8|Bi>0oJ-m)d&|Mew|rm}moN@sqnG$iF!5M0GN| zJ3g9|dJHW~p%r$PZG$dvV_M`u5vfE;E&+3vd|}vQZ=~Op>C`?^17GJ8%7rkb z44w&keet}XDkU7xNt!e^^)r#3SjZWTL9jXVSt9-9Y;ekU$u(Am$X-|O+$t0v%7x&A z9)}Ghe+36A>rG{a2cjs;DwgBLBywf=f|0h%c;2|iWi&ZHtwd6c@7#zp4~Yw-VTND! zfwwTqUfG%>FoW!d7?Y{RzQS=M%E3q_;v`FU5EYZf;bQ+abd-5>CGSHoj=?hRA!4@#XkRCy*X)(2b{3*bkfO|zJ6VHw8scxd1R28 zUX8iBaeNi!b$*i3TPl-IWfVE9p?a0h-O|-5hoCCT;Ki3&L4LpFLMpAQePwj1J^TZe z8#eAuf2uN}fWglWv@o)5gt{Q?SJ`dPeoX{5bgBj;5Tz7GY^WOlz*iV6C%s0rX$$w` zXt@S$pFIF6r-nkDrB7BkldljrL8h6$cZxN?HEUib_NQ9zVCc>`5(dyx_`EcjB zcfIh-yupzK{qh$+q9FFg_Pw`@;#jW0IqC2BHJOZPq{j|`%)`aa6 zBWiGlrc#2%=(8>VZ`Jf*+15|7ix6pv3n#YAJO5T>r#J-#wZC) zy`$O|v@9E`9j7~bJ4_kjQ2)OgGVU>l!+qMdW}F=>BMWC3_jFVwHol8X$I zhVW*WECYtak{D2=?&`)ykId=mTV03piP=JTgyArguFq`Dk_n&KCLD(J@^t!v>ubbo zF$aQ6V%?W3l}5E0kNNChl9x_?{qDWozT%<$bFR|3De~cP`b(`+%{sgwx6E*m_NR^}w^N2l`f=8csAjykiAb|o=Uj+)C7@I$DVH$r6?m)SJNb?q zWYMX;#SKC$_$+gB>|lZX+i9VJhb>swz@Ev`f@Q`f5Zc0ZB;)+ncqt35hs9s*G`R>3 z(N_qnR|D;yKl-P1!QS*JuyLOITL(?($dZ)s7emJh&WssWl#~8*6@L|FgiO{`T1i4U zk3lRZu2{EtcN7C;ksx7yR|deddMG+X5+N9Ga{(5;j;7eOrT#+aiEgO6zKpsdPM?GNM>8OHLbIU%v3ID6Se$i_zc7` zd2hb46-zF9ZkLxlNc++PdV$H<(O}EGMxhf8Q1*CtBD29SSap+t&$yW$&vN5Kh?T8K0Qt(OWkP*jBK~ z$cbHi2atHzXnOf)f2`o6)k0DTExSL_K5#W7x6(Wwtq-NA%0)jZ*1mUu2>B5D#UC#9 z3nLg4OYEmWbb@@R8;CvCxv)Go-Z618HUl)SHvqn^z-p|ZSatV*-0ygzf&lX05SJ)n{Kse!hI$bB9~6l)VmMti zsrlo6g^?V9k$m|;423aXtRWvlF=cqGd=<~IB(S)I!h?!)$Q^IvZOuk|L#8qS$N7C= zO!xZU^^%1fpzfQg$r3Haa{(-0G2ivUR2}NXliOIqXxRvZ)AD!1>tysFnFqy=9l!(B z0r}LyR6$jG6!0U<0mjqyxh3%S6PiO2{iXmKQRBV=mhaX_nwC#iXX4R{UO=}e1;7b& zA%XFOrV$@;(Rv84Q0pyms{c^2bWz;%IIEY&i=!lf=mZzX<9szq<0^Xm82EUZ0P!O0 zn)-ox+25sm7^dg+LkA9kXj-KQx_$TXyv1Nnb|8}fTTis$@#5t>8HC@_pkT1t6S6Yd z3f<~J5VD%TMsz(ad-j9oiyLkMXhrTJM+=>hnsqAT{02MPzu^LN8n#Eu3?L*D1vh$M0XT|?*}1Or@n$+&(;9-FcW-ymx&em__78hTJlGh7}Zz1 zdy4~WRW&i10(TNB@bljV&y?-p_J!bj=Aw2*w5PY<@ zBo$%dqw&AX!_tjr(*V?9RdXxgaQDx3^x8Aq5C+?MEA9(f=yg|G{D_wLyAC%X5|oWi z&(MsbcQw9*mg5-{BBOg}$Vrcmti54oP@L8f!De?*-^kuIs0OVbv;{dfoYtS!;^BWW zQ?IYwv0aI{0@9s8;y=WcFbkSggrJE|pXK{)t>+(`I)#q?kXs`qOjB;lDI8xQEWEY1 zd4J->lJI&boc7|2V@qfsN`JELjE9vJhn?<)zf2)H8vUtLZKJ0T5G*Xx{>Mld|z-Z0iZ~Fkt6odRe0!3>Ae$poT0K+w=DT0LX>9yn) z%zAd3yM^l$A%^D~DRqz8g0f&|G&i_si#-p|qwuy=F54#K_V0{)v-q(zPY_ zR72`qwP9}0XZP1|4XXBya3>MhO_8Z5kwCKqlGZCl;$ZNaKtMx!(*+vh37Fu%P?`st z2?oFvv+La*mE>Ihsta@rvGuOo+426+w=15lb2X$s#LvRRR!|QPh@SVPinVpgS5kX~ z-rs8@IV2C=9?~)X#u{!c<-F0bi(Mi{deoHUM6)dc1t<>`QY0*n*-2(qM2=_@M9t)@ zkl+a7#U@o{R3=4hrrA|a)6Lz5ADOKXf9DkeZf@q&Yk2}?cvNLuw-jTZI#zeY`D`%h zmPxx3@vpN~3#20DsUZj0h463sw(hd*Sp2_OySjl-Z~Czu{e5lfG8)&@qittEzvOE@ zGW2`*A@W0qyenC8%gD9Bo1+0*YJsFS^yg~@C7N$tGhCNb-e?}A1B zQWEz25p5Xsmt7B;k$@M^DER(VIg` zp`kQSA+e+3N@G8~_y$G-6^4C7PKJpCd|GdIqv-m0u6Jqw2g7wH>{E7TsS0oz3!L-J z2ysOae6{K3vyKR`Y;4-|GCRm_lpIB+HFvpI@HD>ldDQE+inC<5$AN)0*~D?+sYyN- z&6kHTFEaG&78}%rXU_zx1+jUP2SnY%&Uw?$MSsMBxB+k9)sQ%4bcP>4BgzbG!hp^q zX>Ih`+LvFUA%5Nt)@>=|u?iQ*{(Xa zk1iBgP=>zV7?s4Ba>IP)T9rtT8l3Nc$6b`iR6>)sSyw-hqS;^!?}447=Ph(Rx1JP0 zQd}(EsW&8AK5W!_Ye(#Z8UU)g|g*8J`K zYXQK#R`GH@6L!qNx54pWMk%R+aknuOR1FzW5ZziD>(zi^e^5z{@7IZ11&^EkQsZRttk1hC&+rQY5l0`z z|Hm@e*1#ijd=z+vUQOMiZc*{{L{e06XPON@vi0G%<3cvnc?MH}Kl)?zhoHMb z%qV&IZC^j4kt21~RoB+{cMWTw77cbb=?K?;uHc_D;qn-hEM$)VDKyI#2iEv9Q-(O8 zt@*nFa>aB2KyGpvR?BjcC+SFh$ORJlCt3l!Ck(>QwX7ao`Z@HVzU(J`6Djvl4#+D8 zU4=~(?Er>MDPkL50fTwcbo@5yz+zrXromWj6j{r>iE*;!G;R z5frTNKs#m+pytQ5M2A0KA;tX{pQMSWfUMN6J!v-jng9Rz@x;L}l@D;X43_qiVNnUrq7KG+zme0SlMZ`|0R?-|T&=VCETb6r07w z17b1WohR_u&Mzte>!!qGnN`n0k3%Vm-&WnNS1F!HSvL4ahN#=lQ?S(9Xuj>r4v0w` zvJ&0Ewc8l43aPdklS(j7F8n}&r{cD$O#EvO$8M+pg|Qrt_<^Fy{&}2%U0I;7;V&8= zz!s3IJZ-!1V?~F(YU_3jX;XU6FoAG;pqy~8NjW+T7Eg-QvGeCOOM8Ma)kHb%?Yo-o zPHg#*A7$W)dK{H@cKRey&9$K)8i_^r zuQB1!>5ee`Q_w%ni{~GET987Q#7kX<99^lgpAoNy7=y2t5~_=6RXRwQ^aG2CJt1+p zMT!5QZDX{IZ4-(~4sJAWdK0xvfr!asz1Nwib>j+~iRMkZfDUCiZPipRQeIKkt&d-L zzp)NgdFVQDXQ?z5X&0VUu$n2}IWu-AV)zcEj=dlt=UxkWx9r zP;%_g%q9S@b|^`dpbl3D+VuAUOEFxoVl3i8DanF#z1iS3X&$W>CRheH7!k^C%i? z2~N{@)#ZI)u!*242&Q)11-_?8#^!pcU8c{v>`=5kGgN>ZVBq9YYT<6GXOO^}G(M}3 z<~|pzIce&c&c2At+MpV)^b}~WaZY0n-pB#-drv`=i^HhS`7}i|mTH1I`^SRdW3S*Q z+`|QPk^3;f`%v2@mym%W+1IVrNM;iM!Yf{sN3+kcFfG#E9qBZfKvEyK<@ z2@2Fb(61%qU;J!B!F1=`ZlXFY)nn%|#bIEUgzAamr4iCsx3O9e?B$v;%Vi*)dr4 zQU`+)wC%c#P!oh->=1)$sIIQprtpGjq^ga7Q$4_|#8NLL(UVegg(47oHyTZ!#+M^J z6m$`0C`<*W64Hv7k)aNN4XvnaDcNdArAC$l={~tBnzZNQV8Xs&^klubUd492&0-4c z>b0jqWVdp`;yTa0OY%c36KDYgS3E zpJ<^(7Mpb+8a8+c#qt=9=kbWz$JZkT)+Xe9oTKCAZ1F!L|epys~Q1#v|XqN9Il5)rj58BbxCXvtT$w{fve+6onaa-#}P|jnFFY=-V zx?=zrmM>i)gUXS)3--*>QD-YTyzV}WI4{a_W@<$(A5(#!JC80$9eX>=~&;;%>;aob29p+CRmd6 zHe=J{ad+tmnZ%Rr5ZiM56QH1++2B~Fft)liMG#*xz@O`ch*4DG&(5O8 zzOmaP#Sn3)66~8ypi)~<4Hyr$3#RZ`=-=wixXJ)X`mkr<$N)_+cYb5@E@XzGdc9q# z7y`oB??at>9rW*-r}}&$-sV2AG{v7%Jo`9U%a|W*cWE`Qh{TAy6py4zr3D`?WTLjf z;UstWSNi-g{g{1wFj57r!tgA!K-eUq0V1mvpujK^uDYefmoGXZSq6cm=T_ZB)k}Gi z*Beh+gu&q<*ip zcl89{<9|EIFO~XkVYyUofEp=o;I`ZeAy$0_WaakF=H-FAYFJoC`I;PS;Bce0b5H^( zqAY)Wls#jZW7^&%X`RJ^oB;+(yWkwdQA-RJ8N`EwHB`Zo0orq?pmbOhC~OA>oRA($ z`g9)WGyI$TnlRMpK$9@DY?r08)hF*!L+zYsCkA zI5l&w8j~?TNq;ntk?{U5Xz~mxO0sbz;vxI;Y}clx{DhDLiDfH`p5JGKN6NlBy~OA~1g z@C+#+iKQ2M)ymg%SIwvJKG1@33ggesAO4<@2jU&=cOQ^4O-helB?GUo(E}!@sNn-u z;U*v(c-?TA=*+<3IFCi%Y-&Op0V=YVZ2*c69TSv4@&YV^Jg~tkUo9%`yI>i!F@FIl z5e1HEyGJYRoSRcj?+mYNlXV*#mPQf4#J(|n+BC#&uXhSz#^cix_?5yUr+@ z!05ihvi}OxHK0f%_K9#v2fJ1}=oI+us{y#S(uB*T=_`?noT+*l-%PZ93Q(yUoi*6J zX^_qipi;GkVn|;LQkqPwx ze^?EiS{^6fH+$cS^iyu=co;Tx^z?~@j08BaPXghM!nrE6ty2OoQn!@qkX|q&a(Ime zrzUxiU5|d{M|ESN{{n2AxnDU96zW}*SP1kGTjkfW@CL!k+1s$NZ~;o#6xq>EdC>=? zK#LN0s_PlfFLvBheb8ejZoc!RQKbR1+Ch@c^pqTwB4JBXDx8bUMQ?4aKPVr>7KfXn zZ(+lXyg22)&MhUFf!)s2*~Q-jwz@H2i&A*Q=Is)}y<3p97VbboA9>_v;T5}S!?1&Z zV9+Uxc&a_f0MQ;CT!Rb!t<7|L*#Z)Nwe8~%TlbFWl8Zt zqrkxfgYs+`gs;H(@gcL&QO^eeVp!OGpMBC{2pEP>nhcL*WF2Zrl{J|cUVxBbz2!U7 z7HUsdHU#RcvOAi09>Vddy3=#tOu0g4xM9DS!lQeXz*)jB>wsu&gs{z%VNc2+e-CWW*=+XMvTtg~cJ+8U zGXC*`Sd*#IJyHkovQLDamagxEN+o+Zlrkiu@?+p(^1WwL_wK-$Gv(@3eoFizdDN85 zrkeg_@Lw&!BB#0=I5es+o;_hPIEh(iJ{w_EFLZT!pW?T<$aCe-h=q;yX_nu1J*?=d zHUoE16S0~A?S_DTF&Gk17aHomv7iAjHSrN1<1c^oLNiZ2VA|`%)_StG?2|=y@ogwD z3W*nXUU>u*)Q>551-~8usax1v72hQ3!B6o)s2e|&+{RR}WzUL(JjRHaBtIisX&3_i%NYgL~BM3gca^#d;20zPbN6ykF=%Acoje{NGd`)l509^YUldmo_ zbS1)jN=k}Krc~H}KywQiv~WFn-|;=KM0ZG5P!)(CpU)TZkkdW8W9os?MM^oK7cCY# zqJGSsgCpOv0M?K{OPt?#2#9|Zg|K=1JK)zo6L@=rN$7r|$ti0+>;?k`B?;e0*Rg#Q zDXxr($c0i%E_TL-aPE}(20mpF4g7}>B$4PR!Fq0%2ZXlAw0^zGPea9IS^;M$2dqr! zLkfkWdhuZwOW>EhRH)N@624!~z_S?;f86sGU}qsN@ACmRHWF+y=|^|qQVS6bBHG#A zp|MO7_irw67DG|H68&3M_A{y3wQnE-DQ+CF2^ClW6WcG~eU#q}L^!{roLub#;bra4 zlQ&=F-AOSQl83Y2eh1)+qiUV)^n6jTYPfON;#x)2jsHM9f&74 z9&C<)V!NTe2zeMcBVenwXX$~O{K@{k!bgGuJ=cPUjzOZ#e_8;lp7PGP zXxhQ%uEr=*_;G0Hvlj3_Z|uN+XDD|$dUO2zxfe*5lEn{Tr1!rcF{>X>eJchODOL$zqm;n=@fygDSOvnKYDW?3c45=FV6hCb7V80+BUWH}-Nxj|f z4hH_jyn3$CA~q4kir{;n|F;D$iB!XO?iu#+Si(?!05%Evl4Ap0NT(Hg4luzgz%@zrO$Sq*6*eMI%+U?d+W1w?h3wSc}Sp=5SZsfYk*E~G&CMjOc(Fz2ftUu6f2h5ixJ7A|}b z+e)1ToS6v_MBi|cftCUslnU4!5u|_%Ths{p{(yao0HQ{M_Kok*0!sRKoJMtg%E^LJ zyq3SXo!FpI$TO&GlhUUvl1M9TlEe|<*M{j$e**%_SI4yj-gGu#$U`zcE0Gy>6XNw_ z*bz)pR0WYe;{%*jHaPt;7<50jZaVi@Goz>>UbER|S01!as}E&>T}%X(2Iq0@~!ib!5W~7 zzxj+Q&Y*2`IosfM{Zs^~cZM4w(4qD=D z3y=1!TW#EBMU>)4Y`E}Ls^4a#ctKg8;$2B^4v1GLJ~O@f+H=!5E9}$Ou8t-GRpoj= zLnJnq#BRSzAXxVTUZyc!-K$CP4rMAfAVflce9fT2A&M>mxuHo_Gf0< zQ$huQC_QL~3wAa$v_fw~Yul76n&GaUt&*+RC6RL2y7(UcGbU~4(P^qOMr9EK=aIdQ z7sh&;)Dbr%y+t6P`|Y5w0G0pxHfGck7gD#)`|4-2h6v)(IO@Ik6k>s2yndsM8~yv| zuuoE7!XNqA(wYKh=bU+4A(@p4oZQNJClvX7|Xa^wd)+~#YV zrg7%jfXp!WOW+63KXISuj*N7c5$oRX1P-ef4(!^QQvXbqZbW4l4b~>YS$edqBBtWQ zwE_Mq*VQ3zH%_#UjgYh%B|XdOOc5j_qEO_aqNQ_z(yBa3V91q`!Yj>vwHkLQJVLMN zz~!WPMEA?cJ9~>Cf&06Qxy0dkvFc$VYH4I4gg!Id!^y0B-wqzF&VU#=YhKIJtS0|O zmX=TuqV4uW65N*$rNRw*@K_+SY~*}|KOi2bHM6I%DqJCUzgIJ4vEVN0O#!iRqp?Fc z#0+T$-xrrc;=|aUk)$4X+<(sQ>wOhk%ZV;?qZG(~D)~OC`D_0d>fy2hDyx=Fr~B)= zZO;xf2-eaIpRSGx2I&v=*j4F$g_9^yV@k4{oHzY5@l-PmSER9P(Y$2(vw6fR%I}|p z|I^K?)~uToke3$aE|s)35nki&LemWcwnAF(lhD31YB0^uLuJAD%LzhL-`)KBiB8cx z>&NdM16%E4Dr_=2W;MxF{SOwdG}G@L_+~j`2s{I?fKMaECAyzc@gwp&n~4C7@J*qf zEcneoUAYjY4`f|s}X|=Lw&7939m{20+U^FkJ(qxKb=(Vfiw=u7qL^Ir`>S+%ka$q%H zn`K|7p_n-Je#D9hrLRE`5~nho0Y7_rWI0mcI4^4;>#WjYD$ieTuvFcG_SD4Aw66HAyM#;kD`XEo-I-EGD;$8unozN9>S+( zb!`O5)`saK`4TWl=tZz2FrX^N_%yklO#gASo5czxd z(BKv;WRC4l-)zPg#rJ;~P5SQkjpB-Q%ZwYd5?=`82(W#yNfl%wzpr&dVJ#(8;kN}H zUwtYeMJL8bq3~5bf7|g2`hnXh`Au2tbc5HxJJ*r?O|Don1udP+ya@!vJ6u;Mi)4k?hIx7|@?bw#aL`JB z8&tP8&9p_>h_!P^gH-$m+B$DuYu>E3we;AXh!O%eT~H{RPq%?JlLEVt6;+Frw|-Js zoW$`ul{2KJ%HdvMA1f}LMpkic45T?dte;@+J~!n7hwUFcESMWjP$qg%HT9NRnJ);z zm8JyCFYkh98xy|v?zOJ1`$G-HjUxv8F3A+TE~|FKJT9oN95R$ypkGfE{=SBV{-JN~ zARa3OzQapqKWjxI_OzWIa6fLVIx(25Aenc&RSm*Q(lfnk~olK!R&RL z)dVc{kNf{VLNr?QP7kR;ii|foEQLO2sG5Q5hyp%pHQo+%U1^*^fS=^B)hPa2U#Wsc z_;sU60wcnZL;)M|6J$fetgavFgv_X0Ft%>;RN91ld1O#*!R?O^P#`K77G4ST{D9;) zFvas&XtC!#-O14NGgt`c+F?6Uy&IyE%c$~8R*r$g*Si=>-95}gKjiYh&5nWeUT0TF z8J?<3vTd0j37Qf_YKh?-vaEdnguj;>kmo@?l|+cnEp^AZg3+M)o7P$q^`CBM7c(RA zu`<){1!NX}Q4ZY}!o&wWKlI>oI7ayNZnu*vq1d!^!M4ihi&wDx^TP=z zS0TpmFhjg_j>+x#uFBQccX<@_#R1aQSx|@_3#Mesl#&u$8@{e2APrjtssy;*)ZZ_=xocCJO5G+gO#A!w#G}d}!mqP9xb`f+bzOg?au9mlN#R6TrT& zf>S>;fsC($;txP=3Niq-J#?~R`qUMhN$*M{)ZMJXB-CIbJ=?5N@0K^lN6kM1VtiuJ zRZ8Xab85oWw}M|>52a8acBai1&w*3+`=;pNV`guPd3 zxs|*S|DMPUvZC-Yaklj>VOJI*8+raAeOZmP_3a=>aE`_v@W`x~aP3-se~W(Td(N2! zhd}tNIr9ZW0+3eEjzhn{OntR=WYnOsNN$Rz5+QMraw^W68KMvf~15N4YILI#}w_7d{ zrUOMuNE40U&8Y$GFXcWeSJ^>&(n~QQZwgUHV`eYyDs{W}YGK#<5Y%y8aI+iJ+^X*dcVJ}-|8 zr~itOKmrSTV<+gs&rdS3B}Grdwi^7bhijX9+rSh-CXgw|PH{G{%oA!W#(9(Cm{+-42sJ3Q~lQt>=SZ+unjxa*25mXMTq z%Ed)UzbAZVzproITZ^Mr2oHI6d(p8Ad-eNm>~MPVN9#H~nQcy%Plq@u^1uv`qk$a$ zAUKp<15Ev~b8v9jc?xiXE9&Mt+B?Zlku9Ir=COJ28n)Wh-$ z$yGtmtbKrTz!?OXJ{ty>f@T5yuUM~MX#7Stl?+Tv-^uHCI-6Y4K6b$Rcj)MP&?%^c z)dKJ-MxPIsDMfxTsqXATuabq=p%7TBQsBccnP+vmwhfdl!?M2aHpW}#MF{>($pJk8 zd5w5ecF?iZGMSuYzcH?Pa`rKs&vejr?V(4x*6yOa#lNFetai0T&CdU*`UFEr9U1^! z)vFD(5A6-gOm?GLC4r-zx27fX<}0Qa_15TK!x^ELscWrDFryYHpNOXVvJ9AqhHVX_ zXYSu|Fgds>%u~c084=QuUi31qAL-?3S{=@3{J`>8R)gn>ut~t5+h2iTI6N^;<14E$ zMA$J#1ja^6nTo%@A?5u*6-XkCoJ^}30EjCtywc%0^Sl*aaX?qhG_DyEZN2q!+!qY- zG2cIY-#3uMItTsWY%87iE0J;F1^w_g;EwYVM_$~ow|VL5W^JYkIUYz(g0YJz+DZ32 z-_Ew*RPVD8ZcC+0@6uUq`<9r}282~(dQqUEx6M0!`{l-hdF|t7wWQnZsBY}oOqFfk z{uKWltUC$HEySFvw!S^P7P@#IclwZ$+Je3KvcPs%;q6RyR1#U?Mt?Q|W{~-%XSHtgv$ z2h8;r#0Y{6EcH-#rmN0$Oy>il60Av&4$OM;WQzrE2>^ZaA83uNV5f`4WR((C_%$sB zsOr?>)sgAi9ll%y-#Qp==w`jWKgNc{+^;tMIS$DzGF7~oI`{9r9)o*HT|G!~ShRZT zW;Of_2&zC4Pt6cB857F7xaDqWuW0xAS5Q3AKrj7V@6&lI)ENu(5BsRF3GHjKId=0K zn(r`jR#lLBQ(lpO8;?yQBfSF$2QGw2*y++~hQC?_46^_N-YD+dsc~k(S=R=1&b-mQ z;m@(15gcPF6y61qk?{^JoFu&gnZOSS>M?81hh4ifXa_%bIRKmja5<3?JsMK(4hvaV z<``z*pj=4C2%?ux1)x#QQq)ze zlNkR@4hYxDReV?J^tq5f z5K4al1oew$R2QAjtEKXb@$D_hUFML*E#u`B+0ZXTW)o!!Odn068J$@#OP`LS(B{aW4L z2coC3-?whoIR~l46e$1`rq_;x!}Y*V1@{)0w{d`|3Wonx=zXmFs2Um4`n3;DI>diTsHBQT@rnk4LI% zqliZA9vQe3g{|W1rt}~@9~6B$eVEdd8jb(%UMjEsgu+Dq^{2)Vj9ZstIQ^qLo!XxZ zlST&C9vct*$(pqA?5a4%EG0;DQ2gvXWjMaPP|0=SZna{>^i z$+=zcs_^cJSFr#rkok;5&nvx2V!HrKvsdipX5X(=inH8-2Ty9}K>UDqc^Ax+Ux zk=>J|xgT}SIaft`eAtu3EK5D{OQtIj;9SvRnSC?MO#kq8(&hR$@6|>Jo_ox@qQmF2f@uj5O{_5WwV^ax{VG-q~At)aw{t3*`q3*_v0>Bj!@ zdg4&V16jr1<0z){-(^%Vvh2s1=LxpI6*))Nt&Vrk<0`irk4Xr zR^&o8jx(qAB-?q_>i4Lgv$$pP4L#@5AJx5uLC!;;1}ZK3N1X3#ENR~D0&YjK^yg$L zoO*HL+9k>z`A%O<)bu4xX|+5KNIM-&m~7Ykjg8VA4^=o1rjsfUo2n}d%yDt$Um6bx zqxZT+?S5#W#>9-)|jVX1Xc;1N=_B>&p}hS6Cb6s?fqD`gHT<2 z(<7fQh1)=_+sT#D<1}3BZQ&-<7$dtfbE>n(!reLR@px}?kmEE!NlBA$U zdbm6b{`o*vGyXRwdyx}QCd*R&UHH!1L+Rr*e^>mDm|NVn*iRha88lOm%@1TzC+4@! z>3=|1Ts6us7bQ!xWMKXPzSmP%O!RnWWD0fgj2rLnYVDnU7YiCTV8}8M_nc_?eE`_j zW$kvD!5Afp4=c5^Jzg8qAQ1(CEOFMsyKXku2~fHf14rszAa_cB5BjzPHm;TgBS{~V zfMRsd?2wy&QTV~;cs58#fFR7--0^v7Z|Fv=2~XSy(`pu^(Z?I4D6kB044;-m0v$|L znRKoXUgioQmYO>lcWI8zkzCBy!!5@c_~DRjo4h_g(@R@d{-Incw5=}XJelk!c8t#{ z{}rR6XGAzobf)hN+9s#0VP>B9@??M$zTIb?`d1nsxrN=`Y|w>-obp0quz624dT|d| z!|yf@>nVE&-)Gv2?ojN>Fz1R6IaL=|>SH$;Td223g?fhz+yd;T0|nU*8IKGOq|Skl z9T7wEf0oBsg#WjB?9%0D%LeR(R~ldp(`zF&Ak)yY-KgFfBtM|%A2bqo_#MWj3f5kQ z7nm}}P=jI+%xCc?Py}=8L>sH~!9M>!mChu*UgaNf@CdU?zzly|%UL($XU3M>Pk&{J zT5Nwj{`uc57aocgXB*WyX8`Tz_o5s#9I*7hs|x043XHvUio~FSAbgNr#=8hrif}&V zBri1u-~GGzk8TMj5F`KW{IIVX057;Mj=ByB7LD~_|sv?r- z04rAT6hYb>@Y|dqn2z_~E1peuvj)ydz8O|HnDMDG?y_1`nxyfSxLMrz$pYd+H4}eixoVd^D06eCLX0Nx`-IO*SiVOU&-m)<#Q@ zbXE*rD|a3BR}m%Tgmu?_z-(NuN33&A+$QQXe`f+S0sAm(2Jo!dz`0$IJ*bM5M6eqg z#nj}e2(TKr2VoT$iOR7i_#>Fz>My}l4~@%-qRpcUrnlR4e>WA=*!*?TMkZ*$)p8P$UO;;hrW(slECzJ4~V%Xgr*I2IG1t(u*C4VM<{=M3%S{ z_B9tP*_3e!$%ii|Nk-Jc*kf_oZ7zp7^V-#BwP^CymNm&G@HN1m2L-US@A-o)=3TIS ztuH@^six@Osz9WoR-RIoz12dzTMU@f`R~Q=oax-9Ye*0%KwcOg4pmG28{B5|s`)}+ zx<eAnCsSM|gMUIl2*v;Jy3nWRzv@DuUM{Zv_WIwd(C6b@xYiaCTV4;crqps*wtb; zz)u}p#?xQLaDnYyKf8atPpX-^CZN%Gm(ph|{!@(OO}6y;|HKReH6QPPo}2)d$%2f< z5GExR_TYPhM)X`$iJi5bAtueqJ-b97dQAe$jgq-Vn8FtP07;XSKlfi@N@%`_P)40g z@WJta8+@SsT-hI>ggXE1lGVlBCQJdf0nwy8?6+I`UA08M8^?sv9>79ZSg)M>_sU?B z3fxoqr(5H3cSFp{1gs?2sZIe;U*I(*Md}Ve#qo;g_&sMx3i}(|V0Wtsrfk!#hQ~q| z^)bc5Z{^M!GN(eEzc@HnG%V?Rg8Fmh3NtGP6xKNK)los^Gv}LEE3HJ1x?Az9z7P_0M#paPW8?@D+Y{9^Ka`R4n5S?N^f*Oql6rA^UaRV?i_gk zEZX##q}A;Ug8|1tYD&yu!I<;Hah-EvXCs(^NM6HXjYBrNA1TGG*W16)DF_Gn)t!^q zjc>LMH9|Uo*qcc3>{5PQz9P7q%mT_BW%@<%{Reh=r>QV!<$X^v`E_RuUv*x^coJW7 z(aO+-FOM0LHYO8br^qFt7w+PoKUmC>E#aP^?O7VA*f_3Sw0U7aOzeE$^IAUo)GWqv zi)TM6>Z92{KIcS@pSvDXNjkTc>?n3zAd_$TS3hjaG zotAh9({y5;Nr`8VFe2Y=UXSGFzR^OG@Gq_3*GW=9tlokcr)CF}vtZs4oFmVR<&7Af_g|$A19_Mfyyvq~~5m!Awi})^5P@4Gk zv5aVzm}6*ggM=^e^u1-|vRN2>Hnaai z1b2FOL1-jasZ$8y!6oGvAuI6RYAPJ0OH7?K??n=b(FFXYwAcOkU`Flj)S{^~}UV#xP#h3MY z(-gl=N0+W1l!zp?rM#`+I>J=Q&A`VPupu%h_Vub&Xiw!-hzaN019s^{(s4#7&pV5nYiPUz6#TZNnsEBCS(V_CIi>EUY zF}70|nu$NBB3u-rWO2EU+#3GGl-ud6HWGS$2-m6ODOPPGCqCJ<%20RuoZJ&7gGZb0 z5#Y!GCL*bm%fCDmjtAfDj){z@Ln7h-&60(c@uy0o#5!0*UxWDXz!d|-lOB}F8%r); z!m$xQ0XqX4&D6TE3XL-i;t`JxM84@%-m})RQ~W3^4~rm7tpfjT&aycI7F=i9hsRj3 z!Q}YV&06B)bR`&Bq?^pv!6oh8{_-4jK8vNK;LQxq9$|^Y|5TkXWf-ZH7cq71Dr;8p zYcS`AIVxbTNo(fW1kFg2Iydq!Ny#E?lQ;M2+2#HM$1u=P=)U=h$zmV(Z|dFy9_=-~ zHNWG7ku?nJcj*=`%R8qO!CCf^PmUuz;KN`@M3;DC!?*LD1L-^cg=v9Efslkj%ZMwO zd!T$jQatj%`u4ciH2pG7n&Zq+)Ql997ynPdb*tx+eEZxT6i@bf{(r;cf}y)i&?FdLXFHR3&8N#u zE)xJq;F|KG!G0`q3lQkQBsd9};E<93I7t$d5qqzk4iM+0yrH}E4Ij1-#J&F@D6<16 zVB&~z<@uq&DJV~Pp5yqpv<4k}M9>>1VGLvp-J_o6uqXQZjKMN>yjw{Cz4w{;vWN)R zfx87T0{bpb?Lyrr+$Vc<3fcLC+2b{QI}oSU0ZDMk$sUfaQANzPgLU$X_!N;h{lFJB z51b`Qq;H0>`b)-V0y1us+^Z;$97Dz{7?Ye`V&lm=bxC6E+o3D+f1E5n+ZhRef9N`1 zX{GtswE;d^XQv4T1>tNi-E?aW@f2o<%ejZ5V_+X9OG8C2fw~hwQNUX)5o5SiZ7jdW zB<_)LyuY4fS}<>08GuFdr(mQo=USM-)3&v$o4048^c<_zTJehiV}!?dbF-|8^6f(u z7nlS4MZ2Q59wBx>t^A3%Epu99OIxthPLW!tolm%i|;g;Ivw3UoZIkq<1% zv>K{;A2ytX*FFUP{P0QDM?X#&Qhe3kDZ|fT1=5Ou5 z&EYJf`{57Zd)%V?5Esi6@cEcrQCP7-S-_?Gm%erBKA4>B`_A<42^d=va1GETpoK_( zAj$>~!x~_uO85F1khEf~c?}M)R=()-y+%kY1Ga(1{dNa7|ICdq{wb35g6=3J7U;|u z7aAinnpj3siMEa05`}F)e4=OrR5u=z*cgj2Ak(+-d#KF_re_}m^9d8P zXzJ#xKEM-g;NbU!UePp6zfxgwUS6B2z7vQ)^Aw_3W4Ls9>Juxg0OwfbaGZaihdzJ- zwFqn_Y+9%g+=jLH3P2>&!q71MXARaJT&Z(zLSCn?25zebL3nhbpf(^)5PN&@^y2&s z@2kgr_RobEjiU?KAqrtZCCb2UzTBN-Q61=KWoz}q8O0Jq9*0kWJ}JHFT@h4k9}tuJ zFZHCU0x5TOobLFuO7tMnjkA&W@uC!P+QU1>_#W4p^sWHM=v-~iT#UZ`9Dr1oQIEHz0^z3F4nUy2*GjH>MCrtK6-a0# zIRsr0OB}92C1~4;YC@tFyzs$tBxdMUOY}xOxnzr@w-IJ;S>OffA3XO!VxAVlf8_!L zo`+M*exz+o&AvDU&WFCa# zgFYijTIb+%=gCJfTAOiw_oW*E+j=GA1LH>rhH&zQQstwVKpS2#*+>vti<`ARQj!Bq z-+DL3Dj4KO-FKVS)~VMfr6L7`#a_a`?UAu?tJbiJ4%xPER0$}ER#6FU>4S3Ue=zkQ z@Kpc(|2WR+;23dG_TEZXGP3t7q(VqSM%iR!9(!afLPS*3P*ln~qL58PGLD^n>?7j; zcy+x$-{1ds>$ZauMVZh6`cAeC|` zPy)F_g(305m6kV-Z|L^hr!^`J*yKHj4aUN@?+cDAv28=|ohfjQZ(3ZTk14ir*yR<9X4doEdE2!?!0;yh@)kLJIwwocW(QFOM}A z0Z`c#m;mMc*;r;}5&j_n*|WuC5!zJeEH+ zf7R~;@rY=(;P3|^o8T!aGyU`!gq%L)q%yU9c{i<J&mfU zNSSXtM_dEa+Y&e|N=Zj8J}Q+hoCQ|50iBFVzx(tm6mXc$uhHP{2nxg|bQknw5%4XZxFnuWCe79yY(C0LZK;jd={T))fGk?Tn&MklkBg zi&T}WxKKV36w#0SN1d>0?sB{2r)NH1e-f12&3$^9-u+fk$XFdr(E$5@As}C$C%9$# z(Mwl`ikP6e&-_=+(d3$ZH_>=wr|28d!#*k9zTWSXPvc)vG@%?!IvI?Hs; z7Pytv_WoqP2MVa;7SH&cde{{_xsHmNX{1b}QBG~?n%u7ib+K4<5MSg)?9^5`5V8lc30WYI)^82nK(qyJEx(^d zA4VQTyKCbaaR-eDe@(fu{S}r{aK7()d^XaKitNTL{(9FlV0s%s^a5Oj=%CM>UUlbO z#6AUR;^oo%e-@;GyspS&>7ECs$C$`;;G=qYbTPNIfzSRZd4X5-dSa2hB0R$l1^^pr4NlW4ebb7h=HLQZ|i_Lki4 zFB`A=GWtu|vt@5~vMc`L$X?H=thO+Jt#UDaYVkZzopQQKjEkAXb6fYr5%i!yWKf9v zaPzVIN8r^6II^I`ln~N{#Q!D(?kcLyi_aqt_EqTmztUKu_hEAgUC=*I#aQ)RFfuFC zr8Bo!%Te(TTw&$_gj2dF^cI79!C&dvN}s3XC!%Z@4*0bTjp1QGywg*GTns}9-T}Ae zJdmU+-YY9R@dC4Fa{qMs*>})_^5`yKVqa7wFC%{b8nno{4pv?b=PJByi0x$}SCIlf zces{sD1|sw^Wmw;EzxRm$#mqzbsnvr_a%d&TlRJ zFugXIA%G#VlM|VC&1zhNnC<@jq`l^zf#I9s30?*oPD#RGLgWn;ul!_W5yz?PgRTv7 z!Wg2qJ#`u9!zFjypg8Re7wQ@hafJOUjlO5k~s}ZB>{Hg8-gbUEqJOGk4 zCeZs_3N7B&7Mn{&mTrC5(9juDV!`Jx33s3%mwDh!C%YEQp`D>e>+VTQ8e~0rz*yjV z_%k%ue4`O9%sJSOIM9b{;KwZ$hdMs8q6TQeMq_}>ONS8uIVRr%*lx}mAip<* zH~GqM(dM_(iYkt=9cL0CwEo6ze>d;H+t(AZD`9Kvpa=p z)TOa%WV!7H)NV0{BNDP&ERz7pxpT8dR`M{QW58EH#Y1bdP{hqOWho$AecWJXKS@N8 zg(X3OFl7F@De!DA`RLFl>pi!_7meDh7^QR-YEX7SzZ&asPM)d7BIdVfFhT|F2J zbC`>7jYylVM{cG7W-yUVXAzvWId~z<C>{;E1or1horGPT;z;=J zl0P*0&YhCx1mSHvgfM2lg2JHyaa+F~n%I7|dL@a^UKB+T-txrdE7tP2H1@7D)x|j1 zg1K5+v>1|2AT!4CdMd9y1ZA7c)M3L9uuQnu;eT)4F{c(~{1}YTzizG=yx({k@z?n8 zrTxp^(hCg6cO{C=-X5-Vn#dBM*~1>+hC7!4N-;?}Kz?hkumvn2Pb0=EuSS2QQwSO_ zBKAlem{;&l2(tuK_72nrdW~Pp&SwnT>C@-!1ODOS?tE%hDY!2zfzM6~*s_cPo`ILY zkG2!I!S!+c*o7zQdv!6q6QM`NM+NY9gb>=a4N2(5Lg49qXMoQ3v0Z4o02_lw!X=d> zWyrq=k__m;5d`iaPn|N=e^x!g6?g-TzD~N6w>)%sN!frd=KD&Gg2_QX3QrKwC_p7? zTpT)rM&s~;k)~e<=@Jh<((ZH5M#jh&Qt*h8BbNi*c=!Oc4&fJlxttjCBz|lchp4B( zqMv6^0FU^oy{2#E-`(x=%fs&}ME?CY@(?3Dq>QSNJNECPYiOWMsfbbQR~Q9HbOSoi zF)IHJJ1Wu<0~Z~gwVM_EUR0{a_VS1u&?6uO8c%^n*1En38eM&@`yNaRP9mNF{De0p zIaJaJ$DY%`J&$Mn4Ma~Kqji_AbvIp=0;r++)zfVc{@sB8`!o}L+N9(-bm&KE@G<@# zet&Xl`2G3BU9B6ZfXsxgo(8(1Bhok!nLJZ; zIz6+fvb<|AzUWZ)IqLtdfon7nA9*}7qB{!DzQDjKi-C4KEogN+Q4gv$wp~YRVL*K3 zXi7RZ4ZwMexaZ{vV17rzQIlFfSInmxnd_f~96ZnZQBkKyrck>N*PQ#LN0{eEf$SB1HX*d*inS1Z!lQ__XLD3t$ERtPYfc_``Y%vv1tye z2l0zfvw?Z?%yP{n{Z9}qLS%1zfA{jg2bjHK3i^z3Bx%zesZQt zF&ro)eS!UQad-@9qgc3w<8@ImyF&KDutpYnXH#QgP6_CACMkLOyW%p3OztuWk;6MM zVOYp%_UtpJ{R|_~FdFzHp=%Ef(fv;^9v270RQCB(lT{iSMZ@Mywx9qx6ERBz0%yc) zjT#J*N4`NCBX|9)0g>tz$z90a8Gg>Wkb6iEC4}`wDV|AQWCvTCar{M%z|`L=ZTS2; zQ?8;<$s(|NLDR6aouC^#bpnP$?aCqKyS({(fsBh?EwrYkr6rw~8m7Tk%O3dD*MJGb z7(4^RIP{sbFnM1Gv?u>J-2eWY&jqH#?{l(c78$iwI@y|y;Mh&b-knf9O`rJh#qgf+ zfhGA#E-BbFS$J?y-vph7o(m`7m=MxNChI2?@YpM$bt|EXK4a-kT_pOWF08ig)K7Rs zV~~0g_~05#Aui?)uyG}Wf8~V5&wd7W*kd4Zl|&cqR(68gjPT2MNXZp>mw$h*P!GeG zz6RbO@pC0HFVhHNLf8v1N1C5H;$ft@DP(nChp6bYA+*Zvk{g0?VENJMdLT`8JyYsR z1Ljq(^H2)R{)ltIq%b)r^WcY@6`zNJR>bnw#_y|+OkAgtIZBVb1HWt&9H0a%f-}b8 z{jy*0*!~|D;0n-XK`Zj1cI1=Dhp!9GSzlld`g{*j#Bx%F~H~`(^bt(P7|u!CEsd< zk{6VQ6w%!9`znno^*v&`2W*uQ`_U}ncU#E*JfacE z*wX$Q4)4EryN`h-aC@s{!Bk0(V-(agfZggLZBCy7Hi{0hQR3k2%SVe>M2=9Ut)KATZWM_Ge z%T2=~nKeC_buO%8_+HJU_C46Ce%`E7!Z-r+13ZJ8;}#r4tN;o;taPAIihZIG@Nc&X zT>Fxp0TECKLTbY$WX}P+;?X@C4lR@rftI}VV<@;#Vejv3BX5N)nlQ3xzSc@&OqEnP z@Pds)L+k1wbGXGOt#A16tZSg`U}CmWOj2MHq*-xOQ%=EqtF2|;;~!-`hUEO%^>4s* zy6Y2w4-jlwHN}i7m z6LF2yj6@Z!pkU{>U=;%vbB{oLTk`Os;AdwJBQP~#sIWg98ITaLwhJ1kSYWs-c>DkF zh9KZ^hx#9A9&-Es>>i?_g@zZ^Kt1BG1opy94WaLy2d)Z|pM)1sXqM=ugq@j&284g{ zL<5i?tu0pdWU~C52V^R!V5&k@k*yI$Mn3>+swCiEmVuZ%WU5WzZ^uh}8ere5FhLX+ zec`3u{cPlg@Ed4s$vUvAw!>J?dL2Jptp@yN7I7l$?pgK%VdisBPMSsJ0Y@I)rroy^ttOHwB2E zy(2V=;al}k<3buZUXwqI`7oizD4>c>=M5)mf*Y-;^>4T`=s#Oa$3c9qQU&NQ7#&X@ zej@^-y3u`HUl&G20AYtrpzqWvU6RZD&q8- zl<}tjra=SM0NZc-KKBY7Rty#gS|vB?RQHF$nI^w=jQ8IbEEZ1B%R2^_?j49 zfB&=*LFQH%X6vJgJw#0Xb-{k(W)Cz5J&MVZ;Sv~d5V5#wHt6$)eJyGw+A$H6;RIhH z1)coAk6-+y*MBNzEeYL!}H@>)jw)R5XP#a3g~masHXL}@mu31RI8ao(`m z7^rS;8G^y`p8qQDHdt9aQ==~)_`iY4;($`XXy6)>1F+x`X?1phsSzpU%F{zx@DT@0 z)eRx{R2u&Gk1S9&AE!%nL<~O8oq7u7C&EeNRF4k2?jDiCP$KuM+4M=+jsHl8N9@ikBZqS9{tLQPkHDU5{5N_G`>{^S*J)-Xjf7a0D{`pIlECE)IMUq;(1nt$(eeMQkX@~ zq=Pv&x+ieyy$`g$tQzppc)+RT!#%)~zywp= z`;;CUnS$0ruJ8+`KN+a4vhk3R6o7Ey_3z(`63E?0Oe0JB8wQFUvNyg<9PqT%8uC%W z_xq70=VXHCA%vtfEFMjF{96kJfIJiQ?pi6c6y@f^sY(=$f0PmV69eqweK)+`8WyYmw=2@&)>OMf{m)49&70mo zEsPL~i2v_8=f&{thr@#RzxA-;U7k51l=S5ICu{9 z*F+(2wmvCdcL!Z3sDWGI7PZqwy%my%*dN-!QckpXYl54TuE9tBK$`H%v2_YK#u%m? zcO;nFGLd(ng9S6jxQ{hl1~*Lm*#F!0j7BiCTB51WSsK!q2SW~U)F98v-w$rCy?>Jn ztj_=LMf5$`y6NQWt3p_D&_yIU9^D3m`2nHCHSc_MANfA43|iz&E+0EP`x?l$>fG*< zlaqx@Pk*Ma?>ZBP2FIXA(%_JHXUle3FGfCZABX}5vm9V_ag*l;lMmK-&%S@U%8(C* z*&|J+nV0A!Vk)ymkTGaDF)R8^#AHicGEa}nO+iN)!mxAwcifqzm2eR7eSwSP8n~ed zHE-^)b)G(QNHRwE`s2D!4YA(q?Us|V`9HH)xHKDVe=NNAQ^Yf_l6tK z|C!aucd#P$!k-ol@D6AwFo^-nb8Ysqi(ecBC%utVid=A`@R=pmga5aU(ls#D1!`f| zz(+0tg(A$=my2Ki&kuh2occkO0|fEc@R8(iXYft`7D|zN7y@=#sA<1H*xzk-quhNf z4fcoDR5D%<^#ToECqKC31G&3!Ct>OT4GwIr%J#6I@3)JV?a5LNY$CLS&%YtwN?4qd z8WSmA1Cs4%pKbd1=rjKt=SKr1JNYk8vZ}Oadv5>3LIf<|^v~(R|6b(Z>$oFUKHz*e z`L9?FYwJ8Y*Io@o8Hwff>yA>t0+>ifT>?a3zM$c%m)#{NFAK*?|FcS;5mOr@4pPtM zHTwK_46{V0f7{ue`DakEp(sdP~gMVL}f>Yx3FowYQ<+Gmo$+3e+_CucOFj|3#X}BJw*7`y}_mw1wV&9|Gj~$yB_alR!S7T~xBd+R=>Y*4lBFDTKr49G< zgEVy{nKN`46Soh1LQ%*~NYF@h#YOC|G%6vx@c=tm8~CygrdHOe4QqVzy+oiVBTcAr z&#NUW)}z9E;Z?0R-ar`ieo?k96gD3nZqU*$V{p)lW)Dx8I&Z*NhepCd$RROLS-{%) z1*K5?u0T9N+1VOj1ZROX8_ANc5d%SP8h|Z-3Z);#L2j-CXtcy2pna{vM-$p%Dnn&} z^Qs6$TrzXe?XpR^7k!uqwmIR|bQauGQ3kM!6TEj^{SbGoRV4;bQ+g5+QJjYD{7jk$ zH5%|c3N=Pw)hT10hX))4h`D-Z!2jCZD!50b4_`g^cGg~;=y7TV%E@;;M?M(B8Spse zNiVMm8#KE4j;Sy%9gDv@#)NIB84$aya2e|}Ar9t^GBNlnpEKlAJE!!rE4>1w}TAuQtA z;~uoRq-#V{3yFvrRXKM}LA59w)bP-xx$i+-6b+|lG9D5&U~=!iYz5j>Cs34EV`E(!|Qc)b&(S?4K+fR_-1~f zSnQk7laZ0(Xy~i)adct7vFASiG4@YYObl8aM4~sAM{Hhvm_JtZxjFx%zCDyW`d?hG zC9-3$s-7q4561Kj7r-0!D4iVDLI8`% zCQi=dQgy^Ah>|X+i!ciC$V1q5?)W{2@hB}eP~dyA?g9#D>5T{bHgShJMd0mDX{&_h zb@h&Iz;KP=@K~Nb!fqq(KIy%uGj?w1?)6Z z^||rly&j;X9;!Eg`2!Ob{)tee8e-u%&w9eTv=(*G#u=A(@(>VPFa<>d?&9(r24xD; z_Ajr-%YmG0GhGY6IGbvce!RhxsB*4FilqU0TL z>%Ck2>T3@%S7-z6zbm@34&OMDbo@ekGi_>+yi=Z!htg4!l^8yH4BX?e&Z+jSP49}A zmxAbQ%)X*_dKSKxKRoaUjYT9S;lDJrsqV8y+D5WML9!v+K3=fMrzj7A_;Pdg4Z)*{ zFS7DEC~_KB3BTGg1xptZ7aG+LM<72m%9V#YHz#tMq9H?s`R6t8>ja^CfBOn5Vsri* zM>NQsUpTteW|K-0Q-L3=3$9vbBb@en_pP!w(p8^@TfzuyoEDvvJa`w~#R)fPcW@;ah3{F}oB>{U*T-u>0 z3HcFoWjUo#CY{IWFRIYj2((PU?vOk`PLZbKFfW;rLl!M4F^voidL)tP$5As+ZPh8^G#y_0Gs-) zOZe0z9O>_35SG+DabPN40;tHuT+H-J~Ss}DVO4I{}L&42Nv$|`tt1NSGS8c14k^xRL2??Dc( z_l##-1FDrl_7&UgXTHS@3DoEYxKS0F5N`I;d3sCo$6yV~2>x0QG>a^phf_uNnk4<; zBshV~Wx}9V;P9JS`PZOo+`t0(JbgB;X}m8D`OVT9U7Cr0rML7tUYOb*3VG#7gu8GG z`{0yhd=-O0BWMA9XT$0IN0;byC*WgXm@-mj0`grpMvM#eFc@99rwZm^s(TU%meJm*)T@zfTL?tNW8p`U#@_ z+KvkBwcpy*;@y1#sh0_mCAT&)*>~eyLx{4o@`zpneZZ&H^?06J1_w(JI1iXzd{`Xu z^=A8G)ArZLJ}UrZ+Stp?_$fI&z%;)Pc$hDjcg(ckX+#_$vE=H|H-(}R4)4N~PI?Zs z^oXzMJ$?SH4n7(mPsNMkDGV;4-UY=?C%9z$p#a+TI?nGZ@lT0n~^zpWN5Drf%GL z)0NtQ=ORfOuc-<=n&~4K$ZZ2rW9N%T`Cbp4B6&7RVmPUTAmSh6G^&e(V9Ou2+*t)- zIk8@J=n)c;@K9P%>&ENr%jkW`fj%t^5_%YkUjeT5N0;O9`ip8Rq=X;(&6=0F3Stw~ zA!vLQtT*rM^rCqg_4<9VAW?Ob-L)WNPer2I`w{uYg`Wm^PI&E6vx!qe>4$Nk<~g5J z^6IFGeG}rYg??V-??iqTtbKrj)AGlw)}fN)_-Fj2b3%+0zY^ZW|KO&&;-f12#vVt; zpc-%p=t4_<)?v5)IO8u?lE8*Q9NHl#xT&N__UpNgf~rCCW7-dNonbr zV^7~X#*;@Nn?p(~JzbKir}BpO{EgC4y{FgYnTT=Vg>o8oGB0*5G~du5yP^W>Mc=Fj z5B{!1#4rgU8y(f<`B5LQk4sa$6*L4`g%{(pm>wLN{%sYGem%iDs*hY+oOV~aQa^2C z`v9z5CP@kQpfrjns@|zRQ#Nqf_GqpZ9pA-uNd9fCBs|prO<$M;TvBs`uO2?i`h1c= z($Lo9jhW*UqMD29W9WBw{vo>jCWM7sI+D2@L~CwU-UAx}=42&~X&oK1{)fnc#ajg7 zP!#^k`6rOY({9aEjZM@@Un9i3$t~W61Z@mj;yCf1sA7y4W32#A8gZHr`3JCD!|N1J zca0k0wf^^kK`pOx>iB=r;%Ox=HAh;K*az_3vfDiYUzr-?k{0 zJ#C(B0Zjk5Eh-lb`0Qn8sX6 zhcX8<`7fH#8R{7+JrfMymFjads-oMgLj)v{;;~WtDFm}7buR&`Rj$?!2{wDI!G8e9 zv*`00lmetc2{`%@^)*3qWyZ?qc$Y$Z<@%ZZl$!t!x5iQ`6`e!Jb#cM-T`~;S0txUB zVJxIJPIXY-`dRIkWieTXXP5~2qMx4`d5xk$+-=}Vf?%Y>3B=J#+RgWBP?}hL_qqwZ zjKa&QXQ@eI#B=$RUjT&lCrCfc|NUI=T0>Pk28>9)BR7AzT~)vE<`f4#4zfs-vVfd=55<-G zJe>uBhYAV|J(x0d{#wC?u0aY z4?OdaU?VOn0~mr2fs{e+M!$XwbFO~zb;pJa)Q-+c1%dB-fgq^OrZPi(+N=!-8|`3l zVJ~*Rw0&<5MX$%(Ox+l@PhT&&d72QmQOJ8N?y ze1v$segd6CPak#-$!h&oM$4*95c>tWC=bGB?zvpx(k3(l@?;GOdSIVyV2|$~cHk^w z(&A;ey-oR2E#Q!Cv%jy=VN?WAoFV0e`lGbTKvFh8E2Z)c>V!VuIgtqBo84skiN&~+ zpIsSLH_{0lG3#tySkhzD?Yo-`59?*6tUjukX|W}PW%>MjLV}-Plb2IW)8p-I(5b6+ zV#Z)5LS+o_u^Iqjl3PZ~V8X0f5J@QPp!-w~sL`6>l2eXu=h)pcq)^kk%C}d=Od`%V zJKiS!(U8Z4UT}d1gJ)_i#hTb@;`fJA5QXJ zIq19N z)4zK$+hd#3&!MG_kH|Qd7X1cP2)@Qp>alQhq4Y#fZiIk0-2n^p{9Zu%uYJhQ9a}!v zw+T5_GpU9QRlqU%-3nqmTzk~D?RB&tC1wFCrvGZ;gA%*um=;bn@ef}Z$6Fu4`lgO( zcRJU$50O%dwaY=zyo~-23-H0XkuWF_4?B0N^XQ4qj7Vr%YaRXQ%Zc6xOq;(CQpUj| z{ua*BElKno!A1RW8x8IZt-;B=qNY3}h4O1EE)fbZrD?%hRdu45FdXFhm{6`~(=Oh< zqa_KqDi`>#cy(*(d?jePKl7jn7dnb+QkIF>554r({T`2y7CCdNR4pX3!X(j|BO!4f zX6{)NjR}VOY_xUTaa6qVW+_K5w>`gY+mg_yLno>t(>53xmwRTY&%Ed{)dhm=^RFB)S7lYLi^q+8j`x3cjgty0b-H#Im z%du^cyeRWYGw`64N$NylpI~=zV(83o3o&eG6UIQ#rX4DgrJszwE2&Wk?#D6&TR|O? z?#Ql`lYi>QBl!twfCB&Bcy-T@i)#7X=8~aeWGn^7-QU9?K9S-JbcD?OiVn7HgMAEm zOJ%xKV5|J2esy3ZHVG-E!3r>y*wd{GM7M|GjpMhG4XWR>0b&j zE)6{DW*W!tBV}U{y12gYkPs%JcOm`SK)1`(& zu_}mb=Q_1>1-aSxyNa~KC+&+nHQemxer|tl{~UOQ{j;jz)KO%f*Ny=)iC?XAhSX`v~tquf}jpn;a`m- zsP_3XZa^z#D+Zjg?kSl2rd)5oAL?>x0|~!iAhrH;{h#iM{&d$Ge7Xf@n58BGHHOM+ z;|I@cjvEteR*GlXn~HT9MR~nZDl)(o#o`s<_3KWzX;nFVqUkD$t_ixptK%bbg%;mYWB+kR}e$ZP|9-yxq zOcT}K&9q;Dehu5ON1=HQ9lAk?7c}O|l(FC5g<@HMPiV~blFB0-i<(=qM_y|=zkvXB zA&@QU{CMp(aNbb*EC3J?q2Vvo^UKksS8r8?;$l{I_K_3mu_F$XA#E4MQhDmnVKW!b za>|L>sU4_T7lz4SdUPi7+n0CRaYDv@C39=DMdxI_?F$xvGM`{j%|5ofr37KC*AGeE z@6Qq$zy6pXI;DI4``yOOL) zz*5p(fOM8#>p0>zD1C9a!bTxSWy|X6GaFXH%O8X~>VjYS?o6}GdRUi?CPx&VI&NW7 zcsr9xxF1p!LgKb0hr2bgw2n?qQC5KQH=@C}h{wetruCPAYX! zVaGoMtJ8J*Wc6`-Btjc+V~SU+xLEP1WFN$l&H7SrU&HQ_*tygUTCm*Q&y{NpIL?7r zi1@&r;MzShEqXzJ{K5m>KFtEE^=wW0@iI{UN{2F5c^364du&vqE@fVYS%KnevhlIy zTX{v6c8OVOYDpRTJcf55-2o}xAn{8~$>}=xr1?!)q*5~qRkMV}#f=`Brri26T0+4PEXz|^=>S!l)(#Ift&MFna*A2QwRG!{J-8Q!zZ2Q21M;f!!T{VMt6*IinS*pr6I?zC~e59vb?yu6$sTQ8}px6^l?S1f_9MP zw$}oiCQ@)D?}!yD9IUQOVsz!_R8R19Humn^fE>k-(YCrc|4+mZfeYWy)^90V#&Nq< zL*tqOxa>EJ3F6Vi(D8WXcn7N^ZB|$H# zQIT5v>%f=VrvpvWmq)V746`omEW8vuIz#MyX~ih`Q14TFOQInZ<37;r6>DdqFW~}K z0)7iT2>)v<8y=b&x>Ke-)01{b$&cxqHbch#TF2_zXh>pap=AGun^BVHy%+A{h4qv=vK;_9u zZic`ud;S)d{}<#!NyfH^DJW}Glt39X2aHo!Q9Yj8`gjH$@4SQ$cJ8Eer&*ecHMpwx zV;T908N+1dcl**4#t-&a4&+KH3xk5T(zujL_?1ag<|hP&s~*$&5(fcR_$zOD(DjK| z)CgJ39O*bw!BgcWzx2y`mdE2_*i9_yhh0AZz+=(tIhQ^Ww1c3}NMzxMen&e0>ahpo z-{SErgEG2=bLydk{2oI>2D9DbYg1X`w=`I<$~XRHDf`G(7|it@K-bAAki->&Y_aQR zIs{2oX{>u;_5H*DXtc>PCQFPl5`iuBaXEL*Sn$nYozs9u%7-iH4|LOW#Eb z8F{B=+pm4wJ4(W2h@0_g|JiP2%kl~$!j)W76Pp_+`uAytX`KQ}74N%JFr(p3iRWuFoF)(1u@+v)4w; zmn@B~r2njPykD$N3JfA1)=Vo3{O#QroputvyokfEj40a$EZ@F+v7+19{aG996`h5T zXqi3Cr8Osph(2w=E2~%6{p6%Z7l7-C&sf6=+0&zdpQ7mt6P1?=ed{QoTseHB*+RG? zk;)BCsF0^doeD?YZbGwu7^~@4&=a>)Qa^Vxuh|+yr?P0Y~zYJes|xMha~SWo_rkU zcJXQMkOjxK6jF;gEM$M~zvcc4{#T#5pH;KM@owj&D6eV~*Fy4jTd^cl_4@)RT|PhA z4kO^50AHSb$)A$0CRdkg)XS+=GpJ+uJ?m66gCo4G;#{BLrISDK#-kOMuP;2$6VXqt zVI^Vun#c!(*BykesAaI)8DxTh`qZFI{F%anCs9A|3HoEsv5BoV%|a;@n+V zQ-K>H<6@#@;39kyXi8WKbV<{V&_1gwK@)o3)YLVFHqPvf>zG}-zQpSNKq2SN>xrB) zTA80yvZ`s0YAGwyZnNADPDb0_Zc2-}RFV?TNIXZ_{pN!8jKenqq<9MYYz}`MIJ9e^ zO@Pj$U>`-)u6vbAm!ebhIvIx3p$Z^80H~}meMl1r9@n`CKX z-W3bl^W#=-{Z{ZN^Bc|LC#Gn>Rf>q(S&>FJ__TfBV;Ru8f*G_RRAJjb{rGf~cD-r- z+x?aFR(A<@b|z?bZ;`MOdE%cD-lLj1{O+Zbtxv4Z-<`)Thq=W1LbeR%wHXo}#~KcT zi06miWIcIx9#e&c?g&4^Z^v@1@Z&Ii5W%b&mf#tbd;{2oP;ZZtB+$OA zDOwks0BG==|B2C^;~~OCN3}l&6L!J6=6MP^8|Gy~;lIr^JHz|&I3|UwQ7sa7678xX zMiwfhHl@_a6w<-#9+ha zOe%)EB(*T1)unf*#;mk*BtHxqc#PUkd``VcUZ{9;z7d>}8%u%#GiTn2`h2FlOgD9X z=z{FpRL0b7QBLmWg7B>Y5yL!b&3=QmiZ| z`deqO51Q$|zCjnh7D=9I{V=Wt5HLPjqQ-zb&OBH7fM)OOpH1QSro29jBgd)lD;xW6 zFJ}&wChG8C)6|Q*HREXUEWPI3Lm{^b9YZOLGZf+dy~RxR9{Gv^E8lfBUcj?%dqFLPZuhK|rHSmmA%XMeUJd-9K2OF}jvZ4g z?ogDsV9>^^4nlWf_3LY&n2QF*&kqkp$EHhljnowrX_FeDEzb%%^YkE9!;PFDw z>)TDd>URBKa&msdbceTLkM0NSr z*g*eArRSgL@9d6{9_kx!D2%xpnDrCKGb`*={|0a9^IoQ-`f?%oP;yfB?sC?klAndht zqZm}9=jAe`Afo;AYRh+?m(yYu05C1&Rks=!3fY~!E87r3Fz@{W`Q2$iHf&o3HQyZS zG{dx5;&{hu-|4Iji_cm0-7&*6hQ6wwYxRc zWOI!8QvdD>2W%}yXh+y9tB?*bxAs&Zw*@Xmz|!#0A>2wLgVtS@p~==2`8AW5Wcf9U zn!5TA$5)`(n{|xEI1CUstsNh21f$_~ojRGH(;=9w$l_`*VVjnM+I7OOGAzbUu!Zo> zZ9orTG(vY)8pn2=gHJngR)xNAN#S`uy1da}c8W*6UW;P0K=1_*mF4*lH<>S>nywCS zXvIpVG)bM)s-Jk3&~&hAnbH)vx0P=LN-Ym?Q%^73)bAzqq~*)%Hb-HPMaM?PP1dXu z9DQ(?W2)!B2~V%Kep_Ew^f$=-e4DZL?g=J!_uM#i*_eM;%QNdTs`#r6$8fH6;b2II zUlz5v4|>Z`^43xc8-I8sTIS=p*;_jq-7xYq+Ds&Z!`F;~T~-Hac-IV~=TO-qov*Ms z@g7{*H|V;H93WBfZ_7h%>j1%lgB0$DA6dD0$Oa%mrVu=7;2+}O+<`c0jZn(*b*eGZ zaepx`RgSWqk%K?SjSe1IyW85D=(j9rpSADdIYKh(^#qVd`n3+pm?5Mm@QrbXhpL%wjQmdmo(p*rf>$EcZ{}lvAevXd}tD&<#oV z;ar)_;fL|gFL3ed?=7VaE37Ya>8A$KA$NRk@H+aAMy6em=EUa|=GIq^nTo-$X`BO5 zygZw~Q#geKKGXRCfr89=lgB3iUGXLA`&NP6{V!`x%!l6IspdFUz<;{@=0ehBE3fc; z&K2}Q(b~p9lo8+Z;(KYoM@~n?2zMFMVLHipPqZb)9LvD;G_juG2q+^8r(G7z4%gRa z>$&l=UPMeNTK$OzyITrMz2Euv)=Vyvh5S8xKX5by^o0e5;Quh+aRR(5n+1-+=Fe72 z&dbJBiz*HUlW(XC?lU6L9$Fw_$MsO}?umR6W>Gu;3$J$c@DPA2*f!N3&!oYj)1rb4 z7jHLtjac_k=ve1VUyY)JN(I1+sxR#Is2_y3V*%LqJc@UEgz>sO%g0t!CJJwAYMPVCEcyj1OyUTVm=MiyniXokUccU+t>~Mm zDh6Vj`IN@_rxpG>H7)?H`STtrDzzW%Z>ztD)^tgB4-3=*)C6TiN#07Kq)NLGNa!20 z3LV$@`89%riykz2da;8Vau{8V-osPZvyU;T7aS`yDx;cM4b>w9oL};pd=^n!R=XqH zu7^2Fzf|@3o||ECMsS1_#L-TEZSqh#-swZQ&Up8*Ae}+4pMWWG?$UH40WQ(j{Nu=d zA0XpM4u7S+=SW31e%*DlHH}q;p@g?s4S)@_yHHPSR-=&P8$^=t3J_tz!~K4a&<%ie zfUwIr#7NHb6Zr|?q27EC>31h+DzGzat z=;ne0e0>C~;}6>`enEW^!_^7f+dJ#EklkgdCmV%sZjD^>pzNb)HEBQ|ei2MlDY-I) zoo$d0`p&nvN_B+b4oLdsqC^bAI=4Z|Q5@K9azOw5OO4OuhD*XqYZzubIVI($ zz!xLJ%gePEGu`)(li2&u>+4^kyH6eP>$R}8-tB}en4`eQ_}^I{ha;}?JfY}K<))t- z{p+x(IyCso(D%n>x2AAaiG$GpfVik8K{8Wy`e{wlWHO-Fpq%;LBRpuAf1e+X8i5Ya zv)Mf-x&IA(0;+NXKz+;@-&}m~X*@&`#YaT2qgRz&k>CM(^V|k`RJIUmB3o#jH96Dv zJ$8ibhctJf42d`aM#?&sSN+9o6pRK}O&oxQh_rQKavFSaF4>=GoV zaVNufO_W{#CRc8&be4wo(X1M_k@Y{gC(@8fX0wg`i#>o&e;=M2ZKF%>9oid$LS(Cr z4e~UhSK3z_68N!bTv|Fu(AX6%a`kxaTU(3eQSfq#57eDI*>tDon-COVz0sljBm6co zbPlsGp;=(9SDQh%^FnVgfl-4_t%{4HPzo;8duxe8SaZ~<1s{F+InUG4LDCL;MYra~ zT?Vl#dvWi0CmMn5;AfsOuIPk-9d-`czN%$7c?aV7omQ>AJC=5Rhu$I?^*so?AY)*9 zeNSa#eKn-cxr^g)t&hhg>QEJic;Ac{tTGTFY7E!X>_IX4dci&*4+X-$yQ)*?MnPF! z{1HwsJ#qytU`VLt6s9NVELLlqPU}>6#q7w?Z$UEOqJCfRA%6Ss{qg&ZT5vCnl^QZ{ z_D?dPPhG!?)QHbd-ZDCrlFo$*bgIVtHox2jF~0 zV+~mkE@*3a=*|htCy|3ZYmzd7}=$;T(4qOmT z?GCCe4OfhnCc#B_9x6p^fX+Ovy05tCBImSTv);@wy%oGZg`?t0M^VtkP|An!K zj*9Qs*@9kSdlS@M7LvhxikIXnG;QyGeYkm}>qulA8y;Ms4giXLs~}sUG!J6uiG@yX z%Xhqezy$Ns?c>BgHB-KNL<7?7d@_4D;Fm`pEeYhF0=uOhjzV?o*(IvmA)QKp53G;g zBtB4l>yh=O%0+k{K#MH?Mx^ZvwLh2MRfMHN{WgY*r+3_Csz-`dcDnYRKn~E3wF9Gs z6qWV6yCW$akY`Kw&m|rc$R;aRBSqItm~l;12{@*+#ja_vBiY6uc^r{W{1F_bvV7B;V><#g=sR7?IUkI^-lvn0i;EtMXt|) zNm>xF!l6McjK&Gq3v`oN>SNRa!?Z=5f5_h>^>iI@^JjIH-|nRp@h^J=!1vg^4;jWM z(TeUtu^GmJi&=uO6lWG=FJ}$*D-*EJrDI|5*vBr1{L0Q1^EpIm(3vfM&~5qHBy-O+ z^|~^CRi2WU*3)u4ja$ANQvTMJ=UsyKUe&k=lAi}?hr&1iUN)u#A!G-L&ERS-Us%>& z0F=!@2o>%M)U%N;(oGk?na;jw4PuL62bnK9XRHPv!RG;{=N2Gb-*g6maI%0~Vg2i) zS5LoRt{Yf-Ga8ieP<{8z#EoxjgS?1U&)r!Uei0TTI(s{cLKrN+q15@$74P4E)E%V> z4Rnm{jepEL!MnOY=%a1(${{w~8K#U11yd#g>)RZ4|9)oVQH+%}q1FUTvqs2PR|x<# z!~pg_ldzEiAUZZ=V@4kE$FV-%882(qZ0Pg=WV87bGyd5yD;MP!8)W2`PiwiR@$a2`^W({U zB8A$=u-j%u(Blx|aa>A>ya<7!$D}Ag8(RX5s+n`_%?(S0UE+BHOp*l0)r#-Ajzl8LkePF}$wh$ijVN~{ri<}K_h8GQj@}Bp!=AlOqH`CXn0Kw)>@W?@1Bj&!FloEcyE#_TmFA!y=7FCUD*B& z!vI6i0Ma!yf=V|-4W*=rf`pWS0#Z^k(lK-hC>^4hNT`5>G=m^z08&E;NDe)O^51hm z&$HIM*89PGEkAf)bH(2KjN|y7!7BoH_Sdf5EK<#KXl~d*4MmF(rlY!?9E8483-Y@M z48v(Xu5HAP@s~w-3B%zKQ8pqe*4VOeS*Dv!SFR5N)+DIwipoR7>UC|(TNU(yf76Cu zdN&sX(?r`SgukZVr)57k{79!%NJ$zMJ4|SgRSpYTHMe;OnoT6BpZiaS>?H>vm+c|2 z1FZ?Dqc2PKrZUN6!(B%tET5;i&B}Cy64ng|%30DLr$0Q_7LYI!o(XvAr&-L0sE-G) zGzQeNw^TI#k<{j@w8Pad*m;!x3#Qtjn*Z_2g{3A#W<9Sp4~vqtP#Ry6(0?GS5hLmV zoC7mf9pZ){ZsOV-9bNaLR(wDq%HeLsj=avplyZkmC5rdTE6|Sr4uyCn?g(E|2F}%t z%fEiaq-Wfy$mmH61yBi+iSx05j>r@+MJuzn&k*RIYuM~FB?x5Q5jeL6^wO`@{KYxk zLI~5(hWV6zOP|l^%SVowFVeY$kBbzJ7Rc;!fk~I-wsfaJN;CFngb8YliUDe}`_O@l zgy~Ez>r%vO%wCcH0pGd+*GxDzB9^*#qEHJ<*;*+-43r6@b?)$nF}r}&=f&M&>*ooQ zR<9}&&u3Iabl8)lURQnkqLTQ+Aq-58SFRhrt<=>1QS9}ItNf-lhhY_%L>@SnTF4iBgi2V<7oqWO*Z4fFoxQAicLqKvmM=ZYT(YZSd{NE;{ zers>*S05U-&D|90y}{U=4#gmWL~V;sY6iHH9r+;FS-|TAb*1l|3#6dTBa2M_R1K=x zr70U-W0Uc}kBvl?0Hr_ES+)0o8}=mwse5r7-x-ry!RA47p#x?9v6c13CkN^$>vgpe zXW<{tsj-k0if0Bd7zTeTc#WH%rST>K9ng!m!)zVyG3V`=)qTUwsE#KMd_&IvUvsc9 zUWvR3cKSvO0R67FB=z}*j&--I|6OJ?T;5=>qFca?J!d>9uhWb(GzddQSx?a_wLh1+>lxN`z`eLFf7&@@4^E< zp#?ID+Fy7wB8!g9Hi z$lM*p#Zv{kos+^BE|`!|xS*41*0Lh9_t>%j++;dQ{1akSK_TZKqPhRwa;K9?(-1%v z_&9X*p$i1od8xU%HmA@3>$&*}_x+k18CIj0%oH}pMeZ1T;^*;)Bmw7w=yODgh3S|j zXyg}xQ1MJImY$y&GbE1jV6`Tyn6e^t+!u;KaAK^|@+W&6XrL#koj}XVrkxdVbiVr5->? zKE)+kcamTW-Kx#}xT?qv6CPuoVpm;!X8<HeL4@eV3=rmv4Q9E@2m)o_wj z+a}8X#DzmXnSZb~nX6~WUX}1(x=z@wUwX$r$nusgLZkz(Q0! z{RDa}%jiM~H7%*08G5B4+vQ4zZ7DQ=!^!&su1#uiT=3+N7^X`T01RWr>XOW28%}T* za+m%x4P=T|ajOEfl0s@qgA|Ukl05i7piK+nYrp_0;Rjje*fSHQX;!Yz^m<=J(KAt!Dgu1&RUrKl{N=gOst3l=hxmHss6&Deu0UE?#$C zaFg-ehI#jp6@Hv5%V@66jHg>4cfb|>Gdx2fd?d)#7rY9;$L$>Fk@4&K4V1eQp?`xmAgD_r#N7aL+r-c*okqvg3zD`diQ24N4X;3tSM6CUMD+>yw z582psJ%EJ5y|8&KADmBLRo?j)3b-;ca2nFjrJJL$5%S8yDLnu3G?fNaUP;^NMe&Og zfJ|GkB+{D_rM{1qQ|NlOcJ0CW`{Yhd0NKdUuN`CF%g$~(zzu3zcQ7?`j^{52a3^>H zDAQ|)2eD8QQPEc~;@P<&pTUNFd^V2IYV_gp(&N!&Bbzuk2J7*BVuKe-p`vy}`~)eg zfrJlFS{$L5KX#krYf;vHLJE~M`j!T##%mQ&~w zZC$=QqyhcJ6>~6f)=)r>B>3(cCOHDRBx!wAV4)M-Qc*3DIA!^PkL|uk{l$| zUcl(|=ZZR!Ft`x6kKJ=!Za;k9K;^uE=uFeo705>t zwg5ur>!Vcbn=ZYbVBIP~k~#L8JVpdH)YpLt%z@vX0iN$tF-0Hy789=l4inq3nas;$ zhAqjl6odm8naDG)G-zRHG(m|CmMf$a<L>8?Mb+}l zVaiy|+GTt=?5>xJkliwH)Hb8>E)Zm_1r&-@#i64zSYZ#l3Z0X=^6tPr(>v>nb?y`1 ztS5OAv-EnuU7?CffAs|04`#MYMUt}H`i73mh+3at_knpRX&yk(?Pv0Z+am+Bg@uKE>!F`?2x9hCG`J zI??{W((hD(!s6F!!*FJ-JgJkGsc7;3G1L%$6`WxhA$o}WP;nsNPQo%jUj!^d*(alT z%%kMtLEeMY)VRn$G!NiJ>tTWK2rHM&8P32fcfY?V)72qL3_f=1rSQstaOS1qtF?Et zgE&nk)>9m1;!^I3&tO>x^>uL)K~6PtUm+gPBVs&CQ^Av-4B1h+&x~~i*UeN9E34sk zFG<-ed{H&s`Y?=PyFE>kcDu|;JF>_M7BTi}vaCkBz(F18fGytv3~SN@Rzz2typAdI zfAZ_qdy+vm7Cz9qOhXJ_VncQ?8;Lo0_46R(S z;E$XE!~rb~jwNiNq9sQv;qp^~F~FWynB-Ld2S0jY799`Q3#r-miCd>JXm+~+!IK_8=$8k zcF4iGHb;tE7*~I8vMj#_PR^GxXa&F6Pi?zQmyQ6=rs6E^K8_TH)2@^%F+uo0*!q5? zF=8C{t)ep$92Ow>iGFzzn|8nd1rMWHbLIPf!>=g=dio!>OViVN!F@mhrYn4Sc&8m) z2>glSH~xhAs7>bJ6>`|0BUyO{^-*RPE>sP?Ou0uA+@GaUk~RMYmI+-vJ~R{;G*qmS z|AM?`U-=eURQi`i{dpKEp`9!M7vi{4rW03q=!A%(7$7PEHbE2)fMWP7j2&F=3Du#0{$WysWhE~@MVBKPDDpxeRd-oOJ8*kv5X5&G}v|TbU zjrcnk6#p-$^wYo$NU0*Wxke3{;k1}bw;-;}Z3!Mr zxhzpWQ^_HwU8ZajSG)RWLIK!Hi$tZvhzTZ=)gEa1j)8#ltNss%uA>EWNgnhIjp&aL zPlC?<1Ac2khxirOC9Aa5eQ%utJ;!YeO!DPuijRY=;IM!Sic8xW>*7!s{HQZoG#dBIK69zz7o?1L!GafnP?)h*1D=0eujgS@|k~&^Dpj zT$|1B$vC9vFEP>}d>OECJa|OeILE!)bjXrvo|eH#o|V00KyUuFr4%WYW)0paLM>2U zog3miCD3|STYLN2uqK4xNfGpqo`!FB_PKWbb8#2gy(bMAGcw>NCuvVfu-S%70EgU4 ziAy2do@xmT21X4{lvbw8_=k2|My3!@3Mf3(HU!bbJhyE%F#wQ)-H+mLix6ruteZn> zXfYuU1Z0}EiK|}Z7~lLfHf9~z+&881D)24r1{$Nu6^nAdxJ{mZIY#nE)qTE$4pWLv z-H!=?-qHG37(N1aJfR229?eQn*|?R`yBDs8&HyA3$y~6)g8z~5h<26}$c6(qITpg& z0_%SkY=$rZMp7V2cQW?I#!}bUe2|;rcjFvyl(fEX4|$j2^_N$FXHDp@&@uA;m(cQ3 zXW_MFpr{USe0wkX`3klej^2ct|JpFw)I(gV_g~!mG0&&Ya+F*X87ap9*O8t4th@b! z*KbB{MdNqgQw=piQPXk2J^4o&Ut`b!f2GK^0Scf$r^&(zw#$#ltAv07eQjXUW6Gfi zFVsan_q=NMhT#||KmR+m;c-CCS9%w%2IANF7!r=px&W-4a5pAYhg0>1q{|WUZ3%~{ z06SIZAILP2BfcIpDOR@;$d_{;(KZ|cJT&M8ZHjtd&+BZ<|7jgq#PfiVi5+1}6Qv2} zH#CCYTA>=Pp-(&$2vJ?tNNARxvAWF2A0Xhj_10G@K7$jk;+upL2xcLtfU)M(0(Gy# z!#j>-U}qrkRq^R&RQ%?!!z)a__FuFwZE&iBmQ&}#IY#QfCkz6UDgXdpzoha}2f2QujYh6;68BdXM^yE z=KC?Rk=IZ8C(k6W`TcD3aE$bQ3XGO2bdA=_`%8vz06cW|OI?4B1UXO*t0ftZrEF0Y za(0?-0hZ6}xgx{wWLv1gyJb#?9E`&9*9hIg6^NNcJxHrsHTR}jXE1x~V|aNICDpP1 zO$*^%X_wC6yFiNl8`y4t%K5#ngH-2?3-C@V!tmAHP2#N$3%M?N1H{q^eCQVeWTi z$L8Rn`Z&A2{4i#l3liF({^iDevp zK4*ST#~bdRILwbBn=ic(tJZP9`iWKpB0L;~%k~YW?WQ67$OA+jO_&|iH`!!U;nVz4 z{rX!#OckpO=&^9&s4dz7!~K*kfx)SsBe4sbp*s~lE8vUDp&5?(k4BUY8q@rYQtMYc<&xdt!2jMw zbsM+sl*YvBx)8cv(|bCj__a}lbi)6y;M0}^20FtC9a^Vkf@$GwG>1XFQtY>o%lq_E zOi?NJw?bYZoRc~m0U%@;&MW*p68ONBLaM0gIZB<`)jG$^^gYhY)=vX%=*@4pfnJ!a zHM+Ja4%5_t=GkYV|ojzrv`tL0EAjB~5E@%?Z$`af= zxYNw7V0N8X&VYUK&NwYpELL6iSW8N$YhH~;hX|@s-*@jC+C@RKJQ5QV^ZziEXMintw3#I93egG?-CVX)9GgR^_!jBV^MCW?r|q_ zvGu)Df*ADkNJ@?Ewjyl+}f{yW*LoDhD8U}1Z$NnqnI-qul6 zzQKgLs?D!@-%wATi*ZAzYXSduL?E{2=_%;-{?XKa-w}{XNv8fC#uYEOns_uZIU5#8F|WHX61oT z)$|xolv(8n$?snub#-AUr=yK^DdU$CJXMk66;GRckhV5K<-@3!&_7tX2io8)H=BWz zy-gn6or&Bzd0}hPEy2BE1=@-VA)|8|@p!#lF0+@#sT`QDG~IP%Hq{+j#QFKv_FiA) zCj?dWlZV&`b`&E?a*l*%At5Fmck?LTArK6%?JdJwtLdWoBH%IUaNh9*FJEk>K?AKi z0oEvStF^i31^3aZpEvj!|sM_FdCu?D5g|k-*G#q!GhWhlsfah+|Rs z-i8`$?FNRP1P;u2JS5jzDyj@gsHr1&^MHxym=pux_JjBR-|QYd66Kya@(m)RlPct` z4tUz8w6K0cB=_hk8x{p&-4?0(Jd|`pU!0u(^DG{zkL?FN<9B?1u6Z)(H(=ep*oGie zPcpZkJGMu2RmF6^$)iLSQ)ts6)8Z&b0-6gAuwf}Q_VMManVDie*%uzbOhvH`+IdFClHo$x^B{6mU7Z_>Tw;q%EG zRn%r3i2~LK?|o)x{!DM*#^4I*fjpY-5mENJ?{ti=B{56KOGI4*n>t`;qGIy&*(0^r^tdW&srPi2xy$L#dOB^^exzuQyQC`KkFBib-!bkC|T-Oh!R| z8C47IP7+>W9akQGi*LoZYY#eHo4mc79%+H(w4EGZjc-$ubs`?Ru1$kU$}K9tv9cuk z!CFqF5R)1+{Gaa*C&PpbNzYU4&l*L4t*+dho|Ripvo~qi?GMK(kjbnb@L!e^`biJr zZ*;J?y$<1@d^GOR^;49wuHgZpCN7kK*9PO*M#{Oy@5R76gY1GB^mOoLdg0DI+3oFY zooZ_@r4z-%9ECt`SIo4XrTjfcy-EMJy_q6XgR_>a%_YT)Y8)7hkI}!RFw)L7kto?q zX}%YNi~z1|#W*aNm*U@)9Qk5$55}>Q=H({g-nk1mv&U;eiD780V+X9h7};*@es3^$ zmu83Mu&*0==?l!;NF)_J!JU>>yp)bQQa!8NUVsrB z7n!R)Bd1!d(IMxJ=ll3h>HT6usC3dm!&cXLqI(;;pAC7y@vArH& zKZ{#PAG8pr69io_?SW5~OXpcX6Hfy|3GLJh55;-z;zowtNE~LjLUtzpNcbmgjS_8u z)?WE=0*IpZkfhWvJxZW60TNqX8Nc(5fD=r8Vi8MQd5T>0E$y*%%323jZY9%6ZarCl zRTCSU(}6obGbGvhBXO#Z@+rKioU<`QMh!Vd4HhK$x3}tb z+S6(?_1Y1RnQ+Wl=gWk6IX^>|yRpZPh{#+U@p*z&5OuYuCa6INcGcvO==MiT=&m6q zEi)M+aEbtx@AV-R*Q?z!Y3z1zFcwc4nW30ug;#~SU{5{;uH986i=8ONCp6{eXf&|R z@#rg(-FhxA8C5dDw=Q3bO^`9gnuTonVL8<}L2v3Vcn)Qvga33)+lIF4GbTNsGG=a2 zb?$k4}gg>&3nFxY$L`?ek z+h7*UsFdP3r8r1q)4qO?PL#0*r^`&(t-7}%3+)C~F0qROe>u2xvNLE0emwQ*8H-W0 z0hof;z~x+_lZ42US)UnS^e6q;I>E>4Jmyu5Qe!6oPnBa;QxG=kUH^wuA+`hqruWR; zcPa3+i<_5gdGIIr2=s&ZPM9yCP!2;M+G~^mZHyLe21ox~@$>W5?dyy#I(`zWNA z8p4fKEmkMmOpw~Ml}dazws3x!t;IS-jZS$bF&q(qt)sJkQmRbN#fW!)Jz_bax!IjY zA8OFTAi-3icsNABqTzKK!dnk|C4Yd=?%ACD=z zr#?GAQsvO{Nx+4#^>t>|grVg6AfI|K3;*fVBMo9>+vuec~9TGO2zaOzW8D`;&H>zw<5;{KEfaTT>LPHMTVEZ zi13{*jyx3!8kg<^g!wG@q1-*St?Vabw;ngNJ`}lg;Puz$d*7q1UrqHDdmFE%cng%S zOa%-jx}1d@UAlQKOUyW9?_}%o%CjpXRc8D~pHJ@jVfJ$>=y}9zE;S3>`C4#gXS_!6 z&K+hoS8NwZAkm=nT=Da*WS`}8tX)tZO3CdDYhsbSQF-`ZtI_0hkFsyv%6SiTb5H(! z6WPpj7Y+qQ)y6{2OMS!%%}*Ug8X?ccP!kg_%LDg8HUHyLG+xW7tsHEb7gTF_etsr} zU&Xbxpp|PH{5aEwG2z%xnYDZzBm})(X|i2MU}#6+W435ebh?uRq*)I%B;!{bsvw_% z=#R@k3Gxu9p7838@AtO6Sk``V;#6os=w`=(5(Y7obWqe>|1A< zC`u~l@ytT!k#aqf{K@1zd1eY~+qfdWr%F61Cp9EE_<*cR3{|`?BTDsAsJAuKt8+gm z^{i4*icPK_u($z#Wpp4^{8Jn~#^&~Z^4h4L^U~Kxi!UujR+mdef>zAWO}*q*dj(6+ zQ?(M9e+p(Ai~~zf3p()%H2m)C>si-jLWg)mF6DL0!?d|+#o_&)i{54O=57>o{ zd4mm)b(xF?R!02ix@Z!Mlr5YA!gn!KNTdZ5nh5HMS0_K_WFvLEwmX@!%v+l)91|){ zc@ue2cI9b&IbH8sN6iIOIZhaglI8^XKE9_Fqk(>?~wg#Oxsh|s0eM8E6Sd2{)I2Ogo#v1FW55wjc`icvU?Uy{K-S+!& z8T~uUkKZSq6S>n(o4I#C(V^+OY}IFnZeP%*T9x<=SboPiyY3_-{h8*+uv0o}#+ChT zlYGOyFFF4I#%_rKPs}->UGKcYM>lkI<&x1n`>hp0GBpa$?QP{FNxzNBD+3*IT>a@M zNWvu^jq}?Io~s>5VAn_ldiuc;K&H`*wchA@M$DRttfb2 z!;4kfiv7tsZ#2N|@PQ7eti91^WvAlk%4U1%tcrGYXZh%K?59#oNCw00gS-NFeAHILty~eK&`tf zG>D}Kyt7qB!k_nZD-=DKe)5ekXPWh8@omuTVf03KLHYU4M-NF*QwF3k)Gexh79jr( z`|^c~NkN?mbJEt;q3lWk`w;THOHay3$?f0m32kQ)2ZNM@ul)WlY{~z~zMP;Va_&5B z-?P`E!JiBpuMve3b&|fRbfE7k%$;WT9R({_D#uO<+Fu*=lMwD=J#U=#Gt&eU>VH+pK9}dE2hTKsB6P_(=_Md~l&U`)$jTqWO-fi`r)7b?{64y>5_C3ov z1Hk|=6gjz-zbHZ&TEZKr4^+TM_fpjrdXlbD$I!x1OHA-kLE_3Pa`StBJ52C#7_H>}!|0LmL|MXRVLL&hZGHa6t%wW4cCfQ{>c{DzoaXi= zUr|Bj?@7UWo?KiZT^XY`VFH2s^W#c|ya@1^-7gl$2tz;X*WVLeYuXy{30TgJDNRXQ zo0upcskE~B^!-}XSvcp=!8u9egQl&`L_8qixsBa^ocD@F^vP=dv?u~INTOqKxe<^YydVqoS`+49m!>eo_*Qg zOU3zC!|IDs^)vCOIV)O(#6jIG2}oQXl;)ka_z+M)!IXD+1h>#*oq7Zn#`!ei1lMcF z)rwhC(x8pPD$W?U#`D)EIvqn@sU&y^@Lz1HsLY7^Z9;X!+=3$CNoiwaZTF7!b$;dH zYekACSypx7|7@~u8fi@jE~GL~i28kqozE{&D!9v}>2*0>y|$U^thQfxv{){rdSbe& zXaY!H$r-}+5T|caxlK%GTiwP!jC~3Hk-(^6v-Xqb&d{&-eG1V-^G{X#KDqQf8GU4? zXtlem$Je!~d+_>QexY^@dhFJ(PalNmzod11$q^0Pv_lfh+dnTbHuj$P{E71v9*^f@ zO8k6}cD?Te#-ctnVQ(p;(4kNgT{+YRk7hV=78dJyebflsaRwr>j0zZX+3(n4Qd2CEb_&!=0 zW@j!Wc|QUHTM? z=RtCOYFz2sd7<;_bt;EbEr5{~1_Vv}#z+AD*{uEji=U@lBTj3Oc^|*9-+J<9f9>nR zvnwWpk>sLaQbL`a=Ik|Yjhp2}wUW@D?2jIEmkD6lm4eFIuBIq=)L)G*||jA}+5NleGwEr^}jGcB&M&gy6&Yp?r~#c@Ca8`+j=L zb%b2qqcV(*_U^>9F|^k&*A~GfdcM`y=8F(B4BzyTG$}s0XnQY(XOL2j%fWfl#|ts7fqhRsDAzR+1dY8t8=wBd7$Z7ruM zR5dph8hi21H_jn2n{m_BOR3K1+Yy+LncF^YP3%l=(s<>ydgBqfN#EO!33$LT%+-vO zY(BU8=idVVnK+kK`7Rt&wj$>QPljn*9mc6;~j4t4PRYFhw{wnN^&_V_8X226dTyYu5!&b1~zik0iavAg0P zQ3*akddqN*{AIO4_RooEn(zj|*^;i{VeQk>q{)j*dFoS9yO}LvSz!pf2*GI!)WGk% z@+s5!zLS#0X4k<5j9v*wGbI`*#%TJ#f}wIwheHIvMZtE}IH~}45O!g)FtqyXg=im1 zZdDqTKYI|(aB{6>R17p(*bueI72(e2Do=2pkx0?b(dUvC?xE!Sxauo>c9Vyxz1IR# zn;KZ)?IqzN*1Ji4JuqVhKV682q5h4i7m%>(e$FhtI*jYeXBJ82>%T<$+&w%(b0>=C z@_tX?6@nN%uEn{wZ{vj0aa;F>-aJl@1?zRu=(`IhHIf-~%j6at&C|FzXG9`Z3z(1A zb^dBWJ>tdAcCRzdWy#mKZY4!M)9ts8=J$F;ITE98%M6{XyvaA>Qz)5x87cm1%^<6;FOla7InhQZZifAQV?!=zu$a4xB{Gi~ z#4yoR++q_#k0u`I?mRLfv~%rX*T`wEqrcdsNGsZLct~9rn5r!FzdQgf!6_s<2L;Ek z)mq22?Y02foik0ZhHmrcqa|+S*|uU39nPovz+2(1d$mlhugY93i|b&6L2(^k&r!PF zJ*u~G+pdPp#p$x{ag3Ql<^w|vW89UL%oUAfA-4Q=_}q=Je`WZV6qS_53IhgJ{NGH< zt^Kz=7am^T#|#OTLT7g^S=HML=`LpISiQH`O~@}|Laj7~;og%^-ff>c+?=wja~Ujr zA1UBM6Yg6^hZ_b;;|q%kLBP66(-;d=DX`6ir^Of}+Qn3jjl6NndK!gq1*5l?WVO0- zVe-suFs|9mEUjUI;5{abXPM23#l}~P7NXzw$emn; zEj3OuN3idm7F&0nQ?P_!s2fYFuHdm{bP0_01Ctr)$$a}}If8#>S}6X*-g`3e@BY=U zr1`7A)Z@vb8b+RW$IDCKilI)HG*^9}B0>bagWwoiYkTLJHX(6_+OussDOQ;uR^Ey3 zMfw_g6kfO)on8^YqITerp;TUz;Q&L)GGUUuC;S%0GYny>Blcf{a37e*_b6KU{p@z(AX4 zYQpZ`U?|u%0Y)+w-kY}DvOh`WFL`yT+3}T{@x?D(3bJ4>xJ~o%x3b$(?UmJ3$9E0o zXp+#{i5!zF-r~_)*M*_;hUG#P%rBucEvI~!+UsXp*Tvy4yEmw45Q$j;(K>}DCJwT^ z6p>{l7tZfjk%C~e0xtGG9bz;fv?u6C7={yM<9)?c`+gpo#a`!){-Q8$g8al}6sWq6 zzCtjPbsOnqZ(_sw3DlwwykqbIBvad_P~SYK({@)hwJfiXi(gMi-}TH2&84iaJBAO$ zqlX|&c?gm5JkMgu;!DN1{4ZuWAo)8f^7WvHYxnOugNbXefC1CryTA<4ow&CiiY;J7 zh<#dCCwr_A2?3(1nDUI8?_sUlgsGITwvcpWVa_2>5Gr)`mDKBH-`!Yr&X5d>|43>s zAaf{?tK_gbVn^jGqjHJHfkx)xjUY|n&pO{!*u2GZip+51UfyYC;P(wjMIn~?0o1|m znnk7xe(csEr)bw{k4u__QZYUz8{c%rl{wBVhY2OEaKLp}np(|mK)UJh6-S50t?&>H z7aCM7wGQ-;$a%(^F7K|7=Fr)lS^0aos_VG}6Ux+jlkwj^KqFu}emMdd zMiA_*6^go zH3TTjkXw!x+8ttjpVmW$nf{InETqVN_e5vBHF48B<1y4z<{$0jmb@8w@;rHgUik4P zf{R_VlUxD@d25cMvY|m)zz8CgRF!c*`M0n?8=J3GMxoCaVbd6-#r?2%L719PE7$Hv z>~+g6rMwB?31z;95ndg9atBwtPEETT8G}GPK$(XshW702?*3Lg)VWB(6-L`ut#u3x zw7>JTXk9&d#~bHRMNO=v17fx-8y^-Lbl;0cyUhh!au_}n1Iq|0!n;&!rTqe9A0?~& zF%IK)QZQ7O-T-fgW^IhwKdwe=_K*E> zk>;Tt8x{8{<$yNHcTX&tRB_jATBpo4a;e%sztv6fU$BTiQ-i#m%EJL2`}wU+AlX~r z9MPqXeP#*HpAyqYE6waUDF=eb-Uq#SGMXN^ldI~VsRjdVOTb~<(BW(7>`+~$OFAry zg9b$|RX&zsfmQzad69klY|Cxm7~F?$8j={-#NPB6%8fMzP4w4Gpl5#0^H#LWAx|lt14{NYB<%d>0i)=QT;mT?9%vzeQ;2H z5wyg`kEgRQZTc2UsDXcQx!;H|@yE@=X}3i=&5@p*Cx0?T@B!OiX_rv7OM*;Tss!_F z=v>og#V8TLUc z6Bpc+#*7FwJQ-`TG3-&IL0)}l;LiSn+@^nDU>+y#{g8QQ(aXhn98%}Wd#6*Zdt1!kq znqMsP!KXt6HWUBIhoZ55$6WB3l?i=g(`}!=-9Rx3L6(Y@Z&Y8|L=p{YY^1avOd#$v zW7wGOC_-1+Z()$GJfZX;)1}JSBfEId(9HvP`44wFubHk}=aY;8W}K7Vl;(ve_&OY( zFVoQ9-q(?uoaTI}2@Ywqg1$`1e0lXoe5*D6$W#x@(8JZnBrS1p3u|>{FmXsLT8pOj zQl$wsF9Sq|`P7lyV_9VSO)>K0HKakVZp@3xQ{gLwM8i66+@Oq#JvfS+ZZY4?gl29* zUy4?fwFlQlxgk3gyd7B4PnxS7*YF;WbirQI?FRa~9a3s@;7beCGW;1w+`YY)G92*& zhGLtLx;Oq;EBT!X;)N9W6*t&+9@y(?A+U5HAMk6h6gwdYUvzjigFMYY`78qIrx4L~ zBL&~iC*gGonMZi@ctu{dC4ihxzQX=Nxq7QRiG4bZRO_IWidNCI0#D(QI=$lc9*;*} z8zIsc$ynS?3X))`!>`e>lox!Y%Nmm|D|-CEn{-)Da9J7MO0i=~(n88VNaa>af@XTC zcAXndM-P2now!Soh{yf}BCMJ$$K>WAP7v0ZMKshQ=AN5UE4cbf>hfKNo`8YI%tmGn zFjVgZPNlDLPg}P7TNbCcI#&=IL!w76z64CBUD~Lh&0VzNnaTL3EJR$_ug1=OQqGY< zkq+Vq5A7agELk7cqf@L-%%RSDS~UM07RN;^8sI*WWzCUqKe*~SKDBxCCgN`MrqM^e z-X{azCd&^4Or+Y=l;>fgyUQjse;%~ec=eZ$tdBfRD7^bHdql@st;aKj#`dj{VrF|P zj&`>e82CrhvOM(eKLeu4EYHZLM?F8K_o+f2-RAT4)sp+OC%+Vvu)_Qx3{zP1{k==w z*l)X~(ZLTbV$7CCSB0V`W(@R($R~eT2c>Z-yqfA~HCadoIVqF;RN63ZXMN761Xglq zv*c}nprLi^1RE`X{dU*!^Q>39+4xG@J1wTqkZ_^)<-f^?GJ;<+t}#1%*Jvuec0ZjZ z6&qj*nSqO{*79mJs1U-;<#490(A&3f3mQeO?6&rY!3X8X6(3>NL`TXly$Lh&|NRNw zD)DATo&wvg*XunfK(xNTYtwZiOeZiEw*JtDAA^#6!5dA1IPw;T)RC3htlbG;3SjRQ z3^FK8CUemJ)wx`xs3-6P1TeG7A;QpE6_sY>oLpLQjVzbX3)4#h@+qwEFQ02qc%n=2 zyU8K%k>T}Cl{=Qdm$KkZw4-dv=kZ&-&FixlXo3;U!I1;DAgr^^*#6b1(FZ zPa&_TT1rQYW7Wo7ujIc-mg9=q=k}RG=RDDztPWnl{JU2WpsuV&8tXh2L>%^CJ-;3E zeDorJX?rSE4n-&&f?dBWl}03U$<(P}P*dokM{OS)?~->W^!-R%#oQw{q*lrnuX$9x zglclV-?P3Y2p;U=@0QilbL4sgWiVH%_Jr~3oSxwOfpFFGL&a$RFLdMOw-D_K1KP4k z4E&zg7lF8sknz2B@Yp>_4KmuZc#d+uQg; zV|N?bD#rC~lpn{6-GAh0DqJFtNNwz*iC)1}SIX?BPIJ;RbN^*BAZ&z;2i+Sk)o#37 zKf;XRNf&>kBaGG8mTb-oEhan;+RbA;EE&pGZ}@ft_+=ppO{6N^=8t|8n%2ke2EDm( zu%N}D`Wt8j4h@yf#j{Mu2{_MEZpT;X{FG{E`c1j6Pe{Bh)G0ulj7mofV=w*KlS%~d zcJ<{0gYUU33a+!M_<)OOj?tbFTY#0V;=1^*7wnn`zFT&hT#1NY-A|d#)x0O5Uq<;a zqK!pE)V9hiDqfIyN=nXj1~OkYATU8<7I z1Mj>u!FF!%wX9R_oeX}VlY6TTae}J75~@zuXpZ`Zdk|6;&7F8BXsxK5oGm~KQuLJp zET71_#|6xoOG9aC7m(m%HG`%6N)*gUB)#Bz*+c-oa9HBy4J|=Egg`a3*?pCx&3=CZ zGrhhz!R>L9wQ#PHu=0K>Na3-E1Dyc@dB;%oO`00fDK!V*dp!K8Ar8e0N$-_)==_-3 zr5DRHT~SAAKnP9krc~nu!5vH1=J)NV5$~<8{6+K(Xe8?Uc<+L=Z}N3lGmcZGobW9VT`HqlcDQj4efB{|-ERoL%O*`H~; z2Xyv&t2Vkvuf!Us0s))Ji-a=p@8IL?)PeH0v0ZQ3GWM1 z;x`vm_kph~`TEp9#&&eE&UI8Ou3qHX91KNAjV-wF)_wvyyAdUlKXgyuxBQMCl)sVw zI501MYsArImK>xC_BUB_G@Q=bt^$5L1@o>vXk9Cue`PBaCHRXmdlQ(4bK zwsz?DR|}d}c6YtrY;)3bVKv<_Mu1)$X}OT=$E2(E1IfaoTE(xI?eB1O%6!pGjl%g! zDbXINT4rH>aNM#_qPc?Xg`=G>^wji;5NU}vJCy^xRGGB0=ejPC0XHS zFc@3s)3n|FHd^MCH*u~mU(z4xM%|~nnp}Vw30>1PmX)P>-LMB0o2a$*T`5b*eT1vX z!aoX0y;r=<&8j?%C zS|EmRW>WI;toQtd0wLMXwH29GeE~{n2|6suj^=u7EtaYiJx+Y?f6>=}As~m0d8;09 zp{1c;%Dpq^S*9*M=UyLRF;A-3bH2j6rLcFprweKtsGLKASYTPn!8Pt*7fV`Sq69T5 zu~qt`uBuFQKcF5^!qj8g8)wn=9R zhkL&7h3o9R_2J%GZ!cIorw1P6T{IG>C?PnVerAmO)r-r>%BW^bS>7u~$MLRUVxj|c z+Mk5@CH@nW&4eS@5yyvwA}bmdekRxE#OFdM0GNXD*!mzViL=S0x2uvZ{eRiB)O zdb)p!PQAqSY#=kid%Rlvji$~0YFq%=I+fC!2NUFmbbPbka-;4o)QmxG|J^wMDiFST zR8w19I*$w&5X7&To>CMCgxlZwCS5oY%{z zEn*rdInq)s;rwQ+5=`?_q~=ldRqKa$uP{J@mJ_;*3f_n-JT>}f1wJ!=mlhonA#gg0 zwP{)XBypjDmLm65I+_C-Gfw^t#zuJ%l4@_k-Oh9yVM5mNwdZk>d5}rH_oV2S<`ZUf zF-i&1P`yQCgtg0!{P2sd#QDf|=9jC5DgjV0h`hCwS}>i9#Ql8|9+puew`=OsVWJD? z1y17drDXC}*n6j|Z5k6WlyWu8mnw-U^=MmYw%0ICsA$uu{LbF^bp7Uia6ghq5`WDN-DTpTDi^ z>nG$wox1YjgOA(e{IScIFIQZaRP4^hC)8r&!GE?XZv}~G&KHF)<>}m&sHQHv9h~ne zl;cDl=^P6NgRnySImWkLgV_WJ)1=fUQmDDzCY9L)jW1*wa@PtSyZf~rf+|H)CKb4$ zYN7tWb=GqlPl~RwY2@PT0*t-c*f?2DT_5LllhJX~E8)oRr=?TUM9Henn1B49IbYU$ zn-6lCAm=k^8PtnTG1dIL7fCOAn|%;Z=RN(&3MGYoLTOv`oLlNL;hVSqe)xg50Auu) zFjjb9cd5@MnxLcQt-<8bnK!_EYR)!tUB~bqhwv#NOJpA&9_=poS5Hv3tvaUov>hLb#9PL~A!hjgFXlB%JyS9E_pm5B!JW~J1ZVMWKp(i`q?ccCAyA#; zPvaj@!#&+AzSW|pC*X}Rk1qLW`C7>hfsMiI^1R*UKyxR;EL`?hzDLvqwZqM&iY}18 ztBnDKj)RUs)T#It0tx$X@qqR~+YtF)BG(6IAdDka0ANymi{WMVBtiZ!u%LHi;L9ofzcX^ysufs3qJ zerLbj-2_UZnsQ4vTUQZ=YG9wOoX%clpcH~rw;NR;w(by4lHe6Oj*cTa-q)ozsiC1pYqIjiX&m6=IO_jK zz5o~xD$Nkyrjm!bGR6}QON6rUxlKcV{5Hf4UmNHyD*tLJeJF%TmD%?=e}UteuWyG} zNM^FWK;_>>(@&>DH;sgEcUchjZxk<`pbPt&Hb)m`0CxlP+$W;lGM^_6Ql>yi%OqLyL;{Xz#}82$$+}A z$%0jyImB9P4jhKcNTdRkAIu*{~0_Y@0v1nFQ1Q`&+02ST_pIznswb!D;AmJ zElOIoFRgbfI-BQz+Ad`OJ7>01(sI0Og@?);6YBTt%DDHyJOtw)-h8vo>7DmCj2BYG ze8CBBirdC(RJ!=Pd#LvbgF93`Ed{wWGy}TVx=uvo8^-eFso$tFZ0;#k0q%@juWp7% zBGPx1BflzkVLaIJQu}qfi6-*B7%`RC5}_5Bg6GP9Ca7SC^iDylTkq8cyxv)KzrJz1w(#Ytj1G ztKGG%jGOT4rl7g2d&py&YL)zWeY}wGLfN4q$y=X)46Il73u|stGBV26r5}HWABQ4nxj&@aTZ}8> zoAy2)3*XDKogQ3RPZB4V-y4Po)yj8mA zKYX=V%#V{FOGzi>;Fs#Imw|U2fy8a2*tqAMGAQ!+gvD`vrfMp38`utZ*Pw}a7#=E& zRHpnvB6y!5sbpBe&IRLB97^PE{U`Aq*pV0Riie37y|?bCzIJXYn|Dmj17d>I8~V7g z>w;!jud{Sv*q*OVn(^>xlparnKi*c|3uZv7HgfNT-1SlKmiQZk;PZCKkT<#b;y9!G zqeHLcojh- zksOxMocSTDDg8S|M$ihQctiZuT?1wMxRE24T*im5C| z_L6wY=d&bZaK5f1@;BTXqP?&zrwE03rRTC1|MYZIjkEXJp{FZDN`8qV8iKjgU7w!L z=`(@Ugl2mp)jBfgk~AEZWM@4dULr<5m!!A!fMgor>b{#K6B@#onE5j+EAVt-#cAzHVRtS6;#p%S^^EAkG-u};njkB>Nw(aS_5nKZ zpVHRwok;6!`#m4d;`D37FI1S7-LHIGI$}01{f3elz)7W79T`HQ!0C!KHV;hPdTFs(mD4bS8b;)pvTkaqK6G_F{ZXn{(|8=!5Q*_BoExXhM} z%rHf({su9@vr>@HDv*XHaiJ6nNLF{>eLhq3=0kT0Aq#Th#=@b(rVsdY{o0i)ira5U-SEY!rb`4c~KtA~}U8*a1pKw&~< z^h=ZnX{ioW4OdiPP7;j z2r`k$d#9vvf&_~)y)`P~)z;Wq=3bbDzVdCCZ^7%!{d^*sA`dj_KJfWkhoG)IAfz;H zyT)7anw`^|HPp;bR0FGb?kG8IXX|V2E0OysyTa6GCQ!ESf8?@L0^C32-WZ;j=^BhW z<(|A*GgEsK{K6M9+Aesfyyq*7DdpIfTs!5F5ROgbhXZDR!Oo?b%TC&4>C5KWSbM{g z`uc|fyJain{s>&XZ2G9oOn@_EYt%PYjTBDJ@Wk?CO3b5}<{slz!6QaTD}S53H=@0% z<&^)~ANQ5@ri4&9>)6X#-5(x#t76Bp@k3tjM@=6&I}mT+pOs0gKDi0$^T1%sr;BLzV*a(ph$ulIfEz%Gx7`HoQ)Xf-bRSU$iCQ|%Is&b^ECfs8LF=(6dxRx zkhrBe0MaO<-O#g}5wPVPSs|u}gf0fWS<%Ncd*L-N1Ymy+Baetb%FS;cfWU;V+B6u& z)H*(K-9bUAwueN}Hz4#}mWR%VzZ5_<=F^ArKp$i|u1+RJnnSQOF3tee3>o$lD>1%& zl;TxiK(8=pFJ=1PjCV$Pr=_;zO#ZJQjpal2M|>`O^~B~fbJvKLGAd0zu+!-RcM{hD zx`!9HX!@T1)R>5yc@&=b_4a(tqP^_JZ$?}PWe$lapewHF$Tpqphs2#5eQ$yuv~_&< zIHBtB;_68K*FG;CkNJueZp(M(jzSKVH71nx+AqF>3fHr=73e_rhW;*(I4Z(r-TpjQZVU9>ZH~D6Hj4UaiwQKyc@?Gjk37s}C>F(a&WT;r;S{u)bo3|99 zOR2%hLMV?3r_)=G(m7T7TXhWC$R*sE>3)DL*zng`;MrJ)#QnV#@ZC8@XD?9Xvl3o~ zB?P@k!)0@|Q*&KtdSl>P-dNa3$wKw{^`q;%>k>I%77mJ&`D5!w#`ZQ<(tmfiiaO?m zj~yXO>?a*wUF0Th8fQGtXH-HJYRhMk1m$u`Ubwl5@D{C%d>6jIR_wi!eXGAl7v5^c zQmF5CX+OMtkiM>iKr`g>cW3SCGXg=tY=uqzntT(bdJ9EQK;}=J-<#n)?0qx}6Fyb) z@r4IekS;Tk%*Ih^th;gcy^*^~x=Dpz)*8!W&CRo254H(;)Bvc+2KzPH-RM6}VEFbK zGue4B{_V0tS=sZ?B4?h@znK1+^H3s+By{7eDz}4usWKD_@`#7L;kpEj>_6YIdm$y^d`uCF6YJH<{5bgCem!T{wjM68~Ow{I0mvAYw+0ENX@7GMj73w zLZy9n?z$`aq$a-}CapE9l0;_ubU{)8@`R|4lB}2|P1M)H7%cI&f)USEat~y9NR-<) zPPX6L^KBk#@J&5a=-;jPy1h-}mRr^7e5(_z2122G-NK!C$Lq6*8})H0>aiI>j!b71 zVJ6%)So!XL-}@oGSkL&6m$aV@3$48*^j-p?J|e{X)vVuaH*wh_GR2N1?jRh_mPC#r zV%il}LJiMr7%UnTupc3gy}I#m8@`bF`t(u?)g}S6duw-6h_yjv-0jDkpfwk~Lnf@b z?9=?AkSx~SN>XoRi6KRvO>#-9vU7`&EbBeIyW(w<3Efk{B^~5P_Ka5ceXH>ryEJiZ zS>nm^m8H|Er%{L5D3x{}?2df~oY3LH?rtbj>ri&@6lGf$^7m>_{r^fPS&Vqs8~4RTzrEj#XpfzUoEbj zso+3p0;FgIwErzz9}3flL0|LR0#VoR@M>$eFz}tZ{3>AVHmfG!b9FcA8YHBIPNk+W zvzkvZM{yJ%#}ypN1M|Ouk12^*q1o-h`42SVk53-{;y;{!2;YnZy8doa#-&h7Q(bX4 zWfIgJyc<0Kpmz+aQQVBdBU8w}1NtRY&RNN>tueIpOpWB-w0p(Fst|&mqZ?}C` zc)d4siz=1f=k`oN$O%@TN6tAnS9SKcQ?G0bFI*WA>2GLg(D#h5DdCG+1b{3|=mEW@ zcSca!^yu~XjQYFj^B5KhZ0d8an-lKWH-}tl(b|T07W7tn4%T~qzmSnCJOVKBF#gz6 zkzE{L?L)Y3HmL*WPNq#ML4_Z`W3XbXKM^?Gy#Ln=%-p!oCnfQ#DPYEDYzNIxGKL*! zlnUBCbDI7{hZ~qx%^w;Uu<3i3Be`%h0$mg^R|R-8+vj^1c4s>^T25 zyopOcxrCyZ(@Ea?cVuW^+z&f27oUDPnZov3eo6_klpH_;c}3cw9#)iq)-UZQmIOAJ zb*;W}OW?yX?!!P&r?g`DYR-51hoSyX1dMit2^Phy=|P_p0Y^FSB>x*sI=f=T$eo{G z8BW8qd^nNSHRDLzNqov7Wy0iJNK10Y4?BCm|0_y#*rIB{6nq5fOA|Zyxb~H#)K%z~ zrkhJPVdx7dv)-}P4Keu3O*IeoD@MZH5efLcWR)eQzdK?U)tQ?posdrqnZ9jegd0P9 zfAW7VeEYRNRir3EU`(%=5K4af1K|i=9jLx%-BUuay&lP4;%rhy?Jcf<;_5PZiheG_ z--4?qV_2v#HoNA5D)e0ycS%n_4j71gWJ}8E4AjWkqm->0^i-{yubI;VVX*RDomIGp zQb>%v^XVrCRJkF};(q+51^@^2-tSN0fP+H#CDv2nr}|Zk72j7k6oGfts+m3YGcTj@ z6uZ&_Bd-!BgOQ~rh2F=DPW!{|(=RfLK)Ah45YC>3#f{C-Oe~EtUQL6Q)8mGrWe=B4djT)(40$UhRW>#W}ydMSlB|158sCX_biw3je(O zl(dZZzl#zQa$e)^Ubtq3I)&g<)}qw|$JD6kgvo5;)+!%AiqH9kJ2`-n5AXJ)QmQ8~ zcoe{JYrlz+lpZbNB%Y2v__8BEM|}5N?;UUm)GL5i*!c^UjKJ)ocQvu=#iK{Zu^Sja zsh)BExCO8g6r)Q05FHa0-E0t}mTg2;t)!&1S%2Yvf`mu=Nj`r9wS zlEW)Bzc%%|{!s5kWa0^LZ}0m`D%IL}k5QaD0M$vUahi$DqmhR&R1an}IPXmHl8Wp6 ze_c5LtOgB!fi&qJ^TnAf`zF8S4#Ficq0APXw?Iwc$kxOBN8M6ev`|#obhCVImG14) z=Y?Cc8jI_+qa{ zRpy30GJ)rdID9PUP8ASs=+=lo;0&FNIjgavjvzKw*Cs|O^X}i6dpa^Tvy2XD@l@`n z=aOL$P+{Ojdx~7rn!O)cYC+IL-8}Qe_QP}hyn(mpdlKP-oKiau zEz^TOWY{w(Tinf8Xq}_B3xqg z?_K8==lE(#ygv@vHe7WFN^5deP6{1RQ9CYmj{RhG_9X{@Vtlt%I7^5des>mlhP#xe zQpp0>Ini(p?{*baW*!tH+upi=BlH9+;}ZEWd}~a2V~1oH-Bz4P-FOGpHN5#@#@Otp zYBY|Pa?#@^zz*PaOKL*?{O(e={1*`!WnEB#6+lmhW-qH1M=49S`@ z*cEnT!}){TGp3VXBfN6s=%!egW3k2P zz=ILb#K^dB8TQAduJwL!w`&@;db?l`ecD@r2onDtyB8i(0R-02n}lA$2yS%Pb<2AZ z)W;a?((6wucN;EBnBQS_MmfB0lWJ=iTj$bFW{YmAw}NHyHqcM=w5CI1k?VB|rWR zAHJj(BWvd$7x4Uz{-C>382a?icp})~=LOA`t@PeaIKb&no%~JUo+ir{DW>!`5LxWp zQ9@gMrK*F*7YTOhCj^skLRB%fn{bD9cUFi`%YcO<;+Z&E{UfyIBrjAIv!JwT{WE(3 zXxVOcJx(+PjBD*yL>Ar?0Tl}64jfnA0ba? z1Lw+etRsbpqGhp8ObGPvTD@_CgK|tn*z4r>wt}at0Dah5!t(36uaqWd-EMIDXJgp?ox8$q?O5=VREtd=c~<=hg0MI=GWn{Wb(fper!OoYUS4sC zN?thac~Bkh;O+j$Qg$a~Lz77@6qN-NvapN&xz^3k#UmHz`J!Qbf?PRpsZ{RvkV&|IH(( z?aK(y`e~$Qy~jxv&nl;7!-8jph0jDi@?Q`@PAyj%KU{+DL5yz5t&RSpQ)Yl;d=%@m zBnZ8af=}Ys?XzN!Kp`~+G2AWS2OMurThoRnrDz-vWySG|`}VO{N0gQGPF!FOJ!o^9 zPwiCs{YU0}b?ZdhOa*}pXq;q?TSbDdA)r{PXmPF9F61xQHo^4Qu^MorzEaanpTDFY zyC=3^E`_El&W8aOp*Q`#9-M(y>7l3!iA^)8C;l+@ed z^xiR!vo<0d2lbCio90NL$8IpWmyJ7pcU1WE)7I+m5a- z6#;OqnLde~ACc^fuV;O}PIc9E<@;R~tp31H;Ug$kSwe6VHjNurM&f|UuaEE1hPo?> ziFL;vrd3U1MI1NoHkIQj+mHR8 z`YUKeBHsb`sk3d+PmYEpF1oPNF3P9#FrjU`ER#!u905!8Ug7?(t1rYY&=Ykw2OUeI zmB(9}SB+0AdfMEW%h_Kb5N7Ak4^m1JQ+?c?VenbubK?oR8Gr^B;_RPaTo%Sq676b) zN0NANi#n#LNXi!QA%-OF)XbXp35khitVhz<>pInsy&7%0`8o&+iw-c3I_~dZ56&m4 z4J+^HsdEq_Nc+MQ5MzaE?aY>Rc)a-T0wV~~)7QW;!Si}GpN9XV^NqpJ)xb@=0~lg5 zGctgAYwnRJ%S%f?Ao1j#bu>?f-+?<`R#=twgUjmvi`=TIl^YJR0^+Crw}*I$vPYs% za^T0(S|L~KyA8AjK3Z`TS1azhfCJg3_%Jp`Fwdz4G(_WlN-Xsr5SKnTS_P@x!OQm( zWE-Yf<4L}D_C_IK!iYL+dkdl+Xmk!Hu_V`TQL0r0a+BM^q0R=?eD;a03%Dp9SWgqw z3dXa@PcyYkp859ds+HBbbKD$6*PQTLcW|7RbJ0Y~*`%DR57PW$yQ?OBPo0nDF{c#O zl8R6Aa%2o$YJi53x~mIrPA7R|4(Fz({k;(4D)_Nw|C!o`oX2cUSA^5Ae-)}^IeHjg zz$SZm>)h6_k+Vygo9&8SXks%s>$E%PToOk$pRaRS-Q~kg*(CIDEZbS_B$&iLLFtI} z72P`or*joDB6Rz(5iDsUOhribQnq^*Sie5msDhW7;yY!5;?4+d5ePs#>@L{ty8-5R z$S{|NqI3tmvRD+ls#W{F)ylzxM?XyDTiOlYHFmr@Ya;xct|+p`Kv=}tuw0}j?eoV( zt}od4mQ5Rt|KS4E|7Z;PvcO@qB9fLi2}So0P3W5_rc_LH4><9!UH3s_uD!%fPG?U6e*saM?5psTZk zktO#rmC*5=MsqT*9gnou0$f-Bn@sbr&o*BvFRKMA`+)bauMhgPcLMQ3mLn#DL53+K zwT`iA0F=z!aqHid4jhHOV(+(v$d=($VnPBHN~o7^uI`R+N{Jn(3}zRsZoh^?YdQTz zwmHiPt-)fLd)_9B&*BlAfm1=8Iz8W~Gj-E~Yg?cAyjmwBy)hB4>Fa}zscqOf=o>J_ zIydI6iChE*mlFgFln~*cNRxNe6CMFlvqJBHmUjzG);kcIuVc8hs8DfjBDJM!(_2BT zXUjpl*d!h3^jM#}ihQJU`kW={0sw^1r2<+`v^0juWy;EWm@~rgCw%Yid&$1gy+{mo z+?Uru@<;NZW#8@W0^@^#XjTFLuw1X2pF_@%r15=aFnYzuQCxWFkxECIbkf?8``Nh)7ph`LMyKE^L|LU zqQgobr}Ok+tBX-`r7V5a=;)}AtbvQjH^UnqqWb1v2aFxwvxKIYlt(PxsjXFYaHUW% z%M#S@2>8_22)f(@vKOAUA}q*fBfH5o^iyi1ffFB9$)U$q&QD=F>eD^J^dYqMadMNv zF0tKo1x)gUKNM8%15_%Uh6c&iJRy*_nQbtE#iI=GSL(}Ze6T0pW8kPLA zhwwQUcyN24)YV=i?n)A$xIFzlSh3>xki;mLC!-z~g>QL;CL?s{umUZC5F6xhe(nEV z_ImvG`mN*zpVhm^r?~QH;`7q{;uPs>($tznOTv${I*3giTRR*jbVP$UvTmWVdMXPfY6npQ&;KG{oH0Cl+Q93HENR=)O zc#2||Qx6EOF`J63Q0RB?>wne(Q_b^>aw@a)--p4L(6L6G=HePnE6aW6Pt;dklMs3? zu2sQ5PHWx#_q`dKBiezAf^lkHjEQn8gw&J**KpTVen!82O<(2JylAd%zC}ze8vDvC zGi>tX*uu;;)YyVz>7*4(fghHNXOF^Z?bn|;QZw|v6E0K{CM^sO^*;1Kj<}-P-;*iY zC9>xBe9bV=^wHFFgszD%=zklVKw3Yit1a?8?Ix>ne7#2IS&^JH!%rWu;f_~N5V{&> zXGLZ(p`YU1QFF}nnBp;Mitbkieg>v~afY$5<;bWP?a@dRU3)0<#7`W<-3_JWG_~(6 zrp;*o>lo6Ob1A&2!S0pcgO_X~J+%eoc7e@|&I81mU{6Lu${RYLlwG=+0?5ZXOo}nE z7Ubb#3GgE;Hxsl){@|0Fz-iI}2CvXoSjkEIsj)#9%>LwC?4bYgio% zy0EEpbLm%O!c);Z_N@(%SP!q&j$V0Pv;D`Cqq%IOn4HldunS% zifAG5Em$m71Zpsewr}<4=qCaft8b{XV4+^tzpwCHPK+#hH}D8*!ZtVRLEH^`MQahv z&a;mvzg1t78p(1WD?0yGUwH2jsu-mc?R6?x(hGT~t#9EslpcWQA?$F^=Gb!}=v?oF z!f&2F0^k$lQ67Ai-q|=#=6e$7|FZXX>kI#o{co=a1>nW?q7qBgsVH4w3A!~XDw|6ox)n_nt++O!-7AMtemm*?g--=+mK&JCErBVQWfmjUj6<$Kn0mU-Y1`V>7A|^ld3~0*{pnc zaw7HCz@>mZ#V*ahshOE&tD?atTiLVTQp7DD;=n&k`c70oy2&(CoM)lfpq4ek{)$N2 z`zfuqe?DJgN7U%gIzN1G$ASzfmrnY7w!J5u)L(g!J%5|?f16HpH2To9Bu94T96nvu zT8kPKCbKJOz)&!nBowk(%d(Mql@``SZ$;AQw8s=aNbvLUGkFc6P`SPW#zs3M;a^Bi zxoP!CW!8$Y|1-z#4&p(B-V}-D$KJL=ER-Xaj%sKiS;Zz*EvL4BPbA7_f3C$`A5Ia_ zI1YmLvS8_%-nd;(4vu(vAooj(PanMN_^0B6RB}Rj+LzN0&$KOD9~XPY%)_W6ShaB~ z{q{GAMI553PVR`8cL7mx#UOad1%HCQ|bE~)Yotsd)ixE;E^1T*< zl*#n@TI+oy_0Otr<7-cOb3Ri2#6xUh^aQI(=qn7X-uqlydI+tOgTsWny`yj6TmCm> zEp~0|L**hj%k*V_ePFZ5s;UFQea_ee@#ClMr8U7N+>vMf>jNw58NC(a4nV4zwaVAN zNiUvLT%`~rw+GXBh5uXrIhA*)qvL4vubUXfRgs>MGpJq;*emDM>w@hlki>jmrVP+VkiiJ7$L;+0FoL{_8IKEOJas|9V_Q|`+mR?uWi+VfO5u!O3Fp2P}s zT>gQejEUUED|q{>(wEP*UvK`txh4m2&Kqf!L8u60&lkcdxu?7KpZ5w!kPQ4GjX;*gHLl46^9oAc{%4RCy@!$n4ob}t z@p<|OQE2_kzQF8Lo4Lz2(5O2q(slIOHzlwZ&uT)nq&GYBRV=W-*zr+!^}l)e@-ee) zMo`m!ZUiSxZOpwFGpyS`yqdMv5z7E}?pIbONFEv#4RPx!{>s2tuQ=b?+e>RFrf#W28%)5G5H9E+&(D5u zd$1%$0&|Z+w!RXVBlWS~coN%bafM5kNK=PIZ6S@-hkp8?V^zbjOX3_I9U|)Jk7TT3 zB0iPH8qv#W88)`@g_DXkVpn=q3MJOG4RpYOTJ17&B73uyJj}LF=+1m3}g-#gyT$!ig|L!Yn80Pio#)z^QxrbO` z@`s1AkTq0V;GZ^vCx-*Q4!S+JY(TfFmrSI9( z#rx6DvC{MZCWp$@TptfmiD%mCbmfy##~p$s!n2;q->DAQwjA!MZF`DC)!i?CQtQ}~ zpmG`9o{b>dP4M>^9*O%7x@!}M{6(S$DS;LN*gD7%{kE5&Uh%C0fIg0LJt{d_f$l1(7889}n%ZTql>ZqRKuSzJ#Q)WWUO=TIqS89J?q z)nCjsth~$EA!3KiVfm?dEEuQ>iOq^5HOhokMM5y8$zCeV2z(y>ZdEc6HbC#ePKu+u zy>&)*P(0W}vq!rhw}BO^svO5fmEte%h$5BTwRIPRTB!@?zrQN?dvf%wvqi-Y>Q9#h z8KhGjDirm0wPEk)NJ!tpMX8kNZdBKCT;V*VxEQ@}JtIwnC)d0)4?l7N-z-K(uHLO; z0M=J}OZw0e48mlL<&U#oFUMnEK9*;bU$^D6z%H5oX+4hW%^-!`INUE0XhEZ=vyHNf zdmpT_PHCve`pF`L8zmTke$>5UJnedzr#KIH&{=fXLUYwn?PfXrye*+Bl#cqF-?K{k zu-)o}l9&B6XlRh00cYm(~`KMt+z z;FvDViZHQ4?(5pmFXk{z|FS*!T7(`%rix&2LOSU?b1&eI%%xYkQq?c9RbZbZ#9$ZkLWF1q~weciQW;hhs}HkP-&f5^VRu^=9{ zi3GsBOz<}Eq!%ZHo#rd+6V9P{VUca?*JT%^SOpK0u&Gl4DqkGB%drb!u`lg%00?8d zyH^8l$H0(>L4<6({&QnEFk?fg5~Xv#RlM8|Sf518kX1R5l!wO2qY4kH{sQDS>zxS* z$pW6QMw9AP@8g9&a?3NmWpwW?HJuKpW)I#&fwD*U8?$W9}g>$06-)Hc>VugVfY657xVQ7Ly`a?HSlY=jvJ%)3^ zwuYOlJNPM_a)JN-nx>Q3p2uc+4Lvm~-jq7?C=IV_TI;FY1ZUF`t|J$4Gxm(_W5VBo zL2Q4HusphISRC0Qr*Xd%H*(>*8{IZyq~Pq{dOeLTSwQ0(g3tR?kVo3{6n}cTsof>H zn!R49foIYHv%3faj1wb)O?|P!*TqsdKY`;ZkPvrbA$fp14B&@MFz&xsfefw==I1MrE&o7CQ92^rg5ZcYp zKr0!4jpE7b3rgw(Ss}ZJc;5}wwg~do57GvIDK5r(>vor;OWFQ|LV_33iOh#v6De@X zZ|WnYXJbUbX^|_Z_jIF;6l>2csCh0S;w}`rRANTzFlTb~L}qnWr=<*nmaZG=t1|zQ z^-Cc$a&dCr>C)M^k#0VWC0maHH#%DHojf_sp_|pwK)9{$hplXf?IAY>9d%j*xfo&3V8_! zx>yf#n$7+tCei{BUIzwyHcHt;9Raqzhx;&d;t?mp_sXZ%=w z#rej+&4?MK-k|G5Mr2RMS$5%o&PM`D!c2IniJcuCM*^zBql&52Oo%1)V6^>) zk))?)SY)OK&KfOwA3)u{_kVn3xPkXX~wI0CPx>pCWOl( zpiAP_UOjPYxZx1ob$k{YAL7d|3`6FZy9RNlRxf21rkWc>vN*00Qkcuqj44%)IU$cb1c*h6cnzzcP zlsi>{R6U{5zMPH;#co2Flw#7gz1{7na&OP(M&ofGwc|FCMAZ!M=l}aPjteK5^cTi5 zG5tKwrTGDpBF2zu8JKYvHS`scsSyp$$O|tnk3Vn%iv?w8{D3)zxBa)YS!;HpNrg}q z9xb7>-mVBUB(+`%BaIklAsHFfaV&XzdmFnPQTfEK*|vDwrowlolZE5YNiKm~yYJ@? zm6*VG{yTS$=PE&03=>-X)2sP!Jq7;fMW6mh>m%=?>BYPr0gS;<8gvksNRD}y)g@vx zBNtCvUw>5BnK96+R+1s}^z7(DTG?f{h>Q2ueKQM@x=+!4Q0@T_b!~-(|3Ga{IWmyF ze8M50^Oaeq>m{!eeyWcj&zDzTNj<9ZDG@kX#>pq)rmjryWkY5Gr{H?nN!BbFHHPaZ z(CtC|)U~CiKb~!(UphOLtv?tkCyb*?|Dl?xc02q%u9Xdw!F)uBlE$mx6sOv3`DPUe zJ~@|G0bTg4t&G*O%?xJ*Z^zz#p!Kot-@w+PrLo+x&{hB@kZ0wnlxowA}UEiBY7po9uHN91%_+ zdK9v@+C7k)6#rEl`3}KwSeT^d6^tvQ4v(+@f;KCIUoJtnz^~U-e-rMZTk(TdA8I|n zzoNvn@rj!4uXpJnkbnpjhfQ>b02Kth{4a#K{?P=_N(Bk~z4F>}qS8i`PpqLrn&s+-M#5_FNfu1i z!w+LIczNPJSVRNaiDUHY*Vt{~EJl%#8Y07~B~{2P1t2N;%J7v;>0>3TINn{D04F-2 zctcB-udIAWYik^k#-&YRz89+bmbZQW%*v%UobS~nxf7hA*V_fExi7z{?4>%2Ct)&t zim@qH@RER8D17#RfslGH5Z;H?W6{Zw-rw#WGT zdV7zS@0ydok&zJ=;zwqnP-7s$lp+4+)H4+?M9+#McG=dvo$^}{ud}|Be=RIuv4u}D zyVMc@SRCcU@4>vV`4toh3fN#Vuv>6(E}6QOjn{+f@g!|v#?C+Nb)3zZT z&b6BU0xg0r3e|jVVCA9ZZkP*N{ko$&>k+3{+4H!~}6Jue|UmkDk zDUzgWGx#^$l9~3S*ysU|vb*b7u4Bi9cAd=p0L=f|l?%r`jt0J!X~b!b4TO`^sKlqv zY4r9iyj!ZztN)^U+(lb|>0*u>Z|CQJz}C~WHG?;)1oaKPh5Yd(VE0fe zX&JG^u~3Svg^WBIsRyW^#YC$k7P$`NrE_TaPLN)U_;y#01g9r+IzAiSdDFx9_D z7QbtLBfoy9miLe&aEs>idce zXq$gbYfqjyMkoXpRnZ{F<42yCtcJ_Y--#cGfdYfsFFZ>e8one8v6Qw<6c%?YuOI)b zM9muY|bi{ znEG2y*y}0ueNS_t|6r3i!A{tRmk-^A9Z!EHRcmwbO3}vRDo}~E66=|c!%bC;P{9zH z@*Hv#-6F5>M!Y_9i2S&BY4`PVlOi)1f=hzB_5AtstZF(7kLsK>;e=9wsX)#7diT6G z*sgEx{Fdjk4Ro-7gp4wRfJx6t`3!cc;Zgq)k8&tlT(*|J12JLgz_ocmW$wV@=L6N~ zbk#yHr;N=N!CSlDR;Ny3PY0+i-;ZR<82C3or}8tf3R>O<89rtPi@d%X2ZMNf4N-Dg zSsBuJ?Zg-sAoOAg#m>IABK|_#KQeCyq`)$$#hFyLaHe^%|Ar4Dp8q`AN08(oCl}5FA$i|KRDcuEA06 z9Po4g;S(lJolJ4_jE{rl)yba=2tSn{UgFYVQIHB9ahvbAvSVBiqHIcx8S6ldst91C z&psQRFt`7dYQdqezw+eulDhAZa}FKR3rhX6zszDg@1TNCuH3AEf}9Q!cb{)W#iAL8^gj&F^-ncim|F@9e>fxCq zBF&gk#e^PQf@BNRIoCAuO|3=AK1TcE~|5v0O{yWLW_NsdJ8+5h+@#2M= J`->43;Cm!{pdW9Or4*z}_EJ{r9QJ28H?Jc;7YcQLj75bydbcrsbVX_crN z_qMJT31f6h|4pP7LiZGcUY|s-cR&%O+5~3eKG4gY#wl0pUvvCs9)AMkdXNQ8!$XuN zubZ9D+%xzwqEgY$o|)x^V|0|kCx{L|A3t1|c9~KE#vye#7S1%tbWzizIO9l(I669x zMBU=Gw2JhvEa7i)(T25T^2kdy|9X}xBX&;j86?TUUNOAWr8dLA%|l>t zFfK5NJje3O3oKIE=S=vr-6>b|6? z%qg+Rplcl8Yf!n}soKDW*6tyJ3OSEM>P(z}esisTgeEWviVKhY-|}qE8T`NJ*+`h| zwaZX+arxjlZ=&1E zt>|Ht!DE47MoQ?_B>etwcsJk8g*`FG#r;JWR0+&#&NHN3#}pPV?Ewb~s&{mzF(lEp z2)`JZTy4Bhmgq^%5hL5f1b5~PH*grKq(!==o9?2Tfz>Q7D{J8|mnJQ5fxqZZZ?y7# ziF)FM<5`ODyC9vCSkhWVt!N61f<86uK79eR55;xb%eBPfNF&bIs711H^HWrzT5Y27 z2obVsIspDCwR3eGAZ%L^>^huo*IJCXO!uki>gYdqJcD`z9d)Q|M@)*CB$_6kTZ1}t#iR&i5yx)G$Bx$;+`%Wue@*{!bWA49xLU? zx$4)Lsy=5l*JLe(Bo(u)VdKsCL!yl$v=nsohFN$yRH3{+c`jc&zPFDyk>&+jTRDIH zQ{(|5szfK$w%Hr7a8n{<{NzxsrD}+vN4iS*1K#BvTExJcbr<^IV;=+=wWowKj(bu> z-BF9mLIj6T2OT`S`(Hl~_z~1%Z+@_Tiat$LS15_Lx0Aa|b&|m}&3O?MGMzuG(M5kD zbj3vz<6FST>EMYH642_R=n_=po!twXcArqBmt9jC6uHb|=r<<*_fz66g{}d#_Uz6q zFq4=gHy@*&c;OLI^!TK>5_a?V_9j?*Cg#(j1O*hMhCUw=PP|gGYHebXq8Z!zu>Uig zvv_UBz(2~`pfl7$hY4MM6x&f>^*2qq#fF!T+eX{yv+}SdQlyjt0y9$#UWp(97JbT6 zsw1r%7gk51yTH`LDApswD&VOJKC@B5Ye)XS<=WJr9RnvNDProMAt1Cg8*nFE)My=} zkNEHx{b@9UbvL)DkL@o320%@A!sZp;2d=I3&HYgLYnw%i1!_Y>m9kH%rqI6u(Ok(4~)f$tCUpb~f)APNSo=J>;`MS@~I6Y$V_cCvZxd=_8;j{xEWf37dzI$mr=G zqKv~wV(Qz0N*}h<`J6ShkR=p!0XzD_h$uhcK!#g4NAX948PhX5(MrH6xrMxg#Tc_n zV_oBaXlN7zyW7o&{Se!O14z~VA5zHw$g=^ej~Mytie&i<(!Zgyh$O>*^$e9VTmRKF zsFs#7H=Pwz2iDO>v~C>m!;Qw1q1wp#LrbiJT9^C+69@hO*zyLd8T0^SUW(w?c^v7Q2*&H|&cp0Np}=!yYim?GKxXRL zPqQs!Mb1#ySl(sd(lpyE);G$67m$jsX>sb3L&Mx{;~QfYg<%+QD{ddkx+0%zP7vzXDikt+CBa!ec?a+GwQZAHSy zUCpSKQD>JbLdS4YsU0r}d%{HpsRi#;Bpb6{U{)lF`%oox0Mk(v^#^GE>F>;ty4+z# zT|?vI;k;8e3GeU#Al0&G6T&5nLkAWe?iYvxSdqBa7r7f%Be9cp0lFG!scvxCG%k8Ps zK})@)c#`AJ+a(illPrD7{*gF%-i4p^;hZF%<_!O263LOX*5{|$gHE&6L)BkyTDBdv zSpZK}RI@fvVCo>GOVaP-#@nPnMSh{YZ$cQ&Aw*`Opa3&0{M2Jv6^pzcyv|LAM<)k!YJg(aL)9tL>++eNaTmE-HWa;@=d|5-+zNFpP{)g#82po8L)?$7n z>Y6mJQ)|m6?;+?a+pvVDv5;l02cBQvMPH(>VjQTneE)lKsPA~5f>G7R(;5mZPELau z29{n>V>({eYx%3th1rc{Qa?0${r3h=`2Uz3KQqtA=~i4q>`PnnN@(F~7o=kEy;Nq#AyNxk;b}-Vh`(kgyp%Qr`qMj5 zbDE6-vFf%q;{KKHKVPAHLS&)7AvqTP=WPV_Put-aUP5u5BfjzV5GpS?a;jC%a^s?B zgOO8F+O0JzZrVwQS_is4bCvJ7EoS0!cTptba@O|ci=yFsAn;;W{Ku&qY@0$G($=pt z5}3E^eSO|DsF&e-~qbDOa4M{ zb)=;J7i(`G4fX&3e=`Q7EMwo98M1_|*^PaR5G6{8u~Z0=ZOEE^uk5=>QK%4-Ek<^g zNEl0~#Mo+NyPmJ!@6Y%9yUz8!uHX5cbN%-|=Y6Wzyq4$l@q9dP_xoMqzwZT|^0h3m z&{-IRcjcKdai{)%>VM9;J{)ZO*4La3TqF5dd}G<(R<#)1Wb}EV$HQVIya$OWHdtO8 zo{GHh0|t)zZ8{&6a~O2amWUZCdldNqw%7P9L7ZHBN8b}m!a_|h?9S=0!C1a>YKD6xMa~nGQs_QrFqs$%fU2* z<{CH4EKq`|Bu&$ETmVF_{KYH&_g9N$u;n!%8{+liv-HLOA$m1dqiXl0JUb;5Ql^UA z=Kw!w;bD~4xCBo0mPueuWutnUwZ#ZyM&h5*a{xvrW?!4HFQi_$U`6_z3+6rhOE5Rh zg9u>4iO*h(QdQTX;yWohhWe@W3!@%gKb|ZB4ymT?S_!QgiVUF_n~(>@Hp7N~VRTCx zfLXm7y{3h)JLJFts=Ll4*`3D3TxIAL%@?~-FBFA~#ltvs2975xVaPxdXv-!BI2}(KyhB4WtCR zqARqedqy^j|MD?$N-aE1P1>ceXwrj95Av;Y;DRkH%t8inrpKnS3;#X5=kH@z29J~#Tf4sI3B`&=Y z-8KXzHQGf=n|Q|mnTl!mRrSMsX&U=W#Bj#@#jYqpL`>SXf27Gx0qP{5QMC3e&b{_MedrL=uKL3&K_Evd)1~jBC4SqZDEt` zd6#K8opGu`>U13eIzf10JAd7akI{Acw@CFu%rJw}Oy@Vm4a^$}yw|yCOmVqo zn*)Z$kT+nl@(kTGXEf)3O3!Qo3LtP@l#Iu0u2!SEMwc|e)0P*-22*VV1tJcOEJd-q zybCEH-Z;m&{<=tK}G-9|K zR@jci?ARxAlgi};jV;(JOCQlrJ!boa`f&#g^Gj*@G{f+(?x@T(`TF_}0KHy`zKSKu z^*nbLm{f2-`f=I{qyl5^r-ZF~-7@iAxL0qE4pE# z#?NH`4!6g;(=kqu-&lBC60u<9UAoxF@imUkVOcUID1HBtjvgAeTfsHC z%u08Vl5+EC@OD?fJg-yo42UHD^-r{`P*D1`ujpo`Bh7uoE$k#*hwg^3GQr8+<8m12 zBSye@^hV<6t$ao%JJ~x>@hx z`7vA{ai{DN;?Db0zPXW|+;mi1LIDzAf-7A-iQ--oW*eT zxWf2^`Q+vxUF@_Qd{!E~^Fn0YRN1rM&q?6d2bdQ99I6wb1+1r%$iJh#bE@7@s26@t?}udlU#j&XOnq zivrd_dbm0ewq0E4NoJk>0Fsar?ekV7qZO+L|G5M)Z*4h=OSjdNA;YrC7&OnC0cjdM zm*%{;f&$IL?eizV0Fnj2AK1rGdAz%hmMA_;_s|3@)Lt~8Fp&RGl06U)FlhX{;oIw}h zcpwi9S+b2zAjV2WZh*lE$ZTFRDgt2bz3Bo#wkxhW zYr~H~87u36--u!RsC8uSD|MhXAO8WENSXX8Ukn6-7ner#m%vVY8zzz@IK^tHomQ>Y zLle=FH*a2ho4{`Q(hpFk>@z1iULTikl>yG_EqoEe1V~!W5>xuC41p7d(Fu{+gNkW| zavuv{es-O-5ZXERngn7Sk^6ut&n?sl={a6jxGoQW{B@bu!0w+}M%%E6Wh}1-l<8p6 z&Q3`hwCqzI4z>APa$Sw#&dvM;Z6vD zgEZpmsPyn@a+Tuf;^+MM1DilJ^movv-2+!V_1f8`&rAD}L8s)H^i1)_+Q$dB2t@kq zRM>O_U^-pT-f^Av1W9)HYGl;%ysUeWc;VDVW&WVWbWF)UAk~m*!!q1;$o-4JAPzx`;^9ih;MV*lK>fBQ#l(fxTzUyB7dTB}z|hnJyr7H=(?P8o zzd`le0`#kQdF8A)7>HneIyDgXV0%rTF(R6Iznzx!>`g8t>D1)SIF@f-+i!ZQpCYnI zJVxnDA@fWzFva}&hVY&r9Fe!TSBM$$T72*~n9ElQ&-3ofE|Ni#9N<-`eJ`F@D^d@t zM_!w9LX+LU7lo{;^bHDqHrjq!onI5Mc`glSEn@183Y5NnV5^11Dntkd3?1*E41tKAt-VsVIi zGX()FxZS9XMXlukn89+V`p*gdN%f&kWaKe^y-M+(b9r-`M_c%+;tLaKfF_(oJkktV zhVfqr`ne(9gJhGb5wq5_-$V*B>z_-9Z%{jxT)?ADrb??Xoh!YcYhBwAX|QciU3S6b z*#)@ZYwQM{r30N2^9bs^g;2r;wqxUtTG2D!pKf+3X+-C!xVa7u=Z)ad386g3b;q)jOt?x+v;M!G3GyLVp>!pe?E}LJnJ42G zM#n+ETAo}{EswE-Gp?ZvKcOsKP50jgG_GnD@cF z=n_r^A`>Qo#!@IspUUtmeZL3LER&8f>a7$9npFblKyfP<18Oc)bj>myA55Ta_Tmm3knz*rw94 z%$LuarLuZ&Hu#5Bl#9G=806}YhzJ;W(kAthulkFK>Tm^%(wZ`{bY|{Sq__d(;nd+zQ2Wj#+E8!tK}D z_#iUEUNOr_sAcGcUV-mzjE_F#q?|=yr74ctk941A z;=z&TiCI&>jW(&X3Cb)d$Y4!)sBpaF;43ii*+?d4eza-1UDLYQswYB+mG->Cv*UZw zFHaBB?ua=xhYq3A7JpyHP9AgqVLT@#$IqaTx`=f5kd)ckq4re)K2GmzE?aeGE%9~; z0(K(^V6Fo|S;D(f?4qzwzaa<~$o;?nwvmwNgF}!RE3B`IJ0>;~QC>06+8TBcERQF9 zZ!Db%F}piMBaPWk?S|y|*~+9{VCXH}GFow#*Vfu)H$9h{$K+h2))LG_wx>^Vh#t#Y z2?}q5$5g}Fge)$#ur83BiZ~Q4!ef?CICuJIp1x`aClg&sGa>8o3zQ4RL3AC;HldDk zy}sYURy+t2&C9jp8D3=W?Cdz*tN><&k>JfcKk`0QH(4uV_U7)qYR`4}cpiRvlfS$K zv3LyT?;lf9T55BuGGM(TZvRyJkF8{4*=uPD30Ap;JEu%sB>S}wQ4iUM1Haki5{NES z!Vcj^cNIM*93PGK7AAhxGE!hoKaY=0f1?h1QuABK(ZDozPzRwN4B0mHC&u+L`c{Y* z2I>sm8P(_P3j@Pl1xR|M;NO|b^|OQlAusJE+7a9RcM_z6^k;0GzMTiGAZh}Snt^Kg ztqp32fq}s(`L5!aqTto~PDN7+o0wN1KG((L*QKUsD}o_A-%m83d2rcN=C%m3@r$oX zXXE(*3wS<)lu=*j*%?iE^K$e)R#YYR@6XpFWl3EYnY~Cw%c!8xG-d(!rHjyF#)p69 zWQ|<6wE5o^vk%7qUv4jmnmaf?i&hGfxjx}S(8{P-l`kj5;IlZ z=w{V_9L15AwC`~A=OVfjCt!2B7A{yiut>2_3h8Ca-jLPy79uk+vRwQR3s9|NgJGyL z)QvZZ;O!6%8IDfs6vjsvbl{?IFFsV`kIU9WB10qNk9`_%eI%}-WSjaJvHG`tdD2%P zLio+BuK?$3EqzokGlH*U_}C1%yuJ@YvMNC99>9CM9@|lks68JiP-~KP%92kHSX6}c zxt0y~Qn%B<1usj>k=ex3b1H8+LcEy{yaTeqx&nfS?#L;21LKhYYy!v^zo_9Xaia!rCyshFRA$?(##z;TL$px5En#lEXCL zgNKqZ(066f`$*IZ&PF-Qi~tOn2zEf{ z^$byRS5J2vSQ?<4@p&Dhw^CfVZ1?<_|L*kGh%_PQiR{vc%SU+uoCY_Bg`?Q1WWrmc zm?Y>%VUgy)3O~~PZ?2)YY+OptCDShH2(s6iNQ2bEo zB`xS7W_V!zDcqO)*0z)wjML~qjI3VlkJ@OEDkERj?mGBXph<)G&nrxt7+5-4^-UX@ z_`!b+0%wR*v-hj3Z-~KmBejYVhmh!GO{c`oKW)a|R27X|pvyrXpIV}6Ftv`4DM+FdJSEla_FnCWPOafn8 zFNGxBzj=au_0@A#GfgCT_^~J*dvqFivM^oPf4>yq{Hs>p&kio# zdlp{K^xzYm(c9?s&tn?XE4I@QMG}WzRKupCIV5_7Z+j1wZJb~+yU{Z6PIkxMNVCqt z;MAXfheONjuMy=X&jUo}ZK!*zVYU+8h0Yn#Kzw)K!(_hi+Odd5wpgVG7eVlfaDs|p z_{#anqT|E$_EDHL8R&W3H4Q-U!nj4qpv#^y^pZ>8u~v)WlQu2CaLTnb-!IE6V=0T7 z>?@EPvNU@nR?e-!{{bM5vLUh7TOS5nu?LfctG4LMrDS7DfBW%EIrJL!=YlJ==7Cs+ z#-~wt>`X0e2#&UE0U-X3JoOR4=m3I6u@jkj@M zdc9)Psm6zH%W1n7ZK1!R!RX*?-&oWVCG;=no}TMwkx0?_WpjiE>3^CP!2rP1f=&nA zc}kv^k%Gk`WxB@lFk2KkefA}A{+SM#m9i{=`03YjszGvVgU-O+AfuylJ#mn@5EjKo zJLPUdD*LuLI?)I+Req&%fU;#j>D$~b4oU~QHjc@q)mx>U6$l)JirYrno9BDNru(0Y37_DvN2l<<7vS%gGjY++BgJJa(`kc!XrYTJkE8 zHxO6}8dCuISqzxS8wtM|AN5@O@~ZS%@8g^WqR8@#uZ2oC3r$raaOfk^Z7=BK7VR;p zp3zqVR96p-d|ZPdmn#KQ4~&oB<_@F0E|m7-i@wGq3=)Fw z{&<5w&wU8~d_MH&!_gxXHjdwa9LJsiPaP<eox^%kLstdJRP45 zP;{Ek&0r;g5JS*G4Pb;2^)CExztse-F~o?h1 zPhcn`Or-`GtO4+RBzX3_7}X@dB__Xe=CBX{3|PsjM~3I&PUv=%cRdYsteB&5gNuI# z7S;d$&LGnr7<&W~g;b7Q?_LG!!<4j8oRnULou8cldhPa@0V+r5@^BeqH31w6LovQk z!j|~-e4bZx-~OVgrYrz)zI;^<+y*H-L1(nq-8ZwH6v-C@RG!APsi*b1!;4()z^s8YgU2t@oK+TqW$0-VX+0{7 zDS{U*p76gn3eL1)U`X-s+a%=nHJmyKiM8QH@x34-Iht7cP%~qhnL3~gRC>+B=#Hy* zC)KatxWVvTTkY~M_OXAi;#q(>Qw2g8a_=lJdC%V!30;L{#Mex7tSlgVo?P?rxM%hP zH)IYX?Nt{4qThf|+X8iXPG_f1)qAz&{9BmHGG}PbxIqrn0L1O!&9E(Az}oSCyA-mN ztn?CS_Zr=xQwE|%AGb`*?V-t%`{5FbFVLaPnraTu-2*Z)k z&-^C8&H^8p579HPE~O0OoQUlakbicTGsJ_>Bgw#UNbfp5NbN%{VZ0XRwt)EN%+rFQ zmU!ILZyf4Q-H5r1NPI1ZP@Ac&FEIV|irV|vApu)p?bMeef3gp$A^sjSkR9|zQ_h>oMT(qUH(QuxH- zZva02bfK@80j5vdjae#jas35E{kdafmO&`@D}(j)N-9uSRZMfN#RUHyz}Yx3!1S zK<+m;YhbwM8q#aRZLG5Hro0Za=Fp-2CcI&&RY+$f!0rC!H)J!mt7&vvxXq=XpP@Ca zlKAeEonR^>zf1c)aVwmf+$Y?V<$)aMgoK2}qj~0iu&n~mX(0y(2SGI1^c1QDS47va z&}3?dA_LZ^5;t&k`1?)2aP)nI+_>qfFFl#7udlk40mMwM?y3h+g=X;`^P<*11c2X3 zP4hg(vm#jVqjG8kz3B(c$dhY=H`V)C@FIF(CY-MMK}9*r7(w=xQkO`IqATuM1BVIP zOfhLKur&WDMJ!^$l6vb%VEKSO7)Ad2RV_~JoYixc8)@3=eTYX0O&1VskB=%+Qn~DYJHdm6$>;&xL{3Ur_pRw!2CJW9A9+2gS4w+A=WVHWD!chso`|v0I0MXGr&n z%lo=dIUgaqfn2c-%L~q>afr4p`{I{|4~_@TfG$!@F%Wa1A74;>8HCi9SO^a+y|8?Q zKw!IDUbzz#7kyivQTtVBnwdo*Da_{-EEnroW z+ba+{n18I!M-adj3N~<#6IWU_S6Z;3l zcMh!*)WzvfEHF5roRc2#GhpwG5NClp4l^22-XlcjRr0zQPeonQ)DQWCArz^r6i2#+ zHpHc+m3S3B;&8vvZMX+jl;Pp?6}psC@L7&hUyl^y2ZhcuwsBbbpgr zdJ^?6PKwvYgk{f{DF24at|Fkr*Zy+k3DlNeUH#|NYgq7&Y9&PyusQAGM~_d+O?WQ! z8XnfbHPwGx80HE?%h}fQFlQq#LY+<%A)j*-)|q-c6!=MG7#kGcddq%M@5Ls;z}s_9 z9&xh#^L3)L3rXb^#}`w2)E=(5uR$&GqEFsc-;V5Tc&*f&FnQHPj>VS|?V6N!xm?p! z%f{Fdrd1+kdtk%DZ+#Mcg4Ky;)s?Zq)C7fP7wUQ7HJ%yIc1rgHyVly?&JQ=pG`4U4 zc|vD-k4fz@U5+A2%}c>D9YOB%-SL(;DJoW0Ittq9;_p*hduF@)U=@CSVSU-T9ocj= z1^S3V?O*1Bot^)b@W$|ks3K+86phUqW+U}l@5?CO$SDfxQ}|e0#if3fHPND-6DF;& z7jtinp>J%+PT~|ho?BT6k!&1ISWgx- z9ePBlRO#&xCFaQCla!1|$M6#RU-lIkM7J~{j*h}?}_O6Xx$xQcpi+(92B)ysD)X$5$p&V}gx-`e2s z&DO8X4i^w9!JS*;jlcs3RLt`Tt;~ma_e-~;ak^fwMAQ7iqF?KS@@Bzr$24hcwcfaX zy*nUV$JhEXq-LtK(p~;`pZGlYZQ#mg;Fz&!e8B|3Si9}lAV>9Ho#gQezC135DhmL`K@A(xos3NeL(tNSZw zcKM2~zbK!0#H{UZ3JkhBI6?ZS8q`H92&uYQ3dFV0;b?Mz*Q-gJKy_H_m%qTo{(-Hw zye20Cw6O2WCI_j4}Q6$}&O zN1$gKJXMrD_+7t0?6$6%+d=xS(o@D!Vbywxzlw=`(KFc^J#72R z@jk_rT;v6cxRxE=9&?_?Xm_|~M6a4>9*6|seGyD77DAS;Hx}zw0gt;)25?r!Dq6aS zrK26+!fkzTS3a+)45d~nm)7!x&;PJOZaKOW(4UkS9p4y;giY4FEZws*V-BL{#!G&# z)rr!|JRQKY-w{QdC@uHE*?p$YQuniykTs3w%t!4nbUl$?VL-Nm;dVL^zG3-1GT;S+ zu>1!Ki-k^Hlrim}P;0uh4;Gq^bXW=ZhLG}ULA7S-HnL924|;2E4?HruN z098JFlg^=X4U%}bp#D9*EgoHJRIIq%$lr=i2m5YIfwGl;gVzFKeEBm%cq>^;Oi`aD ziX*f;j=9vWGL2K5UN!ibDH^w66{7g?EYKf3) zgN`#K*>*Udpo(XzR}oP(MP-j5{T#b7nSJD9?I_`vn~7o1cUtDvpShBvygB{oZMC6o zlkViYf;D{SDowf(w?W#5uECU(h;AXt^UDKr+t3h6=55c-7FZz6BmxmG7K z;`ox)a{RVe2u`TJl}T~N<`{2PTSe|6VhZA+aiNYCo&5`&F zVMWwYD^(dN-QvfqPEXd=Iy95!)L@arzd0pZTYFD;rh+bNn4~hAi2$NE(7}`y3{)Dm zm^4VGH{6w1cr%NybX`d_;w?pZvj9TT)|*A7DH45F3Sgn^X9kpv0`q|oz|sv)Df&n$ z(oe_%=KvEImNQObZl$p_%@AL8PBqWCiz?CX^ntn?AJ17|&p{{)adhEvIy*T^uYp=pU`ymZzpgM-% z^A?$B@q;TfainGB6MK?~UlyVhMt%PA=^3XjYP_{NT8^^VnhBP2o+i&8wwdIS{_zYi zaA0099vZ$e@+NvqPK9{0x)n4@wh3$bg0EaK+<5DAvTlz0_u}7yWDE4p<=gj9&rE=K zBpaitFK{||+g;Xs1fdCw0Aa=aVzv6s;cYp@ERJuKEOcQV`N_5TC{6Hk%s#bcDhHpG z0jN~rP^H>7i>gh#@+Su68ND*4C2F={VoHhlagt`Vx1p)DRHr<>`j@Uv|GRg$<&PU} z+h69Cv+Dx^EGK(KoNo$oN2W^N0#;PMQbydsLipxz*BMUiw!ikR;hj25rz;ccOr}co z!q62Be0cNw^*XoTl9@51iT8fJWo_qX{yOtlYNNm1dzFt+_kO2g3a?osU;(D9_0X2T zI{(aVm;#0tA`5THymw?jiW5qSqv$?^#bsA3X`_$8WK3AxDD5@Kd$m8Y(&s}oN&Hc` z@Zf|lX_EgIO>kwqn~YHS>U-p<6pDPcv-fB4_SQBA36hnih3Qad)k`t?1DHGJ;3ms|UI#)!byagK(Ag=jSSy3e? zG^FzQot^%CU|e0S!nW2iLOF{3UtQX0h+ewdWTM)04@uG@_4gkMu~t=0=8t`|oSd}P zOJJ9YsZ`WWy<6z@d7DkUIZ=<6F7?E|kqJ1vUsLt}F8tKVeW4)YwQO!MOH3Gbug>RA zu5(9%r*Hp&8C=h(?EfesOJnFFBO;<8qr}93rgQqdw#9he6SaA3nBbU%&QSo+$jGEM}AH z|ExEzfxo{Qn(j}64ub}NMMJ|lFmKS< zAa7sOO4Bq}SP{1&O9KrVO$BNP^Ob!{lcM4gw!so_>R!fp#9olK#CfDZvPE5a9o_~X z$6K1g%fH{ulAEqP2@}A=zyJ6x^YNn;jyT<-lvC&2D1v8h+amQs{nPVqM*7*G7|D6#xY!QE;V;%}l~p?gA2@YYRM$Oc8BX{X7*v&iuByPpss< zsu4EMj9~6CXP3yohWdpgoxW|TwJ})pF6pJ-|rN$^ei0F zkG@BwofgebI{v2<9KPc(%KmbKIuNbleI99VwIuq=8IGJ<6Fq6(Jbh}`U|Z)DLxwv` z7&wbWu0>1)Zm2Zae{h>{i#n)&dveXhl#f8n+(7R^)9Guc^sP>-zL_sdDw4D1u;b+t zol=%3OJ;@vU7qEKw+SrEF$zNBw~h}*#asr}3eIZv!^ne8`Q=6BYCEJteyK_6pgP26 z@whYNYHO39_rFv&C^mk$!A6#It+$5nX_Hhqo;c-~fzFYa?%=oK#K(^@pBg>)$%HrH zjJ)mnDUm7y!~gl~ib{UR$h-e%#El{8-tEV&-FGDA5i>x#$@XydZe5 zLfq6eI}pQtD=`pWbcvOqk#l>Bjsp)Fz~vBq4@?+1)ZZbHTLhk~ED& zR!LcX>_u16v>n60rPa}!YR+2~XU~wP1?&#g1CRu&$kgqa07b>mz@6A&?TcE`Q@xE1 zI)b`-67W}=+4kEdiiFT@0N_1H9@kvCOzqIdLO> z0|edS`aGl95DF52h|@)@W?K& zm6G5#HOmh4W>ia`YZm|DRH~ac8B2KnxMU8Rss|+bCH@VVPMNWJEvb(>Kr9M(NV~hs zD70W2vhHP96ijc8F}dx_NB7Zks?gbb@uVWtd|-_f!Xh}5r!3p0P``x-M4@xq_PXHH zz`X{;)l+cJX_aWs9}2(9l2xvxdD0ZEp5A0P@@qg~b3>jEW<1~x{0|FY=4*p>9k9;4 zxvuz$!dk(-WA`q53Wz{9EH@l1I`*_k%kA0!%T9-qU`%S>&j&nC9g%B<6dfh4EYLV#Z5@sGIx_`O`p+L8@;e>8>{LW^l#EJ6>GGK*s2Gl3Re)Vwx3zyWdKa52Sgqwqn)ax+_ z?59|Lo=l|NHnF>Dx9hZ=tX4)8r@|AKYMZpLqJY2tb-Wz7GPe5t;L~aE`h8hnzn?&T z@uKLA>DZIwJg2oMbQM!UtU33gI`BH*9P$Evb#^o6@5k|FDn%ANqUvTvVo@{Dk;byt-+D5 zZ^MSa{Tq!2)Rza2T^V1QNqN-CSH;%5ALa!kpfefC+P#N#hV2u8!P4|7={M zqjA~xmG490TY*^y$61sJO0QY)suP$S-4+y7)NKJavMC=wB>b+aF_}L&^O}nl&H(C) z36SE0z`n`eAJgt8eBL@v+gB;ZGH)*N9e}9h)H^ix;fGABuY?{mdHFVdv!8?95t`PE zQn1wUjQy~M7S3yEjt*gTtz6GWB3$pJVy~UC9I%V2k?9toqHUt;Xy7_c&n>5yp{Ssa za|vI32T-E7F3C8Sm*%+jzCTmtjCRhzJf68dN6hh|>AVpyO$CYX`%`Yi;<~!_51-cE z$>Ys0Wv81@pHQa(muVX!SH;Gd@k+lmN?{JT@vx}rdT^x6kp%#+5-Mnz%k&7_bOXG@+A2 z-;wGjL2qpM-Tf?Y?yQ_-RMljsF-SDG?-W`+BfiAwonrHZ&eY3-DV+96El+}$4f4?w z2?cicw{l-*Ugw$0ZK^?Sg^>N89PRf>h;KBPJi~l)Ld#~IuMI*d*~*^sSfJKU$62AK z;FWeEX!@oY`?^Z8`PhEVd&&wB4KsTad8z)__+!pw)uAgF?R@mlfZJ*I&+T-^Z8G;$ z716ar5(?F$r8xtVRgA-L(7K7yA|ehyUUZO3P9y0Va>i;nfjC9bWxX{3{FLt#-F_I4 zZk>dw#DPkdueB=JxCN62>sW5#3_2EZjnZG2%R%_YEf3lU^c`-q02r)Zd;bbgCe58& z$39`gZIoU6bD#J_+Vm*~8VJ(7a3xJ6$A$q`-!1{l(bt%VQ!yM7Y=|&m{)y>9G?g+p z<(#;2EB_?u`6@NC_gHdQDo){%bc`a5Y(K_Um;g|N&AkiMFYZMi{7uK9l6+--IuauY zAw0$F3nNs}nr;A*bmy9OaFbfl(-KxSa)DxixKEGS#`yy6QD%dRSq!oFIeSkqD{VbA zw(;9E3I?RH3qb!ERb12~hj9^y;(qFtIjB?DC*-EIqRe~!^S`8kSVWI6+8ULZG&bm; z9+H>bLz+maV|z07E7pIE5N42uo9TxV_30!37E~;M(C4bp2qO`SU9>-4m?h@GP%jSh02LS=A&T0Gh`=f3L3EM;BfJx1B{F)c=B(=AvV74u0MoOCIV05lH2- z)V9vK7r{5rvJZ|p<6lC5PrHtm>W`T&0BR;H<#DxY`4|Fs@MmY9Cj68)(6VHdG@b+7 z1hpAFQ2WJhK1}EV_r@Q6r9P8gFiL#j^|T2!LS6s^=X*Z&$?{Rd*C_{0YqiQYXq_v4 z5Yrb6;Ik=nk;hM^RbxM;+#x|@8tz#zO%df5v`e-5tjF57=>^pb^b^@CH3R!=g+KP5 zm(UJe1HW*2`{&0*$i3|kpBOACUMuuKprW}s#twMDoONBDsG(aR<}qQZPmw2e`0{8! zl2(3ZfvFn*pQ5U5#s4EWHs(!PS%F7nNdln8}AOv zc3F#LT~ne`GL|s=n!O8u3gAT>8emcG8r_Sp- z)vm(gn(@ID`Y0Nh{)G$a=Kpd@_)y$m#zEyyA_PYGp@CRlP}#fq+f?JamviWkn#S^X z8o)otRO8P!XkG_4Ziq}FqS+}IRUoGzFpo(@Uh`Wwr6Bl~4T7oqve0rAQFD_Zl+ob^ z({n9kj{rXT@sfAxs2R?DK|sXh9z16kyj>T=ZgWh^QI({L#*nN4?20;mF;Bx}f(W;GKK4$OJy@t!4Wk_fNvG zLg2y!crT|R4&#`Ggl|B$Der3gYFUA}1$8yBw?L5&r}sXhU5s&eFg&9-hh3;*^WIB*pD9(=h8-uu1e zM*`9c;ALPMtQm_q3Df?I%|z5(F5dm6k@108R#J)j1vwQ}pPuwL(uDs4-efEcSk(ha z@9V}^rh1y}KGPQnnX-@BP|+P9R3(w#{{BRh8)B`f47kZ%fYsk0P+!c`ness5pX}WP z`LHYdwkpMQh(HS}t&&qbc)3G>s@{K-zN$=zCF8;Q9y17-+%X6R@k2l9Wl#sV=6|Im z6T$=jo`vfxDtfBzU#pG@WM=1ULQezw>X^uJb_|9Y$Ea|9 z`iudDIw9Sr!yJzbji5>svIHmp~1WB zYGcwMSNzMgj61o5DGZe?C0mX%qDkLI_k9AXYru_3tS(BVHm-t$;hX4~g{g%1r?N?* zh#WXy1ga$3mb%_LPNO&4;^xAR>o|A^UU|qkB0u+`eWD+ILV92?5 ze3;ICAX~#Y#ba3KR!bg z<&(yLC`z;E`qsrzlAwkiVzvBRS)?}A*;BGNYAvl7Yn%GcZS|CgUsAG;xfW@xgX~)c z8-}RxA@DOpZqW+Yz$J*LNGBK4WRWqI2>TgQ)Gvx3f%Wmwnnhz8*k}0O+4(oYB*Lt^ z&G4>6cw4FiK#IUr5!}OPOhz-9w7GSq@*Lbum6G%6naPcOdXgyK~5JP4q7RUorV+zft#|L>UBAM8gvddA9xE}+3b5p4 z>Wa3o*ytMn8_v!CH_niQLxsqab9MVYn^(5976w)a=&WvUT4@?P@LDhTye@Us^6j)6k0fw3ZR#6ZWJq>u!~s@oFBB2kC_hZ=#)EapVn&hQ4+C8!Lyfw>-?ko_HPvbefJLG?fv`|@ z>N-H)DQt(;qdPAyT+0uzs2KI0M4$rR=PC{BF1oN)R_@olCY0Ei|Zl>D7F-}uFj-BrjaaAFPET3&_wlJ}&pO8U|2 zqf#M$c`)TaC9wLjDHD@H>ms$9$L8Jw2M@kIq~QQBSj6kosxpxC|2k(zFrPl^JXqvY zwMoZs+&H^2l=9@Pi;HXj5Dib*mn*;%o=RT>h=riMAFE!ARHXMm)g^z!J-8Py59BQy z233I(zyRNye8*?|m@iZ+z48&kZV+Ga>UkdO)0ic_285((u*k~w? z-i9`q1E~+ky`gYZULA;n&qRPj<6o@grT>4h68}F-QAMPjgU{%9$7Kpw_?K&eGyb!@ zN4;Ra2U)?+XMYnwL-&N>mCbmO83E`4o) zAh;Ou>9YR^_UOEz0eSuJF^P45nB&TOED%9VBQ$#!SxQdXM2 z3#H!Fv4356IQV{XsZX@X^tz3L$ieS7M^d2-xwH!%#|pv>8y0G_GDkint@Xayn_ln! z{^Gxw$B&*FkoQMqah17uWcatG1g(VsCYp;dh0P&x;5ogS8r-}tIGB=UA6)gn;T@Np zX$A4#=Rx4dEXek}7W@E&VS*JThA6+m1(h#0IUu{%v~oY6*nfdVr|E zd_ri&KUAYpF@Nw%$&%8W-DIFLNcO9N4E`9#KPZgaf%sB|;qqOMI^Y4g1)3tGkbo)I zx30)Djhguoz&DG^3O`lnqFFI}=^e4HV=h^D)ab&jtTL;fK7HUUpTh-;wO7Dl?<*dP zs8<1&(}Uo|c=c2p+3DKQM_@>~1#%?E{nb54;0)dO@_mp}`oQVwNW&y9G&la~@oI$J zs)9%eu=+Oz;VGuTxp5FwXPkXRsPh>)Ya;y`2f|thfra59ux(xYvq$YyStj#@XVq|- z2>5`&tOF-zeC;b>3qh^MlPOD+o5c*sS7hW-qQib?5}%4CA`C2I zQ&TUle>_8wUPRWLp=cw@9sECDr~n-d8W`i%qfc)RPY7>3ZCPW18UG3few}iCwGm)! zU*DS}oY&d3k-hzUzBP76~d*$2+WK+b})vHZ8QL68_=7lE&gxeKiQ zbfN#at$`I82+Cc8YSMU$^%cavuss0E*r`C=2s;9&qIg0tFcL|3ENe!CzG< zzxGA2hwE%<*DGh{ePCsMtPpg|i>jVHkq4OznSIC&Q0?{MQbXR!Q*<=4%1JA)D8{4H zGU0FjmzwO139GJtqcx}*+93j5rcsAB;~HE?kAxoQGB0+%S$IEYL|fG)AdFM@FSfV> z+M#)gr<#opnJgLNf!FTC{Ur6TAeidQS%_RLApieFE-rMvGf3o+!Z~$K04b8R3R~`N z#HRV3KUTXzV2mZVQ3o&Ti+&sBobS`az4?4vb0#q31lcqWia5RV=$1E4CX)2^QlK># zEvr`YHVXV`O%020=O9~y|B^*%L2Kp3^Q)*%Jpj5_Gw{yG3*^x&YyaZXG1f{$4-jr4 z;shoSL6iOx!5wr@7;LWKn>wo$`zNw2MfD^mz^St{ z5FFmVfXX!?|3Rm9gZ5C@fhy1yVTxo&o$hxU0OPvPG;(A|KM+sV2d{D)E(~*M78s@i zE3QU(c^*XQ{{o9vk;dxIfUQA0RmboqqO4G$M&t7h!WIZpl7NST4Cxl6QHzk7;{Q3r zxCM0HBlm%RGXPxn?}Qs}a$C=fz*ObzRmn^aU{{C^0-0T3RXu2mifsAOnM1gq9_-$# z&=amI7JQ556B&SrltNCB10a&&pv_OCkQQuUAr!_+=fI1%3Kgp%6F+jECjsyh>&u>S zP7O>zC$GlIQ`frJ?d}19ypk=*lIu)Q-uwl*9{0&n;ubB#A(YF^!f$k4{jVG(N7(|I z&@zy~E)nf$m3Ey9a@^b8^nHG{>#mWj;Jk@z4i5!F%uAJRM2F;wKH0mJjr*oTlNl4p zjVIJi1%i;w%3a{eeiQ0--4i_Zf5IfbC(6r07F!dkmic?Y#-<-+HoZo*{ZRb39K~1t zryNE7BWb<6YGf*QmEq@T$9Y+Yy40d@&cp$u}TAN7xu9Ml) zQ8p2}ln#j2kyTDhEXM6LJ5SB=HCCjV& z-vb1b`XAVrpXgBoR2^>wS)WuFQI_3*n~omS@%s-S9=wYgo4Lvqo}CmCB9! zX(qzg8Az!Ayv^Q%_O#v0)c{cjMdJ>A^a}N(WG%T_#3E#Fm%{+hl{~%3Ss*IR>)Wpp zQJwipBr2IIB9mAq-OT`-zhY%2)Yb*a;L}k2unK6QyftGesJs|yE^5%Do7J0XrfR&} z4dUZOfB6dHNfF8hjxKkzWX{?UIy3bv%Y^odLvWFi_V+jBx!Z5?Br3j0UojZHYS|vs zsPHLSF~dhLz5O(dU;ML8Zs~~M-HkGdueb1zM>Cwt4B-FVeJ61DA#`Z#w~n+0GXVw5 z?MO}UsQlY7{*v(X;X~ISo{cV~hHmVAMOYtR2>JPj6^^7>&F+)sU~oFqdjT0u{yS=MX>0_5aNe(nbQCWP>3S4GLDcu#Uo| ztNQaG;U>*14F{5%ak~W_ST_5)Vl9i+U%+J9VE?}eLN5r0L_Yh!7<=omsP;DMpBM?L zA*4G+Is^&n6cj}zM0x~Nx=WCukw)njK@>?9RBA*@kx~Jr8x#aYWF+5vd(L_4dVlfX zxjYIpd-iZ!sk0q-n*A2R_91MBM3&O z9(f`jdn3l|p$QH}?~!?h+aEPtU4C)U*p|qg!zMjodBb_<84`+!QG6a5sEl=sRUJ(} zS)a%Dn}w*)pL?|{nEUX#i#((yb6uwj>1kkEm&b6?x#bDxGHG6<739^snfu;gByIt^SW6KWgJ!%q-caepYqC2gw>Uwr(snhxwIrK8Hwi0 zDiY!Tl?jd>m>G1B+{Z=lX{K(NN1d_9kXo4%q@SQ{>D+#5}o7Ljqt8}+1-p1c~DVoEI)tJtbE?`oysqK zV;lK|AM>&PTj_J;vFw6Gy)oMB-6fm(f{MN?$Mt!C`B5~7Hc&SkxJ%wIPrtP``=%qP z;2yC%@8@tCb`mr?=$HMour+4vnpn@V7=LO{SNp*}FT}C<;#R28 zoxgPX*o&xtECfI6O(a^W!q9Ykke+kqN0_sWNEAvd2sS3QH)6b)V(A14ye~)I*0l22 z;N|YCxRKgZjXNVia>P4ri*fvAr{~S1ffK9 zDY?)IPK+e`BK?5_tk~@ADM=a%2f`SgC5>9iu*d8%S^QSX=jotKq*GdOVhNVK=97JF z8est4%8246fZJzF0Vyuj_#k7^(Rt#jK6yytS~_*FDM|xm_pe6C>wb_A)T=wd9^l~F zM@rfuZ-Uf2kIl8UBP((>oOmi=NB-&)oj}~caYH& z1{UX^a}8NM_WF(x%+5md%)(5BPrlfMyw<4Yu0H|;mAYBTdE$yBp?am}Srge8M#^wF z$NwCe{%6;2McWmC{LVeGA&U4&wJtIhEv9;DeyNYA%~Q{ja6Q?9-3Q~=8J_2e;_v51 z;nL)F?cB;}K69*Fr)CpfMi|m?o!KLAUEMp>T0m(&rJGPiaAVxS{Z+nz>-ty-!FXCvsta{2Mo zjmH#kz{E2KiD$Vb{(})Yc#CILjnjd9V;=r?Ioya|b;oDQ%(q9XGe(YmG2?+jM%)B@ z0QJ5Cc+hxfBi#|(-kb!`lwMucl?L9R8g~uPoh19S;Fi4h<8~c0$9oo1dfM8bCLzRG zLL<~dGSV9jg_gj_O^vO$-HnqK`eul#R$g$6QLW{-$b~2}9KA65oQ9#+aXSQMIxc=e zwa*I~ZmkoNS@5B&`U^AoU%jRvOW+I6!ino}BL4_eCFKEmS{^ufB) z9O_4fRO0VQh=@viXiW)NUcP({Xw!IZDuwg*5HR>~v?#orp9shZg3)QcUMcURM$&M@ z$XDef@<>;V%1=VeV%LVq-h7KZ5;>JNkAP~teW{EgB}G(TbN#wm(moRw4k@8YvMY<;qj@qw#%v4&cqb~D@%y#jMF(g89aWg_C0hmCA4JCI9Uf?Cz?bf ziGusXjnh$Vye9T5(Y`5#Q)G6ABn1MN)i+`8Bb%`^O%r-N&-!|@MjJ!=g_>*jxz4ja zQ}@I7V9@jCS=hpbTqcdPLZRCeTOO26aa5~_(eLF<&NSVuv? z0?UBzAp3j-{W(cQ2dGvMBip6Fo z0fi}<2~~f}45K>XCul+61JuKQrcHr`p8j}chK2}Pv%?d+Nh&L2x#wpy(ZB2EUU~*$ zw>JCvl1WjjKezQH)M)jPR$DWD`u0@x16r~YHcz()d}!ks(FBeDWKBI6k|3B$XxtXV z!IWwEfh}nmWenq(vu3hO{d>SdU_K$>y-k>tk^e4}jGYmPOZ>Bf9VTKeQd=K*Jloug z?}~yAtn~VGV&1vzdq=unSU~u~em93x{-TRN_Rpw=pBNs;%@Wcll?13$z(UZ+1@67zIK2>yH!QE^azoh5jD4XY6Dt53FPX6auoCjVE#y*FRkMyWD%++ z))X=#nS9An_(}JoD>>36YyT1U$^On%x%KkJTJx@!h>-)A8|IhhYzQogb*1r`=YMTp zo$2^b;$~(VMDp1c}(4qhyFwU5&GOaWEB+Fvl4FWXKU-LGY?Sw`yoTd_Z3yd9zFf>f^g# zFB5ptB!9iNzrD*ZqW`QMRz8O{6xiNCX^?PmoIa<}@S*?K8W>I)*#|n)KI_~dj@7Nz z5X?uEdI?Iqvk4IS85Hmt8GCu}u!kQbxO$y?n_=0Gse7X76$Y9DHRTuu zX)*!4f+ydWv>o5yeamOA`UI^u(C2kp{*OLy4>mg%U(5Z=>hSl7j?Ul7E$7EQ?!}6lqQN{`#cidibWV&6%(<`eA7k zozJAYvR6Nn^>9>6B|gh;og=v};Lp83a31v<`;VlCvOZ{AP{wyMa*;T?_i`+1-H6q?A2~RjhZ8WW~vvJ zJf=1hO~chZ`o6g3=wmeI>*w?qU@d&@!nBHobNzT#)nWbcri#wXdKH}U$w>PEyNHKK^nPK{GYw0519BTNceVa|LCo54~_t<{%`#In&MeW4mz-DBKK&4RfM)s(x(L&GWBryjq!X}gtN85 zo4hlRib^&6*41$`L5fqE0~@)VG7WjQ@0^LLR5*q`Je6Seu=z`BkfP5BeyfoGWI^E9 za!Y5XoV7%&e3vf0saXR;^!Sx-f~!>Jo)ik;8N!ls?%!S~eo7@!n|ETY9Q#9n+*|(v$M>(f;%IOiPj(hv7C* z&?(Uh4uuuQr4?z9?xJc&U73|6;is#g9%rn)^FW3)16#9=nt5SVJ_7@j#lbq(A~wsgEXLhIvq`CjuaHZ8 zG6XTo)#!rEr5Y?k2eFyt;;>|;)eD;{aDPoc=wX3qC4T&=d0p#Mi}PT(^=>dvrpswF z`r2#YWMqC;V#B0d;HIA})~4w09scwwHbc@N3J@%J=T*~>cpNRvDX?UmA7S7xij26F z)~mL8`3=$-n2OZdp_gaiswvSQU{Gg_I1BGV#^y z2E@XWpU$IZ;C`|$NNUuEQGOM$c*4qF2Hx3Yv(O8k9C!?DvN*L6-sB7v7u~`4S8Hc3 z(43X-s)W=T<64fV??PZ(NU5#wUsw+-t=b=EK?9!{utuAFDyqOr>M7Ar<27cNO)_My zNMJIE^CeI1ykjyc%2Vr1zTwD{DvnJagZ-F+`o|PqPT|A8qqpBbm*M#g?5OgJio0cZ zrTSmo`xN$jbFq_MWK_Dy<_AHyEUvT;+TVB@VbHC~Wd1z*>W6DdUqfcxLU(g86B{go55MY0BNM3KEW<>#>uCS0#NaSt z&&GQc*h)PF>CR?(G>zAW5ms>^dA*d?v-6K)5+P&LwZGHAP z?Lt=+p(^ViWLm;;imG3a(uRVoC}LdzbyeZp-25Q@6i}K+0 z4A;L4*plO(-G{CCHPeuhO|U-pP^LK#q)FoFylOU$mYt5Pm7T^ zFXpO?*n3YaBJOZ}FsMy!i$4LKT=k2gw$K`uov;VjwR+>iT*RhKLEK+)}8| z)9R5zA*g;5u!Nu8y;osxR*(78-Fh(@MIH)@!MQ^hsqZlJmMF>XdqToxJqbL;^GEfC ztF`4cr-i@ZA3Zuz2e1nLQeY9mvGub?dWy0EI|%V)PIeW|Hbb!_wlPo8kk&0*f>j+Fo1=FGw`i3}L@W9uhLP z*x39QAwQDWA!*HqY)h?=f!j#|ulyp|Vcf5}B+s8`H;^m%9srY}v_;a_MpELg(f~qB zBN_%|g|(h=w(mhfykb_!oFeACN3uGdrU|<~H71u*<-ImMG|(0Yg5r(quk}Pc08aI* zD_{tmYVGB{mv5WdCxC>-87$qr7wxnOmXlo~UOT!rw{P?=>#Tp$7P7Xwu-tS`0KeRB zn#4(fDBXgd89B7)u3Zwq6Z&)dkg*_gjiaCq^V2s%9#z+o^q(s>pTbGV!$i@+wezxx zywzbHF1~-yG}AZyv~!o*OB`x)SGvQ7q&rTw1MMAkYbI8xPZU=0xPe&s_9WJkr`2 zznn3j{jB!tv$*|n9D?w(cFFj#jbBcwLv#Sls1^j$xb`#H4Y7gx%2m9bcrqx1R2q}; zTepEwW;KO4GQWNO6+_8ZpKyuj#fdDx8=X6V_wz`d&BT1DNzob7=hhS;PVzv#XW}{B z8~qi6pr-agib(Y|XSJ3MXEYz^W3A1N>|;!^Tpt}Tc0M30;ea9H^=LDrWEVuQ^W#So z-~9@3eEx`$x=`2=7nOACk@~NiSV4QRRBjtIWl_bS=o4Ksr^1JB{aX2K$1tGm z9>5oFWPL0L*SQ~+xB?0b)2kV${`^3~5EUP%>zaklAv6~)$2Afsn}X6e&HC{I8IcAd|l}3&#;B@6J~!eCgVrsX#8Ox#$O}ZG+Y3-3!8gDW!$}@GUj2_v^{MjzGBDkgDGC0WLP8B(E zMP~|;nbx0Qj)WVzwG#fP(M)VVv zKa65FBnmcCO__-_Pg6C#101T4evahUVnP-Vj=TKy_M~>3gkQo zc!CS(NdVT&W(D;Bl9sQ0PXg&&3ds$9Fs!zTpFG^R-rVTqGMFUzH@nW8O604@vSH)( z4}zPoCil5*e*8(^2Be!X<~b zysDgNQP@x@n~vffJ*6OyoCmMV?xD3FI!q*|A%Gu4-y$Sgsrfj4oG1&q0UX?i>M*on z=vytffP}a(WZ`x)At3Eq{%|2{JDevyR{jJBvit0AYZJhw}m0Xq*wpYgOTy z!P8U>IWc|>E%rr5L;9vC&t&pWR1^tls3E#2`L*iaG>rp1Ts0!H2-F_AHE!teY<5|# zy;$V@TnbMUjS!xpWFI03pxuA;xKbMOcj>U?f9beTKS0NYJd0&8Rfz66jgn1uXCP^A z(}`uzK>UT4AO{k%6EGD<%1g)5wF}OK<0YZ62U!4{i!89L?AP>S(<@lcjgc z9#;t~-DK@N^{&yJFrdI^TzCAvmyp!uEZFHelhO)UoObMGzj)~q{~=S-R1%x~xvtYJ zzjtB;WEEeJ=dnTW}r-%Ei0D z_%l_r$S+#?@MoFoy@Il{IfC?D2J2adX3D2JSR8YCR7dyi6QW~}&P|-Tc}ghLxC~9h zmDs5j?!qk5N4#j4T9$$w$A1+RmraglBO2SGQC9;>^Km4piU|3#w@&hM*gPVOfcu3A zAav6L|Ax>d3WQ;HVKQ)o=RQnwTDyeFaOoIH zn5sLSN*5#g{~_g)`A5oS^e9PlbH7~T6ohf{dI|}O#t(xp=rrUgcDXV%)lzwKejf(M z3DfWRSjNW;sH{NDg|5tOUFG1t=H_6P{C`y~U>WJtxoDpCSvts9MO7$p72&SJ*)HVv z*y=7lvaP9fP3q!oa6$3(f4P4cr3_fSW{rL!!%Q?!a<9}3#tY|@k7ssL6rVDIkUBlcci+EAbop{*d{mcT;v>Tg&cES*{J{`v}#dO>Z^WU_vL?t@K< zj(lH9J6*7eo)yH^y`av)EjpU$cGYSQxqu*CCACN>nb@NR;Zpt1+EsU#n4O;8-kRUs zxI5*weZP?aMyi`tkyb&-JWJL3yEmtG*`JA<<{6ETADw$y&AKl4-|+kTQD09H_;Vu= z{%dq_02$-LwKPXHyZtXVp6Or~Q(_H1D8wPjz}m&c`cVBMwFrGQGV}Tmu~#|yAFU&QjX$6>{Vz|Y?t{>rB(2Y6e!@eP9qv6OlHk({f!I1KlcuB$Kcz=8YhpFl| z;#U{IPZXpxTP)-lxhuCG@0hu!FGcr7-MPE##KtwW$Z?x1AmR$HF?;XuR$hRDbn1$* z9ROq|#T{vC%WCNMM3s*RzU|=!p{goG=%x6mO$}Vt%Y-JkO_Jw^&8O`#d1jQk0&!ONGU$cC~lr@|Z*BRY^fXLT3Z;?d<*uP>o^X z^E~z+P`U{Dbgk%>=cE3qJ_E5^{THfQkCOc_lrEqQ3EW6+7ZT|!=@7#o+`pZ9nf$7E z=Ge%wbbtJL)RSqNe}o22)c+D1v??rz7p$=aZ;>ai(yuV#It7THl)3QIX+ds}0v@0R z^Y1bx;ucV(cu{D>aESlT>HAHBNgOxzDFr_;TpamY&V8AuDlA_O{e=8=X{qP^u^pnK)^*N?OghdJ6@9Yp zmY6KF$_1K86SMmlFzr66=55oPCL7=X(E@}SvL0~(%Uf|<#gixZDiURmH_j-=a{IF~ zUB0Xx^$#J#aee+l*;_;eUl;x=|1xdYAFt-RAou_lZedjq_q@!nMEs$>)lyP+M~!(J zaeu>9cH3Mmoa4uf?i!1dY*dz)(^}Lbfiu9jH8`5=ZehbRa+9P$M=U;CP*Nz*I~6Zc zWl?`_Oh^SkMJ#;0ZJD6YRyn&Hlv<8icW;^9aKOIyl$&*wzAeB(O%2WL-D}ss5H!Bo#KN_B6&tr!;s%?^NOD3)LB7lSodkA{& zGvRbG^{Y3!?i2r)Kjv@|jB5N>y-KFWCMfw@vk`{WrOdo!$M+?SGTo#F!INP7@wi)c&8yZ8KK4 zpXL4`w}GKmN07}>v45@Z%9YTZkl*y3=Y2VPX6V3P;3WZ7;vuCa5>QL9_YoEUJ zZva2C9p@lB-(vD~F~8JC$!=0I3cz^h#SLy@W!C5oYlKsopDy9?|3hkXV#S`$Oq+sO z$1BpmoYr{`lQGUC+#f5t)ZW${4>y~gj;~a)YCPEaMsm|wIr9~e+9uwxhsTnX<5J(-{^%EZYGV|?C_ zRI_&X`I07kh64iRUhkzG=T@s!1CYt#%HWZ-MVV37y(h!6{SxBc1v$78@d+ za`}i|H~@0uv{lWH_URwhdu)c4|p2BOJNZb|=oo9zCwRWae%OkOB>B|Og=qRj-ZJ^N(E zq|caW3b8{_ZjzVd?0%?+pP&;xb$@^=E5DlS3W|! zmk5viq^q#wwZJ?ZBr(=v?^+W4c5=!3bwC#Ty8JC|?jI=Qv+?u~tL!ug!yKP6#`20Z zkI-CvrCnxrP_Q}pNzN)5w)wRht&b$Ld`+Shs83oMvL}D4R=o2j?SLOrVfx6{r|Lk4 z<*sQLhq?g|7rVj2l+l0qejUuf_shb}`N5=nmu2AbMU+Mo!5^sJ*%wa-E-$$~vAWHM z(9trF^-G|PDF^DqqV|x%WhN1N=rUz-l8XoQ7J-cgnYdX{|8A>L;FnERVCt+sE z`87|~UvRsYH<;TQr@Ov9j$O#VY2!Tb%ZPz%TxA3f`Nabe7c;MP>g$$~MIfV->6g}u zU-;2v1KVt@!L^fU3?+qVz``Ryc{)rFy)`Y%oyRN%eBlbw~B=SG*0N^f~@ zKljn=-L&yl>);9}Av+l=)4T8;^S)C@^;q_?3HLUt3(;ru7({4+lmTF)0&(`A4LD@W z4f`~eJVj}3?)S*_LZ&qkzru%>#u_cUt;FzjF|(N=!mZ9)Gij!=HxnVdF4sRsKhCHd z)>CvP@?`30i@>&;C$y$!#vZNZ=mDWtg?N9^n}n$krYJsQ5i#EIBn3?vW6hmH2Ycgd zzLLRubM{>Hb|wOWj^1AWlp;bTjC?(`34FGvklzb&vNc3H*>*BN0o`7jAW8yhvK&a7 zET59P95GABWlFJ^GW|XG%gfA$k{TBCgGvMm=#us#kD>Hb`SYb_WIIYQNS@&FJ9iQ; zvT?a6V-B8>6io1=LuBzL5^Fl^FXS+%Y@+fDj?nI3pxgE`l|HnhNh+By38FYmX!; zzaysnOpkxIn0`$`gU!f@-h{$^AgmE&xb3wR_83tC0Rd^kqDlf?uA?q41MU?Y{T4@l z81YDnD^}2rP=9qraYggg;td(TJa-YXj}hb0g-Ua9X&Zuz@ z;<<4{F8b?_6`Ns@uS>H$-33)x=$C_zX&xdiy5o6O9>|yFYD*X zxnN>P#LS9L9pGml;Nxtr-`gB5@NjDF%OW67zwyM9mFiM|$qc``kj2rrD;ZZBH{RSF zn~|Hix(6wn;uDvfN-Yr#*z+=lgdMR|S1Q5bVXCWy%aNHO>rD^p$*MTcd z@T~OY?!-2RNR5(1@lr0TUr6H%&l8i~7@;4@Jeri_B&L;Dh2Jo}k_94-Rt-lb>7J@9 zsobmg5g3S5DD`w4KVtvz$&+L9;eYK0XKgibr?`KI`@MbHh*_Vd{WIh*iTd`2F|jgo z=@iIawX!nL8|p;s3d&vOdU?lI@F;)2Dg)i;+pV*qhIMFz>h7hhfy)n4S;{0SqKOkn$84v;UBIvy1{8Vy`*gqV zA5$FdWD{8|<&HzPg;hzKK&zS0us&)f>kA+YH@Fo#b#!T>hR#qa$tf2DtREU3? z|JKmfV0q#4Wz5KEM^e(B1$hYbT4|yJ53;2PkPG5uIYfquBn45`vP3-Z42$Ub8SYep zSV~XCtxG5?!1+zr`sle(w9#G?&kuC43SanHQS0V0n=FTG5?$_@lvBrBG11~-(9z36 zvfLn+O+SH3Z~wBX<+XFCw=OMf=?SUtr9S!C9`G&GS+b)yPwCwV&u)d{8#WQ3%7{ZL6r<{$* z&2k{KWAl##3>}Px-J@oUMzsmZ2y;OcdjT1FpW^?M^20zrs4om|0q8FDH&$rN$ zp>d8U(*s2)Q4hm-&2cBzJ{YMs-6?1?B^zAErt4!162Gm-6}gs?z0v%ie&b#`$&9c& z-5#72<&PfGnX+LNoZfaXX!Zb;UdT)XFWm9KmaM%PQ=O8j3 zpGkGcq>I$%$BufFm#AT$|ISgXN=na%*^)DHkrk+BPf4FLCA3R8c}#=KcJW=}O&ea# z^8|)%47C=r!nQ81@>sQ#%6Cc;H$`? zCSa0PtozX`v|&dq6M@j%gT(d3om`=;6>6RxNnC<73I3>+Z5`2!Q`vSGWZLxQFP=29sktav zq~$oe&~$NxOT>}x?7&<}z^>@GNJn-;$`f(F#y&7OJLU<;Pfs6V?s#`Y#R1kZFJ9@T zldOhRv{t6wBTwi-8bX;<-^__()aWMNmc7+?VVtb(sb|R;R*`DOukNMp#fPuZvEdGr zO+ALWQR3X7P_+-j6Q+0exr4F{VcdL$EPLFOd3p99bhvpM^cCMcMZTIY{a?ev6V>QY zs(Ip0jN(7SJ~~{`Y=0~5+oJBNz6(ZkN-cI?H{vo`{<{?_Xw1fS_m}|Gvx^_yYj~#@ z8l&z;i-wq+oA(!Ra#+XE{*%6WAaqk3;+o_Z))mv;zZp1I+V^VbAGW=bbDH~A660H?*0GL4d}Rq=ZmtD@$0XD zn8xZ%3v;{{KKFoLK{0eEsQ(bP8kl^w-^{jkW3_R!m)=w^o=Wgs?r}l%(xOK|+aAHN zob1m(i-e#9xTY__ieS(cd@AF{dfkRW8rvc``EMCQgehCnZL3J$irQHSjpSX#?G=(G4L$-R-8JRy0Hy*p`1gP8nHqZG9||=h&zHVsKG8i=QQYh)n|rrBu{hjVxXN3Fz3N|+xYI`Sl{8nq$ml5Lvy=2 zJa;;kx_LzwTP3cjusNDF#C925K!`06OHivO=L@oxp)|WX!)wDP-@Z9Fn0K#OuvP(J z5*G08{3x#9Dv0Mv=En7n_-`d3S6cx7c$*oweUT4su)Zv4QEJE%ULcij>pwob$Y}&N zL{hQSa+i@B{a9)sM;{c;lXVyG6JJCPB6XJ}n3GHecDCqOXg-zO{{pubjohQcj6ulN`y^0gm;-dKR z-jk1wAH;cIP55{`5-V^t#_Tw3K#W$}x%bau4`HLjT=CXS*6F`~dKfcVh`RbfYqG~^ zKTJRf51}|!RYyddJ+!Dp$lEw+qw&fmMj=ucF*T$8_d+Bf#;4}U>?G7d^lVbHyK-%+ z5YZLBU3+7F|FCGLObMVLWuFIY1S8ZaQ7zhc?1?k-Pw%RdHznjF5{wB%pom3WT6;R{ zE-6oZw4A_IpX}3^;K5xu%JrRWr1CT{G5+ifk=<0eMSbN9n4iqcW7EgHPhG;UJ)wzl z=1QR3b1txLYJtxf)Q4V>*1$y+8L53XP{?j84*`~nNgk8Zps1dWZNbhQ5AX2a)bpK}r1{47Wl!1Yj5@^W2lYk7_ax#C~uw3Xs zCdl|%aY+$0t4#w_X#>$C+bf?hO7B2YZk27GzEmZ!ODKs1e*7RRTVs>tUI+9?cj2SO z9V8uT0Sa{a(B08V%%T}0(zDa$b|ep}lqZVSS?8Os^INm3`A{;FP8}snUNkPx8Uky4 z`dx`4P8JqIo*#z^hbt3LP}W?TW9r;@$W^FH;~0eLe8izzkhxy@0Hu23Qs8Y~cNCF= z!CyWo!cas?%e7f@-;kJyg&U7~hmb>FWSzNpA`2W`Bo%iT-{%PFP&Sm=mhT~enz?M$ zvn%|*NR}@A9f1x?D3nu&B|ZF?A7s3FXba0wlkv1#9jZG18{UExDub0>H{4wpxv{k zs;pZklz)xbJB&4Wl$zDpISiKQFTkTf9@yQq#v$*->i;kyKp4u($}WE8X%{6Znp}xq zVQ$nfMI|o3<~mp>MS{XU&%;{t@r%{xqb0GtcZe!7X*iXg;uytlboJX$sP!@E?Lnx) z8(-K#_Ikp^>Xs=Z*|ao1x38G6j`SvLQQtja4>70RL$!4MT8(lreN*&f{yk37$GZ}t zRw7XR?d3Pi)oHq|*RY4MmOrzQ7x)NL-dRQOx+I_N6lC!`iCu`oOgX;K78rW_7r6=g zA%VrsLtt-wdg?mf|5d4D!h z>74X0`_9CZ^Y1cf8c78yFw2mo=RiWqY*T8OMfYRBd{m=yb|u%9W=@$^(&H~ zeiCHMm0grbZbLqv>F!|*Q`FvVSnR0wq$Wy+bAVa3)~1wa)MRjhV3p=v&mzD7ax%J-O5WMs5u>D%;^r8ho>drW4-0l>34 zLH?C~rAc9c_5n5At>taTRdX_^tAco^`DM#m5Ic6i0krSF)PE3{K%GPF^wpe*e;0?E7Vg?@@NQ_p8$!%ea zdV$ErDABOA>TR-?ZL)Vmo?$tg#rMzfz+>eiS*g_-+I0j3D5JIx+#}T%ZpFbB#98Ru zS2bbj;QR0x*fMCl5VuVmB6PdAG;HLsbUshf0SfUl@8|G(AGm*QX9$Zx4pOvpU}^z! zkQO)?bUq?0p@fTSOBPb2;APnT+~vqp`1_n3YsX$gmZe?_;1Yo(@6HJ#DWskg%>2o% z%7-S#NgO{lrjXw1qKO=wd&tHc95e=p@7BgJL@O@tEDw=p7N)!6U?_N?Pv7|J!h z!?1ICidnGb9ra$D7>~HY%(yBI$$IHlghPpUzm5Bjx(rrvU$xPPs1@U7KN}?~-5@TB z=H8~k(INm5=~5;M-t@Ld!1V2bCnY5MWN&=zqOXT{LyWS%g+0Bo@{OAztnCh7Gn#>1 zn>z@;=b<0RI)JwnizL}4*F$=N!&Qiqs#Tc3dr{eF{W>p+{sSEIcEomQBP|UMd?>G2 zR6`oGlmluT6PUqLpxJtU}=|s9f*bVujB}PF4w~@OMDeT1#7Lt-q}J1+RnP0!{W$1~{XSOw|ym}qJ=MI%%?(%|3F=;YTnLrugs8{&u;f1z62-=edjg9;wD%_3RcH48D^I;j;nLkxV_X?<)~zTHOEc*OE5O{TkFJY2W&v zK8Cozu+6S3;CajX180)KlhO=cI&NJ_72CK{`>qEG)EtgDZUqC31!(>5t;l~~m+3v# zrVNOO3!QC*uyL##Q_Az3?PmBO)&QqI2zi=Ye_44Zp_S#@>wt8l^*b^NFb-8YbQP)3g4 z_>9UoqU>^R6V8uJtHd{PQYZ>>)&~PTS>yLW0>!t!7tf(&!f5Y(VWAHmIzp!=K)vKy zpv(p3QraRA;vEK$Ke<$H?9iL5;81wq@93Hr$dlgQq64|cBlI4aJ3Ep1M~(Hzm&#`z z7~L-ENPUKCxis5Rx>S3Os5c+nd|?ak(}>O(0mO)KoA$VhEmm03F;U^ti&_Qu`FjgC zd5U4`P~(QG{N%9IY1Kt>rBeKk`j|!-#gA+2yiFVFf0XQCJCI%|r9LQnVtO5{k}P$AOiIqWtPprxa0X=c{lU{|I~ta0)ulJhJrDG|xgcfv{yC1WBOMju32*fb@#zJ-Xd#_BSG~LUFJ| z`^vQ0_GSaUSJ$ifZVZri3(!Smcp(`3tYs*RS!(y<9l>n7017j?%rJLO3^Y!iQaWW? z(XMZh^uHOH2RVmh9su^n(-m&JYv;@i-(=$)2|Hd#w~vG9OSHlDb8S1Kh+Ftp)xya8 zeumf(mmdJB-WrKszt$$*c786~mjdK(E;rz|JD)x);FE=|n;Wn97lxHvw=|cV)f9`e zo3TruW2A$5ccroyBAZwqdXl4jGyV`pKpDf|K9=)YW*Mpc#^Bt*zfpU9ie)w{Vruk= zT~FHVG;v+}6qncbcq#s775UGgXeyaq$YNZVUFt_fiwr~pNeLM=~EO(!+?4g9brS8>_}w11+@}|`Kzs#meJ%Jdde(jL^EH& z<#7M!MzTunb3Wa~yR)E@<-_<6RhXA*#nSz#(?cBstqbjGq|;noEeS@;ykwQf3X7*I z-@QJ9RRuz49j2SE&aVBac|^Qg`PM_jbW7N+CR8yBPhO#3J9a z<1JR!g7+qQ$cAISRHyRK4CT1b0pmwnBm;VnO}Yda`-U zNpnj!%){=TFQU${hur9r?YESXV-#AURMY7m)O`3m$HP+sq~O^qaI>G}P-3JlLlsu?aHCXKM`3 z*J8Tcg53^70-BTgYV=C1!p4iwv+iB$iyp4bc!rw1eyCv{)@@ex zl^>T!>KX51`-i%Cc0w-NV4*>$RUS;TBR?(L`)PmIRV?w04}cvE7Se5sZkyy4(4vE5 z{Hh%|W=L%LEjYxBX@k3jl}O>}&0pFG;PU0=&P@gjUn;wvqb!7js($e|-j}_CrFri1 z`ITzIenokE{BGiV5VPXR`VBOD-OY-Id`m-gW_a+?b$F@0riTpW} zX)ixK&*M&>PjiF7oz$)dR~)X*_zJax=WYRZm_p4RouDw7p=BbVV4pl(<&n)YR7-@g zXUwgmsyY!gYCm5&tqrU}qaAm<1%3S2!EPyLu~&NO{JtnqiEpa-JSFFz;=>v(Fl8Ny z%u4n~&|Jw12Rb^0S^RBhIx=Q1lc`?umBsIYvs^_3ehnuXik478EiKlp;&n3r| zPJJZF_UBgtLUOX`v-c}$7jyzOsR-y8V~%Ipkg3>5tb4gsLQ?3>YO|QTxK0O|FTmu| zv`LBQw$_&u#p#CAsFP~C>txq~9mxEWdQ3`BHCiKTh93_V@4fibtifHX{8?WJ$WD5Z z30pU~7m502vUzW%%phUWW@%WlEuxQP?o=Jb*B#@=&W@BMHsaprJ z)ZzC z7Zb1cRFM@VtEg+IiKNhb6HPQXKgI6tS<8@gI~Lp$bQMJVd-tFnMCxyg7AJIE-8C{$I!|rD?bjS$-ph@@DxC{pvsZ^n9;oU$H@kIy!Ljqtu+y8Q(oeV zfn5ry0x4cL^_@1Nh!uGXf~VinX~4m6(1$^}HZdk!SuD%_>k%{lFoipu zdeYlTs{)}bM3nFMp$-!GuJs4+;uYOgBEyYuv@W?_$mfR58T<^1})jp)T_N;JHDJC25asVI-WA!{)GU06Tm%h$eKZmKGz+;&pzkV zxo1idSncbw`EBpFZe9gS)mM2-uK+3C%jg$-t07Y`+)Hrc+HfUrY4v*beWTB{9haU2 zHazFa2Y&<26@AXf32MCZf`k)Vz?MBpG@bpNFc`*YU^_ky5e4QDQ*r>=ct}yJ}>$JOv9sOTI3;7Enz=bKP25wYMvaV ziE4(JdnKggzde<9OnJ^JetKaRW?ujk*XZD66JvEl_=viO@+Q|HzrH0w?e5g8-^&ly z?$IJGD9ZQNMQ!d6@XwRdrkFxkPlgM|c9`yxQ1>kDUFSplO_kM<c)b$ErUO8de%zePX2Nr7$3#p}*~bOE#7=MPp$4U_Lpn|gpj0LL?jE;NR#N&* zV&q!hk#Kh4+Pk?58~aw!u;V2Bf|^1VOxgd63go7fe0HUfuVD~A|Lej%LDyM6p@5(Q2Lnx7J{ufUzB z9Q$SK=iu+3z~D~qtwFonL4d+S_RY$a?=`swduxXWRl;Btw~sI*78YDM znrmNWRBsYu^4!^6bKBH0d3=#3&ppL=kT4zbHkUm9ibw~xt^&?wki`!M45h{o&O}`i zSFECw%R=Q9)4M^Rbr$}7{$vlw0c;*};9-?Yf_1^JR_y7NRI5_sq-lRli?@cHn z8f1@*>=NM+$Id#4C@YjStR#|gY-MCzA$tqS%u(X^c=aB?@8|dV-hO{vx7&67aXII_ z&htEb> z&B`8bG9iCMJwzq|&~Q??kbe3GnE3644Kw-Taj^eiZNCDzxwPwNzS}Sl#HG6H>rP*O&PPS0FDloA+8c+Ag%$xvco8iO1o>aIPza;xnAvIX zLpbwMixXqLBLP1qJ%0gJH*8OBt`b4_BFx8e!JLITfh2VbtT>A58Pos>K}JWA#gI>) z*cT|da57T=<2P7jz+{n$+@>na9qn8Gz){>;PKcMw_1QkY9qBR}{vW*f+JEt4+S|S4 zysSB?o?lT#N(CLZZ%;e+D*5(h=H{cF7>L^@?@c{e8@E`GjgqUZHMcH>O{G5~{~aR- zQ67|`PeLkx58Na@_NgCEHU@9PQ*8JE^0>ZyX~1{SMGZSrCP6eW>dBRffN9jmxA?SInx$q*JUzyHtIDo+&La zPoj)plz$uAyV&;C_h{ml`k)<)nN1zl0ASc6aQf8rYwHtr7fuFB&FwhPgw~2HqkEg! zW`V1fhZK?0guLw$q8Wm|;{wM}cEz9_kpB58=tHoAp1LcU2X>4HIh35x#{F5mx2X~Q zOwMqZsKniuK_1;|=59?PL#HB^4y7ySadvq#er<$-kHu+A7A`bRswiA{y2sd;@?~M# zFQuM8yB|G8P1TEh4%kwHZV7_^Wn8g_N>|WW*t{mHi@?|^FAkBy{H3VTV6-f|N9@@w z+sc)1=*n(*(c_HwL!{nXfWWH^XIwdhNUx|HPn6R4A=oEkH6HG?WBT+^LLRw#Y5Z(< zh-z!Xl2Qr1yyy$om|W4Vg1Yth7ldTx_;%rrkcWgj-yj-zaYa-*qk_4( zJL5s)DG1L>PF&1$?+2BH{yRgYCd-FB-0s%B=cXr+6I(JZ_*@A)CSdVAa&Qn5rnR@q z1%=0G`rioUtSO{sUVnQt3dQ7rWSb7=Ym)fxYCTKs)kumMqZ7ickJru~z5#bwpskQ9 z90ANF20j`aDQyVcXhYdMJvL{->|>0xb5d7$$G^#+W?iJ8QdsH^$^suDWxVE-vy3c% zX>}5U4#&$AgahufV0tRHn{=gM3t^|+%4a!q9VJ8X#~gtPr?n(DiZedz-r;thXn%raedL&I{Zvn|DymF@Si>*df1(c|_oGcxz$Ty_J$;Ly%a) zl;%Ev1d0PkL~$?*ii5-H2)d6(nljkN8=qJPUt7wj(2i=^m9o3N>Y~T2=h?YL1v(mm z=D>WYTpPbm622f451NA{Ze^Vnod|=Q%s6sQf(VWb-Xr3Xtm1mAYOZh4H10t z-Qy2A#%()M+S7W895*sRrWSO2UJ>F^>8-12I+aPYaNnoJ0tCdBQgTnLT}v4(_t7vj zlH+xlta39`Oq4W9xDX8lzto7n;bhS28UEJ?*`(xIp_LO?QynG(KL0NKQ(WD2J{?D# zGP>wZT(5RLNPq*s#D?C5jb)*(InKDILRkAb{~9yvZ8|Y0^t@ROr{ulv<5X^p7zV|+ zuf)|#-2z2jB<^cdIIn+if|2&d{hb0EEkKymEk-_ut zQ$p?oXA8jim~0(XviTS89!p!Wj~`%n5+A8<3klN_I>b%zrM0m<>zPg?!cM=T0YD$c zJ8`2LMf&#X&7rf4gt?yFIHtLb1qENgmCqPQP3oDW9QX+J!ADmU9_1aql)&p?v0NyN zgk8hEg+zkg=N%q9Js~T*;jZ4u=p%qyd2i5GY-$(tmZ>QW-8h54mN&h&LE&8YKthNz zfp6Z=KqulWXxjMqJuYn9-)BjGy#iFndAVFVlUNm7CszeGN|5U)^JT{*F-$q3L#sn( z_y=bKPG42-_Gv6Q51c1yQ2;$D@Td(_k~%FtOS~MrPOl~ZBt-%Dp??D4a>0BN|3?=e z#U}EKEC0?7RT42J{`)32DTLj<>r*#1+mjNsgc^!`OzavOf3GWxoSIYc+m&(G9CMRC z@MA5y{p19}isT!E5#_Q%g#0B&G5`z9)-G0h=&)n5)`@AR%2E?VR*#KnC#S*&W>TOktLD zX7;pyb1s@^4^_F%%vZ77-RI+&Pl5LL`Cw#HE_-)l`WQZWFKg@TrdV`ZPd5yNpxo3w zpS+D&7ekcqDEa8}v7;y{?`KWcU-)=pjQP8+@t_%<%L zr*=DUr0r{T4%Om|6^W_n*D<)I57Va8!9jL3PaDcr4*`>#`f z3hq94W2d(*V&s2K9sZQxR4I1<)|n#M1H(p0WM0TC+|$9u5k}0-CxBu)UM;gZBkuCQ z^bY}9(_?WF;|~a{whN8Hg??Y=Mh5J^-X2@!{+AfRxpfI^jk-xJ5M|S$iuonyYR4h( z<>4C0bGq_Q10SKoZ`FV34DXA}U2CiTEnGF<<^Gv2USYB+14)|3h9~(|f65#@ZQ*z9 zO-`vzrNrl*t!dwdmEnrQH{o1D_u_I%>YCKR>s|1v(g!}7csp+uD{ZRF$T`H9GBQ%_ z`B+HqxoRa%Qq3atEeNgKWm10bN?(5lAd@77-7r6ty!i<25wp);_XzO?DSA-;d)RH_6m(T6gYF@#Bu+V*#_}`*cxv3OSd7Zo z!t{3z+9lr;Dgm1#y#=M?=^ul(@~I!eFs$hJy6(pi&=#p<0RP&t!hQ< zZnT^^B{0)cO2m#?k8f&cqex-IOdZMXuaHqv#XmYYaCGYa&rK}&U5MY(gK2J$;8p3i$&+63(b&!VK{_0k2@J5Z& zBc;b=?Y^8EFtV4Qn`lb?!P!V1aeb0po~|emx?m(J-YdJttRqO+q1IeAdN#pA9N}IB z!xKnzwDIB`7=M)}oP)j8FC%Q9yO{o|b_lBa0BK2>vfhNG3&<(}9L|;(_ezgX6H70_ZD~IR_61bAHUA7l{1mf)R_JZT)Y5atg4f|Jix}L;T56NgaVvkGx z6$+*8HU?VwM!+eYd9IkwtLPkG4I22x#pj}?)2+N}CY$t=Z`~haT;-B0sKm2a{8F#F zbdvLA+`x=Pe_c%%c~&YYs5o|(no zW9iZ<=F=NTPs2_!U>kuZ>Jrje?g* z*`_r5Qij-FdFS9SWd2UfY`%@HkUAtb_T1yO!|&%$&;*3B4$Ttbc4*WD&jtjn@ikpKpihC)W`9xJHUe7sj;@b!9 zEHm8o5|kyz?;fKA2*2awCbQa1RJW>~=DBH#?t&S}1ylT(r!ag^ap$le($_HFeaT7U zq{xdC#RdVJFOPiSWL>uTT-rM*_K*WDw0qA5^~3*F+5k9<1@36d;-7t&62U|BF-`>jfmK zpLZy(ydAUn2(R&awF`TOuB1p$)Vq((0vupxk7j_#``Q<>*!SU5-fFv@l*Dpmv4>FkyV%`95}y7Ej*j)RkJW z-?;vMwdjQ0MFZTl2Glhc)}Di%orICQm`-ge05oBg5xy;PL+Hr8Lf6b!>K&J=R;fW8 z)Z|@B%l|L;#N*uglPD>TjjoJ)K-Oacv`lxBuN4!~aqNbkyKX9p4|dQ}bdo}3rFP`I z@Y!%XJtT?PVOB$`1oRtQ*EKc6{XVqKhjU0#^FEgc>Lcd^PcH>6NcViJMa`ZiaDDQU zk-Z^z2tdH!vYzTzS?BB5D-w-aLl!U3UY=#{$XG((%hrGNMl{(qSgvx2PU*3;*^d)K@Pe3j4Qtk0y)dIag0I4)^AIVR zVGj;^@j${>?d;G*UBHEm$)69NoI7(F=c-lu7knD;;9>2d5Wh}ln+N51od0?u&OG<} zle&Du@U!edrSZ(WAYb|!tD%zq@7yK5ko{)&`F&ruqM6kq))5-7Qt`#M@jcUXp%Ki* ze%}U6hsp4CPt%eYQN^gKKDzQG)l2Y)MDfNuqy5{=L{p&ao!T>dE>BPMSMrOuH=qt5 zJ0~Rxay;$6J*o>UM1c`udY#Pxk790rcEb5ffM3p+oiY^63^ zo1>^`SO%f#P+6hg;-V@Q(feoH@;aQWZuf7;L9Onvpd#L$BGK?UG%LvbK@xC&yMmZD zZRo*j7mu!8Z?=Q;bW@95R?j2uvw?xZvpjBbA5_c=?)sX|(@!RZCa|d;`*YWFGr@v0 zycJON$Nysop7JgcA}BE-;1>*|5Z?(gZiFpv zN<#YSoFN34d7KPQyStz)O<-_R(Q5 z)4)p5QuG}m#ASRM7R$-7B48em$*4k0i6s0*GwSCyy}`)W(~&4@PtYE}I~O|ma@gj_ zL`;%CoJ4 zS~2J|!CxOly|VeDjQn+Z&(jV{cTS(VSo%nI%-id|NT{;vk8z)ydRt)Mc!ZS1fjXJn z*Zz@Dj^BX6Vo7msYTlsE%e$Mii$BM_jlUl=^^@gDxU(|Ul^ybP^cl2f^D7hU1;0d! zBBaX^(>X4CJ_t!$lM$j~-QhBaloA=4Lh^9ih~TGzmY~c^t|5JY3by4jq}p!4lKl!uf0K{<-Iry2%gDigVf#7hj*KUH{;MLFmghgoiWrh^&=|Rsu zP)SIm1(7t6bPTVrDWqWpwV ze61b!2;!|N@b^HM)&tq;XHZ9=xIVo8@RHyv1L%W};M4b%+qZOs zm)|f&Kv(GYB%s?_1-7C4Z3%3W&9AI#2be{#c3VL}-XbJ5(HTxbr&Z7Ymh4HxVSo1j zBHFWl_&-E@tK$D3(O&wy|3kF5ST$&rMqMo(ac^T<6QL-EqjhXx&s;>pN8PoUCF*xf zx4}4n{^w&40y28>KuFZ{xtTCAQD$+bU~T{ctn++5cUVxyMvPy&qtcP1#)OffnfzRBuf`&q= zNnpQCSZE*1d&~Xv^4i1wX*Mh=YYy%jR0h|HLiwcL^ghG|z7lJPG_c0bJRnPqd_F)c zH`H|k0BBWUt;ngzCQj>Qk{&Di1u1f@Qpsz(5aE&T2r!M{JgWnZ#YrGl22#w2URju- zxg)tDJW^1aCOqmgsOR-r*yT!V7dZk|FO%^SQz#z(o;vaZw;d~PS!0s_3GNVN%#R*> zH;e-7Hw4(hC%?6k{+;y+OUPe7wq5}^h}I+i`~B_PKkgk_qLmbeUS5+})JvJ;C%EL? z$*6^}=Wfu|i7y%DsaZ=60(mO!w9npG?K##BsFLSdk{Ojwr7ZU(#{{FeUN{(CAa_!< zH5deHWL@WBN?90(EAyx9^2}J-h(6+kD7yGMI6=&PCMs{l2Z9Ts8+=P=bRM;lXJS>i z4Ow!cxR%zY8mFMFxWh0<=}M>K_OY2ma46W5(eZeEd??`!)*!o;;WF>jS6`i?B4>$7 zNQ-Jk$e_OQ^lLV5&><|`$1#T#tS{^;-C4z5n<`RMebPiD0cW-vxw6lfXyNcB0@8P5$Y6( z%t1?ai-{K`p6ovv*R8^Shea;oHOvCHN&r0VoQ?3sg}bCc#$I2iz&DuDSqqNfx`A$> zcRG8U5hF#QKX~gt{IQv2)-yn~MlDjE8Gi&kd=FgkZ&2fI%xk0S6zK0PQmygN`oF^Y zbb^4fUf3d-#Z*iKC|^6s>=pU(X}gIyoK%1yV4$VdCV+dTGM_8fD6OyrCwS@vHFkd= z`Z)eNvyYahAfw7$KaN&ey<2`{Yfb7ZOHo!re_^tucl{Ns=+D*PL#6FBW=qZCm!X*U zYZ5t>&LW3ow2|=+6|tHCf!?4VoC8%Ns8YeDKHeHpiV zBm6VW#DX@_-CD1@R=)CSU;HxcY!8J}NqW!1*B@0>DA>CklRE$JvdZ9aKoe}r?ZT1t zLDzTt>{KkBx2rabepLYJrAs^N@%Wo?JPkf({Og)>D{pr;>Ig~Eq7SY-9JUU~5o28I zrEsLt+xXt682zf&x_t8*uLx$-NJf}WFFOS*h%N#_u36( z_qUnw0oNx5`m!YF9v1PipCB0Ei%%lu^Z&=?3_fXIw}ccGr8NIHdKXGN{@VwM3B*zc z(dh{iZ7(4_Y|V9JKZ1;xf=no-8<7GycF^({F}FLY4wGq`7bGg2D+xMSf8Sfg@Z9 zt=1N#;9GMXiL$QA+Q^DcVsG|XV8Q8D_1}F_cv3G{d-0!}3tduX=3VZ0R==e8-T0So zuYvFXPq(M`hM$nYFp3`FW#Mybv^bW3CuTW1Z5bB_YF8a=zviANCkSF3e1pqfU6E!I zO#&3%7CRcLwf7oU3689GuIr7zRz!VIY85!DZe}U}tz}#KBS4Dc4rH3=+Gpxj1yuk| z%?vJm={GJxSqWoli~Vo6OTs{L8HVNP8e`k!e|R&-KFQ`yUuML}Bw}JC1qgPZ z$WQW$fi~S^Z{gxX*#k&PFDMz_5+bGmt!wiIiQ^9q?O=$8?E9~Tk7>6bV>LUE;&_3!+&p=kKTRHBAY35m~2(vjwx!CLum0C^d5@i!1-|6vZ zufB7^GJNY!*&`VXD%DCN;M+I1F}${W#U5~adz3awJ|dar^9ssMpb|p2HNvx4LjtW2 zW#jh0!}v~}&a-?C#mGkhWd;N9ciwv{YpuR>iN7zF*~D3X-#`A|iVeJ!YmaV#d9z!@ z8OC_oI6URL156hocXUp`hyusxtUbADTE&$}nk{mw;2{}(SjSzT1u3g(DghpQPc!hT2 zgj2#g`_e9j|I|rBzaDRT!I?3Z90zes;u(C$)QDW`^rieyJRN*X_6^y&b^0OMa^BFA zdi_=!zSt+0Se%Re6m}pM2?^eud8+rDJ{(1%7Vs4=U}h#Jm%SeO?>Tn7xZDwA90uXk z{|*O!I35Q%Yh0lfxrwrD?3mO06qJr7>+ev!!^b!rrXRXX6RI`IIH2`L&c{vTwzuFV ztao4Wd)_4ykFU#%#SFaghUZ!?B>&_mHlW8;|4vz1n+!33q20p8u>`2Sod9lvjNu>hdMy1^fs<0Enq;C$U&OH~fydRqf zaZ}%3==q9g*d0L1j94shU;^U-L2^(U6Ado;JY|}Um7TR(`5V6`?SxYDXG4XIN80zY=5#;?5}5);4q_CRx;UOfp;dupxiG;^#qTP4A1zkfiRP|?E$?g|78 zx+^hPcVRIvQ^B{Os<&N)+}mN2?zQ@45A)&8R>Y@9*cd1biG@CbJmuu}>YP{OR< zA@azS%>WPG3{_;)UWYCUo4$S9yTL-mGb@-XHx25~g!hT?&qjH74F%67oE6dqmy>Ym zjUnBBwr_~?a2}1y2D-vU>d^&Xam_{6O8vAFbCLk5@j-=Tu18Pf0_YZbudj5Ip z3WcLW)Mbc6cDAa0T@1y$B%LbnR|}S(RKKTBC7Bqyh_k+m3KjYdov3K>s`UKWChO~6 zILe}sBGddUtM{c}-&1_X_lA#NlIm~XI?S|xysOXSdvV;q;b7~6S*ev`O5`~zMr)y$ zh=Qzcu$!i*V_B&?NkOi^_n4!9(I`XlF!eoskN(dIP8mJZFKJ|?W*mzyyym>P{9sxy zcwqMqJcs9QR|MX=E&HMfVy=bq>3bLG!fwnB<2iEmLcW$>GzZT0awPm2c2Ih^Y zkHwW=LK`f7iBLSf(xjC5`**Y)GW!1JfX<4c@+vlza_m#C@c^QEt5HXY%4wZ_Nrr_T zX$R5w=WiC{YFexw^OA=7G$U{D_^|UByYd}Jq+VdpjvBTX28;=5B&TRcWxn0!u`Z~_ zeVxsFfh;fZ>xaNM)9qNeN7V7^moGC?b-DDn6DPJYQsF?gTR-&;H@u_ySovI11`$Ui2z`ub4>6@>u$mv|8q#)hyz^t;V_OUCGg!H%Na2o{|2a` zq;_sppz7eKHeit3{XzEb^n1CN-akU%vTnK*HO4u&q~fp9baKF4k=KA4AvY^#ySQ*~ z-?91JGgp{`oBmeA(94SamK0hs#UxVh__Hh~axHzOM9PdA(@QUCs?x^C6nzTSqu_Jr z-BmRLP)8*SkXGZ1F|@+mh+<9flBP24rr($2{b#tCh%50Z9lCw8)^xsLIs>5y8V2Dv z7q6;vhaZq$la-oA@`v`pfg4$)72Op**pp4IJ_my+pL}?y_C_F$1#Jbi7H?hJ#ksWt{ng#bozEj-||{7u^Y^2TT1v@>?4^A8;&7I9!jO4UT0U@ zVT*W~o_P3Y?~(s^)4}f3j8}mj{bn!_45fZZa3*k9wLve%p^+}Zui@f3R>O0w0tpw` zN7sQ6BGXqI^%aurRTJnHi5k(Omee$_bc&o1k;l(FXt`9u|E1J}p`58T z$8JWuJh(|^;2o&Z&wyL`h4b&9-wQVN#`*|d)5#pN=aa}qvlDKm|7#NHVE0N1o`lK{ zC3W4MugLvL4q&%pn(OuD*XVuXrqk@RCFgwjy@pGFGv4~@%QBk!k1&%bqSo`35S&Yx z7NtXsfZ<*Ggp{699jrn`d}C?101Z<r;BLTul)Q@$U(fT#xO_FqFrF_a;vo@os{b#OwoQe<6X9 zl>@C}G4b?Y?Vd((0V!(whislZ*uK->CqtQmHAmmE@6#uyE($90SRioM9#j|#vyo)E zdL(t3noU#MKdoUNrBHXlYd+&;v~0ixDoUi1N}2K%T{yUyKm3j{6M`w{QnpP9lotN8 zOkVKWFf2^^IP*+dph51tm+0TH$Fc(aZwYWa<~ji%Fgl7-~%1?OGT%g zqH4aD&c)g2^f)lpT&hgv6Ue|O@``*f4dX|OjA0l)+o7t%UF1E&+{LAv>7my z(wB?B=yWs(ersNW4ki|C@@w`|QluCvjs02fDq^q*b`tPo`=;GQiuEQp7S4NS?z)*n z)UQ{a)?1(G(`y!<*BmqI`@1TGA0Aa#h=*~*PF#6*l9rJs2m zKIxKBRgaz8@r60~=w0KViy+qG2se4a^pTy9?U&z-b_;z~IufmNSInKj3qRn)Jnj03 zQ|KD~4R(sQY6MZ>aQr^`hqIqo%E|a^RZFKlE~mWi(JR?!YQ`m+VbrkE7go&gMdXNQ z`;Ondn%pa%h>QLEZ$} zhe*e85o?0UP}D9PsTOnaCAOlOd+@k_-=p27{a~<>Q-~dE)%t2#rLe}yD}G9*yYEPL zR?_L*x1S}$6p3N8b2&@o-@dlS^|Ll)9VD1xe+c}dH&>S-Hi!nR)fel_y@Okio!N0A z2PuV7QyX#q>c4`55FpWYmRQ^V1h-iZP0@x_armWjy+8q4w@(j$$WGLi$uE4oBCj04GZAlxo1dXA>izHDyvDwr)qs(kG8i$r_HpjJNP_|0jzz4CN`*>{nECcLou!r5 z;%0f7%^%znMaWrySeXa(NG^~a{vn$iRnPhI)*}9xH=k@^+RhGyt_r@~dKBqpze#}9 zl*nfbSE?uP4pEg9Y}gl{$YjxI^hGk z-UnJYoT4pAJAYLGP*UMVBKf~ zJ`Cs^@c`X`AzHWnD}=(Rr=*)`FHlJ3wY#O#@wiV7J*C?M*sXl7(d`jn_!0)x z{XTJRk%JSSi``iResNAii1FSZ=yvTh?}ulpwWrsv7M`Pd11xTY?nyH0pMMbB4$b=- za%{!6w7*iPTnr~&$4|h#^~D^2Lwq}_EelsG^lge>{S3u+jMRLu{^sK7-|xr72UWlz z$&-Rt_glt&K>Ymk=I@VYsXJ6tg$m8Ywl!HT79X!7VxuI|K^Pu5O7WZEQ&)A+)Z4R; zm8Fo6zbHl*tl*-AKKzT1k0FJE2$evLIs)s6%wI<{f=-{AK8|3F(Yj3xch7K;{G(8?_8$!`NQ-ejJnNhUXD=}cNLtt6sd|rUVPg|*yIGVtu7<+96Nre!D8cwLiF?O4%GZE zM4IOwl3kz=(U;1y(dVa1*Nh;Gt0K83^WiK1TcK<$iB|jxDOBp3nowpGk-;f_`8}G= z4q6Th5$k|&Lm!EWB@PkZ`&m{d^lNDL%G{dQ-v-Jeikq%O>j9qYU%s6w=Z!#PC4$xp zuwvqSU0Thf=`4odaH|iNHzpzbGbQS{GLulp+10QescKeROn(Bey3He_R4YODj(VnX z<*9i3(DpWLwC)2M774CTBR<{nvdxNvni~~Q)V@~@607=oknwud43qO>&Thzc)5W&} z)u4w71#j}5k7hu`^Fr+YIL^ODH>u<>cwPmpj31=UJ5_l{Erp@ zoJ=3L7Ve2g3k_cUnF*E-UM(6CJw}pEo%`C(Yy(}jQ1n8dS{<}|Gvy)QkML=2^OTZ} z+fpA661rs=IsKV%59yZ&Q*I-{45|Ik8saCg)6+`KK3aD@%j=HS7Aw2^2%fB*)xYJj zbd`k}cRBX%aH(}vm4N)rqnY-W-#)xz{v&75wD)t_~?Z>k2!lQR* zfsx9{NriyQPk$;-Rc;BicM;Z(`fIc+@fo_r%BMP?4;%;MedZHs$-T~XY7Wd|>i!#L z!G!QArq(>}C;ENHRn(#H*E@a;U%~#^V;|ZSC00bnOs*V))q77m2F%Uk&oyanw6sD> z8TprLs7eY=n7>h>@(iU$Z=ok9>w|S0v>b#vAglW}cjYD?f0u7ajM>9SMT*Ty0nd&> zD_d*;0DkTSVfk0eM~w{&oe^td{hKwL=GeX|@QWI^7C-nUx3l58nzQRx;RNg1d9x+Y z6*Wam?9j7L85FR>GH@Is7W$=AMOkFP*}=#X)|rUjITD+%A@}zeP*n%kiZjn# zL;y~TMvn6tiqG$i)rooFF-@gh=1Wvq%DycmN;jit{6mNkru8MuP-=hyt!B#8xe47Z zCnIUCRh>S7)`&rxFhns;94S-`kf1YMdkf}-2aV2UZ)#AB@Kyf`=~PsvbV{}`@e#A| zt{fISJwg+kf!CY;>%y7S`0`qf`xdvC3e5lUC>D0a15_~;wEDV|%AU$W5f~@Ey*@26 z2Wp0A&#(#d@x}TmUgDm0GDhCs76SH2nL>>NQ>wkUkGTK-sK9?xDb8P4k?*W~9%$F! zrc$2PoV|6(OsW?g3hD(pS@_&77yC>h^_?1tv1?(<_s$DN4M5_>S7xt*_>TUz^Ik}| zOMg<#ZGG(mEE&%&S~u;}+Ru2lI1NAZ1<_wG?N~)7J!7bf+a|3!QzWbT|scD`r3|L3*cnWOoKA4y0r|5mxNe6h&M0j)G z!|4KEiLsb?-&1gMwt#6JAHax73U7V$T&qBv?|8u^mh>_J13Fu+#3#RQli-Wxn%R+y zfEc4}GxDjUi(6`6&>lwJ%I($ejJfJ#a=C?9t%yLnq@8~Jwu14niFOd=GB_%6NTk;RoAcSLSZ@j4 za5~Ee%pW2tt7UvCV$-J~Ov(J}dyg9_Ieem8SFvk4&zl6gow>}}-<)YD)`S3OMAYOm z=&@2Cse@#ZvLUxqJ|wuVzp3I++f{zc^lMhWQLgKJ?H*4wb1S|VROr6F`A=&S)d1w5 zt6L;#M#M(rJPwfRuS~2CT1|(G*w~&<^BR(D1&Ou{c z@?QbuO_5)G(zJoa1ky8hpyw@GkWJrT?kN9&B@|i$JJ`(T7t73XsS10!-7P*Hjn~UV z*LDa{)9^$YCd5QD;QXw}JAcJcQYQM)Xh@sCalWWH^{Z__C9)!F?4o{Ef7DW*2e;i9 zV-1~FKSIEw=T9N!m@=1x~d-&X1|6Yq@hz8H^5YLbZ&2i()Zud3Xk!@g8 zO=V)|XRcDkDv(^Y*uKzYz7G3@| zrXLA628#p4m{S$K6680rsM zf}x_>W#dMvK}1xzEi`a2`2)xKOoT@O+E2^!jof1qe=;qB!c8#&v* zfjI9Pai!L};5|w8TWl>R4t4;03 zKnKyJovKfkx0(Zz(t@`!P153>s`tt#TN^X{znVK$ZkL13QQDyu9oPqgvo_+;Is*Ni zWrA+;8$N}962&Wj%IkV}--b7^H~wHb%rp_e_Ipr+>_FxVg+4l5SDr7Pe_}LUN8(8C zvJv96@S>z)*SzeZIH#-AJDD%dL)hqRGN13W$ov%e7E|=Fw?bxVzyu`~g{;L;w|%>2 z5%QB9YA83yi=V?2sTD7(@@>*^$eg9Wk?qEJShM7#rdv_<`dq1_^-TrXRg2Sd@8D-5 zQiYGZ9)4@?Fe19B2vpW<&p3+Zq8+DF(q zaA-){=@vR7HR94 ze<~)2z~BaAuZn#W1j9+~msF87#vhknUOkhNH_#)?B=^~AhP`gk=238y(&LFcj1i!# zPxD2ToTh@TTu;!4$dunD$A}rRbwyl}#Eq8P#T$gUp?7--br2$B zp>!EwVFC@j`(H9ns>BqG7@vyN-f1wObGoj4!j-9kUJIfZrJO(KYtF7jWBxgJ3ft#E zoQ9q+Tw{cAO7o(p<|czPe?U<#3Hl?JRX4O)9R!%Yb?tBN&XU%%S}xCf_AJ3S4;cJg zJSh~M%RX=2;Xg_=jXrrPDgQCIc*AFRX*R(G#Y??>iKGNcg%e}O&1(I`lXji$C_gqm zi4Skj0D)=&@-tvoR5gZeTMC(65}M1r*U5<8_z3WXB{Ma=jr_ATo+&}6Dt|s4%`lUk z&T?59M*4ByKxz|Q1bFuqErNrs(WxOjn>sLz&hVm&7^FM`?L=ERtLB%)_|SEozH^rb_d3;(qhKm~ zg`wr1q=ruBqy)j=?p`UF9D?MbZEghBIYmbk>A3AsodyH!hO-$kZjI( zx#+|6(N(wF^}FOm{?B9XUQ&c15!8i{@w=@%s+FwajDdmAbT((7b@KnrJzz7y9Q(ED zf2SMvejmh5_Dcv>mI0>m67%^AQ42dt%P#uuBX}@4_wj&EQ?tzYYMw*|p)cnOP)AU# zD1+#$KjG@5Ytm}<_L7CjM`=m%YiCWKyDN9OG?OWD;ay_pkVt5)q{lpUO!~3dRW2>i z#EG8jd~H=**@=0snZFJp&6Hd6Mj=*t0PT;_g z|K;#5HNsd{wld!o5LNtEl-5d0cJeX2y>WT#(J6?B=#ST~LPFIOdB0H%Lfl(8**98t z56s4EM;NKqzJ@9_YCAoaaTEB8aVPl$H|@uCY6Oi^>&DO!-}Ue{rmXUCTD=XV*h9e~ zo@W16e@EpRGRs;ps@qLHiHx@SC{FJ+gdY4GZJ3bvit2*gi6dW)T=7U z)Yd`1#8bC;l<3^0)D$`~z^B$Ca&@b$Z(I$1LgucXehja4L(u`Pdljzz9#Z70IMU!0BM4L!>&=?Ni-4^Ce>zgGFG{wk>P zFOL*{FPtH@#K`>4f1isU#-KopKl-!fU8dd3KDQQKR;%ipieE;{g~Ba|&6tV$5H?ua z>HfP*;^+owvrzxc6OqUVb7~s=3}4$o7deRZ zk-i^hfT-NGGasQx?E2c65?yYWsGUdZ;hS&T0GVmH_R%Y&p zNq|uKN4(pBrZAEdta?{Qk!!dvB5@qRJh} z24tdoC1Un05~G(*gIfj5i7MY-1^GF5wE~oSxWSMa29iTDQ^rpwHDutC>1pfRBCSwo zksWfjzv&!xpIwns)vI%>YET!V-L9%o?J+9Aj(ODt;K3eekphqgEUQxJ&{H{>szpJt z`)2B>{__qw<-ZSoYjV_ufoMzh=%Sh zN7&8k-qfQ0I;lsIQc2k*HV;}^dsN` zqKFos{3(Wp${lyeO*eC6BhzcTOtP*KGKt6ycW*(~thF0NmLsv@*_{RH#r>qBcmJm` z-~<)Xs9v?UsQb$j7yh#tf{z9LwDaH_7vnCc5xOKFc8m?(FeY-CV5cK|8Bj4^IT;Eh z41^@$nYfa#Q6htE{Pc#)^s6_$QZl}`oXcx%PGOh0|`?#@Ub6cclV)7$bv}CBgi>TGxL4^ zrc06058$~E89JOJUkNKO%Y-8O#E&X(e2Ux{e9KBpghz};4&13c(0Bp9Gzc*sWI_^H zsF20kEVj%xa365kJW2aR!&@Q>5y{NX-IGKqS-p+!EXM&Y&OMK^ zknE!M4J1W0>)U)+njj-gO}bT`Ch^Q70$UH`#oz0rUrWPvKc_4$5TO$lX&HGr;hh0k z&QBwlqg#?gh%D)AsdMuapWQz8HMyX$*M9NqwY=t`&G{7Zf|N1&`PV8pEStNmIQ z>~9{+$35gGDTT{l;k}cV#?j+SpM8ylhTq@iGuj!k!zO2ua)fk?c_3nMq*9b3w196l z&;ao@i4b<2V9xDCd zAE17_EvWPPJ0znGRj=ak+(Yn2^{x2c!$ktup4@qvjdRZTV=z3NQEN5@pZtV9;2D&N zf4Y|5;pz^#ZM!@R2U*^L8VH)JExwYyOok?Zulq}*HGsGHx_$O8y<7*SK#x!F!T9cGM#}|$#BD<9= z&O_|prKCo%qT*KR@caJtA7~e{8eiDJyQFBc-hK7#6Nu@l$SCjecV@eD>Pnq^r4~rT z%Et*rlMLhw6&=W@x^tJEgnWK4VGCE23mWFdDu+Ertz9Sc#29lRewLzE9hiy8wQX`% zZW`StPuNoG5^Fjplj%PG$H5{g#dVxuYN!0r?so<@AMpPP#{i$Z3jG{uBbsSZ%{^ff zrw>EH{Xz0|@UFL*sUUm1_Rr-9W=4t*oJwDVH*V1yD*;JVOgTX_n1xs2(+C(!^aseb~W{6(LP z&Z*iZM`_UY_W#4)o5xetwr}IvHnt)5MuyBY5jKU&Jj+xmkurx$hB9WzHp@2FEy@s; zA(WwHp2w1*!H{U9GP6Z&k$&g8d!GAyzrTOpzuy18&!^Alaa;Dh2a;M z`p-SiL8(i}s)U3{&V83W^d4C7;vFL$Z2nqOWk_|Kb%^i?VCOlR(eEe8-_Cxtyfp49 z&Y${L;K=@ZXxwYxM~HooL0@a~7(ZIW++hqwXy!}<+MK2HJJDM&Zc(aHQYzs?D{Ol8e0cD+ZN(HilVQWXzOrPr_0AK zkXKy7F7P<-rxFNhy14&9$W3?=R9LCrkm@M9nusa;jazpFdl1*A`G5#=)#`KgOhiBwC|7d)hu>up%bw9DuJNhqFqf6{*x zlOH5PO~X*5v~Es2b{`iXPAWeyNbYLA^&YPzggceT;vMz;dnoo?#T@VkQuUG&PAiM& z+B2P{E`xeO&a7TftW6xVsmVZ2#REs8XVWjgUhYk=1m`?l@ZwzaZ5KQubvHS1Y)2@RaQhM^*XMNU!#1N#%o{fXq8#ZaNO^Modb9=^yuJ+E7B-y*c5{~3ph^^`R_BSXl-z58)^Co^;Ui4 z&oihJxL-=@6m8Mb?`uKj2CjWGsAo48Nd_KA$*~Gt#RM^-gC3nhRI$F{$3Xcb;G0Q{ zgjt9x%DHQyo#9b5bh@8wMdhuY_Iqg7R=j`DIbb&aD$A-&-u8Zr>nDo(*n_gaoRZh% z?UW}1oJfZ`&MlrAF}y>v7}KN`AkIqmpyr_4%XihNH}U1E@EpFiLVp~hbMP|&A%KR% zGcm_jJDclVN!sQ49y}E%!Jq~82qVFzAhfbLAt3Ke3@_{7n!97iWfC;ufyYYxTW`0! zQRHu{^P2Ckq+!pj_^IgH#~znr(-Kr3(EKFffqa@?lo(|(*`Bj2WQ0lmx2u1o$$Lpc7DY|N9|0=RMsMoK%2_R5u%FR+X{Z_3yuWB(qEq{mVA0d4YCp#$-$(7pVRLaxD7+Y$D{Y_Nv7CL5W9vgRrBFwtI> z*&n=M_IIQ=YZKo8EBPzkmF^7)HAVK?JI29_uvn9O%iTJC@KvTBk6r_C8&iHuww{Vt8e6z1}4r)^zh?yl%I7Hmz-0&Mc2O*WyK5<2}e*#9gM*7)0GaALFGdfYUJoU6%QK!d1%o3fmpnnA&uE-M4*omHV!D)f}IU|jzZ1Qys z-YN2v@QXuGIMNvc`^0TShe@VLtm%rq1fEB$)dksWEdZb ziXd_YrgVHP2#h!(d_DpD=*}YkSgE0UkBrVCNS}BIzfphyard0KpX8{6{!sWOR5X?I zS8q|LyJ_xzz=Wli_3Rt-Roc%)LmhHHIa>IuL)YQAR!dvFPD`24o!4z70X?R5FFzSW zuVUhU*3m$TmX{$GR*$eU=;RQ`_hjI`@c2nut&cQ^X6RAHb3Do;vc&ZPek+{MwHR&_ zF7P^FD$r-$3d#Ndu1%1Pku2%jN1uUJ!DQO-q}4qczW1Z_8!<**+y_(N;D4M{X-e@n zu#J_W8^#-lf%>gfgQ+?g5LE`Qxy&8Z=E!!s3ic@7r{^qM1q|hJmr7Qfx#q1Vl4Op) z5l*o<#kP;P1ubwy?AX5d9pw*K!Fq~5kxP-39Agp#_LBxw^vu6qX2OF!-{mJPT52x0 zKnIQ!w|T{8$;=F5jCK7=;}4I0gc8JLa>(TIgrN!&FXN4x_cYJH>S?r)Kx-v9r)pmB zw6}4%k&|50X*t5}V_48fA8zc_WQ9?8p;pO0Y2hOB)@nS!4Fc6?)l&@R#C-f$5b+Y_ z2ItnH@QkSoOgzbICkn~M`{ebobIBgm~&@rCC}5^@y-@{H|xR6e@q=5buy(I zERi>$j&j~!$ZyEnmGA4`#VKrJfV6^m@QIUK{d8xb;+HtWpYY`&@uK)W69_|_U5`rc zX$!x@@|k2`GArJ%zU}@F>NM5k910>H;j%BRUcMF?(!opSz=VmUHkc0hacCl&#<~DocbW}DM9Y- z(&WWnCE=No1q(r4-L;e7(qStxIcmnBi-?K5ew&d+ubeLsO3_-TEhlon zcsw89cQ4V>kKT;pT|<e_#iBdZk z!_3!yF^oCfULK^MR&{83s)8W1J~&RjYU5Ho%;9`;wq028G_8OMtcnlI0^5FQwf%<6 z+x*Qt&j)F?>%aYW(?~jgDGi}DD+U*Yz|Asmr+t#{vDWC9L*5)Dh97fs$ z3?g5eU0z36{$y0Xn)0wLAavZN75DtU zD((4+!7YlPHwB|U^VU9c!Rpzh+w|aY1M=io`co3+jnJy1gGS%Y)_oxN@ZyH|SQceQ zo8vd+)>ZVgAn3u~N8-uW6%%M0RUgK7|8&71bDg~kBeOkH>sp2Bgfo*W8bT%as`sfPW*dm;FRv0vuU`rS--OBFc<(x`2y-fY zh3FR}x2-|a7*nW9xcRovJz^M7*QrciV+YSCWp5X%k!P`qANKd*CNM!&Dy{94+FPSV zp%kKHX-3b3Q1{K{@l!!jArc9XKn6`zXKMS-iae{MM5b4G4JZ@0-w_&b*&Ip7JH7p- zgcPC?p_7N&cWS9K(Kl!MvfiT6Ip^;=YLdJ|twD!k_D#wR>6uNQZd~&oN*TasDcyD@ zYL7zjz)E-X=jbyCB$HYcm^p-<=Sp{I70l)P5X)roU2w7bi&oNk>T2ZLK^LL082OvG z9~-1O$6p5u3R#yS@;3IA#I5wL=HYHcd&70*(|p!1&i~vG@z%#uh^^zKBG$I0*skk4 zGKMFGNPGixFzJx5$HHhZvX|49NIyJzyrcFPzwlYnGmOI(68tJzgU^-E^^ z;=jt9T{q?Hc2+yEqUZ>tm>>FQoqc=JNOHrBbW*Q$CMil<+xM5usy_lPZ?hsX~?xDqdX z7AvmjpjV7GvYjoqzJLAdN-}Gc3%_vb@=7z^s1$w4iD=vr^67k_l?lIW;k#mIYatwNz%*p5nq?+z%V z-_l#Z@V!v}UQx(55a^t!&w82WNw}Ydnr?_s`=j(IG$UQNYJ1x5f%nekLF0(nY`+p~ z`HjvygmKTyBgsD6H#&>fhxOcxJI?BM#JeVPcT-<1YcnBpJT4pK?W7UTAakis_gW=O(?$@yhIWpZT)m4AFcj>0 z>U#%3x%Q!O2duXo6GN+}6xRu^W9mXqhX_epT4UDI-&_xpB2KoOT|Xdvf)_1Hj*ocq zR0<`0ViEb7uiPWq14YjeT)(mSwhGMs*zN=nAqwDL^Qg3(>lR63ib~W~!s7@zpOv~w z8ni$815AomkGI=Wj^g`>t=gS&c5*3pSzdi>EcZ2N&T34ya*&3{FaVdd+ICb?q3E_2 zUv5%zjP%XQlhOS~Y^J12C8AdrEEbXT5h*P$?B}G@7AJ;(s8Z1fC(z)qgc4CfLe5{~ z`NuhszjG4E(uT;hnc0G13Nyhz-)-_0pG)cF^SVhFLn+wz< zDe`T&8ZCW?=!nF9gjQ^FvmDbwK53M2G9CsZe%*Q$e zgK>S4J7va9tG@z@<7QmTi$06x3!*eqF8DYKJ>qMBS4rY1e4t8%#9|N|mshiKChq%hd4}%5FUzP&9 z`SP#BwH^thI`lMAP1W7=wZej^j^psN)$05d7;$J3a_G;T<R?D=cGTJMk^5EW}|&$pNPPGmPfHtXr>=MtVoAna`j&W>)Gd%h?Hht)zJ-$SUV$<6&`%qM-}J{jOjn(Q-uPIIuq zFQ@t}KIjQHpkEPr=U&~BT{`~d>({T3Uml>JxFFXIXE}uH_JGS79l!Y`gbFjeZMnzr zs4iTi2I@QM%6YgNXEFG?l0~I6weoP+j5@|$x56-y@O{HxSSnoiarm(H=+!4a4j4lc zXLMTPJ}TrG7jn!GIq^iqlbh$^2XEZm9vJx@`tAs*Wy5X!1*fa57&|@pulGcdrLi*% z2Dl2j^bP*gzouc>lHv{N;6mVYctMmfRuE1`HE(iQ>?ZuMpBtVxJ?n{Bvv88nJ^18* z2Pzu{N2S6_Y+^O#VeIHC?LHj9VPI|8Y4$JQFG=|OT>l-)LvUgLeL!k7a9tLAtY2aY z_VDE_E=^C)2x9}_Y=jlZ!zWO7PMZIEgz)YEoftt3Faia8Dm;WxEwJ(k^oE`())gef z*{DBvTz-K5b7%jZI{%uGo9wVqm{U2AYoo)BU`C|>dSV%;NkXo=G5&)ltY%dInzC_W zm`8VEg520zSe6om8G!1pjl!UB!>{DG&y=zfnqll%f;o!`FgnzKj}Dw2n`y>Ig)@|c z`(OV)JZ_;0-$Rq^%ZS5>!>{Z&iqDCm2oiAB6j@7ZG4f$rxIR|V6E0s2&*uAFIx8#( z|7*GOX!AL%*xR5#vp57@vHs#Ob^F@umbihMZBgHtYyfGiTMXaQSj~m z{F{?VZs6Zpue8_;r~aRRg3~Z+uIKL-YX9?>$PdXXa9Qo6b0#>LNdNrnR)NRBJuLPl z83PmIpMRoea5vt5M=kz!UQM(@A`D5VZ`}XyI^tk>%rmq4*#C82EL=B?r1yn91{mId z&KnDYaPXqL7AWfd?Jo)b3 zyYH6HPQyDHZYo$zvkQ$x5#z@$XZ`(fG~d$ieE$6_=XSICQOgS_V5yEk7MK&Rg$MpD zFdgQwJyN(AnBhPFyL7{zj_f+OkWH4W+Cu`?pKHkWVepg(Haj@>2w7f_&Mi#9hl-Fz zm*G`y%%2Z+!s6YZ{`7<%tZBF6;Lq@c4;TNe7YfLyTsGn_i@^z1;ev89AY#!>H5#1!*Hl%S^bPH9NL2% z+G7;+XSqTnp@`SdQwJgb=*|g4H|j_Kxjr)_G$>lO1WuI%+~17z%i_VmSJ*oE{PmYB zAFkoL_rbN*ynT1M;O~!wAse_REV`JZ@O2M1%Ra(@z?xjf2a)w&4D3Y@kS*p0OK}MD zYh*wEd!so5A(%!%txW=i0PyK#x<6axEPPMIMBC+@KPOlHzx**uiVXCNYr|mN;}2OM z$CKZkMDA&XexV$?L($GUn;aomO+xuY)ul?IK9`8b@LGRaC?{TQ#jU(0{vcQ3_6KLH95Tj}r@viuV2p5h6MD6)I z@yu|*5^-Z2xFOZh40g}Tw%jpld;vNYWRtlHJanR&ym^#8d;%JNujTwsPtJV~_71wB z&Zn_F0WP~Q-0Luo=Ur{p~KsJ27 z>rBi16ilrj>X34-y^?wgh(c!pu!WaFd(-F5edVOrF{^Fq*5E#>Q|$-ptKb_Z(=IWj z$QQ3w()eX-Gvqs5Z!PFKj?8Pu9nOB~KI%9<1(o8;X-+v?cSX+K3%G=SP#ff%2+F<> zdF@z8zy@&Qg10w8JK|zLz3(J94^)hyv434mV{>7+zjFi*F9PjN)`{;KFzq|~zK%Z5 z;i9AC4|tFj1Qy93w|nk_7WPpo#cLyb@A1m@=%xh{tQJ4 zpDedu9ozRo>RYJt_D%zI25$oNJeIwgqQ}1z&`|stc~_1RG#Pg+%+)@qJI79d>-ac+cp=Gha60Daz%bX97~X5Kb?{=nA;q7uEIoJ837U{jhG~W%c3SBc*xu(4 z!T3PrSGm12KpU@9(_%UR|?5$U``*VSB_}dL2XT9_eJFx&8SmDtvNEwMKZc7gC z4!VNsbDZrWj(4zpk;157*+I+r!OuYZetGTdai@L7O##vJg@G>E?&mM`;yJ`ASjx`* z8SG>{*or1DOg6#{fj?=n(3?Q2uAw3{+iStzM2Y4rNW zy)zXrcOJYGrxIO9rZCk|CRob)>Uzy_H8(Yu0-1R&uyj4nx)S#@-5@fdHS|SRbhBBO z+pNA>9|aZFH}qw%W$PlBxmS*~csJMWc^Fx4eDiNa0v2qS5&W{Gyx{3Y&_MMDQO)P& zf!}*1?iY3mwH^TrzZ0_;;)I&7^K<91!R?#{_U)T2zmJ!czF{}Hrilib0<4tP8x~ER zum|lYG&2N8&LDE^j&!*z#;SxQuhvQE_WA*gVCL~l#W@ApLHDD17}) zGLo3{y8A16GtGZ*uhs#*>;9Q(*G^*@_1beIELgvr9rF=5myi~E^eAaj5O1s^{c z6{|KG$<|9wJe-}R{?kn{-CJBY%`I3(_Fl7VT3pau&&H+qbl@GoX#KrU=QL+NuU9jb z(sH05ynMq0MfJZCz1J0;-TVeFl^xTTAafh8wH@^eT{cSjPeuYuok)`ETPAZRE~b0n z_;znLN#;9=Grz#M1Oh1~R!m8=LE&`64WPcgdAE!_G%jJ8nDBJRTFz&j7O!{u;?Jo& z#&XtM21sYH$05Lr2g+MuRSh+&*6mB)v6+H?~T)pp1Tcmpr2N#-$`gWU>x_Cr- zWhnAWBSHQ04)$FBNEgPeW$!h>hpWgR*-$#w7)316sBd*z_T0_M3$DGa^U+i>f_kwE zjaW)#fFLDgax{V}KE`QvG9XecS1r^0exZPG{0|e($8tD|ssxudd#Ch-bN$cN&NZ-4 zb(*?=B9F`(&(fvGMnSYA17EFXZ@&fdGuvjFz07L1eVW)Op~<9w@l+d*|%41xDY5N`G zX4g>y-do_OGP3rip>M}z_PC7tq)f)7wzGwXb{O(12^|gSXmfQE-ex7R*sWMSSPm#& z?d{^}#{?&K#4^a0Q@qYQs8YDT_3jZ4ZD=mgfR4`J&kQ1Fcq8t%_F_ZWtmV})_ovm! zK>i$ihR?57SELRYTqVuC^x8!LR4+azoQBIo`W>rZVg!Me(tKod{POo|*JLW5?V)NR zK~0POFBhQQ&!S#X-$YOln;2`{)Xm|^I&sO-2PYMYqMYDu@=9F54foTaco>kMKZbB8M*EogftQNl#X zHpEeF50`4JcdLY?jOZ?`M@wyHYOG(`j7cKTu`bX58eq}(k$+WBl3)+#3TQ3FO2_W~ zDPegb{(CY!vyk3n{Fj+DT+U;1^O7V1)pzO+>c!_HE~HLNYV%xvu^x`;xJIxHD=&NQ zmE;zRY7%r3)iC4IfavCVInnjb41?T3TR&Y@{`n${f{!RG)uxhMdzVg67M{|k^pp0y zUb2pCC%cc7l#)J14wF59^5c8%QU6rs7Z_^AN}Dw~1c;D}lu;T(=?RDRr+$c`1mqN^ zS>=>-kIKk&Jcw7$Mn*cgP}t{KJljGx#!$WS)9s&*=<>DX&3ds{5>OWi z>efEur#!XGWys|kdXVZ@{A}C9gml4n8YMS<;Z>{)3HFgfPAm~gdS6#5 z*n8JTFW${K@de^6wV}?~n-j>Z=Ok)rCrq{u?pN&H-s4%5gD3vy`R%-jyXq>W7A?3a z?;**>z0Q`GIdI3uqehI~GpDe5rq)Z;ImUY5M8OU(KXV*@Mu+!T-H-;|y7YAF^|;c_ zN&6lCXX+a7TwG!>mFq%ogFlT?8iUIB`T!cm93W^i>^$(ZEb*H>D4N;Ul|Lg){lXfp zK-hy4h=`BctvR3W;2?gRuRx(xl2+s04c#A*X&Wj*;^)j}I{1TZsPH3k^FB9q{r!OQ z)|Un*C(ros!15}Og0r!e)kOF~w z9yGiCjbG>^sQK*j$*%xKq#_6}J=T~0Z}gP#Ked7|j3`p7zS zM>rvkI97$wUqx^q>*nwl)z2+cRvBaad3w=ZqMIY#orgO+BV|cy)FL~iP0C%drz(9> zu`hIDahuHrrWi*e{qfe1U zCX8y^43cOHQ&2g4vLmAemO03Zu6nvLpmLa=FOI)q=E9V$H2t2~C9ihK+dp&Nhl|w< z>*)K!Ac(&$hja)pttAdP_Iquly8ia;l>F^t=e2Rm)zd3AN9aa(M>D0vUMvMiUw=B> zxVg#6OJ6V)HzlemQduk~s;2Yl_JTb$ERVI=_m-D^kkDfME!NX*MZVQ7L4W?h@#nRv zMw}1R{H1)wRyl&kZx8Qrii!+`cCQ0N2Oz_jw!Ls3rUo?z?qb^76D%qh#_L{AJF!ZU zR~K(nW|m%lR!U0Sj1(ptRkUqo8hks;N)dbt6-N8#E=K#lvr$fGwOeo{2Ao0SK4T8w zPHxu`^+{XOws>k>hDjOG39zR*im;{PcfPho;XK~y`5qa`k7_avSpL0rb9+ooxy zKBIy0&NmN3E2!gDryTy$rn#f}Y@^tn~0)(J3(iK$iCXeW`bmOBKm8yl5`o z1E7NAF+vh&_UE-pQSQAN;td7+2YA~E>iPkvLYUO?k0-Z#A(Ys20!Q#d z!JgJ7xNQAe>%6*SG^;FG+2hkJBsxx8d%yMPtWdbkT@e+om-bA%K>ZEpmq)>@v+lg$ zKmI`*4WypS3y0IUgbo!zcDm6h%&(E%ETH6`9t32Xr4W4ziy{i_Qlu?!49*|D_==tJ znDFU@n~JaJa~LzFRoRH!02W2h$7n?mQhW2TSj zd`w|QJTHoMBP7if0|>_P`3R`r+Z-~UZG#+GCz}FPw6Uo!a6gGcs|ySO(Rc(8$Rx#z zz4Q%~=DWFq?WpN@@_y(kW${UoM_b~Pd3EuXM`cPb5|l1N&#KGLx zF}#j5fzMQC@H}(=PmPowTKS6gW&6v=zYb{#TAy{ubpt?lnf`(~ylHpo(dS6-$|I-A z1W|qsu&Ggf9MHMbcf`ad2XH@*{5B8f$XSxsW5MrYHfA{WZE!dR~Nn3JyW0xst2VBd&67fW`Xxi=-bI{(V? zIlA zaKp>^yRZ_$XqxOLNRgqH+n8WrQsk^nmZPWEw8%mLq67O~ehq}3<2;m2jI`Qz2NrF%!N z=a%kMYLr`k2KOgxYpqh{=n#_i}2_8n*5tW-IA=6WT0oklC)ut#`vx2@%SuK!5$$i9r zi?h4*1hg7Qd?3MlPCFiz>=jJ>018=2h)yR zgD!RiG4p^%*0himZOa}2h~HfWoI53O6<&Hjz$%^k_7uy4;&ruY(i{9E;v%C3g-}jNvq1#IyqZJo9;!9JH zp%HJ}O1;RhCt`CafPeN0B-(jxO^NF5>eJWY=AME>m=KQh>W+LifRzGAlf2J@B>7AG zmhftnfXE%n3=|%Q=5;7^)4?DTS@TZH`r})74oc+od7nW&5AYLQ214ZvZV8t|ab3S5 z5q4_KvG{NS;<`^|TKF(YsAQFM}A7l9T+EAsI$S?K>|##y_gNAE?pUNahWuN0|Rfq1=sPhNA$Yl&Zl3(_l2z1>J1l3Ob=l-I3S^AyRDMKN!7r)gL z^g;=kOIfr`L?pTTK-=~Uy(^)dAO>+SQgwCWwgw0sEkrc^-W@BwP&c$|T2hfz-ULmU zy%C?AZvGzd4tN+llXK`&Y8_Cd53+|(<5?}Ao`;p=>B+48ZB{2c*joqO2P*s3uS|#k z5NMxdV~+mY2UDpNx|<#NO*y{m^JolqDh@+U0h+luY;Ho!yjAgPdNx2f91-Sj`+uA) zw7N&Di}Seu2K5_3WszK*E4Z_klfqS_qT|?*6CYM`q5z^#hP%wL%yw&|2yB$du|w`7 z1LHpLOrL^+!7?J7x*{3g4H)Hi`+JL#IfwU!Cqe7N4_~(et=HuNF0H&!BXXXKW8t*D zq3umRS02w0O_Lq!_}x0780K4SaToTBXZx`$_`ahHGB(dzC?dOATsIuoL80^H{lkte zS4&pej$~VRzX@>BXCP=mctA}ggq;lR7K%|Jo77}?54#Ov8~LsF}<&fg>_wvU`q6ZzNDi#8)sv*P;vqS}wJ9Wy_FXi^dE+wMAX zvO2DhMxnKIN6>ln<`(s>+IUB!Q9U|Onc<5($Lh;sk1)i$!#eG_R^sR!(0{r9%ED%g zV3=0VvP#J>`;t|~V{w9H^CMr`6_2ith&)GTjZFZL=MH7Hm)Vpnt$ommOOQ-2C-Rbd z8tbV>qZ6*yE%iPM|JsBjo*!?>{qR7B0SU|yJ5kCZ zc_*QZ-zja=qaOB>{r}!eAmA=w%wLv94PpezqddGV7Vout)hAA%8*jm0GYMH8XLd#( z6k@w&VecFLVE5!|8mlI$QcaxEGyWV@oR6QFnL#SN=LM)PH(}6FG!7+iTOI7(1Aopv zK>s8h0}Tl1#^`W3-2j}9t|8`w`k!5V33jq8ny~7kv*1$$K~_tULL)Wb|1iI#;1owb zj9&&Y7^!Lhr{=3A0h?`(UYsb;_ifVcIi-|_W}{E7LwC29Ja6`h-K`$B4SG|h>bz~b z!FRUjAWCD09C7if*pV|1#}%g67_Wa0=t}L`%JQFx^S3&6^EbTvAC7%$AqYF$;)gbtZAa*&;#QO{=>vyh?s{c`Q13awh6|+4~u!! z+_hl1aYC1Ql-Xtp;a5`L1@H|PRqQ}wrnt)4)aKm~)A$Jg%a^1G~U6~kH1AqFb zP>#$MwZcwuvA%?^pI4KC^a?jJ1CH_4VERo7=F~`-61FOQ$2tkSx8t z+4ZC!0*9;6n6@TA3*s$-?-(7unyEROcD~2kKr_&X4$kZsU@u{1Q>Za|6Sp);Gac6M zfC(nDF@$rogo5xhQ961P`i!cM$=z%2G7LM&U5uCx1|{(ZkO&=0ZZMT}UIHVzi%H}w z1ilL88s1awj&-iy=hL}MbXe5h6LnJS_0nL+oq__u@EtmhGmm@)AfocY!MM*%tNYBJ z81v6eQfvLhNLvvqd;&y29P;>cBll>Z&ws!G^?9kckth^i|f}_37;XvFbJTXO7E?VJQR?m;B=?A+u3k;v9e$0 zio&q8U1NO|_2H`@-}8l1CdCK*M=x@+J#u#82cI8qPDIZ*$|(6mAZ#?g&L$mL(vg-c zL^r*9$MqX>(zJ$<3yr^hZ2X%>#WJLj+IRW}#zVCla$ecfr=E5v?`66V`;t}kWv0st zdz=adL)#ep#kh+@l&pT&A%PiF+K#r5ia3nqtd@_3Kz7j22v_AZUm*jyZXgiyd||ZG z^@&Huoo%v52j=3&B=~YxHFm{KWeJr}^cxjvQppMv?EXU|EvAQcmu)+4m|Z9#tZN$l zAYUw@LnCPSegblZCtug+fc*xEyxcEE*!EEZR_FsswvYlQ1qtvaXQAtxyRL^q3X~qh z`wS2KYKcs$K~t+&K!W%LA+Ec|^x(yMR*7ZTjMz8)!05jM;rOi8@>i!mH8r%la5@>o zpiix6?bMaHH|ieFPv5#Jw1M8*F2)V;JYmZa7=bU#^=oGL^_t`F=g=wJuIMxD06+XK zB9aYKGnf3H+{`ax%=Uc+!R~orMeqt8h0;;K-%G>OB#{&Jw_B=P<&R&~64U^FIu{>@ z?u!-<^$vv!eP(JuPJ>eRX2#(0*?rT}L$DDo-Dx)8cWE4f`z|1<66e8l&K95ATeh{+ zTRzoX*NJzxj_jg~y|%sdvJT`1hU#Yc0(hhuf^(u*D(wC$Z1a;YLXi5SCmWvDaVK7F^9$Vw}%v5kG?wleeRtoSP$F|vi;xlIH+xI2iM;)p$TG9&%R#o z8G4pk+lOMvR%x=D9gJ3?!8d|t-nKu$Dxb`bEwRtv*QE&!mTndRPY#FCEnqPzA|-_n#Iosfl$@Tk)mFAK9~=3Yro()s zy#ZqGts5U3d5wxb=-H$7oR=S)K&a|NT-6pe`c9FuXm zz*q5$&4J|Lo7KBBfj(TWjdra-e|^PsMhjBH*}_F zlRiUmlWiE~hV~~6K^tx1&%hWar8={=-MWT}TVBOlq#v^j6l$8^5~u5C{dmL*epO)@|68Uztyoj7bHzo-YFkWpn8L`ajWo!o?{Rvs4>(cPFd9~K-8VbmAAcHgAc&808%660QOmdKlG@Qy1nTfaAz zq;(J@N;o#Gk`*muUr2;zq+es`l6S!bY6+IReTYdHJ3#(*{Th_YM?H>Uj@NgE`tyhO zf5dT&vl=9kb?du|S!kK^n1xI)Sj#nEXKggz#~-2{5-YbEbwWkserGRDu8yj-Y8@mo zhh!tGR!yAbagJz%ddm9@?i4qhcM4ZJ*E%bE-1#Tv7Mt4^8GK1K&iU=}^E8Yc)sUl| zYbfIU(JV=tf!SGNA0b3|C@)#aGxd&>sO=(NQ7TW}&h^!kNw2jkV5Kt%=3Umm=N+WU zwf6Ikf3T1Y1TtCMHav`gF8dMCW%b?YpF`G2o#6FRlMle>lRSigE+X&f{~Q`c8XN8h znfn6hB8iaS!cV-#|2aeeyhkAY>1iFvit>OPIXnPyArgiDPfYj)L2xn`{SKc)DiGHY zHv95M!#P=0jW(RNq(Fp?I*KLSs!95leFo%2S0Mu4KF}kkna5Wpagk;D^quaj3XC`U zyJcR|s6DGr64^@o9N$#E^7UDN=r+<3C6;gerE#&vY~%dAdG7B~hP75kxt-y!-89W< z*$DG*>0S16tK}on_YWK0%U}CFaPPT)HA)plcG-Rl7Q^o2h>J}lx7X?_B9Kmz@ubI1IFUlgMzEh@ZJxXpweZK8%D0=(V zVL}em@*<8&qpDg7fe?-7%iY;01IlKt^zSpy(=TqB zx7II@*NyRi=aL?^>>e^@ZD|bIK35^Xn`|MJ-MsRl`0hoa0v3EQ?DSWD0h+`IoKYV1 zlV=bsokx6oeEaJLAoAm~b&HNO-%s)@IL9I~5*G_Q)}jj^BrisNQ3r{z zhZo;}5kXeuv6WUeiZf@&6@y<}WgXJa%j=_>fNk5Y?QA(~JgfhH>mo%IkEJIV9-^jZ zpigt7^Q{3yNM>*lwm-g1!~MG2VyxRI^3Ag5J%FoYY@#c~&B|>q1qkr-+G|uQJ4ClG=_q?-iTGS<6c~?iTbL`o~g%?#Huci}7 zG37X$x=o5AuD?|jKk74!edQeVN~$3JK@ePY0#$}PKvy1T#KsQBjTz2I&ZIoLtS+>V z->=gGxWR6P}%V0E{nLb+X3gPJM{k z&HK$Md)_TtguVe=NP`!gq>^C0fall=NZ1VC+HF?}(W?2(OsDS*Z1uX%IJhlX% z13aP-A?(*}R^v7({JMP$3?ym@)O7a-^wG%~X-RXCeSW*AboH_THj*jBi|6oV=b|~B zN~~3A%!wetBbP+_Aw+xT>Bv6WVGz59O^#KJd=EgKhcKP_5JV8}DEgMDY6%+`Q18@6 z2@DdsjzU0nkj<^Wb?Z63Q$&)aHlN&CFaRkT50(tMuoaeC{w`<$+Z3z)F%)ES(lran zEgkYxPB*?~xPJT%<&rlaAyFT`htv6H@idC0rJFi>TgsPV^O32uml}l>VOqj2sv{qb zx+-*AqLp*?7L_GnCF#HRtz-k4Al!N*kWiB|7GIV6-m!lRrdj0v9CBal<@~MIc3Yux2fc5fKT|naHWTa zT@LS87rmPx7~`BfSEL?@(BvZdfpnGDk({>uKxDd7L<$=sHgqDxLE|d{5**-wJg0CC+B#pMgm_n zaUYDa$t_NEYLs?93l?=}8Yu{5vjTgt8}Mk(cTe}#sLMoQ_c3G#>N4adCW$R+%1=2= z>u}m)T|66pqy+&Vj?ZB~*WTxBQ zoGv=$x4&0jP@dWG$JzRzd}Gq|5u_*Cop7=i=6Ke~jiu!m3?rnA=t?FaqXn>O_u-i~9?|xJ3(Z;bNGWVKc<`Nn(z-?d_?b!WsjP^WczM~_l*}i!`@X0X zm6^_TjusVUUtWBS}ro_AG>s-QzjX;Q>MwOlMZe1_dn-pNBV(yLTMwH z%qNz9HJWGYSHDjl3%Ip9vb|7Njcbd_i0uv^c+|j-(B1gw4Am!RI_KpQ<9+*mFu@}& z=r+-R5{!hicGaEWbj0gm6VTh%c82S9Z>oGEwrX85uRF$d>q`6n%GFN8Lj?0e%aWg7 zo=k2ovTiIPX0c`tx|bg{C;bQx#3x~0OuXHljZTxM^}51B;N|XhBYo_B|2VJ2s=U?Z z85bSsFPn87yzR#_Ay?nU)xQqnC7f}(_UtCUC$-2&30gp`7U z3eq8+5`rR%NHc_l)BplQzsJ`9y`KBI-}m#oo)6Dj*J8;v!JfVMoab@;>Y(7vj*6pu zc|4Rr)f-1OeXzIfiNp_(_$qm$FE$ZY zxlpnmoGbp=cL62G^@|KLFdFW;*IKRH0`>66;J0oqwj42(0VmtU*{}IFagt?THqc3V z`{qW@qXQ;(B32onapliFgpD2X5kJh*Sr;=GDDmZCc-JYRlvdt=G@YN>ybyCYRg!SmF~Dr~WiP14n(0@*@)`@YE;F3wA>%vcyPaT%(y< z9+TbyXuCB_5GagGy`sJM{4=*yjyDtuwo6#}$`}EH?xmP^V}3e?S>@|4-ZlM1G-vx2 zaoThu1RyT?f&mfK9AQ%EZw!Upbef!1ofoILOnF?zc}Y5q9ymDfoqkgYwa@~h9MP5} z&19{tZ*A2&E6Zw=>l=BSr=4XosMC&s$J8%3feP)bfd0|^J?3ZDJzE*=wK{dV%v8MD z)H+EYn@qb;LuAqH9gOJCT^c)Wa@on_lTi+-QcXs2INKZM1;J}UIvt7En!C3FS9y+Z zUW^i&J&X`;5rIBMOJ|D2cYp#H&PXm-Cl^he$v};nUZb;m%67TlPA+LM_6J=XKSwmf zy*6@rSLapl@s&|OrDBRYEylcES*YWj^{8!$v!%@oy#2Y70Bq39&{NWgHmW0 z20j&;@JzI78<$$vMD~+-fvJ*IEz5#F;y)78yf=baouh>~11WQu`;&J!_;jF4qLgZ> zej})KGAQoKNqT%Jy+uF%_*8-)<$2t+tXVV_X_?;N&giV@HTC~=MiU%H9)pePo!*Hid3^KpNf)cOo*sqSwHIt7J73{*VK`vq*}>Rt=?`I7xgR9uuK9$X*Aiu zu|DT$Vw7AgMf=I9om43cB9v_mmOUsdI+16Nz zbV=>pz_q7S?;8-(L7dqlmZ3erjhI;R(L4`|W4z zpJo+guBMRNY#vlVeZ0`0VNX33JeTHkQ@t^tVr{LhrHqD@Foq&`!V3VKx#9AR_bVKp z&CP*qr%BY>&e71d?FNB3M9)Y_6SuxDp%i`GyNzA<@A5BV6=-g&M;z2-xzK`nLbgX8 zF-pujO_mc8_6RP6k?_Tq4{XsStqH}{ztsyQSp3_m4k8JzYD$G!85}8ek`YF#<}*Y; zRXHv<*M5d`{}7loiIT1!N|*VFZOmqKpkhD!tG5>yk7HNt>?E@q)XKwp*{Db#y?8C; z+AJEroh-)ww-wuwHS2jIT$oA~X?Ik%rT?>So zp7U>2S&S!6^ku-u+d>Lnw!#-MabJ&QoO_F4f}N zEKO~dwKnO=xk2VMT^~Z$dWHRR*vr>e2)~zrP?+K#p}+b564>!)07id1Sx}46n*hc8 z?+z}0`PXILLriEV=4)OcIJGK2u(2azk&xIhQV7E{wW`ZzX(LWIlMxQ5{@Jov7pY&7Hs`9 z(K&!Bz>G)Ox4^ECLi z--%!5*g;P7?A(oPh1(bin;iu!6n~Hz+nvi-P$~%1bcPr%u_7;sUgFeQdh!3CPPW~o z|L$aSf(ZB(kU|IxA3{x81BB2}Y(0oxMiCvs5pdhDBiH`BmF-f3DGUmdfI)pfK~%QK zZ0S8-=Y=rag;0U6fceW4Mr|~CSmhTJW?eRQ|8aW-^0nPSvQt%I(U+}itDFfQo@jj#E9>4cc?j zBqT9u4=R^2Xm;P5fDNt+k@I~ncq(8rYyw323Iv{1?6EI_B;o#I5ggdikrqURE(2Y> zwQAt5^`{w;P}oDPB;r{_IQk`1BrW%%VX5`#wIuIfwEDI=*?;H+9$sT~t< z@2qqzbZGQ1>~D({SPdi^t*qaA_?lO1_)AFRul)^^F2P1O?uPwuj}0c&9wrJOTIB&{BeRIWA@{?XO076Bxadctq6QtaN8xEwC9 zJKyN%u)+NSK7CuPg8uqqk%{T%Z>NnP`nCM}#uknFI{8T>9$ys&iuCyPy{A;FHaj`T za>2If1?HSGIghcszUfhhSs$?OfoK&3V2THH34Qs$TxD&SvbW(ZRW?NP;;244*R4IW zhy3qFFyrN$*4L1Dg-*NwG_eczhK9xshADc2xF!V{jl4Y|But=j9))k2!<+>w`i$5o zmHKTJfPF^lcYo(=Y+wBa|hs4ddAK#XS-hB*Qc=jDGv4U{p zPc)eon+HjQ*=!WXRot~?5;g^t2HvffV@>f5qGn(Q4H3eqqZze|Tkr0!b+E$C@xPhQ zV8)2_(K@!{o&61#`|T8)FN=f+PD#yBdcqx*)=Z!O)S$HO_TAxrN5>zZ9IMx_>gPI} z>02(}sq*jR(7!)?Nx#ruzv{J*XH|utkN(!TYbKFm_LVevuiN=GT5zOFRsCdwA7XU# zYTi2ze;}=}m#%Wk7!ffK?A~iUa<%c<_An}J7Y+Oj6mE zN83qgMhCuRnCyT6hh|6LEkpfEb&4vlrejsh5uC<5Rsh#8pjL9J}?#Y~mSy$F6FpW-)!FZs-}K z+-U5G%w3(H-L@&+hhNfDHqT!T_&qv`>^o@(g?dp60#u0C2-5%ptU~=F(^G#fV)7uO z-0;_>>_ls$xYqW2j!~*>H=~<2WS|q>21!cln>qi}b$tN`oBEw)pN~G$z!YHY_Ugn$ z-2iY?r9R`3-@k%P8DU>i)=9`hMNhEKBXUMdweK-UQR;C#I4i_IC^=1hqUWrR(4XN- z$HgyldrK+F{H_51*G~w!=8kyaCua`?AmQI>u}{6WG-3igp^J09o?aF9wGZdARJmCKnx8x(V;LH`T=(hO*dr;|r9)a+5u+nwwB?sahbs!(9-R~@ zx=O2Wxo}f|vWZ9wVkGgS{kz3mreY*Pt^$tBeZz5yJa=fa%`0EQEOv+Gi66w9-hOzGmO3* zaOmp^n9pVZIaPj+%y=dxG1#)ZWfM0D-7UIqBwgnvRnQeol z@p;oxZ@{6|EfMj7OcjUxEfUv3YHtt2Q_Yfm>#X1{cKV#U@}@G ze{`v@u|k(uA4c66-&5+VE2(f~(#sx^^eP=0u6%zH0^I0!L;4g_z7Eh_f52ooV_cH0 zjGbFe*@@1*cH%`*MAbqwF~45J z{$zOVXV|ztD~GqA#{UA7$uolwUmd}j3d_A$%l*SDCT+eTZRze&@B9(`r;AQTpP#dz z@&eTR1e(SB@Toy%&olph7ya2JqtV)t}Ttg%neE^0|~2pn)2Daqrmmo+6G&)_TG zd?|?@hFF5JLUm*LmmKNaumwqmo_i@veD*;g%yoK_u>rtpXgfs-4*Xo(jO<^OMsYX6 zwHK(4pL*Rc-Ft^GA0&iQ_ntDfABPSRWvS^UhHlvg!N=%C)Pg;o&c^ zX*+UP*4t_8&+abr>oKz%-=Edo$aY@%_OWjASIV=Uz13wa+(mtlwD#y7-V*_zPFGF- z(C2UCNg{5PdDQmPtoCGER~*ya)#)JlLrm@U>Pp8PZ=CSuo@0^+ZdE(kQJQVZ#>GvJ zT&U&M#?Lx~-YeBk-2ec$jF##fPm0T}g^mCn!GpV2V0^FaBxa?XoN4PQ<}t(O`L*xb zH+G&HmvY_rLu=op3TlDSCP6-7WF{m|fUE2n^wRlsPgSK#t=yJ2+ga(1he#8US6SCI#W ze-5^5H`^9C-=tkCN!hzuW>-rcMPJLGY)m~&BcIe{7oJ|KZ}MyO(Mh@@v-PrGhrEm! zSldKA#~E*9%O?Eab|pFnIdp+d#xC=-#Mn$_rM=zYRj4&H%wT~Cuw1KkcTEc?-?7ur zItrVo%+lU>(amgaMJO5Uf$5NzT)_c2jstk%H%YNRzhJLSVq_6bcGv%ibGG+&L za@NHR`tj>^gM56OB@5M^3Xa$1325*>OOxU~DUg5h<^%^NG=B|`Aid%m<-08#1^uv{%iJ%WAE4uZQ}*K)Rp$+)2OVC!U-`EOy#Vd4@v>Ki z5$YilsIu~>;?9M?Jt+3d8ZID-R~5A6(1$|6octTve2;&9=nQ;DGa}{xS0CEsfvg5> z|5k$sgjofzm%8P^Pt#p^^&Z4+a|Zpmq0Ve0pGY1lyD~)Yk07~nPCpAXxMQBn`cxjU z?sb+G?ZG6T`4gwk-i^#?A@dn5b8fimK==Uhm>juN`uYu+QM>H4$ZfL*?3eh!epwKQ z$gmiLJcsY`j%;MMSp(Ltn>Ewf& z;Y3z4E6@5xJcvx_yFKb5Kc#zBXOw$*FMRJwR0zLwS6=gA05*u2Z^*Q~vRJIJ_xp{} z%~Rhtxp5)=Y4@YxlKG<`zlCZQc>UVQs~#zlr}0QG2)E`1-P~xS2H)zBRyQf}5D-%8 zP{H-~@>B%pC~#cP0hD;(6pdG}o-hzlTeE+;()o+*gdwP^`1b_6YC2o4WeZY=|0pt> zP&vi;P}=?Xj9!6(g~lhK^RTvoCYmHO%rVIgG4kM*DQ9f%Q>Bk@s#bn7hyFP$n1DW=VFc?S|(9_&h zORW4;6bL4Y$Mycz55RX~Ja1ER1NATsF%o6E%f*A9$^-OFyU9%*W!mDyF!ORm#vgod zSFd0bQf|I($%h^}(dlAys<*MabyX$0%jPr#p-Uy#poI`Gq%f+C8+Qc>emVX}&>~ z`T;r{@2k`wReg@kedD{%P{@VN1&WK?4`>p~cM;u&9}4bLun=;WaAf9Rt^C|@n(E(QnNgB+|G zv`iZ{5+L3YW`+=$zn-nkIN6s{nw}d$a}#YF9H-FD)iC*n?<{fLLnz&q2($2mIF*~Q zo7OWPyKtP7QVZ)U&f>Le1Dp}hs2|BTmb2lD}Q>G$_o4+ew*H^%)V-4qSKrO7(4JN_18YKKh$=krVTNrp&_o~r{*Gj06E8`n(Pl;6D*H>JSUV9|i!K2==;{LOPv_bc1Int8Fo+X&7CnbDb zj;xYobG>%W97hx0tF$+DPELjwmLQie92?+f8(yZu_wYb`XQKqXVec({5UzCc5eCuW zU$kHBs7!8~POOf<=M)+=Ar*e_0LejWal08mIkao#y+hCL2!FBPw)V)ApV^&GhFfO%>^=4POh^ zEfDPuf1%|ojN!%&4Sn<>IoN6IdK6A*ZE$3J%l)x3HeXxhT~yR-kwZqF=tn~QZI`R5 zFWh=#Ub4~ir9`}yKKfW)o#eOf|Iz~NqGzF`90^b-^&LOnwgMqMcD0wwDnshGx+>zA zTDCIb=IjwKO*_4a77|Cx*33S)cW4ff48eJqF83+~cEihW7Bl~$#rKf7MDag0CGaU9Bd#WNxf7_g0IQ8>ggs9* zhxHcXN1B4CnfW@n+x&xs$&s!-C?op!c*C16zb*g0VJd>`e)qYu3GnkdWE0M& z?%(ty>Ny8Hg4!0>yi`8aT*E0AJSNnW+zs2xh%^>pcjbqyEU zEC>PPkFEDehSj&_-+DU*4au`7q$OJCBYm&Pc??>F8`6CP0Usy#_JCGtRudAyB2b4@ zLi))&PNb#UtPm7tI8=;OjhWKa_8DO8+ghdOzPg{!?=aWxe~3t1<~_OAvd0`Km<|t9 z!6&!)JufbV#s}|8QMvc3X%QgAa;|_2M+iW5qF`Ag)+_VUP2YK67% z`lYh>zJL1E5dpO)Lmk5)xSC33x9=HQlhEU>n-Acm6dXhx!tNl+g#&AoV8=#JCZ21# zW+69i6gcJSOy6}4lKpc;u;Pwnp8ZeQ9I|=Gf#~THjE}5cnnEmTS3{Lcxt|PS7g}RF;W_YZo3P4BdbE?0uVW;{z%7`=!#R@ z8lvZ>3MC+N{RDwozYx{XE@P0`fAHUYBEuVqyqB;_gE%+#nAOx*Wr_-6N=I9M_lonw z8JLD`Nb*6F>kb!{qV2}X{AD)Ahl(viWT&mCSRhNS{wIbNSGA4R&ba);w#^hQ2DP)0 zGsbsq6(C*nt|lCIBXhdi0GheQiN?lS@CZE~i06|x0x6!f&JV=ocnz#ZS(*u&H#kTC zA|w_qsBO61HSqec2xiQ4nzG(^33rSml=Ju@=o}m^pC8R}Mqxlv7`_f!NA7bOo~AER z{>#G^{L?%fSvAqD@^W0ARQaoQ8?kAUF4>VrD_z8Cdm91hG1zpC(kN>%`$Cr`>6%U7 z-w{jmUIoXhmbB=vI15*>X?=jLZls!l60?1eI1_l5j#y?B7&WY^bF09h4~jPV6?2V0 z$J)rJ&Q7<{|BimsZXSGblP3y?z@|NGdIN86w1S?fH&n(iaxM6ZomyZ)?jPoMqDKTu zZ@^8#vrZ)qz3R5PS`Z^@MYUc?uC$Lc{ce`^cP~-fp5kIj@*5&Z88x2d^?XihgtV8t zC4IXIuD3-m7Eze@s^c-^pj(TvYZ+=$cDpNBW)6!!@3GZODfnQ5lNX~&G6gd|GoO8g zCMhTO@~2Gpv!vBvj&ZG+`{6{@$&APd&}(3&PR*m&rp}RF3Y&=wWjAx`> z%xS!)^D01tvzr|kA9XNNX|i8plMj;jTrrNC0H`JC5WG{~xeO6#FMX`8V3*TG z?P5pZ-urAm4`$0f_~x6`dv6xVV(=*%S5zZTv6HkBg-?*Jgi?c+3q92yejX8ffzCmR z!2#{N8>{3)wJP-L5Hq%Rs5|d!03$>&Nd#$m`)ktZp$G2hq1Q~1g;E}XL@Dw!3@57#&RP~%-(jD zXWUEjGFqzr;lBpc4E%i8EZ0p^o9Qk|SUM&wV(35RYE z(1-Hp!?%6gumgK1&wUb!CZ@=_%oN|;-nEvU;6e329OH;RH|_e|QOd&(Wvw@{^mve4 ztu&&JWxB|%x21-YvB`upFuw%@=M<$==+F4zwN_2S>`-8@N%dQ(&V^D^Rt9<|7L${s zWIlxQe?75#lKuKTMS!T4;>*TJ;r5WY8;{skN5zk~2Kvusjz$g9MJqn*;cbdO!*2hF zTaD$!MM74q*tJBhupzz{RQ-*|O?GzYmAMF@Hn`YQo3KTPo%i$52nkC{GIpTFr<`%+9ndFO5 zma)Y2Q4R2YF@csdtP4xf4x&XN9`H(>ekR9iXnPWjP2?EnNFR0cQqe0gcx1V?hr^(5 z50gYia8{Xtkkt<*u&FjPMes@s79T=}lAdz;IrCe%0Kqz3a6XK9C$UkojZn5Drl_07 zY^7?Piw(xI4P+deTL=ZGe=IH!2|m_5sQ8MlNw}TTnL)m%n9?$o{RHQi*AzmOcj+^= zWuBqgr*^2c|L{jeQi((;({Wn~eVs67MwOHJr1-<0Mwk8Vz$=QgG}hsfN?@ z6_#f$UkP38s6|gF22X;u)Pb5cZsQm+!6W`v?z2>|FOiPvg+KSZVRy^jAI!dssXB zTG6-T)cDkXrgsPma=YmL_{m_m||gTi!xJb^DbHn*hPOV)4QG z9S+L6nFpPEK}P_?42^Fy)WHa>l6`WnG|uWYl>g|<_;MoXsUF>=r-oCfLl|@#kJKnO zfgdlbZC#V`pJegfGU6@c)Ubvk&7ASpwPzP-tD-vF9&bDTC(|IQz!t!+(v8jnjc<%~ z50RNlRHcyc&|YsVa5j+lNvN(+Uc7a*92+~M#+&;C1^34No){#&34dkDC6*n(Rte4+ zr>-a516eTtBgba`b<=Yd8|(A7;(-ONXSJWO?w$}91EZ&O(oveMkH@?nimt1t(^r%E zN_rc4+|^ksHi9#~EL~qQhj?kD9VNF!HK&?V^kY2N<3p7Lw(eX6?fi@Q2Ci znm`sbTiM^tAAXk0Zf|VDRIv$@E^AXCUs5vHby&x$_9t%kM8kyKZ#zQ2)aNMP0)W8F5JB}+r^D34%>uupVSWXNP+rS!j1sV(ZJelrw9cF%; zx36F&?tSB$iQLZgrK`Cy_|^)ViLF&o-Go^#3Dz!&Y#?m9yxUn8v(Vp2J~y<(4i+qJ zGge%bYS|PU@*&LAB+8K$RHw<=l%7)xhED#AjU-clVQ$bT7*S1hO0tO@1vjM?&Y!wA zj*|CJcq1{O;B%8Z#?Ts(>;n>112H(gS}8)!{vDUeTKv&NQ>NN7YoX9PUbU5td8cKJ zTu2wnvyel=o^v`VPBfUZ96sTrym3yKaMhO-utr@H$v`j*8{{*RRNVt^CoI5iWpwDT zAMZVPL926O7x9l(Y%p=^SE^9uf|hJej{q-7bT$UFwWb?&c54THoKVRQ7;wZ|3AN=3 zlY}T~=b&A%DOurr>HvmJl7|EQlu62pU@;6vtN;1ViEq zJ!_+mFWEae$+|{L_;k!=P&J%F@aLFCI5VHe$m)_3Ie%>vg38%i8$W11z9x`H(}>LL zbr`UQuO*#c2wJ_)C#bhY!*UvoklJb7YonABJ;$)syq*MP_W`GIrP)DqwJ^^c(Jixp zt7w4UIIV%VEnF^FdvG%%Y+!ueGTOYDYFD@7{%udt6c}c`vt}V6FS#aA-FD#-dxv+l zh|Q-#*C?ti$K$Pus@Q_c>0Ya}gN-2hwAsPwxu z*n&imb(EJVx-j7~A$UMOpgk{KwcrU8I{P;s0hB`kO%;_uP!OdX?8u6%lYDns39~lI zFq2U?AsazSd+xEq3T$J&84Y^ZRN3@lU!S|*Yo}AojT3z%dW)8-lK>%(zINTuM$aI_ zO3tz#yWjAaLqc41#)~c$`lyG9Xme$=`p#cKO$tdW%BwYtp$|$&wsM-W0=0kR1(4}7 zXUG8VQbb0u{9)1CgY6#vt3(cx;NL+?vEzoEGpyC*`e;e42X*Ud>{xpgeOQ zJ2B%H!M8-J8Z5_MbJbt)+J9RrU?M>utF<~eqiz9tg>^NzZXxP$%Uo0uoA^sop8a+N zyLtF25FGq~nEGFY0nNe3oe(+-lp7Zvo-sS}HZyb$cam!EfSt$~Kp5sn_T^7Vrc;-` zTIXc~^3E+<3mX7G%_frc|CxP^7XiSMMoBsI5iuWp2c01IHmTl_P~N>BqCYMV)Fi^4 z;Qe<;O{jxFdaa%eNNAs3Cfmj)>O}he?4llmSZx~FzsU(i=)L@+hT%G=2o_RCaadQ6 zmcMjsQ$o-8)dY~Gq}R_bBjQTEBI65_gjt6vpnO0WB>ptJ8Szw*go&@UH{dZit4(^w zyQz#X${mKjjd?G=FNAD+C)in%&Tj)u;Yx4fmS9sNyYMQJ@8Uf8D3*>oEoFSR9I7A; z36Usz!bci$Z#dkw?+)!b;zT)B|4PzH8iD;hB}kp;w#Yln&d-pv-v6xccPoKzuBJ|Ux9|##Zw(pUygS98tsm%zDxOJ z0YLAyjHKoaFst==k3bj$MP^y{^ba}k zKHhRP@gWnCR2cEuFH|0 zS!n+$;oDJ43TQ5R!k3(XOEJ4nHd#tc!?}b@Dm43m_~Mf5sbg( zjH0#%-rhC6+Ou?uT#*roSP8|4<0dG9k6q6loQ2W+kxfZS*cGNQkCx+#(Mp}i5-A7c z0QFZB!CUvm-!-@Pgxl^KQkosbhH|heSt~<=7t6i87pNm-C`SfAbqqGD_5~AnqW$g+ zjMp3~FTd3Lf|K_x{H>WO3`$-iKoOHLh+v;19Ln15nLU#2?l4Ycf8;kgD(eTc-U*=y zft&pPX3$03;S85a9AIgX)2?w_m6S^CWqd((Itw}SXZ0nvO;GD%+g*w zhalv{0ZWNk_vmK~v(a!GGI;;TX<#&PTT?ud)gv`@vs06;FX-0s%fHwXDF`Ugeb7!E z9Nn$XiM*>w;$Pm4gs3DhvK_D`CSN$w%qs0szz-a6#WL_;80&c?VTg{6F#U}>}4qH6|$N^(+To73)B?&vERw7AXOH{4I27ES9rsvhCyyo3%0fv-vf4cY@|QvS+%I zLf4A|Q6;xndcSs_*fkSxRO~O!N=i($P5tFbh<>GPEf1nAOEP)^H-3+I1}~b6^oueV zA3Hc8(QqEy>zRw7=7#Qf`n8@n;eL6rd~;*RkmSU`pTJ5dVzN^?tj5 zT(Wy*JELLxD_#bQ3R$fA@~Bhg5|CFmk=~8>-d?FEYqpyfP7^W=%RX32 zqG=9CaZH<25HQ&qx?3K9tSO`@U;tU1T%`%(HCnN*+ze_BP-2Qrv|W2Z>)0if#Yo@V z1MJYk^u;e~Y~2lH{v!I2IK(_e)RsvpvUBA#+ZHOffaiQj7A7tV%gdr$aSj!#@Q|JI zaxg`mucFe^uOF1!CEthk2&bK8kE3)%i=)C5Z?WCKqCUl#`V|R;(Y6R$a~noFc) zx7Am1{o`4-b68I_#ymb?I4p&j$vYuVkx&>d*_QPatq`9`B~<#b>HLi)sh3%IXN1ME zrG(G?D3t!lOx9yfYTWKkFewzLIc%hAE#o=8#rSc`id!N}3v;McItJI^(~Jk^v6wKEz8W(Uzw3tY#S!vTv2f$477(};v-7EaM+Vx&O%n2hQQ4FRWq?lcf?u$)G8PbEm?oBA{y8n#)~GVYeNnI3`_G`0-=33TP1-vf_IM@a28u5eyTu+4;sUI? zRKq!v!yol!iT|Yq05P*QRD?TB%@j@nP*)Fu6~&XOIqqkc?2e4Jjg4U>HTQad7$`xy zGGQt&)xONj*AgTI?8lIMn)vTM9rTsPS7hC~9kywA#L%#u9VJkTs1F4o0dJeX2pQ6H zD2+lUrc^ACRE)WbBJzXPMdg)@+=#pDB_6P~F?JKXp`YshFPL{D|<@;n=BE6}4d z5lt}Y=XNa;gS;GbpKR_DR+nK=W z2#jRxFQk6~2&2{y2w%BEfRnmGopssF@VrS)`6h7c$PT(4GH3=9bytdjhToeog&s(H zy_56csATufS7L;2tAfy|L13YlVUyeNL4wlsO0Bz6w7z5*4X@N!6_|LM)W~jVGLF>z zSbCTL4W@x>j+C^*XD;3Po&S0X*IMQPO5DN4qSfcktAoO=14V-O76+aci+<@W&o8i*raB7-_JdH=Bblg zg?`bxUe(Gea4X$Z8_5bV{noHw`U*yKiNob%77y^hmnQN$`rwZ)Nxr^w_?R0_K<~P+ zfRWxO7T@A%zYQ&o{5xXZHSQx$CNA~Jupj+U zQV$KAKeJ*J*re0@iBsp12qsf#z(Q?sb}%!IfcRjj&a12f+AK(`_`RMDu7aTuvK7yz zl8#|S&$BxHylHBaH;>pH`pW^x#r8Qm*v@cJZb#-1fr;DXB`)aA1OzX6Kp@EAV2Kr{ zBF!MxB@p;FcR+FzPe3nHm{?n(7yxtrG9dzEZ|Q;KZ7TZe0o;4 z)XwxhV_x9StxxUIL$)2PSqq`h%_pa#L<@X(_?RRL3)~lzEc8uNoqFZXhTCKL?e$Ho z$G(04%%d$})%wVB<3+aT>bjZ(=|cFeIF|a)YSHKH>%ASTtm4Kj4)&6(KzP3tOeP{D z#IBGc>0%$+_Oqz5RiFV=TfcGryw7CVx5G083dsVrOWPCI@b-+1wsEq0?MD-}+gm9I zJBNGJ13Bu)?T0UYukrX?ytl%MzF<*i`+8`&g4=2Q%ELzznI2 z@2D-GVOw$EW(YU=Ybi$J>ar59t5g}+RldETJ1+3SycX~Q4_-X&McW}=?PabB$Kj=4 zTweWe9O0;wGEG}9-2PO`>VM9H-qfI3pxV0jCQbaiO>C~;JDXM)0QMHj`NWCwfbGUkRScJr{VEEZ8;xSwj zSH#-mSe=oW>}z5V%aALEIs){3^L7XlHtoCvKqesW-RW3)*f<2VL3rZFF}8#70<+Ye zZg9z}TqwT^nGP$_p@xc}etn5cl@DkjK4+dfhSqFKrzIlq>NB*n63><&E3MW1(_2B2h6{ej`k=y#e9Kb$X#SVUqAT|`QcBqWVh0KzgkuC zMJ1def#mNu%;MHpq>PHqqf8!ZcuD_ys=4a83<7j`XZF_@re#P|D>u zx%26@JnCd;sDZn=%?sb13{=2`Us?K_)wcZLYhh?p%gykuOrOGXt*lJj3j3p@z#wt8 zSe1sSJ^@oZx72xj9SBK509gjaMS-XBLKr)lnv&kQ7n!JzdUR+_*o)vgx>7wpQSxjddI46fsxea-WaTzi>lY&?|f=;t9HGSauO2F z9T_U$c47DXjA=p^X}60Sbcy0_=O_IlKr?zDIu7czxF2cD8MX(zDB{=;MC}e`L!fyY zr6B1$*mmB0c3kg*ULoH-ek0r6NTrfpn%*pL6STFu0?N!)>2GaKc_*0lY@$YPn%0ZE zwIzX+aV)aK07Db=#>RJ>C-Q%zDE;n$vsUJ1av(Z?Ds&52ia%RlH#uo1YXAZL$d-M@ zCst~f!3S!FgXb{CPGy$j4kp%>Ymc1Dj?=09S27mv!3OKMU0=$62Kp%A0)W$6rD!Jg zAsiIy+3Sd5N}fBMxwJ-tVVkyUk6ue2y0Q;pnU|aMSdV|?rE9j+Mt$RK|73fI>oUql zQH|N_^cjrxEoj{;dU`aswF?^PW?u?8oM1f(m`FIw1*^vWQr(b7UoVZ`?5cZ-7QC4$ z_qIoMzr=M!^WZxlbPnb;hwp2r6#qCQHcXX>HS@mbm0T0eT>E1aYOtqU56$e%MNgFqm(scr<6#GTDDQ{jmQ%WJS&OpsVNMJ1xZ}Dg2Xs0*8I4{vG}X9|LkEL);U6A1f^qsj3r-A%uueB$}Z>ZAQR`ROh2-z;!GVO!}sp&QJ0 zo#h?OH#O7K@dadJMnq!tK}eQU+W|>D`b%&<3QynC`^Z*YzKp#F_Oioq13&)xkUIv! zas7`@R#`U{ec%|oQ#J1O_zQkbTDwnbDuPd5(ROR;o4Ak1^VhGRJy4371?uwV*5=PE zZXz6ZQyrhMoCl1exdq+!g&!)OROk2vi95viv3{W4-!{NkzogoMMnh0DI; zCun@lH)h#ipSO}fGH$8B7YIB!nYyQtt(&8k#BXZ+N$jxo(d&_;eeI-c2@e-3Bq={o zQlj$a+03TK4GIoV_b1l~xKmL!Ynx2i&nU6orbKf$YtzoD#wGHJGhHKc^uAyj-(&Tq zuvvSyfiBmX(zo+~h+?T9+a0O^*$}~5(WW=DpuxNX^BnMzDy|ZhAN$gmgxU5!KUUn^ zT0Su71taDIi7ln7PZtT_s^D@YhGuD5>$TBK*I|3{!A@Mf36P5RwYo6{tk0LC8yyz=poG9u*C?*IMZqdLk!zBiPQe&CL{W{Wbv!SSC z;IV9~+djx+NcY_}OSP%~@3F^#Iq!3X>Bo>ke~m)kE{F!EYW?l62mRppRk;FpUif{B z905B|i%ku-1BK?b#tqfgK>N-XGNkC}i{ZlApstbxFfqijO%$3)%j;%7u3+fUe8@Ko zV(q2eG^cgfiJTZt0VzFQIwpb#Q1-_iZSmlK-PL&<>~cN8{A|b$WQPp{8at+bi#pfb z01D27E1JLQo?g_yE~k+Yxec>$@qKwcLD2E_LUyf8QMgX#x9TIivx%Qvs`~GK*kA8H zC~9_>YHnN056TEQ+%YA02Ij_p-~NV?QK@|z|2fs^$d{f@Yf@74 zW{)LlOBL5Tr4geHE8Zw_p~LG~Ayn-8g`vRb8CYH6XB+)*ukV5wl8JST0mWLXA$;*m zu{;nEOJMKV450LFNRhHL(Gx;R%CkOf_c{ooiWh^-_oxt;Bze`EH$)011Q!UDAD(;h zqr^%bSjpA8N6jz?;vTpyo@kTKKC<+rok`+w=~qHDs|Z74+RE`}rzsH8(|=rI+c_=O z?bIWE{du;BO1bdSo4NRe1Pe%TD1$z+qI0clc_UnEB1fG9ao)U#$uF7a*s^d1i6(Ox zv!FWOc2SKz#1Tkf_;gV$01hRA!+!r%(46)8$3s8f+^|Z1`AZE4=Hg!}yKOYuXT}_>m6Z0F7|uh=;bpt30spB+r>*>`*?e#}~F8=!0XXtj$io8bx(A6M-Yn?e$ ze`}b}sItctDwq$9zHazVSEQY{htXF*-s*NBGU?t~`ez?^_Z73-=SX}A4HT_k6LNfF zIW05~*%;6GA%wjG@ahp@CLX`bf9a%;lJ8&nfrUj(sO&5CKGGbPHT$w>zik8v!7mY< z+COB zrZNoWZ&IU*CvQ?`prm-Z(z+tCqUr{(bDEeQ$^@^&pkF3e@`pWgarb|`Q&mgSJ;C~Z z@1(>i_6dvl&-F~0K|2a_mJ2zQxd!Gae86*L$!!lP|G1$4T$v{0ep`(SO1=d1P7D^U zClo$6_a=fOY+>h^Hct96$eO?RgA@w!rOPAe**=i34Dt#}PpK;x2I? zZMdd$Y1U5XrK=ABv+z?E&l}%|CF?Mtd=cfxFzl}9;k~uQn20U2V0x?n;0y(UJZw0_ z^kK)>(?{h3Qsx{)vVd%zUB8R_3xgG<2 zz!g&1(DxL05EvAFk?PrpgGY(ALK30Gacp#N378E zUUiu%;tPNsXpTa#-Ut@vsc@j2{MTHAy!rkix3yq&*&mCnT;WEAHf1U(0MGHF&BUR334jmhi%`*(GN8OYEr?4*%hqCSeW=xE1W8aDyOV%Q?#@M$kQ$k3H zK_!)xsIko0vXntcmMLkIqT%jNi6%><)KICEG$dM7gJNjn{a)SA^ZUKe^ZxPPf857) z93wT?Tyvh^^Rv#aPz>QL8*QoWde_q#-(EGj43-J`H#gSAK5*fd44QsMhJ{@C7uaoa zudfb~1{_em0s0(5ceDVLm~ZRY3y0l6vW@`<=mtAB!S9*I>M7{$Kwdy z0SjP3`Vr{dxe40!#fo2kLr?Z;@TQ|(nPG}dBg@3wzQ~6_f#rA!l=YF@b{sB^>DEnm z>M^p;fTLl~Yb4$P-t*n+9k43Xuij5Dv_x;(+W7+YKO|6f|chQgf|Z#pSJ_#@tZ2^IRMNP-#_g4 z+aQ^m6OQN!%4%x9rJU8*H?U!hJEiMCw>buyH11>}z1u7!4d|3N(r&9p>BONE3uG{% zxK5bL{{;z9_2Q}0ULOBJjumrC)~bObZCg@i{|I>ft)~z`Z3?HJV7%RC;{O}piaqW z^SIZqpj7Uo?}~a1NQ2nIH}H|$&$~+88O>I$VikfuA!7r|Bmm7_?QbWlh5;EE2%?S@ z7xLj;G^y4nO@4Uyqasse&^xO4gn)s4cg3GX zHfu<0z=K~U$0XWZ+_MEfY#4hFp_ieOYepOa!mH6UF0J6tp91WrY4$?iw5fr?BUh*% zcxFoalc}RqrIaZ{?En?*!gmq^!S-69mJ*E&QMO#N&XqD**n6nc@=7Gkj!VnHEVyi#Lrj`6ToG`oC95lk9sdkWMI2;@CRT`? zAyd≪L8Dt0B=E%&AwZ6lnqcZfYB@lfq%2F?;j@qePKUrSz=^3zHfSOS_@sG$;F& z;JoUSlpUfb?Ks&t3FhYsblS{#@6Dv?2*i=r(L)d;*}31m^%7>+C^!{*U%#~FZf?GP zybezb11yDt=q5Z*OOtl?!v?yj%}*jm#SL88p)^HDl^C;s?u>nvXJFi>BiQu5`gN8f%{~uC9}4R?Z+9EZD4Z6(q;_M#8gu)!CVH=X8m!6D5aCt$%dd=#!5-g{ zI21Lq9%VHbc}j0<4}Ck8Hx`Yq*^7$l5c$Im%j-{=Cd>k?c@xme1&=Tasp-}U(~4gP zRC?dYmUqY&i$(!8m@D8KH$_0RII;)uNAwGB6jQ}Ts78Ny;qmNrO#4JO>~A{(m9m>s zTPOTJPvFc z{@!0zD+A+wEMo2^NN$X2+y?cp9glMsfBV&4I4!rLaV8Q`w#h#aTWBSwj^r=v)f|Sj zzllkPV@LTWWL3TP@}TbV^>$VpYii>tTg)sk8?SmB&3HVyEi}11ziOJirK&(h?s>EB z$vGGtj>g+gVE|YJX9;7|zPBo_yN{qmbW%mY9>6tfCkyCO?=L1#! zDK`)=F=CoQUvCXE>5Hj%+FHE`5#F;A(FkI<={W?*xwgl@dZtL`GDSj%^Kh~M8Zur( z?%=1vN*C@C`Vqi9)In^BU>?Gh%`#+@Vr0N5X@#)2qc>r~AUbG2*>P#pI8vWjhvZ(b zw-vb20bXpIFT$UgmzzKu=l1R34xh|j5GbG6Ex#?PG+^;ZW7_XWze+bLcyp&8{hewModRarG6_ zN*XGE2wb;I7WGOFzbltF z;pTzcbq=Z6o|e=J;e~%@cTlSbf?$#yIuYd(34%-LK`C6|<11CGnK^Y~@H>R#>r@iJ zRCPDvKw{*QBMqeKyVE|w#AD8w_nWx}DSXRcUu@rhct#l!uO#~frC=7b)1cyILt|dZ za6xkSLm^~-VJqnGNBob_82L3J5&r~vziq5Y{fsZfId3SDf6pF7N?2eYEr0L35JjG=~n8E`>1%xubUS!za<2R`? zd|NBJ=me&T8~cu@5y$*3Hcn2QTzG*6x8N1v!7&Go59jD$^mIG;Sj?!2>;~WANwrt%@U50I{ z^@&2RE4O<`|B2#Wvy_<;RaRVZ@R^wYq7pyQXdR6I^DN=o-1q7SG<=LJI;Rs!j9&Hr zHbqEeDekZ^mHk=LKuF11@rpu6Bcz|3@*3vQh-!qc0}7^TyANfc-%x+8Be3&=z`_7@ znj#QcvhaExY8VXD57KP~m3%d|oEk!Q_SrJ#8v-c_E%K*0->)o)??d9tJp!=((Okx**y7EAk^#6u$%MXTK8-cPjH3X@I>}MZv$Q2@HP*ZL7b@Ct4CTx`)FT} zsy;~T%}E!@L>3_c5{3tyTh5~GUVl(e<(+$Un3hWG^_0)x6oPM9!7z~X8kzrWOnyL8 z#w*cXv?3TXQmX4jRFLPQrzCcudy+)c<9psyjJts$1f?*>s$LXVX6ge$+i2&YZenU0 z2LcZt7y%~-XNs9IS!fSQo#t@9^UC`K>J&)FI&>yXZy(ay!a8``vwQHBTR#|EFu6IL zl7jGcO{lIIxl1c{l4h29@1@|xbKoW>RXV+X4YF+zr+5b4`^<(LSz?N}6*l@C z_H!U(h<&&lii0aftSM5sG19=mjgq&SJYhrvihpKblM$_K=T%|S?U8%{y~hDV`0dc~ zWMTwlLY4!QYNVclg~+{`TDuu?Ku3VT_O0$ssP7{)7~d(ZpY2hs?<&oeXLoX=5&GAN7x|0TxznPv%Ill=46i* z3$#gi4|1|*iT&RayR|vE=7%Y)=knvLmXdyZPP{x%9H1OvY>~Yp8Ay=|^8E1ttHNlh znsYS9vcpJ(Ac^n`*~jd&cZqZPYlXprt!`U;^*YhG8KWp zcedjeA|$bL+>K}(W0u`l$G!}SyQ~K|iV}L&7&{h?WA$Q`f~+{Fbmyx2i5q5=R1XAE z9=40%kJfNQkyy5t?-ICI-ZvNeE++D&@19ZMox>edp8JtN^%5(va>y%4ZOUVfh>ly4 zQ*W4bk4X7oAGRNfAkN!~&nq&eG8D~(VVR6THXft$f1s0eEK33zswDnDLzOeP^V`l7 z#D9nj=YQ0E?s^Cr|M0C!156GupXp{aTH#=4dj-sm)SVnN0{l*xGXE_VJmm-@#^T<0R_4Y!@4Zq^c99Wi9db3g z6H^bYQuhzbpt`%*(!h#2$S#@mI(9OfKi{z&2Ns5PpWoE~rs3@eN8tFLWEQ?BP|o6g z;MSAe3(+XI8sE13U}38vAt@Wq&yBTg=Hw#Ck{1R#2IzJzZcK~MYd5!^zKvl@*A5xD zC|dO8)pmd)ms0;4d9o@1#A0hQiJqPbzU*?-$srw0<~!gMYM#u-pv*y62X(LwZymW; zEc!tmW;EvHu>MIDf*WXd_Y{AkYz`|{q!bj0H;QtBqg8S_m#4{`?E3}I!~2$R=%~== zy`XZbUGCdAfTq_TH3(>Eg7ww<)1^5ld+rrM=B(FY(r5S02^q)0N8MQ|$6WqZrNAVP zOorGZ(D!=eN%lbOOohS^NrtNho)<6mpe#(4y}7{#bmdni=Dqb@Yl=GFL5OUn(1esYP%L` z2Pm+5rU@*GijDOreJ|&0>>$lgK!j^no9<0#re;C?9nvJ;3B><%5ms$#4Z+St^_9+R zcii5v-c&y)0H@FS3FGUptA3%j0MW+*!4yg^@IQ4{<10{9WNmF-LzfGId?f1INDT_h zjg8i2OYhopc{ea4?JC^DG=8VSBLYB5K2%Kjy;a-6Pq6KwL~4x7C=6mhBX<1|EOxmN z88o9|FTZ<2psRGU)>4L?S25j&q^?meHi72)@f1Yr&oA^%IvUd#hR;Qqvw5a$oA-mK zM8uUibP>KdW(io>?* z31lt4LO2SC;UEy}Rkb)AT0kL~)t|A{Xsf$XSfU?y{O1(91CR!viwhgW7_^xB>oYFM zsc^CUU@B$_pR;PrK}0JJ0=+{=&2i(j!6PHd4-Bdt_qFQ${;4MXdX zvl;VHJU0}hzHzFlvyGt_7K}uIA!x?cTZs^4*{Ge4yTi8aMLXhIF^59+9i`PwkFdY?$pK@BeKia}ry8IB=c$v0pYx+=Cx%oeg` z2Dum-)#)E?0ukiI4#>O%nn!?h|x96B00NsX2tx7RE;I`?W5*|mZ#A!l0pJ!M%&MDVWh)P@|6WK;P zVqCXOdfU4cNt$Qst1)#{SGGI!$a?LT8dK*P9wwmj9=KO@pzCeaX(IqO1zBm_i(V3@ z7|m9&Gwe&) z7%xV7?tp7StD8Jl97p08mA`?z$J&9DZ-l#P809p@kR>nqWGl6b4<#}}nc4lrm{jh3 z_-CBpN_UGOx0YrY&*#0&yX2|8)>e3yQm_Pg z)!j!066RvF+Sx^SbGitCPP_nBnhv8ue`+0-of2qepP{a#++*)1F&bWi8*&wYDz$)j z*NjE_#9tb~0Pu914rUk5wV|boJk#5gy>svXLUCSeDlagNjLA>*Pmrv8p{2emFU=wQ z_AOa*jSEU4K-t3)t-6?ShOLDklO|8tet+MZ24<`j?Si`dP!%BG%V~D(p2YPuvu73f zHDooTMYUXxqHWLZ4CI`vKeCQq;o)-%+jp=p>qpN%{sORlqIonWa%pzRv8>^IRyjROv3-P-Anc~x*+ zPVlaYAj(&pBvv5U;m}pI#4V8*E9lcvi8SG%qtL#)=GUWy_)lzG6|b(H+CkH{Lrpbv zth7G%aO2@H%U_rn((~!blY444mtx;18Z7i^H$gV3N0S%s#1-28z&a&jW_oUKD@PUUr)K=RbgyADjUrFx4*5XCc|F69S+aG^DDfY zNb|jhn!87Bt4=Y?3Fz`vr-K78G-Kd^guh3N`tdu&X*2U(D3ndJRUA(h^__K!nO#2= zZ}O`_OD@a7vR_40H6qeDv+8kEflx*|cFNut$3c@wW1^{nJDF+di14nzcNlD2PiaR3F;x8HQir(U0DqsdQoH*yBNKJ?D0-wXW@61^Qn82@2D5h|L_?Y)-N(?$O0Qze2cRwe z^PL3@$bFG4!fY_wZgRnpO6Rj%k0REbQ7mRH7=pC5cEAm{=i+rg&WJ@eG&EhY#__Ut zd%lGE4+{-r+|DoCPj~^USB*S^B9(hTd>nVfZOKY<=9avB33uFG7=EaPDfSup#=0Pi z-7U3OZ_Z}IMNRnAN(8{f0;DL4@_mdDi!I3Bp|kes=hMi84RV_m@S?}4@L@ztrvY~4 z4h+-(8N7h}Fd1PX>BQ%nFr&&q`%7xXv|sGMUK>YVYt49W3$Lw2z{qoWKV(vcKeriV z5lkbW5#KO4j_G1h=d^iM=bvn;sM@&tkuKrz6RXLj(r0Z-F-=={myUV6B<~}KF8ZFH zAD5@!AJ$w9uKzq%h!iB}ABaB{1(5@?gKe)xtx$56=UK{!C3SrgVOv88uq}Jtyay~k z6WTzpa;Jc@C@I(ael|Dq1AtG4aTI^|0#+f=lG=YbbxF0Yl`>U6J{bmZ&n!SBp+>oATwnM)~! z=8;8EoeUZZX+E1rm|7sR8X%O|98w+FAEQX5B<)u~(*H7}r_5Ev@Tl>{>o4_h18=3Q zXgw3@EF9TuNY0DMJ4mpVk3u}b`)l;?%l!o|$kKUe6bwEo>c8hom&aguPv&pn2VdMd zvcJ(?94Vn|hP=36vA_`@34t2y@vDL)n`AqG8>Xn2`mw3?X+~*aWti9KcyRxcuJPu| z6>jN!uKPJ&M49y@B)#5!DcOH~EP8ElOVi@dFWUhVL`qoRWN}Ic=#ZHFxXNmFeMdSs*eV}fmWIH^g%#J~xjvhO9N!AXkRA>8Oca~b%=fN1!br#NXRRAo z%$5AW>0RgK41ua7q#eYn7la~-)B0hOa6eDg+Ce*T_4ASwZPPuf)h$keMXT-i;A;_p zZ2@r$T=OVdGm;h{j$|++Nj6un-+q`l^=fonUemw_+ZhOw2FtV)MyVw6ww|9z!0sbu zCE0O&(=b;r?~`YTvcYc*jHZ$a!=mk$B6M7ScwYOn82zV6MxKztaX!F5e<*pFyPsv4 zqAZSMnL-pn14A^4Mjp2VlWQnfZ{@1z(Gl2mr4E?c?(|J|wY2>TKDTW1Ye1QgXzCFT zPbviS3h1~7n_pP6jF=XV_R`@SNWde}^$b3u5lHJhlm#n5w;1J=CeM(fDKBag22=8n zg7s*1jg>=Ml?C`-WvheZ@9G4=tV5~@(o1hBxtni(+|s$+UMMmP$MTacug={_RoSx!J;vR`-ryUKoceD$FcCk{itOEU=cTyE#8?*pSu z*kOSL*3oo90v@vmkjzFud#>n1W0VR@@;y{)#*+OtxQat5)C*v{HR8!NKCnD|)bs4a zm8)i|fUr<_Lz#5uRYdR`pt!f;e$9L|L-dmI9<)k~rQ;4$yCo9X49_bzj zY^HiETKNf(0lN{)Lx+QNhWA~p|F;KbO2sPf@FaRik_TASkpqUZN9TMrvpPIsi3G2K~o_ti~ELh>I*mVP2{iWq+0Ec>}&ZKj+)eDZWROS7X7 z3HcDn{5;C4FL!JoK;wWw4hy{vm%^gy>DeWBKr{3u9{8iND=(0JkANud2jWF!uqUe` zHFdIAiL{D58o}NJbUry+naVDYZ5on&YcP&wsF5aw=09w!?Yy`dJmQz}pFVzpn%|Em zvgY?0xmd?MxfQvRm7j3@kFi6t>3)}ifblsLi@Ll7J9BY5xC4DyL*lA?*JN^2!Hka~b_|13C$`$P%8BIEwMs+gs}80QqJbVYI=O60E~V|3u%(Tbf3ddx5-LmeuS zn*NN6=}S%(-G$jTRH;Fw2R4HBSAqB!D5mUZ@SuyAD$1~N`o?>jiH<-*x?_`y`kVA* z?|WV3f#+}LC4PzYw7;)JFVjnQRg>U4mar2RlA0#I!=UalJUfhkjaL1!qC&J{61VFW z*sx-!@U>N^GHjy2$}fG*rmlR6t8!Ixub+gLGu(&8?j%u3oZFAm$H~n46^)bG#4X-i z#^no2Br=f+jCMqv%65%znXcN};-I-!G+Cj^BKuzT zLg5B1lGTy2RD$8u;l@dOd5WNZEC=NCf7Op`nMA_n!Sln|bWT&Pw0q&oJUJ3ZSrQ%g zM`ZT@pnM_XJtEA>Lvjpm)_Q6F{j-;1frrJx@@d`v!ad8r9#LY|*AQ#t;l3A{K9|SS zPzbr3USj5qr;Q<(Jaj+z4`F)+SmMe`FSBk6BZ+TOx*^A@%@@A5ekwl zA7|NBO@k!FS-_&?t(BkNLk~MDQoCLJlk!U;-oR9f@uFXCnKz z|$^$Q;SPGtO3CO=009^aLe2yXx>xBeTX)DovOG&E%Xbs_uD5B6r?TNX+Q%~c@` S*V+lde;zJA&et8oss9JAh>QII literal 0 HcmV?d00001 diff --git a/docs/multitenant/usecase03/README.md b/docs/multitenant/usecase03/README.md new file mode 100644 index 00000000..3703a9ff --- /dev/null +++ b/docs/multitenant/usecase03/README.md @@ -0,0 +1,266 @@ + + + +# STEP BY STEP (NAMESPACE SEGREGATION) + +- [STEP BY STEP (NAMESPACE SEGREGATION)](#step-by-step-namespace-segregation) + - [INTRODUCTION](#introduction) + - [GIT CLONE ORACLE DATABASE OPERATOR PROJECT](#git-clone-oracle-database-operator-project) + - [NAMESPACE CREATION](#namespace-creation) + - [WEBHOOK CERTIFICATES](#webhook-certificates) + - [ORACLE DATABASE OPERATOR](#oracle-database-operator) + - [CREATE PDB AND CDB SECRETS](#create-pdb-and-cdb-secrets) + - [CREATE TLS CERTIFICATE](#create-tls-certificate) + - [REST SERVER IMAGE CREATION](#rest-server-image-creation) + - [CDB POD CREATION](#cdb-pod-creation) + - [PDB CREATION](#pdb-creation) + - [MAKEFILE](#makefile) + + +### INTRODUCTION + +> ☞ This folder contains the yaml files required to configure and manage cdb and pdb in different namespaces. The main change here is the possibility to specify the namespace where CDB will be created, this implies the introduction of a new parameter at PDB level in order to specify the CDB namespace. + +Tasks performed in the usecase03 are the same ones of the other usecases with the exception that controller pods cdb pods and pdb crd are running in different namespaces. You must be aware of the fact that secrets must be created in the proper namespaces; cdb secrets go into cdb namespace , pdb secrets go into pdbnamespace while certificate secrets need to be created in every namespace. + + +| yaml file parameters | value | description /ords parameter | +|-------------- |--------------------------- |-------------------------------------------------| +| ☞ cdbNamespace | | Cdb namespace | +| dbserver | or | [--db-hostname][1] | +| dbTnsurl | | [--db-custom-url/db.customURL][dbtnsurl] | +| port | | [--db-port][2] | +| cdbName | | Container Name | +| name | | Ords podname prefix in cdb.yaml | +| name | | pdb resource in pdb.yaml | +| ordsImage | /ords-dboper:latest|My public container registry | +| pdbName | | Pluggable database name | +| servicename | | [--db-servicename][3] | +| sysadmin_user | | [--admin-user][adminuser] | +| sysadmin_pwd | | [--password-stdin][pwdstdin] | +| cdbadmin_user | | [db.cdb.adminUser][1] | +| cdbadmin_pwd | | [db.cdb.adminUser.password][cdbadminpwd] | +| webserver_user| | [https user][http] NOT A DB USER | +| webserver_pwd | | [http user password][http] | +| ords_pwd | | [ORDS_PUBLIC_USER password][public_user] | +| pdbTlsKey | | [standalone.https.cert.key][key] | +| pdbTlsCrt | | [standalone.https.cert][cr] | +| pdbTlsCat | | certificate authority | +| xmlFileName | | path for the unplug and plug operation | +| srcPdbName | | name of the database to be cloned | +| fileNameConversions | | used for database cloning | +| tdeKeystorePath | | [tdeKeystorePath][tdeKeystorePath] | +| tdeExport | | [tdeExport] | +| tdeSecret | | [tdeSecret][tdeSecret] | +| tdePassword | | [tdeSecret][tdeSecret] | + +![generla schema](./NamespaceSegregation.png) + +### GIT CLONE ORACLE DATABASE OPERATOR PROJECT + +```bash +git clone https://github.com/oracle/oracle-database-operator.git +cd oracle-database-operator/docs/multitenant/usecase03 +``` +### NAMESPACE CREATION + +We need first to create two different namespaces (**cdbnamespace**,**pdbnamespace**) using ns_pdb_namespace.yaml and ns_cdb_namespace.yaml + +```bash +kubectl apply -f ns_pdb_namespace.yaml +kubectl apply -f ns_cdb_namespace.yaml +``` + +### WEBHOOK CERTIFICATES +Create cert manager and verify the status + +```bash +kubectl apply -f https://github.com/jetstack/cert-manager/releases/latest/download/cert-manager.yaml +``` + +```bash +kubectl get pods --namespace cert-manager +NAME READY STATUS RESTARTS AGE +cert-manager-75997f4b44-4nf5c 1/1 Running 1 9d +cert-manager-cainjector-769785cd7b-mzfq5 1/1 Running 1 9d +cert-manager-webhook-6bc9944d78-tslrp 1/1 Running 1 9d +``` + +### ORACLE DATABASE OPERATOR + +Create the oracle database operator using oracle-database-operator.yaml +```bash +cd oracle-database-operator +kubectl apply -f oracle-database-operator.yaml +cd - +``` + +[operator creation log](operator_creation_log.txt) +### CREATE PDB AND CDB SECRETS + +Update secrets files with your base64 encodede password. + +```bash +echo ImAdemoPassword | base64 +SW1BZGVtb1Bhc3N3b3JkCg== +``` +Apply the cdb_secret and pdb_secret yaml file to generate credential information in each namespace. + +``` +kubectl apply -f cdb_secret.yaml +kubectl apply -f pdb_secret.yaml +``` +> ☞ Note that https credential needs to be replicated in any secret file. It is possible to improve configuration by creating a dedicated namespace for https credential in order to specify this information only once. + +Namespace segregation enables the capability of deploying and manages pluggable database without the cdb administrative passwords. + +### CREATE TLS CERTIFICATE + +Here follow an example of script shell that can be used to create secret certificates in each namespace involved in the kubernets multi tenant architecture + +```bash +#!/bin/bash +export CDB_NAMESPACE=cdbnamespace +export PDB_NAMESPACE=pdbnamespace +export OPR_NAMESPACE=oracle-database-operator-system +export SKEY=tls.key +export SCRT=tls.crt +export CART=ca.crt +export COMPANY=oracle +export REST_SERVER=ords + +openssl genrsa -out ca.key 2048 +openssl req -new -x509 -days 365 -key ca.key -subj "/C=CN/ST=GD/L=SZ/O=${COMPANY}, Inc./CN=${COMPANY} Root CA" -out ca.crt +openssl req -newkey rsa:2048 -nodes -keyout ${SKEY} -subj "/C=CN/ST=GD/L=SZ/O=${COMPANY}, Inc./CN=cdb-dev-${REST_SERVER}.${CDB_NAMESPACE}" -out server.csr +echo "subjectAltName=DNS:cdb-dev-${REST_SERVER}.${CDB_NAMESPACE},DNS:www.example.com" > extfile.txt +openssl x509 -req -extfile extfile.txt -days 365 -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out ${SCRT} + +kubectl create secret tls db-tls --key="${SKEY}" --cert="${SCRT}" -n ${CDB_NAMESPACE} +kubectl create secret generic db-ca --from-file="${CART}" -n ${CDB_NAMESPACE} +kubectl create secret tls db-tls --key="${SKEY}" --cert="${SCRT}" -n ${PDB_NAMESPACE} +kubectl create secret generic db-ca --from-file="${CART}" -n ${PDB_NAMESPACE} +kubectl create secret tls db-tls --key="${SKEY}" --cert="${SCRT}" -n ${OPR_NAMESPACE} +kubectl create secret generic db-ca --from-file="${CART}" -n ${OPR_NAMESPACE} +``` +after all secrets creation you shoud have the following pattern + +```bash +kubectl get secrets -n oracle-database-operator-system +NAME TYPE DATA AGE +db-ca Opaque 1 6d5h +db-tls kubernetes.io/tls 2 6d5h +webhook-server-cert kubernetes.io/tls 3 6d15h + + +kubectl get secrets -n cdbnamespace +NAME TYPE DATA AGE +cdb1-secret Opaque 6 6d15h +db-ca Opaque 1 6d6h +db-tls kubernetes.io/tls 2 6d6h + + +kubectl get secrets -n pdbnamespace +NAME TYPE DATA AGE +db-ca Opaque 1 6d6h +db-tls kubernetes.io/tls 2 6d6h +pdb1-secret Opaque 4 2d16h +tde1-secret Opaque 2 22h +``` +### REST SERVER IMAGE CREATION + +```bash +cd oracle-database-operator/ords +docker build -t oracle/ords-dboper:latest . +docker tag oracle/ords-dboper:latest [path_of_your_registry]/ords-dboper:latest +docker push [path_of_your_registry]/ords-dboper.latest +cd - +``` + +### CDB POD CREATION + +**note:** + Before creating the CDB pod make sure that all the pluggable databases in the container DB are open. + + + +Update the cdb_create.yaml with the path of the image generated before to create CDB pod + +```bash +kubectl apply -f cdb_create.yaml +``` + +Verify the status of the operation and cdb pod existence using the following commands + +```bash +## check the pod creation +kubectl get pods -n cdbnamespace + +## check the rest server log after pod creation +kubectl logs -f `/usr/bin/kubectl get pods -n cdbnamespace|grep ords|cut -d ' ' -f 1` -n cdbnamespace + +##login to the pod for further debug and information gathering +kubectl exec -it `kubectl get pods -n cdbnamespace |grep ords|cut -d ' ' -f 1` -n cdbnamespace bash +``` + +[log cdb creation](./cdb_creation_log.txt) + +### PDB CREATION + +Apply the the pdb_create.yaml file to create a new pdb , after pdb creation you should be able to get pdb details using **kubectl get** command + +```bash +kubectl apply -f pdb_create.yaml +``` + +```bash +#!/bin/bash +#checkpdbs.sh +kubectl get pdbs -n pdbnamespace -o=jsonpath='{range .items[*]} +{"\n==================================================================\n"} +{"CDB="}{.metadata.labels.cdb} +{"K8SNAME="}{.metadata.name} +{"PDBNAME="}{.spec.pdbName} +{"OPENMODE="}{.status.openMode} +{"ACTION="}{.status.action} +{"MSG="}{.status.msg} +{"\n"}{end}' +``` + +```bash +./checkpdbs.sh +================================================================== +CDB=cdb-dev +K8SNAME=pdb1 +PDBNAME=pdbdev +OPENMODE=READ WRITE +ACTION=CREATE +MSG=Success + +``` +[pdb creation log](./pdb_creation_log.txt) + +### MAKEFILE + +In order to facilitate the command execution use the [makefile](./makefile) available target details are exposed in the following tables. + +|target |Action | +|-----------------------------|-------------------------------------| +|step1 | Build rest server images | +|step2 | Tag the immages | +|step3 | Push the image into the repository | +|step4 | Load webhook certmanager | +|step5 | Create the db operator | +|step6 | Create tls certificates | +|step7 | Create tls secret | +|step8 | Create database secrets | +|step9 | Create restserver pod | +|checkstep9 | Monitor the executions | +|step10 | Create pluggable database | +|checkpdb | Monitor PDB status | +|dump | Dump pods info into a file | +|reloadop | Reload the db operator | +|login | Login into cdb pod | + + + + diff --git a/docs/multitenant/usecase03/cdb_create.yaml b/docs/multitenant/usecase03/cdb_create.yaml new file mode 100644 index 00000000..09dd7f86 --- /dev/null +++ b/docs/multitenant/usecase03/cdb_create.yaml @@ -0,0 +1,44 @@ +apiVersion: database.oracle.com/v1alpha1 +kind: CDB +metadata: + name: cdb-dev + namespace: cdbnamespace +spec: + cdbName: "DB12" + ordsImage: "lin.ocir.io/intsanjaysingh/mmalvezz/testppr/ords-dboper:latest" + ordsImagePullPolicy: "Always" + dbTnsurl : "(DESCRIPTION=(CONNECT_TIMEOUT=90)(RETRY_COUNT=30)(RETRY_DELAY=10)(TRANSPORT_CONNECT_TIMEOUT=70)(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan12.testrac.com)(PORT=1521)(IP=V4_ONLY))(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan34.testrac.com)(PORT=1521)(IP=V4_ONLY))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=TESTORDS)))" + replicas: 1 + sysAdminPwd: + secret: + secretName: "cdb1-secret" + key: "sysadmin_pwd" + ordsPwd: + secret: + secretName: "cdb1-secret" + key: "ords_pwd" + cdbAdminUser: + secret: + secretName: "cdb1-secret" + key: "cdbadmin_user" + cdbAdminPwd: + secret: + secretName: "cdb1-secret" + key: "cdbadmin_pwd" + webServerUser: + secret: + secretName: "cdb1-secret" + key: "webserver_user" + webServerPwd: + secret: + secretName: "cdb1-secret" + key: "webserver_pwd" + cdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + cdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + diff --git a/docs/multitenant/usecase03/cdb_creation_log.txt b/docs/multitenant/usecase03/cdb_creation_log.txt new file mode 100644 index 00000000..8c7dc161 --- /dev/null +++ b/docs/multitenant/usecase03/cdb_creation_log.txt @@ -0,0 +1,336 @@ +kubectl get pods -n cdbnamespace +NAME READY STATUS RESTARTS AGE +cdb-dev-ords-rs-pgqqh 0/1 ContainerCreating 0 1s + +kubectl get pods -n cdbnamespace +NAME READY STATUS RESTARTS AGE +cdb-dev-ords-rs-pgqqh 1/1 Running 0 6s + +kubectl logs -f `/usr/bin/kubectl get pods -n cdbnamespace|grep ords|cut -d ' ' -f 1` -n cdbnamespace +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M +NOT_INSTALLED=2 + SETUP +==================================================== +CONFIG=/etc/ords/config +total 0 +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 23.3 Production on Thu Jan 25 17:17:20 2024 + +Copyright (c) 2010, 2024, Oracle. + +Configuration: + /etc/ords/config/ + +The setting named: db.connectionType was set to: customurl in configuration: default +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 23.3 Production on Thu Jan 25 17:17:21 2024 + +Copyright (c) 2010, 2024, Oracle. + +Configuration: + /etc/ords/config/ + +The setting named: db.customURL was set to: jdbc:oracle:thin:@(DESCRIPTION=(CONNECT_TIMEOUT=90)(RETRY_COUNT=30)(RETRY_DELAY=10)(TRANSPORT_CONNECT_TIMEOUT=70)(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan12.testrac.com)(PORT=1521)(IP=V4_ONLY))(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan34.testrac.com)(PORT=1521)(IP=V4_ONLY))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=TESTORDS))) in configuration: default +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 23.3 Production on Thu Jan 25 17:17:23 2024 + +Copyright (c) 2010, 2024, Oracle. + +Configuration: + /etc/ords/config/ + +The setting named: security.requestValidationFunction was set to: false in configuration: default +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 23.3 Production on Thu Jan 25 17:17:25 2024 + +Copyright (c) 2010, 2024, Oracle. + +Configuration: + /etc/ords/config/ + +The setting named: jdbc.MaxLimit was set to: 100 in configuration: default +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 23.3 Production on Thu Jan 25 17:17:27 2024 + +Copyright (c) 2010, 2024, Oracle. + +Configuration: + /etc/ords/config/ + +The setting named: jdbc.InitialLimit was set to: 50 in configuration: default +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 23.3 Production on Thu Jan 25 17:17:29 2024 + +Copyright (c) 2010, 2024, Oracle. + +Configuration: + /etc/ords/config/ + +The global setting named: error.externalPath was set to: /opt/oracle/ords/error +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 23.3 Production on Thu Jan 25 17:17:31 2024 + +Copyright (c) 2010, 2024, Oracle. + +Configuration: + /etc/ords/config/ + +The global setting named: standalone.access.log was set to: /home/oracle +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 23.3 Production on Thu Jan 25 17:17:32 2024 + +Copyright (c) 2010, 2024, Oracle. + +Configuration: + /etc/ords/config/ + +The global setting named: standalone.https.port was set to: 8888 +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 23.3 Production on Thu Jan 25 17:17:34 2024 + +Copyright (c) 2010, 2024, Oracle. + +Configuration: + /etc/ords/config/ + +The global setting named: standalone.https.cert was set to: /opt/oracle/ords//secrets/tls.crt +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 23.3 Production on Thu Jan 25 17:17:36 2024 + +Copyright (c) 2010, 2024, Oracle. + +Configuration: + /etc/ords/config/ + +The global setting named: standalone.https.cert.key was set to: /opt/oracle/ords//secrets/tls.key +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 23.3 Production on Thu Jan 25 17:17:38 2024 + +Copyright (c) 2010, 2024, Oracle. + +Configuration: + /etc/ords/config/ + +The setting named: restEnabledSql.active was set to: true in configuration: default +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 23.3 Production on Thu Jan 25 17:17:40 2024 + +Copyright (c) 2010, 2024, Oracle. + +Configuration: + /etc/ords/config/ + +The global setting named: security.verifySSL was set to: true +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 23.3 Production on Thu Jan 25 17:17:42 2024 + +Copyright (c) 2010, 2024, Oracle. + +Configuration: + /etc/ords/config/ + +The global setting named: database.api.enabled was set to: true +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 23.3 Production on Thu Jan 25 17:17:43 2024 + +Copyright (c) 2010, 2024, Oracle. + +Configuration: + /etc/ords/config/ + +The setting named: plsql.gateway.mode was set to: disabled in configuration: default +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 23.3 Production on Thu Jan 25 17:17:45 2024 + +Copyright (c) 2010, 2024, Oracle. + +Configuration: + /etc/ords/config/ + +The global setting named: database.api.management.services.disabled was set to: false +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 23.3 Production on Thu Jan 25 17:17:47 2024 + +Copyright (c) 2010, 2024, Oracle. + +Configuration: + /etc/ords/config/ + +The setting named: misc.pagination.maxRows was set to: 1000 in configuration: default +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 23.3 Production on Thu Jan 25 17:17:49 2024 + +Copyright (c) 2010, 2024, Oracle. + +Configuration: + /etc/ords/config/ + +The setting named: db.cdb.adminUser was set to: C##DBAPI_CDB_ADMIN AS SYSDBA in configuration: default +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 23.3 Production on Thu Jan 25 17:17:51 2024 + +Copyright (c) 2010, 2024, Oracle. + +Configuration: + /etc/ords/config/ + +The setting named: db.cdb.adminUser.password was set to: ****** in configuration: default +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 23.3 Production on Thu Jan 25 17:17:53 2024 + +Copyright (c) 2010, 2024, Oracle. + +Configuration: + /etc/ords/config/ + +Created user welcome in file /etc/ords/config/global/credentials +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 23.3 Production on Thu Jan 25 17:17:55 2024 + +Copyright (c) 2010, 2024, Oracle. + +Configuration: + /etc/ords/config/ + +Oracle REST Data Services - Non-Interactive Install + +Retrieving information.. +Completed verifying Oracle REST Data Services schema version 23.3.0.r2891830. +Connecting to database user: ORDS_PUBLIC_USER url: jdbc:oracle:thin:@(DESCRIPTION=(CONNECT_TIMEOUT=90)(RETRY_COUNT=30)(RETRY_DELAY=10)(TRANSPORT_CONNECT_TIMEOUT=70)(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan12.testrac.com)(PORT=1521)(IP=V4_ONLY))(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan34.testrac.com)(PORT=1521)(IP=V4_ONLY))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=TESTORDS))) +The setting named: db.serviceNameSuffix was set to: in configuration: default +The setting named: db.username was set to: ORDS_PUBLIC_USER in configuration: default +The setting named: db.password was set to: ****** in configuration: default +The setting named: security.requestValidationFunction was set to: ords_util.authorize_plsql_gateway in configuration: default +2024-01-25T17:17:58.898Z INFO Oracle REST Data Services schema version 23.3.0.r2891830 is installed. +2024-01-25T17:17:58.900Z INFO To run in standalone mode, use the ords serve command: +2024-01-25T17:17:58.900Z INFO ords --config /etc/ords/config serve +2024-01-25T17:17:58.900Z INFO Visit the ORDS Documentation to access tutorials, developer guides and more to help you get started with the new ORDS Command Line Interface (http://oracle.com/rest). +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 23.3 Production on Thu Jan 25 17:18:00 2024 + +Copyright (c) 2010, 2024, Oracle. + +Configuration: + /etc/ords/config/ + +2024-01-25T17:18:00.960Z INFO HTTP and HTTP/2 cleartext listening on host: 0.0.0.0 port: 8080 +2024-01-25T17:18:00.963Z INFO HTTPS and HTTPS/2 listening on host: 0.0.0.0 port: 8888 +2024-01-25T17:18:00.980Z INFO Disabling document root because the specified folder does not exist: /etc/ords/config/global/doc_root +2024-01-25T17:18:00.981Z INFO Default forwarding from / to contextRoot configured. +2024-01-25T17:18:06.634Z INFO Configuration properties for: |default|lo| +db.serviceNameSuffix= +java.specification.version=21 +conf.use.wallet=true +database.api.management.services.disabled=false +sun.jnu.encoding=UTF-8 +user.region=US +java.class.path=/opt/oracle/ords/ords.war +java.vm.vendor=Oracle Corporation +standalone.https.cert.key=/opt/oracle/ords//secrets/tls.key +sun.arch.data.model=64 +nashorn.args=--no-deprecation-warning +java.vendor.url=https://java.oracle.com/ +resource.templates.enabled=false +user.timezone=UTC +java.vm.specification.version=21 +os.name=Linux +sun.java.launcher=SUN_STANDARD +user.country=US +sun.boot.library.path=/usr/java/jdk-21/lib +sun.java.command=/opt/oracle/ords/ords.war --config /etc/ords/config serve --port 8888 --secure +jdk.debug=release +sun.cpu.endian=little +user.home=/home/oracle +oracle.dbtools.launcher.executable.jar.path=/opt/oracle/ords/ords.war +user.language=en +db.cdb.adminUser.password=****** +java.specification.vendor=Oracle Corporation +java.version.date=2023-10-17 +database.api.enabled=true +java.home=/usr/java/jdk-21 +db.username=ORDS_PUBLIC_USER +file.separator=/ +java.vm.compressedOopsMode=32-bit +line.separator= + +restEnabledSql.active=true +java.specification.name=Java Platform API Specification +java.vm.specification.vendor=Oracle Corporation +java.awt.headless=true +standalone.https.cert=/opt/oracle/ords//secrets/tls.crt +db.password=****** +sun.management.compiler=HotSpot 64-Bit Tiered Compilers +security.requestValidationFunction=ords_util.authorize_plsql_gateway +misc.pagination.maxRows=1000 +java.runtime.version=21.0.1+12-LTS-29 +user.name=oracle +error.externalPath=/opt/oracle/ords/error +stdout.encoding=UTF-8 +path.separator=: +db.cdb.adminUser=C##DBAPI_CDB_ADMIN AS SYSDBA +os.version=5.4.17-2136.323.8.1.el7uek.x86_64 +java.runtime.name=Java(TM) SE Runtime Environment +file.encoding=UTF-8 +plsql.gateway.mode=disabled +security.verifySSL=true +standalone.https.port=8888 +java.vm.name=Java HotSpot(TM) 64-Bit Server VM +java.vendor.url.bug=https://bugreport.java.com/bugreport/ +java.io.tmpdir=/tmp +oracle.dbtools.cmdline.ShellCommand=ords +java.version=21.0.1 +user.dir=/home/oracle/keystore +os.arch=amd64 +java.vm.specification.name=Java Virtual Machine Specification +jdbc.MaxLimit=100 +oracle.dbtools.cmdline.home=/opt/oracle/ords +native.encoding=UTF-8 +java.library.path=/usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib +java.vendor=Oracle Corporation +java.vm.info=mixed mode, sharing +stderr.encoding=UTF-8 +java.vm.version=21.0.1+12-LTS-29 +sun.io.unicode.encoding=UnicodeLittle +jdbc.InitialLimit=50 +db.connectionType=customurl +java.class.version=65.0 +db.customURL=jdbc:oracle:thin:@(DESCRIPTION=(CONNECT_TIMEOUT=90)(RETRY_COUNT=30)(RETRY_DELAY=10)(TRANSPORT_CONNECT_TIMEOUT=70)(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan12.testrac.com)(PORT=1521)(IP=V4_ONLY))(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan34.testrac.com)(PORT=1521)(IP=V4_ONLY))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=TESTORDS))) +standalone.access.log=/home/oracle + +2024-01-25T17:18:10.381Z INFO + +Mapped local pools from /etc/ords/config/databases: + /ords/ => default => VALID + + +2024-01-25T17:18:10.532Z INFO Oracle REST Data Services initialized +Oracle REST Data Services version : 23.3.0.r2891830 +Oracle REST Data Services server info: jetty/10.0.17 +Oracle REST Data Services java info: Java HotSpot(TM) 64-Bit Server VM 21.0.1+12-LTS-29 + + +exec -it `kubectl get pods -n cdbnamespace |grep ords|cut -d ' ' -f 1` -n cdbnamespace bash +[oracle@cdb-dev-ords-rs-pgqqh ~]$ ps -ef|grep java +oracle 1147 1116 10 17:17 ? 00:00:21 /usr/java/jdk-21/bin/java -Doracle.dbtools.cmdline.home=/opt/oracle/ords -Duser.language=en -Duser.region=US -Dfile.encoding=UTF-8 -Djava.awt.headless=true -Dnashorn.args=--no-deprecation-warning -Doracle.dbtools.cmdline.ShellCommand=ords -Duser.timezone=UTC -jar /opt/oracle/ords/ords.war --config /etc/ords/config serve --port 8888 --secure +oracle 1227 1200 0 17:21 pts/0 00:00:00 grep --color=auto java diff --git a/docs/multitenant/usecase03/cdb_secret.yaml b/docs/multitenant/usecase03/cdb_secret.yaml new file mode 100644 index 00000000..8f1b6fc9 --- /dev/null +++ b/docs/multitenant/usecase03/cdb_secret.yaml @@ -0,0 +1,17 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: v1 +kind: Secret +metadata: + name: cdb1-secret + namespace: cdbnamespace +type: Opaque +data: + ords_pwd: "[...base64 encoded password...]" + sysadmin_pwd: "[...base64 encoded password...]" + cdbadmin_user: "[...base64 encoded password...]" + cdbadmin_pwd: "[...base64 encoded password...]" + webserver_user: "[...base64 encoded password...]" + webserver_pwd: "[...base64 encoded password...]" diff --git a/docs/multitenant/usecase03/gentlscert.sh b/docs/multitenant/usecase03/gentlscert.sh new file mode 100644 index 00000000..49e29147 --- /dev/null +++ b/docs/multitenant/usecase03/gentlscert.sh @@ -0,0 +1,23 @@ +#!/bin/bash +export CDB_NAMESPACE=cdbnamespace +export PDB_NAMESPACE=pdbnamespace +export OPR_NAMESPACE=oracle-database-operator-system +export SKEY=tls.key +export SCRT=tls.crt +export CART=ca.crt +export COMPANY=oracle +export REST_SERVER=ords + +openssl genrsa -out ca.key 2048 +openssl req -new -x509 -days 365 -key ca.key -subj "/C=CN/ST=GD/L=SZ/O=${COMPANY}, Inc./CN=${COMPANY} Root CA" -out ca.crt +openssl req -newkey rsa:2048 -nodes -keyout ${SKEY} -subj "/C=CN/ST=GD/L=SZ/O=${COMPANY}, Inc./CN=cdb-dev-${REST_SERVER}.${CDB_NAMESPACE}" -out server.csr +echo "subjectAltName=DNS:cdb-dev-${REST_SERVER}.${CDB_NAMESPACE},DNS:www.example.com" > extfile.txt +openssl x509 -req -extfile extfile.txt -days 365 -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out ${SCRT} + +kubectl create secret tls db-tls --key="${SKEY}" --cert="${SCRT}" -n ${CDB_NAMESPACE} +kubectl create secret generic db-ca --from-file="${CART}" -n ${CDB_NAMESPACE} +kubectl create secret tls db-tls --key="${SKEY}" --cert="${SCRT}" -n ${PDB_NAMESPACE} +kubectl create secret generic db-ca --from-file="${CART}" -n ${PDB_NAMESPACE} +kubectl create secret tls db-tls --key="${SKEY}" --cert="${SCRT}" -n ${OPR_NAMESPACE} +kubectl create secret generic db-ca --from-file="${CART}" -n ${OPR_NAMESPACE} + diff --git a/docs/multitenant/usecase03/makefile b/docs/multitenant/usecase03/makefile new file mode 100644 index 00000000..fc95cfa0 --- /dev/null +++ b/docs/multitenant/usecase03/makefile @@ -0,0 +1,291 @@ +# __ __ _ __ _ _ +# | \/ | __ _| | _____ / _(_) | ___ +# | |\/| |/ _` | |/ / _ \ |_| | |/ _ \ +# | | | | (_| | < __/ _| | | __/ +# |_| |_|\__,_|_|\_\___|_| |_|_|\___| +# +# ___ +# / _ \ _ __ _ __ _ __ ___ _ __ ___ +# | | | | '_ \| '_ \| '__/ _ \ '_ ` _ \ +# | |_| | | | | |_) | | | __/ | | | | | +# \___/|_| |_| .__/|_| \___|_| |_| |_| +# |_| +# ____ _ _ _ +# / ___|___ _ __ | |_ _ __ ___ | | | ___ _ __ +# | | / _ \| '_ \| __| '__/ _ \| | |/ _ \ '__| +# | |__| (_) | | | | |_| | | (_) | | | __/ | +# \____\___/|_| |_|\__|_| \___/|_|_|\___|_| +# +# +# This makefile helps to speed up the kubectl commands executions to deploy and test +# the mutlitenant operator. Although it has few functionality you can adapt to your needs +# by adding much more targets. +# +# Quick start: +# ~~~~~~~~~~~ +# +# - Copy files of tab.1 in the makefile directory. +# - Edit the secret files and other yaml files with the correct credential as +# specified in the documentation. +# - Edit makefile updating variables of tab.2 +# - Execute commands of tab.3 "make step1" "make step2" "make step3".... +# +# Tab.1 - List of required files +# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +# +# +-----------------------------+---------------------------------------------+ +# |oracle-database-operator.yaml| Opertaor yaml file | +# +-----------------------------+---------------------------------------------+ +# |cdb_secret.yaml | Secret file for the rest server pod | +# +-----------------------------+---------------------------------------------+ +# |pdb_secret.yaml | Secret file for the pdb creation | +# +-----------------------------+---------------------------------------------+ +# |cdb_create.yaml | Rest server pod creation | +# +-----------------------------+---------------------------------------------+ +# |pdb_create.yaml | Pluggable database creation | +# +-----------------------------+---------------------------------------------+ +# |oracle-database-operator.yaml| Database operator | +# +-----------------------------+---------------------------------------------+ +# |Dockerfiles | Dockerfile for CBD | +# +-----------------------------+---------------------------------------------+ +# |runOrdsSSL.sh | Init script executed by Dockerfile | +# +-----------------------------+---------------------------------------------+ +# +# Tab.2 - List of variables +# ~~~~~~~~~~~~~~~~~~~~~~~~~ +# +# +-----------------------------+---------------------------------------------+ +# |OCIR | Your image registry | +# +-----------------------------+---------------------------------------------+ +# |OCIRPATH | Path of the image in your registry | +# +-----------------------------+---------------------------------------------+ +# +# Tab.3 - Execution steps +# ~~~~~~~~~~~~~~~~~~~~~~~ +# +# +-----------------------------+---------------------------------------------+ +# | MAKEFILE TARGETS LIST | +# | ----- ooo ----- | +# | - TARGET - - DESCRIPTION - | +# +-----------------------------+-------------------------------------+-------+ +# |step1 | Build rest server images | | +# +-----------------------------+-------------------------------------+ REST | +# |step2 | Tag the immages | SRV | +# +-----------------------------+-------------------------------------+ IMG | +# |step3 | Push the image into the repository | | +# +-----------------------------+-------------------------------------+-------+ +# |step4 | Load webhook certmanager | DB | +# +-----------------------------+-------------------------------------+ OPER | +# |step5 | Create the db operator | | +# +-----------------------------+-------------------------------------+-------+ +# |step6 | Create tls certificates | T | +# +-----------------------------+-------------------------------------+ L | +# |step7 | Create tls secret | S | +# +-----------------------------+---------------------------------------------+ +# |step8 | Create database secrets | +# +-----------------------------+---------------------------------------------+ +# |step9 | Create restserver pod | +# | | +---------------------------------------------+ +# | +---> checkstep9 | Monitor the executions | +# +-----------------------------+---------------------------------------------+ +# |step10 | Create pluggable database | +# | | +---------------------------------------------+ +# | +---> checkpdb | Monitor PDB status | +# +-----------------------------+---------------------------------------------+ +# | DIAGNOSTIC TARGETS | +# +-----------------------------+---------------------------------------------+ +# | dump | Dump pods info into a file | +# +-----------------------------+---------------------------------------------+ +# | reloadop | Reload the db operator | +# +-----------------------------+---------------------------------------------+ +# | login | Login into cdb pod | +# +-----------------------------+---------------------------------------------+ + + +################ TAB 2 VARIABLES ############ +REST_SERVER=ords +ORDSVERSION=latest + +OCIR=[container registry] +OCIRPATH=$(REST_SERVER)-dboper:$(ORDSVERSION) + +#examples: +#OCIR=lin.ocir.io +#OCIRPATH=/sampletenancy/samplepath/sampledir/$(REST_SERVER)-dboper:$(ORDSVERSION) +############################################# +DOCKER=/usr/bin/docker +KUBECTL=/usr/bin/kubectl +ORDS=/usr/local/bin/ords +CONFIG=/etc/ords/config +IMAGE=oracle/$(REST_SERVER)-dboper:$(ORDSVERSION) +DBOPERATOR=oracle-database-operator.yaml +URLPATH=/_/db-api/stable/database/pdbs/ +OPENSSL=/usr/bin/openssl +ORDSPORT=8888 +MAKE=/usr/bin/make +DOCKERFILE=Dockerfile +RM=/usr/bin/rm +ECHO=/usr/bin/echo +CERTMANAGER=https://github.com/jetstack/cert-manager/releases/latest/download/cert-manager.yaml +CDB_SECRET_YAML=cdb_secret.yaml +PDB_SECRET_YAML=pdb_secret.yaml +TDE_SECRET_YAML=tde_secret.yaml +CDB_NAMESPACE_YAML=ns_namespace_cdb.yaml +PDB_NAMESPACE_YAML=ns_namespace_pdb.yaml +OPR_NAMESPACE=oracle-database-operator-system +PDB_NAMESPACE=$(shell grep namespace $(PDB_NAMESPACE_YAML) |cut -d: -f 2| tr -d ' ') +CDB_NAMESPACE=$(shell grep namespace $(CDB_NAMESPACE_YAML) |cut -d: -f 2| tr -d ' ') +CDB=cdb_create.yaml +PDB=pdb_create.yaml +SKEY=tls.key +SCRT=tls.crt +CART=ca.crt +COMPANY=oracle +LOCALHOST=localhost +RESTPREFIX=cdb-dev + + +step1: createimage +step2: tagimage +step3: push +step4: certmanager +step5: dboperator +step6: tlscert +step7: tlssecret +step8: dbsecret +step9: cdb +step10: pdb + +checkstep9: checkcdb + + +createimage: + @echo "BUILDING CDB IMAGES" + @if [[ ! -f ./Dockerfile ]]; \ + then\ + echo "DOCKERFILE DOES NOT EXISTS";\ + exit 1; \ + fi; + @if [[ ! -f ../runOrdsSSL.sh ]]; \ + then\ + echo "DOCKERFILE DOES NOT EXISTS";\ + exit 1; \ + fi; + $(DOCKER) build -t $(IMAGE) . + +tagimage: + @echo "TAG IMAGE" + $(DOCKER) tag $(IMAGE) $(OCIR)$(OCIRPATH) + +push: + @echo "PUSH IMAGE INTO THE REGISTRY" + $(DOCKER) push $(OCIR)$(OCIRPATH) + +certmanager: + @echo "WEBHOOK CERT MANAGER" + $(KUBECTL) apply -f $(CERTMANAGER) + +dboperator: + @echo "ORACLE DATABASE OPERATOR" + $(KUBECTL) apply -f $(DBOPERATOR) + +namespace: + $(KUBECTL) get namespaces + $(KUBECTL) apply -f $(CDB_NAMESPACE_YAML) + $(KUBECTL) apply -f $(PDB_NAMESPACE_YAML) + $(KUBECTL) get namespaces + + +tlscert: + @echo "CREATING TLS CERTIFICATES" + $(OPENSSL) genrsa -out ca.key 2048 + $(OPENSSL) req -new -x509 -days 365 -key ca.key -subj "/C=US/ST=California/L=SanFrancisco/O=$(COMPANY) /CN=$(RESTPREFIX)-$(REST_SERVER).$(CDB_NAMESPACE) /CN=$(LOCALHOST) Root CA " -out ca.crt + $(OPENSSL) req -newkey rsa:2048 -nodes -keyout $(SKEY) -subj "/C=US/ST=California/L=SanFrancisco/O=$(COMPANY) /CN=$(RESTPREFIX)-$(REST_SERVER).$(CDB_NAMESPACE) /CN=$(LOCALHOST)" -out server.csr + $(ECHO) "subjectAltName=DNS:cdb-dev-$(REST_SERVER),DNS:www.example.com" > extfile.txt + $(OPENSSL) x509 -req -extfile extfile.txt -days 365 -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out $(SCRT) + + +tlssecret: + $(KUBECTL) create secret tls db-tls --key="$(SKEY)" --cert="$(SCRT)" -n $(CDB_NAMESPACE) + $(KUBECTL) create secret generic db-ca --from-file="$(CART)" -n $(CDB_NAMESPACE) + $(KUBECTL) create secret tls db-tls --key="$(SKEY)" --cert="$(SCRT)" -n $(PDB_NAMESPACE) + $(KUBECTL) create secret generic db-ca --from-file="$(CART)" -n $(PDB_NAMESPACE) + $(KUBECTL) create secret tls db-tls --key="$(SKEY)" --cert="$(SCRT)" -n $(OPR_NAMESPACE) + $(KUBECTL) create secret generic db-ca --from-file="$(CART)" -n $(OPR_NAMESPACE) + + +dbsecret: + @echo "CREATING DB SECRETS" + $(KUBECTL) apply -f $(CDB_SECRET_YAML) + $(KUBECTL) apply -f $(PDB_SECRET_YAML) + $(KUBECTL) apply -f $(TDE_SECRET_YAML) + + +cdb: + @echo "CREATING REST SRV POD" + $(KUBECTL) apply -f $(CDB) + +checkcdb: + $(KUBECTL) logs -f `$(KUBECTL) get pods -n $(CDB_NAMESPACE)|grep $(REST_SERVER)|cut -d ' ' -f 1` -n $(CDB_NAMESPACE) + +pdb: + $(KUBECTL) apply -f $(PDB) + +checkpdb: + $(KUBECTL) get pdbs -n $(OPR_NAMESPACE) + +dump: + @$(eval TMPSP := $(shell date "+%y%m%d%H%M%S" )) + @$(eval DIAGFILE := ./opdmp.$(TMPSP)) + @>$(DIAGFILE) + @echo "OPERATOR DUMP" >> $(DIAGFILE) + @echo "~~~~~~~~~~~~~" >> $(DIAGFILE) + $(KUBECTL) logs pod/`$(KUBECTL) get pods -n $(OPR_NAMESPACE)|grep oracle-database-operator-controller|head -1|cut -d ' ' -f 1` -n $(OPR_NAMESPACE) >>$(DIAGFILE) + $(KUBECTL) logs pod/`$(KUBECTL) get pods -n $(OPR_NAMESPACE)|grep oracle-database-operator-controller|head -2|tail -1 | cut -d ' ' -f 1` -n $(OPR_NAMESPACE) >>$(DIAGFILE) + $(KUBECTL) logs pod/`$(KUBECTL) get pods -n $(OPR_NAMESPACE)|grep oracle-database-operator-controller|tail -1|cut -d ' ' -f 1` -n $(OPR_NAMESPACE) >>$(DIAGFILE) + @echo "CDB LOG DUMP" >> $(DIAGFILE) + @echo "~~~~~~~~" >> $(DIAGFILE) + $(KUBECTL) logs `$(KUBECTL) get pods -n $(OPR_NAMESPACE)|grep $(REST_SERVER)| cut -d ' ' -f 1` -n $(OPR_NAMESPACE) >>$(DIAGFILE) + @echo "SECRET DMP" >>$(DIAGFILE) + @echo "~~~~~~~~" >> $(DIAGFILE) + $(KUBECTL) get secrets -o yaml -n $(OPR_NAMESPACE) >> $(DIAGFILE) + @echo "CDB/PDB DMP" >> $(DIAGFILE) + $(KUBECTL) get pdbs -o yaml -n $(OPR_NAMESPACE) >> $(DIAGFILE) + $(KUBECTL) get cdb -o yaml -n $(OPR_NAMESPACE) >> $(DIAGFILE) + @echo "CLUSTER INFO" >> $(DIAGFILE) + $(KUBECTL) get nodes -o wide + $(KUBECTL) get svc --namespace=kube-system + +reloadop: + echo "RESTARTING OPERATOR" + $(eval OP1 := $(shell $(KUBECTL) get pods -n $(OPR_NAMESPACE)|grep oracle-database-operator-controller|head -1|cut -d ' ' -f 1 )) + $(eval OP2 := $(shell $(KUBECTL) get pods -n $(OPR_NAMESPACE)|grep oracle-database-operator-controller|head -2|tail -1|cut -d ' ' -f 1 )) + $(eval OP3 := $(shell $(KUBECTL) get pods -n $(OPR_NAMESPACE)|grep oracle-database-operator-controller|tail -1|cut -d ' ' -f 1 )) + $(KUBECTL) get pod $(OP1) -n $(OPR_NAMESPACE) -o yaml | kubectl replace --force -f - + $(KUBECTL) get pod $(OP2) -n $(OPR_NAMESPACE) -o yaml | kubectl replace --force -f - + $(KUBECTL) get pod $(OP3) -n $(OPR_NAMESPACE) -o yaml | kubectl replace --force -f - + +login: + $(KUBECTL) exec -it `$(KUBECTL) get pods -n $(CDB_NAMESPACE) |grep $(REST_SERVER)|cut -d ' ' -f 1` -n $(CDB_NAMESPACE) bash + +cdblog: + $(KUBECTL) logs -f `$(KUBECTL) get pods -n $(CDB_NAMESPACE)|grep $(REST_SERVER)|cut -d ' ' -f 1` -n $(CDB_NAMESPACE) + + + +xlog1: + $(KUBECTL) logs -f pod/`$(KUBECTL) get pods -n $(OPR_NAMESPACE)|grep oracle-database-operator-controller|head -1|cut -d ' ' -f 1` -n $(OPR_NAMESPACE) + +xlog2: + $(KUBECTL) logs -f pod/`$(KUBECTL) get pods -n $(OPR_NAMESPACE)|grep oracle-database-operator-controller|head -2|tail -1|cut -d ' ' -f 1` -n $(OPR_NAMESPACE) + +xlog3: + $(KUBECTL) logs -f pod/`$(KUBECTL) get pods -n $(OPR_NAMESPACE)|grep oracle-database-operator-controller|tail -1|cut -d ' ' -f 1` -n $(OPR_NAMESPACE) + +checkdep: + $(KUBECTL) api-resources --verbs=list --namespaced -o name | xargs -n 1 $(KUBECTL) get -n $(OPR_NAMESPACE) + $(KUBECTL) api-resources --verbs=list --namespaced -o name | xargs -n 1 $(KUBECTL) get -n $(CBD_NAMESPACE) + $(KUBECTL) api-resources --verbs=list --namespaced -o name | xargs -n 1 $(KUBECTL) get -n $(PDB_NAMESPACE) + + + diff --git a/docs/multitenant/usecase03/ns_namespace_cdb.yaml b/docs/multitenant/usecase03/ns_namespace_cdb.yaml new file mode 100644 index 00000000..f4c6d77b --- /dev/null +++ b/docs/multitenant/usecase03/ns_namespace_cdb.yaml @@ -0,0 +1,7 @@ +apiVersion: v1 +kind: Namespace +metadata: + labels: + control-plane: controller-manager + name: cdbnamespace + diff --git a/docs/multitenant/usecase03/ns_namespace_pdb.yaml b/docs/multitenant/usecase03/ns_namespace_pdb.yaml new file mode 100644 index 00000000..b22245f9 --- /dev/null +++ b/docs/multitenant/usecase03/ns_namespace_pdb.yaml @@ -0,0 +1,7 @@ +apiVersion: v1 +kind: Namespace +metadata: + labels: + control-plane: controller-manager + name: pdbnamespace + diff --git a/docs/multitenant/usecase03/operator_creation_log.txt b/docs/multitenant/usecase03/operator_creation_log.txt new file mode 100644 index 00000000..36ed02ac --- /dev/null +++ b/docs/multitenant/usecase03/operator_creation_log.txt @@ -0,0 +1,27 @@ +kubectl apply -f oracle-database-operator.yaml +namespace/oracle-database-operator-system created +customresourcedefinition.apiextensions.k8s.io/autonomouscontainerdatabases.database.oracle.com configured +customresourcedefinition.apiextensions.k8s.io/autonomousdatabasebackups.database.oracle.com configured +customresourcedefinition.apiextensions.k8s.io/autonomousdatabaserestores.database.oracle.com configured +customresourcedefinition.apiextensions.k8s.io/autonomousdatabases.database.oracle.com configured +customresourcedefinition.apiextensions.k8s.io/cdbs.database.oracle.com configured +customresourcedefinition.apiextensions.k8s.io/dataguardbrokers.database.oracle.com configured +customresourcedefinition.apiextensions.k8s.io/dbcssystems.database.oracle.com configured +customresourcedefinition.apiextensions.k8s.io/oraclerestdataservices.database.oracle.com configured +customresourcedefinition.apiextensions.k8s.io/pdbs.database.oracle.com configured +customresourcedefinition.apiextensions.k8s.io/shardingdatabases.database.oracle.com configured +customresourcedefinition.apiextensions.k8s.io/singleinstancedatabases.database.oracle.com configured +role.rbac.authorization.k8s.io/oracle-database-operator-leader-election-role created +clusterrole.rbac.authorization.k8s.io/oracle-database-operator-manager-role created +clusterrole.rbac.authorization.k8s.io/oracle-database-operator-metrics-reader created +clusterrole.rbac.authorization.k8s.io/oracle-database-operator-oracle-database-operator-proxy-role created +rolebinding.rbac.authorization.k8s.io/oracle-database-operator-oracle-database-operator-leader-election-rolebinding created +clusterrolebinding.rbac.authorization.k8s.io/oracle-database-operator-oracle-database-operator-manager-rolebinding created +clusterrolebinding.rbac.authorization.k8s.io/oracle-database-operator-oracle-database-operator-proxy-rolebinding created +service/oracle-database-operator-controller-manager-metrics-service created +service/oracle-database-operator-webhook-service created +certificate.cert-manager.io/oracle-database-operator-serving-cert created +issuer.cert-manager.io/oracle-database-operator-selfsigned-issuer created +mutatingwebhookconfiguration.admissionregistration.k8s.io/oracle-database-operator-mutating-webhook-configuration created +validatingwebhookconfiguration.admissionregistration.k8s.io/oracle-database-operator-validating-webhook-configuration created +deployment.apps/oracle-database-operator-controller-manager created diff --git a/docs/multitenant/usecase03/pdb_create.yaml b/docs/multitenant/usecase03/pdb_create.yaml new file mode 100644 index 00000000..200f3712 --- /dev/null +++ b/docs/multitenant/usecase03/pdb_create.yaml @@ -0,0 +1,46 @@ +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + cdbName: "DB12" + pdbName: "pdbdev" + adminName: + secret: + secretName: "pdb1-secret" + key: "sysadmin_user" + adminPwd: + secret: + secretName: "pdb1-secret" + key: "sysadmin_pwd" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "pdb1-secret" + key: "webserver_user" + webServerPwd: + secret: + secretName: "pdb1-secret" + key: "webserver_pwd" + fileNameConversions: "NONE" + tdeImport: false + totalSize: "1G" + tempSize: "100M" + action: "Create" + diff --git a/docs/multitenant/usecase03/pdb_creation_log.txt b/docs/multitenant/usecase03/pdb_creation_log.txt new file mode 100644 index 00000000..71d0eb4f --- /dev/null +++ b/docs/multitenant/usecase03/pdb_creation_log.txt @@ -0,0 +1,6 @@ +kubectl apply -f pdb_create.yaml +pdb.database.oracle.com/pdb1 created + +kubectl get pdbs -n pdbnamespace +NAME CONNECT_STRING CDB NAME PDB NAME PDB STATE PDB SIZE STATUS MESSAGE +pdb1 (DESCRIPTION=(CONNECT_TIMEOUT=90)(RETRY_COUNT=30)(RETRY_DELAY=10)(TRANSPORT_CONNECT_TIMEOUT=70)(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan12.testrac.com)(PORT=1521)(IP=V4_ONLY))(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan34.testrac.com)(PORT=1521)(IP=V4_ONLY))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=pdbdev))) DB12 pdbdev READ WRITE 0.78G Ready Success diff --git a/docs/multitenant/usecase03/pdb_secret.yaml b/docs/multitenant/usecase03/pdb_secret.yaml new file mode 100644 index 00000000..f1dfdac6 --- /dev/null +++ b/docs/multitenant/usecase03/pdb_secret.yaml @@ -0,0 +1,16 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: v1 +kind: Secret +metadata: + name: pdb1-secret + namespace: pdbnamespace +type: Opaque +data: + sysadmin_user: "[...base64 encoded password...]" + sysadmin_pwd: "[...base64 encoded password...]" + webserver_user: "[...base64 encoded password...]" + webserver_pwd: "[...base64 encoded password...]" + diff --git a/docs/observability/README.md b/docs/observability/README.md new file mode 100644 index 00000000..986b1885 --- /dev/null +++ b/docs/observability/README.md @@ -0,0 +1,256 @@ +# Managing Observability on Kubernetes for Oracle Databases + +Oracle Database Operator for Kubernetes (`OraOperator`) includes the +Observability controller for Oracle Databases and adds the `DatabaseObserver` CRD, which enables users to observe +Oracle Databases by scraping database metrics using SQL queries. The controller +automates the deployment and maintenance of the metrics exporter container image, +metrics exporter service and a Prometheus servicemonitor. + +The following sections explains the configuration and functionality +of the controller. + +* [Prerequisites](#prerequisites) +* [The DatabaseObserver Custom Resource Definition](#the-databaseobserver-custom-resource) +* [Configuration of DatabaseObservers](#configuration) + * [Create](#create-resource) + * [List](#list-resource) + * [Get Status](#get-detailed-status) + * [Update](#patch-resource) + * [Delete](#delete-resource) +* [Mandatory Roles and Privileges](#mandatory-roles-and-privileges-requirements-for-observability-controller) +* [Debugging and troubleshooting](#debugging-and-troubleshooting) + +## Prerequisites +The `DatabaseObserver` custom resource has the following pre-requisites: + +1. Prometheus and its `servicemonitor` custom resource definition must be installed on the cluster. + +- The Observability controller creates multiple Kubernetes resources that include + a Prometheus `servicemonitor`. In order for the controller + to create ServiceMonitors, the ServiceMonitor custom resource must exist. + +2. A pre-existing Oracle Database and the proper database grants and privileges. + +- The controller exports metrics through SQL queries that the user can control + and specify through a _toml_ file. The necessary access privileges to the tables used in the queries + are not provided and applied automatically. + +### The DatabaseObserver Custom Resource +The Oracle Database Operator (__v1.1.0__) includes the Oracle Database Observability controller which automates +the deployment and setting up of the Oracle Database metrics exporter and the related resources to make Oracle databases observable. + +In the sample YAML file found in +[./config/samples/observability/databaseobserver.yaml](../../config/samples/observability/databaseobserver.yaml), +the databaseObserver custom resource offers the following properties to be configured: + +| Attribute | Type | Default | Required? | Example | +|-------------------------------------------------------|---------|-----------------|--------------|-----------------------------------------------------------------------| +| `spec.database.dbUser.key` | string | user | Optional | _username_ | +| `spec.database.dbUser.secret` | string | - | Yes | _db-secret_ | +| `spec.database.dbPassword.key` | string | password | Optional | _admin-password_ | +| `spec.database.dbPassword.secret` | string | - | Conditional | _db-secret_ | +| `spec.database.dbPassword.vaultOCID` | string | - | Conditional | _ocid1.vault.oc1..._ | +| `spec.database.dbPassword.vaultSecretName` | string | - | Conditional | _db-vault_ | +| `spec.database.dbWallet.secret` | string | - | Conditional | _devsec-oradevdb-wallet_ | +| `spec.database.dbConnectionString.key` | string | connection | Optional | _connection_ | +| `spec.database.dbConnectionString.secret` | string | - | Yes | _db-secretg_ | +| `spec.exporter.image` | string | - | Optional | _container-registry.oracle.com/database/observability-exporter:1.0.2_ | +| `spec.exporter.configuration.configmap.key` | string | config.toml | Optional | _config.toml_ | +| `spec.exporter.configuration.configmap.configmapName` | string | - | Optional | _devcm-oradevdb-config_ | +| `spec.exporter.service.port` | number | 9161 | Optional | _9161_ | +| `spec.prometheus.port` | string | metrics | Optional | _metrics_ | +| `spec.prometheus.labels` | map | app: obs-{name} | Optional | _app: oradevdb-apps_ | +| `spec.replicas` | number | 1 | Optional | _1_ | +| `spec.ociConfig.configMapName` | string | - | Conditional | _oci-cred_ | +| `spec.ociConfig.secretName` | string | - | Conditional | _oci-privatekey_ | + + + + + +### Configuration +The `databaseObserver` custom resource has the following fields for all configurations that are required: +* `spec.database.dbUser.secret` - secret containing the database username. The corresponding key can be any value but must match the key in the secret provided. +* `spec.database.dbPassword.secret` - secret containing the database password (if vault is NOT used). The corresponding key field can be any value but must match the key in the secret provided +* `spec.database.dbConnectionString.secret` - secret containing the database connection string. The corresponding key field can be any value but must match the key in the secret provided i + +If a database wallet is required to connect, the following field containing the secret is required: +* `spec.database.dbWallet.secret` - secret containing the database wallet. The filenames must be used as the keys + +If vault is used to store the database password instead, the following fields are required: +* `spec.database.dbPassword.vaultOCID` - OCID of the vault used +* `spec.database.dbPassword.vaultSecretName` - Name of the secret inside the desired vault +* `spec.ociConfig.configMapName` - holds the rest of the information of the OCI API signing key. The following keys must be used: `fingerprint`, `region`, `tenancy` and `user` +* `spec.ociConfig.secretName` - holds the private key of the OCI API signing key. The key to the file containing the user private key must be: `privatekey` + +The `databaseObserver` provides the remaining multiple fields that are optional: +* `spec.prometheus.labels` - labels to use for Service, ServiceMonitor and Deployment +* `spec.prometheus.port` - port to use for ServiceMonitor +* `spec.replicas` - number of replicas to deploy +* `spec.exporter.service.port` - port of service +* `spec.exporter.image` - image version of observability exporter to use + + +### Create Resource +Follow the steps below to create a new databaseObserver resource object. + +1. To begin, creating a databaseObserver requires you to create and provide kubernetes Secrets to provide connection details: +```bash +kubectl create secret generic db-secret \ + --from-literal=username='username' \ + --from-literal=password='password_here' \ + --from-literal=connection='dbsample_tp' +``` + +2. (Conditional) Create a Kubernetes secret for the wallet (if a wallet is required to connect to the database). + +You can create this secret by using a command similar to the following example below. +If you are connecting to an Autunomous Database and the operator is used to manage the Oracle Autonomous Database, +a client wallet can also be downloaded as a secret through kubectl commands. You can find out how, [here](../../docs/adb/README.md#download-wallets). + +Otherwise, you can create the wallet secret from a local directory containing the wallet files. +```bash +kubectl create secret generic db-wallet --from-file=wallet_dir +``` + +3. Finally, update the databaseObserver manifest with the resources you have created. You can use the example manifest +inside config/samples/observability to specify and create your databaseObserver object with a +YAML file. + +```YAML +# example +apiVersion: observability.oracle.com/v1alpha1 +kind: DatabaseObserver +metadata: + name: obs-sample +spec: + database: + dbUser: + key: "username" + secret: db-secret + + dbPassword: + key: "password" + secret: db-secret + + dbConnectionString: + key: "connection" + secret: db-secret + + dbWallet: + secret: db-wallet +``` + +```bash + kubectl apply -f databaseobserver.yaml +``` + +### List Resource +To list the Observability custom resources, use the following command as an example: +```bash +kubectl get databaseobserver -A +``` + +### Get Detailed Status +To obtain a quick status, use the following command as an example: + +> Note: The databaseobserver custom resource is named `obs-sample` in the next following sections. +> We will use this name as an example. + +```sh +$ kubectl get databaseobserver obs-sample +NAME EXPORTERCONFIG STATUS +obs-sample default READY +``` + + +To obtain a more detailed status, use the following command as an example: + +```bash +kubectl describe databaseobserver obs-sample +``` + +This provides details of the current state of your databaseObserver resource object. A successful +deployment of the databaseObserver resource object should display `READY` as the status and all conditions with a `True` +value for every ConditionType. + + +### Patch Resource +The Observability controller currently supports updates for most of the fields in the manifest. An example of patching the databaseObserver resource is as follows: +```bash +kubectl --type=merge -p '{"spec":{"exporter":{"image":"container-registry.oracle.com/database/observability-exporter:latest"}}}' patch databaseobserver obs-sample +``` + +The fields listed below can be updated with the given example command: + +* spec.exporter.image +* spec.exporter.configuration.configmap.configmapName +* spec.exporter.configuration.configmap.key +* spec.database.dbUser.secret +* spec.database.dbPassword.secret +* spec.database.dbConnectionString.secret +* spec.database.dbWallet.secret +* spec.ociConfig.configMapName +* spec.ociConfig.secretName +* spec.replicas +* spec.database.dbPassword.vaultOCID +* spec.database.dbPassword.vaultSecretName + + +### Delete Resource + +To delete the DatabaseObserver custom resource and all related resources: + +```bash +kubectl delete databaseobserver obs-sample +``` + +## Mandatory roles and privileges requirements for Observability Controller + +The Observability controller issues the following policy rules for the following resources. Besides +databaseobserver resources, the controller manages its own service, deployment, pods and servicemonitor +and gets and lists configmaps and secrets. + +| Resources | Verbs | +|-------------------------------------------------------|-------------------------------------------| +| services | create delete get list patch update watch | +| deployments | create delete get list patch update watch | +| pods | create delete get list patch update watch | +| events | create delete get list patch update watch | +| services.apps | create delete get list patch update watch | +| deployments.apps | create delete get list patch update watch | +| pods.apps | create delete get list patch update watch | +| servicemonitors.monitoring.coreos.com | create delete get list patch update watch | +| databaseobservers.observability.oracle.com | create delete get list patch update watch | +| databaseobservers.observability.oracle.com/status | get patch update | +| configmaps | get list | +| secrets | get list | +| configmaps.apps | get list | +| databaseobservers.observability.oracle.com/finalizers | update | + +## Debugging and troubleshooting + +### Show the details of the resource +To get the verbose output of the current spec, use the command below: + +```sh +kubectl describe databaseobserver/database-observer-sample +``` + +If any error occurs during the reconciliation loop, the Operator either reports +the error using the resource's event stream, or will show the error under conditions. + +### Check the logs of the pod where the operator deploys +Follow the steps to check the logs. + +1. List the pod replicas + + ```sh + kubectl get pods -n oracle-database-operator-system + ``` + +2. Use the below command to check the logs of the deployment + + ```sh + kubectl logs deployment.apps/oracle-database-operator-controller-manager -n oracle-database-operator-system + ``` diff --git a/docs/sharding/README.md b/docs/sharding/README.md index beb155d6..3d3320ee 100644 --- a/docs/sharding/README.md +++ b/docs/sharding/README.md @@ -26,16 +26,16 @@ The Oracle Sharding database controller provides end-to-end automation of Oracle To create a Sharding Topology, complete the steps in the following sections below: -1. [Prerequsites for running Oracle Sharding Database Controller](#prerequsites-for-running-oracle-sharding-database-controller) +1. [Prerequisites for running Oracle Sharding Database Controller](#prerequisites-for-running-oracle-sharding-database-controller) 2. [Provisioning Sharding Topology in a Cloud based Kubernetes Cluster (OKE in this case)](#provisioning-sharding-topology-in-a-cloud-based-kubernetes-cluster-oke-in-this-case) 3. [Connecting to Shard Databases](#connecting-to-shard-databases) 4. [Debugging and Troubleshooting](#debugging-and-troubleshooting) **Note** Before proceeding to the next section, you must complete the instructions given in each section, based on your enviornment, before proceeding to next section. -## Prerequsites for Running Oracle Sharding Database Controller +## Prerequisites for Running Oracle Sharding Database Controller -**IMPORTANT :** You must make the changes specified in this section before you proceed to the next section. +**IMPORTANT:** You must make the changes specified in this section before you proceed to the next section. ### 1. Kubernetes Cluster: To deploy Oracle Sharding database controller with Oracle Database Operator, you need a Kubernetes Cluster which can be one of the following: @@ -44,9 +44,24 @@ To create a Sharding Topology, complete the steps in the following sections belo To use Oracle Sharding Database Controller, ensure that your system is provisioned with a supported Kubernetes release. Refer to the [Release Status Section](../../README.md#release-status). +#### Mandatory roles and privileges requirements for Oracle Sharding Database Controller + + Oracle Sharding Database Controller uses Kubernetes objects such as :- + + | Resources | Verbs | + | --- | --- | + | Pods | create delete get list patch update watch | + | Containers | create delete get list patch update watch | + | PersistentVolumeClaims | create delete get list patch update watch | + | Services | create delete get list patch update watch | + | Secrets | create delete get list patch update watch | + | Events | create patch | + ### 2. Deploy Oracle Database Operator -To deploy Oracle Database Operator in a Kubernetes cluster, go to the section [Quick Install of the Operator](../../README.md#oracle-database-kubernetes-operator-deployment) in the README, and complete the operator deployment before you proceed further. If you have already deployed the operator, then proceed to the next section. +To deploy Oracle Database Operator in a Kubernetes cluster, go to the section [Install Oracle DB Operator](../../README.md#install-oracle-db-operator) in the README, and complete the operator deployment before you proceed further. If you have already deployed the operator, then proceed to the next section. + +**IMPORTANT:** Make sure you have completed the steps for [Role Binding for access management](../../README.md#role-binding-for-access-management) as well before installing the Oracle DB Operator. ### 3. Oracle Database and Global Data Services Docker Images Choose one of the following deployment options: @@ -84,22 +99,46 @@ You can either download the images and push them to your Docker Images Repositor ### 5. Create a Kubernetes secret for the database installation owner for the database Sharding Deployment -Create a Kubernetes secret named `db-user-pass` using these steps: [Create Kubernetes Secret](./provisioning/create_kubernetes_secret_for_db_user.md) +Create a Kubernetes secret named `db-user-pass-rsa` using these steps: [Create Kubernetes Secret](./provisioning/create_kubernetes_secret_for_db_user.md) After you have the above prerequsites completed, you can proceed to the next section for your environment to provision the Oracle Database Sharding Topology. -## Provisioning Sharding Topology in a Cloud-Based Kubernetes Cluster (OKE in this case) +### 6. Provisioning a Persistent Volume having an Oracle Database Gold Image + +This step is needed when you want to provision a Persistent Volume having an Oracle Database Gold Image for Database Cloning. + +In case of an `OCI OKE` cluster, you can use this Persistent Volume during provisioning Shard Databases by cloning in the same Availability Domain or you can use a Full Backup of this Persistent Volume during provisioning Shard Databases by cloning in different Availability Domains. + +You can refer [here](./provisioning/provisioning_persistent_volume_having_db_gold_image.md) for the steps involved. + +## Provisioning Sharding Topology with System Sharding in a Cloud-Based Kubernetes Cluster + +Deploy Oracle Database Sharding Topology with `System Sharding` on your Cloud based Kubernetes cluster. + +In this example, the deployment uses the YAML file based on `OCI OKE` cluster. There are multiple use case possible for deploying the Oracle Database Sharding Topology covered by below examples: + +[1. Provisioning Oracle Sharded Database with System Sharding without Database Gold Image](./provisioning/system_sharding/ssharding_provisioning_without_db_gold_image.md) +[2. Provisioning Oracle Sharded Database with System Sharding with additional control on resources like Memory and CPU allocated to Pods](./provisioning/system_sharding/ssharding_provisioning_with_control_on_resources.md) +[3. Provisioning Oracle Sharded Database with System Sharding by cloning database from your own Database Gold Image in the same Availability Domain(AD)](./provisioning/system_sharding/ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md) +[4. Provisioning Oracle Sharded Database with System Sharding by cloning database from your own Database Gold Image across Availability Domains(ADs)](./provisioning/system_sharding/ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md) +[5. Provisioning Oracle Sharded Database with System Sharding and send Notification using OCI Notification Service](./provisioning/system_sharding/ssharding_provisioning_with_notification_using_oci_notification.md) +[6. Scale Out - Add Shards to an existing Oracle Sharded Database provisioned earlier with System Sharding](./provisioning/system_sharding/ssharding_scale_out_add_shards.md) +[7. Scale In - Delete an existing Shard from a working Oracle Sharded Database provisioned earlier with System Sharding](./provisioning/system_sharding/ssharding_scale_in_delete_an_existing_shard.md) + + +## Provisioning Sharding Topology with User Defined Sharding in a Cloud-Based Kubernetes Cluster + +Deploy Oracle Database Sharding Topology with `User Defined Sharding` on your Cloud based Kubernetes cluster. -Deploy Oracle Database sharding topology on your Cloud based Kubernetes cluster. In this example, the deployment uses the YAML file based on `OCI OKE` cluster. There are multiple use case possible for deploying the Oracle Database sharding topology. +In this example, the deployment uses the YAML file based on `OCI OKE` cluster. There are multiple use case possible for deploying the Oracle Database Sharding Topology covered by below examples: -[1. Provisioning Oracle Database sharding topology without Database Gold Image](./provisioning/provisioning_without_db_gold_image.md) -[2. Provisioning Oracle Database sharding topology with additional control on resources like Memory and CPU allocated to Pods](./provisioning/provisioning_with_control_on_resources.md) -[3. Provisioning a Persistent Volume having an Oracle Database Gold Image](./provisioning/provisioning_persistent_volume_having_db_gold_image.md) -[4. Provisioning Oracle Database sharding topology by cloning database from your own Database Gold Image in the same Availability Domain(AD)](./provisioning/provisioning_by_cloning_db_gold_image_in_same_ad.md) -[5. Provisioning Oracle Database sharding topology by cloning database from your own Database Gold Image across Availability Domains(ADs)](./provisioning/provisioning_by_cloning_db_from_gold_image_across_ads.md) -[6. Provisioning Oracle Database sharding topology and send Notification using OCI Notification Service](./provisioning/provisioning_with_notification_using_oci_notification.md) -[7. Scale Out - Add Shards to an existing Oracle Database Sharding Topology](./provisioning/scale_out_add_shards.md) -[8. Scale In - Delete an existing Shard from a working Oracle Database sharding topology](./provisioning/scale_in_delete_an_existing_shard.md) +[1. Provisioning Oracle Sharded Database with User Defined Sharding without Database Gold Image](./provisioning/user-defined-sharding/udsharding_provisioning_without_db_gold_image.md) +[2. Provisioning Oracle Sharded Database with User Defined Sharding with additional control on resources like Memory and CPU allocated to Pods](./provisioning/user-defined-sharding/udsharding_provisioning_with_control_on_resources.md) +[3. Provisioning Oracle Sharded Database with User Defined Sharding by cloning database from your own Database Gold Image in the same Availability Domain(AD)](./provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_gold_image_in_same_ad.md) +[4. Provisioning Oracle Sharded Database with User Defined Sharding by cloning database from your own Database Gold Image across Availability Domains(ADs)](./provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_from_gold_image_across_ads.md) +[5. Provisioning Oracle Sharded Database with User Defined Sharding and send Notification using OCI Notification Service](./provisioning/user-defined-sharding/udsharding_provisioning_with_notification_using_oci_notification.md) +[6. Scale Out - Add Shards to an existing Oracle Sharded Database provisioned earlier with User Defined Sharding](./provisioning/user-defined-sharding/udsharding_scale_out_add_shards.md) +[7. Scale In - Delete an existing Shard from a working Oracle Sharded Database provisioned earlier with User Defined Sharding](./provisioning/user-defined-sharding/udsharding_scale_in_delete_an_existing_shard.md) ## Connecting to Shard Databases diff --git a/docs/sharding/provisioning/create_kubernetes_secret_for_db_user.md b/docs/sharding/provisioning/create_kubernetes_secret_for_db_user.md index 0d66f49b..99620f04 100644 --- a/docs/sharding/provisioning/create_kubernetes_secret_for_db_user.md +++ b/docs/sharding/provisioning/create_kubernetes_secret_for_db_user.md @@ -1,25 +1,46 @@ # Create kubernetes secret for db user -Create a Kubernetes secret named "db-user-pass" using a password in a text file and then encrypt it using an `openssl` key. The text file will be removed after secret is created. +Below are the steps to create an encrypted file with a password for the DB User: + +- Create a text file which is having the password which you want to use for the DB user. +- Create an RSA key pair using `openssl`. +- Encrypt the text file with password using `openssl` with the RSA key pair generated earlier. +- Remove the initial text file. +- Create the Kubernetes secret named `db-user-pass-rsa` using the encrypted file. + +Please refer the below example for the above steps: ```sh +# Create a directory for files for the secret: +rm -rf /tmp/.secrets/ mkdir /tmp/.secrets/ -# Generate a random openssl key -openssl rand -hex 64 -out /tmp/.secrets/pwd.key +# Create directories and initialize the variables +RSADIR="/tmp/.secrets" +PRIVKEY="${RSADIR}"/"key.pem" +PUBKEY="${RSADIR}"/"key.pub" +NAMESPACE="shns" +PWDFILE="${RSADIR}"/"pwdfile.txt" +PWDFILE_ENC="${RSADIR}"/"pwdfile.enc" +SECRET_NAME="db-user-pass-rsa" + +# Generate the RSA Key +openssl genrsa -out "${RSADIR}"/key.pem +openssl rsa -in "${RSADIR}"/key.pem -out "${RSADIR}"/key.pub -pubout -# Use a password you want and add it to a text file -echo ORacle_21c > /tmp/.secrets/common_os_pwdfile +# Create a text file with the password +rm -f $PWDFILE_ENC +echo ORacle_23c > ${RSADIR}/pwdfile.txt -# Encrypt the file with the password with the random openssl key generated above -openssl enc -aes-256-cbc -md md5 -salt -in /tmp/.secrets/common_os_pwdfile -out /tmp/.secrets/common_os_pwdfile.enc -pass file:/tmp/.secrets/pwd.key +# Create encrypted file from the text file using the RSA key +openssl pkeyutl -in $PWDFILE -out $PWDFILE_ENC -pubin -inkey $PUBKEY -encrypt -# Remove the password text file -rm -f /tmp/.secrets/common_os_pwdfile +# Remove the initial text file: +rm -f $PWDFILE -# Create the Kubernetes secret in namespace "shns" -kubectl create secret generic db-user-pass --from-file=/tmp/.secrets/common_os_pwdfile.enc --from-file=/tmp/.secrets/pwd.key -n shns +# Deleting the existing secret if existing +kubectl delete secret $SECRET_NAME -n $NAMESPACE -# Check the secret details -kubectl get secret -n shns -``` +# Create the Kubernetes secret in namespace "NAMESPACE" +kubectl create secret generic $SECRET_NAME --from-file=$PWDFILE_ENC --from-file=${PRIVKEY} -n $NAMESPACE +``` \ No newline at end of file diff --git a/docs/sharding/provisioning/oraclesi.yaml b/docs/sharding/provisioning/oraclesi.yaml index 2e0607b7..cac70ffa 100644 --- a/docs/sharding/provisioning/oraclesi.yaml +++ b/docs/sharding/provisioning/oraclesi.yaml @@ -1,14 +1,14 @@ # -# Copyright (c) 2022, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # apiVersion: v1 kind: PersistentVolumeClaim metadata: - name: oshard-gold-image-pvc19c + name: oshard-gold-image-pvc21c namespace: shns labels: - app: oshard19cdb-dep + app: oshard21cdb-dep spec: accessModes: - ReadWriteOnce @@ -18,28 +18,28 @@ spec: storageClassName: oci selector: matchLabels: - failure-domain.beta.kubernetes.io/zone: "EU-FRANKFURT-1-AD-1" + topology.kubernetes.io/zone: "PHX-AD-1" --- apiVersion: apps/v1 kind: StatefulSet metadata: - name: oshard19cdb + name: oshard21cdb namespace: shns labels: - app: oshard19cdb-dep + app: oshard21cdb-dep spec: selector: matchLabels: - app: oshard19cdb-dep + app: oshard21cdb-dep serviceName: gold-shard template: metadata: labels: - app: oshard19cdb-dep + app: oshard21cdb-dep spec: containers: - image: container-registry.oracle.com/database/enterprise:latest - name: oshard19cdb + name: oshard21cdb ports: - containerPort: 1521 name: db1-dbport @@ -68,17 +68,17 @@ spec: volumes: - name: data persistentVolumeClaim: - claimName: oshard-gold-image-pvc19c + claimName: oshard-gold-image-pvc21c - name: dshm emptyDir: medium: Memory nodeSelector: - failure-domain.beta.kubernetes.io/zone: "EU-FRANKFURT-1-AD-1" + topology.kubernetes.io/zone: "PHX-AD-1" --- apiVersion: v1 kind: Service metadata: - name: oshard19cdb + name: oshard21cdb namespace: shns spec: ports: @@ -95,5 +95,4 @@ spec: port: 6234 targetPort: db1-onsrport selector: - app: oshard19cdb-dep - + app: oshard21cdb-dep \ No newline at end of file diff --git a/docs/sharding/provisioning/oraclesi_pvc_commented.yaml b/docs/sharding/provisioning/oraclesi_pvc_commented.yaml index 89707a08..43b50a5e 100644 --- a/docs/sharding/provisioning/oraclesi_pvc_commented.yaml +++ b/docs/sharding/provisioning/oraclesi_pvc_commented.yaml @@ -1,46 +1,45 @@ # -# Copyright (c) 2022, Oracle and/or its affiliates. +# Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # - -# apiVersion: v1 -# kind: PersistentVolumeClaim -# metadata: -# name: oshard-gold-image-pvc19c -# namespace: shns -# labels: -# app: oshard19cdb-dep -# spec: -# accessModes: -# - ReadWriteOnce -# resources: -# requests: -# storage: 50Gi -# storageClassName: oci -# selector: -# matchLabels: -# failure-domain.beta.kubernetes.io/zone: "EU-FRANKFURT-1-AD-1" -# --- +#apiVersion: v1 +#kind: PersistentVolumeClaim +#metadata: +# name: oshard-gold-image-pvc21c +# namespace: shns +# labels: +# app: oshard21cdb-dep +#spec: +# accessModes: +# - ReadWriteOnce +# resources: +# requests: +# storage: 50Gi +# storageClassName: oci +# selector: +# matchLabels: +# topology.kubernetes.io/zone: "PHX-AD-1" +#--- apiVersion: apps/v1 kind: StatefulSet metadata: - name: oshard19cdb + name: oshard21cdb namespace: shns labels: - app: oshard19cdb-dep + app: oshard21cdb-dep spec: selector: matchLabels: - app: oshard19cdb-dep + app: oshard21cdb-dep serviceName: gold-shard template: metadata: labels: - app: oshard19cdb-dep + app: oshard21cdb-dep spec: containers: - image: container-registry.oracle.com/database/enterprise:latest - name: oshard19cdb + name: oshard21cdb ports: - containerPort: 1521 name: db1-dbport @@ -69,17 +68,17 @@ spec: volumes: - name: data persistentVolumeClaim: - claimName: oshard-gold-image-pvc19c + claimName: oshard-gold-image-pvc21c - name: dshm emptyDir: medium: Memory nodeSelector: - failure-domain.beta.kubernetes.io/zone: "EU-FRANKFURT-1-AD-1" + topology.kubernetes.io/zone: "PHX-AD-1" --- apiVersion: v1 kind: Service metadata: - name: oshard19cdb + name: oshard21cdb namespace: shns spec: ports: @@ -96,5 +95,4 @@ spec: port: 6234 targetPort: db1-onsrport selector: - app: oshard19cdb-dep - + app: oshard21cdb-dep \ No newline at end of file diff --git a/docs/sharding/provisioning/provisioning_persistent_volume_having_db_gold_image.md b/docs/sharding/provisioning/provisioning_persistent_volume_having_db_gold_image.md index 581fc62c..7d3312a6 100644 --- a/docs/sharding/provisioning/provisioning_persistent_volume_having_db_gold_image.md +++ b/docs/sharding/provisioning/provisioning_persistent_volume_having_db_gold_image.md @@ -9,7 +9,7 @@ This example uses file `oraclesi.yaml` to provision a single instance Oracle Dat * A Persistent Volume Claim * Repository location for Database Docker Image: `image: container-registry.oracle.com/database/enterprise:latest` * Namespace: `shns` -* Tag `nodeSelector` to deploy the Single Oracle Database in AD `EU-FRANKFURT-1-AD-1` +* Tag `nodeSelector` to deploy the Single Oracle Database in AD `PHX-AD-1` In this example, we are using pre-built Oracle Database image available on [Oracle Container Registry](https://container-registry.oracle.com/) * To pull the above image from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. diff --git a/docs/sharding/provisioning/shard_prov.yaml b/docs/sharding/provisioning/shard_prov.yaml deleted file mode 100644 index 032abd68..00000000 --- a/docs/sharding/provisioning/shard_prov.yaml +++ /dev/null @@ -1,46 +0,0 @@ -# -# Copyright (c) 2022, Oracle and/or its affiliates. -# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. -# -apiVersion: database.oracle.com/v1alpha1 -kind: ShardingDatabase -metadata: - name: shardingdatabase-sample - namespace: shns -spec: - shard: - - name: shard1 - storageSizeInGb: 50 - - name: shard2 - storageSizeInGb: 50 - catalog: - - name: catalog - storageSizeInGb: 50 - gsm: - - name: gsm1 - storageSizeInGb: 50 - replicas: 1 - envVars: - - name: "SERVICE1_PARAMS" - value: "service_name=oltp_rw_svc;service_role=primary" - - name: "SERVICE2_PARAMS" - value: "service_name=oltp_ro_svc;service_role=primary" - - name: gsm2 - storageSizeInGb: 50 - replicas: 1 - envVars: - - name: "SERVICE1_PARAMS" - value: "service_name=oltp_rw_svc;service_role=primary" - - name: "SERVICE2_PARAMS" - value: "service_name=oltp_ro_svc;service_role=primary" - storageClass: oci - dbImage: container-registry.oracle.com/database/enterprise:latest - dbImagePullSecret: ocr-reg-cred - gsmImage: container-registry.oracle.com/database/gsm:latest - gsmImagePullSecret: ocr-reg-cred - scriptsLocation: "set -ex;curl https://codeload.github.com/oracle/db-sharding/tar.gz/master | tar -xz --strip=4 db-sharding-master/docker-based-sharding-deployment/dockerfiles/21.3.0/scripts; cp -i -r scripts/* /opt/oracle/scripts/sharding/scripts/;cp -i -r scripts/*py /opt/oracle/scripts/sharding" - secret: db-user-pass - isExternalSvc: false - isDeleteOraPvc: True - namespace: shns - diff --git a/docs/sharding/provisioning/shard_prov_clone.yaml b/docs/sharding/provisioning/shard_prov_clone.yaml deleted file mode 100644 index f9d3f826..00000000 --- a/docs/sharding/provisioning/shard_prov_clone.yaml +++ /dev/null @@ -1,64 +0,0 @@ -# -# Copyright (c) 2022, Oracle and/or its affiliates. -# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. -# -apiVersion: database.oracle.com/v1alpha1 -kind: ShardingDatabase -metadata: - name: shardingdatabase-sample - namespace: shns -spec: - shard: - - name: shard1 - storageSizeInGb: 50 - nodeSelector: - "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-1" - pvMatchLabels: - "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-1" - pvAnnotations: - volume.beta.kubernetes.io/oci-volume-source: ocid1.volume.oc1.eu-frankfurt-1.abtheljtmwcwf7liuhaibzgdcoxqcwwfpsqiqlsumrjlzkin7y4zx3x2idua - - name: shard2 - storageSizeInGb: 50 - nodeSelector: - "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-1" - pvMatchLabels: - "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-1" - pvAnnotations: - volume.beta.kubernetes.io/oci-volume-source: ocid1.volume.oc1.eu-frankfurt-1.abtheljtmwcwf7liuhaibzgdcoxqcwwfpsqiqlsumrjlzkin7y4zx3x2idua - catalog: - - name: catalog - storageSizeInGb: 50 - nodeSelector: - "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-1" - pvMatchLabels: - "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-1" - pvAnnotations: - volume.beta.kubernetes.io/oci-volume-source: ocid1.volume.oc1.eu-frankfurt-1.abtheljtmwcwf7liuhaibzgdcoxqcwwfpsqiqlsumrjlzkin7y4zx3x2idua - gsm: - - name: gsm1 - storageSizeInGb: 50 - replicas: 1 - envVars: - - name: "SERVICE1_PARAMS" - value: "service_name=oltp_rw_svc;service_role=primary" - - name: "SERVICE2_PARAMS" - value: "service_name=oltp_ro_svc;service_role=primary" - - name: gsm2 - storageSizeInGb: 50 - replicas: 1 - envVars: - - name: "SERVICE1_PARAMS" - value: "service_name=oltp_rw_svc;service_role=primary" - - name: "SERVICE2_PARAMS" - value: "service_name=oltp_ro_svc;service_role=primary" - storageClass: oci - dbImage: container-registry.oracle.com/database/enterprise:latest - dbImagePullSecret: ocr-reg-cred - gsmImage: container-registry.oracle.com/database/gsm:latest - gsmImagePullSecret: ocr-reg-cred - scriptsLocation: "set -ex;curl https://codeload.github.com/oracle/db-sharding/tar.gz/master | tar -xz --strip=4 db-sharding-master/docker-based-sharding-deployment/dockerfiles/21.3.0/scripts; cp -i -r scripts/* /opt/oracle/scripts/sharding/scripts/;cp -i -r scripts/*py /opt/oracle/scripts/sharding" - secret: db-user-pass - isExternalSvc: false - isClone: True - isDeleteOraPvc: True - namespace: shns diff --git a/docs/sharding/provisioning/shard_prov_clone_across_ads.yaml b/docs/sharding/provisioning/shard_prov_clone_across_ads.yaml deleted file mode 100644 index 6b815541..00000000 --- a/docs/sharding/provisioning/shard_prov_clone_across_ads.yaml +++ /dev/null @@ -1,72 +0,0 @@ -# -# Copyright (c) 2022, Oracle and/or its affiliates. -# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. -# -apiVersion: database.oracle.com/v1alpha1 -kind: ShardingDatabase -metadata: - name: shardingdatabase-sample - namespace: shns -spec: - shard: - - name: shard1 - storageSizeInGb: 50 - nodeSelector: - "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-2" - pvMatchLabels: - "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-2" - pvAnnotations: - volume.beta.kubernetes.io/oci-volume-source: ocid1.volumebackup.oc1.eu-frankfurt-1.abtheljtjlc7oce3sgq55vnskb4sjdip5sdaighm54hpmlcg7avgc76pjbea - - name: shard2 - storageSizeInGb: 50 - nodeSelector: - "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-1" - pvMatchLabels: - "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-1" - pvAnnotations: - volume.beta.kubernetes.io/oci-volume-source: ocid1.volumebackup.oc1.eu-frankfurt-1.abtheljtjlc7oce3sgq55vnskb4sjdip5sdaighm54hpmlcg7avgc76pjbea - catalog: - - name: catalog - storageSizeInGb: 50 - nodeSelector: - "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-2" - pvMatchLabels: - "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-2" - pvAnnotations: - volume.beta.kubernetes.io/oci-volume-source: ocid1.volumebackup.oc1.eu-frankfurt-1.abtheljtjlc7oce3sgq55vnskb4sjdip5sdaighm54hpmlcg7avgc76pjbea - gsm: - - name: gsm1 - storageSizeInGb: 50 - replicas: 1 - envVars: - - name: "SERVICE1_PARAMS" - value: "service_name=oltp_rw_svc;service_role=primary" - - name: "SERVICE2_PARAMS" - value: "service_name=oltp_ro_svc;service_role=primary" - nodeSelector: - "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-2" - pvMatchLabels: - "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-2" - - name: gsm2 - storageSizeInGb: 50 - replicas: 1 - envVars: - - name: "SERVICE1_PARAMS" - value: "service_name=oltp_rw_svc;service_role=primary" - - name: "SERVICE2_PARAMS" - value: "service_name=oltp_ro_svc;service_role=primary" - nodeSelector: - "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-3" - pvMatchLabels: - "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-3" - storageClass: oci - dbImage: container-registry.oracle.com/database/enterprise:latest - dbImagePullSecret: ocr-reg-cred - gsmImage: container-registry.oracle.com/database/gsm:latest - gsmImagePullSecret: ocr-reg-cred - scriptsLocation: "set -ex;curl https://codeload.github.com/oracle/db-sharding/tar.gz/master | tar -xz --strip=4 db-sharding-master/docker-based-sharding-deployment/dockerfiles/21.3.0/scripts; cp -i -r scripts/* /opt/oracle/scripts/sharding/scripts/;cp -i -r scripts/*py /opt/oracle/scripts/sharding" - secret: db-user-pass - isExternalSvc: false - isClone: True - isDeleteOraPvc: True - namespace: shns diff --git a/docs/sharding/provisioning/shard_prov_send_notification.yaml b/docs/sharding/provisioning/shard_prov_send_notification.yaml deleted file mode 100644 index 19f3c02c..00000000 --- a/docs/sharding/provisioning/shard_prov_send_notification.yaml +++ /dev/null @@ -1,75 +0,0 @@ -# -# Copyright (c) 2022, Oracle and/or its affiliates. -# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. -# -apiVersion: database.oracle.com/v1alpha1 -kind: ShardingDatabase -metadata: - name: shardingdatabase-sample - namespace: shns -spec: - shard: - - name: shard1 - storageSizeInGb: 50 - nodeSelector: - "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-2" - pvMatchLabels: - "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-2" - pvAnnotations: - volume.beta.kubernetes.io/oci-volume-source: ocid1.volumebackup.oc1.eu-frankfurt-1.abtheljtjlc7oce3sgq55vnskb4sjdip5sdaighm54hpmlcg7avgc76pjbea - - name: shard2 - storageSizeInGb: 50 - nodeSelector: - "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-1" - pvMatchLabels: - "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-1" - pvAnnotations: - volume.beta.kubernetes.io/oci-volume-source: ocid1.volumebackup.oc1.eu-frankfurt-1.abtheljtjlc7oce3sgq55vnskb4sjdip5sdaighm54hpmlcg7avgc76pjbea - catalog: - - name: catalog - storageSizeInGb: 50 - nodeSelector: - "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-2" - pvMatchLabels: - "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-2" - pvAnnotations: - volume.beta.kubernetes.io/oci-volume-source: ocid1.volumebackup.oc1.eu-frankfurt-1.abtheljtjlc7oce3sgq55vnskb4sjdip5sdaighm54hpmlcg7avgc76pjbea - gsm: - - name: gsm1 - storageSizeInGb: 50 - replicas: 1 - envVars: - - name: "SERVICE1_PARAMS" - value: "service_name=oltp_rw_svc;service_role=primary" - - name: "SERVICE2_PARAMS" - value: "service_name=oltp_ro_svc;service_role=primary" - nodeSelector: - "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-2" - pvMatchLabels: - "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-2" - - name: gsm2 - storageSizeInGb: 50 - replicas: 1 - envVars: - - name: "SERVICE1_PARAMS" - value: "service_name=oltp_rw_svc;service_role=primary" - - name: "SERVICE2_PARAMS" - value: "service_name=oltp_ro_svc;service_role=primary" - nodeSelector: - "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-3" - pvMatchLabels: - "failure-domain.beta.kubernetes.io/zone": "EU-FRANKFURT-1-AD-3" - storageClass: oci - dbImage: container-registry.oracle.com/database/enterprise:latest - dbImagePullSecret: ocr-reg-cred - gsmImage: container-registry.oracle.com/database/gsm:latest - gsmImagePullSecret: ocr-reg-cred - scriptsLocation: "set -ex;curl https://codeload.github.com/oracle/db-sharding/tar.gz/master | tar -xz --strip=4 db-sharding-master/docker-based-sharding-deployment/dockerfiles/21.3.0/scripts; cp -i -r scripts/* /opt/oracle/scripts/sharding/scripts/;cp -i -r scripts/*py /opt/oracle/scripts/sharding" - secret: db-user-pass - isExternalSvc: false - isClone: True - isDeleteOraPvc: True - namespace: shns - nsConfigMap: onsconfigmap - nsSecret: my-secret - diff --git a/docs/sharding/provisioning/provisioning_by_cloning_db_from_gold_image_across_ads.md b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md similarity index 57% rename from docs/sharding/provisioning/provisioning_by_cloning_db_from_gold_image_across_ads.md rename to docs/sharding/provisioning/system_sharding/ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md index 75a6f743..9ae05d50 100644 --- a/docs/sharding/provisioning/provisioning_by_cloning_db_from_gold_image_across_ads.md +++ b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md @@ -1,6 +1,8 @@ -# Provisioning Oracle Database Sharding Topology by Cloning the Database from Your Own Database Gold Image Across Availability Domains (ADs) +# Provisioning Oracle Sharded Database with System Sharding by cloning database from your own Database Gold Image across Availability Domains(ADs) -In this test case, you provision the Oracle Database sharding topology while provisioning the Catalog and Shard Databases by cloning from an existing Oracle Database Gold Image created earlier. +**IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. + +In this test case, you provision the Oracle Database sharding topology with System Sharding while provisioning the Catalog and Shard Databases by cloning from an existing Oracle Database Gold Image created earlier. This use case applies when you want to provision the database Pods on a Kubernetes Node in any availability domain (AD), which can also be different from the availability domain (AD) of the Block Volume that has the Oracle Database Gold Image provisioned earlier. @@ -17,25 +19,28 @@ NOTE: ```sh kubectl get pv -n shns ``` -2. Create a Block Volume Backup for this Block Volume, and use the OCID of the Block Volume Backup in the next step. This example uses `shard_prov_clone_across_ads.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: +2. Create a Block Volume Backup for this Block Volume, and use the OCID of the Block Volume Backup in the next step. This example uses `ssharding_shard_prov_clone_across_ads.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Two sharding Pods: `shard1` and `shard2` +* Three sharding Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` * Database Cloning from the `BLOCK VOLUME FULL BACKUP` of the Persistent Volume which had the Gold Image. -* OCID of the Block Volume Backup: `ocid1.volumebackup.oc1.eu-frankfurt-1.abtheljtjlc7oce3sgq55vnskb4sjdip5sdaighm54hpmlcg7avgc76pjbea` +* OCID of the Block Volume Backup: `ocid1.volumebackup.oc1.phx.abyhqljrxtv7tu5swqb3lzc7vpzwbwzdktd2y4k2vjjy2srmgu2w7bqdftjq` + +NOTE: In this case, the Persistent Volume with DB Gold Image was provisioned in the Availability Domain `PHX-AD-1`. The Shards and Catalog will be provisioned across multiple Availability Domains by cloning the database. In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. - * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `shard_prov_clone_across_ads.yaml`. - * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../README.md#3-oracle-database-and-global-data-services-docker-images) + * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `ssharding_shard_prov_clone_across_ads.yaml`. + * The `dbImage` used during provisioning the Persistent Volume with Database Gold Image and the `dbImage` used for deploying the Shard or Catalog Database by cloning should be same. + * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) -Use the file: [shard_prov_clone_across_ads.yaml](./shard_prov_clone_across_ads.yaml) for this use case as below: +Use the file: [ssharding_shard_prov_clone_across_ads.yaml](./ssharding_shard_prov_clone_across_ads.yaml) for this use case as below: -1. Deploy the `shard_prov_clone_across_ads.yaml` file: +1. Deploy the `ssharding_shard_prov_clone_across_ads.yaml` file: ```sh - kubectl apply -f shard_prov_clone_across_ads.yaml + kubectl apply -f ssharding_shard_prov_clone_across_ads.yaml ``` 2. Check the status of the deployment: ```sh diff --git a/docs/sharding/provisioning/provisioning_by_cloning_db_gold_image_in_same_ad.md b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md similarity index 54% rename from docs/sharding/provisioning/provisioning_by_cloning_db_gold_image_in_same_ad.md rename to docs/sharding/provisioning/system_sharding/ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md index 531c3839..cb67addb 100644 --- a/docs/sharding/provisioning/provisioning_by_cloning_db_gold_image_in_same_ad.md +++ b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md @@ -1,4 +1,6 @@ -# Provisioning Oracle Database Sharding Topology by Cloning the Database from Your Own Database Gold Image in the same Availability Domain (AD) +# Provisioning Oracle Sharded Database with System Sharding by cloning database from your own Database Gold Image in the same Availability Domain(AD) + +**IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. In this case, the database is created automatically by cloning from an existing Oracle Database Gold Image during the provisioning of the shard databases and the catalog database when the Oracle Sharding topology is deployed using Oracle Sharding controller. @@ -8,30 +10,33 @@ Choosing this option takes substantially less time during the Oracle Database Sh **NOTE** For this step, the Persistent Volume that has the Oracle Database Gold Image is identified using its OCID. -1. Check the OCID of the Persistent Volume provisioned by above step using below command: +1. Check the OCID of the Persistent Volume provisioned earlier using below command: ```sh kubectl get pv -n shns ``` -2. This example uses `shard_prov_clone.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: +2. This example uses `ssharding_shard_prov_clone.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Two sharding Pods: `shard1` and `shard2` +* Three sharding Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` -* Database Cloning from the Database Gold Image present in Persistent Volume having OCID: `ocid1.volume.oc1.eu-frankfurt-1.abtheljtmwcwf7liuhaibzgdcoxqcwwfpsqiqlsumrjlzkin7y4zx3x2idua` +* Database Cloning from the Database Gold Image present in Persistent Volume having OCID: `ocid1.volume.oc1.phx.abyhqljr3z3w72t6ay5eud7d5w3kdfhktfp6gwb6euy5tzwfaxgmbvwqlvsq` + +NOTE: In this case, the Persistent Volume with DB Gold Image was provisioned in the Availability Domain `PHX-AD-1`. The Shards and Catalog will be provisioned in the same Availability Domain `PHX-AD-1` by cloning the database. In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. - * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `shard_prov.yaml`. - * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../README.md#3-oracle-database-and-global-data-services-docker-images) + * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `ssharding_shard_prov_clone.yaml`. + * The `dbImage` used during provisioning the Persistent Volume with Database Gold Image and the `dbImage` used for deploying the Shard or Catalog Database by cloning should be same. + * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) -Use the file: [shard_prov_clone.yaml](./shard_prov_clone.yaml) for this use case as below: +Use the file: [ssharding_shard_prov_clone.yaml](./ssharding_shard_prov_clone.yaml) for this use case as below: -1. Deploy the `shard_prov_clone.yaml` file: +1. Deploy the `ssharding_shard_prov_clone.yaml` file: ```sh - kubectl apply -f shard_prov_clone.yaml + kubectl apply -f ssharding_shard_prov_clone.yaml ``` 2. Check the status of the deployment: ```sh diff --git a/docs/sharding/provisioning/provisioning_with_control_on_resources.md b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_control_on_resources.md similarity index 58% rename from docs/sharding/provisioning/provisioning_with_control_on_resources.md rename to docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_control_on_resources.md index 734cad89..e14e76a4 100644 --- a/docs/sharding/provisioning/provisioning_with_control_on_resources.md +++ b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_control_on_resources.md @@ -1,11 +1,13 @@ -# Provisioning Oracle Database Sharding Topology with Additional Control on Resources Allocated to Pods +# Provisioning Oracle Sharded Database with System Sharding with additional control on resources like Memory and CPU allocated to Pods -In this use case, there are additional tags used to control resources such as CPU and Memory used by the different Pods when the Oracle Sharding topology is deployed using Oracle Sharding controller. +**IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. -This example uses `shard_prov_memory_cpu.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: +In this use case, there are additional tags used to control resources such as CPU and Memory used by the different Pods when the Oracle Sharding topology with System Sharding is deployed using Oracle Sharding controller. + +This example uses `ssharding_shard_prov_memory_cpu.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Two sharding Pods: `shard1` and `shard2` +* Three sharding Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` * Tags `memory` and `cpu` to control the Memory and CPU of the PODs @@ -13,15 +15,15 @@ This example uses `shard_prov_memory_cpu.yaml` to provision an Oracle Database s In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. - * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `shard_prov_memory_cpu.yaml`. - * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../README.md#3-oracle-database-and-global-data-services-docker-images) + * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `ssharding_shard_prov_memory_cpu.yaml`. + * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) -Use the YAML file [shard_prov_memory_cpu.yaml](./shard_prov_memory_cpu.yaml). +Use the YAML file [ssharding_shard_prov_memory_cpu.yaml](./ssharding_shard_prov_memory_cpu.yaml). -1. Deploy the `shard_prov_memory_cpu.yaml` file: +1. Deploy the `ssharding_shard_prov_memory_cpu.yaml` file: ```sh - kubectl apply -f shard_prov_memory_cpu.yaml + kubectl apply -f ssharding_shard_prov_memory_cpu.yaml ``` 1. Check the details of a POD. For example: To check the details of Pod `shard1-0`: diff --git a/docs/sharding/provisioning/provisioning_with_notification_using_oci_notification.md b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_notification_using_oci_notification.md similarity index 71% rename from docs/sharding/provisioning/provisioning_with_notification_using_oci_notification.md rename to docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_notification_using_oci_notification.md index 7df7bc05..90a8f803 100644 --- a/docs/sharding/provisioning/provisioning_with_notification_using_oci_notification.md +++ b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_notification_using_oci_notification.md @@ -1,15 +1,17 @@ -# Provisioning Oracle Database Sharding Topology and Send Notification Using OCI Notification Service +# Provisioning Oracle Sharded Database with System Sharding and send Notification using OCI Notification Service -This use case demonstrates how to use a notification service like OCI Notification service to send an email notification when a particular operation is completed on an Oracle Database sharding topology provisioned using the Oracle Database sharding controller. +**IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. -This example uses `shard_prov_send_notification.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: +This use case demonstrates how to use a notification service like OCI Notification service to send an email notification when a particular operation is completed on an Oracle Database sharding topology provisioned using the Oracle Database sharding controller. + +This example uses `ssharding_shard_prov_send_notification.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Two sharding Pods: `shard1` and `shard2` +* Three sharding Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` * Database Cloning from the `BLOCK VOLUME FULL BACKUP` of the Persistent Volume that has the Database Gold Image created earlier. -* OCID of the Block Volume Backup: `ocid1.volumebackup.oc1.eu-frankfurt-1.abtheljtjlc7oce3sgq55vnskb4sjdip5sdaighm54hpmlcg7avgc76pjbea` +* OCID of the Block Volume Backup: `ocid1.volumebackup.oc1.phx.abyhqljrxtv7tu5swqb3lzc7vpzwbwzdktd2y4k2vjjy2srmgu2w7bqdftjq` * Configmap to send notification email when a particular operation is completed. For example: When a shard is added. **NOTE:** @@ -26,8 +28,8 @@ To do this: user=ocid1.user.oc1........fx7omxfq fingerprint=fa:18:98:...............:8a tenancy=ocid1.tenancy.oc1..aaaa.......orpn7inq - region=eu-frankfurt-1 - topicid=ocid1.onstopic.oc1.eu-frankfurt-1.aaa............6xrq + region=us-phoenix-1 + topicid=ocid1.onstopic.oc1.phx.aaa............6xrq ``` 2. Create a configmap using the below command using the file created above: ```sh @@ -61,14 +63,14 @@ To do this: In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. - * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `shard_prov.yaml`. - * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../README.md#3-oracle-database-and-global-data-services-docker-images) + * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `ssharding_shard_prov_send_notification.yaml`. + * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) -Use the file: [shard_prov_send_notification.yaml](./shard_prov_send_notification.yaml) for this use case as below: +Use the file: [ssharding_shard_prov_send_notification.yaml](./ssharding_shard_prov_send_notification.yaml) for this use case as below: -1. Deploy the `shard_prov_send_notification.yaml` file: +1. Deploy the `ssharding_shard_prov_send_notification.yaml` file: ```sh - kubectl apply -f shard_prov_send_notification.yaml + kubectl apply -f ssharding_shard_prov_send_notification.yaml ``` 2. Check the status of the deployment: ```sh diff --git a/docs/sharding/provisioning/provisioning_without_db_gold_image.md b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_without_db_gold_image.md similarity index 59% rename from docs/sharding/provisioning/provisioning_without_db_gold_image.md rename to docs/sharding/provisioning/system_sharding/ssharding_provisioning_without_db_gold_image.md index 0a908b52..8f10fd8c 100644 --- a/docs/sharding/provisioning/provisioning_without_db_gold_image.md +++ b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_without_db_gold_image.md @@ -1,28 +1,30 @@ -# Provisioning Oracle Database Sharding Topology Without Database Gold Image +# Provisioning Oracle Sharded Database with System Sharding without Database Gold Image -In this use case, the database is created automatically using DBCA during the provisioning of the shard databases and the catalog database when the Oracle Sharding topology is deployed using Oracle Sharding controller. +**IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. + +In this use case, the database is created automatically using DBCA during the provisioning of the shard databases and the catalog database when the Oracle Sharding topology with System Sharding is deployed using Oracle Sharding controller. **NOTE** In this use case, because DBCA creates the database automatically during the deployment, the time required to create the database is greater than the time it takes when the database is created by cloning from a Database Gold Image. -This example uses `shard_prov.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: +This example uses `ssharding_shard_prov.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Two sharding Pods: `shard1` and `shard2` +* Three sharding Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. - * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `shard_prov.yaml`. - * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../README.md#3-oracle-database-and-global-data-services-docker-images) + * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `ssharding_shard_prov.yaml`. + * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) -Use the file: [shard_prov.yaml](./shard_prov.yaml) for this use case as below: +Use the file: [ssharding_shard_prov.yaml](./ssharding_shard_prov.yaml) for this use case as below: -1. Deploy the `shard_prov.yaml` file: +1. Deploy the `ssharding_shard_prov.yaml` file: ```sh - kubectl apply -f shard_prov.yaml + kubectl apply -f ssharding_shard_prov.yaml ``` 1. Check the status of the deployment: ```sh diff --git a/docs/sharding/provisioning/scale_in_delete_an_existing_shard.md b/docs/sharding/provisioning/system_sharding/ssharding_scale_in_delete_an_existing_shard.md similarity index 56% rename from docs/sharding/provisioning/scale_in_delete_an_existing_shard.md rename to docs/sharding/provisioning/system_sharding/ssharding_scale_in_delete_an_existing_shard.md index 9334741b..4d5713d4 100644 --- a/docs/sharding/provisioning/scale_in_delete_an_existing_shard.md +++ b/docs/sharding/provisioning/system_sharding/ssharding_scale_in_delete_an_existing_shard.md @@ -1,6 +1,8 @@ -# Scale In - Delete an existing Shard From a Working Oracle Database Sharding Topology +# Scale In - Delete an existing Shard from a working Oracle Sharded Database provisioned earlier with System Sharding -This use case demonstrates how to delete an existing Shard from an existing Oracle Database sharding topology provisioned using Oracle Database Sharding controller. +**IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. + +This use case demonstrates how to delete an existing Shard from an existing Oracle Database sharding topology with System Sharding provisioned using Oracle Database Sharding controller. **NOTE** The deletion of a shard is done after verifying the Chunks have been moved out of that shard. @@ -13,25 +15,25 @@ In this use case, the existing database Sharding is having: In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. - * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `shard_prov.yaml`. - * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../README.md#3-oracle-database-and-global-data-services-docker-images) + * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `ssharding_shard_prov_delshard.yaml`. + * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) NOTE: Use tag `isDelete: true` to delete the shard you want. -This use case deletes the shard `shard2` from the above Sharding Topology. +This use case deletes the shard `shard4` from the above Sharding Topology. -Use the file: [shard_prov_delshard.yaml](./shard_prov_delshard.yaml) for this use case as below: +Use the file: [ssharding_shard_prov_delshard.yaml](./ssharding_shard_prov_delshard.yaml) for this use case as below: -1. Deploy the `shard_prov_delshard.yaml` file: +1. Deploy the `ssharding_shard_prov_delshard.yaml` file: ```sh - kubectl apply -f shard_prov_delshard.yaml + kubectl apply -f ssharding_shard_prov_delshard.yaml ``` 2. Check the status of the deployment: ```sh # Check the status of the Kubernetes Pods: kubectl get all -n shns -**NOTE:** After you apply `shard_prov_delshard.yaml`, the change may not be visible immediately. When the shard is removed, first the chunks will be moved out of that shard that is going to be deleted. +**NOTE:** After you apply `ssharding_shard_prov_delshard.yaml`, the change may not be visible immediately. When the shard is removed, first the chunks will be moved out of that shard that is going to be deleted. To monitor the chunk movement, use the following command: diff --git a/docs/sharding/provisioning/scale_out_add_shards.md b/docs/sharding/provisioning/system_sharding/ssharding_scale_out_add_shards.md similarity index 51% rename from docs/sharding/provisioning/scale_out_add_shards.md rename to docs/sharding/provisioning/system_sharding/ssharding_scale_out_add_shards.md index 174d8c6c..5c349847 100644 --- a/docs/sharding/provisioning/scale_out_add_shards.md +++ b/docs/sharding/provisioning/system_sharding/ssharding_scale_out_add_shards.md @@ -1,31 +1,33 @@ -# Scale Out - Add Shards to an existing Oracle Database Sharding Topology +# Scale Out - Add Shards to an existing Oracle Sharded Database provisioned earlier with System Sharding -This use case demonstrates adding a new shard to an existing Oracle Database sharding topology provisioned earlier using Oracle Database Sharding controller. +**IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. + +This use case demonstrates adding a new shard to an existing Oracle Database sharding topology with System Sharding provisioned earlier using Oracle Database Sharding controller. In this use case, the existing Oracle Database sharding topology is having: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Two sharding Pods: `shard1` and `shard2` +* Three sharding Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. - * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `shard_prov.yaml`. - * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../README.md#3-oracle-database-and-global-data-services-docker-images) + * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `ssharding_shard_prov_extshard.yaml`. + * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) -This use case adds three new shards `shard3`,`shard4`,`shard4` to above Sharding Topology. +This use case adds two new shards `shard4`,`shard5` to above Sharding Topology. -Use the file: [shard_prov_extshard.yaml](./shard_prov_extshard.yaml) for this use case as below: +Use the file: [ssharding_shard_prov_extshard.yaml](./ssharding_shard_prov_extshard.yaml) for this use case as below: -1. Deploy the `shard_prov_extshard.yaml` file: +1. Deploy the `ssharding_shard_prov_extshard.yaml` file: ```sh - kubectl apply -f shard_prov_extshard.yaml + kubectl apply -f ssharding_shard_prov_extshard.yaml ``` 2. Check the status of the deployment: ```sh # Check the status of the Kubernetes Pods: kubectl get all -n shns - # Check the logs of a particular pod. For example, to check status of pod "shard3-0": - kubectl logs -f pod/shard3-0 -n shns + # Check the logs of a particular pod. For example, to check status of pod "shard4-0": + kubectl logs -f pod/shard4-0 -n shns diff --git a/docs/sharding/provisioning/system_sharding/ssharding_shard_prov.yaml b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov.yaml new file mode 100644 index 00000000..f803ba42 --- /dev/null +++ b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov.yaml @@ -0,0 +1,58 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +--- +apiVersion: database.oracle.com/v1alpha1 +kind: ShardingDatabase +metadata: + name: shardingdatabase-sample + namespace: shns +spec: + shard: + - name: shard1 + storageSizeInGb: 50 + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + - name: shard2 + storageSizeInGb: 50 + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + - name: shard3 + storageSizeInGb: 50 + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + catalog: + - name: catalog + storageSizeInGb: 50 + imagePullPolicy: "Always" + gsm: + - name: gsm1 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: primary + - name: gsm2 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: standby + storageClass: oci + dbImage: container-registry.oracle.com/database/enterprise:latest + dbImagePullSecret: ocr-reg-cred + gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImagePullSecret: ocr-reg-cred + isExternalSvc: False + isDeleteOraPvc: True + dbSecret: + name: db-user-pass-rsa + pwdFileName: pwdfile.enc + keyFileName: key.pem + gsmService: + - name: oltp_rw_svc + role: primary + - name: oltp_ro_svc + role: primary + namespace: shns + diff --git a/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_clone.yaml b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_clone.yaml new file mode 100644 index 00000000..9a690626 --- /dev/null +++ b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_clone.yaml @@ -0,0 +1,82 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +--- +apiVersion: database.oracle.com/v1alpha1 +kind: ShardingDatabase +metadata: + name: shardingdatabase-sample + namespace: shns +spec: + shard: + - name: shard1 + storageSizeInGb: 50 + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volume.oc1.phx.abyhqljr3z3w72t6ay5eud7d5w3kdfhktfp6gwb6euy5tzwfaxgmbvwqlvsq + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + - name: shard2 + storageSizeInGb: 50 + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volume.oc1.phx.abyhqljr3z3w72t6ay5eud7d5w3kdfhktfp6gwb6euy5tzwfaxgmbvwqlvsq + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + - name: shard3 + storageSizeInGb: 50 + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volume.oc1.phx.abyhqljr3z3w72t6ay5eud7d5w3kdfhktfp6gwb6euy5tzwfaxgmbvwqlvsq + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + catalog: + - name: catalog + storageSizeInGb: 50 + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volume.oc1.phx.abyhqljr3z3w72t6ay5eud7d5w3kdfhktfp6gwb6euy5tzwfaxgmbvwqlvsq + imagePullPolicy: "Always" + gsm: + - name: gsm1 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: primary + - name: gsm2 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: standby + storageClass: oci + dbImage: container-registry.oracle.com/database/enterprise:latest + dbImagePullSecret: ocr-reg-cred + gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImagePullSecret: ocr-reg-cred + isExternalSvc: False + isDeleteOraPvc: True + isClone: True + dbSecret: + name: db-user-pass-rsa + pwdFileName: pwdfile.enc + keyFileName: key.pem + gsmService: + - name: oltp_rw_svc + role: primary + - name: oltp_ro_svc + role: primary + namespace: shns diff --git a/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_clone_across_ads.yaml b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_clone_across_ads.yaml new file mode 100644 index 00000000..d2ae0ba9 --- /dev/null +++ b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_clone_across_ads.yaml @@ -0,0 +1,82 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +--- +apiVersion: database.oracle.com/v1alpha1 +kind: ShardingDatabase +metadata: + name: shardingdatabase-sample + namespace: shns +spec: + shard: + - name: shard1 + storageSizeInGb: 50 + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volumebackup.oc1.phx.abyhqljrxtv7tu5swqb3lzc7vpzwbwzdktd2y4k2vjjy2srmgu2w7bqdftjq + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + - name: shard2 + storageSizeInGb: 50 + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-2" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-2" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volumebackup.oc1.phx.abyhqljrxtv7tu5swqb3lzc7vpzwbwzdktd2y4k2vjjy2srmgu2w7bqdftjq + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + - name: shard3 + storageSizeInGb: 50 + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-3" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-3" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volumebackup.oc1.phx.abyhqljrxtv7tu5swqb3lzc7vpzwbwzdktd2y4k2vjjy2srmgu2w7bqdftjq + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + catalog: + - name: catalog + storageSizeInGb: 50 + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volumebackup.oc1.phx.abyhqljrxtv7tu5swqb3lzc7vpzwbwzdktd2y4k2vjjy2srmgu2w7bqdftjq + imagePullPolicy: "Always" + gsm: + - name: gsm1 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: primary + - name: gsm2 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: standby + storageClass: oci + dbImage: container-registry.oracle.com/database/enterprise:latest + dbImagePullSecret: ocr-reg-cred + gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImagePullSecret: ocr-reg-cred + isExternalSvc: False + isDeleteOraPvc: True + isClone: True + dbSecret: + name: db-user-pass-rsa + pwdFileName: pwdfile.enc + keyFileName: key.pem + gsmService: + - name: oltp_rw_svc + role: primary + - name: oltp_ro_svc + role: primary + namespace: shns \ No newline at end of file diff --git a/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_delshard.yaml b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_delshard.yaml new file mode 100644 index 00000000..5f600d3f --- /dev/null +++ b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_delshard.yaml @@ -0,0 +1,68 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +--- +apiVersion: database.oracle.com/v1alpha1 +kind: ShardingDatabase +metadata: + name: shardingdatabase-sample + namespace: shns +spec: + shard: + - name: shard1 + storageSizeInGb: 50 + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + - name: shard2 + storageSizeInGb: 50 + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + - name: shard3 + storageSizeInGb: 50 + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + - name: shard4 + isDelete: True + storageSizeInGb: 50 + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + - name: shard5 + storageSizeInGb: 50 + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + catalog: + - name: catalog + storageSizeInGb: 50 + imagePullPolicy: "Always" + gsm: + - name: gsm1 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: primary + - name: gsm2 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: standby + storageClass: oci + dbImage: container-registry.oracle.com/database/enterprise:latest + dbImagePullSecret: ocr-reg-cred + gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImagePullSecret: ocr-reg-cred + isExternalSvc: False + isDeleteOraPvc: True + dbSecret: + name: db-user-pass-rsa + pwdFileName: pwdfile.enc + keyFileName: key.pem + gsmService: + - name: oltp_rw_svc + role: primary + - name: oltp_ro_svc + role: primary + namespace: shns \ No newline at end of file diff --git a/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_extshard.yaml b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_extshard.yaml new file mode 100644 index 00000000..d25dc901 --- /dev/null +++ b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_extshard.yaml @@ -0,0 +1,67 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +--- +apiVersion: database.oracle.com/v1alpha1 +kind: ShardingDatabase +metadata: + name: shardingdatabase-sample + namespace: shns +spec: + shard: + - name: shard1 + storageSizeInGb: 50 + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + - name: shard2 + storageSizeInGb: 50 + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + - name: shard3 + storageSizeInGb: 50 + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + - name: shard4 + storageSizeInGb: 50 + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + - name: shard5 + storageSizeInGb: 50 + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + catalog: + - name: catalog + storageSizeInGb: 50 + imagePullPolicy: "Always" + gsm: + - name: gsm1 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: primary + - name: gsm2 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: standby + storageClass: oci + dbImage: container-registry.oracle.com/database/enterprise:latest + dbImagePullSecret: ocr-reg-cred + gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImagePullSecret: ocr-reg-cred + isExternalSvc: False + isDeleteOraPvc: True + dbSecret: + name: db-user-pass-rsa + pwdFileName: pwdfile.enc + keyFileName: key.pem + gsmService: + - name: oltp_rw_svc + role: primary + - name: oltp_ro_svc + role: primary + namespace: shns \ No newline at end of file diff --git a/docs/sharding/provisioning/shard_prov_memory_cpu.yaml b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_memory_cpu.yaml similarity index 60% rename from docs/sharding/provisioning/shard_prov_memory_cpu.yaml rename to docs/sharding/provisioning/system_sharding/ssharding_shard_prov_memory_cpu.yaml index 0e66c563..793a3e4d 100644 --- a/docs/sharding/provisioning/shard_prov_memory_cpu.yaml +++ b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_memory_cpu.yaml @@ -2,6 +2,7 @@ # Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # +--- apiVersion: database.oracle.com/v1alpha1 kind: ShardingDatabase metadata: @@ -20,6 +21,9 @@ spec: value: "600" - name: "INIT_PGA_SIZE" value: "400" + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary - name: shard2 storageSizeInGb: 50 resources: @@ -31,6 +35,23 @@ spec: value: "600" - name: "INIT_PGA_SIZE" value: "400" + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + - name: shard3 + storageSizeInGb: 50 + resources: + requests: + memory: "1000Mi" + cpu: "1000m" + envVars: + - name: "INIT_SGA_SIZE" + value: "600" + - name: "INIT_PGA_SIZE" + value: "400" + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary catalog: - name: catalog storageSizeInGb: 50 @@ -38,30 +59,30 @@ spec: requests: memory: "1000Mi" cpu: "1000m" + imagePullPolicy: "Always" gsm: - name: gsm1 + imagePullPolicy: "Always" storageSizeInGb: 50 - replicas: 1 - envVars: - - name: "SERVICE1_PARAMS" - value: "service_name=oltp_rw_svc;service_role=primary" - - name: "SERVICE2_PARAMS" - value: "service_name=oltp_ro_svc;service_role=primary" + region: primary - name: gsm2 + imagePullPolicy: "Always" storageSizeInGb: 50 - replicas: 1 - envVars: - - name: "SERVICE1_PARAMS" - value: "service_name=oltp_rw_svc;service_role=primary" - - name: "SERVICE2_PARAMS" - value: "service_name=oltp_ro_svc;service_role=primary" + region: standby storageClass: oci dbImage: container-registry.oracle.com/database/enterprise:latest dbImagePullSecret: ocr-reg-cred gsmImage: container-registry.oracle.com/database/gsm:latest gsmImagePullSecret: ocr-reg-cred - scriptsLocation: "set -ex;curl https://codeload.github.com/oracle/db-sharding/tar.gz/master | tar -xz --strip=4 db-sharding-master/docker-based-sharding-deployment/dockerfiles/21.3.0/scripts; cp -i -r scripts/* /opt/oracle/scripts/sharding/scripts/;cp -i -r scripts/*py /opt/oracle/scripts/sharding" - secret: db-user-pass - isExternalSvc: false + isExternalSvc: False isDeleteOraPvc: True + dbSecret: + name: db-user-pass-rsa + pwdFileName: pwdfile.enc + keyFileName: key.pem + gsmService: + - name: oltp_rw_svc + role: primary + - name: oltp_ro_svc + role: primary namespace: shns diff --git a/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_send_notification.yaml b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_send_notification.yaml new file mode 100644 index 00000000..96217b74 --- /dev/null +++ b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_send_notification.yaml @@ -0,0 +1,85 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +--- +apiVersion: database.oracle.com/v1alpha1 +kind: ShardingDatabase +metadata: + name: shardingdatabase-sample + namespace: shns +spec: + shard: + - name: shard1 + storageSizeInGb: 50 + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volumebackup.oc1.phx.abyhqljrxtv7tu5swqb3lzc7vpzwbwzdktd2y4k2vjjy2srmgu2w7bqdftjq + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + - name: shard2 + storageSizeInGb: 50 + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-2" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-2" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volumebackup.oc1.phx.abyhqljrxtv7tu5swqb3lzc7vpzwbwzdktd2y4k2vjjy2srmgu2w7bqdftjq + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + - name: shard3 + storageSizeInGb: 50 + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-3" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-3" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volumebackup.oc1.phx.abyhqljrxtv7tu5swqb3lzc7vpzwbwzdktd2y4k2vjjy2srmgu2w7bqdftjq + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + catalog: + - name: catalog + storageSizeInGb: 50 + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volumebackup.oc1.phx.abyhqljrxtv7tu5swqb3lzc7vpzwbwzdktd2y4k2vjjy2srmgu2w7bqdftjq + imagePullPolicy: "Always" + gsm: + - name: gsm1 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: primary + - name: gsm2 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: standby + storageClass: oci + dbImage: container-registry.oracle.com/database/enterprise:latest + dbImagePullSecret: ocr-reg-cred + gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImagePullSecret: ocr-reg-cred + isExternalSvc: False + isDeleteOraPvc: True + isClone: True + dbSecret: + name: db-user-pass-rsa + pwdFileName: pwdfile.enc + keyFileName: key.pem + nsConfigMap: onsconfigmap + nsSecret: my-secret + gsmService: + - name: oltp_rw_svc + role: primary + - name: oltp_ro_svc + role: primary + namespace: shns + diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_from_gold_image_across_ads.md b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_from_gold_image_across_ads.md new file mode 100644 index 00000000..0beecbca --- /dev/null +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_from_gold_image_across_ads.md @@ -0,0 +1,53 @@ +# Provisioning Oracle Sharded Database with User Defined Sharding by cloning database from your own Database Gold Image across Availability Domains(ADs) + +**IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. + +In this test case, you provision the Oracle Database sharding topology with User Defined Sharding while provisioning the Catalog and Shard Databases by cloning from an existing Oracle Database Gold Image created earlier. + +This use case applies when you want to provision the database Pods on a Kubernetes Node in any availability domain (AD), which can also be different from the availability domain (AD) of the Block Volume that has the Oracle Database Gold Image provisioned earlier. + +Choosing this option takes substantially less time during the Oracle Database Sharding Topology setup across ADs. + +NOTE: + +* Cloning from Block Volume Backup in OCI enables the new Persistent Volumes to be created in other ADs. +* To specify the AD where you want to provision the database Pod, use the tag `nodeSelector` and the POD will be provisioned in a node running in that AD. +* To specify GSM containers, you can also use the tag `nodeSelector` to specify the AD. +* Before you can provision with the Gold Image, you need the OCID of the Persistent Volume that has the Oracle Database Gold Image. + +1. Check the OCID of the Persistent Volume provisioned for the Oracle Database Gold Image: + ```sh + kubectl get pv -n shns + ``` +2. Create a Block Volume Backup for this Block Volume, and use the OCID of the Block Volume Backup in the next step. This example uses `udsharding_shard_prov_clone_across_ads.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: + +* Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` +* Three sharding Pods: `shard1`, `shard2` and `shard3` +* One Catalog Pod: `catalog` +* Namespace: `shns` +* Database Cloning from the `BLOCK VOLUME FULL BACKUP` of the Persistent Volume which had the Gold Image. +* OCID of the Block Volume Backup: `ocid1.volumebackup.oc1.phx.abyhqljrxtv7tu5swqb3lzc7vpzwbwzdktd2y4k2vjjy2srmgu2w7bqdftjq` +* User Defined Sharding is specified using `shardingType: USER` + +NOTE: In this case, the Persistent Volume with DB Gold Image was provisioned in the Availability Domain `PHX-AD-1`. The Shards and Catalog will be provisioned across multiple Availability Domains by cloning the database. + +In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) + * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. + * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `udsharding_shard_prov_clone_across_ads.yaml`. + * The `dbImage` used during provisioning the Persistent Volume with Database Gold Image and the `dbImage` used for deploying the Shard or Catalog Database by cloning should be same. + * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) + +Use the file: [udsharding_shard_prov_clone_across_ads.yaml](./udsharding_shard_prov_clone_across_ads.yaml) for this use case as below: + +1. Deploy the `udsharding_shard_prov_clone_across_ads.yaml` file: + ```sh + kubectl apply -f udsharding_shard_prov_clone_across_ads.yaml + ``` +2. Check the status of the deployment: + ```sh + # Check the status of the Kubernetes Pods: + kubectl get all -n shns + + # Check the logs of a particular pod. For example, to check status of pod "shard1-0": + kubectl logs -f pod/shard1-0 -n shns + diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_gold_image_in_same_ad.md b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_gold_image_in_same_ad.md new file mode 100644 index 00000000..445d0105 --- /dev/null +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_gold_image_in_same_ad.md @@ -0,0 +1,49 @@ +# Provisioning Oracle Sharded Database with User Defined Sharding by cloning database from your own Database Gold Image in the same Availability Domain(AD) + +**IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. + +In this case, the database is created automatically by cloning from an existing Oracle Database Gold Image during the provisioning of the shard databases and the catalog database when the Oracle Sharding topology is deployed using Oracle Sharding controller. + +This use case applies when you are cloning from a Block Volume, and you can clone _only_ in the same availability domain (AD). The result is that the cloned shard database PODs can be created _only_ in the same AD where the Gold Image Block Volume is present. + +Choosing this option takes substantially less time during the Oracle Database Sharding Topology setup. + +**NOTE** For this step, the Persistent Volume that has the Oracle Database Gold Image is identified using its OCID. + +1. Check the OCID of the Persistent Volume provisioned earlier using below command: + + ```sh + kubectl get pv -n shns + ``` + +2. This example uses `udsharding_shard_prov_clone.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: + +* Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` +* Three sharding Pods: `shard1`, `shard2` and `shard3` +* One Catalog Pod: `catalog` +* Namespace: `shns` +* Database Cloning from the Database Gold Image present in Persistent Volume having OCID: `ocid1.volume.oc1.phx.abyhqljr3z3w72t6ay5eud7d5w3kdfhktfp6gwb6euy5tzwfaxgmbvwqlvsq` +* User Defined Sharding is specified using `shardingType: USER` + +NOTE: In this case, the Persistent Volume with DB Gold Image was provisioned in the Availability Domain `PHX-AD-1`. The Shards and Catalog will be provisioned in the same Availability Domain `PHX-AD-1` by cloning the database. + +In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) + * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. + * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `ssharding_shard_prov_clone.yaml`. + * The `dbImage` used during provisioning the Persistent Volume with Database Gold Image and the `dbImage` used for deploying the Shard or Catalog Database by cloning should be same. + * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) + +Use the file: [udsharding_shard_prov_clone.yaml](./udsharding_shard_prov_clone.yaml) for this use case as below: + +1. Deploy the `udsharding_shard_prov_clone.yaml` file: + ```sh + kubectl apply -f udsharding_shard_prov_clone.yaml + ``` +2. Check the status of the deployment: + ```sh + # Check the status of the Kubernetes Pods: + kubectl get all -n shns + + # Check the logs of a particular pod. For example, to check status of pod "shard1-0": + kubectl logs -f pod/shard1-0 -n shns + ``` diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_with_control_on_resources.md b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_with_control_on_resources.md new file mode 100644 index 00000000..d37368f8 --- /dev/null +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_with_control_on_resources.md @@ -0,0 +1,42 @@ +# Provisioning Oracle Sharded Database with User Defined Sharding with additional control on resources like Memory and CPU allocated to Pods + +**IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. + +In this use case, there are additional tags used to control resources such as CPU and Memory used by the different Pods when the Oracle Sharding topology with User Defined Sharding is deployed using Oracle Sharding controller. + +This example uses `udsharding_shard_prov_memory_cpu.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: + +* Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` +* Three sharding Pods: `shard1`, `shard2` and `shard3` +* One Catalog Pod: `catalog` +* Namespace: `shns` +* Tags `memory` and `cpu` to control the Memory and CPU of the PODs +* Additional tags `INIT_SGA_SIZE` and `INIT_PGA_SIZE` to control the SGA and PGA allocation at the database level +* User Defined Sharding is specified using `shardingType: USER` + +In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) + * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. + * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `udsharding_shard_prov_memory_cpu.yaml`. + * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) + +Use the YAML file [udsharding_shard_prov_memory_cpu.yaml](./udsharding_shard_prov_memory_cpu.yaml). + +1. Deploy the `udsharding_shard_prov_memory_cpu.yaml` file: + + ```sh + kubectl apply -f udsharding_shard_prov_memory_cpu.yaml + ``` + +1. Check the details of a POD. For example: To check the details of Pod `shard1-0`: + + ```sh + kubectl describe pod/shard1-0 -n shns + ``` +3. Check the status of the deployment: + ```sh + # Check the status of the Kubernetes Pods: + kubectl get all -n shns + + # Check the logs of a particular pod. For example, to check status of pod "shard1-0": + kubectl logs -f pod/shard1-0 -n shns + ``` diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_with_notification_using_oci_notification.md b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_with_notification_using_oci_notification.md new file mode 100644 index 00000000..c7da6aa5 --- /dev/null +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_with_notification_using_oci_notification.md @@ -0,0 +1,82 @@ +# Provisioning Oracle Sharded Database with User Defined Sharding and send Notification using OCI Notification Service + +**IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. + +This use case demonstrates how to use a notification service like OCI Notification service to send an email notification when a particular operation is completed on an Oracle Database sharding topology provisioned using the Oracle Database sharding controller. + +This example uses `udsharding_shard_prov_send_notification.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: + +* Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` +* Three sharding Pods: `shard1`, `shard2` and `shard3` +* One Catalog Pod: `catalog` +* Namespace: `shns` +* Database Cloning from the `BLOCK VOLUME FULL BACKUP` of the Persistent Volume that has the Database Gold Image created earlier. +* OCID of the Block Volume Backup: `ocid1.volumebackup.oc1.phx.abyhqljrxtv7tu5swqb3lzc7vpzwbwzdktd2y4k2vjjy2srmgu2w7bqdftjq` +* Configmap to send notification email when a particular operation is completed. For example: When a shard is added. +* User Defined Sharding is specified using `shardingType: USER` + +**NOTE:** + +* The notification will be sent using a configmap created with the credentials of the OCI user account in this use case. + +We will create a topic in Notification Service of the OCI Console and use its OCID. + +To do this: + +1. Create a `configmap_data.txt` file, such as the following, which has the OCI User details that will be used to send notfication: + + ```sh + user=ocid1.user.oc1........fx7omxfq + fingerprint=fa:18:98:...............:8a + tenancy=ocid1.tenancy.oc1..aaaa.......orpn7inq + region=us-phoenix-1 + topicid=ocid1.onstopic.oc1.phx.aaa............6xrq + ``` +2. Create a configmap using the below command using the file created above: + ```sh + kubectl create configmap onsconfigmap --from-file=./configmap_data.txt -n shns + ``` + +3. Create a key file `priavatekey` having the PEM key of the OCI user being used to send notification: + ```sh + -----BEGIN PRIVATE KEY-G---- + MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQCXYxA0DJvEwtVR + +o4OxrunL3L2NZJRADTFR+TDHqrNF1JwbaFBizSdL+EXbxQW1faZs5lXZ/sVmQF9 + . + . + . + zn/xWC0FzXGRzfvYHhq8XT3omf6L47KqIzqo3jDKdgvVq4u+lb+fXJlhj6Rwi99y + QEp36HnZiUxAQnR331DacN+YSTE+vpzSwZ38OP49khAB1xQsbiv1adG7CbNpkxpI + nS7CkDLg4Hcs4b9bGLHYJVY= + -----END PRIVATE KEY----- + ``` +4. Use the key file `privatekey` to create a Kubernetes secret in namespace `shns`: + + ```sh + kubectl create secret generic my-secret --from-file=./privatekey -n shns + ``` + +5. Use this command to check details of the secret that you created: + + ```sh + kubectl describe secret my-secret -n shns + ``` + +In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) + * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. + * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `udsharding_shard_prov_send_notification.yaml`. + * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) + +Use the file: [udsharding_shard_prov_send_notification.yaml](./udsharding_shard_prov_send_notification.yaml) for this use case as below: + +1. Deploy the `udsharding_shard_prov_send_notification.yaml` file: + ```sh + kubectl apply -f udsharding_shard_prov_send_notification.yaml + ``` +2. Check the status of the deployment: + ```sh + # Check the status of the Kubernetes Pods: + kubectl get all -n shns + + # Check the logs of a particular pod. For example, to check status of pod "shard1-0": + kubectl logs -f pod/shard1-0 -n shns diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_without_db_gold_image.md b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_without_db_gold_image.md new file mode 100644 index 00000000..a275155f --- /dev/null +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_without_db_gold_image.md @@ -0,0 +1,37 @@ +# Provisioning Oracle Sharded Database with User Defined Sharding without Database Gold Image + +**IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. + +In this use case, the database is created automatically using DBCA during the provisioning of the shard databases and the catalog database when the Oracle Sharding topology with User Defined Sharding is deployed using Oracle Sharding controller. + +**NOTE** In this use case, because DBCA creates the database automatically during the deployment, the time required to create the database is greater than the time it takes when the database is created by cloning from a Database Gold Image. + +This example uses `udsharding_shard_prov.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: + +* Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` +* Three sharding Pods: `shard1`, `shard2` and `shard3` +* One Catalog Pod: `catalog` +* Namespace: `shns` +* User Defined Sharding is specified using `shardingType: USER` + + +In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) + * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. + * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `udsharding_shard_prov.yaml`. + * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) + + +Use the file: [udsharding_shard_prov.yaml](./udsharding_shard_prov.yaml) for this use case as below: + +1. Deploy the `udsharding_shard_prov.yaml` file: + ```sh + kubectl apply -f udsharding_shard_prov.yaml + ``` +1. Check the status of the deployment: + ```sh + # Check the status of the Kubernetes Pods: + kubectl get all -n shns + + # Check the logs of a particular pod. For example, to check status of pod "shard1-0": + kubectl logs -f pod/shard1-0 -n shns + ``` diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_scale_in_delete_an_existing_shard.md b/docs/sharding/provisioning/user-defined-sharding/udsharding_scale_in_delete_an_existing_shard.md new file mode 100644 index 00000000..946a0ab9 --- /dev/null +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_scale_in_delete_an_existing_shard.md @@ -0,0 +1,47 @@ +# Scale In - Delete an existing Shard from a working Oracle Sharded Database provisioned earlier with User Defined Sharding + +**IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. + +This use case demonstrates how to delete an existing Shard from an existing Oracle Database sharding topology with User Defined Sharding provisioned using Oracle Database Sharding controller. + +**NOTE** The deletion of a shard is done after verifying the Chunks have been moved out of that shard. + +In this use case, the existing database Sharding is having: + +* Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` +* Five sharding Pods: `shard1`,`shard2`,`shard3`,`shard4` and `shard5` +* One Catalog Pod: `catalog` +* Namespace: `shns` +* User Defined Sharding is specified using `shardingType: USER` + +In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) + * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. + * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `udsharding_shard_prov_delshard.yaml`. + * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) + +NOTE: Use tag `isDelete: true` to delete the shard you want. + +This use case deletes the shard `shard4` from the above Sharding Topology. + +Use the file: [udsharding_shard_prov_delshard.yaml](./udsharding_shard_prov_delshard.yaml) for this use case as below: + +1. Deploy the `udsharding_shard_prov_delshard.yaml` file: + ```sh + kubectl apply -f udsharding_shard_prov_delshard.yaml + ``` +2. Check the status of the deployment: + ```sh + # Check the status of the Kubernetes Pods: + kubectl get all -n shns + +**NOTE:** After you apply `udsharding_shard_prov_delshard.yaml`, the change may not be visible immediately. When the shard is removed, first the chunks will be moved out of that shard that is going to be deleted. + +To monitor the chunk movement, use the following command: + +```sh +# Switch to the primary GSM Container: +kubectl exec -i -t gsm1-0 -n shns /bin/bash + +# Check the status of the chunks and repeat to observe the chunk movement: +gdsctl config chunks +``` diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_scale_out_add_shards.md b/docs/sharding/provisioning/user-defined-sharding/udsharding_scale_out_add_shards.md new file mode 100644 index 00000000..e4200d72 --- /dev/null +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_scale_out_add_shards.md @@ -0,0 +1,34 @@ +# Scale Out - Add Shards to an existing Oracle Sharded Database provisioned earlier with User Defined Sharding + +**IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. + +This use case demonstrates adding a new shard to an existing Oracle Database sharding topology with User Defined Sharding provisioned earlier using Oracle Database Sharding controller. + +In this use case, the existing Oracle Database sharding topology is having: + +* Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` +* Three sharding Pods: `shard1`, `shard2` and `shard3` +* One Catalog Pod: `catalog` +* Namespace: `shns` +* User Defined Sharding is specified using `shardingType: USER` + +In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) + * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. + * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `udsharding_shard_prov_extshard.yaml`. + * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) + +This use case adds two new shards `shard4`,`shard5` to above Sharding Topology. + +Use the file: [udsharding_shard_prov_extshard.yaml](./udsharding_shard_prov_extshard.yaml) for this use case as below: + +1. Deploy the `udsharding_shard_prov_extshard.yaml` file: + ```sh + kubectl apply -f udsharding_shard_prov_extshard.yaml + ``` +2. Check the status of the deployment: + ```sh + # Check the status of the Kubernetes Pods: + kubectl get all -n shns + + # Check the logs of a particular pod. For example, to check status of pod "shard4-0": + kubectl logs -f pod/shard4-0 -n shns diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov.yaml b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov.yaml new file mode 100644 index 00000000..019fc887 --- /dev/null +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov.yaml @@ -0,0 +1,59 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +--- +apiVersion: database.oracle.com/v1alpha1 +kind: ShardingDatabase +metadata: + name: shardingdatabase-sample + namespace: shns +spec: + shard: + - name: shard1 + storageSizeInGb: 50 + imagePullPolicy: "Always" + shardSpace: sspace1 + shardRegion: primary + - name: shard2 + storageSizeInGb: 50 + imagePullPolicy: "Always" + shardSpace: sspace2 + shardRegion: primary + - name: shard3 + storageSizeInGb: 50 + imagePullPolicy: "Always" + shardSpace: sspace3 + shardRegion: primary + catalog: + - name: catalog + storageSizeInGb: 50 + imagePullPolicy: "Always" + gsm: + - name: gsm1 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: primary + - name: gsm2 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: standby + storageClass: oci + dbImage: container-registry.oracle.com/database/enterprise:latest + dbImagePullSecret: ocr-reg-cred + gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImagePullSecret: ocr-reg-cred + shardingType: USER + isExternalSvc: False + isDeleteOraPvc: True + dbSecret: + name: db-user-pass-rsa + pwdFileName: pwdfile.enc + keyFileName: key.pem + gsmService: + - name: oltp_rw_svc + role: primary + - name: oltp_ro_svc + role: primary + namespace: shns + diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_clone.yaml b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_clone.yaml new file mode 100644 index 00000000..adc2271f --- /dev/null +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_clone.yaml @@ -0,0 +1,83 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +--- +apiVersion: database.oracle.com/v1alpha1 +kind: ShardingDatabase +metadata: + name: shardingdatabase-sample + namespace: shns +spec: + shard: + - name: shard1 + storageSizeInGb: 50 + imagePullPolicy: "Always" + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volume.oc1.phx.abyhqljr3z3w72t6ay5eud7d5w3kdfhktfp6gwb6euy5tzwfaxgmbvwqlvsq + shardSpace: sspace1 + shardRegion: primary + - name: shard2 + storageSizeInGb: 50 + imagePullPolicy: "Always" + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volume.oc1.phx.abyhqljr3z3w72t6ay5eud7d5w3kdfhktfp6gwb6euy5tzwfaxgmbvwqlvsq + shardSpace: sspace2 + shardRegion: primary + - name: shard3 + storageSizeInGb: 50 + imagePullPolicy: "Always" + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volume.oc1.phx.abyhqljr3z3w72t6ay5eud7d5w3kdfhktfp6gwb6euy5tzwfaxgmbvwqlvsq + shardSpace: sspace3 + shardRegion: primary + catalog: + - name: catalog + storageSizeInGb: 50 + imagePullPolicy: "Always" + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volume.oc1.phx.abyhqljr3z3w72t6ay5eud7d5w3kdfhktfp6gwb6euy5tzwfaxgmbvwqlvsq + gsm: + - name: gsm1 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: primary + - name: gsm2 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: standby + storageClass: oci + dbImage: container-registry.oracle.com/database/enterprise:latest + dbImagePullSecret: ocr-reg-cred + gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImagePullSecret: ocr-reg-cred + shardingType: USER + isExternalSvc: False + isDeleteOraPvc: True + isClone: True + dbSecret: + name: db-user-pass-rsa + pwdFileName: pwdfile.enc + keyFileName: key.pem + gsmService: + - name: oltp_rw_svc + role: primary + - name: oltp_ro_svc + role: primary + namespace: shns diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_clone_across_ads.yaml b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_clone_across_ads.yaml new file mode 100644 index 00000000..f2d10e23 --- /dev/null +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_clone_across_ads.yaml @@ -0,0 +1,83 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +--- +apiVersion: database.oracle.com/v1alpha1 +kind: ShardingDatabase +metadata: + name: shardingdatabase-sample + namespace: shns +spec: + shard: + - name: shard1 + storageSizeInGb: 50 + imagePullPolicy: "Always" + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volumebackup.oc1.phx.abyhqljrxtv7tu5swqb3lzc7vpzwbwzdktd2y4k2vjjy2srmgu2w7bqdftjq + shardSpace: sspace1 + shardRegion: primary + - name: shard2 + storageSizeInGb: 50 + imagePullPolicy: "Always" + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-2" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-2" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volumebackup.oc1.phx.abyhqljrxtv7tu5swqb3lzc7vpzwbwzdktd2y4k2vjjy2srmgu2w7bqdftjq + shardSpace: sspace2 + shardRegion: primary + - name: shard3 + storageSizeInGb: 50 + imagePullPolicy: "Always" + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-3" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-3" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volumebackup.oc1.phx.abyhqljrxtv7tu5swqb3lzc7vpzwbwzdktd2y4k2vjjy2srmgu2w7bqdftjq + shardSpace: sspace3 + shardRegion: primary + catalog: + - name: catalog + storageSizeInGb: 50 + imagePullPolicy: "Always" + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volumebackup.oc1.phx.abyhqljrxtv7tu5swqb3lzc7vpzwbwzdktd2y4k2vjjy2srmgu2w7bqdftjq + gsm: + - name: gsm1 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: primary + - name: gsm2 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: standby + storageClass: oci + dbImage: container-registry.oracle.com/database/enterprise:latest + dbImagePullSecret: ocr-reg-cred + gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImagePullSecret: ocr-reg-cred + shardingType: USER + isExternalSvc: False + isDeleteOraPvc: True + isClone: True + dbSecret: + name: db-user-pass-rsa + pwdFileName: pwdfile.enc + keyFileName: key.pem + gsmService: + - name: oltp_rw_svc + role: primary + - name: oltp_ro_svc + role: primary + namespace: shns diff --git a/docs/sharding/provisioning/shard_prov_extshard.yaml b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_delshard.yaml similarity index 51% rename from docs/sharding/provisioning/shard_prov_extshard.yaml rename to docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_delshard.yaml index 449ba90b..51f1c292 100644 --- a/docs/sharding/provisioning/shard_prov_extshard.yaml +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_delshard.yaml @@ -2,6 +2,7 @@ # Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # +--- apiVersion: database.oracle.com/v1alpha1 kind: ShardingDatabase metadata: @@ -11,41 +12,58 @@ spec: shard: - name: shard1 storageSizeInGb: 50 + imagePullPolicy: "Always" + shardSpace: sspace1 + shardRegion: primary - name: shard2 storageSizeInGb: 50 + imagePullPolicy: "Always" + shardSpace: sspace2 + shardRegion: primary - name: shard3 storageSizeInGb: 50 + imagePullPolicy: "Always" + shardSpace: sspace3 + shardRegion: primary - name: shard4 storageSizeInGb: 50 + imagePullPolicy: "Always" + shardSpace: sspace4 + shardRegion: primary + isDelete: True - name: shard5 - storageSizeInGb: 50 + storageSizeInGb: 50 + imagePullPolicy: "Always" + shardSpace: sspace5 + shardRegion: primary catalog: - name: catalog storageSizeInGb: 50 + imagePullPolicy: "Always" gsm: - name: gsm1 + imagePullPolicy: "Always" storageSizeInGb: 50 - replicas: 1 - envVars: - - name: "SERVICE1_PARAMS" - value: "service_name=oltp_rw_svc;service_role=primary" - - name: "SERVICE2_PARAMS" - value: "service_name=oltp_ro_svc;service_role=primary" + region: primary - name: gsm2 + imagePullPolicy: "Always" storageSizeInGb: 50 - replicas: 1 - envVars: - - name: "SERVICE1_PARAMS" - value: "service_name=oltp_rw_svc;service_role=primary" - - name: "SERVICE2_PARAMS" - value: "service_name=oltp_ro_svc;service_role=primary" + region: standby storageClass: oci dbImage: container-registry.oracle.com/database/enterprise:latest dbImagePullSecret: ocr-reg-cred gsmImage: container-registry.oracle.com/database/gsm:latest gsmImagePullSecret: ocr-reg-cred - scriptsLocation: "set -ex;curl https://codeload.github.com/oracle/db-sharding/tar.gz/master | tar -xz --strip=4 db-sharding-master/docker-based-sharding-deployment/dockerfiles/21.3.0/scripts; cp -i -r scripts/* /opt/oracle/scripts/sharding/scripts/;cp -i -r scripts/*py /opt/oracle/scripts/sharding" - secret: db-user-pass - isExternalSvc: false + shardingType: USER + isExternalSvc: False isDeleteOraPvc: True + dbSecret: + name: db-user-pass-rsa + pwdFileName: pwdfile.enc + keyFileName: key.pem + gsmService: + - name: oltp_rw_svc + role: primary + - name: oltp_ro_svc + role: primary namespace: shns diff --git a/docs/sharding/provisioning/shard_prov_delshard.yaml b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_extshard.yaml similarity index 50% rename from docs/sharding/provisioning/shard_prov_delshard.yaml rename to docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_extshard.yaml index 09cbd549..f45d421f 100644 --- a/docs/sharding/provisioning/shard_prov_delshard.yaml +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_extshard.yaml @@ -11,42 +11,57 @@ spec: shard: - name: shard1 storageSizeInGb: 50 + imagePullPolicy: "Always" + shardSpace: sspace1 + shardRegion: primary - name: shard2 storageSizeInGb: 50 - isDelete: true + imagePullPolicy: "Always" + shardSpace: sspace2 + shardRegion: primary - name: shard3 storageSizeInGb: 50 + imagePullPolicy: "Always" + shardSpace: sspace3 + shardRegion: primary - name: shard4 storageSizeInGb: 50 + imagePullPolicy: "Always" + shardSpace: sspace4 + shardRegion: primary - name: shard5 - storageSizeInGb: 50 + storageSizeInGb: 50 + imagePullPolicy: "Always" + shardSpace: sspace5 + shardRegion: primary catalog: - name: catalog storageSizeInGb: 50 + imagePullPolicy: "Always" gsm: - name: gsm1 + imagePullPolicy: "Always" storageSizeInGb: 50 - replicas: 1 - envVars: - - name: "SERVICE1_PARAMS" - value: "service_name=oltp_rw_svc;service_role=primary" - - name: "SERVICE2_PARAMS" - value: "service_name=oltp_ro_svc;service_role=primary" + region: primary - name: gsm2 + imagePullPolicy: "Always" storageSizeInGb: 50 - replicas: 1 - envVars: - - name: "SERVICE1_PARAMS" - value: "service_name=oltp_rw_svc;service_role=primary" - - name: "SERVICE2_PARAMS" - value: "service_name=oltp_ro_svc;service_role=primary" + region: standby storageClass: oci dbImage: container-registry.oracle.com/database/enterprise:latest dbImagePullSecret: ocr-reg-cred gsmImage: container-registry.oracle.com/database/gsm:latest gsmImagePullSecret: ocr-reg-cred - scriptsLocation: "set -ex;curl https://codeload.github.com/oracle/db-sharding/tar.gz/master | tar -xz --strip=4 db-sharding-master/docker-based-sharding-deployment/dockerfiles/21.3.0/scripts; cp -i -r scripts/* /opt/oracle/scripts/sharding/scripts/;cp -i -r scripts/*py /opt/oracle/scripts/sharding" - secret: db-user-pass - isExternalSvc: false + shardingType: USER + isExternalSvc: False isDeleteOraPvc: True + dbSecret: + name: db-user-pass-rsa + pwdFileName: pwdfile.enc + keyFileName: key.pem + gsmService: + - name: oltp_rw_svc + role: primary + - name: oltp_ro_svc + role: primary namespace: shns diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_memory_cpu.yaml b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_memory_cpu.yaml new file mode 100644 index 00000000..15022925 --- /dev/null +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_memory_cpu.yaml @@ -0,0 +1,90 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +--- +apiVersion: database.oracle.com/v1alpha1 +kind: ShardingDatabase +metadata: + name: shardingdatabase-sample + namespace: shns +spec: + shard: + - name: shard1 + storageSizeInGb: 50 + resources: + requests: + memory: "1000Mi" + cpu: "1000m" + envVars: + - name: "INIT_SGA_SIZE" + value: "600" + - name: "INIT_PGA_SIZE" + value: "400" + imagePullPolicy: "Always" + shardSpace: sspace1 + shardRegion: primary + - name: shard2 + storageSizeInGb: 50 + resources: + requests: + memory: "1000Mi" + cpu: "1000m" + envVars: + - name: "INIT_SGA_SIZE" + value: "600" + - name: "INIT_PGA_SIZE" + value: "400" + imagePullPolicy: "Always" + shardSpace: sspace2 + shardRegion: primary + - name: shard3 + storageSizeInGb: 50 + resources: + requests: + memory: "1000Mi" + cpu: "1000m" + envVars: + - name: "INIT_SGA_SIZE" + value: "600" + - name: "INIT_PGA_SIZE" + value: "400" + imagePullPolicy: "Always" + shardSpace: sspace3 + shardRegion: primary + catalog: + - name: catalog + storageSizeInGb: 50 + resources: + requests: + memory: "1000Mi" + cpu: "1000m" + imagePullPolicy: "Always" + gsm: + - name: gsm1 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: primary + - name: gsm2 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: standby + storageClass: oci + dbImage: container-registry.oracle.com/database/enterprise:latest + dbImagePullSecret: ocr-reg-cred + gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImagePullSecret: ocr-reg-cred + shardingType: USER + isExternalSvc: False + isDeleteOraPvc: True + dbSecret: + name: db-user-pass-rsa + pwdFileName: pwdfile.enc + keyFileName: key.pem + gsmService: + - name: oltp_rw_svc + role: primary + - name: oltp_ro_svc + role: primary + namespace: shns + diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_send_notification.yaml b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_send_notification.yaml new file mode 100644 index 00000000..afd951fe --- /dev/null +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_send_notification.yaml @@ -0,0 +1,85 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +--- +apiVersion: database.oracle.com/v1alpha1 +kind: ShardingDatabase +metadata: + name: shardingdatabase-sample + namespace: shns +spec: + shard: + - name: shard1 + storageSizeInGb: 50 + imagePullPolicy: "Always" + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volumebackup.oc1.phx.abyhqljrxtv7tu5swqb3lzc7vpzwbwzdktd2y4k2vjjy2srmgu2w7bqdftjq + shardSpace: sspace1 + shardRegion: primary + - name: shard2 + storageSizeInGb: 50 + imagePullPolicy: "Always" + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-2" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-2" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volumebackup.oc1.phx.abyhqljrxtv7tu5swqb3lzc7vpzwbwzdktd2y4k2vjjy2srmgu2w7bqdftjq + shardSpace: sspace2 + shardRegion: primary + - name: shard3 + storageSizeInGb: 50 + imagePullPolicy: "Always" + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-3" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-3" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volumebackup.oc1.phx.abyhqljrxtv7tu5swqb3lzc7vpzwbwzdktd2y4k2vjjy2srmgu2w7bqdftjq + shardSpace: sspace3 + shardRegion: primary + catalog: + - name: catalog + storageSizeInGb: 50 + imagePullPolicy: "Always" + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volumebackup.oc1.phx.abyhqljrxtv7tu5swqb3lzc7vpzwbwzdktd2y4k2vjjy2srmgu2w7bqdftjq + gsm: + - name: gsm1 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: primary + - name: gsm2 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: standby + storageClass: oci + dbImage: container-registry.oracle.com/database/enterprise:latest + dbImagePullSecret: ocr-reg-cred + gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImagePullSecret: ocr-reg-cred + shardingType: USER + isExternalSvc: False + isDeleteOraPvc: True + isClone: True + dbSecret: + name: db-user-pass-rsa + pwdFileName: pwdfile.enc + keyFileName: key.pem + nsConfigMap: onsconfigmap + nsSecret: my-secret + gsmService: + - name: oltp_rw_svc + role: primary + - name: oltp_ro_svc + role: primary + namespace: shns diff --git a/docs/sidb/PREREQUISITES.md b/docs/sidb/PREREQUISITES.md index 6315d308..b904f5da 100644 --- a/docs/sidb/PREREQUISITES.md +++ b/docs/sidb/PREREQUISITES.md @@ -1,17 +1,21 @@ -## Prerequisites for Oracle Docker Image Deployment -To deploy Oracle Database Operator for Kubernetes on Oracle Docker images, complete these steps. +## Deployment Prerequisites +To deploy Oracle Single Instance Database in Kubernetes using the OraOperator, complete these steps. -* ### Prepare Oracle Docker Images +* ### Prepare Oracle Container Images - Build Single Instance Database Docker Images from source, following the instructions at [https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance), or + Build Single Instance Database Container Images from source, following the instructions at [https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance), or use the pre-built images available at [https://container-registry.oracle.com](https://container-registry.oracle.com) by signing in and accepting the required license agreement. Oracle Database Releases Supported: Enterprise and Standard Edition for Oracle Database 19c, and later releases. Express Edition for Oracle Database 21.3.0 only. Oracle Database Free 23.2.0 and later Free releases - Build Oracle REST Data Service Docker Images from source following the instructions at [https://github.com/oracle/docker-images/tree/main/OracleRestDataServices](https://github.com/oracle/docker-images/tree/main/OracleRestDataServices). + Build Oracle REST Data Service Container Images from source following the instructions at [https://github.com/oracle/docker-images/tree/main/OracleRestDataServices](https://github.com/oracle/docker-images/tree/main/OracleRestDataServices). Supported Oracle REST Data Service version is 21.4.2 -* ### Set Up Kubernetes and Volumes +* ### Ensure Sufficient Disk Space in Kubernetes Worker Nodes + + Provision Kubernetes worker nodes with recommended 250 GiB or more of free disk space required for pulling the base and patched database container images. If deploying on cloud you may choose to increase the custom boot volume size of the worker nodes. + +* ### Set Up Kubernetes and Volumes for Database Persistence Set up an on-premises Kubernetes cluster, or subscribe to a managed Kubernetes service, such as Oracle Cloud Infrastructure Container Engine for Kubernetes. Use a dynamic volume provisioner or pre-provision static persistent volumes manually. These volumes are required for persistent storage of the database files. diff --git a/docs/sidb/README.md b/docs/sidb/README.md index eb5116a4..81db03b8 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -3,6 +3,8 @@ Oracle Database Operator for Kubernetes (`OraOperator`) includes the Single Instance Database Controller, which enables provisioning, cloning, and patching of Oracle Single Instance Databases on Kubernetes. It also enables configuring the database for Oracle REST Data Services with Oracle APEX development platform. The following sections explain the setup and functionality of the operator * [Prerequisites](#prerequisites) + * [Mandatory Resource Privileges](#mandatory-resource-privileges) + * [Optional Resource Privileges](#optional-resource-privileges) * [SingleInstanceDatabase Resource](#singleinstancedatabase-resource) * [Create a Database](#create-a-database) * [New Database](#new-database) @@ -12,6 +14,7 @@ Oracle Database Operator for Kubernetes (`OraOperator`) includes the Single Inst * [Connecting to Database](#connecting-to-database) * [Database Persistence (Storage) Configuration Options](#database-persistence-storage-configuration-options) * [Dynamic Persistence](#dynamic-persistence) + * [Storage Expansion](#storage-expansion) * [Static Persistence](#static-persistence) * [Configuring a Database](#configuring-a-database) * [Switching Database Modes](#switching-database-modes) @@ -30,6 +33,7 @@ Oracle Database Operator for Kubernetes (`OraOperator`) includes the Single Inst * [Perform a Switchover](#perform-a-switchover) * [Patch Primary and Standby databases in Data Guard configuration](#patch-primary-and-standby-databases-in-data-guard-configuration) * [Delete the Data Guard Configuration](#delete-the-data-guard-configuration) + * [Execute Custom Scripts](#execute-custom-scripts) * [OracleRestDataService Resource](#oraclerestdataservice-resource) * [REST Enable a Database](#rest-enable-a-database) * [Provision ORDS](#provision-ords) @@ -45,7 +49,47 @@ Oracle Database Operator for Kubernetes (`OraOperator`) includes the Single Inst ## Prerequisites -Oracle strongly recommends that you follow the [prerequisites](./PREREQUISITES.md). +Oracle strongly recommends to comply with the [prerequisites](./PREREQUISITES.md) and the following requirements + + ### Mandatory Resource Privileges + + Single Instance Database(sidb) controller mandatorily requires the following Kubernetes resource privileges: + + | Resources | Privileges | + | --- | --- | + | Pods | create delete get list patch update watch | + | Containers | create delete get list patch update watch | + | PersistentVolumeClaims | create delete get list patch update watch | + | Services | create delete get list patch update watch | + | Secrets | create delete get list patch update watch | + | Events | create patch | + + For managing the required levels of access, configure [role binding](../../README.md#role-binding-for-access-management) + + ### Optional Resource Privileges + + Single Instance Database(sidb) controller optionally requires the following Kubernetes resource privileges depending on the functionality being used: + + | Functionality | Resources | Privileges | + | --- | --- | --- | + | NodePort Services | Nodes | list watch | + | Storage Expansion with block volumes | StorageClasses | get list watch | + | Custom Scripts Execution | PersistentVolumes | get list watch | + + + For exposing the database via Nodeport services, apply [RBAC](../../rbac/node-rbac.yaml) + ```sh + kubectl apply -f rbac/node-rbac.yaml + ``` + For automatic storage expansion of block volumes, apply [RBAC](../../rbac/storage-class-rbac.yaml) + ```sh + kubectl apply -f rbac/storage-class-rbac.yaml + ``` + For automatic execution of custom scripts post database setup or startup, apply [RBAC](../../rbac/persistent-volume-rbac.yaml) + ```sh + kubectl apply -f rbac/persistent-volume-rbac.yaml + ``` + ## SingleInstanceDatabase Resource @@ -159,10 +203,12 @@ To provision a new database instance on the Kubernetes cluster, use the example 2. If you have not already done so, create an image pull secret for the Oracle Container Registry: ```sh - $ kubectl create secret docker-registry oracle-container-registry-secret --docker-server=container-registry.oracle.com --docker-username='' --docker-password='' --docker-email='' + $ kubectl create secret docker-registry oracle-container-registry-secret --docker-server=container-registry.oracle.com --docker-username='' --docker-password='' --docker-email='' - secret/oracle-container-registry-secret created + secret/oracle-container-registry-secret created ``` + Note: Generate the auth token from user profile section on top right of the page after logging into container-registry.oracle.com + This secret can also be created from the docker config.json or from podman auth.json after a successful login ```sh docker login container-registry.oracle.com @@ -171,7 +217,7 @@ To provision a new database instance on the Kubernetes cluster, use the example or ```sh podman login container-registry.oracle.com - podman create secret generic oracle-container-registry-secret --from-file=.dockerconfigjson=${XDG_RUNTIME_DIR}/containers/auth.json --type=kubernetes.io/dockerconfigjson + kubectl create secret generic oracle-container-registry-secret --from-file=.dockerconfigjson=${XDG_RUNTIME_DIR}/containers/auth.json --type=kubernetes.io/dockerconfigjson ``` 3. Provision a new database instance on the cluster by using the following command: @@ -221,10 +267,10 @@ To provision new Oracle Database Free database, use the sample **[config/samples This command pulls the Free image uploaded on the [Oracle Container Registry](https://container-registry.oracle.com/). **Note:** -- Provisioning Oracle Database Free is supported for release 23c (23.2.0) and later releases. +- Provisioning Oracle Database Free is supported for release 23.3.0 and later releases. - For Free database, only single replica mode (i.e. `replicas: 1`) is supported. - For Free database, you **cannot change** the init parameters i.e. `cpuCount, processes, sgaTarget or pgaAggregateTarget`. -- Oracle Enterprise Manager is not supported from release 23c and later release. +- Oracle Enterprise Manager Express (OEM Express) is not supported from release 23.3.0 and later releases. #### Additional Information You are required to specify the database admin password secret in the corresponding YAML file. The default values mentioned in the `adminPassword.secretName` fields of [singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml), [singleinstancedatabase_prebuiltdb.yaml](../../config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml), [singleinstancedatabase_express.yaml](../../config/samples/sidb/singleinstancedatabase_express.yaml) and [singleinstancedatabse_free.yaml](../../config/samples/sidb/singleinstancedatabase_free.yaml) files are `db-admin-secret`, `prebuiltdb-admin-secret`, `xedb-admin-secret` and `free-admin-secret` respectively. You can create these secrets manually by using the sample command mentioned in the [Template YAML](#template-yaml) section. Alternatively, you can create these secrets by filling the passwords in the **[singleinstancedatabase_secrets.yaml](../../config/samples/sidb/singleinstancedatabase_secrets.yaml)** file and applying it using the command below: @@ -281,7 +327,7 @@ $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.oemExpres https://10.0.25.54:5500/em ``` -**Note:** OEM Express is not available for 23c and later releases +**Note:** OEM Express is not available for 23.3.0 and later releases ### Database Persistence (Storage) Configuration Options The database persistence can be achieved in the following two ways: @@ -295,8 +341,19 @@ In **Dynamic Persistence Provisioning**, a persistent volume is provisioned by m - Generally, the `Reclaim Policy` of such dynamically provisioned volumes is `Delete`. These volumes are deleted when their corresponding database deployment is deleted. To retain volumes, use static provisioning, as explained in the Block Volume Static Provisioning section. - In **Minikube**, the dynamic persistence provisioning class is **standard**. +#### Storage Expansion +When using dynamic persistence, you can at any time scale up your persistent volumes by simply patching the singleinstancedatabase resource using the following command : +```sh +$ kubectl patch singleinstancedatabase sidb-sample -p '{"spec":{"persistence":{"size":"100Gi"}}}' --type=merge +``` + +**Note:** +- Storage expansion requires the storage class to be configured with `allowVolumeExpansion:true` +- Storage expansion requires read and watch access for storage account as mentioned in [prerequisites](#prerequisites) +- User can only scale up a volume/storage and not scale down + #### Static Persistence -In **Static Persistence Provisioning**, you have to create a volume manually, and then use the name of this volume with the `<.spec.persistence.volumeName>` field which corresponds to the `volumeName` field of the persistence section in the **[singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml)**. The `Reclaim Policy` of such volume can be set to `Retain`. So, this volume does not get deleted with the deletion of its corresponding deployment. +In **Static Persistence Provisioning**, you have to create a volume manually, and then use the name of this volume with the `<.spec.persistence.datafilesVolumeName>` field which corresponds to the `datafilesVolumeName` field of the persistence section in the **[singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml)**. The `Reclaim Policy` of such volume can be set to `Retain`. So, this volume does not get deleted with the deletion of its corresponding deployment. For example in **Minikube**, a persistent volume can be provisioned using the sample yaml file below: ```yaml apiVersion: v1 @@ -312,7 +369,7 @@ spec: hostPath: path: /data/oradata ``` -The persistent volume name (i.e. db-vol) can be mentioned in the `volumeName` field of the **[singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml)**. `storageClass` field is not required in this case, and can be left empty. +The persistent volume name (i.e. db-vol) can be mentioned in the `datafilesVolumeName` field of the **[singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml)**. `storageClass` field is not required in this case, and can be left empty. Static Persistence Provisioning in Oracle Cloud Infrastructure (OCI) is explained in the following subsections: @@ -423,7 +480,7 @@ The following attributes cannot be modified after creating the Single Instance D - `edition` - `charset` - `pdbName` -- `cloneFrom` +- `primaryDatabaseRef` If you attempt to changing one of these attributes, then you receive an error similar to the following: @@ -439,8 +496,6 @@ To create copies of your existing database quickly, you can use the cloning func To quickly clone the existing database sidb-sample created above, use the sample **[config/samples/sidb/singleinstancedatabase_clone.yaml](../../config/samples/sidb/singleinstancedatabase_clone.yaml)** file. -**Note**: To clone a database, the source database must have archiveLog mode set to true. - For example: ```sh @@ -450,7 +505,10 @@ $ kubectl apply -f singleinstancedatabase_clone.yaml singleinstancedatabase.database.oracle.com/sidb-sample-clone created ``` -**Note:** The clone database can specify a database image that is different from the source database. In such cases, cloning is supported only between databases of the same major release. +**Note:** +- To clone a database, the source database must have archiveLog mode set to true. +- The clone database can specify a database image that is different from the source database. In such cases, cloning is supported only between databases of the same major release. +- Only enterprise and standard editions support cloning. ### Patch a Database @@ -469,7 +527,10 @@ singleinstancedatabase.database.oracle.com/sidb-sample patched ``` -After patching is complete, the database pods are restarted with the new release update image. For minimum downtime, ensure that you have multiple replicas of the database pods running before you start the patch operation. +After patching is complete, the database pods are restarted with the new release update image. + +**Note:** +- Only enterprise and standard editions support patching. #### Patch after Cloning @@ -518,11 +579,25 @@ The command above will delete the database pods and associated service. Some advanced database configuration scenarios are as follows: #### Run Database with Multiple Replicas -In multiple replicas mode, more than one pod is created for the database. The database is open and mounted by one of the replica pods. Other replica pods have instances started but not mounted, and serve to provide a quick cold fail-over in case the active pod goes down. Multiple replicas are also helpful in [patching](#patch-existing-database) operation. Ensure that you have multiple replicas of the database pods running before you start the patching operation for minimum downtime. +In multiple replicas mode, more than one pod is created for the database. Setting the replica count equal to or more than the number of worker nodes helps in distributing the replicas accross all the nodes that have access to the database persistent storage volume. +The database is open and mounted by one of the replica pods. Other replica pods have the database instance started but not mounted, and serve to provide a quick cold fail-over in case the active pod goes down. To enable multiple replicas, update the replica attribute in the `.yaml`, and apply by using the `kubectl apply` or `kubectl scale` commands. +The following table depicts the fail over matrix for any destructive operation to the primary replica pod + +| Pod Destructive Operation | Pod Restart/FailOver| + | --- | --- | + | Database instance crash | Yes | + | Force delete pod with zero grace period | Yes | + | Gracefully delete pod | Yes | + | Node running primary replica dies | Yes | + | Direct shutdown [All modes] | Yes | + | Maintenance shutdown [All modes] | No | + | PDB close | No | + **Note:** +- Maintence shutdown/startup can be executed using the scripts /home/oracle/shutDown.sh and /home/oracle/startUp.sh - This functionality requires the [k8s extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/k8s) extended images. The database image from the container registry `container-registry.oracle.com` includes the K8s extension. - Because Oracle Database Express Edition (XE) does not support [k8s extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/k8s), it does not support multiple replicas. - If the `ReadWriteOnce` access mode is used, all the replicas will be scheduled on the same node where the persistent volume would be mounted. @@ -540,39 +615,49 @@ $ kubectl --type=merge -p '{"spec":{"loadBalancer": true}}' patch singleinstance ``` ### Enabling TCPS Connections -You can enable TCPS connections in the database by setting the `enableTCPS` field to `true` in the [config/samples/sidb/singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml) file, and applying it using `kubectl apply` command. +You can enable TCPS connections in the database by setting the `enableTCPS` field to `true` in the [config/samples/sidb/singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml) file, and applying it. Alternatively, you can use the following command: ```bash kubectl patch --type=merge singleinstancedatabases.database.oracle.com sidb-sample -p '{"spec": {"enableTCPS": true}}' ``` -Once TCPS connections are enabled, the database connect string will change accordingly. The TCPS connections status can also be queried by the following command: +By default self signed certs are used for TCPS connections. The TCPS connections status can also be queried by the following command: ```bash kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.isTcpsEnabled}" true ``` -The following steps are required to connect the Database using TCPS: -- You need to download the wallet from the Persistent Volume (PV) attached with the database pod. The location of the wallet inside the pod is as `/opt/oracle/oradata/clientWallet/$ORACLE_SID`. **Let us assume the `ORACLE_SID` is `ORCL1`, and singleinstance database resource name is `sidb-sample` for the upcoming example command**. You can copy the wallet to the destination directory by the following command: +**With Self Signed Certs** +- When TCPS is enabled, a self-signed certificate is generated and stored in wallets. For users' convenience, a client-side wallet is generated in location `/opt/oracle/oradata/clientWallet/$ORACLE_SID` in the pod. +- The self-signed certificate used with TCPS has validity for 1 year. After the certificate is expired, it will be renewed by the `OraOperator` automatically. Download the wallet again after auto-renewal. +- You can set the certificate renew interval with the help of `tcpsCertRenewInterval` field in the **[config/samples/sidb/singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml)** file. The minimum accepted value is 24h, and the maximum value is 8760h (1 year). The certificates used with TCPS will automatically be renewed after this interval. If this field is omitted/commented in the yaml file, the certificates will not be renewed automatically. +- When the certificate gets created/renewed, the `.status.certCreationTimestamp` status variable gets updated accordingly. You can see this timestamp by using the following command: ```bash - kubectl cp $(kubectl get pods -l app=sidb-sample -o=jsonpath='{.items[0].metadata.name}'):/opt/oracle/oradata/clientWallet/ORCL1 + kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.certCreationTimestamp}" ``` -- This wallet includes the sample `tnsnames.ora` and `sqlnet.ora` files. All the TNS entries for the database (corresponding to the CDB and PDB) resides in `tnsnames.ora` file. You need to go inside the downloaded wallet directory and set the `TNS_ADMIN` environment variable to point to the current directory as follows: + +**With User Provided Certs** +- Users can provide custom certs to be used for TCPS connections instead of self signed ones. +- Specify the certs by creating a Kubernetes tls secret resource using following command: ```bash - # After going inside the downloaded wallet directory - export TNS_ADMIN=$(pwd) + kubectl create secret tls my-tls-secret --cert=path/to/cert/tls.crt --key=path/to/key/tls.key ``` - After this, you can connect using SQL\*Plus using the following sample commands: +- `tls.crt` is a certificate chain in the order of client, followed by intermediate and then root certificate and `tls.key` is client key. +- Specify the secret created above (`my-tls-secret`) as the value for the attribute `tcpsTlsSecret` in the [config/samples/sidb/singleinstancedatabase_tcps.yaml](../../config/samples/sidb/singleinstancedatabase_tcps.yaml) file, and apply it. + +**Connecting to the Database using TCPS** +- Download the wallet from the Persistent Volume (PV) attached with the database pod. The location of the wallet inside the pod is as `/opt/oracle/oradata/clientWallet/$ORACLE_SID`. Let us assume the `ORACLE_SID` is `ORCL1`, and singleinstance database resource name is `sidb-sample` for the upcoming example command. You can copy the wallet to the destination directory by the following command: ```bash - sqlplus sys@ORCL1 as sysdba + kubectl cp $(kubectl get pods -l app=sidb-sample -o=jsonpath='{.items[0].metadata.name}'):/opt/oracle/oradata/clientWallet/ORCL1 ``` -**Note:** -- When TCPS is enabled, a self-signed certificate is generated and stored inside the wallets. For users' convenience, a client-side wallet is generated and stored at `/opt/oracle/oradata/clientWallet/$ORACLE_SID` location in the pod. -- The self-signed certificate used with TCPS has validity for 1 year. After the certificate is expired, it will be renewed by the `OraOperator` automatically. You need to download the wallet again after the auto-renewal. -- You can set the certificate renew interval with the help of `tcpsCertRenewInterval` field in the **[config/samples/sidb/singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml)** file. The minimum accepted value is 24h, and the maximum value is 8760h (1 year). The certificates used with TCPS will automatically be renewed after this interval. If this field is omitted/commented in the yaml file, the certificates will not be renewed automatically. -- When the certificate gets created/renewed, the `.status.certCreationTimestamp` status variable gets updated accordingly. You can see this timestamp by using the following command: +- This wallet includes the sample `tnsnames.ora` and `sqlnet.ora` files. All the TNS entries for the database (corresponding to the CDB and PDB) reside in the `tnsnames.ora` file. Switch to the downloaded wallet directory and set the `TNS_ADMIN` environment variable to point to the current directory as follows: ```bash - kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.certCreationTimestamp}" + cd + export TNS_ADMIN=$(pwd) + ``` + After this, connect using SQL\*Plus using the following sample commands: + ```bash + sqlplus sys@ORCL1 as sysdba ``` ### Specifying Custom Ports @@ -595,7 +680,8 @@ In case of `NodePort` service, `listenerPort`, and `tcpsListenerPort` will be th ### Create a Standby Database #### Prerequisites -Before creating a standby, ArchiveLog, FlashBack, and ForceLog on primary Single Instance Database(`.spec.primaryDatabaseRef`) should be turned on. +- Before creating a Standby, ensure that ArchiveLog, FlashBack, and ForceLog on primary Single Instance Database(`.spec.primaryDatabaseRef`) are turned on. +- Standby database is not supported for TCPS enabled Primary databases. #### Template YAML To create a standby database, edit and apply the sample yaml file [config/samples/sidb/singleinstancedatabase_standby.yaml](../../config/samples/sidb/singleinstancedatabase_standby.yaml). @@ -603,7 +689,7 @@ To create a standby database, edit and apply the sample yaml file [config/sample **Note:** - The `adminPassword` field of the above [config/samples/sidb/singleinstancedatabase_standby.yaml](../../config/samples/sidb/singleinstancedatabase_standby.yaml) contains an admin password secret of the primary database ref for Standby Database creation. This secret will get deleted after the database pod becomes ready if the `keepSecret` attribute of `adminPassword` field is set to `false`. By default `keepSecret` is set to `true`. - Mention referred primary database in `.spec.primaryDatabaseRef` in the yaml file. -- `.spec.createAsStandby` field of the yaml file should be true. +- `.spec.createAs` field of the yaml file should be set to "standby". - Database configuration like `Archivelog`, `FlashBack`, `ForceLog`, `TCPS connections` are not supported for standby database. #### List Standby Databases @@ -766,7 +852,7 @@ $ kubectl --type=merge -p '{"spec":{"setAsPrimaryDatabase":"ORCLS1"}}' patch dat ### Patch Primary and Standby databases in Data Guard configuration -Databases (both primary and standby) running in you cluster and managed by the Oracle Database operator can be patched or rolled back between release updates of the same major release. While patching databases configured with the dataguard broker you need to first patch the Primary database followed by seconday/standby databases in any order. +Databases (both primary and standby) running in you cluster and managed by the Oracle Database operator can be patched between release updates of the same major release. To patch an existing database, edit and apply the **[config/samples/sidb/singleinstancedatabase_patch.yaml](../../config/samples/sidb/singleinstancedatabase_patch.yaml)** file of the database resource/object either by specifying a new release update for image attributes, or by running the following command: @@ -774,6 +860,15 @@ To patch an existing database, edit and apply the **[config/samples/sidb/singlei kubectl --type=merge -p '{"spec":{"image":{"pullFrom":"patched-image:tag","pullSecrets":"pull-secret"}}}' patch singleinstancedatabase ``` +Follow these steps for patching databases configured with the dataguard broker: +1. First patch all the standby databases by replacing the image with the new release update image +2. Perform switch over of the primary to one of the standby databases +3. Now patch the original primary database (currently standby after #2) + After #3 the software for primary and standby databases is at the same release update +4. Now bounce the current primary database by updating the replica count to 0 and then 1 + #4 will trigger a datapatch execution resulting in patching of the datafiles +5. Finally perform switch over of the current primary back to the original primary (current standby) + ### Delete the Data Guard Configuration @@ -785,7 +880,8 @@ $ kubectl delete dataguardbroker dgbroker-sample dataguardbroker.database.oracle.com/dgbroker-sample deleted ``` -**Note:** Deleting of DataGuardBroker resource is allowed only when role of `.spec.primaryDatabaseRef` is PRIMARY + +**Note:** If a switch over to standby was performed, make sure to switch back to the original primary database before deleting the dataguard broker resource #### Delete Standby Database ```sh @@ -794,6 +890,14 @@ $ kubectl delete singleinstancedatabase stdby-1 singleinstancedatabase.database.oracle.com "stdby-1" deleted ``` +### Execute Custom Scripts + +Custom scripts (sql and/or shell scripts) can be executed after the initial database setup and/or after each startup of the database. SQL scripts will be executed as sysdba, shell scripts will be executed as the current user. To ensure proper order it is recommended to prefix your scripts with a number. For example `01_users.sql`, `02_permissions.sql`, etc. Place all such scripts in setup and startup folders created in a persistent volume to execute them post setup and post startup respectively. + +Create a persistent volume using [static provisioning](#static-persistence) and then specify the name of this volume with the `<.spec.persistence.scriptsVolumeName>` field which corresponds to the `scriptsVolumeName` field of the persistence section in the **[singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml)**. + +**Note:** Executing custom scripts requires read and list access for persistent volumes as mentioned in [prerequisites](#prerequisites) + ## OracleRestDataService Resource The Oracle Database Operator creates the `OracleRestDataService` as a custom resource. We will refer `OracleRestDataService` as ORDS from now onwards. Creating ORDS as a custom resource enables the RESTful API access to the Oracle Database in K8s and enables it to be managed as a native Kubernetes object. diff --git a/go.mod b/go.mod index 81c4d435..1f30279f 100644 --- a/go.mod +++ b/go.mod @@ -1,82 +1,106 @@ module github.com/oracle/oracle-database-operator -go 1.19 +go 1.21 require ( - github.com/go-logr/logr v1.2.3 - github.com/onsi/ginkgo/v2 v2.5.0 - github.com/onsi/gomega v1.24.1 - github.com/oracle/oci-go-sdk/v65 v65.26.1 - go.uber.org/zap v1.23.0 - gopkg.in/yaml.v2 v2.4.0 - k8s.io/api v0.25.4 - k8s.io/apimachinery v0.25.4 - k8s.io/client-go v0.25.4 - sigs.k8s.io/controller-runtime v0.13.1 + github.com/go-logr/logr v1.3.0 + github.com/onsi/ginkgo/v2 v2.13.0 + github.com/onsi/gomega v1.29.0 + github.com/oracle/oci-go-sdk/v65 v65.49.3 + github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring v0.62.0 + go.uber.org/zap v1.26.0 + golang.org/x/text v0.14.0 + gopkg.in/yaml.v3 v3.0.1 + k8s.io/api v0.29.2 + k8s.io/apimachinery v0.29.2 + k8s.io/cli-runtime v0.29.2 + k8s.io/client-go v0.29.2 + k8s.io/kubectl v0.29.2 + sigs.k8s.io/controller-runtime v0.16.2 sigs.k8s.io/yaml v1.3.0 ) require ( - github.com/PuerkitoBio/purell v1.1.1 // indirect - github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578 // indirect + github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 // indirect + github.com/MakeNowJust/heredoc v1.0.0 // indirect github.com/beorn7/perks v1.0.1 // indirect - github.com/cespare/xxhash/v2 v2.1.2 // indirect + github.com/cespare/xxhash/v2 v2.2.0 // indirect + github.com/chai2010/gettext-go v1.0.2 // indirect github.com/davecgh/go-spew v1.1.1 // indirect - github.com/emicklei/go-restful/v3 v3.8.0 // indirect + github.com/emicklei/go-restful/v3 v3.11.0 // indirect + github.com/evanphx/json-patch v5.6.0+incompatible // indirect github.com/evanphx/json-patch/v5 v5.6.0 // indirect - github.com/fsnotify/fsnotify v1.5.4 // indirect - github.com/go-openapi/jsonpointer v0.19.5 // indirect - github.com/go-openapi/jsonreference v0.19.5 // indirect - github.com/go-openapi/swag v0.19.14 // indirect + github.com/exponent-io/jsonpath v0.0.0-20151013193312-d6023ce2651d // indirect + github.com/fatih/camelcase v1.0.0 // indirect + github.com/fsnotify/fsnotify v1.6.0 // indirect + github.com/fvbommel/sortorder v1.1.0 // indirect + github.com/go-errors/errors v1.4.2 // indirect + github.com/go-logr/zapr v1.2.4 // indirect + github.com/go-openapi/jsonpointer v0.19.6 // indirect + github.com/go-openapi/jsonreference v0.20.2 // indirect + github.com/go-openapi/swag v0.22.3 // indirect + github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572 // indirect github.com/gofrs/flock v0.8.1 // indirect github.com/gogo/protobuf v1.3.2 // indirect github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect - github.com/golang/protobuf v1.5.2 // indirect - github.com/google/gnostic v0.5.7-v3refs // indirect - github.com/google/go-cmp v0.5.9 // indirect - github.com/imdario/mergo v0.3.12 // indirect + github.com/golang/protobuf v1.5.3 // indirect + github.com/google/btree v1.0.1 // indirect + github.com/google/gnostic-models v0.6.8 // indirect + github.com/google/go-cmp v0.6.0 // indirect + github.com/google/gofuzz v1.2.0 // indirect + github.com/google/pprof v0.0.0-20210720184732-4bb14d4b1be1 // indirect + github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 // indirect + github.com/google/uuid v1.3.0 // indirect + github.com/gorilla/websocket v1.5.0 // indirect + github.com/gregjones/httpcache v0.0.0-20180305231024-9cad4c3443a7 // indirect + github.com/imdario/mergo v0.3.6 // indirect + github.com/inconshreveable/mousetrap v1.1.0 // indirect github.com/josharian/intern v1.0.0 // indirect github.com/json-iterator/go v1.1.12 // indirect - github.com/mailru/easyjson v0.7.6 // indirect - github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 // indirect + github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de // indirect + github.com/mailru/easyjson v0.7.7 // indirect + github.com/matttproud/golang_protobuf_extensions v1.0.4 // indirect + github.com/mitchellh/go-wordwrap v1.0.1 // indirect github.com/moby/spdystream v0.2.0 // indirect + github.com/moby/term v0.0.0-20221205130635-1aeaba878587 // indirect github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect github.com/modern-go/reflect2 v1.0.2 // indirect + github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00 // indirect github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect + github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f // indirect + github.com/peterbourgon/diskv v2.0.1+incompatible // indirect github.com/pkg/errors v0.9.1 // indirect - github.com/prometheus/client_golang v1.12.2 // indirect - github.com/prometheus/client_model v0.2.0 // indirect - github.com/prometheus/common v0.32.1 // indirect - github.com/prometheus/procfs v0.7.3 // indirect + github.com/prometheus/client_golang v1.16.0 // indirect + github.com/prometheus/client_model v0.4.0 // indirect + github.com/prometheus/common v0.44.0 // indirect + github.com/prometheus/procfs v0.10.1 // indirect + github.com/russross/blackfriday/v2 v2.1.0 // indirect github.com/sony/gobreaker v0.5.0 // indirect + github.com/spf13/cobra v1.7.0 // indirect github.com/spf13/pflag v1.0.5 // indirect - github.com/stretchr/objx v0.4.0 // indirect - go.uber.org/atomic v1.9.0 // indirect - go.uber.org/multierr v1.7.0 // indirect - golang.org/x/net v0.7.0 // indirect - golang.org/x/oauth2 v0.0.0-20211104180415-d3ed0bb246c8 // indirect - golang.org/x/sys v0.5.0 // indirect - golang.org/x/term v0.5.0 // indirect - golang.org/x/text v0.7.0 // indirect - golang.org/x/time v0.0.0-20220609170525-579cf78fd858 // indirect - gomodules.xyz/jsonpatch/v2 v2.2.0 // indirect + github.com/xlab/treeprint v1.2.0 // indirect + go.starlark.net v0.0.0-20230525235612-a134d8f9ddca // indirect + go.uber.org/multierr v1.11.0 // indirect + golang.org/x/exp v0.0.0-20220722155223-a9213eeb770e // indirect + golang.org/x/net v0.19.0 // indirect + golang.org/x/oauth2 v0.10.0 // indirect + golang.org/x/sync v0.5.0 // indirect + golang.org/x/sys v0.15.0 // indirect + golang.org/x/term v0.15.0 // indirect + golang.org/x/time v0.3.0 // indirect + golang.org/x/tools v0.16.1 // indirect + gomodules.xyz/jsonpatch/v2 v2.4.0 // indirect google.golang.org/appengine v1.6.7 // indirect - google.golang.org/protobuf v1.28.0 // indirect + google.golang.org/protobuf v1.31.0 // indirect gopkg.in/inf.v0 v0.9.1 // indirect - gopkg.in/yaml.v3 v3.0.1 // indirect - k8s.io/apiextensions-apiserver v0.25.0 // indirect - k8s.io/component-base v0.25.0 // indirect - k8s.io/klog/v2 v2.70.1 // indirect - k8s.io/kube-openapi v0.0.0-20220803162953-67bda5d908f1 // indirect - k8s.io/utils v0.0.0-20220728103510-ee6ede2d64ed // indirect - sigs.k8s.io/json v0.0.0-20220713155537-f223a00ba0e2 // indirect - sigs.k8s.io/structured-merge-diff/v4 v4.2.3 // indirect -) - -require ( - cloud.google.com/go/compute v1.2.0 // indirect - github.com/evanphx/json-patch v5.6.0+incompatible // indirect - github.com/go-logr/zapr v1.2.3 // indirect - github.com/google/gofuzz v1.2.0 // indirect - github.com/google/uuid v1.3.0 // indirect + gopkg.in/yaml.v2 v2.4.0 // indirect + k8s.io/apiextensions-apiserver v0.28.0 // indirect + k8s.io/component-base v0.29.2 // indirect + k8s.io/klog/v2 v2.110.1 // indirect + k8s.io/kube-openapi v0.0.0-20231010175941-2dd684a91f00 // indirect + k8s.io/utils v0.0.0-20230726121419-3b25d923346b // indirect + sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd // indirect + sigs.k8s.io/kustomize/api v0.13.5-0.20230601165947-6ce0bf390ce3 // indirect + sigs.k8s.io/kustomize/kyaml v0.14.3-0.20230601165947-6ce0bf390ce3 // indirect + sigs.k8s.io/structured-merge-diff/v4 v4.4.1 // indirect ) diff --git a/go.sum b/go.sum index aacb116a..08185f3e 100644 --- a/go.sum +++ b/go.sum @@ -1,809 +1,362 @@ cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw= -cloud.google.com/go v0.34.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw= -cloud.google.com/go v0.38.0/go.mod h1:990N+gfupTy94rShfmMCWGDn0LpTmnzTp2qbd1dvSRU= -cloud.google.com/go v0.44.1/go.mod h1:iSa0KzasP4Uvy3f1mN/7PiObzGgflwredwwASm/v6AU= -cloud.google.com/go v0.44.2/go.mod h1:60680Gw3Yr4ikxnPRS/oxxkBccT6SA1yMk63TGekxKY= -cloud.google.com/go v0.45.1/go.mod h1:RpBamKRgapWJb87xiFSdk4g1CME7QZg3uwTez+TSTjc= -cloud.google.com/go v0.46.3/go.mod h1:a6bKKbmY7er1mI7TEI4lsAkts/mkhTSZK8w33B4RAg0= -cloud.google.com/go v0.50.0/go.mod h1:r9sluTvynVuxRIOHXQEHMFffphuXHOMZMycpNR5e6To= -cloud.google.com/go v0.52.0/go.mod h1:pXajvRH/6o3+F9jDHZWQ5PbGhn+o8w9qiu/CffaVdO4= -cloud.google.com/go v0.53.0/go.mod h1:fp/UouUEsRkN6ryDKNW/Upv/JBKnv6WDthjR6+vze6M= -cloud.google.com/go v0.54.0/go.mod h1:1rq2OEkV3YMf6n/9ZvGWI3GWw0VoqH/1x2nd8Is/bPc= -cloud.google.com/go v0.56.0/go.mod h1:jr7tqZxxKOVYizybht9+26Z/gUq7tiRzu+ACVAMbKVk= -cloud.google.com/go v0.57.0/go.mod h1:oXiQ6Rzq3RAkkY7N6t3TcE6jE+CIBBbA36lwQ1JyzZs= -cloud.google.com/go v0.62.0/go.mod h1:jmCYTdRCQuc1PHIIJ/maLInMho30T/Y0M4hTdTShOYc= -cloud.google.com/go v0.65.0/go.mod h1:O5N8zS7uWy9vkA9vayVHs65eM1ubvY4h553ofrNHObY= -cloud.google.com/go v0.72.0/go.mod h1:M+5Vjvlc2wnp6tjzE102Dw08nGShTscUx2nZMufOKPI= -cloud.google.com/go v0.74.0/go.mod h1:VV1xSbzvo+9QJOxLDaJfTjx5e+MePCpCWwvftOeQmWk= -cloud.google.com/go v0.78.0/go.mod h1:QjdrLG0uq+YwhjoVOLsS1t7TW8fs36kLs4XO5R5ECHg= -cloud.google.com/go v0.79.0/go.mod h1:3bzgcEeQlzbuEAYu4mrWhKqWjmpprinYgKJLgKHnbb8= -cloud.google.com/go v0.81.0/go.mod h1:mk/AM35KwGk/Nm2YSeZbxXdrNK3KZOYHmLkOqC2V6E0= -cloud.google.com/go v0.83.0/go.mod h1:Z7MJUsANfY0pYPdw0lbnivPx4/vhy/e2FEkSkF7vAVY= -cloud.google.com/go v0.84.0/go.mod h1:RazrYuxIK6Kb7YrzzhPoLmCVzl7Sup4NrbKPg8KHSUM= -cloud.google.com/go v0.87.0/go.mod h1:TpDYlFy7vuLzZMMZ+B6iRiELaY7z/gJPaqbMx6mlWcY= -cloud.google.com/go v0.90.0/go.mod h1:kRX0mNRHe0e2rC6oNakvwQqzyDmg57xJ+SZU1eT2aDQ= -cloud.google.com/go v0.93.3/go.mod h1:8utlLll2EF5XMAV15woO4lSbWQlk8rer9aLOfLh7+YI= -cloud.google.com/go v0.94.1/go.mod h1:qAlAugsXlC+JWO+Bke5vCtc9ONxjQT3drlTTnAplMW4= -cloud.google.com/go v0.97.0/go.mod h1:GF7l59pYBVlXQIBLx3a761cZ41F9bBH3JUlihCt2Udc= -cloud.google.com/go v0.99.0/go.mod h1:w0Xx2nLzqWJPuozYQX+hFfCSI8WioryfRDzkoI/Y2ZA= -cloud.google.com/go v0.100.2/go.mod h1:4Xra9TjzAeYHrl5+oeLlzbM2k3mjVhZh4UqTZ//w99A= -cloud.google.com/go/bigquery v1.0.1/go.mod h1:i/xbL2UlR5RvWAURpBYZTtm/cXjCha9lbfbpx4poX+o= -cloud.google.com/go/bigquery v1.3.0/go.mod h1:PjpwJnslEMmckchkHFfq+HTD2DmtT67aNFKH1/VBDHE= -cloud.google.com/go/bigquery v1.4.0/go.mod h1:S8dzgnTigyfTmLBfrtrhyYhwRxG72rYxvftPBK2Dvzc= -cloud.google.com/go/bigquery v1.5.0/go.mod h1:snEHRnqQbz117VIFhE8bmtwIDY80NLUZUMb4Nv6dBIg= -cloud.google.com/go/bigquery v1.7.0/go.mod h1://okPTzCYNXSlb24MZs83e2Do+h+VXtc4gLoIoXIAPc= -cloud.google.com/go/bigquery v1.8.0/go.mod h1:J5hqkt3O0uAFnINi6JXValWIb1v0goeZM77hZzJN/fQ= -cloud.google.com/go/compute v0.1.0/go.mod h1:GAesmwr110a34z04OlxYkATPBEfVhkymfTBXtfbBFow= -cloud.google.com/go/compute v1.2.0 h1:EKki8sSdvDU0OO9mAXGwPXOTOgPz2l08R0/IutDH11I= -cloud.google.com/go/compute v1.2.0/go.mod h1:xlogom/6gr8RJGBe7nT2eGsQYAFUbbv8dbC29qE3Xmw= -cloud.google.com/go/datastore v1.0.0/go.mod h1:LXYbyblFSglQ5pkeyhO+Qmw7ukd3C+pD7TKLgZqpHYE= -cloud.google.com/go/datastore v1.1.0/go.mod h1:umbIZjpQpHh4hmRpGhH4tLFup+FVzqBi1b3c64qFpCk= -cloud.google.com/go/pubsub v1.0.1/go.mod h1:R0Gpsv3s54REJCy4fxDixWD93lHJMoZTyQ2kNxGRt3I= -cloud.google.com/go/pubsub v1.1.0/go.mod h1:EwwdRX2sKPjnvnqCa270oGRyludottCI76h+R3AArQw= -cloud.google.com/go/pubsub v1.2.0/go.mod h1:jhfEVHT8odbXTkndysNHCcx0awwzvfOlguIAii9o8iA= -cloud.google.com/go/pubsub v1.3.1/go.mod h1:i+ucay31+CNRpDW4Lu78I4xXG+O1r/MAHgjpRVR+TSU= -cloud.google.com/go/storage v1.0.0/go.mod h1:IhtSnM/ZTZV8YYJWCY8RULGVqBDmpoyjwiyrjsg+URw= -cloud.google.com/go/storage v1.5.0/go.mod h1:tpKbwo567HUNpVclU5sGELwQWBDZ8gh0ZeosJ0Rtdos= -cloud.google.com/go/storage v1.6.0/go.mod h1:N7U0C8pVQ/+NIKOBQyamJIeKQKkZ+mxpohlUTyfDhBk= -cloud.google.com/go/storage v1.8.0/go.mod h1:Wv1Oy7z6Yz3DshWRJFhqM/UCfaWIRTdp0RXyy7KQOVs= -cloud.google.com/go/storage v1.10.0/go.mod h1:FLPqc6j+Ki4BU591ie1oL6qBQGu2Bl/tZ9ullr3+Kg0= -dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU= +github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 h1:UQHMgLO+TxOElx5B5HZ4hJQsoJ/PvUvKRhJHDQXO8P8= +github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1/go.mod h1:xomTg63KZ2rFqZQzSB4Vz2SUXa1BpHTVz9L5PTmPC4E= github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU= -github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo= -github.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAElWljhcU= -github.com/PuerkitoBio/purell v1.1.1 h1:WEQqlqaGbrPkxLJWfBwQmfEAE1Z7ONdDLqrN38tNFfI= -github.com/PuerkitoBio/purell v1.1.1/go.mod h1:c11w/QuzBsJSee3cPx9rAFu61PvFxuPbtSwDGJws/X0= -github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578 h1:d+Bc7a5rLufV/sSk/8dngufqelfh6jnri85riMAaF/M= -github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578/go.mod h1:uGdkoq3SwY9Y+13GIhn11/XLaGBb4BfwItxLd5jeuXE= -github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc= -github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc= -github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0= -github.com/alecthomas/units v0.0.0-20190717042225-c3de453c63f4/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0= -github.com/alecthomas/units v0.0.0-20190924025748-f65c72e2690d/go.mod h1:rBZYJk541a8SKzHPHnH3zbiI+7dagKZ0cgpgrD7Fyho= -github.com/antihax/optional v1.0.0/go.mod h1:uupD/76wgC+ih3iEmQUL+0Ugr19nfwCT1kdvxnR2qWY= +github.com/MakeNowJust/heredoc v1.0.0 h1:cXCdzVdstXyiTqTvfqk9SDHpKNjxuom+DOlyEeQ4pzQ= +github.com/MakeNowJust/heredoc v1.0.0/go.mod h1:mG5amYoWBHf8vpLOuehzbGGw0EHxpZZ6lCpQ4fNJ8LE= github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 h1:0CwZNZbxp69SHPdPJAN/hZIm0C4OItdklCFmMRWYpio= -github.com/benbjohnson/clock v1.1.0 h1:Q92kusRqC1XV2MjkWETPvjJVqKetz1OzxZB7mHJLju8= +github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5/go.mod h1:wHh0iHkYZB8zMSxRWpUBQtwG5a7fFgvEO+odwuTv2gs= github.com/benbjohnson/clock v1.1.0/go.mod h1:J11/hYXuz8f4ySSvYwY0FKfm+ezbsZBKZxNJlLklBHA= -github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q= -github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8= github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU= -github.com/cespare/xxhash v1.1.0/go.mod h1:XrSqR1VqqWfGrhpAt58auRo0WTKS1nRRg3ghfAqPWnc= -github.com/cespare/xxhash/v2 v2.1.1/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= -github.com/cespare/xxhash/v2 v2.1.2 h1:YRXhKfTDauu4ajMg1TPgFO5jnlC2HCbmLXMcTG5cbYE= -github.com/cespare/xxhash/v2 v2.1.2/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= +github.com/cespare/xxhash/v2 v2.2.0 h1:DC2CZ1Ep5Y4k3ZQ899DldepgrayRUGE6BBZ/cd9Cj44= +github.com/cespare/xxhash/v2 v2.2.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= +github.com/chai2010/gettext-go v1.0.2 h1:1Lwwip6Q2QGsAdl/ZKPCwTe9fe0CjlUbqj5bFNSjIRk= +github.com/chai2010/gettext-go v1.0.2/go.mod h1:y+wnP2cHYaVj19NZhYKAwEMH2CI1gNHeQQ+5AjwawxA= github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI= github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI= github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU= github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw= -github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc= -github.com/cncf/udpa/go v0.0.0-20200629203442-efcf912fb354/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk= -github.com/cncf/udpa/go v0.0.0-20201120205902-5459f2c99403/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk= -github.com/cncf/xds/go v0.0.0-20210312221358-fbca930ec8ed/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs= +github.com/cpuguy83/go-md2man/v2 v2.0.2/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o= github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= +github.com/creack/pty v1.1.18 h1:n56/Zwd5o6whRC5PMGretI4IdRLlmBXYNjScPaBgsbY= +github.com/creack/pty v1.1.18/go.mod h1:MOBLtS5ELjhRRrroQr9kyvTxUAFNvYEK993ew/Vr4O4= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= -github.com/docopt/docopt-go v0.0.0-20180111231733-ee0de3bc6815/go.mod h1:WwZ+bS3ebgob9U8Nd0kOddGdZWjyMGR8Wziv+TBNwSE= -github.com/elazarl/goproxy v0.0.0-20180725130230-947c36da3153 h1:yUdfgN0XgIJw7foRItutHYUIhlcKzcSf5vDpdhQAKTc= -github.com/emicklei/go-restful/v3 v3.8.0 h1:eCZ8ulSerjdAiaNpF7GxXIE7ZCMo1moN1qX+S609eVw= -github.com/emicklei/go-restful/v3 v3.8.0/go.mod h1:6n3XBCmQQb25CM2LCACGz8ukIrRry+4bhvbpWn3mrbc= -github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4= +github.com/emicklei/go-restful/v3 v3.11.0 h1:rAQeMHw1c7zTmncogyy8VvRZwtkmkZ4FxERmMY4rD+g= +github.com/emicklei/go-restful/v3 v3.11.0/go.mod h1:6n3XBCmQQb25CM2LCACGz8ukIrRry+4bhvbpWn3mrbc= github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4= -github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98= -github.com/envoyproxy/go-control-plane v0.9.7/go.mod h1:cwu0lG7PUMfa9snN8LXBig5ynNVH9qI8YYLbd1fK2po= -github.com/envoyproxy/go-control-plane v0.9.9-0.20201210154907-fd9021fe5dad/go.mod h1:cXg6YxExXjJnVBQHBLXeUAgxn2UodCpnH306RInaBQk= -github.com/envoyproxy/go-control-plane v0.9.9-0.20210217033140-668b12f5399d/go.mod h1:cXg6YxExXjJnVBQHBLXeUAgxn2UodCpnH306RInaBQk= -github.com/envoyproxy/go-control-plane v0.9.9-0.20210512163311-63b5d3c536b0/go.mod h1:hliV/p42l8fGbc6Y9bQ70uLwIvmJyVE5k4iMKlh8wCQ= github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c= -github.com/evanphx/json-patch v0.5.2/go.mod h1:ZWS5hhDbVDyob71nXKNL0+PWn6ToqBHMikGIFbs31qQ= github.com/evanphx/json-patch v5.6.0+incompatible h1:jBYDEEiFBPxA0v50tFdvOzQQTCvpL6mnFh5mB2/l16U= github.com/evanphx/json-patch v5.6.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk= github.com/evanphx/json-patch/v5 v5.6.0 h1:b91NhWfaz02IuVxO9faSllyAtNXHMPkC5J8sJCLunww= github.com/evanphx/json-patch/v5 v5.6.0/go.mod h1:G79N1coSVB93tBe7j6PhzjmR3/2VvlbKOFpnXhI9Bw4= -github.com/fsnotify/fsnotify v1.5.4 h1:jRbGcIw6P2Meqdwuo0H1p6JVLbL5DHKAKlYndzMwVZI= -github.com/fsnotify/fsnotify v1.5.4/go.mod h1:OVB6XrOHzAwXMpEM7uPOzcehqUV2UqJxmVXmkdnm1bU= -github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04= -github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU= -github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8= -github.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8= -github.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as= -github.com/go-kit/kit v0.9.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as= -github.com/go-kit/log v0.1.0/go.mod h1:zbhenjAZHb184qTLMA9ZjW7ThYL0H2mk7Q6pNt4vbaY= -github.com/go-logfmt/logfmt v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9GBnD5lWE= -github.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V4qmtdjCk= -github.com/go-logfmt/logfmt v0.5.0/go.mod h1:wCYkCAKZfumFQihp8CzCvQ3paCTfi41vtzG1KdI/P7A= -github.com/go-logr/logr v0.1.0/go.mod h1:ixOQHD9gLJUVQQ2ZOR7zLEifBX6tGkNJF4QyIY7sIas= -github.com/go-logr/logr v1.2.0/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A= -github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A= -github.com/go-logr/logr v1.2.3 h1:2DntVwHkVopvECVRSlL5PSo9eG+cAkDCuckLubN+rq0= -github.com/go-logr/logr v1.2.3/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A= -github.com/go-logr/zapr v1.2.3 h1:a9vnzlIBPQBBkeaR9IuMUfmVOrQlkoC4YfPoFkX3T7A= -github.com/go-logr/zapr v1.2.3/go.mod h1:eIauM6P8qSvTw5o2ez6UEAfGjQKrxQTl5EoK+Qa2oG4= -github.com/go-openapi/jsonpointer v0.19.3/go.mod h1:Pl9vOtqEWErmShwVjC8pYs9cog34VGT37dQOVbmoatg= -github.com/go-openapi/jsonpointer v0.19.5 h1:gZr+CIYByUqjcgeLXnQu2gHYQC9o73G2XUeOFYEICuY= -github.com/go-openapi/jsonpointer v0.19.5/go.mod h1:Pl9vOtqEWErmShwVjC8pYs9cog34VGT37dQOVbmoatg= -github.com/go-openapi/jsonreference v0.19.5 h1:1WJP/wi4OjB4iV8KVbH73rQaoialJrqv8gitZLxGLtM= -github.com/go-openapi/jsonreference v0.19.5/go.mod h1:RdybgQwPxbL4UEjuAruzK1x3nE69AqPYEJeo/TWfEeg= -github.com/go-openapi/swag v0.19.5/go.mod h1:POnQmlKehdgb5mhVOsnJFsivZCEZ/vjK9gh66Z9tfKk= -github.com/go-openapi/swag v0.19.14 h1:gm3vOOXfiuw5i9p5N9xJvfjvuofpyvLA9Wr6QfK5Fng= -github.com/go-openapi/swag v0.19.14/go.mod h1:QYRuS/SOXUCsnplDa677K7+DxSOj6IPNl/eQntq43wQ= -github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY= +github.com/exponent-io/jsonpath v0.0.0-20151013193312-d6023ce2651d h1:105gxyaGwCFad8crR9dcMQWvV9Hvulu6hwUh4tWPJnM= +github.com/exponent-io/jsonpath v0.0.0-20151013193312-d6023ce2651d/go.mod h1:ZZMPRZwes7CROmyNKgQzC3XPs6L/G2EJLHddWejkmf4= +github.com/fatih/camelcase v1.0.0 h1:hxNvNX/xYBp0ovncs8WyWZrOrpBNub/JfaMvbURyft8= +github.com/fatih/camelcase v1.0.0/go.mod h1:yN2Sb0lFhZJUdVvtELVWefmrXpuZESvPmqwoZc+/fpc= +github.com/fsnotify/fsnotify v1.6.0 h1:n+5WquG0fcWoWp6xPWfHdbskMCQaFnG6PfBrh1Ky4HY= +github.com/fsnotify/fsnotify v1.6.0/go.mod h1:sl3t1tCWJFWoRz9R8WJCbQihKKwmorjAbSClcnxKAGw= +github.com/fvbommel/sortorder v1.1.0 h1:fUmoe+HLsBTctBDoaBwpQo5N+nrCp8g/BjKb/6ZQmYw= +github.com/fvbommel/sortorder v1.1.0/go.mod h1:uk88iVf1ovNn1iLfgUVU2F9o5eO30ui720w+kxuqRs0= +github.com/go-errors/errors v1.4.2 h1:J6MZopCL4uSllY1OfXM374weqZFFItUbrImctkmUxIA= +github.com/go-errors/errors v1.4.2/go.mod h1:sIVyrIiJhuEF+Pj9Ebtd6P/rEYROXFi3BopGUQ5a5Og= +github.com/go-logr/logr v1.2.4/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A= +github.com/go-logr/logr v1.3.0 h1:2y3SDp0ZXuc6/cjLSZ+Q3ir+QB9T/iG5yYRXqsagWSY= +github.com/go-logr/logr v1.3.0/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY= +github.com/go-logr/zapr v1.2.4 h1:QHVo+6stLbfJmYGkQ7uGHUCu5hnAFAj6mDe6Ea0SeOo= +github.com/go-logr/zapr v1.2.4/go.mod h1:FyHWQIzQORZ0QVE1BtVHv3cKtNLuXsbNLtpuhNapBOA= +github.com/go-openapi/jsonpointer v0.19.6 h1:eCs3fxoIi3Wh6vtgmLTOjdhSpiqphQ+DaPn38N2ZdrE= +github.com/go-openapi/jsonpointer v0.19.6/go.mod h1:osyAmYz/mB/C3I+WsTTSgw1ONzaLJoLCyoi6/zppojs= +github.com/go-openapi/jsonreference v0.20.2 h1:3sVjiK66+uXK/6oQ8xgcRKcFgQ5KXa2KvnJRumpMGbE= +github.com/go-openapi/jsonreference v0.20.2/go.mod h1:Bl1zwGIM8/wsvqjsOQLJ/SH+En5Ap4rVB5KVcIDZG2k= +github.com/go-openapi/swag v0.22.3 h1:yMBqmnQ0gyZvEb/+KzuWZOXgllrXT4SADYbvDaXHv/g= +github.com/go-openapi/swag v0.22.3/go.mod h1:UzaqsxGiab7freDnrUUra0MwWfN/q7tE4j+VcZ0yl14= +github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572 h1:tfuBGBXKqDEevZMzYi5KSi8KkcZtzBcTgAUUtapy0OI= +github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572/go.mod h1:9Pwr4B2jHnOSGXyyzV8ROjYa2ojvAY6HCGYYfMoC3Ls= github.com/gofrs/flock v0.8.1 h1:+gYjHKf32LDeiEEFhQaotPbLuUXjY5ZqxKgXy7n59aw= github.com/gofrs/flock v0.8.1/go.mod h1:F1TvTiK9OcQqauNUHlbJvyl9Qa1QvF/gOUDKA14jxHU= -github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ= github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q= github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q= github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q= -github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= -github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= -github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da h1:oI5xCqsCo564l8iNU+DwB5epxmsaqB+rhGL0m5jtYqE= github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A= -github.com/golang/mock v1.2.0/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A= -github.com/golang/mock v1.3.1/go.mod h1:sBzyDLLjw3U8JLTeZvSv8jJB+tU5PVekmnlKIyFUx0Y= -github.com/golang/mock v1.4.0/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw= -github.com/golang/mock v1.4.1/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw= -github.com/golang/mock v1.4.3/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw= -github.com/golang/mock v1.4.4/go.mod h1:l3mdAwkq5BuhzHwde/uurv3sEJeZMXNpwsxVWU71h+4= -github.com/golang/mock v1.5.0/go.mod h1:CWnOUgYIOo4TcNZ0wHX3YZCqsaM1I1Jvs6v3mP3KVu8= -github.com/golang/mock v1.6.0/go.mod h1:p6yTPP+5HYm5mzsMV8JkE6ZKdX+/wYM6Hr+LicevLPs= github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= -github.com/golang/protobuf v1.3.3/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw= -github.com/golang/protobuf v1.3.4/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw= -github.com/golang/protobuf v1.3.5/go.mod h1:6O5/vntMXwX2lRkT1hjjk0nAC1IDOTvTlVgjlRvqsdk= github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8= github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA= github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs= github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w= github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0= github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8= -github.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI= -github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI= github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk= -github.com/golang/protobuf v1.5.1/go.mod h1:DopwsBzvsk0Fs44TXzsVbJyPhcCPeIwnvohx4u74HPM= -github.com/golang/protobuf v1.5.2 h1:ROPKBNFfQgOUMifHyP+KYbvpjbdoFNs+aK7DXlji0Tw= -github.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY= -github.com/golang/snappy v0.0.3/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q= -github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= -github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= -github.com/google/gnostic v0.5.7-v3refs h1:FhTMOKj2VhjpouxvWJAV1TL304uMlb9zcDqkl6cEI54= -github.com/google/gnostic v0.5.7-v3refs/go.mod h1:73MKFl6jIHelAJNaBGFzt3SPtZULs9dYrGFt8OiIsHQ= +github.com/golang/protobuf v1.5.3 h1:KhyjKVUg7Usr/dYsdSqoFveMYd5ko72D+zANwlG1mmg= +github.com/golang/protobuf v1.5.3/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY= +github.com/google/btree v1.0.1 h1:gK4Kx5IaGY9CD5sPJ36FHiBJ6ZXl0kilRiiCj+jdYp4= +github.com/google/btree v1.0.1/go.mod h1:xXMiIv4Fb/0kKde4SpL7qlzvu5cMJDRkFDxJfI9uaxA= +github.com/google/gnostic-models v0.6.8 h1:yo/ABAfM5IMRsS1VnXjTBvUb61tFIHozhlYvRgGre9I= +github.com/google/gnostic-models v0.6.8/go.mod h1:5n7qKqH0f5wFt+aWF8CW6pZLLNOfYuF5OpfBSENuI8U= github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M= github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= -github.com/google/go-cmp v0.4.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= -github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= -github.com/google/go-cmp v0.5.3/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= -github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= -github.com/google/go-cmp v0.5.6/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= -github.com/google/go-cmp v0.5.7/go.mod h1:n+brtR0CgQNWTVd5ZUFpTBC8YFBDLK/h/bpaJ8/DtOE= -github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38= github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= +github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI= +github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= github.com/google/gofuzz v1.2.0 h1:xRy4A+RhZaiKjJ1bPfwQ8sedCA+YS2YcCHW6ec7JMi0= github.com/google/gofuzz v1.2.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= -github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs= -github.com/google/martian/v3 v3.0.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0= -github.com/google/martian/v3 v3.1.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0= -github.com/google/martian/v3 v3.2.1/go.mod h1:oBOf6HBosgwRXnUGWUB05QECsc6uvmMiJ3+6W4l/CUk= -github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc= -github.com/google/pprof v0.0.0-20190515194954-54271f7e092f/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc= -github.com/google/pprof v0.0.0-20191218002539-d4f498aebedc/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= -github.com/google/pprof v0.0.0-20200212024743-f11f1df84d12/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= -github.com/google/pprof v0.0.0-20200229191704-1ebb73c60ed3/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= -github.com/google/pprof v0.0.0-20200430221834-fc25d7d30c6d/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= -github.com/google/pprof v0.0.0-20200708004538-1a94d8640e99/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= -github.com/google/pprof v0.0.0-20201023163331-3e6fc7fc9c4c/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= -github.com/google/pprof v0.0.0-20201203190320-1bf35d6f28c2/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= -github.com/google/pprof v0.0.0-20210122040257-d980be63207e/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= -github.com/google/pprof v0.0.0-20210226084205-cbba55b83ad5/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= -github.com/google/pprof v0.0.0-20210601050228-01bbb1931b22/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= -github.com/google/pprof v0.0.0-20210609004039-a478d1d731e9/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= +github.com/google/pprof v0.0.0-20210720184732-4bb14d4b1be1 h1:K6RDEckDVWvDI9JAJYCmNdQXq6neHJOYx3V6jnqNEec= github.com/google/pprof v0.0.0-20210720184732-4bb14d4b1be1/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= -github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI= -github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= +github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 h1:El6M4kTTCOh6aBiKaUGG7oYTSPP8MxqL4YI3kZKwcP4= +github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510/go.mod h1:pupxD2MaaD3pAXIBCelhxNneeOaAeabZDe5s4K6zSpQ= github.com/google/uuid v1.3.0 h1:t6JiXgmwXMjEs8VusXIJk2BXHsn+wx8BZdTaoZ5fu7I= github.com/google/uuid v1.3.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= -github.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg= -github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk= -github.com/googleapis/gax-go/v2 v2.1.0/go.mod h1:Q3nei7sK6ybPYH7twZdmQpAd1MKb7pfu6SK+H1/DsU0= -github.com/googleapis/gax-go/v2 v2.1.1/go.mod h1:hddJymUZASv3XPyGkUpKj8pPO47Rmb0eJc8R6ouapiM= github.com/gorilla/websocket v1.4.2/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE= -github.com/grpc-ecosystem/grpc-gateway v1.16.0/go.mod h1:BDjrQk3hbvj6Nolgz8mAMFbcEtjT1g+wF4CSlocrBnw= -github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8= -github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8= -github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc= +github.com/gorilla/websocket v1.5.0 h1:PPwGk2jz7EePpoHN/+ClbZu8SPxiqlu12wZP/3sWmnc= +github.com/gorilla/websocket v1.5.0/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE= +github.com/gregjones/httpcache v0.0.0-20180305231024-9cad4c3443a7 h1:pdN6V1QBWetyv/0+wjACpqVH+eVULgEjkurDLq3goeM= +github.com/gregjones/httpcache v0.0.0-20180305231024-9cad4c3443a7/go.mod h1:FecbI9+v66THATjSRHfNgh1IVFe/9kFxbXtjV0ctIMA= github.com/ianlancetaylor/demangle v0.0.0-20200824232613-28f6c0f3b639/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc= -github.com/imdario/mergo v0.3.12 h1:b6R2BslTbIEToALKP7LxUvijTsNI9TAe80pLWN2g/HU= -github.com/imdario/mergo v0.3.12/go.mod h1:jmQim1M+e3UYxmgPu/WyfjB3N3VflVyUjjjwH0dnCYA= +github.com/imdario/mergo v0.3.6 h1:xTNEAn+kxVO7dTZGu0CegyqKZmoWFI0rF8UxjlB2d28= +github.com/imdario/mergo v0.3.6/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA= +github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8= +github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw= github.com/jessevdk/go-flags v1.4.0/go.mod h1:4FA24M0QyGHXBuZZK/XkWh8h0e1EYbRYJSGM75WSRxI= github.com/josharian/intern v1.0.0 h1:vlS4z54oSdjm0bgjRigI+G1HpF+tI+9rE5LLzOg8HmY= github.com/josharian/intern v1.0.0/go.mod h1:5DoeVV0s6jJacbCEi61lwdGj/aVlrQvzHFFd8Hwg//Y= -github.com/jpillora/backoff v1.0.0/go.mod h1:J/6gKK9jxlEcS3zixgDgUAsiuZ7yrSoa/FX5e0EB2j4= -github.com/json-iterator/go v1.1.6/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU= -github.com/json-iterator/go v1.1.10/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4= -github.com/json-iterator/go v1.1.11/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4= github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM= github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo= -github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU= -github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk= -github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w= -github.com/julienschmidt/httprouter v1.3.0/go.mod h1:JR6WtHb+2LUe8TCKY3cZOxFyyO8IZAc4RVcycCCAKdM= github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8= github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck= -github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= -github.com/konsorten/go-windows-terminal-sequences v1.0.3/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= -github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc= github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo= -github.com/kr/pretty v0.2.0/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI= +github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI= +github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= +github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ= github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI= github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= -github.com/mailru/easyjson v0.0.0-20190614124828-94de47d64c63/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc= -github.com/mailru/easyjson v0.0.0-20190626092158-b2ccc519800e/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc= -github.com/mailru/easyjson v0.7.6 h1:8yTIVnZgCoiM1TgqoeTl+LfU5Jg6/xL3QhGQnimLYnA= -github.com/mailru/easyjson v0.7.6/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc= -github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0= -github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 h1:I0XW9+e1XWDxdcEniV4rQAIOPUGDq67JSCiRCgGCZLI= -github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369/go.mod h1:BSXmuO+STAnVfrANrmjBb36TMTDstsz7MSK+HVaYKv4= +github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de h1:9TO3cAIGXtEhnIaL+V+BEER86oLrvS+kWobKpbJuye0= +github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de/go.mod h1:zAbeS9B/r2mtpb6U+EI2rYA5OAXxsYw6wTamcNW+zcE= +github.com/mailru/easyjson v0.7.7 h1:UGYAvKxe3sBsEDzO8ZeWOSlIQfWFlxbzLZe7hwFURr0= +github.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc= +github.com/matttproud/golang_protobuf_extensions v1.0.4 h1:mmDVorXM7PCGKw94cs5zkfA9PSy5pEvNWRP0ET0TIVo= +github.com/matttproud/golang_protobuf_extensions v1.0.4/go.mod h1:BSXmuO+STAnVfrANrmjBb36TMTDstsz7MSK+HVaYKv4= +github.com/mitchellh/go-wordwrap v1.0.1 h1:TLuKupo69TCn6TQSyGxwI1EblZZEsQ0vMlAFQflz0v0= +github.com/mitchellh/go-wordwrap v1.0.1/go.mod h1:R62XHJLzvMFRBbcrT7m7WgmE1eOyTSsCt+hzestvNj0= github.com/moby/spdystream v0.2.0 h1:cjW1zVyyoiM0T7b6UoySUFqzXMoqRckQtXwGPiBhOM8= github.com/moby/spdystream v0.2.0/go.mod h1:f7i0iNDQJ059oMTcWxx8MA/zKFIuD/lY+0GqbN2Wy8c= +github.com/moby/term v0.0.0-20221205130635-1aeaba878587 h1:HfkjXDfhgVaN5rmueG8cL8KKeFNecRCXFhaJ2qZ5SKA= +github.com/moby/term v0.0.0-20221205130635-1aeaba878587/go.mod h1:8FzsFHVUBGZdbDsJw/ot+X+d5HLUbvklYLJ9uGfcI3Y= github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg= github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= -github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0= -github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0= github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M= github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk= +github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00 h1:n6/2gBQ3RWajuToeY6ZtZTIKv2v7ThUy5KKusIT0yc0= +github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00/go.mod h1:Pm3mSP3c5uWn86xMLZ5Sa7JB9GsEZySvHYXCTK4E9q4= github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA= github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ= -github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U= -github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U= -github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e h1:fD57ERR4JtEqsWbfPhv4DMiApHyliiK5xCTNVSPiaAs= -github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e/go.mod h1:zD1mROLANZcx1PVRCS0qkT7pwLkGfwJo4zjcN/Tysno= -github.com/nxadm/tail v1.4.8 h1:nPr65rt6Y5JFSKQO7qToXr7pePgD6Gwiw05lkbyAQTE= -github.com/onsi/ginkgo v1.16.5 h1:8xi0RTUf59SOSfEtZMvwTvXYMzG4gV23XVHOZiXNtnE= -github.com/onsi/ginkgo/v2 v2.5.0 h1:TRtrvv2vdQqzkwrQ1ke6vtXf7IK34RBUJafIy1wMwls= -github.com/onsi/ginkgo/v2 v2.5.0/go.mod h1:Luc4sArBICYCS8THh8v3i3i5CuSZO+RaQRaJoeNwomw= -github.com/onsi/gomega v1.24.1 h1:KORJXNNTzJXzu4ScJWssJfJMnJ+2QJqhoQSRwNlze9E= -github.com/onsi/gomega v1.24.1/go.mod h1:3AOiACssS3/MajrniINInwbfOOtfZvplPzuRSmvt1jM= -github.com/oracle/oci-go-sdk/v65 v65.26.1 h1:Ms20RSRj+CuvQmw5ET1TkmzxLBI+bmLjZ6NYANA3gkk= -github.com/oracle/oci-go-sdk/v65 v65.26.1/go.mod h1:oyMrMa1vOzzKTmPN+kqrTR9y9kPA2tU1igN3NUSNTIE= -github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= +github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f h1:y5//uYreIhSUg3J1GEMiLbxo1LJaP8RfCpH6pymGZus= +github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f/go.mod h1:ZdcZmHo+o7JKHSa8/e818NopupXU1YMK5fe1lsApnBw= +github.com/onsi/ginkgo/v2 v2.13.0 h1:0jY9lJquiL8fcf3M4LAXN5aMlS/b2BV86HFFPCPMgE4= +github.com/onsi/ginkgo/v2 v2.13.0/go.mod h1:TE309ZR8s5FsKKpuB1YAQYBzCaAfUgatB/xlT/ETL/o= +github.com/onsi/gomega v1.29.0 h1:KIA/t2t5UBzoirT4H9tsML45GEbo3ouUnBHsCfD2tVg= +github.com/onsi/gomega v1.29.0/go.mod h1:9sxs+SwGrKI0+PWe4Fxa9tFQQBG5xSsSbMXOI8PPpoQ= +github.com/oracle/oci-go-sdk/v65 v65.49.3 h1:HHv+XMZiBYHtoU8Ac/fURdp9v1vJPPCpIbJAWeadREw= +github.com/oracle/oci-go-sdk/v65 v65.49.3/go.mod h1:IBEV9l1qBzUpo7zgGaRUhbB05BVfcDGYRFBCPlTcPp0= +github.com/peterbourgon/diskv v2.0.1+incompatible h1:UBdAOUP5p4RWqPBg048CAvpKN+vxiaj6gdUUzhl4XmI= +github.com/peterbourgon/diskv v2.0.1+incompatible/go.mod h1:uqqh8zWWbv1HBMNONnaR/tNboyR3/BZd58JJSHlUSCU= github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4= github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= -github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw= -github.com/prometheus/client_golang v1.0.0/go.mod h1:db9x61etRT2tGnBNRi70OPL5FsnadC4Ky3P0J6CfImo= -github.com/prometheus/client_golang v1.7.1/go.mod h1:PY5Wy2awLA44sXw4AOSfFBetzPP4j5+D6mVACh+pe2M= -github.com/prometheus/client_golang v1.11.0/go.mod h1:Z6t4BnS23TR94PD6BsDNk8yVqroYurpAkEiz0P2BEV0= -github.com/prometheus/client_golang v1.12.2 h1:51L9cDoUHVrXx4zWYlcLQIZ+d+VXHgqnYKkIuq4g/34= -github.com/prometheus/client_golang v1.12.2/go.mod h1:3Z9XVyYiZYEO+YQWt3RD2R3jrbd179Rt297l4aS6nDY= -github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo= -github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= +github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring v0.62.0 h1:55138zTXw/yRYizPxZ672I/aDD7Yte3uYRAfUjWUu2M= +github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring v0.62.0/go.mod h1:j51242bf6LQwvJ1JPKWApzTnifmCwcQq0i1p29ylWiM= +github.com/prometheus/client_golang v1.16.0 h1:yk/hx9hDbrGHovbci4BY+pRMfSuuat626eFsHb7tmT8= +github.com/prometheus/client_golang v1.16.0/go.mod h1:Zsulrv/L9oM40tJ7T815tM89lFEugiJ9HzIqaAx4LKc= github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= -github.com/prometheus/client_model v0.2.0 h1:uq5h0d+GuxiXLJLNABMgp2qUWDPiLvgCzz2dUR+/W/M= -github.com/prometheus/client_model v0.2.0/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= -github.com/prometheus/common v0.4.1/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4= -github.com/prometheus/common v0.10.0/go.mod h1:Tlit/dnDKsSWFlCLTWaA1cyBgKHSMdTB80sz/V91rCo= -github.com/prometheus/common v0.26.0/go.mod h1:M7rCNAaPfAosfx8veZJCuw84e35h3Cfd9VFqTh1DIvc= -github.com/prometheus/common v0.32.1 h1:hWIdL3N2HoUx3B8j3YN9mWor0qhY/NlEKZEaXxuIRh4= -github.com/prometheus/common v0.32.1/go.mod h1:vu+V0TpY+O6vW9J44gczi3Ap/oXXR10b+M/gUGO4Hls= -github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk= -github.com/prometheus/procfs v0.0.2/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA= -github.com/prometheus/procfs v0.1.3/go.mod h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4OA4YeYWdaU= -github.com/prometheus/procfs v0.6.0/go.mod h1:cz+aTbrPOrUb4q7XlbU9ygM+/jj0fzG6c1xBZuNvfVA= -github.com/prometheus/procfs v0.7.3 h1:4jVXhlkAyzOScmCkXBTOLRLTz8EeU+eyjrwB/EPq0VU= -github.com/prometheus/procfs v0.7.3/go.mod h1:cz+aTbrPOrUb4q7XlbU9ygM+/jj0fzG6c1xBZuNvfVA= -github.com/rogpeppe/fastuuid v1.2.0/go.mod h1:jVj6XXZzXRy/MSR5jhDC/2q6DgLz+nrA6LYCDYWNEvQ= -github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4= -github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo= -github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE= -github.com/sirupsen/logrus v1.6.0/go.mod h1:7uNnSEd1DgxDLC74fIahvMZmmYsHGZGEOFrfsX/uA88= +github.com/prometheus/client_model v0.4.0 h1:5lQXD3cAg1OXBf4Wq03gTrXHeaV0TQvGfUooCfx1yqY= +github.com/prometheus/client_model v0.4.0/go.mod h1:oMQmHW1/JoDwqLtg57MGgP/Fb1CJEYF2imWWhWtMkYU= +github.com/prometheus/common v0.44.0 h1:+5BrQJwiBB9xsMygAB3TNvpQKOwlkc25LbISbrdOOfY= +github.com/prometheus/common v0.44.0/go.mod h1:ofAIvZbQ1e/nugmZGz4/qCb9Ap1VoSTIO7x0VV9VvuY= +github.com/prometheus/procfs v0.10.1 h1:kYK1Va/YMlutzCGazswoHKo//tZVlFpKYh+PymziUAg= +github.com/prometheus/procfs v0.10.1/go.mod h1:nwNm2aOCAYw8uTR/9bWRREkZFxAUcWzPHWJq+XBB/FM= +github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjRBZyWFQ= +github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog= +github.com/russross/blackfriday/v2 v2.1.0 h1:JIOH55/0cWyOuilr9/qlrm0BSXldqnqwMsf35Ld67mk= +github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM= +github.com/sergi/go-diff v1.1.0 h1:we8PVUC3FE2uYfodKH/nBHMSetSfHDR6scGdBi+erh0= +github.com/sergi/go-diff v1.1.0/go.mod h1:STckp+ISIX8hZLjrqAeVduY0gWCT9IjLuqbuNXdaHfM= github.com/sony/gobreaker v0.5.0 h1:dRCvqm0P490vZPmy7ppEk2qCnCieBooFJ+YoXGYB+yg= github.com/sony/gobreaker v0.5.0/go.mod h1:ZKptC7FHNvhBz7dN2LGjPVBz2sZJmc0/PkyDJOjmxWY= -github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA= -github.com/spf13/afero v1.2.2/go.mod h1:9ZxEEn6pIJ8Rxe320qSDBk6AsU0r9pR7Q4OcevTdifk= +github.com/spf13/cobra v1.7.0 h1:hyqWnYt1ZQShIddO5kBpj3vu05/++x6tJ6dg8EC572I= +github.com/spf13/cobra v1.7.0/go.mod h1:uLxZILRyS/50WlhOIKD7W6V5bgeIt+4sICxh6uRMrb0= github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA= github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= -github.com/stoewer/go-strcase v1.2.0/go.mod h1:IBiWB2sKIp3wVVQ3Y035++gc+knqhUQag1KpM8ahLw8= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= -github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= -github.com/stretchr/objx v0.4.0 h1:M2gUjqZET1qApGOWNSnZ49BAIMX4F/1plDv3+l31EJ4= github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw= -github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs= +github.com/stretchr/objx v0.5.0 h1:1zr/of2m5FGMsad5YfcqgdqdWrIhu+EBEJRhR1U7z/c= +github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo= github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= -github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4= -github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA= github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= -github.com/stretchr/testify v1.8.0 h1:pSgiaMZlXftHpm5L7V1+rVB+AZJydKsMxsQBIJw4PKk= -github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= +github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU= +github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4= +github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcUk= +github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo= +github.com/xlab/treeprint v1.2.0 h1:HzHnuAF1plUN2zGlAFHbSQP2qJ0ZAD3XF5XD7OesXRQ= +github.com/xlab/treeprint v1.2.0/go.mod h1:gj5Gd3gPdKtR1ikdDK6fnFLdmIS0X30kTTuNd/WEJu0= github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= -github.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.3.5/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k= -go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU= -go.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8= -go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= -go.opencensus.io v0.22.3/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= -go.opencensus.io v0.22.4/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= -go.opencensus.io v0.22.5/go.mod h1:5pWMHQbX5EPX2/62yrJeAkowc+lfs/XD7Uxpq3pI6kk= -go.opencensus.io v0.23.0/go.mod h1:XItmlyltB5F7CS4xOC1DcqMoFqwtC6OG2xF7mCv7P7E= -go.opentelemetry.io/proto/otlp v0.7.0/go.mod h1:PqfVotwruBrMGOCsRd/89rSnXhoiJIqeYNgFYFoEGnI= +go.starlark.net v0.0.0-20230525235612-a134d8f9ddca h1:VdD38733bfYv5tUZwEIskMM93VanwNIi5bIKnDrJdEY= +go.starlark.net v0.0.0-20230525235612-a134d8f9ddca/go.mod h1:jxU+3+j+71eXOW14274+SmmuW82qJzl6iZSeqEtTGds= go.uber.org/atomic v1.7.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc= -go.uber.org/atomic v1.9.0 h1:ECmE8Bn/WFTYwEW/bpKD3M8VtR/zQVbavAoalC1PYyE= -go.uber.org/atomic v1.9.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc= -go.uber.org/goleak v1.1.10/go.mod h1:8a7PlsEVH3e/a/GLqe5IIrQx6GzcnRmZEufDUTk4A7A= -go.uber.org/goleak v1.1.12 h1:gZAh5/EyT/HQwlpkCy6wTpqfH9H8Lz8zbm3dZh+OyzA= +go.uber.org/goleak v1.1.11/go.mod h1:cwTWslyiVhfpKIDGSZEM2HlOvcqm+tG4zioyIeLoqMQ= +go.uber.org/goleak v1.2.1 h1:NBol2c7O1ZokfZ0LEU9K6Whx/KnwvepVetCUhtKja4A= +go.uber.org/goleak v1.2.1/go.mod h1:qlT2yGI9QafXHhZZLxlSuNsMw3FFLxBr+tBRlmO1xH4= go.uber.org/multierr v1.6.0/go.mod h1:cdWPpRnG4AhwMwsgIHip0KRBQjJy5kYEpYjJxpXp9iU= -go.uber.org/multierr v1.7.0 h1:zaiO/rmgFjbmCXdSYJWQcdvOCsthmdaHfr3Gm2Kx4Ec= -go.uber.org/multierr v1.7.0/go.mod h1:7EAYxJLBy9rStEaz58O2t4Uvip6FSURkq8/ppBp95ak= -go.uber.org/zap v1.19.0/go.mod h1:xg/QME4nWcxGxrpdeYfq7UvYrLh66cuVKdrbD1XF/NI= -go.uber.org/zap v1.23.0 h1:OjGQ5KQDEUawVHxNwQgPpiypGHOxo2mNZsOqTak4fFY= -go.uber.org/zap v1.23.0/go.mod h1:D+nX8jyLsMHMYrln8A0rJjFt/T/9/bGgIhAqxv5URuY= -golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= +go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0= +go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y= +go.uber.org/zap v1.24.0/go.mod h1:2kMP+WWQ8aoFoedH3T2sq6iJ2yDWpHbP0f6MQbS9Gkg= +go.uber.org/zap v1.26.0 h1:sI7k6L95XOKS281NhVKOFCUNIvv9e0w4BF8N3u+tCRo= +go.uber.org/zap v1.26.0/go.mod h1:dtElttAiwGvoJ/vj4IwHBS/gXsEu/pZ50mUIRWuG0so= golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= -golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= -golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= -golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= -golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8= -golang.org/x/exp v0.0.0-20190829153037-c13cbed26979/go.mod h1:86+5VVa7VpoJ4kLfm080zCjGlMRFzhUhsZKEZO7MGek= -golang.org/x/exp v0.0.0-20191030013958-a1ab85dbe136/go.mod h1:JXzH8nQsPlswgeRAPE3MuO9GYsAcnJvJ4vnMwN/5qkY= -golang.org/x/exp v0.0.0-20191129062945-2f5052295587/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4= -golang.org/x/exp v0.0.0-20191227195350-da58074b4299/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4= -golang.org/x/exp v0.0.0-20200119233911-0405dc783f0a/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4= -golang.org/x/exp v0.0.0-20200207192155-f17229e696bd/go.mod h1:J/WKrq2StrnmMY6+EHIKF9dgMWnmCNThgcyBT1FY9mM= -golang.org/x/exp v0.0.0-20200224162631-6cc2880d07d6/go.mod h1:3jZMyOhIsHpP37uCMkUooju7aAi5cS1Q23tOzKc+0MU= -golang.org/x/image v0.0.0-20190227222117-0694c2d4d067/go.mod h1:kZ7UVZpmo3dzQBMxlp+ypCbDeSB+sBbTgSJuh5dn5js= -golang.org/x/image v0.0.0-20190802002840-cff245a6509b/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0= +golang.org/x/exp v0.0.0-20220722155223-a9213eeb770e h1:+WEEuIdZHnUeJJmEUjyYC2gfUMj69yZXw17EnHg/otA= +golang.org/x/exp v0.0.0-20220722155223-a9213eeb770e/go.mod h1:Kr81I6Kryrl9sr8s2FK3vxD90NdsKWRuOIl2O4CvYbA= golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE= golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU= -golang.org/x/lint v0.0.0-20190301231843-5614ed5bae6f/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE= golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= -golang.org/x/lint v0.0.0-20190409202823-959b441ac422/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= -golang.org/x/lint v0.0.0-20190909230951-414d861bb4ac/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= golang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= -golang.org/x/lint v0.0.0-20191125180803-fdd1cda4f05f/go.mod h1:5qLYkcX4OjUUV8bRuDixDT3tpyyb+LUpUlRWLxfhWrs= -golang.org/x/lint v0.0.0-20200130185559-910be7a94367/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY= -golang.org/x/lint v0.0.0-20200302205851-738671d3881b/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY= -golang.org/x/lint v0.0.0-20201208152925-83fdc39ff7b5/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY= -golang.org/x/lint v0.0.0-20210508222113-6edffad5e616/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY= -golang.org/x/mobile v0.0.0-20190312151609-d3739f865fa6/go.mod h1:z+o9i4GpDbdi3rU15maQ/Ox0txvL9dWGYEHz965HBQE= -golang.org/x/mobile v0.0.0-20190719004257-d2bd2a29d028/go.mod h1:E/iHnbuqvinMTCcRqshq8CkpyQDoeVncDDYHnLhea+o= -golang.org/x/mod v0.0.0-20190513183733-4bf6d317e70e/go.mod h1:mXi4GBBbnImb6dmsKGUJ2LatrhH/nqhxcFungHvyanc= -golang.org/x/mod v0.1.0/go.mod h1:0QHyrYULN0/3qlju5TqG8bIK38QM8yzMo5ekMj3DlcY= -golang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg= -golang.org/x/mod v0.1.1-0.20191107180719-034126e5016b/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg= golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= -golang.org/x/mod v0.4.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= -golang.org/x/mod v0.4.1/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= -golang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= -golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= -golang.org/x/net v0.0.0-20190501004415-9ce7a6920f09/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= -golang.org/x/net v0.0.0-20190503192946-f4e77d36d62c/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks= -golang.org/x/net v0.0.0-20190613194153-d28f0bde5980/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= -golang.org/x/net v0.0.0-20190628185345-da137c7871d7/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= -golang.org/x/net v0.0.0-20190724013045-ca1201d0de80/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= -golang.org/x/net v0.0.0-20190827160401-ba9fcec4b297/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= -golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= -golang.org/x/net v0.0.0-20200114155413-6afb5195e5aa/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= -golang.org/x/net v0.0.0-20200202094626-16171245cfb2/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= -golang.org/x/net v0.0.0-20200222125558-5a598a2470a0/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= -golang.org/x/net v0.0.0-20200301022130-244492dfa37a/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= -golang.org/x/net v0.0.0-20200324143707-d3edc9973b7e/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= -golang.org/x/net v0.0.0-20200501053045-e0ff5e5a1de5/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= -golang.org/x/net v0.0.0-20200506145744-7e3656a0809f/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= -golang.org/x/net v0.0.0-20200513185701-a91f0712d120/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= -golang.org/x/net v0.0.0-20200520182314-0ba52f642ac2/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= -golang.org/x/net v0.0.0-20200625001655-4c5254603344/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= -golang.org/x/net v0.0.0-20200707034311-ab3426394381/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= -golang.org/x/net v0.0.0-20200822124328-c89045814202/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= -golang.org/x/net v0.0.0-20201031054903-ff519b6c9102/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= -golang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= -golang.org/x/net v0.0.0-20201209123823-ac852fbbde11/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= -golang.org/x/net v0.0.0-20210119194325-5f4716e94777/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= -golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= -golang.org/x/net v0.0.0-20210316092652-d523dce5a7f4/go.mod h1:RBQZq4jEuRlivfhVLdyRGr576XBO4/greRjx4P4O3yc= golang.org/x/net v0.0.0-20210405180319-a5a99cb37ef4/go.mod h1:p54w0d4576C0XHj96bSt6lcn1PtDYWL6XObtHCRCNQM= -golang.org/x/net v0.0.0-20210503060351-7fd8e65b6420/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= -golang.org/x/net v0.0.0-20210525063256-abc453219eb5/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= -golang.org/x/net v0.7.0 h1:rJrUqqhjsgNp7KqAIc25s9pZnjU7TUcSY7HcVZjdn1g= -golang.org/x/net v0.7.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs= +golang.org/x/net v0.19.0 h1:zTwKpTd2XuCqf8huc7Fo2iSy+4RHPd10s4KzeTnVr1c= +golang.org/x/net v0.19.0/go.mod h1:CfAk/cbD4CthTvqiEl8NpboMuiuOYsAr/7NOjZJtv1U= golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U= -golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= -golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= -golang.org/x/oauth2 v0.0.0-20191202225959-858c2ad4c8b6/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= -golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= -golang.org/x/oauth2 v0.0.0-20200902213428-5d25da1a8d43/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= -golang.org/x/oauth2 v0.0.0-20201109201403-9fd604954f58/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= -golang.org/x/oauth2 v0.0.0-20201208152858-08078c50e5b5/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= -golang.org/x/oauth2 v0.0.0-20210218202405-ba52d332ba99/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= -golang.org/x/oauth2 v0.0.0-20210220000619-9bb904979d93/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= -golang.org/x/oauth2 v0.0.0-20210313182246-cd4f82c27b84/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= -golang.org/x/oauth2 v0.0.0-20210514164344-f6687ab2804c/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= -golang.org/x/oauth2 v0.0.0-20210628180205-a41e5a781914/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= -golang.org/x/oauth2 v0.0.0-20210805134026-6f1e6394065a/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= -golang.org/x/oauth2 v0.0.0-20210819190943-2bc19b11175f/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= -golang.org/x/oauth2 v0.0.0-20211104180415-d3ed0bb246c8 h1:RerP+noqYHUQ8CMRcPlC2nvTa4dcBIjegkuWdcUDuqg= -golang.org/x/oauth2 v0.0.0-20211104180415-d3ed0bb246c8/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= +golang.org/x/oauth2 v0.10.0 h1:zHCpF2Khkwy4mMB4bv0U37YtJdTGW8jI0glAApi0Kh8= +golang.org/x/oauth2 v0.10.0/go.mod h1:kTpgurOux7LqtuxjuyZa4Gj2gdezIt/jQtGnNFfypQI= golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.0.0-20200317015054-43a5402ce75a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.5.0 h1:60k92dhOjHxJkrqnwsfl8KuaHbn/5dl0lUPUklKo3qE= +golang.org/x/sync v0.5.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk= golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= -golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= -golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= -golang.org/x/sys v0.0.0-20190312061237-fead79001313/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20190422165155-953cdadca894/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20190502145724-3ef323f4f1fd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20190606165138-5da285871e9c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20190624142023-c5567b49c5d0/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20191001151750-bb3f8db39f24/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20191228213918-04cbcbbfeed8/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200106162015-b016eb3dc98e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200113162924-86b910548bc1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200122134326-e047566fdf82/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200212091648-12a6c2dcc1e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200302150141-5c8b2ff67527/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200331124033-c3d80250170d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200501052902-10377860bb8e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200511232937-7e40ca221e25/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200515095857-1151b9dac4a9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200523222454-059865788121/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200615200032-f1bc736245b1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200625212154-ddb9806d33ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200803210538-64077c9b5642/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200905004654-be1d3432aa8f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20201201145000-ef89a241ccb3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20210104204734-6f8348627aad/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20210119212857-b64e53b001e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20210220050731-9a76102bfb43/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20210305230114-8fe3ee5dd75b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20210315160823-c6e025ad8005/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20210320140829-1e4c9ba3b0c4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210510120138-977fb7262007/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20210514084401-e8d321eab015/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20210603081109-ebe580a85c40/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20210603125802-9665404d3644/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20210616094352-59db8d763f22/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20210806184541-e5e7981a1069/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20210823070655-63515b42dcdf/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20210908233432-aa78b53d3365/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20211124211545-fe61309f8881/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20211210111614-af8b64212486/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20220114195835-da31bd327af9/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20220209214540-3681064d5158/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20220412211240-33da011f77ad/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.5.0 h1:MUK/U/4lj1t1oPg0HfuXDN/Z1wv31ZJ/YcPiGccS4DU= -golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.0.0-20220908164124-27713097b956/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.8.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.15.0 h1:h48lPFYpsTvQJZF4EKyI4aLHaev3CxivZmv7yZig9pc= +golang.org/x/sys v0.15.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= -golang.org/x/term v0.5.0 h1:n2a8QNdAb0sZNpU9R1ALUXBbY+w51fCQDN+7EdxNBsY= -golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k= -golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= +golang.org/x/term v0.0.0-20220526004731-065cf7ba2467/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8= +golang.org/x/term v0.15.0 h1:y/Oo/a/q3IXu26lQgl04j/gjuBDOBlx7X6Om1j2CPW4= +golang.org/x/term v0.15.0/go.mod h1:BDl952bC7+uMoWR75FIrCDx79TPU9oHkTZ9yRbYOrX0= golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= -golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk= golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= -golang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= -golang.org/x/text v0.3.5/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= -golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= -golang.org/x/text v0.7.0 h1:4BRB4x83lYWy72KwLD/qYDuTu7q9PjSagHvijDw7cLo= -golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8= -golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= -golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= -golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= -golang.org/x/time v0.0.0-20220609170525-579cf78fd858 h1:Dpdu/EMxGMFgq0CeYMh4fazTD2vtlZRYE7wyynxJb9U= -golang.org/x/time v0.0.0-20220609170525-579cf78fd858/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= +golang.org/x/text v0.14.0 h1:ScX5w1eTa3QqT8oi6+ziP7dTV1S2+ALU0bI+0zXKWiQ= +golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU= +golang.org/x/time v0.3.0 h1:rg5rLMjNzMS1RkNLzCG38eapWhnYLFYXDXj2gOlr8j4= +golang.org/x/time v0.3.0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY= golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= -golang.org/x/tools v0.0.0-20190312151545-0bb0c0a6e846/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= -golang.org/x/tools v0.0.0-20190312170243-e65039ee4138/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= -golang.org/x/tools v0.0.0-20190425150028-36563e24a262/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= -golang.org/x/tools v0.0.0-20190506145303-2d16b83fe98c/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= -golang.org/x/tools v0.0.0-20190606124116-d0a3d012864b/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= -golang.org/x/tools v0.0.0-20190621195816-6e04913cbbac/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= -golang.org/x/tools v0.0.0-20190628153133-6cdbf07be9d0/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= -golang.org/x/tools v0.0.0-20190816200558-6889da9d5479/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= -golang.org/x/tools v0.0.0-20190911174233-4f2ddba30aff/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= -golang.org/x/tools v0.0.0-20191012152004-8de300cfc20a/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= -golang.org/x/tools v0.0.0-20191108193012-7d206e10da11/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= -golang.org/x/tools v0.0.0-20191113191852-77e3bb0ad9e7/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= -golang.org/x/tools v0.0.0-20191115202509-3a792d9c32b2/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= -golang.org/x/tools v0.0.0-20191125144606-a911d9008d1f/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= -golang.org/x/tools v0.0.0-20191130070609-6e064ea0cf2d/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= -golang.org/x/tools v0.0.0-20191216173652-a0e659d51361/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= -golang.org/x/tools v0.0.0-20191227053925-7b8e75db28f4/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= -golang.org/x/tools v0.0.0-20200117161641-43d50277825c/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= -golang.org/x/tools v0.0.0-20200122220014-bf1340f18c4a/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= -golang.org/x/tools v0.0.0-20200130002326-2f3ba24bd6e7/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= -golang.org/x/tools v0.0.0-20200204074204-1cc6d1ef6c74/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= -golang.org/x/tools v0.0.0-20200207183749-b753a1ba74fa/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= -golang.org/x/tools v0.0.0-20200212150539-ea181f53ac56/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= -golang.org/x/tools v0.0.0-20200224181240-023911ca70b2/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= -golang.org/x/tools v0.0.0-20200227222343-706bc42d1f0d/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= -golang.org/x/tools v0.0.0-20200304193943-95d2e580d8eb/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw= -golang.org/x/tools v0.0.0-20200312045724-11d5b4c81c7d/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw= -golang.org/x/tools v0.0.0-20200331025713-a30bf2db82d4/go.mod h1:Sl4aGygMT6LrqrWclx+PTx3U+LnKx/seiNR+3G19Ar8= -golang.org/x/tools v0.0.0-20200501065659-ab2804fb9c9d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= -golang.org/x/tools v0.0.0-20200512131952-2bc93b1c0c88/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= -golang.org/x/tools v0.0.0-20200515010526-7d3b6ebf133d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= -golang.org/x/tools v0.0.0-20200618134242-20370b0cb4b2/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= -golang.org/x/tools v0.0.0-20200729194436-6467de6f59a7/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA= -golang.org/x/tools v0.0.0-20200804011535-6c149bb5ef0d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA= -golang.org/x/tools v0.0.0-20200825202427-b303f430e36d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA= -golang.org/x/tools v0.0.0-20200904185747-39188db58858/go.mod h1:Cj7w3i3Rnn0Xh82ur9kSqwfTHTeVxaDqrfMjpcNT6bE= -golang.org/x/tools v0.0.0-20201110124207-079ba7bd75cd/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= -golang.org/x/tools v0.0.0-20201201161351-ac6f37ff4c2a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= -golang.org/x/tools v0.0.0-20201208233053-a543418bbed2/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= -golang.org/x/tools v0.0.0-20210105154028-b0ab187a4818/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= -golang.org/x/tools v0.1.0/go.mod h1:xkSsbof2nBLbhDlRMhhhyNLN/zl3eTqcnHD5viDpcZ0= -golang.org/x/tools v0.1.1/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= -golang.org/x/tools v0.1.2/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= -golang.org/x/tools v0.1.3/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= -golang.org/x/tools v0.1.4/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= golang.org/x/tools v0.1.5/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= +golang.org/x/tools v0.16.1 h1:TLyB3WofjdOEepBHAU20JdNC1Zbg87elYofWYAY5oZA= +golang.org/x/tools v0.16.1/go.mod h1:kYVVN6I1mBNoB1OX+noeBjbRk4IUEPa7JJ+TJMEooJ0= golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= -gomodules.xyz/jsonpatch/v2 v2.2.0 h1:4pT439QV83L+G9FkcCriY6EkpcK6r6bK+A5FBUMI7qY= -gomodules.xyz/jsonpatch/v2 v2.2.0/go.mod h1:WXp+iVDkoLQqPudfQ9GBlwB2eZ5DKOnjQZCYdOS8GPY= -google.golang.org/api v0.4.0/go.mod h1:8k5glujaEP+g9n7WNsDg8QP6cUVNI86fCNMcbazEtwE= -google.golang.org/api v0.7.0/go.mod h1:WtwebWUNSVBH/HAw79HIFXZNqEvBhG+Ra+ax0hx3E3M= -google.golang.org/api v0.8.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg= -google.golang.org/api v0.9.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg= -google.golang.org/api v0.13.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI= -google.golang.org/api v0.14.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI= -google.golang.org/api v0.15.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI= -google.golang.org/api v0.17.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= -google.golang.org/api v0.18.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= -google.golang.org/api v0.19.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= -google.golang.org/api v0.20.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= -google.golang.org/api v0.22.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= -google.golang.org/api v0.24.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE= -google.golang.org/api v0.28.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE= -google.golang.org/api v0.29.0/go.mod h1:Lcubydp8VUV7KeIHD9z2Bys/sm/vGKnG1UHuDBSrHWM= -google.golang.org/api v0.30.0/go.mod h1:QGmEvQ87FHZNiUVJkT14jQNYJ4ZJjdRF23ZXz5138Fc= -google.golang.org/api v0.35.0/go.mod h1:/XrVsuzM0rZmrsbjJutiuftIzeuTQcEeaYcSk/mQ1dg= -google.golang.org/api v0.36.0/go.mod h1:+z5ficQTmoYpPn8LCUNVpK5I7hwkpjbcgqA7I34qYtE= -google.golang.org/api v0.40.0/go.mod h1:fYKFpnQN0DsDSKRVRcQSDQNtqWPfM9i+zNPxepjRCQ8= -google.golang.org/api v0.41.0/go.mod h1:RkxM5lITDfTzmyKFPt+wGrCJbVfniCr2ool8kTBzRTU= -google.golang.org/api v0.43.0/go.mod h1:nQsDGjRXMo4lvh5hP0TKqF244gqhGcr/YSIykhUk/94= -google.golang.org/api v0.47.0/go.mod h1:Wbvgpq1HddcWVtzsVLyfLp8lDg6AA241LmgIL59tHXo= -google.golang.org/api v0.48.0/go.mod h1:71Pr1vy+TAZRPkPs/xlCf5SsU8WjuAWv1Pfjbtukyy4= -google.golang.org/api v0.50.0/go.mod h1:4bNT5pAuq5ji4SRZm+5QIkjny9JAyVD/3gaSihNefaw= -google.golang.org/api v0.51.0/go.mod h1:t4HdrdoNgyN5cbEfm7Lum0lcLDLiise1F8qDKX00sOU= -google.golang.org/api v0.54.0/go.mod h1:7C4bFFOvVDGXjfDTAsgGwDgAxRDeQ4X8NvUedIt6z3k= -google.golang.org/api v0.55.0/go.mod h1:38yMfeP1kfjsl8isn0tliTjIb1rJXcQi4UXlbqivdVE= -google.golang.org/api v0.56.0/go.mod h1:38yMfeP1kfjsl8isn0tliTjIb1rJXcQi4UXlbqivdVE= -google.golang.org/api v0.57.0/go.mod h1:dVPlbZyBo2/OjBpmvNdpn2GRm6rPy75jyU7bmhdrMgI= -google.golang.org/api v0.61.0/go.mod h1:xQRti5UdCmoCEqFxcz93fTl338AVqDgyaDRuOZ3hg9I= -google.golang.org/api v0.63.0/go.mod h1:gs4ij2ffTRXwuzzgJl/56BdwJaA194ijkfn++9tDuPo= -google.golang.org/api v0.66.0/go.mod h1:I1dmXYpX7HGwz/ejRxwQp2qj5bFAz93HiCU1C1oYd9M= +gomodules.xyz/jsonpatch/v2 v2.4.0 h1:Ci3iUJyx9UeRx7CeFN8ARgGbkESwJK+KB9lLcWxY/Zw= +gomodules.xyz/jsonpatch/v2 v2.4.0/go.mod h1:AH3dM2RI6uoBZxn3LVrfvJ3E0/9dG4cSrbuBJT4moAY= google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM= google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4= -google.golang.org/appengine v1.5.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4= -google.golang.org/appengine v1.6.1/go.mod h1:i06prIuMbXzDqacNJfV5OdTW448YApPu5ww/cMBSeb0= -google.golang.org/appengine v1.6.5/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc= -google.golang.org/appengine v1.6.6/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc= google.golang.org/appengine v1.6.7 h1:FZR1q0exgwxzPzp/aF+VccGrSfxfPpkBqjIIEq3ru6c= google.golang.org/appengine v1.6.7/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc= google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc= -google.golang.org/genproto v0.0.0-20190307195333-5fe7a883aa19/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE= -google.golang.org/genproto v0.0.0-20190418145605-e7d98fc518a7/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE= -google.golang.org/genproto v0.0.0-20190425155659-357c62f0e4bb/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE= -google.golang.org/genproto v0.0.0-20190502173448-54afdca5d873/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE= -google.golang.org/genproto v0.0.0-20190801165951-fa694d86fc64/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc= google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc= -google.golang.org/genproto v0.0.0-20190911173649-1774047e7e51/go.mod h1:IbNlFCBrqXvoKpeg0TB2l7cyZUmoaFKYIwrEpbDKLA8= -google.golang.org/genproto v0.0.0-20191108220845-16a3f7862a1a/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= -google.golang.org/genproto v0.0.0-20191115194625-c23dd37a84c9/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= -google.golang.org/genproto v0.0.0-20191216164720-4f79533eabd1/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= -google.golang.org/genproto v0.0.0-20191230161307-f3c370f40bfb/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= -google.golang.org/genproto v0.0.0-20200115191322-ca5a22157cba/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= -google.golang.org/genproto v0.0.0-20200122232147-0452cf42e150/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= -google.golang.org/genproto v0.0.0-20200204135345-fa8e72b47b90/go.mod h1:GmwEX6Z4W5gMy59cAlVYjN9JhxgbQH6Gn+gFDQe2lzA= -google.golang.org/genproto v0.0.0-20200212174721-66ed5ce911ce/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= -google.golang.org/genproto v0.0.0-20200224152610-e50cd9704f63/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= -google.golang.org/genproto v0.0.0-20200228133532-8c2c7df3a383/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= -google.golang.org/genproto v0.0.0-20200305110556-506484158171/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= -google.golang.org/genproto v0.0.0-20200312145019-da6875a35672/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= -google.golang.org/genproto v0.0.0-20200331122359-1ee6d9798940/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= -google.golang.org/genproto v0.0.0-20200430143042-b979b6f78d84/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= -google.golang.org/genproto v0.0.0-20200511104702-f5ebc3bea380/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= -google.golang.org/genproto v0.0.0-20200513103714-09dca8ec2884/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= -google.golang.org/genproto v0.0.0-20200515170657-fc4c6c6a6587/go.mod h1:YsZOwe1myG/8QRHRsmBRE1LrgQY60beZKjly0O1fX9U= google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo= -google.golang.org/genproto v0.0.0-20200618031413-b414f8b61790/go.mod h1:jDfRM7FcilCzHH/e9qn6dsT145K34l5v+OpcnNgKAAA= -google.golang.org/genproto v0.0.0-20200729003335-053ba62fc06f/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= -google.golang.org/genproto v0.0.0-20200804131852-c06518451d9c/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= -google.golang.org/genproto v0.0.0-20200825200019-8632dd797987/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= -google.golang.org/genproto v0.0.0-20200904004341-0bd0a958aa1d/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= -google.golang.org/genproto v0.0.0-20201019141844-1ed22bb0c154/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= -google.golang.org/genproto v0.0.0-20201109203340-2640f1f9cdfb/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= -google.golang.org/genproto v0.0.0-20201201144952-b05cb90ed32e/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= -google.golang.org/genproto v0.0.0-20201210142538-e3217bee35cc/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= -google.golang.org/genproto v0.0.0-20201214200347-8c77b98c765d/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= -google.golang.org/genproto v0.0.0-20210222152913-aa3ee6e6a81c/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= -google.golang.org/genproto v0.0.0-20210303154014-9728d6b83eeb/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= -google.golang.org/genproto v0.0.0-20210310155132-4ce2db91004e/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= -google.golang.org/genproto v0.0.0-20210319143718-93e7006c17a6/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= -google.golang.org/genproto v0.0.0-20210402141018-6c239bbf2bb1/go.mod h1:9lPAdzaEmUacj36I+k7YKbEc5CXzPIeORRgDAUOu28A= -google.golang.org/genproto v0.0.0-20210513213006-bf773b8c8384/go.mod h1:P3QM42oQyzQSnHPnZ/vqoCdDmzH28fzWByN9asMeM8A= -google.golang.org/genproto v0.0.0-20210602131652-f16073e35f0c/go.mod h1:UODoCrxHCcBojKKwX1terBiRUaqAsFqJiF615XL43r0= -google.golang.org/genproto v0.0.0-20210604141403-392c879c8b08/go.mod h1:UODoCrxHCcBojKKwX1terBiRUaqAsFqJiF615XL43r0= -google.golang.org/genproto v0.0.0-20210608205507-b6d2f5bf0d7d/go.mod h1:UODoCrxHCcBojKKwX1terBiRUaqAsFqJiF615XL43r0= -google.golang.org/genproto v0.0.0-20210624195500-8bfb893ecb84/go.mod h1:SzzZ/N+nwJDaO1kznhnlzqS8ocJICar6hYhVyhi++24= -google.golang.org/genproto v0.0.0-20210713002101-d411969a0d9a/go.mod h1:AxrInvYm1dci+enl5hChSFPOmmUF1+uAa/UsgNRWd7k= -google.golang.org/genproto v0.0.0-20210716133855-ce7ef5c701ea/go.mod h1:AxrInvYm1dci+enl5hChSFPOmmUF1+uAa/UsgNRWd7k= -google.golang.org/genproto v0.0.0-20210728212813-7823e685a01f/go.mod h1:ob2IJxKrgPT52GcgX759i1sleT07tiKowYBGbczaW48= -google.golang.org/genproto v0.0.0-20210805201207-89edb61ffb67/go.mod h1:ob2IJxKrgPT52GcgX759i1sleT07tiKowYBGbczaW48= -google.golang.org/genproto v0.0.0-20210813162853-db860fec028c/go.mod h1:cFeNkxwySK631ADgubI+/XFU/xp8FD5KIVV4rj8UC5w= -google.golang.org/genproto v0.0.0-20210821163610-241b8fcbd6c8/go.mod h1:eFjDcFEctNawg4eG61bRv87N7iHBWyVhJu7u1kqDUXY= -google.golang.org/genproto v0.0.0-20210828152312-66f60bf46e71/go.mod h1:eFjDcFEctNawg4eG61bRv87N7iHBWyVhJu7u1kqDUXY= -google.golang.org/genproto v0.0.0-20210831024726-fe130286e0e2/go.mod h1:eFjDcFEctNawg4eG61bRv87N7iHBWyVhJu7u1kqDUXY= -google.golang.org/genproto v0.0.0-20210903162649-d08c68adba83/go.mod h1:eFjDcFEctNawg4eG61bRv87N7iHBWyVhJu7u1kqDUXY= -google.golang.org/genproto v0.0.0-20210909211513-a8c4777a87af/go.mod h1:eFjDcFEctNawg4eG61bRv87N7iHBWyVhJu7u1kqDUXY= -google.golang.org/genproto v0.0.0-20210924002016-3dee208752a0/go.mod h1:5CzLGKJ67TSI2B9POpiiyGha0AjJvZIUgRMt1dSmuhc= -google.golang.org/genproto v0.0.0-20211118181313-81c1377c94b1/go.mod h1:5CzLGKJ67TSI2B9POpiiyGha0AjJvZIUgRMt1dSmuhc= -google.golang.org/genproto v0.0.0-20211206160659-862468c7d6e0/go.mod h1:5CzLGKJ67TSI2B9POpiiyGha0AjJvZIUgRMt1dSmuhc= -google.golang.org/genproto v0.0.0-20211208223120-3a66f561d7aa/go.mod h1:5CzLGKJ67TSI2B9POpiiyGha0AjJvZIUgRMt1dSmuhc= -google.golang.org/genproto v0.0.0-20211221195035-429b39de9b1c/go.mod h1:5CzLGKJ67TSI2B9POpiiyGha0AjJvZIUgRMt1dSmuhc= -google.golang.org/genproto v0.0.0-20220114231437-d2e6a121cae0/go.mod h1:5CzLGKJ67TSI2B9POpiiyGha0AjJvZIUgRMt1dSmuhc= -google.golang.org/genproto v0.0.0-20220201184016-50beb8ab5c44/go.mod h1:5CzLGKJ67TSI2B9POpiiyGha0AjJvZIUgRMt1dSmuhc= google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c= -google.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38= -google.golang.org/grpc v1.21.1/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM= google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg= -google.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY= -google.golang.org/grpc v1.26.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk= google.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk= -google.golang.org/grpc v1.27.1/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk= -google.golang.org/grpc v1.28.0/go.mod h1:rpkK4SK4GF4Ach/+MFLZUBavHOvF2JJB5uozKKal+60= -google.golang.org/grpc v1.29.1/go.mod h1:itym6AZVZYACWQqET3MqgPpjcuV5QH3BxFS3IjizoKk= -google.golang.org/grpc v1.30.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak= -google.golang.org/grpc v1.31.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak= -google.golang.org/grpc v1.31.1/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak= -google.golang.org/grpc v1.33.1/go.mod h1:fr5YgcSWrqhRRxogOsw7RzIpsmvOZ6IcH4kBYTpR3n0= -google.golang.org/grpc v1.33.2/go.mod h1:JMHMWHQWaTccqQQlmk3MJZS+GWXOdAesneDmEnv2fbc= -google.golang.org/grpc v1.34.0/go.mod h1:WotjhfgOW/POjDeRt8vscBtXq+2VjORFy659qA51WJ8= -google.golang.org/grpc v1.35.0/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU= -google.golang.org/grpc v1.36.0/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU= -google.golang.org/grpc v1.36.1/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU= -google.golang.org/grpc v1.37.0/go.mod h1:NREThFqKR1f3iQ6oBuvc5LadQuXVGo9rkm5ZGrQdJfM= -google.golang.org/grpc v1.37.1/go.mod h1:NREThFqKR1f3iQ6oBuvc5LadQuXVGo9rkm5ZGrQdJfM= -google.golang.org/grpc v1.38.0/go.mod h1:NREThFqKR1f3iQ6oBuvc5LadQuXVGo9rkm5ZGrQdJfM= -google.golang.org/grpc v1.39.0/go.mod h1:PImNr+rS9TWYb2O4/emRugxiyHZ5JyHW5F+RPnDzfrE= -google.golang.org/grpc v1.39.1/go.mod h1:PImNr+rS9TWYb2O4/emRugxiyHZ5JyHW5F+RPnDzfrE= -google.golang.org/grpc v1.40.0/go.mod h1:ogyxbiOoUXAkP+4+xa6PZSE9DZgIHtSpzjDTB9KAK34= -google.golang.org/grpc v1.40.1/go.mod h1:ogyxbiOoUXAkP+4+xa6PZSE9DZgIHtSpzjDTB9KAK34= -google.golang.org/grpc/cmd/protoc-gen-go-grpc v1.1.0/go.mod h1:6Kw0yEErY5E/yWrBtf03jp27GLLJujG4z/JK95pnjjw= google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8= google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0= google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM= google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE= google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo= google.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= -google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= -google.golang.org/protobuf v1.24.0/go.mod h1:r/3tXBNzIEhYS9I1OUVjXDlt8tc493IdKGjtUeSXeh4= google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c= google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw= google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc= -google.golang.org/protobuf v1.27.1/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc= -google.golang.org/protobuf v1.28.0 h1:w43yiav+6bVFTBQFZX0r7ipe9JQ1QsbMgHwbBziscLw= -google.golang.org/protobuf v1.28.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I= -gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw= +google.golang.org/protobuf v1.31.0 h1:g0LDEJHgrBl9N9r17Ru3sqWhkIx2NB67okBHPwC7hs8= +google.golang.org/protobuf v1.31.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= -gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= -gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f h1:BLraFXnmrev5lT+xlilqcH8XK9/i0At2xKjWk4p6zsU= -gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= -gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= gopkg.in/inf.v0 v0.9.1 h1:73M5CoZyi3ZLMOyDlQh031Cx6N9NDJ2Vvfl76EDAgDc= gopkg.in/inf.v0 v0.9.1/go.mod h1:cWUDdTG/fYaXco+Dcufb5Vnc6Gp2YChqWtbxRZE0mXw= -gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 h1:uRGJdciOHaEIrze2W8Q3AKkepLTh2hOroT7a+7czfdQ= -gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= -gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= -gopkg.in/yaml.v2 v2.2.3/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= -gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= -gopkg.in/yaml.v2 v2.2.5/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= -gopkg.in/yaml.v2 v2.3.0/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY= gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ= gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= -gopkg.in/yaml.v3 v3.0.0-20200615113413-eeeca48fe776/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= -gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= -honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= -honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= -honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg= -honnef.co/go/tools v0.0.1-2020.1.3/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k= -honnef.co/go/tools v0.0.1-2020.1.4/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k= -k8s.io/api v0.25.4 h1:3YO8J4RtmG7elEgaWMb4HgmpS2CfY1QlaOz9nwB+ZSs= -k8s.io/api v0.25.4/go.mod h1:IG2+RzyPQLllQxnhzD8KQNEu4c4YvyDTpSMztf4A0OQ= -k8s.io/apiextensions-apiserver v0.25.0 h1:CJ9zlyXAbq0FIW8CD7HHyozCMBpDSiH7EdrSTCZcZFY= -k8s.io/apiextensions-apiserver v0.25.0/go.mod h1:3pAjZiN4zw7R8aZC5gR0y3/vCkGlAjCazcg1me8iB/E= -k8s.io/apimachinery v0.25.4 h1:CtXsuaitMESSu339tfhVXhQrPET+EiWnIY1rcurKnAc= -k8s.io/apimachinery v0.25.4/go.mod h1:jaF9C/iPNM1FuLl7Zuy5b9v+n35HGSh6AQ4HYRkCqwo= -k8s.io/client-go v0.25.4 h1:3RNRDffAkNU56M/a7gUfXaEzdhZlYhoW8dgViGy5fn8= -k8s.io/client-go v0.25.4/go.mod h1:8trHCAC83XKY0wsBIpbirZU4NTUpbuhc2JnI7OruGZw= -k8s.io/component-base v0.25.0 h1:haVKlLkPCFZhkcqB6WCvpVxftrg6+FK5x1ZuaIDaQ5Y= -k8s.io/component-base v0.25.0/go.mod h1:F2Sumv9CnbBlqrpdf7rKZTmmd2meJq0HizeyY/yAFxk= -k8s.io/klog/v2 v2.0.0/go.mod h1:PBfzABfn139FHAV07az/IF9Wp1bkk3vpT2XSJ76fSDE= -k8s.io/klog/v2 v2.70.1 h1:7aaoSdahviPmR+XkS7FyxlkkXs6tHISSG03RxleQAVQ= -k8s.io/klog/v2 v2.70.1/go.mod h1:y1WjHnz7Dj687irZUWR/WLkLc5N1YHtjLdmgWjndZn0= -k8s.io/kube-openapi v0.0.0-20220803162953-67bda5d908f1 h1:MQ8BAZPZlWk3S9K4a9NCkIFQtZShWqoha7snGixVgEA= -k8s.io/kube-openapi v0.0.0-20220803162953-67bda5d908f1/go.mod h1:C/N6wCaBHeBHkHUesQOQy2/MZqGgMAFPqGsGQLdbZBU= -k8s.io/utils v0.0.0-20220728103510-ee6ede2d64ed h1:jAne/RjBTyawwAy0utX5eqigAwz/lQhTmy+Hr/Cpue4= -k8s.io/utils v0.0.0-20220728103510-ee6ede2d64ed/go.mod h1:jPW/WVKK9YHAvNhRxK0md/EJ228hCsBRufyofKtW8HA= -rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8= -rsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0= -rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA= -sigs.k8s.io/controller-runtime v0.13.1 h1:tUsRCSJVM1QQOOeViGeX3GMT3dQF1eePPw6sEE3xSlg= -sigs.k8s.io/controller-runtime v0.13.1/go.mod h1:Zbz+el8Yg31jubvAEyglRZGdLAjplZl+PgtYNI6WNTI= -sigs.k8s.io/json v0.0.0-20220713155537-f223a00ba0e2 h1:iXTIw73aPyC+oRdyqqvVJuloN1p0AC/kzH07hu3NE+k= -sigs.k8s.io/json v0.0.0-20220713155537-f223a00ba0e2/go.mod h1:B8JuhiUyNFVKdsE8h686QcCxMaH6HrOAZj4vswFpcB0= -sigs.k8s.io/structured-merge-diff/v4 v4.2.3 h1:PRbqxJClWWYMNV1dhaG4NsibJbArud9kFxnAMREiWFE= -sigs.k8s.io/structured-merge-diff/v4 v4.2.3/go.mod h1:qjx8mGObPmV2aSZepjQjbmb2ihdVs8cGKBraizNC69E= +k8s.io/api v0.29.2 h1:hBC7B9+MU+ptchxEqTNW2DkUosJpp1P+Wn6YncZ474A= +k8s.io/api v0.29.2/go.mod h1:sdIaaKuU7P44aoyyLlikSLayT6Vb7bvJNCX105xZXY0= +k8s.io/apiextensions-apiserver v0.28.0 h1:CszgmBL8CizEnj4sj7/PtLGey6Na3YgWyGCPONv7E9E= +k8s.io/apiextensions-apiserver v0.28.0/go.mod h1:uRdYiwIuu0SyqJKriKmqEN2jThIJPhVmOWETm8ud1VE= +k8s.io/apimachinery v0.29.2 h1:EWGpfJ856oj11C52NRCHuU7rFDwxev48z+6DSlGNsV8= +k8s.io/apimachinery v0.29.2/go.mod h1:6HVkd1FwxIagpYrHSwJlQqZI3G9LfYWRPAkUvLnXTKU= +k8s.io/cli-runtime v0.29.2 h1:smfsOcT4QujeghsNjECKN3lwyX9AwcFU0nvJ7sFN3ro= +k8s.io/cli-runtime v0.29.2/go.mod h1:KLisYYfoqeNfO+MkTWvpqIyb1wpJmmFJhioA0xd4MW8= +k8s.io/client-go v0.29.2 h1:FEg85el1TeZp+/vYJM7hkDlSTFZ+c5nnK44DJ4FyoRg= +k8s.io/client-go v0.29.2/go.mod h1:knlvFZE58VpqbQpJNbCbctTVXcd35mMyAAwBdpt4jrA= +k8s.io/component-base v0.29.2 h1:lpiLyuvPA9yV1aQwGLENYyK7n/8t6l3nn3zAtFTJYe8= +k8s.io/component-base v0.29.2/go.mod h1:BfB3SLrefbZXiBfbM+2H1dlat21Uewg/5qtKOl8degM= +k8s.io/klog/v2 v2.110.1 h1:U/Af64HJf7FcwMcXyKm2RPM22WZzyR7OSpYj5tg3cL0= +k8s.io/klog/v2 v2.110.1/go.mod h1:YGtd1984u+GgbuZ7e08/yBuAfKLSO0+uR1Fhi6ExXjo= +k8s.io/kube-openapi v0.0.0-20231010175941-2dd684a91f00 h1:aVUu9fTY98ivBPKR9Y5w/AuzbMm96cd3YHRTU83I780= +k8s.io/kube-openapi v0.0.0-20231010175941-2dd684a91f00/go.mod h1:AsvuZPBlUDVuCdzJ87iajxtXuR9oktsTctW/R9wwouA= +k8s.io/kubectl v0.29.2 h1:uaDYaBhumvkwz0S2XHt36fK0v5IdNgL7HyUniwb2IUo= +k8s.io/kubectl v0.29.2/go.mod h1:BhizuYBGcKaHWyq+G7txGw2fXg576QbPrrnQdQDZgqI= +k8s.io/utils v0.0.0-20230726121419-3b25d923346b h1:sgn3ZU783SCgtaSJjpcVVlRqd6GSnlTLKgpAAttJvpI= +k8s.io/utils v0.0.0-20230726121419-3b25d923346b/go.mod h1:OLgZIPagt7ERELqWJFomSt595RzquPNLL48iOWgYOg0= +sigs.k8s.io/controller-runtime v0.16.2 h1:mwXAVuEk3EQf478PQwQ48zGOXvW27UJc8NHktQVuIPU= +sigs.k8s.io/controller-runtime v0.16.2/go.mod h1:vpMu3LpI5sYWtujJOa2uPK61nB5rbwlN7BAB8aSLvGU= +sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd h1:EDPBXCAspyGV4jQlpZSudPeMmr1bNJefnuqLsRAsHZo= +sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd/go.mod h1:B8JuhiUyNFVKdsE8h686QcCxMaH6HrOAZj4vswFpcB0= +sigs.k8s.io/kustomize/api v0.13.5-0.20230601165947-6ce0bf390ce3 h1:XX3Ajgzov2RKUdc5jW3t5jwY7Bo7dcRm+tFxT+NfgY0= +sigs.k8s.io/kustomize/api v0.13.5-0.20230601165947-6ce0bf390ce3/go.mod h1:9n16EZKMhXBNSiUC5kSdFQJkdH3zbxS/JoO619G1VAY= +sigs.k8s.io/kustomize/kyaml v0.14.3-0.20230601165947-6ce0bf390ce3 h1:W6cLQc5pnqM7vh3b7HvGNfXrJ/xL6BDMS0v1V/HHg5U= +sigs.k8s.io/kustomize/kyaml v0.14.3-0.20230601165947-6ce0bf390ce3/go.mod h1:JWP1Fj0VWGHyw3YUPjXSQnRnrwezrZSrApfX5S0nIag= +sigs.k8s.io/structured-merge-diff/v4 v4.4.1 h1:150L+0vs/8DA78h1u02ooW1/fFq/Lwr+sGiqlzvrtq4= +sigs.k8s.io/structured-merge-diff/v4 v4.4.1/go.mod h1:N8hJocpFajUSSeSJ9bOZ77VzejKZaXsTtZo4/u7Io08= sigs.k8s.io/yaml v1.3.0 h1:a2VclLzOGrwOHDiV8EfBGhvjHvP46CtW5j6POvhYGGo= sigs.k8s.io/yaml v1.3.0/go.mod h1:GeOyir5tyXNByN85N/dRIT9es5UQNerPYEKK56eTBm8= diff --git a/image.png b/image.png new file mode 100644 index 0000000000000000000000000000000000000000..8bd1bbd5b33a7353cdc97863c2096670ce5f2f52 GIT binary patch literal 6913 zcmb_gS5?5nroaXO0C-A@vKjyYlGQ)Xgo*wy-eB>J0|2P_U>O;8 zI~fHTCwnJXO=nYcO9e|uOINVDhJrKz0E&;-HnO48CKk_Xsb_yX9yVK2l7#=vpVDBj zF%`bF`)pb7u)>jS-yrpE=x{X$%*40(SOjJjCzc|F{z4=pUJRhjbhhP3v7Chs>V>Fb zEVq;Q3I)?-SO+b9^=-)zyN;2SyT92dfjet%$7?v>ARGp1YmCSo6jhwtGaK~-e|vj{ z-Jt=Kcmb1~>}4iM<=xXYxCF5H!s!)Rq6^z(W?;)6`*1I4NJf?Nnf2& zFsjtUAlAIuEmp6HeKzd|5v&oQ3(O@rt((aF&Kqqk$HDSD?zwk48e@myS|1g|?Q}Uo zl>b*HEB%PvkU;qlRfd_4u!o=~xmCN~%RzoYr9B`AMN-s~9Pc zC#WeoXy1jjAD^rvT@x9xji1Q?mcyLGBG!jgvf9x;uzoQG@eNh z2I)337ZDmaxd$a~tPFl!P98?e>&m0pR!^o$J~9;VCCP{%l7xo)9FlOGyq5h&O91va zF!@=EO|ffJuv=_XaQg>w2=8INdj>RX***^*>`rk@m%K2XPX6vWvC(E|#|`wcDfgJf zvT|mayi(<07dN>~}6e;|$C&EleEYSssdK_gEvwmo*& zORE3b+9CtdFwv)+CrG$3u`ppboI3CLML57wH?~b)C$DYJZO{F!HH9sw`{2@Sj_brr zkMYiYjq#44KJGI{AWG1|vQ9Jq!TV7i?0{A$gITb_dLtk21N zyW{Ggwb*~b>7Q2er1cj#MOiE&#Cl*o7Ak7HB$44*rtf?`1Z(ypl+zqs8F)J~t^pSOJHEU;JoyG(pv=Mnko*^v7e;TKKJ&Se ze-8p=si$Q1@gsokAIAhBMS%gR{}|FgQ2YY`fSix~KNTdae3bv;0H*&mNw}um0RU24 zC0QvgZ=@qbEN`m5)T-aP)I+4aCieCmpWSRMxcdPHm*>_Tlv~Y z460uR{H&_-z52Z$RQ*M}<2R|funp>aWD2DCV0lS~{8y8h|K~L19U0Q|wvH~O$=Sgn zY+bOQWPJ69t!0H)Me+7#C2L#+%$#5=MvW*mQ75Llp7`Z0b>cg;cnW@7tv@xthZ$Bb zSInl9>RQTLe1xVT7cAmcO-=H5w_1$elB^6E!5T+ z6$zz~4!L4+)qbCAh}e?rfgTi@xd~YZZuCK8rL~JQ%L&kT-ZK&n?N z5Mg=VrlxG(7nY@%UWA7kb(5k)4JYBs{rUrnfSZ({4UEltG%5+Uoq5g$FYD?FR+ zxtByJ5LTL|?U>GXo~Ld+&B%P{OcKX4J`e z9kas((9kv8I+Yzh30+qZxh8LLJZtakwglYq_CxczM_0?Ub5m7i0OlhmJ1|J^Mb-}*_o?S0k{5l2h*7GN3?MI1N?n6$%&!b;V!xfR$`sdkGlg;=Pzg)t!AFlU zz=-5yOE&C?Hx|IF91nn{B&-*C)L zWme1sq2Q-}MhHa|_S)6BOqHR*z-ujsGm0R~0&@0ZTq73Nxh7Z(@9_8uXUCXvK(+-n%O zNv@pD8Mox)qF=mhu-8@Uo-0~f#tK|=@f}$2goTrv7Cva!nDdBR6+AZ=*7kbz;#h-T z$791q8*GT~>)G$Gx)!v>7l{+$3HDg?a3dBslO{et`S!{qKDJHN-BJ3j_))!cU>Dt& zV85Pkq<*jE@1>*A+r~5VTZm2`LQjNuFJevFUR{CSimAN=;?aovFPs{cJ8hEI{Iz|H z9c1C+;YN^5|6RjZL9U|#3hm5xkp$h8mJV6)ekocEsN)tG!NFVacKLpfc>Vo!VI=hQ zAXV4*p)6u6iiqs?M{YNYx$|;p;x_&vTiLU!8tc~6bsJH$04fU3E@2IO*CpH;ay$?k zd#8Rc(_*4xdI%rru7uI{BSnk3#YOfUCtiwSJ7>j7H6$jFbAJBVkfz3g{4{pU3X~BR zUDRZ8QG07B*c_!(CT5yHeumQ&uHx}lcOMTJ#{bJN?R~(>pFe+02|dwpbFI;KF1U3j z(XT2@wn#VN0ugtCZ;3z)1tJbZnE=nJk;k!bMmOLhM3OKy38f zJP{LZ=<#827NL6jH@XyfX$|rWYbybOP61?2QA9zhZ4*mErj*@;sB9!tn=M za}L4`F@XaV{+V!ayNovyB?Ra00(j}cUDGVz6Hf<~o6Y@^1GOpg{=W68fNwx* z!%m}YmI-mjoTYjiEfbNm14i88?u%mjUXbk$G2iB>LWJIrzpgtC{qA(j1xwqIe1T7X z8QqK9ujns=5V~@1t8$ciQ7;GI<+r=2E33W1Z5JQm9W7Ab<;FC9h37vW_Y&glbfG^b zIc4fGYq>(~v_zc?9JKaKCCi)UBee@8D_3J-f}^5|7~BQ8upB4jK@nRTSD5`re^4G5 zk}0!@n9U;iD|vEPiW!oHYPA-h&rPYWQw?(n8kx(2eI5nv!Q!*;16#NRE~fG^hcwsYm9ev)cR$Oc83 zq&-Bmy+H`!qc{|4xaue{2_g%cQcz|kVW@Bg$+?p@R~6sraok~e2{GeuGuso$xWSJ` z_53`V?Uw#tC#(xuwr@@P_}t?K^2Q;k^`Ot={q)olXvgJZjDns<B zVVQ-X^F+qA?Q6<_ev9AwPclt$^x~xTtxO!dDASb7B7AK!RZ)N_#~eb5d|#W?NP+3& z_|HOjoDFV7PZ>!HP)RH*#TVvnAvzkkbh#y&gVLRFgN3Z=AobYr=xvGK22l|XoxfK; z9?4|(qMlUH>m%Vtv8*hB(JkQ-s7zeX?}Oi*FYtsMa^XvEwfo~Kfl?W#TjlJ-Ft8C$n&&_$bOGB-;Ar+=qPnWQzSuOC$y>8sEV5jMFdE>Bfdt^RP z>T%;N&64|ij5z(nmtgkfBjQpeDoCRwXmku#^ywdS324%U+xyb!g8UA4D>^eeY#!20+k6NpQ*Vy(rmc};oTyiKFn4( z+|&@f&Z$cFk7rxeQ)HAuQP$7(DQi9sVOiC>LJoJh5Gizgdqv>;cWBYO!uF(L&!N7mM$f{jNB!SUSKiN#F@M) zXA#X|LfR9+>eY`b&a`KP#fS^+Z!Y$jXk^lV`2tpMJOe=Af}!53=Z9mpdIoHX?ehms zmu>ZYHj)=67iys#{xU?8<-t_Uu6g?sM@85Mj5^l#5z7dY}A6*+v*BcZyvp zUCp+790=UB8+;R?NSstSR<@8p#6)({w%R*YOy}!?d_cg;mbq3VX2>d*v=8%=Wcamw%R4?fVvHjCNqy zQ?B2nxD{Bj*L?KD)U~cU{o)(UA{6_InJV1lAzJKI+m_YgWY=1eG}JCXsi{csJJ(V5 z`$}S$Nl&!|EPLxzw3lOWK0o%9xynf43-JLjF!-*^*(P0m9B?#20`ny;Lp@%QL{riJ-6~V1YF^|rYrZm z&}&52J9zFK>u5xW3}Z)6*NX1(2hXD#t@M?jU;j#&8w$#_sLH*{liI<>OdDQPn+S!T zEdk_8WQFzZd}?K2VQ z!zCCI^4$juj#fP9U6fU2EnIU;t#w*5HDixGi)*QcV-o{keR+wn1LHmker z_~H$Z7sDayV}VHz)FDVh^hB8cC@8PP0CV4AG>6#y{QPaE+z%=VkH@?q9IRI~hT@8l zh0Cw<@`=<~o!Y8wr2OTs z;gf2Gjn-UTuc8khT8&VvO_=$Lrr&d+H3?$Fytwo_5ms4bG#PHI01uds~Wz7z^&R=E`0k)?H4V(sumw zWsWx%dzML$g4LOuK5q7wl-z0Sed>K1y)z)@m6=~{-1+)NJ%Jp~wC_N#b9!kGZqs^K{T+K_S2BiuXKo3b-oclPD*w`y*b2oteEnU4U%um-{J#tiWWIp=Nk!U{H#GZw7aX!oK{;Q6Vf+ zb&*SH#k2-je}ZPk&)^l6c5hrylYosLt`X;Eo__f>N6o%;aJZ-xLA z8hswL$Q|Pk>p=v~%%n0>;xG>+LnNz*ZW^Bq)0joxtgFM#UhSVEE-cZR#vQoZv|dVR z{+Utc-AA#1GS%6<7|A1~ZQ^mz@?h)g6az_4g&#yxY>Cna-10ZUj&;Q!*)_**QAM%E zUo~8D;-<-|KW}n>BPa^dv{^7S5MZ47(pVw>{_>U9Gurh}iO>lrL6tJJ)5>e!!i6=7 zc$qPbx8@_xW-=Hb-?P-MqzG#IrA`D1R^b?ilRVf{S>VHp&O~*F{SdM^#v@3aU?V)< zSesb1?CF_tB)P4Y)2ripMuDog_tz>2D7My*i&3PyIvGJwdjaY{F;}67b{|a1p7(_8 zH)|>csRw~?&bo#)|n<#&wm=i!+U?iH{^ zNbj+w{Pw}*iHOULmDsf-A6?d7_kre$ulFRYnwNvr1;tuo_^5yrctUi2i^O-@)X@WS z>}e9zG4%8f>itY4>IIDxrA*#(JiZ%bm%lX@?2=l}7unEi5W{&BfMX73TVX6k&|Y0g zoA-gU$T}!7g-9qcoMVEH<037w11p43Sd$gE zJ2~db=ge_`OqgkUWD?2NrWM|B=_g@Kb01#u`C~T4Ad&}(lurI1Ha@){PZD6XUGenk z-RIegjO#8d(SMetfrWlLRrU$5I31ONjIL zFtr`p|`;g;>-Yo6Kouf59|04|uM5Qe`M%!?cYOauGS zOtF7pM_@V)A-Gf6Ot`_lWQ=NDc#zglWQdoZ!(qxWOgcSv3je^R&_-uW3tfmJ+~rKw z{(2DoDk5&!Fy>{u=9Mc5ssEeKB^iNY*HM;9+|`cVrHAJdh@VNLs~5lMbzOVG$nM!* z(><$P>2N(t68svh7{uQtx6nEB$#T)91AmJco;I3SS z;>dF&vi1iOa)v)_eu_)w<0;vL6>1|qsH;fURsBP1r^n#2Yd&CN=CNOQE7qRx7TaA zfe}QUtO~69IbEA{`P*%qL-zUl_9u4eF`0jV{IbW0V@Ym;e8It^X_AHlz~sQG&&GCdxy$IO9L= Ovyz;uY?ZW0=zjql#Zw>v literal 0 HcmV?d00001 diff --git a/main.go b/main.go index 6dbf2305..acdd2719 100644 --- a/main.go +++ b/main.go @@ -41,21 +41,31 @@ package main import ( "context" "flag" + "fmt" "os" "strconv" + "strings" "time" + monitorv1 "github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring/v1" + "go.uber.org/zap/zapcore" "k8s.io/apimachinery/pkg/runtime" utilruntime "k8s.io/apimachinery/pkg/util/runtime" clientgoscheme "k8s.io/client-go/kubernetes/scheme" _ "k8s.io/client-go/plugin/pkg/client/auth/gcp" + "k8s.io/client-go/rest" ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/cache" "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/controller-runtime/pkg/log/zap" + metricsserver "sigs.k8s.io/controller-runtime/pkg/metrics/server" databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" databasecontroller "github.com/oracle/oracle-database-operator/controllers/database" + + observabilityv1alpha1 "github.com/oracle/oracle-database-operator/apis/observability/v1alpha1" + observabilitycontroller "github.com/oracle/oracle-database-operator/controllers/observability" // +kubebuilder:scaffold:imports ) @@ -66,7 +76,8 @@ var ( func init() { utilruntime.Must(clientgoscheme.AddToScheme(scheme)) - + utilruntime.Must(observabilityv1alpha1.AddToScheme(scheme)) + utilruntime.Must(monitorv1.AddToScheme(scheme)) utilruntime.Must(databasev1alpha1.AddToScheme(scheme)) // +kubebuilder:scaffold:scheme } @@ -88,13 +99,25 @@ func main() { ctrl.SetLogger(zap.New(func(o *zap.Options) { *o = *options })) - mgr, err := ctrl.NewManager(ctrl.GetConfigOrDie(), ctrl.Options{ - Scheme: scheme, - MetricsBindAddress: metricsAddr, - Port: 9443, - LeaderElection: enableLeaderElection, - LeaderElectionID: "a9d608ea.oracle.com", - }) + watchNamespaces, err := getWatchNamespace() + if err != nil { + setupLog.Error(err, "Failed to get watch namespaces") + os.Exit(1) + } + opt := ctrl.Options{ + Scheme: scheme, + Metrics: metricsserver.Options{ + BindAddress: metricsAddr, + }, + LeaderElection: enableLeaderElection, + LeaderElectionID: "a9d608ea.oracle.com", + NewCache: func(config *rest.Config, opts cache.Options) (cache.Cache, error) { + opts.DefaultNamespaces = watchNamespaces + return cache.New(config, opts) + }, + } + + mgr, err := ctrl.NewManager(ctrl.GetConfigOrDie(), opt) if err != nil { setupLog.Error(err, "unable to start manager") os.Exit(1) @@ -226,6 +249,10 @@ func main() { setupLog.Error(err, "unable to create webhook", "webhook", "DataguardBroker") os.Exit(1) } + if err = (&observabilityv1alpha1.DatabaseObserver{}).SetupWebhookWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create webhook", "webhook", "DatabaseObserver") + os.Exit(1) + } } // PDB Reconciler @@ -263,6 +290,17 @@ func main() { os.Exit(1) } + // Observability DatabaseObserver Reconciler + if err = (&observabilitycontroller.DatabaseObserverReconciler{ + Client: mgr.GetClient(), + Log: ctrl.Log.WithName("controllers").WithName("observability").WithName("DatabaseObserver"), + Scheme: mgr.GetScheme(), + Recorder: mgr.GetEventRecorderFor("DatabaseObserver"), + }).SetupWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create controller", "controller", "DatabaseObserver") + os.Exit(1) + } + // +kubebuilder:scaffold:builder // Add index for PDB CR to enable mgr to cache PDBs @@ -280,3 +318,35 @@ func main() { os.Exit(1) } } + +func getWatchNamespace() (map[string]cache.Config, error) { + // WatchNamespaceEnvVar is the constant for env variable WATCH_NAMESPACE + // which specifies the Namespace to watch. + // An empty value means the operator is running with cluster scope. + + var watchNamespaceEnvVar = "WATCH_NAMESPACE" + var nsmap map[string]cache.Config + ns, found := os.LookupEnv(watchNamespaceEnvVar) + values := strings.Split(ns, ",") + if len(values) == 1 && values[0] == "" { + fmt.Printf(":CLUSTER SCOPED:\n") + return nil, nil + } + fmt.Printf(":NAMESPACE SCOPED:\n") + fmt.Printf("WATCH LIST=%s\n", values) + nsmap = make(map[string]cache.Config, len(values)) + if !found { + return nsmap, fmt.Errorf("%s must be set", watchNamespaceEnvVar) + } + + if ns == "" { + return nil, nil + } + + for _, ns := range values { + nsmap[ns] = cache.Config{} + } + + return nsmap, nil + +} diff --git a/oracle-database-operator.yaml b/oracle-database-operator.yaml index 70f026c1..c2714b6e 100644 --- a/oracle-database-operator.yaml +++ b/oracle-database-operator.yaml @@ -572,11 +572,59 @@ spec: - connectionStrings type: object type: array + conditions: + items: + description: "Condition contains details for one aspect of the current state of this API Resource. --- This struct is intended for direct use as an array at the field path .status.conditions. For example, \n \ttype FooStatus struct{ \t // Represents the observations of a foo's current state. \t // Known .status.conditions.type are: \"Available\", \"Progressing\", and \"Degraded\" \t // +patchMergeKey=type \t // +patchStrategy=merge \t // +listType=map \t // +listMapKey=type \t Conditions []metav1.Condition `json:\"conditions,omitempty\" patchStrategy:\"merge\" patchMergeKey:\"type\" protobuf:\"bytes,1,rep,name=conditions\"` \n \t // other fields \t}" + properties: + lastTransitionTime: + description: lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: message is a human readable message indicating details about the transition. This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. --- Many .condition.type values are consistent across resources like Available, but because arbitrary conditions can be useful (see .node.status.conditions), the ability to deconflict is important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map lifecycleState: description: 'INSERT ADDITIONAL STATUS FIELD - define observed state of cluster Important: Run "make" to regenerate code after modifying this file' type: string timeCreated: type: string + walletExpiringDate: + type: string type: object type: object served: true @@ -853,6 +901,195 @@ status: --- apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.6.1 + creationTimestamp: null + name: databaseobservers.observability.oracle.com +spec: + group: observability.oracle.com + names: + kind: DatabaseObserver + listKind: DatabaseObserverList + plural: databaseobservers + singular: databaseobserver + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .status.exporterConfig + name: ExporterConfig + type: string + - jsonPath: .status.status + name: Status + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + description: DatabaseObserver is the Schema for the databaseobservers API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: DatabaseObserverSpec defines the desired state of DatabaseObserver + properties: + database: + description: DatabaseObserverDatabase defines the database details used for DatabaseObserver + properties: + dbConnectionString: + properties: + key: + type: string + secret: + type: string + type: object + dbPassword: + properties: + key: + type: string + secret: + type: string + vaultOCID: + type: string + vaultSecretName: + type: string + type: object + dbUser: + properties: + key: + type: string + secret: + type: string + type: object + dbWallet: + properties: + key: + type: string + secret: + type: string + type: object + type: object + exporter: + description: DatabaseObserverExporterConfig defines the configuration details related to the exporters of DatabaseObserver + properties: + configuration: + properties: + configmap: + description: ConfigMapDetails defines the configmap name + properties: + configmapName: + type: string + key: + type: string + type: object + type: object + image: + type: string + service: + description: DatabaseObserverService defines the exporter service component of DatabaseObserver + properties: + port: + format: int32 + type: integer + type: object + type: object + ociConfig: + properties: + configMapName: + type: string + secretName: + type: string + type: object + prometheus: + description: PrometheusConfig defines the generated resources for Prometheus + properties: + labels: + additionalProperties: + type: string + type: object + port: + type: string + type: object + replicas: + format: int32 + type: integer + type: object + status: + description: DatabaseObserverStatus defines the observed state of DatabaseObserver + properties: + conditions: + description: 'INSERT ADDITIONAL STATUS FIELD - define observed state of cluster Important: Run "make" to regenerate code after modifying this file' + items: + description: "Condition contains details for one aspect of the current state of this API Resource. --- This struct is intended for direct use as an array at the field path .status.conditions. For example, \n \ttype FooStatus struct{ \t // Represents the observations of a foo's current state. \t // Known .status.conditions.type are: \"Available\", \"Progressing\", and \"Degraded\" \t // +patchMergeKey=type \t // +patchStrategy=merge \t // +listType=map \t // +listMapKey=type \t Conditions []metav1.Condition `json:\"conditions,omitempty\" patchStrategy:\"merge\" patchMergeKey:\"type\" protobuf:\"bytes,1,rep,name=conditions\"` \n \t // other fields \t}" + properties: + lastTransitionTime: + description: lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: message is a human readable message indicating details about the transition. This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. --- Many .condition.type values are consistent across resources like Available, but because arbitrary conditions can be useful (see .node.status.conditions), the ability to deconflict is important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + exporterConfig: + type: string + replicas: + type: integer + status: + type: string + required: + - conditions + - exporterConfig + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition metadata: annotations: controller-gen.kubebuilder.io/version: v0.6.1 @@ -1536,6 +1773,9 @@ spec: cdbName: description: Name of the CDB type: string + cdbNamespace: + description: CDB Namespace + type: string cdbResName: description: Name of the CDB Custom Resource that runs the ORDS container type: string @@ -1688,6 +1928,40 @@ spec: unlimitedStorage: description: Relevant for Create and Plug operations. True for unlimited storage. Even when set to true, totalSize and tempSize MUST be specified in the request if Action is Create. type: boolean + webServerPwd: + description: Password for the Web ServerPDB User + properties: + secret: + description: PDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + webServerUser: + description: Web Server User with SQL Administrator role to allow us to authenticate to the PDB Lifecycle Management REST endpoints + properties: + secret: + description: PDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object xmlFileName: description: XML metadata filename to be used for Plug or Unplug operations type: string @@ -1753,7 +2027,18 @@ spec: singular: shardingdatabase scope: Namespaced versions: - - name: v1alpha1 + - additionalPrinterColumns: + - jsonPath: .status.gsm.state + name: Gsm State + type: string + - jsonPath: .status.gsm.services + name: Services + type: string + - jsonPath: .status.gsm.shards + name: shards + priority: 1 + type: string + name: v1alpha1 schema: openAPIV3Schema: description: ShardingDatabase is the Schema for the shardingdatabases API @@ -1769,6 +2054,8 @@ spec: spec: description: ShardingDatabaseSpec defines the desired state of ShardingDatabase properties: + InvitedNodeSubnet: + type: string catalog: items: description: CatalogSpec defines the desired state of CatalogSpec @@ -1812,6 +2099,21 @@ spec: resources: description: ResourceRequirements describes the compute resource requirements. properties: + claims: + description: "Claims lists the names of resources, defined in spec.resourceClaims, that are used by this container. \n This is an alpha field and requires enabling the DynamicResourceAllocation feature gate. \n This field is immutable. It can only be set for containers." + items: + description: ResourceClaim references one entry in PodSpec.ResourceClaims. + properties: + name: + description: Name must match the name of one entry in pod.spec.resourceClaims of the Pod where this field is used. It makes that resource available inside a container. + type: string + required: + - name + type: object + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map limits: additionalProperties: anyOf: @@ -1828,7 +2130,7 @@ spec: - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true - description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. Requests cannot exceed Limits. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' type: object type: object storageSizeInGb: @@ -1842,11 +2144,39 @@ spec: type: string dbImagePullSecret: type: string + dbSecret: + description: Secret Details + properties: + encryptionType: + type: string + keyFileMountLocation: + type: string + keyFileName: + type: string + keySecretName: + type: string + name: + type: string + nsConfigMap: + type: string + nsSecret: + type: string + pwdFileMountLocation: + type: string + pwdFileName: + type: string + required: + - name + - pwdFileName + type: object gsm: items: description: GsmSpec defines the desired state of GsmSpec properties: + directorName: + type: string envVars: + description: Replicas int32 `json:"replicas,omitempty"` // Gsm Replicas. If you set OraGsmPvcName then it is set default to 1. items: description: EnvironmentVariable represents a named variable accessible for containers. properties: @@ -1878,12 +2208,26 @@ spec: type: object pvcName: type: string - replicas: - format: int32 - type: integer + region: + type: string resources: description: ResourceRequirements describes the compute resource requirements. properties: + claims: + description: "Claims lists the names of resources, defined in spec.resourceClaims, that are used by this container. \n This is an alpha field and requires enabling the DynamicResourceAllocation feature gate. \n This field is immutable. It can only be set for containers." + items: + description: ResourceClaim references one entry in PodSpec.ResourceClaims. + properties: + name: + description: Name must match the name of one entry in pod.spec.resourceClaims of the Pod where this field is used. It makes that resource available inside a container. + type: string + required: + - name + type: object + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map limits: additionalProperties: anyOf: @@ -1900,7 +2244,7 @@ spec: - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true - description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. Requests cannot exceed Limits. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' type: object type: object storageSizeInGb: @@ -1910,10 +2254,109 @@ spec: - name type: object type: array + gsmDevMode: + type: string gsmImage: type: string gsmImagePullSecret: type: string + gsmService: + items: + description: Service Definition + properties: + available: + type: string + clbGoal: + type: string + commitOutcome: + type: string + drainTimeout: + type: string + dtp: + type: string + edition: + type: string + failoverDelay: + type: string + failoverMethod: + type: string + failoverPrimary: + type: string + failoverRestore: + type: string + failoverRetry: + type: string + failoverType: + type: string + gdsPool: + type: string + lag: + type: integer + locality: + type: string + name: + type: string + notification: + type: string + pdbName: + type: string + policy: + type: string + preferred: + type: string + prferredAll: + type: string + regionFailover: + type: string + retention: + type: string + role: + type: string + sessionState: + type: string + sqlTransactionProfile: + type: string + stopOption: + type: string + tableFamily: + type: string + tfaPolicy: + type: string + required: + - name + type: object + type: array + gsmShardGroup: + items: + properties: + deployAs: + type: string + name: + type: string + region: + type: string + required: + - name + type: object + type: array + gsmShardSpace: + items: + description: ShardSpace Specs + properties: + chunks: + type: integer + name: + type: string + protectionMode: + type: string + shardGroup: + type: string + required: + - name + type: object + type: array + invitedNodeSubnetFlag: + type: string isClone: type: boolean isDataGuard: @@ -1922,14 +2365,16 @@ spec: type: boolean isDeleteOraPvc: type: boolean + isDownloadScripts: + type: boolean isExternalSvc: type: boolean + isTdeWallet: + type: boolean + liveinessCheckPeriod: + type: integer namespace: type: string - nsConfigMap: - type: string - nsSecret: - type: string portMappings: items: description: PortMapping is a specification of port mapping for an application deployment. @@ -1949,15 +2394,19 @@ spec: - targetPort type: object type: array - scriptsLocation: + readinessCheckPeriod: + type: integer + replicationType: type: string - secret: + scriptsLocation: type: string shard: description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster Important: Run "make" to regenerate code after modifying this file' items: description: ShardSpec is a specification of Shards for an application deployment. properties: + deployAs: + type: string envVars: items: description: EnvironmentVariable represents a named variable accessible for containers. @@ -1997,6 +2446,21 @@ spec: resources: description: ResourceRequirements describes the compute resource requirements. properties: + claims: + description: "Claims lists the names of resources, defined in spec.resourceClaims, that are used by this container. \n This is an alpha field and requires enabling the DynamicResourceAllocation feature gate. \n This field is immutable. It can only be set for containers." + items: + description: ResourceClaim references one entry in PodSpec.ResourceClaims. + properties: + name: + description: Name must match the name of one entry in pod.spec.resourceClaims of the Pod where this field is used. It makes that resource available inside a container. + type: string + required: + - name + type: object + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map limits: additionalProperties: anyOf: @@ -2013,9 +2477,15 @@ spec: - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true - description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. Requests cannot exceed Limits. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' type: object type: object + shardGroup: + type: string + shardRegion: + type: string + shardSpace: + type: string storageSizeInGb: format: int32 type: integer @@ -2023,6 +2493,16 @@ spec: - name type: object type: array + shardBuddyRegion: + type: string + shardConfigName: + type: string + shardRegion: + items: + type: string + type: array + shardingType: + type: string stagePvcName: type: string storageClass: @@ -2032,7 +2512,6 @@ spec: - dbImage - gsm - gsmImage - - secret - shard type: object status: @@ -2207,10 +2686,12 @@ spec: type: boolean charset: type: string - cloneFrom: + createAs: + enum: + - primary + - standby + - clone type: string - createAsStandby: - type: boolean dgBrokerConfigured: type: boolean edition: @@ -2270,14 +2751,18 @@ spec: - ReadWriteOnce - ReadWriteMany type: string + datafilesVolumeName: + type: string + scriptsVolumeName: + type: string + setWritePermissions: + type: boolean size: type: string storageClass: type: string volumeClaimAnnotation: type: string - volumeName: - type: string type: object primaryDatabaseRef: type: string @@ -2300,6 +2785,8 @@ spec: type: string tcpsListenerPort: type: integer + tcpsTlsSecret: + type: string required: - image type: object @@ -2318,8 +2805,6 @@ spec: type: string clientWalletLoc: type: string - cloneFrom: - type: string clusterConnectString: type: string conditions: @@ -2370,6 +2855,8 @@ spec: x-kubernetes-list-type: map connectString: type: string + createdAs: + type: string datafilesCreated: default: "false" type: string @@ -2423,14 +2910,18 @@ spec: - ReadWriteOnce - ReadWriteMany type: string + datafilesVolumeName: + type: string + scriptsVolumeName: + type: string + setWritePermissions: + type: boolean size: type: string storageClass: type: string volumeClaimAnnotation: type: string - volumeName: - type: string type: object prebuiltDB: type: boolean @@ -2454,9 +2945,13 @@ spec: type: string tcpsPdbConnectString: type: string + tcpsTlsSecret: + default: "" + type: string required: - isTcpsEnabled - persistence + - tcpsTlsSecret type: object type: object served: true @@ -2525,6 +3020,23 @@ metadata: creationTimestamp: null name: oracle-database-operator-manager-role rules: +- apiGroups: + - "" + resources: + - configmaps + - deployments + - events + - pods + - secrets + - services + verbs: + - create + - delete + - get + - list + - patch + - update + - watch - apiGroups: - "" resources: @@ -2574,7 +3086,12 @@ rules: - "" resources: - events - - nodes + verbs: + - create + - patch +- apiGroups: + - "" + resources: - persistentvolumeclaims - pods - pods/exec @@ -2588,6 +3105,13 @@ rules: - patch - update - watch +- apiGroups: + - "" + resources: + - persistentvolumes + verbs: + - get + - list - apiGroups: - '''''' resources: @@ -2600,6 +3124,21 @@ rules: - patch - update - watch +- apiGroups: + - apps + resources: + - configmaps + - deployments + - pods + - services + verbs: + - create + - delete + - get + - list + - patch + - update + - watch - apiGroups: - apps resources: @@ -2637,9 +3176,9 @@ rules: - "" resources: - configmaps + - containers - events - namespaces - - nodes - persistentvolumeclaims - pods - pods/exec @@ -2947,6 +3486,53 @@ rules: - get - patch - update +- apiGroups: + - monitoring.coreos.com + resources: + - prometheusrules + - servicemonitors + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - observability.oracle.com + resources: + - databaseobservers + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - observability.oracle.com + resources: + - databaseobservers/finalizers + verbs: + - update +- apiGroups: + - observability.oracle.com + resources: + - databaseobservers/status + verbs: + - get + - patch + - update +- apiGroups: + - storage.k8s.io + resources: + - storageclasses + verbs: + - get + - list + - watch --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole @@ -2991,9 +3577,10 @@ subjects: namespace: oracle-database-operator-system --- apiVersion: rbac.authorization.k8s.io/v1 -kind: ClusterRoleBinding +kind: RoleBinding metadata: name: oracle-database-operator-oracle-database-operator-manager-rolebinding + namespace: oracle-database-operator-system roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole @@ -3006,11 +3593,11 @@ subjects: apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: - name: oracle-database-operator-oracle-database-operator-proxy-rolebinding + name: oracle-database-operator-proxy-rolebinding roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole - name: proxy-role + name: oracle-database-operator-oracle-database-operator-proxy-role subjects: - kind: ServiceAccount name: default @@ -3435,7 +4022,10 @@ spec: - --enable-leader-election command: - /manager - image: container-registry.oracle.com/database/operator:1.0.0 + env: + - name: WATCH_NAMESPACE + value: "" + image: container-registry.oracle.com/database/operator:latest imagePullPolicy: Always name: manager ports: diff --git a/rbac/cluster-role-binding.yaml b/rbac/cluster-role-binding.yaml new file mode 100644 index 00000000..1c609012 --- /dev/null +++ b/rbac/cluster-role-binding.yaml @@ -0,0 +1,15 @@ +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: oracle-database-operator-oracle-database-operator-manager-rolebinding + namespace: oracle-database-operator-system +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: oracle-database-operator-manager-role +subjects: +- kind: ServiceAccount + name: default + namespace: oracle-database-operator-system +--- diff --git a/rbac/default-ns-role-binding.yaml b/rbac/default-ns-role-binding.yaml new file mode 100644 index 00000000..b737e1f1 --- /dev/null +++ b/rbac/default-ns-role-binding.yaml @@ -0,0 +1,15 @@ +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: oracle-database-operator-oracle-database-operator-manager-rolebinding + namespace: default +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: oracle-database-operator-manager-role +subjects: +- kind: ServiceAccount + name: default + namespace: oracle-database-operator-system +--- diff --git a/rbac/node-rbac.yaml b/rbac/node-rbac.yaml new file mode 100644 index 00000000..ac474873 --- /dev/null +++ b/rbac/node-rbac.yaml @@ -0,0 +1,27 @@ +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: oracle-database-operator-manager-role-node +rules: +- apiGroups: + - "" + resources: + - nodes + verbs: + - list + - watch +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: oracle-database-operator-manager-role-node-cluster-role-binding +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: oracle-database-operator-manager-role-node +subjects: +- kind: ServiceAccount + name: default + namespace: oracle-database-operator-system +--- diff --git a/rbac/persistent-volume-rbac.yaml b/rbac/persistent-volume-rbac.yaml new file mode 100644 index 00000000..bce9733d --- /dev/null +++ b/rbac/persistent-volume-rbac.yaml @@ -0,0 +1,28 @@ +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: oracle-database-operator-manager-role-persistent-volume +rules: +- apiGroups: + - "" + resources: + - persistentvolumes + verbs: + - get + - list + - watch +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: oracle-database-operator-manager-role-persistent-volume-cluster-role-binding +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: oracle-database-operator-manager-role-persistent-volume +subjects: +- kind: ServiceAccount + name: default + namespace: oracle-database-operator-system +--- diff --git a/rbac/storage-class-rbac.yaml b/rbac/storage-class-rbac.yaml new file mode 100644 index 00000000..a34f67d4 --- /dev/null +++ b/rbac/storage-class-rbac.yaml @@ -0,0 +1,28 @@ +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: oracle-database-operator-manager-role-storage-class +rules: +- apiGroups: + - storage.k8s.io + resources: + - storageclasses + verbs: + - get + - list + - watch +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: oracle-database-operator-manager-role-storage-class-cluster-role-binding +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: oracle-database-operator-manager-role-storage-class +subjects: +- kind: ServiceAccount + name: default + namespace: oracle-database-operator-system +--- diff --git a/test/e2e/resource/test_config.yaml b/test/e2e/resource/test_config.yaml index 1907af47..dc2768e2 100644 --- a/test/e2e/resource/test_config.yaml +++ b/test/e2e/resource/test_config.yaml @@ -18,7 +18,7 @@ instanceWalletPasswordOCID: ocid1.vaultsecret... subnetOCID: ocid1.subnet... # The OCID of the network security group used to test the network access settings nsgOCID: ocid1.networksecuritygroup... -# The URL of the bucket used for configure ADB manual backup +# The URL of the bucket used for configure ADB on-demand backup bucketURL: https://swiftobjectstorage.region.oraclecloud.com/v1/namespace-string/bucket_name # The auth token generated in OCI Console > Profile > User Settings > Auth Token authToken: token diff --git a/test/e2e/util/util.go b/test/e2e/util/util.go index 81b5259d..b9e76aa7 100644 --- a/test/e2e/util/util.go +++ b/test/e2e/util/util.go @@ -44,7 +44,7 @@ import ( "strings" "time" - goyaml "gopkg.in/yaml.v2" + goyaml "gopkg.in/yaml.v3" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" From 31b5e9b656a5973269785ee4307d79bb9801893e Mon Sep 17 00:00:00 2001 From: Kuassi Mensah Date: Wed, 15 May 2024 16:20:07 -0700 Subject: [PATCH 604/628] Update README.md (#99) Reviewed and updated the Readme. --- README.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index ed684a8a..6629d976 100644 --- a/README.md +++ b/README.md @@ -9,7 +9,7 @@ In this v1.1.0 production release, `OraOperator` supports the following database * Oracle Autonomous Database: * Oracle Autonomous Database shared Oracle Cloud Infrastructure (OCI) (ADB-S) * Oracle Autonomous Database on dedicated Cloud infrastructure (ADB-D) - * Oracle Autonomous Container Database (ACD) (infrastructure) the infrastructure for provisioning Autonomous Databases. + * Oracle Autonomous Container Database (ACD) (infrastructure) is the infrastructure for provisioning Autonomous Databases. * Containerized Single Instance databases (SIDB) deployed in the Oracle Kubernetes Engine (OKE) and any k8s where OraOperator is deployed * Containerized Sharded databases (SHARDED) deployed in OKE and any k8s where OraOperator is deployed * Oracle Multitenant Databases (CDB/PDBs) @@ -20,7 +20,7 @@ In this v1.1.0 production release, `OraOperator` supports the following database Oracle will continue to extend `OraOperator` to support additional Oracle Database configurations. ## New in V1.1.0 Release -* Namespace scope deployment option +* Enhanced security with namespace scope deployment option * Support for Oracle Database 23ai Free (with SIDB) * Automatic Storage Expansion for SIDB and Sharded DB * User-Defined Sharding @@ -28,9 +28,9 @@ Oracle will continue to extend `OraOperator` to support additional Oracle Databa * Execute custom scripts during DB setup/startup * Patching for SIDB Primary/Standby in Data Guard * Long-term backup for Autonomous Databases (ADB): Moves to long-term backup and removes the deprecated mandatory backup -* Wallet expiry date for ADB: A user-freindly enhancement to display wallet expiry date in the status of assiciated ADB -* Wait-for-Completion option for ADB: Supports `kubectl wait` command that allows user to wait a specific condition on ADB -* OKE workload Identify: Supports OKE workload indentity authentication method. For more details, refer to [Oracle Autonomous Database (ADB) Prerequisites](docs/adb/ADB_PREREQUISITES.md#authorized-with-oke-workload-identity) +* Wallet expiry date for ADB: A user-friendly enhancement to display the wallet expiry date in the status of the associated ADB +* Wait-for-Completion option for ADB: Supports `kubectl wait` command that allows the user to wait for a specific condition on ADB +* OKE workload Identify: Supports OKE workload identity authentication method (i.e., uses OKE credentials). For more details, refer to [Oracle Autonomous Database (ADB) Prerequisites](docs/adb/ADB_PREREQUISITES.md#authorized-with-oke-workload-identity) * Database Observability (Preview - Metrics) ## Features Summary @@ -47,7 +47,7 @@ This release of Oracle Database Operator for Kubernetes (the operator) supports * Oracle Database Observability: create, patch, delete databaseObserver resources * Watch over a set of namespaces or all the namespaces in the cluster using the "WATCH_NAMESPACE" env variable of the operator deployment -The upcoming releases will support new configurations, operations and capabilities. +The upcoming releases will support new configurations, operations, and capabilities. ## Release Status From fba9b35696bb9f6d6905a06511dc862e8ac65670 Mon Sep 17 00:00:00 2001 From: Ting-Lan Wang Date: Wed, 29 May 2024 16:17:23 -0400 Subject: [PATCH 605/628] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 6629d976..38c48aad 100644 --- a/README.md +++ b/README.md @@ -27,7 +27,7 @@ Oracle will continue to extend `OraOperator` to support additional Oracle Databa * TCPS support customer provided certs * Execute custom scripts during DB setup/startup * Patching for SIDB Primary/Standby in Data Guard -* Long-term backup for Autonomous Databases (ADB): Moves to long-term backup and removes the deprecated mandatory backup +* Long-term backup for Autonomous Databases (ADB): Support for [long-term retention backup](https://docs.oracle.com/en/cloud/paas/autonomous-database/serverless/adbsb/backup-long-term.html) and removed support for the deprecated mandatory backup * Wallet expiry date for ADB: A user-friendly enhancement to display the wallet expiry date in the status of the associated ADB * Wait-for-Completion option for ADB: Supports `kubectl wait` command that allows the user to wait for a specific condition on ADB * OKE workload Identify: Supports OKE workload identity authentication method (i.e., uses OKE credentials). For more details, refer to [Oracle Autonomous Database (ADB) Prerequisites](docs/adb/ADB_PREREQUISITES.md#authorized-with-oke-workload-identity) From f8e273c5ab097a69ab196b4b6536bca826360c81 Mon Sep 17 00:00:00 2001 From: Ishaan Date: Thu, 4 Jul 2024 12:33:42 +0530 Subject: [PATCH 606/628] Doc and comments enhancement to support openshift --- config/samples/sidb/openshift_rbac.yaml | 72 ++++++++++++------- .../samples/sidb/singleinstancedatabase.yaml | 4 +- docs/sidb/README.md | 33 +++++++-- 3 files changed, 77 insertions(+), 32 deletions(-) diff --git a/config/samples/sidb/openshift_rbac.yaml b/config/samples/sidb/openshift_rbac.yaml index 8c88f78e..2e0d0cb2 100644 --- a/config/samples/sidb/openshift_rbac.yaml +++ b/config/samples/sidb/openshift_rbac.yaml @@ -1,73 +1,93 @@ # -# Copyright (c) 2023, Oracle and/or its affiliates. +# Copyright (c) 2024, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # - --- # Create a Security Context Contraint kind: SecurityContextConstraints - apiVersion: v1 + apiVersion: security.openshift.io/v1 + metadata: + name: sidb-oracle-user-scc + allowPrivilegedContainer: false + allowedCapabilities: + - SYS_NICE + runAsUser: + type: MustRunAs + uid: 54321 + seLinuxContext: + type: RunAsAny + fsGroup: + type: MustRunAs + ranges: + - min: 54321 + max: 54321 + supplementalGroups: + type: MustRunAs + ranges: + - min: 54321 + max: 54321 +--- +# Create a Security Context Contraint + + kind: SecurityContextConstraints + apiVersion: security.openshift.io/v1 metadata: - name: sidb-scc - namespace: default + name: sidb-oracle-root-user-scc allowPrivilegedContainer: false - users: - - system:serviceaccount:default:sidb-sa - - system:serviceaccount:default:oracle-database-operator + allowedCapabilities: + - SYS_NICE runAsUser: type: MustRunAsRange uidRangeMin: 0 - uidRangeMax: 60000 + uidRangeMax: 54321 seLinuxContext: type: RunAsAny fsGroup: type: MustRunAs ranges: - min: 0 - max: 60000 + max: 54321 supplementalGroups: type: MustRunAs ranges: - min: 0 - max: 60000 - + max: 54321 --- -# Create Service Account - apiVersion: v1 kind: ServiceAccount metadata: name: sidb-sa - namespace: default + namespace: sidb-ns --- -# Create a rbac role - kind: Role apiVersion: rbac.authorization.k8s.io/v1 metadata: name: use-sidb-scc - namespace: default + namespace: sidb-ns rules: - - apiGroups: ["security.openshift.io"] - resources: ["securitycontextconstraints"] - resourceNames: ["sidb-scc"] - verbs: ["use"] + - verbs: + - use + apiGroups: + - security.openshift.io + resources: + - securitycontextconstraints + resourceNames: + - oracle-user-scc + - oracle-root-scc --- -# Create a rbac role binding - kind: RoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: use-sidb-scc - namespace: default + namespace: sidb-ns subjects: - kind: ServiceAccount name: sidb-sa + namespace: sidb-ns roleRef: kind: Role name: use-sidb-scc apiGroup: rbac.authorization.k8s.io - \ No newline at end of file diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index 3fcf7d97..746b09d0 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -64,9 +64,9 @@ spec: tcpsCertRenewInterval: 8760h ## N/A for createAs clone or standby - ## Specify both sgaSize and pgaSize (in MB) or dont specify both ## Specify Non-Zero value to use - ## You cannot change these initParams for Oracle Database Express (XE) edition + ## sgaTarget and pagAggregateTarget must be in MB + ## You cannot change these initParams for Oracle Database Express (XE) and Oracle Database Free edition initParams: cpuCount: 0 processes: 0 diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 81db03b8..895c26a9 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -5,6 +5,7 @@ Oracle Database Operator for Kubernetes (`OraOperator`) includes the Single Inst * [Prerequisites](#prerequisites) * [Mandatory Resource Privileges](#mandatory-resource-privileges) * [Optional Resource Privileges](#optional-resource-privileges) + * [OpenShift Security Context Constraints](#openshift-security-context-constraints) * [SingleInstanceDatabase Resource](#singleinstancedatabase-resource) * [Create a Database](#create-a-database) * [New Database](#new-database) @@ -47,6 +48,7 @@ Oracle Database Operator for Kubernetes (`OraOperator`) includes the Single Inst * [Maintenance Operations](#maintenance-operations) * [Additional Information](#additional-information) + ## Prerequisites Oracle strongly recommends to comply with the [prerequisites](./PREREQUISITES.md) and the following requirements @@ -89,7 +91,30 @@ Oracle strongly recommends to comply with the [prerequisites](./PREREQUISITES.md ```sh kubectl apply -f rbac/persistent-volume-rbac.yaml ``` + + ### OpenShift Security Context Constraints + + OpenShift requires additional Security Context Constraints (SCC) for deploying and managing the SingleInstanceDatabase resource. Follow these steps to create the appropriate SCCs before deploying the SingleInstanceDatabase resource. + + 1. Create a new project/namespace for deploying the SingleInstanceDatabase resource + + ```sh + oc new-project sidb-ns + ``` + + **Note:** OpenShift recommends not to deploy in namespaces starting with `kube`, `openshift` and the `default` namespace. + + 2. Apply the file [openshift_rbac.yaml](../../config/samples/sidb/openshift_rbac.yaml) with cluster-admin user privileges. + + ```sh + oc apply -f openshift-rbac.yaml + ``` + + This would result in creation of SCC (Security Context Constraints) and serviceaccount `sidb-sa` in the namespace `sidb-ns` which has access to the SCC. + + **Note:** The above config yaml file will bind the SCC to the serviceaccount `sidb-sa` in namespace `sidb-ns`. For any other project/namespace update the file appropriately with the namespace before applying. + 3. Set the `serviceAccountName` attribute to `sidb-sa` and the namespace to `sidb-ns` in **[config/samples/sidb/singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml)** before deploying the SingleInstanceDatabase resource. ## SingleInstanceDatabase Resource @@ -961,12 +986,12 @@ $ kubectl describe oraclerestdataservice ords-sample ### Template YAML -The template `.yaml` file for Oracle Rest Data Services (`OracleRestDataService` kind), including all the configurable options, is available at **[config/samples/sidb/oraclerestdataservice.yaml](config/samples/sidb/oraclerestdataservice.yaml)**. +The template `.yaml` file for Oracle Rest Data Services (`OracleRestDataService` kind), including all the configurable options, is available at **[config/samples/sidb/oraclerestdataservice.yaml](../../config/samples/sidb/oraclerestdataservice.yaml)**. **Note:** - The `adminPassword` and `ordsPassword` fields in the `oraclerestdataservice.yaml` file contains secrets for authenticating the Single Instance Database and the ORDS user with the following roles: `SQL Administrator, System Administrator, SQL Developer, oracle.dbtools.autorest.any.schema`. - To build the ORDS image, use the following instructions: [Building Oracle REST Data Services Install Images](https://github.com/oracle/docker-images/tree/main/OracleRestDataServices#building-oracle-rest-data-services-install-images). -- By default, ORDS uses self-signed certificates. To use certificates from the Certificate Authority, the ORDS image needs to be rebuilt after specifying the values of `ssl.cert` and `ssl.cert.key` in the [standalone.properties](https://github.com/oracle/docker-images/blob/main/OracleRestDataServices/dockerfiles/standalone.properties.tmpl) file. After you rebuild the ORDS image, use the rebuilt image in the **[config/samples/sidb/oraclerestdataservice.yaml](config/samples/sidb/oraclerestdataservice.yaml)** file. +- By default, ORDS uses self-signed certificates. To use certificates from the Certificate Authority, the ORDS image needs to be rebuilt after specifying the values of `ssl.cert` and `ssl.cert.key` in the [standalone.properties](https://github.com/oracle/docker-images/blob/main/OracleRestDataServices/dockerfiles/standalone.properties.tmpl) file. After you rebuild the ORDS image, use the rebuilt image in the **[config/samples/sidb/oraclerestdataservice.yaml](../../config/samples/sidb/oraclerestdataservice.yaml)** file. - If you want to install ORDS in a [prebuilt database](#provision-a-pre-built-database), make sure to attach the **database persistence** by uncommenting the `persistence` section in the **[config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml](../../config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml)** file, while provisioning the prebuilt database. ### REST Enable a Database @@ -1114,7 +1139,7 @@ Fetch all entries from 'DEPT' table by calling the following API Database Actions is a web-based interface that uses Oracle REST Data Services to provide development, data tools, administration and monitoring features for Oracle Database. * To use Database Actions, you must sign in as a database user whose schema has been REST-enabled. -* To enable a schema for REST, you can specify appropriate values for the `.spec.restEnableSchemas` attributes details in the sample `yaml` **[config/samples/sidb/oraclerestdataservice.yaml](config/samples/sidb/oraclerestdataservice.yaml)**, which are needed for authorizing Database Actions. +* To enable a schema for REST, you can specify appropriate values for the `.spec.restEnableSchemas` attributes details in the sample `yaml` **[config/samples/sidb/oraclerestdataservice.yaml](../../config/samples/sidb/oraclerestdataservice.yaml)**, which are needed for authorizing Database Actions. * Schema are created (if they exist) with the username as `.spec.restEnableSchema[].schema` and password as `.spec.ordsPassword.`. * UrlMapping `.spec.restEnableSchema[].urlMapping` is optional and is defaulted to `.spec.restEnableSchema[].schema`. @@ -1148,7 +1173,7 @@ Using APEX, developers can quickly develop and deploy compelling apps that solve The `OraOperator` facilitates installation of APEX in the database and also configures ORDS for it. The following section will explain installing APEX with configured ORDS: -* For quick provisioning, use the sample **[config/samples/sidb/oraclerestdataservice_apex.yaml](../../confi/samples/sidb/oraclerestdataservice_apex.yaml)** file. For example: +* For quick provisioning, use the sample **[config/samples/sidb/oraclerestdataservice_apex.yaml](../../config/samples/sidb/oraclerestdataservice_apex.yaml)** file. For example: kubectl apply -f oraclerestdataservice_apex.yaml From 9695ec2bace5f71676a6033511f139ab13b9da87 Mon Sep 17 00:00:00 2001 From: Ishaan Date: Fri, 5 Jul 2024 11:14:22 +0530 Subject: [PATCH 607/628] enhancing formatting --- config/samples/sidb/openshift_rbac.yaml | 136 ++++++++++++------------ 1 file changed, 67 insertions(+), 69 deletions(-) diff --git a/config/samples/sidb/openshift_rbac.yaml b/config/samples/sidb/openshift_rbac.yaml index 2e0d0cb2..f35188b0 100644 --- a/config/samples/sidb/openshift_rbac.yaml +++ b/config/samples/sidb/openshift_rbac.yaml @@ -4,55 +4,53 @@ # --- # Create a Security Context Contraint - - kind: SecurityContextConstraints - apiVersion: security.openshift.io/v1 - metadata: - name: sidb-oracle-user-scc - allowPrivilegedContainer: false - allowedCapabilities: - - SYS_NICE - runAsUser: - type: MustRunAs - uid: 54321 - seLinuxContext: - type: RunAsAny - fsGroup: - type: MustRunAs - ranges: - - min: 54321 - max: 54321 - supplementalGroups: +kind: SecurityContextConstraints +apiVersion: security.openshift.io/v1 +metadata: + name: sidb-oracle-user-scc +allowPrivilegedContainer: false +allowedCapabilities: + - SYS_NICE +runAsUser: + type: MustRunAs + uid: 54321 +seLinuxContext: + type: RunAsAny +fsGroup: + type: MustRunAs + ranges: + - min: 54321 + max: 54321 +supplementalGroups: type: MustRunAs ranges: - min: 54321 max: 54321 --- # Create a Security Context Contraint - - kind: SecurityContextConstraints - apiVersion: security.openshift.io/v1 - metadata: - name: sidb-oracle-root-user-scc - allowPrivilegedContainer: false - allowedCapabilities: - - SYS_NICE - runAsUser: - type: MustRunAsRange - uidRangeMin: 0 - uidRangeMax: 54321 - seLinuxContext: - type: RunAsAny - fsGroup: - type: MustRunAs - ranges: - - min: 0 - max: 54321 - supplementalGroups: - type: MustRunAs - ranges: - - min: 0 - max: 54321 +kind: SecurityContextConstraints +apiVersion: security.openshift.io/v1 +metadata: + name: sidb-oracle-root-user-scc +allowPrivilegedContainer: false +allowedCapabilities: + - SYS_NICE +runAsUser: + type: MustRunAsRange + uidRangeMin: 0 + uidRangeMax: 54321 +seLinuxContext: + type: RunAsAny +fsGroup: + type: MustRunAs + ranges: + - min: 0 + max: 54321 +supplementalGroups: + type: MustRunAs + ranges: + - min: 0 + max: 5432 --- apiVersion: v1 kind: ServiceAccount @@ -61,33 +59,33 @@ metadata: namespace: sidb-ns --- - kind: Role - apiVersion: rbac.authorization.k8s.io/v1 - metadata: - name: use-sidb-scc - namespace: sidb-ns - rules: - - verbs: - - use - apiGroups: - - security.openshift.io +kind: Role +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: use-sidb-scc + namespace: sidb-ns +rules: + - apiGroups: + - security.openshift.io + verbs: + - use resources: - - securitycontextconstraints + - securitycontextconstraints resourceNames: - - oracle-user-scc - - oracle-root-scc + - sidb-oracle-user-scc + - sidb-oracle-root-scc --- - kind: RoleBinding - apiVersion: rbac.authorization.k8s.io/v1 - metadata: - name: use-sidb-scc - namespace: sidb-ns - subjects: - - kind: ServiceAccount - name: sidb-sa - namespace: sidb-ns - roleRef: - kind: Role - name: use-sidb-scc - apiGroup: rbac.authorization.k8s.io +kind: RoleBinding +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: use-sidb-scc + namespace: sidb-ns +subjects: + - kind: ServiceAccount + name: sidb-sa + namespace: sidb-ns +roleRef: + kind: Role + name: use-sidb-scc + apiGroup: rbac.authorization.k8s.io From c8139d9f9cc74ed074dd21b1d106754be2ed1600 Mon Sep 17 00:00:00 2001 From: Ishaan Date: Fri, 5 Jul 2024 11:15:47 +0530 Subject: [PATCH 608/628] enhancing formatting openshift_rbac --- config/samples/sidb/openshift_rbac.yaml | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/config/samples/sidb/openshift_rbac.yaml b/config/samples/sidb/openshift_rbac.yaml index f35188b0..021f73a2 100644 --- a/config/samples/sidb/openshift_rbac.yaml +++ b/config/samples/sidb/openshift_rbac.yaml @@ -3,6 +3,7 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # --- + # Create a Security Context Contraint kind: SecurityContextConstraints apiVersion: security.openshift.io/v1 @@ -27,6 +28,7 @@ supplementalGroups: - min: 54321 max: 54321 --- + # Create a Security Context Contraint kind: SecurityContextConstraints apiVersion: security.openshift.io/v1 @@ -52,13 +54,14 @@ supplementalGroups: - min: 0 max: 5432 --- + apiVersion: v1 kind: ServiceAccount metadata: name: sidb-sa namespace: sidb-ns - --- + kind: Role apiVersion: rbac.authorization.k8s.io/v1 metadata: @@ -74,8 +77,8 @@ rules: resourceNames: - sidb-oracle-user-scc - sidb-oracle-root-scc - --- + kind: RoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: From 3cd5250b839718e3793ceaaee6d8e3e2613fab8d Mon Sep 17 00:00:00 2001 From: Ishaan Date: Sat, 6 Jul 2024 00:16:14 +0530 Subject: [PATCH 609/628] correcting scc ref --- config/samples/sidb/openshift_rbac.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/config/samples/sidb/openshift_rbac.yaml b/config/samples/sidb/openshift_rbac.yaml index 021f73a2..6dddb80d 100644 --- a/config/samples/sidb/openshift_rbac.yaml +++ b/config/samples/sidb/openshift_rbac.yaml @@ -76,7 +76,7 @@ rules: - securitycontextconstraints resourceNames: - sidb-oracle-user-scc - - sidb-oracle-root-scc + - sidb-oracle-root-user-scc --- kind: RoleBinding From 72161cd5596fdb27e902cf10f5f9f241a4d23e20 Mon Sep 17 00:00:00 2001 From: Ishaan Date: Mon, 8 Jul 2024 10:16:19 +0530 Subject: [PATCH 610/628] fixing typo --- config/samples/sidb/singleinstancedatabase.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index 746b09d0..b66082e1 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -65,7 +65,7 @@ spec: ## N/A for createAs clone or standby ## Specify Non-Zero value to use - ## sgaTarget and pagAggregateTarget must be in MB + ## sgaTarget and pgaAggregateTarget must be in MB ## You cannot change these initParams for Oracle Database Express (XE) and Oracle Database Free edition initParams: cpuCount: 0 From fc51187fc4b9f90821ddb1f5306f68c61f49fba7 Mon Sep 17 00:00:00 2001 From: Saurabh Ahuja Date: Wed, 4 Sep 2024 02:01:35 +0530 Subject: [PATCH 611/628] Squashed commit with all contents from new_stuff branch (#141) * Squashed commit with all contents from new_stuff branch * fixed missed conflict * Oracle's instructions for reporting security vulnerabilities * Installation added --- .gitignore | 4 + PROJECT | 8 + README.md | 3 +- SECURITY.md | 4 +- .../v1alpha1/adbfamily_common_utils.go | 6 +- .../autonomousdatabasebackup_types.go | 12 +- .../v1alpha1/dataguardbroker_types.go | 16 +- apis/database/v1alpha1/groupversion_info.go | 4 +- apis/database/v1alpha1/pdb_types.go | 4 + .../v1alpha1/shardingdatabase_types.go | 105 +- .../v1alpha1/shardingdatabase_webhook.go | 270 ++++ .../v1alpha1/singleinstancedatabase_types.go | 11 + apis/database/v1alpha1/webhook_suite_test.go | 8 +- .../v1alpha1/zz_generated.deepcopy.go | 41 + commons/dbcssystem/dbcs_reconciler.go | 10 +- commons/observability/constants.go | 1 + commons/oci/wallet.go | 2 +- commons/sharding/catalog.go | 50 +- commons/sharding/exec.go | 21 + commons/sharding/gsm.go | 2 +- commons/sharding/scommon.go | 219 ++- commons/sharding/shard.go | 21 +- .../crd/bases/database.oracle.com_pdbs.yaml | 5 + ...database.oracle.com_shardingdatabases.yaml | 21 +- ...se.oracle.com_singleinstancedatabases.yaml | 17 + config/database.oracle.com_DbcsSystem.yaml | 240 ++++ ...acle.com_autonomouscontainerdatabases.yaml | 117 ++ ....oracle.com_autonomousdatabasebackups.yaml | 138 ++ ...oracle.com_autonomousdatabaserestores.yaml | 138 ++ ...tabase.oracle.com_autonomousdatabases.yaml | 324 +++++ config/database.oracle.com_cdbs.yaml | 270 ++++ .../database.oracle.com_dataguardbrokers.yaml | 134 ++ ...ase.oracle.com_oraclerestdataservices.yaml | 224 +++ config/database.oracle.com_pdbs.yaml | 383 ++++++ ...database.oracle.com_shardingdatabases.yaml | 688 ++++++++++ ...se.oracle.com_singleinstancedatabases.yaml | 421 ++++++ config/manager/kustomization.yaml | 4 +- ...tabase-operator.clusterserviceversion.yaml | 97 +- ...vability.oracle.com_databaseobservers.yaml | 227 ++++ config/samples/kustomization.yaml | 32 +- .../samples/sidb/singleinstancedatabase.yaml | 15 + config/webhook/manifests.yaml | 41 + controllers/database/pdb_controller.go | 132 +- .../database/shardingdatabase_controller.go | 231 ++-- .../singleinstancedatabase_controller.go | 32 + .../databaseobserver_resource.go | 6 +- docs/multitenant/README.md | 142 +- docs/multitenant/provisioning/add_replica.log | 192 --- docs/multitenant/provisioning/add_replica.md | 36 - .../multitenant/provisioning/add_replica.yaml | 40 - docs/multitenant/provisioning/cdb.log | 279 ---- docs/multitenant/provisioning/cdb.yaml | 41 - .../provisioning/cdb_crd_resource.md | 38 - docs/multitenant/provisioning/cdb_secret.yaml | 17 - docs/multitenant/provisioning/clone_pdb.log | 137 -- docs/multitenant/provisioning/clone_pdb.md | 38 - docs/multitenant/provisioning/clone_pdb.yaml | 20 - docs/multitenant/provisioning/create_pdb.log | 139 -- docs/multitenant/provisioning/create_pdb.md | 37 - docs/multitenant/provisioning/create_pdb.yaml | 27 - docs/multitenant/provisioning/delete_pdb.log | 157 --- docs/multitenant/provisioning/delete_pdb.md | 37 - docs/multitenant/provisioning/delete_pdb.yaml | 16 - .../example_setup_using_oci_oke_cluster.md | 3 +- docs/multitenant/provisioning/known_issues.md | 49 - docs/multitenant/provisioning/modify_pdb.log | 181 --- docs/multitenant/provisioning/modify_pdb.md | 69 - .../provisioning/modify_pdb_close.yaml | 18 - .../provisioning/modify_pdb_open.yaml | 18 - .../multinamespace/cdb_create.yaml | 44 + .../multinamespace/pdb_clone.yaml | 50 + .../multinamespace/pdb_close.yaml | 44 + .../multinamespace/pdb_create.yaml | 46 + .../multinamespace/pdb_delete.yaml | 34 + .../provisioning/multinamespace/pdb_open.yaml | 43 + .../provisioning/multinamespace/pdb_plug.yaml | 46 + .../multinamespace/pdb_unplug.yaml | 39 + docs/multitenant/provisioning/ords_image.log | 503 ------- docs/multitenant/provisioning/ords_image.md | 49 +- .../provisioning/pdb_crd_resource.md | 0 docs/multitenant/provisioning/plug_pdb.log | 100 -- docs/multitenant/provisioning/plug_pdb.md | 44 - docs/multitenant/provisioning/plug_pdb.yaml | 22 - .../provisioning/quickOKEcreation.md | 136 ++ .../singlenamespace/cdb_create.yaml | 44 + .../singlenamespace/cdb_secret.yaml | 17 + .../singlenamespace/pdb_clone.yaml | 50 + .../singlenamespace/pdb_close.yaml | 44 + .../singlenamespace/pdb_create.yaml | 47 + .../singlenamespace/pdb_delete.yaml | 34 + .../singlenamespace/pdb_open.yaml | 43 + .../singlenamespace/pdb_plug.yaml | 46 + .../{ => singlenamespace}/pdb_secret.yaml | 11 +- .../pdb_unplug.yaml} | 32 +- docs/multitenant/provisioning/unplug_pdb.log | 165 --- docs/multitenant/provisioning/unplug_pdb.md | 39 - docs/multitenant/provisioning/unplug_pdb.yaml | 17 - .../provisioning/validation_error.md | 73 - docs/multitenant/usecase01/README.md | 150 ++- docs/multitenant/usecase01/ca.crt | 25 + docs/multitenant/usecase01/ca.key | 27 + docs/multitenant/usecase01/ca.srl | 1 + docs/multitenant/usecase01/extfile.txt | 1 + .../usecase01/logfiles/BuildImage.log | 1199 +++++++++++------ .../usecase01/logfiles/cdb_creation.log | 357 +++++ .../usecase01/logfiles/openssl_execution.log | 27 +- .../usecase01/logfiles/ordsconfig.log | 58 +- .../usecase01/logfiles/tagandpush.log | 14 + .../usecase01/logfiles/testapi.log | 29 +- docs/multitenant/usecase01/makefile | 29 +- docs/multitenant/usecase01/pdb_create.yaml | 1 + docs/multitenant/usecase01/pdb_delete.yaml | 34 + docs/multitenant/usecase01/pdb_map.yaml | 1 + docs/multitenant/usecase01/server.csr | 18 + docs/multitenant/usecase01/tls.crt | 24 + docs/multitenant/usecase01/tls.key | 28 + docs/multitenant/usecase02/README.md | 10 +- docs/multitenant/usecase02/pdb_clone.yaml | 1 + docs/multitenant/usecase02/pdb_plug.yaml | 1 + docs/multitenant/usecase03/Dockerfile | 63 +- docs/multitenant/usecase03/README.md | 8 +- docs/multitenant/usecase03/cdb_create.yaml | 4 +- docs/multitenant/usecase03/makefile | 18 +- docs/multitenant/usecase03/runOrdsSSL.sh | 190 +++ docs/sharding/README.md | 73 +- docs/sharding/provisioning/debugging.md | 7 + .../sharding_provisioning_with_db_events.md | 40 + .../sharding_provisioning_with_db_events.yaml | 69 + .../sharding_provisioning_with_free_images.md | 40 + ...harding_provisioning_with_free_images.yaml | 58 + ...y_cloning_db_from_gold_image_across_ads.md | 57 + ...ing_by_cloning_db_gold_image_in_same_ad.md | 53 + ...ding_provisioning_with_chunks_specified.md | 43 + ..._provisioning_with_control_on_resources.md | 47 + ...ith_notification_using_oci_notification.md | 87 ++ ...ding_provisioning_without_db_gold_image.md | 40 + ...rding_scale_in_delete_an_existing_shard.md | 50 + .../snr_ssharding_scale_out_add_shards.md | 37 + .../snr_ssharding_shard_prov.yaml | 58 + .../snr_ssharding_shard_prov_chunks.yaml | 61 + .../snr_ssharding_shard_prov_clone.yaml | 83 ++ ...ssharding_shard_prov_clone_across_ads.yaml | 91 ++ .../snr_ssharding_shard_prov_delshard.yaml | 69 + .../snr_ssharding_shard_prov_extshard.yaml | 68 + .../snr_ssharding_shard_prov_memory_cpu.yaml | 89 ++ ...sharding_shard_prov_send_notification.yaml | 85 ++ ...y_cloning_db_from_gold_image_across_ads.md | 7 +- ...ing_by_cloning_db_gold_image_in_same_ad.md | 4 +- ...ding_provisioning_with_chunks_specified.md | 40 + ..._provisioning_with_control_on_resources.md | 7 +- ...ith_notification_using_oci_notification.md | 5 +- ...ding_provisioning_without_db_gold_image.md | 9 +- ...rding_scale_in_delete_an_existing_shard.md | 7 +- .../ssharding_scale_out_add_shards.md | 5 +- .../system_sharding/ssharding_shard_prov.yaml | 1 - ...ssharding_shard_prov_clone_across_ads.yaml | 10 +- .../ssharding_shard_prov_delshard.yaml | 2 +- ...y_cloning_db_from_gold_image_across_ads.md | 2 + ...ing_by_cloning_db_gold_image_in_same_ad.md | 2 + ..._provisioning_with_control_on_resources.md | 3 + ...ith_notification_using_oci_notification.md | 3 + ...ding_provisioning_without_db_gold_image.md | 2 +- ...rding_scale_in_delete_an_existing_shard.md | 44 +- .../udsharding_scale_out_add_shards.md | 1 + .../udsharding_shard_prov.yaml | 1 - ...dsharding_shard_prov_clone_across_ads.yaml | 8 + .../udsharding_shard_prov_delshard.yaml | 4 +- .../udsharding_shard_prov_memory_cpu.yaml | 1 - docs/sidb/README.md | 4 + main.go | 3 + oracle-database-operator.yaml | 169 ++- ords/Dockerfile | 61 +- ords/runOrdsSSL.sh | 121 +- 173 files changed, 8888 insertions(+), 3613 deletions(-) create mode 100644 apis/database/v1alpha1/shardingdatabase_webhook.go create mode 100644 config/database.oracle.com_DbcsSystem.yaml create mode 100644 config/database.oracle.com_autonomouscontainerdatabases.yaml create mode 100644 config/database.oracle.com_autonomousdatabasebackups.yaml create mode 100644 config/database.oracle.com_autonomousdatabaserestores.yaml create mode 100644 config/database.oracle.com_autonomousdatabases.yaml create mode 100644 config/database.oracle.com_cdbs.yaml create mode 100644 config/database.oracle.com_dataguardbrokers.yaml create mode 100644 config/database.oracle.com_oraclerestdataservices.yaml create mode 100644 config/database.oracle.com_pdbs.yaml create mode 100644 config/database.oracle.com_shardingdatabases.yaml create mode 100644 config/database.oracle.com_singleinstancedatabases.yaml create mode 100644 config/observability.oracle.com_databaseobservers.yaml delete mode 100644 docs/multitenant/provisioning/add_replica.log delete mode 100644 docs/multitenant/provisioning/add_replica.md delete mode 100644 docs/multitenant/provisioning/add_replica.yaml delete mode 100644 docs/multitenant/provisioning/cdb.log delete mode 100644 docs/multitenant/provisioning/cdb.yaml delete mode 100644 docs/multitenant/provisioning/cdb_crd_resource.md delete mode 100644 docs/multitenant/provisioning/cdb_secret.yaml delete mode 100644 docs/multitenant/provisioning/clone_pdb.log delete mode 100644 docs/multitenant/provisioning/clone_pdb.md delete mode 100644 docs/multitenant/provisioning/clone_pdb.yaml delete mode 100644 docs/multitenant/provisioning/create_pdb.log delete mode 100644 docs/multitenant/provisioning/create_pdb.md delete mode 100644 docs/multitenant/provisioning/create_pdb.yaml delete mode 100644 docs/multitenant/provisioning/delete_pdb.log delete mode 100644 docs/multitenant/provisioning/delete_pdb.md delete mode 100644 docs/multitenant/provisioning/delete_pdb.yaml delete mode 100644 docs/multitenant/provisioning/known_issues.md delete mode 100644 docs/multitenant/provisioning/modify_pdb.log delete mode 100644 docs/multitenant/provisioning/modify_pdb.md delete mode 100644 docs/multitenant/provisioning/modify_pdb_close.yaml delete mode 100644 docs/multitenant/provisioning/modify_pdb_open.yaml create mode 100644 docs/multitenant/provisioning/multinamespace/cdb_create.yaml create mode 100644 docs/multitenant/provisioning/multinamespace/pdb_clone.yaml create mode 100644 docs/multitenant/provisioning/multinamespace/pdb_close.yaml create mode 100644 docs/multitenant/provisioning/multinamespace/pdb_create.yaml create mode 100644 docs/multitenant/provisioning/multinamespace/pdb_delete.yaml create mode 100644 docs/multitenant/provisioning/multinamespace/pdb_open.yaml create mode 100644 docs/multitenant/provisioning/multinamespace/pdb_plug.yaml create mode 100644 docs/multitenant/provisioning/multinamespace/pdb_unplug.yaml delete mode 100644 docs/multitenant/provisioning/ords_image.log delete mode 100644 docs/multitenant/provisioning/pdb_crd_resource.md delete mode 100644 docs/multitenant/provisioning/plug_pdb.log delete mode 100644 docs/multitenant/provisioning/plug_pdb.md delete mode 100644 docs/multitenant/provisioning/plug_pdb.yaml create mode 100644 docs/multitenant/provisioning/quickOKEcreation.md create mode 100644 docs/multitenant/provisioning/singlenamespace/cdb_create.yaml create mode 100644 docs/multitenant/provisioning/singlenamespace/cdb_secret.yaml create mode 100644 docs/multitenant/provisioning/singlenamespace/pdb_clone.yaml create mode 100644 docs/multitenant/provisioning/singlenamespace/pdb_close.yaml create mode 100644 docs/multitenant/provisioning/singlenamespace/pdb_create.yaml create mode 100644 docs/multitenant/provisioning/singlenamespace/pdb_delete.yaml create mode 100644 docs/multitenant/provisioning/singlenamespace/pdb_open.yaml create mode 100644 docs/multitenant/provisioning/singlenamespace/pdb_plug.yaml rename docs/multitenant/provisioning/{ => singlenamespace}/pdb_secret.yaml (54%) rename docs/multitenant/provisioning/{pdb.yaml => singlenamespace/pdb_unplug.yaml} (50%) delete mode 100644 docs/multitenant/provisioning/unplug_pdb.log delete mode 100644 docs/multitenant/provisioning/unplug_pdb.md delete mode 100644 docs/multitenant/provisioning/unplug_pdb.yaml delete mode 100644 docs/multitenant/provisioning/validation_error.md create mode 100644 docs/multitenant/usecase01/ca.crt create mode 100644 docs/multitenant/usecase01/ca.key create mode 100644 docs/multitenant/usecase01/ca.srl create mode 100644 docs/multitenant/usecase01/extfile.txt create mode 100644 docs/multitenant/usecase01/logfiles/cdb_creation.log create mode 100644 docs/multitenant/usecase01/logfiles/tagandpush.log create mode 100644 docs/multitenant/usecase01/pdb_delete.yaml create mode 100644 docs/multitenant/usecase01/server.csr create mode 100644 docs/multitenant/usecase01/tls.crt create mode 100644 docs/multitenant/usecase01/tls.key create mode 100644 docs/multitenant/usecase03/runOrdsSSL.sh create mode 100644 docs/sharding/provisioning/debugging/sharding_provisioning_with_db_events.md create mode 100644 docs/sharding/provisioning/debugging/sharding_provisioning_with_db_events.yaml create mode 100644 docs/sharding/provisioning/free/sharding_provisioning_with_free_images.md create mode 100644 docs/sharding/provisioning/free/sharding_provisioning_with_free_images.yaml create mode 100644 docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md create mode 100644 docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md create mode 100644 docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_chunks_specified.md create mode 100644 docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_control_on_resources.md create mode 100644 docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_notification_using_oci_notification.md create mode 100644 docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_without_db_gold_image.md create mode 100644 docs/sharding/provisioning/snr_system_sharding/snr_ssharding_scale_in_delete_an_existing_shard.md create mode 100644 docs/sharding/provisioning/snr_system_sharding/snr_ssharding_scale_out_add_shards.md create mode 100644 docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov.yaml create mode 100644 docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_chunks.yaml create mode 100644 docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_clone.yaml create mode 100644 docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_clone_across_ads.yaml create mode 100644 docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_delshard.yaml create mode 100644 docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_extshard.yaml create mode 100644 docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_memory_cpu.yaml create mode 100644 docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_send_notification.yaml create mode 100644 docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_chunks_specified.md diff --git a/.gitignore b/.gitignore index 618e3efb..98fbc1c4 100644 --- a/.gitignore +++ b/.gitignore @@ -7,3 +7,7 @@ ords/*zip .gitattributes .vscode .gitlab-ci.yml + +# development +.idea +.local diff --git a/PROJECT b/PROJECT index 7b3ec718..fbf861db 100644 --- a/PROJECT +++ b/PROJECT @@ -1,3 +1,7 @@ +# Code generated by tool. DO NOT EDIT. +# This file is used to track the info used to scaffold your project +# and allow the plugins properly work. +# More info: https://book.kubebuilder.io/reference/project-config.html domain: oracle.com layout: - go.kubebuilder.io/v2 @@ -67,6 +71,10 @@ resources: kind: ShardingDatabase path: github.com/oracle/oracle-database-operator/apis/database/v1alpha1 version: v1alpha1 + webhooks: + defaulting: true + validation: true + webhookVersion: v1beta1 - api: crdVersion: v1 namespaced: true diff --git a/README.md b/README.md index 38c48aad..959045cb 100644 --- a/README.md +++ b/README.md @@ -20,6 +20,7 @@ In this v1.1.0 production release, `OraOperator` supports the following database Oracle will continue to extend `OraOperator` to support additional Oracle Database configurations. ## New in V1.1.0 Release +* Namespace scope deployment option * Enhanced security with namespace scope deployment option * Support for Oracle Database 23ai Free (with SIDB) * Automatic Storage Expansion for SIDB and Sharded DB @@ -125,7 +126,7 @@ Oracle strongly recommends that you ensure your system meets the following [Prer ```sh kubectl apply -f rbac/node-rbac.yaml ``` - +# Installation ## Install Oracle DB Operator After you have completed the preceding prerequisite changes, you can install the operator. To install the operator in the cluster quickly, you can apply the modified `oracle-database-operator.yaml` file from the preceding step. diff --git a/SECURITY.md b/SECURITY.md index fb238413..30159518 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -1,4 +1,4 @@ -# Reporting security vulnerabilities +# Oracle's instructions for reporting security vulnerabilities Oracle values the independent security research community and believes that responsible disclosure of security vulnerabilities helps us ensure the security @@ -35,4 +35,4 @@ sufficiently hardened for production use. [1]: mailto:secalert_us@oracle.com [2]: https://www.oracle.com/corporate/security-practices/assurance/vulnerability/reporting.html [3]: https://www.oracle.com/security-alerts/encryptionkey.html -[4]: https://www.oracle.com/security-alerts/ +[4]: https://www.oracle.com/security-alerts/ \ No newline at end of file diff --git a/apis/database/v1alpha1/adbfamily_common_utils.go b/apis/database/v1alpha1/adbfamily_common_utils.go index e6691a12..d4d3ae9f 100644 --- a/apis/database/v1alpha1/adbfamily_common_utils.go +++ b/apis/database/v1alpha1/adbfamily_common_utils.go @@ -173,9 +173,9 @@ func traverse(lastSpec interface{}, curSpec interface{}) bool { return changed } -// 1. If the current field is with a zero value, then the field is unchanged. -// 2. If the current field is NOT with a zero value, then we want to comapre it with the last field. -// In this case if the last field is with a zero value, then the field is changed +// 1. If the current field is with a zero value, then the field is unchanged. +// 2. If the current field is NOT with a zero value, then we want to comapre it with the last field. +// In this case if the last field is with a zero value, then the field is changed func hasChanged(lastField reflect.Value, curField reflect.Value) bool { zero := reflect.Zero(lastField.Type()).Interface() lastFieldIsZero := reflect.DeepEqual(lastField.Interface(), zero) diff --git a/apis/database/v1alpha1/autonomousdatabasebackup_types.go b/apis/database/v1alpha1/autonomousdatabasebackup_types.go index 876cb811..95c77560 100644 --- a/apis/database/v1alpha1/autonomousdatabasebackup_types.go +++ b/apis/database/v1alpha1/autonomousdatabasebackup_types.go @@ -52,12 +52,12 @@ import ( type AutonomousDatabaseBackupSpec struct { // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster // Important: Run "make" to regenerate code after modifying this file - Target TargetSpec `json:"target,omitempty"` - DisplayName *string `json:"displayName,omitempty"` - AutonomousDatabaseBackupOCID *string `json:"autonomousDatabaseBackupOCID,omitempty"` - IsLongTermBackup *bool `json:"isLongTermBackup,omitempty"` - RetentionPeriodInDays *int `json:"retentionPeriodInDays,omitempty"` - OCIConfig OCIConfigSpec `json:"ociConfig,omitempty"` + Target TargetSpec `json:"target,omitempty"` + DisplayName *string `json:"displayName,omitempty"` + AutonomousDatabaseBackupOCID *string `json:"autonomousDatabaseBackupOCID,omitempty"` + IsLongTermBackup *bool `json:"isLongTermBackup,omitempty"` + RetentionPeriodInDays *int `json:"retentionPeriodInDays,omitempty"` + OCIConfig OCIConfigSpec `json:"ociConfig,omitempty"` } // AutonomousDatabaseBackupStatus defines the observed state of AutonomousDatabaseBackup diff --git a/apis/database/v1alpha1/dataguardbroker_types.go b/apis/database/v1alpha1/dataguardbroker_types.go index 138e2bdb..37d71b92 100644 --- a/apis/database/v1alpha1/dataguardbroker_types.go +++ b/apis/database/v1alpha1/dataguardbroker_types.go @@ -50,15 +50,15 @@ type DataguardBrokerSpec struct { // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster // Important: Run "make" to regenerate code after modifying this file - PrimaryDatabaseRef string `json:"primaryDatabaseRef"` - StandbyDatabaseRefs []string `json:"standbyDatabaseRefs"` - SetAsPrimaryDatabase string `json:"setAsPrimaryDatabase,omitempty"` - LoadBalancer bool `json:"loadBalancer,omitempty"` - ServiceAnnotations map[string]string `json:"serviceAnnotations,omitempty"` + PrimaryDatabaseRef string `json:"primaryDatabaseRef"` + StandbyDatabaseRefs []string `json:"standbyDatabaseRefs"` + SetAsPrimaryDatabase string `json:"setAsPrimaryDatabase,omitempty"` + LoadBalancer bool `json:"loadBalancer,omitempty"` + ServiceAnnotations map[string]string `json:"serviceAnnotations,omitempty"` // +kubebuilder:validation:Enum=MaxPerformance;MaxAvailability - ProtectionMode string `json:"protectionMode"` - NodeSelector map[string]string `json:"nodeSelector,omitempty"` - FastStartFailOver DataguardBrokerFastStartFailOver `json:"fastStartFailOver,omitempty"` + ProtectionMode string `json:"protectionMode"` + NodeSelector map[string]string `json:"nodeSelector,omitempty"` + FastStartFailOver DataguardBrokerFastStartFailOver `json:"fastStartFailOver,omitempty"` } type DataguardBrokerFastStartFailOver struct { diff --git a/apis/database/v1alpha1/groupversion_info.go b/apis/database/v1alpha1/groupversion_info.go index d72ee908..3c4b1804 100644 --- a/apis/database/v1alpha1/groupversion_info.go +++ b/apis/database/v1alpha1/groupversion_info.go @@ -37,8 +37,8 @@ */ // Package v1alpha1 contains API Schema definitions for the database v1alpha1 API group -//+kubebuilder:object:generate=true -//+groupName=database.oracle.com +// +kubebuilder:object:generate=true +// +groupName=database.oracle.com package v1alpha1 import ( diff --git a/apis/database/v1alpha1/pdb_types.go b/apis/database/v1alpha1/pdb_types.go index 1e4da0a1..8de9db52 100644 --- a/apis/database/v1alpha1/pdb_types.go +++ b/apis/database/v1alpha1/pdb_types.go @@ -114,6 +114,10 @@ type PDBSpec struct { // The target state of the PDB // +kubebuilder:validation:Enum=OPEN;CLOSE PDBState string `json:"pdbState,omitempty"` + // turn on the assertive approach to delete pdb resource + // kubectl delete pdb ..... automatically triggers the pluggable database + // deletion + AssertivePdbDeletion bool `json:"assertivePdbDeletion,omitempty"` } // PDBAdminName defines the secret containing Sys Admin User mapped to key 'adminName' for PDB diff --git a/apis/database/v1alpha1/shardingdatabase_types.go b/apis/database/v1alpha1/shardingdatabase_types.go index 3b4f9c17..ffc17ab0 100644 --- a/apis/database/v1alpha1/shardingdatabase_types.go +++ b/apis/database/v1alpha1/shardingdatabase_types.go @@ -58,39 +58,43 @@ import ( type ShardingDatabaseSpec struct { // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster // Important: Run "make" to regenerate code after modifying this file - Shard []ShardSpec `json:"shard"` - Catalog []CatalogSpec `json:"catalog"` // The catalogSpes accept all the catalog parameters - Gsm []GsmSpec `json:"gsm"` // The GsmSpec will accept all the Gsm parameter - StorageClass string `json:"storageClass,omitempty"` // Optional Accept storage class name - DbImage string `json:"dbImage"` // Accept DB Image name - DbImagePullSecret string `json:"dbImagePullSecret,omitempty"` // Optional The name of an image pull secret in case of a private docker repository. - GsmImage string `json:"gsmImage"` // Acccept the GSM image name - GsmImagePullSecret string `json:"gsmImagePullSecret,omitempty"` // Optional The name of an image pull secret in case of a private docker repository. - StagePvcName string `json:"stagePvcName,omitempty"` // the Stagepvc for the backup of cluster - PortMappings []PortMapping `json:"portMappings,omitempty"` // Port mappings for the service that is created. The service is created if there is at least - Namespace string `json:"namespace,omitempty"` // Target namespace of the application. - IsDebug bool `json:"isDebug,omitempty"` // Optional parameter to enable logining - IsExternalSvc bool `json:"isExternalSvc,omitempty"` - IsClone bool `json:"isClone,omitempty"` - IsDataGuard bool `json:"isDataGuard,omitempty"` - ScriptsLocation string `json:"scriptsLocation,omitempty"` - IsDeleteOraPvc bool `json:"isDeleteOraPvc,omitempty"` - ReadinessCheckPeriod int `json:"readinessCheckPeriod,omitempty"` - LivenessCheckPeriod int `json:"liveinessCheckPeriod,omitempty"` - ReplicationType string `json:"replicationType,omitempty"` - IsDownloadScripts bool `json:"isDownloadScripts,omitempty"` - InvitedNodeSubnetFlag string `json:"invitedNodeSubnetFlag,omitempty"` - InvitedNodeSubnet string `json:"InvitedNodeSubnet,omitempty"` - ShardingType string `json:"shardingType,omitempty"` - GsmShardSpace []GsmShardSpaceSpec `json:"gsmShardSpace,omitempty"` - GsmShardGroup []GsmShardGroupSpec `json:"gsmShardGroup,omitempty"` - ShardRegion []string `json:"shardRegion,omitempty"` - ShardBuddyRegion string `json:"shardBuddyRegion,omitempty"` - GsmService []GsmServiceSpec `json:"gsmService,omitempty"` - ShardConfigName string `json:"shardConfigName,omitempty"` - GsmDevMode string `json:"gsmDevMode,omitempty"` - DbSecret *SecretDetails `json:"dbSecret,omitempty"` // Secret Name to be used with Shard - IsTdeWallet bool `json:"isTdeWallet,omitempty"` + Shard []ShardSpec `json:"shard"` + Catalog []CatalogSpec `json:"catalog"` // The catalogSpes accept all the catalog parameters + Gsm []GsmSpec `json:"gsm"` // The GsmSpec will accept all the Gsm parameter + StorageClass string `json:"storageClass,omitempty"` // Optional Accept storage class name + DbImage string `json:"dbImage"` // Accept DB Image name + DbImagePullSecret string `json:"dbImagePullSecret,omitempty"` // Optional The name of an image pull secret in case of a private docker repository. + GsmImage string `json:"gsmImage"` // Acccept the GSM image name + GsmImagePullSecret string `json:"gsmImagePullSecret,omitempty"` // Optional The name of an image pull secret in case of a private docker repository. + StagePvcName string `json:"stagePvcName,omitempty"` // the Stagepvc for the backup of cluster + PortMappings []PortMapping `json:"portMappings,omitempty"` // Port mappings for the service that is created. The service is created if there is at least + Namespace string `json:"namespace,omitempty"` // Target namespace of the application. + IsDebug bool `json:"isDebug,omitempty"` // Optional parameter to enable logining + IsExternalSvc bool `json:"isExternalSvc,omitempty"` + IsClone bool `json:"isClone,omitempty"` + IsDataGuard bool `json:"isDataGuard,omitempty"` + ScriptsLocation string `json:"scriptsLocation,omitempty"` + IsDeleteOraPvc bool `json:"isDeleteOraPvc,omitempty"` + ReadinessCheckPeriod int `json:"readinessCheckPeriod,omitempty"` + LivenessCheckPeriod int `json:"liveinessCheckPeriod,omitempty"` + ReplicationType string `json:"replicationType,omitempty"` + IsDownloadScripts bool `json:"isDownloadScripts,omitempty"` + InvitedNodeSubnetFlag string `json:"invitedNodeSubnetFlag,omitempty"` + InvitedNodeSubnet string `json:"InvitedNodeSubnet,omitempty"` + ShardingType string `json:"shardingType,omitempty"` + GsmShardSpace []GsmShardSpaceSpec `json:"gsmShardSpace,omitempty"` + GsmShardGroup []GsmShardGroupSpec `json:"gsmShardGroup,omitempty"` + ShardRegion []string `json:"shardRegion,omitempty"` + ShardBuddyRegion string `json:"shardBuddyRegion,omitempty"` + GsmService []GsmServiceSpec `json:"gsmService,omitempty"` + ShardConfigName string `json:"shardConfigName,omitempty"` + GsmDevMode string `json:"gsmDevMode,omitempty"` + DbSecret *SecretDetails `json:"dbSecret,omitempty"` // Secret Name to be used with Shard + IsTdeWallet string `json:"isTdeWallet,omitempty"` + TdeWalletPvc string `json:"tdeWalletPvc,omitempty"` + FssStorageClass string `json:"fssStorageClass,omitempty"` + TdeWalletPvcMountLocation string `json:"tdeWalletPvcMountLocation,omitempty"` + DbEdition string `json:"dbEdition,omitempty"` } // To understand Metav1.Condition, please refer the link https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1 @@ -164,21 +168,22 @@ type ShardingDatabaseList struct { // ShardSpec is a specification of Shards for an application deployment. // +k8s:openapi-gen=true type ShardSpec struct { - Name string `json:"name"` // Shard name that will be used deploy StatefulSet - StorageSizeInGb int32 `json:"storageSizeInGb,omitempty"` // Optional Shard Storage Size - EnvVars []EnvironmentVariable `json:"envVars,omitempty"` //Optional Env variables for Shards - Resources *corev1.ResourceRequirements `json:"resources,omitempty" protobuf:"bytes,1,opt,name=resources"` //Optional resource requirement for the container. - PvcName string `json:"pvcName,omitempty"` - Label string `json:"label,omitempty"` - IsDelete bool `json:"isDelete,omitempty"` - NodeSelector map[string]string `json:"nodeSelector,omitempty"` - PvAnnotations map[string]string `json:"pvAnnotations,omitempty"` - PvMatchLabels map[string]string `json:"pvMatchLabels,omitempty"` - ImagePulllPolicy *corev1.PullPolicy `json:"imagePullPolicy,omitempty"` - ShardSpace string `json:"shardSpace,omitempty"` - ShardGroup string `json:"shardGroup,omitempty"` - ShardRegion string `json:"shardRegion,omitempty"` - DeployAs string `json:"deployAs,omitempty"` + Name string `json:"name"` // Shard name that will be used deploy StatefulSet + StorageSizeInGb int32 `json:"storageSizeInGb,omitempty"` // Optional Shard Storage Size + EnvVars []EnvironmentVariable `json:"envVars,omitempty"` //Optional Env variables for Shards + Resources *corev1.ResourceRequirements `json:"resources,omitempty" protobuf:"bytes,1,opt,name=resources"` //Optional resource requirement for the container. + PvcName string `json:"pvcName,omitempty"` + Label string `json:"label,omitempty"` + // +kubebuilder:validation:Enum=enable;disable;failed;force + IsDelete string `json:"isDelete,omitempty"` + NodeSelector map[string]string `json:"nodeSelector,omitempty"` + PvAnnotations map[string]string `json:"pvAnnotations,omitempty"` + PvMatchLabels map[string]string `json:"pvMatchLabels,omitempty"` + ImagePulllPolicy *corev1.PullPolicy `json:"imagePullPolicy,omitempty"` + ShardSpace string `json:"shardSpace,omitempty"` + ShardGroup string `json:"shardGroup,omitempty"` + ShardRegion string `json:"shardRegion,omitempty"` + DeployAs string `json:"deployAs,omitempty"` } // CatalogSpec defines the desired state of CatalogSpec @@ -190,7 +195,7 @@ type CatalogSpec struct { Resources *corev1.ResourceRequirements `json:"resources,omitempty" protobuf:"bytes,1,opt,name=resources"` // Optional resource requirement for the container. PvcName string `json:"pvcName,omitempty"` Label string `json:"label,omitempty"` - IsDelete bool `json:"isDelete,omitempty"` + IsDelete string `json:"isDelete,omitempty"` NodeSelector map[string]string `json:"nodeSelector,omitempty"` PvAnnotations map[string]string `json:"pvAnnotations,omitempty"` PvMatchLabels map[string]string `json:"pvMatchLabels,omitempty"` @@ -208,7 +213,7 @@ type GsmSpec struct { Resources *corev1.ResourceRequirements `json:"resources,omitempty" protobuf:"bytes,1,opt,name=resources"` // Optional resource requirement for the container. PvcName string `json:"pvcName,omitempty"` Label string `json:"label,omitempty"` // Optional GSM Label - IsDelete bool `json:"isDelete,omitempty"` + IsDelete string `json:"isDelete,omitempty"` NodeSelector map[string]string `json:"nodeSelector,omitempty"` PvMatchLabels map[string]string `json:"pvMatchLabels,omitempty"` ImagePulllPolicy *corev1.PullPolicy `json:"imagePullPolicy,omitempty"` diff --git a/apis/database/v1alpha1/shardingdatabase_webhook.go b/apis/database/v1alpha1/shardingdatabase_webhook.go new file mode 100644 index 00000000..8b91fb0c --- /dev/null +++ b/apis/database/v1alpha1/shardingdatabase_webhook.go @@ -0,0 +1,270 @@ +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v1alpha1 + +import ( + "strings" + + apierrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/runtime/schema" + "k8s.io/apimachinery/pkg/util/validation/field" + ctrl "sigs.k8s.io/controller-runtime" + logf "sigs.k8s.io/controller-runtime/pkg/log" + "sigs.k8s.io/controller-runtime/pkg/webhook" + "sigs.k8s.io/controller-runtime/pkg/webhook/admission" +) + +// log is for logging in this package. +var shardingdatabaselog = logf.Log.WithName("shardingdatabase-resource") + +func (r *ShardingDatabase) SetupWebhookWithManager(mgr ctrl.Manager) error { + return ctrl.NewWebhookManagedBy(mgr). + For(r). + Complete() +} + +// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! + +//+kubebuilder:webhook:path=/mutate-database-oracle-com-v1alpha1-shardingdatabase,mutating=true,failurePolicy=fail,sideEffects=none,groups=database.oracle.com,resources=shardingdatabases,verbs=create;update,versions=v1alpha1,name=mshardingdatabase.kb.io,admissionReviewVersions={v1} + +var _ webhook.Defaulter = &ShardingDatabase{} + +// Default implements webhook.Defaulter so a webhook will be registered for the type +func (r *ShardingDatabase) Default() { + shardingdatabaselog.Info("default", "name", r.Name) + + // TODO(user): fill in your defaulting logic. + if r.Spec.GsmDevMode != "" { + r.Spec.GsmDevMode = "dev" + } + + if r.Spec.IsTdeWallet == "" { + r.Spec.IsTdeWallet = "disable" + } + for pindex := range r.Spec.Shard { + if strings.ToLower(r.Spec.Shard[pindex].IsDelete) == "" { + r.Spec.Shard[pindex].IsDelete = "disable" + } + } + +} + +// TODO(user): change verbs to "verbs=create;update;delete" if you want to enable deletion validation. +//+kubebuilder:webhook:verbs=create;update;delete,path=/validate-database-oracle-com-v1alpha1-shardingdatabase,mutating=false,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=shardingdatabases,versions=v1alpha1,name=vshardingdatabase.kb.io,admissionReviewVersions={v1} + +var _ webhook.Validator = &ShardingDatabase{} + +// ValidateCreate implements webhook.Validator so a webhook will be registered for the type +func (r *ShardingDatabase) ValidateCreate() (admission.Warnings, error) { + shardingdatabaselog.Info("validate create", "name", r.Name) + + // TODO(user): fill in your validation logic upon object creation. + // Check Secret configuration + var validationErr field.ErrorList + var validationErrs1 field.ErrorList + + //namespaces := db.GetWatchNamespaces() + //_, containsNamespace := namespaces[r.Namespace] + // Check if the allowed namespaces maps contains the required namespace + // if len(namespaces) != 0 && !containsNamespace { + // validationErr = append(validationErr, + // field.Invalid(field.NewPath("metadata").Child("namespace"), r.Namespace, + // "Oracle database operator doesn't watch over this namespace")) + //} + + if r.Spec.DbSecret == nil { + validationErr = append(validationErr, + field.Invalid(field.NewPath("spec").Child("DbSecret"), r.Spec.DbSecret, + "DbSecret cannot be set to nil")) + } else { + if len(r.Spec.DbSecret.Name) == 0 { + validationErr = append(validationErr, + field.Invalid(field.NewPath("spec").Child("DbSecret").Child("Name"), r.Spec.DbSecret.Name, + "Secret name cannot be set empty")) + } + if len(r.Spec.DbSecret.PwdFileName) == 0 { + validationErr = append(validationErr, + field.Invalid(field.NewPath("spec").Child("DbSecret").Child("PwdFileName"), r.Spec.DbSecret.PwdFileName, + "Password file name cannot be set empty")) + } + if strings.ToLower(r.Spec.DbSecret.EncryptionType) != "base64" { + if strings.ToLower(r.Spec.DbSecret.KeyFileName) == "" { + validationErr = append(validationErr, + field.Invalid(field.NewPath("spec").Child("DbSecret").Child("KeyFileName"), r.Spec.DbSecret.KeyFileName, + "Key file name cannot be empty")) + } + } + + /** + if len(r.Spec.DbSecret.PwdFileMountLocation) == 0 { + validationErr = append(validationErr, + field.Invalid(field.NewPath("spec").Child("DbSecret").Child("PwdFileMountLocation"), r.Spec.DbSecret.PwdFileMountLocation, + "Password file mount location cannot be empty")) + } + + if len(r.Spec.DbSecret.KeyFileMountLocation) == 0 { + validationErr = append(validationErr, + field.Invalid(field.NewPath("spec").Child("DbSecret").Child("KeyFileMountLocation"), r.Spec.DbSecret.KeyFileMountLocation, + "KeyFileMountLocation file mount location cannot be empty")) + } + **/ + } + + if r.Spec.IsTdeWallet == "enable" { + if (len(r.Spec.FssStorageClass) == 0) && (len(r.Spec.TdeWalletPvc) == 0) { + validationErr = append(validationErr, + field.Invalid(field.NewPath("spec").Child("FssStorageClass"), r.Spec.FssStorageClass, + "FssStorageClass or TdeWalletPvc cannot be set empty if isTdeWallet set to true")) + + validationErr = append(validationErr, + field.Invalid(field.NewPath("spec").Child("TdeWalletPvc"), r.Spec.TdeWalletPvc, + "FssStorageClass or TdeWalletPvc cannot be set empty if isTdeWallet set to true")) + } + } + + if r.Spec.IsTdeWallet != "" { + if (strings.ToLower(strings.TrimSpace(r.Spec.IsTdeWallet)) != "enable") && (strings.ToLower(strings.TrimSpace(r.Spec.IsTdeWallet)) != "disable") { + validationErr = append(validationErr, + field.Invalid(field.NewPath("spec").Child("isTdeWallet"), r.Spec.IsTdeWallet, + "isTdeWallet can be set to only \"enable\" or \"disable\"")) + } + } + + validationErrs1 = r.validateShardIsDelete() + if validationErrs1 != nil { + validationErr = append(validationErr, validationErrs1...) + } + + validationErrs1 = r.validateFreeEdition() + if validationErrs1 != nil { + validationErr = append(validationErr, validationErrs1...) + } + + // TODO(user): fill in your validation logic upon object creation. + if len(validationErr) == 0 { + return nil, nil + } + + return nil, apierrors.NewInvalid( + schema.GroupKind{Group: "database.oracle.com", Kind: "ShardingDatabase"}, + r.Name, validationErr) +} + +// ValidateUpdate implements webhook.Validator so a webhook will be registered for the type +func (r *ShardingDatabase) ValidateUpdate(old runtime.Object) (admission.Warnings, error) { + shardingdatabaselog.Info("validate update", "name", r.Name) + + // TODO(user): fill in your validation logic upon object update. + return nil, nil +} + +// ValidateDelete implements webhook.Validator so a webhook will be registered for the type +func (r *ShardingDatabase) ValidateDelete() (admission.Warnings, error) { + shardingdatabaselog.Info("validate delete", "name", r.Name) + + // TODO(user): fill in your validation logic upon object deletion. + return nil, nil +} + +// ###### Vlaidation Block ################# + +func (r *ShardingDatabase) validateShardIsDelete() field.ErrorList { + + var validationErrs field.ErrorList + + for pindex := range r.Spec.Shard { + if (strings.ToLower(strings.TrimSpace(r.Spec.Shard[pindex].IsDelete)) != "enable") && (strings.ToLower(strings.TrimSpace(r.Spec.Shard[pindex].IsDelete)) != "disable") && (strings.ToLower(strings.TrimSpace(r.Spec.Shard[pindex].IsDelete)) != "failed") { + validationErrs = append(validationErrs, + field.Invalid(field.NewPath("spec").Child("shard").Child("isDelete"), r.Spec.Shard[pindex].IsDelete, + "r.Spec.Shard[pindex].IsDelete can be set to only enable|disable|failed")) + } + } + + if len(validationErrs) > 0 { + return validationErrs + } + return nil +} + +func (r *ShardingDatabase) validateFreeEdition() field.ErrorList { + + var validationErrs field.ErrorList + if strings.ToLower(r.Spec.DbEdition) == "free" { + // Shard Spec Checks + for i := 0; i < len(r.Spec.Shard); i++ { + for index, variable := range r.Spec.Shard[i].EnvVars { + if variable.Name == "ORACLE_SID" { + if strings.ToLower(variable.Value) != "free" { + validationErrs = append(validationErrs, field.Invalid(field.NewPath("spec").Child("shard").Child("EnvVars"), r.Spec.Shard[i].EnvVars[index].Name, + "r.Spec.Shard[i].EnvVars[index].Name ORACLE_SID value can only be set to free")) + } + } + if variable.Name == "ORACLE_PDB" { + if strings.ToLower(variable.Value) != "freepdb" { + validationErrs = append(validationErrs, field.Invalid(field.NewPath("spec").Child("shard").Child("EnvVars"), r.Spec.Shard[i].EnvVars[index].Name, + "r.Spec.Shard[i].EnvVars[index].Name ORACLE_PDB value can only be set to freepdb")) + } + } + } + } + // Catalog Spec Checks + for i := 0; i < len(r.Spec.Catalog); i++ { + for index, variable := range r.Spec.Catalog[i].EnvVars { + if variable.Name == "ORACLE_SID" { + if strings.ToLower(variable.Value) != "free" { + validationErrs = append(validationErrs, field.Invalid(field.NewPath("spec").Child("catalog").Child("EnvVars"), r.Spec.Catalog[i].EnvVars[index].Name, + "r.Spec.Catalog[i].EnvVars[index].Name ORACLE_SID value can only be set to free")) + } + } + if variable.Name == "ORACLE_PDB" { + if strings.ToLower(variable.Value) != "freepdb" { + validationErrs = append(validationErrs, field.Invalid(field.NewPath("spec").Child("catalog").Child("EnvVars"), r.Spec.Catalog[i].EnvVars[index].Name, + "r.Spec.Catalog[i].EnvVars[index].Name ORACLE_PDB value can only be set to freepdb")) + } + } + } + } + } + + if len(validationErrs) > 0 { + return validationErrs + } + return nil +} diff --git a/apis/database/v1alpha1/singleinstancedatabase_types.go b/apis/database/v1alpha1/singleinstancedatabase_types.go index 0d49aa6b..7c6c1ea5 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_types.go +++ b/apis/database/v1alpha1/singleinstancedatabase_types.go @@ -86,6 +86,17 @@ type SingleInstanceDatabaseSpec struct { Image SingleInstanceDatabaseImage `json:"image"` Persistence SingleInstanceDatabasePersistence `json:"persistence,omitempty"` InitParams *SingleInstanceDatabaseInitParams `json:"initParams,omitempty"` + Resources SingleInstanceDatabaseResources `json:"resources,omitempty"` +} + +type SingleInstanceDatabaseResource struct { + Cpu string `json:"cpu,omitempty"` + Memory string `json:"memory,omitempty"` +} + +type SingleInstanceDatabaseResources struct { + Requests *SingleInstanceDatabaseResource `json:"requests,omitempty"` + Limits *SingleInstanceDatabaseResource `json:"limits,omitempty"` } // SingleInstanceDatabasePersistence defines the storage size and class for PVC diff --git a/apis/database/v1alpha1/webhook_suite_test.go b/apis/database/v1alpha1/webhook_suite_test.go index 3f740a41..e28925e6 100644 --- a/apis/database/v1alpha1/webhook_suite_test.go +++ b/apis/database/v1alpha1/webhook_suite_test.go @@ -61,8 +61,8 @@ import ( "sigs.k8s.io/controller-runtime/pkg/envtest" logf "sigs.k8s.io/controller-runtime/pkg/log" "sigs.k8s.io/controller-runtime/pkg/log/zap" - "sigs.k8s.io/controller-runtime/pkg/webhook" metricsserver "sigs.k8s.io/controller-runtime/pkg/metrics/server" + "sigs.k8s.io/controller-runtime/pkg/webhook" ) // To avoid dot import @@ -137,14 +137,14 @@ var _ = BeforeSuite(func() { // start webhook server using Manager webhookInstallOptions := &testEnv.WebhookInstallOptions mgr, err := ctrl.NewManager(cfg, ctrl.Options{ - Scheme: scheme, + Scheme: scheme, WebhookServer: webhook.NewServer(webhook.Options{ Port: webhookInstallOptions.LocalServingPort, Host: webhookInstallOptions.LocalServingHost, CertDir: webhookInstallOptions.LocalServingCertDir, }), - LeaderElection: false, - Metrics: metricsserver.Options{ + LeaderElection: false, + Metrics: metricsserver.Options{ BindAddress: "0", }, }) diff --git a/apis/database/v1alpha1/zz_generated.deepcopy.go b/apis/database/v1alpha1/zz_generated.deepcopy.go index f1eaf503..10b34ca7 100644 --- a/apis/database/v1alpha1/zz_generated.deepcopy.go +++ b/apis/database/v1alpha1/zz_generated.deepcopy.go @@ -2470,6 +2470,46 @@ func (in *SingleInstanceDatabasePersistence) DeepCopy() *SingleInstanceDatabaseP return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *SingleInstanceDatabaseResource) DeepCopyInto(out *SingleInstanceDatabaseResource) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SingleInstanceDatabaseResource. +func (in *SingleInstanceDatabaseResource) DeepCopy() *SingleInstanceDatabaseResource { + if in == nil { + return nil + } + out := new(SingleInstanceDatabaseResource) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *SingleInstanceDatabaseResources) DeepCopyInto(out *SingleInstanceDatabaseResources) { + *out = *in + if in.Requests != nil { + in, out := &in.Requests, &out.Requests + *out = new(SingleInstanceDatabaseResource) + **out = **in + } + if in.Limits != nil { + in, out := &in.Limits, &out.Limits + *out = new(SingleInstanceDatabaseResource) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SingleInstanceDatabaseResources. +func (in *SingleInstanceDatabaseResources) DeepCopy() *SingleInstanceDatabaseResources { + if in == nil { + return nil + } + out := new(SingleInstanceDatabaseResources) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *SingleInstanceDatabaseSpec) DeepCopyInto(out *SingleInstanceDatabaseSpec) { *out = *in @@ -2510,6 +2550,7 @@ func (in *SingleInstanceDatabaseSpec) DeepCopyInto(out *SingleInstanceDatabaseSp *out = new(SingleInstanceDatabaseInitParams) **out = **in } + in.Resources.DeepCopyInto(&out.Resources) } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SingleInstanceDatabaseSpec. diff --git a/commons/dbcssystem/dbcs_reconciler.go b/commons/dbcssystem/dbcs_reconciler.go index 70e50294..60905c76 100644 --- a/commons/dbcssystem/dbcs_reconciler.go +++ b/commons/dbcssystem/dbcs_reconciler.go @@ -94,7 +94,7 @@ func CreateAndGetDbcsId(logger logr.Logger, kubeClient client.Client, dbClient d dbcsDetails.CompartmentId = common.String(dbcs.Spec.DbSystem.CompartmentId) dbcsDetails.SubnetId = common.String(dbcs.Spec.DbSystem.SubnetId) dbcsDetails.Shape = common.String(dbcs.Spec.DbSystem.Shape) - dbcsDetails.Domain = common.String(dbcs.Spec.DbSystem.Domain) + dbcsDetails.Domain = common.String(dbcs.Spec.DbSystem.Domain) if dbcs.Spec.DbSystem.DisplayName != "" { dbcsDetails.DisplayName = common.String(dbcs.Spec.DbSystem.DisplayName) } @@ -536,10 +536,10 @@ func GetResourceState(logger logr.Logger, dbClient database.DatabaseClient, Id s func SetDBCSStatus(dbClient database.DatabaseClient, dbcs *databasev1alpha1.DbcsSystem, nwClient core.VirtualNetworkClient, wrClient workrequests.WorkRequestClient) error { - if dbcs.Spec.Id == nil { - dbcs.Status.State = "FAILED" - return nil - } + if dbcs.Spec.Id == nil { + dbcs.Status.State = "FAILED" + return nil + } dbcsId := *dbcs.Spec.Id diff --git a/commons/observability/constants.go b/commons/observability/constants.go index a4d85b71..89ecb946 100644 --- a/commons/observability/constants.go +++ b/commons/observability/constants.go @@ -38,6 +38,7 @@ const ( DefaultExporterImage = "container-registry.oracle.com/database/observability-exporter:1.1.0" DefaultServicePort = 9161 + DefaultServiceTargetPort = 9161 DefaultPrometheusPort = "metrics" DefaultReplicaCount = 1 DefaultExporterConfigMountRootPath = "/oracle/observability" diff --git a/commons/oci/wallet.go b/commons/oci/wallet.go index a5f9235e..076460b1 100644 --- a/commons/oci/wallet.go +++ b/commons/oci/wallet.go @@ -107,4 +107,4 @@ func WalletExpiringDate(files map[string][]byte) string { line := data[strings.Index(data, "this wallet will expire on"):strings.Index(data, ".\nIn order to avoid")] return strings.TrimSpace(strings.TrimPrefix(line, "this wallet will expire on")) -} \ No newline at end of file +} diff --git a/commons/sharding/catalog.go b/commons/sharding/catalog.go index 8a0019dd..58f07490 100644 --- a/commons/sharding/catalog.go +++ b/commons/sharding/catalog.go @@ -43,9 +43,8 @@ import ( "reflect" "strconv" - databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" - "github.com/go-logr/logr" + databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" appsv1 "k8s.io/api/apps/v1" corev1 "k8s.io/api/core/v1" "k8s.io/apimachinery/pkg/api/resource" @@ -198,6 +197,12 @@ func buildVolumeSpecForCatalog(instance *databasev1alpha1.ShardingDatabase, OraC result = append(result, corev1.Volume{Name: OraCatalogSpex.Name + "orascript-vol5", VolumeSource: corev1.VolumeSource{EmptyDir: &corev1.EmptyDirVolumeSource{}}}) } + if checkTdeWalletFlag(instance) { + if len(instance.Spec.FssStorageClass) == 0 && len(instance.Spec.TdeWalletPvc) > 0 { + result = append(result, corev1.Volume{Name: OraCatalogSpex.Name + "shared-storage-vol8", VolumeSource: corev1.VolumeSource{PersistentVolumeClaim: &corev1.PersistentVolumeClaimVolumeSource{ClaimName: instance.Spec.TdeWalletPvc}}}) + } + } + return result } @@ -210,7 +215,7 @@ func buildContainerSpecForCatalog(instance *databasev1alpha1.ShardingDatabase, O Image: instance.Spec.DbImage, SecurityContext: &corev1.SecurityContext{ Capabilities: &corev1.Capabilities{ - Add: []corev1.Capability{"NET_RAW"}, + Add: []corev1.Capability{corev1.Capability("NET_ADMIN"), corev1.Capability("SYS_NICE")}, }, }, Resources: corev1.ResourceRequirements{ @@ -328,6 +333,15 @@ func buildVolumeMountSpecForCatalog(instance *databasev1alpha1.ShardingDatabase, result = append(result, corev1.VolumeMount{Name: OraCatalogSpex.Name + "orastage-vol7", MountPath: oraStage}) } + if checkTdeWalletFlag(instance) { + if len(instance.Spec.FssStorageClass) > 0 && len(instance.Spec.TdeWalletPvc) == 0 { + result = append(result, corev1.VolumeMount{Name: instance.Name + "shared-storage", MountPath: getTdeWalletMountLoc(instance)}) + } else { + if len(instance.Spec.FssStorageClass) == 0 && len(instance.Spec.TdeWalletPvc) > 0 { + result = append(result, corev1.VolumeMount{Name: OraCatalogSpex.Name + "shared-storage-vol8", MountPath: getTdeWalletMountLoc(instance)}) + } + } + } return result } @@ -372,6 +386,34 @@ func volumeClaimTemplatesForCatalog(instance *databasev1alpha1.ShardingDatabase, claims[0].Spec.Selector = &metav1.LabelSelector{MatchLabels: OraCatalogSpex.PvMatchLabels} } + if checkTdeWalletFlag(instance) { + if len(instance.Spec.FssStorageClass) > 0 && len(instance.Spec.TdeWalletPvc) == 0 { + { + pvcClaim := corev1.PersistentVolumeClaim{ + ObjectMeta: metav1.ObjectMeta{ + Name: instance.Name + "shared-storage", + Namespace: instance.Spec.Namespace, + OwnerReferences: getOwnerRef(instance), + Labels: buildLabelsForCatalog(instance, "sharding"), + }, + Spec: corev1.PersistentVolumeClaimSpec{ + AccessModes: []corev1.PersistentVolumeAccessMode{ + corev1.ReadWriteMany, + }, + StorageClassName: &instance.Spec.FssStorageClass, + Resources: corev1.VolumeResourceRequirements{ + Requests: corev1.ResourceList{ + corev1.ResourceStorage: resource.MustParse(strconv.FormatInt(int64(OraCatalogSpex.StorageSizeInGb), 10) + "Gi"), + }, + }, + }, + } + + claims = append(claims, pvcClaim) + } + } + } + return claims } @@ -454,7 +496,7 @@ func UpdateProvForCatalog(instance *databasev1alpha1.ShardingDatabase, oraSpexRes := OraCatalogSpex.Resources if !reflect.DeepEqual(shardContaineRes, oraSpexRes) { - isUpdate = true + isUpdate = false } } } diff --git a/commons/sharding/exec.go b/commons/sharding/exec.go index c1921018..44f91e51 100644 --- a/commons/sharding/exec.go +++ b/commons/sharding/exec.go @@ -63,12 +63,33 @@ import ( // ExecCMDInContainer execute command in first container of a pod func ExecCommand(podName string, cmd []string, kubeClient kubernetes.Interface, kubeConfig clientcmd.ClientConfig, instance *databasealphav1.ShardingDatabase, logger logr.Logger) (string, string, error) { + var err1 error = nil var msg string var ( execOut bytes.Buffer execErr bytes.Buffer ) + for i := 0; i < 5; i++ { + if scheme.Scheme == nil { + time.Sleep(time.Second * 40) + } else { + break + } + } + + if kubeClient == nil { + msg = "ExecCommand() : kubeClient is nil" + err1 = fmt.Errorf(msg) + return "Error:","kubeClient is nil",err1 + } + if kubeConfig == nil { + msg = "ExecCommand() : kubeConfig is nil" + err1 = fmt.Errorf(msg) + return "Error:","kubeConfig is nil",err1 + } + + msg = "" req := kubeClient.CoreV1().RESTClient(). Post(). Namespace(instance.Spec.Namespace). diff --git a/commons/sharding/gsm.go b/commons/sharding/gsm.go index 6ac1414f..bcdc8866 100644 --- a/commons/sharding/gsm.go +++ b/commons/sharding/gsm.go @@ -466,7 +466,7 @@ func UpdateProvForGsm(instance *databasev1alpha1.ShardingDatabase, oraSpexRes := OraGsmSpex.Resources if !reflect.DeepEqual(shardContaineRes, oraSpexRes) { - isUpdate = true + isUpdate = false } } } diff --git a/commons/sharding/scommon.go b/commons/sharding/scommon.go index d5d438b1..99987661 100644 --- a/commons/sharding/scommon.go +++ b/commons/sharding/scommon.go @@ -40,6 +40,7 @@ package commons import ( "context" + "encoding/json" "fmt" "slices" @@ -106,6 +107,8 @@ func buildEnvVarsSpec(instance *databasealphav1.ShardingDatabase, variables []da var result []corev1.EnvVar var varinfo string var sidFlag bool = false + //var sidValue string + var pdbValue string var pdbFlag bool = false var sDirectParam bool = false var sGroup1Params bool = false @@ -115,13 +118,17 @@ func buildEnvVarsSpec(instance *databasealphav1.ShardingDatabase, variables []da var oldSidFlag bool = false var archiveLogFlag bool = false var shardSetupFlag bool = false + var dbUnameFlag bool = false + var ofreePdbFlag bool = false for _, variable := range variables { if variable.Name == "ORACLE_SID" { sidFlag = true + //sidValue = variable.Value } if variable.Name == "ORACLE_PDB" { pdbFlag = true + pdbValue = variable.Value } if variable.Name == "SHARD_DIRECTOR_PARAMS" { sDirectParam = true @@ -144,9 +151,32 @@ func buildEnvVarsSpec(instance *databasealphav1.ShardingDatabase, variables []da if variable.Name == "OLD_ORACLE_PDB" { archiveLogFlag = true } + if variable.Name == "DB_UNIQUE_NAME" { + dbUnameFlag = true + } + if variable.Name == "ORACLE_FREE_PDB" { + ofreePdbFlag = true + } + result = append(result, corev1.EnvVar{Name: variable.Name, Value: variable.Value}) } + if !dbUnameFlag { + if strings.ToLower(instance.Spec.DbEdition) == "free" { + result = append(result, corev1.EnvVar{Name: "DB_UNIQUE_NAME", Value: strings.ToUpper(name)}) + } + } + + if !ofreePdbFlag { + if strings.ToLower(instance.Spec.DbEdition) == "free" { + if pdbFlag { + result = append(result, corev1.EnvVar{Name: "ORACLE_FREE_PDB", Value: pdbValue}) + } else { + result = append(result, corev1.EnvVar{Name: "ORACLE_FREE_PDB", Value: strings.ToUpper(name) + "PDB"}) + } + } + } + if !shardSetupFlag { if restype == "SHARD" { result = append(result, corev1.EnvVar{Name: "SHARD_SETUP", Value: "true"}) @@ -167,19 +197,27 @@ func buildEnvVarsSpec(instance *databasealphav1.ShardingDatabase, variables []da } } if !sidFlag { - if restype == "SHARD" { - result = append(result, corev1.EnvVar{Name: "ORACLE_SID", Value: strings.ToUpper(name)}) - } - if restype == "CATALOG" { - result = append(result, corev1.EnvVar{Name: "ORACLE_SID", Value: strings.ToUpper(name)}) + if strings.ToLower(instance.Spec.DbEdition) == "free" { + result = append(result, corev1.EnvVar{Name: "ORACLE_SID", Value: "FREE"}) + } else { + if restype == "SHARD" { + result = append(result, corev1.EnvVar{Name: "ORACLE_SID", Value: strings.ToUpper(name)}) + } + if restype == "CATALOG" { + result = append(result, corev1.EnvVar{Name: "ORACLE_SID", Value: strings.ToUpper(name)}) + } } } if !pdbFlag { - if restype == "SHARD" { - result = append(result, corev1.EnvVar{Name: "ORACLE_PDB", Value: strings.ToUpper(name) + "PDB"}) - } - if restype == "CATALOG" { - result = append(result, corev1.EnvVar{Name: "ORACLE_PDB", Value: strings.ToUpper(name) + "PDB"}) + if strings.ToLower(instance.Spec.DbEdition) == "free" { + result = append(result, corev1.EnvVar{Name: "ORACLE_PDB", Value: "FREEPDB"}) + } else { + if restype == "SHARD" { + result = append(result, corev1.EnvVar{Name: "ORACLE_PDB", Value: strings.ToUpper(name) + "PDB"}) + } + if restype == "CATALOG" { + result = append(result, corev1.EnvVar{Name: "ORACLE_PDB", Value: strings.ToUpper(name) + "PDB"}) + } } } // Secret Settings @@ -256,6 +294,10 @@ func buildEnvVarsSpec(instance *databasealphav1.ShardingDatabase, variables []da result = append(result, corev1.EnvVar{Name: "DEV_MODE", Value: "TRUE"}) } + if instance.Spec.InvitedNodeSubnetFlag == "" { + instance.Spec.InvitedNodeSubnetFlag = "FALSE" + + } if strings.ToUpper(instance.Spec.InvitedNodeSubnetFlag) != "FALSE" { result = append(result, corev1.EnvVar{Name: "INVITED_NODE_SUBNET_FLAG", Value: "TRUE"}) if instance.Spec.InvitedNodeSubnet != "" { @@ -727,13 +769,26 @@ func buildCatalogParams(instance *databasealphav1.ShardingDatabase) string { } for _, variable := range variables { - if variable.Name == "ORACLE_SID" { + if variable.Name == "DB_UNIQUE_NAME" { sidFlag = true sidName = variable.Value + } else { + if variable.Name == "ORACLE_SID" { + sidFlag = true + sidName = variable.Value + } } - if variable.Name == "ORACLE_PDB" { - pdbFlag = true - pdbName = variable.Value + if variable.Name == "ORACLE_FREE_PDB" { + if strings.ToLower(instance.Spec.DbEdition) == "free" { + pdbFlag = true + pdbName = variable.Value + } + } + if strings.ToLower(instance.Spec.DbEdition) != "free" { + if variable.Name == "ORACLE_PDB" { + pdbFlag = true + pdbName = variable.Value + } } if variable.Name == "CATALOG_PORT" { portFlag = true @@ -754,16 +809,26 @@ func buildCatalogParams(instance *databasealphav1.ShardingDatabase) string { varinfo = "catalog_db=" + strings.ToUpper(instance.Spec.Catalog[0].Name) + ";" result = result + varinfo } else { - varinfo = "catalog_db=" + strings.ToUpper(sidName) + ";" - result = result + varinfo + if strings.ToLower(instance.Spec.DbEdition) == "free" { + varinfo = "catalog_db=" + strings.ToUpper(instance.Spec.Catalog[0].Name) + ";" + result = result + varinfo + } else { + varinfo = "catalog_db=" + strings.ToUpper(sidName) + ";" + result = result + varinfo + } } if !pdbFlag { varinfo = "catalog_pdb=" + strings.ToUpper(instance.Spec.Catalog[0].Name) + "PDB" + ";" result = result + varinfo } else { - varinfo = "catalog_pdb=" + strings.ToUpper(pdbName) + ";" - result = result + varinfo + if strings.ToLower(instance.Spec.DbEdition) == "free" { + varinfo = "catalog_pdb=" + strings.ToUpper(instance.Spec.Catalog[0].Name) + "PDB" + ";" + result = result + varinfo + } else { + varinfo = "catalog_pdb=" + strings.ToUpper(pdbName) + ";" + result = result + varinfo + } } if !portFlag { @@ -781,8 +846,13 @@ func buildCatalogParams(instance *databasealphav1.ShardingDatabase) string { varinfo = "catalog_name=" + strings.ToUpper(cname) + ";" result = result + varinfo } + if chunksFlag { result = result + "catalog_chunks=" + catchunks + ";" + } else { + if strings.ToLower(instance.Spec.DbEdition) == "free" && strings.ToUpper(instance.Spec.ShardingType) != "USER" && strings.ToUpper(instance.Spec.ShardingType) != "NATIVE" { + result = result + "catalog_chunks=12;" + } } result = strings.TrimSuffix(result, ";") return result @@ -840,6 +910,15 @@ func BuildShardParams(instance *databasealphav1.ShardingDatabase, sfSet *appsv1. var result string var varinfo string var isShardPort bool = false + var freePdbFlag bool = false + var freePdbValue string + var pdbFlag bool = false + var pdbValue string + var dbUnameFlag bool = false + var sidFlag bool = false + var dbUname string + var sidName string + //var isShardGrp bool = false //var i int32 //var isShardSpace bool = false @@ -847,14 +926,25 @@ func BuildShardParams(instance *databasealphav1.ShardingDatabase, sfSet *appsv1. result = "shard_host=" + sfSet.Name + "-0" + "." + sfSet.Name + ";" for _, variable := range variables { - if variable.Name == "ORACLE_SID" { - varinfo = "shard_db=" + variable.Value + ";" - result = result + varinfo + if variable.Name == "DB_UNIQUE_NAME" { + dbUnameFlag = true + dbUname = variable.Value + } else { + if variable.Name == "ORACLE_SID" { + sidFlag = true + sidName = variable.Value + } } + if variable.Name == "ORACLE_FREE_PDB" { + freePdbFlag = true + freePdbValue = variable.Value + } + if variable.Name == "ORACLE_PDB" { - varinfo = "shard_pdb=" + variable.Value + ";" - result = result + varinfo + pdbFlag = true + pdbValue = variable.Value } + if variable.Name == "SHARD_PORT" { varinfo = "shard_port=" + variable.Value + ";" result = result + varinfo @@ -862,6 +952,41 @@ func BuildShardParams(instance *databasealphav1.ShardingDatabase, sfSet *appsv1. } } + + if dbUnameFlag { + varinfo = "shard_db=" + dbUname + ";" + result = result + varinfo + } + + if sidFlag && !dbUnameFlag { + if strings.ToLower(instance.Spec.DbEdition) != "free" { + varinfo = "shard_db=" + sidName + ";" + result = result + varinfo + } else { + varinfo = "shard_db=" + sfSet.Name + ";" + result = result + varinfo + } + } + + if !sidFlag && !dbUnameFlag { + if strings.ToLower(instance.Spec.DbEdition) != "free" { + varinfo = "shard_db=" + sfSet.Name + ";" + result = result + varinfo + } + } + + if freePdbFlag { + if strings.ToLower(instance.Spec.DbEdition) == "free" { + varinfo = "shard_pdb=" + freePdbValue + ";" + result = result + varinfo + } + } else { + if pdbFlag { + varinfo = "shard_pdb=" + pdbValue + ";" + result = result + varinfo + } + } + if OraShardSpex.ShardGroup != "" { varinfo = "shard_group=" + OraShardSpex.ShardGroup + ";" result = result + varinfo @@ -1205,7 +1330,7 @@ func CheckOnlineShardInGsm(gsmPodName string, sparams string, instance *database _, _, err := ExecCommand(gsmPodName, getOnlineShardCmd(sparams), kubeClient, kubeconfig, instance, logger) if err != nil { - msg := "Shard: " + GetFmtStr(sparams) + " is not onine in GSM." + msg := "Shard: " + GetFmtStr(sparams) + " is not online in GSM." LogMessages("INFO", msg, nil, instance, logger) return err } @@ -1355,6 +1480,28 @@ func SfsetLabelPatch(sfSetFound *appsv1.StatefulSet, sfSetPod *corev1.Pod, insta return nil } +func InstanceShardPatch(obj client.Object, instance *databasealphav1.ShardingDatabase, kClient client.Client, id int32, field string, value string, +) error { + + var err error + instSpec := instance.Spec + instSpec.Shard[id].IsDelete = "failed" + instshardM, _ := json.Marshal(struct { + Spec *databasealphav1.ShardingDatabaseSpec `json:"spec":` + }{ + Spec: &instSpec, + }) + + patch1 := client.RawPatch(types.MergePatchType, instshardM) + err = kClient.Patch(context.TODO(), obj, patch1) + + if err != nil { + return err + } + + return err +} + // Send Notification func SendNotification(title string, body string, instance *databasealphav1.ShardingDatabase, topicId string, rclient ons.NotificationDataPlaneClient, logger logr.Logger, @@ -1377,3 +1524,27 @@ func SendNotification(title string, body string, instance *databasealphav1.Shard func GetSecretMount() string { return oraSecretMount } + +func checkTdeWalletFlag(instance *databasev1alpha1.ShardingDatabase) bool { + if strings.ToLower(instance.Spec.IsTdeWallet) == "enable" { + return true + } + return false +} + +func CheckIsDeleteFlag(delStr string, instance *databasealphav1.ShardingDatabase, logger logr.Logger) bool { + if strings.ToLower(delStr) == "enable" { + return true + } + if strings.ToLower(delStr) == "failed" { + // LogMessages("INFO", "manual intervention required", nil, instance, logger) + } + return false +} + +func getTdeWalletMountLoc(instance *databasev1alpha1.ShardingDatabase) string { + if len(instance.Spec.TdeWalletPvcMountLocation) > 0 { + return instance.Spec.TdeWalletPvcMountLocation + } + return "/tdewallet/" + instance.Name +} diff --git a/commons/sharding/shard.go b/commons/sharding/shard.go index 6dd7a6ea..c76fc0e5 100644 --- a/commons/sharding/shard.go +++ b/commons/sharding/shard.go @@ -197,6 +197,13 @@ func buildVolumeSpecForShard(instance *databasev1alpha1.ShardingDatabase, OraSha if instance.Spec.IsDownloadScripts { result = append(result, corev1.Volume{Name: OraShardSpex.Name + "orascript-vol5", VolumeSource: corev1.VolumeSource{EmptyDir: &corev1.EmptyDirVolumeSource{}}}) } + + if checkTdeWalletFlag(instance) { + if len(instance.Spec.FssStorageClass) == 0 && len(instance.Spec.TdeWalletPvc) > 0 { + result = append(result, corev1.Volume{Name: OraShardSpex.Name + "shared-storage-vol8", VolumeSource: corev1.VolumeSource{PersistentVolumeClaim: &corev1.PersistentVolumeClaimVolumeSource{ClaimName: instance.Spec.TdeWalletPvc}}}) + } + } + return result } @@ -209,7 +216,7 @@ func buildContainerSpecForShard(instance *databasev1alpha1.ShardingDatabase, Ora Image: instance.Spec.DbImage, SecurityContext: &corev1.SecurityContext{ Capabilities: &corev1.Capabilities{ - Add: []corev1.Capability{"NET_RAW"}, + Add: []corev1.Capability{corev1.Capability("NET_ADMIN"), corev1.Capability("SYS_NICE")}, }, }, Resources: corev1.ResourceRequirements{ @@ -331,6 +338,16 @@ func buildVolumeMountSpecForShard(instance *databasev1alpha1.ShardingDatabase, O result = append(result, corev1.VolumeMount{Name: OraShardSpex.Name + "orastage-vol7", MountPath: oraStage}) } + if checkTdeWalletFlag(instance) { + if len(instance.Spec.FssStorageClass) > 0 && len(instance.Spec.TdeWalletPvc) == 0 { + result = append(result, corev1.VolumeMount{Name: instance.Name + "shared-storage" + instance.Spec.Catalog[0].Name + "-0", MountPath: getTdeWalletMountLoc(instance)}) + } else { + if len(instance.Spec.FssStorageClass) == 0 && len(instance.Spec.TdeWalletPvc) > 0 { + result = append(result, corev1.VolumeMount{Name: OraShardSpex.Name + "shared-storage-vol8", MountPath: getTdeWalletMountLoc(instance)}) + } + } + } + return result } @@ -463,7 +480,7 @@ func UpdateProvForShard(instance *databasev1alpha1.ShardingDatabase, OraShardSpe oraSpexRes := OraShardSpex.Resources if !reflect.DeepEqual(shardContaineRes, oraSpexRes) { - isUpdate = true + isUpdate = false } } } diff --git a/config/crd/bases/database.oracle.com_pdbs.yaml b/config/crd/bases/database.oracle.com_pdbs.yaml index 8e6cb94f..85af8c1b 100644 --- a/config/crd/bases/database.oracle.com_pdbs.yaml +++ b/config/crd/bases/database.oracle.com_pdbs.yaml @@ -119,6 +119,11 @@ spec: to plug in a PDB. This property is applicable when the Action property is PLUG but not required. type: boolean + assertivePdbDeletion: + description: turn on the assertive approach to delete pdb resource + kubectl delete pdb ..... automatically triggers the pluggable database + deletion + type: boolean cdbName: description: Name of the CDB type: string diff --git a/config/crd/bases/database.oracle.com_shardingdatabases.yaml b/config/crd/bases/database.oracle.com_shardingdatabases.yaml index 554ed506..641629a0 100644 --- a/config/crd/bases/database.oracle.com_shardingdatabases.yaml +++ b/config/crd/bases/database.oracle.com_shardingdatabases.yaml @@ -72,7 +72,7 @@ spec: a container image type: string isDelete: - type: boolean + type: string label: type: string name: @@ -148,6 +148,8 @@ spec: - name type: object type: array + dbEdition: + type: string dbImage: type: string dbImagePullSecret: @@ -177,6 +179,8 @@ spec: - name - pwdFileName type: object + fssStorageClass: + type: string gsm: items: description: GsmSpec defines the desired state of GsmSpec @@ -205,7 +209,7 @@ spec: a container image type: string isDelete: - type: boolean + type: string label: type: string name: @@ -395,7 +399,7 @@ spec: isExternalSvc: type: boolean isTdeWallet: - type: boolean + type: string liveinessCheckPeriod: type: integer namespace: @@ -454,7 +458,12 @@ spec: a container image type: string isDelete: - type: boolean + enum: + - enable + - disable + - failed + - force + type: string label: type: string name: @@ -550,6 +559,10 @@ spec: type: string storageClass: type: string + tdeWalletPvc: + type: string + tdeWalletPvcMountLocation: + type: string required: - catalog - dbImage diff --git a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml index acfa2bea..1c011e17 100644 --- a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml +++ b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml @@ -175,6 +175,23 @@ spec: type: integer replicas: type: integer + resources: + properties: + limits: + properties: + cpu: + type: string + memory: + type: string + type: object + requests: + properties: + cpu: + type: string + memory: + type: string + type: object + type: object serviceAccountName: type: string serviceAnnotations: diff --git a/config/database.oracle.com_DbcsSystem.yaml b/config/database.oracle.com_DbcsSystem.yaml new file mode 100644 index 00000000..e933d5a4 --- /dev/null +++ b/config/database.oracle.com_DbcsSystem.yaml @@ -0,0 +1,240 @@ + +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.6.1 + creationTimestamp: null + name: DbcsSystem.database.oracle.com +spec: + group: database.oracle.com + names: + kind: DbcsSystem + listKind: DbcsSystemList + plural: DbcsSystem + singular: dbcssystem + scope: Namespaced + versions: + - name: v1alpha1 + schema: + openAPIV3Schema: + description: DbcsSystem is the Schema for the dbcssystems API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: DbcsSystemSpec defines the desired state of DbcsSystem + properties: + dbSystem: + properties: + availabilityDomain: + type: string + backupSubnetId: + type: string + clusterName: + type: string + compartmentId: + type: string + cpuCoreCount: + type: integer + dbAdminPaswordSecret: + type: string + dbBackupConfig: + description: DB Backup COnfig Network Struct + properties: + autoBackupEnabled: + type: boolean + autoBackupWindow: + type: string + backupDestinationDetails: + type: string + recoveryWindowsInDays: + type: integer + type: object + dbDomain: + type: string + dbEdition: + type: string + dbName: + type: string + dbUniqueName: + type: string + dbVersion: + type: string + dbWorkload: + type: string + diskRedundancy: + type: string + displayName: + type: string + domain: + type: string + faultDomains: + items: + type: string + type: array + hostName: + type: string + initialDataStorageSizeInGB: + type: integer + kmsKeyId: + type: string + kmsKeyVersionId: + type: string + licenseModel: + type: string + nodeCount: + type: integer + pdbName: + type: string + privateIp: + type: string + shape: + type: string + sshPublicKeys: + items: + type: string + type: array + storageManagement: + type: string + subnetId: + type: string + tags: + additionalProperties: + type: string + type: object + tdeWalletPasswordSecret: + type: string + timeZone: + type: string + required: + - availabilityDomain + - compartmentId + - dbAdminPaswordSecret + - hostName + - shape + - sshPublicKeys + - subnetId + type: object + hardLink: + type: boolean + id: + type: string + ociConfigMap: + type: string + ociSecret: + type: string + required: + - ociConfigMap + type: object + status: + description: DbcsSystemStatus defines the observed state of DbcsSystem + properties: + availabilityDomain: + type: string + cpuCoreCount: + type: integer + dataStoragePercentage: + type: integer + dataStorageSizeInGBs: + type: integer + dbEdition: + type: string + dbInfo: + items: + description: DbcsSystemStatus defines the observed state of DbcsSystem + properties: + dbHomeId: + type: string + dbName: + type: string + dbUniqueName: + type: string + dbWorkload: + type: string + id: + type: string + type: object + type: array + displayName: + type: string + id: + type: string + licenseModel: + type: string + network: + properties: + clientSubnet: + type: string + domainName: + type: string + hostName: + type: string + listenerPort: + type: integer + networkSG: + type: string + scanDnsName: + type: string + vcnName: + type: string + type: object + nodeCount: + type: integer + recoStorageSizeInGB: + type: integer + shape: + type: string + state: + type: string + storageManagement: + type: string + subnetId: + type: string + timeZone: + type: string + workRequests: + items: + properties: + operationId: + type: string + operationType: + type: string + percentComplete: + type: string + timeAccepted: + type: string + timeFinished: + type: string + timeStarted: + type: string + required: + - operationId + - operationType + type: object + type: array + required: + - state + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] diff --git a/config/database.oracle.com_autonomouscontainerdatabases.yaml b/config/database.oracle.com_autonomouscontainerdatabases.yaml new file mode 100644 index 00000000..bac3a28c --- /dev/null +++ b/config/database.oracle.com_autonomouscontainerdatabases.yaml @@ -0,0 +1,117 @@ + +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.6.1 + creationTimestamp: null + name: autonomouscontainerdatabases.database.oracle.com +spec: + group: database.oracle.com + names: + kind: AutonomousContainerDatabase + listKind: AutonomousContainerDatabaseList + plural: autonomouscontainerdatabases + shortNames: + - acd + - acds + singular: autonomouscontainerdatabase + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .spec.displayName + name: DisplayName + type: string + - jsonPath: .status.lifecycleState + name: State + type: string + - jsonPath: .status.timeCreated + name: Created + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + description: AutonomousContainerDatabase is the Schema for the autonomouscontainerdatabases + API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: AutonomousContainerDatabaseSpec defines the desired state + of AutonomousContainerDatabase + properties: + action: + enum: + - SYNC + - RESTART + - TERMINATE + type: string + autonomousContainerDatabaseOCID: + description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster + Important: Run "make" to regenerate code after modifying this file' + type: string + autonomousExadataVMClusterOCID: + type: string + compartmentOCID: + type: string + displayName: + type: string + freeformTags: + additionalProperties: + type: string + type: object + hardLink: + default: false + type: boolean + ociConfig: + description: "*********************** *\tOCI config ***********************" + properties: + configMapName: + type: string + secretName: + type: string + type: object + patchModel: + description: 'AutonomousContainerDatabasePatchModelEnum Enum with + underlying type: string' + enum: + - RELEASE_UPDATES + - RELEASE_UPDATE_REVISIONS + type: string + type: object + status: + description: AutonomousContainerDatabaseStatus defines the observed state + of AutonomousContainerDatabase + properties: + lifecycleState: + description: 'INSERT ADDITIONAL STATUS FIELD - define observed state + of cluster Important: Run "make" to regenerate code after modifying + this file' + type: string + timeCreated: + type: string + required: + - lifecycleState + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] diff --git a/config/database.oracle.com_autonomousdatabasebackups.yaml b/config/database.oracle.com_autonomousdatabasebackups.yaml new file mode 100644 index 00000000..a5c37507 --- /dev/null +++ b/config/database.oracle.com_autonomousdatabasebackups.yaml @@ -0,0 +1,138 @@ + +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.6.1 + creationTimestamp: null + name: autonomousdatabasebackups.database.oracle.com +spec: + group: database.oracle.com + names: + kind: AutonomousDatabaseBackup + listKind: AutonomousDatabaseBackupList + plural: autonomousdatabasebackups + shortNames: + - adbbu + - adbbus + singular: autonomousdatabasebackup + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .status.lifecycleState + name: State + type: string + - jsonPath: .status.dbDisplayName + name: DB DisplayName + type: string + - jsonPath: .status.type + name: Type + type: string + - jsonPath: .status.timeStarted + name: Started + type: string + - jsonPath: .status.timeEnded + name: Ended + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + description: AutonomousDatabaseBackup is the Schema for the autonomousdatabasebackups + API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: AutonomousDatabaseBackupSpec defines the desired state of + AutonomousDatabaseBackup + properties: + autonomousDatabaseBackupOCID: + type: string + displayName: + type: string + isLongTermBackup: + type: boolean + ociConfig: + description: "*********************** *\tOCI config ***********************" + properties: + configMapName: + type: string + secretName: + type: string + type: object + retentionPeriodInDays: + type: integer + target: + description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster + Important: Run "make" to regenerate code after modifying this file' + properties: + k8sADB: + description: "*********************** *\tADB spec ***********************" + properties: + name: + type: string + type: object + ociADB: + properties: + ocid: + type: string + type: object + type: object + type: object + status: + description: AutonomousDatabaseBackupStatus defines the observed state + of AutonomousDatabaseBackup + properties: + autonomousDatabaseOCID: + type: string + compartmentOCID: + type: string + dbDisplayName: + type: string + dbName: + type: string + isAutomatic: + type: boolean + lifecycleState: + description: 'AutonomousDatabaseBackupLifecycleStateEnum Enum with + underlying type: string' + type: string + timeEnded: + type: string + timeStarted: + type: string + type: + description: 'AutonomousDatabaseBackupTypeEnum Enum with underlying + type: string' + type: string + required: + - autonomousDatabaseOCID + - compartmentOCID + - dbDisplayName + - dbName + - isAutomatic + - lifecycleState + - type + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] diff --git a/config/database.oracle.com_autonomousdatabaserestores.yaml b/config/database.oracle.com_autonomousdatabaserestores.yaml new file mode 100644 index 00000000..5e9f2c73 --- /dev/null +++ b/config/database.oracle.com_autonomousdatabaserestores.yaml @@ -0,0 +1,138 @@ + +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.6.1 + creationTimestamp: null + name: autonomousdatabaserestores.database.oracle.com +spec: + group: database.oracle.com + names: + kind: AutonomousDatabaseRestore + listKind: AutonomousDatabaseRestoreList + plural: autonomousdatabaserestores + shortNames: + - adbr + - adbrs + singular: autonomousdatabaserestore + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .status.status + name: Status + type: string + - jsonPath: .status.displayName + name: DbDisplayName + type: string + - jsonPath: .status.dbName + name: DbName + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + description: AutonomousDatabaseRestore is the Schema for the autonomousdatabaserestores + API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: AutonomousDatabaseRestoreSpec defines the desired state of + AutonomousDatabaseRestore + properties: + ociConfig: + description: "*********************** *\tOCI config ***********************" + properties: + configMapName: + type: string + secretName: + type: string + type: object + source: + properties: + k8sADBBackup: + description: 'EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO + OWN! NOTE: json tags are required. Any new fields you add must + have json tags for the fields to be serialized.' + properties: + name: + type: string + type: object + pointInTime: + properties: + timestamp: + description: 'The timestamp must follow this format: YYYY-MM-DD + HH:MM:SS GMT' + type: string + type: object + type: object + target: + description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster + Important: Run "make" to regenerate code after modifying this file' + properties: + k8sADB: + description: "*********************** *\tADB spec ***********************" + properties: + name: + type: string + type: object + ociADB: + properties: + ocid: + type: string + type: object + type: object + required: + - source + - target + type: object + status: + description: AutonomousDatabaseRestoreStatus defines the observed state + of AutonomousDatabaseRestore + properties: + dbName: + type: string + displayName: + description: 'INSERT ADDITIONAL STATUS FIELD - define observed state + of cluster Important: Run "make" to regenerate code after modifying + this file' + type: string + status: + description: 'WorkRequestStatusEnum Enum with underlying type: string' + type: string + timeAccepted: + type: string + timeEnded: + type: string + timeStarted: + type: string + workRequestOCID: + type: string + required: + - dbName + - displayName + - status + - workRequestOCID + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] diff --git a/config/database.oracle.com_autonomousdatabases.yaml b/config/database.oracle.com_autonomousdatabases.yaml new file mode 100644 index 00000000..f77407f3 --- /dev/null +++ b/config/database.oracle.com_autonomousdatabases.yaml @@ -0,0 +1,324 @@ + +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.6.1 + creationTimestamp: null + name: autonomousdatabases.database.oracle.com +spec: + group: database.oracle.com + names: + kind: AutonomousDatabase + listKind: AutonomousDatabaseList + plural: autonomousdatabases + shortNames: + - adb + - adbs + singular: autonomousdatabase + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .spec.details.displayName + name: Display Name + type: string + - jsonPath: .spec.details.dbName + name: Db Name + type: string + - jsonPath: .status.lifecycleState + name: State + type: string + - jsonPath: .spec.details.isDedicated + name: Dedicated + type: string + - jsonPath: .spec.details.cpuCoreCount + name: OCPUs + type: integer + - jsonPath: .spec.details.dataStorageSizeInTBs + name: Storage (TB) + type: integer + - jsonPath: .spec.details.dbWorkload + name: Workload Type + type: string + - jsonPath: .status.timeCreated + name: Created + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + description: AutonomousDatabase is the Schema for the autonomousdatabases + API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: 'AutonomousDatabaseSpec defines the desired state of AutonomousDatabase + Important: Run "make" to regenerate code after modifying this file' + properties: + details: + description: AutonomousDatabaseDetails defines the detail information + of AutonomousDatabase, corresponding to oci-go-sdk/database/AutonomousDatabase + properties: + adminPassword: + properties: + k8sSecret: + description: "*********************** *\tSecret specs ***********************" + properties: + name: + type: string + type: object + ociSecret: + properties: + ocid: + type: string + type: object + type: object + autonomousContainerDatabase: + description: ACDSpec defines the spec of the target for backup/restore + runs. The name could be the name of an AutonomousDatabase or + an AutonomousDatabaseBackup + properties: + k8sACD: + description: "*********************** *\tACD specs ***********************" + properties: + name: + type: string + type: object + ociACD: + properties: + ocid: + type: string + type: object + type: object + autonomousDatabaseOCID: + type: string + compartmentOCID: + type: string + cpuCoreCount: + type: integer + dataStorageSizeInTBs: + type: integer + dbName: + type: string + dbVersion: + type: string + dbWorkload: + description: 'AutonomousDatabaseDbWorkloadEnum Enum with underlying + type: string' + enum: + - OLTP + - DW + - AJD + - APEX + type: string + displayName: + type: string + freeformTags: + additionalProperties: + type: string + type: object + isAutoScalingEnabled: + type: boolean + isDedicated: + type: boolean + licenseModel: + description: 'AutonomousDatabaseLicenseModelEnum Enum with underlying + type: string' + enum: + - LICENSE_INCLUDED + - BRING_YOUR_OWN_LICENSE + type: string + lifecycleState: + description: 'AutonomousDatabaseLifecycleStateEnum Enum with underlying + type: string' + type: string + networkAccess: + properties: + accessControlList: + items: + type: string + type: array + accessType: + enum: + - "" + - PUBLIC + - RESTRICTED + - PRIVATE + type: string + isAccessControlEnabled: + type: boolean + isMTLSConnectionRequired: + type: boolean + privateEndpoint: + properties: + hostnamePrefix: + type: string + nsgOCIDs: + items: + type: string + type: array + subnetOCID: + type: string + type: object + type: object + wallet: + properties: + name: + type: string + password: + properties: + k8sSecret: + description: "*********************** *\tSecret specs + ***********************" + properties: + name: + type: string + type: object + ociSecret: + properties: + ocid: + type: string + type: object + type: object + type: object + type: object + hardLink: + default: false + type: boolean + ociConfig: + description: "*********************** *\tOCI config ***********************" + properties: + configMapName: + type: string + secretName: + type: string + type: object + required: + - details + type: object + status: + description: AutonomousDatabaseStatus defines the observed state of AutonomousDatabase + properties: + allConnectionStrings: + items: + properties: + connectionStrings: + items: + properties: + connectionString: + type: string + tnsName: + type: string + type: object + type: array + tlsAuthentication: + type: string + required: + - connectionStrings + type: object + type: array + conditions: + items: + description: "Condition contains details for one aspect of the current + state of this API Resource. --- This struct is intended for direct + use as an array at the field path .status.conditions. For example, + \n \ttype FooStatus struct{ \t // Represents the observations + of a foo's current state. \t // Known .status.conditions.type + are: \"Available\", \"Progressing\", and \"Degraded\" \t // + +patchMergeKey=type \t // +patchStrategy=merge \t // +listType=map + \t // +listMapKey=type \t Conditions []metav1.Condition + `json:\"conditions,omitempty\" patchStrategy:\"merge\" patchMergeKey:\"type\" + protobuf:\"bytes,1,rep,name=conditions\"` \n \t // other fields + \t}" + properties: + lastTransitionTime: + description: lastTransitionTime is the last time the condition + transitioned from one status to another. This should be when + the underlying condition changed. If that is not known, then + using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: message is a human readable message indicating + details about the transition. This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: observedGeneration represents the .metadata.generation + that the condition was set based upon. For instance, if .metadata.generation + is currently 12, but the .status.conditions[x].observedGeneration + is 9, the condition is out of date with respect to the current + state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: reason contains a programmatic identifier indicating + the reason for the condition's last transition. Producers + of specific condition types may define expected values and + meanings for this field, and whether the values are considered + a guaranteed API. The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + --- Many .condition.type values are consistent across resources + like Available, but because arbitrary conditions can be useful + (see .node.status.conditions), the ability to deconflict is + important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + lifecycleState: + description: 'INSERT ADDITIONAL STATUS FIELD - define observed state + of cluster Important: Run "make" to regenerate code after modifying + this file' + type: string + timeCreated: + type: string + walletExpiringDate: + type: string + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] diff --git a/config/database.oracle.com_cdbs.yaml b/config/database.oracle.com_cdbs.yaml new file mode 100644 index 00000000..6b1c350c --- /dev/null +++ b/config/database.oracle.com_cdbs.yaml @@ -0,0 +1,270 @@ + +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.6.1 + creationTimestamp: null + name: cdbs.database.oracle.com +spec: + group: database.oracle.com + names: + kind: CDB + listKind: CDBList + plural: cdbs + singular: cdb + scope: Namespaced + versions: + - additionalPrinterColumns: + - description: Name of the CDB + jsonPath: .spec.cdbName + name: CDB Name + type: string + - description: ' Name of the DB Server' + jsonPath: .spec.dbServer + name: DB Server + type: string + - description: DB server port + jsonPath: .spec.dbPort + name: DB Port + type: integer + - description: ' string of the tnsalias' + jsonPath: .spec.dbTnsurl + name: TNS STRING + type: string + - description: Replicas + jsonPath: .spec.replicas + name: Replicas + type: integer + - description: Status of the CDB Resource + jsonPath: .status.phase + name: Status + type: string + - description: Error message, if any + jsonPath: .status.msg + name: Message + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + description: CDB is the Schema for the cdbs API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: CDBSpec defines the desired state of CDB + properties: + cdbAdminPwd: + description: Password for the CDB Administrator to manage PDB lifecycle + properties: + secret: + description: CDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + cdbAdminUser: + description: User in the root container with sysdba priviledges to + manage PDB lifecycle + properties: + secret: + description: CDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + cdbName: + description: Name of the CDB + type: string + cdbTlsCrt: + properties: + secret: + description: CDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + cdbTlsKey: + properties: + secret: + description: CDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + dbPort: + description: DB server port + type: integer + dbServer: + description: Name of the DB server + type: string + dbTnsurl: + type: string + nodeSelector: + additionalProperties: + type: string + description: Node Selector for running the Pod + type: object + ordsImage: + description: ORDS Image Name + type: string + ordsImagePullPolicy: + description: ORDS Image Pull Policy + enum: + - Always + - Never + type: string + ordsImagePullSecret: + description: The name of the image pull secret in case of a private + docker repository. + type: string + ordsPort: + description: ORDS server port. For now, keep it as 8888. TO BE USED + IN FUTURE RELEASE. + type: integer + ordsPwd: + description: Password for user ORDS_PUBLIC_USER + properties: + secret: + description: CDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + replicas: + description: Number of ORDS Containers to create + type: integer + serviceName: + description: Name of the CDB Service + type: string + sysAdminPwd: + description: Password for the CDB System Administrator + properties: + secret: + description: CDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + webServerPwd: + description: Password for the Web Server User + properties: + secret: + description: CDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + webServerUser: + description: Web Server User with SQL Administrator role to allow + us to authenticate to the PDB Lifecycle Management REST endpoints + properties: + secret: + description: CDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + type: object + status: + description: CDBStatus defines the observed state of CDB + properties: + msg: + description: Message + type: string + phase: + description: Phase of the CDB Resource + type: string + status: + description: CDB Resource Status + type: boolean + required: + - phase + - status + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] diff --git a/config/database.oracle.com_dataguardbrokers.yaml b/config/database.oracle.com_dataguardbrokers.yaml new file mode 100644 index 00000000..f19a3e22 --- /dev/null +++ b/config/database.oracle.com_dataguardbrokers.yaml @@ -0,0 +1,134 @@ + +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.6.1 + creationTimestamp: null + name: dataguardbrokers.database.oracle.com +spec: + group: database.oracle.com + names: + kind: DataguardBroker + listKind: DataguardBrokerList + plural: dataguardbrokers + singular: dataguardbroker + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .status.primaryDatabase + name: Primary + type: string + - jsonPath: .status.standbyDatabases + name: Standbys + type: string + - jsonPath: .spec.protectionMode + name: Protection Mode + type: string + - jsonPath: .status.clusterConnectString + name: Cluster Connect Str + priority: 1 + type: string + - jsonPath: .status.externalConnectString + name: Connect Str + type: string + - jsonPath: .spec.primaryDatabaseRef + name: Primary Database + priority: 1 + type: string + - jsonPath: .status.status + name: Status + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + description: DataguardBroker is the Schema for the dataguardbrokers API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: DataguardBrokerSpec defines the desired state of DataguardBroker + properties: + fastStartFailOver: + properties: + enable: + type: boolean + strategy: + items: + description: FSFO strategy + properties: + sourceDatabaseRef: + type: string + targetDatabaseRefs: + type: string + type: object + type: array + type: object + loadBalancer: + type: boolean + nodeSelector: + additionalProperties: + type: string + type: object + primaryDatabaseRef: + type: string + protectionMode: + enum: + - MaxPerformance + - MaxAvailability + type: string + serviceAnnotations: + additionalProperties: + type: string + type: object + setAsPrimaryDatabase: + type: string + standbyDatabaseRefs: + items: + type: string + type: array + required: + - primaryDatabaseRef + - protectionMode + - standbyDatabaseRefs + type: object + status: + description: DataguardBrokerStatus defines the observed state of DataguardBroker + properties: + clusterConnectString: + type: string + externalConnectString: + type: string + primaryDatabase: + type: string + primaryDatabaseRef: + type: string + protectionMode: + type: string + standbyDatabases: + type: string + status: + type: string + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] diff --git a/config/database.oracle.com_oraclerestdataservices.yaml b/config/database.oracle.com_oraclerestdataservices.yaml new file mode 100644 index 00000000..121383fd --- /dev/null +++ b/config/database.oracle.com_oraclerestdataservices.yaml @@ -0,0 +1,224 @@ + +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.6.1 + creationTimestamp: null + name: oraclerestdataservices.database.oracle.com +spec: + group: database.oracle.com + names: + kind: OracleRestDataService + listKind: OracleRestDataServiceList + plural: oraclerestdataservices + singular: oraclerestdataservice + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .status.status + name: Status + type: string + - jsonPath: .spec.databaseRef + name: Database + type: string + - jsonPath: .status.databaseApiUrl + name: Database API URL + type: string + - jsonPath: .status.databaseActionsUrl + name: Database Actions URL + type: string + - jsonPath: .status.apexUrl + name: Apex URL + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + description: OracleRestDataService is the Schema for the oraclerestdataservices + API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: OracleRestDataServiceSpec defines the desired state of OracleRestDataService + properties: + adminPassword: + description: OracleRestDataServicePassword defines the secret containing + Password mapped to secretKey + properties: + keepSecret: + type: boolean + secretKey: + default: oracle_pwd + type: string + secretName: + type: string + required: + - secretName + type: object + apexPassword: + description: OracleRestDataServicePassword defines the secret containing + Password mapped to secretKey + properties: + keepSecret: + type: boolean + secretKey: + default: oracle_pwd + type: string + secretName: + type: string + required: + - secretName + type: object + databaseRef: + type: string + image: + description: OracleRestDataServiceImage defines the Image source and + pullSecrets for POD + properties: + pullFrom: + type: string + pullSecrets: + type: string + version: + type: string + required: + - pullFrom + type: object + loadBalancer: + type: boolean + nodeSelector: + additionalProperties: + type: string + type: object + oracleService: + type: string + ordsPassword: + description: OracleRestDataServicePassword defines the secret containing + Password mapped to secretKey + properties: + keepSecret: + type: boolean + secretKey: + default: oracle_pwd + type: string + secretName: + type: string + required: + - secretName + type: object + ordsUser: + type: string + persistence: + description: OracleRestDataServicePersistence defines the storage + releated params + properties: + accessMode: + enum: + - ReadWriteOnce + - ReadWriteMany + type: string + size: + type: string + storageClass: + type: string + volumeName: + type: string + type: object + replicas: + minimum: 1 + type: integer + restEnableSchemas: + items: + description: OracleRestDataServicePDBSchemas defines the PDB Schemas + to be ORDS Enabled + properties: + enable: + type: boolean + pdbName: + type: string + schemaName: + type: string + urlMapping: + type: string + required: + - enable + - schemaName + type: object + type: array + serviceAccountName: + type: string + serviceAnnotations: + additionalProperties: + type: string + type: object + required: + - adminPassword + - databaseRef + - ordsPassword + type: object + status: + description: OracleRestDataServiceStatus defines the observed state of + OracleRestDataService + properties: + apexConfigured: + type: boolean + apexUrl: + type: string + commonUsersCreated: + type: boolean + databaseActionsUrl: + type: string + databaseApiUrl: + type: string + databaseRef: + type: string + image: + description: OracleRestDataServiceImage defines the Image source and + pullSecrets for POD + properties: + pullFrom: + type: string + pullSecrets: + type: string + version: + type: string + required: + - pullFrom + type: object + loadBalancer: + type: string + ordsInstalled: + type: boolean + replicas: + type: integer + serviceIP: + type: string + status: + description: 'INSERT ADDITIONAL STATUS FIELD - define observed state + of cluster Important: Run "make" to regenerate code after modifying + this file' + type: string + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] diff --git a/config/database.oracle.com_pdbs.yaml b/config/database.oracle.com_pdbs.yaml new file mode 100644 index 00000000..85af8c1b --- /dev/null +++ b/config/database.oracle.com_pdbs.yaml @@ -0,0 +1,383 @@ + +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.6.1 + creationTimestamp: null + name: pdbs.database.oracle.com +spec: + group: database.oracle.com + names: + kind: PDB + listKind: PDBList + plural: pdbs + singular: pdb + scope: Namespaced + versions: + - additionalPrinterColumns: + - description: The connect string to be used + jsonPath: .status.connString + name: Connect_String + type: string + - description: Name of the CDB + jsonPath: .spec.cdbName + name: CDB Name + type: string + - description: Name of the PDB + jsonPath: .spec.pdbName + name: PDB Name + type: string + - description: PDB Open Mode + jsonPath: .status.openMode + name: PDB State + type: string + - description: Total Size of the PDB + jsonPath: .status.totalSize + name: PDB Size + type: string + - description: Status of the PDB Resource + jsonPath: .status.phase + name: Status + type: string + - description: Error message, if any + jsonPath: .status.msg + name: Message + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + description: PDB is the Schema for the pdbs API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: PDBSpec defines the desired state of PDB + properties: + action: + description: 'Action to be taken: Create/Clone/Plug/Unplug/Delete/Modify/Status/Map. + Map is used to map a Databse PDB to a Kubernetes PDB CR.' + enum: + - Create + - Clone + - Plug + - Unplug + - Delete + - Modify + - Status + - Map + type: string + adminName: + description: The administrator username for the new PDB. This property + is required when the Action property is Create. + properties: + secret: + description: PDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + adminPwd: + description: The administrator password for the new PDB. This property + is required when the Action property is Create. + properties: + secret: + description: PDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + asClone: + description: Indicate if 'AS CLONE' option should be used in the command + to plug in a PDB. This property is applicable when the Action property + is PLUG but not required. + type: boolean + assertivePdbDeletion: + description: turn on the assertive approach to delete pdb resource + kubectl delete pdb ..... automatically triggers the pluggable database + deletion + type: boolean + cdbName: + description: Name of the CDB + type: string + cdbNamespace: + description: CDB Namespace + type: string + cdbResName: + description: Name of the CDB Custom Resource that runs the ORDS container + type: string + copyAction: + description: To copy files or not while cloning a PDB + enum: + - COPY + - NOCOPY + - MOVE + type: string + dropAction: + description: Specify if datafiles should be removed or not. The value + can be INCLUDING or KEEP (default). + enum: + - INCLUDING + - KEEP + type: string + fileNameConversions: + description: Relevant for Create and Plug operations. As defined in + the Oracle Multitenant Database documentation. Values can be a + filename convert pattern or NONE. + type: string + getScript: + description: Whether you need the script only or execute the script + type: boolean + modifyOption: + description: Extra options for opening and closing a PDB + enum: + - IMMEDIATE + - NORMAL + - READ ONLY + - READ WRITE + - RESTRICTED + type: string + pdbName: + description: The name of the new PDB. Relevant for both Create and + Plug Actions. + type: string + pdbState: + description: The target state of the PDB + enum: + - OPEN + - CLOSE + type: string + pdbTlsCat: + properties: + secret: + description: PDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + pdbTlsCrt: + properties: + secret: + description: PDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + pdbTlsKey: + properties: + secret: + description: PDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + reuseTempFile: + description: Whether to reuse temp file + type: boolean + sourceFileNameConversions: + description: This property is required when the Action property is + Plug. As defined in the Oracle Multitenant Database documentation. + Values can be a source filename convert pattern or NONE. + type: string + sparseClonePath: + description: A Path specified for sparse clone snapshot copy. (Optional) + type: string + srcPdbName: + description: Name of the Source PDB from which to clone + type: string + tdeExport: + description: TDE export for unplug operations + type: boolean + tdeImport: + description: TDE import for plug operations + type: boolean + tdeKeystorePath: + description: TDE keystore path is required if the tdeImport or tdeExport + flag is set to true. Can be used in plug or unplug operations. + type: string + tdePassword: + description: TDE password if the tdeImport or tdeExport flag is set + to true. Can be used in create, plug or unplug operations + properties: + secret: + description: PDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + tdeSecret: + description: TDE secret is required if the tdeImport or tdeExport + flag is set to true. Can be used in plug or unplug operations. + properties: + secret: + description: PDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + tempSize: + description: Relevant for Create and Clone operations. Total size + for temporary tablespace as defined in the Oracle Multitenant Database + documentation. See size_clause description in Database SQL Language + Reference documentation. + type: string + totalSize: + description: Relevant for create and plug operations. Total size as + defined in the Oracle Multitenant Database documentation. See size_clause + description in Database SQL Language Reference documentation. + type: string + unlimitedStorage: + description: Relevant for Create and Plug operations. True for unlimited + storage. Even when set to true, totalSize and tempSize MUST be specified + in the request if Action is Create. + type: boolean + webServerPwd: + description: Password for the Web ServerPDB User + properties: + secret: + description: PDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + webServerUser: + description: Web Server User with SQL Administrator role to allow + us to authenticate to the PDB Lifecycle Management REST endpoints + properties: + secret: + description: PDBSecret defines the secretName + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + xmlFileName: + description: XML metadata filename to be used for Plug or Unplug operations + type: string + required: + - action + type: object + status: + description: PDBStatus defines the observed state of PDB + properties: + action: + description: Last Completed Action + type: string + connString: + description: PDB Connect String + type: string + modifyOption: + description: Modify Option of the PDB + type: string + msg: + description: Message + type: string + openMode: + description: Open mode of the PDB + type: string + phase: + description: Phase of the PDB Resource + type: string + status: + description: PDB Resource Status + type: boolean + totalSize: + description: Total size of the PDB + type: string + required: + - phase + - status + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] diff --git a/config/database.oracle.com_shardingdatabases.yaml b/config/database.oracle.com_shardingdatabases.yaml new file mode 100644 index 00000000..641629a0 --- /dev/null +++ b/config/database.oracle.com_shardingdatabases.yaml @@ -0,0 +1,688 @@ + +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.6.1 + creationTimestamp: null + name: shardingdatabases.database.oracle.com +spec: + group: database.oracle.com + names: + kind: ShardingDatabase + listKind: ShardingDatabaseList + plural: shardingdatabases + singular: shardingdatabase + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .status.gsm.state + name: Gsm State + type: string + - jsonPath: .status.gsm.services + name: Services + type: string + - jsonPath: .status.gsm.shards + name: shards + priority: 1 + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + description: ShardingDatabase is the Schema for the shardingdatabases API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: ShardingDatabaseSpec defines the desired state of ShardingDatabase + properties: + InvitedNodeSubnet: + type: string + catalog: + items: + description: CatalogSpec defines the desired state of CatalogSpec + properties: + envVars: + items: + description: EnvironmentVariable represents a named variable + accessible for containers. + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + imagePullPolicy: + description: PullPolicy describes a policy for if/when to pull + a container image + type: string + isDelete: + type: string + label: + type: string + name: + type: string + nodeSelector: + additionalProperties: + type: string + type: object + pvAnnotations: + additionalProperties: + type: string + type: object + pvMatchLabels: + additionalProperties: + type: string + type: object + pvcName: + type: string + resources: + description: ResourceRequirements describes the compute resource + requirements. + properties: + claims: + description: "Claims lists the names of resources, defined + in spec.resourceClaims, that are used by this container. + \n This is an alpha field and requires enabling the DynamicResourceAllocation + feature gate. \n This field is immutable. It can only + be set for containers." + items: + description: ResourceClaim references one entry in PodSpec.ResourceClaims. + properties: + name: + description: Name must match the name of one entry + in pod.spec.resourceClaims of the Pod where this + field is used. It makes that resource available + inside a container. + type: string + required: + - name + type: object + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + limits: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + description: 'Limits describes the maximum amount of compute + resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + type: object + requests: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + description: 'Requests describes the minimum amount of compute + resources required. If Requests is omitted for a container, + it defaults to Limits if that is explicitly specified, + otherwise to an implementation-defined value. Requests + cannot exceed Limits. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + type: object + type: object + storageSizeInGb: + format: int32 + type: integer + required: + - name + type: object + type: array + dbEdition: + type: string + dbImage: + type: string + dbImagePullSecret: + type: string + dbSecret: + description: Secret Details + properties: + encryptionType: + type: string + keyFileMountLocation: + type: string + keyFileName: + type: string + keySecretName: + type: string + name: + type: string + nsConfigMap: + type: string + nsSecret: + type: string + pwdFileMountLocation: + type: string + pwdFileName: + type: string + required: + - name + - pwdFileName + type: object + fssStorageClass: + type: string + gsm: + items: + description: GsmSpec defines the desired state of GsmSpec + properties: + directorName: + type: string + envVars: + description: Replicas int32 `json:"replicas,omitempty"` // + Gsm Replicas. If you set OraGsmPvcName then it is set default + to 1. + items: + description: EnvironmentVariable represents a named variable + accessible for containers. + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + imagePullPolicy: + description: PullPolicy describes a policy for if/when to pull + a container image + type: string + isDelete: + type: string + label: + type: string + name: + type: string + nodeSelector: + additionalProperties: + type: string + type: object + pvMatchLabels: + additionalProperties: + type: string + type: object + pvcName: + type: string + region: + type: string + resources: + description: ResourceRequirements describes the compute resource + requirements. + properties: + claims: + description: "Claims lists the names of resources, defined + in spec.resourceClaims, that are used by this container. + \n This is an alpha field and requires enabling the DynamicResourceAllocation + feature gate. \n This field is immutable. It can only + be set for containers." + items: + description: ResourceClaim references one entry in PodSpec.ResourceClaims. + properties: + name: + description: Name must match the name of one entry + in pod.spec.resourceClaims of the Pod where this + field is used. It makes that resource available + inside a container. + type: string + required: + - name + type: object + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + limits: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + description: 'Limits describes the maximum amount of compute + resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + type: object + requests: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + description: 'Requests describes the minimum amount of compute + resources required. If Requests is omitted for a container, + it defaults to Limits if that is explicitly specified, + otherwise to an implementation-defined value. Requests + cannot exceed Limits. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + type: object + type: object + storageSizeInGb: + format: int32 + type: integer + required: + - name + type: object + type: array + gsmDevMode: + type: string + gsmImage: + type: string + gsmImagePullSecret: + type: string + gsmService: + items: + description: Service Definition + properties: + available: + type: string + clbGoal: + type: string + commitOutcome: + type: string + drainTimeout: + type: string + dtp: + type: string + edition: + type: string + failoverDelay: + type: string + failoverMethod: + type: string + failoverPrimary: + type: string + failoverRestore: + type: string + failoverRetry: + type: string + failoverType: + type: string + gdsPool: + type: string + lag: + type: integer + locality: + type: string + name: + type: string + notification: + type: string + pdbName: + type: string + policy: + type: string + preferred: + type: string + prferredAll: + type: string + regionFailover: + type: string + retention: + type: string + role: + type: string + sessionState: + type: string + sqlTransactionProfile: + type: string + stopOption: + type: string + tableFamily: + type: string + tfaPolicy: + type: string + required: + - name + type: object + type: array + gsmShardGroup: + items: + properties: + deployAs: + type: string + name: + type: string + region: + type: string + required: + - name + type: object + type: array + gsmShardSpace: + items: + description: ShardSpace Specs + properties: + chunks: + type: integer + name: + type: string + protectionMode: + type: string + shardGroup: + type: string + required: + - name + type: object + type: array + invitedNodeSubnetFlag: + type: string + isClone: + type: boolean + isDataGuard: + type: boolean + isDebug: + type: boolean + isDeleteOraPvc: + type: boolean + isDownloadScripts: + type: boolean + isExternalSvc: + type: boolean + isTdeWallet: + type: string + liveinessCheckPeriod: + type: integer + namespace: + type: string + portMappings: + items: + description: PortMapping is a specification of port mapping for + an application deployment. + properties: + port: + format: int32 + type: integer + protocol: + default: TCP + type: string + targetPort: + format: int32 + type: integer + required: + - port + - protocol + - targetPort + type: object + type: array + readinessCheckPeriod: + type: integer + replicationType: + type: string + scriptsLocation: + type: string + shard: + description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster + Important: Run "make" to regenerate code after modifying this file' + items: + description: ShardSpec is a specification of Shards for an application + deployment. + properties: + deployAs: + type: string + envVars: + items: + description: EnvironmentVariable represents a named variable + accessible for containers. + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + imagePullPolicy: + description: PullPolicy describes a policy for if/when to pull + a container image + type: string + isDelete: + enum: + - enable + - disable + - failed + - force + type: string + label: + type: string + name: + type: string + nodeSelector: + additionalProperties: + type: string + type: object + pvAnnotations: + additionalProperties: + type: string + type: object + pvMatchLabels: + additionalProperties: + type: string + type: object + pvcName: + type: string + resources: + description: ResourceRequirements describes the compute resource + requirements. + properties: + claims: + description: "Claims lists the names of resources, defined + in spec.resourceClaims, that are used by this container. + \n This is an alpha field and requires enabling the DynamicResourceAllocation + feature gate. \n This field is immutable. It can only + be set for containers." + items: + description: ResourceClaim references one entry in PodSpec.ResourceClaims. + properties: + name: + description: Name must match the name of one entry + in pod.spec.resourceClaims of the Pod where this + field is used. It makes that resource available + inside a container. + type: string + required: + - name + type: object + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + limits: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + description: 'Limits describes the maximum amount of compute + resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + type: object + requests: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + description: 'Requests describes the minimum amount of compute + resources required. If Requests is omitted for a container, + it defaults to Limits if that is explicitly specified, + otherwise to an implementation-defined value. Requests + cannot exceed Limits. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + type: object + type: object + shardGroup: + type: string + shardRegion: + type: string + shardSpace: + type: string + storageSizeInGb: + format: int32 + type: integer + required: + - name + type: object + type: array + shardBuddyRegion: + type: string + shardConfigName: + type: string + shardRegion: + items: + type: string + type: array + shardingType: + type: string + stagePvcName: + type: string + storageClass: + type: string + tdeWalletPvc: + type: string + tdeWalletPvcMountLocation: + type: string + required: + - catalog + - dbImage + - gsm + - gsmImage + - shard + type: object + status: + description: To understand Metav1.Condition, please refer the link https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1 + ShardingDatabaseStatus defines the observed state of ShardingDatabase + properties: + catalogs: + additionalProperties: + type: string + type: object + conditions: + items: + description: "Condition contains details for one aspect of the current + state of this API Resource. --- This struct is intended for direct + use as an array at the field path .status.conditions. For example, + \n \ttype FooStatus struct{ \t // Represents the observations + of a foo's current state. \t // Known .status.conditions.type + are: \"Available\", \"Progressing\", and \"Degraded\" \t // + +patchMergeKey=type \t // +patchStrategy=merge \t // +listType=map + \t // +listMapKey=type \t Conditions []metav1.Condition + `json:\"conditions,omitempty\" patchStrategy:\"merge\" patchMergeKey:\"type\" + protobuf:\"bytes,1,rep,name=conditions\"` \n \t // other fields + \t}" + properties: + lastTransitionTime: + description: lastTransitionTime is the last time the condition + transitioned from one status to another. This should be when + the underlying condition changed. If that is not known, then + using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: message is a human readable message indicating + details about the transition. This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: observedGeneration represents the .metadata.generation + that the condition was set based upon. For instance, if .metadata.generation + is currently 12, but the .status.conditions[x].observedGeneration + is 9, the condition is out of date with respect to the current + state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: reason contains a programmatic identifier indicating + the reason for the condition's last transition. Producers + of specific condition types may define expected values and + meanings for this field, and whether the values are considered + a guaranteed API. The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + --- Many .condition.type values are consistent across resources + like Available, but because arbitrary conditions can be useful + (see .node.status.conditions), the ability to deconflict is + important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + gsm: + properties: + details: + additionalProperties: + type: string + type: object + externalConnectStr: + type: string + internalConnectStr: + type: string + services: + type: string + shards: + additionalProperties: + type: string + type: object + state: + type: string + type: object + shards: + additionalProperties: + type: string + type: object + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] diff --git a/config/database.oracle.com_singleinstancedatabases.yaml b/config/database.oracle.com_singleinstancedatabases.yaml new file mode 100644 index 00000000..1c011e17 --- /dev/null +++ b/config/database.oracle.com_singleinstancedatabases.yaml @@ -0,0 +1,421 @@ + +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.6.1 + creationTimestamp: null + name: singleinstancedatabases.database.oracle.com +spec: + group: database.oracle.com + names: + kind: SingleInstanceDatabase + listKind: SingleInstanceDatabaseList + plural: singleinstancedatabases + singular: singleinstancedatabase + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .status.edition + name: Edition + type: string + - jsonPath: .status.sid + name: Sid + priority: 1 + type: string + - jsonPath: .status.status + name: Status + type: string + - jsonPath: .status.role + name: Role + type: string + - jsonPath: .status.releaseUpdate + name: Version + type: string + - jsonPath: .status.connectString + name: Connect Str + type: string + - jsonPath: .status.pdbConnectString + name: Pdb Connect Str + priority: 1 + type: string + - jsonPath: .status.tcpsConnectString + name: TCPS Connect Str + type: string + - jsonPath: .status.tcpsPdbConnectString + name: TCPS Pdb Connect Str + priority: 1 + type: string + - jsonPath: .status.oemExpressUrl + name: Oem Express Url + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + description: SingleInstanceDatabase is the Schema for the singleinstancedatabases + API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: SingleInstanceDatabaseSpec defines the desired state of SingleInstanceDatabase + properties: + adminPassword: + description: SingleInsatnceAdminPassword defines the secret containing + Admin Password mapped to secretKey for Database + properties: + keepSecret: + type: boolean + secretKey: + default: oracle_pwd + type: string + secretName: + type: string + required: + - secretName + type: object + archiveLog: + type: boolean + charset: + type: string + createAs: + enum: + - primary + - standby + - clone + type: string + dgBrokerConfigured: + type: boolean + edition: + enum: + - standard + - enterprise + - express + - free + type: string + enableTCPS: + type: boolean + flashBack: + type: boolean + forceLog: + type: boolean + image: + description: SingleInstanceDatabaseImage defines the Image source + and pullSecrets for POD + properties: + prebuiltDB: + type: boolean + pullFrom: + type: string + pullSecrets: + type: string + version: + type: string + required: + - pullFrom + type: object + initParams: + description: SingleInstanceDatabaseInitParams defines the Init Parameters + properties: + cpuCount: + type: integer + pgaAggregateTarget: + type: integer + processes: + type: integer + sgaTarget: + type: integer + type: object + listenerPort: + type: integer + loadBalancer: + type: boolean + nodeSelector: + additionalProperties: + type: string + type: object + pdbName: + type: string + persistence: + description: SingleInstanceDatabasePersistence defines the storage + size and class for PVC + properties: + accessMode: + enum: + - ReadWriteOnce + - ReadWriteMany + type: string + datafilesVolumeName: + type: string + scriptsVolumeName: + type: string + setWritePermissions: + type: boolean + size: + type: string + storageClass: + type: string + volumeClaimAnnotation: + type: string + type: object + primaryDatabaseRef: + type: string + readinessCheckPeriod: + type: integer + replicas: + type: integer + resources: + properties: + limits: + properties: + cpu: + type: string + memory: + type: string + type: object + requests: + properties: + cpu: + type: string + memory: + type: string + type: object + type: object + serviceAccountName: + type: string + serviceAnnotations: + additionalProperties: + type: string + type: object + sid: + description: SID must be alphanumeric (no special characters, only + a-z, A-Z, 0-9), and no longer than 12 characters. + maxLength: 12 + pattern: ^[a-zA-Z0-9]+$ + type: string + tcpsCertRenewInterval: + type: string + tcpsListenerPort: + type: integer + tcpsTlsSecret: + type: string + required: + - image + type: object + status: + description: SingleInstanceDatabaseStatus defines the observed state of + SingleInstanceDatabase + properties: + apexInstalled: + type: boolean + archiveLog: + type: string + certCreationTimestamp: + type: string + certRenewInterval: + type: string + charset: + type: string + clientWalletLoc: + type: string + clusterConnectString: + type: string + conditions: + items: + description: "Condition contains details for one aspect of the current + state of this API Resource. --- This struct is intended for direct + use as an array at the field path .status.conditions. For example, + \n \ttype FooStatus struct{ \t // Represents the observations + of a foo's current state. \t // Known .status.conditions.type + are: \"Available\", \"Progressing\", and \"Degraded\" \t // + +patchMergeKey=type \t // +patchStrategy=merge \t // +listType=map + \t // +listMapKey=type \t Conditions []metav1.Condition + `json:\"conditions,omitempty\" patchStrategy:\"merge\" patchMergeKey:\"type\" + protobuf:\"bytes,1,rep,name=conditions\"` \n \t // other fields + \t}" + properties: + lastTransitionTime: + description: lastTransitionTime is the last time the condition + transitioned from one status to another. This should be when + the underlying condition changed. If that is not known, then + using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: message is a human readable message indicating + details about the transition. This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: observedGeneration represents the .metadata.generation + that the condition was set based upon. For instance, if .metadata.generation + is currently 12, but the .status.conditions[x].observedGeneration + is 9, the condition is out of date with respect to the current + state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: reason contains a programmatic identifier indicating + the reason for the condition's last transition. Producers + of specific condition types may define expected values and + meanings for this field, and whether the values are considered + a guaranteed API. The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + --- Many .condition.type values are consistent across resources + like Available, but because arbitrary conditions can be useful + (see .node.status.conditions), the ability to deconflict is + important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + connectString: + type: string + createdAs: + type: string + datafilesCreated: + default: "false" + type: string + datafilesPatched: + default: "false" + type: string + dgBrokerConfigured: + type: boolean + edition: + type: string + flashBack: + type: string + forceLog: + type: string + initParams: + description: SingleInstanceDatabaseInitParams defines the Init Parameters + properties: + cpuCount: + type: integer + pgaAggregateTarget: + type: integer + processes: + type: integer + sgaTarget: + type: integer + type: object + initPgaSize: + type: integer + initSgaSize: + type: integer + isTcpsEnabled: + default: false + type: boolean + nodes: + items: + type: string + type: array + oemExpressUrl: + type: string + ordsReference: + type: string + pdbConnectString: + type: string + pdbName: + type: string + persistence: + description: SingleInstanceDatabasePersistence defines the storage + size and class for PVC + properties: + accessMode: + enum: + - ReadWriteOnce + - ReadWriteMany + type: string + datafilesVolumeName: + type: string + scriptsVolumeName: + type: string + setWritePermissions: + type: boolean + size: + type: string + storageClass: + type: string + volumeClaimAnnotation: + type: string + type: object + prebuiltDB: + type: boolean + primaryDatabase: + type: string + releaseUpdate: + type: string + replicas: + type: integer + role: + type: string + sid: + type: string + standbyDatabases: + additionalProperties: + type: string + type: object + status: + type: string + tcpsConnectString: + type: string + tcpsPdbConnectString: + type: string + tcpsTlsSecret: + default: "" + type: string + required: + - isTcpsEnabled + - persistence + - tcpsTlsSecret + type: object + type: object + served: true + storage: true + subresources: + scale: + specReplicasPath: .spec.replicas + statusReplicasPath: .status.replicas + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] diff --git a/config/manager/kustomization.yaml b/config/manager/kustomization.yaml index e7ed68ce..2aed83d4 100644 --- a/config/manager/kustomization.yaml +++ b/config/manager/kustomization.yaml @@ -8,5 +8,5 @@ apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization images: - name: controller - newName: phx.ocir.io/intsanjaysingh/db-repo/oracle/database - newTag: sharding-operator + newName: container-registry.oracle.com/database/operator + newTag: latest diff --git a/config/manifests/bases/oracle-database-operator.clusterserviceversion.yaml b/config/manifests/bases/oracle-database-operator.clusterserviceversion.yaml index 6ec37dd1..23cd7c00 100644 --- a/config/manifests/bases/oracle-database-operator.clusterserviceversion.yaml +++ b/config/manifests/bases/oracle-database-operator.clusterserviceversion.yaml @@ -3,25 +3,98 @@ kind: ClusterServiceVersion metadata: annotations: alm-examples: '[]' - capabilities: Basic Install + capabilities: Seamless Upgrades operators.operatorframework.io/builder: operator-sdk-v1.2.0 operators.operatorframework.io/project_layout: go.kubebuilder.io/v2 - name: oracle-database-operator.v0.0.0 - namespace: placeholder + name: oracle-database-operator.v1.1.0 + namespace: oracle-database-operator-system spec: apiservicedefinitions: {} customresourcedefinitions: owned: + - description: DbcsSystem is the Schema for the dbcssystems API + displayName: Dbcs System + kind: DbcsSystem + name: DbcsSystem.database.oracle.com + version: v1alpha1 + - description: AutonomousContainerDatabase is the Schema for the autonomouscontainerdatabases + API + displayName: Autonomous Container Database + kind: AutonomousContainerDatabase + name: autonomouscontainerdatabases.database.oracle.com + version: v1alpha1 + - description: AutonomousDatabaseBackup is the Schema for the autonomousdatabasebackups + API + displayName: Autonomous Database Backup + kind: AutonomousDatabaseBackup + name: autonomousdatabasebackups.database.oracle.com + version: v1alpha1 + - description: AutonomousDatabaseRestore is the Schema for the autonomousdatabaserestores + API + displayName: Autonomous Database Restore + kind: AutonomousDatabaseRestore + name: autonomousdatabaserestores.database.oracle.com + version: v1alpha1 - description: AutonomousDatabase is the Schema for the autonomousdatabases API displayName: Autonomous Database kind: AutonomousDatabase name: autonomousdatabases.database.oracle.com version: v1alpha1 - description: Operator to manage Oracle sharding - displayName: Oracle Sharding DB Operator + - description: CDB is the Schema for the cdbs API + displayName: CDB + kind: CDB + name: cdbs.database.oracle.com + version: v1alpha1 + - description: DatabaseObserver is the Schema for the databaseobservers API + displayName: Database Observer + kind: DatabaseObserver + name: databaseobservers.observability.oracle.com + version: v1alpha1 + - description: DataguardBroker is the Schema for the dataguardbrokers API + displayName: Dataguard Broker + kind: DataguardBroker + name: dataguardbrokers.database.oracle.com + version: v1alpha1 + - description: OracleRestDataService is the Schema for the oraclerestdataservices + API + displayName: Oracle Rest Data Service + kind: OracleRestDataService + name: oraclerestdataservices.database.oracle.com + version: v1alpha1 + - description: PDB is the Schema for the pdbs API + displayName: PDB + kind: PDB + name: pdbs.database.oracle.com + version: v1alpha1 + - description: ShardingDatabase is the Schema for the shardingdatabases API + displayName: Sharding Database + kind: ShardingDatabase + name: shardingdatabases.database.oracle.com + version: v1alpha1 + - description: SingleInstanceDatabase is the Schema for the singleinstancedatabases + API + displayName: Single Instance Database + kind: SingleInstanceDatabase + name: singleinstancedatabases.database.oracle.com + version: v1alpha1 + description: | + As part of Oracle's resolution to make Oracle Database Kubernetes native (that is, observable and operable by Kubernetes), Oracle released Oracle Database Operator for Kubernetes (OraOperator or the operator). OraOperator extends the Kubernetes API with custom resources and controllers for automating Oracle Database lifecycle management. + In this v1.1.0 production release, OraOperator supports the following database configurations and infrastructure: + ## Oracle Autonomous Database: + * Oracle Autonomous Database shared Oracle Cloud Infrastructure (OCI) (ADB-S) + * Oracle Autonomous Database on dedicated Cloud infrastructure (ADB-D) + * Oracle Autonomous Container Database (ACD) (infrastructure) is the infrastructure for provisioning Autonomous Databases. + * Containerized Single Instance databases (SIDB) deployed in the Oracle Kubernetes Engine (OKE) and any k8s where OraOperator is deployed + * Containerized Sharded databases (SHARDED) deployed in OKE and any k8s where OraOperator is deployed + * Oracle Multitenant Databases (CDB/PDBs) + * Oracle Base Database Cloud Service (BDBCS) + * Oracle Data Guard (Preview status) + * Oracle Database Observability (Preview status) + * Oracle will continue to extend OraOperator to support additional Oracle Database configurations. + displayName: Oracle Database Operator icon: - - base64data: "" - mediatype: "" + - base64data: iVBORw0KGgoAAAANSUhEUgAAALQAAAC0CAYAAAA9zQYyAAAABGdBTUEAALGPC/xhBQAAACBjSFJNAAB6JgAAgIQAAPoAAACA6AAAdTAAAOpgAAA6mAAAF3CculE8AAAABmJLR0QAAAAAAAD5Q7t/AAAJjUlEQVR42u3cfcwcRQHH8S9PH0BokZfCVBgpOgjyFjRoQIQQkLeA0PLWqgQMFDVgja9AChIKKCEKSgQEQVsQJGKxtNCAvAi2vJiCqAQMUpQRMKM4vFiCQEUo/jH7kOt19m7vbveK8fdJLukzMzuzczc7OzszWxAREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREZH/X2tVSRStmwi8B5gErN1nWS8DAVhmgl9ZsdwpVc+xoteKc/iTCf7VujKN1o0A+xef5cDPTfCP1XjeY+VsAWwFTATGDZjdPSb4F6J1U9sjTPA31n3uXeq1MfBe4F30376ADo0lWjcBOAa4EHhHzXW4FzgDuNsE/2aHc3gJmFBz2WMuBc4ywT87SCbRunWAB4APtEV92gR/zaAnGa2bBJwInFVz/SeY4F+O1q32/Zvg6+xEyuq1PnA08F1gg7ryHSkpbDrwEnAZ9TdmgD2BxcDj0bptG8i/is8DMVq394D5zGb1xgxwdbRum34zjdaNi9adAjxD/Y15PxP8yzXn2UvdDiPdsX9IjY0ZOHOVKzFaNw64Apgx5DoelrvNNdxDt9rDBP/rXg+K1u0EPNIhyWPADp3uQiX5bggsIX+hDOp6E/z0lrKG1kMXQ7OLgJkNZL8SWO+tHjpatxbwM4bfmAEWRuuOWAPljrmvGGJVVlz8d3ZJth1wXI/5jgeW0UxjhnRnGrqifV1FM40ZYDcT/GujLQGnAUd2OGAu6Qd8oY/CxgGTgWOB3UvSzI/W7WiCf7RLXgcNUOnJwOUlcScCF/SQ10mAqZBubrTuVhP837sljNZB6lQmdUh2AbCUdMvu1Ssm+Of6OK4OXyb9/mV+DNxOf+0LE/yDUDwURuu2A/5YknYWcLEJ/pU6ahWt2wq4FtgjE/08YMZmQUqGHHua4O8boPx1gNuAvTPRI1WGB0UdnsxEPQzsnAm/HTjQBN8t36nAwpLoTwDzTfBv9Fv3kjIbH3JE6xzwREn0bOA7dY3px4Ycl5bEH2CC/1ZdjRnABP8UsBfwg0z0RODgusoqKf810uxNzsbdjm/pRXOOIP1A7Q4ADu2S7yjljXl7E/y8uhvzEF1YEj7FBH9OnQ+oI9G6zYF9MnEzTfB3NFG7ogf+ApDrsr7XRJlt/lYSvmGFY6cDu2XCZ5ngnwC+XXLcTcXDXpmyodheTcxpD0u0bjNgSibqZBP8orrLGyH1HjlXNFnRorc5KhPlioWcJo2WhL/e6aBo3abke+eVFL2QCX4F+eEMpDnXMsdnwu4ywd/T8HfRtLLv4pImChsFDsmEn2aCf73XzPrwUEn4+4Gep9F68LGS8G4PTN8vCf9oMZQBwAS/JFp3PTCtLd2MaN3ckmeAXIP+ZoPfwbCUDSFXFMO3Om03QlrkaLd4GDUtHsByPd7WTZUZrdsLuDUTdX+n5fBo3b6k4Ua7K0zw92fCy6an7o3WVV2seqRiurez3QfPopI5Jvhlo+SniPqaOunT05mwjmPZaN2WwKPAmy0fOvw99u/NOmR7aofyxgO/7OU4E/yz0brjSHOv7c4oPt3U9jC+Bk0aPItKvgZpDJ27zVZ5OKrLFpmwlzodYIL/K2k+egPgncX5bghsRJqp2KT4TAQ2LT6dGvONJvi7O8R/oyR8qgn+xQ7HXUP+wffrxSpjN+v195W+rQxj3vuQsd9hhDRJ326P3vLrTzGGOjoT9Zdux5rg7yWtxA3qsZJzGDvHXYCvZKIWAzd1OceVlE/X3VmsNnayfQ31W9MeaDj/W0zwN4/9MQosYvUv/YJo3UVVt3kOYAfyO/4qTVOZ4JcVu9EeAjbvo/xLSNNH/85FRuvWBsp67guBKRUfbM4jrcSucvqk1caxp/3rgE+2pfkqaWfi/7JfkO8w1m5i4mGUtGrWbhzwKdKKXiOKtf0rM1HPAbFqPib4GK17H2lRYv8Kh7xIWkj6kQned0n7JWB8JnwmaSfiFgzm4mjdomKxaQ6rN+jDo3UfMsH/dsBy1qSy/S4zaGBqeMQE/zTwu0zcT6J1u/WaYRVFr3YOsGsmema3JeJ2xUrmQcDFHZKdU9R3IxP86d0ac7Rua+D8XBRpP0iVi6eKecXFXXYneDBaN7mmsoau2MOyJBN1eTHjVKuxpe/PlMQvjdadUGGsV1mxcjSP8qf8Bf3ka4J/wwT/RdImmJwzSbv6uj5oFQ1sYUn0fkVZjwJn1/CV7ApM77Ik/1S0rq4LaE0o2+G3JFp3UjG0q8Vb49do3fnAyR3SngvcQ3rFqFfjAEtaaJjWId2uJvjftJxTX5uTonWHAzeURQMf7LT7LVp3LHB1Jup8E/ypLenWBVb08X3kbEbanLWU/J0L0uzP2aQ76qBTeq+a4B/ObU4CPlJTnVodTOpUypxH6smXD1JIa4MeBX5FfqFlGGaY4FcZUw+y264YLi3tkGQXE/zvM8dNIr0lkjO+faNWtG5n0sXai1syYfNN8EdF6zYhNeymHWmCv6GkQdftKtIo4GbgwCYLan9jZV1gPvDxIVSy1fEm+KvaAwfdPlqMg//cIcnhJviFbccsIr8dYD8TfLcN/ZVE6y4HPpeJOsAEf0dxUf2BNH/ehMXAPiZ4htSgNzbBLy+27l5Lfg9PLVZ5p7CYvjqU4b3V8B/gw7nGXIdi99umlO/FXRCtmzU29RatO4h8Y15QV2MuzCoJvz1aN8EE/w9gS3p74aAXx/T64D2AKSb45fDW1t1plD+zDWrf1V6SNcG/aYK/jDSmm917npU8Tdo7vH7TU1Im+OeBnShfBDkP+Gm0bgPyQwFIb7PUeU7/pHwx59wizQoT/CmAI793vF+fNcGHlr+P7zun7m4jrXO01h0T/BzSKu7pNZZ1nQn+rqr/L8dkYFvSvGu/T6T/Ap4CHjfBV9orEq07IRN8pwn+yV4LL17Q7PTj3UV+F95zTfw/FcVMSvb9zeIHz53/1sA2pEWZfmeermxfMIvW7Uh6EB3pL8tSN5vgn+mWqNibsw3wbvpvXwuqtisREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREREWnxX2ox1/vZSvwPAAAAJXRFWHRkYXRlOmNyZWF0ZQAyMDI0LTA4LTEzVDE5OjUyOjMxKzAwOjAwsDIMcAAAACV0RVh0ZGF0ZTptb2RpZnkAMjAyNC0wOC0xM1QxOTo1MjozMSswMDowMMFvtMwAAABVdEVYdHN2Zzpjb21tZW50ACBVcGxvYWRlZCB0bzogU1ZHIFJlcG8sIHd3dy5zdmdyZXBvLmNvbSwgR2VuZXJhdG9yOiBTVkcgUmVwbyBNaXhlciBUb29scyBFB1wTAAAAAElFTkSuQmCC + mediatype: png install: spec: deployments: null @@ -37,12 +110,12 @@ spec: type: AllNamespaces keywords: - Oracle - - sharding - - db + - Database + - Operator links: - name: Oracle Database Operator - url: https://oracle-database-operator.domain + url: https://github.com/oracle/oracle-database-operator maturity: alpha provider: - name: ShardingDatabase - version: 0.0.0 + name: Oracle + version: 1.2.0 diff --git a/config/observability.oracle.com_databaseobservers.yaml b/config/observability.oracle.com_databaseobservers.yaml new file mode 100644 index 00000000..b0801738 --- /dev/null +++ b/config/observability.oracle.com_databaseobservers.yaml @@ -0,0 +1,227 @@ + +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.6.1 + creationTimestamp: null + name: databaseobservers.observability.oracle.com +spec: + group: observability.oracle.com + names: + kind: DatabaseObserver + listKind: DatabaseObserverList + plural: databaseobservers + singular: databaseobserver + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .status.exporterConfig + name: ExporterConfig + type: string + - jsonPath: .status.status + name: Status + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + description: DatabaseObserver is the Schema for the databaseobservers API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: DatabaseObserverSpec defines the desired state of DatabaseObserver + properties: + database: + description: DatabaseObserverDatabase defines the database details + used for DatabaseObserver + properties: + dbConnectionString: + properties: + key: + type: string + secret: + type: string + type: object + dbPassword: + properties: + key: + type: string + secret: + type: string + vaultOCID: + type: string + vaultSecretName: + type: string + type: object + dbUser: + properties: + key: + type: string + secret: + type: string + type: object + dbWallet: + properties: + key: + type: string + secret: + type: string + type: object + type: object + exporter: + description: DatabaseObserverExporterConfig defines the configuration + details related to the exporters of DatabaseObserver + properties: + configuration: + properties: + configmap: + description: ConfigMapDetails defines the configmap name + properties: + configmapName: + type: string + key: + type: string + type: object + type: object + image: + type: string + service: + description: DatabaseObserverService defines the exporter service + component of DatabaseObserver + properties: + port: + format: int32 + type: integer + type: object + type: object + ociConfig: + properties: + configMapName: + type: string + secretName: + type: string + type: object + prometheus: + description: PrometheusConfig defines the generated resources for + Prometheus + properties: + labels: + additionalProperties: + type: string + type: object + port: + type: string + type: object + replicas: + format: int32 + type: integer + type: object + status: + description: DatabaseObserverStatus defines the observed state of DatabaseObserver + properties: + conditions: + description: 'INSERT ADDITIONAL STATUS FIELD - define observed state + of cluster Important: Run "make" to regenerate code after modifying + this file' + items: + description: "Condition contains details for one aspect of the current + state of this API Resource. --- This struct is intended for direct + use as an array at the field path .status.conditions. For example, + \n \ttype FooStatus struct{ \t // Represents the observations + of a foo's current state. \t // Known .status.conditions.type + are: \"Available\", \"Progressing\", and \"Degraded\" \t // + +patchMergeKey=type \t // +patchStrategy=merge \t // +listType=map + \t // +listMapKey=type \t Conditions []metav1.Condition + `json:\"conditions,omitempty\" patchStrategy:\"merge\" patchMergeKey:\"type\" + protobuf:\"bytes,1,rep,name=conditions\"` \n \t // other fields + \t}" + properties: + lastTransitionTime: + description: lastTransitionTime is the last time the condition + transitioned from one status to another. This should be when + the underlying condition changed. If that is not known, then + using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: message is a human readable message indicating + details about the transition. This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: observedGeneration represents the .metadata.generation + that the condition was set based upon. For instance, if .metadata.generation + is currently 12, but the .status.conditions[x].observedGeneration + is 9, the condition is out of date with respect to the current + state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: reason contains a programmatic identifier indicating + the reason for the condition's last transition. Producers + of specific condition types may define expected values and + meanings for this field, and whether the values are considered + a guaranteed API. The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + --- Many .condition.type values are consistent across resources + like Available, but because arbitrary conditions can be useful + (see .node.status.conditions), the ability to deconflict is + important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + exporterConfig: + type: string + replicas: + type: integer + status: + type: string + required: + - conditions + - exporterConfig + type: object + type: object + served: true + storage: true + subresources: + status: {} +status: + acceptedNames: + kind: "" + plural: "" + conditions: [] + storedVersions: [] diff --git a/config/samples/kustomization.yaml b/config/samples/kustomization.yaml index 3bf0eab5..ac5e158f 100644 --- a/config/samples/kustomization.yaml +++ b/config/samples/kustomization.yaml @@ -5,17 +5,29 @@ ## Append samples you want in your CSV to this file as resources ## resources: - - onpremdb/pdb.yaml - - onpremdb/cdb.yaml - - adb/autonomousdatabase_create.yaml + - multitenant/pdb_plug.yaml + - multitenant/cdb_secret.yaml + - multitenant/pdb_secret.yaml + - multitenant/pdb_clone.yaml + - multitenant/cdb.yaml + - sidb/singleinstancedatabase_patch.yaml + - sidb/oraclerestdataservice_apex.yaml + - sidb/singleinstancedatabase_express.yaml + - sidb/singleinstancedatabase_secrets.yaml + - sidb/singleinstancedatabase_clone.yaml + - sidb/singleinstancedatabase_prebuiltdb.yaml + - sidb/dataguardbroker.yaml + - sidb/oraclerestdataservice_secrets.yaml + - sidb/singleinstancedatabase_free.yaml + - sidb/singleinstancedatabase_standby.yaml + - sidb/openshift_rbac.yaml + - sharding/sharding_v1alpha1_provshard_clonespec1.yaml + - sharding/shardingdatabase.yaml + - sharding/sharding_v1alpha1_provshard_clonespec.yaml + - observability/databaseobserver_vault.yaml + - observability/databaseobserver_minimal.yaml - adb/autonomousdatabase_bind.yaml - adb/autonomousdatabase_backup.yaml - adb/autonomousdatabase_restore.yaml - - acd/autonomouscontainerdatabase_create.yaml - - sidb/singleinstancedatabase.yaml - - sharding/shardingdatabase.yaml - - sharding/sharding_v1alpha1_provshard.yaml - - dbcs/database_v1alpha1_dbcssystem.yaml - - database_v1alpha1_dataguardbroker.yaml - - observability/databaseobserver.yaml + - acd/autonomouscontainerdatabase_restart_terminate.yaml # +kubebuilder:scaffold:manifestskustomizesamples diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index b66082e1..4425acea 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -103,6 +103,21 @@ spec: ## Optionally specify a volume containing scripts in 'setup' and 'startup' folders to be executed during database setup and startup respectively. scriptsVolumeName: "" + ## Database pod resource details + ## cpu can be expressed in terms of cpu units and can be a plain integer or fractional value + ## memory is measured in bytes and can be expressed in plain integer or as a fixed-point number + ## using one of these quantity suffixes: E, P, T, G, M, k. + ## You can also use the power-of-two equivalents: Ei, Pi, Ti, Gi, Mi, Ki. + resources: + ## requests denotes minimum node resources required/to be utilized by the database pod + requests: + cpu: + memory: + ## limits specifies the maximum node resources that can be utilized by the database pod + limits: + cpu: + memory: + ## Type of service . Applicable on cloud enviroments only ## if loadBalService : false, service type = "NodePort" else "LoadBalancer" loadBalancer: false diff --git a/config/webhook/manifests.yaml b/config/webhook/manifests.yaml index 175abd47..c150867f 100644 --- a/config/webhook/manifests.yaml +++ b/config/webhook/manifests.yaml @@ -130,6 +130,26 @@ webhooks: resources: - pdbs sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: webhook-service + namespace: system + path: /mutate-database-oracle-com-v1alpha1-shardingdatabase + failurePolicy: Fail + name: mshardingdatabase.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + resources: + - shardingdatabases + sideEffects: None - admissionReviewVersions: - v1 - v1beta1 @@ -343,6 +363,27 @@ webhooks: resources: - pdbs sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: webhook-service + namespace: system + path: /validate-database-oracle-com-v1alpha1-shardingdatabase + failurePolicy: Fail + name: vshardingdatabase.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + - DELETE + resources: + - shardingdatabases + sideEffects: None - admissionReviewVersions: - v1 - v1beta1 diff --git a/controllers/database/pdb_controller.go b/controllers/database/pdb_controller.go index 5c173740..f0b4fd46 100644 --- a/controllers/database/pdb_controller.go +++ b/controllers/database/pdb_controller.go @@ -44,6 +44,7 @@ import ( "crypto/tls" "crypto/x509" "encoding/json" + //"encoding/pem" "errors" "fmt" @@ -59,6 +60,7 @@ import ( "github.com/go-logr/logr" corev1 "k8s.io/api/core/v1" apierrors "k8s.io/apimachinery/pkg/api/errors" + //metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/runtime" "k8s.io/apimachinery/pkg/types" @@ -119,10 +121,13 @@ var ( ) const PDBFinalizer = "database.oracle.com/PDBfinalizer" +const ONE = 1 +const ZERO = 0 var tdePassword string var tdeSecret string var floodcontrol bool = false +var assertivePdbDeletion bool = false /* Global variable for assertive pdb deletion */ //+kubebuilder:rbac:groups=database.oracle.com,resources=pdbs,verbs=get;list;watch;create;update;patch;delete //+kubebuilder:rbac:groups=database.oracle.com,resources=pdbs/status,verbs=get;update;patch @@ -133,7 +138,6 @@ var floodcontrol bool = false // +kubebuilder:rbac:groups=apps,resources=statefulsets,verbs=get;list;watch;create;update;patch;delete // +kubebuilder:rbac:groups='',resources=statefulsets/finalizers,verbs=get;list;watch;create;update;patch;delete - // Reconcile is part of the main kubernetes reconciliation loop which aims to // move the current state of the cluster closer to the desired state. // TODO(user): Modify the Reconcile function to compare the state specified by @@ -183,9 +187,9 @@ func (r *PDBReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.R } // Finalizer section - err = r.managePDBDeletion(ctx, req, pdb) + err = r.managePDBDeletion2(ctx, req, pdb) if err != nil { - log.Info("Reconcile queued") + log.Info("managePDBDeletion2 Error Deleting resource ") return requeueY, nil } @@ -652,6 +656,9 @@ func (r *PDBReconciler) createPDB(ctx context.Context, req ctrl.Request, pdb *db return nil } + pdbAdminName = strings.TrimSuffix(pdbAdminName, "\n") + pdbAdminPwd = strings.TrimSuffix(pdbAdminPwd, "\n") + values := map[string]string{ "method": "CREATE", "pdb_name": pdb.Spec.PDBName, @@ -681,7 +688,6 @@ func (r *PDBReconciler) createPDB(ctx context.Context, req ctrl.Request, pdb *db values["tdeSecret"] = tdeSecret } - //url := "https://"+ pdb.Spec.CDBNamespace + "." + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" url := "https://" + pdb.Spec.CDBResName + "-ords." + pdb.Spec.CDBNamespace + ":" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" pdb.Status.TotalSize = pdb.Spec.TotalSize @@ -705,6 +711,10 @@ func (r *PDBReconciler) createPDB(ctx context.Context, req ctrl.Request, pdb *db ParseTnsAlias(&(pdb.Status.ConnString), &(pdb.Spec.PDBName)) } + assertivePdbDeletion = pdb.Spec.AssertivePdbDeletion + if pdb.Spec.AssertivePdbDeletion == true { + r.Recorder.Eventf(pdb, corev1.EventTypeNormal, "Created", "PDB '%s' assertive pdb deletion turned on", pdb.Spec.PDBName) + } log.Info("New connect strinng", "tnsurl", cdb.Spec.DBTnsurl) log.Info("Created PDB Resource", "PDB Name", pdb.Spec.PDBName) r.getPDBState(ctx, req, pdb) @@ -786,6 +796,11 @@ func (r *PDBReconciler) clonePDB(ctx context.Context, req ctrl.Request, pdb *dba ParseTnsAlias(&(pdb.Status.ConnString), &(pdb.Spec.PDBName)) } + assertivePdbDeletion = pdb.Spec.AssertivePdbDeletion + if pdb.Spec.AssertivePdbDeletion == true { + r.Recorder.Eventf(pdb, corev1.EventTypeNormal, "Clone", "PDB '%s' assertive pdb deletion turned on", pdb.Spec.PDBName) + } + log.Info("Cloned PDB successfully", "Source PDB Name", pdb.Spec.SrcPDBName, "Clone PDB Name", pdb.Spec.PDBName) r.getPDBState(ctx, req, pdb) return nil @@ -845,8 +860,6 @@ func (r *PDBReconciler) plugPDB(ctx context.Context, req ctrl.Request, pdb *dbap values["asClone"] = strconv.FormatBool(*(pdb.Spec.AsClone)) } - //url := "https://"+ pdb.Spec.CDBNamespace + "." + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" - //url := "https://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" url := "https://" + pdb.Spec.CDBResName + "-ords." + pdb.Spec.CDBNamespace + ":" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" pdb.Status.TotalSize = pdb.Spec.TotalSize @@ -868,6 +881,11 @@ func (r *PDBReconciler) plugPDB(ctx context.Context, req ctrl.Request, pdb *dbap pdb.Status.ConnString = cdb.Spec.DBTnsurl } + assertivePdbDeletion = pdb.Spec.AssertivePdbDeletion + if pdb.Spec.AssertivePdbDeletion == true { + r.Recorder.Eventf(pdb, corev1.EventTypeNormal, "Plugged", "PDB '%s' assertive pdb deletion turned on", pdb.Spec.PDBName) + } + log.Info("Successfully plugged PDB", "PDB Name", pdb.Spec.PDBName) r.getPDBState(ctx, req, pdb) return nil @@ -915,7 +933,6 @@ func (r *PDBReconciler) unplugPDB(ctx context.Context, req ctrl.Request, pdb *db values["tdeExport"] = strconv.FormatBool(*(pdb.Spec.TDEExport)) } - //url := "https://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdb.Spec.PDBName + "/" url := "https://" + pdb.Spec.CDBResName + "-ords." + pdb.Spec.CDBNamespace + ":" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdb.Spec.PDBName + "/" log.Info("CallAPI(url)", "url", url) @@ -1011,7 +1028,6 @@ func (r *PDBReconciler) modifyPDB(ctx context.Context, req ctrl.Request, pdb *db log.Info("PDB STATUS OPENMODE", "pdb.Status.OpenMode=", pdb.Status.OpenMode) pdbName := pdb.Spec.PDBName - //url := "https://" + pdb.Spec.CDBNamespace + "." + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdbName + "/status" url := "https://" + pdb.Spec.CDBResName + "-ords." + pdb.Spec.CDBNamespace + ":" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdbName + "/status" pdb.Status.Phase = pdbPhaseModify @@ -1055,8 +1071,6 @@ func (r *PDBReconciler) getPDBState(ctx context.Context, req ctrl.Request, pdb * } pdbName := pdb.Spec.PDBName - //url := "https://"+ pdb.Spec.CDBNamespace + "." + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdbName + "/status" - //url := "https://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdbName + "/status" url := "https://" + pdb.Spec.CDBResName + "-ords." + pdb.Spec.CDBNamespace + ":" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdbName + "/status" pdb.Status.Msg = "Getting PDB state" @@ -1108,8 +1122,6 @@ func (r *PDBReconciler) mapPDB(ctx context.Context, req ctrl.Request, pdb *dbapi } pdbName := pdb.Spec.PDBName - //url := "https://"+ pdb.Spec.CDBNamespace + "." + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdbName + "/" - //url := "https://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdbName + "/" url := "https://" + pdb.Spec.CDBResName + "-ords." + pdb.Spec.CDBNamespace + ":" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdbName + "/" pdb.Status.Msg = "Mapping PDB" @@ -1129,12 +1141,15 @@ func (r *PDBReconciler) mapPDB(ctx context.Context, req ctrl.Request, pdb *dbapi log.Error(err, "Failed to get state of PDB :"+pdbName, "err", err.Error()) } - //fmt.Printf("%+v\n", objmap) totSizeInBytes := objmap["total_size"].(float64) totSizeInGB := totSizeInBytes / 1024 / 1024 / 1024 pdb.Status.OpenMode = objmap["open_mode"].(string) pdb.Status.TotalSize = fmt.Sprintf("%.2f", totSizeInGB) + "G" + assertivePdbDeletion = pdb.Spec.AssertivePdbDeletion + if pdb.Spec.AssertivePdbDeletion == true { + r.Recorder.Eventf(pdb, corev1.EventTypeNormal, "Mapped", "PDB '%s' assertive pdb deletion turned on", pdb.Spec.PDBName) + } if cdb.Spec.DBServer != "" { pdb.Status.ConnString = cdb.Spec.DBServer + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName @@ -1188,49 +1203,72 @@ func (r *PDBReconciler) deletePDB(ctx context.Context, req ctrl.Request, pdb *db return nil } -/* -************************************************ +/************************************************* - Check PDB deletion - /*********************************************** -*/ -func (r *PDBReconciler) managePDBDeletion(ctx context.Context, req ctrl.Request, pdb *dbapi.PDB) error { - log := r.Log.WithValues("managePDBDeletion", req.NamespacedName) - - // Check if the PDB instance is marked to be deleted, which is - // indicated by the deletion timestamp being set. - isPDBMarkedToBeDeleted := pdb.GetDeletionTimestamp() != nil - if isPDBMarkedToBeDeleted { - log.Info("Marked to be deleted") - pdb.Status.Phase = pdbPhaseDelete - pdb.Status.Status = true - r.Status().Update(ctx, pdb) +**************************************************/ +func (r *PDBReconciler) managePDBDeletion2(ctx context.Context, req ctrl.Request, pdb *dbapi.PDB) error { + log := r.Log.WithValues("managePDBDeletion", req.NamespacedName) + if pdb.ObjectMeta.DeletionTimestamp.IsZero() { + if !controllerutil.ContainsFinalizer(pdb, PDBFinalizer) { + controllerutil.AddFinalizer(pdb, PDBFinalizer) + if err := r.Update(ctx, pdb); err != nil { + return err + } + } + } else { + log.Info("Pdb marked to be delted") if controllerutil.ContainsFinalizer(pdb, PDBFinalizer) { - // Remove PDBFinalizer. Once all finalizers have been - // removed, the object will be deleted. - log.Info("Removing finalizer") + if assertivePdbDeletion == true { + log.Info("Deleting pdb CRD: Assertive approach is turned on ") + cdb, err := r.getCDBResource(ctx, req, pdb) + if err != nil { + log.Error(err, "Cannont find cdb resource ", "err", err.Error()) + return err + } + + pdbName := pdb.Spec.PDBName + if pdb.Status.OpenMode == "READ WRITE" { + valuesclose := map[string]string{ + "state": "CLOSE", + "modifyOption": "IMMEDIATE", + "getScript": "FALSE"} + url := "https://" + pdb.Spec.CDBResName + "-ords." + pdb.Spec.CDBNamespace + ":" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdbName + "/status" + _, errclose := r.callAPI(ctx, req, pdb, url, valuesclose, "POST") + if errclose != nil { + log.Info("Warning error closing pdb continue anyway") + } + } + + valuesdrop := map[string]string{ + "action": "INCLUDING", + "getScript": "FALSE"} + url := "https://" + pdb.Spec.CDBResName + "-ords." + pdb.Spec.CDBNamespace + ":" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdbName + "/" + + log.Info("Call Delete()") + _, errdelete := r.callAPI(ctx, req, pdb, url, valuesdrop, "DELETE") + if errdelete != nil { + log.Error(errdelete, "Fail to delete pdb :"+pdb.Name, "err", err.Error()) + return errdelete + } + } /* END OF ASSERTIVE SECTION */ + + log.Info("Marked to be deleted") + pdb.Status.Phase = pdbPhaseDelete + pdb.Status.Status = true + r.Status().Update(ctx, pdb) + controllerutil.RemoveFinalizer(pdb, PDBFinalizer) - err := r.Update(ctx, pdb) - if err != nil { - log.Info("Could not remove finalizer", "err", err.Error()) + if err := r.Update(ctx, pdb); err != nil { + log.Info("Cannot remove finalizer") return err } - log.Info("Successfully removed PDB resource") - return nil - } - } - // Add finalizer for this CR - if !controllerutil.ContainsFinalizer(pdb, PDBFinalizer) { - log.Info("Adding finalizer") - controllerutil.AddFinalizer(pdb, PDBFinalizer) - err := r.Update(ctx, pdb) - if err != nil { - log.Info("Could not add finalizer", "err", err.Error()) - return err } - pdb.Status.Status = false + + return nil } + return nil } diff --git a/controllers/database/shardingdatabase_controller.go b/controllers/database/shardingdatabase_controller.go index 88823afe..7fcaac2b 100644 --- a/controllers/database/shardingdatabase_controller.go +++ b/controllers/database/shardingdatabase_controller.go @@ -94,6 +94,9 @@ type ShardingDatabaseReconciler struct { Namespace string } +var sentFailMsg = make(map[string]bool) +var sentCompleteMsg = make(map[string]bool) + // +kubebuilder:rbac:groups=database.oracle.com,resources=shardingdatabases,verbs=get;list;watch;create;update;patch;delete // +kubebuilder:rbac:groups=database.oracle.com,resources=shardingdatabases/status,verbs=get;update;patch // +kubebuilder:rbac:groups=database.oracle.com,resources=shardingdatabases/finalizers,verbs=get;create;update;patch;delete @@ -166,6 +169,7 @@ func (r *ShardingDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.Req r.osh = append(r.osh, osh) } defer r.setCrdLifeCycleState(instance, &result, &err, &stateType) + defer r.updateShardTopologyStatus(instance) // =============================== Check Deletion TimeStamp======== // Check if the ProvOShard instance is marked to be deleted, which is // // indicated by the deletion timestamp being set. @@ -286,7 +290,7 @@ func (r *ShardingDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.Req // if user set replicasize greater than 1 but also set instance.Spec.OraDbPvcName then only one service will be created and one pod for i = 0; i < int32(len(instance.Spec.Shard)); i++ { OraShardSpex = instance.Spec.Shard[i] - if OraShardSpex.IsDelete != true { + if !shardingv1.CheckIsDeleteFlag(OraShardSpex.IsDelete, instance, r.Log) { result, err = r.createService(instance, shardingv1.BuildServiceDefForShard(instance, 0, OraShardSpex, "local")) if err != nil { result = resultNq @@ -306,7 +310,7 @@ func (r *ShardingDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.Req if len(instance.Spec.Shard) > 0 { for i = 0; i < int32(len(instance.Spec.Shard)); i++ { OraShardSpex = instance.Spec.Shard[i] - if OraShardSpex.IsDelete != true { + if !shardingv1.CheckIsDeleteFlag(OraShardSpex.IsDelete, instance, r.Log) { result, err = r.deployStatefulSet(instance, shardingv1.BuildStatefulSetForShard(instance, OraShardSpex), "SHARD") if err != nil { result = resultNq @@ -327,6 +331,13 @@ func (r *ShardingDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.Req return result, err } + err = r.checkShardState(instance) + if err != nil { + err = nilErr + result = resultQ + return result, err + } + //set the Waiting state for Reconcile loop // Loop will be requeued only if Shard Statefulset is not ready or not configured. // Till that time Reconcilation loop will remain in blocked state @@ -380,7 +391,7 @@ func (r *ShardingDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.Req // ====================== Update Setup for Shard ============================== for i = 0; i < int32(len(instance.Spec.Shard)); i++ { OraShardSpex = instance.Spec.Shard[i] - if OraShardSpex.IsDelete != true { + if !shardingv1.CheckIsDeleteFlag(OraShardSpex.IsDelete, instance, r.Log) { sfSet, shardPod, err := r.validateShard(instance, OraShardSpex, int(i)) if err != nil { shardingv1.LogMessages("INFO", "Shard "+sfSet.Name+" is not in available state.", nil, instance, r.Log) @@ -413,18 +424,6 @@ func (r *ShardingDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.Req } } - // Calling updateShardTopology to update the entire sharding topology - // This is required because we just executed updateShard,updateCatalog and UpdateGsm - // If some state has changed it will update the topology - - err = r.updateShardTopologyStatus(instance) - if err != nil { - // time.Sleep(30 * time.Second) - result = resultQ - err = nilErr - return result, err - } - stateType = string(databasev1alpha1.CrdReconcileCompeleteState) // r.setCrdLifeCycleState(instance, &result, &err, stateType) // Set error to ni to avoid reconcilation state reconcilation error as we are passing err to setCrdLifeCycleState @@ -889,7 +888,7 @@ func (r *ShardingDatabaseReconciler) validateSpex(instance *databasev1alpha1.Sha if len(instance.Spec.Shard) > 0 { for i = 0; i < int32(len(instance.Spec.Shard)); i++ { OraShardSpex := instance.Spec.Shard[i] - if OraShardSpex.IsDelete != true { + if !shardingv1.CheckIsDeleteFlag(OraShardSpex.IsDelete, instance, r.Log) { err = r.checkShardSpace(instance, OraShardSpex) if err != nil { return err @@ -902,32 +901,6 @@ func (r *ShardingDatabaseReconciler) validateSpex(instance *databasev1alpha1.Sha } } - // Check Secret configuration - if instance.Spec.DbSecret == nil { - return fmt.Errorf("Secret specification cannot be null, you need to set secret details") - } else { - if len(instance.Spec.DbSecret.Name) == 0 { - return fmt.Errorf("instance.Spec.DbSecret.Name cannot be empty") - } - if len(instance.Spec.DbSecret.PwdFileName) == 0 { - return fmt.Errorf("instance.Spec.DbSecret.PwdFileName cannot be empty") - } - if strings.ToLower(instance.Spec.DbSecret.EncryptionType) != "base64" { - if strings.ToLower(instance.Spec.DbSecret.KeyFileName) == "" { - return fmt.Errorf("instance.Spec.DbSecret.KeyFileName cannot be empty") - } - } - if len(instance.Spec.DbSecret.PwdFileMountLocation) == 0 { - msg := "instance.Spec.DbSecret.PwdFileMountLocation is not set. Setting it to default " + shardingv1.GetSecretMount() - shardingv1.LogMessages("INFO", msg, nil, instance, r.Log) - } - - if len(instance.Spec.DbSecret.KeyFileMountLocation) == 0 { - msg := "instance.Spec.DbSecret.KeyFileMountLocation is not set. Setting it to default " + shardingv1.GetSecretMount() - shardingv1.LogMessages("INFO", msg, nil, instance, r.Log) - } - } - // Once the initial Spec is been validated then update the last Sucessful Spec err = instance.UpdateLastSuccessfulSpec(r.Client) if err != nil { @@ -1350,23 +1323,21 @@ func (r *ShardingDatabaseReconciler) validateShard(instance *databasev1alpha1.Sh } // This function updates the shard topology over all -func (r *ShardingDatabaseReconciler) updateShardTopologyStatus(instance *databasev1alpha1.ShardingDatabase) error { +func (r *ShardingDatabaseReconciler) updateShardTopologyStatus(instance *databasev1alpha1.ShardingDatabase) { //shardPod := &corev1.Pod{} //gsmSfSet := &appsv1.StatefulSet{} gsmPod := &corev1.Pod{} var err error _, _, err = r.validateCatalog(instance) if err != nil { - return err + } _, gsmPod, err = r.validateGsm(instance) if err != nil { - return err + } r.updateShardTopologyShardsInGsm(instance, gsmPod) - return nil - } func (r *ShardingDatabaseReconciler) updateShardTopologyShardsInGsm(instance *databasev1alpha1.ShardingDatabase, gsmPod *corev1.Pod) { @@ -1378,8 +1349,11 @@ func (r *ShardingDatabaseReconciler) updateShardTopologyShardsInGsm(instance *da if len(instance.Spec.Shard) > 0 { for i = 0; i < int32(len(instance.Spec.Shard)); i++ { OraShardSpex := instance.Spec.Shard[i] + if strings.ToLower(OraShardSpex.IsDelete) == "failed" { + continue + } // stateStr := shardingv1.GetGsmShardStatus(instance, OraShardSpex.Name) - if OraShardSpex.IsDelete != true { + if !shardingv1.CheckIsDeleteFlag(OraShardSpex.IsDelete, instance, r.Log) { shardSfSet, _, err = r.validateShard(instance, OraShardSpex, int(i)) if err != nil { continue @@ -1532,7 +1506,7 @@ func (r *ShardingDatabaseReconciler) addPrimaryShards(instance *databasev1alpha1 // stateStr := shardingv1.GetGsmShardStatus(instance, OraShardSpex.Name) // strings.Contains(stateStr, "DELETE") - if OraShardSpex.IsDelete != true { + if !shardingv1.CheckIsDeleteFlag(OraShardSpex.IsDelete, instance, r.Log) { if setLifeCycleFlag != true { setLifeCycleFlag = true stateType := string(databasev1alpha1.CrdReconcileWaitingState) @@ -1561,33 +1535,36 @@ func (r *ShardingDatabaseReconciler) addPrimaryShards(instance *databasev1alpha1 continue } + /** // Copy file from pod to FS - // configrest, kclientset, err := shardingv1.GetPodCopyConfig(r.kubeClient, r.kubeConfig, instance, r.Log) - // if err != nil { - // return fmt.Errorf("Error occurred in getting KubeConfig, cannot perform copy operation from the pod") - // } - - // _, _, err = shardingv1.ExecCommand(gsmPod.Name, shardingv1.GetTdeKeyLocCmd(), r.kubeClient, r.kubeConfig, instance, r.Log) - // if err != nil { - // fmt.Printf("Error occurred during the while getting the TDE key from the pod " + gsmPod.Name) - // //return err - // } - // fileName := "/tmp/tde_key" - // last := fileName[strings.LastIndex(fileName, "/")+1:] - // fileName1 := last - // fsLoc := shardingv1.TmpLoc + "/" + fileName1 - // _, _, _, err = shardingv1.KctlCopyFile(r.kubeClient, r.kubeConfig, instance, configrest, kclientset, r.Log, fmt.Sprintf("%s/%s:/%s", instance.Spec.Namespace, gsmPod.Name, fileName), fsLoc, "") - // if err != nil { - // fmt.Printf("failed to copy file") - // //return err - // } + configrest, kclientset, err := shardingv1.GetPodCopyConfig(r.kubeClient, r.kubeConfig, instance, r.Log) + if err != nil { + return fmt.Errorf("Error occurred in getting KubeConfig, cannot perform copy operation from the pod") + } + + _, _, err = shardingv1.ExecCommand(gsmPod.Name, shardingv1.GetTdeKeyLocCmd(), r.kubeClient, r.kubeConfig, instance, r.Log) + if err != nil { + fmt.Printf("Error occurred during the while getting the TDE key from the pod " + gsmPod.Name) + //return err + } + fileName := "/tmp/tde_key" + last := fileName[strings.LastIndex(fileName, "/")+1:] + fileName1 := last + fsLoc := shardingv1.TmpLoc + "/" + fileName1 + _, _, _, err = shardingv1.KctlCopyFile(r.kubeClient, r.kubeConfig, instance, configrest, kclientset, r.Log, fmt.Sprintf("%s/%s:/%s", instance.Spec.Namespace, gsmPod.Name, fileName), fsLoc, "") + if err != nil { + fmt.Printf("failed to copy file") + //return err + } // Copying it to Shard Pod - // _, _, _, err = shardingv1.KctlCopyFile(r.kubeClient, r.kubeConfig, instance, configrest, kclientset, r.Log, fsLoc, fmt.Sprintf("%s/%s:/%s", instance.Spec.Namespace, OraShardSpex.Name+"-0", fsLoc), "") - // if err != nil { - // fmt.Printf("failed to copy file") - // //return err - /// } + _, _, _, err = shardingv1.KctlCopyFile(r.kubeClient, r.kubeConfig, instance, configrest, kclientset, r.Log, fsLoc, fmt.Sprintf("%s/%s:/%s", instance.Spec.Namespace, OraShardSpex.Name+"-0", fsLoc), "") + if err != nil { + fmt.Printf("failed to copy file") + //return err + } + + **/ // If the shard doesn't exist in GSM then just add the shard statefulset and update GSM shard status // ADD Shard in GSM @@ -1598,7 +1575,12 @@ func (r *ShardingDatabaseReconciler) addPrimaryShards(instance *databasev1alpha1 r.updateGsmShardStatus(instance, OraShardSpex.Name, string(databasev1alpha1.AddingShardErrorState)) title = "Shard Addition Failure" message = "Error occurred during shard " + shardingv1.GetFmtStr(OraShardSpex.Name) + " addition." - r.sendMessage(instance, title, message) + shardingv1.LogMessages("INFO", title+":"+message, nil, instance, r.Log) + if sentFailMsg[OraShardSpex.Name] != true { + r.sendMessage(instance, title, message) + } + sentFailMsg[OraShardSpex.Name] = true + sentCompleteMsg[OraShardSpex.Name] = false deployFlag = false } } @@ -1649,7 +1631,13 @@ func (r *ShardingDatabaseReconciler) verifyShards(instance *databasev1alpha1.Sha if oldStateStr != string(databasev1alpha1.ShardOnlineState) { title = "Shard Addition Completed" message = "Shard addition completed for shard " + shardingv1.GetFmtStr(shardSfSet.Name) + " in GSM." - r.sendMessage(instance, title, message) + shardingv1.LogMessages("INFO", title+":"+message, nil, instance, r.Log) + if sentCompleteMsg[shardSfSet.Name] != true { + r.sendMessage(instance, title, message) + } + + sentCompleteMsg[shardSfSet.Name] = true + sentFailMsg[shardSfSet.Name] = false } return nil } @@ -1680,7 +1668,7 @@ func (r *ShardingDatabaseReconciler) delGsmShard(instance *databasev1alpha1.Shar if len(instance.Spec.Shard) > 0 { for i = 0; i < int32(len(instance.Spec.Shard)); i++ { OraShardSpex := instance.Spec.Shard[i] - if OraShardSpex.IsDelete == true { + if shardingv1.CheckIsDeleteFlag(OraShardSpex.IsDelete, instance, r.Log) { if setLifeCycleFlag != true { setLifeCycleFlag = true stateType := string(databasev1alpha1.CrdReconcileWaitingState) @@ -1736,6 +1724,13 @@ func (r *ShardingDatabaseReconciler) delGsmShard(instance *databasev1alpha1.Shar title = "Chunk Movement Failure" message = "Error occurred during chunk movement in shard " + shardingv1.GetFmtStr(OraShardSpex.Name) + " deletion." r.sendMessage(instance, title, message) + instance.Spec.Shard[i].IsDelete = "failed" + err = shardingv1.InstanceShardPatch(instance, instance, r.Client, i, "isDelete", "failed") + if err != nil { + msg = "Error occurred while changing the isDelete value to failed in Spec struct" + shardingv1.LogMessages("INFO", msg, nil, instance, r.Log) + return err + } continue } // 6th Step @@ -1743,13 +1738,26 @@ func (r *ShardingDatabaseReconciler) delGsmShard(instance *databasev1alpha1.Shar // This is a loop and will check unless there is a error or chunks has moved // Validate if the chunks has moved before performing shard deletion for { + msg = "Sleeping for 120 seconds and will check status again of chunks movement in gsm for shard: " + shardingv1.GetFmtStr(OraShardSpex.Name) + "ShardType=" + strings.TrimSpace(strings.ToUpper(instance.Spec.ShardingType)) + shardingv1.LogMessages("INFO", msg, nil, instance, r.Log) + time.Sleep(120 * time.Second) err = shardingv1.VerifyChunks(gsmPod.Name, sparams, instance, r.kubeClient, r.kubeConfig, r.Log) if err == nil { break } else { - msg = "Sleeping for 120 seconds and will check status again of chunks movement in gsm for shard: " + shardingv1.GetFmtStr(OraShardSpex.Name) - shardingv1.LogMessages("INFO", msg, nil, instance, r.Log) - time.Sleep(120 * time.Second) + if strings.TrimSpace(strings.ToUpper(instance.Spec.ShardingType)) != "USER" { + // If ShardingType is not "USER", do not perform the patching.. continue + continue + } + instance.Spec.Shard[i].IsDelete = "failed" + err = shardingv1.InstanceShardPatch(instance, instance, r.Client, i, "isDelete", "failed") + if err != nil { + // r.updateGsmShardStatus(instance, OraShardSpex.Name, string(databasev1alpha1.ChunkMoveError)) + msg = "Error occurred while changing the isDelete value to failed in Spec struct" + shardingv1.LogMessages("INFO", msg, nil, instance, r.Log) + // return err + } + return err } } } @@ -1764,6 +1772,7 @@ func (r *ShardingDatabaseReconciler) delGsmShard(instance *databasev1alpha1.Shar msg = "Error occurred during shard" + shardingv1.GetFmtStr(OraShardSpex.Name) + "removal from Gsm" shardingv1.LogMessages("Error", msg, nil, instance, r.Log) r.updateShardStatus(instance, int(i), string(databasev1alpha1.ShardRemoveError)) + instance.Spec.Shard[i].IsDelete = "failed" continue } @@ -1939,6 +1948,15 @@ func (r *ShardingDatabaseReconciler) deployStatefulSet(instance *databasev1alpha message := "Inside the deployStatefulSet function" shardingv1.LogMessages("DEBUG", message, nil, instance, r.Log) // See if StatefulSets already exists and create if it doesn't + // Error : invalid memory address or nil pointer dereference" (runtime error: invalid memory address or nil pointer dereference) + // This happens during unit test cases + for i := 0; i < 5; i++ { + if r.Scheme == nil { + time.Sleep(time.Second * 40) + } else { + break + } + } controllerutil.SetControllerReference(instance, dep, r.Scheme) found := &appsv1.StatefulSet{} err := r.Client.Get(context.TODO(), types.NamespacedName{ @@ -1974,3 +1992,58 @@ func (r *ShardingDatabaseReconciler) deployStatefulSet(instance *databasev1alpha return ctrl.Result{}, nil } + +func (r *ShardingDatabaseReconciler) checkShardState(instance *databasev1alpha1.ShardingDatabase) error { + + var i int32 + var err error = nil + var OraShardSpex databasev1alpha1.ShardSpec + var currState string + var eventMsg string + var msg string + + currState = "" + eventMsg = "" + + msg = "checkShardState():ShardType=" + strings.TrimSpace(strings.ToUpper(instance.Spec.ShardingType)) + shardingv1.LogMessages("INFO", msg, nil, instance, r.Log) + if strings.TrimSpace(strings.ToUpper(instance.Spec.ShardingType)) != "USER" { + // ShardingType is not "USER", so return + return err + } + + if len(instance.Status.Gsm.Shards) > 0 { + for i = 0; i < int32(len(instance.Spec.Shard)); i++ { + OraShardSpex = instance.Spec.Shard[i] + currState = shardingv1.GetGsmShardStatus(instance, OraShardSpex.Name) + if currState == string(databasev1alpha1.AddingShardState) { + eventMsg = "Shard Addition in progress. Requeuing" + err = fmt.Errorf(eventMsg) + break + } else if currState == string(databasev1alpha1.DeletingState) { + eventMsg = "Shard Deletion in progress. Requeuing" + err = fmt.Errorf(eventMsg) + err = nil + break + } else if OraShardSpex.IsDelete == "failed" { + eventMsg = "Shard Deletion failed. Manual intervention required. Requeuing" + err = fmt.Errorf(eventMsg) + break + } else if currState == string(databasev1alpha1.DeleteErrorState) { + eventMsg = "Shard Deletion Error. Manual intervention required. Requeuing" + err = fmt.Errorf(eventMsg) + break + } else if currState == string(databasev1alpha1.ShardRemoveError) { + eventMsg = "Shard Deletion Error. Manual intervention required. Requeuing" + err = fmt.Errorf(eventMsg) + break + } else { + eventMsg = "checkShardState() : Shard State=[" + currState + "]" + shardingv1.LogMessages("INFO", eventMsg, nil, instance, r.Log) + err = nil + } + } + r.publishEvents(instance, eventMsg, currState) + } + return err +} diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index fa01ae7e..a20fa1fd 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -1096,6 +1096,38 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns } }(), + + Resources: func() corev1.ResourceRequirements { + if m.Spec.Resources.Requests != nil && m.Spec.Resources.Limits != nil { + return corev1.ResourceRequirements{ + Requests: corev1.ResourceList{ + "cpu": resource.MustParse(m.Spec.Resources.Requests.Cpu), + "memory": resource.MustParse(m.Spec.Resources.Requests.Memory), + }, + Limits: corev1.ResourceList{ + "cpu": resource.MustParse(m.Spec.Resources.Limits.Cpu), + "memory": resource.MustParse(m.Spec.Resources.Requests.Memory), + }, + } + } else if m.Spec.Resources.Requests != nil { + return corev1.ResourceRequirements{ + Requests: corev1.ResourceList{ + "cpu": resource.MustParse(m.Spec.Resources.Requests.Cpu), + "memory": resource.MustParse(m.Spec.Resources.Requests.Memory), + }, + } + } else if m.Spec.Resources.Limits != nil { + return corev1.ResourceRequirements{ + Limits: corev1.ResourceList{ + "cpu": resource.MustParse(m.Spec.Resources.Limits.Cpu), + "memory": resource.MustParse(m.Spec.Resources.Requests.Memory), + }, + } + } else { + return corev1.ResourceRequirements{} + } + + }(), }}, TerminationGracePeriodSeconds: func() *int64 { i := int64(30); return &i }(), diff --git a/controllers/observability/databaseobserver_resource.go b/controllers/observability/databaseobserver_resource.go index 75e05330..8c20ebe5 100644 --- a/controllers/observability/databaseobserver_resource.go +++ b/controllers/observability/databaseobserver_resource.go @@ -10,6 +10,7 @@ import ( "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" "k8s.io/apimachinery/pkg/runtime" "k8s.io/apimachinery/pkg/runtime/schema" + "k8s.io/apimachinery/pkg/util/intstr" "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" ) @@ -100,8 +101,9 @@ func (resource *ObservabilityServiceResource) generate(api *apiv1.DatabaseObserv Selector: rSelector, Ports: []corev1.ServicePort{ { - Name: "metrics", - Port: rPort, + Name: "metrics", + Port: rPort, + TargetPort: intstr.FromInt32(constants.DefaultServiceTargetPort), }, }, }, diff --git a/docs/multitenant/README.md b/docs/multitenant/README.md index b20eaa4a..6c9a6756 100644 --- a/docs/multitenant/README.md +++ b/docs/multitenant/README.md @@ -1,31 +1,37 @@ +# Oracle Multitenant Database Controllers +The Oracle Database Operator for kubernetes uses two controllers to manage [Pluggable Database life cycle][oradocpdb] -# Oracle Multitenant Database Controller +- CDB controller +- PDB controller -> WARNING: Examples with https are located in the use case directories +By usigng CDB/PDB controllers you can perform the following actions **CREATE**,**MODIFY(OPEN/COSE)**,**DELETE**,**CLONE**,**PLUG** and **UNPLUG** -Detailed examples can be found here +This file examplains how to setup CDB and PDB controllers, additional details can be found in the README files under usecases directories.. -- [Usecase01](./usecase01) pdb crd and cdb pod are running in the same namesaoce -- [Usecase02](./usecase02) unplug and plug operation examples -- [Usecase03](./usecase03) multiple namespace example cdb pod ,pdb crd and pod operator are running in different namespaces +- [Usecase01][uc01] pdb crd and cdb pod are running in the same namesaoce +- [Usecase02][uc02] unplug and plug operation examples +- [Usecase03][uc03] multiple namespace example cdb pod ,pdb crd and pod operator are running in different namespaces. +> **NOTE** that there is no controller for Container Database Operations -CDBs and PDBs are part of the Oracle Database [Multitenant Architecture](https://docs.oracle.com/en/database/oracle/oracle-database/21/multi/introduction-to-the-multitenant-architecture.html#GUID-AB84D6C9-4BBE-4D36-992F-2BB85739329F). The Multitenant Database Controller is a feature of Oracle DB Operator for Kubernetes (`OraOperator`), which helps to manage the lifecycle of Pluggable Databases (PDBs) in an Oracle Container Database (CDB). +## Macro steps for setup -The target CDB for which PDB lifecycle management is needed can be running on a machine on-premises. To manage the PDBs of that target CDB, you can run the Oracle DB Operator on a Kubernetes system on-premises (For Example: [Oracle Linux Cloud Native Environment or OLCNE](https://docs.oracle.com/en/operating-systems/olcne/)). +- Deply the Oracle Database Operator +- Create Ords based image for CDB pod +- Container DB user creation +- Create secrets for credentials +- Create certificates for https connection +- Create CDB pod -NOTE: The target CDB can also run in a Cloud environment, such as an OCI [Oracle Base Database Service](https://docs.oracle.com/en-us/iaas/dbcs/doc/bare-metal-and-virtual-machine-db-systems.html)). To manage PDBs on the target CDB, the Oracle DB Operator can run on a Kubernetes Cluster running in the cloud, such as OCI's [Container Engine for Kubernetes or OKE](https://docs.oracle.com/en-us/iaas/Content/ContEng/Concepts/contengoverview.htm#Overview_of_Container_Engine_for_Kubernetes)) - - - -# Oracle DB Operator Multitenant Database Controller Deployment +## Oracle DB Operator Multitenant Database Controller Deployment To deploy OraOperator, use this [Oracle Database Operator for Kubernetes](https://github.com/oracle/oracle-database-operator/blob/main/README.md) step-by-step procedure. After the Oracle Database Operator is deployed, you can see the DB Operator Pods running in the Kubernetes Cluster. As part of the `OraOperator` deployment, the multitenant Database Controller is deployed. You can see the CRDs (Custom Resource Definition) for the CDB and PDBs in the list of CRDs. The following output is an example of such a deployment: + ```bash [root@test-server oracle-database-operator]# kubectl get ns NAME STATUS AGE @@ -36,7 +42,6 @@ kube-public Active 245d kube-system Active 245d oracle-database-operator-system Active 24h <<<< namespace to deploy the Oracle Database Operator - [root@test-server oracle-database-operator]# kubectl get all -n oracle-database-operator-system NAME READY STATUS RESTARTS AGE pod/oracle-database-operator-controller-manager-665874bd57-dlhls 1/1 Running 0 28s @@ -54,7 +59,6 @@ NAME DESIRED replicaset.apps/oracle-database-operator-controller-manager-665874bd57 3 3 3 29s [root@docker-test-server oracle-database-operator]# - [root@test-server oracle-database-operator]# kubectl get crd NAME CREATED AT autonomouscontainerdatabases.database.oracle.com 2022-06-22T01:21:36Z @@ -75,23 +79,18 @@ shardingdatabases.database.oracle.com 2022-06-22T01:21:39Z singleinstancedatabases.database.oracle.com 2022-06-22T01:21:40Z ``` -The following sections explain the setup and functionality of this controller. - -# Prerequsites to manage PDB Life Cycle using Oracle DB Operator Multitenant Database Controller +## Prerequsites to manage PDB Life Cycle using Oracle DB Operator Multitenant Database Controller -**CAUTION :** You must complete the following steps before managing the lifecycle of a PDB in a CDB using the Oracle DB Operator Multitenant Database Controller. - -* [Prepare CDB for PDB Lifecycle Management or PDB-LM](#prepare-cdb-for-pdb-lifecycle-management-pdb-lm) +* [Prepare the container database for PDB Lifecycle Management or PDB-LM](#prepare-cdb-for-pdb-lifecycle-management-pdb-lm) * [Oracle REST Data Service or ORDS Image](#oracle-rest-data-service-ords-image) * [Kubernetes Secrets](#kubernetes-secrets) * [Kubernetes CRD for CDB](#kubernetes-crd-for-cdb) * [Kubernetes CRD for PDB](#kubernetes-crd-for-pdb) +## Prepare the container database for PDB Lifecycle Management (PDB-LM) -+ ## Prepare CDB for PDB Lifecycle Management (PDB-LM) - -Pluggable Database (PDB) management operations are performed in the Container Database (CDB). These operations include create, clone, plug, unplug, delete, modify and map operations. +Pluggable Database (PDB) management operations are performed in the Container Database (CDB). These operations include create, clone, plug, unplug, delete, modify and map pdb. You cannot have an ORDS-enabled schema in the container database. To perform the PDB lifecycle management operations, you must first use the following steps to define the default CDB administrator credentials on target CDBs: @@ -117,29 +116,31 @@ col account_status for a30 select username, account_status from dba_users where username in ('ORDS_PUBLIC_USER','C##DBAPI_CDB_ADMIN','APEX_PUBLIC_USER','APEX_REST_PUBLIC_USER'); ``` -### Reference Setup: Example of a setup using OCI OKE(Kubernetes Cluster) and a CDB in Cloud (OCI Exadata Database Cluster) +## OCI OKE(Kubernetes Cluster) -See this [provisioning example setup](./provisioning/example_setup_using_oci_oke_cluster.md) for steps to configure a Kubernetes Cluster and a CDB. This example uses an OCI OKE Cluster as the Kubernetes Cluster and a CDB in OCI Exadata Database service. +You can use an [OKE in Oracle Cloud Infrastructure][okelink] to configure the operator for PDB lifecycle management. **Note that there is no restriction about container database location; it can be anywhere (on cloud or premises , on any supported platform).** +To quickly create an OKE cluster in your OCI cloud environment you can use the following [link](./provisioning/quickOKEcreation.md). +In this setup example [provisioning example setup](./provisioning/example_setup_using_oci_oke_cluster.md), the Container database is running on a OCI Exadata Database Cluster. -+ ## Oracle REST Data Service (ORDS) Image + +## Oracle REST Data Service (ORDS) Image - Oracle DB Operator Multitenant Database controller requires that the Oracle REST Data Services (ORDS) image for PDB Lifecycle Management is present in the target CDB. - - You can build this image by using the ORDS [Dockerfile](../../../ords/Dockerfile) + The PDB Database controllers require a pod running a dedicated rest server image based on [ORDS][ordsdoc]. Read the following [link](./provisioning/ords_image.md) to build the ords images. - For the steps to build the ORDS Docker image, see [ORDS_image](./provisioning/ords_image.md) +## Kubernetes Secrets + Multitenant Controllers use Kubernetes Secrets to store the required credential. The https certificates are stored in Kubernetes Secrets as well. -+ ## Kubernetes Secrets + **Note** In multi namespace enviroment you have to create specific secrets for each namespaces - Oracle DB Operator Multitenant Database Controller uses Kubernetes Secrets to store usernames and passwords that you must have to manage the lifecycle operations of a PDB in the target CDB. In addition, to use https protocol, all certificates need to be stored using Kubernetes Secret. + **Note** In multi namespace enviroment you have to create specific secrets for each namespaces **Note** In multi namespace enviroment you have to create specific secrets for each namespaces ### Secrets for CDB CRD - Create a secret file as shown here: [config/samples/multitenant/cdb_secret.yaml](../../config/samples/multitenant/cdb_secret.yaml). Modify this file with the `base64` encoded values of the required passwords for CDB, and use this file to create the required secrets. + Create a secret file as shown here: [config/samples/multitenant/cdb_secret.yaml](../multitenant/provisioning/singlenamespace/cdb_create.yaml). Modify this file with the `base64` encoded values of the required passwords for CDB, and use this file to create the required secrets. ```bash kubectl apply -f cdb_secret.yaml @@ -150,17 +151,16 @@ See this [provisioning example setup](./provisioning/example_setup_using_oci_oke ```bash echo -n "" | base64 ``` - The value that is returned is the base64-encoded value for that password string. **Note:** After successful creation of the CDB Resource, the CDB secrets are deleted from the Kubernetes system . ### Secrets for PDB CRD - Create a secret file as shown here: [config/samples/multitenant/pdb_secret.yaml](../../config/samples/multitenant/pdb_secret.yaml). Modify this file with the `base64` encoded values of the required passwords for PDB and use it to create the required secrets. + + Create a secret file as shown here: [pdb_secret.yaml](../multitenant/provisioning/singlenamespace/pdb_secret.yaml). Edit the file using your base64 credential and apply it. ```bash kubectl apply -f pdb_secret.yaml ``` - **NOTE:** To encode the password using `base64`, see the command example in the preceding **Secrets for CDB CRD** section. **NOTE:** Don't leave plaintext files containing sensitive data on disk. After loading the Secret, remove the plaintext file or move it to secure storage. @@ -182,43 +182,73 @@ kubectl create secret generic db-ca --from-file=ca.crt -n oracle-database-operat ``` image_not_found + **Note:** On successful creation of the certificates secret creation remove files or move to secure storage . -+ ## Kubernetes CRD for CDB +## Kubernetes CRD for CDB -The Oracle Database Operator Multitenant Controller creates the CDB kind as a custom resource that models a target CDB as a native Kubernetes object. This kind is used only to create Pods to connect to the target CDB to perform PDB-LM operations. These CDB resources can be scaled, based on the expected load, using replicas. Each CDB resource follows the CDB CRD as defined here: [config/crd/bases/database.oracle.com_cdbs.yaml](../../config/crd/bases/database.oracle.com_cdbs.yaml) +The Oracle Database Operator Multitenant Controller creates the CDB kind as a custom resource that models a target CDB as a native Kubernetes object. This kind is used only to create Pods to connect to the target CDB to perform PDB-LM operations. Each CDB resource follows the CDB CRD as defined here: [config/crd/bases/database.oracle.com_cdbs.yaml](../../config/crd/bases/database.oracle.com_cdbs.yaml) -To create a CDB CRD, see this example `.yaml` file: [config/samples/multitenant/cdb.yaml](../../config/samples/multitenant/cdb.yaml) +To create a CDB CRD, see this example `.yaml` file: [cdb_create.yaml](../multitenant/provisioning/singlenamespace/cdb_create.yaml) **Note:** The password and username fields in this *cdb.yaml* Yaml are the Kubernetes Secrets created earlier in this procedure. For more information, see the section [Kubernetes Secrets](https://kubernetes.io/docs/concepts/configuration/secret/). To understand more about creating secrets for pulling images from a Docker private registry, see [Kubernetes Private Registry Documenation]( https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/). -1. [Use Case: Create a CDB CRD Resource](./provisioning/cdb_crd_resource.md) -2. [Use Case: Add another replica to an existing CDB CRD Resource](./provisioning/add_replica.md) +Create a CDB CRD Resource example +```bash +kubectl apply -f cdb_create.yaml +``` -+ ## Kubernetes CRD for PDB +see [usecase01][uc01] and usecase03[uc03] for more information about file configuration -The Oracle Database Operator Multitenant Controller creates the PDB kind as a custom resource that models a PDB as a native Kubernetes object. There is a one-to-one mapping between the actual PDB and the Kubernetes PDB Custom Resource. You cannot have more than one Kubernetes resource for a target PDB. This PDB resource can be used to perform PDB-LM operations by specifying the action attribute in the PDB Specs. Each PDB resource follows the PDB CRD as defined here: [config/crd/bases/database.oracle.com_pdbs.yaml](../../../config/crd/bases/database.oracle.com_pdbs.yaml) +## Kubernetes CRD for PDB -To create a PDB CRD Resource, a sample .yaml file is available here: [config/samples/multitenant/pdb_create.yaml](../../config/samples/multitenant/pdb_create.yaml) +The Oracle Database Operator Multitenant Controller creates the PDB kind as a custom resource that models a PDB as a native Kubernetes object. There is a one-to-one mapping between the actual PDB and the Kubernetes PDB Custom Resource. You cannot have more than one Kubernetes resource for a target PDB. This PDB resource can be used to perform PDB-LM operations by specifying the action attribute in the PDB Specs. Each PDB resource follows the PDB CRD as defined here: [config/crd/bases/database.oracle.com_pdbs.yaml](../../../config/crd/bases/database.oracle.com_pdbs.yaml) -# Use Cases for PDB Lifecycle Management Operations using Oracle DB Operator Multitenant Controller +To create a PDB CRD Resource, a sample .yaml file is available here: [pdb_create.yaml](../multitenant/provisioning/singlenamespace/pdb_create.yaml) -Using the Oracle DB Operator Multitenant Controller, you can perform the following PDB-LM operations: CREATE, CLONE, MODIFY, DELETE, UNPLUG, PLUG. +```bash +kubectl apply -f cdb_create.yaml +``` -1. [Create PDB](./provisioning/create_pdb.md) -2. [Clone PDB](./provisioning/clone_pdb.md) -3. [Modify PDB](./provisioning/modify_pdb.md) -4. [Delete PDB](./provisioning/delete_pdb.md) -5. [Unplug PDB](./provisioning/unplug_pdb.md) -6. [Plug PDB](./provisioning/plug_pdb.md) +## Usecases files list +### Single Namespace -## Validation and Errors +1. [Create CDB](./provisioning/singlenamespace/cdb_create.yaml) +2. [Create PDB](./provisioning/singlenamespace/pdb_create.yaml) +3. [Clone PDB](./provisioning/singlenamespace/pdb_clone.yaml) +4. [Open PDB](./provisioning/singlenamespace/pdb_open.yaml) +4. [Close PDB](./provisioning/singlenamespace/pdb_close.yaml) +5. [Delete PDB](./provisioning/singlenamespace/pdb_delete.yaml) +6. [Unplug PDB](./provisioning/singlenamespace/pdb_unplug.yaml) +7. [Plug PDB](./provisioning/singlenamespace/pdb_plug.yaml) -To see how to look for any validation errors, see [validation_error](./provisioning/validation_error.md). +### Multiple namespace (cdbnamespace,dbnamespace) +1. [Create CDB](./provisioning/multinamespace/cdb_create.yaml) +2. [Create PDB](./provisioning/multinamespace/pdb_create.yaml) +3. [Clone PDB](./provisioning/multinamespace/pdb_clone.yaml) +4. [Open PDB](./provisioning/multinamespace/pdb_open.yaml) +4. [Close PDB](./provisioning/multinamespace/pdb_close.yaml) +5. [Delete PDB](./provisioning/multinamespace/pdb_delete.yaml) +6. [Unplug PDB](./provisioning/multinamespace/pdb_unplug.yaml) ## Known issues -To find out about known issue related to Oracle DB Operator Multitenant Controller, see [known_issues](./provisioning/known_issues.md). + - Ords installatian failure if pluaggable databases in the container db are not opened + + - Version 1.1.0: encoded password for https authentication may include carriege return as consequence the https request fails with http 404 error. W/A generate encoded password using **printf** instead of **echo**. + + - pdb controller authentication suddenly failes without any system change. Check the certificate expiration date **openssl .... -days 365** + + - Nothing happens after cdb yaml file applying: Make sure to have properly configure the WHATCH_NAMESPACE list in the operator yaml file + + [okelink]:https://docs.oracle.com/en-us/iaas/Content/ContEng/Concepts/contengoverview.htm + [ordsdoc]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/23.1/index.html + [uc01]:../multitenant/usecase01/README.md + [uc02]:../multitenant/usecase02/README.md + [uc03]:../multitenant/usecase03/README.md + [oradocpdb]:https://docs.oracle.com/en/database/oracle/oracle-database/21/multi/introduction-to-the-multitenant-architecture.html#GUID-AB84D6C9-4BBE-4D36-992F-2BB85739329F + + \ No newline at end of file diff --git a/docs/multitenant/provisioning/add_replica.log b/docs/multitenant/provisioning/add_replica.log deleted file mode 100644 index 53971443..00000000 --- a/docs/multitenant/provisioning/add_replica.log +++ /dev/null @@ -1,192 +0,0 @@ --- Check the status of CDB CRD Pod(s): - -% kubectl get all -n oracle-database-operator-system -NAME READY STATUS RESTARTS AGE -pod/cdb-dev-ords-rs-q2b68 1/1 Running 0 29s -pod/oracle-database-operator-controller-manager-76cb674c5c-4nrh8 1/1 Running 0 4d10h -pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd 1/1 Running 1 4d10h -pod/oracle-database-operator-controller-manager-76cb674c5c-xsv9g 1/1 Running 2 4d10h - -NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE -service/oracle-database-operator-controller-manager-metrics-service ClusterIP 10.96.98.47 8443/TCP 5d1h -service/oracle-database-operator-webhook-service ClusterIP 10.96.166.163 443/TCP 5d1h - -NAME READY UP-TO-DATE AVAILABLE AGE -deployment.apps/oracle-database-operator-controller-manager 3/3 3 3 5d1h - -NAME DESIRED CURRENT READY AGE -replicaset.apps/cdb-dev-ords-rs 1 1 1 31s -replicaset.apps/oracle-database-operator-controller-manager-76cb674c5c 3 3 3 5d1h - - --- .yaml file for the add replica use case: - -% cat add_replica.yaml -# -# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. -# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. -# -apiVersion: database.oracle.com/v1alpha1 -kind: CDB -metadata: - name: cdb-dev - namespace: oracle-database-operator-system -spec: - cdbName: "goldcdb" - scanName: "goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com" - dbServer: "goldhost1.lbsub52b3b1cae.okecluster.oraclevcn.com" - ordsImage: phx.ocir.io//oracle/ords:21.4.3 - dbPort: 1521 - replicas: 2 - serviceName: "goldcdb_phx1pw.lbsub52b3b1cae.okecluster.oraclevcn.com" - sysAdminPwd: - secret: - secretName: "cdb1-secret" - key: "sysadmin_pwd" - ordsPwd: - secret: - secretName: "cdb1-secret" - key: "ords_pwd" - cdbAdminUser: - secret: - secretName: "cdb1-secret" - key: "cdbadmin_user" - cdbAdminPwd: - secret: - secretName: "cdb1-secret" - key: "cdbadmin_pwd" - webServerUser: - secret: - secretName: "cdb1-secret" - key: "webserver_user" - webServerPwd: - secret: - secretName: "cdb1-secret" - key: "webserver_pwd" - - - - --- Apply the .yaml file: - -% kubectl apply -f add_replica.yaml -cdb.database.oracle.com/cdb-dev configured - - - --- Check the status of the CDB CRD Pod(s): - -% kubectl get all -n oracle-database-operator-system -NAME READY STATUS RESTARTS AGE -pod/cdb-dev-ords-rs-5bztb 1/1 Running 0 21s << New Pod Added -pod/cdb-dev-ords-rs-q2b68 1/1 Running 0 7m40s -pod/oracle-database-operator-controller-manager-76cb674c5c-4nrh8 1/1 Running 0 4d10h -pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd 1/1 Running 1 4d10h -pod/oracle-database-operator-controller-manager-76cb674c5c-xsv9g 1/1 Running 2 4d10h - -NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE -service/cdb-dev-ords ClusterIP None 6m25s -service/oracle-database-operator-controller-manager-metrics-service ClusterIP 10.96.98.47 8443/TCP 5d2h -service/oracle-database-operator-webhook-service ClusterIP 10.96.166.163 443/TCP 5d2h - -NAME READY UP-TO-DATE AVAILABLE AGE -deployment.apps/oracle-database-operator-controller-manager 3/3 3 3 5d2h - -NAME DESIRED CURRENT READY AGE -replicaset.apps/cdb-dev-ords-rs 2 2 2 7m42s -replicaset.apps/oracle-database-operator-controller-manager-76cb674c5c 3 3 3 5d2h - - - - - --- Logs from Oracle DB Operator Pod: - -% kubectl logs -f pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd -n oracle-database-operator-system -. -. -2022-06-27T03:24:34Z DEBUG controller-runtime.webhook.webhooks received request {"webhook": "/validate-database-oracle-com-v1alpha1-cdb", "UID": "19a3fbb6-57e4-4ad2-92c9-a90bb66cefae", "kind": "database.oracle.com/v1alpha1, Kind=CDB", "resource": {"group":"database.oracle.com","version":"v1alpha1","resource":"cdbs"}} -2022-06-27T03:24:34Z INFO cdb-webhook validate update {"name": "cdb-dev"} -2022-06-27T03:24:34Z DEBUG controller-runtime.webhook.webhooks wrote response {"webhook": "/validate-database-oracle-com-v1alpha1-cdb", "code": 200, "reason": "", "UID": "19a3fbb6-57e4-4ad2-92c9-a90bb66cefae", "allowed": true} -2022-06-27T03:24:34Z INFO controllers.CDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:24:34Z INFO controllers.CDB Res Status: {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "Ready", "Status": "true"} -2022-06-27T03:24:34Z INFO controllers.CDB Existing Replicas: 1, New Replicas: 2 {"evaluateSpecChange": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:24:34Z INFO controllers.CDB Current Phase:ValidatingPods {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev"} -2022-06-27T03:24:34Z INFO controllers.CDB Validating Pod creation for :cdb-dev {"validateORDSPod": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:24:34Z INFO controller.cdb Executing Command : {"reconciler group": "database.oracle.com", "reconciler kind": "CDB", "name": "cdb-dev", "namespace": "oracle-database-operator-system", "ExecCommand": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:24:34Z INFO controller.cdb bash -c curl -sSkv -k -X GET https://localhost:8888/ords/_/db-api/stable/metadata-catalog/ || curl -sSkv -X GET http://localhost:8888/ords/_/db-api/stable/metadata-catalog/ {"reconciler group": "database.oracle.com", "reconciler kind": "CDB", "name": "cdb-dev", "namespace": "oracle-database-operator-system", "ExecCommand": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:24:35Z INFO controllers.CDB Replicas: 2 {"validateORDSPod": "oracle-database-operator-system/cdb-dev", "Ready Pods: ": 1} -2022-06-27T03:24:35Z INFO controllers.CDB Reconcile queued {"onpremdboperator": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:24:35Z INFO controllers.CDB DEFER {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "ValidatingPods", "Status": "false"} -2022-06-27T03:24:50Z INFO controllers.CDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:24:50Z INFO controllers.CDB Res Status: {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "ValidatingPods", "Status": "false"} -2022-06-27T03:24:50Z INFO controllers.CDB Current Phase:ValidatingPods {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev"} -2022-06-27T03:24:50Z INFO controllers.CDB Validating Pod creation for :cdb-dev {"validateORDSPod": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:24:50Z INFO controller.cdb Executing Command : {"reconciler group": "database.oracle.com", "reconciler kind": "CDB", "name": "cdb-dev", "namespace": "oracle-database-operator-system", "ExecCommand": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:24:50Z INFO controller.cdb bash -c curl -sSkv -k -X GET https://localhost:8888/ords/_/db-api/stable/metadata-catalog/ || curl -sSkv -X GET http://localhost:8888/ords/_/db-api/stable/metadata-catalog/ {"reconciler group": "database.oracle.com", "reconciler kind": "CDB", "name": "cdb-dev", "namespace": "oracle-database-operator-system", "ExecCommand": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:24:50Z INFO controller.cdb Executing Command : {"reconciler group": "database.oracle.com", "reconciler kind": "CDB", "name": "cdb-dev", "namespace": "oracle-database-operator-system", "ExecCommand": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:24:50Z INFO controller.cdb bash -c curl -sSkv -k -X GET https://localhost:8888/ords/_/db-api/stable/metadata-catalog/ || curl -sSkv -X GET http://localhost:8888/ords/_/db-api/stable/metadata-catalog/ {"reconciler group": "database.oracle.com", "reconciler kind": "CDB", "name": "cdb-dev", "namespace": "oracle-database-operator-system", "ExecCommand": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:24:50Z INFO controllers.CDB Replicas: 2 {"validateORDSPod": "oracle-database-operator-system/cdb-dev", "Ready Pods: ": 1} -2022-06-27T03:24:50Z INFO controllers.CDB Reconcile queued {"onpremdboperator": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:24:50Z INFO controllers.CDB DEFER {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "ValidatingPods", "Status": "false"} -2022-06-27T03:25:05Z INFO controllers.CDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:25:05Z INFO controllers.CDB Res Status: {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "ValidatingPods", "Status": "false"} -2022-06-27T03:25:05Z INFO controllers.CDB Current Phase:ValidatingPods {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev"} -2022-06-27T03:25:05Z INFO controllers.CDB Validating Pod creation for :cdb-dev {"validateORDSPod": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:25:05Z INFO controller.cdb Executing Command : {"reconciler group": "database.oracle.com", "reconciler kind": "CDB", "name": "cdb-dev", "namespace": "oracle-database-operator-system", "ExecCommand": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:25:05Z INFO controller.cdb bash -c curl -sSkv -k -X GET https://localhost:8888/ords/_/db-api/stable/metadata-catalog/ || curl -sSkv -X GET http://localhost:8888/ords/_/db-api/stable/metadata-catalog/ {"reconciler group": "database.oracle.com", "reconciler kind": "CDB", "name": "cdb-dev", "namespace": "oracle-database-operator-system", "ExecCommand": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:25:05Z INFO controller.cdb Executing Command : {"reconciler group": "database.oracle.com", "reconciler kind": "CDB", "name": "cdb-dev", "namespace": "oracle-database-operator-system", "ExecCommand": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:25:05Z INFO controller.cdb bash -c curl -sSkv -k -X GET https://localhost:8888/ords/_/db-api/stable/metadata-catalog/ || curl -sSkv -X GET http://localhost:8888/ords/_/db-api/stable/metadata-catalog/ {"reconciler group": "database.oracle.com", "reconciler kind": "CDB", "name": "cdb-dev", "namespace": "oracle-database-operator-system", "ExecCommand": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:25:06Z INFO controllers.CDB DEFER {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "CreatingService", "Status": "false"} -2022-06-27T03:25:21Z INFO controllers.CDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:25:21Z INFO controllers.CDB Res Status: {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "CreatingService", "Status": "false"} -2022-06-27T03:25:21Z INFO controllers.CDB Current Phase:CreatingService {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev"} -2022-06-27T03:25:21Z INFO controllers.CDB ORDS Cluster Service already exists {"createORDSSVC": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:25:21Z INFO controllers.CDB DEFER {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "Ready", "Status": "false"} -2022-06-27T03:25:36Z INFO controllers.CDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:25:36Z INFO controllers.CDB Res Status: {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "Ready", "Status": "false"} -2022-06-27T03:25:36Z INFO controllers.CDB Current Phase:Ready {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev"} -2022-06-27T03:25:36Z INFO controllers.CDB DEFER {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "Ready", "Status": "true"} - - - --- Logs of the newly added CDB CRD Pod: - -% kubectl logs -f pod/cdb-dev-ords-rs-5bztb -n oracle-database-operator-system - -Retrieving information. -Requires to login with administrator privileges to verify Oracle REST Data Services schema. - -Connecting to database user: SYS AS SYSDBA url: jdbc:oracle:thin:@//goldhost1.lbsub52b3b1cae.okecluster.oraclevcn.com:1521/goldcdb_phx1pw.lbsub52b3b1cae.okecluster.oraclevcn.com - -Retrieving information.. -Completed verifying Oracle REST Data Services schema version 21.4.3.r1170405. -2022-06-27T03:24:40.351Z INFO reloaded pools: [] -2022-06-27T03:24:40.353Z INFO Oracle REST Data Services schema version 21.4.3.r1170405 is installed. -spawn java -jar /opt/oracle/ords/ords.war user sql_admin SQL Administrator -Enter a password for user sql_admin: -Confirm password for user sql_admin: -2022-06-27T03:24:42.034Z INFO Created user: sql_admin in file: /opt/oracle/ords/config/ords/credentials -2022-06-27T03:24:43.666Z INFO Modified: /opt/oracle/ords/config/ords/conf/apex_pu.xml, updated properties: database.api.admin.enabled, db.cdb.adminUser, db.cdb.adminUser.password -2022-06-27T03:24:45.455Z INFO HTTP and HTTP/2 cleartext listening on host: localhost port: 8888 -2022-06-27T03:24:45.520Z INFO The document root is serving static resources located in: /opt/oracle/ords/doc_root -2022-06-27T03:24:47.515Z INFO Configuration properties for: |apex|pu| -db.servicename=goldcdb_phx1pw.lbsub52b3b1cae.okecluster.oraclevcn.com -db.hostname=goldhost1.lbsub52b3b1cae.okecluster.oraclevcn.com -database.api.admin.enabled=true -db.password=****** -db.cdb.adminUser.password=****** -database.api.enabled=true -db.cdb.adminUser=C##DBAPI_CDB_ADMIN as SYSDBA -db.username=ORDS_PUBLIC_USER -restEnabledSql.active=true -resource.templates.enabled=true -db.port=1521 -feature.sdw=true -db.connectionType=basic - -2022-06-27T03:24:47.517Z WARNING *** jdbc.MaxLimit in configuration |apex|pu| is using a value of 10, this setting may not be sized adequately for a production environment *** -2022-06-27T03:24:47.517Z WARNING *** jdbc.InitialLimit in configuration |apex|pu| is using a value of 3, this setting may not be sized adequately for a production environment *** -2022-06-27T03:24:51.761Z INFO Oracle REST Data Services initialized -Oracle REST Data Services version : 21.4.3.r1170405 -Oracle REST Data Services server info: jetty/9.4.44.v20210927 diff --git a/docs/multitenant/provisioning/add_replica.md b/docs/multitenant/provisioning/add_replica.md deleted file mode 100644 index 1315dc9f..00000000 --- a/docs/multitenant/provisioning/add_replica.md +++ /dev/null @@ -1,36 +0,0 @@ -# Add a new replicate to an existing CDB CRD Resource using Oracle DB Operator On-Prem Controller - -In this use case, using the Oracle Database Operator On-Prem Controller, you will add a new replica to an existing CDB CRD resource. - -**NOTE:** It is assumed that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-manage-pdb-life-cycle-using-oracle-db-operator-on-prem-database-controller) steps. - -This example uses `add_replica.yaml` with: - -- CDB CRD resource Name as `cdb-dev` -- Container Database (CDB) Name as `goldcdb` -- Scan Name as `goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com` -- Database Server Name as `goldhost1.lbsub52b3b1cae.okecluster.oraclevcn.com` -- ORDS Docker Image as `phx.ocir.io//oracle/ords:21.4.3` -- Database Listener Port as `1521` -- Number of replicas for CDB CRD Resource as 2. - -**NOTE:** For the details of the parameters to be used in the .yaml file, please refer [config/crd/bases/database.oracle.com_cdbs.yaml](../../../config/crd/bases/database.oracle.com_cdbs.yaml) - -Use the file: [add_replica.yaml](./add_replica.yaml) for this use case as below: - -1. Deploy the .yaml file: -```sh -[root@test-server oracle-database-operator]# kubectl apply -f add_replica.yaml -``` - -2. Monitor the Oracle DB Operator Pod for the progress of the CDB CRD Resource creation. - -NOTE: Check the DB Operator Pod name in your environment. - -``` -[root@test-server oracle-database-operator]# kubectl logs -f pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd -n oracle-database-operator-system -``` - -## Sample Output - -[Here](./add_replica.log) is the sample output for a PDB created using Oracle DB Operator On-Prem Controller using file [add_replica.yaml](./add_replica.yaml) diff --git a/docs/multitenant/provisioning/add_replica.yaml b/docs/multitenant/provisioning/add_replica.yaml deleted file mode 100644 index fac2d7ba..00000000 --- a/docs/multitenant/provisioning/add_replica.yaml +++ /dev/null @@ -1,40 +0,0 @@ -# -# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. -# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. -# -apiVersion: database.oracle.com/v1alpha1 -kind: CDB -metadata: - name: cdb-dev - namespace: oracle-database-operator-system -spec: - cdbName: "goldcdb" - dbServer: "goldhost1.lbsub52b3b1cae.okecluster.oraclevcn.com" - ordsImage: phx.ocir.io//oracle/ords:21.4.3 - dbPort: 1521 - replicas: 2 - serviceName: "goldcdb_phx1pw.lbsub52b3b1cae.okecluster.oraclevcn.com" - sysAdminPwd: - secret: - secretName: "cdb1-secret" - key: "sysadmin_pwd" - ordsPwd: - secret: - secretName: "cdb1-secret" - key: "ords_pwd" - cdbAdminUser: - secret: - secretName: "cdb1-secret" - key: "cdbadmin_user" - cdbAdminPwd: - secret: - secretName: "cdb1-secret" - key: "cdbadmin_pwd" - webServerUser: - secret: - secretName: "cdb1-secret" - key: "webserver_user" - webServerPwd: - secret: - secretName: "cdb1-secret" - key: "webserver_pwd" diff --git a/docs/multitenant/provisioning/cdb.log b/docs/multitenant/provisioning/cdb.log deleted file mode 100644 index 8c3cbdc5..00000000 --- a/docs/multitenant/provisioning/cdb.log +++ /dev/null @@ -1,279 +0,0 @@ --- Check the status of the Oracle DB Operator Pods: - -% kubectl get all -n oracle-database-operator-system -NAME READY STATUS RESTARTS AGE -pod/oracle-database-operator-controller-manager-76cb674c5c-4nrh8 1/1 Running 0 29h -pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd 1/1 Running 1 29h -pod/oracle-database-operator-controller-manager-76cb674c5c-xsv9g 1/1 Running 2 29h - -NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE -service/oracle-database-operator-controller-manager-metrics-service ClusterIP 10.96.98.47 8443/TCP 45h -service/oracle-database-operator-webhook-service ClusterIP 10.96.166.163 443/TCP 45h - -NAME READY UP-TO-DATE AVAILABLE AGE -deployment.apps/oracle-database-operator-controller-manager 3/3 3 3 45h - -NAME DESIRED CURRENT READY AGE -replicaset.apps/oracle-database-operator-controller-manager-76cb674c5c 3 3 3 45h - - - --- Get the base64 values for the required passwords as below: - -% echo -n "WElcome_21##" | base64 -V0VsY29tZV8yMSMj - -% echo -n "C##DBAPI_CDB_ADMIN" | base64 -QyMjREJBUElfQ0RCX0FETUlO - -% echo -n "sql_admin" | base64 -c3FsX2FkbWlu - -% echo -n "welcome1" | base64 -d2VsY29tZTE= - - --- Add the base64 encoded values against the required variables in cdb_secret.yaml file for this use case: - -% cat cdb_secret.yaml -# -# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. -# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. -# -apiVersion: v1 -kind: Secret -metadata: - name: cdb1-secret - namespace: oracle-database-operator-system -type: Opaque -data: - ords_pwd: "V0VsY29tZV8yMSMj" - sysadmin_pwd: "V0VsY29tZV8yMSMj" - cdbadmin_user: "QyMjREJBUElfQ0RCX0FETUlO" - cdbadmin_pwd: "V0VsY29tZV8yMSMj" - webserver_user: "c3FsX2FkbWlu" - webserver_pwd: "d2VsY29tZTE=" - - --- Check the contents of the cdb.yaml file: -% cat cdb.yaml -# -# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. -# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. -# -apiVersion: database.oracle.com/v1alpha1 -kind: CDB -metadata: - name: cdb-dev - namespace: oracle-database-operator-system -spec: - cdbName: "goldcdb" - scanName: "goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com" - dbServer: "goldhost1.lbsub52b3b1cae.okecluster.oraclevcn.com" - ordsImage: phx.ocir.io//oracle/ords:21.4.3 - ordsImagePullSecret: "container-registry-secret" - dbPort: 1521 - replicas: 1 - serviceName: "goldcdb_phx1pw.lbsub52b3b1cae.okecluster.oraclevcn.com" - sysAdminPwd: - secret: - secretName: "cdb1-secret" - key: "sysadmin_pwd" - ordsPwd: - secret: - secretName: "cdb1-secret" - key: "ords_pwd" - cdbAdminUser: - secret: - secretName: "cdb1-secret" - key: "cdbadmin_user" - cdbAdminPwd: - secret: - secretName: "cdb1-secret" - key: "cdbadmin_pwd" - webServerUser: - secret: - secretName: "cdb1-secret" - key: "webserver_user" - webServerPwd: - secret: - secretName: "cdb1-secret" - key: "webserver_pwd" - - - --- Apply the .yaml files: - -% kubectl apply -f cdb_secret.yaml -secret/cdb1-secret created - -% kubectl apply -f cdb.yaml -cdb.database.oracle.com/cdb-dev created - - - --- Monitor the Oracle DB Operator Pod during the period when the CDB CRD is getting deployed: - -% kubectl logs -f pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd -n oracle-database-operator-system -. -. -2022-06-27T03:16:44Z INFO controllers.CDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:16:44Z INFO controllers.CDB Res Status: {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "", "Status": "false"} -2022-06-27T03:16:44Z INFO controllers.CDB Adding finalizer {"manageCDBDeletion": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:16:44Z INFO controllers.CDB Current Phase: {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev"} -2022-06-27T03:16:44Z INFO controllers.CDB DEFAULT: {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "", "Status": "false"} -2022-06-27T03:16:44Z INFO controllers.CDB DEFER {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "Initializing", "Status": "false"} -2022-06-27T03:17:00Z INFO controllers.CDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:17:00Z INFO controllers.CDB Res Status: {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "Initializing", "Status": "false"} -2022-06-27T03:17:00Z INFO controllers.CDB Current Phase:Initializing {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev"} -2022-06-27T03:17:00Z INFO controllers.CDB Verified secrets successfully {"verifySecrets": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:17:00Z INFO controllers.CDB DEFER {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "CreatingPod", "Status": "false"} -2022-06-27T03:17:15Z INFO controllers.CDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:17:15Z INFO controllers.CDB Res Status: {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "CreatingPod", "Status": "false"} -2022-06-27T03:17:15Z INFO controllers.CDB Current Phase:CreatingPod {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev"} -2022-06-27T03:17:15Z INFO controllers.CDB Creating ORDS Replicaset: cdb-dev-ords-rs {"createORDSInstances": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:17:15Z INFO controllers.CDB Created ORDS ReplicaSet successfully {"createORDSInstances": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:17:15Z DEBUG events Normal {"object": {"kind":"CDB","namespace":"oracle-database-operator-system","name":"cdb-dev","uid":"c36e8d5f-6103-4a70-a840-16c6683755ec","apiVersion":"database.oracle.com/v1alpha1","resourceVersion":"101443216"}, "reason": "CreatedORDSReplicaSet", "message": "Created ORDS Replicaset (Replicas - 1) for cdb-dev"} -2022-06-27T03:17:15Z INFO controllers.CDB DEFER {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "ValidatingPods", "Status": "false"} -2022-06-27T03:17:30Z INFO controllers.CDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:17:30Z INFO controllers.CDB Res Status: {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "ValidatingPods", "Status": "false"} -2022-06-27T03:17:30Z INFO controllers.CDB Current Phase:ValidatingPods {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev"} -2022-06-27T03:17:30Z INFO controllers.CDB Validating Pod creation for :cdb-dev {"validateORDSPod": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:17:30Z INFO controller.cdb Executing Command : {"reconciler group": "database.oracle.com", "reconciler kind": "CDB", "name": "cdb-dev", "namespace": "oracle-database-operator-system", "ExecCommand": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:17:30Z INFO controller.cdb bash -c curl -sSkv -k -X GET https://localhost:8888/ords/_/db-api/stable/metadata-catalog/ || curl -sSkv -X GET http://localhost:8888/ords/_/db-api/stable/metadata-catalog/ {"reconciler group": "database.oracle.com", "reconciler kind": "CDB", "name": "cdb-dev", "namespace": "oracle-database-operator-system", "ExecCommand": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:17:30Z INFO controllers.CDB Replicas: 1 {"validateORDSPod": "oracle-database-operator-system/cdb-dev", "Ready Pods: ": 0} -2022-06-27T03:17:30Z INFO controllers.CDB Reconcile queued {"onpremdboperator": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:17:30Z INFO controllers.CDB DEFER {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "ValidatingPods", "Status": "false"} -2022-06-27T03:17:45Z INFO controllers.CDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:17:45Z INFO controllers.CDB Res Status: {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "ValidatingPods", "Status": "false"} -2022-06-27T03:17:45Z INFO controllers.CDB Current Phase:ValidatingPods {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev"} -2022-06-27T03:17:45Z INFO controllers.CDB Validating Pod creation for :cdb-dev {"validateORDSPod": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:17:45Z INFO controller.cdb Executing Command : {"reconciler group": "database.oracle.com", "reconciler kind": "CDB", "name": "cdb-dev", "namespace": "oracle-database-operator-system", "ExecCommand": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:17:45Z INFO controller.cdb bash -c curl -sSkv -k -X GET https://localhost:8888/ords/_/db-api/stable/metadata-catalog/ || curl -sSkv -X GET http://localhost:8888/ords/_/db-api/stable/metadata-catalog/ {"reconciler group": "database.oracle.com", "reconciler kind": "CDB", "name": "cdb-dev", "namespace": "oracle-database-operator-system", "ExecCommand": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:17:45Z INFO controllers.CDB Replicas: 1 {"validateORDSPod": "oracle-database-operator-system/cdb-dev", "Ready Pods: ": 0} -2022-06-27T03:17:45Z INFO controllers.CDB Reconcile queued {"onpremdboperator": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:17:45Z INFO controllers.CDB DEFER {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "ValidatingPods", "Status": "false"} -2022-06-27T03:18:00Z INFO controllers.CDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:18:00Z INFO controllers.CDB Res Status: {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "ValidatingPods", "Status": "false"} -2022-06-27T03:18:00Z INFO controllers.CDB Current Phase:ValidatingPods {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev"} -2022-06-27T03:18:00Z INFO controllers.CDB Validating Pod creation for :cdb-dev {"validateORDSPod": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:18:00Z INFO controller.cdb Executing Command : {"reconciler group": "database.oracle.com", "reconciler kind": "CDB", "name": "cdb-dev", "namespace": "oracle-database-operator-system", "ExecCommand": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:18:00Z INFO controller.cdb bash -c curl -sSkv -k -X GET https://localhost:8888/ords/_/db-api/stable/metadata-catalog/ || curl -sSkv -X GET http://localhost:8888/ords/_/db-api/stable/metadata-catalog/ {"reconciler group": "database.oracle.com", "reconciler kind": "CDB", "name": "cdb-dev", "namespace": "oracle-database-operator-system", "ExecCommand": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:18:01Z INFO controllers.CDB Replicas: 1 {"validateORDSPod": "oracle-database-operator-system/cdb-dev", "Ready Pods: ": 0} -2022-06-27T03:18:01Z INFO controllers.CDB Reconcile queued {"onpremdboperator": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:18:01Z INFO controllers.CDB DEFER {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "ValidatingPods", "Status": "false"} -2022-06-27T03:18:16Z INFO controllers.CDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:18:16Z INFO controllers.CDB Res Status: {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "ValidatingPods", "Status": "false"} -2022-06-27T03:18:16Z INFO controllers.CDB Current Phase:ValidatingPods {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev"} -2022-06-27T03:18:16Z INFO controllers.CDB Validating Pod creation for :cdb-dev {"validateORDSPod": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:18:16Z INFO controller.cdb Executing Command : {"reconciler group": "database.oracle.com", "reconciler kind": "CDB", "name": "cdb-dev", "namespace": "oracle-database-operator-system", "ExecCommand": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:18:16Z INFO controller.cdb bash -c curl -sSkv -k -X GET https://localhost:8888/ords/_/db-api/stable/metadata-catalog/ || curl -sSkv -X GET http://localhost:8888/ords/_/db-api/stable/metadata-catalog/ {"reconciler group": "database.oracle.com", "reconciler kind": "CDB", "name": "cdb-dev", "namespace": "oracle-database-operator-system", "ExecCommand": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:18:16Z INFO controllers.CDB DEFER {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "CreatingService", "Status": "false"} -2022-06-27T03:18:31Z INFO controllers.CDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:18:31Z INFO controllers.CDB Res Status: {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "CreatingService", "Status": "false"} -2022-06-27T03:18:31Z INFO controllers.CDB Current Phase:CreatingService {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev"} -2022-06-27T03:18:31Z INFO controllers.CDB Creating a new Cluster Service for: cdb-dev {"createORDSSVC": "oracle-database-operator-system/cdb-dev", "Svc.Namespace": "oracle-database-operator-system", "Service.Name": "cdb-dev-ords"} -2022-06-27T03:18:31Z INFO controllers.CDB Created ORDS Cluster Service successfully {"createORDSSVC": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:18:31Z DEBUG events Normal {"object": {"kind":"CDB","namespace":"oracle-database-operator-system","name":"cdb-dev","uid":"c36e8d5f-6103-4a70-a840-16c6683755ec","apiVersion":"database.oracle.com/v1alpha1","resourceVersion":"101443627"}, "reason": "CreatedORDSService", "message": "Created ORDS Service for cdb-dev"} -2022-06-27T03:18:31Z INFO controllers.CDB DEFER {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "Ready", "Status": "false"} -2022-06-27T03:18:46Z INFO controllers.CDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/cdb-dev"} -2022-06-27T03:18:46Z INFO controllers.CDB Res Status: {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "Ready", "Status": "false"} -2022-06-27T03:18:46Z INFO controllers.CDB Current Phase:Ready {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev"} -2022-06-27T03:18:46Z INFO controllers.CDB DEFER {"onpremdboperator": "oracle-database-operator-system/cdb-dev", "Name": "cdb-dev", "Phase": "Ready", "Status": "true"} - - - - --- Check the status of the CDB CRD Pod after few minutes of running above commands: - -% kubectl get all -n oracle-database-operator-system -NAME READY STATUS RESTARTS AGE -pod/cdb-dev-ords-rs-q2b68 1/1 Running 0 29s <<< CDB CRD Resource Pod -pod/oracle-database-operator-controller-manager-76cb674c5c-4nrh8 1/1 Running 0 4d10h -pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd 1/1 Running 1 4d10h -pod/oracle-database-operator-controller-manager-76cb674c5c-xsv9g 1/1 Running 2 4d10h - -NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE -service/oracle-database-operator-controller-manager-metrics-service ClusterIP 10.96.98.47 8443/TCP 5d1h -service/oracle-database-operator-webhook-service ClusterIP 10.96.166.163 443/TCP 5d1h - -NAME READY UP-TO-DATE AVAILABLE AGE -deployment.apps/oracle-database-operator-controller-manager 3/3 3 3 5d1h - -NAME DESIRED CURRENT READY AGE -replicaset.apps/cdb-dev-ords-rs 1 1 1 31s -replicaset.apps/oracle-database-operator-controller-manager-76cb674c5c 3 3 3 5d1h - - - - --- Check the logs of the CDB CRD Pod created above: - -% kubectl logs -f pod/cdb-dev-ords-rs-q2b68 -n oracle-database-operator-system -Requires to login with administrator privileges to verify Oracle REST Data Services schema. - -Connecting to database user: SYS AS SYSDBA url: jdbc:oracle:thin:@//goldhost1.lbsub52b3b1cae.okecluster.oraclevcn.com:1521/goldcdb_phx1pw.lbsub52b3b1cae.okecluster.oraclevcn.com - -Retrieving information.. -Your database connection is to a CDB. ORDS common user ORDS_PUBLIC_USER will be created in the CDB. ORDS schema will be installed in the PDBs. -Root CDB$ROOT - create ORDS common user -PDB PDB$SEED - install ORDS 21.4.3.r1170405 - -2022-06-27T03:17:22.015Z INFO reloaded pools: [] -2022-06-27T03:17:22.024Z INFO - -2022-06-27T03:17:23.200Z INFO Installing Oracle REST Data Services version 21.4.3.r1170405 in CDB$ROOT -2022-06-27T03:17:23.234Z INFO ... Log file written to /home/oracle/ords_cdb_install_core_CDB_ROOT_2022-06-27_031723_00234.log -2022-06-27T03:17:24.352Z INFO ... Verified database prerequisites -2022-06-27T03:17:24.662Z INFO ... Created Oracle REST Data Services proxy user -2022-06-27T03:17:24.708Z INFO Completed installation for Oracle REST Data Services version 21.4.3.r1170405. Elapsed time: 00:00:01.504 - -2022-06-27T03:17:24.722Z INFO Installing Oracle REST Data Services version 21.4.3.r1170405 in PDB$SEED -2022-06-27T03:17:24.731Z INFO ... Log file written to /home/oracle/ords_cdb_install_core_PDB_SEED_2022-06-27_031724_00731.log -2022-06-27T03:17:24.863Z INFO ... Verified database prerequisites -2022-06-27T03:17:25.123Z INFO ... Created Oracle REST Data Services proxy user -2022-06-27T03:17:25.568Z INFO ... Created Oracle REST Data Services schema -2022-06-27T03:17:26.252Z INFO ... Granted privileges to Oracle REST Data Services -2022-06-27T03:17:34.493Z INFO ... Created Oracle REST Data Services database objects -2022-06-27T03:17:43.730Z INFO ... Log file written to /home/oracle/ords_cdb_install_datamodel_PDB_SEED_2022-06-27_031743_00730.log -2022-06-27T03:17:45.883Z INFO ... Log file written to /home/oracle/ords_cdb_install_scheduler_PDB_SEED_2022-06-27_031745_00883.log -2022-06-27T03:17:49.296Z INFO ... Log file written to /home/oracle/ords_cdb_install_apex_PDB_SEED_2022-06-27_031749_00296.log -2022-06-27T03:17:51.492Z INFO Completed installation for Oracle REST Data Services version 21.4.3.r1170405. Elapsed time: 00:00:26.768 - -2022-06-27T03:17:51.492Z INFO Completed CDB installation for Oracle REST Data Services version 21.4.3.r1170405. Total elapsed time: 00:00:28.297 - -spawn java -jar /opt/oracle/ords/ords.war user sql_admin SQL Administrator -Enter a password for user sql_admin: -Confirm password for user sql_admin: -2022-06-27T03:17:53.192Z INFO Created user: sql_admin in file: /opt/oracle/ords/config/ords/credentials -2022-06-27T03:17:54.816Z INFO Modified: /opt/oracle/ords/config/ords/conf/apex_pu.xml, updated properties: database.api.admin.enabled, db.cdb.adminUser, db.cdb.adminUser.password -2022-06-27T03:17:56.583Z INFO HTTP and HTTP/2 cleartext listening on host: localhost port: 8888 -2022-06-27T03:17:56.647Z INFO The document root is serving static resources located in: /opt/oracle/ords/doc_root -2022-06-27T03:17:58.593Z INFO Configuration properties for: |apex|pu| -db.servicename=goldcdb_phx1pw.lbsub52b3b1cae.okecluster.oraclevcn.com -db.hostname=goldhost1.lbsub52b3b1cae.okecluster.oraclevcn.com -database.api.admin.enabled=true -db.password=****** -db.cdb.adminUser.password=****** -database.api.enabled=true -db.cdb.adminUser=C##DBAPI_CDB_ADMIN as SYSDBA -db.username=ORDS_PUBLIC_USER -restEnabledSql.active=true -resource.templates.enabled=true -db.port=1521 -feature.sdw=true -db.connectionType=basic - -2022-06-27T03:17:58.595Z WARNING *** jdbc.MaxLimit in configuration |apex|pu| is using a value of 10, this setting may not be sized adequately for a production environment *** -2022-06-27T03:17:58.595Z WARNING *** jdbc.InitialLimit in configuration |apex|pu| is using a value of 3, this setting may not be sized adequately for a production environment *** -2022-06-27T03:18:02.803Z INFO Oracle REST Data Services initialized -Oracle REST Data Services version : 21.4.3.r1170405 -Oracle REST Data Services server info: jetty/9.4.44.v20210927 - - - --- Check the CDB CRD Resource and it should be in "Ready" status for a successful deployment: - -% kubectl get cdbs -A -NAMESPACE NAME CDB NAME DB SERVER DB PORT SCAN NAME REPLICAS STATUS MESSAGE -oracle-database-operator-system cdb-dev goldcdb goldhost1.lbsub52b3b1cae.okecluster.oraclevcn.com 1521 goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com 1 Ready diff --git a/docs/multitenant/provisioning/cdb.yaml b/docs/multitenant/provisioning/cdb.yaml deleted file mode 100644 index 8e25a763..00000000 --- a/docs/multitenant/provisioning/cdb.yaml +++ /dev/null @@ -1,41 +0,0 @@ -# -# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. -# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. -# -apiVersion: database.oracle.com/v1alpha1 -kind: CDB -metadata: - name: cdb-dev - namespace: oracle-database-operator-system -spec: - cdbName: "goldcdb" - dbServer: "goldhost1.lbsub52b3b1cae.okecluster.oraclevcn.com" - ordsImage: phx.ocir.io//oracle/ords:21.4.3 - ordsImagePullSecret: "container-registry-secret" - dbPort: 1521 - replicas: 1 - serviceName: "goldcdb_phx1pw.lbsub52b3b1cae.okecluster.oraclevcn.com" - sysAdminPwd: - secret: - secretName: "cdb1-secret" - key: "sysadmin_pwd" - ordsPwd: - secret: - secretName: "cdb1-secret" - key: "ords_pwd" - cdbAdminUser: - secret: - secretName: "cdb1-secret" - key: "cdbadmin_user" - cdbAdminPwd: - secret: - secretName: "cdb1-secret" - key: "cdbadmin_pwd" - webServerUser: - secret: - secretName: "cdb1-secret" - key: "webserver_user" - webServerPwd: - secret: - secretName: "cdb1-secret" - key: "webserver_pwd" diff --git a/docs/multitenant/provisioning/cdb_crd_resource.md b/docs/multitenant/provisioning/cdb_crd_resource.md deleted file mode 100644 index f1f36404..00000000 --- a/docs/multitenant/provisioning/cdb_crd_resource.md +++ /dev/null @@ -1,38 +0,0 @@ -# Create a CDB CRD Resource using Oracle DB Operator On-Prem Controller - -In this use case, using the Oracle Database Operator On-Prem Controller, you will create the CDB kind as a custom resource that will model a CDB as a native Kubernetes object. - -**NOTE:** It is assumed that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-manage-pdb-life-cycle-using-oracle-db-operator-on-prem-database-controller) steps. - -This example uses `create_cdb.yaml` with: - -- CDB CRD resource Name as `cdb-dev` -- Container Database (CDB) Name as `goldcdb` -- Scan Name as `goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com` -- Database Server Name as `goldhost1.lbsub52b3b1cae.okecluster.oraclevcn.com` -- ORDS Docker Image as `phx.ocir.io//oracle/ords:21.4.3` -- Image Pull Secret as `container-registry-secret` -- Database Listener Port as `1521` -- Database Service Name as `goldcdb_phx1pw.lbsub52b3b1cae.okecluster.oraclevcn.com` -- Number of replicas for CDB CRD Resource as 1 - -**NOTE:** For the details of the parameters to be used in the .yaml file, please refer [config/crd/bases/database.oracle.com_cdbs.yaml](../../../config/crd/bases/database.oracle.com_cdbs.yaml) - -Use the file: [cdb.yaml](./cdb.yaml) for this use case as below: - -1. Deploy the .yaml file: -```sh -[root@test-server oracle-database-operator]# kubectl apply -f cdb.yaml -``` - -2. Monitor the Oracle DB Operator Pod for the progress of the CDB CRD Resource creation. - -NOTE: Check the DB Operator Pod name in your environment. - -``` -[root@test-server oracle-database-operator]# kubectl logs -f pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd -n oracle-database-operator-system -``` - -## Sample Output - -[Here](./cdb.log) is the sample output for a PDB created using Oracle DB Operator On-Prem Controller using file [cdb.yaml](./cdb.yaml) diff --git a/docs/multitenant/provisioning/cdb_secret.yaml b/docs/multitenant/provisioning/cdb_secret.yaml deleted file mode 100644 index 4d03499e..00000000 --- a/docs/multitenant/provisioning/cdb_secret.yaml +++ /dev/null @@ -1,17 +0,0 @@ -# -# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. -# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. -# -apiVersion: v1 -kind: Secret -metadata: - name: cdb1-secret - namespace: oracle-database-operator-system -type: Opaque -data: - ords_pwd: "[ base64 encode values ]" - sysadmin_pwd: "[ base64 encode values ]" - cdbadmin_user: "[ base64 encode values ]" - cdbadmin_pwd: "[ base64 encode values ]" - webserver_user: "[ base64 encode values ]" - webserver_pwd: "[base64 encode values ]" diff --git a/docs/multitenant/provisioning/clone_pdb.log b/docs/multitenant/provisioning/clone_pdb.log deleted file mode 100644 index 54b54ef6..00000000 --- a/docs/multitenant/provisioning/clone_pdb.log +++ /dev/null @@ -1,137 +0,0 @@ --- Check the Oracle DB Operator Pod and CDB CRD Pod status: - -% kubectl get all -n oracle-database-operator-system -NAME READY STATUS RESTARTS AGE -pod/cdb-dev-ords-rs-5bztb 1/1 Running 0 5m23s -pod/cdb-dev-ords-rs-q2b68 1/1 Running 0 12m -pod/oracle-database-operator-controller-manager-76cb674c5c-4nrh8 1/1 Running 0 4d10h -pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd 1/1 Running 1 4d10h -pod/oracle-database-operator-controller-manager-76cb674c5c-xsv9g 1/1 Running 2 4d10h - -NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE -service/cdb-dev-ords ClusterIP None 11m -service/oracle-database-operator-controller-manager-metrics-service ClusterIP 10.96.98.47 8443/TCP 5d2h -service/oracle-database-operator-webhook-service ClusterIP 10.96.166.163 443/TCP 5d2h - -NAME READY UP-TO-DATE AVAILABLE AGE -deployment.apps/oracle-database-operator-controller-manager 3/3 3 3 5d2h - -NAME DESIRED CURRENT READY AGE -replicaset.apps/cdb-dev-ords-rs 2 2 2 12m -replicaset.apps/oracle-database-operator-controller-manager-76cb674c5c 3 3 3 5d2h - - --- Check the current PDB CRD resource: - -% kubectl get pdbs -A -NAMESPACE NAME CONNECT STRING CDB NAME PDB NAME PDB STATE PDB SIZE STATUS MESSAGE -oracle-database-operator-system pdb1 goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com:1521/pdbnew goldcdb pdbnew READ WRITE 1G Ready Success - - --- Check the current PDBs in the target CDB: - -SQL> show pdbs - - CON_ID CON_NAME OPEN MODE RESTRICTED ----------- ------------------------------ ---------- ---------- - 2 PDB$SEED READ WRITE NO - 3 PDBNEW READ WRITE NO - - - --- .yaml file used in this use case to clone a PDB: - -% cat clone_pdb.yaml -# -# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. -# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. -# -apiVersion: database.oracle.com/v1alpha1 -kind: PDB -metadata: - name: pdb1-clone - namespace: oracle-database-operator-system - labels: - cdb: cdb-dev -spec: - cdbResName: "cdb-dev" - cdbName: "goldcdb" - pdbName: "pdbnewclone" - srcPdbName: "pdbnew" - fileNameConversions: "NONE" - totalSize: "UNLIMITED" - tempSize: "UNLIMITED" - action: "Clone" - - --- Apply the .yaml file: - -% kubectl apply -f clone_pdb.yaml -pdb.database.oracle.com/pdb1-clone created - - - -- Monitor the logs from the Oracle DB Operator Pod: - -% kubectl logs -f pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd -n oracle-database-operator-system -. -. -2022-06-27T03:37:21Z DEBUG controller-runtime.webhook.webhooks received request {"webhook": "/mutate-database-oracle-com-v1alpha1-pdb", "UID": "7fbbd983-0309-4603-9c3e-77f7ffded000", "kind": "database.oracle.com/v1alpha1, Kind=PDB", "resource": {"group":"database.oracle.com","version":"v1alpha1","resource":"pdbs"}} -2022-06-27T03:37:21Z INFO pdb-webhook Setting default values in PDB spec for : pdb1-clone -2022-06-27T03:37:21Z INFO pdb-webhook - reuseTempFile : true -2022-06-27T03:37:21Z INFO pdb-webhook - unlimitedStorage : true -2022-06-27T03:37:21Z INFO pdb-webhook - tdeImport : false -2022-06-27T03:37:21Z INFO pdb-webhook - tdeExport : false -2022-06-27T03:37:21Z INFO pdb-webhook - asClone : false -2022-06-27T03:37:21Z INFO pdb-webhook - getScript : false -2022-06-27T03:37:21Z DEBUG controller-runtime.webhook.webhooks wrote response {"webhook": "/mutate-database-oracle-com-v1alpha1-pdb", "code": 200, "reason": "", "UID": "7fbbd983-0309-4603-9c3e-77f7ffded000", "allowed": true} -2022-06-27T03:37:21Z DEBUG controller-runtime.webhook.webhooks received request {"webhook": "/validate-database-oracle-com-v1alpha1-pdb", "UID": "12aa43ea-df81-4931-b29b-d665c121590f", "kind": "database.oracle.com/v1alpha1, Kind=PDB", "resource": {"group":"database.oracle.com","version":"v1alpha1","resource":"pdbs"}} -2022-06-27T03:37:21Z INFO pdb-webhook ValidateCreate-Validating PDB spec for : pdb1-clone -2022-06-27T03:37:21Z INFO pdb-webhook validateCommon {"name": "pdb1-clone"} -2022-06-27T03:37:21Z INFO pdb-webhook Valdiating PDB Resource Action : CLONE -2022-06-27T03:37:21Z INFO pdb-webhook PDB Resource : pdb1-clone successfully validated for Action : CLONE -2022-06-27T03:37:21Z DEBUG controller-runtime.webhook.webhooks wrote response {"webhook": "/validate-database-oracle-com-v1alpha1-pdb", "code": 200, "reason": "", "UID": "12aa43ea-df81-4931-b29b-d665c121590f", "allowed": true} -2022-06-27T03:37:21Z INFO controllers.PDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/pdb1-clone"} -2022-06-27T03:37:21Z INFO controllers.PDB Adding finalizer {"managePDBDeletion": "oracle-database-operator-system/pdb1-clone"} -2022-06-27T03:37:21Z DEBUG controller-runtime.webhook.webhooks received request {"webhook": "/mutate-database-oracle-com-v1alpha1-pdb", "UID": "b3bbaa0d-6865-4dd1-9ca9-d54f916a8d66", "kind": "database.oracle.com/v1alpha1, Kind=PDB", "resource": {"group":"database.oracle.com","version":"v1alpha1","resource":"pdbs"}} -2022-06-27T03:37:21Z INFO pdb-webhook Setting default values in PDB spec for : pdb1-clone -2022-06-27T03:37:21Z DEBUG controller-runtime.webhook.webhooks wrote response {"webhook": "/mutate-database-oracle-com-v1alpha1-pdb", "code": 200, "reason": "", "UID": "b3bbaa0d-6865-4dd1-9ca9-d54f916a8d66", "allowed": true} -2022-06-27T03:37:21Z DEBUG controller-runtime.webhook.webhooks received request {"webhook": "/validate-database-oracle-com-v1alpha1-pdb", "UID": "5924d376-7a5c-4d4a-8ca9-040c585fb4b6", "kind": "database.oracle.com/v1alpha1, Kind=PDB", "resource": {"group":"database.oracle.com","version":"v1alpha1","resource":"pdbs"}} -2022-06-27T03:37:21Z INFO pdb-webhook ValidateUpdate-Validating PDB spec for : pdb1-clone -2022-06-27T03:37:21Z INFO pdb-webhook validateCommon {"name": "pdb1-clone"} -2022-06-27T03:37:21Z INFO pdb-webhook Valdiating PDB Resource Action : CLONE -2022-06-27T03:37:21Z DEBUG controller-runtime.webhook.webhooks wrote response {"webhook": "/validate-database-oracle-com-v1alpha1-pdb", "code": 200, "reason": "", "UID": "5924d376-7a5c-4d4a-8ca9-040c585fb4b6", "allowed": true} -2022-06-27T03:37:21Z INFO controllers.PDB Found PDB: pdb1-clone {"checkDuplicatePDB": "oracle-database-operator-system/pdb1-clone"} -2022-06-27T03:37:21Z INFO controllers.PDB Validating PDB phase for: pdb1-clone {"validatePhase": "oracle-database-operator-system/pdb1-clone", "Action": "CLONE"} -2022-06-27T03:37:21Z INFO controllers.PDB Validation complete {"validatePhase": "oracle-database-operator-system/pdb1-clone"} -2022-06-27T03:37:21Z INFO controllers.PDB PDB: {"onpremdboperator": "oracle-database-operator-system/pdb1-clone", "Name": "pdb1-clone", "Phase": "Cloning", "Status": "false"} -2022-06-27T03:37:21Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1-clone", "Name": "cdb-dev", "CR Name": "cdb-dev"} -2022-06-27T03:37:21Z INFO controllers.PDB Issuing REST call {"callAPI": "oracle-database-operator-system/pdb1-clone", "URL": "http://cdb-dev-ords:8888/ords/_/db-api/latest/database/pdbs/pdbnew/", "Action": "POST"} -2022-06-27T03:37:21Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1-clone", "Name": "cdb-dev", "CR Name": "cdb-dev"} -2022-06-27T03:38:01Z INFO controllers.PDB Cloned PDB successfully {"clonePDB": "oracle-database-operator-system/pdb1-clone", "Source PDB Name": "pdbnew", "Clone PDB Name": "pdbnewclone"} -2022-06-27T03:38:01Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1-clone", "Name": "cdb-dev", "CR Name": "cdb-dev"} -2022-06-27T03:38:01Z DEBUG events Normal {"object": {"kind":"PDB","namespace":"oracle-database-operator-system","name":"pdb1-clone","uid":"16276c26-60a3-463b-bdd5-10bd328f9d43","apiVersion":"database.oracle.com/v1alpha1","resourceVersion":"101449482"}, "reason": "Created", "message": "PDB 'pdbnewclone' cloned successfully"} -2022-06-27T03:38:01Z INFO controllers.PDB Issuing REST call {"callAPI": "oracle-database-operator-system/pdb1-clone", "URL": "http://cdb-dev-ords:8888/ords/_/db-api/latest/database/pdbs/pdbnewclone/status", "Action": "GET"} -2022-06-27T03:38:01Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1-clone", "Name": "cdb-dev", "CR Name": "cdb-dev"} -2022-06-27T03:38:01Z INFO controllers.PDB Successfully obtained PDB state {"getPDBState": "oracle-database-operator-system/pdb1-clone", "PDB Name": "pdbnewclone", "State": "READ WRITE"} -2022-06-27T03:38:01Z INFO controllers.PDB Reconcile completed {"onpremdboperator": "oracle-database-operator-system/pdb1-clone"} - - - --- Check the status of the new PDB CRD resource created by cloning: - -% kubectl get pdbs -A -NAMESPACE NAME CONNECT STRING CDB NAME PDB NAME PDB STATE PDB SIZE STATUS MESSAGE -oracle-database-operator-system pdb1 goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com:1521/pdbnew goldcdb pdbnew READ WRITE 1G Ready Success -oracle-database-operator-system pdb1-clone goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com:1521/pdbnewclone goldcdb pdbnewclone READ WRITE Ready Success - - --- Verify the new PDB created from the target CDB: - -SQL> show pdbs - - CON_ID CON_NAME OPEN MODE RESTRICTED ----------- ------------------------------ ---------- ---------- - 2 PDB$SEED READ WRITE NO - 3 PDBNEW READ WRITE NO - 4 PDBNEWCLONE READ WRITE NO diff --git a/docs/multitenant/provisioning/clone_pdb.md b/docs/multitenant/provisioning/clone_pdb.md deleted file mode 100644 index b731d301..00000000 --- a/docs/multitenant/provisioning/clone_pdb.md +++ /dev/null @@ -1,38 +0,0 @@ -# Clone a PDB using Oracle DB Operator On-Prem Controller in a target CDB - -In this use case, a PDB is cloned using Oracle DB Operator On-Prem controller. - -To clone a PDB CRD Resource, a sample .yaml file is available here: [config/samples/onpremdb/pdb_clone.yaml](../../../config/samples/onpremdb/pdb_clone.yaml) - -**NOTE:** It is assumed that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-manage-pdb-life-cycle-using-oracle-db-operator-on-prem-database-controller) steps. - -This example uses `clone_pdb.yaml` to clone a PDB using Oracle DB Operator On-Prem Controller with: - -- PDB CRD resource Name as `pdb1-clone` -- Pluggable Database (PDB) Name as `pdbnewclone` -- Total Size of the PDB as `UNLIMITED` -- Total size for temporary tablespace as `UNLIMITED` -- Target CDB CRD Resource Name as `cdb-dev` -- Target CDB name as `goldcdb` -- Source PDB Name as `pdbnew` - -**NOTE:** For the details of the parameters to be used in the .yaml file, please refer [config/crd/bases/database.oracle.com_pdbs.yaml](../../../config/crd/bases/database.oracle.com_pdbs.yaml) - -Use the file: [clone_pdb.yaml](./clone_pdb.yaml) for this use case as below: - -1. Deploy the .yaml file: -```sh -[root@test-server oracle-database-operator]# kubectl apply -f clone_pdb.yaml -``` - -2. Monitor the Oracle DB Operator Pod for the progress of the PDB creation. - -NOTE: Check the DB Operator Pod name in your environment. - -``` -[root@test-server oracle-database-operator]# kubectl logs -f pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd -n oracle-database-operator-system -``` - -## Sample Output - -[Here](./clone_pdb.log) is the sample output for a PDB created using Oracle DB Operator On-Prem Controller using file [clone_pdb.yaml](./clone_pdb.yaml) diff --git a/docs/multitenant/provisioning/clone_pdb.yaml b/docs/multitenant/provisioning/clone_pdb.yaml deleted file mode 100644 index 7d3cfff9..00000000 --- a/docs/multitenant/provisioning/clone_pdb.yaml +++ /dev/null @@ -1,20 +0,0 @@ -# -# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. -# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. -# -apiVersion: database.oracle.com/v1alpha1 -kind: PDB -metadata: - name: pdb1-clone - namespace: oracle-database-operator-system - labels: - cdb: cdb-dev -spec: - cdbResName: "cdb-dev" - cdbName: "goldcdb" - pdbName: "pdbnewclone" - srcPdbName: "pdbnew" - fileNameConversions: "NONE" - totalSize: "UNLIMITED" - tempSize: "UNLIMITED" - action: "Clone" diff --git a/docs/multitenant/provisioning/create_pdb.log b/docs/multitenant/provisioning/create_pdb.log deleted file mode 100644 index ebba9e66..00000000 --- a/docs/multitenant/provisioning/create_pdb.log +++ /dev/null @@ -1,139 +0,0 @@ --- Check the Oracle DB Operator Pod and CDB CRD Pod status: - -% kubectl get all -n oracle-database-operator-system -NAME READY STATUS RESTARTS AGE -pod/cdb-dev-ords-rs-5bztb 1/1 Running 0 5m23s -pod/cdb-dev-ords-rs-q2b68 1/1 Running 0 12m -pod/oracle-database-operator-controller-manager-76cb674c5c-4nrh8 1/1 Running 0 4d10h -pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd 1/1 Running 1 4d10h -pod/oracle-database-operator-controller-manager-76cb674c5c-xsv9g 1/1 Running 2 4d10h - -NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE -service/cdb-dev-ords ClusterIP None 11m -service/oracle-database-operator-controller-manager-metrics-service ClusterIP 10.96.98.47 8443/TCP 5d2h -service/oracle-database-operator-webhook-service ClusterIP 10.96.166.163 443/TCP 5d2h - -NAME READY UP-TO-DATE AVAILABLE AGE -deployment.apps/oracle-database-operator-controller-manager 3/3 3 3 5d2h - -NAME DESIRED CURRENT READY AGE -replicaset.apps/cdb-dev-ords-rs 2 2 2 12m -replicaset.apps/oracle-database-operator-controller-manager-76cb674c5c 3 3 3 5d2h - - - --- PDB secrets in this use case were created using the below file: - -% cat pdb_secret.yaml -# -# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. -# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. -# -apiVersion: v1 -kind: Secret -metadata: - name: pdb1-secret - namespace: oracle-database-operator-system -type: Opaque -data: - sysadmin_user: "cGRiYWRtaW4=" - sysadmin_pwd: "V0VsY29tZV8yMSMj" - - --- This is the .yaml file used to create a PDB CRD resource in this use case: - -% cat pdb_create.yaml -# -# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. -# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. -# -apiVersion: database.oracle.com/v1alpha1 -kind: PDB -metadata: - name: pdb1 - namespace: oracle-database-operator-system - labels: - cdb: cdb-dev -spec: - cdbResName: "cdb-dev" - cdbName: "goldcdb" - pdbName: "pdbnew" - adminName: - secret: - secretName: "pdb1-secret" - key: "sysadmin_user" - adminPwd: - secret: - secretName: "pdb1-secret" - key: "sysadmin_pwd" - fileNameConversions: "NONE" - totalSize: "1G" - tempSize: "100M" - action: "Create" - - - - --- Apply the .yaml files: - -% kubectl apply -f pdb_secret.yaml -secret/pdb1-secret created - -% kubectl apply -f pdb_create.yaml -pdb.database.oracle.com/pdb1 created - - --- Monitor the logs from the Oracle DB Operator Pod: - -% kubectl logs -f pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd -n oracle-database-operator-system -. -. -2022-06-27T03:28:30Z DEBUG controller-runtime.webhook.webhooks received request {"webhook": "/mutate-database-oracle-com-v1alpha1-pdb", "UID": "6fca0e37-8fd9-4ccd-86ad-2edec604a28b", "kind": "database.oracle.com/v1alpha1, Kind=PDB", "resource": {"group":"database.oracle.com","version":"v1alpha1","resource":"pdbs"}} -2022-06-27T03:28:30Z INFO pdb-webhook Setting default values in PDB spec for : pdb1 -2022-06-27T03:28:30Z INFO pdb-webhook - reuseTempFile : true -2022-06-27T03:28:30Z INFO pdb-webhook - unlimitedStorage : true -2022-06-27T03:28:30Z INFO pdb-webhook - tdeImport : false -2022-06-27T03:28:30Z INFO pdb-webhook - tdeExport : false -2022-06-27T03:28:30Z INFO pdb-webhook - asClone : false -2022-06-27T03:28:30Z DEBUG controller-runtime.webhook.webhooks wrote response {"webhook": "/mutate-database-oracle-com-v1alpha1-pdb", "code": 200, "reason": "", "UID": "6fca0e37-8fd9-4ccd-86ad-2edec604a28b", "allowed": true} -2022-06-27T03:28:30Z DEBUG controller-runtime.webhook.webhooks received request {"webhook": "/validate-database-oracle-com-v1alpha1-pdb", "UID": "6ae20043-fa1a-4eba-b943-a2266183da48", "kind": "database.oracle.com/v1alpha1, Kind=PDB", "resource": {"group":"database.oracle.com","version":"v1alpha1","resource":"pdbs"}} -2022-06-27T03:28:30Z INFO pdb-webhook ValidateUpdate-Validating PDB spec for : pdb1 -2022-06-27T03:28:30Z INFO pdb-webhook validateCommon {"name": "pdb1"} -2022-06-27T03:28:30Z INFO pdb-webhook Valdiating PDB Resource Action : CREATE -2022-06-27T03:28:30Z DEBUG controller-runtime.webhook.webhooks wrote response {"webhook": "/validate-database-oracle-com-v1alpha1-pdb", "code": 200, "reason": "", "UID": "6ae20043-fa1a-4eba-b943-a2266183da48", "allowed": true} -2022-06-27T03:28:30Z INFO controllers.PDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/pdb1"} -2022-06-27T03:28:30Z INFO controllers.PDB Validating PDB phase for: pdb1 {"validatePhase": "oracle-database-operator-system/pdb1", "Action": "CREATE"} -2022-06-27T03:28:30Z INFO controllers.PDB Validation complete {"validatePhase": "oracle-database-operator-system/pdb1"} -2022-06-27T03:28:30Z INFO controllers.PDB PDB: {"onpremdboperator": "oracle-database-operator-system/pdb1", "Name": "pdb1", "Phase": "Creating", "Status": "false"} -2022-06-27T03:28:30Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} -2022-06-27T03:28:30Z INFO controllers.PDB Issuing REST call {"callAPI": "oracle-database-operator-system/pdb1", "URL": "http://cdb-dev-ords:8888/ords/_/db-api/latest/database/pdbs/", "Action": "POST"} -2022-06-27T03:28:30Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} -2022-06-27T03:29:04Z INFO controllers.PDB Created PDB Resource {"createPDB": "oracle-database-operator-system/pdb1", "PDB Name": "pdbnew"} -2022-06-27T03:29:04Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} -2022-06-27T03:29:04Z DEBUG events Normal {"object": {"kind":"PDB","namespace":"oracle-database-operator-system","name":"pdb1","uid":"81f2e686-6e1b-4e2c-8a2f-e20c2f99d6b9","apiVersion":"database.oracle.com/v1alpha1","resourceVersion":"101446779"}, "reason": "Created", "message": "PDB 'pdbnew' created successfully"} -2022-06-27T03:29:04Z INFO controllers.PDB Issuing REST call {"callAPI": "oracle-database-operator-system/pdb1", "URL": "http://cdb-dev-ords:8888/ords/_/db-api/latest/database/pdbs/pdbnew/status", "Action": "GET"} -2022-06-27T03:29:04Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} -2022-06-27T03:29:04Z INFO controllers.PDB Successfully obtained PDB state {"getPDBState": "oracle-database-operator-system/pdb1", "PDB Name": "pdbnew", "State": "READ WRITE"} -2022-06-27T03:29:04Z INFO controllers.PDB Reconcile completed {"onpremdboperator": "oracle-database-operator-system/pdb1"} - - - - --- Check the status of the PDB CRD Resource created: - -% kubectl get pdbs -A -NAMESPACE NAME CONNECT STRING CDB NAME PDB NAME PDB STATE PDB SIZE STATUS MESSAGE -oracle-database-operator-system pdb1 goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com:1521/pdbnew goldcdb pdbnew READ WRITE 1G Ready Success - - --- The status as "Ready" and message as "Success" confirms that the resource has been created successfully. - - --- Verify that the PDB is created from CDB: - -SQL> show pdbs - - CON_ID CON_NAME OPEN MODE RESTRICTED ----------- ------------------------------ ---------- ---------- - 2 PDB$SEED READ WRITE NO - 3 PDBNEW READ WRITE NO diff --git a/docs/multitenant/provisioning/create_pdb.md b/docs/multitenant/provisioning/create_pdb.md deleted file mode 100644 index 4f6e57aa..00000000 --- a/docs/multitenant/provisioning/create_pdb.md +++ /dev/null @@ -1,37 +0,0 @@ -# Create a PDB using Oracle DB Operator On-Prem Controller in a target CDB - -The Oracle Database Operator On-Prem Controller creates the PDB kind as a custom resource that models a PDB as a native Kubernetes object. There is a one-to-one mapping between the actual PDB and the Kubernetes PDB Custom Resource. Each PDB resource follows the PDB CRD as defined here: [config/crd/bases/database.oracle.com_pdbs.yaml](../../../config/crd/bases/database.oracle.com_pdbs.yaml) - -To create a PDB CRD Resource, a sample .yaml file is available here: [config/samples/onpremdb/pdb_create.yaml](../../../config/samples/onpremdb/pdb_create.yaml) - -**NOTE:** It is assumed that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-manage-pdb-life-cycle-using-oracle-db-operator-on-prem-database-controller) steps. - -This example uses `create_pdb.yaml` to create a PDB using Oracle DB Operator On-Prem Controller with: - -- PDB CRD resource Name as `pdb1` -- Pluggable Database (PDB) Name as `pdbnew` -- Total Size of the PDB as `1GB` -- Total size for temporary tablespace as `100M` -- Target CDB CRD Resource Name as `cdb-dev` -- Target CDB name as `goldcdb` - -**NOTE:** For the details of the parameters to be used in the .yaml file, please refer [config/crd/bases/database.oracle.com_pdbs.yaml](../../../config/crd/bases/database.oracle.com_pdbs.yaml) - -Use the file: [create_pdb.yaml](./create_pdb.yaml) for this use case as below: - -1. Deploy the .yaml file: -```sh -[root@test-server oracle-database-operator]# kubectl apply -f create_pdb.yaml -``` - -2. Monitor the Oracle DB Operator Pod for the progress of the PDB creation. - -NOTE: Check the DB Operator Pod name in your environment. - -``` -[root@test-server oracle-database-operator]# kubectl logs -f pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd -n oracle-database-operator-system -``` - -## Sample Output - -[Here](./create_pdb.log) is the sample output for a PDB created using Oracle DB Operator On-Prem Controller using file [create_pdb.yaml](./create_pdb.yaml) diff --git a/docs/multitenant/provisioning/create_pdb.yaml b/docs/multitenant/provisioning/create_pdb.yaml deleted file mode 100644 index 82941185..00000000 --- a/docs/multitenant/provisioning/create_pdb.yaml +++ /dev/null @@ -1,27 +0,0 @@ -# -# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. -# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. -# -apiVersion: database.oracle.com/v1alpha1 -kind: PDB -metadata: - name: pdb1 - namespace: oracle-database-operator-system - labels: - cdb: cdb-dev -spec: - cdbResName: "cdb-dev" - cdbName: "goldcdb" - pdbName: "pdbnew" - adminName: - secret: - secretName: "pdb1-secret" - key: "sysadmin_user" - adminPwd: - secret: - secretName: "pdb1-secret" - key: "sysadmin_pwd" - fileNameConversions: "NONE" - totalSize: "1G" - tempSize: "100M" - action: "Create" diff --git a/docs/multitenant/provisioning/delete_pdb.log b/docs/multitenant/provisioning/delete_pdb.log deleted file mode 100644 index 7f361871..00000000 --- a/docs/multitenant/provisioning/delete_pdb.log +++ /dev/null @@ -1,157 +0,0 @@ --- Check the existing PDB CRD resources: - -% kubectl get pdbs -A -NAMESPACE NAME CONNECT STRING CDB NAME PDB NAME PDB STATE PDB SIZE STATUS MESSAGE -oracle-database-operator-system pdb1 goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com:1521/pdbnew goldcdb pdbnew READ WRITE 1G Ready Success -oracle-database-operator-system pdb1-clone goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com:1521/pdbnewclone goldcdb pdbnewclone READ WRITE Ready Success - - --- Also check from the database as well: - -SQL> show pdbs - - CON_ID CON_NAME OPEN MODE RESTRICTED ----------- ------------------------------ ---------- ---------- - 2 PDB$SEED READ WRITE NO - 3 PDBNEW READ WRITE NO - 4 PDBNEWCLONE READ WRITE NO - - - -% cat modify_pdb_close.yaml -# -# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. -# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. -# -apiVersion: database.oracle.com/v1alpha1 -kind: PDB -metadata: - name: pdb1-clone - namespace: oracle-database-operator-system - labels: - cdb: cdb-dev -spec: - cdbResName: "cdb-dev" - cdbName: "goldcdb" - pdbName: "pdbnewclone" - pdbState: "CLOSE" - modifyOption: "IMMEDIATE" - action: "Modify" - - -% kubectl apply -f modify_pdb_close.yaml -pdb.database.oracle.com/pdb1-clone configured - - - -% kubectl logs -f pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd -n oracle-database-operator-system -. -. -2022-06-27T04:19:36Z DEBUG controller-runtime.webhook.webhooks received request {"webhook": "/mutate-database-oracle-com-v1alpha1-pdb", "UID": "24842cc8-0047-46cc-86a5-2782a95e3e36", "kind": "database.oracle.com/v1alpha1, Kind=PDB", "resource": {"group":"database.oracle.com","version":"v1alpha1","resource":"pdbs"}} -2022-06-27T04:19:36Z INFO pdb-webhook Setting default values in PDB spec for : pdb1-clone -2022-06-27T04:19:36Z DEBUG controller-runtime.webhook.webhooks wrote response {"webhook": "/mutate-database-oracle-com-v1alpha1-pdb", "code": 200, "reason": "", "UID": "24842cc8-0047-46cc-86a5-2782a95e3e36", "allowed": true} -2022-06-27T04:19:36Z INFO controllers.PDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/pdb1-clone"} -2022-06-27T04:19:36Z INFO controllers.PDB Validating PDB phase for: pdb1-clone {"validatePhase": "oracle-database-operator-system/pdb1-clone", "Action": "MODIFY"} -2022-06-27T04:19:36Z INFO controllers.PDB Validation complete {"validatePhase": "oracle-database-operator-system/pdb1-clone"} -2022-06-27T04:19:36Z INFO controllers.PDB PDB: {"onpremdboperator": "oracle-database-operator-system/pdb1-clone", "Name": "pdb1-clone", "Phase": "Modifying", "Status": "false"} -2022-06-27T04:19:36Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1-clone", "Name": "cdb-dev", "CR Name": "cdb-dev"} -2022-06-27T04:19:36Z INFO controllers.PDB Issuing REST call {"callAPI": "oracle-database-operator-system/pdb1-clone", "URL": "http://cdb-dev-ords:8888/ords/_/db-api/latest/database/pdbs/pdbnewclone/status", "Action": "GET"} -2022-06-27T04:19:36Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1-clone", "Name": "cdb-dev", "CR Name": "cdb-dev"} -2022-06-27T04:19:38Z INFO controllers.PDB Successfully obtained PDB state {"getPDBState": "oracle-database-operator-system/pdb1-clone", "PDB Name": "pdbnewclone", "State": "READ WRITE"} -2022-06-27T04:19:38Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1-clone", "Name": "cdb-dev", "CR Name": "cdb-dev"} -2022-06-27T04:19:38Z INFO controllers.PDB Issuing REST call {"callAPI": "oracle-database-operator-system/pdb1-clone", "URL": "http://cdb-dev-ords:8888/ords/_/db-api/latest/database/pdbs/pdbnewclone/status", "Action": "POST"} -2022-06-27T04:19:38Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1-clone", "Name": "cdb-dev", "CR Name": "cdb-dev"} -2022-06-27T04:19:39Z INFO controllers.PDB Successfully modified PDB state {"modifyPDB": "oracle-database-operator-system/pdb1-clone", "PDB Name": "pdbnewclone"} -2022-06-27T04:19:39Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1-clone", "Name": "cdb-dev", "CR Name": "cdb-dev"} -2022-06-27T04:19:39Z DEBUG events Normal {"object": {"kind":"PDB","namespace":"oracle-database-operator-system","name":"pdb1-clone","uid":"309dd711-198b-45b6-a34b-da5069af70fb","apiVersion":"database.oracle.com/v1alpha1","resourceVersion":"101462484"}, "reason": "Modified", "message": "PDB 'pdbnewclone' modified successfully"} -2022-06-27T04:19:39Z INFO controllers.PDB Issuing REST call {"callAPI": "oracle-database-operator-system/pdb1-clone", "URL": "http://cdb-dev-ords:8888/ords/_/db-api/latest/database/pdbs/pdbnewclone/status", "Action": "GET"} -2022-06-27T04:19:39Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1-clone", "Name": "cdb-dev", "CR Name": "cdb-dev"} -2022-06-27T04:19:39Z INFO controllers.PDB Successfully obtained PDB state {"getPDBState": "oracle-database-operator-system/pdb1-clone", "PDB Name": "pdbnewclone", "State": "MOUNTED"} -2022-06-27T04:19:39Z INFO controllers.PDB Reconcile completed {"onpremdboperator": "oracle-database-operator-system/pdb1-clone"} - - - -SQL> show pdbs - - CON_ID CON_NAME OPEN MODE RESTRICTED ----------- ------------------------------ ---------- ---------- - 2 PDB$SEED READ WRITE NO - 3 PDBNEW READ WRITE NO - 4 PDBNEWCLONE MOUNTED - - --- Check the .yaml file to be used to delete a PDB: - -% cat delete_pdb.yaml -# -# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. -# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. -# -apiVersion: database.oracle.com/v1alpha1 -kind: PDB -metadata: - name: pdb1-clone - namespace: oracle-database-operator-system - labels: - cdb: cdb-dev -spec: - cdbResName: "cdb-dev" - pdbName: "pdbnewclone" - action: "Delete" - dropAction: "INCLUDING" - - - --- Apply the .yaml file: - -% kubectl apply -f delete_pdb.yaml -pdb.database.oracle.com/pdb1-clone configured - - --- Monitor the Oracle DB Operator Pod logs: - -% kubectl logs -f pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd -n oracle-database-operator-system -. -. -2022-06-27T04:21:37Z DEBUG controller-runtime.webhook.webhooks received request {"webhook": "/validate-database-oracle-com-v1alpha1-pdb", "UID": "64148dda-d0df-4e03-88e3-98b1ce7b7aaf", "kind": "database.oracle.com/v1alpha1, Kind=PDB", "resource": {"group":"database.oracle.com","version":"v1alpha1","resource":"pdbs"}} -2022-06-27T04:21:37Z INFO pdb-webhook ValidateUpdate-Validating PDB spec for : pdb1-clone -2022-06-27T04:21:37Z INFO pdb-webhook validateCommon {"name": "pdb1-clone"} -2022-06-27T04:21:37Z INFO pdb-webhook Valdiating PDB Resource Action : DELETE -2022-06-27T04:21:37Z DEBUG controller-runtime.webhook.webhooks wrote response {"webhook": "/validate-database-oracle-com-v1alpha1-pdb", "code": 200, "reason": "", "UID": "64148dda-d0df-4e03-88e3-98b1ce7b7aaf", "allowed": true} -2022-06-27T04:21:37Z INFO controllers.PDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/pdb1-clone"} -2022-06-27T04:21:37Z INFO controllers.PDB Validating PDB phase for: pdb1-clone {"validatePhase": "oracle-database-operator-system/pdb1-clone", "Action": "DELETE"} -2022-06-27T04:21:37Z INFO controllers.PDB Validation complete {"validatePhase": "oracle-database-operator-system/pdb1-clone"} -2022-06-27T04:21:37Z INFO controllers.PDB PDB: {"onpremdboperator": "oracle-database-operator-system/pdb1-clone", "Name": "pdb1-clone", "Phase": "Deleting", "Status": "false"} -2022-06-27T04:21:37Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1-clone", "Name": "cdb-dev", "CR Name": "cdb-dev"} -2022-06-27T04:21:37Z INFO controllers.PDB Issuing REST call {"callAPI": "oracle-database-operator-system/pdb1-clone", "URL": "http://cdb-dev-ords:8888/ords/_/db-api/latest/database/pdbs/pdbnewclone/", "Action": "DELETE"} -2022-06-27T04:21:37Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1-clone", "Name": "cdb-dev", "CR Name": "cdb-dev"} -2022-06-27T04:21:39Z INFO controllers.PDB Successfully dropped PDB {"deletePDBInstance": "oracle-database-operator-system/pdb1-clone", "PDB Name": "pdbnewclone"} -2022-06-27T04:21:39Z INFO controllers.PDB Removing finalizer {"deletePDB": "oracle-database-operator-system/pdb1-clone"} -2022-06-27T04:21:39Z DEBUG controller-runtime.webhook.webhooks received request {"webhook": "/validate-database-oracle-com-v1alpha1-pdb", "UID": "15bd320b-3f9f-46a7-8493-c586310b7d84", "kind": "database.oracle.com/v1alpha1, Kind=PDB", "resource": {"group":"database.oracle.com","version":"v1alpha1","resource":"pdbs"}} -2022-06-27T04:21:39Z INFO pdb-webhook ValidateUpdate-Validating PDB spec for : pdb1-clone -2022-06-27T04:21:39Z INFO pdb-webhook validateCommon {"name": "pdb1-clone"} -2022-06-27T04:21:39Z INFO pdb-webhook Valdiating PDB Resource Action : DELETE -2022-06-27T04:21:39Z DEBUG controller-runtime.webhook.webhooks wrote response {"webhook": "/validate-database-oracle-com-v1alpha1-pdb", "code": 200, "reason": "", "UID": "15bd320b-3f9f-46a7-8493-c586310b7d84", "allowed": true} -2022-06-27T04:21:39Z INFO controllers.PDB Successfully deleted PDB resource {"deletePDB": "oracle-database-operator-system/pdb1-clone"} -2022-06-27T04:21:39Z INFO controllers.PDB Reconcile completed {"onpremdboperator": "oracle-database-operator-system/pdb1-clone"} -2022-06-27T04:21:39Z DEBUG events Normal {"object": {"kind":"PDB","namespace":"oracle-database-operator-system","name":"pdb1-clone","uid":"309dd711-198b-45b6-a34b-da5069af70fb","apiVersion":"database.oracle.com/v1alpha1","resourceVersion":"101463106"}, "reason": "Deleted", "message": "PDB 'pdbnewclone' dropped successfully"} - - - - - --- Check the PDB CRD resources: - -% kubectl get pdbs -A -NAMESPACE NAME CONNECT STRING CDB NAME PDB NAME PDB STATE PDB SIZE STATUS MESSAGE -oracle-database-operator-system pdb1 goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com:1521/pdbnew goldcdb pdbnew READ WRITE 1G Ready Success - - --- Verify from the CDB: - -SQL> show pdbs - - CON_ID CON_NAME OPEN MODE RESTRICTED ----------- ------------------------------ ---------- ---------- - 2 PDB$SEED READ WRITE NO - 3 PDBNEW READ WRITE NO diff --git a/docs/multitenant/provisioning/delete_pdb.md b/docs/multitenant/provisioning/delete_pdb.md deleted file mode 100644 index e36bb473..00000000 --- a/docs/multitenant/provisioning/delete_pdb.md +++ /dev/null @@ -1,37 +0,0 @@ -# Delete a PDB using Oracle DB Operator On-Prem Controller in a target CDB - -In this use case, a PDB is deleted using Oracle DB Operator On-Prem controller. - -To delete a PDB CRD Resource, a sample .yaml file is available here: [config/samples/onpremdb/pdb_delete.yaml](../../../config/samples/onpremdb/pdb_delete.yaml) - -**NOTE:** It is assumed that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-manage-pdb-life-cycle-using-oracle-db-operator-on-prem-database-controller) steps. - -This example uses `delete_pdb.yaml` to delete a PDB using Oracle DB Operator On-Prem Controller with: - -- Pluggable Database (PDB) Name as `pdbnewclone` -- Target CDB CRD Resource Name as `cdb-dev` -- Action to be taken on the PDB as `Delete` -- Option to specify if datafiles should be removed as `INCLUDING` - -**NOTE:** You need to *modify* the PDB status to MOUNTED, as described earlier, on the target CDB before you want to delete that PDB. - -**NOTE:** For the details of the parameters to be used in the .yaml file, please refer [config/crd/bases/database.oracle.com_pdbs.yaml](../../../config/crd/bases/database.oracle.com_pdbs.yaml) - -Use the file: [delete_pdb.yaml](./delete_pdb.yaml) for this use case as below: - -1. Deploy the .yaml file: -```sh -% kubectl apply -f delete_pdb.yaml -``` - -2. Monitor the Oracle DB Operator Pod `pod/oracle-database-operator-controller-manager-665874bd57-g2cgw` for the progress of the PDB deletion. - -NOTE: Check the DB Operator Pod name in your environment. - -```sh -% kubectl logs -f pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd -n oracle-database-operator-system -``` - -## Sample Output - -[Here](./delete_pdb.log) is the sample output for a PDB created using Oracle DB Operator On-Prem Controller using file [delete_pdb.yaml](./delete_pdb.yaml) diff --git a/docs/multitenant/provisioning/delete_pdb.yaml b/docs/multitenant/provisioning/delete_pdb.yaml deleted file mode 100644 index d16084bb..00000000 --- a/docs/multitenant/provisioning/delete_pdb.yaml +++ /dev/null @@ -1,16 +0,0 @@ -# -# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. -# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. -# -apiVersion: database.oracle.com/v1alpha1 -kind: PDB -metadata: - name: pdb1-clone - namespace: oracle-database-operator-system - labels: - cdb: cdb-dev -spec: - cdbResName: "cdb-dev" - pdbName: "pdbnewclone" - action: "Delete" - dropAction: "INCLUDING" diff --git a/docs/multitenant/provisioning/example_setup_using_oci_oke_cluster.md b/docs/multitenant/provisioning/example_setup_using_oci_oke_cluster.md index f3ed2489..d56efacb 100644 --- a/docs/multitenant/provisioning/example_setup_using_oci_oke_cluster.md +++ b/docs/multitenant/provisioning/example_setup_using_oci_oke_cluster.md @@ -18,7 +18,7 @@ Below are the main steps that will be involved in this setup: Check the [Oracle Documentation](https://docs.oracle.com/en-us/iaas/Content/ContEng/Concepts/contengnetworkconfigexample.htm#example-privatek8sapi-privateworkers-publiclb) for the OKE rules settings. -Create OKE cluster with CUSTOM option to use same VCN where ExaCS is provisioned. +Create OKE cluster with CUSTOM option to use same VCN where ExaCS is provisioned. **NOTE:** Make sure you choose same VCN exaphxvcn where ExaCS is provisioned. @@ -35,3 +35,4 @@ NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL- ``` Once this setup is ready, you can proceed with the installation of [Oracle Database Operator for Kubernetes](https://github.com/oracle/oracle-database-operator/blob/main/README.md) to use the Oracle On-prem controller to manage PDBs in this CDB. + diff --git a/docs/multitenant/provisioning/known_issues.md b/docs/multitenant/provisioning/known_issues.md deleted file mode 100644 index 3ab19897..00000000 --- a/docs/multitenant/provisioning/known_issues.md +++ /dev/null @@ -1,49 +0,0 @@ -# Known Issues - -Please refer to the below list of known issues related to Oracle DB Operator On-Prem Controller: - -1. **ORA-20002: ERROR: The user ORDS_PUBLIC_USER already exists in the logs of CDB CRD Pods.** - -This error is expected when you are deploying `2` replicas of CDB CRD during the deployment of the CDB CRD. Below is the snippet of a possible error: -``` -2022-06-22T20:06:32.616Z INFO Installing Oracle REST Data Services version 21.4.2.r0621806 in CDB$ROOT -2022-06-22T20:06:32.663Z INFO ... Log file written to /home/oracle/ords_cdb_install_core_CDB_ROOT_2022-06-22_200632_00662.log -2022-06-22T20:06:33.835Z INFO CDB restart file created in /home/oracle/ords_restart_2022-06-22_200633_00835.properties -2022-06-22T20:06:33.837Z SEVERE Error executing script: ords_prereq_env.sql Error: ORA-20002: ERROR: The user ORDS_PUBLIC_USER already exists. You must first uninstall ORDS using ords_uninstall.sql prior to running the install scripts. -ORA-06512: at line 8 -ORA-06512: at line 8 - - Refer to log file /home/oracle/ords_cdb_install_core_CDB_ROOT_2022-06-22_200632_00662.log for details - -java.io.IOException: Error executing script: ords_prereq_env.sql Error: ORA-20002: ERROR: The user ORDS_PUBLIC_USER already exists. You must first uninstall ORDS using ords_uninstall.sql prior to running the install scripts. -ORA-06512: at line 8 -ORA-06512: at line 8 -``` -This error is seen in the logs of one of the two CDB CRD pods. The other Pod `does not` show this error and the ORDS installation is done successfully. - -To avoid this error, you need to initially deploy the CDB CRD with a single replica and later add another replica as per need. - -2. **PDB create failure with error "Failed: Unauthorized"** - -It was observed that PDB creation fails with the below error when special characters like "_" or "#" were used in the password for user SQL_ADMIN: -``` -2022-06-22T20:10:09Z INFO controllers.PDB ORDS Error - HTTP Status Code :401 {"callAPI": "oracle-database-operator-system/pdb1", "Err": "\n{\n \"code\": \"Unauthorized\",\n \"message\": \"Unauthorized\",\n \"type\": \"tag:oracle.com,2020:error/Unauthorized\",\n \"instance\": \"tag:oracle.com,2020:ecid/OoqA0Zw3oBWdabzP8wUMcQ\"\n}"} -2022-06-22T20:10:09Z INFO controllers.PDB Reconcile completed {"onpremdboperator": "oracle-database-operator-system/pdb1"} -2022-06-22T20:10:09Z DEBUG events Warning {"object": {"kind":"PDB","namespace":"oracle-database-operator-system","name":"pdb1","uid":"19fc98b1-ca7f-4e63-a6c7-fdeb14b8c275","apiVersion":"database.oracle.com/v1alpha1","resourceVersion":"99558229"}, "reason": "ORDSError", "message": "Failed: Unauthorized"} -``` - -In testing, we have used the password `welcome1` for the user SQL_ADMIN. - -To avoid this error, please avoid password `welcome1` for SQL_ADMIN user. - - -3. **After cloning a PDB from another PDB, PDB SIZE field is show as empty even if the .yaml file used during the PDB cloning specifies the PDB size:** - -```sh -% kubectl get pdbs -A -NAMESPACE NAME CONNECT STRING CDB NAME PDB NAME PDB STATE PDB SIZE STATUS MESSAGE -oracle-database-operator-system pdb1 goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com:1521/pdbnew goldcdb pdbnew READ WRITE 1G Ready Success -oracle-database-operator-system pdb1-clone goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com:1521/pdbnewclone goldcdb pdbnewclone READ WRITE Ready Success -``` - -In the above example the PDB `pdbnewclone` is cloned from PDB `pdbnew` and is showing the size column as EMPTY. This will be fixed in future version. diff --git a/docs/multitenant/provisioning/modify_pdb.log b/docs/multitenant/provisioning/modify_pdb.log deleted file mode 100644 index 151393a7..00000000 --- a/docs/multitenant/provisioning/modify_pdb.log +++ /dev/null @@ -1,181 +0,0 @@ ------ Closing a PDB ------ - --- Check the existing PDB CRD resources - -jyotiprakashverma@jyotiprakashverma-mac onprem_test % kubectl get pdbs -A -NAMESPACE NAME CONNECT STRING CDB NAME PDB NAME PDB STATE PDB SIZE STATUS MESSAGE -oracle-database-operator-system pdb1 goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com:1521/pdbnew goldcdb pdbnew READ WRITE 1G Ready Success -oracle-database-operator-system pdb1-clone goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com:1521/pdbnewclone goldcdb pdbnewclone READ WRITE Ready Success - - - - --- Check the status of the PDBs in the CDB: - -SQL> show pdbs - - CON_ID CON_NAME OPEN MODE RESTRICTED ----------- ------------------------------ ---------- ---------- - 2 PDB$SEED READ WRITE NO - 3 PDBNEW READ WRITE NO - 5 PDBNEWCLONE READ WRITE NO - - - --- Check the file to modify the PDB state to CLOSE: - -% cat modify_pdb_close.yaml -# -# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. -# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. -# -apiVersion: database.oracle.com/v1alpha1 -kind: PDB -metadata: - name: pdb1 - namespace: oracle-database-operator-system - labels: - cdb: cdb-dev -spec: - cdbResName: "cdb-dev" - cdbName: "goldcdb" - pdbName: "pdbnew" - pdbState: "CLOSE" - modifyOption: "IMMEDIATE" - action: "Modify" - - --- Apply the file: - -% kubectl apply -f modify_pdb_close.yaml -pdb.database.oracle.com/pdb1 configured - - --- Monitor the logs from the Oracle DB Operator Pod: - -% kubectl logs -f pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd -n oracle-database-operator-system -. -. -2022-06-27T03:44:36Z DEBUG controller-runtime.webhook.webhooks received request {"webhook": "/mutate-database-oracle-com-v1alpha1-pdb", "UID": "08f95926-6bf1-4c70-b319-1b17015ce22a", "kind": "database.oracle.com/v1alpha1, Kind=PDB", "resource": {"group":"database.oracle.com","version":"v1alpha1","resource":"pdbs"}} -2022-06-27T03:44:36Z INFO pdb-webhook Setting default values in PDB spec for : pdb1 -2022-06-27T03:44:36Z DEBUG controller-runtime.webhook.webhooks wrote response {"webhook": "/mutate-database-oracle-com-v1alpha1-pdb", "code": 200, "reason": "", "UID": "08f95926-6bf1-4c70-b319-1b17015ce22a", "allowed": true} -2022-06-27T03:44:36Z DEBUG controller-runtime.webhook.webhooks received request {"webhook": "/validate-database-oracle-com-v1alpha1-pdb", "UID": "64ebebe2-87f2-4237-8928-532365f3cca9", "kind": "database.oracle.com/v1alpha1, Kind=PDB", "resource": {"group":"database.oracle.com","version":"v1alpha1","resource":"pdbs"}} -2022-06-27T03:44:36Z INFO pdb-webhook ValidateUpdate-Validating PDB spec for : pdb1 -2022-06-27T03:44:36Z INFO pdb-webhook validateCommon {"name": "pdb1"} -2022-06-27T03:44:36Z INFO pdb-webhook Valdiating PDB Resource Action : MODIFY -2022-06-27T03:44:36Z DEBUG controller-runtime.webhook.webhooks wrote response {"webhook": "/validate-database-oracle-com-v1alpha1-pdb", "code": 200, "reason": "", "UID": "64ebebe2-87f2-4237-8928-532365f3cca9", "allowed": true} -2022-06-27T03:44:36Z INFO controllers.PDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/pdb1"} -2022-06-27T03:44:36Z INFO controllers.PDB Validating PDB phase for: pdb1 {"validatePhase": "oracle-database-operator-system/pdb1", "Action": "MODIFY"} -2022-06-27T03:44:36Z INFO controllers.PDB Validation complete {"validatePhase": "oracle-database-operator-system/pdb1"} -2022-06-27T03:44:36Z INFO controllers.PDB PDB: {"onpremdboperator": "oracle-database-operator-system/pdb1", "Name": "pdb1", "Phase": "Modifying", "Status": "false"} -2022-06-27T03:44:36Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} -2022-06-27T03:44:36Z INFO controllers.PDB Issuing REST call {"callAPI": "oracle-database-operator-system/pdb1", "URL": "http://cdb-dev-ords:8888/ords/_/db-api/latest/database/pdbs/pdbnew/status", "Action": "GET"} -2022-06-27T03:44:36Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} -2022-06-27T03:44:36Z INFO controllers.PDB Successfully obtained PDB state {"getPDBState": "oracle-database-operator-system/pdb1", "PDB Name": "pdbnew", "State": "READ WRITE"} -2022-06-27T03:44:36Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} -2022-06-27T03:44:36Z INFO controllers.PDB Issuing REST call {"callAPI": "oracle-database-operator-system/pdb1", "URL": "http://cdb-dev-ords:8888/ords/_/db-api/latest/database/pdbs/pdbnew/status", "Action": "POST"} -2022-06-27T03:44:36Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} -2022-06-27T03:44:38Z INFO controllers.PDB Successfully modified PDB state {"modifyPDB": "oracle-database-operator-system/pdb1", "PDB Name": "pdbnew"} -2022-06-27T03:44:38Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} -2022-06-27T03:44:38Z DEBUG events Normal {"object": {"kind":"PDB","namespace":"oracle-database-operator-system","name":"pdb1","uid":"81f2e686-6e1b-4e2c-8a2f-e20c2f99d6b9","apiVersion":"database.oracle.com/v1alpha1","resourceVersion":"101451707"}, "reason": "Modified", "message": "PDB 'pdbnew' modified successfully"} -2022-06-27T03:44:38Z INFO controllers.PDB Issuing REST call {"callAPI": "oracle-database-operator-system/pdb1", "URL": "http://cdb-dev-ords:8888/ords/_/db-api/latest/database/pdbs/pdbnew/status", "Action": "GET"} -2022-06-27T03:44:38Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} -2022-06-27T03:44:38Z INFO controllers.PDB Successfully obtained PDB state {"getPDBState": "oracle-database-operator-system/pdb1", "PDB Name": "pdbnew", "State": "MOUNTED"} -2022-06-27T03:44:38Z INFO controllers.PDB Reconcile completed {"onpremdboperator": "oracle-database-operator-system/pdb1"} - - - - --- Check the status of PDB CRD resources - -jyotiprakashverma@jyotiprakashverma-mac onprem_test % kubectl get pdbs -A -NAMESPACE NAME CONNECT STRING CDB NAME PDB NAME PDB STATE PDB SIZE STATUS MESSAGE -oracle-database-operator-system pdb1 goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com:1521/pdbnew goldcdb pdbnew MOUNTED 1G Ready Success -oracle-database-operator-system pdb1-clone goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com:1521/pdbnewclone goldcdb pdbnewclone READ WRITE Ready Success - - - --- Confirm the status of the PDB in the CDB: - -SQL> show pdbs - - CON_ID CON_NAME OPEN MODE RESTRICTED ----------- ------------------------------ ---------- ---------- - 2 PDB$SEED READ WRITE NO - 3 PDBNEW MOUNTED - 4 PDBNEWCLONE READ WRITE NO - - - - - - - - ------ Opening a PDB ------ - --- Check the .yaml file to open the PDB: - -% cat modify_pdb_open.yaml -# -# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. -# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. -# -apiVersion: database.oracle.com/v1alpha1 -kind: PDB -metadata: - name: pdb1 - namespace: oracle-database-operator-system - labels: - cdb: cdb-dev -spec: - cdbResName: "cdb-dev" - cdbName: "goldcdb" - pdbName: "pdbnew" - pdbState: "OPEN" - modifyOption: "READ WRITE" - action: "Modify" - - - --- Apply the file: - -% kubectl apply -f modify_pdb_open.yaml -pdb.database.oracle.com/pdb1 configured - - --- Monitor the logs from the Oracle DB Operator Pod: - -% kubectl logs -f pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd -n oracle-database-operator-system -. -. -2022-06-27T03:48:38Z INFO controllers.PDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/pdb1"} -2022-06-27T03:48:38Z INFO controllers.PDB Validating PDB phase for: pdb1 {"validatePhase": "oracle-database-operator-system/pdb1", "Action": "MODIFY"} -2022-06-27T03:48:38Z INFO controllers.PDB Validation complete {"validatePhase": "oracle-database-operator-system/pdb1"} -2022-06-27T03:48:38Z INFO controllers.PDB PDB: {"onpremdboperator": "oracle-database-operator-system/pdb1", "Name": "pdb1", "Phase": "Modifying", "Status": "false"} -2022-06-27T03:48:38Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} -2022-06-27T03:48:38Z INFO controllers.PDB Issuing REST call {"callAPI": "oracle-database-operator-system/pdb1", "URL": "http://cdb-dev-ords:8888/ords/_/db-api/latest/database/pdbs/pdbnew/status", "Action": "GET"} -2022-06-27T03:48:38Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} -2022-06-27T03:48:38Z INFO controllers.PDB Successfully obtained PDB state {"getPDBState": "oracle-database-operator-system/pdb1", "PDB Name": "pdbnew", "State": "MOUNTED"} -2022-06-27T03:48:38Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} -2022-06-27T03:48:38Z INFO controllers.PDB Issuing REST call {"callAPI": "oracle-database-operator-system/pdb1", "URL": "http://cdb-dev-ords:8888/ords/_/db-api/latest/database/pdbs/pdbnew/status", "Action": "POST"} -2022-06-27T03:48:38Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} -2022-06-27T03:48:41Z INFO controllers.PDB Successfully modified PDB state {"modifyPDB": "oracle-database-operator-system/pdb1", "PDB Name": "pdbnew"} -2022-06-27T03:48:41Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} -2022-06-27T03:48:41Z DEBUG events Normal {"object": {"kind":"PDB","namespace":"oracle-database-operator-system","name":"pdb1","uid":"81f2e686-6e1b-4e2c-8a2f-e20c2f99d6b9","apiVersion":"database.oracle.com/v1alpha1","resourceVersion":"101452939"}, "reason": "Modified", "message": "PDB 'pdbnew' modified successfully"} -2022-06-27T03:48:41Z INFO controllers.PDB Issuing REST call {"callAPI": "oracle-database-operator-system/pdb1", "URL": "http://cdb-dev-ords:8888/ords/_/db-api/latest/database/pdbs/pdbnew/status", "Action": "GET"} -2022-06-27T03:48:41Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} -2022-06-27T03:48:41Z INFO controllers.PDB Successfully obtained PDB state {"getPDBState": "oracle-database-operator-system/pdb1", "PDB Name": "pdbnew", "State": "READ WRITE"} -2022-06-27T03:48:41Z INFO controllers.PDB Reconcile completed {"onpremdboperator": "oracle-database-operator-system/pdb1"} - - - --- Verify the status of the PDB: - -SQL> show pdbs - - CON_ID CON_NAME OPEN MODE RESTRICTED ----------- ------------------------------ ---------- ---------- - 2 PDB$SEED READ WRITE NO - 3 PDBNEW READ WRITE NO - 4 PDBNEWCLONE READ WRITE NO diff --git a/docs/multitenant/provisioning/modify_pdb.md b/docs/multitenant/provisioning/modify_pdb.md deleted file mode 100644 index a0432efb..00000000 --- a/docs/multitenant/provisioning/modify_pdb.md +++ /dev/null @@ -1,69 +0,0 @@ -# Modify a PDB using Oracle DB Operator On-Prem Controller in a target CDB - -In this use case, the state of an existing PDB is modified using Oracle DB Operator On-Prem controller. - -To modify a PDB CRD Resource, a sample .yaml file is available here: [config/samples/onpremdb/pdb_modify.yaml](../../../config/samples/onpremdb/pdb_modify.yaml) - -**NOTE:** It is assumed that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-manage-pdb-life-cycle-using-oracle-db-operator-on-prem-database-controller) steps. - -Subcase 1: This example uses `modify_pdb_close.yaml` to close a PDB using Oracle DB Operator On-Prem Controller with: - -- PDB CRD resource Name as `pdb1` -- Pluggable Database (PDB) Name as `pdbnew` -- Target CDB CRD Resource Name as `cdb-dev` -- Target CDB name as `goldcdb` -- Action to be taken on the PDB as `MODIFY` -- Target state of the PDB as `CLOSE` -- Option to close the state (i.e. modify) as `IMMEDIATE` - - -**NOTE:** For the details of the parameters to be used in the .yaml file, please refer [config/crd/bases/database.oracle.com_pdbs.yaml](../../../config/crd/bases/database.oracle.com_pdbs.yaml) - -Use the file: [modify_pdb_close.yaml](./modify_pdb_close.yaml) for this use case as below: - -1. Deploy the .yaml file: -```sh -% kubectl apply -f modify_pdb_close.yaml -pdb.database.oracle.com/pdb1 configured -``` - -2. Monitor the Oracle DB Operator Pod for the progress of the PDB creation. - -NOTE: Check the DB Operator Pod name in your environment. - -``` -[root@test-server oracle-database-operator]# kubectl logs -f pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd -n oracle-database-operator-system -``` - -Subcase 2: This example uses `modify_pdb_open.yaml` to open a PDB using Oracle DB Operator On-Prem Controller with: - -- PDB CRD resource Name as `pdb1` -- Pluggable Database (PDB) Name as `pdbnew` -- Target CDB CRD Resource Name as `cdb-dev` -- Target CDB name as `goldcdb` -- Action to be taken on the PDB as `MODIFY` -- Target state of the PDB as `OPEN` -- Option to close the state (i.e. modify) as `READ WRITE` - - -**NOTE:** For the details of the parameters to be used in the .yaml file, please refer [config/crd/bases/database.oracle.com_pdbs.yaml](../../../config/crd/bases/database.oracle.com_pdbs.yaml) - -Use the file: [modify_pdb_open.yaml](./modify_pdb_open.yaml) for this use case as below: - -1. Deploy the .yaml file: -```sh -% kubectl apply -f modify_pdb_open.yaml -pdb.database.oracle.com/pdb1 configured -``` - -2. Monitor the Oracle DB Operator Pod for the progress of the PDB creation. - -NOTE: Check the DB Operator Pod name in your environment. - -``` -[root@test-server oracle-database-operator]# kubectl logs -f pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd -n oracle-database-operator-system -``` - -## Sample Output - -[Here](./modify_pdb.log) is the sample output for a PDB created using Oracle DB Operator On-Prem Controller using file [modify_pdb_close.yaml](./modify_pdb_close.yaml) and [modify_pdb_open.yaml](./modify_pdb_open.yaml) diff --git a/docs/multitenant/provisioning/modify_pdb_close.yaml b/docs/multitenant/provisioning/modify_pdb_close.yaml deleted file mode 100644 index 8897b482..00000000 --- a/docs/multitenant/provisioning/modify_pdb_close.yaml +++ /dev/null @@ -1,18 +0,0 @@ -# -# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. -# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. -# -apiVersion: database.oracle.com/v1alpha1 -kind: PDB -metadata: - name: pdb1 - namespace: oracle-database-operator-system - labels: - cdb: cdb-dev -spec: - cdbResName: "cdb-dev" - cdbName: "goldcdb" - pdbName: "pdbnew" - pdbState: "CLOSE" - modifyOption: "IMMEDIATE" - action: "Modify" diff --git a/docs/multitenant/provisioning/modify_pdb_open.yaml b/docs/multitenant/provisioning/modify_pdb_open.yaml deleted file mode 100644 index 0ec640d4..00000000 --- a/docs/multitenant/provisioning/modify_pdb_open.yaml +++ /dev/null @@ -1,18 +0,0 @@ -# -# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. -# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. -# -apiVersion: database.oracle.com/v1alpha1 -kind: PDB -metadata: - name: pdb1 - namespace: oracle-database-operator-system - labels: - cdb: cdb-dev -spec: - cdbResName: "cdb-dev" - cdbName: "goldcdb" - pdbName: "pdbnew" - pdbState: "OPEN" - modifyOption: "READ WRITE" - action: "Modify" diff --git a/docs/multitenant/provisioning/multinamespace/cdb_create.yaml b/docs/multitenant/provisioning/multinamespace/cdb_create.yaml new file mode 100644 index 00000000..d3b5e04f --- /dev/null +++ b/docs/multitenant/provisioning/multinamespace/cdb_create.yaml @@ -0,0 +1,44 @@ +apiVersion: database.oracle.com/v1alpha1 +kind: CDB +metadata: + name: cdb-dev + namespace: cdbnamespace +spec: + cdbName: "DB12" + ordsImage: ".............your registry............./ords-dboper:latest" + ordsImagePullPolicy: "Always" + dbTnsurl : "...Container tns alias....." + replicas: 1 + sysAdminPwd: + secret: + secretName: "cdb1-secret" + key: "sysadmin_pwd" + ordsPwd: + secret: + secretName: "cdb1-secret" + key: "ords_pwd" + cdbAdminUser: + secret: + secretName: "cdb1-secret" + key: "cdbadmin_user" + cdbAdminPwd: + secret: + secretName: "cdb1-secret" + key: "cdbadmin_pwd" + webServerUser: + secret: + secretName: "cdb1-secret" + key: "webserver_user" + webServerPwd: + secret: + secretName: "cdb1-secret" + key: "webserver_pwd" + cdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + cdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + diff --git a/docs/multitenant/provisioning/multinamespace/pdb_clone.yaml b/docs/multitenant/provisioning/multinamespace/pdb_clone.yaml new file mode 100644 index 00000000..b88fb71b --- /dev/null +++ b/docs/multitenant/provisioning/multinamespace/pdb_clone.yaml @@ -0,0 +1,50 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb2 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + cdbName: "DB12" + pdbName: "pdb2_clone" + srcPdbName: "pdbdev" + fileNameConversions: "NONE" + totalSize: "UNLIMITED" + tempSize: "UNLIMITED" + assertivePdbDeletion: true + adminName: + secret: + secretName: "pdb1-secret" + key: "sysadmin_user" + adminPwd: + secret: + secretName: "pdb1-secret" + key: "sysadmin_pwd" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "pdb1-secret" + key: "webserver_user" + webServerPwd: + secret: + secretName: "pdb1-secret" + key: "webserver_pwd" + action: "Clone" diff --git a/docs/multitenant/provisioning/multinamespace/pdb_close.yaml b/docs/multitenant/provisioning/multinamespace/pdb_close.yaml new file mode 100644 index 00000000..a823f5d9 --- /dev/null +++ b/docs/multitenant/provisioning/multinamespace/pdb_close.yaml @@ -0,0 +1,44 @@ +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + cdbName: "DB12" + pdbName: "pdbdev" + adminName: + secret: + secretName: "pdb1-secret" + key: "sysadmin_user" + adminPwd: + secret: + secretName: "pdb1-secret" + key: "sysadmin_pwd" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "pdb1-secret" + key: "webserver_user" + webServerPwd: + secret: + secretName: "pdb1-secret" + key: "webserver_pwd" + pdbState: "CLOSE" + modifyOption: "IMMEDIATE" + action: "Modify" + diff --git a/docs/multitenant/provisioning/multinamespace/pdb_create.yaml b/docs/multitenant/provisioning/multinamespace/pdb_create.yaml new file mode 100644 index 00000000..200f3712 --- /dev/null +++ b/docs/multitenant/provisioning/multinamespace/pdb_create.yaml @@ -0,0 +1,46 @@ +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + cdbName: "DB12" + pdbName: "pdbdev" + adminName: + secret: + secretName: "pdb1-secret" + key: "sysadmin_user" + adminPwd: + secret: + secretName: "pdb1-secret" + key: "sysadmin_pwd" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "pdb1-secret" + key: "webserver_user" + webServerPwd: + secret: + secretName: "pdb1-secret" + key: "webserver_pwd" + fileNameConversions: "NONE" + tdeImport: false + totalSize: "1G" + tempSize: "100M" + action: "Create" + diff --git a/docs/multitenant/provisioning/multinamespace/pdb_delete.yaml b/docs/multitenant/provisioning/multinamespace/pdb_delete.yaml new file mode 100644 index 00000000..282885b0 --- /dev/null +++ b/docs/multitenant/provisioning/multinamespace/pdb_delete.yaml @@ -0,0 +1,34 @@ +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + pdbName: "pdbdev" + action: "Delete" + dropAction: "INCLUDING" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "pdb1-secret" + key: "webserver_user" + webServerPwd: + secret: + secretName: "pdb1-secret" + key: "webserver_pwd" + diff --git a/docs/multitenant/provisioning/multinamespace/pdb_open.yaml b/docs/multitenant/provisioning/multinamespace/pdb_open.yaml new file mode 100644 index 00000000..85fb2ce4 --- /dev/null +++ b/docs/multitenant/provisioning/multinamespace/pdb_open.yaml @@ -0,0 +1,43 @@ +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + cdbName: "DB12" + pdbName: "pdbdev" + adminName: + secret: + secretName: "pdb1-secret" + key: "sysadmin_user" + adminPwd: + secret: + secretName: "pdb1-secret" + key: "sysadmin_pwd" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "pdb1-secret" + key: "webserver_user" + webServerPwd: + secret: + secretName: "pdb1-secret" + key: "webserver_pwd" + action: "Modify" + pdbState: "OPEN" + modifyOption: "READ WRITE" diff --git a/docs/multitenant/provisioning/multinamespace/pdb_plug.yaml b/docs/multitenant/provisioning/multinamespace/pdb_plug.yaml new file mode 100644 index 00000000..d9135f13 --- /dev/null +++ b/docs/multitenant/provisioning/multinamespace/pdb_plug.yaml @@ -0,0 +1,46 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + cdbName: "DB12" + pdbName: "pdbdev" + xmlFileName: "/tmp/pdb.xml" + fileNameConversions: "NONE" + sourceFileNameConversions: "NONE" + copyAction: "MOVE" + totalSize: "1G" + tempSize: "100M" + action: "Plug" + assertivePdbDeletion: true + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "pdb1-secret" + key: "webserver_user" + webServerPwd: + secret: + secretName: "pdb1-secret" + key: "webserver_pwd" + + diff --git a/docs/multitenant/provisioning/multinamespace/pdb_unplug.yaml b/docs/multitenant/provisioning/multinamespace/pdb_unplug.yaml new file mode 100644 index 00000000..f3667dad --- /dev/null +++ b/docs/multitenant/provisioning/multinamespace/pdb_unplug.yaml @@ -0,0 +1,39 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + cdbName: "DB12" + pdbName: "pdbdev" + xmlFileName: "/tmp/pdb.xml" + action: "Unplug" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "pdb1-secret" + key: "webserver_user" + webServerPwd: + secret: + secretName: "pdb1-secret" + key: "webserver_pwd" + diff --git a/docs/multitenant/provisioning/ords_image.log b/docs/multitenant/provisioning/ords_image.log deleted file mode 100644 index 771a22ca..00000000 --- a/docs/multitenant/provisioning/ords_image.log +++ /dev/null @@ -1,503 +0,0 @@ -/usr/bin/docker build -t oracle/ords-dboper:22.2.1 . -Sending build context to Docker daemon 280.6kB -Step 1/10 : FROM container-registry.oracle.com/java/jdk:latest - ---> 44b86e8925c4 -Step 2/10 : ENV ORDS_HOME=/opt/oracle/ords RUN_FILE="runOrdsSSL.sh" - ---> Running in e22c5a03b869 -Removing intermediate container e22c5a03b869 - ---> 1421497abef8 -Step 3/10 : COPY $RUN_FILE $ORDS_HOME/ - ---> d96ac1477d2d -Step 4/10 : RUN yum -y install yum-utils bind-utils tree hostname openssl net-tools zip unzip tar wget vim-minimal which sudo expect procps && yum-config-manager --add-repo=http://yum.oracle.com/repo/OracleLinux/OL8/oracle/software/x86_64 && yum -y install java-11-openjdk-devel && yum -y install ords && yum -y install iproute && yum clean all - ---> Running in c08b8dac80a5 -Oracle Linux 8 BaseOS Latest (x86_64) 72 MB/s | 49 MB 00:00 -Oracle Linux 8 Application Stream (x86_64) 88 MB/s | 37 MB 00:00 -Last metadata expiration check: 0:00:07 ago on Mon 12 Sep 2022 03:23:32 PM UTC. -Package yum-utils-4.0.21-11.0.1.el8.noarch is already installed. -Package vim-minimal-2:8.0.1763-19.0.1.el8_6.2.x86_64 is already installed. -Package procps-ng-3.3.15-6.0.1.el8.x86_64 is already installed. -Dependencies resolved. -================================================================================ - Package Arch Version Repository Size -================================================================================ -Installing: - bind-utils x86_64 32:9.11.36-3.el8 ol8_appstream 452 k - expect x86_64 5.45.4-5.el8 ol8_baseos_latest 266 k - hostname x86_64 3.20-6.el8 ol8_baseos_latest 32 k - net-tools x86_64 2.0-0.52.20160912git.el8 ol8_baseos_latest 322 k - openssl x86_64 1:1.1.1k-7.el8_6 ol8_baseos_latest 709 k - sudo x86_64 1.8.29-8.el8 ol8_baseos_latest 925 k - tar x86_64 2:1.30-5.el8 ol8_baseos_latest 838 k - tree x86_64 1.7.0-15.el8 ol8_baseos_latest 59 k - unzip x86_64 6.0-46.0.1.el8 ol8_baseos_latest 196 k - wget x86_64 1.19.5-10.0.1.el8 ol8_appstream 734 k - which x86_64 2.21-17.el8 ol8_baseos_latest 49 k - zip x86_64 3.0-23.el8 ol8_baseos_latest 270 k -Upgrading: - openssl-libs x86_64 1:1.1.1k-7.el8_6 ol8_baseos_latest 1.5 M - vim-minimal x86_64 2:8.0.1763-19.0.1.el8_6.4 ol8_baseos_latest 575 k -Installing dependencies: - bind-libs x86_64 32:9.11.36-3.el8 ol8_appstream 175 k - bind-libs-lite x86_64 32:9.11.36-3.el8 ol8_appstream 1.2 M - bind-license noarch 32:9.11.36-3.el8 ol8_appstream 103 k - fstrm x86_64 0.6.1-2.el8 ol8_appstream 29 k - libmaxminddb x86_64 1.2.0-10.el8 ol8_appstream 33 k - libmetalink x86_64 0.1.3-7.el8 ol8_baseos_latest 32 k - protobuf-c x86_64 1.3.0-6.el8 ol8_appstream 37 k - python3-bind noarch 32:9.11.36-3.el8 ol8_appstream 150 k - python3-ply noarch 3.9-9.el8 ol8_baseos_latest 111 k - tcl x86_64 1:8.6.8-2.el8 ol8_baseos_latest 1.1 M - -Transaction Summary -================================================================================ -Install 22 Packages -Upgrade 2 Packages - -Total download size: 9.7 M -Downloading Packages: -(1/24): expect-5.45.4-5.el8.x86_64.rpm 158 kB/s | 266 kB 00:01 -(2/24): hostname-3.20-6.el8.x86_64.rpm 18 kB/s | 32 kB 00:01 -(3/24): libmetalink-0.1.3-7.el8.x86_64.rpm 18 kB/s | 32 kB 00:01 -(4/24): net-tools-2.0-0.52.20160912git.el8.x86_ 2.3 MB/s | 322 kB 00:00 -(5/24): openssl-1.1.1k-7.el8_6.x86_64.rpm 4.0 MB/s | 709 kB 00:00 -(6/24): python3-ply-3.9-9.el8.noarch.rpm 538 kB/s | 111 kB 00:00 -(7/24): sudo-1.8.29-8.el8.x86_64.rpm 5.0 MB/s | 925 kB 00:00 -(8/24): tar-1.30-5.el8.x86_64.rpm 4.2 MB/s | 838 kB 00:00 -(9/24): unzip-6.0-46.0.1.el8.x86_64.rpm 3.6 MB/s | 196 kB 00:00 -(10/24): tcl-8.6.8-2.el8.x86_64.rpm 4.1 MB/s | 1.1 MB 00:00 -(11/24): which-2.21-17.el8.x86_64.rpm 613 kB/s | 49 kB 00:00 -(12/24): tree-1.7.0-15.el8.x86_64.rpm 208 kB/s | 59 kB 00:00 -(13/24): bind-libs-9.11.36-3.el8.x86_64.rpm 1.3 MB/s | 175 kB 00:00 -(14/24): bind-license-9.11.36-3.el8.noarch.rpm 2.6 MB/s | 103 kB 00:00 -(15/24): bind-libs-lite-9.11.36-3.el8.x86_64.rp 6.8 MB/s | 1.2 MB 00:00 -(16/24): bind-utils-9.11.36-3.el8.x86_64.rpm 3.6 MB/s | 452 kB 00:00 -(17/24): zip-3.0-23.el8.x86_64.rpm 804 kB/s | 270 kB 00:00 -(18/24): libmaxminddb-1.2.0-10.el8.x86_64.rpm 529 kB/s | 33 kB 00:00 -(19/24): fstrm-0.6.1-2.el8.x86_64.rpm 161 kB/s | 29 kB 00:00 -(20/24): python3-bind-9.11.36-3.el8.noarch.rpm 2.0 MB/s | 150 kB 00:00 -(21/24): protobuf-c-1.3.0-6.el8.x86_64.rpm 351 kB/s | 37 kB 00:00 -(22/24): vim-minimal-8.0.1763-19.0.1.el8_6.4.x8 6.4 MB/s | 575 kB 00:00 -(23/24): wget-1.19.5-10.0.1.el8.x86_64.rpm 3.3 MB/s | 734 kB 00:00 -(24/24): openssl-libs-1.1.1k-7.el8_6.x86_64.rpm 6.8 MB/s | 1.5 MB 00:00 --------------------------------------------------------------------------------- -Total 3.3 MB/s | 9.7 MB 00:02 -Running transaction check -Transaction check succeeded. -Running transaction test -Transaction test succeeded. -Running transaction - Preparing : 1/1 - Upgrading : openssl-libs-1:1.1.1k-7.el8_6.x86_64 1/26 - Running scriptlet: openssl-libs-1:1.1.1k-7.el8_6.x86_64 1/26 - Installing : protobuf-c-1.3.0-6.el8.x86_64 2/26 - Installing : libmaxminddb-1.2.0-10.el8.x86_64 3/26 - Running scriptlet: libmaxminddb-1.2.0-10.el8.x86_64 3/26 - Installing : fstrm-0.6.1-2.el8.x86_64 4/26 - Installing : bind-license-32:9.11.36-3.el8.noarch 5/26 - Installing : bind-libs-lite-32:9.11.36-3.el8.x86_64 6/26 - Installing : bind-libs-32:9.11.36-3.el8.x86_64 7/26 - Upgrading : vim-minimal-2:8.0.1763-19.0.1.el8_6.4.x86_64 8/26 - Installing : unzip-6.0-46.0.1.el8.x86_64 9/26 - Installing : tcl-1:8.6.8-2.el8.x86_64 10/26 - Running scriptlet: tcl-1:8.6.8-2.el8.x86_64 10/26 - Installing : python3-ply-3.9-9.el8.noarch 11/26 - Installing : python3-bind-32:9.11.36-3.el8.noarch 12/26 - Installing : libmetalink-0.1.3-7.el8.x86_64 13/26 - Installing : wget-1.19.5-10.0.1.el8.x86_64 14/26 - Running scriptlet: wget-1.19.5-10.0.1.el8.x86_64 14/26 - Installing : bind-utils-32:9.11.36-3.el8.x86_64 15/26 - Installing : expect-5.45.4-5.el8.x86_64 16/26 - Installing : zip-3.0-23.el8.x86_64 17/26 - Installing : sudo-1.8.29-8.el8.x86_64 18/26 - Running scriptlet: sudo-1.8.29-8.el8.x86_64 18/26 - Installing : openssl-1:1.1.1k-7.el8_6.x86_64 19/26 - Installing : which-2.21-17.el8.x86_64 20/26 - Installing : tree-1.7.0-15.el8.x86_64 21/26 - Installing : tar-2:1.30-5.el8.x86_64 22/26 - Running scriptlet: tar-2:1.30-5.el8.x86_64 22/26 - Installing : net-tools-2.0-0.52.20160912git.el8.x86_64 23/26 - Running scriptlet: net-tools-2.0-0.52.20160912git.el8.x86_64 23/26 - Installing : hostname-3.20-6.el8.x86_64 24/26 - Running scriptlet: hostname-3.20-6.el8.x86_64 24/26 - Cleanup : vim-minimal-2:8.0.1763-19.0.1.el8_6.2.x86_64 25/26 - Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 26/26 - Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 26/26 - Verifying : expect-5.45.4-5.el8.x86_64 1/26 - Verifying : hostname-3.20-6.el8.x86_64 2/26 - Verifying : libmetalink-0.1.3-7.el8.x86_64 3/26 - Verifying : net-tools-2.0-0.52.20160912git.el8.x86_64 4/26 - Verifying : openssl-1:1.1.1k-7.el8_6.x86_64 5/26 - Verifying : python3-ply-3.9-9.el8.noarch 6/26 - Verifying : sudo-1.8.29-8.el8.x86_64 7/26 - Verifying : tar-2:1.30-5.el8.x86_64 8/26 - Verifying : tcl-1:8.6.8-2.el8.x86_64 9/26 - Verifying : tree-1.7.0-15.el8.x86_64 10/26 - Verifying : unzip-6.0-46.0.1.el8.x86_64 11/26 - Verifying : which-2.21-17.el8.x86_64 12/26 - Verifying : zip-3.0-23.el8.x86_64 13/26 - Verifying : bind-libs-32:9.11.36-3.el8.x86_64 14/26 - Verifying : bind-libs-lite-32:9.11.36-3.el8.x86_64 15/26 - Verifying : bind-license-32:9.11.36-3.el8.noarch 16/26 - Verifying : bind-utils-32:9.11.36-3.el8.x86_64 17/26 - Verifying : fstrm-0.6.1-2.el8.x86_64 18/26 - Verifying : libmaxminddb-1.2.0-10.el8.x86_64 19/26 - Verifying : protobuf-c-1.3.0-6.el8.x86_64 20/26 - Verifying : python3-bind-32:9.11.36-3.el8.noarch 21/26 - Verifying : wget-1.19.5-10.0.1.el8.x86_64 22/26 - Verifying : openssl-libs-1:1.1.1k-7.el8_6.x86_64 23/26 - Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 24/26 - Verifying : vim-minimal-2:8.0.1763-19.0.1.el8_6.4.x86_64 25/26 - Verifying : vim-minimal-2:8.0.1763-19.0.1.el8_6.2.x86_64 26/26 - -Upgraded: - openssl-libs-1:1.1.1k-7.el8_6.x86_64 - vim-minimal-2:8.0.1763-19.0.1.el8_6.4.x86_64 -Installed: - bind-libs-32:9.11.36-3.el8.x86_64 - bind-libs-lite-32:9.11.36-3.el8.x86_64 - bind-license-32:9.11.36-3.el8.noarch - bind-utils-32:9.11.36-3.el8.x86_64 - expect-5.45.4-5.el8.x86_64 - fstrm-0.6.1-2.el8.x86_64 - hostname-3.20-6.el8.x86_64 - libmaxminddb-1.2.0-10.el8.x86_64 - libmetalink-0.1.3-7.el8.x86_64 - net-tools-2.0-0.52.20160912git.el8.x86_64 - openssl-1:1.1.1k-7.el8_6.x86_64 - protobuf-c-1.3.0-6.el8.x86_64 - python3-bind-32:9.11.36-3.el8.noarch - python3-ply-3.9-9.el8.noarch - sudo-1.8.29-8.el8.x86_64 - tar-2:1.30-5.el8.x86_64 - tcl-1:8.6.8-2.el8.x86_64 - tree-1.7.0-15.el8.x86_64 - unzip-6.0-46.0.1.el8.x86_64 - wget-1.19.5-10.0.1.el8.x86_64 - which-2.21-17.el8.x86_64 - zip-3.0-23.el8.x86_64 - -Complete! -Adding repo from: http://yum.oracle.com/repo/OracleLinux/OL8/oracle/software/x86_64 -created by dnf config-manager from http://yum.o 221 kB/s | 45 kB 00:00 -Dependencies resolved. -============================================================================================= - Package Arch Version Repository Size -============================================================================================= -Installing: - java-11-openjdk-devel x86_64 1:11.0.16.1.1-1.el8_6 ol8_appstream 3.4 M -Installing dependencies: - alsa-lib x86_64 1.2.6.1-3.el8 ol8_appstream 491 k - avahi-libs x86_64 0.7-20.el8 ol8_baseos_latest 62 k - copy-jdk-configs noarch 4.0-2.el8 ol8_appstream 30 k - crypto-policies-scripts noarch 20211116-1.gitae470d6.el8 ol8_baseos_latest 83 k - cups-libs x86_64 1:2.2.6-45.el8_6.2 ol8_baseos_latest 434 k - giflib x86_64 5.1.4-3.el8 ol8_appstream 51 k - graphite2 x86_64 1.3.10-10.el8 ol8_appstream 122 k - harfbuzz x86_64 1.7.5-3.el8 ol8_appstream 295 k - java-11-openjdk x86_64 1:11.0.16.1.1-1.el8_6 ol8_appstream 272 k - java-11-openjdk-headless x86_64 1:11.0.16.1.1-1.el8_6 ol8_appstream 40 M - javapackages-filesystem noarch 5.3.0-1.module+el8+5136+7ff78f74 ol8_appstream 30 k - lcms2 x86_64 2.9-2.el8 ol8_appstream 164 k - libX11 x86_64 1.6.8-5.el8 ol8_appstream 611 k - libX11-common noarch 1.6.8-5.el8 ol8_appstream 158 k - libXau x86_64 1.0.9-3.el8 ol8_appstream 37 k - libXcomposite x86_64 0.4.4-14.el8 ol8_appstream 28 k - libXext x86_64 1.3.4-1.el8 ol8_appstream 45 k - libXi x86_64 1.7.10-1.el8 ol8_appstream 49 k - libXrender x86_64 0.9.10-7.el8 ol8_appstream 33 k - libXtst x86_64 1.2.3-7.el8 ol8_appstream 22 k - libfontenc x86_64 1.1.3-8.el8 ol8_appstream 37 k - libjpeg-turbo x86_64 1.5.3-12.el8 ol8_appstream 157 k - libpkgconf x86_64 1.4.2-1.el8 ol8_baseos_latest 35 k - libxcb x86_64 1.13.1-1.el8 ol8_appstream 231 k - lksctp-tools x86_64 1.0.18-3.el8 ol8_baseos_latest 100 k - lua x86_64 5.3.4-12.el8 ol8_appstream 192 k - nspr x86_64 4.32.0-1.el8_4 ol8_appstream 142 k - nss x86_64 3.67.0-7.el8_5 ol8_appstream 741 k - nss-softokn x86_64 3.67.0-7.el8_5 ol8_appstream 487 k - nss-softokn-freebl x86_64 3.67.0-7.el8_5 ol8_appstream 395 k - nss-sysinit x86_64 3.67.0-7.el8_5 ol8_appstream 73 k - nss-util x86_64 3.67.0-7.el8_5 ol8_appstream 137 k - pkgconf x86_64 1.4.2-1.el8 ol8_baseos_latest 38 k - pkgconf-m4 noarch 1.4.2-1.el8 ol8_baseos_latest 17 k - pkgconf-pkg-config x86_64 1.4.2-1.el8 ol8_baseos_latest 15 k - ttmkfdir x86_64 3.0.9-54.el8 ol8_appstream 62 k - tzdata-java noarch 2022c-1.el8 ol8_appstream 186 k - xorg-x11-font-utils x86_64 1:7.5-41.el8 ol8_appstream 104 k - xorg-x11-fonts-Type1 noarch 7.5-19.el8 ol8_appstream 522 k -Enabling module streams: - javapackages-runtime 201801 - -Transaction Summary -============================================================================================= -Install 40 Packages - -Total download size: 50 M -Installed size: 194 M -Downloading Packages: -(1/40): crypto-policies-scripts-20211116-1.gita 1.3 MB/s | 83 kB 00:00 -(2/40): avahi-libs-0.7-20.el8.x86_64.rpm 952 kB/s | 62 kB 00:00 -(3/40): libpkgconf-1.4.2-1.el8.x86_64.rpm 2.2 MB/s | 35 kB 00:00 -(4/40): cups-libs-2.2.6-45.el8_6.2.x86_64.rpm 4.9 MB/s | 434 kB 00:00 -(5/40): lksctp-tools-1.0.18-3.el8.x86_64.rpm 3.9 MB/s | 100 kB 00:00 -(6/40): pkgconf-1.4.2-1.el8.x86_64.rpm 2.3 MB/s | 38 kB 00:00 -(7/40): pkgconf-m4-1.4.2-1.el8.noarch.rpm 1.1 MB/s | 17 kB 00:00 -(8/40): pkgconf-pkg-config-1.4.2-1.el8.x86_64.r 1.1 MB/s | 15 kB 00:00 -(9/40): copy-jdk-configs-4.0-2.el8.noarch.rpm 1.8 MB/s | 30 kB 00:00 -(10/40): giflib-5.1.4-3.el8.x86_64.rpm 3.0 MB/s | 51 kB 00:00 -(11/40): alsa-lib-1.2.6.1-3.el8.x86_64.rpm 12 MB/s | 491 kB 00:00 -(12/40): graphite2-1.3.10-10.el8.x86_64.rpm 5.9 MB/s | 122 kB 00:00 -(13/40): harfbuzz-1.7.5-3.el8.x86_64.rpm 13 MB/s | 295 kB 00:00 -(14/40): java-11-openjdk-11.0.16.1.1-1.el8_6.x8 15 MB/s | 272 kB 00:00 -(15/40): javapackages-filesystem-5.3.0-1.module 2.1 MB/s | 30 kB 00:00 -(16/40): lcms2-2.9-2.el8.x86_64.rpm 9.5 MB/s | 164 kB 00:00 -(17/40): libX11-1.6.8-5.el8.x86_64.rpm 24 MB/s | 611 kB 00:00 -(18/40): java-11-openjdk-devel-11.0.16.1.1-1.el 40 MB/s | 3.4 MB 00:00 -(19/40): libX11-common-1.6.8-5.el8.noarch.rpm 8.6 MB/s | 158 kB 00:00 -(20/40): libXau-1.0.9-3.el8.x86_64.rpm 2.6 MB/s | 37 kB 00:00 -(21/40): libXcomposite-0.4.4-14.el8.x86_64.rpm 2.2 MB/s | 28 kB 00:00 -(22/40): libXext-1.3.4-1.el8.x86_64.rpm 2.7 MB/s | 45 kB 00:00 -(23/40): libXi-1.7.10-1.el8.x86_64.rpm 2.8 MB/s | 49 kB 00:00 -(24/40): libXrender-0.9.10-7.el8.x86_64.rpm 2.4 MB/s | 33 kB 00:00 -(25/40): libXtst-1.2.3-7.el8.x86_64.rpm 1.6 MB/s | 22 kB 00:00 -(26/40): libfontenc-1.1.3-8.el8.x86_64.rpm 2.7 MB/s | 37 kB 00:00 -(27/40): libjpeg-turbo-1.5.3-12.el8.x86_64.rpm 9.6 MB/s | 157 kB 00:00 -(28/40): libxcb-1.13.1-1.el8.x86_64.rpm 13 MB/s | 231 kB 00:00 -(29/40): lua-5.3.4-12.el8.x86_64.rpm 11 MB/s | 192 kB 00:00 -(30/40): nspr-4.32.0-1.el8_4.x86_64.rpm 9.2 MB/s | 142 kB 00:00 -(31/40): nss-3.67.0-7.el8_5.x86_64.rpm 31 MB/s | 741 kB 00:00 -(32/40): nss-softokn-3.67.0-7.el8_5.x86_64.rpm 24 MB/s | 487 kB 00:00 -(33/40): nss-softokn-freebl-3.67.0-7.el8_5.x86_ 18 MB/s | 395 kB 00:00 -(34/40): nss-sysinit-3.67.0-7.el8_5.x86_64.rpm 4.3 MB/s | 73 kB 00:00 -(35/40): nss-util-3.67.0-7.el8_5.x86_64.rpm 8.7 MB/s | 137 kB 00:00 -(36/40): ttmkfdir-3.0.9-54.el8.x86_64.rpm 4.0 MB/s | 62 kB 00:00 -(37/40): tzdata-java-2022c-1.el8.noarch.rpm 12 MB/s | 186 kB 00:00 -(38/40): xorg-x11-font-utils-7.5-41.el8.x86_64. 6.0 MB/s | 104 kB 00:00 -(39/40): xorg-x11-fonts-Type1-7.5-19.el8.noarch 23 MB/s | 522 kB 00:00 -(40/40): java-11-openjdk-headless-11.0.16.1.1-1 73 MB/s | 40 MB 00:00 --------------------------------------------------------------------------------- -Total 71 MB/s | 50 MB 00:00 -Running transaction check -Transaction check succeeded. -Running transaction test -Transaction test succeeded. -Running transaction - Running scriptlet: copy-jdk-configs-4.0-2.el8.noarch 1/1 - Running scriptlet: java-11-openjdk-headless-1:11.0.16.1.1-1.el8_6.x86_6 1/1 - Preparing : 1/1 - Installing : nspr-4.32.0-1.el8_4.x86_64 1/40 - Running scriptlet: nspr-4.32.0-1.el8_4.x86_64 1/40 - Installing : nss-util-3.67.0-7.el8_5.x86_64 2/40 - Installing : libjpeg-turbo-1.5.3-12.el8.x86_64 3/40 - Installing : nss-softokn-freebl-3.67.0-7.el8_5.x86_64 4/40 - Installing : nss-softokn-3.67.0-7.el8_5.x86_64 5/40 - Installing : tzdata-java-2022c-1.el8.noarch 6/40 - Installing : ttmkfdir-3.0.9-54.el8.x86_64 7/40 - Installing : lua-5.3.4-12.el8.x86_64 8/40 - Installing : copy-jdk-configs-4.0-2.el8.noarch 9/40 - Installing : libfontenc-1.1.3-8.el8.x86_64 10/40 - Installing : libXau-1.0.9-3.el8.x86_64 11/40 - Installing : libxcb-1.13.1-1.el8.x86_64 12/40 - Installing : libX11-common-1.6.8-5.el8.noarch 13/40 - Installing : libX11-1.6.8-5.el8.x86_64 14/40 - Installing : libXext-1.3.4-1.el8.x86_64 15/40 - Installing : libXi-1.7.10-1.el8.x86_64 16/40 - Installing : libXtst-1.2.3-7.el8.x86_64 17/40 - Installing : libXcomposite-0.4.4-14.el8.x86_64 18/40 - Installing : libXrender-0.9.10-7.el8.x86_64 19/40 - Installing : lcms2-2.9-2.el8.x86_64 20/40 - Running scriptlet: lcms2-2.9-2.el8.x86_64 20/40 - Installing : javapackages-filesystem-5.3.0-1.module+el8+5136+7f 21/40 - Installing : graphite2-1.3.10-10.el8.x86_64 22/40 - Installing : harfbuzz-1.7.5-3.el8.x86_64 23/40 - Running scriptlet: harfbuzz-1.7.5-3.el8.x86_64 23/40 - Installing : giflib-5.1.4-3.el8.x86_64 24/40 - Installing : alsa-lib-1.2.6.1-3.el8.x86_64 25/40 - Running scriptlet: alsa-lib-1.2.6.1-3.el8.x86_64 25/40 - Installing : pkgconf-m4-1.4.2-1.el8.noarch 26/40 - Installing : lksctp-tools-1.0.18-3.el8.x86_64 27/40 - Running scriptlet: lksctp-tools-1.0.18-3.el8.x86_64 27/40 - Installing : libpkgconf-1.4.2-1.el8.x86_64 28/40 - Installing : pkgconf-1.4.2-1.el8.x86_64 29/40 - Installing : pkgconf-pkg-config-1.4.2-1.el8.x86_64 30/40 - Installing : xorg-x11-font-utils-1:7.5-41.el8.x86_64 31/40 - Installing : xorg-x11-fonts-Type1-7.5-19.el8.noarch 32/40 - Running scriptlet: xorg-x11-fonts-Type1-7.5-19.el8.noarch 32/40 - Installing : crypto-policies-scripts-20211116-1.gitae470d6.el8. 33/40 - Installing : nss-sysinit-3.67.0-7.el8_5.x86_64 34/40 - Installing : nss-3.67.0-7.el8_5.x86_64 35/40 - Installing : avahi-libs-0.7-20.el8.x86_64 36/40 - Installing : cups-libs-1:2.2.6-45.el8_6.2.x86_64 37/40 - Installing : java-11-openjdk-headless-1:11.0.16.1.1-1.el8_6.x86 38/40 - Running scriptlet: java-11-openjdk-headless-1:11.0.16.1.1-1.el8_6.x86 38/40 - Installing : java-11-openjdk-1:11.0.16.1.1-1.el8_6.x86_64 39/40 - Running scriptlet: java-11-openjdk-1:11.0.16.1.1-1.el8_6.x86_64 39/40 - Installing : java-11-openjdk-devel-1:11.0.16.1.1-1.el8_6.x86_64 40/40 - Running scriptlet: java-11-openjdk-devel-1:11.0.16.1.1-1.el8_6.x86_64 40/40 - Running scriptlet: copy-jdk-configs-4.0-2.el8.noarch 40/40 - Running scriptlet: crypto-policies-scripts-20211116-1.gitae470d6.el8. 40/40 - Running scriptlet: nss-3.67.0-7.el8_5.x86_64 40/40 - Running scriptlet: java-11-openjdk-headless-1:11.0.16.1.1-1.el8_6.x86 40/40 - Running scriptlet: java-11-openjdk-1:11.0.16.1.1-1.el8_6.x86_64 40/40 - Running scriptlet: java-11-openjdk-devel-1:11.0.16.1.1-1.el8_6.x86_64 40/40 - Verifying : avahi-libs-0.7-20.el8.x86_64 1/40 - Verifying : crypto-policies-scripts-20211116-1.gitae470d6.el8. 2/40 - Verifying : cups-libs-1:2.2.6-45.el8_6.2.x86_64 3/40 - Verifying : libpkgconf-1.4.2-1.el8.x86_64 4/40 - Verifying : lksctp-tools-1.0.18-3.el8.x86_64 5/40 - Verifying : pkgconf-1.4.2-1.el8.x86_64 6/40 - Verifying : pkgconf-m4-1.4.2-1.el8.noarch 7/40 - Verifying : pkgconf-pkg-config-1.4.2-1.el8.x86_64 8/40 - Verifying : alsa-lib-1.2.6.1-3.el8.x86_64 9/40 - Verifying : copy-jdk-configs-4.0-2.el8.noarch 10/40 - Verifying : giflib-5.1.4-3.el8.x86_64 11/40 - Verifying : graphite2-1.3.10-10.el8.x86_64 12/40 - Verifying : harfbuzz-1.7.5-3.el8.x86_64 13/40 - Verifying : java-11-openjdk-1:11.0.16.1.1-1.el8_6.x86_64 14/40 - Verifying : java-11-openjdk-devel-1:11.0.16.1.1-1.el8_6.x86_64 15/40 - Verifying : java-11-openjdk-headless-1:11.0.16.1.1-1.el8_6.x86 16/40 - Verifying : javapackages-filesystem-5.3.0-1.module+el8+5136+7f 17/40 - Verifying : lcms2-2.9-2.el8.x86_64 18/40 - Verifying : libX11-1.6.8-5.el8.x86_64 19/40 - Verifying : libX11-common-1.6.8-5.el8.noarch 20/40 - Verifying : libXau-1.0.9-3.el8.x86_64 21/40 - Verifying : libXcomposite-0.4.4-14.el8.x86_64 22/40 - Verifying : libXext-1.3.4-1.el8.x86_64 23/40 - Verifying : libXi-1.7.10-1.el8.x86_64 24/40 - Verifying : libXrender-0.9.10-7.el8.x86_64 25/40 - Verifying : libXtst-1.2.3-7.el8.x86_64 26/40 - Verifying : libfontenc-1.1.3-8.el8.x86_64 27/40 - Verifying : libjpeg-turbo-1.5.3-12.el8.x86_64 28/40 - Verifying : libxcb-1.13.1-1.el8.x86_64 29/40 - Verifying : lua-5.3.4-12.el8.x86_64 30/40 - Verifying : nspr-4.32.0-1.el8_4.x86_64 31/40 - Verifying : nss-3.67.0-7.el8_5.x86_64 32/40 - Verifying : nss-softokn-3.67.0-7.el8_5.x86_64 33/40 - Verifying : nss-softokn-freebl-3.67.0-7.el8_5.x86_64 34/40 - Verifying : nss-sysinit-3.67.0-7.el8_5.x86_64 35/40 - Verifying : nss-util-3.67.0-7.el8_5.x86_64 36/40 - Verifying : ttmkfdir-3.0.9-54.el8.x86_64 37/40 - Verifying : tzdata-java-2022c-1.el8.noarch 38/40 - Verifying : xorg-x11-font-utils-1:7.5-41.el8.x86_64 39/40 - Verifying : xorg-x11-fonts-Type1-7.5-19.el8.noarch 40/40 - -Installed: - alsa-lib-1.2.6.1-3.el8.x86_64 - avahi-libs-0.7-20.el8.x86_64 - copy-jdk-configs-4.0-2.el8.noarch - crypto-policies-scripts-20211116-1.gitae470d6.el8.noarch - cups-libs-1:2.2.6-45.el8_6.2.x86_64 - giflib-5.1.4-3.el8.x86_64 - graphite2-1.3.10-10.el8.x86_64 - harfbuzz-1.7.5-3.el8.x86_64 - java-11-openjdk-1:11.0.16.1.1-1.el8_6.x86_64 - java-11-openjdk-devel-1:11.0.16.1.1-1.el8_6.x86_64 - java-11-openjdk-headless-1:11.0.16.1.1-1.el8_6.x86_64 - javapackages-filesystem-5.3.0-1.module+el8+5136+7ff78f74.noarch - lcms2-2.9-2.el8.x86_64 - libX11-1.6.8-5.el8.x86_64 - libX11-common-1.6.8-5.el8.noarch - libXau-1.0.9-3.el8.x86_64 - libXcomposite-0.4.4-14.el8.x86_64 - libXext-1.3.4-1.el8.x86_64 - libXi-1.7.10-1.el8.x86_64 - libXrender-0.9.10-7.el8.x86_64 - libXtst-1.2.3-7.el8.x86_64 - libfontenc-1.1.3-8.el8.x86_64 - libjpeg-turbo-1.5.3-12.el8.x86_64 - libpkgconf-1.4.2-1.el8.x86_64 - libxcb-1.13.1-1.el8.x86_64 - lksctp-tools-1.0.18-3.el8.x86_64 - lua-5.3.4-12.el8.x86_64 - nspr-4.32.0-1.el8_4.x86_64 - nss-3.67.0-7.el8_5.x86_64 - nss-softokn-3.67.0-7.el8_5.x86_64 - nss-softokn-freebl-3.67.0-7.el8_5.x86_64 - nss-sysinit-3.67.0-7.el8_5.x86_64 - nss-util-3.67.0-7.el8_5.x86_64 - pkgconf-1.4.2-1.el8.x86_64 - pkgconf-m4-1.4.2-1.el8.noarch - pkgconf-pkg-config-1.4.2-1.el8.x86_64 - ttmkfdir-3.0.9-54.el8.x86_64 - tzdata-java-2022c-1.el8.noarch - xorg-x11-font-utils-1:7.5-41.el8.x86_64 - xorg-x11-fonts-Type1-7.5-19.el8.noarch - -Complete! -Last metadata expiration check: 0:00:10 ago on Mon 12 Sep 2022 03:23:49 PM UTC. -Dependencies resolved. -============================================================================================== - Package - Arch Version Repository Size -============================================================================================== -Installing: - ords noarch 22.2.1-2.el8 yum.oracle.com_repo_OracleLinux_OL8_oracle_software_x86_64 83 M -Installing dependencies: - lsof x86_64 4.93.2-1.el8 ol8_baseos_latest 253 k - -Transaction Summary -============================================================================================== -Install 2 Packages - -Total download size: 83 M -Installed size: 87 M -Downloading Packages: -(1/2): lsof-4.93.2-1.el8.x86_64.rpm 3.0 MB/s | 253 kB 00:00 -(2/2): ords-22.2.1-2.el8.noarch.rpm 56 MB/s | 83 MB 00:01 --------------------------------------------------------------------------------- -Total 56 MB/s | 83 MB 00:01 -Running transaction check -Transaction check succeeded. -Running transaction test -Transaction test succeeded. -Running transaction - Preparing : 1/1 - Installing : lsof-4.93.2-1.el8.x86_64 1/2 - Running scriptlet: ords-22.2.1-2.el8.noarch 2/2 - Installing : ords-22.2.1-2.el8.noarch 2/2 - Running scriptlet: ords-22.2.1-2.el8.noarch 2/2 -INFO: Before starting ORDS service, run the below command as user oracle: - ords --config /etc/ords/config install - - Verifying : lsof-4.93.2-1.el8.x86_64 1/2 - Verifying : ords-22.2.1-2.el8.noarch 2/2 - -Installed: - lsof-4.93.2-1.el8.x86_64 ords-22.2.1-2.el8.noarch - -Complete! -Last metadata expiration check: 0:00:15 ago on Mon 12 Sep 2022 03:23:49 PM UTC. -Package iproute-5.15.0-4.el8.x86_64 is already installed. -Dependencies resolved. -Nothing to do. -Complete! -24 files removed -Removing intermediate container c08b8dac80a5 - ---> bb1a717f3e6e -Step 5/10 : RUN mkdir -p $ORDS_HOME/doc_root && mkdir -p $ORDS_HOME/error && mkdir -p $ORDS_HOME/secrets && chmod ug+x $ORDS_HOME/*.sh && groupadd -g 54322 dba && usermod -u 54321 -d /home/oracle -g dba -m -s /bin/bash oracle && chown -R oracle:dba $ORDS_HOME && echo "oracle ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers - ---> Running in 0103c070f4b6 -Removing intermediate container 0103c070f4b6 - ---> 089d06d9b198 -Step 6/10 : USER oracle - ---> Running in 51b1846c8c6f -Removing intermediate container 51b1846c8c6f - ---> 6c7b115954a4 -Step 7/10 : WORKDIR /home/oracle - ---> Running in 5862e2bc8df9 -Removing intermediate container 5862e2bc8df9 - ---> 28543543a88c -Step 8/10 : VOLUME ["$ORDS_HOME/config/ords"] - ---> Running in 465398d6f2bb -Removing intermediate container 465398d6f2bb - ---> 4037eb7f2f12 -Step 9/10 : EXPOSE 8888 - ---> Running in 2813ab5473f6 -Removing intermediate container 2813ab5473f6 - ---> 3410f1be2fff -Step 10/10 : CMD $ORDS_HOME/$RUN_FILE - ---> Running in 0a9a72408177 -Removing intermediate container 0a9a72408177 - ---> 2ef5dc95701b -Successfully built 2ef5dc95701b -Successfully tagged oracle/ords-dboper:22.2.1 - diff --git a/docs/multitenant/provisioning/ords_image.md b/docs/multitenant/provisioning/ords_image.md index 21abcbab..e2d1dcef 100644 --- a/docs/multitenant/provisioning/ords_image.md +++ b/docs/multitenant/provisioning/ords_image.md @@ -2,43 +2,60 @@ # Build ORDS Docker Image -In the below steps, we are building an ORDS Docker Image for ORDS Software. The image built can be later pushed to a local repository to be used later for a deployment. +This file contains the steps to create an ORDS based image to be used solely by the PDB life cycle multitentant controllers. **NOTE:** It is assumed that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-manage-pdb-life-cycle-using-oracle-db-operator-on-prem-database-controller) steps. -1. Clone the software using git: +#### Clone the software using git: + +> Under directory ./oracle-database-operator/ords you will find the [Dockerfile](../../../ords/Dockerfile) and [runOrdsSSL.sh](../../../ords/runOrdsSSL.sh) required to build the image. + ```sh git clone git@orahub.oci.oraclecorp.com:rac-docker-dev/oracle-database-operator.git cd oracle-database-operator/ords/ ``` -2. Login to the registry: container-registry.oracle.com +#### Login to the registry: container-registry.oracle.com **NOTE:** To login to this registry, you will need to the URL https://container-registry.oracle.com , Sign in, then click on "Java" and then accept the agreement. ```bash docker login container-registry.oracle.com -``` +``` -3. Login to a repo where you want to push your docker image (if needed) to pull during deployment in your environment. +#### Login to the your container registry + +Login to a repo where you want to push your docker image (if needed) to pull during deployment in your environment. ```bash docker login ``` -4. Build the docker image by using below command: +#### Build the image + +Build the docker image by using below command: ```bash -docker build -t oracle/ords-dboper:ords-latest . +docker build -t oracle/ords-dboper:latest . ``` +> If your are working behind a proxy mind to specify https_proxy and http_proxy during image creation -5. Check the docker image details using: +Check the docker image details using: ```bash docker images ``` -6. Tag and push the image to your docker repository. +> OUTPUT EXAMPLE +```bash +REPOSITORY TAG IMAGE ID CREATED SIZE +oracle/ords-dboper latest fdb17aa242f8 4 hours ago 1.46GB + +``` + +#### Tag and push the image + +Tag and push the image to your image repository. NOTE: We have the repo as `phx.ocir.io//oracle/ords:latest`. Please change as per your environment. @@ -47,18 +64,18 @@ docker tag oracle/ords-dboper:ords-latest phx.ocir.io//oracle/ords:la docker push phx.ocir.io//oracle/ords:latest ``` -7. Verify the image pushed to your docker repository. - -You can refer to below sample output for above steps as well. +#### In case of private image -8. Create a Kubernetes Secret for your docker repository to pull the image during deployment using the below command: +If you the image not be public then yuo need to create a secret containing the password of your image repository. +Create a Kubernetes Secret for your docker repository to pull the image during deployment using the below command: ```bash kubectl create secret generic container-registry-secret --from-file=.dockerconfigjson=./.docker/config.json --type=kubernetes.io/dockerconfigjson -n oracle-database-operator-system ``` -This Kubernetes secret will be provided in the .yaml file against the parameter `ordsImagePullSecret` to pull the ORDS Docker Image from your docker repository (if its a private repository). +Use the parameter `ordsImagePullSecret` to specify the container secrets in pod creation yaml file + +#### [Image createion example](../usecase01/logfiles/BuildImage.log) -## Sample Output -[Here](./ords_image.log) is the sample output for docker image created for ORDS latest version + diff --git a/docs/multitenant/provisioning/pdb_crd_resource.md b/docs/multitenant/provisioning/pdb_crd_resource.md deleted file mode 100644 index e69de29b..00000000 diff --git a/docs/multitenant/provisioning/plug_pdb.log b/docs/multitenant/provisioning/plug_pdb.log deleted file mode 100644 index 3a0be247..00000000 --- a/docs/multitenant/provisioning/plug_pdb.log +++ /dev/null @@ -1,100 +0,0 @@ --- Check the status of the PDB CRD Resources: - -% kubectl get pdbs -A -No resources found - - - --- Verify from the CDB: - -SQL> show pdbs - - CON_ID CON_NAME OPEN MODE RESTRICTED ----------- ------------------------------ ---------- ---------- - 2 PDB$SEED READ WRITE NO - - - --- Confirm the availability of the required .xml file: - -[oracle@goldhost1 ~]$ ls -lrt /tmp/pdbnewclone.xml --rw-r--r-- 1 oracle asmadmin 9920 Jun 27 06:26 /tmp/pdbnewclone.xml - - --- Use the below .yaml file for the plug in operation: - -% cat plug_pdb.yaml -# -# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. -# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. -# -apiVersion: database.oracle.com/v1alpha1 -kind: PDB -metadata: - name: pdb1 - namespace: oracle-database-operator-system - labels: - cdb: cdb-dev -spec: - cdbResName: "cdb-dev" - cdbName: "goldcdb" - pdbName: "pdbnew" - xmlFileName: "/tmp/pdbnewclone.xml" - fileNameConversions: "NONE" - sourceFileNameConversions: "NONE" - copyAction: "MOVE" - totalSize: "1G" - tempSize: "100M" - action: "Plug" - - - --- Apply the .yaml file: - -% kubectl apply -f plug_pdb.yaml -pdb.database.oracle.com/pdb1 created - - --- Monitor the logs from the Oracle DB Operator Pod: - -% kubectl logs -f pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd -n oracle-database-operator-system -. -. -2022-06-27T04:28:36Z DEBUG controller-runtime.webhook.webhooks received request {"webhook": "/validate-database-oracle-com-v1alpha1-pdb", "UID": "bfad69af-36be-4792-87e3-639323300167", "kind": "database.oracle.com/v1alpha1, Kind=PDB", "resource": {"group":"database.oracle.com","version":"v1alpha1","resource":"pdbs"}} -2022-06-27T04:28:36Z INFO pdb-webhook ValidateCreate-Validating PDB spec for : pdb1 -2022-06-27T04:28:36Z INFO pdb-webhook validateCommon {"name": "pdb1"} -2022-06-27T04:28:36Z INFO pdb-webhook Valdiating PDB Resource Action : PLUG -2022-06-27T04:28:36Z INFO pdb-webhook PDB Resource : pdb1 successfully validated for Action : PLUG -2022-06-27T04:28:36Z DEBUG controller-runtime.webhook.webhooks wrote response {"webhook": "/validate-database-oracle-com-v1alpha1-pdb", "code": 200, "reason": "", "UID": "bfad69af-36be-4792-87e3-639323300167", "allowed": true} -2022-06-27T04:28:36Z INFO controllers.PDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/pdb1"} -2022-06-27T04:28:36Z INFO controllers.PDB Adding finalizer {"managePDBDeletion": "oracle-database-operator-system/pdb1"} -2022-06-27T04:28:36Z INFO controllers.PDB Found PDB: pdb1 {"checkDuplicatePDB": "oracle-database-operator-system/pdb1"} -2022-06-27T04:28:36Z INFO controllers.PDB Validating PDB phase for: pdb1 {"validatePhase": "oracle-database-operator-system/pdb1", "Action": "PLUG"} -2022-06-27T04:28:36Z INFO controllers.PDB Validation complete {"validatePhase": "oracle-database-operator-system/pdb1"} -2022-06-27T04:28:36Z INFO controllers.PDB PDB: {"onpremdboperator": "oracle-database-operator-system/pdb1", "Name": "pdb1", "Phase": "Plugging", "Status": "false"} -2022-06-27T04:28:36Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} -2022-06-27T04:28:36Z INFO controllers.PDB Issuing REST call {"callAPI": "oracle-database-operator-system/pdb1", "URL": "http://cdb-dev-ords:8888/ords/_/db-api/latest/database/pdbs/", "Action": "POST"} -2022-06-27T04:28:36Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} -2022-06-27T04:29:07Z INFO controllers.PDB Successfully plugged PDB {"plugPDB": "oracle-database-operator-system/pdb1", "PDB Name": "pdbnew"} -2022-06-27T04:29:07Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} -2022-06-27T04:29:07Z DEBUG events Normal {"object": {"kind":"PDB","namespace":"oracle-database-operator-system","name":"pdb1","uid":"dd9bef3c-e493-4d5a-ae82-b24cbf5d0be3","apiVersion":"database.oracle.com/v1alpha1","resourceVersion":"101465242"}, "reason": "Created", "message": "PDB 'pdbnew' plugged successfully"} -2022-06-27T04:29:07Z INFO controllers.PDB Issuing REST call {"callAPI": "oracle-database-operator-system/pdb1", "URL": "http://cdb-dev-ords:8888/ords/_/db-api/latest/database/pdbs/pdbnew/status", "Action": "GET"} -2022-06-27T04:29:07Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} -2022-06-27T04:29:07Z INFO controllers.PDB Successfully obtained PDB state {"getPDBState": "oracle-database-operator-system/pdb1", "PDB Name": "pdbnew", "State": "READ WRITE"} -2022-06-27T04:29:07Z INFO controllers.PDB Reconcile completed {"onpremdboperator": "oracle-database-operator-system/pdb1"} - - - --- Confirm the PDB CRD resource has been created and the PDB has been plugged in to the target CDB: - -% kubectl get pdbs -A -NAMESPACE NAME CONNECT STRING CDB NAME PDB NAME PDB STATE PDB SIZE STATUS MESSAGE -oracle-database-operator-system pdb1 goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com:1521/pdbnew goldcdb pdbnew READ WRITE 1G Ready Success - - -SQL> show pdbs - - CON_ID CON_NAME OPEN MODE RESTRICTED ----------- ------------------------------ ---------- ---------- - 2 PDB$SEED READ WRITE NO - 4 PDBNEW READ WRITE NO diff --git a/docs/multitenant/provisioning/plug_pdb.md b/docs/multitenant/provisioning/plug_pdb.md deleted file mode 100644 index 02645f52..00000000 --- a/docs/multitenant/provisioning/plug_pdb.md +++ /dev/null @@ -1,44 +0,0 @@ -# Plug in a PDB using Oracle DB Operator On-Prem Controller in a target CDB - -In this use case, a PDB is plugged in using Oracle DB Operator On-Prem controller using an existing .xml file which was generated when the PDB was unplugged from this target CDB or another CDB. - -To plug in a PDB CRD Resource, a sample .yaml file is available here: [config/samples/onpremdb/pdb_plug.yaml](../../../config/samples/onpremdb/pdb_plug.yaml) - -**NOTE:** It is assumed that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-manage-pdb-life-cycle-using-oracle-db-operator-on-prem-database-controller) steps. - -This example uses `plug_pdb.yaml` to plug in a PDB to a target CDB using Oracle DB Operator On-Prem Controller with: - -- Pluggable Database CRD Resource Name as `pdb1` -- Pluggable Database (PDB) Name as `pdbnew` -- Target CDB CRD Resource Name as `cdb-dev` -- CDB Name as `goldcdb` -- Action to be taken on the PDB as `Plug` -- XML metadata filename as `/tmp/pdbnewclone.xml` -- Source File Name Conversion as `NONE` -- File Name Conversion as `NONE` -- Copy Action as `MOVE` -- PDB Size as `1G` -- Temporary tablespace Size as `100M` - -**NOTE:** For the details of the parameters to be used in the .yaml file, please refer [config/crd/bases/database.oracle.com_pdbs.yaml](../../../config/crd/bases/database.oracle.com_pdbs.yaml) - -**NOTE:** Before performing the plug inoperation, you will first need to confirm the availability of the .xml file and the PDB datafiles. - -Use the file: [plug_pdb.yaml](./plug_pdb.yaml) for this use case as below: - -1. Deploy the .yaml file: -```sh -% kubectl apply -f plug_pdb.yaml -``` - -2. Monitor the Oracle DB Operator Pod for the progress of the PDB Unplug operation: - -NOTE: Check the DB Operator Pod name in your environment. - -```sh -% kubectl logs -f pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd -n oracle-database-operator-system -``` - -## Sample Output - -[Here](./plug_pdb.log) is the sample output for a PDB created using Oracle DB Operator On-Prem Controller using file [plug_pdb.yaml](./plug_pdb.yaml) diff --git a/docs/multitenant/provisioning/plug_pdb.yaml b/docs/multitenant/provisioning/plug_pdb.yaml deleted file mode 100644 index a27d3255..00000000 --- a/docs/multitenant/provisioning/plug_pdb.yaml +++ /dev/null @@ -1,22 +0,0 @@ -# -# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. -# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. -# -apiVersion: database.oracle.com/v1alpha1 -kind: PDB -metadata: - name: pdb1 - namespace: oracle-database-operator-system - labels: - cdb: cdb-dev -spec: - cdbResName: "cdb-dev" - cdbName: "goldcdb" - pdbName: "pdbnew" - xmlFileName: "/tmp/pdbnewclone.xml" - fileNameConversions: "NONE" - sourceFileNameConversions: "NONE" - copyAction: "MOVE" - totalSize: "1G" - tempSize: "100M" - action: "Plug" diff --git a/docs/multitenant/provisioning/quickOKEcreation.md b/docs/multitenant/provisioning/quickOKEcreation.md new file mode 100644 index 00000000..19d9323e --- /dev/null +++ b/docs/multitenant/provisioning/quickOKEcreation.md @@ -0,0 +1,136 @@ + + +### Quick Oke creation script + +Use this script to create quickly an OKE cluster in your OCI. + +#### Prerequisties: +- ocicli is properly configured on your client +- make is installed on your client +- vnc is already configured +- ssh key is configured (public key available under directory ~/.ssh) +- edit make providing all the information about your compartment, vnc,subnet,lb subnet and nd subnet (exported variables in the header section) + + +#### Execution: + +```bash +make all +``` + +Monitor the OKE from OCI console + +#### Makefile +```makefile +.EXPORT_ALL_VARIABLES: + +export CMPID=[.... COMPARTMENT ID.............] +export VNCID=[.... VNC ID ....................] +export ENDID=[.... SUBNET END POINT ID .......] +export LBSID=[.....LB SUBNET ID...............] +export NDSID=[.....NODE SUBNET ID.............] + + +#ssh public key +export KEYFL=~/.ssh/id_rsa.pub + +#cluster version +export KSVER=v1.27.2 + +#cluster name +export CLUNM=myoke + +#pool name +export PLNAM=Pool1 + +#logfile +export LOGFILE=./clustoke.log + +#shape +export SHAPE=VM.Standard.E4.Flex + +OCI=/home/oracle/bin/oci +CUT=/usr/bin/cut +KUBECTL=/usr/bin/kubectl +CAT=/usr/bin/cat + +all: cluster waitcluster pool waitpool config desccluster + +cluster: + @echo " - CREATING CLUSTER " + @$(OCI) ce cluster create \ + --compartment-id $(CMPID) \ + --kubernetes-version $(KSVER) \ + --name $(CLUNM) \ + --vcn-id $(VNCID) \ + --endpoint-subnet-id $(ENDID) \ + --service-lb-subnet-ids '["'$(LBSID)'"]' \ + --endpoint-public-ip-enabled true \ + --persistent-volume-freeform-tags '{"$(CLUNM)" : "OKE"}' 1>$(LOGFILE) 2>&1 + +waitcluster: + @while [ `$(OCI) ce cluster list --compartment-id $(CMPID) \ + --name $(CLUNM) --lifecycle-state ACTIVE --query data[0].id \ + --raw-output |wc -l ` -eq 0 ] ; do sleep 5 ; done + @echo " - CLUSTER CREATED" + + +pool: + @echo " - CREATING POOL" + @$(eval PBKEY :=$(shell $(CAT) $(KEYFL)|grep -v " PUBLIC KEY")) + @$(OCI) ce node-pool create \ + --cluster-id `$(OCI) ce cluster list --compartment-id $(CMPID) \ + --name $(CLUNM) --lifecycle-state ACTIVE --query data[0].id --raw-output` \ + --compartment-id $(CMPID) \ + --kubernetes-version $(KSVER) \ + --name $(PLNAM) \ + --node-shape $(SHAPE) \ + --node-shape-config '{"memoryInGBs": 8.0, "ocpus": 1.0}' \ + --node-image-id `$(OCI) compute image list \ + --operating-system 'Oracle Linux' --operating-system-version 7.9 \ + --sort-by TIMECREATED --compartment-id $(CMPID) --shape $(SHAPE) \ + --query data[1].id --raw-output` \ + --node-boot-volume-size-in-gbs 50 \ + --ssh-public-key "$(PBKEY)" \ + --size 3 \ + --placement-configs '[{"availabilityDomain": "'`oci iam availability-domain list \ + --compartment-id $(CMPID) \ + --query data[0].name --raw-output`'", "subnetId": "'$(NDSID)'"}]' 1>>$(LOGFILE) 2>&1 + +waitpool: + $(eval CLSID :=$(shell $(OCI) ce cluster list --compartment-id $(CMPID) \ + --name $(CLUNM) --lifecycle-state ACTIVE --query data[0].id --raw-output)) + @while [ `$(OCI) ce node-pool list --compartment-id $(CMPID) \ + --lifecycle-state ACTIVE --cluster-id $(CLSID) \ + --query data[0].id --raw-output |wc -l ` -eq 0 ] ; do sleep 5 ; done + @sleep 10 + $(eval PLLID :=$(shell $(OCI) ce node-pool list --compartment-id $(CMPID) \ + --lifecycle-state ACTIVE --cluster-id $(CLSID) --query data[0].id --raw-output)) + @echo " - POOL CREATED" + +config: + @$(OCI) ce cluster create-kubeconfig --cluster-id \ + `$(OCI) ce cluster list \ + --compartment-id $(CMPID) --name $(CLUNM) --lifecycle-state ACTIVE \ + --query data[0].id --raw-output` \ + --file $(HOME)/.kube/config --region \ + `$(OCI) ce cluster list \ + --compartment-id $(CMPID) --name $(CLUNM) --lifecycle-state ACTIVE \ + --query data[0].id --raw-output|$(CUT) -f4 -d. ` \ + --token-version 2.0.0 --kube-endpoint PUBLIC_ENDPOINT + @echo " - KUBECTL PUBLIC ENDPOINT CONFIGURED" + + +desccluster: + @$(eval TMPSP := $(shell date "+%y/%m/%d:%H:%M" )) + $(KUBECTL) get nodes -o wide + $(KUBECTL) get storageclass + +checkvol: + $(OCI) bv volume list \ + --compartment-id $(CMPID) \ + --lifecycle-state AVAILABLE \ + --query 'data[?"freeform-tags".stackgres == '\''OKE'\''].id' +``` + + diff --git a/docs/multitenant/provisioning/singlenamespace/cdb_create.yaml b/docs/multitenant/provisioning/singlenamespace/cdb_create.yaml new file mode 100644 index 00000000..01fc0a18 --- /dev/null +++ b/docs/multitenant/provisioning/singlenamespace/cdb_create.yaml @@ -0,0 +1,44 @@ +apiVersion: database.oracle.com/v1alpha1 +kind: CDB +metadata: + name: cdb-dev + namespace: oracle-database-operator-system +spec: + cdbName: "DB12" + ordsImage: ".............your registry............./ords-dboper:latest" + ordsImagePullPolicy: "Always" + dbTnsurl : "...Container tns alias....." + replicas: 1 + sysAdminPwd: + secret: + secretName: "cdb1-secret" + key: "sysadmin_pwd" + ordsPwd: + secret: + secretName: "cdb1-secret" + key: "ords_pwd" + cdbAdminUser: + secret: + secretName: "cdb1-secret" + key: "cdbadmin_user" + cdbAdminPwd: + secret: + secretName: "cdb1-secret" + key: "cdbadmin_pwd" + webServerUser: + secret: + secretName: "cdb1-secret" + key: "webserver_user" + webServerPwd: + secret: + secretName: "cdb1-secret" + key: "webserver_pwd" + cdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + cdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + diff --git a/docs/multitenant/provisioning/singlenamespace/cdb_secret.yaml b/docs/multitenant/provisioning/singlenamespace/cdb_secret.yaml new file mode 100644 index 00000000..567b90a4 --- /dev/null +++ b/docs/multitenant/provisioning/singlenamespace/cdb_secret.yaml @@ -0,0 +1,17 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: v1 +kind: Secret +metadata: + name: cdb1-secret + namespace: oracle-database-operator-system +type: Opaque +data: + ords_pwd: ".....base64 encoded password...." + sysadmin_pwd: ".....base64 encoded password...." + cdbadmin_user: ".....base64 encoded password...." + cdbadmin_pwd: ".....base64 encoded password...." + webserver_user: ".....base64 encoded password...." + webserver_pwd: ".....base64 encoded password...." diff --git a/docs/multitenant/provisioning/singlenamespace/pdb_clone.yaml b/docs/multitenant/provisioning/singlenamespace/pdb_clone.yaml new file mode 100644 index 00000000..0ecc3c70 --- /dev/null +++ b/docs/multitenant/provisioning/singlenamespace/pdb_clone.yaml @@ -0,0 +1,50 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb2 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "oracle-database-operator-system" + cdbName: "DB12" + pdbName: "pdb2_clone" + srcPdbName: "pdbdev" + fileNameConversions: "NONE" + totalSize: "UNLIMITED" + tempSize: "UNLIMITED" + assertivePdbDeletion: true + adminName: + secret: + secretName: "pdb1-secret" + key: "sysadmin_user" + adminPwd: + secret: + secretName: "pdb1-secret" + key: "sysadmin_pwd" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "pdb1-secret" + key: "webserver_user" + webServerPwd: + secret: + secretName: "pdb1-secret" + key: "webserver_pwd" + action: "Clone" diff --git a/docs/multitenant/provisioning/singlenamespace/pdb_close.yaml b/docs/multitenant/provisioning/singlenamespace/pdb_close.yaml new file mode 100644 index 00000000..5917d33a --- /dev/null +++ b/docs/multitenant/provisioning/singlenamespace/pdb_close.yaml @@ -0,0 +1,44 @@ +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "oracle-database-operator-system" + cdbName: "DB12" + pdbName: "pdbdev" + adminName: + secret: + secretName: "pdb1-secret" + key: "sysadmin_user" + adminPwd: + secret: + secretName: "pdb1-secret" + key: "sysadmin_pwd" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "pdb1-secret" + key: "webserver_user" + webServerPwd: + secret: + secretName: "pdb1-secret" + key: "webserver_pwd" + pdbState: "CLOSE" + modifyOption: "IMMEDIATE" + action: "Modify" + diff --git a/docs/multitenant/provisioning/singlenamespace/pdb_create.yaml b/docs/multitenant/provisioning/singlenamespace/pdb_create.yaml new file mode 100644 index 00000000..be3581ad --- /dev/null +++ b/docs/multitenant/provisioning/singlenamespace/pdb_create.yaml @@ -0,0 +1,47 @@ +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "oracle-database-operator-system" + cdbName: "DB12" + pdbName: "pdbdev" + adminName: + secret: + secretName: "pdb1-secret" + key: "sysadmin_user" + adminPwd: + secret: + secretName: "pdb1-secret" + key: "sysadmin_pwd" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "pdb1-secret" + key: "webserver_user" + webServerPwd: + secret: + secretName: "pdb1-secret" + key: "webserver_pwd" + fileNameConversions: "NONE" + tdeImport: false + totalSize: "1G" + tempSize: "100M" + action: "Create" + assertivePdbDeletion: true + diff --git a/docs/multitenant/provisioning/singlenamespace/pdb_delete.yaml b/docs/multitenant/provisioning/singlenamespace/pdb_delete.yaml new file mode 100644 index 00000000..c22b546a --- /dev/null +++ b/docs/multitenant/provisioning/singlenamespace/pdb_delete.yaml @@ -0,0 +1,34 @@ +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "oracle-database-operator-system" + pdbName: "pdbdev" + action: "Delete" + dropAction: "INCLUDING" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "pdb1-secret" + key: "webserver_user" + webServerPwd: + secret: + secretName: "pdb1-secret" + key: "webserver_pwd" + diff --git a/docs/multitenant/provisioning/singlenamespace/pdb_open.yaml b/docs/multitenant/provisioning/singlenamespace/pdb_open.yaml new file mode 100644 index 00000000..25fdccc4 --- /dev/null +++ b/docs/multitenant/provisioning/singlenamespace/pdb_open.yaml @@ -0,0 +1,43 @@ +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "oracle-database-operator-system" + cdbName: "DB12" + pdbName: "pdbdev" + adminName: + secret: + secretName: "pdb1-secret" + key: "sysadmin_user" + adminPwd: + secret: + secretName: "pdb1-secret" + key: "sysadmin_pwd" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "pdb1-secret" + key: "webserver_user" + webServerPwd: + secret: + secretName: "pdb1-secret" + key: "webserver_pwd" + action: "Modify" + pdbState: "OPEN" + modifyOption: "READ WRITE" diff --git a/docs/multitenant/provisioning/singlenamespace/pdb_plug.yaml b/docs/multitenant/provisioning/singlenamespace/pdb_plug.yaml new file mode 100644 index 00000000..77c00b9c --- /dev/null +++ b/docs/multitenant/provisioning/singlenamespace/pdb_plug.yaml @@ -0,0 +1,46 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "oracle-database-operator-system" + cdbName: "DB12" + pdbName: "pdbdev" + xmlFileName: "/tmp/pdb.xml" + fileNameConversions: "NONE" + sourceFileNameConversions: "NONE" + copyAction: "MOVE" + totalSize: "1G" + tempSize: "100M" + action: "Plug" + assertivePdbDeletion: true + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "pdb1-secret" + key: "webserver_user" + webServerPwd: + secret: + secretName: "pdb1-secret" + key: "webserver_pwd" + + diff --git a/docs/multitenant/provisioning/pdb_secret.yaml b/docs/multitenant/provisioning/singlenamespace/pdb_secret.yaml similarity index 54% rename from docs/multitenant/provisioning/pdb_secret.yaml rename to docs/multitenant/provisioning/singlenamespace/pdb_secret.yaml index cf385dda..60d95d76 100644 --- a/docs/multitenant/provisioning/pdb_secret.yaml +++ b/docs/multitenant/provisioning/singlenamespace/pdb_secret.yaml @@ -7,7 +7,10 @@ kind: Secret metadata: name: pdb1-secret namespace: oracle-database-operator-system -type: Opaque -data: - sysadmin_user: "[ base64 encode values ]" - sysadmin_pwd: "[base64 encode values ]" +type: Opaque +data: + sysadmin_user: ".....base64 encoded password...." + sysadmin_pwd: ".....base64 encoded password...." + webserver_user: ".....base64 encoded password...." + webserver_pwd: ".....base64 encoded password...." + diff --git a/docs/multitenant/provisioning/pdb.yaml b/docs/multitenant/provisioning/singlenamespace/pdb_unplug.yaml similarity index 50% rename from docs/multitenant/provisioning/pdb.yaml rename to docs/multitenant/provisioning/singlenamespace/pdb_unplug.yaml index 82941185..085d337e 100644 --- a/docs/multitenant/provisioning/pdb.yaml +++ b/docs/multitenant/provisioning/singlenamespace/pdb_unplug.yaml @@ -11,17 +11,29 @@ metadata: cdb: cdb-dev spec: cdbResName: "cdb-dev" - cdbName: "goldcdb" - pdbName: "pdbnew" - adminName: + cdbNamespace: "oracle-database-operator-system" + cdbName: "DB12" + pdbName: "pdbdev" + xmlFileName: "/tmp/pdb.xml" + action: "Unplug" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: secret: secretName: "pdb1-secret" - key: "sysadmin_user" - adminPwd: + key: "webserver_user" + webServerPwd: secret: secretName: "pdb1-secret" - key: "sysadmin_pwd" - fileNameConversions: "NONE" - totalSize: "1G" - tempSize: "100M" - action: "Create" + key: "webserver_pwd" + diff --git a/docs/multitenant/provisioning/unplug_pdb.log b/docs/multitenant/provisioning/unplug_pdb.log deleted file mode 100644 index c0995f83..00000000 --- a/docs/multitenant/provisioning/unplug_pdb.log +++ /dev/null @@ -1,165 +0,0 @@ --- Check the status of the PDB CRD resource: - -% kubectl get pdbs -A -NAMESPACE NAME CONNECT STRING CDB NAME PDB NAME PDB STATE PDB SIZE STATUS MESSAGE -oracle-database-operator-system pdb1 goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com:1521/pdbnew goldcdb pdbnew READ WRITE 1G Ready Success - - --- Verify the status from the CDB: - -SQL> show pdbs - - CON_ID CON_NAME OPEN MODE RESTRICTED ----------- ------------------------------ ---------- ---------- - 2 PDB$SEED READ WRITE NO - 3 PDBNEW READ WRITE NO - - --- Use the below .yaml file to close the PDB: - -% cat modify_pdb_close.yaml -# -# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. -# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. -# -apiVersion: database.oracle.com/v1alpha1 -kind: PDB -metadata: - name: pdb1 - namespace: oracle-database-operator-system - labels: - cdb: cdb-dev -spec: - cdbResName: "cdb-dev" - cdbName: "goldcdb" - pdbName: "pdbnew" - pdbState: "CLOSE" - modifyOption: "IMMEDIATE" - action: "Modify" - - --- Apply the .yaml file: - -% kubectl apply -f modify_pdb_close.yaml -pdb.database.oracle.com/pdb1 configured - - --- Monitor the Oracle DB Operator Pod logs: - -% kubectl logs -f pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd -n oracle-database-operator-system -. -. -2022-06-27T04:25:00Z DEBUG controller-runtime.webhook.webhooks received request {"webhook": "/mutate-database-oracle-com-v1alpha1-pdb", "UID": "1623a6e2-d7dc-4b0f-8aa8-efada76cac13", "kind": "database.oracle.com/v1alpha1, Kind=PDB", "resource": {"group":"database.oracle.com","version":"v1alpha1","resource":"pdbs"}} -2022-06-27T04:25:00Z INFO pdb-webhook Setting default values in PDB spec for : pdb1 -2022-06-27T04:25:00Z DEBUG controller-runtime.webhook.webhooks wrote response {"webhook": "/mutate-database-oracle-com-v1alpha1-pdb", "code": 200, "reason": "", "UID": "1623a6e2-d7dc-4b0f-8aa8-efada76cac13", "allowed": true} -2022-06-27T04:25:00Z INFO controllers.PDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/pdb1"} -2022-06-27T04:25:00Z INFO controllers.PDB Validating PDB phase for: pdb1 {"validatePhase": "oracle-database-operator-system/pdb1", "Action": "MODIFY"} -2022-06-27T04:25:00Z INFO controllers.PDB Validation complete {"validatePhase": "oracle-database-operator-system/pdb1"} -2022-06-27T04:25:00Z INFO controllers.PDB PDB: {"onpremdboperator": "oracle-database-operator-system/pdb1", "Name": "pdb1", "Phase": "Modifying", "Status": "false"} -2022-06-27T04:25:00Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} -2022-06-27T04:25:00Z INFO controllers.PDB Issuing REST call {"callAPI": "oracle-database-operator-system/pdb1", "URL": "http://cdb-dev-ords:8888/ords/_/db-api/latest/database/pdbs/pdbnew/status", "Action": "GET"} -2022-06-27T04:25:00Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} -2022-06-27T04:25:00Z INFO controllers.PDB Successfully obtained PDB state {"getPDBState": "oracle-database-operator-system/pdb1", "PDB Name": "pdbnew", "State": "READ WRITE"} -2022-06-27T04:25:00Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} -2022-06-27T04:25:01Z INFO controllers.PDB Issuing REST call {"callAPI": "oracle-database-operator-system/pdb1", "URL": "http://cdb-dev-ords:8888/ords/_/db-api/latest/database/pdbs/pdbnew/status", "Action": "POST"} -2022-06-27T04:25:01Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} -2022-06-27T04:25:02Z INFO controllers.PDB Successfully modified PDB state {"modifyPDB": "oracle-database-operator-system/pdb1", "PDB Name": "pdbnew"} -2022-06-27T04:25:02Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} -2022-06-27T04:25:02Z DEBUG events Normal {"object": {"kind":"PDB","namespace":"oracle-database-operator-system","name":"pdb1","uid":"447346c7-cfb0-43ed-abb2-a0fac844a3e4","apiVersion":"database.oracle.com/v1alpha1","resourceVersion":"101464133"}, "reason": "Modified", "message": "PDB 'pdbnew' modified successfully"} -2022-06-27T04:25:02Z INFO controllers.PDB Issuing REST call {"callAPI": "oracle-database-operator-system/pdb1", "URL": "http://cdb-dev-ords:8888/ords/_/db-api/latest/database/pdbs/pdbnew/status", "Action": "GET"} -2022-06-27T04:25:02Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} -2022-06-27T04:25:02Z INFO controllers.PDB Successfully obtained PDB state {"getPDBState": "oracle-database-operator-system/pdb1", "PDB Name": "pdbnew", "State": "MOUNTED"} -2022-06-27T04:25:02Z INFO controllers.PDB Reconcile completed {"onpremdboperator": "oracle-database-operator-system/pdb1"} - - - --- Confirm the PDB is now in MOUNT status: - -% kubectl get pdbs -A -NAMESPACE NAME CONNECT STRING CDB NAME PDB NAME PDB STATE PDB SIZE STATUS MESSAGE -oracle-database-operator-system pdb1 goldhost-scan.lbsub52b3b1cae.okecluster.oraclevcn.com:1521/pdbnew goldcdb pdbnew MOUNTED 1G Ready Success - - -SQL> show pdbs - - CON_ID CON_NAME OPEN MODE RESTRICTED ----------- ------------------------------ ---------- ---------- - 2 PDB$SEED READ WRITE NO - 3 PDBNEW MOUNTED - - - - --- Use the below .yaml file to unplug the PDB: - -% cat unplug_pdb.yaml -# -# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. -# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. -# -apiVersion: database.oracle.com/v1alpha1 -kind: PDB -metadata: - name: pdb1 - namespace: oracle-database-operator-system - labels: - cdb: cdb-dev -spec: - cdbResName: "cdb-dev" - cdbName: "goldcdb" - pdbName: "pdbnew" - xmlFileName: "/tmp/pdbnewclone.xml" - action: "Unplug" - - --- Apply the .yaml file: - -% kubectl apply -f unplug_pdb.yaml -pdb.database.oracle.com/pdb1 configured - - --- Monitor the Oracle DB Operator Pod logs: - -% kubectl logs -f pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd -n oracle-database-operator-system -. -. -2022-06-27T04:26:10Z DEBUG controller-runtime.webhook.webhooks received request {"webhook": "/mutate-database-oracle-com-v1alpha1-pdb", "UID": "0f292426-8839-46b6-ba30-b3ffeee7e644", "kind": "database.oracle.com/v1alpha1, Kind=PDB", "resource": {"group":"database.oracle.com","version":"v1alpha1","resource":"pdbs"}} -2022-06-27T04:26:10Z INFO pdb-webhook Setting default values in PDB spec for : pdb1 -2022-06-27T04:26:10Z DEBUG controller-runtime.webhook.webhooks wrote response {"webhook": "/mutate-database-oracle-com-v1alpha1-pdb", "code": 200, "reason": "", "UID": "0f292426-8839-46b6-ba30-b3ffeee7e644", "allowed": true} -2022-06-27T04:26:10Z INFO controllers.PDB Reconcile requested {"onpremdboperator": "oracle-database-operator-system/pdb1"} -2022-06-27T04:26:10Z INFO controllers.PDB Validating PDB phase for: pdb1 {"validatePhase": "oracle-database-operator-system/pdb1", "Action": "UNPLUG"} -2022-06-27T04:26:10Z INFO controllers.PDB Validation complete {"validatePhase": "oracle-database-operator-system/pdb1"} -2022-06-27T04:26:10Z INFO controllers.PDB PDB: {"onpremdboperator": "oracle-database-operator-system/pdb1", "Name": "pdb1", "Phase": "Unplugging", "Status": "false"} -2022-06-27T04:26:10Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} -2022-06-27T04:26:10Z INFO controllers.PDB Issuing REST call {"callAPI": "oracle-database-operator-system/pdb1", "URL": "http://cdb-dev-ords:8888/ords/_/db-api/latest/database/pdbs/pdbnew/", "Action": "POST"} -2022-06-27T04:26:10Z INFO controllers.PDB Found CR for CDB {"getCDBResource": "oracle-database-operator-system/pdb1", "Name": "cdb-dev", "CR Name": "cdb-dev"} -2022-06-27T04:26:18Z INFO controllers.PDB Removing finalizer {"unplugPDB": "oracle-database-operator-system/pdb1"} -2022-06-27T04:26:19Z DEBUG controller-runtime.webhook.webhooks received request {"webhook": "/validate-database-oracle-com-v1alpha1-pdb", "UID": "ce9e1a52-a372-4e3b-b148-c1e31bcb26f8", "kind": "database.oracle.com/v1alpha1, Kind=PDB", "resource": {"group":"database.oracle.com","version":"v1alpha1","resource":"pdbs"}} -2022-06-27T04:26:19Z INFO pdb-webhook ValidateUpdate-Validating PDB spec for : pdb1 -2022-06-27T04:26:19Z INFO pdb-webhook validateCommon {"name": "pdb1"} -2022-06-27T04:26:19Z INFO pdb-webhook Valdiating PDB Resource Action : UNPLUG -2022-06-27T04:26:19Z DEBUG controller-runtime.webhook.webhooks wrote response {"webhook": "/validate-database-oracle-com-v1alpha1-pdb", "code": 200, "reason": "", "UID": "ce9e1a52-a372-4e3b-b148-c1e31bcb26f8", "allowed": true} -2022-06-27T04:26:19Z INFO controllers.PDB Successfully unplugged PDB resource {"unplugPDB": "oracle-database-operator-system/pdb1"} -2022-06-27T04:26:19Z INFO controllers.PDB Reconcile completed {"onpremdboperator": "oracle-database-operator-system/pdb1"} -2022-06-27T04:26:19Z DEBUG events Normal {"object": {"kind":"PDB","namespace":"oracle-database-operator-system","name":"pdb1","uid":"447346c7-cfb0-43ed-abb2-a0fac844a3e4","apiVersion":"database.oracle.com/v1alpha1","resourceVersion":"101464533"}, "reason": "Unplugged", "message": "PDB 'pdbnew' unplugged successfully"} - - - --- Confirm the PDB has been unplugged: - -% kubectl get pdbs -A -No resources found - - -SQL> show pdbs - - CON_ID CON_NAME OPEN MODE RESTRICTED ----------- ------------------------------ ---------- ---------- - 2 PDB$SEED READ WRITE NO - - - --- Confirm the .xml file generated in the CDB host: - -[oracle@goldhost1 ~]$ ls -lrt /tmp/pdbnewclone.xml --rw-r--r-- 1 oracle asmadmin 9920 Jun 27 06:26 /tmp/pdbnewclone.xml diff --git a/docs/multitenant/provisioning/unplug_pdb.md b/docs/multitenant/provisioning/unplug_pdb.md deleted file mode 100644 index fb98fc8b..00000000 --- a/docs/multitenant/provisioning/unplug_pdb.md +++ /dev/null @@ -1,39 +0,0 @@ -# Unplug a PDB using Oracle DB Operator On-Prem Controller in a target CDB - -In this use case, a PDB is unplugged using Oracle DB Operator On-Prem controller. - -To unplug a PDB CRD Resource, a sample .yaml file is available here: [config/samples/onpremdb/pdb_unplug.yaml](../../../config/samples/onpremdb/pdb_unplug.yaml) - -**NOTE:** It is assumed that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-manage-pdb-life-cycle-using-oracle-db-operator-on-prem-database-controller) steps. - -This example uses `unplug_pdb.yaml` to unplug a PDB from a target CDB using Oracle DB Operator On-Prem Controller with: - -- Pluggable Database CRD Resource Name as `pdb1` -- Pluggable Database (PDB) Name as `pdbnew` -- Target CDB CRD Resource Name as `cdb-dev` -- CDB Name as `goldcdb` -- Action to be taken on the PDB as `Unplug` -- XML metadata filename as `/tmp/pdbnewclone.xml` - -**NOTE:** For the details of the parameters to be used in the .yaml file, please refer [config/crd/bases/database.oracle.com_pdbs.yaml](../../../config/crd/bases/database.oracle.com_pdbs.yaml) - -**NOTE:** Before performing the unplug operation on the PDB CRD Resource, you will first need to perform the Modify Operation on that PDB CRD resource to Close the the PDB. After that you will be able to perform the Unplug operation. Please refer to the use case to modify the PDB state to Close. - -Use the file: [unplug_pdb.yaml](./unplug_pdb.yaml) for this use case as below: - -1. Deploy the .yaml file: -```sh -% kubectl apply -f unplug_pdb.yaml -``` - -2. Monitor the Oracle DB Operator Pod for the progress of the PDB Unplug operation: - -NOTE: Check the DB Operator Pod name in your environment. - -```sh -% kubectl logs -f pod/oracle-database-operator-controller-manager-76cb674c5c-f9wsd -n oracle-database-operator-system -``` - -## Sample Output - -[Here](./unplug_pdb.log) is the sample output for a PDB created using Oracle DB Operator On-Prem Controller using file [unplug_pdb.yaml](./unplug_pdb.yaml) diff --git a/docs/multitenant/provisioning/unplug_pdb.yaml b/docs/multitenant/provisioning/unplug_pdb.yaml deleted file mode 100644 index c9915b28..00000000 --- a/docs/multitenant/provisioning/unplug_pdb.yaml +++ /dev/null @@ -1,17 +0,0 @@ -# -# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. -# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. -# -apiVersion: database.oracle.com/v1alpha1 -kind: PDB -metadata: - name: pdb1 - namespace: oracle-database-operator-system - labels: - cdb: cdb-dev -spec: - cdbResName: "cdb-dev" - cdbName: "goldcdb" - pdbName: "pdbnew" - xmlFileName: "/tmp/pdbnewclone.xml" - action: "Unplug" diff --git a/docs/multitenant/provisioning/validation_error.md b/docs/multitenant/provisioning/validation_error.md deleted file mode 100644 index ec527cdb..00000000 --- a/docs/multitenant/provisioning/validation_error.md +++ /dev/null @@ -1,73 +0,0 @@ -#Validation and Errors - -## Kubernetes Events -You can check Kubernetes events for any errors or status updates as shown below: -```sh -$ kubectl get events -A -NAMESPACE LAST SEEN TYPE REASON OBJECT MESSAGE -oracle-database-operator-system 58m Warning Failed pod/cdb-dev-ords-qiigr Error: secret "cdb1-secret" not found -oracle-database-operator-system 56m Normal DeletedORDSPod cdb/cdb-dev Deleted ORDS Pod(s) for cdb-dev -oracle-database-operator-system 56m Normal DeletedORDSService cdb/cdb-dev Deleted ORDS Service for cdb-dev -... -oracle-database-operator-system 26m Warning OraError pdb/pdb1 ORA-65016: FILE_NAME_CONVERT must be specified... -oracle-database-operator-system 24m Warning OraError pdb/pdb2 ORA-65011: Pluggable database DEMOTEST does not exist. -... -oracle-database-operator-system 20m Normal Created pdb/pdb1 PDB 'demotest' created successfully -... -oracle-database-operator-system 17m Warning OraError pdb/pdb3 ORA-65012: Pluggable database DEMOTEST already exists... -``` - -In case of successfull operation, you can see messages like below: - -```sh -% kubectl get events -A -NAMESPACE LAST SEEN TYPE REASON OBJECT MESSAGE -kube-system 33s Warning BackOff pod/kube-apiserver Back-off restarting failed container -oracle-database-operator-system 59m Normal CreatedORDSService cdb/cdb-dev Created ORDS Service for cdb-dev -oracle-database-operator-system 51m Normal Created pdb/pdb1-clone PDB 'pdbnewclone' cloned successfully -oracle-database-operator-system 49m Normal Modified pdb/pdb1-clone PDB 'pdbnewclone' modified successfully -oracle-database-operator-system 47m Normal Deleted pdb/pdb1-clone PDB 'pdbnewclone' dropped successfully -oracle-database-operator-system 53m Normal Created pdb/pdb1 PDB 'pdbnew' created successfully -oracle-database-operator-system 44m Normal Modified pdb/pdb1 PDB 'pdbnew' modified successfully -oracle-database-operator-system 42m Normal Unplugged pdb/pdb1 PDB 'pdbnew' unplugged successfully -oracle-database-operator-system 39m Normal Created pdb/pdb1 PDB 'pdbnew' plugged successfully -``` - -## CDB Validation and Errors - -Validation is done at the time of CDB resource creation as shown below: -```sh -$ kubectl apply -f cdb1.yaml -The PDB "cdb-dev" is invalid: -* spec.dbServer: Required value: Please specify Database Server Name or IP Address -* spec.dbPort: Required value: Please specify DB Server Port -* spec.ordsImage: Required value: Please specify name of ORDS Image to be used -``` - -Apart from events, listing of CDBs will also show the possible reasons why a particular CDB CR could not be created as shown below: -```sh - $ kubectl get cdbs -A - - NAMESPACE NAME CDB NAME DB SERVER DB PORT SCAN NAME STATUS MESSAGE - oracle-database-operator-system cdb-dev devdb 172.17.0.4 1521 devdb Failed Secret not found:cdb1-secret -``` - -## PDB Validation and Errors - -Validation is done at the time of PDB resource creation as shown below: -```sh -$ kubectl apply -f pdb1.yaml -The PDB "pdb1" is invalid: -* spec.cdbResName: Required value: Please specify the name of the CDB Kubernetes resource to use for PDB operations -* spec.pdbName: Required value: Please specify name of the PDB to be created -* spec.adminPwd: Required value: Please specify PDB System Administrator Password -* spec.fileNameConversions: Required value: Please specify a value for fileNameConversions. Values can be a filename convert pattern or NONE -``` - -Similarly, for PDBs, listing of PDBs will also show the possible reasons why a particular PDB CR could not be created as shown below: -```sh -$ kubectl get pdbs -A -NAMESPACE NAME CONNECT STRING CDB NAME PDB NAME PDB SIZE STATUS MESSAGE -oracle-database-operator-system pdb1 democdb demotest1 Failed Secret not found:pdb12-secret -oracle-database-operator-system pdb2 democdb demotest2 Failed ORA-65016: FILE_NAME_CONVERT must be specified... -``` diff --git a/docs/multitenant/usecase01/README.md b/docs/multitenant/usecase01/README.md index e8109110..7352257e 100644 --- a/docs/multitenant/usecase01/README.md +++ b/docs/multitenant/usecase01/README.md @@ -3,20 +3,23 @@ # STEP BY STEP USE CASE -- [INTRODUCTION](#introduction) -- [OPERATION STEPS ](#operation-steps) -- [Download latest version from github ](#download-latest-version-from-orahub-a-namedownloada) -- [Upload webhook certificates](#upload-webhook-certificates-a-namewebhooka) -- [Create the dboperator](#create-the-dboperator-a-namedboperatora) -- [Create Secret for container registry](#create-secret-for-container-registry) -- [Build ords immage ](#build-ords-immage-a-nameordsimagea) -- [Database Configuration](#database-configuration) -- [Create CDB secret ](#create-cdb-secret) -- [Create Certificates](#create-certificates) -- [Apply cdb.yaml](#apply-cdbyaml) -- [Logs and throuble shutting](#cdb---logs-and-throuble-shutting) -- [Create PDB secret](#create-pdb-secret) -- [Other action ](#other-actions) +- [STEP BY STEP USE CASE](#step-by-step-use-case) + - [INTRODUCTION](#introduction) + - [OPERATIONAL STEPS](#operational-steps) + - [Download latest version from github ](#download-latest-version-from-github-) + - [Upload webhook certificates ](#upload-webhook-certificates-) + - [Create the dboperator ](#create-the-dboperator-) + - [Create secret for container registry](#create-secret-for-container-registry) + - [Build ords immage ](#build-ords-immage-) + - [Database Configuration](#database-configuration) + - [Create CDB secret](#create-cdb-secret) + - [Create Certificates](#create-certificates) + - [Apply cdb.yaml](#apply-cdbyaml) + - [CDB - Logs and throuble shutting](#cdb---logs-and-throuble-shutting) + - [Create PDB secret](#create-pdb-secret) + - [Apply pdb yaml file to create pdb](#apply-pdb-yaml-file-to-create-pdb) + - [Other actions](#other-actions) + - [Imperative approach on pdb deletion - will be avilable in 1.2.0 ](#imperative-approach-on-pdb-deletion) @@ -47,6 +50,7 @@ The following table reports the parameters required to configure and use oracle | pdbTlsKey | | [standalone.https.cert.key][key] | | pdbTlsCrt | | [standalone.https.cert][cr] | | pdbTlsCat | | certificate authority | +| assertivePdbDeletion | boolean | [turn on imperative approach on crd deleteion][imperative] | > A [makfile](./makefile) is available to sped up the command execution for the multitenant setup and test. See the comments in the header of file @@ -78,6 +82,7 @@ make operator-yaml IMG=operator:latest > **NOTE:** If you are using oracle-container-registry make sure to accept the license agreement otherwise the operator image pull fails. ---- + #### Upload webhook certificates ```bash @@ -101,6 +106,7 @@ oracle-database-operator-controller-manager-557ff6c659-xpswv 1/1 Running ``` ---- + #### Create secret for container registry + Make sure to login to your container registry and then create the secret for you container registry. @@ -119,6 +125,7 @@ container-registry-secret kubernetes.io/dockerconfigjson 1 19s webhook-server-cert kubernetes.io/tls ``` ---- + #### Build ords immage + Build the ords image, downloading ords software is no longer needed; just build the image and push it to your repository @@ -128,16 +135,17 @@ cd oracle-database-operator/ords docker build -t oracle/ords-dboper:latest . ``` -[example of execution](./BuildImage.log) +[Example of execution](./logfiles/BuildImage.log) + Login to your container registry and push the ords image. ```bash docker tag /ords-dboper:latest docker push /ords-dboper:latest ``` -[example of execution](./ImagePush.log) +[Example of execution](./logfiles/tagandpush.log) ---- + #### Database Configuration + Configure Database @@ -153,6 +161,7 @@ GRANT SYSDBA TO CONTAINER = ALL; GRANT CREATE SESSION TO CONTAINER = ALL; ``` ---- + #### Create CDB secret + Create secret for CDB connection @@ -208,6 +217,7 @@ webhook-server-cert kubernetes.io/tls 3 4m55s >**TIPS:** Use the following commands to analyze contents of an existing secret ```bash kubectl get secret -o yaml -n ``` ---- + #### Create Certificates + Create certificates: At this stage we need to create certificates on our local machine and upload into kubernetes cluster by creating new secrets. @@ -258,10 +268,11 @@ kubectl create secret generic db-ca --from-file= -n oracle-database-op ``` -[example of execution:](./openssl_execution.log) +[Example of execution:](./logfiles/openssl_execution.log) ---- + #### Apply cdb.yaml @@ -272,56 +283,54 @@ kubectl create secret generic db-ca --from-file= -n oracle-database-op + Create ords container ```bash -/usr/bin/kubectl apply -f cdb.yaml -n oracle-database-operator-system +/usr/bin/kubectl apply -f cdb_create.yaml -n oracle-database-operator-system ``` -Example: **cdb.yaml** +Example: **cdb_create.yaml** ```yaml apiVersion: database.oracle.com/v1alpha1 -kind: CDB -metadata: - name: +kind: CDB +metadata: + name: cdb-dev namespace: oracle-database-operator-system spec: - cdbName: "" - dbServer: "" or - dbPort: - ordsImage: "/ords-dboper:.latest" + cdbName: "DB12" + ordsImage: ".............your registry............./ords-dboper:latest" ordsImagePullPolicy: "Always" - serviceName: + dbTnsurl : "...Container tns alias....." replicas: 1 - sysAdminPwd: - secret: + sysAdminPwd: + secret: secretName: "cdb1-secret" key: "sysadmin_pwd" ordsPwd: - secret: + secret: secretName: "cdb1-secret" - key: "ords_pwd" - cdbAdminUser: - secret: + key: "ords_pwd" + cdbAdminUser: + secret: secretName: "cdb1-secret" key: "cdbadmin_user" - cdbAdminPwd: - secret: + cdbAdminPwd: + secret: secretName: "cdb1-secret" key: "cdbadmin_pwd" - webServerUser: - secret: + webServerUser: + secret: secretName: "cdb1-secret" key: "webserver_user" - webServerPwd: - secret: + webServerPwd: + secret: secretName: "cdb1-secret" - key: "webserver_pwd" + key: "webserver_pwd" cdbTlsKey: secret: secretName: "db-tls" - key: "" + key: "tls.key" cdbTlsCrt: secret: secretName: "db-tls" - key: ":" + key: "tls.crt" ``` > **Note** if you are working in dataguard environment with multiple sites (AC/DR) specifying the host name (dbServer/dbPort/serviceName) may not be the suitable solution for this kind of configuration, use **dbTnsurl** instead. Specify the whole tns string which includes the hosts/scan list. @@ -342,9 +351,11 @@ spec: dbtnsurl:((DESCRIPTION=(CONNECT_TIMEOUT=90)(RETRY_COUNT=30)(RETRY_DELAY=10)(TRANSPORT_CONNECT_TIMEOUT=70)(TRANS...... ``` -[example of cdb.yaml](./cdb.yaml) +[Example of cdb.yaml](./cdb_create.yaml) + ---- + #### CDB - Logs and throuble shutting + Check the status of ords container @@ -379,14 +390,14 @@ NAME CDB NAME DB SERVER DB PORT REPLICAS STATUS MESSAG ```bash /usr/bin/kubectl logs `/usr/bin/kubectl get pods -n oracle-database-operator-system|grep ords|cut -d ' ' -f 1` -n oracle-database-operator-system ``` -[example of execution](./cdb.log) +[Example of cdb creation log](./logfiles/cdb_creation.log) + Test REST API from the pod. By querying the metadata catalog you can verify the status of https setting ```bash /usr/bin/kubectl exec -it `/usr/bin/kubectl get pods -n oracle-database-operator-system|grep ords|cut -d ' ' -f 1` -n oracle-database-operator-system -i -t -- /usr/bin/curl -sSkv -k -X GET https://localhost:8888/ords/_/db-api/stable/metadata-catalog/ ``` -[example of execution](./testapi.log) +[Example of execution](./logfiles/testapi.log) + Verify the pod environment varaibles ```bash @@ -403,9 +414,10 @@ NAME CDB NAME DB SERVER DB PORT REPLICAS STATUS MESSAG ```bash /usr/bin/kubectl exec -it `/usr/bin/kubectl get pods -n oracle-database-operator-system|grep ords|cut -d ' ' -f 1` -n oracle-database-operator-system -i -t -- /usr/local/bin/ords --config /etc/ords/config config list ``` -[Example of executions](./ordsconfig.log) +[Example of executions](./logfiles/ordsconfig.log) ----- + #### Create PDB secret @@ -439,30 +451,32 @@ pdb1-secret Opaque 2 79m <--- webhook-server-cert kubernetes.io/tls 3 79m ``` --- + #### Apply pdb yaml file to create pdb ```bash /usr/bin/kubectl apply -f pdb.yaml -n oracle-database-operator-system ``` -Example: **pdb.yaml** +Example: **pdb_create.yaml** ```yaml apiVersion: database.oracle.com/v1alpha1 kind: PDB metadata: - name: + name: pdb1 namespace: oracle-database-operator-system labels: - cdb: + cdb: cdb-dev spec: - cdbResName: "" - cdbName: "" - pdbName: + cdbResName: "cdb-dev" + cdbNamespace: "oracle-database-operator-system" + cdbName: "DB12" + pdbName: "pdbdev" adminName: secret: secretName: "pdb1-secret" - key: "sysadmin_user" + key: "sysadmin_user" adminPwd: secret: secretName: "pdb1-secret" @@ -470,19 +484,29 @@ spec: pdbTlsKey: secret: secretName: "db-tls" - key: "" + key: "tls.key" pdbTlsCrt: secret: secretName: "db-tls" - key: "" + key: "tls.crt" pdbTlsCat: secret: secretName: "db-ca" - key: "" + key: "ca.crt" + webServerUser: + secret: + secretName: "pdb1-secret" + key: "webserver_user" + webServerPwd: + secret: + secretName: "pdb1-secret" + key: "webserver_pwd" fileNameConversions: "NONE" + tdeImport: false totalSize: "1G" tempSize: "100M" action: "Create" + assertivePdbDeletion: true ``` + Monitor the pdb creation status until message is success @@ -529,13 +553,17 @@ kubectl logs -f $(kubectl get pods -n oracle-database-operator-system|grep oracl ``` --- + #### Other actions -Configure and use other yaml files to perform pluggable database life cycle managment action **modify_pdb_open.yaml** **modify_pdb_close.yaml** +Configure and use other yaml files to perform pluggable database life cycle managment action **pdb_open.yaml** **pdb_close.yaml** -> **Note** sql command *"alter pluggable database open instances=all;"* acts only on closed databases, so you want get any oracle error in case of execution against an pluggable database already opened +> **Note** sql command *"alter pluggable database open instances=all;"* acts only on closed databases, so you don't get any oracle error in case of execution against an pluggable database already opened +#### Imperative approach on pdb deletion +If **assertivePdbDeletion** is true then the command execution **kubectl delete pdbs crd_pdb_name** automatically deletes the pluggable database on the container database. By default this option is disabled. You can use this option during **create**,**map**,**plug** and **clone** operation. If the option is disabled then **kubectl delete** only deletes the crd but not the pluggable on the container db. Database deletion uses the option **including datafiles**. +If you drop the CRD without dropping the pluggable database and you need to recreate the CRD then you can use the [pdb_map.yaml](./pdb_map.yaml) [1]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/installing-and-configuring-oracle-rest-data-services.html#GUID-E9625FAB-9BC8-468B-9FF9-443C88D76FA1:~:text=Table%202%2D2%20Command%20Options%20for%20Command%2DLine%20Interface%20Installation @@ -558,4 +586,8 @@ Configure and use other yaml files to perform pluggable database life cycle mana [http]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/installing-and-configuring-oracle-rest-data-services.html#GUID-BEECC057-A8F5-4EAB-B88E-9828C2809CD8:~:text=Example%3A%20delete%20%5B%2D%2Dglobal%5D-,user%20add,-Add%20a%20user -[dbtnsurl]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/installing-and-configuring-oracle-rest-data-services.html#GUID-A9AED253-4EEC-4E13-A0C4-B7CE82EC1C22 \ No newline at end of file +[dbtnsurl]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/installing-and-configuring-oracle-rest-data-services.html#GUID-A9AED253-4EEC-4E13-A0C4-B7CE82EC1C22 + +[imperative]:https://kubernetes.io/docs/concepts/overview/working-with-objects/object-management/ + + diff --git a/docs/multitenant/usecase01/ca.crt b/docs/multitenant/usecase01/ca.crt new file mode 100644 index 00000000..cc9aa8bb --- /dev/null +++ b/docs/multitenant/usecase01/ca.crt @@ -0,0 +1,25 @@ +-----BEGIN CERTIFICATE----- +MIIEJTCCAw2gAwIBAgIUNXPtpnNEFBCMcnxRP5kJsBDpafcwDQYJKoZIhvcNAQEL +BQAwgaExCzAJBgNVBAYTAlVTMRMwEQYDVQQIDApDYWxpZm9ybmlhMRUwEwYDVQQH +DAxTYW5GcmFuY2lzY28xEDAOBgNVBAoMB29yYWNsZSAxNjA0BgNVBAMMLWNkYi1k +ZXYtb3Jkcy5vcmFjbGUtZGF0YWJhc2Utb3BlcmF0b3Itc3lzdGVtIDEcMBoGA1UE +AwwTbG9jYWxob3N0ICBSb290IENBIDAeFw0yNDA4MTIxNTMyMzVaFw0yNTA4MTIx +NTMyMzVaMIGhMQswCQYDVQQGEwJVUzETMBEGA1UECAwKQ2FsaWZvcm5pYTEVMBMG +A1UEBwwMU2FuRnJhbmNpc2NvMRAwDgYDVQQKDAdvcmFjbGUgMTYwNAYDVQQDDC1j +ZGItZGV2LW9yZHMub3JhY2xlLWRhdGFiYXNlLW9wZXJhdG9yLXN5c3RlbSAxHDAa +BgNVBAMME2xvY2FsaG9zdCAgUm9vdCBDQSAwggEiMA0GCSqGSIb3DQEBAQUAA4IB +DwAwggEKAoIBAQCmnGVApwUBF1kpqcyr2nYeED0VKvefpoHLtxHSP+vP0lWhW7NU +NJlb1YuUagjJ4/rpGRQmPxcVU51n3aAW3a5qHazIpNxNa3fvgB1rMOPFxGmdel2d +8lIt+u19q19DknX/GNgH9Mog8RcyZyPeA7d2icT8TBo74ognr+8p68O3CjBHQ8EM +SnRQR7/bh1c10Uia317ilKvs+I7oErTq5JFLeIuPDdAJ6UncaeblTf1XJ/1FrpHG +fSS7xmR8x0/MblBQlku4eImYmN35g+eRgf8bLDDwC+GPzDnAqqMLjx6h2N+btDxr +tnn05qyqmN9G08uUlP4d4BXi9ISb/toYypklAgMBAAGjUzBRMB0GA1UdDgQWBBS+ +a4X2XTmdPivdQtqDWNpfOtHypDAfBgNVHSMEGDAWgBS+a4X2XTmdPivdQtqDWNpf +OtHypDAPBgNVHRMBAf8EBTADAQH/MA0GCSqGSIb3DQEBCwUAA4IBAQAZIrGBNdSw +pe+1agefHfaR8hjZQiXBxdwHM1gR2LWOaFzMS8Q/eRETHTO6+VwQ0/FNaXbAqgqk +G317gZMXS5ZmXuOi28fTpAQtuzokkEKpoK0puTnbXOKGA2QSbBlpSFPqb3aJXvVt +afXFQb5P/0mhr4kuVt7Ech82WM/o5ryFgObygDayDmLatTp+VaRmBZPksnSMhslq +3zPyS7bx2YhbPTLkDxq8Mfr/Msxme8LvSXUpFf4PpQ5zwp1RE32gekct6eRQLmqU +5LXY2aPtqpMF0fBpcwPWbqA9gOYCRKcvXXIr+u1x8hf6Er6grZegHkM9TQ8s0hJd +sxi5tK0lPMHJ +-----END CERTIFICATE----- diff --git a/docs/multitenant/usecase01/ca.key b/docs/multitenant/usecase01/ca.key new file mode 100644 index 00000000..1a0ef89d --- /dev/null +++ b/docs/multitenant/usecase01/ca.key @@ -0,0 +1,27 @@ +-----BEGIN RSA PRIVATE KEY----- +MIIEowIBAAKCAQEAppxlQKcFARdZKanMq9p2HhA9FSr3n6aBy7cR0j/rz9JVoVuz +VDSZW9WLlGoIyeP66RkUJj8XFVOdZ92gFt2uah2syKTcTWt374AdazDjxcRpnXpd +nfJSLfrtfatfQ5J1/xjYB/TKIPEXMmcj3gO3donE/EwaO+KIJ6/vKevDtwowR0PB +DEp0UEe/24dXNdFImt9e4pSr7PiO6BK06uSRS3iLjw3QCelJ3Gnm5U39Vyf9Ra6R +xn0ku8ZkfMdPzG5QUJZLuHiJmJjd+YPnkYH/Gyww8Avhj8w5wKqjC48eodjfm7Q8 +a7Z59OasqpjfRtPLlJT+HeAV4vSEm/7aGMqZJQIDAQABAoIBAGXRGYdjCgnarOBr +Jeq3vIsuvUVcVqs35AYMQFXOPltoXHAZTAPfiQC4BW6TRf+q1MDyVH/y+jZMPNsm +cxjGLDopHFgZd4/QZyDzmAbTf75yA2D7UI6fcV0sBUpRGgx/SqC0HADwtT1gWB6z +LRYWC13jX4AXOcjy7OXj/DIQJDCMivedt3dv0rDWJUcBCnVot5tr6zjycefxGKa8 +mG9LZQb3x71FxwpFUau3WLDSwOjtXCeMytaGXnGmIiofJmXnFi0KA4ApzKL7QV6I +cCBS1WBLLXeVM9vOfrtzKVLWGe0qADyLm35p5Fnl3j+vimkk8h/2DEvCZ75c987m +O3PEgdkCgYEA0Scg+KINTA78sdZL5v2+8fT4b+EfoCgUqfr10ReUPKrz3HfrVHcj +7Vf00RT52TkfmkL3mIdLyBUzQ9vzPgweo1o4yKCKNCpR9G3ydNW+KI5jSYnq2efz +Gpe3wTt+8YoyCgm9eUxNWjfO9fipS91sSotY0PovkBohj9aezfcWp1sCgYEAy+3n +MIvW/9PoYxCvQ9fDGLvx3B4/uy0ZYPh7j5edDuaRzwFd2YXUysXhJVuqTp0KT2tv +dRPFRE9Oq5N8e5ITIUiKLQ5PIRNBZm8CiAof+XS1fIuU+MTDaTfXwyGQo0xSg8MB +ITnJulmUlkcTWEtGyBi9sIjor5ve8kqvyrdAKX8CgYA9ZUUSd0978jJPad6iEf6J +PCXpgaYs91cJhre+BzPmkzA+mZ0lEEwlkdo1vfiRwWj7eYkA50Zhl4eS9e/zWM9t +mEBu9GFdasbf/55amZvWf+W5YpjkGmiMd9jjCjn7YVvLAozyHGngf91q6vGXaYou +X7VUsvxfSqxrcs7vGwc1XQKBgB0qaD80MMqj5v+MGlTsndWCw8OEe/7sI04QG7Pc +rjS8Wyws+NwsXNOnW1z5cDEQGrJjHiyzaCot4YV+cXZG3P+MnV52RnDnjRn2VHla +YVpPC8nFOMgfdAcvWmdo/IOuXbrEf/vdhPFm8G5Ruf2NvpDNoQuHeSfsdgVXEy89 +6CpHAoGBAMZInYD0XjcnZNqiQnQdcIJN3CqDIU76Z45OOpcUrYrvTos2xhGLrRI5 +qrk5Od/sovJfse+oUIIbgsABieqtyfxM03iu8fvbahIY6Un1iw2KN9t+mcPrSZJK +jTXKf7XxZ1+yN9kvohdLc65ySyXFSm++glDq8WGrmnOtLUlr0oMm +-----END RSA PRIVATE KEY----- diff --git a/docs/multitenant/usecase01/ca.srl b/docs/multitenant/usecase01/ca.srl new file mode 100644 index 00000000..7c9868bb --- /dev/null +++ b/docs/multitenant/usecase01/ca.srl @@ -0,0 +1 @@ +77D97AB4C4B6D5A9377B84B455D3E16348C6DE04 diff --git a/docs/multitenant/usecase01/extfile.txt b/docs/multitenant/usecase01/extfile.txt new file mode 100644 index 00000000..c51d22a3 --- /dev/null +++ b/docs/multitenant/usecase01/extfile.txt @@ -0,0 +1 @@ +subjectAltName=DNS:cdb-dev-ords.oracle-database-operator-system,DNS:www.example.com diff --git a/docs/multitenant/usecase01/logfiles/BuildImage.log b/docs/multitenant/usecase01/logfiles/BuildImage.log index 4ee2fa05..f35c66d8 100644 --- a/docs/multitenant/usecase01/logfiles/BuildImage.log +++ b/docs/multitenant/usecase01/logfiles/BuildImage.log @@ -1,487 +1,896 @@ -/usr/bin/docker build -t oracle/ords-dboper:latest . -Sending build context to Docker daemon 92.38MB -Step 1/10 : FROM container-registry.oracle.com/java/jdk:latest -Trying to pull repository container-registry.oracle.com/java/jdk ... -latest: Pulling from container-registry.oracle.com/java/jdk -7cb069903b8a: Pull complete -a98ca67f4239: Pull complete -1b4060d1d804: Pull complete -Digest: sha256:8e7161bbd6a3a3beb77ee6f2d80c17ae4c80d88e0f5af667a19a0271c33f1b5e -Status: Downloaded newer image for container-registry.oracle.com/java/jdk:latest - ---> ad9ff1bbe92a -Step 2/10 : ENV ORDS_HOME=/opt/oracle/ords/ RUN_FILE="runOrdsSSL.sh" - ---> Running in e6f76deab66e -Removing intermediate container e6f76deab66e - ---> 0b26c489e4fd -Step 3/10 : COPY $RUN_FILE $ORDS_HOME - ---> ee472155adab -Step 4/10 : RUN yum -y install yum-utils bind-utils tree hostname openssl net-tools zip unzip tar wget vim-minimal which sudo expect procps && yum-config-manager --add-repo=http://yum.oracle.com/repo/OracleLinux/OL8/oracle/software/x86_64 && yum -y install java-11-openjdk-devel && yum -y install ords && yum -y install iproute && yum clean all - ---> Running in d38a69d2cc70 -Oracle Linux 8 BaseOS Latest (x86_64) 105 MB/s | 50 MB 00:00 -Oracle Linux 8 Application Stream (x86_64) 90 MB/s | 38 MB 00:00 -Last metadata expiration check: 0:00:07 ago on Mon 10 Oct 2022 04:06:15 PM UTC. -Package yum-utils-4.0.21-11.0.1.el8.noarch is already installed. -Package tar-2:1.30-5.el8.x86_64 is already installed. +/usr/bin/docker build -t oracle/ords-dboper:latest ../../../ords +Sending build context to Docker daemon 13.82kB +Step 1/12 : FROM container-registry.oracle.com/java/jdk:latest + ---> b8457e2f0b73 +Step 2/12 : ENV ORDS_HOME=/opt/oracle/ords/ RUN_FILE="runOrdsSSL.sh" ORDSVERSION=23.4.0-8 + ---> Using cache + ---> 3317a16cd6f8 +Step 3/12 : COPY $RUN_FILE $ORDS_HOME + ---> 7995edec33cc +Step 4/12 : RUN yum -y install yum-utils bind-utils tree hostname openssl net-tools zip unzip tar wget vim-minimal which sudo expect procps curl lsof && yum-config-manager --add-repo=http://yum.oracle.com/repo/OracleLinux/OL8/oracle/software/x86_64 && yum -y install java-11-openjdk-devel && yum -y install iproute && yum clean all + ---> Running in fe168b01f3ad +Oracle Linux 8 BaseOS Latest (x86_64) 91 MB/s | 79 MB 00:00 +Oracle Linux 8 Application Stream (x86_64) 69 MB/s | 62 MB 00:00 +Last metadata expiration check: 0:00:12 ago on Tue 20 Aug 2024 08:54:50 AM UTC. +Package yum-utils-4.0.21-23.0.1.el8.noarch is already installed. +Package tar-2:1.30-9.el8.x86_64 is already installed. Package vim-minimal-2:8.0.1763-19.0.1.el8_6.4.x86_64 is already installed. -Package procps-ng-3.3.15-6.0.1.el8.x86_64 is already installed. +Package procps-ng-3.3.15-14.0.1.el8.x86_64 is already installed. +Package curl-7.61.1-33.el8_9.5.x86_64 is already installed. Dependencies resolved. ================================================================================ - Package Arch Version Repository Size + Package Arch Version Repository Size ================================================================================ Installing: - bind-utils x86_64 32:9.11.36-3.el8_6.1 ol8_appstream 452 k - expect x86_64 5.45.4-5.el8 ol8_baseos_latest 266 k - hostname x86_64 3.20-6.el8 ol8_baseos_latest 32 k - net-tools x86_64 2.0-0.52.20160912git.el8 ol8_baseos_latest 322 k - openssl x86_64 1:1.1.1k-7.el8_6 ol8_baseos_latest 709 k - sudo x86_64 1.8.29-8.el8 ol8_baseos_latest 925 k - tree x86_64 1.7.0-15.el8 ol8_baseos_latest 59 k - unzip x86_64 6.0-46.0.1.el8 ol8_baseos_latest 196 k - wget x86_64 1.19.5-10.0.1.el8 ol8_appstream 734 k - which x86_64 2.21-17.el8 ol8_baseos_latest 49 k - zip x86_64 3.0-23.el8 ol8_baseos_latest 270 k + bind-utils x86_64 32:9.11.36-16.el8_10.2 ol8_appstream 453 k + expect x86_64 5.45.4-5.el8 ol8_baseos_latest 266 k + hostname x86_64 3.20-6.el8 ol8_baseos_latest 32 k + lsof x86_64 4.93.2-1.el8 ol8_baseos_latest 253 k + net-tools x86_64 2.0-0.52.20160912git.el8 ol8_baseos_latest 322 k + openssl x86_64 1:1.1.1k-12.el8_9 ol8_baseos_latest 710 k + sudo x86_64 1.9.5p2-1.el8_9 ol8_baseos_latest 1.0 M + tree x86_64 1.7.0-15.el8 ol8_baseos_latest 59 k + unzip x86_64 6.0-46.0.1.el8 ol8_baseos_latest 196 k + wget x86_64 1.19.5-12.0.1.el8_10 ol8_appstream 733 k + which x86_64 2.21-20.el8 ol8_baseos_latest 50 k + zip x86_64 3.0-23.el8 ol8_baseos_latest 270 k +Upgrading: + curl x86_64 7.61.1-34.el8 ol8_baseos_latest 352 k + dnf-plugins-core noarch 4.0.21-25.0.1.el8 ol8_baseos_latest 76 k + libcurl x86_64 7.61.1-34.el8 ol8_baseos_latest 303 k + python3-dnf-plugins-core + noarch 4.0.21-25.0.1.el8 ol8_baseos_latest 263 k + yum-utils noarch 4.0.21-25.0.1.el8 ol8_baseos_latest 75 k Installing dependencies: - bind-libs x86_64 32:9.11.36-3.el8_6.1 ol8_appstream 175 k - bind-libs-lite x86_64 32:9.11.36-3.el8_6.1 ol8_appstream 1.2 M - bind-license noarch 32:9.11.36-3.el8_6.1 ol8_appstream 103 k - fstrm x86_64 0.6.1-2.el8 ol8_appstream 29 k - libmaxminddb x86_64 1.2.0-10.el8 ol8_appstream 33 k - libmetalink x86_64 0.1.3-7.el8 ol8_baseos_latest 32 k - protobuf-c x86_64 1.3.0-6.el8 ol8_appstream 37 k - python3-bind noarch 32:9.11.36-3.el8_6.1 ol8_appstream 150 k - python3-ply noarch 3.9-9.el8 ol8_baseos_latest 111 k - tcl x86_64 1:8.6.8-2.el8 ol8_baseos_latest 1.1 M + bind-libs x86_64 32:9.11.36-16.el8_10.2 ol8_appstream 176 k + bind-libs-lite x86_64 32:9.11.36-16.el8_10.2 ol8_appstream 1.2 M + bind-license noarch 32:9.11.36-16.el8_10.2 ol8_appstream 104 k + fstrm x86_64 0.6.1-3.el8 ol8_appstream 29 k + libmaxminddb x86_64 1.2.0-10.el8_9.1 ol8_appstream 32 k + libmetalink x86_64 0.1.3-7.el8 ol8_baseos_latest 32 k + protobuf-c x86_64 1.3.0-8.el8 ol8_appstream 37 k + python3-bind noarch 32:9.11.36-16.el8_10.2 ol8_appstream 151 k + python3-ply noarch 3.9-9.el8 ol8_baseos_latest 111 k + tcl x86_64 1:8.6.8-2.el8 ol8_baseos_latest 1.1 M +Installing weak dependencies: + geolite2-city noarch 20180605-1.el8 ol8_appstream 19 M + geolite2-country noarch 20180605-1.el8 ol8_appstream 1.0 M Transaction Summary ================================================================================ -Install 21 Packages +Install 24 Packages +Upgrade 5 Packages -Total download size: 6.9 M -Installed size: 20 M +Total download size: 28 M Downloading Packages: -(1/21): hostname-3.20-6.el8.x86_64.rpm 555 kB/s | 32 kB 00:00 -(2/21): libmetalink-0.1.3-7.el8.x86_64.rpm 492 kB/s | 32 kB 00:00 -(3/21): expect-5.45.4-5.el8.x86_64.rpm 3.2 MB/s | 266 kB 00:00 -(4/21): python3-ply-3.9-9.el8.noarch.rpm 5.5 MB/s | 111 kB 00:00 -(5/21): net-tools-2.0-0.52.20160912git.el8.x86_ 6.7 MB/s | 322 kB 00:00 -(6/21): openssl-1.1.1k-7.el8_6.x86_64.rpm 12 MB/s | 709 kB 00:00 -(7/21): tree-1.7.0-15.el8.x86_64.rpm 4.1 MB/s | 59 kB 00:00 -(8/21): sudo-1.8.29-8.el8.x86_64.rpm 19 MB/s | 925 kB 00:00 -(9/21): which-2.21-17.el8.x86_64.rpm 2.5 MB/s | 49 kB 00:00 -(10/21): unzip-6.0-46.0.1.el8.x86_64.rpm 5.9 MB/s | 196 kB 00:00 -(11/21): tcl-8.6.8-2.el8.x86_64.rpm 15 MB/s | 1.1 MB 00:00 -(12/21): zip-3.0-23.el8.x86_64.rpm 15 MB/s | 270 kB 00:00 -(13/21): bind-libs-9.11.36-3.el8_6.1.x86_64.rpm 7.9 MB/s | 175 kB 00:00 -(14/21): bind-license-9.11.36-3.el8_6.1.noarch. 4.9 MB/s | 103 kB 00:00 -(15/21): bind-utils-9.11.36-3.el8_6.1.x86_64.rp 21 MB/s | 452 kB 00:00 -(16/21): bind-libs-lite-9.11.36-3.el8_6.1.x86_6 28 MB/s | 1.2 MB 00:00 -(17/21): libmaxminddb-1.2.0-10.el8.x86_64.rpm 1.8 MB/s | 33 kB 00:00 -(18/21): fstrm-0.6.1-2.el8.x86_64.rpm 1.0 MB/s | 29 kB 00:00 -(19/21): protobuf-c-1.3.0-6.el8.x86_64.rpm 1.4 MB/s | 37 kB 00:00 -(20/21): python3-bind-9.11.36-3.el8_6.1.noarch. 9.2 MB/s | 150 kB 00:00 -(21/21): wget-1.19.5-10.0.1.el8.x86_64.rpm 7.5 MB/s | 734 kB 00:00 +(1/29): hostname-3.20-6.el8.x86_64.rpm 268 kB/s | 32 kB 00:00 +(2/29): libmetalink-0.1.3-7.el8.x86_64.rpm 257 kB/s | 32 kB 00:00 +(3/29): expect-5.45.4-5.el8.x86_64.rpm 1.4 MB/s | 266 kB 00:00 +(4/29): lsof-4.93.2-1.el8.x86_64.rpm 3.2 MB/s | 253 kB 00:00 +(5/29): net-tools-2.0-0.52.20160912git.el8.x86_ 3.6 MB/s | 322 kB 00:00 +(6/29): python3-ply-3.9-9.el8.noarch.rpm 2.7 MB/s | 111 kB 00:00 +(7/29): openssl-1.1.1k-12.el8_9.x86_64.rpm 10 MB/s | 710 kB 00:00 +(8/29): tree-1.7.0-15.el8.x86_64.rpm 2.2 MB/s | 59 kB 00:00 +(9/29): sudo-1.9.5p2-1.el8_9.x86_64.rpm 14 MB/s | 1.0 MB 00:00 +(10/29): unzip-6.0-46.0.1.el8.x86_64.rpm 6.8 MB/s | 196 kB 00:00 +(11/29): which-2.21-20.el8.x86_64.rpm 2.0 MB/s | 50 kB 00:00 +(12/29): tcl-8.6.8-2.el8.x86_64.rpm 13 MB/s | 1.1 MB 00:00 +(13/29): bind-libs-9.11.36-16.el8_10.2.x86_64.r 6.7 MB/s | 176 kB 00:00 +(14/29): zip-3.0-23.el8.x86_64.rpm 8.4 MB/s | 270 kB 00:00 +(15/29): bind-libs-lite-9.11.36-16.el8_10.2.x86 29 MB/s | 1.2 MB 00:00 +(16/29): bind-license-9.11.36-16.el8_10.2.noarc 3.3 MB/s | 104 kB 00:00 +(17/29): bind-utils-9.11.36-16.el8_10.2.x86_64. 13 MB/s | 453 kB 00:00 +(18/29): fstrm-0.6.1-3.el8.x86_64.rpm 1.2 MB/s | 29 kB 00:00 +(19/29): libmaxminddb-1.2.0-10.el8_9.1.x86_64.r 1.3 MB/s | 32 kB 00:00 +(20/29): geolite2-country-20180605-1.el8.noarch 17 MB/s | 1.0 MB 00:00 +(21/29): protobuf-c-1.3.0-8.el8.x86_64.rpm 1.5 MB/s | 37 kB 00:00 +(22/29): python3-bind-9.11.36-16.el8_10.2.noarc 5.8 MB/s | 151 kB 00:00 +(23/29): wget-1.19.5-12.0.1.el8_10.x86_64.rpm 17 MB/s | 733 kB 00:00 +(24/29): curl-7.61.1-34.el8.x86_64.rpm 12 MB/s | 352 kB 00:00 +(25/29): dnf-plugins-core-4.0.21-25.0.1.el8.noa 2.4 MB/s | 76 kB 00:00 +(26/29): libcurl-7.61.1-34.el8.x86_64.rpm 8.6 MB/s | 303 kB 00:00 +(27/29): python3-dnf-plugins-core-4.0.21-25.0.1 9.8 MB/s | 263 kB 00:00 +(28/29): yum-utils-4.0.21-25.0.1.el8.noarch.rpm 3.0 MB/s | 75 kB 00:00 +(29/29): geolite2-city-20180605-1.el8.noarch.rp 66 MB/s | 19 MB 00:00 -------------------------------------------------------------------------------- -Total 20 MB/s | 6.9 MB 00:00 +Total 43 MB/s | 28 MB 00:00 Running transaction check Transaction check succeeded. Running transaction test Transaction test succeeded. Running transaction Preparing : 1/1 - Installing : protobuf-c-1.3.0-6.el8.x86_64 1/21 - Installing : libmaxminddb-1.2.0-10.el8.x86_64 2/21 - Running scriptlet: libmaxminddb-1.2.0-10.el8.x86_64 2/21 - Installing : fstrm-0.6.1-2.el8.x86_64 3/21 - Installing : bind-license-32:9.11.36-3.el8_6.1.noarch 4/21 - Installing : bind-libs-lite-32:9.11.36-3.el8_6.1.x86_64 5/21 - Installing : bind-libs-32:9.11.36-3.el8_6.1.x86_64 6/21 - Installing : unzip-6.0-46.0.1.el8.x86_64 7/21 - Installing : tcl-1:8.6.8-2.el8.x86_64 8/21 - Running scriptlet: tcl-1:8.6.8-2.el8.x86_64 8/21 - Installing : python3-ply-3.9-9.el8.noarch 9/21 - Installing : python3-bind-32:9.11.36-3.el8_6.1.noarch 10/21 - Installing : libmetalink-0.1.3-7.el8.x86_64 11/21 - Installing : wget-1.19.5-10.0.1.el8.x86_64 12/21 - Running scriptlet: wget-1.19.5-10.0.1.el8.x86_64 12/21 - Installing : bind-utils-32:9.11.36-3.el8_6.1.x86_64 13/21 - Installing : expect-5.45.4-5.el8.x86_64 14/21 - Installing : zip-3.0-23.el8.x86_64 15/21 - Installing : which-2.21-17.el8.x86_64 16/21 - Installing : tree-1.7.0-15.el8.x86_64 17/21 - Installing : sudo-1.8.29-8.el8.x86_64 18/21 - Running scriptlet: sudo-1.8.29-8.el8.x86_64 18/21 - Installing : openssl-1:1.1.1k-7.el8_6.x86_64 19/21 - Installing : net-tools-2.0-0.52.20160912git.el8.x86_64 20/21 - Running scriptlet: net-tools-2.0-0.52.20160912git.el8.x86_64 20/21 - Installing : hostname-3.20-6.el8.x86_64 21/21 - Running scriptlet: hostname-3.20-6.el8.x86_64 21/21 - Verifying : expect-5.45.4-5.el8.x86_64 1/21 - Verifying : hostname-3.20-6.el8.x86_64 2/21 - Verifying : libmetalink-0.1.3-7.el8.x86_64 3/21 - Verifying : net-tools-2.0-0.52.20160912git.el8.x86_64 4/21 - Verifying : openssl-1:1.1.1k-7.el8_6.x86_64 5/21 - Verifying : python3-ply-3.9-9.el8.noarch 6/21 - Verifying : sudo-1.8.29-8.el8.x86_64 7/21 - Verifying : tcl-1:8.6.8-2.el8.x86_64 8/21 - Verifying : tree-1.7.0-15.el8.x86_64 9/21 - Verifying : unzip-6.0-46.0.1.el8.x86_64 10/21 - Verifying : which-2.21-17.el8.x86_64 11/21 - Verifying : zip-3.0-23.el8.x86_64 12/21 - Verifying : bind-libs-32:9.11.36-3.el8_6.1.x86_64 13/21 - Verifying : bind-libs-lite-32:9.11.36-3.el8_6.1.x86_64 14/21 - Verifying : bind-license-32:9.11.36-3.el8_6.1.noarch 15/21 - Verifying : bind-utils-32:9.11.36-3.el8_6.1.x86_64 16/21 - Verifying : fstrm-0.6.1-2.el8.x86_64 17/21 - Verifying : libmaxminddb-1.2.0-10.el8.x86_64 18/21 - Verifying : protobuf-c-1.3.0-6.el8.x86_64 19/21 - Verifying : python3-bind-32:9.11.36-3.el8_6.1.noarch 20/21 - Verifying : wget-1.19.5-10.0.1.el8.x86_64 21/21 + Running scriptlet: protobuf-c-1.3.0-8.el8.x86_64 1/1 + Installing : protobuf-c-1.3.0-8.el8.x86_64 1/34 + Installing : fstrm-0.6.1-3.el8.x86_64 2/34 + Installing : bind-license-32:9.11.36-16.el8_10.2.noarch 3/34 + Upgrading : python3-dnf-plugins-core-4.0.21-25.0.1.el8.noarch 4/34 + Upgrading : dnf-plugins-core-4.0.21-25.0.1.el8.noarch 5/34 + Upgrading : libcurl-7.61.1-34.el8.x86_64 6/34 + Installing : geolite2-country-20180605-1.el8.noarch 7/34 + Installing : geolite2-city-20180605-1.el8.noarch 8/34 + Installing : libmaxminddb-1.2.0-10.el8_9.1.x86_64 9/34 + Running scriptlet: libmaxminddb-1.2.0-10.el8_9.1.x86_64 9/34 + Installing : bind-libs-lite-32:9.11.36-16.el8_10.2.x86_64 10/34 + Installing : bind-libs-32:9.11.36-16.el8_10.2.x86_64 11/34 + Installing : unzip-6.0-46.0.1.el8.x86_64 12/34 + Installing : tcl-1:8.6.8-2.el8.x86_64 13/34 + Running scriptlet: tcl-1:8.6.8-2.el8.x86_64 13/34 + Installing : python3-ply-3.9-9.el8.noarch 14/34 + Installing : python3-bind-32:9.11.36-16.el8_10.2.noarch 15/34 + Installing : libmetalink-0.1.3-7.el8.x86_64 16/34 + Installing : wget-1.19.5-12.0.1.el8_10.x86_64 17/34 + Running scriptlet: wget-1.19.5-12.0.1.el8_10.x86_64 17/34 + Installing : bind-utils-32:9.11.36-16.el8_10.2.x86_64 18/34 + Installing : expect-5.45.4-5.el8.x86_64 19/34 + Installing : zip-3.0-23.el8.x86_64 20/34 + Upgrading : curl-7.61.1-34.el8.x86_64 21/34 + Upgrading : yum-utils-4.0.21-25.0.1.el8.noarch 22/34 + Installing : which-2.21-20.el8.x86_64 23/34 + Installing : tree-1.7.0-15.el8.x86_64 24/34 + Installing : sudo-1.9.5p2-1.el8_9.x86_64 25/34 + Running scriptlet: sudo-1.9.5p2-1.el8_9.x86_64 25/34 + Installing : openssl-1:1.1.1k-12.el8_9.x86_64 26/34 + Installing : net-tools-2.0-0.52.20160912git.el8.x86_64 27/34 + Running scriptlet: net-tools-2.0-0.52.20160912git.el8.x86_64 27/34 + Installing : lsof-4.93.2-1.el8.x86_64 28/34 + Installing : hostname-3.20-6.el8.x86_64 29/34 + Running scriptlet: hostname-3.20-6.el8.x86_64 29/34 + Cleanup : curl-7.61.1-33.el8_9.5.x86_64 30/34 + Cleanup : yum-utils-4.0.21-23.0.1.el8.noarch 31/34 + Cleanup : dnf-plugins-core-4.0.21-23.0.1.el8.noarch 32/34 + Cleanup : python3-dnf-plugins-core-4.0.21-23.0.1.el8.noarch 33/34 + Cleanup : libcurl-7.61.1-33.el8_9.5.x86_64 34/34 + Running scriptlet: libcurl-7.61.1-33.el8_9.5.x86_64 34/34 + Verifying : expect-5.45.4-5.el8.x86_64 1/34 + Verifying : hostname-3.20-6.el8.x86_64 2/34 + Verifying : libmetalink-0.1.3-7.el8.x86_64 3/34 + Verifying : lsof-4.93.2-1.el8.x86_64 4/34 + Verifying : net-tools-2.0-0.52.20160912git.el8.x86_64 5/34 + Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 6/34 + Verifying : python3-ply-3.9-9.el8.noarch 7/34 + Verifying : sudo-1.9.5p2-1.el8_9.x86_64 8/34 + Verifying : tcl-1:8.6.8-2.el8.x86_64 9/34 + Verifying : tree-1.7.0-15.el8.x86_64 10/34 + Verifying : unzip-6.0-46.0.1.el8.x86_64 11/34 + Verifying : which-2.21-20.el8.x86_64 12/34 + Verifying : zip-3.0-23.el8.x86_64 13/34 + Verifying : bind-libs-32:9.11.36-16.el8_10.2.x86_64 14/34 + Verifying : bind-libs-lite-32:9.11.36-16.el8_10.2.x86_64 15/34 + Verifying : bind-license-32:9.11.36-16.el8_10.2.noarch 16/34 + Verifying : bind-utils-32:9.11.36-16.el8_10.2.x86_64 17/34 + Verifying : fstrm-0.6.1-3.el8.x86_64 18/34 + Verifying : geolite2-city-20180605-1.el8.noarch 19/34 + Verifying : geolite2-country-20180605-1.el8.noarch 20/34 + Verifying : libmaxminddb-1.2.0-10.el8_9.1.x86_64 21/34 + Verifying : protobuf-c-1.3.0-8.el8.x86_64 22/34 + Verifying : python3-bind-32:9.11.36-16.el8_10.2.noarch 23/34 + Verifying : wget-1.19.5-12.0.1.el8_10.x86_64 24/34 + Verifying : curl-7.61.1-34.el8.x86_64 25/34 + Verifying : curl-7.61.1-33.el8_9.5.x86_64 26/34 + Verifying : dnf-plugins-core-4.0.21-25.0.1.el8.noarch 27/34 + Verifying : dnf-plugins-core-4.0.21-23.0.1.el8.noarch 28/34 + Verifying : libcurl-7.61.1-34.el8.x86_64 29/34 + Verifying : libcurl-7.61.1-33.el8_9.5.x86_64 30/34 + Verifying : python3-dnf-plugins-core-4.0.21-25.0.1.el8.noarch 31/34 + Verifying : python3-dnf-plugins-core-4.0.21-23.0.1.el8.noarch 32/34 + Verifying : yum-utils-4.0.21-25.0.1.el8.noarch 33/34 + Verifying : yum-utils-4.0.21-23.0.1.el8.noarch 34/34 +Upgraded: + curl-7.61.1-34.el8.x86_64 + dnf-plugins-core-4.0.21-25.0.1.el8.noarch + libcurl-7.61.1-34.el8.x86_64 + python3-dnf-plugins-core-4.0.21-25.0.1.el8.noarch + yum-utils-4.0.21-25.0.1.el8.noarch Installed: - bind-libs-32:9.11.36-3.el8_6.1.x86_64 - bind-libs-lite-32:9.11.36-3.el8_6.1.x86_64 - bind-license-32:9.11.36-3.el8_6.1.noarch - bind-utils-32:9.11.36-3.el8_6.1.x86_64 + bind-libs-32:9.11.36-16.el8_10.2.x86_64 + bind-libs-lite-32:9.11.36-16.el8_10.2.x86_64 + bind-license-32:9.11.36-16.el8_10.2.noarch + bind-utils-32:9.11.36-16.el8_10.2.x86_64 expect-5.45.4-5.el8.x86_64 - fstrm-0.6.1-2.el8.x86_64 + fstrm-0.6.1-3.el8.x86_64 + geolite2-city-20180605-1.el8.noarch + geolite2-country-20180605-1.el8.noarch hostname-3.20-6.el8.x86_64 - libmaxminddb-1.2.0-10.el8.x86_64 + libmaxminddb-1.2.0-10.el8_9.1.x86_64 libmetalink-0.1.3-7.el8.x86_64 + lsof-4.93.2-1.el8.x86_64 net-tools-2.0-0.52.20160912git.el8.x86_64 - openssl-1:1.1.1k-7.el8_6.x86_64 - protobuf-c-1.3.0-6.el8.x86_64 - python3-bind-32:9.11.36-3.el8_6.1.noarch + openssl-1:1.1.1k-12.el8_9.x86_64 + protobuf-c-1.3.0-8.el8.x86_64 + python3-bind-32:9.11.36-16.el8_10.2.noarch python3-ply-3.9-9.el8.noarch - sudo-1.8.29-8.el8.x86_64 + sudo-1.9.5p2-1.el8_9.x86_64 tcl-1:8.6.8-2.el8.x86_64 tree-1.7.0-15.el8.x86_64 unzip-6.0-46.0.1.el8.x86_64 - wget-1.19.5-10.0.1.el8.x86_64 - which-2.21-17.el8.x86_64 + wget-1.19.5-12.0.1.el8_10.x86_64 + which-2.21-20.el8.x86_64 zip-3.0-23.el8.x86_64 Complete! Adding repo from: http://yum.oracle.com/repo/OracleLinux/OL8/oracle/software/x86_64 -created by dnf config-manager from http://yum.o 194 kB/s | 49 kB 00:00 +created by dnf config-manager from http://yum.o 496 kB/s | 139 kB 00:00 +Last metadata expiration check: 0:00:01 ago on Tue 20 Aug 2024 08:55:14 AM UTC. Dependencies resolved. -============================================================================================= - Package Arch Version Repository Size -============================================================================================= +============================================================================================== + Package Arch Version Repository Size +============================================================================================== Installing: - java-11-openjdk-devel x86_64 1:11.0.16.1.1-1.el8_6 ol8_appstream 3.4 M + java-11-openjdk-devel x86_64 1:11.0.24.0.8-3.0.1.el8 ol8_appstream 3.4 M Installing dependencies: - alsa-lib x86_64 1.2.6.1-3.el8 ol8_appstream 491 k - avahi-libs x86_64 0.7-20.el8 ol8_baseos_latest 62 k - copy-jdk-configs noarch 4.0-2.el8 ol8_appstream 30 k - crypto-policies-scripts noarch 20211116-1.gitae470d6.el8 ol8_baseos_latest 83 k - cups-libs x86_64 1:2.2.6-45.el8_6.2 ol8_baseos_latest 434 k - giflib x86_64 5.1.4-3.el8 ol8_appstream 51 k - graphite2 x86_64 1.3.10-10.el8 ol8_appstream 122 k - harfbuzz x86_64 1.7.5-3.el8 ol8_appstream 295 k - java-11-openjdk x86_64 1:11.0.16.1.1-1.el8_6 ol8_appstream 272 k - java-11-openjdk-headless x86_64 1:11.0.16.1.1-1.el8_6 ol8_appstream 40 M - javapackages-filesystem noarch 5.3.0-1.module+el8+5136+7ff78f74 ol8_appstream 30 k - lcms2 x86_64 2.9-2.el8 ol8_appstream 164 k - libX11 x86_64 1.6.8-5.el8 ol8_appstream 611 k - libX11-common noarch 1.6.8-5.el8 ol8_appstream 158 k - libXau x86_64 1.0.9-3.el8 ol8_appstream 37 k - libXcomposite x86_64 0.4.4-14.el8 ol8_appstream 28 k - libXext x86_64 1.3.4-1.el8 ol8_appstream 45 k - libXi x86_64 1.7.10-1.el8 ol8_appstream 49 k - libXrender x86_64 0.9.10-7.el8 ol8_appstream 33 k - libXtst x86_64 1.2.3-7.el8 ol8_appstream 22 k - libfontenc x86_64 1.1.3-8.el8 ol8_appstream 37 k - libjpeg-turbo x86_64 1.5.3-12.el8 ol8_appstream 157 k - libpkgconf x86_64 1.4.2-1.el8 ol8_baseos_latest 35 k - libxcb x86_64 1.13.1-1.el8 ol8_appstream 231 k - lksctp-tools x86_64 1.0.18-3.el8 ol8_baseos_latest 100 k - lua x86_64 5.3.4-12.el8 ol8_appstream 192 k - nspr x86_64 4.34.0-3.el8_6 ol8_appstream 143 k - nss x86_64 3.79.0-10.el8_6 ol8_appstream 747 k - nss-softokn x86_64 3.79.0-10.el8_6 ol8_appstream 1.2 M - nss-softokn-freebl x86_64 3.79.0-10.el8_6 ol8_appstream 398 k - nss-sysinit x86_64 3.79.0-10.el8_6 ol8_appstream 74 k - nss-util x86_64 3.79.0-10.el8_6 ol8_appstream 138 k - pkgconf x86_64 1.4.2-1.el8 ol8_baseos_latest 38 k - pkgconf-m4 noarch 1.4.2-1.el8 ol8_baseos_latest 17 k - pkgconf-pkg-config x86_64 1.4.2-1.el8 ol8_baseos_latest 15 k - ttmkfdir x86_64 3.0.9-54.el8 ol8_appstream 62 k - tzdata-java noarch 2022d-1.el8 ol8_appstream 186 k - xorg-x11-font-utils x86_64 1:7.5-41.el8 ol8_appstream 104 k - xorg-x11-fonts-Type1 noarch 7.5-19.el8 ol8_appstream 522 k + adwaita-cursor-theme noarch 3.28.0-3.el8 ol8_appstream 647 k + adwaita-icon-theme noarch 3.28.0-3.el8 ol8_appstream 11 M + alsa-lib x86_64 1.2.10-2.el8 ol8_appstream 500 k + at-spi2-atk x86_64 2.26.2-1.el8 ol8_appstream 89 k + at-spi2-core x86_64 2.28.0-1.el8 ol8_appstream 169 k + atk x86_64 2.28.1-1.el8 ol8_appstream 272 k + avahi-libs x86_64 0.7-27.el8 ol8_baseos_latest 61 k + cairo x86_64 1.15.12-6.el8 ol8_appstream 719 k + cairo-gobject x86_64 1.15.12-6.el8 ol8_appstream 33 k + colord-libs x86_64 1.4.2-1.el8 ol8_appstream 236 k + copy-jdk-configs noarch 4.0-2.el8 ol8_appstream 30 k + cpio x86_64 2.12-11.el8 ol8_baseos_latest 266 k + crypto-policies-scripts noarch 20230731-1.git3177e06.el8 ol8_baseos_latest 84 k + cups-libs x86_64 1:2.2.6-60.el8_10 ol8_baseos_latest 435 k + dracut x86_64 049-233.git20240115.0.1.el8 ol8_baseos_latest 382 k + file x86_64 5.33-25.el8 ol8_baseos_latest 77 k + fribidi x86_64 1.0.4-9.el8 ol8_appstream 89 k + gdk-pixbuf2 x86_64 2.36.12-6.el8_10 ol8_baseos_latest 465 k + gdk-pixbuf2-modules x86_64 2.36.12-6.el8_10 ol8_appstream 108 k + gettext x86_64 0.19.8.1-17.el8 ol8_baseos_latest 1.1 M + gettext-libs x86_64 0.19.8.1-17.el8 ol8_baseos_latest 312 k + glib-networking x86_64 2.56.1-1.1.el8 ol8_baseos_latest 155 k + graphite2 x86_64 1.3.10-10.el8 ol8_appstream 122 k + grub2-common noarch 1:2.02-156.0.2.el8 ol8_baseos_latest 897 k + grub2-tools x86_64 1:2.02-156.0.2.el8 ol8_baseos_latest 2.0 M + grub2-tools-minimal x86_64 1:2.02-156.0.2.el8 ol8_baseos_latest 215 k + gsettings-desktop-schemas x86_64 3.32.0-6.el8 ol8_baseos_latest 633 k + gtk-update-icon-cache x86_64 3.22.30-11.el8 ol8_appstream 32 k + harfbuzz x86_64 1.7.5-4.el8 ol8_appstream 295 k + hicolor-icon-theme noarch 0.17-2.el8 ol8_appstream 48 k + jasper-libs x86_64 2.0.14-5.el8 ol8_appstream 167 k + java-11-openjdk x86_64 1:11.0.24.0.8-3.0.1.el8 ol8_appstream 475 k + java-11-openjdk-headless x86_64 1:11.0.24.0.8-3.0.1.el8 ol8_appstream 42 M + javapackages-filesystem noarch 5.3.0-1.module+el8+5136+7ff78f74 ol8_appstream 30 k + jbigkit-libs x86_64 2.1-14.el8 ol8_appstream 55 k + json-glib x86_64 1.4.4-1.el8 ol8_baseos_latest 144 k + kbd-legacy noarch 2.0.4-11.el8 ol8_baseos_latest 481 k + kbd-misc noarch 2.0.4-11.el8 ol8_baseos_latest 1.5 M + lcms2 x86_64 2.9-2.el8 ol8_appstream 164 k + libX11 x86_64 1.6.8-8.el8 ol8_appstream 611 k + libX11-common noarch 1.6.8-8.el8 ol8_appstream 157 k + libXau x86_64 1.0.9-3.el8 ol8_appstream 37 k + libXcomposite x86_64 0.4.4-14.el8 ol8_appstream 28 k + libXcursor x86_64 1.1.15-3.el8 ol8_appstream 36 k + libXdamage x86_64 1.1.4-14.el8 ol8_appstream 27 k + libXext x86_64 1.3.4-1.el8 ol8_appstream 45 k + libXfixes x86_64 5.0.3-7.el8 ol8_appstream 25 k + libXft x86_64 2.3.3-1.el8 ol8_appstream 67 k + libXi x86_64 1.7.10-1.el8 ol8_appstream 49 k + libXinerama x86_64 1.1.4-1.el8 ol8_appstream 15 k + libXrandr x86_64 1.5.2-1.el8 ol8_appstream 34 k + libXrender x86_64 0.9.10-7.el8 ol8_appstream 33 k + libXtst x86_64 1.2.3-7.el8 ol8_appstream 22 k + libcroco x86_64 0.6.12-4.el8_2.1 ol8_baseos_latest 113 k + libdatrie x86_64 0.2.9-7.el8 ol8_appstream 33 k + libepoxy x86_64 1.5.8-1.el8 ol8_appstream 225 k + libfontenc x86_64 1.1.3-8.el8 ol8_appstream 37 k + libgomp x86_64 8.5.0-22.0.1.el8_10 ol8_baseos_latest 218 k + libgusb x86_64 0.3.0-1.el8 ol8_baseos_latest 49 k + libjpeg-turbo x86_64 1.5.3-12.el8 ol8_appstream 157 k + libkcapi x86_64 1.4.0-2.0.1.el8 ol8_baseos_latest 52 k + libkcapi-hmaccalc x86_64 1.4.0-2.0.1.el8 ol8_baseos_latest 31 k + libmodman x86_64 2.0.1-17.el8 ol8_baseos_latest 36 k + libpkgconf x86_64 1.4.2-1.el8 ol8_baseos_latest 35 k + libproxy x86_64 0.4.15-5.2.el8 ol8_baseos_latest 75 k + libsoup x86_64 2.62.3-5.el8 ol8_baseos_latest 424 k + libthai x86_64 0.1.27-2.el8 ol8_appstream 203 k + libtiff x86_64 4.0.9-32.el8_10 ol8_appstream 189 k + libwayland-client x86_64 1.21.0-1.el8 ol8_appstream 41 k + libwayland-cursor x86_64 1.21.0-1.el8 ol8_appstream 26 k + libwayland-egl x86_64 1.21.0-1.el8 ol8_appstream 19 k + libxcb x86_64 1.13.1-1.el8 ol8_appstream 231 k + libxkbcommon x86_64 0.9.1-1.el8 ol8_appstream 116 k + lksctp-tools x86_64 1.0.18-3.el8 ol8_baseos_latest 100 k + lua x86_64 5.3.4-12.el8 ol8_appstream 192 k + nspr x86_64 4.35.0-1.el8_8 ol8_appstream 143 k + nss x86_64 3.90.0-7.el8_10 ol8_appstream 750 k + nss-softokn x86_64 3.90.0-7.el8_10 ol8_appstream 1.2 M + nss-softokn-freebl x86_64 3.90.0-7.el8_10 ol8_appstream 375 k + nss-sysinit x86_64 3.90.0-7.el8_10 ol8_appstream 74 k + nss-util x86_64 3.90.0-7.el8_10 ol8_appstream 139 k + os-prober x86_64 1.74-9.0.1.el8 ol8_baseos_latest 51 k + pango x86_64 1.42.4-8.el8 ol8_appstream 297 k + pixman x86_64 0.38.4-4.el8 ol8_appstream 256 k + pkgconf x86_64 1.4.2-1.el8 ol8_baseos_latest 38 k + pkgconf-m4 noarch 1.4.2-1.el8 ol8_baseos_latest 17 k + pkgconf-pkg-config x86_64 1.4.2-1.el8 ol8_baseos_latest 15 k + rest x86_64 0.8.1-2.el8 ol8_appstream 70 k + shared-mime-info x86_64 1.9-4.el8 ol8_baseos_latest 328 k + systemd-udev x86_64 239-78.0.4.el8 ol8_baseos_latest 1.6 M + ttmkfdir x86_64 3.0.9-54.el8 ol8_appstream 62 k + tzdata-java noarch 2024a-1.0.1.el8 ol8_appstream 186 k + xkeyboard-config noarch 2.28-1.el8 ol8_appstream 782 k + xorg-x11-font-utils x86_64 1:7.5-41.el8 ol8_appstream 104 k + xorg-x11-fonts-Type1 noarch 7.5-19.el8 ol8_appstream 522 k + xz x86_64 5.2.4-4.el8_6 ol8_baseos_latest 153 k +Installing weak dependencies: + abattis-cantarell-fonts noarch 0.0.25-6.el8 ol8_appstream 155 k + dconf x86_64 0.28.0-4.0.1.el8 ol8_appstream 108 k + dejavu-sans-mono-fonts noarch 2.35-7.el8 ol8_baseos_latest 447 k + grubby x86_64 8.40-49.0.2.el8 ol8_baseos_latest 50 k + gtk3 x86_64 3.22.30-11.el8 ol8_appstream 4.5 M + hardlink x86_64 1:1.3-6.el8 ol8_baseos_latest 29 k + kbd x86_64 2.0.4-11.el8 ol8_baseos_latest 390 k + memstrack x86_64 0.2.5-2.el8 ol8_baseos_latest 51 k + pigz x86_64 2.4-4.el8 ol8_baseos_latest 80 k Enabling module streams: - javapackages-runtime 201801 + javapackages-runtime 201801 Transaction Summary -============================================================================================= -Install 40 Packages +============================================================================================== +Install 106 Packages -Total download size: 50 M -Installed size: 196 M +Total download size: 86 M +Installed size: 312 M Downloading Packages: -(1/40): crypto-policies-scripts-20211116-1.gita 1.3 MB/s | 83 kB 00:00 -(2/40): avahi-libs-0.7-20.el8.x86_64.rpm 879 kB/s | 62 kB 00:00 -(3/40): libpkgconf-1.4.2-1.el8.x86_64.rpm 2.0 MB/s | 35 kB 00:00 -(4/40): cups-libs-2.2.6-45.el8_6.2.x86_64.rpm 4.5 MB/s | 434 kB 00:00 -(5/40): lksctp-tools-1.0.18-3.el8.x86_64.rpm 3.7 MB/s | 100 kB 00:00 -(6/40): pkgconf-1.4.2-1.el8.x86_64.rpm 2.2 MB/s | 38 kB 00:00 -(7/40): pkgconf-m4-1.4.2-1.el8.noarch.rpm 1.2 MB/s | 17 kB 00:00 -(8/40): pkgconf-pkg-config-1.4.2-1.el8.x86_64.r 929 kB/s | 15 kB 00:00 -(9/40): copy-jdk-configs-4.0-2.el8.noarch.rpm 2.2 MB/s | 30 kB 00:00 -(10/40): giflib-5.1.4-3.el8.x86_64.rpm 3.3 MB/s | 51 kB 00:00 -(11/40): graphite2-1.3.10-10.el8.x86_64.rpm 7.7 MB/s | 122 kB 00:00 -(12/40): alsa-lib-1.2.6.1-3.el8.x86_64.rpm 12 MB/s | 491 kB 00:00 -(13/40): java-11-openjdk-11.0.16.1.1-1.el8_6.x8 14 MB/s | 272 kB 00:00 -(14/40): harfbuzz-1.7.5-3.el8.x86_64.rpm 8.7 MB/s | 295 kB 00:00 -(15/40): javapackages-filesystem-5.3.0-1.module 2.0 MB/s | 30 kB 00:00 -(16/40): lcms2-2.9-2.el8.x86_64.rpm 6.7 MB/s | 164 kB 00:00 -(17/40): java-11-openjdk-devel-11.0.16.1.1-1.el 46 MB/s | 3.4 MB 00:00 -(18/40): libX11-common-1.6.8-5.el8.noarch.rpm 8.4 MB/s | 158 kB 00:00 -(19/40): libX11-1.6.8-5.el8.x86_64.rpm 17 MB/s | 611 kB 00:00 -(20/40): libXau-1.0.9-3.el8.x86_64.rpm 2.6 MB/s | 37 kB 00:00 -(21/40): libXcomposite-0.4.4-14.el8.x86_64.rpm 2.0 MB/s | 28 kB 00:00 -(22/40): libXi-1.7.10-1.el8.x86_64.rpm 2.2 MB/s | 49 kB 00:00 -(23/40): libXext-1.3.4-1.el8.x86_64.rpm 1.6 MB/s | 45 kB 00:00 -(24/40): libXtst-1.2.3-7.el8.x86_64.rpm 1.1 MB/s | 22 kB 00:00 -(25/40): libXrender-0.9.10-7.el8.x86_64.rpm 1.3 MB/s | 33 kB 00:00 -(26/40): libfontenc-1.1.3-8.el8.x86_64.rpm 2.2 MB/s | 37 kB 00:00 -(27/40): libjpeg-turbo-1.5.3-12.el8.x86_64.rpm 8.6 MB/s | 157 kB 00:00 -(28/40): libxcb-1.13.1-1.el8.x86_64.rpm 13 MB/s | 231 kB 00:00 -(29/40): lua-5.3.4-12.el8.x86_64.rpm 11 MB/s | 192 kB 00:00 -(30/40): nspr-4.34.0-3.el8_6.x86_64.rpm 7.8 MB/s | 143 kB 00:00 -(31/40): nss-3.79.0-10.el8_6.x86_64.rpm 23 MB/s | 747 kB 00:00 -(32/40): nss-softokn-3.79.0-10.el8_6.x86_64.rpm 42 MB/s | 1.2 MB 00:00 -(33/40): nss-softokn-freebl-3.79.0-10.el8_6.x86 19 MB/s | 398 kB 00:00 -(34/40): nss-sysinit-3.79.0-10.el8_6.x86_64.rpm 5.3 MB/s | 74 kB 00:00 -(35/40): nss-util-3.79.0-10.el8_6.x86_64.rpm 8.7 MB/s | 138 kB 00:00 -(36/40): ttmkfdir-3.0.9-54.el8.x86_64.rpm 4.2 MB/s | 62 kB 00:00 -(37/40): tzdata-java-2022d-1.el8.noarch.rpm 11 MB/s | 186 kB 00:00 -(38/40): xorg-x11-font-utils-7.5-41.el8.x86_64. 6.7 MB/s | 104 kB 00:00 -(39/40): xorg-x11-fonts-Type1-7.5-19.el8.noarch 24 MB/s | 522 kB 00:00 -(40/40): java-11-openjdk-headless-11.0.16.1.1-1 77 MB/s | 40 MB 00:00 +(1/106): crypto-policies-scripts-20230731-1.git 862 kB/s | 84 kB 00:00 +(2/106): avahi-libs-0.7-27.el8.x86_64.rpm 602 kB/s | 61 kB 00:00 +(3/106): cpio-2.12-11.el8.x86_64.rpm 1.8 MB/s | 266 kB 00:00 +(4/106): cups-libs-2.2.6-60.el8_10.x86_64.rpm 5.7 MB/s | 435 kB 00:00 +(5/106): dejavu-sans-mono-fonts-2.35-7.el8.noar 5.1 MB/s | 447 kB 00:00 +(6/106): dracut-049-233.git20240115.0.1.el8.x86 7.0 MB/s | 382 kB 00:00 +(7/106): gdk-pixbuf2-2.36.12-6.el8_10.x86_64.rp 12 MB/s | 465 kB 00:00 +(8/106): gettext-libs-0.19.8.1-17.el8.x86_64.rp 9.3 MB/s | 312 kB 00:00 +(9/106): gettext-0.19.8.1-17.el8.x86_64.rpm 16 MB/s | 1.1 MB 00:00 +(10/106): glib-networking-2.56.1-1.1.el8.x86_64 6.0 MB/s | 155 kB 00:00 +(11/106): grub2-common-2.02-156.0.2.el8.noarch. 26 MB/s | 897 kB 00:00 +(12/106): grub2-tools-minimal-2.02-156.0.2.el8. 8.2 MB/s | 215 kB 00:00 +(13/106): grubby-8.40-49.0.2.el8.x86_64.rpm 2.1 MB/s | 50 kB 00:00 +(14/106): grub2-tools-2.02-156.0.2.el8.x86_64.r 26 MB/s | 2.0 MB 00:00 +(15/106): gsettings-desktop-schemas-3.32.0-6.el 19 MB/s | 633 kB 00:00 +(16/106): hardlink-1.3-6.el8.x86_64.rpm 1.1 MB/s | 29 kB 00:00 +(17/106): json-glib-1.4.4-1.el8.x86_64.rpm 5.9 MB/s | 144 kB 00:00 +(18/106): kbd-2.0.4-11.el8.x86_64.rpm 14 MB/s | 390 kB 00:00 +(19/106): kbd-legacy-2.0.4-11.el8.noarch.rpm 17 MB/s | 481 kB 00:00 +(20/106): kbd-misc-2.0.4-11.el8.noarch.rpm 41 MB/s | 1.5 MB 00:00 +(21/106): libcroco-0.6.12-4.el8_2.1.x86_64.rpm 4.7 MB/s | 113 kB 00:00 +(22/106): libgomp-8.5.0-22.0.1.el8_10.x86_64.rp 9.1 MB/s | 218 kB 00:00 +(23/106): libgusb-0.3.0-1.el8.x86_64.rpm 2.1 MB/s | 49 kB 00:00 +(24/106): libkcapi-1.4.0-2.0.1.el8.x86_64.rpm 1.6 MB/s | 52 kB 00:00 +(25/106): libkcapi-hmaccalc-1.4.0-2.0.1.el8.x86 822 kB/s | 31 kB 00:00 +(26/106): libmodman-2.0.1-17.el8.x86_64.rpm 1.6 MB/s | 36 kB 00:00 +(27/106): libpkgconf-1.4.2-1.el8.x86_64.rpm 1.2 MB/s | 35 kB 00:00 +(28/106): libproxy-0.4.15-5.2.el8.x86_64.rpm 3.0 MB/s | 75 kB 00:00 +(29/106): libsoup-2.62.3-5.el8.x86_64.rpm 15 MB/s | 424 kB 00:00 +(30/106): lksctp-tools-1.0.18-3.el8.x86_64.rpm 3.5 MB/s | 100 kB 00:00 +(31/106): memstrack-0.2.5-2.el8.x86_64.rpm 2.2 MB/s | 51 kB 00:00 +(32/106): os-prober-1.74-9.0.1.el8.x86_64.rpm 2.2 MB/s | 51 kB 00:00 +(33/106): pigz-2.4-4.el8.x86_64.rpm 3.5 MB/s | 80 kB 00:00 +(34/106): pkgconf-1.4.2-1.el8.x86_64.rpm 1.7 MB/s | 38 kB 00:00 +(35/106): pkgconf-m4-1.4.2-1.el8.noarch.rpm 761 kB/s | 17 kB 00:00 +(36/106): pkgconf-pkg-config-1.4.2-1.el8.x86_64 691 kB/s | 15 kB 00:00 +(37/106): shared-mime-info-1.9-4.el8.x86_64.rpm 13 MB/s | 328 kB 00:00 +(38/106): systemd-udev-239-78.0.4.el8.x86_64.rp 32 MB/s | 1.6 MB 00:00 +(39/106): xz-5.2.4-4.el8_6.x86_64.rpm 5.2 MB/s | 153 kB 00:00 +(40/106): abattis-cantarell-fonts-0.0.25-6.el8. 6.4 MB/s | 155 kB 00:00 +(41/106): adwaita-cursor-theme-3.28.0-3.el8.noa 22 MB/s | 647 kB 00:00 +(42/106): alsa-lib-1.2.10-2.el8.x86_64.rpm 18 MB/s | 500 kB 00:00 +(43/106): at-spi2-atk-2.26.2-1.el8.x86_64.rpm 3.8 MB/s | 89 kB 00:00 +(44/106): at-spi2-core-2.28.0-1.el8.x86_64.rpm 6.9 MB/s | 169 kB 00:00 +(45/106): atk-2.28.1-1.el8.x86_64.rpm 9.2 MB/s | 272 kB 00:00 +(46/106): cairo-1.15.12-6.el8.x86_64.rpm 24 MB/s | 719 kB 00:00 +(47/106): adwaita-icon-theme-3.28.0-3.el8.noarc 65 MB/s | 11 MB 00:00 +(48/106): cairo-gobject-1.15.12-6.el8.x86_64.rp 914 kB/s | 33 kB 00:00 +(49/106): colord-libs-1.4.2-1.el8.x86_64.rpm 9.5 MB/s | 236 kB 00:00 +(50/106): copy-jdk-configs-4.0-2.el8.noarch.rpm 1.1 MB/s | 30 kB 00:00 +(51/106): dconf-0.28.0-4.0.1.el8.x86_64.rpm 4.4 MB/s | 108 kB 00:00 +(52/106): fribidi-1.0.4-9.el8.x86_64.rpm 3.9 MB/s | 89 kB 00:00 +(53/106): graphite2-1.3.10-10.el8.x86_64.rpm 5.1 MB/s | 122 kB 00:00 +(54/106): gdk-pixbuf2-modules-2.36.12-6.el8_10. 3.6 MB/s | 108 kB 00:00 +(55/106): gtk-update-icon-cache-3.22.30-11.el8. 1.4 MB/s | 32 kB 00:00 +(56/106): harfbuzz-1.7.5-4.el8.x86_64.rpm 11 MB/s | 295 kB 00:00 +(57/106): gtk3-3.22.30-11.el8.x86_64.rpm 68 MB/s | 4.5 MB 00:00 +(58/106): hicolor-icon-theme-0.17-2.el8.noarch. 2.1 MB/s | 48 kB 00:00 +(59/106): java-11-openjdk-11.0.24.0.8-3.0.1.el8 17 MB/s | 475 kB 00:00 +(60/106): jasper-libs-2.0.14-5.el8.x86_64.rpm 5.0 MB/s | 167 kB 00:00 +(61/106): java-11-openjdk-devel-11.0.24.0.8-3.0 61 MB/s | 3.4 MB 00:00 +(62/106): javapackages-filesystem-5.3.0-1.modul 1.2 MB/s | 30 kB 00:00 +(63/106): jbigkit-libs-2.1-14.el8.x86_64.rpm 2.1 MB/s | 55 kB 00:00 +(64/106): lcms2-2.9-2.el8.x86_64.rpm 3.8 MB/s | 164 kB 00:00 +(65/106): libX11-1.6.8-8.el8.x86_64.rpm 20 MB/s | 611 kB 00:00 +(66/106): libX11-common-1.6.8-8.el8.noarch.rpm 6.8 MB/s | 157 kB 00:00 +(67/106): libXau-1.0.9-3.el8.x86_64.rpm 1.6 MB/s | 37 kB 00:00 +(68/106): libXcomposite-0.4.4-14.el8.x86_64.rpm 1.3 MB/s | 28 kB 00:00 +(69/106): libXcursor-1.1.15-3.el8.x86_64.rpm 1.6 MB/s | 36 kB 00:00 +(70/106): libXdamage-1.1.4-14.el8.x86_64.rpm 1.2 MB/s | 27 kB 00:00 +(71/106): libXext-1.3.4-1.el8.x86_64.rpm 2.0 MB/s | 45 kB 00:00 +(72/106): libXfixes-5.0.3-7.el8.x86_64.rpm 1.1 MB/s | 25 kB 00:00 +(73/106): libXft-2.3.3-1.el8.x86_64.rpm 2.9 MB/s | 67 kB 00:00 +(74/106): libXi-1.7.10-1.el8.x86_64.rpm 2.2 MB/s | 49 kB 00:00 +(75/106): libXinerama-1.1.4-1.el8.x86_64.rpm 717 kB/s | 15 kB 00:00 +(76/106): libXrandr-1.5.2-1.el8.x86_64.rpm 1.5 MB/s | 34 kB 00:00 +(77/106): libXrender-0.9.10-7.el8.x86_64.rpm 1.4 MB/s | 33 kB 00:00 +(78/106): libXtst-1.2.3-7.el8.x86_64.rpm 957 kB/s | 22 kB 00:00 +(79/106): java-11-openjdk-headless-11.0.24.0.8- 71 MB/s | 42 MB 00:00 +(80/106): libdatrie-0.2.9-7.el8.x86_64.rpm 274 kB/s | 33 kB 00:00 +(81/106): libepoxy-1.5.8-1.el8.x86_64.rpm 9.1 MB/s | 225 kB 00:00 +(82/106): libfontenc-1.1.3-8.el8.x86_64.rpm 1.5 MB/s | 37 kB 00:00 +(83/106): libthai-0.1.27-2.el8.x86_64.rpm 8.2 MB/s | 203 kB 00:00 +(84/106): libjpeg-turbo-1.5.3-12.el8.x86_64.rpm 5.1 MB/s | 157 kB 00:00 +(85/106): libtiff-4.0.9-32.el8_10.x86_64.rpm 7.8 MB/s | 189 kB 00:00 +(86/106): libwayland-client-1.21.0-1.el8.x86_64 1.7 MB/s | 41 kB 00:00 +(87/106): libwayland-cursor-1.21.0-1.el8.x86_64 1.2 MB/s | 26 kB 00:00 +(88/106): libwayland-egl-1.21.0-1.el8.x86_64.rp 801 kB/s | 19 kB 00:00 +(89/106): libxcb-1.13.1-1.el8.x86_64.rpm 9.7 MB/s | 231 kB 00:00 +(90/106): libxkbcommon-0.9.1-1.el8.x86_64.rpm 5.0 MB/s | 116 kB 00:00 +(91/106): nspr-4.35.0-1.el8_8.x86_64.rpm 6.0 MB/s | 143 kB 00:00 +(92/106): lua-5.3.4-12.el8.x86_64.rpm 5.9 MB/s | 192 kB 00:00 +(93/106): nss-softokn-3.90.0-7.el8_10.x86_64.rp 38 MB/s | 1.2 MB 00:00 +(94/106): nss-3.90.0-7.el8_10.x86_64.rpm 17 MB/s | 750 kB 00:00 +(95/106): nss-softokn-freebl-3.90.0-7.el8_10.x8 14 MB/s | 375 kB 00:00 +(96/106): nss-sysinit-3.90.0-7.el8_10.x86_64.rp 3.2 MB/s | 74 kB 00:00 +(97/106): nss-util-3.90.0-7.el8_10.x86_64.rpm 5.8 MB/s | 139 kB 00:00 +(98/106): pango-1.42.4-8.el8.x86_64.rpm 11 MB/s | 297 kB 00:00 +(99/106): pixman-0.38.4-4.el8.x86_64.rpm 10 MB/s | 256 kB 00:00 +(100/106): rest-0.8.1-2.el8.x86_64.rpm 3.1 MB/s | 70 kB 00:00 +(101/106): ttmkfdir-3.0.9-54.el8.x86_64.rpm 2.5 MB/s | 62 kB 00:00 +(102/106): tzdata-java-2024a-1.0.1.el8.noarch.r 7.4 MB/s | 186 kB 00:00 +(103/106): xkeyboard-config-2.28-1.el8.noarch.r 27 MB/s | 782 kB 00:00 +(104/106): xorg-x11-font-utils-7.5-41.el8.x86_6 3.9 MB/s | 104 kB 00:00 +(105/106): xorg-x11-fonts-Type1-7.5-19.el8.noar 1.3 MB/s | 522 kB 00:00 +(106/106): file-5.33-25.el8.x86_64.rpm 26 kB/s | 77 kB 00:02 -------------------------------------------------------------------------------- -Total 74 MB/s | 50 MB 00:00 +Total 27 MB/s | 86 MB 00:03 Running transaction check Transaction check succeeded. Running transaction test Transaction test succeeded. Running transaction Running scriptlet: copy-jdk-configs-4.0-2.el8.noarch 1/1 - Running scriptlet: java-11-openjdk-headless-1:11.0.16.1.1-1.el8_6.x86_6 1/1 + Running scriptlet: java-11-openjdk-headless-1:11.0.24.0.8-3.0.1.el8.x86 1/1 Preparing : 1/1 - Installing : nspr-4.34.0-3.el8_6.x86_64 1/40 - Running scriptlet: nspr-4.34.0-3.el8_6.x86_64 1/40 - Installing : nss-util-3.79.0-10.el8_6.x86_64 2/40 - Installing : libjpeg-turbo-1.5.3-12.el8.x86_64 3/40 - Installing : nss-softokn-freebl-3.79.0-10.el8_6.x86_64 4/40 - Installing : nss-softokn-3.79.0-10.el8_6.x86_64 5/40 - Installing : tzdata-java-2022d-1.el8.noarch 6/40 - Installing : ttmkfdir-3.0.9-54.el8.x86_64 7/40 - Installing : lua-5.3.4-12.el8.x86_64 8/40 - Installing : copy-jdk-configs-4.0-2.el8.noarch 9/40 - Installing : libfontenc-1.1.3-8.el8.x86_64 10/40 - Installing : libXau-1.0.9-3.el8.x86_64 11/40 - Installing : libxcb-1.13.1-1.el8.x86_64 12/40 - Installing : libX11-common-1.6.8-5.el8.noarch 13/40 - Installing : libX11-1.6.8-5.el8.x86_64 14/40 - Installing : libXext-1.3.4-1.el8.x86_64 15/40 - Installing : libXi-1.7.10-1.el8.x86_64 16/40 - Installing : libXtst-1.2.3-7.el8.x86_64 17/40 - Installing : libXcomposite-0.4.4-14.el8.x86_64 18/40 - Installing : libXrender-0.9.10-7.el8.x86_64 19/40 - Installing : lcms2-2.9-2.el8.x86_64 20/40 - Running scriptlet: lcms2-2.9-2.el8.x86_64 20/40 - Installing : javapackages-filesystem-5.3.0-1.module+el8+5136+7f 21/40 - Installing : graphite2-1.3.10-10.el8.x86_64 22/40 - Installing : harfbuzz-1.7.5-3.el8.x86_64 23/40 - Running scriptlet: harfbuzz-1.7.5-3.el8.x86_64 23/40 - Installing : giflib-5.1.4-3.el8.x86_64 24/40 - Installing : alsa-lib-1.2.6.1-3.el8.x86_64 25/40 - Running scriptlet: alsa-lib-1.2.6.1-3.el8.x86_64 25/40 - Installing : pkgconf-m4-1.4.2-1.el8.noarch 26/40 - Installing : lksctp-tools-1.0.18-3.el8.x86_64 27/40 - Running scriptlet: lksctp-tools-1.0.18-3.el8.x86_64 27/40 - Installing : libpkgconf-1.4.2-1.el8.x86_64 28/40 - Installing : pkgconf-1.4.2-1.el8.x86_64 29/40 - Installing : pkgconf-pkg-config-1.4.2-1.el8.x86_64 30/40 - Installing : xorg-x11-font-utils-1:7.5-41.el8.x86_64 31/40 - Installing : xorg-x11-fonts-Type1-7.5-19.el8.noarch 32/40 - Running scriptlet: xorg-x11-fonts-Type1-7.5-19.el8.noarch 32/40 - Installing : crypto-policies-scripts-20211116-1.gitae470d6.el8. 33/40 - Installing : nss-sysinit-3.79.0-10.el8_6.x86_64 34/40 - Installing : nss-3.79.0-10.el8_6.x86_64 35/40 - Installing : avahi-libs-0.7-20.el8.x86_64 36/40 - Installing : cups-libs-1:2.2.6-45.el8_6.2.x86_64 37/40 - Installing : java-11-openjdk-headless-1:11.0.16.1.1-1.el8_6.x86 38/40 - Running scriptlet: java-11-openjdk-headless-1:11.0.16.1.1-1.el8_6.x86 38/40 - Installing : java-11-openjdk-1:11.0.16.1.1-1.el8_6.x86_64 39/40 - Running scriptlet: java-11-openjdk-1:11.0.16.1.1-1.el8_6.x86_64 39/40 - Installing : java-11-openjdk-devel-1:11.0.16.1.1-1.el8_6.x86_64 40/40 - Running scriptlet: java-11-openjdk-devel-1:11.0.16.1.1-1.el8_6.x86_64 40/40 - Running scriptlet: copy-jdk-configs-4.0-2.el8.noarch 40/40 - Running scriptlet: crypto-policies-scripts-20211116-1.gitae470d6.el8. 40/40 - Running scriptlet: nss-3.79.0-10.el8_6.x86_64 40/40 - Running scriptlet: java-11-openjdk-headless-1:11.0.16.1.1-1.el8_6.x86 40/40 - Running scriptlet: java-11-openjdk-1:11.0.16.1.1-1.el8_6.x86_64 40/40 - Running scriptlet: java-11-openjdk-devel-1:11.0.16.1.1-1.el8_6.x86_64 40/40 - Verifying : avahi-libs-0.7-20.el8.x86_64 1/40 - Verifying : crypto-policies-scripts-20211116-1.gitae470d6.el8. 2/40 - Verifying : cups-libs-1:2.2.6-45.el8_6.2.x86_64 3/40 - Verifying : libpkgconf-1.4.2-1.el8.x86_64 4/40 - Verifying : lksctp-tools-1.0.18-3.el8.x86_64 5/40 - Verifying : pkgconf-1.4.2-1.el8.x86_64 6/40 - Verifying : pkgconf-m4-1.4.2-1.el8.noarch 7/40 - Verifying : pkgconf-pkg-config-1.4.2-1.el8.x86_64 8/40 - Verifying : alsa-lib-1.2.6.1-3.el8.x86_64 9/40 - Verifying : copy-jdk-configs-4.0-2.el8.noarch 10/40 - Verifying : giflib-5.1.4-3.el8.x86_64 11/40 - Verifying : graphite2-1.3.10-10.el8.x86_64 12/40 - Verifying : harfbuzz-1.7.5-3.el8.x86_64 13/40 - Verifying : java-11-openjdk-1:11.0.16.1.1-1.el8_6.x86_64 14/40 - Verifying : java-11-openjdk-devel-1:11.0.16.1.1-1.el8_6.x86_64 15/40 - Verifying : java-11-openjdk-headless-1:11.0.16.1.1-1.el8_6.x86 16/40 - Verifying : javapackages-filesystem-5.3.0-1.module+el8+5136+7f 17/40 - Verifying : lcms2-2.9-2.el8.x86_64 18/40 - Verifying : libX11-1.6.8-5.el8.x86_64 19/40 - Verifying : libX11-common-1.6.8-5.el8.noarch 20/40 - Verifying : libXau-1.0.9-3.el8.x86_64 21/40 - Verifying : libXcomposite-0.4.4-14.el8.x86_64 22/40 - Verifying : libXext-1.3.4-1.el8.x86_64 23/40 - Verifying : libXi-1.7.10-1.el8.x86_64 24/40 - Verifying : libXrender-0.9.10-7.el8.x86_64 25/40 - Verifying : libXtst-1.2.3-7.el8.x86_64 26/40 - Verifying : libfontenc-1.1.3-8.el8.x86_64 27/40 - Verifying : libjpeg-turbo-1.5.3-12.el8.x86_64 28/40 - Verifying : libxcb-1.13.1-1.el8.x86_64 29/40 - Verifying : lua-5.3.4-12.el8.x86_64 30/40 - Verifying : nspr-4.34.0-3.el8_6.x86_64 31/40 - Verifying : nss-3.79.0-10.el8_6.x86_64 32/40 - Verifying : nss-softokn-3.79.0-10.el8_6.x86_64 33/40 - Verifying : nss-softokn-freebl-3.79.0-10.el8_6.x86_64 34/40 - Verifying : nss-sysinit-3.79.0-10.el8_6.x86_64 35/40 - Verifying : nss-util-3.79.0-10.el8_6.x86_64 36/40 - Verifying : ttmkfdir-3.0.9-54.el8.x86_64 37/40 - Verifying : tzdata-java-2022d-1.el8.noarch 38/40 - Verifying : xorg-x11-font-utils-1:7.5-41.el8.x86_64 39/40 - Verifying : xorg-x11-fonts-Type1-7.5-19.el8.noarch 40/40 + Installing : nspr-4.35.0-1.el8_8.x86_64 1/106 + Running scriptlet: nspr-4.35.0-1.el8_8.x86_64 1/106 + Installing : nss-util-3.90.0-7.el8_10.x86_64 2/106 + Installing : libjpeg-turbo-1.5.3-12.el8.x86_64 3/106 + Installing : pixman-0.38.4-4.el8.x86_64 4/106 + Installing : libwayland-client-1.21.0-1.el8.x86_64 5/106 + Installing : atk-2.28.1-1.el8.x86_64 6/106 + Installing : libgomp-8.5.0-22.0.1.el8_10.x86_64 7/106 + Running scriptlet: libgomp-8.5.0-22.0.1.el8_10.x86_64 7/106 + Installing : libcroco-0.6.12-4.el8_2.1.x86_64 8/106 + Running scriptlet: libcroco-0.6.12-4.el8_2.1.x86_64 8/106 + Installing : grub2-common-1:2.02-156.0.2.el8.noarch 9/106 + Installing : gettext-libs-0.19.8.1-17.el8.x86_64 10/106 + Installing : gettext-0.19.8.1-17.el8.x86_64 11/106 + Running scriptlet: gettext-0.19.8.1-17.el8.x86_64 11/106 + Installing : grub2-tools-minimal-1:2.02-156.0.2.el8.x86_64 12/106 + Installing : libwayland-cursor-1.21.0-1.el8.x86_64 13/106 + Installing : jasper-libs-2.0.14-5.el8.x86_64 14/106 + Installing : nss-softokn-freebl-3.90.0-7.el8_10.x86_64 15/106 + Installing : nss-softokn-3.90.0-7.el8_10.x86_64 16/106 + Installing : xkeyboard-config-2.28-1.el8.noarch 17/106 + Installing : libxkbcommon-0.9.1-1.el8.x86_64 18/106 + Installing : tzdata-java-2024a-1.0.1.el8.noarch 19/106 + Installing : ttmkfdir-3.0.9-54.el8.x86_64 20/106 + Installing : lua-5.3.4-12.el8.x86_64 21/106 + Installing : copy-jdk-configs-4.0-2.el8.noarch 22/106 + Installing : libwayland-egl-1.21.0-1.el8.x86_64 23/106 + Installing : libfontenc-1.1.3-8.el8.x86_64 24/106 + Installing : libepoxy-1.5.8-1.el8.x86_64 25/106 + Installing : libdatrie-0.2.9-7.el8.x86_64 26/106 + Running scriptlet: libdatrie-0.2.9-7.el8.x86_64 26/106 + Installing : libthai-0.1.27-2.el8.x86_64 27/106 + Running scriptlet: libthai-0.1.27-2.el8.x86_64 27/106 + Installing : libXau-1.0.9-3.el8.x86_64 28/106 + Installing : libxcb-1.13.1-1.el8.x86_64 29/106 + Installing : libX11-common-1.6.8-8.el8.noarch 30/106 + Installing : libX11-1.6.8-8.el8.x86_64 31/106 + Installing : libXext-1.3.4-1.el8.x86_64 32/106 + Installing : libXrender-0.9.10-7.el8.x86_64 33/106 + Installing : cairo-1.15.12-6.el8.x86_64 34/106 + Installing : libXi-1.7.10-1.el8.x86_64 35/106 + Installing : libXfixes-5.0.3-7.el8.x86_64 36/106 + Installing : libXtst-1.2.3-7.el8.x86_64 37/106 + Installing : libXcomposite-0.4.4-14.el8.x86_64 38/106 + Installing : at-spi2-core-2.28.0-1.el8.x86_64 39/106 + Running scriptlet: at-spi2-core-2.28.0-1.el8.x86_64 39/106 + Installing : at-spi2-atk-2.26.2-1.el8.x86_64 40/106 + Running scriptlet: at-spi2-atk-2.26.2-1.el8.x86_64 40/106 + Installing : libXcursor-1.1.15-3.el8.x86_64 41/106 + Installing : libXdamage-1.1.4-14.el8.x86_64 42/106 + Installing : cairo-gobject-1.15.12-6.el8.x86_64 43/106 + Installing : libXft-2.3.3-1.el8.x86_64 44/106 + Installing : libXrandr-1.5.2-1.el8.x86_64 45/106 + Installing : libXinerama-1.1.4-1.el8.x86_64 46/106 + Installing : lcms2-2.9-2.el8.x86_64 47/106 + Running scriptlet: lcms2-2.9-2.el8.x86_64 47/106 + Installing : jbigkit-libs-2.1-14.el8.x86_64 48/106 + Running scriptlet: jbigkit-libs-2.1-14.el8.x86_64 48/106 + Installing : libtiff-4.0.9-32.el8_10.x86_64 49/106 + Installing : javapackages-filesystem-5.3.0-1.module+el8+5136+ 50/106 + Installing : hicolor-icon-theme-0.17-2.el8.noarch 51/106 + Installing : graphite2-1.3.10-10.el8.x86_64 52/106 + Installing : harfbuzz-1.7.5-4.el8.x86_64 53/106 + Running scriptlet: harfbuzz-1.7.5-4.el8.x86_64 53/106 + Installing : fribidi-1.0.4-9.el8.x86_64 54/106 + Installing : pango-1.42.4-8.el8.x86_64 55/106 + Running scriptlet: pango-1.42.4-8.el8.x86_64 55/106 + Installing : dconf-0.28.0-4.0.1.el8.x86_64 56/106 + Installing : alsa-lib-1.2.10-2.el8.x86_64 57/106 + Running scriptlet: alsa-lib-1.2.10-2.el8.x86_64 57/106 + Installing : adwaita-cursor-theme-3.28.0-3.el8.noarch 58/106 + Installing : adwaita-icon-theme-3.28.0-3.el8.noarch 59/106 + Installing : abattis-cantarell-fonts-0.0.25-6.el8.noarch 60/106 + Installing : xz-5.2.4-4.el8_6.x86_64 61/106 + Installing : shared-mime-info-1.9-4.el8.x86_64 62/106 + Running scriptlet: shared-mime-info-1.9-4.el8.x86_64 62/106 + Installing : gdk-pixbuf2-2.36.12-6.el8_10.x86_64 63/106 + Running scriptlet: gdk-pixbuf2-2.36.12-6.el8_10.x86_64 63/106 + Installing : gdk-pixbuf2-modules-2.36.12-6.el8_10.x86_64 64/106 + Installing : gtk-update-icon-cache-3.22.30-11.el8.x86_64 65/106 + Installing : pkgconf-m4-1.4.2-1.el8.noarch 66/106 + Installing : pigz-2.4-4.el8.x86_64 67/106 + Installing : memstrack-0.2.5-2.el8.x86_64 68/106 + Installing : lksctp-tools-1.0.18-3.el8.x86_64 69/106 + Running scriptlet: lksctp-tools-1.0.18-3.el8.x86_64 69/106 + Installing : libpkgconf-1.4.2-1.el8.x86_64 70/106 + Installing : pkgconf-1.4.2-1.el8.x86_64 71/106 + Installing : pkgconf-pkg-config-1.4.2-1.el8.x86_64 72/106 + Installing : xorg-x11-font-utils-1:7.5-41.el8.x86_64 73/106 + Installing : xorg-x11-fonts-Type1-7.5-19.el8.noarch 74/106 + Running scriptlet: xorg-x11-fonts-Type1-7.5-19.el8.noarch 74/106 + Installing : libmodman-2.0.1-17.el8.x86_64 75/106 + Running scriptlet: libmodman-2.0.1-17.el8.x86_64 75/106 + Installing : libproxy-0.4.15-5.2.el8.x86_64 76/106 + Running scriptlet: libproxy-0.4.15-5.2.el8.x86_64 76/106 + Installing : libkcapi-1.4.0-2.0.1.el8.x86_64 77/106 + Installing : libkcapi-hmaccalc-1.4.0-2.0.1.el8.x86_64 78/106 + Installing : libgusb-0.3.0-1.el8.x86_64 79/106 + Installing : colord-libs-1.4.2-1.el8.x86_64 80/106 + Installing : kbd-misc-2.0.4-11.el8.noarch 81/106 + Installing : kbd-legacy-2.0.4-11.el8.noarch 82/106 + Installing : kbd-2.0.4-11.el8.x86_64 83/106 + Installing : systemd-udev-239-78.0.4.el8.x86_64 84/106 + Running scriptlet: systemd-udev-239-78.0.4.el8.x86_64 84/106 + Installing : os-prober-1.74-9.0.1.el8.x86_64 85/106 + Installing : json-glib-1.4.4-1.el8.x86_64 86/106 + Installing : hardlink-1:1.3-6.el8.x86_64 87/106 + Installing : file-5.33-25.el8.x86_64 88/106 + Installing : dejavu-sans-mono-fonts-2.35-7.el8.noarch 89/106 + Installing : gsettings-desktop-schemas-3.32.0-6.el8.x86_64 90/106 + Installing : glib-networking-2.56.1-1.1.el8.x86_64 91/106 + Installing : libsoup-2.62.3-5.el8.x86_64 92/106 + Installing : rest-0.8.1-2.el8.x86_64 93/106 + Running scriptlet: rest-0.8.1-2.el8.x86_64 93/106 + Installing : cpio-2.12-11.el8.x86_64 94/106 + Installing : dracut-049-233.git20240115.0.1.el8.x86_64 95/106 + Running scriptlet: grub2-tools-1:2.02-156.0.2.el8.x86_64 96/106 + Installing : grub2-tools-1:2.02-156.0.2.el8.x86_64 96/106 + Running scriptlet: grub2-tools-1:2.02-156.0.2.el8.x86_64 96/106 + Installing : grubby-8.40-49.0.2.el8.x86_64 97/106 + Installing : crypto-policies-scripts-20230731-1.git3177e06.el 98/106 + Installing : nss-sysinit-3.90.0-7.el8_10.x86_64 99/106 + Installing : nss-3.90.0-7.el8_10.x86_64 100/106 + Installing : avahi-libs-0.7-27.el8.x86_64 101/106 + Installing : cups-libs-1:2.2.6-60.el8_10.x86_64 102/106 + Installing : java-11-openjdk-headless-1:11.0.24.0.8-3.0.1.el8 103/106 + Running scriptlet: java-11-openjdk-headless-1:11.0.24.0.8-3.0.1.el8 103/106 + Installing : gtk3-3.22.30-11.el8.x86_64 104/106 + Installing : java-11-openjdk-1:11.0.24.0.8-3.0.1.el8.x86_64 105/106 + Running scriptlet: java-11-openjdk-1:11.0.24.0.8-3.0.1.el8.x86_64 105/106 + Installing : java-11-openjdk-devel-1:11.0.24.0.8-3.0.1.el8.x8 106/106 + Running scriptlet: java-11-openjdk-devel-1:11.0.24.0.8-3.0.1.el8.x8 106/106 + Running scriptlet: copy-jdk-configs-4.0-2.el8.noarch 106/106 + Running scriptlet: dconf-0.28.0-4.0.1.el8.x86_64 106/106 + Running scriptlet: crypto-policies-scripts-20230731-1.git3177e06.el 106/106 + Running scriptlet: nss-3.90.0-7.el8_10.x86_64 106/106 + Running scriptlet: java-11-openjdk-headless-1:11.0.24.0.8-3.0.1.el8 106/106 + Running scriptlet: java-11-openjdk-1:11.0.24.0.8-3.0.1.el8.x86_64 106/106 + Running scriptlet: java-11-openjdk-devel-1:11.0.24.0.8-3.0.1.el8.x8 106/106 + Running scriptlet: hicolor-icon-theme-0.17-2.el8.noarch 106/106 + Running scriptlet: adwaita-icon-theme-3.28.0-3.el8.noarch 106/106 + Running scriptlet: shared-mime-info-1.9-4.el8.x86_64 106/106 + Running scriptlet: gdk-pixbuf2-2.36.12-6.el8_10.x86_64 106/106 + Running scriptlet: systemd-udev-239-78.0.4.el8.x86_64 106/106 + Verifying : avahi-libs-0.7-27.el8.x86_64 1/106 + Verifying : cpio-2.12-11.el8.x86_64 2/106 + Verifying : crypto-policies-scripts-20230731-1.git3177e06.el 3/106 + Verifying : cups-libs-1:2.2.6-60.el8_10.x86_64 4/106 + Verifying : dejavu-sans-mono-fonts-2.35-7.el8.noarch 5/106 + Verifying : dracut-049-233.git20240115.0.1.el8.x86_64 6/106 + Verifying : file-5.33-25.el8.x86_64 7/106 + Verifying : gdk-pixbuf2-2.36.12-6.el8_10.x86_64 8/106 + Verifying : gettext-0.19.8.1-17.el8.x86_64 9/106 + Verifying : gettext-libs-0.19.8.1-17.el8.x86_64 10/106 + Verifying : glib-networking-2.56.1-1.1.el8.x86_64 11/106 + Verifying : grub2-common-1:2.02-156.0.2.el8.noarch 12/106 + Verifying : grub2-tools-1:2.02-156.0.2.el8.x86_64 13/106 + Verifying : grub2-tools-minimal-1:2.02-156.0.2.el8.x86_64 14/106 + Verifying : grubby-8.40-49.0.2.el8.x86_64 15/106 + Verifying : gsettings-desktop-schemas-3.32.0-6.el8.x86_64 16/106 + Verifying : hardlink-1:1.3-6.el8.x86_64 17/106 + Verifying : json-glib-1.4.4-1.el8.x86_64 18/106 + Verifying : kbd-2.0.4-11.el8.x86_64 19/106 + Verifying : kbd-legacy-2.0.4-11.el8.noarch 20/106 + Verifying : kbd-misc-2.0.4-11.el8.noarch 21/106 + Verifying : libcroco-0.6.12-4.el8_2.1.x86_64 22/106 + Verifying : libgomp-8.5.0-22.0.1.el8_10.x86_64 23/106 + Verifying : libgusb-0.3.0-1.el8.x86_64 24/106 + Verifying : libkcapi-1.4.0-2.0.1.el8.x86_64 25/106 + Verifying : libkcapi-hmaccalc-1.4.0-2.0.1.el8.x86_64 26/106 + Verifying : libmodman-2.0.1-17.el8.x86_64 27/106 + Verifying : libpkgconf-1.4.2-1.el8.x86_64 28/106 + Verifying : libproxy-0.4.15-5.2.el8.x86_64 29/106 + Verifying : libsoup-2.62.3-5.el8.x86_64 30/106 + Verifying : lksctp-tools-1.0.18-3.el8.x86_64 31/106 + Verifying : memstrack-0.2.5-2.el8.x86_64 32/106 + Verifying : os-prober-1.74-9.0.1.el8.x86_64 33/106 + Verifying : pigz-2.4-4.el8.x86_64 34/106 + Verifying : pkgconf-1.4.2-1.el8.x86_64 35/106 + Verifying : pkgconf-m4-1.4.2-1.el8.noarch 36/106 + Verifying : pkgconf-pkg-config-1.4.2-1.el8.x86_64 37/106 + Verifying : shared-mime-info-1.9-4.el8.x86_64 38/106 + Verifying : systemd-udev-239-78.0.4.el8.x86_64 39/106 + Verifying : xz-5.2.4-4.el8_6.x86_64 40/106 + Verifying : abattis-cantarell-fonts-0.0.25-6.el8.noarch 41/106 + Verifying : adwaita-cursor-theme-3.28.0-3.el8.noarch 42/106 + Verifying : adwaita-icon-theme-3.28.0-3.el8.noarch 43/106 + Verifying : alsa-lib-1.2.10-2.el8.x86_64 44/106 + Verifying : at-spi2-atk-2.26.2-1.el8.x86_64 45/106 + Verifying : at-spi2-core-2.28.0-1.el8.x86_64 46/106 + Verifying : atk-2.28.1-1.el8.x86_64 47/106 + Verifying : cairo-1.15.12-6.el8.x86_64 48/106 + Verifying : cairo-gobject-1.15.12-6.el8.x86_64 49/106 + Verifying : colord-libs-1.4.2-1.el8.x86_64 50/106 + Verifying : copy-jdk-configs-4.0-2.el8.noarch 51/106 + Verifying : dconf-0.28.0-4.0.1.el8.x86_64 52/106 + Verifying : fribidi-1.0.4-9.el8.x86_64 53/106 + Verifying : gdk-pixbuf2-modules-2.36.12-6.el8_10.x86_64 54/106 + Verifying : graphite2-1.3.10-10.el8.x86_64 55/106 + Verifying : gtk-update-icon-cache-3.22.30-11.el8.x86_64 56/106 + Verifying : gtk3-3.22.30-11.el8.x86_64 57/106 + Verifying : harfbuzz-1.7.5-4.el8.x86_64 58/106 + Verifying : hicolor-icon-theme-0.17-2.el8.noarch 59/106 + Verifying : jasper-libs-2.0.14-5.el8.x86_64 60/106 + Verifying : java-11-openjdk-1:11.0.24.0.8-3.0.1.el8.x86_64 61/106 + Verifying : java-11-openjdk-devel-1:11.0.24.0.8-3.0.1.el8.x8 62/106 + Verifying : java-11-openjdk-headless-1:11.0.24.0.8-3.0.1.el8 63/106 + Verifying : javapackages-filesystem-5.3.0-1.module+el8+5136+ 64/106 + Verifying : jbigkit-libs-2.1-14.el8.x86_64 65/106 + Verifying : lcms2-2.9-2.el8.x86_64 66/106 + Verifying : libX11-1.6.8-8.el8.x86_64 67/106 + Verifying : libX11-common-1.6.8-8.el8.noarch 68/106 + Verifying : libXau-1.0.9-3.el8.x86_64 69/106 + Verifying : libXcomposite-0.4.4-14.el8.x86_64 70/106 + Verifying : libXcursor-1.1.15-3.el8.x86_64 71/106 + Verifying : libXdamage-1.1.4-14.el8.x86_64 72/106 + Verifying : libXext-1.3.4-1.el8.x86_64 73/106 + Verifying : libXfixes-5.0.3-7.el8.x86_64 74/106 + Verifying : libXft-2.3.3-1.el8.x86_64 75/106 + Verifying : libXi-1.7.10-1.el8.x86_64 76/106 + Verifying : libXinerama-1.1.4-1.el8.x86_64 77/106 + Verifying : libXrandr-1.5.2-1.el8.x86_64 78/106 + Verifying : libXrender-0.9.10-7.el8.x86_64 79/106 + Verifying : libXtst-1.2.3-7.el8.x86_64 80/106 + Verifying : libdatrie-0.2.9-7.el8.x86_64 81/106 + Verifying : libepoxy-1.5.8-1.el8.x86_64 82/106 + Verifying : libfontenc-1.1.3-8.el8.x86_64 83/106 + Verifying : libjpeg-turbo-1.5.3-12.el8.x86_64 84/106 + Verifying : libthai-0.1.27-2.el8.x86_64 85/106 + Verifying : libtiff-4.0.9-32.el8_10.x86_64 86/106 + Verifying : libwayland-client-1.21.0-1.el8.x86_64 87/106 + Verifying : libwayland-cursor-1.21.0-1.el8.x86_64 88/106 + Verifying : libwayland-egl-1.21.0-1.el8.x86_64 89/106 + Verifying : libxcb-1.13.1-1.el8.x86_64 90/106 + Verifying : libxkbcommon-0.9.1-1.el8.x86_64 91/106 + Verifying : lua-5.3.4-12.el8.x86_64 92/106 + Verifying : nspr-4.35.0-1.el8_8.x86_64 93/106 + Verifying : nss-3.90.0-7.el8_10.x86_64 94/106 + Verifying : nss-softokn-3.90.0-7.el8_10.x86_64 95/106 + Verifying : nss-softokn-freebl-3.90.0-7.el8_10.x86_64 96/106 + Verifying : nss-sysinit-3.90.0-7.el8_10.x86_64 97/106 + Verifying : nss-util-3.90.0-7.el8_10.x86_64 98/106 + Verifying : pango-1.42.4-8.el8.x86_64 99/106 + Verifying : pixman-0.38.4-4.el8.x86_64 100/106 + Verifying : rest-0.8.1-2.el8.x86_64 101/106 + Verifying : ttmkfdir-3.0.9-54.el8.x86_64 102/106 + Verifying : tzdata-java-2024a-1.0.1.el8.noarch 103/106 + Verifying : xkeyboard-config-2.28-1.el8.noarch 104/106 + Verifying : xorg-x11-font-utils-1:7.5-41.el8.x86_64 105/106 + Verifying : xorg-x11-fonts-Type1-7.5-19.el8.noarch 106/106 Installed: - alsa-lib-1.2.6.1-3.el8.x86_64 - avahi-libs-0.7-20.el8.x86_64 + abattis-cantarell-fonts-0.0.25-6.el8.noarch + adwaita-cursor-theme-3.28.0-3.el8.noarch + adwaita-icon-theme-3.28.0-3.el8.noarch + alsa-lib-1.2.10-2.el8.x86_64 + at-spi2-atk-2.26.2-1.el8.x86_64 + at-spi2-core-2.28.0-1.el8.x86_64 + atk-2.28.1-1.el8.x86_64 + avahi-libs-0.7-27.el8.x86_64 + cairo-1.15.12-6.el8.x86_64 + cairo-gobject-1.15.12-6.el8.x86_64 + colord-libs-1.4.2-1.el8.x86_64 copy-jdk-configs-4.0-2.el8.noarch - crypto-policies-scripts-20211116-1.gitae470d6.el8.noarch - cups-libs-1:2.2.6-45.el8_6.2.x86_64 - giflib-5.1.4-3.el8.x86_64 + cpio-2.12-11.el8.x86_64 + crypto-policies-scripts-20230731-1.git3177e06.el8.noarch + cups-libs-1:2.2.6-60.el8_10.x86_64 + dconf-0.28.0-4.0.1.el8.x86_64 + dejavu-sans-mono-fonts-2.35-7.el8.noarch + dracut-049-233.git20240115.0.1.el8.x86_64 + file-5.33-25.el8.x86_64 + fribidi-1.0.4-9.el8.x86_64 + gdk-pixbuf2-2.36.12-6.el8_10.x86_64 + gdk-pixbuf2-modules-2.36.12-6.el8_10.x86_64 + gettext-0.19.8.1-17.el8.x86_64 + gettext-libs-0.19.8.1-17.el8.x86_64 + glib-networking-2.56.1-1.1.el8.x86_64 graphite2-1.3.10-10.el8.x86_64 - harfbuzz-1.7.5-3.el8.x86_64 - java-11-openjdk-1:11.0.16.1.1-1.el8_6.x86_64 - java-11-openjdk-devel-1:11.0.16.1.1-1.el8_6.x86_64 - java-11-openjdk-headless-1:11.0.16.1.1-1.el8_6.x86_64 + grub2-common-1:2.02-156.0.2.el8.noarch + grub2-tools-1:2.02-156.0.2.el8.x86_64 + grub2-tools-minimal-1:2.02-156.0.2.el8.x86_64 + grubby-8.40-49.0.2.el8.x86_64 + gsettings-desktop-schemas-3.32.0-6.el8.x86_64 + gtk-update-icon-cache-3.22.30-11.el8.x86_64 + gtk3-3.22.30-11.el8.x86_64 + hardlink-1:1.3-6.el8.x86_64 + harfbuzz-1.7.5-4.el8.x86_64 + hicolor-icon-theme-0.17-2.el8.noarch + jasper-libs-2.0.14-5.el8.x86_64 + java-11-openjdk-1:11.0.24.0.8-3.0.1.el8.x86_64 + java-11-openjdk-devel-1:11.0.24.0.8-3.0.1.el8.x86_64 + java-11-openjdk-headless-1:11.0.24.0.8-3.0.1.el8.x86_64 javapackages-filesystem-5.3.0-1.module+el8+5136+7ff78f74.noarch + jbigkit-libs-2.1-14.el8.x86_64 + json-glib-1.4.4-1.el8.x86_64 + kbd-2.0.4-11.el8.x86_64 + kbd-legacy-2.0.4-11.el8.noarch + kbd-misc-2.0.4-11.el8.noarch lcms2-2.9-2.el8.x86_64 - libX11-1.6.8-5.el8.x86_64 - libX11-common-1.6.8-5.el8.noarch + libX11-1.6.8-8.el8.x86_64 + libX11-common-1.6.8-8.el8.noarch libXau-1.0.9-3.el8.x86_64 libXcomposite-0.4.4-14.el8.x86_64 + libXcursor-1.1.15-3.el8.x86_64 + libXdamage-1.1.4-14.el8.x86_64 libXext-1.3.4-1.el8.x86_64 + libXfixes-5.0.3-7.el8.x86_64 + libXft-2.3.3-1.el8.x86_64 libXi-1.7.10-1.el8.x86_64 + libXinerama-1.1.4-1.el8.x86_64 + libXrandr-1.5.2-1.el8.x86_64 libXrender-0.9.10-7.el8.x86_64 libXtst-1.2.3-7.el8.x86_64 + libcroco-0.6.12-4.el8_2.1.x86_64 + libdatrie-0.2.9-7.el8.x86_64 + libepoxy-1.5.8-1.el8.x86_64 libfontenc-1.1.3-8.el8.x86_64 + libgomp-8.5.0-22.0.1.el8_10.x86_64 + libgusb-0.3.0-1.el8.x86_64 libjpeg-turbo-1.5.3-12.el8.x86_64 + libkcapi-1.4.0-2.0.1.el8.x86_64 + libkcapi-hmaccalc-1.4.0-2.0.1.el8.x86_64 + libmodman-2.0.1-17.el8.x86_64 libpkgconf-1.4.2-1.el8.x86_64 + libproxy-0.4.15-5.2.el8.x86_64 + libsoup-2.62.3-5.el8.x86_64 + libthai-0.1.27-2.el8.x86_64 + libtiff-4.0.9-32.el8_10.x86_64 + libwayland-client-1.21.0-1.el8.x86_64 + libwayland-cursor-1.21.0-1.el8.x86_64 + libwayland-egl-1.21.0-1.el8.x86_64 libxcb-1.13.1-1.el8.x86_64 + libxkbcommon-0.9.1-1.el8.x86_64 lksctp-tools-1.0.18-3.el8.x86_64 lua-5.3.4-12.el8.x86_64 - nspr-4.34.0-3.el8_6.x86_64 - nss-3.79.0-10.el8_6.x86_64 - nss-softokn-3.79.0-10.el8_6.x86_64 - nss-softokn-freebl-3.79.0-10.el8_6.x86_64 - nss-sysinit-3.79.0-10.el8_6.x86_64 - nss-util-3.79.0-10.el8_6.x86_64 + memstrack-0.2.5-2.el8.x86_64 + nspr-4.35.0-1.el8_8.x86_64 + nss-3.90.0-7.el8_10.x86_64 + nss-softokn-3.90.0-7.el8_10.x86_64 + nss-softokn-freebl-3.90.0-7.el8_10.x86_64 + nss-sysinit-3.90.0-7.el8_10.x86_64 + nss-util-3.90.0-7.el8_10.x86_64 + os-prober-1.74-9.0.1.el8.x86_64 + pango-1.42.4-8.el8.x86_64 + pigz-2.4-4.el8.x86_64 + pixman-0.38.4-4.el8.x86_64 pkgconf-1.4.2-1.el8.x86_64 pkgconf-m4-1.4.2-1.el8.noarch pkgconf-pkg-config-1.4.2-1.el8.x86_64 + rest-0.8.1-2.el8.x86_64 + shared-mime-info-1.9-4.el8.x86_64 + systemd-udev-239-78.0.4.el8.x86_64 ttmkfdir-3.0.9-54.el8.x86_64 - tzdata-java-2022d-1.el8.noarch + tzdata-java-2024a-1.0.1.el8.noarch + xkeyboard-config-2.28-1.el8.noarch xorg-x11-font-utils-1:7.5-41.el8.x86_64 xorg-x11-fonts-Type1-7.5-19.el8.noarch + xz-5.2.4-4.el8_6.x86_64 Complete! -Last metadata expiration check: 0:00:10 ago on Mon 10 Oct 2022 04:06:28 PM UTC. +Last metadata expiration check: 0:00:23 ago on Tue 20 Aug 2024 08:55:14 AM UTC. +Package iproute-6.2.0-5.el8_9.x86_64 is already installed. Dependencies resolved. -============================================================================================== - Package - Arch Version Repository Size -============================================================================================== -Installing: - ords noarch 22.3.0-7.el8 yum.oracle.com_repo_OracleLinux_OL8_oracle_software_x86_64 87 M -Installing dependencies: - lsof x86_64 4.93.2-1.el8 ol8_baseos_latest 253 k +================================================================================ + Package Architecture Version Repository Size +================================================================================ +Upgrading: + iproute x86_64 6.2.0-6.el8_10 ol8_baseos_latest 853 k Transaction Summary -============================================================================================== -Install 2 Packages +================================================================================ +Upgrade 1 Package -Total download size: 87 M -Installed size: 92 M +Total download size: 853 k Downloading Packages: -(1/2): lsof-4.93.2-1.el8.x86_64.rpm 3.0 MB/s | 253 kB 00:00 -(2/2): ords-22.3.0-7.el8.noarch.rpm 66 MB/s | 87 MB 00:01 +iproute-6.2.0-6.el8_10.x86_64.rpm 4.2 MB/s | 853 kB 00:00 -------------------------------------------------------------------------------- -Total 66 MB/s | 87 MB 00:01 +Total 4.2 MB/s | 853 kB 00:00 Running transaction check Transaction check succeeded. Running transaction test Transaction test succeeded. Running transaction Preparing : 1/1 - Installing : lsof-4.93.2-1.el8.x86_64 1/2 - Running scriptlet: ords-22.3.0-7.el8.noarch 2/2 - Installing : ords-22.3.0-7.el8.noarch 2/2 - Running scriptlet: ords-22.3.0-7.el8.noarch 2/2 -INFO: Before starting ORDS service, run the below command as user oracle: - ords --config /etc/ords/config install + Upgrading : iproute-6.2.0-6.el8_10.x86_64 1/2 + Cleanup : iproute-6.2.0-5.el8_9.x86_64 2/2 + Running scriptlet: iproute-6.2.0-5.el8_9.x86_64 2/2 + Verifying : iproute-6.2.0-6.el8_10.x86_64 1/2 + Verifying : iproute-6.2.0-5.el8_9.x86_64 2/2 - Verifying : lsof-4.93.2-1.el8.x86_64 1/2 - Verifying : ords-22.3.0-7.el8.noarch 2/2 - -Installed: - lsof-4.93.2-1.el8.x86_64 ords-22.3.0-7.el8.noarch +Upgraded: + iproute-6.2.0-6.el8_10.x86_64 -Complete! -Last metadata expiration check: 0:00:15 ago on Mon 10 Oct 2022 04:06:28 PM UTC. -Package iproute-5.15.0-4.el8_6.1.x86_64 is already installed. -Dependencies resolved. -Nothing to do. Complete! 24 files removed -Removing intermediate container d38a69d2cc70 - ---> 3a7b8edb327e -Step 5/10 : RUN mkdir -p $ORDS_HOME/doc_root && mkdir -p $ORDS_HOME/error && mkdir -p $ORDS_HOME/secrets && chmod ug+x $ORDS_HOME/*.sh && groupadd -g 54322 dba && usermod -u 54321 -d /home/oracle -g dba -m -s /bin/bash oracle && chown -R oracle:dba $ORDS_HOME && echo "oracle ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers - ---> Running in 1d05951f8252 -Removing intermediate container 1d05951f8252 - ---> 265cb7ab4f2c -Step 6/10 : USER oracle - ---> Running in 180d432ae42d -Removing intermediate container 180d432ae42d - ---> a9caee3d9426 -Step 7/10 : WORKDIR /home/oracle - ---> Running in bf8ac95c724a -Removing intermediate container bf8ac95c724a - ---> 4623d696e603 -Step 8/10 : VOLUME ["$ORDS_HOME/config/ords"] - ---> Running in 3afce627e4c0 -Removing intermediate container 3afce627e4c0 - ---> 914d4ee42ede -Step 9/10 : EXPOSE 8888 - ---> Running in 13460b132c52 -Removing intermediate container 13460b132c52 - ---> 4c9edba5aade -Step 10/10 : CMD $ORDS_HOME/$RUN_FILE - ---> Running in f97b17d8cea4 -Removing intermediate container f97b17d8cea4 - ---> c8e95aadf5e3 -Successfully built c8e95aadf5e3 +Removing intermediate container fe168b01f3ad + ---> 791878694a50 +Step 5/12 : RUN curl -o /tmp/ords-$ORDSVERSION.el8.noarch.rpm https://yum.oracle.com/repo/OracleLinux/OL8/oracle/software/x86_64/getPackage/ords-$ORDSVERSION.el8.noarch.rpm + ---> Running in 59d7143da358 + % Total % Received % Xferd Average Speed Time Time Time Current + Dload Upload Total Spent Left Speed +100 108M 100 108M 0 0 1440k 0 0:01:16 0:01:16 --:--:-- 1578k +Removing intermediate container 59d7143da358 + ---> 17c4534293e5 +Step 6/12 : RUN rpm -ivh /tmp/ords-$ORDSVERSION.el8.noarch.rpm + ---> Running in 84b1cbffdc51 +Verifying... ######################################## +Preparing... ######################################## +Updating / installing... +ords-23.4.0-8.el8 ######################################## +INFO: Before starting ORDS service, run the below command as user oracle: + ords --config /etc/ords/config install +Removing intermediate container 84b1cbffdc51 + ---> 6e7151b79588 +Step 7/12 : RUN mkdir -p $ORDS_HOME/doc_root && mkdir -p $ORDS_HOME/error && mkdir -p $ORDS_HOME/secrets && chmod ug+x $ORDS_HOME/*.sh && groupadd -g 54322 dba && usermod -u 54321 -d /home/oracle -g dba -m -s /bin/bash oracle && chown -R oracle:dba $ORDS_HOME && echo "oracle ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers + ---> Running in 66e5db5f343f +Removing intermediate container 66e5db5f343f + ---> 0523dc897bf4 +Step 8/12 : USER oracle + ---> Running in ffda8495ac77 +Removing intermediate container ffda8495ac77 + ---> 162acd4d0b93 +Step 9/12 : WORKDIR /home/oracle + ---> Running in 8c14310ffbc7 +Removing intermediate container 8c14310ffbc7 + ---> c8dae809e772 +Step 10/12 : VOLUME ["$ORDS_HOME/config/ords"] + ---> Running in ed64548fd997 +Removing intermediate container ed64548fd997 + ---> 22e2c99247b0 +Step 11/12 : EXPOSE 8888 + ---> Running in 921f7c85d61d +Removing intermediate container 921f7c85d61d + ---> e5d503c92224 +Step 12/12 : CMD $ORDS_HOME/$RUN_FILE + ---> Running in cad487298d63 +Removing intermediate container cad487298d63 + ---> fdb17aa242f8 +Successfully built fdb17aa242f8 Successfully tagged oracle/ords-dboper:latest +08:57:18 oracle@mitk01:# diff --git a/docs/multitenant/usecase01/logfiles/cdb_creation.log b/docs/multitenant/usecase01/logfiles/cdb_creation.log new file mode 100644 index 00000000..b4602f54 --- /dev/null +++ b/docs/multitenant/usecase01/logfiles/cdb_creation.log @@ -0,0 +1,357 @@ +/usr/local/go/bin/kubectl logs -f `/usr/local/go/bin/kubectl get pods -n oracle-database-operator-system|grep ords|cut -d ' ' -f 1` -n oracle-database-operator-system +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. +ORDSVERSIN:23.4.0-8 +NOT_INSTALLED=2 + SETUP +==================================================== +CONFIG=/etc/ords/config +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 23.4 Production on Tue Aug 20 07:21:16 2024 + +Copyright (c) 2010, 2024, Oracle. + +Configuration: + /etc/ords/config/ + +The setting named: db.connectionType was set to: customurl in configuration: default +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 23.4 Production on Tue Aug 20 07:21:18 2024 + +Copyright (c) 2010, 2024, Oracle. + +Configuration: + /etc/ords/config/ + +The setting named: db.customURL was set to: jdbc:oracle:thin:@(DESCRIPTION=(CONNECT_TIMEOUT=90)(RETRY_COUNT=30)(RETRY_DELAY=10)(TRANSPORT_CONNECT_TIMEOUT=70)(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan12.testrac.com)(PORT=1521)(IP=V4_ONLY))(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan34.testrac.com)(PORT=1521)(IP=V4_ONLY))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=TESTORDS))) in configuration: default +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 23.4 Production on Tue Aug 20 07:21:20 2024 + +Copyright (c) 2010, 2024, Oracle. + +Configuration: + /etc/ords/config/ + +The setting named: security.requestValidationFunction was set to: false in configuration: default +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 23.4 Production on Tue Aug 20 07:21:22 2024 + +Copyright (c) 2010, 2024, Oracle. + +Configuration: + /etc/ords/config/ + +The setting named: jdbc.MaxLimit was set to: 100 in configuration: default +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 23.4 Production on Tue Aug 20 07:21:24 2024 + +Copyright (c) 2010, 2024, Oracle. + +Configuration: + /etc/ords/config/ + +The setting named: jdbc.InitialLimit was set to: 50 in configuration: default +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 23.4 Production on Tue Aug 20 07:21:25 2024 + +Copyright (c) 2010, 2024, Oracle. + +Configuration: + /etc/ords/config/ + +The global setting named: error.externalPath was set to: /opt/oracle/ords/error +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 23.4 Production on Tue Aug 20 07:21:27 2024 + +Copyright (c) 2010, 2024, Oracle. + +Configuration: + /etc/ords/config/ + +The global setting named: standalone.access.log was set to: /home/oracle +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 23.4 Production on Tue Aug 20 07:21:29 2024 + +Copyright (c) 2010, 2024, Oracle. + +Configuration: + /etc/ords/config/ + +The global setting named: standalone.https.port was set to: 8888 +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 23.4 Production on Tue Aug 20 07:21:31 2024 + +Copyright (c) 2010, 2024, Oracle. + +Configuration: + /etc/ords/config/ + +The global setting named: standalone.https.cert was set to: /opt/oracle/ords//secrets/tls.crt +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 23.4 Production on Tue Aug 20 07:21:33 2024 + +Copyright (c) 2010, 2024, Oracle. + +Configuration: + /etc/ords/config/ + +The global setting named: standalone.https.cert.key was set to: /opt/oracle/ords//secrets/tls.key +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 23.4 Production on Tue Aug 20 07:21:35 2024 + +Copyright (c) 2010, 2024, Oracle. + +Configuration: + /etc/ords/config/ + +The setting named: restEnabledSql.active was set to: true in configuration: default +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 23.4 Production on Tue Aug 20 07:21:37 2024 + +Copyright (c) 2010, 2024, Oracle. + +Configuration: + /etc/ords/config/ + +The global setting named: security.verifySSL was set to: true +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 23.4 Production on Tue Aug 20 07:21:39 2024 + +Copyright (c) 2010, 2024, Oracle. + +Configuration: + /etc/ords/config/ + +The global setting named: database.api.enabled was set to: true +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 23.4 Production on Tue Aug 20 07:21:41 2024 + +Copyright (c) 2010, 2024, Oracle. + +Configuration: + /etc/ords/config/ + +The setting named: plsql.gateway.mode was set to: disabled in configuration: default +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 23.4 Production on Tue Aug 20 07:21:43 2024 + +Copyright (c) 2010, 2024, Oracle. + +Configuration: + /etc/ords/config/ + +The global setting named: database.api.management.services.disabled was set to: false +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 23.4 Production on Tue Aug 20 07:21:45 2024 + +Copyright (c) 2010, 2024, Oracle. + +Configuration: + /etc/ords/config/ + +The setting named: misc.pagination.maxRows was set to: 1000 in configuration: default +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 23.4 Production on Tue Aug 20 07:21:47 2024 + +Copyright (c) 2010, 2024, Oracle. + +Configuration: + /etc/ords/config/ + +The setting named: db.cdb.adminUser was set to: C##DBAPI_CDB_ADMIN AS SYSDBA in configuration: default +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 23.4 Production on Tue Aug 20 07:21:49 2024 + +Copyright (c) 2010, 2024, Oracle. + +Configuration: + /etc/ords/config/ + +The setting named: db.cdb.adminUser.password was set to: ****** in configuration: default +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 23.4 Production on Tue Aug 20 07:21:51 2024 + +Copyright (c) 2010, 2024, Oracle. + +Configuration: + /etc/ords/config/ + +Created user welcome in file /etc/ords/config/global/credentials +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 23.4 Production on Tue Aug 20 07:21:53 2024 + +Copyright (c) 2010, 2024, Oracle. + +Configuration: + /etc/ords/config/ + +Oracle REST Data Services - Non-Interactive Install + +Retrieving information... +Completed verifying Oracle REST Data Services schema version 23.4.0.r3461619. +Connecting to database user: ORDS_PUBLIC_USER url: jdbc:oracle:thin:@(DESCRIPTION=(CONNECT_TIMEOUT=90)(RETRY_COUNT=30)(RETRY_DELAY=10)(TRANSPORT_CONNECT_TIMEOUT=70)(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan12.testrac.com)(PORT=1521)(IP=V4_ONLY))(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan34.testrac.com)(PORT=1521)(IP=V4_ONLY))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=TESTORDS))) +The setting named: db.serviceNameSuffix was set to: in configuration: default +The setting named: db.username was set to: ORDS_PUBLIC_USER in configuration: default +The setting named: db.password was set to: ****** in configuration: default +The setting named: security.requestValidationFunction was set to: ords_util.authorize_plsql_gateway in configuration: default +2024-08-20T07:21:57.563Z INFO Oracle REST Data Services schema version 23.4.0.r3461619 is installed. +2024-08-20T07:21:57.565Z INFO To run in standalone mode, use the ords serve command: +2024-08-20T07:21:57.565Z INFO ords --config /etc/ords/config serve +2024-08-20T07:21:57.565Z INFO Visit the ORDS Documentation to access tutorials, developer guides and more to help you get started with the new ORDS Command Line Interface (http://oracle.com/rest). +Picked up _JAVA_OPTIONS: -Xms1126M -Xmx1126M + +ORDS: Release 23.4 Production on Tue Aug 20 07:21:59 2024 + +Copyright (c) 2010, 2024, Oracle. + +Configuration: + /etc/ords/config/ + +2024-08-20T07:21:59.739Z INFO HTTP and HTTP/2 cleartext listening on host: 0.0.0.0 port: 8080 +2024-08-20T07:21:59.741Z INFO HTTPS and HTTPS/2 listening on host: 0.0.0.0 port: 8888 +2024-08-20T07:21:59.765Z INFO Disabling document root because the specified folder does not exist: /etc/ords/config/global/doc_root +2024-08-20T07:21:59.765Z INFO Default forwarding from / to contextRoot configured. +2024-08-20T07:22:05.313Z INFO Configuration properties for: |default|lo| +db.serviceNameSuffix= +java.specification.version=22 +conf.use.wallet=true +database.api.management.services.disabled=false +sun.jnu.encoding=UTF-8 +user.region=US +java.class.path=/opt/oracle/ords/ords.war +java.vm.vendor=Oracle Corporation +standalone.https.cert.key=/opt/oracle/ords//secrets/tls.key +sun.arch.data.model=64 +nashorn.args=--no-deprecation-warning +java.vendor.url=https://java.oracle.com/ +resource.templates.enabled=false +user.timezone=UTC +java.vm.specification.version=22 +os.name=Linux +sun.java.launcher=SUN_STANDARD +user.country=US +sun.boot.library.path=/usr/java/jdk-22/lib +sun.java.command=/opt/oracle/ords/ords.war --config /etc/ords/config serve --port 8888 --secure +jdk.debug=release +sun.cpu.endian=little +user.home=/home/oracle +oracle.dbtools.launcher.executable.jar.path=/opt/oracle/ords/ords.war +user.language=en +db.cdb.adminUser.password=****** +java.specification.vendor=Oracle Corporation +java.version.date=2024-07-16 +database.api.enabled=true +java.home=/usr/java/jdk-22 +db.username=ORDS_PUBLIC_USER +file.separator=/ +java.vm.compressedOopsMode=32-bit +line.separator= + +restEnabledSql.active=true +java.specification.name=Java Platform API Specification +java.vm.specification.vendor=Oracle Corporation +java.awt.headless=true +standalone.https.cert=/opt/oracle/ords//secrets/tls.crt +db.password=****** +sun.management.compiler=HotSpot 64-Bit Tiered Compilers +security.requestValidationFunction=ords_util.authorize_plsql_gateway +misc.pagination.maxRows=1000 +java.runtime.version=22.0.2+9-70 +user.name=oracle +error.externalPath=/opt/oracle/ords/error +stdout.encoding=UTF-8 +path.separator=: +db.cdb.adminUser=C##DBAPI_CDB_ADMIN AS SYSDBA +os.version=5.4.17-2136.329.3.1.el7uek.x86_64 +java.runtime.name=Java(TM) SE Runtime Environment +file.encoding=UTF-8 +plsql.gateway.mode=disabled +security.verifySSL=true +standalone.https.port=8888 +java.vm.name=Java HotSpot(TM) 64-Bit Server VM +java.vendor.url.bug=https://bugreport.java.com/bugreport/ +java.io.tmpdir=/tmp +oracle.dbtools.cmdline.ShellCommand=ords +java.version=22.0.2 +user.dir=/home/oracle +os.arch=amd64 +java.vm.specification.name=Java Virtual Machine Specification +jdbc.MaxLimit=100 +oracle.dbtools.cmdline.home=/opt/oracle/ords +native.encoding=UTF-8 +java.library.path=/usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib +java.vendor=Oracle Corporation +java.vm.info=mixed mode, sharing +stderr.encoding=UTF-8 +java.vm.version=22.0.2+9-70 +sun.io.unicode.encoding=UnicodeLittle +jdbc.InitialLimit=50 +db.connectionType=customurl +java.class.version=66.0 +db.customURL=jdbc:oracle:thin:@(DESCRIPTION=(CONNECT_TIMEOUT=90)(RETRY_COUNT=30)(RETRY_DELAY=10)(TRANSPORT_CONNECT_TIMEOUT=70)(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan12.testrac.com)(PORT=1521)(IP=V4_ONLY))(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan34.testrac.com)(PORT=1521)(IP=V4_ONLY))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=TESTORDS))) +standalone.access.log=/home/oracle + +2024-08-20T07:22:09.268Z INFO + +Mapped local pools from /etc/ords/config/databases: + /ords/ => default => VALID + + +2024-08-20T07:22:09.414Z INFO Oracle REST Data Services initialized +Oracle REST Data Services version : 23.4.0.r3461619 +Oracle REST Data Services server info: jetty/10.0.18 +Oracle REST Data Services java info: Java HotSpot(TM) 64-Bit Server VM 22.0.2+9-70 + diff --git a/docs/multitenant/usecase01/logfiles/openssl_execution.log b/docs/multitenant/usecase01/logfiles/openssl_execution.log index 30a1c5d4..e3915a21 100644 --- a/docs/multitenant/usecase01/logfiles/openssl_execution.log +++ b/docs/multitenant/usecase01/logfiles/openssl_execution.log @@ -1,22 +1,19 @@ +CREATING TLS CERTIFICATES /usr/bin/openssl genrsa -out ca.key 2048 -Generating RSA private key, 2048 bit long modulus -......................................................................................................................................................................................+++ -...................................+++ -e is 65537 (0x10001) -/usr/bin/openssl req -new -x509 -days 365 -key ca.key -subj "/C=CN/ST=GD/L=SZ/O=oracle, Inc./CN=oracle Root CA" -out ca.crt -/usr/bin/openssl req -newkey rsa:2048 -nodes -keyout tls.key -subj "/C=CN/ST=GD/L=SZ/O=oracle, Inc./CN=cdb-dev-ords" -out server.csr -Generating a 2048 bit RSA private key -...................................+++ -........................................+++ +Generating RSA private key, 2048 bit long modulus (2 primes) +......................+++++ +..................................................+++++ +e is 65537 (0x010001) +/usr/bin/openssl req -new -x509 -days 365 -key ca.key -subj "/C=US/ST=California/L=SanFrancisco/O=oracle /CN=cdb-dev-ords.oracle-database-operator-system /CN=localhost Root CA " -out ca.crt +/usr/bin/openssl req -newkey rsa:2048 -nodes -keyout tls.key -subj "/C=US/ST=California/L=SanFrancisco/O=oracle /CN=cdb-dev-ords.oracle-database-operator-system /CN=localhost" -out server.csr +Generating a RSA private key +...........+++++ +...........................................+++++ writing new private key to 'tls.key' ----- -/usr/bin/echo "subjectAltName=DNS:cdb-dev-ords,DNS:www.example.com" > extfile.txt +/usr/bin/echo "subjectAltName=DNS:cdb-dev-ords.oracle-database-operator-system,DNS:www.example.com" > extfile.txt /usr/bin/openssl x509 -req -extfile extfile.txt -days 365 -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out tls.crt Signature ok -subject=/C=CN/ST=GD/L=SZ/O=oracle, Inc./CN=cdb-dev-ords +subject=C = US, ST = California, L = SanFrancisco, O = "oracle ", CN = "cdb-dev-ords.oracle-database-operator-system ", CN = localhost Getting CA Private Key -/usr/bin/kubectl create secret tls db-tls --key="tls.key" --cert="tls.crt" -n oracle-database-operator-system -secret/db-tls created -/usr/bin/kubectl create secret generic db-ca --from-file="ca.crt" -n oracle-database-operator-system -secret/db-ca created diff --git a/docs/multitenant/usecase01/logfiles/ordsconfig.log b/docs/multitenant/usecase01/logfiles/ordsconfig.log index ad5e7bab..b787b752 100644 --- a/docs/multitenant/usecase01/logfiles/ordsconfig.log +++ b/docs/multitenant/usecase01/logfiles/ordsconfig.log @@ -1,35 +1,39 @@ -: Release 22.3 Production on Tue Oct 11 12:51:50 2022 +ORDS: Release 23.4 Production on Tue Aug 20 07:48:44 2024 -Copyright (c) 2010, 2022, Oracle. +Copyright (c) 2010, 2024, Oracle. Configuration: /etc/ords/config/ Database pool: default -Setting Value Source ------------------------------------------ -------------------------------------- ----------- -database.api.enabled true Global -database.api.management.services.disabled false Global -db.cdb.adminUser C##DBAPI_CDB_ADMIN AS SYSDBA Pool -db.cdb.adminUser.password ****** Pool Wallet -db.connectionType basic Pool -db.hostname racnode1.testrac.com Pool -db.password ****** Pool Wallet -db.port 1521 Pool -db.serviceNameSuffix Pool -db.servicename TESTORDS Pool -db.username ORDS_PUBLIC_USER Pool -error.externalPath /opt/oracle/ords/error Global -jdbc.InitialLimit 50 Pool -jdbc.MaxLimit 100 Pool -misc.pagination.maxRows 1000 Pool -plsql.gateway.mode proxied Pool -restEnabledSql.active true Pool -security.requestValidationFunction wwv_flow_epg_include_modules.authorize Pool -security.verifySSL true Global -standalone.access.log /home/oracle Global -standalone.https.cert /opt/oracle/ords//secrets/tls.crt Global -standalone.https.cert.key /opt/oracle/ords//secrets/tls.key Global -standalone.https.port 8888 Global +Setting Value Source +----------------------------------------- -------------------------------------------------- ----------- +database.api.enabled true Global +database.api.management.services.disabled false Global +db.cdb.adminUser C##DBAPI_CDB_ADMIN AS SYSDBA Pool +db.cdb.adminUser.password ****** Pool Wallet +db.connectionType customurl Pool +db.customURL jdbc:oracle:thin:@(DESCRIPTION=(CONNECT_TIMEOUT=90 Pool + )(RETRY_COUNT=30)(RETRY_DELAY=10)(TRANSPORT_CONNEC + T_TIMEOUT=70)(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL= + TCP)(HOST=scan12.testrac.com)(PORT=1521)(IP=V4_ONL + Y))(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST= + scan34.testrac.com)(PORT=1521)(IP=V4_ONLY))(CONNEC + T_DATA=(SERVER=DEDICATED)(SERVICE_NAME=TESTORDS))) +db.password ****** Pool Wallet +db.serviceNameSuffix Pool +db.username ORDS_PUBLIC_USER Pool +error.externalPath /opt/oracle/ords/error Global +jdbc.InitialLimit 50 Pool +jdbc.MaxLimit 100 Pool +misc.pagination.maxRows 1000 Pool +plsql.gateway.mode disabled Pool +restEnabledSql.active true Pool +security.requestValidationFunction ords_util.authorize_plsql_gateway Pool +security.verifySSL true Global +standalone.access.log /home/oracle Global +standalone.https.cert /opt/oracle/ords//secrets/tls.crt Global +standalone.https.cert.key /opt/oracle/ords//secrets/tls.key Global +standalone.https.port 8888 Global diff --git a/docs/multitenant/usecase01/logfiles/tagandpush.log b/docs/multitenant/usecase01/logfiles/tagandpush.log new file mode 100644 index 00000000..232d5bb2 --- /dev/null +++ b/docs/multitenant/usecase01/logfiles/tagandpush.log @@ -0,0 +1,14 @@ +/usr/bin/docker tag oracle/ords-dboper:latest [.......]/ords-dboper:latest + +/usr/bin/docker push [your container registry]/ords-dboper:latest +The push refers to repository [your container registry] +0405aac3af1c: Pushed +6be46e8e1e21: Pushed +c9884830a66d: Pushed +a46244557bb9: Pushing [===========================> ] 261.8MB/469.9MB +f988845e261e: Pushed +fe07ec0b1f5a: Layer already exists +2ac63de5f950: Layer already exists +386cd7a64c01: Layer already exists +826c69252b8b: Layer already exists + diff --git a/docs/multitenant/usecase01/logfiles/testapi.log b/docs/multitenant/usecase01/logfiles/testapi.log index 4c95b457..cb42ecc3 100644 --- a/docs/multitenant/usecase01/logfiles/testapi.log +++ b/docs/multitenant/usecase01/logfiles/testapi.log @@ -1,6 +1,7 @@ -* Trying 127.0.0.1... +kubectl exec -it `kubectl get pods -n oracle-database-operator-system|grep ords|cut -d ' ' -f 1` -n oracle-database-operator-system -i -t -- /usr/bin/curl -sSkv -k -X GET https://localhost:8888/ords/_/db-api/stable/metadata-catalog/ +* Trying ::1... * TCP_NODELAY set -* Connected to localhost (127.0.0.1) port 8888 (#0) +* Connected to localhost (::1) port 8888 (#0) * ALPN, offering h2 * ALPN, offering http/1.1 * successfully set certificate verify locations: @@ -19,10 +20,10 @@ * SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384 * ALPN, server accepted to use h2 * Server certificate: -* subject: C=CN; ST=GD; L=SZ; O=oracle, Inc.; CN=cdb-dev-ords -* start date: Oct 11 07:44:38 2022 GMT -* expire date: Oct 11 07:44:38 2023 GMT -* issuer: C=CN; ST=GD; L=SZ; O=oracle, Inc.; CN=oracle Root CA +* subject: C=US; ST=California; L=SanFrancisco; O=oracle ; CN=cdb-dev-ords.oracle-database-operator-system ; CN=localhost +* start date: Aug 20 07:14:04 2024 GMT +* expire date: Aug 20 07:14:04 2025 GMT +* issuer: C=US; ST=California; L=SanFrancisco; O=oracle ; CN=cdb-dev-ords.oracle-database-operator-system ; CN=localhost Root CA * SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway. * Using HTTP2, server supports multi-use * Connection state changed (HTTP/2 confirmed) @@ -30,7 +31,7 @@ * TLSv1.3 (OUT), TLS app data, [no content] (0): * TLSv1.3 (OUT), TLS app data, [no content] (0): * TLSv1.3 (OUT), TLS app data, [no content] (0): -* Using Stream ID: 1 (easy handle 0x564be7b0b970) +* Using Stream ID: 1 (easy handle 0x55d14a7dea90) * TLSv1.3 (OUT), TLS app data, [no content] (0): > GET /ords/_/db-api/stable/metadata-catalog/ HTTP/2 > Host: localhost:8888 @@ -46,4 +47,16 @@ * TLSv1.3 (OUT), TLS app data, [no content] (0): * TLSv1.3 (IN), TLS app data, [no content] (0): * TLSv1.3 (IN), TLS app data, [no content] (0): - +< HTTP/2 200 +< content-type: application/json +< +* TLSv1.3 (IN), TLS handshake, [no content] (0): +* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4): +* TLSv1.3 (IN), TLS app data, [no content] (0): +* TLSv1.3 (IN), TLS app data, [no content] (0): +* TLSv1.3 (IN), TLS app data, [no content] (0): +* TLSv1.3 (IN), TLS app data, [no content] (0): +* TLSv1.3 (IN), TLS app data, [no content] (0): +* TLSv1.3 (IN), TLS app data, [no content] (0): +* Connection #0 to host localhost left intact +{"items":[{"name":"default","links":[{"rel":"canonical","href":"https://localhost:8888/ords/_/db-api/stable/metadata-catalog/openapi.json","mediaType":"application/vnd.oai.openapi+json;version=3.0"}]}],"links":[{"rel":"self","href":"https://localhost:8888/ords/_/db-api/stable/metadata-catalog/"},{"rel":"describes","href":"https://localhost:8888/ords/_/db-api/stable/"}]} diff --git a/docs/multitenant/usecase01/makefile b/docs/multitenant/usecase01/makefile index e468ef62..d4176c75 100644 --- a/docs/multitenant/usecase01/makefile +++ b/docs/multitenant/usecase01/makefile @@ -109,6 +109,8 @@ # | Before testing step13 delete the crd: | # | kubectl delete pdb pdb1 -n oracle-database-operator-system | # +---------------------------------------------------------------------------+ +# |step14 | delete pdb | +# +-----------------------------+---------------------------------------------+ # | DIAGNOSTIC TARGETS | # +-----------------------------+---------------------------------------------+ # | dump | Dump pods info into a file | @@ -137,6 +139,7 @@ ORDSPORT=8888 MAKE=/usr/bin/make DOCKERFILE=../../../ords/Dockerfile RUNSCRIPT=../../../ords/runOrdsSSL.sh +ORDSIMGDIR=../../../ords RM=/usr/bin/rm CP=/usr/bin/cp ECHO=/usr/bin/echo @@ -170,26 +173,13 @@ step10: pdb step11: close step12: open step13: map +step14: delete checkstep9: checkcdb createimage: - $(CP) $(DOCKERFILE) . - $(CP) $(RUNSCRIPT) . - @echo "BUILDING CDB IMAGES" - @if [[ ! -f ./Dockerfile ]]; \ - then\ - echo "DOCKERFILE DOES NOT EXISTS";\ - exit 1; \ - fi; - @if [[ ! -f ./runOrdsSSL.sh ]]; \ - then\ - echo "DOCKERFILE DOES NOT EXISTS";\ - exit 1; \ - fi; - $(DOCKER) build -t $(IMAGE) . - $(RM) ./Dockerfile ./runOrdsSSL.sh + $(DOCKER) build -t $(IMAGE) $(ORDSIMGDIR) tagimage: @echo "TAG IMAGE" @@ -217,9 +207,9 @@ dboperator: tlscert: @echo "CREATING TLS CERTIFICATES" $(OPENSSL) genrsa -out ca.key 2048 - $(OPENSSL) req -new -x509 -days 365 -key ca.key -subj "/C=US/ST=California/L=SanFrancisco/O=$(COMPANY) /CN=$(RESTPREFIX)-$(REST_SERVER) /CN=$(LOCALHOST) Root CA " -out ca.crt - $(OPENSSL) req -newkey rsa:2048 -nodes -keyout $(SKEY) -subj "/C=US/ST=California/L=SanFrancisco/O=$(COMPANY) /CN=$(RESTPREFIX)-$(REST_SERVER) /CN=$(LOCALHOST)" -out server.csr - $(ECHO) "subjectAltName=DNS:cdb-dev-$(REST_SERVER),DNS:www.example.com" > extfile.txt + $(OPENSSL) req -new -x509 -days 365 -key ca.key -subj "/C=US/ST=California/L=SanFrancisco/O=$(COMPANY) /CN=$(RESTPREFIX)-$(REST_SERVER).$(NAMESPACE) /CN=$(LOCALHOST) Root CA " -out ca.crt + $(OPENSSL) req -newkey rsa:2048 -nodes -keyout $(SKEY) -subj "/C=US/ST=California/L=SanFrancisco/O=$(COMPANY) /CN=$(RESTPREFIX)-$(REST_SERVER).$(NAMESPACE) /CN=$(LOCALHOST)" -out server.csr + $(ECHO) "subjectAltName=DNS:$(RESTPREFIX)-$(REST_SERVER).$(NAMESPACE),DNS:www.example.com" > extfile.txt $(OPENSSL) x509 -req -extfile extfile.txt -days 365 -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out $(SCRT) tlssecret: @@ -255,6 +245,9 @@ map: checkpdb: $(KUBECTL) get pdbs -n $(NAMESPACE) +delete: + $(KUBECTL) apply -f pdb_delete.yaml + dump: @$(eval TMPSP := $(shell date "+%y%m%d%H%M%S" )) @$(eval DIAGFILE := ./opdmp.$(TMPSP)) diff --git a/docs/multitenant/usecase01/pdb_create.yaml b/docs/multitenant/usecase01/pdb_create.yaml index 7953118f..be3581ad 100644 --- a/docs/multitenant/usecase01/pdb_create.yaml +++ b/docs/multitenant/usecase01/pdb_create.yaml @@ -43,4 +43,5 @@ spec: totalSize: "1G" tempSize: "100M" action: "Create" + assertivePdbDeletion: true diff --git a/docs/multitenant/usecase01/pdb_delete.yaml b/docs/multitenant/usecase01/pdb_delete.yaml new file mode 100644 index 00000000..c22b546a --- /dev/null +++ b/docs/multitenant/usecase01/pdb_delete.yaml @@ -0,0 +1,34 @@ +apiVersion: database.oracle.com/v1alpha1 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "oracle-database-operator-system" + pdbName: "pdbdev" + action: "Delete" + dropAction: "INCLUDING" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "pdb1-secret" + key: "webserver_user" + webServerPwd: + secret: + secretName: "pdb1-secret" + key: "webserver_pwd" + diff --git a/docs/multitenant/usecase01/pdb_map.yaml b/docs/multitenant/usecase01/pdb_map.yaml index cd6d4ffb..3300a7fa 100644 --- a/docs/multitenant/usecase01/pdb_map.yaml +++ b/docs/multitenant/usecase01/pdb_map.yaml @@ -42,3 +42,4 @@ spec: totalSize: "1G" tempSize: "100M" action: "Map" + assertivePdbDeletion: true diff --git a/docs/multitenant/usecase01/server.csr b/docs/multitenant/usecase01/server.csr new file mode 100644 index 00000000..e308d301 --- /dev/null +++ b/docs/multitenant/usecase01/server.csr @@ -0,0 +1,18 @@ +-----BEGIN CERTIFICATE REQUEST----- +MIIC3TCCAcUCAQAwgZcxCzAJBgNVBAYTAlVTMRMwEQYDVQQIDApDYWxpZm9ybmlh +MRUwEwYDVQQHDAxTYW5GcmFuY2lzY28xEDAOBgNVBAoMB29yYWNsZSAxNjA0BgNV +BAMMLWNkYi1kZXYtb3Jkcy5vcmFjbGUtZGF0YWJhc2Utb3BlcmF0b3Itc3lzdGVt +IDESMBAGA1UEAwwJbG9jYWxob3N0MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIB +CgKCAQEAm9nlNSQNsPTVqH57MkWKZEyaVtzVKQ8Z3oDK6hWXfB24p0jVj6sTOJkf +NVAxnqmU8DpW3odpbU6qWe/n+B5vJpqdXUGdsq9NKyus2fGb/xf1UnskpA2FUuWZ +o3upyCFxDAOvE4eZUzlxIn+54XXaNAdQiU9E8VXPr5YxrvZ15T/xCXLtJPs/RCOF +cJ8+gvZGcjMbdP16auJDVWZzBaur3eKbiHN7LXNCCRzGO++dv0kGY8vH7MyFfgp3 +qYBiSHS3WDiFUJjYIvfa8lLfP1hnlCyHn8TnU9gjGjmd1YcccSKqWIAT24wPUKVU +Lme4n91jxDPp7g8nRtDw0Smj9gYCtQIDAQABoAAwDQYJKoZIhvcNAQELBQADggEB +AGOG/9IJJRvT2JLcuzE5Arai1XHc6Jh65iuDRqXQav47Bz38FFF2gZNO69gzDmhq +6k7tie+5bPcAHuuJZ0dAa71a9SLjKl+XNkkI0vS6te6OK3DCVUoMqNCk5VdwrJw0 +RORbKUwgLEG6mu80Gc/6wCdeR/36hoYTMeNPjm6M9e+X5ppsXqxCNsgDxasJFT82 +FejuJE2sZ6RCradlDToUHNS1dMLoW0WAIISqOmrDvEI6snm9ZZr3Sxo1auEtpI6v +NllBM4AgEghy/2mAtke+By4WHCfXBpxEGv9S7ATqJHYrR5Qa3nwx0eojWW1vmn0/ +aEzslX1tAH6oz2jA6QZ0sNo= +-----END CERTIFICATE REQUEST----- diff --git a/docs/multitenant/usecase01/tls.crt b/docs/multitenant/usecase01/tls.crt new file mode 100644 index 00000000..6bf8aef4 --- /dev/null +++ b/docs/multitenant/usecase01/tls.crt @@ -0,0 +1,24 @@ +-----BEGIN CERTIFICATE----- +MIIEFDCCAvygAwIBAgIUd9l6tMS21ak3e4S0VdPhY0jG3gQwDQYJKoZIhvcNAQEL +BQAwgaExCzAJBgNVBAYTAlVTMRMwEQYDVQQIDApDYWxpZm9ybmlhMRUwEwYDVQQH +DAxTYW5GcmFuY2lzY28xEDAOBgNVBAoMB29yYWNsZSAxNjA0BgNVBAMMLWNkYi1k +ZXYtb3Jkcy5vcmFjbGUtZGF0YWJhc2Utb3BlcmF0b3Itc3lzdGVtIDEcMBoGA1UE +AwwTbG9jYWxob3N0ICBSb290IENBIDAeFw0yNDA4MTIxNTMyMzVaFw0yNTA4MTIx +NTMyMzVaMIGXMQswCQYDVQQGEwJVUzETMBEGA1UECAwKQ2FsaWZvcm5pYTEVMBMG +A1UEBwwMU2FuRnJhbmNpc2NvMRAwDgYDVQQKDAdvcmFjbGUgMTYwNAYDVQQDDC1j +ZGItZGV2LW9yZHMub3JhY2xlLWRhdGFiYXNlLW9wZXJhdG9yLXN5c3RlbSAxEjAQ +BgNVBAMMCWxvY2FsaG9zdDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEB +AJvZ5TUkDbD01ah+ezJFimRMmlbc1SkPGd6AyuoVl3wduKdI1Y+rEziZHzVQMZ6p +lPA6Vt6HaW1Oqlnv5/gebyaanV1BnbKvTSsrrNnxm/8X9VJ7JKQNhVLlmaN7qcgh +cQwDrxOHmVM5cSJ/ueF12jQHUIlPRPFVz6+WMa72deU/8Qly7ST7P0QjhXCfPoL2 +RnIzG3T9emriQ1VmcwWrq93im4hzey1zQgkcxjvvnb9JBmPLx+zMhX4Kd6mAYkh0 +t1g4hVCY2CL32vJS3z9YZ5Qsh5/E51PYIxo5ndWHHHEiqliAE9uMD1ClVC5nuJ/d +Y8Qz6e4PJ0bQ8NEpo/YGArUCAwEAAaNMMEowSAYDVR0RBEEwP4IsY2RiLWRldi1v +cmRzLm9yYWNsZS1kYXRhYmFzZS1vcGVyYXRvci1zeXN0ZW2CD3d3dy5leGFtcGxl +LmNvbTANBgkqhkiG9w0BAQsFAAOCAQEAh7Lsu2ITS6Bc2q/Ef4No5Us0Vo9BWKoL +AlrfQPjsv1erMGsyEEyZ0Cg8l3QrXlscQ1ESvx0BnRGjoqZGE4+PoVZTEYSkokXP +aAr69epPzXQRyyAGCg5GeL6IFAj1AzqJGNnKOrPaLpcTri4MboiWmW+MHmgLdyPK +iwl8bNa8841nK/L/m6QET15BI+MIAvn7pgcpztum5jmkB+eceXzXnKUGg77TaFiX +bXqVBR4EvexC4DgUfQJI4zJLFdcH/GHxCpaaXNjbXeVz1ZK/qo2TCrXp2UXVrznU +9VTUuCaQA2VYZCitvAbupt+1OvMFYhWiIAroJSmzrvH4oK+IXgY6GA== +-----END CERTIFICATE----- diff --git a/docs/multitenant/usecase01/tls.key b/docs/multitenant/usecase01/tls.key new file mode 100644 index 00000000..666c5639 --- /dev/null +++ b/docs/multitenant/usecase01/tls.key @@ -0,0 +1,28 @@ +-----BEGIN PRIVATE KEY----- +MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQCb2eU1JA2w9NWo +fnsyRYpkTJpW3NUpDxnegMrqFZd8HbinSNWPqxM4mR81UDGeqZTwOlbeh2ltTqpZ +7+f4Hm8mmp1dQZ2yr00rK6zZ8Zv/F/VSeySkDYVS5Zmje6nIIXEMA68Th5lTOXEi +f7nhddo0B1CJT0TxVc+vljGu9nXlP/EJcu0k+z9EI4Vwnz6C9kZyMxt0/Xpq4kNV +ZnMFq6vd4puIc3stc0IJHMY7752/SQZjy8fszIV+CnepgGJIdLdYOIVQmNgi99ry +Ut8/WGeULIefxOdT2CMaOZ3VhxxxIqpYgBPbjA9QpVQuZ7if3WPEM+nuDydG0PDR +KaP2BgK1AgMBAAECggEAKUwl1l0FW7yk2Q8a6glPUKCTzSybN1QPEMyj+D9ccsEV +aw57uKQmZbr9cA0d+OMK2lU7K6BKKXLM5SQTHcZCwcH6rPl0JiMZmbTrCp1hLslU +clS7MtV6XKsGeTGNncBuyjY3sD8gO9NezTt3L+0gsuS1TI06wZBxhh+QbsJUHzjW +bC3mNjD4SqXree4Snp05nlFaT2s2isIjj25mKDwBu8IX0BN2VjsaSiQcjb8Dmzmu +42Xh7bcWBebns8Ehuq9TIl6ZjQht+pmVOMlB862baVpW/9CxkknzM+UQhIkXTSJk +Jt/mGeO89V4/Zh2N4ixIOE1hw87EvRFBoYh2VF58QQKBgQDMujXYblh+eEdsB1LG +kY0LerFHuQgdzifYmjPl0jtBsWDmh5i6q9PRUs2JZ/Fsq4QMQ8SLinGzaIBq5FKr +CL067X5blrFA9H0D6exJI3iHBTQpeMFwtqvu3j+zpCmgzonaUDQrczUpc0hxU7YI +/jhDe9LSWknPrzzMoWWKuy0sTQKBgQDC4g8F2krqm9Q5ug8bRKTAvMrY0skFIwrP +5LXBq9C8YCnLnT4S4tYQfbnWaBeG7YpkkmkZe30c9MUjsr1OHZbo+jlxHBU+oRYZ +e1j0UorVGt7FfNe/zjW0fLd72CBO741EDvV6pVeItkAwH6P5/cbRu085dwvyFbxv +JmOaYddECQKBgQCuid6YG1NE10SE3CV89uAZtktny18ZEgY0ixrNx5MPaaskPtw9 +4Xofjol+qOhR7lQQpMHu+WQAQYqiFvBHspapo4pDiVCrAQWIDamNnTkHW69h3/qD +HqmsZzxF6iI3X351akVf+cOMCCXtwCGEvz+2gN12ytT8w/iAuOS6BuP3TQKBgBlf +v57+diSn13EQtajSPjVOH4ctorjFgEHjQHsP+OSeDLMTLSLeYArTo9+zu+R4hz1j +BsYnmvmrMQPd4OIL3jtFYTdF9coqxSraMZHWMXdfwUOrZpf1rG5skqNQV5yPejAz +Vmj6oDQPrrnVVM9W6I0kO0N7KZYCmH9MW0mdlZ6pAoGAB60f2sk35VUBpvh7qzTY +70WDbNnCCU3I3KZ7LCUwUPWzGLQwMXRlAb5ZMheT/SGPChX4QXCNUCjXkR3Am3NO +yURHqZIRy0bwZRVjYnlCtc9YQ8pB0isZ1z2a9FXRD75o2WboFZ+VsG0FU81IE2ZO +gW802gT76NRnz851B7/nFNs= +-----END PRIVATE KEY----- diff --git a/docs/multitenant/usecase02/README.md b/docs/multitenant/usecase02/README.md index 0b060df7..c434271f 100644 --- a/docs/multitenant/usecase02/README.md +++ b/docs/multitenant/usecase02/README.md @@ -13,7 +13,7 @@ > ☞ The examples of this folder are based on single namespace **oracle-database-operator-system** -This page explains how to plug and unplug database a pdb; it assumes that you have already configured a pluggable database (see usecase01) +This page explains how to plug and unplug database a pdb; it assumes that you have already configured a pluggable database (see [usecase01](../usecase01/README.md)) The following table reports the parameters required to configure and use oracle multi tenant controller for pluggable database lifecycle management. | yaml file parameters | value | description /ords parameter | @@ -78,7 +78,7 @@ The following table reports the parameters required to configure and use oracle ### UNPLUG DATABASE -Use the following command to check kubernets pdb resources. Note that the output of the commands can be tailored to fit to your needs. Just check the structure of pdb resource **kubectl get pdbs -n oracle-database-operator-system -o=json** and modify the script accordingly. For the sake of simplicity put this command in a single script **checkpdbs.sh**. +Use the following command to check kubernets pdb resources. Note that the output of the commands can be tailored to meet your needs. Just check the structure of pdb resource **kubectl get pdbs -n oracle-database-operator-system -o=json** and modify the script accordingly. For the sake of simplicity put this command in a single script **checkpdbs.sh**. ```bash kubectl get pdbs -n oracle-database-operator-system -o=jsonpath='{range .items[*]} @@ -92,7 +92,7 @@ kubectl get pdbs -n oracle-database-operator-system -o=jsonpath='{range .items[* {"\n"}{end}' ``` -We assume that the pluggable database pdbdev is already configured on opened in read write mode +We assume that the pluggable database pdbdev is already configured and opened in read write mode ```bash ./checkpdbs.sh @@ -106,7 +106,7 @@ MSG=Success ``` -Prepare a new yaml file **pdb_unplug.yaml** to unplug the pdbdev database. Make sure that the path of the xml file is correct and check the existence of all the required secrets. +Prepare a new yaml file **pdb_unplug.yaml** to unplug the pdbdev database. Make sure that the path of the xml file is correct and check the existence of all the required secrets. Do not reuse an existing xml files. ```yaml # Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. @@ -248,7 +248,7 @@ Completed: DROP PLUGGABLE DATABASE "pdbdev" KEEP DATAFILES ``` -login to the server and check xml file existence. Get the datafile path on the ASM filesystem. +login to the server and check xml file existence. Verify the datafile path on the ASM filesystem. ```bash ls -ltr /tmp/pdbunplug.xml diff --git a/docs/multitenant/usecase02/pdb_clone.yaml b/docs/multitenant/usecase02/pdb_clone.yaml index be22020b..0ecc3c70 100644 --- a/docs/multitenant/usecase02/pdb_clone.yaml +++ b/docs/multitenant/usecase02/pdb_clone.yaml @@ -18,6 +18,7 @@ spec: fileNameConversions: "NONE" totalSize: "UNLIMITED" tempSize: "UNLIMITED" + assertivePdbDeletion: true adminName: secret: secretName: "pdb1-secret" diff --git a/docs/multitenant/usecase02/pdb_plug.yaml b/docs/multitenant/usecase02/pdb_plug.yaml index 0d0d3b37..77c00b9c 100644 --- a/docs/multitenant/usecase02/pdb_plug.yaml +++ b/docs/multitenant/usecase02/pdb_plug.yaml @@ -21,6 +21,7 @@ spec: totalSize: "1G" tempSize: "100M" action: "Plug" + assertivePdbDeletion: true pdbTlsKey: secret: secretName: "db-tls" diff --git a/docs/multitenant/usecase03/Dockerfile b/docs/multitenant/usecase03/Dockerfile index 5c27f11b..772a7e6d 100644 --- a/docs/multitenant/usecase03/Dockerfile +++ b/docs/multitenant/usecase03/Dockerfile @@ -1,34 +1,63 @@ -#LICENSE UPL 1.0 -# -# Copyright (c) 1982-2017 Oracle and/or its affiliates. All rights reserved. -# -# ORACLE DOCKERFILES PROJECT -# -------------------------- -# This is the Dockerfile for Oracle Rest Data Services 22.2 -# +## Copyright (c) 2022 Oracle and/or its affiliates. +## +## The Universal Permissive License (UPL), Version 1.0 +## +## Subject to the condition set forth below, permission is hereby granted to any +## person obtaining a copy of this software, associated documentation and/or data +## (collectively the "Software"), free of charge and under any and all copyright +## rights in the Software, and any and all patent rights owned or freely +## licensable by each licensor hereunder covering either (i) the unmodified +## Software as contributed to or provided by such licensor, or (ii) the Larger +## Works (as defined below), to deal in both +## +## (a) the Software, and +## (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +## one is included with the Software (each a "Larger Work" to which the Software +## is contributed by such licensors), +## +## without restriction, including without limitation the rights to copy, create +## derivative works of, display, perform, and distribute the Software and make, +## use, sell, offer for sale, import, export, have made, and have sold the +## Software and the Larger Work(s), and to sublicense the foregoing rights on +## either these or other terms. +## +## This license is subject to the following condition: +## The above copyright notice and either this complete permission notice or at +## a minimum a reference to the UPL must be included in all copies or +## substantial portions of the Software. +## +## THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +## IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +## FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +## AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +## LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +## OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +## SOFTWARE. + FROM container-registry.oracle.com/java/jdk:latest # Environment variables required for this build (do NOT change) # ------------------------------------------------------------- ENV ORDS_HOME=/opt/oracle/ords/ \ - RUN_FILE="runOrdsSSL.sh" - -#RUN_FILE_NOSSL="runOrdsNOSSL.sh" + RUN_FILE="runOrdsSSL.sh" \ + ORDSVERSION=23.4.0-8 # Copy binaries # ------------- COPY $RUN_FILE $ORDS_HOME -#COPY $RUN_FILE_NOSSL $ORDS_HOME -RUN yum -y install yum-utils bind-utils tree hostname openssl net-tools zip unzip tar wget vim-minimal which sudo expect procps && \ +RUN yum -y install yum-utils bind-utils tree hostname openssl net-tools zip unzip tar wget vim-minimal which sudo expect procps curl lsof && \ yum-config-manager --add-repo=http://yum.oracle.com/repo/OracleLinux/OL8/oracle/software/x86_64 && \ yum -y install java-11-openjdk-devel && \ - yum -y install ords && \ yum -y install iproute && \ yum clean all +RUN curl -o /tmp/ords-$ORDSVERSION.el8.noarch.rpm https://yum.oracle.com/repo/OracleLinux/OL8/oracle/software/x86_64/getPackage/ords-$ORDSVERSION.el8.noarch.rpm + +RUN rpm -ivh /tmp/ords-$ORDSVERSION.el8.noarch.rpm + # Setup filesystem and oracle user -# ------------------------------------------------------------ +# -------------------------------- RUN mkdir -p $ORDS_HOME/doc_root && \ mkdir -p $ORDS_HOME/error && \ mkdir -p $ORDS_HOME/secrets && \ @@ -43,11 +72,9 @@ RUN mkdir -p $ORDS_HOME/doc_root && \ USER oracle WORKDIR /home/oracle -#VOLUME ["$ORDS_HOME/config/ords"] +VOLUME ["$ORDS_HOME/config/ords"] EXPOSE 8888 # Define default command to start Ords Services CMD $ORDS_HOME/$RUN_FILE -## ONLY FOR DEVELOPMENT STAGE -#CMD ["/usr/sbin/init"] diff --git a/docs/multitenant/usecase03/README.md b/docs/multitenant/usecase03/README.md index 3703a9ff..c06368cd 100644 --- a/docs/multitenant/usecase03/README.md +++ b/docs/multitenant/usecase03/README.md @@ -19,9 +19,9 @@ ### INTRODUCTION -> ☞ This folder contains the yaml files required to configure and manage cdb and pdb in different namespaces. The main change here is the possibility to specify the namespace where CDB will be created, this implies the introduction of a new parameter at PDB level in order to specify the CDB namespace. +> ☞ This folder contains the yaml files required to configure and manage cdb and pdb in different namespaces. The main change here is the possibility to specify the namespace where CDB will be created, this implies the introduction of new parameter at PDB level in order to specify the CDB namespace. -Tasks performed in the usecase03 are the same ones of the other usecases with the exception that controller pods cdb pods and pdb crd are running in different namespaces. You must be aware of the fact that secrets must be created in the proper namespaces; cdb secrets go into cdb namespace , pdb secrets go into pdbnamespace while certificate secrets need to be created in every namespace. +Tasks performed in the usecase03 are the same ones of the other usecase01 with the exception that controller pods cdb pods and pdb crd are running in different namespaces. You must be aware of the fact that secrets must be created in the proper namespaces; cdb secrets go into cdb namespace , pdb secrets go into pdbnamespace while certificate secrets need to be created in every namespace. | yaml file parameters | value | description /ords parameter | @@ -53,6 +53,7 @@ Tasks performed in the usecase03 are the same ones of the other usecases with th | tdeExport | | [tdeExport] | | tdeSecret | | [tdeSecret][tdeSecret] | | tdePassword | | [tdeSecret][tdeSecret] | +| assertivePdbDeletion | boolean | [turn on imperative approach on crd deleteion][imperative] | ![generla schema](./NamespaceSegregation.png) @@ -261,6 +262,7 @@ In order to facilitate the command execution use the [makefile](./makefile) avai |reloadop | Reload the db operator | |login | Login into cdb pod | - +[imperative]:https://kubernetes.io/docs/concepts/overview/working-with-objects/object-management/ + diff --git a/docs/multitenant/usecase03/cdb_create.yaml b/docs/multitenant/usecase03/cdb_create.yaml index 09dd7f86..d3b5e04f 100644 --- a/docs/multitenant/usecase03/cdb_create.yaml +++ b/docs/multitenant/usecase03/cdb_create.yaml @@ -5,9 +5,9 @@ metadata: namespace: cdbnamespace spec: cdbName: "DB12" - ordsImage: "lin.ocir.io/intsanjaysingh/mmalvezz/testppr/ords-dboper:latest" + ordsImage: ".............your registry............./ords-dboper:latest" ordsImagePullPolicy: "Always" - dbTnsurl : "(DESCRIPTION=(CONNECT_TIMEOUT=90)(RETRY_COUNT=30)(RETRY_DELAY=10)(TRANSPORT_CONNECT_TIMEOUT=70)(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan12.testrac.com)(PORT=1521)(IP=V4_ONLY))(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan34.testrac.com)(PORT=1521)(IP=V4_ONLY))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=TESTORDS)))" + dbTnsurl : "...Container tns alias....." replicas: 1 sysAdminPwd: secret: diff --git a/docs/multitenant/usecase03/makefile b/docs/multitenant/usecase03/makefile index fc95cfa0..7270a5e0 100644 --- a/docs/multitenant/usecase03/makefile +++ b/docs/multitenant/usecase03/makefile @@ -123,8 +123,10 @@ URLPATH=/_/db-api/stable/database/pdbs/ OPENSSL=/usr/bin/openssl ORDSPORT=8888 MAKE=/usr/bin/make -DOCKERFILE=Dockerfile +DOCKERFILE=../../../ords/Dockerfile +RUNSCRIPT=../../../ords/runOrdsSSL.sh RM=/usr/bin/rm +CP=/bin/cp ECHO=/usr/bin/echo CERTMANAGER=https://github.com/jetstack/cert-manager/releases/latest/download/cert-manager.yaml CDB_SECRET_YAML=cdb_secret.yaml @@ -161,16 +163,8 @@ checkstep9: checkcdb createimage: @echo "BUILDING CDB IMAGES" - @if [[ ! -f ./Dockerfile ]]; \ - then\ - echo "DOCKERFILE DOES NOT EXISTS";\ - exit 1; \ - fi; - @if [[ ! -f ../runOrdsSSL.sh ]]; \ - then\ - echo "DOCKERFILE DOES NOT EXISTS";\ - exit 1; \ - fi; + $(CP) $(DOCKERFILE) . + $(CP) $(RUNSCRIPT) . $(DOCKER) build -t $(IMAGE) . tagimage: @@ -201,7 +195,7 @@ tlscert: $(OPENSSL) genrsa -out ca.key 2048 $(OPENSSL) req -new -x509 -days 365 -key ca.key -subj "/C=US/ST=California/L=SanFrancisco/O=$(COMPANY) /CN=$(RESTPREFIX)-$(REST_SERVER).$(CDB_NAMESPACE) /CN=$(LOCALHOST) Root CA " -out ca.crt $(OPENSSL) req -newkey rsa:2048 -nodes -keyout $(SKEY) -subj "/C=US/ST=California/L=SanFrancisco/O=$(COMPANY) /CN=$(RESTPREFIX)-$(REST_SERVER).$(CDB_NAMESPACE) /CN=$(LOCALHOST)" -out server.csr - $(ECHO) "subjectAltName=DNS:cdb-dev-$(REST_SERVER),DNS:www.example.com" > extfile.txt + $(ECHO) "subjectAltName=DNS:$(RESTPREFIX)-$(REST_SERVER).$(CDB_NAMESPACE),DNS:www.example.com" > extfile.txt $(OPENSSL) x509 -req -extfile extfile.txt -days 365 -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out $(SCRT) diff --git a/docs/multitenant/usecase03/runOrdsSSL.sh b/docs/multitenant/usecase03/runOrdsSSL.sh new file mode 100644 index 00000000..35f1b77b --- /dev/null +++ b/docs/multitenant/usecase03/runOrdsSSL.sh @@ -0,0 +1,190 @@ +#!/bin/bash + +cat <$TNSNAME + + +function SetParameter() { + ##ords config info <--- Use this command to get the list + +[[ ! -z "${ORACLE_HOST}" && -z "${DBTNSURL}" ]] && { + $ORDS --config ${CONFIG} config set db.hostname ${ORACLE_HOST:-racnode1} + $ORDS --config ${CONFIG} config set db.port ${ORACLE_PORT:-1521} + $ORDS --config ${CONFIG} config set db.servicename ${ORACLE_SERVICE:-TESTORDS} +} + +[[ -z "${ORACLE_HOST}" && ! -z "${DBTNSURL}" ]] && { + #$ORDS --config ${CONFIG} config set db.tnsAliasName ${TNSALIAS} + #$ORDS --config ${CONFIG} config set db.tnsDirectory ${TNS_ADMIN} + #$ORDS --config ${CONFIG} config set db.connectionType tns + + $ORDS --config ${CONFIG} config set db.connectionType customurl + $ORDS --config ${CONFIG} config set db.customURL jdbc:oracle:thin:@${DBTNSURL} +} + + $ORDS --config ${CONFIG} config set security.requestValidationFunction false + $ORDS --config ${CONFIG} config set jdbc.MaxLimit 100 + $ORDS --config ${CONFIG} config set jdbc.InitialLimit 50 + $ORDS --config ${CONFIG} config set error.externalPath ${ERRORFOLDER} + $ORDS --config ${CONFIG} config set standalone.access.log /home/oracle + $ORDS --config ${CONFIG} config set standalone.https.port 8888 + $ORDS --config ${CONFIG} config set standalone.https.cert ${CERTIFICATE} + $ORDS --config ${CONFIG} config set standalone.https.cert.key ${KEY} + $ORDS --config ${CONFIG} config set restEnabledSql.active true + $ORDS --config ${CONFIG} config set security.verifySSL true + $ORDS --config ${CONFIG} config set database.api.enabled true + $ORDS --config ${CONFIG} config set plsql.gateway.mode disabled + $ORDS --config ${CONFIG} config set database.api.management.services.disabled false + $ORDS --config ${CONFIG} config set misc.pagination.maxRows 1000 + $ORDS --config ${CONFIG} config set db.cdb.adminUser "${CDBADMIN_USER:-C##DBAPI_CDB_ADMIN} AS SYSDBA" + $ORDS --config ${CONFIG} config secret --password-stdin db.cdb.adminUser.password << EOF +${CDBADMIN_PWD:-PROVIDE_A_PASSWORD} +EOF + +$ORDS --config ${CONFIG} config user add --password-stdin ${WEBSERVER_USER:-ordspdbadmin} "SQL Administrator, System Administrator" <${CKF} 2>&1 +echo "checkfile" >> ${CKF} +NOT_INSTALLED=`cat ${CKF} | grep "INFO: The" |wc -l ` +echo NOT_INSTALLED=$NOT_INSTALLED + + +function StartUp () { + $ORDS --config $CONFIG serve --port 8888 --secure +} + +# Check whether ords is already setup +if [ $NOT_INSTALLED -ne 0 ] +then + echo " SETUP " + setupOrds; + StartUp; +fi + +if [ $NOT_INSTALLED -eq 0 ] +then + echo " STARTUP " + StartUp; +fi + + diff --git a/docs/sharding/README.md b/docs/sharding/README.md index 3d3320ee..0c817467 100644 --- a/docs/sharding/README.md +++ b/docs/sharding/README.md @@ -22,18 +22,21 @@ The Sharding Database controller in Oracle Database Operator deploys Oracle Shar The Oracle Sharding database controller provides end-to-end automation of Oracle Database sharding topology deployment in Kubernetes clusters. -## Using Oracle Sharding Database Operator +## Using Oracle Database Operator Sharding Controller -To create a Sharding Topology, complete the steps in the following sections below: +Following sections provide the details for deploying Oracle Globally Distributed Database (Oracle Sharded Database) using Oracle Database Operator Sharding Controller with different use cases: -1. [Prerequisites for running Oracle Sharding Database Controller](#prerequisites-for-running-oracle-sharding-database-controller) -2. [Provisioning Sharding Topology in a Cloud based Kubernetes Cluster (OKE in this case)](#provisioning-sharding-topology-in-a-cloud-based-kubernetes-cluster-oke-in-this-case) -3. [Connecting to Shard Databases](#connecting-to-shard-databases) -4. [Debugging and Troubleshooting](#debugging-and-troubleshooting) +* [Prerequisites for running Oracle Sharding Database Controller](#prerequisites-for-running-oracle-sharding-database-controller) +* [Oracle Database 23ai Free](#oracle-database-23ai-free) +* [Provisioning Sharding Topology with System-Managed Sharding in a Cloud-Based Kubernetes Cluster](#provisioning-sharding-topology-with-system-managed-sharding-in-a-cloud-based-kubernetes-cluster) +* [Provisioning Sharding Topology with User Defined Sharding in a Cloud-Based Kubernetes Cluster](#provisioning-sharding-topology-with-user-defined-sharding-in-a-cloud-based-kubernetes-cluster) +* [Provisioning System-Managed Sharding Topology with Raft replication enabled in a Cloud-Based Kubernetes Cluster](#provisioning-system-managed-sharding-topology-with-raft-replication-enabled-in-a-cloud-based-kubernetes-cluster) +* [Connecting to Shard Databases](#connecting-to-shard-databases) +* [Debugging and Troubleshooting](#debugging-and-troubleshooting) **Note** Before proceeding to the next section, you must complete the instructions given in each section, based on your enviornment, before proceeding to next section. -## Prerequisites for Running Oracle Sharding Database Controller +## Prerequisites for running Oracle Sharding Database Controller **IMPORTANT:** You must make the changes specified in this section before you proceed to the next section. @@ -85,6 +88,8 @@ You can either download the images and push them to your Docker Images Repositor **Note**: In the sharding example yaml files, we are using GDS and database images available on [Oracle Container Registry](https://container-registry.oracle.com/ords/f?p=113:10::::::). +**Note:** In case you want to use the `Oracle Database 23ai Free` Image for Database and GSM, refer to section [Oracle Database 23ai Free](#oracle-database-23ai-free) for more details. + ### 4. Create a namespace for the Oracle DB Sharding Setup Create a Kubernetes namespace named `shns`. All the resources belonging to the Oracle Database Sharding Setup will be provisioned in this namespace named `shns`. For example: @@ -101,7 +106,7 @@ You can either download the images and push them to your Docker Images Repositor Create a Kubernetes secret named `db-user-pass-rsa` using these steps: [Create Kubernetes Secret](./provisioning/create_kubernetes_secret_for_db_user.md) -After you have the above prerequsites completed, you can proceed to the next section for your environment to provision the Oracle Database Sharding Topology. +After you have the above prerequisites completed, you can proceed to the next section for your environment to provision the Oracle Database Sharding Topology. ### 6. Provisioning a Persistent Volume having an Oracle Database Gold Image @@ -111,19 +116,35 @@ In case of an `OCI OKE` cluster, you can use this Persistent Volume during provi You can refer [here](./provisioning/provisioning_persistent_volume_having_db_gold_image.md) for the steps involved. -## Provisioning Sharding Topology with System Sharding in a Cloud-Based Kubernetes Cluster +**NOTE:** Provisioning the Sharded Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. So, this step will not be needed if you are deploying Oracle Sharded Database using Oracle 23ai Free Database and GSM Images. + +## Oracle Database 23ai Free + +Please refer to [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) documentation for more details. + +If you want to use Oracle Database 23ai Free Image for Database and GSM for deployment of the Sharded Database using Sharding Controller in Oracle Database Kubernetes Operator, you need to consider the below points: + +* To deploy using the FREE Database and GSM Image, you will need to add the additional parameter `dbEdition: "free"` to the .yaml file. +* Refer to [Sample Sharded Database Deployment using Oracle 23ai FREE Database and GSM Images](./provisioning/free/sharding_provisioning_with_free_images.md) for an example. +* For Oracle Database 23ai Free, you can control the `CPU` and `Memory` allocation of the PODs using tags `cpu` and `memory` respectively but tags `INIT_SGA_SIZE` and `INIT_PGA_SIZE` to control the SGA and PGA allocation at the database level are `not` supported. +* Provisioning the Sharded Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. +* Total number of chunks for FREE Database defaults to `12` if `CATALOG_CHUNKS` parameter is not specified. This default value is determined considering limitation of 12 GB of user data on disk for oracle free database. + + +## Provisioning Sharding Topology with System-Managed Sharding in a Cloud-Based Kubernetes Cluster -Deploy Oracle Database Sharding Topology with `System Sharding` on your Cloud based Kubernetes cluster. +Deploy Oracle Database Sharding Topology with `System-Managed Sharding` on your Cloud based Kubernetes cluster. In this example, the deployment uses the YAML file based on `OCI OKE` cluster. There are multiple use case possible for deploying the Oracle Database Sharding Topology covered by below examples: -[1. Provisioning Oracle Sharded Database with System Sharding without Database Gold Image](./provisioning/system_sharding/ssharding_provisioning_without_db_gold_image.md) -[2. Provisioning Oracle Sharded Database with System Sharding with additional control on resources like Memory and CPU allocated to Pods](./provisioning/system_sharding/ssharding_provisioning_with_control_on_resources.md) -[3. Provisioning Oracle Sharded Database with System Sharding by cloning database from your own Database Gold Image in the same Availability Domain(AD)](./provisioning/system_sharding/ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md) -[4. Provisioning Oracle Sharded Database with System Sharding by cloning database from your own Database Gold Image across Availability Domains(ADs)](./provisioning/system_sharding/ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md) -[5. Provisioning Oracle Sharded Database with System Sharding and send Notification using OCI Notification Service](./provisioning/system_sharding/ssharding_provisioning_with_notification_using_oci_notification.md) -[6. Scale Out - Add Shards to an existing Oracle Sharded Database provisioned earlier with System Sharding](./provisioning/system_sharding/ssharding_scale_out_add_shards.md) -[7. Scale In - Delete an existing Shard from a working Oracle Sharded Database provisioned earlier with System Sharding](./provisioning/system_sharding/ssharding_scale_in_delete_an_existing_shard.md) +[1. Provisioning Oracle Sharded Database with System-Managed Sharding without Database Gold Image](./provisioning/system_sharding/ssharding_provisioning_without_db_gold_image.md) +[2. Provisioning Oracle Sharded Database with System-Managed Sharding with number of chunks specified](./provisioning/system_sharding/ssharding_provisioning_with_chunks_specified.md) +[3. Provisioning Oracle Sharded Database with System-Managed Sharding with additional control on resources like Memory and CPU allocated to Pods](./provisioning/system_sharding/ssharding_provisioning_with_control_on_resources.md) +[4. Provisioning Oracle Sharded Database with System-Managed Sharding by cloning database from your own Database Gold Image in the same Availability Domain(AD)](./provisioning/system_sharding/ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md) +[5. Provisioning Oracle Sharded Database with System-Managed Sharding by cloning database from your own Database Gold Image across Availability Domains(ADs)](./provisioning/system_sharding/ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md) +[6. Provisioning Oracle Sharded Database with System-Managed Sharding and send Notification using OCI Notification Service](./provisioning/system_sharding/ssharding_provisioning_with_notification_using_oci_notification.md) +[7. Scale Out - Add Shards to an existing Oracle Sharded Database provisioned earlier with System-Managed Sharding](./provisioning/system_sharding/ssharding_scale_out_add_shards.md) +[8. Scale In - Delete an existing Shard from a working Oracle Sharded Database provisioned earlier with System-Managed Sharding](./provisioning/system_sharding/ssharding_scale_in_delete_an_existing_shard.md) ## Provisioning Sharding Topology with User Defined Sharding in a Cloud-Based Kubernetes Cluster @@ -140,6 +161,24 @@ In this example, the deployment uses the YAML file based on `OCI OKE` cluster. T [6. Scale Out - Add Shards to an existing Oracle Sharded Database provisioned earlier with User Defined Sharding](./provisioning/user-defined-sharding/udsharding_scale_out_add_shards.md) [7. Scale In - Delete an existing Shard from a working Oracle Sharded Database provisioned earlier with User Defined Sharding](./provisioning/user-defined-sharding/udsharding_scale_in_delete_an_existing_shard.md) + +## Provisioning System-Managed Sharding Topology with Raft replication enabled in a Cloud-Based Kubernetes Cluster + +Deploy Oracle Database Sharding Topology with `System-Managed Sharding with SNR RAFT enabled` on your Cloud based Kubernetes cluster. + +**NOTE: SNR RAFT Feature is available only for Oracle 23ai RDBMS and Oracle 23ai GSM version.** + +In this example, the deployment uses the YAML file based on `OCI OKE` cluster. There are multiple use case possible for deploying the Oracle Database Sharding Topology covered by below examples: + +[1. Provisioning System-Managed Sharding Topology with Raft replication enabled without Database Gold Image](./provisioning/snr_system_sharding/snr_ssharding_provisioning_without_db_gold_image.md) +[2. Provisioning System-Managed Sharding Topology with Raft replication enabled with number of chunks specified](./provisioning/snr_system_sharding/snr_ssharding_provisioning_with_chunks_specified.md) +[3. Provisioning System-Managed Sharding Topology with Raft replication enabled with additional control on resources like Memory and CPU allocated to Pods](./provisioning/snr_system_sharding/snr_ssharding_provisioning_with_control_on_resources.md) +[4. Provisioning System-Managed Sharding Topology with Raft replication enabled by cloning database from your own Database Gold Image in the same Availability Domain(AD)](./provisioning/snr_system_sharding/snr_ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md) +[5. Provisioning System-Managed Sharding Topology with Raft replication enabled by cloning database from your own Database Gold Image across Availability Domains(ADs)](./provisioning/snr_system_sharding/snr_ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md) +[6. Provisioning System-Managed Sharding Topology with Raft replication enabled and send Notification using OCI Notification Service](./provisioning/snr_system_sharding/snr_ssharding_provisioning_with_notification_using_oci_notification.md) +[7. Scale Out - Add Shards to an existing Oracle Sharded Database provisioned earlier with System-Managed Sharding with RAFT replication enabled](./provisioning/snr_system_sharding/snr_ssharding_scale_out_add_shards.md) +[8. Scale In - Delete an existing Shard from a working Oracle Sharded Database provisioned earlier with System-Managed Sharding with RAFT reolication enabled](./provisioning/snr_system_sharding/snr_ssharding_scale_in_delete_an_existing_shard.md) + ## Connecting to Shard Databases After the Oracle Database Sharding Topology has been provisioned using the Sharding Controller in Oracle Database Kubernetes Operator, you can follow the steps in this document to connect to the Sharded Database or to the individual Shards: [Database Connectivity](./provisioning/database_connection.md) diff --git a/docs/sharding/provisioning/debugging.md b/docs/sharding/provisioning/debugging.md index 545bf034..63e02b6a 100644 --- a/docs/sharding/provisioning/debugging.md +++ b/docs/sharding/provisioning/debugging.md @@ -41,3 +41,10 @@ kubectl exec -it catalog-0 -n shns /bin/bash ``` Now, you can troubleshooting the corresponding component using the alert log or the trace files etc just like a normal Sharding Database Deployment. Please refer to [Oracle Database Sharding Documentation](https://docs.oracle.com/en/database/oracle/oracle-database/19/shard/sharding-troubleshooting.html#GUID-629262E5-7910-4690-A726-A565C59BA73E) for this purpose. + + +## Debugging using Database Events + +* You can enable database events as part of the Sharded Database Deployment +* This can be enabled using the `envVars` +* One example of enabling Database Events is [sharding_provisioning_with_db_events.md](./debugging/sharding_provisioning_with_db_events.md) \ No newline at end of file diff --git a/docs/sharding/provisioning/debugging/sharding_provisioning_with_db_events.md b/docs/sharding/provisioning/debugging/sharding_provisioning_with_db_events.md new file mode 100644 index 00000000..fa73920f --- /dev/null +++ b/docs/sharding/provisioning/debugging/sharding_provisioning_with_db_events.md @@ -0,0 +1,40 @@ +# Example of provisioning Oracle Sharded Database along with DB Events set at Database Level + +**IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. + +This example sets a Database Event at the Database Level for Catalog and Shard Databases. + +The sharded database in this example is deployed with System-Managed Sharding type. In this use case, the database is created automatically using DBCA during the provisioning of the shard databases and the catalog database when the Oracle Sharding topology with System-Managed Sharding is deployed using Oracle Sharding controller. + +**NOTE:** In this use case, because DBCA creates the database automatically during the deployment, the time required to create the database is greater than the time it takes when the database is created by cloning from a Database Gold Image. + +This example uses `sharding_provisioning_with_db_events.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: + +* Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` +* Three sharding Pods: `shard1`, `shard2` and `shard3` +* One Catalog Pod: `catalog` +* Namespace: `shns` +* Database Event: `10798 trace name context forever, level 7` set along with `GWM_TRACE level 263` + + +In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) + * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. + * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `sharding_provisioning_with_db_events.yaml`. + * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) + + +Use the file: [sharding_provisioning_with_db_events.yaml](./sharding_provisioning_with_db_events.yaml) for this use case as below: + +1. Deploy the `sharding_provisioning_with_db_events.yaml` file: + ```sh + kubectl apply -f sharding_provisioning_with_db_events.yaml + ``` +2. Check the status of the deployment: + ```sh + # Check the status of the Kubernetes Pods: + kubectl get all -n shns + + # Check the logs of a particular pod. For example, to check status of pod "shard1-0": + kubectl logs -f pod/shard1-0 -n shns + ``` +3. You can confirm the Database event and the tracing enabled in the RDBMS alert log file of the Database. \ No newline at end of file diff --git a/docs/sharding/provisioning/debugging/sharding_provisioning_with_db_events.yaml b/docs/sharding/provisioning/debugging/sharding_provisioning_with_db_events.yaml new file mode 100644 index 00000000..7d136d58 --- /dev/null +++ b/docs/sharding/provisioning/debugging/sharding_provisioning_with_db_events.yaml @@ -0,0 +1,69 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +--- +apiVersion: database.oracle.com/v1alpha1 +kind: ShardingDatabase +metadata: + name: shardingdatabase-sample + namespace: shns +spec: + shard: + - name: shard1 + storageSizeInGb: 50 + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + envVars: + - name: "DB_EVENTS" + value: "10798 trace name context forever, level 7:scope=spfile;immediate trace name GWM_TRACE level 263" + - name: shard2 + storageSizeInGb: 50 + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + envVars: + - name: "DB_EVENTS" + value: "10798 trace name context forever, level 7:scope=spfile;immediate trace name GWM_TRACE level 263" + - name: shard3 + storageSizeInGb: 50 + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + envVars: + - name: "DB_EVENTS" + value: "10798 trace name context forever, level 7:scope=spfile;immediate trace name GWM_TRACE level 263" + catalog: + - name: catalog + storageSizeInGb: 50 + imagePullPolicy: "Always" + envVars: + - name: "DB_EVENTS" + value: "10798 trace name context forever, level 7:scope=spfile;immediate trace name GWM_TRACE level 263" + gsm: + - name: gsm1 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: primary + - name: gsm2 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: standby + storageClass: oci + dbImage: container-registry.oracle.com/database/enterprise:latest + dbImagePullSecret: ocr-reg-cred + gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImagePullSecret: ocr-reg-cred + isExternalSvc: False + isDeleteOraPvc: True + dbSecret: + name: db-user-pass-rsa + pwdFileName: pwdfile.enc + keyFileName: key.pem + gsmService: + - name: oltp_rw_svc + role: primary + - name: oltp_ro_svc + role: primary + namespace: shns diff --git a/docs/sharding/provisioning/free/sharding_provisioning_with_free_images.md b/docs/sharding/provisioning/free/sharding_provisioning_with_free_images.md new file mode 100644 index 00000000..61641312 --- /dev/null +++ b/docs/sharding/provisioning/free/sharding_provisioning_with_free_images.md @@ -0,0 +1,40 @@ +# Example of provisioning Oracle Sharded Database with Oracle 23ai FREE Database and GSM Images + +**IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. + +This example uses the Oracle 23ai FREE Database and GSM Images. + +The sharded database in this example is deployed with System-Managed Sharding type. In this use case, the database is created automatically using DBCA during the provisioning of the shard databases and the catalog database when the Oracle Sharding topology with System-Managed Sharding is deployed using Oracle Sharding controller. + +**NOTE:** In this use case, because DBCA creates the database automatically during the deployment, the time required to create the database is greater than the time it takes when the database is created by cloning from a Database Gold Image. + +This example uses `sharding_provisioning_with_free_images.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: + +* Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` +* Three sharding Pods: `shard1`, `shard2` and `shard3` +* One Catalog Pod: `catalog` +* Namespace: `shns` + + +To get the Oracle 23ai FREE Database and GSM Images: + * The Oracle 23ai FREE RDBMS Image used is `container-registry.oracle.com/database/free:latest`. Check [Oracle Database Free Get Started](https://www.oracle.com/database/free/get-started/?source=v0-DBFree-ChatCTA-j2032-20240709) for details. + * To pull the above image from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. + * Use the Oracle 23ai FREE GSM Binaries `LINUX.X64_234000_gsm.zip` as listed on page [Oracle Database Free Get Started](https://www.oracle.com/database/free/get-started/?source=v0-DBFree-ChatCTA-j2032-20240709) and prepare the GSM Container Image following [Oracle Global Data Services Image](https://github.com/oracle/db-sharding/tree/master/docker-based-sharding-deployment/dockerfiles) + * You need to change `dbImage` and `gsmImage` tag with the images you want to use in your enviornment in file `sharding_provisioning_with_free_images.yaml`. + + + +Use the file: [sharding_provisioning_with_free_images.yaml](./sharding_provisioning_with_free_images.yaml) for this use case as below: + +1. Deploy the `sharding_provisioning_with_free_images.yaml` file: + ```sh + kubectl apply -f sharding_provisioning_with_free_images.yaml + ``` +2. Check the status of the deployment: + ```sh + # Check the status of the Kubernetes Pods: + kubectl get all -n shns + + # Check the logs of a particular pod. For example, to check status of pod "shard1-0": + kubectl logs -f pod/shard1-0 -n shns + ``` \ No newline at end of file diff --git a/docs/sharding/provisioning/free/sharding_provisioning_with_free_images.yaml b/docs/sharding/provisioning/free/sharding_provisioning_with_free_images.yaml new file mode 100644 index 00000000..7e39b3b2 --- /dev/null +++ b/docs/sharding/provisioning/free/sharding_provisioning_with_free_images.yaml @@ -0,0 +1,58 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +--- +apiVersion: database.oracle.com/v1alpha1 +kind: ShardingDatabase +metadata: + name: shardingdatabase-sample + namespace: shns +spec: + shard: + - name: shard1 + storageSizeInGb: 50 + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + - name: shard2 + storageSizeInGb: 50 + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + - name: shard3 + storageSizeInGb: 50 + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + catalog: + - name: catalog + storageSizeInGb: 50 + imagePullPolicy: "Always" + gsm: + - name: gsm1 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: primary + - name: gsm2 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: standby + storageClass: oci + dbImage: container-registry.oracle.com/database/free:latest + dbImagePullSecret: ocr-reg-cred + gsmImage: + gsmImagePullSecret: + dbEdition: "free" + isExternalSvc: False + isDeleteOraPvc: True + dbSecret: + name: db-user-pass-rsa + pwdFileName: pwdfile.enc + keyFileName: key.pem + gsmService: + - name: oltp_rw_svc + role: primary + - name: oltp_ro_svc + role: primary + namespace: shns diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md new file mode 100644 index 00000000..ba72be25 --- /dev/null +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md @@ -0,0 +1,57 @@ +# Provisioning System managed Sharding Topology with Raft replication enabled by cloning database from your own Database Gold Image across Availability Domains(ADs) + +**NOTE: RAFT Replication Feature is available only for Oracle 23c RDBMS and Oracle 23c GSM version.** + +**IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. + +In this test case, you provision the System managed Sharding Topology with Raft replication enabled while provisioning the Catalog and Shard Databases by cloning from an existing Oracle Database Gold Image created earlier. + +This use case applies when you want to provision the database Pods on a Kubernetes Node in any availability domain (AD), which can also be different from the availability domain (AD) of the Block Volume that has the Oracle Database Gold Image provisioned earlier. + +Choosing this option takes substantially less time during the Oracle Database Sharding Topology setup across ADs. + +NOTE: + +* Cloning from Block Volume Backup in OCI enables the new Persistent Volumes to be created in other ADs. +* To specify the AD where you want to provision the database Pod, use the tag `nodeSelector` and the POD will be provisioned in a node running in that AD. +* To specify GSM containers, you can also use the tag `nodeSelector` to specify the AD. +* Before you can provision with the Gold Image, you need the OCID of the Persistent Volume that has the Oracle Database Gold Image. + +1. Check the OCID of the Persistent Volume provisioned for the Oracle Database Gold Image: + ```sh + kubectl get pv -n shns + ``` +2. Create a Block Volume Backup for this Block Volume, and use the OCID of the Block Volume Backup in the next step. This example uses `snr_ssharding_shard_prov_clone_across_ads.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: + +* Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` +* Three sharding Pods: `shard1`, `shard2` and `shard3` +* One Catalog Pod: `catalog` +* Namespace: `shns` +* Database Cloning from the `BLOCK VOLUME FULL BACKUP` of the Persistent Volume which had the Gold Image. +* OCID of the Block Volume Backup: `ocid1.volumebackup.oc1.phx.abyhqljrxtv7tu5swqb3lzc7vpzwbwzdktd2y4k2vjjy2srmgu2w7bqdftjq` +* `RAFT Replication` enabled + +NOTE: In this case, the Persistent Volume with DB Gold Image was provisioned in the Availability Domain `PHX-AD-1`. The Shards and Catalog will be provisioned across multiple Availability Domains by cloning the database. + +**NOTE:** Provisioning the Sharded Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. + +In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) + * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. + * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `snr_ssharding_shard_prov_clone_across_ads.yaml`. + * The `dbImage` used during provisioning the Persistent Volume with Database Gold Image and the `dbImage` used for deploying the Shard or Catalog Database by cloning should be same. + * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) + +Use the file: [snr_ssharding_shard_prov_clone_across_ads.yaml](./snr_ssharding_shard_prov_clone_across_ads.yaml) for this use case as below: + +1. Deploy the `snr_ssharding_shard_prov_clone_across_ads.yaml` file: + ```sh + kubectl apply -f snr_ssharding_shard_prov_clone_across_ads.yaml + ``` +2. Check the status of the deployment: + ```sh + # Check the status of the Kubernetes Pods: + kubectl get all -n shns + + # Check the logs of a particular pod. For example, to check status of pod "shard1-0": + kubectl logs -f pod/shard1-0 -n shns + diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md new file mode 100644 index 00000000..cf4240f7 --- /dev/null +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md @@ -0,0 +1,53 @@ +# Provisioning System managed Sharding Topology with Raft replication enabled by cloning database from your own Database Gold Image in the same Availability Domain(AD) + +**NOTE: RAFT Replication Feature is available only for Oracle 23c RDBMS and Oracle 23c GSM version.** + +**IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. + +In this test case, you provision the System managed Sharding Topology with Raft replication enabled while provisioning the Catalog and Shard Databases by cloning from an existing Oracle Database Gold Image created earlier. + +This use case applies when you are cloning from a Block Volume, and you can clone _only_ in the same availability domain (AD). The result is that the cloned shard database PODs can be created _only_ in the same AD where the Gold Image Block Volume is present. + +Choosing this option takes substantially less time during the Oracle Database Sharding Topology setup. + +**NOTE** For this step, the Persistent Volume that has the Oracle Database Gold Image is identified using its OCID. + +1. Check the OCID of the Persistent Volume provisioned earlier using below command: + + ```sh + kubectl get pv -n shns + ``` + +2. This example uses `snr_ssharding_shard_prov_clone.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: + +* Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` +* Three sharding Pods: `shard1`, `shard2` and `shard3` +* One Catalog Pod: `catalog` +* Namespace: `shns` +* Database Cloning from the Database Gold Image present in Persistent Volume having OCID: `ocid1.volume.oc1.phx.abyhqljr3z3w72t6ay5eud7d5w3kdfhktfp6gwb6euy5tzwfaxgmbvwqlvsq` +* `RAFT Replication` enabled + +NOTE: In this case, the Persistent Volume with DB Gold Image was provisioned in the Availability Domain `PHX-AD-1`. The Shards and Catalog will be provisioned in the same Availability Domain `PHX-AD-1` by cloning the database. + +In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) + * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. + * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `snr_ssharding_shard_prov_clone.yaml`. + * The `dbImage` used during provisioning the Persistent Volume with Database Gold Image and the `dbImage` used for deploying the Shard or Catalog Database by cloning should be same. + * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) + +**NOTE:** Provisioning the Sharded Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. + +Use the file: [snr_ssharding_shard_prov_clone.yaml](./snr_ssharding_shard_prov_clone.yaml) for this use case as below: + +1. Deploy the `snr_ssharding_shard_prov_clone.yaml` file: + ```sh + kubectl apply -f snr_ssharding_shard_prov_clone.yaml + ``` +2. Check the status of the deployment: + ```sh + # Check the status of the Kubernetes Pods: + kubectl get all -n shns + + # Check the logs of a particular pod. For example, to check status of pod "shard1-0": + kubectl logs -f pod/shard1-0 -n shns + ``` diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_chunks_specified.md b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_chunks_specified.md new file mode 100644 index 00000000..44972090 --- /dev/null +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_chunks_specified.md @@ -0,0 +1,43 @@ +# Provisioning System-Managed Sharding Topology with Raft replication enabled with number of chunks specified + +**NOTE: RAFT Replication Feature is available only for Oracle 23c RDBMS and Oracle 23c GSM version.** + +**IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. + +In this use case, the database is created automatically using DBCA during the provisioning of the shard databases and the catalog database when the Oracle Sharding topology with System-Managed with RAFT Replication enabled is deployed using Oracle Sharding controller. + +**NOTE** In this use case, because DBCA creates the database automatically during the deployment, the time required to create the database is greater than the time it takes when the database is created by cloning from a Database Gold Image. + +By default, the System-Managed with RAFT Replication deploys the Sharded Database with 360 chunks per Shard Database (because there are 3 chunks created for each replication unit). In this example, the Sharded Database will be deployed with non-default number of chunks specified using parameter `CATALOG_CHUNKS`. + +This example uses `snr_ssharding_shard_prov_chunks.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: + +* Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` +* Three sharding Pods: `shard1`, `shard2` and `shard3` +* One Catalog Pod: `catalog` +* Total number of chunks as `120` specified by variable `CATALOG_CHUNKS` (it will be 120 chunks per shard) +* Namespace: `shns` +* `RAFT Replication` enabled + + +In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) + * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. + * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `snr_ssharding_shard_prov.yaml`. + * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) + * In case you want to use the [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then you will need to add the additional parameter `dbEdition: "free"` to the below .yaml file. + + +Use the file: [snr_ssharding_shard_prov_chunks.yaml](./snr_ssharding_shard_prov_chunks.yaml) for this use case as below: + +1. Deploy the `snr_ssharding_shard_prov_chunks.yaml` file: + ```sh + kubectl apply -f snr_ssharding_shard_prov_chunks.yaml + ``` +1. Check the status of the deployment: + ```sh + # Check the status of the Kubernetes Pods: + kubectl get all -n shns + + # Check the logs of a particular pod. For example, to check status of pod "shard1-0": + kubectl logs -f pod/shard1-0 -n shns + ``` diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_control_on_resources.md b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_control_on_resources.md new file mode 100644 index 00000000..9cfd6afb --- /dev/null +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_control_on_resources.md @@ -0,0 +1,47 @@ +# Provisioning System-Managed Sharding Topology with Raft replication enabled with additional control on resources like Memory and CPU allocated to Pods + +**NOTE: RAFT Replication Feature is available only for Oracle 23c RDBMS and Oracle 23c GSM version.** + +**IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. + +In this use case, there are additional tags used to control resources such as CPU and Memory used by the different Pods when the Oracle Sharding topology with System-Managed with RAFT Replication is deployed using Oracle Sharding controller. + +This example uses `snr_ssharding_shard_prov_memory_cpu.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: + +* Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` +* Three sharding Pods: `shard1`, `shard2` and `shard3` +* One Catalog Pod: `catalog` +* Namespace: `shns` +* Tags `memory` and `cpu` to control the Memory and CPU of the PODs +* Additional tags `INIT_SGA_SIZE` and `INIT_PGA_SIZE` to control the SGA and PGA allocation at the database level +* `RAFT Replication` enabled + +In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) + * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. + * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `snr_ssharding_shard_prov_memory_cpu.yaml`. + * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) + * In case you want to use the [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then you will need to add the additional parameter `dbEdition: "free"` to the below .yaml file. + +**NOTE:** For Oracle Database 23ai Free, you can control the `CPU` and `Memory` allocation of the PODs using tags `cpu` and `memory` respectively but tags `INIT_SGA_SIZE` and `INIT_PGA_SIZE` to control the SGA and PGA allocation at the database level are `not` supported. + +Use the YAML file [snr_ssharding_shard_prov_memory_cpu.yaml](./snr_ssharding_shard_prov_memory_cpu.yaml). + +1. Deploy the `snr_ssharding_shard_prov_memory_cpu.yaml` file: + + ```sh + kubectl apply -f snr_ssharding_shard_prov_memory_cpu.yaml + ``` + +1. Check the details of a POD. For example: To check the details of Pod `shard1-0`: + + ```sh + kubectl describe pod/shard1-0 -n shns + ``` +3. Check the status of the deployment: + ```sh + # Check the status of the Kubernetes Pods: + kubectl get all -n shns + + # Check the logs of a particular pod. For example, to check status of pod "shard1-0": + kubectl logs -f pod/shard1-0 -n shns + ``` diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_notification_using_oci_notification.md b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_notification_using_oci_notification.md new file mode 100644 index 00000000..d4cb11de --- /dev/null +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_notification_using_oci_notification.md @@ -0,0 +1,87 @@ +# Provisioning System managed Sharding Topology with Raft replication enabled and send Notification using OCI Notification Service + +**NOTE: RAFT Replication Feature is available only for Oracle 23c RDBMS and Oracle 23c GSM version.** + +**IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. + +This use case demonstrates how to use a notification service like OCI Notification service to send an email notification when a particular operation is completed on an Oracle Database sharding topology provisioned using the Oracle Database sharding controller. + +This example uses `snr_ssharding_shard_prov_send_notification.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: + +* Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` +* Three sharding Pods: `shard1`, `shard2` and `shard3` +* One Catalog Pod: `catalog` +* Namespace: `shns` +* Database Cloning from the `BLOCK VOLUME FULL BACKUP` of the Persistent Volume that has the Database Gold Image created earlier. +* OCID of the Block Volume Backup: `ocid1.volumebackup.oc1.phx.abyhqljrxtv7tu5swqb3lzc7vpzwbwzdktd2y4k2vjjy2srmgu2w7bqdftjq` +* Configmap to send notification email when a particular operation is completed. For example: When a shard is added. +* `RAFT Replication` enabled + +**NOTE:** + +* The notification will be sent using a configmap created with the credentials of the OCI user account in this use case. + +We will create a topic in Notification Service of the OCI Console and use its OCID. + +To do this: + +1. Create a `configmap_data.txt` file, such as the following, which has the OCI User details that will be used to send notfication: + + ```sh + user=ocid1.user.oc1........fx7omxfq + fingerprint=fa:18:98:...............:8a + tenancy=ocid1.tenancy.oc1..aaaa.......orpn7inq + region=us-phoenix-1 + topicid=ocid1.onstopic.oc1.phx.aaa............6xrq + ``` +2. Create a configmap using the below command using the file created above: + ```sh + kubectl create configmap onsconfigmap --from-file=./configmap_data.txt -n shns + ``` + +3. Create a key file `priavatekey` having the PEM key of the OCI user being used to send notification: + ```sh + -----BEGIN PRIVATE KEY-G---- + MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQCXYxA0DJvEwtVR + +o4OxrunL3L2NZJRADTFR+TDHqrNF1JwbaFBizSdL+EXbxQW1faZs5lXZ/sVmQF9 + . + . + . + zn/xWC0FzXGRzfvYHhq8XT3omf6L47KqIzqo3jDKdgvVq4u+lb+fXJlhj6Rwi99y + QEp36HnZiUxAQnR331DacN+YSTE+vpzSwZ38OP49khAB1xQsbiv1adG7CbNpkxpI + nS7CkDLg4Hcs4b9bGLHYJVY= + -----END PRIVATE KEY----- + ``` +4. Use the key file `privatekey` to create a Kubernetes secret in namespace `shns`: + + ```sh + kubectl create secret generic my-secret --from-file=./privatekey -n shns + ``` + +5. Use this command to check details of the secret that you created: + + ```sh + kubectl describe secret my-secret -n shns + ``` + +In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) + * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. + * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `snr_ssharding_shard_prov_send_notification.yaml`. + * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) + * In case you want to use the [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then you will need to add the additional parameter `dbEdition: "free"` to the below .yaml file. + +**NOTE:** Provisioning the Sharded Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. + +Use the file: [snr_ssharding_shard_prov_send_notification.yaml](./snr_ssharding_shard_prov_send_notification.yaml) for this use case as below: + +1. Deploy the `snr_ssharding_shard_prov_send_notification.yaml` file: + ```sh + kubectl apply -f snr_ssharding_shard_prov_send_notification.yaml + ``` +2. Check the status of the deployment: + ```sh + # Check the status of the Kubernetes Pods: + kubectl get all -n shns + + # Check the logs of a particular pod. For example, to check status of pod "shard1-0": + kubectl logs -f pod/shard1-0 -n shns diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_without_db_gold_image.md b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_without_db_gold_image.md new file mode 100644 index 00000000..892741a5 --- /dev/null +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_without_db_gold_image.md @@ -0,0 +1,40 @@ +# Provisioning System-Managed Sharding Topology with Raft replication enabled without Database Gold Image + +**NOTE: RAFT Replication Feature is available only for Oracle 23c RDBMS and Oracle 23c GSM version.** + +**IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. + +In this use case, the database is created automatically using DBCA during the provisioning of the shard databases and the catalog database when the Oracle Sharding topology with System-Managed with RAFT Replication enabled is deployed using Oracle Sharding controller. + +**NOTE** In this use case, because DBCA creates the database automatically during the deployment, the time required to create the database is greater than the time it takes when the database is created by cloning from a Database Gold Image. + +This example uses `snr_ssharding_shard_prov.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: + +* Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` +* Three sharding Pods: `shard1`, `shard2` and `shard3` +* One Catalog Pod: `catalog` +* Namespace: `shns` +* `RAFT Replication` enabled + + +In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) + * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. + * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `snr_ssharding_shard_prov.yaml`. + * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) + * In case you want to use the [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then you will need to add the additional parameter `dbEdition: "free"` to the below .yaml file. + + +Use the file: [snr_ssharding_shard_prov.yaml](./snr_ssharding_shard_prov.yaml) for this use case as below: + +1. Deploy the `snr_ssharding_shard_prov.yaml` file: + ```sh + kubectl apply -f snr_ssharding_shard_prov.yaml + ``` +1. Check the status of the deployment: + ```sh + # Check the status of the Kubernetes Pods: + kubectl get all -n shns + + # Check the logs of a particular pod. For example, to check status of pod "shard1-0": + kubectl logs -f pod/shard1-0 -n shns + ``` diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_scale_in_delete_an_existing_shard.md b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_scale_in_delete_an_existing_shard.md new file mode 100644 index 00000000..fe3157ec --- /dev/null +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_scale_in_delete_an_existing_shard.md @@ -0,0 +1,50 @@ +# Scale In - Delete an existing Shard from a working Oracle Sharded Database provisioned earlier with System-Managed Sharding with RAFT reolication enabled + +**NOTE: RAFT Replication Feature is available only for Oracle 23c RDBMS and Oracle 23c GSM version.** + +**IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. + +This use case demonstrates how to delete an existing Shard from an existing Oracle Database sharding topology with System-Managed with RAFT Replication enabled provisioned using Oracle Database Sharding controller. + +**NOTE** The deletion of a shard is done after verifying the Chunks have been moved out of that shard. + +In this use case, the existing database Sharding is having: + +* Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` +* Five sharding Pods: `shard1`,`shard2`,`shard3`,`shard4` and `shard5` +* One Catalog Pod: `catalog` +* Namespace: `shns` +* `RAFT Replication` enabled + +In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) + * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. + * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `snr_ssharding_shard_prov_delshard.yaml`. + * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) + * In case you want to use the [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then you will need to add the additional parameter `dbEdition: "free"` to the below .yaml file. + +NOTE: Use tag `isDelete: enable` to delete the shard you want. + +This use case deletes the shard `shard4` from the above Sharding Topology. + +Use the file: [snr_ssharding_shard_prov_delshard.yaml](./snr_ssharding_shard_prov_delshard.yaml) for this use case as below: + +1. Deploy the `snr_ssharding_shard_prov_delshard.yaml` file: + ```sh + kubectl apply -f snr_ssharding_shard_prov_delshard.yaml + ``` +2. Check the status of the deployment: + ```sh + # Check the status of the Kubernetes Pods: + kubectl get all -n shns + +**NOTE:** After you apply `snr_ssharding_shard_prov_delshard.yaml`, the change may not be visible immediately. When the shard is removed, first the chunks will be moved out of that shard that is going to be deleted. + +To monitor the chunk movement, use the following command: + +```sh +# Switch to the primary GSM Container: +kubectl exec -i -t gsm1-0 -n shns /bin/bash + +# Check the status of the chunks and repeat to observe the chunk movement: +gdsctl config chunks +``` diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_scale_out_add_shards.md b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_scale_out_add_shards.md new file mode 100644 index 00000000..03423e72 --- /dev/null +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_scale_out_add_shards.md @@ -0,0 +1,37 @@ +# Scale Out - Add Shards to an existing Oracle Sharded Database provisioned earlier with System-Managed Sharding with RAFT replication enabled + +**NOTE: RAFT Replication Feature is available only for Oracle 23c RDBMS and Oracle 23c GSM version.** + +**IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. + +This use case demonstrates adding a new shard to an existing Oracle Database sharding topology with System-Managed with RAFT Replication enabled provisioned earlier using Oracle Database Sharding controller. + +In this use case, the existing Oracle Database sharding topology is having: + +* Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` +* Three sharding Pods: `shard1`, `shard2` and `shard3` +* One Catalog Pod: `catalog` +* Namespace: `shns` +* `RAFT Replication` enabled + +In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) + * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. + * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `snr_ssharding_shard_prov_extshard.yaml`. + * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) + * If the existing Sharding Topology was deployed using [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then the additional parameter `dbEdition: "free"` will be needed for the below .yaml file as well. + +This use case adds two new shards `shard4`,`shard5` to above Sharding Topology. + +Use the file: [snr_ssharding_shard_prov_extshard.yaml](./snr_ssharding_shard_prov_extshard.yaml) for this use case as below: + +1. Deploy the `snr_ssharding_shard_prov_extshard.yaml` file: + ```sh + kubectl apply -f snr_ssharding_shard_prov_extshard.yaml + ``` +2. Check the status of the deployment: + ```sh + # Check the status of the Kubernetes Pods: + kubectl get all -n shns + + # Check the logs of a particular pod. For example, to check status of pod "shard4-0": + kubectl logs -f pod/shard4-0 -n shns diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov.yaml b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov.yaml new file mode 100644 index 00000000..efe3abec --- /dev/null +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov.yaml @@ -0,0 +1,58 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +--- +apiVersion: database.oracle.com/v1alpha1 +kind: ShardingDatabase +metadata: + name: shardingdatabase-sample + namespace: shns +spec: + shard: + - name: shard1 + storageSizeInGb: 50 + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + - name: shard2 + storageSizeInGb: 50 + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + - name: shard3 + storageSizeInGb: 50 + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + catalog: + - name: catalog + storageSizeInGb: 50 + imagePullPolicy: "Always" + gsm: + - name: gsm1 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: primary + - name: gsm2 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: standby + storageClass: oci + dbImage: container-registry.oracle.com/database/enterprise:latest + dbImagePullSecret: ocr-reg-cred + gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImagePullSecret: ocr-reg-cred + replicationType: "native" + isExternalSvc: False + isDeleteOraPvc: True + dbSecret: + name: db-user-pass-rsa + pwdFileName: pwdfile.enc + keyFileName: key.pem + gsmService: + - name: oltp_rw_svc + role: primary + - name: oltp_ro_svc + role: primary + namespace: shns \ No newline at end of file diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_chunks.yaml b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_chunks.yaml new file mode 100644 index 00000000..a79eafdc --- /dev/null +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_chunks.yaml @@ -0,0 +1,61 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +--- +apiVersion: database.oracle.com/v1alpha1 +kind: ShardingDatabase +metadata: + name: shardingdatabase-sample + namespace: shns +spec: + shard: + - name: shard1 + storageSizeInGb: 50 + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + - name: shard2 + storageSizeInGb: 50 + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + - name: shard3 + storageSizeInGb: 50 + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + catalog: + - name: catalog + storageSizeInGb: 50 + imagePullPolicy: "Always" + envVars: + - name: "CATALOG_CHUNKS" + value: "120" + gsm: + - name: gsm1 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: primary + - name: gsm2 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: standby + storageClass: oci + dbImage: container-registry.oracle.com/database/enterprise:latest + dbImagePullSecret: ocr-reg-cred + gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImagePullSecret: ocr-reg-cred + replicationType: "native" + isExternalSvc: False + isDeleteOraPvc: True + dbSecret: + name: db-user-pass-rsa + pwdFileName: pwdfile.enc + keyFileName: key.pem + gsmService: + - name: oltp_rw_svc + role: primary + - name: oltp_ro_svc + role: primary + namespace: shns diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_clone.yaml b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_clone.yaml new file mode 100644 index 00000000..218fda0a --- /dev/null +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_clone.yaml @@ -0,0 +1,83 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +--- +apiVersion: database.oracle.com/v1alpha1 +kind: ShardingDatabase +metadata: + name: shardingdatabase-sample + namespace: shns +spec: + shard: + - name: shard1 + storageSizeInGb: 50 + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volume.oc1.phx.abyhqljr3z3w72t6ay5eud7d5w3kdfhktfp6gwb6euy5tzwfaxgmbvwqlvsq + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + - name: shard2 + storageSizeInGb: 50 + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volume.oc1.phx.abyhqljr3z3w72t6ay5eud7d5w3kdfhktfp6gwb6euy5tzwfaxgmbvwqlvsq + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + - name: shard3 + storageSizeInGb: 50 + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volume.oc1.phx.abyhqljr3z3w72t6ay5eud7d5w3kdfhktfp6gwb6euy5tzwfaxgmbvwqlvsq + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + catalog: + - name: catalog + storageSizeInGb: 50 + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volume.oc1.phx.abyhqljr3z3w72t6ay5eud7d5w3kdfhktfp6gwb6euy5tzwfaxgmbvwqlvsq + imagePullPolicy: "Always" + gsm: + - name: gsm1 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: primary + - name: gsm2 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: standby + storageClass: oci + dbImage: container-registry.oracle.com/database/enterprise:latest + dbImagePullSecret: ocr-reg-cred + gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImagePullSecret: ocr-reg-cred + replicationType: "native" + isExternalSvc: False + isDeleteOraPvc: True + isClone: True + dbSecret: + name: db-user-pass-rsa + pwdFileName: pwdfile.enc + keyFileName: key.pem + gsmService: + - name: oltp_rw_svc + role: primary + - name: oltp_ro_svc + role: primary + namespace: shns \ No newline at end of file diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_clone_across_ads.yaml b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_clone_across_ads.yaml new file mode 100644 index 00000000..4eb3954a --- /dev/null +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_clone_across_ads.yaml @@ -0,0 +1,91 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +--- +apiVersion: database.oracle.com/v1alpha1 +kind: ShardingDatabase +metadata: + name: shardingdatabase-sample + namespace: shns +spec: + shard: + - name: shard1 + storageSizeInGb: 50 + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volumebackup.oc1.phx.abyhqljrxtv7tu5swqb3lzc7vpzwbwzdktd2y4k2vjjy2srmgu2w7bqdftjq + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + - name: shard2 + storageSizeInGb: 50 + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-2" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-2" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volumebackup.oc1.phx.abyhqljrxtv7tu5swqb3lzc7vpzwbwzdktd2y4k2vjjy2srmgu2w7bqdftjq + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + - name: shard3 + storageSizeInGb: 50 + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-3" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-3" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volumebackup.oc1.phx.abyhqljrxtv7tu5swqb3lzc7vpzwbwzdktd2y4k2vjjy2srmgu2w7bqdftjq + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + catalog: + - name: catalog + storageSizeInGb: 50 + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volumebackup.oc1.phx.abyhqljrxtv7tu5swqb3lzc7vpzwbwzdktd2y4k2vjjy2srmgu2w7bqdftjq + imagePullPolicy: "Always" + gsm: + - name: gsm1 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: primary + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-3" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-3" + - name: gsm2 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: standby + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-3" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-3" + storageClass: oci + dbImage: container-registry.oracle.com/database/enterprise:latest + dbImagePullSecret: ocr-reg-cred + gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImagePullSecret: ocr-reg-cred + replicationType: "native" + isExternalSvc: False + isDeleteOraPvc: True + isClone: True + dbSecret: + name: db-user-pass-rsa + pwdFileName: pwdfile.enc + keyFileName: key.pem + gsmService: + - name: oltp_rw_svc + role: primary + - name: oltp_ro_svc + role: primary + namespace: shns \ No newline at end of file diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_delshard.yaml b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_delshard.yaml new file mode 100644 index 00000000..145ef616 --- /dev/null +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_delshard.yaml @@ -0,0 +1,69 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +--- +apiVersion: database.oracle.com/v1alpha1 +kind: ShardingDatabase +metadata: + name: shardingdatabase-sample + namespace: shns +spec: + shard: + - name: shard1 + storageSizeInGb: 50 + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + - name: shard2 + storageSizeInGb: 50 + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + - name: shard3 + storageSizeInGb: 50 + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + - name: shard4 + isDelete: enable + storageSizeInGb: 50 + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + - name: shard5 + storageSizeInGb: 50 + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + catalog: + - name: catalog + storageSizeInGb: 50 + imagePullPolicy: "Always" + gsm: + - name: gsm1 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: primary + - name: gsm2 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: standby + storageClass: oci + dbImage: container-registry.oracle.com/database/enterprise:latest + dbImagePullSecret: ocr-reg-cred + gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImagePullSecret: ocr-reg-cred + replicationType: "native" + isExternalSvc: False + isDeleteOraPvc: True + dbSecret: + name: db-user-pass-rsa + pwdFileName: pwdfile.enc + keyFileName: key.pem + gsmService: + - name: oltp_rw_svc + role: primary + - name: oltp_ro_svc + role: primary + namespace: shns \ No newline at end of file diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_extshard.yaml b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_extshard.yaml new file mode 100644 index 00000000..ea0c05a5 --- /dev/null +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_extshard.yaml @@ -0,0 +1,68 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +--- +apiVersion: database.oracle.com/v1alpha1 +kind: ShardingDatabase +metadata: + name: shardingdatabase-sample + namespace: shns +spec: + shard: + - name: shard1 + storageSizeInGb: 50 + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + - name: shard2 + storageSizeInGb: 50 + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + - name: shard3 + storageSizeInGb: 50 + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + - name: shard4 + storageSizeInGb: 50 + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + - name: shard5 + storageSizeInGb: 50 + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + catalog: + - name: catalog + storageSizeInGb: 50 + imagePullPolicy: "Always" + gsm: + - name: gsm1 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: primary + - name: gsm2 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: standby + storageClass: oci + dbImage: container-registry.oracle.com/database/enterprise:latest + dbImagePullSecret: ocr-reg-cred + gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImagePullSecret: ocr-reg-cred + replicationType: "native" + isExternalSvc: False + isDeleteOraPvc: True + dbSecret: + name: db-user-pass-rsa + pwdFileName: pwdfile.enc + keyFileName: key.pem + gsmService: + - name: oltp_rw_svc + role: primary + - name: oltp_ro_svc + role: primary + namespace: shns \ No newline at end of file diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_memory_cpu.yaml b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_memory_cpu.yaml new file mode 100644 index 00000000..5c15c724 --- /dev/null +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_memory_cpu.yaml @@ -0,0 +1,89 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +--- +apiVersion: database.oracle.com/v1alpha1 +kind: ShardingDatabase +metadata: + name: shardingdatabase-sample + namespace: shns +spec: + shard: + - name: shard1 + storageSizeInGb: 50 + resources: + requests: + memory: "1000Mi" + cpu: "1000m" + envVars: + - name: "INIT_SGA_SIZE" + value: "600" + - name: "INIT_PGA_SIZE" + value: "400" + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + - name: shard2 + storageSizeInGb: 50 + resources: + requests: + memory: "1000Mi" + cpu: "1000m" + envVars: + - name: "INIT_SGA_SIZE" + value: "600" + - name: "INIT_PGA_SIZE" + value: "400" + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + - name: shard3 + storageSizeInGb: 50 + resources: + requests: + memory: "1000Mi" + cpu: "1000m" + envVars: + - name: "INIT_SGA_SIZE" + value: "600" + - name: "INIT_PGA_SIZE" + value: "400" + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + catalog: + - name: catalog + storageSizeInGb: 50 + resources: + requests: + memory: "1000Mi" + cpu: "1000m" + imagePullPolicy: "Always" + gsm: + - name: gsm1 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: primary + - name: gsm2 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: standby + storageClass: oci + dbImage: container-registry.oracle.com/database/enterprise:latest + dbImagePullSecret: ocr-reg-cred + gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImagePullSecret: ocr-reg-cred + replicationType: "native" + isExternalSvc: False + isDeleteOraPvc: True + dbSecret: + name: db-user-pass-rsa + pwdFileName: pwdfile.enc + keyFileName: key.pem + gsmService: + - name: oltp_rw_svc + role: primary + - name: oltp_ro_svc + role: primary + namespace: shns \ No newline at end of file diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_send_notification.yaml b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_send_notification.yaml new file mode 100644 index 00000000..50c85443 --- /dev/null +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_send_notification.yaml @@ -0,0 +1,85 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +--- +apiVersion: database.oracle.com/v1alpha1 +kind: ShardingDatabase +metadata: + name: shardingdatabase-sample + namespace: shns +spec: + shard: + - name: shard1 + storageSizeInGb: 50 + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volumebackup.oc1.phx.abyhqljrxtv7tu5swqb3lzc7vpzwbwzdktd2y4k2vjjy2srmgu2w7bqdftjq + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + - name: shard2 + storageSizeInGb: 50 + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-2" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-2" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volumebackup.oc1.phx.abyhqljrxtv7tu5swqb3lzc7vpzwbwzdktd2y4k2vjjy2srmgu2w7bqdftjq + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + - name: shard3 + storageSizeInGb: 50 + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-3" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-3" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volumebackup.oc1.phx.abyhqljrxtv7tu5swqb3lzc7vpzwbwzdktd2y4k2vjjy2srmgu2w7bqdftjq + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + catalog: + - name: catalog + storageSizeInGb: 50 + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-1" + pvAnnotations: + volume.beta.kubernetes.io/oci-volume-source: ocid1.volumebackup.oc1.phx.abyhqljrxtv7tu5swqb3lzc7vpzwbwzdktd2y4k2vjjy2srmgu2w7bqdftjq + imagePullPolicy: "Always" + gsm: + - name: gsm1 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: primary + - name: gsm2 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: standby + storageClass: oci + dbImage: container-registry.oracle.com/database/enterprise:latest + dbImagePullSecret: ocr-reg-cred + gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImagePullSecret: ocr-reg-cred + replicationType: "native" + isExternalSvc: False + isDeleteOraPvc: True + isClone: True + dbSecret: + name: db-user-pass-rsa + pwdFileName: pwdfile.enc + keyFileName: key.pem + nsConfigMap: onsconfigmap + nsSecret: my-secret + gsmService: + - name: oltp_rw_svc + role: primary + - name: oltp_ro_svc + role: primary + namespace: shns \ No newline at end of file diff --git a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md index 9ae05d50..64e2f4eb 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md +++ b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md @@ -1,8 +1,8 @@ -# Provisioning Oracle Sharded Database with System Sharding by cloning database from your own Database Gold Image across Availability Domains(ADs) +# Provisioning Oracle Sharded Database with System-Managed Sharding by cloning database from your own Database Gold Image across Availability Domains(ADs) **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. -In this test case, you provision the Oracle Database sharding topology with System Sharding while provisioning the Catalog and Shard Databases by cloning from an existing Oracle Database Gold Image created earlier. +In this test case, you provision the Oracle Database sharding topology with System-Managed Sharding while provisioning the Catalog and Shard Databases by cloning from an existing Oracle Database Gold Image created earlier. This use case applies when you want to provision the database Pods on a Kubernetes Node in any availability domain (AD), which can also be different from the availability domain (AD) of the Block Volume that has the Oracle Database Gold Image provisioned earlier. @@ -30,6 +30,9 @@ NOTE: NOTE: In this case, the Persistent Volume with DB Gold Image was provisioned in the Availability Domain `PHX-AD-1`. The Shards and Catalog will be provisioned across multiple Availability Domains by cloning the database. + +**NOTE:** Provisioning the Sharded Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. + In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `ssharding_shard_prov_clone_across_ads.yaml`. diff --git a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md index cb67addb..f7aef949 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md +++ b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md @@ -1,4 +1,4 @@ -# Provisioning Oracle Sharded Database with System Sharding by cloning database from your own Database Gold Image in the same Availability Domain(AD) +# Provisioning Oracle Sharded Database with System-Managed Sharding by cloning database from your own Database Gold Image in the same Availability Domain(AD) **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. @@ -24,6 +24,8 @@ Choosing this option takes substantially less time during the Oracle Database Sh * Namespace: `shns` * Database Cloning from the Database Gold Image present in Persistent Volume having OCID: `ocid1.volume.oc1.phx.abyhqljr3z3w72t6ay5eud7d5w3kdfhktfp6gwb6euy5tzwfaxgmbvwqlvsq` +**NOTE:** Provisioning the Sharded Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. + NOTE: In this case, the Persistent Volume with DB Gold Image was provisioned in the Availability Domain `PHX-AD-1`. The Shards and Catalog will be provisioned in the same Availability Domain `PHX-AD-1` by cloning the database. In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) diff --git a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_chunks_specified.md b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_chunks_specified.md new file mode 100644 index 00000000..0c6ea8fe --- /dev/null +++ b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_chunks_specified.md @@ -0,0 +1,40 @@ +# Provisioning Oracle Sharded Database with System-Managed Sharding with number of chunks specified + +**IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. + +In this use case, the database is created automatically using DBCA during the provisioning of the shard databases and the catalog database when the Oracle Sharding topology with System-Managed Sharding is deployed using Oracle Sharding controller. + +**NOTE:** In this use case, because DBCA creates the database automatically during the deployment, the time required to create the database is greater than the time it takes when the database is created by cloning from a Database Gold Image. + +By default, the System-Managed Sharding deploys the Sharded Database with 120 chunks per Shard Database. If, for example, we have three shards in the Sharded Database, it will be total of 360 chunks. In this example, the Sharded Database will be deployed with non-default number of chunks specified using parameter `CATALOG_CHUNKS`. + +This example uses `ssharding_shard_prov_chunks.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: + +* Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` +* Three sharding Pods: `shard1`, `shard2` and `shard3` +* One Catalog Pod: `catalog` +* Total number of chunks as `120` specified by variable `CATALOG_CHUNKS` (it will be 40 chunks per shard) +* Namespace: `shns` + + +In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) + * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. + * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `ssharding_shard_prov.yaml`. + * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) + * In case you want to use the [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then you will need to add the additional parameter `dbEdition: "free"` to the below .yaml file. + + +Use the file: [ssharding_shard_prov_chunks.yaml](./ssharding_shard_prov_chunks.yaml) for this use case as below: + +1. Deploy the `ssharding_shard_prov_chunks.yaml` file: + ```sh + kubectl apply -f ssharding_shard_prov_chunks.yaml + ``` +1. Check the status of the deployment: + ```sh + # Check the status of the Kubernetes Pods: + kubectl get all -n shns + + # Check the logs of a particular pod. For example, to check status of pod "shard1-0": + kubectl logs -f pod/shard1-0 -n shns + ``` diff --git a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_control_on_resources.md b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_control_on_resources.md index e14e76a4..c4f45a48 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_control_on_resources.md +++ b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_control_on_resources.md @@ -1,8 +1,8 @@ -# Provisioning Oracle Sharded Database with System Sharding with additional control on resources like Memory and CPU allocated to Pods +# Provisioning Oracle Sharded Database with System-Managed Sharding with additional control on resources like Memory and CPU allocated to Pods **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. -In this use case, there are additional tags used to control resources such as CPU and Memory used by the different Pods when the Oracle Sharding topology with System Sharding is deployed using Oracle Sharding controller. +In this use case, there are additional tags used to control resources such as CPU and Memory used by the different Pods when the Oracle Sharding topology with System-Managed Sharding is deployed using Oracle Sharding controller. This example uses `ssharding_shard_prov_memory_cpu.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: @@ -17,6 +17,9 @@ In this example, we are using pre-built Oracle Database and Global Data Services * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `ssharding_shard_prov_memory_cpu.yaml`. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) + * In case you want to use the [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then you will need to add the additional parameter `dbEdition: "free"` to the below .yaml file. + + **NOTE:** For Oracle Database 23ai Free, you can control the `CPU` and `Memory` allocation of the PODs using tags `cpu` and `memory` respectively but tags `INIT_SGA_SIZE` and `INIT_PGA_SIZE` to control the SGA and PGA allocation at the database level are `not` supported. Use the YAML file [ssharding_shard_prov_memory_cpu.yaml](./ssharding_shard_prov_memory_cpu.yaml). diff --git a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_notification_using_oci_notification.md b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_notification_using_oci_notification.md index 90a8f803..1a6a1ee3 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_notification_using_oci_notification.md +++ b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_notification_using_oci_notification.md @@ -1,4 +1,4 @@ -# Provisioning Oracle Sharded Database with System Sharding and send Notification using OCI Notification Service +# Provisioning Oracle Sharded Database with System-Managed Sharding and send Notification using OCI Notification Service **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. @@ -65,6 +65,9 @@ In this example, we are using pre-built Oracle Database and Global Data Services * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `ssharding_shard_prov_send_notification.yaml`. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) + * In case you want to use the [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then you will need to add the additional parameter `dbEdition: "free"` to the below .yaml file. + +**NOTE:** Provisioning the Sharded Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. Use the file: [ssharding_shard_prov_send_notification.yaml](./ssharding_shard_prov_send_notification.yaml) for this use case as below: diff --git a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_without_db_gold_image.md b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_without_db_gold_image.md index 8f10fd8c..b223d1af 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_without_db_gold_image.md +++ b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_without_db_gold_image.md @@ -1,10 +1,10 @@ -# Provisioning Oracle Sharded Database with System Sharding without Database Gold Image +# Provisioning Oracle Sharded Database with System-Managed Sharding without Database Gold Image **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. -In this use case, the database is created automatically using DBCA during the provisioning of the shard databases and the catalog database when the Oracle Sharding topology with System Sharding is deployed using Oracle Sharding controller. +In this use case, the database is created automatically using DBCA during the provisioning of the shard databases and the catalog database when the Oracle Sharding topology with System-Managed Sharding is deployed using Oracle Sharding controller. -**NOTE** In this use case, because DBCA creates the database automatically during the deployment, the time required to create the database is greater than the time it takes when the database is created by cloning from a Database Gold Image. +**NOTE:** In this use case, because DBCA creates the database automatically during the deployment, the time required to create the database is greater than the time it takes when the database is created by cloning from a Database Gold Image. This example uses `ssharding_shard_prov.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: @@ -18,6 +18,7 @@ In this example, we are using pre-built Oracle Database and Global Data Services * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `ssharding_shard_prov.yaml`. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) + * In case you want to use the [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then you will need to add the additional parameter `dbEdition: "free"` to the below .yaml file. Use the file: [ssharding_shard_prov.yaml](./ssharding_shard_prov.yaml) for this use case as below: @@ -33,4 +34,4 @@ Use the file: [ssharding_shard_prov.yaml](./ssharding_shard_prov.yaml) for this # Check the logs of a particular pod. For example, to check status of pod "shard1-0": kubectl logs -f pod/shard1-0 -n shns - ``` + ``` \ No newline at end of file diff --git a/docs/sharding/provisioning/system_sharding/ssharding_scale_in_delete_an_existing_shard.md b/docs/sharding/provisioning/system_sharding/ssharding_scale_in_delete_an_existing_shard.md index 4d5713d4..bca34253 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_scale_in_delete_an_existing_shard.md +++ b/docs/sharding/provisioning/system_sharding/ssharding_scale_in_delete_an_existing_shard.md @@ -1,8 +1,8 @@ -# Scale In - Delete an existing Shard from a working Oracle Sharded Database provisioned earlier with System Sharding +# Scale In - Delete an existing Shard from a working Oracle Sharded Database provisioned earlier with System-Managed Sharding **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. -This use case demonstrates how to delete an existing Shard from an existing Oracle Database sharding topology with System Sharding provisioned using Oracle Database Sharding controller. +This use case demonstrates how to delete an existing Shard from an existing Oracle Database sharding topology with System-Managed Sharding provisioned using Oracle Database Sharding controller. **NOTE** The deletion of a shard is done after verifying the Chunks have been moved out of that shard. @@ -17,8 +17,9 @@ In this example, we are using pre-built Oracle Database and Global Data Services * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `ssharding_shard_prov_delshard.yaml`. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) + * In case you want to use the [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then you will need to add the additional parameter `dbEdition: "free"` to the below .yaml file. -NOTE: Use tag `isDelete: true` to delete the shard you want. +NOTE: Use tag `isDelete: enable` to delete the shard you want. This use case deletes the shard `shard4` from the above Sharding Topology. diff --git a/docs/sharding/provisioning/system_sharding/ssharding_scale_out_add_shards.md b/docs/sharding/provisioning/system_sharding/ssharding_scale_out_add_shards.md index 5c349847..1db8e6c3 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_scale_out_add_shards.md +++ b/docs/sharding/provisioning/system_sharding/ssharding_scale_out_add_shards.md @@ -1,8 +1,8 @@ -# Scale Out - Add Shards to an existing Oracle Sharded Database provisioned earlier with System Sharding +# Scale Out - Add Shards to an existing Oracle Sharded Database provisioned earlier with System-Managed Sharding **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. -This use case demonstrates adding a new shard to an existing Oracle Database sharding topology with System Sharding provisioned earlier using Oracle Database Sharding controller. +This use case demonstrates adding a new shard to an existing Oracle Database sharding topology with System-Managed Sharding provisioned earlier using Oracle Database Sharding controller. In this use case, the existing Oracle Database sharding topology is having: @@ -15,6 +15,7 @@ In this example, we are using pre-built Oracle Database and Global Data Services * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `ssharding_shard_prov_extshard.yaml`. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) + * If the existing Sharding Topology was deployed using [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then the additional parameter `dbEdition: "free"` will be needed for the below .yaml file as well. This use case adds two new shards `shard4`,`shard5` to above Sharding Topology. diff --git a/docs/sharding/provisioning/system_sharding/ssharding_shard_prov.yaml b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov.yaml index f803ba42..7d4e16ec 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_shard_prov.yaml +++ b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov.yaml @@ -55,4 +55,3 @@ spec: - name: oltp_ro_svc role: primary namespace: shns - diff --git a/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_clone_across_ads.yaml b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_clone_across_ads.yaml index d2ae0ba9..b7dd1397 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_clone_across_ads.yaml +++ b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_clone_across_ads.yaml @@ -58,10 +58,18 @@ spec: imagePullPolicy: "Always" storageSizeInGb: 50 region: primary + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-3" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-3" - name: gsm2 imagePullPolicy: "Always" storageSizeInGb: 50 region: standby + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-3" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-3" storageClass: oci dbImage: container-registry.oracle.com/database/enterprise:latest dbImagePullSecret: ocr-reg-cred @@ -79,4 +87,4 @@ spec: role: primary - name: oltp_ro_svc role: primary - namespace: shns \ No newline at end of file + namespace: shns diff --git a/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_delshard.yaml b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_delshard.yaml index 5f600d3f..75caca31 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_delshard.yaml +++ b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_delshard.yaml @@ -26,7 +26,7 @@ spec: shardGroup: shardgroup1 shardRegion: primary - name: shard4 - isDelete: True + isDelete: enable storageSizeInGb: 50 imagePullPolicy: "Always" shardGroup: shardgroup1 diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_from_gold_image_across_ads.md b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_from_gold_image_across_ads.md index 0beecbca..9b2905e8 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_from_gold_image_across_ads.md +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_from_gold_image_across_ads.md @@ -37,6 +37,8 @@ In this example, we are using pre-built Oracle Database and Global Data Services * The `dbImage` used during provisioning the Persistent Volume with Database Gold Image and the `dbImage` used for deploying the Shard or Catalog Database by cloning should be same. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) +**NOTE:** Provisioning the Sharded Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. + Use the file: [udsharding_shard_prov_clone_across_ads.yaml](./udsharding_shard_prov_clone_across_ads.yaml) for this use case as below: 1. Deploy the `udsharding_shard_prov_clone_across_ads.yaml` file: diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_gold_image_in_same_ad.md b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_gold_image_in_same_ad.md index 445d0105..a4669667 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_gold_image_in_same_ad.md +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_gold_image_in_same_ad.md @@ -33,6 +33,8 @@ In this example, we are using pre-built Oracle Database and Global Data Services * The `dbImage` used during provisioning the Persistent Volume with Database Gold Image and the `dbImage` used for deploying the Shard or Catalog Database by cloning should be same. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) +**NOTE:** Provisioning the Sharded Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. + Use the file: [udsharding_shard_prov_clone.yaml](./udsharding_shard_prov_clone.yaml) for this use case as below: 1. Deploy the `udsharding_shard_prov_clone.yaml` file: diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_with_control_on_resources.md b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_with_control_on_resources.md index d37368f8..b52b8745 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_with_control_on_resources.md +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_with_control_on_resources.md @@ -18,6 +18,9 @@ In this example, we are using pre-built Oracle Database and Global Data Services * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `udsharding_shard_prov_memory_cpu.yaml`. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) + * In case you want to use the [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then you will need to add the additional parameter `dbEdition: "free"` to the below .yaml file. + +**NOTE:** For Oracle Database 23ai Free, you can control the `CPU` and `Memory` allocation of the PODs using tags `cpu` and `memory` respectively but tags `INIT_SGA_SIZE` and `INIT_PGA_SIZE` to control the SGA and PGA allocation at the database level are `not` supported. Use the YAML file [udsharding_shard_prov_memory_cpu.yaml](./udsharding_shard_prov_memory_cpu.yaml). diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_with_notification_using_oci_notification.md b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_with_notification_using_oci_notification.md index c7da6aa5..640301a2 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_with_notification_using_oci_notification.md +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_with_notification_using_oci_notification.md @@ -66,6 +66,9 @@ In this example, we are using pre-built Oracle Database and Global Data Services * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `udsharding_shard_prov_send_notification.yaml`. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) + * In case you want to use the [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then you will need to add the additional parameter `dbEdition: "free"` to the below .yaml file. + +**NOTE:** Provisioning the Sharded Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. Use the file: [udsharding_shard_prov_send_notification.yaml](./udsharding_shard_prov_send_notification.yaml) for this use case as below: diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_without_db_gold_image.md b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_without_db_gold_image.md index a275155f..2be5ac9f 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_without_db_gold_image.md +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_without_db_gold_image.md @@ -19,7 +19,7 @@ In this example, we are using pre-built Oracle Database and Global Data Services * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `udsharding_shard_prov.yaml`. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) - + * In case you want to use the [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then you will need to add the additional parameter `dbEdition: "free"` to the below .yaml file. Use the file: [udsharding_shard_prov.yaml](./udsharding_shard_prov.yaml) for this use case as below: diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_scale_in_delete_an_existing_shard.md b/docs/sharding/provisioning/user-defined-sharding/udsharding_scale_in_delete_an_existing_shard.md index 946a0ab9..2c4cbfc2 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_scale_in_delete_an_existing_shard.md +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_scale_in_delete_an_existing_shard.md @@ -4,8 +4,6 @@ This use case demonstrates how to delete an existing Shard from an existing Oracle Database sharding topology with User Defined Sharding provisioned using Oracle Database Sharding controller. -**NOTE** The deletion of a shard is done after verifying the Chunks have been moved out of that shard. - In this use case, the existing database Sharding is having: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` @@ -18,30 +16,50 @@ In this example, we are using pre-built Oracle Database and Global Data Services * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `udsharding_shard_prov_delshard.yaml`. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) + * In case you want to use the [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then you will need to add the additional parameter `dbEdition: "free"` to the below .yaml file. -NOTE: Use tag `isDelete: true` to delete the shard you want. +**NOTE:** Use tag `isDelete: enable` to delete the shard you want. This use case deletes the shard `shard4` from the above Sharding Topology. Use the file: [udsharding_shard_prov_delshard.yaml](./udsharding_shard_prov_delshard.yaml) for this use case as below: -1. Deploy the `udsharding_shard_prov_delshard.yaml` file: +1. Move out the chunks from the shard to be deleted to another shard. For example, in the current case, before deleting the `shard4`, if you want to move the chunks from `shard4` to `shard2`, then you can run the below `kubectl` command where `/u01/app/oracle/product/23ai/gsmhome_1` is the GSM HOME: + ```sh + kubectl exec -it pod/gsm1-0 -n shns -- /u01/app/oracle/product/23ai/gsmhome_1/bin/gdsctl "move chunk -chunk all -source shard4_shard4pdb -target shard4_shard4pdb" + ``` +2. Confirm the shard to be deleted (`shard4` in this case) is not having any chunk using below command: + ```sh + kubectl exec -it pod/gsm1-0 -n shns -- /u01/app/oracle/product/23ai/gsmhome_1/bin/gdsctl "config chunks" + ``` + If there is no chunk present in the shard to be deleted, you can move to the next step. + +3. Apply the `udsharding_shard_prov_delshard.yaml` file: ```sh kubectl apply -f udsharding_shard_prov_delshard.yaml ``` -2. Check the status of the deployment: +4. Check the status of the deployment: ```sh # Check the status of the Kubernetes Pods: kubectl get all -n shns + ``` -**NOTE:** After you apply `udsharding_shard_prov_delshard.yaml`, the change may not be visible immediately. When the shard is removed, first the chunks will be moved out of that shard that is going to be deleted. +**NOTE:** +- After you apply `udsharding_shard_prov_delshard.yaml`, the change may not be visible immediately and it may take some time for the delete operation to complete. +- If the shard, that you are trying to delete, is still having chunks, then the you will see message like below in the logs of the Oracle Database Operator Pod. + ```sh + INFO controllers.database.ShardingDatabase manual intervention required + ``` + In this case, you will need to first move out the chunks from the shard to be deleted using Step 2 above and then apply the file in Step 3 to delete that shard. -To monitor the chunk movement, use the following command: +To check the status, use the following command: + ```sh + # Switch to the primary GSM Container: + kubectl exec -i -t gsm1-0 -n shns /bin/bash -```sh -# Switch to the primary GSM Container: -kubectl exec -i -t gsm1-0 -n shns /bin/bash + # Check the status shards: + gdsctl config shard -# Check the status of the chunks and repeat to observe the chunk movement: -gdsctl config chunks -``` + # Check the status of the chunks: + gdsctl config chunks + ``` diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_scale_out_add_shards.md b/docs/sharding/provisioning/user-defined-sharding/udsharding_scale_out_add_shards.md index e4200d72..20f50b29 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_scale_out_add_shards.md +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_scale_out_add_shards.md @@ -16,6 +16,7 @@ In this example, we are using pre-built Oracle Database and Global Data Services * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `udsharding_shard_prov_extshard.yaml`. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) + * If the existing Sharding Topology was deployed using [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then the additional parameter `dbEdition: "free"` will be needed for the below .yaml file as well. This use case adds two new shards `shard4`,`shard5` to above Sharding Topology. diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov.yaml b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov.yaml index 019fc887..9b565b73 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov.yaml +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov.yaml @@ -56,4 +56,3 @@ spec: - name: oltp_ro_svc role: primary namespace: shns - diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_clone_across_ads.yaml b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_clone_across_ads.yaml index f2d10e23..28f36608 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_clone_across_ads.yaml +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_clone_across_ads.yaml @@ -58,10 +58,18 @@ spec: imagePullPolicy: "Always" storageSizeInGb: 50 region: primary + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-3" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-3" - name: gsm2 imagePullPolicy: "Always" storageSizeInGb: 50 region: standby + nodeSelector: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-3" + pvMatchLabels: + "failure-domain.beta.kubernetes.io/zone": "PHX-AD-3" storageClass: oci dbImage: container-registry.oracle.com/database/enterprise:latest dbImagePullSecret: ocr-reg-cred diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_delshard.yaml b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_delshard.yaml index 51f1c292..2342dc55 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_delshard.yaml +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_delshard.yaml @@ -30,7 +30,7 @@ spec: imagePullPolicy: "Always" shardSpace: sspace4 shardRegion: primary - isDelete: True + isDelete: enable - name: shard5 storageSizeInGb: 50 imagePullPolicy: "Always" @@ -66,4 +66,4 @@ spec: role: primary - name: oltp_ro_svc role: primary - namespace: shns + namespace: shns \ No newline at end of file diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_memory_cpu.yaml b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_memory_cpu.yaml index 15022925..e663aa65 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_memory_cpu.yaml +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_memory_cpu.yaml @@ -87,4 +87,3 @@ spec: - name: oltp_ro_svc role: primary namespace: shns - diff --git a/docs/sidb/README.md b/docs/sidb/README.md index 895c26a9..ff357195 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -25,6 +25,7 @@ Oracle Database Operator for Kubernetes (`OraOperator`) includes the Single Inst * [Delete a Database](#delete-a-database) * [Advanced Database Configurations](#advanced-database-configurations) * [Run Database with Multiple Replicas](#run-database-with-multiple-replicas) + * [Database Pod Resource Management](#database-pod-resource-management) * [Setup Database with LoadBalancer](#setup-database-with-loadbalancer) * [Enabling TCPS Connections](#enabling-tcps-connections) * [Specifying Custom Ports](#specifying-custom-ports) @@ -628,6 +629,9 @@ The following table depicts the fail over matrix for any destructive operation t - If the `ReadWriteOnce` access mode is used, all the replicas will be scheduled on the same node where the persistent volume would be mounted. - If the `ReadWriteMany` access mode is used, all the replicas will be distributed on different nodes. So, it is recommended to have replicas more than or equal to the number of the nodes as the database image is downloaded on all those nodes. This is beneficial in quick cold fail-over scenario (when the active pod dies) as the image would already be available on that node. +#### Database Pod Resource Management +When creating a Single Instance Database you can specify the cpu and memory resources needed by the database pod. These specified resources are passed to the `kube-scheduler` so that the pod gets scheduled on one of the pods that has the required resources available. To use database pod resource management specify values for the `resources` attributes in the [config/samples/sidb/singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml) file, and apply it. + #### Setup Database with LoadBalancer For the Single Instance Database, the default service is the `NodePort` service. You can enable the `LoadBalancer` service by using `kubectl patch` command. diff --git a/main.go b/main.go index acdd2719..4174e97d 100644 --- a/main.go +++ b/main.go @@ -249,6 +249,9 @@ func main() { setupLog.Error(err, "unable to create webhook", "webhook", "DataguardBroker") os.Exit(1) } + if err = (&databasev1alpha1.ShardingDatabase{}).SetupWebhookWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create webhook", "webhook", "ShardingDatabase") + } if err = (&observabilityv1alpha1.DatabaseObserver{}).SetupWebhookWithManager(mgr); err != nil { setupLog.Error(err, "unable to create webhook", "webhook", "DatabaseObserver") os.Exit(1) diff --git a/oracle-database-operator.yaml b/oracle-database-operator.yaml index c2714b6e..504fc7cd 100644 --- a/oracle-database-operator.yaml +++ b/oracle-database-operator.yaml @@ -165,6 +165,8 @@ spec: type: string displayName: type: string + isLongTermBackup: + type: boolean ociConfig: description: "*********************** *\tOCI config ***********************" properties: @@ -173,6 +175,8 @@ spec: secretName: type: string type: object + retentionPeriodInDays: + type: integer target: description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster Important: Run "make" to regenerate code after modifying this file' properties: @@ -1770,6 +1774,9 @@ spec: asClone: description: Indicate if 'AS CLONE' option should be used in the command to plug in a PDB. This property is applicable when the Action property is PLUG but not required. type: boolean + assertivePdbDeletion: + description: turn on the assertive approach to delete pdb resource kubectl delete pdb ..... automatically triggers the pluggable database deletion + type: boolean cdbName: description: Name of the CDB type: string @@ -2077,7 +2084,7 @@ spec: description: PullPolicy describes a policy for if/when to pull a container image type: string isDelete: - type: boolean + type: string label: type: string name: @@ -2140,6 +2147,8 @@ spec: - name type: object type: array + dbEdition: + type: string dbImage: type: string dbImagePullSecret: @@ -2169,6 +2178,8 @@ spec: - name - pwdFileName type: object + fssStorageClass: + type: string gsm: items: description: GsmSpec defines the desired state of GsmSpec @@ -2193,7 +2204,7 @@ spec: description: PullPolicy describes a policy for if/when to pull a container image type: string isDelete: - type: boolean + type: string label: type: string name: @@ -2370,7 +2381,7 @@ spec: isExternalSvc: type: boolean isTdeWallet: - type: boolean + type: string liveinessCheckPeriod: type: integer namespace: @@ -2424,7 +2435,12 @@ spec: description: PullPolicy describes a policy for if/when to pull a container image type: string isDelete: - type: boolean + enum: + - enable + - disable + - failed + - force + type: string label: type: string name: @@ -2507,6 +2523,10 @@ spec: type: string storageClass: type: string + tdeWalletPvc: + type: string + tdeWalletPvcMountLocation: + type: string required: - catalog - dbImage @@ -2770,6 +2790,23 @@ spec: type: integer replicas: type: integer + resources: + properties: + limits: + properties: + cpu: + type: string + memory: + type: string + type: object + requests: + properties: + cpu: + type: string + memory: + type: string + type: object + type: object serviceAccountName: type: string serviceAnnotations: @@ -3082,6 +3119,21 @@ rules: - patch - update - watch +- apiGroups: + - "" + resources: + - deployments + - events + - pods + - services + verbs: + - create + - delete + - get + - list + - patch + - update + - watch - apiGroups: - "" resources: @@ -3112,6 +3164,7 @@ rules: verbs: - get - list + - watch - apiGroups: - '''''' resources: @@ -3128,6 +3181,12 @@ rules: - apps resources: - configmaps + verbs: + - get + - list +- apiGroups: + - apps + resources: - deployments - pods - services @@ -3193,6 +3252,26 @@ rules: - patch - update - watch +- apiGroups: + - "" + resources: + - configmaps + - containers + - events + - namespaces + - pods + - pods/exec + - pods/log + - secrets + - services + verbs: + - create + - delete + - get + - list + - patch + - update + - watch - apiGroups: - "" resources: @@ -3489,7 +3568,6 @@ rules: - apiGroups: - monitoring.coreos.com resources: - - prometheusrules - servicemonitors verbs: - create @@ -3784,6 +3862,26 @@ webhooks: resources: - pdbs sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /mutate-database-oracle-com-v1alpha1-shardingdatabase + failurePolicy: Fail + name: mshardingdatabase.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + resources: + - shardingdatabases + sideEffects: None - admissionReviewVersions: - v1 - v1beta1 @@ -3805,6 +3903,26 @@ webhooks: resources: - singleinstancedatabases sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /mutate-observability-oracle-com-v1alpha1-databaseobserver + failurePolicy: Fail + name: mdatabaseobserver.kb.io + rules: + - apiGroups: + - observability.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + resources: + - databaseobservers + sideEffects: None --- apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingWebhookConfiguration @@ -3977,6 +4095,27 @@ webhooks: resources: - pdbs sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /validate-database-oracle-com-v1alpha1-shardingdatabase + failurePolicy: Fail + name: vshardingdatabase.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + - DELETE + resources: + - shardingdatabases + sideEffects: None - admissionReviewVersions: - v1 - v1beta1 @@ -3999,6 +4138,26 @@ webhooks: resources: - singleinstancedatabases sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /validate-observability-oracle-com-v1alpha1-databaseobserver + failurePolicy: Fail + name: vdatabaseobserver.kb.io + rules: + - apiGroups: + - observability.oracle.com + apiVersions: + - v1alpha1 + operations: + - CREATE + - UPDATE + resources: + - databaseobservers + sideEffects: None --- apiVersion: apps/v1 kind: Deployment diff --git a/ords/Dockerfile b/ords/Dockerfile index d4e16b6c..772a7e6d 100644 --- a/ords/Dockerfile +++ b/ords/Dockerfile @@ -1,34 +1,63 @@ -#LICENSE UPL 1.0 -# -# Copyright (c) 1982-2017 Oracle and/or its affiliates. All rights reserved. -# -# ORACLE DOCKERFILES PROJECT -# -------------------------- -# This is the Dockerfile for Oracle Rest Data Services 22.2 -# +## Copyright (c) 2022 Oracle and/or its affiliates. +## +## The Universal Permissive License (UPL), Version 1.0 +## +## Subject to the condition set forth below, permission is hereby granted to any +## person obtaining a copy of this software, associated documentation and/or data +## (collectively the "Software"), free of charge and under any and all copyright +## rights in the Software, and any and all patent rights owned or freely +## licensable by each licensor hereunder covering either (i) the unmodified +## Software as contributed to or provided by such licensor, or (ii) the Larger +## Works (as defined below), to deal in both +## +## (a) the Software, and +## (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +## one is included with the Software (each a "Larger Work" to which the Software +## is contributed by such licensors), +## +## without restriction, including without limitation the rights to copy, create +## derivative works of, display, perform, and distribute the Software and make, +## use, sell, offer for sale, import, export, have made, and have sold the +## Software and the Larger Work(s), and to sublicense the foregoing rights on +## either these or other terms. +## +## This license is subject to the following condition: +## The above copyright notice and either this complete permission notice or at +## a minimum a reference to the UPL must be included in all copies or +## substantial portions of the Software. +## +## THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +## IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +## FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +## AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +## LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +## OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +## SOFTWARE. + FROM container-registry.oracle.com/java/jdk:latest # Environment variables required for this build (do NOT change) # ------------------------------------------------------------- ENV ORDS_HOME=/opt/oracle/ords/ \ - RUN_FILE="runOrdsSSL.sh" - -#RUN_FILE_NOSSL="runOrdsNOSSL.sh" + RUN_FILE="runOrdsSSL.sh" \ + ORDSVERSION=23.4.0-8 # Copy binaries # ------------- COPY $RUN_FILE $ORDS_HOME -#COPY $RUN_FILE_NOSSL $ORDS_HOME -RUN yum -y install yum-utils bind-utils tree hostname openssl net-tools zip unzip tar wget vim-minimal which sudo expect procps && \ +RUN yum -y install yum-utils bind-utils tree hostname openssl net-tools zip unzip tar wget vim-minimal which sudo expect procps curl lsof && \ yum-config-manager --add-repo=http://yum.oracle.com/repo/OracleLinux/OL8/oracle/software/x86_64 && \ yum -y install java-11-openjdk-devel && \ - yum -y install ords && \ yum -y install iproute && \ yum clean all +RUN curl -o /tmp/ords-$ORDSVERSION.el8.noarch.rpm https://yum.oracle.com/repo/OracleLinux/OL8/oracle/software/x86_64/getPackage/ords-$ORDSVERSION.el8.noarch.rpm + +RUN rpm -ivh /tmp/ords-$ORDSVERSION.el8.noarch.rpm + # Setup filesystem and oracle user -# ------------------------------------------------------------ +# -------------------------------- RUN mkdir -p $ORDS_HOME/doc_root && \ mkdir -p $ORDS_HOME/error && \ mkdir -p $ORDS_HOME/secrets && \ @@ -49,5 +78,3 @@ EXPOSE 8888 # Define default command to start Ords Services CMD $ORDS_HOME/$RUN_FILE -## ONLY FOR DEVELOPMENT STAGE -#CMD ["/usr/sbin/init"] diff --git a/ords/runOrdsSSL.sh b/ords/runOrdsSSL.sh index 23b99f1e..35f1b77b 100644 --- a/ords/runOrdsSSL.sh +++ b/ords/runOrdsSSL.sh @@ -1,16 +1,44 @@ #!/bin/bash -# -# Since: June, 2022 -# Author: matteo.malvezzi@oracle.com -# Description: Setup and runs Oracle Rest Data Services 22.2. -# -# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS HEADER. -# -# Copyright (c) 2014-2017 Oracle and/or its affiliates. All rights reserved. -# -# MODIFIED (DD-Mon-YY) -# mmalvezz 25-Jun-22 - Initial version -# mmalvezz 17-Oct-22 - db.customURL utilization + +cat <$TNSNAME - - function SetParameter() { ##ords config info <--- Use this command to get the list @@ -67,65 +91,16 @@ function SetParameter() { $ORDS --config ${CONFIG} config set misc.pagination.maxRows 1000 $ORDS --config ${CONFIG} config set db.cdb.adminUser "${CDBADMIN_USER:-C##DBAPI_CDB_ADMIN} AS SYSDBA" $ORDS --config ${CONFIG} config secret --password-stdin db.cdb.adminUser.password << EOF -${CDBADMIN_PWD:-WElcome_12##} +${CDBADMIN_PWD:-PROVIDE_A_PASSWORD} EOF -## $ORDS --config ${CONFIG} config set db.username "SYS AS SYSDBA" -## $ORDS --config ${CONFIG} config secret --password-stdin db.password <$PASSFILE -welcome1 -EOF - -## $JAVA_HOME/bin/keytool -genkey -keyalg RSA -alias selfsigned -keystore keystore.jks \ -## -dname "CN=${HN}, OU=Example Department, O=Example Company, L=Birmingham, ST=West Midlands, C=GB" \ -## -storepass welcome1 -validity 3600 -keysize 2048 -keypass welcome1 -## -## -## $JAVA_HOME/bin/keytool -importkeystore -srckeystore keystore.jks -srcalias selfsigned -srcstorepass welcome1 \ -## -destkeystore keystore.p12 -deststoretype PKCS12 -deststorepass welcome1 -destkeypass welcome1 -## -## -## ${OPENSSL} pkcs12 -in ${KEYSTORE}/keystore.p12 -nodes -nocerts -out ${KEYSTORE}/${HN}-key.pem -passin file:${PASSFILE} -## ${OPENSSL} pkcs12 -in ${KEYSTORE}/keystore.p12 -nokeys -out ${KEYSTORE}/${HN}.pem -passin file:${PASSFILE} -## ${OPENSSL} pkcs8 -topk8 -inform PEM -outform DER -in ${HN}-key.pem -out ${HN}-key.der -nocrypt -## ${OPENSSL} x509 -inform PEM -outform DER -in ${HN}.pem -out ${HN}.der - - - - - - - - -rm $PASSFILE -ls -ltr $KEYSTORE - - - -} - - function setupOrds() { echo "====================================================" @@ -163,7 +138,6 @@ export ORDS_LOGS=/tmp ORDS_PASSWORD=`cat $ORDS_HOME/secrets/$ORDS_PWD_KEY` } -setupHTTPS; SetParameter; $ORDS --config ${CONFIG} install \ @@ -173,8 +147,8 @@ $ORDS --config ${CONFIG} install \ --log-folder ${ORDS_LOGS} \ --proxy-user \ --password-stdin <${CKF} 2>&1 +echo "checkfile" >> ${CKF} +NOT_INSTALLED=`cat ${CKF} | grep "INFO: The" |wc -l ` echo NOT_INSTALLED=$NOT_INSTALLED + function StartUp () { $ORDS --config $CONFIG serve --port 8888 --secure } From ec60d939f6cbcc35e015d757237af04056eb9c54 Mon Sep 17 00:00:00 2001 From: Oleksandra Pavlusieva Date: Wed, 4 Sep 2024 12:15:10 +0300 Subject: [PATCH 612/628] Update README.md --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 959045cb..26c4ead8 100644 --- a/README.md +++ b/README.md @@ -126,8 +126,8 @@ Oracle strongly recommends that you ensure your system meets the following [Prer ```sh kubectl apply -f rbac/node-rbac.yaml ``` -# Installation -## Install Oracle DB Operator +## Installation +### Install Oracle DB Operator After you have completed the preceding prerequisite changes, you can install the operator. To install the operator in the cluster quickly, you can apply the modified `oracle-database-operator.yaml` file from the preceding step. From ccc898289059345c37c68a09df11762dba5acda9 Mon Sep 17 00:00:00 2001 From: Saurabh Ahuja Date: Wed, 4 Sep 2024 15:21:13 +0530 Subject: [PATCH 613/628] Update README.md --- README.md | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 26c4ead8..d707bc87 100644 --- a/README.md +++ b/README.md @@ -154,7 +154,7 @@ Oracle strongly recommends that you ensure your system meets the following [Prer You should see that the operator is up and running, along with the shipped controllers. For more details, see [Oracle Database Operator Installation Instructions](./docs/installation/OPERATOR_INSTALLATION_README.md). - +## Documentation ## Getting Started with the Operator (Quickstart) The following quickstarts are designed for specific database configurations: @@ -170,6 +170,7 @@ The following quickstarts are designed for specific database configurations: The following quickstart is designed for non-database configurations: * [Oracle Database Observability](./docs/observability/README.md) +## Examples YAML file templates are available under [`/config/samples`](./config/samples/). You can copy and edit these template files to configure them for your use cases. ## Uninstall the Operator @@ -221,15 +222,15 @@ YAML file templates are available under [`/config/samples`](./config/samples/). ## Contributing -See [Contributing to this Repository](./CONTRIBUTING.md) +This project welcomes contributions from the community. Before submitting a pull request, please [review our contribution guide](./CONTRIBUTING.md) -## Support +## Help You can submit a GitHub issue, oir submit an issue and then file an [Oracle Support service](https://support.oracle.com/portal/) request. To file an issue or a service request, use the following product ID: 14430. ## Security -Secure platforms are an important basis for general system security. Ensure that your deployment is in compliance with common security practices. +Please consult the [security guide](./SECURITY.md) for our responsible security vulnerability disclosure process ### Managing Sensitive Data @@ -244,10 +245,6 @@ The following is an example of a YAML file fragment for specifying Oracle Cloud Examples in this repository where passwords are entered on the command line are for demonstration purposes only. -### Reporting a Security Issue - -See [Reporting security vulnerabilities](./SECURITY.md) - ## License Copyright (c) 2022, 2024 Oracle and/or its affiliates. From 29a38c49fced4fc626e17d3b82f6326180551ee0 Mon Sep 17 00:00:00 2001 From: Saurabh Ahuja Date: Wed, 4 Sep 2024 15:23:02 +0530 Subject: [PATCH 614/628] Update SECURITY.md --- SECURITY.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/SECURITY.md b/SECURITY.md index 30159518..2ca81027 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -1,4 +1,4 @@ -# Oracle's instructions for reporting security vulnerabilities +# Reporting security vulnerabilities Oracle values the independent security research community and believes that responsible disclosure of security vulnerabilities helps us ensure the security @@ -21,7 +21,7 @@ security features are welcome on GitHub Issues. Security updates will be released on a regular cadence. Many of our projects will typically release security fixes in conjunction with the -[Oracle Critical Patch Update][3] program. Additional +Oracle Critical Patch Update program. Additional information, including past advisories, is available on our [security alerts][4] page. @@ -35,4 +35,4 @@ sufficiently hardened for production use. [1]: mailto:secalert_us@oracle.com [2]: https://www.oracle.com/corporate/security-practices/assurance/vulnerability/reporting.html [3]: https://www.oracle.com/security-alerts/encryptionkey.html -[4]: https://www.oracle.com/security-alerts/ \ No newline at end of file +[4]: https://www.oracle.com/security-alerts/ From 2479f3c2eb8092d6cbbeccc9502d884fa9705c56 Mon Sep 17 00:00:00 2001 From: jpverma85 Date: Fri, 6 Sep 2024 15:04:55 -0400 Subject: [PATCH 615/628] gdd-changes (#146) --- PREREQUISITES.md | 6 +- README.md | 10 +- docs/sharding/README.md | 119 +++++++++--------- .../provisioning/database_connection.md | 6 +- docs/sharding/provisioning/debugging.md | 8 +- .../sharding_provisioning_with_db_events.md | 8 +- .../sharding_provisioning_with_free_images.md | 8 +- ...harding_provisioning_with_free_images.yaml | 4 +- ..._persistent_volume_having_db_gold_image.md | 2 +- ...y_cloning_db_from_gold_image_across_ads.md | 14 +-- ...ing_by_cloning_db_gold_image_in_same_ad.md | 14 +-- ...ding_provisioning_with_chunks_specified.md | 10 +- ..._provisioning_with_control_on_resources.md | 6 +- ...ith_notification_using_oci_notification.md | 12 +- ...ding_provisioning_without_db_gold_image.md | 8 +- ...rding_scale_in_delete_an_existing_shard.md | 6 +- .../snr_ssharding_scale_out_add_shards.md | 10 +- ...y_cloning_db_from_gold_image_across_ads.md | 10 +- ...ing_by_cloning_db_gold_image_in_same_ad.md | 8 +- ...ding_provisioning_with_chunks_specified.md | 6 +- ..._provisioning_with_control_on_resources.md | 4 +- ...ith_notification_using_oci_notification.md | 8 +- ...ding_provisioning_without_db_gold_image.md | 4 +- ...rding_scale_in_delete_an_existing_shard.md | 8 +- .../ssharding_scale_out_add_shards.md | 10 +- ...y_cloning_db_from_gold_image_across_ads.md | 14 +-- ...ing_by_cloning_db_gold_image_in_same_ad.md | 12 +- ..._provisioning_with_control_on_resources.md | 8 +- ...ith_notification_using_oci_notification.md | 12 +- ...ding_provisioning_without_db_gold_image.md | 10 +- ...rding_scale_in_delete_an_existing_shard.md | 10 +- .../udsharding_scale_out_add_shards.md | 14 +-- 32 files changed, 195 insertions(+), 194 deletions(-) diff --git a/PREREQUISITES.md b/PREREQUISITES.md index 01bb94b4..bc333357 100644 --- a/PREREQUISITES.md +++ b/PREREQUISITES.md @@ -6,7 +6,7 @@ Oracle Database Operator for Kubernetes (OraOperator) manages all Cloud deployme * Oracle Autonomous Database (ADB) * Containerized Oracle Database Single Instance (SIDB) -* Containerized Sharded Oracle Database (SHARDING) +* Containerized Oracle Globally Distributed Database (GDD) ### Setting Up a Kubernetes Cluster and Volumes Review and complete each step as needed. @@ -29,6 +29,6 @@ If you intent to use `OraOperator` to handle Oracle Autonomous Database lifecycl If you intent to use `OraOperator` to handle Oracle Database Single Instance lifecycles, then read [Single Instance Database Prerequisites](./docs/sidb/PREREQUISITES.md) -### Prerequites for Sharded Databases (SHARDING) +### Prerequites for Oracle Globally Distributed Databases(GDD) - If you intent to use OraOperator to handle the lifecycle of Oracle Database deployed with Oracle Sharding, then read [Sharded Database Prerequisites](./docs/sharding/README.md#prerequsites-for-running-oracle-sharding-database-controller) + If you intent to use OraOperator to handle the lifecycle of Oracle Globally Distributed Database(GDD), then read [Oracle Globally Distributed Database Prerequisites](./docs/sharding/README.md#prerequsites-for-running-oracle-sharding-database-controller) diff --git a/README.md b/README.md index d707bc87..3409463b 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,7 @@ In this v1.1.0 production release, `OraOperator` supports the following database * Oracle Autonomous Database on dedicated Cloud infrastructure (ADB-D) * Oracle Autonomous Container Database (ACD) (infrastructure) is the infrastructure for provisioning Autonomous Databases. * Containerized Single Instance databases (SIDB) deployed in the Oracle Kubernetes Engine (OKE) and any k8s where OraOperator is deployed -* Containerized Sharded databases (SHARDED) deployed in OKE and any k8s where OraOperator is deployed +* Containerized Oracle Globally Distributed Databases(GDD) deployed in OKE and any k8s where OraOperator is deployed * Oracle Multitenant Databases (CDB/PDBs) * Oracle Base Database Cloud Service (BDBCS) * Oracle Data Guard (Preview status) @@ -23,7 +23,7 @@ Oracle will continue to extend `OraOperator` to support additional Oracle Databa * Namespace scope deployment option * Enhanced security with namespace scope deployment option * Support for Oracle Database 23ai Free (with SIDB) -* Automatic Storage Expansion for SIDB and Sharded DB +* Automatic Storage Expansion for SIDB and Oracle Globally Distributed Database * User-Defined Sharding * TCPS support customer provided certs * Execute custom scripts during DB setup/startup @@ -41,7 +41,7 @@ This release of Oracle Database Operator for Kubernetes (the operator) supports * ADB-S/ADB-D: Provision, bind, start, stop, terminate (soft/hard), scale (up/down), long-term backup, manual restore * ACD: provision, bind, restart, terminate (soft/hard) * SIDB: Provision, clone, patch (in-place/out-of-place), update database initialization parameters, update database configuration (Flashback, archiving), Oracle Enterprise Manager (EM) Express (a basic observability console), Oracle REST Data Service (ORDS) to support REST based SQL, PDB management, SQL Developer Web, and Application Express (Apex) -* SHARDED: Provision/deploy sharded databases and the shard topology, Add a new shard, Delete an existing shard +* GDD: Provision/deploy Oracle Globally Distributed Databases and the GDD topology, Add a new shard, Delete an existing shard * Oracle Multitenant Database: Bind to a CDB, Create a  PDB, Plug a  PDB, Unplug a PDB, Delete a PDB, Clone a PDB, Open/Close a PDB * Oracle Base Database Cloud Service (BDBCS): provision, bind, scale shape Up/Down, Scale Storage Up, Terminate and Update License * Oracle Data Guard: Provision a Standby for the SIDB resource, Create a Data Guard Configuration, Perform a Switchover, Patch Primary and Standby databases in Data Guard Configuration @@ -162,7 +162,7 @@ The following quickstarts are designed for specific database configurations: * [Oracle Autonomous Database](./docs/adb/README.md) * [Oracle Autonomous Container Database](./docs/adb/ACD.md) * [Containerized Oracle Single Instance Database and Data Guard](./docs/sidb/README.md) -* [Containerized Oracle Sharded Database](./docs/sharding/README.md) +* [Containerized Oracle Globally Distributed Database](./docs/sharding/README.md) * [Oracle Multitenant Database](./docs/multitenant/README.md) * [Oracle Base Database Cloud Service (BDBCS)](./docs/dbcs/README.md) @@ -217,7 +217,7 @@ YAML file templates are available under [`/config/samples`](./config/samples/). * [Oracle Autonomous Database](https://docs.oracle.com/en-us/iaas/Content/Database/Concepts/adboverview.htm) * [Components of Dedicated Autonomous Database](https://docs.oracle.com/en-us/iaas/autonomous-database/doc/components.html) * [Oracle Database Single Instance](https://docs.oracle.com/en/database/oracle/oracle-database/) -* [Oracle Database Sharding](https://docs.oracle.com/en/database/oracle/oracle-database/21/shard/index.html) +* [Oracle Globally Distributed Database](https://docs.oracle.com/en/database/oracle/oracle-database/21/shard/index.html) * [Oracle Database Cloud Service](https://docs.oracle.com/en/database/database-cloud-services.html) ## Contributing diff --git a/docs/sharding/README.md b/docs/sharding/README.md index 0c817467..a5c7c470 100644 --- a/docs/sharding/README.md +++ b/docs/sharding/README.md @@ -1,36 +1,36 @@ -# Using Oracle Sharding with Oracle Database Operator for Kubernetes +# Using Oracle Globally Distributed Database with Oracle Database Operator for Kubernetes -Oracle Sharding distributes segments of a data set across many databases (shards) on different computers, either on-premises or in cloud. Sharding enables globally distributed, linearly scalable, multimodel databases. It requires no specialized hardware or software. Oracle Sharding does all this while rendering the strong consistency, full power of SQL, support for structured and unstructured data, and the Oracle Database ecosystem. It meets data sovereignty requirements, and supports applications that require low latency and high availability. +Oracle Globally Distributed Database distributes segments of a data set across many databases (shards) on different computers, either on-premises or in cloud. This feature enables globally distributed, linearly scalable, multimodel databases. It requires no specialized hardware or software. Oracle Globally Distributed Database does all this while rendering the strong consistency, full power of SQL, support for structured and unstructured data, and the Oracle Database ecosystem. It meets data sovereignty requirements, and supports applications that require low latency and high availability. -All of the shards together make up a single logical database, which is referred to as a sharded database (SDB). +All of the shards together make up a single logical database, which is referred to as a Oracle Globally Distributed Database (GDD). -Kubernetes provides infrastructure building blocks, such as compute, storage, and networks. Kubernetes makes the infrastructure available as code. It enables rapid provisioning of multi-node topolgies. Additionally, Kubernetes also provides statefulsets, which are the workload API objects that are used to manage stateful applications. This provides us lifecycle management elasticity for databases as a stateful application for various database topologies, such as sharded databases, Oracle Real Application Clusters (Oracle RAC), single instance Oracle Database, and other Oracle features and configurations. +Kubernetes provides infrastructure building blocks, such as compute, storage, and networks. Kubernetes makes the infrastructure available as code. It enables rapid provisioning of multi-node topolgies. Additionally, Kubernetes also provides statefulsets, which are the workload API objects that are used to manage stateful applications. This provides us lifecycle management elasticity for databases as a stateful application for various database topologies, such as Oracle Globally Distributed Database, Oracle Real Application Clusters (Oracle RAC), single instance Oracle Database, and other Oracle features and configurations. -The Sharding Database controller in Oracle Database Operator deploys Oracle Sharding topology as a statefulset in the Kubernetes clusters, using Oracle Database and Global Data Services Docker images. The Oracle Sharding database controller manages the typical lifecycle of Oracle Sharding topology in the Kubernetes cluster, as shown below: +The Sharding Database controller in Oracle Database Operator deploys Oracle Globally Distributed Database Topology as a statefulset in the Kubernetes clusters, using Oracle Database and Global Data Services Docker images. The Oracle Sharding database controller manages the typical lifecycle of Oracle Globally Distributed Database topology in the Kubernetes cluster, as shown below: * Create primary statefulsets shards * Create master and standby Global Data Services statefulsets * Create persistent storage, along with statefulset * Create services * Create load balancer service -* Provision sharding topology by creating and configuring the following: +* Provision Oracle Globally Distributed Database topology by creating and configuring the following: * Catalog database * Shard Databases * GSMs * Shard scale up and scale down * Shard topology cleanup -The Oracle Sharding database controller provides end-to-end automation of Oracle Database sharding topology deployment in Kubernetes clusters. +The Oracle Sharding database controller provides end-to-end automation of Oracle Globally Distributed Database topology deployment in Kubernetes clusters. ## Using Oracle Database Operator Sharding Controller -Following sections provide the details for deploying Oracle Globally Distributed Database (Oracle Sharded Database) using Oracle Database Operator Sharding Controller with different use cases: +Following sections provide the details for deploying Oracle Globally Distributed Database using Oracle Database Operator Sharding Controller with different use cases: * [Prerequisites for running Oracle Sharding Database Controller](#prerequisites-for-running-oracle-sharding-database-controller) * [Oracle Database 23ai Free](#oracle-database-23ai-free) -* [Provisioning Sharding Topology with System-Managed Sharding in a Cloud-Based Kubernetes Cluster](#provisioning-sharding-topology-with-system-managed-sharding-in-a-cloud-based-kubernetes-cluster) -* [Provisioning Sharding Topology with User Defined Sharding in a Cloud-Based Kubernetes Cluster](#provisioning-sharding-topology-with-user-defined-sharding-in-a-cloud-based-kubernetes-cluster) -* [Provisioning System-Managed Sharding Topology with Raft replication enabled in a Cloud-Based Kubernetes Cluster](#provisioning-system-managed-sharding-topology-with-raft-replication-enabled-in-a-cloud-based-kubernetes-cluster) +* [Provisioning Oracle Globally Distributed Database Topology System-Managed Sharding in a Cloud-Based Kubernetes Cluster](#provisioning-oracle-globally-distributed-database-topology-with-system-managed-sharding-in-a-cloud-based-kubernetes-cluster) +* [Provisioning Oracle Globally Distributed Database Topology User Defined Sharding in a Cloud-Based Kubernetes Cluster](#provisioning-oracle-globally-distributed-database-topology-with-user-defined-sharding-in-a-cloud-based-kubernetes-cluster) +* [Provisioning Oracle Globally Distributed Database System-Managed Sharding with Raft replication enabled in a Cloud-Based Kubernetes Cluster](#provisioning-oracle-globally-distributed-database-topology-with-system-managed-sharding-and-raft-replication-enabled-in-a-cloud-based-kubernetes-cluster) * [Connecting to Shard Databases](#connecting-to-shard-databases) * [Debugging and Troubleshooting](#debugging-and-troubleshooting) @@ -72,8 +72,9 @@ Choose one of the following deployment options: **Use Oracle-Supplied Docker Images:** The Oracle Sharding Database controller uses Oracle Global Data Services and Oracle Database images to provision the sharding topology. - You can also download the pre-built Oracle Global Data Services `container-registry.oracle.com/database/gsm:latest` and Oracle Database images `container-registry.oracle.com/database/enterprise:latest` from [Oracle Container Registry](https://container-registry.oracle.com/ords/f?p=113:10::::::). These images are functionally tested and evaluated with various use cases of sharding topology by deploying on OKE and OLCNE. - + You can also download the pre-built Oracle Global Data Services `container-registry.oracle.com/database/gsm:latest` and Oracle Database images `container-registry.oracle.com/database/enterprise:latest` from [Oracle Container Registry](https://container-registry.oracle.com/ords/f?p=113:10::::::). These images are functionally tested and evaluated with various use cases of Oracle Globally Distributed Database topology by deploying on OKE and OLCNE. + + **Note:** You will need to accept Agreement from container-registry.orcale.com to be able to pull the pre-built container images. **OR** @@ -82,17 +83,17 @@ Choose one of the following deployment options: * [Oracle Global Data Services Image](https://github.com/oracle/db-sharding/tree/master/docker-based-sharding-deployment/dockerfiles) * [Oracle Database Image](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance) -After the images are ready, push them to your Docker Images Repository, so that you can pull them during Oracle Database Sharding topology provisioning. +After the images are ready, push them to your Docker Images Repository, so that you can pull them during Oracle Globally Distributed Database topology provisioning. You can either download the images and push them to your Docker Images Repository, or, if your Kubernetes cluster can reach OCR, you can download these images directly from OCR. -**Note**: In the sharding example yaml files, we are using GDS and database images available on [Oracle Container Registry](https://container-registry.oracle.com/ords/f?p=113:10::::::). +**Note**: In the Oracle Globally Distributed Database Topology example yaml files, we are using GDS and database images available on [Oracle Container Registry](https://container-registry.oracle.com/ords/f?p=113:10::::::). **Note:** In case you want to use the `Oracle Database 23ai Free` Image for Database and GSM, refer to section [Oracle Database 23ai Free](#oracle-database-23ai-free) for more details. -### 4. Create a namespace for the Oracle DB Sharding Setup +### 4. Create a namespace for the Oracle Globally Distributed Database Setup - Create a Kubernetes namespace named `shns`. All the resources belonging to the Oracle Database Sharding Setup will be provisioned in this namespace named `shns`. For example: + Create a Kubernetes namespace named `shns`. All the resources belonging to the Oracle Globally Distributed Database Topology Setup will be provisioned in this namespace named `shns`. For example: ```sh #### Create the namespace @@ -102,11 +103,11 @@ You can either download the images and push them to your Docker Images Repositor kubectl get ns ``` -### 5. Create a Kubernetes secret for the database installation owner for the database Sharding Deployment +### 5. Create a Kubernetes secret for the database installation owner for the Oracle Globally Distributed Database Topology Deployment Create a Kubernetes secret named `db-user-pass-rsa` using these steps: [Create Kubernetes Secret](./provisioning/create_kubernetes_secret_for_db_user.md) -After you have the above prerequisites completed, you can proceed to the next section for your environment to provision the Oracle Database Sharding Topology. +After you have the above prerequisites completed, you can proceed to the next section for your environment to provision the Oracle Globally Distributed Database Topology. ### 6. Provisioning a Persistent Volume having an Oracle Database Gold Image @@ -116,73 +117,73 @@ In case of an `OCI OKE` cluster, you can use this Persistent Volume during provi You can refer [here](./provisioning/provisioning_persistent_volume_having_db_gold_image.md) for the steps involved. -**NOTE:** Provisioning the Sharded Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. So, this step will not be needed if you are deploying Oracle Sharded Database using Oracle 23ai Free Database and GSM Images. +**NOTE:** Provisioning the Oracle Globally Distributed Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. So, this step will not be needed if you are deploying Oracle Globally Distributed Database using Oracle 23ai Free Database and GSM Images. ## Oracle Database 23ai Free Please refer to [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) documentation for more details. -If you want to use Oracle Database 23ai Free Image for Database and GSM for deployment of the Sharded Database using Sharding Controller in Oracle Database Kubernetes Operator, you need to consider the below points: +If you want to use Oracle Database 23ai Free Image for Database and GSM for deployment of the Oracle Globally Distributed Database using Sharding Controller in Oracle Database Kubernetes Operator, you need to consider the below points: * To deploy using the FREE Database and GSM Image, you will need to add the additional parameter `dbEdition: "free"` to the .yaml file. -* Refer to [Sample Sharded Database Deployment using Oracle 23ai FREE Database and GSM Images](./provisioning/free/sharding_provisioning_with_free_images.md) for an example. +* Refer to [Sample Oracle Globally Distributed Database Deployment using Oracle 23ai FREE Database and GSM Images](./provisioning/free/sharding_provisioning_with_free_images.md) for an example. * For Oracle Database 23ai Free, you can control the `CPU` and `Memory` allocation of the PODs using tags `cpu` and `memory` respectively but tags `INIT_SGA_SIZE` and `INIT_PGA_SIZE` to control the SGA and PGA allocation at the database level are `not` supported. -* Provisioning the Sharded Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. +* Provisioning the Oracle Globally Distributed Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. * Total number of chunks for FREE Database defaults to `12` if `CATALOG_CHUNKS` parameter is not specified. This default value is determined considering limitation of 12 GB of user data on disk for oracle free database. -## Provisioning Sharding Topology with System-Managed Sharding in a Cloud-Based Kubernetes Cluster +## Provisioning Oracle Globally Distributed Database Topology with System-Managed Sharding in a Cloud-Based Kubernetes Cluster -Deploy Oracle Database Sharding Topology with `System-Managed Sharding` on your Cloud based Kubernetes cluster. +Deploy Oracle Globally Distributed Database Topology with `System-Managed Sharding` on your Cloud based Kubernetes cluster. -In this example, the deployment uses the YAML file based on `OCI OKE` cluster. There are multiple use case possible for deploying the Oracle Database Sharding Topology covered by below examples: +In this example, the deployment uses the YAML file based on `OCI OKE` cluster. There are multiple use case possible for deploying the Oracle Globally Distributed Database Topology covered by below examples: -[1. Provisioning Oracle Sharded Database with System-Managed Sharding without Database Gold Image](./provisioning/system_sharding/ssharding_provisioning_without_db_gold_image.md) -[2. Provisioning Oracle Sharded Database with System-Managed Sharding with number of chunks specified](./provisioning/system_sharding/ssharding_provisioning_with_chunks_specified.md) -[3. Provisioning Oracle Sharded Database with System-Managed Sharding with additional control on resources like Memory and CPU allocated to Pods](./provisioning/system_sharding/ssharding_provisioning_with_control_on_resources.md) -[4. Provisioning Oracle Sharded Database with System-Managed Sharding by cloning database from your own Database Gold Image in the same Availability Domain(AD)](./provisioning/system_sharding/ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md) -[5. Provisioning Oracle Sharded Database with System-Managed Sharding by cloning database from your own Database Gold Image across Availability Domains(ADs)](./provisioning/system_sharding/ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md) -[6. Provisioning Oracle Sharded Database with System-Managed Sharding and send Notification using OCI Notification Service](./provisioning/system_sharding/ssharding_provisioning_with_notification_using_oci_notification.md) -[7. Scale Out - Add Shards to an existing Oracle Sharded Database provisioned earlier with System-Managed Sharding](./provisioning/system_sharding/ssharding_scale_out_add_shards.md) -[8. Scale In - Delete an existing Shard from a working Oracle Sharded Database provisioned earlier with System-Managed Sharding](./provisioning/system_sharding/ssharding_scale_in_delete_an_existing_shard.md) +[1. Provisioning Oracle Globally Distributed Database with System-Managed Sharding without Database Gold Image](./provisioning/system_sharding/ssharding_provisioning_without_db_gold_image.md) +[2. Provisioning Oracle Globally Distributed Database with System-Managed Sharding with number of chunks specified](./provisioning/system_sharding/ssharding_provisioning_with_chunks_specified.md) +[3. Provisioning Oracle Globally Distributed Database with System-Managed Sharding with additional control on resources like Memory and CPU allocated to Pods](./provisioning/system_sharding/ssharding_provisioning_with_control_on_resources.md) +[4. Provisioning Oracle Globally Distributed Database with System-Managed Sharding by cloning database from your own Database Gold Image in the same Availability Domain(AD)](./provisioning/system_sharding/ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md) +[5. Provisioning Oracle Globally Distributed Database with System-Managed Sharding by cloning database from your own Database Gold Image across Availability Domains(ADs)](./provisioning/system_sharding/ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md) +[6. Provisioning Oracle Globally Distributed Database with System-Managed Sharding and send Notification using OCI Notification Service](./provisioning/system_sharding/ssharding_provisioning_with_notification_using_oci_notification.md) +[7. Scale Out - Add Shards to an existing Oracle Globally Distributed Database provisioned earlier with System-Managed Sharding](./provisioning/system_sharding/ssharding_scale_out_add_shards.md) +[8. Scale In - Delete an existing Shard from a working Oracle Globally Distributed Database provisioned earlier with System-Managed Sharding](./provisioning/system_sharding/ssharding_scale_in_delete_an_existing_shard.md) -## Provisioning Sharding Topology with User Defined Sharding in a Cloud-Based Kubernetes Cluster +## Provisioning Oracle Globally Distributed Database Topology with User-Defined Sharding in a Cloud-Based Kubernetes Cluster -Deploy Oracle Database Sharding Topology with `User Defined Sharding` on your Cloud based Kubernetes cluster. +Deploy Oracle Globally Distributed Database Topology with `User-Defined Sharding` on your Cloud based Kubernetes cluster. -In this example, the deployment uses the YAML file based on `OCI OKE` cluster. There are multiple use case possible for deploying the Oracle Database Sharding Topology covered by below examples: +In this example, the deployment uses the YAML file based on `OCI OKE` cluster. There are multiple use case possible for deploying the Oracle Globally Distributed Database Topology covered by below examples: -[1. Provisioning Oracle Sharded Database with User Defined Sharding without Database Gold Image](./provisioning/user-defined-sharding/udsharding_provisioning_without_db_gold_image.md) -[2. Provisioning Oracle Sharded Database with User Defined Sharding with additional control on resources like Memory and CPU allocated to Pods](./provisioning/user-defined-sharding/udsharding_provisioning_with_control_on_resources.md) -[3. Provisioning Oracle Sharded Database with User Defined Sharding by cloning database from your own Database Gold Image in the same Availability Domain(AD)](./provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_gold_image_in_same_ad.md) -[4. Provisioning Oracle Sharded Database with User Defined Sharding by cloning database from your own Database Gold Image across Availability Domains(ADs)](./provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_from_gold_image_across_ads.md) -[5. Provisioning Oracle Sharded Database with User Defined Sharding and send Notification using OCI Notification Service](./provisioning/user-defined-sharding/udsharding_provisioning_with_notification_using_oci_notification.md) -[6. Scale Out - Add Shards to an existing Oracle Sharded Database provisioned earlier with User Defined Sharding](./provisioning/user-defined-sharding/udsharding_scale_out_add_shards.md) -[7. Scale In - Delete an existing Shard from a working Oracle Sharded Database provisioned earlier with User Defined Sharding](./provisioning/user-defined-sharding/udsharding_scale_in_delete_an_existing_shard.md) +[1. Provisioning Oracle Globally Distributed Database with User-Defined Sharding without Database Gold Image](./provisioning/user-defined-sharding/udsharding_provisioning_without_db_gold_image.md) +[2. Provisioning Oracle Globally Distributed Database with User-Defined Sharding with additional control on resources like Memory and CPU allocated to Pods](./provisioning/user-defined-sharding/udsharding_provisioning_with_control_on_resources.md) +[3. Provisioning Oracle Globally Distributed Database with User-Defined Sharding by cloning database from your own Database Gold Image in the same Availability Domain(AD)](./provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_gold_image_in_same_ad.md) +[4. Provisioning Oracle Globally Distributed Database with User-Defined Sharding by cloning database from your own Database Gold Image across Availability Domains(ADs)](./provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_from_gold_image_across_ads.md) +[5. Provisioning Oracle Globally Distributed Database with User-Defined Sharding and send Notification using OCI Notification Service](./provisioning/user-defined-sharding/udsharding_provisioning_with_notification_using_oci_notification.md) +[6. Scale Out - Add Shards to an existing Oracle Globally Distributed Database provisioned earlier with User-Defined Sharding](./provisioning/user-defined-sharding/udsharding_scale_out_add_shards.md) +[7. Scale In - Delete an existing Shard from a working Oracle Globally Distributed Database provisioned earlier with User-Defined Sharding](./provisioning/user-defined-sharding/udsharding_scale_in_delete_an_existing_shard.md) -## Provisioning System-Managed Sharding Topology with Raft replication enabled in a Cloud-Based Kubernetes Cluster +## Provisioning Oracle Globally Distributed Database Topology with System-Managed Sharding and Raft replication enabled in a Cloud-Based Kubernetes Cluster -Deploy Oracle Database Sharding Topology with `System-Managed Sharding with SNR RAFT enabled` on your Cloud based Kubernetes cluster. +Deploy Oracle Globally Distributed Database Topology with `System-Managed Sharding` and with `RAFT Replication` enabled on your Cloud based Kubernetes cluster. -**NOTE: SNR RAFT Feature is available only for Oracle 23ai RDBMS and Oracle 23ai GSM version.** +**NOTE: RAFT Replication Feature is available only for Oracle 23ai RDBMS and Oracle 23ai GSM version.** -In this example, the deployment uses the YAML file based on `OCI OKE` cluster. There are multiple use case possible for deploying the Oracle Database Sharding Topology covered by below examples: +In this example, the deployment uses the YAML file based on `OCI OKE` cluster. There are multiple use case possible for deploying the Oracle Globally Distributed Database Topology covered by below examples: -[1. Provisioning System-Managed Sharding Topology with Raft replication enabled without Database Gold Image](./provisioning/snr_system_sharding/snr_ssharding_provisioning_without_db_gold_image.md) -[2. Provisioning System-Managed Sharding Topology with Raft replication enabled with number of chunks specified](./provisioning/snr_system_sharding/snr_ssharding_provisioning_with_chunks_specified.md) -[3. Provisioning System-Managed Sharding Topology with Raft replication enabled with additional control on resources like Memory and CPU allocated to Pods](./provisioning/snr_system_sharding/snr_ssharding_provisioning_with_control_on_resources.md) -[4. Provisioning System-Managed Sharding Topology with Raft replication enabled by cloning database from your own Database Gold Image in the same Availability Domain(AD)](./provisioning/snr_system_sharding/snr_ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md) -[5. Provisioning System-Managed Sharding Topology with Raft replication enabled by cloning database from your own Database Gold Image across Availability Domains(ADs)](./provisioning/snr_system_sharding/snr_ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md) -[6. Provisioning System-Managed Sharding Topology with Raft replication enabled and send Notification using OCI Notification Service](./provisioning/snr_system_sharding/snr_ssharding_provisioning_with_notification_using_oci_notification.md) -[7. Scale Out - Add Shards to an existing Oracle Sharded Database provisioned earlier with System-Managed Sharding with RAFT replication enabled](./provisioning/snr_system_sharding/snr_ssharding_scale_out_add_shards.md) -[8. Scale In - Delete an existing Shard from a working Oracle Sharded Database provisioned earlier with System-Managed Sharding with RAFT reolication enabled](./provisioning/snr_system_sharding/snr_ssharding_scale_in_delete_an_existing_shard.md) +[1. Provisioning Oracle Globally Distributed Database Topology with System-Managed Sharding and Raft replication enabled without Database Gold Image](./provisioning/snr_system_sharding/snr_ssharding_provisioning_without_db_gold_image.md) +[2. Provisioning Oracle Globally Distributed Database Topology with System-Managed Sharding and Raft replication enabled with number of chunks specified](./provisioning/snr_system_sharding/snr_ssharding_provisioning_with_chunks_specified.md) +[3. Provisioning Oracle Globally Distributed Database Topology with System-Managed Sharding and Raft replication enabled with additional control on resources like Memory and CPU allocated to Pods](./provisioning/snr_system_sharding/snr_ssharding_provisioning_with_control_on_resources.md) +[4. Provisioning Oracle Globally Distributed Database Topology with System-Managed Sharding and Raft replication enabled by cloning database from your own Database Gold Image in the same Availability Domain(AD)](./provisioning/snr_system_sharding/snr_ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md) +[5. Provisioning Oracle Globally Distributed Database Topology with System-Managed Sharding and Raft replication enabled by cloning database from your own Database Gold Image across Availability Domains(ADs)](./provisioning/snr_system_sharding/snr_ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md) +[6. Provisioning Oracle Globally Distributed Database Topology with System-Managed Sharding and Raft replication enabled send Notification using OCI Notification Service](./provisioning/snr_system_sharding/snr_ssharding_provisioning_with_notification_using_oci_notification.md) +[7. Scale Out - Add Shards to an existing Oracle Globally Distributed Database provisioned earlier with System-Managed Sharding and RAFT replication enabled](./provisioning/snr_system_sharding/snr_ssharding_scale_out_add_shards.md) +[8. Scale In - Delete an existing Shard from a working Oracle Globally Distributed Database provisioned earlier with System-Managed Sharding and RAFT reolication enabled](./provisioning/snr_system_sharding/snr_ssharding_scale_in_delete_an_existing_shard.md) -## Connecting to Shard Databases +## Connecting to Oracle Globally Distributed Database -After the Oracle Database Sharding Topology has been provisioned using the Sharding Controller in Oracle Database Kubernetes Operator, you can follow the steps in this document to connect to the Sharded Database or to the individual Shards: [Database Connectivity](./provisioning/database_connection.md) +After the Oracle Globally Distributed Database Topology has been provisioned using the Sharding Controller in Oracle Database Kubernetes Operator, you can follow the steps in this document to connect to the Oracle Globally Distributed Database or to the individual Shards: [Database Connectivity](./provisioning/database_connection.md) ## Debugging and Troubleshooting -To debug the Oracle Database Sharding Topology provisioned using the Sharding Controller of Oracle Database Kubernetes Operator, follow this document: [Debugging and troubleshooting](./provisioning/debugging.md) +To debug the Oracle Globally Distributed Database Topology provisioned using the Sharding Controller of Oracle Database Kubernetes Operator, follow this document: [Debugging and troubleshooting](./provisioning/debugging.md) diff --git a/docs/sharding/provisioning/database_connection.md b/docs/sharding/provisioning/database_connection.md index 7f64bbd5..5671520b 100644 --- a/docs/sharding/provisioning/database_connection.md +++ b/docs/sharding/provisioning/database_connection.md @@ -1,10 +1,10 @@ # Database Connectivity -The Oracle Database Sharding Topology deployed by Sharding Controller in Oracle Database Operator has an external IP available for each of the container. +The Oracle Globally Distributed Database Topology deployed by Sharding Controller in Oracle Database Operator has an external IP available for each of the container. ## Below is an example setup with connection details -Check the details of the Sharding Topology provisioned using Sharding Controller: +Check the details of the Oracle Globally Distributed Database Topology provisioned using Sharding Controller: ```sh $ kubectl get all -n shns @@ -40,5 +40,5 @@ After you have the external IP address, you can use the services shown below to 1. **Direct connection to the CATALOG Database**: Connect to the service `catalogpdb` on catalog container external IP `xx.xx.xx.116` on port `1521` 2. **Direct connection to the shard Database SHARD1**: Connect to the service `shard1pdb` on catalog container external IP `xx.xx.xx.187` on port `1521` 3. **Direct connection to the shard Database SHARD2**: Connect to the service `shard2pdb` on catalog container external IP `xx.xx.xx.197` on port `1521` -4. **Connection to SHARDED Database for DML activity (INSERT/UPDATE/DELETE)**: Connect to the service `oltp_rw_svc.catalog.oradbcloud` either on primary gsm GSM1 container external IP `xx.xx.xx.38` on port `1522` **or** on standby gsm GSM2 container external IP `xx.xx.xx.66` on port `1522` +4. **Connection to Oracle Globally Distributed Database for DML activity (INSERT/UPDATE/DELETE)**: Connect to the service `oltp_rw_svc.catalog.oradbcloud` either on primary gsm GSM1 container external IP `xx.xx.xx.38` on port `1522` **or** on standby gsm GSM2 container external IP `xx.xx.xx.66` on port `1522` 5. **Connection to the catalog database for DDL activity**: Connect to the service `GDS$CATALOG.oradbcloud` on catalog container external IP `xx.xx.xx.116` on port `1521` diff --git a/docs/sharding/provisioning/debugging.md b/docs/sharding/provisioning/debugging.md index 63e02b6a..330cfc0e 100644 --- a/docs/sharding/provisioning/debugging.md +++ b/docs/sharding/provisioning/debugging.md @@ -1,6 +1,6 @@ # Debugging and Troubleshooting -When the Oracle Database Sharding Topology is provisioned using the Oracle Database Kubernetes Operator, the debugging of an issue with the deployment depends on at which stage the issue has been seen. +When the Oracle Globally Distributed Database Topology is provisioned using the Oracle Database Kubernetes Operator, the debugging of an issue with the deployment depends on at which stage the issue has been seen. Below are the possible cases and the steps to debug such an issue: @@ -24,7 +24,7 @@ kubectl describe pod/catalog-0 -n shns In case the failure is related to the Cloud Infrastructure, you will need to troubleshooting that using the documentation from the cloud provider. -## Failure in the provisioning of the Sharded Database +## Failure in the provisioning of the Oracle Globally Distributed Database In case the failure occures after the Kubernetes Pods are created but during the execution of the scripts to create the shard databases, catalog database or the GSM, you will need to trobleshoot that at the individual Pod level. @@ -40,11 +40,11 @@ To check the logs at the GSM or at the Database level or at the host level, swit kubectl exec -it catalog-0 -n shns /bin/bash ``` -Now, you can troubleshooting the corresponding component using the alert log or the trace files etc just like a normal Sharding Database Deployment. Please refer to [Oracle Database Sharding Documentation](https://docs.oracle.com/en/database/oracle/oracle-database/19/shard/sharding-troubleshooting.html#GUID-629262E5-7910-4690-A726-A565C59BA73E) for this purpose. +Now, you can troubleshooting the corresponding component using the alert log or the trace files etc just like a normal Oracle Globally Distributed Database Deployment. Please refer to [Oracle Globally Distributed Database Documentation](https://docs.oracle.com/en/database/oracle/oracle-database/19/shard/sharding-troubleshooting.html#GUID-629262E5-7910-4690-A726-A565C59BA73E) for this purpose. ## Debugging using Database Events -* You can enable database events as part of the Sharded Database Deployment +* You can enable database events as part of the Oracle Globally Distributed Database Deployment * This can be enabled using the `envVars` * One example of enabling Database Events is [sharding_provisioning_with_db_events.md](./debugging/sharding_provisioning_with_db_events.md) \ No newline at end of file diff --git a/docs/sharding/provisioning/debugging/sharding_provisioning_with_db_events.md b/docs/sharding/provisioning/debugging/sharding_provisioning_with_db_events.md index fa73920f..763f2dc8 100644 --- a/docs/sharding/provisioning/debugging/sharding_provisioning_with_db_events.md +++ b/docs/sharding/provisioning/debugging/sharding_provisioning_with_db_events.md @@ -1,17 +1,17 @@ -# Example of provisioning Oracle Sharded Database along with DB Events set at Database Level +# Example of provisioning Oracle Globally Distributed Database along with DB Events set at Database Level **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. This example sets a Database Event at the Database Level for Catalog and Shard Databases. -The sharded database in this example is deployed with System-Managed Sharding type. In this use case, the database is created automatically using DBCA during the provisioning of the shard databases and the catalog database when the Oracle Sharding topology with System-Managed Sharding is deployed using Oracle Sharding controller. +The Oracle Globally Distributed Database in this example is deployed with System-Managed Sharding type. In this use case, the database is created automatically using DBCA during the provisioning of the shard databases and the catalog database when the Oracle Globally Distributed Database topology with System-Managed Sharding is deployed using Oracle Sharding controller. **NOTE:** In this use case, because DBCA creates the database automatically during the deployment, the time required to create the database is greater than the time it takes when the database is created by cloning from a Database Gold Image. -This example uses `sharding_provisioning_with_db_events.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: +This example uses `sharding_provisioning_with_db_events.yaml` to provision an Oracle Globally Distributed Database topology using Oracle Sharding controller with: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three sharding Pods: `shard1`, `shard2` and `shard3` +* Three Shard Database Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` * Database Event: `10798 trace name context forever, level 7` set along with `GWM_TRACE level 263` diff --git a/docs/sharding/provisioning/free/sharding_provisioning_with_free_images.md b/docs/sharding/provisioning/free/sharding_provisioning_with_free_images.md index 61641312..f6b53462 100644 --- a/docs/sharding/provisioning/free/sharding_provisioning_with_free_images.md +++ b/docs/sharding/provisioning/free/sharding_provisioning_with_free_images.md @@ -1,25 +1,25 @@ -# Example of provisioning Oracle Sharded Database with Oracle 23ai FREE Database and GSM Images +# Example of provisioning Oracle Globally Distributed Database with Oracle 23ai FREE Database and GSM Images **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. This example uses the Oracle 23ai FREE Database and GSM Images. -The sharded database in this example is deployed with System-Managed Sharding type. In this use case, the database is created automatically using DBCA during the provisioning of the shard databases and the catalog database when the Oracle Sharding topology with System-Managed Sharding is deployed using Oracle Sharding controller. +The Oracle Globally Distributed Database in this example is deployed with System-Managed Sharding type. In this use case, the database is created automatically using DBCA during the provisioning of the shard databases and the catalog database when the Oracle Sharding topology with System-Managed Sharding is deployed using Oracle Sharding controller. **NOTE:** In this use case, because DBCA creates the database automatically during the deployment, the time required to create the database is greater than the time it takes when the database is created by cloning from a Database Gold Image. This example uses `sharding_provisioning_with_free_images.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three sharding Pods: `shard1`, `shard2` and `shard3` +* Three Shard Database Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` To get the Oracle 23ai FREE Database and GSM Images: * The Oracle 23ai FREE RDBMS Image used is `container-registry.oracle.com/database/free:latest`. Check [Oracle Database Free Get Started](https://www.oracle.com/database/free/get-started/?source=v0-DBFree-ChatCTA-j2032-20240709) for details. + * Use the Oracle 23ai FREE GSM Image used is `container-registry.oracle.com/database/gsm:latest`. * To pull the above image from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. - * Use the Oracle 23ai FREE GSM Binaries `LINUX.X64_234000_gsm.zip` as listed on page [Oracle Database Free Get Started](https://www.oracle.com/database/free/get-started/?source=v0-DBFree-ChatCTA-j2032-20240709) and prepare the GSM Container Image following [Oracle Global Data Services Image](https://github.com/oracle/db-sharding/tree/master/docker-based-sharding-deployment/dockerfiles) * You need to change `dbImage` and `gsmImage` tag with the images you want to use in your enviornment in file `sharding_provisioning_with_free_images.yaml`. diff --git a/docs/sharding/provisioning/free/sharding_provisioning_with_free_images.yaml b/docs/sharding/provisioning/free/sharding_provisioning_with_free_images.yaml index 7e39b3b2..53e4191f 100644 --- a/docs/sharding/provisioning/free/sharding_provisioning_with_free_images.yaml +++ b/docs/sharding/provisioning/free/sharding_provisioning_with_free_images.yaml @@ -41,8 +41,8 @@ spec: storageClass: oci dbImage: container-registry.oracle.com/database/free:latest dbImagePullSecret: ocr-reg-cred - gsmImage: - gsmImagePullSecret: + gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImagePullSecret: ocr-reg-cred dbEdition: "free" isExternalSvc: False isDeleteOraPvc: True diff --git a/docs/sharding/provisioning/provisioning_persistent_volume_having_db_gold_image.md b/docs/sharding/provisioning/provisioning_persistent_volume_having_db_gold_image.md index 7d3312a6..0a453c15 100644 --- a/docs/sharding/provisioning/provisioning_persistent_volume_having_db_gold_image.md +++ b/docs/sharding/provisioning/provisioning_persistent_volume_having_db_gold_image.md @@ -2,7 +2,7 @@ In this use case, a Persistent Volume with a Oracle Database Gold Image is created. - This is required when you do not already have a Persistent Volume with a Database Gold Image from which you can clone database to save time while deploying Oracle Sharding topology using Oracle Sharding controller. + This is required when you do not already have a Persistent Volume with a Database Gold Image from which you can clone database to save time while deploying Oracle Globally Distributed Database topology using Oracle Sharding controller. This example uses file `oraclesi.yaml` to provision a single instance Oracle Database: diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md index ba72be25..2698bce9 100644 --- a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md @@ -1,14 +1,14 @@ -# Provisioning System managed Sharding Topology with Raft replication enabled by cloning database from your own Database Gold Image across Availability Domains(ADs) +# Provisioning Oracle Globally Distributed Database Topology with System-managed sharding and Raft replication enabled by cloning database from your own Database Gold Image across Availability Domains(ADs) **NOTE: RAFT Replication Feature is available only for Oracle 23c RDBMS and Oracle 23c GSM version.** **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. -In this test case, you provision the System managed Sharding Topology with Raft replication enabled while provisioning the Catalog and Shard Databases by cloning from an existing Oracle Database Gold Image created earlier. +In this test case, you provision the Oracle Globally Distributed Database Topology with System-managed sharding and Raft replication enabled while provisioning the Catalog and Shard Databases by cloning from an existing Oracle Database Gold Image created earlier. This use case applies when you want to provision the database Pods on a Kubernetes Node in any availability domain (AD), which can also be different from the availability domain (AD) of the Block Volume that has the Oracle Database Gold Image provisioned earlier. -Choosing this option takes substantially less time during the Oracle Database Sharding Topology setup across ADs. +Choosing this option takes substantially less time during the Oracle Globally Distributed Database Topology setup across ADs. NOTE: @@ -21,10 +21,10 @@ NOTE: ```sh kubectl get pv -n shns ``` -2. Create a Block Volume Backup for this Block Volume, and use the OCID of the Block Volume Backup in the next step. This example uses `snr_ssharding_shard_prov_clone_across_ads.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: +2. Create a Block Volume Backup for this Block Volume, and use the OCID of the Block Volume Backup in the next step. This example uses `snr_ssharding_shard_prov_clone_across_ads.yaml` to provision an Oracle Globally Distributed Database topology using Oracle Sharding Database controller with: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three sharding Pods: `shard1`, `shard2` and `shard3` +* Three Shard Database Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` * Database Cloning from the `BLOCK VOLUME FULL BACKUP` of the Persistent Volume which had the Gold Image. @@ -33,12 +33,12 @@ NOTE: NOTE: In this case, the Persistent Volume with DB Gold Image was provisioned in the Availability Domain `PHX-AD-1`. The Shards and Catalog will be provisioned across multiple Availability Domains by cloning the database. -**NOTE:** Provisioning the Sharded Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. +**NOTE:** Provisioning the Oracle Globally Distributed Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `snr_ssharding_shard_prov_clone_across_ads.yaml`. - * The `dbImage` used during provisioning the Persistent Volume with Database Gold Image and the `dbImage` used for deploying the Shard or Catalog Database by cloning should be same. + * The `dbImage` used during provisioning the Persistent Volume with Database Gold Image and the `dbImage` used for deploying the Shard or Catalog Database by cloning should be same. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) Use the file: [snr_ssharding_shard_prov_clone_across_ads.yaml](./snr_ssharding_shard_prov_clone_across_ads.yaml) for this use case as below: diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md index cf4240f7..b43a9158 100644 --- a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md @@ -1,14 +1,14 @@ -# Provisioning System managed Sharding Topology with Raft replication enabled by cloning database from your own Database Gold Image in the same Availability Domain(AD) +# Provisioning Oracle Globally Distributed Database Topology with System-managed sharding and Raft replication enabled by cloning database from your own Database Gold Image in the same Availability Domain(AD) **NOTE: RAFT Replication Feature is available only for Oracle 23c RDBMS and Oracle 23c GSM version.** **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. -In this test case, you provision the System managed Sharding Topology with Raft replication enabled while provisioning the Catalog and Shard Databases by cloning from an existing Oracle Database Gold Image created earlier. +In this test case, you provision the Oracle Globally Distributed Database Topology with System-managed sharding and Raft replication enabled while provisioning the Catalog and Shard Databases by cloning from an existing Oracle Database Gold Image created earlier. This use case applies when you are cloning from a Block Volume, and you can clone _only_ in the same availability domain (AD). The result is that the cloned shard database PODs can be created _only_ in the same AD where the Gold Image Block Volume is present. -Choosing this option takes substantially less time during the Oracle Database Sharding Topology setup. +Choosing this option takes substantially less time during the Oracle Globally Distributed Database Topology setup. **NOTE** For this step, the Persistent Volume that has the Oracle Database Gold Image is identified using its OCID. @@ -18,10 +18,10 @@ Choosing this option takes substantially less time during the Oracle Database Sh kubectl get pv -n shns ``` -2. This example uses `snr_ssharding_shard_prov_clone.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: +2. This example uses `snr_ssharding_shard_prov_clone.yaml` to provision an Oracle Globally Distributed Database topology using Oracle Sharding controller with: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three sharding Pods: `shard1`, `shard2` and `shard3` +* Three Shard Database Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` * Database Cloning from the Database Gold Image present in Persistent Volume having OCID: `ocid1.volume.oc1.phx.abyhqljr3z3w72t6ay5eud7d5w3kdfhktfp6gwb6euy5tzwfaxgmbvwqlvsq` @@ -29,14 +29,14 @@ Choosing this option takes substantially less time during the Oracle Database Sh NOTE: In this case, the Persistent Volume with DB Gold Image was provisioned in the Availability Domain `PHX-AD-1`. The Shards and Catalog will be provisioned in the same Availability Domain `PHX-AD-1` by cloning the database. +**NOTE:** Provisioning the Oracle Globally Distributed Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. + In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `snr_ssharding_shard_prov_clone.yaml`. * The `dbImage` used during provisioning the Persistent Volume with Database Gold Image and the `dbImage` used for deploying the Shard or Catalog Database by cloning should be same. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) -**NOTE:** Provisioning the Sharded Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. - Use the file: [snr_ssharding_shard_prov_clone.yaml](./snr_ssharding_shard_prov_clone.yaml) for this use case as below: 1. Deploy the `snr_ssharding_shard_prov_clone.yaml` file: diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_chunks_specified.md b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_chunks_specified.md index 44972090..d6171986 100644 --- a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_chunks_specified.md +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_chunks_specified.md @@ -1,19 +1,19 @@ -# Provisioning System-Managed Sharding Topology with Raft replication enabled with number of chunks specified +# Provisioning Oracle Globally Distributed Database Topology with System-managed sharding and Raft replication enabled with number of chunks specified **NOTE: RAFT Replication Feature is available only for Oracle 23c RDBMS and Oracle 23c GSM version.** **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. -In this use case, the database is created automatically using DBCA during the provisioning of the shard databases and the catalog database when the Oracle Sharding topology with System-Managed with RAFT Replication enabled is deployed using Oracle Sharding controller. +In this use case, the database is created automatically using DBCA during the provisioning of the shard databases and the catalog database when the Oracle Globally Distributed Database topology with System-Managed sharding and RAFT Replication enabled is deployed using Oracle Sharding controller. **NOTE** In this use case, because DBCA creates the database automatically during the deployment, the time required to create the database is greater than the time it takes when the database is created by cloning from a Database Gold Image. -By default, the System-Managed with RAFT Replication deploys the Sharded Database with 360 chunks per Shard Database (because there are 3 chunks created for each replication unit). In this example, the Sharded Database will be deployed with non-default number of chunks specified using parameter `CATALOG_CHUNKS`. +By default, the System-Managed with RAFT Replication deploys the Oracle Globally Distributed Database with 360 chunks per Shard Database (because there are 3 chunks created for each replication unit). In this example, the Oracle Globally Distributed Database will be deployed with non-default number of chunks specified using parameter `CATALOG_CHUNKS`. -This example uses `snr_ssharding_shard_prov_chunks.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: +This example uses `snr_ssharding_shard_prov_chunks.yaml` to provision an Oracle Globally Distributed Database topology using Oracle Sharding controller with: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three sharding Pods: `shard1`, `shard2` and `shard3` +* Three Shard Database Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Total number of chunks as `120` specified by variable `CATALOG_CHUNKS` (it will be 120 chunks per shard) * Namespace: `shns` diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_control_on_resources.md b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_control_on_resources.md index 9cfd6afb..c432310d 100644 --- a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_control_on_resources.md +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_control_on_resources.md @@ -1,15 +1,15 @@ -# Provisioning System-Managed Sharding Topology with Raft replication enabled with additional control on resources like Memory and CPU allocated to Pods +# Provisioning Oracle Globally Distributed Database Topology with System-managed sharding and Raft replication enabled with additional control on resources like Memory and CPU allocated to Pods **NOTE: RAFT Replication Feature is available only for Oracle 23c RDBMS and Oracle 23c GSM version.** **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. -In this use case, there are additional tags used to control resources such as CPU and Memory used by the different Pods when the Oracle Sharding topology with System-Managed with RAFT Replication is deployed using Oracle Sharding controller. +In this use case, there are additional tags used to control resources such as CPU and Memory used by the different Pods when the Oracle Globally Distributed Database Topology with System-managed sharding and RAFT Replication is deployed using Oracle Sharding controller. This example uses `snr_ssharding_shard_prov_memory_cpu.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three sharding Pods: `shard1`, `shard2` and `shard3` +* Three Shard Database Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` * Tags `memory` and `cpu` to control the Memory and CPU of the PODs diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_notification_using_oci_notification.md b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_notification_using_oci_notification.md index d4cb11de..d45b2911 100644 --- a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_notification_using_oci_notification.md +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_notification_using_oci_notification.md @@ -1,15 +1,15 @@ -# Provisioning System managed Sharding Topology with Raft replication enabled and send Notification using OCI Notification Service +# Provisioning Oracle Globally Distributed Database Topology with System-managed sharding and Raft replication enabled and send Notification using OCI Notification Service **NOTE: RAFT Replication Feature is available only for Oracle 23c RDBMS and Oracle 23c GSM version.** **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. -This use case demonstrates how to use a notification service like OCI Notification service to send an email notification when a particular operation is completed on an Oracle Database sharding topology provisioned using the Oracle Database sharding controller. +This use case demonstrates how to use a notification service like OCI Notification service to send an email notification when a particular operation is completed on an Oracle Globally Distributed Database Topology provisioned using the Oracle Database sharding controller. -This example uses `snr_ssharding_shard_prov_send_notification.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: +This example uses `snr_ssharding_shard_prov_send_notification.yaml` to provision an Oracle Globally Distributed Database Topology using Oracle Sharding controller with: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three sharding Pods: `shard1`, `shard2` and `shard3` +* Three Shard Database Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` * Database Cloning from the `BLOCK VOLUME FULL BACKUP` of the Persistent Volume that has the Database Gold Image created earlier. @@ -64,14 +64,14 @@ To do this: kubectl describe secret my-secret -n shns ``` +**NOTE:** Provisioning the Oracle Globally Distributed Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. + In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `snr_ssharding_shard_prov_send_notification.yaml`. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) * In case you want to use the [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then you will need to add the additional parameter `dbEdition: "free"` to the below .yaml file. -**NOTE:** Provisioning the Sharded Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. - Use the file: [snr_ssharding_shard_prov_send_notification.yaml](./snr_ssharding_shard_prov_send_notification.yaml) for this use case as below: 1. Deploy the `snr_ssharding_shard_prov_send_notification.yaml` file: diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_without_db_gold_image.md b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_without_db_gold_image.md index 892741a5..c8568f1e 100644 --- a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_without_db_gold_image.md +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_without_db_gold_image.md @@ -1,17 +1,17 @@ -# Provisioning System-Managed Sharding Topology with Raft replication enabled without Database Gold Image +# Provisioning Oracle Globally Distributed Database Topology with System-managed sharding and Raft replication enabled without Database Gold Image **NOTE: RAFT Replication Feature is available only for Oracle 23c RDBMS and Oracle 23c GSM version.** **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. -In this use case, the database is created automatically using DBCA during the provisioning of the shard databases and the catalog database when the Oracle Sharding topology with System-Managed with RAFT Replication enabled is deployed using Oracle Sharding controller. +In this use case, the database is created automatically using DBCA during the provisioning of the shard databases and the catalog database when the Oracle Globally Distributed Database topology with System-Managed sharding and RAFT Replication enabled is deployed using Oracle Sharding controller. **NOTE** In this use case, because DBCA creates the database automatically during the deployment, the time required to create the database is greater than the time it takes when the database is created by cloning from a Database Gold Image. -This example uses `snr_ssharding_shard_prov.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: +This example uses `snr_ssharding_shard_prov.yaml` to provision an Oracle Globally Distributed Database topology using Oracle Sharding controller with: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three sharding Pods: `shard1`, `shard2` and `shard3` +* Three Shard Database Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` * `RAFT Replication` enabled diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_scale_in_delete_an_existing_shard.md b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_scale_in_delete_an_existing_shard.md index fe3157ec..dc026a7c 100644 --- a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_scale_in_delete_an_existing_shard.md +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_scale_in_delete_an_existing_shard.md @@ -1,4 +1,4 @@ -# Scale In - Delete an existing Shard from a working Oracle Sharded Database provisioned earlier with System-Managed Sharding with RAFT reolication enabled +# Scale In - Delete an existing Shard from a working Oracle Globally Distributed Database provisioned earlier with System-Managed Sharding and RAFT reolication enabled **NOTE: RAFT Replication Feature is available only for Oracle 23c RDBMS and Oracle 23c GSM version.** @@ -8,10 +8,10 @@ This use case demonstrates how to delete an existing Shard from an existing Orac **NOTE** The deletion of a shard is done after verifying the Chunks have been moved out of that shard. -In this use case, the existing database Sharding is having: +In this use case, the existing Oracle Globally Distributed Database is having: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Five sharding Pods: `shard1`,`shard2`,`shard3`,`shard4` and `shard5` +* Five Shard Database Pods: `shard1`,`shard2`,`shard3`,`shard4` and `shard5` * One Catalog Pod: `catalog` * Namespace: `shns` * `RAFT Replication` enabled diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_scale_out_add_shards.md b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_scale_out_add_shards.md index 03423e72..962bf64c 100644 --- a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_scale_out_add_shards.md +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_scale_out_add_shards.md @@ -1,15 +1,15 @@ -# Scale Out - Add Shards to an existing Oracle Sharded Database provisioned earlier with System-Managed Sharding with RAFT replication enabled +# Scale Out - Add Shards to an existing Oracle Globally Distributed Database Topology provisioned earlier with System-Managed Sharding and RAFT replication enabled **NOTE: RAFT Replication Feature is available only for Oracle 23c RDBMS and Oracle 23c GSM version.** **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. -This use case demonstrates adding a new shard to an existing Oracle Database sharding topology with System-Managed with RAFT Replication enabled provisioned earlier using Oracle Database Sharding controller. +This use case demonstrates adding a new shard to an existing Oracle Globally Distributed Database topology with System-Managed sharding with RAFT Replication enabled provisioned earlier using Oracle Database Sharding controller. -In this use case, the existing Oracle Database sharding topology is having: +In this use case, the existing Oracle Globally Distributed Database topology is having: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three sharding Pods: `shard1`, `shard2` and `shard3` +* Three Shard Database Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` * `RAFT Replication` enabled @@ -18,7 +18,7 @@ In this example, we are using pre-built Oracle Database and Global Data Services * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `snr_ssharding_shard_prov_extshard.yaml`. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) - * If the existing Sharding Topology was deployed using [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then the additional parameter `dbEdition: "free"` will be needed for the below .yaml file as well. + * If the existing Oracle Globally Distributed Database Topology was deployed using [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then the additional parameter `dbEdition: "free"` will be needed for the below .yaml file as well. This use case adds two new shards `shard4`,`shard5` to above Sharding Topology. diff --git a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md index 64e2f4eb..e015f916 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md +++ b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md @@ -1,4 +1,4 @@ -# Provisioning Oracle Sharded Database with System-Managed Sharding by cloning database from your own Database Gold Image across Availability Domains(ADs) +# Provisioning Oracle Globally Distributed Database with System-Managed Sharding by cloning database from your own Database Gold Image across Availability Domains(ADs) **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. @@ -6,7 +6,7 @@ In this test case, you provision the Oracle Database sharding topology with Syst This use case applies when you want to provision the database Pods on a Kubernetes Node in any availability domain (AD), which can also be different from the availability domain (AD) of the Block Volume that has the Oracle Database Gold Image provisioned earlier. -Choosing this option takes substantially less time during the Oracle Database Sharding Topology setup across ADs. +Choosing this option takes substantially less time during the Oracle Globally Distributed Database Topology setup across ADs. NOTE: @@ -22,7 +22,7 @@ NOTE: 2. Create a Block Volume Backup for this Block Volume, and use the OCID of the Block Volume Backup in the next step. This example uses `ssharding_shard_prov_clone_across_ads.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three sharding Pods: `shard1`, `shard2` and `shard3` +* Three Shard Database Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` * Database Cloning from the `BLOCK VOLUME FULL BACKUP` of the Persistent Volume which had the Gold Image. @@ -31,11 +31,11 @@ NOTE: NOTE: In this case, the Persistent Volume with DB Gold Image was provisioned in the Availability Domain `PHX-AD-1`. The Shards and Catalog will be provisioned across multiple Availability Domains by cloning the database. -**NOTE:** Provisioning the Sharded Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. +**NOTE:** Provisioning the Oracle Globally Distributed Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. - * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `ssharding_shard_prov_clone_across_ads.yaml`. + * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `ssharding_shard_prov_clone_across_ads.yaml`. * The `dbImage` used during provisioning the Persistent Volume with Database Gold Image and the `dbImage` used for deploying the Shard or Catalog Database by cloning should be same. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) diff --git a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md index f7aef949..fb16e3cd 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md +++ b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md @@ -1,4 +1,4 @@ -# Provisioning Oracle Sharded Database with System-Managed Sharding by cloning database from your own Database Gold Image in the same Availability Domain(AD) +# Provisioning Oracle Globally Distributed Database with System-Managed Sharding by cloning database from your own Database Gold Image in the same Availability Domain(AD) **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. @@ -6,7 +6,7 @@ In this case, the database is created automatically by cloning from an existing This use case applies when you are cloning from a Block Volume, and you can clone _only_ in the same availability domain (AD). The result is that the cloned shard database PODs can be created _only_ in the same AD where the Gold Image Block Volume is present. -Choosing this option takes substantially less time during the Oracle Database Sharding Topology setup. +Choosing this option takes substantially less time during the Oracle Globally Distributed Database Topology setup. **NOTE** For this step, the Persistent Volume that has the Oracle Database Gold Image is identified using its OCID. @@ -19,12 +19,12 @@ Choosing this option takes substantially less time during the Oracle Database Sh 2. This example uses `ssharding_shard_prov_clone.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three sharding Pods: `shard1`, `shard2` and `shard3` +* Three Shard Database Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` * Database Cloning from the Database Gold Image present in Persistent Volume having OCID: `ocid1.volume.oc1.phx.abyhqljr3z3w72t6ay5eud7d5w3kdfhktfp6gwb6euy5tzwfaxgmbvwqlvsq` -**NOTE:** Provisioning the Sharded Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. +**NOTE:** Provisioning the Oracle Globally Distributed Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. NOTE: In this case, the Persistent Volume with DB Gold Image was provisioned in the Availability Domain `PHX-AD-1`. The Shards and Catalog will be provisioned in the same Availability Domain `PHX-AD-1` by cloning the database. diff --git a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_chunks_specified.md b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_chunks_specified.md index 0c6ea8fe..b824ab03 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_chunks_specified.md +++ b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_chunks_specified.md @@ -1,4 +1,4 @@ -# Provisioning Oracle Sharded Database with System-Managed Sharding with number of chunks specified +# Provisioning Oracle Globally Distributed Database with System-Managed Sharding with number of chunks specified **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. @@ -6,12 +6,12 @@ In this use case, the database is created automatically using DBCA during the pr **NOTE:** In this use case, because DBCA creates the database automatically during the deployment, the time required to create the database is greater than the time it takes when the database is created by cloning from a Database Gold Image. -By default, the System-Managed Sharding deploys the Sharded Database with 120 chunks per Shard Database. If, for example, we have three shards in the Sharded Database, it will be total of 360 chunks. In this example, the Sharded Database will be deployed with non-default number of chunks specified using parameter `CATALOG_CHUNKS`. +By default, the System-Managed Sharding deploys the Oracle Globally Distributed Database with 120 chunks per Shard Database. For example, if we have three shards in the Oracle Globally Distributed Database, it will be total of 360 chunks. In this example, the Oracle Globally Distributed Database will be deployed with non-default number of chunks specified using parameter `CATALOG_CHUNKS`. This example uses `ssharding_shard_prov_chunks.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three sharding Pods: `shard1`, `shard2` and `shard3` +* Three Shard Database Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Total number of chunks as `120` specified by variable `CATALOG_CHUNKS` (it will be 40 chunks per shard) * Namespace: `shns` diff --git a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_control_on_resources.md b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_control_on_resources.md index c4f45a48..153a40a9 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_control_on_resources.md +++ b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_control_on_resources.md @@ -1,4 +1,4 @@ -# Provisioning Oracle Sharded Database with System-Managed Sharding with additional control on resources like Memory and CPU allocated to Pods +# Provisioning Oracle Globally Distributed Database with System-Managed Sharding with additional control on resources like Memory and CPU allocated to Pods **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. @@ -7,7 +7,7 @@ In this use case, there are additional tags used to control resources such as CP This example uses `ssharding_shard_prov_memory_cpu.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three sharding Pods: `shard1`, `shard2` and `shard3` +* Three Shard Database Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` * Tags `memory` and `cpu` to control the Memory and CPU of the PODs diff --git a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_notification_using_oci_notification.md b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_notification_using_oci_notification.md index 1a6a1ee3..7ec24439 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_notification_using_oci_notification.md +++ b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_notification_using_oci_notification.md @@ -1,13 +1,13 @@ -# Provisioning Oracle Sharded Database with System-Managed Sharding and send Notification using OCI Notification Service +# Provisioning Oracle Globally Distributed Database with System-Managed Sharding and send Notification using OCI Notification Service **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. -This use case demonstrates how to use a notification service like OCI Notification service to send an email notification when a particular operation is completed on an Oracle Database sharding topology provisioned using the Oracle Database sharding controller. +This use case demonstrates how to use a notification service like OCI Notification service to send an email notification when a particular operation is completed on an Oracle Globally Distributed Database topology provisioned using the Oracle Database sharding controller. This example uses `ssharding_shard_prov_send_notification.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three sharding Pods: `shard1`, `shard2` and `shard3` +* Three Shard Database Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` * Database Cloning from the `BLOCK VOLUME FULL BACKUP` of the Persistent Volume that has the Database Gold Image created earlier. @@ -67,7 +67,7 @@ In this example, we are using pre-built Oracle Database and Global Data Services * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) * In case you want to use the [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then you will need to add the additional parameter `dbEdition: "free"` to the below .yaml file. -**NOTE:** Provisioning the Sharded Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. +**NOTE:** Provisioning the Oracle Globally Distributed Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. Use the file: [ssharding_shard_prov_send_notification.yaml](./ssharding_shard_prov_send_notification.yaml) for this use case as below: diff --git a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_without_db_gold_image.md b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_without_db_gold_image.md index b223d1af..b262407f 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_without_db_gold_image.md +++ b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_without_db_gold_image.md @@ -1,4 +1,4 @@ -# Provisioning Oracle Sharded Database with System-Managed Sharding without Database Gold Image +# Provisioning Oracle Globally Distributed Database with System-Managed Sharding without Database Gold Image **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. @@ -9,7 +9,7 @@ In this use case, the database is created automatically using DBCA during the pr This example uses `ssharding_shard_prov.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three sharding Pods: `shard1`, `shard2` and `shard3` +* Three Shard Database Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` diff --git a/docs/sharding/provisioning/system_sharding/ssharding_scale_in_delete_an_existing_shard.md b/docs/sharding/provisioning/system_sharding/ssharding_scale_in_delete_an_existing_shard.md index bca34253..8cac01cd 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_scale_in_delete_an_existing_shard.md +++ b/docs/sharding/provisioning/system_sharding/ssharding_scale_in_delete_an_existing_shard.md @@ -1,4 +1,4 @@ -# Scale In - Delete an existing Shard from a working Oracle Sharded Database provisioned earlier with System-Managed Sharding +# Scale In - Delete an existing Shard from a working Oracle Globally Distributed Database provisioned earlier with System-Managed Sharding **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. @@ -6,10 +6,10 @@ This use case demonstrates how to delete an existing Shard from an existing Orac **NOTE** The deletion of a shard is done after verifying the Chunks have been moved out of that shard. -In this use case, the existing database Sharding is having: +In this use case, the existing Oracle Globally Distributed Database is having: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Five sharding Pods: `shard1`,`shard2`,`shard3`,`shard4` and `shard5` +* Five Shard Database Pods: `shard1`,`shard2`,`shard3`,`shard4` and `shard5` * One Catalog Pod: `catalog` * Namespace: `shns` @@ -21,7 +21,7 @@ In this example, we are using pre-built Oracle Database and Global Data Services NOTE: Use tag `isDelete: enable` to delete the shard you want. -This use case deletes the shard `shard4` from the above Sharding Topology. +This use case deletes the shard `shard4` from the above Oracle Globally Distributed Database Topology. Use the file: [ssharding_shard_prov_delshard.yaml](./ssharding_shard_prov_delshard.yaml) for this use case as below: diff --git a/docs/sharding/provisioning/system_sharding/ssharding_scale_out_add_shards.md b/docs/sharding/provisioning/system_sharding/ssharding_scale_out_add_shards.md index 1db8e6c3..091a01a0 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_scale_out_add_shards.md +++ b/docs/sharding/provisioning/system_sharding/ssharding_scale_out_add_shards.md @@ -1,13 +1,13 @@ -# Scale Out - Add Shards to an existing Oracle Sharded Database provisioned earlier with System-Managed Sharding +# Scale Out - Add Shards to an existing Oracle Globally Distributed Database provisioned earlier with System-Managed Sharding **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. This use case demonstrates adding a new shard to an existing Oracle Database sharding topology with System-Managed Sharding provisioned earlier using Oracle Database Sharding controller. -In this use case, the existing Oracle Database sharding topology is having: +In this use case, the existing Oracle Globally Distributed Database topology is having: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three sharding Pods: `shard1`, `shard2` and `shard3` +* Three Shard Database Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` @@ -15,9 +15,9 @@ In this example, we are using pre-built Oracle Database and Global Data Services * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `ssharding_shard_prov_extshard.yaml`. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) - * If the existing Sharding Topology was deployed using [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then the additional parameter `dbEdition: "free"` will be needed for the below .yaml file as well. + * If the existing Oracle Globally Distributed Database Topology was deployed using [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then the additional parameter `dbEdition: "free"` will be needed for the below .yaml file as well. -This use case adds two new shards `shard4`,`shard5` to above Sharding Topology. +This use case adds two new shards `shard4`,`shard5` to above Oracle Globally Distributed Database Topology. Use the file: [ssharding_shard_prov_extshard.yaml](./ssharding_shard_prov_extshard.yaml) for this use case as below: diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_from_gold_image_across_ads.md b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_from_gold_image_across_ads.md index 9b2905e8..dda8a350 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_from_gold_image_across_ads.md +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_from_gold_image_across_ads.md @@ -1,12 +1,12 @@ -# Provisioning Oracle Sharded Database with User Defined Sharding by cloning database from your own Database Gold Image across Availability Domains(ADs) +# Provisioning Oracle Globally Distributed Database with User-Defined Sharding by cloning database from your own Database Gold Image across Availability Domains(ADs) **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. -In this test case, you provision the Oracle Database sharding topology with User Defined Sharding while provisioning the Catalog and Shard Databases by cloning from an existing Oracle Database Gold Image created earlier. +In this test case, you provision the Oracle Globally Distributed Database topology with User-Defined Sharding while provisioning the Catalog and Shard Databases by cloning from an existing Oracle Database Gold Image created earlier. This use case applies when you want to provision the database Pods on a Kubernetes Node in any availability domain (AD), which can also be different from the availability domain (AD) of the Block Volume that has the Oracle Database Gold Image provisioned earlier. -Choosing this option takes substantially less time during the Oracle Database Sharding Topology setup across ADs. +Choosing this option takes substantially less time during the Oracle Globally Distributed Database Topology setup across ADs. NOTE: @@ -19,15 +19,15 @@ NOTE: ```sh kubectl get pv -n shns ``` -2. Create a Block Volume Backup for this Block Volume, and use the OCID of the Block Volume Backup in the next step. This example uses `udsharding_shard_prov_clone_across_ads.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: +2. Create a Block Volume Backup for this Block Volume, and use the OCID of the Block Volume Backup in the next step. This example uses `udsharding_shard_prov_clone_across_ads.yaml` to provision an Oracle Globally Distributed Database topology using Oracle Sharding controller with: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three sharding Pods: `shard1`, `shard2` and `shard3` +* Three Shard Database Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` * Database Cloning from the `BLOCK VOLUME FULL BACKUP` of the Persistent Volume which had the Gold Image. * OCID of the Block Volume Backup: `ocid1.volumebackup.oc1.phx.abyhqljrxtv7tu5swqb3lzc7vpzwbwzdktd2y4k2vjjy2srmgu2w7bqdftjq` -* User Defined Sharding is specified using `shardingType: USER` +* User-Defined Sharding is specified using `shardingType: USER` NOTE: In this case, the Persistent Volume with DB Gold Image was provisioned in the Availability Domain `PHX-AD-1`. The Shards and Catalog will be provisioned across multiple Availability Domains by cloning the database. @@ -37,7 +37,7 @@ In this example, we are using pre-built Oracle Database and Global Data Services * The `dbImage` used during provisioning the Persistent Volume with Database Gold Image and the `dbImage` used for deploying the Shard or Catalog Database by cloning should be same. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) -**NOTE:** Provisioning the Sharded Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. +**NOTE:** Provisioning the Oracle Globally Distributed Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. Use the file: [udsharding_shard_prov_clone_across_ads.yaml](./udsharding_shard_prov_clone_across_ads.yaml) for this use case as below: diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_gold_image_in_same_ad.md b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_gold_image_in_same_ad.md index a4669667..34fa2867 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_gold_image_in_same_ad.md +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_gold_image_in_same_ad.md @@ -1,4 +1,4 @@ -# Provisioning Oracle Sharded Database with User Defined Sharding by cloning database from your own Database Gold Image in the same Availability Domain(AD) +# Provisioning Oracle Globally Distributed Database with User-Defined Sharding by cloning database from your own Database Gold Image in the same Availability Domain(AD) **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. @@ -6,7 +6,7 @@ In this case, the database is created automatically by cloning from an existing This use case applies when you are cloning from a Block Volume, and you can clone _only_ in the same availability domain (AD). The result is that the cloned shard database PODs can be created _only_ in the same AD where the Gold Image Block Volume is present. -Choosing this option takes substantially less time during the Oracle Database Sharding Topology setup. +Choosing this option takes substantially less time during the Oracle Globally Distributed Database Topology setup. **NOTE** For this step, the Persistent Volume that has the Oracle Database Gold Image is identified using its OCID. @@ -16,14 +16,14 @@ Choosing this option takes substantially less time during the Oracle Database Sh kubectl get pv -n shns ``` -2. This example uses `udsharding_shard_prov_clone.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: +2. This example uses `udsharding_shard_prov_clone.yaml` to provision an Oracle Globally Distributed Database topology using Oracle Sharding controller with: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three sharding Pods: `shard1`, `shard2` and `shard3` +* Three Shard Database Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` * Database Cloning from the Database Gold Image present in Persistent Volume having OCID: `ocid1.volume.oc1.phx.abyhqljr3z3w72t6ay5eud7d5w3kdfhktfp6gwb6euy5tzwfaxgmbvwqlvsq` -* User Defined Sharding is specified using `shardingType: USER` +* User-Defined Sharding is specified using `shardingType: USER` NOTE: In this case, the Persistent Volume with DB Gold Image was provisioned in the Availability Domain `PHX-AD-1`. The Shards and Catalog will be provisioned in the same Availability Domain `PHX-AD-1` by cloning the database. @@ -33,7 +33,7 @@ In this example, we are using pre-built Oracle Database and Global Data Services * The `dbImage` used during provisioning the Persistent Volume with Database Gold Image and the `dbImage` used for deploying the Shard or Catalog Database by cloning should be same. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) -**NOTE:** Provisioning the Sharded Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. +**NOTE:** Provisioning the Oracle Globally Distributed Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. Use the file: [udsharding_shard_prov_clone.yaml](./udsharding_shard_prov_clone.yaml) for this use case as below: diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_with_control_on_resources.md b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_with_control_on_resources.md index b52b8745..8836baeb 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_with_control_on_resources.md +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_with_control_on_resources.md @@ -1,18 +1,18 @@ -# Provisioning Oracle Sharded Database with User Defined Sharding with additional control on resources like Memory and CPU allocated to Pods +# Provisioning Oracle Globally Distributed Database with User-Defined Sharding with additional control on resources like Memory and CPU allocated to Pods **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. -In this use case, there are additional tags used to control resources such as CPU and Memory used by the different Pods when the Oracle Sharding topology with User Defined Sharding is deployed using Oracle Sharding controller. +In this use case, there are additional tags used to control resources such as CPU and Memory used by the different Pods when the Oracle Globally Distributed Database topology with User-Defined Sharding is deployed using Oracle Sharding controller. This example uses `udsharding_shard_prov_memory_cpu.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three sharding Pods: `shard1`, `shard2` and `shard3` +* Three Shard Database Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` * Tags `memory` and `cpu` to control the Memory and CPU of the PODs * Additional tags `INIT_SGA_SIZE` and `INIT_PGA_SIZE` to control the SGA and PGA allocation at the database level -* User Defined Sharding is specified using `shardingType: USER` +* User-Defined Sharding is specified using `shardingType: USER` In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_with_notification_using_oci_notification.md b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_with_notification_using_oci_notification.md index 640301a2..ea3a2802 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_with_notification_using_oci_notification.md +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_with_notification_using_oci_notification.md @@ -1,19 +1,19 @@ -# Provisioning Oracle Sharded Database with User Defined Sharding and send Notification using OCI Notification Service +# Provisioning Oracle Globally Distributed Database with User-Defined Sharding and send Notification using OCI Notification Service **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. -This use case demonstrates how to use a notification service like OCI Notification service to send an email notification when a particular operation is completed on an Oracle Database sharding topology provisioned using the Oracle Database sharding controller. +This use case demonstrates how to use a notification service like OCI Notification service to send an email notification when a particular operation is completed on an Oracle Globally Distributed Database topology provisioned using the Oracle Database sharding controller. -This example uses `udsharding_shard_prov_send_notification.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: +This example uses `udsharding_shard_prov_send_notification.yaml` to provision an Oracle Globally Distributed Database topology using Oracle Sharding controller with: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three sharding Pods: `shard1`, `shard2` and `shard3` +* Three Shard Database Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` * Database Cloning from the `BLOCK VOLUME FULL BACKUP` of the Persistent Volume that has the Database Gold Image created earlier. * OCID of the Block Volume Backup: `ocid1.volumebackup.oc1.phx.abyhqljrxtv7tu5swqb3lzc7vpzwbwzdktd2y4k2vjjy2srmgu2w7bqdftjq` * Configmap to send notification email when a particular operation is completed. For example: When a shard is added. -* User Defined Sharding is specified using `shardingType: USER` +* User-Defined Sharding is specified using `shardingType: USER` **NOTE:** @@ -68,7 +68,7 @@ In this example, we are using pre-built Oracle Database and Global Data Services * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) * In case you want to use the [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then you will need to add the additional parameter `dbEdition: "free"` to the below .yaml file. -**NOTE:** Provisioning the Sharded Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. +**NOTE:** Provisioning the Oracle Globally Distributed Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. Use the file: [udsharding_shard_prov_send_notification.yaml](./udsharding_shard_prov_send_notification.yaml) for this use case as below: diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_without_db_gold_image.md b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_without_db_gold_image.md index 2be5ac9f..5b1d2db0 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_without_db_gold_image.md +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_without_db_gold_image.md @@ -1,18 +1,18 @@ -# Provisioning Oracle Sharded Database with User Defined Sharding without Database Gold Image +# Provisioning Oracle Globally Distributed Database with User-Defined Sharding without Database Gold Image **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. -In this use case, the database is created automatically using DBCA during the provisioning of the shard databases and the catalog database when the Oracle Sharding topology with User Defined Sharding is deployed using Oracle Sharding controller. +In this use case, the database is created automatically using DBCA during the provisioning of the shard databases and the catalog database when the Oracle Sharding topology with User-Defined Sharding is deployed using Oracle Sharding controller. **NOTE** In this use case, because DBCA creates the database automatically during the deployment, the time required to create the database is greater than the time it takes when the database is created by cloning from a Database Gold Image. -This example uses `udsharding_shard_prov.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: +This example uses `udsharding_shard_prov.yaml` to provision an Oracle Globally Distributed Database topology using Oracle Sharding controller with: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three sharding Pods: `shard1`, `shard2` and `shard3` +* Three Shard Database Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` -* User Defined Sharding is specified using `shardingType: USER` +* User-Defined Sharding is specified using `shardingType: USER` In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_scale_in_delete_an_existing_shard.md b/docs/sharding/provisioning/user-defined-sharding/udsharding_scale_in_delete_an_existing_shard.md index 2c4cbfc2..adb8af30 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_scale_in_delete_an_existing_shard.md +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_scale_in_delete_an_existing_shard.md @@ -1,16 +1,16 @@ -# Scale In - Delete an existing Shard from a working Oracle Sharded Database provisioned earlier with User Defined Sharding +# Scale In - Delete an existing Shard from a working Oracle Globally Distributed Database provisioned earlier with User-Defined Sharding **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. -This use case demonstrates how to delete an existing Shard from an existing Oracle Database sharding topology with User Defined Sharding provisioned using Oracle Database Sharding controller. +This use case demonstrates how to delete an existing Shard from an existing Oracle Globally Distributed Database topology with User-Defined Sharding provisioned using Oracle Database Sharding controller. In this use case, the existing database Sharding is having: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Five sharding Pods: `shard1`,`shard2`,`shard3`,`shard4` and `shard5` +* Five Shard Database Pods: `shard1`,`shard2`,`shard3`,`shard4` and `shard5` * One Catalog Pod: `catalog` * Namespace: `shns` -* User Defined Sharding is specified using `shardingType: USER` +* User-Defined Sharding is specified using `shardingType: USER` In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. @@ -20,7 +20,7 @@ In this example, we are using pre-built Oracle Database and Global Data Services **NOTE:** Use tag `isDelete: enable` to delete the shard you want. -This use case deletes the shard `shard4` from the above Sharding Topology. +This use case deletes the shard `shard4` from the above Oracle Globally Distributed Database Topology. Use the file: [udsharding_shard_prov_delshard.yaml](./udsharding_shard_prov_delshard.yaml) for this use case as below: diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_scale_out_add_shards.md b/docs/sharding/provisioning/user-defined-sharding/udsharding_scale_out_add_shards.md index 20f50b29..371b2438 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_scale_out_add_shards.md +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_scale_out_add_shards.md @@ -1,24 +1,24 @@ -# Scale Out - Add Shards to an existing Oracle Sharded Database provisioned earlier with User Defined Sharding +# Scale Out - Add Shards to an existing Oracle Globally Distributed Database provisioned earlier with User-Defined Sharding **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. -This use case demonstrates adding a new shard to an existing Oracle Database sharding topology with User Defined Sharding provisioned earlier using Oracle Database Sharding controller. +This use case demonstrates adding a new shard to an existing Oracle Globally Distributed Database topology with User-Defined Sharding provisioned earlier using Oracle Database Sharding controller. -In this use case, the existing Oracle Database sharding topology is having: +In this use case, the existing Oracle Globally Distributed Database topology is having: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three sharding Pods: `shard1`, `shard2` and `shard3` +* Three Shard Database Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` -* User Defined Sharding is specified using `shardingType: USER` +* User-Defined Sharding is specified using `shardingType: USER` In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `udsharding_shard_prov_extshard.yaml`. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) - * If the existing Sharding Topology was deployed using [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then the additional parameter `dbEdition: "free"` will be needed for the below .yaml file as well. + * If the existing Oracle Globally Distributed Database Topology was deployed using [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then the additional parameter `dbEdition: "free"` will be needed for the below .yaml file as well. -This use case adds two new shards `shard4`,`shard5` to above Sharding Topology. +This use case adds two new shards `shard4`,`shard5` to above Oracle Globally Distributed Database Topology. Use the file: [udsharding_shard_prov_extshard.yaml](./udsharding_shard_prov_extshard.yaml) for this use case as below: From 93099928ffcc8763998264727d864908088d74f5 Mon Sep 17 00:00:00 2001 From: jpverma85 Date: Wed, 11 Sep 2024 17:29:36 -0400 Subject: [PATCH 616/628] Known issues change (#147) * added known issues for sharding * added the known issue change --- docs/sharding/README.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/docs/sharding/README.md b/docs/sharding/README.md index a5c7c470..076d7e32 100644 --- a/docs/sharding/README.md +++ b/docs/sharding/README.md @@ -33,6 +33,7 @@ Following sections provide the details for deploying Oracle Globally Distributed * [Provisioning Oracle Globally Distributed Database System-Managed Sharding with Raft replication enabled in a Cloud-Based Kubernetes Cluster](#provisioning-oracle-globally-distributed-database-topology-with-system-managed-sharding-and-raft-replication-enabled-in-a-cloud-based-kubernetes-cluster) * [Connecting to Shard Databases](#connecting-to-shard-databases) * [Debugging and Troubleshooting](#debugging-and-troubleshooting) +* [Known Issues](#known-issues) **Note** Before proceeding to the next section, you must complete the instructions given in each section, based on your enviornment, before proceeding to next section. @@ -187,3 +188,10 @@ After the Oracle Globally Distributed Database Topology has been provisioned usi ## Debugging and Troubleshooting To debug the Oracle Globally Distributed Database Topology provisioned using the Sharding Controller of Oracle Database Kubernetes Operator, follow this document: [Debugging and troubleshooting](./provisioning/debugging.md) + +## Known Issues + +* For both ENTERPRISE and FREE Images, if the GSM POD is stopped using `crictl stopp` at the worker node level, it leaves GSM in failed state with the `gdsctl` commands failing with error **GSM-45034: Connection to GDS catalog is not established**. It is beacause with change, the network namespace is lost if we check from the GSM Pod. +* For both ENTERPRISE and FREE Images, reboot of node running CATALOG using `/sbin/reboot -f` results in **GSM-45076: GSM IS NOT RUNNING**. Once you hit this issue, after waiting for a certain time, the `gdsctl` commands start working as the DB connection start working. Once the stack comes up fine after the node reboot, after some time, unexpected restart of GSM Pod is also observed. +* For both ENTERPRISE and FREE Images, reboot of node running the SHARD Pod using `/sbin/reboot -f` or stopping the Shard Database Pod from worker node using `crictl stopp` command leaves the shard in error state. +* For both ENTERPRISE and FREE Images, GSM pod restarts multiple times after force rebooting the node running GSM Pod. Its because when the worker node comes up, the GSM pod was recreated but it does not get DB connection to Catalog and meanwhile, the Liveness Probe fails which restart the Pod. \ No newline at end of file From d6ac2321e8a15a354a5e4958c281cd3be14afdc3 Mon Sep 17 00:00:00 2001 From: jpverma85 Date: Fri, 11 Oct 2024 14:53:06 -0400 Subject: [PATCH 617/628] doc_change (#151) --- docs/sharding/README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/sharding/README.md b/docs/sharding/README.md index 076d7e32..134d338b 100644 --- a/docs/sharding/README.md +++ b/docs/sharding/README.md @@ -191,7 +191,7 @@ To debug the Oracle Globally Distributed Database Topology provisioned using the ## Known Issues -* For both ENTERPRISE and FREE Images, if the GSM POD is stopped using `crictl stopp` at the worker node level, it leaves GSM in failed state with the `gdsctl` commands failing with error **GSM-45034: Connection to GDS catalog is not established**. It is beacause with change, the network namespace is lost if we check from the GSM Pod. -* For both ENTERPRISE and FREE Images, reboot of node running CATALOG using `/sbin/reboot -f` results in **GSM-45076: GSM IS NOT RUNNING**. Once you hit this issue, after waiting for a certain time, the `gdsctl` commands start working as the DB connection start working. Once the stack comes up fine after the node reboot, after some time, unexpected restart of GSM Pod is also observed. -* For both ENTERPRISE and FREE Images, reboot of node running the SHARD Pod using `/sbin/reboot -f` or stopping the Shard Database Pod from worker node using `crictl stopp` command leaves the shard in error state. -* For both ENTERPRISE and FREE Images, GSM pod restarts multiple times after force rebooting the node running GSM Pod. Its because when the worker node comes up, the GSM pod was recreated but it does not get DB connection to Catalog and meanwhile, the Liveness Probe fails which restart the Pod. \ No newline at end of file +* For both ENTERPRISE and FREE Images, if the Oracle Global Service Manager (GSM) POD is stopped using `crictl stopp` at the worker node level, it leaves GSM in failed state. The `gdsctl` commands fail with error **GSM-45034: Connection to GDS catalog is not established**. This is because with change, the network namespace is lost when checked from the GSM Pod. +* For both ENTERPRISE and FREE Images, restart of the node running CATALOG using `/sbin/reboot -f` results in **GSM-45076: GSM IS NOT RUNNING**. After you encounter this issue, wait until the `gdsctl` commands start working as the database connection start working. When the stack comes up again after the node restart, you can encounter an unexpected restart of the GSM Pod. +* For both ENTERPRISE and FREE Images, either restart of node running the SHARD Pod using `/sbin/reboot -f` or stopping the Shard Database Pod from the worker node using `crictl stopp` command can leave the shard in an error state. +* For both ENTERPRISE and FREE Images, after force restarts of the node running GSM Pod, the GSM pod restarts multiple times, and then becomes stable. The GSM pod restarts itself because when the worker node comes up, the GSM pod is recreated, but does not obtain DB connection to the Catalog. The Liveness Probe fails which restarts the Pod. Be aware of this issue, and permit the GSM pod to become stable. \ No newline at end of file From 44f2afe186da2cd2295aa689f15d46a3f8b176fc Mon Sep 17 00:00:00 2001 From: jpverma85 Date: Thu, 17 Oct 2024 18:40:18 -0400 Subject: [PATCH 618/628] added known issues (#152) --- docs/sharding/README.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/sharding/README.md b/docs/sharding/README.md index 134d338b..239dd767 100644 --- a/docs/sharding/README.md +++ b/docs/sharding/README.md @@ -193,5 +193,7 @@ To debug the Oracle Globally Distributed Database Topology provisioned using the * For both ENTERPRISE and FREE Images, if the Oracle Global Service Manager (GSM) POD is stopped using `crictl stopp` at the worker node level, it leaves GSM in failed state. The `gdsctl` commands fail with error **GSM-45034: Connection to GDS catalog is not established**. This is because with change, the network namespace is lost when checked from the GSM Pod. * For both ENTERPRISE and FREE Images, restart of the node running CATALOG using `/sbin/reboot -f` results in **GSM-45076: GSM IS NOT RUNNING**. After you encounter this issue, wait until the `gdsctl` commands start working as the database connection start working. When the stack comes up again after the node restart, you can encounter an unexpected restart of the GSM Pod. +* For both ENTERPRISE and FREE Images, if the CATALOG Database Pod is stopped from the worker node using the command `crictl stopp`, then it can leave the CATALOG in an error state. This error state results in GSM reporting the error message **GSM-45034: Connection to GDS catalog is not established.** * For both ENTERPRISE and FREE Images, either restart of node running the SHARD Pod using `/sbin/reboot -f` or stopping the Shard Database Pod from the worker node using `crictl stopp` command can leave the shard in an error state. -* For both ENTERPRISE and FREE Images, after force restarts of the node running GSM Pod, the GSM pod restarts multiple times, and then becomes stable. The GSM pod restarts itself because when the worker node comes up, the GSM pod is recreated, but does not obtain DB connection to the Catalog. The Liveness Probe fails which restarts the Pod. Be aware of this issue, and permit the GSM pod to become stable. \ No newline at end of file +* For both ENTERPRISE and FREE Images, after force restarts of the node running GSM Pod, the GSM pod restarts multiple times, and then becomes stable. The GSM pod restarts itself because when the worker node comes up, the GSM pod is recreated, but does not obtain DB connection to the Catalog. The Liveness Probe fails which restarts the Pod. Be aware of this issue, and permit the GSM pod to become stable. +* **DDL Propagation from Catalog to Shards:** DDL Propagation from the Catalog Database to the Shard Databases can take several minutes to complete. To see faster propagation of DDLs such as the tablespace set from the Catalog Database to the Shard Databases, Oracle recommends that you set smaller chunk values by using the `CATALOG_CHUNKS` attribute in the .yaml file while creating the Sharded Database Topology. \ No newline at end of file From a834cfb518f3db3455628080dcb5fc1ef05969ee Mon Sep 17 00:00:00 2001 From: jpverma85 Date: Mon, 21 Oct 2024 16:33:52 -0400 Subject: [PATCH 619/628] Uds doc change (#153) * added known issues * uds doc change --- .../udsharding_scale_in_delete_an_existing_shard.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_scale_in_delete_an_existing_shard.md b/docs/sharding/provisioning/user-defined-sharding/udsharding_scale_in_delete_an_existing_shard.md index adb8af30..e01e606f 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_scale_in_delete_an_existing_shard.md +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_scale_in_delete_an_existing_shard.md @@ -26,7 +26,7 @@ Use the file: [udsharding_shard_prov_delshard.yaml](./udsharding_shard_prov_dels 1. Move out the chunks from the shard to be deleted to another shard. For example, in the current case, before deleting the `shard4`, if you want to move the chunks from `shard4` to `shard2`, then you can run the below `kubectl` command where `/u01/app/oracle/product/23ai/gsmhome_1` is the GSM HOME: ```sh - kubectl exec -it pod/gsm1-0 -n shns -- /u01/app/oracle/product/23ai/gsmhome_1/bin/gdsctl "move chunk -chunk all -source shard4_shard4pdb -target shard4_shard4pdb" + kubectl exec -it pod/gsm1-0 -n shns -- /u01/app/oracle/product/23ai/gsmhome_1/bin/gdsctl "move chunk -chunk all -source shard4_shard4pdb -target shard2_shard2pdb" ``` 2. Confirm the shard to be deleted (`shard4` in this case) is not having any chunk using below command: ```sh @@ -48,7 +48,7 @@ Use the file: [udsharding_shard_prov_delshard.yaml](./udsharding_shard_prov_dels - After you apply `udsharding_shard_prov_delshard.yaml`, the change may not be visible immediately and it may take some time for the delete operation to complete. - If the shard, that you are trying to delete, is still having chunks, then the you will see message like below in the logs of the Oracle Database Operator Pod. ```sh - INFO controllers.database.ShardingDatabase manual intervention required + DEBUG events Shard Deletion failed for [shard4]. Retry shard deletion after manually moving the chunks. Requeuing ``` In this case, you will need to first move out the chunks from the shard to be deleted using Step 2 above and then apply the file in Step 3 to delete that shard. From b2cd9b9a9f62318f13a24cc69027a2796de4b7ec Mon Sep 17 00:00:00 2001 From: Paramdeep Saini Date: Mon, 10 Mar 2025 15:52:13 +0000 Subject: [PATCH 620/628] Added Features of Oracle DB Operator Release 1.2.0 --- .gitignore | 2 +- Dockerfile | 18 +- Makefile | 113 +- PREREQUISITES.md | 8 +- PROJECT | 102 + README.md | 136 +- THIRD_PARTY_LICENSES.txt | 65 +- .../v1alpha1/adbfamily_common_spec.go | 67 + .../autonomouscontainerdatabase_types.go | 25 +- .../autonomouscontainerdatabase_webhook.go | 9 +- .../v1alpha1/autonomousdatabase_conversion.go | 371 + .../v1alpha1/autonomousdatabase_types.go | 333 +- .../v1alpha1/autonomousdatabase_webhook.go | 172 +- .../autonomousdatabase_webhook_test.go | 150 +- .../autonomousdatabasebackup_types.go | 15 +- .../autonomousdatabasebackup_webhook.go | 21 +- .../autonomousdatabasebackup_webhook_test.go | 18 +- .../autonomousdatabaserestore_types.go | 19 +- .../autonomousdatabaserestore_webhook.go | 16 +- .../autonomousdatabaserestore_webhook_test.go | 12 +- .../v1alpha1/dataguardbroker_conversion.go | 14 + .../v1alpha1/dataguardbroker_types.go | 73 +- .../v1alpha1/dataguardbroker_webhook.go | 10 + .../v1alpha1/dbcssystem_conversion.go | 14 + .../database/v1alpha1/dbcssystem_kms_types.go | 141 + .../v1alpha1/dbcssystem_pdbconfig_types.go | 83 + apis/database/v1alpha1/dbcssystem_types.go | 98 +- apis/database/v1alpha1/dbcssystem_webhook.go | 98 + .../oraclerestdataservice_conversion.go | 14 + .../v1alpha1/oraclerestdataservice_types.go | 35 +- .../v1alpha1/oraclerestdataservice_webhook.go | 3 - .../v1alpha1/shardingdatabase_conversion.go | 14 + .../v1alpha1/shardingdatabase_types.go | 25 +- .../v1alpha1/shardingdatabase_webhook.go | 48 +- .../singleinstancedatabase_conversion.go | 14 + .../v1alpha1/singleinstancedatabase_types.go | 16 +- .../singleinstancedatabase_webhook.go | 20 +- .../v1alpha1/zz_generated.deepcopy.go | 958 +- apis/database/v4/adbfamily_common_spec.go | 67 + .../adbfamily_utils.go} | 55 +- .../v4/autonomouscontainerdatabase_types.go | 226 + .../v4/autonomouscontainerdatabase_webhook.go | 110 + apis/database/v4/autonomousdatabase_types.go | 393 + .../database/v4/autonomousdatabase_webhook.go | 170 + .../v4/autonomousdatabasebackup_types.go | 129 + .../v4/autonomousdatabasebackup_webhook.go | 158 + .../v4/autonomousdatabaserestore_types.go | 142 + .../v4/autonomousdatabaserestore_webhook.go | 146 + apis/database/{v1alpha1 => v4}/cdb_types.go | 20 +- apis/database/{v1alpha1 => v4}/cdb_webhook.go | 11 +- .../database/v4/dataguardbroker_conversion.go | 4 + apis/database/v4/dataguardbroker_types.go | 163 + apis/database/v4/dataguardbroker_webhook.go | 55 + apis/database/v4/dbcssystem_conversion.go | 4 + apis/database/v4/dbcssystem_kms_types.go | 141 + .../database/v4/dbcssystem_pdbconfig_types.go | 83 + apis/database/v4/dbcssystem_types.go | 292 + apis/database/v4/dbcssystem_webhook.go | 98 + apis/database/v4/groupversion_info.go | 58 + apis/database/v4/lrest_types.go | 191 + apis/database/v4/lrest_webhook.go | 219 + apis/database/v4/lrpdb_types.go | 256 + apis/database/v4/lrpdb_webhook.go | 370 + .../v4/oraclerestdataservice_conversion.go | 4 + .../v4/oraclerestdataservice_types.go | 158 + .../v4/oraclerestdataservice_webhook.go | 55 + apis/database/v4/ordssrvs_types.go | 693 + apis/database/{v1alpha1 => v4}/pdb_types.go | 17 +- apis/database/{v1alpha1 => v4}/pdb_webhook.go | 40 +- .../v4/shardingdatabase_conversion.go | 4 + apis/database/v4/shardingdatabase_types.go | 427 + apis/database/v4/shardingdatabase_webhook.go | 314 + .../v4/singleinstancedatabase_conversion.go | 4 + .../v4/singleinstancedatabase_types.go | 231 + .../v4/singleinstancedatabase_webhook.go | 55 + apis/database/v4/zz_generated.deepcopy.go | 4213 +++++ .../v1/databaseobserver_types.go | 195 + .../v1/databaseobserver_webhook.go | 185 + apis/observability/v1/groupversion_info.go | 58 + .../observability/v1/zz_generated.deepcopy.go | 481 + .../v1alpha1/databaseobserver_types.go | 83 +- .../v1alpha1/databaseobserver_webhook.go | 8 +- .../v1alpha1/zz_generated.deepcopy.go | 201 +- .../v4/databaseobserver_types.go | 196 + .../v4/databaseobserver_webhook.go | 182 + apis/observability/v4/groupversion_info.go | 58 + .../observability/v4/zz_generated.deepcopy.go | 481 + commons/adb_family/utils.go | 18 +- commons/database/constants.go | 34 +- commons/database/podbuilder.go | 108 + commons/database/svcbuilder.go | 99 + commons/database/utils.go | 26 - commons/dbcssystem/dbcs_reconciler.go | 974 +- commons/dbcssystem/dcommon.go | 61 +- commons/k8s/create.go | 27 +- commons/k8s/fetch.go | 30 +- commons/multitenant/lrest/common.go | 113 + commons/observability/constants.go | 68 +- commons/observability/utils.go | 503 +- commons/oci/containerdatabase.go | 12 +- commons/oci/database.go | 331 +- commons/oci/provider.go | 8 +- commons/sharding/catalog.go | 58 +- commons/sharding/exec.go | 40 +- commons/sharding/gsm.go | 64 +- commons/sharding/provstatus.go | 227 +- commons/sharding/scommon.go | 160 +- commons/sharding/shard.go | 50 +- ...acle.com_autonomouscontainerdatabases.yaml | 100 +- ....oracle.com_autonomousdatabasebackups.yaml | 121 +- ...oracle.com_autonomousdatabaserestores.yaml | 121 +- ...tabase.oracle.com_autonomousdatabases.yaml | 597 +- .../crd/bases/database.oracle.com_cdbs.yaml | 92 +- .../database.oracle.com_dataguardbrokers.yaml | 140 +- .../database.oracle.com_dbcssystems.yaml | 562 +- .../crd/bases/database.oracle.com_lrests.yaml | 254 + .../crd/bases/database.oracle.com_lrpdbs.yaml | 369 + ...ase.oracle.com_oraclerestdataservices.yaml | 214 +- .../bases/database.oracle.com_ordssrvs.yaml | 488 + .../crd/bases/database.oracle.com_pdbs.yaml | 130 +- ...database.oracle.com_shardingdatabases.yaml | 701 +- ...se.oracle.com_singleinstancedatabases.yaml | 427 +- ...vability.oracle.com_databaseobservers.yaml | 6919 +++++++- config/crd/kustomization.yaml | 25 +- ...ction_in_autonomouscontainerdatabases.yaml | 2 +- ...njection_in_autonomousdatabasebackups.yaml | 2 +- ...jection_in_autonomousdatabaserestores.yaml | 2 +- ...njection_in_database_dataguardbrokers.yaml | 8 + .../cainjection_in_database_lrests.yaml | 8 + .../cainjection_in_database_lrpdbs.yaml | 8 + ...on_in_database_oraclerestdataservices.yaml | 8 + .../cainjection_in_database_ordssrvs.yaml | 8 + ...n_in_database_singleinstancedatabases.yaml | 8 + .../patches/cainjection_in_dbcssystems.yaml | 2 +- ...on_in_observability_databaseobservers.yaml | 8 + ...bhook_in_autonomouscontainerdatabases.yaml | 20 +- .../webhook_in_autonomousdatabasebackups.yaml | 20 +- ...webhook_in_autonomousdatabaserestores.yaml | 20 +- .../webhook_in_autonomousdatabases.yaml | 18 +- config/crd/patches/webhook_in_lrests.yaml | 17 + config/crd/patches/webhook_in_lrpdbs.yaml | 17 + config/crd/patches/webhook_in_ordssrvs.yaml | 17 + config/database.oracle.com_DbcsSystem.yaml | 209 +- ...database.oracle.com_shardingdatabases.yaml | 4 +- config/manager/kustomization.yaml | 2 +- ...tabase-operator.clusterserviceversion.yaml | 4 +- ...vability.oracle.com_databaseobservers.yaml | 9350 ++++++++++- config/rbac/lrest_editor_role.yaml | 24 + config/rbac/lrest_viewer_role.yaml | 20 + config/rbac/lrpdb_editor_role.yaml | 24 + config/rbac/lrpdb_viewer_role.yaml | 20 + config/rbac/ordssrvs_editor_role.yaml | 24 + config/rbac/ordssrvs_viewer_role.yaml | 20 + config/rbac/role.yaml | 398 +- .../samples/adb/autonomousdatabase_bind.yaml | 3 +- .../samples/adb/autonomousdatabase_clone.yaml | 35 + .../adb/autonomousdatabase_create.yaml | 3 +- .../autonomousdatabase_delete_resource.yaml | 2 +- .../adb/autonomousdatabase_rename.yaml | 3 +- .../samples/adb/autonomousdatabase_scale.yaml | 3 +- ...tonomousdatabase_stop_start_terminate.yaml | 6 +- ...onomousdatabase_update_admin_password.yaml | 5 +- .../adb/autonomousdatabase_update_mtls.yaml | 9 +- ...onomousdatabase_update_network_access.yaml | 57 +- .../adb/autonomousdatabase_wallet.yaml | 25 +- config/samples/kustomization.yaml | 18 + .../observability/v1/databaseobserver.yaml | 81 + ...databaseobserver_customization_fields.yaml | 54 + .../v1/databaseobserver_logs_promtail.yaml | 74 + .../v1alpha1/databaseobserver.yaml | 80 + .../databaseobserver_custom_config.yaml | 46 + .../databaseobserver_logs_promtail.yaml | 74 + .../v1alpha1/databaseobserver_minimal.yaml | 26 + .../v1alpha1/databaseobserver_vault.yaml | 30 + .../observability/v4/databaseobserver.yaml | 79 + .../v4/databaseobserver_custom_config.yaml | 46 + .../v4/databaseobserver_logs_promtail.yaml | 76 + .../v4/databaseobserver_minimal.yaml | 26 + .../v4/databaseobserver_vault.yaml | 39 + config/samples/sidb/dataguardbroker.yaml | 8 +- .../samples/sidb/oraclerestdataservice.yaml | 21 +- .../sidb/oraclerestdataservice_apex.yaml | 42 - .../sidb/oraclerestdataservice_create.yaml | 9 +- .../sidb/oraclerestdataservice_secrets.yaml | 18 - .../samples/sidb/singleinstancedatabase.yaml | 18 +- .../sidb/singleinstancedatabase_clone.yaml | 2 +- .../sidb/singleinstancedatabase_create.yaml | 4 +- .../sidb/singleinstancedatabase_express.yaml | 2 +- .../singleinstancedatabase_free-lite.yaml | 35 + ...singleinstancedatabase_free-truecache.yaml | 48 + .../sidb/singleinstancedatabase_free.yaml | 7 +- .../sidb/singleinstancedatabase_patch.yaml | 2 +- .../singleinstancedatabase_prebuiltdb.yaml | 2 +- .../sidb/singleinstancedatabase_standby.yaml | 4 +- .../sidb/singleinstancedatabase_tcps.yaml | 2 +- config/webhook/manifests.yaml | 412 +- .../autonomouscontainerdatabase_controller.go | 92 +- .../database/autonomousdatabase_controller.go | 1255 +- .../autonomousdatabasebackup_controller.go | 50 +- .../autonomousdatabaserestore_controller.go | 52 +- controllers/database/cdb_controller.go | 174 +- .../database/dataguardbroker_controller.go | 1199 -- controllers/database/dbcssystem_controller.go | 1345 +- controllers/database/lrest_controller.go | 1105 ++ controllers/database/lrpdb_controller.go | 2381 +++ .../oraclerestdataservice_controller.go | 593 +- controllers/database/ordssrvs_controller.go | 1116 ++ controllers/database/ordssrvs_ordsconfig.go | 258 + controllers/database/pdb_controller.go | 340 +- .../database/shardingdatabase_controller.go | 480 +- .../singleinstancedatabase_controller.go | 381 +- controllers/dataguard/datagauard_errors.go | 47 + controllers/dataguard/dataguard_utils.go | 1061 ++ .../dataguard/dataguardbroker_controller.go | 513 + .../databaseobserver_controller.go | 293 +- .../databaseobserver_resource.go | 132 +- docs/adb/ADB_LONG_TERM_BACKUP.md | 4 +- docs/adb/ADB_PREREQUISITES.md | 4 +- docs/adb/NETWORK_ACCESS_OPTIONS.md | 222 +- docs/adb/README.md | 173 +- docs/dbcs/README.md | 144 +- .../bind_to_existing_dbcs_system.md | 22 +- .../bind_to_existing_dbcs_system.yaml | 4 +- ..._to_existing_dbcs_system_sample_output.log | 141 +- docs/dbcs/provisioning/clone_dbcs_system.yaml | 20 + .../clone_dbcs_system_from_backup.yaml | 22 + ..._dbcs_system_from_backup_sample_output.log | 75 + .../clone_dbcs_system_from_database.yaml | 22 + ...bcs_system_from_database_sample_output.log | 39 + .../clone_dbcs_system_sample_output.log | 60 + .../provisioning/clone_from_backup_dbcs.md | 36 + docs/dbcs/provisioning/clone_from_database.md | 35 + .../provisioning/clone_from_existing_dbcs.md | 36 + .../dbcs/provisioning/create_dbcs_with_kms.md | 73 + .../dbcs/provisioning/create_dbcs_with_pdb.md | 55 + docs/dbcs/provisioning/create_kms.md | 50 + docs/dbcs/provisioning/create_pdb.md | 55 + .../create_pdb_to_existing_dbcs_system.md | 55 + ..._in_existing_dbcs_system_sample_output.log | 1 + ...reatepdb_in_existing_dbcs_system_list.yaml | 27 + ...xisting_dbcs_system_list_sample_output.log | 185 + .../dbcs_controller_parameters.md | 2 +- .../dbcs_service_migrate_to_kms.log | 132 + .../dbcs_service_migrate_to_kms.yaml | 16 + .../dbcs_service_with_2_node_rac.md | 42 +- .../dbcs_service_with_2_node_rac.yaml | 21 +- .../dbcs_service_with_all_parameters_asm.md | 41 +- .../dbcs_service_with_all_parameters_asm.yaml | 29 +- ..._with_all_parameters_asm_sample_output.log | 225 +- .../dbcs_service_with_all_parameters_lvm.md | 42 +- .../dbcs_service_with_all_parameters_lvm.yaml | 27 +- .../provisioning/dbcs_service_with_kms.yaml | 27 + .../dbcs_service_with_kms_sample_output.log | 91 + .../dbcs_service_with_minimal_parameters.md | 22 +- .../dbcs_service_with_minimal_parameters.yaml | 22 +- ..._with_minimal_parameters_sample_output.log | 175 +- .../provisioning/dbcs_service_with_pdb.yaml | 38 + .../dbcs_service_with_pdb_sample_output.log | 137 + docs/dbcs/provisioning/delete_pdb.md | 50 + ...eletepdb_in_existing_dbcs_system_list.yaml | 13 + ...xisting_dbcs_system_list_sample_output.log | 8 + docs/dbcs/provisioning/migrate_to_kms.md | 49 + .../scale_down_dbcs_system_shape.md | 30 +- .../scale_down_dbcs_system_shape.yaml | 17 +- ...e_down_dbcs_system_shape_sample_output.log | 371 +- .../scale_up_dbcs_system_shape.md | 32 +- .../scale_up_dbcs_system_shape.yaml | 17 +- docs/dbcs/provisioning/scale_up_storage.md | 30 +- docs/dbcs/provisioning/scale_up_storage.yaml | 19 +- .../scale_up_storage_sample_output.log | 440 +- .../provisioning/terminate_dbcs_system.md | 24 +- .../provisioning/terminate_dbcs_system.yaml | 2 +- .../terminate_dbcs_system_sample_output.log | 8 +- docs/dbcs/provisioning/update_license.md | 32 +- docs/dbcs/provisioning/update_license.yaml | 38 +- docs/multitenant/README.md | 259 +- docs/multitenant/lrest-based/README.md | 500 + .../lrest-based/images/Generalschema2.jpg | Bin 0 -> 96239 bytes .../lrest-based/images/UsecaseSchema.jpg | Bin 0 -> 185861 bytes .../multitenant/lrest-based/usecase/README.md | 139 + .../usecase/altersystem_pdb1_resource.yaml | 50 + .../usecase/cdbnamespace_binding.yaml | 13 + .../usecase/clone_pdb1_resource.yaml | 51 + .../usecase/clone_pdb2_resource.yaml | 51 + .../usecase/close_pdb1_resource.yaml | 47 + .../usecase/close_pdb2_resource.yaml | 47 + .../usecase/close_pdb3_resource.yaml | 47 + .../lrest-based/usecase/config-map-pdb.yaml | 11 + .../lrest-based/usecase/config_map_pdb.yaml | 11 + .../lrest-based/usecase/create_lrest_pod.yaml | 44 + .../usecase/create_pdb1_resource.yaml | 52 + .../usecase/create_pdb2_resource.yaml | 52 + .../usecase/delete_pdb1_resource.yaml | 45 + .../usecase/delete_pdb2_resource.yaml | 45 + docs/multitenant/lrest-based/usecase/makefile | 911 + .../usecase/map_pdb1_resource.yaml | 49 + .../usecase/map_pdb2_resource.yaml | 49 + .../usecase/map_pdb3_resource.yaml | 49 + .../usecase/open_pdb1_resource.yaml | 47 + .../usecase/open_pdb2_resource.yaml | 47 + .../usecase/open_pdb3_resource.yaml | 47 + .../lrest-based/usecase/parameters.txt | 52 + .../usecase/pdbnamespace_binding.yaml | 13 + .../usecase/plug_pdb1_resource.yaml | 54 + .../usecase/unplug_pdb1_resource.yaml | 46 + docs/multitenant/ords-based/NamespaceSeg.md | 14 + docs/multitenant/ords-based/README.md | 411 + .../images/K8S_NAMESPACE_SEG.png | Bin .../{ => ords-based}/images/K8S_SECURE1.png | Bin .../{ => ords-based}/images/K8S_SECURE2.png | Bin .../{ => ords-based}/images/K8S_SECURE3.png | Bin .../{ => ords-based}/images/K8S_SECURE4.png | Bin .../ords-based/images/makerunall.png | Bin 0 -> 211874 bytes .../ords-based/images/makesecrets_1_1.png | Bin 0 -> 117953 bytes .../{ => ords-based}/openssl_schema.jpg | Bin .../example_setup_using_oci_oke_cluster.md | 0 .../multinamespace/cdb_create.yaml | 30 +- .../multinamespace/pdb_clone.yaml | 22 +- .../multinamespace/pdb_close.yaml | 22 +- .../multinamespace/pdb_create.yaml | 22 +- .../multinamespace/pdb_delete.yaml | 15 +- .../provisioning/multinamespace/pdb_open.yaml | 22 +- .../provisioning/multinamespace/pdb_plug.yaml | 15 +- .../multinamespace/pdb_unplug.yaml | 14 +- .../ords-based/provisioning/ords_image.md | 81 + .../provisioning/quickOKEcreation.md | 0 .../singlenamespace}/cdb_create.yaml | 39 +- .../singlenamespace/cdb_secret.yaml | 0 .../singlenamespace/pdb_clone.yaml | 18 +- .../singlenamespace}/pdb_close.yaml | 22 +- .../singlenamespace}/pdb_create.yaml | 22 +- .../singlenamespace}/pdb_delete.yaml | 15 +- .../singlenamespace}/pdb_open.yaml | 22 +- .../singlenamespace}/pdb_plug.yaml | 13 +- .../singlenamespace/pdb_secret.yaml | 0 .../singlenamespace}/pdb_unplug.yaml | 12 +- docs/multitenant/ords-based/usecase/README.md | 112 + .../usecase/cdbnamespace_binding.yaml | 13 + .../usecase/clone_pdb1_resource.yaml | 50 + .../usecase/clone_pdb2_resource.yaml | 50 + .../usecase/close_pdb1_resource.yaml | 47 + .../usecase/close_pdb2_resource.yaml | 47 + .../usecase/close_pdb3_resource.yaml | 47 + .../ords-based/usecase/create_ords_pod.yaml | 48 + .../usecase/create_pdb1_resource.yaml | 51 + .../usecase/create_pdb2_resource.yaml | 51 + .../usecase/delete_pdb1_resource.yaml | 45 + .../usecase/delete_pdb2_resource.yaml | 45 + docs/multitenant/ords-based/usecase/makefile | 915 + .../ords-based/usecase/map_pdb1_resource.yaml | 49 + .../ords-based/usecase/map_pdb2_resource.yaml | 49 + .../ords-based/usecase/map_pdb3_resource.yaml | 49 + .../usecase/open_pdb1_resource.yaml | 47 + .../usecase/open_pdb2_resource.yaml | 47 + .../usecase/open_pdb3_resource.yaml | 47 + .../ords-based/usecase/parameters.txt | 61 + .../usecase/pdbnamespace_binding.yaml | 13 + .../usecase/plug_pdb1_resource.yaml | 53 + .../usecase/unplug_pdb1_resource.yaml | 46 + .../{ => ords-based}/usecase01/README.md | 199 +- .../{ => ords-based}/usecase01/ca.crt | 0 .../{ => ords-based}/usecase01/ca.key | 0 .../{ => ords-based}/usecase01/ca.srl | 0 .../usecase01}/cdb_create.yaml | 0 .../usecase01/cdb_secret.yaml | 0 .../usecase01/clone_pdb1_resource.yaml} | 32 +- .../usecase01/clone_pdb2_resource.yaml | 50 + .../usecase01/close_pdb1_resource.yaml | 47 + .../usecase01/close_pdb2_resource.yaml | 47 + .../usecase01/close_pdb3_resource.yaml | 47 + .../ords-based/usecase01/create_ords_pod.yaml | 48 + .../usecase01/create_pdb1_resource.yaml | 51 + .../usecase01/create_pdb2_resource.yaml | 51 + .../usecase01/delete_pdb1_resource.yaml | 45 + .../usecase01/delete_pdb2_resource.yaml | 45 + .../{ => ords-based}/usecase01/extfile.txt | 0 .../usecase01/logfiles/BuildImage.log | 896 + .../usecase01/logfiles/ImagePush.log | 0 .../usecase01/logfiles/cdb.log | 0 .../usecase01/logfiles/cdb_creation.log | 0 .../usecase01/logfiles/openssl_execution.log | 19 + .../usecase01/logfiles/ordsconfig.log | 39 + .../usecase01/logfiles/tagandpush.log | 0 .../usecase01/logfiles/testapi.log | 0 .../multitenant/ords-based/usecase01/makefile | 906 + .../usecase01/map_pdb1_resource.yaml | 49 + .../usecase01/map_pdb2_resource.yaml | 49 + .../usecase01/map_pdb3_resource.yaml | 49 + .../usecase01/open_pdb1_resource.yaml | 47 + .../usecase01/open_pdb2_resource.yaml | 47 + .../usecase01/open_pdb3_resource.yaml | 47 + ...acle-database-operator-system_binding.yaml | 13 + .../usecase01/oracle-database-operator.yaml | 0 .../ords-based/usecase01/parameters.txt | 61 + .../usecase01}/pdb_close.yaml | 0 .../usecase01}/pdb_create.yaml | 0 .../usecase01}/pdb_delete.yaml | 0 .../{ => ords-based}/usecase01/pdb_map.yaml | 0 .../usecase01}/pdb_open.yaml | 0 .../usecase01/pdb_secret.yaml | 0 .../usecase01/plug_pdb1_resource.yaml} | 31 +- .../{ => ords-based}/usecase01/server.csr | 0 .../usecase01/tde_secret.yaml | 0 .../{ => ords-based}/usecase01/tls.crt | 0 .../{ => ords-based}/usecase01/tls.key | 0 .../usecase01/unplug_pdb1_resource.yaml} | 27 +- .../{ => ords-based}/usecase02/README.md | 144 +- .../ords-based/usecase02/pdb_clone.yaml | 50 + .../ords-based/usecase02/pdb_plug.yaml | 53 + .../usecase02/pdb_plugtde.yaml | 2 +- .../ords-based/usecase02/pdb_unplug.yaml | 46 + .../usecase02/pdb_unplugtde.yaml | 2 +- docs/multitenant/usecase02/tde_secret.yaml | 15 - docs/observability/README.md | 550 +- docs/ordsservices/README.md | 67 + docs/ordsservices/TROUBLESHOOTING.md | 129 + docs/ordsservices/api.md | 1388 ++ docs/ordsservices/autoupgrade.md | 57 + docs/ordsservices/examples/adb.md | 108 + docs/ordsservices/examples/adb_oraoper.md | 176 + docs/ordsservices/examples/mongo_api.md | 160 + docs/ordsservices/examples/multi_pool.md | 203 + docs/ordsservices/examples/sidb_container.md | 154 + .../usecase01/create_mong_schema.sql | 9 + docs/ordsservices/usecase01/help | 1 + docs/ordsservices/usecase01/makefile | 778 + .../usecase01/tnsadmin/tnsnames.ora | 3 + .../usecase01/tnsadmin/tnsnames.ora.offline | 1 + docs/sharding/README.md | 78 +- .../create_kubernetes_secret_for_db_user.md | 12 +- .../provisioning/database_connection.md | 6 +- docs/sharding/provisioning/debugging.md | 28 +- .../sharding_provisioning_with_db_events.md | 8 +- .../sharding_provisioning_with_db_events.yaml | 3 +- .../sharding_provisioning_with_free_images.md | 8 +- ...harding_provisioning_with_free_images.yaml | 7 +- ...y_cloning_db_from_gold_image_across_ads.md | 14 +- ...ing_by_cloning_db_gold_image_in_same_ad.md | 14 +- ...ding_provisioning_with_chunks_specified.md | 10 +- ..._provisioning_with_control_on_resources.md | 6 +- ...ith_notification_using_oci_notification.md | 12 +- ...ding_provisioning_without_db_gold_image.md | 8 +- ...rding_scale_in_delete_an_existing_shard.md | 6 +- .../snr_ssharding_scale_out_add_shards.md | 10 +- .../snr_ssharding_shard_prov.yaml | 3 +- .../snr_ssharding_shard_prov_chunks.yaml | 3 +- .../snr_ssharding_shard_prov_clone.yaml | 3 +- ...ssharding_shard_prov_clone_across_ads.yaml | 3 +- .../snr_ssharding_shard_prov_delshard.yaml | 3 +- .../snr_ssharding_shard_prov_extshard.yaml | 3 +- .../snr_ssharding_shard_prov_memory_cpu.yaml | 3 +- ...sharding_shard_prov_send_notification.yaml | 3 +- ...y_cloning_db_from_gold_image_across_ads.md | 8 +- ...ing_by_cloning_db_gold_image_in_same_ad.md | 6 +- ...ding_provisioning_with_chunks_specified.md | 6 +- ..._provisioning_with_control_on_resources.md | 4 +- ...ith_notification_using_oci_notification.md | 8 +- ...ding_provisioning_without_db_gold_image.md | 4 +- ...rding_scale_in_delete_an_existing_shard.md | 8 +- .../ssharding_scale_out_add_shards.md | 10 +- .../system_sharding/ssharding_shard_prov.yaml | 3 +- .../ssharding_shard_prov_chunks.yaml | 59 + .../ssharding_shard_prov_clone.yaml | 3 +- ...ssharding_shard_prov_clone_across_ads.yaml | 3 +- .../ssharding_shard_prov_delshard.yaml | 3 +- .../ssharding_shard_prov_extshard.yaml | 3 +- .../ssharding_shard_prov_memory_cpu.yaml | 3 +- ...sharding_shard_prov_send_notification.yaml | 3 +- ...y_cloning_db_from_gold_image_across_ads.md | 14 +- ...ing_by_cloning_db_gold_image_in_same_ad.md | 12 +- ..._provisioning_with_control_on_resources.md | 8 +- ...ith_notification_using_oci_notification.md | 12 +- ...ding_provisioning_without_db_gold_image.md | 10 +- ...rding_scale_in_delete_an_existing_shard.md | 14 +- .../udsharding_scale_out_add_shards.md | 14 +- .../udsharding_shard_prov.yaml | 3 +- .../udsharding_shard_prov_clone.yaml | 3 +- ...dsharding_shard_prov_clone_across_ads.yaml | 3 +- .../udsharding_shard_prov_delshard.yaml | 3 +- .../udsharding_shard_prov_extshard.yaml | 3 +- .../udsharding_shard_prov_memory_cpu.yaml | 3 +- ...sharding_shard_prov_send_notification.yaml | 3 +- docs/sidb/PREREQUISITES.md | 13 +- docs/sidb/README.md | 377 +- go.mod | 98 +- go.sum | 245 +- main.go | 123 +- oracle-database-operator.yaml | 13962 +++++++++++++--- ords/Dockerfile | 14 +- ords/ords_init.sh | 484 + ords/runOrdsSSL.sh | 19 +- test/e2e/autonomouscontainerdatabase_test.go | 8 +- ...autonomousdatabase_controller_bind_test.go | 44 +- ...tonomousdatabase_controller_create_test.go | 109 +- test/e2e/behavior/shared_behaviors.go | 133 +- test/e2e/suite_test.go | 6 +- 496 files changed, 74952 insertions(+), 11904 deletions(-) create mode 100644 apis/database/v1alpha1/adbfamily_common_spec.go create mode 100644 apis/database/v1alpha1/autonomousdatabase_conversion.go create mode 100644 apis/database/v1alpha1/dataguardbroker_conversion.go create mode 100644 apis/database/v1alpha1/dbcssystem_conversion.go create mode 100644 apis/database/v1alpha1/dbcssystem_kms_types.go create mode 100644 apis/database/v1alpha1/dbcssystem_pdbconfig_types.go create mode 100644 apis/database/v1alpha1/dbcssystem_webhook.go create mode 100644 apis/database/v1alpha1/oraclerestdataservice_conversion.go create mode 100644 apis/database/v1alpha1/shardingdatabase_conversion.go create mode 100644 apis/database/v1alpha1/singleinstancedatabase_conversion.go create mode 100644 apis/database/v4/adbfamily_common_spec.go rename apis/database/{v1alpha1/adbfamily_common_utils.go => v4/adbfamily_utils.go} (87%) create mode 100644 apis/database/v4/autonomouscontainerdatabase_types.go create mode 100644 apis/database/v4/autonomouscontainerdatabase_webhook.go create mode 100644 apis/database/v4/autonomousdatabase_types.go create mode 100644 apis/database/v4/autonomousdatabase_webhook.go create mode 100644 apis/database/v4/autonomousdatabasebackup_types.go create mode 100644 apis/database/v4/autonomousdatabasebackup_webhook.go create mode 100644 apis/database/v4/autonomousdatabaserestore_types.go create mode 100644 apis/database/v4/autonomousdatabaserestore_webhook.go rename apis/database/{v1alpha1 => v4}/cdb_types.go (93%) rename apis/database/{v1alpha1 => v4}/cdb_webhook.go (92%) create mode 100644 apis/database/v4/dataguardbroker_conversion.go create mode 100644 apis/database/v4/dataguardbroker_types.go create mode 100644 apis/database/v4/dataguardbroker_webhook.go create mode 100644 apis/database/v4/dbcssystem_conversion.go create mode 100644 apis/database/v4/dbcssystem_kms_types.go create mode 100644 apis/database/v4/dbcssystem_pdbconfig_types.go create mode 100644 apis/database/v4/dbcssystem_types.go create mode 100644 apis/database/v4/dbcssystem_webhook.go create mode 100644 apis/database/v4/groupversion_info.go create mode 100644 apis/database/v4/lrest_types.go create mode 100644 apis/database/v4/lrest_webhook.go create mode 100644 apis/database/v4/lrpdb_types.go create mode 100644 apis/database/v4/lrpdb_webhook.go create mode 100644 apis/database/v4/oraclerestdataservice_conversion.go create mode 100644 apis/database/v4/oraclerestdataservice_types.go create mode 100644 apis/database/v4/oraclerestdataservice_webhook.go create mode 100644 apis/database/v4/ordssrvs_types.go rename apis/database/{v1alpha1 => v4}/pdb_types.go (96%) rename apis/database/{v1alpha1 => v4}/pdb_webhook.go (85%) create mode 100644 apis/database/v4/shardingdatabase_conversion.go create mode 100644 apis/database/v4/shardingdatabase_types.go create mode 100644 apis/database/v4/shardingdatabase_webhook.go create mode 100644 apis/database/v4/singleinstancedatabase_conversion.go create mode 100644 apis/database/v4/singleinstancedatabase_types.go create mode 100644 apis/database/v4/singleinstancedatabase_webhook.go create mode 100644 apis/database/v4/zz_generated.deepcopy.go create mode 100644 apis/observability/v1/databaseobserver_types.go create mode 100644 apis/observability/v1/databaseobserver_webhook.go create mode 100644 apis/observability/v1/groupversion_info.go create mode 100644 apis/observability/v1/zz_generated.deepcopy.go create mode 100644 apis/observability/v4/databaseobserver_types.go create mode 100644 apis/observability/v4/databaseobserver_webhook.go create mode 100644 apis/observability/v4/groupversion_info.go create mode 100644 apis/observability/v4/zz_generated.deepcopy.go create mode 100644 commons/database/podbuilder.go create mode 100644 commons/database/svcbuilder.go create mode 100644 commons/multitenant/lrest/common.go create mode 100644 config/crd/bases/database.oracle.com_lrests.yaml create mode 100644 config/crd/bases/database.oracle.com_lrpdbs.yaml create mode 100644 config/crd/bases/database.oracle.com_ordssrvs.yaml create mode 100644 config/crd/patches/cainjection_in_database_dataguardbrokers.yaml create mode 100644 config/crd/patches/cainjection_in_database_lrests.yaml create mode 100644 config/crd/patches/cainjection_in_database_lrpdbs.yaml create mode 100644 config/crd/patches/cainjection_in_database_oraclerestdataservices.yaml create mode 100644 config/crd/patches/cainjection_in_database_ordssrvs.yaml create mode 100644 config/crd/patches/cainjection_in_database_singleinstancedatabases.yaml create mode 100644 config/crd/patches/cainjection_in_observability_databaseobservers.yaml create mode 100644 config/crd/patches/webhook_in_lrests.yaml create mode 100644 config/crd/patches/webhook_in_lrpdbs.yaml create mode 100644 config/crd/patches/webhook_in_ordssrvs.yaml create mode 100644 config/rbac/lrest_editor_role.yaml create mode 100644 config/rbac/lrest_viewer_role.yaml create mode 100644 config/rbac/lrpdb_editor_role.yaml create mode 100644 config/rbac/lrpdb_viewer_role.yaml create mode 100644 config/rbac/ordssrvs_editor_role.yaml create mode 100644 config/rbac/ordssrvs_viewer_role.yaml create mode 100644 config/samples/adb/autonomousdatabase_clone.yaml create mode 100644 config/samples/observability/v1/databaseobserver.yaml create mode 100644 config/samples/observability/v1/databaseobserver_customization_fields.yaml create mode 100644 config/samples/observability/v1/databaseobserver_logs_promtail.yaml create mode 100644 config/samples/observability/v1alpha1/databaseobserver.yaml create mode 100644 config/samples/observability/v1alpha1/databaseobserver_custom_config.yaml create mode 100644 config/samples/observability/v1alpha1/databaseobserver_logs_promtail.yaml create mode 100644 config/samples/observability/v1alpha1/databaseobserver_minimal.yaml create mode 100644 config/samples/observability/v1alpha1/databaseobserver_vault.yaml create mode 100644 config/samples/observability/v4/databaseobserver.yaml create mode 100644 config/samples/observability/v4/databaseobserver_custom_config.yaml create mode 100644 config/samples/observability/v4/databaseobserver_logs_promtail.yaml create mode 100644 config/samples/observability/v4/databaseobserver_minimal.yaml create mode 100644 config/samples/observability/v4/databaseobserver_vault.yaml delete mode 100644 config/samples/sidb/oraclerestdataservice_apex.yaml create mode 100644 config/samples/sidb/singleinstancedatabase_free-lite.yaml create mode 100644 config/samples/sidb/singleinstancedatabase_free-truecache.yaml delete mode 100644 controllers/database/dataguardbroker_controller.go create mode 100644 controllers/database/lrest_controller.go create mode 100644 controllers/database/lrpdb_controller.go create mode 100644 controllers/database/ordssrvs_controller.go create mode 100644 controllers/database/ordssrvs_ordsconfig.go create mode 100644 controllers/dataguard/datagauard_errors.go create mode 100644 controllers/dataguard/dataguard_utils.go create mode 100644 controllers/dataguard/dataguardbroker_controller.go create mode 100644 docs/dbcs/provisioning/clone_dbcs_system.yaml create mode 100644 docs/dbcs/provisioning/clone_dbcs_system_from_backup.yaml create mode 100644 docs/dbcs/provisioning/clone_dbcs_system_from_backup_sample_output.log create mode 100644 docs/dbcs/provisioning/clone_dbcs_system_from_database.yaml create mode 100644 docs/dbcs/provisioning/clone_dbcs_system_from_database_sample_output.log create mode 100644 docs/dbcs/provisioning/clone_dbcs_system_sample_output.log create mode 100644 docs/dbcs/provisioning/clone_from_backup_dbcs.md create mode 100644 docs/dbcs/provisioning/clone_from_database.md create mode 100644 docs/dbcs/provisioning/clone_from_existing_dbcs.md create mode 100644 docs/dbcs/provisioning/create_dbcs_with_kms.md create mode 100644 docs/dbcs/provisioning/create_dbcs_with_pdb.md create mode 100644 docs/dbcs/provisioning/create_kms.md create mode 100644 docs/dbcs/provisioning/create_pdb.md create mode 100644 docs/dbcs/provisioning/create_pdb_to_existing_dbcs_system.md create mode 100644 docs/dbcs/provisioning/createkms_in_existing_dbcs_system_sample_output.log create mode 100644 docs/dbcs/provisioning/createpdb_in_existing_dbcs_system_list.yaml create mode 100644 docs/dbcs/provisioning/createpdb_in_existing_dbcs_system_list_sample_output.log create mode 100644 docs/dbcs/provisioning/dbcs_service_migrate_to_kms.log create mode 100644 docs/dbcs/provisioning/dbcs_service_migrate_to_kms.yaml create mode 100644 docs/dbcs/provisioning/dbcs_service_with_kms.yaml create mode 100644 docs/dbcs/provisioning/dbcs_service_with_kms_sample_output.log create mode 100644 docs/dbcs/provisioning/dbcs_service_with_pdb.yaml create mode 100644 docs/dbcs/provisioning/dbcs_service_with_pdb_sample_output.log create mode 100644 docs/dbcs/provisioning/delete_pdb.md create mode 100644 docs/dbcs/provisioning/deletepdb_in_existing_dbcs_system_list.yaml create mode 100644 docs/dbcs/provisioning/deletepdb_in_existing_dbcs_system_list_sample_output.log create mode 100644 docs/dbcs/provisioning/migrate_to_kms.md create mode 100644 docs/multitenant/lrest-based/README.md create mode 100644 docs/multitenant/lrest-based/images/Generalschema2.jpg create mode 100644 docs/multitenant/lrest-based/images/UsecaseSchema.jpg create mode 100644 docs/multitenant/lrest-based/usecase/README.md create mode 100644 docs/multitenant/lrest-based/usecase/altersystem_pdb1_resource.yaml create mode 100644 docs/multitenant/lrest-based/usecase/cdbnamespace_binding.yaml create mode 100644 docs/multitenant/lrest-based/usecase/clone_pdb1_resource.yaml create mode 100644 docs/multitenant/lrest-based/usecase/clone_pdb2_resource.yaml create mode 100644 docs/multitenant/lrest-based/usecase/close_pdb1_resource.yaml create mode 100644 docs/multitenant/lrest-based/usecase/close_pdb2_resource.yaml create mode 100644 docs/multitenant/lrest-based/usecase/close_pdb3_resource.yaml create mode 100644 docs/multitenant/lrest-based/usecase/config-map-pdb.yaml create mode 100644 docs/multitenant/lrest-based/usecase/config_map_pdb.yaml create mode 100644 docs/multitenant/lrest-based/usecase/create_lrest_pod.yaml create mode 100644 docs/multitenant/lrest-based/usecase/create_pdb1_resource.yaml create mode 100644 docs/multitenant/lrest-based/usecase/create_pdb2_resource.yaml create mode 100644 docs/multitenant/lrest-based/usecase/delete_pdb1_resource.yaml create mode 100644 docs/multitenant/lrest-based/usecase/delete_pdb2_resource.yaml create mode 100644 docs/multitenant/lrest-based/usecase/makefile create mode 100644 docs/multitenant/lrest-based/usecase/map_pdb1_resource.yaml create mode 100644 docs/multitenant/lrest-based/usecase/map_pdb2_resource.yaml create mode 100644 docs/multitenant/lrest-based/usecase/map_pdb3_resource.yaml create mode 100644 docs/multitenant/lrest-based/usecase/open_pdb1_resource.yaml create mode 100644 docs/multitenant/lrest-based/usecase/open_pdb2_resource.yaml create mode 100644 docs/multitenant/lrest-based/usecase/open_pdb3_resource.yaml create mode 100644 docs/multitenant/lrest-based/usecase/parameters.txt create mode 100644 docs/multitenant/lrest-based/usecase/pdbnamespace_binding.yaml create mode 100644 docs/multitenant/lrest-based/usecase/plug_pdb1_resource.yaml create mode 100644 docs/multitenant/lrest-based/usecase/unplug_pdb1_resource.yaml create mode 100644 docs/multitenant/ords-based/NamespaceSeg.md create mode 100644 docs/multitenant/ords-based/README.md rename docs/multitenant/{ => ords-based}/images/K8S_NAMESPACE_SEG.png (100%) rename docs/multitenant/{ => ords-based}/images/K8S_SECURE1.png (100%) rename docs/multitenant/{ => ords-based}/images/K8S_SECURE2.png (100%) rename docs/multitenant/{ => ords-based}/images/K8S_SECURE3.png (100%) rename docs/multitenant/{ => ords-based}/images/K8S_SECURE4.png (100%) create mode 100644 docs/multitenant/ords-based/images/makerunall.png create mode 100644 docs/multitenant/ords-based/images/makesecrets_1_1.png rename docs/multitenant/{ => ords-based}/openssl_schema.jpg (100%) rename docs/multitenant/{ => ords-based}/provisioning/example_setup_using_oci_oke_cluster.md (100%) rename docs/multitenant/{ => ords-based}/provisioning/multinamespace/cdb_create.yaml (58%) rename docs/multitenant/{ => ords-based}/provisioning/multinamespace/pdb_clone.yaml (74%) rename docs/multitenant/{ => ords-based}/provisioning/multinamespace/pdb_close.yaml (66%) rename docs/multitenant/{ => ords-based}/provisioning/multinamespace/pdb_create.yaml (67%) rename docs/multitenant/{ => ords-based}/provisioning/multinamespace/pdb_delete.yaml (70%) rename docs/multitenant/{ => ords-based}/provisioning/multinamespace/pdb_open.yaml (66%) rename docs/multitenant/{ => ords-based}/provisioning/multinamespace/pdb_plug.yaml (80%) rename docs/multitenant/{ => ords-based}/provisioning/multinamespace/pdb_unplug.yaml (77%) create mode 100644 docs/multitenant/ords-based/provisioning/ords_image.md rename docs/multitenant/{ => ords-based}/provisioning/quickOKEcreation.md (100%) rename docs/multitenant/{usecase01 => ords-based/provisioning/singlenamespace}/cdb_create.yaml (50%) rename docs/multitenant/{ => ords-based}/provisioning/singlenamespace/cdb_secret.yaml (100%) rename docs/multitenant/{ => ords-based}/provisioning/singlenamespace/pdb_clone.yaml (78%) rename docs/multitenant/{usecase01 => ords-based/provisioning/singlenamespace}/pdb_close.yaml (67%) rename docs/multitenant/{usecase01 => ords-based/provisioning/singlenamespace}/pdb_create.yaml (69%) rename docs/multitenant/{usecase01 => ords-based/provisioning/singlenamespace}/pdb_delete.yaml (72%) rename docs/multitenant/{usecase01 => ords-based/provisioning/singlenamespace}/pdb_open.yaml (67%) rename docs/multitenant/{usecase02 => ords-based/provisioning/singlenamespace}/pdb_plug.yaml (81%) rename docs/multitenant/{ => ords-based}/provisioning/singlenamespace/pdb_secret.yaml (100%) rename docs/multitenant/{usecase02 => ords-based/provisioning/singlenamespace}/pdb_unplug.yaml (78%) create mode 100644 docs/multitenant/ords-based/usecase/README.md create mode 100644 docs/multitenant/ords-based/usecase/cdbnamespace_binding.yaml create mode 100644 docs/multitenant/ords-based/usecase/clone_pdb1_resource.yaml create mode 100644 docs/multitenant/ords-based/usecase/clone_pdb2_resource.yaml create mode 100644 docs/multitenant/ords-based/usecase/close_pdb1_resource.yaml create mode 100644 docs/multitenant/ords-based/usecase/close_pdb2_resource.yaml create mode 100644 docs/multitenant/ords-based/usecase/close_pdb3_resource.yaml create mode 100644 docs/multitenant/ords-based/usecase/create_ords_pod.yaml create mode 100644 docs/multitenant/ords-based/usecase/create_pdb1_resource.yaml create mode 100644 docs/multitenant/ords-based/usecase/create_pdb2_resource.yaml create mode 100644 docs/multitenant/ords-based/usecase/delete_pdb1_resource.yaml create mode 100644 docs/multitenant/ords-based/usecase/delete_pdb2_resource.yaml create mode 100644 docs/multitenant/ords-based/usecase/makefile create mode 100644 docs/multitenant/ords-based/usecase/map_pdb1_resource.yaml create mode 100644 docs/multitenant/ords-based/usecase/map_pdb2_resource.yaml create mode 100644 docs/multitenant/ords-based/usecase/map_pdb3_resource.yaml create mode 100644 docs/multitenant/ords-based/usecase/open_pdb1_resource.yaml create mode 100644 docs/multitenant/ords-based/usecase/open_pdb2_resource.yaml create mode 100644 docs/multitenant/ords-based/usecase/open_pdb3_resource.yaml create mode 100644 docs/multitenant/ords-based/usecase/parameters.txt create mode 100644 docs/multitenant/ords-based/usecase/pdbnamespace_binding.yaml create mode 100644 docs/multitenant/ords-based/usecase/plug_pdb1_resource.yaml create mode 100644 docs/multitenant/ords-based/usecase/unplug_pdb1_resource.yaml rename docs/multitenant/{ => ords-based}/usecase01/README.md (85%) rename docs/multitenant/{ => ords-based}/usecase01/ca.crt (100%) rename docs/multitenant/{ => ords-based}/usecase01/ca.key (100%) rename docs/multitenant/{ => ords-based}/usecase01/ca.srl (100%) rename docs/multitenant/{provisioning/singlenamespace => ords-based/usecase01}/cdb_create.yaml (100%) rename docs/multitenant/{ => ords-based}/usecase01/cdb_secret.yaml (100%) rename docs/multitenant/{usecase02/pdb_clone.yaml => ords-based/usecase01/clone_pdb1_resource.yaml} (57%) create mode 100644 docs/multitenant/ords-based/usecase01/clone_pdb2_resource.yaml create mode 100644 docs/multitenant/ords-based/usecase01/close_pdb1_resource.yaml create mode 100644 docs/multitenant/ords-based/usecase01/close_pdb2_resource.yaml create mode 100644 docs/multitenant/ords-based/usecase01/close_pdb3_resource.yaml create mode 100644 docs/multitenant/ords-based/usecase01/create_ords_pod.yaml create mode 100644 docs/multitenant/ords-based/usecase01/create_pdb1_resource.yaml create mode 100644 docs/multitenant/ords-based/usecase01/create_pdb2_resource.yaml create mode 100644 docs/multitenant/ords-based/usecase01/delete_pdb1_resource.yaml create mode 100644 docs/multitenant/ords-based/usecase01/delete_pdb2_resource.yaml rename docs/multitenant/{ => ords-based}/usecase01/extfile.txt (100%) create mode 100644 docs/multitenant/ords-based/usecase01/logfiles/BuildImage.log rename docs/multitenant/{ => ords-based}/usecase01/logfiles/ImagePush.log (100%) rename docs/multitenant/{ => ords-based}/usecase01/logfiles/cdb.log (100%) rename docs/multitenant/{ => ords-based}/usecase01/logfiles/cdb_creation.log (100%) create mode 100644 docs/multitenant/ords-based/usecase01/logfiles/openssl_execution.log create mode 100644 docs/multitenant/ords-based/usecase01/logfiles/ordsconfig.log rename docs/multitenant/{ => ords-based}/usecase01/logfiles/tagandpush.log (100%) rename docs/multitenant/{ => ords-based}/usecase01/logfiles/testapi.log (100%) create mode 100644 docs/multitenant/ords-based/usecase01/makefile create mode 100644 docs/multitenant/ords-based/usecase01/map_pdb1_resource.yaml create mode 100644 docs/multitenant/ords-based/usecase01/map_pdb2_resource.yaml create mode 100644 docs/multitenant/ords-based/usecase01/map_pdb3_resource.yaml create mode 100644 docs/multitenant/ords-based/usecase01/open_pdb1_resource.yaml create mode 100644 docs/multitenant/ords-based/usecase01/open_pdb2_resource.yaml create mode 100644 docs/multitenant/ords-based/usecase01/open_pdb3_resource.yaml create mode 100644 docs/multitenant/ords-based/usecase01/oracle-database-operator-system_binding.yaml rename docs/multitenant/{ => ords-based}/usecase01/oracle-database-operator.yaml (100%) create mode 100644 docs/multitenant/ords-based/usecase01/parameters.txt rename docs/multitenant/{provisioning/singlenamespace => ords-based/usecase01}/pdb_close.yaml (100%) rename docs/multitenant/{provisioning/singlenamespace => ords-based/usecase01}/pdb_create.yaml (100%) rename docs/multitenant/{provisioning/singlenamespace => ords-based/usecase01}/pdb_delete.yaml (100%) rename docs/multitenant/{ => ords-based}/usecase01/pdb_map.yaml (100%) rename docs/multitenant/{provisioning/singlenamespace => ords-based/usecase01}/pdb_open.yaml (100%) rename docs/multitenant/{ => ords-based}/usecase01/pdb_secret.yaml (100%) rename docs/multitenant/{provisioning/singlenamespace/pdb_plug.yaml => ords-based/usecase01/plug_pdb1_resource.yaml} (63%) rename docs/multitenant/{ => ords-based}/usecase01/server.csr (100%) rename docs/multitenant/{ => ords-based}/usecase01/tde_secret.yaml (100%) rename docs/multitenant/{ => ords-based}/usecase01/tls.crt (100%) rename docs/multitenant/{ => ords-based}/usecase01/tls.key (100%) rename docs/multitenant/{provisioning/singlenamespace/pdb_unplug.yaml => ords-based/usecase01/unplug_pdb1_resource.yaml} (59%) rename docs/multitenant/{ => ords-based}/usecase02/README.md (69%) create mode 100644 docs/multitenant/ords-based/usecase02/pdb_clone.yaml create mode 100644 docs/multitenant/ords-based/usecase02/pdb_plug.yaml rename docs/multitenant/{ => ords-based}/usecase02/pdb_plugtde.yaml (96%) create mode 100644 docs/multitenant/ords-based/usecase02/pdb_unplug.yaml rename docs/multitenant/{ => ords-based}/usecase02/pdb_unplugtde.yaml (96%) delete mode 100644 docs/multitenant/usecase02/tde_secret.yaml create mode 100644 docs/ordsservices/README.md create mode 100644 docs/ordsservices/TROUBLESHOOTING.md create mode 100644 docs/ordsservices/api.md create mode 100644 docs/ordsservices/autoupgrade.md create mode 100644 docs/ordsservices/examples/adb.md create mode 100644 docs/ordsservices/examples/adb_oraoper.md create mode 100644 docs/ordsservices/examples/mongo_api.md create mode 100644 docs/ordsservices/examples/multi_pool.md create mode 100644 docs/ordsservices/examples/sidb_container.md create mode 100644 docs/ordsservices/usecase01/create_mong_schema.sql create mode 100644 docs/ordsservices/usecase01/help create mode 100644 docs/ordsservices/usecase01/makefile create mode 100644 docs/ordsservices/usecase01/tnsadmin/tnsnames.ora create mode 100644 docs/ordsservices/usecase01/tnsadmin/tnsnames.ora.offline create mode 100644 docs/sharding/provisioning/system_sharding/ssharding_shard_prov_chunks.yaml create mode 100644 ords/ords_init.sh diff --git a/.gitignore b/.gitignore index 98fbc1c4..51923538 100644 --- a/.gitignore +++ b/.gitignore @@ -7,7 +7,7 @@ ords/*zip .gitattributes .vscode .gitlab-ci.yml - +.DS_Store # development .idea .local diff --git a/Dockerfile b/Dockerfile index 11a56962..f444d508 100644 --- a/Dockerfile +++ b/Dockerfile @@ -6,13 +6,16 @@ ARG BUILDER_IMG FROM ${BUILDER_IMG} as builder -# Download golang if BUILD_INTERNAL is set to true +ARG TARGETARCH +# Download golang if INSTALL_GO is set to true ARG INSTALL_GO ARG GOLANG_VERSION RUN if [ "$INSTALL_GO" = "true" ]; then \ - curl -LJO https://go.dev/dl/go${GOLANG_VERSION}.linux-amd64.tar.gz &&\ - rm -rf /usr/local/go && tar -C /usr/local -xzf go${GOLANG_VERSION}.linux-amd64.tar.gz &&\ - rm go${GOLANG_VERSION}.linux-amd64.tar.gz; \ + echo -e "\nCurrent Arch: $(arch), Downloading Go for linux/${TARGETARCH}" &&\ + curl -LJO https://go.dev/dl/go${GOLANG_VERSION}.linux-${TARGETARCH}.tar.gz &&\ + rm -rf /usr/local/go && tar -C /usr/local -xzf go${GOLANG_VERSION}.linux-${TARGETARCH}.tar.gz &&\ + rm go${GOLANG_VERSION}.linux-${TARGETARCH}.tar.gz; \ + echo "Go Arch: $(/usr/local/go/bin/go env GOARCH)"; \ fi ENV PATH=${GOLANG_VERSION:+"${PATH}:/usr/local/go/bin"} @@ -33,16 +36,17 @@ COPY LICENSE.txt LICENSE.txt COPY THIRD_PARTY_LICENSES_DOCKER.txt THIRD_PARTY_LICENSES_DOCKER.txt # Build -RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 GO111MODULE=on go build -a -o manager main.go +RUN CGO_ENABLED=0 GOOS=linux GOARCH=${TARGETARCH} GO111MODULE=on go build -a -o manager main.go -# Use oraclelinux:8-slim as base image to package the manager binary -FROM oraclelinux:8-slim +# Use oraclelinux:9 as base image to package the manager binary +FROM oraclelinux:9 ARG CI_COMMIT_SHA ARG CI_COMMIT_BRANCH ENV COMMIT_SHA=${CI_COMMIT_SHA} \ COMMIT_BRANCH=${CI_COMMIT_BRANCH} WORKDIR / COPY --from=builder /workspace/manager . +COPY ords/ords_init.sh . RUN useradd -u 1002 nonroot USER nonroot diff --git a/Makefile b/Makefile index 88e14843..b9755e6f 100644 --- a/Makefile +++ b/Makefile @@ -1,5 +1,5 @@ # -# Copyright (c) 2022, Oracle and/or its affiliates. +# Copyright (c) 2025, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # @@ -18,14 +18,14 @@ BUNDLE_METADATA_OPTS ?= $(BUNDLE_CHANNELS) $(BUNDLE_DEFAULT_CHANNEL) # Image URL to use all building/pushing image targets IMG ?= controller:latest -# Produce CRDs that work back to Kubernetes 1.11 (no version conversion) -# API version has to be v1 to use defaulting (https://github.com/kubernetes-sigs/controller-tools/issues/478) -CRD_OPTIONS ?= "crd:trivialVersions=true,preserveUnknownFields=false" +# Enable allowDangerousTypes to use float type in CRD +# Remove the Desc to avoid YAML getting too long. See the discussion: +# https://github.com/kubernetes-sigs/kubebuilder/issues/1140 +CRD_OPTIONS ?= "crd:maxDescLen=0,allowDangerousTypes=true" # ENVTEST_K8S_VERSION refers to the version of kubebuilder assets to be downloaded by envtest binary. -ENVTEST_K8S_VERSION = 1.21 +ENVTEST_K8S_VERSION = 1.29.0 # Operator YAML file OPERATOR_YAML=$$(basename $$(pwd)).yaml - # Get the currently used golang install path (in GOPATH/bin, unless GOBIN is set) ifeq (,$(shell go env GOBIN)) GOBIN=$(shell go env GOPATH)/bin @@ -40,69 +40,82 @@ SHELL = /usr/bin/env bash -o pipefail .SHELLFLAGS = -ec all: build - ##@ Development manifests: controller-gen ## Generate WebhookConfiguration, ClusterRole and CustomResourceDefinition objects. $(CONTROLLER_GEN) $(CRD_OPTIONS) rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases - + generate: controller-gen ## Generate code containing DeepCopy, DeepCopyInto, and DeepCopyObject method implementations. $(CONTROLLER_GEN) object:headerFile="hack/boilerplate.go.txt" paths="./..." - + fmt: ## Run go fmt against code. go fmt ./... - + vet: ## Run go vet against code. go vet ./... - + TEST ?= ./apis/database/v1alpha1 ./commons/... ./controllers/... test: manifests generate fmt vet envtest ## Run unit tests. KUBEBUILDER_ASSETS="$(shell $(ENVTEST) use $(ENVTEST_K8S_VERSION) -p path)" go test $(TEST) -coverprofile cover.out - + E2ETEST ?= ./test/e2e/ e2e: manifests generate fmt vet envtest ## Run e2e tests. KUBEBUILDER_ASSETS="$(shell $(ENVTEST) use $(ENVTEST_K8S_VERSION) -p path)" go test $(E2ETEST) -test.timeout 0 -test.v --ginkgo.fail-fast - + ##@ Build - + build: generate fmt vet ## Build manager binary. go build -o bin/manager main.go - + run: manifests generate fmt vet ## Run a controller from your host. go run ./main.go - -GOLANG_VERSION ?= 1.21.7 + +GOLANG_VERSION ?= 1.23.3 ## Download golang in the Dockerfile if BUILD_INTERNAL is set to true. ## Otherwise, use golang image from docker hub as the builder. ifeq ($(BUILD_INTERNAL), true) -BUILDER_IMG = oraclelinux:8 +BUILDER_IMG = oraclelinux:9 BUILD_ARGS = --build-arg BUILDER_IMG=$(BUILDER_IMG) --build-arg GOLANG_VERSION=$(GOLANG_VERSION) --build-arg INSTALL_GO=true else BUILDER_IMG = golang:$(GOLANG_VERSION) -BUILD_ARGS = --build-arg BUILDER_IMG=$(BUILDER_IMG) --build-arg INSTALL_GO=false +BUILD_ARGS = --build-arg BUILDER_IMG=$(BUILDER_IMG) --build-arg INSTALL_GO="false" --build-arg GOLANG_VERSION=$(GOLANG_VERSION) +endif +ifeq ($(BUILD_MANIFEST), true) +BUILD_ARGS := $(BUILD_ARGS) --platform=linux/arm64,linux/amd64 --jobs=2 --manifest +PUSH_ARGS := manifest +else +BUILD_ARGS := $(BUILD_ARGS) --platform=linux/amd64 --tag endif docker-build: #manifests generate fmt vet #test ## Build docker image with the manager. Disable the test but keep the validations to fail fast docker build --no-cache=true --build-arg http_proxy=$(HTTP_PROXY) --build-arg https_proxy=$(HTTPS_PROXY) \ - --build-arg CI_COMMIT_SHA=$(CI_COMMIT_SHA) --build-arg CI_COMMIT_BRANCH=$(CI_COMMIT_BRANCH) \ - $(BUILD_ARGS) . -t $(IMG) - + --build-arg CI_COMMIT_SHA=$(CI_COMMIT_SHA) --build-arg CI_COMMIT_BRANCH=$(CI_COMMIT_BRANCH) \ + $(BUILD_ARGS) $(IMG) . + docker-push: ## Push docker image with the manager. - docker push $(IMG) + docker $(PUSH_ARGS) push $(IMG) -##@ Deployment +# Push to minikube's local registry enabled by registry add-on +minikube-push: + docker tag $(IMG) $$(minikube ip):5000/$(IMG) + docker push --tls-verify=false $$(minikube ip):5000/$(IMG) +##@ Deployment + install: manifests kustomize ## Install CRDs into the K8s cluster specified in ~/.kube/config. $(KUSTOMIZE) build config/crd | kubectl apply -f - - + uninstall: manifests kustomize ## Uninstall CRDs from the K8s cluster specified in ~/.kube/config. $(KUSTOMIZE) build config/crd | kubectl delete -f - - + deploy: manifests kustomize ## Deploy controller to the K8s cluster specified in ~/.kube/config. cd config/manager && $(KUSTOMIZE) edit set image controller=$(IMG) $(KUSTOMIZE) build config/default | kubectl apply -f - +minikube-deploy: minikube-operator-yaml minikube-push + kubectl apply -f $(OPERATOR_YAML) + # Bug:34265574 -# Used sed to reposition the controller-manager Deployment after the certificate creation in the OPERATOR_YAML +# Used sed to reposition the controller-manager Deployment after the certificate creation in the OPERATOR_YAML operator-yaml: manifests kustomize cd config/manager && $(KUSTOMIZE) edit set image controller=$(IMG) $(KUSTOMIZE) build config/default > "$(OPERATOR_YAML)" @@ -110,57 +123,62 @@ operator-yaml: manifests kustomize (echo --- && sed '/^apiVersion: apps\/v1/,/---/!d' "$(OPERATOR_YAML).bak") >> "$(OPERATOR_YAML)" rm "$(OPERATOR_YAML).bak" +minikube-operator-yaml: IMG:=localhost:5000/$(IMG) +minikube-operator-yaml: operator-yaml + sed -i.bak 's/\(replicas.\) 3/\1 1/g' "$(OPERATOR_YAML)" + rm "$(OPERATOR_YAML).bak" + undeploy: ## Undeploy controller from the K8s cluster specified in ~/.kube/config. $(KUSTOMIZE) build config/default | kubectl delete -f - - + ##@ Build Dependencies - + ## Location to install dependencies to LOCALBIN ?= $(shell pwd)/bin $(LOCALBIN): mkdir -p $(LOCALBIN) - + ## Tool Binaries KUSTOMIZE ?= $(LOCALBIN)/kustomize CONTROLLER_GEN ?= $(LOCALBIN)/controller-gen ENVTEST ?= $(LOCALBIN)/setup-envtest - + ## Tool Versions -KUSTOMIZE_VERSION ?= v3.8.7 -CONTROLLER_TOOLS_VERSION ?= v0.6.1 - +KUSTOMIZE_VERSION ?= v5.3.0 +CONTROLLER_TOOLS_VERSION ?= v0.16.5 + KUSTOMIZE_INSTALL_SCRIPT ?= "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh" .PHONY: kustomize kustomize: $(KUSTOMIZE) ## Download kustomize locally if necessary. $(KUSTOMIZE): $(LOCALBIN) curl -s $(KUSTOMIZE_INSTALL_SCRIPT) | bash -s -- $(subst v,,$(KUSTOMIZE_VERSION)) $(LOCALBIN) - + .PHONY: controller-gen controller-gen: $(CONTROLLER_GEN) ## Download controller-gen locally if necessary. $(CONTROLLER_GEN): $(LOCALBIN) GOBIN=$(LOCALBIN) go install sigs.k8s.io/controller-tools/cmd/controller-gen@$(CONTROLLER_TOOLS_VERSION) - + .PHONY: envtest envtest: $(ENVTEST) ## Download envtest-setup locally if necessary. $(ENVTEST): $(LOCALBIN) GOBIN=$(LOCALBIN) go install sigs.k8s.io/controller-runtime/tools/setup-envtest@latest - - + + .PHONY: bundle bundle: manifests kustomize ## Generate bundle manifests and metadata, then validate generated files. operator-sdk generate kustomize manifests -q cd config/manager && $(KUSTOMIZE) edit set image controller=$(IMG) $(KUSTOMIZE) build config/manifests | operator-sdk generate bundle -q --overwrite --version $(VERSION) $(BUNDLE_METADATA_OPTS) operator-sdk bundle validate ./bundle - + .PHONY: bundle-build bundle-build: ## Build the bundle image. docker build -f bundle.Dockerfile -t $(BUNDLE_IMG) . - + .PHONY: bundle-push bundle-push: ## Push the bundle image. $(MAKE) docker-push IMG=$(BUNDLE_IMG) - + .PHONY: opm OPM = ./bin/opm opm: ## Download opm locally if necessary. @@ -172,33 +190,32 @@ ifeq (,$(shell which opm 2>/dev/null)) OS=$(shell go env GOOS) && ARCH=$(shell go env GOARCH) && \ curl -sSLo $(OPM) https://github.com/operator-framework/operator-registry/releases/download/v1.15.1/$${OS}-$${ARCH}-opm ;\ chmod +x $(OPM) ;\ - } + } else OPM = $(shell which opm) endif endif - + # A comma-separated list of bundle images (e.g. make catalog-build BUNDLE_IMGS=example.com/operator-bundle:v0.1.0,example.com/operator-bundle:v0.2.0). # These images MUST exist in a registry and be pull-able. BUNDLE_IMGS ?= $(BUNDLE_IMG) - + # The image tag given to the resulting catalog image (e.g. make catalog-build CATALOG_IMG=example.com/operator-catalog:v0.2.0). CATALOG_IMG ?= $(IMAGE_TAG_BASE)-catalog:v$(VERSION) - + # Set CATALOG_BASE_IMG to an existing catalog image tag to add $BUNDLE_IMGS to that image. ifneq ($(origin CATALOG_BASE_IMG), undefined) FROM_INDEX_OPT := --from-index $(CATALOG_BASE_IMG) endif - + # Build a catalog image by adding bundle images to an empty catalog using the operator package manager tool, 'opm'. # This recipe invokes 'opm' in 'semver' bundle add mode. For more information on add modes, see: # https://github.com/operator-framework/community-operators/blob/7f1438c/docs/packaging-operator.md#updating-your-existing-operator .PHONY: catalog-build catalog-build: opm ## Build a catalog image. $(OPM) index add --container-tool docker --mode semver --tag $(CATALOG_IMG) --bundles $(BUNDLE_IMGS) $(FROM_INDEX_OPT) - + # Push the catalog image. .PHONY: catalog-push catalog-push: ## Push a catalog image. $(MAKE) docker-push IMG=$(CATALOG_IMG) - diff --git a/PREREQUISITES.md b/PREREQUISITES.md index bc333357..3c73ad4b 100644 --- a/PREREQUISITES.md +++ b/PREREQUISITES.md @@ -2,7 +2,7 @@ ## Prerequisites for Using Oracle Database Operator for Kubernetes -Oracle Database Operator for Kubernetes (OraOperator) manages all Cloud deployments of Oracle Database, including: +Oracle Database Operator for Kubernetes (`OraOperator`) manages all Cloud deployments of Oracle Database, including: * Oracle Autonomous Database (ADB) * Containerized Oracle Database Single Instance (SIDB) @@ -19,15 +19,15 @@ To set up a Kubernetes cluster on Oracle Cloud Infrastructure: 1. Create an OKE Cluster 1. Provision persistent storage for data files (NFS or Block) -Note: You must provision persistent storage if you intend to deploy containerized databases over the OKE cluster. +Note: If you intend to deploy containerized databases over the OKE cluster, then you must provision persistent storage. ### Prerequites for Oracle Autonomous Database (ADB) -If you intent to use `OraOperator` to handle Oracle Autonomous Database lifecycles, then read [Oracle Autonomous Database prerequisites](./docs/adb/ADB_PREREQUISITES.md) +If you intend to use `OraOperator` to handle Oracle Autonomous Database lifecycles, then read [Oracle Autonomous Database prerequisites](./docs/adb/ADB_PREREQUISITES.md) ### Prerequites for Single Instance Databases (SIDB) -If you intent to use `OraOperator` to handle Oracle Database Single Instance lifecycles, then read [Single Instance Database Prerequisites](./docs/sidb/PREREQUISITES.md) +If you intend to use `OraOperator` to handle Oracle Database Single Instance lifecycles, then read [Single Instance Database Prerequisites](./docs/sidb/PREREQUISITES.md) ### Prerequites for Oracle Globally Distributed Databases(GDD) diff --git a/PROJECT b/PROJECT index fbf861db..97e9409c 100644 --- a/PROJECT +++ b/PROJECT @@ -131,6 +131,10 @@ resources: kind: DbcsSystem path: github.com/oracle/oracle-database-operator/apis/database/v1alpha1 version: v1alpha1 + webhooks: + defaulting: true + validation: true + webhookVersion: v1beta1 - api: crdVersion: v1beta1 namespaced: true @@ -157,4 +161,102 @@ resources: defaulting: true validation: true webhookVersion: v1beta1 +- api: + crdVersion: v1beta1 + namespaced: true + domain: oracle.com + group: database + kind: ShardingDatabase + path: github.com/oracle/oracle-database-operator/apis/database/v4 + version: v4 + webhooks: + conversion: true + webhookVersion: v1beta1 +- api: + crdVersion: v1beta1 + namespaced: true + domain: oracle.com + group: database + kind: DbcsSystem + path: github.com/oracle/oracle-database-operator/apis/database/v4 + version: v4 +- api: + crdVersion: v1beta1 + namespaced: true + domain: oracle.com + group: database + kind: LREST + path: github.com/oracle/oracle-database-operator/apis/database/v4 + version: v4 +- api: + crdVersion: v1beta1 + namespaced: true + domain: oracle.com + group: database + kind: LRPDB + path: github.com/oracle/oracle-database-operator/apis/database/v4 + version: v4 +- api: + crdVersion: v1beta1 + namespaced: true + domain: oracle.com + group: database + kind: OrdsSrvs + path: github.com/oracle/oracle-database-operator/apis/database/v4 + version: v4 +- api: + crdVersion: v1beta1 + namespaced: true + domain: oracle.com + group: observability + kind: DatabaseObserver + path: github.com/oracle/oracle-database-operator/apis/observability/v1 + version: v1 + webhooks: + conversion: true + webhookVersion: v1beta1 +- api: + crdVersion: v1beta1 + namespaced: true + domain: oracle.com + group: observability + kind: DatabaseObserver + path: github.com/oracle/oracle-database-operator/apis/observability/v4 + version: v4 + webhooks: + conversion: true + webhookVersion: v1beta1 +- api: + crdVersion: v1beta1 + namespaced: true + domain: oracle.com + group: database + kind: SingleInstanceDatabase + path: github.com/oracle/oracle-database-operator/apis/database/v4 + version: v4 + webhooks: + conversion: true + webhookVersion: v1beta1 +- api: + crdVersion: v1beta1 + namespaced: true + domain: oracle.com + group: database + kind: DataguardBroker + path: github.com/oracle/oracle-database-operator/apis/database/v4 + version: v4 + webhooks: + conversion: true + webhookVersion: v1beta1 +- api: + crdVersion: v1beta1 + namespaced: true + domain: oracle.com + group: database + kind: OracleRestDataService + path: github.com/oracle/oracle-database-operator/apis/database/v4 + version: v4 + webhooks: + conversion: true + webhookVersion: v1beta1 version: "3" diff --git a/README.md b/README.md index 3409463b..7afa79e8 100644 --- a/README.md +++ b/README.md @@ -2,53 +2,68 @@ ## Make Oracle Database Kubernetes Native -As part of Oracle's resolution to make Oracle Database Kubernetes native (that is, observable and operable by Kubernetes), Oracle released _Oracle Database Operator for Kubernetes_ (`OraOperator` or the operator). OraOperator extends the Kubernetes API with custom resources and controllers for automating Oracle Database lifecycle management. +As part of Oracle's resolution to make Oracle Database Kubernetes native (that is, observable and operable by Kubernetes), Oracle released the _Oracle Database Operator for Kubernetes_ (`OraOperator` or the operator). OraOperator extends the Kubernetes API with custom resources and controllers for automating the management of the Oracle Database lifecycle. -In this v1.1.0 production release, `OraOperator` supports the following database configurations and infrastructure: +## Supported Database Configurations in V1.2.0 +In this v1.2.0 production release, `OraOperator` supports the following database configurations, and controllers: * Oracle Autonomous Database: * Oracle Autonomous Database shared Oracle Cloud Infrastructure (OCI) (ADB-S) * Oracle Autonomous Database on dedicated Cloud infrastructure (ADB-D) - * Oracle Autonomous Container Database (ACD) (infrastructure) is the infrastructure for provisioning Autonomous Databases. + * Oracle Autonomous Container Database (ACD), the infrastructure for provisioning Autonomous Databases. * Containerized Single Instance databases (SIDB) deployed in the Oracle Kubernetes Engine (OKE) and any k8s where OraOperator is deployed * Containerized Oracle Globally Distributed Databases(GDD) deployed in OKE and any k8s where OraOperator is deployed * Oracle Multitenant Databases (CDB/PDBs) -* Oracle Base Database Cloud Service (BDBCS) -* Oracle Data Guard (Preview status) -* Oracle Database Observability (Preview status) - -Oracle will continue to extend `OraOperator` to support additional Oracle Database configurations. - -## New in V1.1.0 Release -* Namespace scope deployment option -* Enhanced security with namespace scope deployment option -* Support for Oracle Database 23ai Free (with SIDB) -* Automatic Storage Expansion for SIDB and Oracle Globally Distributed Database -* User-Defined Sharding -* TCPS support customer provided certs -* Execute custom scripts during DB setup/startup -* Patching for SIDB Primary/Standby in Data Guard -* Long-term backup for Autonomous Databases (ADB): Support for [long-term retention backup](https://docs.oracle.com/en/cloud/paas/autonomous-database/serverless/adbsb/backup-long-term.html) and removed support for the deprecated mandatory backup -* Wallet expiry date for ADB: A user-friendly enhancement to display the wallet expiry date in the status of the associated ADB -* Wait-for-Completion option for ADB: Supports `kubectl wait` command that allows the user to wait for a specific condition on ADB -* OKE workload Identify: Supports OKE workload identity authentication method (i.e., uses OKE credentials). For more details, refer to [Oracle Autonomous Database (ADB) Prerequisites](docs/adb/ADB_PREREQUISITES.md#authorized-with-oke-workload-identity) -* Database Observability (Preview - Metrics) - -## Features Summary +* Oracle Base Database Service (OBDS) on Oracle Cloud Infrastructure (OCI) +* Oracle Data Guard +* Oracle Database Observability +* Oracle Database Rest Service (ORDS) instances + +## New Lifecycle Features in V1.2.0 Release (Controllers Enhancements) +* ORDSSERVICES + - Install on SIDB and ADB + - Provision and Delete ORDS instances +* SIDB + - Oracle Database 23ai Free support + - Oracle Database 23ai Free-lite support + - SIDB resource management + - True Cache support for Free SIDB databases (Preview) + - Observer for FastStartFailover with Data Guard + - Snapshot Standby support in Data Guard setup +* Globally Distributed Database : Support for Oracle Database 23ai Raft replication +* Autonomous Database: support for Database cloning +* Multitenant DB: + - ORDS-based Controller: assertive deletion policy. + - New LRES based Controller (ARM & AM) + - PDBs settings with init parameters config map + - Assertive deletion policy. +* Database Observability (preview) + - Support for Database Logs (in addition to Metrics) + - Support for the latest Exporter container images + - Bug Fix: Prometheus label config +* Oracle Base Database Service: support for Oracle Database 23ai Cloning, using KMS Vaults, PDB creation. + +## New Product Features +*The Operator itself, as a product, brings the following new features: +* Published on `operatorhub.io` +* Operator Lifecycle Manager (OLM) support (install from `operatorhub.io`) +* Validated on Google Kubernetes Engine + +## Overall Features Summary This release of Oracle Database Operator for Kubernetes (the operator) supports the following lifecycle operations: -* ADB-S/ADB-D: Provision, bind, start, stop, terminate (soft/hard), scale (up/down), long-term backup, manual restore -* ACD: provision, bind, restart, terminate (soft/hard) -* SIDB: Provision, clone, patch (in-place/out-of-place), update database initialization parameters, update database configuration (Flashback, archiving), Oracle Enterprise Manager (EM) Express (a basic observability console), Oracle REST Data Service (ORDS) to support REST based SQL, PDB management, SQL Developer Web, and Application Express (Apex) -* GDD: Provision/deploy Oracle Globally Distributed Databases and the GDD topology, Add a new shard, Delete an existing shard -* Oracle Multitenant Database: Bind to a CDB, Create a  PDB, Plug a  PDB, Unplug a PDB, Delete a PDB, Clone a PDB, Open/Close a PDB -* Oracle Base Database Cloud Service (BDBCS): provision, bind, scale shape Up/Down, Scale Storage Up, Terminate and Update License +* ADB-S/ADB-D: Provision, bind, start, stop, terminate (soft/hard), scale (up/down), long-term backup, manual restore, cloning. +* ACD: Provision, bind, restart, terminate (soft/hard) +* SIDB: Provision, clone, patch (in-place/out-of-place), update database initialization parameters, update database configuration (Flashback, archiving), Oracle Enterprise Manager (EM) Express (basic console), Oracle REST Data Service (ORDS) to support REST based SQL, PDB management, SQL Developer Web, Application Express (Apex), Resource management, True Cache, Observer for FastStartFailover (Data Guard), and Snapshot Standby (Data Guard) +* ORDS Services: Provision and delete ORDS instances +* Globally Distrib. (Sharded): Provision/deploy sharded databases and the shard topology, Add a new shard, Delete an existing shard, Raft replication. +* Oracle Multitenant Database (choice of controller): Bind to a CDB, Create a  PDB, Plug a  PDB, Unplug a PDB, Delete a PDB, Clone a PDB, Open/Close a PDB, Assertive deletion policy +* Oracle Base Database Service (OBDS): Provision, bind, scale shape Up/Down, Scale Storage Up, Terminate and Update License, Cloning, PDB creation, using KMS Vaults on Oracle Cloud Infrastructure (OCI) * Oracle Data Guard: Provision a Standby for the SIDB resource, Create a Data Guard Configuration, Perform a Switchover, Patch Primary and Standby databases in Data Guard Configuration -* Oracle Database Observability: create, patch, delete databaseObserver resources -* Watch over a set of namespaces or all the namespaces in the cluster using the "WATCH_NAMESPACE" env variable of the operator deployment +* Oracle Database Observability: create, patch, delete `databaseObserver` resources (Logs and Metrics) +* Watch over a set of namespaces or all the namespaces in the cluster using the `WATCH_NAMESPACE` environment variable of the operator deployment -The upcoming releases will support new configurations, operations, and capabilities. ## Release Status @@ -56,11 +71,11 @@ This production release has been installed and tested on the following Kubernete * [Oracle Container Engine for Kubernetes (OKE)](https://www.oracle.com/cloud-native/container-engine-kubernetes/) with Kubernetes 1.24 * [Oracle Linux Cloud Native Environment(OLCNE)](https://docs.oracle.com/en/operating-systems/olcne/) 1.6 -* [Minikube](https://minikube.sigs.k8s.io/docs/) with version v1.29.0 -* [Azure Kubernetes Service](https://azure.microsoft.com/en-us/services/kubernetes-service/) +* [Azure Kubernetes Service](https://azure.microsoft.com/en-us/services/kubernetes-service/) * [Amazon Elastic Kubernetes Service](https://aws.amazon.com/eks/) +* [Google Kubernetes Engine](https://cloud.google.com/kubernetes-engine/docs) * [Red Hat OKD](https://www.okd.io/) -* [Red Hat OpenShift](https://www.redhat.com/en/technologies/cloud-computing/openshift/) +* [Minikube](https://minikube.sigs.k8s.io/docs/) with version v1.29.0 ## Prerequisites @@ -68,12 +83,12 @@ Oracle strongly recommends that you ensure your system meets the following [Prer * ### Install cert-manager - The operator uses webhooks for validating user input before persisting it in etcd. Webhooks require TLS certificates that are generated and managed by a certificate manager. + The operator uses webhooks for validating user input before persisting it in `etcd`. Webhooks require TLS certificates that are generated and managed by a certificate manager. Install the certificate manager with the following command: ```sh - kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.14.4/cert-manager.yaml + kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.16.2/cert-manager.yaml ``` * ### Create Role Bindings for Access Management @@ -83,8 +98,8 @@ Oracle strongly recommends that you ensure your system meets the following [Prer This is the default mode, in which OraOperator is deployed to operate in a cluster, and to monitor all the namespaces in the cluster. - - Grant the `serviceaccount:oracle-database-operator-system:default` cluster wide access for the resources by applying [cluster-role-binding.yaml](./rbac/cluster-role-binding.yaml) - + - Grant the `serviceaccount:oracle-database-operator-system:default` clusterwide access for the resources by applying [cluster-role-binding.yaml](./rbac/cluster-role-binding.yaml) + ```sh kubectl apply -f rbac/cluster-role-binding.yaml ``` @@ -97,31 +112,30 @@ Oracle strongly recommends that you ensure your system meets the following [Prer ##### 2. Namespace Scoped Deployment - In this mode, OraOperator can be deployed to operate in a namespace, and to monitor one or many namespaces. + In this mode, `OraOperator` can be deployed to operate in a namespace, and to monitor one or many namespaces. - - Grant `serviceaccount:oracle-database-operator-system:default` service account with resource access in the required namespaces. For example, to monitor only the default namespace, apply the [default-ns-role-binding.yaml](./rbac/default-ns-role-binding.yaml) + - Grant `serviceaccount:oracle-database-operator-system:default` service account with resource access in the required namespaces. For example, to monitor only the default namespace, apply the [`default-ns-role-binding.yaml`](./rbac/default-ns-role-binding.yaml) ```sh kubectl apply -f rbac/default-ns-role-binding.yaml ``` To watch additional namespaces, create different role binding files for each namespace, using [default-ns-role-binding.yaml](./rbac/default-ns-role-binding.yaml) as a template, and changing the `metadata.name` and `metadata.namespace` fields - - Next, edit the [oracle-database-operator.yaml](./oracle-database-operator.yaml) to add the required namespaces under `WATCH_NAMESPACE`. Use comma-delimited values for multiple namespaces. + - Next, edit the [`oracle-database-operator.yaml`](./oracle-database-operator.yaml) to add the required namespaces under `WATCH_NAMESPACE`. Use comma-delimited values for multiple namespaces. ```sh - name: WATCH_NAMESPACE value: "default" ``` - - Finally, apply the edited [oracle-database-operator.yaml](./oracle-database-operator.yaml) to deploy the Operator + - Finally, apply the edited [`oracle-database-operator.yaml`](./oracle-database-operator.yaml) to deploy the Operator ```sh kubectl apply -f oracle-database-operator.yaml ``` - * ### ClusterRole and ClusterRoleBinding for NodePort services - To expose services on each node's IP and port (the NodePort) apply the [node-rbac.yaml](./rbac/node-rbac.yaml). Note that this step is not required for LoadBalancer services. + To expose services on each node's IP and port (the NodePort), apply the [`node-rbac.yaml`](./rbac/node-rbac.yaml). Note that this step is not required for LoadBalancer services. ```sh kubectl apply -f rbac/node-rbac.yaml @@ -137,11 +151,21 @@ Oracle strongly recommends that you ensure your system meets the following [Prer kubectl apply -f oracle-database-operator.yaml ``` - Ensure that the operator pods are up and running. For high availability, Operator pod replicas are set to a default of 3. You can scale this setting up or down. +## Install Oracle DB Operator + + After you have completed the preceding prerequisite changes, you can install the operator. To install the operator in the cluster quickly, you can apply the modified `oracle-database-operator.yaml` file from the preceding step. + + Run the following command + + ```sh + kubectl apply -f oracle-database-operator.yaml + ``` + + Ensure that the operator pods are up and running. For high availability, operator pod replicas are set to a default of 3. You can scale this setting up or down. ```sh $ kubectl get pods -n oracle-database-operator-system - + NAME READY STATUS RESTARTS AGE pod/oracle-database-operator-controller-manager-78666fdddb-s4xcm 1/1 Running 0 11d pod/oracle-database-operator-controller-manager-78666fdddb-5k6n4 1/1 Running 0 11d @@ -164,7 +188,11 @@ The following quickstarts are designed for specific database configurations: * [Containerized Oracle Single Instance Database and Data Guard](./docs/sidb/README.md) * [Containerized Oracle Globally Distributed Database](./docs/sharding/README.md) * [Oracle Multitenant Database](./docs/multitenant/README.md) -* [Oracle Base Database Cloud Service (BDBCS)](./docs/dbcs/README.md) +* [Oracle Base Database Service (OBDS)](./docs/dbcs/README.md) + + +The following quickstart is designed for non-database configurations: +* [Oracle Database Observability](./docs/observability/README.md) The following quickstart is designed for non-database configurations: @@ -175,7 +203,7 @@ YAML file templates are available under [`/config/samples`](./config/samples/). ## Uninstall the Operator - To uninstall the operator, the final step consists of deciding whether you want to delete the custom resource definitions (CRDs) and Kubernetes APIServices introduced into the cluster by the operator. Choose one of the following options: + To uninstall the operator, the final step consists of deciding whether you want to delete the custom resource definitions (CRDs) and Kubernetes `APIServices` introduced into the cluster by the operator. Choose one of the following options: * ### Delete the CRDs and APIServices @@ -226,7 +254,7 @@ This project welcomes contributions from the community. Before submitting a pull ## Help -You can submit a GitHub issue, oir submit an issue and then file an [Oracle Support service](https://support.oracle.com/portal/) request. To file an issue or a service request, use the following product ID: 14430. +You can submit a GitHub issue, or submit an issue and then file an [Oracle Support service](https://support.oracle.com/portal/) request. To file an issue or a service request, use the following product ID: 14430. ## Security @@ -243,9 +271,9 @@ The following is an example of a YAML file fragment for specifying Oracle Cloud ociSecretOCID: ocid1.vaultsecret.oc1... ``` -Examples in this repository where passwords are entered on the command line are for demonstration purposes only. +Examples in this repository where passwords are entered on the command line are for demonstration purposes only. ## License -Copyright (c) 2022, 2024 Oracle and/or its affiliates. +Copyright (c) 2022, 2025 Oracle and/or its affiliates. Released under the Universal Permissive License v1.0 as shown at [https://oss.oracle.com/licenses/upl/](https://oss.oracle.com/licenses/upl/) diff --git a/THIRD_PARTY_LICENSES.txt b/THIRD_PARTY_LICENSES.txt index d75f3946..14e4308f 100644 --- a/THIRD_PARTY_LICENSES.txt +++ b/THIRD_PARTY_LICENSES.txt @@ -1,11 +1,11 @@ ------------------------------------- -Operator SDK 1.32.0 +Operator SDK 1.37.0 https://github.com/operator-framework/operator-sdk Apache 2.0 ------------------------------------- -Apache License: +Apache License: Version 2.0, January 2004 http://www.apache.org/licenses/ @@ -206,13 +206,13 @@ Apache License: WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. - + ------------------------------ - GO lang 1.21.4 + GO lang 1.23.3 https://github.com/golang - Copyright (c) 2009 The Go Authors. + Copyright (c) 2009 The Go Authors. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are @@ -241,22 +241,20 @@ THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. ------------------------- -apimachinery 0.28.4 +apimachinery 0.31.3 https://github.com/kubernetes/apimachinery/tr Apache 2.0 ------------------------- -controller-runtime 0.16.3 +controller-runtime 0.19.3 https://github.com/kubernetes-sigs/controller-runtime/releases/tag/v0.16.3 Apache 2.0 ------------------------- -golang 1.21.4 +golang 1.23.3 https://github.com/golang/go/releases/tag/go1.21.4 -BSD 2-clause or 3-clause - -BSD 2-clause or 3-clause License: +BSD 2-clause or 3-clause License: Copyright (c) 2009 The Go Authors. All rights reserved. @@ -1008,23 +1006,23 @@ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. limitations under the License. -------------------------- -Logr 1.3.0 +Logr 1.4.2 https://pkg.go.dev/github.com/go-logr/logr https://github.com/go-logr/logr/tree/v1.3.0 Apache 2.0 License ------------------------- -OCI Go SDK 65.53.0 +OCI Go SDK 65.77.1 https://github.com/oracle/oci-go-sdk/releases/tag/v65.53.0 Dual-License: UPL + Apache 2.0 -UPL license: +UPL license: -Copyright (c) 2016, 2018, 2020, Oracle and/or its affiliates. -This software is dual-licensed to you under the Universal Permissive License (UPL) 1.0 +Copyright (c) 2016, 2018, 2020, Oracle and/or its affiliates. +This software is dual-licensed to you under the Universal Permissive License (UPL) 1.0 as shown at https://oss.oracle.com/licenses/upl -or Apache License 2.0 as shown at http://www.apache.org/licenses/LICENSE-2.0. +or Apache License 2.0 as shown at http://www.apache.org/licenses/LICENSE-2.0. You may choose either license. Copyright (c) 2019, 2020 Oracle and/or its affiliates. @@ -1065,58 +1063,58 @@ The Universal Permissive License (UPL), Version 1.0 ------------------------- -ginkgo 2.13.1 +ginkgo 2.202. https://github.com/onsi/ginkgo/releases/tag/v2.13.1 MIT ------------------------------------ Gomega github.com/onsi/gomega -MIT License +MIT License Copyright (c) 2013-2014 Onsi Fakhouri ---------------------------- -gomega 1.30.0 +gomega 1.34.2 http://onsi.github.io/gomega/ MIT ------------------------- -Kubernetes api 0.28.4 +Kubernetes api 0.31.3 https://pkg.go.dev/k8s.io/api Apache 2.0 ---------------------------------- -Kubernetes apimachinery 0.28.4 +Kubernetes apimachinery 0.31.3 https://pkg.go.dev/k8s.io/apimachinery Apache 2.0 ----------------------------------- -Kubernetes client-go 0.28.4 +Kubernetes client-go 0.31.3 https://pkg.go.dev/k8s.io/client-go Apache 2.0 ------------------------------------- -Kubernetes controller-runtime project 0.16.3 +Kubernetes controller-runtime project 0.19.3 https://pkg.go.dev/sigs.k8s.io/controller-runtime Apache 2.0 ------------------------------------ kubernetes-sigs/yaml 1.4.0 https://github.com/kubernetes-sigs/yaml/tree/v1.3.0 -MIT +MIT ------------------------- -OCI SDK for Go 65.53.0 +OCI SDK for Go 65.77.1 https://github.com/oracle/oci-go-sdk Multiple Licenses: Apache 2.0, UPL ------------------------------ -Operator Lifecycle Manager (OLM) +Operator Lifecycle Manager (OLM) 0.30.0 github.com/operator-framework/operator-lifecycle-manager Apache 2.0 ------------------------------------ -Prometheus Operator 0.65.2 +Prometheus Operator 0.78.2 https://github.com/prometheus-operator/prometheus-operator Apache 2.0 @@ -1135,8 +1133,8 @@ https://pkg.go.dev/sigs.k8s.io/yaml Dual license: BSD-3-Clause, MIT ------------------------------------ -zap 1.26.0 -https://github.com/uber-go/zap/releases/tag/v1.26.0 +zap 1.27.0 +https://github.com/uber-go/zap/releases/tag/v1.27.0 MIT ------------------------------------ @@ -1163,7 +1161,7 @@ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. -Copyright (c) 2012 The Go Authors. +Copyright (c) 2012 The Go Authors. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are @@ -1192,9 +1190,9 @@ THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. ------------------------ -Ginkgo 2.13.1 +Ginkgo 2.20.2 github.com/onsi/ginkgo -MIT License +MIT License Copyright (c) 2013-2014 Onsi Fakhouri Permission is hereby granted, free of charge, to any person obtaining @@ -1216,4 +1214,3 @@ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. --------------------------- - diff --git a/apis/database/v1alpha1/adbfamily_common_spec.go b/apis/database/v1alpha1/adbfamily_common_spec.go new file mode 100644 index 00000000..74eb9f94 --- /dev/null +++ b/apis/database/v1alpha1/adbfamily_common_spec.go @@ -0,0 +1,67 @@ +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v1alpha1 + +// LastSuccessfulSpec is an annotation key which maps to the value of last successful spec +const LastSuccessfulSpec string = "lastSuccessfulSpec" + +/************************ +* OCI config +************************/ +type OciConfigSpec struct { + ConfigMapName *string `json:"configMapName,omitempty"` + SecretName *string `json:"secretName,omitempty"` +} + +/************************ +* ADB spec +************************/ +type K8sAdbSpec struct { + Name *string `json:"name,omitempty"` +} + +type OciAdbSpec struct { + Ocid *string `json:"ocid,omitempty"` +} + +// TargetSpec defines the spec of the target for backup/restore runs. +type TargetSpec struct { + K8sAdb K8sAdbSpec `json:"k8sADB,omitempty"` + OciAdb OciAdbSpec `json:"ociADB,omitempty"` +} diff --git a/apis/database/v1alpha1/autonomouscontainerdatabase_types.go b/apis/database/v1alpha1/autonomouscontainerdatabase_types.go index abf0ee0d..fd71b210 100644 --- a/apis/database/v1alpha1/autonomouscontainerdatabase_types.go +++ b/apis/database/v1alpha1/autonomouscontainerdatabase_types.go @@ -42,6 +42,8 @@ import ( "encoding/json" "reflect" + dbv4 "github.com/oracle/oracle-database-operator/apis/database/v4" + "github.com/oracle/oci-go-sdk/v65/database" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" ) @@ -60,6 +62,17 @@ const ( AcdActionTerminate AcdActionEnum = "TERMINATE" ) +func GetAcdActionEnumFromString(val string) (AcdActionEnum, bool) { + var mappingAcdActionEnum = map[string]AcdActionEnum{ + "RESTART": AcdActionRestart, + "TERMINATE": AcdActionTerminate, + "": AcdActionBlank, + } + + enum, ok := mappingAcdActionEnum[val] + return enum, ok +} + // AutonomousContainerDatabaseSpec defines the desired state of AutonomousContainerDatabase type AutonomousContainerDatabaseSpec struct { // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster @@ -74,7 +87,7 @@ type AutonomousContainerDatabaseSpec struct { Action AcdActionEnum `json:"action,omitempty"` FreeformTags map[string]string `json:"freeformTags,omitempty"` - OCIConfig OCIConfigSpec `json:"ociConfig,omitempty"` + OCIConfig OciConfigSpec `json:"ociConfig,omitempty"` // +kubebuilder:default:=false HardLink *bool `json:"hardLink,omitempty"` } @@ -157,13 +170,13 @@ func (acd *AutonomousContainerDatabase) UpdateLastSuccessfulSpec() error { } // UpdateStatusFromOCIACD updates the status subresource -func (acd *AutonomousContainerDatabase) UpdateStatusFromOCIACD(ociObj database.AutonomousContainerDatabase) { +func (acd *AutonomousContainerDatabase) UpdateStatusFromOciAcd(ociObj database.AutonomousContainerDatabase) { acd.Status.LifecycleState = ociObj.LifecycleState - acd.Status.TimeCreated = FormatSDKTime(ociObj.TimeCreated) + acd.Status.TimeCreated = dbv4.FormatSDKTime(ociObj.TimeCreated) } // UpdateFromOCIADB updates the attributes using database.AutonomousContainerDatabase object -func (acd *AutonomousContainerDatabase) UpdateFromOCIACD(ociObj database.AutonomousContainerDatabase) (specChanged bool) { +func (acd *AutonomousContainerDatabase) UpdateFromOciAcd(ociObj database.AutonomousContainerDatabase) (specChanged bool) { oldACD := acd.DeepCopy() /*********************************** @@ -186,14 +199,14 @@ func (acd *AutonomousContainerDatabase) UpdateFromOCIACD(ociObj database.Autonom /*********************************** * update the status subresource ***********************************/ - acd.UpdateStatusFromOCIACD(ociObj) + acd.UpdateStatusFromOciAcd(ociObj) return !reflect.DeepEqual(oldACD.Spec, acd.Spec) } // RemoveUnchangedSpec removes the unchanged fields in spec, and returns if the spec has been changed. func (acd *AutonomousContainerDatabase) RemoveUnchangedSpec(prevSpec AutonomousContainerDatabaseSpec) (bool, error) { - changed, err := removeUnchangedFields(prevSpec, &acd.Spec) + changed, err := dbv4.RemoveUnchangedFields(prevSpec, &acd.Spec) if err != nil { return changed, err } diff --git a/apis/database/v1alpha1/autonomouscontainerdatabase_webhook.go b/apis/database/v1alpha1/autonomouscontainerdatabase_webhook.go index 37e1d819..10a16cd1 100644 --- a/apis/database/v1alpha1/autonomouscontainerdatabase_webhook.go +++ b/apis/database/v1alpha1/autonomouscontainerdatabase_webhook.go @@ -39,6 +39,7 @@ package v1alpha1 import ( + dbv4 "github.com/oracle/oracle-database-operator/apis/database/v4" apierrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/runtime" "k8s.io/apimachinery/pkg/runtime/schema" @@ -58,15 +59,13 @@ func (r *AutonomousContainerDatabase) SetupWebhookWithManager(mgr ctrl.Manager) Complete() } -//+kubebuilder:webhook:verbs=create;update,path=/validate-database-oracle-com-v1alpha1-autonomouscontainerdatabase,mutating=false,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=autonomouscontainerdatabases,versions=v1alpha1,name=vautonomouscontainerdatabase.kb.io,admissionReviewVersions={v1} +//+kubebuilder:webhook:verbs=create;update,path=/validate-database-oracle-com-v1alpha1-autonomouscontainerdatabase,mutating=false,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=autonomouscontainerdatabases,versions=v1alpha1,name=vautonomouscontainerdatabasev1alpha1.kb.io,admissionReviewVersions=v1 var _ webhook.Validator = &AutonomousContainerDatabase{} // ValidateCreate implements webhook.Validator so a webhook will be registered for the type func (r *AutonomousContainerDatabase) ValidateCreate() (admission.Warnings, error) { autonomouscontainerdatabaselog.Info("validate create", "name", r.Name) - - // TODO(user): fill in your validation logic upon object creation. return nil, nil } @@ -84,12 +83,12 @@ func (r *AutonomousContainerDatabase) ValidateUpdate(old runtime.Object) (admiss // cannot update when the old state is in intermediate state, except for the terminate operatrion var copiedSpec *AutonomousContainerDatabaseSpec = r.Spec.DeepCopy() - changed, err := removeUnchangedFields(oldACD.Spec, copiedSpec) + changed, err := dbv4.RemoveUnchangedFields(oldACD.Spec, copiedSpec) if err != nil { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec"), err.Error())) } - if IsACDIntermediateState(oldACD.Status.LifecycleState) && changed { + if dbv4.IsACDIntermediateState(oldACD.Status.LifecycleState) && changed { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec"), "cannot change the spec when the lifecycleState is in an intermdeiate state")) diff --git a/apis/database/v1alpha1/autonomousdatabase_conversion.go b/apis/database/v1alpha1/autonomousdatabase_conversion.go new file mode 100644 index 00000000..ffccc181 --- /dev/null +++ b/apis/database/v1alpha1/autonomousdatabase_conversion.go @@ -0,0 +1,371 @@ +package v1alpha1 + +import ( + "errors" + + v4 "github.com/oracle/oracle-database-operator/apis/database/v4" + "sigs.k8s.io/controller-runtime/pkg/conversion" +) + +// ConvertTo converts this AutonomousDatabase to the Hub version (v4). +func (src *AutonomousDatabase) ConvertTo(dstRaw conversion.Hub) error { + dst := dstRaw.(*v4.AutonomousDatabase) + // Convert the Spec + dst.Spec.Action = src.Spec.Action + + // Details + dst.Spec.Details.Id = src.Spec.Details.Id + dst.Spec.Details.CompartmentId = src.Spec.Details.CompartmentId + dst.Spec.Details.AutonomousContainerDatabase.K8sAcd.Name = src.Spec.Details.AutonomousContainerDatabase.K8sAcd.Name + dst.Spec.Details.AutonomousContainerDatabase.OciAcd.Id = src.Spec.Details.AutonomousContainerDatabase.OciAcd.Id + dst.Spec.Details.DisplayName = src.Spec.Details.DisplayName + dst.Spec.Details.DbName = src.Spec.Details.DbName + dst.Spec.Details.DbWorkload = src.Spec.Details.DbWorkload + dst.Spec.Details.LicenseModel = src.Spec.Details.LicenseModel + dst.Spec.Details.DbVersion = src.Spec.Details.DbVersion + dst.Spec.Details.DataStorageSizeInTBs = src.Spec.Details.DataStorageSizeInTBs + dst.Spec.Details.CpuCoreCount = src.Spec.Details.CpuCoreCount + dst.Spec.Details.ComputeModel = src.Spec.Details.ComputeModel + dst.Spec.Details.ComputeCount = src.Spec.Details.ComputeCount + dst.Spec.Details.OcpuCount = src.Spec.Details.OcpuCount + dst.Spec.Details.AdminPassword.K8sSecret.Name = src.Spec.Details.AdminPassword.K8sSecret.Name + dst.Spec.Details.AdminPassword.OciSecret.Id = src.Spec.Details.AdminPassword.OciSecret.Id + dst.Spec.Details.IsAutoScalingEnabled = src.Spec.Details.IsAutoScalingEnabled + dst.Spec.Details.IsDedicated = src.Spec.Details.IsDedicated + dst.Spec.Details.IsFreeTier = src.Spec.Details.IsFreeTier + dst.Spec.Details.IsAccessControlEnabled = src.Spec.Details.IsAccessControlEnabled + dst.Spec.Details.WhitelistedIps = src.Spec.Details.WhitelistedIps + dst.Spec.Details.SubnetId = src.Spec.Details.SubnetId + dst.Spec.Details.NsgIds = src.Spec.Details.NsgIds + dst.Spec.Details.PrivateEndpointLabel = src.Spec.Details.PrivateEndpointLabel + dst.Spec.Details.IsMtlsConnectionRequired = src.Spec.Details.IsMtlsConnectionRequired + dst.Spec.Details.FreeformTags = src.Spec.Details.FreeformTags + + // Clone + dst.Spec.Clone.CompartmentId = src.Spec.Clone.CompartmentId + dst.Spec.Clone.AutonomousContainerDatabase.K8sAcd.Name = src.Spec.Clone.AutonomousContainerDatabase.K8sAcd.Name + dst.Spec.Clone.AutonomousContainerDatabase.OciAcd.Id = src.Spec.Clone.AutonomousContainerDatabase.OciAcd.Id + dst.Spec.Clone.DisplayName = src.Spec.Clone.DisplayName + dst.Spec.Clone.DbName = src.Spec.Clone.DbName + dst.Spec.Clone.DbWorkload = src.Spec.Clone.DbWorkload + dst.Spec.Clone.LicenseModel = src.Spec.Clone.LicenseModel + dst.Spec.Clone.DbVersion = src.Spec.Clone.DbVersion + dst.Spec.Clone.DataStorageSizeInTBs = src.Spec.Clone.DataStorageSizeInTBs + dst.Spec.Clone.CpuCoreCount = src.Spec.Clone.CpuCoreCount + dst.Spec.Clone.ComputeModel = src.Spec.Clone.ComputeModel + dst.Spec.Clone.ComputeCount = src.Spec.Clone.ComputeCount + dst.Spec.Clone.OcpuCount = src.Spec.Clone.OcpuCount + dst.Spec.Clone.AdminPassword.K8sSecret.Name = src.Spec.Clone.AdminPassword.K8sSecret.Name + dst.Spec.Clone.AdminPassword.OciSecret.Id = src.Spec.Clone.AdminPassword.OciSecret.Id + dst.Spec.Clone.IsAutoScalingEnabled = src.Spec.Clone.IsAutoScalingEnabled + dst.Spec.Clone.IsDedicated = src.Spec.Clone.IsDedicated + dst.Spec.Clone.IsFreeTier = src.Spec.Clone.IsFreeTier + dst.Spec.Clone.IsAccessControlEnabled = src.Spec.Clone.IsAccessControlEnabled + dst.Spec.Clone.WhitelistedIps = src.Spec.Clone.WhitelistedIps + dst.Spec.Clone.SubnetId = src.Spec.Clone.SubnetId + dst.Spec.Clone.NsgIds = src.Spec.Clone.NsgIds + dst.Spec.Clone.PrivateEndpointLabel = src.Spec.Clone.PrivateEndpointLabel + dst.Spec.Clone.IsMtlsConnectionRequired = src.Spec.Clone.IsMtlsConnectionRequired + dst.Spec.Clone.FreeformTags = src.Spec.Clone.FreeformTags + dst.Spec.Clone.CloneType = src.Spec.Clone.CloneType + + // Wallet + dst.Spec.Wallet.Name = src.Spec.Wallet.Name + dst.Spec.Wallet.Password.K8sSecret.Name = src.Spec.Wallet.Password.K8sSecret.Name + dst.Spec.Wallet.Password.OciSecret.Id = src.Spec.Wallet.Password.OciSecret.Id + + dst.Spec.OciConfig.ConfigMapName = src.Spec.OciConfig.ConfigMapName + dst.Spec.OciConfig.SecretName = src.Spec.OciConfig.SecretName + + dst.Spec.HardLink = src.Spec.HardLink + + // Convert the Status + dst.Status.LifecycleState = src.Status.LifecycleState + dst.Status.TimeCreated = src.Status.TimeCreated + dst.Status.WalletExpiringDate = src.Status.WalletExpiringDate + + // convert status.allConnectionStrings + if src.Status.AllConnectionStrings != nil { + for _, srcProfile := range src.Status.AllConnectionStrings { + dstProfile := v4.ConnectionStringProfile{} + + // convert status.allConnectionStrings[i].tlsAuthentication + if val, ok := v4.GetTLSAuthenticationEnumFromString(string(srcProfile.TLSAuthentication)); !ok { + return errors.New("Unable to convert to TLSAuthenticationEnum: " + string(srcProfile.TLSAuthentication)) + } else { + dstProfile.TLSAuthentication = val + } + + // convert status.allConnectionStrings[i].connectionStrings + dstProfile.ConnectionStrings = make([]v4.ConnectionStringSpec, len(srcProfile.ConnectionStrings)) + for i, v := range srcProfile.ConnectionStrings { + dstProfile.ConnectionStrings[i].TNSName = v.TNSName + dstProfile.ConnectionStrings[i].ConnectionString = v.ConnectionString + } + + dst.Status.AllConnectionStrings = append(dst.Status.AllConnectionStrings, dstProfile) + } + } + + dst.Status.Conditions = src.Status.Conditions + + dst.ObjectMeta = src.ObjectMeta + return nil +} + +// ConvertFrom converts from the Hub version (v4) to v1alpha1 +func (dst *AutonomousDatabase) ConvertFrom(srcRaw conversion.Hub) error { + src := srcRaw.(*v4.AutonomousDatabase) + + // Convert the Spec + dst.Spec.Action = src.Spec.Action + + // Details + dst.Spec.Details.Id = src.Spec.Details.Id + dst.Spec.Details.CompartmentId = src.Spec.Details.CompartmentId + dst.Spec.Details.AutonomousContainerDatabase.K8sAcd.Name = src.Spec.Details.AutonomousContainerDatabase.K8sAcd.Name + dst.Spec.Details.AutonomousContainerDatabase.OciAcd.Id = src.Spec.Details.AutonomousContainerDatabase.OciAcd.Id + dst.Spec.Details.DisplayName = src.Spec.Details.DisplayName + dst.Spec.Details.DbName = src.Spec.Details.DbName + dst.Spec.Details.DbWorkload = src.Spec.Details.DbWorkload + dst.Spec.Details.LicenseModel = src.Spec.Details.LicenseModel + dst.Spec.Details.DbVersion = src.Spec.Details.DbVersion + dst.Spec.Details.DataStorageSizeInTBs = src.Spec.Details.DataStorageSizeInTBs + dst.Spec.Details.CpuCoreCount = src.Spec.Details.CpuCoreCount + dst.Spec.Details.ComputeModel = src.Spec.Details.ComputeModel + dst.Spec.Details.ComputeCount = src.Spec.Details.ComputeCount + dst.Spec.Details.OcpuCount = src.Spec.Details.OcpuCount + dst.Spec.Details.AdminPassword.K8sSecret.Name = src.Spec.Details.AdminPassword.K8sSecret.Name + dst.Spec.Details.AdminPassword.OciSecret.Id = src.Spec.Details.AdminPassword.OciSecret.Id + dst.Spec.Details.IsAutoScalingEnabled = src.Spec.Details.IsAutoScalingEnabled + dst.Spec.Details.IsDedicated = src.Spec.Details.IsDedicated + dst.Spec.Details.IsFreeTier = src.Spec.Details.IsFreeTier + dst.Spec.Details.IsAccessControlEnabled = src.Spec.Details.IsAccessControlEnabled + dst.Spec.Details.WhitelistedIps = src.Spec.Details.WhitelistedIps + dst.Spec.Details.SubnetId = src.Spec.Details.SubnetId + dst.Spec.Details.NsgIds = src.Spec.Details.NsgIds + dst.Spec.Details.PrivateEndpointLabel = src.Spec.Details.PrivateEndpointLabel + dst.Spec.Details.IsMtlsConnectionRequired = src.Spec.Details.IsMtlsConnectionRequired + dst.Spec.Details.FreeformTags = src.Spec.Details.FreeformTags + + // Clone + dst.Spec.Clone.CompartmentId = src.Spec.Clone.CompartmentId + dst.Spec.Clone.AutonomousContainerDatabase.K8sAcd.Name = src.Spec.Clone.AutonomousContainerDatabase.K8sAcd.Name + dst.Spec.Clone.AutonomousContainerDatabase.OciAcd.Id = src.Spec.Clone.AutonomousContainerDatabase.OciAcd.Id + dst.Spec.Clone.DisplayName = src.Spec.Clone.DisplayName + dst.Spec.Clone.DbName = src.Spec.Clone.DbName + dst.Spec.Clone.DbWorkload = src.Spec.Clone.DbWorkload + dst.Spec.Clone.LicenseModel = src.Spec.Clone.LicenseModel + dst.Spec.Clone.DbVersion = src.Spec.Clone.DbVersion + dst.Spec.Clone.DataStorageSizeInTBs = src.Spec.Clone.DataStorageSizeInTBs + dst.Spec.Clone.CpuCoreCount = src.Spec.Clone.CpuCoreCount + dst.Spec.Clone.ComputeModel = src.Spec.Clone.ComputeModel + dst.Spec.Clone.ComputeCount = src.Spec.Clone.ComputeCount + dst.Spec.Clone.OcpuCount = src.Spec.Clone.OcpuCount + dst.Spec.Clone.AdminPassword.K8sSecret.Name = src.Spec.Clone.AdminPassword.K8sSecret.Name + dst.Spec.Clone.AdminPassword.OciSecret.Id = src.Spec.Clone.AdminPassword.OciSecret.Id + dst.Spec.Clone.IsAutoScalingEnabled = src.Spec.Clone.IsAutoScalingEnabled + dst.Spec.Clone.IsDedicated = src.Spec.Clone.IsDedicated + dst.Spec.Clone.IsFreeTier = src.Spec.Clone.IsFreeTier + dst.Spec.Clone.IsAccessControlEnabled = src.Spec.Clone.IsAccessControlEnabled + dst.Spec.Clone.WhitelistedIps = src.Spec.Clone.WhitelistedIps + dst.Spec.Clone.SubnetId = src.Spec.Clone.SubnetId + dst.Spec.Clone.NsgIds = src.Spec.Clone.NsgIds + dst.Spec.Clone.PrivateEndpointLabel = src.Spec.Clone.PrivateEndpointLabel + dst.Spec.Clone.IsMtlsConnectionRequired = src.Spec.Clone.IsMtlsConnectionRequired + dst.Spec.Clone.FreeformTags = src.Spec.Clone.FreeformTags + dst.Spec.Clone.CloneType = src.Spec.Clone.CloneType + + // Wallet + dst.Spec.Wallet.Name = src.Spec.Wallet.Name + dst.Spec.Wallet.Password.K8sSecret.Name = src.Spec.Wallet.Password.K8sSecret.Name + dst.Spec.Wallet.Password.OciSecret.Id = src.Spec.Wallet.Password.OciSecret.Id + + dst.Spec.OciConfig.ConfigMapName = src.Spec.OciConfig.ConfigMapName + dst.Spec.OciConfig.SecretName = src.Spec.OciConfig.SecretName + + dst.Spec.HardLink = src.Spec.HardLink + + // Convert the Status + dst.Status.LifecycleState = src.Status.LifecycleState + dst.Status.TimeCreated = src.Status.TimeCreated + dst.Status.WalletExpiringDate = src.Status.WalletExpiringDate + + // convert status.allConnectionStrings + if src.Status.AllConnectionStrings != nil { + for _, srcProfile := range src.Status.AllConnectionStrings { + dstProfile := ConnectionStringProfile{} + + // convert status.allConnectionStrings[i].tlsAuthentication + if val, ok := GetTLSAuthenticationEnumFromString(string(srcProfile.TLSAuthentication)); !ok { + return errors.New("Unable to convert to TLSAuthenticationEnum: " + string(srcProfile.TLSAuthentication)) + } else { + dstProfile.TLSAuthentication = val + } + + // convert status.allConnectionStrings[i].connectionStrings + dstProfile.ConnectionStrings = make([]ConnectionStringSpec, len(srcProfile.ConnectionStrings)) + for i, v := range srcProfile.ConnectionStrings { + dstProfile.ConnectionStrings[i].TNSName = v.TNSName + dstProfile.ConnectionStrings[i].ConnectionString = v.ConnectionString + } + + dst.Status.AllConnectionStrings = append(dst.Status.AllConnectionStrings, dstProfile) + } + } + + dst.Status.Conditions = src.Status.Conditions + + dst.ObjectMeta = src.ObjectMeta + return nil +} + +func (src *AutonomousDatabaseBackup) ConvertTo(dstRaw conversion.Hub) error { + dst := dstRaw.(*v4.AutonomousDatabaseBackup) + + dst.Spec.Target.K8sAdb.Name = src.Spec.Target.K8sAdb.Name + dst.Spec.Target.OciAdb.OCID = src.Spec.Target.OciAdb.Ocid + dst.Spec.DisplayName = src.Spec.DisplayName + dst.Spec.AutonomousDatabaseBackupOCID = src.Spec.AutonomousDatabaseBackupOCID + dst.Spec.IsLongTermBackup = src.Spec.IsLongTermBackup + dst.Spec.RetentionPeriodInDays = src.Spec.RetentionPeriodInDays + dst.Spec.OCIConfig.ConfigMapName = src.Spec.OCIConfig.ConfigMapName + dst.Spec.OCIConfig.SecretName = src.Spec.OCIConfig.SecretName + + dst.Status.LifecycleState = src.Status.LifecycleState + dst.Status.Type = src.Status.Type + dst.Status.IsAutomatic = src.Status.IsAutomatic + dst.Status.TimeStarted = src.Status.TimeStarted + dst.Status.TimeEnded = src.Status.TimeEnded + dst.Status.AutonomousDatabaseOCID = src.Status.AutonomousDatabaseOCID + dst.Status.CompartmentOCID = src.Status.CompartmentOCID + dst.Status.DBName = src.Status.DBName + dst.Status.DBDisplayName = src.Status.DBDisplayName + + dst.ObjectMeta = src.ObjectMeta + return nil +} + +func (dst *AutonomousDatabaseBackup) ConvertFrom(srcRaw conversion.Hub) error { + src := srcRaw.(*v4.AutonomousDatabaseBackup) + + dst.Spec.Target.K8sAdb.Name = src.Spec.Target.K8sAdb.Name + dst.Spec.Target.OciAdb.Ocid = src.Spec.Target.OciAdb.OCID + dst.Spec.DisplayName = src.Spec.DisplayName + dst.Spec.AutonomousDatabaseBackupOCID = src.Spec.AutonomousDatabaseBackupOCID + dst.Spec.IsLongTermBackup = src.Spec.IsLongTermBackup + dst.Spec.RetentionPeriodInDays = src.Spec.RetentionPeriodInDays + dst.Spec.OCIConfig.ConfigMapName = src.Spec.OCIConfig.ConfigMapName + dst.Spec.OCIConfig.SecretName = src.Spec.OCIConfig.SecretName + + dst.Status.LifecycleState = src.Status.LifecycleState + dst.Status.Type = src.Status.Type + dst.Status.IsAutomatic = src.Status.IsAutomatic + dst.Status.TimeStarted = src.Status.TimeStarted + dst.Status.TimeEnded = src.Status.TimeEnded + dst.Status.AutonomousDatabaseOCID = src.Status.AutonomousDatabaseOCID + dst.Status.CompartmentOCID = src.Status.CompartmentOCID + dst.Status.DBName = src.Status.DBName + dst.Status.DBDisplayName = src.Status.DBDisplayName + + dst.ObjectMeta = src.ObjectMeta + return nil +} + +func (src *AutonomousDatabaseRestore) ConvertTo(dstRaw conversion.Hub) error { + dst := dstRaw.(*v4.AutonomousDatabaseRestore) + + dst.Spec.Target.K8sAdb.Name = src.Spec.Target.K8sAdb.Name + dst.Spec.Target.OciAdb.OCID = src.Spec.Target.OciAdb.Ocid + dst.Spec.Source.K8sAdbBackup.Name = src.Spec.Source.K8sAdbBackup.Name + dst.Spec.Source.PointInTime.Timestamp = src.Spec.Source.PointInTime.Timestamp + dst.Spec.OCIConfig.ConfigMapName = src.Spec.OCIConfig.ConfigMapName + dst.Spec.OCIConfig.SecretName = src.Spec.OCIConfig.SecretName + + dst.Status.DisplayName = src.Status.DisplayName + dst.Status.TimeAccepted = src.Status.TimeAccepted + dst.Status.TimeStarted = src.Status.TimeStarted + dst.Status.TimeEnded = src.Status.TimeEnded + dst.Status.DbName = src.Status.DbName + dst.Status.WorkRequestOCID = src.Status.WorkRequestOCID + dst.Status.Status = src.Status.Status + + dst.ObjectMeta = src.ObjectMeta + return nil +} + +func (dst *AutonomousDatabaseRestore) ConvertFrom(srcRaw conversion.Hub) error { + src := srcRaw.(*v4.AutonomousDatabaseRestore) + + dst.Spec.Target.K8sAdb.Name = src.Spec.Target.K8sAdb.Name + dst.Spec.Target.OciAdb.Ocid = src.Spec.Target.OciAdb.OCID + dst.Spec.Source.K8sAdbBackup.Name = src.Spec.Source.K8sAdbBackup.Name + dst.Spec.Source.PointInTime.Timestamp = src.Spec.Source.PointInTime.Timestamp + dst.Spec.OCIConfig.ConfigMapName = src.Spec.OCIConfig.ConfigMapName + dst.Spec.OCIConfig.SecretName = src.Spec.OCIConfig.SecretName + + dst.Status.DisplayName = src.Status.DisplayName + dst.Status.TimeAccepted = src.Status.TimeAccepted + dst.Status.TimeStarted = src.Status.TimeStarted + dst.Status.TimeEnded = src.Status.TimeEnded + dst.Status.DbName = src.Status.DbName + dst.Status.WorkRequestOCID = src.Status.WorkRequestOCID + dst.Status.Status = src.Status.Status + + dst.ObjectMeta = src.ObjectMeta + return nil +} + +func (src *AutonomousContainerDatabase) ConvertTo(dstRaw conversion.Hub) error { + dst := dstRaw.(*v4.AutonomousContainerDatabase) + + dst.Spec.AutonomousContainerDatabaseOCID = src.Spec.AutonomousContainerDatabaseOCID + dst.Spec.CompartmentOCID = src.Spec.CompartmentOCID + dst.Spec.DisplayName = src.Spec.DisplayName + dst.Spec.AutonomousExadataVMClusterOCID = src.Spec.AutonomousExadataVMClusterOCID + dst.Spec.PatchModel = src.Spec.PatchModel + + if val, ok := v4.GetAcdActionEnumFromString(string(src.Spec.Action)); !ok { + return errors.New("Unable to convert to AcdActionEnum: " + string(src.Spec.Action)) + } else { + dst.Spec.Action = val + } + + dst.Spec.FreeformTags = src.Spec.FreeformTags + dst.Spec.OCIConfig.ConfigMapName = src.Spec.OCIConfig.ConfigMapName + dst.Spec.OCIConfig.SecretName = src.Spec.OCIConfig.SecretName + dst.Spec.HardLink = src.Spec.HardLink + + dst.Status.LifecycleState = src.Status.LifecycleState + dst.Status.TimeCreated = src.Status.TimeCreated + + dst.ObjectMeta = src.ObjectMeta + return nil +} + +func (dst *AutonomousContainerDatabase) ConvertFrom(srcRaw conversion.Hub) error { + src := srcRaw.(*v4.AutonomousContainerDatabase) + + dst.Spec.AutonomousContainerDatabaseOCID = src.Spec.AutonomousContainerDatabaseOCID + dst.Spec.CompartmentOCID = src.Spec.CompartmentOCID + dst.Spec.DisplayName = src.Spec.DisplayName + dst.Spec.AutonomousExadataVMClusterOCID = src.Spec.AutonomousExadataVMClusterOCID + dst.Spec.PatchModel = src.Spec.PatchModel + + if val, ok := GetAcdActionEnumFromString(string(src.Spec.Action)); !ok { + return errors.New("Unable to convert to AcdActionEnum: " + string(src.Spec.Action)) + } else { + dst.Spec.Action = val + } + + dst.Spec.FreeformTags = src.Spec.FreeformTags + dst.Spec.OCIConfig.ConfigMapName = src.Spec.OCIConfig.ConfigMapName + dst.Spec.OCIConfig.SecretName = src.Spec.OCIConfig.SecretName + dst.Spec.HardLink = src.Spec.HardLink + + dst.Status.LifecycleState = src.Status.LifecycleState + dst.Status.TimeCreated = src.Status.TimeCreated + + dst.ObjectMeta = src.ObjectMeta + return nil +} diff --git a/apis/database/v1alpha1/autonomousdatabase_types.go b/apis/database/v1alpha1/autonomousdatabase_types.go index cd23b3f3..099703c2 100644 --- a/apis/database/v1alpha1/autonomousdatabase_types.go +++ b/apis/database/v1alpha1/autonomousdatabase_types.go @@ -39,9 +39,6 @@ package v1alpha1 import ( - "encoding/json" - "reflect" - "github.com/oracle/oci-go-sdk/v65/database" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" ) @@ -49,34 +46,79 @@ import ( // EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! // NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized. -// name of our custom finalizer -const ADB_FINALIZER = "database.oracle.com/adb-finalizer" - // AutonomousDatabaseSpec defines the desired state of AutonomousDatabase // Important: Run "make" to regenerate code after modifying this file type AutonomousDatabaseSpec struct { - Details AutonomousDatabaseDetails `json:"details"` - OCIConfig OCIConfigSpec `json:"ociConfig,omitempty"` + // +kubebuilder:validation:Enum:="";Create;Sync;Update;Stop;Start;Terminate;Clone + Action string `json:"action"` + Details AutonomousDatabaseDetails `json:"details,omitempty"` + Clone AutonomousDatabaseClone `json:"clone,omitempty"` + Wallet WalletSpec `json:"wallet,omitempty"` + OciConfig OciConfigSpec `json:"ociConfig,omitempty"` // +kubebuilder:default:=false HardLink *bool `json:"hardLink,omitempty"` } +type AutonomousDatabaseDetails struct { + AutonomousDatabaseBase `json:",inline"` + Id *string `json:"id,omitempty"` +} + +type AutonomousDatabaseClone struct { + AutonomousDatabaseBase `json:",inline"` + // +kubebuilder:validation:Enum:="FULL";"METADATA" + CloneType database.CreateAutonomousDatabaseCloneDetailsCloneTypeEnum `json:"cloneType,omitempty"` +} + +// AutonomousDatabaseBase defines the detail information of AutonomousDatabase, corresponding to oci-go-sdk/database/AutonomousDatabase +type AutonomousDatabaseBase struct { + CompartmentId *string `json:"compartmentId,omitempty"` + AutonomousContainerDatabase AcdSpec `json:"autonomousContainerDatabase,omitempty"` + DisplayName *string `json:"displayName,omitempty"` + DbName *string `json:"dbName,omitempty"` + // +kubebuilder:validation:Enum:="OLTP";"DW";"AJD";"APEX" + DbWorkload database.AutonomousDatabaseDbWorkloadEnum `json:"dbWorkload,omitempty"` + // +kubebuilder:validation:Enum:="LICENSE_INCLUDED";"BRING_YOUR_OWN_LICENSE" + LicenseModel database.AutonomousDatabaseLicenseModelEnum `json:"licenseModel,omitempty"` + DbVersion *string `json:"dbVersion,omitempty"` + DataStorageSizeInTBs *int `json:"dataStorageSizeInTBs,omitempty"` + CpuCoreCount *int `json:"cpuCoreCount,omitempty"` + // +kubebuilder:validation:Enum:="ECPU";"OCPU" + ComputeModel database.AutonomousDatabaseComputeModelEnum `json:"computeModel,omitempty"` + ComputeCount *float32 `json:"computeCount,omitempty"` + OcpuCount *float32 `json:"ocpuCount,omitempty"` + AdminPassword PasswordSpec `json:"adminPassword,omitempty"` + IsAutoScalingEnabled *bool `json:"isAutoScalingEnabled,omitempty"` + IsDedicated *bool `json:"isDedicated,omitempty"` + IsFreeTier *bool `json:"isFreeTier,omitempty"` + + // NetworkAccess + IsAccessControlEnabled *bool `json:"isAccessControlEnabled,omitempty"` + WhitelistedIps []string `json:"whitelistedIps,omitempty"` + SubnetId *string `json:"subnetId,omitempty"` + NsgIds []string `json:"nsgIds,omitempty"` + PrivateEndpointLabel *string `json:"privateEndpointLabel,omitempty"` + IsMtlsConnectionRequired *bool `json:"isMtlsConnectionRequired,omitempty"` + + FreeformTags map[string]string `json:"freeformTags,omitempty"` +} + /************************ * ACD specs ************************/ -type K8sACDSpec struct { +type K8sAcdSpec struct { Name *string `json:"name,omitempty"` } -type OCIACDSpec struct { - OCID *string `json:"ocid,omitempty"` +type OciAcdSpec struct { + Id *string `json:"id,omitempty"` } -// ACDSpec defines the spec of the target for backup/restore runs. +// AcdSpec defines the spec of the target for backup/restore runs. // The name could be the name of an AutonomousDatabase or an AutonomousDatabaseBackup -type ACDSpec struct { - K8sACD K8sACDSpec `json:"k8sACD,omitempty"` - OCIACD OCIACDSpec `json:"ociACD,omitempty"` +type AcdSpec struct { + K8sAcd K8sAcdSpec `json:"k8sAcd,omitempty"` + OciAcd OciAcdSpec `json:"ociAcd,omitempty"` } /************************ @@ -86,13 +128,13 @@ type K8sSecretSpec struct { Name *string `json:"name,omitempty"` } -type OCISecretSpec struct { - OCID *string `json:"ocid,omitempty"` +type OciSecretSpec struct { + Id *string `json:"id,omitempty"` } type PasswordSpec struct { K8sSecret K8sSecretSpec `json:"k8sSecret,omitempty"` - OCISecret OCISecretSpec `json:"ociSecret,omitempty"` + OciSecret OciSecretSpec `json:"ociSecret,omitempty"` } type WalletSpec struct { @@ -100,67 +142,16 @@ type WalletSpec struct { Password PasswordSpec `json:"password,omitempty"` } -/************************ -* Network Access specs -************************/ - -type NetworkAccessTypeEnum string - -const ( - NetworkAccessTypePublic NetworkAccessTypeEnum = "PUBLIC" - NetworkAccessTypeRestricted NetworkAccessTypeEnum = "RESTRICTED" - NetworkAccessTypePrivate NetworkAccessTypeEnum = "PRIVATE" -) - -type NetworkAccessSpec struct { - // +kubebuilder:validation:Enum:="";"PUBLIC";"RESTRICTED";"PRIVATE" - AccessType NetworkAccessTypeEnum `json:"accessType,omitempty"` - IsAccessControlEnabled *bool `json:"isAccessControlEnabled,omitempty"` - AccessControlList []string `json:"accessControlList,omitempty"` - PrivateEndpoint PrivateEndpointSpec `json:"privateEndpoint,omitempty"` - IsMTLSConnectionRequired *bool `json:"isMTLSConnectionRequired,omitempty"` -} - -type PrivateEndpointSpec struct { - SubnetOCID *string `json:"subnetOCID,omitempty"` - NsgOCIDs []string `json:"nsgOCIDs,omitempty"` - HostnamePrefix *string `json:"hostnamePrefix,omitempty"` -} - -// AutonomousDatabaseDetails defines the detail information of AutonomousDatabase, corresponding to oci-go-sdk/database/AutonomousDatabase -type AutonomousDatabaseDetails struct { - AutonomousDatabaseOCID *string `json:"autonomousDatabaseOCID,omitempty"` - CompartmentOCID *string `json:"compartmentOCID,omitempty"` - AutonomousContainerDatabase ACDSpec `json:"autonomousContainerDatabase,omitempty"` - DisplayName *string `json:"displayName,omitempty"` - DbName *string `json:"dbName,omitempty"` - // +kubebuilder:validation:Enum:="OLTP";"DW";"AJD";"APEX" - DbWorkload database.AutonomousDatabaseDbWorkloadEnum `json:"dbWorkload,omitempty"` - // +kubebuilder:validation:Enum:="LICENSE_INCLUDED";"BRING_YOUR_OWN_LICENSE" - LicenseModel database.AutonomousDatabaseLicenseModelEnum `json:"licenseModel,omitempty"` - DbVersion *string `json:"dbVersion,omitempty"` - DataStorageSizeInTBs *int `json:"dataStorageSizeInTBs,omitempty"` - CPUCoreCount *int `json:"cpuCoreCount,omitempty"` - AdminPassword PasswordSpec `json:"adminPassword,omitempty"` - IsAutoScalingEnabled *bool `json:"isAutoScalingEnabled,omitempty"` - IsDedicated *bool `json:"isDedicated,omitempty"` - LifecycleState database.AutonomousDatabaseLifecycleStateEnum `json:"lifecycleState,omitempty"` - - NetworkAccess NetworkAccessSpec `json:"networkAccess,omitempty"` - - FreeformTags map[string]string `json:"freeformTags,omitempty"` - - Wallet WalletSpec `json:"wallet,omitempty"` -} - // AutonomousDatabaseStatus defines the observed state of AutonomousDatabase type AutonomousDatabaseStatus struct { - // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster - // Important: Run "make" to regenerate code after modifying this file - LifecycleState database.AutonomousDatabaseLifecycleStateEnum `json:"lifecycleState,omitempty"` - TimeCreated string `json:"timeCreated,omitempty"` - WalletExpiringDate string `json:"walletExpiringDate,omitempty"` - AllConnectionStrings []ConnectionStringProfile `json:"allConnectionStrings,omitempty"` + // Lifecycle State of the ADB + LifecycleState database.AutonomousDatabaseLifecycleStateEnum `json:"lifecycleState,omitempty"` + // Creation time of the ADB + TimeCreated string `json:"timeCreated,omitempty"` + // Expiring date of the instance wallet + WalletExpiringDate string `json:"walletExpiringDate,omitempty"` + // Connection Strings of the ADB + AllConnectionStrings []ConnectionStringProfile `json:"allConnectionStrings,omitempty"` // +patchMergeKey=type // +patchStrategy=merge // +listType=map @@ -175,6 +166,16 @@ const ( tlsAuthenticationMTLS TLSAuthenticationEnum = "Mutual TLS" ) +func GetTLSAuthenticationEnumFromString(val string) (TLSAuthenticationEnum, bool) { + var mappingTLSAuthenticationEnum = map[string]TLSAuthenticationEnum{ + "TLS": tlsAuthenticationTLS, + "Mutual TLS": tlsAuthenticationMTLS, + } + + enum, ok := mappingTLSAuthenticationEnum[val] + return enum, ok +} + type ConnectionStringProfile struct { TLSAuthentication TLSAuthenticationEnum `json:"tlsAuthentication,omitempty"` ConnectionStrings []ConnectionStringSpec `json:"connectionStrings"` @@ -217,181 +218,3 @@ type AutonomousDatabaseList struct { func init() { SchemeBuilder.Register(&AutonomousDatabase{}, &AutonomousDatabaseList{}) } - -// GetLastSuccessfulSpec returns spec from the lass successful reconciliation. -// Returns nil, nil if there is no lastSuccessfulSpec. -func (adb *AutonomousDatabase) GetLastSuccessfulSpec() (*AutonomousDatabaseSpec, error) { - val, ok := adb.GetAnnotations()[LastSuccessfulSpec] - if !ok { - return nil, nil - } - - specBytes := []byte(val) - sucSpec := AutonomousDatabaseSpec{} - - err := json.Unmarshal(specBytes, &sucSpec) - if err != nil { - return nil, err - } - - return &sucSpec, nil -} - -func (adb *AutonomousDatabase) UpdateLastSuccessfulSpec() error { - specBytes, err := json.Marshal(adb.Spec) - if err != nil { - return err - } - - anns := adb.GetAnnotations() - - if anns == nil { - anns = map[string]string{ - LastSuccessfulSpec: string(specBytes), - } - } else { - anns[LastSuccessfulSpec] = string(specBytes) - } - - adb.SetAnnotations(anns) - - return nil -} - -// UpdateStatusFromOCIADB updates the status subresource -func (adb *AutonomousDatabase) UpdateStatusFromOCIADB(ociObj database.AutonomousDatabase) { - adb.Status.LifecycleState = ociObj.LifecycleState - adb.Status.TimeCreated = FormatSDKTime(ociObj.TimeCreated) - - if *ociObj.IsDedicated { - conns := make([]ConnectionStringSpec, len(ociObj.ConnectionStrings.AllConnectionStrings)) - for key, val := range ociObj.ConnectionStrings.AllConnectionStrings { - conns = append(conns, ConnectionStringSpec{TNSName: key, ConnectionString: val}) - } - - adb.Status.AllConnectionStrings = []ConnectionStringProfile{ - {ConnectionStrings: conns}, - } - } else { - var mTLSConns []ConnectionStringSpec - var tlsConns []ConnectionStringSpec - - var conns []ConnectionStringProfile - - for _, profile := range ociObj.ConnectionStrings.Profiles { - if profile.TlsAuthentication == database.DatabaseConnectionStringProfileTlsAuthenticationMutual { - mTLSConns = append(mTLSConns, ConnectionStringSpec{TNSName: *profile.DisplayName, ConnectionString: *profile.Value}) - } else { - tlsConns = append(tlsConns, ConnectionStringSpec{TNSName: *profile.DisplayName, ConnectionString: *profile.Value}) - } - } - - if len(mTLSConns) > 0 { - conns = append(conns, ConnectionStringProfile{ - TLSAuthentication: tlsAuthenticationMTLS, - ConnectionStrings: mTLSConns, - }) - } - - if len(tlsConns) > 0 { - conns = append(conns, ConnectionStringProfile{ - TLSAuthentication: tlsAuthenticationTLS, - ConnectionStrings: tlsConns, - }) - } - - adb.Status.AllConnectionStrings = conns - } -} - -// UpdateFromOCIADB updates the attributes using database.AutonomousDatabase object -func (adb *AutonomousDatabase) UpdateFromOCIADB(ociObj database.AutonomousDatabase) (specChanged bool) { - oldADB := adb.DeepCopy() - - /*********************************** - * update the spec - ***********************************/ - adb.Spec.Details.AutonomousDatabaseOCID = ociObj.Id - adb.Spec.Details.CompartmentOCID = ociObj.CompartmentId - adb.Spec.Details.AutonomousContainerDatabase.OCIACD.OCID = ociObj.AutonomousContainerDatabaseId - adb.Spec.Details.DisplayName = ociObj.DisplayName - adb.Spec.Details.DbName = ociObj.DbName - adb.Spec.Details.DbWorkload = ociObj.DbWorkload - adb.Spec.Details.LicenseModel = ociObj.LicenseModel - adb.Spec.Details.DbVersion = ociObj.DbVersion - adb.Spec.Details.DataStorageSizeInTBs = ociObj.DataStorageSizeInTBs - adb.Spec.Details.CPUCoreCount = ociObj.CpuCoreCount - adb.Spec.Details.IsAutoScalingEnabled = ociObj.IsAutoScalingEnabled - adb.Spec.Details.IsDedicated = ociObj.IsDedicated - adb.Spec.Details.LifecycleState = NextADBStableState(ociObj.LifecycleState) - // Special case: an emtpy map will be nil after unmarshalling while the OCI always returns an emty map. - if len(ociObj.FreeformTags) != 0 { - adb.Spec.Details.FreeformTags = ociObj.FreeformTags - } else { - adb.Spec.Details.FreeformTags = nil - } - - // Determine network.accessType - if *ociObj.IsDedicated { - adb.Spec.Details.NetworkAccess.AccessType = NetworkAccessTypePrivate - } else { - if ociObj.NsgIds != nil || ociObj.PrivateEndpoint != nil || ociObj.PrivateEndpointIp != nil || ociObj.PrivateEndpointLabel != nil { - adb.Spec.Details.NetworkAccess.AccessType = NetworkAccessTypePrivate - } else if ociObj.WhitelistedIps != nil { - adb.Spec.Details.NetworkAccess.AccessType = NetworkAccessTypeRestricted - } else { - adb.Spec.Details.NetworkAccess.AccessType = NetworkAccessTypePublic - } - } - - adb.Spec.Details.NetworkAccess.IsAccessControlEnabled = ociObj.IsAccessControlEnabled - if len(ociObj.WhitelistedIps) != 0 { - adb.Spec.Details.NetworkAccess.AccessControlList = ociObj.WhitelistedIps - } else { - adb.Spec.Details.NetworkAccess.AccessControlList = nil - } - adb.Spec.Details.NetworkAccess.IsMTLSConnectionRequired = ociObj.IsMtlsConnectionRequired - adb.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID = ociObj.SubnetId - if len(ociObj.NsgIds) != 0 { - adb.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs = ociObj.NsgIds - } else { - adb.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs = nil - } - adb.Spec.Details.NetworkAccess.PrivateEndpoint.HostnamePrefix = ociObj.PrivateEndpointLabel - - // The admin password is not going to be updated in a bind operation. Erase the field if the lastSucSpec is nil. - // Leave the wallet field as is because the download wallet operation is independent from the update operation. - lastSucSpec, _ := adb.GetLastSuccessfulSpec() - if lastSucSpec == nil { - adb.Spec.Details.AdminPassword = PasswordSpec{} - } else { - adb.Spec.Details.AdminPassword = lastSucSpec.Details.AdminPassword - } - - /*********************************** - * update the status subresource - ***********************************/ - adb.UpdateStatusFromOCIADB(ociObj) - - return !reflect.DeepEqual(oldADB.Spec, adb.Spec) -} - -// RemoveUnchangedDetails removes the unchanged fields in spec.details, and returns if the details has been changed. -func (adb *AutonomousDatabase) RemoveUnchangedDetails(prevSpec AutonomousDatabaseSpec) (bool, error) { - - changed, err := removeUnchangedFields(prevSpec.Details, &adb.Spec.Details) - if err != nil { - return changed, err - } - - return changed, nil -} - -// A helper function which is useful for debugging. The function prints out a structural JSON format. -func (adb *AutonomousDatabase) String() (string, error) { - out, err := json.MarshalIndent(adb, "", " ") - if err != nil { - return "", err - } - return string(out), nil -} diff --git a/apis/database/v1alpha1/autonomousdatabase_webhook.go b/apis/database/v1alpha1/autonomousdatabase_webhook.go index b25e8104..e209ae7a 100644 --- a/apis/database/v1alpha1/autonomousdatabase_webhook.go +++ b/apis/database/v1alpha1/autonomousdatabase_webhook.go @@ -39,10 +39,6 @@ package v1alpha1 import ( - "fmt" - - "github.com/oracle/oci-go-sdk/v65/common" - "github.com/oracle/oci-go-sdk/v65/database" dbcommons "github.com/oracle/oracle-database-operator/commons/database" apierrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/runtime" @@ -63,37 +59,7 @@ func (r *AutonomousDatabase) SetupWebhookWithManager(mgr ctrl.Manager) error { Complete() } -//+kubebuilder:webhook:verbs=create;update,path=/mutate-database-oracle-com-v1alpha1-autonomousdatabase,mutating=true,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=autonomousdatabases,versions=v1alpha1,name=mautonomousdatabase.kb.io,admissionReviewVersions=v1 - -var _ webhook.Defaulter = &AutonomousDatabase{} - -// Default implements webhook.Defaulter so a webhook will be registered for the type -func (r *AutonomousDatabase) Default() { - autonomousdatabaselog.Info("default", "name", r.Name) - - if !isDedicated(r) { // Shared database - // AccessType is PUBLIC by default - if r.Spec.Details.NetworkAccess.AccessType == NetworkAccessTypePublic { - r.Spec.Details.NetworkAccess.IsMTLSConnectionRequired = common.Bool(true) - r.Spec.Details.NetworkAccess.AccessControlList = nil - r.Spec.Details.NetworkAccess.PrivateEndpoint.HostnamePrefix = nil - r.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs = nil - r.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID = nil - } else if r.Spec.Details.NetworkAccess.AccessType == NetworkAccessTypeRestricted { - r.Spec.Details.NetworkAccess.PrivateEndpoint.HostnamePrefix = nil - r.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs = nil - r.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID = nil - } else if r.Spec.Details.NetworkAccess.AccessType == NetworkAccessTypePrivate { - r.Spec.Details.NetworkAccess.AccessControlList = nil - } - } else { // Dedicated database - // AccessType can only be PRIVATE for a dedicated database - r.Spec.Details.NetworkAccess.AccessType = NetworkAccessTypePrivate - } - -} - -//+kubebuilder:webhook:verbs=create;update,path=/validate-database-oracle-com-v1alpha1-autonomousdatabase,mutating=false,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=autonomousdatabases,versions=v1alpha1,name=vautonomousdatabase.kb.io,admissionReviewVersions={v1} +//+kubebuilder:webhook:verbs=create;update,path=/validate-database-oracle-com-v1alpha1-autonomousdatabase,mutating=false,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=autonomousdatabases,versions=v1alpha1,name=vautonomousdatabasev1alpha1.kb.io,admissionReviewVersions=v1 var _ webhook.Validator = &AutonomousDatabase{} @@ -117,17 +83,6 @@ func (r *AutonomousDatabase) ValidateCreate() (admission.Warnings, error) { } } - if r.Spec.Details.AutonomousDatabaseOCID == nil { // provisioning operation - allErrs = validateCommon(r, allErrs) - allErrs = validateNetworkAccess(r, allErrs) - - if r.Spec.Details.LifecycleState != "" { - allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec").Child("details").Child("lifecycleState"), - "cannot apply lifecycleState to a provision operation")) - } - } - if len(allErrs) == 0 { return nil, nil } @@ -139,66 +94,43 @@ func (r *AutonomousDatabase) ValidateCreate() (admission.Warnings, error) { // ValidateUpdate implements webhook.Validator so a webhook will be registered for the type func (r *AutonomousDatabase) ValidateUpdate(old runtime.Object) (admission.Warnings, error) { var allErrs field.ErrorList - var oldADB *AutonomousDatabase = old.(*AutonomousDatabase) + var oldAdb *AutonomousDatabase = old.(*AutonomousDatabase) autonomousdatabaselog.Info("validate update", "name", r.Name) // skip the update of adding ADB OCID or binding - if oldADB.Status.LifecycleState == "" { - return nil, nil - } + // if oldAdb.Status.LifecycleState == "" { + // return nil, nil + // } // cannot update when the old state is in intermediate, except for the change to the hardLink or the terminate operatrion during valid lifecycleState - var copySpec *AutonomousDatabaseSpec = r.Spec.DeepCopy() - specChanged, err := removeUnchangedFields(oldADB.Spec, copySpec) - if err != nil { - allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec"), err.Error())) - } + // var copySpec *AutonomousDatabaseSpec = r.Spec.DeepCopy() + // specChanged, err := dbv4.RemoveUnchangedFields(oldAdb.Spec, copySpec) + // if err != nil { + // allErrs = append(allErrs, + // field.Forbidden(field.NewPath("spec"), err.Error())) + // } - hardLinkChanged := copySpec.HardLink != nil + // hardLinkChanged := copySpec.HardLink != nil - terminateOp := ValidADBTerminateState(oldADB.Status.LifecycleState) && copySpec.Details.LifecycleState == database.AutonomousDatabaseLifecycleStateTerminated + // isTerminateOp := dbv4.CanBeTerminated(oldAdb.Status.LifecycleState) && copySpec.Action == "Terminate" - if specChanged && IsADBIntermediateState(oldADB.Status.LifecycleState) && !terminateOp && !hardLinkChanged { - allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec"), - "cannot change the spec when the lifecycleState is in an intermdeiate state")) - } + // if specChanged && dbv4.IsAdbIntermediateState(oldAdb.Status.LifecycleState) && !isTerminateOp && !hardLinkChanged { + // allErrs = append(allErrs, + // field.Forbidden(field.NewPath("spec"), + // "cannot change the spec when the lifecycleState is in an intermdeiate state")) + // } // cannot modify autonomousDatabaseOCID - if r.Spec.Details.AutonomousDatabaseOCID != nil && - oldADB.Spec.Details.AutonomousDatabaseOCID != nil && - *r.Spec.Details.AutonomousDatabaseOCID != *oldADB.Spec.Details.AutonomousDatabaseOCID { + if r.Spec.Details.Id != nil && + oldAdb.Spec.Details.Id != nil && + *r.Spec.Details.Id != *oldAdb.Spec.Details.Id { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("details").Child("autonomousDatabaseOCID"), "autonomousDatabaseOCID cannot be modified")) } - // cannot change lifecycleState with other fields together (except the oci config) - var lifecycleChanged, otherFieldsChanged bool - - lifecycleChanged = oldADB.Spec.Details.LifecycleState != "" && - r.Spec.Details.LifecycleState != "" && - oldADB.Spec.Details.LifecycleState != r.Spec.Details.LifecycleState - var copiedADB *AutonomousDatabaseSpec = r.Spec.DeepCopy() - copiedADB.Details.LifecycleState = oldADB.Spec.Details.LifecycleState - copiedADB.OCIConfig = oldADB.Spec.OCIConfig - - otherFieldsChanged, err = removeUnchangedFields(oldADB.Spec, copiedADB) - if err != nil { - allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec"), err.Error())) - } - - if lifecycleChanged && otherFieldsChanged { - allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec").Child("details").Child("LifecycleState"), - "cannot change lifecycleState with other spec attributes at the same time")) - } - allErrs = validateCommon(r, allErrs) - allErrs = validateNetworkAccess(r, allErrs) if len(allErrs) == 0 { return nil, nil @@ -210,13 +142,13 @@ func (r *AutonomousDatabase) ValidateUpdate(old runtime.Object) (admission.Warni func validateCommon(adb *AutonomousDatabase, allErrs field.ErrorList) field.ErrorList { // password - if adb.Spec.Details.AdminPassword.K8sSecret.Name != nil && adb.Spec.Details.AdminPassword.OCISecret.OCID != nil { + if adb.Spec.Details.AdminPassword.K8sSecret.Name != nil && adb.Spec.Details.AdminPassword.OciSecret.Id != nil { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("details").Child("adminPassword"), "cannot apply k8sSecret.name and ociSecret.ocid at the same time")) } - if adb.Spec.Details.Wallet.Password.K8sSecret.Name != nil && adb.Spec.Details.Wallet.Password.OCISecret.OCID != nil { + if adb.Spec.Wallet.Password.K8sSecret.Name != nil && adb.Spec.Wallet.Password.OciSecret.Id != nil { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("details").Child("wallet").Child("password"), "cannot apply k8sSecret.name and ociSecret.ocid at the same time")) @@ -225,69 +157,15 @@ func validateCommon(adb *AutonomousDatabase, allErrs field.ErrorList) field.Erro return allErrs } -func validateNetworkAccess(adb *AutonomousDatabase, allErrs field.ErrorList) field.ErrorList { - if !isDedicated(adb) { - // Shared database - if adb.Spec.Details.NetworkAccess.AccessType == NetworkAccessTypeRestricted { - if adb.Spec.Details.NetworkAccess.AccessControlList == nil { - allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec").Child("details").Child("networkAccess").Child("accessControlList"), - fmt.Sprintf("accessControlList cannot be empty when the network access type is %s", NetworkAccessTypeRestricted))) - } - } else if adb.Spec.Details.NetworkAccess.AccessType == NetworkAccessTypePrivate { // the accessType is PRIVATE - if adb.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID == nil { - allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec").Child("details").Child("networkAccess").Child("privateEndpoint").Child("subnetOCID"), - fmt.Sprintf("subnetOCID cannot be empty when the network access type is %s", NetworkAccessTypePrivate))) - } - } - - // NsgOCIDs only applies to PRIVATE accessType - if adb.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs != nil && adb.Spec.Details.NetworkAccess.AccessType != NetworkAccessTypePrivate { - allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec").Child("details").Child("networkAccess").Child("privateEndpoint").Child("nsgOCIDs"), - fmt.Sprintf("NsgOCIDs cannot only be applied when network access type is %s.", NetworkAccessTypePrivate))) - } - - // IsAccessControlEnabled is not applicable to a shared database - if adb.Spec.Details.NetworkAccess.IsAccessControlEnabled != nil { - allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec").Child("details").Child("networkAccess").Child("IsAccessControlEnabled"), - "isAccessControlEnabled is not applicable on a shared Autonomous Database")) - } - } else { - // Dedicated database - - // accessControlList cannot be provided when Autonomous Database's access control is disabled - if adb.Spec.Details.NetworkAccess.AccessControlList != nil && - (adb.Spec.Details.NetworkAccess.IsAccessControlEnabled == nil || !*adb.Spec.Details.NetworkAccess.IsAccessControlEnabled) { - allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec").Child("details").Child("networkAccess").Child("accessControlList"), - "access control list cannot be provided when Autonomous Database's access control is disabled")) - } - - // IsMTLSConnectionRequired is not supported by dedicated database - if adb.Spec.Details.NetworkAccess.IsMTLSConnectionRequired != nil { - allErrs = append(allErrs, - field.Forbidden(field.NewPath("spec").Child("details").Child("networkAccess").Child("isMTLSConnectionRequired"), - "isMTLSConnectionRequired is not supported on a dedicated database")) - } - } - - return allErrs -} - // ValidateDelete implements webhook.Validator so a webhook will be registered for the type func (r *AutonomousDatabase) ValidateDelete() (admission.Warnings, error) { autonomousdatabaselog.Info("validate delete", "name", r.Name) - - // TODO(user): fill in your validation logic upon object deletion. return nil, nil } // Returns true if AutonomousContainerDatabaseOCID has value. // We don't use Details.IsDedicated because the parameter might be null when it's a provision operation. func isDedicated(adb *AutonomousDatabase) bool { - return adb.Spec.Details.AutonomousContainerDatabase.K8sACD.Name != nil || - adb.Spec.Details.AutonomousContainerDatabase.OCIACD.OCID != nil + return adb.Spec.Details.AutonomousContainerDatabase.K8sAcd.Name != nil || + adb.Spec.Details.AutonomousContainerDatabase.OciAcd.Id != nil } diff --git a/apis/database/v1alpha1/autonomousdatabase_webhook_test.go b/apis/database/v1alpha1/autonomousdatabase_webhook_test.go index ee26021f..8949f8f4 100644 --- a/apis/database/v1alpha1/autonomousdatabase_webhook_test.go +++ b/apis/database/v1alpha1/autonomousdatabase_webhook_test.go @@ -50,55 +50,6 @@ import ( ) var _ = Describe("test AutonomousDatabase webhook", func() { - Describe("Test AutonomousDatabase mutating webhook", func() { - var ( - resourceName = "testadb" - namespace = "default" - adbLookupKey = types.NamespacedName{Name: resourceName, Namespace: namespace} - - timeout = time.Second * 5 - - adb *AutonomousDatabase - ) - - BeforeEach(func() { - adb = &AutonomousDatabase{ - TypeMeta: metav1.TypeMeta{ - APIVersion: "database.oracle.com/v1alpha1", - Kind: "AutonomousDatabase", - }, - ObjectMeta: metav1.ObjectMeta{ - Name: resourceName, - Namespace: namespace, - }, - Spec: AutonomousDatabaseSpec{ - Details: AutonomousDatabaseDetails{}, - }, - } - }) - - AfterEach(func() { - Expect(k8sClient.Delete(context.TODO(), adb)).To(Succeed()) - }) - - It("Should set the default network access type to PRIVATE, if it's a dedicated ADB", func() { - By("Creating an AutonomousDatabase with ACD_OCID") - adb.Spec.Details.AutonomousContainerDatabase.OCIACD.OCID = common.String("ocid1.autonomouscontainerdatabase.oc1.dummy-acd-ocid") - - Expect(k8sClient.Create(context.TODO(), adb)).To(Succeed()) - - By("Checking the AutonomousDatabase has a network access type PRIVATE") - Eventually(func() NetworkAccessTypeEnum { - err := k8sClient.Get(context.TODO(), adbLookupKey, adb) - if err != nil { - return "" - } - - return adb.Spec.Details.NetworkAccess.AccessType - }, timeout).Should(Equal(NetworkAccessTypePrivate)) - }) - }) - Describe("Test ValidateCreate of the AutonomousDatabase validating webhook", func() { var ( resourceName = "testadb" @@ -119,16 +70,18 @@ var _ = Describe("test AutonomousDatabase webhook", func() { }, Spec: AutonomousDatabaseSpec{ Details: AutonomousDatabaseDetails{ - CompartmentOCID: common.String("fake-compartment-ocid"), - DbName: common.String("fake-dbName"), - DisplayName: common.String("fake-displayName"), - CPUCoreCount: common.Int(1), - AdminPassword: PasswordSpec{ - K8sSecret: K8sSecretSpec{ - Name: common.String("fake-admin-password"), + AutonomousDatabaseBase: AutonomousDatabaseBase{ + CompartmentId: common.String("fake-compartment-ocid"), + DbName: common.String("fake-dbName"), + DisplayName: common.String("fake-displayName"), + CpuCoreCount: common.Int(1), + AdminPassword: PasswordSpec{ + K8sSecret: K8sSecretSpec{ + Name: common.String("fake-admin-password"), + }, }, + DataStorageSizeInTBs: common.Int(1), }, - DataStorageSizeInTBs: common.Int(1), }, }, } @@ -139,7 +92,7 @@ var _ = Describe("test AutonomousDatabase webhook", func() { var errMsg string = "cannot apply k8sSecret.name and ociSecret.ocid at the same time" adb.Spec.Details.AdminPassword.K8sSecret.Name = common.String("test-admin-password") - adb.Spec.Details.AdminPassword.OCISecret.OCID = common.String("fake.ocid1.vaultsecret.oc1...") + adb.Spec.Details.AdminPassword.OciSecret.Id = common.String("fake.ocid1.vaultsecret.oc1...") validateInvalidTest(adb, false, errMsg) }) @@ -147,54 +100,23 @@ var _ = Describe("test AutonomousDatabase webhook", func() { It("Should not apply values to wallet.password.k8sSecret and wallet.password.ociSecret at the same time", func() { var errMsg string = "cannot apply k8sSecret.name and ociSecret.ocid at the same time" - adb.Spec.Details.Wallet.Password.K8sSecret.Name = common.String("test-wallet-password") - adb.Spec.Details.Wallet.Password.OCISecret.OCID = common.String("fake.ocid1.vaultsecret.oc1...") + adb.Spec.Wallet.Password.K8sSecret.Name = common.String("test-wallet-password") + adb.Spec.Wallet.Password.OciSecret.Id = common.String("fake.ocid1.vaultsecret.oc1...") validateInvalidTest(adb, false, errMsg) }) - // Network validation - Context("Shared Autonomous Database", func() { - It("AccessControlList cannot be empty when the network access type is RESTRICTED", func() { - var errMsg string = "accessControlList cannot be empty when the network access type is " + string(NetworkAccessTypeRestricted) - - adb.Spec.Details.NetworkAccess.AccessType = NetworkAccessTypeRestricted - adb.Spec.Details.NetworkAccess.AccessControlList = nil - - validateInvalidTest(adb, false, errMsg) - }) - - It("SubnetOCID and nsgOCIDs cannot be empty when the network access type is PRIVATE", func() { - var errMsg1 string = "subnetOCID cannot be empty when the network access type is " + string(NetworkAccessTypePrivate) - var errMsg2 string = "nsgOCIDs cannot be empty when the network access type is " + string(NetworkAccessTypePrivate) - - adb.Spec.Details.NetworkAccess.AccessType = NetworkAccessTypePrivate - adb.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID = nil - adb.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs = nil - - validateInvalidTest(adb, false, errMsg1, errMsg2) - }) - - It("IsAccessControlEnabled is not applicable on a shared Autonomous Database", func() { - var errMsg string = "isAccessControlEnabled is not applicable on a shared Autonomous Database" - - adb.Spec.Details.NetworkAccess.IsAccessControlEnabled = common.Bool(true) - - validateInvalidTest(adb, false, errMsg) - }) - }) - Context("Dedicated Autonomous Database", func() { BeforeEach(func() { - adb.Spec.Details.AutonomousContainerDatabase.K8sACD.Name = common.String("testACD") - adb.Spec.Details.AutonomousContainerDatabase.OCIACD.OCID = common.String("fake-acd-ocid") + adb.Spec.Details.AutonomousContainerDatabase.K8sAcd.Name = common.String("testACD") + adb.Spec.Details.AutonomousContainerDatabase.OciAcd.Id = common.String("fake-acd-ocid") }) It("AccessControlList cannot be empty when the network access type is RESTRICTED", func() { var errMsg string = "access control list cannot be provided when Autonomous Database's access control is disabled" - adb.Spec.Details.NetworkAccess.IsAccessControlEnabled = common.Bool(false) - adb.Spec.Details.NetworkAccess.AccessControlList = []string{"192.168.1.1"} + adb.Spec.Details.IsAccessControlEnabled = common.Bool(false) + adb.Spec.Details.WhitelistedIps = []string{"192.168.1.1"} validateInvalidTest(adb, false, errMsg) }) @@ -202,21 +124,12 @@ var _ = Describe("test AutonomousDatabase webhook", func() { It("AccessControlList cannot be empty when the network access type is RESTRICTED", func() { var errMsg string = "isMTLSConnectionRequired is not supported on a dedicated database" - adb.Spec.Details.NetworkAccess.IsMTLSConnectionRequired = common.Bool(true) + adb.Spec.Details.IsMtlsConnectionRequired = common.Bool(true) validateInvalidTest(adb, false, errMsg) }) }) - - // Others - It("Cannot apply lifecycleState to a provision operation", func() { - var errMsg string = "cannot apply lifecycleState to a provision operation" - - adb.Spec.Details.LifecycleState = database.AutonomousDatabaseLifecycleStateStopped - - validateInvalidTest(adb, false, errMsg) - }) }) // Skip the common and network validations since they're already verified in the test for ValidateCreate @@ -242,14 +155,16 @@ var _ = Describe("test AutonomousDatabase webhook", func() { Namespace: namespace, }, Spec: AutonomousDatabaseSpec{ + Action: "Create", Details: AutonomousDatabaseDetails{ - CompartmentOCID: common.String("fake-compartment-ocid"), - AutonomousDatabaseOCID: common.String("fake-adb-ocid"), - DbName: common.String("fake-dbName"), - DisplayName: common.String("fake-displayName"), - CPUCoreCount: common.Int(1), - DataStorageSizeInTBs: common.Int(1), - LifecycleState: database.AutonomousDatabaseLifecycleStateAvailable, + Id: common.String("fake-adb-ocid"), + AutonomousDatabaseBase: AutonomousDatabaseBase{ + CompartmentId: common.String("fake-compartment-ocid"), + DbName: common.String("fake-dbName"), + DisplayName: common.String("fake-displayName"), + CpuCoreCount: common.Int(1), + DataStorageSizeInTBs: common.Int(1), + }, }, }, } @@ -293,16 +208,7 @@ var _ = Describe("test AutonomousDatabase webhook", func() { It("AutonomousDatabaseOCID cannot be modified", func() { var errMsg string = "autonomousDatabaseOCID cannot be modified" - adb.Spec.Details.AutonomousDatabaseOCID = common.String("modified-adb-ocid") - - validateInvalidTest(adb, true, errMsg) - }) - - It("Cannot change lifecycleState with other spec attributes at the same time", func() { - var errMsg string = "cannot change lifecycleState with other spec attributes at the same time" - - adb.Spec.Details.LifecycleState = database.AutonomousDatabaseLifecycleStateStopped - adb.Spec.Details.CPUCoreCount = common.Int(2) + adb.Spec.Details.Id = common.String("modified-adb-ocid") validateInvalidTest(adb, true, errMsg) }) diff --git a/apis/database/v1alpha1/autonomousdatabasebackup_types.go b/apis/database/v1alpha1/autonomousdatabasebackup_types.go index 95c77560..aa70c2d5 100644 --- a/apis/database/v1alpha1/autonomousdatabasebackup_types.go +++ b/apis/database/v1alpha1/autonomousdatabasebackup_types.go @@ -43,6 +43,7 @@ import ( "github.com/oracle/oci-go-sdk/v65/common" "github.com/oracle/oci-go-sdk/v65/database" + dbv4 "github.com/oracle/oracle-database-operator/apis/database/v4" ) // EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! @@ -57,7 +58,7 @@ type AutonomousDatabaseBackupSpec struct { AutonomousDatabaseBackupOCID *string `json:"autonomousDatabaseBackupOCID,omitempty"` IsLongTermBackup *bool `json:"isLongTermBackup,omitempty"` RetentionPeriodInDays *int `json:"retentionPeriodInDays,omitempty"` - OCIConfig OCIConfigSpec `json:"ociConfig,omitempty"` + OCIConfig OciConfigSpec `json:"ociConfig,omitempty"` } // AutonomousDatabaseBackupStatus defines the observed state of AutonomousDatabaseBackup @@ -104,7 +105,7 @@ func init() { SchemeBuilder.Register(&AutonomousDatabaseBackup{}, &AutonomousDatabaseBackupList{}) } -func (b *AutonomousDatabaseBackup) UpdateStatusFromOCIBackup(ociBackup database.AutonomousDatabaseBackup, ociADB database.AutonomousDatabase) { +func (b *AutonomousDatabaseBackup) UpdateStatusFromOCIBackup(ociBackup database.AutonomousDatabaseBackup, ociAdb database.AutonomousDatabase) { b.Status.AutonomousDatabaseOCID = *ociBackup.AutonomousDatabaseId b.Status.CompartmentOCID = *ociBackup.CompartmentId b.Status.Type = ociBackup.Type @@ -112,14 +113,14 @@ func (b *AutonomousDatabaseBackup) UpdateStatusFromOCIBackup(ociBackup database. b.Status.LifecycleState = ociBackup.LifecycleState - b.Status.TimeStarted = FormatSDKTime(ociBackup.TimeStarted) - b.Status.TimeEnded = FormatSDKTime(ociBackup.TimeEnded) + b.Status.TimeStarted = dbv4.FormatSDKTime(ociBackup.TimeStarted) + b.Status.TimeEnded = dbv4.FormatSDKTime(ociBackup.TimeEnded) - b.Status.DBDisplayName = *ociADB.DisplayName - b.Status.DBName = *ociADB.DbName + b.Status.DBDisplayName = *ociAdb.DisplayName + b.Status.DBName = *ociAdb.DbName } // GetTimeEnded returns the status.timeEnded in SDKTime format func (b *AutonomousDatabaseBackup) GetTimeEnded() (*common.SDKTime, error) { - return parseDisplayTime(b.Status.TimeEnded) + return dbv4.ParseDisplayTime(b.Status.TimeEnded) } diff --git a/apis/database/v1alpha1/autonomousdatabasebackup_webhook.go b/apis/database/v1alpha1/autonomousdatabasebackup_webhook.go index 99bf3815..ffa9b888 100644 --- a/apis/database/v1alpha1/autonomousdatabasebackup_webhook.go +++ b/apis/database/v1alpha1/autonomousdatabasebackup_webhook.go @@ -59,21 +59,16 @@ func (r *AutonomousDatabaseBackup) SetupWebhookWithManager(mgr ctrl.Manager) err Complete() } -// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! - -//+kubebuilder:webhook:path=/mutate-database-oracle-com-v1alpha1-autonomousdatabasebackup,mutating=true,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=autonomousdatabasebackups,verbs=create;update,versions=v1alpha1,name=mautonomousdatabasebackup.kb.io,admissionReviewVersions={v1} +//+kubebuilder:webhook:path=/mutate-database-oracle-com-v1alpha1-autonomousdatabasebackup,mutating=true,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=autonomousdatabasebackups,verbs=create;update,versions=v1alpha1,name=mautonomousdatabasebackupv1alpha1.kb.io,admissionReviewVersions=v1 var _ webhook.Defaulter = &AutonomousDatabaseBackup{} // Default implements webhook.Defaulter so a webhook will be registered for the type func (r *AutonomousDatabaseBackup) Default() { autonomousdatabasebackuplog.Info("default", "name", r.Name) - - // TODO(user): fill in your defaulting logic. } -// TODO(user): change verbs to "verbs=create;update;delete" if you want to enable deletion validation. -//+kubebuilder:webhook:verbs=create;update,path=/validate-database-oracle-com-v1alpha1-autonomousdatabasebackup,mutating=false,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=autonomousdatabasebackups,versions=v1alpha1,name=vautonomousdatabasebackup.kb.io,admissionReviewVersions={v1} +//+kubebuilder:webhook:verbs=create;update,path=/validate-database-oracle-com-v1alpha1-autonomousdatabasebackup,mutating=false,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=autonomousdatabasebackups,versions=v1alpha1,name=vautonomousdatabasebackupv1alpha1.kb.io,admissionReviewVersions=v1 var _ webhook.Validator = &AutonomousDatabaseBackup{} @@ -96,12 +91,12 @@ func (r *AutonomousDatabaseBackup) ValidateCreate() (admission.Warnings, error) } } - if r.Spec.Target.K8sADB.Name == nil && r.Spec.Target.OCIADB.OCID == nil { + if r.Spec.Target.K8sAdb.Name == nil && r.Spec.Target.OciAdb.Ocid == nil { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("target"), "target ADB is empty")) } - if r.Spec.Target.K8sADB.Name != nil && r.Spec.Target.OCIADB.OCID != nil { + if r.Spec.Target.K8sAdb.Name != nil && r.Spec.Target.OciAdb.Ocid != nil { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("target"), "specify either k8sADB or ociADB, but not both")) } @@ -128,14 +123,14 @@ func (r *AutonomousDatabaseBackup) ValidateUpdate(old runtime.Object) (admission "cannot assign a new autonomousDatabaseBackupOCID to this backup")) } - if oldBackup.Spec.Target.K8sADB.Name != nil && r.Spec.Target.K8sADB.Name != nil && - *oldBackup.Spec.Target.K8sADB.Name != *r.Spec.Target.K8sADB.Name { + if oldBackup.Spec.Target.K8sAdb.Name != nil && r.Spec.Target.K8sAdb.Name != nil && + *oldBackup.Spec.Target.K8sAdb.Name != *r.Spec.Target.K8sAdb.Name { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("target").Child("k8sADB").Child("name"), "cannot assign a new name to the target")) } - if oldBackup.Spec.Target.OCIADB.OCID != nil && r.Spec.Target.OCIADB.OCID != nil && - *oldBackup.Spec.Target.OCIADB.OCID != *r.Spec.Target.OCIADB.OCID { + if oldBackup.Spec.Target.OciAdb.Ocid != nil && r.Spec.Target.OciAdb.Ocid != nil && + *oldBackup.Spec.Target.OciAdb.Ocid != *r.Spec.Target.OciAdb.Ocid { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("target").Child("ociADB").Child("ocid"), "cannot assign a new ocid to the target")) } diff --git a/apis/database/v1alpha1/autonomousdatabasebackup_webhook_test.go b/apis/database/v1alpha1/autonomousdatabasebackup_webhook_test.go index 497c3f28..87eb1618 100644 --- a/apis/database/v1alpha1/autonomousdatabasebackup_webhook_test.go +++ b/apis/database/v1alpha1/autonomousdatabasebackup_webhook_test.go @@ -75,8 +75,8 @@ var _ = Describe("test AutonomousDatabaseBackup webhook", func() { It("Should specify at least one of the k8sADB and ociADB", func() { var errMsg string = "target ADB is empty" - backup.Spec.Target.K8sADB.Name = nil - backup.Spec.Target.OCIADB.OCID = nil + backup.Spec.Target.K8sAdb.Name = nil + backup.Spec.Target.OciAdb.Ocid = nil validateInvalidTest(backup, false, errMsg) }) @@ -84,8 +84,8 @@ var _ = Describe("test AutonomousDatabaseBackup webhook", func() { It("Should specify either k8sADB or ociADB, but not both", func() { var errMsg string = "specify either k8sADB or ociADB, but not both" - backup.Spec.Target.K8sADB.Name = common.String("fake-target-adb") - backup.Spec.Target.OCIADB.OCID = common.String("fake.ocid1.autonomousdatabase.oc1...") + backup.Spec.Target.K8sAdb.Name = common.String("fake-target-adb") + backup.Spec.Target.OciAdb.Ocid = common.String("fake.ocid1.autonomousdatabase.oc1...") validateInvalidTest(backup, false, errMsg) }) @@ -133,15 +133,15 @@ var _ = Describe("test AutonomousDatabaseBackup webhook", func() { Expect(k8sClient.Delete(context.TODO(), backup)).To(Succeed()) }) - Context("The bakcup is using target.k8sADB.name", func() { + Context("The bakcup is using target.k8sAdb.name", func() { BeforeEach(func() { - backup.Spec.Target.K8sADB.Name = common.String("fake-target-adb") + backup.Spec.Target.K8sAdb.Name = common.String("fake-target-adb") }) It("Cannot assign a new name to the target", func() { var errMsg string = "cannot assign a new name to the target" - backup.Spec.Target.K8sADB.Name = common.String("modified-target-adb") + backup.Spec.Target.K8sAdb.Name = common.String("modified-target-adb") validateInvalidTest(backup, true, errMsg) }) @@ -157,13 +157,13 @@ var _ = Describe("test AutonomousDatabaseBackup webhook", func() { Context("The bakcup is using target.ociADB.ocid", func() { BeforeEach(func() { - backup.Spec.Target.OCIADB.OCID = common.String("fake.ocid1.autonomousdatabase.oc1...") + backup.Spec.Target.OciAdb.Ocid = common.String("fake.ocid1.autonomousdatabase.oc1...") }) It("Cannot assign a new ocid to the target", func() { var errMsg string = "cannot assign a new ocid to the target" - backup.Spec.Target.OCIADB.OCID = common.String("modified.ocid1.autonomousdatabase.oc1...") + backup.Spec.Target.OciAdb.Ocid = common.String("modified.ocid1.autonomousdatabase.oc1...") validateInvalidTest(backup, true, errMsg) }) diff --git a/apis/database/v1alpha1/autonomousdatabaserestore_types.go b/apis/database/v1alpha1/autonomousdatabaserestore_types.go index 4bea0043..ef8698b2 100644 --- a/apis/database/v1alpha1/autonomousdatabaserestore_types.go +++ b/apis/database/v1alpha1/autonomousdatabaserestore_types.go @@ -46,22 +46,23 @@ import ( "github.com/oracle/oci-go-sdk/v65/common" "github.com/oracle/oci-go-sdk/v65/database" "github.com/oracle/oci-go-sdk/v65/workrequests" + dbv4 "github.com/oracle/oracle-database-operator/apis/database/v4" ) // EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! // NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized. -type K8sADBBackupSpec struct { +type K8sAdbBackupSpec struct { Name *string `json:"name,omitempty"` } -type PITSpec struct { +type PitSpec struct { // The timestamp must follow this format: YYYY-MM-DD HH:MM:SS GMT Timestamp *string `json:"timestamp,omitempty"` } type SourceSpec struct { - K8sADBBackup K8sADBBackupSpec `json:"k8sADBBackup,omitempty"` - PointInTime PITSpec `json:"pointInTime,omitempty"` + K8sAdbBackup K8sAdbBackupSpec `json:"k8sADBBackup,omitempty"` + PointInTime PitSpec `json:"pointInTime,omitempty"` } // AutonomousDatabaseRestoreSpec defines the desired state of AutonomousDatabaseRestore @@ -70,7 +71,7 @@ type AutonomousDatabaseRestoreSpec struct { // Important: Run "make" to regenerate code after modifying this file Target TargetSpec `json:"target"` Source SourceSpec `json:"source"` - OCIConfig OCIConfigSpec `json:"ociConfig,omitempty"` + OCIConfig OciConfigSpec `json:"ociConfig,omitempty"` } // AutonomousDatabaseRestoreStatus defines the observed state of AutonomousDatabaseRestore @@ -120,7 +121,7 @@ func (r *AutonomousDatabaseRestore) GetPIT() (*common.SDKTime, error) { if r.Spec.Source.PointInTime.Timestamp == nil { return nil, errors.New("the timestamp is empty") } - return parseDisplayTime(*r.Spec.Source.PointInTime.Timestamp) + return dbv4.ParseDisplayTime(*r.Spec.Source.PointInTime.Timestamp) } func (r *AutonomousDatabaseRestore) UpdateStatus( @@ -132,7 +133,7 @@ func (r *AutonomousDatabaseRestore) UpdateStatus( r.Status.WorkRequestOCID = *workResp.Id r.Status.Status = workResp.Status - r.Status.TimeAccepted = FormatSDKTime(workResp.TimeAccepted) - r.Status.TimeStarted = FormatSDKTime(workResp.TimeStarted) - r.Status.TimeEnded = FormatSDKTime(workResp.TimeFinished) + r.Status.TimeAccepted = dbv4.FormatSDKTime(workResp.TimeAccepted) + r.Status.TimeStarted = dbv4.FormatSDKTime(workResp.TimeStarted) + r.Status.TimeEnded = dbv4.FormatSDKTime(workResp.TimeFinished) } diff --git a/apis/database/v1alpha1/autonomousdatabaserestore_webhook.go b/apis/database/v1alpha1/autonomousdatabaserestore_webhook.go index 4f96bd7b..dcd57137 100644 --- a/apis/database/v1alpha1/autonomousdatabaserestore_webhook.go +++ b/apis/database/v1alpha1/autonomousdatabaserestore_webhook.go @@ -39,6 +39,7 @@ package v1alpha1 import ( + dbv4 "github.com/oracle/oracle-database-operator/apis/database/v4" dbcommons "github.com/oracle/oracle-database-operator/commons/database" apierrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/runtime" @@ -59,10 +60,7 @@ func (r *AutonomousDatabaseRestore) SetupWebhookWithManager(mgr ctrl.Manager) er Complete() } -// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! - -// TODO(user): change verbs to "verbs=create;update;delete" if you want to enable deletion validation. -//+kubebuilder:webhook:verbs=create;update,path=/validate-database-oracle-com-v1alpha1-autonomousdatabaserestore,mutating=false,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=autonomousdatabaserestores,versions=v1alpha1,name=vautonomousdatabaserestore.kb.io,admissionReviewVersions={v1} +//+kubebuilder:webhook:verbs=create;update,path=/validate-database-oracle-com-v1alpha1-autonomousdatabaserestore,mutating=false,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=autonomousdatabaserestores,versions=v1alpha1,name=vautonomousdatabaserestorev1alpha1.kb.io,admissionReviewVersions=v1 var _ webhook.Validator = &AutonomousDatabaseRestore{} @@ -86,24 +84,24 @@ func (r *AutonomousDatabaseRestore) ValidateCreate() (admission.Warnings, error) } // Validate the target ADB - if r.Spec.Target.K8sADB.Name == nil && r.Spec.Target.OCIADB.OCID == nil { + if r.Spec.Target.K8sAdb.Name == nil && r.Spec.Target.OciAdb.Ocid == nil { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("target"), "target ADB is empty")) } - if r.Spec.Target.K8sADB.Name != nil && r.Spec.Target.OCIADB.OCID != nil { + if r.Spec.Target.K8sAdb.Name != nil && r.Spec.Target.OciAdb.Ocid != nil { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("target"), "specify either k8sADB.name or ociADB.ocid, but not both")) } // Validate the restore source - if r.Spec.Source.K8sADBBackup.Name == nil && + if r.Spec.Source.K8sAdbBackup.Name == nil && r.Spec.Source.PointInTime.Timestamp == nil { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("source"), "retore source is empty")) } - if r.Spec.Source.K8sADBBackup.Name != nil && + if r.Spec.Source.K8sAdbBackup.Name != nil && r.Spec.Source.PointInTime.Timestamp != nil { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("source"), "cannot apply backupName and the PITR parameters at the same time")) @@ -111,7 +109,7 @@ func (r *AutonomousDatabaseRestore) ValidateCreate() (admission.Warnings, error) // Verify the timestamp format if it's PITR if r.Spec.Source.PointInTime.Timestamp != nil { - _, err := parseDisplayTime(*r.Spec.Source.PointInTime.Timestamp) + _, err := dbv4.ParseDisplayTime(*r.Spec.Source.PointInTime.Timestamp) if err != nil { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("source").Child("pointInTime").Child("timestamp"), "invalid timestamp format")) diff --git a/apis/database/v1alpha1/autonomousdatabaserestore_webhook_test.go b/apis/database/v1alpha1/autonomousdatabaserestore_webhook_test.go index aed87d71..0cc9b692 100644 --- a/apis/database/v1alpha1/autonomousdatabaserestore_webhook_test.go +++ b/apis/database/v1alpha1/autonomousdatabaserestore_webhook_test.go @@ -71,8 +71,8 @@ var _ = Describe("test AutonomousDatabaseRestore webhook", func() { It("Should specify at least one of the k8sADB and ociADB", func() { var errMsg string = "target ADB is empty" - restore.Spec.Target.K8sADB.Name = nil - restore.Spec.Target.OCIADB.OCID = nil + restore.Spec.Target.K8sAdb.Name = nil + restore.Spec.Target.OciAdb.Ocid = nil validateInvalidTest(restore, false, errMsg) }) @@ -80,8 +80,8 @@ var _ = Describe("test AutonomousDatabaseRestore webhook", func() { It("Should specify either k8sADB.name or ociADB.ocid, but not both", func() { var errMsg string = "specify either k8sADB.name or ociADB.ocid, but not both" - restore.Spec.Target.K8sADB.Name = common.String("fake-target-adb") - restore.Spec.Target.OCIADB.OCID = common.String("fake.ocid1.autonomousdatabase.oc1...") + restore.Spec.Target.K8sAdb.Name = common.String("fake-target-adb") + restore.Spec.Target.OciAdb.Ocid = common.String("fake.ocid1.autonomousdatabase.oc1...") validateInvalidTest(restore, false, errMsg) }) @@ -89,7 +89,7 @@ var _ = Describe("test AutonomousDatabaseRestore webhook", func() { It("Should select at least one restore source", func() { var errMsg string = "retore source is empty" - restore.Spec.Source.K8sADBBackup.Name = nil + restore.Spec.Source.K8sAdbBackup.Name = nil restore.Spec.Source.PointInTime.Timestamp = nil validateInvalidTest(restore, false, errMsg) @@ -98,7 +98,7 @@ var _ = Describe("test AutonomousDatabaseRestore webhook", func() { It("Cannot apply backupName and the PITR parameters at the same time", func() { var errMsg string = "cannot apply backupName and the PITR parameters at the same time" - restore.Spec.Source.K8sADBBackup.Name = common.String("fake-source-adb-backup") + restore.Spec.Source.K8sAdbBackup.Name = common.String("fake-source-adb-backup") restore.Spec.Source.PointInTime.Timestamp = common.String("2021-12-23 11:03:13 UTC") validateInvalidTest(restore, false, errMsg) diff --git a/apis/database/v1alpha1/dataguardbroker_conversion.go b/apis/database/v1alpha1/dataguardbroker_conversion.go new file mode 100644 index 00000000..39751a05 --- /dev/null +++ b/apis/database/v1alpha1/dataguardbroker_conversion.go @@ -0,0 +1,14 @@ +package v1alpha1 + +import ( + "sigs.k8s.io/controller-runtime/pkg/conversion" +) + +func (src *DataguardBroker) ConvertTo(dst conversion.Hub) error { + return nil +} + +// ConvertFrom converts v1 to v1alpha1 +func (dst *DataguardBroker) ConvertFrom(src conversion.Hub) error { + return nil +} diff --git a/apis/database/v1alpha1/dataguardbroker_types.go b/apis/database/v1alpha1/dataguardbroker_types.go index 37d71b92..768d6dd3 100644 --- a/apis/database/v1alpha1/dataguardbroker_types.go +++ b/apis/database/v1alpha1/dataguardbroker_types.go @@ -56,20 +56,10 @@ type DataguardBrokerSpec struct { LoadBalancer bool `json:"loadBalancer,omitempty"` ServiceAnnotations map[string]string `json:"serviceAnnotations,omitempty"` // +kubebuilder:validation:Enum=MaxPerformance;MaxAvailability - ProtectionMode string `json:"protectionMode"` - NodeSelector map[string]string `json:"nodeSelector,omitempty"` - FastStartFailOver DataguardBrokerFastStartFailOver `json:"fastStartFailOver,omitempty"` -} - -type DataguardBrokerFastStartFailOver struct { - Enable bool `json:"enable,omitempty"` - Strategy []DataguardBrokerStrategy `json:"strategy,omitempty"` -} + ProtectionMode string `json:"protectionMode"` + NodeSelector map[string]string `json:"nodeSelector,omitempty"` -// FSFO strategy -type DataguardBrokerStrategy struct { - SourceDatabaseRef string `json:"sourceDatabaseRef,omitempty"` - TargetDatabaseRefs string `json:"targetDatabaseRefs,omitempty"` + FastStartFailover bool `json:"fastStartFailover,omitempty"` } // DataguardBrokerStatus defines the observed state of DataguardBroker @@ -84,10 +74,13 @@ type DataguardBrokerStatus struct { ExternalConnectString string `json:"externalConnectString,omitempty"` ClusterConnectString string `json:"clusterConnectString,omitempty"` Status string `json:"status,omitempty"` + + FastStartFailover string `json:"fastStartFailover,omitempty"` + DatabasesInDataguardConfig map[string]string `json:"databasesInDataguardConfig,omitempty"` } -//+kubebuilder:object:root=true -//+kubebuilder:subresource:status +// +kubebuilder:object:root=true +// +kubebuilder:subresource:status // +kubebuilder:printcolumn:JSONPath=".status.primaryDatabase",name="Primary",type="string" // +kubebuilder:printcolumn:JSONPath=".status.standbyDatabases",name="Standbys",type="string" // +kubebuilder:printcolumn:JSONPath=".spec.protectionMode",name="Protection Mode",type="string" @@ -95,6 +88,7 @@ type DataguardBrokerStatus struct { // +kubebuilder:printcolumn:JSONPath=".status.externalConnectString",name="Connect Str",type="string" // +kubebuilder:printcolumn:JSONPath=".spec.primaryDatabaseRef",name="Primary Database",type="string", priority=1 // +kubebuilder:printcolumn:JSONPath=".status.status",name="Status",type="string" +// +kubebuilder:printcolumn:JSONPath=".status.fastStartFailover",name="FSFO", type="string" // DataguardBroker is the Schema for the dataguardbrokers API type DataguardBroker struct { @@ -105,6 +99,55 @@ type DataguardBroker struct { Status DataguardBrokerStatus `json:"status,omitempty"` } +// ////////////////////////////////////////////////////////////////////////////////////////////////// +// Returns the current primary database in the dataguard configuration from the resource status/spec +// ////////////////////////////////////////////////////////////////////////////////////////////////// +func (broker *DataguardBroker) GetCurrentPrimaryDatabase() string { + if broker.Status.PrimaryDatabase != "" { + return broker.Status.DatabasesInDataguardConfig[broker.Status.PrimaryDatabase] + } + return broker.Spec.PrimaryDatabaseRef +} + +// ////////////////////////////////////////////////////////////////////////////////////////////////// +// Returns databases in Dataguard configuration from the resource status/spec +// ////////////////////////////////////////////////////////////////////////////////////////////////// +func (broker *DataguardBroker) GetDatabasesInDataGuardConfiguration() []string { + var databases []string + if len(broker.Status.DatabasesInDataguardConfig) > 0 { + for _, value := range broker.Status.DatabasesInDataguardConfig { + if value != "" { + databases = append(databases, value) + } + } + + return databases + } + + databases = append(databases, broker.Spec.PrimaryDatabaseRef) + databases = append(databases, broker.Spec.StandbyDatabaseRefs...) + return databases +} + +// ////////////////////////////////////////////////////////////////////////////////////////////////// +// Returns standby databases in the dataguard configuration from the resource status/spec +// ////////////////////////////////////////////////////////////////////////////////////////////////// +func (broker *DataguardBroker) GetStandbyDatabasesInDgConfig() []string { + var databases []string + if len(broker.Status.DatabasesInDataguardConfig) > 0 { + for _, value := range broker.Status.DatabasesInDataguardConfig { + if value != "" && value != broker.Status.PrimaryDatabase { + databases = append(databases, value) + } + } + + return databases + } + + databases = append(databases, broker.Spec.StandbyDatabaseRefs...) + return databases +} + //+kubebuilder:object:root=true // DataguardBrokerList contains a list of DataguardBroker diff --git a/apis/database/v1alpha1/dataguardbroker_webhook.go b/apis/database/v1alpha1/dataguardbroker_webhook.go index a9d59286..89a9d3fd 100644 --- a/apis/database/v1alpha1/dataguardbroker_webhook.go +++ b/apis/database/v1alpha1/dataguardbroker_webhook.go @@ -39,6 +39,7 @@ package v1alpha1 import ( + "strconv" "strings" dbcommons "github.com/oracle/oracle-database-operator/commons/database" @@ -89,6 +90,10 @@ func (r *DataguardBroker) Default() { r.Spec.ServiceAnnotations["service.beta.kubernetes.io/oci-load-balancer-shape-flex-max"] = "100" } } + + if r.Spec.SetAsPrimaryDatabase != "" { + r.Spec.SetAsPrimaryDatabase = strings.ToUpper(r.Spec.SetAsPrimaryDatabase) + } } // TODO(user): change verbs to "verbs=create;update;delete" if you want to enable deletion validation. @@ -154,6 +159,11 @@ func (r *DataguardBroker) ValidateUpdate(old runtime.Object) (admission.Warnings allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("primaryDatabaseRef"), "cannot be changed")) } + fastStartFailoverStatus, _ := strconv.ParseBool(oldObj.Status.FastStartFailover) + if (fastStartFailoverStatus || r.Spec.FastStartFailover) && r.Spec.SetAsPrimaryDatabase != "" { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("setAsPrimaryDatabase"), "switchover not supported when fastStartFailover is true")) + } if len(allErrs) == 0 { return nil, nil diff --git a/apis/database/v1alpha1/dbcssystem_conversion.go b/apis/database/v1alpha1/dbcssystem_conversion.go new file mode 100644 index 00000000..0aa6a258 --- /dev/null +++ b/apis/database/v1alpha1/dbcssystem_conversion.go @@ -0,0 +1,14 @@ +package v1alpha1 + +import ( + "sigs.k8s.io/controller-runtime/pkg/conversion" +) + +func (src *DbcsSystem) ConvertTo(dst conversion.Hub) error { + return nil +} + +// ConvertFrom converts v1 to v1alpha1 +func (dst *DbcsSystem) ConvertFrom(src conversion.Hub) error { + return nil +} diff --git a/apis/database/v1alpha1/dbcssystem_kms_types.go b/apis/database/v1alpha1/dbcssystem_kms_types.go new file mode 100644 index 00000000..c90726e3 --- /dev/null +++ b/apis/database/v1alpha1/dbcssystem_kms_types.go @@ -0,0 +1,141 @@ +/* +** Copyright (c) 2022-2024 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ +package v1alpha1 + +import "encoding/json" + +type KMSConfig struct { + VaultName string `json:"vaultName,omitempty"` + CompartmentId string `json:"compartmentId,omitempty"` + KeyName string `json:"keyName,omitempty"` + EncryptionAlgo string `json:"encryptionAlgo,omitempty"` + VaultType string `json:"vaultType,omitempty"` +} +type KMSDetailsStatus struct { + VaultId string `json:"vaultId,omitempty"` + ManagementEndpoint string `json:"managementEndpoint,omitempty"` + KeyId string `json:"keyId,omitempty"` + VaultName string `json:"vaultName,omitempty"` + CompartmentId string `json:"compartmentId,omitempty"` + KeyName string `json:"keyName,omitempty"` + EncryptionAlgo string `json:"encryptionAlgo,omitempty"` + VaultType string `json:"vaultType,omitempty"` +} + +const ( + lastSuccessfulKMSConfig = "lastSuccessfulKMSConfig" + lastSuccessfulKMSStatus = "lastSuccessfulKMSStatus" +) + +// GetLastSuccessfulKMSConfig returns the KMS config from the last successful reconciliation. +// Returns nil, nil if there is no lastSuccessfulKMSConfig. +func (dbcs *DbcsSystem) GetLastSuccessfulKMSConfig() (*KMSConfig, error) { + val, ok := dbcs.GetAnnotations()[lastSuccessfulKMSConfig] + if !ok { + return nil, nil + } + + configBytes := []byte(val) + kmsConfig := KMSConfig{} + + err := json.Unmarshal(configBytes, &kmsConfig) + if err != nil { + return nil, err + } + + return &kmsConfig, nil +} + +// GetLastSuccessfulKMSStatus returns the KMS status from the last successful reconciliation. +// Returns nil, nil if there is no lastSuccessfulKMSStatus. +func (dbcs *DbcsSystem) GetLastSuccessfulKMSStatus() (*KMSDetailsStatus, error) { + val, ok := dbcs.GetAnnotations()[lastSuccessfulKMSStatus] + if !ok { + return nil, nil + } + + statusBytes := []byte(val) + kmsStatus := KMSDetailsStatus{} + + err := json.Unmarshal(statusBytes, &kmsStatus) + if err != nil { + return nil, err + } + + return &kmsStatus, nil +} + +// SetLastSuccessfulKMSConfig saves the given KMSConfig to the annotations. +func (dbcs *DbcsSystem) SetLastSuccessfulKMSConfig(kmsConfig *KMSConfig) error { + configBytes, err := json.Marshal(kmsConfig) + if err != nil { + return err + } + + annotations := dbcs.GetAnnotations() + if annotations == nil { + annotations = make(map[string]string) + } + annotations[lastSuccessfulKMSConfig] = string(configBytes) + dbcs.SetAnnotations(annotations) + return nil +} + +// SetLastSuccessfulKMSStatus saves the given KMSDetailsStatus to the annotations. +func (dbcs *DbcsSystem) SetLastSuccessfulKMSStatus(kmsStatus *KMSDetailsStatus) error { + statusBytes, err := json.Marshal(kmsStatus) + if err != nil { + return err + } + + annotations := dbcs.GetAnnotations() + if annotations == nil { + annotations = make(map[string]string) + } + annotations[lastSuccessfulKMSStatus] = string(statusBytes) + dbcs.SetAnnotations(annotations) + // Update KMSDetailsStatus in DbcsSystemStatus + dbcs.Status.KMSDetailsStatus = KMSDetailsStatus{ + VaultName: kmsStatus.VaultName, + CompartmentId: kmsStatus.CompartmentId, + KeyName: kmsStatus.KeyName, + EncryptionAlgo: kmsStatus.EncryptionAlgo, + VaultType: kmsStatus.VaultType, + } + return nil +} diff --git a/apis/database/v1alpha1/dbcssystem_pdbconfig_types.go b/apis/database/v1alpha1/dbcssystem_pdbconfig_types.go new file mode 100644 index 00000000..1b745e09 --- /dev/null +++ b/apis/database/v1alpha1/dbcssystem_pdbconfig_types.go @@ -0,0 +1,83 @@ +/* +** Copyright (c) 2022-2024 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ +package v1alpha1 + +// PDBConfig defines details of PDB struct for DBCS systems +type PDBConfig struct { + // The name for the pluggable database (PDB). The name is unique in the context of a Database. The name must begin with an alphabetic character and can contain a maximum of thirty alphanumeric characters. Special characters are not permitted. The pluggable database name should not be same as the container database name. + PdbName *string `mandatory:"true" json:"pdbName"` + + // The OCID (https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm) of the CDB + // ContainerDatabaseId *string `mandatory:"false" json:"containerDatabaseId"` + + // // A strong password for PDB Admin. The password must be at least nine characters and contain at least two uppercase, two lowercase, two numbers, and two special characters. The special characters must be _, \#, or -. + PdbAdminPassword *string `mandatory:"false" json:"pdbAdminPassword"` + + // // The existing TDE wallet password of the CDB. + TdeWalletPassword *string `mandatory:"false" json:"tdeWalletPassword"` + + // // The locked mode of the pluggable database admin account. If false, the user needs to provide the PDB Admin Password to connect to it. + // // If true, the pluggable database will be locked and user cannot login to it. + ShouldPdbAdminAccountBeLocked *bool `mandatory:"false" json:"shouldPdbAdminAccountBeLocked"` + + // // Free-form tags for this resource. Each tag is a simple key-value pair with no predefined name, type, or namespace. + // // For more information, see Resource Tags (https://docs.cloud.oracle.com/Content/General/Concepts/resourcetags.htm). + // // Example: `{"Department": "Finance"}` + FreeformTags map[string]string `mandatory:"false" json:"freeformTags"` + + // // Defined tags for this resource. Each key is predefined and scoped to a namespace. + // // For more information, see Resource Tags (https://docs.cloud.oracle.com/Content/General/Concepts/resourcetags.htm). + // DefinedTags map[string]map[string]interface{} `mandatory:"false" json:"definedTags"` + + // To specify whether to delete the PDB + IsDelete *bool `mandatory:"false" json:"isDelete,omitempty"` + + // The OCID of the PDB for deletion purposes. + PluggableDatabaseId *string `mandatory:"false" json:"pluggableDatabaseId,omitempty"` +} + +type PDBConfigStatus struct { + PdbName *string `mandatory:"true" json:"pdbName"` + ShouldPdbAdminAccountBeLocked *bool `mandatory:"false" json:"shouldPdbAdminAccountBeLocked"` + FreeformTags map[string]string `mandatory:"false" json:"freeformTags"` + PluggableDatabaseId *string `mandatory:"false" json:"pluggableDatabaseId,omitempty"` + PdbLifecycleState LifecycleState `json:"pdbState,omitempty"` +} +type PDBDetailsStatus struct { + PDBConfigStatus []PDBConfigStatus `json:"pdbConfigStatus,omitempty"` +} diff --git a/apis/database/v1alpha1/dbcssystem_types.go b/apis/database/v1alpha1/dbcssystem_types.go index 37e80a6b..d49fde8c 100644 --- a/apis/database/v1alpha1/dbcssystem_types.go +++ b/apis/database/v1alpha1/dbcssystem_types.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2022 Oracle and/or its affiliates. +** Copyright (c) 2022-2024 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** @@ -42,6 +42,7 @@ import ( metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "github.com/go-logr/logr" dbcsv1 "github.com/oracle/oracle-database-operator/commons/annotations" "sigs.k8s.io/controller-runtime/pkg/client" @@ -52,11 +53,17 @@ import ( // DbcsSystemSpec defines the desired state of DbcsSystem type DbcsSystemSpec struct { - DbSystem DbSystemDetails `json:"dbSystem,omitempty"` - Id *string `json:"id,omitempty"` - OCIConfigMap string `json:"ociConfigMap"` - OCISecret string `json:"ociSecret,omitempty"` - HardLink bool `json:"hardLink,omitempty"` + DbSystem DbSystemDetails `json:"dbSystem,omitempty"` + Id *string `json:"id,omitempty"` + OCIConfigMap *string `json:"ociConfigMap"` + OCISecret *string `json:"ociSecret,omitempty"` + DbClone *DbCloneConfig `json:"dbClone,omitempty"` + HardLink bool `json:"hardLink,omitempty"` + PdbConfigs []PDBConfig `json:"pdbConfigs,omitempty"` + SetupDBCloning bool `json:"setupDBCloning,omitempty"` + DbBackupId *string `json:"dbBackupId,omitempty"` + DatabaseId *string `json:"databaseId,omitempty"` + KMSConfig KMSConfig `json:"kmsConfig,omitempty"` } // DbSystemDetails Spec @@ -66,7 +73,7 @@ type DbSystemDetails struct { AvailabilityDomain string `json:"availabilityDomain"` SubnetId string `json:"subnetId"` Shape string `json:"shape"` - SshPublicKeys []string `json:"sshPublicKeys"` + SshPublicKeys []string `json:"sshPublicKeys,omitempty"` HostName string `json:"hostName"` CpuCoreCount int `json:"cpuCoreCount,omitempty"` FaultDomains []string `json:"faultDomains,omitempty"` @@ -78,8 +85,6 @@ type DbSystemDetails struct { Domain string `json:"domain,omitempty"` InitialDataStorageSizeInGB int `json:"initialDataStorageSizeInGB,omitempty"` ClusterName string `json:"clusterName,omitempty"` - KmsKeyId string `json:"kmsKeyId,omitempty"` - KmsKeyVersionId string `json:"kmsKeyVersionId,omitempty"` DbAdminPaswordSecret string `json:"dbAdminPaswordSecret"` DbName string `json:"dbName,omitempty"` PdbName string `json:"pdbName,omitempty"` @@ -94,9 +99,10 @@ type DbSystemDetails struct { TdeWalletPasswordSecret string `json:"tdeWalletPasswordSecret,omitempty"` Tags map[string]string `json:"tags,omitempty"` DbBackupConfig Backupconfig `json:"dbBackupConfig,omitempty"` + KMSConfig KMSConfig `json:"kmsConfig,omitempty"` } -// DB Backup COnfig Network Struct +// DB Backup Config Network Struct type Backupconfig struct { AutoBackupEnabled *bool `json:"autoBackupEnabled,omitempty"` RecoveryWindowsInDays *int `json:"recoveryWindowsInDays,omitempty"` @@ -121,11 +127,14 @@ type DbcsSystemStatus struct { DataStorageSizeInGBs *int `json:"dataStorageSizeInGBs,omitempty"` RecoStorageSizeInGB *int `json:"recoStorageSizeInGB,omitempty"` - Shape *string `json:"shape,omitempty"` - State LifecycleState `json:"state"` - DbInfo []DbStatus `json:"dbInfo,omitempty"` - Network VmNetworkDetails `json:"network,omitempty"` - WorkRequests []DbWorkrequests `json:"workRequests,omitempty"` + Shape *string `json:"shape,omitempty"` + State LifecycleState `json:"state"` + DbInfo []DbStatus `json:"dbInfo,omitempty"` + Network VmNetworkDetails `json:"network,omitempty"` + WorkRequests []DbWorkrequests `json:"workRequests,omitempty"` + KMSDetailsStatus KMSDetailsStatus `json:"kmsDetailsStatus,omitempty"` + DbCloneStatus DbCloneStatus `json:"dbCloneStatus,omitempty"` + PdbDetailsStatus []PDBDetailsStatus `json:"pdbDetailsStatus,omitempty"` } // DbcsSystemStatus defines the observed state of DbcsSystem @@ -156,17 +165,49 @@ type VmNetworkDetails struct { NetworkSG string `json:"networkSG,omitempty"` } +// DbCloneConfig defines the configuration for the database clone +type DbCloneConfig struct { + DbAdminPaswordSecret string `json:"dbAdminPaswordSecret,omitempty"` + TdeWalletPasswordSecret string `json:"tdeWalletPasswordSecret,omitempty"` + DbName string `json:"dbName"` + HostName string `json:"hostName"` + DbUniqueName string `json:"dbDbUniqueName"` + DisplayName string `json:"displayName"` + LicenseModel string `json:"licenseModel,omitempty"` + Domain string `json:"domain,omitempty"` + SshPublicKeys []string `json:"sshPublicKeys,omitempty"` + SubnetId string `json:"subnetId"` + SidPrefix string `json:"sidPrefix,omitempty"` + InitialDataStorageSizeInGB int `json:"initialDataStorageSizeInGB,omitempty"` + KmsKeyId string `json:"kmsKeyId,omitempty"` + KmsKeyVersionId string `json:"kmsKeyVersionId,omitempty"` + PrivateIp string `json:"privateIp,omitempty"` +} + +// DbCloneStatus defines the observed state of DbClone +type DbCloneStatus struct { + Id *string `json:"id,omitempty"` + DbAdminPaswordSecret string `json:"dbAdminPaswordSecret,omitempty"` + DbName string `json:"dbName,omitempty"` + HostName string `json:"hostName"` + DbUniqueName string `json:"dbDbUniqueName"` + DisplayName string `json:"displayName,omitempty"` + LicenseModel string `json:"licenseModel,omitempty"` + Domain string `json:"domain,omitempty"` + SshPublicKeys []string `json:"sshPublicKeys,omitempty"` + SubnetId string `json:"subnetId,omitempty"` +} + // +kubebuilder:object:root=true // +kubebuilder:subresource:status -// +kubebuilder:resource:path=DbcsSystem,scope=Namespaced +// +kubebuilder:resource:path=dbcssystems,scope=Namespaced // DbcsSystem is the Schema for the dbcssystems API type DbcsSystem struct { metav1.TypeMeta `json:",inline"` metav1.ObjectMeta `json:"metadata,omitempty"` - - Spec DbcsSystemSpec `json:"spec,omitempty"` - Status DbcsSystemStatus `json:"status,omitempty"` + Spec DbcsSystemSpec `json:"spec,omitempty"` + Status DbcsSystemStatus `json:"status,omitempty"` } //+kubebuilder:object:root=true @@ -208,6 +249,25 @@ func (dbcs *DbcsSystem) GetLastSuccessfulSpec() (*DbcsSystemSpec, error) { return &sucSpec, nil } +func (dbcs *DbcsSystem) GetLastSuccessfulSpecWithLog(log logr.Logger) (*DbcsSystemSpec, error) { + val, ok := dbcs.GetAnnotations()[lastSuccessfulSpec] + if !ok { + log.Info("No last successful spec annotation found") + return nil, nil + } + + specBytes := []byte(val) + sucSpec := DbcsSystemSpec{} + + err := json.Unmarshal(specBytes, &sucSpec) + if err != nil { + log.Error(err, "Failed to unmarshal last successful spec") + return nil, err + } + + log.Info("Successfully retrieved last successful spec", "spec", sucSpec) + return &sucSpec, nil +} // UpdateLastSuccessfulSpec updates lastSuccessfulSpec with the current spec. func (dbcs *DbcsSystem) UpdateLastSuccessfulSpec(kubeClient client.Client) error { diff --git a/apis/database/v1alpha1/dbcssystem_webhook.go b/apis/database/v1alpha1/dbcssystem_webhook.go new file mode 100644 index 00000000..dc9f8934 --- /dev/null +++ b/apis/database/v1alpha1/dbcssystem_webhook.go @@ -0,0 +1,98 @@ +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v1alpha1 + +import ( + "k8s.io/apimachinery/pkg/runtime" + ctrl "sigs.k8s.io/controller-runtime" + logf "sigs.k8s.io/controller-runtime/pkg/log" + "sigs.k8s.io/controller-runtime/pkg/webhook" + "sigs.k8s.io/controller-runtime/pkg/webhook/admission" +) + +// log is for logging in this package. +var dbcssystemlog = logf.Log.WithName("dbcssystem-resource") + +func (r *DbcsSystem) SetupWebhookWithManager(mgr ctrl.Manager) error { + return ctrl.NewWebhookManagedBy(mgr). + For(r). + Complete() +} + +// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! + +//+kubebuilder:webhook:path=/mutate-database-oracle-com-v4-dbcssystem,mutating=true,failurePolicy=fail,sideEffects=none,groups=database.oracle.com,resources=dbcssystems,verbs=create;update,versions=v4,name=mdbcssystemv1alpha1.kb.io,admissionReviewVersions=v1 + +var _ webhook.Defaulter = &DbcsSystem{} + +// Default implements webhook.Defaulter so a webhook will be registered for the type +func (r *DbcsSystem) Default() { + dbcssystemlog.Info("default", "name", r.Name) + + // TODO(user): fill in your defaulting logic. +} + +// TODO(user): change verbs to "verbs=create;update;delete" if you want to enable deletion validation. + +// +kubebuilder:webhook:verbs=create;update;delete,path=/validate-database-oracle-com-v4-dbcssystem,mutating=false,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=dbcssystems,versions=v4,name=vdbcssystemv1alpha1.kb.io,admissionReviewVersions=v1 +var _ webhook.Validator = &DbcsSystem{} + +// ValidateCreate implements webhook.Validator so a webhook will be registered for the type +func (r *DbcsSystem) ValidateCreate() (admission.Warnings, error) { + dbcssystemlog.Info("validate create", "name", r.Name) + + // // TODO(user): fill in your validation logic upon object creation. + return nil, nil +} + +// // ValidateUpdate implements webhook.Validator so a webhook will be registered for the type +func (r *DbcsSystem) ValidateUpdate(old runtime.Object) (admission.Warnings, error) { + dbcssystemlog.Info("validate update", "name", r.Name) + + // // TODO(user): fill in your validation logic upon object update. + return nil, nil +} + +// ValidateDelete implements webhook.Validator so a webhook will be registered for the type +func (r *DbcsSystem) ValidateDelete() (admission.Warnings, error) { + dbcssystemlog.Info("validate delete", "name", r.Name) + + // TODO(user): fill in your validation logic upon object deletion. + return nil, nil +} diff --git a/apis/database/v1alpha1/oraclerestdataservice_conversion.go b/apis/database/v1alpha1/oraclerestdataservice_conversion.go new file mode 100644 index 00000000..a16e1ff6 --- /dev/null +++ b/apis/database/v1alpha1/oraclerestdataservice_conversion.go @@ -0,0 +1,14 @@ +package v1alpha1 + +import ( + "sigs.k8s.io/controller-runtime/pkg/conversion" +) + +func (src *OracleRestDataService) ConvertTo(dst conversion.Hub) error { + return nil +} + +// ConvertFrom converts v1 to v1alpha1 +func (dst *OracleRestDataService) ConvertFrom(src conversion.Hub) error { + return nil +} diff --git a/apis/database/v1alpha1/oraclerestdataservice_types.go b/apis/database/v1alpha1/oraclerestdataservice_types.go index 37f61a8a..bab04092 100644 --- a/apis/database/v1alpha1/oraclerestdataservice_types.go +++ b/apis/database/v1alpha1/oraclerestdataservice_types.go @@ -56,17 +56,18 @@ type OracleRestDataServiceSpec struct { NodeSelector map[string]string `json:"nodeSelector,omitempty"` Image OracleRestDataServiceImage `json:"image,omitempty"` OrdsPassword OracleRestDataServicePassword `json:"ordsPassword"` - ApexPassword OracleRestDataServicePassword `json:"apexPassword,omitempty"` AdminPassword OracleRestDataServicePassword `json:"adminPassword"` OrdsUser string `json:"ordsUser,omitempty"` RestEnableSchemas []OracleRestDataServiceRestEnableSchemas `json:"restEnableSchemas,omitempty"` OracleService string `json:"oracleService,omitempty"` ServiceAccountName string `json:"serviceAccountName,omitempty"` Persistence OracleRestDataServicePersistence `json:"persistence,omitempty"` + MongoDbApi bool `json:"mongoDbApi,omitempty"` // +k8s:openapi-gen=true // +kubebuilder:validation:Minimum=1 - Replicas int `json:"replicas,omitempty"` + Replicas int `json:"replicas,omitempty"` + ReadinessCheckPeriod int `json:"readinessCheckPeriod,omitempty"` } // OracleRestDataServicePersistence defines the storage releated params @@ -75,8 +76,9 @@ type OracleRestDataServicePersistence struct { StorageClass string `json:"storageClass,omitempty"` // +kubebuilder:validation:Enum=ReadWriteOnce;ReadWriteMany - AccessMode string `json:"accessMode,omitempty"` - VolumeName string `json:"volumeName,omitempty"` + AccessMode string `json:"accessMode,omitempty"` + VolumeName string `json:"volumeName,omitempty"` + SetWritePermissions *bool `json:"setWritePermissions,omitempty"` } // OracleRestDataServiceImage defines the Image source and pullSecrets for POD @@ -106,17 +108,19 @@ type OracleRestDataServiceRestEnableSchemas struct { type OracleRestDataServiceStatus struct { // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster // Important: Run "make" to regenerate code after modifying this file - Status string `json:"status,omitempty"` - DatabaseApiUrl string `json:"databaseApiUrl,omitempty"` - LoadBalancer string `json:"loadBalancer,omitempty"` - DatabaseRef string `json:"databaseRef,omitempty"` - ServiceIP string `json:"serviceIP,omitempty"` - DatabaseActionsUrl string `json:"databaseActionsUrl,omitempty"` - OrdsInstalled bool `json:"ordsInstalled,omitempty"` - ApexConfigured bool `json:"apexConfigured,omitempty"` - ApxeUrl string `json:"apexUrl,omitempty"` - CommonUsersCreated bool `json:"commonUsersCreated,omitempty"` - Replicas int `json:"replicas,omitempty"` + Status string `json:"status,omitempty"` + DatabaseApiUrl string `json:"databaseApiUrl,omitempty"` + LoadBalancer string `json:"loadBalancer,omitempty"` + DatabaseRef string `json:"databaseRef,omitempty"` + ServiceIP string `json:"serviceIP,omitempty"` + DatabaseActionsUrl string `json:"databaseActionsUrl,omitempty"` + MongoDbApiAccessUrl string `json:"mongoDbApiAccessUrl,omitempty"` + OrdsInstalled bool `json:"ordsInstalled,omitempty"` + ApexConfigured bool `json:"apexConfigured,omitempty"` + ApxeUrl string `json:"apexUrl,omitempty"` + MongoDbApi bool `json:"mongoDbApi,omitempty"` + CommonUsersCreated bool `json:"commonUsersCreated,omitempty"` + Replicas int `json:"replicas,omitempty"` Image OracleRestDataServiceImage `json:"image,omitempty"` } @@ -128,6 +132,7 @@ type OracleRestDataServiceStatus struct { // +kubebuilder:printcolumn:JSONPath=".status.databaseApiUrl",name="Database API URL",type="string" // +kubebuilder:printcolumn:JSONPath=".status.databaseActionsUrl",name="Database Actions URL",type="string" // +kubebuilder:printcolumn:JSONPath=".status.apexUrl",name="Apex URL",type="string" +// +kubebuilder:printcolumn:JSONPath=".status.mongoDbApiAccessUrl",name="MongoDbApi Access URL",type="string" // OracleRestDataService is the Schema for the oraclerestdataservices API type OracleRestDataService struct { diff --git a/apis/database/v1alpha1/oraclerestdataservice_webhook.go b/apis/database/v1alpha1/oraclerestdataservice_webhook.go index bfe3208c..c5ecde1c 100644 --- a/apis/database/v1alpha1/oraclerestdataservice_webhook.go +++ b/apis/database/v1alpha1/oraclerestdataservice_webhook.go @@ -74,9 +74,6 @@ func (r *OracleRestDataService) Default() { if r.Spec.OrdsPassword.KeepSecret == nil { r.Spec.OrdsPassword.KeepSecret = &keepSecret } - if r.Spec.ApexPassword.KeepSecret == nil { - r.Spec.ApexPassword.KeepSecret = &keepSecret - } if r.Spec.AdminPassword.KeepSecret == nil { r.Spec.AdminPassword.KeepSecret = &keepSecret } diff --git a/apis/database/v1alpha1/shardingdatabase_conversion.go b/apis/database/v1alpha1/shardingdatabase_conversion.go new file mode 100644 index 00000000..d8db75ca --- /dev/null +++ b/apis/database/v1alpha1/shardingdatabase_conversion.go @@ -0,0 +1,14 @@ +package v1alpha1 + +import ( + "sigs.k8s.io/controller-runtime/pkg/conversion" +) + +func (src *ShardingDatabase) ConvertTo(dst conversion.Hub) error { + return nil +} + +// ConvertFrom converts v1 to v1alpha1 +func (dst *ShardingDatabase) ConvertFrom(src conversion.Hub) error { + return nil +} diff --git a/apis/database/v1alpha1/shardingdatabase_types.go b/apis/database/v1alpha1/shardingdatabase_types.go index ffc17ab0..ae9066fc 100644 --- a/apis/database/v1alpha1/shardingdatabase_types.go +++ b/apis/database/v1alpha1/shardingdatabase_types.go @@ -68,7 +68,6 @@ type ShardingDatabaseSpec struct { GsmImagePullSecret string `json:"gsmImagePullSecret,omitempty"` // Optional The name of an image pull secret in case of a private docker repository. StagePvcName string `json:"stagePvcName,omitempty"` // the Stagepvc for the backup of cluster PortMappings []PortMapping `json:"portMappings,omitempty"` // Port mappings for the service that is created. The service is created if there is at least - Namespace string `json:"namespace,omitempty"` // Target namespace of the application. IsDebug bool `json:"isDebug,omitempty"` // Optional parameter to enable logining IsExternalSvc bool `json:"isExternalSvc,omitempty"` IsClone bool `json:"isClone,omitempty"` @@ -95,6 +94,7 @@ type ShardingDatabaseSpec struct { FssStorageClass string `json:"fssStorageClass,omitempty"` TdeWalletPvcMountLocation string `json:"tdeWalletPvcMountLocation,omitempty"` DbEdition string `json:"dbEdition,omitempty"` + TopicId string `json:"topicId,omitempty"` } // To understand Metav1.Condition, please refer the link https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1 @@ -215,6 +215,7 @@ type GsmSpec struct { Label string `json:"label,omitempty"` // Optional GSM Label IsDelete string `json:"isDelete,omitempty"` NodeSelector map[string]string `json:"nodeSelector,omitempty"` + PvAnnotations map[string]string `json:"pvAnnotations,omitempty"` PvMatchLabels map[string]string `json:"pvMatchLabels,omitempty"` ImagePulllPolicy *corev1.PullPolicy `json:"imagePullPolicy,omitempty"` Region string `json:"region,omitempty"` @@ -364,6 +365,7 @@ const ( var KubeConfigOnce sync.Once // #const lastSuccessfulSpec = "lastSuccessfulSpec" +const lastSuccessfulSpecOnsInfo = "lastSuccessfulSpeOnsInfo" // GetLastSuccessfulSpec returns spec from the lass successful reconciliation. // Returns nil, nil if there is no lastSuccessfulSpec. @@ -398,6 +400,27 @@ func (shardingv1 *ShardingDatabase) UpdateLastSuccessfulSpec(kubeClient client.C return annsv1.PatchAnnotations(kubeClient, shardingv1, anns) } +// GetLastSuccessfulOnsInfo returns spec from the lass successful reconciliation. +// Returns nil, nil if there is no lastSuccessfulSpec. +func (shardingv1 *ShardingDatabase) GetLastSuccessfulOnsInfo() ([]byte, error) { + val, ok := shardingv1.GetAnnotations()[lastSuccessfulSpecOnsInfo] + if !ok { + return nil, nil + } + specBytes := []byte(val) + return specBytes, nil +} + +// UpdateLastSuccessfulSpec updates lastSuccessfulSpec with the current spec. +func (shardingv1 *ShardingDatabase) UpdateLastSuccessfulSpecOnsInfo(kubeClient client.Client, specBytes []byte) error { + + anns := map[string]string{ + lastSuccessfulSpecOnsInfo: string(specBytes), + } + + return annsv1.PatchAnnotations(kubeClient, shardingv1, anns) +} + func init() { SchemeBuilder.Register(&ShardingDatabase{}, &ShardingDatabaseList{}) } diff --git a/apis/database/v1alpha1/shardingdatabase_webhook.go b/apis/database/v1alpha1/shardingdatabase_webhook.go index 8b91fb0c..4e7ea2e7 100644 --- a/apis/database/v1alpha1/shardingdatabase_webhook.go +++ b/apis/database/v1alpha1/shardingdatabase_webhook.go @@ -62,7 +62,7 @@ func (r *ShardingDatabase) SetupWebhookWithManager(mgr ctrl.Manager) error { // EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! -//+kubebuilder:webhook:path=/mutate-database-oracle-com-v1alpha1-shardingdatabase,mutating=true,failurePolicy=fail,sideEffects=none,groups=database.oracle.com,resources=shardingdatabases,verbs=create;update,versions=v1alpha1,name=mshardingdatabase.kb.io,admissionReviewVersions={v1} +//+kubebuilder:webhook:path=/mutate-database-oracle-com-v1alpha1-shardingdatabase,mutating=true,failurePolicy=fail,sideEffects=none,groups=database.oracle.com,resources=shardingdatabases,verbs=create;update,versions=v1alpha1,name=mshardingdatabasev1alpha1.kb.io,admissionReviewVersions=v1 var _ webhook.Defaulter = &ShardingDatabase{} @@ -87,7 +87,7 @@ func (r *ShardingDatabase) Default() { } // TODO(user): change verbs to "verbs=create;update;delete" if you want to enable deletion validation. -//+kubebuilder:webhook:verbs=create;update;delete,path=/validate-database-oracle-com-v1alpha1-shardingdatabase,mutating=false,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=shardingdatabases,versions=v1alpha1,name=vshardingdatabase.kb.io,admissionReviewVersions={v1} +//+kubebuilder:webhook:verbs=create;update;delete,path=/validate-database-oracle-com-v1alpha1-shardingdatabase,mutating=false,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=shardingdatabases,versions=v1alpha1,name=vshardingdatabasev1alpha1.kb.io,admissionReviewVersions=v1 var _ webhook.Validator = &ShardingDatabase{} @@ -177,6 +177,16 @@ func (r *ShardingDatabase) ValidateCreate() (admission.Warnings, error) { validationErr = append(validationErr, validationErrs1...) } + validationErrs1 = r.validateCatalogName() + if validationErrs1 != nil { + validationErr = append(validationErr, validationErrs1...) + } + + validationErrs1 = r.validateShardName() + if validationErrs1 != nil { + validationErr = append(validationErr, validationErrs1...) + } + // TODO(user): fill in your validation logic upon object creation. if len(validationErr) == 0 { return nil, nil @@ -268,3 +278,37 @@ func (r *ShardingDatabase) validateFreeEdition() field.ErrorList { } return nil } + +func (r *ShardingDatabase) validateShardName() field.ErrorList { + var validationErrs field.ErrorList + + for pindex := range r.Spec.Shard { + if len(r.Spec.Shard[pindex].Name) > 9 { + validationErrs = append(validationErrs, + field.Invalid(field.NewPath("spec").Child("shard").Child("Name"), r.Spec.Shard[pindex].Name, + "Shard Name cannot be greater than 9 characters.")) + } + } + + if len(validationErrs) > 0 { + return validationErrs + } + return nil +} + +func (r *ShardingDatabase) validateCatalogName() field.ErrorList { + var validationErrs field.ErrorList + + for pindex := range r.Spec.Catalog { + if len(r.Spec.Catalog[pindex].Name) > 9 { + validationErrs = append(validationErrs, + field.Invalid(field.NewPath("spec").Child("catalog").Child("Name"), r.Spec.Catalog[pindex].Name, + "Catalog Name cannot be greater than 9 characters.")) + } + } + + if len(validationErrs) > 0 { + return validationErrs + } + return nil +} diff --git a/apis/database/v1alpha1/singleinstancedatabase_conversion.go b/apis/database/v1alpha1/singleinstancedatabase_conversion.go new file mode 100644 index 00000000..76968dce --- /dev/null +++ b/apis/database/v1alpha1/singleinstancedatabase_conversion.go @@ -0,0 +1,14 @@ +package v1alpha1 + +import ( + "sigs.k8s.io/controller-runtime/pkg/conversion" +) + +func (src *SingleInstanceDatabase) ConvertTo(dst conversion.Hub) error { + return nil +} + +// ConvertFrom converts v1 to v1alpha1 +func (dst *SingleInstanceDatabase) ConvertFrom(src conversion.Hub) error { + return nil +} diff --git a/apis/database/v1alpha1/singleinstancedatabase_types.go b/apis/database/v1alpha1/singleinstancedatabase_types.go index 7c6c1ea5..36125d37 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_types.go +++ b/apis/database/v1alpha1/singleinstancedatabase_types.go @@ -70,13 +70,13 @@ type SingleInstanceDatabaseSpec struct { EnableTCPS bool `json:"enableTCPS,omitempty"` TcpsCertRenewInterval string `json:"tcpsCertRenewInterval,omitempty"` TcpsTlsSecret string `json:"tcpsTlsSecret,omitempty"` - DgBrokerConfigured bool `json:"dgBrokerConfigured,omitempty"` PrimaryDatabaseRef string `json:"primaryDatabaseRef,omitempty"` - // +kubebuilder:validation:Enum=primary;standby;clone - CreateAs string `json:"createAs,omitempty"` - ReadinessCheckPeriod int `json:"readinessCheckPeriod,omitempty"` - ServiceAccountName string `json:"serviceAccountName,omitempty"` + // +kubebuilder:validation:Enum=primary;standby;clone;truecache + CreateAs string `json:"createAs,omitempty"` + ReadinessCheckPeriod int `json:"readinessCheckPeriod,omitempty"` + ServiceAccountName string `json:"serviceAccountName,omitempty"` + TrueCacheServices []string `json:"trueCacheServices,omitempty"` // +k8s:openapi-gen=true Replicas int `json:"replicas,omitempty"` @@ -87,6 +87,8 @@ type SingleInstanceDatabaseSpec struct { Persistence SingleInstanceDatabasePersistence `json:"persistence,omitempty"` InitParams *SingleInstanceDatabaseInitParams `json:"initParams,omitempty"` Resources SingleInstanceDatabaseResources `json:"resources,omitempty"` + + ConvertToSnapshotStandby bool `json:"convertToSnapshotStandby,omitempty"` } type SingleInstanceDatabaseResource struct { @@ -145,6 +147,7 @@ type SingleInstanceDatabaseStatus struct { Status string `json:"status,omitempty"` Replicas int `json:"replicas,omitempty"` ReleaseUpdate string `json:"releaseUpdate,omitempty"` + DgBroker *string `json:"dgBroker,omitempty"` // +kubebuilder:default:="false" DatafilesPatched string `json:"datafilesPatched,omitempty"` ConnectString string `json:"connectString,omitempty"` @@ -175,7 +178,6 @@ type SingleInstanceDatabaseStatus struct { CertRenewInterval string `json:"certRenewInterval,omitempty"` ClientWalletLoc string `json:"clientWalletLoc,omitempty"` PrimaryDatabase string `json:"primaryDatabase,omitempty"` - DgBrokerConfigured bool `json:"dgBrokerConfigured,omitempty"` // +kubebuilder:default:="" TcpsTlsSecret string `json:"tcpsTlsSecret"` @@ -187,6 +189,8 @@ type SingleInstanceDatabaseStatus struct { InitParams SingleInstanceDatabaseInitParams `json:"initParams,omitempty"` Persistence SingleInstanceDatabasePersistence `json:"persistence"` + + ConvertToSnapshotStandby bool `json:"convertToSnapshotStandby,omitempty"` } //+kubebuilder:object:root=true diff --git a/apis/database/v1alpha1/singleinstancedatabase_webhook.go b/apis/database/v1alpha1/singleinstancedatabase_webhook.go index 1a47207d..bc095f7c 100644 --- a/apis/database/v1alpha1/singleinstancedatabase_webhook.go +++ b/apis/database/v1alpha1/singleinstancedatabase_webhook.go @@ -135,6 +135,10 @@ func (r *SingleInstanceDatabase) Default() { r.Spec.Replicas = 1 } } + + if r.Spec.TrueCacheServices == nil { + r.Spec.TrueCacheServices = make([]string, 0) + } } // TODO(user): change verbs to "verbs=create;update;delete" if you want to enable deletion validation. @@ -293,6 +297,14 @@ func (r *SingleInstanceDatabase) ValidateCreate() (admission.Warnings, error) { } } + if r.Spec.CreateAs != "truecache" { + if len(r.Spec.TrueCacheServices) > 0 { + allErrs = append(allErrs, + field.Invalid(field.NewPath("spec").Child("trueCacheServices"), r.Spec.TrueCacheServices, + "Creation of trueCacheServices only supported with True Cache instances")) + } + } + if r.Status.FlashBack == "true" && r.Spec.FlashBack != nil && *r.Spec.FlashBack { if r.Spec.ArchiveLog != nil && !*r.Spec.ArchiveLog { allErrs = append(allErrs, @@ -470,7 +482,7 @@ func (r *SingleInstanceDatabase) ValidateUpdate(oldRuntimeObject runtime.Object) // if Db is in a dataguard configuration or referred by Standby databases then Restrict enabling Tcps on the Primary DB if r.Spec.EnableTCPS { - if old.Status.DgBrokerConfigured { + if old.Status.DgBroker != nil { allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("enableTCPS"), "cannot enable tcps as database is in a dataguard configuration")) } else if len(old.Status.StandbyDatabases) != 0 { @@ -509,6 +521,12 @@ func (r *SingleInstanceDatabase) ValidateUpdate(oldRuntimeObject runtime.Object) allErrs = append(allErrs, field.Forbidden(field.NewPath("spec").Child("persistence"), "uninstall ORDS to change Persistence")) } + + if old.Status.Replicas != r.Spec.Replicas && old.Status.DgBroker != nil { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("replicas"), "cannot be updated for a database in a Data Guard configuration")) + } + if len(allErrs) == 0 { return nil, nil } diff --git a/apis/database/v1alpha1/zz_generated.deepcopy.go b/apis/database/v1alpha1/zz_generated.deepcopy.go index 10b34ca7..d0426da8 100644 --- a/apis/database/v1alpha1/zz_generated.deepcopy.go +++ b/apis/database/v1alpha1/zz_generated.deepcopy.go @@ -1,5 +1,4 @@ //go:build !ignore_autogenerated -// +build !ignore_autogenerated /* ** Copyright (c) 2022 Oracle and/or its affiliates. @@ -50,18 +49,18 @@ import ( ) // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *ACDSpec) DeepCopyInto(out *ACDSpec) { +func (in *AcdSpec) DeepCopyInto(out *AcdSpec) { *out = *in - in.K8sACD.DeepCopyInto(&out.K8sACD) - in.OCIACD.DeepCopyInto(&out.OCIACD) + in.K8sAcd.DeepCopyInto(&out.K8sAcd) + in.OciAcd.DeepCopyInto(&out.OciAcd) } -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ACDSpec. -func (in *ACDSpec) DeepCopy() *ACDSpec { +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AcdSpec. +func (in *AcdSpec) DeepCopy() *AcdSpec { if in == nil { return nil } - out := new(ACDSpec) + out := new(AcdSpec) in.DeepCopyInto(out) return out } @@ -327,15 +326,10 @@ func (in *AutonomousDatabaseBackupStatus) DeepCopy() *AutonomousDatabaseBackupSt } // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *AutonomousDatabaseDetails) DeepCopyInto(out *AutonomousDatabaseDetails) { +func (in *AutonomousDatabaseBase) DeepCopyInto(out *AutonomousDatabaseBase) { *out = *in - if in.AutonomousDatabaseOCID != nil { - in, out := &in.AutonomousDatabaseOCID, &out.AutonomousDatabaseOCID - *out = new(string) - **out = **in - } - if in.CompartmentOCID != nil { - in, out := &in.CompartmentOCID, &out.CompartmentOCID + if in.CompartmentId != nil { + in, out := &in.CompartmentId, &out.CompartmentId *out = new(string) **out = **in } @@ -360,11 +354,21 @@ func (in *AutonomousDatabaseDetails) DeepCopyInto(out *AutonomousDatabaseDetails *out = new(int) **out = **in } - if in.CPUCoreCount != nil { - in, out := &in.CPUCoreCount, &out.CPUCoreCount + if in.CpuCoreCount != nil { + in, out := &in.CpuCoreCount, &out.CpuCoreCount *out = new(int) **out = **in } + if in.ComputeCount != nil { + in, out := &in.ComputeCount, &out.ComputeCount + *out = new(float32) + **out = **in + } + if in.OcpuCount != nil { + in, out := &in.OcpuCount, &out.OcpuCount + *out = new(float32) + **out = **in + } in.AdminPassword.DeepCopyInto(&out.AdminPassword) if in.IsAutoScalingEnabled != nil { in, out := &in.IsAutoScalingEnabled, &out.IsAutoScalingEnabled @@ -376,7 +380,41 @@ func (in *AutonomousDatabaseDetails) DeepCopyInto(out *AutonomousDatabaseDetails *out = new(bool) **out = **in } - in.NetworkAccess.DeepCopyInto(&out.NetworkAccess) + if in.IsFreeTier != nil { + in, out := &in.IsFreeTier, &out.IsFreeTier + *out = new(bool) + **out = **in + } + if in.IsAccessControlEnabled != nil { + in, out := &in.IsAccessControlEnabled, &out.IsAccessControlEnabled + *out = new(bool) + **out = **in + } + if in.WhitelistedIps != nil { + in, out := &in.WhitelistedIps, &out.WhitelistedIps + *out = make([]string, len(*in)) + copy(*out, *in) + } + if in.SubnetId != nil { + in, out := &in.SubnetId, &out.SubnetId + *out = new(string) + **out = **in + } + if in.NsgIds != nil { + in, out := &in.NsgIds, &out.NsgIds + *out = make([]string, len(*in)) + copy(*out, *in) + } + if in.PrivateEndpointLabel != nil { + in, out := &in.PrivateEndpointLabel, &out.PrivateEndpointLabel + *out = new(string) + **out = **in + } + if in.IsMtlsConnectionRequired != nil { + in, out := &in.IsMtlsConnectionRequired, &out.IsMtlsConnectionRequired + *out = new(bool) + **out = **in + } if in.FreeformTags != nil { in, out := &in.FreeformTags, &out.FreeformTags *out = make(map[string]string, len(*in)) @@ -384,7 +422,43 @@ func (in *AutonomousDatabaseDetails) DeepCopyInto(out *AutonomousDatabaseDetails (*out)[key] = val } } - in.Wallet.DeepCopyInto(&out.Wallet) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseBase. +func (in *AutonomousDatabaseBase) DeepCopy() *AutonomousDatabaseBase { + if in == nil { + return nil + } + out := new(AutonomousDatabaseBase) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousDatabaseClone) DeepCopyInto(out *AutonomousDatabaseClone) { + *out = *in + in.AutonomousDatabaseBase.DeepCopyInto(&out.AutonomousDatabaseBase) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseClone. +func (in *AutonomousDatabaseClone) DeepCopy() *AutonomousDatabaseClone { + if in == nil { + return nil + } + out := new(AutonomousDatabaseClone) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousDatabaseDetails) DeepCopyInto(out *AutonomousDatabaseDetails) { + *out = *in + in.AutonomousDatabaseBase.DeepCopyInto(&out.AutonomousDatabaseBase) + if in.Id != nil { + in, out := &in.Id, &out.Id + *out = new(string) + **out = **in + } } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseDetails. @@ -525,7 +599,9 @@ func (in *AutonomousDatabaseRestoreStatus) DeepCopy() *AutonomousDatabaseRestore func (in *AutonomousDatabaseSpec) DeepCopyInto(out *AutonomousDatabaseSpec) { *out = *in in.Details.DeepCopyInto(&out.Details) - in.OCIConfig.DeepCopyInto(&out.OCIConfig) + in.Clone.DeepCopyInto(&out.Clone) + in.Wallet.DeepCopyInto(&out.Wallet) + in.OciConfig.DeepCopyInto(&out.OciConfig) if in.HardLink != nil { in, out := &in.HardLink, &out.HardLink *out = new(bool) @@ -607,205 +683,6 @@ func (in *Backupconfig) DeepCopy() *Backupconfig { return out } -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *CDB) DeepCopyInto(out *CDB) { - *out = *in - out.TypeMeta = in.TypeMeta - in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) - in.Spec.DeepCopyInto(&out.Spec) - out.Status = in.Status -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDB. -func (in *CDB) DeepCopy() *CDB { - if in == nil { - return nil - } - out := new(CDB) - in.DeepCopyInto(out) - return out -} - -// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. -func (in *CDB) DeepCopyObject() runtime.Object { - if c := in.DeepCopy(); c != nil { - return c - } - return nil -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *CDBAdminPassword) DeepCopyInto(out *CDBAdminPassword) { - *out = *in - out.Secret = in.Secret -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDBAdminPassword. -func (in *CDBAdminPassword) DeepCopy() *CDBAdminPassword { - if in == nil { - return nil - } - out := new(CDBAdminPassword) - in.DeepCopyInto(out) - return out -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *CDBAdminUser) DeepCopyInto(out *CDBAdminUser) { - *out = *in - out.Secret = in.Secret -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDBAdminUser. -func (in *CDBAdminUser) DeepCopy() *CDBAdminUser { - if in == nil { - return nil - } - out := new(CDBAdminUser) - in.DeepCopyInto(out) - return out -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *CDBList) DeepCopyInto(out *CDBList) { - *out = *in - out.TypeMeta = in.TypeMeta - in.ListMeta.DeepCopyInto(&out.ListMeta) - if in.Items != nil { - in, out := &in.Items, &out.Items - *out = make([]CDB, len(*in)) - for i := range *in { - (*in)[i].DeepCopyInto(&(*out)[i]) - } - } -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDBList. -func (in *CDBList) DeepCopy() *CDBList { - if in == nil { - return nil - } - out := new(CDBList) - in.DeepCopyInto(out) - return out -} - -// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. -func (in *CDBList) DeepCopyObject() runtime.Object { - if c := in.DeepCopy(); c != nil { - return c - } - return nil -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *CDBSecret) DeepCopyInto(out *CDBSecret) { - *out = *in -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDBSecret. -func (in *CDBSecret) DeepCopy() *CDBSecret { - if in == nil { - return nil - } - out := new(CDBSecret) - in.DeepCopyInto(out) - return out -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *CDBSpec) DeepCopyInto(out *CDBSpec) { - *out = *in - out.SysAdminPwd = in.SysAdminPwd - out.CDBAdminUser = in.CDBAdminUser - out.CDBAdminPwd = in.CDBAdminPwd - out.CDBTlsKey = in.CDBTlsKey - out.CDBTlsCrt = in.CDBTlsCrt - out.ORDSPwd = in.ORDSPwd - out.WebServerUser = in.WebServerUser - out.WebServerPwd = in.WebServerPwd - if in.NodeSelector != nil { - in, out := &in.NodeSelector, &out.NodeSelector - *out = make(map[string]string, len(*in)) - for key, val := range *in { - (*out)[key] = val - } - } -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDBSpec. -func (in *CDBSpec) DeepCopy() *CDBSpec { - if in == nil { - return nil - } - out := new(CDBSpec) - in.DeepCopyInto(out) - return out -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *CDBStatus) DeepCopyInto(out *CDBStatus) { - *out = *in -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDBStatus. -func (in *CDBStatus) DeepCopy() *CDBStatus { - if in == nil { - return nil - } - out := new(CDBStatus) - in.DeepCopyInto(out) - return out -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *CDBSysAdminPassword) DeepCopyInto(out *CDBSysAdminPassword) { - *out = *in - out.Secret = in.Secret -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDBSysAdminPassword. -func (in *CDBSysAdminPassword) DeepCopy() *CDBSysAdminPassword { - if in == nil { - return nil - } - out := new(CDBSysAdminPassword) - in.DeepCopyInto(out) - return out -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *CDBTLSCRT) DeepCopyInto(out *CDBTLSCRT) { - *out = *in - out.Secret = in.Secret -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDBTLSCRT. -func (in *CDBTLSCRT) DeepCopy() *CDBTLSCRT { - if in == nil { - return nil - } - out := new(CDBTLSCRT) - in.DeepCopyInto(out) - return out -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *CDBTLSKEY) DeepCopyInto(out *CDBTLSKEY) { - *out = *in - out.Secret = in.Secret -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDBTLSKEY. -func (in *CDBTLSKEY) DeepCopy() *CDBTLSKEY { - if in == nil { - return nil - } - out := new(CDBTLSKEY) - in.DeepCopyInto(out) - return out -} - // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *CatalogSpec) DeepCopyInto(out *CatalogSpec) { *out = *in @@ -898,7 +775,7 @@ func (in *DataguardBroker) DeepCopyInto(out *DataguardBroker) { out.TypeMeta = in.TypeMeta in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) in.Spec.DeepCopyInto(&out.Spec) - out.Status = in.Status + in.Status.DeepCopyInto(&out.Status) } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DataguardBroker. @@ -919,26 +796,6 @@ func (in *DataguardBroker) DeepCopyObject() runtime.Object { return nil } -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *DataguardBrokerFastStartFailOver) DeepCopyInto(out *DataguardBrokerFastStartFailOver) { - *out = *in - if in.Strategy != nil { - in, out := &in.Strategy, &out.Strategy - *out = make([]DataguardBrokerStrategy, len(*in)) - copy(*out, *in) - } -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DataguardBrokerFastStartFailOver. -func (in *DataguardBrokerFastStartFailOver) DeepCopy() *DataguardBrokerFastStartFailOver { - if in == nil { - return nil - } - out := new(DataguardBrokerFastStartFailOver) - in.DeepCopyInto(out) - return out -} - // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *DataguardBrokerList) DeepCopyInto(out *DataguardBrokerList) { *out = *in @@ -993,7 +850,6 @@ func (in *DataguardBrokerSpec) DeepCopyInto(out *DataguardBrokerSpec) { (*out)[key] = val } } - in.FastStartFailOver.DeepCopyInto(&out.FastStartFailOver) } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DataguardBrokerSpec. @@ -1009,6 +865,13 @@ func (in *DataguardBrokerSpec) DeepCopy() *DataguardBrokerSpec { // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *DataguardBrokerStatus) DeepCopyInto(out *DataguardBrokerStatus) { *out = *in + if in.DatabasesInDataguardConfig != nil { + in, out := &in.DatabasesInDataguardConfig, &out.DatabasesInDataguardConfig + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DataguardBrokerStatus. @@ -1022,16 +885,46 @@ func (in *DataguardBrokerStatus) DeepCopy() *DataguardBrokerStatus { } // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *DataguardBrokerStrategy) DeepCopyInto(out *DataguardBrokerStrategy) { +func (in *DbCloneConfig) DeepCopyInto(out *DbCloneConfig) { *out = *in + if in.SshPublicKeys != nil { + in, out := &in.SshPublicKeys, &out.SshPublicKeys + *out = make([]string, len(*in)) + copy(*out, *in) + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DbCloneConfig. +func (in *DbCloneConfig) DeepCopy() *DbCloneConfig { + if in == nil { + return nil + } + out := new(DbCloneConfig) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DbCloneStatus) DeepCopyInto(out *DbCloneStatus) { + *out = *in + if in.Id != nil { + in, out := &in.Id, &out.Id + *out = new(string) + **out = **in + } + if in.SshPublicKeys != nil { + in, out := &in.SshPublicKeys, &out.SshPublicKeys + *out = make([]string, len(*in)) + copy(*out, *in) + } } -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DataguardBrokerStrategy. -func (in *DataguardBrokerStrategy) DeepCopy() *DataguardBrokerStrategy { +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DbCloneStatus. +func (in *DbCloneStatus) DeepCopy() *DbCloneStatus { if in == nil { return nil } - out := new(DataguardBrokerStrategy) + out := new(DbCloneStatus) in.DeepCopyInto(out) return out } @@ -1082,6 +975,7 @@ func (in *DbSystemDetails) DeepCopyInto(out *DbSystemDetails) { } } in.DbBackupConfig.DeepCopyInto(&out.DbBackupConfig) + out.KMSConfig = in.KMSConfig } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DbSystemDetails. @@ -1187,6 +1081,39 @@ func (in *DbcsSystemSpec) DeepCopyInto(out *DbcsSystemSpec) { *out = new(string) **out = **in } + if in.OCIConfigMap != nil { + in, out := &in.OCIConfigMap, &out.OCIConfigMap + *out = new(string) + **out = **in + } + if in.OCISecret != nil { + in, out := &in.OCISecret, &out.OCISecret + *out = new(string) + **out = **in + } + if in.DbClone != nil { + in, out := &in.DbClone, &out.DbClone + *out = new(DbCloneConfig) + (*in).DeepCopyInto(*out) + } + if in.PdbConfigs != nil { + in, out := &in.PdbConfigs, &out.PdbConfigs + *out = make([]PDBConfig, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } + if in.DbBackupId != nil { + in, out := &in.DbBackupId, &out.DbBackupId + *out = new(string) + **out = **in + } + if in.DatabaseId != nil { + in, out := &in.DatabaseId, &out.DatabaseId + *out = new(string) + **out = **in + } + out.KMSConfig = in.KMSConfig } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DbcsSystemSpec. @@ -1242,6 +1169,15 @@ func (in *DbcsSystemStatus) DeepCopyInto(out *DbcsSystemStatus) { (*in)[i].DeepCopyInto(&(*out)[i]) } } + out.KMSDetailsStatus = in.KMSDetailsStatus + in.DbCloneStatus.DeepCopyInto(&out.DbCloneStatus) + if in.PdbDetailsStatus != nil { + in, out := &in.PdbDetailsStatus, &out.PdbDetailsStatus + *out = make([]PDBDetailsStatus, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DbcsSystemStatus. @@ -1349,6 +1285,13 @@ func (in *GsmSpec) DeepCopyInto(out *GsmSpec) { (*out)[key] = val } } + if in.PvAnnotations != nil { + in, out := &in.PvAnnotations, &out.PvAnnotations + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } if in.PvMatchLabels != nil { in, out := &in.PvMatchLabels, &out.PvMatchLabels *out = make(map[string]string, len(*in)) @@ -1418,7 +1361,7 @@ func (in *GsmStatusDetails) DeepCopy() *GsmStatusDetails { } // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *K8sACDSpec) DeepCopyInto(out *K8sACDSpec) { +func (in *K8sAcdSpec) DeepCopyInto(out *K8sAcdSpec) { *out = *in if in.Name != nil { in, out := &in.Name, &out.Name @@ -1427,18 +1370,18 @@ func (in *K8sACDSpec) DeepCopyInto(out *K8sACDSpec) { } } -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new K8sACDSpec. -func (in *K8sACDSpec) DeepCopy() *K8sACDSpec { +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new K8sAcdSpec. +func (in *K8sAcdSpec) DeepCopy() *K8sAcdSpec { if in == nil { return nil } - out := new(K8sACDSpec) + out := new(K8sAcdSpec) in.DeepCopyInto(out) return out } // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *K8sADBBackupSpec) DeepCopyInto(out *K8sADBBackupSpec) { +func (in *K8sAdbBackupSpec) DeepCopyInto(out *K8sAdbBackupSpec) { *out = *in if in.Name != nil { in, out := &in.Name, &out.Name @@ -1447,18 +1390,18 @@ func (in *K8sADBBackupSpec) DeepCopyInto(out *K8sADBBackupSpec) { } } -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new K8sADBBackupSpec. -func (in *K8sADBBackupSpec) DeepCopy() *K8sADBBackupSpec { +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new K8sAdbBackupSpec. +func (in *K8sAdbBackupSpec) DeepCopy() *K8sAdbBackupSpec { if in == nil { return nil } - out := new(K8sADBBackupSpec) + out := new(K8sAdbBackupSpec) in.DeepCopyInto(out) return out } // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *K8sADBSpec) DeepCopyInto(out *K8sADBSpec) { +func (in *K8sAdbSpec) DeepCopyInto(out *K8sAdbSpec) { *out = *in if in.Name != nil { in, out := &in.Name, &out.Name @@ -1467,12 +1410,12 @@ func (in *K8sADBSpec) DeepCopyInto(out *K8sADBSpec) { } } -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new K8sADBSpec. -func (in *K8sADBSpec) DeepCopy() *K8sADBSpec { +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new K8sAdbSpec. +func (in *K8sAdbSpec) DeepCopy() *K8sAdbSpec { if in == nil { return nil } - out := new(K8sADBSpec) + out := new(K8sAdbSpec) in.DeepCopyInto(out) return out } @@ -1498,78 +1441,77 @@ func (in *K8sSecretSpec) DeepCopy() *K8sSecretSpec { } // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *NetworkAccessSpec) DeepCopyInto(out *NetworkAccessSpec) { +func (in *KMSConfig) DeepCopyInto(out *KMSConfig) { *out = *in - if in.IsAccessControlEnabled != nil { - in, out := &in.IsAccessControlEnabled, &out.IsAccessControlEnabled - *out = new(bool) - **out = **in - } - if in.AccessControlList != nil { - in, out := &in.AccessControlList, &out.AccessControlList - *out = make([]string, len(*in)) - copy(*out, *in) - } - in.PrivateEndpoint.DeepCopyInto(&out.PrivateEndpoint) - if in.IsMTLSConnectionRequired != nil { - in, out := &in.IsMTLSConnectionRequired, &out.IsMTLSConnectionRequired - *out = new(bool) - **out = **in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new KMSConfig. +func (in *KMSConfig) DeepCopy() *KMSConfig { + if in == nil { + return nil } + out := new(KMSConfig) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *KMSDetailsStatus) DeepCopyInto(out *KMSDetailsStatus) { + *out = *in } -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new NetworkAccessSpec. -func (in *NetworkAccessSpec) DeepCopy() *NetworkAccessSpec { +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new KMSDetailsStatus. +func (in *KMSDetailsStatus) DeepCopy() *KMSDetailsStatus { if in == nil { return nil } - out := new(NetworkAccessSpec) + out := new(KMSDetailsStatus) in.DeepCopyInto(out) return out } // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *OCIACDSpec) DeepCopyInto(out *OCIACDSpec) { +func (in *OciAcdSpec) DeepCopyInto(out *OciAcdSpec) { *out = *in - if in.OCID != nil { - in, out := &in.OCID, &out.OCID + if in.Id != nil { + in, out := &in.Id, &out.Id *out = new(string) **out = **in } } -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new OCIACDSpec. -func (in *OCIACDSpec) DeepCopy() *OCIACDSpec { +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new OciAcdSpec. +func (in *OciAcdSpec) DeepCopy() *OciAcdSpec { if in == nil { return nil } - out := new(OCIACDSpec) + out := new(OciAcdSpec) in.DeepCopyInto(out) return out } // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *OCIADBSpec) DeepCopyInto(out *OCIADBSpec) { +func (in *OciAdbSpec) DeepCopyInto(out *OciAdbSpec) { *out = *in - if in.OCID != nil { - in, out := &in.OCID, &out.OCID + if in.Ocid != nil { + in, out := &in.Ocid, &out.Ocid *out = new(string) **out = **in } } -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new OCIADBSpec. -func (in *OCIADBSpec) DeepCopy() *OCIADBSpec { +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new OciAdbSpec. +func (in *OciAdbSpec) DeepCopy() *OciAdbSpec { if in == nil { return nil } - out := new(OCIADBSpec) + out := new(OciAdbSpec) in.DeepCopyInto(out) return out } // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *OCIConfigSpec) DeepCopyInto(out *OCIConfigSpec) { +func (in *OciConfigSpec) DeepCopyInto(out *OciConfigSpec) { *out = *in if in.ConfigMapName != nil { in, out := &in.ConfigMapName, &out.ConfigMapName @@ -1583,48 +1525,32 @@ func (in *OCIConfigSpec) DeepCopyInto(out *OCIConfigSpec) { } } -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new OCIConfigSpec. -func (in *OCIConfigSpec) DeepCopy() *OCIConfigSpec { +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new OciConfigSpec. +func (in *OciConfigSpec) DeepCopy() *OciConfigSpec { if in == nil { return nil } - out := new(OCIConfigSpec) + out := new(OciConfigSpec) in.DeepCopyInto(out) return out } // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *OCISecretSpec) DeepCopyInto(out *OCISecretSpec) { +func (in *OciSecretSpec) DeepCopyInto(out *OciSecretSpec) { *out = *in - if in.OCID != nil { - in, out := &in.OCID, &out.OCID + if in.Id != nil { + in, out := &in.Id, &out.Id *out = new(string) **out = **in } } -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new OCISecretSpec. -func (in *OCISecretSpec) DeepCopy() *OCISecretSpec { - if in == nil { - return nil - } - out := new(OCISecretSpec) - in.DeepCopyInto(out) - return out -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *ORDSPassword) DeepCopyInto(out *ORDSPassword) { - *out = *in - out.Secret = in.Secret -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ORDSPassword. -func (in *ORDSPassword) DeepCopy() *ORDSPassword { +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new OciSecretSpec. +func (in *OciSecretSpec) DeepCopy() *OciSecretSpec { if in == nil { return nil } - out := new(ORDSPassword) + out := new(OciSecretSpec) in.DeepCopyInto(out) return out } @@ -1726,6 +1652,11 @@ func (in *OracleRestDataServicePassword) DeepCopy() *OracleRestDataServicePasswo // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *OracleRestDataServicePersistence) DeepCopyInto(out *OracleRestDataServicePersistence) { *out = *in + if in.SetWritePermissions != nil { + in, out := &in.SetWritePermissions, &out.SetWritePermissions + *out = new(bool) + **out = **in + } } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new OracleRestDataServicePersistence. @@ -1772,14 +1703,13 @@ func (in *OracleRestDataServiceSpec) DeepCopyInto(out *OracleRestDataServiceSpec } out.Image = in.Image in.OrdsPassword.DeepCopyInto(&out.OrdsPassword) - in.ApexPassword.DeepCopyInto(&out.ApexPassword) in.AdminPassword.DeepCopyInto(&out.AdminPassword) if in.RestEnableSchemas != nil { in, out := &in.RestEnableSchemas, &out.RestEnableSchemas *out = make([]OracleRestDataServiceRestEnableSchemas, len(*in)) copy(*out, *in) } - out.Persistence = in.Persistence + in.Persistence.DeepCopyInto(&out.Persistence) } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new OracleRestDataServiceSpec. @@ -1809,230 +1739,135 @@ func (in *OracleRestDataServiceStatus) DeepCopy() *OracleRestDataServiceStatus { } // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *PDB) DeepCopyInto(out *PDB) { +func (in *PDBConfig) DeepCopyInto(out *PDBConfig) { *out = *in - out.TypeMeta = in.TypeMeta - in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) - in.Spec.DeepCopyInto(&out.Spec) - out.Status = in.Status -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDB. -func (in *PDB) DeepCopy() *PDB { - if in == nil { - return nil + if in.PdbName != nil { + in, out := &in.PdbName, &out.PdbName + *out = new(string) + **out = **in } - out := new(PDB) - in.DeepCopyInto(out) - return out -} - -// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. -func (in *PDB) DeepCopyObject() runtime.Object { - if c := in.DeepCopy(); c != nil { - return c + if in.PdbAdminPassword != nil { + in, out := &in.PdbAdminPassword, &out.PdbAdminPassword + *out = new(string) + **out = **in } - return nil -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *PDBAdminName) DeepCopyInto(out *PDBAdminName) { - *out = *in - out.Secret = in.Secret -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBAdminName. -func (in *PDBAdminName) DeepCopy() *PDBAdminName { - if in == nil { - return nil + if in.TdeWalletPassword != nil { + in, out := &in.TdeWalletPassword, &out.TdeWalletPassword + *out = new(string) + **out = **in } - out := new(PDBAdminName) - in.DeepCopyInto(out) - return out -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *PDBAdminPassword) DeepCopyInto(out *PDBAdminPassword) { - *out = *in - out.Secret = in.Secret -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBAdminPassword. -func (in *PDBAdminPassword) DeepCopy() *PDBAdminPassword { - if in == nil { - return nil + if in.ShouldPdbAdminAccountBeLocked != nil { + in, out := &in.ShouldPdbAdminAccountBeLocked, &out.ShouldPdbAdminAccountBeLocked + *out = new(bool) + **out = **in } - out := new(PDBAdminPassword) - in.DeepCopyInto(out) - return out -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *PDBList) DeepCopyInto(out *PDBList) { - *out = *in - out.TypeMeta = in.TypeMeta - in.ListMeta.DeepCopyInto(&out.ListMeta) - if in.Items != nil { - in, out := &in.Items, &out.Items - *out = make([]PDB, len(*in)) - for i := range *in { - (*in)[i].DeepCopyInto(&(*out)[i]) + if in.FreeformTags != nil { + in, out := &in.FreeformTags, &out.FreeformTags + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val } } -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBList. -func (in *PDBList) DeepCopy() *PDBList { - if in == nil { - return nil + if in.IsDelete != nil { + in, out := &in.IsDelete, &out.IsDelete + *out = new(bool) + **out = **in } - out := new(PDBList) - in.DeepCopyInto(out) - return out -} - -// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. -func (in *PDBList) DeepCopyObject() runtime.Object { - if c := in.DeepCopy(); c != nil { - return c + if in.PluggableDatabaseId != nil { + in, out := &in.PluggableDatabaseId, &out.PluggableDatabaseId + *out = new(string) + **out = **in } - return nil -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *PDBSecret) DeepCopyInto(out *PDBSecret) { - *out = *in } -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBSecret. -func (in *PDBSecret) DeepCopy() *PDBSecret { +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBConfig. +func (in *PDBConfig) DeepCopy() *PDBConfig { if in == nil { return nil } - out := new(PDBSecret) + out := new(PDBConfig) in.DeepCopyInto(out) return out } // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *PDBSpec) DeepCopyInto(out *PDBSpec) { +func (in *PDBConfigStatus) DeepCopyInto(out *PDBConfigStatus) { *out = *in - out.PDBTlsKey = in.PDBTlsKey - out.PDBTlsCrt = in.PDBTlsCrt - out.PDBTlsCat = in.PDBTlsCat - out.AdminName = in.AdminName - out.AdminPwd = in.AdminPwd - out.WebServerUsr = in.WebServerUsr - out.WebServerPwd = in.WebServerPwd - if in.ReuseTempFile != nil { - in, out := &in.ReuseTempFile, &out.ReuseTempFile - *out = new(bool) - **out = **in - } - if in.UnlimitedStorage != nil { - in, out := &in.UnlimitedStorage, &out.UnlimitedStorage - *out = new(bool) - **out = **in - } - if in.AsClone != nil { - in, out := &in.AsClone, &out.AsClone - *out = new(bool) + if in.PdbName != nil { + in, out := &in.PdbName, &out.PdbName + *out = new(string) **out = **in } - if in.TDEImport != nil { - in, out := &in.TDEImport, &out.TDEImport + if in.ShouldPdbAdminAccountBeLocked != nil { + in, out := &in.ShouldPdbAdminAccountBeLocked, &out.ShouldPdbAdminAccountBeLocked *out = new(bool) **out = **in } - if in.TDEExport != nil { - in, out := &in.TDEExport, &out.TDEExport - *out = new(bool) - **out = **in + if in.FreeformTags != nil { + in, out := &in.FreeformTags, &out.FreeformTags + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } } - out.TDEPassword = in.TDEPassword - out.TDESecret = in.TDESecret - if in.GetScript != nil { - in, out := &in.GetScript, &out.GetScript - *out = new(bool) + if in.PluggableDatabaseId != nil { + in, out := &in.PluggableDatabaseId, &out.PluggableDatabaseId + *out = new(string) **out = **in } } -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBSpec. -func (in *PDBSpec) DeepCopy() *PDBSpec { - if in == nil { - return nil - } - out := new(PDBSpec) - in.DeepCopyInto(out) - return out -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *PDBStatus) DeepCopyInto(out *PDBStatus) { - *out = *in -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBStatus. -func (in *PDBStatus) DeepCopy() *PDBStatus { +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBConfigStatus. +func (in *PDBConfigStatus) DeepCopy() *PDBConfigStatus { if in == nil { return nil } - out := new(PDBStatus) + out := new(PDBConfigStatus) in.DeepCopyInto(out) return out } // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *PDBTLSCAT) DeepCopyInto(out *PDBTLSCAT) { +func (in *PDBDetailsStatus) DeepCopyInto(out *PDBDetailsStatus) { *out = *in - out.Secret = in.Secret -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBTLSCAT. -func (in *PDBTLSCAT) DeepCopy() *PDBTLSCAT { - if in == nil { - return nil + if in.PDBConfigStatus != nil { + in, out := &in.PDBConfigStatus, &out.PDBConfigStatus + *out = make([]PDBConfigStatus, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } } - out := new(PDBTLSCAT) - in.DeepCopyInto(out) - return out -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *PDBTLSCRT) DeepCopyInto(out *PDBTLSCRT) { - *out = *in - out.Secret = in.Secret } -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBTLSCRT. -func (in *PDBTLSCRT) DeepCopy() *PDBTLSCRT { +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBDetailsStatus. +func (in *PDBDetailsStatus) DeepCopy() *PDBDetailsStatus { if in == nil { return nil } - out := new(PDBTLSCRT) + out := new(PDBDetailsStatus) in.DeepCopyInto(out) return out } // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *PDBTLSKEY) DeepCopyInto(out *PDBTLSKEY) { +func (in *PasswordSpec) DeepCopyInto(out *PasswordSpec) { *out = *in - out.Secret = in.Secret + in.K8sSecret.DeepCopyInto(&out.K8sSecret) + in.OciSecret.DeepCopyInto(&out.OciSecret) } -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBTLSKEY. -func (in *PDBTLSKEY) DeepCopy() *PDBTLSKEY { +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PasswordSpec. +func (in *PasswordSpec) DeepCopy() *PasswordSpec { if in == nil { return nil } - out := new(PDBTLSKEY) + out := new(PasswordSpec) in.DeepCopyInto(out) return out } // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *PITSpec) DeepCopyInto(out *PITSpec) { +func (in *PitSpec) DeepCopyInto(out *PitSpec) { *out = *in if in.Timestamp != nil { in, out := &in.Timestamp, &out.Timestamp @@ -2041,29 +1876,12 @@ func (in *PITSpec) DeepCopyInto(out *PITSpec) { } } -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PITSpec. -func (in *PITSpec) DeepCopy() *PITSpec { - if in == nil { - return nil - } - out := new(PITSpec) - in.DeepCopyInto(out) - return out -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *PasswordSpec) DeepCopyInto(out *PasswordSpec) { - *out = *in - in.K8sSecret.DeepCopyInto(&out.K8sSecret) - in.OCISecret.DeepCopyInto(&out.OCISecret) -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PasswordSpec. -func (in *PasswordSpec) DeepCopy() *PasswordSpec { +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PitSpec. +func (in *PitSpec) DeepCopy() *PitSpec { if in == nil { return nil } - out := new(PasswordSpec) + out := new(PitSpec) in.DeepCopyInto(out) return out } @@ -2083,36 +1901,6 @@ func (in *PortMapping) DeepCopy() *PortMapping { return out } -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *PrivateEndpointSpec) DeepCopyInto(out *PrivateEndpointSpec) { - *out = *in - if in.SubnetOCID != nil { - in, out := &in.SubnetOCID, &out.SubnetOCID - *out = new(string) - **out = **in - } - if in.NsgOCIDs != nil { - in, out := &in.NsgOCIDs, &out.NsgOCIDs - *out = make([]string, len(*in)) - copy(*out, *in) - } - if in.HostnamePrefix != nil { - in, out := &in.HostnamePrefix, &out.HostnamePrefix - *out = new(string) - **out = **in - } -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PrivateEndpointSpec. -func (in *PrivateEndpointSpec) DeepCopy() *PrivateEndpointSpec { - if in == nil { - return nil - } - out := new(PrivateEndpointSpec) - in.DeepCopyInto(out) - return out -} - // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *SecretDetails) DeepCopyInto(out *SecretDetails) { *out = *in @@ -2535,6 +2323,11 @@ func (in *SingleInstanceDatabaseSpec) DeepCopyInto(out *SingleInstanceDatabaseSp *out = new(bool) **out = **in } + if in.TrueCacheServices != nil { + in, out := &in.TrueCacheServices, &out.TrueCacheServices + *out = make([]string, len(*in)) + copy(*out, *in) + } if in.NodeSelector != nil { in, out := &in.NodeSelector, &out.NodeSelector *out = make(map[string]string, len(*in)) @@ -2571,6 +2364,11 @@ func (in *SingleInstanceDatabaseStatus) DeepCopyInto(out *SingleInstanceDatabase *out = make([]string, len(*in)) copy(*out, *in) } + if in.DgBroker != nil { + in, out := &in.DgBroker, &out.DgBroker + *out = new(string) + **out = **in + } if in.StandbyDatabases != nil { in, out := &in.StandbyDatabases, &out.StandbyDatabases *out = make(map[string]string, len(*in)) @@ -2602,7 +2400,7 @@ func (in *SingleInstanceDatabaseStatus) DeepCopy() *SingleInstanceDatabaseStatus // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *SourceSpec) DeepCopyInto(out *SourceSpec) { *out = *in - in.K8sADBBackup.DeepCopyInto(&out.K8sADBBackup) + in.K8sAdbBackup.DeepCopyInto(&out.K8sAdbBackup) in.PointInTime.DeepCopyInto(&out.PointInTime) } @@ -2616,43 +2414,11 @@ func (in *SourceSpec) DeepCopy() *SourceSpec { return out } -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *TDEPwd) DeepCopyInto(out *TDEPwd) { - *out = *in - out.Secret = in.Secret -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new TDEPwd. -func (in *TDEPwd) DeepCopy() *TDEPwd { - if in == nil { - return nil - } - out := new(TDEPwd) - in.DeepCopyInto(out) - return out -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *TDESecret) DeepCopyInto(out *TDESecret) { - *out = *in - out.Secret = in.Secret -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new TDESecret. -func (in *TDESecret) DeepCopy() *TDESecret { - if in == nil { - return nil - } - out := new(TDESecret) - in.DeepCopyInto(out) - return out -} - // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *TargetSpec) DeepCopyInto(out *TargetSpec) { *out = *in - in.K8sADB.DeepCopyInto(&out.K8sADB) - in.OCIADB.DeepCopyInto(&out.OCIADB) + in.K8sAdb.DeepCopyInto(&out.K8sAdb) + in.OciAdb.DeepCopyInto(&out.OciAdb) } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new TargetSpec. @@ -2720,67 +2486,3 @@ func (in *WalletSpec) DeepCopy() *WalletSpec { in.DeepCopyInto(out) return out } - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *WebServerPassword) DeepCopyInto(out *WebServerPassword) { - *out = *in - out.Secret = in.Secret -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new WebServerPassword. -func (in *WebServerPassword) DeepCopy() *WebServerPassword { - if in == nil { - return nil - } - out := new(WebServerPassword) - in.DeepCopyInto(out) - return out -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *WebServerPasswordPDB) DeepCopyInto(out *WebServerPasswordPDB) { - *out = *in - out.Secret = in.Secret -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new WebServerPasswordPDB. -func (in *WebServerPasswordPDB) DeepCopy() *WebServerPasswordPDB { - if in == nil { - return nil - } - out := new(WebServerPasswordPDB) - in.DeepCopyInto(out) - return out -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *WebServerUser) DeepCopyInto(out *WebServerUser) { - *out = *in - out.Secret = in.Secret -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new WebServerUser. -func (in *WebServerUser) DeepCopy() *WebServerUser { - if in == nil { - return nil - } - out := new(WebServerUser) - in.DeepCopyInto(out) - return out -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *WebServerUserPDB) DeepCopyInto(out *WebServerUserPDB) { - *out = *in - out.Secret = in.Secret -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new WebServerUserPDB. -func (in *WebServerUserPDB) DeepCopy() *WebServerUserPDB { - if in == nil { - return nil - } - out := new(WebServerUserPDB) - in.DeepCopyInto(out) - return out -} diff --git a/apis/database/v4/adbfamily_common_spec.go b/apis/database/v4/adbfamily_common_spec.go new file mode 100644 index 00000000..87434852 --- /dev/null +++ b/apis/database/v4/adbfamily_common_spec.go @@ -0,0 +1,67 @@ +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v4 + +// LastSuccessfulSpec is an annotation key which maps to the value of last successful spec +const LastSuccessfulSpec string = "lastSuccessfulSpec" + +/************************ +* OCI config +************************/ +type OciConfigSpec struct { + ConfigMapName *string `json:"configMapName,omitempty"` + SecretName *string `json:"secretName,omitempty"` +} + +/************************ +* ADB spec +************************/ +type K8sAdbSpec struct { + Name *string `json:"name,omitempty"` +} + +type OciAdbSpec struct { + OCID *string `json:"ocid,omitempty"` +} + +// TargetSpec defines the spec of the target for backup/restore runs. +type TargetSpec struct { + K8sAdb K8sAdbSpec `json:"k8sADB,omitempty"` + OciAdb OciAdbSpec `json:"ociADB,omitempty"` +} diff --git a/apis/database/v1alpha1/adbfamily_common_utils.go b/apis/database/v4/adbfamily_utils.go similarity index 87% rename from apis/database/v1alpha1/adbfamily_common_utils.go rename to apis/database/v4/adbfamily_utils.go index d4d3ae9f..380dab35 100644 --- a/apis/database/v1alpha1/adbfamily_common_utils.go +++ b/apis/database/v4/adbfamily_utils.go @@ -36,61 +36,20 @@ ** SOFTWARE. */ -package v1alpha1 +package v4 import ( "errors" "reflect" "time" - metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" - "sigs.k8s.io/controller-runtime/pkg/client" - "github.com/oracle/oci-go-sdk/v65/common" "github.com/oracle/oci-go-sdk/v65/database" "github.com/oracle/oci-go-sdk/v65/workrequests" ) -// LastSuccessfulSpec is an annotation key which maps to the value of last successful spec -const LastSuccessfulSpec string = "lastSuccessfulSpec" - -// File the meta condition and return the meta view -func CreateMetaCondition(obj client.Object, err error, lifecycleState string, stateMsg string) metav1.Condition { - - return metav1.Condition{ - Type: lifecycleState, - LastTransitionTime: metav1.Now(), - ObservedGeneration: obj.GetGeneration(), - Reason: stateMsg, - Message: err.Error(), - Status: metav1.ConditionTrue, - } -} - -/************************ -* OCI config -************************/ -type OCIConfigSpec struct { - ConfigMapName *string `json:"configMapName,omitempty"` - SecretName *string `json:"secretName,omitempty"` -} - -/************************ -* ADB spec -************************/ -type K8sADBSpec struct { - Name *string `json:"name,omitempty"` -} - -type OCIADBSpec struct { - OCID *string `json:"ocid,omitempty"` -} - -// TargetSpec defines the spec of the target for backup/restore runs. -type TargetSpec struct { - K8sADB K8sADBSpec `json:"k8sADB,omitempty"` - OCIADB OCIADBSpec `json:"ociADB,omitempty"` -} +// This file contains the util functions that are shared by specs in both +// apis/database/v1alpha1 and apis/database/v4. /************************** * Remove Unchanged Fields @@ -99,7 +58,7 @@ type TargetSpec struct { // removeUnchangedFields removes the unchanged fields in the struct and returns if the struct is changed. // lastSpec should be a derefereced struct that is the last successful spec, e.g. AutonomousDatabaseSpec. // curSpec should be a pointer pointing to the struct that is being proccessed, e.g., *AutonomousDatabaseSpec. -func removeUnchangedFields(lastSpec interface{}, curSpec interface{}) (bool, error) { +func RemoveUnchangedFields(lastSpec interface{}, curSpec interface{}) (bool, error) { if reflect.ValueOf(lastSpec).Kind() != reflect.Struct { return false, errors.New("lastSpec should be a struct") } @@ -217,7 +176,7 @@ func FormatSDKTime(sdkTime *common.SDKTime) string { return time.Format(displayFormat) } -func parseDisplayTime(val string) (*common.SDKTime, error) { +func ParseDisplayTime(val string) (*common.SDKTime, error) { parsedTime, err := time.Parse(displayFormat, val) if err != nil { return nil, err @@ -229,7 +188,7 @@ func parseDisplayTime(val string) (*common.SDKTime, error) { /************************ * LifecycleState check ************************/ -func IsADBIntermediateState(state database.AutonomousDatabaseLifecycleStateEnum) bool { +func IsAdbIntermediateState(state database.AutonomousDatabaseLifecycleStateEnum) bool { if state == database.AutonomousDatabaseLifecycleStateProvisioning || state == database.AutonomousDatabaseLifecycleStateUpdating || state == database.AutonomousDatabaseLifecycleStateScaleInProgress || @@ -248,7 +207,7 @@ func IsADBIntermediateState(state database.AutonomousDatabaseLifecycleStateEnum) return false } -func ValidADBTerminateState(state database.AutonomousDatabaseLifecycleStateEnum) bool { +func CanBeTerminated(state database.AutonomousDatabaseLifecycleStateEnum) bool { if state == database.AutonomousDatabaseLifecycleStateProvisioning || state == database.AutonomousDatabaseLifecycleStateAvailable || state == database.AutonomousDatabaseLifecycleStateStopped || diff --git a/apis/database/v4/autonomouscontainerdatabase_types.go b/apis/database/v4/autonomouscontainerdatabase_types.go new file mode 100644 index 00000000..be9cc615 --- /dev/null +++ b/apis/database/v4/autonomouscontainerdatabase_types.go @@ -0,0 +1,226 @@ +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v4 + +import ( + "encoding/json" + "reflect" + + "github.com/oracle/oci-go-sdk/v65/database" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! +// NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized. + +// name of our custom finalizer +const ACDFinalizer = "database.oracle.com/acd-finalizer" + +type AcdActionEnum string + +const ( + AcdActionBlank AcdActionEnum = "" + AcdActionRestart AcdActionEnum = "RESTART" + AcdActionTerminate AcdActionEnum = "TERMINATE" +) + +func GetAcdActionEnumFromString(val string) (AcdActionEnum, bool) { + var mappingAcdActionEnum = map[string]AcdActionEnum{ + "RESTART": AcdActionRestart, + "TERMINATE": AcdActionTerminate, + "": AcdActionBlank, + } + + enum, ok := mappingAcdActionEnum[val] + return enum, ok +} + +// AutonomousContainerDatabaseSpec defines the desired state of AutonomousContainerDatabase +type AutonomousContainerDatabaseSpec struct { + // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster + // Important: Run "make" to regenerate code after modifying this file + AutonomousContainerDatabaseOCID *string `json:"autonomousContainerDatabaseOCID,omitempty"` + CompartmentOCID *string `json:"compartmentOCID,omitempty"` + DisplayName *string `json:"displayName,omitempty"` + AutonomousExadataVMClusterOCID *string `json:"autonomousExadataVMClusterOCID,omitempty"` + // +kubebuilder:validation:Enum:="RELEASE_UPDATES";"RELEASE_UPDATE_REVISIONS" + PatchModel database.AutonomousContainerDatabasePatchModelEnum `json:"patchModel,omitempty"` + // +kubebuilder:validation:Enum:="SYNC";"RESTART";"TERMINATE" + Action AcdActionEnum `json:"action,omitempty"` + FreeformTags map[string]string `json:"freeformTags,omitempty"` + + OCIConfig OciConfigSpec `json:"ociConfig,omitempty"` + // +kubebuilder:default:=false + HardLink *bool `json:"hardLink,omitempty"` +} + +// AutonomousContainerDatabaseStatus defines the observed state of AutonomousContainerDatabase +type AutonomousContainerDatabaseStatus struct { + // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster + // Important: Run "make" to regenerate code after modifying this file + LifecycleState database.AutonomousContainerDatabaseLifecycleStateEnum `json:"lifecycleState"` + TimeCreated string `json:"timeCreated,omitempty"` +} + +//+kubebuilder:object:root=true +//+kubebuilder:subresource:status +// +kubebuilder:resource:shortName="acd";"acds" +// +kubebuilder:printcolumn:JSONPath=".spec.displayName",name="DisplayName",type=string +// +kubebuilder:printcolumn:JSONPath=".status.lifecycleState",name="State",type=string +// +kubebuilder:printcolumn:JSONPath=".status.timeCreated",name="Created",type=string +// +kubebuilder:storageversion + +// AutonomousContainerDatabase is the Schema for the autonomouscontainerdatabases API +type AutonomousContainerDatabase struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + Spec AutonomousContainerDatabaseSpec `json:"spec,omitempty"` + Status AutonomousContainerDatabaseStatus `json:"status,omitempty"` +} + +//+kubebuilder:object:root=true + +// AutonomousContainerDatabaseList contains a list of AutonomousContainerDatabase +type AutonomousContainerDatabaseList struct { + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata,omitempty"` + Items []AutonomousContainerDatabase `json:"items"` +} + +func init() { + SchemeBuilder.Register(&AutonomousContainerDatabase{}, &AutonomousContainerDatabaseList{}) +} + +// Implement conversion.Hub interface, which means any resource version can convert into v4 +func (*AutonomousContainerDatabase) Hub() {} + +// GetLastSuccessfulSpec returns spec from the lass successful reconciliation. +// Returns nil, nil if there is no lastSuccessfulSpec. +func (acd *AutonomousContainerDatabase) GetLastSuccessfulSpec() (*AutonomousContainerDatabaseSpec, error) { + val, ok := acd.GetAnnotations()[LastSuccessfulSpec] + if !ok { + return nil, nil + } + + specBytes := []byte(val) + sucSpec := AutonomousContainerDatabaseSpec{} + + err := json.Unmarshal(specBytes, &sucSpec) + if err != nil { + return nil, err + } + + return &sucSpec, nil +} + +func (acd *AutonomousContainerDatabase) UpdateLastSuccessfulSpec() error { + specBytes, err := json.Marshal(acd.Spec) + if err != nil { + return err + } + + anns := acd.GetAnnotations() + + if anns == nil { + anns = map[string]string{ + LastSuccessfulSpec: string(specBytes), + } + } else { + anns[LastSuccessfulSpec] = string(specBytes) + } + + acd.SetAnnotations(anns) + + return nil +} + +// UpdateStatusFromOCIACD updates the status subresource +func (acd *AutonomousContainerDatabase) UpdateStatusFromOCIACD(ociObj database.AutonomousContainerDatabase) { + acd.Status.LifecycleState = ociObj.LifecycleState + acd.Status.TimeCreated = FormatSDKTime(ociObj.TimeCreated) +} + +// UpdateFromOCIADB updates the attributes using database.AutonomousContainerDatabase object +func (acd *AutonomousContainerDatabase) UpdateFromOCIACD(ociObj database.AutonomousContainerDatabase) (specChanged bool) { + oldACD := acd.DeepCopy() + + /*********************************** + * update the spec + ***********************************/ + acd.Spec.Action = AcdActionBlank + acd.Spec.AutonomousContainerDatabaseOCID = ociObj.Id + acd.Spec.CompartmentOCID = ociObj.CompartmentId + acd.Spec.DisplayName = ociObj.DisplayName + acd.Spec.AutonomousExadataVMClusterOCID = ociObj.CloudAutonomousVmClusterId + acd.Spec.PatchModel = ociObj.PatchModel + + // special case: an emtpy map will be nil after unmarshalling while the OCI always returns an emty map. + if len(ociObj.FreeformTags) != 0 { + acd.Spec.FreeformTags = ociObj.FreeformTags + } else { + acd.Spec.FreeformTags = nil + } + + /*********************************** + * update the status subresource + ***********************************/ + acd.UpdateStatusFromOCIACD(ociObj) + + return !reflect.DeepEqual(oldACD.Spec, acd.Spec) +} + +// RemoveUnchangedSpec removes the unchanged fields in spec, and returns if the spec has been changed. +func (acd *AutonomousContainerDatabase) RemoveUnchangedSpec(prevSpec AutonomousContainerDatabaseSpec) (bool, error) { + changed, err := RemoveUnchangedFields(prevSpec, &acd.Spec) + if err != nil { + return changed, err + } + + return changed, nil +} + +// A helper function which is useful for debugging. The function prints out a structural JSON format. +func (acd *AutonomousContainerDatabase) String() (string, error) { + out, err := json.MarshalIndent(acd, "", " ") + if err != nil { + return "", err + } + return string(out), nil +} diff --git a/apis/database/v4/autonomouscontainerdatabase_webhook.go b/apis/database/v4/autonomouscontainerdatabase_webhook.go new file mode 100644 index 00000000..9fcb9d8b --- /dev/null +++ b/apis/database/v4/autonomouscontainerdatabase_webhook.go @@ -0,0 +1,110 @@ +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v4 + +import ( + apierrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/runtime/schema" + "k8s.io/apimachinery/pkg/util/validation/field" + ctrl "sigs.k8s.io/controller-runtime" + logf "sigs.k8s.io/controller-runtime/pkg/log" + "sigs.k8s.io/controller-runtime/pkg/webhook" + "sigs.k8s.io/controller-runtime/pkg/webhook/admission" +) + +// log is for logging in this package. +var autonomouscontainerdatabaselog = logf.Log.WithName("autonomouscontainerdatabase-resource") + +func (r *AutonomousContainerDatabase) SetupWebhookWithManager(mgr ctrl.Manager) error { + return ctrl.NewWebhookManagedBy(mgr). + For(r). + Complete() +} + +//+kubebuilder:webhook:verbs=create;update,path=/validate-database-oracle-com-v4-autonomouscontainerdatabase,mutating=false,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=autonomouscontainerdatabases,versions=v4,name=vautonomouscontainerdatabasev4.kb.io,admissionReviewVersions=v1 + +var _ webhook.Validator = &AutonomousContainerDatabase{} + +// ValidateCreate implements webhook.Validator so a webhook will be registered for the type +func (r *AutonomousContainerDatabase) ValidateCreate() (admission.Warnings, error) { + autonomouscontainerdatabaselog.Info("validate create", "name", r.Name) + return nil, nil +} + +// ValidateUpdate implements webhook.Validator so a webhook will be registered for the type +func (r *AutonomousContainerDatabase) ValidateUpdate(old runtime.Object) (admission.Warnings, error) { + var allErrs field.ErrorList + var oldACD *AutonomousContainerDatabase = old.(*AutonomousContainerDatabase) + + autonomouscontainerdatabaselog.Info("validate update", "name", r.Name) + + // skip the update of adding ADB OCID or binding + if oldACD.Status.LifecycleState == "" { + return nil, nil + } + + // cannot update when the old state is in intermediate state, except for the terminate operatrion + var copiedSpec *AutonomousContainerDatabaseSpec = r.Spec.DeepCopy() + changed, err := RemoveUnchangedFields(oldACD.Spec, copiedSpec) + if err != nil { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec"), err.Error())) + } + if IsACDIntermediateState(oldACD.Status.LifecycleState) && changed { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec"), + "cannot change the spec when the lifecycleState is in an intermdeiate state")) + } + + if len(allErrs) == 0 { + return nil, nil + } + return nil, apierrors.NewInvalid( + schema.GroupKind{Group: "database.oracle.com", Kind: "AutonomousContainerDatabase"}, + r.Name, allErrs) +} + +// ValidateDelete implements webhook.Validator so a webhook will be registered for the type +func (r *AutonomousContainerDatabase) ValidateDelete() (admission.Warnings, error) { + autonomouscontainerdatabaselog.Info("validate delete", "name", r.Name) + + // TODO(user): fill in your validation logic upon object deletion. + return nil, nil +} diff --git a/apis/database/v4/autonomousdatabase_types.go b/apis/database/v4/autonomousdatabase_types.go new file mode 100644 index 00000000..628dd882 --- /dev/null +++ b/apis/database/v4/autonomousdatabase_types.go @@ -0,0 +1,393 @@ +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v4 + +import ( + "encoding/json" + "reflect" + + "github.com/oracle/oci-go-sdk/v65/database" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! +// NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized. + +// AutonomousDatabaseSpec defines the desired state of AutonomousDatabase +// Important: Run "make" to regenerate code after modifying this file +type AutonomousDatabaseSpec struct { + // +kubebuilder:validation:Enum:="";Create;Sync;Update;Stop;Start;Terminate;Clone + Action string `json:"action"` + Details AutonomousDatabaseDetails `json:"details,omitempty"` + Clone AutonomousDatabaseClone `json:"clone,omitempty"` + Wallet WalletSpec `json:"wallet,omitempty"` + OciConfig OciConfigSpec `json:"ociConfig,omitempty"` + // +kubebuilder:default:=false + HardLink *bool `json:"hardLink,omitempty"` +} + +type AutonomousDatabaseDetails struct { + AutonomousDatabaseBase `json:",inline"` + Id *string `json:"id,omitempty"` +} + +type AutonomousDatabaseClone struct { + AutonomousDatabaseBase `json:",inline"` + // +kubebuilder:validation:Enum:="FULL";"METADATA" + CloneType database.CreateAutonomousDatabaseCloneDetailsCloneTypeEnum `json:"cloneType,omitempty"` +} + +// AutonomousDatabaseBase defines the detail information of AutonomousDatabase, corresponding to oci-go-sdk/database/AutonomousDatabase +type AutonomousDatabaseBase struct { + CompartmentId *string `json:"compartmentId,omitempty"` + AutonomousContainerDatabase AcdSpec `json:"autonomousContainerDatabase,omitempty"` + DisplayName *string `json:"displayName,omitempty"` + DbName *string `json:"dbName,omitempty"` + // +kubebuilder:validation:Enum:="OLTP";"DW";"AJD";"APEX" + DbWorkload database.AutonomousDatabaseDbWorkloadEnum `json:"dbWorkload,omitempty"` + // +kubebuilder:validation:Enum:="LICENSE_INCLUDED";"BRING_YOUR_OWN_LICENSE" + LicenseModel database.AutonomousDatabaseLicenseModelEnum `json:"licenseModel,omitempty"` + DbVersion *string `json:"dbVersion,omitempty"` + DataStorageSizeInTBs *int `json:"dataStorageSizeInTBs,omitempty"` + CpuCoreCount *int `json:"cpuCoreCount,omitempty"` + // +kubebuilder:validation:Enum:="ECPU";"OCPU" + ComputeModel database.AutonomousDatabaseComputeModelEnum `json:"computeModel,omitempty"` + ComputeCount *float32 `json:"computeCount,omitempty"` + OcpuCount *float32 `json:"ocpuCount,omitempty"` + AdminPassword PasswordSpec `json:"adminPassword,omitempty"` + IsAutoScalingEnabled *bool `json:"isAutoScalingEnabled,omitempty"` + IsDedicated *bool `json:"isDedicated,omitempty"` + IsFreeTier *bool `json:"isFreeTier,omitempty"` + + // NetworkAccess + IsAccessControlEnabled *bool `json:"isAccessControlEnabled,omitempty"` + WhitelistedIps []string `json:"whitelistedIps,omitempty"` + SubnetId *string `json:"subnetId,omitempty"` + NsgIds []string `json:"nsgIds,omitempty"` + PrivateEndpointLabel *string `json:"privateEndpointLabel,omitempty"` + IsMtlsConnectionRequired *bool `json:"isMtlsConnectionRequired,omitempty"` + + FreeformTags map[string]string `json:"freeformTags,omitempty"` +} + +/************************ +* ACD specs +************************/ +type K8sAcdSpec struct { + Name *string `json:"name,omitempty"` +} + +type OciAcdSpec struct { + Id *string `json:"id,omitempty"` +} + +// AcdSpec defines the spec of the target for backup/restore runs. +// The name could be the name of an AutonomousDatabase or an AutonomousDatabaseBackup +type AcdSpec struct { + K8sAcd K8sAcdSpec `json:"k8sAcd,omitempty"` + OciAcd OciAcdSpec `json:"ociAcd,omitempty"` +} + +/************************ +* Secret specs +************************/ +type K8sSecretSpec struct { + Name *string `json:"name,omitempty"` +} + +type OciSecretSpec struct { + Id *string `json:"id,omitempty"` +} + +type PasswordSpec struct { + K8sSecret K8sSecretSpec `json:"k8sSecret,omitempty"` + OciSecret OciSecretSpec `json:"ociSecret,omitempty"` +} + +type WalletSpec struct { + Name *string `json:"name,omitempty"` + Password PasswordSpec `json:"password,omitempty"` +} + +// AutonomousDatabaseStatus defines the observed state of AutonomousDatabase +type AutonomousDatabaseStatus struct { + // Lifecycle State of the ADB + LifecycleState database.AutonomousDatabaseLifecycleStateEnum `json:"lifecycleState,omitempty"` + // Creation time of the ADB + TimeCreated string `json:"timeCreated,omitempty"` + // Expiring date of the instance wallet + WalletExpiringDate string `json:"walletExpiringDate,omitempty"` + // Connection Strings of the ADB + AllConnectionStrings []ConnectionStringProfile `json:"allConnectionStrings,omitempty"` + // +patchMergeKey=type + // +patchStrategy=merge + // +listType=map + // +listMapKey=type + Conditions []metav1.Condition `json:"conditions,omitempty"` +} + +type TLSAuthenticationEnum string + +const ( + tlsAuthenticationTLS TLSAuthenticationEnum = "TLS" + tlsAuthenticationMTLS TLSAuthenticationEnum = "Mutual TLS" +) + +func GetTLSAuthenticationEnumFromString(val string) (TLSAuthenticationEnum, bool) { + var mappingTLSAuthenticationEnum = map[string]TLSAuthenticationEnum{ + "TLS": tlsAuthenticationTLS, + "Mutual TLS": tlsAuthenticationMTLS, + } + + enum, ok := mappingTLSAuthenticationEnum[val] + return enum, ok +} + +type ConnectionStringProfile struct { + TLSAuthentication TLSAuthenticationEnum `json:"tlsAuthentication,omitempty"` + ConnectionStrings []ConnectionStringSpec `json:"connectionStrings"` +} + +type ConnectionStringSpec struct { + TNSName string `json:"tnsName,omitempty"` + ConnectionString string `json:"connectionString,omitempty"` +} + +// AutonomousDatabase is the Schema for the autonomousdatabases API +// +kubebuilder:object:root=true +// +kubebuilder:resource:shortName="adb";"adbs" +// +kubebuilder:subresource:status +// +kubebuilder:printcolumn:JSONPath=".spec.details.displayName",name="Display Name",type=string +// +kubebuilder:printcolumn:JSONPath=".spec.details.dbName",name="Db Name",type=string +// +kubebuilder:printcolumn:JSONPath=".status.lifecycleState",name="State",type=string +// +kubebuilder:printcolumn:JSONPath=".spec.details.isDedicated",name="Dedicated",type=string +// +kubebuilder:printcolumn:JSONPath=".spec.details.cpuCoreCount",name="OCPUs",type=integer +// +kubebuilder:printcolumn:JSONPath=".spec.details.dataStorageSizeInTBs",name="Storage (TB)",type=integer +// +kubebuilder:printcolumn:JSONPath=".spec.details.dbWorkload",name="Workload Type",type=string +// +kubebuilder:printcolumn:JSONPath=".status.timeCreated",name="Created",type=string +// +kubebuilder:storageversion +type AutonomousDatabase struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + Spec AutonomousDatabaseSpec `json:"spec,omitempty"` + Status AutonomousDatabaseStatus `json:"status,omitempty"` +} + +// +kubebuilder:object:root=true + +// AutonomousDatabaseList contains a list of AutonomousDatabase +type AutonomousDatabaseList struct { + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata,omitempty"` + Items []AutonomousDatabase `json:"items"` +} + +func init() { + SchemeBuilder.Register(&AutonomousDatabase{}, &AutonomousDatabaseList{}) +} + +// Implement conversion.Hub interface, which means any resource version can convert into v4 +func (*AutonomousDatabase) Hub() {} + +// UpdateStatusFromOCIADB updates the status subresource +func (adb *AutonomousDatabase) UpdateStatusFromOciAdb(ociObj database.AutonomousDatabase) { + adb.Status.LifecycleState = ociObj.LifecycleState + adb.Status.TimeCreated = FormatSDKTime(ociObj.TimeCreated) + + if *ociObj.IsDedicated { + conns := make([]ConnectionStringSpec, len(ociObj.ConnectionStrings.AllConnectionStrings)) + for key, val := range ociObj.ConnectionStrings.AllConnectionStrings { + conns = append(conns, ConnectionStringSpec{TNSName: key, ConnectionString: val}) + } + + adb.Status.AllConnectionStrings = []ConnectionStringProfile{ + {ConnectionStrings: conns}, + } + } else { + var mTLSConns []ConnectionStringSpec + var tlsConns []ConnectionStringSpec + + var conns []ConnectionStringProfile + + for _, profile := range ociObj.ConnectionStrings.Profiles { + if profile.TlsAuthentication == database.DatabaseConnectionStringProfileTlsAuthenticationMutual { + mTLSConns = append(mTLSConns, ConnectionStringSpec{TNSName: *profile.DisplayName, ConnectionString: *profile.Value}) + } else { + tlsConns = append(tlsConns, ConnectionStringSpec{TNSName: *profile.DisplayName, ConnectionString: *profile.Value}) + } + } + + if len(mTLSConns) > 0 { + conns = append(conns, ConnectionStringProfile{ + TLSAuthentication: tlsAuthenticationMTLS, + ConnectionStrings: mTLSConns, + }) + } + + if len(tlsConns) > 0 { + conns = append(conns, ConnectionStringProfile{ + TLSAuthentication: tlsAuthenticationTLS, + ConnectionStrings: tlsConns, + }) + } + + adb.Status.AllConnectionStrings = conns + } +} + +// UpdateFromOciAdb updates the attributes using database.AutonomousDatabase object +func (adb *AutonomousDatabase) UpdateFromOciAdb(ociObj database.AutonomousDatabase, overwrite bool) (specChanged bool) { + oldADB := adb.DeepCopy() + + /*********************************** + * update the spec + ***********************************/ + if overwrite || adb.Spec.Details.Id == nil { + adb.Spec.Details.Id = ociObj.Id + } + if overwrite || adb.Spec.Details.CompartmentId == nil { + adb.Spec.Details.CompartmentId = ociObj.CompartmentId + } + if overwrite || adb.Spec.Details.AutonomousContainerDatabase.OciAcd.Id == nil { + adb.Spec.Details.AutonomousContainerDatabase.OciAcd.Id = ociObj.AutonomousContainerDatabaseId + } + if overwrite || adb.Spec.Details.DisplayName == nil { + adb.Spec.Details.DisplayName = ociObj.DisplayName + } + if overwrite || adb.Spec.Details.DbName == nil { + adb.Spec.Details.DbName = ociObj.DbName + } + if overwrite || adb.Spec.Details.DbWorkload == "" { + adb.Spec.Details.DbWorkload = ociObj.DbWorkload + } + if overwrite || adb.Spec.Details.LicenseModel == "" { + adb.Spec.Details.LicenseModel = ociObj.LicenseModel + } + if overwrite || adb.Spec.Details.DbVersion == nil { + adb.Spec.Details.DbVersion = ociObj.DbVersion + } + if overwrite || adb.Spec.Details.DataStorageSizeInTBs == nil { + adb.Spec.Details.DataStorageSizeInTBs = ociObj.DataStorageSizeInTBs + } + if overwrite || adb.Spec.Details.CpuCoreCount == nil { + adb.Spec.Details.CpuCoreCount = ociObj.CpuCoreCount + } + if overwrite || adb.Spec.Details.ComputeModel == "" { + adb.Spec.Details.ComputeModel = ociObj.ComputeModel + } + if overwrite || adb.Spec.Details.OcpuCount == nil { + adb.Spec.Details.OcpuCount = ociObj.OcpuCount + } + if overwrite || adb.Spec.Details.ComputeCount == nil { + adb.Spec.Details.ComputeCount = ociObj.ComputeCount + } + if overwrite || adb.Spec.Details.IsAutoScalingEnabled == nil { + adb.Spec.Details.IsAutoScalingEnabled = ociObj.IsAutoScalingEnabled + } + if overwrite || adb.Spec.Details.IsDedicated == nil { + adb.Spec.Details.IsDedicated = ociObj.IsDedicated + } + if overwrite || adb.Spec.Details.IsFreeTier == nil { + adb.Spec.Details.IsFreeTier = ociObj.IsFreeTier + } + if overwrite || adb.Spec.Details.FreeformTags == nil { + // Special case: an emtpy map will be nil after unmarshalling while the OCI always returns an emty map. + if len(ociObj.FreeformTags) != 0 { + adb.Spec.Details.FreeformTags = ociObj.FreeformTags + } else { + adb.Spec.Details.FreeformTags = nil + } + } + + if overwrite || adb.Spec.Details.IsAccessControlEnabled == nil { + adb.Spec.Details.IsAccessControlEnabled = ociObj.IsAccessControlEnabled + } + + if overwrite || adb.Spec.Details.WhitelistedIps == nil { + if len(ociObj.WhitelistedIps) != 0 { + adb.Spec.Details.WhitelistedIps = ociObj.WhitelistedIps + } else { + adb.Spec.Details.WhitelistedIps = nil + } + } + if overwrite || adb.Spec.Details.IsMtlsConnectionRequired == nil { + adb.Spec.Details.IsMtlsConnectionRequired = ociObj.IsMtlsConnectionRequired + } + if overwrite || adb.Spec.Details.SubnetId == nil { + adb.Spec.Details.SubnetId = ociObj.SubnetId + } + if overwrite || adb.Spec.Details.NsgIds == nil { + if len(ociObj.NsgIds) != 0 { + adb.Spec.Details.NsgIds = ociObj.NsgIds + } else { + adb.Spec.Details.NsgIds = nil + } + } + if overwrite || adb.Spec.Details.PrivateEndpointLabel == nil { + adb.Spec.Details.PrivateEndpointLabel = ociObj.PrivateEndpointLabel + } + + /*********************************** + * update the status subresource + ***********************************/ + adb.UpdateStatusFromOciAdb(ociObj) + + return !reflect.DeepEqual(oldADB.Spec, adb.Spec) +} + +// RemoveUnchangedDetails removes the unchanged fields in spec.details, and returns if the details has been changed. +func (adb *AutonomousDatabase) RemoveUnchangedDetails(prevSpec AutonomousDatabaseSpec) (bool, error) { + + changed, err := RemoveUnchangedFields(prevSpec.Details, &adb.Spec.Details) + if err != nil { + return changed, err + } + + return changed, nil +} + +// A helper function which is useful for debugging. The function prints out a structural JSON format. +func (adb *AutonomousDatabase) String() (string, error) { + out, err := json.MarshalIndent(adb, "", " ") + if err != nil { + return "", err + } + return string(out), nil +} diff --git a/apis/database/v4/autonomousdatabase_webhook.go b/apis/database/v4/autonomousdatabase_webhook.go new file mode 100644 index 00000000..f7eb60aa --- /dev/null +++ b/apis/database/v4/autonomousdatabase_webhook.go @@ -0,0 +1,170 @@ +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v4 + +import ( + dbcommons "github.com/oracle/oracle-database-operator/commons/database" + apierrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/runtime/schema" + "k8s.io/apimachinery/pkg/util/validation/field" + ctrl "sigs.k8s.io/controller-runtime" + logf "sigs.k8s.io/controller-runtime/pkg/log" + "sigs.k8s.io/controller-runtime/pkg/webhook" + "sigs.k8s.io/controller-runtime/pkg/webhook/admission" +) + +// log is for logging in this package. +var autonomousdatabaselog = logf.Log.WithName("autonomousdatabase-resource") + +func (r *AutonomousDatabase) SetupWebhookWithManager(mgr ctrl.Manager) error { + return ctrl.NewWebhookManagedBy(mgr). + For(r). + Complete() +} + +// +kubebuilder:webhook:verbs=create;update,path=/validate-database-oracle-com-v4-autonomousdatabase,mutating=false,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=autonomousdatabases,versions=v4,name=vautonomousdatabasev4.kb.io,admissionReviewVersions=v1 +var _ webhook.Validator = &AutonomousDatabase{} + +// ValidateCreate implements webhook.Validator so a webhook will be registered for the type +// ValidateCreate checks if the spec is valid for a provisioning or a binding operation +func (r *AutonomousDatabase) ValidateCreate() (admission.Warnings, error) { + var allErrs field.ErrorList + + autonomousdatabaselog.Info("validate create", "name", r.Name) + + namespaces := dbcommons.GetWatchNamespaces() + _, hasEmptyString := namespaces[""] + isClusterScoped := len(namespaces) == 1 && hasEmptyString + if !isClusterScoped { + _, containsNamespace := namespaces[r.Namespace] + // Check if the allowed namespaces maps contains the required namespace + if len(namespaces) != 0 && !containsNamespace { + allErrs = append(allErrs, + field.Invalid(field.NewPath("metadata").Child("namespace"), r.Namespace, + "Oracle database operator doesn't watch over this namespace")) + } + } + + if len(allErrs) == 0 { + return nil, nil + } + return nil, apierrors.NewInvalid( + schema.GroupKind{Group: "database.oracle.com", Kind: "AutonomousDatabase"}, + r.Name, allErrs) +} + +// ValidateUpdate implements webhook.Validator so a webhook will be registered for the type +func (r *AutonomousDatabase) ValidateUpdate(old runtime.Object) (admission.Warnings, error) { + var allErrs field.ErrorList + var oldADB *AutonomousDatabase = old.(*AutonomousDatabase) + + autonomousdatabaselog.Info("validate update", "name", r.Name) + + // skip the verification of adding ADB OCID or binding + // if oldADB.Status.LifecycleState == "" { + // return nil, nil + // } + + // cannot update when the old state is in intermediate, except for the change to the hardLink or the terminate operatrion during valid lifecycleState + // var copySpec *AutonomousDatabaseSpec = r.Spec.DeepCopy() + // specChanged, err := RemoveUnchangedFields(oldADB.Spec, copySpec) + // if err != nil { + // allErrs = append(allErrs, + // field.Forbidden(field.NewPath("spec"), err.Error())) + // } + + // hardLinkChanged := copySpec.HardLink != nil + + // isTerminateOp := CanBeTerminated(oldADB.Status.LifecycleState) && copySpec.Action == "Terminate" + + // if specChanged && IsAdbIntermediateState(oldADB.Status.LifecycleState) && !isTerminateOp && !hardLinkChanged { + // allErrs = append(allErrs, + // field.Forbidden(field.NewPath("spec"), + // "cannot change the spec when the lifecycleState is in an intermdeiate state")) + // } + + // cannot modify autonomousDatabaseOCID + if r.Spec.Details.Id != nil && + oldADB.Spec.Details.Id != nil && + *r.Spec.Details.Id != *oldADB.Spec.Details.Id { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("details").Child("autonomousDatabaseOCID"), + "autonomousDatabaseOCID cannot be modified")) + } + + allErrs = validateCommon(r, allErrs) + + if len(allErrs) == 0 { + return nil, nil + } + return nil, apierrors.NewInvalid( + schema.GroupKind{Group: "database.oracle.com", Kind: "AutonomousDatabase"}, + r.Name, allErrs) +} + +func validateCommon(adb *AutonomousDatabase, allErrs field.ErrorList) field.ErrorList { + // password + if adb.Spec.Details.AdminPassword.K8sSecret.Name != nil && adb.Spec.Details.AdminPassword.OciSecret.Id != nil { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("details").Child("adminPassword"), + "cannot apply k8sSecret.name and ociSecret.ocid at the same time")) + } + + if adb.Spec.Wallet.Password.K8sSecret.Name != nil && adb.Spec.Wallet.Password.OciSecret.Id != nil { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("details").Child("wallet").Child("password"), + "cannot apply k8sSecret.name and ociSecret.ocid at the same time")) + } + + return allErrs +} + +// ValidateDelete implements webhook.Validator so a webhook will be registered for the type +func (r *AutonomousDatabase) ValidateDelete() (admission.Warnings, error) { + autonomousdatabaselog.Info("validate delete", "name", r.Name) + return nil, nil +} + +// Returns true if AutonomousContainerDatabaseOCID has value. +// We don't use Details.IsDedicated because the parameter might be null when it's a provision operation. +func isDedicated(adb *AutonomousDatabase) bool { + return adb.Spec.Details.AutonomousContainerDatabase.K8sAcd.Name != nil || + adb.Spec.Details.AutonomousContainerDatabase.OciAcd.Id != nil +} diff --git a/apis/database/v4/autonomousdatabasebackup_types.go b/apis/database/v4/autonomousdatabasebackup_types.go new file mode 100644 index 00000000..925256c0 --- /dev/null +++ b/apis/database/v4/autonomousdatabasebackup_types.go @@ -0,0 +1,129 @@ +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v4 + +import ( + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + + "github.com/oracle/oci-go-sdk/v65/common" + "github.com/oracle/oci-go-sdk/v65/database" +) + +// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! +// NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized. + +// AutonomousDatabaseBackupSpec defines the desired state of AutonomousDatabaseBackup +type AutonomousDatabaseBackupSpec struct { + // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster + // Important: Run "make" to regenerate code after modifying this file + Target TargetSpec `json:"target,omitempty"` + DisplayName *string `json:"displayName,omitempty"` + AutonomousDatabaseBackupOCID *string `json:"autonomousDatabaseBackupOCID,omitempty"` + IsLongTermBackup *bool `json:"isLongTermBackup,omitempty"` + RetentionPeriodInDays *int `json:"retentionPeriodInDays,omitempty"` + OCIConfig OciConfigSpec `json:"ociConfig,omitempty"` +} + +// AutonomousDatabaseBackupStatus defines the observed state of AutonomousDatabaseBackup +type AutonomousDatabaseBackupStatus struct { + LifecycleState database.AutonomousDatabaseBackupLifecycleStateEnum `json:"lifecycleState"` + Type database.AutonomousDatabaseBackupTypeEnum `json:"type"` + IsAutomatic bool `json:"isAutomatic"` + TimeStarted string `json:"timeStarted,omitempty"` + TimeEnded string `json:"timeEnded,omitempty"` + AutonomousDatabaseOCID string `json:"autonomousDatabaseOCID"` + CompartmentOCID string `json:"compartmentOCID"` + DBName string `json:"dbName"` + DBDisplayName string `json:"dbDisplayName"` +} + +//+kubebuilder:object:root=true +//+kubebuilder:subresource:status +//+kubebuilder:resource:shortName="adbbu";"adbbus" +//+kubebuilder:printcolumn:JSONPath=".status.lifecycleState",name="State",type=string +//+kubebuilder:printcolumn:JSONPath=".status.dbDisplayName",name="DB DisplayName",type=string +//+kubebuilder:printcolumn:JSONPath=".status.type",name="Type",type=string +//+kubebuilder:printcolumn:JSONPath=".status.timeStarted",name="Started",type=string +//+kubebuilder:printcolumn:JSONPath=".status.timeEnded",name="Ended",type=string +// +kubebuilder:storageversion + +// AutonomousDatabaseBackup is the Schema for the autonomousdatabasebackups API +type AutonomousDatabaseBackup struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + Spec AutonomousDatabaseBackupSpec `json:"spec,omitempty"` + Status AutonomousDatabaseBackupStatus `json:"status,omitempty"` +} + +//+kubebuilder:object:root=true + +// AutonomousDatabaseBackupList contains a list of AutonomousDatabaseBackup +type AutonomousDatabaseBackupList struct { + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata,omitempty"` + Items []AutonomousDatabaseBackup `json:"items"` +} + +func init() { + SchemeBuilder.Register(&AutonomousDatabaseBackup{}, &AutonomousDatabaseBackupList{}) +} + +// Implement conversion.Hub interface, which means any resource version can convert into v4 +func (*AutonomousDatabaseBackup) Hub() {} + +func (b *AutonomousDatabaseBackup) UpdateStatusFromOCIBackup(ociBackup database.AutonomousDatabaseBackup, ociADB database.AutonomousDatabase) { + b.Status.AutonomousDatabaseOCID = *ociBackup.AutonomousDatabaseId + b.Status.CompartmentOCID = *ociBackup.CompartmentId + b.Status.Type = ociBackup.Type + b.Status.IsAutomatic = *ociBackup.IsAutomatic + + b.Status.LifecycleState = ociBackup.LifecycleState + + b.Status.TimeStarted = FormatSDKTime(ociBackup.TimeStarted) + b.Status.TimeEnded = FormatSDKTime(ociBackup.TimeEnded) + + b.Status.DBDisplayName = *ociADB.DisplayName + b.Status.DBName = *ociADB.DbName +} + +// GetTimeEnded returns the status.timeEnded in SDKTime format +func (b *AutonomousDatabaseBackup) GetTimeEnded() (*common.SDKTime, error) { + return ParseDisplayTime(b.Status.TimeEnded) +} diff --git a/apis/database/v4/autonomousdatabasebackup_webhook.go b/apis/database/v4/autonomousdatabasebackup_webhook.go new file mode 100644 index 00000000..7858adce --- /dev/null +++ b/apis/database/v4/autonomousdatabasebackup_webhook.go @@ -0,0 +1,158 @@ +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v4 + +import ( + dbcommons "github.com/oracle/oracle-database-operator/commons/database" + apierrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/runtime/schema" + "k8s.io/apimachinery/pkg/util/validation/field" + ctrl "sigs.k8s.io/controller-runtime" + logf "sigs.k8s.io/controller-runtime/pkg/log" + "sigs.k8s.io/controller-runtime/pkg/webhook" + "sigs.k8s.io/controller-runtime/pkg/webhook/admission" +) + +// log is for logging in this package. +var autonomousdatabasebackuplog = logf.Log.WithName("autonomousdatabasebackup-resource") + +func (r *AutonomousDatabaseBackup) SetupWebhookWithManager(mgr ctrl.Manager) error { + return ctrl.NewWebhookManagedBy(mgr). + For(r). + Complete() +} + +//+kubebuilder:webhook:path=/mutate-database-oracle-com-v4-autonomousdatabasebackup,mutating=true,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=autonomousdatabasebackups,verbs=create;update,versions=v4,name=mautonomousdatabasebackupv4.kb.io,admissionReviewVersions=v1 + +var _ webhook.Defaulter = &AutonomousDatabaseBackup{} + +// Default implements webhook.Defaulter so a webhook will be registered for the type +func (r *AutonomousDatabaseBackup) Default() { + autonomousdatabasebackuplog.Info("default", "name", r.Name) +} + +//+kubebuilder:webhook:verbs=create;update,path=/validate-database-oracle-com-v4-autonomousdatabasebackup,mutating=false,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=autonomousdatabasebackups,versions=v4,name=vautonomousdatabasebackupv4.kb.io,admissionReviewVersions=v1 + +var _ webhook.Validator = &AutonomousDatabaseBackup{} + +// ValidateCreate implements webhook.Validator so a webhook will be registered for the type +func (r *AutonomousDatabaseBackup) ValidateCreate() (admission.Warnings, error) { + autonomousdatabasebackuplog.Info("validate create", "name", r.Name) + + var allErrs field.ErrorList + + namespaces := dbcommons.GetWatchNamespaces() + _, hasEmptyString := namespaces[""] + isClusterScoped := len(namespaces) == 1 && hasEmptyString + if !isClusterScoped { + _, containsNamespace := namespaces[r.Namespace] + // Check if the allowed namespaces maps contains the required namespace + if len(namespaces) != 0 && !containsNamespace { + allErrs = append(allErrs, + field.Invalid(field.NewPath("metadata").Child("namespace"), r.Namespace, + "Oracle database operator doesn't watch over this namespace")) + } + } + + if r.Spec.Target.K8sAdb.Name == nil && r.Spec.Target.OciAdb.OCID == nil { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("target"), "target ADB is empty")) + } + + if r.Spec.Target.K8sAdb.Name != nil && r.Spec.Target.OciAdb.OCID != nil { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("target"), "specify either k8sADB or ociADB, but not both")) + } + + if len(allErrs) == 0 { + return nil, nil + } + return nil, apierrors.NewInvalid( + schema.GroupKind{Group: "database.oracle.com", Kind: "AutonomousDatabaseBackup"}, + r.Name, allErrs) +} + +// ValidateUpdate implements webhook.Validator so a webhook will be registered for the type +func (r *AutonomousDatabaseBackup) ValidateUpdate(old runtime.Object) (admission.Warnings, error) { + autonomousdatabasebackuplog.Info("validate update", "name", r.Name) + + var allErrs field.ErrorList + oldBackup := old.(*AutonomousDatabaseBackup) + + if oldBackup.Spec.AutonomousDatabaseBackupOCID != nil && r.Spec.AutonomousDatabaseBackupOCID != nil && + *oldBackup.Spec.AutonomousDatabaseBackupOCID != *r.Spec.AutonomousDatabaseBackupOCID { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("autonomousDatabaseBackupOCID"), + "cannot assign a new autonomousDatabaseBackupOCID to this backup")) + } + + if oldBackup.Spec.Target.K8sAdb.Name != nil && r.Spec.Target.K8sAdb.Name != nil && + *oldBackup.Spec.Target.K8sAdb.Name != *r.Spec.Target.K8sAdb.Name { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("target").Child("k8sADB").Child("name"), "cannot assign a new name to the target")) + } + + if oldBackup.Spec.Target.OciAdb.OCID != nil && r.Spec.Target.OciAdb.OCID != nil && + *oldBackup.Spec.Target.OciAdb.OCID != *r.Spec.Target.OciAdb.OCID { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("target").Child("ociADB").Child("ocid"), "cannot assign a new ocid to the target")) + } + + if oldBackup.Spec.DisplayName != nil && r.Spec.DisplayName != nil && + *oldBackup.Spec.DisplayName != *r.Spec.DisplayName { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("displayName"), "cannot assign a new displayName to this backup")) + } + + if len(allErrs) == 0 { + return nil, nil + } + return nil, apierrors.NewInvalid( + schema.GroupKind{Group: "database.oracle.com", Kind: "AutonomousDatabaseBackup"}, + r.Name, allErrs) +} + +// ValidateDelete implements webhook.Validator so a webhook will be registered for the type +func (r *AutonomousDatabaseBackup) ValidateDelete() (admission.Warnings, error) { + autonomousdatabasebackuplog.Info("validate delete", "name", r.Name) + + // TODO(user): fill in your validation logic upon object deletion. + return nil, nil +} diff --git a/apis/database/v4/autonomousdatabaserestore_types.go b/apis/database/v4/autonomousdatabaserestore_types.go new file mode 100644 index 00000000..3337c983 --- /dev/null +++ b/apis/database/v4/autonomousdatabaserestore_types.go @@ -0,0 +1,142 @@ +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v4 + +import ( + "errors" + + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + + "github.com/oracle/oci-go-sdk/v65/common" + "github.com/oracle/oci-go-sdk/v65/database" + "github.com/oracle/oci-go-sdk/v65/workrequests" +) + +// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! +// NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized. +type K8sADBBackupSpec struct { + Name *string `json:"name,omitempty"` +} + +type PITSpec struct { + // The timestamp must follow this format: YYYY-MM-DD HH:MM:SS GMT + Timestamp *string `json:"timestamp,omitempty"` +} + +type SourceSpec struct { + K8sAdbBackup K8sADBBackupSpec `json:"k8sADBBackup,omitempty"` + PointInTime PITSpec `json:"pointInTime,omitempty"` +} + +// AutonomousDatabaseRestoreSpec defines the desired state of AutonomousDatabaseRestore +type AutonomousDatabaseRestoreSpec struct { + // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster + // Important: Run "make" to regenerate code after modifying this file + Target TargetSpec `json:"target"` + Source SourceSpec `json:"source"` + OCIConfig OciConfigSpec `json:"ociConfig,omitempty"` +} + +// AutonomousDatabaseRestoreStatus defines the observed state of AutonomousDatabaseRestore +type AutonomousDatabaseRestoreStatus struct { + // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster + // Important: Run "make" to regenerate code after modifying this file + DisplayName string `json:"displayName"` + TimeAccepted string `json:"timeAccepted,omitempty"` + TimeStarted string `json:"timeStarted,omitempty"` + TimeEnded string `json:"timeEnded,omitempty"` + DbName string `json:"dbName"` + WorkRequestOCID string `json:"workRequestOCID"` + Status workrequests.WorkRequestStatusEnum `json:"status"` +} + +//+kubebuilder:object:root=true +//+kubebuilder:subresource:status +//+kubebuilder:resource:shortName="adbr";"adbrs" +// +kubebuilder:printcolumn:JSONPath=".status.status",name="Status",type=string +// +kubebuilder:printcolumn:JSONPath=".status.displayName",name="DbDisplayName",type=string +// +kubebuilder:printcolumn:JSONPath=".status.dbName",name="DbName",type=string +// +kubebuilder:storageversion + +// AutonomousDatabaseRestore is the Schema for the autonomousdatabaserestores API +type AutonomousDatabaseRestore struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + Spec AutonomousDatabaseRestoreSpec `json:"spec,omitempty"` + Status AutonomousDatabaseRestoreStatus `json:"status,omitempty"` +} + +//+kubebuilder:object:root=true + +// AutonomousDatabaseRestoreList contains a list of AutonomousDatabaseRestore +type AutonomousDatabaseRestoreList struct { + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata,omitempty"` + Items []AutonomousDatabaseRestore `json:"items"` +} + +func init() { + SchemeBuilder.Register(&AutonomousDatabaseRestore{}, &AutonomousDatabaseRestoreList{}) +} + +// Implement conversion.Hub interface, which means any resource version can convert into v4 +func (*AutonomousDatabaseRestore) Hub() {} + +// GetPIT returns the spec.pointInTime.timeStamp in SDKTime format +func (r *AutonomousDatabaseRestore) GetPIT() (*common.SDKTime, error) { + if r.Spec.Source.PointInTime.Timestamp == nil { + return nil, errors.New("the timestamp is empty") + } + return ParseDisplayTime(*r.Spec.Source.PointInTime.Timestamp) +} + +func (r *AutonomousDatabaseRestore) UpdateStatus( + adb database.AutonomousDatabase, + workResp workrequests.GetWorkRequestResponse) { + + r.Status.DisplayName = *adb.DisplayName + r.Status.DbName = *adb.DbName + + r.Status.WorkRequestOCID = *workResp.Id + r.Status.Status = workResp.Status + r.Status.TimeAccepted = FormatSDKTime(workResp.TimeAccepted) + r.Status.TimeStarted = FormatSDKTime(workResp.TimeStarted) + r.Status.TimeEnded = FormatSDKTime(workResp.TimeFinished) +} diff --git a/apis/database/v4/autonomousdatabaserestore_webhook.go b/apis/database/v4/autonomousdatabaserestore_webhook.go new file mode 100644 index 00000000..6e3b4656 --- /dev/null +++ b/apis/database/v4/autonomousdatabaserestore_webhook.go @@ -0,0 +1,146 @@ +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v4 + +import ( + dbcommons "github.com/oracle/oracle-database-operator/commons/database" + apierrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/runtime/schema" + "k8s.io/apimachinery/pkg/util/validation/field" + ctrl "sigs.k8s.io/controller-runtime" + logf "sigs.k8s.io/controller-runtime/pkg/log" + "sigs.k8s.io/controller-runtime/pkg/webhook" + "sigs.k8s.io/controller-runtime/pkg/webhook/admission" +) + +// log is for logging in this package. +var autonomousdatabaserestorelog = logf.Log.WithName("autonomousdatabaserestore-resource") + +func (r *AutonomousDatabaseRestore) SetupWebhookWithManager(mgr ctrl.Manager) error { + return ctrl.NewWebhookManagedBy(mgr). + For(r). + Complete() +} + +//+kubebuilder:webhook:verbs=create;update,path=/validate-database-oracle-com-v4-autonomousdatabaserestore,mutating=false,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=autonomousdatabaserestores,versions=v4,name=vautonomousdatabaserestorev4.kb.io,admissionReviewVersions=v1 + +var _ webhook.Validator = &AutonomousDatabaseRestore{} + +// ValidateCreate implements webhook.Validator so a webhook will be registered for the type +func (r *AutonomousDatabaseRestore) ValidateCreate() (admission.Warnings, error) { + autonomousdatabaserestorelog.Info("validate create", "name", r.Name) + + var allErrs field.ErrorList + + namespaces := dbcommons.GetWatchNamespaces() + _, hasEmptyString := namespaces[""] + isClusterScoped := len(namespaces) == 1 && hasEmptyString + if !isClusterScoped { + _, containsNamespace := namespaces[r.Namespace] + // Check if the allowed namespaces maps contains the required namespace + if len(namespaces) != 0 && !containsNamespace { + allErrs = append(allErrs, + field.Invalid(field.NewPath("metadata").Child("namespace"), r.Namespace, + "Oracle database operator doesn't watch over this namespace")) + } + } + + // Validate the target ADB + if r.Spec.Target.K8sAdb.Name == nil && r.Spec.Target.OciAdb.OCID == nil { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("target"), "target ADB is empty")) + } + + if r.Spec.Target.K8sAdb.Name != nil && r.Spec.Target.OciAdb.OCID != nil { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("target"), "specify either k8sADB.name or ociADB.ocid, but not both")) + } + + // Validate the restore source + if r.Spec.Source.K8sAdbBackup.Name == nil && + r.Spec.Source.PointInTime.Timestamp == nil { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("source"), "retore source is empty")) + } + + if r.Spec.Source.K8sAdbBackup.Name != nil && + r.Spec.Source.PointInTime.Timestamp != nil { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("source"), "cannot apply backupName and the PITR parameters at the same time")) + } + + // Verify the timestamp format if it's PITR + if r.Spec.Source.PointInTime.Timestamp != nil { + _, err := ParseDisplayTime(*r.Spec.Source.PointInTime.Timestamp) + if err != nil { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("source").Child("pointInTime").Child("timestamp"), "invalid timestamp format")) + } + } + + if len(allErrs) == 0 { + return nil, nil + } + return nil, apierrors.NewInvalid( + schema.GroupKind{Group: "database.oracle.com", Kind: "AutonomousDatabaseRestore"}, + r.Name, allErrs) +} + +// ValidateUpdate implements webhook.Validator so a webhook will be registered for the type +func (r *AutonomousDatabaseRestore) ValidateUpdate(old runtime.Object) (admission.Warnings, error) { + autonomousdatabaserestorelog.Info("validate update", "name", r.Name) + + var allErrs field.ErrorList + + if len(allErrs) == 0 { + return nil, nil + } + return nil, apierrors.NewInvalid( + schema.GroupKind{Group: "database.oracle.com", Kind: "AutonomousDatabaseRestore"}, + r.Name, allErrs) +} + +// ValidateDelete implements webhook.Validator so a webhook will be registered for the type +func (r *AutonomousDatabaseRestore) ValidateDelete() (admission.Warnings, error) { + autonomousdatabaserestorelog.Info("validate delete", "name", r.Name) + + // TODO(user): fill in your validation logic upon object deletion. + return nil, nil +} diff --git a/apis/database/v1alpha1/cdb_types.go b/apis/database/v4/cdb_types.go similarity index 93% rename from apis/database/v1alpha1/cdb_types.go rename to apis/database/v4/cdb_types.go index 206781b2..ce3f6f28 100644 --- a/apis/database/v1alpha1/cdb_types.go +++ b/apis/database/v4/cdb_types.go @@ -36,7 +36,7 @@ ** SOFTWARE. */ -package v1alpha1 +package v4 import ( metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" @@ -84,8 +84,11 @@ type CDBSpec struct { // DB server port DBPort int `json:"dbPort,omitempty"` // Node Selector for running the Pod - NodeSelector map[string]string `json:"nodeSelector,omitempty"` - DBTnsurl string `json:"dbTnsurl,omitempty"` + NodeSelector map[string]string `json:"nodeSelector,omitempty"` + DeletePDBCascade bool `json:"deletePdbCascade,omitempty"` + DBTnsurl string `json:"dbTnsurl,omitempty"` + CDBPubKey CDBPUBKEY `json:"cdbOrdsPubKey,omitempty"` + CDBPriKey CDBPRIVKEY `json:"cdbOrdsPrvKey,omitempty"` } // CDBSecret defines the secretName @@ -132,6 +135,14 @@ type CDBTLSCRT struct { Secret CDBSecret `json:"secret"` } +type CDBPUBKEY struct { + Secret CDBSecret `json:"secret"` +} + +type CDBPRIVKEY struct { + Secret CDBSecret `json:"secret"` +} + // CDBStatus defines the observed state of CDB type CDBStatus struct { // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster @@ -150,11 +161,12 @@ type CDBStatus struct { // +kubebuilder:printcolumn:JSONPath=".spec.cdbName",name="CDB Name",type="string",description="Name of the CDB" // +kubebuilder:printcolumn:JSONPath=".spec.dbServer",name="DB Server",type="string",description=" Name of the DB Server" // +kubebuilder:printcolumn:JSONPath=".spec.dbPort",name="DB Port",type="integer",description="DB server port" -// +kubebuilder:printcolumn:JSONPath=".spec.dbTnsurl",name="TNS STRING",type="string",description=" string of the tnsalias" // +kubebuilder:printcolumn:JSONPath=".spec.replicas",name="Replicas",type="integer",description="Replicas" // +kubebuilder:printcolumn:JSONPath=".status.phase",name="Status",type="string",description="Status of the CDB Resource" // +kubebuilder:printcolumn:JSONPath=".status.msg",name="Message",type="string",description="Error message, if any" +// +kubebuilder:printcolumn:JSONPath=".spec.dbTnsurl",name="TNS STRING",type="string",description=" string of the tnsalias" // +kubebuilder:resource:path=cdbs,scope=Namespaced +// +kubebuilder:storageversion // CDB is the Schema for the cdbs API type CDB struct { diff --git a/apis/database/v1alpha1/cdb_webhook.go b/apis/database/v4/cdb_webhook.go similarity index 92% rename from apis/database/v1alpha1/cdb_webhook.go rename to apis/database/v4/cdb_webhook.go index 345b6f75..235b2627 100644 --- a/apis/database/v1alpha1/cdb_webhook.go +++ b/apis/database/v4/cdb_webhook.go @@ -36,7 +36,7 @@ ** SOFTWARE. */ -package v1alpha1 +package v4 import ( "reflect" @@ -61,7 +61,7 @@ func (r *CDB) SetupWebhookWithManager(mgr ctrl.Manager) error { Complete() } -//+kubebuilder:webhook:path=/mutate-database-oracle-com-v1alpha1-cdb,mutating=true,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=cdbs,verbs=create;update,versions=v1alpha1,name=mcdb.kb.io,admissionReviewVersions={v1,v1beta1} +//+kubebuilder:webhook:path=/mutate-database-oracle-com-v4-cdb,mutating=true,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=cdbs,verbs=create;update,versions=v4,name=mcdb.kb.io,admissionReviewVersions={v1,v1beta1} var _ webhook.Defaulter = &CDB{} @@ -79,7 +79,7 @@ func (r *CDB) Default() { } // TODO(user): change verbs to "verbs=create;update;delete" if you want to enable deletion validation. -//+kubebuilder:webhook:path=/validate-database-oracle-com-v1alpha1-cdb,mutating=false,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=cdbs,verbs=create;update,versions=v1alpha1,name=vcdb.kb.io,admissionReviewVersions={v1,v1beta1} +//+kubebuilder:webhook:path=/validate-database-oracle-com-v4-cdb,mutating=false,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=cdbs,verbs=create;update,versions=v4,name=vcdb.kb.io,admissionReviewVersions={v1,v1beta1} var _ webhook.Validator = &CDB{} @@ -104,6 +104,11 @@ func (r *CDB) ValidateCreate() (admission.Warnings, error) { field.Required(field.NewPath("spec").Child("cdbTlsCrt"), "Please specify CDB Tls Certificate(secret)")) } + if reflect.ValueOf(r.Spec.CDBPriKey).IsZero() { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("CDBPriKey"), "Please specify CDB CDBPriKey(secret)")) + } + /*if r.Spec.SCANName == "" { allErrs = append(allErrs, field.Required(field.NewPath("spec").Child("scanName"), "Please specify SCAN Name for CDB")) diff --git a/apis/database/v4/dataguardbroker_conversion.go b/apis/database/v4/dataguardbroker_conversion.go new file mode 100644 index 00000000..c63210e0 --- /dev/null +++ b/apis/database/v4/dataguardbroker_conversion.go @@ -0,0 +1,4 @@ +package v4 + +// Hub defines v1 as the hub version +func (*DataguardBroker) Hub() {} diff --git a/apis/database/v4/dataguardbroker_types.go b/apis/database/v4/dataguardbroker_types.go new file mode 100644 index 00000000..cec11ca4 --- /dev/null +++ b/apis/database/v4/dataguardbroker_types.go @@ -0,0 +1,163 @@ +/* +** Copyright (c) 2023 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v4 + +import ( + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! +// NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized. + +// DataguardBrokerSpec defines the desired state of DataguardBroker +type DataguardBrokerSpec struct { + // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster + // Important: Run "make" to regenerate code after modifying this file + + PrimaryDatabaseRef string `json:"primaryDatabaseRef"` + StandbyDatabaseRefs []string `json:"standbyDatabaseRefs"` + SetAsPrimaryDatabase string `json:"setAsPrimaryDatabase,omitempty"` + LoadBalancer bool `json:"loadBalancer,omitempty"` + ServiceAnnotations map[string]string `json:"serviceAnnotations,omitempty"` + // +kubebuilder:validation:Enum=MaxPerformance;MaxAvailability + ProtectionMode string `json:"protectionMode"` + NodeSelector map[string]string `json:"nodeSelector,omitempty"` + + FastStartFailover bool `json:"fastStartFailover,omitempty"` +} + +// DataguardBrokerStatus defines the observed state of DataguardBroker +type DataguardBrokerStatus struct { + // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster + // Important: Run "make" to regenerate code after modifying this file + + PrimaryDatabaseRef string `json:"primaryDatabaseRef,omitempty"` + ProtectionMode string `json:"protectionMode,omitempty"` + PrimaryDatabase string `json:"primaryDatabase,omitempty"` + StandbyDatabases string `json:"standbyDatabases,omitempty"` + ExternalConnectString string `json:"externalConnectString,omitempty"` + ClusterConnectString string `json:"clusterConnectString,omitempty"` + Status string `json:"status,omitempty"` + + FastStartFailover string `json:"fastStartFailover,omitempty"` + DatabasesInDataguardConfig map[string]string `json:"databasesInDataguardConfig,omitempty"` +} + +// +kubebuilder:object:root=true +// +kubebuilder:subresource:status +// +kubebuilder:printcolumn:JSONPath=".status.primaryDatabase",name="Primary",type="string" +// +kubebuilder:printcolumn:JSONPath=".status.standbyDatabases",name="Standbys",type="string" +// +kubebuilder:printcolumn:JSONPath=".spec.protectionMode",name="Protection Mode",type="string" +// +kubebuilder:printcolumn:JSONPath=".status.clusterConnectString",name="Cluster Connect Str",type="string",priority=1 +// +kubebuilder:printcolumn:JSONPath=".status.externalConnectString",name="Connect Str",type="string" +// +kubebuilder:printcolumn:JSONPath=".spec.primaryDatabaseRef",name="Primary Database",type="string", priority=1 +// +kubebuilder:printcolumn:JSONPath=".status.status",name="Status",type="string" +// +kubebuilder:printcolumn:JSONPath=".status.fastStartFailover",name="FSFO", type="string" + +// DataguardBroker is the Schema for the dataguardbrokers API +// +kubebuilder:storageversion +type DataguardBroker struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + Spec DataguardBrokerSpec `json:"spec,omitempty"` + Status DataguardBrokerStatus `json:"status,omitempty"` +} + +// ////////////////////////////////////////////////////////////////////////////////////////////////// +// Returns the current primary database in the dataguard configuration from the resource status/spec +// ////////////////////////////////////////////////////////////////////////////////////////////////// +func (broker *DataguardBroker) GetCurrentPrimaryDatabase() string { + if broker.Status.PrimaryDatabase != "" { + return broker.Status.DatabasesInDataguardConfig[broker.Status.PrimaryDatabase] + } + return broker.Spec.PrimaryDatabaseRef +} + +// ////////////////////////////////////////////////////////////////////////////////////////////////// +// Returns databases in Dataguard configuration from the resource status/spec +// ////////////////////////////////////////////////////////////////////////////////////////////////// +func (broker *DataguardBroker) GetDatabasesInDataGuardConfiguration() []string { + var databases []string + if len(broker.Status.DatabasesInDataguardConfig) > 0 { + for _, value := range broker.Status.DatabasesInDataguardConfig { + if value != "" { + databases = append(databases, value) + } + } + + return databases + } + + databases = append(databases, broker.Spec.PrimaryDatabaseRef) + databases = append(databases, broker.Spec.StandbyDatabaseRefs...) + return databases +} + +// ////////////////////////////////////////////////////////////////////////////////////////////////// +// Returns standby databases in the dataguard configuration from the resource status/spec +// ////////////////////////////////////////////////////////////////////////////////////////////////// +func (broker *DataguardBroker) GetStandbyDatabasesInDgConfig() []string { + var databases []string + if len(broker.Status.DatabasesInDataguardConfig) > 0 { + for _, value := range broker.Status.DatabasesInDataguardConfig { + if value != "" && value != broker.Status.PrimaryDatabase { + databases = append(databases, value) + } + } + + return databases + } + + databases = append(databases, broker.Spec.StandbyDatabaseRefs...) + return databases +} + +//+kubebuilder:object:root=true + +// DataguardBrokerList contains a list of DataguardBroker +type DataguardBrokerList struct { + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata,omitempty"` + Items []DataguardBroker `json:"items"` +} + +func init() { + SchemeBuilder.Register(&DataguardBroker{}, &DataguardBrokerList{}) +} diff --git a/apis/database/v4/dataguardbroker_webhook.go b/apis/database/v4/dataguardbroker_webhook.go new file mode 100644 index 00000000..bcd35de9 --- /dev/null +++ b/apis/database/v4/dataguardbroker_webhook.go @@ -0,0 +1,55 @@ +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v4 + +import ( + ctrl "sigs.k8s.io/controller-runtime" + logf "sigs.k8s.io/controller-runtime/pkg/log" +) + +// log is for logging in this package. +var dataguardbrokerlog = logf.Log.WithName("dataguardbroker-resource") + +func (r *DataguardBroker) SetupWebhookWithManager(mgr ctrl.Manager) error { + return ctrl.NewWebhookManagedBy(mgr). + For(r). + Complete() +} + +// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! diff --git a/apis/database/v4/dbcssystem_conversion.go b/apis/database/v4/dbcssystem_conversion.go new file mode 100644 index 00000000..e5919f54 --- /dev/null +++ b/apis/database/v4/dbcssystem_conversion.go @@ -0,0 +1,4 @@ +package v4 + +// Hub defines v1 as the hub version +func (*DbcsSystem) Hub() {} diff --git a/apis/database/v4/dbcssystem_kms_types.go b/apis/database/v4/dbcssystem_kms_types.go new file mode 100644 index 00000000..8cbff504 --- /dev/null +++ b/apis/database/v4/dbcssystem_kms_types.go @@ -0,0 +1,141 @@ +/* +** Copyright (c) 2022-2024 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ +package v4 + +import "encoding/json" + +type KMSConfig struct { + VaultName string `json:"vaultName,omitempty"` + CompartmentId string `json:"compartmentId,omitempty"` + KeyName string `json:"keyName,omitempty"` + EncryptionAlgo string `json:"encryptionAlgo,omitempty"` + VaultType string `json:"vaultType,omitempty"` +} +type KMSDetailsStatus struct { + VaultId string `json:"vaultId,omitempty"` + ManagementEndpoint string `json:"managementEndpoint,omitempty"` + KeyId string `json:"keyId,omitempty"` + VaultName string `json:"vaultName,omitempty"` + CompartmentId string `json:"compartmentId,omitempty"` + KeyName string `json:"keyName,omitempty"` + EncryptionAlgo string `json:"encryptionAlgo,omitempty"` + VaultType string `json:"vaultType,omitempty"` +} + +const ( + lastSuccessfulKMSConfig = "lastSuccessfulKMSConfig" + lastSuccessfulKMSStatus = "lastSuccessfulKMSStatus" +) + +// GetLastSuccessfulKMSConfig returns the KMS config from the last successful reconciliation. +// Returns nil, nil if there is no lastSuccessfulKMSConfig. +func (dbcs *DbcsSystem) GetLastSuccessfulKMSConfig() (*KMSConfig, error) { + val, ok := dbcs.GetAnnotations()[lastSuccessfulKMSConfig] + if !ok { + return nil, nil + } + + configBytes := []byte(val) + kmsConfig := KMSConfig{} + + err := json.Unmarshal(configBytes, &kmsConfig) + if err != nil { + return nil, err + } + + return &kmsConfig, nil +} + +// GetLastSuccessfulKMSStatus returns the KMS status from the last successful reconciliation. +// Returns nil, nil if there is no lastSuccessfulKMSStatus. +func (dbcs *DbcsSystem) GetLastSuccessfulKMSStatus() (*KMSDetailsStatus, error) { + val, ok := dbcs.GetAnnotations()[lastSuccessfulKMSStatus] + if !ok { + return nil, nil + } + + statusBytes := []byte(val) + kmsStatus := KMSDetailsStatus{} + + err := json.Unmarshal(statusBytes, &kmsStatus) + if err != nil { + return nil, err + } + + return &kmsStatus, nil +} + +// SetLastSuccessfulKMSConfig saves the given KMSConfig to the annotations. +func (dbcs *DbcsSystem) SetLastSuccessfulKMSConfig(kmsConfig *KMSConfig) error { + configBytes, err := json.Marshal(kmsConfig) + if err != nil { + return err + } + + annotations := dbcs.GetAnnotations() + if annotations == nil { + annotations = make(map[string]string) + } + annotations[lastSuccessfulKMSConfig] = string(configBytes) + dbcs.SetAnnotations(annotations) + return nil +} + +// SetLastSuccessfulKMSStatus saves the given KMSDetailsStatus to the annotations. +func (dbcs *DbcsSystem) SetLastSuccessfulKMSStatus(kmsStatus *KMSDetailsStatus) error { + statusBytes, err := json.Marshal(kmsStatus) + if err != nil { + return err + } + + annotations := dbcs.GetAnnotations() + if annotations == nil { + annotations = make(map[string]string) + } + annotations[lastSuccessfulKMSStatus] = string(statusBytes) + dbcs.SetAnnotations(annotations) + // Update KMSDetailsStatus in DbcsSystemStatus + dbcs.Status.KMSDetailsStatus = KMSDetailsStatus{ + VaultName: kmsStatus.VaultName, + CompartmentId: kmsStatus.CompartmentId, + KeyName: kmsStatus.KeyName, + EncryptionAlgo: kmsStatus.EncryptionAlgo, + VaultType: kmsStatus.VaultType, + } + return nil +} diff --git a/apis/database/v4/dbcssystem_pdbconfig_types.go b/apis/database/v4/dbcssystem_pdbconfig_types.go new file mode 100644 index 00000000..2ae361e5 --- /dev/null +++ b/apis/database/v4/dbcssystem_pdbconfig_types.go @@ -0,0 +1,83 @@ +/* +** Copyright (c) 2022-2024 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ +package v4 + +// PDBConfig defines details of PDB struct for DBCS systems +type PDBConfig struct { + // The name for the pluggable database (PDB). The name is unique in the context of a Database. The name must begin with an alphabetic character and can contain a maximum of thirty alphanumeric characters. Special characters are not permitted. The pluggable database name should not be same as the container database name. + PdbName *string `mandatory:"true" json:"pdbName"` + + // The OCID (https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm) of the CDB + // ContainerDatabaseId *string `mandatory:"false" json:"containerDatabaseId"` + + // // A strong password for PDB Admin. The password must be at least nine characters and contain at least two uppercase, two lowercase, two numbers, and two special characters. The special characters must be _, \#, or -. + PdbAdminPassword *string `mandatory:"false" json:"pdbAdminPassword"` + + // // The existing TDE wallet password of the CDB. + TdeWalletPassword *string `mandatory:"false" json:"tdeWalletPassword"` + + // // The locked mode of the pluggable database admin account. If false, the user needs to provide the PDB Admin Password to connect to it. + // // If true, the pluggable database will be locked and user cannot login to it. + ShouldPdbAdminAccountBeLocked *bool `mandatory:"false" json:"shouldPdbAdminAccountBeLocked"` + + // // Free-form tags for this resource. Each tag is a simple key-value pair with no predefined name, type, or namespace. + // // For more information, see Resource Tags (https://docs.cloud.oracle.com/Content/General/Concepts/resourcetags.htm). + // // Example: `{"Department": "Finance"}` + FreeformTags map[string]string `mandatory:"false" json:"freeformTags"` + + // // Defined tags for this resource. Each key is predefined and scoped to a namespace. + // // For more information, see Resource Tags (https://docs.cloud.oracle.com/Content/General/Concepts/resourcetags.htm). + // DefinedTags map[string]map[string]interface{} `mandatory:"false" json:"definedTags"` + + // To specify whether to delete the PDB + IsDelete *bool `mandatory:"false" json:"isDelete,omitempty"` + + // The OCID of the PDB for deletion purposes. + PluggableDatabaseId *string `mandatory:"false" json:"pluggableDatabaseId,omitempty"` +} + +type PDBConfigStatus struct { + PdbName *string `mandatory:"false" json:"pdbName,omitempty"` + ShouldPdbAdminAccountBeLocked *bool `mandatory:"false" json:"shouldPdbAdminAccountBeLocked,omitempty"` + FreeformTags map[string]string `mandatory:"false" json:"freeformTags,omitempty"` + PluggableDatabaseId *string `mandatory:"false" json:"pluggableDatabaseId,omitempty"` + PdbLifecycleState LifecycleState `json:"pdbState,omitempty"` +} +type PDBDetailsStatus struct { + PDBConfigStatus []PDBConfigStatus `json:"pdbConfigStatus,omitempty"` +} diff --git a/apis/database/v4/dbcssystem_types.go b/apis/database/v4/dbcssystem_types.go new file mode 100644 index 00000000..9810a3b7 --- /dev/null +++ b/apis/database/v4/dbcssystem_types.go @@ -0,0 +1,292 @@ +/* +** Copyright (c) 2022-2024 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ +package v4 + +import ( + "encoding/json" + + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + + "github.com/go-logr/logr" + dbcsv1 "github.com/oracle/oracle-database-operator/commons/annotations" + + "sigs.k8s.io/controller-runtime/pkg/client" +) + +// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! +// NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized. + +// DbcsSystemSpec defines the desired state of DbcsSystem +type DbcsSystemSpec struct { + DbSystem DbSystemDetails `json:"dbSystem,omitempty"` + Id *string `json:"id,omitempty"` + OCIConfigMap *string `json:"ociConfigMap"` + OCISecret *string `json:"ociSecret,omitempty"` + DbClone *DbCloneConfig `json:"dbClone,omitempty"` + HardLink bool `json:"hardLink,omitempty"` + PdbConfigs []PDBConfig `json:"pdbConfigs,omitempty"` + SetupDBCloning bool `json:"setupDBCloning,omitempty"` + DbBackupId *string `json:"dbBackupId,omitempty"` + DatabaseId *string `json:"databaseId,omitempty"` + KMSConfig KMSConfig `json:"kmsConfig,omitempty"` +} + +// DbSystemDetails Spec + +type DbSystemDetails struct { + CompartmentId string `json:"compartmentId"` + AvailabilityDomain string `json:"availabilityDomain"` + SubnetId string `json:"subnetId"` + Shape string `json:"shape"` + SshPublicKeys []string `json:"sshPublicKeys,omitempty"` + HostName string `json:"hostName"` + CpuCoreCount int `json:"cpuCoreCount,omitempty"` + FaultDomains []string `json:"faultDomains,omitempty"` + DisplayName string `json:"displayName,omitempty"` + BackupSubnetId string `json:"backupSubnetId,omitempty"` + TimeZone string `json:"timeZone,omitempty"` + NodeCount *int `json:"nodeCount,omitempty"` + PrivateIp string `json:"privateIp,omitempty"` + Domain string `json:"domain,omitempty"` + InitialDataStorageSizeInGB int `json:"initialDataStorageSizeInGB,omitempty"` + ClusterName string `json:"clusterName,omitempty"` + DbAdminPasswordSecret string `json:"dbAdminPasswordSecret"` + DbName string `json:"dbName,omitempty"` + PdbName string `json:"pdbName,omitempty"` + DbDomain string `json:"dbDomain,omitempty"` + DbUniqueName string `json:"dbUniqueName,omitempty"` + StorageManagement string `json:"storageManagement,omitempty"` + DbVersion string `json:"dbVersion,omitempty"` + DbEdition string `json:"dbEdition,omitempty"` + DiskRedundancy string `json:"diskRedundancy,omitempty"` + DbWorkload string `json:"dbWorkload,omitempty"` + LicenseModel string `json:"licenseModel,omitempty"` + TdeWalletPasswordSecret string `json:"tdeWalletPasswordSecret,omitempty"` + Tags map[string]string `json:"tags,omitempty"` + DbBackupConfig Backupconfig `json:"dbBackupConfig,omitempty"` + KMSConfig KMSConfig `json:"kmsConfig,omitempty"` +} + +// DB Backup Config Network Struct +type Backupconfig struct { + AutoBackupEnabled *bool `json:"autoBackupEnabled,omitempty"` + RecoveryWindowsInDays *int `json:"recoveryWindowsInDays,omitempty"` + AutoBackupWindow *string `json:"autoBackupWindow,omitempty"` + BackupDestinationDetails *string `json:"backupDestinationDetails,omitempty"` +} + +// DbcsSystemStatus defines the observed state of DbcsSystem +type DbcsSystemStatus struct { + Id *string `json:"id,omitempty"` + DisplayName string `json:"displayName,omitempty"` + AvailabilityDomain string `json:"availabilityDomain,omitempty"` + SubnetId string `json:"subnetId,omitempty"` + StorageManagement string `json:"storageManagement,omitempty"` + NodeCount int `json:"nodeCount,omitempty"` + CpuCoreCount int `json:"cpuCoreCount,omitempty"` + + DbEdition string `json:"dbEdition,omitempty"` + TimeZone string `json:"timeZone,omitempty"` + DataStoragePercentage *int `json:"dataStoragePercentage,omitempty"` + LicenseModel string `json:"licenseModel,omitempty"` + DataStorageSizeInGBs *int `json:"dataStorageSizeInGBs,omitempty"` + RecoStorageSizeInGB *int `json:"recoStorageSizeInGB,omitempty"` + + Shape *string `json:"shape,omitempty"` + State LifecycleState `json:"state"` + DbInfo []DbStatus `json:"dbInfo,omitempty"` + Network VmNetworkDetails `json:"network,omitempty"` + WorkRequests []DbWorkrequests `json:"workRequests,omitempty"` + KMSDetailsStatus KMSDetailsStatus `json:"kmsDetailsStatus,omitempty"` + DbCloneStatus DbCloneStatus `json:"dbCloneStatus,omitempty"` + PdbDetailsStatus []PDBDetailsStatus `json:"pdbDetailsStatus,omitempty"` +} + +// DbcsSystemStatus defines the observed state of DbcsSystem +type DbStatus struct { + Id *string `json:"id,omitempty"` + DbName string `json:"dbName,omitempty"` + DbUniqueName string `json:"dbUniqueName,omitempty"` + DbWorkload string `json:"dbWorkload,omitempty"` + DbHomeId string `json:"dbHomeId,omitempty"` +} + +type DbWorkrequests struct { + OperationType *string `json:"operationType,omitmpty"` + OperationId *string `json:"operationId,omitemty"` + PercentComplete string `json:"percentComplete,omitempty"` + TimeAccepted string `json:"timeAccepted,omitempty"` + TimeStarted string `json:"timeStarted,omitempty"` + TimeFinished string `json:"timeFinished,omitempty"` +} + +type VmNetworkDetails struct { + VcnName *string `json:"vcnName,omitempty"` + SubnetName *string `json:"clientSubnet,omitempty"` + ScanDnsName *string `json:"scanDnsName,omitempty"` + HostName string `json:"hostName,omitempty"` + DomainName string `json:"domainName,omitempty"` + ListenerPort *int `json:"listenerPort,omitempty"` + NetworkSG string `json:"networkSG,omitempty"` +} + +// DbCloneConfig defines the configuration for the database clone +type DbCloneConfig struct { + DbAdminPasswordSecret string `json:"dbAdminPasswordSecret,omitempty"` + TdeWalletPasswordSecret string `json:"tdeWalletPasswordSecret,omitempty"` + DbName string `json:"dbName"` + HostName string `json:"hostName"` + DbUniqueName string `json:"dbDbUniqueName"` + DisplayName string `json:"displayName"` + LicenseModel string `json:"licenseModel,omitempty"` + Domain string `json:"domain,omitempty"` + SshPublicKeys []string `json:"sshPublicKeys,omitempty"` + SubnetId string `json:"subnetId"` + SidPrefix string `json:"sidPrefix,omitempty"` + InitialDataStorageSizeInGB int `json:"initialDataStorageSizeInGB,omitempty"` + KmsKeyId string `json:"kmsKeyId,omitempty"` + KmsKeyVersionId string `json:"kmsKeyVersionId,omitempty"` + PrivateIp string `json:"privateIp,omitempty"` +} + +// DbCloneStatus defines the observed state of DbClone +type DbCloneStatus struct { + Id *string `json:"id,omitempty"` + DbAdminPaswordSecret string `json:"dbAdminPaswordSecret,omitempty"` + DbName string `json:"dbName,omitempty"` + HostName string `json:"hostName"` + DbUniqueName string `json:"dbDbUniqueName"` + DisplayName string `json:"displayName,omitempty"` + LicenseModel string `json:"licenseModel,omitempty"` + Domain string `json:"domain,omitempty"` + SshPublicKeys []string `json:"sshPublicKeys,omitempty"` + SubnetId string `json:"subnetId,omitempty"` +} + +// +kubebuilder:object:root=true +// +kubebuilder:subresource:status +// +kubebuilder:resource:path=dbcssystems,scope=Namespaced +// +kubebuilder:storageversion +// +kubebuilder:storageversion + +// DbcsSystem is the Schema for the dbcssystems API +type DbcsSystem struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + Spec DbcsSystemSpec `json:"spec,omitempty"` + Status DbcsSystemStatus `json:"status,omitempty"` +} + +//+kubebuilder:object:root=true + +// DbcsSystemList contains a list of DbcsSystem +type DbcsSystemList struct { + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata,omitempty"` + Items []DbcsSystem `json:"items"` +} + +type LifecycleState string + +const ( + Available LifecycleState = "AVAILABLE" + Failed LifecycleState = "FAILED" + Update LifecycleState = "UPDATING" + Provision LifecycleState = "PROVISIONING" + Terminate LifecycleState = "TERMINATED" +) + +const lastSuccessfulSpec = "lastSuccessfulSpec" + +// GetLastSuccessfulSpec returns spec from the lass successful reconciliation. +// Returns nil, nil if there is no lastSuccessfulSpec. +func (dbcs *DbcsSystem) GetLastSuccessfulSpec() (*DbcsSystemSpec, error) { + val, ok := dbcs.GetAnnotations()[lastSuccessfulSpec] + if !ok { + return nil, nil + } + + specBytes := []byte(val) + sucSpec := DbcsSystemSpec{} + + err := json.Unmarshal(specBytes, &sucSpec) + if err != nil { + return nil, err + } + + return &sucSpec, nil +} +func (dbcs *DbcsSystem) GetLastSuccessfulSpecWithLog(log logr.Logger) (*DbcsSystemSpec, error) { + val, ok := dbcs.GetAnnotations()[lastSuccessfulSpec] + if !ok { + log.Info("No last successful spec annotation found") + return nil, nil + } + + specBytes := []byte(val) + sucSpec := DbcsSystemSpec{} + + err := json.Unmarshal(specBytes, &sucSpec) + if err != nil { + log.Error(err, "Failed to unmarshal last successful spec") + return nil, err + } + + log.Info("Successfully retrieved last successful spec", "spec", sucSpec) + return &sucSpec, nil +} + +// UpdateLastSuccessfulSpec updates lastSuccessfulSpec with the current spec. +func (dbcs *DbcsSystem) UpdateLastSuccessfulSpec(kubeClient client.Client) error { + specBytes, err := json.Marshal(dbcs.Spec) + if err != nil { + return err + } + + anns := map[string]string{ + lastSuccessfulSpec: string(specBytes), + } + + // return dbcsv1.SetAnnotations(kubeClient, dbcs, anns) + return dbcsv1.PatchAnnotations(kubeClient, dbcs, anns) + +} + +func init() { + SchemeBuilder.Register(&DbcsSystem{}, &DbcsSystemList{}) +} diff --git a/apis/database/v4/dbcssystem_webhook.go b/apis/database/v4/dbcssystem_webhook.go new file mode 100644 index 00000000..c3ff8ddb --- /dev/null +++ b/apis/database/v4/dbcssystem_webhook.go @@ -0,0 +1,98 @@ +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v4 + +import ( + "k8s.io/apimachinery/pkg/runtime" + ctrl "sigs.k8s.io/controller-runtime" + logf "sigs.k8s.io/controller-runtime/pkg/log" + "sigs.k8s.io/controller-runtime/pkg/webhook" + "sigs.k8s.io/controller-runtime/pkg/webhook/admission" +) + +// log is for logging in this package. +var dbcssystemlog = logf.Log.WithName("dbcssystem-resource") + +func (r *DbcsSystem) SetupWebhookWithManager(mgr ctrl.Manager) error { + return ctrl.NewWebhookManagedBy(mgr). + For(r). + Complete() +} + +// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! + +//+kubebuilder:webhook:path=/mutate-database-oracle-com-v4-dbcssystem,mutating=true,failurePolicy=fail,sideEffects=none,groups=database.oracle.com,resources=dbcssystems,verbs=create;update,versions=v4,name=mdbcssystemv4.kb.io,admissionReviewVersions=v1 + +var _ webhook.Defaulter = &DbcsSystem{} + +// Default implements webhook.Defaulter so a webhook will be registered for the type +func (r *DbcsSystem) Default() { + dbcssystemlog.Info("default", "name", r.Name) + + // TODO(user): fill in your defaulting logic. +} + +// TODO(user): change verbs to "verbs=create;update;delete" if you want to enable deletion validation. + +// +kubebuilder:webhook:verbs=create;update;delete,path=/validate-database-oracle-com-v4-dbcssystem,mutating=false,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=dbcssystems,versions=v4,name=vdbcssystemv4.kb.io,admissionReviewVersions=v1 +var _ webhook.Validator = &DbcsSystem{} + +// ValidateCreate implements webhook.Validator so a webhook will be registered for the type +func (r *DbcsSystem) ValidateCreate() (admission.Warnings, error) { + dbcssystemlog.Info("validate create", "name", r.Name) + + // // TODO(user): fill in your validation logic upon object creation. + return nil, nil +} + +// // ValidateUpdate implements webhook.Validator so a webhook will be registered for the type +func (r *DbcsSystem) ValidateUpdate(old runtime.Object) (admission.Warnings, error) { + dbcssystemlog.Info("validate update", "name", r.Name) + + // // TODO(user): fill in your validation logic upon object update. + return nil, nil +} + +// ValidateDelete implements webhook.Validator so a webhook will be registered for the type +func (r *DbcsSystem) ValidateDelete() (admission.Warnings, error) { + dbcssystemlog.Info("validate delete", "name", r.Name) + + // TODO(user): fill in your validation logic upon object deletion. + return nil, nil +} diff --git a/apis/database/v4/groupversion_info.go b/apis/database/v4/groupversion_info.go new file mode 100644 index 00000000..6644b93c --- /dev/null +++ b/apis/database/v4/groupversion_info.go @@ -0,0 +1,58 @@ +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +// Package v4 contains API Schema definitions for the database v4 API group +// +kubebuilder:object:generate=true +// +groupName=database.oracle.com +package v4 + +import ( + "k8s.io/apimachinery/pkg/runtime/schema" + "sigs.k8s.io/controller-runtime/pkg/scheme" +) + +var ( + // GroupVersion is group version used to register these objects + GroupVersion = schema.GroupVersion{Group: "database.oracle.com", Version: "v4"} + + // SchemeBuilder is used to add go types to the GroupVersionKind scheme + SchemeBuilder = &scheme.Builder{GroupVersion: GroupVersion} + + // AddToScheme adds the types in this group-version to the given scheme. + AddToScheme = SchemeBuilder.AddToScheme +) diff --git a/apis/database/v4/lrest_types.go b/apis/database/v4/lrest_types.go new file mode 100644 index 00000000..421a3ea1 --- /dev/null +++ b/apis/database/v4/lrest_types.go @@ -0,0 +1,191 @@ +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v4 + +import ( + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +// LRESTSpec defines the desired state of LREST +type LRESTSpec struct { + // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster + // Important: Run "make" to regenerate code after modifying this file + + // Name of the LREST + LRESTName string `json:"cdbName,omitempty"` + // Name of the LREST Service + ServiceName string `json:"serviceName,omitempty"` + + // Password for the LREST System Administrator + SysAdminPwd LRESTSysAdminPassword `json:"sysAdminPwd,omitempty"` + // User in the root container with sysdba priviledges to manage PDB lifecycle + LRESTAdminUser LRESTAdminUser `json:"cdbAdminUser,omitempty"` + // Password for the LREST Administrator to manage PDB lifecycle + LRESTAdminPwd LRESTAdminPassword `json:"cdbAdminPwd,omitempty"` + + LRESTTlsKey LRESTTLSKEY `json:"cdbTlsKey,omitempty"` + LRESTTlsCrt LRESTTLSCRT `json:"cdbTlsCrt,omitempty"` + LRESTPubKey LRESTPUBKEY `json:"cdbPubKey,omitempty"` + LRESTPriKey LRESTPRVKEY `json:"cdbPrvKey,omitempty"` + + // Password for user LREST_PUBLIC_USER + LRESTPwd LRESTPassword `json:"lrestPwd,omitempty"` + // LREST server port. For now, keep it as 8888. TO BE USED IN FUTURE RELEASE. + LRESTPort int `json:"lrestPort,omitempty"` + // LREST Image Name + LRESTImage string `json:"lrestImage,omitempty"` + // The name of the image pull secret in case of a private docker repository. + LRESTImagePullSecret string `json:"lrestImagePullSecret,omitempty"` + // LREST Image Pull Policy + // +kubebuilder:validation:Enum=Always;Never + LRESTImagePullPolicy string `json:"lrestImagePullPolicy,omitempty"` + // Number of LREST Containers to create + Replicas int `json:"replicas,omitempty"` + // Web Server User with SQL Administrator role to allow us to authenticate to the PDB Lifecycle Management REST endpoints + WebLrestServerUser WebLrestServerUser `json:"webServerUser,omitempty"` + // Password for the Web Server User + WebLrestServerPwd WebLrestServerPassword `json:"webServerPwd,omitempty"` + // Name of the DB server + DBServer string `json:"dbServer,omitempty"` + // DB server port + DBPort int `json:"dbPort,omitempty"` + // Node Selector for running the Pod + NodeSelector map[string]string `json:"nodeSelector,omitempty"` + DBTnsurl string `json:"dbTnsurl,omitempty"` + DeletePDBCascade bool `json:"deletePdbCascade,omitempty"` +} + +// LRESTSecret defines the secretName +type LRESTSecret struct { + SecretName string `json:"secretName"` + Key string `json:"key"` +} + +// LRESTSysAdminPassword defines the secret containing SysAdmin Password mapped to key 'sysAdminPwd' for LREST +type LRESTSysAdminPassword struct { + Secret LRESTSecret `json:"secret"` +} + +// LRESTAdminUser defines the secret containing LREST Administrator User mapped to key 'lrestAdminUser' to manage PDB lifecycle +type LRESTAdminUser struct { + Secret LRESTSecret `json:"secret"` +} + +// LRESTAdminPassword defines the secret containing LREST Administrator Password mapped to key 'lrestAdminPwd' to manage PDB lifecycle +type LRESTAdminPassword struct { + Secret LRESTSecret `json:"secret"` +} + +// LRESTPassword defines the secret containing LREST_PUBLIC_USER Password mapped to key 'ordsPwd' +type LRESTPassword struct { + Secret LRESTSecret `json:"secret"` +} + +// WebLrestServerUser defines the secret containing Web Server User mapped to key 'webServerUser' to manage PDB lifecycle +type WebLrestServerUser struct { + Secret LRESTSecret `json:"secret"` +} + +// WebLrestServerPassword defines the secret containing password for Web Server User mapped to key 'webServerPwd' to manage PDB lifecycle +type WebLrestServerPassword struct { + Secret LRESTSecret `json:"secret"` +} + +type LRESTTLSKEY struct { + Secret LRESTSecret `json:"secret"` +} + +type LRESTTLSCRT struct { + Secret LRESTSecret `json:"secret"` +} + +type LRESTPUBKEY struct { + Secret LRESTSecret `json:"secret"` +} + +type LRESTPRVKEY struct { + Secret LRESTSecret `json:"secret"` +} + +// LRESTStatus defines the observed state of LREST +type LRESTStatus struct { + // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster + // Important: Run "make" to regenerate code after modifying this file + + // Phase of the LREST Resource + Phase string `json:"phase"` + // LREST Resource Status + Status bool `json:"status"` + // Message + Msg string `json:"msg,omitempty"` +} + +// +kubebuilder:object:root=true +// +kubebuilder:subresource:status +// +kubebuilder:printcolumn:JSONPath=".spec.cdbName",name="CDB NAME",type="string",description="Name of the LREST" +// +kubebuilder:printcolumn:JSONPath=".spec.dbServer",name="DB Server",type="string",description=" Name of the DB Server" +// +kubebuilder:printcolumn:JSONPath=".spec.dbPort",name="DB Port",type="integer",description="DB server port" +// +kubebuilder:printcolumn:JSONPath=".spec.replicas",name="Replicas",type="integer",description="Replicas" +// +kubebuilder:printcolumn:JSONPath=".status.phase",name="Status",type="string",description="Status of the LREST Resource" +// +kubebuilder:printcolumn:JSONPath=".status.msg",name="Message",type="string",description="Error message if any" +// +kubebuilder:printcolumn:JSONPath=".spec.dbTnsurl",name="TNS STRING",type="string",description="string of the tnsalias" +// +kubebuilder:resource:path=lrests,scope=Namespaced +// +kubebuilder:storageversion + +// LREST is the Schema for the lrests API +type LREST struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + Spec LRESTSpec `json:"spec,omitempty"` + Status LRESTStatus `json:"status,omitempty"` +} + +//+kubebuilder:object:root=true + +// LRESTList contains a list of LREST +type LRESTList struct { + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata,omitempty"` + Items []LREST `json:"items"` +} + +func init() { + SchemeBuilder.Register(&LREST{}, &LRESTList{}) +} diff --git a/apis/database/v4/lrest_webhook.go b/apis/database/v4/lrest_webhook.go new file mode 100644 index 00000000..9d65a1d6 --- /dev/null +++ b/apis/database/v4/lrest_webhook.go @@ -0,0 +1,219 @@ +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v4 + +import ( + "reflect" + "strings" + + apierrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/runtime/schema" + "k8s.io/apimachinery/pkg/util/validation/field" + ctrl "sigs.k8s.io/controller-runtime" + logf "sigs.k8s.io/controller-runtime/pkg/log" + "sigs.k8s.io/controller-runtime/pkg/webhook" + "sigs.k8s.io/controller-runtime/pkg/webhook/admission" +) + +// log is for logging in this package. +var lrestlog = logf.Log.WithName("lrest-webhook") + +func (r *LREST) SetupWebhookWithManager(mgr ctrl.Manager) error { + return ctrl.NewWebhookManagedBy(mgr). + For(r). + Complete() +} + +//+kubebuilder:webhook:path=/mutate-database-oracle-com-v4-lrest,mutating=true,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=lrests,verbs=create;update,versions=v4,name=mlrest.kb.io,admissionReviewVersions={v4,v1beta1} + +var _ webhook.Defaulter = &LREST{} + +// Default implements webhook.Defaulter so a webhook will be registered for the type +func (r *LREST) Default() { + lrestlog.Info("Setting default values in LREST spec for : " + r.Name) + + if r.Spec.LRESTPort == 0 { + r.Spec.LRESTPort = 8888 + } + + if r.Spec.Replicas == 0 { + r.Spec.Replicas = 1 + } +} + +// TODO(user): change verbs to "verbs=create;update;delete" if you want to enable deletion validation. +//+kubebuilder:webhook:path=/validate-database-oracle-com-v4-lrest,mutating=false,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=lrests,verbs=create;update,versions=v4,name=vlrest.kb.io,admissionReviewVersions={v4,v1beta1} + +var _ webhook.Validator = &LREST{} + +// ValidateCreate implements webhook.Validator so a webhook will be registered for the type +func (r *LREST) ValidateCreate() (admission.Warnings, error) { + lrestlog.Info("ValidateCreate", "name", r.Name) + + var allErrs field.ErrorList + + if r.Spec.ServiceName == "" && r.Spec.DBServer != "" { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("serviceName"), "Please specify LREST Service name")) + } + + if reflect.ValueOf(r.Spec.LRESTTlsKey).IsZero() { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("lrestTlsKey"), "Please specify LREST Tls key(secret)")) + } + + if reflect.ValueOf(r.Spec.LRESTTlsCrt).IsZero() { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("lrestTlsCrt"), "Please specify LREST Tls Certificate(secret)")) + } + + /*if r.Spec.SCANName == "" { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("scanName"), "Please specify SCAN Name for LREST")) + }*/ + + if (r.Spec.DBServer == "" && r.Spec.DBTnsurl == "") || (r.Spec.DBServer != "" && r.Spec.DBTnsurl != "") { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("dbServer"), "Please specify Database Server Name/IP Address or tnsalias string")) + } + + if r.Spec.DBTnsurl != "" && (r.Spec.DBServer != "" || r.Spec.DBPort != 0 || r.Spec.ServiceName != "") { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("dbServer"), "DBtnsurl is orthogonal to (DBServer,DBport,Services)")) + } + + if r.Spec.DBPort == 0 && r.Spec.DBServer != "" { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("dbPort"), "Please specify DB Server Port")) + } + if r.Spec.DBPort < 0 && r.Spec.DBServer != "" { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("dbPort"), "Please specify a valid DB Server Port")) + } + if r.Spec.LRESTPort < 0 { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("ordsPort"), "Please specify a valid LREST Port")) + } + if r.Spec.Replicas < 0 { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("replicas"), "Please specify a valid value for Replicas")) + } + if r.Spec.LRESTImage == "" { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("ordsImage"), "Please specify name of LREST Image to be used")) + } + if reflect.ValueOf(r.Spec.LRESTAdminUser).IsZero() { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("lrestAdminUser"), "Please specify user in the root container with sysdba priviledges to manage PDB lifecycle")) + } + if reflect.ValueOf(r.Spec.LRESTAdminPwd).IsZero() { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("lrestAdminPwd"), "Please specify password for the LREST Administrator to manage PDB lifecycle")) + } + /* if reflect.ValueOf(r.Spec.LRESTPwd).IsZero() { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("ordsPwd"), "Please specify password for user LREST_PUBLIC_USER")) + } */ + if reflect.ValueOf(r.Spec.WebLrestServerUser).IsZero() { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("webLrestServerUser"), "Please specify the Web Server User having SQL Administrator role")) + } + if reflect.ValueOf(r.Spec.WebLrestServerPwd).IsZero() { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("webServerPwd"), "Please specify password for the Web Server User having SQL Administrator role")) + } + if len(allErrs) == 0 { + return nil, nil + } + return nil, apierrors.NewInvalid( + schema.GroupKind{Group: "database.oracle.com", Kind: "LREST"}, + r.Name, allErrs) +} + +// ValidateUpdate implements webhook.Validator so a webhook will be registered for the type +func (r *LREST) ValidateUpdate(old runtime.Object) (admission.Warnings, error) { + lrestlog.Info("validate update", "name", r.Name) + + isLRESTMarkedToBeDeleted := r.GetDeletionTimestamp() != nil + if isLRESTMarkedToBeDeleted { + return nil, nil + } + + var allErrs field.ErrorList + + // Check for updation errors + oldLREST, ok := old.(*LREST) + if !ok { + return nil, nil + } + + if r.Spec.DBPort < 0 { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("dbPort"), "Please specify a valid DB Server Port")) + } + if r.Spec.LRESTPort < 0 { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("ordsPort"), "Please specify a valid LREST Port")) + } + if r.Spec.Replicas < 0 { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("replicas"), "Please specify a valid value for Replicas")) + } + if !strings.EqualFold(oldLREST.Spec.ServiceName, r.Spec.ServiceName) { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("replicas"), "cannot be changed")) + } + + if len(allErrs) == 0 { + return nil, nil + } + + return nil, apierrors.NewInvalid( + schema.GroupKind{Group: "database.oracle.com", Kind: "LREST"}, + r.Name, allErrs) +} + +// ValidateDelete implements webhook.Validator so a webhook will be registered for the type +func (r *LREST) ValidateDelete() (admission.Warnings, error) { + lrestlog.Info("validate delete", "name", r.Name) + + // TODO(user): fill in your validation logic upon object deletion. + return nil, nil +} diff --git a/apis/database/v4/lrpdb_types.go b/apis/database/v4/lrpdb_types.go new file mode 100644 index 00000000..d37bebdc --- /dev/null +++ b/apis/database/v4/lrpdb_types.go @@ -0,0 +1,256 @@ +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v4 + +import ( + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +// LRPDBSpec defines the desired state of LRPDB +type LRPDBSpec struct { + // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster + // Important: Run "make" to regenerate code after modifying this file + + LRPDBTlsKey LRPDBTLSKEY `json:"lrpdbTlsKey,omitempty"` + LRPDBTlsCrt LRPDBTLSCRT `json:"lrpdbTlsCrt,omitempty"` + LRPDBTlsCat LRPDBTLSCAT `json:"lrpdbTlsCat,omitempty"` + LRPDBPriKey LRPDBPRVKEY `json:"cdbPrvKey,omitempty"` + + // Namespace of the rest server + CDBNamespace string `json:"cdbNamespace,omitempty"` + // Name of the CDB Custom Resource that runs the LREST container + CDBResName string `json:"cdbResName,omitempty"` + // Name of the CDB + CDBName string `json:"cdbName,omitempty"` + // The name of the new LRPDB. Relevant for both Create and Plug Actions. + LRPDBName string `json:"pdbName,omitempty"` + // Name of the Source LRPDB from which to clone + SrcLRPDBName string `json:"srcPdbName,omitempty"` + // The administrator username for the new LRPDB. This property is required when the Action property is Create. + AdminName LRPDBAdminName `json:"adminName,omitempty"` + // The administrator password for the new LRPDB. This property is required when the Action property is Create. + AdminPwd LRPDBAdminPassword `json:"adminPwd,omitempty"` + // Relevant for Create and Plug operations. As defined in the Oracle Multitenant Database documentation. Values can be a filename convert pattern or NONE. + AdminpdbUser AdminpdbUser `json:"adminpdbUser,omitempty"` + AdminpdbPass AdminpdbPass `json:"adminpdbPass,omitempty"` + + FileNameConversions string `json:"fileNameConversions,omitempty"` + // This property is required when the Action property is Plug. As defined in the Oracle Multitenant Database documentation. Values can be a source filename convert pattern or NONE. + SourceFileNameConversions string `json:"sourceFileNameConversions,omitempty"` + // XML metadata filename to be used for Plug or Unplug operations + XMLFileName string `json:"xmlFileName,omitempty"` + // To copy files or not while cloning a LRPDB + // +kubebuilder:validation:Enum=COPY;NOCOPY;MOVE + CopyAction string `json:"copyAction,omitempty"` + // Specify if datafiles should be removed or not. The value can be INCLUDING or KEEP (default). + // +kubebuilder:validation:Enum=INCLUDING;KEEP + DropAction string `json:"dropAction,omitempty"` + // A Path specified for sparse clone snapshot copy. (Optional) + SparseClonePath string `json:"sparseClonePath,omitempty"` + // Whether to reuse temp file + ReuseTempFile *bool `json:"reuseTempFile,omitempty"` + // Relevant for Create and Plug operations. True for unlimited storage. Even when set to true, totalSize and tempSize MUST be specified in the request if Action is Create. + UnlimitedStorage *bool `json:"unlimitedStorage,omitempty"` + // Indicate if 'AS CLONE' option should be used in the command to plug in a LRPDB. This property is applicable when the Action property is PLUG but not required. + AsClone *bool `json:"asClone,omitempty"` + // Relevant for create and plug operations. Total size as defined in the Oracle Multitenant Database documentation. See size_clause description in Database SQL Language Reference documentation. + TotalSize string `json:"totalSize,omitempty"` + // Relevant for Create and Clone operations. Total size for temporary tablespace as defined in the Oracle Multitenant Database documentation. See size_clause description in Database SQL Language Reference documentation. + TempSize string `json:"tempSize,omitempty"` + // Web Server User with SQL Administrator role to allow us to authenticate to the PDB Lifecycle Management REST endpoints + WebLrpdbServerUser WebLrpdbServerUser `json:"webServerUser,omitempty"` + // Password for the Web Server User + WebLrpdbServerPwd WebLrpdbServerPassword `json:"webServerPwd,omitempt"` + // TDE import for plug operations + LTDEImport *bool `json:"tdeImport,omitempty"` + // LTDE export for unplug operations + LTDEExport *bool `json:"tdeExport,omitempty"` + // TDE password if the tdeImport or tdeExport flag is set to true. Can be used in create, plug or unplug operations + LTDEPassword LTDEPwd `json:"tdePassword,omitempty"` + // LTDE keystore path is required if the tdeImport or tdeExport flag is set to true. Can be used in plug or unplug operations. + LTDEKeystorePath string `json:"tdeKeystorePath,omitempty"` + // LTDE secret is required if the tdeImport or tdeExport flag is set to true. Can be used in plug or unplug operations. + LTDESecret LTDESecret `json:"tdeSecret,omitempty"` + // Whether you need the script only or execute the script + GetScript *bool `json:"getScript,omitempty"` + // Action to be taken: Create/Clone/Plug/Unplug/Delete/Modify/Status/Map/Alter. Map is used to map a Databse LRPDB to a Kubernetes LRPDB CR. + // +kubebuilder:validation:Enum=Create;Clone;Plug;Unplug;Delete;Modify;Status;Map;Alter;Noaction + Action string `json:"action"` + // Extra options for opening and closing a LRPDB + // +kubebuilder:validation:Enum=IMMEDIATE;NORMAL;READ ONLY;READ WRITE;RESTRICTED + ModifyOption string `json:"modifyOption,omitempty"` + // to be used with ALTER option - obsolete do not use + AlterSystem string `json:"alterSystem,omitempty"` + // to be used with ALTER option - the name of the parameter + AlterSystemParameter string `json:"alterSystemParameter"` + // to be used with ALTER option - the value of the parameter + AlterSystemValue string `json:"alterSystemValue"` + // parameter scope + ParameterScope string `json:"parameterScope,omitempty"` + // The target state of the LRPDB + // +kubebuilder:validation:Enum=OPEN;CLOSE;ALTER + LRPDBState string `json:"pdbState,omitempty"` + // turn on the assertive approach to delete pdb resource + // kubectl delete pdb ..... automatically triggers the pluggable database + // deletion + AssertiveLrpdbDeletion bool `json:"assertiveLrpdbDeletion,omitempty"` + PDBConfigMap string `json:"pdbconfigmap,omitempty"` +} + +// LRPDBAdminName defines the secret containing Sys Admin User mapped to key 'adminName' for LRPDB +type LRPDBAdminName struct { + Secret LRPDBSecret `json:"secret"` +} + +// LRPDBAdminPassword defines the secret containing Sys Admin Password mapped to key 'adminPwd' for LRPDB +type LRPDBAdminPassword struct { + Secret LRPDBSecret `json:"secret"` +} + +// TDEPwd defines the secret containing TDE Wallet Password mapped to key 'tdePassword' for LRPDB +type LTDEPwd struct { + Secret LRPDBSecret `json:"secret"` +} + +// TDESecret defines the secret containing TDE Secret to key 'tdeSecret' for LRPDB +type LTDESecret struct { + Secret LRPDBSecret `json:"secret"` +} + +type WebLrpdbServerUser struct { + Secret LRPDBSecret `json:"secret"` +} + +type WebLrpdbServerPassword struct { + Secret LRPDBSecret `json:"secret"` +} + +type AdminpdbUser struct { + Secret LRPDBSecret `json:"secret"` +} + +type AdminpdbPass struct { + Secret LRPDBSecret `json:"secret"` +} + +// LRPDBSecret defines the secretName +type LRPDBSecret struct { + SecretName string `json:"secretName"` + Key string `json:"key"` +} + +type LRPDBTLSKEY struct { + Secret LRPDBSecret `json:"secret"` +} + +type LRPDBTLSCRT struct { + Secret LRPDBSecret `json:"secret"` +} + +type LRPDBTLSCAT struct { + Secret LRPDBSecret `json:"secret"` +} + +type LRPDBPRVKEY struct { + Secret LRPDBSecret `json:"secret"` +} + +// LRPDBStatus defines the observed state of LRPDB +type LRPDBStatus struct { + // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster + // Important: Run "make" to regenerate code after modifying this file + + // LRPDB Connect String + ConnString string `json:"connString,omitempty"` + // Phase of the LRPDB Resource + Phase string `json:"phase"` + // LRPDB Resource Status + Status bool `json:"status"` + // Total size of the LRPDB + TotalSize string `json:"totalSize,omitempty"` + // Open mode of the LRPDB + OpenMode string `json:"openMode,omitempty"` + // Modify Option of the LRPDB + ModifyOption string `json:"modifyOption,omitempty"` + // Message + Msg string `json:"msg,omitempty"` + // Last Completed Action + Action string `json:"action,omitempty"` + // Last Completed alter system + AlterSystem string `json:"alterSystem,omitempty"` + // Last ORA- + SqlCode int `json:"sqlCode"` + Bitstat int `json:"bitstat,omitempty"` /* Bitmask */ + BitStatStr string `json:"bitstatstr,omitempty"` /* Decoded bitmask */ +} + +// +kubebuilder:object:root=true +// +kubebuilder:subresource:status +// +kubebuilder:printcolumn:JSONPath=".spec.cdbName",name="CDB Name",type="string",description="Name of the CDB" +// +kubebuilder:printcolumn:JSONPath=".spec.pdbName",name="PDB Name",type="string",description="Name of the PDB" +// +kubebuilder:printcolumn:JSONPath=".status.openMode",name="PDB State",type="string",description="PDB Open Mode" +// +kubebuilder:printcolumn:JSONPath=".status.totalSize",name="PDB Size",type="string",description="Total Size of the PDB" +// +kubebuilder:printcolumn:JSONPath=".status.phase",name="Status",type="string",description="Status of the LRPDB Resource" +// +kubebuilder:printcolumn:JSONPath=".status.msg",name="Message",type="string",description="Error message, if any" +// +kubebuilder:printcolumn:JSONPath=".status.sqlCode",name="last sqlcode",type="integer",description="last sqlcode" +// +kubebuilder:printcolumn:JSONPath=".status.connString",name="Connect_String",type="string",description="The connect string to be used" +// +kubebuilder:resource:path=lrpdbs,scope=Namespaced +// +kubebuilder:storageversion + +// LRPDB is the Schema for the pdbs API +type LRPDB struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + Spec LRPDBSpec `json:"spec,omitempty"` + Status LRPDBStatus `json:"status,omitempty"` +} + +//+kubebuilder:object:root=true + +// LRPDBList contains a list of LRPDB +type LRPDBList struct { + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata,omitempty"` + Items []LRPDB `json:"items"` +} + +func init() { + SchemeBuilder.Register(&LRPDB{}, &LRPDBList{}) +} diff --git a/apis/database/v4/lrpdb_webhook.go b/apis/database/v4/lrpdb_webhook.go new file mode 100644 index 00000000..d6807926 --- /dev/null +++ b/apis/database/v4/lrpdb_webhook.go @@ -0,0 +1,370 @@ +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +/* MODIFIED (MM/DD/YY) +** rcitton 07/14/22 - 33822886 + */ + +package v4 + +import ( + "context" + "fmt" + "reflect" + "strconv" + "strings" + + apierrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/runtime/schema" + "k8s.io/apimachinery/pkg/util/validation/field" + ctrl "sigs.k8s.io/controller-runtime" + logf "sigs.k8s.io/controller-runtime/pkg/log" + "sigs.k8s.io/controller-runtime/pkg/webhook" + "sigs.k8s.io/controller-runtime/pkg/webhook/admission" +) + +// log is for logging in this package. +var lrpdblog = logf.Log.WithName("lrpdb-webhook") + +func (r *LRPDB) SetupWebhookWithManager(mgr ctrl.Manager) error { + return ctrl.NewWebhookManagedBy(mgr). + WithValidator(&LRPDB{}). + WithDefaulter(&LRPDB{}). + For(r). + Complete() +} + +//+kubebuilder:webhook:path=/mutate-database-oracle-com-v4-lrpdb,mutating=true,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=lrpdbs,verbs=create;update,versions=v4,name=mlrpdb.kb.io,admissionReviewVersions={v4,v1beta1} + +var _ webhook.CustomDefaulter = &LRPDB{} + +// Default implements webhook.Defaulter so a webhook will be registered for the type +func (r *LRPDB) Default(ctx context.Context, obj runtime.Object) error { + pdb, ok := obj.(*LRPDB) + if !ok { + return fmt.Errorf("expected an LRPDB object but got %T", obj) + } + lrpdblog.Info("Setting default values in LRPDB spec for : " + pdb.Name) + + action := strings.ToUpper(pdb.Spec.Action) + + if action == "DELETE" { + if pdb.Spec.DropAction == "" { + pdb.Spec.DropAction = "KEEP" + lrpdblog.Info(" - dropAction : KEEP") + } + } else if action != "MODIFY" && action != "STATUS" { + if pdb.Spec.ReuseTempFile == nil { + pdb.Spec.ReuseTempFile = new(bool) + *pdb.Spec.ReuseTempFile = true + lrpdblog.Info(" - reuseTempFile : " + strconv.FormatBool(*(pdb.Spec.ReuseTempFile))) + } + if pdb.Spec.UnlimitedStorage == nil { + pdb.Spec.UnlimitedStorage = new(bool) + *pdb.Spec.UnlimitedStorage = true + lrpdblog.Info(" - unlimitedStorage : " + strconv.FormatBool(*(pdb.Spec.UnlimitedStorage))) + } + if pdb.Spec.LTDEImport == nil { + pdb.Spec.LTDEImport = new(bool) + *pdb.Spec.LTDEImport = false + lrpdblog.Info(" - tdeImport : " + strconv.FormatBool(*(pdb.Spec.LTDEImport))) + } + if pdb.Spec.LTDEExport == nil { + pdb.Spec.LTDEExport = new(bool) + *pdb.Spec.LTDEExport = false + lrpdblog.Info(" - tdeExport : " + strconv.FormatBool(*(pdb.Spec.LTDEExport))) + } + if pdb.Spec.AsClone == nil { + pdb.Spec.AsClone = new(bool) + *pdb.Spec.AsClone = false + lrpdblog.Info(" - asClone : " + strconv.FormatBool(*(pdb.Spec.AsClone))) + } + } + + if pdb.Spec.GetScript == nil { + pdb.Spec.GetScript = new(bool) + *pdb.Spec.GetScript = false + lrpdblog.Info(" - getScript : " + strconv.FormatBool(*(pdb.Spec.GetScript))) + } + return nil +} + +// TODO(user): change verbs to "verbs=create;update;delete" if you want to enable deletion validation. +//+kubebuilder:webhook:path=/validate-database-oracle-com-v4-lrpdb,mutating=false,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=lrpdbs,verbs=create;update,versions=v4,name=vlrpdb.kb.io,admissionReviewVersions={v4,v1beta1} + +var _ webhook.CustomValidator = &LRPDB{} + +// ValidateCreate implements webhook.Validator so a webhook will be registered for the type +func (r *LRPDB) ValidateCreate(ctx context.Context, obj runtime.Object) (admission.Warnings, error) { + lrpdblog.Info("ValidateCreate-Validating LRPDB spec for : " + r.Name) + pdb := obj.(*LRPDB) + + var allErrs field.ErrorList + + r.validateCommon(&allErrs, ctx, *pdb) + + r.validateAction(&allErrs, ctx, *pdb) + + action := strings.ToUpper(pdb.Spec.Action) + + if len(allErrs) == 0 { + lrpdblog.Info("LRPDB Resource : " + r.Name + " successfully validated for Action : " + action) + return nil, nil + } + return nil, apierrors.NewInvalid( + schema.GroupKind{Group: "database.oracle.com", Kind: "LRPDB"}, + r.Name, allErrs) + return nil, nil +} + +// Validate Action for required parameters +func (r *LRPDB) validateAction(allErrs *field.ErrorList, ctx context.Context, pdb LRPDB) { + action := strings.ToUpper(pdb.Spec.Action) + + lrpdblog.Info("Valdiating LRPDB Resource Action : " + action) + + if reflect.ValueOf(pdb.Spec.LRPDBTlsKey).IsZero() { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("lrpdbTlsKey"), "Please specify LRPDB Tls Key(secret)")) + } + + if reflect.ValueOf(pdb.Spec.LRPDBTlsCrt).IsZero() { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("lrpdbTlsCrt"), "Please specify LRPDB Tls Certificate(secret)")) + } + + if reflect.ValueOf(pdb.Spec.LRPDBTlsCat).IsZero() { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("lrpdbTlsCat"), "Please specify LRPDB Tls Certificate Authority(secret)")) + } + + switch action { + case "DELETE": + /* BUG 36752336 - LREST OPERATOR - DELETE NON-EXISTENT PDB SHOWS LRPDB CREATED MESSAGE */ + if pdb.Status.OpenMode == "READ WRITE" { + lrpdblog.Info("Cannot delete: pdb is open ") + *allErrs = append(*allErrs, field.Invalid(field.NewPath("status").Child("OpenMode"), "READ WRITE", "pdb "+pdb.Spec.LRPDBName+" "+pdb.Status.OpenMode)) + } + r.CheckObjExistence("DELETE", allErrs, ctx, pdb) + case "CREATE": + if reflect.ValueOf(pdb.Spec.AdminpdbUser).IsZero() { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("adminpdbUser"), "Please specify LRPDB System Administrator user")) + } + if reflect.ValueOf(pdb.Spec.AdminpdbPass).IsZero() { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("adminpdbPass"), "Please specify LRPDB System Administrator Password")) + } + if pdb.Spec.FileNameConversions == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("fileNameConversions"), "Please specify a value for fileNameConversions. Values can be a filename convert pattern or NONE")) + } + if pdb.Spec.TotalSize == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("totalSize"), "When the storage is not UNLIMITED the Total Size must be specified")) + } + if pdb.Spec.TempSize == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("tempSize"), "When the storage is not UNLIMITED the Temp Size must be specified")) + } + if *(pdb.Spec.LTDEImport) { + r.validateTDEInfo(allErrs, ctx, pdb) + } + case "CLONE": + // Sample Err: The LRPDB "lrpdb1-clone" is invalid: spec.srcPdbName: Required value: Please specify source LRPDB for Cloning + if pdb.Spec.SrcLRPDBName == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("srcPdbName"), "Please specify source LRPDB name for Cloning")) + } + if pdb.Spec.TotalSize == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("totalSize"), "When the storage is not UNLIMITED the Total Size must be specified")) + } + if pdb.Spec.TempSize == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("tempSize"), "When the storage is not UNLIMITED the Temp Size must be specified")) + } + if pdb.Status.OpenMode == "MOUNT" { + lrpdblog.Info("Cannot clone: pdb is mount ") + *allErrs = append(*allErrs, field.Invalid(field.NewPath("status").Child("OpenMode"), "READ WRITE", "pdb "+pdb.Spec.LRPDBName+" "+pdb.Status.OpenMode)) + } + case "PLUG": + if pdb.Spec.XMLFileName == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("xmlFileName"), "Please specify XML metadata filename")) + } + if pdb.Spec.FileNameConversions == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("fileNameConversions"), "Please specify a value for fileNameConversions. Values can be a filename convert pattern or NONE")) + } + if pdb.Spec.SourceFileNameConversions == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("sourceFileNameConversions"), "Please specify a value for sourceFileNameConversions. Values can be a filename convert pattern or NONE")) + } + if pdb.Spec.CopyAction == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("copyAction"), "Please specify a value for copyAction. Values can be COPY, NOCOPY or MOVE")) + } + if *(pdb.Spec.LTDEImport) { + r.validateTDEInfo(allErrs, ctx, pdb) + } + case "UNPLUG": + if pdb.Spec.XMLFileName == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("xmlFileName"), "Please specify XML metadata filename")) + } + if *(pdb.Spec.LTDEExport) { + r.validateTDEInfo(allErrs, ctx, pdb) + } + if pdb.Status.OpenMode == "READ WRITE" { + lrpdblog.Info("Cannot unplug: pdb is open ") + *allErrs = append(*allErrs, field.Invalid(field.NewPath("status").Child("OpenMode"), "READ WRITE", "pdb "+pdb.Spec.LRPDBName+" "+pdb.Status.OpenMode)) + } + r.CheckObjExistence("UNPLUG", allErrs, ctx, pdb) + case "MODIFY": + + if pdb.Spec.LRPDBState == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("lrpdbState"), "Please specify target state of LRPDB")) + } + if pdb.Spec.ModifyOption == "" && pdb.Spec.AlterSystem == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("modifyOption"), "Please specify an option for opening/closing a LRPDB or alter system parameter")) + } + r.CheckObjExistence("MODIFY", allErrs, ctx, pdb) + } +} + +func (r *LRPDB) CheckObjExistence(action string, allErrs *field.ErrorList, ctx context.Context, pdb LRPDB) { + /* BUG 36752465 - lrest operator - open non-existent pdb creates a lrpdb with status failed */ + lrpdblog.Info("Action [" + action + "] checkin " + pdb.Spec.LRPDBName + " existence") + if pdb.Status.OpenMode == "" { + *allErrs = append(*allErrs, field.NotFound(field.NewPath("Spec").Child("LRPDBName"), " "+pdb.Spec.LRPDBName+" does not exist : action "+action+" failure")) + + } +} + +// ValidateUpdate implements webhook.Validator so a webhook will be registered for the type +func (r *LRPDB) ValidateUpdate(ctx context.Context, obj runtime.Object, old runtime.Object) (admission.Warnings, error) { + lrpdblog.Info("ValidateUpdate-Validating LRPDB spec for : " + r.Name) + pdb := old.(*LRPDB) + + isLRPDBMarkedToBeDeleted := r.GetDeletionTimestamp() != nil + if isLRPDBMarkedToBeDeleted { + return nil, nil + } + + var allErrs field.ErrorList + action := strings.ToUpper(pdb.Spec.Action) + + // If LRPDB CR has been created and in Ready state, only allow updates if the "action" value has changed as well + if (pdb.Status.Phase == "Ready") && (pdb.Status.Action != "MODIFY") && (pdb.Status.Action != "STATUS") && (pdb.Status.Action != "NOACTION") && (pdb.Status.Action == action) { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("action"), "New action also needs to be specified after LRPDB is in Ready state")) + } else { + + // Check Common Validations + r.validateCommon(&allErrs, ctx, *pdb) + + // Validate required parameters for Action specified + r.validateAction(&allErrs, ctx, *pdb) + + // Check TDE requirements + if (action != "DELETE") && (action != "MODIFY") && (action != "STATUS") && (*(pdb.Spec.LTDEImport) || *(pdb.Spec.LTDEExport)) { + r.validateTDEInfo(&allErrs, ctx, *pdb) + } + } + + if len(allErrs) == 0 { + return nil, nil + } + return nil, apierrors.NewInvalid( + schema.GroupKind{Group: "database.oracle.com", Kind: "LRPDB"}, + r.Name, allErrs) + return nil, nil +} + +// ValidateDelete implements webhook.Validator so a webhook will be registered for the type +func (r *LRPDB) ValidateDelete(ctx context.Context, obj runtime.Object) (admission.Warnings, error) { + lrpdblog.Info("ValidateDelete-Validating LRPDB spec for : " + r.Name) + + // TODO(user): fill in your validation logic upon object deletion. + return nil, nil +} + +// Validate common specs needed for all LRPDB Actions +func (r *LRPDB) validateCommon(allErrs *field.ErrorList, ctx context.Context, pdb LRPDB) { + lrpdblog.Info("validateCommon", "name", pdb.Name) + + if pdb.Spec.Action == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("action"), "Please specify LRPDB operation to be performed")) + } + if pdb.Spec.CDBResName == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("cdbResName"), "Please specify the name of the CDB Kubernetes resource to use for LRPDB operations")) + } + if pdb.Spec.CDBNamespace == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("cdbNamespace"), "Please specify the namespace of the rest server to use for LRPDB operations")) + } + if pdb.Spec.LRPDBName == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("lrpdbName"), "Please specify name of the LRPDB to be created")) + } +} + +// Validate TDE information for Create, Plug and Unplug Actions +func (r *LRPDB) validateTDEInfo(allErrs *field.ErrorList, ctx context.Context, pdb LRPDB) { + lrpdblog.Info("validateTDEInfo", "name", r.Name) + + if reflect.ValueOf(pdb.Spec.LTDEPassword).IsZero() { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("tdePassword"), "Please specify a value for tdePassword.")) + } + if pdb.Spec.LTDEKeystorePath == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("tdeKeystorePath"), "Please specify a value for tdeKeystorePath.")) + } + if reflect.ValueOf(pdb.Spec.LTDESecret).IsZero() { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("tdeSecret"), "Please specify a value for tdeSecret.")) + } + +} diff --git a/apis/database/v4/oraclerestdataservice_conversion.go b/apis/database/v4/oraclerestdataservice_conversion.go new file mode 100644 index 00000000..a19cdfd5 --- /dev/null +++ b/apis/database/v4/oraclerestdataservice_conversion.go @@ -0,0 +1,4 @@ +package v4 + +// Hub defines v1 as the hub version +func (*OracleRestDataService) Hub() {} diff --git a/apis/database/v4/oraclerestdataservice_types.go b/apis/database/v4/oraclerestdataservice_types.go new file mode 100644 index 00000000..20cc7a74 --- /dev/null +++ b/apis/database/v4/oraclerestdataservice_types.go @@ -0,0 +1,158 @@ +/* +** Copyright (c) 2023 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v4 + +import ( + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! +// NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized. + +// OracleRestDataServiceSpec defines the desired state of OracleRestDataService +type OracleRestDataServiceSpec struct { + // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster + // Important: Run "make" to regenerate code after modifying this file + + DatabaseRef string `json:"databaseRef"` + LoadBalancer bool `json:"loadBalancer,omitempty"` + ServiceAnnotations map[string]string `json:"serviceAnnotations,omitempty"` + NodeSelector map[string]string `json:"nodeSelector,omitempty"` + Image OracleRestDataServiceImage `json:"image,omitempty"` + OrdsPassword OracleRestDataServicePassword `json:"ordsPassword"` + AdminPassword OracleRestDataServicePassword `json:"adminPassword"` + OrdsUser string `json:"ordsUser,omitempty"` + RestEnableSchemas []OracleRestDataServiceRestEnableSchemas `json:"restEnableSchemas,omitempty"` + OracleService string `json:"oracleService,omitempty"` + ServiceAccountName string `json:"serviceAccountName,omitempty"` + Persistence OracleRestDataServicePersistence `json:"persistence,omitempty"` + MongoDbApi bool `json:"mongoDbApi,omitempty"` + + // +k8s:openapi-gen=true + // +kubebuilder:validation:Minimum=1 + Replicas int `json:"replicas,omitempty"` + ReadinessCheckPeriod int `json:"readinessCheckPeriod,omitempty"` +} + +// OracleRestDataServicePersistence defines the storage releated params +type OracleRestDataServicePersistence struct { + Size string `json:"size,omitempty"` + StorageClass string `json:"storageClass,omitempty"` + + // +kubebuilder:validation:Enum=ReadWriteOnce;ReadWriteMany + AccessMode string `json:"accessMode,omitempty"` + VolumeName string `json:"volumeName,omitempty"` + SetWritePermissions *bool `json:"setWritePermissions,omitempty"` +} + +// OracleRestDataServiceImage defines the Image source and pullSecrets for POD +type OracleRestDataServiceImage struct { + Version string `json:"version,omitempty"` + PullFrom string `json:"pullFrom"` + PullSecrets string `json:"pullSecrets,omitempty"` +} + +// OracleRestDataServicePassword defines the secret containing Password mapped to secretKey +type OracleRestDataServicePassword struct { + SecretName string `json:"secretName"` + // +kubebuilder:default:="oracle_pwd" + SecretKey string `json:"secretKey,omitempty"` + KeepSecret *bool `json:"keepSecret,omitempty"` +} + +// OracleRestDataServicePDBSchemas defines the PDB Schemas to be ORDS Enabled +type OracleRestDataServiceRestEnableSchemas struct { + PdbName string `json:"pdbName,omitempty"` + SchemaName string `json:"schemaName"` + UrlMapping string `json:"urlMapping,omitempty"` + Enable bool `json:"enable"` +} + +// OracleRestDataServiceStatus defines the observed state of OracleRestDataService +type OracleRestDataServiceStatus struct { + // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster + // Important: Run "make" to regenerate code after modifying this file + Status string `json:"status,omitempty"` + DatabaseApiUrl string `json:"databaseApiUrl,omitempty"` + LoadBalancer string `json:"loadBalancer,omitempty"` + DatabaseRef string `json:"databaseRef,omitempty"` + ServiceIP string `json:"serviceIP,omitempty"` + DatabaseActionsUrl string `json:"databaseActionsUrl,omitempty"` + MongoDbApiAccessUrl string `json:"mongoDbApiAccessUrl,omitempty"` + OrdsInstalled bool `json:"ordsInstalled,omitempty"` + ApexConfigured bool `json:"apexConfigured,omitempty"` + ApxeUrl string `json:"apexUrl,omitempty"` + MongoDbApi bool `json:"mongoDbApi,omitempty"` + CommonUsersCreated bool `json:"commonUsersCreated,omitempty"` + Replicas int `json:"replicas,omitempty"` + + Image OracleRestDataServiceImage `json:"image,omitempty"` +} + +//+kubebuilder:object:root=true +//+kubebuilder:subresource:status +// +kubebuilder:printcolumn:JSONPath=".status.status",name="Status",type="string" +// +kubebuilder:printcolumn:JSONPath=".spec.databaseRef",name="Database",type="string" +// +kubebuilder:printcolumn:JSONPath=".status.databaseApiUrl",name="Database API URL",type="string" +// +kubebuilder:printcolumn:JSONPath=".status.databaseActionsUrl",name="Database Actions URL",type="string" +// +kubebuilder:printcolumn:JSONPath=".status.apexUrl",name="Apex URL",type="string" +// +kubebuilder:printcolumn:JSONPath=".status.mongoDbApiAccessUrl",name="MongoDbApi Access URL",type="string" + +// OracleRestDataService is the Schema for the oraclerestdataservices API +// +kubebuilder:storageversion +type OracleRestDataService struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + Spec OracleRestDataServiceSpec `json:"spec,omitempty"` + Status OracleRestDataServiceStatus `json:"status,omitempty"` +} + +//+kubebuilder:object:root=true + +// OracleRestDataServiceList contains a list of OracleRestDataService +type OracleRestDataServiceList struct { + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata,omitempty"` + Items []OracleRestDataService `json:"items"` +} + +func init() { + SchemeBuilder.Register(&OracleRestDataService{}, &OracleRestDataServiceList{}) +} diff --git a/apis/database/v4/oraclerestdataservice_webhook.go b/apis/database/v4/oraclerestdataservice_webhook.go new file mode 100644 index 00000000..5211528a --- /dev/null +++ b/apis/database/v4/oraclerestdataservice_webhook.go @@ -0,0 +1,55 @@ +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v4 + +import ( + ctrl "sigs.k8s.io/controller-runtime" + logf "sigs.k8s.io/controller-runtime/pkg/log" +) + +// log is for logging in this package. +var oraclerestdataservicelog = logf.Log.WithName("oraclerestdataservice-resource") + +func (r *OracleRestDataService) SetupWebhookWithManager(mgr ctrl.Manager) error { + return ctrl.NewWebhookManagedBy(mgr). + For(r). + Complete() +} + +// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! diff --git a/apis/database/v4/ordssrvs_types.go b/apis/database/v4/ordssrvs_types.go new file mode 100644 index 00000000..1fbf820a --- /dev/null +++ b/apis/database/v4/ordssrvs_types.go @@ -0,0 +1,693 @@ +/* +** Copyright (c) 2024 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v4 + +import ( + "time" + + corev1 "k8s.io/api/core/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +// OrdsSrvsSpec defines the desired state of OrdsSrvs +// +kubebuilder:resource:shortName="ords" +type OrdsSrvsSpec struct { + // Specifies the desired Kubernetes Workload + //+kubebuilder:validation:Enum=Deployment;StatefulSet;DaemonSet + //+kubebuilder:default=Deployment + WorkloadType string `json:"workloadType,omitempty"` + // Defines the number of desired Replicas when workloadType is Deployment or StatefulSet + //+kubebuilder:validation:Minimum=1 + //+kubebuilder:default=1 + Replicas int32 `json:"replicas,omitempty"` + // Specifies whether to restart pods when Global or Pool configurations change + ForceRestart bool `json:"forceRestart,omitempty"` + // Specifies the ORDS container image + //+kubecbuilder:default=container-registry.oracle.com/database/ords:latest + Image string `json:"image"` + // Specifies the ORDS container image pull policy + //+kubebuilder:validation:Enum=IfNotPresent;Always;Never + //+kubebuilder:default=IfNotPresent + ImagePullPolicy corev1.PullPolicy `json:"imagePullPolicy,omitempty"` + // Specifies the Secret Name for pulling the ORDS container image + ImagePullSecrets string `json:"imagePullSecrets,omitempty"` + // Contains settings that are configured across the entire ORDS instance. + GlobalSettings GlobalSettings `json:"globalSettings"` + // Contains settings for individual pools/databases + // Private key + EncPrivKey PasswordSecret `json:"encPrivKey,omitempty"` + PoolSettings []*PoolSettings `json:"poolSettings,omitempty"` + // +k8s:openapi-gen=true + +} + +type GlobalSettings struct { + // Specifies the setting to enable or disable metadata caching. + CacheMetadataEnabled *bool `json:"cache.metadata.enabled,omitempty"` + + // Specifies the duration after a GraphQL schema is not accessed from the cache that it expires. + CacheMetadataGraphQLExpireAfterAccess *time.Duration `json:"cache.metadata.graphql.expireAfterAccess,omitempty"` + + // Specifies the duration after a GraphQL schema is cached that it expires and has to be loaded again. + CacheMetadataGraphQLExpireAfterWrite *time.Duration `json:"cache.metadata.graphql.expireAfterWrite,omitempty"` + + // Specifies the setting to determine for how long a metadata record remains in the cache. + // Longer duration means, it takes longer to view the applied changes. + // The formats accepted are based on the ISO-8601 duration format. + CacheMetadataTimeout *time.Duration `json:"cache.metadata.timeout,omitempty"` + + // Specifies the setting to enable or disable JWKS caching. + CacheMetadataJWKSEnabled *bool `json:"cache.metadata.jwks.enabled,omitempty"` + + // Specifies the initial capacity of the JWKS cache. + CacheMetadataJWKSInitialCapacity *int32 `json:"cache.metadata.jwks.initialCapacity,omitempty"` + + // Specifies the maximum capacity of the JWKS cache. + CacheMetadataJWKSMaximumSize *int32 `json:"cache.metadata.jwks.maximumSize,omitempty"` + + // Specifies the duration after a JWK is not accessed from the cache that it expires. + // By default this is disabled. + CacheMetadataJWKSExpireAfterAccess *time.Duration `json:"cache.metadata.jwks.expireAfterAccess,omitempty"` + + // Specifies the duration after a JWK is cached, that is, it expires and has to be loaded again. + CacheMetadataJWKSExpireAfterWrite *time.Duration `json:"cache.metadata.jwks.expireAfterWrite,omitempty"` + + // Specifies whether the Database API is enabled. + DatabaseAPIEnabled *bool `json:"database.api.enabled,omitempty"` + + // Specifies to disable the Database API administration related services. + // Only applicable when Database API is enabled. + DatabaseAPIManagementServicesDisabled *bool `json:"database.api.management.services.disabled,omitempty"` + + // Specifies how long to wait before retrying an invalid pool. + DBInvalidPoolTimeout *time.Duration `json:"db.invalidPoolTimeout,omitempty"` + + // Specifies the maximum join nesting depth limit for GraphQL queries. + FeatureGraphQLMaxNestingDepth *int32 `json:"feature.grahpql.max.nesting.depth,omitempty"` + + // Specifies the name of the HTTP request header that uniquely identifies the request end to end as + // it passes through the various layers of the application stack. + // In Oracle this header is commonly referred to as the ECID (Entity Context ID). + RequestTraceHeaderName string `json:"request.traceHeaderName,omitempty"` + + // Specifies the maximum number of unsuccessful password attempts allowed. + // Enabled by setting a positive integer value. + SecurityCredentialsAttempts *int32 `json:"security.credentials.attempts,omitempty"` + + // Specifies the period to lock the account that has exceeded maximum attempts. + SecurityCredentialsLockTime *time.Duration `json:"security.credentials.lock.time,omitempty"` + + // Specifies the HTTP listen port. + //+kubebuilder:default:=8080 + StandaloneHTTPPort *int32 `json:"standalone.http.port,omitempty"` + + // Specifies the SSL certificate hostname. + StandaloneHTTPSHost string `json:"standalone.https.host,omitempty"` + + // Specifies the HTTPS listen port. + //+kubebuilder:default:=8443 + StandaloneHTTPSPort *int32 `json:"standalone.https.port,omitempty"` + + // Specifies the period for Standalone Mode to wait until it is gracefully shutdown. + StandaloneStopTimeout *time.Duration `json:"standalone.stop.timeout,omitempty"` + + // Specifies whether to display error messages on the browser. + DebugPrintDebugToScreen *bool `json:"debug.printDebugToScreen,omitempty"` + + // Specifies how the HTTP error responses must be formatted. + // html - Force all responses to be in HTML format + // json - Force all responses to be in JSON format + // auto - Automatically determines most appropriate format for the request (default). + ErrorResponseFormat string `json:"error.responseFormat,omitempty"` + + // Specifies the Internet Content Adaptation Protocol (ICAP) port to virus scan files. + // Either icap.port or icap.secure.port are required to have a value. + ICAPPort *int32 `json:"icap.port,omitempty"` + + // Specifies the Internet Content Adaptation Protocol (ICAP) port to virus scan files. + // Either icap.port or icap.secure.port are required to have a value. + // If values for both icap.port and icap.secure.port are provided, then the value of icap.port is ignored. + ICAPSecurePort *int32 `json:"icap.secure.port,omitempty"` + + // Specifies the Internet Content Adaptation Protocol (ICAP) server name or IP address to virus scan files. + // The icap.server is required to have a value. + ICAPServer string `json:"icap.server,omitempty"` + + // Specifies whether procedures are to be logged. + LogProcedure bool `json:"log.procedure,omitempty"` + + // Specifies to enable the API for MongoDB. + //+kubebuider:default=false + MongoEnabled bool `json:"mongo.enabled,omitempty"` + + // Specifies the API for MongoDB listen port. + //+kubebuilder:default:=27017 + MongoPort *int32 `json:"mongo.port,omitempty"` + + // Specifies the maximum idle time for a Mongo connection in milliseconds. + MongoIdleTimeout *time.Duration `json:"mongo.idle.timeout,omitempty"` + + // Specifies the maximum time for a Mongo database operation in milliseconds. + MongoOpTimeout *time.Duration `json:"mongo.op.timeout,omitempty"` + + // If this value is set to true, then the Oracle REST Data Services internal exclusion list is not enforced. + // Oracle recommends that you do not set this value to true. + SecurityDisableDefaultExclusionList *bool `json:"security.disableDefaultExclusionList,omitempty"` + + // Specifies a pattern for procedures, packages, or schema names which are forbidden to be directly executed from a browser. + SecurityExclusionList string `json:"security.exclusionList,omitempty"` + + // Specifies a pattern for procedures, packages, or schema names which are allowed to be directly executed from a browser. + SecurityInclusionList string `json:"security.inclusionList,omitempty"` + + // Specifies the maximum number of cached procedure validations. + // Set this value to 0 to force the validation procedure to be invoked on each request. + SecurityMaxEntries *int32 `json:"security.maxEntries,omitempty"` + + // Specifies whether HTTPS is available in your environment. + SecurityVerifySSL *bool `json:"security.verifySSL,omitempty"` + + // Specifies the context path where ords is located. + //+kubebuilder:default:="/ords" + StandaloneContextPath string `json:"standalone.context.path,omitempty"` + + /************************************************* + * Undocumented + /************************************************/ + + // Specifies that the HTTP Header contains the specified text + // Usually set to 'X-Forwarded-Proto: https' coming from a load-balancer + SecurityHTTPSHeaderCheck string `json:"security.httpsHeaderCheck,omitempty"` + + // Specifies to force HTTPS; this is set to default to false as in real-world TLS should + // terminiate at the LoadBalancer + SecurityForceHTTPS bool `json:"security.forceHTTPS,omitempty"` + + // Specifies to trust Access from originating domains + SecuirtyExternalSessionTrustedOrigins string `json:"security.externalSessionTrustedOrigins,omitempty"` + + /************************************************* + * Customised + /************************************************/ + /* Below are settings with physical path/file locations to be replaced by ConfigMaps/Secrets, Boolean or HardCoded */ + + /* + // Specifies the path to the folder to store HTTP request access logs. + // If not specified, then no access log is generated. + // HARDCODED + // StandaloneAccessLog string `json:"standalone.access.log,omitempty"` + */ + + // Specifies if HTTP request access logs should be enabled + // If enabled, logs will be written to /opt/oracle/sa/log/global + //+kubebuilder:default:=false + EnableStandaloneAccessLog bool `json:"enable.standalone.access.log,omitempty"` + + // Specifies if HTTP request access logs should be enabled + // If enabled, logs will be written to /opt/oracle/sa/log/global + //+kubebuilder:default:=false + EnableMongoAccessLog bool `json:"enable.mongo.access.log,omitempty"` + + /* + //Specifies the SSL certificate path. + // If you are providing the SSL certificate, then you must specify the certificate location. + // Replaced with: CertSecret *CertificateSecret `json:"certSecret,omitempty"` + //StandaloneHTTPSCert string `json:"standalone.https.cert"` + + // Specifies the SSL certificate key path. + // If you are providing the SSL certificate, you must specify the certificate key location. + // Replaced with: CertSecret *CertificateSecret `json:"certSecret,omitempty"` + //StandaloneHTTPSCertKey string `json:"standalone.https.cert.key"` + */ + + // Specifies the Secret containing the SSL Certificates + // Replaces: standalone.https.cert and standalone.https.cert.key + CertSecret *CertificateSecret `json:"certSecret,omitempty"` + + /************************************************* + * Disabled + /************************************************* + // Specifies the comma separated list of host names or IP addresses to identify a specific network + // interface on which to listen. + //+kubebuilder:default:="0.0.0.0" + //StandaloneBinds string `json:"standalone.binds,omitempty"` + // This is disabled as containerised + + // Specifies the file where credentials are stored. + //SecurityCredentialsFile string `json:"security.credentials.file,omitempty"` + // WTF does this do?!?! + + // Points to the location where static resources to be served under the / root server path are located. + // StandaloneDocRoot string `json:"standalone.doc.root,omitempty"` + // Maybe this gets implemented; difficult to predict valid use case + + // Specifies the path to a folder that contains the custom error page. + // ErrorExternalPath string `json:"error.externalPath,omitempty"` + // Can see use-case; but wait for implementation + + // Specifies the Context path where APEX static resources are located. + //+kubebuilder:default:="/i" + // StandaloneStaticContextPath string `json:"standalone.static.context.path,omitempty"` + // Does anyone ever change this? If so, need to also change the APEX install configmap to update path + */ + + // Specifies the path to the folder containing static resources required by APEX. + // StandaloneStaticPath string `json:"standalone.static.path,omitempty"` + // This is disabled as will use the container image path (/opt/oracle/apex/$ORDS_VER/images) + // HARDCODED into the entrypoint + + // Specifies a comma separated list of host names or IP addresses to identify a specific + // network interface on which to listen. + //+kubebuilder:default:="0.0.0.0" + // MongoHost string `json:"mongo.host,omitempty"` + // This is disabled as containerised + + // Specifies the path to the folder where you want to store the API for MongoDB access logs. + // MongoAccessLog string `json:"mongo.access.log,omitempty"` + // HARDCODED to global/logs +} + +type PoolSettings struct { + // Specifies the Pool Name + PoolName string `json:"poolName"` + + // Specify whether to perform ORDS installation/upgrades automatically + // The db.adminUser and db.adminUser.secret must be set, otherwise setting is ignored + // This setting will be ignored for ADB + //+kubebuilder:default:=false + AutoUpgradeORDS bool `json:"autoUpgradeORDS,omitempty"` + + // Specify whether to perform APEX installation/upgrades automatically + // The db.adminUser and db.adminUser.secret must be set, otherwise setting is ignored + // This setting will be ignored for ADB + //+kubebuilder:default:=false + AutoUpgradeAPEX bool `json:"autoUpgradeAPEX,omitempty"` + + // Specifies the name of the database user for the connection. + // For non-ADB this will default to ORDS_PUBLIC_USER + // For ADBs this must be specified and not ORDS_PUBLIC_USER + // If ORDS_PUBLIC_USER is specified for an ADB, the workload will fail + //+kubebuilder:default:="ORDS_PUBLIC_USER" + DBUsername string `json:"db.username,omitempty"` + + // Specifies the password of the specified database user. + // Replaced by: DBSecret PasswordSecret `json:"dbSecret"` + // DBPassword struct{} `json:"dbPassword,omitempty"` + + // Specifies the Secret with the dbUsername and dbPassword values + // for the connection. + DBSecret PasswordSecret `json:"db.secret"` + + // Specifies the username for the database account that ORDS uses for administration operations in the database. + DBAdminUser string `json:"db.adminUser,omitempty"` + + // Specifies the password for the database account that ORDS uses for administration operations in the database. + // Replaced by: DBAdminUserSecret PasswordSecret `json:"dbAdminUserSecret,omitempty"` + // DBAdminUserPassword struct{} `json:"db.adminUser.password,omitempty"` + + // Specifies the Secret with the dbAdminUser (SYS) and dbAdminPassword values + // for the database account that ORDS uses for administration operations in the database. + // replaces: db.adminUser.password + DBAdminUserSecret PasswordSecret `json:"db.adminUser.secret,omitempty"` + + // Specifies the username for the database account that ORDS uses for the Pluggable Database Lifecycle Management. + DBCDBAdminUser string `json:"db.cdb.adminUser,omitempty"` + + // Specifies the password for the database account that ORDS uses for the Pluggable Database Lifecycle Management. + // Replaced by: DBCdbAdminUserSecret PasswordSecret `json:"dbCdbAdminUserSecret,omitempty"` + // DBCdbAdminUserPassword struct{} `json:"db.cdb.adminUser.password,omitempty"` + + // Specifies the Secret with the dbCdbAdminUser (SYS) and dbCdbAdminPassword values + // Specifies the username for the database account that ORDS uses for the Pluggable Database Lifecycle Management. + // Replaces: db.cdb.adminUser.password + DBCDBAdminUserSecret PasswordSecret `json:"db.cdb.adminUser.secret,omitempty"` + + // Specifies the comma delimited list of additional roles to assign authenticated APEX administrator type users. + ApexSecurityAdministratorRoles string `json:"apex.security.administrator.roles,omitempty"` + + // Specifies the comma delimited list of additional roles to assign authenticated regular APEX users. + ApexSecurityUserRoles string `json:"apex.security.user.roles,omitempty"` + + // Specifies the source for database credentials when creating a direct connection for running SQL statements. + // Value can be one of pool or request. + // If the value is pool, then the credentials defined in this pool is used to create a JDBC connection. + // If the value request is used, then the credentials in the request is used to create a JDBC connection and if successful, grants the requestor SQL Developer role. + //+kubebuilder:validation:Enum=pool;request + DBCredentialsSource string `json:"db.credentialsSource,omitempty"` + + // Indicates how long to wait to gracefully destroy a pool before moving to forcefully destroy all connections including borrowed ones. + DBPoolDestroyTimeout *time.Duration `json:"db.poolDestroyTimeout,omitempty"` + + // Specifies to enable tracking of JDBC resources. + // If not released causes in resource leaks or exhaustion in the database. + // Tracking imposes a performance overhead. + DebugTrackResources *bool `json:"debug.trackResources,omitempty"` + + // Specifies to disable the Open Service Broker services available for the pool. + FeatureOpenservicebrokerExclude *bool `json:"feature.openservicebroker.exclude,omitempty"` + + // Specifies to enable the Database Actions feature. + FeatureSDW *bool `json:"feature.sdw,omitempty"` + + // Specifies a comma separated list of HTTP Cookies to exclude when initializing an Oracle Web Agent environment. + HttpCookieFilter string `json:"http.cookie.filter,omitempty"` + + // Identifies the database role that indicates that the database user must get the SQL Administrator role. + JDBCAuthAdminRole string `json:"jdbc.auth.admin.role,omitempty"` + + // Specifies how a pooled JDBC connection and corresponding database session, is released when a request has been processed. + JDBCCleanupMode string `json:"jdbc.cleanup.mode,omitempty"` + + // If it is true, then it causes a trace of the SQL statements performed by Oracle Web Agent to be echoed to the log. + OwaTraceSql *bool `json:"owa.trace.sql,omitempty"` + + // Indicates if the PL/SQL Gateway functionality should be available for a pool or not. + // Value can be one of disabled, direct, or proxied. + // If the value is direct, then the pool serves the PL/SQL Gateway requests directly. + // If the value is proxied, the PLSQL_GATEWAY_CONFIG view is used to determine the user to whom to proxy. + //+kubebuilder:validation:Enum=disabled;direct;proxied + PlsqlGatewayMode string `json:"plsql.gateway.mode,omitempty"` + + // Specifies whether the JWT Profile authentication is available. Supported values: + SecurityJWTProfileEnabled *bool `json:"security.jwt.profile.enabled,omitempty"` + + // Specifies the maximum number of bytes read from the JWK url. + SecurityJWKSSize *int32 `json:"security.jwks.size,omitempty"` + + // Specifies the maximum amount of time before timing-out when accessing a JWK url. + SecurityJWKSConnectionTimeout *time.Duration `json:"security.jwks.connection.timeout,omitempty"` + + // Specifies the maximum amount of time reading a response from the JWK url before timing-out. + SecurityJWKSReadTimeout *time.Duration `json:"security.jwks.read.timeout,omitempty"` + + // Specifies the minimum interval between refreshing the JWK cached value. + SecurityJWKSRefreshInterval *time.Duration `json:"security.jwks.refresh.interval,omitempty"` + + // Specifies the maximum skew the JWT time claims are accepted. + // This is useful if the clock on the JWT issuer and ORDS differs by a few seconds. + SecurityJWTAllowedSkew *time.Duration `json:"security.jwt.allowed.skew,omitempty"` + + // Specifies the maximum allowed age of a JWT in seconds, regardless of expired claim. + // The age of the JWT is taken from the JWT issued at claim. + SecurityJWTAllowedAge *time.Duration `json:"security.jwt.allowed.age,omitempty"` + + // Indicates the type of security.requestValidationFunction: javascript or plsql. + //+kubebuilder:validation:Enum=plsql;javascript + SecurityValidationFunctionType string `json:"security.validationFunctionType,omitempty"` + + // The type of connection. + //+kubebuilder:validation:Enum=basic;tns;customurl + DBConnectionType string `json:"db.connectionType,omitempty"` + + // Specifies the JDBC URL connection to connect to the database. + DBCustomURL string `json:"db.customURL,omitempty"` + + // Specifies the host system for the Oracle database. + DBHostname string `json:"db.hostname,omitempty"` + + // Specifies the database listener port. + DBPort *int32 `json:"db.port,omitempty"` + + // Specifies the network service name of the database. + DBServicename string `json:"db.servicename,omitempty"` + + // Specifies the name of the database. + DBSid string `json:"db.sid,omitempty"` + + // Specifies the TNS alias name that matches the name in the tnsnames.ora file. + DBTnsAliasName string `json:"db.tnsAliasName,omitempty"` + + // Specifies the service name in the wallet archive for the pool. + DBWalletZipService string `json:"db.wallet.zip.service,omitempty"` + + // Specifies the JDBC driver type. + //+kubebuilder:validation:Enum=thin;oci8 + JDBCDriverType string `json:"jdbc.DriverType,omitempty"` + + // Specifies how long an available connection can remain idle before it is closed. The inactivity connection timeout is in seconds. + JDBCInactivityTimeout *int32 `json:"jdbc.InactivityTimeout,omitempty"` + + // Specifies the initial size for the number of connections that will be created. + // The default is low, and should probably be set higher in most production environments. + JDBCInitialLimit *int32 `json:"jdbc.InitialLimit,omitempty"` + + // Specifies the maximum number of times to reuse a connection before it is discarded and replaced with a new connection. + JDBCMaxConnectionReuseCount *int32 `json:"jdbc.MaxConnectionReuseCount,omitempty"` + + // Sets the maximum connection reuse time property. + JDBCMaxConnectionReuseTime *int32 `json:"jdbc.MaxConnectionReuseTime,omitempty"` + + // Sets the time in seconds to trust an idle connection to skip a validation test. + JDBCSecondsToTrustIdleConnection *int32 `json:"jdbc.SecondsToTrustIdleConnection,omitempty"` + + // Specifies the maximum number of connections. + // Might be too low for some production environments. + JDBCMaxLimit *int32 `json:"jdbc.MaxLimit,omitempty"` + + // Specifies if the PL/SQL Gateway calls can be authenticated using database users. + // If the value is true then this feature is enabled. If the value is false, then this feature is disabled. + // Oracle recommends not to use this feature. + // This feature used only to facilitate customers migrating from mod_plsql. + JDBCAuthEnabled *bool `json:"jdbc.auth.enabled,omitempty"` + + // Specifies the maximum number of statements to cache for each connection. + JDBCMaxStatementsLimit *int32 `json:"jdbc.MaxStatementsLimit,omitempty"` + + // Specifies the minimum number of connections. + JDBCMinLimit *int32 `json:"jdbc.MinLimit,omitempty"` + + // Specifies a timeout period on a statement. + // An abnormally long running query or script, executed by a request, may leave it in a hanging state unless a timeout is + // set on the statement. Setting a timeout on the statement ensures that all the queries automatically timeout if + // they are not completed within the specified time period. + JDBCStatementTimeout *int32 `json:"jdbc.statementTimeout,omitempty"` + + // Specifies the default page to display. The Oracle REST Data Services Landing Page. + MiscDefaultPage string `json:"misc.defaultPage,omitempty"` + + // Specifies the maximum number of rows that will be returned from a query when processing a RESTful service + // and that will be returned from a nested cursor in a result set. + // Affects all RESTful services generated through a SQL query, regardless of whether the resource is paginated. + MiscPaginationMaxRows *int32 `json:"misc.pagination.maxRows,omitempty"` + + // Specifies the procedure name(s) to execute after executing the procedure specified on the URL. + // Multiple procedure names must be separated by commas. + ProcedurePostProcess string `json:"procedurePostProcess,omitempty"` + + // Specifies the procedure name(s) to execute prior to executing the procedure specified on the URL. + // Multiple procedure names must be separated by commas. + ProcedurePreProcess string `json:"procedure.preProcess,omitempty"` + + // Specifies the function to be invoked prior to dispatching each Oracle REST Data Services based REST Service. + // The function can perform configuration of the database session, perform additional validation or authorization of the request. + // If the function returns true, then processing of the request continues. + // If the function returns false, then processing of the request is aborted and an HTTP 403 Forbidden status is returned. + ProcedureRestPreHook string `json:"procedure.rest.preHook,omitempty"` + + // Specifies an authentication function to determine if the requested procedure in the URL should be allowed or disallowed for processing. + // The function should return true if the procedure is allowed; otherwise, it should return false. + // If it returns false, Oracle REST Data Services will return WWW-Authenticate in the response header. + SecurityRequestAuthenticationFunction string `json:"security.requestAuthenticationFunction,omitempty"` + + // Specifies a validation function to determine if the requested procedure in the URL should be allowed or disallowed for processing. + // The function should return true if the procedure is allowed; otherwise, return false. + //+kubebuilder:default:="ords_util.authorize_plsql_gateway" + SecurityRequestValidationFunction string `json:"security.requestValidationFunction,omitempty"` + + // When using the SODA REST API, specifies the default number of documents returned for a GET request on a collection when a + // limit is not specified in the URL. Must be a positive integer, or "unlimited" for no limit. + SODADefaultLimit string `json:"soda.defaultLimit,omitempty"` + + // When using the SODA REST API, specifies the maximum number of documents that will be returned for a GET request on a collection URL, + // regardless of any limit specified in the URL. Must be a positive integer, or "unlimited" for no limit. + SODAMaxLimit string `json:"soda.maxLimit,omitempty"` + + // Specifies whether the REST-Enabled SQL service is active. + RestEnabledSqlActive *bool `json:"restEnabledSql.active,omitempty"` + + /************************************************* + * Customised + /************************************************/ + /* Below are settings with physical path/file locations to be replaced by ConfigMaps/Secrets, Boolean or HardCoded */ + + /* + // Specifies the wallet archive (provided in BASE64 encoding) containing connection details for the pool. + // Replaced with: DBWalletSecret *DBWalletSecret `json:"dbWalletSecret,omitempty"` + DBWalletZip string `json:"db.wallet.zip,omitempty"` + + // Specifies the path to a wallet archive containing connection details for the pool. + // HARDCODED + DBWalletZipPath string `json:"db.wallet.zip.path,omitempty"` + */ + + // Specifies the Secret containing the wallet archive containing connection details for the pool. + // Replaces: db.wallet.zip + DBWalletSecret *DBWalletSecret `json:"dbWalletSecret,omitempty"` + + /* + // The directory location of your tnsnames.ora file. + // Replaced with: TNSAdminSecret *TNSAdminSecret `json:"tnsAdminSecret,omitempty"` + // DBTnsDirectory string `json:"db.tnsDirectory,omitempty"` + */ + + // Specifies the Secret containing the TNS_ADMIN directory + // Replaces: db.tnsDirectory + TNSAdminSecret *TNSAdminSecret `json:"tnsAdminSecret,omitempty"` + + /************************************************* + * Disabled + /************************************************* + // specifies a configuration setting for AutoUpgrade.jar location. + // AutoupgradeAPIAulocation string `json:"autoupgrade.api.aulocation,omitempty"` + // As of 23.4; AutoUpgrade.jar is not part of the container image + + // Specifies a configuration setting to enable AutoUpgrade REST API features. + // AutoupgradeAPIEnabled *bool `json:"autoupgrade.api.enabled,omitempty"` + // Guess this has to do with autoupgrade.api.aulocation which is not implemented + + // Specifies a configuration setting for AutoUpgrade REST API JVM location. + // AutoupgradeAPIJvmlocation string `json:"autoupgrade.api.jvmlocation,omitempty"` + // Guess this has to do with autoupgrade.api.aulocation which is not implemented + + // Specifies a configuration setting for AutoUpgrade REST API log location. + // AutoupgradeAPILoglocation string `json:"autoupgrade.api.loglocation,omitempty"` + // Guess this has to do with autoupgrade.api.aulocation which is not implemented + + // Specifies that the pool points to a CDB, and that the PDBs connected to that CDB should be made addressable + // by Oracle REST Data Services + // DBServiceNameSuffix string `json:"db.serviceNameSuffix,omitempty"` + // Not sure of use case here?!? + */ +} + +type PriVKey struct { + Secret PasswordSecret `json:"secret"` +} + +// Defines the secret containing Password mapped to secretKey +type PasswordSecret struct { + // Specifies the name of the password Secret + SecretName string `json:"secretName"` + // Specifies the key holding the value of the Secret + //+kubebuilder:default:="password" + PasswordKey string `json:"passwordKey,omitempty"` +} + +// Defines the secret containing Certificates +type CertificateSecret struct { + // Specifies the name of the certificate Secret + SecretName string `json:"secretName"` + // Specifies the Certificate + Certificate string `json:"cert"` + // Specifies the Certificate Key + CertificateKey string `json:"key"` +} + +// Defines the secret containing Certificates +type TNSAdminSecret struct { + // Specifies the name of the TNS_ADMIN Secret + SecretName string `json:"secretName"` +} + +// Defines the secret containing Certificates +type DBWalletSecret struct { + // Specifies the name of the Database Wallet Secret + SecretName string `json:"secretName"` + // Specifies the Secret key name containing the Wallet + WalletName string `json:"walletName"` +} + +// OrdsSrvsStatus defines the observed state of OrdsSrvs +type OrdsSrvsStatus struct { + //** PLACE HOLDER + OrdsInstalled bool `json:"ordsInstalled,omitempty"` + //** PLACE HOLDER + // Indicates the current status of the resource + Status string `json:"status,omitempty"` + // Indicates the current Workload type of the resource + WorkloadType string `json:"workloadType,omitempty"` + // Indicates the ORDS version + ORDSVersion string `json:"ordsVersion,omitempty"` + // Indicates the HTTP port of the resource exposed by the pods + HTTPPort *int32 `json:"httpPort,omitempty"` + // Indicates the HTTPS port of the resource exposed by the pods + HTTPSPort *int32 `json:"httpsPort,omitempty"` + // Indicates the MongoAPI port of the resource exposed by the pods (if enabled) + MongoPort int32 `json:"mongoPort,omitempty"` + // Indicates if the resource is out-of-sync with the configuration + RestartRequired bool `json:"restartRequired"` + + // +operator-sdk:csv:customresourcedefinitions:type=status + Conditions []metav1.Condition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type" protobuf:"bytes,1,rep,name=conditions"` +} + +//+kubebuilder:object:root=true +//+kubebuilder:subresource:status +//+kubebuilder:printcolumn:JSONPath=".status.status",name="status",type="string" +//+kubebuilder:printcolumn:JSONPath=".status.workloadType",name="workloadType",type="string" +//+kubebuilder:printcolumn:JSONPath=".status.ordsVersion",name="ordsVersion",type="string" +//+kubebuilder:printcolumn:JSONPath=".status.httpPort",name="httpPort",type="integer" +//+kubebuilder:printcolumn:JSONPath=".status.httpsPort",name="httpsPort",type="integer" +//+kubebuilder:printcolumn:JSONPath=".status.mongoPort",name="MongoPort",type="integer" +//+kubebuilder:printcolumn:JSONPath=".status.restartRequired",name="restartRequired",type="boolean" +//+kubebuilder:printcolumn:JSONPath=".metadata.creationTimestamp",name="AGE",type="date" +//+kubebuilder:printcolumn:JSONPath=".status.ordsInstalled",name="OrdsInstalled",type="boolean" +//+kubebuilder:resource:path=ordssrvs,scope=Namespaced + +// OrdsSrvs is the Schema for the ordssrvs API +type OrdsSrvs struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + Spec OrdsSrvsSpec `json:"spec,omitempty"` + Status OrdsSrvsStatus `json:"status,omitempty"` +} + +//+kubebuilder:object:root=true + +// OrdsSrvsList contains a list of OrdsSrvs +type OrdsSrvsList struct { + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata,omitempty"` + Items []OrdsSrvs `json:"items"` +} + +func init() { + SchemeBuilder.Register(&OrdsSrvs{}, &OrdsSrvsList{}) +} diff --git a/apis/database/v1alpha1/pdb_types.go b/apis/database/v4/pdb_types.go similarity index 96% rename from apis/database/v1alpha1/pdb_types.go rename to apis/database/v4/pdb_types.go index 8de9db52..16021f12 100644 --- a/apis/database/v1alpha1/pdb_types.go +++ b/apis/database/v4/pdb_types.go @@ -36,7 +36,7 @@ ** SOFTWARE. */ -package v1alpha1 +package v4 import ( metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" @@ -117,7 +117,9 @@ type PDBSpec struct { // turn on the assertive approach to delete pdb resource // kubectl delete pdb ..... automatically triggers the pluggable database // deletion - AssertivePdbDeletion bool `json:"assertivePdbDeletion,omitempty"` + AssertivePdbDeletion bool `json:"assertivePdbDeletion,omitempty"` + PDBPubKey PDBPUBKEY `json:"pdbOrdsPubKey,omitempty"` + PDBPriKey PDBPRIVKEY `json:"pdbOrdsPrvKey,omitempty"` } // PDBAdminName defines the secret containing Sys Admin User mapped to key 'adminName' for PDB @@ -194,14 +196,15 @@ type PDBStatus struct { // +kubebuilder:object:root=true // +kubebuilder:subresource:status -// +kubebuilder:printcolumn:JSONPath=".status.connString",name="Connect_String",type="string",description="The connect string to be used" // +kubebuilder:printcolumn:JSONPath=".spec.cdbName",name="CDB Name",type="string",description="Name of the CDB" // +kubebuilder:printcolumn:JSONPath=".spec.pdbName",name="PDB Name",type="string",description="Name of the PDB" // +kubebuilder:printcolumn:JSONPath=".status.openMode",name="PDB State",type="string",description="PDB Open Mode" // +kubebuilder:printcolumn:JSONPath=".status.totalSize",name="PDB Size",type="string",description="Total Size of the PDB" // +kubebuilder:printcolumn:JSONPath=".status.phase",name="Status",type="string",description="Status of the PDB Resource" // +kubebuilder:printcolumn:JSONPath=".status.msg",name="Message",type="string",description="Error message, if any" +// +kubebuilder:printcolumn:JSONPath=".status.connString",name="Connect_String",type="string",description="The connect string to be used" // +kubebuilder:resource:path=pdbs,scope=Namespaced +// +kubebuilder:storageversion // PDB is the Schema for the pdbs API type PDB struct { @@ -221,6 +224,14 @@ type PDBList struct { Items []PDB `json:"items"` } +type PDBPUBKEY struct { + Secret PDBSecret `json:"secret"` +} + +type PDBPRIVKEY struct { + Secret PDBSecret `json:"secret"` +} + func init() { SchemeBuilder.Register(&PDB{}, &PDBList{}) } diff --git a/apis/database/v1alpha1/pdb_webhook.go b/apis/database/v4/pdb_webhook.go similarity index 85% rename from apis/database/v1alpha1/pdb_webhook.go rename to apis/database/v4/pdb_webhook.go index 1577198e..f651accf 100644 --- a/apis/database/v1alpha1/pdb_webhook.go +++ b/apis/database/v4/pdb_webhook.go @@ -40,7 +40,7 @@ ** rcitton 07/14/22 - 33822886 */ -package v1alpha1 +package v4 import ( "reflect" @@ -66,7 +66,7 @@ func (r *PDB) SetupWebhookWithManager(mgr ctrl.Manager) error { Complete() } -//+kubebuilder:webhook:path=/mutate-database-oracle-com-v1alpha1-pdb,mutating=true,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=pdbs,verbs=create;update,versions=v1alpha1,name=mpdb.kb.io,admissionReviewVersions={v1,v1beta1} +//+kubebuilder:webhook:path=/mutate-database-oracle-com-v4-pdb,mutating=true,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=pdbs,verbs=create;update,versions=v4,name=mpdb.kb.io,admissionReviewVersions={v1,v1beta1} var _ webhook.Defaulter = &PDB{} @@ -107,6 +107,7 @@ func (r *PDB) Default() { *r.Spec.AsClone = false pdblog.Info(" - asClone : " + strconv.FormatBool(*(r.Spec.AsClone))) } + } if r.Spec.GetScript == nil { @@ -117,7 +118,7 @@ func (r *PDB) Default() { } // TODO(user): change verbs to "verbs=create;update;delete" if you want to enable deletion validation. -//+kubebuilder:webhook:path=/validate-database-oracle-com-v1alpha1-pdb,mutating=false,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=pdbs,verbs=create;update,versions=v1alpha1,name=vpdb.kb.io,admissionReviewVersions={v1,v1beta1} +//+kubebuilder:webhook:path=/validate-database-oracle-com-v4-pdb,mutating=false,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=pdbs,verbs=create;update,versions=v4,name=vpdb.kb.io,admissionReviewVersions={v1,v1beta1} var _ webhook.Validator = &PDB{} @@ -162,8 +163,19 @@ func (r *PDB) validateAction(allErrs *field.ErrorList) { *allErrs = append(*allErrs, field.Required(field.NewPath("spec").Child("pdbTlsCat"), "Please specify PDB Tls Certificate Authority(secret)")) } + if reflect.ValueOf(r.Spec.PDBPriKey).IsZero() { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("pdbOrdsPrvKey"), "Please specify PDB Tls Certificate Authority(secret)")) + } switch action { + case "DELETE": + /* BUG 36752336 - LREST OPERATOR - DELETE NON-EXISTENT PDB SHOWS LRPDB CREATED MESSAGE */ + if r.Status.OpenMode == "READ WRITE" { + pdblog.Info("Cannot delete: pdb is open ") + *allErrs = append(*allErrs, field.Invalid(field.NewPath("status").Child("OpenMode"), "READ WRITE", "pdb "+r.Spec.PDBName+" "+r.Status.OpenMode)) + } + r.CheckObjExistence("DELETE", allErrs, r) case "CREATE": if reflect.ValueOf(r.Spec.AdminName).IsZero() { *allErrs = append(*allErrs, @@ -211,6 +223,13 @@ func (r *PDB) validateAction(allErrs *field.ErrorList) { *allErrs = append(*allErrs, field.Required(field.NewPath("spec").Child("tempSize"), "When the storage is not UNLIMITED the Temp Size must be specified")) } + /* We don't need this check as ords open the pdb before cloninig */ + /* + if r.Status.OpenMode == "MOUNTED" { + pdblog.Info("Cannot clone: pdb is mount ") + *allErrs = append(*allErrs, field.Invalid(field.NewPath("status").Child("OpenMode"), "READ WRITE", "pdb "+r.Spec.PDBName+" "+r.Status.OpenMode)) + } + */ case "PLUG": if r.Spec.XMLFileName == "" { *allErrs = append(*allErrs, @@ -239,6 +258,11 @@ func (r *PDB) validateAction(allErrs *field.ErrorList) { if *(r.Spec.TDEExport) { r.validateTDEInfo(allErrs) } + if r.Status.OpenMode == "READ WRITE" { + pdblog.Info("Cannot unplug: pdb is open ") + *allErrs = append(*allErrs, field.Invalid(field.NewPath("status").Child("OpenMode"), "READ WRITE", "pdb "+r.Spec.PDBName+" "+r.Status.OpenMode)) + } + r.CheckObjExistence("UNPLUG", allErrs, r) case "MODIFY": if r.Spec.PDBState == "" { *allErrs = append(*allErrs, @@ -248,6 +272,7 @@ func (r *PDB) validateAction(allErrs *field.ErrorList) { *allErrs = append(*allErrs, field.Required(field.NewPath("spec").Child("modifyOption"), "Please specify an option for opening/closing a PDB")) } + r.CheckObjExistence("MODIY", allErrs, r) } } @@ -333,3 +358,12 @@ func (r *PDB) validateTDEInfo(allErrs *field.ErrorList) { } } + +func (r *PDB) CheckObjExistence(action string, allErrs *field.ErrorList, pdb *PDB) { + /* BUG 36752465 - lrest operator - open non-existent pdb creates a lrpdb with status failed */ + pdblog.Info("Action [" + action + "] checkin " + pdb.Spec.PDBName + " existence") + if pdb.Status.OpenMode == "" { + *allErrs = append(*allErrs, field.NotFound(field.NewPath("Spec").Child("PDBName"), " "+pdb.Spec.PDBName+" does not exist : action "+action+" failure")) + + } +} diff --git a/apis/database/v4/shardingdatabase_conversion.go b/apis/database/v4/shardingdatabase_conversion.go new file mode 100644 index 00000000..7b2c17ac --- /dev/null +++ b/apis/database/v4/shardingdatabase_conversion.go @@ -0,0 +1,4 @@ +package v4 + +// Hub defines v1 as the hub version +func (*ShardingDatabase) Hub() {} diff --git a/apis/database/v4/shardingdatabase_types.go b/apis/database/v4/shardingdatabase_types.go new file mode 100644 index 00000000..cc01b24d --- /dev/null +++ b/apis/database/v4/shardingdatabase_types.go @@ -0,0 +1,427 @@ +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v4 + +import ( + "sync" + + "encoding/json" + + "sigs.k8s.io/controller-runtime/pkg/client" + + annsv1 "github.com/oracle/oracle-database-operator/commons/annotations" + + corev1 "k8s.io/api/core/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! +// NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized. + +// ShardingDatabaseSpec defines the desired state of ShardingDatabase +type ShardingDatabaseSpec struct { + // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster + // Important: Run "make" to regenerate code after modifying this file + Shard []ShardSpec `json:"shard"` + Catalog []CatalogSpec `json:"catalog"` // The catalogSpes accept all the catalog parameters + Gsm []GsmSpec `json:"gsm"` // The GsmSpec will accept all the Gsm parameter + StorageClass string `json:"storageClass,omitempty"` // Optional Accept storage class name + DbImage string `json:"dbImage"` // Accept DB Image name + DbImagePullSecret string `json:"dbImagePullSecret,omitempty"` // Optional The name of an image pull secret in case of a private docker repository. + GsmImage string `json:"gsmImage"` // Acccept the GSM image name + GsmImagePullSecret string `json:"gsmImagePullSecret,omitempty"` // Optional The name of an image pull secret in case of a private docker repository. + StagePvcName string `json:"stagePvcName,omitempty"` // the Stagepvc for the backup of cluster + PortMappings []PortMapping `json:"portMappings,omitempty"` // Port mappings for the service that is created. The service is created if there is at least + IsDebug bool `json:"isDebug,omitempty"` // Optional parameter to enable logining + IsExternalSvc bool `json:"isExternalSvc,omitempty"` + IsClone bool `json:"isClone,omitempty"` + IsDataGuard bool `json:"isDataGuard,omitempty"` + ScriptsLocation string `json:"scriptsLocation,omitempty"` + IsDeleteOraPvc bool `json:"isDeleteOraPvc,omitempty"` + ReadinessCheckPeriod int `json:"readinessCheckPeriod,omitempty"` + LivenessCheckPeriod int `json:"liveinessCheckPeriod,omitempty"` + ReplicationType string `json:"replicationType,omitempty"` + IsDownloadScripts bool `json:"isDownloadScripts,omitempty"` + InvitedNodeSubnetFlag string `json:"invitedNodeSubnetFlag,omitempty"` + InvitedNodeSubnet string `json:"InvitedNodeSubnet,omitempty"` + ShardingType string `json:"shardingType,omitempty"` + GsmShardSpace []GsmShardSpaceSpec `json:"gsmShardSpace,omitempty"` + GsmShardGroup []GsmShardGroupSpec `json:"gsmShardGroup,omitempty"` + ShardRegion []string `json:"shardRegion,omitempty"` + ShardBuddyRegion string `json:"shardBuddyRegion,omitempty"` + GsmService []GsmServiceSpec `json:"gsmService,omitempty"` + ShardConfigName string `json:"shardConfigName,omitempty"` + GsmDevMode string `json:"gsmDevMode,omitempty"` + DbSecret *SecretDetails `json:"dbSecret,omitempty"` // Secret Name to be used with Shard + IsTdeWallet string `json:"isTdeWallet,omitempty"` + TdeWalletPvc string `json:"tdeWalletPvc,omitempty"` + FssStorageClass string `json:"fssStorageClass,omitempty"` + TdeWalletPvcMountLocation string `json:"tdeWalletPvcMountLocation,omitempty"` + DbEdition string `json:"dbEdition,omitempty"` + TopicId string `json:"topicId,omitempty"` +} + +// To understand Metav1.Condition, please refer the link https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1 +// ShardingDatabaseStatus defines the observed state of ShardingDatabase +type ShardingDatabaseStatus struct { + // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster + // Important: Run "make" to regenerate code after modifying this file + + Shard map[string]string `json:"shards,omitempty"` + Catalog map[string]string `json:"catalogs,omitempty"` + + Gsm GsmStatus `json:"gsm,omitempty"` + + // +patchMergeKey=type + // +patchStrategy=merge + // +listType=map + // +listMapKey=type + CrdStatus []metav1.Condition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type"` +} + +type GsmStatus struct { + InternalconnectStr string `json:"internalConnectStr,omitempty"` + ExternalConnectStr string `json:"externalConnectStr,omitempty"` + State string `json:"state,omitempty"` + Shards map[string]string `json:"shards,omitempty"` + Details map[string]string `json:"details,omitempty"` + Services string `json:"services,omitempty"` +} + +type GsmShardDetails struct { + Name string `json:"name,omitempty"` + Available string `json:"available,omitempty"` + State string `json:"State,omitempty"` +} + +type GsmStatusDetails struct { + Name string `json:"name,omitempty"` + K8sInternalSvc string `json:"k8sInternalSvc,omitempty"` + K8sExternalSvc string `json:"k8sExternalSvc,omitempty"` + K8sInternalSvcIP string `json:"k8sInternalIP,omitempty"` + K8sExternalSvcIP string `json:"k8sExternalIP,omitempty"` + Role string `json:"role,omitempty"` + DbPasswordSecret string `json:"dbPasswordSecret"` +} + +//+kubebuilder:object:root=true +//+kubebuilder:subresource:status +//+kubebuilder:printcolumn:JSONPath=".status.gsm.state",name="Gsm State",type=string +//+kubebuilder:printcolumn:JSONPath=".status.gsm.services",name="Services",type=string +//+kubebuilder:printcolumn:JSONPath=".status.gsm.shards",name="shards",type=string,priority=1 + +// ShardingDatabase is the Schema for the shardingdatabases API +// +kubebuilder:resource:path=shardingdatabases,scope=Namespaced +// +kubebuilder:storageversion +type ShardingDatabase struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + Spec ShardingDatabaseSpec `json:"spec,omitempty"` + Status ShardingDatabaseStatus `json:"status,omitempty"` +} + +//+kubebuilder:object:root=true + +// ShardingDatabaseList contains a list of ShardingDatabase +type ShardingDatabaseList struct { + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata,omitempty"` + Items []ShardingDatabase `json:"items"` +} + +// ShardSpec is a specification of Shards for an application deployment. +// +k8s:openapi-gen=true +type ShardSpec struct { + Name string `json:"name"` // Shard name that will be used deploy StatefulSet + StorageSizeInGb int32 `json:"storageSizeInGb,omitempty"` // Optional Shard Storage Size + EnvVars []EnvironmentVariable `json:"envVars,omitempty"` //Optional Env variables for Shards + Resources *corev1.ResourceRequirements `json:"resources,omitempty" protobuf:"bytes,1,opt,name=resources"` //Optional resource requirement for the container. + PvcName string `json:"pvcName,omitempty"` + Label string `json:"label,omitempty"` + // +kubebuilder:validation:Enum=enable;disable;failed;force + IsDelete string `json:"isDelete,omitempty"` + NodeSelector map[string]string `json:"nodeSelector,omitempty"` + PvAnnotations map[string]string `json:"pvAnnotations,omitempty"` + PvMatchLabels map[string]string `json:"pvMatchLabels,omitempty"` + ImagePulllPolicy *corev1.PullPolicy `json:"imagePullPolicy,omitempty"` + ShardSpace string `json:"shardSpace,omitempty"` + ShardGroup string `json:"shardGroup,omitempty"` + ShardRegion string `json:"shardRegion,omitempty"` + DeployAs string `json:"deployAs,omitempty"` +} + +// CatalogSpec defines the desired state of CatalogSpec +// +k8s:openapi-gen=true +type CatalogSpec struct { + Name string `json:"name"` // Catalog name that will be used deploy StatefulSet + StorageSizeInGb int32 `json:"storageSizeInGb,omitempty"` // Optional Catalog Storage Size and This parameter will not be used if you use PvcName + EnvVars []EnvironmentVariable `json:"envVars,omitempty"` //Optional Env variables for Catalog + Resources *corev1.ResourceRequirements `json:"resources,omitempty" protobuf:"bytes,1,opt,name=resources"` // Optional resource requirement for the container. + PvcName string `json:"pvcName,omitempty"` + Label string `json:"label,omitempty"` + IsDelete string `json:"isDelete,omitempty"` + NodeSelector map[string]string `json:"nodeSelector,omitempty"` + PvAnnotations map[string]string `json:"pvAnnotations,omitempty"` + PvMatchLabels map[string]string `json:"pvMatchLabels,omitempty"` + ImagePulllPolicy *corev1.PullPolicy `json:"imagePullPolicy,omitempty"` +} + +// GsmSpec defines the desired state of GsmSpec +// +k8s:openapi-gen=true +type GsmSpec struct { + Name string `json:"name"` // Gsm name that will be used deploy StatefulSet + + //Replicas int32 `json:"replicas,omitempty"` // Gsm Replicas. If you set OraGsmPvcName then it is set default to 1. + EnvVars []EnvironmentVariable `json:"envVars,omitempty"` //Optional Env variables for GSM + StorageSizeInGb int32 `json:"storageSizeInGb,omitempty"` // This parameter will not be used if you use OraGsmPvcName + Resources *corev1.ResourceRequirements `json:"resources,omitempty" protobuf:"bytes,1,opt,name=resources"` // Optional resource requirement for the container. + PvcName string `json:"pvcName,omitempty"` + Label string `json:"label,omitempty"` // Optional GSM Label + IsDelete string `json:"isDelete,omitempty"` + NodeSelector map[string]string `json:"nodeSelector,omitempty"` + PvAnnotations map[string]string `json:"pvAnnotations,omitempty"` + PvMatchLabels map[string]string `json:"pvMatchLabels,omitempty"` + ImagePulllPolicy *corev1.PullPolicy `json:"imagePullPolicy,omitempty"` + Region string `json:"region,omitempty"` + DirectorName string `json:"directorName,omitempty"` +} + +// ShardGroupSpec Specification + +type GsmShardGroupSpec struct { + Name string `json:"name"` // Name of the shardgroup. + Region string `json:"region,omitempty"` + DeployAs string `json:"deployAs,omitempty"` +} + +// ShardSpace Specs +type GsmShardSpaceSpec struct { + Name string `json:"name"` // Name of the shardSpace. + Chunks int `json:"chunks,omitempty"` //chunks is optional + ProtectionMode string `json:"protectionMode,omitempty"` // Data guard protection mode + ShardGroup string `json:"shardGroup,omitempty"` +} + +// Service Definition +type GsmServiceSpec struct { + Name string `json:"name"` // Name of the shardSpace. + Available string `json:"available,omitempty"` + ClbGoal string `json:"clbGoal,omitempty"` + CommitOutcome string `json:"commitOutcome,omitempty"` + DrainTimeout string `json:"drainTimeout,omitempty"` + Dtp string `json:"dtp,omitempty"` + Edition string `json:"edition,omitempty"` + FailoverPrimary string `json:"failoverPrimary,omitempty"` + FailoverRestore string `json:"failoverRestore,omitempty"` + FailoverDelay string `json:"failoverDelay,omitempty"` + FailoverMethod string `json:"failoverMethod,omitempty"` + FailoverRetry string `json:"failoverRetry,omitempty"` + FailoverType string `json:"failoverType,omitempty"` + GdsPool string `json:"gdsPool,omitempty"` + Role string `json:"role,omitempty"` + SessionState string `json:"sessionState,omitempty"` + Lag int `json:"lag,omitempty"` + Locality string `json:"locality,omitempty"` + Notification string `json:"notification,omitempty"` + PdbName string `json:"pdbName,omitempty"` + Policy string `json:"policy,omitempty"` + Preferrred string `json:"preferred,omitempty"` + PreferredAll string `json:"prferredAll,omitempty"` + RegionFailover string `json:"regionFailover,omitempty"` + StopOption string `json:"stopOption,omitempty"` + SqlTrasactionProfile string `json:"sqlTransactionProfile,omitempty"` + TableFamily string `json:"tableFamily,omitempty"` + Retention string `json:"retention,omitempty"` + TfaPolicy string `json:"tfaPolicy,omitempty"` +} + +// Secret Details +type SecretDetails struct { + Name string `json:"name"` // Name of the secret. + KeyFileName string `json:"keyFileName,omitempty"` // Name of the key. + NsConfigMap string `json:"nsConfigMap,omitempty"` + NsSecret string `json:"nsSecret,omitempty"` + PwdFileName string `json:"pwdFileName"` + PwdFileMountLocation string `json:"pwdFileMountLocation,omitempty"` + KeyFileMountLocation string `json:"keyFileMountLocation,omitempty"` + KeySecretName string `json:"keySecretName,omitempty"` + EncryptionType string `json:"encryptionType,omitempty"` +} + +// EnvironmentVariable represents a named variable accessible for containers. +// +k8s:openapi-gen=true +type EnvironmentVariable struct { + Name string `json:"name"` // Name of the variable. Must be a C_IDENTIFIER. + Value string `json:"value"` // Value of the variable, as defined in Kubernetes core API. +} + +// PortMapping is a specification of port mapping for an application deployment. +// +k8s:openapi-gen=true +type PortMapping struct { + Port int32 `json:"port"` // Port that will be exposed on the service. + TargetPort int32 `json:"targetPort"` // Docker image port for the application. + Protocol corev1.Protocol `json:"protocol"` // IP protocol for the mapping, e.g., "TCP" or "UDP". +} + +type SfsetLabel string + +const ( + ShardingDelLabelKey SfsetLabel = "sharding.oracle.com/delflag" + ShardingDelLabelTrueValue SfsetLabel = "true" + ShardingDelLabelFalseValue SfsetLabel = "false" +) + +type ShardStatusMapKeys string + +const ( + Name ShardStatusMapKeys = "Name" + K8sInternalSvc ShardStatusMapKeys = "K8sInternalSvc" + K8sExternalSvc ShardStatusMapKeys = "K8sExternalSvc" + K8sInternalSvcIP ShardStatusMapKeys = "K8sInternalSvcIP" + K8sExternalSvcIP ShardStatusMapKeys = "K8sExternalSvcIP" + OracleSid ShardStatusMapKeys = "OracleSid" + OraclePdb ShardStatusMapKeys = "OraclePdb" + Role ShardStatusMapKeys = "Role" + DbPasswordSecret ShardStatusMapKeys = "DbPasswordSecret" + State ShardStatusMapKeys = "State" + OpenMode ShardStatusMapKeys = "OpenMode" +) + +type ShardLifecycleState string + +const ( + AvailableState ShardLifecycleState = "AVAILABLE" + FailedState ShardLifecycleState = "FAILED" + UpdateState ShardLifecycleState = "UPDATING" + ProvisionState ShardLifecycleState = "PROVISIONING" + PodNotReadyState ShardLifecycleState = "PODNOTREADY" + PodFailureState ShardLifecycleState = "PODFAILURE" + PodNotFound ShardLifecycleState = "PODNOTFOUND" + StatefulSetFailure ShardLifecycleState = "STATEFULSETFAILURE" + StatefulSetNotFound ShardLifecycleState = "STATEFULSETNOTFOUND" + DeletingState ShardLifecycleState = "DELETING" + DeleteErrorState ShardLifecycleState = "DELETE_ERROR" + ChunkMoveError ShardLifecycleState = "CHUNK_MOVE_ERROR_IN_GSM" + Terminated ShardLifecycleState = "TERMINATED" + LabelPatchingError ShardLifecycleState = "LABELPATCHINGERROR" + DeletePVCError ShardLifecycleState = "DELETEPVCERROR" + AddingShardState ShardLifecycleState = "SHARD_ADDITION" + AddingShardErrorState ShardLifecycleState = "SHARD_ADDITION_ERROR_IN_GSM" + ShardOnlineErrorState ShardLifecycleState = "SHARD_ONLINE_ERROR_IN_GSM" + ShardOnlineState ShardLifecycleState = "ONLINE_SHARD" + ShardRemoveError ShardLifecycleState = "SHARD_DELETE_ERROR_FROM_GSM" +) + +type CrdReconcileState string + +const ( + CrdReconcileErrorState CrdReconcileState = "ReconcileError" + CrdReconcileErrorReason CrdReconcileState = "LastReconcileCycleFailed" + CrdReconcileQueuedState CrdReconcileState = "ReconcileQueued" + CrdReconcileQueuedReason CrdReconcileState = "LastReconcileCycleQueued" + CrdReconcileCompeleteState CrdReconcileState = "ReconcileComplete" + CrdReconcileCompleteReason CrdReconcileState = "LastReconcileCycleCompleted" + CrdReconcileWaitingState CrdReconcileState = "ReconcileWaiting" + CrdReconcileWaitingReason CrdReconcileState = "LastReconcileCycleWaiting" +) + +// var +var KubeConfigOnce sync.Once + +// #const lastSuccessfulSpec = "lastSuccessfulSpec" +const lastSuccessfulSpecOnsInfo = "lastSuccessfulSpeOnsInfo" + +// GetLastSuccessfulSpec returns spec from the lass successful reconciliation. +// Returns nil, nil if there is no lastSuccessfulSpec. +func (shardingv1 *ShardingDatabase) GetLastSuccessfulSpec() (*ShardingDatabaseSpec, error) { + val, ok := shardingv1.GetAnnotations()[lastSuccessfulSpec] + if !ok { + return nil, nil + } + + specBytes := []byte(val) + sucSpec := ShardingDatabaseSpec{} + + err := json.Unmarshal(specBytes, &sucSpec) + if err != nil { + return nil, err + } + + return &sucSpec, nil +} + +// UpdateLastSuccessfulSpec updates lastSuccessfulSpec with the current spec. +func (shardingv1 *ShardingDatabase) UpdateLastSuccessfulSpec(kubeClient client.Client) error { + specBytes, err := json.Marshal(shardingv1.Spec) + if err != nil { + return err + } + + anns := map[string]string{ + lastSuccessfulSpec: string(specBytes), + } + + return annsv1.PatchAnnotations(kubeClient, shardingv1, anns) +} + +// GetLastSuccessfulOnsInfo returns spec from the lass successful reconciliation. +// Returns nil, nil if there is no lastSuccessfulSpec. +func (shardingv1 *ShardingDatabase) GetLastSuccessfulOnsInfo() ([]byte, error) { + val, ok := shardingv1.GetAnnotations()[lastSuccessfulSpecOnsInfo] + if !ok { + return nil, nil + } + specBytes := []byte(val) + return specBytes, nil +} + +// UpdateLastSuccessfulSpec updates lastSuccessfulSpec with the current spec. +func (shardingv1 *ShardingDatabase) UpdateLastSuccessfulSpecOnsInfo(kubeClient client.Client, specBytes []byte) error { + + anns := map[string]string{ + lastSuccessfulSpecOnsInfo: string(specBytes), + } + + return annsv1.PatchAnnotations(kubeClient, shardingv1, anns) +} + +func init() { + SchemeBuilder.Register(&ShardingDatabase{}, &ShardingDatabaseList{}) +} diff --git a/apis/database/v4/shardingdatabase_webhook.go b/apis/database/v4/shardingdatabase_webhook.go new file mode 100644 index 00000000..1ac74d08 --- /dev/null +++ b/apis/database/v4/shardingdatabase_webhook.go @@ -0,0 +1,314 @@ +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v4 + +import ( + "strings" + + apierrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/runtime/schema" + "k8s.io/apimachinery/pkg/util/validation/field" + ctrl "sigs.k8s.io/controller-runtime" + logf "sigs.k8s.io/controller-runtime/pkg/log" + "sigs.k8s.io/controller-runtime/pkg/webhook" + "sigs.k8s.io/controller-runtime/pkg/webhook/admission" +) + +// log is for logging in this package. +var shardingdatabaselog = logf.Log.WithName("shardingdatabase-resource") + +func (r *ShardingDatabase) SetupWebhookWithManager(mgr ctrl.Manager) error { + return ctrl.NewWebhookManagedBy(mgr). + For(r). + Complete() +} + +// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! + +//+kubebuilder:webhook:path=/mutate-database-oracle-com-v4-shardingdatabase,mutating=true,failurePolicy=fail,sideEffects=none,groups=database.oracle.com,resources=shardingdatabases,verbs=create;update,versions=v4,name=mshardingdatabasev4.kb.io,admissionReviewVersions=v1 + +var _ webhook.Defaulter = &ShardingDatabase{} + +// Default implements webhook.Defaulter so a webhook will be registered for the type +func (r *ShardingDatabase) Default() { + shardingdatabaselog.Info("default", "name", r.Name) + + // TODO(user): fill in your defaulting logic. + if r.Spec.GsmDevMode != "" { + r.Spec.GsmDevMode = "dev" + } + + if r.Spec.IsTdeWallet == "" { + r.Spec.IsTdeWallet = "disable" + } + for pindex := range r.Spec.Shard { + if strings.ToLower(r.Spec.Shard[pindex].IsDelete) == "" { + r.Spec.Shard[pindex].IsDelete = "disable" + } + } + +} + +// TODO(user): change verbs to "verbs=create;update;delete" if you want to enable deletion validation. +//+kubebuilder:webhook:verbs=create;update;delete,path=/validate-database-oracle-com-v4-shardingdatabase,mutating=false,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=shardingdatabases,versions=v4,name=vshardingdatabasev4.kb.io,admissionReviewVersions={v1} + +var _ webhook.Validator = &ShardingDatabase{} + +// ValidateCreate implements webhook.Validator so a webhook will be registered for the type +func (r *ShardingDatabase) ValidateCreate() (admission.Warnings, error) { + shardingdatabaselog.Info("validate create", "name", r.Name) + + // TODO(user): fill in your validation logic upon object creation. + // Check Secret configuration + var validationErr field.ErrorList + var validationErrs1 field.ErrorList + + //namespaces := db.GetWatchNamespaces() + //_, containsNamespace := namespaces[r.Namespace] + // Check if the allowed namespaces maps contains the required namespace + // if len(namespaces) != 0 && !containsNamespace { + // validationErr = append(validationErr, + // field.Invalid(field.NewPath("metadata").Child("namespace"), r.Namespace, + // "Oracle database operator doesn't watch over this namespace")) + //} + + if r.Spec.DbSecret == nil { + validationErr = append(validationErr, + field.Invalid(field.NewPath("spec").Child("DbSecret"), r.Spec.DbSecret, + "DbSecret cannot be set to nil")) + } else { + if len(r.Spec.DbSecret.Name) == 0 { + validationErr = append(validationErr, + field.Invalid(field.NewPath("spec").Child("DbSecret").Child("Name"), r.Spec.DbSecret.Name, + "Secret name cannot be set empty")) + } + if len(r.Spec.DbSecret.PwdFileName) == 0 { + validationErr = append(validationErr, + field.Invalid(field.NewPath("spec").Child("DbSecret").Child("PwdFileName"), r.Spec.DbSecret.PwdFileName, + "Password file name cannot be set empty")) + } + if strings.ToLower(r.Spec.DbSecret.EncryptionType) != "base64" { + if strings.ToLower(r.Spec.DbSecret.KeyFileName) == "" { + validationErr = append(validationErr, + field.Invalid(field.NewPath("spec").Child("DbSecret").Child("KeyFileName"), r.Spec.DbSecret.KeyFileName, + "Key file name cannot be empty")) + } + } + + /** + if len(r.Spec.DbSecret.PwdFileMountLocation) == 0 { + validationErr = append(validationErr, + field.Invalid(field.NewPath("spec").Child("DbSecret").Child("PwdFileMountLocation"), r.Spec.DbSecret.PwdFileMountLocation, + "Password file mount location cannot be empty")) + } + + if len(r.Spec.DbSecret.KeyFileMountLocation) == 0 { + validationErr = append(validationErr, + field.Invalid(field.NewPath("spec").Child("DbSecret").Child("KeyFileMountLocation"), r.Spec.DbSecret.KeyFileMountLocation, + "KeyFileMountLocation file mount location cannot be empty")) + } + **/ + } + + if r.Spec.IsTdeWallet == "enable" { + if (len(r.Spec.FssStorageClass) == 0) && (len(r.Spec.TdeWalletPvc) == 0) { + validationErr = append(validationErr, + field.Invalid(field.NewPath("spec").Child("FssStorageClass"), r.Spec.FssStorageClass, + "FssStorageClass or TdeWalletPvc cannot be set empty if isTdeWallet set to true")) + + validationErr = append(validationErr, + field.Invalid(field.NewPath("spec").Child("TdeWalletPvc"), r.Spec.TdeWalletPvc, + "FssStorageClass or TdeWalletPvc cannot be set empty if isTdeWallet set to true")) + } + } + + if r.Spec.IsTdeWallet != "" { + if (strings.ToLower(strings.TrimSpace(r.Spec.IsTdeWallet)) != "enable") && (strings.ToLower(strings.TrimSpace(r.Spec.IsTdeWallet)) != "disable") { + validationErr = append(validationErr, + field.Invalid(field.NewPath("spec").Child("isTdeWallet"), r.Spec.IsTdeWallet, + "isTdeWallet can be set to only \"enable\" or \"disable\"")) + } + } + + validationErrs1 = r.validateShardIsDelete() + if validationErrs1 != nil { + validationErr = append(validationErr, validationErrs1...) + } + + validationErrs1 = r.validateFreeEdition() + if validationErrs1 != nil { + validationErr = append(validationErr, validationErrs1...) + } + + validationErrs1 = r.validateCatalogName() + if validationErrs1 != nil { + validationErr = append(validationErr, validationErrs1...) + } + + validationErrs1 = r.validateShardName() + if validationErrs1 != nil { + validationErr = append(validationErr, validationErrs1...) + } + + // TODO(user): fill in your validation logic upon object creation. + if len(validationErr) == 0 { + return nil, nil + } + + return nil, apierrors.NewInvalid( + schema.GroupKind{Group: "database.oracle.com", Kind: "ShardingDatabase"}, + r.Name, validationErr) +} + +// ValidateUpdate implements webhook.Validator so a webhook will be registered for the type +func (r *ShardingDatabase) ValidateUpdate(old runtime.Object) (admission.Warnings, error) { + shardingdatabaselog.Info("validate update", "name", r.Name) + + // TODO(user): fill in your validation logic upon object update. + return nil, nil +} + +// ValidateDelete implements webhook.Validator so a webhook will be registered for the type +func (r *ShardingDatabase) ValidateDelete() (admission.Warnings, error) { + shardingdatabaselog.Info("validate delete", "name", r.Name) + + // TODO(user): fill in your validation logic upon object deletion. + return nil, nil +} + +// ###### Vlaidation Block ################# + +func (r *ShardingDatabase) validateShardIsDelete() field.ErrorList { + + var validationErrs field.ErrorList + + for pindex := range r.Spec.Shard { + if (strings.ToLower(strings.TrimSpace(r.Spec.Shard[pindex].IsDelete)) != "enable") && (strings.ToLower(strings.TrimSpace(r.Spec.Shard[pindex].IsDelete)) != "disable") && (strings.ToLower(strings.TrimSpace(r.Spec.Shard[pindex].IsDelete)) != "failed") { + validationErrs = append(validationErrs, + field.Invalid(field.NewPath("spec").Child("shard").Child("isDelete"), r.Spec.Shard[pindex].IsDelete, + "r.Spec.Shard[pindex].IsDelete can be set to only enable|disable|failed")) + } + } + + if len(validationErrs) > 0 { + return validationErrs + } + return nil +} + +func (r *ShardingDatabase) validateFreeEdition() field.ErrorList { + + var validationErrs field.ErrorList + if strings.ToLower(r.Spec.DbEdition) == "free" { + // Shard Spec Checks + for i := 0; i < len(r.Spec.Shard); i++ { + for index, variable := range r.Spec.Shard[i].EnvVars { + if variable.Name == "ORACLE_SID" { + if strings.ToLower(variable.Value) != "free" { + validationErrs = append(validationErrs, field.Invalid(field.NewPath("spec").Child("shard").Child("EnvVars"), r.Spec.Shard[i].EnvVars[index].Name, + "r.Spec.Shard[i].EnvVars[index].Name ORACLE_SID value can only be set to free")) + } + } + if variable.Name == "ORACLE_PDB" { + if strings.ToLower(variable.Value) != "freepdb" { + validationErrs = append(validationErrs, field.Invalid(field.NewPath("spec").Child("shard").Child("EnvVars"), r.Spec.Shard[i].EnvVars[index].Name, + "r.Spec.Shard[i].EnvVars[index].Name ORACLE_PDB value can only be set to freepdb")) + } + } + } + } + // Catalog Spec Checks + for i := 0; i < len(r.Spec.Catalog); i++ { + for index, variable := range r.Spec.Catalog[i].EnvVars { + if variable.Name == "ORACLE_SID" { + if strings.ToLower(variable.Value) != "free" { + validationErrs = append(validationErrs, field.Invalid(field.NewPath("spec").Child("catalog").Child("EnvVars"), r.Spec.Catalog[i].EnvVars[index].Name, + "r.Spec.Catalog[i].EnvVars[index].Name ORACLE_SID value can only be set to free")) + } + } + if variable.Name == "ORACLE_PDB" { + if strings.ToLower(variable.Value) != "freepdb" { + validationErrs = append(validationErrs, field.Invalid(field.NewPath("spec").Child("catalog").Child("EnvVars"), r.Spec.Catalog[i].EnvVars[index].Name, + "r.Spec.Catalog[i].EnvVars[index].Name ORACLE_PDB value can only be set to freepdb")) + } + } + } + } + } + + if len(validationErrs) > 0 { + return validationErrs + } + return nil +} + +func (r *ShardingDatabase) validateShardName() field.ErrorList { + var validationErrs field.ErrorList + + for pindex := range r.Spec.Shard { + if len(r.Spec.Shard[pindex].Name) > 9 { + validationErrs = append(validationErrs, + field.Invalid(field.NewPath("spec").Child("shard").Child("Name"), r.Spec.Shard[pindex].Name, + "Shard Name cannot be greater than 9 characters.")) + } + } + + if len(validationErrs) > 0 { + return validationErrs + } + return nil +} + +func (r *ShardingDatabase) validateCatalogName() field.ErrorList { + var validationErrs field.ErrorList + + for pindex := range r.Spec.Catalog { + if len(r.Spec.Catalog[pindex].Name) > 9 { + validationErrs = append(validationErrs, + field.Invalid(field.NewPath("spec").Child("catalog").Child("Name"), r.Spec.Catalog[pindex].Name, + "Catalog Name cannot be greater than 9 characters.")) + } + } + + if len(validationErrs) > 0 { + return validationErrs + } + return nil +} diff --git a/apis/database/v4/singleinstancedatabase_conversion.go b/apis/database/v4/singleinstancedatabase_conversion.go new file mode 100644 index 00000000..93638482 --- /dev/null +++ b/apis/database/v4/singleinstancedatabase_conversion.go @@ -0,0 +1,4 @@ +package v4 + +// Hub defines v1 as the hub version +func (*SingleInstanceDatabase) Hub() {} diff --git a/apis/database/v4/singleinstancedatabase_types.go b/apis/database/v4/singleinstancedatabase_types.go new file mode 100644 index 00000000..4f4836d7 --- /dev/null +++ b/apis/database/v4/singleinstancedatabase_types.go @@ -0,0 +1,231 @@ +/* +** Copyright (c) 2023 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v4 + +import ( + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! +// NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized. + +// SingleInstanceDatabaseSpec defines the desired state of SingleInstanceDatabase +type SingleInstanceDatabaseSpec struct { + // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster + // Important: Run "make" to regenerate code after modifying this file + + // +kubebuilder:validation:Enum=standard;enterprise;express;free + Edition string `json:"edition,omitempty"` + + // SID must be alphanumeric (no special characters, only a-z, A-Z, 0-9), and no longer than 12 characters. + // +k8s:openapi-gen=true + // +kubebuilder:validation:Pattern=`^[a-zA-Z0-9]+$` + // +kubebuilder:validation:MaxLength:=12 + Sid string `json:"sid,omitempty"` + Charset string `json:"charset,omitempty"` + Pdbname string `json:"pdbName,omitempty"` + LoadBalancer bool `json:"loadBalancer,omitempty"` + ListenerPort int `json:"listenerPort,omitempty"` + TcpsListenerPort int `json:"tcpsListenerPort,omitempty"` + ServiceAnnotations map[string]string `json:"serviceAnnotations,omitempty"` + FlashBack *bool `json:"flashBack,omitempty"` + ArchiveLog *bool `json:"archiveLog,omitempty"` + ForceLogging *bool `json:"forceLog,omitempty"` + EnableTCPS bool `json:"enableTCPS,omitempty"` + TcpsCertRenewInterval string `json:"tcpsCertRenewInterval,omitempty"` + TcpsTlsSecret string `json:"tcpsTlsSecret,omitempty"` + + PrimaryDatabaseRef string `json:"primaryDatabaseRef,omitempty"` + // +kubebuilder:validation:Enum=primary;standby;clone;truecache + CreateAs string `json:"createAs,omitempty"` + ReadinessCheckPeriod int `json:"readinessCheckPeriod,omitempty"` + ServiceAccountName string `json:"serviceAccountName,omitempty"` + TrueCacheServices []string `json:"trueCacheServices,omitempty"` + + // +k8s:openapi-gen=true + Replicas int `json:"replicas,omitempty"` + + NodeSelector map[string]string `json:"nodeSelector,omitempty"` + AdminPassword SingleInstanceDatabaseAdminPassword `json:"adminPassword,omitempty"` + Image SingleInstanceDatabaseImage `json:"image"` + Persistence SingleInstanceDatabasePersistence `json:"persistence,omitempty"` + InitParams *SingleInstanceDatabaseInitParams `json:"initParams,omitempty"` + Resources SingleInstanceDatabaseResources `json:"resources,omitempty"` + + ConvertToSnapshotStandby bool `json:"convertToSnapshotStandby,omitempty"` +} + +type SingleInstanceDatabaseResource struct { + Cpu string `json:"cpu,omitempty"` + Memory string `json:"memory,omitempty"` +} + +type SingleInstanceDatabaseResources struct { + Requests *SingleInstanceDatabaseResource `json:"requests,omitempty"` + Limits *SingleInstanceDatabaseResource `json:"limits,omitempty"` +} + +// SingleInstanceDatabasePersistence defines the storage size and class for PVC +type SingleInstanceDatabasePersistence struct { + Size string `json:"size,omitempty"` + StorageClass string `json:"storageClass,omitempty"` + // +kubebuilder:validation:Enum=ReadWriteOnce;ReadWriteMany + AccessMode string `json:"accessMode,omitempty"` + DatafilesVolumeName string `json:"datafilesVolumeName,omitempty"` + ScriptsVolumeName string `json:"scriptsVolumeName,omitempty"` + VolumeClaimAnnotation string `json:"volumeClaimAnnotation,omitempty"` + SetWritePermissions *bool `json:"setWritePermissions,omitempty"` +} + +// SingleInstanceDatabaseInitParams defines the Init Parameters +type SingleInstanceDatabaseInitParams struct { + SgaTarget int `json:"sgaTarget,omitempty"` + PgaAggregateTarget int `json:"pgaAggregateTarget,omitempty"` + CpuCount int `json:"cpuCount,omitempty"` + Processes int `json:"processes,omitempty"` +} + +// SingleInstanceDatabaseImage defines the Image source and pullSecrets for POD +type SingleInstanceDatabaseImage struct { + Version string `json:"version,omitempty"` + PullFrom string `json:"pullFrom"` + PullSecrets string `json:"pullSecrets,omitempty"` + PrebuiltDB bool `json:"prebuiltDB,omitempty"` +} + +// SingleInsatnceAdminPassword defines the secret containing Admin Password mapped to secretKey for Database +type SingleInstanceDatabaseAdminPassword struct { + SecretName string `json:"secretName"` + // +kubebuilder:default:="oracle_pwd" + SecretKey string `json:"secretKey,omitempty"` + KeepSecret *bool `json:"keepSecret,omitempty"` +} + +// SingleInstanceDatabaseStatus defines the observed state of SingleInstanceDatabase +type SingleInstanceDatabaseStatus struct { + // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster + // Important: Run "make" to regenerate code after modifying this file + + Nodes []string `json:"nodes,omitempty"` + Role string `json:"role,omitempty"` + Status string `json:"status,omitempty"` + Replicas int `json:"replicas,omitempty"` + ReleaseUpdate string `json:"releaseUpdate,omitempty"` + DgBroker *string `json:"dgBroker,omitempty"` + // +kubebuilder:default:="false" + DatafilesPatched string `json:"datafilesPatched,omitempty"` + ConnectString string `json:"connectString,omitempty"` + ClusterConnectString string `json:"clusterConnectString,omitempty"` + TcpsConnectString string `json:"tcpsConnectString,omitempty"` + StandbyDatabases map[string]string `json:"standbyDatabases,omitempty"` + // +kubebuilder:default:="false" + DatafilesCreated string `json:"datafilesCreated,omitempty"` + Sid string `json:"sid,omitempty"` + Edition string `json:"edition,omitempty"` + Charset string `json:"charset,omitempty"` + Pdbname string `json:"pdbName,omitempty"` + InitSgaSize int `json:"initSgaSize,omitempty"` + InitPgaSize int `json:"initPgaSize,omitempty"` + CreatedAs string `json:"createdAs,omitempty"` + FlashBack string `json:"flashBack,omitempty"` + ArchiveLog string `json:"archiveLog,omitempty"` + ForceLogging string `json:"forceLog,omitempty"` + OemExpressUrl string `json:"oemExpressUrl,omitempty"` + OrdsReference string `json:"ordsReference,omitempty"` + PdbConnectString string `json:"pdbConnectString,omitempty"` + TcpsPdbConnectString string `json:"tcpsPdbConnectString,omitempty"` + ApexInstalled bool `json:"apexInstalled,omitempty"` + PrebuiltDB bool `json:"prebuiltDB,omitempty"` + // +kubebuilder:default:=false + IsTcpsEnabled bool `json:"isTcpsEnabled"` + CertCreationTimestamp string `json:"certCreationTimestamp,omitempty"` + CertRenewInterval string `json:"certRenewInterval,omitempty"` + ClientWalletLoc string `json:"clientWalletLoc,omitempty"` + PrimaryDatabase string `json:"primaryDatabase,omitempty"` + // +kubebuilder:default:="" + TcpsTlsSecret string `json:"tcpsTlsSecret"` + + // +patchMergeKey=type + // +patchStrategy=merge + // +listType=map + // +listMapKey=type + Conditions []metav1.Condition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type"` + + InitParams SingleInstanceDatabaseInitParams `json:"initParams,omitempty"` + Persistence SingleInstanceDatabasePersistence `json:"persistence"` + + ConvertToSnapshotStandby bool `json:"convertToSnapshotStandby,omitempty"` +} + +//+kubebuilder:object:root=true +//+kubebuilder:subresource:status +// +kubebuilder:subresource:scale:specpath=.spec.replicas,statuspath=.status.replicas +// +kubebuilder:printcolumn:JSONPath=".status.edition",name="Edition",type="string" +// +kubebuilder:printcolumn:JSONPath=".status.sid",name="Sid",type="string",priority=1 +// +kubebuilder:printcolumn:JSONPath=".status.status",name="Status",type="string" +// +kubebuilder:printcolumn:JSONPath=".status.role",name="Role",type="string" +// +kubebuilder:printcolumn:JSONPath=".status.releaseUpdate",name="Version",type="string" +// +kubebuilder:printcolumn:JSONPath=".status.connectString",name="Connect Str",type="string" +// +kubebuilder:printcolumn:JSONPath=".status.pdbConnectString",name="Pdb Connect Str",type="string",priority=1 +// +kubebuilder:printcolumn:JSONPath=".status.tcpsConnectString",name="TCPS Connect Str",type="string" +// +kubebuilder:printcolumn:JSONPath=".status.tcpsPdbConnectString",name="TCPS Pdb Connect Str",type="string", priority=1 +// +kubebuilder:printcolumn:JSONPath=".status.oemExpressUrl",name="Oem Express Url",type="string" + +// SingleInstanceDatabase is the Schema for the singleinstancedatabases API +// +kubebuilder:storageversion +type SingleInstanceDatabase struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + Spec SingleInstanceDatabaseSpec `json:"spec,omitempty"` + Status SingleInstanceDatabaseStatus `json:"status,omitempty"` +} + +//+kubebuilder:object:root=true + +// SingleInstanceDatabaseList contains a list of SingleInstanceDatabase +type SingleInstanceDatabaseList struct { + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata,omitempty"` + Items []SingleInstanceDatabase `json:"items"` +} + +func init() { + SchemeBuilder.Register(&SingleInstanceDatabase{}, &SingleInstanceDatabaseList{}) +} diff --git a/apis/database/v4/singleinstancedatabase_webhook.go b/apis/database/v4/singleinstancedatabase_webhook.go new file mode 100644 index 00000000..b327d7d4 --- /dev/null +++ b/apis/database/v4/singleinstancedatabase_webhook.go @@ -0,0 +1,55 @@ +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v4 + +import ( + ctrl "sigs.k8s.io/controller-runtime" + logf "sigs.k8s.io/controller-runtime/pkg/log" +) + +// log is for logging in this package. +var singleinstancedatabaselog = logf.Log.WithName("singleinstancedatabase-resource") + +func (r *SingleInstanceDatabase) SetupWebhookWithManager(mgr ctrl.Manager) error { + return ctrl.NewWebhookManagedBy(mgr). + For(r). + Complete() +} + +// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! diff --git a/apis/database/v4/zz_generated.deepcopy.go b/apis/database/v4/zz_generated.deepcopy.go new file mode 100644 index 00000000..4eb9425d --- /dev/null +++ b/apis/database/v4/zz_generated.deepcopy.go @@ -0,0 +1,4213 @@ +//go:build !ignore_autogenerated + +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +// Code generated by controller-gen. DO NOT EDIT. + +package v4 + +import ( + corev1 "k8s.io/api/core/v1" + "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/runtime" + timex "time" +) + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AcdSpec) DeepCopyInto(out *AcdSpec) { + *out = *in + in.K8sAcd.DeepCopyInto(&out.K8sAcd) + in.OciAcd.DeepCopyInto(&out.OciAcd) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AcdSpec. +func (in *AcdSpec) DeepCopy() *AcdSpec { + if in == nil { + return nil + } + out := new(AcdSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AdminpdbPass) DeepCopyInto(out *AdminpdbPass) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AdminpdbPass. +func (in *AdminpdbPass) DeepCopy() *AdminpdbPass { + if in == nil { + return nil + } + out := new(AdminpdbPass) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AdminpdbUser) DeepCopyInto(out *AdminpdbUser) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AdminpdbUser. +func (in *AdminpdbUser) DeepCopy() *AdminpdbUser { + if in == nil { + return nil + } + out := new(AdminpdbUser) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousContainerDatabase) DeepCopyInto(out *AutonomousContainerDatabase) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + in.Spec.DeepCopyInto(&out.Spec) + out.Status = in.Status +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousContainerDatabase. +func (in *AutonomousContainerDatabase) DeepCopy() *AutonomousContainerDatabase { + if in == nil { + return nil + } + out := new(AutonomousContainerDatabase) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *AutonomousContainerDatabase) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousContainerDatabaseList) DeepCopyInto(out *AutonomousContainerDatabaseList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]AutonomousContainerDatabase, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousContainerDatabaseList. +func (in *AutonomousContainerDatabaseList) DeepCopy() *AutonomousContainerDatabaseList { + if in == nil { + return nil + } + out := new(AutonomousContainerDatabaseList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *AutonomousContainerDatabaseList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousContainerDatabaseSpec) DeepCopyInto(out *AutonomousContainerDatabaseSpec) { + *out = *in + if in.AutonomousContainerDatabaseOCID != nil { + in, out := &in.AutonomousContainerDatabaseOCID, &out.AutonomousContainerDatabaseOCID + *out = new(string) + **out = **in + } + if in.CompartmentOCID != nil { + in, out := &in.CompartmentOCID, &out.CompartmentOCID + *out = new(string) + **out = **in + } + if in.DisplayName != nil { + in, out := &in.DisplayName, &out.DisplayName + *out = new(string) + **out = **in + } + if in.AutonomousExadataVMClusterOCID != nil { + in, out := &in.AutonomousExadataVMClusterOCID, &out.AutonomousExadataVMClusterOCID + *out = new(string) + **out = **in + } + if in.FreeformTags != nil { + in, out := &in.FreeformTags, &out.FreeformTags + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + in.OCIConfig.DeepCopyInto(&out.OCIConfig) + if in.HardLink != nil { + in, out := &in.HardLink, &out.HardLink + *out = new(bool) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousContainerDatabaseSpec. +func (in *AutonomousContainerDatabaseSpec) DeepCopy() *AutonomousContainerDatabaseSpec { + if in == nil { + return nil + } + out := new(AutonomousContainerDatabaseSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousContainerDatabaseStatus) DeepCopyInto(out *AutonomousContainerDatabaseStatus) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousContainerDatabaseStatus. +func (in *AutonomousContainerDatabaseStatus) DeepCopy() *AutonomousContainerDatabaseStatus { + if in == nil { + return nil + } + out := new(AutonomousContainerDatabaseStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousDatabase) DeepCopyInto(out *AutonomousDatabase) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + in.Spec.DeepCopyInto(&out.Spec) + in.Status.DeepCopyInto(&out.Status) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabase. +func (in *AutonomousDatabase) DeepCopy() *AutonomousDatabase { + if in == nil { + return nil + } + out := new(AutonomousDatabase) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *AutonomousDatabase) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousDatabaseBackup) DeepCopyInto(out *AutonomousDatabaseBackup) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + in.Spec.DeepCopyInto(&out.Spec) + out.Status = in.Status +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseBackup. +func (in *AutonomousDatabaseBackup) DeepCopy() *AutonomousDatabaseBackup { + if in == nil { + return nil + } + out := new(AutonomousDatabaseBackup) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *AutonomousDatabaseBackup) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousDatabaseBackupList) DeepCopyInto(out *AutonomousDatabaseBackupList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]AutonomousDatabaseBackup, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseBackupList. +func (in *AutonomousDatabaseBackupList) DeepCopy() *AutonomousDatabaseBackupList { + if in == nil { + return nil + } + out := new(AutonomousDatabaseBackupList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *AutonomousDatabaseBackupList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousDatabaseBackupSpec) DeepCopyInto(out *AutonomousDatabaseBackupSpec) { + *out = *in + in.Target.DeepCopyInto(&out.Target) + if in.DisplayName != nil { + in, out := &in.DisplayName, &out.DisplayName + *out = new(string) + **out = **in + } + if in.AutonomousDatabaseBackupOCID != nil { + in, out := &in.AutonomousDatabaseBackupOCID, &out.AutonomousDatabaseBackupOCID + *out = new(string) + **out = **in + } + if in.IsLongTermBackup != nil { + in, out := &in.IsLongTermBackup, &out.IsLongTermBackup + *out = new(bool) + **out = **in + } + if in.RetentionPeriodInDays != nil { + in, out := &in.RetentionPeriodInDays, &out.RetentionPeriodInDays + *out = new(int) + **out = **in + } + in.OCIConfig.DeepCopyInto(&out.OCIConfig) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseBackupSpec. +func (in *AutonomousDatabaseBackupSpec) DeepCopy() *AutonomousDatabaseBackupSpec { + if in == nil { + return nil + } + out := new(AutonomousDatabaseBackupSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousDatabaseBackupStatus) DeepCopyInto(out *AutonomousDatabaseBackupStatus) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseBackupStatus. +func (in *AutonomousDatabaseBackupStatus) DeepCopy() *AutonomousDatabaseBackupStatus { + if in == nil { + return nil + } + out := new(AutonomousDatabaseBackupStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousDatabaseBase) DeepCopyInto(out *AutonomousDatabaseBase) { + *out = *in + if in.CompartmentId != nil { + in, out := &in.CompartmentId, &out.CompartmentId + *out = new(string) + **out = **in + } + in.AutonomousContainerDatabase.DeepCopyInto(&out.AutonomousContainerDatabase) + if in.DisplayName != nil { + in, out := &in.DisplayName, &out.DisplayName + *out = new(string) + **out = **in + } + if in.DbName != nil { + in, out := &in.DbName, &out.DbName + *out = new(string) + **out = **in + } + if in.DbVersion != nil { + in, out := &in.DbVersion, &out.DbVersion + *out = new(string) + **out = **in + } + if in.DataStorageSizeInTBs != nil { + in, out := &in.DataStorageSizeInTBs, &out.DataStorageSizeInTBs + *out = new(int) + **out = **in + } + if in.CpuCoreCount != nil { + in, out := &in.CpuCoreCount, &out.CpuCoreCount + *out = new(int) + **out = **in + } + if in.ComputeCount != nil { + in, out := &in.ComputeCount, &out.ComputeCount + *out = new(float32) + **out = **in + } + if in.OcpuCount != nil { + in, out := &in.OcpuCount, &out.OcpuCount + *out = new(float32) + **out = **in + } + in.AdminPassword.DeepCopyInto(&out.AdminPassword) + if in.IsAutoScalingEnabled != nil { + in, out := &in.IsAutoScalingEnabled, &out.IsAutoScalingEnabled + *out = new(bool) + **out = **in + } + if in.IsDedicated != nil { + in, out := &in.IsDedicated, &out.IsDedicated + *out = new(bool) + **out = **in + } + if in.IsFreeTier != nil { + in, out := &in.IsFreeTier, &out.IsFreeTier + *out = new(bool) + **out = **in + } + if in.IsAccessControlEnabled != nil { + in, out := &in.IsAccessControlEnabled, &out.IsAccessControlEnabled + *out = new(bool) + **out = **in + } + if in.WhitelistedIps != nil { + in, out := &in.WhitelistedIps, &out.WhitelistedIps + *out = make([]string, len(*in)) + copy(*out, *in) + } + if in.SubnetId != nil { + in, out := &in.SubnetId, &out.SubnetId + *out = new(string) + **out = **in + } + if in.NsgIds != nil { + in, out := &in.NsgIds, &out.NsgIds + *out = make([]string, len(*in)) + copy(*out, *in) + } + if in.PrivateEndpointLabel != nil { + in, out := &in.PrivateEndpointLabel, &out.PrivateEndpointLabel + *out = new(string) + **out = **in + } + if in.IsMtlsConnectionRequired != nil { + in, out := &in.IsMtlsConnectionRequired, &out.IsMtlsConnectionRequired + *out = new(bool) + **out = **in + } + if in.FreeformTags != nil { + in, out := &in.FreeformTags, &out.FreeformTags + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseBase. +func (in *AutonomousDatabaseBase) DeepCopy() *AutonomousDatabaseBase { + if in == nil { + return nil + } + out := new(AutonomousDatabaseBase) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousDatabaseClone) DeepCopyInto(out *AutonomousDatabaseClone) { + *out = *in + in.AutonomousDatabaseBase.DeepCopyInto(&out.AutonomousDatabaseBase) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseClone. +func (in *AutonomousDatabaseClone) DeepCopy() *AutonomousDatabaseClone { + if in == nil { + return nil + } + out := new(AutonomousDatabaseClone) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousDatabaseDetails) DeepCopyInto(out *AutonomousDatabaseDetails) { + *out = *in + in.AutonomousDatabaseBase.DeepCopyInto(&out.AutonomousDatabaseBase) + if in.Id != nil { + in, out := &in.Id, &out.Id + *out = new(string) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseDetails. +func (in *AutonomousDatabaseDetails) DeepCopy() *AutonomousDatabaseDetails { + if in == nil { + return nil + } + out := new(AutonomousDatabaseDetails) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousDatabaseList) DeepCopyInto(out *AutonomousDatabaseList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]AutonomousDatabase, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseList. +func (in *AutonomousDatabaseList) DeepCopy() *AutonomousDatabaseList { + if in == nil { + return nil + } + out := new(AutonomousDatabaseList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *AutonomousDatabaseList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousDatabaseRestore) DeepCopyInto(out *AutonomousDatabaseRestore) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + in.Spec.DeepCopyInto(&out.Spec) + out.Status = in.Status +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseRestore. +func (in *AutonomousDatabaseRestore) DeepCopy() *AutonomousDatabaseRestore { + if in == nil { + return nil + } + out := new(AutonomousDatabaseRestore) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *AutonomousDatabaseRestore) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousDatabaseRestoreList) DeepCopyInto(out *AutonomousDatabaseRestoreList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]AutonomousDatabaseRestore, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseRestoreList. +func (in *AutonomousDatabaseRestoreList) DeepCopy() *AutonomousDatabaseRestoreList { + if in == nil { + return nil + } + out := new(AutonomousDatabaseRestoreList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *AutonomousDatabaseRestoreList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousDatabaseRestoreSpec) DeepCopyInto(out *AutonomousDatabaseRestoreSpec) { + *out = *in + in.Target.DeepCopyInto(&out.Target) + in.Source.DeepCopyInto(&out.Source) + in.OCIConfig.DeepCopyInto(&out.OCIConfig) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseRestoreSpec. +func (in *AutonomousDatabaseRestoreSpec) DeepCopy() *AutonomousDatabaseRestoreSpec { + if in == nil { + return nil + } + out := new(AutonomousDatabaseRestoreSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousDatabaseRestoreStatus) DeepCopyInto(out *AutonomousDatabaseRestoreStatus) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseRestoreStatus. +func (in *AutonomousDatabaseRestoreStatus) DeepCopy() *AutonomousDatabaseRestoreStatus { + if in == nil { + return nil + } + out := new(AutonomousDatabaseRestoreStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousDatabaseSpec) DeepCopyInto(out *AutonomousDatabaseSpec) { + *out = *in + in.Details.DeepCopyInto(&out.Details) + in.Clone.DeepCopyInto(&out.Clone) + in.Wallet.DeepCopyInto(&out.Wallet) + in.OciConfig.DeepCopyInto(&out.OciConfig) + if in.HardLink != nil { + in, out := &in.HardLink, &out.HardLink + *out = new(bool) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseSpec. +func (in *AutonomousDatabaseSpec) DeepCopy() *AutonomousDatabaseSpec { + if in == nil { + return nil + } + out := new(AutonomousDatabaseSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *AutonomousDatabaseStatus) DeepCopyInto(out *AutonomousDatabaseStatus) { + *out = *in + if in.AllConnectionStrings != nil { + in, out := &in.AllConnectionStrings, &out.AllConnectionStrings + *out = make([]ConnectionStringProfile, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } + if in.Conditions != nil { + in, out := &in.Conditions, &out.Conditions + *out = make([]v1.Condition, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AutonomousDatabaseStatus. +func (in *AutonomousDatabaseStatus) DeepCopy() *AutonomousDatabaseStatus { + if in == nil { + return nil + } + out := new(AutonomousDatabaseStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *Backupconfig) DeepCopyInto(out *Backupconfig) { + *out = *in + if in.AutoBackupEnabled != nil { + in, out := &in.AutoBackupEnabled, &out.AutoBackupEnabled + *out = new(bool) + **out = **in + } + if in.RecoveryWindowsInDays != nil { + in, out := &in.RecoveryWindowsInDays, &out.RecoveryWindowsInDays + *out = new(int) + **out = **in + } + if in.AutoBackupWindow != nil { + in, out := &in.AutoBackupWindow, &out.AutoBackupWindow + *out = new(string) + **out = **in + } + if in.BackupDestinationDetails != nil { + in, out := &in.BackupDestinationDetails, &out.BackupDestinationDetails + *out = new(string) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Backupconfig. +func (in *Backupconfig) DeepCopy() *Backupconfig { + if in == nil { + return nil + } + out := new(Backupconfig) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CDB) DeepCopyInto(out *CDB) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + in.Spec.DeepCopyInto(&out.Spec) + out.Status = in.Status +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDB. +func (in *CDB) DeepCopy() *CDB { + if in == nil { + return nil + } + out := new(CDB) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *CDB) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CDBAdminPassword) DeepCopyInto(out *CDBAdminPassword) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDBAdminPassword. +func (in *CDBAdminPassword) DeepCopy() *CDBAdminPassword { + if in == nil { + return nil + } + out := new(CDBAdminPassword) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CDBAdminUser) DeepCopyInto(out *CDBAdminUser) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDBAdminUser. +func (in *CDBAdminUser) DeepCopy() *CDBAdminUser { + if in == nil { + return nil + } + out := new(CDBAdminUser) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CDBList) DeepCopyInto(out *CDBList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]CDB, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDBList. +func (in *CDBList) DeepCopy() *CDBList { + if in == nil { + return nil + } + out := new(CDBList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *CDBList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CDBPRIVKEY) DeepCopyInto(out *CDBPRIVKEY) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDBPRIVKEY. +func (in *CDBPRIVKEY) DeepCopy() *CDBPRIVKEY { + if in == nil { + return nil + } + out := new(CDBPRIVKEY) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CDBPUBKEY) DeepCopyInto(out *CDBPUBKEY) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDBPUBKEY. +func (in *CDBPUBKEY) DeepCopy() *CDBPUBKEY { + if in == nil { + return nil + } + out := new(CDBPUBKEY) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CDBSecret) DeepCopyInto(out *CDBSecret) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDBSecret. +func (in *CDBSecret) DeepCopy() *CDBSecret { + if in == nil { + return nil + } + out := new(CDBSecret) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CDBSpec) DeepCopyInto(out *CDBSpec) { + *out = *in + out.SysAdminPwd = in.SysAdminPwd + out.CDBAdminUser = in.CDBAdminUser + out.CDBAdminPwd = in.CDBAdminPwd + out.CDBTlsKey = in.CDBTlsKey + out.CDBTlsCrt = in.CDBTlsCrt + out.ORDSPwd = in.ORDSPwd + out.WebServerUser = in.WebServerUser + out.WebServerPwd = in.WebServerPwd + if in.NodeSelector != nil { + in, out := &in.NodeSelector, &out.NodeSelector + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + out.CDBPubKey = in.CDBPubKey + out.CDBPriKey = in.CDBPriKey +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDBSpec. +func (in *CDBSpec) DeepCopy() *CDBSpec { + if in == nil { + return nil + } + out := new(CDBSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CDBStatus) DeepCopyInto(out *CDBStatus) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDBStatus. +func (in *CDBStatus) DeepCopy() *CDBStatus { + if in == nil { + return nil + } + out := new(CDBStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CDBSysAdminPassword) DeepCopyInto(out *CDBSysAdminPassword) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDBSysAdminPassword. +func (in *CDBSysAdminPassword) DeepCopy() *CDBSysAdminPassword { + if in == nil { + return nil + } + out := new(CDBSysAdminPassword) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CDBTLSCRT) DeepCopyInto(out *CDBTLSCRT) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDBTLSCRT. +func (in *CDBTLSCRT) DeepCopy() *CDBTLSCRT { + if in == nil { + return nil + } + out := new(CDBTLSCRT) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CDBTLSKEY) DeepCopyInto(out *CDBTLSKEY) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDBTLSKEY. +func (in *CDBTLSKEY) DeepCopy() *CDBTLSKEY { + if in == nil { + return nil + } + out := new(CDBTLSKEY) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CatalogSpec) DeepCopyInto(out *CatalogSpec) { + *out = *in + if in.EnvVars != nil { + in, out := &in.EnvVars, &out.EnvVars + *out = make([]EnvironmentVariable, len(*in)) + copy(*out, *in) + } + if in.Resources != nil { + in, out := &in.Resources, &out.Resources + *out = new(corev1.ResourceRequirements) + (*in).DeepCopyInto(*out) + } + if in.NodeSelector != nil { + in, out := &in.NodeSelector, &out.NodeSelector + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + if in.PvAnnotations != nil { + in, out := &in.PvAnnotations, &out.PvAnnotations + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + if in.PvMatchLabels != nil { + in, out := &in.PvMatchLabels, &out.PvMatchLabels + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + if in.ImagePulllPolicy != nil { + in, out := &in.ImagePulllPolicy, &out.ImagePulllPolicy + *out = new(corev1.PullPolicy) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CatalogSpec. +func (in *CatalogSpec) DeepCopy() *CatalogSpec { + if in == nil { + return nil + } + out := new(CatalogSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CertificateSecret) DeepCopyInto(out *CertificateSecret) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CertificateSecret. +func (in *CertificateSecret) DeepCopy() *CertificateSecret { + if in == nil { + return nil + } + out := new(CertificateSecret) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *ConnectionStringProfile) DeepCopyInto(out *ConnectionStringProfile) { + *out = *in + if in.ConnectionStrings != nil { + in, out := &in.ConnectionStrings, &out.ConnectionStrings + *out = make([]ConnectionStringSpec, len(*in)) + copy(*out, *in) + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ConnectionStringProfile. +func (in *ConnectionStringProfile) DeepCopy() *ConnectionStringProfile { + if in == nil { + return nil + } + out := new(ConnectionStringProfile) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *ConnectionStringSpec) DeepCopyInto(out *ConnectionStringSpec) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ConnectionStringSpec. +func (in *ConnectionStringSpec) DeepCopy() *ConnectionStringSpec { + if in == nil { + return nil + } + out := new(ConnectionStringSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DBWalletSecret) DeepCopyInto(out *DBWalletSecret) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DBWalletSecret. +func (in *DBWalletSecret) DeepCopy() *DBWalletSecret { + if in == nil { + return nil + } + out := new(DBWalletSecret) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DataguardBroker) DeepCopyInto(out *DataguardBroker) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + in.Spec.DeepCopyInto(&out.Spec) + in.Status.DeepCopyInto(&out.Status) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DataguardBroker. +func (in *DataguardBroker) DeepCopy() *DataguardBroker { + if in == nil { + return nil + } + out := new(DataguardBroker) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *DataguardBroker) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DataguardBrokerList) DeepCopyInto(out *DataguardBrokerList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]DataguardBroker, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DataguardBrokerList. +func (in *DataguardBrokerList) DeepCopy() *DataguardBrokerList { + if in == nil { + return nil + } + out := new(DataguardBrokerList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *DataguardBrokerList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DataguardBrokerSpec) DeepCopyInto(out *DataguardBrokerSpec) { + *out = *in + if in.StandbyDatabaseRefs != nil { + in, out := &in.StandbyDatabaseRefs, &out.StandbyDatabaseRefs + *out = make([]string, len(*in)) + copy(*out, *in) + } + if in.ServiceAnnotations != nil { + in, out := &in.ServiceAnnotations, &out.ServiceAnnotations + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + if in.NodeSelector != nil { + in, out := &in.NodeSelector, &out.NodeSelector + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DataguardBrokerSpec. +func (in *DataguardBrokerSpec) DeepCopy() *DataguardBrokerSpec { + if in == nil { + return nil + } + out := new(DataguardBrokerSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DataguardBrokerStatus) DeepCopyInto(out *DataguardBrokerStatus) { + *out = *in + if in.DatabasesInDataguardConfig != nil { + in, out := &in.DatabasesInDataguardConfig, &out.DatabasesInDataguardConfig + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DataguardBrokerStatus. +func (in *DataguardBrokerStatus) DeepCopy() *DataguardBrokerStatus { + if in == nil { + return nil + } + out := new(DataguardBrokerStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DbCloneConfig) DeepCopyInto(out *DbCloneConfig) { + *out = *in + if in.SshPublicKeys != nil { + in, out := &in.SshPublicKeys, &out.SshPublicKeys + *out = make([]string, len(*in)) + copy(*out, *in) + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DbCloneConfig. +func (in *DbCloneConfig) DeepCopy() *DbCloneConfig { + if in == nil { + return nil + } + out := new(DbCloneConfig) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DbCloneStatus) DeepCopyInto(out *DbCloneStatus) { + *out = *in + if in.Id != nil { + in, out := &in.Id, &out.Id + *out = new(string) + **out = **in + } + if in.SshPublicKeys != nil { + in, out := &in.SshPublicKeys, &out.SshPublicKeys + *out = make([]string, len(*in)) + copy(*out, *in) + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DbCloneStatus. +func (in *DbCloneStatus) DeepCopy() *DbCloneStatus { + if in == nil { + return nil + } + out := new(DbCloneStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DbStatus) DeepCopyInto(out *DbStatus) { + *out = *in + if in.Id != nil { + in, out := &in.Id, &out.Id + *out = new(string) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DbStatus. +func (in *DbStatus) DeepCopy() *DbStatus { + if in == nil { + return nil + } + out := new(DbStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DbSystemDetails) DeepCopyInto(out *DbSystemDetails) { + *out = *in + if in.SshPublicKeys != nil { + in, out := &in.SshPublicKeys, &out.SshPublicKeys + *out = make([]string, len(*in)) + copy(*out, *in) + } + if in.FaultDomains != nil { + in, out := &in.FaultDomains, &out.FaultDomains + *out = make([]string, len(*in)) + copy(*out, *in) + } + if in.NodeCount != nil { + in, out := &in.NodeCount, &out.NodeCount + *out = new(int) + **out = **in + } + if in.Tags != nil { + in, out := &in.Tags, &out.Tags + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + in.DbBackupConfig.DeepCopyInto(&out.DbBackupConfig) + out.KMSConfig = in.KMSConfig +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DbSystemDetails. +func (in *DbSystemDetails) DeepCopy() *DbSystemDetails { + if in == nil { + return nil + } + out := new(DbSystemDetails) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DbWorkrequests) DeepCopyInto(out *DbWorkrequests) { + *out = *in + if in.OperationType != nil { + in, out := &in.OperationType, &out.OperationType + *out = new(string) + **out = **in + } + if in.OperationId != nil { + in, out := &in.OperationId, &out.OperationId + *out = new(string) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DbWorkrequests. +func (in *DbWorkrequests) DeepCopy() *DbWorkrequests { + if in == nil { + return nil + } + out := new(DbWorkrequests) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DbcsSystem) DeepCopyInto(out *DbcsSystem) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + in.Spec.DeepCopyInto(&out.Spec) + in.Status.DeepCopyInto(&out.Status) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DbcsSystem. +func (in *DbcsSystem) DeepCopy() *DbcsSystem { + if in == nil { + return nil + } + out := new(DbcsSystem) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *DbcsSystem) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DbcsSystemList) DeepCopyInto(out *DbcsSystemList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]DbcsSystem, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DbcsSystemList. +func (in *DbcsSystemList) DeepCopy() *DbcsSystemList { + if in == nil { + return nil + } + out := new(DbcsSystemList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *DbcsSystemList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DbcsSystemSpec) DeepCopyInto(out *DbcsSystemSpec) { + *out = *in + in.DbSystem.DeepCopyInto(&out.DbSystem) + if in.Id != nil { + in, out := &in.Id, &out.Id + *out = new(string) + **out = **in + } + if in.OCIConfigMap != nil { + in, out := &in.OCIConfigMap, &out.OCIConfigMap + *out = new(string) + **out = **in + } + if in.OCISecret != nil { + in, out := &in.OCISecret, &out.OCISecret + *out = new(string) + **out = **in + } + if in.DbClone != nil { + in, out := &in.DbClone, &out.DbClone + *out = new(DbCloneConfig) + (*in).DeepCopyInto(*out) + } + if in.PdbConfigs != nil { + in, out := &in.PdbConfigs, &out.PdbConfigs + *out = make([]PDBConfig, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } + if in.DbBackupId != nil { + in, out := &in.DbBackupId, &out.DbBackupId + *out = new(string) + **out = **in + } + if in.DatabaseId != nil { + in, out := &in.DatabaseId, &out.DatabaseId + *out = new(string) + **out = **in + } + out.KMSConfig = in.KMSConfig +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DbcsSystemSpec. +func (in *DbcsSystemSpec) DeepCopy() *DbcsSystemSpec { + if in == nil { + return nil + } + out := new(DbcsSystemSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DbcsSystemStatus) DeepCopyInto(out *DbcsSystemStatus) { + *out = *in + if in.Id != nil { + in, out := &in.Id, &out.Id + *out = new(string) + **out = **in + } + if in.DataStoragePercentage != nil { + in, out := &in.DataStoragePercentage, &out.DataStoragePercentage + *out = new(int) + **out = **in + } + if in.DataStorageSizeInGBs != nil { + in, out := &in.DataStorageSizeInGBs, &out.DataStorageSizeInGBs + *out = new(int) + **out = **in + } + if in.RecoStorageSizeInGB != nil { + in, out := &in.RecoStorageSizeInGB, &out.RecoStorageSizeInGB + *out = new(int) + **out = **in + } + if in.Shape != nil { + in, out := &in.Shape, &out.Shape + *out = new(string) + **out = **in + } + if in.DbInfo != nil { + in, out := &in.DbInfo, &out.DbInfo + *out = make([]DbStatus, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } + in.Network.DeepCopyInto(&out.Network) + if in.WorkRequests != nil { + in, out := &in.WorkRequests, &out.WorkRequests + *out = make([]DbWorkrequests, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } + out.KMSDetailsStatus = in.KMSDetailsStatus + in.DbCloneStatus.DeepCopyInto(&out.DbCloneStatus) + if in.PdbDetailsStatus != nil { + in, out := &in.PdbDetailsStatus, &out.PdbDetailsStatus + *out = make([]PDBDetailsStatus, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DbcsSystemStatus. +func (in *DbcsSystemStatus) DeepCopy() *DbcsSystemStatus { + if in == nil { + return nil + } + out := new(DbcsSystemStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *EnvironmentVariable) DeepCopyInto(out *EnvironmentVariable) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new EnvironmentVariable. +func (in *EnvironmentVariable) DeepCopy() *EnvironmentVariable { + if in == nil { + return nil + } + out := new(EnvironmentVariable) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *GlobalSettings) DeepCopyInto(out *GlobalSettings) { + *out = *in + if in.CacheMetadataEnabled != nil { + in, out := &in.CacheMetadataEnabled, &out.CacheMetadataEnabled + *out = new(bool) + **out = **in + } + if in.CacheMetadataGraphQLExpireAfterAccess != nil { + in, out := &in.CacheMetadataGraphQLExpireAfterAccess, &out.CacheMetadataGraphQLExpireAfterAccess + *out = new(timex.Duration) + **out = **in + } + if in.CacheMetadataGraphQLExpireAfterWrite != nil { + in, out := &in.CacheMetadataGraphQLExpireAfterWrite, &out.CacheMetadataGraphQLExpireAfterWrite + *out = new(timex.Duration) + **out = **in + } + if in.CacheMetadataTimeout != nil { + in, out := &in.CacheMetadataTimeout, &out.CacheMetadataTimeout + *out = new(timex.Duration) + **out = **in + } + if in.CacheMetadataJWKSEnabled != nil { + in, out := &in.CacheMetadataJWKSEnabled, &out.CacheMetadataJWKSEnabled + *out = new(bool) + **out = **in + } + if in.CacheMetadataJWKSInitialCapacity != nil { + in, out := &in.CacheMetadataJWKSInitialCapacity, &out.CacheMetadataJWKSInitialCapacity + *out = new(int32) + **out = **in + } + if in.CacheMetadataJWKSMaximumSize != nil { + in, out := &in.CacheMetadataJWKSMaximumSize, &out.CacheMetadataJWKSMaximumSize + *out = new(int32) + **out = **in + } + if in.CacheMetadataJWKSExpireAfterAccess != nil { + in, out := &in.CacheMetadataJWKSExpireAfterAccess, &out.CacheMetadataJWKSExpireAfterAccess + *out = new(timex.Duration) + **out = **in + } + if in.CacheMetadataJWKSExpireAfterWrite != nil { + in, out := &in.CacheMetadataJWKSExpireAfterWrite, &out.CacheMetadataJWKSExpireAfterWrite + *out = new(timex.Duration) + **out = **in + } + if in.DatabaseAPIEnabled != nil { + in, out := &in.DatabaseAPIEnabled, &out.DatabaseAPIEnabled + *out = new(bool) + **out = **in + } + if in.DatabaseAPIManagementServicesDisabled != nil { + in, out := &in.DatabaseAPIManagementServicesDisabled, &out.DatabaseAPIManagementServicesDisabled + *out = new(bool) + **out = **in + } + if in.DBInvalidPoolTimeout != nil { + in, out := &in.DBInvalidPoolTimeout, &out.DBInvalidPoolTimeout + *out = new(timex.Duration) + **out = **in + } + if in.FeatureGraphQLMaxNestingDepth != nil { + in, out := &in.FeatureGraphQLMaxNestingDepth, &out.FeatureGraphQLMaxNestingDepth + *out = new(int32) + **out = **in + } + if in.SecurityCredentialsAttempts != nil { + in, out := &in.SecurityCredentialsAttempts, &out.SecurityCredentialsAttempts + *out = new(int32) + **out = **in + } + if in.SecurityCredentialsLockTime != nil { + in, out := &in.SecurityCredentialsLockTime, &out.SecurityCredentialsLockTime + *out = new(timex.Duration) + **out = **in + } + if in.StandaloneHTTPPort != nil { + in, out := &in.StandaloneHTTPPort, &out.StandaloneHTTPPort + *out = new(int32) + **out = **in + } + if in.StandaloneHTTPSPort != nil { + in, out := &in.StandaloneHTTPSPort, &out.StandaloneHTTPSPort + *out = new(int32) + **out = **in + } + if in.StandaloneStopTimeout != nil { + in, out := &in.StandaloneStopTimeout, &out.StandaloneStopTimeout + *out = new(timex.Duration) + **out = **in + } + if in.DebugPrintDebugToScreen != nil { + in, out := &in.DebugPrintDebugToScreen, &out.DebugPrintDebugToScreen + *out = new(bool) + **out = **in + } + if in.ICAPPort != nil { + in, out := &in.ICAPPort, &out.ICAPPort + *out = new(int32) + **out = **in + } + if in.ICAPSecurePort != nil { + in, out := &in.ICAPSecurePort, &out.ICAPSecurePort + *out = new(int32) + **out = **in + } + if in.MongoPort != nil { + in, out := &in.MongoPort, &out.MongoPort + *out = new(int32) + **out = **in + } + if in.MongoIdleTimeout != nil { + in, out := &in.MongoIdleTimeout, &out.MongoIdleTimeout + *out = new(timex.Duration) + **out = **in + } + if in.MongoOpTimeout != nil { + in, out := &in.MongoOpTimeout, &out.MongoOpTimeout + *out = new(timex.Duration) + **out = **in + } + if in.SecurityDisableDefaultExclusionList != nil { + in, out := &in.SecurityDisableDefaultExclusionList, &out.SecurityDisableDefaultExclusionList + *out = new(bool) + **out = **in + } + if in.SecurityMaxEntries != nil { + in, out := &in.SecurityMaxEntries, &out.SecurityMaxEntries + *out = new(int32) + **out = **in + } + if in.SecurityVerifySSL != nil { + in, out := &in.SecurityVerifySSL, &out.SecurityVerifySSL + *out = new(bool) + **out = **in + } + if in.CertSecret != nil { + in, out := &in.CertSecret, &out.CertSecret + *out = new(CertificateSecret) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new GlobalSettings. +func (in *GlobalSettings) DeepCopy() *GlobalSettings { + if in == nil { + return nil + } + out := new(GlobalSettings) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *GsmServiceSpec) DeepCopyInto(out *GsmServiceSpec) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new GsmServiceSpec. +func (in *GsmServiceSpec) DeepCopy() *GsmServiceSpec { + if in == nil { + return nil + } + out := new(GsmServiceSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *GsmShardDetails) DeepCopyInto(out *GsmShardDetails) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new GsmShardDetails. +func (in *GsmShardDetails) DeepCopy() *GsmShardDetails { + if in == nil { + return nil + } + out := new(GsmShardDetails) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *GsmShardGroupSpec) DeepCopyInto(out *GsmShardGroupSpec) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new GsmShardGroupSpec. +func (in *GsmShardGroupSpec) DeepCopy() *GsmShardGroupSpec { + if in == nil { + return nil + } + out := new(GsmShardGroupSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *GsmShardSpaceSpec) DeepCopyInto(out *GsmShardSpaceSpec) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new GsmShardSpaceSpec. +func (in *GsmShardSpaceSpec) DeepCopy() *GsmShardSpaceSpec { + if in == nil { + return nil + } + out := new(GsmShardSpaceSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *GsmSpec) DeepCopyInto(out *GsmSpec) { + *out = *in + if in.EnvVars != nil { + in, out := &in.EnvVars, &out.EnvVars + *out = make([]EnvironmentVariable, len(*in)) + copy(*out, *in) + } + if in.Resources != nil { + in, out := &in.Resources, &out.Resources + *out = new(corev1.ResourceRequirements) + (*in).DeepCopyInto(*out) + } + if in.NodeSelector != nil { + in, out := &in.NodeSelector, &out.NodeSelector + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + if in.PvAnnotations != nil { + in, out := &in.PvAnnotations, &out.PvAnnotations + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + if in.PvMatchLabels != nil { + in, out := &in.PvMatchLabels, &out.PvMatchLabels + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + if in.ImagePulllPolicy != nil { + in, out := &in.ImagePulllPolicy, &out.ImagePulllPolicy + *out = new(corev1.PullPolicy) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new GsmSpec. +func (in *GsmSpec) DeepCopy() *GsmSpec { + if in == nil { + return nil + } + out := new(GsmSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *GsmStatus) DeepCopyInto(out *GsmStatus) { + *out = *in + if in.Shards != nil { + in, out := &in.Shards, &out.Shards + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + if in.Details != nil { + in, out := &in.Details, &out.Details + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new GsmStatus. +func (in *GsmStatus) DeepCopy() *GsmStatus { + if in == nil { + return nil + } + out := new(GsmStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *GsmStatusDetails) DeepCopyInto(out *GsmStatusDetails) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new GsmStatusDetails. +func (in *GsmStatusDetails) DeepCopy() *GsmStatusDetails { + if in == nil { + return nil + } + out := new(GsmStatusDetails) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *K8sADBBackupSpec) DeepCopyInto(out *K8sADBBackupSpec) { + *out = *in + if in.Name != nil { + in, out := &in.Name, &out.Name + *out = new(string) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new K8sADBBackupSpec. +func (in *K8sADBBackupSpec) DeepCopy() *K8sADBBackupSpec { + if in == nil { + return nil + } + out := new(K8sADBBackupSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *K8sAcdSpec) DeepCopyInto(out *K8sAcdSpec) { + *out = *in + if in.Name != nil { + in, out := &in.Name, &out.Name + *out = new(string) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new K8sAcdSpec. +func (in *K8sAcdSpec) DeepCopy() *K8sAcdSpec { + if in == nil { + return nil + } + out := new(K8sAcdSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *K8sAdbSpec) DeepCopyInto(out *K8sAdbSpec) { + *out = *in + if in.Name != nil { + in, out := &in.Name, &out.Name + *out = new(string) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new K8sAdbSpec. +func (in *K8sAdbSpec) DeepCopy() *K8sAdbSpec { + if in == nil { + return nil + } + out := new(K8sAdbSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *K8sSecretSpec) DeepCopyInto(out *K8sSecretSpec) { + *out = *in + if in.Name != nil { + in, out := &in.Name, &out.Name + *out = new(string) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new K8sSecretSpec. +func (in *K8sSecretSpec) DeepCopy() *K8sSecretSpec { + if in == nil { + return nil + } + out := new(K8sSecretSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *KMSConfig) DeepCopyInto(out *KMSConfig) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new KMSConfig. +func (in *KMSConfig) DeepCopy() *KMSConfig { + if in == nil { + return nil + } + out := new(KMSConfig) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *KMSDetailsStatus) DeepCopyInto(out *KMSDetailsStatus) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new KMSDetailsStatus. +func (in *KMSDetailsStatus) DeepCopy() *KMSDetailsStatus { + if in == nil { + return nil + } + out := new(KMSDetailsStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *LREST) DeepCopyInto(out *LREST) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + in.Spec.DeepCopyInto(&out.Spec) + out.Status = in.Status +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new LREST. +func (in *LREST) DeepCopy() *LREST { + if in == nil { + return nil + } + out := new(LREST) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *LREST) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *LRESTAdminPassword) DeepCopyInto(out *LRESTAdminPassword) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new LRESTAdminPassword. +func (in *LRESTAdminPassword) DeepCopy() *LRESTAdminPassword { + if in == nil { + return nil + } + out := new(LRESTAdminPassword) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *LRESTAdminUser) DeepCopyInto(out *LRESTAdminUser) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new LRESTAdminUser. +func (in *LRESTAdminUser) DeepCopy() *LRESTAdminUser { + if in == nil { + return nil + } + out := new(LRESTAdminUser) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *LRESTList) DeepCopyInto(out *LRESTList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]LREST, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new LRESTList. +func (in *LRESTList) DeepCopy() *LRESTList { + if in == nil { + return nil + } + out := new(LRESTList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *LRESTList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *LRESTPRVKEY) DeepCopyInto(out *LRESTPRVKEY) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new LRESTPRVKEY. +func (in *LRESTPRVKEY) DeepCopy() *LRESTPRVKEY { + if in == nil { + return nil + } + out := new(LRESTPRVKEY) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *LRESTPUBKEY) DeepCopyInto(out *LRESTPUBKEY) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new LRESTPUBKEY. +func (in *LRESTPUBKEY) DeepCopy() *LRESTPUBKEY { + if in == nil { + return nil + } + out := new(LRESTPUBKEY) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *LRESTPassword) DeepCopyInto(out *LRESTPassword) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new LRESTPassword. +func (in *LRESTPassword) DeepCopy() *LRESTPassword { + if in == nil { + return nil + } + out := new(LRESTPassword) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *LRESTSecret) DeepCopyInto(out *LRESTSecret) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new LRESTSecret. +func (in *LRESTSecret) DeepCopy() *LRESTSecret { + if in == nil { + return nil + } + out := new(LRESTSecret) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *LRESTSpec) DeepCopyInto(out *LRESTSpec) { + *out = *in + out.SysAdminPwd = in.SysAdminPwd + out.LRESTAdminUser = in.LRESTAdminUser + out.LRESTAdminPwd = in.LRESTAdminPwd + out.LRESTTlsKey = in.LRESTTlsKey + out.LRESTTlsCrt = in.LRESTTlsCrt + out.LRESTPubKey = in.LRESTPubKey + out.LRESTPriKey = in.LRESTPriKey + out.LRESTPwd = in.LRESTPwd + out.WebLrestServerUser = in.WebLrestServerUser + out.WebLrestServerPwd = in.WebLrestServerPwd + if in.NodeSelector != nil { + in, out := &in.NodeSelector, &out.NodeSelector + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new LRESTSpec. +func (in *LRESTSpec) DeepCopy() *LRESTSpec { + if in == nil { + return nil + } + out := new(LRESTSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *LRESTStatus) DeepCopyInto(out *LRESTStatus) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new LRESTStatus. +func (in *LRESTStatus) DeepCopy() *LRESTStatus { + if in == nil { + return nil + } + out := new(LRESTStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *LRESTSysAdminPassword) DeepCopyInto(out *LRESTSysAdminPassword) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new LRESTSysAdminPassword. +func (in *LRESTSysAdminPassword) DeepCopy() *LRESTSysAdminPassword { + if in == nil { + return nil + } + out := new(LRESTSysAdminPassword) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *LRESTTLSCRT) DeepCopyInto(out *LRESTTLSCRT) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new LRESTTLSCRT. +func (in *LRESTTLSCRT) DeepCopy() *LRESTTLSCRT { + if in == nil { + return nil + } + out := new(LRESTTLSCRT) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *LRESTTLSKEY) DeepCopyInto(out *LRESTTLSKEY) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new LRESTTLSKEY. +func (in *LRESTTLSKEY) DeepCopy() *LRESTTLSKEY { + if in == nil { + return nil + } + out := new(LRESTTLSKEY) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *LRPDB) DeepCopyInto(out *LRPDB) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + in.Spec.DeepCopyInto(&out.Spec) + out.Status = in.Status +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new LRPDB. +func (in *LRPDB) DeepCopy() *LRPDB { + if in == nil { + return nil + } + out := new(LRPDB) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *LRPDB) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *LRPDBAdminName) DeepCopyInto(out *LRPDBAdminName) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new LRPDBAdminName. +func (in *LRPDBAdminName) DeepCopy() *LRPDBAdminName { + if in == nil { + return nil + } + out := new(LRPDBAdminName) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *LRPDBAdminPassword) DeepCopyInto(out *LRPDBAdminPassword) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new LRPDBAdminPassword. +func (in *LRPDBAdminPassword) DeepCopy() *LRPDBAdminPassword { + if in == nil { + return nil + } + out := new(LRPDBAdminPassword) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *LRPDBList) DeepCopyInto(out *LRPDBList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]LRPDB, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new LRPDBList. +func (in *LRPDBList) DeepCopy() *LRPDBList { + if in == nil { + return nil + } + out := new(LRPDBList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *LRPDBList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *LRPDBPRVKEY) DeepCopyInto(out *LRPDBPRVKEY) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new LRPDBPRVKEY. +func (in *LRPDBPRVKEY) DeepCopy() *LRPDBPRVKEY { + if in == nil { + return nil + } + out := new(LRPDBPRVKEY) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *LRPDBSecret) DeepCopyInto(out *LRPDBSecret) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new LRPDBSecret. +func (in *LRPDBSecret) DeepCopy() *LRPDBSecret { + if in == nil { + return nil + } + out := new(LRPDBSecret) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *LRPDBSpec) DeepCopyInto(out *LRPDBSpec) { + *out = *in + out.LRPDBTlsKey = in.LRPDBTlsKey + out.LRPDBTlsCrt = in.LRPDBTlsCrt + out.LRPDBTlsCat = in.LRPDBTlsCat + out.LRPDBPriKey = in.LRPDBPriKey + out.AdminName = in.AdminName + out.AdminPwd = in.AdminPwd + out.AdminpdbUser = in.AdminpdbUser + out.AdminpdbPass = in.AdminpdbPass + if in.ReuseTempFile != nil { + in, out := &in.ReuseTempFile, &out.ReuseTempFile + *out = new(bool) + **out = **in + } + if in.UnlimitedStorage != nil { + in, out := &in.UnlimitedStorage, &out.UnlimitedStorage + *out = new(bool) + **out = **in + } + if in.AsClone != nil { + in, out := &in.AsClone, &out.AsClone + *out = new(bool) + **out = **in + } + out.WebLrpdbServerUser = in.WebLrpdbServerUser + out.WebLrpdbServerPwd = in.WebLrpdbServerPwd + if in.LTDEImport != nil { + in, out := &in.LTDEImport, &out.LTDEImport + *out = new(bool) + **out = **in + } + if in.LTDEExport != nil { + in, out := &in.LTDEExport, &out.LTDEExport + *out = new(bool) + **out = **in + } + out.LTDEPassword = in.LTDEPassword + out.LTDESecret = in.LTDESecret + if in.GetScript != nil { + in, out := &in.GetScript, &out.GetScript + *out = new(bool) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new LRPDBSpec. +func (in *LRPDBSpec) DeepCopy() *LRPDBSpec { + if in == nil { + return nil + } + out := new(LRPDBSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *LRPDBStatus) DeepCopyInto(out *LRPDBStatus) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new LRPDBStatus. +func (in *LRPDBStatus) DeepCopy() *LRPDBStatus { + if in == nil { + return nil + } + out := new(LRPDBStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *LRPDBTLSCAT) DeepCopyInto(out *LRPDBTLSCAT) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new LRPDBTLSCAT. +func (in *LRPDBTLSCAT) DeepCopy() *LRPDBTLSCAT { + if in == nil { + return nil + } + out := new(LRPDBTLSCAT) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *LRPDBTLSCRT) DeepCopyInto(out *LRPDBTLSCRT) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new LRPDBTLSCRT. +func (in *LRPDBTLSCRT) DeepCopy() *LRPDBTLSCRT { + if in == nil { + return nil + } + out := new(LRPDBTLSCRT) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *LRPDBTLSKEY) DeepCopyInto(out *LRPDBTLSKEY) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new LRPDBTLSKEY. +func (in *LRPDBTLSKEY) DeepCopy() *LRPDBTLSKEY { + if in == nil { + return nil + } + out := new(LRPDBTLSKEY) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *LTDEPwd) DeepCopyInto(out *LTDEPwd) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new LTDEPwd. +func (in *LTDEPwd) DeepCopy() *LTDEPwd { + if in == nil { + return nil + } + out := new(LTDEPwd) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *LTDESecret) DeepCopyInto(out *LTDESecret) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new LTDESecret. +func (in *LTDESecret) DeepCopy() *LTDESecret { + if in == nil { + return nil + } + out := new(LTDESecret) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *ORDSPassword) DeepCopyInto(out *ORDSPassword) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ORDSPassword. +func (in *ORDSPassword) DeepCopy() *ORDSPassword { + if in == nil { + return nil + } + out := new(ORDSPassword) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *OciAcdSpec) DeepCopyInto(out *OciAcdSpec) { + *out = *in + if in.Id != nil { + in, out := &in.Id, &out.Id + *out = new(string) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new OciAcdSpec. +func (in *OciAcdSpec) DeepCopy() *OciAcdSpec { + if in == nil { + return nil + } + out := new(OciAcdSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *OciAdbSpec) DeepCopyInto(out *OciAdbSpec) { + *out = *in + if in.OCID != nil { + in, out := &in.OCID, &out.OCID + *out = new(string) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new OciAdbSpec. +func (in *OciAdbSpec) DeepCopy() *OciAdbSpec { + if in == nil { + return nil + } + out := new(OciAdbSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *OciConfigSpec) DeepCopyInto(out *OciConfigSpec) { + *out = *in + if in.ConfigMapName != nil { + in, out := &in.ConfigMapName, &out.ConfigMapName + *out = new(string) + **out = **in + } + if in.SecretName != nil { + in, out := &in.SecretName, &out.SecretName + *out = new(string) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new OciConfigSpec. +func (in *OciConfigSpec) DeepCopy() *OciConfigSpec { + if in == nil { + return nil + } + out := new(OciConfigSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *OciSecretSpec) DeepCopyInto(out *OciSecretSpec) { + *out = *in + if in.Id != nil { + in, out := &in.Id, &out.Id + *out = new(string) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new OciSecretSpec. +func (in *OciSecretSpec) DeepCopy() *OciSecretSpec { + if in == nil { + return nil + } + out := new(OciSecretSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *OracleRestDataService) DeepCopyInto(out *OracleRestDataService) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + in.Spec.DeepCopyInto(&out.Spec) + out.Status = in.Status +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new OracleRestDataService. +func (in *OracleRestDataService) DeepCopy() *OracleRestDataService { + if in == nil { + return nil + } + out := new(OracleRestDataService) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *OracleRestDataService) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *OracleRestDataServiceImage) DeepCopyInto(out *OracleRestDataServiceImage) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new OracleRestDataServiceImage. +func (in *OracleRestDataServiceImage) DeepCopy() *OracleRestDataServiceImage { + if in == nil { + return nil + } + out := new(OracleRestDataServiceImage) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *OracleRestDataServiceList) DeepCopyInto(out *OracleRestDataServiceList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]OracleRestDataService, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new OracleRestDataServiceList. +func (in *OracleRestDataServiceList) DeepCopy() *OracleRestDataServiceList { + if in == nil { + return nil + } + out := new(OracleRestDataServiceList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *OracleRestDataServiceList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *OracleRestDataServicePassword) DeepCopyInto(out *OracleRestDataServicePassword) { + *out = *in + if in.KeepSecret != nil { + in, out := &in.KeepSecret, &out.KeepSecret + *out = new(bool) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new OracleRestDataServicePassword. +func (in *OracleRestDataServicePassword) DeepCopy() *OracleRestDataServicePassword { + if in == nil { + return nil + } + out := new(OracleRestDataServicePassword) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *OracleRestDataServicePersistence) DeepCopyInto(out *OracleRestDataServicePersistence) { + *out = *in + if in.SetWritePermissions != nil { + in, out := &in.SetWritePermissions, &out.SetWritePermissions + *out = new(bool) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new OracleRestDataServicePersistence. +func (in *OracleRestDataServicePersistence) DeepCopy() *OracleRestDataServicePersistence { + if in == nil { + return nil + } + out := new(OracleRestDataServicePersistence) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *OracleRestDataServiceRestEnableSchemas) DeepCopyInto(out *OracleRestDataServiceRestEnableSchemas) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new OracleRestDataServiceRestEnableSchemas. +func (in *OracleRestDataServiceRestEnableSchemas) DeepCopy() *OracleRestDataServiceRestEnableSchemas { + if in == nil { + return nil + } + out := new(OracleRestDataServiceRestEnableSchemas) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *OracleRestDataServiceSpec) DeepCopyInto(out *OracleRestDataServiceSpec) { + *out = *in + if in.ServiceAnnotations != nil { + in, out := &in.ServiceAnnotations, &out.ServiceAnnotations + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + if in.NodeSelector != nil { + in, out := &in.NodeSelector, &out.NodeSelector + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + out.Image = in.Image + in.OrdsPassword.DeepCopyInto(&out.OrdsPassword) + in.AdminPassword.DeepCopyInto(&out.AdminPassword) + if in.RestEnableSchemas != nil { + in, out := &in.RestEnableSchemas, &out.RestEnableSchemas + *out = make([]OracleRestDataServiceRestEnableSchemas, len(*in)) + copy(*out, *in) + } + in.Persistence.DeepCopyInto(&out.Persistence) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new OracleRestDataServiceSpec. +func (in *OracleRestDataServiceSpec) DeepCopy() *OracleRestDataServiceSpec { + if in == nil { + return nil + } + out := new(OracleRestDataServiceSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *OracleRestDataServiceStatus) DeepCopyInto(out *OracleRestDataServiceStatus) { + *out = *in + out.Image = in.Image +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new OracleRestDataServiceStatus. +func (in *OracleRestDataServiceStatus) DeepCopy() *OracleRestDataServiceStatus { + if in == nil { + return nil + } + out := new(OracleRestDataServiceStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *OrdsSrvs) DeepCopyInto(out *OrdsSrvs) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + in.Spec.DeepCopyInto(&out.Spec) + in.Status.DeepCopyInto(&out.Status) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new OrdsSrvs. +func (in *OrdsSrvs) DeepCopy() *OrdsSrvs { + if in == nil { + return nil + } + out := new(OrdsSrvs) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *OrdsSrvs) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *OrdsSrvsList) DeepCopyInto(out *OrdsSrvsList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]OrdsSrvs, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new OrdsSrvsList. +func (in *OrdsSrvsList) DeepCopy() *OrdsSrvsList { + if in == nil { + return nil + } + out := new(OrdsSrvsList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *OrdsSrvsList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *OrdsSrvsSpec) DeepCopyInto(out *OrdsSrvsSpec) { + *out = *in + in.GlobalSettings.DeepCopyInto(&out.GlobalSettings) + out.EncPrivKey = in.EncPrivKey + if in.PoolSettings != nil { + in, out := &in.PoolSettings, &out.PoolSettings + *out = make([]*PoolSettings, len(*in)) + for i := range *in { + if (*in)[i] != nil { + in, out := &(*in)[i], &(*out)[i] + *out = new(PoolSettings) + (*in).DeepCopyInto(*out) + } + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new OrdsSrvsSpec. +func (in *OrdsSrvsSpec) DeepCopy() *OrdsSrvsSpec { + if in == nil { + return nil + } + out := new(OrdsSrvsSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *OrdsSrvsStatus) DeepCopyInto(out *OrdsSrvsStatus) { + *out = *in + if in.HTTPPort != nil { + in, out := &in.HTTPPort, &out.HTTPPort + *out = new(int32) + **out = **in + } + if in.HTTPSPort != nil { + in, out := &in.HTTPSPort, &out.HTTPSPort + *out = new(int32) + **out = **in + } + if in.Conditions != nil { + in, out := &in.Conditions, &out.Conditions + *out = make([]v1.Condition, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new OrdsSrvsStatus. +func (in *OrdsSrvsStatus) DeepCopy() *OrdsSrvsStatus { + if in == nil { + return nil + } + out := new(OrdsSrvsStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PDB) DeepCopyInto(out *PDB) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + in.Spec.DeepCopyInto(&out.Spec) + out.Status = in.Status +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDB. +func (in *PDB) DeepCopy() *PDB { + if in == nil { + return nil + } + out := new(PDB) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *PDB) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PDBAdminName) DeepCopyInto(out *PDBAdminName) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBAdminName. +func (in *PDBAdminName) DeepCopy() *PDBAdminName { + if in == nil { + return nil + } + out := new(PDBAdminName) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PDBAdminPassword) DeepCopyInto(out *PDBAdminPassword) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBAdminPassword. +func (in *PDBAdminPassword) DeepCopy() *PDBAdminPassword { + if in == nil { + return nil + } + out := new(PDBAdminPassword) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PDBConfig) DeepCopyInto(out *PDBConfig) { + *out = *in + if in.PdbName != nil { + in, out := &in.PdbName, &out.PdbName + *out = new(string) + **out = **in + } + if in.PdbAdminPassword != nil { + in, out := &in.PdbAdminPassword, &out.PdbAdminPassword + *out = new(string) + **out = **in + } + if in.TdeWalletPassword != nil { + in, out := &in.TdeWalletPassword, &out.TdeWalletPassword + *out = new(string) + **out = **in + } + if in.ShouldPdbAdminAccountBeLocked != nil { + in, out := &in.ShouldPdbAdminAccountBeLocked, &out.ShouldPdbAdminAccountBeLocked + *out = new(bool) + **out = **in + } + if in.FreeformTags != nil { + in, out := &in.FreeformTags, &out.FreeformTags + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + if in.IsDelete != nil { + in, out := &in.IsDelete, &out.IsDelete + *out = new(bool) + **out = **in + } + if in.PluggableDatabaseId != nil { + in, out := &in.PluggableDatabaseId, &out.PluggableDatabaseId + *out = new(string) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBConfig. +func (in *PDBConfig) DeepCopy() *PDBConfig { + if in == nil { + return nil + } + out := new(PDBConfig) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PDBConfigStatus) DeepCopyInto(out *PDBConfigStatus) { + *out = *in + if in.PdbName != nil { + in, out := &in.PdbName, &out.PdbName + *out = new(string) + **out = **in + } + if in.ShouldPdbAdminAccountBeLocked != nil { + in, out := &in.ShouldPdbAdminAccountBeLocked, &out.ShouldPdbAdminAccountBeLocked + *out = new(bool) + **out = **in + } + if in.FreeformTags != nil { + in, out := &in.FreeformTags, &out.FreeformTags + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + if in.PluggableDatabaseId != nil { + in, out := &in.PluggableDatabaseId, &out.PluggableDatabaseId + *out = new(string) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBConfigStatus. +func (in *PDBConfigStatus) DeepCopy() *PDBConfigStatus { + if in == nil { + return nil + } + out := new(PDBConfigStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PDBDetailsStatus) DeepCopyInto(out *PDBDetailsStatus) { + *out = *in + if in.PDBConfigStatus != nil { + in, out := &in.PDBConfigStatus, &out.PDBConfigStatus + *out = make([]PDBConfigStatus, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBDetailsStatus. +func (in *PDBDetailsStatus) DeepCopy() *PDBDetailsStatus { + if in == nil { + return nil + } + out := new(PDBDetailsStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PDBList) DeepCopyInto(out *PDBList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]PDB, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBList. +func (in *PDBList) DeepCopy() *PDBList { + if in == nil { + return nil + } + out := new(PDBList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *PDBList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PDBPRIVKEY) DeepCopyInto(out *PDBPRIVKEY) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBPRIVKEY. +func (in *PDBPRIVKEY) DeepCopy() *PDBPRIVKEY { + if in == nil { + return nil + } + out := new(PDBPRIVKEY) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PDBPUBKEY) DeepCopyInto(out *PDBPUBKEY) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBPUBKEY. +func (in *PDBPUBKEY) DeepCopy() *PDBPUBKEY { + if in == nil { + return nil + } + out := new(PDBPUBKEY) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PDBSecret) DeepCopyInto(out *PDBSecret) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBSecret. +func (in *PDBSecret) DeepCopy() *PDBSecret { + if in == nil { + return nil + } + out := new(PDBSecret) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PDBSpec) DeepCopyInto(out *PDBSpec) { + *out = *in + out.PDBTlsKey = in.PDBTlsKey + out.PDBTlsCrt = in.PDBTlsCrt + out.PDBTlsCat = in.PDBTlsCat + out.AdminName = in.AdminName + out.AdminPwd = in.AdminPwd + out.WebServerUsr = in.WebServerUsr + out.WebServerPwd = in.WebServerPwd + if in.ReuseTempFile != nil { + in, out := &in.ReuseTempFile, &out.ReuseTempFile + *out = new(bool) + **out = **in + } + if in.UnlimitedStorage != nil { + in, out := &in.UnlimitedStorage, &out.UnlimitedStorage + *out = new(bool) + **out = **in + } + if in.AsClone != nil { + in, out := &in.AsClone, &out.AsClone + *out = new(bool) + **out = **in + } + if in.TDEImport != nil { + in, out := &in.TDEImport, &out.TDEImport + *out = new(bool) + **out = **in + } + if in.TDEExport != nil { + in, out := &in.TDEExport, &out.TDEExport + *out = new(bool) + **out = **in + } + out.TDEPassword = in.TDEPassword + out.TDESecret = in.TDESecret + if in.GetScript != nil { + in, out := &in.GetScript, &out.GetScript + *out = new(bool) + **out = **in + } + out.PDBPubKey = in.PDBPubKey + out.PDBPriKey = in.PDBPriKey +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBSpec. +func (in *PDBSpec) DeepCopy() *PDBSpec { + if in == nil { + return nil + } + out := new(PDBSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PDBStatus) DeepCopyInto(out *PDBStatus) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBStatus. +func (in *PDBStatus) DeepCopy() *PDBStatus { + if in == nil { + return nil + } + out := new(PDBStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PDBTLSCAT) DeepCopyInto(out *PDBTLSCAT) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBTLSCAT. +func (in *PDBTLSCAT) DeepCopy() *PDBTLSCAT { + if in == nil { + return nil + } + out := new(PDBTLSCAT) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PDBTLSCRT) DeepCopyInto(out *PDBTLSCRT) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBTLSCRT. +func (in *PDBTLSCRT) DeepCopy() *PDBTLSCRT { + if in == nil { + return nil + } + out := new(PDBTLSCRT) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PDBTLSKEY) DeepCopyInto(out *PDBTLSKEY) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBTLSKEY. +func (in *PDBTLSKEY) DeepCopy() *PDBTLSKEY { + if in == nil { + return nil + } + out := new(PDBTLSKEY) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PITSpec) DeepCopyInto(out *PITSpec) { + *out = *in + if in.Timestamp != nil { + in, out := &in.Timestamp, &out.Timestamp + *out = new(string) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PITSpec. +func (in *PITSpec) DeepCopy() *PITSpec { + if in == nil { + return nil + } + out := new(PITSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PasswordSecret) DeepCopyInto(out *PasswordSecret) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PasswordSecret. +func (in *PasswordSecret) DeepCopy() *PasswordSecret { + if in == nil { + return nil + } + out := new(PasswordSecret) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PasswordSpec) DeepCopyInto(out *PasswordSpec) { + *out = *in + in.K8sSecret.DeepCopyInto(&out.K8sSecret) + in.OciSecret.DeepCopyInto(&out.OciSecret) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PasswordSpec. +func (in *PasswordSpec) DeepCopy() *PasswordSpec { + if in == nil { + return nil + } + out := new(PasswordSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PoolSettings) DeepCopyInto(out *PoolSettings) { + *out = *in + out.DBSecret = in.DBSecret + out.DBAdminUserSecret = in.DBAdminUserSecret + out.DBCDBAdminUserSecret = in.DBCDBAdminUserSecret + if in.DBPoolDestroyTimeout != nil { + in, out := &in.DBPoolDestroyTimeout, &out.DBPoolDestroyTimeout + *out = new(timex.Duration) + **out = **in + } + if in.DebugTrackResources != nil { + in, out := &in.DebugTrackResources, &out.DebugTrackResources + *out = new(bool) + **out = **in + } + if in.FeatureOpenservicebrokerExclude != nil { + in, out := &in.FeatureOpenservicebrokerExclude, &out.FeatureOpenservicebrokerExclude + *out = new(bool) + **out = **in + } + if in.FeatureSDW != nil { + in, out := &in.FeatureSDW, &out.FeatureSDW + *out = new(bool) + **out = **in + } + if in.OwaTraceSql != nil { + in, out := &in.OwaTraceSql, &out.OwaTraceSql + *out = new(bool) + **out = **in + } + if in.SecurityJWTProfileEnabled != nil { + in, out := &in.SecurityJWTProfileEnabled, &out.SecurityJWTProfileEnabled + *out = new(bool) + **out = **in + } + if in.SecurityJWKSSize != nil { + in, out := &in.SecurityJWKSSize, &out.SecurityJWKSSize + *out = new(int32) + **out = **in + } + if in.SecurityJWKSConnectionTimeout != nil { + in, out := &in.SecurityJWKSConnectionTimeout, &out.SecurityJWKSConnectionTimeout + *out = new(timex.Duration) + **out = **in + } + if in.SecurityJWKSReadTimeout != nil { + in, out := &in.SecurityJWKSReadTimeout, &out.SecurityJWKSReadTimeout + *out = new(timex.Duration) + **out = **in + } + if in.SecurityJWKSRefreshInterval != nil { + in, out := &in.SecurityJWKSRefreshInterval, &out.SecurityJWKSRefreshInterval + *out = new(timex.Duration) + **out = **in + } + if in.SecurityJWTAllowedSkew != nil { + in, out := &in.SecurityJWTAllowedSkew, &out.SecurityJWTAllowedSkew + *out = new(timex.Duration) + **out = **in + } + if in.SecurityJWTAllowedAge != nil { + in, out := &in.SecurityJWTAllowedAge, &out.SecurityJWTAllowedAge + *out = new(timex.Duration) + **out = **in + } + if in.DBPort != nil { + in, out := &in.DBPort, &out.DBPort + *out = new(int32) + **out = **in + } + if in.JDBCInactivityTimeout != nil { + in, out := &in.JDBCInactivityTimeout, &out.JDBCInactivityTimeout + *out = new(int32) + **out = **in + } + if in.JDBCInitialLimit != nil { + in, out := &in.JDBCInitialLimit, &out.JDBCInitialLimit + *out = new(int32) + **out = **in + } + if in.JDBCMaxConnectionReuseCount != nil { + in, out := &in.JDBCMaxConnectionReuseCount, &out.JDBCMaxConnectionReuseCount + *out = new(int32) + **out = **in + } + if in.JDBCMaxConnectionReuseTime != nil { + in, out := &in.JDBCMaxConnectionReuseTime, &out.JDBCMaxConnectionReuseTime + *out = new(int32) + **out = **in + } + if in.JDBCSecondsToTrustIdleConnection != nil { + in, out := &in.JDBCSecondsToTrustIdleConnection, &out.JDBCSecondsToTrustIdleConnection + *out = new(int32) + **out = **in + } + if in.JDBCMaxLimit != nil { + in, out := &in.JDBCMaxLimit, &out.JDBCMaxLimit + *out = new(int32) + **out = **in + } + if in.JDBCAuthEnabled != nil { + in, out := &in.JDBCAuthEnabled, &out.JDBCAuthEnabled + *out = new(bool) + **out = **in + } + if in.JDBCMaxStatementsLimit != nil { + in, out := &in.JDBCMaxStatementsLimit, &out.JDBCMaxStatementsLimit + *out = new(int32) + **out = **in + } + if in.JDBCMinLimit != nil { + in, out := &in.JDBCMinLimit, &out.JDBCMinLimit + *out = new(int32) + **out = **in + } + if in.JDBCStatementTimeout != nil { + in, out := &in.JDBCStatementTimeout, &out.JDBCStatementTimeout + *out = new(int32) + **out = **in + } + if in.MiscPaginationMaxRows != nil { + in, out := &in.MiscPaginationMaxRows, &out.MiscPaginationMaxRows + *out = new(int32) + **out = **in + } + if in.RestEnabledSqlActive != nil { + in, out := &in.RestEnabledSqlActive, &out.RestEnabledSqlActive + *out = new(bool) + **out = **in + } + if in.DBWalletSecret != nil { + in, out := &in.DBWalletSecret, &out.DBWalletSecret + *out = new(DBWalletSecret) + **out = **in + } + if in.TNSAdminSecret != nil { + in, out := &in.TNSAdminSecret, &out.TNSAdminSecret + *out = new(TNSAdminSecret) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PoolSettings. +func (in *PoolSettings) DeepCopy() *PoolSettings { + if in == nil { + return nil + } + out := new(PoolSettings) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PortMapping) DeepCopyInto(out *PortMapping) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PortMapping. +func (in *PortMapping) DeepCopy() *PortMapping { + if in == nil { + return nil + } + out := new(PortMapping) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PriVKey) DeepCopyInto(out *PriVKey) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PriVKey. +func (in *PriVKey) DeepCopy() *PriVKey { + if in == nil { + return nil + } + out := new(PriVKey) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *SecretDetails) DeepCopyInto(out *SecretDetails) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SecretDetails. +func (in *SecretDetails) DeepCopy() *SecretDetails { + if in == nil { + return nil + } + out := new(SecretDetails) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *ShardSpec) DeepCopyInto(out *ShardSpec) { + *out = *in + if in.EnvVars != nil { + in, out := &in.EnvVars, &out.EnvVars + *out = make([]EnvironmentVariable, len(*in)) + copy(*out, *in) + } + if in.Resources != nil { + in, out := &in.Resources, &out.Resources + *out = new(corev1.ResourceRequirements) + (*in).DeepCopyInto(*out) + } + if in.NodeSelector != nil { + in, out := &in.NodeSelector, &out.NodeSelector + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + if in.PvAnnotations != nil { + in, out := &in.PvAnnotations, &out.PvAnnotations + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + if in.PvMatchLabels != nil { + in, out := &in.PvMatchLabels, &out.PvMatchLabels + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + if in.ImagePulllPolicy != nil { + in, out := &in.ImagePulllPolicy, &out.ImagePulllPolicy + *out = new(corev1.PullPolicy) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ShardSpec. +func (in *ShardSpec) DeepCopy() *ShardSpec { + if in == nil { + return nil + } + out := new(ShardSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *ShardingDatabase) DeepCopyInto(out *ShardingDatabase) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + in.Spec.DeepCopyInto(&out.Spec) + in.Status.DeepCopyInto(&out.Status) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ShardingDatabase. +func (in *ShardingDatabase) DeepCopy() *ShardingDatabase { + if in == nil { + return nil + } + out := new(ShardingDatabase) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *ShardingDatabase) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *ShardingDatabaseList) DeepCopyInto(out *ShardingDatabaseList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]ShardingDatabase, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ShardingDatabaseList. +func (in *ShardingDatabaseList) DeepCopy() *ShardingDatabaseList { + if in == nil { + return nil + } + out := new(ShardingDatabaseList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *ShardingDatabaseList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *ShardingDatabaseSpec) DeepCopyInto(out *ShardingDatabaseSpec) { + *out = *in + if in.Shard != nil { + in, out := &in.Shard, &out.Shard + *out = make([]ShardSpec, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } + if in.Catalog != nil { + in, out := &in.Catalog, &out.Catalog + *out = make([]CatalogSpec, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } + if in.Gsm != nil { + in, out := &in.Gsm, &out.Gsm + *out = make([]GsmSpec, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } + if in.PortMappings != nil { + in, out := &in.PortMappings, &out.PortMappings + *out = make([]PortMapping, len(*in)) + copy(*out, *in) + } + if in.GsmShardSpace != nil { + in, out := &in.GsmShardSpace, &out.GsmShardSpace + *out = make([]GsmShardSpaceSpec, len(*in)) + copy(*out, *in) + } + if in.GsmShardGroup != nil { + in, out := &in.GsmShardGroup, &out.GsmShardGroup + *out = make([]GsmShardGroupSpec, len(*in)) + copy(*out, *in) + } + if in.ShardRegion != nil { + in, out := &in.ShardRegion, &out.ShardRegion + *out = make([]string, len(*in)) + copy(*out, *in) + } + if in.GsmService != nil { + in, out := &in.GsmService, &out.GsmService + *out = make([]GsmServiceSpec, len(*in)) + copy(*out, *in) + } + if in.DbSecret != nil { + in, out := &in.DbSecret, &out.DbSecret + *out = new(SecretDetails) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ShardingDatabaseSpec. +func (in *ShardingDatabaseSpec) DeepCopy() *ShardingDatabaseSpec { + if in == nil { + return nil + } + out := new(ShardingDatabaseSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *ShardingDatabaseStatus) DeepCopyInto(out *ShardingDatabaseStatus) { + *out = *in + if in.Shard != nil { + in, out := &in.Shard, &out.Shard + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + if in.Catalog != nil { + in, out := &in.Catalog, &out.Catalog + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + in.Gsm.DeepCopyInto(&out.Gsm) + if in.CrdStatus != nil { + in, out := &in.CrdStatus, &out.CrdStatus + *out = make([]v1.Condition, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ShardingDatabaseStatus. +func (in *ShardingDatabaseStatus) DeepCopy() *ShardingDatabaseStatus { + if in == nil { + return nil + } + out := new(ShardingDatabaseStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *SingleInstanceDatabase) DeepCopyInto(out *SingleInstanceDatabase) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + in.Spec.DeepCopyInto(&out.Spec) + in.Status.DeepCopyInto(&out.Status) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SingleInstanceDatabase. +func (in *SingleInstanceDatabase) DeepCopy() *SingleInstanceDatabase { + if in == nil { + return nil + } + out := new(SingleInstanceDatabase) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *SingleInstanceDatabase) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *SingleInstanceDatabaseAdminPassword) DeepCopyInto(out *SingleInstanceDatabaseAdminPassword) { + *out = *in + if in.KeepSecret != nil { + in, out := &in.KeepSecret, &out.KeepSecret + *out = new(bool) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SingleInstanceDatabaseAdminPassword. +func (in *SingleInstanceDatabaseAdminPassword) DeepCopy() *SingleInstanceDatabaseAdminPassword { + if in == nil { + return nil + } + out := new(SingleInstanceDatabaseAdminPassword) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *SingleInstanceDatabaseImage) DeepCopyInto(out *SingleInstanceDatabaseImage) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SingleInstanceDatabaseImage. +func (in *SingleInstanceDatabaseImage) DeepCopy() *SingleInstanceDatabaseImage { + if in == nil { + return nil + } + out := new(SingleInstanceDatabaseImage) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *SingleInstanceDatabaseInitParams) DeepCopyInto(out *SingleInstanceDatabaseInitParams) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SingleInstanceDatabaseInitParams. +func (in *SingleInstanceDatabaseInitParams) DeepCopy() *SingleInstanceDatabaseInitParams { + if in == nil { + return nil + } + out := new(SingleInstanceDatabaseInitParams) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *SingleInstanceDatabaseList) DeepCopyInto(out *SingleInstanceDatabaseList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]SingleInstanceDatabase, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SingleInstanceDatabaseList. +func (in *SingleInstanceDatabaseList) DeepCopy() *SingleInstanceDatabaseList { + if in == nil { + return nil + } + out := new(SingleInstanceDatabaseList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *SingleInstanceDatabaseList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *SingleInstanceDatabasePersistence) DeepCopyInto(out *SingleInstanceDatabasePersistence) { + *out = *in + if in.SetWritePermissions != nil { + in, out := &in.SetWritePermissions, &out.SetWritePermissions + *out = new(bool) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SingleInstanceDatabasePersistence. +func (in *SingleInstanceDatabasePersistence) DeepCopy() *SingleInstanceDatabasePersistence { + if in == nil { + return nil + } + out := new(SingleInstanceDatabasePersistence) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *SingleInstanceDatabaseResource) DeepCopyInto(out *SingleInstanceDatabaseResource) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SingleInstanceDatabaseResource. +func (in *SingleInstanceDatabaseResource) DeepCopy() *SingleInstanceDatabaseResource { + if in == nil { + return nil + } + out := new(SingleInstanceDatabaseResource) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *SingleInstanceDatabaseResources) DeepCopyInto(out *SingleInstanceDatabaseResources) { + *out = *in + if in.Requests != nil { + in, out := &in.Requests, &out.Requests + *out = new(SingleInstanceDatabaseResource) + **out = **in + } + if in.Limits != nil { + in, out := &in.Limits, &out.Limits + *out = new(SingleInstanceDatabaseResource) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SingleInstanceDatabaseResources. +func (in *SingleInstanceDatabaseResources) DeepCopy() *SingleInstanceDatabaseResources { + if in == nil { + return nil + } + out := new(SingleInstanceDatabaseResources) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *SingleInstanceDatabaseSpec) DeepCopyInto(out *SingleInstanceDatabaseSpec) { + *out = *in + if in.ServiceAnnotations != nil { + in, out := &in.ServiceAnnotations, &out.ServiceAnnotations + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + if in.FlashBack != nil { + in, out := &in.FlashBack, &out.FlashBack + *out = new(bool) + **out = **in + } + if in.ArchiveLog != nil { + in, out := &in.ArchiveLog, &out.ArchiveLog + *out = new(bool) + **out = **in + } + if in.ForceLogging != nil { + in, out := &in.ForceLogging, &out.ForceLogging + *out = new(bool) + **out = **in + } + if in.TrueCacheServices != nil { + in, out := &in.TrueCacheServices, &out.TrueCacheServices + *out = make([]string, len(*in)) + copy(*out, *in) + } + if in.NodeSelector != nil { + in, out := &in.NodeSelector, &out.NodeSelector + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + in.AdminPassword.DeepCopyInto(&out.AdminPassword) + out.Image = in.Image + in.Persistence.DeepCopyInto(&out.Persistence) + if in.InitParams != nil { + in, out := &in.InitParams, &out.InitParams + *out = new(SingleInstanceDatabaseInitParams) + **out = **in + } + in.Resources.DeepCopyInto(&out.Resources) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SingleInstanceDatabaseSpec. +func (in *SingleInstanceDatabaseSpec) DeepCopy() *SingleInstanceDatabaseSpec { + if in == nil { + return nil + } + out := new(SingleInstanceDatabaseSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *SingleInstanceDatabaseStatus) DeepCopyInto(out *SingleInstanceDatabaseStatus) { + *out = *in + if in.Nodes != nil { + in, out := &in.Nodes, &out.Nodes + *out = make([]string, len(*in)) + copy(*out, *in) + } + if in.DgBroker != nil { + in, out := &in.DgBroker, &out.DgBroker + *out = new(string) + **out = **in + } + if in.StandbyDatabases != nil { + in, out := &in.StandbyDatabases, &out.StandbyDatabases + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + if in.Conditions != nil { + in, out := &in.Conditions, &out.Conditions + *out = make([]v1.Condition, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } + out.InitParams = in.InitParams + in.Persistence.DeepCopyInto(&out.Persistence) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SingleInstanceDatabaseStatus. +func (in *SingleInstanceDatabaseStatus) DeepCopy() *SingleInstanceDatabaseStatus { + if in == nil { + return nil + } + out := new(SingleInstanceDatabaseStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *SourceSpec) DeepCopyInto(out *SourceSpec) { + *out = *in + in.K8sAdbBackup.DeepCopyInto(&out.K8sAdbBackup) + in.PointInTime.DeepCopyInto(&out.PointInTime) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SourceSpec. +func (in *SourceSpec) DeepCopy() *SourceSpec { + if in == nil { + return nil + } + out := new(SourceSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *TDEPwd) DeepCopyInto(out *TDEPwd) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new TDEPwd. +func (in *TDEPwd) DeepCopy() *TDEPwd { + if in == nil { + return nil + } + out := new(TDEPwd) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *TDESecret) DeepCopyInto(out *TDESecret) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new TDESecret. +func (in *TDESecret) DeepCopy() *TDESecret { + if in == nil { + return nil + } + out := new(TDESecret) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *TNSAdminSecret) DeepCopyInto(out *TNSAdminSecret) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new TNSAdminSecret. +func (in *TNSAdminSecret) DeepCopy() *TNSAdminSecret { + if in == nil { + return nil + } + out := new(TNSAdminSecret) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *TargetSpec) DeepCopyInto(out *TargetSpec) { + *out = *in + in.K8sAdb.DeepCopyInto(&out.K8sAdb) + in.OciAdb.DeepCopyInto(&out.OciAdb) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new TargetSpec. +func (in *TargetSpec) DeepCopy() *TargetSpec { + if in == nil { + return nil + } + out := new(TargetSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *VmNetworkDetails) DeepCopyInto(out *VmNetworkDetails) { + *out = *in + if in.VcnName != nil { + in, out := &in.VcnName, &out.VcnName + *out = new(string) + **out = **in + } + if in.SubnetName != nil { + in, out := &in.SubnetName, &out.SubnetName + *out = new(string) + **out = **in + } + if in.ScanDnsName != nil { + in, out := &in.ScanDnsName, &out.ScanDnsName + *out = new(string) + **out = **in + } + if in.ListenerPort != nil { + in, out := &in.ListenerPort, &out.ListenerPort + *out = new(int) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new VmNetworkDetails. +func (in *VmNetworkDetails) DeepCopy() *VmNetworkDetails { + if in == nil { + return nil + } + out := new(VmNetworkDetails) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *WalletSpec) DeepCopyInto(out *WalletSpec) { + *out = *in + if in.Name != nil { + in, out := &in.Name, &out.Name + *out = new(string) + **out = **in + } + in.Password.DeepCopyInto(&out.Password) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new WalletSpec. +func (in *WalletSpec) DeepCopy() *WalletSpec { + if in == nil { + return nil + } + out := new(WalletSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *WebLrestServerPassword) DeepCopyInto(out *WebLrestServerPassword) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new WebLrestServerPassword. +func (in *WebLrestServerPassword) DeepCopy() *WebLrestServerPassword { + if in == nil { + return nil + } + out := new(WebLrestServerPassword) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *WebLrestServerUser) DeepCopyInto(out *WebLrestServerUser) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new WebLrestServerUser. +func (in *WebLrestServerUser) DeepCopy() *WebLrestServerUser { + if in == nil { + return nil + } + out := new(WebLrestServerUser) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *WebLrpdbServerPassword) DeepCopyInto(out *WebLrpdbServerPassword) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new WebLrpdbServerPassword. +func (in *WebLrpdbServerPassword) DeepCopy() *WebLrpdbServerPassword { + if in == nil { + return nil + } + out := new(WebLrpdbServerPassword) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *WebLrpdbServerUser) DeepCopyInto(out *WebLrpdbServerUser) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new WebLrpdbServerUser. +func (in *WebLrpdbServerUser) DeepCopy() *WebLrpdbServerUser { + if in == nil { + return nil + } + out := new(WebLrpdbServerUser) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *WebServerPassword) DeepCopyInto(out *WebServerPassword) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new WebServerPassword. +func (in *WebServerPassword) DeepCopy() *WebServerPassword { + if in == nil { + return nil + } + out := new(WebServerPassword) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *WebServerPasswordPDB) DeepCopyInto(out *WebServerPasswordPDB) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new WebServerPasswordPDB. +func (in *WebServerPasswordPDB) DeepCopy() *WebServerPasswordPDB { + if in == nil { + return nil + } + out := new(WebServerPasswordPDB) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *WebServerUser) DeepCopyInto(out *WebServerUser) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new WebServerUser. +func (in *WebServerUser) DeepCopy() *WebServerUser { + if in == nil { + return nil + } + out := new(WebServerUser) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *WebServerUserPDB) DeepCopyInto(out *WebServerUserPDB) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new WebServerUserPDB. +func (in *WebServerUserPDB) DeepCopy() *WebServerUserPDB { + if in == nil { + return nil + } + out := new(WebServerUserPDB) + in.DeepCopyInto(out) + return out +} diff --git a/apis/observability/v1/databaseobserver_types.go b/apis/observability/v1/databaseobserver_types.go new file mode 100644 index 00000000..642ff18b --- /dev/null +++ b/apis/observability/v1/databaseobserver_types.go @@ -0,0 +1,195 @@ +/* +** Copyright (c) 2024 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v1 + +import ( + monitorv1 "github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring/v1" + corev1 "k8s.io/api/core/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +type StatusEnum string + +// DatabaseObserverSpec defines the desired state of DatabaseObserver +type DatabaseObserverSpec struct { + Database DatabaseObserverDatabase `json:"database,omitempty"` + Exporter DatabaseObserverExporterConfig `json:"exporter,omitempty"` + ExporterConfig DatabaseObserverConfigMap `json:"configuration,omitempty"` + Prometheus PrometheusConfig `json:"prometheus,omitempty"` + OCIConfig OCIConfigSpec `json:"ociConfig,omitempty"` + Replicas int32 `json:"replicas,omitempty"` + Log LogConfig `json:"log,omitempty"` + InheritLabels []string `json:"inheritLabels,omitempty"` + ExporterSidecars []corev1.Container `json:"sidecars,omitempty"` + SideCarVolumes []corev1.Volume `json:"sidecarVolumes,omitempty"` +} + +// LogConfig defines the configuration details relation to the logs of DatabaseObserver +type LogConfig struct { + Path string `json:"path,omitempty"` + Filename string `json:"filename,omitempty"` + Volume LogVolume `json:"volume,omitempty"` +} + +type LogVolume struct { + Name string `json:"name,omitempty"` + PersistentVolumeClaim LogVolumePVClaim `json:"persistentVolumeClaim,omitempty"` +} + +type LogVolumePVClaim struct { + ClaimName string `json:"claimName,omitempty"` +} + +// DatabaseObserverDatabase defines the database details used for DatabaseObserver +type DatabaseObserverDatabase struct { + DBUser DBSecret `json:"dbUser,omitempty"` + DBPassword DBSecretWithVault `json:"dbPassword,omitempty"` + DBWallet DBSecret `json:"dbWallet,omitempty"` + DBConnectionString DBSecret `json:"dbConnectionString,omitempty"` +} + +// DatabaseObserverExporterConfig defines the configuration details related to the exporters of DatabaseObserver +type DatabaseObserverExporterConfig struct { + Deployment DatabaseObserverDeployment `json:"deployment,omitempty"` + Service DatabaseObserverService `json:"service,omitempty"` +} + +// DatabaseObserverDeployment defines the exporter deployment component of DatabaseObserver +type DatabaseObserverDeployment struct { + ExporterImage string `json:"image,omitempty"` + SecurityContext *corev1.SecurityContext `json:"securityContext,omitempty"` + ExporterArgs []string `json:"args,omitempty"` + ExporterCommands []string `json:"commands,omitempty"` + ExporterEnvs map[string]string `json:"env,omitempty"` + Labels map[string]string `json:"labels,omitempty"` + DeploymentPodTemplate DeploymentPodTemplate `json:"podTemplate,omitempty"` +} + +// DeploymentPodTemplate defines the labels for the DatabaseObserver pods component of a deployment +type DeploymentPodTemplate struct { + SecurityContext *corev1.PodSecurityContext `json:"securityContext,omitempty"` + Labels map[string]string `json:"labels,omitempty"` +} + +// DatabaseObserverService defines the exporter service component of DatabaseObserver +type DatabaseObserverService struct { + Ports []corev1.ServicePort `json:"ports,omitempty"` + Labels map[string]string `json:"labels,omitempty"` +} + +// PrometheusConfig defines the generated resources for Prometheus +type PrometheusConfig struct { + ServiceMonitor PrometheusServiceMonitor `json:"serviceMonitor,omitempty"` +} + +// PrometheusServiceMonitor defines DatabaseObserver servicemonitor spec +type PrometheusServiceMonitor struct { + Labels map[string]string `json:"labels,omitempty"` + NamespaceSelector *monitorv1.NamespaceSelector `json:"namespaceSelector,omitempty"` + Endpoints []monitorv1.Endpoint `json:"endpoints,omitempty"` +} + +// DBSecret defines secrets used in reference +type DBSecret struct { + Key string `json:"key,omitempty"` + SecretName string `json:"secret,omitempty"` +} + +// DBSecretWithVault defines secrets used in reference with vault fields +type DBSecretWithVault struct { + Key string `json:"key,omitempty"` + SecretName string `json:"secret,omitempty"` + VaultOCID string `json:"vaultOCID,omitempty"` + VaultSecretName string `json:"vaultSecretName,omitempty"` +} + +// DatabaseObserverConfigMap defines configMap used for metrics configuration +type DatabaseObserverConfigMap struct { + Configmap ConfigMapDetails `json:"configMap,omitempty"` +} + +// ConfigMapDetails defines the configmap name +type ConfigMapDetails struct { + Key string `json:"key,omitempty"` + Name string `json:"name,omitempty"` +} + +// OCIConfigSpec defines the configmap name and secret name used for connecting to OCI +type OCIConfigSpec struct { + ConfigMapName string `json:"configMapName,omitempty"` + SecretName string `json:"secretName,omitempty"` +} + +// DatabaseObserverStatus defines the observed state of DatabaseObserver +type DatabaseObserverStatus struct { + Conditions []metav1.Condition `json:"conditions"` + Status string `json:"status,omitempty"` + ExporterConfig string `json:"exporterConfig"` + Version string `json:"version"` + Replicas int `json:"replicas,omitempty"` +} + +//+kubebuilder:object:root=true +//+kubebuilder:subresource:status +// +kubebuilder:resource:shortName="dbobserver";"dbobservers" + +// DatabaseObserver is the Schema for the databaseobservers API +// +kubebuilder:printcolumn:JSONPath=".status.exporterConfig",name="ExporterConfig",type=string +// +kubebuilder:printcolumn:JSONPath=".status.status",name="Status",type=string +// +kubebuilder:printcolumn:JSONPath=".status.version",name="Version",type=string +type DatabaseObserver struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + Spec DatabaseObserverSpec `json:"spec,omitempty"` + Status DatabaseObserverStatus `json:"status,omitempty"` +} + +//+kubebuilder:object:root=true + +// DatabaseObserverList contains a list of DatabaseObserver +type DatabaseObserverList struct { + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata,omitempty"` + Items []DatabaseObserver `json:"items"` +} + +func init() { + SchemeBuilder.Register(&DatabaseObserver{}, &DatabaseObserverList{}) +} diff --git a/apis/observability/v1/databaseobserver_webhook.go b/apis/observability/v1/databaseobserver_webhook.go new file mode 100644 index 00000000..286d6ed6 --- /dev/null +++ b/apis/observability/v1/databaseobserver_webhook.go @@ -0,0 +1,185 @@ +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v1 + +import ( + dbcommons "github.com/oracle/oracle-database-operator/commons/database" + apierrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/runtime/schema" + "k8s.io/apimachinery/pkg/util/validation/field" + ctrl "sigs.k8s.io/controller-runtime" + logf "sigs.k8s.io/controller-runtime/pkg/log" + "sigs.k8s.io/controller-runtime/pkg/webhook" + "sigs.k8s.io/controller-runtime/pkg/webhook/admission" + "strings" +) + +// log is for logging in this package. +var databaseobserverlog = logf.Log.WithName("databaseobserver-resource") + +const ( + AllowedExporterImage = "container-registry.oracle.com/database/observability-exporter" + ErrorSpecValidationMissingConnString = "a required field for database connection string secret is missing or does not have a value" + ErrorSpecValidationMissingDBUser = "a required field for database user secret is missing or does not have a value" + ErrorSpecValidationMissingDBVaultField = "a field for the OCI vault has a value but the other required field is missing or does not have a value" + ErrorSpecValidationMissingOCIConfig = "a field(s) for the OCI Config is missing or does not have a value when fields for the OCI vault has values" + ErrorSpecValidationMissingDBPasswordSecret = "a required field for the database password secret is missing or does not have a value" + ErrorSpecExporterImageNotAllowed = "a different exporter image was found, only official database exporter container images are currently supported" +) + +func (r *DatabaseObserver) SetupWebhookWithManager(mgr ctrl.Manager) error { + return ctrl.NewWebhookManagedBy(mgr). + For(r). + Complete() +} + +// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! + +//+kubebuilder:webhook:path=/mutate-observability-oracle-com-v1-databaseobserver,mutating=true,sideEffects=none,failurePolicy=fail,groups=observability.oracle.com,resources=databaseobservers,verbs=create;update,versions=v1,name=mdatabaseobserver.kb.io,admissionReviewVersions=v1 + +var _ webhook.Defaulter = &DatabaseObserver{} + +// Default implements webhook.Defaulter so a webhook will be registered for the type +func (r *DatabaseObserver) Default() { + databaseobserverlog.Info("default", "name", r.Name) + + // TODO(user): fill in your defaulting logic. +} + +// TODO(user): change verbs to "verbs=create;update;delete" if you want to enable deletion validation. +//+kubebuilder:webhook:verbs=create;update,path=/validate-observability-oracle-com-v1-databaseobserver,mutating=false,sideEffects=none,failurePolicy=fail,groups=observability.oracle.com,resources=databaseobservers,versions=v1,name=vdatabaseobserver.kb.io,admissionReviewVersions=v1 + +var _ webhook.Validator = &DatabaseObserver{} + +// ValidateCreate implements webhook.Validator so a webhook will be registered for the type +func (r *DatabaseObserver) ValidateCreate() (admission.Warnings, error) { + databaseobserverlog.Info("validate create", "name", r.Name) + + var e field.ErrorList + ns := dbcommons.GetWatchNamespaces() + + // Check for namespace/cluster scope access + if _, isDesiredNamespaceWithinScope := ns[r.Namespace]; !isDesiredNamespaceWithinScope && len(ns) > 0 { + e = append(e, + field.Invalid(field.NewPath("metadata").Child("namespace"), r.Namespace, + "Oracle database operator doesn't watch over this namespace")) + } + + // Check required secret for db user has value + if r.Spec.Database.DBUser.SecretName == "" { + e = append(e, + field.Invalid(field.NewPath("spec").Child("database").Child("dbUser").Child("secret"), r.Spec.Database.DBUser.SecretName, + ErrorSpecValidationMissingDBUser)) + } + + // Check required secret for db connection string has value + if r.Spec.Database.DBConnectionString.SecretName == "" { + e = append(e, + field.Invalid(field.NewPath("spec").Child("database").Child("dbConnectionString").Child("secret"), r.Spec.Database.DBConnectionString.SecretName, + ErrorSpecValidationMissingConnString)) + } + + // The other vault field must have value if one does + if (r.Spec.Database.DBPassword.VaultOCID != "" && r.Spec.Database.DBPassword.VaultSecretName == "") || + (r.Spec.Database.DBPassword.VaultSecretName != "" && r.Spec.Database.DBPassword.VaultOCID == "") { + + e = append(e, + field.Invalid(field.NewPath("spec").Child("database").Child("dbPassword"), r.Spec.Database.DBPassword, + ErrorSpecValidationMissingDBVaultField)) + } + + // if vault fields have value, ociConfig must have values + if r.Spec.Database.DBPassword.VaultOCID != "" && r.Spec.Database.DBPassword.VaultSecretName != "" && + (r.Spec.OCIConfig.SecretName == "" || r.Spec.OCIConfig.ConfigMapName == "") { + + e = append(e, + field.Invalid(field.NewPath("spec").Child("ociConfig"), r.Spec.OCIConfig, + ErrorSpecValidationMissingOCIConfig)) + } + + // If all of {DB Password Secret Name and vaultOCID+vaultSecretName} have no value, then error out + if r.Spec.Database.DBPassword.SecretName == "" && + r.Spec.Database.DBPassword.VaultOCID == "" && + r.Spec.Database.DBPassword.VaultSecretName == "" { + + e = append(e, + field.Invalid(field.NewPath("spec").Child("database").Child("dbPassword").Child("secret"), r.Spec.Database.DBPassword.SecretName, + ErrorSpecValidationMissingDBPasswordSecret)) + } + + // disallow usage of any other image than the observability-exporter + if r.Spec.Exporter.Deployment.ExporterImage != "" && !strings.HasPrefix(r.Spec.Exporter.Deployment.ExporterImage, AllowedExporterImage) { + e = append(e, + field.Invalid(field.NewPath("spec").Child("exporter").Child("image"), r.Spec.Exporter.Deployment.ExporterImage, + ErrorSpecExporterImageNotAllowed)) + } + + // Return if any errors + if len(e) > 0 { + return nil, apierrors.NewInvalid(schema.GroupKind{Group: "observability.oracle.com", Kind: "DatabaseObserver"}, r.Name, e) + } + return nil, nil + +} + +// ValidateUpdate implements webhook.Validator so a webhook will be registered for the type +func (r *DatabaseObserver) ValidateUpdate(old runtime.Object) (admission.Warnings, error) { + databaseobserverlog.Info("validate update", "name", r.Name) + var e field.ErrorList + + // disallow usage of any other image than the observability-exporter + if r.Spec.Exporter.Deployment.ExporterImage != "" && !strings.HasPrefix(r.Spec.Exporter.Deployment.ExporterImage, AllowedExporterImage) { + e = append(e, + field.Invalid(field.NewPath("spec").Child("exporter").Child("image"), r.Spec.Exporter.Deployment.ExporterImage, + ErrorSpecExporterImageNotAllowed)) + } + // Return if any errors + if len(e) > 0 { + return nil, apierrors.NewInvalid(schema.GroupKind{Group: "observability.oracle.com", Kind: "DatabaseObserver"}, r.Name, e) + } + return nil, nil +} + +// ValidateDelete implements webhook.Validator so a webhook will be registered for the type +func (r *DatabaseObserver) ValidateDelete() (admission.Warnings, error) { + databaseobserverlog.Info("validate delete", "name", r.Name) + + return nil, nil +} diff --git a/apis/observability/v1/groupversion_info.go b/apis/observability/v1/groupversion_info.go new file mode 100644 index 00000000..3f332c05 --- /dev/null +++ b/apis/observability/v1/groupversion_info.go @@ -0,0 +1,58 @@ +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +// Package v1 contains API Schema definitions for the observability v1 API group +// +kubebuilder:object:generate=true +// +groupName=observability.oracle.com +package v1 + +import ( + "k8s.io/apimachinery/pkg/runtime/schema" + "sigs.k8s.io/controller-runtime/pkg/scheme" +) + +var ( + // GroupVersion is group version used to register these objects + GroupVersion = schema.GroupVersion{Group: "observability.oracle.com", Version: "v1"} + + // SchemeBuilder is used to add go types to the GroupVersionKind scheme + SchemeBuilder = &scheme.Builder{GroupVersion: GroupVersion} + + // AddToScheme adds the types in this group-version to the given scheme. + AddToScheme = SchemeBuilder.AddToScheme +) diff --git a/apis/observability/v1/zz_generated.deepcopy.go b/apis/observability/v1/zz_generated.deepcopy.go new file mode 100644 index 00000000..4924216f --- /dev/null +++ b/apis/observability/v1/zz_generated.deepcopy.go @@ -0,0 +1,481 @@ +//go:build !ignore_autogenerated + +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +// Code generated by controller-gen. DO NOT EDIT. + +package v1 + +import ( + monitoringv1 "github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring/v1" + corev1 "k8s.io/api/core/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/runtime" +) + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *ConfigMapDetails) DeepCopyInto(out *ConfigMapDetails) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ConfigMapDetails. +func (in *ConfigMapDetails) DeepCopy() *ConfigMapDetails { + if in == nil { + return nil + } + out := new(ConfigMapDetails) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DBSecret) DeepCopyInto(out *DBSecret) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DBSecret. +func (in *DBSecret) DeepCopy() *DBSecret { + if in == nil { + return nil + } + out := new(DBSecret) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DBSecretWithVault) DeepCopyInto(out *DBSecretWithVault) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DBSecretWithVault. +func (in *DBSecretWithVault) DeepCopy() *DBSecretWithVault { + if in == nil { + return nil + } + out := new(DBSecretWithVault) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DatabaseObserver) DeepCopyInto(out *DatabaseObserver) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + in.Spec.DeepCopyInto(&out.Spec) + in.Status.DeepCopyInto(&out.Status) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserver. +func (in *DatabaseObserver) DeepCopy() *DatabaseObserver { + if in == nil { + return nil + } + out := new(DatabaseObserver) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *DatabaseObserver) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DatabaseObserverConfigMap) DeepCopyInto(out *DatabaseObserverConfigMap) { + *out = *in + out.Configmap = in.Configmap +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserverConfigMap. +func (in *DatabaseObserverConfigMap) DeepCopy() *DatabaseObserverConfigMap { + if in == nil { + return nil + } + out := new(DatabaseObserverConfigMap) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DatabaseObserverDatabase) DeepCopyInto(out *DatabaseObserverDatabase) { + *out = *in + out.DBUser = in.DBUser + out.DBPassword = in.DBPassword + out.DBWallet = in.DBWallet + out.DBConnectionString = in.DBConnectionString +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserverDatabase. +func (in *DatabaseObserverDatabase) DeepCopy() *DatabaseObserverDatabase { + if in == nil { + return nil + } + out := new(DatabaseObserverDatabase) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DatabaseObserverDeployment) DeepCopyInto(out *DatabaseObserverDeployment) { + *out = *in + if in.SecurityContext != nil { + in, out := &in.SecurityContext, &out.SecurityContext + *out = new(corev1.SecurityContext) + (*in).DeepCopyInto(*out) + } + if in.ExporterArgs != nil { + in, out := &in.ExporterArgs, &out.ExporterArgs + *out = make([]string, len(*in)) + copy(*out, *in) + } + if in.ExporterCommands != nil { + in, out := &in.ExporterCommands, &out.ExporterCommands + *out = make([]string, len(*in)) + copy(*out, *in) + } + if in.ExporterEnvs != nil { + in, out := &in.ExporterEnvs, &out.ExporterEnvs + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + if in.Labels != nil { + in, out := &in.Labels, &out.Labels + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + in.DeploymentPodTemplate.DeepCopyInto(&out.DeploymentPodTemplate) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserverDeployment. +func (in *DatabaseObserverDeployment) DeepCopy() *DatabaseObserverDeployment { + if in == nil { + return nil + } + out := new(DatabaseObserverDeployment) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DatabaseObserverExporterConfig) DeepCopyInto(out *DatabaseObserverExporterConfig) { + *out = *in + in.Deployment.DeepCopyInto(&out.Deployment) + in.Service.DeepCopyInto(&out.Service) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserverExporterConfig. +func (in *DatabaseObserverExporterConfig) DeepCopy() *DatabaseObserverExporterConfig { + if in == nil { + return nil + } + out := new(DatabaseObserverExporterConfig) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DatabaseObserverList) DeepCopyInto(out *DatabaseObserverList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]DatabaseObserver, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserverList. +func (in *DatabaseObserverList) DeepCopy() *DatabaseObserverList { + if in == nil { + return nil + } + out := new(DatabaseObserverList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *DatabaseObserverList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DatabaseObserverService) DeepCopyInto(out *DatabaseObserverService) { + *out = *in + if in.Ports != nil { + in, out := &in.Ports, &out.Ports + *out = make([]corev1.ServicePort, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } + if in.Labels != nil { + in, out := &in.Labels, &out.Labels + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserverService. +func (in *DatabaseObserverService) DeepCopy() *DatabaseObserverService { + if in == nil { + return nil + } + out := new(DatabaseObserverService) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DatabaseObserverSpec) DeepCopyInto(out *DatabaseObserverSpec) { + *out = *in + out.Database = in.Database + in.Exporter.DeepCopyInto(&out.Exporter) + out.ExporterConfig = in.ExporterConfig + in.Prometheus.DeepCopyInto(&out.Prometheus) + out.OCIConfig = in.OCIConfig + out.Log = in.Log + if in.InheritLabels != nil { + in, out := &in.InheritLabels, &out.InheritLabels + *out = make([]string, len(*in)) + copy(*out, *in) + } + if in.ExporterSidecars != nil { + in, out := &in.ExporterSidecars, &out.ExporterSidecars + *out = make([]corev1.Container, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } + if in.SideCarVolumes != nil { + in, out := &in.SideCarVolumes, &out.SideCarVolumes + *out = make([]corev1.Volume, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserverSpec. +func (in *DatabaseObserverSpec) DeepCopy() *DatabaseObserverSpec { + if in == nil { + return nil + } + out := new(DatabaseObserverSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DatabaseObserverStatus) DeepCopyInto(out *DatabaseObserverStatus) { + *out = *in + if in.Conditions != nil { + in, out := &in.Conditions, &out.Conditions + *out = make([]metav1.Condition, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserverStatus. +func (in *DatabaseObserverStatus) DeepCopy() *DatabaseObserverStatus { + if in == nil { + return nil + } + out := new(DatabaseObserverStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DeploymentPodTemplate) DeepCopyInto(out *DeploymentPodTemplate) { + *out = *in + if in.SecurityContext != nil { + in, out := &in.SecurityContext, &out.SecurityContext + *out = new(corev1.PodSecurityContext) + (*in).DeepCopyInto(*out) + } + if in.Labels != nil { + in, out := &in.Labels, &out.Labels + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DeploymentPodTemplate. +func (in *DeploymentPodTemplate) DeepCopy() *DeploymentPodTemplate { + if in == nil { + return nil + } + out := new(DeploymentPodTemplate) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *LogConfig) DeepCopyInto(out *LogConfig) { + *out = *in + out.Volume = in.Volume +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new LogConfig. +func (in *LogConfig) DeepCopy() *LogConfig { + if in == nil { + return nil + } + out := new(LogConfig) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *LogVolume) DeepCopyInto(out *LogVolume) { + *out = *in + out.PersistentVolumeClaim = in.PersistentVolumeClaim +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new LogVolume. +func (in *LogVolume) DeepCopy() *LogVolume { + if in == nil { + return nil + } + out := new(LogVolume) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *LogVolumePVClaim) DeepCopyInto(out *LogVolumePVClaim) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new LogVolumePVClaim. +func (in *LogVolumePVClaim) DeepCopy() *LogVolumePVClaim { + if in == nil { + return nil + } + out := new(LogVolumePVClaim) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *OCIConfigSpec) DeepCopyInto(out *OCIConfigSpec) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new OCIConfigSpec. +func (in *OCIConfigSpec) DeepCopy() *OCIConfigSpec { + if in == nil { + return nil + } + out := new(OCIConfigSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PrometheusConfig) DeepCopyInto(out *PrometheusConfig) { + *out = *in + in.ServiceMonitor.DeepCopyInto(&out.ServiceMonitor) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PrometheusConfig. +func (in *PrometheusConfig) DeepCopy() *PrometheusConfig { + if in == nil { + return nil + } + out := new(PrometheusConfig) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PrometheusServiceMonitor) DeepCopyInto(out *PrometheusServiceMonitor) { + *out = *in + if in.Labels != nil { + in, out := &in.Labels, &out.Labels + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + if in.NamespaceSelector != nil { + in, out := &in.NamespaceSelector, &out.NamespaceSelector + *out = new(monitoringv1.NamespaceSelector) + (*in).DeepCopyInto(*out) + } + if in.Endpoints != nil { + in, out := &in.Endpoints, &out.Endpoints + *out = make([]monitoringv1.Endpoint, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PrometheusServiceMonitor. +func (in *PrometheusServiceMonitor) DeepCopy() *PrometheusServiceMonitor { + if in == nil { + return nil + } + out := new(PrometheusServiceMonitor) + in.DeepCopyInto(out) + return out +} diff --git a/apis/observability/v1alpha1/databaseobserver_types.go b/apis/observability/v1alpha1/databaseobserver_types.go index 97827d17..f4c62900 100644 --- a/apis/observability/v1alpha1/databaseobserver_types.go +++ b/apis/observability/v1alpha1/databaseobserver_types.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2022 Oracle and/or its affiliates. +** Copyright (c) 2024 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** @@ -39,6 +39,8 @@ package v1alpha1 import ( + monitorv1 "github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring/v1" + corev1 "k8s.io/api/core/v1" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" ) @@ -46,11 +48,32 @@ type StatusEnum string // DatabaseObserverSpec defines the desired state of DatabaseObserver type DatabaseObserverSpec struct { - Database DatabaseObserverDatabase `json:"database,omitempty"` - Exporter DatabaseObserverExporterConfig `json:"exporter,omitempty"` - Prometheus PrometheusConfig `json:"prometheus,omitempty"` - OCIConfig OCIConfigSpec `json:"ociConfig,omitempty"` - Replicas int32 `json:"replicas,omitempty"` + Database DatabaseObserverDatabase `json:"database,omitempty"` + Exporter DatabaseObserverExporterConfig `json:"exporter,omitempty"` + ExporterConfig DatabaseObserverConfigMap `json:"configuration,omitempty"` + Prometheus PrometheusConfig `json:"prometheus,omitempty"` + OCIConfig OCIConfigSpec `json:"ociConfig,omitempty"` + Replicas int32 `json:"replicas,omitempty"` + Log LogConfig `json:"log,omitempty"` + InheritLabels []string `json:"inheritLabels,omitempty"` + ExporterSidecars []corev1.Container `json:"sidecars,omitempty"` + SideCarVolumes []corev1.Volume `json:"sidecarVolumes,omitempty"` +} + +// LogConfig defines the configuration details relation to the logs of DatabaseObserver +type LogConfig struct { + Path string `json:"path,omitempty"` + Filename string `json:"filename,omitempty"` + Volume LogVolume `json:"volume,omitempty"` +} + +type LogVolume struct { + Name string `json:"name,omitempty"` + PersistentVolumeClaim LogVolumePVClaim `json:"persistentVolumeClaim,omitempty"` +} + +type LogVolumePVClaim struct { + ClaimName string `json:"claimName,omitempty"` } // DatabaseObserverDatabase defines the database details used for DatabaseObserver @@ -63,27 +86,52 @@ type DatabaseObserverDatabase struct { // DatabaseObserverExporterConfig defines the configuration details related to the exporters of DatabaseObserver type DatabaseObserverExporterConfig struct { - ExporterImage string `json:"image,omitempty"` - ExporterConfig DatabaseObserverConfigMap `json:"configuration,omitempty"` - Service DatabaseObserverService `json:"service,omitempty"` + Deployment DatabaseObserverDeployment `json:"deployment,omitempty"` + Service DatabaseObserverService `json:"service,omitempty"` +} + +// DatabaseObserverDeployment defines the exporter deployment component of DatabaseObserver +type DatabaseObserverDeployment struct { + ExporterImage string `json:"image,omitempty"` + SecurityContext *corev1.SecurityContext `json:"securityContext,omitempty"` + ExporterArgs []string `json:"args,omitempty"` + ExporterCommands []string `json:"commands,omitempty"` + ExporterEnvs map[string]string `json:"env,omitempty"` + Labels map[string]string `json:"labels,omitempty"` + DeploymentPodTemplate DeploymentPodTemplate `json:"podTemplate,omitempty"` +} + +// DeploymentPodTemplate defines the labels for the DatabaseObserver pods component of a deployment +type DeploymentPodTemplate struct { + SecurityContext *corev1.PodSecurityContext `json:"securityContext,omitempty"` + Labels map[string]string `json:"labels,omitempty"` } // DatabaseObserverService defines the exporter service component of DatabaseObserver type DatabaseObserverService struct { - Port int32 `json:"port,omitempty"` + Ports []corev1.ServicePort `json:"ports,omitempty"` + Labels map[string]string `json:"labels,omitempty"` } // PrometheusConfig defines the generated resources for Prometheus type PrometheusConfig struct { - Labels map[string]string `json:"labels,omitempty"` - Port string `json:"port,omitempty"` + ServiceMonitor PrometheusServiceMonitor `json:"serviceMonitor,omitempty"` +} + +// PrometheusServiceMonitor defines DatabaseObserver servicemonitor spec +type PrometheusServiceMonitor struct { + Labels map[string]string `json:"labels,omitempty"` + NamespaceSelector *monitorv1.NamespaceSelector `json:"namespaceSelector,omitempty"` + Endpoints []monitorv1.Endpoint `json:"endpoints,omitempty"` } +// DBSecret defines secrets used in reference type DBSecret struct { Key string `json:"key,omitempty"` SecretName string `json:"secret,omitempty"` } +// DBSecretWithVault defines secrets used in reference with vault fields type DBSecretWithVault struct { Key string `json:"key,omitempty"` SecretName string `json:"secret,omitempty"` @@ -91,16 +139,18 @@ type DBSecretWithVault struct { VaultSecretName string `json:"vaultSecretName,omitempty"` } +// DatabaseObserverConfigMap defines configMap used for metrics configuration type DatabaseObserverConfigMap struct { - Configmap ConfigMapDetails `json:"configmap,omitempty"` + Configmap ConfigMapDetails `json:"configMap,omitempty"` } // ConfigMapDetails defines the configmap name type ConfigMapDetails struct { Key string `json:"key,omitempty"` - Name string `json:"configmapName,omitempty"` + Name string `json:"name,omitempty"` } +// OCIConfigSpec defines the configmap name and secret name used for connecting to OCI type OCIConfigSpec struct { ConfigMapName string `json:"configMapName,omitempty"` SecretName string `json:"secretName,omitempty"` @@ -108,20 +158,21 @@ type OCIConfigSpec struct { // DatabaseObserverStatus defines the observed state of DatabaseObserver type DatabaseObserverStatus struct { - // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster - // Important: Run "make" to regenerate code after modifying this file Conditions []metav1.Condition `json:"conditions"` Status string `json:"status,omitempty"` ExporterConfig string `json:"exporterConfig"` + Version string `json:"version"` Replicas int `json:"replicas,omitempty"` } //+kubebuilder:object:root=true //+kubebuilder:subresource:status +// +kubebuilder:resource:shortName="dbobserver";"dbobservers" // DatabaseObserver is the Schema for the databaseobservers API // +kubebuilder:printcolumn:JSONPath=".status.exporterConfig",name="ExporterConfig",type=string // +kubebuilder:printcolumn:JSONPath=".status.status",name="Status",type=string +// +kubebuilder:printcolumn:JSONPath=".status.version",name="Version",type=string type DatabaseObserver struct { metav1.TypeMeta `json:",inline"` metav1.ObjectMeta `json:"metadata,omitempty"` diff --git a/apis/observability/v1alpha1/databaseobserver_webhook.go b/apis/observability/v1alpha1/databaseobserver_webhook.go index 2ab9b732..585ad3bf 100644 --- a/apis/observability/v1alpha1/databaseobserver_webhook.go +++ b/apis/observability/v1alpha1/databaseobserver_webhook.go @@ -145,9 +145,9 @@ func (r *DatabaseObserver) ValidateCreate() (admission.Warnings, error) { } // disallow usage of any other image than the observability-exporter - if r.Spec.Exporter.ExporterImage != "" && !strings.HasPrefix(r.Spec.Exporter.ExporterImage, AllowedExporterImage) { + if r.Spec.Exporter.Deployment.ExporterImage != "" && !strings.HasPrefix(r.Spec.Exporter.Deployment.ExporterImage, AllowedExporterImage) { e = append(e, - field.Invalid(field.NewPath("spec").Child("exporter").Child("image"), r.Spec.Exporter.ExporterImage, + field.Invalid(field.NewPath("spec").Child("exporter").Child("image"), r.Spec.Exporter.Deployment.ExporterImage, ErrorSpecExporterImageNotAllowed)) } @@ -165,9 +165,9 @@ func (r *DatabaseObserver) ValidateUpdate(old runtime.Object) (admission.Warning var e field.ErrorList // disallow usage of any other image than the observability-exporter - if r.Spec.Exporter.ExporterImage != "" && !strings.HasPrefix(r.Spec.Exporter.ExporterImage, AllowedExporterImage) { + if r.Spec.Exporter.Deployment.ExporterImage != "" && !strings.HasPrefix(r.Spec.Exporter.Deployment.ExporterImage, AllowedExporterImage) { e = append(e, - field.Invalid(field.NewPath("spec").Child("exporter").Child("image"), r.Spec.Exporter.ExporterImage, + field.Invalid(field.NewPath("spec").Child("exporter").Child("image"), r.Spec.Exporter.Deployment.ExporterImage, ErrorSpecExporterImageNotAllowed)) } // Return if any errors diff --git a/apis/observability/v1alpha1/zz_generated.deepcopy.go b/apis/observability/v1alpha1/zz_generated.deepcopy.go index 39b438eb..4b2a29b0 100644 --- a/apis/observability/v1alpha1/zz_generated.deepcopy.go +++ b/apis/observability/v1alpha1/zz_generated.deepcopy.go @@ -1,5 +1,4 @@ //go:build !ignore_autogenerated -// +build !ignore_autogenerated /* ** Copyright (c) 2022 Oracle and/or its affiliates. @@ -44,7 +43,9 @@ package v1alpha1 import ( - "k8s.io/apimachinery/pkg/apis/meta/v1" + monitoringv1 "github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring/v1" + "k8s.io/api/core/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/runtime" ) @@ -155,11 +156,56 @@ func (in *DatabaseObserverDatabase) DeepCopy() *DatabaseObserverDatabase { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DatabaseObserverDeployment) DeepCopyInto(out *DatabaseObserverDeployment) { + *out = *in + if in.SecurityContext != nil { + in, out := &in.SecurityContext, &out.SecurityContext + *out = new(v1.SecurityContext) + (*in).DeepCopyInto(*out) + } + if in.ExporterArgs != nil { + in, out := &in.ExporterArgs, &out.ExporterArgs + *out = make([]string, len(*in)) + copy(*out, *in) + } + if in.ExporterCommands != nil { + in, out := &in.ExporterCommands, &out.ExporterCommands + *out = make([]string, len(*in)) + copy(*out, *in) + } + if in.ExporterEnvs != nil { + in, out := &in.ExporterEnvs, &out.ExporterEnvs + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + if in.Labels != nil { + in, out := &in.Labels, &out.Labels + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + in.DeploymentPodTemplate.DeepCopyInto(&out.DeploymentPodTemplate) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserverDeployment. +func (in *DatabaseObserverDeployment) DeepCopy() *DatabaseObserverDeployment { + if in == nil { + return nil + } + out := new(DatabaseObserverDeployment) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *DatabaseObserverExporterConfig) DeepCopyInto(out *DatabaseObserverExporterConfig) { *out = *in - out.ExporterConfig = in.ExporterConfig - out.Service = in.Service + in.Deployment.DeepCopyInto(&out.Deployment) + in.Service.DeepCopyInto(&out.Service) } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserverExporterConfig. @@ -207,6 +253,20 @@ func (in *DatabaseObserverList) DeepCopyObject() runtime.Object { // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *DatabaseObserverService) DeepCopyInto(out *DatabaseObserverService) { *out = *in + if in.Ports != nil { + in, out := &in.Ports, &out.Ports + *out = make([]v1.ServicePort, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } + if in.Labels != nil { + in, out := &in.Labels, &out.Labels + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserverService. @@ -223,9 +283,30 @@ func (in *DatabaseObserverService) DeepCopy() *DatabaseObserverService { func (in *DatabaseObserverSpec) DeepCopyInto(out *DatabaseObserverSpec) { *out = *in out.Database = in.Database - out.Exporter = in.Exporter + in.Exporter.DeepCopyInto(&out.Exporter) + out.ExporterConfig = in.ExporterConfig in.Prometheus.DeepCopyInto(&out.Prometheus) out.OCIConfig = in.OCIConfig + out.Log = in.Log + if in.InheritLabels != nil { + in, out := &in.InheritLabels, &out.InheritLabels + *out = make([]string, len(*in)) + copy(*out, *in) + } + if in.ExporterSidecars != nil { + in, out := &in.ExporterSidecars, &out.ExporterSidecars + *out = make([]v1.Container, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } + if in.SideCarVolumes != nil { + in, out := &in.SideCarVolumes, &out.SideCarVolumes + *out = make([]v1.Volume, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserverSpec. @@ -243,7 +324,7 @@ func (in *DatabaseObserverStatus) DeepCopyInto(out *DatabaseObserverStatus) { *out = *in if in.Conditions != nil { in, out := &in.Conditions, &out.Conditions - *out = make([]v1.Condition, len(*in)) + *out = make([]metav1.Condition, len(*in)) for i := range *in { (*in)[i].DeepCopyInto(&(*out)[i]) } @@ -260,6 +341,80 @@ func (in *DatabaseObserverStatus) DeepCopy() *DatabaseObserverStatus { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DeploymentPodTemplate) DeepCopyInto(out *DeploymentPodTemplate) { + *out = *in + if in.SecurityContext != nil { + in, out := &in.SecurityContext, &out.SecurityContext + *out = new(v1.PodSecurityContext) + (*in).DeepCopyInto(*out) + } + if in.Labels != nil { + in, out := &in.Labels, &out.Labels + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DeploymentPodTemplate. +func (in *DeploymentPodTemplate) DeepCopy() *DeploymentPodTemplate { + if in == nil { + return nil + } + out := new(DeploymentPodTemplate) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *LogConfig) DeepCopyInto(out *LogConfig) { + *out = *in + out.Volume = in.Volume +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new LogConfig. +func (in *LogConfig) DeepCopy() *LogConfig { + if in == nil { + return nil + } + out := new(LogConfig) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *LogVolume) DeepCopyInto(out *LogVolume) { + *out = *in + out.PersistentVolumeClaim = in.PersistentVolumeClaim +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new LogVolume. +func (in *LogVolume) DeepCopy() *LogVolume { + if in == nil { + return nil + } + out := new(LogVolume) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *LogVolumePVClaim) DeepCopyInto(out *LogVolumePVClaim) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new LogVolumePVClaim. +func (in *LogVolumePVClaim) DeepCopy() *LogVolumePVClaim { + if in == nil { + return nil + } + out := new(LogVolumePVClaim) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *OCIConfigSpec) DeepCopyInto(out *OCIConfigSpec) { *out = *in @@ -277,6 +432,22 @@ func (in *OCIConfigSpec) DeepCopy() *OCIConfigSpec { // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *PrometheusConfig) DeepCopyInto(out *PrometheusConfig) { + *out = *in + in.ServiceMonitor.DeepCopyInto(&out.ServiceMonitor) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PrometheusConfig. +func (in *PrometheusConfig) DeepCopy() *PrometheusConfig { + if in == nil { + return nil + } + out := new(PrometheusConfig) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PrometheusServiceMonitor) DeepCopyInto(out *PrometheusServiceMonitor) { *out = *in if in.Labels != nil { in, out := &in.Labels, &out.Labels @@ -285,14 +456,26 @@ func (in *PrometheusConfig) DeepCopyInto(out *PrometheusConfig) { (*out)[key] = val } } + if in.NamespaceSelector != nil { + in, out := &in.NamespaceSelector, &out.NamespaceSelector + *out = new(monitoringv1.NamespaceSelector) + (*in).DeepCopyInto(*out) + } + if in.Endpoints != nil { + in, out := &in.Endpoints, &out.Endpoints + *out = make([]monitoringv1.Endpoint, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } } -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PrometheusConfig. -func (in *PrometheusConfig) DeepCopy() *PrometheusConfig { +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PrometheusServiceMonitor. +func (in *PrometheusServiceMonitor) DeepCopy() *PrometheusServiceMonitor { if in == nil { return nil } - out := new(PrometheusConfig) + out := new(PrometheusServiceMonitor) in.DeepCopyInto(out) return out } diff --git a/apis/observability/v4/databaseobserver_types.go b/apis/observability/v4/databaseobserver_types.go new file mode 100644 index 00000000..2b9df606 --- /dev/null +++ b/apis/observability/v4/databaseobserver_types.go @@ -0,0 +1,196 @@ +/* +** Copyright (c) 2024 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v4 + +import ( + monitorv1 "github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring/v1" + corev1 "k8s.io/api/core/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +type StatusEnum string + +// DatabaseObserverSpec defines the desired state of DatabaseObserver +type DatabaseObserverSpec struct { + Database DatabaseObserverDatabase `json:"database,omitempty"` + Exporter DatabaseObserverExporterConfig `json:"exporter,omitempty"` + ExporterConfig DatabaseObserverConfigMap `json:"configuration,omitempty"` + Prometheus PrometheusConfig `json:"prometheus,omitempty"` + OCIConfig OCIConfigSpec `json:"ociConfig,omitempty"` + Replicas int32 `json:"replicas,omitempty"` + Log LogConfig `json:"log,omitempty"` + InheritLabels []string `json:"inheritLabels,omitempty"` + ExporterSidecars []corev1.Container `json:"sidecars,omitempty"` + SideCarVolumes []corev1.Volume `json:"sidecarVolumes,omitempty"` +} + +// LogConfig defines the configuration details relation to the logs of DatabaseObserver +type LogConfig struct { + Path string `json:"path,omitempty"` + Filename string `json:"filename,omitempty"` + Volume LogVolume `json:"volume,omitempty"` +} + +type LogVolume struct { + Name string `json:"name,omitempty"` + PersistentVolumeClaim LogVolumePVClaim `json:"persistentVolumeClaim,omitempty"` +} + +type LogVolumePVClaim struct { + ClaimName string `json:"claimName,omitempty"` +} + +// DatabaseObserverDatabase defines the database details used for DatabaseObserver +type DatabaseObserverDatabase struct { + DBUser DBSecret `json:"dbUser,omitempty"` + DBPassword DBSecretWithVault `json:"dbPassword,omitempty"` + DBWallet DBSecret `json:"dbWallet,omitempty"` + DBConnectionString DBSecret `json:"dbConnectionString,omitempty"` +} + +// DatabaseObserverExporterConfig defines the configuration details related to the exporters of DatabaseObserver +type DatabaseObserverExporterConfig struct { + Deployment DatabaseObserverDeployment `json:"deployment,omitempty"` + Service DatabaseObserverService `json:"service,omitempty"` +} + +// DatabaseObserverDeployment defines the exporter deployment component of DatabaseObserver +type DatabaseObserverDeployment struct { + ExporterImage string `json:"image,omitempty"` + SecurityContext *corev1.SecurityContext `json:"securityContext,omitempty"` + ExporterArgs []string `json:"args,omitempty"` + ExporterCommands []string `json:"commands,omitempty"` + ExporterEnvs map[string]string `json:"env,omitempty"` + Labels map[string]string `json:"labels,omitempty"` + DeploymentPodTemplate DeploymentPodTemplate `json:"podTemplate,omitempty"` +} + +// DeploymentPodTemplate defines the labels for the DatabaseObserver pods component of a deployment +type DeploymentPodTemplate struct { + SecurityContext *corev1.PodSecurityContext `json:"securityContext,omitempty"` + Labels map[string]string `json:"labels,omitempty"` +} + +// DatabaseObserverService defines the exporter service component of DatabaseObserver +type DatabaseObserverService struct { + Ports []corev1.ServicePort `json:"ports,omitempty"` + Labels map[string]string `json:"labels,omitempty"` +} + +// PrometheusConfig defines the generated resources for Prometheus +type PrometheusConfig struct { + ServiceMonitor PrometheusServiceMonitor `json:"serviceMonitor,omitempty"` +} + +// PrometheusServiceMonitor defines DatabaseObserver servicemonitor spec +type PrometheusServiceMonitor struct { + Labels map[string]string `json:"labels,omitempty"` + NamespaceSelector *monitorv1.NamespaceSelector `json:"namespaceSelector,omitempty"` + Endpoints []monitorv1.Endpoint `json:"endpoints,omitempty"` +} + +// DBSecret defines secrets used in reference +type DBSecret struct { + Key string `json:"key,omitempty"` + SecretName string `json:"secret,omitempty"` +} + +// DBSecretWithVault defines secrets used in reference with vault fields +type DBSecretWithVault struct { + Key string `json:"key,omitempty"` + SecretName string `json:"secret,omitempty"` + VaultOCID string `json:"vaultOCID,omitempty"` + VaultSecretName string `json:"vaultSecretName,omitempty"` +} + +// DatabaseObserverConfigMap defines configMap used for metrics configuration +type DatabaseObserverConfigMap struct { + Configmap ConfigMapDetails `json:"configMap,omitempty"` +} + +// ConfigMapDetails defines the configmap name +type ConfigMapDetails struct { + Key string `json:"key,omitempty"` + Name string `json:"name,omitempty"` +} + +// OCIConfigSpec defines the configmap name and secret name used for connecting to OCI +type OCIConfigSpec struct { + ConfigMapName string `json:"configMapName,omitempty"` + SecretName string `json:"secretName,omitempty"` +} + +// DatabaseObserverStatus defines the observed state of DatabaseObserver +type DatabaseObserverStatus struct { + Conditions []metav1.Condition `json:"conditions"` + Status string `json:"status,omitempty"` + ExporterConfig string `json:"exporterConfig"` + Version string `json:"version"` + Replicas int `json:"replicas,omitempty"` +} + +//+kubebuilder:object:root=true +//+kubebuilder:subresource:status +// +kubebuilder:resource:shortName="dbobserver";"dbobservers" + +// DatabaseObserver is the Schema for the databaseobservers API +// +kubebuilder:printcolumn:JSONPath=".status.exporterConfig",name="ExporterConfig",type=string +// +kubebuilder:printcolumn:JSONPath=".status.status",name="Status",type=string +// +kubebuilder:printcolumn:JSONPath=".status.version",name="Version",type=string +// +kubebuilder:storageversion +type DatabaseObserver struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + Spec DatabaseObserverSpec `json:"spec,omitempty"` + Status DatabaseObserverStatus `json:"status,omitempty"` +} + +//+kubebuilder:object:root=true + +// DatabaseObserverList contains a list of DatabaseObserver +type DatabaseObserverList struct { + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata,omitempty"` + Items []DatabaseObserver `json:"items"` +} + +func init() { + SchemeBuilder.Register(&DatabaseObserver{}, &DatabaseObserverList{}) +} diff --git a/apis/observability/v4/databaseobserver_webhook.go b/apis/observability/v4/databaseobserver_webhook.go new file mode 100644 index 00000000..c0a5d8b7 --- /dev/null +++ b/apis/observability/v4/databaseobserver_webhook.go @@ -0,0 +1,182 @@ +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v4 + +import ( + dbcommons "github.com/oracle/oracle-database-operator/commons/database" + apierrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/runtime/schema" + "k8s.io/apimachinery/pkg/util/validation/field" + ctrl "sigs.k8s.io/controller-runtime" + logf "sigs.k8s.io/controller-runtime/pkg/log" + "sigs.k8s.io/controller-runtime/pkg/webhook" + "sigs.k8s.io/controller-runtime/pkg/webhook/admission" + "strings" +) + +// log is for logging in this package. +var databaseobserverlog = logf.Log.WithName("databaseobserver-resource") + +const ( + AllowedExporterImage = "container-registry.oracle.com/database/observability-exporter" + ErrorSpecValidationMissingConnString = "a required field for database connection string secret is missing or does not have a value" + ErrorSpecValidationMissingDBUser = "a required field for database user secret is missing or does not have a value" + ErrorSpecValidationMissingDBVaultField = "a field for the OCI vault has a value but the other required field is missing or does not have a value" + ErrorSpecValidationMissingOCIConfig = "a field(s) for the OCI Config is missing or does not have a value when fields for the OCI vault has values" + ErrorSpecValidationMissingDBPasswordSecret = "a required field for the database password secret is missing or does not have a value" + ErrorSpecExporterImageNotAllowed = "a different exporter image was found, only official database exporter container images are currently supported" +) + +func (r *DatabaseObserver) SetupWebhookWithManager(mgr ctrl.Manager) error { + return ctrl.NewWebhookManagedBy(mgr). + For(r). + Complete() +} + +//+kubebuilder:webhook:path=/mutate-observability-oracle-com-v4-databaseobserver,mutating=true,sideEffects=none,failurePolicy=fail,groups=observability.oracle.com,resources=databaseobservers,verbs=create;update,versions=v4,name=mdatabaseobserver.kb.io,admissionReviewVersions=v1 + +var _ webhook.Defaulter = &DatabaseObserver{} + +// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! + +// Default implements webhook.Defaulter so a webhook will be registered for the type +func (r *DatabaseObserver) Default() { + databaseobserverlog.Info("default", "name", r.Name) +} + +//+kubebuilder:webhook:verbs=create;update,path=/validate-observability-oracle-com-v4-databaseobserver,mutating=false,sideEffects=none,failurePolicy=fail,groups=observability.oracle.com,resources=databaseobservers,versions=v4,name=vdatabaseobserver.kb.io,admissionReviewVersions=v1 + +var _ webhook.Validator = &DatabaseObserver{} + +// ValidateCreate implements webhook.Validator so a webhook will be registered for the type +func (r *DatabaseObserver) ValidateCreate() (admission.Warnings, error) { + databaseobserverlog.Info("validate create", "name", r.Name) + + var e field.ErrorList + ns := dbcommons.GetWatchNamespaces() + + // Check for namespace/cluster scope access + if _, isDesiredNamespaceWithinScope := ns[r.Namespace]; !isDesiredNamespaceWithinScope && len(ns) > 0 { + e = append(e, + field.Invalid(field.NewPath("metadata").Child("namespace"), r.Namespace, + "Oracle database operator doesn't watch over this namespace")) + } + + // Check required secret for db user has value + if r.Spec.Database.DBUser.SecretName == "" { + e = append(e, + field.Invalid(field.NewPath("spec").Child("database").Child("dbUser").Child("secret"), r.Spec.Database.DBUser.SecretName, + ErrorSpecValidationMissingDBUser)) + } + + // Check required secret for db connection string has value + if r.Spec.Database.DBConnectionString.SecretName == "" { + e = append(e, + field.Invalid(field.NewPath("spec").Child("database").Child("dbConnectionString").Child("secret"), r.Spec.Database.DBConnectionString.SecretName, + ErrorSpecValidationMissingConnString)) + } + + // The other vault field must have value if one does + if (r.Spec.Database.DBPassword.VaultOCID != "" && r.Spec.Database.DBPassword.VaultSecretName == "") || + (r.Spec.Database.DBPassword.VaultSecretName != "" && r.Spec.Database.DBPassword.VaultOCID == "") { + + e = append(e, + field.Invalid(field.NewPath("spec").Child("database").Child("dbPassword"), r.Spec.Database.DBPassword, + ErrorSpecValidationMissingDBVaultField)) + } + + // if vault fields have value, ociConfig must have values + if r.Spec.Database.DBPassword.VaultOCID != "" && r.Spec.Database.DBPassword.VaultSecretName != "" && + (r.Spec.OCIConfig.SecretName == "" || r.Spec.OCIConfig.ConfigMapName == "") { + + e = append(e, + field.Invalid(field.NewPath("spec").Child("ociConfig"), r.Spec.OCIConfig, + ErrorSpecValidationMissingOCIConfig)) + } + + // If all of {DB Password Secret Name and vaultOCID+vaultSecretName} have no value, then error out + if r.Spec.Database.DBPassword.SecretName == "" && + r.Spec.Database.DBPassword.VaultOCID == "" && + r.Spec.Database.DBPassword.VaultSecretName == "" { + + e = append(e, + field.Invalid(field.NewPath("spec").Child("database").Child("dbPassword").Child("secret"), r.Spec.Database.DBPassword.SecretName, + ErrorSpecValidationMissingDBPasswordSecret)) + } + + // disallow usage of any other image than the observability-exporter + if r.Spec.Exporter.Deployment.ExporterImage != "" && !strings.HasPrefix(r.Spec.Exporter.Deployment.ExporterImage, AllowedExporterImage) { + e = append(e, + field.Invalid(field.NewPath("spec").Child("exporter").Child("image"), r.Spec.Exporter.Deployment.ExporterImage, + ErrorSpecExporterImageNotAllowed)) + } + + // Return if any errors + if len(e) > 0 { + return nil, apierrors.NewInvalid(schema.GroupKind{Group: "observability.oracle.com", Kind: "DatabaseObserver"}, r.Name, e) + } + return nil, nil + +} + +// ValidateUpdate implements webhook.Validator so a webhook will be registered for the type +func (r *DatabaseObserver) ValidateUpdate(old runtime.Object) (admission.Warnings, error) { + databaseobserverlog.Info("validate update", "name", r.Name) + var e field.ErrorList + + // disallow usage of any other image than the observability-exporter + if r.Spec.Exporter.Deployment.ExporterImage != "" && !strings.HasPrefix(r.Spec.Exporter.Deployment.ExporterImage, AllowedExporterImage) { + e = append(e, + field.Invalid(field.NewPath("spec").Child("exporter").Child("image"), r.Spec.Exporter.Deployment.ExporterImage, + ErrorSpecExporterImageNotAllowed)) + } + // Return if any errors + if len(e) > 0 { + return nil, apierrors.NewInvalid(schema.GroupKind{Group: "observability.oracle.com", Kind: "DatabaseObserver"}, r.Name, e) + } + return nil, nil +} + +// ValidateDelete implements webhook.Validator so a webhook will be registered for the type +func (r *DatabaseObserver) ValidateDelete() (admission.Warnings, error) { + databaseobserverlog.Info("validate delete", "name", r.Name) + + return nil, nil +} diff --git a/apis/observability/v4/groupversion_info.go b/apis/observability/v4/groupversion_info.go new file mode 100644 index 00000000..155b1c11 --- /dev/null +++ b/apis/observability/v4/groupversion_info.go @@ -0,0 +1,58 @@ +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +// Package v4 contains API Schema definitions for the observability v4 API group +// +kubebuilder:object:generate=true +// +groupName=observability.oracle.com +package v4 + +import ( + "k8s.io/apimachinery/pkg/runtime/schema" + "sigs.k8s.io/controller-runtime/pkg/scheme" +) + +var ( + // GroupVersion is group version used to register these objects + GroupVersion = schema.GroupVersion{Group: "observability.oracle.com", Version: "v4"} + + // SchemeBuilder is used to add go types to the GroupVersionKind scheme + SchemeBuilder = &scheme.Builder{GroupVersion: GroupVersion} + + // AddToScheme adds the types in this group-version to the given scheme. + AddToScheme = SchemeBuilder.AddToScheme +) diff --git a/apis/observability/v4/zz_generated.deepcopy.go b/apis/observability/v4/zz_generated.deepcopy.go new file mode 100644 index 00000000..d9892643 --- /dev/null +++ b/apis/observability/v4/zz_generated.deepcopy.go @@ -0,0 +1,481 @@ +//go:build !ignore_autogenerated + +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +// Code generated by controller-gen. DO NOT EDIT. + +package v4 + +import ( + monitoringv1 "github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring/v1" + "k8s.io/api/core/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/runtime" +) + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *ConfigMapDetails) DeepCopyInto(out *ConfigMapDetails) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ConfigMapDetails. +func (in *ConfigMapDetails) DeepCopy() *ConfigMapDetails { + if in == nil { + return nil + } + out := new(ConfigMapDetails) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DBSecret) DeepCopyInto(out *DBSecret) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DBSecret. +func (in *DBSecret) DeepCopy() *DBSecret { + if in == nil { + return nil + } + out := new(DBSecret) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DBSecretWithVault) DeepCopyInto(out *DBSecretWithVault) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DBSecretWithVault. +func (in *DBSecretWithVault) DeepCopy() *DBSecretWithVault { + if in == nil { + return nil + } + out := new(DBSecretWithVault) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DatabaseObserver) DeepCopyInto(out *DatabaseObserver) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + in.Spec.DeepCopyInto(&out.Spec) + in.Status.DeepCopyInto(&out.Status) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserver. +func (in *DatabaseObserver) DeepCopy() *DatabaseObserver { + if in == nil { + return nil + } + out := new(DatabaseObserver) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *DatabaseObserver) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DatabaseObserverConfigMap) DeepCopyInto(out *DatabaseObserverConfigMap) { + *out = *in + out.Configmap = in.Configmap +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserverConfigMap. +func (in *DatabaseObserverConfigMap) DeepCopy() *DatabaseObserverConfigMap { + if in == nil { + return nil + } + out := new(DatabaseObserverConfigMap) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DatabaseObserverDatabase) DeepCopyInto(out *DatabaseObserverDatabase) { + *out = *in + out.DBUser = in.DBUser + out.DBPassword = in.DBPassword + out.DBWallet = in.DBWallet + out.DBConnectionString = in.DBConnectionString +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserverDatabase. +func (in *DatabaseObserverDatabase) DeepCopy() *DatabaseObserverDatabase { + if in == nil { + return nil + } + out := new(DatabaseObserverDatabase) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DatabaseObserverDeployment) DeepCopyInto(out *DatabaseObserverDeployment) { + *out = *in + if in.SecurityContext != nil { + in, out := &in.SecurityContext, &out.SecurityContext + *out = new(v1.SecurityContext) + (*in).DeepCopyInto(*out) + } + if in.ExporterArgs != nil { + in, out := &in.ExporterArgs, &out.ExporterArgs + *out = make([]string, len(*in)) + copy(*out, *in) + } + if in.ExporterCommands != nil { + in, out := &in.ExporterCommands, &out.ExporterCommands + *out = make([]string, len(*in)) + copy(*out, *in) + } + if in.ExporterEnvs != nil { + in, out := &in.ExporterEnvs, &out.ExporterEnvs + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + if in.Labels != nil { + in, out := &in.Labels, &out.Labels + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + in.DeploymentPodTemplate.DeepCopyInto(&out.DeploymentPodTemplate) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserverDeployment. +func (in *DatabaseObserverDeployment) DeepCopy() *DatabaseObserverDeployment { + if in == nil { + return nil + } + out := new(DatabaseObserverDeployment) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DatabaseObserverExporterConfig) DeepCopyInto(out *DatabaseObserverExporterConfig) { + *out = *in + in.Deployment.DeepCopyInto(&out.Deployment) + in.Service.DeepCopyInto(&out.Service) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserverExporterConfig. +func (in *DatabaseObserverExporterConfig) DeepCopy() *DatabaseObserverExporterConfig { + if in == nil { + return nil + } + out := new(DatabaseObserverExporterConfig) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DatabaseObserverList) DeepCopyInto(out *DatabaseObserverList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]DatabaseObserver, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserverList. +func (in *DatabaseObserverList) DeepCopy() *DatabaseObserverList { + if in == nil { + return nil + } + out := new(DatabaseObserverList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *DatabaseObserverList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DatabaseObserverService) DeepCopyInto(out *DatabaseObserverService) { + *out = *in + if in.Ports != nil { + in, out := &in.Ports, &out.Ports + *out = make([]v1.ServicePort, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } + if in.Labels != nil { + in, out := &in.Labels, &out.Labels + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserverService. +func (in *DatabaseObserverService) DeepCopy() *DatabaseObserverService { + if in == nil { + return nil + } + out := new(DatabaseObserverService) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DatabaseObserverSpec) DeepCopyInto(out *DatabaseObserverSpec) { + *out = *in + out.Database = in.Database + in.Exporter.DeepCopyInto(&out.Exporter) + out.ExporterConfig = in.ExporterConfig + in.Prometheus.DeepCopyInto(&out.Prometheus) + out.OCIConfig = in.OCIConfig + out.Log = in.Log + if in.InheritLabels != nil { + in, out := &in.InheritLabels, &out.InheritLabels + *out = make([]string, len(*in)) + copy(*out, *in) + } + if in.ExporterSidecars != nil { + in, out := &in.ExporterSidecars, &out.ExporterSidecars + *out = make([]v1.Container, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } + if in.SideCarVolumes != nil { + in, out := &in.SideCarVolumes, &out.SideCarVolumes + *out = make([]v1.Volume, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserverSpec. +func (in *DatabaseObserverSpec) DeepCopy() *DatabaseObserverSpec { + if in == nil { + return nil + } + out := new(DatabaseObserverSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DatabaseObserverStatus) DeepCopyInto(out *DatabaseObserverStatus) { + *out = *in + if in.Conditions != nil { + in, out := &in.Conditions, &out.Conditions + *out = make([]metav1.Condition, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DatabaseObserverStatus. +func (in *DatabaseObserverStatus) DeepCopy() *DatabaseObserverStatus { + if in == nil { + return nil + } + out := new(DatabaseObserverStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *DeploymentPodTemplate) DeepCopyInto(out *DeploymentPodTemplate) { + *out = *in + if in.SecurityContext != nil { + in, out := &in.SecurityContext, &out.SecurityContext + *out = new(v1.PodSecurityContext) + (*in).DeepCopyInto(*out) + } + if in.Labels != nil { + in, out := &in.Labels, &out.Labels + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new DeploymentPodTemplate. +func (in *DeploymentPodTemplate) DeepCopy() *DeploymentPodTemplate { + if in == nil { + return nil + } + out := new(DeploymentPodTemplate) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *LogConfig) DeepCopyInto(out *LogConfig) { + *out = *in + out.Volume = in.Volume +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new LogConfig. +func (in *LogConfig) DeepCopy() *LogConfig { + if in == nil { + return nil + } + out := new(LogConfig) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *LogVolume) DeepCopyInto(out *LogVolume) { + *out = *in + out.PersistentVolumeClaim = in.PersistentVolumeClaim +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new LogVolume. +func (in *LogVolume) DeepCopy() *LogVolume { + if in == nil { + return nil + } + out := new(LogVolume) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *LogVolumePVClaim) DeepCopyInto(out *LogVolumePVClaim) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new LogVolumePVClaim. +func (in *LogVolumePVClaim) DeepCopy() *LogVolumePVClaim { + if in == nil { + return nil + } + out := new(LogVolumePVClaim) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *OCIConfigSpec) DeepCopyInto(out *OCIConfigSpec) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new OCIConfigSpec. +func (in *OCIConfigSpec) DeepCopy() *OCIConfigSpec { + if in == nil { + return nil + } + out := new(OCIConfigSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PrometheusConfig) DeepCopyInto(out *PrometheusConfig) { + *out = *in + in.ServiceMonitor.DeepCopyInto(&out.ServiceMonitor) +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PrometheusConfig. +func (in *PrometheusConfig) DeepCopy() *PrometheusConfig { + if in == nil { + return nil + } + out := new(PrometheusConfig) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PrometheusServiceMonitor) DeepCopyInto(out *PrometheusServiceMonitor) { + *out = *in + if in.Labels != nil { + in, out := &in.Labels, &out.Labels + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + if in.NamespaceSelector != nil { + in, out := &in.NamespaceSelector, &out.NamespaceSelector + *out = new(monitoringv1.NamespaceSelector) + (*in).DeepCopyInto(*out) + } + if in.Endpoints != nil { + in, out := &in.Endpoints, &out.Endpoints + *out = make([]monitoringv1.Endpoint, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PrometheusServiceMonitor. +func (in *PrometheusServiceMonitor) DeepCopy() *PrometheusServiceMonitor { + if in == nil { + return nil + } + out := new(PrometheusServiceMonitor) + in.DeepCopyInto(out) + return out +} diff --git a/commons/adb_family/utils.go b/commons/adb_family/utils.go index 8218502e..591b3130 100644 --- a/commons/adb_family/utils.go +++ b/commons/adb_family/utils.go @@ -39,35 +39,35 @@ package adbfamily import ( - dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + dbv4 "github.com/oracle/oracle-database-operator/apis/database/v4" "github.com/oracle/oracle-database-operator/commons/k8s" "sigs.k8s.io/controller-runtime/pkg/client" ) -// VerifyTargetADB searches if the target ADB is in the cluster, and set the owner reference to the ADB if it exists. +// VerifyTargetAdb searches if the target ADB is in the cluster. // The function returns two values in the following order: // ocid: the OCID of the target ADB. An empty string is returned if the ocid is nil. // ownerADB: the resource of the targetADB if it's found in the cluster -func VerifyTargetADB(kubeClient client.Client, target dbv1alpha1.TargetSpec, namespace string) (*dbv1alpha1.AutonomousDatabase, error) { +func VerifyTargetAdb(kubeClient client.Client, target dbv4.TargetSpec, namespace string) (*dbv4.AutonomousDatabase, error) { var err error - var ownerADB *dbv1alpha1.AutonomousDatabase + var ownerAdb *dbv4.AutonomousDatabase // Get the target ADB OCID - if target.K8sADB.Name != nil { + if target.K8sAdb.Name != nil { // Find the target ADB using the name of the k8s ADB - ownerADB = &dbv1alpha1.AutonomousDatabase{} - if err := k8s.FetchResource(kubeClient, namespace, *target.K8sADB.Name, ownerADB); err != nil { + ownerAdb = &dbv4.AutonomousDatabase{} + if err := k8s.FetchResource(kubeClient, namespace, *target.K8sAdb.Name, ownerAdb); err != nil { return nil, err } } else { // Find the target ADB using the ADB OCID - ownerADB, err = k8s.FetchAutonomousDatabaseWithOCID(kubeClient, namespace, *target.OCIADB.OCID) + ownerAdb, err = k8s.FetchAutonomousDatabaseWithOCID(kubeClient, namespace, *target.OciAdb.OCID) if err != nil { return nil, err } } - return ownerADB, nil + return ownerAdb, nil } diff --git a/commons/database/constants.go b/commons/database/constants.go index 6f27750d..940a2727 100644 --- a/commons/database/constants.go +++ b/commons/database/constants.go @@ -50,6 +50,8 @@ const DBA_GUID int64 = 54322 const SQLPlusCLI string = "sqlplus -s / as sysdba" +const SQLCLI string = "sql -s / as sysdba" + const GetVersionSQL string = "SELECT VERSION_FULL FROM V\\$INSTANCE;" const CheckModesSQL string = "SELECT 'log_mode:' || log_mode AS log_mode ,'flashback_on:' || flashback_on AS flashback_on ,'force_logging:' || force_logging AS force_logging FROM v\\$database;" @@ -182,10 +184,6 @@ const DataguardBrokerMaxPerformanceCMD string = "CREATE CONFIGURATION dg_config "\nADD DATABASE ${ORACLE_SID} AS CONNECT IDENTIFIER IS ${SVC_HOST}:1521/${ORACLE_SID} MAINTAINED AS PHYSICAL;" + "\nEDIT DATABASE ${PRIMARY_SID} SET PROPERTY LogXptMode='ASYNC';" + "\nEDIT DATABASE ${ORACLE_SID} SET PROPERTY LogXptMode='ASYNC';" + - "\nEDIT DATABASE ${PRIMARY_SID} SET PROPERTY STATICCONNECTIDENTIFIER='(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=${PRIMARY_IP})(PORT=1521))" + - "(CONNECT_DATA=(SERVICE_NAME=${PRIMARY_SID}_DGMGRL)(INSTANCE_NAME=${PRIMARY_SID})(SERVER=DEDICATED)))';" + - "\nEDIT DATABASE ${ORACLE_SID} SET PROPERTY STATICCONNECTIDENTIFIER='(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=${SVC_HOST})(PORT=1521))" + - "(CONNECT_DATA=(SERVICE_NAME=${ORACLE_SID}_DGMGRL)(INSTANCE_NAME=${ORACLE_SID})(SERVER=DEDICATED)))';" + "\nEDIT CONFIGURATION SET PROTECTION MODE AS MAXPERFORMANCE;" + "\nENABLE CONFIGURATION;" @@ -193,10 +191,6 @@ const DataguardBrokerMaxAvailabilityCMD string = "CREATE CONFIGURATION dg_config "\nADD DATABASE ${ORACLE_SID} AS CONNECT IDENTIFIER IS ${SVC_HOST}:1521/${ORACLE_SID} MAINTAINED AS PHYSICAL;" + "\nEDIT DATABASE ${PRIMARY_SID} SET PROPERTY LogXptMode='SYNC';" + "\nEDIT DATABASE ${ORACLE_SID} SET PROPERTY LogXptMode='SYNC';" + - "\nEDIT DATABASE ${PRIMARY_SID} SET PROPERTY STATICCONNECTIDENTIFIER='(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=${PRIMARY_IP})(PORT=1521))" + - "(CONNECT_DATA=(SERVICE_NAME=${PRIMARY_SID}_DGMGRL)(INSTANCE_NAME=${PRIMARY_SID})(SERVER=DEDICATED)))';" + - "\nEDIT DATABASE ${ORACLE_SID} SET PROPERTY STATICCONNECTIDENTIFIER='(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=${SVC_HOST})(PORT=1521))" + - "(CONNECT_DATA=(SERVICE_NAME=${ORACLE_SID}_DGMGRL)(INSTANCE_NAME=${ORACLE_SID})(SERVER=DEDICATED)))';" + "\nEDIT CONFIGURATION SET PROTECTION MODE AS MAXAVAILABILITY;" + "\nENABLE CONFIGURATION;" @@ -221,6 +215,9 @@ const DataguardBrokerGetDatabaseCMD string = "SELECT DATABASE || ':' || DATAGUAR const EnableFSFOCMD string = "ENABLE FAST_START FAILOVER;" +const DisableFSFOCMD string = "STOP OBSERVER %s" + + "\nDISABLE FAST_START FAILOVER;" + const RemoveDataguardConfiguration string = "DISABLE FAST_START FAILOVER;" + "\nEDIT CONFIGURATION SET PROTECTION MODE AS MAXPERFORMANCE;" + "\nREMOVE CONFIGURATION;" @@ -345,6 +342,8 @@ const InitORDSCMD string = "if [ -f $ORDS_HOME/config/ords/defaults.xml ]; then "\nrm -f sqladmin.passwd" + "\numask 022" +const DbConnectString string = "CONN_STRING=sys/%[1]s@%[2]s:1521/%[3]s" + const GetSessionInfoSQL string = "select s.sid || ',' || s.serial# as Info FROM v\\$session s, v\\$process p " + "WHERE (s.username = 'ORDS_PUBLIC_USER' or " + "s.username = 'APEX_PUBLIC_USER' or " + @@ -369,7 +368,9 @@ const UninstallORDSCMD string = "\numask 177" + "\nrm -rf /opt/oracle/ords/config/ords/standalone" + "\nrm -rf /opt/oracle/ords/config/ords/apex" -const GetORDSStatus string = "curl -sSkv -k -X GET https://localhost:8443/ords/_/db-api/stable/metadata-catalog/" +const GetORDSStatus string = "curl -sSkvf -k -X GET http://localhost:8181/ords/_/db-api/stable/metadata-catalog/" + +const ORDSReadinessProbe string = "curl -sSkvf -k -X GET http://localhost:8181/ords/_/landing" const ValidateAdminPassword string = "conn sys/\\\"%s\\\"@${ORACLE_SID} as sysdba\nshow user" @@ -403,6 +404,8 @@ const StatusReady string = "Healthy" const StatusError string = "Error" +const StatusUnknown string = "Unknown" + const ValueUnavailable string = "Unavailable" const NoExternalIp string = "Node ExternalIP unavailable" @@ -442,18 +445,18 @@ const ChownApex string = " chown oracle:oinstall /opt/oracle/oradata/${ORACLE_SI const InstallApex string = "if [ -f /opt/oracle/oradata/${ORACLE_SID^^}/apex/apexins.sql ]; then ( while true; do sleep 60; echo \"Installing Apex...\" ; done ) & " + " cd /opt/oracle/oradata/${ORACLE_SID^^}/apex && echo -e \"@apexins.sql SYSAUX SYSAUX TEMP /i/\" | %[1]s && kill -9 $!; else echo \"Apex Folder doesn't exist\" ; fi ;" -const InstallApexInContainer string = "cd ${ORDS_HOME}/config/apex/ && echo -e \"@apxsilentins.sql SYSAUX SYSAUX TEMP /i/ %[1]s %[1]s %[1]s %[1]s;\n" + +const InstallApexInContainer string = "cd ${APEX_HOME}/${APEX_VER} && echo -e \"@apxsilentins.sql SYSAUX SYSAUX TEMP /i/ %[1]s %[1]s %[1]s %[1]s;\n" + "@apex_rest_config_core.sql;\n" + "exec APEX_UTIL.set_workspace(p_workspace => 'INTERNAL');\n" + "exec APEX_UTIL.EDIT_USER(p_user_id => APEX_UTIL.GET_USER_ID('ADMIN'), p_user_name => 'ADMIN', p_change_password_on_first_use => 'Y');\n" + - "\" | sqlplus -s sys/%[2]s@${ORACLE_HOST}:${ORACLE_PORT}/%[3]s as sysdba;" + "\" | sql -s sys/%[2]s@${ORACLE_HOST}:${ORACLE_PORT}/%[3]s as sysdba;" const IsApexInstalled string = "echo -e \"select 'APEXVERSION:'||version as version FROM DBA_REGISTRY WHERE COMP_ID='APEX';\"" + - " | sqlplus -s sys/%[1]s@${ORACLE_HOST}:${ORACLE_PORT}/%[2]s as sysdba;" + " | sql -s sys/%[1]s@${ORACLE_HOST}:${ORACLE_PORT}/%[2]s as sysdba;" -const UninstallApex string = "cd ${ORDS_HOME}/config/apex/ && echo -e \"@apxremov.sql\n\" | sqlplus -s sys/%[1]s@${ORACLE_HOST}:${ORACLE_PORT}/%[2]s as sysdba;" +const UninstallApex string = "cd ${APEX_HOME}/${APEX_VER} && echo -e \"@apxremov.sql\n\" | sql -s sys/%[1]s@${ORACLE_HOST}:${ORACLE_PORT}/%[2]s as sysdba;" -const ConfigureApexRest string = "if [ -f ${ORDS_HOME}/config/apex/apex_rest_config.sql ]; then cd ${ORDS_HOME}/config/apex && " + +const ConfigureApexRest string = "if [ -f ${APEX_HOME}/${APEX_VER}/apex_rest_config.sql ]; then cd ${ORDS_HOME}/config/apex && " + "echo -e \"%[1]s\n%[1]s\" | %[2]s ; else echo \"Apex Folder doesn't exist\" ; fi ;" const AlterApexUsers string = "\nALTER SESSION SET CONTAINER=%[2]s;" + @@ -508,6 +511,9 @@ const SetApexUsers string = "\numask 177" + "\nrm -f apexPublicUser" + "\numask 022" +// Command to enable/disable MongoDB API support in ords pods +const ConfigMongoDb string = "ords config set mongo.enabled %[1]s" + // Get Sid, Pdbname, Edition for prebuilt db const GetSidPdbEditionCMD string = "echo $ORACLE_SID,$ORACLE_PDB,$ORACLE_EDITION;" diff --git a/commons/database/podbuilder.go b/commons/database/podbuilder.go new file mode 100644 index 00000000..c704c4fc --- /dev/null +++ b/commons/database/podbuilder.go @@ -0,0 +1,108 @@ +/* +** Copyright (c) 2024 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package commons + +import ( + "k8s.io/apimachinery/pkg/types" + + corev1 "k8s.io/api/core/v1" +) + +type PodBuilder interface { + SetNamespacedName(types.NamespacedName) *PodBuilder + SetLabels(map[string]string) *PodBuilder + SetTerminationGracePeriodSeconds(int64) *PodBuilder + SetNodeSelector(map[string]string) *PodBuilder + SetSecurityContext(corev1.PodSecurityContext) *PodBuilder + SetImagePullSecrets(string) *PodBuilder + AppendContainers(corev1.Container) *PodBuilder + Build() corev1.Pod +} + +type RealPodBuilder struct { + pod corev1.Pod +} + +func (rpb *RealPodBuilder) SetNamespacedName(namespacedName types.NamespacedName) *RealPodBuilder { + rpb.pod.ObjectMeta.Name = namespacedName.Name + rpb.pod.ObjectMeta.Namespace = namespacedName.Namespace + return rpb +} + +func (rpb *RealPodBuilder) SetLabels(labels map[string]string) *RealPodBuilder { + rpb.pod.ObjectMeta.Labels = labels + return rpb +} + +func (rpb *RealPodBuilder) SetTerminationGracePeriodSeconds(terminationGracePeriod int64) *RealPodBuilder { + rpb.pod.Spec.TerminationGracePeriodSeconds = &terminationGracePeriod + return rpb +} + +func (rpb *RealPodBuilder) SetNodeSelector(nsRule map[string]string) *RealPodBuilder { + rpb.pod.Spec.NodeSelector = nsRule + return rpb +} + +func (rpb *RealPodBuilder) SetSecurityContext(podSecurityContext corev1.PodSecurityContext) *RealPodBuilder { + rpb.pod.Spec.SecurityContext = &podSecurityContext + return rpb +} + +func (rpb *RealPodBuilder) SetImagePullSecrets(imagePullSecret string) *RealPodBuilder { + rpb.pod.Spec.ImagePullSecrets = []corev1.LocalObjectReference{ + { + Name: imagePullSecret, + }, + } + return rpb +} + +func (rpb *RealPodBuilder) AppendContainers(container corev1.Container) *RealPodBuilder { + rpb.pod.Spec.Containers = append(rpb.pod.Spec.Containers, container) + return rpb +} + +func (rpb *RealPodBuilder) Build() corev1.Pod { + return rpb.pod +} + +func NewRealPodBuilder() *RealPodBuilder { + return &RealPodBuilder{} +} diff --git a/commons/database/svcbuilder.go b/commons/database/svcbuilder.go new file mode 100644 index 00000000..8029c8ee --- /dev/null +++ b/commons/database/svcbuilder.go @@ -0,0 +1,99 @@ +/* +** Copyright (c) 2024 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package commons + +import ( + corev1 "k8s.io/api/core/v1" +) + +type ServiceBuilder interface { + SetName(string) *ServiceBuilder + SetNamespace(string) *ServiceBuilder + SetLabels(map[string]string) *ServiceBuilder + SetAnnotation(map[string]string) *ServiceBuilder + SetPorts([]corev1.ServicePort) *ServiceBuilder + SetSelector(map[string]string) *ServiceBuilder + SetPublishNotReadyAddresses(bool) *ServiceBuilder + SetServiceType(corev1.ServiceType) *ServiceBuilder + Build() *corev1.Service +} + +type RealServiceBuilder struct { + service corev1.Service +} + +func (rsb *RealServiceBuilder) SetName(name string) *RealServiceBuilder { + rsb.service.ObjectMeta.Name = name + return rsb +} +func (rsb *RealServiceBuilder) SetNamespace(namespace string) *RealServiceBuilder { + rsb.service.ObjectMeta.Namespace = namespace + return rsb +} +func (rsb *RealServiceBuilder) SetLabels(labels map[string]string) *RealServiceBuilder { + rsb.service.ObjectMeta.Labels = labels + return rsb +} +func (rsb *RealServiceBuilder) SetAnnotation(annotations map[string]string) *RealServiceBuilder { + rsb.service.ObjectMeta.Annotations = annotations + return rsb +} +func (rsb *RealServiceBuilder) SetPorts(ports []corev1.ServicePort) *RealServiceBuilder { + rsb.service.Spec.Ports = ports + return rsb +} +func (rsb *RealServiceBuilder) SetSelector(selector map[string]string) *RealServiceBuilder { + rsb.service.Spec.Selector = selector + return rsb +} +func (rsb *RealServiceBuilder) SetPublishNotReadyAddresses(flag bool) *RealServiceBuilder { + rsb.service.Spec.PublishNotReadyAddresses = flag + return rsb +} +func (rsb *RealServiceBuilder) SetType(serviceType corev1.ServiceType) *RealServiceBuilder { + rsb.service.Spec.Type = serviceType + return rsb +} +func (rsb *RealServiceBuilder) Build() corev1.Service { + return rsb.service +} + +func NewRealServiceBuilder() *RealServiceBuilder { + return &RealServiceBuilder{} +} diff --git a/commons/database/utils.go b/commons/database/utils.go index 1723bc90..e0536642 100644 --- a/commons/database/utils.go +++ b/commons/database/utils.go @@ -502,32 +502,6 @@ func GetPrimaryDatabase(databases []string) string { return primary } -// Returns the databases in DG config . -func GetDatabasesInDgConfig(readyPod corev1.Pod, r client.Reader, - config *rest.Config, ctx context.Context, req ctrl.Request) ([]string, string, error) { - log := ctrllog.FromContext(ctx).WithValues("GetDatabasesInDgConfig", req.NamespacedName) - - // ## FIND DATABASES PRESENT IN DG CONFIGURATION - out, err := ExecCommand(r, config, readyPod.Name, readyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf("echo -e \"%s\" | sqlplus -s / as sysdba ", DataguardBrokerGetDatabaseCMD)) - if err != nil { - return []string{}, "", err - } - log.Info("GetDatabasesInDgConfig Output") - log.Info(out) - - if !strings.Contains(out, "no rows selected") && !strings.Contains(out, "ORA-") { - out1 := strings.Replace(out, " ", "_", -1) - // filtering output and storing databses in dg configuration in "databases" slice - databases := strings.Fields(out1) - - // first 2 values in the slice will be column name(DATABASES) and a seperator(--------------) . so take the slice from position [2:] - databases = databases[2:] - return databases, out, nil - } - return []string{}, out, errors.New("databases in DG config is nil") -} - // Returns Database version func GetDatabaseVersion(readyPod corev1.Pod, r client.Reader, config *rest.Config, ctx context.Context, req ctrl.Request) (string, error) { diff --git a/commons/dbcssystem/dbcs_reconciler.go b/commons/dbcssystem/dbcs_reconciler.go index 60905c76..6c498320 100644 --- a/commons/dbcssystem/dbcs_reconciler.go +++ b/commons/dbcssystem/dbcs_reconciler.go @@ -43,6 +43,8 @@ import ( "encoding/json" "errors" "fmt" + "reflect" + "strings" "time" "github.com/go-logr/logr" @@ -51,68 +53,102 @@ import ( "github.com/oracle/oci-go-sdk/v65/database" "github.com/oracle/oci-go-sdk/v65/workrequests" + apierrors "k8s.io/apimachinery/pkg/api/errors" "k8s.io/client-go/tools/record" - "k8s.io/client-go/util/retry" "sigs.k8s.io/controller-runtime/pkg/client" - databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + databasev4 "github.com/oracle/oracle-database-operator/apis/database/v4" "github.com/oracle/oracle-database-operator/commons/annotations" corev1 "k8s.io/api/core/v1" "k8s.io/apimachinery/pkg/types" ) -func CreateAndGetDbcsId(logger logr.Logger, kubeClient client.Client, dbClient database.DatabaseClient, dbcs *databasev1alpha1.DbcsSystem, nwClient core.VirtualNetworkClient, wrClient workrequests.WorkRequestClient) (string, error) { +const ( + checkInterval = 30 * time.Second + timeout = 15 * time.Minute +) + +func CreateAndGetDbcsId(logger logr.Logger, kubeClient client.Client, dbClient database.DatabaseClient, dbcs *databasev4.DbcsSystem, nwClient core.VirtualNetworkClient, wrClient workrequests.WorkRequestClient, kmsDetails *databasev4.KMSDetailsStatus) (string, error) { - //var provisionedDbcsSystemId string ctx := context.TODO() - // Get DB System Details - dbcsDetails := database.LaunchDbSystemDetails{} + // Check if DBCS system already exists using the displayName + listDbcsRequest := database.ListDbSystemsRequest{ + CompartmentId: common.String(dbcs.Spec.DbSystem.CompartmentId), + DisplayName: common.String(dbcs.Spec.DbSystem.DisplayName), + } + + listDbcsResponse, err := dbClient.ListDbSystems(ctx, listDbcsRequest) + if err != nil { + return "", err + } + + // Check if any DBCS system matches the display name + if len(listDbcsResponse.Items) > 0 { + for _, dbcsItem := range listDbcsResponse.Items { + if dbcsItem.DisplayName != nil && *dbcsItem.DisplayName == dbcs.Spec.DbSystem.DisplayName { + logger.Info("DBCS system already exists", "DBCS ID", *dbcsItem.Id) + return *dbcsItem.Id, nil + } + } + } + // Get the admin password from OCI key sshPublicKeys, err := getPublicSSHKey(kubeClient, dbcs) if err != nil { return "", err } - // Get Db SystemOption + + // Get DB SystemOptions dbSystemReq := GetDBSystemopts(dbcs) licenceModel := getLicenceModel(dbcs) - if dbcs.Spec.DbSystem.ClusterName != "" { - dbcsDetails.ClusterName = &dbcs.Spec.DbSystem.ClusterName - } - - if dbcs.Spec.DbSystem.TimeZone != "" { - dbcsDetails.TimeZone = &dbcs.Spec.DbSystem.TimeZone - } // Get DB Home Details dbHomeReq, err := GetDbHomeDetails(kubeClient, dbClient, dbcs) if err != nil { return "", err } - //tenancyOcid, _ := provider.TenancyOCID() - dbcsDetails.AvailabilityDomain = common.String(dbcs.Spec.DbSystem.AvailabilityDomain) - dbcsDetails.CompartmentId = common.String(dbcs.Spec.DbSystem.CompartmentId) - dbcsDetails.SubnetId = common.String(dbcs.Spec.DbSystem.SubnetId) - dbcsDetails.Shape = common.String(dbcs.Spec.DbSystem.Shape) - dbcsDetails.Domain = common.String(dbcs.Spec.DbSystem.Domain) - if dbcs.Spec.DbSystem.DisplayName != "" { - dbcsDetails.DisplayName = common.String(dbcs.Spec.DbSystem.DisplayName) - } - dbcsDetails.SshPublicKeys = []string{sshPublicKeys} - dbcsDetails.Hostname = common.String(dbcs.Spec.DbSystem.HostName) - dbcsDetails.CpuCoreCount = common.Int(dbcs.Spec.DbSystem.CpuCoreCount) - //dbcsDetails.SourceDbSystemId = common.String(r.tenancyOcid) - dbcsDetails.NodeCount = common.Int(GetNodeCount(dbcs)) - dbcsDetails.InitialDataStorageSizeInGB = common.Int(GetInitialStorage(dbcs)) - dbcsDetails.DbSystemOptions = &dbSystemReq - dbcsDetails.DbHome = &dbHomeReq - dbcsDetails.DatabaseEdition = GetDBEdition(dbcs) - dbcsDetails.DiskRedundancy = GetDBbDiskRedundancy(dbcs) - dbcsDetails.LicenseModel = database.LaunchDbSystemDetailsLicenseModelEnum(licenceModel) + + // Determine CpuCoreCount + cpuCoreCount := 2 // default value + if dbcs.Spec.DbSystem.CpuCoreCount > 0 { + cpuCoreCount = dbcs.Spec.DbSystem.CpuCoreCount + } + + // Set up DB system details + dbcsDetails := database.LaunchDbSystemDetails{ + AvailabilityDomain: common.String(dbcs.Spec.DbSystem.AvailabilityDomain), + CompartmentId: common.String(dbcs.Spec.DbSystem.CompartmentId), + SubnetId: common.String(dbcs.Spec.DbSystem.SubnetId), + Shape: common.String(dbcs.Spec.DbSystem.Shape), + Domain: common.String(dbcs.Spec.DbSystem.Domain), + DisplayName: common.String(dbcs.Spec.DbSystem.DisplayName), + SshPublicKeys: []string{sshPublicKeys}, + Hostname: common.String(dbcs.Spec.DbSystem.HostName), + CpuCoreCount: common.Int(cpuCoreCount), + NodeCount: common.Int(GetNodeCount(dbcs)), + InitialDataStorageSizeInGB: common.Int(GetInitialStorage(dbcs)), + DbSystemOptions: &dbSystemReq, + DbHome: &dbHomeReq, + DatabaseEdition: GetDBEdition(dbcs), + DiskRedundancy: GetDBbDiskRedundancy(dbcs), + LicenseModel: database.LaunchDbSystemDetailsLicenseModelEnum(licenceModel), + } + if len(dbcs.Spec.DbSystem.Tags) != 0 { dbcsDetails.FreeformTags = dbcs.Spec.DbSystem.Tags } + // Add KMS details if available + if kmsDetails != nil && kmsDetails.VaultId != "" { + dbcsDetails.KmsKeyId = common.String(kmsDetails.KeyId) + dbcsDetails.DbHome.Database.KmsKeyId = common.String(kmsDetails.KeyId) + dbcsDetails.DbHome.Database.VaultId = common.String(kmsDetails.VaultId) + } + + // Log dbcsDetails for debugging + logger.Info("Launching DB System with details", "dbcsDetails", dbcsDetails) + req := database.LaunchDbSystemRequest{LaunchDbSystemDetails: dbcsDetails} // Send the request using the service client @@ -122,12 +158,14 @@ func CreateAndGetDbcsId(logger logr.Logger, kubeClient client.Client, dbClient d } dbcs.Spec.Id = resp.DbSystem.Id + // Change the phase to "Provisioning" - if statusErr := SetLifecycleState(kubeClient, dbClient, dbcs, databasev1alpha1.Provision, nwClient, wrClient); statusErr != nil { + if statusErr := SetLifecycleState(kubeClient, dbClient, dbcs, databasev4.Provision, nwClient, wrClient); statusErr != nil { return "", statusErr } + // Check the State - _, err = CheckResourceState(logger, dbClient, *resp.DbSystem.Id, string(databasev1alpha1.Provision), string(databasev1alpha1.Available)) + _, err = CheckResourceState(logger, dbClient, *resp.DbSystem.Id, string(databasev4.Provision), string(databasev4.Available)) if err != nil { return "", err } @@ -135,16 +173,418 @@ func CreateAndGetDbcsId(logger logr.Logger, kubeClient client.Client, dbClient d return *resp.DbSystem.Id, nil } +func parseLicenseModel(licenseModelStr string) (database.DbSystemLicenseModelEnum, error) { + switch licenseModelStr { + case "LICENSE_INCLUDED": + return database.DbSystemLicenseModelLicenseIncluded, nil + case "BRING_YOUR_OWN_LICENSE": + return database.DbSystemLicenseModelBringYourOwnLicense, nil + default: + return "", fmt.Errorf("invalid license model: %s", licenseModelStr) + } +} +func convertLicenseModel(licenseModel database.DbSystemLicenseModelEnum) (database.LaunchDbSystemFromDbSystemDetailsLicenseModelEnum, error) { + switch licenseModel { + case database.DbSystemLicenseModelLicenseIncluded: + return database.LaunchDbSystemFromDbSystemDetailsLicenseModelLicenseIncluded, nil + case database.DbSystemLicenseModelBringYourOwnLicense: + return database.LaunchDbSystemFromDbSystemDetailsLicenseModelBringYourOwnLicense, nil + default: + return "", fmt.Errorf("unsupported license model: %s", licenseModel) + } +} + +func CloneAndGetDbcsId(logger logr.Logger, kubeClient client.Client, dbClient database.DatabaseClient, dbcs *databasev4.DbcsSystem, nwClient core.VirtualNetworkClient, wrClient workrequests.WorkRequestClient) (string, error) { + ctx := context.TODO() + var err error + dbAdminPassword := "" + // tdePassword := "" + logger.Info("Starting the clone process for DBCS", "dbcs", dbcs) + // Get the admin password from Kubernetes secret + if dbcs.Spec.DbClone.DbAdminPasswordSecret != "" { + dbAdminPassword, err = GetCloningAdminPassword(kubeClient, dbcs) + if err != nil { + logger.Error(err, "Failed to get DB Admin password") + } + // logger.Info(dbAdminPassword) + } + // // Log retrieved passwords + logger.Info("Retrieved passwords from Kubernetes secrets") + + // // // Retrieve the TDE wallet password from Kubernetes secrets + // // tdePassword, err := GetTdePassword(kubeClient, dbcs.Namespace, dbcs.Spec.TdeWalletPasswordSecretName) + // // if err != nil { + // // logger.Error(err, "Failed to get TDE wallet password from Kubernetes secret", "namespace", dbcs.Namespace, "secretName", dbcs.Spec.TdeWalletPasswordSecretName) + // // return "", err + // // } + sshPublicKeys, err := getCloningPublicSSHKey(kubeClient, dbcs) + if err != nil { + logger.Error(err, "failed to get SSH public key") + } + + // Fetch the existing DB system details + existingDbSystem, err := dbClient.GetDbSystem(ctx, database.GetDbSystemRequest{ + DbSystemId: dbcs.Spec.Id, + }) + if err != nil { + return "", err + } + logger.Info("Retrieved existing Db System Details from OCI using Spec.Id") + + // // Create the clone request payload + // // Create the DbHome details + // Prepare CreateDatabaseFromDbSystemDetails + databaseDetails := &database.CreateDatabaseFromDbSystemDetails{ + AdminPassword: &dbAdminPassword, + DbName: &dbcs.Spec.DbClone.DbName, + DbDomain: existingDbSystem.DbSystem.Domain, + DbUniqueName: &dbcs.Spec.DbClone.DbUniqueName, + FreeformTags: existingDbSystem.DbSystem.FreeformTags, + DefinedTags: existingDbSystem.DbSystem.DefinedTags, + } + licenseModelEnum, err := parseLicenseModel(dbcs.Spec.DbClone.LicenseModel) + if err != nil { + return "", err + } + launchLicenseModel, err := convertLicenseModel(licenseModelEnum) + if err != nil { + return "", err + } + + cloneRequest := database.LaunchDbSystemFromDbSystemDetails{ + CompartmentId: existingDbSystem.DbSystem.CompartmentId, + AvailabilityDomain: existingDbSystem.DbSystem.AvailabilityDomain, + SubnetId: &dbcs.Spec.DbClone.SubnetId, + Shape: existingDbSystem.DbSystem.Shape, + SshPublicKeys: []string{sshPublicKeys}, + Hostname: &dbcs.Spec.DbClone.HostName, + CpuCoreCount: existingDbSystem.DbSystem.CpuCoreCount, + SourceDbSystemId: existingDbSystem.DbSystem.Id, + DbHome: &database.CreateDbHomeFromDbSystemDetails{ + Database: databaseDetails, + DisplayName: existingDbSystem.DbSystem.DisplayName, + FreeformTags: existingDbSystem.DbSystem.FreeformTags, + DefinedTags: existingDbSystem.DbSystem.DefinedTags, + }, + FaultDomains: existingDbSystem.DbSystem.FaultDomains, + DisplayName: &dbcs.Spec.DbClone.DisplayName, + BackupSubnetId: existingDbSystem.DbSystem.BackupSubnetId, + NsgIds: existingDbSystem.DbSystem.NsgIds, + BackupNetworkNsgIds: existingDbSystem.DbSystem.BackupNetworkNsgIds, + TimeZone: existingDbSystem.DbSystem.TimeZone, + DbSystemOptions: existingDbSystem.DbSystem.DbSystemOptions, + SparseDiskgroup: existingDbSystem.DbSystem.SparseDiskgroup, + Domain: &dbcs.Spec.DbClone.Domain, + ClusterName: existingDbSystem.DbSystem.ClusterName, + DataStoragePercentage: existingDbSystem.DbSystem.DataStoragePercentage, + // KmsKeyId: existingDbSystem.DbSystem.KmsKeyId, + // KmsKeyVersionId: existingDbSystem.DbSystem.KmsKeyVersionId, + NodeCount: existingDbSystem.DbSystem.NodeCount, + FreeformTags: existingDbSystem.DbSystem.FreeformTags, + DefinedTags: existingDbSystem.DbSystem.DefinedTags, + DataCollectionOptions: existingDbSystem.DbSystem.DataCollectionOptions, + LicenseModel: launchLicenseModel, + } + + // Execute the clone request + response, err := dbClient.LaunchDbSystem(ctx, database.LaunchDbSystemRequest{ + LaunchDbSystemDetails: cloneRequest, + }) + if err != nil { + return "", err + } + + dbcs.Status.DbCloneStatus.Id = response.DbSystem.Id + + // Change the phase to "Provisioning" + if statusErr := SetLifecycleState(kubeClient, dbClient, dbcs, databasev4.Provision, nwClient, wrClient); statusErr != nil { + return "", statusErr + } + + // Check the state + _, err = CheckResourceState(logger, dbClient, *response.DbSystem.Id, string(databasev4.Provision), string(databasev4.Available)) + if err != nil { + return "", err + } + + return *response.DbSystem.Id, nil + // return "", nil +} + +// CloneFromBackupAndGetDbcsId clones a DB system from a backup and returns the new DB system's OCID. +func CloneFromBackupAndGetDbcsId(logger logr.Logger, kubeClient client.Client, dbClient database.DatabaseClient, dbcs *databasev4.DbcsSystem, nwClient core.VirtualNetworkClient, wrClient workrequests.WorkRequestClient) (string, error) { + ctx := context.TODO() + + var err error + var dbAdminPassword string + var tdePassword string + logger.Info("Starting the clone process for DBCS from backup", "dbcs", dbcs) + backupResp, err := dbClient.GetBackup(ctx, database.GetBackupRequest{ + BackupId: dbcs.Spec.DbBackupId, + }) + + if err != nil { + fmt.Println("Error getting backup details:", err) + return "", err + } + databaseId := backupResp.Backup.DatabaseId + // Fetch the existing Database details + existingDatabase, err := dbClient.GetDatabase(ctx, database.GetDatabaseRequest{ + DatabaseId: databaseId, + }) + if err != nil { + logger.Error(err, "Failed to retrieve existing Database details") + return "", err + } + // Check if DbSystemId is available + dbSystemId := existingDatabase.DbSystemId + if dbSystemId == nil { + // handle the case where DbSystemId is not available + logger.Error(err, "DBSystemId not found") + return "", err + } + + // Fetch the existing DB system details + existingDbSystem, err := dbClient.GetDbSystem(ctx, database.GetDbSystemRequest{ + DbSystemId: dbSystemId, + }) + if err != nil { + return "", err + } + // Get the admin password from Kubernetes secret + if dbcs.Spec.DbClone.DbAdminPasswordSecret != "" { + dbAdminPassword, err = GetCloningAdminPassword(kubeClient, dbcs) + if err != nil { + logger.Error(err, "Failed to get DB Admin password") + } + // logger.Info(dbAdminPassword) + } + // // // Retrieve the TDE wallet password from Kubernetes secrets to open backup DB using TDE Wallet + if dbcs.Spec.DbClone.TdeWalletPasswordSecret != "" { + tdePassword, err = GetCloningTdePassword(kubeClient, dbcs) + if err != nil { + logger.Error(err, "Failed to get TDE wallet password from Kubernetes secret") + return "", err + } + } + + sshPublicKeys, err := getCloningPublicSSHKey(kubeClient, dbcs) + if err != nil { + logger.Error(err, "failed to get SSH public key") + return "", err + } + + // Create the clone request payload + cloneRequest := database.LaunchDbSystemFromBackupDetails{ + CompartmentId: existingDbSystem.DbSystem.CompartmentId, + AvailabilityDomain: existingDbSystem.DbSystem.AvailabilityDomain, + SubnetId: &dbcs.Spec.DbClone.SubnetId, + Shape: existingDbSystem.DbSystem.Shape, + SshPublicKeys: []string{sshPublicKeys}, + Hostname: &dbcs.Spec.DbClone.HostName, + CpuCoreCount: existingDbSystem.DbSystem.CpuCoreCount, + DbHome: &database.CreateDbHomeFromBackupDetails{ + Database: &database.CreateDatabaseFromBackupDetails{ // Corrected type here + BackupId: dbcs.Spec.DbBackupId, + AdminPassword: &dbAdminPassword, + BackupTDEPassword: &tdePassword, + DbName: &dbcs.Spec.DbClone.DbName, + // DbDomain: existingDbSystem.DbSystem.Domain, + DbUniqueName: &dbcs.Spec.DbClone.DbUniqueName, + // FreeformTags: existingDbSystem.DbSystem.FreeformTags, + // DefinedTags: existingDbSystem.DbSystem.DefinedTags, + SidPrefix: &dbcs.Spec.DbClone.SidPrefix, + }, + DisplayName: existingDbSystem.DbSystem.DisplayName, + FreeformTags: existingDbSystem.DbSystem.FreeformTags, + DefinedTags: existingDbSystem.DbSystem.DefinedTags, + }, + FaultDomains: existingDbSystem.DbSystem.FaultDomains, + DisplayName: &dbcs.Spec.DbClone.DisplayName, + BackupSubnetId: existingDbSystem.DbSystem.BackupSubnetId, + NsgIds: existingDbSystem.DbSystem.NsgIds, + BackupNetworkNsgIds: existingDbSystem.DbSystem.BackupNetworkNsgIds, + TimeZone: existingDbSystem.DbSystem.TimeZone, + DbSystemOptions: existingDbSystem.DbSystem.DbSystemOptions, + SparseDiskgroup: existingDbSystem.DbSystem.SparseDiskgroup, + Domain: &dbcs.Spec.DbClone.Domain, + ClusterName: existingDbSystem.DbSystem.ClusterName, + DataStoragePercentage: existingDbSystem.DbSystem.DataStoragePercentage, + InitialDataStorageSizeInGB: &dbcs.Spec.DbClone.InitialDataStorageSizeInGB, + KmsKeyId: &dbcs.Spec.DbClone.KmsKeyId, + KmsKeyVersionId: &dbcs.Spec.DbClone.KmsKeyVersionId, + NodeCount: existingDbSystem.DbSystem.NodeCount, + FreeformTags: existingDbSystem.DbSystem.FreeformTags, + DefinedTags: existingDbSystem.DbSystem.DefinedTags, + DataCollectionOptions: existingDbSystem.DbSystem.DataCollectionOptions, + DatabaseEdition: database.LaunchDbSystemFromBackupDetailsDatabaseEditionEnum(existingDbSystem.DbSystem.DatabaseEdition), + LicenseModel: database.LaunchDbSystemFromBackupDetailsLicenseModelEnum(existingDbSystem.DbSystem.LicenseModel), + StorageVolumePerformanceMode: database.LaunchDbSystemBaseStorageVolumePerformanceModeEnum(existingDbSystem.DbSystem.StorageVolumePerformanceMode), + } + + // Execute the clone request + response, err := dbClient.LaunchDbSystem(ctx, database.LaunchDbSystemRequest{ + LaunchDbSystemDetails: cloneRequest, + }) + if err != nil { + return "", err + } + + dbcs.Status.DbCloneStatus.Id = response.DbSystem.Id + + // Change the phase to "Provisioning" + if statusErr := SetLifecycleState(kubeClient, dbClient, dbcs, databasev4.Provision, nwClient, wrClient); statusErr != nil { + return "", statusErr + } + + // Check the state + _, err = CheckResourceState(logger, dbClient, *response.DbSystem.Id, string(databasev4.Provision), string(databasev4.Available)) + if err != nil { + return "", err + } + + return *response.DbSystem.Id, nil +} + // Sync the DbcsSystem Database details +func CloneFromDatabaseAndGetDbcsId(logger logr.Logger, kubeClient client.Client, dbClient database.DatabaseClient, dbcs *databasev4.DbcsSystem, nwClient core.VirtualNetworkClient, wrClient workrequests.WorkRequestClient) (string, error) { + ctx := context.TODO() + var err error + dbAdminPassword := "" + tdePassword := "" + logger.Info("Starting the clone process for Database", "dbcs", dbcs) + + // Get the admin password from Kubernetes secret + if dbcs.Spec.DbClone.DbAdminPasswordSecret != "" { + dbAdminPassword, err = GetCloningAdminPassword(kubeClient, dbcs) + if err != nil { + logger.Error(err, "Failed to get DB Admin password") + return "", err + } + } + // // // Retrieve the TDE wallet password from Kubernetes secrets to open backup DB using TDE Wallet + if dbcs.Spec.DbClone.TdeWalletPasswordSecret != "" { + tdePassword, err = GetCloningTdePassword(kubeClient, dbcs) + if err != nil { + logger.Error(err, "Failed to get TDE wallet password from Kubernetes secret") + return "", err + } + } + + logger.Info("Retrieved passwords from Kubernetes secrets") + + // Fetch the existing Database details + existingDatabase, err := dbClient.GetDatabase(ctx, database.GetDatabaseRequest{ + DatabaseId: dbcs.Spec.DatabaseId, + }) + if err != nil { + logger.Error(err, "Failed to retrieve existing Database details") + return "", err + } + // Check if DbSystemId is available + dbSystemId := existingDatabase.DbSystemId + if dbSystemId == nil { + // handle the case where DbSystemId is not available + logger.Error(err, "DBSystemId not found") + return "", err + } + + // Fetch the existing DB system details + existingDbSystem, err := dbClient.GetDbSystem(ctx, database.GetDbSystemRequest{ + DbSystemId: dbSystemId, + }) + if err != nil { + return "", err + } + logger.Info("Retrieved existing Database details from OCI", "DatabaseId", dbcs.Spec.DatabaseId) + + // Get SSH public key + sshPublicKeys, err := getCloningPublicSSHKey(kubeClient, dbcs) + if err != nil { + logger.Error(err, "Failed to get SSH public key") + return "", err + } + + // Create the clone request payload + cloneRequest := database.LaunchDbSystemFromDatabaseDetails{ + CompartmentId: existingDatabase.CompartmentId, + AvailabilityDomain: existingDbSystem.DbSystem.AvailabilityDomain, + SubnetId: existingDbSystem.DbSystem.SubnetId, + Shape: existingDbSystem.DbSystem.Shape, + SshPublicKeys: []string{sshPublicKeys}, + Hostname: &dbcs.Spec.DbClone.HostName, + CpuCoreCount: existingDbSystem.DbSystem.CpuCoreCount, + DatabaseEdition: database.LaunchDbSystemFromDatabaseDetailsDatabaseEditionEnum(existingDbSystem.DbSystem.DatabaseEdition), + DbHome: &database.CreateDbHomeFromDatabaseDetails{ + Database: &database.CreateDatabaseFromAnotherDatabaseDetails{ + // Mandatory fields + DatabaseId: dbcs.Spec.DatabaseId, // Source database ID + // Optionally fill in other fields if needed + DbName: &dbcs.Spec.DbClone.DbName, + AdminPassword: &dbAdminPassword, // Admin password for the new database + // The password to open the TDE wallet. + BackupTDEPassword: &tdePassword, + + DbUniqueName: &dbcs.Spec.DbClone.DbUniqueName, + }, + + // Provide a display name for the new Database Home + DisplayName: existingDbSystem.DbSystem.DisplayName, + FreeformTags: existingDbSystem.DbSystem.FreeformTags, + DefinedTags: existingDbSystem.DbSystem.DefinedTags, + }, + + FaultDomains: existingDbSystem.DbSystem.FaultDomains, + DisplayName: &dbcs.Spec.DbClone.DisplayName, + BackupSubnetId: existingDbSystem.DbSystem.BackupSubnetId, + NsgIds: existingDbSystem.DbSystem.NsgIds, + BackupNetworkNsgIds: existingDbSystem.DbSystem.BackupNetworkNsgIds, + TimeZone: existingDbSystem.DbSystem.TimeZone, + KmsKeyId: &dbcs.Spec.DbClone.KmsKeyId, + KmsKeyVersionId: &dbcs.Spec.DbClone.KmsKeyVersionId, + NodeCount: existingDbSystem.DbSystem.NodeCount, + FreeformTags: existingDbSystem.DbSystem.FreeformTags, + DefinedTags: existingDbSystem.DbSystem.DefinedTags, + // PrivateIp: &dbcs.Spec.DbClone.PrivateIp, + InitialDataStorageSizeInGB: &dbcs.Spec.DbClone.InitialDataStorageSizeInGB, + LicenseModel: database.LaunchDbSystemFromDatabaseDetailsLicenseModelEnum(existingDbSystem.DbSystem.LicenseModel), + StorageVolumePerformanceMode: database.LaunchDbSystemBaseStorageVolumePerformanceModeEnum(existingDbSystem.DbSystem.StorageVolumePerformanceMode), + } + + // logger.Info("Launching database clone", "cloneRequest", cloneRequest) + + // Execute the clone request + response, err := dbClient.LaunchDbSystem(ctx, database.LaunchDbSystemRequest{ + LaunchDbSystemDetails: cloneRequest, + }) + if err != nil { + return "", err + } + + dbcs.Status.DbCloneStatus.Id = response.DbSystem.Id + + // Change the phase to "Provisioning" + if statusErr := SetLifecycleState(kubeClient, dbClient, dbcs, databasev4.Provision, nwClient, wrClient); statusErr != nil { + return "", statusErr + } + + // Check the state + _, err = CheckResourceState(logger, dbClient, *response.DbSystem.Id, string(databasev4.Provision), string(databasev4.Available)) + if err != nil { + return "", err + } + + return *response.DbSystem.Id, nil +} // Get admin password from Secret then OCI valut secret -func GetAdminPassword(kubeClient client.Client, dbcs *databasev1alpha1.DbcsSystem) (string, error) { - if dbcs.Spec.DbSystem.DbAdminPaswordSecret != "" { +func GetCloningAdminPassword(kubeClient client.Client, dbcs *databasev4.DbcsSystem) (string, error) { + if dbcs.Spec.DbClone.DbAdminPasswordSecret != "" { // Get the Admin Secret adminSecret := &corev1.Secret{} err := kubeClient.Get(context.TODO(), types.NamespacedName{ Namespace: dbcs.GetNamespace(), - Name: dbcs.Spec.DbSystem.DbAdminPaswordSecret, + Name: dbcs.Spec.DbClone.DbAdminPasswordSecret, }, adminSecret) if err != nil { @@ -154,7 +594,7 @@ func GetAdminPassword(kubeClient client.Client, dbcs *databasev1alpha1.DbcsSyste // Get the admin password key := "admin-password" if val, ok := adminSecret.Data[key]; ok { - return string(val), nil + return strings.TrimSpace(string(val)), nil } else { msg := "secret item not found: admin-password" return "", errors.New(msg) @@ -164,7 +604,33 @@ func GetAdminPassword(kubeClient client.Client, dbcs *databasev1alpha1.DbcsSyste } // Get admin password from Secret then OCI valut secret -func GetTdePassword(kubeClient client.Client, dbcs *databasev1alpha1.DbcsSystem) (string, error) { +func GetAdminPassword(kubeClient client.Client, dbcs *databasev4.DbcsSystem) (string, error) { + if dbcs.Spec.DbSystem.DbAdminPasswordSecret != "" { + // Get the Admin Secret + adminSecret := &corev1.Secret{} + err := kubeClient.Get(context.TODO(), types.NamespacedName{ + Namespace: dbcs.GetNamespace(), + Name: dbcs.Spec.DbSystem.DbAdminPasswordSecret, + }, adminSecret) + + if err != nil { + return "", err + } + + // Get the admin password + key := "admin-password" + if val, ok := adminSecret.Data[key]; ok { + return strings.TrimSpace(string(val)), nil + } else { + msg := "secret item not found: admin-password" + return "", errors.New(msg) + } + } + return "", errors.New("should provide either a Secret name or a Valut Secret ID") +} + +// Get admin password from Secret then OCI valut secret +func GetTdePassword(kubeClient client.Client, dbcs *databasev4.DbcsSystem) (string, error) { if dbcs.Spec.DbSystem.TdeWalletPasswordSecret != "" { // Get the Admin Secret tdeSecret := &corev1.Secret{} @@ -180,7 +646,7 @@ func GetTdePassword(kubeClient client.Client, dbcs *databasev1alpha1.DbcsSystem) // Get the admin password key := "tde-password" if val, ok := tdeSecret.Data[key]; ok { - return string(val), nil + return strings.TrimSpace(string(val)), nil } else { msg := "secret item not found: tde-password" return "", errors.New(msg) @@ -190,7 +656,33 @@ func GetTdePassword(kubeClient client.Client, dbcs *databasev1alpha1.DbcsSystem) } // Get admin password from Secret then OCI valut secret -func getPublicSSHKey(kubeClient client.Client, dbcs *databasev1alpha1.DbcsSystem) (string, error) { +func GetCloningTdePassword(kubeClient client.Client, dbcs *databasev4.DbcsSystem) (string, error) { + if dbcs.Spec.DbClone.TdeWalletPasswordSecret != "" { + // Get the Admin Secret + tdeSecret := &corev1.Secret{} + err := kubeClient.Get(context.TODO(), types.NamespacedName{ + Namespace: dbcs.GetNamespace(), + Name: dbcs.Spec.DbClone.TdeWalletPasswordSecret, + }, tdeSecret) + + if err != nil { + return "", err + } + + // Get the admin password + key := "tde-password" + if val, ok := tdeSecret.Data[key]; ok { + return strings.TrimSpace(string(val)), nil + } else { + msg := "secret item not found: tde-password" + return "", errors.New(msg) + } + } + return "", errors.New("should provide either a Secret name or a Valut Secret ID") +} + +// Get admin password from Secret then OCI valut secret +func getPublicSSHKey(kubeClient client.Client, dbcs *databasev4.DbcsSystem) (string, error) { if dbcs.Spec.DbSystem.SshPublicKeys[0] != "" { // Get the Admin Secret sshkeysecret := &corev1.Secret{} @@ -215,6 +707,32 @@ func getPublicSSHKey(kubeClient client.Client, dbcs *databasev1alpha1.DbcsSystem return "", errors.New("should provide either a Secret name or a Valut Secret ID") } +// Get admin password from Secret then OCI valut secret +func getCloningPublicSSHKey(kubeClient client.Client, dbcs *databasev4.DbcsSystem) (string, error) { + if dbcs.Spec.DbClone.SshPublicKeys[0] != "" { + // Get the Admin Secret + sshkeysecret := &corev1.Secret{} + err := kubeClient.Get(context.TODO(), types.NamespacedName{ + Namespace: dbcs.GetNamespace(), + Name: dbcs.Spec.DbClone.SshPublicKeys[0], + }, sshkeysecret) + + if err != nil { + return "", err + } + + // Get the admin password` + key := "publickey" + if val, ok := sshkeysecret.Data[key]; ok { + return string(val), nil + } else { + msg := "secret item not found: " + return "", errors.New(msg) + } + } + return "", errors.New("should provide either a Secret name or a Valut Secret ID") +} + // Delete DbcsSystem System func DeleteDbcsSystemSystem(dbClient database.DatabaseClient, Id string) error { @@ -233,24 +751,142 @@ func DeleteDbcsSystemSystem(dbClient database.DatabaseClient, Id string) error { } // SetLifecycleState set status.state of the reosurce. -func SetLifecycleState(kubeClient client.Client, dbClient database.DatabaseClient, dbcs *databasev1alpha1.DbcsSystem, state databasev1alpha1.LifecycleState, nwClient core.VirtualNetworkClient, wrClient workrequests.WorkRequestClient) error { - return retry.RetryOnConflict(retry.DefaultRetry, func() error { - dbcs.Status.State = state - // Set the status +func SetLifecycleState(kubeClient client.Client, dbClient database.DatabaseClient, dbcs *databasev4.DbcsSystem, state databasev4.LifecycleState, nwClient core.VirtualNetworkClient, wrClient workrequests.WorkRequestClient) error { + maxRetries := 5 + retryDelay := time.Second * 2 + + for attempt := 0; attempt < maxRetries; attempt++ { + // Fetch the latest version of the object + latestInstance := &databasev4.DbcsSystem{} + err := kubeClient.Get(context.TODO(), client.ObjectKeyFromObject(dbcs), latestInstance) + if err != nil { + // Log and return error if fetching the latest version fails + return fmt.Errorf("failed to fetch the latest version of DBCS instance: %w", err) + } + + // Merge the instance fields into latestInstance + err = mergeInstancesFromLatest(dbcs, latestInstance) + if err != nil { + return fmt.Errorf("failed to merge instances: %w", err) + } + + // Set the status using the dbcs object if statusErr := SetDBCSStatus(dbClient, dbcs, nwClient, wrClient); statusErr != nil { return statusErr } - if err := kubeClient.Status().Update(context.TODO(), dbcs); err != nil { - return err + + // Update the ResourceVersion of dbcs from latestInstance to avoid conflict + dbcs.ResourceVersion = latestInstance.ResourceVersion + + // Attempt to patch the status of the instance + err = kubeClient.Status().Patch(context.TODO(), dbcs, client.MergeFrom(latestInstance)) + if err != nil { + if apierrors.IsConflict(err) { + // Handle the conflict and retry + time.Sleep(retryDelay) + continue + } + // For other errors, log and return the error + return fmt.Errorf("failed to update the DBCS instance status: %w", err) } - return nil - }) + // If no error, break the loop + break + } + + return nil +} +func mergeInstancesFromLatest(instance, latestInstance *databasev4.DbcsSystem) error { + instanceVal := reflect.ValueOf(instance).Elem() + latestVal := reflect.ValueOf(latestInstance).Elem() + + // Fields to exclude from merging + excludeFields := map[string]bool{ + "ReleaseUpdate": true, + "AsmStorageStatus": true, + } + + // Loop through the fields in instance + for i := 0; i < instanceVal.NumField(); i++ { + field := instanceVal.Type().Field(i) + instanceField := instanceVal.Field(i) + latestField := latestVal.FieldByName(field.Name) + + // Skip unexported fields + if !isExported(field) { + continue + } + + // Ensure latestField is valid + if !latestField.IsValid() || !instanceField.CanSet() { + continue + } + + // Skip fields that are in the exclusion list + if excludeFields[field.Name] { + continue + } + + // Handle pointer fields + if latestField.Kind() == reflect.Ptr { + if !latestField.IsNil() && instanceField.IsNil() { + // If instance's field is nil and latest's field is not nil, set the latest's field value + instanceField.Set(latestField) + } + // If instance's field is not nil, do not overwrite + } else if latestField.Kind() == reflect.String { + if latestField.String() != "" && latestField.String() != "NOT_DEFINED" && instanceField.String() == "" { + // If latest's string field is non-empty and not "NOT_DEFINED", and instance's string field is empty, set the value + instanceField.Set(latestField) + } + } else if latestField.Kind() == reflect.Struct { + // Handle struct types recursively + mergeStructFields(instanceField, latestField) + } else { + // Handle other types if instance's field is zero value + if reflect.DeepEqual(instanceField.Interface(), reflect.Zero(instanceField.Type()).Interface()) { + instanceField.Set(latestField) + } + } + } + return nil +} + +func mergeStructFields(instanceField, latestField reflect.Value) { + for i := 0; i < instanceField.NumField(); i++ { + subField := instanceField.Type().Field(i) + instanceSubField := instanceField.Field(i) + latestSubField := latestField.Field(i) + + if !isExported(subField) || !instanceSubField.CanSet() { + continue + } + + if latestSubField.Kind() == reflect.Ptr { + if !latestSubField.IsNil() && instanceSubField.IsNil() { + instanceSubField.Set(latestSubField) + } + } else if latestSubField.Kind() == reflect.String { + if latestSubField.String() != "" && latestSubField.String() != "NOT_DEFINED" && instanceSubField.String() == "" { + instanceSubField.Set(latestSubField) + } + } else if latestSubField.Kind() == reflect.Struct { + mergeStructFields(instanceSubField, latestSubField) + } else { + if reflect.DeepEqual(instanceSubField.Interface(), reflect.Zero(instanceSubField.Type()).Interface()) { + instanceSubField.Set(latestSubField) + } + } + } +} + +func isExported(field reflect.StructField) bool { + return field.PkgPath == "" } // SetDBCSSystem LifeCycle state when state is provisioning -func SetDBCSDatabaseLifecycleState(logger logr.Logger, kubeClient client.Client, dbClient database.DatabaseClient, dbcs *databasev1alpha1.DbcsSystem, nwClient core.VirtualNetworkClient, wrClient workrequests.WorkRequestClient) error { +func SetDBCSDatabaseLifecycleState(logger logr.Logger, kubeClient client.Client, dbClient database.DatabaseClient, dbcs *databasev4.DbcsSystem, nwClient core.VirtualNetworkClient, wrClient workrequests.WorkRequestClient) error { dbcsId := *dbcs.Spec.Id @@ -266,47 +902,47 @@ func SetDBCSDatabaseLifecycleState(logger logr.Logger, kubeClient client.Client, // Return if the desired lifecycle state is the same as the current lifecycle state if string(dbcs.Status.State) == string(resp.LifecycleState) { return nil - } else if string(resp.LifecycleState) == string(databasev1alpha1.Available) { + } else if string(resp.LifecycleState) == string(databasev4.Available) { // Change the phase to "Available" - if statusErr := SetLifecycleState(kubeClient, dbClient, dbcs, databasev1alpha1.Available, nwClient, wrClient); statusErr != nil { + if statusErr := SetLifecycleState(kubeClient, dbClient, dbcs, databasev4.Available, nwClient, wrClient); statusErr != nil { return statusErr } - } else if string(resp.LifecycleState) == string(databasev1alpha1.Provision) { + } else if string(resp.LifecycleState) == string(databasev4.Provision) { // Change the phase to "Provisioning" - if statusErr := SetLifecycleState(kubeClient, dbClient, dbcs, databasev1alpha1.Provision, nwClient, wrClient); statusErr != nil { + if statusErr := SetLifecycleState(kubeClient, dbClient, dbcs, databasev4.Provision, nwClient, wrClient); statusErr != nil { return statusErr } // Check the State - _, err = CheckResourceState(logger, dbClient, *resp.DbSystem.Id, string(databasev1alpha1.Provision), string(databasev1alpha1.Available)) + _, err = CheckResourceState(logger, dbClient, *resp.DbSystem.Id, string(databasev4.Provision), string(databasev4.Available)) if err != nil { return err } - } else if string(resp.LifecycleState) == string(databasev1alpha1.Update) { + } else if string(resp.LifecycleState) == string(databasev4.Update) { // Change the phase to "Updating" - if statusErr := SetLifecycleState(kubeClient, dbClient, dbcs, databasev1alpha1.Update, nwClient, wrClient); statusErr != nil { + if statusErr := SetLifecycleState(kubeClient, dbClient, dbcs, databasev4.Update, nwClient, wrClient); statusErr != nil { return statusErr } // Check the State - _, err = CheckResourceState(logger, dbClient, *resp.DbSystem.Id, string(databasev1alpha1.Update), string(databasev1alpha1.Available)) + _, err = CheckResourceState(logger, dbClient, *resp.DbSystem.Id, string(databasev4.Update), string(databasev4.Available)) if err != nil { return err } - } else if string(resp.LifecycleState) == string(databasev1alpha1.Failed) { + } else if string(resp.LifecycleState) == string(databasev4.Failed) { // Change the phase to "Updating" - if statusErr := SetLifecycleState(kubeClient, dbClient, dbcs, databasev1alpha1.Failed, nwClient, wrClient); statusErr != nil { + if statusErr := SetLifecycleState(kubeClient, dbClient, dbcs, databasev4.Failed, nwClient, wrClient); statusErr != nil { return statusErr } return fmt.Errorf("DbSystem is in Failed State") - } else if string(resp.LifecycleState) == string(databasev1alpha1.Terminated) { + } else if string(resp.LifecycleState) == string(databasev4.Terminated) { // Change the phase to "Terminated" - if statusErr := SetLifecycleState(kubeClient, dbClient, dbcs, databasev1alpha1.Terminate, nwClient, wrClient); statusErr != nil { + if statusErr := SetLifecycleState(kubeClient, dbClient, dbcs, databasev4.Terminate, nwClient, wrClient); statusErr != nil { return statusErr } } return nil } -func GetDbSystemId(logger logr.Logger, dbClient database.DatabaseClient, dbcs *databasev1alpha1.DbcsSystem) error { +func GetDbSystemId(logger logr.Logger, dbClient database.DatabaseClient, dbcs *databasev4.DbcsSystem) error { dbcsId := *dbcs.Spec.Id dbcsReq := database.GetDbSystemRequest{ @@ -353,7 +989,9 @@ func GetDbSystemId(logger logr.Logger, dbClient database.DatabaseClient, dbcs *d } dbcs.Spec.DbSystem.SubnetId = *response.SubnetId dbcs.Spec.DbSystem.AvailabilityDomain = *response.AvailabilityDomain - + if response.KmsKeyId != nil { + dbcs.Status.KMSDetailsStatus.KeyId = *response.KmsKeyId + } err = PopulateDBDetails(logger, dbClient, dbcs) if err != nil { logger.Info("Error Occurred while collecting the DB details") @@ -362,7 +1000,7 @@ func GetDbSystemId(logger logr.Logger, dbClient database.DatabaseClient, dbcs *d return nil } -func PopulateDBDetails(logger logr.Logger, dbClient database.DatabaseClient, dbcs *databasev1alpha1.DbcsSystem) error { +func PopulateDBDetails(logger logr.Logger, dbClient database.DatabaseClient, dbcs *databasev4.DbcsSystem) error { listDbHomeRsp, err := GetListDbHomeRsp(logger, dbClient, dbcs) if err != nil { @@ -383,7 +1021,7 @@ func PopulateDBDetails(logger logr.Logger, dbClient database.DatabaseClient, dbc return nil } -func GetListDbHomeRsp(logger logr.Logger, dbClient database.DatabaseClient, dbcs *databasev1alpha1.DbcsSystem) (database.ListDbHomesResponse, error) { +func GetListDbHomeRsp(logger logr.Logger, dbClient database.DatabaseClient, dbcs *databasev4.DbcsSystem) (database.ListDbHomesResponse, error) { dbcsId := *dbcs.Spec.Id CompartmentId := dbcs.Spec.DbSystem.CompartmentId @@ -401,7 +1039,7 @@ func GetListDbHomeRsp(logger logr.Logger, dbClient database.DatabaseClient, dbcs return response, nil } -func GetListDatabaseRsp(logger logr.Logger, dbClient database.DatabaseClient, dbcs *databasev1alpha1.DbcsSystem, dbHomeId string) (database.ListDatabasesResponse, error) { +func GetListDatabaseRsp(logger logr.Logger, dbClient database.DatabaseClient, dbcs *databasev4.DbcsSystem, dbHomeId string) (database.ListDatabasesResponse, error) { CompartmentId := dbcs.Spec.DbSystem.CompartmentId @@ -418,36 +1056,136 @@ func GetListDatabaseRsp(logger logr.Logger, dbClient database.DatabaseClient, db return response, nil } -func UpdateDbcsSystemIdInst(log logr.Logger, dbClient database.DatabaseClient, dbcs *databasev1alpha1.DbcsSystem, kubeClient client.Client, nwClient core.VirtualNetworkClient, wrClient workrequests.WorkRequestClient) error { - //logger := log.WithName("UpdateDbcsSystemInstance") - +func UpdateDbcsSystemIdInst(log logr.Logger, dbClient database.DatabaseClient, dbcs *databasev4.DbcsSystem, kubeClient client.Client, nwClient core.VirtualNetworkClient, wrClient workrequests.WorkRequestClient, databaseID string) error { + // log.Info("Existing DB System Getting Updated with new details in UpdateDbcsSystemIdInst") + var err error updateFlag := false updateDbcsDetails := database.UpdateDbSystemDetails{} - oldSpec, err := dbcs.GetLastSuccessfulSpec() + // log.Info("Current annotations", "annotations", dbcs.GetAnnotations()) + oldSpec, err := dbcs.GetLastSuccessfulSpecWithLog(log) // Use the new method if err != nil { + log.Error(err, "Failed to get last successful spec") return err } - if dbcs.Spec.DbSystem.CpuCoreCount > 0 && dbcs.Spec.DbSystem.CpuCoreCount != oldSpec.DbSystem.CpuCoreCount { + if oldSpec == nil { + log.Info("oldSpec is nil") + } else { + log.Info("Details of oldSpec", "oldSpec", oldSpec) + } + log.Info("Details of updateFlag -> " + fmt.Sprint(updateFlag)) + + if dbcs.Spec.DbSystem.CpuCoreCount > 0 && ((dbcs.Spec.DbSystem.CpuCoreCount != oldSpec.DbSystem.CpuCoreCount) || (dbcs.Spec.DbSystem.CpuCoreCount != *&dbcs.Status.CpuCoreCount)) { + log.Info("DB System cpu core count is: " + fmt.Sprint(dbcs.Spec.DbSystem.CpuCoreCount) + " DB System old cpu count is: " + fmt.Sprint(oldSpec.DbSystem.CpuCoreCount)) updateDbcsDetails.CpuCoreCount = common.Int(dbcs.Spec.DbSystem.CpuCoreCount) updateFlag = true } - if dbcs.Spec.DbSystem.Shape != "" && dbcs.Spec.DbSystem.Shape != oldSpec.DbSystem.Shape { + if dbcs.Spec.DbSystem.Shape != "" && ((dbcs.Spec.DbSystem.Shape != oldSpec.DbSystem.Shape) || (dbcs.Spec.DbSystem.Shape != *dbcs.Status.Shape)) { + // log.Info("DB System desired shape is :" + string(dbcs.Spec.DbSystem.Shape) + "DB System old shape is " + string(oldSpec.DbSystem.Shape)) updateDbcsDetails.Shape = common.String(dbcs.Spec.DbSystem.Shape) updateFlag = true } - if dbcs.Spec.DbSystem.LicenseModel != "" && dbcs.Spec.DbSystem.LicenseModel != oldSpec.DbSystem.LicenseModel { + if dbcs.Spec.DbSystem.LicenseModel != "" && ((dbcs.Spec.DbSystem.LicenseModel != oldSpec.DbSystem.LicenseModel) || (dbcs.Spec.DbSystem.LicenseModel != *&dbcs.Status.LicenseModel)) { licenceModel := getLicenceModel(dbcs) + // log.Info("DB System desired License Model is :" + string(dbcs.Spec.DbSystem.LicenseModel) + "DB Sytsem old License Model is " + string(oldSpec.DbSystem.LicenseModel)) updateDbcsDetails.LicenseModel = database.UpdateDbSystemDetailsLicenseModelEnum(licenceModel) updateFlag = true } if dbcs.Spec.DbSystem.InitialDataStorageSizeInGB != 0 && dbcs.Spec.DbSystem.InitialDataStorageSizeInGB != oldSpec.DbSystem.InitialDataStorageSizeInGB { + // log.Info("DB System desired Storage Size is :" + fmt.Sprint(dbcs.Spec.DbSystem.InitialDataStorageSizeInGB) + "DB System old Storage Size is " + fmt.Sprint(oldSpec.DbSystem.InitialDataStorageSizeInGB)) updateDbcsDetails.DataStorageSizeInGBs = &dbcs.Spec.DbSystem.InitialDataStorageSizeInGB updateFlag = true } + // // Check and update KMS details if necessary + if (dbcs.Spec.KMSConfig != databasev4.KMSConfig{}) { + if dbcs.Spec.KMSConfig != oldSpec.DbSystem.KMSConfig { + log.Info("Updating KMS details in Existing Database") + + kmsKeyID := dbcs.Status.KMSDetailsStatus.KeyId + vaultID := dbcs.Status.KMSDetailsStatus.VaultId + tdeWalletPassword := "" + if dbcs.Spec.DbSystem.TdeWalletPasswordSecret != "" { + tdeWalletPassword, err = GetTdePassword(kubeClient, dbcs) + if err != nil { + log.Error(err, "Failed to get TDE wallet password") + } + } else { + log.Info("Its mandatory to define Tde wallet password when KMS Vault is defined. Not updating existing database") + return nil + } + dbAdminPassword := "" + if dbcs.Spec.DbSystem.DbAdminPasswordSecret != "" { + dbAdminPassword, err = GetAdminPassword(kubeClient, dbcs) + if err != nil { + log.Error(err, "Failed to get DB Admin password") + } + } + + // Assign all available fields to KMSConfig + dbcs.Spec.DbSystem.KMSConfig = databasev4.KMSConfig{ + VaultName: dbcs.Spec.KMSConfig.VaultName, + CompartmentId: dbcs.Spec.KMSConfig.CompartmentId, + KeyName: dbcs.Spec.KMSConfig.KeyName, + EncryptionAlgo: dbcs.Spec.KMSConfig.EncryptionAlgo, + VaultType: dbcs.Spec.KMSConfig.VaultType, + } + + // Create the migrate vault key request + migrateRequest := database.MigrateVaultKeyRequest{ + DatabaseId: common.String(databaseID), + MigrateVaultKeyDetails: database.MigrateVaultKeyDetails{ + KmsKeyId: common.String(kmsKeyID), + VaultId: common.String(vaultID), + }, + } + if tdeWalletPassword != "" { + migrateRequest.TdeWalletPassword = common.String(tdeWalletPassword) + } + if dbAdminPassword != "" { + migrateRequest.AdminPassword = common.String(dbAdminPassword) + } + // Change the phase to "Updating" + if statusErr := SetLifecycleState(kubeClient, dbClient, dbcs, databasev4.Update, nwClient, wrClient); statusErr != nil { + return statusErr + } + // Send the request + migrateResponse, err := dbClient.MigrateVaultKey(context.TODO(), migrateRequest) + if err != nil { + log.Error(err, "Failed to migrate vault key") + return err + } + + // // Check for additional response details (if any) + if migrateResponse.RawResponse.StatusCode != 200 { + log.Error(fmt.Errorf("unexpected status code"), "Migrate vault key request failed", "StatusCode", migrateResponse.RawResponse.StatusCode) + return fmt.Errorf("MigrateVaultKey request failed with status code %d", migrateResponse.RawResponse.StatusCode) + } + + log.Info("MigrateVaultKey request succeeded, waiting for database to reach the desired state") + + // // Wait for the database to reach the desired state after migration, timeout for 2 hours + // Define timeout and check interval + timeout := 2 * time.Hour + checkInterval := 1 * time.Minute + + err = WaitForDatabaseState(log, dbClient, databaseID, "AVAILABLE", timeout, checkInterval) + if err != nil { + log.Error(err, "Database did not reach the desired state within the timeout period") + return err + } + // Change the phase to "Available" + if statusErr := SetLifecycleState(kubeClient, dbClient, dbcs, databasev4.Available, nwClient, wrClient); statusErr != nil { + return statusErr + } + + log.Info("KMS migration process completed successfully") + } + } + + log.Info("Details of updateFlag after validations is " + fmt.Sprint(updateFlag)) if updateFlag { updateDbcsRequest := database.UpdateDbSystemRequest{ DbSystemId: common.String(*dbcs.Spec.Id), @@ -459,7 +1197,7 @@ func UpdateDbcsSystemIdInst(log logr.Logger, dbClient database.DatabaseClient, d } // Change the phase to "Provisioning" - if statusErr := SetLifecycleState(kubeClient, dbClient, dbcs, databasev1alpha1.Update, nwClient, wrClient); statusErr != nil { + if statusErr := SetLifecycleState(kubeClient, dbClient, dbcs, databasev4.Update, nwClient, wrClient); statusErr != nil { return statusErr } // Check the State @@ -467,13 +1205,58 @@ func UpdateDbcsSystemIdInst(log logr.Logger, dbClient database.DatabaseClient, d if err != nil { return err } - } return nil } -func UpdateDbcsSystemId(kubeClient client.Client, dbcs *databasev1alpha1.DbcsSystem) error { +func WaitForDatabaseState( + log logr.Logger, + dbClient database.DatabaseClient, + databaseId string, + desiredState database.DbHomeLifecycleStateEnum, + timeout time.Duration, + checkInterval time.Duration, +) error { + // Set a deadline for the timeout + deadline := time.Now().Add(timeout) + + log.Info("Starting to wait for the database to reach the desired state", "DatabaseID", databaseId, "DesiredState", desiredState, "Timeout", timeout) + + for time.Now().Before(deadline) { + // Prepare the request to fetch database details + getDatabaseReq := database.GetDatabaseRequest{ + DatabaseId: &databaseId, + } + + // Fetch database details + databaseResp, err := dbClient.GetDatabase(context.TODO(), getDatabaseReq) + if err != nil { + log.Error(err, "Failed to get database details", "DatabaseID", databaseId) + return err + } + + // Log the current database state + log.Info("Database State", "DatabaseID", databaseId, "CurrentState", databaseResp.LifecycleState) + + // Check if the database has reached the desired state + if databaseResp.LifecycleState == database.DatabaseLifecycleStateEnum(desiredState) { + log.Info("Database reached the desired state", "DatabaseID", databaseId, "State", desiredState) + return nil + } + + // Wait for the specified interval before checking again + log.Info("Database not in the desired state yet, waiting...", "DatabaseID", databaseId, "CurrentState", databaseResp.LifecycleState, "DesiredState", desiredState, "NextCheckIn", checkInterval) + time.Sleep(checkInterval) + } + + // Return an error if the timeout is reached + err := fmt.Errorf("timed out waiting for database to reach the desired state: %s", desiredState) + log.Error(err, "Timeout reached while waiting for the database to reach the desired state", "DatabaseID", databaseId) + return err +} + +func UpdateDbcsSystemId(kubeClient client.Client, dbcs *databasev4.DbcsSystem) error { payload := []annotations.PatchValue{{ Op: "replace", Path: "/spec/details", @@ -491,7 +1274,6 @@ func UpdateDbcsSystemId(kubeClient client.Client, dbcs *databasev1alpha1.DbcsSys func CheckResourceState(logger logr.Logger, dbClient database.DatabaseClient, Id string, currentState string, expectedState string) (string, error) { // The database OCID is not available when the provisioning is onging. // Retry until the new DbcsSystem is ready. - // Retry up to 18 times every 10 seconds. var state string var err error @@ -534,7 +1316,12 @@ func GetResourceState(logger logr.Logger, dbClient database.DatabaseClient, Id s return state, nil } -func SetDBCSStatus(dbClient database.DatabaseClient, dbcs *databasev1alpha1.DbcsSystem, nwClient core.VirtualNetworkClient, wrClient workrequests.WorkRequestClient) error { +func SetDBCSStatus(dbClient database.DatabaseClient, dbcs *databasev4.DbcsSystem, nwClient core.VirtualNetworkClient, wrClient workrequests.WorkRequestClient) error { + + if dbcs.Spec.Id == nil { + dbcs.Status.State = "FAILED" + return nil + } if dbcs.Spec.Id == nil { dbcs.Status.State = "FAILED" @@ -571,6 +1358,15 @@ func SetDBCSStatus(dbClient database.DatabaseClient, dbcs *databasev1alpha1.Dbcs dbcs.Status.Network.ListenerPort = resp.ListenerPort dbcs.Status.Network.HostName = *resp.Hostname dbcs.Status.Network.DomainName = *resp.Domain + if dbcs.Spec.KMSConfig.CompartmentId != "" { + dbcs.Status.KMSDetailsStatus.CompartmentId = dbcs.Spec.KMSConfig.CompartmentId + dbcs.Status.KMSDetailsStatus.VaultName = dbcs.Spec.KMSConfig.VaultName + } + dbcs.Status.State = databasev4.LifecycleState(resp.LifecycleState) + if dbcs.Spec.KMSConfig.CompartmentId != "" { + dbcs.Status.KMSDetailsStatus.CompartmentId = dbcs.Spec.KMSConfig.CompartmentId + dbcs.Status.KMSDetailsStatus.VaultName = dbcs.Spec.KMSConfig.VaultName + } sname, vcnId, err := getSubnetName(*resp.SubnetId, nwClient) @@ -585,7 +1381,7 @@ func SetDBCSStatus(dbClient database.DatabaseClient, dbcs *databasev1alpha1.Dbcs } // Work Request Ststaus - dbWorkRequest := databasev1alpha1.DbWorkrequests{} + dbWorkRequest := databasev4.DbWorkrequests{} dbWorks, err := getWorkRequest(*resp.OpcRequestId, wrClient, dbcs) if err == nil { @@ -605,11 +1401,11 @@ func SetDBCSStatus(dbClient database.DatabaseClient, dbcs *databasev1alpha1.Dbcs dbWorkRequest.TimeStarted = dbWork.TimeStarted.String() } - if dbWorkRequest != (databasev1alpha1.DbWorkrequests{}) { + if dbWorkRequest != (databasev4.DbWorkrequests{}) { status := checkValue(dbcs, dbWork.Id) if status == 0 { dbcs.Status.WorkRequests = append(dbcs.Status.WorkRequests, dbWorkRequest) - dbWorkRequest = databasev1alpha1.DbWorkrequests{} + dbWorkRequest = databasev4.DbWorkrequests{} } else { setValue(dbcs, dbWorkRequest) } @@ -620,7 +1416,7 @@ func SetDBCSStatus(dbClient database.DatabaseClient, dbcs *databasev1alpha1.Dbcs // DB Home Status dbcs.Status.DbInfo = dbcs.Status.DbInfo[:0] - dbStatus := databasev1alpha1.DbStatus{} + dbStatus := databasev4.DbStatus{} dbHomes, err := getDbHomeList(dbClient, dbcs) @@ -636,14 +1432,14 @@ func SetDBCSStatus(dbClient database.DatabaseClient, dbcs *databasev1alpha1.Dbcs dbStatus.DbWorkload = *dbDetail.DbWorkload } dbcs.Status.DbInfo = append(dbcs.Status.DbInfo, dbStatus) - dbStatus = databasev1alpha1.DbStatus{} + dbStatus = databasev4.DbStatus{} } } } return nil } -func getDbHomeList(dbClient database.DatabaseClient, dbcs *databasev1alpha1.DbcsSystem) ([]database.DbHomeSummary, error) { +func getDbHomeList(dbClient database.DatabaseClient, dbcs *databasev4.DbcsSystem) ([]database.DbHomeSummary, error) { var items []database.DbHomeSummary dbcsId := *dbcs.Spec.Id @@ -661,7 +1457,7 @@ func getDbHomeList(dbClient database.DatabaseClient, dbcs *databasev1alpha1.Dbcs return resp.Items, nil } -func getDList(dbClient database.DatabaseClient, dbcs *databasev1alpha1.DbcsSystem, dbHomeId *string) ([]database.DatabaseSummary, error) { +func getDList(dbClient database.DatabaseClient, dbcs *databasev4.DbcsSystem, dbHomeId *string) ([]database.DatabaseSummary, error) { dbcsId := *dbcs.Spec.Id var items []database.DatabaseSummary @@ -710,7 +1506,7 @@ func getVcnName(vcnId *string, nwClient core.VirtualNetworkClient) (*string, err } // =========== validate Specs ============ -func ValidateSpex(logger logr.Logger, kubeClient client.Client, dbClient database.DatabaseClient, dbcs *databasev1alpha1.DbcsSystem, nwClient core.VirtualNetworkClient, eRecord record.EventRecorder) error { +func ValidateSpex(logger logr.Logger, kubeClient client.Client, dbClient database.DatabaseClient, dbcs *databasev4.DbcsSystem, nwClient core.VirtualNetworkClient, eRecord record.EventRecorder) error { //var str1 string var eventMsg string diff --git a/commons/dbcssystem/dcommon.go b/commons/dbcssystem/dcommon.go index 3af3a6b4..beaa7c38 100644 --- a/commons/dbcssystem/dcommon.go +++ b/commons/dbcssystem/dcommon.go @@ -49,10 +49,10 @@ import ( "github.com/oracle/oci-go-sdk/v65/workrequests" "sigs.k8s.io/controller-runtime/pkg/client" - databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + databasev4 "github.com/oracle/oracle-database-operator/apis/database/v4" ) -func GetDbHomeDetails(kubeClient client.Client, dbClient database.DatabaseClient, dbcs *databasev1alpha1.DbcsSystem) (database.CreateDbHomeDetails, error) { +func GetDbHomeDetails(kubeClient client.Client, dbClient database.DatabaseClient, dbcs *databasev4.DbcsSystem) (database.CreateDbHomeDetails, error) { dbHomeDetails := database.CreateDbHomeDetails{} @@ -72,7 +72,7 @@ func GetDbHomeDetails(kubeClient client.Client, dbClient database.DatabaseClient return dbHomeDetails, nil } -func GetDbLatestVersion(dbClient database.DatabaseClient, dbcs *databasev1alpha1.DbcsSystem, dbSystemId string) (string, error) { +func GetDbLatestVersion(dbClient database.DatabaseClient, dbcs *databasev4.DbcsSystem, dbSystemId string) (string, error) { //var provisionedDbcsSystemId string ctx := context.TODO() @@ -105,27 +105,34 @@ func GetDbLatestVersion(dbClient database.DatabaseClient, dbcs *databasev1alpha1 s2 := getStr(dbcs.Spec.DbSystem.DbVersion, 2) if strings.EqualFold(s1, s2) { val, _ = strconv.Atoi(s1) - if val >= 18 { + if val >= 18 && val <= 21 { s3 := s1 + "c" if strings.EqualFold(s3, dbcs.Spec.DbSystem.DbVersion) { sFlag = 1 break } + } else if val >= 23 { + s3 := s1 + "ai" + if strings.EqualFold(s3, dbcs.Spec.DbSystem.DbVersion) { + sFlag = 1 + break + } + } else if val < 18 && val >= 11 { + s4 := getStr(*version.Version, 4) + if strings.EqualFold(s4, dbcs.Spec.DbSystem.DbVersion) { + sFlag = 1 + break + } } - } else if val < 18 && val >= 11 { - s4 := getStr(*version.Version, 4) - if strings.EqualFold(s4, dbcs.Spec.DbSystem.DbVersion) { - sFlag = 1 - break - } - } + } } } if sFlag == 1 { return *version.Version, nil } + return *version.Version, fmt.Errorf("no database version matched") } @@ -133,7 +140,7 @@ func getStr(str1 string, num int) string { return str1[0:num] } -func GetDBDetails(kubeClient client.Client, dbcs *databasev1alpha1.DbcsSystem) (database.CreateDatabaseDetails, error) { +func GetDBDetails(kubeClient client.Client, dbcs *databasev4.DbcsSystem) (database.CreateDatabaseDetails, error) { dbDetails := database.CreateDatabaseDetails{} var val database.CreateDatabaseDetailsDbWorkloadEnum @@ -188,7 +195,7 @@ func GetDBDetails(kubeClient client.Client, dbcs *databasev1alpha1.DbcsSystem) ( return dbDetails, nil } -func getBackupConfig(kubeClient client.Client, dbcs *databasev1alpha1.DbcsSystem) (database.DbBackupConfig, error) { +func getBackupConfig(kubeClient client.Client, dbcs *databasev4.DbcsSystem) (database.DbBackupConfig, error) { backupConfig := database.DbBackupConfig{} if dbcs.Spec.DbSystem.DbBackupConfig.AutoBackupEnabled != nil { @@ -216,7 +223,7 @@ func getBackupConfig(kubeClient client.Client, dbcs *databasev1alpha1.DbcsSystem return backupConfig, nil } -func getBackupWindowEnum(dbcs *databasev1alpha1.DbcsSystem) (database.DbBackupConfigAutoBackupWindowEnum, error) { +func getBackupWindowEnum(dbcs *databasev4.DbcsSystem) (database.DbBackupConfigAutoBackupWindowEnum, error) { if strings.ToUpper(*dbcs.Spec.DbSystem.DbBackupConfig.AutoBackupWindow) == "SLOT_ONE" { return database.DbBackupConfigAutoBackupWindowOne, nil @@ -251,7 +258,7 @@ func getBackupWindowEnum(dbcs *databasev1alpha1.DbcsSystem) (database.DbBackupCo //return database.DbBackupConfigAutoBackupWindowEight, fmt.Errorf("AutoBackupWindow values can be SLOT_ONE|SLOT_TWO|SLOT_THREE|SLOT_FOUR|SLOT_FIVE|SLOT_SIX|SLOT_SEVEN|SLOT_EIGHT|SLOT_NINE|SLOT_TEN|SLOT_ELEVEN|SLOT_TWELEVE. The current value set to " + *dbcs.Spec.DbSystem.DbBackupConfig.AutoBackupWindow) } -func getRecoveryWindowsInDays(dbcs *databasev1alpha1.DbcsSystem) (int, error) { +func getRecoveryWindowsInDays(dbcs *databasev4.DbcsSystem) (int, error) { var days int @@ -274,7 +281,7 @@ func getRecoveryWindowsInDays(dbcs *databasev1alpha1.DbcsSystem) (int, error) { } func GetDBSystemopts( - dbcs *databasev1alpha1.DbcsSystem) database.DbSystemOptions { + dbcs *databasev4.DbcsSystem) database.DbSystemOptions { dbSystemOpt := database.DbSystemOptions{} @@ -294,7 +301,7 @@ func GetDBSystemopts( return dbSystemOpt } -func getLicenceModel(dbcs *databasev1alpha1.DbcsSystem) database.DbSystemLicenseModelEnum { +func getLicenceModel(dbcs *databasev4.DbcsSystem) database.DbSystemLicenseModelEnum { if dbcs.Spec.DbSystem.LicenseModel == "BRING_YOUR_OWN_LICENSE" { return database.DbSystemLicenseModelBringYourOwnLicense @@ -302,7 +309,7 @@ func getLicenceModel(dbcs *databasev1alpha1.DbcsSystem) database.DbSystemLicense return database.DbSystemLicenseModelLicenseIncluded } -func getDbWorkLoadType(dbcs *databasev1alpha1.DbcsSystem) (database.CreateDatabaseDetailsDbWorkloadEnum, error) { +func getDbWorkLoadType(dbcs *databasev4.DbcsSystem) (database.CreateDatabaseDetailsDbWorkloadEnum, error) { if strings.ToUpper(dbcs.Spec.DbSystem.DbWorkload) == "OLTP" { @@ -317,7 +324,7 @@ func getDbWorkLoadType(dbcs *databasev1alpha1.DbcsSystem) (database.CreateDataba } func GetNodeCount( - dbcs *databasev1alpha1.DbcsSystem) int { + dbcs *databasev4.DbcsSystem) int { if dbcs.Spec.DbSystem.NodeCount != nil { return *dbcs.Spec.DbSystem.NodeCount @@ -327,7 +334,7 @@ func GetNodeCount( } func GetInitialStorage( - dbcs *databasev1alpha1.DbcsSystem) int { + dbcs *databasev4.DbcsSystem) int { if dbcs.Spec.DbSystem.InitialDataStorageSizeInGB > 0 { return dbcs.Spec.DbSystem.InitialDataStorageSizeInGB @@ -335,7 +342,7 @@ func GetInitialStorage( return 256 } -func GetDBEdition(dbcs *databasev1alpha1.DbcsSystem) database.LaunchDbSystemDetailsDatabaseEditionEnum { +func GetDBEdition(dbcs *databasev4.DbcsSystem) database.LaunchDbSystemDetailsDatabaseEditionEnum { if dbcs.Spec.DbSystem.ClusterName != "" { return database.LaunchDbSystemDetailsDatabaseEditionEnterpriseEditionExtremePerformance @@ -360,7 +367,7 @@ func GetDBEdition(dbcs *databasev1alpha1.DbcsSystem) database.LaunchDbSystemDeta } func GetDBbDiskRedundancy( - dbcs *databasev1alpha1.DbcsSystem) database.LaunchDbSystemDetailsDiskRedundancyEnum { + dbcs *databasev4.DbcsSystem) database.LaunchDbSystemDetailsDiskRedundancyEnum { if dbcs.Spec.DbSystem.ClusterName != "" { return database.LaunchDbSystemDetailsDiskRedundancyHigh @@ -376,7 +383,7 @@ func GetDBbDiskRedundancy( return database.LaunchDbSystemDetailsDiskRedundancyNormal } -func getWorkRequest(workId string, wrClient workrequests.WorkRequestClient, dbcs *databasev1alpha1.DbcsSystem) ([]workrequests.WorkRequestSummary, error) { +func getWorkRequest(workId string, wrClient workrequests.WorkRequestClient, dbcs *databasev4.DbcsSystem) ([]workrequests.WorkRequestSummary, error) { var workReq []workrequests.WorkRequestSummary req := workrequests.ListWorkRequestsRequest{CompartmentId: &dbcs.Spec.DbSystem.CompartmentId, OpcRequestId: &workId, ResourceId: dbcs.Spec.Id} @@ -405,10 +412,10 @@ func GetFmtStr(pstr string) string { return "[" + pstr + "]" } -func checkValue(dbcs *databasev1alpha1.DbcsSystem, workId *string) int { +func checkValue(dbcs *databasev4.DbcsSystem, workId *string) int { var status int = 0 - //dbWorkRequest := databasev1alpha1.DbWorkrequests{} + //dbWorkRequest := databasev4.DbWorkrequests{} if len(dbcs.Status.WorkRequests) > 0 { for _, v := range dbcs.Status.WorkRequests { @@ -420,10 +427,10 @@ func checkValue(dbcs *databasev1alpha1.DbcsSystem, workId *string) int { return status } -func setValue(dbcs *databasev1alpha1.DbcsSystem, dbWorkRequest databasev1alpha1.DbWorkrequests) { +func setValue(dbcs *databasev4.DbcsSystem, dbWorkRequest databasev4.DbWorkrequests) { //var status int = 1 - //dbWorkRequest := databasev1alpha1.DbWorkrequests{} + //dbWorkRequest := databasev4.DbWorkrequests{} var counter int = 0 if len(dbcs.Status.WorkRequests) > 0 { for _, v := range dbcs.Status.WorkRequests { diff --git a/commons/k8s/create.go b/commons/k8s/create.go index 5055bc0e..cd836af7 100644 --- a/commons/k8s/create.go +++ b/commons/k8s/create.go @@ -41,7 +41,7 @@ package k8s import ( "context" - dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + dbv4 "github.com/oracle/oracle-database-operator/apis/database/v4" "github.com/oracle/oci-go-sdk/v65/common" "github.com/oracle/oci-go-sdk/v65/database" @@ -78,25 +78,28 @@ func CreateSecret(kubeClient client.Client, namespace string, name string, data func CreateAutonomousBackup(kubeClient client.Client, backupName string, backupSummary database.AutonomousDatabaseBackupSummary, - ownerADB *dbv1alpha1.AutonomousDatabase) error { + ownerAdb *dbv4.AutonomousDatabase) error { - backup := &dbv1alpha1.AutonomousDatabaseBackup{ + backup := &dbv4.AutonomousDatabaseBackup{ ObjectMeta: metav1.ObjectMeta{ - Namespace: ownerADB.GetNamespace(), + Namespace: ownerAdb.GetNamespace(), Name: backupName, - OwnerReferences: NewOwnerReference(ownerADB), + OwnerReferences: NewOwnerReference(ownerAdb), + Labels: map[string]string{ + "adb": ownerAdb.Name, + }, }, - Spec: dbv1alpha1.AutonomousDatabaseBackupSpec{ - Target: dbv1alpha1.TargetSpec{ - K8sADB: dbv1alpha1.K8sADBSpec{ - Name: common.String(ownerADB.Name), + Spec: dbv4.AutonomousDatabaseBackupSpec{ + Target: dbv4.TargetSpec{ + K8sAdb: dbv4.K8sAdbSpec{ + Name: common.String(ownerAdb.Name), }, }, DisplayName: backupSummary.DisplayName, AutonomousDatabaseBackupOCID: backupSummary.Id, - OCIConfig: dbv1alpha1.OCIConfigSpec{ - ConfigMapName: ownerADB.Spec.OCIConfig.ConfigMapName, - SecretName: ownerADB.Spec.OCIConfig.SecretName, + OCIConfig: dbv4.OciConfigSpec{ + ConfigMapName: ownerAdb.Spec.OciConfig.ConfigMapName, + SecretName: ownerAdb.Spec.OciConfig.SecretName, }, }, } diff --git a/commons/k8s/fetch.go b/commons/k8s/fetch.go index 05792cad..617abdb5 100644 --- a/commons/k8s/fetch.go +++ b/commons/k8s/fetch.go @@ -44,10 +44,11 @@ import ( corev1 "k8s.io/api/core/v1" apiErrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/labels" "k8s.io/apimachinery/pkg/types" "sigs.k8s.io/controller-runtime/pkg/client" - dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + dbv4 "github.com/oracle/oracle-database-operator/apis/database/v4" ) func FetchResource(kubeClient client.Client, namespace string, name string, object client.Object) error { @@ -64,16 +65,16 @@ func FetchResource(kubeClient client.Client, namespace string, name string, obje // Returns the first AutonomousDatabase resource that matches the AutonomousDatabaseOCID of the backup // Sometimes the AutonomousDatabase doesn't exist. It could happen if a user simply want to restore or -// backup the ADB without creating an ADB rersource in the cluster. +// backup the AutonomousDatabase without creating an AutonomousDatabase rersource in the cluster. // If there isn't an AutonomousDatabase with the same OCID, a nil is returned. -func FetchAutonomousDatabaseWithOCID(kubeClient client.Client, namespace string, ocid string) (*dbv1alpha1.AutonomousDatabase, error) { +func FetchAutonomousDatabaseWithOCID(kubeClient client.Client, namespace string, ocid string) (*dbv4.AutonomousDatabase, error) { adbList, err := fetchAutonomousDatabases(kubeClient, namespace) if err != nil { return nil, err } for _, adb := range adbList.Items { - if adb.Spec.Details.AutonomousDatabaseOCID != nil && *adb.Spec.Details.AutonomousDatabaseOCID == ocid { + if adb.Spec.Details.Id != nil && *adb.Spec.Details.Id == ocid { return &adb, nil } } @@ -81,9 +82,9 @@ func FetchAutonomousDatabaseWithOCID(kubeClient client.Client, namespace string, return nil, nil } -func fetchAutonomousDatabases(kubeClient client.Client, namespace string) (*dbv1alpha1.AutonomousDatabaseList, error) { +func fetchAutonomousDatabases(kubeClient client.Client, namespace string) (*dbv4.AutonomousDatabaseList, error) { // Get the list of AutonomousDatabaseBackupOCID in the same namespace - adbList := &dbv1alpha1.AutonomousDatabaseList{} + adbList := &dbv4.AutonomousDatabaseList{} if err := kubeClient.List(context.TODO(), adbList, &client.ListOptions{Namespace: namespace}); err != nil { // Ignore not-found errors, since they can't be fixed by an immediate requeue. @@ -96,11 +97,20 @@ func fetchAutonomousDatabases(kubeClient client.Client, namespace string) (*dbv1 return adbList, nil } -func FetchAutonomousDatabaseBackups(kubeClient client.Client, namespace string) (*dbv1alpha1.AutonomousDatabaseBackupList, error) { +func FetchAutonomousDatabaseBackups(kubeClient client.Client, namespace string, adbName string) (*dbv4.AutonomousDatabaseBackupList, error) { // Get the list of AutonomousDatabaseBackupOCID in the same namespace - backupList := &dbv1alpha1.AutonomousDatabaseBackupList{} - - if err := kubeClient.List(context.TODO(), backupList, &client.ListOptions{Namespace: namespace}); err != nil { + backupList := &dbv4.AutonomousDatabaseBackupList{} + + // Create a label selector + selector := labels.Set{"adb": adbName}.AsSelector() + + if err := kubeClient.List( + context.TODO(), + backupList, + &client.ListOptions{ + Namespace: namespace, + LabelSelector: selector, + }); err != nil { // Ignore not-found errors, since they can't be fixed by an immediate requeue. // No need to change the since we don't know if we obtain the object. if !apiErrors.IsNotFound(err) { diff --git a/commons/multitenant/lrest/common.go b/commons/multitenant/lrest/common.go new file mode 100644 index 00000000..e72e85b0 --- /dev/null +++ b/commons/multitenant/lrest/common.go @@ -0,0 +1,113 @@ +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if + one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. +*/ + +package lrest + +import ( + "crypto/rsa" + "crypto/x509" + "encoding/base64" + "encoding/pem" + "fmt" + "regexp" + "strings" + + corev1 "k8s.io/api/core/v1" + + ctrl "sigs.k8s.io/controller-runtime" +) + +func CommonDecryptWithPrivKey(Key string, Buffer string, req ctrl.Request) (string, error) { + + Debug := 0 + block, _ := pem.Decode([]byte(Key)) + pkcs8PrivateKey, err := x509.ParsePKCS8PrivateKey(block.Bytes) + if err != nil { + fmt.Printf("Failed to parse private key %s \n", err.Error()) + return "", err + } + if Debug == 1 { + fmt.Printf("======================================\n") + fmt.Printf("%s\n", Key) + fmt.Printf("======================================\n") + } + + encString64, err := base64.StdEncoding.DecodeString(string(Buffer)) + if err != nil { + fmt.Printf("Failed to decode encrypted string to base64: %s\n", err.Error()) + return "", err + } + + decryptedB, err := rsa.DecryptPKCS1v15(nil, pkcs8PrivateKey.(*rsa.PrivateKey), encString64) + if err != nil { + fmt.Printf("Failed to decrypt string %s\n", err.Error()) + return "", err + } + if Debug == 1 { + fmt.Printf("[%s]\n", string(decryptedB)) + } + return strings.TrimSpace(string(decryptedB)), err + +} + +func ParseConfigMapData(cfgmap *corev1.ConfigMap) []string { + + var tokens []string + for Key, Value := range cfgmap.Data { + fmt.Printf("KEY:%s\n", Key) + re0 := regexp.MustCompile("\\n") + re1 := regexp.MustCompile(";") + re2 := regexp.MustCompile(",") /* Additional separator for future use */ + + Value = re0.ReplaceAllString(Value, " ") + tokens = strings.Split(Value, " ") + + for cnt := range tokens { + if len(tokens[cnt]) != 0 { + tokens[cnt] = re1.ReplaceAllString(tokens[cnt], " ") + tokens[cnt] = re2.ReplaceAllString(tokens[cnt], " ") + + } + + } + + } + + return tokens + +} diff --git a/commons/observability/constants.go b/commons/observability/constants.go index 89ecb946..45f06e49 100644 --- a/commons/observability/constants.go +++ b/commons/observability/constants.go @@ -1,6 +1,8 @@ package observability -import "github.com/oracle/oracle-database-operator/apis/observability/v1alpha1" +import ( + v4 "github.com/oracle/oracle-database-operator/apis/observability/v4" +) const ( UnknownValue = "UNKNOWN" @@ -9,9 +11,9 @@ const ( // Observability Status const ( - StatusObservabilityPending v1alpha1.StatusEnum = "PENDING" - StatusObservabilityError v1alpha1.StatusEnum = "ERROR" - StatusObservabilityReady v1alpha1.StatusEnum = "READY" + StatusObservabilityPending v4.StatusEnum = "PENDING" + StatusObservabilityError v4.StatusEnum = "ERROR" + StatusObservabilityReady v4.StatusEnum = "READY" ) // Log Names @@ -27,8 +29,9 @@ const ( DefaultDbUserKey = "username" DefaultDBPasswordKey = "password" DefaultDBConnectionStringKey = "connection" - DefaultLabelKey = "app" DefaultConfigVolumeString = "config-volume" + DefaultLogFilename = "alert.log" + DefaultLogVolumeString = "log-volume" DefaultWalletVolumeString = "creds" DefaultOCIPrivateKeyVolumeString = "ocikey" DefaultOCIConfigFingerprintKey = "fingerprint" @@ -36,10 +39,12 @@ const ( DefaultOCIConfigTenancyKey = "tenancy" DefaultOCIConfigUserKey = "user" - DefaultExporterImage = "container-registry.oracle.com/database/observability-exporter:1.1.0" + DefaultExporterImage = "container-registry.oracle.com/database/observability-exporter:1.5.2" DefaultServicePort = 9161 DefaultServiceTargetPort = 9161 + DefaultAppPort = 8080 DefaultPrometheusPort = "metrics" + DefaultServiceType = "ClusterIP" DefaultReplicaCount = 1 DefaultExporterConfigMountRootPath = "/oracle/observability" DefaultOracleHome = "/lib/oracle/21/client64/lib" @@ -52,12 +57,15 @@ const ( DefaultExporterConfigmapAbsolutePath = DefaultExporterConfigMountRootPath + "/" + DefaultExporterConfigmapFilename ) -// default resource prefixes +// labeling +const ( + DefaultSelectorLabelKey = "app" + DefaultReleaseLabelKey = "release" +) + +// default resource const ( - DefaultServiceMonitorPrefix = "obs-servicemonitor-" - DefaultLabelPrefix = "obs-" - DefaultExporterDeploymentPrefix = "obs-deploy-" - DefaultExporterContainerName = "observability-exporter" + DefaultExporterContainerName = "observability-exporter" ) // Known environment variables @@ -66,6 +74,7 @@ const ( EnvVarDataSourceUser = "DB_USERNAME" EnvVarDataSourcePassword = "DB_PASSWORD" EnvVarDataSourceConnectString = "DB_CONNECT_STRING" + EnvVarDataSourceLogDestination = "LOG_DESTINATION" EnvVarDataSourcePwdVaultSecretName = "VAULT_SECRET_NAME" EnvVarDataSourcePwdVaultId = "VAULT_ID" EnvVarCustomConfigmap = "CUSTOM_METRICS" @@ -93,11 +102,11 @@ const ( ReasonReadyFailed = "ReadinessValidationFailed" ReasonDeploymentSpecValidationFailed = "SpecValidationFailed" - ReasonDeploymentSuccessful = "ResourceDeployed" - ReasonDeploymentUpdated = "ResourceDeploymentUpdated" - ReasonDeploymentUpdateFailed = "ResourceDeploymentUpdateFailed" - ReasonDeploymentFailed = "ResourceDeploymentFailed" - ReasonDeploymentPending = "ResourceDeploymentInProgress" + ReasonDeploymentSuccessful = "ResourceDeployed" + ReasonResourceUpdated = "ResourceUpdated" + ReasonResourceUpdateFailed = "ResourceUpdateFailed" + ReasonDeploymentFailed = "ResourceDeploymentFailed" + ReasonDeploymentPending = "ResourceDeploymentInProgress" ReasonGeneralResourceGenerationFailed = "ResourceGenerationFailed" ReasonGeneralResourceCreated = "ResourceCreated" @@ -112,18 +121,19 @@ const ( ErrorStatusUpdate = "an error occurred with updating the cr status" ErrorSpecValidationFailedDueToAnError = "an error occurred with validating the exporter deployment spec" ErrorDeploymentPodsFailure = "an error occurred with deploying exporter deployment pods" - ErrorDeploymentUpdate = "an error occurred with updating exporter deployment" ErrorResourceCreationFailure = "an error occurred with creating databaseobserver resource" ErrorResourceRetrievalFailureDueToAnError = "an error occurred with retrieving databaseobserver resource" + LogErrorWithResourceUpdate = "an error occurred with updating resource" ) // Log Infos const ( - LogCRStart = "Started DatabaseObserver instance reconciliation" - LogCREnd = "Ended DatabaseObserver instance reconciliation, resource must have been deleted." - LogResourceCreated = "Created DatabaseObserver resource successfully" - LogResourceUpdated = "Updated DatabaseObserver resource successfully" - LogResourceFound = "Validated DatabaseObserver resource readiness" + LogCRStart = "Started DatabaseObserver instance reconciliation" + LogCREnd = "Ended DatabaseObserver instance reconciliation, resource must have been deleted." + LogResourceCreated = "Created DatabaseObserver resource successfully" + LogResourceUpdated = "Updated DatabaseObserver resource successfully" + LogResourceFound = "Validated DatabaseObserver resource readiness" + LogSuccessWithResourceUpdate = "Updated DatabaseObserver resource successfully" ) // Messages @@ -140,11 +150,8 @@ const ( MessageResourceGenerationFailed = "Failed to generate resource due to an error" MessageExporterDeploymentSpecValidationFailed = "Failed to validate export deployment spec due to an error with the spec" - MessageExporterDeploymentImageUpdated = "Completed updating exporter deployment image successfully" - MessageExporterDeploymentEnvironmentUpdated = "Completed updating exporter deployment environment values successfully" - MessageExporterDeploymentReplicaUpdated = "Completed updating exporter deployment replicaCount successfully" - MessageExporterDeploymentVolumesUpdated = "Completed updating exporter deployment volumes successfully" - MessageExporterDeploymentUpdateFailed = "Failed to update exporter deployment due to an error" + MessageExporterResourceUpdateFailed = "Failed to update exporter resource due to an error" + MessageExporterResourceUpdated = "Updated exporter resource successfully" MessageExporterDeploymentValidationFailed = "Failed to validate exporter deployment due to an error retrieving resource" MessageExporterDeploymentSuccessful = "Completed validation of exporter deployment readiness" MessageExporterDeploymentFailed = "Failed to deploy exporter deployment due to PodFailure" @@ -158,16 +165,11 @@ const ( EventMessageFailedCRRetrieval = "Encountered error retrieving databaseObserver instance" EventReasonSpecError = "DeploymentSpecValidationFailed" - EventMessageSpecErrorDBPasswordMissing = "Spec validation failed due to missing dbPassword field values" EventMessageSpecErrorDBPasswordSecretMissing = "Spec validation failed due to required dbPassword secret not found" EventMessageSpecErrorDBConnectionStringSecretMissing = "Spec validation failed due to required dbConnectionString secret not found" EventMessageSpecErrorDBPUserSecretMissing = "Spec validation failed due to dbUser secret not found" EventMessageSpecErrorConfigmapMissing = "Spec validation failed due to custom config configmap not found" EventMessageSpecErrorDBWalletSecretMissing = "Spec validation failed due to provided dbWallet secret not found" - EventReasonUpdateSucceeded = "ExporterDeploymentUpdated" - EventMessageUpdatedImageSucceeded = "Exporter deployment image updated successfully" - EventMessageUpdatedEnvironmentSucceeded = "Exporter deployment environment values updated successfully" - EventMessageUpdatedVolumesSucceeded = "Exporter deployment volumes updated successfully" - EventMessageUpdatedReplicaSucceeded = "Exporter deployment replicaCount updated successfully" + EventReasonUpdateSucceeded = "ExporterDeploymentUpdated" ) diff --git a/commons/observability/utils.go b/commons/observability/utils.go index f396b95a..6eccb261 100644 --- a/commons/observability/utils.go +++ b/commons/observability/utils.go @@ -1,87 +1,230 @@ package observability import ( - apiv1 "github.com/oracle/oracle-database-operator/apis/observability/v1alpha1" - appsv1 "k8s.io/api/apps/v1" + api "github.com/oracle/oracle-database-operator/apis/observability/v4" + monitorv1 "github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring/v1" corev1 "k8s.io/api/core/v1" + "k8s.io/apimachinery/pkg/util/intstr" + "path/filepath" + "strings" ) -// GetExporterLabels function retrieves exporter labels from api or provides default -func GetExporterLabels(api *apiv1.DatabaseObserver) map[string]string { +func AddSidecarContainers(a *api.DatabaseObserver, listing *[]corev1.Container) { + + if containers := a.Spec.ExporterSidecars; len(containers) > 0 { + for _, container := range containers { + *listing = append(*listing, container) + } + + } +} + +func AddSidecarVolumes(a *api.DatabaseObserver, listing *[]corev1.Volume) { + + if volumes := a.Spec.SideCarVolumes; len(volumes) > 0 { + for _, v := range volumes { + *listing = append(*listing, v) + } + + } +} + +// GetLabels retrieves labels from the spec +func GetLabels(a *api.DatabaseObserver, customResourceLabels map[string]string) map[string]string { + var l = make(map[string]string) - if labels := api.Spec.Prometheus.Labels; labels != nil && len(labels) > 0 { - for k, v := range labels { - l[k] = v + // get inherited labels + if iLabels := a.Spec.InheritLabels; iLabels != nil { + for _, v := range iLabels { + if v != DefaultSelectorLabelKey { + l[v] = a.Labels[v] + } + } + } + + if customResourceLabels != nil { + for k, v := range customResourceLabels { + if k != DefaultSelectorLabelKey { + l[k] = v + } } - l["release"] = "stable" - return l } - return map[string]string{ - DefaultLabelKey: DefaultLabelPrefix + api.Name, - "release": "stable", + + // add app label + l[DefaultSelectorLabelKey] = a.Name + return l +} + +// GetSelectorLabel adds selector label +func GetSelectorLabel(a *api.DatabaseObserver) map[string]string { + selectors := make(map[string]string) + selectors[DefaultSelectorLabelKey] = a.Name + return selectors +} + +// GetExporterVersion retrieves version of exporter used +func GetExporterVersion(a *api.DatabaseObserver) string { + appVersion := "latest" + whichImage := DefaultExporterImage + if img := a.Spec.Exporter.Deployment.ExporterImage; img != "" { + whichImage = img } + // return tag in image:tag + if str := strings.Split(whichImage, ":"); len(str) == 2 { + appVersion = str[1] + } + return appVersion +} + +// GetExporterArgs retrieves args +func GetExporterArgs(a *api.DatabaseObserver) []string { + if args := a.Spec.Exporter.Deployment.ExporterArgs; args != nil || len(args) > 0 { + return args + } + return nil } -// GetExporterServicePort function retrieves exporter service port from api or provides default -func GetExporterServicePort(api *apiv1.DatabaseObserver) int32 { - if rPort := api.Spec.Exporter.Service.Port; rPort != 0 { - return rPort +// GetExporterDeploymentSecurityContext retrieves security context for container +func GetExporterDeploymentSecurityContext(a *api.DatabaseObserver) *corev1.SecurityContext { + if sc := a.Spec.Exporter.Deployment.SecurityContext; sc != nil { + return sc } - return int32(DefaultServicePort) + return &corev1.SecurityContext{} } -// GetExporterServiceMonitorPort function retrieves exporter service monitor port from api or provides default -func GetExporterServiceMonitorPort(api *apiv1.DatabaseObserver) string { - if rPort := api.Spec.Prometheus.Port; rPort != "" { - return rPort +// GetExporterPodSecurityContext retrieves security context for pods +func GetExporterPodSecurityContext(a *api.DatabaseObserver) *corev1.PodSecurityContext { + if sc := a.Spec.Exporter.Deployment.DeploymentPodTemplate.SecurityContext; sc != nil { + return sc } - return DefaultPrometheusPort + return &corev1.PodSecurityContext{} +} +// GetExporterCommands retrieves commands +func GetExporterCommands(a *api.DatabaseObserver) []string { + if c := a.Spec.Exporter.Deployment.ExporterCommands; c != nil || len(c) > 0 { + return c + } + return nil } -// GetExporterDeploymentVolumeMounts function retrieves volume mounts from api or provides default -func GetExporterDeploymentVolumeMounts(api *apiv1.DatabaseObserver) []corev1.VolumeMount { +// GetExporterServicePort function retrieves exporter service port from a or provides default +func GetExporterServicePort(a *api.DatabaseObserver) []corev1.ServicePort { + + servicePorts := make([]corev1.ServicePort, 0) + + // get service ports + if ports := a.Spec.Exporter.Service.Ports; len(ports) > 0 { + for _, port := range ports { + servicePorts = append(servicePorts, port) + } + + } else { + // if not, provide default service port + servicePorts = append(servicePorts, corev1.ServicePort{ + Name: DefaultPrometheusPort, + Port: DefaultServicePort, + TargetPort: intstr.FromInt32(DefaultServiceTargetPort), + }) + } + + return servicePorts + +} + +// GetEndpoints function +func GetEndpoints(a *api.DatabaseObserver) []monitorv1.Endpoint { + + endpoints := make([]monitorv1.Endpoint, 0) + + // get endpoints + if es := a.Spec.Prometheus.ServiceMonitor.Endpoints; len(es) > 0 { + for _, e := range es { + endpoints = append(endpoints, e) + } + } + + // if not, provide default endpoint + endpoints = append(endpoints, monitorv1.Endpoint{ + Port: DefaultPrometheusPort, + Interval: "20s", + }) + + return endpoints +} + +func AddNamespaceSelector(a *api.DatabaseObserver, spec *monitorv1.ServiceMonitorSpec) { + + if ns := a.Spec.Prometheus.ServiceMonitor.NamespaceSelector; ns != nil { + a.Spec.Prometheus.ServiceMonitor.NamespaceSelector.DeepCopyInto(&spec.NamespaceSelector) + } + +} + +// GetExporterDeploymentVolumeMounts function retrieves volume mounts from a or provides default +func GetExporterDeploymentVolumeMounts(a *api.DatabaseObserver) []corev1.VolumeMount { volM := make([]corev1.VolumeMount, 0) - if cVolumeSourceName := api.Spec.Exporter.ExporterConfig.Configmap.Name; cVolumeSourceName != "" { + if cVolumeSourceName := a.Spec.ExporterConfig.Configmap.Name; cVolumeSourceName != "" { volM = append(volM, corev1.VolumeMount{ Name: DefaultConfigVolumeString, MountPath: DefaultExporterConfigMountRootPath, }) } - // api.Spec.Database.DBWallet.SecretName optional + // a.Spec.Database.DBWallet.SecretName optional // if null, consider the database NON-ADB and connect as such - if secretName := api.Spec.Database.DBWallet.SecretName; secretName != "" { + if secretName := a.Spec.Database.DBWallet.SecretName; secretName != "" { + + p := DefaultOracleTNSAdmin + + // Determine what the value of TNS_ADMIN + // if custom TNS_ADMIN environment variable is set and found, use that instead as the path + if rCustomEnvs := a.Spec.Exporter.Deployment.ExporterEnvs; rCustomEnvs != nil { + if v, f := rCustomEnvs[EnvVarTNSAdmin]; f { + p = v + } + } + volM = append(volM, corev1.VolumeMount{ Name: DefaultWalletVolumeString, - MountPath: DefaultOracleTNSAdmin, + MountPath: p, }) } - // api.Spec.OCIConfig.SecretName required if vault is used - if secretName := api.Spec.OCIConfig.SecretName; secretName != "" { + // a.Spec.OCIConfig.SecretName required if vault is used + if secretName := a.Spec.OCIConfig.SecretName; secretName != "" { volM = append(volM, corev1.VolumeMount{ Name: DefaultOCIPrivateKeyVolumeString, MountPath: DefaultVaultPrivateKeyRootPath, }) } + + // a.Spec.Log.Path path to mount for a custom log path, a volume is required + if rLogPath := a.Spec.Log.Path; rLogPath != "" { + vName := GetLogName(a) + volM = append(volM, corev1.VolumeMount{ + Name: vName, + MountPath: rLogPath, + }) + } + return volM } -// GetExporterDeploymentVolumes function retrieves volumes from api or provides default -func GetExporterDeploymentVolumes(api *apiv1.DatabaseObserver) []corev1.Volume { +// GetExporterDeploymentVolumes function retrieves volumes from a or provides default +func GetExporterDeploymentVolumes(a *api.DatabaseObserver) []corev1.Volume { vol := make([]corev1.Volume, 0) // config-volume Volume // if null, the exporter uses the default built-in config - if cVolumeSourceName := api.Spec.Exporter.ExporterConfig.Configmap.Name; cVolumeSourceName != "" { + if cVolumeSourceName := a.Spec.ExporterConfig.Configmap.Name; cVolumeSourceName != "" { - cVolumeSourceKey := api.Spec.Exporter.ExporterConfig.Configmap.Key + cVolumeSourceKey := a.Spec.ExporterConfig.Configmap.Key cMSource := &corev1.ConfigMapVolumeSource{ LocalObjectReference: corev1.LocalObjectReference{ Name: cVolumeSourceName, @@ -96,9 +239,9 @@ func GetExporterDeploymentVolumes(api *apiv1.DatabaseObserver) []corev1.Volume { } // creds Volume - // api.Spec.Database.DBWallet.SecretName optional + // a.Spec.Database.DBWallet.SecretName optional // if null, consider the database NON-ADB and connect as such - if secretName := api.Spec.Database.DBWallet.SecretName; secretName != "" { + if secretName := a.Spec.Database.DBWallet.SecretName; secretName != "" { vol = append(vol, corev1.Volume{ Name: DefaultWalletVolumeString, @@ -111,8 +254,8 @@ func GetExporterDeploymentVolumes(api *apiv1.DatabaseObserver) []corev1.Volume { } // ocikey Volume - // api.Spec.Database.DBWallet.SecretName optional - if secretName := api.Spec.OCIConfig.SecretName; secretName != "" { + // a.Spec.Database.DBWallet.SecretName optional + if secretName := a.Spec.OCIConfig.SecretName; secretName != "" { OCIConfigSource := &corev1.SecretVolumeSource{ SecretName: secretName, @@ -127,39 +270,104 @@ func GetExporterDeploymentVolumes(api *apiv1.DatabaseObserver) []corev1.Volume { VolumeSource: corev1.VolumeSource{Secret: OCIConfigSource}, }) } + + // log-volume Volume + if rLogPath := a.Spec.Log.Path; rLogPath != "" { + vs := GetLogVolumeSource(a) + vName := GetLogName(a) + + vol = append(vol, corev1.Volume{ + Name: vName, + VolumeSource: vs, + }) + } + return vol } -// GetExporterSelector function retrieves labels from api or provides default -func GetExporterSelector(api *apiv1.DatabaseObserver) map[string]string { - var s = make(map[string]string) - if labels := api.Spec.Prometheus.Labels; labels != nil && len(labels) > 0 { - for k, v := range labels { - s[k] = v +// GetExporterConfig function retrieves config name for status +func GetExporterConfig(a *api.DatabaseObserver) string { + + configName := DefaultValue + if cmName := a.Spec.ExporterConfig.Configmap.Name; cmName != "" { + configName = cmName + } + + return configName +} + +func GetLogName(a *api.DatabaseObserver) string { + if name := a.Spec.Log.Volume.Name; name != "" { + return name + } + return DefaultLogVolumeString +} + +// GetLogVolumeSource function retrieves the source to help GetExporterDeploymentVolumes +func GetLogVolumeSource(a *api.DatabaseObserver) corev1.VolumeSource { + + vs := corev1.VolumeSource{} + rLogVolumeClaimName := a.Spec.Log.Volume.PersistentVolumeClaim.ClaimName + + // volume claims take precedence + if rLogVolumeClaimName != "" { + vs.PersistentVolumeClaim = &corev1.PersistentVolumeClaimVolumeSource{ + ClaimName: rLogVolumeClaimName, } - return s + return vs + + } else { + vs.EmptyDir = &corev1.EmptyDirVolumeSource{} + return vs + } +} +// AddEnv is a helper method that appends an Env Var value +func AddEnv(env []corev1.EnvVar, existing map[string]string, name string, v string) []corev1.EnvVar { + + // Evaluate if env already exists + if _, f := existing[name]; !f { + env = append(env, corev1.EnvVar{Name: name, Value: v}) } - return map[string]string{DefaultLabelKey: DefaultLabelPrefix + api.Name} + return env +} + +// AddEnvFrom is a helper method that appends an Env Var value source +func AddEnvFrom(env []corev1.EnvVar, existing map[string]string, name string, v *corev1.EnvVarSource) []corev1.EnvVar { + // Evaluate if env already exists + if _, f := existing[name]; !f { + env = append(env, corev1.EnvVar{Name: name, ValueFrom: v}) + } + return env } -// GetExporterEnvs function retrieves env from api or provides default -func GetExporterEnvs(api *apiv1.DatabaseObserver) []corev1.EnvVar { +// GetExporterEnvs function retrieves env from a or provides default +func GetExporterEnvs(a *api.DatabaseObserver) []corev1.EnvVar { optional := true - rDBPasswordKey := api.Spec.Database.DBPassword.Key - rDBPasswordName := api.Spec.Database.DBPassword.SecretName - rDBConnectStrKey := api.Spec.Database.DBConnectionString.Key - rDBConnectStrName := api.Spec.Database.DBConnectionString.SecretName - rDBVaultSecretName := api.Spec.Database.DBPassword.VaultSecretName - rDBVaultOCID := api.Spec.Database.DBPassword.VaultOCID - rDBUserSKey := api.Spec.Database.DBUser.Key - rDBUserSName := api.Spec.Database.DBUser.SecretName - rOCIConfigCMName := api.Spec.OCIConfig.ConfigMapName + rDBPasswordKey := a.Spec.Database.DBPassword.Key + rDBPasswordName := a.Spec.Database.DBPassword.SecretName + rDBConnectStrKey := a.Spec.Database.DBConnectionString.Key + rDBConnectStrName := a.Spec.Database.DBConnectionString.SecretName + rDBVaultSecretName := a.Spec.Database.DBPassword.VaultSecretName + rDBVaultOCID := a.Spec.Database.DBPassword.VaultOCID + rDBUserSKey := a.Spec.Database.DBUser.Key + rDBUserSName := a.Spec.Database.DBUser.SecretName + rOCIConfigCMName := a.Spec.OCIConfig.ConfigMapName + rLogPath := a.Spec.Log.Path + rLogFilename := a.Spec.Log.Filename + rCustomEnvs := a.Spec.Exporter.Deployment.ExporterEnvs var env = make([]corev1.EnvVar, 0) + // add CustomEnvs + if rCustomEnvs != nil { + for k, v := range rCustomEnvs { + env = append(env, corev1.EnvVar{Name: k, Value: v}) + } + } + // DB_USERNAME environment variable if rDBUserSKey == "" { // overwrite rDBUserSKey = DefaultDbUserKey @@ -170,7 +378,7 @@ func GetExporterEnvs(api *apiv1.DatabaseObserver) []corev1.EnvVar { LocalObjectReference: corev1.LocalObjectReference{Name: rDBUserSName}, Optional: &optional, }} - env = append(env, corev1.EnvVar{Name: EnvVarDataSourceUser, ValueFrom: envUser}) + env = AddEnvFrom(env, rCustomEnvs, EnvVarDataSourceUser, envUser) // DB_CONNECT_STRING environment variable if rDBConnectStrKey == "" { @@ -182,15 +390,15 @@ func GetExporterEnvs(api *apiv1.DatabaseObserver) []corev1.EnvVar { LocalObjectReference: corev1.LocalObjectReference{Name: rDBConnectStrName}, Optional: &optional, }} - env = append(env, corev1.EnvVar{Name: EnvVarDataSourceConnectString, ValueFrom: envConnectStr}) + env = AddEnvFrom(env, rCustomEnvs, EnvVarDataSourceConnectString, envConnectStr) // DB_PASSWORD environment variable // if useVault, add environment variables for Vault ID and Vault Secret Name useVault := rDBVaultSecretName != "" && rDBVaultOCID != "" if useVault { - env = append(env, corev1.EnvVar{Name: EnvVarDataSourcePwdVaultSecretName, Value: rDBVaultSecretName}) - env = append(env, corev1.EnvVar{Name: EnvVarDataSourcePwdVaultId, Value: rDBVaultOCID}) + env = AddEnv(env, rCustomEnvs, EnvVarDataSourcePwdVaultSecretName, rDBVaultSecretName) + env = AddEnv(env, rCustomEnvs, EnvVarDataSourcePwdVaultId, rDBVaultOCID) // Configuring the configProvider prefixed with vault_ // https://github.com/oracle/oracle-db-appdev-monitoring/blob/main/vault/vault.go @@ -222,12 +430,11 @@ func GetExporterEnvs(api *apiv1.DatabaseObserver) []corev1.EnvVar { Optional: &optional, }, } - - env = append(env, corev1.EnvVar{Name: EnvVarVaultFingerprint, ValueFrom: configSourceFingerprintValue}) - env = append(env, corev1.EnvVar{Name: EnvVarVaultUserOCID, ValueFrom: configSourceUserValue}) - env = append(env, corev1.EnvVar{Name: EnvVarVaultTenancyOCID, ValueFrom: configSourceTenancyValue}) - env = append(env, corev1.EnvVar{Name: EnvVarVaultRegion, ValueFrom: configSourceRegionValue}) - env = append(env, corev1.EnvVar{Name: EnvVarVaultPrivateKeyPath, Value: DefaultVaultPrivateKeyAbsolutePath}) + env = AddEnvFrom(env, rCustomEnvs, EnvVarVaultFingerprint, configSourceFingerprintValue) + env = AddEnvFrom(env, rCustomEnvs, EnvVarVaultUserOCID, configSourceUserValue) + env = AddEnvFrom(env, rCustomEnvs, EnvVarVaultTenancyOCID, configSourceTenancyValue) + env = AddEnvFrom(env, rCustomEnvs, EnvVarVaultRegion, configSourceRegionValue) + env = AddEnv(env, rCustomEnvs, EnvVarVaultPrivateKeyPath, DefaultVaultPrivateKeyAbsolutePath) } else { @@ -241,162 +448,46 @@ func GetExporterEnvs(api *apiv1.DatabaseObserver) []corev1.EnvVar { Optional: &optional, }} - env = append(env, corev1.EnvVar{Name: EnvVarDataSourcePassword, ValueFrom: dbPassword}) + env = AddEnvFrom(env, rCustomEnvs, EnvVarDataSourcePassword, dbPassword) } // CUSTOM_METRICS environment variable - if customMetricsName := api.Spec.Exporter.ExporterConfig.Configmap.Name; customMetricsName != "" { + if customMetricsName := a.Spec.ExporterConfig.Configmap.Name; customMetricsName != "" { customMetrics := DefaultExporterConfigmapAbsolutePath - env = append(env, corev1.EnvVar{Name: EnvVarCustomConfigmap, Value: customMetrics}) + + env = AddEnv(env, rCustomEnvs, EnvVarCustomConfigmap, customMetrics) + } + + env = AddEnv(env, rCustomEnvs, EnvVarOracleHome, DefaultOracleHome) + env = AddEnv(env, rCustomEnvs, EnvVarTNSAdmin, DefaultOracleTNSAdmin) + + // LOG_DESTINATION environment variable + if rLogPath != "" { + if rLogFilename == "" { + rLogFilename = DefaultLogFilename + } + d := filepath.Join(rLogPath, rLogFilename) + env = AddEnv(env, rCustomEnvs, EnvVarDataSourceLogDestination, d) } - env = append(env, corev1.EnvVar{Name: EnvVarOracleHome, Value: DefaultOracleHome}) - env = append(env, corev1.EnvVar{Name: EnvVarTNSAdmin, Value: DefaultOracleTNSAdmin}) return env } -// GetExporterReplicas function retrieves replicaCount from api or provides default -func GetExporterReplicas(api *apiv1.DatabaseObserver) int32 { - if rc := api.Spec.Replicas; rc != 0 { +// GetExporterReplicas function retrieves replicaCount from a or provides default +func GetExporterReplicas(a *api.DatabaseObserver) int32 { + if rc := a.Spec.Replicas; rc != 0 { return rc } return int32(DefaultReplicaCount) } -// GetExporterImage function retrieves image from api or provides default -func GetExporterImage(api *apiv1.DatabaseObserver) string { - if img := api.Spec.Exporter.ExporterImage; img != "" { +// GetExporterImage function retrieves image from a or provides default +func GetExporterImage(a *api.DatabaseObserver) string { + if img := a.Spec.Exporter.Deployment.ExporterImage; img != "" { return img } - return DefaultExporterImage - -} - -func IsUpdateRequiredForContainerImage(desired *appsv1.Deployment, found *appsv1.Deployment) bool { - foundImage := found.Spec.Template.Spec.Containers[0].Image - desiredImage := desired.Spec.Template.Spec.Containers[0].Image - - return foundImage != desiredImage -} - -func IsUpdateRequiredForEnvironmentVars(desired *appsv1.Deployment, found *appsv1.Deployment) bool { - var updateEnvsRequired bool - desiredEnvValues := make(map[string]string) - - foundEnvs := found.Spec.Template.Spec.Containers[0].Env - desiredEnvs := desired.Spec.Template.Spec.Containers[0].Env - if len(foundEnvs) != len(desiredEnvs) { - updateEnvsRequired = true - } else { - for _, v := range desiredEnvs { - - if v.Name == EnvVarDataSourceUser || - v.Name == EnvVarDataSourceConnectString || - v.Name == EnvVarDataSourcePassword { - - ref := *(*v.ValueFrom).SecretKeyRef - desiredEnvValues[v.Name] = ref.Key + "-" + ref.Name - - } else if v.Name == EnvVarVaultFingerprint || - v.Name == EnvVarVaultRegion || - v.Name == EnvVarVaultTenancyOCID || - v.Name == EnvVarVaultUserOCID { - - ref := *(*v.ValueFrom).ConfigMapKeyRef - desiredEnvValues[v.Name] = ref.Key + "-" + ref.Name - - } else if v.Name == EnvVarDataSourcePwdVaultId || - v.Name == EnvVarDataSourcePwdVaultSecretName || - v.Name == EnvVarCustomConfigmap { - - desiredEnvValues[v.Name] = v.Value - } - } - - for _, v := range foundEnvs { - var foundValue string - - if v.Name == EnvVarDataSourceUser || - v.Name == EnvVarDataSourceConnectString || - v.Name == EnvVarDataSourcePassword { - - ref := *(*v.ValueFrom).SecretKeyRef - foundValue = ref.Key + "-" + ref.Name - - } else if v.Name == EnvVarVaultFingerprint || - v.Name == EnvVarVaultRegion || - v.Name == EnvVarVaultTenancyOCID || - v.Name == EnvVarVaultUserOCID { - - ref := *(*v.ValueFrom).ConfigMapKeyRef - foundValue = ref.Key + "-" + ref.Name - - } else if v.Name == EnvVarDataSourcePwdVaultId || - v.Name == EnvVarDataSourcePwdVaultSecretName || - v.Name == EnvVarCustomConfigmap { - - foundValue = v.Value - } - - if desiredEnvValues[v.Name] != foundValue { - updateEnvsRequired = true - } - } - } - return updateEnvsRequired -} - -func IsUpdateRequiredForVolumes(desired *appsv1.Deployment, found *appsv1.Deployment) bool { - var updateVolumesRequired bool - var foundConfigmap, desiredConfigmap string - var foundWalletSecret, desiredWalletSecret string - var foundOCIConfig, desiredOCIConfig string - desiredVolumes := desired.Spec.Template.Spec.Volumes - foundVolumes := found.Spec.Template.Spec.Volumes - - if len(desiredVolumes) != len(foundVolumes) { - updateVolumesRequired = true - } else { - for _, v := range desiredVolumes { - if v.Name == DefaultConfigVolumeString { - desiredConfigmap = v.ConfigMap.Name - for _, key := range v.ConfigMap.Items { - desiredConfigmap += key.Key - } - } else if v.Name == DefaultWalletVolumeString { - desiredWalletSecret = v.VolumeSource.Secret.SecretName - - } else if v.Name == DefaultOCIPrivateKeyVolumeString { - desiredOCIConfig = v.VolumeSource.Secret.SecretName - } - } - - for _, v := range foundVolumes { - if v.Name == DefaultConfigVolumeString { - foundConfigmap = v.ConfigMap.Name - for _, key := range v.ConfigMap.Items { - foundConfigmap += key.Key - } - } else if v.Name == DefaultWalletVolumeString { - foundWalletSecret = v.VolumeSource.Secret.SecretName - - } else if v.Name == DefaultOCIPrivateKeyVolumeString { - foundOCIConfig = v.VolumeSource.Secret.SecretName - } - } - } - - return updateVolumesRequired || - desiredConfigmap != foundConfigmap || - desiredWalletSecret != foundWalletSecret || - desiredOCIConfig != foundOCIConfig -} - -func IsUpdateRequiredForReplicas(desired *appsv1.Deployment, found *appsv1.Deployment) bool { - foundReplicas := *found.Spec.Replicas - desiredReplicas := *desired.Spec.Replicas + return DefaultExporterImage - return desiredReplicas != foundReplicas } diff --git a/commons/oci/containerdatabase.go b/commons/oci/containerdatabase.go index a5313d41..9391d6f8 100644 --- a/commons/oci/containerdatabase.go +++ b/commons/oci/containerdatabase.go @@ -44,13 +44,13 @@ import ( "github.com/oracle/oci-go-sdk/v65/common" "github.com/oracle/oci-go-sdk/v65/database" - dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + dbv4 "github.com/oracle/oracle-database-operator/apis/database/v4" ) /******************************** * Autonomous Container Database *******************************/ -func (d *databaseService) CreateAutonomousContainerDatabase(acd *dbv1alpha1.AutonomousContainerDatabase) (database.CreateAutonomousContainerDatabaseResponse, error) { +func (d *DatabaseService) CreateAutonomousContainerDatabase(acd *dbv4.AutonomousContainerDatabase) (database.CreateAutonomousContainerDatabaseResponse, error) { createAutonomousContainerDatabaseRequest := database.CreateAutonomousContainerDatabaseRequest{ CreateAutonomousContainerDatabaseDetails: database.CreateAutonomousContainerDatabaseDetails{ CompartmentId: acd.Spec.CompartmentOCID, @@ -63,7 +63,7 @@ func (d *databaseService) CreateAutonomousContainerDatabase(acd *dbv1alpha1.Auto return d.dbClient.CreateAutonomousContainerDatabase(context.TODO(), createAutonomousContainerDatabaseRequest) } -func (d *databaseService) GetAutonomousContainerDatabase(acdOCID string) (database.GetAutonomousContainerDatabaseResponse, error) { +func (d *DatabaseService) GetAutonomousContainerDatabase(acdOCID string) (database.GetAutonomousContainerDatabaseResponse, error) { getAutonomousContainerDatabaseRequest := database.GetAutonomousContainerDatabaseRequest{ AutonomousContainerDatabaseId: common.String(acdOCID), } @@ -71,7 +71,7 @@ func (d *databaseService) GetAutonomousContainerDatabase(acdOCID string) (databa return d.dbClient.GetAutonomousContainerDatabase(context.TODO(), getAutonomousContainerDatabaseRequest) } -func (d *databaseService) UpdateAutonomousContainerDatabase(acdOCID string, difACD *dbv1alpha1.AutonomousContainerDatabase) (database.UpdateAutonomousContainerDatabaseResponse, error) { +func (d *DatabaseService) UpdateAutonomousContainerDatabase(acdOCID string, difACD *dbv4.AutonomousContainerDatabase) (database.UpdateAutonomousContainerDatabaseResponse, error) { updateAutonomousContainerDatabaseRequest := database.UpdateAutonomousContainerDatabaseRequest{ AutonomousContainerDatabaseId: common.String(acdOCID), UpdateAutonomousContainerDatabaseDetails: database.UpdateAutonomousContainerDatabaseDetails{ @@ -84,7 +84,7 @@ func (d *databaseService) UpdateAutonomousContainerDatabase(acdOCID string, difA return d.dbClient.UpdateAutonomousContainerDatabase(context.TODO(), updateAutonomousContainerDatabaseRequest) } -func (d *databaseService) RestartAutonomousContainerDatabase(acdOCID string) (database.RestartAutonomousContainerDatabaseResponse, error) { +func (d *DatabaseService) RestartAutonomousContainerDatabase(acdOCID string) (database.RestartAutonomousContainerDatabaseResponse, error) { restartRequest := database.RestartAutonomousContainerDatabaseRequest{ AutonomousContainerDatabaseId: common.String(acdOCID), } @@ -92,7 +92,7 @@ func (d *databaseService) RestartAutonomousContainerDatabase(acdOCID string) (da return d.dbClient.RestartAutonomousContainerDatabase(context.TODO(), restartRequest) } -func (d *databaseService) TerminateAutonomousContainerDatabase(acdOCID string) (database.TerminateAutonomousContainerDatabaseResponse, error) { +func (d *DatabaseService) TerminateAutonomousContainerDatabase(acdOCID string) (database.TerminateAutonomousContainerDatabaseResponse, error) { terminateRequest := database.TerminateAutonomousContainerDatabaseRequest{ AutonomousContainerDatabaseId: common.String(acdOCID), } diff --git a/commons/oci/database.go b/commons/oci/database.go index 9c3cd4d8..e43afb56 100644 --- a/commons/oci/database.go +++ b/commons/oci/database.go @@ -47,38 +47,11 @@ import ( "github.com/oracle/oci-go-sdk/v65/database" "sigs.k8s.io/controller-runtime/pkg/client" - dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + dbv4 "github.com/oracle/oracle-database-operator/apis/database/v4" "github.com/oracle/oracle-database-operator/commons/k8s" ) -type DatabaseService interface { - CreateAutonomousDatabase(adb *dbv1alpha1.AutonomousDatabase) (database.CreateAutonomousDatabaseResponse, error) - GetAutonomousDatabase(adbOCID string) (database.GetAutonomousDatabaseResponse, error) - UpdateAutonomousDatabaseGeneralFields(adbOCID string, difADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) - UpdateAutonomousDatabaseDBWorkload(adbOCID string, difADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) - UpdateAutonomousDatabaseLicenseModel(adbOCID string, difADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) - UpdateAutonomousDatabaseAdminPassword(adbOCID string, difADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) - UpdateAutonomousDatabaseScalingFields(adbOCID string, difADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) - UpdateNetworkAccessMTLSRequired(adbOCID string) (resp database.UpdateAutonomousDatabaseResponse, err error) - UpdateNetworkAccessMTLS(adbOCID string, difADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) - UpdateNetworkAccessPublic(lastAccessType dbv1alpha1.NetworkAccessTypeEnum, adbOCID string) (resp database.UpdateAutonomousDatabaseResponse, err error) - UpdateNetworkAccess(adbOCID string, difADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) - StartAutonomousDatabase(adbOCID string) (database.StartAutonomousDatabaseResponse, error) - StopAutonomousDatabase(adbOCID string) (database.StopAutonomousDatabaseResponse, error) - DeleteAutonomousDatabase(adbOCID string) (database.DeleteAutonomousDatabaseResponse, error) - DownloadWallet(adb *dbv1alpha1.AutonomousDatabase) (database.GenerateAutonomousDatabaseWalletResponse, error) - RestoreAutonomousDatabase(adbOCID string, sdkTime common.SDKTime) (database.RestoreAutonomousDatabaseResponse, error) - ListAutonomousDatabaseBackups(adbOCID string) (database.ListAutonomousDatabaseBackupsResponse, error) - CreateAutonomousDatabaseBackup(adbBackup *dbv1alpha1.AutonomousDatabaseBackup, adbOCID string) (database.CreateAutonomousDatabaseBackupResponse, error) - GetAutonomousDatabaseBackup(backupOCID string) (database.GetAutonomousDatabaseBackupResponse, error) - CreateAutonomousContainerDatabase(acd *dbv1alpha1.AutonomousContainerDatabase) (database.CreateAutonomousContainerDatabaseResponse, error) - GetAutonomousContainerDatabase(acdOCID string) (database.GetAutonomousContainerDatabaseResponse, error) - UpdateAutonomousContainerDatabase(acdOCID string, difACD *dbv1alpha1.AutonomousContainerDatabase) (database.UpdateAutonomousContainerDatabaseResponse, error) - RestartAutonomousContainerDatabase(acdOCID string) (database.RestartAutonomousContainerDatabaseResponse, error) - TerminateAutonomousContainerDatabase(acdOCID string) (database.TerminateAutonomousContainerDatabaseResponse, error) -} - -type databaseService struct { +type DatabaseService struct { logger logr.Logger kubeClient client.Client dbClient database.DatabaseClient @@ -88,19 +61,19 @@ type databaseService struct { func NewDatabaseService( logger logr.Logger, kubeClient client.Client, - provider common.ConfigurationProvider) (DatabaseService, error) { + provider common.ConfigurationProvider) (databaseService DatabaseService, err error) { dbClient, err := database.NewDatabaseClientWithConfigurationProvider(provider) if err != nil { - return nil, err + return databaseService, err } vaultService, err := NewVaultService(logger, provider) if err != nil { - return nil, err + return databaseService, err } - return &databaseService{ + return DatabaseService{ logger: logger.WithName("dbService"), kubeClient: kubeClient, dbClient: dbClient, @@ -114,7 +87,7 @@ func NewDatabaseService( // ReadPassword reads the password from passwordSpec, and returns the pointer to the read password string. // The function returns a nil if nothing is read -func (d *databaseService) readPassword(namespace string, passwordSpec dbv1alpha1.PasswordSpec) (*string, error) { +func (d *DatabaseService) readPassword(namespace string, passwordSpec dbv4.PasswordSpec) (*string, error) { logger := d.logger.WithName("readPassword") if passwordSpec.K8sSecret.Name != nil { @@ -129,10 +102,10 @@ func (d *databaseService) readPassword(namespace string, passwordSpec dbv1alpha1 return common.String(password), nil } - if passwordSpec.OCISecret.OCID != nil { - logger.Info(fmt.Sprintf("Getting password from OCI Vault Secret OCID %s", *passwordSpec.OCISecret.OCID)) + if passwordSpec.OciSecret.Id != nil { + logger.Info(fmt.Sprintf("Getting password from OCI Vault Secret OCID %s", *passwordSpec.OciSecret.Id)) - password, err := d.vaultService.GetSecretValue(*passwordSpec.OCISecret.OCID) + password, err := d.vaultService.GetSecretValue(*passwordSpec.OciSecret.Id) if err != nil { return nil, err } @@ -142,14 +115,14 @@ func (d *databaseService) readPassword(namespace string, passwordSpec dbv1alpha1 return nil, nil } -func (d *databaseService) readACD_OCID(acd *dbv1alpha1.ACDSpec, namespace string) (*string, error) { - if acd.OCIACD.OCID != nil { - return acd.OCIACD.OCID, nil +func (d *DatabaseService) readACD_OCID(acd *dbv4.AcdSpec, namespace string) (*string, error) { + if acd.OciAcd.Id != nil { + return acd.OciAcd.Id, nil } - if acd.K8sACD.Name != nil { - fetchedACD := &dbv1alpha1.AutonomousContainerDatabase{} - if err := k8s.FetchResource(d.kubeClient, namespace, *acd.K8sACD.Name, fetchedACD); err != nil { + if acd.K8sAcd.Name != nil { + fetchedACD := &dbv4.AutonomousContainerDatabase{} + if err := k8s.FetchResource(d.kubeClient, namespace, *acd.K8sAcd.Name, fetchedACD); err != nil { return nil, err } @@ -160,7 +133,7 @@ func (d *databaseService) readACD_OCID(acd *dbv1alpha1.ACDSpec, namespace string } // CreateAutonomousDatabase sends a request to OCI to provision a database and returns the AutonomousDatabase OCID. -func (d *databaseService) CreateAutonomousDatabase(adb *dbv1alpha1.AutonomousDatabase) (resp database.CreateAutonomousDatabaseResponse, err error) { +func (d *DatabaseService) CreateAutonomousDatabase(adb *dbv4.AutonomousDatabase) (resp database.CreateAutonomousDatabaseResponse, err error) { adminPassword, err := d.readPassword(adb.Namespace, adb.Spec.Details.AdminPassword) if err != nil { return resp, err @@ -172,9 +145,12 @@ func (d *databaseService) CreateAutonomousDatabase(adb *dbv1alpha1.AutonomousDat } createAutonomousDatabaseDetails := database.CreateAutonomousDatabaseDetails{ - CompartmentId: adb.Spec.Details.CompartmentOCID, + CompartmentId: adb.Spec.Details.CompartmentId, DbName: adb.Spec.Details.DbName, - CpuCoreCount: adb.Spec.Details.CPUCoreCount, + CpuCoreCount: adb.Spec.Details.CpuCoreCount, + ComputeModel: database.CreateAutonomousDatabaseBaseComputeModelEnum(adb.Spec.Details.ComputeModel), + ComputeCount: adb.Spec.Details.ComputeCount, + OcpuCount: adb.Spec.Details.OcpuCount, DataStorageSizeInTBs: adb.Spec.Details.DataStorageSizeInTBs, AdminPassword: adminPassword, DisplayName: adb.Spec.Details.DisplayName, @@ -182,21 +158,26 @@ func (d *databaseService) CreateAutonomousDatabase(adb *dbv1alpha1.AutonomousDat IsDedicated: adb.Spec.Details.IsDedicated, AutonomousContainerDatabaseId: acdOCID, DbVersion: adb.Spec.Details.DbVersion, - DbWorkload: database.CreateAutonomousDatabaseBaseDbWorkloadEnum( - adb.Spec.Details.DbWorkload), - LicenseModel: database.CreateAutonomousDatabaseBaseLicenseModelEnum(adb.Spec.Details.LicenseModel), - IsAccessControlEnabled: adb.Spec.Details.NetworkAccess.IsAccessControlEnabled, - WhitelistedIps: adb.Spec.Details.NetworkAccess.AccessControlList, - IsMtlsConnectionRequired: adb.Spec.Details.NetworkAccess.IsMTLSConnectionRequired, - SubnetId: adb.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID, - NsgIds: adb.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs, - PrivateEndpointLabel: adb.Spec.Details.NetworkAccess.PrivateEndpoint.HostnamePrefix, + DbWorkload: database.CreateAutonomousDatabaseBaseDbWorkloadEnum(adb.Spec.Details.DbWorkload), + LicenseModel: database.CreateAutonomousDatabaseBaseLicenseModelEnum(adb.Spec.Details.LicenseModel), + IsFreeTier: adb.Spec.Details.IsFreeTier, + IsAccessControlEnabled: adb.Spec.Details.IsAccessControlEnabled, + WhitelistedIps: adb.Spec.Details.WhitelistedIps, + IsMtlsConnectionRequired: adb.Spec.Details.IsMtlsConnectionRequired, + SubnetId: adb.Spec.Details.SubnetId, + NsgIds: adb.Spec.Details.NsgIds, + PrivateEndpointLabel: adb.Spec.Details.PrivateEndpointLabel, FreeformTags: adb.Spec.Details.FreeformTags, } + retryPolicy := common.DefaultRetryPolicy() + createAutonomousDatabaseRequest := database.CreateAutonomousDatabaseRequest{ CreateAutonomousDatabaseDetails: createAutonomousDatabaseDetails, + RequestMetadata: common.RequestMetadata{ + RetryPolicy: &retryPolicy, + }, } resp, err = d.dbClient.CreateAutonomousDatabase(context.TODO(), createAutonomousDatabaseRequest) @@ -207,166 +188,116 @@ func (d *databaseService) CreateAutonomousDatabase(adb *dbv1alpha1.AutonomousDat return resp, nil } -func (d *databaseService) GetAutonomousDatabase(adbOCID string) (database.GetAutonomousDatabaseResponse, error) { - getAutonomousDatabaseRequest := database.GetAutonomousDatabaseRequest{ - AutonomousDatabaseId: common.String(adbOCID), - } +func (d *DatabaseService) GetAutonomousDatabase(adbOCID string) (database.GetAutonomousDatabaseResponse, error) { + retryPolicy := common.DefaultRetryPolicy() - return d.dbClient.GetAutonomousDatabase(context.TODO(), getAutonomousDatabaseRequest) -} - -func (d *databaseService) UpdateAutonomousDatabaseGeneralFields(adbOCID string, difADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { - updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ - AutonomousDatabaseId: common.String(adbOCID), - UpdateAutonomousDatabaseDetails: database.UpdateAutonomousDatabaseDetails{ - DisplayName: difADB.Spec.Details.DisplayName, - DbName: difADB.Spec.Details.DbName, - DbVersion: difADB.Spec.Details.DbVersion, - FreeformTags: difADB.Spec.Details.FreeformTags, - }, - } - return d.dbClient.UpdateAutonomousDatabase(context.TODO(), updateAutonomousDatabaseRequest) -} - -func (d *databaseService) UpdateAutonomousDatabaseDBWorkload(adbOCID string, difADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { - updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ + getAutonomousDatabaseRequest := database.GetAutonomousDatabaseRequest{ AutonomousDatabaseId: common.String(adbOCID), - UpdateAutonomousDatabaseDetails: database.UpdateAutonomousDatabaseDetails{ - DbWorkload: database.UpdateAutonomousDatabaseDetailsDbWorkloadEnum(difADB.Spec.Details.DbWorkload), + RequestMetadata: common.RequestMetadata{ + RetryPolicy: &retryPolicy, }, } - return d.dbClient.UpdateAutonomousDatabase(context.TODO(), updateAutonomousDatabaseRequest) -} -func (d *databaseService) UpdateAutonomousDatabaseLicenseModel(adbOCID string, difADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { - updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ - AutonomousDatabaseId: common.String(adbOCID), - UpdateAutonomousDatabaseDetails: database.UpdateAutonomousDatabaseDetails{ - LicenseModel: database.UpdateAutonomousDatabaseDetailsLicenseModelEnum(difADB.Spec.Details.LicenseModel), - }, - } - return d.dbClient.UpdateAutonomousDatabase(context.TODO(), updateAutonomousDatabaseRequest) + return d.dbClient.GetAutonomousDatabase(context.TODO(), getAutonomousDatabaseRequest) } -func (d *databaseService) UpdateAutonomousDatabaseAdminPassword(adbOCID string, difADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { - adminPassword, err := d.readPassword(difADB.Namespace, difADB.Spec.Details.AdminPassword) +func (d *DatabaseService) UpdateAutonomousDatabase(adbOCID string, adb *dbv4.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { + // Retrieve admin password + adminPassword, err := d.readPassword(adb.Namespace, adb.Spec.Details.AdminPassword) if err != nil { return resp, err } - updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ - AutonomousDatabaseId: common.String(adbOCID), - UpdateAutonomousDatabaseDetails: database.UpdateAutonomousDatabaseDetails{ - AdminPassword: adminPassword, - }, - } - return d.dbClient.UpdateAutonomousDatabase(context.TODO(), updateAutonomousDatabaseRequest) -} + retryPolicy := common.DefaultRetryPolicy() -func (d *databaseService) UpdateAutonomousDatabaseScalingFields(adbOCID string, difADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ AutonomousDatabaseId: common.String(adbOCID), UpdateAutonomousDatabaseDetails: database.UpdateAutonomousDatabaseDetails{ - DataStorageSizeInTBs: difADB.Spec.Details.DataStorageSizeInTBs, - CpuCoreCount: difADB.Spec.Details.CPUCoreCount, - IsAutoScalingEnabled: difADB.Spec.Details.IsAutoScalingEnabled, + DisplayName: adb.Spec.Details.DisplayName, + DbName: adb.Spec.Details.DbName, + DbVersion: adb.Spec.Details.DbVersion, + FreeformTags: adb.Spec.Details.FreeformTags, + DbWorkload: database.UpdateAutonomousDatabaseDetailsDbWorkloadEnum(adb.Spec.Details.DbWorkload), + LicenseModel: database.UpdateAutonomousDatabaseDetailsLicenseModelEnum(adb.Spec.Details.LicenseModel), + AdminPassword: adminPassword, + DataStorageSizeInTBs: adb.Spec.Details.DataStorageSizeInTBs, + CpuCoreCount: adb.Spec.Details.CpuCoreCount, + ComputeModel: database.UpdateAutonomousDatabaseDetailsComputeModelEnum(adb.Spec.Details.ComputeModel), + ComputeCount: adb.Spec.Details.ComputeCount, + OcpuCount: adb.Spec.Details.OcpuCount, + IsAutoScalingEnabled: adb.Spec.Details.IsAutoScalingEnabled, + IsFreeTier: adb.Spec.Details.IsFreeTier, + IsMtlsConnectionRequired: adb.Spec.Details.IsMtlsConnectionRequired, + IsAccessControlEnabled: adb.Spec.Details.IsAccessControlEnabled, + WhitelistedIps: adb.Spec.Details.WhitelistedIps, + SubnetId: adb.Spec.Details.SubnetId, + NsgIds: adb.Spec.Details.NsgIds, + PrivateEndpointLabel: adb.Spec.Details.PrivateEndpointLabel, }, - } - return d.dbClient.UpdateAutonomousDatabase(context.TODO(), updateAutonomousDatabaseRequest) -} - -func (d *databaseService) UpdateNetworkAccessMTLSRequired(adbOCID string) (resp database.UpdateAutonomousDatabaseResponse, err error) { - updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ - AutonomousDatabaseId: common.String(adbOCID), - UpdateAutonomousDatabaseDetails: database.UpdateAutonomousDatabaseDetails{ - IsMtlsConnectionRequired: common.Bool(true), - }, - } - return d.dbClient.UpdateAutonomousDatabase(context.TODO(), updateAutonomousDatabaseRequest) -} - -func (d *databaseService) UpdateNetworkAccessMTLS(adbOCID string, difADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { - updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ - AutonomousDatabaseId: common.String(adbOCID), - UpdateAutonomousDatabaseDetails: database.UpdateAutonomousDatabaseDetails{ - IsMtlsConnectionRequired: difADB.Spec.Details.NetworkAccess.IsMTLSConnectionRequired, + RequestMetadata: common.RequestMetadata{ + RetryPolicy: &retryPolicy, }, } return d.dbClient.UpdateAutonomousDatabase(context.TODO(), updateAutonomousDatabaseRequest) } -func (d *databaseService) UpdateNetworkAccessPublic( - lastAccessType dbv1alpha1.NetworkAccessTypeEnum, - adbOCID string) (resp database.UpdateAutonomousDatabaseResponse, err error) { - - updateAutonomousDatabaseDetails := database.UpdateAutonomousDatabaseDetails{} +func (d *DatabaseService) StartAutonomousDatabase(adbOCID string) (database.StartAutonomousDatabaseResponse, error) { + retryPolicy := common.DefaultRetryPolicy() - if lastAccessType == dbv1alpha1.NetworkAccessTypeRestricted { - updateAutonomousDatabaseDetails.WhitelistedIps = []string{""} - } else if lastAccessType == dbv1alpha1.NetworkAccessTypePrivate { - updateAutonomousDatabaseDetails.PrivateEndpointLabel = common.String("") - } - - updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ - AutonomousDatabaseId: common.String(adbOCID), - UpdateAutonomousDatabaseDetails: updateAutonomousDatabaseDetails, - } - - return d.dbClient.UpdateAutonomousDatabase(context.TODO(), updateAutonomousDatabaseRequest) -} - -func (d *databaseService) UpdateNetworkAccess(adbOCID string, difADB *dbv1alpha1.AutonomousDatabase) (resp database.UpdateAutonomousDatabaseResponse, err error) { - updateAutonomousDatabaseRequest := database.UpdateAutonomousDatabaseRequest{ - AutonomousDatabaseId: common.String(adbOCID), - UpdateAutonomousDatabaseDetails: database.UpdateAutonomousDatabaseDetails{ - IsAccessControlEnabled: difADB.Spec.Details.NetworkAccess.IsAccessControlEnabled, - WhitelistedIps: difADB.Spec.Details.NetworkAccess.AccessControlList, - SubnetId: difADB.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID, - NsgIds: difADB.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs, - PrivateEndpointLabel: difADB.Spec.Details.NetworkAccess.PrivateEndpoint.HostnamePrefix, - }, - } - - return d.dbClient.UpdateAutonomousDatabase(context.TODO(), updateAutonomousDatabaseRequest) -} - -func (d *databaseService) StartAutonomousDatabase(adbOCID string) (database.StartAutonomousDatabaseResponse, error) { startRequest := database.StartAutonomousDatabaseRequest{ AutonomousDatabaseId: common.String(adbOCID), + RequestMetadata: common.RequestMetadata{ + RetryPolicy: &retryPolicy, + }, } return d.dbClient.StartAutonomousDatabase(context.TODO(), startRequest) } -func (d *databaseService) StopAutonomousDatabase(adbOCID string) (database.StopAutonomousDatabaseResponse, error) { +func (d *DatabaseService) StopAutonomousDatabase(adbOCID string) (database.StopAutonomousDatabaseResponse, error) { + retryPolicy := common.DefaultRetryPolicy() + stopRequest := database.StopAutonomousDatabaseRequest{ AutonomousDatabaseId: common.String(adbOCID), + RequestMetadata: common.RequestMetadata{ + RetryPolicy: &retryPolicy, + }, } return d.dbClient.StopAutonomousDatabase(context.TODO(), stopRequest) } -func (d *databaseService) DeleteAutonomousDatabase(adbOCID string) (database.DeleteAutonomousDatabaseResponse, error) { +func (d *DatabaseService) DeleteAutonomousDatabase(adbOCID string) (database.DeleteAutonomousDatabaseResponse, error) { + retryPolicy := common.DefaultRetryPolicy() + deleteRequest := database.DeleteAutonomousDatabaseRequest{ AutonomousDatabaseId: common.String(adbOCID), + RequestMetadata: common.RequestMetadata{ + RetryPolicy: &retryPolicy, + }, } return d.dbClient.DeleteAutonomousDatabase(context.TODO(), deleteRequest) } -func (d *databaseService) DownloadWallet(adb *dbv1alpha1.AutonomousDatabase) (resp database.GenerateAutonomousDatabaseWalletResponse, err error) { +func (d *DatabaseService) DownloadWallet(adb *dbv4.AutonomousDatabase) (resp database.GenerateAutonomousDatabaseWalletResponse, err error) { // Prepare wallet password - walletPassword, err := d.readPassword(adb.Namespace, adb.Spec.Details.Wallet.Password) + walletPassword, err := d.readPassword(adb.Namespace, adb.Spec.Wallet.Password) if err != nil { return resp, err } + retryPolicy := common.DefaultRetryPolicy() + // Download a Wallet req := database.GenerateAutonomousDatabaseWalletRequest{ - AutonomousDatabaseId: adb.Spec.Details.AutonomousDatabaseOCID, + AutonomousDatabaseId: adb.Spec.Details.Id, GenerateAutonomousDatabaseWalletDetails: database.GenerateAutonomousDatabaseWalletDetails{ Password: walletPassword, }, + RequestMetadata: common.RequestMetadata{ + RetryPolicy: &retryPolicy, + }, } // Send the request using the service client @@ -382,12 +313,17 @@ func (d *databaseService) DownloadWallet(adb *dbv1alpha1.AutonomousDatabase) (re * Autonomous Database Restore *******************************/ -func (d *databaseService) RestoreAutonomousDatabase(adbOCID string, sdkTime common.SDKTime) (database.RestoreAutonomousDatabaseResponse, error) { +func (d *DatabaseService) RestoreAutonomousDatabase(adbOCID string, sdkTime common.SDKTime) (database.RestoreAutonomousDatabaseResponse, error) { + retryPolicy := common.DefaultRetryPolicy() + request := database.RestoreAutonomousDatabaseRequest{ AutonomousDatabaseId: common.String(adbOCID), RestoreAutonomousDatabaseDetails: database.RestoreAutonomousDatabaseDetails{ Timestamp: &sdkTime, }, + RequestMetadata: common.RequestMetadata{ + RetryPolicy: &retryPolicy, + }, } return d.dbClient.RestoreAutonomousDatabase(context.TODO(), request) } @@ -396,21 +332,31 @@ func (d *databaseService) RestoreAutonomousDatabase(adbOCID string, sdkTime comm * Autonomous Database Backup *******************************/ -func (d *databaseService) ListAutonomousDatabaseBackups(adbOCID string) (database.ListAutonomousDatabaseBackupsResponse, error) { +func (d *DatabaseService) ListAutonomousDatabaseBackups(adbOCID string) (database.ListAutonomousDatabaseBackupsResponse, error) { + retryPolicy := common.DefaultRetryPolicy() + listBackupRequest := database.ListAutonomousDatabaseBackupsRequest{ AutonomousDatabaseId: common.String(adbOCID), + RequestMetadata: common.RequestMetadata{ + RetryPolicy: &retryPolicy, + }, } return d.dbClient.ListAutonomousDatabaseBackups(context.TODO(), listBackupRequest) } -func (d *databaseService) CreateAutonomousDatabaseBackup(adbBackup *dbv1alpha1.AutonomousDatabaseBackup, adbOCID string) (database.CreateAutonomousDatabaseBackupResponse, error) { +func (d *DatabaseService) CreateAutonomousDatabaseBackup(adbBackup *dbv4.AutonomousDatabaseBackup, adbOCID string) (database.CreateAutonomousDatabaseBackupResponse, error) { + retryPolicy := common.DefaultRetryPolicy() + createBackupRequest := database.CreateAutonomousDatabaseBackupRequest{ CreateAutonomousDatabaseBackupDetails: database.CreateAutonomousDatabaseBackupDetails{ AutonomousDatabaseId: common.String(adbOCID), IsLongTermBackup: adbBackup.Spec.IsLongTermBackup, RetentionPeriodInDays: adbBackup.Spec.RetentionPeriodInDays, }, + RequestMetadata: common.RequestMetadata{ + RetryPolicy: &retryPolicy, + }, } // Use the spec.displayName as the displayName of the backup if is provided, @@ -424,10 +370,63 @@ func (d *databaseService) CreateAutonomousDatabaseBackup(adbBackup *dbv1alpha1.A return d.dbClient.CreateAutonomousDatabaseBackup(context.TODO(), createBackupRequest) } -func (d *databaseService) GetAutonomousDatabaseBackup(backupOCID string) (database.GetAutonomousDatabaseBackupResponse, error) { +func (d *DatabaseService) GetAutonomousDatabaseBackup(backupOCID string) (database.GetAutonomousDatabaseBackupResponse, error) { + retryPolicy := common.DefaultRetryPolicy() + getBackupRequest := database.GetAutonomousDatabaseBackupRequest{ AutonomousDatabaseBackupId: common.String(backupOCID), + RequestMetadata: common.RequestMetadata{ + RetryPolicy: &retryPolicy, + }, } return d.dbClient.GetAutonomousDatabaseBackup(context.TODO(), getBackupRequest) } + +func (d *DatabaseService) CreateAutonomousDatabaseClone(adb *dbv4.AutonomousDatabase) (resp database.CreateAutonomousDatabaseResponse, err error) { + adminPassword, err := d.readPassword(adb.Namespace, adb.Spec.Clone.AdminPassword) + if err != nil { + return resp, err + } + + acdOCID, err := d.readACD_OCID(&adb.Spec.Clone.AutonomousContainerDatabase, adb.Namespace) + if err != nil { + return resp, err + } + + retryPolicy := common.DefaultRetryPolicy() + request := database.CreateAutonomousDatabaseRequest{ + CreateAutonomousDatabaseDetails: database.CreateAutonomousDatabaseCloneDetails{ + CompartmentId: adb.Spec.Clone.CompartmentId, + SourceId: adb.Spec.Details.Id, + AutonomousContainerDatabaseId: acdOCID, + DisplayName: adb.Spec.Clone.DisplayName, + DbName: adb.Spec.Clone.DbName, + DbWorkload: database.CreateAutonomousDatabaseBaseDbWorkloadEnum(adb.Spec.Clone.DbWorkload), + LicenseModel: database.CreateAutonomousDatabaseBaseLicenseModelEnum(adb.Spec.Clone.LicenseModel), + DbVersion: adb.Spec.Clone.DbVersion, + DataStorageSizeInTBs: adb.Spec.Clone.DataStorageSizeInTBs, + CpuCoreCount: adb.Spec.Clone.CpuCoreCount, + ComputeModel: database.CreateAutonomousDatabaseBaseComputeModelEnum(adb.Spec.Clone.ComputeModel), + ComputeCount: adb.Spec.Clone.ComputeCount, + OcpuCount: adb.Spec.Clone.OcpuCount, + AdminPassword: adminPassword, + IsAutoScalingEnabled: adb.Spec.Clone.IsAutoScalingEnabled, + IsDedicated: adb.Spec.Clone.IsDedicated, + IsFreeTier: adb.Spec.Clone.IsFreeTier, + IsAccessControlEnabled: adb.Spec.Clone.IsAccessControlEnabled, + WhitelistedIps: adb.Spec.Clone.WhitelistedIps, + SubnetId: adb.Spec.Clone.SubnetId, + NsgIds: adb.Spec.Clone.NsgIds, + PrivateEndpointLabel: adb.Spec.Clone.PrivateEndpointLabel, + IsMtlsConnectionRequired: adb.Spec.Clone.IsMtlsConnectionRequired, + FreeformTags: adb.Spec.Clone.FreeformTags, + CloneType: adb.Spec.Clone.CloneType, + }, + RequestMetadata: common.RequestMetadata{ + RetryPolicy: &retryPolicy, + }, + } + + return d.dbClient.CreateAutonomousDatabase(context.TODO(), request) +} diff --git a/commons/oci/provider.go b/commons/oci/provider.go index 152f1efd..f466f226 100644 --- a/commons/oci/provider.go +++ b/commons/oci/provider.go @@ -60,13 +60,13 @@ const ( privatekeyKey = "privatekey" ) -type APIKeyAuth struct { +type ApiKeyAuth struct { ConfigMapName *string SecretName *string Namespace string } -func GetOCIProvider(kubeClient client.Client, authData APIKeyAuth) (common.ConfigurationProvider, error) { +func GetOciProvider(kubeClient client.Client, authData ApiKeyAuth) (common.ConfigurationProvider, error) { if authData.ConfigMapName != nil && authData.SecretName == nil { return getWorkloadIdentityProvider(kubeClient, authData) } else if authData.ConfigMapName != nil && authData.SecretName != nil { @@ -84,7 +84,7 @@ func GetOCIProvider(kubeClient client.Client, authData APIKeyAuth) (common.Confi } } -func getWorkloadIdentityProvider(kubeClient client.Client, authData APIKeyAuth) (common.ConfigurationProvider, error) { +func getWorkloadIdentityProvider(kubeClient client.Client, authData ApiKeyAuth) (common.ConfigurationProvider, error) { ociConfigMap, err := k8s.FetchConfigMap(kubeClient, authData.Namespace, *authData.ConfigMapName) if err != nil { return nil, err @@ -108,7 +108,7 @@ func getWorkloadIdentityProvider(kubeClient client.Client, authData APIKeyAuth) return auth.OkeWorkloadIdentityConfigurationProvider() } -func getProviderWithAPIKey(kubeClient client.Client, authData APIKeyAuth) (common.ConfigurationProvider, error) { +func getProviderWithAPIKey(kubeClient client.Client, authData ApiKeyAuth) (common.ConfigurationProvider, error) { var region, fingerprint, user, tenancy, passphrase, privatekeyValue string // Prepare ConfigMap diff --git a/commons/sharding/catalog.go b/commons/sharding/catalog.go index 58f07490..646c89b8 100644 --- a/commons/sharding/catalog.go +++ b/commons/sharding/catalog.go @@ -44,7 +44,7 @@ import ( "strconv" "github.com/go-logr/logr" - databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + databasev4 "github.com/oracle/oracle-database-operator/apis/database/v4" appsv1 "k8s.io/api/apps/v1" corev1 "k8s.io/api/core/v1" "k8s.io/apimachinery/pkg/api/resource" @@ -53,7 +53,7 @@ import ( "sigs.k8s.io/controller-runtime/pkg/client" ) -func buildLabelsForCatalog(instance *databasev1alpha1.ShardingDatabase, label string) map[string]string { +func buildLabelsForCatalog(instance *databasev4.ShardingDatabase, label string, catalogName string) map[string]string { return map[string]string{ "app": "OracleSharding", "type": "Catalog", @@ -61,7 +61,7 @@ func buildLabelsForCatalog(instance *databasev1alpha1.ShardingDatabase, label st } } -func getLabelForCatalog(instance *databasev1alpha1.ShardingDatabase) string { +func getLabelForCatalog(instance *databasev4.ShardingDatabase) string { // if len(OraCatalogSpex.Label) !=0 { // return OraCatalogSpex.Label @@ -70,7 +70,7 @@ func getLabelForCatalog(instance *databasev1alpha1.ShardingDatabase) string { return instance.Name } -func BuildStatefulSetForCatalog(instance *databasev1alpha1.ShardingDatabase, OraCatalogSpex databasev1alpha1.CatalogSpec) *appsv1.StatefulSet { +func BuildStatefulSetForCatalog(instance *databasev4.ShardingDatabase, OraCatalogSpex databasev4.CatalogSpec) *appsv1.StatefulSet { sfset := &appsv1.StatefulSet{ TypeMeta: buildTypeMetaForCatalog(), ObjectMeta: builObjectMetaForCatalog(instance, OraCatalogSpex), @@ -91,29 +91,29 @@ func buildTypeMetaForCatalog() metav1.TypeMeta { } // Function to build ObjectMeta -func builObjectMetaForCatalog(instance *databasev1alpha1.ShardingDatabase, OraCatalogSpex databasev1alpha1.CatalogSpec) metav1.ObjectMeta { +func builObjectMetaForCatalog(instance *databasev4.ShardingDatabase, OraCatalogSpex databasev4.CatalogSpec) metav1.ObjectMeta { // building objectMeta objmeta := metav1.ObjectMeta{ Name: OraCatalogSpex.Name, - Namespace: instance.Spec.Namespace, + Namespace: instance.Namespace, OwnerReferences: getOwnerRef(instance), - Labels: buildLabelsForCatalog(instance, "sharding"), + Labels: buildLabelsForCatalog(instance, "sharding", OraCatalogSpex.Name), } return objmeta } // Function to build Stateful Specs -func buildStatefulSpecForCatalog(instance *databasev1alpha1.ShardingDatabase, OraCatalogSpex databasev1alpha1.CatalogSpec) *appsv1.StatefulSetSpec { +func buildStatefulSpecForCatalog(instance *databasev4.ShardingDatabase, OraCatalogSpex databasev4.CatalogSpec) *appsv1.StatefulSetSpec { // building Stateful set Specs sfsetspec := &appsv1.StatefulSetSpec{ ServiceName: OraCatalogSpex.Name, Selector: &metav1.LabelSelector{ - MatchLabels: buildLabelsForCatalog(instance, "sharding"), + MatchLabels: buildLabelsForCatalog(instance, "sharding", OraCatalogSpex.Name), }, Template: corev1.PodTemplateSpec{ ObjectMeta: metav1.ObjectMeta{ - Labels: buildLabelsForCatalog(instance, "sharding"), + Labels: buildLabelsForCatalog(instance, "sharding", OraCatalogSpex.Name), }, Spec: *buildPodSpecForCatalog(instance, OraCatalogSpex), }, @@ -131,7 +131,7 @@ func buildStatefulSpecForCatalog(instance *databasev1alpha1.ShardingDatabase, Or // Function to build PodSpec -func buildPodSpecForCatalog(instance *databasev1alpha1.ShardingDatabase, OraCatalogSpex databasev1alpha1.CatalogSpec) *corev1.PodSpec { +func buildPodSpecForCatalog(instance *databasev4.ShardingDatabase, OraCatalogSpex databasev4.CatalogSpec) *corev1.PodSpec { user := oraRunAsUser group := oraFsGroup @@ -166,7 +166,7 @@ func buildPodSpecForCatalog(instance *databasev1alpha1.ShardingDatabase, OraCata } // Function to build Volume Spec -func buildVolumeSpecForCatalog(instance *databasev1alpha1.ShardingDatabase, OraCatalogSpex databasev1alpha1.CatalogSpec) []corev1.Volume { +func buildVolumeSpecForCatalog(instance *databasev4.ShardingDatabase, OraCatalogSpex databasev4.CatalogSpec) []corev1.Volume { var result []corev1.Volume result = []corev1.Volume{ { @@ -207,7 +207,7 @@ func buildVolumeSpecForCatalog(instance *databasev1alpha1.ShardingDatabase, OraC } // Function to build the container Specification -func buildContainerSpecForCatalog(instance *databasev1alpha1.ShardingDatabase, OraCatalogSpex databasev1alpha1.CatalogSpec) []corev1.Container { +func buildContainerSpecForCatalog(instance *databasev4.ShardingDatabase, OraCatalogSpex databasev4.CatalogSpec) []corev1.Container { // building Continer spec var result []corev1.Container containerSpec := corev1.Container{ @@ -284,7 +284,7 @@ func buildContainerSpecForCatalog(instance *databasev1alpha1.ShardingDatabase, O } // Function to build the init Container Spec -func buildInitContainerSpecForCatalog(instance *databasev1alpha1.ShardingDatabase, OraCatalogSpex databasev1alpha1.CatalogSpec) []corev1.Container { +func buildInitContainerSpecForCatalog(instance *databasev4.ShardingDatabase, OraCatalogSpex databasev4.CatalogSpec) []corev1.Container { var result []corev1.Container // building the init Container Spec privFlag := true @@ -320,7 +320,7 @@ func buildInitContainerSpecForCatalog(instance *databasev1alpha1.ShardingDatabas return result } -func buildVolumeMountSpecForCatalog(instance *databasev1alpha1.ShardingDatabase, OraCatalogSpex databasev1alpha1.CatalogSpec) []corev1.VolumeMount { +func buildVolumeMountSpecForCatalog(instance *databasev4.ShardingDatabase, OraCatalogSpex databasev4.CatalogSpec) []corev1.VolumeMount { var result []corev1.VolumeMount result = append(result, corev1.VolumeMount{Name: OraCatalogSpex.Name + "secretmap-vol3", MountPath: oraSecretMount, ReadOnly: true}) result = append(result, corev1.VolumeMount{Name: OraCatalogSpex.Name + "-oradata-vol4", MountPath: oraDataMount}) @@ -345,7 +345,7 @@ func buildVolumeMountSpecForCatalog(instance *databasev1alpha1.ShardingDatabase, return result } -func volumeClaimTemplatesForCatalog(instance *databasev1alpha1.ShardingDatabase, OraCatalogSpex databasev1alpha1.CatalogSpec) []corev1.PersistentVolumeClaim { +func volumeClaimTemplatesForCatalog(instance *databasev4.ShardingDatabase, OraCatalogSpex databasev4.CatalogSpec) []corev1.PersistentVolumeClaim { var claims []corev1.PersistentVolumeClaim @@ -357,9 +357,9 @@ func volumeClaimTemplatesForCatalog(instance *databasev1alpha1.ShardingDatabase, { ObjectMeta: metav1.ObjectMeta{ Name: OraCatalogSpex.Name + "-oradata-vol4", - Namespace: instance.Spec.Namespace, + Namespace: instance.Namespace, OwnerReferences: getOwnerRef(instance), - Labels: buildLabelsForCatalog(instance, "sharding"), + Labels: buildLabelsForCatalog(instance, "sharding", OraCatalogSpex.Name), }, Spec: corev1.PersistentVolumeClaimSpec{ AccessModes: []corev1.PersistentVolumeAccessMode{ @@ -392,9 +392,9 @@ func volumeClaimTemplatesForCatalog(instance *databasev1alpha1.ShardingDatabase, pvcClaim := corev1.PersistentVolumeClaim{ ObjectMeta: metav1.ObjectMeta{ Name: instance.Name + "shared-storage", - Namespace: instance.Spec.Namespace, + Namespace: instance.Namespace, OwnerReferences: getOwnerRef(instance), - Labels: buildLabelsForCatalog(instance, "sharding"), + Labels: buildLabelsForCatalog(instance, "sharding", OraCatalogSpex.Name), }, Spec: corev1.PersistentVolumeClaimSpec{ AccessModes: []corev1.PersistentVolumeAccessMode{ @@ -417,7 +417,7 @@ func volumeClaimTemplatesForCatalog(instance *databasev1alpha1.ShardingDatabase, return claims } -func BuildServiceDefForCatalog(instance *databasev1alpha1.ShardingDatabase, replicaCount int32, OraCatalogSpex databasev1alpha1.CatalogSpec, svctype string) *corev1.Service { +func BuildServiceDefForCatalog(instance *databasev4.ShardingDatabase, replicaCount int32, OraCatalogSpex databasev4.CatalogSpec, svctype string) *corev1.Service { //service := &corev1.Service{} service := &corev1.Service{ ObjectMeta: buildSvcObjectMetaForCatalog(instance, replicaCount, OraCatalogSpex, svctype), @@ -432,7 +432,7 @@ func BuildServiceDefForCatalog(instance *databasev1alpha1.ShardingDatabase, repl if svctype == "local" { service.Spec.ClusterIP = corev1.ClusterIPNone - service.Spec.Selector = buildLabelsForCatalog(instance, "sharding") + service.Spec.Selector = getSvcLabelsForCatalog(replicaCount, OraCatalogSpex) } // build Service Ports Specs to be exposed. If the PortMappings is not set then default ports will be exposed. @@ -441,7 +441,7 @@ func BuildServiceDefForCatalog(instance *databasev1alpha1.ShardingDatabase, repl } // Function to build Service ObjectMeta -func buildSvcObjectMetaForCatalog(instance *databasev1alpha1.ShardingDatabase, replicaCount int32, OraCatalogSpex databasev1alpha1.CatalogSpec, svctype string) metav1.ObjectMeta { +func buildSvcObjectMetaForCatalog(instance *databasev4.ShardingDatabase, replicaCount int32, OraCatalogSpex databasev4.CatalogSpec, svctype string) metav1.ObjectMeta { // building objectMeta var svcName string if svctype == "local" { @@ -454,14 +454,14 @@ func buildSvcObjectMetaForCatalog(instance *databasev1alpha1.ShardingDatabase, r objmeta := metav1.ObjectMeta{ Name: svcName, - Namespace: instance.Spec.Namespace, + Namespace: instance.Namespace, OwnerReferences: getOwnerRef(instance), - Labels: buildLabelsForCatalog(instance, "sharding"), + Labels: buildLabelsForCatalog(instance, "sharding", OraCatalogSpex.Name), } return objmeta } -func getSvcLabelsForCatalog(replicaCount int32, OraCatalogSpex databasev1alpha1.CatalogSpec) map[string]string { +func getSvcLabelsForCatalog(replicaCount int32, OraCatalogSpex databasev4.CatalogSpec) map[string]string { var labelStr map[string]string = make(map[string]string) if replicaCount == -1 { @@ -475,8 +475,8 @@ func getSvcLabelsForCatalog(replicaCount int32, OraCatalogSpex databasev1alpha1. } // ======================== update Section ======================== -func UpdateProvForCatalog(instance *databasev1alpha1.ShardingDatabase, - OraCatalogSpex databasev1alpha1.CatalogSpec, kClient client.Client, sfSet *appsv1.StatefulSet, catalogPod *corev1.Pod, logger logr.Logger, +func UpdateProvForCatalog(instance *databasev4.ShardingDatabase, + OraCatalogSpex databasev4.CatalogSpec, kClient client.Client, sfSet *appsv1.StatefulSet, catalogPod *corev1.Pod, logger logr.Logger, ) (ctrl.Result, error) { var isUpdate bool = false @@ -485,7 +485,7 @@ func UpdateProvForCatalog(instance *databasev1alpha1.ShardingDatabase, var msg string //msg = "Inside the updateProvForCatalog" - //reqLogger := r.Log.WithValues("Instance.Namespace", instance.Spec.Namespace, "Instance.Name", instance.Name) + //reqLogger := r.Log.WithValues("Instance.Namespace", instance.Namespace, "Instance.Name", instance.Name) LogMessages("DEBUG", msg, nil, instance, logger) // Memory Check diff --git a/commons/sharding/exec.go b/commons/sharding/exec.go index 44f91e51..00caa995 100644 --- a/commons/sharding/exec.go +++ b/commons/sharding/exec.go @@ -44,7 +44,7 @@ import ( "net/http" "time" - databasealphav1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + databasev4 "github.com/oracle/oracle-database-operator/apis/database/v4" "github.com/go-logr/logr" corev1 "k8s.io/api/core/v1" @@ -61,9 +61,9 @@ import ( ) // ExecCMDInContainer execute command in first container of a pod -func ExecCommand(podName string, cmd []string, kubeClient kubernetes.Interface, kubeConfig clientcmd.ClientConfig, instance *databasealphav1.ShardingDatabase, logger logr.Logger) (string, string, error) { +func ExecCommand(podName string, cmd []string, kubeClient kubernetes.Interface, kubeConfig clientcmd.ClientConfig, instance *databasev4.ShardingDatabase, logger logr.Logger) (string, string, error) { - var err1 error = nil + var err1 error = nil var msg string var ( execOut bytes.Buffer @@ -71,28 +71,28 @@ func ExecCommand(podName string, cmd []string, kubeClient kubernetes.Interface, ) for i := 0; i < 5; i++ { - if scheme.Scheme == nil { - time.Sleep(time.Second * 40) - } else { - break - } - } + if scheme.Scheme == nil { + time.Sleep(time.Second * 40) + } else { + break + } + } if kubeClient == nil { - msg = "ExecCommand() : kubeClient is nil" - err1 = fmt.Errorf(msg) - return "Error:","kubeClient is nil",err1 - } + msg = "ExecCommand() : kubeClient is nil" + err1 = fmt.Errorf(msg) + return "Error:", "kubeClient is nil", err1 + } if kubeConfig == nil { - msg = "ExecCommand() : kubeConfig is nil" - err1 = fmt.Errorf(msg) - return "Error:","kubeConfig is nil",err1 + msg = "ExecCommand() : kubeConfig is nil" + err1 = fmt.Errorf(msg) + return "Error:", "kubeConfig is nil", err1 } msg = "" req := kubeClient.CoreV1().RESTClient(). Post(). - Namespace(instance.Spec.Namespace). + Namespace(instance.Namespace). Resource("pods"). Name(podName). SubResource("exec"). @@ -105,6 +105,8 @@ func ExecCommand(podName string, cmd []string, kubeClient kubernetes.Interface, config, err := kubeConfig.ClientConfig() if err != nil { + msg = "Error after executing kubeConfig.ClientConfig" + LogMessages("Error", msg, err, instance, logger) return "Error Occurred", "Error Occurred", err } @@ -136,7 +138,7 @@ func ExecCommand(podName string, cmd []string, kubeClient kubernetes.Interface, return execOut.String(), execErr.String(), nil } -func GetPodCopyConfig(kubeClient kubernetes.Interface, kubeConfig clientcmd.ClientConfig, instance *databasealphav1.ShardingDatabase, logger logr.Logger) (*rest.Config, *kubernetes.Clientset, error) { +func GetPodCopyConfig(kubeClient kubernetes.Interface, kubeConfig clientcmd.ClientConfig, instance *databasev4.ShardingDatabase, logger logr.Logger) (*rest.Config, *kubernetes.Clientset, error) { var clientSet *kubernetes.Clientset config, err := kubeConfig.ClientConfig() @@ -152,7 +154,7 @@ func GetPodCopyConfig(kubeClient kubernetes.Interface, kubeConfig clientcmd.Clie } -func KctlCopyFile(kubeClient kubernetes.Interface, kubeConfig clientcmd.ClientConfig, instance *databasealphav1.ShardingDatabase, restConfig *rest.Config, kclientset *kubernetes.Clientset, logger logr.Logger, src string, dst string, containername string) (*bytes.Buffer, *bytes.Buffer, *bytes.Buffer, error) { +func KctlCopyFile(kubeClient kubernetes.Interface, kubeConfig clientcmd.ClientConfig, instance *databasev4.ShardingDatabase, restConfig *rest.Config, kclientset *kubernetes.Clientset, logger logr.Logger, src string, dst string, containername string) (*bytes.Buffer, *bytes.Buffer, *bytes.Buffer, error) { var in, out, errOut *bytes.Buffer var ioStreams genericclioptions.IOStreams diff --git a/commons/sharding/gsm.go b/commons/sharding/gsm.go index bcdc8866..e6be8770 100644 --- a/commons/sharding/gsm.go +++ b/commons/sharding/gsm.go @@ -44,7 +44,7 @@ import ( "reflect" "strconv" - databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + databasev4 "github.com/oracle/oracle-database-operator/apis/database/v4" "github.com/go-logr/logr" appsv1 "k8s.io/api/apps/v1" @@ -56,7 +56,7 @@ import ( ) // Constants for hello-stateful StatefulSet & Volumes -func buildLabelsForGsm(instance *databasev1alpha1.ShardingDatabase, label string) map[string]string { +func buildLabelsForGsm(instance *databasev4.ShardingDatabase, label string, gsmName string) map[string]string { return map[string]string{ "app": "OracleGsming", "shard_name": "Gsm", @@ -64,7 +64,7 @@ func buildLabelsForGsm(instance *databasev1alpha1.ShardingDatabase, label string } } -func getLabelForGsm(instance *databasev1alpha1.ShardingDatabase) string { +func getLabelForGsm(instance *databasev4.ShardingDatabase) string { // if len(OraGsmSpex.Label) !=0 { // return OraGsmSpex.Label @@ -73,7 +73,7 @@ func getLabelForGsm(instance *databasev1alpha1.ShardingDatabase) string { return instance.Name } -func BuildStatefulSetForGsm(instance *databasev1alpha1.ShardingDatabase, OraGsmSpex databasev1alpha1.GsmSpec) *appsv1.StatefulSet { +func BuildStatefulSetForGsm(instance *databasev4.ShardingDatabase, OraGsmSpex databasev4.GsmSpec) *appsv1.StatefulSet { sfset := &appsv1.StatefulSet{ TypeMeta: buildTypeMetaForGsm(), ObjectMeta: builObjectMetaForGsm(instance, OraGsmSpex), @@ -93,29 +93,29 @@ func buildTypeMetaForGsm() metav1.TypeMeta { } // Function to build ObjectMeta -func builObjectMetaForGsm(instance *databasev1alpha1.ShardingDatabase, OraGsmSpex databasev1alpha1.GsmSpec) metav1.ObjectMeta { +func builObjectMetaForGsm(instance *databasev4.ShardingDatabase, OraGsmSpex databasev4.GsmSpec) metav1.ObjectMeta { // building objectMeta objmeta := metav1.ObjectMeta{ Name: OraGsmSpex.Name, - Namespace: instance.Spec.Namespace, - Labels: buildLabelsForGsm(instance, "sharding"), + Namespace: instance.Namespace, + Labels: buildLabelsForGsm(instance, "sharding", OraGsmSpex.Name), OwnerReferences: getOwnerRef(instance), } return objmeta } // Function to build Stateful Specs -func buildStatefulSpecForGsm(instance *databasev1alpha1.ShardingDatabase, OraGsmSpex databasev1alpha1.GsmSpec) *appsv1.StatefulSetSpec { +func buildStatefulSpecForGsm(instance *databasev4.ShardingDatabase, OraGsmSpex databasev4.GsmSpec) *appsv1.StatefulSetSpec { // building Stateful set Specs sfsetspec := &appsv1.StatefulSetSpec{ ServiceName: OraGsmSpex.Name, Selector: &metav1.LabelSelector{ - MatchLabels: buildLabelsForGsm(instance, "sharding"), + MatchLabels: buildLabelsForGsm(instance, "sharding", OraGsmSpex.Name), }, Template: corev1.PodTemplateSpec{ ObjectMeta: metav1.ObjectMeta{ - Labels: buildLabelsForGsm(instance, "sharding"), + Labels: buildLabelsForGsm(instance, "sharding", OraGsmSpex.Name), }, Spec: *buildPodSpecForGsm(instance, OraGsmSpex), }, @@ -136,7 +136,7 @@ func buildStatefulSpecForGsm(instance *databasev1alpha1.ShardingDatabase, OraGsm // Function to build PodSpec -func buildPodSpecForGsm(instance *databasev1alpha1.ShardingDatabase, OraGsmSpex databasev1alpha1.GsmSpec) *corev1.PodSpec { +func buildPodSpecForGsm(instance *databasev4.ShardingDatabase, OraGsmSpex databasev4.GsmSpec) *corev1.PodSpec { user := oraRunAsUser group := oraFsGroup @@ -170,7 +170,7 @@ func buildPodSpecForGsm(instance *databasev1alpha1.ShardingDatabase, OraGsmSpex } // Function to build Volume Spec -func buildVolumeSpecForGsm(instance *databasev1alpha1.ShardingDatabase, OraGsmSpex databasev1alpha1.GsmSpec) []corev1.Volume { +func buildVolumeSpecForGsm(instance *databasev4.ShardingDatabase, OraGsmSpex databasev4.GsmSpec) []corev1.Volume { var result []corev1.Volume result = []corev1.Volume{ { @@ -204,7 +204,7 @@ func buildVolumeSpecForGsm(instance *databasev1alpha1.ShardingDatabase, OraGsmSp } // Function to build the container Specification -func buildContainerSpecForGsm(instance *databasev1alpha1.ShardingDatabase, OraGsmSpex databasev1alpha1.GsmSpec) []corev1.Container { +func buildContainerSpecForGsm(instance *databasev4.ShardingDatabase, OraGsmSpex databasev4.GsmSpec) []corev1.Container { // building Continer spec var result []corev1.Container var masterGsmFlag = false @@ -272,7 +272,7 @@ func buildContainerSpecForGsm(instance *databasev1alpha1.ShardingDatabase, OraGs } // Function to build the init Container Spec -func buildInitContainerSpecForGsm(instance *databasev1alpha1.ShardingDatabase, OraGsmSpex databasev1alpha1.GsmSpec) []corev1.Container { +func buildInitContainerSpecForGsm(instance *databasev4.ShardingDatabase, OraGsmSpex databasev4.GsmSpec) []corev1.Container { var result []corev1.Container // building the init Container Spec privFlag := true @@ -309,7 +309,7 @@ func buildInitContainerSpecForGsm(instance *databasev1alpha1.ShardingDatabase, O return result } -func buildVolumeMountSpecForGsm(instance *databasev1alpha1.ShardingDatabase, OraGsmSpex databasev1alpha1.GsmSpec) []corev1.VolumeMount { +func buildVolumeMountSpecForGsm(instance *databasev4.ShardingDatabase, OraGsmSpex databasev4.GsmSpec) []corev1.VolumeMount { var result []corev1.VolumeMount result = append(result, corev1.VolumeMount{Name: OraGsmSpex.Name + "secretmap-vol3", MountPath: oraSecretMount, ReadOnly: true}) result = append(result, corev1.VolumeMount{Name: OraGsmSpex.Name + "-oradata-vol4", MountPath: oraGsmDataMount}) @@ -325,7 +325,7 @@ func buildVolumeMountSpecForGsm(instance *databasev1alpha1.ShardingDatabase, Ora return result } -func volumeClaimTemplatesForGsm(instance *databasev1alpha1.ShardingDatabase, OraGsmSpex databasev1alpha1.GsmSpec) []corev1.PersistentVolumeClaim { +func volumeClaimTemplatesForGsm(instance *databasev4.ShardingDatabase, OraGsmSpex databasev4.GsmSpec) []corev1.PersistentVolumeClaim { var claims []corev1.PersistentVolumeClaim @@ -337,8 +337,8 @@ func volumeClaimTemplatesForGsm(instance *databasev1alpha1.ShardingDatabase, Ora { ObjectMeta: metav1.ObjectMeta{ Name: OraGsmSpex.Name + "-oradata-vol4", - Namespace: instance.Spec.Namespace, - Labels: buildLabelsForGsm(instance, "sharding"), + Namespace: instance.Namespace, + Labels: buildLabelsForGsm(instance, "sharding", OraGsmSpex.Name), OwnerReferences: getOwnerRef(instance), }, Spec: corev1.PersistentVolumeClaimSpec{ @@ -354,6 +354,14 @@ func volumeClaimTemplatesForGsm(instance *databasev1alpha1.ShardingDatabase, Ora }, }, } + + if len(OraGsmSpex.PvAnnotations) > 0 { + claims[0].ObjectMeta.Annotations = make(map[string]string) + for key, value := range OraGsmSpex.PvAnnotations { + claims[0].ObjectMeta.Annotations[key] = value + } + } + if len(OraGsmSpex.PvMatchLabels) > 0 { claims[0].Spec.Selector = &metav1.LabelSelector{MatchLabels: OraGsmSpex.PvMatchLabels} } @@ -361,7 +369,7 @@ func volumeClaimTemplatesForGsm(instance *databasev1alpha1.ShardingDatabase, Ora return claims } -func BuildServiceDefForGsm(instance *databasev1alpha1.ShardingDatabase, replicaCount int32, OraGsmSpex databasev1alpha1.GsmSpec, svctype string) *corev1.Service { +func BuildServiceDefForGsm(instance *databasev4.ShardingDatabase, replicaCount int32, OraGsmSpex databasev4.GsmSpec, svctype string) *corev1.Service { //service := &corev1.Service{} service := &corev1.Service{ ObjectMeta: buildSvcObjectMetaForGsm(instance, replicaCount, OraGsmSpex, svctype), @@ -376,7 +384,7 @@ func BuildServiceDefForGsm(instance *databasev1alpha1.ShardingDatabase, replicaC if svctype == "local" { service.Spec.ClusterIP = corev1.ClusterIPNone - service.Spec.Selector = buildLabelsForGsm(instance, "sharding") + service.Spec.Selector = getSvcLabelsForGsm(replicaCount, OraGsmSpex) } // build Service Ports Specs to be exposed. If the PortMappings is not set then default ports will be exposed. @@ -385,7 +393,7 @@ func BuildServiceDefForGsm(instance *databasev1alpha1.ShardingDatabase, replicaC } // Function to build Service ObjectMeta -func buildSvcObjectMetaForGsm(instance *databasev1alpha1.ShardingDatabase, replicaCount int32, OraGsmSpex databasev1alpha1.GsmSpec, svctype string) metav1.ObjectMeta { +func buildSvcObjectMetaForGsm(instance *databasev4.ShardingDatabase, replicaCount int32, OraGsmSpex databasev4.GsmSpec, svctype string) metav1.ObjectMeta { // building objectMeta var svcName string if svctype == "local" { @@ -398,14 +406,14 @@ func buildSvcObjectMetaForGsm(instance *databasev1alpha1.ShardingDatabase, repli objmeta := metav1.ObjectMeta{ Name: svcName, - Namespace: instance.Spec.Namespace, - Labels: buildLabelsForGsm(instance, "sharding"), + Namespace: instance.Namespace, + Labels: buildLabelsForGsm(instance, "sharding", OraGsmSpex.Name), OwnerReferences: getOwnerRef(instance), } return objmeta } -func getSvcLabelsForGsm(replicaCount int32, OraGsmSpex databasev1alpha1.GsmSpec) map[string]string { +func getSvcLabelsForGsm(replicaCount int32, OraGsmSpex databasev4.GsmSpec) map[string]string { var labelStr map[string]string = make(map[string]string) if replicaCount == -1 { @@ -419,8 +427,8 @@ func getSvcLabelsForGsm(replicaCount int32, OraGsmSpex databasev1alpha1.GsmSpec) } // This function cleanup the shard from GSM -func OraCleanupForGsm(instance *databasev1alpha1.ShardingDatabase, - OraGsmSpex databasev1alpha1.GsmSpec, +func OraCleanupForGsm(instance *databasev4.ShardingDatabase, + OraGsmSpex databasev4.GsmSpec, oldReplicaSize int32, newReplicaSize int32, ) string { @@ -435,8 +443,8 @@ func OraCleanupForGsm(instance *databasev1alpha1.ShardingDatabase, return err1 } -func UpdateProvForGsm(instance *databasev1alpha1.ShardingDatabase, - OraGsmSpex databasev1alpha1.GsmSpec, kClient client.Client, sfSet *appsv1.StatefulSet, gsmPod *corev1.Pod, logger logr.Logger, +func UpdateProvForGsm(instance *databasev4.ShardingDatabase, + OraGsmSpex databasev4.GsmSpec, kClient client.Client, sfSet *appsv1.StatefulSet, gsmPod *corev1.Pod, logger logr.Logger, ) (ctrl.Result, error) { var msg string diff --git a/commons/sharding/provstatus.go b/commons/sharding/provstatus.go index 87796553..44544c60 100644 --- a/commons/sharding/provstatus.go +++ b/commons/sharding/provstatus.go @@ -42,7 +42,7 @@ import ( "fmt" "strconv" - databasealphav1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + databasev4 "github.com/oracle/oracle-database-operator/apis/database/v4" "github.com/go-logr/logr" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" @@ -52,7 +52,7 @@ import ( ) // CHeck if record exist in a struct -func CheckGsmStatusInst(instSpex []databasealphav1.GsmStatusDetails, name string, +func CheckGsmStatusInst(instSpex []databasev4.GsmStatusDetails, name string, ) (int, bool) { var status bool = false @@ -69,9 +69,9 @@ func CheckGsmStatusInst(instSpex []databasealphav1.GsmStatusDetails, name string return idx, status } -func UpdateGsmStatusData(instance *databasealphav1.ShardingDatabase, Specidx int, state string, kubeClient kubernetes.Interface, kubeConfig clientcmd.ClientConfig, logger logr.Logger, +func UpdateGsmStatusData(instance *databasev4.ShardingDatabase, Specidx int, state string, kubeClient kubernetes.Interface, kubeConfig clientcmd.ClientConfig, logger logr.Logger, ) { - if state == string(databasealphav1.AvailableState) { + if state == string(databasev4.AvailableState) { // Evaluate following values only if state is set to available svcName := instance.Spec.Gsm[Specidx].Name + "-0." + instance.Spec.Gsm[Specidx].Name k8sExternalSvcName := svcName + strconv.FormatInt(int64(0), 10) + "-svc." + getInstanceNs(instance) + ".svc.cluster.local" @@ -85,40 +85,40 @@ func UpdateGsmStatusData(instance *databasealphav1.ShardingDatabase, Specidx int // internIp := strings.Replace(K8sExternalSvcIP, "/r/n", "", -1) // Populate the Maps - insertOrUpdateGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasealphav1.Name), instance.Spec.Gsm[Specidx].Name) - insertOrUpdateGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasealphav1.DbPasswordSecret), DbPasswordSecret) + insertOrUpdateGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasev4.Name), instance.Spec.Gsm[Specidx].Name) + insertOrUpdateGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasev4.DbPasswordSecret), DbPasswordSecret) if instance.Spec.IsExternalSvc == true { - insertOrUpdateGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasealphav1.K8sExternalSvc), k8sExternalSvcName) - insertOrUpdateGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasealphav1.K8sExternalSvcIP), K8sExternalSvcIP) + insertOrUpdateGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasev4.K8sExternalSvc), k8sExternalSvcName) + insertOrUpdateGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasev4.K8sExternalSvcIP), K8sExternalSvcIP) } - insertOrUpdateGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasealphav1.K8sInternalSvc), K8sInternalSvcName) - insertOrUpdateGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasealphav1.K8sInternalSvcIP), K8sInternalSvcIP) - insertOrUpdateGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasealphav1.State), state) - } else if state == string(databasealphav1.Terminated) { - removeGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasealphav1.Name)) - removeGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasealphav1.K8sInternalSvc)) - removeGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasealphav1.K8sExternalSvc)) - removeGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasealphav1.K8sExternalSvcIP)) - removeGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasealphav1.K8sInternalSvcIP)) - removeGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasealphav1.Role)) + insertOrUpdateGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasev4.K8sInternalSvc), K8sInternalSvcName) + insertOrUpdateGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasev4.K8sInternalSvcIP), K8sInternalSvcIP) + insertOrUpdateGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasev4.State), state) + } else if state == string(databasev4.Terminated) { + removeGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasev4.Name)) + removeGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasev4.K8sInternalSvc)) + removeGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasev4.K8sExternalSvc)) + removeGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasev4.K8sExternalSvcIP)) + removeGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasev4.K8sInternalSvcIP)) + removeGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasev4.Role)) instance.Status.Gsm.Services = "" } else { - insertOrUpdateGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasealphav1.Name), instance.Spec.Gsm[Specidx].Name) - removeGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasealphav1.K8sInternalSvc)) - removeGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasealphav1.K8sExternalSvc)) - removeGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasealphav1.K8sExternalSvcIP)) - removeGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasealphav1.K8sInternalSvcIP)) - removeGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasealphav1.Role)) + insertOrUpdateGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasev4.Name), instance.Spec.Gsm[Specidx].Name) + removeGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasev4.K8sInternalSvc)) + removeGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasev4.K8sExternalSvc)) + removeGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasev4.K8sExternalSvcIP)) + removeGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasev4.K8sInternalSvcIP)) + removeGsmKeys(instance, instance.Spec.Gsm[Specidx].Name, string(databasev4.Role)) instance.Status.Gsm.Services = "" } } -func UpdateCatalogStatusData(instance *databasealphav1.ShardingDatabase, Specidx int, state string, kubeClient kubernetes.Interface, kubeConfig clientcmd.ClientConfig, logger logr.Logger, +func UpdateCatalogStatusData(instance *databasev4.ShardingDatabase, Specidx int, state string, kubeClient kubernetes.Interface, kubeConfig clientcmd.ClientConfig, logger logr.Logger, ) { mode := GetDbOpenMode(instance.Spec.Catalog[Specidx].Name+"-0", instance, kubeClient, kubeConfig, logger) - if state == string(databasealphav1.AvailableState) { + if state == string(databasev4.AvailableState) { // Evaluate following values only if state is set to available svcName := instance.Spec.Catalog[Specidx].Name + "-0." + instance.Spec.Catalog[Specidx].Name k8sExternalSvcName := svcName + strconv.FormatInt(int64(0), 10) + "-svc." + getInstanceNs(instance) + ".svc.cluster.local" @@ -133,52 +133,52 @@ func UpdateCatalogStatusData(instance *databasealphav1.ShardingDatabase, Specidx // internIp := strings.Replace(K8sExternalSvcIP, "/r/n", "", -1) // Populate the Maps - insertOrUpdateCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.Name), instance.Spec.Catalog[Specidx].Name) - insertOrUpdateCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.DbPasswordSecret), DbPasswordSecret) + insertOrUpdateCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasev4.Name), instance.Spec.Catalog[Specidx].Name) + insertOrUpdateCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasev4.DbPasswordSecret), DbPasswordSecret) if instance.Spec.IsExternalSvc == true { - insertOrUpdateCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.K8sExternalSvc), k8sExternalSvcName) - insertOrUpdateCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.K8sExternalSvcIP), K8sExternalSvcIP) + insertOrUpdateCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasev4.K8sExternalSvc), k8sExternalSvcName) + insertOrUpdateCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasev4.K8sExternalSvcIP), K8sExternalSvcIP) } - insertOrUpdateCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.K8sInternalSvc), K8sInternalSvcName) - insertOrUpdateCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.K8sInternalSvcIP), K8sInternalSvcIP) - insertOrUpdateCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.State), state) - insertOrUpdateCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.OracleSid), oracleSid) - insertOrUpdateCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.OraclePdb), oraclePdb) - insertOrUpdateCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.Role), role) - insertOrUpdateCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.OpenMode), mode) - } else if state == string(databasealphav1.Terminated) { - removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.State)) - removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.Name)) - removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.K8sInternalSvc)) - removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.K8sExternalSvc)) - removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.K8sExternalSvcIP)) - removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.K8sInternalSvcIP)) - removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.Role)) - removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.OraclePdb)) - removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.OracleSid)) - removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.Role)) - removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.OpenMode)) + insertOrUpdateCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasev4.K8sInternalSvc), K8sInternalSvcName) + insertOrUpdateCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasev4.K8sInternalSvcIP), K8sInternalSvcIP) + insertOrUpdateCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasev4.State), state) + insertOrUpdateCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasev4.OracleSid), oracleSid) + insertOrUpdateCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasev4.OraclePdb), oraclePdb) + insertOrUpdateCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasev4.Role), role) + insertOrUpdateCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasev4.OpenMode), mode) + } else if state == string(databasev4.Terminated) { + removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasev4.State)) + removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasev4.Name)) + removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasev4.K8sInternalSvc)) + removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasev4.K8sExternalSvc)) + removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasev4.K8sExternalSvcIP)) + removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasev4.K8sInternalSvcIP)) + removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasev4.Role)) + removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasev4.OraclePdb)) + removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasev4.OracleSid)) + removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasev4.Role)) + removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasev4.OpenMode)) } else { - insertOrUpdateCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.State), state) - insertOrUpdateCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.Name), instance.Spec.Catalog[Specidx].Name) - insertOrUpdateCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.OpenMode), mode) - removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.K8sInternalSvc)) - removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.K8sExternalSvc)) - removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.K8sExternalSvcIP)) - removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.K8sInternalSvcIP)) - removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.Role)) - removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.OraclePdb)) - removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.OracleSid)) - removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasealphav1.Role)) + insertOrUpdateCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasev4.State), state) + insertOrUpdateCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasev4.Name), instance.Spec.Catalog[Specidx].Name) + insertOrUpdateCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasev4.OpenMode), mode) + removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasev4.K8sInternalSvc)) + removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasev4.K8sExternalSvc)) + removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasev4.K8sExternalSvcIP)) + removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasev4.K8sInternalSvcIP)) + removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasev4.Role)) + removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasev4.OraclePdb)) + removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasev4.OracleSid)) + removeCatalogKeys(instance, instance.Spec.Catalog[Specidx].Name, string(databasev4.Role)) } } -func UpdateShardStatusData(instance *databasealphav1.ShardingDatabase, Specidx int, state string, kubeClient kubernetes.Interface, kubeConfig clientcmd.ClientConfig, logger logr.Logger, +func UpdateShardStatusData(instance *databasev4.ShardingDatabase, Specidx int, state string, kubeClient kubernetes.Interface, kubeConfig clientcmd.ClientConfig, logger logr.Logger, ) { mode := GetDbOpenMode(instance.Spec.Shard[Specidx].Name+"-0", instance, kubeClient, kubeConfig, logger) - if state == string(databasealphav1.AvailableState) { + if state == string(databasev4.AvailableState) { // Evaluate following values only if state is set to available svcName := instance.Spec.Shard[Specidx].Name + "-0." + instance.Spec.Shard[Specidx].Name k8sExternalSvcName := svcName + strconv.FormatInt(int64(0), 10) + "-svc." + getInstanceNs(instance) + ".svc.cluster.local" @@ -191,49 +191,49 @@ func UpdateShardStatusData(instance *databasealphav1.ShardingDatabase, Specidx i role := GetDbRole(instance.Spec.Shard[Specidx].Name+"-0", instance, kubeClient, kubeConfig, logger) // Populate the Maps - insertOrUpdateShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.Name), instance.Spec.Shard[Specidx].Name) - insertOrUpdateShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.DbPasswordSecret), DbPasswordSecret) + insertOrUpdateShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasev4.Name), instance.Spec.Shard[Specidx].Name) + insertOrUpdateShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasev4.DbPasswordSecret), DbPasswordSecret) if instance.Spec.IsExternalSvc == true { - insertOrUpdateShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.K8sExternalSvc), k8sExternalSvcName) - insertOrUpdateShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.K8sExternalSvcIP), K8sExternalSvcIP) + insertOrUpdateShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasev4.K8sExternalSvc), k8sExternalSvcName) + insertOrUpdateShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasev4.K8sExternalSvcIP), K8sExternalSvcIP) } - insertOrUpdateShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.K8sInternalSvc), K8sInternalSvcName) - insertOrUpdateShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.K8sInternalSvcIP), K8sInternalSvcIP) - insertOrUpdateShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.State), state) - insertOrUpdateShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.OracleSid), oracleSid) - insertOrUpdateShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.OraclePdb), oraclePdb) - insertOrUpdateShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.Role), role) - insertOrUpdateShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.OpenMode), mode) - } else if state == string(databasealphav1.Terminated) { - removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.State)) - removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.Name)) - removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.K8sInternalSvc)) - removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.K8sExternalSvc)) - removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.K8sExternalSvcIP)) - removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.K8sInternalSvcIP)) - removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.Role)) - removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.OraclePdb)) - removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.OracleSid)) - removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.Role)) - removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.OpenMode)) + insertOrUpdateShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasev4.K8sInternalSvc), K8sInternalSvcName) + insertOrUpdateShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasev4.K8sInternalSvcIP), K8sInternalSvcIP) + insertOrUpdateShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasev4.State), state) + insertOrUpdateShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasev4.OracleSid), oracleSid) + insertOrUpdateShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasev4.OraclePdb), oraclePdb) + insertOrUpdateShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasev4.Role), role) + insertOrUpdateShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasev4.OpenMode), mode) + } else if state == string(databasev4.Terminated) { + removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasev4.State)) + removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasev4.Name)) + removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasev4.K8sInternalSvc)) + removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasev4.K8sExternalSvc)) + removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasev4.K8sExternalSvcIP)) + removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasev4.K8sInternalSvcIP)) + removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasev4.Role)) + removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasev4.OraclePdb)) + removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasev4.OracleSid)) + removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasev4.Role)) + removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasev4.OpenMode)) } else { - insertOrUpdateShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.State), state) - insertOrUpdateShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.Name), instance.Spec.Shard[Specidx].Name) - insertOrUpdateShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.OpenMode), mode) - removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.K8sInternalSvc)) - removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.K8sExternalSvc)) - removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.K8sExternalSvcIP)) - removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.K8sInternalSvcIP)) - removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.Role)) - removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.OraclePdb)) - removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.OracleSid)) - removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasealphav1.Role)) + insertOrUpdateShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasev4.State), state) + insertOrUpdateShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasev4.Name), instance.Spec.Shard[Specidx].Name) + insertOrUpdateShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasev4.OpenMode), mode) + removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasev4.K8sInternalSvc)) + removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasev4.K8sExternalSvc)) + removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasev4.K8sExternalSvcIP)) + removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasev4.K8sInternalSvcIP)) + removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasev4.Role)) + removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasev4.OraclePdb)) + removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasev4.OracleSid)) + removeShardKeys(instance, instance.Spec.Shard[Specidx].Name, string(databasev4.Role)) } } -func insertOrUpdateShardKeys(instance *databasealphav1.ShardingDatabase, name string, key string, value string) { +func insertOrUpdateShardKeys(instance *databasev4.ShardingDatabase, name string, key string, value string) { newKey := name + "_" + key if len(instance.Status.Shard) > 0 { if _, ok := instance.Status.Shard[newKey]; ok { @@ -248,7 +248,7 @@ func insertOrUpdateShardKeys(instance *databasealphav1.ShardingDatabase, name st } -func removeShardKeys(instance *databasealphav1.ShardingDatabase, name string, key string) { +func removeShardKeys(instance *databasev4.ShardingDatabase, name string, key string) { newKey := name + "_" + key if len(instance.Status.Shard) > 0 { if _, ok := instance.Status.Shard[newKey]; ok { @@ -258,7 +258,7 @@ func removeShardKeys(instance *databasealphav1.ShardingDatabase, name string, ke } } -func insertOrUpdateCatalogKeys(instance *databasealphav1.ShardingDatabase, name string, key string, value string) { +func insertOrUpdateCatalogKeys(instance *databasev4.ShardingDatabase, name string, key string, value string) { newKey := name + "_" + key if len(instance.Status.Catalog) > 0 { if _, ok := instance.Status.Catalog[newKey]; ok { @@ -273,7 +273,7 @@ func insertOrUpdateCatalogKeys(instance *databasealphav1.ShardingDatabase, name } -func removeCatalogKeys(instance *databasealphav1.ShardingDatabase, name string, key string) { +func removeCatalogKeys(instance *databasev4.ShardingDatabase, name string, key string) { newKey := name + "_" + key if len(instance.Status.Catalog) > 0 { if _, ok := instance.Status.Catalog[newKey]; ok { @@ -283,7 +283,7 @@ func removeCatalogKeys(instance *databasealphav1.ShardingDatabase, name string, } } -func insertOrUpdateGsmKeys(instance *databasealphav1.ShardingDatabase, name string, key string, value string) { +func insertOrUpdateGsmKeys(instance *databasev4.ShardingDatabase, name string, key string, value string) { newKey := name + "_" + key if len(instance.Status.Gsm.Details) > 0 { if _, ok := instance.Status.Gsm.Details[newKey]; ok { @@ -298,7 +298,7 @@ func insertOrUpdateGsmKeys(instance *databasealphav1.ShardingDatabase, name stri } -func removeGsmKeys(instance *databasealphav1.ShardingDatabase, name string, key string) { +func removeGsmKeys(instance *databasev4.ShardingDatabase, name string, key string) { newKey := name + "_" + key if len(instance.Status.Gsm.Details) > 0 { if _, ok := instance.Status.Gsm.Details[newKey]; ok { @@ -308,18 +308,18 @@ func removeGsmKeys(instance *databasealphav1.ShardingDatabase, name string, key } } -func getInstanceNs(instance *databasealphav1.ShardingDatabase) string { +func getInstanceNs(instance *databasev4.ShardingDatabase) string { var namespace string - if instance.Spec.Namespace == "" { + if instance.Namespace == "" { namespace = "default" } else { - namespace = instance.Spec.Namespace + namespace = instance.Namespace } return namespace } // File the meta condition and return the meta view -func GetMetaCondition(instance *databasealphav1.ShardingDatabase, result *ctrl.Result, err *error, stateType string, stateMsg string) metav1.Condition { +func GetMetaCondition(instance *databasev4.ShardingDatabase, result *ctrl.Result, err *error, stateType string, stateMsg string) metav1.Condition { return metav1.Condition{ Type: stateType, @@ -332,7 +332,7 @@ func GetMetaCondition(instance *databasealphav1.ShardingDatabase, result *ctrl.R } // ======================= CHeck GSM Director Status ============== -func CheckGsmStatus(gname string, instance *databasealphav1.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, +func CheckGsmStatus(gname string, instance *databasev4.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, ) error { var err error var msg string = "Inside the checkGsmStatus. Checking GSM director in " + GetFmtStr(gname) + " pod." @@ -349,17 +349,18 @@ func CheckGsmStatus(gname string, instance *databasealphav1.ShardingDatabase, ku // ============ Functiont o check the status of the Shard and catalog ========= // ================================ Validate shard =========================== -func ValidateDbSetup(podName string, instance *databasealphav1.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, +func ValidateDbSetup(podName string, instance *databasev4.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, ) error { _, _, err := ExecCommand(podName, shardValidationCmd(), kubeClient, kubeconfig, instance, logger) if err != nil { + return fmt.Errorf("error ocurred while validating the DB Setup") } return nil } -func UpdateGsmShardStatus(instance *databasealphav1.ShardingDatabase, name string, state string) { +func UpdateGsmShardStatus(instance *databasev4.ShardingDatabase, name string, state string) { //smap := make(map[string]string) if _, ok := instance.Status.Gsm.Shards[name]; ok { instance.Status.Gsm.Shards[name] = state @@ -383,7 +384,7 @@ func UpdateGsmShardStatus(instance *databasealphav1.ShardingDatabase, name strin } -func GetGsmShardStatus(instance *databasealphav1.ShardingDatabase, name string) string { +func GetGsmShardStatus(instance *databasev4.ShardingDatabase, name string) string { if _, ok := instance.Status.Gsm.Shards[name]; ok { return instance.Status.Gsm.Shards[name] @@ -392,7 +393,7 @@ func GetGsmShardStatus(instance *databasealphav1.ShardingDatabase, name string) } -func GetGsmShardStatusKey(instance *databasealphav1.ShardingDatabase, key string) string { +func GetGsmShardStatusKey(instance *databasev4.ShardingDatabase, key string) string { if _, ok := instance.Status.Shard[key]; ok { return instance.Status.Shard[key] @@ -401,7 +402,7 @@ func GetGsmShardStatusKey(instance *databasealphav1.ShardingDatabase, key string } -func GetGsmCatalogStatusKey(instance *databasealphav1.ShardingDatabase, key string) string { +func GetGsmCatalogStatusKey(instance *databasev4.ShardingDatabase, key string) string { if _, ok := instance.Status.Catalog[key]; ok { return instance.Status.Catalog[key] @@ -410,7 +411,7 @@ func GetGsmCatalogStatusKey(instance *databasealphav1.ShardingDatabase, key stri } -func GetGsmDetailsSttausKey(instance *databasealphav1.ShardingDatabase, key string) string { +func GetGsmDetailsSttausKey(instance *databasev4.ShardingDatabase, key string) string { if _, ok := instance.Status.Gsm.Details[key]; ok { return instance.Status.Gsm.Details[key] diff --git a/commons/sharding/scommon.go b/commons/sharding/scommon.go index 99987661..3b3f1b04 100644 --- a/commons/sharding/scommon.go +++ b/commons/sharding/scommon.go @@ -44,8 +44,7 @@ import ( "fmt" "slices" - databasealphav1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" - databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + databasev4 "github.com/oracle/oracle-database-operator/apis/database/v4" "regexp" "strconv" @@ -103,7 +102,7 @@ const ( ) // Function to build the env var specification -func buildEnvVarsSpec(instance *databasealphav1.ShardingDatabase, variables []databasealphav1.EnvironmentVariable, name string, restype string, masterFlag bool, directorParams string) []corev1.EnvVar { +func buildEnvVarsSpec(instance *databasev4.ShardingDatabase, variables []databasev4.EnvironmentVariable, name string, restype string, masterFlag bool, directorParams string) []corev1.EnvVar { var result []corev1.EnvVar var varinfo string var sidFlag bool = false @@ -295,7 +294,7 @@ func buildEnvVarsSpec(instance *databasealphav1.ShardingDatabase, variables []da } if instance.Spec.InvitedNodeSubnetFlag == "" { - instance.Spec.InvitedNodeSubnetFlag = "FALSE" + instance.Spec.InvitedNodeSubnetFlag = "TRUE" } if strings.ToUpper(instance.Spec.InvitedNodeSubnetFlag) != "FALSE" { @@ -351,7 +350,7 @@ func buildEnvVarsSpec(instance *databasealphav1.ShardingDatabase, variables []da } // FUnction to build the svc definition for catalog/shard and GSM -func buildSvcPortsDef(instance *databasealphav1.ShardingDatabase, resType string) []corev1.ServicePort { +func buildSvcPortsDef(instance *databasev4.ShardingDatabase, resType string) []corev1.ServicePort { var result []corev1.ServicePort if len(instance.Spec.PortMappings) > 0 { for _, portMapping := range instance.Spec.PortMappings { @@ -393,12 +392,12 @@ func generateName(base string) string { } // Function to generate the port mapping -func generatePortMapping(portMapping databasealphav1.PortMapping) string { +func generatePortMapping(portMapping databasev4.PortMapping) string { return generateName(fmt.Sprintf("%s-%d-%d-", "tcp", portMapping.Port, portMapping.TargetPort)) } -func LogMessages(msgtype string, msg string, err error, instance *databasealphav1.ShardingDatabase, logger logr.Logger) { +func LogMessages(msgtype string, msg string, err error, instance *databasev4.ShardingDatabase, logger logr.Logger) { // setting logrus formatter //logrus.SetFormatter(&logrus.JSONFormatter{}) //logrus.SetOutput(os.Stdout) @@ -411,6 +410,8 @@ func LogMessages(msgtype string, msg string, err error, instance *databasealphav } } else if msgtype == "INFO" { logger.Info(msg) + } else if msgtype == "Error" { + logger.Error(err, msg) } } @@ -419,7 +420,7 @@ func GetGsmPodName(gsmName string) string { return podName } -func GetSidName(variables []databasealphav1.EnvironmentVariable, name string) string { +func GetSidName(variables []databasev4.EnvironmentVariable, name string) string { var result string for _, variable := range variables { @@ -433,7 +434,7 @@ func GetSidName(variables []databasealphav1.EnvironmentVariable, name string) st return result } -func GetPdbName(variables []databasealphav1.EnvironmentVariable, name string) string { +func GetPdbName(variables []databasev4.EnvironmentVariable, name string) string { var result string for _, variable := range variables { @@ -447,34 +448,34 @@ func GetPdbName(variables []databasealphav1.EnvironmentVariable, name string) st return result } -func getlabelsForGsm(instance *databasealphav1.ShardingDatabase) map[string]string { - return buildLabelsForGsm(instance, "sharding") +func getlabelsForGsm(instance *databasev4.ShardingDatabase) map[string]string { + return buildLabelsForGsm(instance, "sharding", "gsm") } -func getlabelsForShard(instance *databasealphav1.ShardingDatabase) map[string]string { - return buildLabelsForShard(instance, "sharding") +func getlabelsForShard(instance *databasev4.ShardingDatabase) map[string]string { + return buildLabelsForShard(instance, "sharding", "shard") } -func getlabelsForCatalog(instance *databasealphav1.ShardingDatabase) map[string]string { - return buildLabelsForCatalog(instance, "sharding") +func getlabelsForCatalog(instance *databasev4.ShardingDatabase) map[string]string { + return buildLabelsForCatalog(instance, "sharding", "catalog") } -func LabelsForProvShardKind(instance *databasealphav1.ShardingDatabase, sftype string, +func LabelsForProvShardKind(instance *databasev4.ShardingDatabase, sftype string, ) map[string]string { if sftype == "shard" { - return buildLabelsForShard(instance, "sharding") + return buildLabelsForShard(instance, "sharding", "shard") } return nil } -func CheckSfset(sfsetName string, instance *databasealphav1.ShardingDatabase, kClient client.Client) (*appsv1.StatefulSet, error) { +func CheckSfset(sfsetName string, instance *databasev4.ShardingDatabase, kClient client.Client) (*appsv1.StatefulSet, error) { sfSetFound := &appsv1.StatefulSet{} err := kClient.Get(context.TODO(), types.NamespacedName{ Name: sfsetName, - Namespace: instance.Spec.Namespace, + Namespace: instance.Namespace, }, sfSetFound) if err != nil { return sfSetFound, err @@ -482,11 +483,11 @@ func CheckSfset(sfsetName string, instance *databasealphav1.ShardingDatabase, kC return sfSetFound, nil } -func checkPvc(pvcName string, instance *databasealphav1.ShardingDatabase, kClient client.Client) (*corev1.PersistentVolumeClaim, error) { +func checkPvc(pvcName string, instance *databasev4.ShardingDatabase, kClient client.Client) (*corev1.PersistentVolumeClaim, error) { pvcFound := &corev1.PersistentVolumeClaim{} err := kClient.Get(context.TODO(), types.NamespacedName{ Name: pvcName, - Namespace: instance.Spec.Namespace, + Namespace: instance.Namespace, }, pvcFound) if err != nil { return pvcFound, err @@ -494,7 +495,7 @@ func checkPvc(pvcName string, instance *databasealphav1.ShardingDatabase, kClien return pvcFound, nil } -func DelPvc(pvcName string, instance *databasealphav1.ShardingDatabase, kClient client.Client, logger logr.Logger) error { +func DelPvc(pvcName string, instance *databasev4.ShardingDatabase, kClient client.Client, logger logr.Logger) error { LogMessages("DEBUG", "Inside the delPvc and received param: "+GetFmtStr(pvcName), nil, instance, logger) pvcFound, err := checkPvc(pvcName, instance, kClient) @@ -510,7 +511,7 @@ func DelPvc(pvcName string, instance *databasealphav1.ShardingDatabase, kClient return nil } -func DelSvc(pvcName string, instance *databasealphav1.ShardingDatabase, kClient client.Client, logger logr.Logger) error { +func DelSvc(pvcName string, instance *databasev4.ShardingDatabase, kClient client.Client, logger logr.Logger) error { LogMessages("DEBUG", "Inside the delPvc and received param: "+GetFmtStr(pvcName), nil, instance, logger) pvcFound, err := checkPvc(pvcName, instance, kClient) @@ -526,11 +527,11 @@ func DelSvc(pvcName string, instance *databasealphav1.ShardingDatabase, kClient return nil } -func CheckSvc(svcName string, instance *databasealphav1.ShardingDatabase, kClient client.Client) (*corev1.Service, error) { +func CheckSvc(svcName string, instance *databasev4.ShardingDatabase, kClient client.Client) (*corev1.Service, error) { svcFound := &corev1.Service{} err := kClient.Get(context.TODO(), types.NamespacedName{ Name: svcName, - Namespace: instance.Spec.Namespace, + Namespace: instance.Namespace, }, svcFound) if err != nil { return svcFound, err @@ -538,7 +539,7 @@ func CheckSvc(svcName string, instance *databasealphav1.ShardingDatabase, kClien return svcFound, nil } -func PodListValidation(podList *corev1.PodList, sfName string, instance *databasealphav1.ShardingDatabase, kClient client.Client, +func PodListValidation(podList *corev1.PodList, sfName string, instance *databasev4.ShardingDatabase, kClient client.Client, ) (bool, *corev1.Pod) { var isPodExist bool = false @@ -574,7 +575,7 @@ func PodListValidation(podList *corev1.PodList, sfName string, instance *databas return isPodExist, podInfo } -func GetPodList(sfsetName string, resType string, instance *databasealphav1.ShardingDatabase, kClient client.Client, +func GetPodList(sfsetName string, resType string, instance *databasev4.ShardingDatabase, kClient client.Client, ) (*corev1.PodList, error) { podList := &corev1.PodList{} //labelSelector := labels.SelectorFromSet(getlabelsForGsm(instance)) @@ -595,7 +596,7 @@ func GetPodList(sfsetName string, resType string, instance *databasealphav1.Shar return nil, err1 } - listOps := &client.ListOptions{Namespace: instance.Spec.Namespace, LabelSelector: labelSelector} + listOps := &client.ListOptions{Namespace: instance.Namespace, LabelSelector: labelSelector} err := kClient.List(context.TODO(), podList, listOps) if err != nil { @@ -604,11 +605,11 @@ func GetPodList(sfsetName string, resType string, instance *databasealphav1.Shar return podList, nil } -func checkPod(instance *databasealphav1.ShardingDatabase, pod *corev1.Pod, kClient client.Client, +func checkPod(instance *databasev4.ShardingDatabase, pod *corev1.Pod, kClient client.Client, ) error { err := kClient.Get(context.TODO(), types.NamespacedName{ Name: pod.Name, - Namespace: instance.Spec.Namespace, + Namespace: instance.Namespace, }, pod) if err != nil { @@ -664,15 +665,15 @@ func checkContainerStatus(pod *corev1.Pod, kClient client.Client, // Namespace related function -func AddNamespace(instance *databasealphav1.ShardingDatabase, kClient client.Client, logger logr.Logger, +func AddNamespace(instance *databasev4.ShardingDatabase, kClient client.Client, logger logr.Logger, ) error { var msg string ns := &corev1.Namespace{} - err := kClient.Get(context.TODO(), types.NamespacedName{Name: instance.Spec.Namespace}, ns) + err := kClient.Get(context.TODO(), types.NamespacedName{Name: instance.Namespace}, ns) if err != nil { - //msg = "Namespace " + instance.Spec.Namespace + " doesn't exist! creating namespace" + //msg = "Namespace " + instance.Namespace + " doesn't exist! creating namespace" if errors.IsNotFound(err) { - err = kClient.Create(context.TODO(), NewNamespace(instance.Spec.Namespace)) + err = kClient.Create(context.TODO(), NewNamespace(instance.Namespace)) if err != nil { msg = "Error in creating namespace!" LogMessages("Error", msg, nil, instance, logger) @@ -700,7 +701,7 @@ func NewNamespace(name string) *corev1.Namespace { } } -func getOwnerRef(instance *databasealphav1.ShardingDatabase, +func getOwnerRef(instance *databasev4.ShardingDatabase, ) []metav1.OwnerReference { var ownerRef []metav1.OwnerReference @@ -708,8 +709,8 @@ func getOwnerRef(instance *databasealphav1.ShardingDatabase, return ownerRef } -func buildCatalogParams(instance *databasealphav1.ShardingDatabase) string { - var variables []databasealphav1.EnvironmentVariable = instance.Spec.Catalog[0].EnvVars +func buildCatalogParams(instance *databasev4.ShardingDatabase) string { + var variables []databasev4.EnvironmentVariable = instance.Spec.Catalog[0].EnvVars var result string var varinfo string var sidFlag bool = false @@ -858,8 +859,8 @@ func buildCatalogParams(instance *databasealphav1.ShardingDatabase) string { return result } -func buildDirectorParams(instance *databasealphav1.ShardingDatabase, oraGsmSpex databasealphav1.GsmSpec, idx int) string { - var variables []databasealphav1.EnvironmentVariable +func buildDirectorParams(instance *databasev4.ShardingDatabase, oraGsmSpex databasev4.GsmSpec, idx int) string { + var variables []databasev4.EnvironmentVariable var result string var varinfo string var dnameFlag bool = false @@ -905,7 +906,7 @@ func buildDirectorParams(instance *databasealphav1.ShardingDatabase, oraGsmSpex return result } -func BuildShardParams(instance *databasealphav1.ShardingDatabase, sfSet *appsv1.StatefulSet, OraShardSpex databasev1alpha1.ShardSpec) string { +func BuildShardParams(instance *databasev4.ShardingDatabase, sfSet *appsv1.StatefulSet, OraShardSpex databasev4.ShardSpec) string { var variables []corev1.EnvVar = sfSet.Spec.Template.Spec.Containers[0].Env var result string var varinfo string @@ -1014,11 +1015,11 @@ func BuildShardParams(instance *databasealphav1.ShardingDatabase, sfSet *appsv1. return result } -func labelsForShardingDatabaseKind(instance *databasealphav1.ShardingDatabase, sftype string, +func labelsForShardingDatabaseKind(instance *databasev4.ShardingDatabase, sftype string, ) map[string]string { if sftype == "shard" { - return buildLabelsForShard(instance, "sharding") + return buildLabelsForShard(instance, "sharding", "shard") } return nil @@ -1199,7 +1200,7 @@ func GetFmtStr(pstr string, return "[" + pstr + "]" } -func ReadConfigMap(cmName string, instance *databasealphav1.ShardingDatabase, kClient client.Client, logger logr.Logger, +func ReadConfigMap(cmName string, instance *databasev4.ShardingDatabase, kClient client.Client, logger logr.Logger, ) (string, string, string, string, string, string) { var region, fingerprint, user, tenancy, passphrase, str1, topicid, k, value string @@ -1210,7 +1211,7 @@ func ReadConfigMap(cmName string, instance *databasealphav1.ShardingDatabase, kC // Reding a config map err = kClient.Get(context.TODO(), types.NamespacedName{ Name: cmName, - Namespace: instance.Spec.Namespace, + Namespace: instance.Namespace, }, cm) if err != nil { @@ -1253,7 +1254,7 @@ func ReadConfigMap(cmName string, instance *databasealphav1.ShardingDatabase, kC return region, user, tenancy, passphrase, fingerprint, topicid } -func ReadSecret(secName string, instance *databasealphav1.ShardingDatabase, kClient client.Client, logger logr.Logger, +func ReadSecret(secName string, instance *databasev4.ShardingDatabase, kClient client.Client, logger logr.Logger, ) string { var value string @@ -1263,7 +1264,7 @@ func ReadSecret(secName string, instance *databasealphav1.ShardingDatabase, kCli // Reading a Secret var err error = kClient.Get(context.TODO(), types.NamespacedName{ Name: secName, - Namespace: instance.Spec.Namespace, + Namespace: instance.Namespace, }, sc) if err != nil { @@ -1285,20 +1286,17 @@ func GetK8sClientConfig(kClient client.Client) (clientcmd.ClientConfig, kubernet var kubeConfig clientcmd.ClientConfig var kubeClient kubernetes.Interface - databasealphav1.KubeConfigOnce.Do(func() { - loadingRules := clientcmd.NewDefaultClientConfigLoadingRules() - configOverrides := &clientcmd.ConfigOverrides{} - kubeConfig = clientcmd.NewNonInteractiveDeferredLoadingClientConfig(loadingRules, configOverrides) - config, err := kubeConfig.ClientConfig() - if err != nil { - err1 = err - } - kubeClient, err = kubernetes.NewForConfig(config) - if err != nil { - err1 = err - } - - }) + loadingRules := clientcmd.NewDefaultClientConfigLoadingRules() + configOverrides := &clientcmd.ConfigOverrides{} + kubeConfig = clientcmd.NewNonInteractiveDeferredLoadingClientConfig(loadingRules, configOverrides) + config, err := kubeConfig.ClientConfig() + if err != nil { + err1 = err + } + kubeClient, err = kubernetes.NewForConfig(config) + if err != nil { + err1 = err + } return kubeConfig, kubeClient, err1 } @@ -1312,7 +1310,7 @@ func Contains(list []string, s string) bool { } // Function to check shadrd in GSM -func CheckShardInGsm(gsmPodName string, sparams string, instance *databasealphav1.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, +func CheckShardInGsm(gsmPodName string, sparams string, instance *databasev4.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, ) error { _, _, err := ExecCommand(gsmPodName, getShardCheckCmd(sparams), kubeClient, kubeconfig, instance, logger) @@ -1325,7 +1323,7 @@ func CheckShardInGsm(gsmPodName string, sparams string, instance *databasealphav } // Function to check the online Shard -func CheckOnlineShardInGsm(gsmPodName string, sparams string, instance *databasealphav1.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, +func CheckOnlineShardInGsm(gsmPodName string, sparams string, instance *databasev4.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, ) error { _, _, err := ExecCommand(gsmPodName, getOnlineShardCmd(sparams), kubeClient, kubeconfig, instance, logger) @@ -1338,7 +1336,7 @@ func CheckOnlineShardInGsm(gsmPodName string, sparams string, instance *database } // Function to move the chunks -func MoveChunks(gsmPodName string, sparams string, instance *databasealphav1.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, +func MoveChunks(gsmPodName string, sparams string, instance *databasev4.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, ) error { _, _, err := ExecCommand(gsmPodName, getMoveChunksCmd(sparams), kubeClient, kubeconfig, instance, logger) @@ -1351,7 +1349,7 @@ func MoveChunks(gsmPodName string, sparams string, instance *databasealphav1.Sha } // Function to verify the chunks -func VerifyChunks(gsmPodName string, sparams string, instance *databasealphav1.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, +func VerifyChunks(gsmPodName string, sparams string, instance *databasev4.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, ) error { _, _, err := ExecCommand(gsmPodName, getNoChunksCmd(sparams), kubeClient, kubeconfig, instance, logger) if err != nil { @@ -1363,7 +1361,7 @@ func VerifyChunks(gsmPodName string, sparams string, instance *databasealphav1.S } // Function to verify the chunks -func AddShardInGsm(gsmPodName string, sparams string, instance *databasealphav1.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, +func AddShardInGsm(gsmPodName string, sparams string, instance *databasev4.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, ) error { _, _, err := ExecCommand(gsmPodName, getShardAddCmd(sparams), kubeClient, kubeconfig, instance, logger) if err != nil { @@ -1375,7 +1373,7 @@ func AddShardInGsm(gsmPodName string, sparams string, instance *databasealphav1. } // Function to deploy the Shards -func DeployShardInGsm(gsmPodName string, sparams string, instance *databasealphav1.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, +func DeployShardInGsm(gsmPodName string, sparams string, instance *databasev4.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, ) error { _, _, err := ExecCommand(gsmPodName, getdeployShardCmd(), kubeClient, kubeconfig, instance, logger) if err != nil { @@ -1387,7 +1385,7 @@ func DeployShardInGsm(gsmPodName string, sparams string, instance *databasealpha } // Function to verify the chunks -func CancelChunksInGsm(gsmPodName string, sparams string, instance *databasealphav1.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, +func CancelChunksInGsm(gsmPodName string, sparams string, instance *databasev4.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, ) error { _, _, err := ExecCommand(gsmPodName, getCancelChunksCmd(sparams), kubeClient, kubeconfig, instance, logger) if err != nil { @@ -1399,7 +1397,7 @@ func CancelChunksInGsm(gsmPodName string, sparams string, instance *databasealph } // Function to delete the shard -func RemoveShardFromGsm(gsmPodName string, sparams string, instance *databasealphav1.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, +func RemoveShardFromGsm(gsmPodName string, sparams string, instance *databasev4.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, ) error { _, _, err := ExecCommand(gsmPodName, getShardDelCmd(sparams), kubeClient, kubeconfig, instance, logger) if err != nil { @@ -1410,7 +1408,7 @@ func RemoveShardFromGsm(gsmPodName string, sparams string, instance *databasealp return nil } -func GetSvcIp(PodName string, sparams string, instance *databasealphav1.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, +func GetSvcIp(PodName string, sparams string, instance *databasev4.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, ) (string, string, error) { stdoutput, stderror, err := ExecCommand(PodName, GetIpCmd(sparams), kubeClient, kubeconfig, instance, logger) if err != nil { @@ -1421,7 +1419,7 @@ func GetSvcIp(PodName string, sparams string, instance *databasealphav1.Sharding return strings.Replace(stdoutput, "\r\n", "", -1), strings.Replace(stderror, "/r/n", "", -1), nil } -func GetGsmServices(PodName string, instance *databasealphav1.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, +func GetGsmServices(PodName string, instance *databasev4.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, ) string { stdoutput, _, err := ExecCommand(PodName, getGsmSvcCmd(), kubeClient, kubeconfig, instance, logger) if err != nil { @@ -1432,7 +1430,7 @@ func GetGsmServices(PodName string, instance *databasealphav1.ShardingDatabase, return stdoutput } -func GetDbRole(PodName string, instance *databasealphav1.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, +func GetDbRole(PodName string, instance *databasev4.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, ) string { stdoutput, _, err := ExecCommand(PodName, getDbRoleCmd(), kubeClient, kubeconfig, instance, logger) if err != nil { @@ -1443,7 +1441,7 @@ func GetDbRole(PodName string, instance *databasealphav1.ShardingDatabase, kubeC return strings.TrimSpace(stdoutput) } -func GetDbOpenMode(PodName string, instance *databasealphav1.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, +func GetDbOpenMode(PodName string, instance *databasev4.ShardingDatabase, kubeClient kubernetes.Interface, kubeconfig clientcmd.ClientConfig, logger logr.Logger, ) string { stdoutput, _, err := ExecCommand(PodName, getDbModeCmd(), kubeClient, kubeconfig, instance, logger) if err != nil { @@ -1454,7 +1452,7 @@ func GetDbOpenMode(PodName string, instance *databasealphav1.ShardingDatabase, k return strings.TrimSpace(stdoutput) } -func SfsetLabelPatch(sfSetFound *appsv1.StatefulSet, sfSetPod *corev1.Pod, instance *databasealphav1.ShardingDatabase, kClient client.Client, +func SfsetLabelPatch(sfSetFound *appsv1.StatefulSet, sfSetPod *corev1.Pod, instance *databasev4.ShardingDatabase, kClient client.Client, ) error { //var msg string @@ -1462,7 +1460,7 @@ func SfsetLabelPatch(sfSetFound *appsv1.StatefulSet, sfSetPod *corev1.Pod, insta var err error sfsetCopy := sfSetFound.DeepCopy() - sfsetCopy.Labels[string(databasealphav1.ShardingDelLabelKey)] = string(databasealphav1.ShardingDelLabelTrueValue) + sfsetCopy.Labels[string(databasev4.ShardingDelLabelKey)] = string(databasev4.ShardingDelLabelTrueValue) patch := client.MergeFrom(sfSetFound) err = kClient.Patch(context.Background(), sfsetCopy, patch) if err != nil { @@ -1470,7 +1468,7 @@ func SfsetLabelPatch(sfSetFound *appsv1.StatefulSet, sfSetPod *corev1.Pod, insta } podCopy := sfSetPod.DeepCopy() - podCopy.Labels[string(databasealphav1.ShardingDelLabelKey)] = string(databasealphav1.ShardingDelLabelTrueValue) + podCopy.Labels[string(databasev4.ShardingDelLabelKey)] = string(databasev4.ShardingDelLabelTrueValue) podPatch := client.MergeFrom(sfSetPod.DeepCopy()) err = kClient.Patch(context.Background(), podCopy, podPatch) if err != nil { @@ -1480,14 +1478,14 @@ func SfsetLabelPatch(sfSetFound *appsv1.StatefulSet, sfSetPod *corev1.Pod, insta return nil } -func InstanceShardPatch(obj client.Object, instance *databasealphav1.ShardingDatabase, kClient client.Client, id int32, field string, value string, +func InstanceShardPatch(obj client.Object, instance *databasev4.ShardingDatabase, kClient client.Client, id int32, field string, value string, ) error { var err error instSpec := instance.Spec instSpec.Shard[id].IsDelete = "failed" instshardM, _ := json.Marshal(struct { - Spec *databasealphav1.ShardingDatabaseSpec `json:"spec":` + Spec *databasev4.ShardingDatabaseSpec `json:"spec":` }{ Spec: &instSpec, }) @@ -1504,7 +1502,7 @@ func InstanceShardPatch(obj client.Object, instance *databasealphav1.ShardingDat // Send Notification -func SendNotification(title string, body string, instance *databasealphav1.ShardingDatabase, topicId string, rclient ons.NotificationDataPlaneClient, logger logr.Logger, +func SendNotification(title string, body string, instance *databasev4.ShardingDatabase, topicId string, rclient ons.NotificationDataPlaneClient, logger logr.Logger, ) { var msg string req := ons.PublishMessageRequest{TopicId: common.String(topicId), @@ -1525,14 +1523,14 @@ func GetSecretMount() string { return oraSecretMount } -func checkTdeWalletFlag(instance *databasev1alpha1.ShardingDatabase) bool { +func checkTdeWalletFlag(instance *databasev4.ShardingDatabase) bool { if strings.ToLower(instance.Spec.IsTdeWallet) == "enable" { return true } return false } -func CheckIsDeleteFlag(delStr string, instance *databasealphav1.ShardingDatabase, logger logr.Logger) bool { +func CheckIsDeleteFlag(delStr string, instance *databasev4.ShardingDatabase, logger logr.Logger) bool { if strings.ToLower(delStr) == "enable" { return true } @@ -1542,7 +1540,7 @@ func CheckIsDeleteFlag(delStr string, instance *databasealphav1.ShardingDatabase return false } -func getTdeWalletMountLoc(instance *databasev1alpha1.ShardingDatabase) string { +func getTdeWalletMountLoc(instance *databasev4.ShardingDatabase) string { if len(instance.Spec.TdeWalletPvcMountLocation) > 0 { return instance.Spec.TdeWalletPvcMountLocation } diff --git a/commons/sharding/shard.go b/commons/sharding/shard.go index c76fc0e5..e48b56dd 100644 --- a/commons/sharding/shard.go +++ b/commons/sharding/shard.go @@ -43,7 +43,7 @@ import ( "reflect" "strconv" - databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + databasev4 "github.com/oracle/oracle-database-operator/apis/database/v4" "github.com/go-logr/logr" appsv1 "k8s.io/api/apps/v1" @@ -54,7 +54,7 @@ import ( "sigs.k8s.io/controller-runtime/pkg/client" ) -func buildLabelsForShard(instance *databasev1alpha1.ShardingDatabase, label string) map[string]string { +func buildLabelsForShard(instance *databasev4.ShardingDatabase, label string, shardName string) map[string]string { return map[string]string{ "app": "OracleSharding", "type": "Shard", @@ -62,7 +62,7 @@ func buildLabelsForShard(instance *databasev1alpha1.ShardingDatabase, label stri } } -func getLabelForShard(instance *databasev1alpha1.ShardingDatabase) string { +func getLabelForShard(instance *databasev4.ShardingDatabase) string { // if len(OraShardSpex.Label) !=0 { // return OraShardSpex.Label @@ -71,7 +71,7 @@ func getLabelForShard(instance *databasev1alpha1.ShardingDatabase) string { return instance.Name } -func BuildStatefulSetForShard(instance *databasev1alpha1.ShardingDatabase, OraShardSpex databasev1alpha1.ShardSpec) *appsv1.StatefulSet { +func BuildStatefulSetForShard(instance *databasev4.ShardingDatabase, OraShardSpex databasev4.ShardSpec) *appsv1.StatefulSet { sfset := &appsv1.StatefulSet{ TypeMeta: buildTypeMetaForShard(), ObjectMeta: builObjectMetaForShard(instance, OraShardSpex), @@ -92,29 +92,29 @@ func buildTypeMetaForShard() metav1.TypeMeta { } // Function to build ObjectMeta -func builObjectMetaForShard(instance *databasev1alpha1.ShardingDatabase, OraShardSpex databasev1alpha1.ShardSpec) metav1.ObjectMeta { +func builObjectMetaForShard(instance *databasev4.ShardingDatabase, OraShardSpex databasev4.ShardSpec) metav1.ObjectMeta { // building objectMeta objmeta := metav1.ObjectMeta{ Name: OraShardSpex.Name, - Namespace: instance.Spec.Namespace, + Namespace: instance.Namespace, OwnerReferences: getOwnerRef(instance), - Labels: buildLabelsForShard(instance, "sharding"), + Labels: buildLabelsForShard(instance, "sharding", OraShardSpex.Name), } return objmeta } // Function to build Stateful Specs -func buildStatefulSpecForShard(instance *databasev1alpha1.ShardingDatabase, OraShardSpex databasev1alpha1.ShardSpec) *appsv1.StatefulSetSpec { +func buildStatefulSpecForShard(instance *databasev4.ShardingDatabase, OraShardSpex databasev4.ShardSpec) *appsv1.StatefulSetSpec { // building Stateful set Specs var size int32 = 1 sfsetspec := &appsv1.StatefulSetSpec{ ServiceName: OraShardSpex.Name, Selector: &metav1.LabelSelector{ - MatchLabels: buildLabelsForShard(instance, "sharding"), + MatchLabels: buildLabelsForShard(instance, "sharding", OraShardSpex.Name), }, Template: corev1.PodTemplateSpec{ ObjectMeta: metav1.ObjectMeta{ - Labels: buildLabelsForShard(instance, "sharding"), + Labels: buildLabelsForShard(instance, "sharding", OraShardSpex.Name), }, Spec: *buildPodSpecForShard(instance, OraShardSpex), }, @@ -132,7 +132,7 @@ func buildStatefulSpecForShard(instance *databasev1alpha1.ShardingDatabase, OraS // Function to build PodSpec -func buildPodSpecForShard(instance *databasev1alpha1.ShardingDatabase, OraShardSpex databasev1alpha1.ShardSpec) *corev1.PodSpec { +func buildPodSpecForShard(instance *databasev4.ShardingDatabase, OraShardSpex databasev4.ShardSpec) *corev1.PodSpec { user := oraRunAsUser group := oraFsGroup @@ -168,7 +168,7 @@ func buildPodSpecForShard(instance *databasev1alpha1.ShardingDatabase, OraShardS } // Function to build Volume Spec -func buildVolumeSpecForShard(instance *databasev1alpha1.ShardingDatabase, OraShardSpex databasev1alpha1.ShardSpec) []corev1.Volume { +func buildVolumeSpecForShard(instance *databasev4.ShardingDatabase, OraShardSpex databasev4.ShardSpec) []corev1.Volume { var result []corev1.Volume result = []corev1.Volume{ { @@ -208,7 +208,7 @@ func buildVolumeSpecForShard(instance *databasev1alpha1.ShardingDatabase, OraSha } // Function to build the container Specification -func buildContainerSpecForShard(instance *databasev1alpha1.ShardingDatabase, OraShardSpex databasev1alpha1.ShardSpec) []corev1.Container { +func buildContainerSpecForShard(instance *databasev4.ShardingDatabase, OraShardSpex databasev4.ShardSpec) []corev1.Container { // building Continer spec var result []corev1.Container containerSpec := corev1.Container{ @@ -287,7 +287,7 @@ func buildContainerSpecForShard(instance *databasev1alpha1.ShardingDatabase, Ora } // Function to build the init Container Spec -func buildInitContainerSpecForShard(instance *databasev1alpha1.ShardingDatabase, OraShardSpex databasev1alpha1.ShardSpec) []corev1.Container { +func buildInitContainerSpecForShard(instance *databasev4.ShardingDatabase, OraShardSpex databasev4.ShardSpec) []corev1.Container { var result []corev1.Container privFlag := false var uid int64 = 0 @@ -325,7 +325,7 @@ func buildInitContainerSpecForShard(instance *databasev1alpha1.ShardingDatabase, return result } -func buildVolumeMountSpecForShard(instance *databasev1alpha1.ShardingDatabase, OraShardSpex databasev1alpha1.ShardSpec) []corev1.VolumeMount { +func buildVolumeMountSpecForShard(instance *databasev4.ShardingDatabase, OraShardSpex databasev4.ShardSpec) []corev1.VolumeMount { var result []corev1.VolumeMount result = append(result, corev1.VolumeMount{Name: OraShardSpex.Name + "secretmap-vol3", MountPath: oraSecretMount, ReadOnly: true}) result = append(result, corev1.VolumeMount{Name: OraShardSpex.Name + "-oradata-vol4", MountPath: oraDataMount}) @@ -351,7 +351,7 @@ func buildVolumeMountSpecForShard(instance *databasev1alpha1.ShardingDatabase, O return result } -func volumeClaimTemplatesForShard(instance *databasev1alpha1.ShardingDatabase, OraShardSpex databasev1alpha1.ShardSpec) []corev1.PersistentVolumeClaim { +func volumeClaimTemplatesForShard(instance *databasev4.ShardingDatabase, OraShardSpex databasev4.ShardSpec) []corev1.PersistentVolumeClaim { var claims []corev1.PersistentVolumeClaim @@ -363,9 +363,9 @@ func volumeClaimTemplatesForShard(instance *databasev1alpha1.ShardingDatabase, O { ObjectMeta: metav1.ObjectMeta{ Name: OraShardSpex.Name + "-oradata-vol4", - Namespace: instance.Spec.Namespace, + Namespace: instance.Namespace, OwnerReferences: getOwnerRef(instance), - Labels: buildLabelsForShard(instance, "sharding"), + Labels: buildLabelsForShard(instance, "sharding", OraShardSpex.Name), }, Spec: corev1.PersistentVolumeClaimSpec{ AccessModes: []corev1.PersistentVolumeAccessMode{ @@ -395,7 +395,7 @@ func volumeClaimTemplatesForShard(instance *databasev1alpha1.ShardingDatabase, O return claims } -func BuildServiceDefForShard(instance *databasev1alpha1.ShardingDatabase, replicaCount int32, OraShardSpex databasev1alpha1.ShardSpec, svctype string) *corev1.Service { +func BuildServiceDefForShard(instance *databasev4.ShardingDatabase, replicaCount int32, OraShardSpex databasev4.ShardSpec, svctype string) *corev1.Service { //service := &corev1.Service{} service := &corev1.Service{ ObjectMeta: buildSvcObjectMetaForShard(instance, replicaCount, OraShardSpex, svctype), @@ -410,7 +410,7 @@ func BuildServiceDefForShard(instance *databasev1alpha1.ShardingDatabase, replic if svctype == "local" { service.Spec.ClusterIP = corev1.ClusterIPNone - service.Spec.Selector = buildLabelsForShard(instance, "sharding") + service.Spec.Selector = getSvcLabelsForShard(replicaCount, OraShardSpex) } // build Service Ports Specs to be exposed. If the PortMappings is not set then default ports will be exposed. @@ -419,7 +419,7 @@ func BuildServiceDefForShard(instance *databasev1alpha1.ShardingDatabase, replic } // Function to build Service ObjectMeta -func buildSvcObjectMetaForShard(instance *databasev1alpha1.ShardingDatabase, replicaCount int32, OraShardSpex databasev1alpha1.ShardSpec, svctype string) metav1.ObjectMeta { +func buildSvcObjectMetaForShard(instance *databasev4.ShardingDatabase, replicaCount int32, OraShardSpex databasev4.ShardSpec, svctype string) metav1.ObjectMeta { // building objectMeta var svcName string @@ -434,14 +434,14 @@ func buildSvcObjectMetaForShard(instance *databasev1alpha1.ShardingDatabase, rep objmeta := metav1.ObjectMeta{ Name: svcName, - Namespace: instance.Spec.Namespace, - Labels: buildLabelsForShard(instance, "sharding"), + Namespace: instance.Namespace, + Labels: buildLabelsForShard(instance, "sharding", OraShardSpex.Name), OwnerReferences: getOwnerRef(instance), } return objmeta } -func getSvcLabelsForShard(replicaCount int32, OraShardSpex databasev1alpha1.ShardSpec) map[string]string { +func getSvcLabelsForShard(replicaCount int32, OraShardSpex databasev4.ShardSpec) map[string]string { var labelStr map[string]string = make(map[string]string) if replicaCount == -1 { @@ -455,7 +455,7 @@ func getSvcLabelsForShard(replicaCount int32, OraShardSpex databasev1alpha1.Shar } // ======================== Update Section ======================== -func UpdateProvForShard(instance *databasev1alpha1.ShardingDatabase, OraShardSpex databasev1alpha1.ShardSpec, kClient client.Client, sfSet *appsv1.StatefulSet, shardPod *corev1.Pod, logger logr.Logger, +func UpdateProvForShard(instance *databasev4.ShardingDatabase, OraShardSpex databasev4.ShardSpec, kClient client.Client, sfSet *appsv1.StatefulSet, shardPod *corev1.Pod, logger logr.Logger, ) (ctrl.Result, error) { var msg string var size int32 = 1 diff --git a/config/crd/bases/database.oracle.com_autonomouscontainerdatabases.yaml b/config/crd/bases/database.oracle.com_autonomouscontainerdatabases.yaml index bac3a28c..1e078b63 100644 --- a/config/crd/bases/database.oracle.com_autonomouscontainerdatabases.yaml +++ b/config/crd/bases/database.oracle.com_autonomouscontainerdatabases.yaml @@ -1,11 +1,9 @@ - --- apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - controller-gen.kubebuilder.io/version: v0.6.1 - creationTimestamp: null + controller-gen.kubebuilder.io/version: v0.16.5 name: autonomouscontainerdatabases.database.oracle.com spec: group: database.oracle.com @@ -32,24 +30,14 @@ spec: name: v1alpha1 schema: openAPIV3Schema: - description: AutonomousContainerDatabase is the Schema for the autonomouscontainerdatabases - API properties: apiVersion: - description: 'APIVersion defines the versioned schema of this representation - of an object. Servers should convert recognized schemas to the latest - internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: - description: 'Kind is a string value representing the REST resource this - object represents. Servers may infer this from the endpoint the client - submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: - description: AutonomousContainerDatabaseSpec defines the desired state - of AutonomousContainerDatabase properties: action: enum: @@ -58,8 +46,6 @@ spec: - TERMINATE type: string autonomousContainerDatabaseOCID: - description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster - Important: Run "make" to regenerate code after modifying this file' type: string autonomousExadataVMClusterOCID: type: string @@ -75,7 +61,6 @@ spec: default: false type: boolean ociConfig: - description: "*********************** *\tOCI config ***********************" properties: configMapName: type: string @@ -83,21 +68,84 @@ spec: type: string type: object patchModel: - description: 'AutonomousContainerDatabasePatchModelEnum Enum with - underlying type: string' enum: - RELEASE_UPDATES - RELEASE_UPDATE_REVISIONS type: string type: object status: - description: AutonomousContainerDatabaseStatus defines the observed state - of AutonomousContainerDatabase properties: lifecycleState: - description: 'INSERT ADDITIONAL STATUS FIELD - define observed state - of cluster Important: Run "make" to regenerate code after modifying - this file' + type: string + timeCreated: + type: string + required: + - lifecycleState + type: object + type: object + served: true + storage: false + subresources: + status: {} + - additionalPrinterColumns: + - jsonPath: .spec.displayName + name: DisplayName + type: string + - jsonPath: .status.lifecycleState + name: State + type: string + - jsonPath: .status.timeCreated + name: Created + type: string + name: v4 + schema: + openAPIV3Schema: + properties: + apiVersion: + type: string + kind: + type: string + metadata: + type: object + spec: + properties: + action: + enum: + - SYNC + - RESTART + - TERMINATE + type: string + autonomousContainerDatabaseOCID: + type: string + autonomousExadataVMClusterOCID: + type: string + compartmentOCID: + type: string + displayName: + type: string + freeformTags: + additionalProperties: + type: string + type: object + hardLink: + default: false + type: boolean + ociConfig: + properties: + configMapName: + type: string + secretName: + type: string + type: object + patchModel: + enum: + - RELEASE_UPDATES + - RELEASE_UPDATE_REVISIONS + type: string + type: object + status: + properties: + lifecycleState: type: string timeCreated: type: string @@ -109,9 +157,3 @@ spec: storage: true subresources: status: {} -status: - acceptedNames: - kind: "" - plural: "" - conditions: [] - storedVersions: [] diff --git a/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml b/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml index a5c37507..b0d6f8ed 100644 --- a/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml +++ b/config/crd/bases/database.oracle.com_autonomousdatabasebackups.yaml @@ -1,11 +1,9 @@ - --- apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - controller-gen.kubebuilder.io/version: v0.6.1 - creationTimestamp: null + controller-gen.kubebuilder.io/version: v0.16.5 name: autonomousdatabasebackups.database.oracle.com spec: group: database.oracle.com @@ -38,24 +36,14 @@ spec: name: v1alpha1 schema: openAPIV3Schema: - description: AutonomousDatabaseBackup is the Schema for the autonomousdatabasebackups - API properties: apiVersion: - description: 'APIVersion defines the versioned schema of this representation - of an object. Servers should convert recognized schemas to the latest - internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: - description: 'Kind is a string value representing the REST resource this - object represents. Servers may infer this from the endpoint the client - submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: - description: AutonomousDatabaseBackupSpec defines the desired state of - AutonomousDatabaseBackup properties: autonomousDatabaseBackupOCID: type: string @@ -64,7 +52,6 @@ spec: isLongTermBackup: type: boolean ociConfig: - description: "*********************** *\tOCI config ***********************" properties: configMapName: type: string @@ -74,11 +61,8 @@ spec: retentionPeriodInDays: type: integer target: - description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster - Important: Run "make" to regenerate code after modifying this file' properties: k8sADB: - description: "*********************** *\tADB spec ***********************" properties: name: type: string @@ -91,8 +75,6 @@ spec: type: object type: object status: - description: AutonomousDatabaseBackupStatus defines the observed state - of AutonomousDatabaseBackup properties: autonomousDatabaseOCID: type: string @@ -105,16 +87,103 @@ spec: isAutomatic: type: boolean lifecycleState: - description: 'AutonomousDatabaseBackupLifecycleStateEnum Enum with - underlying type: string' type: string timeEnded: type: string timeStarted: type: string type: - description: 'AutonomousDatabaseBackupTypeEnum Enum with underlying - type: string' + type: string + required: + - autonomousDatabaseOCID + - compartmentOCID + - dbDisplayName + - dbName + - isAutomatic + - lifecycleState + - type + type: object + type: object + served: true + storage: false + subresources: + status: {} + - additionalPrinterColumns: + - jsonPath: .status.lifecycleState + name: State + type: string + - jsonPath: .status.dbDisplayName + name: DB DisplayName + type: string + - jsonPath: .status.type + name: Type + type: string + - jsonPath: .status.timeStarted + name: Started + type: string + - jsonPath: .status.timeEnded + name: Ended + type: string + name: v4 + schema: + openAPIV3Schema: + properties: + apiVersion: + type: string + kind: + type: string + metadata: + type: object + spec: + properties: + autonomousDatabaseBackupOCID: + type: string + displayName: + type: string + isLongTermBackup: + type: boolean + ociConfig: + properties: + configMapName: + type: string + secretName: + type: string + type: object + retentionPeriodInDays: + type: integer + target: + properties: + k8sADB: + properties: + name: + type: string + type: object + ociADB: + properties: + ocid: + type: string + type: object + type: object + type: object + status: + properties: + autonomousDatabaseOCID: + type: string + compartmentOCID: + type: string + dbDisplayName: + type: string + dbName: + type: string + isAutomatic: + type: boolean + lifecycleState: + type: string + timeEnded: + type: string + timeStarted: + type: string + type: type: string required: - autonomousDatabaseOCID @@ -130,9 +199,3 @@ spec: storage: true subresources: status: {} -status: - acceptedNames: - kind: "" - plural: "" - conditions: [] - storedVersions: [] diff --git a/config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml b/config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml index 5e9f2c73..3bfc5a4e 100644 --- a/config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml +++ b/config/crd/bases/database.oracle.com_autonomousdatabaserestores.yaml @@ -1,11 +1,9 @@ - --- apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - controller-gen.kubebuilder.io/version: v0.6.1 - creationTimestamp: null + controller-gen.kubebuilder.io/version: v0.16.5 name: autonomousdatabaserestores.database.oracle.com spec: group: database.oracle.com @@ -32,27 +30,16 @@ spec: name: v1alpha1 schema: openAPIV3Schema: - description: AutonomousDatabaseRestore is the Schema for the autonomousdatabaserestores - API properties: apiVersion: - description: 'APIVersion defines the versioned schema of this representation - of an object. Servers should convert recognized schemas to the latest - internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: - description: 'Kind is a string value representing the REST resource this - object represents. Servers may infer this from the endpoint the client - submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: - description: AutonomousDatabaseRestoreSpec defines the desired state of - AutonomousDatabaseRestore properties: ociConfig: - description: "*********************** *\tOCI config ***********************" properties: configMapName: type: string @@ -62,9 +49,6 @@ spec: source: properties: k8sADBBackup: - description: 'EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO - OWN! NOTE: json tags are required. Any new fields you add must - have json tags for the fields to be serialized.' properties: name: type: string @@ -72,17 +56,12 @@ spec: pointInTime: properties: timestamp: - description: 'The timestamp must follow this format: YYYY-MM-DD - HH:MM:SS GMT' type: string type: object type: object target: - description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster - Important: Run "make" to regenerate code after modifying this file' properties: k8sADB: - description: "*********************** *\tADB spec ***********************" properties: name: type: string @@ -98,18 +77,98 @@ spec: - target type: object status: - description: AutonomousDatabaseRestoreStatus defines the observed state - of AutonomousDatabaseRestore properties: dbName: type: string displayName: - description: 'INSERT ADDITIONAL STATUS FIELD - define observed state - of cluster Important: Run "make" to regenerate code after modifying - this file' type: string status: - description: 'WorkRequestStatusEnum Enum with underlying type: string' + type: string + timeAccepted: + type: string + timeEnded: + type: string + timeStarted: + type: string + workRequestOCID: + type: string + required: + - dbName + - displayName + - status + - workRequestOCID + type: object + type: object + served: true + storage: false + subresources: + status: {} + - additionalPrinterColumns: + - jsonPath: .status.status + name: Status + type: string + - jsonPath: .status.displayName + name: DbDisplayName + type: string + - jsonPath: .status.dbName + name: DbName + type: string + name: v4 + schema: + openAPIV3Schema: + properties: + apiVersion: + type: string + kind: + type: string + metadata: + type: object + spec: + properties: + ociConfig: + properties: + configMapName: + type: string + secretName: + type: string + type: object + source: + properties: + k8sADBBackup: + properties: + name: + type: string + type: object + pointInTime: + properties: + timestamp: + type: string + type: object + type: object + target: + properties: + k8sADB: + properties: + name: + type: string + type: object + ociADB: + properties: + ocid: + type: string + type: object + type: object + required: + - source + - target + type: object + status: + properties: + dbName: + type: string + displayName: + type: string + status: type: string timeAccepted: type: string @@ -130,9 +189,3 @@ spec: storage: true subresources: status: {} -status: - acceptedNames: - kind: "" - plural: "" - conditions: [] - storedVersions: [] diff --git a/config/crd/bases/database.oracle.com_autonomousdatabases.yaml b/config/crd/bases/database.oracle.com_autonomousdatabases.yaml index f77407f3..1672ae81 100644 --- a/config/crd/bases/database.oracle.com_autonomousdatabases.yaml +++ b/config/crd/bases/database.oracle.com_autonomousdatabases.yaml @@ -1,11 +1,9 @@ - --- apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - controller-gen.kubebuilder.io/version: v0.6.1 - creationTimestamp: null + controller-gen.kubebuilder.io/version: v0.16.5 name: autonomousdatabases.database.oracle.com spec: group: database.oracle.com @@ -47,63 +45,400 @@ spec: name: v1alpha1 schema: openAPIV3Schema: - description: AutonomousDatabase is the Schema for the autonomousdatabases - API properties: apiVersion: - description: 'APIVersion defines the versioned schema of this representation - of an object. Servers should convert recognized schemas to the latest - internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: - description: 'Kind is a string value representing the REST resource this - object represents. Servers may infer this from the endpoint the client - submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: - description: 'AutonomousDatabaseSpec defines the desired state of AutonomousDatabase - Important: Run "make" to regenerate code after modifying this file' properties: + action: + enum: + - "" + - Create + - Sync + - Update + - Stop + - Start + - Terminate + - Clone + type: string + clone: + properties: + adminPassword: + properties: + k8sSecret: + properties: + name: + type: string + type: object + ociSecret: + properties: + id: + type: string + type: object + type: object + autonomousContainerDatabase: + properties: + k8sAcd: + properties: + name: + type: string + type: object + ociAcd: + properties: + id: + type: string + type: object + type: object + cloneType: + enum: + - FULL + - METADATA + type: string + compartmentId: + type: string + computeCount: + type: number + computeModel: + enum: + - ECPU + - OCPU + type: string + cpuCoreCount: + type: integer + dataStorageSizeInTBs: + type: integer + dbName: + type: string + dbVersion: + type: string + dbWorkload: + enum: + - OLTP + - DW + - AJD + - APEX + type: string + displayName: + type: string + freeformTags: + additionalProperties: + type: string + type: object + isAccessControlEnabled: + type: boolean + isAutoScalingEnabled: + type: boolean + isDedicated: + type: boolean + isFreeTier: + type: boolean + isMtlsConnectionRequired: + type: boolean + licenseModel: + enum: + - LICENSE_INCLUDED + - BRING_YOUR_OWN_LICENSE + type: string + nsgIds: + items: + type: string + type: array + ocpuCount: + type: number + privateEndpointLabel: + type: string + subnetId: + type: string + whitelistedIps: + items: + type: string + type: array + type: object details: - description: AutonomousDatabaseDetails defines the detail information - of AutonomousDatabase, corresponding to oci-go-sdk/database/AutonomousDatabase properties: adminPassword: properties: k8sSecret: - description: "*********************** *\tSecret specs ***********************" properties: name: type: string type: object ociSecret: properties: - ocid: + id: + type: string + type: object + type: object + autonomousContainerDatabase: + properties: + k8sAcd: + properties: + name: + type: string + type: object + ociAcd: + properties: + id: + type: string + type: object + type: object + compartmentId: + type: string + computeCount: + type: number + computeModel: + enum: + - ECPU + - OCPU + type: string + cpuCoreCount: + type: integer + dataStorageSizeInTBs: + type: integer + dbName: + type: string + dbVersion: + type: string + dbWorkload: + enum: + - OLTP + - DW + - AJD + - APEX + type: string + displayName: + type: string + freeformTags: + additionalProperties: + type: string + type: object + id: + type: string + isAccessControlEnabled: + type: boolean + isAutoScalingEnabled: + type: boolean + isDedicated: + type: boolean + isFreeTier: + type: boolean + isMtlsConnectionRequired: + type: boolean + licenseModel: + enum: + - LICENSE_INCLUDED + - BRING_YOUR_OWN_LICENSE + type: string + nsgIds: + items: + type: string + type: array + ocpuCount: + type: number + privateEndpointLabel: + type: string + subnetId: + type: string + whitelistedIps: + items: + type: string + type: array + type: object + hardLink: + default: false + type: boolean + ociConfig: + properties: + configMapName: + type: string + secretName: + type: string + type: object + wallet: + properties: + name: + type: string + password: + properties: + k8sSecret: + properties: + name: + type: string + type: object + ociSecret: + properties: + id: + type: string + type: object + type: object + type: object + required: + - action + type: object + status: + properties: + allConnectionStrings: + items: + properties: + connectionStrings: + items: + properties: + connectionString: + type: string + tnsName: + type: string + type: object + type: array + tlsAuthentication: + type: string + required: + - connectionStrings + type: object + type: array + conditions: + items: + properties: + lastTransitionTime: + format: date-time + type: string + message: + maxLength: 32768 + type: string + observedGeneration: + format: int64 + minimum: 0 + type: integer + reason: + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + enum: + - "True" + - "False" + - Unknown + type: string + type: + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + lifecycleState: + type: string + timeCreated: + type: string + walletExpiringDate: + type: string + type: object + type: object + served: true + storage: false + subresources: + status: {} + - additionalPrinterColumns: + - jsonPath: .spec.details.displayName + name: Display Name + type: string + - jsonPath: .spec.details.dbName + name: Db Name + type: string + - jsonPath: .status.lifecycleState + name: State + type: string + - jsonPath: .spec.details.isDedicated + name: Dedicated + type: string + - jsonPath: .spec.details.cpuCoreCount + name: OCPUs + type: integer + - jsonPath: .spec.details.dataStorageSizeInTBs + name: Storage (TB) + type: integer + - jsonPath: .spec.details.dbWorkload + name: Workload Type + type: string + - jsonPath: .status.timeCreated + name: Created + type: string + name: v4 + schema: + openAPIV3Schema: + properties: + apiVersion: + type: string + kind: + type: string + metadata: + type: object + spec: + properties: + action: + enum: + - "" + - Create + - Sync + - Update + - Stop + - Start + - Terminate + - Clone + type: string + clone: + properties: + adminPassword: + properties: + k8sSecret: + properties: + name: + type: string + type: object + ociSecret: + properties: + id: type: string type: object type: object autonomousContainerDatabase: - description: ACDSpec defines the spec of the target for backup/restore - runs. The name could be the name of an AutonomousDatabase or - an AutonomousDatabaseBackup properties: - k8sACD: - description: "*********************** *\tACD specs ***********************" + k8sAcd: properties: name: type: string type: object - ociACD: + ociAcd: properties: - ocid: + id: type: string type: object type: object - autonomousDatabaseOCID: + cloneType: + enum: + - FULL + - METADATA type: string - compartmentOCID: + compartmentId: + type: string + computeCount: + type: number + computeModel: + enum: + - ECPU + - OCPU type: string cpuCoreCount: type: integer @@ -114,8 +449,6 @@ spec: dbVersion: type: string dbWorkload: - description: 'AutonomousDatabaseDbWorkloadEnum Enum with underlying - type: string' enum: - OLTP - DW @@ -128,87 +461,158 @@ spec: additionalProperties: type: string type: object + isAccessControlEnabled: + type: boolean isAutoScalingEnabled: type: boolean isDedicated: type: boolean + isFreeTier: + type: boolean + isMtlsConnectionRequired: + type: boolean licenseModel: - description: 'AutonomousDatabaseLicenseModelEnum Enum with underlying - type: string' enum: - LICENSE_INCLUDED - BRING_YOUR_OWN_LICENSE type: string - lifecycleState: - description: 'AutonomousDatabaseLifecycleStateEnum Enum with underlying - type: string' + nsgIds: + items: + type: string + type: array + ocpuCount: + type: number + privateEndpointLabel: + type: string + subnetId: type: string - networkAccess: + whitelistedIps: + items: + type: string + type: array + type: object + details: + properties: + adminPassword: properties: - accessControlList: - items: - type: string - type: array - accessType: - enum: - - "" - - PUBLIC - - RESTRICTED - - PRIVATE - type: string - isAccessControlEnabled: - type: boolean - isMTLSConnectionRequired: - type: boolean - privateEndpoint: + k8sSecret: properties: - hostnamePrefix: + name: type: string - nsgOCIDs: - items: - type: string - type: array - subnetOCID: + type: object + ociSecret: + properties: + id: type: string type: object type: object - wallet: + autonomousContainerDatabase: properties: - name: - type: string - password: + k8sAcd: properties: - k8sSecret: - description: "*********************** *\tSecret specs - ***********************" - properties: - name: - type: string - type: object - ociSecret: - properties: - ocid: - type: string - type: object + name: + type: string type: object + ociAcd: + properties: + id: + type: string + type: object + type: object + compartmentId: + type: string + computeCount: + type: number + computeModel: + enum: + - ECPU + - OCPU + type: string + cpuCoreCount: + type: integer + dataStorageSizeInTBs: + type: integer + dbName: + type: string + dbVersion: + type: string + dbWorkload: + enum: + - OLTP + - DW + - AJD + - APEX + type: string + displayName: + type: string + freeformTags: + additionalProperties: + type: string type: object + id: + type: string + isAccessControlEnabled: + type: boolean + isAutoScalingEnabled: + type: boolean + isDedicated: + type: boolean + isFreeTier: + type: boolean + isMtlsConnectionRequired: + type: boolean + licenseModel: + enum: + - LICENSE_INCLUDED + - BRING_YOUR_OWN_LICENSE + type: string + nsgIds: + items: + type: string + type: array + ocpuCount: + type: number + privateEndpointLabel: + type: string + subnetId: + type: string + whitelistedIps: + items: + type: string + type: array type: object hardLink: default: false type: boolean ociConfig: - description: "*********************** *\tOCI config ***********************" properties: configMapName: type: string secretName: type: string type: object + wallet: + properties: + name: + type: string + password: + properties: + k8sSecret: + properties: + name: + type: string + type: object + ociSecret: + properties: + id: + type: string + type: object + type: object + type: object required: - - details + - action type: object status: - description: AutonomousDatabaseStatus defines the observed state of AutonomousDatabase properties: allConnectionStrings: items: @@ -230,63 +634,29 @@ spec: type: array conditions: items: - description: "Condition contains details for one aspect of the current - state of this API Resource. --- This struct is intended for direct - use as an array at the field path .status.conditions. For example, - \n \ttype FooStatus struct{ \t // Represents the observations - of a foo's current state. \t // Known .status.conditions.type - are: \"Available\", \"Progressing\", and \"Degraded\" \t // - +patchMergeKey=type \t // +patchStrategy=merge \t // +listType=map - \t // +listMapKey=type \t Conditions []metav1.Condition - `json:\"conditions,omitempty\" patchStrategy:\"merge\" patchMergeKey:\"type\" - protobuf:\"bytes,1,rep,name=conditions\"` \n \t // other fields - \t}" properties: lastTransitionTime: - description: lastTransitionTime is the last time the condition - transitioned from one status to another. This should be when - the underlying condition changed. If that is not known, then - using the time when the API field changed is acceptable. format: date-time type: string message: - description: message is a human readable message indicating - details about the transition. This may be an empty string. maxLength: 32768 type: string observedGeneration: - description: observedGeneration represents the .metadata.generation - that the condition was set based upon. For instance, if .metadata.generation - is currently 12, but the .status.conditions[x].observedGeneration - is 9, the condition is out of date with respect to the current - state of the instance. format: int64 minimum: 0 type: integer reason: - description: reason contains a programmatic identifier indicating - the reason for the condition's last transition. Producers - of specific condition types may define expected values and - meanings for this field, and whether the values are considered - a guaranteed API. The value should be a CamelCase string. - This field may not be empty. maxLength: 1024 minLength: 1 pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ type: string status: - description: status of the condition, one of True, False, Unknown. enum: - "True" - "False" - Unknown type: string type: - description: type of condition in CamelCase or in foo.example.com/CamelCase. - --- Many .condition.type values are consistent across resources - like Available, but because arbitrary conditions can be useful - (see .node.status.conditions), the ability to deconflict is - important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) maxLength: 316 pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ type: string @@ -302,9 +672,6 @@ spec: - type x-kubernetes-list-type: map lifecycleState: - description: 'INSERT ADDITIONAL STATUS FIELD - define observed state - of cluster Important: Run "make" to regenerate code after modifying - this file' type: string timeCreated: type: string @@ -316,9 +683,3 @@ spec: storage: true subresources: status: {} -status: - acceptedNames: - kind: "" - plural: "" - conditions: [] - storedVersions: [] diff --git a/config/crd/bases/database.oracle.com_cdbs.yaml b/config/crd/bases/database.oracle.com_cdbs.yaml index 6b1c350c..8ea594e6 100644 --- a/config/crd/bases/database.oracle.com_cdbs.yaml +++ b/config/crd/bases/database.oracle.com_cdbs.yaml @@ -1,11 +1,9 @@ - --- apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - controller-gen.kubebuilder.io/version: v0.6.1 - creationTimestamp: null + controller-gen.kubebuilder.io/version: v0.16.5 name: cdbs.database.oracle.com spec: group: database.oracle.com @@ -29,10 +27,6 @@ spec: jsonPath: .spec.dbPort name: DB Port type: integer - - description: ' string of the tnsalias' - jsonPath: .spec.dbTnsurl - name: TNS STRING - type: string - description: Replicas jsonPath: .spec.replicas name: Replicas @@ -45,31 +39,25 @@ spec: jsonPath: .status.msg name: Message type: string - name: v1alpha1 + - description: ' string of the tnsalias' + jsonPath: .spec.dbTnsurl + name: TNS STRING + type: string + name: v4 schema: openAPIV3Schema: - description: CDB is the Schema for the cdbs API properties: apiVersion: - description: 'APIVersion defines the versioned schema of this representation - of an object. Servers should convert recognized schemas to the latest - internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: - description: 'Kind is a string value representing the REST resource this - object represents. Servers may infer this from the endpoint the client - submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: - description: CDBSpec defines the desired state of CDB properties: cdbAdminPwd: - description: Password for the CDB Administrator to manage PDB lifecycle properties: secret: - description: CDBSecret defines the secretName properties: key: type: string @@ -83,11 +71,8 @@ spec: - secret type: object cdbAdminUser: - description: User in the root container with sysdba priviledges to - manage PDB lifecycle properties: secret: - description: CDBSecret defines the secretName properties: key: type: string @@ -101,12 +86,40 @@ spec: - secret type: object cdbName: - description: Name of the CDB type: string + cdbOrdsPrvKey: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + cdbOrdsPubKey: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object cdbTlsCrt: properties: secret: - description: CDBSecret defines the secretName properties: key: type: string @@ -122,7 +135,6 @@ spec: cdbTlsKey: properties: secret: - description: CDBSecret defines the secretName properties: key: type: string @@ -136,40 +148,31 @@ spec: - secret type: object dbPort: - description: DB server port type: integer dbServer: - description: Name of the DB server type: string dbTnsurl: type: string + deletePdbCascade: + type: boolean nodeSelector: additionalProperties: type: string - description: Node Selector for running the Pod type: object ordsImage: - description: ORDS Image Name type: string ordsImagePullPolicy: - description: ORDS Image Pull Policy enum: - Always - Never type: string ordsImagePullSecret: - description: The name of the image pull secret in case of a private - docker repository. type: string ordsPort: - description: ORDS server port. For now, keep it as 8888. TO BE USED - IN FUTURE RELEASE. type: integer ordsPwd: - description: Password for user ORDS_PUBLIC_USER properties: secret: - description: CDBSecret defines the secretName properties: key: type: string @@ -183,16 +186,12 @@ spec: - secret type: object replicas: - description: Number of ORDS Containers to create type: integer serviceName: - description: Name of the CDB Service type: string sysAdminPwd: - description: Password for the CDB System Administrator properties: secret: - description: CDBSecret defines the secretName properties: key: type: string @@ -206,10 +205,8 @@ spec: - secret type: object webServerPwd: - description: Password for the Web Server User properties: secret: - description: CDBSecret defines the secretName properties: key: type: string @@ -223,11 +220,8 @@ spec: - secret type: object webServerUser: - description: Web Server User with SQL Administrator role to allow - us to authenticate to the PDB Lifecycle Management REST endpoints properties: secret: - description: CDBSecret defines the secretName properties: key: type: string @@ -242,16 +236,12 @@ spec: type: object type: object status: - description: CDBStatus defines the observed state of CDB properties: msg: - description: Message type: string phase: - description: Phase of the CDB Resource type: string status: - description: CDB Resource Status type: boolean required: - phase @@ -262,9 +252,3 @@ spec: storage: true subresources: status: {} -status: - acceptedNames: - kind: "" - plural: "" - conditions: [] - storedVersions: [] diff --git a/config/crd/bases/database.oracle.com_dataguardbrokers.yaml b/config/crd/bases/database.oracle.com_dataguardbrokers.yaml index f19a3e22..5efceff4 100644 --- a/config/crd/bases/database.oracle.com_dataguardbrokers.yaml +++ b/config/crd/bases/database.oracle.com_dataguardbrokers.yaml @@ -1,11 +1,9 @@ - --- apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - controller-gen.kubebuilder.io/version: v0.6.1 - creationTimestamp: null + controller-gen.kubebuilder.io/version: v0.16.5 name: dataguardbrokers.database.oracle.com spec: group: database.oracle.com @@ -40,41 +38,120 @@ spec: - jsonPath: .status.status name: Status type: string + - jsonPath: .status.fastStartFailover + name: FSFO + type: string name: v1alpha1 schema: openAPIV3Schema: - description: DataguardBroker is the Schema for the dataguardbrokers API properties: apiVersion: - description: 'APIVersion defines the versioned schema of this representation - of an object. Servers should convert recognized schemas to the latest - internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: - description: 'Kind is a string value representing the REST resource this - object represents. Servers may infer this from the endpoint the client - submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: - description: DataguardBrokerSpec defines the desired state of DataguardBroker properties: - fastStartFailOver: - properties: - enable: - type: boolean - strategy: - items: - description: FSFO strategy - properties: - sourceDatabaseRef: - type: string - targetDatabaseRefs: - type: string - type: object - type: array + fastStartFailover: + type: boolean + loadBalancer: + type: boolean + nodeSelector: + additionalProperties: + type: string + type: object + primaryDatabaseRef: + type: string + protectionMode: + enum: + - MaxPerformance + - MaxAvailability + type: string + serviceAnnotations: + additionalProperties: + type: string + type: object + setAsPrimaryDatabase: + type: string + standbyDatabaseRefs: + items: + type: string + type: array + required: + - primaryDatabaseRef + - protectionMode + - standbyDatabaseRefs + type: object + status: + properties: + clusterConnectString: + type: string + databasesInDataguardConfig: + additionalProperties: + type: string type: object + externalConnectString: + type: string + fastStartFailover: + type: boolean + primaryDatabase: + type: string + primaryDatabaseRef: + type: string + protectionMode: + type: string + standbyDatabases: + type: string + status: + type: string + type: object + type: object + served: true + storage: false + subresources: + status: {} + - additionalPrinterColumns: + - jsonPath: .status.primaryDatabase + name: Primary + type: string + - jsonPath: .status.standbyDatabases + name: Standbys + type: string + - jsonPath: .spec.protectionMode + name: Protection Mode + type: string + - jsonPath: .status.clusterConnectString + name: Cluster Connect Str + priority: 1 + type: string + - jsonPath: .status.externalConnectString + name: Connect Str + type: string + - jsonPath: .spec.primaryDatabaseRef + name: Primary Database + priority: 1 + type: string + - jsonPath: .status.status + name: Status + type: string + - jsonPath: .status.fastStartFailover + name: FSFO + type: string + name: v4 + schema: + openAPIV3Schema: + properties: + apiVersion: + type: string + kind: + type: string + metadata: + type: object + spec: + properties: + fastStartFailover: + type: boolean loadBalancer: type: boolean nodeSelector: @@ -104,12 +181,17 @@ spec: - standbyDatabaseRefs type: object status: - description: DataguardBrokerStatus defines the observed state of DataguardBroker properties: clusterConnectString: type: string + databasesInDataguardConfig: + additionalProperties: + type: string + type: object externalConnectString: type: string + fastStartFailover: + type: boolean primaryDatabase: type: string primaryDatabaseRef: @@ -126,9 +208,3 @@ spec: storage: true subresources: status: {} -status: - acceptedNames: - kind: "" - plural: "" - conditions: [] - storedVersions: [] diff --git a/config/crd/bases/database.oracle.com_dbcssystems.yaml b/config/crd/bases/database.oracle.com_dbcssystems.yaml index 3f4b1c46..468d7612 100644 --- a/config/crd/bases/database.oracle.com_dbcssystems.yaml +++ b/config/crd/bases/database.oracle.com_dbcssystems.yaml @@ -1,11 +1,9 @@ - --- apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - controller-gen.kubebuilder.io/version: v0.6.1 - creationTimestamp: null + controller-gen.kubebuilder.io/version: v0.16.5 name: dbcssystems.database.oracle.com spec: group: database.oracle.com @@ -19,23 +17,60 @@ spec: - name: v1alpha1 schema: openAPIV3Schema: - description: DbcsSystem is the Schema for the dbcssystems API properties: apiVersion: - description: 'APIVersion defines the versioned schema of this representation - of an object. Servers should convert recognized schemas to the latest - internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: - description: 'Kind is a string value representing the REST resource this - object represents. Servers may infer this from the endpoint the client - submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: - description: DbcsSystemSpec defines the desired state of DbcsSystem properties: + databaseId: + type: string + dbBackupId: + type: string + dbClone: + properties: + dbAdminPaswordSecret: + type: string + dbDbUniqueName: + type: string + dbName: + type: string + displayName: + type: string + domain: + type: string + hostName: + type: string + initialDataStorageSizeInGB: + type: integer + kmsKeyId: + type: string + kmsKeyVersionId: + type: string + licenseModel: + type: string + privateIp: + type: string + sidPrefix: + type: string + sshPublicKeys: + items: + type: string + type: array + subnetId: + type: string + tdeWalletPasswordSecret: + type: string + required: + - dbDbUniqueName + - dbName + - displayName + - hostName + - subnetId + type: object dbSystem: properties: availabilityDomain: @@ -51,7 +86,6 @@ spec: dbAdminPaswordSecret: type: string dbBackupConfig: - description: DB Backup COnfig Network Struct properties: autoBackupEnabled: type: boolean @@ -88,12 +122,394 @@ spec: type: string initialDataStorageSizeInGB: type: integer + kmsConfig: + properties: + compartmentId: + type: string + encryptionAlgo: + type: string + keyName: + type: string + vaultName: + type: string + vaultType: + type: string + type: object + licenseModel: + type: string + nodeCount: + type: integer + pdbName: + type: string + privateIp: + type: string + shape: + type: string + sshPublicKeys: + items: + type: string + type: array + storageManagement: + type: string + subnetId: + type: string + tags: + additionalProperties: + type: string + type: object + tdeWalletPasswordSecret: + type: string + timeZone: + type: string + required: + - availabilityDomain + - compartmentId + - dbAdminPaswordSecret + - hostName + - shape + - subnetId + type: object + hardLink: + type: boolean + id: + type: string + kmsConfig: + properties: + compartmentId: + type: string + encryptionAlgo: + type: string + keyName: + type: string + vaultName: + type: string + vaultType: + type: string + type: object + ociConfigMap: + type: string + ociSecret: + type: string + pdbConfigs: + items: + properties: + freeformTags: + additionalProperties: + type: string + type: object + isDelete: + type: boolean + pdbAdminPassword: + type: string + pdbName: + type: string + pluggableDatabaseId: + type: string + shouldPdbAdminAccountBeLocked: + type: boolean + tdeWalletPassword: + type: string + required: + - freeformTags + - pdbAdminPassword + - pdbName + - shouldPdbAdminAccountBeLocked + - tdeWalletPassword + type: object + type: array + setupDBCloning: + type: boolean + required: + - ociConfigMap + type: object + status: + properties: + availabilityDomain: + type: string + cpuCoreCount: + type: integer + dataStoragePercentage: + type: integer + dataStorageSizeInGBs: + type: integer + dbCloneStatus: + properties: + dbAdminPaswordSecret: + type: string + dbDbUniqueName: + type: string + dbName: + type: string + displayName: + type: string + domain: + type: string + hostName: + type: string + id: + type: string + licenseModel: + type: string + sshPublicKeys: + items: + type: string + type: array + subnetId: + type: string + required: + - dbDbUniqueName + - hostName + type: object + dbEdition: + type: string + dbInfo: + items: + properties: + dbHomeId: + type: string + dbName: + type: string + dbUniqueName: + type: string + dbWorkload: + type: string + id: + type: string + type: object + type: array + displayName: + type: string + id: + type: string + kmsDetailsStatus: + properties: + compartmentId: + type: string + encryptionAlgo: + type: string + keyId: + type: string + keyName: + type: string + managementEndpoint: + type: string + vaultId: + type: string + vaultName: + type: string + vaultType: + type: string + type: object + licenseModel: + type: string + network: + properties: + clientSubnet: + type: string + domainName: + type: string + hostName: + type: string + listenerPort: + type: integer + networkSG: + type: string + scanDnsName: + type: string + vcnName: + type: string + type: object + nodeCount: + type: integer + pdbDetailsStatus: + items: + properties: + pdbConfigStatus: + items: + properties: + freeformTags: + additionalProperties: + type: string + type: object + pdbName: + type: string + pdbState: + type: string + pluggableDatabaseId: + type: string + shouldPdbAdminAccountBeLocked: + type: boolean + required: + - freeformTags + - pdbName + - shouldPdbAdminAccountBeLocked + type: object + type: array + type: object + type: array + recoStorageSizeInGB: + type: integer + shape: + type: string + state: + type: string + storageManagement: + type: string + subnetId: + type: string + timeZone: + type: string + workRequests: + items: + properties: + operationId: + type: string + operationType: + type: string + percentComplete: + type: string + timeAccepted: + type: string + timeFinished: + type: string + timeStarted: + type: string + required: + - operationId + - operationType + type: object + type: array + required: + - state + type: object + type: object + served: true + storage: false + subresources: + status: {} + - name: v4 + schema: + openAPIV3Schema: + properties: + apiVersion: + type: string + kind: + type: string + metadata: + type: object + spec: + properties: + databaseId: + type: string + dbBackupId: + type: string + dbClone: + properties: + dbAdminPasswordSecret: + type: string + dbDbUniqueName: + type: string + dbName: + type: string + displayName: + type: string + domain: + type: string + hostName: + type: string + initialDataStorageSizeInGB: + type: integer kmsKeyId: type: string kmsKeyVersionId: type: string licenseModel: type: string + privateIp: + type: string + sidPrefix: + type: string + sshPublicKeys: + items: + type: string + type: array + subnetId: + type: string + tdeWalletPasswordSecret: + type: string + required: + - dbDbUniqueName + - dbName + - displayName + - hostName + - subnetId + type: object + dbSystem: + properties: + availabilityDomain: + type: string + backupSubnetId: + type: string + clusterName: + type: string + compartmentId: + type: string + cpuCoreCount: + type: integer + dbAdminPasswordSecret: + type: string + dbBackupConfig: + properties: + autoBackupEnabled: + type: boolean + autoBackupWindow: + type: string + backupDestinationDetails: + type: string + recoveryWindowsInDays: + type: integer + type: object + dbDomain: + type: string + dbEdition: + type: string + dbName: + type: string + dbUniqueName: + type: string + dbVersion: + type: string + dbWorkload: + type: string + diskRedundancy: + type: string + displayName: + type: string + domain: + type: string + faultDomains: + items: + type: string + type: array + hostName: + type: string + initialDataStorageSizeInGB: + type: integer + kmsConfig: + properties: + compartmentId: + type: string + encryptionAlgo: + type: string + keyName: + type: string + vaultName: + type: string + vaultType: + type: string + type: object + licenseModel: + type: string nodeCount: type: integer pdbName: @@ -121,25 +537,65 @@ spec: required: - availabilityDomain - compartmentId - - dbAdminPaswordSecret + - dbAdminPasswordSecret - hostName - shape - - sshPublicKeys - subnetId type: object hardLink: type: boolean id: type: string + kmsConfig: + properties: + compartmentId: + type: string + encryptionAlgo: + type: string + keyName: + type: string + vaultName: + type: string + vaultType: + type: string + type: object ociConfigMap: type: string ociSecret: type: string + pdbConfigs: + items: + properties: + freeformTags: + additionalProperties: + type: string + type: object + isDelete: + type: boolean + pdbAdminPassword: + type: string + pdbName: + type: string + pluggableDatabaseId: + type: string + shouldPdbAdminAccountBeLocked: + type: boolean + tdeWalletPassword: + type: string + required: + - freeformTags + - pdbAdminPassword + - pdbName + - shouldPdbAdminAccountBeLocked + - tdeWalletPassword + type: object + type: array + setupDBCloning: + type: boolean required: - ociConfigMap type: object status: - description: DbcsSystemStatus defines the observed state of DbcsSystem properties: availabilityDomain: type: string @@ -149,11 +605,38 @@ spec: type: integer dataStorageSizeInGBs: type: integer + dbCloneStatus: + properties: + dbAdminPaswordSecret: + type: string + dbDbUniqueName: + type: string + dbName: + type: string + displayName: + type: string + domain: + type: string + hostName: + type: string + id: + type: string + licenseModel: + type: string + sshPublicKeys: + items: + type: string + type: array + subnetId: + type: string + required: + - dbDbUniqueName + - hostName + type: object dbEdition: type: string dbInfo: items: - description: DbcsSystemStatus defines the observed state of DbcsSystem properties: dbHomeId: type: string @@ -171,6 +654,25 @@ spec: type: string id: type: string + kmsDetailsStatus: + properties: + compartmentId: + type: string + encryptionAlgo: + type: string + keyId: + type: string + keyName: + type: string + managementEndpoint: + type: string + vaultId: + type: string + vaultName: + type: string + vaultType: + type: string + type: object licenseModel: type: string network: @@ -192,6 +694,28 @@ spec: type: object nodeCount: type: integer + pdbDetailsStatus: + items: + properties: + pdbConfigStatus: + items: + properties: + freeformTags: + additionalProperties: + type: string + type: object + pdbName: + type: string + pdbState: + type: string + pluggableDatabaseId: + type: string + shouldPdbAdminAccountBeLocked: + type: boolean + type: object + type: array + type: object + type: array recoStorageSizeInGB: type: integer shape: @@ -232,9 +756,3 @@ spec: storage: true subresources: status: {} -status: - acceptedNames: - kind: "" - plural: "" - conditions: [] - storedVersions: [] diff --git a/config/crd/bases/database.oracle.com_lrests.yaml b/config/crd/bases/database.oracle.com_lrests.yaml new file mode 100644 index 00000000..c20356e7 --- /dev/null +++ b/config/crd/bases/database.oracle.com_lrests.yaml @@ -0,0 +1,254 @@ +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.16.5 + name: lrests.database.oracle.com +spec: + group: database.oracle.com + names: + kind: LREST + listKind: LRESTList + plural: lrests + singular: lrest + scope: Namespaced + versions: + - additionalPrinterColumns: + - description: Name of the LREST + jsonPath: .spec.cdbName + name: CDB NAME + type: string + - description: ' Name of the DB Server' + jsonPath: .spec.dbServer + name: DB Server + type: string + - description: DB server port + jsonPath: .spec.dbPort + name: DB Port + type: integer + - description: Replicas + jsonPath: .spec.replicas + name: Replicas + type: integer + - description: Status of the LREST Resource + jsonPath: .status.phase + name: Status + type: string + - description: Error message if any + jsonPath: .status.msg + name: Message + type: string + - description: string of the tnsalias + jsonPath: .spec.dbTnsurl + name: TNS STRING + type: string + name: v4 + schema: + openAPIV3Schema: + properties: + apiVersion: + type: string + kind: + type: string + metadata: + type: object + spec: + properties: + cdbAdminPwd: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + cdbAdminUser: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + cdbName: + type: string + cdbPrvKey: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + cdbPubKey: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + cdbTlsCrt: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + cdbTlsKey: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + dbPort: + type: integer + dbServer: + type: string + dbTnsurl: + type: string + deletePdbCascade: + type: boolean + lrestImage: + type: string + lrestImagePullPolicy: + enum: + - Always + - Never + type: string + lrestImagePullSecret: + type: string + lrestPort: + type: integer + lrestPwd: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + nodeSelector: + additionalProperties: + type: string + type: object + replicas: + type: integer + serviceName: + type: string + sysAdminPwd: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + webServerPwd: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + webServerUser: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + type: object + status: + properties: + msg: + type: string + phase: + type: string + status: + type: boolean + required: + - phase + - status + type: object + type: object + served: true + storage: true + subresources: + status: {} diff --git a/config/crd/bases/database.oracle.com_lrpdbs.yaml b/config/crd/bases/database.oracle.com_lrpdbs.yaml new file mode 100644 index 00000000..14ad7f29 --- /dev/null +++ b/config/crd/bases/database.oracle.com_lrpdbs.yaml @@ -0,0 +1,369 @@ +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.16.5 + name: lrpdbs.database.oracle.com +spec: + group: database.oracle.com + names: + kind: LRPDB + listKind: LRPDBList + plural: lrpdbs + singular: lrpdb + scope: Namespaced + versions: + - additionalPrinterColumns: + - description: Name of the CDB + jsonPath: .spec.cdbName + name: CDB Name + type: string + - description: Name of the PDB + jsonPath: .spec.pdbName + name: PDB Name + type: string + - description: PDB Open Mode + jsonPath: .status.openMode + name: PDB State + type: string + - description: Total Size of the PDB + jsonPath: .status.totalSize + name: PDB Size + type: string + - description: Status of the LRPDB Resource + jsonPath: .status.phase + name: Status + type: string + - description: Error message, if any + jsonPath: .status.msg + name: Message + type: string + - description: last sqlcode + jsonPath: .status.sqlCode + name: last sqlcode + type: integer + - description: The connect string to be used + jsonPath: .status.connString + name: Connect_String + type: string + name: v4 + schema: + openAPIV3Schema: + properties: + apiVersion: + type: string + kind: + type: string + metadata: + type: object + spec: + properties: + action: + enum: + - Create + - Clone + - Plug + - Unplug + - Delete + - Modify + - Status + - Map + - Alter + - Noaction + type: string + adminName: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + adminPwd: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + adminpdbPass: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + adminpdbUser: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + alterSystem: + type: string + alterSystemParameter: + type: string + alterSystemValue: + type: string + asClone: + type: boolean + assertiveLrpdbDeletion: + type: boolean + cdbName: + type: string + cdbNamespace: + type: string + cdbPrvKey: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + cdbResName: + type: string + copyAction: + enum: + - COPY + - NOCOPY + - MOVE + type: string + dropAction: + enum: + - INCLUDING + - KEEP + type: string + fileNameConversions: + type: string + getScript: + type: boolean + lrpdbTlsCat: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + lrpdbTlsCrt: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + lrpdbTlsKey: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + modifyOption: + enum: + - IMMEDIATE + - NORMAL + - READ ONLY + - READ WRITE + - RESTRICTED + type: string + parameterScope: + type: string + pdbName: + type: string + pdbState: + enum: + - OPEN + - CLOSE + - ALTER + type: string + pdbconfigmap: + type: string + reuseTempFile: + type: boolean + sourceFileNameConversions: + type: string + sparseClonePath: + type: string + srcPdbName: + type: string + tdeExport: + type: boolean + tdeImport: + type: boolean + tdeKeystorePath: + type: string + tdePassword: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + tdeSecret: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + tempSize: + type: string + totalSize: + type: string + unlimitedStorage: + type: boolean + webServerPwd: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + webServerUser: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + xmlFileName: + type: string + required: + - action + - alterSystemParameter + - alterSystemValue + - webServerPwd + type: object + status: + properties: + action: + type: string + alterSystem: + type: string + bitstat: + type: integer + bitstatstr: + type: string + connString: + type: string + modifyOption: + type: string + msg: + type: string + openMode: + type: string + phase: + type: string + sqlCode: + type: integer + status: + type: boolean + totalSize: + type: string + required: + - phase + - sqlCode + - status + type: object + type: object + served: true + storage: true + subresources: + status: {} diff --git a/config/crd/bases/database.oracle.com_oraclerestdataservices.yaml b/config/crd/bases/database.oracle.com_oraclerestdataservices.yaml index 121383fd..fe93a531 100644 --- a/config/crd/bases/database.oracle.com_oraclerestdataservices.yaml +++ b/config/crd/bases/database.oracle.com_oraclerestdataservices.yaml @@ -1,11 +1,9 @@ - --- apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - controller-gen.kubebuilder.io/version: v0.6.1 - creationTimestamp: null + controller-gen.kubebuilder.io/version: v0.16.5 name: oraclerestdataservices.database.oracle.com spec: group: database.oracle.com @@ -32,30 +30,22 @@ spec: - jsonPath: .status.apexUrl name: Apex URL type: string + - jsonPath: .status.mongoDbApiAccessUrl + name: MongoDbApi Access URL + type: string name: v1alpha1 schema: openAPIV3Schema: - description: OracleRestDataService is the Schema for the oraclerestdataservices - API properties: apiVersion: - description: 'APIVersion defines the versioned schema of this representation - of an object. Servers should convert recognized schemas to the latest - internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: - description: 'Kind is a string value representing the REST resource this - object represents. Servers may infer this from the endpoint the client - submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: - description: OracleRestDataServiceSpec defines the desired state of OracleRestDataService properties: adminPassword: - description: OracleRestDataServicePassword defines the secret containing - Password mapped to secretKey properties: keepSecret: type: boolean @@ -67,9 +57,30 @@ spec: required: - secretName type: object - apexPassword: - description: OracleRestDataServicePassword defines the secret containing - Password mapped to secretKey + databaseRef: + type: string + image: + properties: + pullFrom: + type: string + pullSecrets: + type: string + version: + type: string + required: + - pullFrom + type: object + loadBalancer: + type: boolean + mongoDbApi: + type: boolean + nodeSelector: + additionalProperties: + type: string + type: object + oracleService: + type: string + ordsPassword: properties: keepSecret: type: boolean @@ -81,11 +92,71 @@ spec: required: - secretName type: object + ordsUser: + type: string + persistence: + properties: + accessMode: + enum: + - ReadWriteOnce + - ReadWriteMany + type: string + setWritePermissions: + type: boolean + size: + type: string + storageClass: + type: string + volumeName: + type: string + type: object + readinessCheckPeriod: + type: integer + replicas: + minimum: 1 + type: integer + restEnableSchemas: + items: + properties: + enable: + type: boolean + pdbName: + type: string + schemaName: + type: string + urlMapping: + type: string + required: + - enable + - schemaName + type: object + type: array + serviceAccountName: + type: string + serviceAnnotations: + additionalProperties: + type: string + type: object + required: + - adminPassword + - databaseRef + - ordsPassword + type: object + status: + properties: + apexConfigured: + type: boolean + apexUrl: + type: string + commonUsersCreated: + type: boolean + databaseActionsUrl: + type: string + databaseApiUrl: + type: string databaseRef: type: string image: - description: OracleRestDataServiceImage defines the Image source and - pullSecrets for POD properties: pullFrom: type: string @@ -97,6 +168,84 @@ spec: - pullFrom type: object loadBalancer: + type: string + mongoDbApi: + type: boolean + mongoDbApiAccessUrl: + type: string + ordsInstalled: + type: boolean + replicas: + type: integer + serviceIP: + type: string + status: + type: string + type: object + type: object + served: true + storage: false + subresources: + status: {} + - additionalPrinterColumns: + - jsonPath: .status.status + name: Status + type: string + - jsonPath: .spec.databaseRef + name: Database + type: string + - jsonPath: .status.databaseApiUrl + name: Database API URL + type: string + - jsonPath: .status.databaseActionsUrl + name: Database Actions URL + type: string + - jsonPath: .status.apexUrl + name: Apex URL + type: string + - jsonPath: .status.mongoDbApiAccessUrl + name: MongoDbApi Access URL + type: string + name: v4 + schema: + openAPIV3Schema: + properties: + apiVersion: + type: string + kind: + type: string + metadata: + type: object + spec: + properties: + adminPassword: + properties: + keepSecret: + type: boolean + secretKey: + default: oracle_pwd + type: string + secretName: + type: string + required: + - secretName + type: object + databaseRef: + type: string + image: + properties: + pullFrom: + type: string + pullSecrets: + type: string + version: + type: string + required: + - pullFrom + type: object + loadBalancer: + type: boolean + mongoDbApi: type: boolean nodeSelector: additionalProperties: @@ -105,8 +254,6 @@ spec: oracleService: type: string ordsPassword: - description: OracleRestDataServicePassword defines the secret containing - Password mapped to secretKey properties: keepSecret: type: boolean @@ -121,14 +268,14 @@ spec: ordsUser: type: string persistence: - description: OracleRestDataServicePersistence defines the storage - releated params properties: accessMode: enum: - ReadWriteOnce - ReadWriteMany type: string + setWritePermissions: + type: boolean size: type: string storageClass: @@ -136,13 +283,13 @@ spec: volumeName: type: string type: object + readinessCheckPeriod: + type: integer replicas: minimum: 1 type: integer restEnableSchemas: items: - description: OracleRestDataServicePDBSchemas defines the PDB Schemas - to be ORDS Enabled properties: enable: type: boolean @@ -169,8 +316,6 @@ spec: - ordsPassword type: object status: - description: OracleRestDataServiceStatus defines the observed state of - OracleRestDataService properties: apexConfigured: type: boolean @@ -185,8 +330,6 @@ spec: databaseRef: type: string image: - description: OracleRestDataServiceImage defines the Image source and - pullSecrets for POD properties: pullFrom: type: string @@ -199,6 +342,10 @@ spec: type: object loadBalancer: type: string + mongoDbApi: + type: boolean + mongoDbApiAccessUrl: + type: string ordsInstalled: type: boolean replicas: @@ -206,9 +353,6 @@ spec: serviceIP: type: string status: - description: 'INSERT ADDITIONAL STATUS FIELD - define observed state - of cluster Important: Run "make" to regenerate code after modifying - this file' type: string type: object type: object @@ -216,9 +360,3 @@ spec: storage: true subresources: status: {} -status: - acceptedNames: - kind: "" - plural: "" - conditions: [] - storedVersions: [] diff --git a/config/crd/bases/database.oracle.com_ordssrvs.yaml b/config/crd/bases/database.oracle.com_ordssrvs.yaml new file mode 100644 index 00000000..9c4ab88f --- /dev/null +++ b/config/crd/bases/database.oracle.com_ordssrvs.yaml @@ -0,0 +1,488 @@ +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.16.5 + name: ordssrvs.database.oracle.com +spec: + group: database.oracle.com + names: + kind: OrdsSrvs + listKind: OrdsSrvsList + plural: ordssrvs + singular: ordssrvs + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .status.status + name: status + type: string + - jsonPath: .status.workloadType + name: workloadType + type: string + - jsonPath: .status.ordsVersion + name: ordsVersion + type: string + - jsonPath: .status.httpPort + name: httpPort + type: integer + - jsonPath: .status.httpsPort + name: httpsPort + type: integer + - jsonPath: .status.mongoPort + name: MongoPort + type: integer + - jsonPath: .status.restartRequired + name: restartRequired + type: boolean + - jsonPath: .metadata.creationTimestamp + name: AGE + type: date + - jsonPath: .status.ordsInstalled + name: OrdsInstalled + type: boolean + name: v4 + schema: + openAPIV3Schema: + properties: + apiVersion: + type: string + kind: + type: string + metadata: + type: object + spec: + properties: + encPrivKey: + properties: + passwordKey: + default: password + type: string + secretName: + type: string + required: + - secretName + type: object + forceRestart: + type: boolean + globalSettings: + properties: + cache.metadata.enabled: + type: boolean + cache.metadata.graphql.expireAfterAccess: + format: int64 + type: integer + cache.metadata.graphql.expireAfterWrite: + format: int64 + type: integer + cache.metadata.jwks.enabled: + type: boolean + cache.metadata.jwks.expireAfterAccess: + format: int64 + type: integer + cache.metadata.jwks.expireAfterWrite: + format: int64 + type: integer + cache.metadata.jwks.initialCapacity: + format: int32 + type: integer + cache.metadata.jwks.maximumSize: + format: int32 + type: integer + cache.metadata.timeout: + format: int64 + type: integer + certSecret: + properties: + cert: + type: string + key: + type: string + secretName: + type: string + required: + - cert + - key + - secretName + type: object + database.api.enabled: + type: boolean + database.api.management.services.disabled: + type: boolean + db.invalidPoolTimeout: + format: int64 + type: integer + debug.printDebugToScreen: + type: boolean + enable.mongo.access.log: + default: false + type: boolean + enable.standalone.access.log: + default: false + type: boolean + error.responseFormat: + type: string + feature.grahpql.max.nesting.depth: + format: int32 + type: integer + icap.port: + format: int32 + type: integer + icap.secure.port: + format: int32 + type: integer + icap.server: + type: string + log.procedure: + type: boolean + mongo.enabled: + type: boolean + mongo.idle.timeout: + format: int64 + type: integer + mongo.op.timeout: + format: int64 + type: integer + mongo.port: + default: 27017 + format: int32 + type: integer + request.traceHeaderName: + type: string + security.credentials.attempts: + format: int32 + type: integer + security.credentials.lock.time: + format: int64 + type: integer + security.disableDefaultExclusionList: + type: boolean + security.exclusionList: + type: string + security.externalSessionTrustedOrigins: + type: string + security.forceHTTPS: + type: boolean + security.httpsHeaderCheck: + type: string + security.inclusionList: + type: string + security.maxEntries: + format: int32 + type: integer + security.verifySSL: + type: boolean + standalone.context.path: + default: /ords + type: string + standalone.http.port: + default: 8080 + format: int32 + type: integer + standalone.https.host: + type: string + standalone.https.port: + default: 8443 + format: int32 + type: integer + standalone.stop.timeout: + format: int64 + type: integer + type: object + image: + type: string + imagePullPolicy: + default: IfNotPresent + enum: + - IfNotPresent + - Always + - Never + type: string + imagePullSecrets: + type: string + poolSettings: + items: + properties: + apex.security.administrator.roles: + type: string + apex.security.user.roles: + type: string + autoUpgradeAPEX: + default: false + type: boolean + autoUpgradeORDS: + default: false + type: boolean + db.adminUser: + type: string + db.adminUser.secret: + properties: + passwordKey: + default: password + type: string + secretName: + type: string + required: + - secretName + type: object + db.cdb.adminUser: + type: string + db.cdb.adminUser.secret: + properties: + passwordKey: + default: password + type: string + secretName: + type: string + required: + - secretName + type: object + db.connectionType: + enum: + - basic + - tns + - customurl + type: string + db.credentialsSource: + enum: + - pool + - request + type: string + db.customURL: + type: string + db.hostname: + type: string + db.poolDestroyTimeout: + format: int64 + type: integer + db.port: + format: int32 + type: integer + db.secret: + properties: + passwordKey: + default: password + type: string + secretName: + type: string + required: + - secretName + type: object + db.servicename: + type: string + db.sid: + type: string + db.tnsAliasName: + type: string + db.username: + default: ORDS_PUBLIC_USER + type: string + db.wallet.zip.service: + type: string + dbWalletSecret: + properties: + secretName: + type: string + walletName: + type: string + required: + - secretName + - walletName + type: object + debug.trackResources: + type: boolean + feature.openservicebroker.exclude: + type: boolean + feature.sdw: + type: boolean + http.cookie.filter: + type: string + jdbc.DriverType: + enum: + - thin + - oci8 + type: string + jdbc.InactivityTimeout: + format: int32 + type: integer + jdbc.InitialLimit: + format: int32 + type: integer + jdbc.MaxConnectionReuseCount: + format: int32 + type: integer + jdbc.MaxConnectionReuseTime: + format: int32 + type: integer + jdbc.MaxLimit: + format: int32 + type: integer + jdbc.MaxStatementsLimit: + format: int32 + type: integer + jdbc.MinLimit: + format: int32 + type: integer + jdbc.SecondsToTrustIdleConnection: + format: int32 + type: integer + jdbc.auth.admin.role: + type: string + jdbc.auth.enabled: + type: boolean + jdbc.cleanup.mode: + type: string + jdbc.statementTimeout: + format: int32 + type: integer + misc.defaultPage: + type: string + misc.pagination.maxRows: + format: int32 + type: integer + owa.trace.sql: + type: boolean + plsql.gateway.mode: + enum: + - disabled + - direct + - proxied + type: string + poolName: + type: string + procedure.preProcess: + type: string + procedure.rest.preHook: + type: string + procedurePostProcess: + type: string + restEnabledSql.active: + type: boolean + security.jwks.connection.timeout: + format: int64 + type: integer + security.jwks.read.timeout: + format: int64 + type: integer + security.jwks.refresh.interval: + format: int64 + type: integer + security.jwks.size: + format: int32 + type: integer + security.jwt.allowed.age: + format: int64 + type: integer + security.jwt.allowed.skew: + format: int64 + type: integer + security.jwt.profile.enabled: + type: boolean + security.requestAuthenticationFunction: + type: string + security.requestValidationFunction: + default: ords_util.authorize_plsql_gateway + type: string + security.validationFunctionType: + enum: + - plsql + - javascript + type: string + soda.defaultLimit: + type: string + soda.maxLimit: + type: string + tnsAdminSecret: + properties: + secretName: + type: string + required: + - secretName + type: object + required: + - db.secret + - poolName + type: object + type: array + replicas: + default: 1 + format: int32 + minimum: 1 + type: integer + workloadType: + default: Deployment + enum: + - Deployment + - StatefulSet + - DaemonSet + type: string + required: + - globalSettings + - image + type: object + status: + properties: + conditions: + items: + properties: + lastTransitionTime: + format: date-time + type: string + message: + maxLength: 32768 + type: string + observedGeneration: + format: int64 + minimum: 0 + type: integer + reason: + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + enum: + - "True" + - "False" + - Unknown + type: string + type: + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + httpPort: + format: int32 + type: integer + httpsPort: + format: int32 + type: integer + mongoPort: + format: int32 + type: integer + ordsInstalled: + type: boolean + ordsVersion: + type: string + restartRequired: + type: boolean + status: + type: string + workloadType: + type: string + required: + - restartRequired + type: object + type: object + served: true + storage: true + subresources: + status: {} diff --git a/config/crd/bases/database.oracle.com_pdbs.yaml b/config/crd/bases/database.oracle.com_pdbs.yaml index 85af8c1b..b674f856 100644 --- a/config/crd/bases/database.oracle.com_pdbs.yaml +++ b/config/crd/bases/database.oracle.com_pdbs.yaml @@ -1,11 +1,9 @@ - --- apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - controller-gen.kubebuilder.io/version: v0.6.1 - creationTimestamp: null + controller-gen.kubebuilder.io/version: v0.16.5 name: pdbs.database.oracle.com spec: group: database.oracle.com @@ -17,10 +15,6 @@ spec: scope: Namespaced versions: - additionalPrinterColumns: - - description: The connect string to be used - jsonPath: .status.connString - name: Connect_String - type: string - description: Name of the CDB jsonPath: .spec.cdbName name: CDB Name @@ -45,29 +39,23 @@ spec: jsonPath: .status.msg name: Message type: string - name: v1alpha1 + - description: The connect string to be used + jsonPath: .status.connString + name: Connect_String + type: string + name: v4 schema: openAPIV3Schema: - description: PDB is the Schema for the pdbs API properties: apiVersion: - description: 'APIVersion defines the versioned schema of this representation - of an object. Servers should convert recognized schemas to the latest - internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: - description: 'Kind is a string value representing the REST resource this - object represents. Servers may infer this from the endpoint the client - submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: - description: PDBSpec defines the desired state of PDB properties: action: - description: 'Action to be taken: Create/Clone/Plug/Unplug/Delete/Modify/Status/Map. - Map is used to map a Databse PDB to a Kubernetes PDB CR.' enum: - Create - Clone @@ -79,11 +67,8 @@ spec: - Map type: string adminName: - description: The administrator username for the new PDB. This property - is required when the Action property is Create. properties: secret: - description: PDBSecret defines the secretName properties: key: type: string @@ -97,11 +82,8 @@ spec: - secret type: object adminPwd: - description: The administrator password for the new PDB. This property - is required when the Action property is Create. properties: secret: - description: PDBSecret defines the secretName properties: key: type: string @@ -115,9 +97,8 @@ spec: - secret type: object asClone: - description: Indicate if 'AS CLONE' option should be used in the command - to plug in a PDB. This property is applicable when the Action property - is PLUG but not required. + type: boolean + assertivePdbDeletion: type: boolean assertivePdbDeletion: description: turn on the assertive approach to delete pdb resource @@ -125,38 +106,30 @@ spec: deletion type: boolean cdbName: - description: Name of the CDB + type: string + cdbNamespace: type: string cdbNamespace: description: CDB Namespace type: string cdbResName: - description: Name of the CDB Custom Resource that runs the ORDS container type: string copyAction: - description: To copy files or not while cloning a PDB enum: - COPY - NOCOPY - MOVE type: string dropAction: - description: Specify if datafiles should be removed or not. The value - can be INCLUDING or KEEP (default). enum: - INCLUDING - KEEP type: string fileNameConversions: - description: Relevant for Create and Plug operations. As defined in - the Oracle Multitenant Database documentation. Values can be a - filename convert pattern or NONE. type: string getScript: - description: Whether you need the script only or execute the script type: boolean modifyOption: - description: Extra options for opening and closing a PDB enum: - IMMEDIATE - NORMAL @@ -165,11 +138,38 @@ spec: - RESTRICTED type: string pdbName: - description: The name of the new PDB. Relevant for both Create and - Plug Actions. type: string + pdbOrdsPrvKey: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + pdbOrdsPubKey: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object pdbState: - description: The target state of the PDB enum: - OPEN - CLOSE @@ -177,7 +177,6 @@ spec: pdbTlsCat: properties: secret: - description: PDBSecret defines the secretName properties: key: type: string @@ -193,7 +192,6 @@ spec: pdbTlsCrt: properties: secret: - description: PDBSecret defines the secretName properties: key: type: string @@ -209,7 +207,6 @@ spec: pdbTlsKey: properties: secret: - description: PDBSecret defines the secretName properties: key: type: string @@ -223,35 +220,22 @@ spec: - secret type: object reuseTempFile: - description: Whether to reuse temp file type: boolean sourceFileNameConversions: - description: This property is required when the Action property is - Plug. As defined in the Oracle Multitenant Database documentation. - Values can be a source filename convert pattern or NONE. type: string sparseClonePath: - description: A Path specified for sparse clone snapshot copy. (Optional) type: string srcPdbName: - description: Name of the Source PDB from which to clone type: string tdeExport: - description: TDE export for unplug operations type: boolean tdeImport: - description: TDE import for plug operations type: boolean tdeKeystorePath: - description: TDE keystore path is required if the tdeImport or tdeExport - flag is set to true. Can be used in plug or unplug operations. type: string tdePassword: - description: TDE password if the tdeImport or tdeExport flag is set - to true. Can be used in create, plug or unplug operations properties: secret: - description: PDBSecret defines the secretName properties: key: type: string @@ -265,11 +249,8 @@ spec: - secret type: object tdeSecret: - description: TDE secret is required if the tdeImport or tdeExport - flag is set to true. Can be used in plug or unplug operations. properties: secret: - description: PDBSecret defines the secretName properties: key: type: string @@ -283,26 +264,14 @@ spec: - secret type: object tempSize: - description: Relevant for Create and Clone operations. Total size - for temporary tablespace as defined in the Oracle Multitenant Database - documentation. See size_clause description in Database SQL Language - Reference documentation. type: string totalSize: - description: Relevant for create and plug operations. Total size as - defined in the Oracle Multitenant Database documentation. See size_clause - description in Database SQL Language Reference documentation. type: string unlimitedStorage: - description: Relevant for Create and Plug operations. True for unlimited - storage. Even when set to true, totalSize and tempSize MUST be specified - in the request if Action is Create. type: boolean webServerPwd: - description: Password for the Web ServerPDB User properties: secret: - description: PDBSecret defines the secretName properties: key: type: string @@ -316,11 +285,8 @@ spec: - secret type: object webServerUser: - description: Web Server User with SQL Administrator role to allow - us to authenticate to the PDB Lifecycle Management REST endpoints properties: secret: - description: PDBSecret defines the secretName properties: key: type: string @@ -334,37 +300,27 @@ spec: - secret type: object xmlFileName: - description: XML metadata filename to be used for Plug or Unplug operations type: string required: - action type: object status: - description: PDBStatus defines the observed state of PDB properties: action: - description: Last Completed Action type: string connString: - description: PDB Connect String type: string modifyOption: - description: Modify Option of the PDB type: string msg: - description: Message type: string openMode: - description: Open mode of the PDB type: string phase: - description: Phase of the PDB Resource type: string status: - description: PDB Resource Status type: boolean totalSize: - description: Total size of the PDB type: string required: - phase @@ -375,9 +331,3 @@ spec: storage: true subresources: status: {} -status: - acceptedNames: - kind: "" - plural: "" - conditions: [] - storedVersions: [] diff --git a/config/crd/bases/database.oracle.com_shardingdatabases.yaml b/config/crd/bases/database.oracle.com_shardingdatabases.yaml index 641629a0..e46d883e 100644 --- a/config/crd/bases/database.oracle.com_shardingdatabases.yaml +++ b/config/crd/bases/database.oracle.com_shardingdatabases.yaml @@ -1,11 +1,9 @@ - --- apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - controller-gen.kubebuilder.io/version: v0.6.1 - creationTimestamp: null + controller-gen.kubebuilder.io/version: v0.16.5 name: shardingdatabases.database.oracle.com spec: group: database.oracle.com @@ -30,33 +28,22 @@ spec: name: v1alpha1 schema: openAPIV3Schema: - description: ShardingDatabase is the Schema for the shardingdatabases API properties: apiVersion: - description: 'APIVersion defines the versioned schema of this representation - of an object. Servers should convert recognized schemas to the latest - internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: - description: 'Kind is a string value representing the REST resource this - object represents. Servers may infer this from the endpoint the client - submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: - description: ShardingDatabaseSpec defines the desired state of ShardingDatabase properties: InvitedNodeSubnet: type: string catalog: items: - description: CatalogSpec defines the desired state of CatalogSpec properties: envVars: items: - description: EnvironmentVariable represents a named variable - accessible for containers. properties: name: type: string @@ -68,8 +55,6 @@ spec: type: object type: array imagePullPolicy: - description: PullPolicy describes a policy for if/when to pull - a container image type: string isDelete: type: string @@ -92,23 +77,13 @@ spec: pvcName: type: string resources: - description: ResourceRequirements describes the compute resource - requirements. properties: claims: - description: "Claims lists the names of resources, defined - in spec.resourceClaims, that are used by this container. - \n This is an alpha field and requires enabling the DynamicResourceAllocation - feature gate. \n This field is immutable. It can only - be set for containers." items: - description: ResourceClaim references one entry in PodSpec.ResourceClaims. properties: name: - description: Name must match the name of one entry - in pod.spec.resourceClaims of the Pod where this - field is used. It makes that resource available - inside a container. + type: string + request: type: string required: - name @@ -124,8 +99,6 @@ spec: - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true - description: 'Limits describes the maximum amount of compute - resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' type: object requests: additionalProperties: @@ -134,11 +107,6 @@ spec: - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true - description: 'Requests describes the minimum amount of compute - resources required. If Requests is omitted for a container, - it defaults to Limits if that is explicitly specified, - otherwise to an implementation-defined value. Requests - cannot exceed Limits. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' type: object type: object storageSizeInGb: @@ -155,7 +123,6 @@ spec: dbImagePullSecret: type: string dbSecret: - description: Secret Details properties: encryptionType: type: string @@ -183,7 +150,6 @@ spec: type: string gsm: items: - description: GsmSpec defines the desired state of GsmSpec properties: directorName: type: string @@ -192,8 +158,6 @@ spec: Gsm Replicas. If you set OraGsmPvcName then it is set default to 1. items: - description: EnvironmentVariable represents a named variable - accessible for containers. properties: name: type: string @@ -205,8 +169,6 @@ spec: type: object type: array imagePullPolicy: - description: PullPolicy describes a policy for if/when to pull - a container image type: string isDelete: type: string @@ -218,6 +180,10 @@ spec: additionalProperties: type: string type: object + pvAnnotations: + additionalProperties: + type: string + type: object pvMatchLabels: additionalProperties: type: string @@ -227,23 +193,13 @@ spec: region: type: string resources: - description: ResourceRequirements describes the compute resource - requirements. properties: claims: - description: "Claims lists the names of resources, defined - in spec.resourceClaims, that are used by this container. - \n This is an alpha field and requires enabling the DynamicResourceAllocation - feature gate. \n This field is immutable. It can only - be set for containers." items: - description: ResourceClaim references one entry in PodSpec.ResourceClaims. properties: name: - description: Name must match the name of one entry - in pod.spec.resourceClaims of the Pod where this - field is used. It makes that resource available - inside a container. + type: string + request: type: string required: - name @@ -259,8 +215,6 @@ spec: - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true - description: 'Limits describes the maximum amount of compute - resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' type: object requests: additionalProperties: @@ -269,11 +223,6 @@ spec: - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true - description: 'Requests describes the minimum amount of compute - resources required. If Requests is omitted for a container, - it defaults to Limits if that is explicitly specified, - otherwise to an implementation-defined value. Requests - cannot exceed Limits. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' type: object type: object storageSizeInGb: @@ -291,7 +240,6 @@ spec: type: string gsmService: items: - description: Service Definition properties: available: type: string @@ -370,7 +318,6 @@ spec: type: array gsmShardSpace: items: - description: ShardSpace Specs properties: chunks: type: integer @@ -402,18 +349,13 @@ spec: type: string liveinessCheckPeriod: type: integer - namespace: - type: string portMappings: items: - description: PortMapping is a specification of port mapping for - an application deployment. properties: port: format: int32 type: integer protocol: - default: TCP type: string targetPort: format: int32 @@ -431,18 +373,12 @@ spec: scriptsLocation: type: string shard: - description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster - Important: Run "make" to regenerate code after modifying this file' items: - description: ShardSpec is a specification of Shards for an application - deployment. properties: deployAs: type: string envVars: items: - description: EnvironmentVariable represents a named variable - accessible for containers. properties: name: type: string @@ -454,8 +390,6 @@ spec: type: object type: array imagePullPolicy: - description: PullPolicy describes a policy for if/when to pull - a container image type: string isDelete: enum: @@ -483,23 +417,13 @@ spec: pvcName: type: string resources: - description: ResourceRequirements describes the compute resource - requirements. properties: claims: - description: "Claims lists the names of resources, defined - in spec.resourceClaims, that are used by this container. - \n This is an alpha field and requires enabling the DynamicResourceAllocation - feature gate. \n This field is immutable. It can only - be set for containers." items: - description: ResourceClaim references one entry in PodSpec.ResourceClaims. properties: name: - description: Name must match the name of one entry - in pod.spec.resourceClaims of the Pod where this - field is used. It makes that resource available - inside a container. + type: string + request: type: string required: - name @@ -515,8 +439,6 @@ spec: - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true - description: 'Limits describes the maximum amount of compute - resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' type: object requests: additionalProperties: @@ -525,11 +447,6 @@ spec: - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true - description: 'Requests describes the minimum amount of compute - resources required. If Requests is omitted for a container, - it defaults to Limits if that is explicitly specified, - otherwise to an implementation-defined value. Requests - cannot exceed Limits. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' type: object type: object shardGroup: @@ -563,6 +480,560 @@ spec: type: string tdeWalletPvcMountLocation: type: string + topicId: + type: string + required: + - catalog + - dbImage + - gsm + - gsmImage + - shard + type: object + status: + properties: + catalogs: + additionalProperties: + type: string + type: object + conditions: + items: + properties: + lastTransitionTime: + format: date-time + type: string + message: + maxLength: 32768 + type: string + observedGeneration: + format: int64 + minimum: 0 + type: integer + reason: + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + enum: + - "True" + - "False" + - Unknown + type: string + type: + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + gsm: + properties: + details: + additionalProperties: + type: string + type: object + externalConnectStr: + type: string + internalConnectStr: + type: string + services: + type: string + shards: + additionalProperties: + type: string + type: object + state: + type: string + type: object + shards: + additionalProperties: + type: string + type: object + type: object + type: object + served: true + storage: false + subresources: + status: {} + - additionalPrinterColumns: + - jsonPath: .status.gsm.state + name: Gsm State + type: string + - jsonPath: .status.gsm.services + name: Services + type: string + - jsonPath: .status.gsm.shards + name: shards + priority: 1 + type: string + name: v4 + schema: + openAPIV3Schema: + properties: + apiVersion: + type: string + kind: + type: string + metadata: + type: object + spec: + properties: + InvitedNodeSubnet: + type: string + catalog: + items: + properties: + envVars: + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + imagePullPolicy: + type: string + isDelete: + type: string + label: + type: string + name: + type: string + nodeSelector: + additionalProperties: + type: string + type: object + pvAnnotations: + additionalProperties: + type: string + type: object + pvMatchLabels: + additionalProperties: + type: string + type: object + pvcName: + type: string + resources: + properties: + claims: + items: + properties: + name: + type: string + request: + type: string + required: + - name + type: object + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + limits: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + requests: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + type: object + shardGroup: + type: string + shardRegion: + type: string + shardSpace: + type: string + storageSizeInGb: + format: int32 + type: integer + required: + - name + type: object + type: array + dbEdition: + type: string + dbImage: + type: string + dbImagePullSecret: + type: string + dbSecret: + properties: + encryptionType: + type: string + keyFileMountLocation: + type: string + keyFileName: + type: string + keySecretName: + type: string + name: + type: string + nsConfigMap: + type: string + nsSecret: + type: string + pwdFileMountLocation: + type: string + pwdFileName: + type: string + required: + - name + - pwdFileName + type: object + fssStorageClass: + type: string + gsm: + items: + properties: + directorName: + type: string + envVars: + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + imagePullPolicy: + type: string + isDelete: + type: string + label: + type: string + name: + type: string + nodeSelector: + additionalProperties: + type: string + type: object + pvAnnotations: + additionalProperties: + type: string + type: object + pvMatchLabels: + additionalProperties: + type: string + type: object + pvcName: + type: string + region: + type: string + resources: + properties: + claims: + items: + properties: + name: + type: string + request: + type: string + required: + - name + type: object + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + limits: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + requests: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + type: object + storageSizeInGb: + format: int32 + type: integer + required: + - name + type: object + type: array + gsmDevMode: + type: string + gsmImage: + type: string + gsmImagePullSecret: + type: string + gsmService: + items: + properties: + available: + type: string + clbGoal: + type: string + commitOutcome: + type: string + drainTimeout: + type: string + dtp: + type: string + edition: + type: string + failoverDelay: + type: string + failoverMethod: + type: string + failoverPrimary: + type: string + failoverRestore: + type: string + failoverRetry: + type: string + failoverType: + type: string + gdsPool: + type: string + lag: + type: integer + locality: + type: string + name: + type: string + notification: + type: string + pdbName: + type: string + policy: + type: string + preferred: + type: string + prferredAll: + type: string + regionFailover: + type: string + retention: + type: string + role: + type: string + sessionState: + type: string + sqlTransactionProfile: + type: string + stopOption: + type: string + tableFamily: + type: string + tfaPolicy: + type: string + required: + - name + type: object + type: array + gsmShardGroup: + items: + properties: + deployAs: + type: string + name: + type: string + region: + type: string + required: + - name + type: object + type: array + gsmShardSpace: + items: + properties: + chunks: + type: integer + name: + type: string + protectionMode: + type: string + shardGroup: + type: string + required: + - name + type: object + type: array + invitedNodeSubnetFlag: + type: string + isClone: + type: boolean + isDataGuard: + type: boolean + isDebug: + type: boolean + isDeleteOraPvc: + type: boolean + isDownloadScripts: + type: boolean + isExternalSvc: + type: boolean + isTdeWallet: + type: string + liveinessCheckPeriod: + type: integer + portMappings: + items: + properties: + port: + format: int32 + type: integer + protocol: + type: string + targetPort: + format: int32 + type: integer + required: + - port + - protocol + - targetPort + type: object + type: array + readinessCheckPeriod: + type: integer + replicationType: + type: string + scriptsLocation: + type: string + shard: + items: + properties: + deployAs: + type: string + envVars: + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + imagePullPolicy: + type: string + isDelete: + enum: + - enable + - disable + - failed + - force + type: string + label: + type: string + name: + type: string + nodeSelector: + additionalProperties: + type: string + type: object + pvAnnotations: + additionalProperties: + type: string + type: object + pvMatchLabels: + additionalProperties: + type: string + type: object + pvcName: + type: string + resources: + properties: + claims: + items: + properties: + name: + type: string + request: + type: string + required: + - name + type: object + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + limits: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + requests: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + type: object + shardGroup: + type: string + shardRegion: + type: string + shardSpace: + type: string + storageSizeInGb: + format: int32 + type: integer + required: + - name + type: object + type: array + shardBuddyRegion: + type: string + shardConfigName: + type: string + shardRegion: + items: + type: string + type: array + shardingType: + type: string + stagePvcName: + type: string + storageClass: + type: string + tdeWalletPvc: + type: string + tdeWalletPvcMountLocation: + type: string + topicId: + type: string required: - catalog - dbImage @@ -571,8 +1042,6 @@ spec: - shard type: object status: - description: To understand Metav1.Condition, please refer the link https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1 - ShardingDatabaseStatus defines the observed state of ShardingDatabase properties: catalogs: additionalProperties: @@ -580,63 +1049,29 @@ spec: type: object conditions: items: - description: "Condition contains details for one aspect of the current - state of this API Resource. --- This struct is intended for direct - use as an array at the field path .status.conditions. For example, - \n \ttype FooStatus struct{ \t // Represents the observations - of a foo's current state. \t // Known .status.conditions.type - are: \"Available\", \"Progressing\", and \"Degraded\" \t // - +patchMergeKey=type \t // +patchStrategy=merge \t // +listType=map - \t // +listMapKey=type \t Conditions []metav1.Condition - `json:\"conditions,omitempty\" patchStrategy:\"merge\" patchMergeKey:\"type\" - protobuf:\"bytes,1,rep,name=conditions\"` \n \t // other fields - \t}" properties: lastTransitionTime: - description: lastTransitionTime is the last time the condition - transitioned from one status to another. This should be when - the underlying condition changed. If that is not known, then - using the time when the API field changed is acceptable. format: date-time type: string message: - description: message is a human readable message indicating - details about the transition. This may be an empty string. maxLength: 32768 type: string observedGeneration: - description: observedGeneration represents the .metadata.generation - that the condition was set based upon. For instance, if .metadata.generation - is currently 12, but the .status.conditions[x].observedGeneration - is 9, the condition is out of date with respect to the current - state of the instance. format: int64 minimum: 0 type: integer reason: - description: reason contains a programmatic identifier indicating - the reason for the condition's last transition. Producers - of specific condition types may define expected values and - meanings for this field, and whether the values are considered - a guaranteed API. The value should be a CamelCase string. - This field may not be empty. maxLength: 1024 minLength: 1 pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ type: string status: - description: status of the condition, one of True, False, Unknown. enum: - "True" - "False" - Unknown type: string type: - description: type of condition in CamelCase or in foo.example.com/CamelCase. - --- Many .condition.type values are consistent across resources - like Available, but because arbitrary conditions can be useful - (see .node.status.conditions), the ability to deconflict is - important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) maxLength: 316 pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ type: string @@ -680,9 +1115,3 @@ spec: storage: true subresources: status: {} -status: - acceptedNames: - kind: "" - plural: "" - conditions: [] - storedVersions: [] diff --git a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml index 1c011e17..8357f2c5 100644 --- a/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml +++ b/config/crd/bases/database.oracle.com_singleinstancedatabases.yaml @@ -1,11 +1,9 @@ - --- apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - controller-gen.kubebuilder.io/version: v0.6.1 - creationTimestamp: null + controller-gen.kubebuilder.io/version: v0.16.5 name: singleinstancedatabases.database.oracle.com spec: group: database.oracle.com @@ -53,27 +51,16 @@ spec: name: v1alpha1 schema: openAPIV3Schema: - description: SingleInstanceDatabase is the Schema for the singleinstancedatabases - API properties: apiVersion: - description: 'APIVersion defines the versioned schema of this representation - of an object. Servers should convert recognized schemas to the latest - internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: - description: 'Kind is a string value representing the REST resource this - object represents. Servers may infer this from the endpoint the client - submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: - description: SingleInstanceDatabaseSpec defines the desired state of SingleInstanceDatabase properties: adminPassword: - description: SingleInsatnceAdminPassword defines the secret containing - Admin Password mapped to secretKey for Database properties: keepSecret: type: boolean @@ -89,14 +76,15 @@ spec: type: boolean charset: type: string + convertToSnapshotStandby: + type: boolean createAs: enum: - primary - standby - clone + - truecache type: string - dgBrokerConfigured: - type: boolean edition: enum: - standard @@ -111,8 +99,6 @@ spec: forceLog: type: boolean image: - description: SingleInstanceDatabaseImage defines the Image source - and pullSecrets for POD properties: prebuiltDB: type: boolean @@ -126,7 +112,6 @@ spec: - pullFrom type: object initParams: - description: SingleInstanceDatabaseInitParams defines the Init Parameters properties: cpuCount: type: integer @@ -148,8 +133,6 @@ spec: pdbName: type: string persistence: - description: SingleInstanceDatabasePersistence defines the storage - size and class for PVC properties: accessMode: enum: @@ -199,8 +182,6 @@ spec: type: string type: object sid: - description: SID must be alphanumeric (no special characters, only - a-z, A-Z, 0-9), and no longer than 12 characters. maxLength: 12 pattern: ^[a-zA-Z0-9]+$ type: string @@ -210,12 +191,14 @@ spec: type: integer tcpsTlsSecret: type: string + trueCacheServices: + items: + type: string + type: array required: - image type: object status: - description: SingleInstanceDatabaseStatus defines the observed state of - SingleInstanceDatabase properties: apexInstalled: type: boolean @@ -233,63 +216,29 @@ spec: type: string conditions: items: - description: "Condition contains details for one aspect of the current - state of this API Resource. --- This struct is intended for direct - use as an array at the field path .status.conditions. For example, - \n \ttype FooStatus struct{ \t // Represents the observations - of a foo's current state. \t // Known .status.conditions.type - are: \"Available\", \"Progressing\", and \"Degraded\" \t // - +patchMergeKey=type \t // +patchStrategy=merge \t // +listType=map - \t // +listMapKey=type \t Conditions []metav1.Condition - `json:\"conditions,omitempty\" patchStrategy:\"merge\" patchMergeKey:\"type\" - protobuf:\"bytes,1,rep,name=conditions\"` \n \t // other fields - \t}" properties: lastTransitionTime: - description: lastTransitionTime is the last time the condition - transitioned from one status to another. This should be when - the underlying condition changed. If that is not known, then - using the time when the API field changed is acceptable. format: date-time type: string message: - description: message is a human readable message indicating - details about the transition. This may be an empty string. maxLength: 32768 type: string observedGeneration: - description: observedGeneration represents the .metadata.generation - that the condition was set based upon. For instance, if .metadata.generation - is currently 12, but the .status.conditions[x].observedGeneration - is 9, the condition is out of date with respect to the current - state of the instance. format: int64 minimum: 0 type: integer reason: - description: reason contains a programmatic identifier indicating - the reason for the condition's last transition. Producers - of specific condition types may define expected values and - meanings for this field, and whether the values are considered - a guaranteed API. The value should be a CamelCase string. - This field may not be empty. maxLength: 1024 minLength: 1 pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ type: string status: - description: status of the condition, one of True, False, Unknown. enum: - "True" - "False" - Unknown type: string type: - description: type of condition in CamelCase or in foo.example.com/CamelCase. - --- Many .condition.type values are consistent across resources - like Available, but because arbitrary conditions can be useful - (see .node.status.conditions), the ability to deconflict is - important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) maxLength: 316 pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ type: string @@ -306,6 +255,8 @@ spec: x-kubernetes-list-type: map connectString: type: string + convertToSnapshotStandby: + type: boolean createdAs: type: string datafilesCreated: @@ -314,8 +265,355 @@ spec: datafilesPatched: default: "false" type: string - dgBrokerConfigured: + dgBroker: + type: string + edition: + type: string + flashBack: + type: string + forceLog: + type: string + initParams: + properties: + cpuCount: + type: integer + pgaAggregateTarget: + type: integer + processes: + type: integer + sgaTarget: + type: integer + type: object + initPgaSize: + type: integer + initSgaSize: + type: integer + isTcpsEnabled: + default: false + type: boolean + nodes: + items: + type: string + type: array + oemExpressUrl: + type: string + ordsReference: + type: string + pdbConnectString: + type: string + pdbName: + type: string + persistence: + properties: + accessMode: + enum: + - ReadWriteOnce + - ReadWriteMany + type: string + datafilesVolumeName: + type: string + scriptsVolumeName: + type: string + setWritePermissions: + type: boolean + size: + type: string + storageClass: + type: string + volumeClaimAnnotation: + type: string + type: object + prebuiltDB: + type: boolean + primaryDatabase: + type: string + releaseUpdate: + type: string + replicas: + type: integer + role: + type: string + sid: + type: string + standbyDatabases: + additionalProperties: + type: string + type: object + status: + type: string + tcpsConnectString: + type: string + tcpsPdbConnectString: + type: string + tcpsTlsSecret: + default: "" + type: string + required: + - isTcpsEnabled + - persistence + - tcpsTlsSecret + type: object + type: object + served: true + storage: false + subresources: + scale: + specReplicasPath: .spec.replicas + statusReplicasPath: .status.replicas + status: {} + - additionalPrinterColumns: + - jsonPath: .status.edition + name: Edition + type: string + - jsonPath: .status.sid + name: Sid + priority: 1 + type: string + - jsonPath: .status.status + name: Status + type: string + - jsonPath: .status.role + name: Role + type: string + - jsonPath: .status.releaseUpdate + name: Version + type: string + - jsonPath: .status.connectString + name: Connect Str + type: string + - jsonPath: .status.pdbConnectString + name: Pdb Connect Str + priority: 1 + type: string + - jsonPath: .status.tcpsConnectString + name: TCPS Connect Str + type: string + - jsonPath: .status.tcpsPdbConnectString + name: TCPS Pdb Connect Str + priority: 1 + type: string + - jsonPath: .status.oemExpressUrl + name: Oem Express Url + type: string + name: v4 + schema: + openAPIV3Schema: + properties: + apiVersion: + type: string + kind: + type: string + metadata: + type: object + spec: + properties: + adminPassword: + properties: + keepSecret: + type: boolean + secretKey: + default: oracle_pwd + type: string + secretName: + type: string + required: + - secretName + type: object + archiveLog: + type: boolean + charset: + type: string + convertToSnapshotStandby: + type: boolean + createAs: + enum: + - primary + - standby + - clone + - truecache + type: string + edition: + enum: + - standard + - enterprise + - express + - free + type: string + enableTCPS: + type: boolean + flashBack: + type: boolean + forceLog: + type: boolean + image: + properties: + prebuiltDB: + type: boolean + pullFrom: + type: string + pullSecrets: + type: string + version: + type: string + required: + - pullFrom + type: object + initParams: + properties: + cpuCount: + type: integer + pgaAggregateTarget: + type: integer + processes: + type: integer + sgaTarget: + type: integer + type: object + listenerPort: + type: integer + loadBalancer: type: boolean + nodeSelector: + additionalProperties: + type: string + type: object + pdbName: + type: string + persistence: + properties: + accessMode: + enum: + - ReadWriteOnce + - ReadWriteMany + type: string + datafilesVolumeName: + type: string + scriptsVolumeName: + type: string + setWritePermissions: + type: boolean + size: + type: string + storageClass: + type: string + volumeClaimAnnotation: + type: string + type: object + primaryDatabaseRef: + type: string + readinessCheckPeriod: + type: integer + replicas: + type: integer + resources: + properties: + limits: + properties: + cpu: + type: string + memory: + type: string + type: object + requests: + properties: + cpu: + type: string + memory: + type: string + type: object + type: object + serviceAccountName: + type: string + serviceAnnotations: + additionalProperties: + type: string + type: object + sid: + maxLength: 12 + pattern: ^[a-zA-Z0-9]+$ + type: string + tcpsCertRenewInterval: + type: string + tcpsListenerPort: + type: integer + tcpsTlsSecret: + type: string + trueCacheServices: + items: + type: string + type: array + required: + - image + type: object + status: + properties: + apexInstalled: + type: boolean + archiveLog: + type: string + certCreationTimestamp: + type: string + certRenewInterval: + type: string + charset: + type: string + clientWalletLoc: + type: string + clusterConnectString: + type: string + conditions: + items: + properties: + lastTransitionTime: + format: date-time + type: string + message: + maxLength: 32768 + type: string + observedGeneration: + format: int64 + minimum: 0 + type: integer + reason: + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + enum: + - "True" + - "False" + - Unknown + type: string + type: + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + connectString: + type: string + convertToSnapshotStandby: + type: boolean + createdAs: + type: string + datafilesCreated: + default: "false" + type: string + datafilesPatched: + default: "false" + type: string + dgBroker: + type: string edition: type: string flashBack: @@ -323,7 +621,6 @@ spec: forceLog: type: string initParams: - description: SingleInstanceDatabaseInitParams defines the Init Parameters properties: cpuCount: type: integer @@ -354,8 +651,6 @@ spec: pdbName: type: string persistence: - description: SingleInstanceDatabasePersistence defines the storage - size and class for PVC properties: accessMode: enum: @@ -413,9 +708,3 @@ spec: specReplicasPath: .spec.replicas statusReplicasPath: .status.replicas status: {} -status: - acceptedNames: - kind: "" - plural: "" - conditions: [] - storedVersions: [] diff --git a/config/crd/bases/observability.oracle.com_databaseobservers.yaml b/config/crd/bases/observability.oracle.com_databaseobservers.yaml index b0801738..298f9d4e 100644 --- a/config/crd/bases/observability.oracle.com_databaseobservers.yaml +++ b/config/crd/bases/observability.oracle.com_databaseobservers.yaml @@ -1,11 +1,9 @@ - --- apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - controller-gen.kubebuilder.io/version: v0.6.1 - creationTimestamp: null + controller-gen.kubebuilder.io/version: v0.16.5 name: databaseobservers.observability.oracle.com spec: group: observability.oracle.com @@ -13,6 +11,9 @@ spec: kind: DatabaseObserver listKind: DatabaseObserverList plural: databaseobservers + shortNames: + - dbobserver + - dbobservers singular: databaseobserver scope: Namespaced versions: @@ -23,29 +24,4678 @@ spec: - jsonPath: .status.status name: Status type: string + - jsonPath: .status.version + name: Version + type: string + name: v1 + schema: + openAPIV3Schema: + properties: + apiVersion: + type: string + kind: + type: string + metadata: + type: object + spec: + properties: + configuration: + properties: + configMap: + properties: + key: + type: string + name: + type: string + type: object + type: object + database: + properties: + dbConnectionString: + properties: + key: + type: string + secret: + type: string + type: object + dbPassword: + properties: + key: + type: string + secret: + type: string + vaultOCID: + type: string + vaultSecretName: + type: string + type: object + dbUser: + properties: + key: + type: string + secret: + type: string + type: object + dbWallet: + properties: + key: + type: string + secret: + type: string + type: object + type: object + exporter: + properties: + deployment: + properties: + args: + items: + type: string + type: array + commands: + items: + type: string + type: array + env: + additionalProperties: + type: string + type: object + image: + type: string + labels: + additionalProperties: + type: string + type: object + podTemplate: + properties: + labels: + additionalProperties: + type: string + type: object + securityContext: + properties: + appArmorProfile: + properties: + localhostProfile: + type: string + type: + type: string + required: + - type + type: object + fsGroup: + format: int64 + type: integer + fsGroupChangePolicy: + type: string + runAsGroup: + format: int64 + type: integer + runAsNonRoot: + type: boolean + runAsUser: + format: int64 + type: integer + seLinuxOptions: + properties: + level: + type: string + role: + type: string + type: + type: string + user: + type: string + type: object + seccompProfile: + properties: + localhostProfile: + type: string + type: + type: string + required: + - type + type: object + supplementalGroups: + items: + format: int64 + type: integer + type: array + x-kubernetes-list-type: atomic + supplementalGroupsPolicy: + type: string + sysctls: + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + x-kubernetes-list-type: atomic + windowsOptions: + properties: + gmsaCredentialSpec: + type: string + gmsaCredentialSpecName: + type: string + hostProcess: + type: boolean + runAsUserName: + type: string + type: object + type: object + type: object + securityContext: + properties: + allowPrivilegeEscalation: + type: boolean + appArmorProfile: + properties: + localhostProfile: + type: string + type: + type: string + required: + - type + type: object + capabilities: + properties: + add: + items: + type: string + type: array + x-kubernetes-list-type: atomic + drop: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + privileged: + type: boolean + procMount: + type: string + readOnlyRootFilesystem: + type: boolean + runAsGroup: + format: int64 + type: integer + runAsNonRoot: + type: boolean + runAsUser: + format: int64 + type: integer + seLinuxOptions: + properties: + level: + type: string + role: + type: string + type: + type: string + user: + type: string + type: object + seccompProfile: + properties: + localhostProfile: + type: string + type: + type: string + required: + - type + type: object + windowsOptions: + properties: + gmsaCredentialSpec: + type: string + gmsaCredentialSpecName: + type: string + hostProcess: + type: boolean + runAsUserName: + type: string + type: object + type: object + type: object + service: + properties: + labels: + additionalProperties: + type: string + type: object + ports: + items: + properties: + appProtocol: + type: string + name: + type: string + nodePort: + format: int32 + type: integer + port: + format: int32 + type: integer + protocol: + default: TCP + type: string + targetPort: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + required: + - port + type: object + type: array + type: object + type: object + inheritLabels: + items: + type: string + type: array + log: + properties: + filename: + type: string + path: + type: string + volume: + properties: + name: + type: string + persistentVolumeClaim: + properties: + claimName: + type: string + type: object + type: object + type: object + ociConfig: + properties: + configMapName: + type: string + secretName: + type: string + type: object + prometheus: + properties: + serviceMonitor: + properties: + endpoints: + items: + properties: + authorization: + properties: + credentials: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: + type: string + type: object + basicAuth: + properties: + password: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + username: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: object + bearerTokenFile: + type: string + bearerTokenSecret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + enableHttp2: + type: boolean + filterRunning: + type: boolean + followRedirects: + type: boolean + honorLabels: + type: boolean + honorTimestamps: + type: boolean + interval: + pattern: ^(0|(([0-9]+)y)?(([0-9]+)w)?(([0-9]+)d)?(([0-9]+)h)?(([0-9]+)m)?(([0-9]+)s)?(([0-9]+)ms)?)$ + type: string + metricRelabelings: + items: + properties: + action: + default: replace + enum: + - replace + - Replace + - keep + - Keep + - drop + - Drop + - hashmod + - HashMod + - labelmap + - LabelMap + - labeldrop + - LabelDrop + - labelkeep + - LabelKeep + - lowercase + - Lowercase + - uppercase + - Uppercase + - keepequal + - KeepEqual + - dropequal + - DropEqual + type: string + modulus: + format: int64 + type: integer + regex: + type: string + replacement: + type: string + separator: + type: string + sourceLabels: + items: + pattern: ^[a-zA-Z_][a-zA-Z0-9_]*$ + type: string + type: array + targetLabel: + type: string + type: object + type: array + oauth2: + properties: + clientId: + properties: + configMap: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + secret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: object + clientSecret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + endpointParams: + additionalProperties: + type: string + type: object + noProxy: + type: string + proxyConnectHeader: + additionalProperties: + items: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: array + type: object + x-kubernetes-map-type: atomic + proxyFromEnvironment: + type: boolean + proxyUrl: + pattern: ^http(s)?://.+$ + type: string + scopes: + items: + type: string + type: array + tlsConfig: + properties: + ca: + properties: + configMap: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + secret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: object + cert: + properties: + configMap: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + secret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: object + insecureSkipVerify: + type: boolean + keySecret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + maxVersion: + enum: + - TLS10 + - TLS11 + - TLS12 + - TLS13 + type: string + minVersion: + enum: + - TLS10 + - TLS11 + - TLS12 + - TLS13 + type: string + serverName: + type: string + type: object + tokenUrl: + minLength: 1 + type: string + required: + - clientId + - clientSecret + - tokenUrl + type: object + params: + additionalProperties: + items: + type: string + type: array + type: object + path: + type: string + port: + type: string + proxyUrl: + type: string + relabelings: + items: + properties: + action: + default: replace + enum: + - replace + - Replace + - keep + - Keep + - drop + - Drop + - hashmod + - HashMod + - labelmap + - LabelMap + - labeldrop + - LabelDrop + - labelkeep + - LabelKeep + - lowercase + - Lowercase + - uppercase + - Uppercase + - keepequal + - KeepEqual + - dropequal + - DropEqual + type: string + modulus: + format: int64 + type: integer + regex: + type: string + replacement: + type: string + separator: + type: string + sourceLabels: + items: + pattern: ^[a-zA-Z_][a-zA-Z0-9_]*$ + type: string + type: array + targetLabel: + type: string + type: object + type: array + scheme: + enum: + - http + - https + type: string + scrapeTimeout: + pattern: ^(0|(([0-9]+)y)?(([0-9]+)w)?(([0-9]+)d)?(([0-9]+)h)?(([0-9]+)m)?(([0-9]+)s)?(([0-9]+)ms)?)$ + type: string + targetPort: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + tlsConfig: + properties: + ca: + properties: + configMap: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + secret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: object + caFile: + type: string + cert: + properties: + configMap: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + secret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: object + certFile: + type: string + insecureSkipVerify: + type: boolean + keyFile: + type: string + keySecret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + maxVersion: + enum: + - TLS10 + - TLS11 + - TLS12 + - TLS13 + type: string + minVersion: + enum: + - TLS10 + - TLS11 + - TLS12 + - TLS13 + type: string + serverName: + type: string + type: object + trackTimestampsStaleness: + type: boolean + type: object + type: array + labels: + additionalProperties: + type: string + type: object + namespaceSelector: + properties: + any: + type: boolean + matchNames: + items: + type: string + type: array + type: object + type: object + type: object + replicas: + format: int32 + type: integer + sidecarVolumes: + items: + properties: + awsElasticBlockStore: + properties: + fsType: + type: string + partition: + format: int32 + type: integer + readOnly: + type: boolean + volumeID: + type: string + required: + - volumeID + type: object + azureDisk: + properties: + cachingMode: + type: string + diskName: + type: string + diskURI: + type: string + fsType: + default: ext4 + type: string + kind: + type: string + readOnly: + default: false + type: boolean + required: + - diskName + - diskURI + type: object + azureFile: + properties: + readOnly: + type: boolean + secretName: + type: string + shareName: + type: string + required: + - secretName + - shareName + type: object + cephfs: + properties: + monitors: + items: + type: string + type: array + x-kubernetes-list-type: atomic + path: + type: string + readOnly: + type: boolean + secretFile: + type: string + secretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + user: + type: string + required: + - monitors + type: object + cinder: + properties: + fsType: + type: string + readOnly: + type: boolean + secretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + volumeID: + type: string + required: + - volumeID + type: object + configMap: + properties: + defaultMode: + format: int32 + type: integer + items: + items: + properties: + key: + type: string + mode: + format: int32 + type: integer + path: + type: string + required: + - key + - path + type: object + type: array + x-kubernetes-list-type: atomic + name: + default: "" + type: string + optional: + type: boolean + type: object + x-kubernetes-map-type: atomic + csi: + properties: + driver: + type: string + fsType: + type: string + nodePublishSecretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + readOnly: + type: boolean + volumeAttributes: + additionalProperties: + type: string + type: object + required: + - driver + type: object + downwardAPI: + properties: + defaultMode: + format: int32 + type: integer + items: + items: + properties: + fieldRef: + properties: + apiVersion: + type: string + fieldPath: + type: string + required: + - fieldPath + type: object + x-kubernetes-map-type: atomic + mode: + format: int32 + type: integer + path: + type: string + resourceFieldRef: + properties: + containerName: + type: string + divisor: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + resource: + type: string + required: + - resource + type: object + x-kubernetes-map-type: atomic + required: + - path + type: object + type: array + x-kubernetes-list-type: atomic + type: object + emptyDir: + properties: + medium: + type: string + sizeLimit: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + ephemeral: + properties: + volumeClaimTemplate: + properties: + metadata: + type: object + spec: + properties: + accessModes: + items: + type: string + type: array + x-kubernetes-list-type: atomic + dataSource: + properties: + apiGroup: + type: string + kind: + type: string + name: + type: string + required: + - kind + - name + type: object + x-kubernetes-map-type: atomic + dataSourceRef: + properties: + apiGroup: + type: string + kind: + type: string + name: + type: string + namespace: + type: string + required: + - kind + - name + type: object + resources: + properties: + limits: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + requests: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + type: object + selector: + properties: + matchExpressions: + items: + properties: + key: + type: string + operator: + type: string + values: + items: + type: string + type: array + x-kubernetes-list-type: atomic + required: + - key + - operator + type: object + type: array + x-kubernetes-list-type: atomic + matchLabels: + additionalProperties: + type: string + type: object + type: object + x-kubernetes-map-type: atomic + storageClassName: + type: string + volumeAttributesClassName: + type: string + volumeMode: + type: string + volumeName: + type: string + type: object + required: + - spec + type: object + type: object + fc: + properties: + fsType: + type: string + lun: + format: int32 + type: integer + readOnly: + type: boolean + targetWWNs: + items: + type: string + type: array + x-kubernetes-list-type: atomic + wwids: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + flexVolume: + properties: + driver: + type: string + fsType: + type: string + options: + additionalProperties: + type: string + type: object + readOnly: + type: boolean + secretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + required: + - driver + type: object + flocker: + properties: + datasetName: + type: string + datasetUUID: + type: string + type: object + gcePersistentDisk: + properties: + fsType: + type: string + partition: + format: int32 + type: integer + pdName: + type: string + readOnly: + type: boolean + required: + - pdName + type: object + gitRepo: + properties: + directory: + type: string + repository: + type: string + revision: + type: string + required: + - repository + type: object + glusterfs: + properties: + endpoints: + type: string + path: + type: string + readOnly: + type: boolean + required: + - endpoints + - path + type: object + hostPath: + properties: + path: + type: string + type: + type: string + required: + - path + type: object + image: + properties: + pullPolicy: + type: string + reference: + type: string + type: object + iscsi: + properties: + chapAuthDiscovery: + type: boolean + chapAuthSession: + type: boolean + fsType: + type: string + initiatorName: + type: string + iqn: + type: string + iscsiInterface: + default: default + type: string + lun: + format: int32 + type: integer + portals: + items: + type: string + type: array + x-kubernetes-list-type: atomic + readOnly: + type: boolean + secretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + targetPortal: + type: string + required: + - iqn + - lun + - targetPortal + type: object + name: + type: string + nfs: + properties: + path: + type: string + readOnly: + type: boolean + server: + type: string + required: + - path + - server + type: object + persistentVolumeClaim: + properties: + claimName: + type: string + readOnly: + type: boolean + required: + - claimName + type: object + photonPersistentDisk: + properties: + fsType: + type: string + pdID: + type: string + required: + - pdID + type: object + portworxVolume: + properties: + fsType: + type: string + readOnly: + type: boolean + volumeID: + type: string + required: + - volumeID + type: object + projected: + properties: + defaultMode: + format: int32 + type: integer + sources: + items: + properties: + clusterTrustBundle: + properties: + labelSelector: + properties: + matchExpressions: + items: + properties: + key: + type: string + operator: + type: string + values: + items: + type: string + type: array + x-kubernetes-list-type: atomic + required: + - key + - operator + type: object + type: array + x-kubernetes-list-type: atomic + matchLabels: + additionalProperties: + type: string + type: object + type: object + x-kubernetes-map-type: atomic + name: + type: string + optional: + type: boolean + path: + type: string + signerName: + type: string + required: + - path + type: object + configMap: + properties: + items: + items: + properties: + key: + type: string + mode: + format: int32 + type: integer + path: + type: string + required: + - key + - path + type: object + type: array + x-kubernetes-list-type: atomic + name: + default: "" + type: string + optional: + type: boolean + type: object + x-kubernetes-map-type: atomic + downwardAPI: + properties: + items: + items: + properties: + fieldRef: + properties: + apiVersion: + type: string + fieldPath: + type: string + required: + - fieldPath + type: object + x-kubernetes-map-type: atomic + mode: + format: int32 + type: integer + path: + type: string + resourceFieldRef: + properties: + containerName: + type: string + divisor: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + resource: + type: string + required: + - resource + type: object + x-kubernetes-map-type: atomic + required: + - path + type: object + type: array + x-kubernetes-list-type: atomic + type: object + secret: + properties: + items: + items: + properties: + key: + type: string + mode: + format: int32 + type: integer + path: + type: string + required: + - key + - path + type: object + type: array + x-kubernetes-list-type: atomic + name: + default: "" + type: string + optional: + type: boolean + type: object + x-kubernetes-map-type: atomic + serviceAccountToken: + properties: + audience: + type: string + expirationSeconds: + format: int64 + type: integer + path: + type: string + required: + - path + type: object + type: object + type: array + x-kubernetes-list-type: atomic + type: object + quobyte: + properties: + group: + type: string + readOnly: + type: boolean + registry: + type: string + tenant: + type: string + user: + type: string + volume: + type: string + required: + - registry + - volume + type: object + rbd: + properties: + fsType: + type: string + image: + type: string + keyring: + default: /etc/ceph/keyring + type: string + monitors: + items: + type: string + type: array + x-kubernetes-list-type: atomic + pool: + default: rbd + type: string + readOnly: + type: boolean + secretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + user: + default: admin + type: string + required: + - image + - monitors + type: object + scaleIO: + properties: + fsType: + default: xfs + type: string + gateway: + type: string + protectionDomain: + type: string + readOnly: + type: boolean + secretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + sslEnabled: + type: boolean + storageMode: + default: ThinProvisioned + type: string + storagePool: + type: string + system: + type: string + volumeName: + type: string + required: + - gateway + - secretRef + - system + type: object + secret: + properties: + defaultMode: + format: int32 + type: integer + items: + items: + properties: + key: + type: string + mode: + format: int32 + type: integer + path: + type: string + required: + - key + - path + type: object + type: array + x-kubernetes-list-type: atomic + optional: + type: boolean + secretName: + type: string + type: object + storageos: + properties: + fsType: + type: string + readOnly: + type: boolean + secretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + volumeName: + type: string + volumeNamespace: + type: string + type: object + vsphereVolume: + properties: + fsType: + type: string + storagePolicyID: + type: string + storagePolicyName: + type: string + volumePath: + type: string + required: + - volumePath + type: object + required: + - name + type: object + type: array + sidecars: + items: + properties: + args: + items: + type: string + type: array + x-kubernetes-list-type: atomic + command: + items: + type: string + type: array + x-kubernetes-list-type: atomic + env: + items: + properties: + name: + type: string + value: + type: string + valueFrom: + properties: + configMapKeyRef: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + fieldRef: + properties: + apiVersion: + type: string + fieldPath: + type: string + required: + - fieldPath + type: object + x-kubernetes-map-type: atomic + resourceFieldRef: + properties: + containerName: + type: string + divisor: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + resource: + type: string + required: + - resource + type: object + x-kubernetes-map-type: atomic + secretKeyRef: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: object + required: + - name + type: object + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + envFrom: + items: + properties: + configMapRef: + properties: + name: + default: "" + type: string + optional: + type: boolean + type: object + x-kubernetes-map-type: atomic + prefix: + type: string + secretRef: + properties: + name: + default: "" + type: string + optional: + type: boolean + type: object + x-kubernetes-map-type: atomic + type: object + type: array + x-kubernetes-list-type: atomic + image: + type: string + imagePullPolicy: + type: string + lifecycle: + properties: + postStart: + properties: + exec: + properties: + command: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + httpGet: + properties: + host: + type: string + httpHeaders: + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + x-kubernetes-list-type: atomic + path: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + scheme: + type: string + required: + - port + type: object + sleep: + properties: + seconds: + format: int64 + type: integer + required: + - seconds + type: object + tcpSocket: + properties: + host: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + required: + - port + type: object + type: object + preStop: + properties: + exec: + properties: + command: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + httpGet: + properties: + host: + type: string + httpHeaders: + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + x-kubernetes-list-type: atomic + path: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + scheme: + type: string + required: + - port + type: object + sleep: + properties: + seconds: + format: int64 + type: integer + required: + - seconds + type: object + tcpSocket: + properties: + host: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + required: + - port + type: object + type: object + type: object + livenessProbe: + properties: + exec: + properties: + command: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + failureThreshold: + format: int32 + type: integer + grpc: + properties: + port: + format: int32 + type: integer + service: + default: "" + type: string + required: + - port + type: object + httpGet: + properties: + host: + type: string + httpHeaders: + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + x-kubernetes-list-type: atomic + path: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + scheme: + type: string + required: + - port + type: object + initialDelaySeconds: + format: int32 + type: integer + periodSeconds: + format: int32 + type: integer + successThreshold: + format: int32 + type: integer + tcpSocket: + properties: + host: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + required: + - port + type: object + terminationGracePeriodSeconds: + format: int64 + type: integer + timeoutSeconds: + format: int32 + type: integer + type: object + name: + type: string + ports: + items: + properties: + containerPort: + format: int32 + type: integer + hostIP: + type: string + hostPort: + format: int32 + type: integer + name: + type: string + protocol: + default: TCP + type: string + required: + - containerPort + type: object + type: array + x-kubernetes-list-map-keys: + - containerPort + - protocol + x-kubernetes-list-type: map + readinessProbe: + properties: + exec: + properties: + command: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + failureThreshold: + format: int32 + type: integer + grpc: + properties: + port: + format: int32 + type: integer + service: + default: "" + type: string + required: + - port + type: object + httpGet: + properties: + host: + type: string + httpHeaders: + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + x-kubernetes-list-type: atomic + path: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + scheme: + type: string + required: + - port + type: object + initialDelaySeconds: + format: int32 + type: integer + periodSeconds: + format: int32 + type: integer + successThreshold: + format: int32 + type: integer + tcpSocket: + properties: + host: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + required: + - port + type: object + terminationGracePeriodSeconds: + format: int64 + type: integer + timeoutSeconds: + format: int32 + type: integer + type: object + resizePolicy: + items: + properties: + resourceName: + type: string + restartPolicy: + type: string + required: + - resourceName + - restartPolicy + type: object + type: array + x-kubernetes-list-type: atomic + resources: + properties: + claims: + items: + properties: + name: + type: string + request: + type: string + required: + - name + type: object + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + limits: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + requests: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + type: object + restartPolicy: + type: string + securityContext: + properties: + allowPrivilegeEscalation: + type: boolean + appArmorProfile: + properties: + localhostProfile: + type: string + type: + type: string + required: + - type + type: object + capabilities: + properties: + add: + items: + type: string + type: array + x-kubernetes-list-type: atomic + drop: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + privileged: + type: boolean + procMount: + type: string + readOnlyRootFilesystem: + type: boolean + runAsGroup: + format: int64 + type: integer + runAsNonRoot: + type: boolean + runAsUser: + format: int64 + type: integer + seLinuxOptions: + properties: + level: + type: string + role: + type: string + type: + type: string + user: + type: string + type: object + seccompProfile: + properties: + localhostProfile: + type: string + type: + type: string + required: + - type + type: object + windowsOptions: + properties: + gmsaCredentialSpec: + type: string + gmsaCredentialSpecName: + type: string + hostProcess: + type: boolean + runAsUserName: + type: string + type: object + type: object + startupProbe: + properties: + exec: + properties: + command: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + failureThreshold: + format: int32 + type: integer + grpc: + properties: + port: + format: int32 + type: integer + service: + default: "" + type: string + required: + - port + type: object + httpGet: + properties: + host: + type: string + httpHeaders: + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + x-kubernetes-list-type: atomic + path: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + scheme: + type: string + required: + - port + type: object + initialDelaySeconds: + format: int32 + type: integer + periodSeconds: + format: int32 + type: integer + successThreshold: + format: int32 + type: integer + tcpSocket: + properties: + host: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + required: + - port + type: object + terminationGracePeriodSeconds: + format: int64 + type: integer + timeoutSeconds: + format: int32 + type: integer + type: object + stdin: + type: boolean + stdinOnce: + type: boolean + terminationMessagePath: + type: string + terminationMessagePolicy: + type: string + tty: + type: boolean + volumeDevices: + items: + properties: + devicePath: + type: string + name: + type: string + required: + - devicePath + - name + type: object + type: array + x-kubernetes-list-map-keys: + - devicePath + x-kubernetes-list-type: map + volumeMounts: + items: + properties: + mountPath: + type: string + mountPropagation: + type: string + name: + type: string + readOnly: + type: boolean + recursiveReadOnly: + type: string + subPath: + type: string + subPathExpr: + type: string + required: + - mountPath + - name + type: object + type: array + x-kubernetes-list-map-keys: + - mountPath + x-kubernetes-list-type: map + workingDir: + type: string + required: + - name + type: object + type: array + type: object + status: + properties: + conditions: + items: + properties: + lastTransitionTime: + format: date-time + type: string + message: + maxLength: 32768 + type: string + observedGeneration: + format: int64 + minimum: 0 + type: integer + reason: + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + enum: + - "True" + - "False" + - Unknown + type: string + type: + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + exporterConfig: + type: string + replicas: + type: integer + status: + type: string + version: + type: string + required: + - conditions + - exporterConfig + - version + type: object + type: object + served: true + storage: false + subresources: + status: {} + - additionalPrinterColumns: + - jsonPath: .status.exporterConfig + name: ExporterConfig + type: string + - jsonPath: .status.status + name: Status + type: string + - jsonPath: .status.version + name: Version + type: string name: v1alpha1 schema: openAPIV3Schema: - description: DatabaseObserver is the Schema for the databaseobservers API properties: apiVersion: - description: 'APIVersion defines the versioned schema of this representation - of an object. Servers should convert recognized schemas to the latest - internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: - description: 'Kind is a string value representing the REST resource this - object represents. Servers may infer this from the endpoint the client - submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: - description: DatabaseObserverSpec defines the desired state of DatabaseObserver properties: + configuration: + properties: + configMap: + properties: + key: + type: string + name: + type: string + type: object + type: object + database: + properties: + dbConnectionString: + properties: + key: + type: string + secret: + type: string + type: object + dbPassword: + properties: + key: + type: string + secret: + type: string + vaultOCID: + type: string + vaultSecretName: + type: string + type: object + dbUser: + properties: + key: + type: string + secret: + type: string + type: object + dbWallet: + properties: + key: + type: string + secret: + type: string + type: object + type: object + exporter: + properties: + deployment: + properties: + args: + items: + type: string + type: array + commands: + items: + type: string + type: array + env: + additionalProperties: + type: string + type: object + image: + type: string + labels: + additionalProperties: + type: string + type: object + podTemplate: + properties: + labels: + additionalProperties: + type: string + type: object + securityContext: + properties: + appArmorProfile: + properties: + localhostProfile: + type: string + type: + type: string + required: + - type + type: object + fsGroup: + format: int64 + type: integer + fsGroupChangePolicy: + type: string + runAsGroup: + format: int64 + type: integer + runAsNonRoot: + type: boolean + runAsUser: + format: int64 + type: integer + seLinuxOptions: + properties: + level: + type: string + role: + type: string + type: + type: string + user: + type: string + type: object + seccompProfile: + properties: + localhostProfile: + type: string + type: + type: string + required: + - type + type: object + supplementalGroups: + items: + format: int64 + type: integer + type: array + x-kubernetes-list-type: atomic + supplementalGroupsPolicy: + type: string + sysctls: + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + x-kubernetes-list-type: atomic + windowsOptions: + properties: + gmsaCredentialSpec: + type: string + gmsaCredentialSpecName: + type: string + hostProcess: + type: boolean + runAsUserName: + type: string + type: object + type: object + type: object + securityContext: + properties: + allowPrivilegeEscalation: + type: boolean + appArmorProfile: + properties: + localhostProfile: + type: string + type: + type: string + required: + - type + type: object + capabilities: + properties: + add: + items: + type: string + type: array + x-kubernetes-list-type: atomic + drop: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + privileged: + type: boolean + procMount: + type: string + readOnlyRootFilesystem: + type: boolean + runAsGroup: + format: int64 + type: integer + runAsNonRoot: + type: boolean + runAsUser: + format: int64 + type: integer + seLinuxOptions: + properties: + level: + type: string + role: + type: string + type: + type: string + user: + type: string + type: object + seccompProfile: + properties: + localhostProfile: + type: string + type: + type: string + required: + - type + type: object + windowsOptions: + properties: + gmsaCredentialSpec: + type: string + gmsaCredentialSpecName: + type: string + hostProcess: + type: boolean + runAsUserName: + type: string + type: object + type: object + type: object + service: + properties: + labels: + additionalProperties: + type: string + type: object + ports: + items: + properties: + appProtocol: + type: string + name: + type: string + nodePort: + format: int32 + type: integer + port: + format: int32 + type: integer + protocol: + default: TCP + type: string + targetPort: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + required: + - port + type: object + type: array + type: object + type: object + inheritLabels: + items: + type: string + type: array + log: + properties: + filename: + type: string + path: + type: string + volume: + properties: + name: + type: string + persistentVolumeClaim: + properties: + claimName: + type: string + type: object + type: object + type: object + ociConfig: + properties: + configMapName: + type: string + secretName: + type: string + type: object + prometheus: + properties: + serviceMonitor: + properties: + endpoints: + items: + properties: + authorization: + properties: + credentials: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: + type: string + type: object + basicAuth: + properties: + password: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + username: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: object + bearerTokenFile: + type: string + bearerTokenSecret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + enableHttp2: + type: boolean + filterRunning: + type: boolean + followRedirects: + type: boolean + honorLabels: + type: boolean + honorTimestamps: + type: boolean + interval: + pattern: ^(0|(([0-9]+)y)?(([0-9]+)w)?(([0-9]+)d)?(([0-9]+)h)?(([0-9]+)m)?(([0-9]+)s)?(([0-9]+)ms)?)$ + type: string + metricRelabelings: + items: + properties: + action: + default: replace + enum: + - replace + - Replace + - keep + - Keep + - drop + - Drop + - hashmod + - HashMod + - labelmap + - LabelMap + - labeldrop + - LabelDrop + - labelkeep + - LabelKeep + - lowercase + - Lowercase + - uppercase + - Uppercase + - keepequal + - KeepEqual + - dropequal + - DropEqual + type: string + modulus: + format: int64 + type: integer + regex: + type: string + replacement: + type: string + separator: + type: string + sourceLabels: + items: + pattern: ^[a-zA-Z_][a-zA-Z0-9_]*$ + type: string + type: array + targetLabel: + type: string + type: object + type: array + oauth2: + properties: + clientId: + properties: + configMap: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + secret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: object + clientSecret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + endpointParams: + additionalProperties: + type: string + type: object + noProxy: + type: string + proxyConnectHeader: + additionalProperties: + items: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: array + type: object + x-kubernetes-map-type: atomic + proxyFromEnvironment: + type: boolean + proxyUrl: + pattern: ^http(s)?://.+$ + type: string + scopes: + items: + type: string + type: array + tlsConfig: + properties: + ca: + properties: + configMap: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + secret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: object + cert: + properties: + configMap: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + secret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: object + insecureSkipVerify: + type: boolean + keySecret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + maxVersion: + enum: + - TLS10 + - TLS11 + - TLS12 + - TLS13 + type: string + minVersion: + enum: + - TLS10 + - TLS11 + - TLS12 + - TLS13 + type: string + serverName: + type: string + type: object + tokenUrl: + minLength: 1 + type: string + required: + - clientId + - clientSecret + - tokenUrl + type: object + params: + additionalProperties: + items: + type: string + type: array + type: object + path: + type: string + port: + type: string + proxyUrl: + type: string + relabelings: + items: + properties: + action: + default: replace + enum: + - replace + - Replace + - keep + - Keep + - drop + - Drop + - hashmod + - HashMod + - labelmap + - LabelMap + - labeldrop + - LabelDrop + - labelkeep + - LabelKeep + - lowercase + - Lowercase + - uppercase + - Uppercase + - keepequal + - KeepEqual + - dropequal + - DropEqual + type: string + modulus: + format: int64 + type: integer + regex: + type: string + replacement: + type: string + separator: + type: string + sourceLabels: + items: + pattern: ^[a-zA-Z_][a-zA-Z0-9_]*$ + type: string + type: array + targetLabel: + type: string + type: object + type: array + scheme: + enum: + - http + - https + type: string + scrapeTimeout: + pattern: ^(0|(([0-9]+)y)?(([0-9]+)w)?(([0-9]+)d)?(([0-9]+)h)?(([0-9]+)m)?(([0-9]+)s)?(([0-9]+)ms)?)$ + type: string + targetPort: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + tlsConfig: + properties: + ca: + properties: + configMap: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + secret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: object + caFile: + type: string + cert: + properties: + configMap: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + secret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: object + certFile: + type: string + insecureSkipVerify: + type: boolean + keyFile: + type: string + keySecret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + maxVersion: + enum: + - TLS10 + - TLS11 + - TLS12 + - TLS13 + type: string + minVersion: + enum: + - TLS10 + - TLS11 + - TLS12 + - TLS13 + type: string + serverName: + type: string + type: object + trackTimestampsStaleness: + type: boolean + type: object + type: array + labels: + additionalProperties: + type: string + type: object + namespaceSelector: + properties: + any: + type: boolean + matchNames: + items: + type: string + type: array + type: object + type: object + type: object + replicas: + format: int32 + type: integer + sidecarVolumes: + items: + properties: + awsElasticBlockStore: + properties: + fsType: + type: string + partition: + format: int32 + type: integer + readOnly: + type: boolean + volumeID: + type: string + required: + - volumeID + type: object + azureDisk: + properties: + cachingMode: + type: string + diskName: + type: string + diskURI: + type: string + fsType: + default: ext4 + type: string + kind: + type: string + readOnly: + default: false + type: boolean + required: + - diskName + - diskURI + type: object + azureFile: + properties: + readOnly: + type: boolean + secretName: + type: string + shareName: + type: string + required: + - secretName + - shareName + type: object + cephfs: + properties: + monitors: + items: + type: string + type: array + x-kubernetes-list-type: atomic + path: + type: string + readOnly: + type: boolean + secretFile: + type: string + secretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + user: + type: string + required: + - monitors + type: object + cinder: + properties: + fsType: + type: string + readOnly: + type: boolean + secretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + volumeID: + type: string + required: + - volumeID + type: object + configMap: + properties: + defaultMode: + format: int32 + type: integer + items: + items: + properties: + key: + type: string + mode: + format: int32 + type: integer + path: + type: string + required: + - key + - path + type: object + type: array + x-kubernetes-list-type: atomic + name: + default: "" + type: string + optional: + type: boolean + type: object + x-kubernetes-map-type: atomic + csi: + properties: + driver: + type: string + fsType: + type: string + nodePublishSecretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + readOnly: + type: boolean + volumeAttributes: + additionalProperties: + type: string + type: object + required: + - driver + type: object + downwardAPI: + properties: + defaultMode: + format: int32 + type: integer + items: + items: + properties: + fieldRef: + properties: + apiVersion: + type: string + fieldPath: + type: string + required: + - fieldPath + type: object + x-kubernetes-map-type: atomic + mode: + format: int32 + type: integer + path: + type: string + resourceFieldRef: + properties: + containerName: + type: string + divisor: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + resource: + type: string + required: + - resource + type: object + x-kubernetes-map-type: atomic + required: + - path + type: object + type: array + x-kubernetes-list-type: atomic + type: object + emptyDir: + properties: + medium: + type: string + sizeLimit: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + ephemeral: + properties: + volumeClaimTemplate: + properties: + metadata: + type: object + spec: + properties: + accessModes: + items: + type: string + type: array + x-kubernetes-list-type: atomic + dataSource: + properties: + apiGroup: + type: string + kind: + type: string + name: + type: string + required: + - kind + - name + type: object + x-kubernetes-map-type: atomic + dataSourceRef: + properties: + apiGroup: + type: string + kind: + type: string + name: + type: string + namespace: + type: string + required: + - kind + - name + type: object + resources: + properties: + limits: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + requests: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + type: object + selector: + properties: + matchExpressions: + items: + properties: + key: + type: string + operator: + type: string + values: + items: + type: string + type: array + x-kubernetes-list-type: atomic + required: + - key + - operator + type: object + type: array + x-kubernetes-list-type: atomic + matchLabels: + additionalProperties: + type: string + type: object + type: object + x-kubernetes-map-type: atomic + storageClassName: + type: string + volumeAttributesClassName: + type: string + volumeMode: + type: string + volumeName: + type: string + type: object + required: + - spec + type: object + type: object + fc: + properties: + fsType: + type: string + lun: + format: int32 + type: integer + readOnly: + type: boolean + targetWWNs: + items: + type: string + type: array + x-kubernetes-list-type: atomic + wwids: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + flexVolume: + properties: + driver: + type: string + fsType: + type: string + options: + additionalProperties: + type: string + type: object + readOnly: + type: boolean + secretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + required: + - driver + type: object + flocker: + properties: + datasetName: + type: string + datasetUUID: + type: string + type: object + gcePersistentDisk: + properties: + fsType: + type: string + partition: + format: int32 + type: integer + pdName: + type: string + readOnly: + type: boolean + required: + - pdName + type: object + gitRepo: + properties: + directory: + type: string + repository: + type: string + revision: + type: string + required: + - repository + type: object + glusterfs: + properties: + endpoints: + type: string + path: + type: string + readOnly: + type: boolean + required: + - endpoints + - path + type: object + hostPath: + properties: + path: + type: string + type: + type: string + required: + - path + type: object + image: + properties: + pullPolicy: + type: string + reference: + type: string + type: object + iscsi: + properties: + chapAuthDiscovery: + type: boolean + chapAuthSession: + type: boolean + fsType: + type: string + initiatorName: + type: string + iqn: + type: string + iscsiInterface: + default: default + type: string + lun: + format: int32 + type: integer + portals: + items: + type: string + type: array + x-kubernetes-list-type: atomic + readOnly: + type: boolean + secretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + targetPortal: + type: string + required: + - iqn + - lun + - targetPortal + type: object + name: + type: string + nfs: + properties: + path: + type: string + readOnly: + type: boolean + server: + type: string + required: + - path + - server + type: object + persistentVolumeClaim: + properties: + claimName: + type: string + readOnly: + type: boolean + required: + - claimName + type: object + photonPersistentDisk: + properties: + fsType: + type: string + pdID: + type: string + required: + - pdID + type: object + portworxVolume: + properties: + fsType: + type: string + readOnly: + type: boolean + volumeID: + type: string + required: + - volumeID + type: object + projected: + properties: + defaultMode: + format: int32 + type: integer + sources: + items: + properties: + clusterTrustBundle: + properties: + labelSelector: + properties: + matchExpressions: + items: + properties: + key: + type: string + operator: + type: string + values: + items: + type: string + type: array + x-kubernetes-list-type: atomic + required: + - key + - operator + type: object + type: array + x-kubernetes-list-type: atomic + matchLabels: + additionalProperties: + type: string + type: object + type: object + x-kubernetes-map-type: atomic + name: + type: string + optional: + type: boolean + path: + type: string + signerName: + type: string + required: + - path + type: object + configMap: + properties: + items: + items: + properties: + key: + type: string + mode: + format: int32 + type: integer + path: + type: string + required: + - key + - path + type: object + type: array + x-kubernetes-list-type: atomic + name: + default: "" + type: string + optional: + type: boolean + type: object + x-kubernetes-map-type: atomic + downwardAPI: + properties: + items: + items: + properties: + fieldRef: + properties: + apiVersion: + type: string + fieldPath: + type: string + required: + - fieldPath + type: object + x-kubernetes-map-type: atomic + mode: + format: int32 + type: integer + path: + type: string + resourceFieldRef: + properties: + containerName: + type: string + divisor: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + resource: + type: string + required: + - resource + type: object + x-kubernetes-map-type: atomic + required: + - path + type: object + type: array + x-kubernetes-list-type: atomic + type: object + secret: + properties: + items: + items: + properties: + key: + type: string + mode: + format: int32 + type: integer + path: + type: string + required: + - key + - path + type: object + type: array + x-kubernetes-list-type: atomic + name: + default: "" + type: string + optional: + type: boolean + type: object + x-kubernetes-map-type: atomic + serviceAccountToken: + properties: + audience: + type: string + expirationSeconds: + format: int64 + type: integer + path: + type: string + required: + - path + type: object + type: object + type: array + x-kubernetes-list-type: atomic + type: object + quobyte: + properties: + group: + type: string + readOnly: + type: boolean + registry: + type: string + tenant: + type: string + user: + type: string + volume: + type: string + required: + - registry + - volume + type: object + rbd: + properties: + fsType: + type: string + image: + type: string + keyring: + default: /etc/ceph/keyring + type: string + monitors: + items: + type: string + type: array + x-kubernetes-list-type: atomic + pool: + default: rbd + type: string + readOnly: + type: boolean + secretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + user: + default: admin + type: string + required: + - image + - monitors + type: object + scaleIO: + properties: + fsType: + default: xfs + type: string + gateway: + type: string + protectionDomain: + type: string + readOnly: + type: boolean + secretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + sslEnabled: + type: boolean + storageMode: + default: ThinProvisioned + type: string + storagePool: + type: string + system: + type: string + volumeName: + type: string + required: + - gateway + - secretRef + - system + type: object + secret: + properties: + defaultMode: + format: int32 + type: integer + items: + items: + properties: + key: + type: string + mode: + format: int32 + type: integer + path: + type: string + required: + - key + - path + type: object + type: array + x-kubernetes-list-type: atomic + optional: + type: boolean + secretName: + type: string + type: object + storageos: + properties: + fsType: + type: string + readOnly: + type: boolean + secretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + volumeName: + type: string + volumeNamespace: + type: string + type: object + vsphereVolume: + properties: + fsType: + type: string + storagePolicyID: + type: string + storagePolicyName: + type: string + volumePath: + type: string + required: + - volumePath + type: object + required: + - name + type: object + type: array + sidecars: + items: + properties: + args: + items: + type: string + type: array + x-kubernetes-list-type: atomic + command: + items: + type: string + type: array + x-kubernetes-list-type: atomic + env: + items: + properties: + name: + type: string + value: + type: string + valueFrom: + properties: + configMapKeyRef: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + fieldRef: + properties: + apiVersion: + type: string + fieldPath: + type: string + required: + - fieldPath + type: object + x-kubernetes-map-type: atomic + resourceFieldRef: + properties: + containerName: + type: string + divisor: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + resource: + type: string + required: + - resource + type: object + x-kubernetes-map-type: atomic + secretKeyRef: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: object + required: + - name + type: object + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + envFrom: + items: + properties: + configMapRef: + properties: + name: + default: "" + type: string + optional: + type: boolean + type: object + x-kubernetes-map-type: atomic + prefix: + type: string + secretRef: + properties: + name: + default: "" + type: string + optional: + type: boolean + type: object + x-kubernetes-map-type: atomic + type: object + type: array + x-kubernetes-list-type: atomic + image: + type: string + imagePullPolicy: + type: string + lifecycle: + properties: + postStart: + properties: + exec: + properties: + command: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + httpGet: + properties: + host: + type: string + httpHeaders: + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + x-kubernetes-list-type: atomic + path: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + scheme: + type: string + required: + - port + type: object + sleep: + properties: + seconds: + format: int64 + type: integer + required: + - seconds + type: object + tcpSocket: + properties: + host: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + required: + - port + type: object + type: object + preStop: + properties: + exec: + properties: + command: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + httpGet: + properties: + host: + type: string + httpHeaders: + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + x-kubernetes-list-type: atomic + path: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + scheme: + type: string + required: + - port + type: object + sleep: + properties: + seconds: + format: int64 + type: integer + required: + - seconds + type: object + tcpSocket: + properties: + host: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + required: + - port + type: object + type: object + type: object + livenessProbe: + properties: + exec: + properties: + command: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + failureThreshold: + format: int32 + type: integer + grpc: + properties: + port: + format: int32 + type: integer + service: + default: "" + type: string + required: + - port + type: object + httpGet: + properties: + host: + type: string + httpHeaders: + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + x-kubernetes-list-type: atomic + path: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + scheme: + type: string + required: + - port + type: object + initialDelaySeconds: + format: int32 + type: integer + periodSeconds: + format: int32 + type: integer + successThreshold: + format: int32 + type: integer + tcpSocket: + properties: + host: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + required: + - port + type: object + terminationGracePeriodSeconds: + format: int64 + type: integer + timeoutSeconds: + format: int32 + type: integer + type: object + name: + type: string + ports: + items: + properties: + containerPort: + format: int32 + type: integer + hostIP: + type: string + hostPort: + format: int32 + type: integer + name: + type: string + protocol: + default: TCP + type: string + required: + - containerPort + type: object + type: array + x-kubernetes-list-map-keys: + - containerPort + - protocol + x-kubernetes-list-type: map + readinessProbe: + properties: + exec: + properties: + command: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + failureThreshold: + format: int32 + type: integer + grpc: + properties: + port: + format: int32 + type: integer + service: + default: "" + type: string + required: + - port + type: object + httpGet: + properties: + host: + type: string + httpHeaders: + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + x-kubernetes-list-type: atomic + path: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + scheme: + type: string + required: + - port + type: object + initialDelaySeconds: + format: int32 + type: integer + periodSeconds: + format: int32 + type: integer + successThreshold: + format: int32 + type: integer + tcpSocket: + properties: + host: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + required: + - port + type: object + terminationGracePeriodSeconds: + format: int64 + type: integer + timeoutSeconds: + format: int32 + type: integer + type: object + resizePolicy: + items: + properties: + resourceName: + type: string + restartPolicy: + type: string + required: + - resourceName + - restartPolicy + type: object + type: array + x-kubernetes-list-type: atomic + resources: + properties: + claims: + items: + properties: + name: + type: string + request: + type: string + required: + - name + type: object + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + limits: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + requests: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + type: object + restartPolicy: + type: string + securityContext: + properties: + allowPrivilegeEscalation: + type: boolean + appArmorProfile: + properties: + localhostProfile: + type: string + type: + type: string + required: + - type + type: object + capabilities: + properties: + add: + items: + type: string + type: array + x-kubernetes-list-type: atomic + drop: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + privileged: + type: boolean + procMount: + type: string + readOnlyRootFilesystem: + type: boolean + runAsGroup: + format: int64 + type: integer + runAsNonRoot: + type: boolean + runAsUser: + format: int64 + type: integer + seLinuxOptions: + properties: + level: + type: string + role: + type: string + type: + type: string + user: + type: string + type: object + seccompProfile: + properties: + localhostProfile: + type: string + type: + type: string + required: + - type + type: object + windowsOptions: + properties: + gmsaCredentialSpec: + type: string + gmsaCredentialSpecName: + type: string + hostProcess: + type: boolean + runAsUserName: + type: string + type: object + type: object + startupProbe: + properties: + exec: + properties: + command: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + failureThreshold: + format: int32 + type: integer + grpc: + properties: + port: + format: int32 + type: integer + service: + default: "" + type: string + required: + - port + type: object + httpGet: + properties: + host: + type: string + httpHeaders: + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + x-kubernetes-list-type: atomic + path: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + scheme: + type: string + required: + - port + type: object + initialDelaySeconds: + format: int32 + type: integer + periodSeconds: + format: int32 + type: integer + successThreshold: + format: int32 + type: integer + tcpSocket: + properties: + host: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + required: + - port + type: object + terminationGracePeriodSeconds: + format: int64 + type: integer + timeoutSeconds: + format: int32 + type: integer + type: object + stdin: + type: boolean + stdinOnce: + type: boolean + terminationMessagePath: + type: string + terminationMessagePolicy: + type: string + tty: + type: boolean + volumeDevices: + items: + properties: + devicePath: + type: string + name: + type: string + required: + - devicePath + - name + type: object + type: array + x-kubernetes-list-map-keys: + - devicePath + x-kubernetes-list-type: map + volumeMounts: + items: + properties: + mountPath: + type: string + mountPropagation: + type: string + name: + type: string + readOnly: + type: boolean + recursiveReadOnly: + type: string + subPath: + type: string + subPathExpr: + type: string + required: + - mountPath + - name + type: object + type: array + x-kubernetes-list-map-keys: + - mountPath + x-kubernetes-list-type: map + workingDir: + type: string + required: + - name + type: object + type: array + type: object + status: + properties: + conditions: + items: + properties: + lastTransitionTime: + format: date-time + type: string + message: + maxLength: 32768 + type: string + observedGeneration: + format: int64 + minimum: 0 + type: integer + reason: + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + enum: + - "True" + - "False" + - Unknown + type: string + type: + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + exporterConfig: + type: string + replicas: + type: integer + status: + type: string + version: + type: string + required: + - conditions + - exporterConfig + - version + type: object + type: object + served: true + storage: false + subresources: + status: {} + - additionalPrinterColumns: + - jsonPath: .status.exporterConfig + name: ExporterConfig + type: string + - jsonPath: .status.status + name: Status + type: string + - jsonPath: .status.version + name: Version + type: string + name: v4 + schema: + openAPIV3Schema: + properties: + apiVersion: + type: string + kind: + type: string + metadata: + type: object + spec: + properties: + configuration: + properties: + configMap: + properties: + key: + type: string + name: + type: string + type: object + type: object database: - description: DatabaseObserverDatabase defines the database details - used for DatabaseObserver properties: dbConnectionString: properties: @@ -81,29 +4731,236 @@ spec: type: object type: object exporter: - description: DatabaseObserverExporterConfig defines the configuration - details related to the exporters of DatabaseObserver properties: - configuration: + deployment: properties: - configmap: - description: ConfigMapDetails defines the configmap name + args: + items: + type: string + type: array + commands: + items: + type: string + type: array + env: + additionalProperties: + type: string + type: object + image: + type: string + labels: + additionalProperties: + type: string + type: object + podTemplate: properties: - configmapName: - type: string - key: + labels: + additionalProperties: + type: string + type: object + securityContext: + properties: + appArmorProfile: + properties: + localhostProfile: + type: string + type: + type: string + required: + - type + type: object + fsGroup: + format: int64 + type: integer + fsGroupChangePolicy: + type: string + runAsGroup: + format: int64 + type: integer + runAsNonRoot: + type: boolean + runAsUser: + format: int64 + type: integer + seLinuxOptions: + properties: + level: + type: string + role: + type: string + type: + type: string + user: + type: string + type: object + seccompProfile: + properties: + localhostProfile: + type: string + type: + type: string + required: + - type + type: object + supplementalGroups: + items: + format: int64 + type: integer + type: array + x-kubernetes-list-type: atomic + supplementalGroupsPolicy: + type: string + sysctls: + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + x-kubernetes-list-type: atomic + windowsOptions: + properties: + gmsaCredentialSpec: + type: string + gmsaCredentialSpecName: + type: string + hostProcess: + type: boolean + runAsUserName: + type: string + type: object + type: object + type: object + securityContext: + properties: + allowPrivilegeEscalation: + type: boolean + appArmorProfile: + properties: + localhostProfile: + type: string + type: + type: string + required: + - type + type: object + capabilities: + properties: + add: + items: + type: string + type: array + x-kubernetes-list-type: atomic + drop: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + privileged: + type: boolean + procMount: type: string + readOnlyRootFilesystem: + type: boolean + runAsGroup: + format: int64 + type: integer + runAsNonRoot: + type: boolean + runAsUser: + format: int64 + type: integer + seLinuxOptions: + properties: + level: + type: string + role: + type: string + type: + type: string + user: + type: string + type: object + seccompProfile: + properties: + localhostProfile: + type: string + type: + type: string + required: + - type + type: object + windowsOptions: + properties: + gmsaCredentialSpec: + type: string + gmsaCredentialSpecName: + type: string + hostProcess: + type: boolean + runAsUserName: + type: string + type: object type: object type: object - image: - type: string service: - description: DatabaseObserverService defines the exporter service - component of DatabaseObserver properties: - port: - format: int32 - type: integer + labels: + additionalProperties: + type: string + type: object + ports: + items: + properties: + appProtocol: + type: string + name: + type: string + nodePort: + format: int32 + type: integer + port: + format: int32 + type: integer + protocol: + default: TCP + type: string + targetPort: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + required: + - port + type: object + type: array + type: object + type: object + inheritLabels: + items: + type: string + type: array + log: + properties: + filename: + type: string + path: + type: string + volume: + properties: + name: + type: string + persistentVolumeClaim: + properties: + claimName: + type: string + type: object type: object type: object ociConfig: @@ -114,85 +4971,1992 @@ spec: type: string type: object prometheus: - description: PrometheusConfig defines the generated resources for - Prometheus properties: - labels: - additionalProperties: - type: string + serviceMonitor: + properties: + endpoints: + items: + properties: + authorization: + properties: + credentials: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: + type: string + type: object + basicAuth: + properties: + password: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + username: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: object + bearerTokenFile: + type: string + bearerTokenSecret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + enableHttp2: + type: boolean + filterRunning: + type: boolean + followRedirects: + type: boolean + honorLabels: + type: boolean + honorTimestamps: + type: boolean + interval: + pattern: ^(0|(([0-9]+)y)?(([0-9]+)w)?(([0-9]+)d)?(([0-9]+)h)?(([0-9]+)m)?(([0-9]+)s)?(([0-9]+)ms)?)$ + type: string + metricRelabelings: + items: + properties: + action: + default: replace + enum: + - replace + - Replace + - keep + - Keep + - drop + - Drop + - hashmod + - HashMod + - labelmap + - LabelMap + - labeldrop + - LabelDrop + - labelkeep + - LabelKeep + - lowercase + - Lowercase + - uppercase + - Uppercase + - keepequal + - KeepEqual + - dropequal + - DropEqual + type: string + modulus: + format: int64 + type: integer + regex: + type: string + replacement: + type: string + separator: + type: string + sourceLabels: + items: + pattern: ^[a-zA-Z_][a-zA-Z0-9_]*$ + type: string + type: array + targetLabel: + type: string + type: object + type: array + oauth2: + properties: + clientId: + properties: + configMap: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + secret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: object + clientSecret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + endpointParams: + additionalProperties: + type: string + type: object + noProxy: + type: string + proxyConnectHeader: + additionalProperties: + items: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: array + type: object + x-kubernetes-map-type: atomic + proxyFromEnvironment: + type: boolean + proxyUrl: + pattern: ^http(s)?://.+$ + type: string + scopes: + items: + type: string + type: array + tlsConfig: + properties: + ca: + properties: + configMap: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + secret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: object + cert: + properties: + configMap: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + secret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: object + insecureSkipVerify: + type: boolean + keySecret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + maxVersion: + enum: + - TLS10 + - TLS11 + - TLS12 + - TLS13 + type: string + minVersion: + enum: + - TLS10 + - TLS11 + - TLS12 + - TLS13 + type: string + serverName: + type: string + type: object + tokenUrl: + minLength: 1 + type: string + required: + - clientId + - clientSecret + - tokenUrl + type: object + params: + additionalProperties: + items: + type: string + type: array + type: object + path: + type: string + port: + type: string + proxyUrl: + type: string + relabelings: + items: + properties: + action: + default: replace + enum: + - replace + - Replace + - keep + - Keep + - drop + - Drop + - hashmod + - HashMod + - labelmap + - LabelMap + - labeldrop + - LabelDrop + - labelkeep + - LabelKeep + - lowercase + - Lowercase + - uppercase + - Uppercase + - keepequal + - KeepEqual + - dropequal + - DropEqual + type: string + modulus: + format: int64 + type: integer + regex: + type: string + replacement: + type: string + separator: + type: string + sourceLabels: + items: + pattern: ^[a-zA-Z_][a-zA-Z0-9_]*$ + type: string + type: array + targetLabel: + type: string + type: object + type: array + scheme: + enum: + - http + - https + type: string + scrapeTimeout: + pattern: ^(0|(([0-9]+)y)?(([0-9]+)w)?(([0-9]+)d)?(([0-9]+)h)?(([0-9]+)m)?(([0-9]+)s)?(([0-9]+)ms)?)$ + type: string + targetPort: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + tlsConfig: + properties: + ca: + properties: + configMap: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + secret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: object + caFile: + type: string + cert: + properties: + configMap: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + secret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: object + certFile: + type: string + insecureSkipVerify: + type: boolean + keyFile: + type: string + keySecret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + maxVersion: + enum: + - TLS10 + - TLS11 + - TLS12 + - TLS13 + type: string + minVersion: + enum: + - TLS10 + - TLS11 + - TLS12 + - TLS13 + type: string + serverName: + type: string + type: object + trackTimestampsStaleness: + type: boolean + type: object + type: array + labels: + additionalProperties: + type: string + type: object + namespaceSelector: + properties: + any: + type: boolean + matchNames: + items: + type: string + type: array + type: object type: object - port: - type: string type: object replicas: format: int32 type: integer + sidecarVolumes: + items: + properties: + awsElasticBlockStore: + properties: + fsType: + type: string + partition: + format: int32 + type: integer + readOnly: + type: boolean + volumeID: + type: string + required: + - volumeID + type: object + azureDisk: + properties: + cachingMode: + type: string + diskName: + type: string + diskURI: + type: string + fsType: + default: ext4 + type: string + kind: + type: string + readOnly: + default: false + type: boolean + required: + - diskName + - diskURI + type: object + azureFile: + properties: + readOnly: + type: boolean + secretName: + type: string + shareName: + type: string + required: + - secretName + - shareName + type: object + cephfs: + properties: + monitors: + items: + type: string + type: array + x-kubernetes-list-type: atomic + path: + type: string + readOnly: + type: boolean + secretFile: + type: string + secretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + user: + type: string + required: + - monitors + type: object + cinder: + properties: + fsType: + type: string + readOnly: + type: boolean + secretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + volumeID: + type: string + required: + - volumeID + type: object + configMap: + properties: + defaultMode: + format: int32 + type: integer + items: + items: + properties: + key: + type: string + mode: + format: int32 + type: integer + path: + type: string + required: + - key + - path + type: object + type: array + x-kubernetes-list-type: atomic + name: + default: "" + type: string + optional: + type: boolean + type: object + x-kubernetes-map-type: atomic + csi: + properties: + driver: + type: string + fsType: + type: string + nodePublishSecretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + readOnly: + type: boolean + volumeAttributes: + additionalProperties: + type: string + type: object + required: + - driver + type: object + downwardAPI: + properties: + defaultMode: + format: int32 + type: integer + items: + items: + properties: + fieldRef: + properties: + apiVersion: + type: string + fieldPath: + type: string + required: + - fieldPath + type: object + x-kubernetes-map-type: atomic + mode: + format: int32 + type: integer + path: + type: string + resourceFieldRef: + properties: + containerName: + type: string + divisor: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + resource: + type: string + required: + - resource + type: object + x-kubernetes-map-type: atomic + required: + - path + type: object + type: array + x-kubernetes-list-type: atomic + type: object + emptyDir: + properties: + medium: + type: string + sizeLimit: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + ephemeral: + properties: + volumeClaimTemplate: + properties: + metadata: + type: object + spec: + properties: + accessModes: + items: + type: string + type: array + x-kubernetes-list-type: atomic + dataSource: + properties: + apiGroup: + type: string + kind: + type: string + name: + type: string + required: + - kind + - name + type: object + x-kubernetes-map-type: atomic + dataSourceRef: + properties: + apiGroup: + type: string + kind: + type: string + name: + type: string + namespace: + type: string + required: + - kind + - name + type: object + resources: + properties: + limits: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + requests: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + type: object + selector: + properties: + matchExpressions: + items: + properties: + key: + type: string + operator: + type: string + values: + items: + type: string + type: array + x-kubernetes-list-type: atomic + required: + - key + - operator + type: object + type: array + x-kubernetes-list-type: atomic + matchLabels: + additionalProperties: + type: string + type: object + type: object + x-kubernetes-map-type: atomic + storageClassName: + type: string + volumeAttributesClassName: + type: string + volumeMode: + type: string + volumeName: + type: string + type: object + required: + - spec + type: object + type: object + fc: + properties: + fsType: + type: string + lun: + format: int32 + type: integer + readOnly: + type: boolean + targetWWNs: + items: + type: string + type: array + x-kubernetes-list-type: atomic + wwids: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + flexVolume: + properties: + driver: + type: string + fsType: + type: string + options: + additionalProperties: + type: string + type: object + readOnly: + type: boolean + secretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + required: + - driver + type: object + flocker: + properties: + datasetName: + type: string + datasetUUID: + type: string + type: object + gcePersistentDisk: + properties: + fsType: + type: string + partition: + format: int32 + type: integer + pdName: + type: string + readOnly: + type: boolean + required: + - pdName + type: object + gitRepo: + properties: + directory: + type: string + repository: + type: string + revision: + type: string + required: + - repository + type: object + glusterfs: + properties: + endpoints: + type: string + path: + type: string + readOnly: + type: boolean + required: + - endpoints + - path + type: object + hostPath: + properties: + path: + type: string + type: + type: string + required: + - path + type: object + image: + properties: + pullPolicy: + type: string + reference: + type: string + type: object + iscsi: + properties: + chapAuthDiscovery: + type: boolean + chapAuthSession: + type: boolean + fsType: + type: string + initiatorName: + type: string + iqn: + type: string + iscsiInterface: + default: default + type: string + lun: + format: int32 + type: integer + portals: + items: + type: string + type: array + x-kubernetes-list-type: atomic + readOnly: + type: boolean + secretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + targetPortal: + type: string + required: + - iqn + - lun + - targetPortal + type: object + name: + type: string + nfs: + properties: + path: + type: string + readOnly: + type: boolean + server: + type: string + required: + - path + - server + type: object + persistentVolumeClaim: + properties: + claimName: + type: string + readOnly: + type: boolean + required: + - claimName + type: object + photonPersistentDisk: + properties: + fsType: + type: string + pdID: + type: string + required: + - pdID + type: object + portworxVolume: + properties: + fsType: + type: string + readOnly: + type: boolean + volumeID: + type: string + required: + - volumeID + type: object + projected: + properties: + defaultMode: + format: int32 + type: integer + sources: + items: + properties: + clusterTrustBundle: + properties: + labelSelector: + properties: + matchExpressions: + items: + properties: + key: + type: string + operator: + type: string + values: + items: + type: string + type: array + x-kubernetes-list-type: atomic + required: + - key + - operator + type: object + type: array + x-kubernetes-list-type: atomic + matchLabels: + additionalProperties: + type: string + type: object + type: object + x-kubernetes-map-type: atomic + name: + type: string + optional: + type: boolean + path: + type: string + signerName: + type: string + required: + - path + type: object + configMap: + properties: + items: + items: + properties: + key: + type: string + mode: + format: int32 + type: integer + path: + type: string + required: + - key + - path + type: object + type: array + x-kubernetes-list-type: atomic + name: + default: "" + type: string + optional: + type: boolean + type: object + x-kubernetes-map-type: atomic + downwardAPI: + properties: + items: + items: + properties: + fieldRef: + properties: + apiVersion: + type: string + fieldPath: + type: string + required: + - fieldPath + type: object + x-kubernetes-map-type: atomic + mode: + format: int32 + type: integer + path: + type: string + resourceFieldRef: + properties: + containerName: + type: string + divisor: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + resource: + type: string + required: + - resource + type: object + x-kubernetes-map-type: atomic + required: + - path + type: object + type: array + x-kubernetes-list-type: atomic + type: object + secret: + properties: + items: + items: + properties: + key: + type: string + mode: + format: int32 + type: integer + path: + type: string + required: + - key + - path + type: object + type: array + x-kubernetes-list-type: atomic + name: + default: "" + type: string + optional: + type: boolean + type: object + x-kubernetes-map-type: atomic + serviceAccountToken: + properties: + audience: + type: string + expirationSeconds: + format: int64 + type: integer + path: + type: string + required: + - path + type: object + type: object + type: array + x-kubernetes-list-type: atomic + type: object + quobyte: + properties: + group: + type: string + readOnly: + type: boolean + registry: + type: string + tenant: + type: string + user: + type: string + volume: + type: string + required: + - registry + - volume + type: object + rbd: + properties: + fsType: + type: string + image: + type: string + keyring: + default: /etc/ceph/keyring + type: string + monitors: + items: + type: string + type: array + x-kubernetes-list-type: atomic + pool: + default: rbd + type: string + readOnly: + type: boolean + secretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + user: + default: admin + type: string + required: + - image + - monitors + type: object + scaleIO: + properties: + fsType: + default: xfs + type: string + gateway: + type: string + protectionDomain: + type: string + readOnly: + type: boolean + secretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + sslEnabled: + type: boolean + storageMode: + default: ThinProvisioned + type: string + storagePool: + type: string + system: + type: string + volumeName: + type: string + required: + - gateway + - secretRef + - system + type: object + secret: + properties: + defaultMode: + format: int32 + type: integer + items: + items: + properties: + key: + type: string + mode: + format: int32 + type: integer + path: + type: string + required: + - key + - path + type: object + type: array + x-kubernetes-list-type: atomic + optional: + type: boolean + secretName: + type: string + type: object + storageos: + properties: + fsType: + type: string + readOnly: + type: boolean + secretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + volumeName: + type: string + volumeNamespace: + type: string + type: object + vsphereVolume: + properties: + fsType: + type: string + storagePolicyID: + type: string + storagePolicyName: + type: string + volumePath: + type: string + required: + - volumePath + type: object + required: + - name + type: object + type: array + sidecars: + items: + properties: + args: + items: + type: string + type: array + x-kubernetes-list-type: atomic + command: + items: + type: string + type: array + x-kubernetes-list-type: atomic + env: + items: + properties: + name: + type: string + value: + type: string + valueFrom: + properties: + configMapKeyRef: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + fieldRef: + properties: + apiVersion: + type: string + fieldPath: + type: string + required: + - fieldPath + type: object + x-kubernetes-map-type: atomic + resourceFieldRef: + properties: + containerName: + type: string + divisor: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + resource: + type: string + required: + - resource + type: object + x-kubernetes-map-type: atomic + secretKeyRef: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: object + required: + - name + type: object + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + envFrom: + items: + properties: + configMapRef: + properties: + name: + default: "" + type: string + optional: + type: boolean + type: object + x-kubernetes-map-type: atomic + prefix: + type: string + secretRef: + properties: + name: + default: "" + type: string + optional: + type: boolean + type: object + x-kubernetes-map-type: atomic + type: object + type: array + x-kubernetes-list-type: atomic + image: + type: string + imagePullPolicy: + type: string + lifecycle: + properties: + postStart: + properties: + exec: + properties: + command: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + httpGet: + properties: + host: + type: string + httpHeaders: + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + x-kubernetes-list-type: atomic + path: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + scheme: + type: string + required: + - port + type: object + sleep: + properties: + seconds: + format: int64 + type: integer + required: + - seconds + type: object + tcpSocket: + properties: + host: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + required: + - port + type: object + type: object + preStop: + properties: + exec: + properties: + command: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + httpGet: + properties: + host: + type: string + httpHeaders: + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + x-kubernetes-list-type: atomic + path: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + scheme: + type: string + required: + - port + type: object + sleep: + properties: + seconds: + format: int64 + type: integer + required: + - seconds + type: object + tcpSocket: + properties: + host: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + required: + - port + type: object + type: object + type: object + livenessProbe: + properties: + exec: + properties: + command: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + failureThreshold: + format: int32 + type: integer + grpc: + properties: + port: + format: int32 + type: integer + service: + default: "" + type: string + required: + - port + type: object + httpGet: + properties: + host: + type: string + httpHeaders: + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + x-kubernetes-list-type: atomic + path: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + scheme: + type: string + required: + - port + type: object + initialDelaySeconds: + format: int32 + type: integer + periodSeconds: + format: int32 + type: integer + successThreshold: + format: int32 + type: integer + tcpSocket: + properties: + host: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + required: + - port + type: object + terminationGracePeriodSeconds: + format: int64 + type: integer + timeoutSeconds: + format: int32 + type: integer + type: object + name: + type: string + ports: + items: + properties: + containerPort: + format: int32 + type: integer + hostIP: + type: string + hostPort: + format: int32 + type: integer + name: + type: string + protocol: + default: TCP + type: string + required: + - containerPort + type: object + type: array + x-kubernetes-list-map-keys: + - containerPort + - protocol + x-kubernetes-list-type: map + readinessProbe: + properties: + exec: + properties: + command: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + failureThreshold: + format: int32 + type: integer + grpc: + properties: + port: + format: int32 + type: integer + service: + default: "" + type: string + required: + - port + type: object + httpGet: + properties: + host: + type: string + httpHeaders: + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + x-kubernetes-list-type: atomic + path: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + scheme: + type: string + required: + - port + type: object + initialDelaySeconds: + format: int32 + type: integer + periodSeconds: + format: int32 + type: integer + successThreshold: + format: int32 + type: integer + tcpSocket: + properties: + host: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + required: + - port + type: object + terminationGracePeriodSeconds: + format: int64 + type: integer + timeoutSeconds: + format: int32 + type: integer + type: object + resizePolicy: + items: + properties: + resourceName: + type: string + restartPolicy: + type: string + required: + - resourceName + - restartPolicy + type: object + type: array + x-kubernetes-list-type: atomic + resources: + properties: + claims: + items: + properties: + name: + type: string + request: + type: string + required: + - name + type: object + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + limits: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + requests: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + type: object + restartPolicy: + type: string + securityContext: + properties: + allowPrivilegeEscalation: + type: boolean + appArmorProfile: + properties: + localhostProfile: + type: string + type: + type: string + required: + - type + type: object + capabilities: + properties: + add: + items: + type: string + type: array + x-kubernetes-list-type: atomic + drop: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + privileged: + type: boolean + procMount: + type: string + readOnlyRootFilesystem: + type: boolean + runAsGroup: + format: int64 + type: integer + runAsNonRoot: + type: boolean + runAsUser: + format: int64 + type: integer + seLinuxOptions: + properties: + level: + type: string + role: + type: string + type: + type: string + user: + type: string + type: object + seccompProfile: + properties: + localhostProfile: + type: string + type: + type: string + required: + - type + type: object + windowsOptions: + properties: + gmsaCredentialSpec: + type: string + gmsaCredentialSpecName: + type: string + hostProcess: + type: boolean + runAsUserName: + type: string + type: object + type: object + startupProbe: + properties: + exec: + properties: + command: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + failureThreshold: + format: int32 + type: integer + grpc: + properties: + port: + format: int32 + type: integer + service: + default: "" + type: string + required: + - port + type: object + httpGet: + properties: + host: + type: string + httpHeaders: + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + x-kubernetes-list-type: atomic + path: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + scheme: + type: string + required: + - port + type: object + initialDelaySeconds: + format: int32 + type: integer + periodSeconds: + format: int32 + type: integer + successThreshold: + format: int32 + type: integer + tcpSocket: + properties: + host: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + required: + - port + type: object + terminationGracePeriodSeconds: + format: int64 + type: integer + timeoutSeconds: + format: int32 + type: integer + type: object + stdin: + type: boolean + stdinOnce: + type: boolean + terminationMessagePath: + type: string + terminationMessagePolicy: + type: string + tty: + type: boolean + volumeDevices: + items: + properties: + devicePath: + type: string + name: + type: string + required: + - devicePath + - name + type: object + type: array + x-kubernetes-list-map-keys: + - devicePath + x-kubernetes-list-type: map + volumeMounts: + items: + properties: + mountPath: + type: string + mountPropagation: + type: string + name: + type: string + readOnly: + type: boolean + recursiveReadOnly: + type: string + subPath: + type: string + subPathExpr: + type: string + required: + - mountPath + - name + type: object + type: array + x-kubernetes-list-map-keys: + - mountPath + x-kubernetes-list-type: map + workingDir: + type: string + required: + - name + type: object + type: array type: object status: - description: DatabaseObserverStatus defines the observed state of DatabaseObserver properties: conditions: - description: 'INSERT ADDITIONAL STATUS FIELD - define observed state - of cluster Important: Run "make" to regenerate code after modifying - this file' items: - description: "Condition contains details for one aspect of the current - state of this API Resource. --- This struct is intended for direct - use as an array at the field path .status.conditions. For example, - \n \ttype FooStatus struct{ \t // Represents the observations - of a foo's current state. \t // Known .status.conditions.type - are: \"Available\", \"Progressing\", and \"Degraded\" \t // - +patchMergeKey=type \t // +patchStrategy=merge \t // +listType=map - \t // +listMapKey=type \t Conditions []metav1.Condition - `json:\"conditions,omitempty\" patchStrategy:\"merge\" patchMergeKey:\"type\" - protobuf:\"bytes,1,rep,name=conditions\"` \n \t // other fields - \t}" properties: lastTransitionTime: - description: lastTransitionTime is the last time the condition - transitioned from one status to another. This should be when - the underlying condition changed. If that is not known, then - using the time when the API field changed is acceptable. format: date-time type: string message: - description: message is a human readable message indicating - details about the transition. This may be an empty string. maxLength: 32768 type: string observedGeneration: - description: observedGeneration represents the .metadata.generation - that the condition was set based upon. For instance, if .metadata.generation - is currently 12, but the .status.conditions[x].observedGeneration - is 9, the condition is out of date with respect to the current - state of the instance. format: int64 minimum: 0 type: integer reason: - description: reason contains a programmatic identifier indicating - the reason for the condition's last transition. Producers - of specific condition types may define expected values and - meanings for this field, and whether the values are considered - a guaranteed API. The value should be a CamelCase string. - This field may not be empty. maxLength: 1024 minLength: 1 pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ type: string status: - description: status of the condition, one of True, False, Unknown. enum: - "True" - "False" - Unknown type: string type: - description: type of condition in CamelCase or in foo.example.com/CamelCase. - --- Many .condition.type values are consistent across resources - like Available, but because arbitrary conditions can be useful - (see .node.status.conditions), the ability to deconflict is - important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) maxLength: 316 pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ type: string @@ -210,18 +6974,15 @@ spec: type: integer status: type: string + version: + type: string required: - conditions - exporterConfig + - version type: object type: object served: true storage: true subresources: status: {} -status: - acceptedNames: - kind: "" - plural: "" - conditions: [] - storedVersions: [] diff --git a/config/crd/kustomization.yaml b/config/crd/kustomization.yaml index b9f3aa8c..726521b0 100644 --- a/config/crd/kustomization.yaml +++ b/config/crd/kustomization.yaml @@ -18,37 +18,52 @@ resources: - bases/database.oracle.com_dbcssystems.yaml - bases/database.oracle.com_dataguardbrokers.yaml - bases/observability.oracle.com_databaseobservers.yaml +- bases/database.oracle.com_lrests.yaml +- bases/database.oracle.com_lrpdbs.yaml +- bases/database.oracle.com_ordssrvs.yaml # +kubebuilder:scaffold:crdkustomizeresource patchesStrategicMerge: # [WEBHOOK] To enable webhook, uncomment all the sections with [WEBHOOK] prefix. # patches here are for enabling the conversion webhook for each CRD #- patches/webhook_in_provshards.yaml -#- patches/webhook_in_autonomousdatabases.yaml #- patches/webhook_in_singleinstancedatabases.yaml #- patches/webhook_in_shardingdatabases.yaml #- patches/webhook_in_pdbs.yaml #- patches/webhook_in_cdbs.yaml #- patches/webhook_in_oraclerestdataservices.yaml -#- patches/webhook_in_autonomouscontainerdatabases.yaml #- patches/webhook_in_dbcssystems.yaml #- patches/webhook_in_dataguardbrokers.yaml #- patches/webhook_in_databaseobservers.yaml +- patches/webhook_in_autonomousdatabases.yaml +- patches/webhook_in_autonomousdatabasebackups.yaml +- patches/webhook_in_autonomousdatabaserestores.yaml +- patches/webhook_in_autonomouscontainerdatabases.yaml +#- patches/webhook_in_lrests.yaml +#- patches/webhook_in_lrpdbs.yaml +#- patches/webhook_in_ordssrvs.yaml # +kubebuilder:scaffold:crdkustomizewebhookpatch # [CERTMANAGER] To enable webhook, uncomment all the sections with [CERTMANAGER] prefix. # patches here are for enabling the CA injection for each CRD #- patches/cainjection_in_provshards.yaml -#- patches/cainjection_in_autonomousdatabases.yaml - patches/cainjection_in_singleinstancedatabases.yaml -#- patches/cainjection_in_shardingdatabases.yaml +- patches/cainjection_in_shardingdatabases.yaml - patches/cainjection_in_pdbs.yaml - patches/cainjection_in_cdbs.yaml #- patches/cainjection_in_oraclerestdataservices.yaml #- patches/cainjection_in_autonomouscontainerdatabases.yaml -#- patches/cainjection_in_dbcssystems.yaml +- patches/cainjection_in_dbcssystems.yaml #- patches/cainjection_in_dataguardbrokers.yaml #- patches/cainjection_in_databaseobservers.yaml +- patches/cainjection_in_autonomousdatabases.yaml +- patches/cainjection_in_autonomousdatabasebackups.yaml +- patches/cainjection_in_autonomousdatabaserestores.yaml +- patches/cainjection_in_autonomouscontainerdatabases.yaml +#- patches/cainjection_in_lrests.yaml +#- patches/cainjection_in_lrpdbs.yaml +#- patches/cainjection_in_ordssrvs.yaml +#- patches/cainjection_in_singleinstancedatabases.yaml # +kubebuilder:scaffold:crdkustomizecainjectionpatch # the following config is for teaching kustomize how to do kustomization for CRDs. diff --git a/config/crd/patches/cainjection_in_autonomouscontainerdatabases.yaml b/config/crd/patches/cainjection_in_autonomouscontainerdatabases.yaml index 3985a5ae..734407bc 100644 --- a/config/crd/patches/cainjection_in_autonomouscontainerdatabases.yaml +++ b/config/crd/patches/cainjection_in_autonomouscontainerdatabases.yaml @@ -1,6 +1,6 @@ # The following patch adds a directive for certmanager to inject CA into the CRD # CRD conversion requires k8s 1.13 or later. -apiVersion: apiextensions.k8s.io/v1beta1 +apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: diff --git a/config/crd/patches/cainjection_in_autonomousdatabasebackups.yaml b/config/crd/patches/cainjection_in_autonomousdatabasebackups.yaml index 78280137..9468569d 100644 --- a/config/crd/patches/cainjection_in_autonomousdatabasebackups.yaml +++ b/config/crd/patches/cainjection_in_autonomousdatabasebackups.yaml @@ -1,6 +1,6 @@ # The following patch adds a directive for certmanager to inject CA into the CRD # CRD conversion requires k8s 1.13 or later. -apiVersion: apiextensions.k8s.io/v1beta1 +apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: diff --git a/config/crd/patches/cainjection_in_autonomousdatabaserestores.yaml b/config/crd/patches/cainjection_in_autonomousdatabaserestores.yaml index 75894cbb..cfc941f8 100644 --- a/config/crd/patches/cainjection_in_autonomousdatabaserestores.yaml +++ b/config/crd/patches/cainjection_in_autonomousdatabaserestores.yaml @@ -1,6 +1,6 @@ # The following patch adds a directive for certmanager to inject CA into the CRD # CRD conversion requires k8s 1.13 or later. -apiVersion: apiextensions.k8s.io/v1beta1 +apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: diff --git a/config/crd/patches/cainjection_in_database_dataguardbrokers.yaml b/config/crd/patches/cainjection_in_database_dataguardbrokers.yaml new file mode 100644 index 00000000..6409f54c --- /dev/null +++ b/config/crd/patches/cainjection_in_database_dataguardbrokers.yaml @@ -0,0 +1,8 @@ +# The following patch adds a directive for certmanager to inject CA into the CRD +# CRD conversion requires k8s 1.13 or later. +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + annotations: + cert-manager.io/inject-ca-from: $(CERTIFICATE_NAMESPACE)/$(CERTIFICATE_NAME) + name: dataguardbrokers.database.oracle.com diff --git a/config/crd/patches/cainjection_in_database_lrests.yaml b/config/crd/patches/cainjection_in_database_lrests.yaml new file mode 100644 index 00000000..22f4b410 --- /dev/null +++ b/config/crd/patches/cainjection_in_database_lrests.yaml @@ -0,0 +1,8 @@ +# The following patch adds a directive for certmanager to inject CA into the CRD +# CRD conversion requires k8s 1.13 or later. +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + annotations: + cert-manager.io/inject-ca-from: $(CERTIFICATE_NAMESPACE)/$(CERTIFICATE_NAME) + name: lrests.database.oracle.com diff --git a/config/crd/patches/cainjection_in_database_lrpdbs.yaml b/config/crd/patches/cainjection_in_database_lrpdbs.yaml new file mode 100644 index 00000000..f6f21f4c --- /dev/null +++ b/config/crd/patches/cainjection_in_database_lrpdbs.yaml @@ -0,0 +1,8 @@ +# The following patch adds a directive for certmanager to inject CA into the CRD +# CRD conversion requires k8s 1.13 or later. +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + annotations: + cert-manager.io/inject-ca-from: $(CERTIFICATE_NAMESPACE)/$(CERTIFICATE_NAME) + name: lrpdbs.database.oracle.com diff --git a/config/crd/patches/cainjection_in_database_oraclerestdataservices.yaml b/config/crd/patches/cainjection_in_database_oraclerestdataservices.yaml new file mode 100644 index 00000000..d2b5d4ee --- /dev/null +++ b/config/crd/patches/cainjection_in_database_oraclerestdataservices.yaml @@ -0,0 +1,8 @@ +# The following patch adds a directive for certmanager to inject CA into the CRD +# CRD conversion requires k8s 1.13 or later. +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + annotations: + cert-manager.io/inject-ca-from: $(CERTIFICATE_NAMESPACE)/$(CERTIFICATE_NAME) + name: oraclerestdataservices.database.oracle.com diff --git a/config/crd/patches/cainjection_in_database_ordssrvs.yaml b/config/crd/patches/cainjection_in_database_ordssrvs.yaml new file mode 100644 index 00000000..d2bfc8bf --- /dev/null +++ b/config/crd/patches/cainjection_in_database_ordssrvs.yaml @@ -0,0 +1,8 @@ +# The following patch adds a directive for certmanager to inject CA into the CRD +# CRD conversion requires k8s 1.13 or later. +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + annotations: + cert-manager.io/inject-ca-from: $(CERTIFICATE_NAMESPACE)/$(CERTIFICATE_NAME) + name: ordssrvs.database.oracle.com diff --git a/config/crd/patches/cainjection_in_database_singleinstancedatabases.yaml b/config/crd/patches/cainjection_in_database_singleinstancedatabases.yaml new file mode 100644 index 00000000..b87b9351 --- /dev/null +++ b/config/crd/patches/cainjection_in_database_singleinstancedatabases.yaml @@ -0,0 +1,8 @@ +# The following patch adds a directive for certmanager to inject CA into the CRD +# CRD conversion requires k8s 1.13 or later. +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + annotations: + cert-manager.io/inject-ca-from: $(CERTIFICATE_NAMESPACE)/$(CERTIFICATE_NAME) + name: singleinstancedatabases.database.oracle.com diff --git a/config/crd/patches/cainjection_in_dbcssystems.yaml b/config/crd/patches/cainjection_in_dbcssystems.yaml index 9d8521ac..1c14e1fd 100644 --- a/config/crd/patches/cainjection_in_dbcssystems.yaml +++ b/config/crd/patches/cainjection_in_dbcssystems.yaml @@ -1,6 +1,6 @@ # The following patch adds a directive for certmanager to inject CA into the CRD # CRD conversion requires k8s 1.13 or later. -apiVersion: apiextensions.k8s.io/v1beta1 +apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: diff --git a/config/crd/patches/cainjection_in_observability_databaseobservers.yaml b/config/crd/patches/cainjection_in_observability_databaseobservers.yaml new file mode 100644 index 00000000..bef0b6c0 --- /dev/null +++ b/config/crd/patches/cainjection_in_observability_databaseobservers.yaml @@ -0,0 +1,8 @@ +# The following patch adds a directive for certmanager to inject CA into the CRD +# CRD conversion requires k8s 1.13 or later. +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + annotations: + cert-manager.io/inject-ca-from: $(CERTIFICATE_NAMESPACE)/$(CERTIFICATE_NAME) + name: databaseobservers.observability.oracle.com diff --git a/config/crd/patches/webhook_in_autonomouscontainerdatabases.yaml b/config/crd/patches/webhook_in_autonomouscontainerdatabases.yaml index 03a73384..6ef8f0a6 100644 --- a/config/crd/patches/webhook_in_autonomouscontainerdatabases.yaml +++ b/config/crd/patches/webhook_in_autonomouscontainerdatabases.yaml @@ -1,17 +1,19 @@ # The following patch enables conversion webhook for CRD # CRD conversion requires k8s 1.13 or later. -apiVersion: apiextensions.k8s.io/v1beta1 +apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: name: autonomouscontainerdatabases.database.oracle.com spec: conversion: strategy: Webhook - webhookClientConfig: - # this is "\n" used as a placeholder, otherwise it will be rejected by the apiserver for being blank, - # but we're going to set it later using the cert-manager (or potentially a patch if not using cert-manager) - caBundle: Cg== - service: - namespace: system - name: webhook-service - path: /convert + webhook: + clientConfig: + service: + namespace: oracle-database-operator-system + name: oracle-database-operator-webhook-service + path: /convert + conversionReviewVersions: + - v1alpha1 + - v1 + - v4 diff --git a/config/crd/patches/webhook_in_autonomousdatabasebackups.yaml b/config/crd/patches/webhook_in_autonomousdatabasebackups.yaml index 1a4eacb6..ee363f8f 100644 --- a/config/crd/patches/webhook_in_autonomousdatabasebackups.yaml +++ b/config/crd/patches/webhook_in_autonomousdatabasebackups.yaml @@ -1,17 +1,19 @@ # The following patch enables conversion webhook for CRD # CRD conversion requires k8s 1.13 or later. -apiVersion: apiextensions.k8s.io/v1beta1 +apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: name: autonomousdatabasebackups.database.oracle.com spec: conversion: strategy: Webhook - webhookClientConfig: - # this is "\n" used as a placeholder, otherwise it will be rejected by the apiserver for being blank, - # but we're going to set it later using the cert-manager (or potentially a patch if not using cert-manager) - caBundle: Cg== - service: - namespace: system - name: webhook-service - path: /convert + webhook: + clientConfig: + service: + namespace: oracle-database-operator-system + name: oracle-database-operator-webhook-service + path: /convert + conversionReviewVersions: + - v1alpha1 + - v1 + - v4 \ No newline at end of file diff --git a/config/crd/patches/webhook_in_autonomousdatabaserestores.yaml b/config/crd/patches/webhook_in_autonomousdatabaserestores.yaml index 0a0ed4ad..33329655 100644 --- a/config/crd/patches/webhook_in_autonomousdatabaserestores.yaml +++ b/config/crd/patches/webhook_in_autonomousdatabaserestores.yaml @@ -1,17 +1,19 @@ # The following patch enables conversion webhook for CRD # CRD conversion requires k8s 1.13 or later. -apiVersion: apiextensions.k8s.io/v1beta1 +apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: name: autonomousdatabaserestores.database.oracle.com spec: conversion: strategy: Webhook - webhookClientConfig: - # this is "\n" used as a placeholder, otherwise it will be rejected by the apiserver for being blank, - # but we're going to set it later using the cert-manager (or potentially a patch if not using cert-manager) - caBundle: Cg== - service: - namespace: system - name: webhook-service - path: /convert + webhook: + clientConfig: + service: + namespace: oracle-database-operator-system + name: oracle-database-operator-webhook-service + path: /convert + conversionReviewVersions: + - v1alpha1 + - v1 + - v4 \ No newline at end of file diff --git a/config/crd/patches/webhook_in_autonomousdatabases.yaml b/config/crd/patches/webhook_in_autonomousdatabases.yaml index 230f9f68..c7ec554f 100644 --- a/config/crd/patches/webhook_in_autonomousdatabases.yaml +++ b/config/crd/patches/webhook_in_autonomousdatabases.yaml @@ -11,11 +11,13 @@ metadata: spec: conversion: strategy: Webhook - webhookClientConfig: - # this is "\n" used as a placeholder, otherwise it will be rejected by the apiserver for being blank, - # but we're going to set it later using the cert-manager (or potentially a patch if not using cert-manager) - caBundle: Cg== - service: - namespace: system - name: webhook-service - path: /convert + webhook: + clientConfig: + service: + namespace: oracle-database-operator-system + name: oracle-database-operator-webhook-service + path: /convert + conversionReviewVersions: + - v1alpha1 + - v1 + - v4 diff --git a/config/crd/patches/webhook_in_lrests.yaml b/config/crd/patches/webhook_in_lrests.yaml new file mode 100644 index 00000000..01afd4b5 --- /dev/null +++ b/config/crd/patches/webhook_in_lrests.yaml @@ -0,0 +1,17 @@ +# The following patch enables conversion webhook for CRD +# CRD conversion requires k8s 1.13 or later. +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: lrests.database.oracle.com +spec: + conversion: + strategy: Webhook + webhookClientConfig: + # this is "\n" used as a placeholder, otherwise it will be rejected by the apiserver for being blank, + # but we're going to set it later using the cert-manager (or potentially a patch if not using cert-manager) + caBundle: Cg== + service: + namespace: system + name: webhook-service + path: /convert diff --git a/config/crd/patches/webhook_in_lrpdbs.yaml b/config/crd/patches/webhook_in_lrpdbs.yaml new file mode 100644 index 00000000..4120e72f --- /dev/null +++ b/config/crd/patches/webhook_in_lrpdbs.yaml @@ -0,0 +1,17 @@ +# The following patch enables conversion webhook for CRD +# CRD conversion requires k8s 1.13 or later. +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: lrpdbs.database.oracle.com +spec: + conversion: + strategy: Webhook + webhookClientConfig: + # this is "\n" used as a placeholder, otherwise it will be rejected by the apiserver for being blank, + # but we're going to set it later using the cert-manager (or potentially a patch if not using cert-manager) + caBundle: Cg== + service: + namespace: system + name: webhook-service + path: /convert diff --git a/config/crd/patches/webhook_in_ordssrvs.yaml b/config/crd/patches/webhook_in_ordssrvs.yaml new file mode 100644 index 00000000..0c3d7637 --- /dev/null +++ b/config/crd/patches/webhook_in_ordssrvs.yaml @@ -0,0 +1,17 @@ +# The following patch enables conversion webhook for CRD +# CRD conversion requires k8s 1.13 or later. +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: ordssrvs.database.oracle.com +spec: + conversion: + strategy: Webhook + webhookClientConfig: + # this is "\n" used as a placeholder, otherwise it will be rejected by the apiserver for being blank, + # but we're going to set it later using the cert-manager (or potentially a patch if not using cert-manager) + caBundle: Cg== + service: + namespace: system + name: webhook-service + path: /convert diff --git a/config/database.oracle.com_DbcsSystem.yaml b/config/database.oracle.com_DbcsSystem.yaml index e933d5a4..c342c363 100644 --- a/config/database.oracle.com_DbcsSystem.yaml +++ b/config/database.oracle.com_DbcsSystem.yaml @@ -4,7 +4,7 @@ apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - controller-gen.kubebuilder.io/version: v0.6.1 + controller-gen.kubebuilder.io/version: v0.16.5 creationTimestamp: null name: DbcsSystem.database.oracle.com spec: @@ -16,7 +16,7 @@ spec: singular: dbcssystem scope: Namespaced versions: - - name: v1alpha1 + - name: v4 schema: openAPIV3Schema: description: DbcsSystem is the Schema for the dbcssystems API @@ -36,6 +36,53 @@ spec: spec: description: DbcsSystemSpec defines the desired state of DbcsSystem properties: + databaseId: + type: string + dbBackupId: + type: string + dbClone: + description: DbCloneConfig defines the configuration for the database + clone + properties: + dbAdminPaswordSecret: + type: string + dbDbUniqueName: + type: string + dbName: + type: string + displayName: + type: string + domain: + type: string + hostName: + type: string + initialDataStorageSizeInGB: + type: integer + kmsKeyId: + type: string + kmsKeyVersionId: + type: string + licenseModel: + type: string + privateIp: + type: string + sidPrefix: + type: string + sshPublicKeys: + items: + type: string + type: array + subnetId: + type: string + tdeWalletPasswordSecret: + type: string + required: + - dbDbUniqueName + - dbName + - displayName + - hostName + - subnetId + type: object dbSystem: properties: availabilityDomain: @@ -51,7 +98,7 @@ spec: dbAdminPaswordSecret: type: string dbBackupConfig: - description: DB Backup COnfig Network Struct + description: DB Backup Config Network Struct properties: autoBackupEnabled: type: boolean @@ -88,10 +135,19 @@ spec: type: string initialDataStorageSizeInGB: type: integer - kmsKeyId: - type: string - kmsKeyVersionId: - type: string + kmsConfig: + properties: + compartmentId: + type: string + encryptionAlgo: + type: string + keyName: + type: string + vaultName: + type: string + vaultType: + type: string + type: object licenseModel: type: string nodeCount: @@ -124,17 +180,80 @@ spec: - dbAdminPaswordSecret - hostName - shape - - sshPublicKeys - subnetId type: object hardLink: type: boolean id: type: string + kmsConfig: + properties: + compartmentId: + type: string + encryptionAlgo: + type: string + keyName: + type: string + vaultName: + type: string + vaultType: + type: string + type: object ociConfigMap: type: string ociSecret: type: string + pdbConfigs: + items: + description: PDBConfig defines details of PDB struct for DBCS systems + properties: + freeformTags: + additionalProperties: + type: string + description: '// Free-form tags for this resource. Each tag + is a simple key-value pair with no predefined name, type, + or namespace. // For more information, see Resource Tags (https://docs.cloud.oracle.com/Content/General/Concepts/resourcetags.htm). + // Example: `{"Department": "Finance"}`' + type: object + isDelete: + description: To specify whether to delete the PDB + type: boolean + pdbAdminPassword: + description: // A strong password for PDB Admin. The password + must be at least nine characters and contain at least two + uppercase, two lowercase, two numbers, and two special characters. + The special characters must be _, \#, or -. + type: string + pdbName: + description: The name for the pluggable database (PDB). The + name is unique in the context of a Database. The name must + begin with an alphabetic character and can contain a maximum + of thirty alphanumeric characters. Special characters are + not permitted. The pluggable database name should not be same + as the container database name. + type: string + pluggableDatabaseId: + description: The OCID of the PDB for deletion purposes. + type: string + shouldPdbAdminAccountBeLocked: + description: // The locked mode of the pluggable database admin + account. If false, the user needs to provide the PDB Admin + Password to connect to it. // If true, the pluggable database + will be locked and user cannot login to it. + type: boolean + tdeWalletPassword: + description: // The existing TDE wallet password of the CDB. + type: string + required: + - freeformTags + - pdbAdminPassword + - pdbName + - shouldPdbAdminAccountBeLocked + - tdeWalletPassword + type: object + type: array + setupDBCloning: + type: boolean required: - ociConfigMap type: object @@ -149,6 +268,35 @@ spec: type: integer dataStorageSizeInGBs: type: integer + dbCloneStatus: + description: DbCloneStatus defines the observed state of DbClone + properties: + dbAdminPaswordSecret: + type: string + dbDbUniqueName: + type: string + dbName: + type: string + displayName: + type: string + domain: + type: string + hostName: + type: string + id: + type: string + licenseModel: + type: string + sshPublicKeys: + items: + type: string + type: array + subnetId: + type: string + required: + - dbDbUniqueName + - hostName + type: object dbEdition: type: string dbInfo: @@ -171,6 +319,25 @@ spec: type: string id: type: string + kmsDetailsStatus: + properties: + compartmentId: + type: string + encryptionAlgo: + type: string + keyId: + type: string + keyName: + type: string + managementEndpoint: + type: string + vaultId: + type: string + vaultName: + type: string + vaultType: + type: string + type: object licenseModel: type: string network: @@ -192,6 +359,32 @@ spec: type: object nodeCount: type: integer + pdbDetailsStatus: + items: + properties: + pdbConfigStatus: + items: + properties: + freeformTags: + additionalProperties: + type: string + type: object + pdbName: + type: string + pdbState: + type: string + pluggableDatabaseId: + type: string + shouldPdbAdminAccountBeLocked: + type: boolean + required: + - freeformTags + - pdbName + - shouldPdbAdminAccountBeLocked + type: object + type: array + type: object + type: array recoStorageSizeInGB: type: integer shape: diff --git a/config/database.oracle.com_shardingdatabases.yaml b/config/database.oracle.com_shardingdatabases.yaml index 641629a0..bb9bbd38 100644 --- a/config/database.oracle.com_shardingdatabases.yaml +++ b/config/database.oracle.com_shardingdatabases.yaml @@ -4,7 +4,7 @@ apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - controller-gen.kubebuilder.io/version: v0.6.1 + controller-gen.kubebuilder.io/version: v0.16.5 creationTimestamp: null name: shardingdatabases.database.oracle.com spec: @@ -27,7 +27,7 @@ spec: name: shards priority: 1 type: string - name: v1alpha1 + name: v4 schema: openAPIV3Schema: description: ShardingDatabase is the Schema for the shardingdatabases API diff --git a/config/manager/kustomization.yaml b/config/manager/kustomization.yaml index 2aed83d4..1a9d97d3 100644 --- a/config/manager/kustomization.yaml +++ b/config/manager/kustomization.yaml @@ -8,5 +8,5 @@ apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization images: - name: controller - newName: container-registry.oracle.com/database/operator + newName: lin.ocir.io/intsanjaysingh/mmalvezz/testppr/operatormntnns newTag: latest diff --git a/config/manifests/bases/oracle-database-operator.clusterserviceversion.yaml b/config/manifests/bases/oracle-database-operator.clusterserviceversion.yaml index 23cd7c00..933a2bfa 100644 --- a/config/manifests/bases/oracle-database-operator.clusterserviceversion.yaml +++ b/config/manifests/bases/oracle-database-operator.clusterserviceversion.yaml @@ -16,7 +16,7 @@ spec: displayName: Dbcs System kind: DbcsSystem name: DbcsSystem.database.oracle.com - version: v1alpha1 + version: v4 - description: AutonomousContainerDatabase is the Schema for the autonomouscontainerdatabases API displayName: Autonomous Container Database @@ -70,7 +70,7 @@ spec: displayName: Sharding Database kind: ShardingDatabase name: shardingdatabases.database.oracle.com - version: v1alpha1 + version: v4 - description: SingleInstanceDatabase is the Schema for the singleinstancedatabases API displayName: Single Instance Database diff --git a/config/observability.oracle.com_databaseobservers.yaml b/config/observability.oracle.com_databaseobservers.yaml index b0801738..c69a3b99 100644 --- a/config/observability.oracle.com_databaseobservers.yaml +++ b/config/observability.oracle.com_databaseobservers.yaml @@ -1,4 +1,3 @@ - --- apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition @@ -23,6 +22,3181 @@ spec: - jsonPath: .status.status name: Status type: string + - jsonPath: .status.version + name: Version + type: string + name: v1 + schema: + openAPIV3Schema: + description: DatabaseObserver is the Schema for the databaseobservers API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: DatabaseObserverSpec defines the desired state of DatabaseObserver + properties: + configuration: + properties: + configMap: + description: ConfigMapDetails defines the configmap name + properties: + key: + type: string + name: + type: string + type: object + type: object + database: + description: DatabaseObserverDatabase defines the database details + used for DatabaseObserver + properties: + dbConnectionString: + properties: + key: + type: string + secret: + type: string + type: object + dbPassword: + properties: + key: + type: string + secret: + type: string + vaultOCID: + type: string + vaultSecretName: + type: string + type: object + dbRole: + type: string + dbUser: + properties: + key: + type: string + secret: + type: string + type: object + dbWallet: + properties: + key: + type: string + secret: + type: string + type: object + type: object + exporter: + description: DatabaseObserverExporterConfig defines the configuration + details related to the exporters of DatabaseObserver + properties: + args: + items: + type: string + type: array + commands: + items: + type: string + type: array + image: + type: string + service: + description: DatabaseObserverService defines the exporter service + component of DatabaseObserver + properties: + port: + format: int32 + type: integer + type: object + type: object + inherit_labels: + items: + type: string + type: array + log: + description: LogConfig defines the configuration details relation + to the logs of DatabaseObserver + properties: + filename: + type: string + path: + type: string + volume: + properties: + name: + type: string + persistentVolumeClaim: + properties: + claimName: + type: string + type: object + type: object + type: object + ociConfig: + properties: + configMapName: + type: string + secretName: + type: string + type: object + prometheus: + description: PrometheusConfig defines the generated resources for + Prometheus + properties: + port: + type: string + release: + type: string + type: object + replicas: + format: int32 + type: integer + sidecarVolumes: + items: + description: Volume represents a named volume in a pod that may + be accessed by any container in the pod. + properties: + awsElasticBlockStore: + description: 'awsElasticBlockStore represents an AWS Disk resource + that is attached to a kubelet''s host machine and then exposed + to the pod. More info: https://kubernetes.io/docs/concepts/storage/volumes#awselasticblockstore' + properties: + fsType: + description: 'fsType is the filesystem type of the volume + that you want to mount. Tip: Ensure that the filesystem + type is supported by the host operating system. Examples: + "ext4", "xfs", "ntfs". Implicitly inferred to be "ext4" + if unspecified. More info: https://kubernetes.io/docs/concepts/storage/volumes#awselasticblockstore + TODO: how do we prevent errors in the filesystem from + compromising the machine' + type: string + partition: + description: 'partition is the partition in the volume that + you want to mount. If omitted, the default is to mount + by volume name. Examples: For volume /dev/sda1, you specify + the partition as "1". Similarly, the volume partition + for /dev/sda is "0" (or you can leave the property empty).' + format: int32 + type: integer + readOnly: + description: 'readOnly value true will force the readOnly + setting in VolumeMounts. More info: https://kubernetes.io/docs/concepts/storage/volumes#awselasticblockstore' + type: boolean + volumeID: + description: 'volumeID is unique ID of the persistent disk + resource in AWS (Amazon EBS volume). More info: https://kubernetes.io/docs/concepts/storage/volumes#awselasticblockstore' + type: string + required: + - volumeID + type: object + azureDisk: + description: azureDisk represents an Azure Data Disk mount on + the host and bind mount to the pod. + properties: + cachingMode: + description: 'cachingMode is the Host Caching mode: None, + Read Only, Read Write.' + type: string + diskName: + description: diskName is the Name of the data disk in the + blob storage + type: string + diskURI: + description: diskURI is the URI of data disk in the blob + storage + type: string + fsType: + description: fsType is Filesystem type to mount. Must be + a filesystem type supported by the host operating system. + Ex. "ext4", "xfs", "ntfs". Implicitly inferred to be "ext4" + if unspecified. + type: string + kind: + description: 'kind expected values are Shared: multiple + blob disks per storage account Dedicated: single blob + disk per storage account Managed: azure managed data + disk (only in managed availability set). defaults to shared' + type: string + readOnly: + description: readOnly Defaults to false (read/write). ReadOnly + here will force the ReadOnly setting in VolumeMounts. + type: boolean + required: + - diskName + - diskURI + type: object + azureFile: + description: azureFile represents an Azure File Service mount + on the host and bind mount to the pod. + properties: + readOnly: + description: readOnly defaults to false (read/write). ReadOnly + here will force the ReadOnly setting in VolumeMounts. + type: boolean + secretName: + description: secretName is the name of secret that contains + Azure Storage Account Name and Key + type: string + shareName: + description: shareName is the azure share Name + type: string + required: + - secretName + - shareName + type: object + cephfs: + description: cephFS represents a Ceph FS mount on the host that + shares a pod's lifetime + properties: + monitors: + description: 'monitors is Required: Monitors is a collection + of Ceph monitors More info: https://examples.k8s.io/volumes/cephfs/README.md#how-to-use-it' + items: + type: string + type: array + path: + description: 'path is Optional: Used as the mounted root, + rather than the full Ceph tree, default is /' + type: string + readOnly: + description: 'readOnly is Optional: Defaults to false (read/write). + ReadOnly here will force the ReadOnly setting in VolumeMounts. + More info: https://examples.k8s.io/volumes/cephfs/README.md#how-to-use-it' + type: boolean + secretFile: + description: 'secretFile is Optional: SecretFile is the + path to key ring for User, default is /etc/ceph/user.secret + More info: https://examples.k8s.io/volumes/cephfs/README.md#how-to-use-it' + type: string + secretRef: + description: 'secretRef is Optional: SecretRef is reference + to the authentication secret for User, default is empty. + More info: https://examples.k8s.io/volumes/cephfs/README.md#how-to-use-it' + properties: + name: + description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, uid?' + type: string + type: object + user: + description: 'user is optional: User is the rados user name, + default is admin More info: https://examples.k8s.io/volumes/cephfs/README.md#how-to-use-it' + type: string + required: + - monitors + type: object + cinder: + description: 'cinder represents a cinder volume attached and + mounted on kubelets host machine. More info: https://examples.k8s.io/mysql-cinder-pd/README.md' + properties: + fsType: + description: 'fsType is the filesystem type to mount. Must + be a filesystem type supported by the host operating system. + Examples: "ext4", "xfs", "ntfs". Implicitly inferred to + be "ext4" if unspecified. More info: https://examples.k8s.io/mysql-cinder-pd/README.md' + type: string + readOnly: + description: 'readOnly defaults to false (read/write). ReadOnly + here will force the ReadOnly setting in VolumeMounts. + More info: https://examples.k8s.io/mysql-cinder-pd/README.md' + type: boolean + secretRef: + description: 'secretRef is optional: points to a secret + object containing parameters used to connect to OpenStack.' + properties: + name: + description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, uid?' + type: string + type: object + volumeID: + description: 'volumeID used to identify the volume in cinder. + More info: https://examples.k8s.io/mysql-cinder-pd/README.md' + type: string + required: + - volumeID + type: object + configMap: + description: configMap represents a configMap that should populate + this volume + properties: + defaultMode: + description: 'defaultMode is optional: mode bits used to + set permissions on created files by default. Must be an + octal value between 0000 and 0777 or a decimal value between + 0 and 511. YAML accepts both octal and decimal values, + JSON requires decimal values for mode bits. Defaults to + 0644. Directories within the path are not affected by + this setting. This might be in conflict with other options + that affect the file mode, like fsGroup, and the result + can be other mode bits set.' + format: int32 + type: integer + items: + description: items if unspecified, each key-value pair in + the Data field of the referenced ConfigMap will be projected + into the volume as a file whose name is the key and content + is the value. If specified, the listed keys will be projected + into the specified paths, and unlisted keys will not be + present. If a key is specified which is not present in + the ConfigMap, the volume setup will error unless it is + marked optional. Paths must be relative and may not contain + the '..' path or start with '..'. + items: + description: Maps a string key to a path within a volume. + properties: + key: + description: key is the key to project. + type: string + mode: + description: 'mode is Optional: mode bits used to + set permissions on this file. Must be an octal value + between 0000 and 0777 or a decimal value between + 0 and 511. YAML accepts both octal and decimal values, + JSON requires decimal values for mode bits. If not + specified, the volume defaultMode will be used. + This might be in conflict with other options that + affect the file mode, like fsGroup, and the result + can be other mode bits set.' + format: int32 + type: integer + path: + description: path is the relative path of the file + to map the key to. May not be an absolute path. + May not contain the path element '..'. May not start + with the string '..'. + type: string + required: + - key + - path + type: object + type: array + name: + description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, uid?' + type: string + optional: + description: optional specify whether the ConfigMap or its + keys must be defined + type: boolean + type: object + csi: + description: csi (Container Storage Interface) represents ephemeral + storage that is handled by certain external CSI drivers (Beta + feature). + properties: + driver: + description: driver is the name of the CSI driver that handles + this volume. Consult with your admin for the correct name + as registered in the cluster. + type: string + fsType: + description: fsType to mount. Ex. "ext4", "xfs", "ntfs". + If not provided, the empty value is passed to the associated + CSI driver which will determine the default filesystem + to apply. + type: string + nodePublishSecretRef: + description: nodePublishSecretRef is a reference to the + secret object containing sensitive information to pass + to the CSI driver to complete the CSI NodePublishVolume + and NodeUnpublishVolume calls. This field is optional, + and may be empty if no secret is required. If the secret + object contains more than one secret, all secret references + are passed. + properties: + name: + description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, uid?' + type: string + type: object + readOnly: + description: readOnly specifies a read-only configuration + for the volume. Defaults to false (read/write). + type: boolean + volumeAttributes: + additionalProperties: + type: string + description: volumeAttributes stores driver-specific properties + that are passed to the CSI driver. Consult your driver's + documentation for supported values. + type: object + required: + - driver + type: object + downwardAPI: + description: downwardAPI represents downward API about the pod + that should populate this volume + properties: + defaultMode: + description: 'Optional: mode bits to use on created files + by default. Must be a Optional: mode bits used to set + permissions on created files by default. Must be an octal + value between 0000 and 0777 or a decimal value between + 0 and 511. YAML accepts both octal and decimal values, + JSON requires decimal values for mode bits. Defaults to + 0644. Directories within the path are not affected by + this setting. This might be in conflict with other options + that affect the file mode, like fsGroup, and the result + can be other mode bits set.' + format: int32 + type: integer + items: + description: Items is a list of downward API volume file + items: + description: DownwardAPIVolumeFile represents information + to create the file containing the pod field + properties: + fieldRef: + description: 'Required: Selects a field of the pod: + only annotations, labels, name and namespace are + supported.' + properties: + apiVersion: + description: Version of the schema the FieldPath + is written in terms of, defaults to "v1". + type: string + fieldPath: + description: Path of the field to select in the + specified API version. + type: string + required: + - fieldPath + type: object + mode: + description: 'Optional: mode bits used to set permissions + on this file, must be an octal value between 0000 + and 0777 or a decimal value between 0 and 511. YAML + accepts both octal and decimal values, JSON requires + decimal values for mode bits. If not specified, + the volume defaultMode will be used. This might + be in conflict with other options that affect the + file mode, like fsGroup, and the result can be other + mode bits set.' + format: int32 + type: integer + path: + description: 'Required: Path is the relative path + name of the file to be created. Must not be absolute + or contain the ''..'' path. Must be utf-8 encoded. + The first item of the relative path must not start + with ''..''' + type: string + resourceFieldRef: + description: 'Selects a resource of the container: + only resources limits and requests (limits.cpu, + limits.memory, requests.cpu and requests.memory) + are currently supported.' + properties: + containerName: + description: 'Container name: required for volumes, + optional for env vars' + type: string + divisor: + anyOf: + - type: integer + - type: string + description: Specifies the output format of the + exposed resources, defaults to "1" + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + resource: + description: 'Required: resource to select' + type: string + required: + - resource + type: object + required: + - path + type: object + type: array + type: object + emptyDir: + description: 'emptyDir represents a temporary directory that + shares a pod''s lifetime. More info: https://kubernetes.io/docs/concepts/storage/volumes#emptydir' + properties: + medium: + description: 'medium represents what type of storage medium + should back this directory. The default is "" which means + to use the node''s default medium. Must be an empty string + (default) or Memory. More info: https://kubernetes.io/docs/concepts/storage/volumes#emptydir' + type: string + sizeLimit: + anyOf: + - type: integer + - type: string + description: 'sizeLimit is the total amount of local storage + required for this EmptyDir volume. The size limit is also + applicable for memory medium. The maximum usage on memory + medium EmptyDir would be the minimum value between the + SizeLimit specified here and the sum of memory limits + of all containers in a pod. The default is nil which means + that the limit is undefined. More info: https://kubernetes.io/docs/concepts/storage/volumes#emptydir' + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + ephemeral: + description: "ephemeral represents a volume that is handled + by a cluster storage driver. The volume's lifecycle is tied + to the pod that defines it - it will be created before the + pod starts, and deleted when the pod is removed. \n Use this + if: a) the volume is only needed while the pod runs, b) features + of normal volumes like restoring from snapshot or capacity + \ tracking are needed, c) the storage driver is specified + through a storage class, and d) the storage driver supports + dynamic volume provisioning through a PersistentVolumeClaim + (see EphemeralVolumeSource for more information on the + connection between this volume type and PersistentVolumeClaim). + \n Use PersistentVolumeClaim or one of the vendor-specific + APIs for volumes that persist for longer than the lifecycle + of an individual pod. \n Use CSI for light-weight local ephemeral + volumes if the CSI driver is meant to be used that way - see + the documentation of the driver for more information. \n A + pod can use both types of ephemeral volumes and persistent + volumes at the same time." + properties: + volumeClaimTemplate: + description: "Will be used to create a stand-alone PVC to + provision the volume. The pod in which this EphemeralVolumeSource + is embedded will be the owner of the PVC, i.e. the PVC + will be deleted together with the pod. The name of the + PVC will be `-` where `` is the name from the `PodSpec.Volumes` array entry. + Pod validation will reject the pod if the concatenated + name is not valid for a PVC (for example, too long). \n + An existing PVC with that name that is not owned by the + pod will *not* be used for the pod to avoid using an unrelated + volume by mistake. Starting the pod is then blocked until + the unrelated PVC is removed. If such a pre-created PVC + is meant to be used by the pod, the PVC has to updated + with an owner reference to the pod once the pod exists. + Normally this should not be necessary, but it may be useful + when manually reconstructing a broken cluster. \n This + field is read-only and no changes will be made by Kubernetes + to the PVC after it has been created. \n Required, must + not be nil." + properties: + metadata: + description: May contain labels and annotations that + will be copied into the PVC when creating it. No other + fields are allowed and will be rejected during validation. + type: object + spec: + description: The specification for the PersistentVolumeClaim. + The entire content is copied unchanged into the PVC + that gets created from this template. The same fields + as in a PersistentVolumeClaim are also valid here. + properties: + accessModes: + description: 'accessModes contains the desired access + modes the volume should have. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1' + items: + type: string + type: array + dataSource: + description: 'dataSource field can be used to specify + either: * An existing VolumeSnapshot object (snapshot.storage.k8s.io/VolumeSnapshot) + * An existing PVC (PersistentVolumeClaim) If the + provisioner or an external controller can support + the specified data source, it will create a new + volume based on the contents of the specified + data source. When the AnyVolumeDataSource feature + gate is enabled, dataSource contents will be copied + to dataSourceRef, and dataSourceRef contents will + be copied to dataSource when dataSourceRef.namespace + is not specified. If the namespace is specified, + then dataSourceRef will not be copied to dataSource.' + properties: + apiGroup: + description: APIGroup is the group for the resource + being referenced. If APIGroup is not specified, + the specified Kind must be in the core API + group. For any other third-party types, APIGroup + is required. + type: string + kind: + description: Kind is the type of resource being + referenced + type: string + name: + description: Name is the name of resource being + referenced + type: string + required: + - kind + - name + type: object + dataSourceRef: + description: 'dataSourceRef specifies the object + from which to populate the volume with data, if + a non-empty volume is desired. This may be any + object from a non-empty API group (non core object) + or a PersistentVolumeClaim object. When this field + is specified, volume binding will only succeed + if the type of the specified object matches some + installed volume populator or dynamic provisioner. + This field will replace the functionality of the + dataSource field and as such if both fields are + non-empty, they must have the same value. For + backwards compatibility, when namespace isn''t + specified in dataSourceRef, both fields (dataSource + and dataSourceRef) will be set to the same value + automatically if one of them is empty and the + other is non-empty. When namespace is specified + in dataSourceRef, dataSource isn''t set to the + same value and must be empty. There are three + important differences between dataSource and dataSourceRef: + * While dataSource only allows two specific types + of objects, dataSourceRef allows any non-core + object, as well as PersistentVolumeClaim objects. + * While dataSource ignores disallowed values (dropping + them), dataSourceRef preserves all values, and + generates an error if a disallowed value is specified. + * While dataSource only allows local objects, + dataSourceRef allows objects in any namespaces. + (Beta) Using this field requires the AnyVolumeDataSource + feature gate to be enabled. (Alpha) Using the + namespace field of dataSourceRef requires the + CrossNamespaceVolumeDataSource feature gate to + be enabled.' + properties: + apiGroup: + description: APIGroup is the group for the resource + being referenced. If APIGroup is not specified, + the specified Kind must be in the core API + group. For any other third-party types, APIGroup + is required. + type: string + kind: + description: Kind is the type of resource being + referenced + type: string + name: + description: Name is the name of resource being + referenced + type: string + namespace: + description: Namespace is the namespace of resource + being referenced Note that when a namespace + is specified, a gateway.networking.k8s.io/ReferenceGrant + object is required in the referent namespace + to allow that namespace's owner to accept + the reference. See the ReferenceGrant documentation + for details. (Alpha) This field requires the + CrossNamespaceVolumeDataSource feature gate + to be enabled. + type: string + required: + - kind + - name + type: object + resources: + description: 'resources represents the minimum resources + the volume should have. If RecoverVolumeExpansionFailure + feature is enabled users are allowed to specify + resource requirements that are lower than previous + value but must still be higher than capacity recorded + in the status field of the claim. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#resources' + properties: + limits: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + description: 'Limits describes the maximum amount + of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + type: object + requests: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + description: 'Requests describes the minimum + amount of compute resources required. If Requests + is omitted for a container, it defaults to + Limits if that is explicitly specified, otherwise + to an implementation-defined value. Requests + cannot exceed Limits. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + type: object + type: object + selector: + description: selector is a label query over volumes + to consider for binding. + properties: + matchExpressions: + description: matchExpressions is a list of label + selector requirements. The requirements are + ANDed. + items: + description: A label selector requirement + is a selector that contains values, a key, + and an operator that relates the key and + values. + properties: + key: + description: key is the label key that + the selector applies to. + type: string + operator: + description: operator represents a key's + relationship to a set of values. Valid + operators are In, NotIn, Exists and + DoesNotExist. + type: string + values: + description: values is an array of string + values. If the operator is In or NotIn, + the values array must be non-empty. + If the operator is Exists or DoesNotExist, + the values array must be empty. This + array is replaced during a strategic + merge patch. + items: + type: string + type: array + required: + - key + - operator + type: object + type: array + matchLabels: + additionalProperties: + type: string + description: matchLabels is a map of {key,value} + pairs. A single {key,value} in the matchLabels + map is equivalent to an element of matchExpressions, + whose key field is "key", the operator is + "In", and the values array contains only "value". + The requirements are ANDed. + type: object + type: object + storageClassName: + description: 'storageClassName is the name of the + StorageClass required by the claim. More info: + https://kubernetes.io/docs/concepts/storage/persistent-volumes#class-1' + type: string + volumeAttributesClassName: + description: 'volumeAttributesClassName may be used + to set the VolumeAttributesClass used by this + claim. If specified, the CSI driver will create + or update the volume with the attributes defined + in the corresponding VolumeAttributesClass. This + has a different purpose than storageClassName, + it can be changed after the claim is created. + An empty string value means that no VolumeAttributesClass + will be applied to the claim but it''s not allowed + to reset this field to empty string once it is + set. If unspecified and the PersistentVolumeClaim + is unbound, the default VolumeAttributesClass + will be set by the persistentvolume controller + if it exists. If the resource referred to by volumeAttributesClass + does not exist, this PersistentVolumeClaim will + be set to a Pending state, as reflected by the + modifyVolumeStatus field, until such as a resource + exists. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#volumeattributesclass + (Alpha) Using this field requires the VolumeAttributesClass + feature gate to be enabled.' + type: string + volumeMode: + description: volumeMode defines what type of volume + is required by the claim. Value of Filesystem + is implied when not included in claim spec. + type: string + volumeName: + description: volumeName is the binding reference + to the PersistentVolume backing this claim. + type: string + type: object + required: + - spec + type: object + type: object + fc: + description: fc represents a Fibre Channel resource that is + attached to a kubelet's host machine and then exposed to the + pod. + properties: + fsType: + description: 'fsType is the filesystem type to mount. Must + be a filesystem type supported by the host operating system. + Ex. "ext4", "xfs", "ntfs". Implicitly inferred to be "ext4" + if unspecified. TODO: how do we prevent errors in the + filesystem from compromising the machine' + type: string + lun: + description: 'lun is Optional: FC target lun number' + format: int32 + type: integer + readOnly: + description: 'readOnly is Optional: Defaults to false (read/write). + ReadOnly here will force the ReadOnly setting in VolumeMounts.' + type: boolean + targetWWNs: + description: 'targetWWNs is Optional: FC target worldwide + names (WWNs)' + items: + type: string + type: array + wwids: + description: 'wwids Optional: FC volume world wide identifiers + (wwids) Either wwids or combination of targetWWNs and + lun must be set, but not both simultaneously.' + items: + type: string + type: array + type: object + flexVolume: + description: flexVolume represents a generic volume resource + that is provisioned/attached using an exec based plugin. + properties: + driver: + description: driver is the name of the driver to use for + this volume. + type: string + fsType: + description: fsType is the filesystem type to mount. Must + be a filesystem type supported by the host operating system. + Ex. "ext4", "xfs", "ntfs". The default filesystem depends + on FlexVolume script. + type: string + options: + additionalProperties: + type: string + description: 'options is Optional: this field holds extra + command options if any.' + type: object + readOnly: + description: 'readOnly is Optional: defaults to false (read/write). + ReadOnly here will force the ReadOnly setting in VolumeMounts.' + type: boolean + secretRef: + description: 'secretRef is Optional: secretRef is reference + to the secret object containing sensitive information + to pass to the plugin scripts. This may be empty if no + secret object is specified. If the secret object contains + more than one secret, all secrets are passed to the plugin + scripts.' + properties: + name: + description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, uid?' + type: string + type: object + required: + - driver + type: object + flocker: + description: flocker represents a Flocker volume attached to + a kubelet's host machine. This depends on the Flocker control + service being running + properties: + datasetName: + description: datasetName is Name of the dataset stored as + metadata -> name on the dataset for Flocker should be + considered as deprecated + type: string + datasetUUID: + description: datasetUUID is the UUID of the dataset. This + is unique identifier of a Flocker dataset + type: string + type: object + gcePersistentDisk: + description: 'gcePersistentDisk represents a GCE Disk resource + that is attached to a kubelet''s host machine and then exposed + to the pod. More info: https://kubernetes.io/docs/concepts/storage/volumes#gcepersistentdisk' + properties: + fsType: + description: 'fsType is filesystem type of the volume that + you want to mount. Tip: Ensure that the filesystem type + is supported by the host operating system. Examples: "ext4", + "xfs", "ntfs". Implicitly inferred to be "ext4" if unspecified. + More info: https://kubernetes.io/docs/concepts/storage/volumes#gcepersistentdisk + TODO: how do we prevent errors in the filesystem from + compromising the machine' + type: string + partition: + description: 'partition is the partition in the volume that + you want to mount. If omitted, the default is to mount + by volume name. Examples: For volume /dev/sda1, you specify + the partition as "1". Similarly, the volume partition + for /dev/sda is "0" (or you can leave the property empty). + More info: https://kubernetes.io/docs/concepts/storage/volumes#gcepersistentdisk' + format: int32 + type: integer + pdName: + description: 'pdName is unique name of the PD resource in + GCE. Used to identify the disk in GCE. More info: https://kubernetes.io/docs/concepts/storage/volumes#gcepersistentdisk' + type: string + readOnly: + description: 'readOnly here will force the ReadOnly setting + in VolumeMounts. Defaults to false. More info: https://kubernetes.io/docs/concepts/storage/volumes#gcepersistentdisk' + type: boolean + required: + - pdName + type: object + gitRepo: + description: 'gitRepo represents a git repository at a particular + revision. DEPRECATED: GitRepo is deprecated. To provision + a container with a git repo, mount an EmptyDir into an InitContainer + that clones the repo using git, then mount the EmptyDir into + the Pod''s container.' + properties: + directory: + description: directory is the target directory name. Must + not contain or start with '..'. If '.' is supplied, the + volume directory will be the git repository. Otherwise, + if specified, the volume will contain the git repository + in the subdirectory with the given name. + type: string + repository: + description: repository is the URL + type: string + revision: + description: revision is the commit hash for the specified + revision. + type: string + required: + - repository + type: object + glusterfs: + description: 'glusterfs represents a Glusterfs mount on the + host that shares a pod''s lifetime. More info: https://examples.k8s.io/volumes/glusterfs/README.md' + properties: + endpoints: + description: 'endpoints is the endpoint name that details + Glusterfs topology. More info: https://examples.k8s.io/volumes/glusterfs/README.md#create-a-pod' + type: string + path: + description: 'path is the Glusterfs volume path. More info: + https://examples.k8s.io/volumes/glusterfs/README.md#create-a-pod' + type: string + readOnly: + description: 'readOnly here will force the Glusterfs volume + to be mounted with read-only permissions. Defaults to + false. More info: https://examples.k8s.io/volumes/glusterfs/README.md#create-a-pod' + type: boolean + required: + - endpoints + - path + type: object + hostPath: + description: 'hostPath represents a pre-existing file or directory + on the host machine that is directly exposed to the container. + This is generally used for system agents or other privileged + things that are allowed to see the host machine. Most containers + will NOT need this. More info: https://kubernetes.io/docs/concepts/storage/volumes#hostpath + --- TODO(jonesdl) We need to restrict who can use host directory + mounts and who can/can not mount host directories as read/write.' + properties: + path: + description: 'path of the directory on the host. If the + path is a symlink, it will follow the link to the real + path. More info: https://kubernetes.io/docs/concepts/storage/volumes#hostpath' + type: string + type: + description: 'type for HostPath Volume Defaults to "" More + info: https://kubernetes.io/docs/concepts/storage/volumes#hostpath' + type: string + required: + - path + type: object + iscsi: + description: 'iscsi represents an ISCSI Disk resource that is + attached to a kubelet''s host machine and then exposed to + the pod. More info: https://examples.k8s.io/volumes/iscsi/README.md' + properties: + chapAuthDiscovery: + description: chapAuthDiscovery defines whether support iSCSI + Discovery CHAP authentication + type: boolean + chapAuthSession: + description: chapAuthSession defines whether support iSCSI + Session CHAP authentication + type: boolean + fsType: + description: 'fsType is the filesystem type of the volume + that you want to mount. Tip: Ensure that the filesystem + type is supported by the host operating system. Examples: + "ext4", "xfs", "ntfs". Implicitly inferred to be "ext4" + if unspecified. More info: https://kubernetes.io/docs/concepts/storage/volumes#iscsi + TODO: how do we prevent errors in the filesystem from + compromising the machine' + type: string + initiatorName: + description: initiatorName is the custom iSCSI Initiator + Name. If initiatorName is specified with iscsiInterface + simultaneously, new iSCSI interface : will be created for the connection. + type: string + iqn: + description: iqn is the target iSCSI Qualified Name. + type: string + iscsiInterface: + description: iscsiInterface is the interface Name that uses + an iSCSI transport. Defaults to 'default' (tcp). + type: string + lun: + description: lun represents iSCSI Target Lun number. + format: int32 + type: integer + portals: + description: portals is the iSCSI Target Portal List. The + portal is either an IP or ip_addr:port if the port is + other than default (typically TCP ports 860 and 3260). + items: + type: string + type: array + readOnly: + description: readOnly here will force the ReadOnly setting + in VolumeMounts. Defaults to false. + type: boolean + secretRef: + description: secretRef is the CHAP Secret for iSCSI target + and initiator authentication + properties: + name: + description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, uid?' + type: string + type: object + targetPortal: + description: targetPortal is iSCSI Target Portal. The Portal + is either an IP or ip_addr:port if the port is other than + default (typically TCP ports 860 and 3260). + type: string + required: + - iqn + - lun + - targetPortal + type: object + name: + description: 'name of the volume. Must be a DNS_LABEL and unique + within the pod. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names' + type: string + nfs: + description: 'nfs represents an NFS mount on the host that shares + a pod''s lifetime More info: https://kubernetes.io/docs/concepts/storage/volumes#nfs' + properties: + path: + description: 'path that is exported by the NFS server. More + info: https://kubernetes.io/docs/concepts/storage/volumes#nfs' + type: string + readOnly: + description: 'readOnly here will force the NFS export to + be mounted with read-only permissions. Defaults to false. + More info: https://kubernetes.io/docs/concepts/storage/volumes#nfs' + type: boolean + server: + description: 'server is the hostname or IP address of the + NFS server. More info: https://kubernetes.io/docs/concepts/storage/volumes#nfs' + type: string + required: + - path + - server + type: object + persistentVolumeClaim: + description: 'persistentVolumeClaimVolumeSource represents a + reference to a PersistentVolumeClaim in the same namespace. + More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims' + properties: + claimName: + description: 'claimName is the name of a PersistentVolumeClaim + in the same namespace as the pod using this volume. More + info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims' + type: string + readOnly: + description: readOnly Will force the ReadOnly setting in + VolumeMounts. Default false. + type: boolean + required: + - claimName + type: object + photonPersistentDisk: + description: photonPersistentDisk represents a PhotonController + persistent disk attached and mounted on kubelets host machine + properties: + fsType: + description: fsType is the filesystem type to mount. Must + be a filesystem type supported by the host operating system. + Ex. "ext4", "xfs", "ntfs". Implicitly inferred to be "ext4" + if unspecified. + type: string + pdID: + description: pdID is the ID that identifies Photon Controller + persistent disk + type: string + required: + - pdID + type: object + portworxVolume: + description: portworxVolume represents a portworx volume attached + and mounted on kubelets host machine + properties: + fsType: + description: fSType represents the filesystem type to mount + Must be a filesystem type supported by the host operating + system. Ex. "ext4", "xfs". Implicitly inferred to be "ext4" + if unspecified. + type: string + readOnly: + description: readOnly defaults to false (read/write). ReadOnly + here will force the ReadOnly setting in VolumeMounts. + type: boolean + volumeID: + description: volumeID uniquely identifies a Portworx volume + type: string + required: + - volumeID + type: object + projected: + description: projected items for all in one resources secrets, + configmaps, and downward API + properties: + defaultMode: + description: defaultMode are the mode bits used to set permissions + on created files by default. Must be an octal value between + 0000 and 0777 or a decimal value between 0 and 511. YAML + accepts both octal and decimal values, JSON requires decimal + values for mode bits. Directories within the path are + not affected by this setting. This might be in conflict + with other options that affect the file mode, like fsGroup, + and the result can be other mode bits set. + format: int32 + type: integer + sources: + description: sources is the list of volume projections + items: + description: Projection that may be projected along with + other supported volume types + properties: + clusterTrustBundle: + description: "ClusterTrustBundle allows a pod to access + the `.spec.trustBundle` field of ClusterTrustBundle + objects in an auto-updating file. \n Alpha, gated + by the ClusterTrustBundleProjection feature gate. + \n ClusterTrustBundle objects can either be selected + by name, or by the combination of signer name and + a label selector. \n Kubelet performs aggressive + normalization of the PEM contents written into the + pod filesystem. Esoteric PEM features such as inter-block + comments and block headers are stripped. Certificates + are deduplicated. The ordering of certificates within + the file is arbitrary, and Kubelet may change the + order over time." + properties: + labelSelector: + description: Select all ClusterTrustBundles that + match this label selector. Only has effect + if signerName is set. Mutually-exclusive with + name. If unset, interpreted as "match nothing". If + set but empty, interpreted as "match everything". + properties: + matchExpressions: + description: matchExpressions is a list of + label selector requirements. The requirements + are ANDed. + items: + description: A label selector requirement + is a selector that contains values, a + key, and an operator that relates the + key and values. + properties: + key: + description: key is the label key that + the selector applies to. + type: string + operator: + description: operator represents a key's + relationship to a set of values. Valid + operators are In, NotIn, Exists and + DoesNotExist. + type: string + values: + description: values is an array of string + values. If the operator is In or NotIn, + the values array must be non-empty. + If the operator is Exists or DoesNotExist, + the values array must be empty. This + array is replaced during a strategic + merge patch. + items: + type: string + type: array + required: + - key + - operator + type: object + type: array + matchLabels: + additionalProperties: + type: string + description: matchLabels is a map of {key,value} + pairs. A single {key,value} in the matchLabels + map is equivalent to an element of matchExpressions, + whose key field is "key", the operator is + "In", and the values array contains only + "value". The requirements are ANDed. + type: object + type: object + name: + description: Select a single ClusterTrustBundle + by object name. Mutually-exclusive with signerName + and labelSelector. + type: string + optional: + description: If true, don't block pod startup + if the referenced ClusterTrustBundle(s) aren't + available. If using name, then the named ClusterTrustBundle + is allowed not to exist. If using signerName, + then the combination of signerName and labelSelector + is allowed to match zero ClusterTrustBundles. + type: boolean + path: + description: Relative path from the volume root + to write the bundle. + type: string + signerName: + description: Select all ClusterTrustBundles that + match this signer name. Mutually-exclusive with + name. The contents of all selected ClusterTrustBundles + will be unified and deduplicated. + type: string + required: + - path + type: object + configMap: + description: configMap information about the configMap + data to project + properties: + items: + description: items if unspecified, each key-value + pair in the Data field of the referenced ConfigMap + will be projected into the volume as a file + whose name is the key and content is the value. + If specified, the listed keys will be projected + into the specified paths, and unlisted keys + will not be present. If a key is specified which + is not present in the ConfigMap, the volume + setup will error unless it is marked optional. + Paths must be relative and may not contain the + '..' path or start with '..'. + items: + description: Maps a string key to a path within + a volume. + properties: + key: + description: key is the key to project. + type: string + mode: + description: 'mode is Optional: mode bits + used to set permissions on this file. + Must be an octal value between 0000 and + 0777 or a decimal value between 0 and + 511. YAML accepts both octal and decimal + values, JSON requires decimal values for + mode bits. If not specified, the volume + defaultMode will be used. This might be + in conflict with other options that affect + the file mode, like fsGroup, and the result + can be other mode bits set.' + format: int32 + type: integer + path: + description: path is the relative path of + the file to map the key to. May not be + an absolute path. May not contain the + path element '..'. May not start with + the string '..'. + type: string + required: + - key + - path + type: object + type: array + name: + description: 'Name of the referent. More info: + https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, + uid?' + type: string + optional: + description: optional specify whether the ConfigMap + or its keys must be defined + type: boolean + type: object + downwardAPI: + description: downwardAPI information about the downwardAPI + data to project + properties: + items: + description: Items is a list of DownwardAPIVolume + file + items: + description: DownwardAPIVolumeFile represents + information to create the file containing + the pod field + properties: + fieldRef: + description: 'Required: Selects a field + of the pod: only annotations, labels, + name and namespace are supported.' + properties: + apiVersion: + description: Version of the schema the + FieldPath is written in terms of, + defaults to "v1". + type: string + fieldPath: + description: Path of the field to select + in the specified API version. + type: string + required: + - fieldPath + type: object + mode: + description: 'Optional: mode bits used to + set permissions on this file, must be + an octal value between 0000 and 0777 or + a decimal value between 0 and 511. YAML + accepts both octal and decimal values, + JSON requires decimal values for mode + bits. If not specified, the volume defaultMode + will be used. This might be in conflict + with other options that affect the file + mode, like fsGroup, and the result can + be other mode bits set.' + format: int32 + type: integer + path: + description: 'Required: Path is the relative + path name of the file to be created. Must + not be absolute or contain the ''..'' + path. Must be utf-8 encoded. The first + item of the relative path must not start + with ''..''' + type: string + resourceFieldRef: + description: 'Selects a resource of the + container: only resources limits and requests + (limits.cpu, limits.memory, requests.cpu + and requests.memory) are currently supported.' + properties: + containerName: + description: 'Container name: required + for volumes, optional for env vars' + type: string + divisor: + anyOf: + - type: integer + - type: string + description: Specifies the output format + of the exposed resources, defaults + to "1" + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + resource: + description: 'Required: resource to + select' + type: string + required: + - resource + type: object + required: + - path + type: object + type: array + type: object + secret: + description: secret information about the secret data + to project + properties: + items: + description: items if unspecified, each key-value + pair in the Data field of the referenced Secret + will be projected into the volume as a file + whose name is the key and content is the value. + If specified, the listed keys will be projected + into the specified paths, and unlisted keys + will not be present. If a key is specified which + is not present in the Secret, the volume setup + will error unless it is marked optional. Paths + must be relative and may not contain the '..' + path or start with '..'. + items: + description: Maps a string key to a path within + a volume. + properties: + key: + description: key is the key to project. + type: string + mode: + description: 'mode is Optional: mode bits + used to set permissions on this file. + Must be an octal value between 0000 and + 0777 or a decimal value between 0 and + 511. YAML accepts both octal and decimal + values, JSON requires decimal values for + mode bits. If not specified, the volume + defaultMode will be used. This might be + in conflict with other options that affect + the file mode, like fsGroup, and the result + can be other mode bits set.' + format: int32 + type: integer + path: + description: path is the relative path of + the file to map the key to. May not be + an absolute path. May not contain the + path element '..'. May not start with + the string '..'. + type: string + required: + - key + - path + type: object + type: array + name: + description: 'Name of the referent. More info: + https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, + uid?' + type: string + optional: + description: optional field specify whether the + Secret or its key must be defined + type: boolean + type: object + serviceAccountToken: + description: serviceAccountToken is information about + the serviceAccountToken data to project + properties: + audience: + description: audience is the intended audience + of the token. A recipient of a token must identify + itself with an identifier specified in the audience + of the token, and otherwise should reject the + token. The audience defaults to the identifier + of the apiserver. + type: string + expirationSeconds: + description: expirationSeconds is the requested + duration of validity of the service account + token. As the token approaches expiration, the + kubelet volume plugin will proactively rotate + the service account token. The kubelet will + start trying to rotate the token if the token + is older than 80 percent of its time to live + or if the token is older than 24 hours.Defaults + to 1 hour and must be at least 10 minutes. + format: int64 + type: integer + path: + description: path is the path relative to the + mount point of the file to project the token + into. + type: string + required: + - path + type: object + type: object + type: array + type: object + quobyte: + description: quobyte represents a Quobyte mount on the host + that shares a pod's lifetime + properties: + group: + description: group to map volume access to Default is no + group + type: string + readOnly: + description: readOnly here will force the Quobyte volume + to be mounted with read-only permissions. Defaults to + false. + type: boolean + registry: + description: registry represents a single or multiple Quobyte + Registry services specified as a string as host:port pair + (multiple entries are separated with commas) which acts + as the central registry for volumes + type: string + tenant: + description: tenant owning the given Quobyte volume in the + Backend Used with dynamically provisioned Quobyte volumes, + value is set by the plugin + type: string + user: + description: user to map volume access to Defaults to serivceaccount + user + type: string + volume: + description: volume is a string that references an already + created Quobyte volume by name. + type: string + required: + - registry + - volume + type: object + rbd: + description: 'rbd represents a Rados Block Device mount on the + host that shares a pod''s lifetime. More info: https://examples.k8s.io/volumes/rbd/README.md' + properties: + fsType: + description: 'fsType is the filesystem type of the volume + that you want to mount. Tip: Ensure that the filesystem + type is supported by the host operating system. Examples: + "ext4", "xfs", "ntfs". Implicitly inferred to be "ext4" + if unspecified. More info: https://kubernetes.io/docs/concepts/storage/volumes#rbd + TODO: how do we prevent errors in the filesystem from + compromising the machine' + type: string + image: + description: 'image is the rados image name. More info: + https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it' + type: string + keyring: + description: 'keyring is the path to key ring for RBDUser. + Default is /etc/ceph/keyring. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it' + type: string + monitors: + description: 'monitors is a collection of Ceph monitors. + More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it' + items: + type: string + type: array + pool: + description: 'pool is the rados pool name. Default is rbd. + More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it' + type: string + readOnly: + description: 'readOnly here will force the ReadOnly setting + in VolumeMounts. Defaults to false. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it' + type: boolean + secretRef: + description: 'secretRef is name of the authentication secret + for RBDUser. If provided overrides keyring. Default is + nil. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it' + properties: + name: + description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, uid?' + type: string + type: object + user: + description: 'user is the rados user name. Default is admin. + More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it' + type: string + required: + - image + - monitors + type: object + scaleIO: + description: scaleIO represents a ScaleIO persistent volume + attached and mounted on Kubernetes nodes. + properties: + fsType: + description: fsType is the filesystem type to mount. Must + be a filesystem type supported by the host operating system. + Ex. "ext4", "xfs", "ntfs". Default is "xfs". + type: string + gateway: + description: gateway is the host address of the ScaleIO + API Gateway. + type: string + protectionDomain: + description: protectionDomain is the name of the ScaleIO + Protection Domain for the configured storage. + type: string + readOnly: + description: readOnly Defaults to false (read/write). ReadOnly + here will force the ReadOnly setting in VolumeMounts. + type: boolean + secretRef: + description: secretRef references to the secret for ScaleIO + user and other sensitive information. If this is not provided, + Login operation will fail. + properties: + name: + description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, uid?' + type: string + type: object + sslEnabled: + description: sslEnabled Flag enable/disable SSL communication + with Gateway, default false + type: boolean + storageMode: + description: storageMode indicates whether the storage for + a volume should be ThickProvisioned or ThinProvisioned. + Default is ThinProvisioned. + type: string + storagePool: + description: storagePool is the ScaleIO Storage Pool associated + with the protection domain. + type: string + system: + description: system is the name of the storage system as + configured in ScaleIO. + type: string + volumeName: + description: volumeName is the name of a volume already + created in the ScaleIO system that is associated with + this volume source. + type: string + required: + - gateway + - secretRef + - system + type: object + secret: + description: 'secret represents a secret that should populate + this volume. More info: https://kubernetes.io/docs/concepts/storage/volumes#secret' + properties: + defaultMode: + description: 'defaultMode is Optional: mode bits used to + set permissions on created files by default. Must be an + octal value between 0000 and 0777 or a decimal value between + 0 and 511. YAML accepts both octal and decimal values, + JSON requires decimal values for mode bits. Defaults to + 0644. Directories within the path are not affected by + this setting. This might be in conflict with other options + that affect the file mode, like fsGroup, and the result + can be other mode bits set.' + format: int32 + type: integer + items: + description: items If unspecified, each key-value pair in + the Data field of the referenced Secret will be projected + into the volume as a file whose name is the key and content + is the value. If specified, the listed keys will be projected + into the specified paths, and unlisted keys will not be + present. If a key is specified which is not present in + the Secret, the volume setup will error unless it is marked + optional. Paths must be relative and may not contain the + '..' path or start with '..'. + items: + description: Maps a string key to a path within a volume. + properties: + key: + description: key is the key to project. + type: string + mode: + description: 'mode is Optional: mode bits used to + set permissions on this file. Must be an octal value + between 0000 and 0777 or a decimal value between + 0 and 511. YAML accepts both octal and decimal values, + JSON requires decimal values for mode bits. If not + specified, the volume defaultMode will be used. + This might be in conflict with other options that + affect the file mode, like fsGroup, and the result + can be other mode bits set.' + format: int32 + type: integer + path: + description: path is the relative path of the file + to map the key to. May not be an absolute path. + May not contain the path element '..'. May not start + with the string '..'. + type: string + required: + - key + - path + type: object + type: array + optional: + description: optional field specify whether the Secret or + its keys must be defined + type: boolean + secretName: + description: 'secretName is the name of the secret in the + pod''s namespace to use. More info: https://kubernetes.io/docs/concepts/storage/volumes#secret' + type: string + type: object + storageos: + description: storageOS represents a StorageOS volume attached + and mounted on Kubernetes nodes. + properties: + fsType: + description: fsType is the filesystem type to mount. Must + be a filesystem type supported by the host operating system. + Ex. "ext4", "xfs", "ntfs". Implicitly inferred to be "ext4" + if unspecified. + type: string + readOnly: + description: readOnly defaults to false (read/write). ReadOnly + here will force the ReadOnly setting in VolumeMounts. + type: boolean + secretRef: + description: secretRef specifies the secret to use for obtaining + the StorageOS API credentials. If not specified, default + values will be attempted. + properties: + name: + description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, uid?' + type: string + type: object + volumeName: + description: volumeName is the human-readable name of the + StorageOS volume. Volume names are only unique within + a namespace. + type: string + volumeNamespace: + description: volumeNamespace specifies the scope of the + volume within StorageOS. If no namespace is specified + then the Pod's namespace will be used. This allows the + Kubernetes name scoping to be mirrored within StorageOS + for tighter integration. Set VolumeName to any name to + override the default behaviour. Set to "default" if you + are not using namespaces within StorageOS. Namespaces + that do not pre-exist within StorageOS will be created. + type: string + type: object + vsphereVolume: + description: vsphereVolume represents a vSphere volume attached + and mounted on kubelets host machine + properties: + fsType: + description: fsType is filesystem type to mount. Must be + a filesystem type supported by the host operating system. + Ex. "ext4", "xfs", "ntfs". Implicitly inferred to be "ext4" + if unspecified. + type: string + storagePolicyID: + description: storagePolicyID is the storage Policy Based + Management (SPBM) profile ID associated with the StoragePolicyName. + type: string + storagePolicyName: + description: storagePolicyName is the storage Policy Based + Management (SPBM) profile name. + type: string + volumePath: + description: volumePath is the path that identifies vSphere + volume vmdk + type: string + required: + - volumePath + type: object + required: + - name + type: object + type: array + sidecars: + items: + description: A single application container that you want to run + within a pod. + properties: + args: + description: 'Arguments to the entrypoint. The container image''s + CMD is used if this is not provided. Variable references $(VAR_NAME) + are expanded using the container''s environment. If a variable + cannot be resolved, the reference in the input string will + be unchanged. Double $$ are reduced to a single $, which allows + for escaping the $(VAR_NAME) syntax: i.e. "$$(VAR_NAME)" will + produce the string literal "$(VAR_NAME)". Escaped references + will never be expanded, regardless of whether the variable + exists or not. Cannot be updated. More info: https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell' + items: + type: string + type: array + command: + description: 'Entrypoint array. Not executed within a shell. + The container image''s ENTRYPOINT is used if this is not provided. + Variable references $(VAR_NAME) are expanded using the container''s + environment. If a variable cannot be resolved, the reference + in the input string will be unchanged. Double $$ are reduced + to a single $, which allows for escaping the $(VAR_NAME) syntax: + i.e. "$$(VAR_NAME)" will produce the string literal "$(VAR_NAME)". + Escaped references will never be expanded, regardless of whether + the variable exists or not. Cannot be updated. More info: + https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell' + items: + type: string + type: array + env: + description: List of environment variables to set in the container. + Cannot be updated. + items: + description: EnvVar represents an environment variable present + in a Container. + properties: + name: + description: Name of the environment variable. Must be + a C_IDENTIFIER. + type: string + value: + description: 'Variable references $(VAR_NAME) are expanded + using the previously defined environment variables in + the container and any service environment variables. + If a variable cannot be resolved, the reference in the + input string will be unchanged. Double $$ are reduced + to a single $, which allows for escaping the $(VAR_NAME) + syntax: i.e. "$$(VAR_NAME)" will produce the string + literal "$(VAR_NAME)". Escaped references will never + be expanded, regardless of whether the variable exists + or not. Defaults to "".' + type: string + valueFrom: + description: Source for the environment variable's value. + Cannot be used if value is not empty. + properties: + configMapKeyRef: + description: Selects a key of a ConfigMap. + properties: + key: + description: The key to select. + type: string + name: + description: 'Name of the referent. More info: + https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, + uid?' + type: string + optional: + description: Specify whether the ConfigMap or + its key must be defined + type: boolean + required: + - key + type: object + fieldRef: + description: 'Selects a field of the pod: supports + metadata.name, metadata.namespace, `metadata.labels['''']`, + `metadata.annotations['''']`, spec.nodeName, + spec.serviceAccountName, status.hostIP, status.podIP, + status.podIPs.' + properties: + apiVersion: + description: Version of the schema the FieldPath + is written in terms of, defaults to "v1". + type: string + fieldPath: + description: Path of the field to select in the + specified API version. + type: string + required: + - fieldPath + type: object + resourceFieldRef: + description: 'Selects a resource of the container: + only resources limits and requests (limits.cpu, + limits.memory, limits.ephemeral-storage, requests.cpu, + requests.memory and requests.ephemeral-storage) + are currently supported.' + properties: + containerName: + description: 'Container name: required for volumes, + optional for env vars' + type: string + divisor: + anyOf: + - type: integer + - type: string + description: Specifies the output format of the + exposed resources, defaults to "1" + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + resource: + description: 'Required: resource to select' + type: string + required: + - resource + type: object + secretKeyRef: + description: Selects a key of a secret in the pod's + namespace + properties: + key: + description: The key of the secret to select from. Must + be a valid secret key. + type: string + name: + description: 'Name of the referent. More info: + https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, + uid?' + type: string + optional: + description: Specify whether the Secret or its + key must be defined + type: boolean + required: + - key + type: object + type: object + required: + - name + type: object + type: array + envFrom: + description: List of sources to populate environment variables + in the container. The keys defined within a source must be + a C_IDENTIFIER. All invalid keys will be reported as an event + when the container is starting. When a key exists in multiple + sources, the value associated with the last source will take + precedence. Values defined by an Env with a duplicate key + will take precedence. Cannot be updated. + items: + description: EnvFromSource represents the source of a set + of ConfigMaps + properties: + configMapRef: + description: The ConfigMap to select from + properties: + name: + description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, + uid?' + type: string + optional: + description: Specify whether the ConfigMap must be + defined + type: boolean + type: object + prefix: + description: An optional identifier to prepend to each + key in the ConfigMap. Must be a C_IDENTIFIER. + type: string + secretRef: + description: The Secret to select from + properties: + name: + description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, + uid?' + type: string + optional: + description: Specify whether the Secret must be defined + type: boolean + type: object + type: object + type: array + image: + description: 'Container image name. More info: https://kubernetes.io/docs/concepts/containers/images + This field is optional to allow higher level config management + to default or override container images in workload controllers + like Deployments and StatefulSets.' + type: string + imagePullPolicy: + description: 'Image pull policy. One of Always, Never, IfNotPresent. + Defaults to Always if :latest tag is specified, or IfNotPresent + otherwise. Cannot be updated. More info: https://kubernetes.io/docs/concepts/containers/images#updating-images' + type: string + lifecycle: + description: Actions that the management system should take + in response to container lifecycle events. Cannot be updated. + properties: + postStart: + description: 'PostStart is called immediately after a container + is created. If the handler fails, the container is terminated + and restarted according to its restart policy. Other management + of the container blocks until the hook completes. More + info: https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/#container-hooks' + properties: + exec: + description: Exec specifies the action to take. + properties: + command: + description: Command is the command line to execute + inside the container, the working directory for + the command is root ('/') in the container's + filesystem. The command is simply exec'd, it is + not run inside a shell, so traditional shell instructions + ('|', etc) won't work. To use a shell, you need + to explicitly call out to that shell. Exit status + of 0 is treated as live/healthy and non-zero is + unhealthy. + items: + type: string + type: array + type: object + httpGet: + description: HTTPGet specifies the http request to perform. + properties: + host: + description: Host name to connect to, defaults to + the pod IP. You probably want to set "Host" in + httpHeaders instead. + type: string + httpHeaders: + description: Custom headers to set in the request. + HTTP allows repeated headers. + items: + description: HTTPHeader describes a custom header + to be used in HTTP probes + properties: + name: + description: The header field name. This will + be canonicalized upon output, so case-variant + names will be understood as the same header. + type: string + value: + description: The header field value + type: string + required: + - name + - value + type: object + type: array + path: + description: Path to access on the HTTP server. + type: string + port: + anyOf: + - type: integer + - type: string + description: Name or number of the port to access + on the container. Number must be in the range + 1 to 65535. Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + scheme: + description: Scheme to use for connecting to the + host. Defaults to HTTP. + type: string + required: + - port + type: object + sleep: + description: Sleep represents the duration that the + container should sleep before being terminated. + properties: + seconds: + description: Seconds is the number of seconds to + sleep. + format: int64 + type: integer + required: + - seconds + type: object + tcpSocket: + description: Deprecated. TCPSocket is NOT supported + as a LifecycleHandler and kept for the backward compatibility. + There are no validation of this field and lifecycle + hooks will fail in runtime when tcp handler is specified. + properties: + host: + description: 'Optional: Host name to connect to, + defaults to the pod IP.' + type: string + port: + anyOf: + - type: integer + - type: string + description: Number or name of the port to access + on the container. Number must be in the range + 1 to 65535. Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + required: + - port + type: object + type: object + preStop: + description: 'PreStop is called immediately before a container + is terminated due to an API request or management event + such as liveness/startup probe failure, preemption, resource + contention, etc. The handler is not called if the container + crashes or exits. The Pod''s termination grace period + countdown begins before the PreStop hook is executed. + Regardless of the outcome of the handler, the container + will eventually terminate within the Pod''s termination + grace period (unless delayed by finalizers). Other management + of the container blocks until the hook completes or until + the termination grace period is reached. More info: https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/#container-hooks' + properties: + exec: + description: Exec specifies the action to take. + properties: + command: + description: Command is the command line to execute + inside the container, the working directory for + the command is root ('/') in the container's + filesystem. The command is simply exec'd, it is + not run inside a shell, so traditional shell instructions + ('|', etc) won't work. To use a shell, you need + to explicitly call out to that shell. Exit status + of 0 is treated as live/healthy and non-zero is + unhealthy. + items: + type: string + type: array + type: object + httpGet: + description: HTTPGet specifies the http request to perform. + properties: + host: + description: Host name to connect to, defaults to + the pod IP. You probably want to set "Host" in + httpHeaders instead. + type: string + httpHeaders: + description: Custom headers to set in the request. + HTTP allows repeated headers. + items: + description: HTTPHeader describes a custom header + to be used in HTTP probes + properties: + name: + description: The header field name. This will + be canonicalized upon output, so case-variant + names will be understood as the same header. + type: string + value: + description: The header field value + type: string + required: + - name + - value + type: object + type: array + path: + description: Path to access on the HTTP server. + type: string + port: + anyOf: + - type: integer + - type: string + description: Name or number of the port to access + on the container. Number must be in the range + 1 to 65535. Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + scheme: + description: Scheme to use for connecting to the + host. Defaults to HTTP. + type: string + required: + - port + type: object + sleep: + description: Sleep represents the duration that the + container should sleep before being terminated. + properties: + seconds: + description: Seconds is the number of seconds to + sleep. + format: int64 + type: integer + required: + - seconds + type: object + tcpSocket: + description: Deprecated. TCPSocket is NOT supported + as a LifecycleHandler and kept for the backward compatibility. + There are no validation of this field and lifecycle + hooks will fail in runtime when tcp handler is specified. + properties: + host: + description: 'Optional: Host name to connect to, + defaults to the pod IP.' + type: string + port: + anyOf: + - type: integer + - type: string + description: Number or name of the port to access + on the container. Number must be in the range + 1 to 65535. Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + required: + - port + type: object + type: object + type: object + livenessProbe: + description: 'Periodic probe of container liveness. Container + will be restarted if the probe fails. Cannot be updated. More + info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes' + properties: + exec: + description: Exec specifies the action to take. + properties: + command: + description: Command is the command line to execute + inside the container, the working directory for the + command is root ('/') in the container's filesystem. + The command is simply exec'd, it is not run inside + a shell, so traditional shell instructions ('|', etc) + won't work. To use a shell, you need to explicitly + call out to that shell. Exit status of 0 is treated + as live/healthy and non-zero is unhealthy. + items: + type: string + type: array + type: object + failureThreshold: + description: Minimum consecutive failures for the probe + to be considered failed after having succeeded. Defaults + to 3. Minimum value is 1. + format: int32 + type: integer + grpc: + description: GRPC specifies an action involving a GRPC port. + properties: + port: + description: Port number of the gRPC service. Number + must be in the range 1 to 65535. + format: int32 + type: integer + service: + description: "Service is the name of the service to + place in the gRPC HealthCheckRequest (see https://github.com/grpc/grpc/blob/master/doc/health-checking.md). + \n If this is not specified, the default behavior + is defined by gRPC." + type: string + required: + - port + type: object + httpGet: + description: HTTPGet specifies the http request to perform. + properties: + host: + description: Host name to connect to, defaults to the + pod IP. You probably want to set "Host" in httpHeaders + instead. + type: string + httpHeaders: + description: Custom headers to set in the request. HTTP + allows repeated headers. + items: + description: HTTPHeader describes a custom header + to be used in HTTP probes + properties: + name: + description: The header field name. This will + be canonicalized upon output, so case-variant + names will be understood as the same header. + type: string + value: + description: The header field value + type: string + required: + - name + - value + type: object + type: array + path: + description: Path to access on the HTTP server. + type: string + port: + anyOf: + - type: integer + - type: string + description: Name or number of the port to access on + the container. Number must be in the range 1 to 65535. + Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + scheme: + description: Scheme to use for connecting to the host. + Defaults to HTTP. + type: string + required: + - port + type: object + initialDelaySeconds: + description: 'Number of seconds after the container has + started before liveness probes are initiated. More info: + https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes' + format: int32 + type: integer + periodSeconds: + description: How often (in seconds) to perform the probe. + Default to 10 seconds. Minimum value is 1. + format: int32 + type: integer + successThreshold: + description: Minimum consecutive successes for the probe + to be considered successful after having failed. Defaults + to 1. Must be 1 for liveness and startup. Minimum value + is 1. + format: int32 + type: integer + tcpSocket: + description: TCPSocket specifies an action involving a TCP + port. + properties: + host: + description: 'Optional: Host name to connect to, defaults + to the pod IP.' + type: string + port: + anyOf: + - type: integer + - type: string + description: Number or name of the port to access on + the container. Number must be in the range 1 to 65535. + Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + required: + - port + type: object + terminationGracePeriodSeconds: + description: Optional duration in seconds the pod needs + to terminate gracefully upon probe failure. The grace + period is the duration in seconds after the processes + running in the pod are sent a termination signal and the + time when the processes are forcibly halted with a kill + signal. Set this value longer than the expected cleanup + time for your process. If this value is nil, the pod's + terminationGracePeriodSeconds will be used. Otherwise, + this value overrides the value provided by the pod spec. + Value must be non-negative integer. The value zero indicates + stop immediately via the kill signal (no opportunity to + shut down). This is a beta field and requires enabling + ProbeTerminationGracePeriod feature gate. Minimum value + is 1. spec.terminationGracePeriodSeconds is used if unset. + format: int64 + type: integer + timeoutSeconds: + description: 'Number of seconds after which the probe times + out. Defaults to 1 second. Minimum value is 1. More info: + https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes' + format: int32 + type: integer + type: object + name: + description: Name of the container specified as a DNS_LABEL. + Each container in a pod must have a unique name (DNS_LABEL). + Cannot be updated. + type: string + ports: + description: List of ports to expose from the container. Not + specifying a port here DOES NOT prevent that port from being + exposed. Any port which is listening on the default "0.0.0.0" + address inside a container will be accessible from the network. + Modifying this array with strategic merge patch may corrupt + the data. For more information See https://github.com/kubernetes/kubernetes/issues/108255. + Cannot be updated. + items: + description: ContainerPort represents a network port in a + single container. + properties: + containerPort: + description: Number of port to expose on the pod's IP + address. This must be a valid port number, 0 < x < 65536. + format: int32 + type: integer + hostIP: + description: What host IP to bind the external port to. + type: string + hostPort: + description: Number of port to expose on the host. If + specified, this must be a valid port number, 0 < x < + 65536. If HostNetwork is specified, this must match + ContainerPort. Most containers do not need this. + format: int32 + type: integer + name: + description: If specified, this must be an IANA_SVC_NAME + and unique within the pod. Each named port in a pod + must have a unique name. Name for the port that can + be referred to by services. + type: string + protocol: + default: TCP + description: Protocol for port. Must be UDP, TCP, or SCTP. + Defaults to "TCP". + type: string + required: + - containerPort + type: object + type: array + x-kubernetes-list-map-keys: + - containerPort + - protocol + x-kubernetes-list-type: map + readinessProbe: + description: 'Periodic probe of container service readiness. + Container will be removed from service endpoints if the probe + fails. Cannot be updated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes' + properties: + exec: + description: Exec specifies the action to take. + properties: + command: + description: Command is the command line to execute + inside the container, the working directory for the + command is root ('/') in the container's filesystem. + The command is simply exec'd, it is not run inside + a shell, so traditional shell instructions ('|', etc) + won't work. To use a shell, you need to explicitly + call out to that shell. Exit status of 0 is treated + as live/healthy and non-zero is unhealthy. + items: + type: string + type: array + type: object + failureThreshold: + description: Minimum consecutive failures for the probe + to be considered failed after having succeeded. Defaults + to 3. Minimum value is 1. + format: int32 + type: integer + grpc: + description: GRPC specifies an action involving a GRPC port. + properties: + port: + description: Port number of the gRPC service. Number + must be in the range 1 to 65535. + format: int32 + type: integer + service: + description: "Service is the name of the service to + place in the gRPC HealthCheckRequest (see https://github.com/grpc/grpc/blob/master/doc/health-checking.md). + \n If this is not specified, the default behavior + is defined by gRPC." + type: string + required: + - port + type: object + httpGet: + description: HTTPGet specifies the http request to perform. + properties: + host: + description: Host name to connect to, defaults to the + pod IP. You probably want to set "Host" in httpHeaders + instead. + type: string + httpHeaders: + description: Custom headers to set in the request. HTTP + allows repeated headers. + items: + description: HTTPHeader describes a custom header + to be used in HTTP probes + properties: + name: + description: The header field name. This will + be canonicalized upon output, so case-variant + names will be understood as the same header. + type: string + value: + description: The header field value + type: string + required: + - name + - value + type: object + type: array + path: + description: Path to access on the HTTP server. + type: string + port: + anyOf: + - type: integer + - type: string + description: Name or number of the port to access on + the container. Number must be in the range 1 to 65535. + Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + scheme: + description: Scheme to use for connecting to the host. + Defaults to HTTP. + type: string + required: + - port + type: object + initialDelaySeconds: + description: 'Number of seconds after the container has + started before liveness probes are initiated. More info: + https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes' + format: int32 + type: integer + periodSeconds: + description: How often (in seconds) to perform the probe. + Default to 10 seconds. Minimum value is 1. + format: int32 + type: integer + successThreshold: + description: Minimum consecutive successes for the probe + to be considered successful after having failed. Defaults + to 1. Must be 1 for liveness and startup. Minimum value + is 1. + format: int32 + type: integer + tcpSocket: + description: TCPSocket specifies an action involving a TCP + port. + properties: + host: + description: 'Optional: Host name to connect to, defaults + to the pod IP.' + type: string + port: + anyOf: + - type: integer + - type: string + description: Number or name of the port to access on + the container. Number must be in the range 1 to 65535. + Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + required: + - port + type: object + terminationGracePeriodSeconds: + description: Optional duration in seconds the pod needs + to terminate gracefully upon probe failure. The grace + period is the duration in seconds after the processes + running in the pod are sent a termination signal and the + time when the processes are forcibly halted with a kill + signal. Set this value longer than the expected cleanup + time for your process. If this value is nil, the pod's + terminationGracePeriodSeconds will be used. Otherwise, + this value overrides the value provided by the pod spec. + Value must be non-negative integer. The value zero indicates + stop immediately via the kill signal (no opportunity to + shut down). This is a beta field and requires enabling + ProbeTerminationGracePeriod feature gate. Minimum value + is 1. spec.terminationGracePeriodSeconds is used if unset. + format: int64 + type: integer + timeoutSeconds: + description: 'Number of seconds after which the probe times + out. Defaults to 1 second. Minimum value is 1. More info: + https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes' + format: int32 + type: integer + type: object + resizePolicy: + description: Resources resize policy for the container. + items: + description: ContainerResizePolicy represents resource resize + policy for the container. + properties: + resourceName: + description: 'Name of the resource to which this resource + resize policy applies. Supported values: cpu, memory.' + type: string + restartPolicy: + description: Restart policy to apply when specified resource + is resized. If not specified, it defaults to NotRequired. + type: string + required: + - resourceName + - restartPolicy + type: object + type: array + x-kubernetes-list-type: atomic + resources: + description: 'Compute Resources required by this container. + Cannot be updated. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + properties: + claims: + description: "Claims lists the names of resources, defined + in spec.resourceClaims, that are used by this container. + \n This is an alpha field and requires enabling the DynamicResourceAllocation + feature gate. \n This field is immutable. It can only + be set for containers." + items: + description: ResourceClaim references one entry in PodSpec.ResourceClaims. + properties: + name: + description: Name must match the name of one entry + in pod.spec.resourceClaims of the Pod where this + field is used. It makes that resource available + inside a container. + type: string + required: + - name + type: object + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + limits: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + description: 'Limits describes the maximum amount of compute + resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + type: object + requests: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + description: 'Requests describes the minimum amount of compute + resources required. If Requests is omitted for a container, + it defaults to Limits if that is explicitly specified, + otherwise to an implementation-defined value. Requests + cannot exceed Limits. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + type: object + type: object + restartPolicy: + description: 'RestartPolicy defines the restart behavior of + individual containers in a pod. This field may only be set + for init containers, and the only allowed value is "Always". + For non-init containers or when this field is not specified, + the restart behavior is defined by the Pod''s restart policy + and the container type. Setting the RestartPolicy as "Always" + for the init container will have the following effect: this + init container will be continually restarted on exit until + all regular containers have terminated. Once all regular containers + have completed, all init containers with restartPolicy "Always" + will be shut down. This lifecycle differs from normal init + containers and is often referred to as a "sidecar" container. + Although this init container still starts in the init container + sequence, it does not wait for the container to complete before + proceeding to the next init container. Instead, the next init + container starts immediately after this init container is + started, or after any startupProbe has successfully completed.' + type: string + securityContext: + description: 'SecurityContext defines the security options the + container should be run with. If set, the fields of SecurityContext + override the equivalent fields of PodSecurityContext. More + info: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/' + properties: + allowPrivilegeEscalation: + description: 'AllowPrivilegeEscalation controls whether + a process can gain more privileges than its parent process. + This bool directly controls if the no_new_privs flag will + be set on the container process. AllowPrivilegeEscalation + is true always when the container is: 1) run as Privileged + 2) has CAP_SYS_ADMIN Note that this field cannot be set + when spec.os.name is windows.' + type: boolean + capabilities: + description: The capabilities to add/drop when running containers. + Defaults to the default set of capabilities granted by + the container runtime. Note that this field cannot be + set when spec.os.name is windows. + properties: + add: + description: Added capabilities + items: + description: Capability represent POSIX capabilities + type + type: string + type: array + drop: + description: Removed capabilities + items: + description: Capability represent POSIX capabilities + type + type: string + type: array + type: object + privileged: + description: Run container in privileged mode. Processes + in privileged containers are essentially equivalent to + root on the host. Defaults to false. Note that this field + cannot be set when spec.os.name is windows. + type: boolean + procMount: + description: procMount denotes the type of proc mount to + use for the containers. The default is DefaultProcMount + which uses the container runtime defaults for readonly + paths and masked paths. This requires the ProcMountType + feature flag to be enabled. Note that this field cannot + be set when spec.os.name is windows. + type: string + readOnlyRootFilesystem: + description: Whether this container has a read-only root + filesystem. Default is false. Note that this field cannot + be set when spec.os.name is windows. + type: boolean + runAsGroup: + description: The GID to run the entrypoint of the container + process. Uses runtime default if unset. May also be set + in PodSecurityContext. If set in both SecurityContext + and PodSecurityContext, the value specified in SecurityContext + takes precedence. Note that this field cannot be set when + spec.os.name is windows. + format: int64 + type: integer + runAsNonRoot: + description: Indicates that the container must run as a + non-root user. If true, the Kubelet will validate the + image at runtime to ensure that it does not run as UID + 0 (root) and fail to start the container if it does. If + unset or false, no such validation will be performed. + May also be set in PodSecurityContext. If set in both + SecurityContext and PodSecurityContext, the value specified + in SecurityContext takes precedence. + type: boolean + runAsUser: + description: The UID to run the entrypoint of the container + process. Defaults to user specified in image metadata + if unspecified. May also be set in PodSecurityContext. If + set in both SecurityContext and PodSecurityContext, the + value specified in SecurityContext takes precedence. Note + that this field cannot be set when spec.os.name is windows. + format: int64 + type: integer + seLinuxOptions: + description: The SELinux context to be applied to the container. + If unspecified, the container runtime will allocate a + random SELinux context for each container. May also be + set in PodSecurityContext. If set in both SecurityContext + and PodSecurityContext, the value specified in SecurityContext + takes precedence. Note that this field cannot be set when + spec.os.name is windows. + properties: + level: + description: Level is SELinux level label that applies + to the container. + type: string + role: + description: Role is a SELinux role label that applies + to the container. + type: string + type: + description: Type is a SELinux type label that applies + to the container. + type: string + user: + description: User is a SELinux user label that applies + to the container. + type: string + type: object + seccompProfile: + description: The seccomp options to use by this container. + If seccomp options are provided at both the pod & container + level, the container options override the pod options. + Note that this field cannot be set when spec.os.name is + windows. + properties: + localhostProfile: + description: localhostProfile indicates a profile defined + in a file on the node should be used. The profile + must be preconfigured on the node to work. Must be + a descending path, relative to the kubelet's configured + seccomp profile location. Must be set if type is "Localhost". + Must NOT be set for any other type. + type: string + type: + description: "type indicates which kind of seccomp profile + will be applied. Valid options are: \n Localhost - + a profile defined in a file on the node should be + used. RuntimeDefault - the container runtime default + profile should be used. Unconfined - no profile should + be applied." + type: string + required: + - type + type: object + windowsOptions: + description: The Windows specific settings applied to all + containers. If unspecified, the options from the PodSecurityContext + will be used. If set in both SecurityContext and PodSecurityContext, + the value specified in SecurityContext takes precedence. + Note that this field cannot be set when spec.os.name is + linux. + properties: + gmsaCredentialSpec: + description: GMSACredentialSpec is where the GMSA admission + webhook (https://github.com/kubernetes-sigs/windows-gmsa) + inlines the contents of the GMSA credential spec named + by the GMSACredentialSpecName field. + type: string + gmsaCredentialSpecName: + description: GMSACredentialSpecName is the name of the + GMSA credential spec to use. + type: string + hostProcess: + description: HostProcess determines if a container should + be run as a 'Host Process' container. All of a Pod's + containers must have the same effective HostProcess + value (it is not allowed to have a mix of HostProcess + containers and non-HostProcess containers). In addition, + if HostProcess is true then HostNetwork must also + be set to true. + type: boolean + runAsUserName: + description: The UserName in Windows to run the entrypoint + of the container process. Defaults to the user specified + in image metadata if unspecified. May also be set + in PodSecurityContext. If set in both SecurityContext + and PodSecurityContext, the value specified in SecurityContext + takes precedence. + type: string + type: object + type: object + startupProbe: + description: 'StartupProbe indicates that the Pod has successfully + initialized. If specified, no other probes are executed until + this completes successfully. If this probe fails, the Pod + will be restarted, just as if the livenessProbe failed. This + can be used to provide different probe parameters at the beginning + of a Pod''s lifecycle, when it might take a long time to load + data or warm a cache, than during steady-state operation. + This cannot be updated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes' + properties: + exec: + description: Exec specifies the action to take. + properties: + command: + description: Command is the command line to execute + inside the container, the working directory for the + command is root ('/') in the container's filesystem. + The command is simply exec'd, it is not run inside + a shell, so traditional shell instructions ('|', etc) + won't work. To use a shell, you need to explicitly + call out to that shell. Exit status of 0 is treated + as live/healthy and non-zero is unhealthy. + items: + type: string + type: array + type: object + failureThreshold: + description: Minimum consecutive failures for the probe + to be considered failed after having succeeded. Defaults + to 3. Minimum value is 1. + format: int32 + type: integer + grpc: + description: GRPC specifies an action involving a GRPC port. + properties: + port: + description: Port number of the gRPC service. Number + must be in the range 1 to 65535. + format: int32 + type: integer + service: + description: "Service is the name of the service to + place in the gRPC HealthCheckRequest (see https://github.com/grpc/grpc/blob/master/doc/health-checking.md). + \n If this is not specified, the default behavior + is defined by gRPC." + type: string + required: + - port + type: object + httpGet: + description: HTTPGet specifies the http request to perform. + properties: + host: + description: Host name to connect to, defaults to the + pod IP. You probably want to set "Host" in httpHeaders + instead. + type: string + httpHeaders: + description: Custom headers to set in the request. HTTP + allows repeated headers. + items: + description: HTTPHeader describes a custom header + to be used in HTTP probes + properties: + name: + description: The header field name. This will + be canonicalized upon output, so case-variant + names will be understood as the same header. + type: string + value: + description: The header field value + type: string + required: + - name + - value + type: object + type: array + path: + description: Path to access on the HTTP server. + type: string + port: + anyOf: + - type: integer + - type: string + description: Name or number of the port to access on + the container. Number must be in the range 1 to 65535. + Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + scheme: + description: Scheme to use for connecting to the host. + Defaults to HTTP. + type: string + required: + - port + type: object + initialDelaySeconds: + description: 'Number of seconds after the container has + started before liveness probes are initiated. More info: + https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes' + format: int32 + type: integer + periodSeconds: + description: How often (in seconds) to perform the probe. + Default to 10 seconds. Minimum value is 1. + format: int32 + type: integer + successThreshold: + description: Minimum consecutive successes for the probe + to be considered successful after having failed. Defaults + to 1. Must be 1 for liveness and startup. Minimum value + is 1. + format: int32 + type: integer + tcpSocket: + description: TCPSocket specifies an action involving a TCP + port. + properties: + host: + description: 'Optional: Host name to connect to, defaults + to the pod IP.' + type: string + port: + anyOf: + - type: integer + - type: string + description: Number or name of the port to access on + the container. Number must be in the range 1 to 65535. + Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + required: + - port + type: object + terminationGracePeriodSeconds: + description: Optional duration in seconds the pod needs + to terminate gracefully upon probe failure. The grace + period is the duration in seconds after the processes + running in the pod are sent a termination signal and the + time when the processes are forcibly halted with a kill + signal. Set this value longer than the expected cleanup + time for your process. If this value is nil, the pod's + terminationGracePeriodSeconds will be used. Otherwise, + this value overrides the value provided by the pod spec. + Value must be non-negative integer. The value zero indicates + stop immediately via the kill signal (no opportunity to + shut down). This is a beta field and requires enabling + ProbeTerminationGracePeriod feature gate. Minimum value + is 1. spec.terminationGracePeriodSeconds is used if unset. + format: int64 + type: integer + timeoutSeconds: + description: 'Number of seconds after which the probe times + out. Defaults to 1 second. Minimum value is 1. More info: + https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes' + format: int32 + type: integer + type: object + stdin: + description: Whether this container should allocate a buffer + for stdin in the container runtime. If this is not set, reads + from stdin in the container will always result in EOF. Default + is false. + type: boolean + stdinOnce: + description: Whether the container runtime should close the + stdin channel after it has been opened by a single attach. + When stdin is true the stdin stream will remain open across + multiple attach sessions. If stdinOnce is set to true, stdin + is opened on container start, is empty until the first client + attaches to stdin, and then remains open and accepts data + until the client disconnects, at which time stdin is closed + and remains closed until the container is restarted. If this + flag is false, a container processes that reads from stdin + will never receive an EOF. Default is false + type: boolean + terminationMessagePath: + description: 'Optional: Path at which the file to which the + container''s termination message will be written is mounted + into the container''s filesystem. Message written is intended + to be brief final status, such as an assertion failure message. + Will be truncated by the node if greater than 4096 bytes. + The total message length across all containers will be limited + to 12kb. Defaults to /dev/termination-log. Cannot be updated.' + type: string + terminationMessagePolicy: + description: Indicate how the termination message should be + populated. File will use the contents of terminationMessagePath + to populate the container status message on both success and + failure. FallbackToLogsOnError will use the last chunk of + container log output if the termination message file is empty + and the container exited with an error. The log output is + limited to 2048 bytes or 80 lines, whichever is smaller. Defaults + to File. Cannot be updated. + type: string + tty: + description: Whether this container should allocate a TTY for + itself, also requires 'stdin' to be true. Default is false. + type: boolean + volumeDevices: + description: volumeDevices is the list of block devices to be + used by the container. + items: + description: volumeDevice describes a mapping of a raw block + device within a container. + properties: + devicePath: + description: devicePath is the path inside of the container + that the device will be mapped to. + type: string + name: + description: name must match the name of a persistentVolumeClaim + in the pod + type: string + required: + - devicePath + - name + type: object + type: array + volumeMounts: + description: Pod volumes to mount into the container's filesystem. + Cannot be updated. + items: + description: VolumeMount describes a mounting of a Volume + within a container. + properties: + mountPath: + description: Path within the container at which the volume + should be mounted. Must not contain ':'. + type: string + mountPropagation: + description: mountPropagation determines how mounts are + propagated from the host to container and the other + way around. When not set, MountPropagationNone is used. + This field is beta in 1.10. + type: string + name: + description: This must match the Name of a Volume. + type: string + readOnly: + description: Mounted read-only if true, read-write otherwise + (false or unspecified). Defaults to false. + type: boolean + subPath: + description: Path within the volume from which the container's + volume should be mounted. Defaults to "" (volume's root). + type: string + subPathExpr: + description: Expanded path within the volume from which + the container's volume should be mounted. Behaves similarly + to SubPath but environment variable references $(VAR_NAME) + are expanded using the container's environment. Defaults + to "" (volume's root). SubPathExpr and SubPath are mutually + exclusive. + type: string + required: + - mountPath + - name + type: object + type: array + workingDir: + description: Container's working directory. If not specified, + the container runtime's default will be used, which might + be configured in the container image. Cannot be updated. + type: string + required: + - name + type: object + type: array + type: object + status: + description: DatabaseObserverStatus defines the observed state of DatabaseObserver + properties: + conditions: + items: + description: "Condition contains details for one aspect of the current + state of this API Resource. --- This struct is intended for direct + use as an array at the field path .status.conditions. For example, + \n \ttype FooStatus struct{ \t // Represents the observations + of a foo's current state. \t // Known .status.conditions.type + are: \"Available\", \"Progressing\", and \"Degraded\" \t // + +patchMergeKey=type \t // +patchStrategy=merge \t // +listType=map + \t // +listMapKey=type \t Conditions []metav1.Condition + `json:\"conditions,omitempty\" patchStrategy:\"merge\" patchMergeKey:\"type\" + protobuf:\"bytes,1,rep,name=conditions\"` \n \t // other fields + \t}" + properties: + lastTransitionTime: + description: lastTransitionTime is the last time the condition + transitioned from one status to another. This should be when + the underlying condition changed. If that is not known, then + using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: message is a human readable message indicating + details about the transition. This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: observedGeneration represents the .metadata.generation + that the condition was set based upon. For instance, if .metadata.generation + is currently 12, but the .status.conditions[x].observedGeneration + is 9, the condition is out of date with respect to the current + state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: reason contains a programmatic identifier indicating + the reason for the condition's last transition. Producers + of specific condition types may define expected values and + meanings for this field, and whether the values are considered + a guaranteed API. The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + --- Many .condition.type values are consistent across resources + like Available, but because arbitrary conditions can be useful + (see .node.status.conditions), the ability to deconflict is + important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + exporterConfig: + type: string + replicas: + type: integer + status: + type: string + version: + type: string + required: + - conditions + - exporterConfig + - version + type: object + type: object + served: true + storage: false + subresources: + status: {} + - additionalPrinterColumns: + - jsonPath: .status.exporterConfig + name: ExporterConfig + type: string + - jsonPath: .status.status + name: Status + type: string + - jsonPath: .status.version + name: Version + type: string name: v1alpha1 schema: openAPIV3Schema: @@ -43,6 +3217,3189 @@ spec: spec: description: DatabaseObserverSpec defines the desired state of DatabaseObserver properties: + configuration: + properties: + configMap: + description: ConfigMapDetails defines the configmap name + properties: + key: + type: string + name: + type: string + type: object + type: object + database: + description: DatabaseObserverDatabase defines the database details + used for DatabaseObserver + properties: + dbConnectionString: + properties: + key: + type: string + secret: + type: string + type: object + dbPassword: + properties: + key: + type: string + secret: + type: string + vaultOCID: + type: string + vaultSecretName: + type: string + type: object + dbRole: + type: string + dbUser: + properties: + key: + type: string + secret: + type: string + type: object + dbWallet: + properties: + key: + type: string + secret: + type: string + type: object + type: object + exporter: + description: DatabaseObserverExporterConfig defines the configuration + details related to the exporters of DatabaseObserver + properties: + args: + items: + type: string + type: array + commands: + items: + type: string + type: array + image: + type: string + service: + description: DatabaseObserverService defines the exporter service + component of DatabaseObserver + properties: + port: + format: int32 + type: integer + type: object + type: object + inherit_labels: + items: + type: string + type: array + log: + description: LogConfig defines the configuration details relation + to the logs of DatabaseObserver + properties: + filename: + type: string + path: + type: string + volume: + properties: + name: + type: string + persistentVolumeClaim: + properties: + claimName: + type: string + type: object + type: object + type: object + ociConfig: + properties: + configMapName: + type: string + secretName: + type: string + type: object + prometheus: + description: PrometheusConfig defines the generated resources for + Prometheus + properties: + port: + type: string + release: + type: string + type: object + replicas: + format: int32 + type: integer + sidecarVolumes: + items: + description: Volume represents a named volume in a pod that may + be accessed by any container in the pod. + properties: + awsElasticBlockStore: + description: 'awsElasticBlockStore represents an AWS Disk resource + that is attached to a kubelet''s host machine and then exposed + to the pod. More info: https://kubernetes.io/docs/concepts/storage/volumes#awselasticblockstore' + properties: + fsType: + description: 'fsType is the filesystem type of the volume + that you want to mount. Tip: Ensure that the filesystem + type is supported by the host operating system. Examples: + "ext4", "xfs", "ntfs". Implicitly inferred to be "ext4" + if unspecified. More info: https://kubernetes.io/docs/concepts/storage/volumes#awselasticblockstore + TODO: how do we prevent errors in the filesystem from + compromising the machine' + type: string + partition: + description: 'partition is the partition in the volume that + you want to mount. If omitted, the default is to mount + by volume name. Examples: For volume /dev/sda1, you specify + the partition as "1". Similarly, the volume partition + for /dev/sda is "0" (or you can leave the property empty).' + format: int32 + type: integer + readOnly: + description: 'readOnly value true will force the readOnly + setting in VolumeMounts. More info: https://kubernetes.io/docs/concepts/storage/volumes#awselasticblockstore' + type: boolean + volumeID: + description: 'volumeID is unique ID of the persistent disk + resource in AWS (Amazon EBS volume). More info: https://kubernetes.io/docs/concepts/storage/volumes#awselasticblockstore' + type: string + required: + - volumeID + type: object + azureDisk: + description: azureDisk represents an Azure Data Disk mount on + the host and bind mount to the pod. + properties: + cachingMode: + description: 'cachingMode is the Host Caching mode: None, + Read Only, Read Write.' + type: string + diskName: + description: diskName is the Name of the data disk in the + blob storage + type: string + diskURI: + description: diskURI is the URI of data disk in the blob + storage + type: string + fsType: + description: fsType is Filesystem type to mount. Must be + a filesystem type supported by the host operating system. + Ex. "ext4", "xfs", "ntfs". Implicitly inferred to be "ext4" + if unspecified. + type: string + kind: + description: 'kind expected values are Shared: multiple + blob disks per storage account Dedicated: single blob + disk per storage account Managed: azure managed data + disk (only in managed availability set). defaults to shared' + type: string + readOnly: + description: readOnly Defaults to false (read/write). ReadOnly + here will force the ReadOnly setting in VolumeMounts. + type: boolean + required: + - diskName + - diskURI + type: object + azureFile: + description: azureFile represents an Azure File Service mount + on the host and bind mount to the pod. + properties: + readOnly: + description: readOnly defaults to false (read/write). ReadOnly + here will force the ReadOnly setting in VolumeMounts. + type: boolean + secretName: + description: secretName is the name of secret that contains + Azure Storage Account Name and Key + type: string + shareName: + description: shareName is the azure share Name + type: string + required: + - secretName + - shareName + type: object + cephfs: + description: cephFS represents a Ceph FS mount on the host that + shares a pod's lifetime + properties: + monitors: + description: 'monitors is Required: Monitors is a collection + of Ceph monitors More info: https://examples.k8s.io/volumes/cephfs/README.md#how-to-use-it' + items: + type: string + type: array + path: + description: 'path is Optional: Used as the mounted root, + rather than the full Ceph tree, default is /' + type: string + readOnly: + description: 'readOnly is Optional: Defaults to false (read/write). + ReadOnly here will force the ReadOnly setting in VolumeMounts. + More info: https://examples.k8s.io/volumes/cephfs/README.md#how-to-use-it' + type: boolean + secretFile: + description: 'secretFile is Optional: SecretFile is the + path to key ring for User, default is /etc/ceph/user.secret + More info: https://examples.k8s.io/volumes/cephfs/README.md#how-to-use-it' + type: string + secretRef: + description: 'secretRef is Optional: SecretRef is reference + to the authentication secret for User, default is empty. + More info: https://examples.k8s.io/volumes/cephfs/README.md#how-to-use-it' + properties: + name: + description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, uid?' + type: string + type: object + user: + description: 'user is optional: User is the rados user name, + default is admin More info: https://examples.k8s.io/volumes/cephfs/README.md#how-to-use-it' + type: string + required: + - monitors + type: object + cinder: + description: 'cinder represents a cinder volume attached and + mounted on kubelets host machine. More info: https://examples.k8s.io/mysql-cinder-pd/README.md' + properties: + fsType: + description: 'fsType is the filesystem type to mount. Must + be a filesystem type supported by the host operating system. + Examples: "ext4", "xfs", "ntfs". Implicitly inferred to + be "ext4" if unspecified. More info: https://examples.k8s.io/mysql-cinder-pd/README.md' + type: string + readOnly: + description: 'readOnly defaults to false (read/write). ReadOnly + here will force the ReadOnly setting in VolumeMounts. + More info: https://examples.k8s.io/mysql-cinder-pd/README.md' + type: boolean + secretRef: + description: 'secretRef is optional: points to a secret + object containing parameters used to connect to OpenStack.' + properties: + name: + description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, uid?' + type: string + type: object + volumeID: + description: 'volumeID used to identify the volume in cinder. + More info: https://examples.k8s.io/mysql-cinder-pd/README.md' + type: string + required: + - volumeID + type: object + configMap: + description: configMap represents a configMap that should populate + this volume + properties: + defaultMode: + description: 'defaultMode is optional: mode bits used to + set permissions on created files by default. Must be an + octal value between 0000 and 0777 or a decimal value between + 0 and 511. YAML accepts both octal and decimal values, + JSON requires decimal values for mode bits. Defaults to + 0644. Directories within the path are not affected by + this setting. This might be in conflict with other options + that affect the file mode, like fsGroup, and the result + can be other mode bits set.' + format: int32 + type: integer + items: + description: items if unspecified, each key-value pair in + the Data field of the referenced ConfigMap will be projected + into the volume as a file whose name is the key and content + is the value. If specified, the listed keys will be projected + into the specified paths, and unlisted keys will not be + present. If a key is specified which is not present in + the ConfigMap, the volume setup will error unless it is + marked optional. Paths must be relative and may not contain + the '..' path or start with '..'. + items: + description: Maps a string key to a path within a volume. + properties: + key: + description: key is the key to project. + type: string + mode: + description: 'mode is Optional: mode bits used to + set permissions on this file. Must be an octal value + between 0000 and 0777 or a decimal value between + 0 and 511. YAML accepts both octal and decimal values, + JSON requires decimal values for mode bits. If not + specified, the volume defaultMode will be used. + This might be in conflict with other options that + affect the file mode, like fsGroup, and the result + can be other mode bits set.' + format: int32 + type: integer + path: + description: path is the relative path of the file + to map the key to. May not be an absolute path. + May not contain the path element '..'. May not start + with the string '..'. + type: string + required: + - key + - path + type: object + type: array + name: + description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, uid?' + type: string + optional: + description: optional specify whether the ConfigMap or its + keys must be defined + type: boolean + type: object + csi: + description: csi (Container Storage Interface) represents ephemeral + storage that is handled by certain external CSI drivers (Beta + feature). + properties: + driver: + description: driver is the name of the CSI driver that handles + this volume. Consult with your admin for the correct name + as registered in the cluster. + type: string + fsType: + description: fsType to mount. Ex. "ext4", "xfs", "ntfs". + If not provided, the empty value is passed to the associated + CSI driver which will determine the default filesystem + to apply. + type: string + nodePublishSecretRef: + description: nodePublishSecretRef is a reference to the + secret object containing sensitive information to pass + to the CSI driver to complete the CSI NodePublishVolume + and NodeUnpublishVolume calls. This field is optional, + and may be empty if no secret is required. If the secret + object contains more than one secret, all secret references + are passed. + properties: + name: + description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, uid?' + type: string + type: object + readOnly: + description: readOnly specifies a read-only configuration + for the volume. Defaults to false (read/write). + type: boolean + volumeAttributes: + additionalProperties: + type: string + description: volumeAttributes stores driver-specific properties + that are passed to the CSI driver. Consult your driver's + documentation for supported values. + type: object + required: + - driver + type: object + downwardAPI: + description: downwardAPI represents downward API about the pod + that should populate this volume + properties: + defaultMode: + description: 'Optional: mode bits to use on created files + by default. Must be a Optional: mode bits used to set + permissions on created files by default. Must be an octal + value between 0000 and 0777 or a decimal value between + 0 and 511. YAML accepts both octal and decimal values, + JSON requires decimal values for mode bits. Defaults to + 0644. Directories within the path are not affected by + this setting. This might be in conflict with other options + that affect the file mode, like fsGroup, and the result + can be other mode bits set.' + format: int32 + type: integer + items: + description: Items is a list of downward API volume file + items: + description: DownwardAPIVolumeFile represents information + to create the file containing the pod field + properties: + fieldRef: + description: 'Required: Selects a field of the pod: + only annotations, labels, name and namespace are + supported.' + properties: + apiVersion: + description: Version of the schema the FieldPath + is written in terms of, defaults to "v1". + type: string + fieldPath: + description: Path of the field to select in the + specified API version. + type: string + required: + - fieldPath + type: object + mode: + description: 'Optional: mode bits used to set permissions + on this file, must be an octal value between 0000 + and 0777 or a decimal value between 0 and 511. YAML + accepts both octal and decimal values, JSON requires + decimal values for mode bits. If not specified, + the volume defaultMode will be used. This might + be in conflict with other options that affect the + file mode, like fsGroup, and the result can be other + mode bits set.' + format: int32 + type: integer + path: + description: 'Required: Path is the relative path + name of the file to be created. Must not be absolute + or contain the ''..'' path. Must be utf-8 encoded. + The first item of the relative path must not start + with ''..''' + type: string + resourceFieldRef: + description: 'Selects a resource of the container: + only resources limits and requests (limits.cpu, + limits.memory, requests.cpu and requests.memory) + are currently supported.' + properties: + containerName: + description: 'Container name: required for volumes, + optional for env vars' + type: string + divisor: + anyOf: + - type: integer + - type: string + description: Specifies the output format of the + exposed resources, defaults to "1" + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + resource: + description: 'Required: resource to select' + type: string + required: + - resource + type: object + required: + - path + type: object + type: array + type: object + emptyDir: + description: 'emptyDir represents a temporary directory that + shares a pod''s lifetime. More info: https://kubernetes.io/docs/concepts/storage/volumes#emptydir' + properties: + medium: + description: 'medium represents what type of storage medium + should back this directory. The default is "" which means + to use the node''s default medium. Must be an empty string + (default) or Memory. More info: https://kubernetes.io/docs/concepts/storage/volumes#emptydir' + type: string + sizeLimit: + anyOf: + - type: integer + - type: string + description: 'sizeLimit is the total amount of local storage + required for this EmptyDir volume. The size limit is also + applicable for memory medium. The maximum usage on memory + medium EmptyDir would be the minimum value between the + SizeLimit specified here and the sum of memory limits + of all containers in a pod. The default is nil which means + that the limit is undefined. More info: https://kubernetes.io/docs/concepts/storage/volumes#emptydir' + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + ephemeral: + description: "ephemeral represents a volume that is handled + by a cluster storage driver. The volume's lifecycle is tied + to the pod that defines it - it will be created before the + pod starts, and deleted when the pod is removed. \n Use this + if: a) the volume is only needed while the pod runs, b) features + of normal volumes like restoring from snapshot or capacity + \ tracking are needed, c) the storage driver is specified + through a storage class, and d) the storage driver supports + dynamic volume provisioning through a PersistentVolumeClaim + (see EphemeralVolumeSource for more information on the + connection between this volume type and PersistentVolumeClaim). + \n Use PersistentVolumeClaim or one of the vendor-specific + APIs for volumes that persist for longer than the lifecycle + of an individual pod. \n Use CSI for light-weight local ephemeral + volumes if the CSI driver is meant to be used that way - see + the documentation of the driver for more information. \n A + pod can use both types of ephemeral volumes and persistent + volumes at the same time." + properties: + volumeClaimTemplate: + description: "Will be used to create a stand-alone PVC to + provision the volume. The pod in which this EphemeralVolumeSource + is embedded will be the owner of the PVC, i.e. the PVC + will be deleted together with the pod. The name of the + PVC will be `-` where `` is the name from the `PodSpec.Volumes` array entry. + Pod validation will reject the pod if the concatenated + name is not valid for a PVC (for example, too long). \n + An existing PVC with that name that is not owned by the + pod will *not* be used for the pod to avoid using an unrelated + volume by mistake. Starting the pod is then blocked until + the unrelated PVC is removed. If such a pre-created PVC + is meant to be used by the pod, the PVC has to updated + with an owner reference to the pod once the pod exists. + Normally this should not be necessary, but it may be useful + when manually reconstructing a broken cluster. \n This + field is read-only and no changes will be made by Kubernetes + to the PVC after it has been created. \n Required, must + not be nil." + properties: + metadata: + description: May contain labels and annotations that + will be copied into the PVC when creating it. No other + fields are allowed and will be rejected during validation. + type: object + spec: + description: The specification for the PersistentVolumeClaim. + The entire content is copied unchanged into the PVC + that gets created from this template. The same fields + as in a PersistentVolumeClaim are also valid here. + properties: + accessModes: + description: 'accessModes contains the desired access + modes the volume should have. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1' + items: + type: string + type: array + dataSource: + description: 'dataSource field can be used to specify + either: * An existing VolumeSnapshot object (snapshot.storage.k8s.io/VolumeSnapshot) + * An existing PVC (PersistentVolumeClaim) If the + provisioner or an external controller can support + the specified data source, it will create a new + volume based on the contents of the specified + data source. When the AnyVolumeDataSource feature + gate is enabled, dataSource contents will be copied + to dataSourceRef, and dataSourceRef contents will + be copied to dataSource when dataSourceRef.namespace + is not specified. If the namespace is specified, + then dataSourceRef will not be copied to dataSource.' + properties: + apiGroup: + description: APIGroup is the group for the resource + being referenced. If APIGroup is not specified, + the specified Kind must be in the core API + group. For any other third-party types, APIGroup + is required. + type: string + kind: + description: Kind is the type of resource being + referenced + type: string + name: + description: Name is the name of resource being + referenced + type: string + required: + - kind + - name + type: object + dataSourceRef: + description: 'dataSourceRef specifies the object + from which to populate the volume with data, if + a non-empty volume is desired. This may be any + object from a non-empty API group (non core object) + or a PersistentVolumeClaim object. When this field + is specified, volume binding will only succeed + if the type of the specified object matches some + installed volume populator or dynamic provisioner. + This field will replace the functionality of the + dataSource field and as such if both fields are + non-empty, they must have the same value. For + backwards compatibility, when namespace isn''t + specified in dataSourceRef, both fields (dataSource + and dataSourceRef) will be set to the same value + automatically if one of them is empty and the + other is non-empty. When namespace is specified + in dataSourceRef, dataSource isn''t set to the + same value and must be empty. There are three + important differences between dataSource and dataSourceRef: + * While dataSource only allows two specific types + of objects, dataSourceRef allows any non-core + object, as well as PersistentVolumeClaim objects. + * While dataSource ignores disallowed values (dropping + them), dataSourceRef preserves all values, and + generates an error if a disallowed value is specified. + * While dataSource only allows local objects, + dataSourceRef allows objects in any namespaces. + (Beta) Using this field requires the AnyVolumeDataSource + feature gate to be enabled. (Alpha) Using the + namespace field of dataSourceRef requires the + CrossNamespaceVolumeDataSource feature gate to + be enabled.' + properties: + apiGroup: + description: APIGroup is the group for the resource + being referenced. If APIGroup is not specified, + the specified Kind must be in the core API + group. For any other third-party types, APIGroup + is required. + type: string + kind: + description: Kind is the type of resource being + referenced + type: string + name: + description: Name is the name of resource being + referenced + type: string + namespace: + description: Namespace is the namespace of resource + being referenced Note that when a namespace + is specified, a gateway.networking.k8s.io/ReferenceGrant + object is required in the referent namespace + to allow that namespace's owner to accept + the reference. See the ReferenceGrant documentation + for details. (Alpha) This field requires the + CrossNamespaceVolumeDataSource feature gate + to be enabled. + type: string + required: + - kind + - name + type: object + resources: + description: 'resources represents the minimum resources + the volume should have. If RecoverVolumeExpansionFailure + feature is enabled users are allowed to specify + resource requirements that are lower than previous + value but must still be higher than capacity recorded + in the status field of the claim. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#resources' + properties: + limits: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + description: 'Limits describes the maximum amount + of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + type: object + requests: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + description: 'Requests describes the minimum + amount of compute resources required. If Requests + is omitted for a container, it defaults to + Limits if that is explicitly specified, otherwise + to an implementation-defined value. Requests + cannot exceed Limits. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + type: object + type: object + selector: + description: selector is a label query over volumes + to consider for binding. + properties: + matchExpressions: + description: matchExpressions is a list of label + selector requirements. The requirements are + ANDed. + items: + description: A label selector requirement + is a selector that contains values, a key, + and an operator that relates the key and + values. + properties: + key: + description: key is the label key that + the selector applies to. + type: string + operator: + description: operator represents a key's + relationship to a set of values. Valid + operators are In, NotIn, Exists and + DoesNotExist. + type: string + values: + description: values is an array of string + values. If the operator is In or NotIn, + the values array must be non-empty. + If the operator is Exists or DoesNotExist, + the values array must be empty. This + array is replaced during a strategic + merge patch. + items: + type: string + type: array + required: + - key + - operator + type: object + type: array + matchLabels: + additionalProperties: + type: string + description: matchLabels is a map of {key,value} + pairs. A single {key,value} in the matchLabels + map is equivalent to an element of matchExpressions, + whose key field is "key", the operator is + "In", and the values array contains only "value". + The requirements are ANDed. + type: object + type: object + storageClassName: + description: 'storageClassName is the name of the + StorageClass required by the claim. More info: + https://kubernetes.io/docs/concepts/storage/persistent-volumes#class-1' + type: string + volumeAttributesClassName: + description: 'volumeAttributesClassName may be used + to set the VolumeAttributesClass used by this + claim. If specified, the CSI driver will create + or update the volume with the attributes defined + in the corresponding VolumeAttributesClass. This + has a different purpose than storageClassName, + it can be changed after the claim is created. + An empty string value means that no VolumeAttributesClass + will be applied to the claim but it''s not allowed + to reset this field to empty string once it is + set. If unspecified and the PersistentVolumeClaim + is unbound, the default VolumeAttributesClass + will be set by the persistentvolume controller + if it exists. If the resource referred to by volumeAttributesClass + does not exist, this PersistentVolumeClaim will + be set to a Pending state, as reflected by the + modifyVolumeStatus field, until such as a resource + exists. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#volumeattributesclass + (Alpha) Using this field requires the VolumeAttributesClass + feature gate to be enabled.' + type: string + volumeMode: + description: volumeMode defines what type of volume + is required by the claim. Value of Filesystem + is implied when not included in claim spec. + type: string + volumeName: + description: volumeName is the binding reference + to the PersistentVolume backing this claim. + type: string + type: object + required: + - spec + type: object + type: object + fc: + description: fc represents a Fibre Channel resource that is + attached to a kubelet's host machine and then exposed to the + pod. + properties: + fsType: + description: 'fsType is the filesystem type to mount. Must + be a filesystem type supported by the host operating system. + Ex. "ext4", "xfs", "ntfs". Implicitly inferred to be "ext4" + if unspecified. TODO: how do we prevent errors in the + filesystem from compromising the machine' + type: string + lun: + description: 'lun is Optional: FC target lun number' + format: int32 + type: integer + readOnly: + description: 'readOnly is Optional: Defaults to false (read/write). + ReadOnly here will force the ReadOnly setting in VolumeMounts.' + type: boolean + targetWWNs: + description: 'targetWWNs is Optional: FC target worldwide + names (WWNs)' + items: + type: string + type: array + wwids: + description: 'wwids Optional: FC volume world wide identifiers + (wwids) Either wwids or combination of targetWWNs and + lun must be set, but not both simultaneously.' + items: + type: string + type: array + type: object + flexVolume: + description: flexVolume represents a generic volume resource + that is provisioned/attached using an exec based plugin. + properties: + driver: + description: driver is the name of the driver to use for + this volume. + type: string + fsType: + description: fsType is the filesystem type to mount. Must + be a filesystem type supported by the host operating system. + Ex. "ext4", "xfs", "ntfs". The default filesystem depends + on FlexVolume script. + type: string + options: + additionalProperties: + type: string + description: 'options is Optional: this field holds extra + command options if any.' + type: object + readOnly: + description: 'readOnly is Optional: defaults to false (read/write). + ReadOnly here will force the ReadOnly setting in VolumeMounts.' + type: boolean + secretRef: + description: 'secretRef is Optional: secretRef is reference + to the secret object containing sensitive information + to pass to the plugin scripts. This may be empty if no + secret object is specified. If the secret object contains + more than one secret, all secrets are passed to the plugin + scripts.' + properties: + name: + description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, uid?' + type: string + type: object + required: + - driver + type: object + flocker: + description: flocker represents a Flocker volume attached to + a kubelet's host machine. This depends on the Flocker control + service being running + properties: + datasetName: + description: datasetName is Name of the dataset stored as + metadata -> name on the dataset for Flocker should be + considered as deprecated + type: string + datasetUUID: + description: datasetUUID is the UUID of the dataset. This + is unique identifier of a Flocker dataset + type: string + type: object + gcePersistentDisk: + description: 'gcePersistentDisk represents a GCE Disk resource + that is attached to a kubelet''s host machine and then exposed + to the pod. More info: https://kubernetes.io/docs/concepts/storage/volumes#gcepersistentdisk' + properties: + fsType: + description: 'fsType is filesystem type of the volume that + you want to mount. Tip: Ensure that the filesystem type + is supported by the host operating system. Examples: "ext4", + "xfs", "ntfs". Implicitly inferred to be "ext4" if unspecified. + More info: https://kubernetes.io/docs/concepts/storage/volumes#gcepersistentdisk + TODO: how do we prevent errors in the filesystem from + compromising the machine' + type: string + partition: + description: 'partition is the partition in the volume that + you want to mount. If omitted, the default is to mount + by volume name. Examples: For volume /dev/sda1, you specify + the partition as "1". Similarly, the volume partition + for /dev/sda is "0" (or you can leave the property empty). + More info: https://kubernetes.io/docs/concepts/storage/volumes#gcepersistentdisk' + format: int32 + type: integer + pdName: + description: 'pdName is unique name of the PD resource in + GCE. Used to identify the disk in GCE. More info: https://kubernetes.io/docs/concepts/storage/volumes#gcepersistentdisk' + type: string + readOnly: + description: 'readOnly here will force the ReadOnly setting + in VolumeMounts. Defaults to false. More info: https://kubernetes.io/docs/concepts/storage/volumes#gcepersistentdisk' + type: boolean + required: + - pdName + type: object + gitRepo: + description: 'gitRepo represents a git repository at a particular + revision. DEPRECATED: GitRepo is deprecated. To provision + a container with a git repo, mount an EmptyDir into an InitContainer + that clones the repo using git, then mount the EmptyDir into + the Pod''s container.' + properties: + directory: + description: directory is the target directory name. Must + not contain or start with '..'. If '.' is supplied, the + volume directory will be the git repository. Otherwise, + if specified, the volume will contain the git repository + in the subdirectory with the given name. + type: string + repository: + description: repository is the URL + type: string + revision: + description: revision is the commit hash for the specified + revision. + type: string + required: + - repository + type: object + glusterfs: + description: 'glusterfs represents a Glusterfs mount on the + host that shares a pod''s lifetime. More info: https://examples.k8s.io/volumes/glusterfs/README.md' + properties: + endpoints: + description: 'endpoints is the endpoint name that details + Glusterfs topology. More info: https://examples.k8s.io/volumes/glusterfs/README.md#create-a-pod' + type: string + path: + description: 'path is the Glusterfs volume path. More info: + https://examples.k8s.io/volumes/glusterfs/README.md#create-a-pod' + type: string + readOnly: + description: 'readOnly here will force the Glusterfs volume + to be mounted with read-only permissions. Defaults to + false. More info: https://examples.k8s.io/volumes/glusterfs/README.md#create-a-pod' + type: boolean + required: + - endpoints + - path + type: object + hostPath: + description: 'hostPath represents a pre-existing file or directory + on the host machine that is directly exposed to the container. + This is generally used for system agents or other privileged + things that are allowed to see the host machine. Most containers + will NOT need this. More info: https://kubernetes.io/docs/concepts/storage/volumes#hostpath + --- TODO(jonesdl) We need to restrict who can use host directory + mounts and who can/can not mount host directories as read/write.' + properties: + path: + description: 'path of the directory on the host. If the + path is a symlink, it will follow the link to the real + path. More info: https://kubernetes.io/docs/concepts/storage/volumes#hostpath' + type: string + type: + description: 'type for HostPath Volume Defaults to "" More + info: https://kubernetes.io/docs/concepts/storage/volumes#hostpath' + type: string + required: + - path + type: object + iscsi: + description: 'iscsi represents an ISCSI Disk resource that is + attached to a kubelet''s host machine and then exposed to + the pod. More info: https://examples.k8s.io/volumes/iscsi/README.md' + properties: + chapAuthDiscovery: + description: chapAuthDiscovery defines whether support iSCSI + Discovery CHAP authentication + type: boolean + chapAuthSession: + description: chapAuthSession defines whether support iSCSI + Session CHAP authentication + type: boolean + fsType: + description: 'fsType is the filesystem type of the volume + that you want to mount. Tip: Ensure that the filesystem + type is supported by the host operating system. Examples: + "ext4", "xfs", "ntfs". Implicitly inferred to be "ext4" + if unspecified. More info: https://kubernetes.io/docs/concepts/storage/volumes#iscsi + TODO: how do we prevent errors in the filesystem from + compromising the machine' + type: string + initiatorName: + description: initiatorName is the custom iSCSI Initiator + Name. If initiatorName is specified with iscsiInterface + simultaneously, new iSCSI interface : will be created for the connection. + type: string + iqn: + description: iqn is the target iSCSI Qualified Name. + type: string + iscsiInterface: + description: iscsiInterface is the interface Name that uses + an iSCSI transport. Defaults to 'default' (tcp). + type: string + lun: + description: lun represents iSCSI Target Lun number. + format: int32 + type: integer + portals: + description: portals is the iSCSI Target Portal List. The + portal is either an IP or ip_addr:port if the port is + other than default (typically TCP ports 860 and 3260). + items: + type: string + type: array + readOnly: + description: readOnly here will force the ReadOnly setting + in VolumeMounts. Defaults to false. + type: boolean + secretRef: + description: secretRef is the CHAP Secret for iSCSI target + and initiator authentication + properties: + name: + description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, uid?' + type: string + type: object + targetPortal: + description: targetPortal is iSCSI Target Portal. The Portal + is either an IP or ip_addr:port if the port is other than + default (typically TCP ports 860 and 3260). + type: string + required: + - iqn + - lun + - targetPortal + type: object + name: + description: 'name of the volume. Must be a DNS_LABEL and unique + within the pod. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names' + type: string + nfs: + description: 'nfs represents an NFS mount on the host that shares + a pod''s lifetime More info: https://kubernetes.io/docs/concepts/storage/volumes#nfs' + properties: + path: + description: 'path that is exported by the NFS server. More + info: https://kubernetes.io/docs/concepts/storage/volumes#nfs' + type: string + readOnly: + description: 'readOnly here will force the NFS export to + be mounted with read-only permissions. Defaults to false. + More info: https://kubernetes.io/docs/concepts/storage/volumes#nfs' + type: boolean + server: + description: 'server is the hostname or IP address of the + NFS server. More info: https://kubernetes.io/docs/concepts/storage/volumes#nfs' + type: string + required: + - path + - server + type: object + persistentVolumeClaim: + description: 'persistentVolumeClaimVolumeSource represents a + reference to a PersistentVolumeClaim in the same namespace. + More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims' + properties: + claimName: + description: 'claimName is the name of a PersistentVolumeClaim + in the same namespace as the pod using this volume. More + info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims' + type: string + readOnly: + description: readOnly Will force the ReadOnly setting in + VolumeMounts. Default false. + type: boolean + required: + - claimName + type: object + photonPersistentDisk: + description: photonPersistentDisk represents a PhotonController + persistent disk attached and mounted on kubelets host machine + properties: + fsType: + description: fsType is the filesystem type to mount. Must + be a filesystem type supported by the host operating system. + Ex. "ext4", "xfs", "ntfs". Implicitly inferred to be "ext4" + if unspecified. + type: string + pdID: + description: pdID is the ID that identifies Photon Controller + persistent disk + type: string + required: + - pdID + type: object + portworxVolume: + description: portworxVolume represents a portworx volume attached + and mounted on kubelets host machine + properties: + fsType: + description: fSType represents the filesystem type to mount + Must be a filesystem type supported by the host operating + system. Ex. "ext4", "xfs". Implicitly inferred to be "ext4" + if unspecified. + type: string + readOnly: + description: readOnly defaults to false (read/write). ReadOnly + here will force the ReadOnly setting in VolumeMounts. + type: boolean + volumeID: + description: volumeID uniquely identifies a Portworx volume + type: string + required: + - volumeID + type: object + projected: + description: projected items for all in one resources secrets, + configmaps, and downward API + properties: + defaultMode: + description: defaultMode are the mode bits used to set permissions + on created files by default. Must be an octal value between + 0000 and 0777 or a decimal value between 0 and 511. YAML + accepts both octal and decimal values, JSON requires decimal + values for mode bits. Directories within the path are + not affected by this setting. This might be in conflict + with other options that affect the file mode, like fsGroup, + and the result can be other mode bits set. + format: int32 + type: integer + sources: + description: sources is the list of volume projections + items: + description: Projection that may be projected along with + other supported volume types + properties: + clusterTrustBundle: + description: "ClusterTrustBundle allows a pod to access + the `.spec.trustBundle` field of ClusterTrustBundle + objects in an auto-updating file. \n Alpha, gated + by the ClusterTrustBundleProjection feature gate. + \n ClusterTrustBundle objects can either be selected + by name, or by the combination of signer name and + a label selector. \n Kubelet performs aggressive + normalization of the PEM contents written into the + pod filesystem. Esoteric PEM features such as inter-block + comments and block headers are stripped. Certificates + are deduplicated. The ordering of certificates within + the file is arbitrary, and Kubelet may change the + order over time." + properties: + labelSelector: + description: Select all ClusterTrustBundles that + match this label selector. Only has effect + if signerName is set. Mutually-exclusive with + name. If unset, interpreted as "match nothing". If + set but empty, interpreted as "match everything". + properties: + matchExpressions: + description: matchExpressions is a list of + label selector requirements. The requirements + are ANDed. + items: + description: A label selector requirement + is a selector that contains values, a + key, and an operator that relates the + key and values. + properties: + key: + description: key is the label key that + the selector applies to. + type: string + operator: + description: operator represents a key's + relationship to a set of values. Valid + operators are In, NotIn, Exists and + DoesNotExist. + type: string + values: + description: values is an array of string + values. If the operator is In or NotIn, + the values array must be non-empty. + If the operator is Exists or DoesNotExist, + the values array must be empty. This + array is replaced during a strategic + merge patch. + items: + type: string + type: array + required: + - key + - operator + type: object + type: array + matchLabels: + additionalProperties: + type: string + description: matchLabels is a map of {key,value} + pairs. A single {key,value} in the matchLabels + map is equivalent to an element of matchExpressions, + whose key field is "key", the operator is + "In", and the values array contains only + "value". The requirements are ANDed. + type: object + type: object + name: + description: Select a single ClusterTrustBundle + by object name. Mutually-exclusive with signerName + and labelSelector. + type: string + optional: + description: If true, don't block pod startup + if the referenced ClusterTrustBundle(s) aren't + available. If using name, then the named ClusterTrustBundle + is allowed not to exist. If using signerName, + then the combination of signerName and labelSelector + is allowed to match zero ClusterTrustBundles. + type: boolean + path: + description: Relative path from the volume root + to write the bundle. + type: string + signerName: + description: Select all ClusterTrustBundles that + match this signer name. Mutually-exclusive with + name. The contents of all selected ClusterTrustBundles + will be unified and deduplicated. + type: string + required: + - path + type: object + configMap: + description: configMap information about the configMap + data to project + properties: + items: + description: items if unspecified, each key-value + pair in the Data field of the referenced ConfigMap + will be projected into the volume as a file + whose name is the key and content is the value. + If specified, the listed keys will be projected + into the specified paths, and unlisted keys + will not be present. If a key is specified which + is not present in the ConfigMap, the volume + setup will error unless it is marked optional. + Paths must be relative and may not contain the + '..' path or start with '..'. + items: + description: Maps a string key to a path within + a volume. + properties: + key: + description: key is the key to project. + type: string + mode: + description: 'mode is Optional: mode bits + used to set permissions on this file. + Must be an octal value between 0000 and + 0777 or a decimal value between 0 and + 511. YAML accepts both octal and decimal + values, JSON requires decimal values for + mode bits. If not specified, the volume + defaultMode will be used. This might be + in conflict with other options that affect + the file mode, like fsGroup, and the result + can be other mode bits set.' + format: int32 + type: integer + path: + description: path is the relative path of + the file to map the key to. May not be + an absolute path. May not contain the + path element '..'. May not start with + the string '..'. + type: string + required: + - key + - path + type: object + type: array + name: + description: 'Name of the referent. More info: + https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, + uid?' + type: string + optional: + description: optional specify whether the ConfigMap + or its keys must be defined + type: boolean + type: object + downwardAPI: + description: downwardAPI information about the downwardAPI + data to project + properties: + items: + description: Items is a list of DownwardAPIVolume + file + items: + description: DownwardAPIVolumeFile represents + information to create the file containing + the pod field + properties: + fieldRef: + description: 'Required: Selects a field + of the pod: only annotations, labels, + name and namespace are supported.' + properties: + apiVersion: + description: Version of the schema the + FieldPath is written in terms of, + defaults to "v1". + type: string + fieldPath: + description: Path of the field to select + in the specified API version. + type: string + required: + - fieldPath + type: object + mode: + description: 'Optional: mode bits used to + set permissions on this file, must be + an octal value between 0000 and 0777 or + a decimal value between 0 and 511. YAML + accepts both octal and decimal values, + JSON requires decimal values for mode + bits. If not specified, the volume defaultMode + will be used. This might be in conflict + with other options that affect the file + mode, like fsGroup, and the result can + be other mode bits set.' + format: int32 + type: integer + path: + description: 'Required: Path is the relative + path name of the file to be created. Must + not be absolute or contain the ''..'' + path. Must be utf-8 encoded. The first + item of the relative path must not start + with ''..''' + type: string + resourceFieldRef: + description: 'Selects a resource of the + container: only resources limits and requests + (limits.cpu, limits.memory, requests.cpu + and requests.memory) are currently supported.' + properties: + containerName: + description: 'Container name: required + for volumes, optional for env vars' + type: string + divisor: + anyOf: + - type: integer + - type: string + description: Specifies the output format + of the exposed resources, defaults + to "1" + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + resource: + description: 'Required: resource to + select' + type: string + required: + - resource + type: object + required: + - path + type: object + type: array + type: object + secret: + description: secret information about the secret data + to project + properties: + items: + description: items if unspecified, each key-value + pair in the Data field of the referenced Secret + will be projected into the volume as a file + whose name is the key and content is the value. + If specified, the listed keys will be projected + into the specified paths, and unlisted keys + will not be present. If a key is specified which + is not present in the Secret, the volume setup + will error unless it is marked optional. Paths + must be relative and may not contain the '..' + path or start with '..'. + items: + description: Maps a string key to a path within + a volume. + properties: + key: + description: key is the key to project. + type: string + mode: + description: 'mode is Optional: mode bits + used to set permissions on this file. + Must be an octal value between 0000 and + 0777 or a decimal value between 0 and + 511. YAML accepts both octal and decimal + values, JSON requires decimal values for + mode bits. If not specified, the volume + defaultMode will be used. This might be + in conflict with other options that affect + the file mode, like fsGroup, and the result + can be other mode bits set.' + format: int32 + type: integer + path: + description: path is the relative path of + the file to map the key to. May not be + an absolute path. May not contain the + path element '..'. May not start with + the string '..'. + type: string + required: + - key + - path + type: object + type: array + name: + description: 'Name of the referent. More info: + https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, + uid?' + type: string + optional: + description: optional field specify whether the + Secret or its key must be defined + type: boolean + type: object + serviceAccountToken: + description: serviceAccountToken is information about + the serviceAccountToken data to project + properties: + audience: + description: audience is the intended audience + of the token. A recipient of a token must identify + itself with an identifier specified in the audience + of the token, and otherwise should reject the + token. The audience defaults to the identifier + of the apiserver. + type: string + expirationSeconds: + description: expirationSeconds is the requested + duration of validity of the service account + token. As the token approaches expiration, the + kubelet volume plugin will proactively rotate + the service account token. The kubelet will + start trying to rotate the token if the token + is older than 80 percent of its time to live + or if the token is older than 24 hours.Defaults + to 1 hour and must be at least 10 minutes. + format: int64 + type: integer + path: + description: path is the path relative to the + mount point of the file to project the token + into. + type: string + required: + - path + type: object + type: object + type: array + type: object + quobyte: + description: quobyte represents a Quobyte mount on the host + that shares a pod's lifetime + properties: + group: + description: group to map volume access to Default is no + group + type: string + readOnly: + description: readOnly here will force the Quobyte volume + to be mounted with read-only permissions. Defaults to + false. + type: boolean + registry: + description: registry represents a single or multiple Quobyte + Registry services specified as a string as host:port pair + (multiple entries are separated with commas) which acts + as the central registry for volumes + type: string + tenant: + description: tenant owning the given Quobyte volume in the + Backend Used with dynamically provisioned Quobyte volumes, + value is set by the plugin + type: string + user: + description: user to map volume access to Defaults to serivceaccount + user + type: string + volume: + description: volume is a string that references an already + created Quobyte volume by name. + type: string + required: + - registry + - volume + type: object + rbd: + description: 'rbd represents a Rados Block Device mount on the + host that shares a pod''s lifetime. More info: https://examples.k8s.io/volumes/rbd/README.md' + properties: + fsType: + description: 'fsType is the filesystem type of the volume + that you want to mount. Tip: Ensure that the filesystem + type is supported by the host operating system. Examples: + "ext4", "xfs", "ntfs". Implicitly inferred to be "ext4" + if unspecified. More info: https://kubernetes.io/docs/concepts/storage/volumes#rbd + TODO: how do we prevent errors in the filesystem from + compromising the machine' + type: string + image: + description: 'image is the rados image name. More info: + https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it' + type: string + keyring: + description: 'keyring is the path to key ring for RBDUser. + Default is /etc/ceph/keyring. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it' + type: string + monitors: + description: 'monitors is a collection of Ceph monitors. + More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it' + items: + type: string + type: array + pool: + description: 'pool is the rados pool name. Default is rbd. + More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it' + type: string + readOnly: + description: 'readOnly here will force the ReadOnly setting + in VolumeMounts. Defaults to false. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it' + type: boolean + secretRef: + description: 'secretRef is name of the authentication secret + for RBDUser. If provided overrides keyring. Default is + nil. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it' + properties: + name: + description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, uid?' + type: string + type: object + user: + description: 'user is the rados user name. Default is admin. + More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it' + type: string + required: + - image + - monitors + type: object + scaleIO: + description: scaleIO represents a ScaleIO persistent volume + attached and mounted on Kubernetes nodes. + properties: + fsType: + description: fsType is the filesystem type to mount. Must + be a filesystem type supported by the host operating system. + Ex. "ext4", "xfs", "ntfs". Default is "xfs". + type: string + gateway: + description: gateway is the host address of the ScaleIO + API Gateway. + type: string + protectionDomain: + description: protectionDomain is the name of the ScaleIO + Protection Domain for the configured storage. + type: string + readOnly: + description: readOnly Defaults to false (read/write). ReadOnly + here will force the ReadOnly setting in VolumeMounts. + type: boolean + secretRef: + description: secretRef references to the secret for ScaleIO + user and other sensitive information. If this is not provided, + Login operation will fail. + properties: + name: + description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, uid?' + type: string + type: object + sslEnabled: + description: sslEnabled Flag enable/disable SSL communication + with Gateway, default false + type: boolean + storageMode: + description: storageMode indicates whether the storage for + a volume should be ThickProvisioned or ThinProvisioned. + Default is ThinProvisioned. + type: string + storagePool: + description: storagePool is the ScaleIO Storage Pool associated + with the protection domain. + type: string + system: + description: system is the name of the storage system as + configured in ScaleIO. + type: string + volumeName: + description: volumeName is the name of a volume already + created in the ScaleIO system that is associated with + this volume source. + type: string + required: + - gateway + - secretRef + - system + type: object + secret: + description: 'secret represents a secret that should populate + this volume. More info: https://kubernetes.io/docs/concepts/storage/volumes#secret' + properties: + defaultMode: + description: 'defaultMode is Optional: mode bits used to + set permissions on created files by default. Must be an + octal value between 0000 and 0777 or a decimal value between + 0 and 511. YAML accepts both octal and decimal values, + JSON requires decimal values for mode bits. Defaults to + 0644. Directories within the path are not affected by + this setting. This might be in conflict with other options + that affect the file mode, like fsGroup, and the result + can be other mode bits set.' + format: int32 + type: integer + items: + description: items If unspecified, each key-value pair in + the Data field of the referenced Secret will be projected + into the volume as a file whose name is the key and content + is the value. If specified, the listed keys will be projected + into the specified paths, and unlisted keys will not be + present. If a key is specified which is not present in + the Secret, the volume setup will error unless it is marked + optional. Paths must be relative and may not contain the + '..' path or start with '..'. + items: + description: Maps a string key to a path within a volume. + properties: + key: + description: key is the key to project. + type: string + mode: + description: 'mode is Optional: mode bits used to + set permissions on this file. Must be an octal value + between 0000 and 0777 or a decimal value between + 0 and 511. YAML accepts both octal and decimal values, + JSON requires decimal values for mode bits. If not + specified, the volume defaultMode will be used. + This might be in conflict with other options that + affect the file mode, like fsGroup, and the result + can be other mode bits set.' + format: int32 + type: integer + path: + description: path is the relative path of the file + to map the key to. May not be an absolute path. + May not contain the path element '..'. May not start + with the string '..'. + type: string + required: + - key + - path + type: object + type: array + optional: + description: optional field specify whether the Secret or + its keys must be defined + type: boolean + secretName: + description: 'secretName is the name of the secret in the + pod''s namespace to use. More info: https://kubernetes.io/docs/concepts/storage/volumes#secret' + type: string + type: object + storageos: + description: storageOS represents a StorageOS volume attached + and mounted on Kubernetes nodes. + properties: + fsType: + description: fsType is the filesystem type to mount. Must + be a filesystem type supported by the host operating system. + Ex. "ext4", "xfs", "ntfs". Implicitly inferred to be "ext4" + if unspecified. + type: string + readOnly: + description: readOnly defaults to false (read/write). ReadOnly + here will force the ReadOnly setting in VolumeMounts. + type: boolean + secretRef: + description: secretRef specifies the secret to use for obtaining + the StorageOS API credentials. If not specified, default + values will be attempted. + properties: + name: + description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, uid?' + type: string + type: object + volumeName: + description: volumeName is the human-readable name of the + StorageOS volume. Volume names are only unique within + a namespace. + type: string + volumeNamespace: + description: volumeNamespace specifies the scope of the + volume within StorageOS. If no namespace is specified + then the Pod's namespace will be used. This allows the + Kubernetes name scoping to be mirrored within StorageOS + for tighter integration. Set VolumeName to any name to + override the default behaviour. Set to "default" if you + are not using namespaces within StorageOS. Namespaces + that do not pre-exist within StorageOS will be created. + type: string + type: object + vsphereVolume: + description: vsphereVolume represents a vSphere volume attached + and mounted on kubelets host machine + properties: + fsType: + description: fsType is filesystem type to mount. Must be + a filesystem type supported by the host operating system. + Ex. "ext4", "xfs", "ntfs". Implicitly inferred to be "ext4" + if unspecified. + type: string + storagePolicyID: + description: storagePolicyID is the storage Policy Based + Management (SPBM) profile ID associated with the StoragePolicyName. + type: string + storagePolicyName: + description: storagePolicyName is the storage Policy Based + Management (SPBM) profile name. + type: string + volumePath: + description: volumePath is the path that identifies vSphere + volume vmdk + type: string + required: + - volumePath + type: object + required: + - name + type: object + type: array + sidecars: + items: + description: A single application container that you want to run + within a pod. + properties: + args: + description: 'Arguments to the entrypoint. The container image''s + CMD is used if this is not provided. Variable references $(VAR_NAME) + are expanded using the container''s environment. If a variable + cannot be resolved, the reference in the input string will + be unchanged. Double $$ are reduced to a single $, which allows + for escaping the $(VAR_NAME) syntax: i.e. "$$(VAR_NAME)" will + produce the string literal "$(VAR_NAME)". Escaped references + will never be expanded, regardless of whether the variable + exists or not. Cannot be updated. More info: https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell' + items: + type: string + type: array + command: + description: 'Entrypoint array. Not executed within a shell. + The container image''s ENTRYPOINT is used if this is not provided. + Variable references $(VAR_NAME) are expanded using the container''s + environment. If a variable cannot be resolved, the reference + in the input string will be unchanged. Double $$ are reduced + to a single $, which allows for escaping the $(VAR_NAME) syntax: + i.e. "$$(VAR_NAME)" will produce the string literal "$(VAR_NAME)". + Escaped references will never be expanded, regardless of whether + the variable exists or not. Cannot be updated. More info: + https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell' + items: + type: string + type: array + env: + description: List of environment variables to set in the container. + Cannot be updated. + items: + description: EnvVar represents an environment variable present + in a Container. + properties: + name: + description: Name of the environment variable. Must be + a C_IDENTIFIER. + type: string + value: + description: 'Variable references $(VAR_NAME) are expanded + using the previously defined environment variables in + the container and any service environment variables. + If a variable cannot be resolved, the reference in the + input string will be unchanged. Double $$ are reduced + to a single $, which allows for escaping the $(VAR_NAME) + syntax: i.e. "$$(VAR_NAME)" will produce the string + literal "$(VAR_NAME)". Escaped references will never + be expanded, regardless of whether the variable exists + or not. Defaults to "".' + type: string + valueFrom: + description: Source for the environment variable's value. + Cannot be used if value is not empty. + properties: + configMapKeyRef: + description: Selects a key of a ConfigMap. + properties: + key: + description: The key to select. + type: string + name: + description: 'Name of the referent. More info: + https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, + uid?' + type: string + optional: + description: Specify whether the ConfigMap or + its key must be defined + type: boolean + required: + - key + type: object + fieldRef: + description: 'Selects a field of the pod: supports + metadata.name, metadata.namespace, `metadata.labels['''']`, + `metadata.annotations['''']`, spec.nodeName, + spec.serviceAccountName, status.hostIP, status.podIP, + status.podIPs.' + properties: + apiVersion: + description: Version of the schema the FieldPath + is written in terms of, defaults to "v1". + type: string + fieldPath: + description: Path of the field to select in the + specified API version. + type: string + required: + - fieldPath + type: object + resourceFieldRef: + description: 'Selects a resource of the container: + only resources limits and requests (limits.cpu, + limits.memory, limits.ephemeral-storage, requests.cpu, + requests.memory and requests.ephemeral-storage) + are currently supported.' + properties: + containerName: + description: 'Container name: required for volumes, + optional for env vars' + type: string + divisor: + anyOf: + - type: integer + - type: string + description: Specifies the output format of the + exposed resources, defaults to "1" + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + resource: + description: 'Required: resource to select' + type: string + required: + - resource + type: object + secretKeyRef: + description: Selects a key of a secret in the pod's + namespace + properties: + key: + description: The key of the secret to select from. Must + be a valid secret key. + type: string + name: + description: 'Name of the referent. More info: + https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, + uid?' + type: string + optional: + description: Specify whether the Secret or its + key must be defined + type: boolean + required: + - key + type: object + type: object + required: + - name + type: object + type: array + envFrom: + description: List of sources to populate environment variables + in the container. The keys defined within a source must be + a C_IDENTIFIER. All invalid keys will be reported as an event + when the container is starting. When a key exists in multiple + sources, the value associated with the last source will take + precedence. Values defined by an Env with a duplicate key + will take precedence. Cannot be updated. + items: + description: EnvFromSource represents the source of a set + of ConfigMaps + properties: + configMapRef: + description: The ConfigMap to select from + properties: + name: + description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, + uid?' + type: string + optional: + description: Specify whether the ConfigMap must be + defined + type: boolean + type: object + prefix: + description: An optional identifier to prepend to each + key in the ConfigMap. Must be a C_IDENTIFIER. + type: string + secretRef: + description: The Secret to select from + properties: + name: + description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, + uid?' + type: string + optional: + description: Specify whether the Secret must be defined + type: boolean + type: object + type: object + type: array + image: + description: 'Container image name. More info: https://kubernetes.io/docs/concepts/containers/images + This field is optional to allow higher level config management + to default or override container images in workload controllers + like Deployments and StatefulSets.' + type: string + imagePullPolicy: + description: 'Image pull policy. One of Always, Never, IfNotPresent. + Defaults to Always if :latest tag is specified, or IfNotPresent + otherwise. Cannot be updated. More info: https://kubernetes.io/docs/concepts/containers/images#updating-images' + type: string + lifecycle: + description: Actions that the management system should take + in response to container lifecycle events. Cannot be updated. + properties: + postStart: + description: 'PostStart is called immediately after a container + is created. If the handler fails, the container is terminated + and restarted according to its restart policy. Other management + of the container blocks until the hook completes. More + info: https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/#container-hooks' + properties: + exec: + description: Exec specifies the action to take. + properties: + command: + description: Command is the command line to execute + inside the container, the working directory for + the command is root ('/') in the container's + filesystem. The command is simply exec'd, it is + not run inside a shell, so traditional shell instructions + ('|', etc) won't work. To use a shell, you need + to explicitly call out to that shell. Exit status + of 0 is treated as live/healthy and non-zero is + unhealthy. + items: + type: string + type: array + type: object + httpGet: + description: HTTPGet specifies the http request to perform. + properties: + host: + description: Host name to connect to, defaults to + the pod IP. You probably want to set "Host" in + httpHeaders instead. + type: string + httpHeaders: + description: Custom headers to set in the request. + HTTP allows repeated headers. + items: + description: HTTPHeader describes a custom header + to be used in HTTP probes + properties: + name: + description: The header field name. This will + be canonicalized upon output, so case-variant + names will be understood as the same header. + type: string + value: + description: The header field value + type: string + required: + - name + - value + type: object + type: array + path: + description: Path to access on the HTTP server. + type: string + port: + anyOf: + - type: integer + - type: string + description: Name or number of the port to access + on the container. Number must be in the range + 1 to 65535. Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + scheme: + description: Scheme to use for connecting to the + host. Defaults to HTTP. + type: string + required: + - port + type: object + sleep: + description: Sleep represents the duration that the + container should sleep before being terminated. + properties: + seconds: + description: Seconds is the number of seconds to + sleep. + format: int64 + type: integer + required: + - seconds + type: object + tcpSocket: + description: Deprecated. TCPSocket is NOT supported + as a LifecycleHandler and kept for the backward compatibility. + There are no validation of this field and lifecycle + hooks will fail in runtime when tcp handler is specified. + properties: + host: + description: 'Optional: Host name to connect to, + defaults to the pod IP.' + type: string + port: + anyOf: + - type: integer + - type: string + description: Number or name of the port to access + on the container. Number must be in the range + 1 to 65535. Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + required: + - port + type: object + type: object + preStop: + description: 'PreStop is called immediately before a container + is terminated due to an API request or management event + such as liveness/startup probe failure, preemption, resource + contention, etc. The handler is not called if the container + crashes or exits. The Pod''s termination grace period + countdown begins before the PreStop hook is executed. + Regardless of the outcome of the handler, the container + will eventually terminate within the Pod''s termination + grace period (unless delayed by finalizers). Other management + of the container blocks until the hook completes or until + the termination grace period is reached. More info: https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/#container-hooks' + properties: + exec: + description: Exec specifies the action to take. + properties: + command: + description: Command is the command line to execute + inside the container, the working directory for + the command is root ('/') in the container's + filesystem. The command is simply exec'd, it is + not run inside a shell, so traditional shell instructions + ('|', etc) won't work. To use a shell, you need + to explicitly call out to that shell. Exit status + of 0 is treated as live/healthy and non-zero is + unhealthy. + items: + type: string + type: array + type: object + httpGet: + description: HTTPGet specifies the http request to perform. + properties: + host: + description: Host name to connect to, defaults to + the pod IP. You probably want to set "Host" in + httpHeaders instead. + type: string + httpHeaders: + description: Custom headers to set in the request. + HTTP allows repeated headers. + items: + description: HTTPHeader describes a custom header + to be used in HTTP probes + properties: + name: + description: The header field name. This will + be canonicalized upon output, so case-variant + names will be understood as the same header. + type: string + value: + description: The header field value + type: string + required: + - name + - value + type: object + type: array + path: + description: Path to access on the HTTP server. + type: string + port: + anyOf: + - type: integer + - type: string + description: Name or number of the port to access + on the container. Number must be in the range + 1 to 65535. Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + scheme: + description: Scheme to use for connecting to the + host. Defaults to HTTP. + type: string + required: + - port + type: object + sleep: + description: Sleep represents the duration that the + container should sleep before being terminated. + properties: + seconds: + description: Seconds is the number of seconds to + sleep. + format: int64 + type: integer + required: + - seconds + type: object + tcpSocket: + description: Deprecated. TCPSocket is NOT supported + as a LifecycleHandler and kept for the backward compatibility. + There are no validation of this field and lifecycle + hooks will fail in runtime when tcp handler is specified. + properties: + host: + description: 'Optional: Host name to connect to, + defaults to the pod IP.' + type: string + port: + anyOf: + - type: integer + - type: string + description: Number or name of the port to access + on the container. Number must be in the range + 1 to 65535. Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + required: + - port + type: object + type: object + type: object + livenessProbe: + description: 'Periodic probe of container liveness. Container + will be restarted if the probe fails. Cannot be updated. More + info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes' + properties: + exec: + description: Exec specifies the action to take. + properties: + command: + description: Command is the command line to execute + inside the container, the working directory for the + command is root ('/') in the container's filesystem. + The command is simply exec'd, it is not run inside + a shell, so traditional shell instructions ('|', etc) + won't work. To use a shell, you need to explicitly + call out to that shell. Exit status of 0 is treated + as live/healthy and non-zero is unhealthy. + items: + type: string + type: array + type: object + failureThreshold: + description: Minimum consecutive failures for the probe + to be considered failed after having succeeded. Defaults + to 3. Minimum value is 1. + format: int32 + type: integer + grpc: + description: GRPC specifies an action involving a GRPC port. + properties: + port: + description: Port number of the gRPC service. Number + must be in the range 1 to 65535. + format: int32 + type: integer + service: + description: "Service is the name of the service to + place in the gRPC HealthCheckRequest (see https://github.com/grpc/grpc/blob/master/doc/health-checking.md). + \n If this is not specified, the default behavior + is defined by gRPC." + type: string + required: + - port + type: object + httpGet: + description: HTTPGet specifies the http request to perform. + properties: + host: + description: Host name to connect to, defaults to the + pod IP. You probably want to set "Host" in httpHeaders + instead. + type: string + httpHeaders: + description: Custom headers to set in the request. HTTP + allows repeated headers. + items: + description: HTTPHeader describes a custom header + to be used in HTTP probes + properties: + name: + description: The header field name. This will + be canonicalized upon output, so case-variant + names will be understood as the same header. + type: string + value: + description: The header field value + type: string + required: + - name + - value + type: object + type: array + path: + description: Path to access on the HTTP server. + type: string + port: + anyOf: + - type: integer + - type: string + description: Name or number of the port to access on + the container. Number must be in the range 1 to 65535. + Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + scheme: + description: Scheme to use for connecting to the host. + Defaults to HTTP. + type: string + required: + - port + type: object + initialDelaySeconds: + description: 'Number of seconds after the container has + started before liveness probes are initiated. More info: + https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes' + format: int32 + type: integer + periodSeconds: + description: How often (in seconds) to perform the probe. + Default to 10 seconds. Minimum value is 1. + format: int32 + type: integer + successThreshold: + description: Minimum consecutive successes for the probe + to be considered successful after having failed. Defaults + to 1. Must be 1 for liveness and startup. Minimum value + is 1. + format: int32 + type: integer + tcpSocket: + description: TCPSocket specifies an action involving a TCP + port. + properties: + host: + description: 'Optional: Host name to connect to, defaults + to the pod IP.' + type: string + port: + anyOf: + - type: integer + - type: string + description: Number or name of the port to access on + the container. Number must be in the range 1 to 65535. + Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + required: + - port + type: object + terminationGracePeriodSeconds: + description: Optional duration in seconds the pod needs + to terminate gracefully upon probe failure. The grace + period is the duration in seconds after the processes + running in the pod are sent a termination signal and the + time when the processes are forcibly halted with a kill + signal. Set this value longer than the expected cleanup + time for your process. If this value is nil, the pod's + terminationGracePeriodSeconds will be used. Otherwise, + this value overrides the value provided by the pod spec. + Value must be non-negative integer. The value zero indicates + stop immediately via the kill signal (no opportunity to + shut down). This is a beta field and requires enabling + ProbeTerminationGracePeriod feature gate. Minimum value + is 1. spec.terminationGracePeriodSeconds is used if unset. + format: int64 + type: integer + timeoutSeconds: + description: 'Number of seconds after which the probe times + out. Defaults to 1 second. Minimum value is 1. More info: + https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes' + format: int32 + type: integer + type: object + name: + description: Name of the container specified as a DNS_LABEL. + Each container in a pod must have a unique name (DNS_LABEL). + Cannot be updated. + type: string + ports: + description: List of ports to expose from the container. Not + specifying a port here DOES NOT prevent that port from being + exposed. Any port which is listening on the default "0.0.0.0" + address inside a container will be accessible from the network. + Modifying this array with strategic merge patch may corrupt + the data. For more information See https://github.com/kubernetes/kubernetes/issues/108255. + Cannot be updated. + items: + description: ContainerPort represents a network port in a + single container. + properties: + containerPort: + description: Number of port to expose on the pod's IP + address. This must be a valid port number, 0 < x < 65536. + format: int32 + type: integer + hostIP: + description: What host IP to bind the external port to. + type: string + hostPort: + description: Number of port to expose on the host. If + specified, this must be a valid port number, 0 < x < + 65536. If HostNetwork is specified, this must match + ContainerPort. Most containers do not need this. + format: int32 + type: integer + name: + description: If specified, this must be an IANA_SVC_NAME + and unique within the pod. Each named port in a pod + must have a unique name. Name for the port that can + be referred to by services. + type: string + protocol: + default: TCP + description: Protocol for port. Must be UDP, TCP, or SCTP. + Defaults to "TCP". + type: string + required: + - containerPort + type: object + type: array + x-kubernetes-list-map-keys: + - containerPort + - protocol + x-kubernetes-list-type: map + readinessProbe: + description: 'Periodic probe of container service readiness. + Container will be removed from service endpoints if the probe + fails. Cannot be updated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes' + properties: + exec: + description: Exec specifies the action to take. + properties: + command: + description: Command is the command line to execute + inside the container, the working directory for the + command is root ('/') in the container's filesystem. + The command is simply exec'd, it is not run inside + a shell, so traditional shell instructions ('|', etc) + won't work. To use a shell, you need to explicitly + call out to that shell. Exit status of 0 is treated + as live/healthy and non-zero is unhealthy. + items: + type: string + type: array + type: object + failureThreshold: + description: Minimum consecutive failures for the probe + to be considered failed after having succeeded. Defaults + to 3. Minimum value is 1. + format: int32 + type: integer + grpc: + description: GRPC specifies an action involving a GRPC port. + properties: + port: + description: Port number of the gRPC service. Number + must be in the range 1 to 65535. + format: int32 + type: integer + service: + description: "Service is the name of the service to + place in the gRPC HealthCheckRequest (see https://github.com/grpc/grpc/blob/master/doc/health-checking.md). + \n If this is not specified, the default behavior + is defined by gRPC." + type: string + required: + - port + type: object + httpGet: + description: HTTPGet specifies the http request to perform. + properties: + host: + description: Host name to connect to, defaults to the + pod IP. You probably want to set "Host" in httpHeaders + instead. + type: string + httpHeaders: + description: Custom headers to set in the request. HTTP + allows repeated headers. + items: + description: HTTPHeader describes a custom header + to be used in HTTP probes + properties: + name: + description: The header field name. This will + be canonicalized upon output, so case-variant + names will be understood as the same header. + type: string + value: + description: The header field value + type: string + required: + - name + - value + type: object + type: array + path: + description: Path to access on the HTTP server. + type: string + port: + anyOf: + - type: integer + - type: string + description: Name or number of the port to access on + the container. Number must be in the range 1 to 65535. + Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + scheme: + description: Scheme to use for connecting to the host. + Defaults to HTTP. + type: string + required: + - port + type: object + initialDelaySeconds: + description: 'Number of seconds after the container has + started before liveness probes are initiated. More info: + https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes' + format: int32 + type: integer + periodSeconds: + description: How often (in seconds) to perform the probe. + Default to 10 seconds. Minimum value is 1. + format: int32 + type: integer + successThreshold: + description: Minimum consecutive successes for the probe + to be considered successful after having failed. Defaults + to 1. Must be 1 for liveness and startup. Minimum value + is 1. + format: int32 + type: integer + tcpSocket: + description: TCPSocket specifies an action involving a TCP + port. + properties: + host: + description: 'Optional: Host name to connect to, defaults + to the pod IP.' + type: string + port: + anyOf: + - type: integer + - type: string + description: Number or name of the port to access on + the container. Number must be in the range 1 to 65535. + Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + required: + - port + type: object + terminationGracePeriodSeconds: + description: Optional duration in seconds the pod needs + to terminate gracefully upon probe failure. The grace + period is the duration in seconds after the processes + running in the pod are sent a termination signal and the + time when the processes are forcibly halted with a kill + signal. Set this value longer than the expected cleanup + time for your process. If this value is nil, the pod's + terminationGracePeriodSeconds will be used. Otherwise, + this value overrides the value provided by the pod spec. + Value must be non-negative integer. The value zero indicates + stop immediately via the kill signal (no opportunity to + shut down). This is a beta field and requires enabling + ProbeTerminationGracePeriod feature gate. Minimum value + is 1. spec.terminationGracePeriodSeconds is used if unset. + format: int64 + type: integer + timeoutSeconds: + description: 'Number of seconds after which the probe times + out. Defaults to 1 second. Minimum value is 1. More info: + https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes' + format: int32 + type: integer + type: object + resizePolicy: + description: Resources resize policy for the container. + items: + description: ContainerResizePolicy represents resource resize + policy for the container. + properties: + resourceName: + description: 'Name of the resource to which this resource + resize policy applies. Supported values: cpu, memory.' + type: string + restartPolicy: + description: Restart policy to apply when specified resource + is resized. If not specified, it defaults to NotRequired. + type: string + required: + - resourceName + - restartPolicy + type: object + type: array + x-kubernetes-list-type: atomic + resources: + description: 'Compute Resources required by this container. + Cannot be updated. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + properties: + claims: + description: "Claims lists the names of resources, defined + in spec.resourceClaims, that are used by this container. + \n This is an alpha field and requires enabling the DynamicResourceAllocation + feature gate. \n This field is immutable. It can only + be set for containers." + items: + description: ResourceClaim references one entry in PodSpec.ResourceClaims. + properties: + name: + description: Name must match the name of one entry + in pod.spec.resourceClaims of the Pod where this + field is used. It makes that resource available + inside a container. + type: string + required: + - name + type: object + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + limits: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + description: 'Limits describes the maximum amount of compute + resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + type: object + requests: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + description: 'Requests describes the minimum amount of compute + resources required. If Requests is omitted for a container, + it defaults to Limits if that is explicitly specified, + otherwise to an implementation-defined value. Requests + cannot exceed Limits. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + type: object + type: object + restartPolicy: + description: 'RestartPolicy defines the restart behavior of + individual containers in a pod. This field may only be set + for init containers, and the only allowed value is "Always". + For non-init containers or when this field is not specified, + the restart behavior is defined by the Pod''s restart policy + and the container type. Setting the RestartPolicy as "Always" + for the init container will have the following effect: this + init container will be continually restarted on exit until + all regular containers have terminated. Once all regular containers + have completed, all init containers with restartPolicy "Always" + will be shut down. This lifecycle differs from normal init + containers and is often referred to as a "sidecar" container. + Although this init container still starts in the init container + sequence, it does not wait for the container to complete before + proceeding to the next init container. Instead, the next init + container starts immediately after this init container is + started, or after any startupProbe has successfully completed.' + type: string + securityContext: + description: 'SecurityContext defines the security options the + container should be run with. If set, the fields of SecurityContext + override the equivalent fields of PodSecurityContext. More + info: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/' + properties: + allowPrivilegeEscalation: + description: 'AllowPrivilegeEscalation controls whether + a process can gain more privileges than its parent process. + This bool directly controls if the no_new_privs flag will + be set on the container process. AllowPrivilegeEscalation + is true always when the container is: 1) run as Privileged + 2) has CAP_SYS_ADMIN Note that this field cannot be set + when spec.os.name is windows.' + type: boolean + capabilities: + description: The capabilities to add/drop when running containers. + Defaults to the default set of capabilities granted by + the container runtime. Note that this field cannot be + set when spec.os.name is windows. + properties: + add: + description: Added capabilities + items: + description: Capability represent POSIX capabilities + type + type: string + type: array + drop: + description: Removed capabilities + items: + description: Capability represent POSIX capabilities + type + type: string + type: array + type: object + privileged: + description: Run container in privileged mode. Processes + in privileged containers are essentially equivalent to + root on the host. Defaults to false. Note that this field + cannot be set when spec.os.name is windows. + type: boolean + procMount: + description: procMount denotes the type of proc mount to + use for the containers. The default is DefaultProcMount + which uses the container runtime defaults for readonly + paths and masked paths. This requires the ProcMountType + feature flag to be enabled. Note that this field cannot + be set when spec.os.name is windows. + type: string + readOnlyRootFilesystem: + description: Whether this container has a read-only root + filesystem. Default is false. Note that this field cannot + be set when spec.os.name is windows. + type: boolean + runAsGroup: + description: The GID to run the entrypoint of the container + process. Uses runtime default if unset. May also be set + in PodSecurityContext. If set in both SecurityContext + and PodSecurityContext, the value specified in SecurityContext + takes precedence. Note that this field cannot be set when + spec.os.name is windows. + format: int64 + type: integer + runAsNonRoot: + description: Indicates that the container must run as a + non-root user. If true, the Kubelet will validate the + image at runtime to ensure that it does not run as UID + 0 (root) and fail to start the container if it does. If + unset or false, no such validation will be performed. + May also be set in PodSecurityContext. If set in both + SecurityContext and PodSecurityContext, the value specified + in SecurityContext takes precedence. + type: boolean + runAsUser: + description: The UID to run the entrypoint of the container + process. Defaults to user specified in image metadata + if unspecified. May also be set in PodSecurityContext. If + set in both SecurityContext and PodSecurityContext, the + value specified in SecurityContext takes precedence. Note + that this field cannot be set when spec.os.name is windows. + format: int64 + type: integer + seLinuxOptions: + description: The SELinux context to be applied to the container. + If unspecified, the container runtime will allocate a + random SELinux context for each container. May also be + set in PodSecurityContext. If set in both SecurityContext + and PodSecurityContext, the value specified in SecurityContext + takes precedence. Note that this field cannot be set when + spec.os.name is windows. + properties: + level: + description: Level is SELinux level label that applies + to the container. + type: string + role: + description: Role is a SELinux role label that applies + to the container. + type: string + type: + description: Type is a SELinux type label that applies + to the container. + type: string + user: + description: User is a SELinux user label that applies + to the container. + type: string + type: object + seccompProfile: + description: The seccomp options to use by this container. + If seccomp options are provided at both the pod & container + level, the container options override the pod options. + Note that this field cannot be set when spec.os.name is + windows. + properties: + localhostProfile: + description: localhostProfile indicates a profile defined + in a file on the node should be used. The profile + must be preconfigured on the node to work. Must be + a descending path, relative to the kubelet's configured + seccomp profile location. Must be set if type is "Localhost". + Must NOT be set for any other type. + type: string + type: + description: "type indicates which kind of seccomp profile + will be applied. Valid options are: \n Localhost - + a profile defined in a file on the node should be + used. RuntimeDefault - the container runtime default + profile should be used. Unconfined - no profile should + be applied." + type: string + required: + - type + type: object + windowsOptions: + description: The Windows specific settings applied to all + containers. If unspecified, the options from the PodSecurityContext + will be used. If set in both SecurityContext and PodSecurityContext, + the value specified in SecurityContext takes precedence. + Note that this field cannot be set when spec.os.name is + linux. + properties: + gmsaCredentialSpec: + description: GMSACredentialSpec is where the GMSA admission + webhook (https://github.com/kubernetes-sigs/windows-gmsa) + inlines the contents of the GMSA credential spec named + by the GMSACredentialSpecName field. + type: string + gmsaCredentialSpecName: + description: GMSACredentialSpecName is the name of the + GMSA credential spec to use. + type: string + hostProcess: + description: HostProcess determines if a container should + be run as a 'Host Process' container. All of a Pod's + containers must have the same effective HostProcess + value (it is not allowed to have a mix of HostProcess + containers and non-HostProcess containers). In addition, + if HostProcess is true then HostNetwork must also + be set to true. + type: boolean + runAsUserName: + description: The UserName in Windows to run the entrypoint + of the container process. Defaults to the user specified + in image metadata if unspecified. May also be set + in PodSecurityContext. If set in both SecurityContext + and PodSecurityContext, the value specified in SecurityContext + takes precedence. + type: string + type: object + type: object + startupProbe: + description: 'StartupProbe indicates that the Pod has successfully + initialized. If specified, no other probes are executed until + this completes successfully. If this probe fails, the Pod + will be restarted, just as if the livenessProbe failed. This + can be used to provide different probe parameters at the beginning + of a Pod''s lifecycle, when it might take a long time to load + data or warm a cache, than during steady-state operation. + This cannot be updated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes' + properties: + exec: + description: Exec specifies the action to take. + properties: + command: + description: Command is the command line to execute + inside the container, the working directory for the + command is root ('/') in the container's filesystem. + The command is simply exec'd, it is not run inside + a shell, so traditional shell instructions ('|', etc) + won't work. To use a shell, you need to explicitly + call out to that shell. Exit status of 0 is treated + as live/healthy and non-zero is unhealthy. + items: + type: string + type: array + type: object + failureThreshold: + description: Minimum consecutive failures for the probe + to be considered failed after having succeeded. Defaults + to 3. Minimum value is 1. + format: int32 + type: integer + grpc: + description: GRPC specifies an action involving a GRPC port. + properties: + port: + description: Port number of the gRPC service. Number + must be in the range 1 to 65535. + format: int32 + type: integer + service: + description: "Service is the name of the service to + place in the gRPC HealthCheckRequest (see https://github.com/grpc/grpc/blob/master/doc/health-checking.md). + \n If this is not specified, the default behavior + is defined by gRPC." + type: string + required: + - port + type: object + httpGet: + description: HTTPGet specifies the http request to perform. + properties: + host: + description: Host name to connect to, defaults to the + pod IP. You probably want to set "Host" in httpHeaders + instead. + type: string + httpHeaders: + description: Custom headers to set in the request. HTTP + allows repeated headers. + items: + description: HTTPHeader describes a custom header + to be used in HTTP probes + properties: + name: + description: The header field name. This will + be canonicalized upon output, so case-variant + names will be understood as the same header. + type: string + value: + description: The header field value + type: string + required: + - name + - value + type: object + type: array + path: + description: Path to access on the HTTP server. + type: string + port: + anyOf: + - type: integer + - type: string + description: Name or number of the port to access on + the container. Number must be in the range 1 to 65535. + Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + scheme: + description: Scheme to use for connecting to the host. + Defaults to HTTP. + type: string + required: + - port + type: object + initialDelaySeconds: + description: 'Number of seconds after the container has + started before liveness probes are initiated. More info: + https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes' + format: int32 + type: integer + periodSeconds: + description: How often (in seconds) to perform the probe. + Default to 10 seconds. Minimum value is 1. + format: int32 + type: integer + successThreshold: + description: Minimum consecutive successes for the probe + to be considered successful after having failed. Defaults + to 1. Must be 1 for liveness and startup. Minimum value + is 1. + format: int32 + type: integer + tcpSocket: + description: TCPSocket specifies an action involving a TCP + port. + properties: + host: + description: 'Optional: Host name to connect to, defaults + to the pod IP.' + type: string + port: + anyOf: + - type: integer + - type: string + description: Number or name of the port to access on + the container. Number must be in the range 1 to 65535. + Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + required: + - port + type: object + terminationGracePeriodSeconds: + description: Optional duration in seconds the pod needs + to terminate gracefully upon probe failure. The grace + period is the duration in seconds after the processes + running in the pod are sent a termination signal and the + time when the processes are forcibly halted with a kill + signal. Set this value longer than the expected cleanup + time for your process. If this value is nil, the pod's + terminationGracePeriodSeconds will be used. Otherwise, + this value overrides the value provided by the pod spec. + Value must be non-negative integer. The value zero indicates + stop immediately via the kill signal (no opportunity to + shut down). This is a beta field and requires enabling + ProbeTerminationGracePeriod feature gate. Minimum value + is 1. spec.terminationGracePeriodSeconds is used if unset. + format: int64 + type: integer + timeoutSeconds: + description: 'Number of seconds after which the probe times + out. Defaults to 1 second. Minimum value is 1. More info: + https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes' + format: int32 + type: integer + type: object + stdin: + description: Whether this container should allocate a buffer + for stdin in the container runtime. If this is not set, reads + from stdin in the container will always result in EOF. Default + is false. + type: boolean + stdinOnce: + description: Whether the container runtime should close the + stdin channel after it has been opened by a single attach. + When stdin is true the stdin stream will remain open across + multiple attach sessions. If stdinOnce is set to true, stdin + is opened on container start, is empty until the first client + attaches to stdin, and then remains open and accepts data + until the client disconnects, at which time stdin is closed + and remains closed until the container is restarted. If this + flag is false, a container processes that reads from stdin + will never receive an EOF. Default is false + type: boolean + terminationMessagePath: + description: 'Optional: Path at which the file to which the + container''s termination message will be written is mounted + into the container''s filesystem. Message written is intended + to be brief final status, such as an assertion failure message. + Will be truncated by the node if greater than 4096 bytes. + The total message length across all containers will be limited + to 12kb. Defaults to /dev/termination-log. Cannot be updated.' + type: string + terminationMessagePolicy: + description: Indicate how the termination message should be + populated. File will use the contents of terminationMessagePath + to populate the container status message on both success and + failure. FallbackToLogsOnError will use the last chunk of + container log output if the termination message file is empty + and the container exited with an error. The log output is + limited to 2048 bytes or 80 lines, whichever is smaller. Defaults + to File. Cannot be updated. + type: string + tty: + description: Whether this container should allocate a TTY for + itself, also requires 'stdin' to be true. Default is false. + type: boolean + volumeDevices: + description: volumeDevices is the list of block devices to be + used by the container. + items: + description: volumeDevice describes a mapping of a raw block + device within a container. + properties: + devicePath: + description: devicePath is the path inside of the container + that the device will be mapped to. + type: string + name: + description: name must match the name of a persistentVolumeClaim + in the pod + type: string + required: + - devicePath + - name + type: object + type: array + volumeMounts: + description: Pod volumes to mount into the container's filesystem. + Cannot be updated. + items: + description: VolumeMount describes a mounting of a Volume + within a container. + properties: + mountPath: + description: Path within the container at which the volume + should be mounted. Must not contain ':'. + type: string + mountPropagation: + description: mountPropagation determines how mounts are + propagated from the host to container and the other + way around. When not set, MountPropagationNone is used. + This field is beta in 1.10. + type: string + name: + description: This must match the Name of a Volume. + type: string + readOnly: + description: Mounted read-only if true, read-write otherwise + (false or unspecified). Defaults to false. + type: boolean + subPath: + description: Path within the volume from which the container's + volume should be mounted. Defaults to "" (volume's root). + type: string + subPathExpr: + description: Expanded path within the volume from which + the container's volume should be mounted. Behaves similarly + to SubPath but environment variable references $(VAR_NAME) + are expanded using the container's environment. Defaults + to "" (volume's root). SubPathExpr and SubPath are mutually + exclusive. + type: string + required: + - mountPath + - name + type: object + type: array + workingDir: + description: Container's working directory. If not specified, + the container runtime's default will be used, which might + be configured in the container image. Cannot be updated. + type: string + required: + - name + type: object + type: array + type: object + status: + description: DatabaseObserverStatus defines the observed state of DatabaseObserver + properties: + conditions: + items: + description: "Condition contains details for one aspect of the current + state of this API Resource. --- This struct is intended for direct + use as an array at the field path .status.conditions. For example, + \n \ttype FooStatus struct{ \t // Represents the observations + of a foo's current state. \t // Known .status.conditions.type + are: \"Available\", \"Progressing\", and \"Degraded\" \t // + +patchMergeKey=type \t // +patchStrategy=merge \t // +listType=map + \t // +listMapKey=type \t Conditions []metav1.Condition + `json:\"conditions,omitempty\" patchStrategy:\"merge\" patchMergeKey:\"type\" + protobuf:\"bytes,1,rep,name=conditions\"` \n \t // other fields + \t}" + properties: + lastTransitionTime: + description: lastTransitionTime is the last time the condition + transitioned from one status to another. This should be when + the underlying condition changed. If that is not known, then + using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: message is a human readable message indicating + details about the transition. This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: observedGeneration represents the .metadata.generation + that the condition was set based upon. For instance, if .metadata.generation + is currently 12, but the .status.conditions[x].observedGeneration + is 9, the condition is out of date with respect to the current + state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: reason contains a programmatic identifier indicating + the reason for the condition's last transition. Producers + of specific condition types may define expected values and + meanings for this field, and whether the values are considered + a guaranteed API. The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + --- Many .condition.type values are consistent across resources + like Available, but because arbitrary conditions can be useful + (see .node.status.conditions), the ability to deconflict is + important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + exporterConfig: + type: string + replicas: + type: integer + status: + type: string + version: + type: string + required: + - conditions + - exporterConfig + - version + type: object + type: object + served: true + storage: false + subresources: + status: {} + - additionalPrinterColumns: + - jsonPath: .status.exporterConfig + name: ExporterConfig + type: string + - jsonPath: .status.status + name: Status + type: string + - jsonPath: .status.version + name: Version + type: string + name: v4 + schema: + openAPIV3Schema: + description: DatabaseObserver is the Schema for the databaseobservers API + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + description: DatabaseObserverSpec defines the desired state of DatabaseObserver + properties: + configuration: + properties: + configMap: + description: ConfigMapDetails defines the configmap name + properties: + key: + type: string + name: + type: string + type: object + type: object database: description: DatabaseObserverDatabase defines the database details used for DatabaseObserver @@ -65,6 +6422,8 @@ spec: vaultSecretName: type: string type: object + dbRole: + type: string dbUser: properties: key: @@ -84,17 +6443,14 @@ spec: description: DatabaseObserverExporterConfig defines the configuration details related to the exporters of DatabaseObserver properties: - configuration: - properties: - configmap: - description: ConfigMapDetails defines the configmap name - properties: - configmapName: - type: string - key: - type: string - type: object - type: object + args: + items: + type: string + type: array + commands: + items: + type: string + type: array image: type: string service: @@ -106,6 +6462,29 @@ spec: type: integer type: object type: object + inherit_labels: + items: + type: string + type: array + log: + description: LogConfig defines the configuration details relation + to the logs of DatabaseObserver + properties: + filename: + type: string + path: + type: string + volume: + properties: + name: + type: string + persistentVolumeClaim: + properties: + claimName: + type: string + type: object + type: object + type: object ociConfig: properties: configMapName: @@ -117,24 +6496,2954 @@ spec: description: PrometheusConfig defines the generated resources for Prometheus properties: - labels: - additionalProperties: - type: string - type: object port: type: string + release: + type: string type: object replicas: format: int32 type: integer + sidecarVolumes: + items: + description: Volume represents a named volume in a pod that may + be accessed by any container in the pod. + properties: + awsElasticBlockStore: + description: 'awsElasticBlockStore represents an AWS Disk resource + that is attached to a kubelet''s host machine and then exposed + to the pod. More info: https://kubernetes.io/docs/concepts/storage/volumes#awselasticblockstore' + properties: + fsType: + description: 'fsType is the filesystem type of the volume + that you want to mount. Tip: Ensure that the filesystem + type is supported by the host operating system. Examples: + "ext4", "xfs", "ntfs". Implicitly inferred to be "ext4" + if unspecified. More info: https://kubernetes.io/docs/concepts/storage/volumes#awselasticblockstore + TODO: how do we prevent errors in the filesystem from + compromising the machine' + type: string + partition: + description: 'partition is the partition in the volume that + you want to mount. If omitted, the default is to mount + by volume name. Examples: For volume /dev/sda1, you specify + the partition as "1". Similarly, the volume partition + for /dev/sda is "0" (or you can leave the property empty).' + format: int32 + type: integer + readOnly: + description: 'readOnly value true will force the readOnly + setting in VolumeMounts. More info: https://kubernetes.io/docs/concepts/storage/volumes#awselasticblockstore' + type: boolean + volumeID: + description: 'volumeID is unique ID of the persistent disk + resource in AWS (Amazon EBS volume). More info: https://kubernetes.io/docs/concepts/storage/volumes#awselasticblockstore' + type: string + required: + - volumeID + type: object + azureDisk: + description: azureDisk represents an Azure Data Disk mount on + the host and bind mount to the pod. + properties: + cachingMode: + description: 'cachingMode is the Host Caching mode: None, + Read Only, Read Write.' + type: string + diskName: + description: diskName is the Name of the data disk in the + blob storage + type: string + diskURI: + description: diskURI is the URI of data disk in the blob + storage + type: string + fsType: + description: fsType is Filesystem type to mount. Must be + a filesystem type supported by the host operating system. + Ex. "ext4", "xfs", "ntfs". Implicitly inferred to be "ext4" + if unspecified. + type: string + kind: + description: 'kind expected values are Shared: multiple + blob disks per storage account Dedicated: single blob + disk per storage account Managed: azure managed data + disk (only in managed availability set). defaults to shared' + type: string + readOnly: + description: readOnly Defaults to false (read/write). ReadOnly + here will force the ReadOnly setting in VolumeMounts. + type: boolean + required: + - diskName + - diskURI + type: object + azureFile: + description: azureFile represents an Azure File Service mount + on the host and bind mount to the pod. + properties: + readOnly: + description: readOnly defaults to false (read/write). ReadOnly + here will force the ReadOnly setting in VolumeMounts. + type: boolean + secretName: + description: secretName is the name of secret that contains + Azure Storage Account Name and Key + type: string + shareName: + description: shareName is the azure share Name + type: string + required: + - secretName + - shareName + type: object + cephfs: + description: cephFS represents a Ceph FS mount on the host that + shares a pod's lifetime + properties: + monitors: + description: 'monitors is Required: Monitors is a collection + of Ceph monitors More info: https://examples.k8s.io/volumes/cephfs/README.md#how-to-use-it' + items: + type: string + type: array + path: + description: 'path is Optional: Used as the mounted root, + rather than the full Ceph tree, default is /' + type: string + readOnly: + description: 'readOnly is Optional: Defaults to false (read/write). + ReadOnly here will force the ReadOnly setting in VolumeMounts. + More info: https://examples.k8s.io/volumes/cephfs/README.md#how-to-use-it' + type: boolean + secretFile: + description: 'secretFile is Optional: SecretFile is the + path to key ring for User, default is /etc/ceph/user.secret + More info: https://examples.k8s.io/volumes/cephfs/README.md#how-to-use-it' + type: string + secretRef: + description: 'secretRef is Optional: SecretRef is reference + to the authentication secret for User, default is empty. + More info: https://examples.k8s.io/volumes/cephfs/README.md#how-to-use-it' + properties: + name: + description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, uid?' + type: string + type: object + user: + description: 'user is optional: User is the rados user name, + default is admin More info: https://examples.k8s.io/volumes/cephfs/README.md#how-to-use-it' + type: string + required: + - monitors + type: object + cinder: + description: 'cinder represents a cinder volume attached and + mounted on kubelets host machine. More info: https://examples.k8s.io/mysql-cinder-pd/README.md' + properties: + fsType: + description: 'fsType is the filesystem type to mount. Must + be a filesystem type supported by the host operating system. + Examples: "ext4", "xfs", "ntfs". Implicitly inferred to + be "ext4" if unspecified. More info: https://examples.k8s.io/mysql-cinder-pd/README.md' + type: string + readOnly: + description: 'readOnly defaults to false (read/write). ReadOnly + here will force the ReadOnly setting in VolumeMounts. + More info: https://examples.k8s.io/mysql-cinder-pd/README.md' + type: boolean + secretRef: + description: 'secretRef is optional: points to a secret + object containing parameters used to connect to OpenStack.' + properties: + name: + description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, uid?' + type: string + type: object + volumeID: + description: 'volumeID used to identify the volume in cinder. + More info: https://examples.k8s.io/mysql-cinder-pd/README.md' + type: string + required: + - volumeID + type: object + configMap: + description: configMap represents a configMap that should populate + this volume + properties: + defaultMode: + description: 'defaultMode is optional: mode bits used to + set permissions on created files by default. Must be an + octal value between 0000 and 0777 or a decimal value between + 0 and 511. YAML accepts both octal and decimal values, + JSON requires decimal values for mode bits. Defaults to + 0644. Directories within the path are not affected by + this setting. This might be in conflict with other options + that affect the file mode, like fsGroup, and the result + can be other mode bits set.' + format: int32 + type: integer + items: + description: items if unspecified, each key-value pair in + the Data field of the referenced ConfigMap will be projected + into the volume as a file whose name is the key and content + is the value. If specified, the listed keys will be projected + into the specified paths, and unlisted keys will not be + present. If a key is specified which is not present in + the ConfigMap, the volume setup will error unless it is + marked optional. Paths must be relative and may not contain + the '..' path or start with '..'. + items: + description: Maps a string key to a path within a volume. + properties: + key: + description: key is the key to project. + type: string + mode: + description: 'mode is Optional: mode bits used to + set permissions on this file. Must be an octal value + between 0000 and 0777 or a decimal value between + 0 and 511. YAML accepts both octal and decimal values, + JSON requires decimal values for mode bits. If not + specified, the volume defaultMode will be used. + This might be in conflict with other options that + affect the file mode, like fsGroup, and the result + can be other mode bits set.' + format: int32 + type: integer + path: + description: path is the relative path of the file + to map the key to. May not be an absolute path. + May not contain the path element '..'. May not start + with the string '..'. + type: string + required: + - key + - path + type: object + type: array + name: + description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, uid?' + type: string + optional: + description: optional specify whether the ConfigMap or its + keys must be defined + type: boolean + type: object + csi: + description: csi (Container Storage Interface) represents ephemeral + storage that is handled by certain external CSI drivers (Beta + feature). + properties: + driver: + description: driver is the name of the CSI driver that handles + this volume. Consult with your admin for the correct name + as registered in the cluster. + type: string + fsType: + description: fsType to mount. Ex. "ext4", "xfs", "ntfs". + If not provided, the empty value is passed to the associated + CSI driver which will determine the default filesystem + to apply. + type: string + nodePublishSecretRef: + description: nodePublishSecretRef is a reference to the + secret object containing sensitive information to pass + to the CSI driver to complete the CSI NodePublishVolume + and NodeUnpublishVolume calls. This field is optional, + and may be empty if no secret is required. If the secret + object contains more than one secret, all secret references + are passed. + properties: + name: + description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, uid?' + type: string + type: object + readOnly: + description: readOnly specifies a read-only configuration + for the volume. Defaults to false (read/write). + type: boolean + volumeAttributes: + additionalProperties: + type: string + description: volumeAttributes stores driver-specific properties + that are passed to the CSI driver. Consult your driver's + documentation for supported values. + type: object + required: + - driver + type: object + downwardAPI: + description: downwardAPI represents downward API about the pod + that should populate this volume + properties: + defaultMode: + description: 'Optional: mode bits to use on created files + by default. Must be a Optional: mode bits used to set + permissions on created files by default. Must be an octal + value between 0000 and 0777 or a decimal value between + 0 and 511. YAML accepts both octal and decimal values, + JSON requires decimal values for mode bits. Defaults to + 0644. Directories within the path are not affected by + this setting. This might be in conflict with other options + that affect the file mode, like fsGroup, and the result + can be other mode bits set.' + format: int32 + type: integer + items: + description: Items is a list of downward API volume file + items: + description: DownwardAPIVolumeFile represents information + to create the file containing the pod field + properties: + fieldRef: + description: 'Required: Selects a field of the pod: + only annotations, labels, name and namespace are + supported.' + properties: + apiVersion: + description: Version of the schema the FieldPath + is written in terms of, defaults to "v1". + type: string + fieldPath: + description: Path of the field to select in the + specified API version. + type: string + required: + - fieldPath + type: object + mode: + description: 'Optional: mode bits used to set permissions + on this file, must be an octal value between 0000 + and 0777 or a decimal value between 0 and 511. YAML + accepts both octal and decimal values, JSON requires + decimal values for mode bits. If not specified, + the volume defaultMode will be used. This might + be in conflict with other options that affect the + file mode, like fsGroup, and the result can be other + mode bits set.' + format: int32 + type: integer + path: + description: 'Required: Path is the relative path + name of the file to be created. Must not be absolute + or contain the ''..'' path. Must be utf-8 encoded. + The first item of the relative path must not start + with ''..''' + type: string + resourceFieldRef: + description: 'Selects a resource of the container: + only resources limits and requests (limits.cpu, + limits.memory, requests.cpu and requests.memory) + are currently supported.' + properties: + containerName: + description: 'Container name: required for volumes, + optional for env vars' + type: string + divisor: + anyOf: + - type: integer + - type: string + description: Specifies the output format of the + exposed resources, defaults to "1" + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + resource: + description: 'Required: resource to select' + type: string + required: + - resource + type: object + required: + - path + type: object + type: array + type: object + emptyDir: + description: 'emptyDir represents a temporary directory that + shares a pod''s lifetime. More info: https://kubernetes.io/docs/concepts/storage/volumes#emptydir' + properties: + medium: + description: 'medium represents what type of storage medium + should back this directory. The default is "" which means + to use the node''s default medium. Must be an empty string + (default) or Memory. More info: https://kubernetes.io/docs/concepts/storage/volumes#emptydir' + type: string + sizeLimit: + anyOf: + - type: integer + - type: string + description: 'sizeLimit is the total amount of local storage + required for this EmptyDir volume. The size limit is also + applicable for memory medium. The maximum usage on memory + medium EmptyDir would be the minimum value between the + SizeLimit specified here and the sum of memory limits + of all containers in a pod. The default is nil which means + that the limit is undefined. More info: https://kubernetes.io/docs/concepts/storage/volumes#emptydir' + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + ephemeral: + description: "ephemeral represents a volume that is handled + by a cluster storage driver. The volume's lifecycle is tied + to the pod that defines it - it will be created before the + pod starts, and deleted when the pod is removed. \n Use this + if: a) the volume is only needed while the pod runs, b) features + of normal volumes like restoring from snapshot or capacity + \ tracking are needed, c) the storage driver is specified + through a storage class, and d) the storage driver supports + dynamic volume provisioning through a PersistentVolumeClaim + (see EphemeralVolumeSource for more information on the + connection between this volume type and PersistentVolumeClaim). + \n Use PersistentVolumeClaim or one of the vendor-specific + APIs for volumes that persist for longer than the lifecycle + of an individual pod. \n Use CSI for light-weight local ephemeral + volumes if the CSI driver is meant to be used that way - see + the documentation of the driver for more information. \n A + pod can use both types of ephemeral volumes and persistent + volumes at the same time." + properties: + volumeClaimTemplate: + description: "Will be used to create a stand-alone PVC to + provision the volume. The pod in which this EphemeralVolumeSource + is embedded will be the owner of the PVC, i.e. the PVC + will be deleted together with the pod. The name of the + PVC will be `-` where `` is the name from the `PodSpec.Volumes` array entry. + Pod validation will reject the pod if the concatenated + name is not valid for a PVC (for example, too long). \n + An existing PVC with that name that is not owned by the + pod will *not* be used for the pod to avoid using an unrelated + volume by mistake. Starting the pod is then blocked until + the unrelated PVC is removed. If such a pre-created PVC + is meant to be used by the pod, the PVC has to updated + with an owner reference to the pod once the pod exists. + Normally this should not be necessary, but it may be useful + when manually reconstructing a broken cluster. \n This + field is read-only and no changes will be made by Kubernetes + to the PVC after it has been created. \n Required, must + not be nil." + properties: + metadata: + description: May contain labels and annotations that + will be copied into the PVC when creating it. No other + fields are allowed and will be rejected during validation. + type: object + spec: + description: The specification for the PersistentVolumeClaim. + The entire content is copied unchanged into the PVC + that gets created from this template. The same fields + as in a PersistentVolumeClaim are also valid here. + properties: + accessModes: + description: 'accessModes contains the desired access + modes the volume should have. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1' + items: + type: string + type: array + dataSource: + description: 'dataSource field can be used to specify + either: * An existing VolumeSnapshot object (snapshot.storage.k8s.io/VolumeSnapshot) + * An existing PVC (PersistentVolumeClaim) If the + provisioner or an external controller can support + the specified data source, it will create a new + volume based on the contents of the specified + data source. When the AnyVolumeDataSource feature + gate is enabled, dataSource contents will be copied + to dataSourceRef, and dataSourceRef contents will + be copied to dataSource when dataSourceRef.namespace + is not specified. If the namespace is specified, + then dataSourceRef will not be copied to dataSource.' + properties: + apiGroup: + description: APIGroup is the group for the resource + being referenced. If APIGroup is not specified, + the specified Kind must be in the core API + group. For any other third-party types, APIGroup + is required. + type: string + kind: + description: Kind is the type of resource being + referenced + type: string + name: + description: Name is the name of resource being + referenced + type: string + required: + - kind + - name + type: object + dataSourceRef: + description: 'dataSourceRef specifies the object + from which to populate the volume with data, if + a non-empty volume is desired. This may be any + object from a non-empty API group (non core object) + or a PersistentVolumeClaim object. When this field + is specified, volume binding will only succeed + if the type of the specified object matches some + installed volume populator or dynamic provisioner. + This field will replace the functionality of the + dataSource field and as such if both fields are + non-empty, they must have the same value. For + backwards compatibility, when namespace isn''t + specified in dataSourceRef, both fields (dataSource + and dataSourceRef) will be set to the same value + automatically if one of them is empty and the + other is non-empty. When namespace is specified + in dataSourceRef, dataSource isn''t set to the + same value and must be empty. There are three + important differences between dataSource and dataSourceRef: + * While dataSource only allows two specific types + of objects, dataSourceRef allows any non-core + object, as well as PersistentVolumeClaim objects. + * While dataSource ignores disallowed values (dropping + them), dataSourceRef preserves all values, and + generates an error if a disallowed value is specified. + * While dataSource only allows local objects, + dataSourceRef allows objects in any namespaces. + (Beta) Using this field requires the AnyVolumeDataSource + feature gate to be enabled. (Alpha) Using the + namespace field of dataSourceRef requires the + CrossNamespaceVolumeDataSource feature gate to + be enabled.' + properties: + apiGroup: + description: APIGroup is the group for the resource + being referenced. If APIGroup is not specified, + the specified Kind must be in the core API + group. For any other third-party types, APIGroup + is required. + type: string + kind: + description: Kind is the type of resource being + referenced + type: string + name: + description: Name is the name of resource being + referenced + type: string + namespace: + description: Namespace is the namespace of resource + being referenced Note that when a namespace + is specified, a gateway.networking.k8s.io/ReferenceGrant + object is required in the referent namespace + to allow that namespace's owner to accept + the reference. See the ReferenceGrant documentation + for details. (Alpha) This field requires the + CrossNamespaceVolumeDataSource feature gate + to be enabled. + type: string + required: + - kind + - name + type: object + resources: + description: 'resources represents the minimum resources + the volume should have. If RecoverVolumeExpansionFailure + feature is enabled users are allowed to specify + resource requirements that are lower than previous + value but must still be higher than capacity recorded + in the status field of the claim. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#resources' + properties: + limits: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + description: 'Limits describes the maximum amount + of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + type: object + requests: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + description: 'Requests describes the minimum + amount of compute resources required. If Requests + is omitted for a container, it defaults to + Limits if that is explicitly specified, otherwise + to an implementation-defined value. Requests + cannot exceed Limits. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + type: object + type: object + selector: + description: selector is a label query over volumes + to consider for binding. + properties: + matchExpressions: + description: matchExpressions is a list of label + selector requirements. The requirements are + ANDed. + items: + description: A label selector requirement + is a selector that contains values, a key, + and an operator that relates the key and + values. + properties: + key: + description: key is the label key that + the selector applies to. + type: string + operator: + description: operator represents a key's + relationship to a set of values. Valid + operators are In, NotIn, Exists and + DoesNotExist. + type: string + values: + description: values is an array of string + values. If the operator is In or NotIn, + the values array must be non-empty. + If the operator is Exists or DoesNotExist, + the values array must be empty. This + array is replaced during a strategic + merge patch. + items: + type: string + type: array + required: + - key + - operator + type: object + type: array + matchLabels: + additionalProperties: + type: string + description: matchLabels is a map of {key,value} + pairs. A single {key,value} in the matchLabels + map is equivalent to an element of matchExpressions, + whose key field is "key", the operator is + "In", and the values array contains only "value". + The requirements are ANDed. + type: object + type: object + storageClassName: + description: 'storageClassName is the name of the + StorageClass required by the claim. More info: + https://kubernetes.io/docs/concepts/storage/persistent-volumes#class-1' + type: string + volumeAttributesClassName: + description: 'volumeAttributesClassName may be used + to set the VolumeAttributesClass used by this + claim. If specified, the CSI driver will create + or update the volume with the attributes defined + in the corresponding VolumeAttributesClass. This + has a different purpose than storageClassName, + it can be changed after the claim is created. + An empty string value means that no VolumeAttributesClass + will be applied to the claim but it''s not allowed + to reset this field to empty string once it is + set. If unspecified and the PersistentVolumeClaim + is unbound, the default VolumeAttributesClass + will be set by the persistentvolume controller + if it exists. If the resource referred to by volumeAttributesClass + does not exist, this PersistentVolumeClaim will + be set to a Pending state, as reflected by the + modifyVolumeStatus field, until such as a resource + exists. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#volumeattributesclass + (Alpha) Using this field requires the VolumeAttributesClass + feature gate to be enabled.' + type: string + volumeMode: + description: volumeMode defines what type of volume + is required by the claim. Value of Filesystem + is implied when not included in claim spec. + type: string + volumeName: + description: volumeName is the binding reference + to the PersistentVolume backing this claim. + type: string + type: object + required: + - spec + type: object + type: object + fc: + description: fc represents a Fibre Channel resource that is + attached to a kubelet's host machine and then exposed to the + pod. + properties: + fsType: + description: 'fsType is the filesystem type to mount. Must + be a filesystem type supported by the host operating system. + Ex. "ext4", "xfs", "ntfs". Implicitly inferred to be "ext4" + if unspecified. TODO: how do we prevent errors in the + filesystem from compromising the machine' + type: string + lun: + description: 'lun is Optional: FC target lun number' + format: int32 + type: integer + readOnly: + description: 'readOnly is Optional: Defaults to false (read/write). + ReadOnly here will force the ReadOnly setting in VolumeMounts.' + type: boolean + targetWWNs: + description: 'targetWWNs is Optional: FC target worldwide + names (WWNs)' + items: + type: string + type: array + wwids: + description: 'wwids Optional: FC volume world wide identifiers + (wwids) Either wwids or combination of targetWWNs and + lun must be set, but not both simultaneously.' + items: + type: string + type: array + type: object + flexVolume: + description: flexVolume represents a generic volume resource + that is provisioned/attached using an exec based plugin. + properties: + driver: + description: driver is the name of the driver to use for + this volume. + type: string + fsType: + description: fsType is the filesystem type to mount. Must + be a filesystem type supported by the host operating system. + Ex. "ext4", "xfs", "ntfs". The default filesystem depends + on FlexVolume script. + type: string + options: + additionalProperties: + type: string + description: 'options is Optional: this field holds extra + command options if any.' + type: object + readOnly: + description: 'readOnly is Optional: defaults to false (read/write). + ReadOnly here will force the ReadOnly setting in VolumeMounts.' + type: boolean + secretRef: + description: 'secretRef is Optional: secretRef is reference + to the secret object containing sensitive information + to pass to the plugin scripts. This may be empty if no + secret object is specified. If the secret object contains + more than one secret, all secrets are passed to the plugin + scripts.' + properties: + name: + description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, uid?' + type: string + type: object + required: + - driver + type: object + flocker: + description: flocker represents a Flocker volume attached to + a kubelet's host machine. This depends on the Flocker control + service being running + properties: + datasetName: + description: datasetName is Name of the dataset stored as + metadata -> name on the dataset for Flocker should be + considered as deprecated + type: string + datasetUUID: + description: datasetUUID is the UUID of the dataset. This + is unique identifier of a Flocker dataset + type: string + type: object + gcePersistentDisk: + description: 'gcePersistentDisk represents a GCE Disk resource + that is attached to a kubelet''s host machine and then exposed + to the pod. More info: https://kubernetes.io/docs/concepts/storage/volumes#gcepersistentdisk' + properties: + fsType: + description: 'fsType is filesystem type of the volume that + you want to mount. Tip: Ensure that the filesystem type + is supported by the host operating system. Examples: "ext4", + "xfs", "ntfs". Implicitly inferred to be "ext4" if unspecified. + More info: https://kubernetes.io/docs/concepts/storage/volumes#gcepersistentdisk + TODO: how do we prevent errors in the filesystem from + compromising the machine' + type: string + partition: + description: 'partition is the partition in the volume that + you want to mount. If omitted, the default is to mount + by volume name. Examples: For volume /dev/sda1, you specify + the partition as "1". Similarly, the volume partition + for /dev/sda is "0" (or you can leave the property empty). + More info: https://kubernetes.io/docs/concepts/storage/volumes#gcepersistentdisk' + format: int32 + type: integer + pdName: + description: 'pdName is unique name of the PD resource in + GCE. Used to identify the disk in GCE. More info: https://kubernetes.io/docs/concepts/storage/volumes#gcepersistentdisk' + type: string + readOnly: + description: 'readOnly here will force the ReadOnly setting + in VolumeMounts. Defaults to false. More info: https://kubernetes.io/docs/concepts/storage/volumes#gcepersistentdisk' + type: boolean + required: + - pdName + type: object + gitRepo: + description: 'gitRepo represents a git repository at a particular + revision. DEPRECATED: GitRepo is deprecated. To provision + a container with a git repo, mount an EmptyDir into an InitContainer + that clones the repo using git, then mount the EmptyDir into + the Pod''s container.' + properties: + directory: + description: directory is the target directory name. Must + not contain or start with '..'. If '.' is supplied, the + volume directory will be the git repository. Otherwise, + if specified, the volume will contain the git repository + in the subdirectory with the given name. + type: string + repository: + description: repository is the URL + type: string + revision: + description: revision is the commit hash for the specified + revision. + type: string + required: + - repository + type: object + glusterfs: + description: 'glusterfs represents a Glusterfs mount on the + host that shares a pod''s lifetime. More info: https://examples.k8s.io/volumes/glusterfs/README.md' + properties: + endpoints: + description: 'endpoints is the endpoint name that details + Glusterfs topology. More info: https://examples.k8s.io/volumes/glusterfs/README.md#create-a-pod' + type: string + path: + description: 'path is the Glusterfs volume path. More info: + https://examples.k8s.io/volumes/glusterfs/README.md#create-a-pod' + type: string + readOnly: + description: 'readOnly here will force the Glusterfs volume + to be mounted with read-only permissions. Defaults to + false. More info: https://examples.k8s.io/volumes/glusterfs/README.md#create-a-pod' + type: boolean + required: + - endpoints + - path + type: object + hostPath: + description: 'hostPath represents a pre-existing file or directory + on the host machine that is directly exposed to the container. + This is generally used for system agents or other privileged + things that are allowed to see the host machine. Most containers + will NOT need this. More info: https://kubernetes.io/docs/concepts/storage/volumes#hostpath + --- TODO(jonesdl) We need to restrict who can use host directory + mounts and who can/can not mount host directories as read/write.' + properties: + path: + description: 'path of the directory on the host. If the + path is a symlink, it will follow the link to the real + path. More info: https://kubernetes.io/docs/concepts/storage/volumes#hostpath' + type: string + type: + description: 'type for HostPath Volume Defaults to "" More + info: https://kubernetes.io/docs/concepts/storage/volumes#hostpath' + type: string + required: + - path + type: object + iscsi: + description: 'iscsi represents an ISCSI Disk resource that is + attached to a kubelet''s host machine and then exposed to + the pod. More info: https://examples.k8s.io/volumes/iscsi/README.md' + properties: + chapAuthDiscovery: + description: chapAuthDiscovery defines whether support iSCSI + Discovery CHAP authentication + type: boolean + chapAuthSession: + description: chapAuthSession defines whether support iSCSI + Session CHAP authentication + type: boolean + fsType: + description: 'fsType is the filesystem type of the volume + that you want to mount. Tip: Ensure that the filesystem + type is supported by the host operating system. Examples: + "ext4", "xfs", "ntfs". Implicitly inferred to be "ext4" + if unspecified. More info: https://kubernetes.io/docs/concepts/storage/volumes#iscsi + TODO: how do we prevent errors in the filesystem from + compromising the machine' + type: string + initiatorName: + description: initiatorName is the custom iSCSI Initiator + Name. If initiatorName is specified with iscsiInterface + simultaneously, new iSCSI interface : will be created for the connection. + type: string + iqn: + description: iqn is the target iSCSI Qualified Name. + type: string + iscsiInterface: + description: iscsiInterface is the interface Name that uses + an iSCSI transport. Defaults to 'default' (tcp). + type: string + lun: + description: lun represents iSCSI Target Lun number. + format: int32 + type: integer + portals: + description: portals is the iSCSI Target Portal List. The + portal is either an IP or ip_addr:port if the port is + other than default (typically TCP ports 860 and 3260). + items: + type: string + type: array + readOnly: + description: readOnly here will force the ReadOnly setting + in VolumeMounts. Defaults to false. + type: boolean + secretRef: + description: secretRef is the CHAP Secret for iSCSI target + and initiator authentication + properties: + name: + description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, uid?' + type: string + type: object + targetPortal: + description: targetPortal is iSCSI Target Portal. The Portal + is either an IP or ip_addr:port if the port is other than + default (typically TCP ports 860 and 3260). + type: string + required: + - iqn + - lun + - targetPortal + type: object + name: + description: 'name of the volume. Must be a DNS_LABEL and unique + within the pod. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names' + type: string + nfs: + description: 'nfs represents an NFS mount on the host that shares + a pod''s lifetime More info: https://kubernetes.io/docs/concepts/storage/volumes#nfs' + properties: + path: + description: 'path that is exported by the NFS server. More + info: https://kubernetes.io/docs/concepts/storage/volumes#nfs' + type: string + readOnly: + description: 'readOnly here will force the NFS export to + be mounted with read-only permissions. Defaults to false. + More info: https://kubernetes.io/docs/concepts/storage/volumes#nfs' + type: boolean + server: + description: 'server is the hostname or IP address of the + NFS server. More info: https://kubernetes.io/docs/concepts/storage/volumes#nfs' + type: string + required: + - path + - server + type: object + persistentVolumeClaim: + description: 'persistentVolumeClaimVolumeSource represents a + reference to a PersistentVolumeClaim in the same namespace. + More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims' + properties: + claimName: + description: 'claimName is the name of a PersistentVolumeClaim + in the same namespace as the pod using this volume. More + info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims' + type: string + readOnly: + description: readOnly Will force the ReadOnly setting in + VolumeMounts. Default false. + type: boolean + required: + - claimName + type: object + photonPersistentDisk: + description: photonPersistentDisk represents a PhotonController + persistent disk attached and mounted on kubelets host machine + properties: + fsType: + description: fsType is the filesystem type to mount. Must + be a filesystem type supported by the host operating system. + Ex. "ext4", "xfs", "ntfs". Implicitly inferred to be "ext4" + if unspecified. + type: string + pdID: + description: pdID is the ID that identifies Photon Controller + persistent disk + type: string + required: + - pdID + type: object + portworxVolume: + description: portworxVolume represents a portworx volume attached + and mounted on kubelets host machine + properties: + fsType: + description: fSType represents the filesystem type to mount + Must be a filesystem type supported by the host operating + system. Ex. "ext4", "xfs". Implicitly inferred to be "ext4" + if unspecified. + type: string + readOnly: + description: readOnly defaults to false (read/write). ReadOnly + here will force the ReadOnly setting in VolumeMounts. + type: boolean + volumeID: + description: volumeID uniquely identifies a Portworx volume + type: string + required: + - volumeID + type: object + projected: + description: projected items for all in one resources secrets, + configmaps, and downward API + properties: + defaultMode: + description: defaultMode are the mode bits used to set permissions + on created files by default. Must be an octal value between + 0000 and 0777 or a decimal value between 0 and 511. YAML + accepts both octal and decimal values, JSON requires decimal + values for mode bits. Directories within the path are + not affected by this setting. This might be in conflict + with other options that affect the file mode, like fsGroup, + and the result can be other mode bits set. + format: int32 + type: integer + sources: + description: sources is the list of volume projections + items: + description: Projection that may be projected along with + other supported volume types + properties: + clusterTrustBundle: + description: "ClusterTrustBundle allows a pod to access + the `.spec.trustBundle` field of ClusterTrustBundle + objects in an auto-updating file. \n Alpha, gated + by the ClusterTrustBundleProjection feature gate. + \n ClusterTrustBundle objects can either be selected + by name, or by the combination of signer name and + a label selector. \n Kubelet performs aggressive + normalization of the PEM contents written into the + pod filesystem. Esoteric PEM features such as inter-block + comments and block headers are stripped. Certificates + are deduplicated. The ordering of certificates within + the file is arbitrary, and Kubelet may change the + order over time." + properties: + labelSelector: + description: Select all ClusterTrustBundles that + match this label selector. Only has effect + if signerName is set. Mutually-exclusive with + name. If unset, interpreted as "match nothing". If + set but empty, interpreted as "match everything". + properties: + matchExpressions: + description: matchExpressions is a list of + label selector requirements. The requirements + are ANDed. + items: + description: A label selector requirement + is a selector that contains values, a + key, and an operator that relates the + key and values. + properties: + key: + description: key is the label key that + the selector applies to. + type: string + operator: + description: operator represents a key's + relationship to a set of values. Valid + operators are In, NotIn, Exists and + DoesNotExist. + type: string + values: + description: values is an array of string + values. If the operator is In or NotIn, + the values array must be non-empty. + If the operator is Exists or DoesNotExist, + the values array must be empty. This + array is replaced during a strategic + merge patch. + items: + type: string + type: array + required: + - key + - operator + type: object + type: array + matchLabels: + additionalProperties: + type: string + description: matchLabels is a map of {key,value} + pairs. A single {key,value} in the matchLabels + map is equivalent to an element of matchExpressions, + whose key field is "key", the operator is + "In", and the values array contains only + "value". The requirements are ANDed. + type: object + type: object + name: + description: Select a single ClusterTrustBundle + by object name. Mutually-exclusive with signerName + and labelSelector. + type: string + optional: + description: If true, don't block pod startup + if the referenced ClusterTrustBundle(s) aren't + available. If using name, then the named ClusterTrustBundle + is allowed not to exist. If using signerName, + then the combination of signerName and labelSelector + is allowed to match zero ClusterTrustBundles. + type: boolean + path: + description: Relative path from the volume root + to write the bundle. + type: string + signerName: + description: Select all ClusterTrustBundles that + match this signer name. Mutually-exclusive with + name. The contents of all selected ClusterTrustBundles + will be unified and deduplicated. + type: string + required: + - path + type: object + configMap: + description: configMap information about the configMap + data to project + properties: + items: + description: items if unspecified, each key-value + pair in the Data field of the referenced ConfigMap + will be projected into the volume as a file + whose name is the key and content is the value. + If specified, the listed keys will be projected + into the specified paths, and unlisted keys + will not be present. If a key is specified which + is not present in the ConfigMap, the volume + setup will error unless it is marked optional. + Paths must be relative and may not contain the + '..' path or start with '..'. + items: + description: Maps a string key to a path within + a volume. + properties: + key: + description: key is the key to project. + type: string + mode: + description: 'mode is Optional: mode bits + used to set permissions on this file. + Must be an octal value between 0000 and + 0777 or a decimal value between 0 and + 511. YAML accepts both octal and decimal + values, JSON requires decimal values for + mode bits. If not specified, the volume + defaultMode will be used. This might be + in conflict with other options that affect + the file mode, like fsGroup, and the result + can be other mode bits set.' + format: int32 + type: integer + path: + description: path is the relative path of + the file to map the key to. May not be + an absolute path. May not contain the + path element '..'. May not start with + the string '..'. + type: string + required: + - key + - path + type: object + type: array + name: + description: 'Name of the referent. More info: + https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, + uid?' + type: string + optional: + description: optional specify whether the ConfigMap + or its keys must be defined + type: boolean + type: object + downwardAPI: + description: downwardAPI information about the downwardAPI + data to project + properties: + items: + description: Items is a list of DownwardAPIVolume + file + items: + description: DownwardAPIVolumeFile represents + information to create the file containing + the pod field + properties: + fieldRef: + description: 'Required: Selects a field + of the pod: only annotations, labels, + name and namespace are supported.' + properties: + apiVersion: + description: Version of the schema the + FieldPath is written in terms of, + defaults to "v1". + type: string + fieldPath: + description: Path of the field to select + in the specified API version. + type: string + required: + - fieldPath + type: object + mode: + description: 'Optional: mode bits used to + set permissions on this file, must be + an octal value between 0000 and 0777 or + a decimal value between 0 and 511. YAML + accepts both octal and decimal values, + JSON requires decimal values for mode + bits. If not specified, the volume defaultMode + will be used. This might be in conflict + with other options that affect the file + mode, like fsGroup, and the result can + be other mode bits set.' + format: int32 + type: integer + path: + description: 'Required: Path is the relative + path name of the file to be created. Must + not be absolute or contain the ''..'' + path. Must be utf-8 encoded. The first + item of the relative path must not start + with ''..''' + type: string + resourceFieldRef: + description: 'Selects a resource of the + container: only resources limits and requests + (limits.cpu, limits.memory, requests.cpu + and requests.memory) are currently supported.' + properties: + containerName: + description: 'Container name: required + for volumes, optional for env vars' + type: string + divisor: + anyOf: + - type: integer + - type: string + description: Specifies the output format + of the exposed resources, defaults + to "1" + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + resource: + description: 'Required: resource to + select' + type: string + required: + - resource + type: object + required: + - path + type: object + type: array + type: object + secret: + description: secret information about the secret data + to project + properties: + items: + description: items if unspecified, each key-value + pair in the Data field of the referenced Secret + will be projected into the volume as a file + whose name is the key and content is the value. + If specified, the listed keys will be projected + into the specified paths, and unlisted keys + will not be present. If a key is specified which + is not present in the Secret, the volume setup + will error unless it is marked optional. Paths + must be relative and may not contain the '..' + path or start with '..'. + items: + description: Maps a string key to a path within + a volume. + properties: + key: + description: key is the key to project. + type: string + mode: + description: 'mode is Optional: mode bits + used to set permissions on this file. + Must be an octal value between 0000 and + 0777 or a decimal value between 0 and + 511. YAML accepts both octal and decimal + values, JSON requires decimal values for + mode bits. If not specified, the volume + defaultMode will be used. This might be + in conflict with other options that affect + the file mode, like fsGroup, and the result + can be other mode bits set.' + format: int32 + type: integer + path: + description: path is the relative path of + the file to map the key to. May not be + an absolute path. May not contain the + path element '..'. May not start with + the string '..'. + type: string + required: + - key + - path + type: object + type: array + name: + description: 'Name of the referent. More info: + https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, + uid?' + type: string + optional: + description: optional field specify whether the + Secret or its key must be defined + type: boolean + type: object + serviceAccountToken: + description: serviceAccountToken is information about + the serviceAccountToken data to project + properties: + audience: + description: audience is the intended audience + of the token. A recipient of a token must identify + itself with an identifier specified in the audience + of the token, and otherwise should reject the + token. The audience defaults to the identifier + of the apiserver. + type: string + expirationSeconds: + description: expirationSeconds is the requested + duration of validity of the service account + token. As the token approaches expiration, the + kubelet volume plugin will proactively rotate + the service account token. The kubelet will + start trying to rotate the token if the token + is older than 80 percent of its time to live + or if the token is older than 24 hours.Defaults + to 1 hour and must be at least 10 minutes. + format: int64 + type: integer + path: + description: path is the path relative to the + mount point of the file to project the token + into. + type: string + required: + - path + type: object + type: object + type: array + type: object + quobyte: + description: quobyte represents a Quobyte mount on the host + that shares a pod's lifetime + properties: + group: + description: group to map volume access to Default is no + group + type: string + readOnly: + description: readOnly here will force the Quobyte volume + to be mounted with read-only permissions. Defaults to + false. + type: boolean + registry: + description: registry represents a single or multiple Quobyte + Registry services specified as a string as host:port pair + (multiple entries are separated with commas) which acts + as the central registry for volumes + type: string + tenant: + description: tenant owning the given Quobyte volume in the + Backend Used with dynamically provisioned Quobyte volumes, + value is set by the plugin + type: string + user: + description: user to map volume access to Defaults to serivceaccount + user + type: string + volume: + description: volume is a string that references an already + created Quobyte volume by name. + type: string + required: + - registry + - volume + type: object + rbd: + description: 'rbd represents a Rados Block Device mount on the + host that shares a pod''s lifetime. More info: https://examples.k8s.io/volumes/rbd/README.md' + properties: + fsType: + description: 'fsType is the filesystem type of the volume + that you want to mount. Tip: Ensure that the filesystem + type is supported by the host operating system. Examples: + "ext4", "xfs", "ntfs". Implicitly inferred to be "ext4" + if unspecified. More info: https://kubernetes.io/docs/concepts/storage/volumes#rbd + TODO: how do we prevent errors in the filesystem from + compromising the machine' + type: string + image: + description: 'image is the rados image name. More info: + https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it' + type: string + keyring: + description: 'keyring is the path to key ring for RBDUser. + Default is /etc/ceph/keyring. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it' + type: string + monitors: + description: 'monitors is a collection of Ceph monitors. + More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it' + items: + type: string + type: array + pool: + description: 'pool is the rados pool name. Default is rbd. + More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it' + type: string + readOnly: + description: 'readOnly here will force the ReadOnly setting + in VolumeMounts. Defaults to false. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it' + type: boolean + secretRef: + description: 'secretRef is name of the authentication secret + for RBDUser. If provided overrides keyring. Default is + nil. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it' + properties: + name: + description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, uid?' + type: string + type: object + user: + description: 'user is the rados user name. Default is admin. + More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it' + type: string + required: + - image + - monitors + type: object + scaleIO: + description: scaleIO represents a ScaleIO persistent volume + attached and mounted on Kubernetes nodes. + properties: + fsType: + description: fsType is the filesystem type to mount. Must + be a filesystem type supported by the host operating system. + Ex. "ext4", "xfs", "ntfs". Default is "xfs". + type: string + gateway: + description: gateway is the host address of the ScaleIO + API Gateway. + type: string + protectionDomain: + description: protectionDomain is the name of the ScaleIO + Protection Domain for the configured storage. + type: string + readOnly: + description: readOnly Defaults to false (read/write). ReadOnly + here will force the ReadOnly setting in VolumeMounts. + type: boolean + secretRef: + description: secretRef references to the secret for ScaleIO + user and other sensitive information. If this is not provided, + Login operation will fail. + properties: + name: + description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, uid?' + type: string + type: object + sslEnabled: + description: sslEnabled Flag enable/disable SSL communication + with Gateway, default false + type: boolean + storageMode: + description: storageMode indicates whether the storage for + a volume should be ThickProvisioned or ThinProvisioned. + Default is ThinProvisioned. + type: string + storagePool: + description: storagePool is the ScaleIO Storage Pool associated + with the protection domain. + type: string + system: + description: system is the name of the storage system as + configured in ScaleIO. + type: string + volumeName: + description: volumeName is the name of a volume already + created in the ScaleIO system that is associated with + this volume source. + type: string + required: + - gateway + - secretRef + - system + type: object + secret: + description: 'secret represents a secret that should populate + this volume. More info: https://kubernetes.io/docs/concepts/storage/volumes#secret' + properties: + defaultMode: + description: 'defaultMode is Optional: mode bits used to + set permissions on created files by default. Must be an + octal value between 0000 and 0777 or a decimal value between + 0 and 511. YAML accepts both octal and decimal values, + JSON requires decimal values for mode bits. Defaults to + 0644. Directories within the path are not affected by + this setting. This might be in conflict with other options + that affect the file mode, like fsGroup, and the result + can be other mode bits set.' + format: int32 + type: integer + items: + description: items If unspecified, each key-value pair in + the Data field of the referenced Secret will be projected + into the volume as a file whose name is the key and content + is the value. If specified, the listed keys will be projected + into the specified paths, and unlisted keys will not be + present. If a key is specified which is not present in + the Secret, the volume setup will error unless it is marked + optional. Paths must be relative and may not contain the + '..' path or start with '..'. + items: + description: Maps a string key to a path within a volume. + properties: + key: + description: key is the key to project. + type: string + mode: + description: 'mode is Optional: mode bits used to + set permissions on this file. Must be an octal value + between 0000 and 0777 or a decimal value between + 0 and 511. YAML accepts both octal and decimal values, + JSON requires decimal values for mode bits. If not + specified, the volume defaultMode will be used. + This might be in conflict with other options that + affect the file mode, like fsGroup, and the result + can be other mode bits set.' + format: int32 + type: integer + path: + description: path is the relative path of the file + to map the key to. May not be an absolute path. + May not contain the path element '..'. May not start + with the string '..'. + type: string + required: + - key + - path + type: object + type: array + optional: + description: optional field specify whether the Secret or + its keys must be defined + type: boolean + secretName: + description: 'secretName is the name of the secret in the + pod''s namespace to use. More info: https://kubernetes.io/docs/concepts/storage/volumes#secret' + type: string + type: object + storageos: + description: storageOS represents a StorageOS volume attached + and mounted on Kubernetes nodes. + properties: + fsType: + description: fsType is the filesystem type to mount. Must + be a filesystem type supported by the host operating system. + Ex. "ext4", "xfs", "ntfs". Implicitly inferred to be "ext4" + if unspecified. + type: string + readOnly: + description: readOnly defaults to false (read/write). ReadOnly + here will force the ReadOnly setting in VolumeMounts. + type: boolean + secretRef: + description: secretRef specifies the secret to use for obtaining + the StorageOS API credentials. If not specified, default + values will be attempted. + properties: + name: + description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, uid?' + type: string + type: object + volumeName: + description: volumeName is the human-readable name of the + StorageOS volume. Volume names are only unique within + a namespace. + type: string + volumeNamespace: + description: volumeNamespace specifies the scope of the + volume within StorageOS. If no namespace is specified + then the Pod's namespace will be used. This allows the + Kubernetes name scoping to be mirrored within StorageOS + for tighter integration. Set VolumeName to any name to + override the default behaviour. Set to "default" if you + are not using namespaces within StorageOS. Namespaces + that do not pre-exist within StorageOS will be created. + type: string + type: object + vsphereVolume: + description: vsphereVolume represents a vSphere volume attached + and mounted on kubelets host machine + properties: + fsType: + description: fsType is filesystem type to mount. Must be + a filesystem type supported by the host operating system. + Ex. "ext4", "xfs", "ntfs". Implicitly inferred to be "ext4" + if unspecified. + type: string + storagePolicyID: + description: storagePolicyID is the storage Policy Based + Management (SPBM) profile ID associated with the StoragePolicyName. + type: string + storagePolicyName: + description: storagePolicyName is the storage Policy Based + Management (SPBM) profile name. + type: string + volumePath: + description: volumePath is the path that identifies vSphere + volume vmdk + type: string + required: + - volumePath + type: object + required: + - name + type: object + type: array + sidecars: + items: + description: A single application container that you want to run + within a pod. + properties: + args: + description: 'Arguments to the entrypoint. The container image''s + CMD is used if this is not provided. Variable references $(VAR_NAME) + are expanded using the container''s environment. If a variable + cannot be resolved, the reference in the input string will + be unchanged. Double $$ are reduced to a single $, which allows + for escaping the $(VAR_NAME) syntax: i.e. "$$(VAR_NAME)" will + produce the string literal "$(VAR_NAME)". Escaped references + will never be expanded, regardless of whether the variable + exists or not. Cannot be updated. More info: https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell' + items: + type: string + type: array + command: + description: 'Entrypoint array. Not executed within a shell. + The container image''s ENTRYPOINT is used if this is not provided. + Variable references $(VAR_NAME) are expanded using the container''s + environment. If a variable cannot be resolved, the reference + in the input string will be unchanged. Double $$ are reduced + to a single $, which allows for escaping the $(VAR_NAME) syntax: + i.e. "$$(VAR_NAME)" will produce the string literal "$(VAR_NAME)". + Escaped references will never be expanded, regardless of whether + the variable exists or not. Cannot be updated. More info: + https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell' + items: + type: string + type: array + env: + description: List of environment variables to set in the container. + Cannot be updated. + items: + description: EnvVar represents an environment variable present + in a Container. + properties: + name: + description: Name of the environment variable. Must be + a C_IDENTIFIER. + type: string + value: + description: 'Variable references $(VAR_NAME) are expanded + using the previously defined environment variables in + the container and any service environment variables. + If a variable cannot be resolved, the reference in the + input string will be unchanged. Double $$ are reduced + to a single $, which allows for escaping the $(VAR_NAME) + syntax: i.e. "$$(VAR_NAME)" will produce the string + literal "$(VAR_NAME)". Escaped references will never + be expanded, regardless of whether the variable exists + or not. Defaults to "".' + type: string + valueFrom: + description: Source for the environment variable's value. + Cannot be used if value is not empty. + properties: + configMapKeyRef: + description: Selects a key of a ConfigMap. + properties: + key: + description: The key to select. + type: string + name: + description: 'Name of the referent. More info: + https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, + uid?' + type: string + optional: + description: Specify whether the ConfigMap or + its key must be defined + type: boolean + required: + - key + type: object + fieldRef: + description: 'Selects a field of the pod: supports + metadata.name, metadata.namespace, `metadata.labels['''']`, + `metadata.annotations['''']`, spec.nodeName, + spec.serviceAccountName, status.hostIP, status.podIP, + status.podIPs.' + properties: + apiVersion: + description: Version of the schema the FieldPath + is written in terms of, defaults to "v1". + type: string + fieldPath: + description: Path of the field to select in the + specified API version. + type: string + required: + - fieldPath + type: object + resourceFieldRef: + description: 'Selects a resource of the container: + only resources limits and requests (limits.cpu, + limits.memory, limits.ephemeral-storage, requests.cpu, + requests.memory and requests.ephemeral-storage) + are currently supported.' + properties: + containerName: + description: 'Container name: required for volumes, + optional for env vars' + type: string + divisor: + anyOf: + - type: integer + - type: string + description: Specifies the output format of the + exposed resources, defaults to "1" + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + resource: + description: 'Required: resource to select' + type: string + required: + - resource + type: object + secretKeyRef: + description: Selects a key of a secret in the pod's + namespace + properties: + key: + description: The key of the secret to select from. Must + be a valid secret key. + type: string + name: + description: 'Name of the referent. More info: + https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, + uid?' + type: string + optional: + description: Specify whether the Secret or its + key must be defined + type: boolean + required: + - key + type: object + type: object + required: + - name + type: object + type: array + envFrom: + description: List of sources to populate environment variables + in the container. The keys defined within a source must be + a C_IDENTIFIER. All invalid keys will be reported as an event + when the container is starting. When a key exists in multiple + sources, the value associated with the last source will take + precedence. Values defined by an Env with a duplicate key + will take precedence. Cannot be updated. + items: + description: EnvFromSource represents the source of a set + of ConfigMaps + properties: + configMapRef: + description: The ConfigMap to select from + properties: + name: + description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, + uid?' + type: string + optional: + description: Specify whether the ConfigMap must be + defined + type: boolean + type: object + prefix: + description: An optional identifier to prepend to each + key in the ConfigMap. Must be a C_IDENTIFIER. + type: string + secretRef: + description: The Secret to select from + properties: + name: + description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names + TODO: Add other useful fields. apiVersion, kind, + uid?' + type: string + optional: + description: Specify whether the Secret must be defined + type: boolean + type: object + type: object + type: array + image: + description: 'Container image name. More info: https://kubernetes.io/docs/concepts/containers/images + This field is optional to allow higher level config management + to default or override container images in workload controllers + like Deployments and StatefulSets.' + type: string + imagePullPolicy: + description: 'Image pull policy. One of Always, Never, IfNotPresent. + Defaults to Always if :latest tag is specified, or IfNotPresent + otherwise. Cannot be updated. More info: https://kubernetes.io/docs/concepts/containers/images#updating-images' + type: string + lifecycle: + description: Actions that the management system should take + in response to container lifecycle events. Cannot be updated. + properties: + postStart: + description: 'PostStart is called immediately after a container + is created. If the handler fails, the container is terminated + and restarted according to its restart policy. Other management + of the container blocks until the hook completes. More + info: https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/#container-hooks' + properties: + exec: + description: Exec specifies the action to take. + properties: + command: + description: Command is the command line to execute + inside the container, the working directory for + the command is root ('/') in the container's + filesystem. The command is simply exec'd, it is + not run inside a shell, so traditional shell instructions + ('|', etc) won't work. To use a shell, you need + to explicitly call out to that shell. Exit status + of 0 is treated as live/healthy and non-zero is + unhealthy. + items: + type: string + type: array + type: object + httpGet: + description: HTTPGet specifies the http request to perform. + properties: + host: + description: Host name to connect to, defaults to + the pod IP. You probably want to set "Host" in + httpHeaders instead. + type: string + httpHeaders: + description: Custom headers to set in the request. + HTTP allows repeated headers. + items: + description: HTTPHeader describes a custom header + to be used in HTTP probes + properties: + name: + description: The header field name. This will + be canonicalized upon output, so case-variant + names will be understood as the same header. + type: string + value: + description: The header field value + type: string + required: + - name + - value + type: object + type: array + path: + description: Path to access on the HTTP server. + type: string + port: + anyOf: + - type: integer + - type: string + description: Name or number of the port to access + on the container. Number must be in the range + 1 to 65535. Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + scheme: + description: Scheme to use for connecting to the + host. Defaults to HTTP. + type: string + required: + - port + type: object + sleep: + description: Sleep represents the duration that the + container should sleep before being terminated. + properties: + seconds: + description: Seconds is the number of seconds to + sleep. + format: int64 + type: integer + required: + - seconds + type: object + tcpSocket: + description: Deprecated. TCPSocket is NOT supported + as a LifecycleHandler and kept for the backward compatibility. + There are no validation of this field and lifecycle + hooks will fail in runtime when tcp handler is specified. + properties: + host: + description: 'Optional: Host name to connect to, + defaults to the pod IP.' + type: string + port: + anyOf: + - type: integer + - type: string + description: Number or name of the port to access + on the container. Number must be in the range + 1 to 65535. Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + required: + - port + type: object + type: object + preStop: + description: 'PreStop is called immediately before a container + is terminated due to an API request or management event + such as liveness/startup probe failure, preemption, resource + contention, etc. The handler is not called if the container + crashes or exits. The Pod''s termination grace period + countdown begins before the PreStop hook is executed. + Regardless of the outcome of the handler, the container + will eventually terminate within the Pod''s termination + grace period (unless delayed by finalizers). Other management + of the container blocks until the hook completes or until + the termination grace period is reached. More info: https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/#container-hooks' + properties: + exec: + description: Exec specifies the action to take. + properties: + command: + description: Command is the command line to execute + inside the container, the working directory for + the command is root ('/') in the container's + filesystem. The command is simply exec'd, it is + not run inside a shell, so traditional shell instructions + ('|', etc) won't work. To use a shell, you need + to explicitly call out to that shell. Exit status + of 0 is treated as live/healthy and non-zero is + unhealthy. + items: + type: string + type: array + type: object + httpGet: + description: HTTPGet specifies the http request to perform. + properties: + host: + description: Host name to connect to, defaults to + the pod IP. You probably want to set "Host" in + httpHeaders instead. + type: string + httpHeaders: + description: Custom headers to set in the request. + HTTP allows repeated headers. + items: + description: HTTPHeader describes a custom header + to be used in HTTP probes + properties: + name: + description: The header field name. This will + be canonicalized upon output, so case-variant + names will be understood as the same header. + type: string + value: + description: The header field value + type: string + required: + - name + - value + type: object + type: array + path: + description: Path to access on the HTTP server. + type: string + port: + anyOf: + - type: integer + - type: string + description: Name or number of the port to access + on the container. Number must be in the range + 1 to 65535. Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + scheme: + description: Scheme to use for connecting to the + host. Defaults to HTTP. + type: string + required: + - port + type: object + sleep: + description: Sleep represents the duration that the + container should sleep before being terminated. + properties: + seconds: + description: Seconds is the number of seconds to + sleep. + format: int64 + type: integer + required: + - seconds + type: object + tcpSocket: + description: Deprecated. TCPSocket is NOT supported + as a LifecycleHandler and kept for the backward compatibility. + There are no validation of this field and lifecycle + hooks will fail in runtime when tcp handler is specified. + properties: + host: + description: 'Optional: Host name to connect to, + defaults to the pod IP.' + type: string + port: + anyOf: + - type: integer + - type: string + description: Number or name of the port to access + on the container. Number must be in the range + 1 to 65535. Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + required: + - port + type: object + type: object + type: object + livenessProbe: + description: 'Periodic probe of container liveness. Container + will be restarted if the probe fails. Cannot be updated. More + info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes' + properties: + exec: + description: Exec specifies the action to take. + properties: + command: + description: Command is the command line to execute + inside the container, the working directory for the + command is root ('/') in the container's filesystem. + The command is simply exec'd, it is not run inside + a shell, so traditional shell instructions ('|', etc) + won't work. To use a shell, you need to explicitly + call out to that shell. Exit status of 0 is treated + as live/healthy and non-zero is unhealthy. + items: + type: string + type: array + type: object + failureThreshold: + description: Minimum consecutive failures for the probe + to be considered failed after having succeeded. Defaults + to 3. Minimum value is 1. + format: int32 + type: integer + grpc: + description: GRPC specifies an action involving a GRPC port. + properties: + port: + description: Port number of the gRPC service. Number + must be in the range 1 to 65535. + format: int32 + type: integer + service: + description: "Service is the name of the service to + place in the gRPC HealthCheckRequest (see https://github.com/grpc/grpc/blob/master/doc/health-checking.md). + \n If this is not specified, the default behavior + is defined by gRPC." + type: string + required: + - port + type: object + httpGet: + description: HTTPGet specifies the http request to perform. + properties: + host: + description: Host name to connect to, defaults to the + pod IP. You probably want to set "Host" in httpHeaders + instead. + type: string + httpHeaders: + description: Custom headers to set in the request. HTTP + allows repeated headers. + items: + description: HTTPHeader describes a custom header + to be used in HTTP probes + properties: + name: + description: The header field name. This will + be canonicalized upon output, so case-variant + names will be understood as the same header. + type: string + value: + description: The header field value + type: string + required: + - name + - value + type: object + type: array + path: + description: Path to access on the HTTP server. + type: string + port: + anyOf: + - type: integer + - type: string + description: Name or number of the port to access on + the container. Number must be in the range 1 to 65535. + Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + scheme: + description: Scheme to use for connecting to the host. + Defaults to HTTP. + type: string + required: + - port + type: object + initialDelaySeconds: + description: 'Number of seconds after the container has + started before liveness probes are initiated. More info: + https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes' + format: int32 + type: integer + periodSeconds: + description: How often (in seconds) to perform the probe. + Default to 10 seconds. Minimum value is 1. + format: int32 + type: integer + successThreshold: + description: Minimum consecutive successes for the probe + to be considered successful after having failed. Defaults + to 1. Must be 1 for liveness and startup. Minimum value + is 1. + format: int32 + type: integer + tcpSocket: + description: TCPSocket specifies an action involving a TCP + port. + properties: + host: + description: 'Optional: Host name to connect to, defaults + to the pod IP.' + type: string + port: + anyOf: + - type: integer + - type: string + description: Number or name of the port to access on + the container. Number must be in the range 1 to 65535. + Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + required: + - port + type: object + terminationGracePeriodSeconds: + description: Optional duration in seconds the pod needs + to terminate gracefully upon probe failure. The grace + period is the duration in seconds after the processes + running in the pod are sent a termination signal and the + time when the processes are forcibly halted with a kill + signal. Set this value longer than the expected cleanup + time for your process. If this value is nil, the pod's + terminationGracePeriodSeconds will be used. Otherwise, + this value overrides the value provided by the pod spec. + Value must be non-negative integer. The value zero indicates + stop immediately via the kill signal (no opportunity to + shut down). This is a beta field and requires enabling + ProbeTerminationGracePeriod feature gate. Minimum value + is 1. spec.terminationGracePeriodSeconds is used if unset. + format: int64 + type: integer + timeoutSeconds: + description: 'Number of seconds after which the probe times + out. Defaults to 1 second. Minimum value is 1. More info: + https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes' + format: int32 + type: integer + type: object + name: + description: Name of the container specified as a DNS_LABEL. + Each container in a pod must have a unique name (DNS_LABEL). + Cannot be updated. + type: string + ports: + description: List of ports to expose from the container. Not + specifying a port here DOES NOT prevent that port from being + exposed. Any port which is listening on the default "0.0.0.0" + address inside a container will be accessible from the network. + Modifying this array with strategic merge patch may corrupt + the data. For more information See https://github.com/kubernetes/kubernetes/issues/108255. + Cannot be updated. + items: + description: ContainerPort represents a network port in a + single container. + properties: + containerPort: + description: Number of port to expose on the pod's IP + address. This must be a valid port number, 0 < x < 65536. + format: int32 + type: integer + hostIP: + description: What host IP to bind the external port to. + type: string + hostPort: + description: Number of port to expose on the host. If + specified, this must be a valid port number, 0 < x < + 65536. If HostNetwork is specified, this must match + ContainerPort. Most containers do not need this. + format: int32 + type: integer + name: + description: If specified, this must be an IANA_SVC_NAME + and unique within the pod. Each named port in a pod + must have a unique name. Name for the port that can + be referred to by services. + type: string + protocol: + default: TCP + description: Protocol for port. Must be UDP, TCP, or SCTP. + Defaults to "TCP". + type: string + required: + - containerPort + type: object + type: array + x-kubernetes-list-map-keys: + - containerPort + - protocol + x-kubernetes-list-type: map + readinessProbe: + description: 'Periodic probe of container service readiness. + Container will be removed from service endpoints if the probe + fails. Cannot be updated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes' + properties: + exec: + description: Exec specifies the action to take. + properties: + command: + description: Command is the command line to execute + inside the container, the working directory for the + command is root ('/') in the container's filesystem. + The command is simply exec'd, it is not run inside + a shell, so traditional shell instructions ('|', etc) + won't work. To use a shell, you need to explicitly + call out to that shell. Exit status of 0 is treated + as live/healthy and non-zero is unhealthy. + items: + type: string + type: array + type: object + failureThreshold: + description: Minimum consecutive failures for the probe + to be considered failed after having succeeded. Defaults + to 3. Minimum value is 1. + format: int32 + type: integer + grpc: + description: GRPC specifies an action involving a GRPC port. + properties: + port: + description: Port number of the gRPC service. Number + must be in the range 1 to 65535. + format: int32 + type: integer + service: + description: "Service is the name of the service to + place in the gRPC HealthCheckRequest (see https://github.com/grpc/grpc/blob/master/doc/health-checking.md). + \n If this is not specified, the default behavior + is defined by gRPC." + type: string + required: + - port + type: object + httpGet: + description: HTTPGet specifies the http request to perform. + properties: + host: + description: Host name to connect to, defaults to the + pod IP. You probably want to set "Host" in httpHeaders + instead. + type: string + httpHeaders: + description: Custom headers to set in the request. HTTP + allows repeated headers. + items: + description: HTTPHeader describes a custom header + to be used in HTTP probes + properties: + name: + description: The header field name. This will + be canonicalized upon output, so case-variant + names will be understood as the same header. + type: string + value: + description: The header field value + type: string + required: + - name + - value + type: object + type: array + path: + description: Path to access on the HTTP server. + type: string + port: + anyOf: + - type: integer + - type: string + description: Name or number of the port to access on + the container. Number must be in the range 1 to 65535. + Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + scheme: + description: Scheme to use for connecting to the host. + Defaults to HTTP. + type: string + required: + - port + type: object + initialDelaySeconds: + description: 'Number of seconds after the container has + started before liveness probes are initiated. More info: + https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes' + format: int32 + type: integer + periodSeconds: + description: How often (in seconds) to perform the probe. + Default to 10 seconds. Minimum value is 1. + format: int32 + type: integer + successThreshold: + description: Minimum consecutive successes for the probe + to be considered successful after having failed. Defaults + to 1. Must be 1 for liveness and startup. Minimum value + is 1. + format: int32 + type: integer + tcpSocket: + description: TCPSocket specifies an action involving a TCP + port. + properties: + host: + description: 'Optional: Host name to connect to, defaults + to the pod IP.' + type: string + port: + anyOf: + - type: integer + - type: string + description: Number or name of the port to access on + the container. Number must be in the range 1 to 65535. + Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + required: + - port + type: object + terminationGracePeriodSeconds: + description: Optional duration in seconds the pod needs + to terminate gracefully upon probe failure. The grace + period is the duration in seconds after the processes + running in the pod are sent a termination signal and the + time when the processes are forcibly halted with a kill + signal. Set this value longer than the expected cleanup + time for your process. If this value is nil, the pod's + terminationGracePeriodSeconds will be used. Otherwise, + this value overrides the value provided by the pod spec. + Value must be non-negative integer. The value zero indicates + stop immediately via the kill signal (no opportunity to + shut down). This is a beta field and requires enabling + ProbeTerminationGracePeriod feature gate. Minimum value + is 1. spec.terminationGracePeriodSeconds is used if unset. + format: int64 + type: integer + timeoutSeconds: + description: 'Number of seconds after which the probe times + out. Defaults to 1 second. Minimum value is 1. More info: + https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes' + format: int32 + type: integer + type: object + resizePolicy: + description: Resources resize policy for the container. + items: + description: ContainerResizePolicy represents resource resize + policy for the container. + properties: + resourceName: + description: 'Name of the resource to which this resource + resize policy applies. Supported values: cpu, memory.' + type: string + restartPolicy: + description: Restart policy to apply when specified resource + is resized. If not specified, it defaults to NotRequired. + type: string + required: + - resourceName + - restartPolicy + type: object + type: array + x-kubernetes-list-type: atomic + resources: + description: 'Compute Resources required by this container. + Cannot be updated. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + properties: + claims: + description: "Claims lists the names of resources, defined + in spec.resourceClaims, that are used by this container. + \n This is an alpha field and requires enabling the DynamicResourceAllocation + feature gate. \n This field is immutable. It can only + be set for containers." + items: + description: ResourceClaim references one entry in PodSpec.ResourceClaims. + properties: + name: + description: Name must match the name of one entry + in pod.spec.resourceClaims of the Pod where this + field is used. It makes that resource available + inside a container. + type: string + required: + - name + type: object + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + limits: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + description: 'Limits describes the maximum amount of compute + resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + type: object + requests: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + description: 'Requests describes the minimum amount of compute + resources required. If Requests is omitted for a container, + it defaults to Limits if that is explicitly specified, + otherwise to an implementation-defined value. Requests + cannot exceed Limits. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' + type: object + type: object + restartPolicy: + description: 'RestartPolicy defines the restart behavior of + individual containers in a pod. This field may only be set + for init containers, and the only allowed value is "Always". + For non-init containers or when this field is not specified, + the restart behavior is defined by the Pod''s restart policy + and the container type. Setting the RestartPolicy as "Always" + for the init container will have the following effect: this + init container will be continually restarted on exit until + all regular containers have terminated. Once all regular containers + have completed, all init containers with restartPolicy "Always" + will be shut down. This lifecycle differs from normal init + containers and is often referred to as a "sidecar" container. + Although this init container still starts in the init container + sequence, it does not wait for the container to complete before + proceeding to the next init container. Instead, the next init + container starts immediately after this init container is + started, or after any startupProbe has successfully completed.' + type: string + securityContext: + description: 'SecurityContext defines the security options the + container should be run with. If set, the fields of SecurityContext + override the equivalent fields of PodSecurityContext. More + info: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/' + properties: + allowPrivilegeEscalation: + description: 'AllowPrivilegeEscalation controls whether + a process can gain more privileges than its parent process. + This bool directly controls if the no_new_privs flag will + be set on the container process. AllowPrivilegeEscalation + is true always when the container is: 1) run as Privileged + 2) has CAP_SYS_ADMIN Note that this field cannot be set + when spec.os.name is windows.' + type: boolean + capabilities: + description: The capabilities to add/drop when running containers. + Defaults to the default set of capabilities granted by + the container runtime. Note that this field cannot be + set when spec.os.name is windows. + properties: + add: + description: Added capabilities + items: + description: Capability represent POSIX capabilities + type + type: string + type: array + drop: + description: Removed capabilities + items: + description: Capability represent POSIX capabilities + type + type: string + type: array + type: object + privileged: + description: Run container in privileged mode. Processes + in privileged containers are essentially equivalent to + root on the host. Defaults to false. Note that this field + cannot be set when spec.os.name is windows. + type: boolean + procMount: + description: procMount denotes the type of proc mount to + use for the containers. The default is DefaultProcMount + which uses the container runtime defaults for readonly + paths and masked paths. This requires the ProcMountType + feature flag to be enabled. Note that this field cannot + be set when spec.os.name is windows. + type: string + readOnlyRootFilesystem: + description: Whether this container has a read-only root + filesystem. Default is false. Note that this field cannot + be set when spec.os.name is windows. + type: boolean + runAsGroup: + description: The GID to run the entrypoint of the container + process. Uses runtime default if unset. May also be set + in PodSecurityContext. If set in both SecurityContext + and PodSecurityContext, the value specified in SecurityContext + takes precedence. Note that this field cannot be set when + spec.os.name is windows. + format: int64 + type: integer + runAsNonRoot: + description: Indicates that the container must run as a + non-root user. If true, the Kubelet will validate the + image at runtime to ensure that it does not run as UID + 0 (root) and fail to start the container if it does. If + unset or false, no such validation will be performed. + May also be set in PodSecurityContext. If set in both + SecurityContext and PodSecurityContext, the value specified + in SecurityContext takes precedence. + type: boolean + runAsUser: + description: The UID to run the entrypoint of the container + process. Defaults to user specified in image metadata + if unspecified. May also be set in PodSecurityContext. If + set in both SecurityContext and PodSecurityContext, the + value specified in SecurityContext takes precedence. Note + that this field cannot be set when spec.os.name is windows. + format: int64 + type: integer + seLinuxOptions: + description: The SELinux context to be applied to the container. + If unspecified, the container runtime will allocate a + random SELinux context for each container. May also be + set in PodSecurityContext. If set in both SecurityContext + and PodSecurityContext, the value specified in SecurityContext + takes precedence. Note that this field cannot be set when + spec.os.name is windows. + properties: + level: + description: Level is SELinux level label that applies + to the container. + type: string + role: + description: Role is a SELinux role label that applies + to the container. + type: string + type: + description: Type is a SELinux type label that applies + to the container. + type: string + user: + description: User is a SELinux user label that applies + to the container. + type: string + type: object + seccompProfile: + description: The seccomp options to use by this container. + If seccomp options are provided at both the pod & container + level, the container options override the pod options. + Note that this field cannot be set when spec.os.name is + windows. + properties: + localhostProfile: + description: localhostProfile indicates a profile defined + in a file on the node should be used. The profile + must be preconfigured on the node to work. Must be + a descending path, relative to the kubelet's configured + seccomp profile location. Must be set if type is "Localhost". + Must NOT be set for any other type. + type: string + type: + description: "type indicates which kind of seccomp profile + will be applied. Valid options are: \n Localhost - + a profile defined in a file on the node should be + used. RuntimeDefault - the container runtime default + profile should be used. Unconfined - no profile should + be applied." + type: string + required: + - type + type: object + windowsOptions: + description: The Windows specific settings applied to all + containers. If unspecified, the options from the PodSecurityContext + will be used. If set in both SecurityContext and PodSecurityContext, + the value specified in SecurityContext takes precedence. + Note that this field cannot be set when spec.os.name is + linux. + properties: + gmsaCredentialSpec: + description: GMSACredentialSpec is where the GMSA admission + webhook (https://github.com/kubernetes-sigs/windows-gmsa) + inlines the contents of the GMSA credential spec named + by the GMSACredentialSpecName field. + type: string + gmsaCredentialSpecName: + description: GMSACredentialSpecName is the name of the + GMSA credential spec to use. + type: string + hostProcess: + description: HostProcess determines if a container should + be run as a 'Host Process' container. All of a Pod's + containers must have the same effective HostProcess + value (it is not allowed to have a mix of HostProcess + containers and non-HostProcess containers). In addition, + if HostProcess is true then HostNetwork must also + be set to true. + type: boolean + runAsUserName: + description: The UserName in Windows to run the entrypoint + of the container process. Defaults to the user specified + in image metadata if unspecified. May also be set + in PodSecurityContext. If set in both SecurityContext + and PodSecurityContext, the value specified in SecurityContext + takes precedence. + type: string + type: object + type: object + startupProbe: + description: 'StartupProbe indicates that the Pod has successfully + initialized. If specified, no other probes are executed until + this completes successfully. If this probe fails, the Pod + will be restarted, just as if the livenessProbe failed. This + can be used to provide different probe parameters at the beginning + of a Pod''s lifecycle, when it might take a long time to load + data or warm a cache, than during steady-state operation. + This cannot be updated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes' + properties: + exec: + description: Exec specifies the action to take. + properties: + command: + description: Command is the command line to execute + inside the container, the working directory for the + command is root ('/') in the container's filesystem. + The command is simply exec'd, it is not run inside + a shell, so traditional shell instructions ('|', etc) + won't work. To use a shell, you need to explicitly + call out to that shell. Exit status of 0 is treated + as live/healthy and non-zero is unhealthy. + items: + type: string + type: array + type: object + failureThreshold: + description: Minimum consecutive failures for the probe + to be considered failed after having succeeded. Defaults + to 3. Minimum value is 1. + format: int32 + type: integer + grpc: + description: GRPC specifies an action involving a GRPC port. + properties: + port: + description: Port number of the gRPC service. Number + must be in the range 1 to 65535. + format: int32 + type: integer + service: + description: "Service is the name of the service to + place in the gRPC HealthCheckRequest (see https://github.com/grpc/grpc/blob/master/doc/health-checking.md). + \n If this is not specified, the default behavior + is defined by gRPC." + type: string + required: + - port + type: object + httpGet: + description: HTTPGet specifies the http request to perform. + properties: + host: + description: Host name to connect to, defaults to the + pod IP. You probably want to set "Host" in httpHeaders + instead. + type: string + httpHeaders: + description: Custom headers to set in the request. HTTP + allows repeated headers. + items: + description: HTTPHeader describes a custom header + to be used in HTTP probes + properties: + name: + description: The header field name. This will + be canonicalized upon output, so case-variant + names will be understood as the same header. + type: string + value: + description: The header field value + type: string + required: + - name + - value + type: object + type: array + path: + description: Path to access on the HTTP server. + type: string + port: + anyOf: + - type: integer + - type: string + description: Name or number of the port to access on + the container. Number must be in the range 1 to 65535. + Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + scheme: + description: Scheme to use for connecting to the host. + Defaults to HTTP. + type: string + required: + - port + type: object + initialDelaySeconds: + description: 'Number of seconds after the container has + started before liveness probes are initiated. More info: + https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes' + format: int32 + type: integer + periodSeconds: + description: How often (in seconds) to perform the probe. + Default to 10 seconds. Minimum value is 1. + format: int32 + type: integer + successThreshold: + description: Minimum consecutive successes for the probe + to be considered successful after having failed. Defaults + to 1. Must be 1 for liveness and startup. Minimum value + is 1. + format: int32 + type: integer + tcpSocket: + description: TCPSocket specifies an action involving a TCP + port. + properties: + host: + description: 'Optional: Host name to connect to, defaults + to the pod IP.' + type: string + port: + anyOf: + - type: integer + - type: string + description: Number or name of the port to access on + the container. Number must be in the range 1 to 65535. + Name must be an IANA_SVC_NAME. + x-kubernetes-int-or-string: true + required: + - port + type: object + terminationGracePeriodSeconds: + description: Optional duration in seconds the pod needs + to terminate gracefully upon probe failure. The grace + period is the duration in seconds after the processes + running in the pod are sent a termination signal and the + time when the processes are forcibly halted with a kill + signal. Set this value longer than the expected cleanup + time for your process. If this value is nil, the pod's + terminationGracePeriodSeconds will be used. Otherwise, + this value overrides the value provided by the pod spec. + Value must be non-negative integer. The value zero indicates + stop immediately via the kill signal (no opportunity to + shut down). This is a beta field and requires enabling + ProbeTerminationGracePeriod feature gate. Minimum value + is 1. spec.terminationGracePeriodSeconds is used if unset. + format: int64 + type: integer + timeoutSeconds: + description: 'Number of seconds after which the probe times + out. Defaults to 1 second. Minimum value is 1. More info: + https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes' + format: int32 + type: integer + type: object + stdin: + description: Whether this container should allocate a buffer + for stdin in the container runtime. If this is not set, reads + from stdin in the container will always result in EOF. Default + is false. + type: boolean + stdinOnce: + description: Whether the container runtime should close the + stdin channel after it has been opened by a single attach. + When stdin is true the stdin stream will remain open across + multiple attach sessions. If stdinOnce is set to true, stdin + is opened on container start, is empty until the first client + attaches to stdin, and then remains open and accepts data + until the client disconnects, at which time stdin is closed + and remains closed until the container is restarted. If this + flag is false, a container processes that reads from stdin + will never receive an EOF. Default is false + type: boolean + terminationMessagePath: + description: 'Optional: Path at which the file to which the + container''s termination message will be written is mounted + into the container''s filesystem. Message written is intended + to be brief final status, such as an assertion failure message. + Will be truncated by the node if greater than 4096 bytes. + The total message length across all containers will be limited + to 12kb. Defaults to /dev/termination-log. Cannot be updated.' + type: string + terminationMessagePolicy: + description: Indicate how the termination message should be + populated. File will use the contents of terminationMessagePath + to populate the container status message on both success and + failure. FallbackToLogsOnError will use the last chunk of + container log output if the termination message file is empty + and the container exited with an error. The log output is + limited to 2048 bytes or 80 lines, whichever is smaller. Defaults + to File. Cannot be updated. + type: string + tty: + description: Whether this container should allocate a TTY for + itself, also requires 'stdin' to be true. Default is false. + type: boolean + volumeDevices: + description: volumeDevices is the list of block devices to be + used by the container. + items: + description: volumeDevice describes a mapping of a raw block + device within a container. + properties: + devicePath: + description: devicePath is the path inside of the container + that the device will be mapped to. + type: string + name: + description: name must match the name of a persistentVolumeClaim + in the pod + type: string + required: + - devicePath + - name + type: object + type: array + volumeMounts: + description: Pod volumes to mount into the container's filesystem. + Cannot be updated. + items: + description: VolumeMount describes a mounting of a Volume + within a container. + properties: + mountPath: + description: Path within the container at which the volume + should be mounted. Must not contain ':'. + type: string + mountPropagation: + description: mountPropagation determines how mounts are + propagated from the host to container and the other + way around. When not set, MountPropagationNone is used. + This field is beta in 1.10. + type: string + name: + description: This must match the Name of a Volume. + type: string + readOnly: + description: Mounted read-only if true, read-write otherwise + (false or unspecified). Defaults to false. + type: boolean + subPath: + description: Path within the volume from which the container's + volume should be mounted. Defaults to "" (volume's root). + type: string + subPathExpr: + description: Expanded path within the volume from which + the container's volume should be mounted. Behaves similarly + to SubPath but environment variable references $(VAR_NAME) + are expanded using the container's environment. Defaults + to "" (volume's root). SubPathExpr and SubPath are mutually + exclusive. + type: string + required: + - mountPath + - name + type: object + type: array + workingDir: + description: Container's working directory. If not specified, + the container runtime's default will be used, which might + be configured in the container image. Cannot be updated. + type: string + required: + - name + type: object + type: array type: object status: description: DatabaseObserverStatus defines the observed state of DatabaseObserver properties: conditions: - description: 'INSERT ADDITIONAL STATUS FIELD - define observed state - of cluster Important: Run "make" to regenerate code after modifying - this file' items: description: "Condition contains details for one aspect of the current state of this API Resource. --- This struct is intended for direct @@ -210,9 +9519,12 @@ spec: type: integer status: type: string + version: + type: string required: - conditions - exporterConfig + - version type: object type: object served: true diff --git a/config/rbac/lrest_editor_role.yaml b/config/rbac/lrest_editor_role.yaml new file mode 100644 index 00000000..7f5b2f01 --- /dev/null +++ b/config/rbac/lrest_editor_role.yaml @@ -0,0 +1,24 @@ +# permissions for end users to edit lrests. +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: lrest-editor-role +rules: +- apiGroups: + - database.oracle.com + resources: + - lrests + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - database.oracle.com + resources: + - lrests/status + verbs: + - get diff --git a/config/rbac/lrest_viewer_role.yaml b/config/rbac/lrest_viewer_role.yaml new file mode 100644 index 00000000..d74bc977 --- /dev/null +++ b/config/rbac/lrest_viewer_role.yaml @@ -0,0 +1,20 @@ +# permissions for end users to view lrests. +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: lrest-viewer-role +rules: +- apiGroups: + - database.oracle.com + resources: + - lrests + verbs: + - get + - list + - watch +- apiGroups: + - database.oracle.com + resources: + - lrests/status + verbs: + - get diff --git a/config/rbac/lrpdb_editor_role.yaml b/config/rbac/lrpdb_editor_role.yaml new file mode 100644 index 00000000..20ae714a --- /dev/null +++ b/config/rbac/lrpdb_editor_role.yaml @@ -0,0 +1,24 @@ +# permissions for end users to edit lrpdbs. +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: lrpdb-editor-role +rules: +- apiGroups: + - database.oracle.com + resources: + - lrpdbs + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - database.oracle.com + resources: + - lrpdbs/status + verbs: + - get diff --git a/config/rbac/lrpdb_viewer_role.yaml b/config/rbac/lrpdb_viewer_role.yaml new file mode 100644 index 00000000..95bcaab5 --- /dev/null +++ b/config/rbac/lrpdb_viewer_role.yaml @@ -0,0 +1,20 @@ +# permissions for end users to view lrpdbs. +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: lrpdb-viewer-role +rules: +- apiGroups: + - database.oracle.com + resources: + - lrpdbs + verbs: + - get + - list + - watch +- apiGroups: + - database.oracle.com + resources: + - lrpdbs/status + verbs: + - get diff --git a/config/rbac/ordssrvs_editor_role.yaml b/config/rbac/ordssrvs_editor_role.yaml new file mode 100644 index 00000000..bc4170f6 --- /dev/null +++ b/config/rbac/ordssrvs_editor_role.yaml @@ -0,0 +1,24 @@ +# permissions for end users to edit ordssrvs. +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: ordssrvs-editor-role +rules: +- apiGroups: + - database.oracle.com + resources: + - ordssrvs + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - database.oracle.com + resources: + - ordssrvs/status + verbs: + - get diff --git a/config/rbac/ordssrvs_viewer_role.yaml b/config/rbac/ordssrvs_viewer_role.yaml new file mode 100644 index 00000000..8880c17d --- /dev/null +++ b/config/rbac/ordssrvs_viewer_role.yaml @@ -0,0 +1,20 @@ +# permissions for end users to view ordssrvs. +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: ordssrvs-viewer-role +rules: +- apiGroups: + - database.oracle.com + resources: + - ordssrvs + verbs: + - get + - list + - watch +- apiGroups: + - database.oracle.com + resources: + - ordssrvs/status + verbs: + - get diff --git a/config/rbac/role.yaml b/config/rbac/role.yaml index 2f33c915..3a12386c 100644 --- a/config/rbac/role.yaml +++ b/config/rbac/role.yaml @@ -1,48 +1,24 @@ - --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: - creationTimestamp: null name: manager-role rules: - apiGroups: - "" resources: - configmaps + - containers + - deployments - events + - namespaces + - persistentvolumeclaims - pods - pods/exec - pods/log - replicasets - - services - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - "" - resources: - - configmaps - - namespaces - - secrets - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - "" - resources: - - configmaps - secrets + - services verbs: - create - delete @@ -54,49 +30,29 @@ rules: - apiGroups: - "" resources: - - deployments - - events - - pods - - services + - configmaps/status + - daemonsets/status + - deployments/status + - services/status + - statefulsets/status verbs: - - create - - delete - get - - list - patch - update - - watch -- apiGroups: - - "" - resources: - - events - verbs: - - create - - patch - apiGroups: - "" resources: - - persistentvolumeclaims - - pods - - pods/exec - - pods/log - - services + - persistentvolumes verbs: - - create - - delete - get - list - - patch - - update - watch - apiGroups: - "" resources: - - persistentvolumes + - secrets/status verbs: - get - - list - - watch - apiGroups: - '''''' resources: @@ -119,32 +75,11 @@ rules: - apiGroups: - apps resources: + - daemonsets - deployments - pods - - services - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - apps - resources: - replicasets - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - apps - resources: + - services - statefulsets verbs: - create @@ -163,73 +98,22 @@ rules: - get - list - update -- apiGroups: - - "" - resources: - - configmaps - - containers - - events - - namespaces - - persistentvolumeclaims - - pods - - pods/exec - - pods/log - - secrets - - services - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - "" - resources: - - configmaps - - containers - - events - - namespaces - - pods - - pods/exec - - pods/log - - secrets - - services - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - "" - resources: - - configmaps - - namespaces - - pods - - secrets - - services - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - "" - resources: - - pods/exec - verbs: - - create - apiGroups: - database.oracle.com resources: - autonomouscontainerdatabases + - autonomousdatabases + - cdbs + - dataguardbrokers + - dbcssystems + - events + - lrests + - lrpdbs + - oraclerestdataservices + - ordssrvs + - pdbs + - shardingdatabases + - singleinstancedatabases verbs: - create - delete @@ -242,44 +126,18 @@ rules: - database.oracle.com resources: - autonomouscontainerdatabases/status - verbs: - - get - - patch - - update -- apiGroups: - - database.oracle.com - resources: - - autonomousdatabasebackups - verbs: - - create - - delete - - get - - list - - update - - watch -- apiGroups: - - database.oracle.com - resources: - autonomousdatabasebackups/status - verbs: - - get - - patch - - update -- apiGroups: - - database.oracle.com - resources: - - autonomousdatabaserestores - verbs: - - create - - delete - - get - - list - - update - - watch -- apiGroups: - - database.oracle.com - resources: - autonomousdatabaserestores/status + - cdbs/status + - dataguardbrokers/status + - dbcssystems/status + - lrests/status + - lrpdbs/status + - oraclerestdataservices/status + - ordssrvs/status + - pdbs/status + - shardingdatabases/status + - singleinstancedatabases/status verbs: - get - patch @@ -287,13 +145,13 @@ rules: - apiGroups: - database.oracle.com resources: - - autonomousdatabases + - autonomousdatabasebackups + - autonomousdatabaserestores verbs: - create - delete - get - list - - patch - update - watch - apiGroups: @@ -303,159 +161,23 @@ rules: verbs: - patch - update -- apiGroups: - - database.oracle.com - resources: - - cdbs - verbs: - - create - - delete - - get - - list - - patch - - update - - watch - apiGroups: - database.oracle.com resources: - cdbs/finalizers - verbs: - - update -- apiGroups: - - database.oracle.com - resources: - - cdbs/status - verbs: - - get - - patch - - update -- apiGroups: - - database.oracle.com - resources: - - dataguardbrokers - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - database.oracle.com - resources: - dataguardbrokers/finalizers - verbs: - - update -- apiGroups: - - database.oracle.com - resources: - - dataguardbrokers/status - verbs: - - get - - patch - - update -- apiGroups: - - database.oracle.com - resources: - - dbcssystems - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - database.oracle.com - resources: - - dbcssystems/finalizers - verbs: - - create - - delete - - get - - patch - - update -- apiGroups: - - database.oracle.com - resources: - - dbcssystems/status - verbs: - - get - - patch - - update -- apiGroups: - - database.oracle.com - resources: - - oraclerestdataservices - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - database.oracle.com - resources: + - lrests/finalizers - oraclerestdataservices/finalizers + - ordssrvs/finalizers + - singleinstancedatabases/finalizers verbs: - update - apiGroups: - database.oracle.com resources: - - oraclerestdataservices/status - verbs: - - get - - patch - - update -- apiGroups: - - database.oracle.com - resources: - - pdbs - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - database.oracle.com - resources: + - dbcssystems/finalizers + - lrpdbs/finalizers - pdbs/finalizers - verbs: - - create - - delete - - get - - patch - - update -- apiGroups: - - database.oracle.com - resources: - - pdbs/status - verbs: - - get - - patch - - update -- apiGroups: - - database.oracle.com - resources: - - shardingdatabases - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - database.oracle.com - resources: - shardingdatabases/finalizers verbs: - create @@ -463,40 +185,6 @@ rules: - get - patch - update -- apiGroups: - - database.oracle.com - resources: - - shardingdatabases/status - verbs: - - get - - patch - - update -- apiGroups: - - database.oracle.com - resources: - - singleinstancedatabases - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - database.oracle.com - resources: - - singleinstancedatabases/finalizers - verbs: - - update -- apiGroups: - - database.oracle.com - resources: - - singleinstancedatabases/status - verbs: - - get - - patch - - update - apiGroups: - monitoring.coreos.com resources: diff --git a/config/samples/adb/autonomousdatabase_bind.yaml b/config/samples/adb/autonomousdatabase_bind.yaml index 8d1de0fe..702b8f03 100644 --- a/config/samples/adb/autonomousdatabase_bind.yaml +++ b/config/samples/adb/autonomousdatabase_bind.yaml @@ -7,8 +7,9 @@ kind: AutonomousDatabase metadata: name: autonomousdatabase-sample spec: + action: Sync details: - autonomousDatabaseOCID: ocid1.autonomousdatabase... + id: ocid1.autonomousdatabase... # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. ociConfig: configMapName: oci-cred diff --git a/config/samples/adb/autonomousdatabase_clone.yaml b/config/samples/adb/autonomousdatabase_clone.yaml new file mode 100644 index 00000000..559d7185 --- /dev/null +++ b/config/samples/adb/autonomousdatabase_clone.yaml @@ -0,0 +1,35 @@ +# +# Copyright (c) 2022, 2024, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +apiVersion: database.oracle.com/v1alpha1 +kind: AutonomousDatabase +metadata: + name: autonomousdatabase-sample +spec: + action: Clone + details: + id: ocid1.autonomousdatabase... + clone: + # Update compartmentOCID with your compartment OCID. + compartmentId: ocid1.compartment... OR ocid1.tenancy... + # The dbName must begin with an alphabetic character and can contain a maximum of 14 alphanumeric characters. Special characters are not permitted. The database name must be unique in the tenancy. + dbName: ClonedADB + displayName: ClonedADB + cpuCoreCount: 1 + adminPassword: + # Comment out k8sSecret and uncomment ociSecret if you pass the admin password using OCI Secret. + k8sSecret: + # The Name of the K8s secret where you want to hold the password of the ADMIN account. + name: admin-password + # ociSecret: + # # The OCID of the OCI Secret that holds the password of the ADMIN account. It should start with ocid1.vaultsecret... . + # id: ocid1.vaultsecret... + dataStorageSizeInTBs: 1 + dbWorkload: OLTP + cloneType: METADATA + # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. + ociConfig: + configMapName: oci-cred + # Comment out secretName if using OKE workload identity + secretName: oci-privatekey \ No newline at end of file diff --git a/config/samples/adb/autonomousdatabase_create.yaml b/config/samples/adb/autonomousdatabase_create.yaml index aa77a94c..d633cb84 100644 --- a/config/samples/adb/autonomousdatabase_create.yaml +++ b/config/samples/adb/autonomousdatabase_create.yaml @@ -7,9 +7,10 @@ kind: AutonomousDatabase metadata: name: autonomousdatabase-sample spec: + action: Create details: # Update compartmentOCID with your compartment OCID. - compartmentOCID: ocid1.compartment... OR ocid1.tenancy... + compartmentId: ocid1.compartment... OR ocid1.tenancy... # The dbName must begin with an alphabetic character and can contain a maximum of 14 alphanumeric characters. Special characters are not permitted. The database name must be unique in the tenancy. dbName: NewADB displayName: NewADB diff --git a/config/samples/adb/autonomousdatabase_delete_resource.yaml b/config/samples/adb/autonomousdatabase_delete_resource.yaml index 60a8fe5c..bae1f605 100644 --- a/config/samples/adb/autonomousdatabase_delete_resource.yaml +++ b/config/samples/adb/autonomousdatabase_delete_resource.yaml @@ -8,7 +8,7 @@ metadata: name: autonomousdatabase-sample spec: details: - autonomousDatabaseOCID: ocid1.autonomousdatabase... + id: ocid1.autonomousdatabase... # Delete this resource to terminate database after the changes applied hardLink: true # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. diff --git a/config/samples/adb/autonomousdatabase_rename.yaml b/config/samples/adb/autonomousdatabase_rename.yaml index d3a29998..22dbcc0f 100644 --- a/config/samples/adb/autonomousdatabase_rename.yaml +++ b/config/samples/adb/autonomousdatabase_rename.yaml @@ -7,8 +7,9 @@ kind: AutonomousDatabase metadata: name: autonomousdatabase-sample spec: + action: Update details: - autonomousDatabaseOCID: ocid1.autonomousdatabase... + id: ocid1.autonomousdatabase... # The database name dbName: RenamedADB # The user-friendly name for the Autonomous Database diff --git a/config/samples/adb/autonomousdatabase_scale.yaml b/config/samples/adb/autonomousdatabase_scale.yaml index cd100675..ea53e94d 100644 --- a/config/samples/adb/autonomousdatabase_scale.yaml +++ b/config/samples/adb/autonomousdatabase_scale.yaml @@ -7,8 +7,9 @@ kind: AutonomousDatabase metadata: name: autonomousdatabase-sample spec: + action: Update details: - autonomousDatabaseOCID: ocid1.autonomousdatabase... + id: ocid1.autonomousdatabase... # Your database's OPCU core count cpuCoreCount: 2 # Your database's storage size in TB diff --git a/config/samples/adb/autonomousdatabase_stop_start_terminate.yaml b/config/samples/adb/autonomousdatabase_stop_start_terminate.yaml index a41f8d0b..4a191dd6 100644 --- a/config/samples/adb/autonomousdatabase_stop_start_terminate.yaml +++ b/config/samples/adb/autonomousdatabase_stop_start_terminate.yaml @@ -7,10 +7,10 @@ kind: AutonomousDatabase metadata: name: autonomousdatabase-sample spec: + + action: Stop # Use the value "Start" to start the database details: - autonomousDatabaseOCID: ocid1.autonomousdatabase... - # Change the lifecycleState to "AVAILABLE" to start the database - lifecycleState: STOPPED + id: ocid1.autonomousdatabase... # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. ociConfig: configMapName: oci-cred diff --git a/config/samples/adb/autonomousdatabase_update_admin_password.yaml b/config/samples/adb/autonomousdatabase_update_admin_password.yaml index 67fd142d..be7aca69 100644 --- a/config/samples/adb/autonomousdatabase_update_admin_password.yaml +++ b/config/samples/adb/autonomousdatabase_update_admin_password.yaml @@ -7,8 +7,9 @@ kind: AutonomousDatabase metadata: name: autonomousdatabase-sample spec: + action: Update details: - autonomousDatabaseOCID: ocid1.autonomousdatabase... + id: ocid1.autonomousdatabase... adminPassword: # Comment out k8sSecret and uncomment ociSecret if you pass the admin password using OCI Secret. k8sSecret: @@ -16,7 +17,7 @@ spec: name: new-admin-password # ociSecret: # # The OCID of the OCI Secret that holds the password of the ADMIN account. It should start with ocid1.vaultsecret... . - # ocid: ocid1.vaultsecret... + # id: ocid1.vaultsecret... # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. ociConfig: configMapName: oci-cred diff --git a/config/samples/adb/autonomousdatabase_update_mtls.yaml b/config/samples/adb/autonomousdatabase_update_mtls.yaml index 93db0d8a..25eda529 100644 --- a/config/samples/adb/autonomousdatabase_update_mtls.yaml +++ b/config/samples/adb/autonomousdatabase_update_mtls.yaml @@ -7,12 +7,11 @@ kind: AutonomousDatabase metadata: name: autonomousdatabase-sample spec: + action: Update details: - autonomousDatabaseOCID: ocid1.autonomousdatabase... - networkAccess: - # Set the patameter to false to allow both TLS and mutual TLS (mTLS) authentication. - # Avaiable when the networkAccessType is RESTRICTED or PRIVATE on shared Autnomous Database. - isMTLSConnectionRequired: false + id: ocid1.autonomousdatabase... + # Set the patameter to false to allow both TLS and mutual TLS (mTLS) authentication, or true to require mTLS connections and disallow TLS connections. + isMTLSConnectionRequired: true # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. ociConfig: diff --git a/config/samples/adb/autonomousdatabase_update_network_access.yaml b/config/samples/adb/autonomousdatabase_update_network_access.yaml index f0e98806..7dd3fa0c 100644 --- a/config/samples/adb/autonomousdatabase_update_network_access.yaml +++ b/config/samples/adb/autonomousdatabase_update_network_access.yaml @@ -7,39 +7,38 @@ kind: AutonomousDatabase metadata: name: autonomousdatabase-sample spec: + action: Update details: - autonomousDatabaseOCID: ocid1.autonomousdatabase... - networkAccess: - # Allow secure access from everywhere. - accessType: PUBLIC + id: ocid1.autonomousdatabase... + # # Allow secure access from everywhere. Uncomment one of the following field depends on your network access configuration. + # accessControlList: + # - + # privateEndpoint: "" - # # Uncomment this block to configure the network access type with the RESTRICTED option. - # # This option lets you restrict access by defining access control rules in an Access Control List (ACL). - # # By specifying an ACL, the database will be accessible from a whitelisted set of IP addresses, CIDR (Classless Inter-Domain Routing) blocks, or VCNs. - # # Use a semicolon (;) as a deliminator between the VCN-specific subnets or IPs. - # accessType: RESTRICTED - # accessControlList: - # - 1.1.1.1 - # - 1.1.0.0/16 - # - ocid1.vcn... - # - ocid1.vcn...;1.1.1.1 - # - ocid1.vcn...;1.1.0.0/16 + # # Uncomment this block to configure the network access type with the RESTRICTED option. + # # This option lets you restrict access by defining access control rules in an Access Control List (ACL). + # # By specifying an ACL, the database will be accessible from a whitelisted set of IP addresses, CIDR (Classless Inter-Domain Routing) blocks, or VCNs. + # # Use a semicolon (;) as a deliminator between the VCN-specific subnets or IPs. + # accessControlList: + # - 1.1.1.1 + # - 1.1.0.0/16 + # - ocid1.vcn... + # - ocid1.vcn...;1.1.1.1 + # - ocid1.vcn...;1.1.0.0/16 - # # Uncomment this block to configure the network access type with the PRIVATE option. - # # This option assigns a private endpoint, private IP, and hostname to your database. - # # Specifying this option allows traffic only from the VCN you specify. - # # This allows you to define security rules, ingress/egress, at the Network Security Group (NSG) level and to control traffic to your Autonomous Database. - # accessType: PRIVATE - # privateEndpoint: - # subnetOCID: ocid1.subnet... - # nsgOCIDs: # Optional - # - ocid1.networksecuritygroup... + # # Uncomment this block to configure the network access type with the PRIVATE option. + # # This option assigns a private endpoint, private IP, and hostname to your database. + # # Specifying this option allows traffic only from the VCN you specify. + # # This allows you to define security rules, ingress/egress, at the Network Security Group (NSG) level and to control traffic to your Autonomous Database. + # privateEndpoint: + # subnetOCID: ocid1.subnet... + # nsgOCIDs: # Optional + # - ocid1.networksecuritygroup... - # # Uncomment this block to configure the network access of an dedicated Autonomous Database (ADB-D) with an access control list. - # isAccessControlEnabled: true - # accessControlList: - # - 1.1.1.1 - # - 1.1.0.0/16 + # # Uncomment this block to configure the network access of an dedicated Autonomous Database (ADB-D) with an access control list. + # accessControlList: + # - 1.1.1.1 + # - 1.1.0.0/16 # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. ociConfig: diff --git a/config/samples/adb/autonomousdatabase_wallet.yaml b/config/samples/adb/autonomousdatabase_wallet.yaml index 15fa6ca0..84136647 100644 --- a/config/samples/adb/autonomousdatabase_wallet.yaml +++ b/config/samples/adb/autonomousdatabase_wallet.yaml @@ -7,19 +7,20 @@ kind: AutonomousDatabase metadata: name: autonomousdatabase-sample spec: + action: Update details: - autonomousDatabaseOCID: ocid1.autonomousdatabase... - wallet: - # Insert a name of the secret where you want the wallet to be stored. The default name is -instance-wallet. - name: instance-wallet - password: - # Comment out k8sSecret and uncomment ociSecret if you pass the admin password using OCI Secret. - k8sSecret: - # The Name of the K8s secret where you want to hold the password of the ADMIN account. - name: instance-wallet-password - # ociSecret: - # # The OCID of the OCI Secret that holds the password of the ADMIN account. It should start with ocid1.vaultsecret... . - # ocid: ocid1.vaultsecret... + id: ocid1.autonomousdatabase... + wallet: + # Insert a name of the secret where you want the wallet to be stored. The default name is -instance-wallet. + name: instance-wallet + password: + # Comment out k8sSecret and uncomment ociSecret if you pass the admin password using OCI Secret. + k8sSecret: + # The Name of the K8s secret where you want to hold the password of the ADMIN account. + name: instance-wallet-password + # ociSecret: + # # The OCID of the OCI Secret that holds the password of the ADMIN account. It should start with ocid1.vaultsecret... . + # id: ocid1.vaultsecret... # Authorize the operator with API signing key pair. Comment out the ociConfig fields if your nodes are already authorized with instance principal. ociConfig: configMapName: oci-cred diff --git a/config/samples/kustomization.yaml b/config/samples/kustomization.yaml index ac5e158f..1a032832 100644 --- a/config/samples/kustomization.yaml +++ b/config/samples/kustomization.yaml @@ -29,5 +29,23 @@ resources: - adb/autonomousdatabase_bind.yaml - adb/autonomousdatabase_backup.yaml - adb/autonomousdatabase_restore.yaml + - acd/autonomouscontainerdatabase_create.yaml + - sidb/singleinstancedatabase.yaml + - sharding/shardingdatabase.yaml + - sharding/sharding_v1alpha1_provshard.yaml + - dbcs/database_v1alpha1_dbcssystem.yaml + - database_v1alpha1_dataguardbroker.yaml + - database_v1alpha1_shardingdatabase.yaml + - observability/v1alpha1/databaseobserver.yaml + - observability/v1/databaseobserver.yaml + - observability/v4/databaseobserver.yaml - acd/autonomouscontainerdatabase_restart_terminate.yaml + - database_v4_shardingdatabase.yaml + - database_v4_dbcssystem.yaml +- database_v4_lrest.yaml +- database_v4_lrpdb.yaml +- database_v4_ordssrvs.yaml +- database_v4_singleinstancedatabase.yaml +- database_v4_dataguardbroker.yaml +- database_v4_oraclerestdataservice.yaml # +kubebuilder:scaffold:manifestskustomizesamples diff --git a/config/samples/observability/v1/databaseobserver.yaml b/config/samples/observability/v1/databaseobserver.yaml new file mode 100644 index 00000000..82a7e89e --- /dev/null +++ b/config/samples/observability/v1/databaseobserver.yaml @@ -0,0 +1,81 @@ +# example +apiVersion: observability.oracle.com/v1 +kind: DatabaseObserver +metadata: + name: obs-sample + labels: + app.kubernetes.io/name: observability-exporter + app.kubernetes.io/instance: obs-sample + app.kubernetes.io/version: 1.5.1 +spec: + database: + dbUser: + key: "username" + secret: db-secret + + dbPassword: + key: "password" + secret: db-secret + + dbConnectionString: + key: "connection" + secret: db-secret + + dbWallet: + secret: instance-wallet + + inheritLabels: + - app.kubernetes.io/name + - app.kubernetes.io/instance + - app.kubernetes.io/version + + sidecars: [ ] + sidecarVolumes: [ ] + + exporter: + deployment: + env: + TNS_ADMIN: /some/custom/path + ORACLE_HOME: /some/custom/path + DB_ROLE: SYSDBA + image: "container-registry.oracle.com/database/observability-exporter:1.5.1" + args: [ "--log.level=info" ] + commands: [ "/oracledb_exporter" ] + + labels: + environment: dev + podTemplate: + labels: + environment: dev + + service: + labels: + environment: dev + + configuration: + configMap: + key: "config.toml" + name: "devcm-oradevdb-config" + + prometheus: + serviceMonitor: + labels: + release: prometheus + + + log: + filename: "alert.log" + path: "/log" + + volume: + name: volume + persistentVolumeClaim: + claimName: "my-pvc" + + replicas: 1 + + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey + + diff --git a/config/samples/observability/v1/databaseobserver_customization_fields.yaml b/config/samples/observability/v1/databaseobserver_customization_fields.yaml new file mode 100644 index 00000000..d88caec4 --- /dev/null +++ b/config/samples/observability/v1/databaseobserver_customization_fields.yaml @@ -0,0 +1,54 @@ +# example +apiVersion: observability.oracle.com/v4 +kind: DatabaseObserver +metadata: + name: obs-sample +spec: + database: + dbUser: + key: "username" + secret: db-secret + + dbPassword: + key: "password" + secret: db-secret + + dbConnectionString: + key: "connection" + secret: db-secret + + dbWallet: + secret: instance-wallets + + exporter: + deployment: + image: "container-registry.oracle.com/database/observability-exporter:1.5.1" + args: + - "--log.level=info" + commands: + - "/oracledb_exporter" + env: + TNS_ADMIN: /some/custom/path + labels: + environment: dev + podTemplate: + labels: + environment: dev + service: + ports: + - name: "metrics" + port: 9161 + targetPort: 9161 + labels: + environment: dev + + prometheus: + serviceMonitor: + endpoints: + - bearerTokenSecret: + key: '' + interval: 15s + port: metrics + labels: + release: prometheus + diff --git a/config/samples/observability/v1/databaseobserver_logs_promtail.yaml b/config/samples/observability/v1/databaseobserver_logs_promtail.yaml new file mode 100644 index 00000000..8130f487 --- /dev/null +++ b/config/samples/observability/v1/databaseobserver_logs_promtail.yaml @@ -0,0 +1,74 @@ +# example +apiVersion: observability.oracle.com/v1 +kind: DatabaseObserver +metadata: + name: obs-sample + labels: + app.kubernetes.io/name: observability-exporter + app.kubernetes.io/instance: obs-sample + app.kubernetes.io/version: 1.5.1 +spec: + database: + dbUser: + key: "username" + secret: db-secret + + dbPassword: + key: "password" + secret: db-secret + + dbConnectionString: + key: "connection" + secret: db-secret + + dbWallet: + secret: instance-wallet + + inheritLabels: + - app.kubernetes.io/name + - app.kubernetes.io/instance + - app.kubernetes.io/version + + sidecars: + - name: promtail + image: grafana/promtail + args: + - -config.file=/etc/promtail/promtail.yaml + volumeMounts: + - name: config + mountPath: /etc/promtail + - name: log-volume + mountPath: /log + + sidecarVolumes: + - name: config + configMap: + name: promtail-sidecar-config + exporter: + deployment: + image: "container-registry.oracle.com/database/observability-exporter:1.5.1" + args: [ "--log.level=info" ] + commands: [ "/oracledb_exporter" ] + + configuration: + configMap: + key: "config.toml" + name: "devcm-oradevdb-config" + + prometheus: + serviceMonitor: + labels: + release: prometheus + + log: + filename: "alert.log" + path: "/log" + + volume: + name: log-volume + + replicas: 1 + + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey diff --git a/config/samples/observability/v1alpha1/databaseobserver.yaml b/config/samples/observability/v1alpha1/databaseobserver.yaml new file mode 100644 index 00000000..24672d8b --- /dev/null +++ b/config/samples/observability/v1alpha1/databaseobserver.yaml @@ -0,0 +1,80 @@ +# example +apiVersion: observability.oracle.com/v1alpha1 +kind: DatabaseObserver +metadata: + name: obs-sample + labels: + app.kubernetes.io/name: observability-exporter + app.kubernetes.io/instance: obs-sample + app.kubernetes.io/version: 1.5.1 +spec: + database: + dbUser: + key: "username" + secret: db-secret + + dbPassword: + key: "password" + secret: db-secret + + dbConnectionString: + key: "connection" + secret: db-secret + + dbWallet: + secret: instance-wallet + + + inheritLabels: + - app.kubernetes.io/name + - app.kubernetes.io/instance + - app.kubernetes.io/version + + sidecars: [ ] + sidecarVolumes: [ ] + + exporter: + deployment: + image: "container-registry.oracle.com/database/observability-exporter:1.5.1" + args: [ "--log.level=info" ] + commands: [ "/oracledb_exporter" ] + env: + TNS_ADMIN: /some/custom/path + ORACLE_HOME: /some/custom/path + labels: + environment: dev + podTemplate: + labels: + environment: dev + + service: + labels: + environment: dev + + configuration: + configMap: + key: "config.toml" + name: "devcm-oradevdb-config" + + prometheus: + serviceMonitor: + labels: + release: prometheus + + + log: + filename: "alert.log" + path: "/log" + + volume: + name: volume + persistentVolumeClaim: + claimName: "my-pvc" + + replicas: 1 + + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey + + diff --git a/config/samples/observability/v1alpha1/databaseobserver_custom_config.yaml b/config/samples/observability/v1alpha1/databaseobserver_custom_config.yaml new file mode 100644 index 00000000..8e0d0623 --- /dev/null +++ b/config/samples/observability/v1alpha1/databaseobserver_custom_config.yaml @@ -0,0 +1,46 @@ +# example +apiVersion: observability.oracle.com/v1alpha1 +kind: DatabaseObserver +metadata: + name: obs-sample + labels: + app.kubernetes.io/name: observability-exporter + app.kubernetes.io/instance: obs-sample + app.kubernetes.io/version: 1.5.1 +spec: + database: + dbUser: + key: "username" + secret: db-secret + + dbPassword: + key: "password" + secret: db-secret + + dbConnectionString: + key: "connection" + secret: db-secret + + dbWallet: + secret: instance-wallet + + inherit_labels: + - app.kubernetes.io/name + - app.kubernetes.io/instance + - app.kubernetes.io/version + + exporter: + deployment: + image: "container-registry.oracle.com/database/observability-exporter:1.5.1" + args: [ "--log.level=info" ] + commands: [ "/oracledb_exporter" ] + + prometheus: + serviceMonitor: + labels: + release: prometheus + + configuration: + configMap: + key: "config.toml" + name: "devcm-oradevdb-config" \ No newline at end of file diff --git a/config/samples/observability/v1alpha1/databaseobserver_logs_promtail.yaml b/config/samples/observability/v1alpha1/databaseobserver_logs_promtail.yaml new file mode 100644 index 00000000..28592cb0 --- /dev/null +++ b/config/samples/observability/v1alpha1/databaseobserver_logs_promtail.yaml @@ -0,0 +1,74 @@ +# example +apiVersion: observability.oracle.com/v1alpha1 +kind: DatabaseObserver +metadata: + name: obs-sample + labels: + app.kubernetes.io/name: observability-exporter + app.kubernetes.io/instance: obs-sample + app.kubernetes.io/version: 1.5.1 +spec: + database: + dbUser: + key: "username" + secret: db-secret + + dbPassword: + key: "password" + secret: db-secret + + dbConnectionString: + key: "connection" + secret: db-secret + + dbWallet: + secret: instance-wallet + + inheritLabels: + - app.kubernetes.io/name + - app.kubernetes.io/instance + - app.kubernetes.io/version + + sidecars: + - name: promtail + image: grafana/promtail + args: + - -config.file=/etc/promtail/promtail.yaml + volumeMounts: + - name: config + mountPath: /etc/promtail + - name: log-volume + mountPath: /log + + sidecarVolumes: + - name: config + configMap: + name: promtail-sidecar-config + exporter: + deployment: + image: "container-registry.oracle.com/database/observability-exporter:1.5.1" + args: [ "--log.level=info" ] + commands: [ "/oracledb_exporter" ] + + configuration: + configMap: + key: "config.toml" + name: "devcm-oradevdb-config" + + prometheus: + serviceMonitor: + labels: + release: prometheus + + log: + filename: "alert.log" + path: "/log" + + volume: + name: log-volume + + replicas: 1 + + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey diff --git a/config/samples/observability/v1alpha1/databaseobserver_minimal.yaml b/config/samples/observability/v1alpha1/databaseobserver_minimal.yaml new file mode 100644 index 00000000..74620ac7 --- /dev/null +++ b/config/samples/observability/v1alpha1/databaseobserver_minimal.yaml @@ -0,0 +1,26 @@ +# example +apiVersion: observability.oracle.com/v1alpha1 +kind: DatabaseObserver +metadata: + name: obs-sample +spec: + database: + dbUser: + key: "username" + secret: db-secret + + dbPassword: + key: "password" + secret: db-secret + + dbConnectionString: + key: "connection" + secret: db-secret + + dbWallet: + secret: instance-wallets + + prometheus: + serviceMonitor: + labels: + release: prometheus \ No newline at end of file diff --git a/config/samples/observability/v1alpha1/databaseobserver_vault.yaml b/config/samples/observability/v1alpha1/databaseobserver_vault.yaml new file mode 100644 index 00000000..2fc3c9f0 --- /dev/null +++ b/config/samples/observability/v1alpha1/databaseobserver_vault.yaml @@ -0,0 +1,30 @@ +# example +apiVersion: observability.oracle.com/v1alpha1 +kind: DatabaseObserver +metadata: + name: obs-sample +spec: + database: + dbUser: + key: "username" + secret: db-secret + + dbPassword: + vaultSecretName: sample_secret + vaultOCID: ocid1.vault.oc1.. + + dbConnectionString: + key: "connection" + secret: db-secret + + dbWallet: + secret: instance-wallet + + prometheus: + serviceMonitor: + labels: + release: prometheus + + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey \ No newline at end of file diff --git a/config/samples/observability/v4/databaseobserver.yaml b/config/samples/observability/v4/databaseobserver.yaml new file mode 100644 index 00000000..f7b310f7 --- /dev/null +++ b/config/samples/observability/v4/databaseobserver.yaml @@ -0,0 +1,79 @@ +# example +apiVersion: observability.oracle.com/v4 +kind: DatabaseObserver +metadata: + name: obs-sample + labels: + app.kubernetes.io/name: observability-exporter + app.kubernetes.io/instance: obs-sample + app.kubernetes.io/version: latest +spec: + database: + dbUser: + key: "username" + secret: db-secret + + dbPassword: + key: "password" + secret: db-secret + + dbConnectionString: + key: "connection" + secret: db-secret + + dbWallet: + secret: instance-wallet + + inheritLabels: + - app.kubernetes.io/name + - app.kubernetes.io/instance + - app.kubernetes.io/version + + sidecars: [ ] + sidecarVolumes: [ ] + + exporter: + deployment: + image: "container-registry.oracle.com/database/observability-exporter:1.5.1" + args: [ "--log.level=info" ] + commands: [ "/oracledb_exporter" ] + env: + TNS_ADMIN: /some/custom/path + ORACLE_HOME: /some/custom/path + labels: + environment: dev + podTemplate: + labels: + environment: dev + + service: + labels: + environment: dev + + + configuration: + configMap: + key: "config.toml" + name: "devcm-oradevdb-config" + + prometheus: + serviceMonitor: + labels: + release: prometheus + + log: + filename: "alert.log" + path: "/log" + + volume: + name: volume + persistentVolumeClaim: + claimName: "my-pvc" + + replicas: 1 + + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey + + diff --git a/config/samples/observability/v4/databaseobserver_custom_config.yaml b/config/samples/observability/v4/databaseobserver_custom_config.yaml new file mode 100644 index 00000000..dd2e3da5 --- /dev/null +++ b/config/samples/observability/v4/databaseobserver_custom_config.yaml @@ -0,0 +1,46 @@ +# example +apiVersion: observability.oracle.com/v4 +kind: DatabaseObserver +metadata: + name: obs-sample + labels: + app.kubernetes.io/name: observability-exporter + app.kubernetes.io/instance: obs-sample + app.kubernetes.io/version: latest +spec: + database: + dbUser: + key: "username" + secret: db-secret + + dbPassword: + key: "password" + secret: db-secret + + dbConnectionString: + key: "connection" + secret: db-secret + + dbWallet: + secret: instance-wallet + + inherit_labels: + - app.kubernetes.io/name + - app.kubernetes.io/instance + - app.kubernetes.io/version + + exporter: + deployment: + image: "container-registry.oracle.com/database/observability-exporter:1.5.1" + args: [ "--log.level=info" ] + commands: [ "/oracledb_exporter" ] + + configuration: + configMap: + key: "config.toml" + name: "devcm-oradevdb-config" + + prometheus: + serviceMonitor: + labels: + release: prometheus \ No newline at end of file diff --git a/config/samples/observability/v4/databaseobserver_logs_promtail.yaml b/config/samples/observability/v4/databaseobserver_logs_promtail.yaml new file mode 100644 index 00000000..26a747a3 --- /dev/null +++ b/config/samples/observability/v4/databaseobserver_logs_promtail.yaml @@ -0,0 +1,76 @@ +# example +apiVersion: observability.oracle.com/v4 +kind: DatabaseObserver +metadata: + name: obs-sample + labels: + app.kubernetes.io/name: observability-exporter + app.kubernetes.io/instance: obs-sample + app.kubernetes.io/version: latest +spec: + database: + dbUser: + key: "username" + secret: db-secret + + dbPassword: + key: "password" + secret: db-secret + + dbConnectionString: + key: "connection" + secret: db-secret + + dbWallet: + secret: instance-wallet + + inheritLabels: + - app.kubernetes.io/name + - app.kubernetes.io/instance + - app.kubernetes.io/version + + sidecars: + - name: promtail + image: grafana/promtail + args: + - -config.file=/etc/promtail/promtail.yaml + volumeMounts: + - name: config + mountPath: /etc/promtail + - name: log-volume + mountPath: /log + + sidecarVolumes: + - name: config + configMap: + name: promtail-sidecar-config + exporter: + deployment: + image: "container-registry.oracle.com/database/observability-exporter:1.5.1" + args: [ "--log.level=info" ] + commands: [ "/oracledb_exporter" ] + + configuration: + configMap: + key: "config.toml" + name: "devcm-oradevdb-config" + + prometheus: + serviceMonitor: + labels: + release: prometheus + + log: + filename: "alert.log" + path: "/log" + + volume: + name: log-volume + + replicas: 1 + + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey + + diff --git a/config/samples/observability/v4/databaseobserver_minimal.yaml b/config/samples/observability/v4/databaseobserver_minimal.yaml new file mode 100644 index 00000000..cc14fbea --- /dev/null +++ b/config/samples/observability/v4/databaseobserver_minimal.yaml @@ -0,0 +1,26 @@ +# example +apiVersion: observability.oracle.com/v4 +kind: DatabaseObserver +metadata: + name: obs-sample +spec: + database: + dbUser: + key: "username" + secret: db-secret + + dbPassword: + key: "password" + secret: db-secret + + dbConnectionString: + key: "connection" + secret: db-secret + + dbWallet: + secret: instance-wallets + + prometheus: + serviceMonitor: + labels: + release: prometheus diff --git a/config/samples/observability/v4/databaseobserver_vault.yaml b/config/samples/observability/v4/databaseobserver_vault.yaml new file mode 100644 index 00000000..4f5845f6 --- /dev/null +++ b/config/samples/observability/v4/databaseobserver_vault.yaml @@ -0,0 +1,39 @@ +# example +apiVersion: observability.oracle.com/v4 +kind: DatabaseObserver +metadata: + name: obs-sample + labels: + app.kubernetes.io/name: observability-exporter + app.kubernetes.io/instance: obs-sample + app.kubernetes.io/version: latest +spec: + database: + dbUser: + key: "username" + secret: db-secret + + dbPassword: + vaultSecretName: sample_secret + vaultOCID: ocid1.vault.oc1.. + + dbConnectionString: + key: "connection" + secret: db-secret + + dbWallet: + secret: instance-wallet + + inherit_labels: + - app.kubernetes.io/name + - app.kubernetes.io/instance + - app.kubernetes.io/version + + prometheus: + serviceMonitor: + labels: + release: prometheus + + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey \ No newline at end of file diff --git a/config/samples/sidb/dataguardbroker.yaml b/config/samples/sidb/dataguardbroker.yaml index 5425afb7..644d2d40 100644 --- a/config/samples/sidb/dataguardbroker.yaml +++ b/config/samples/sidb/dataguardbroker.yaml @@ -3,7 +3,7 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: DataguardBroker metadata: name: dataguardbroker-sample @@ -25,5 +25,9 @@ spec: ## Protection Mode for dg configuration . MaxAvailability or MaxPerformance protectionMode: MaxAvailability - ## Manual Switchover to this database to make it primary(if not already), requires target Database SID . + ## Specify the database SID to switchover thereby making it the primary. + ## Switchover is not supported when fastStartFailover is true. setAsPrimaryDatabase: "" + + ## Enable/disable Fast-Start Failover for the dataguard configuration. + fastStartFailover: false diff --git a/config/samples/sidb/oraclerestdataservice.yaml b/config/samples/sidb/oraclerestdataservice.yaml index 911a9b1e..77555f47 100644 --- a/config/samples/sidb/oraclerestdataservice.yaml +++ b/config/samples/sidb/oraclerestdataservice.yaml @@ -2,7 +2,7 @@ # Copyright (c) 2023, Oracle and/or its affiliates. All rights reserved. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: OracleRestDataService metadata: name: ords-sample @@ -26,24 +26,9 @@ spec: secretKey: keepSecret: true - ## To configure APEX with ORDS, specfiy the apexPassword secret details. Leave empty if Apex is not needed - ## This is a secret containing a common password for APEX_PUBLIC_USER, APEX_REST_PUBLIC_USER, APEX_LISTENER and Apex administrator (username: ADMIN) mapped to secretKey - ## This secret will be deleted after ORDS Installation unless keepSecret set to true. - ## This password should complete the following requirements: - ## 1. Contain at least 6 characters. - ## 2. Contain at least one numeric character (0123456789). - ## 3. Contain at least one punctuation character (!"#$%&()``*+,-/:;?_). - ## 4. Contain at least one uppercase alphabetic character. - - apexPassword: - secretName: - secretKey: - keepSecret: true - ## ORDS image details - ## Supported ORDS image is container-registry.oracle.com/database/ords:21.4.2-gh image: - pullFrom: container-registry.oracle.com/database/ords:21.4.2-gh + pullFrom: container-registry.oracle.com/database/ords-developer:latest pullSecrets: ## Dedicated persistent storage is optional. If not specified, ORDS will use persistent storage from .spec.databaseRef @@ -65,6 +50,8 @@ spec: #serviceAnnotations: # service.beta.kubernetes.io/oci-load-balancer-internal: "true" + ## Set this to true to enable MongoDB API + mongoDbApi: true ## Deploy only on nodes having required labels. Format label_name: label_value ## The same lables are applied to the created PVC diff --git a/config/samples/sidb/oraclerestdataservice_apex.yaml b/config/samples/sidb/oraclerestdataservice_apex.yaml deleted file mode 100644 index 6bdc9fb5..00000000 --- a/config/samples/sidb/oraclerestdataservice_apex.yaml +++ /dev/null @@ -1,42 +0,0 @@ -# -# Copyright (c) 2023, Oracle and/or its affiliates. All rights reserved. -# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. -# - -apiVersion: database.oracle.com/v1alpha1 -kind: OracleRestDataService -metadata: - name: ords-sample - namespace: default -spec: - - ## Database ref. This can be of kind SingleInstanceDatabase. - ## Make sure the source database has been created by applying singeinstancedatabase_express.yaml - databaseRef: "xedb-sample" - - ## Secret containing databaseRef password - adminPassword: - secretName: xedb-admin-secret - - ## Secret containing ORDS_PUBLIC_USER password - ordsPassword: - secretName: ords-secret - - ## To configure APEX with ORDS, specfiy the apexPassword secret details. Leave empty if Apex is not needed. - ## This is a secret containing a common password for APEX_PUBLIC_USER, APEX_REST_PUBLIC_USER, APEX_LISTENER and Apex administrator (username: ADMIN) - apexPassword: - secretName: apex-secret - - ## ORDS image details - image: - pullFrom: container-registry.oracle.com/database/ords:21.4.2-gh - - ## PDB Schemas to be ORDS Enabled. - ## Schema will be created (if not exists) with password as .spec.ordsPassword. - restEnableSchemas: - - schemaName: schema1 - enable: true - urlMapping: - - schemaName: schema2 - enable: true - urlMapping: myschema diff --git a/config/samples/sidb/oraclerestdataservice_create.yaml b/config/samples/sidb/oraclerestdataservice_create.yaml index 454abf37..e98ca018 100644 --- a/config/samples/sidb/oraclerestdataservice_create.yaml +++ b/config/samples/sidb/oraclerestdataservice_create.yaml @@ -3,7 +3,7 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: OracleRestDataService metadata: name: ords-sample @@ -24,7 +24,10 @@ spec: ## ORDS image details image: - pullFrom: container-registry.oracle.com/database/ords:21.4.2-gh + pullFrom: container-registry.oracle.com/database/ords-developer:latest + + ## Set this to true to enable MongoDB API + mongoDbApi: true ## PDB Schemas to be ORDS Enabled. ## Schema will be created (if not exists) with password as .spec.ordsPassword. @@ -34,4 +37,4 @@ spec: urlMapping: - schemaName: schema2 enable: true - urlMapping: myschema + urlMapping: myschema \ No newline at end of file diff --git a/config/samples/sidb/oraclerestdataservice_secrets.yaml b/config/samples/sidb/oraclerestdataservice_secrets.yaml index 9e587960..aebca546 100644 --- a/config/samples/sidb/oraclerestdataservice_secrets.yaml +++ b/config/samples/sidb/oraclerestdataservice_secrets.yaml @@ -13,21 +13,3 @@ type: Opaque stringData: ## Specify your ORDS password here oracle_pwd: - ---- - -## APEX password secret -apiVersion: v1 -kind: Secret -metadata: - name: apex-secret - namespace: default -type: Opaque -stringData: - ## Specify your APEX password here - ## This password should complete the following requirements: - ## 1. Contain at least 6 characters. - ## 2. Contain at least one numeric character (0123456789). - ## 3. Contain at least one punctuation character (!"#$%&()``*+,-/:;?_). - ## 4. Contain at least one uppercase alphabetic character. - oracle_pwd: diff --git a/config/samples/sidb/singleinstancedatabase.yaml b/config/samples/sidb/singleinstancedatabase.yaml index 4425acea..368762f5 100644 --- a/config/samples/sidb/singleinstancedatabase.yaml +++ b/config/samples/sidb/singleinstancedatabase.yaml @@ -2,7 +2,7 @@ # Copyright (c) 2023, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: SingleInstanceDatabase metadata: name: sidb-sample @@ -17,14 +17,24 @@ spec: edition: enterprise ## Type of database. - ## Valid values for createAs are primary, clone or standby - ## Valid only for enterprise and standard editions + ## Valid values for createAs are primary, clone, standby or truecache createAs: primary + ## Specify true to convert this standby to a snapshot standby + ## Valid only if createAs is standby + convertToSnapshotStandby: false + ## Reference to a source primary database. - ## Valid only for createAs clone or standby + ## Valid only when createAs is clone, standby or truecache ## The name of a source primary database resource from the same namespace primaryDatabaseRef: "" + + ## Only valid when createAs is set to truecache + ## Accepts a semi colon separated map of `PRIMARY_PDB_SERIVCE_NAME:PRIMARY_SERVICE_NAME:TRUECACHE_SERVICE_NAME` + trueCacheServices: + # - "FREEPDB1:sales1:sales1_tc" + # - "FREEPDB1:sales2:sales2_tc" + # - "FREEPDB1:sales3:sales3_tc" ## Secret containing SIDB password mapped to secretKey. secretKey defaults to oracle_pwd ## Should refer to adminPassword of Source DB if createAs is clone or standby diff --git a/config/samples/sidb/singleinstancedatabase_clone.yaml b/config/samples/sidb/singleinstancedatabase_clone.yaml index f25484d9..438d4ea5 100644 --- a/config/samples/sidb/singleinstancedatabase_clone.yaml +++ b/config/samples/sidb/singleinstancedatabase_clone.yaml @@ -3,7 +3,7 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: SingleInstanceDatabase metadata: name: sidb-sample-clone diff --git a/config/samples/sidb/singleinstancedatabase_create.yaml b/config/samples/sidb/singleinstancedatabase_create.yaml index d09d9c26..2a4e4bae 100644 --- a/config/samples/sidb/singleinstancedatabase_create.yaml +++ b/config/samples/sidb/singleinstancedatabase_create.yaml @@ -3,7 +3,7 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: SingleInstanceDatabase metadata: # Creates base sidb-sample. Use singleinstancedatabase_clone.yaml for cloning @@ -33,7 +33,7 @@ spec: ## Database image details image: - pullFrom: container-registry.oracle.com/database/enterprise:latest + pullFrom: container-registry.oracle.com/database/enterprise_ru:19 pullSecrets: oracle-container-registry-secret ## size is the required minimum size of the persistent volume diff --git a/config/samples/sidb/singleinstancedatabase_express.yaml b/config/samples/sidb/singleinstancedatabase_express.yaml index 64f2e351..2cabbdaf 100644 --- a/config/samples/sidb/singleinstancedatabase_express.yaml +++ b/config/samples/sidb/singleinstancedatabase_express.yaml @@ -3,7 +3,7 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: SingleInstanceDatabase metadata: name: xedb-sample diff --git a/config/samples/sidb/singleinstancedatabase_free-lite.yaml b/config/samples/sidb/singleinstancedatabase_free-lite.yaml new file mode 100644 index 00000000..93b3c4c9 --- /dev/null +++ b/config/samples/sidb/singleinstancedatabase_free-lite.yaml @@ -0,0 +1,35 @@ +# +# Copyright (c) 2024, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +apiVersion: database.oracle.com/v4 +kind: SingleInstanceDatabase +metadata: + name: freedb-lite-sample + namespace: default +spec: + + ## DB edition + edition: free + + ## Secret containing SIDB password mapped to secretKey + adminPassword: + secretName: freedb-admin-secret + + ## Database image details + image: + ## Oracle Database Free Lite is only supported from DB version 23.2 onwards + pullFrom: container-registry.oracle.com/database/free:latest-lite + + ## size is the required minimum size of the persistent volume + ## storageClass is specified for automatic volume provisioning + ## accessMode can only accept one of ReadWriteOnce, ReadWriteMany + persistence: + size: 50Gi + ## oci-bv applies to OCI block volumes. Use "standard" storageClass for dynamic provisioning in Minikube. Update as appropriate for other cloud service providers + storageClass: "oci-bv" + accessMode: "ReadWriteOnce" + + ## Count of Database Pods. Should be 1 for free edition. + replicas: 1 \ No newline at end of file diff --git a/config/samples/sidb/singleinstancedatabase_free-truecache.yaml b/config/samples/sidb/singleinstancedatabase_free-truecache.yaml new file mode 100644 index 00000000..c2481f7c --- /dev/null +++ b/config/samples/sidb/singleinstancedatabase_free-truecache.yaml @@ -0,0 +1,48 @@ +# +# Copyright (c) 2024, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# + +apiVersion: database.oracle.com/v4 +kind: SingleInstanceDatabase +metadata: + name: truecache-sample + namespace: default +spec: + + ## DB edition + edition: free + + ## DB Type + createAs: truecache + + ## Reference to the source primary database. + primaryDatabaseRef: "freedb-sample" + + ## Accepts a semi colon separated list of `PRIMARY_PDB_SERIVCE_NAME:PRIMARY_SERVICE_NAME:TRUECACHE_SERVICE_NAME` + trueCacheServices: + # - "FREEPDB1:sales1:sales1_tc" + # - "FREEPDB1:sales2:sales2_tc" + # - "FREEPDB1:sales3:sales3_tc" + + ## Secret containing SIDB password mapped to secretKey + adminPassword: + secretName: freedb-admin-secret + + ## Database image details + image: + ## Oracle True Cache is only supported with 23ai + pullFrom: container-registry.oracle.com/database/free:latest + + + ## size is the required minimum size of the persistent volume + ## storageClass is specified for automatic volume provisioning + ## accessMode can only accept one of ReadWriteOnce, ReadWriteMany + persistence: + size: 50Gi + ## oci-bv applies to OCI block volumes. Use "standard" storageClass for dynamic provisioning in Minikube. Update as appropriate for other cloud service providers + storageClass: "oci-bv" + accessMode: "ReadWriteOnce" + + ## Count of Database Pods. Should be 1 for free edition. + replicas: 1 diff --git a/config/samples/sidb/singleinstancedatabase_free.yaml b/config/samples/sidb/singleinstancedatabase_free.yaml index 6dd0aa39..6238e52e 100644 --- a/config/samples/sidb/singleinstancedatabase_free.yaml +++ b/config/samples/sidb/singleinstancedatabase_free.yaml @@ -3,16 +3,13 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: SingleInstanceDatabase metadata: name: freedb-sample namespace: default spec: - - ## Use only alphanumeric characters for sid - sid: FREE - + ## DB edition edition: free diff --git a/config/samples/sidb/singleinstancedatabase_patch.yaml b/config/samples/sidb/singleinstancedatabase_patch.yaml index 9a211cdc..455bdc79 100644 --- a/config/samples/sidb/singleinstancedatabase_patch.yaml +++ b/config/samples/sidb/singleinstancedatabase_patch.yaml @@ -3,7 +3,7 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: SingleInstanceDatabase metadata: # sidb-sample should have already been created using singleinstancedatabase_create.yaml diff --git a/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml b/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml index 5e4d0a4f..4eec988a 100644 --- a/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml +++ b/config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml @@ -3,7 +3,7 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: SingleInstanceDatabase metadata: name: prebuiltdb-sample diff --git a/config/samples/sidb/singleinstancedatabase_standby.yaml b/config/samples/sidb/singleinstancedatabase_standby.yaml index 644438b4..d7ad4b23 100644 --- a/config/samples/sidb/singleinstancedatabase_standby.yaml +++ b/config/samples/sidb/singleinstancedatabase_standby.yaml @@ -3,7 +3,7 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: SingleInstanceDatabase metadata: # Creates base standbydatabase-sample. Use singleinstancedatabase_clone.yaml for cloning @@ -27,7 +27,7 @@ spec: ## Database image details image: - pullFrom: container-registry.oracle.com/database/enterprise:latest + pullFrom: container-registry.oracle.com/database/enterprise_ru:19 pullSecrets: oracle-container-registry-secret ## size is the required minimum size of the persistent volume diff --git a/config/samples/sidb/singleinstancedatabase_tcps.yaml b/config/samples/sidb/singleinstancedatabase_tcps.yaml index 06389f96..d3e3100b 100644 --- a/config/samples/sidb/singleinstancedatabase_tcps.yaml +++ b/config/samples/sidb/singleinstancedatabase_tcps.yaml @@ -3,7 +3,7 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: SingleInstanceDatabase metadata: # Creates base sidb-sample. Use singleinstancedatabase_clone.yaml for cloning diff --git a/config/webhook/manifests.yaml b/config/webhook/manifests.yaml index c150867f..3a0f15ec 100644 --- a/config/webhook/manifests.yaml +++ b/config/webhook/manifests.yaml @@ -1,9 +1,7 @@ - --- apiVersion: admissionregistration.k8s.io/v1 kind: MutatingWebhookConfiguration metadata: - creationTimestamp: null name: mutating-webhook-configuration webhooks: - admissionReviewVersions: @@ -12,50 +10,153 @@ webhooks: service: name: webhook-service namespace: system - path: /mutate-database-oracle-com-v1alpha1-autonomousdatabase + path: /mutate-database-oracle-com-v4-autonomousdatabasebackup failurePolicy: Fail - name: mautonomousdatabase.kb.io + name: mautonomousdatabasebackupv4.kb.io rules: - apiGroups: - database.oracle.com apiVersions: - - v1alpha1 + - v4 operations: - CREATE - UPDATE resources: - - autonomousdatabases + - autonomousdatabasebackups sideEffects: None - admissionReviewVersions: - v1 + - v1beta1 clientConfig: service: name: webhook-service namespace: system - path: /mutate-database-oracle-com-v1alpha1-autonomousdatabasebackup + path: /mutate-database-oracle-com-v4-cdb failurePolicy: Fail - name: mautonomousdatabasebackup.kb.io + name: mcdb.kb.io rules: - apiGroups: - database.oracle.com apiVersions: - - v1alpha1 + - v4 operations: - CREATE - UPDATE resources: - - autonomousdatabasebackups + - cdbs sideEffects: None - admissionReviewVersions: - v1 + clientConfig: + service: + name: webhook-service + namespace: system + path: /mutate-database-oracle-com-v4-dbcssystem + failurePolicy: Fail + name: mdbcssystemv4.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v4 + operations: + - CREATE + - UPDATE + resources: + - dbcssystems + sideEffects: None +- admissionReviewVersions: + - v4 - v1beta1 clientConfig: service: name: webhook-service namespace: system - path: /mutate-database-oracle-com-v1alpha1-cdb + path: /mutate-database-oracle-com-v4-lrest failurePolicy: Fail - name: mcdb.kb.io + name: mlrest.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v4 + operations: + - CREATE + - UPDATE + resources: + - lrests + sideEffects: None +- admissionReviewVersions: + - v4 + - v1beta1 + clientConfig: + service: + name: webhook-service + namespace: system + path: /mutate-database-oracle-com-v4-lrpdb + failurePolicy: Fail + name: mlrpdb.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v4 + operations: + - CREATE + - UPDATE + resources: + - lrpdbs + sideEffects: None +- admissionReviewVersions: + - v1 + - v1beta1 + clientConfig: + service: + name: webhook-service + namespace: system + path: /mutate-database-oracle-com-v4-pdb + failurePolicy: Fail + name: mpdb.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v4 + operations: + - CREATE + - UPDATE + resources: + - pdbs + sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: webhook-service + namespace: system + path: /mutate-database-oracle-com-v4-shardingdatabase + failurePolicy: Fail + name: mshardingdatabasev4.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v4 + operations: + - CREATE + - UPDATE + resources: + - shardingdatabases + sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: webhook-service + namespace: system + path: /mutate-database-oracle-com-v1alpha1-autonomousdatabasebackup + failurePolicy: Fail + name: mautonomousdatabasebackupv1alpha1.kb.io rules: - apiGroups: - database.oracle.com @@ -65,7 +166,7 @@ webhooks: - CREATE - UPDATE resources: - - cdbs + - autonomousdatabasebackups sideEffects: None - admissionReviewVersions: - v1 @@ -88,6 +189,26 @@ webhooks: resources: - dataguardbrokers sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: webhook-service + namespace: system + path: /mutate-database-oracle-com-v4-dbcssystem + failurePolicy: Fail + name: mdbcssystemv1alpha1.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v4 + operations: + - CREATE + - UPDATE + resources: + - dbcssystems + sideEffects: None - admissionReviewVersions: - v1 - v1beta1 @@ -111,14 +232,13 @@ webhooks: sideEffects: None - admissionReviewVersions: - v1 - - v1beta1 clientConfig: service: name: webhook-service namespace: system - path: /mutate-database-oracle-com-v1alpha1-pdb + path: /mutate-database-oracle-com-v1alpha1-shardingdatabase failurePolicy: Fail - name: mpdb.kb.io + name: mshardingdatabasev1alpha1.kb.io rules: - apiGroups: - database.oracle.com @@ -128,7 +248,7 @@ webhooks: - CREATE - UPDATE resources: - - pdbs + - shardingdatabases sideEffects: None - admissionReviewVersions: - v1 @@ -171,6 +291,26 @@ webhooks: resources: - singleinstancedatabases sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: webhook-service + namespace: system + path: /mutate-observability-oracle-com-v1-databaseobserver + failurePolicy: Fail + name: mdatabaseobserver.kb.io + rules: + - apiGroups: + - observability.oracle.com + apiVersions: + - v1 + operations: + - CREATE + - UPDATE + resources: + - databaseobservers + sideEffects: None - admissionReviewVersions: - v1 clientConfig: @@ -191,12 +331,30 @@ webhooks: resources: - databaseobservers sideEffects: None - +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: webhook-service + namespace: system + path: /mutate-observability-oracle-com-v4-databaseobserver + failurePolicy: Fail + name: mdatabaseobserver.kb.io + rules: + - apiGroups: + - observability.oracle.com + apiVersions: + - v4 + operations: + - CREATE + - UPDATE + resources: + - databaseobservers + sideEffects: None --- apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingWebhookConfiguration metadata: - creationTimestamp: null name: validating-webhook-configuration webhooks: - admissionReviewVersions: @@ -205,14 +363,14 @@ webhooks: service: name: webhook-service namespace: system - path: /validate-database-oracle-com-v1alpha1-autonomouscontainerdatabase + path: /validate-database-oracle-com-v4-autonomouscontainerdatabase failurePolicy: Fail - name: vautonomouscontainerdatabase.kb.io + name: vautonomouscontainerdatabasev4.kb.io rules: - apiGroups: - database.oracle.com apiVersions: - - v1alpha1 + - v4 operations: - CREATE - UPDATE @@ -225,9 +383,154 @@ webhooks: service: name: webhook-service namespace: system - path: /validate-database-oracle-com-v1alpha1-autonomousdatabase + path: /validate-database-oracle-com-v4-autonomousdatabasebackup + failurePolicy: Fail + name: vautonomousdatabasebackupv4.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v4 + operations: + - CREATE + - UPDATE + resources: + - autonomousdatabasebackups + sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: webhook-service + namespace: system + path: /validate-database-oracle-com-v4-autonomousdatabaserestore + failurePolicy: Fail + name: vautonomousdatabaserestorev4.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v4 + operations: + - CREATE + - UPDATE + resources: + - autonomousdatabaserestores + sideEffects: None +- admissionReviewVersions: + - v1 + - v1beta1 + clientConfig: + service: + name: webhook-service + namespace: system + path: /validate-database-oracle-com-v4-cdb + failurePolicy: Fail + name: vcdb.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v4 + operations: + - CREATE + - UPDATE + resources: + - cdbs + sideEffects: None +- admissionReviewVersions: + - v4 + - v1beta1 + clientConfig: + service: + name: webhook-service + namespace: system + path: /validate-database-oracle-com-v4-lrest + failurePolicy: Fail + name: vlrest.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v4 + operations: + - CREATE + - UPDATE + resources: + - lrests + sideEffects: None +- admissionReviewVersions: + - v4 + - v1beta1 + clientConfig: + service: + name: webhook-service + namespace: system + path: /validate-database-oracle-com-v4-lrpdb + failurePolicy: Fail + name: vlrpdb.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v4 + operations: + - CREATE + - UPDATE + resources: + - lrpdbs + sideEffects: None +- admissionReviewVersions: + - v1 + - v1beta1 + clientConfig: + service: + name: webhook-service + namespace: system + path: /validate-database-oracle-com-v4-pdb + failurePolicy: Fail + name: vpdb.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v4 + operations: + - CREATE + - UPDATE + resources: + - pdbs + sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: webhook-service + namespace: system + path: /validate-database-oracle-com-v4-shardingdatabase failurePolicy: Fail - name: vautonomousdatabase.kb.io + name: vshardingdatabasev4.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v4 + operations: + - CREATE + - UPDATE + - DELETE + resources: + - shardingdatabases + sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: webhook-service + namespace: system + path: /validate-database-oracle-com-v1alpha1-autonomouscontainerdatabase + failurePolicy: Fail + name: vautonomouscontainerdatabasev1alpha1.kb.io rules: - apiGroups: - database.oracle.com @@ -237,7 +540,7 @@ webhooks: - CREATE - UPDATE resources: - - autonomousdatabases + - autonomouscontainerdatabases sideEffects: None - admissionReviewVersions: - v1 @@ -247,7 +550,7 @@ webhooks: namespace: system path: /validate-database-oracle-com-v1alpha1-autonomousdatabasebackup failurePolicy: Fail - name: vautonomousdatabasebackup.kb.io + name: vautonomousdatabasebackupv1alpha1.kb.io rules: - apiGroups: - database.oracle.com @@ -267,7 +570,7 @@ webhooks: namespace: system path: /validate-database-oracle-com-v1alpha1-autonomousdatabaserestore failurePolicy: Fail - name: vautonomousdatabaserestore.kb.io + name: vautonomousdatabaserestorev1alpha1.kb.io rules: - apiGroups: - database.oracle.com @@ -281,14 +584,13 @@ webhooks: sideEffects: None - admissionReviewVersions: - v1 - - v1beta1 clientConfig: service: name: webhook-service namespace: system - path: /validate-database-oracle-com-v1alpha1-cdb + path: /validate-database-oracle-com-v1alpha1-autonomousdatabase failurePolicy: Fail - name: vcdb.kb.io + name: vautonomousdatabasev1alpha1.kb.io rules: - apiGroups: - database.oracle.com @@ -298,7 +600,7 @@ webhooks: - CREATE - UPDATE resources: - - cdbs + - autonomousdatabases sideEffects: None - admissionReviewVersions: - v1 @@ -344,14 +646,13 @@ webhooks: sideEffects: None - admissionReviewVersions: - v1 - - v1beta1 clientConfig: service: name: webhook-service namespace: system - path: /validate-database-oracle-com-v1alpha1-pdb + path: /validate-database-oracle-com-v1alpha1-shardingdatabase failurePolicy: Fail - name: vpdb.kb.io + name: vshardingdatabasev1alpha1.kb.io rules: - apiGroups: - database.oracle.com @@ -360,8 +661,9 @@ webhooks: operations: - CREATE - UPDATE + - DELETE resources: - - pdbs + - shardingdatabases sideEffects: None - admissionReviewVersions: - v1 @@ -406,6 +708,26 @@ webhooks: resources: - singleinstancedatabases sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: webhook-service + namespace: system + path: /validate-observability-oracle-com-v1-databaseobserver + failurePolicy: Fail + name: vdatabaseobserver.kb.io + rules: + - apiGroups: + - observability.oracle.com + apiVersions: + - v1 + operations: + - CREATE + - UPDATE + resources: + - databaseobservers + sideEffects: None - admissionReviewVersions: - v1 clientConfig: @@ -426,3 +748,23 @@ webhooks: resources: - databaseobservers sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: webhook-service + namespace: system + path: /validate-observability-oracle-com-v4-databaseobserver + failurePolicy: Fail + name: vdatabaseobserver.kb.io + rules: + - apiGroups: + - observability.oracle.com + apiVersions: + - v4 + operations: + - CREATE + - UPDATE + resources: + - databaseobservers + sideEffects: None diff --git a/controllers/database/autonomouscontainerdatabase_controller.go b/controllers/database/autonomouscontainerdatabase_controller.go index 3845a03f..73830eee 100644 --- a/controllers/database/autonomouscontainerdatabase_controller.go +++ b/controllers/database/autonomouscontainerdatabase_controller.go @@ -58,7 +58,7 @@ import ( "sigs.k8s.io/controller-runtime/pkg/event" "sigs.k8s.io/controller-runtime/pkg/predicate" - dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + dbv4 "github.com/oracle/oracle-database-operator/apis/database/v4" "github.com/oracle/oracle-database-operator/commons/annotations" "github.com/oracle/oracle-database-operator/commons/k8s" "github.com/oracle/oracle-database-operator/commons/oci" @@ -77,7 +77,7 @@ type AutonomousContainerDatabaseReconciler struct { // SetupWithManager sets up the controller with the Manager. func (r *AutonomousContainerDatabaseReconciler) SetupWithManager(mgr ctrl.Manager) error { return ctrl.NewControllerManagedBy(mgr). - For(&dbv1alpha1.AutonomousContainerDatabase{}). + For(&dbv4.AutonomousContainerDatabase{}). WithEventFilter(r.eventFilterPredicate()). WithOptions(controller.Options{MaxConcurrentReconciles: 5}). Complete(r) @@ -86,13 +86,13 @@ func (r *AutonomousContainerDatabaseReconciler) SetupWithManager(mgr ctrl.Manage func (r *AutonomousContainerDatabaseReconciler) eventFilterPredicate() predicate.Predicate { pred := predicate.Funcs{ UpdateFunc: func(e event.UpdateEvent) bool { - desiredACD, acdOk := e.ObjectNew.(*dbv1alpha1.AutonomousContainerDatabase) + desiredACD, acdOk := e.ObjectNew.(*dbv4.AutonomousContainerDatabase) if acdOk { - oldACD := e.ObjectOld.(*dbv1alpha1.AutonomousContainerDatabase) + oldACD := e.ObjectOld.(*dbv4.AutonomousContainerDatabase) if !reflect.DeepEqual(oldACD.Status, desiredACD.Status) || - (controllerutil.ContainsFinalizer(oldACD, dbv1alpha1.LastSuccessfulSpec) != controllerutil.ContainsFinalizer(desiredACD, dbv1alpha1.LastSuccessfulSpec)) || - (controllerutil.ContainsFinalizer(oldACD, dbv1alpha1.ACDFinalizer) != controllerutil.ContainsFinalizer(desiredACD, dbv1alpha1.ACDFinalizer)) { + (controllerutil.ContainsFinalizer(oldACD, dbv4.LastSuccessfulSpec) != controllerutil.ContainsFinalizer(desiredACD, dbv4.LastSuccessfulSpec)) || + (controllerutil.ContainsFinalizer(oldACD, dbv4.ACDFinalizer) != controllerutil.ContainsFinalizer(desiredACD, dbv4.ACDFinalizer)) { // Don't enqueue if the status, lastSucSpec, or the finalizler changes return false } @@ -103,7 +103,7 @@ func (r *AutonomousContainerDatabaseReconciler) eventFilterPredicate() predicate }, DeleteFunc: func(e event.DeleteEvent) bool { // Do not trigger reconciliation when the object is deleted from the cluster. - _, acdOk := e.Object.(*dbv1alpha1.AutonomousContainerDatabase) + _, acdOk := e.Object.(*dbv4.AutonomousContainerDatabase) return !acdOk }, } @@ -124,10 +124,10 @@ func (r *AutonomousContainerDatabaseReconciler) Reconcile(ctx context.Context, r logger := r.Log.WithValues("Namespace/Name", req.NamespacedName) var err error - var ociACD *dbv1alpha1.AutonomousContainerDatabase + var ociACD *dbv4.AutonomousContainerDatabase // Get the autonomousdatabase instance from the cluster - acd := &dbv1alpha1.AutonomousContainerDatabase{} + acd := &dbv4.AutonomousContainerDatabase{} if err := r.KubeClient.Get(context.TODO(), req.NamespacedName, acd); err != nil { // Ignore not-found errors, since they can't be fixed by an immediate requeue. // No need to change the since we don't know if we obtain the object. @@ -159,7 +159,7 @@ func (r *AutonomousContainerDatabaseReconciler) Reconcile(ctx context.Context, r return r.manageError(logger, acd, err) } - ociACD = &dbv1alpha1.AutonomousContainerDatabase{} + ociACD = &dbv4.AutonomousContainerDatabase{} ociACD.UpdateFromOCIACD(resp.AutonomousContainerDatabase) } @@ -218,7 +218,7 @@ func (r *AutonomousContainerDatabaseReconciler) Reconcile(ctx context.Context, r return r.manageError(logger, acd, err) } - if dbv1alpha1.IsACDIntermediateState(acd.Status.LifecycleState) { + if dbv4.IsACDIntermediateState(acd.Status.LifecycleState) { logger.WithName("IsIntermediateState").Info("Current lifecycleState is " + string(acd.Status.LifecycleState) + "; reconcile queued") return requeueResult, nil } @@ -232,16 +232,16 @@ func (r *AutonomousContainerDatabaseReconciler) Reconcile(ctx context.Context, r return emptyResult, nil } -func (r *AutonomousContainerDatabaseReconciler) setupOCIClients(logger logr.Logger, acd *dbv1alpha1.AutonomousContainerDatabase) error { +func (r *AutonomousContainerDatabaseReconciler) setupOCIClients(logger logr.Logger, acd *dbv4.AutonomousContainerDatabase) error { var err error - authData := oci.APIKeyAuth{ + authData := oci.ApiKeyAuth{ ConfigMapName: acd.Spec.OCIConfig.ConfigMapName, SecretName: acd.Spec.OCIConfig.SecretName, Namespace: acd.GetNamespace(), } - provider, err := oci.GetOCIProvider(r.KubeClient, authData) + provider, err := oci.GetOciProvider(r.KubeClient, authData) if err != nil { return err } @@ -254,7 +254,7 @@ func (r *AutonomousContainerDatabaseReconciler) setupOCIClients(logger logr.Logg return nil } -func (r *AutonomousContainerDatabaseReconciler) manageError(logger logr.Logger, acd *dbv1alpha1.AutonomousContainerDatabase, issue error) (ctrl.Result, error) { +func (r *AutonomousContainerDatabaseReconciler) manageError(logger logr.Logger, acd *dbv4.AutonomousContainerDatabase, issue error) (ctrl.Result, error) { l := logger.WithName("manageError") // Has synced at least once @@ -290,7 +290,7 @@ func (r *AutonomousContainerDatabaseReconciler) manageError(logger logr.Logger, } // validateLifecycleState gets and validates the current lifecycleState -func (r *AutonomousContainerDatabaseReconciler) validateLifecycleState(logger logr.Logger, acd *dbv1alpha1.AutonomousContainerDatabase, ociACD *dbv1alpha1.AutonomousContainerDatabase) (needsRequeue bool, err error) { +func (r *AutonomousContainerDatabaseReconciler) validateLifecycleState(logger logr.Logger, acd *dbv4.AutonomousContainerDatabase, ociACD *dbv4.AutonomousContainerDatabase) (needsRequeue bool, err error) { if ociACD == nil { return false, nil } @@ -313,7 +313,7 @@ func (r *AutonomousContainerDatabaseReconciler) validateLifecycleState(logger lo return false, err } - if dbv1alpha1.IsACDIntermediateState(ociACD.Status.LifecycleState) { + if dbv4.IsACDIntermediateState(ociACD.Status.LifecycleState) { l.Info("LifecycleState is " + string(acd.Status.LifecycleState) + "; reconcile queued") return true, nil } @@ -321,7 +321,7 @@ func (r *AutonomousContainerDatabaseReconciler) validateLifecycleState(logger lo return false, nil } -func (r *AutonomousContainerDatabaseReconciler) validateCleanup(logger logr.Logger, acd *dbv1alpha1.AutonomousContainerDatabase) (exitReconcile bool, err error) { +func (r *AutonomousContainerDatabaseReconciler) validateCleanup(logger logr.Logger, acd *dbv4.AutonomousContainerDatabase) (exitReconcile bool, err error) { l := logger.WithName("validateCleanup") isACDToBeDeleted := acd.GetDeletionTimestamp() != nil @@ -330,7 +330,7 @@ func (r *AutonomousContainerDatabaseReconciler) validateCleanup(logger logr.Logg return false, nil } - if controllerutil.ContainsFinalizer(acd, dbv1alpha1.ACDFinalizer) { + if controllerutil.ContainsFinalizer(acd, dbv4.ACDFinalizer) { if acd.Status.LifecycleState == database.AutonomousContainerDatabaseLifecycleStateTerminating { l.Info("Resource is already in TERMINATING state") // Delete in progress, continue with the reconcile logic @@ -341,7 +341,7 @@ func (r *AutonomousContainerDatabaseReconciler) validateCleanup(logger logr.Logg // The acd has been deleted. Remove the finalizer and exit the reconcile. // Once all finalizers have been removed, the object will be deleted. l.Info("Resource is already in TERMINATED state; remove the finalizer") - if err := k8s.RemoveFinalizerAndPatch(r.KubeClient, acd, dbv1alpha1.ACDFinalizer); err != nil { + if err := k8s.RemoveFinalizerAndPatch(r.KubeClient, acd, dbv4.ACDFinalizer); err != nil { return false, err } return true, nil @@ -350,17 +350,17 @@ func (r *AutonomousContainerDatabaseReconciler) validateCleanup(logger logr.Logg if acd.Spec.AutonomousContainerDatabaseOCID == nil { l.Info("Missing AutonomousContainerDatabaseOCID to terminate Autonomous Container Database; remove the finalizer anyway", "Name", acd.Name, "Namespace", acd.Namespace) // Remove finalizer anyway. - if err := k8s.RemoveFinalizerAndPatch(r.KubeClient, acd, dbv1alpha1.ACDFinalizer); err != nil { + if err := k8s.RemoveFinalizerAndPatch(r.KubeClient, acd, dbv4.ACDFinalizer); err != nil { return false, err } return true, nil } - if acd.Spec.Action != dbv1alpha1.AcdActionTerminate { + if acd.Spec.Action != dbv4.AcdActionTerminate { // Run finalization logic for finalizer. If the finalization logic fails, don't remove the finalizer so // that we can retry during the next reconciliation. l.Info("Terminating Autonomous Container Database") - acd.Spec.Action = dbv1alpha1.AcdActionTerminate + acd.Spec.Action = dbv4.AcdActionTerminate if err := r.KubeClient.Update(context.TODO(), acd); err != nil { return false, err } @@ -376,15 +376,15 @@ func (r *AutonomousContainerDatabaseReconciler) validateCleanup(logger logr.Logg return true, nil } -func (r *AutonomousContainerDatabaseReconciler) validateFinalizer(acd *dbv1alpha1.AutonomousContainerDatabase) error { +func (r *AutonomousContainerDatabaseReconciler) validateFinalizer(acd *dbv4.AutonomousContainerDatabase) error { // Delete is not schduled. Update the finalizer for this CR if hardLink is present if acd.Spec.HardLink != nil { - if *acd.Spec.HardLink && !controllerutil.ContainsFinalizer(acd, dbv1alpha1.ACDFinalizer) { - if err := k8s.AddFinalizerAndPatch(r.KubeClient, acd, dbv1alpha1.ACDFinalizer); err != nil { + if *acd.Spec.HardLink && !controllerutil.ContainsFinalizer(acd, dbv4.ACDFinalizer) { + if err := k8s.AddFinalizerAndPatch(r.KubeClient, acd, dbv4.ACDFinalizer); err != nil { return err } - } else if !*acd.Spec.HardLink && controllerutil.ContainsFinalizer(acd, dbv1alpha1.ACDFinalizer) { - if err := k8s.RemoveFinalizerAndPatch(r.KubeClient, acd, dbv1alpha1.ACDFinalizer); err != nil { + } else if !*acd.Spec.HardLink && controllerutil.ContainsFinalizer(acd, dbv4.ACDFinalizer) { + if err := k8s.RemoveFinalizerAndPatch(r.KubeClient, acd, dbv4.ACDFinalizer); err != nil { return err } } @@ -395,8 +395,8 @@ func (r *AutonomousContainerDatabaseReconciler) validateFinalizer(acd *dbv1alpha func (r *AutonomousContainerDatabaseReconciler) validateOperation( logger logr.Logger, - acd *dbv1alpha1.AutonomousContainerDatabase, - ociACD *dbv1alpha1.AutonomousContainerDatabase) (exitReconcile bool, result ctrl.Result, err error) { + acd *dbv4.AutonomousContainerDatabase, + ociACD *dbv4.AutonomousContainerDatabase) (exitReconcile bool, result ctrl.Result, err error) { l := logger.WithName("validateOperation") @@ -511,7 +511,7 @@ func (r *AutonomousContainerDatabaseReconciler) validateOperation( } } -func (r *AutonomousContainerDatabaseReconciler) updateCR(acd *dbv1alpha1.AutonomousContainerDatabase) error { +func (r *AutonomousContainerDatabaseReconciler) updateCR(acd *dbv4.AutonomousContainerDatabase) error { // Update the lastSucSpec if err := acd.UpdateLastSuccessfulSpec(); err != nil { return err @@ -523,14 +523,14 @@ func (r *AutonomousContainerDatabaseReconciler) updateCR(acd *dbv1alpha1.Autonom return nil } -func (r *AutonomousContainerDatabaseReconciler) patchLastSuccessfulSpec(acd *dbv1alpha1.AutonomousContainerDatabase) error { +func (r *AutonomousContainerDatabaseReconciler) patchLastSuccessfulSpec(acd *dbv4.AutonomousContainerDatabase) error { specBytes, err := json.Marshal(acd.Spec) if err != nil { return err } anns := map[string]string{ - dbv1alpha1.LastSuccessfulSpec: string(specBytes), + dbv4.LastSuccessfulSpec: string(specBytes), } annotations.PatchAnnotations(r.KubeClient, acd, anns) @@ -538,7 +538,7 @@ func (r *AutonomousContainerDatabaseReconciler) patchLastSuccessfulSpec(acd *dbv return nil } -func (r *AutonomousContainerDatabaseReconciler) createACD(logger logr.Logger, acd *dbv1alpha1.AutonomousContainerDatabase) error { +func (r *AutonomousContainerDatabaseReconciler) createACD(logger logr.Logger, acd *dbv4.AutonomousContainerDatabase) error { logger.WithName("createACD").Info("Sending CreateAutonomousContainerDatabase request to OCI") resp, err := r.dbService.CreateAutonomousContainerDatabase(acd) @@ -551,7 +551,7 @@ func (r *AutonomousContainerDatabaseReconciler) createACD(logger logr.Logger, ac return nil } -func (r *AutonomousContainerDatabaseReconciler) getACD(logger logr.Logger, acd *dbv1alpha1.AutonomousContainerDatabase) (bool, error) { +func (r *AutonomousContainerDatabaseReconciler) getACD(logger logr.Logger, acd *dbv4.AutonomousContainerDatabase) (bool, error) { if acd == nil { return false, errors.New("AutonomousContainerDatabase OCID is missing") } @@ -573,10 +573,10 @@ func (r *AutonomousContainerDatabaseReconciler) getACD(logger logr.Logger, acd * // The AutonomousContainerDatabase is updated with the returned object from the OCI requests. func (r *AutonomousContainerDatabaseReconciler) updateACD( logger logr.Logger, - acd *dbv1alpha1.AutonomousContainerDatabase, - difACD *dbv1alpha1.AutonomousContainerDatabase) (ociReqSent bool, specChanged bool, err error) { + acd *dbv4.AutonomousContainerDatabase, + difACD *dbv4.AutonomousContainerDatabase) (ociReqSent bool, specChanged bool, err error) { - validations := []func(logr.Logger, *dbv1alpha1.AutonomousContainerDatabase, *dbv1alpha1.AutonomousContainerDatabase) (bool, bool, error){ + validations := []func(logr.Logger, *dbv4.AutonomousContainerDatabase, *dbv4.AutonomousContainerDatabase) (bool, bool, error){ r.validateGeneralFields, r.validateDesiredLifecycleState, } @@ -597,8 +597,8 @@ func (r *AutonomousContainerDatabaseReconciler) updateACD( func (r *AutonomousContainerDatabaseReconciler) validateGeneralFields( logger logr.Logger, - acd *dbv1alpha1.AutonomousContainerDatabase, - difACD *dbv1alpha1.AutonomousContainerDatabase) (sent bool, requeue bool, err error) { + acd *dbv4.AutonomousContainerDatabase, + difACD *dbv4.AutonomousContainerDatabase) (sent bool, requeue bool, err error) { if difACD.Spec.DisplayName == nil && difACD.Spec.PatchModel == "" && @@ -620,17 +620,17 @@ func (r *AutonomousContainerDatabaseReconciler) validateGeneralFields( func (r *AutonomousContainerDatabaseReconciler) validateDesiredLifecycleState( logger logr.Logger, - acd *dbv1alpha1.AutonomousContainerDatabase, - difACD *dbv1alpha1.AutonomousContainerDatabase) (sent bool, specChanged bool, err error) { + acd *dbv4.AutonomousContainerDatabase, + difACD *dbv4.AutonomousContainerDatabase) (sent bool, specChanged bool, err error) { - if difACD.Spec.Action == dbv1alpha1.AcdActionBlank { + if difACD.Spec.Action == dbv4.AcdActionBlank { return false, false, nil } l := logger.WithName("validateDesiredLifecycleState") switch difACD.Spec.Action { - case dbv1alpha1.AcdActionRestart: + case dbv4.AcdActionRestart: l.Info("Sending RestartAutonomousContainerDatabase request to OCI") resp, err := r.dbService.RestartAutonomousContainerDatabase(*acd.Spec.AutonomousContainerDatabaseOCID) @@ -639,7 +639,7 @@ func (r *AutonomousContainerDatabaseReconciler) validateDesiredLifecycleState( } acd.Status.LifecycleState = resp.LifecycleState - case dbv1alpha1.AcdActionTerminate: + case dbv4.AcdActionTerminate: l.Info("Sending TerminateAutonomousContainerDatabase request to OCI") _, err := r.dbService.TerminateAutonomousContainerDatabase(*acd.Spec.AutonomousContainerDatabaseOCID) @@ -652,7 +652,7 @@ func (r *AutonomousContainerDatabaseReconciler) validateDesiredLifecycleState( return false, false, errors.New("unknown lifecycleState") } - acd.Spec.Action = dbv1alpha1.AcdActionBlank + acd.Spec.Action = dbv4.AcdActionBlank return true, true, nil } diff --git a/controllers/database/autonomousdatabase_controller.go b/controllers/database/autonomousdatabase_controller.go index bf56bfe0..37ae1b14 100644 --- a/controllers/database/autonomousdatabase_controller.go +++ b/controllers/database/autonomousdatabase_controller.go @@ -40,7 +40,6 @@ package controllers import ( "context" - "encoding/json" "errors" "fmt" "reflect" @@ -49,7 +48,6 @@ import ( "time" "github.com/go-logr/logr" - "github.com/oracle/oci-go-sdk/v65/common" "github.com/oracle/oci-go-sdk/v65/database" apiErrors "k8s.io/apimachinery/pkg/api/errors" @@ -68,12 +66,14 @@ import ( "sigs.k8s.io/controller-runtime/pkg/predicate" "sigs.k8s.io/controller-runtime/pkg/reconcile" - dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" - "github.com/oracle/oracle-database-operator/commons/annotations" + dbv4 "github.com/oracle/oracle-database-operator/apis/database/v4" "github.com/oracle/oracle-database-operator/commons/k8s" "github.com/oracle/oracle-database-operator/commons/oci" ) +// name of our custom finalizer +const ADB_FINALIZER = "database.oracle.com/adb-finalizer" + var requeueResult ctrl.Result = ctrl.Result{Requeue: true, RequeueAfter: 15 * time.Second} var emptyResult ctrl.Result = ctrl.Result{} @@ -90,13 +90,9 @@ type AutonomousDatabaseReconciler struct { // SetupWithManager function func (r *AutonomousDatabaseReconciler) SetupWithManager(mgr ctrl.Manager) error { return ctrl.NewControllerManagedBy(mgr). - For(&dbv1alpha1.AutonomousDatabase{}). - Watches( - &dbv1alpha1.AutonomousDatabaseBackup{}, - handler.EnqueueRequestsFromMapFunc(r.enqueueMapFn), - ). + For(&dbv4.AutonomousDatabase{}). Watches( - &dbv1alpha1.AutonomousDatabaseRestore{}, + &dbv4.AutonomousDatabaseRestore{}, handler.EnqueueRequestsFromMapFunc(r.enqueueMapFn), ). WithEventFilter(predicate.And(r.eventFilterPredicate(), r.watchPredicate())). @@ -121,22 +117,15 @@ func (r *AutonomousDatabaseReconciler) enqueueMapFn(ctx context.Context, o clien func (r *AutonomousDatabaseReconciler) watchPredicate() predicate.Predicate { return predicate.Funcs{ CreateFunc: func(e event.CreateEvent) bool { - _, backupOk := e.Object.(*dbv1alpha1.AutonomousDatabaseBackup) - _, restoreOk := e.Object.(*dbv1alpha1.AutonomousDatabaseRestore) + _, restoreOk := e.Object.(*dbv4.AutonomousDatabaseRestore) // Don't enqueue if the event is from Backup or Restore - return !(backupOk || restoreOk) + return !restoreOk }, UpdateFunc: func(e event.UpdateEvent) bool { // Enqueue the update event only when the status changes the first time - desiredBackup, backupOk := e.ObjectNew.(*dbv1alpha1.AutonomousDatabaseBackup) - if backupOk { - oldBackup := e.ObjectOld.(*dbv1alpha1.AutonomousDatabaseBackup) - return oldBackup.Status.LifecycleState == "" && desiredBackup.Status.LifecycleState != "" - } - - desiredRestore, restoreOk := e.ObjectNew.(*dbv1alpha1.AutonomousDatabaseRestore) + desiredRestore, restoreOk := e.ObjectNew.(*dbv4.AutonomousDatabaseRestore) if restoreOk { - oldRestore := e.ObjectOld.(*dbv1alpha1.AutonomousDatabaseRestore) + oldRestore := e.ObjectOld.(*dbv4.AutonomousDatabaseRestore) return oldRestore.Status.Status == "" && desiredRestore.Status.Status != "" } @@ -150,21 +139,17 @@ func (r *AutonomousDatabaseReconciler) eventFilterPredicate() predicate.Predicat return predicate.Funcs{ UpdateFunc: func(e event.UpdateEvent) bool { // source object can be AutonomousDatabase, AutonomousDatabaseBackup, or AutonomousDatabaseRestore - desiredADB, adbOk := e.ObjectNew.(*dbv1alpha1.AutonomousDatabase) + desiredAdb, adbOk := e.ObjectNew.(*dbv4.AutonomousDatabase) if adbOk { - oldADB := e.ObjectOld.(*dbv1alpha1.AutonomousDatabase) - - specChanged := !reflect.DeepEqual(oldADB.Spec, desiredADB.Spec) - statusChanged := !reflect.DeepEqual(oldADB.Status, desiredADB.Status) + oldAdb := e.ObjectOld.(*dbv4.AutonomousDatabase) - oldLastSucSpec := oldADB.GetAnnotations()[dbv1alpha1.LastSuccessfulSpec] - desiredLastSucSpec := desiredADB.GetAnnotations()[dbv1alpha1.LastSuccessfulSpec] - lastSucSpecChanged := oldLastSucSpec != desiredLastSucSpec + specChanged := !reflect.DeepEqual(oldAdb.Spec, desiredAdb.Spec) + statusChanged := !reflect.DeepEqual(oldAdb.Status, desiredAdb.Status) - if (!specChanged && statusChanged) || lastSucSpecChanged || - (controllerutil.ContainsFinalizer(oldADB, dbv1alpha1.ADB_FINALIZER) != controllerutil.ContainsFinalizer(desiredADB, dbv1alpha1.ADB_FINALIZER)) { + if (!specChanged && statusChanged) || + (controllerutil.ContainsFinalizer(oldAdb, ADB_FINALIZER) != controllerutil.ContainsFinalizer(desiredAdb, ADB_FINALIZER)) { // Don't enqueue in the folowing condition: - // 1. only status changes 2. lastSucSpec changes 3. ADB_FINALIZER changes + // 1. only status changes 2. ADB_FINALIZER changes return false } @@ -174,7 +159,7 @@ func (r *AutonomousDatabaseReconciler) eventFilterPredicate() predicate.Predicat }, DeleteFunc: func(e event.DeleteEvent) bool { // Do not trigger reconciliation when the object is deleted from the cluster. - _, adbOk := e.Object.(*dbv1alpha1.AutonomousDatabase) + _, adbOk := e.Object.(*dbv4.AutonomousDatabase) return !adbOk }, } @@ -189,158 +174,205 @@ func (r *AutonomousDatabaseReconciler) eventFilterPredicate() predicate.Predicat // +kubebuilder:rbac:groups="",resources=configmaps;secrets,verbs=get;list;watch;create;update;patch;delete // +kubebuilder:rbac:groups="",resources=events,verbs=create;patch -// Reconcile is the funtion that the operator calls every time when the reconciliation loop is triggered. -// It go to the beggining of the reconcile if an error is returned. We won't return a error if it is related -// to OCI, because the issues cannot be solved by re-run the reconcile. func (r *AutonomousDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { logger := r.Log.WithValues("Namespace/Name", req.NamespacedName) var err error - var ociADB *dbv1alpha1.AutonomousDatabase + // Indicates whether spec has been changed at the end of the reconcile. + var specChanged bool = false // Get the autonomousdatabase instance from the cluster - desiredADB := &dbv1alpha1.AutonomousDatabase{} - if err := r.KubeClient.Get(context.TODO(), req.NamespacedName, desiredADB); err != nil { + desiredAdb := &dbv4.AutonomousDatabase{} + if err := r.KubeClient.Get(context.TODO(), req.NamespacedName, desiredAdb); err != nil { // Ignore not-found errors, since they can't be fixed by an immediate requeue. - // No need to change the since we don't know if we obtain the object. if apiErrors.IsNotFound(err) { return emptyResult, nil } - // Failed to get ADB, so we don't need to update the status return emptyResult, err } /****************************************************************** * Get OCI database client ******************************************************************/ - if err := r.setupOCIClients(logger, desiredADB); err != nil { - logger.Error(err, "Fail to setup OCI clients") - - return r.manageError(logger.WithName("setupOCIClients"), desiredADB, err) + if err := r.setupOCIClients(logger, desiredAdb); err != nil { + return r.manageError( + logger.WithName("setupOCIClients"), + desiredAdb, + fmt.Errorf("Failed to get OCI Database Client: %w", err)) } logger.Info("OCI clients configured succesfully") /****************************************************************** - * Cleanup the resource if the resource is to be deleted. - * Deletion timestamp will be added to a object before it is deleted. - * Kubernetes server calls the clean up function if a finalizer exitsts, and won't delete the real object until - * all the finalizers are removed from the object metadata. - * Refer to this page for more details of using finalizers: https://kubernetes.io/blog/2022/05/14/using-finalizers-to-control-deletion/ + * Fill the empty fields in the local resource at the beginning of + * the reconciliation. ******************************************************************/ - exitReconcile, err := r.validateCleanup(logger, desiredADB) - if err != nil { - return r.manageError(logger.WithName("validateCleanup"), desiredADB, err) - } - - if exitReconcile { - return emptyResult, nil + // Fill the empty fields in the AutonomousDatabase resource by + // syncing up with the Autonomous Database in OCI. Only the fields + // that have nil values will be overwritten. + var stateBeforeFirstSync = desiredAdb.Status.LifecycleState + if _, err = r.syncAutonomousDatabase(logger, desiredAdb, false); err != nil { + return r.manageError( + logger.WithName("syncAutonomousDatabase"), + desiredAdb, + fmt.Errorf("Failed to sync AutonomousDatabase: %w", err)) + } + + // If the lifecycle state changes from any other states to + // AVAILABLE and spec.action is an empty string, it means that + // the resource in OCI just finished the work, and the spec + // of the Autonomous Database in OCI might also change. + // This is because OCI won't update the spec until the work + // completes. In this case, we need to update the spec of + // the resource in local cluster. + if stateBeforeFirstSync != database.AutonomousDatabaseLifecycleStateAvailable && + desiredAdb.Status.LifecycleState == database.AutonomousDatabaseLifecycleStateAvailable { + if specChanged, err = r.syncAutonomousDatabase(logger, desiredAdb, true); err != nil { + return r.manageError( + logger.WithName("syncAutonomousDatabase"), + desiredAdb, + fmt.Errorf("Failed to sync AutonomousDatabase: %w", err)) + } } /****************************************************************** - * Register/unregister the finalizer + * Determine if the external resource needs to be cleaned up. + * If yes, delete the Autonomous Database in OCI and exits the + * reconcile function immediately. + * + * There is no need to check the other fields if the resource is + * under deletion. This method should be executed soon after the OCI + * database client is obtained and the local resource is synced in + * the above two steps. + * + * Kubernetes server calls the clean up function if a finalizer exitsts, + * and won't delete the object until all the finalizers are removed + * from the object metadata. ******************************************************************/ - exit, err := r.validateFinalizer(logger, desiredADB) - if err != nil { - return r.manageError(logger.WithName("validateFinalizer"), desiredADB, err) - } + if desiredAdb.GetDeletionTimestamp().IsZero() { + // The Autonomous Database is not being deleted. Update the finalizer. + if desiredAdb.Spec.HardLink != nil && + *desiredAdb.Spec.HardLink && + !controllerutil.ContainsFinalizer(desiredAdb, ADB_FINALIZER) { + + if err := k8s.AddFinalizerAndPatch(r.KubeClient, desiredAdb, ADB_FINALIZER); err != nil { + return emptyResult, fmt.Errorf("Failed to add finalizer to Autonomous Database "+desiredAdb.Name+": %w", err) + } + } else if desiredAdb.Spec.HardLink != nil && + !*desiredAdb.Spec.HardLink && + controllerutil.ContainsFinalizer(desiredAdb, ADB_FINALIZER) { - if exit { - return emptyResult, nil + if err := k8s.RemoveFinalizerAndPatch(r.KubeClient, desiredAdb, ADB_FINALIZER); err != nil { + return emptyResult, fmt.Errorf("Failed to remove finalizer to Autonomous Database "+desiredAdb.Name+": %w", err) + } + } + } else { + // The Autonomous Database is being deleted + if controllerutil.ContainsFinalizer(desiredAdb, ADB_FINALIZER) { + if dbv4.IsAdbIntermediateState(desiredAdb.Status.LifecycleState) { + // No-op + } else if desiredAdb.Status.LifecycleState == database.AutonomousDatabaseLifecycleStateTerminated { + // The Autonomous Database in OCI has been deleted. Remove the finalizer. + if err := k8s.RemoveFinalizerAndPatch(r.KubeClient, desiredAdb, ADB_FINALIZER); err != nil { + return emptyResult, fmt.Errorf("Failed to remove finalizer to Autonomous Database "+desiredAdb.Name+": %w", err) + } + } else { + // Remove the Autonomous Database in OCI. + // Change the action to Terminate and proceed with the rest of the reconcile logic + desiredAdb.Spec.Action = "Terminate" + } + } } - /****************************************************************** - * Validate operations - ******************************************************************/ - modifiedADB := desiredADB.DeepCopy() // the ADB which stores the changes - exitReconcile, result, err := r.validateOperation(logger, modifiedADB, ociADB) - if err != nil { - return r.manageError(logger.WithName("validateOperation"), modifiedADB, err) - } - if exitReconcile { - return result, nil - } + if !dbv4.IsAdbIntermediateState(desiredAdb.Status.LifecycleState) { + /****************************************************************** + * Perform operations + ******************************************************************/ + var specChangedAfterOperation bool + specChangedAfterOperation, err = r.performOperation(logger, desiredAdb) + if err != nil { + return r.manageError( + logger.WithName("performOperation"), + desiredAdb, + fmt.Errorf("Failed to operate database action: %w", err)) + } - /***************************************************** - * Sync AutonomousDatabase Backups from OCI - *****************************************************/ - if err := r.syncBackupResources(logger, modifiedADB); err != nil { - return r.manageError(logger.WithName("syncBackupResources"), modifiedADB, err) - } + if specChangedAfterOperation { + specChanged = true + } - /***************************************************** - * Validate Wallet - *****************************************************/ - if err := r.validateWallet(logger, modifiedADB); err != nil { - return r.manageError(logger.WithName("validateWallet"), modifiedADB, err) - } + /****************************************************************** + * Sync AutonomousDatabase Backups from OCI. + * The backups will not be synced when the lifecycle state is + * TERMINATING or TERMINATED. + ******************************************************************/ + if desiredAdb.Status.LifecycleState != database.AutonomousDatabaseLifecycleStateTerminating && + desiredAdb.Status.LifecycleState != database.AutonomousDatabaseLifecycleStateTerminated { + if err := r.syncBackupResources(logger, desiredAdb); err != nil { + return r.manageError(logger.WithName("syncBackupResources"), desiredAdb, err) + } + } - /****************************************************************** - * Update the resource if the spec has been changed. - * This will trigger another reconcile, so returns with an empty - * result. - ******************************************************************/ - if !reflect.DeepEqual(modifiedADB.Spec, desiredADB.Spec) { - if err := r.KubeClient.Update(context.TODO(), modifiedADB); err != nil { - return r.manageError(logger.WithName("updateSpec"), modifiedADB, err) + /***************************************************** + * Validate Wallet + *****************************************************/ + if err := r.validateWallet(logger, desiredAdb); err != nil { + return r.manageError( + logger.WithName("validateWallet"), + desiredAdb, + fmt.Errorf("Failed to validate Wallet: %w", err)) } - return emptyResult, nil } /****************************************************************** - * Update the status at the end of every reconcile. + * Update the Autonomous Database at the end of every reconcile. ******************************************************************/ - copiedADB := modifiedADB.DeepCopy() - - updateCondition(modifiedADB, nil) - if err := r.KubeClient.Status().Update(context.TODO(), modifiedADB); err != nil { - return r.manageError(logger.WithName("Status().Update"), modifiedADB, err) + if specChanged { + if err := r.KubeClient.Update(context.TODO(), desiredAdb); err != nil { + return r.manageError( + logger.WithName("updateSpec"), + desiredAdb, + fmt.Errorf("Failed to update AutonomousDatabase spec: %w", err)) + } + // Immediately exit the reconcile loop if the resource is updated, and let + // the next run continue. + return emptyResult, nil } - modifiedADB.Spec = copiedADB.Spec - if dbv1alpha1.IsADBIntermediateState(modifiedADB.Status.LifecycleState) { - logger.WithName("IsADBIntermediateState").Info("LifecycleState is " + string(modifiedADB.Status.LifecycleState) + "; reconcile queued") - return requeueResult, nil + updateCondition(desiredAdb, nil) + if err := r.KubeClient.Status().Update(context.TODO(), desiredAdb); err != nil { + return r.manageError( + logger, + desiredAdb, + fmt.Errorf("Failed to update AutonomousDatabase status: %w", err)) } /****************************************************************** - * Update the lastSucSpec, and then finish the reconcile. - * Requeue if the ADB is terminated, but the finalizer is not yet - * removed. + * Requeue the request in the following cases: + * 1. the ADB is in intermediate state + * 2. the ADB is terminated, but the finalizer is not yet removed. ******************************************************************/ - - var requeue bool = false - if modifiedADB.GetDeletionTimestamp() != nil && - controllerutil.ContainsFinalizer(modifiedADB, dbv1alpha1.ADB_FINALIZER) && - modifiedADB.Status.LifecycleState == database.AutonomousDatabaseLifecycleStateTerminated { - logger.Info("The ADB is TERMINATED. The CR is to be deleted but finalizer is not yet removed; reconcile queued") - requeue = true - } - - if err := r.patchLastSuccessfulSpec(modifiedADB); err != nil { - return r.manageError(logger.WithName("patchLastSuccessfulSpec"), modifiedADB, err) - } - - if requeue { - logger.Info("Reconcile queued") + if dbv4.IsAdbIntermediateState(desiredAdb.Status.LifecycleState) { + logger. + WithName("IsAdbIntermediateState"). + Info("LifecycleState is " + string(desiredAdb.Status.LifecycleState) + "; reconciliation queued") return requeueResult, nil - } else { logger.Info("AutonomousDatabase reconciles successfully") return emptyResult, nil } } -func (r *AutonomousDatabaseReconciler) setupOCIClients(logger logr.Logger, adb *dbv1alpha1.AutonomousDatabase) error { +func (r *AutonomousDatabaseReconciler) setupOCIClients(logger logr.Logger, adb *dbv4.AutonomousDatabase) error { var err error - authData := oci.APIKeyAuth{ - ConfigMapName: adb.Spec.OCIConfig.ConfigMapName, - SecretName: adb.Spec.OCIConfig.SecretName, + authData := oci.ApiKeyAuth{ + ConfigMapName: adb.Spec.OciConfig.ConfigMapName, + SecretName: adb.Spec.OciConfig.SecretName, Namespace: adb.GetNamespace(), } - provider, err := oci.GetOCIProvider(r.KubeClient, authData) + provider, err := oci.GetOciProvider(r.KubeClient, authData) if err != nil { return err } @@ -353,894 +385,282 @@ func (r *AutonomousDatabaseReconciler) setupOCIClients(logger logr.Logger, adb * return nil } -func (r *AutonomousDatabaseReconciler) manageError(logger logr.Logger, adb *dbv1alpha1.AutonomousDatabase, err error) (ctrl.Result, error) { +// Upates the status with the error and returns an empty result +func (r *AutonomousDatabaseReconciler) manageError(logger logr.Logger, adb *dbv4.AutonomousDatabase, err error) (ctrl.Result, error) { l := logger.WithName("manageError") - if adb.Status.LifecycleState == "" { - // First time entering reconcile - updateCondition(adb, err) - l.Error(err, "CreateFailed") - - return emptyResult, nil - } else { - // Has synced at least once - var finalError = err - - // Roll back - ociADB := adb.DeepCopy() - specChanged, err := r.getADB(l, ociADB) - if err != nil { - finalError = k8s.CombineErrors(finalError, err) - } + l.Error(err, "Error occured") - // Will exit the Reconcile anyway after the manageError is called. - if specChanged { - // Clear the lifecycleState first to avoid the webhook error when update during an intermediate state - adb.Status.LifecycleState = "" - if err := r.KubeClient.Status().Update(context.TODO(), adb); err != nil { - finalError = k8s.CombineErrors(finalError, err) - } - - adb.Spec = ociADB.Spec - - if err := r.KubeClient.Update(context.TODO(), adb); err != nil { - finalError = k8s.CombineErrors(finalError, err) - } - } - - updateCondition(adb, err) - - l.Error(finalError, "UpdateFailed") - - return emptyResult, nil + updateCondition(adb, err) + if err := r.KubeClient.Status().Update(context.TODO(), adb); err != nil { + return emptyResult, fmt.Errorf("Failed to update status: %w", err) } + return emptyResult, nil } -const CONDITION_TYPE_COMPLETE = "Complete" -const CONDITION_REASON_COMPLETE = "ReconcileComplete" +const CONDITION_TYPE_AVAILABLE = "Available" +const CONDITION_REASON_AVAILABLE = "Available" +const CONDITION_TYPE_RECONCILE_QUEUED = "ReconcileQueued" +const CONDITION_REASON_RECONCILE_QUEUED = "LastReconcileQueued" +const CONDITION_TYPE_RECONCILE_ERROR = "ReconfileError" +const CONDITION_REASON_RECONCILE_ERROR = "LastReconcileError" -func updateCondition(adb *dbv1alpha1.AutonomousDatabase, err error) { +func updateCondition(adb *dbv4.AutonomousDatabase, err error) { var condition metav1.Condition + var errMsg string - errMsg := func() string { - if err != nil { - return err.Error() + if err != nil { + errMsg = err.Error() + } + + // Clean up the Conditions array + if len(adb.Status.Conditions) > 0 { + var allConditions = []string{ + CONDITION_TYPE_AVAILABLE, + CONDITION_TYPE_RECONCILE_QUEUED, + CONDITION_TYPE_RECONCILE_ERROR} + + for _, conditionType := range allConditions { + meta.RemoveStatusCondition(&adb.Status.Conditions, conditionType) } - return "no reconcile errors" - }() + } - // If error occurs, ReconcileComplete will be marked as true and the error message will still be listed - // If the ADB lifecycleState is intermediate, then ReconcileComplete will be marked as false + // If error occurs, the condition status will be marked as false and the error message will still be listed + // If the ADB lifecycleState is intermediate, then condition status will be marked as true + // Otherwise, then condition status will be marked as true if no error occurs if err != nil { condition = metav1.Condition{ - Type: CONDITION_TYPE_COMPLETE, + Type: CONDITION_TYPE_RECONCILE_ERROR, LastTransitionTime: metav1.Now(), ObservedGeneration: adb.GetGeneration(), - Reason: CONDITION_REASON_COMPLETE, + Reason: CONDITION_REASON_RECONCILE_ERROR, Message: errMsg, - Status: metav1.ConditionTrue, + Status: metav1.ConditionFalse, } - } else if dbv1alpha1.IsADBIntermediateState(adb.Status.LifecycleState) { + } else if dbv4.IsAdbIntermediateState(adb.Status.LifecycleState) { condition = metav1.Condition{ - Type: CONDITION_TYPE_COMPLETE, + Type: CONDITION_TYPE_RECONCILE_QUEUED, LastTransitionTime: metav1.Now(), ObservedGeneration: adb.GetGeneration(), - Reason: CONDITION_REASON_COMPLETE, - Message: errMsg, - Status: metav1.ConditionFalse, + Reason: CONDITION_REASON_RECONCILE_QUEUED, + Message: "no reconcile errors", + Status: metav1.ConditionTrue, } } else { condition = metav1.Condition{ - Type: CONDITION_TYPE_COMPLETE, + Type: CONDITION_TYPE_AVAILABLE, LastTransitionTime: metav1.Now(), ObservedGeneration: adb.GetGeneration(), - Reason: CONDITION_REASON_COMPLETE, - Message: errMsg, + Reason: CONDITION_REASON_AVAILABLE, + Message: "no reconcile errors", Status: metav1.ConditionTrue, } } - if len(adb.Status.Conditions) > 0 { - meta.RemoveStatusCondition(&adb.Status.Conditions, condition.Type) - } meta.SetStatusCondition(&adb.Status.Conditions, condition) } -func (r *AutonomousDatabaseReconciler) validateOperation( +func (r *AutonomousDatabaseReconciler) performOperation( logger logr.Logger, - adb *dbv1alpha1.AutonomousDatabase, - ociADB *dbv1alpha1.AutonomousDatabase) (exit bool, result ctrl.Result, err error) { - - lastSucSpec, err := adb.GetLastSuccessfulSpec() - if err != nil { - return false, emptyResult, err - } + adb *dbv4.AutonomousDatabase) (specChanged bool, err error) { l := logger.WithName("validateOperation") - // If lastSucSpec is nil, then it's CREATE or BIND opertaion - if lastSucSpec == nil { - if adb.Spec.Details.AutonomousDatabaseOCID == nil { - l.Info("Create operation") - err := r.createADB(logger, adb) - if err != nil { - return false, emptyResult, err - } - - // Update the ADB OCID - if err := r.updateCR(adb); err != nil { - return false, emptyResult, err - } - - l.Info("AutonomousDatabaseOCID updated; exit reconcile") - return true, emptyResult, nil - } else { - l.Info("Bind operation") - _, err := r.getADB(logger, adb) - if err != nil { - return false, emptyResult, err - } - - if err := r.updateCR(adb); err != nil { - return false, emptyResult, err - } - - l.Info("spec updated; exit reconcile") - return true, emptyResult, nil + switch adb.Spec.Action { + case "Create": + l.Info("Create operation") + err := r.createAutonomousDatabase(logger, adb) + if err != nil { + return false, err } - } - // If it's not CREATE or BIND opertaion, then it's UPDATE or SYNC operation. - // In most of the case the user changes the spec, and we update the oci ADB, but when the user updates on - // the Cloud Console, the controller cannot tell the direction and how to update the resource. - // Thus we compare the current spec with the lastSucSpec. If the details are different, it means that - // the user updates the spec (UPDATE operation), otherwise it's a SYNC operation. - lastDifADB := adb.DeepCopy() + adb.Spec.Action = "" + return true, nil - lastDetailsChanged, err := lastDifADB.RemoveUnchangedDetails(*lastSucSpec) - if err != nil { - return false, emptyResult, err - } + case "Sync": + l.Info("Sync operation") + _, err = r.syncAutonomousDatabase(logger, adb, true) + if err != nil { + return false, err + } - if lastDetailsChanged { - // Double check if the user input spec is actually different from the spec in OCI. If so, then update the resource. - // When the update completes and the status changes from UPDATING to AVAILABLE, the lastSucSpec is not updated yet, - // so we compare with the oci ADB again to make sure that the updates are completed. + adb.Spec.Action = "" + return true, nil + case "Update": l.Info("Update operation") - - exit, err := r.updateADB(logger, adb) + err = r.updateAutonomousDatabase(logger, adb) if err != nil { - return false, emptyResult, err + return false, err } - return exit, emptyResult, nil + adb.Spec.Action = "" + return true, nil - } else { - l.Info("No operation specified; sync the resource") + case "Stop": + l.Info("Sending StopAutonomousDatabase request to OCI") - // The user doesn't change the spec and the controller should pull the spec from the OCI. - specChanged, err := r.getADB(logger, adb) + resp, err := r.dbService.StopAutonomousDatabase(*adb.Spec.Details.Id) if err != nil { - return false, emptyResult, err + return false, err } - if specChanged { - l.Info("The local spec doesn't match the oci's spec; update the CR") - - // Erase the status.lifecycleState temporarily to avoid the webhook error. - tmpADB := adb.DeepCopy() - adb.Status.LifecycleState = "" - if err := r.KubeClient.Status().Update(context.TODO(), adb); err != nil { - return false, emptyResult, err - } - adb.Spec = tmpADB.Spec + adb.Spec.Action = "" + adb.Status.LifecycleState = resp.LifecycleState + return true, nil - if err := r.updateCR(adb); err != nil { - return false, emptyResult, err - } + case "Start": + l.Info("Sending StartAutonomousDatabase request to OCI") - return true, emptyResult, nil + resp, err := r.dbService.StartAutonomousDatabase(*adb.Spec.Details.Id) + if err != nil { + return false, err } - return false, emptyResult, nil - } -} -func (r *AutonomousDatabaseReconciler) validateCleanup(logger logr.Logger, adb *dbv1alpha1.AutonomousDatabase) (exitReconcile bool, err error) { - l := logger.WithName("validateCleanup") - - isADBToBeDeleted := adb.GetDeletionTimestamp() != nil - - if !isADBToBeDeleted { - return false, nil - } + adb.Spec.Action = "" + adb.Status.LifecycleState = resp.LifecycleState + return true, nil - if controllerutil.ContainsFinalizer(adb, dbv1alpha1.ADB_FINALIZER) { - if adb.Status.LifecycleState == database.AutonomousDatabaseLifecycleStateTerminating { - // Delete in progress, continue with the reconcile logic - return false, nil - } + case "Terminate": + // OCI only allows terminate operation when the ADB is in an valid state, otherwise requeue the reconcile. + if dbv4.CanBeTerminated(adb.Status.LifecycleState) { + l.Info("Sending DeleteAutonomousDatabase request to OCI") - if adb.Status.LifecycleState == database.AutonomousDatabaseLifecycleStateTerminated { - // The adb has been deleted. Remove the finalizer and exit the reconcile. - // Once all finalizers have been removed, the object will be deleted. - l.Info("Resource is in TERMINATED state; remove the finalizer") - if err := k8s.RemoveFinalizerAndPatch(r.KubeClient, adb, dbv1alpha1.ADB_FINALIZER); err != nil { + _, err := r.dbService.DeleteAutonomousDatabase(*adb.Spec.Details.Id) + if err != nil { return false, err } - return true, nil - } - if adb.Spec.Details.AutonomousDatabaseOCID == nil { - l.Info("Missing AutonomousDatabaseOCID to terminate Autonomous Database; remove the finalizer anyway", "Name", adb.Name, "Namespace", adb.Namespace) - // Remove finalizer anyway. - if err := k8s.RemoveFinalizerAndPatch(r.KubeClient, adb, dbv1alpha1.ADB_FINALIZER); err != nil { + if err := r.removeBackupResources(l, adb); err != nil { return false, err } - return true, nil - } - if adb.Spec.Details.LifecycleState != database.AutonomousDatabaseLifecycleStateTerminated { - // Run finalization logic for finalizer. If the finalization logic fails, don't remove the finalizer so - // that we can retry during the next reconciliation. - l.Info("Terminating Autonomous Database") - adb.Spec.Details.LifecycleState = database.AutonomousDatabaseLifecycleStateTerminated - if err := r.KubeClient.Update(context.TODO(), adb); err != nil { - return false, err - } - // Exit the reconcile since we have updated the spec - return true, nil + adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateTerminating + } else if dbv4.IsAdbIntermediateState(adb.Status.LifecycleState) { + l.Info("Can not terminate an ADB in an intermediate state; exit reconcile") } - // Continue with the reconcile logic - return false, nil - } - - // Exit the Reconcile since the to-be-deleted resource doesn't has a finalizer - return true, nil -} - -func (r *AutonomousDatabaseReconciler) validateFinalizer(logger logr.Logger, adb *dbv1alpha1.AutonomousDatabase) (exit bool, err error) { - l := logger.WithName("validateFinalizer") - - // Delete is not schduled. Update the finalizer for this CR if hardLink is present - var finalizerChanged = false - if adb.Spec.HardLink != nil { - if *adb.Spec.HardLink && !controllerutil.ContainsFinalizer(adb, dbv1alpha1.ADB_FINALIZER) { - l.Info("Finalizer added") - if err := k8s.AddFinalizerAndPatch(r.KubeClient, adb, dbv1alpha1.ADB_FINALIZER); err != nil { - return false, err - } - - finalizerChanged = true + adb.Spec.Action = "" + return true, nil - } else if !*adb.Spec.HardLink && controllerutil.ContainsFinalizer(adb, dbv1alpha1.ADB_FINALIZER) { - l.Info("Finalizer removed") + case "Clone": + resp, err := r.dbService.CreateAutonomousDatabaseClone(adb) + if err != nil { + return false, err + } + adb.Status.LifecycleState = resp.LifecycleState - if err := k8s.RemoveFinalizerAndPatch(r.KubeClient, adb, dbv1alpha1.ADB_FINALIZER); err != nil { - return false, err - } + adb.Spec.Action = "" - finalizerChanged = true + // Create cloned Autonomous Database resource + clonedAdb := &dbv4.AutonomousDatabase{ + ObjectMeta: metav1.ObjectMeta{ + Name: *adb.Spec.Clone.DisplayName, + Namespace: adb.Namespace, + }, + Spec: dbv4.AutonomousDatabaseSpec{ + OciConfig: *adb.Spec.OciConfig.DeepCopy(), + }, + } + clonedAdb.UpdateFromOciAdb(resp.AutonomousDatabase, true) + if err := r.KubeClient.Create(context.TODO(), clonedAdb); err != nil { + return false, err } - } - - // If the finalizer is changed during an intermediate state, e.g. set hardLink to true and - // delete the resource, then there must be another ongoing reconcile. In this case we should - // exit the reconcile. - if finalizerChanged && dbv1alpha1.IsADBIntermediateState(adb.Status.LifecycleState) { - l.Info("Finalizer changed during an intermediate state, exit the reconcile") return true, nil - } - return false, nil -} - -// updateCR updates the lastSucSpec and the CR -func (r *AutonomousDatabaseReconciler) updateCR(adb *dbv1alpha1.AutonomousDatabase) error { - // Update the lastSucSpec - // Should patch the lastSuccessfulSpec first, otherwise, the update event will be - // filtered out by predicate since the lastSuccessfulSpec is changed. - if err := r.patchLastSuccessfulSpec(adb); err != nil { - return err - } - - if err := r.KubeClient.Update(context.TODO(), adb); err != nil { - return err - } - return nil -} - -func (r *AutonomousDatabaseReconciler) patchLastSuccessfulSpec(adb *dbv1alpha1.AutonomousDatabase) error { - copyADB := adb.DeepCopy() - - specBytes, err := json.Marshal(adb.Spec) - if err != nil { - return err - } - - anns := map[string]string{ - dbv1alpha1.LastSuccessfulSpec: string(specBytes), + case "": + // No-op + return false, nil + default: + adb.Spec.Action = "" + return true, errors.New("Unknown action: " + adb.Spec.Action) } - - annotations.PatchAnnotations(r.KubeClient, adb, anns) - - adb.Spec = copyADB.Spec - adb.Status = copyADB.Status - - return nil } -func (r *AutonomousDatabaseReconciler) createADB(logger logr.Logger, adb *dbv1alpha1.AutonomousDatabase) error { +func (r *AutonomousDatabaseReconciler) createAutonomousDatabase(logger logr.Logger, adb *dbv4.AutonomousDatabase) error { logger.WithName("createADB").Info("Sending CreateAutonomousDatabase request to OCI") resp, err := r.dbService.CreateAutonomousDatabase(adb) if err != nil { return err } - // Restore the admin password after updating from OCI ADB - adminPass := adb.Spec.Details.AdminPassword - adb.UpdateFromOCIADB(resp.AutonomousDatabase) - adb.Spec.Details.AdminPassword = adminPass + adb.UpdateFromOciAdb(resp.AutonomousDatabase, true) return nil } -// getADB gets the information from OCI and overwrites the spec and the status, but not update the CR in the cluster -func (r *AutonomousDatabaseReconciler) getADB(logger logr.Logger, adb *dbv1alpha1.AutonomousDatabase) (bool, error) { - if adb == nil { - return false, errors.New("AutonomousDatabase OCID is missing") +// syncAutonomousDatabase retrieve the information of AutonomousDatabase from +// OCI and "overwrite" decides whether the spec and the status of "adb" will +// be overwritten. +// It will be a no-op if "Spec.Details.AutonomousDatabaseOCID" of the provided +// AutonomousDatabase is nil. +// This method does not update the actual resource in the cluster. +// +// The returned values are: +// 1. bool: indicates whether the spec is changed after the sync +// 2. error: not nil if an error occurs during the sync +func (r *AutonomousDatabaseReconciler) syncAutonomousDatabase( + logger logr.Logger, + adb *dbv4.AutonomousDatabase, overwrite bool) (specChanged bool, err error) { + if adb.Spec.Details.Id == nil { + return false, nil } - l := logger.WithName("getADB") + l := logger.WithName("syncAutonomousDatabase") // Get the information from OCI l.Info("Sending GetAutonomousDatabase request to OCI") - resp, err := r.dbService.GetAutonomousDatabase(*adb.Spec.Details.AutonomousDatabaseOCID) + resp, err := r.dbService.GetAutonomousDatabase(*adb.Spec.Details.Id) if err != nil { return false, err } - specChanged := adb.UpdateFromOCIADB(resp.AutonomousDatabase) - + specChanged = adb.UpdateFromOciAdb(resp.AutonomousDatabase, overwrite) return specChanged, nil } -// updateADB returns true if an OCI request is sent. +// updateAutonomousDatabase returns true if an OCI request is sent. // The AutonomousDatabase is updated with the returned object from the OCI requests. -func (r *AutonomousDatabaseReconciler) updateADB( +func (r *AutonomousDatabaseReconciler) updateAutonomousDatabase( logger logr.Logger, - adb *dbv1alpha1.AutonomousDatabase) (exit bool, err error) { - - l := logger.WithName("updateADB") + adb *dbv4.AutonomousDatabase) (err error) { // Get OCI AutonomousDatabase and update the lifecycleState of the CR, // so that the validatexx functions know when the state changes back to AVAILABLE - ociADB := adb.DeepCopy() - _, err = r.getADB(logger, ociADB) + ociAdb := adb.DeepCopy() + _, err = r.syncAutonomousDatabase(logger, ociAdb, true) if err != nil { - return false, err + return err } - adb.Status.LifecycleState = ociADB.Status.LifecycleState - // Start update - difADB := adb.DeepCopy() + // difAdb is used to store ONLY the values of Autonomous Database that are + // difference from the one in OCI + difAdb := adb.DeepCopy() - ociDetailsChanged, err := difADB.RemoveUnchangedDetails(ociADB.Spec) + detailsAreChanged, err := difAdb.RemoveUnchangedDetails(ociAdb.Spec) if err != nil { - return false, err + return err } // Do the update request only if the current ADB is actually different from the OCI ADB - if ociDetailsChanged { - // Special case: if the oci ADB is terminating, then update the spec and exit the reconcile. - // This happens when the lifecycleState changes to TERMINATED during an intermediate state, - // whatever is in progress should be abandonded and the desired spec should the same as oci ADB. - if ociADB.Status.LifecycleState == database.AutonomousDatabaseLifecycleStateTerminating { - l.Info("OCI ADB is in TERMINATING state; update the spec and exit the reconcile") - - adb.Status.LifecycleState = "" - if err := r.KubeClient.Status().Update(context.TODO(), adb); err != nil { - return false, err - } - - adb.Spec = ociADB.Spec - if err := r.KubeClient.Update(context.TODO(), adb); err != nil { - return false, err - } - return true, nil - } - - // Special case: if the lifecycleState is changed, it might have to exit the reconcile in some cases. - sent, exit, err := r.validateDesiredLifecycleState(logger, adb, difADB, ociADB) - if err != nil { - return false, err - } - if sent { - return exit, nil - } - - validations := []func(logr.Logger, *dbv1alpha1.AutonomousDatabase, *dbv1alpha1.AutonomousDatabase, *dbv1alpha1.AutonomousDatabase) (bool, error){ - r.validateGeneralFields, - r.validateAdminPassword, - r.validateDbWorkload, - r.validateLicenseModel, - r.validateScalingFields, - r.validateGeneralNetworkAccess, - } - - for _, op := range validations { - sent, err := op(logger, adb, difADB, ociADB) - if err != nil { - return false, err - } - - if sent { - return false, nil - } - } - } - - return false, nil -} - -func (r *AutonomousDatabaseReconciler) validateGeneralFields( - logger logr.Logger, - adb *dbv1alpha1.AutonomousDatabase, - difADB *dbv1alpha1.AutonomousDatabase, - ociADB *dbv1alpha1.AutonomousDatabase) (sent bool, err error) { - - if difADB.Spec.Details.DisplayName == nil && - difADB.Spec.Details.DbName == nil && - difADB.Spec.Details.DbVersion == nil && - difADB.Spec.Details.FreeformTags == nil { - return false, nil - } + if detailsAreChanged { + logger.Info("Sending UpdateAutonomousDatabase request to OCI") - if ociADB.Status.LifecycleState != database.AutonomousDatabaseLifecycleStateAvailable { - return false, nil - } - - l := logger.WithName("validateGeneralFields") - - l.Info("Sending UpdateAutonomousDatabase request to OCI") - resp, err := r.dbService.UpdateAutonomousDatabaseGeneralFields(*adb.Spec.Details.AutonomousDatabaseOCID, difADB) - if err != nil { - return false, err - } - - adb.UpdateFromOCIADB(resp.AutonomousDatabase) - - return true, nil -} - -// Special case: compare with lastSpec but not ociSpec -func (r *AutonomousDatabaseReconciler) validateAdminPassword( - logger logr.Logger, - adb *dbv1alpha1.AutonomousDatabase, - difADB *dbv1alpha1.AutonomousDatabase, - ociADB *dbv1alpha1.AutonomousDatabase) (sent bool, err error) { - - if difADB.Spec.Details.AdminPassword.K8sSecret.Name == nil && - difADB.Spec.Details.AdminPassword.OCISecret.OCID == nil { - return false, nil - } - - if ociADB.Status.LifecycleState != database.AutonomousDatabaseLifecycleStateAvailable { - return false, nil - } - - l := logger.WithName("validateAdminPassword") - - l.Info("Sending UpdateAutonomousDatabase request to OCI") - resp, err := r.dbService.UpdateAutonomousDatabaseAdminPassword(*adb.Spec.Details.AutonomousDatabaseOCID, difADB) - if err != nil { - return false, err - } - - adb.UpdateFromOCIADB(resp.AutonomousDatabase) - // Update the admin password fields because they are missing in the ociADB - adb.Spec.Details.AdminPassword = difADB.Spec.Details.AdminPassword - - return true, nil -} - -func (r *AutonomousDatabaseReconciler) validateDbWorkload( - logger logr.Logger, - adb *dbv1alpha1.AutonomousDatabase, - difADB *dbv1alpha1.AutonomousDatabase, - ociADB *dbv1alpha1.AutonomousDatabase) (sent bool, err error) { - - if difADB.Spec.Details.DbWorkload == "" { - return false, nil - } - - if ociADB.Status.LifecycleState != database.AutonomousDatabaseLifecycleStateAvailable { - return false, nil - } - - l := logger.WithName("validateDbWorkload") - - l.Info("Sending UpdateAutonomousDatabase request to OCI") - resp, err := r.dbService.UpdateAutonomousDatabaseDBWorkload(*adb.Spec.Details.AutonomousDatabaseOCID, difADB) - if err != nil { - return false, err - } - - adb.UpdateFromOCIADB(resp.AutonomousDatabase) - - return true, nil -} - -func (r *AutonomousDatabaseReconciler) validateLicenseModel( - logger logr.Logger, - adb *dbv1alpha1.AutonomousDatabase, - difADB *dbv1alpha1.AutonomousDatabase, - ociADB *dbv1alpha1.AutonomousDatabase) (sent bool, err error) { - - if difADB.Spec.Details.LicenseModel == "" { - return false, nil - } - - if ociADB.Status.LifecycleState != database.AutonomousDatabaseLifecycleStateAvailable { - return false, nil - } - - l := logger.WithName("validateLicenseModel") - - l.Info("Sending UpdateAutonomousDatabase request to OCI") - resp, err := r.dbService.UpdateAutonomousDatabaseLicenseModel(*adb.Spec.Details.AutonomousDatabaseOCID, difADB) - if err != nil { - return false, err - } - - adb.UpdateFromOCIADB(resp.AutonomousDatabase) - - return true, nil -} - -func (r *AutonomousDatabaseReconciler) validateScalingFields( - logger logr.Logger, - adb *dbv1alpha1.AutonomousDatabase, - difADB *dbv1alpha1.AutonomousDatabase, - ociADB *dbv1alpha1.AutonomousDatabase) (sent bool, err error) { - - if difADB.Spec.Details.DataStorageSizeInTBs == nil && - difADB.Spec.Details.CPUCoreCount == nil && - difADB.Spec.Details.IsAutoScalingEnabled == nil { - return false, nil - } - - if ociADB.Status.LifecycleState != database.AutonomousDatabaseLifecycleStateAvailable { - return false, nil - } - - l := logger.WithName("validateScalingFields") - - l.Info("Sending UpdateAutonomousDatabase request to OCI") - resp, err := r.dbService.UpdateAutonomousDatabaseScalingFields(*adb.Spec.Details.AutonomousDatabaseOCID, difADB) - if err != nil { - return false, err - } - - adb.UpdateFromOCIADB(resp.AutonomousDatabase) - - return true, nil -} - -func (r *AutonomousDatabaseReconciler) validateDesiredLifecycleState( - logger logr.Logger, - adb *dbv1alpha1.AutonomousDatabase, - difADB *dbv1alpha1.AutonomousDatabase, - ociADB *dbv1alpha1.AutonomousDatabase) (sent bool, exit bool, err error) { - - if difADB.Spec.Details.LifecycleState == "" { - return false, false, nil - } - - if difADB.Spec.Details.LifecycleState == database.AutonomousDatabaseLifecycleStateTerminated { - // OCI only allows terminate operation when the ADB is in an valid state, otherwise requeue the reconcile. - if !dbv1alpha1.ValidADBTerminateState(adb.Status.LifecycleState) { - return false, false, nil - } - } else if dbv1alpha1.IsADBIntermediateState(ociADB.Status.LifecycleState) { - // Other lifecycle management operation; requeue the reconcile if it's in an intermediate state - return false, false, nil - } - - l := logger.WithName("validateDesiredLifecycleState") - - switch difADB.Spec.Details.LifecycleState { - case database.AutonomousDatabaseLifecycleStateAvailable: - l.Info("Sending StartAutonomousDatabase request to OCI") - - resp, err := r.dbService.StartAutonomousDatabase(*adb.Spec.Details.AutonomousDatabaseOCID) - if err != nil { - return false, false, err - } - - adb.Status.LifecycleState = resp.LifecycleState - case database.AutonomousDatabaseLifecycleStateStopped: - l.Info("Sending StopAutonomousDatabase request to OCI") - - resp, err := r.dbService.StopAutonomousDatabase(*adb.Spec.Details.AutonomousDatabaseOCID) - if err != nil { - return false, false, err - } - - adb.Status.LifecycleState = resp.LifecycleState - case database.AutonomousDatabaseLifecycleStateTerminated: - l.Info("Sending DeleteAutonomousDatabase request to OCI") - - _, err := r.dbService.DeleteAutonomousDatabase(*adb.Spec.Details.AutonomousDatabaseOCID) - if err != nil { - return false, false, err - } - - adb.Status.LifecycleState = database.AutonomousDatabaseLifecycleStateTerminating - - // The controller allows terminate during some intermediate states. - // Exit the reconcile because there is already another ongoing reconcile. - if dbv1alpha1.IsADBIntermediateState(ociADB.Status.LifecycleState) { - l.Info("Terminating an ADB which is in an intermediate state; exit reconcile") - return true, true, nil - } - default: - return false, false, errors.New("unknown lifecycleState") - } - - return true, false, nil -} - -// The logic of updating the network access configurations is as follows: -// -// 1. Shared databases: -// If the network access type changes -// a. to PUBLIC: -// was RESTRICTED: re-enable IsMTLSConnectionRequired if its not. Then set WhitelistedIps to an array with a single empty string entry. -// was PRIVATE: re-enable IsMTLSConnectionRequired if its not. Then set PrivateEndpointLabel to an emtpy string. -// b. to RESTRICTED: -// was PUBLIC: set WhitelistedIps to desired IPs/CIDR blocks/VCN OCID. Configure the IsMTLSConnectionRequired settings if it is set to disabled. -// was PRIVATE: re-enable IsMTLSConnectionRequired if its not. Set the type to PUBLIC first, and then configure the WhitelistedIps. Finally resume the IsMTLSConnectionRequired settings if it was, or is configured as disabled. -// c. to PRIVATE: -// was PUBLIC: set subnetOCID and nsgOCIDs. Configure the IsMTLSConnectionRequired settings if it is set. -// was RESTRICTED: set subnetOCID and nsgOCIDs. Configure the IsMTLSConnectionRequired settings if it is set. -// *Note: OCI requires nsgOCIDs to be an empty string rather than nil when we don't want the adb to be included in any network security group. -// -// Otherwise, if the network access type remains the same, apply the network configuration, and then set the IsMTLSConnectionRequired. -// -// 2. Dedicated databases: -// Apply the configs directly -func (r *AutonomousDatabaseReconciler) validateGeneralNetworkAccess( - logger logr.Logger, - adb *dbv1alpha1.AutonomousDatabase, - difADB *dbv1alpha1.AutonomousDatabase, - ociADB *dbv1alpha1.AutonomousDatabase) (sent bool, err error) { - - if difADB.Spec.Details.NetworkAccess.AccessType == "" && - difADB.Spec.Details.NetworkAccess.IsAccessControlEnabled == nil && - difADB.Spec.Details.NetworkAccess.AccessControlList == nil && - difADB.Spec.Details.NetworkAccess.IsMTLSConnectionRequired == nil && - difADB.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID == nil && - difADB.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs == nil && - difADB.Spec.Details.NetworkAccess.PrivateEndpoint.HostnamePrefix == nil { - return false, nil - } - - if ociADB.Status.LifecycleState != database.AutonomousDatabaseLifecycleStateAvailable { - return false, nil - } - - l := logger.WithName("validateGeneralNetworkAccess") - - if !*adb.Spec.Details.IsDedicated { - var lastAccessType = ociADB.Spec.Details.NetworkAccess.AccessType - var difAccessType = difADB.Spec.Details.NetworkAccess.AccessType - - if difAccessType != "" { - switch difAccessType { - case dbv1alpha1.NetworkAccessTypePublic: - l.Info("Configuring network access type to PUBLIC") - // OCI validation requires IsMTLSConnectionRequired to be enabled before changing the network access type to PUBLIC - if !*ociADB.Spec.Details.NetworkAccess.IsMTLSConnectionRequired { - if err := r.setMTLSRequired(logger, adb); err != nil { - return false, err - } - return true, nil - } - - if err := r.setNetworkAccessPublic(logger, ociADB.Spec.Details.NetworkAccess.AccessType, adb); err != nil { - return false, err - } - return true, nil - case dbv1alpha1.NetworkAccessTypeRestricted: - l.Info("Configuring network access type to RESTRICTED") - // If the access type was PRIVATE, then OCI validation requires IsMTLSConnectionRequired - // to be enabled before setting ACL. Also, we can only change the network access type from - // PRIVATE to PUBLIC, so the steps are PRIVATE->(requeue)->PUBLIC->(requeue)->RESTRICTED. - if lastAccessType == dbv1alpha1.NetworkAccessTypePrivate { - if !*ociADB.Spec.Details.NetworkAccess.IsMTLSConnectionRequired { - if err := r.setMTLSRequired(logger, adb); err != nil { - return false, err - } - return true, nil - } - - if err := r.setNetworkAccessPublic(logger, ociADB.Spec.Details.NetworkAccess.AccessType, adb); err != nil { - return false, err - } - return true, nil - } - - sent, err := r.validateNetworkAccess(logger, adb, difADB, ociADB) - if err != nil { - return false, err - } - if sent { - return true, nil - } - - sent, err = r.validateMTLS(logger, adb, difADB, ociADB) - if err != nil { - return false, err - } - if sent { - return true, nil - } - case dbv1alpha1.NetworkAccessTypePrivate: - l.Info("Configuring network access type to PRIVATE") - - sent, err := r.validateNetworkAccess(logger, adb, difADB, ociADB) - if err != nil { - return false, err - } - if sent { - return true, nil - } - - sent, err = r.validateMTLS(logger, adb, difADB, ociADB) - if err != nil { - return false, err - } - if sent { - return true, nil - } - } - } else { - // Access type doesn't change - sent, err := r.validateNetworkAccess(logger, adb, difADB, ociADB) - if err != nil { - return false, err - } - if sent { - return true, nil - } - - sent, err = r.validateMTLS(logger, adb, difADB, ociADB) - if err != nil { - return false, err - } - if sent { - return true, nil - } - } - } else { - // Dedicated database - sent, err := r.validateNetworkAccess(logger, adb, difADB, ociADB) + resp, err := r.dbService.UpdateAutonomousDatabase(*adb.Spec.Details.Id, difAdb) if err != nil { - return false, err - } - if sent { - return true, nil + return err } + _ = adb.UpdateFromOciAdb(resp.AutonomousDatabase, true) } - return false, nil -} - -// Set the mTLS to true but not changing the spec -func (r *AutonomousDatabaseReconciler) setMTLSRequired(logger logr.Logger, adb *dbv1alpha1.AutonomousDatabase) error { - l := logger.WithName("setMTLSRequired") - - l.Info("Sending request to OCI to set IsMtlsConnectionRequired to true") - - adb.Spec.Details.NetworkAccess.IsMTLSConnectionRequired = common.Bool(true) - - resp, err := r.dbService.UpdateNetworkAccessMTLSRequired(*adb.Spec.Details.AutonomousDatabaseOCID) - if err != nil { - return err - } - - adb.UpdateFromOCIADB(resp.AutonomousDatabase) - return nil } -func (r *AutonomousDatabaseReconciler) validateMTLS( - logger logr.Logger, - adb *dbv1alpha1.AutonomousDatabase, - difADB *dbv1alpha1.AutonomousDatabase, - ociADB *dbv1alpha1.AutonomousDatabase) (sent bool, err error) { - - if difADB.Spec.Details.NetworkAccess.IsMTLSConnectionRequired == nil { - return false, nil - } - - l := logger.WithName("validateMTLS") - - l.Info("Sending request to OCI to configure IsMtlsConnectionRequired") - - resp, err := r.dbService.UpdateNetworkAccessMTLS(*adb.Spec.Details.AutonomousDatabaseOCID, difADB) - if err != nil { - return false, err - } - - adb.UpdateFromOCIADB(resp.AutonomousDatabase) - - return true, nil -} - -func (r *AutonomousDatabaseReconciler) setNetworkAccessPublic(logger logr.Logger, lastAcessType dbv1alpha1.NetworkAccessTypeEnum, adb *dbv1alpha1.AutonomousDatabase) error { - adb.Spec.Details.NetworkAccess.AccessType = dbv1alpha1.NetworkAccessTypePublic - adb.Spec.Details.NetworkAccess.AccessControlList = nil - adb.Spec.Details.NetworkAccess.PrivateEndpoint.HostnamePrefix = common.String("") - adb.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs = nil - adb.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID = nil - - l := logger.WithName("setNetworkAccessPublic") - - l.Info("Sending request to OCI to configure network access options to PUBLIC") - - resp, err := r.dbService.UpdateNetworkAccessPublic(lastAcessType, *adb.Spec.Details.AutonomousDatabaseOCID) - if err != nil { - return err - } - - adb.UpdateFromOCIADB(resp.AutonomousDatabase) - - return nil -} - -func (r *AutonomousDatabaseReconciler) validateNetworkAccess( - logger logr.Logger, - adb *dbv1alpha1.AutonomousDatabase, - difADB *dbv1alpha1.AutonomousDatabase, - ociADB *dbv1alpha1.AutonomousDatabase) (sent bool, err error) { - - if difADB.Spec.Details.NetworkAccess.AccessType == "" && - difADB.Spec.Details.NetworkAccess.IsAccessControlEnabled == nil && - difADB.Spec.Details.NetworkAccess.AccessControlList == nil && - difADB.Spec.Details.NetworkAccess.PrivateEndpoint.SubnetOCID == nil && - difADB.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs == nil && - difADB.Spec.Details.NetworkAccess.PrivateEndpoint.HostnamePrefix == nil { - return false, nil - } - - l := logger.WithName("validateNetworkAccess") - - l.Info("Sending request to OCI to configure network access options") - - // When the network access type is set to PRIVATE, any nil type of nsgOCIDs needs to be set to an empty string, otherwise, OCI SDK returns a 400 error - if difADB.Spec.Details.NetworkAccess.AccessType == dbv1alpha1.NetworkAccessTypePrivate && - difADB.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs == nil { - difADB.Spec.Details.NetworkAccess.PrivateEndpoint.NsgOCIDs = []string{} - } - - resp, err := r.dbService.UpdateNetworkAccess(*adb.Spec.Details.AutonomousDatabaseOCID, difADB) - if err != nil { - return false, err - } - - adb.UpdateFromOCIADB(resp.AutonomousDatabase) - - return true, nil -} - -func (r *AutonomousDatabaseReconciler) validateWallet(logger logr.Logger, adb *dbv1alpha1.AutonomousDatabase) error { - if adb.Spec.Details.Wallet.Name == nil && - adb.Spec.Details.Wallet.Password.K8sSecret.Name == nil && - adb.Spec.Details.Wallet.Password.OCISecret.OCID == nil { +func (r *AutonomousDatabaseReconciler) validateWallet(logger logr.Logger, adb *dbv4.AutonomousDatabase) error { + if adb.Spec.Wallet.Name == nil && + adb.Spec.Wallet.Password.K8sSecret.Name == nil && + adb.Spec.Wallet.Password.OciSecret.Id == nil { return nil } @@ -1253,10 +673,10 @@ func (r *AutonomousDatabaseReconciler) validateWallet(logger logr.Logger, adb *d // lastSucSpec may be nil if this is the first time entering the reconciliation loop var walletName string - if adb.Spec.Details.Wallet.Name == nil { + if adb.Spec.Wallet.Name == nil { walletName = adb.GetName() + "-instance-wallet" } else { - walletName = *adb.Spec.Details.Wallet.Name + walletName = *adb.Spec.Wallet.Name } secret, err := k8s.FetchSecret(r.KubeClient, adb.GetNamespace(), walletName) @@ -1297,11 +717,11 @@ func (r *AutonomousDatabaseReconciler) validateWallet(logger logr.Logger, adb *d // updateBackupResources get the list of AutonomousDatabasBackups and // create a backup object if it's not found in the same namespace -func (r *AutonomousDatabaseReconciler) syncBackupResources(logger logr.Logger, adb *dbv1alpha1.AutonomousDatabase) error { +func (r *AutonomousDatabaseReconciler) syncBackupResources(logger logr.Logger, adb *dbv4.AutonomousDatabase) error { l := logger.WithName("syncBackupResources") // Get the list of AutonomousDatabaseBackupOCID in the same namespace - backupList, err := k8s.FetchAutonomousDatabaseBackups(r.KubeClient, adb.Namespace) + backupList, err := k8s.FetchAutonomousDatabaseBackups(r.KubeClient, adb.Namespace, adb.Name) if err != nil { return err } @@ -1319,7 +739,7 @@ func (r *AutonomousDatabaseReconciler) syncBackupResources(logger logr.Logger, a } } - resp, err := r.dbService.ListAutonomousDatabaseBackups(*adb.Spec.Details.AutonomousDatabaseOCID) + resp, err := r.dbService.ListAutonomousDatabaseBackups(*adb.Spec.Details.Id) if err != nil { return err } @@ -1370,7 +790,7 @@ func (r *AutonomousDatabaseReconciler) getValidBackupName(displayName string, us return finalName, nil } -func (r *AutonomousDatabaseReconciler) ifBackupExists(backupSummary database.AutonomousDatabaseBackupSummary, curBackupOCIDs map[string]bool, backupList *dbv1alpha1.AutonomousDatabaseBackupList) bool { +func (r *AutonomousDatabaseReconciler) ifBackupExists(backupSummary database.AutonomousDatabaseBackupSummary, curBackupOCIDs map[string]bool, backupList *dbv4.AutonomousDatabaseBackupList) bool { _, ok := curBackupOCIDs[*backupSummary.Id] if ok { return true @@ -1391,3 +811,24 @@ func (r *AutonomousDatabaseReconciler) ifBackupExists(backupSummary database.Aut return false } + +// removeBackupResources remove all the AutonomousDatabasBackups that +// are associated with the adb +func (r *AutonomousDatabaseReconciler) removeBackupResources(logger logr.Logger, adb *dbv4.AutonomousDatabase) error { + l := logger.WithName("removeBackupResources") + + // Get the list of AutonomousDatabaseBackupOCID in the same namespace + backupList, err := k8s.FetchAutonomousDatabaseBackups(r.KubeClient, adb.Namespace, adb.Name) + if err != nil { + return err + } + + for _, backup := range backupList.Items { + if err := r.KubeClient.Delete(context.TODO(), &backup); err != nil { + return err + } + l.Info("Delete AutonomousDatabaseBackup " + backup.Name) + } + + return nil +} diff --git a/controllers/database/autonomousdatabasebackup_controller.go b/controllers/database/autonomousdatabasebackup_controller.go index 7fac9e04..9744f3fb 100644 --- a/controllers/database/autonomousdatabasebackup_controller.go +++ b/controllers/database/autonomousdatabasebackup_controller.go @@ -54,8 +54,8 @@ import ( "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" "sigs.k8s.io/controller-runtime/pkg/predicate" - dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" - "github.com/oracle/oracle-database-operator/commons/adb_family" + dbv4 "github.com/oracle/oracle-database-operator/apis/database/v4" + adbfamily "github.com/oracle/oracle-database-operator/commons/adb_family" "github.com/oracle/oracle-database-operator/commons/oci" ) @@ -72,7 +72,7 @@ type AutonomousDatabaseBackupReconciler struct { // SetupWithManager sets up the controller with the Manager. func (r *AutonomousDatabaseBackupReconciler) SetupWithManager(mgr ctrl.Manager) error { return ctrl.NewControllerManagedBy(mgr). - For(&dbv1alpha1.AutonomousDatabaseBackup{}). + For(&dbv4.AutonomousDatabaseBackup{}). WithEventFilter(predicate.GenerationChangedPredicate{}). WithOptions(controller.Options{MaxConcurrentReconciles: 100}). // ReconcileHandler is never invoked concurrently with the same object. Complete(r) @@ -85,14 +85,14 @@ func (r *AutonomousDatabaseBackupReconciler) SetupWithManager(mgr ctrl.Manager) func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { logger := r.Log.WithValues("Namespace/Name", req.NamespacedName) - backup := &dbv1alpha1.AutonomousDatabaseBackup{} + backup := &dbv4.AutonomousDatabaseBackup{} if err := r.KubeClient.Get(context.TODO(), req.NamespacedName, backup); err != nil { // Ignore not-found errors, since they can't be fixed by an immediate requeue. // No need to change the since we don't know if we obtain the object. if apiErrors.IsNotFound(err) { return emptyResult, nil } - // Failed to get ADBBackup, so we don't need to update the status + // Failed to get AutonomousDatabaseBackup, so we don't need to update the status return emptyResult, err } @@ -100,7 +100,7 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req * Look up the owner AutonomousDatabase and set the ownerReference * if the owner hasn't been set yet. ******************************************************************/ - adbOCID, err := r.verifyTargetADB(backup) + adbOCID, err := r.verifyTargetAdb(backup) if err != nil { return r.manageError(backup, err) } @@ -133,10 +133,10 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req /****************************************************************** * Requeue if the Backup is in an intermediate state - * No-op if the ADB OCID is nil + * No-op if the Autonomous Database OCID is nil * To get the latest status, execute before all the reconcile logic ******************************************************************/ - if dbv1alpha1.IsBackupIntermediateState(backup.Status.LifecycleState) { + if dbv4.IsBackupIntermediateState(backup.Status.LifecycleState) { logger.WithName("IsIntermediateState").Info("Current lifecycleState is " + string(backup.Status.LifecycleState) + "; reconcile queued") return requeueResult, nil } @@ -187,7 +187,7 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req return r.manageError(backup, err) } - if dbv1alpha1.IsBackupIntermediateState(backup.Status.LifecycleState) { + if dbv4.IsBackupIntermediateState(backup.Status.LifecycleState) { logger.WithName("IsIntermediateState").Info("Reconcile queued") return requeueResult, nil } @@ -198,7 +198,7 @@ func (r *AutonomousDatabaseBackupReconciler) Reconcile(ctx context.Context, req } // setOwnerAutonomousDatabase sets the owner of the AutonomousDatabaseBackup if the AutonomousDatabase resource with the same database OCID is found -func (r *AutonomousDatabaseBackupReconciler) setOwnerAutonomousDatabase(backup *dbv1alpha1.AutonomousDatabaseBackup, adb *dbv1alpha1.AutonomousDatabase) error { +func (r *AutonomousDatabaseBackupReconciler) setOwnerAutonomousDatabase(backup *dbv4.AutonomousDatabaseBackup, adb *dbv4.AutonomousDatabase) error { logger := r.Log.WithName("set-owner-reference") controllerutil.SetOwnerReference(adb, backup, r.Scheme) @@ -210,43 +210,43 @@ func (r *AutonomousDatabaseBackupReconciler) setOwnerAutonomousDatabase(backup * return nil } -// verifyTargetADB searches if the target ADB is in the cluster, and set the owner reference to the ADB if it exists. -// The function returns the OCID of the target ADB. -func (r *AutonomousDatabaseBackupReconciler) verifyTargetADB(backup *dbv1alpha1.AutonomousDatabaseBackup) (string, error) { +// verifyTargetAdb searches if the target AutonomousDatabase is in the cluster, and set the owner reference to that AutonomousDatabase if it exists. +// The function returns the OCID of the target AutonomousDatabase. +func (r *AutonomousDatabaseBackupReconciler) verifyTargetAdb(backup *dbv4.AutonomousDatabaseBackup) (string, error) { // Get the target ADB OCID and the ADB resource - ownerADB, err := adbfamily.VerifyTargetADB(r.KubeClient, backup.Spec.Target, backup.Namespace) + ownerAdb, err := adbfamily.VerifyTargetAdb(r.KubeClient, backup.Spec.Target, backup.Namespace) if err != nil { return "", err } // Set the owner reference if needed - if len(backup.GetOwnerReferences()) == 0 && ownerADB != nil { - if err := r.setOwnerAutonomousDatabase(backup, ownerADB); err != nil { + if len(backup.GetOwnerReferences()) == 0 && ownerAdb != nil { + if err := r.setOwnerAutonomousDatabase(backup, ownerAdb); err != nil { return "", err } } - if backup.Spec.Target.OCIADB.OCID != nil { - return *backup.Spec.Target.OCIADB.OCID, nil + if backup.Spec.Target.OciAdb.OCID != nil { + return *backup.Spec.Target.OciAdb.OCID, nil } - if ownerADB != nil && ownerADB.Spec.Details.AutonomousDatabaseOCID != nil { - return *ownerADB.Spec.Details.AutonomousDatabaseOCID, nil + if ownerAdb != nil && ownerAdb.Spec.Details.Id != nil { + return *ownerAdb.Spec.Details.Id, nil } - return "", errors.New("cannot get the OCID of the targetADB") + return "", errors.New("cannot get the OCID of the target AutonomousDatabase") } -func (r *AutonomousDatabaseBackupReconciler) setupOCIClients(backup *dbv1alpha1.AutonomousDatabaseBackup) error { +func (r *AutonomousDatabaseBackupReconciler) setupOCIClients(backup *dbv4.AutonomousDatabaseBackup) error { var err error - authData := oci.APIKeyAuth{ + authData := oci.ApiKeyAuth{ ConfigMapName: backup.Spec.OCIConfig.ConfigMapName, SecretName: backup.Spec.OCIConfig.SecretName, Namespace: backup.GetNamespace(), } - provider, err := oci.GetOCIProvider(r.KubeClient, authData) + provider, err := oci.GetOciProvider(r.KubeClient, authData) if err != nil { return err } @@ -259,7 +259,7 @@ func (r *AutonomousDatabaseBackupReconciler) setupOCIClients(backup *dbv1alpha1. return nil } -func (r *AutonomousDatabaseBackupReconciler) manageError(backup *dbv1alpha1.AutonomousDatabaseBackup, issue error) (ctrl.Result, error) { +func (r *AutonomousDatabaseBackupReconciler) manageError(backup *dbv4.AutonomousDatabaseBackup, issue error) (ctrl.Result, error) { // Send event r.Recorder.Event(backup, corev1.EventTypeWarning, "ReconcileFailed", issue.Error()) diff --git a/controllers/database/autonomousdatabaserestore_controller.go b/controllers/database/autonomousdatabaserestore_controller.go index 254731bb..61b84c5d 100644 --- a/controllers/database/autonomousdatabaserestore_controller.go +++ b/controllers/database/autonomousdatabaserestore_controller.go @@ -54,8 +54,8 @@ import ( "sigs.k8s.io/controller-runtime/pkg/predicate" "github.com/oracle/oci-go-sdk/v65/common" - dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" - "github.com/oracle/oracle-database-operator/commons/adb_family" + dbv4 "github.com/oracle/oracle-database-operator/apis/database/v4" + adbfamily "github.com/oracle/oracle-database-operator/commons/adb_family" "github.com/oracle/oracle-database-operator/commons/k8s" "github.com/oracle/oracle-database-operator/commons/oci" ) @@ -74,7 +74,7 @@ type AutonomousDatabaseRestoreReconciler struct { // SetupWithManager sets up the controller with the Manager. func (r *AutonomousDatabaseRestoreReconciler) SetupWithManager(mgr ctrl.Manager) error { return ctrl.NewControllerManagedBy(mgr). - For(&dbv1alpha1.AutonomousDatabaseRestore{}). + For(&dbv4.AutonomousDatabaseRestore{}). WithEventFilter(predicate.GenerationChangedPredicate{}). Complete(r) } @@ -95,14 +95,14 @@ func (r *AutonomousDatabaseRestoreReconciler) SetupWithManager(mgr ctrl.Manager) func (r *AutonomousDatabaseRestoreReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { logger := r.Log.WithValues("Namespace/Name", req.NamespacedName) - restore := &dbv1alpha1.AutonomousDatabaseRestore{} + restore := &dbv4.AutonomousDatabaseRestore{} if err := r.KubeClient.Get(context.TODO(), req.NamespacedName, restore); err != nil { // Ignore not-found errors, since they can't be fixed by an immediate requeue. // No need to change since we don't know if we obtain the object. if apiErrors.IsNotFound(err) { return emptyResult, nil } - // Failed to get ADBRestore, so we don't need to update the status + // Failed to get the resource return emptyResult, err } @@ -110,7 +110,7 @@ func (r *AutonomousDatabaseRestoreReconciler) Reconcile(ctx context.Context, req * Look up the owner AutonomousDatabase and set the ownerReference * if the owner hasn't been set yet. ******************************************************************/ - adbOCID, err := r.verifyTargetADB(restore) + adbOCID, err := r.verifyTargetAdb(restore) if err != nil { return r.manageError(restore, err) } @@ -172,7 +172,7 @@ func (r *AutonomousDatabaseRestoreReconciler) Reconcile(ctx context.Context, req } // Requeue if it's in intermediate state - if dbv1alpha1.IsRestoreIntermediateState(restore.Status.Status) { + if dbv4.IsRestoreIntermediateState(restore.Status.Status) { logger.WithName("IsIntermediateState").Info("Current status is " + string(restore.Status.Status) + "; reconcile queued") return requeueResult, nil } @@ -182,10 +182,10 @@ func (r *AutonomousDatabaseRestoreReconciler) Reconcile(ctx context.Context, req return emptyResult, nil } -func (r *AutonomousDatabaseRestoreReconciler) getRestoreSDKTime(restore *dbv1alpha1.AutonomousDatabaseRestore) (*common.SDKTime, error) { - if restore.Spec.Source.K8sADBBackup.Name != nil { // restore using backupName - backup := &dbv1alpha1.AutonomousDatabaseBackup{} - if err := k8s.FetchResource(r.KubeClient, restore.Namespace, *restore.Spec.Source.K8sADBBackup.Name, backup); err != nil { +func (r *AutonomousDatabaseRestoreReconciler) getRestoreSDKTime(restore *dbv4.AutonomousDatabaseRestore) (*common.SDKTime, error) { + if restore.Spec.Source.K8sAdbBackup.Name != nil { // restore using backupName + backup := &dbv4.AutonomousDatabaseBackup{} + if err := k8s.FetchResource(r.KubeClient, restore.Namespace, *restore.Spec.Source.K8sAdbBackup.Name, backup); err != nil { return nil, err } @@ -207,7 +207,7 @@ func (r *AutonomousDatabaseRestoreReconciler) getRestoreSDKTime(restore *dbv1alp } // setOwnerAutonomousDatabase sets the owner of the AutonomousDatabaseBackup if the AutonomousDatabase resource with the same database OCID is found -func (r *AutonomousDatabaseRestoreReconciler) setOwnerAutonomousDatabase(restore *dbv1alpha1.AutonomousDatabaseRestore, adb *dbv1alpha1.AutonomousDatabase) error { +func (r *AutonomousDatabaseRestoreReconciler) setOwnerAutonomousDatabase(restore *dbv4.AutonomousDatabaseRestore, adb *dbv4.AutonomousDatabase) error { logger := r.Log.WithName("set-owner-reference") controllerutil.SetOwnerReference(adb, restore, r.Scheme) @@ -219,43 +219,43 @@ func (r *AutonomousDatabaseRestoreReconciler) setOwnerAutonomousDatabase(restore return nil } -// verifyTargetADB searches if the target ADB is in the cluster, and set the owner reference to the ADB if it exists. +// verifyTargetAdb searches if the target ADB is in the cluster, and set the owner reference to the ADB if it exists. // The function returns the OCID of the target ADB. -func (r *AutonomousDatabaseRestoreReconciler) verifyTargetADB(restore *dbv1alpha1.AutonomousDatabaseRestore) (string, error) { +func (r *AutonomousDatabaseRestoreReconciler) verifyTargetAdb(restore *dbv4.AutonomousDatabaseRestore) (string, error) { // Get the target ADB OCID and the ADB resource - ownerADB, err := adbfamily.VerifyTargetADB(r.KubeClient, restore.Spec.Target, restore.Namespace) + ownerAdb, err := adbfamily.VerifyTargetAdb(r.KubeClient, restore.Spec.Target, restore.Namespace) if err != nil { return "", err } // Set the owner reference if needed - if len(restore.GetOwnerReferences()) == 0 && ownerADB != nil { - if err := r.setOwnerAutonomousDatabase(restore, ownerADB); err != nil { + if len(restore.GetOwnerReferences()) == 0 && ownerAdb != nil { + if err := r.setOwnerAutonomousDatabase(restore, ownerAdb); err != nil { return "", err } } - if restore.Spec.Target.OCIADB.OCID != nil { - return *restore.Spec.Target.OCIADB.OCID, nil + if restore.Spec.Target.OciAdb.OCID != nil { + return *restore.Spec.Target.OciAdb.OCID, nil } - if ownerADB != nil && ownerADB.Spec.Details.AutonomousDatabaseOCID != nil { - return *ownerADB.Spec.Details.AutonomousDatabaseOCID, nil + if ownerAdb != nil && ownerAdb.Spec.Details.Id != nil { + return *ownerAdb.Spec.Details.Id, nil } - return "", errors.New("cannot get the OCID of the targetADB") + return "", errors.New("cannot get the OCID of the target Autonomous Database") } -func (r *AutonomousDatabaseRestoreReconciler) setupOCIClients(restore *dbv1alpha1.AutonomousDatabaseRestore) error { +func (r *AutonomousDatabaseRestoreReconciler) setupOCIClients(restore *dbv4.AutonomousDatabaseRestore) error { var err error - authData := oci.APIKeyAuth{ + authData := oci.ApiKeyAuth{ ConfigMapName: restore.Spec.OCIConfig.ConfigMapName, SecretName: restore.Spec.OCIConfig.SecretName, Namespace: restore.GetNamespace(), } - provider, err := oci.GetOCIProvider(r.KubeClient, authData) + provider, err := oci.GetOciProvider(r.KubeClient, authData) if err != nil { return err } @@ -274,7 +274,7 @@ func (r *AutonomousDatabaseRestoreReconciler) setupOCIClients(restore *dbv1alpha } // manageError doesn't return the error so that the request won't be requeued -func (r *AutonomousDatabaseRestoreReconciler) manageError(restore *dbv1alpha1.AutonomousDatabaseRestore, issue error) (ctrl.Result, error) { +func (r *AutonomousDatabaseRestoreReconciler) manageError(restore *dbv4.AutonomousDatabaseRestore, issue error) (ctrl.Result, error) { // Send event r.Recorder.Event(restore, corev1.EventTypeWarning, "ReconcileFailed", issue.Error()) diff --git a/controllers/database/cdb_controller.go b/controllers/database/cdb_controller.go index 5e6c0aca..6c5fc747 100644 --- a/controllers/database/cdb_controller.go +++ b/controllers/database/cdb_controller.go @@ -40,7 +40,9 @@ package controllers import ( "context" + "encoding/json" "errors" + "fmt" //"fmt" "strconv" @@ -64,7 +66,7 @@ import ( "sigs.k8s.io/controller-runtime/pkg/event" "sigs.k8s.io/controller-runtime/pkg/predicate" - dbapi "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + dbapi "github.com/oracle/oracle-database-operator/apis/database/v4" dbcommons "github.com/oracle/oracle-database-operator/commons/database" ) @@ -506,6 +508,17 @@ func (r *CDBReconciler) createPodSpec(cdb *dbapi.CDB) corev1.PodSpec { Name: "WEBSERVER_PASSWORD_KEY", Value: cdb.Spec.WebServerPwd.Secret.Key, }, + { + Name: "R1", + ValueFrom: &corev1.EnvVarSource{ + SecretKeyRef: &corev1.SecretKeySelector{ + LocalObjectReference: corev1.LocalObjectReference{ + Name: cdb.Spec.CDBPriKey.Secret.SecretName, + }, + Key: cdb.Spec.CDBPriKey.Secret.Key, + }, + }, + }, } }(), }}, @@ -749,56 +762,46 @@ func (r *CDBReconciler) createSvcSpec(cdb *dbapi.CDB) *corev1.Service { /* ************************************************ - Check CDB deletion - /*********************************************** + +/*********************************************** */ func (r *CDBReconciler) manageCDBDeletion(ctx context.Context, req ctrl.Request, cdb *dbapi.CDB) error { log := r.Log.WithValues("manageCDBDeletion", req.NamespacedName) - // Check if the PDB instance is marked to be deleted, which is - // indicated by the deletion timestamp being set. - isCDBMarkedToBeDeleted := cdb.GetDeletionTimestamp() != nil - if isCDBMarkedToBeDeleted { - log.Info("Marked to be deleted") + /* REGISTER FINALIZER */ + if cdb.ObjectMeta.DeletionTimestamp.IsZero() { + if !controllerutil.ContainsFinalizer(cdb, CDBFinalizer) { + controllerutil.AddFinalizer(cdb, CDBFinalizer) + if err := r.Update(ctx, cdb); err != nil { + return err + } + } + + } else { + log.Info("cdb set to be deleted") cdb.Status.Phase = cdbPhaseDelete cdb.Status.Status = true r.Status().Update(ctx, cdb) + if controllerutil.ContainsFinalizer(cdb, CDBFinalizer) { - // Run finalization logic for CDBFinalizer. If the - // finalization logic fails, don't remove the finalizer so - // that we can retry during the next reconciliation. - err := r.deleteCDBInstance(ctx, req, cdb) - if err != nil { - log.Info("Could not delete CDB Resource", "CDB Name", cdb.Spec.CDBName, "err", err.Error()) + + if err := r.DeletePDBS(ctx, req, cdb); err != nil { + log.Info("Cannot delete pdbs") return err } - // Remove CDBFinalizer. Once all finalizers have been - // removed, the object will be deleted. - log.Info("Removing finalizer") controllerutil.RemoveFinalizer(cdb, CDBFinalizer) - err = r.Update(ctx, cdb) - if err != nil { - log.Info("Could not remove finalizer", "err", err.Error()) + if err := r.Update(ctx, cdb); err != nil { return err } - - log.Info("Successfully removed CDB Resource") - return nil } - } - - // Add finalizer for this CR - if !controllerutil.ContainsFinalizer(cdb, CDBFinalizer) { - log.Info("Adding finalizer") - cdb.Status.Phase = cdbPhaseInit - cdb.Status.Status = false - controllerutil.AddFinalizer(cdb, CDBFinalizer) - err := r.Update(ctx, cdb) + err := r.deleteCDBInstance(ctx, req, cdb) if err != nil { - log.Info("Could not add finalizer", "err", err.Error()) + log.Info("Could not delete CDB Resource", "CDB Name", cdb.Spec.CDBName, "err", err.Error()) return err } + } return nil } @@ -852,7 +855,8 @@ func (r *CDBReconciler) deleteCDBInstance(ctx context.Context, req ctrl.Request, /* ************************************************ - Get Secret Key for a Secret Name - /*********************************************** + +/*********************************************** */ func (r *CDBReconciler) verifySecrets(ctx context.Context, req ctrl.Request, cdb *dbapi.CDB) error { @@ -876,6 +880,9 @@ func (r *CDBReconciler) verifySecrets(ctx context.Context, req ctrl.Request, cdb if err := r.checkSecret(ctx, req, cdb, cdb.Spec.WebServerPwd.Secret.SecretName); err != nil { return err } + if err := r.checkSecret(ctx, req, cdb, cdb.Spec.CDBPriKey.Secret.SecretName); err != nil { + return err + } cdb.Status.Msg = "" log.Info("Verified secrets successfully") @@ -885,7 +892,8 @@ func (r *CDBReconciler) verifySecrets(ctx context.Context, req ctrl.Request, cdb /* ************************************************ - Get Secret Key for a Secret Name - /*********************************************** + +/*********************************************** */ func (r *CDBReconciler) checkSecret(ctx context.Context, req ctrl.Request, cdb *dbapi.CDB, secretName string) error { @@ -909,7 +917,8 @@ func (r *CDBReconciler) checkSecret(ctx context.Context, req ctrl.Request, cdb * /* ************************************************ - Delete Secrets - /*********************************************** + +/*********************************************** */ func (r *CDBReconciler) deleteSecrets(ctx context.Context, req ctrl.Request, cdb *dbapi.CDB) { @@ -966,11 +975,104 @@ func (r *CDBReconciler) deleteSecrets(ctx context.Context, req ctrl.Request, cdb } } +/* Delete cascade option */ + /* ************************************************************* - SetupWithManager sets up the controller with the Manager. - /************************************************************ +/************************************************************ */ + +func (r *CDBReconciler) DeletePDBS(ctx context.Context, req ctrl.Request, cdb *dbapi.CDB) error { + log := r.Log.WithValues("DeletePDBS", req.NamespacedName) + + /* =================== DELETE CASCADE ================ */ + if cdb.Spec.DeletePDBCascade == true { + log.Info("DELETE PDB CASCADE OPTION") + pdbList := &dbapi.PDBList{} + listOpts := []client.ListOption{} + err := r.List(ctx, pdbList, listOpts...) + if err != nil { + log.Info("Failed to get the list of pdbs") + } + + var url string + if err == nil { + for _, pdbitem := range pdbList.Items { + log.Info("pdbitem.Spec.CDBName : " + pdbitem.Spec.CDBName) + log.Info("pdbitem.Spec.CDBNamespace: " + pdbitem.Spec.CDBNamespace) + log.Info("cdb.Spec.CDBName : " + cdb.Spec.CDBName) + log.Info("cdb.Namespace : " + cdb.Namespace) + if pdbitem.Spec.CDBName == cdb.Spec.CDBName && pdbitem.Spec.CDBNamespace == cdb.Namespace { + fmt.Printf("DeletePDBS Call Delete function for %s %s\n", pdbitem.Name, pdbitem.Spec.PDBName) + + var objmap map[string]interface{} /* Used for the return payload */ + values := map[string]string{ + "state": "CLOSE", + "modifyOption": "IMMEDIATE", + "getScript": "FALSE", + } + + //url := "https://" + pdbitem.Spec.CDBResName + "-cdb." + pdbitem.Spec.CDBNamespace + ":" + strconv.Itoa(cdb.Spec.ORDSPort) + "/database/pdbs/" + pdbitem.Spec.PDBName + url = "https://" + pdbitem.Spec.CDBResName + "-ords." + pdbitem.Spec.CDBNamespace + ":" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdbitem.Spec.PDBName + "/status" + + log.Info("callAPI(URL):" + url) + log.Info("pdbitem.Status.OpenMode" + pdbitem.Status.OpenMode) + + if pdbitem.Status.OpenMode != "MOUNTED" { + + log.Info("Force pdb closure") + respData, errapi := NewCallApi(r, ctx, req, &pdbitem, url, values, "POST") + + fmt.Printf("Debug NEWCALL:%s\n", respData) + if err := json.Unmarshal([]byte(respData), &objmap); err != nil { + log.Error(err, "failed to get respData from callAPI", "err", err.Error()) + return err + } + + if errapi != nil { + log.Error(err, "callAPI cannot close pdb "+pdbitem.Spec.PDBName, "err", err.Error()) + return err + } + + r.Recorder.Eventf(cdb, corev1.EventTypeNormal, "close pdb", "pdbname=%s", pdbitem.Spec.PDBName) + } + + /* start dropping pdb */ + log.Info("Drop pluggable database") + values = map[string]string{ + "action": "INCLUDING", + "getScript": "FALSE", + } + url = "https://" + pdbitem.Spec.CDBResName + "-ords." + pdbitem.Spec.CDBNamespace + ":" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdbitem.Spec.PDBName + "/" + respData, errapi := NewCallApi(r, ctx, req, &pdbitem, url, values, "DELETE") + + if err := json.Unmarshal([]byte(respData), &objmap); err != nil { + log.Error(err, "failed to get respData from callAPI", "err", err.Error()) + return err + } + + if errapi != nil { + log.Error(err, "callAPI cannot drop pdb "+pdbitem.Spec.PDBName, "err", err.Error()) + return err + } + r.Recorder.Eventf(cdb, corev1.EventTypeNormal, "drop pdb", "pdbname=%s", pdbitem.Spec.PDBName) + + err = r.Delete(context.Background(), &pdbitem, client.GracePeriodSeconds(0)) + if err != nil { + log.Info("Could not delete PDB resource", "err", err.Error()) + return err + } + + } /* check pdb name */ + } /* end of loop */ + } + + } + /* ================================================ */ + return nil +} + func (r *CDBReconciler) SetupWithManager(mgr ctrl.Manager) error { return ctrl.NewControllerManagedBy(mgr). For(&dbapi.CDB{}). diff --git a/controllers/database/dataguardbroker_controller.go b/controllers/database/dataguardbroker_controller.go deleted file mode 100644 index 9faaefd2..00000000 --- a/controllers/database/dataguardbroker_controller.go +++ /dev/null @@ -1,1199 +0,0 @@ -/* -** Copyright (c) 2023 Oracle and/or its affiliates. -** -** The Universal Permissive License (UPL), Version 1.0 -** -** Subject to the condition set forth below, permission is hereby granted to any -** person obtaining a copy of this software, associated documentation and/or data -** (collectively the "Software"), free of charge and under any and all copyright -** rights in the Software, and any and all patent rights owned or freely -** licensable by each licensor hereunder covering either (i) the unmodified -** Software as contributed to or provided by such licensor, or (ii) the Larger -** Works (as defined below), to deal in both -** -** (a) the Software, and -** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if -** one is included with the Software (each a "Larger Work" to which the Software -** is contributed by such licensors), -** -** without restriction, including without limitation the rights to copy, create -** derivative works of, display, perform, and distribute the Software and make, -** use, sell, offer for sale, import, export, have made, and have sold the -** Software and the Larger Work(s), and to sublicense the foregoing rights on -** either these or other terms. -** -** This license is subject to the following condition: -** The above copyright notice and either this complete permission notice or at -** a minimum a reference to the UPL must be included in all copies or -** substantial portions of the Software. -** -** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -** SOFTWARE. - */ - -package controllers - -import ( - "context" - "errors" - "fmt" - "strings" - "time" - - "github.com/go-logr/logr" - corev1 "k8s.io/api/core/v1" - apierrors "k8s.io/apimachinery/pkg/api/errors" - metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" - "k8s.io/apimachinery/pkg/runtime" - "k8s.io/apimachinery/pkg/types" - "k8s.io/client-go/rest" - "k8s.io/client-go/tools/record" - ctrl "sigs.k8s.io/controller-runtime" - "sigs.k8s.io/controller-runtime/pkg/client" - "sigs.k8s.io/controller-runtime/pkg/controller" - "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" - - dbapi "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" - dbcommons "github.com/oracle/oracle-database-operator/commons/database" -) - -// DataguardBrokerReconciler reconciles a DataguardBroker object -type DataguardBrokerReconciler struct { - client.Client - Log logr.Logger - Scheme *runtime.Scheme - Config *rest.Config - Recorder record.EventRecorder -} - -const dataguardBrokerFinalizer = "database.oracle.com/dataguardbrokerfinalizer" - -//+kubebuilder:rbac:groups=database.oracle.com,resources=dataguardbrokers,verbs=get;list;watch;create;update;patch;delete -//+kubebuilder:rbac:groups=database.oracle.com,resources=dataguardbrokers/status,verbs=get;update;patch -//+kubebuilder:rbac:groups=database.oracle.com,resources=dataguardbrokers/finalizers,verbs=update -//+kubebuilder:rbac:groups="",resources=pods;pods/log;pods/exec;persistentvolumeclaims;services,verbs=create;delete;get;list;patch;update;watch -//+kubebuilder:rbac:groups="",resources=events,verbs=create;patch - -// Reconcile is part of the main kubernetes reconciliation loop which aims to -// move the current state of the cluster closer to the desired state. -// TODO(user): Modify the Reconcile function to compare the state specified by -// the DataguardBroker object against the actual cluster state, and then -// perform operations to make the cluster state reflect the state specified by -// the user. -// -// For more details, check Reconcile and its Result here: -// - https://pkg.go.dev/sigs.k8s.io/controller-runtime@v0.8.3/pkg/reconcile -func (r *DataguardBrokerReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { - - r.Log.Info("Reconcile requested") - - dataguardBroker := &dbapi.DataguardBroker{} - err := r.Get(ctx, types.NamespacedName{Namespace: req.Namespace, Name: req.Name}, dataguardBroker) - if err != nil { - if apierrors.IsNotFound(err) { - r.Log.Info("Resource deleted") - return requeueN, nil - } - return requeueN, err - } - - // Manage DataguardBroker Deletion - result, err := r.manageDataguardBrokerDeletion(req, ctx, dataguardBroker) - if result.Requeue { - r.Log.Info("Reconcile queued") - return result, err - } - if err != nil { - r.Log.Error(err, err.Error()) - return result, err - } - - // Fetch Primary Database Reference - singleInstanceDatabase := &dbapi.SingleInstanceDatabase{} - err = r.Get(ctx, types.NamespacedName{Namespace: req.Namespace, Name: dataguardBroker.Spec.PrimaryDatabaseRef}, singleInstanceDatabase) - if err != nil { - if apierrors.IsNotFound(err) { - r.Log.Info("Resource deleted") - return requeueN, nil - } - return requeueN, err - } - - /* Initialize Status */ - if dataguardBroker.Status.Status == "" { - dataguardBroker.Status.Status = dbcommons.StatusCreating - dataguardBroker.Status.ExternalConnectString = dbcommons.ValueUnavailable - dataguardBroker.Status.ClusterConnectString = dbcommons.ValueUnavailable - r.Status().Update(ctx, dataguardBroker) - } - - // Always refresh status before a reconcile - defer r.Status().Update(ctx, dataguardBroker) - - // Create Service to point to primary database always - result = r.createSVC(ctx, req, dataguardBroker) - if result.Requeue { - r.Log.Info("Reconcile queued") - return result, nil - } - - // Validate if Primary Database Reference is ready - result, sidbReadyPod, adminPassword := r.validateSidbReadiness(dataguardBroker, singleInstanceDatabase, ctx, req) - if result.Requeue { - r.Log.Info("Reconcile queued") - return result, nil - } - - // Setup the DG Configuration - result = r.setupDataguardBrokerConfiguration(dataguardBroker, singleInstanceDatabase, sidbReadyPod, adminPassword, ctx, req) - if result.Requeue { - r.Log.Info("Reconcile queued") - return result, nil - } - - // Set a particular database as primary - result = r.SetAsPrimaryDatabase(singleInstanceDatabase.Spec.Sid, dataguardBroker.Spec.SetAsPrimaryDatabase, dataguardBroker, - singleInstanceDatabase, adminPassword, ctx, req) - if result.Requeue { - r.Log.Info("Reconcile queued") - return result, nil - } - - // If LoadBalancer = true , ensure Connect String is updated - if dataguardBroker.Status.ExternalConnectString == dbcommons.ValueUnavailable { - return requeueY, nil - } - - dataguardBroker.Status.Status = dbcommons.StatusReady - - r.Log.Info("Reconcile completed") - return ctrl.Result{}, nil - -} - -// ##################################################################################################### -// -// Validate Readiness of the primary DB specified -// -// ##################################################################################################### -func (r *DataguardBrokerReconciler) validateSidbReadiness(m *dbapi.DataguardBroker, - n *dbapi.SingleInstanceDatabase, ctx context.Context, req ctrl.Request) (ctrl.Result, corev1.Pod, string) { - - log := r.Log.WithValues("validateSidbReadiness", req.NamespacedName) - adminPassword := "" - // ## FETCH THE SIDB REPLICAS . - sidbReadyPod, _, _, _, err := dbcommons.FindPods(r, n.Spec.Image.Version, - n.Spec.Image.PullFrom, n.Name, n.Namespace, ctx, req) - if err != nil { - log.Error(err, err.Error()) - return requeueY, sidbReadyPod, adminPassword - } - - if n.Status.Status != dbcommons.StatusReady { - - eventReason := "Waiting" - eventMsg := "Waiting for " + n.Name + " to be Ready" - r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) - return requeueY, sidbReadyPod, adminPassword - } - - // Validate databaseRef Admin Password - adminPasswordSecret := &corev1.Secret{} - err = r.Get(ctx, types.NamespacedName{Name: n.Spec.AdminPassword.SecretName, Namespace: n.Namespace}, adminPasswordSecret) - if err != nil { - if apierrors.IsNotFound(err) { - //m.Status.Status = dbcommons.StatusError - eventReason := "Waiting" - eventMsg := "waiting for secret : " + n.Spec.AdminPassword.SecretName + " to get created" - r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) - r.Log.Info("Secret " + n.Spec.AdminPassword.SecretName + " Not Found") - return requeueY, sidbReadyPod, adminPassword - } - log.Error(err, err.Error()) - return requeueY, sidbReadyPod, adminPassword - } - adminPassword = string(adminPasswordSecret.Data[n.Spec.AdminPassword.SecretKey]) - - out, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", - fmt.Sprintf("echo -e \"%s\" | %s", fmt.Sprintf(dbcommons.ValidateAdminPassword, adminPassword), dbcommons.GetSqlClient(n.Spec.Edition))) - if err != nil { - log.Error(err, err.Error()) - return requeueY, sidbReadyPod, adminPassword - } - if strings.Contains(out, "USER is \"SYS\"") { - log.Info("validated Admin password successfully") - } else if strings.Contains(out, "ORA-01017") { - //m.Status.Status = dbcommons.StatusError - eventReason := "Logon denied" - eventMsg := "invalid databaseRef admin password. secret: " + n.Spec.AdminPassword.SecretName - r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) - return requeueY, sidbReadyPod, adminPassword - } else { - return requeueY, sidbReadyPod, adminPassword - } - - return requeueN, sidbReadyPod, adminPassword -} - -// ############################################################################# -// -// Instantiate Service spec from StandbyDatabase spec -// -// ############################################################################# -func (r *DataguardBrokerReconciler) instantiateSVCSpec(m *dbapi.DataguardBroker) *corev1.Service { - svc := &corev1.Service{ - TypeMeta: metav1.TypeMeta{ - Kind: "Service", - }, - ObjectMeta: metav1.ObjectMeta{ - Name: m.Name, - Namespace: m.Namespace, - Labels: map[string]string{ - "app": m.Name, - }, - Annotations: func() map[string]string { - annotations := make(map[string]string) - if len(m.Spec.ServiceAnnotations) != 0 { - for key, value := range m.Spec.ServiceAnnotations { - annotations[key] = value - } - } - return annotations - }(), - }, - Spec: corev1.ServiceSpec{ - Ports: []corev1.ServicePort{ - { - Name: "listener", - Port: 1521, - Protocol: corev1.ProtocolTCP, - }, - { - Name: "xmldb", - Port: 5500, - Protocol: corev1.ProtocolTCP, - }, - }, - Selector: map[string]string{ - "app": m.Name, - }, - Type: corev1.ServiceType(func() string { - if m.Spec.LoadBalancer { - return "LoadBalancer" - } - return "NodePort" - }()), - }, - } - // Set StandbyDatabase instance as the owner and controller - ctrl.SetControllerReference(m, svc, r.Scheme) - return svc -} - -// ############################################################################# -// -// Create a Service for StandbyDatabase -// -// ############################################################################# -func (r *DataguardBrokerReconciler) createSVC(ctx context.Context, req ctrl.Request, - m *dbapi.DataguardBroker) ctrl.Result { - - log := r.Log.WithValues("createSVC", req.NamespacedName) - // Check if the Service already exists, if not create a new one - svc := &corev1.Service{} - // Get retrieves an obj for the given object key from the Kubernetes Cluster. - // obj must be a struct pointer so that obj can be updated with the response returned by the Server. - // Here foundsvc is the struct pointer to corev1.Service{} - err := r.Get(ctx, types.NamespacedName{Name: m.Name, Namespace: m.Namespace}, svc) - if err != nil && apierrors.IsNotFound(err) { - // Define a new Service - svc = r.instantiateSVCSpec(m) - log.Info("Creating a new Service", "Service.Namespace", svc.Namespace, "Service.Name", svc.Name) - err = r.Create(ctx, svc) - //err = r.Update(ctx, svc) - if err != nil { - log.Error(err, "Failed to create new Service", "Service.Namespace", svc.Namespace, "Service.Name", svc.Name) - return requeueY - } else { - timeout := 30 - // Waiting for Service to get created as sometimes it takes some time to create a service . 30 seconds TImeout - err = dbcommons.WaitForStatusChange(r, svc.Name, m.Namespace, ctx, req, time.Duration(timeout)*time.Second, "svc", "creation") - if err != nil { - log.Error(err, "Error in Waiting for svc status for Creation", "svc.Namespace", svc.Namespace, "SVC.Name", svc.Name) - return requeueY - } - log.Info("Succesfully Created New Service ", "Service.Name : ", svc.Name) - } - time.Sleep(10 * time.Second) - - } else if err != nil { - log.Error(err, "Failed to get Service") - return requeueY - } else if err == nil { - log.Info(" ", "Found Existing Service ", svc.Name) - } - - // update service status - log.Info("Updating the service status...") - m.Status.ClusterConnectString = svc.Name + "." + svc.Namespace + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/DATAGUARD" - if m.Spec.LoadBalancer { - if len(svc.Status.LoadBalancer.Ingress) > 0 { - lbAddress := svc.Status.LoadBalancer.Ingress[0].Hostname - if lbAddress == "" { - lbAddress = svc.Status.LoadBalancer.Ingress[0].IP - } - m.Status.ExternalConnectString = lbAddress + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/DATAGUARD" - } - } else { - nodeip := dbcommons.GetNodeIp(r, ctx, req) - if nodeip != "" { - m.Status.ExternalConnectString = nodeip + ":" + fmt.Sprint(svc.Spec.Ports[0].NodePort) + "/DATAGUARD" - } - } - r.Status().Update(ctx, m) - - return requeueN -} - -// ############################################################################# -// -// Setup the requested DG Configuration -// -// ############################################################################# -func (r *DataguardBrokerReconciler) setupDataguardBrokerConfiguration(m *dbapi.DataguardBroker, n *dbapi.SingleInstanceDatabase, - sidbReadyPod corev1.Pod, adminPassword string, ctx context.Context, req ctrl.Request) ctrl.Result { - log := r.Log.WithValues("setupDataguardBrokerConfiguration", req.NamespacedName) - - databases, _, err := dbcommons.GetDatabasesInDgConfig(sidbReadyPod, r, r.Config, ctx, req) - dbSet := make(map[string]struct{}) - if err != nil { - if err.Error() != "databases in DG config is nil" { - return requeueY - } - } - if len(databases) > 0 { - log.Info("Databases in DG config are :") - for i := 0; i < len(databases); i++ { - log.Info(strings.Split(databases[i], ":")[0]) - dbSet[strings.ToUpper(strings.Split(databases[i], ":")[0])] = struct{}{} - } - } - - for i := 0; i < len(m.Spec.StandbyDatabaseRefs); i++ { - - standbyDatabase := &dbapi.SingleInstanceDatabase{} - err := r.Get(ctx, types.NamespacedName{Namespace: req.Namespace, Name: m.Spec.StandbyDatabaseRefs[i]}, standbyDatabase) - if err != nil { - if apierrors.IsNotFound(err) { - eventReason := "Warning" - eventMsg := m.Spec.StandbyDatabaseRefs[i] + "not found" - r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) - continue - } - log.Error(err, err.Error()) - return requeueY - } - - // Check if dataguard broker is already configured for the standby database - if standbyDatabase.Status.DgBrokerConfigured { - log.Info("Dataguard broker for standbyDatabase : " + standbyDatabase.Name + " is already configured") - continue - } - _, ok := dbSet[standbyDatabase.Status.Sid] - if ok { - log.Info("A database with the same SID is already configured in the DG") - r.Recorder.Eventf(m, corev1.EventTypeWarning, "Spec Error", "A database with the same SID "+standbyDatabase.Status.Sid+" is already configured in the DG") - continue - } - - m.Status.Status = dbcommons.StatusCreating - r.Status().Update(ctx, m) - - // ## FETCH THE STANDBY REPLICAS . - standbyDatabaseReadyPod, _, _, _, err := dbcommons.FindPods(r, n.Spec.Image.Version, - n.Spec.Image.PullFrom, standbyDatabase.Name, standbyDatabase.Namespace, ctx, req) - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - - if standbyDatabase.Status.Status != dbcommons.StatusReady { - - eventReason := "Waiting" - eventMsg := "Waiting for " + standbyDatabase.Name + " to be Ready" - r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) - return requeueY - - } - - result := r.setupDataguardBrokerConfigurationForGivenDB(m, n, standbyDatabase, standbyDatabaseReadyPod, sidbReadyPod, ctx, req, adminPassword) - if result.Requeue { - return result - } - - // Update Databases - r.updateReconcileStatus(m, sidbReadyPod, ctx, req) - } - - eventReason := "DG Configuration up to date" - eventMsg := "" - - // Patch DataguardBroker Service to point selector to Current Primary Name - result := r.patchService(m, sidbReadyPod, n, ctx, req) - if result.Requeue { - return result - } - - r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) - - return requeueN -} - -// ############################################################################# -// -// Patch DataguardBroker Service to point selector to Current Primary Name -// -// ############################################################################# -func (r *DataguardBrokerReconciler) patchService(m *dbapi.DataguardBroker, sidbReadyPod corev1.Pod, n *dbapi.SingleInstanceDatabase, - ctx context.Context, req ctrl.Request) ctrl.Result { - log := r.Log.WithValues("patchService", req.NamespacedName) - databases, out, err := dbcommons.GetDatabasesInDgConfig(sidbReadyPod, r, r.Config, ctx, req) - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - if !strings.Contains(out, "ORA-") { - primarySid := strings.ToUpper(dbcommons.GetPrimaryDatabase(databases)) - primaryName := n.Name - if primarySid != n.Spec.Sid { - primaryName = n.Status.StandbyDatabases[primarySid] - } - - // Patch DataguardBroker Service to point selector to Current Primary Name - svc := &corev1.Service{} - err = r.Get(ctx, types.NamespacedName{Name: req.Name, Namespace: req.Namespace}, svc) - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - svc.Spec.Selector["app"] = primaryName - err = r.Update(ctx, svc) - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - - m.Status.ClusterConnectString = svc.Name + "." + svc.Namespace + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/DATAGUARD" - if m.Spec.LoadBalancer { - if len(svc.Status.LoadBalancer.Ingress) > 0 { - lbAddress := svc.Status.LoadBalancer.Ingress[0].Hostname - if lbAddress == "" { - lbAddress = svc.Status.LoadBalancer.Ingress[0].IP - } - m.Status.ExternalConnectString = lbAddress + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/DATAGUARD" - } - } else { - nodeip := dbcommons.GetNodeIp(r, ctx, req) - if nodeip != "" { - m.Status.ExternalConnectString = nodeip + ":" + fmt.Sprint(svc.Spec.Ports[0].NodePort) + "/DATAGUARD" - } - } - } - return requeueN -} - -// ############################################################################# -// -// Set up DG Configuration for a given StandbyDatabase -// -// ############################################################################# -func (r *DataguardBrokerReconciler) setupDataguardBrokerConfigurationForGivenDB(m *dbapi.DataguardBroker, n *dbapi.SingleInstanceDatabase, standbyDatabase *dbapi.SingleInstanceDatabase, - standbyDatabaseReadyPod corev1.Pod, sidbReadyPod corev1.Pod, ctx context.Context, req ctrl.Request, adminPassword string) ctrl.Result { - - log := r.Log.WithValues("setupDataguardBrokerConfigurationForGivenDB", req.NamespacedName) - - if standbyDatabaseReadyPod.Name == "" || sidbReadyPod.Name == "" { - return requeueY - } - - // ## CHECK IF DG CONFIGURATION AVAILABLE IN PRIMARY DATABSE## - out, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf("echo -e \"%s\" | dgmgrl / as sysdba ", dbcommons.DBShowConfigCMD)) - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - log.Info("ShowConfiguration Output") - log.Info(out) - - if strings.Contains(out, "ORA-16525") { - log.Info("ORA-16525: The Oracle Data Guard broker is not yet available on Primary") - return requeueY - } - - _, err = dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, true, "bash", "-c", - fmt.Sprintf(dbcommons.CreateAdminPasswordFile, adminPassword)) - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - log.Info("DB Admin pwd file created") - - // ORA-16532: Oracle Data Guard broker configuration does not exist , so create one - if strings.Contains(out, "ORA-16532") { - if m.Spec.ProtectionMode == "MaxPerformance" { - // Construct the password file and dgbroker command file - out, err := dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf(dbcommons.CreateDGMGRLScriptFile, dbcommons.DataguardBrokerMaxPerformanceCMD)) - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - log.Info("DGMGRL command file creation output") - log.Info(out) - - // ## DG CONFIGURATION FOR PRIMARY DB || MODE : MAXPERFORMANCE ## - out, err = dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, false, "bash", "-c", - "dgmgrl sys@${PRIMARY_DB_CONN_STR} @dgmgrl.cmd < admin.pwd && rm -rf dgmgrl.cmd") - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - log.Info("DgConfigurationMaxPerformance Output") - log.Info(out) - } else if m.Spec.ProtectionMode == "MaxAvailability" { - // ## DG CONFIGURATION FOR PRIMARY DB || MODE : MAX AVAILABILITY ## - out, err := dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf(dbcommons.CreateDGMGRLScriptFile, dbcommons.DataguardBrokerMaxAvailabilityCMD)) - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - log.Info("DGMGRL command file creation output") - log.Info(out) - - // ## DG CONFIGURATION FOR PRIMARY DB || MODE : MAXPERFORMANCE ## - out, err = dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, false, "bash", "-c", - "dgmgrl sys@${PRIMARY_DB_CONN_STR} @dgmgrl.cmd < admin.pwd && rm -rf dgmgrl.cmd") - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - log.Info("DgConfigurationMaxAvailability Output") - log.Info(out) - } else { - log.Info("SPECIFY correct Protection Mode . Either MaxAvailability or MaxPerformance") - return requeueY - } - - // ## SHOW CONFIGURATION DG - out, err := dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf("echo -e \"%s\" | dgmgrl / as sysdba ", dbcommons.DBShowConfigCMD)) - if err != nil { - log.Error(err, err.Error()) - return requeueY - } else { - log.Info("ShowConfiguration Output") - log.Info(out) - } - // Set DG Configured status to true for this standbyDatabase and primary Database. so that in next reconcilation, we dont configure this again - n.Status.DgBrokerConfigured = true - standbyDatabase.Status.DgBrokerConfigured = true - r.Status().Update(ctx, standbyDatabase) - r.Status().Update(ctx, n) - // Remove admin pwd file - _, err = dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, true, "bash", "-c", - dbcommons.RemoveAdminPasswordFile) - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - log.Info("DB Admin pwd file removed") - - return requeueN - } - - // DG Configuration Exists . So add the standbyDatabase to the existing DG Configuration - databases, _, err := dbcommons.GetDatabasesInDgConfig(sidbReadyPod, r, r.Config, ctx, req) - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - - // ## ADD DATABASE TO DG CONFIG , IF NOT PRESENT - found, _ := dbcommons.IsDatabaseFound(standbyDatabase.Spec.Sid, databases, "") - if found { - return requeueN - } - primarySid := dbcommons.GetPrimaryDatabase(databases) - - // If user adds a new standby to a dg config when failover happened to one ot the standbys, we need to have current primary connect string - primaryConnectString := n.Name + ":1521/" + primarySid - if !strings.EqualFold(primarySid, n.Spec.Sid) { - primaryConnectString = n.Status.StandbyDatabases[primarySid] + ":1521/" + primarySid - } - - if m.Spec.ProtectionMode == "MaxPerformance" { - // ## DG CONFIGURATION FOR PRIMARY DB || MODE : MAXPERFORMANCE ## - out, err := dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf(dbcommons.CreateDGMGRLScriptFile, dbcommons.DataguardBrokerAddDBMaxPerformanceCMD)) - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - log.Info("DGMGRL command file creation output") - log.Info(out) - - out, err = dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf("dgmgrl sys@%s @dgmgrl.cmd < admin.pwd && rm -rf dgmgrl.cmd ", primaryConnectString)) - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - log.Info("DgConfigurationMaxPerformance Output") - log.Info(out) - - } else if m.Spec.ProtectionMode == "MaxAvailability" { - // ## DG CONFIGURATION FOR PRIMARY DB || MODE : MAX AVAILABILITY ## - out, err := dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf(dbcommons.CreateDGMGRLScriptFile, dbcommons.DataguardBrokerAddDBMaxAvailabilityCMD)) - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - log.Info("DGMGRL command file creation output") - log.Info(out) - - out, err = dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf("dgmgrl sys@%s @dgmgrl.cmd < admin.pwd && rm -rf dgmgrl.cmd ", primaryConnectString)) - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - log.Info("DgConfigurationMaxAvailability Output") - log.Info(out) - - } else { - log.Info("SPECIFY correct Protection Mode . Either MaxAvailability or MaxPerformance") - log.Error(err, err.Error()) - return requeueY - } - - databases, _, err = dbcommons.GetDatabasesInDgConfig(sidbReadyPod, r, r.Config, ctx, req) - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - - // ## SET PROPERTY FASTSTARTFAILOVERTARGET FOR EACH DATABASE TO ALL OTHER DATABASES IN DG CONFIG . - if m.Spec.FastStartFailOver.Enable { - for i := 0; i < len(databases); i++ { - out, err = dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf("dgmgrl sys@%s \"EDIT DATABASE %s SET PROPERTY FASTSTARTFAILOVERTARGET=%s\"< admin.pwd", primaryConnectString, - strings.Split(databases[i], ":")[0], getFSFOTargets(i, databases))) - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - log.Info("SETTING FSFO TARGET OUTPUT") - log.Info(out) - - out, err = dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf("dgmgrl sys@%s \"SHOW DATABASE %s FASTSTARTFAILOVERTARGET\" < admin.pwd", primaryConnectString, strings.Split(databases[i], ":")[0])) - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - log.Info("FSFO TARGETS OF " + databases[i]) - log.Info(out) - - } - } - // Remove admin pwd file - _, err = dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, true, "bash", "-c", - dbcommons.RemoveAdminPasswordFile) - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - log.Info("DB Admin pwd file removed") - - // Set DG Configured status to true for this standbyDatabase. so that in next reconcilation, we dont configure this again - standbyDatabase.Status.DgBrokerConfigured = true - r.Status().Update(ctx, standbyDatabase) - - return requeueN -} - -// ############################################################################# -// -// Remove a Database from DG Configuration -// -// ############################################################################# -// -//lint:ignore U1000 deferred for next release -func (r *DataguardBrokerReconciler) removeDatabaseFromDGConfig(m *dbapi.DataguardBroker, n *dbapi.SingleInstanceDatabase, standbyDatabase *dbapi.SingleInstanceDatabase, standbyDatabaseReadyPod corev1.Pod, sidbReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) ctrl.Result { - log := r.Log.WithValues("removeDataguardBrokerConfigurationForGivenDB", req.NamespacedName) - - if standbyDatabaseReadyPod.Name == "" || sidbReadyPod.Name == "" { - return requeueY - } - - // ## CHECK IF DG CONFIGURATION IS AVAILABLE IN PRIMARY DATABASE ## - out, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf("echo -e \"%s\" | dgmgrl / as sysdba", dbcommons.DBShowConfigCMD)) - - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - log.Info("Showconfiguration Output") - log.Info(out) - - if strings.Contains(out, "ORA-16525") { - log.Info("ORA-16525: The Oracle Data Guard broker is not yet available on Primary") - return requeueY - } - - // ## REMOVING STANDBY DATABASE FROM DG CONFIGURATION ## - _, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf(dbcommons.CreateDGMGRLScriptFile, dbcommons.RemoveStandbyDBFromDGConfgCMD)) - - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - - // ## SHOW CONFIGURATION - _, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf("echo -e \"%s\" | dgmgrl / as sysdba", dbcommons.DBShowConfigCMD)) - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - log.Info("Showconfiguration Output") - log.Info(out) - // Set DG Configured status to false for this standbyDatabase. so that in next reconcilation, we dont configure this again - standbyDatabase.Status.DgBrokerConfigured = false - r.Status().Update(ctx, standbyDatabase) - - return requeueN -} - -// ############################################################################# -// -// Return FSFO targets of each StandbyDatabase -// Concatenation of all strings in databases slice expecting that of index 1 -// -// ############################################################################# -func getFSFOTargets(index int, databases []string) string { - fsfotargets := "" - for i := 0; i < len(databases); i++ { - if i != index { - splitstr := strings.Split(databases[i], ":") - if fsfotargets == "" { - fsfotargets = splitstr[0] - } else { - fsfotargets = fsfotargets + "," + splitstr[0] - } - } - } - return fsfotargets -} - -// ##################################################################################################### -// -// Switchovers to 'sid' db to make 'sid' db primary -// -// ##################################################################################################### -func (r *DataguardBrokerReconciler) SetAsPrimaryDatabase(sidbSid string, targetSid string, m *dbapi.DataguardBroker, n *dbapi.SingleInstanceDatabase, - adminPassword string, ctx context.Context, req ctrl.Request) ctrl.Result { - - log := r.Log.WithValues("SetAsPrimaryDatabase", req.NamespacedName) - if targetSid == "" { - log.Info("Specified sid is nil") - return requeueN - } - - // Fetch the SIDB Ready Pod - sidbReadyPod, _, _, _, err := dbcommons.FindPods(r, n.Spec.Image.Version, - (n.Spec.Image.PullFrom), n.Name, n.Namespace, ctx, req) - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - - // Fetch databases in dataguard broker configuration - databases, _, err := dbcommons.GetDatabasesInDgConfig(sidbReadyPod, r, r.Config, ctx, req) - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - - dbInDgConfig := false - for i := 0; i < len(databases); i++ { - splitstr := strings.Split(databases[i], ":") - if strings.ToUpper(splitstr[0]) == strings.ToUpper(targetSid) { - dbInDgConfig = true - break - } - } - - if !dbInDgConfig { - eventReason := "Cannot Switchover" - eventMsg := fmt.Sprintf("Database %s not a part of the dataguard configuration", targetSid) - r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) - return requeueN - } - - // Fetch the current Primary database - primarySid := dbcommons.GetPrimaryDatabase(databases) - if strings.EqualFold(primarySid, targetSid) { - log.Info(targetSid + " is already Primary") - return requeueN - } - - m.Status.Status = dbcommons.StatusUpdating - r.Status().Update(ctx, m) - - found, _ := dbcommons.IsDatabaseFound(targetSid, databases, "") - if !found { - log.Info(targetSid + " not yet set in DG config") - return requeueY - } - - // Fetch the PrimarySid Ready Pod to create chk file - var primaryReq ctrl.Request - var primaryReadyPod corev1.Pod - if !strings.EqualFold(primarySid, sidbSid) { - primaryReq = ctrl.Request{ - NamespacedName: types.NamespacedName{ - Namespace: req.Namespace, - Name: n.Status.StandbyDatabases[strings.ToUpper(primarySid)], - }, - } - primaryReadyPod, _, _, _, err = dbcommons.FindPods(r, "", "", primaryReq.Name, primaryReq.Namespace, ctx, req) - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - } else { - primaryReadyPod = sidbReadyPod - } - - // Fetch the targetSid Ready Pod to create chk file - var targetReq ctrl.Request - var targetReadyPod corev1.Pod - if !strings.EqualFold(targetSid, sidbSid) { - targetReq = ctrl.Request{ - NamespacedName: types.NamespacedName{ - Namespace: req.Namespace, - Name: n.Status.StandbyDatabases[strings.ToUpper(targetSid)], - }, - } - targetReadyPod, _, _, _, err = dbcommons.FindPods(r, "", "", targetReq.Name, targetReq.Namespace, ctx, req) - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - } else { - targetReadyPod = sidbReadyPod - } - - // Create a chk File so that no other pods take the lock during Switchover . - out, err := dbcommons.ExecCommand(r, r.Config, primaryReadyPod.Name, primaryReadyPod.Namespace, "", ctx, req, false, "bash", "-c", dbcommons.CreateChkFileCMD) - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - log.Info("Successfully Created chk file " + out) - out, err = dbcommons.ExecCommand(r, r.Config, targetReadyPod.Name, targetReadyPod.Namespace, "", ctx, req, false, "bash", "-c", dbcommons.CreateChkFileCMD) - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - log.Info("Successfully Created chk file " + out) - - eventReason := "Waiting" - eventMsg := "Switchover In Progress" - r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) - - // Connect to 'primarySid' db using dgmgrl and switchover to 'targetSid' db to make 'targetSid' db primary - _, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", - fmt.Sprintf(dbcommons.CreateAdminPasswordFile, adminPassword)) - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - log.Info("DB Admin pwd file created") - - out, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf("dgmgrl sys@%s \"SWITCHOVER TO %s\" < admin.pwd", primarySid, targetSid)) - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - log.Info("SWITCHOVER TO " + targetSid + " Output") - log.Info(out) - - //Delete pwd file - _, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", - dbcommons.RemoveAdminPasswordFile) - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - log.Info("DB Admin pwd file removed") - - eventReason = "Success" - eventMsg = "Switchover Completed Successfully" - r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) - - // Remove the chk File . - _, err = dbcommons.ExecCommand(r, r.Config, primaryReadyPod.Name, primaryReadyPod.Namespace, "", ctx, req, false, "bash", "-c", dbcommons.RemoveChkFileCMD) - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - out, err = dbcommons.ExecCommand(r, r.Config, targetReadyPod.Name, targetReadyPod.Namespace, "", ctx, req, false, "bash", "-c", dbcommons.RemoveChkFileCMD) - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - log.Info("Successfully Removed chk file " + out) - - // Update Databases - r.updateReconcileStatus(m, sidbReadyPod, ctx, req) - - // Update status of Primary true/false on 'primary' db (From which switchover initiated) - if !strings.EqualFold(primarySid, sidbSid) { - - standbyDatabase := &dbapi.SingleInstanceDatabase{} - err = r.Get(ctx, primaryReq.NamespacedName, standbyDatabase) - if err != nil { - return requeueN - } - out, err := dbcommons.GetDatabaseRole(primaryReadyPod, r, r.Config, ctx, primaryReq) - if err == nil { - standbyDatabase.Status.Role = strings.ToUpper(out) - } - r.Status().Update(ctx, standbyDatabase) - - } else { - sidbReq := ctrl.Request{ - NamespacedName: types.NamespacedName{ - Namespace: req.Namespace, - Name: n.Name, - }, - } - out, err := dbcommons.GetDatabaseRole(sidbReadyPod, r, r.Config, ctx, sidbReq) - if err == nil { - n.Status.Role = strings.ToUpper(out) - } - r.Status().Update(ctx, n) - } - - // Update status of Primary true/false on 'sid' db (To which switchover initiated) - if !strings.EqualFold(targetSid, sidbSid) { - - standbyDatabase := &dbapi.SingleInstanceDatabase{} - err = r.Get(ctx, targetReq.NamespacedName, standbyDatabase) - if err != nil { - return requeueN - } - out, err := dbcommons.GetDatabaseRole(targetReadyPod, r, r.Config, ctx, targetReq) - if err == nil { - standbyDatabase.Status.Role = strings.ToUpper(out) - } - r.Status().Update(ctx, standbyDatabase) - - } else { - sidbReq := ctrl.Request{ - NamespacedName: types.NamespacedName{ - Namespace: req.Namespace, - Name: n.Name, - }, - } - out, err := dbcommons.GetDatabaseRole(sidbReadyPod, r, r.Config, ctx, sidbReq) - if err == nil { - n.Status.Role = strings.ToUpper(out) - } - r.Status().Update(ctx, n) - } - - // Patch DataguardBroker Service to point selector to Current Primary Name and updates client db connection strings on dataguardBroker - result := r.patchService(m, sidbReadyPod, n, ctx, req) - if result.Requeue { - return result - } - - return requeueN -} - -// ############################################################################# -// -// Update Reconcile Status -// -// ############################################################################# -func (r *DataguardBrokerReconciler) updateReconcileStatus(m *dbapi.DataguardBroker, sidbReadyPod corev1.Pod, - ctx context.Context, req ctrl.Request) (err error) { - - // ConnectStrings updated in PatchService() - var databases []string - databases, _, err = dbcommons.GetDatabasesInDgConfig(sidbReadyPod, r, r.Config, ctx, req) - if err == nil { - primaryDatabase := "" - standbyDatabases := "" - for i := 0; i < len(databases); i++ { - splitstr := strings.Split(databases[i], ":") - if strings.ToUpper(splitstr[1]) == "PRIMARY" { - primaryDatabase = strings.ToUpper(splitstr[0]) - } - if strings.ToUpper(splitstr[1]) == "PHYSICAL_STANDBY" { - if standbyDatabases != "" { - standbyDatabases += "," + strings.ToUpper(splitstr[0]) - } else { - standbyDatabases = strings.ToUpper(splitstr[0]) - } - } - } - m.Status.PrimaryDatabase = primaryDatabase - m.Status.StandbyDatabases = standbyDatabases - } - - m.Status.PrimaryDatabaseRef = m.Spec.PrimaryDatabaseRef - m.Status.ProtectionMode = m.Spec.ProtectionMode - return -} - -// ############################################################################# -// -// Manage Finalizer to cleanup before deletion of DataguardBroker -// -// ############################################################################# -func (r *DataguardBrokerReconciler) manageDataguardBrokerDeletion(req ctrl.Request, ctx context.Context, m *dbapi.DataguardBroker) (ctrl.Result, error) { - log := r.Log.WithValues("manageDataguardBrokerDeletion", req.NamespacedName) - - // Check if the DataguardBroker instance is marked to be deleted, which is - // indicated by the deletion timestamp being set. - isDataguardBrokerMarkedToBeDeleted := m.GetDeletionTimestamp() != nil - if isDataguardBrokerMarkedToBeDeleted { - if controllerutil.ContainsFinalizer(m, dataguardBrokerFinalizer) { - // Run finalization logic for dataguardBrokerFinalizer. If the - // finalization logic fails, don't remove the finalizer so - // that we can retry during the next reconciliation. - result, err := r.cleanupDataguardBroker(req, ctx, m) - if result.Requeue { - return result, err - } - - // Remove dataguardBrokerFinalizer. Once all finalizers have been - // removed, the object will be deleted. - controllerutil.RemoveFinalizer(m, dataguardBrokerFinalizer) - err = r.Update(ctx, m) - if err != nil { - log.Error(err, err.Error()) - return requeueY, err - } - } - return requeueY, errors.New("deletion pending") - } - - // Add finalizer for this CR - if !controllerutil.ContainsFinalizer(m, dataguardBrokerFinalizer) { - controllerutil.AddFinalizer(m, dataguardBrokerFinalizer) - err := r.Update(ctx, m) - if err != nil { - log.Error(err, err.Error()) - return requeueY, err - } - } - return requeueN, nil -} - -// ############################################################################# -// -// Finalization logic for DataguardBrokerFinalizer -// -// ############################################################################# -func (r *DataguardBrokerReconciler) cleanupDataguardBroker(req ctrl.Request, ctx context.Context, m *dbapi.DataguardBroker) (ctrl.Result, error) { - log := r.Log.WithValues("cleanupDataguardBroker", req.NamespacedName) - - // Fetch Primary Database Reference - singleInstanceDatabase := &dbapi.SingleInstanceDatabase{} - err := r.Get(ctx, types.NamespacedName{Namespace: req.Namespace, Name: m.Spec.PrimaryDatabaseRef}, singleInstanceDatabase) - if err != nil { - if apierrors.IsNotFound(err) { - r.Log.Info("Resource deleted. No need to remove dataguard configuration") - return requeueN, nil - } - return requeueY, err - } - - // Validate if Primary Database Reference is ready - result, sidbReadyPod, _ := r.validateSidbReadiness(m, singleInstanceDatabase, ctx, req) - if result.Requeue { - r.Log.Info("Reconcile queued") - return result, nil - } - - // Get Primary database to remove dataguard configuration - _, _, err = dbcommons.GetDatabasesInDgConfig(sidbReadyPod, r, r.Config, ctx, req) - if err != nil { - log.Error(err, err.Error()) - return requeueY, err - } - - //primarySid := dbcommons.GetPrimaryDatabase(databases) - - out, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", - fmt.Sprintf("echo -e \"%s\" | dgmgrl / as sysdba ", dbcommons.RemoveDataguardConfiguration)) - if err != nil { - log.Error(err, err.Error()) - return requeueY, err - } - log.Info("RemoveDataguardConfiguration Output") - log.Info(out) - - for i := 0; i < len(m.Spec.StandbyDatabaseRefs); i++ { - - standbyDatabase := &dbapi.SingleInstanceDatabase{} - err := r.Get(ctx, types.NamespacedName{Namespace: req.Namespace, Name: m.Spec.StandbyDatabaseRefs[i]}, standbyDatabase) - if err != nil { - if apierrors.IsNotFound(err) { - continue - } - log.Error(err, err.Error()) - return requeueY, err - } - - // Set DgBrokerConfigured to false - standbyDatabase.Status.DgBrokerConfigured = false - r.Status().Update(ctx, standbyDatabase) - } - - singleInstanceDatabase.Status.DgBrokerConfigured = false - r.Status().Update(ctx, singleInstanceDatabase) - - log.Info("Successfully cleaned up Dataguard Broker") - return requeueN, nil -} - -// ############################################################################# -// -// SetupWithManager sets up the controller with the Manager -// -// ############################################################################# -func (r *DataguardBrokerReconciler) SetupWithManager(mgr ctrl.Manager) error { - return ctrl.NewControllerManagedBy(mgr). - For(&dbapi.DataguardBroker{}). - Owns(&corev1.Pod{}). //Watch for deleted pods of DataguardBroker Owner - WithEventFilter(dbcommons.ResourceEventHandler()). - WithOptions(controller.Options{MaxConcurrentReconciles: 100}). //ReconcileHandler is never invoked concurrently with the same object. - Complete(r) -} diff --git a/controllers/database/dbcssystem_controller.go b/controllers/database/dbcssystem_controller.go index 003be62a..1fd94dde 100644 --- a/controllers/database/dbcssystem_controller.go +++ b/controllers/database/dbcssystem_controller.go @@ -1,5 +1,5 @@ /* -** Copyright (c) 2022 Oracle and/or its affiliates. +** Copyright (c) 2022-2024 Oracle and/or its affiliates. ** ** The Universal Permissive License (UPL), Version 1.0 ** @@ -40,26 +40,35 @@ package controllers import ( "context" + "fmt" "reflect" + "strings" + "time" - databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + databasev4 "github.com/oracle/oracle-database-operator/apis/database/v4" dbcsv1 "github.com/oracle/oracle-database-operator/commons/dbcssystem" "github.com/oracle/oracle-database-operator/commons/finalizer" "github.com/oracle/oracle-database-operator/commons/oci" "github.com/go-logr/logr" + "github.com/oracle/oci-go-sdk/v65/common" "github.com/oracle/oci-go-sdk/v65/core" "github.com/oracle/oci-go-sdk/v65/database" + "github.com/oracle/oci-go-sdk/v65/keymanagement" "github.com/oracle/oci-go-sdk/v65/workrequests" + corev1 "k8s.io/api/core/v1" "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/types" "k8s.io/client-go/tools/record" + "k8s.io/client-go/util/retry" ctrl "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/controller-runtime/pkg/controller" "sigs.k8s.io/controller-runtime/pkg/event" "sigs.k8s.io/controller-runtime/pkg/log" "sigs.k8s.io/controller-runtime/pkg/predicate" + "sigs.k8s.io/controller-runtime/pkg/reconcile" ) // DbcsSystemReconciler reconciles a DbcsSystem object @@ -74,7 +83,7 @@ type DbcsSystemReconciler struct { Recorder record.EventRecorder } -//+kubebuilder:rbac:groups=database.oracle.com,resources=dbcssystems,verbs=get;list;watch;create;update;patch;delete +// +kubebuilder:rbac:groups=database.oracle.com,resources=dbcssystems,verbs=get;list;watch;create;update;patch;delete // +kubebuilder:rbac:groups=database.oracle.com,resources=dbcssystems/status,verbs=get;update;patch // +kubebuilder:rbac:groups=database.oracle.com,resources=dbcssystems/finalizers,verbs=get;create;update;patch;delete // +kubebuilder:rbac:groups="",resources=configmaps;secrets;namespaces,verbs=get;list;watch;create;update;patch;delete @@ -91,12 +100,13 @@ type DbcsSystemReconciler struct { func (r *DbcsSystemReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { r.Logger = log.FromContext(ctx) - // your logic here - - //r.Logger = r.Logv1.WithValues("Instance.Namespace", req.NamespacedName) var err error + resultNq := ctrl.Result{Requeue: false} + resultQ := ctrl.Result{Requeue: true, RequeueAfter: 60 * time.Second} + // Get the dbcs instance from the cluster - dbcsInst := &databasev1alpha1.DbcsSystem{} + dbcsInst := &databasev4.DbcsSystem{} + r.Logger.Info("Reconciling DbSystemDetails", "name", req.NamespacedName) if err := r.KubeClient.Get(context.TODO(), req.NamespacedName, dbcsInst); err != nil { if !errors.IsNotFound(err) { @@ -105,28 +115,35 @@ func (r *DbcsSystemReconciler) Reconcile(ctx context.Context, req ctrl.Request) } // Create oci-go-sdk client - authData := oci.APIKeyAuth{ - ConfigMapName: &dbcsInst.Spec.OCIConfigMap, - SecretName: &dbcsInst.Spec.OCISecret, + authData := oci.ApiKeyAuth{ + ConfigMapName: dbcsInst.Spec.OCIConfigMap, + SecretName: dbcsInst.Spec.OCISecret, Namespace: dbcsInst.GetNamespace(), } - provider, err := oci.GetOCIProvider(r.KubeClient, authData) + provider, err := oci.GetOciProvider(r.KubeClient, authData) if err != nil { - return ctrl.Result{}, err + result := resultNq + return result, err } r.dbClient, err = database.NewDatabaseClientWithConfigurationProvider(provider) if err != nil { - return ctrl.Result{}, err + result := resultNq + return result, err } r.nwClient, err = core.NewVirtualNetworkClientWithConfigurationProvider(provider) if err != nil { - return ctrl.Result{}, err + result := resultNq + return result, err } r.wrClient, err = workrequests.NewWorkRequestClientWithConfigurationProvider(provider) + if err != nil { + result := resultNq + return result, err + } r.Logger.Info("OCI provider configured succesfully") /* @@ -148,111 +165,372 @@ func (r *DbcsSystemReconciler) Reconcile(ctx context.Context, req ctrl.Request) if err := dbcsv1.DeleteDbcsSystemSystem(r.dbClient, *dbcsInst.Spec.Id); err != nil { r.Logger.Error(err, "Fail to terminate DbcsSystem Instance") // Change the status to Failed - if statusErr := dbcsv1.SetLifecycleState(r.KubeClient, r.dbClient, dbcsInst, databasev1alpha1.Terminate, r.nwClient, r.wrClient); statusErr != nil { - return ctrl.Result{}, statusErr + if statusErr := dbcsv1.SetLifecycleState(r.KubeClient, r.dbClient, dbcsInst, databasev4.Terminate, r.nwClient, r.wrClient); statusErr != nil { + result := resultNq + return result, err } // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue - return ctrl.Result{}, nil + result := resultNq + return result, err + } + + // Check if PDBConfig is defined + pdbConfigs := dbcsInst.Spec.PdbConfigs + for _, pdbConfig := range pdbConfigs { + if pdbConfig.PdbName != nil { + // Handle PDB deletion if PluggableDatabaseId is defined and isDelete is true + if pdbConfig.IsDelete != nil && pdbConfig.PluggableDatabaseId != nil && *pdbConfig.IsDelete { + // Call deletePluggableDatabase function + dbSystemId := *dbcsInst.Spec.Id + if err := r.deletePluggableDatabase(ctx, pdbConfig, dbSystemId); err != nil { + result := resultNq + return result, err + } + result := resultNq + return result, err + } + } } + // Remove the finalizer and update the object finalizer.Unregister(r.KubeClient, dbcsInst) r.Logger.Info("Finalizer unregistered successfully.") // Stop reconciliation as the item is being deleted - return ctrl.Result{}, nil + result := resultNq + return result, err } /* Determine whether it's a provision or bind operation */ - lastSucSpec, err := dbcsInst.GetLastSuccessfulSpec() + lastSuccessfullSpec, err := dbcsInst.GetLastSuccessfulSpec() + if err != nil { + return ctrl.Result{}, err + } + lastSuccessfullKMSConfig, err := dbcsInst.GetLastSuccessfulKMSConfig() + if err != nil { + return ctrl.Result{}, err + } + lastSuccessfullKMSStatus, err := dbcsInst.GetLastSuccessfulKMSStatus() if err != nil { return ctrl.Result{}, err } - if dbcsInst.Spec.Id == nil && lastSucSpec == nil { - // If no DbcsSystem ID specified, create a DB System - // ======================== Validate Specs ============== - err = dbcsv1.ValidateSpex(r.Logger, r.KubeClient, r.dbClient, dbcsInst, r.nwClient, r.Recorder) - if err != nil { - return ctrl.Result{}, err - } - r.Logger.Info("DbcsSystem DBSystem provisioning") - dbcsID, err := dbcsv1.CreateAndGetDbcsId(r.Logger, r.KubeClient, r.dbClient, dbcsInst, r.nwClient, r.wrClient) - if err != nil { - r.Logger.Error(err, "Fail to provision and get DbcsSystem System ID") + if lastSuccessfullKMSConfig == nil && lastSuccessfullKMSStatus == nil { - // Change the status to Failed - if statusErr := dbcsv1.SetLifecycleState(r.KubeClient, r.dbClient, dbcsInst, databasev1alpha1.Failed, r.nwClient, r.wrClient); statusErr != nil { - return ctrl.Result{}, statusErr + if dbcsInst.Spec.KMSConfig.KeyName != "" { + + kmsVaultClient, err := keymanagement.NewKmsVaultClientWithConfigurationProvider(provider) + + if err != nil { + return ctrl.Result{}, err } - // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue - return ctrl.Result{}, nil - } - assignDBCSID(dbcsInst, dbcsID) - if err := dbcsv1.UpdateDbcsSystemId(r.KubeClient, dbcsInst); err != nil { - // Change the status to Failed - assignDBCSID(dbcsInst, dbcsID) - if statusErr := dbcsv1.SetLifecycleState(r.KubeClient, r.dbClient, dbcsInst, databasev1alpha1.Failed, r.nwClient, r.wrClient); statusErr != nil { - return ctrl.Result{}, statusErr + // Determine the criteria to identify or locate the vault based on provided information + // Example: Using displayName as a unique identifier (assumed to be unique in this context) + displayName := dbcsInst.Spec.KMSConfig.VaultName + + // Check if a vault with the given displayName exists + getVaultReq := keymanagement.ListVaultsRequest{ + CompartmentId: &dbcsInst.Spec.KMSConfig.CompartmentId, // Assuming compartment ID is known or provided } - return ctrl.Result{}, err - } - r.Logger.Info("DbcsSystem system provisioned succesfully") - assignDBCSID(dbcsInst, dbcsID) - if err := dbcsInst.UpdateLastSuccessfulSpec(r.KubeClient); err != nil { - return ctrl.Result{}, err + listResp, err := kmsVaultClient.ListVaults(ctx, getVaultReq) + if err != nil { + return ctrl.Result{}, fmt.Errorf("error listing vaults: %v", err) + } + + var existingVaultId *string + var existingVaultManagementEndpoint *string + var kmsClient keymanagement.KmsManagementClient + // Find the first active vault with matching displayName + for _, vault := range listResp.Items { + if vault.LifecycleState == keymanagement.VaultSummaryLifecycleStateActive && *vault.DisplayName == displayName { + existingVaultId = vault.Id + existingVaultManagementEndpoint = vault.ManagementEndpoint + // Create KMS Management client + kmsClient, err = keymanagement.NewKmsManagementClientWithConfigurationProvider(provider, *existingVaultManagementEndpoint) + if err != nil { + return ctrl.Result{}, err + } + break + } + } + + // If no active vault found, create a new one + if existingVaultId == nil { + + // Create the KMS vault + createResp, err := r.createKMSVault(ctx, &dbcsInst.Spec.KMSConfig, kmsClient, &dbcsInst.Status.KMSDetailsStatus) + if err != nil { + return ctrl.Result{}, fmt.Errorf("error creating vault: %v", err) + } + existingVaultId = createResp.Id + r.Logger.Info("Created vault Id", existingVaultId) + } else { + // Optionally, perform additional checks or operations if needed + r.Logger.Info("Found existing active vault with displayName", "DisplayName", displayName, "VaultId", *existingVaultId) + dbcsInst.Status.KMSDetailsStatus.VaultId = *existingVaultId + dbcsInst.Status.KMSDetailsStatus.ManagementEndpoint = *existingVaultManagementEndpoint + } + if existingVaultId != nil { + + // Find the key ID based on compartmentID in the existing vault + + listKeysReq := keymanagement.ListKeysRequest{ + CompartmentId: &dbcsInst.Spec.KMSConfig.CompartmentId, + } + + var keyId *string + var keyName *string + + // Make a single request to list keys + listKeysResp, err := kmsClient.ListKeys(ctx, listKeysReq) + if err != nil { + r.Logger.Error(err, "Error listing keys in existing vault") + return ctrl.Result{}, err + } + + // Iterate over the keys to find the desired key + for _, key := range listKeysResp.Items { + if key.DisplayName != nil && *key.DisplayName == dbcsInst.Spec.KMSConfig.KeyName { + keyId = key.Id + keyName = key.DisplayName + dbcsInst.Status.KMSDetailsStatus.KeyId = *key.Id + dbcsInst.Status.KMSDetailsStatus.KeyName = *key.DisplayName + break + } + } + + if keyId == nil { + r.Logger.Info("Master key not found in existing vault, creating new key") + + // Create the KMS key in the existing vault + keyResponse, err := r.createKMSKey(ctx, &dbcsInst.Spec.KMSConfig, kmsClient, &dbcsInst.Status.KMSDetailsStatus) + if err != nil { + return ctrl.Result{}, err + } + + // Update the DbSystem with the encryption key ID + dbcsInst.Status.KMSDetailsStatus.KeyId = *keyResponse.Key.Id + dbcsInst.Status.KMSDetailsStatus.KeyName = *keyResponse.Key.DisplayName + } else { + r.Logger.Info("Found existing master key in vault", "KeyName", dbcsInst.Spec.KMSConfig.KeyName, "KeyId", *keyId) + + // Update the DbSystem with the existing encryption key ID + dbcsInst.Status.KMSDetailsStatus.KeyId = *keyId + dbcsInst.Status.KMSDetailsStatus.KeyName = *keyName + } + } else { + r.Logger.Info("Creating new vault") + + // Create the new vault + vaultResponse, err := r.createKMSVault(ctx, &dbcsInst.Spec.KMSConfig, kmsClient, &dbcsInst.Status.KMSDetailsStatus) + if err != nil { + return ctrl.Result{}, err + } + dbcsInst.Status.KMSDetailsStatus.VaultId = *vaultResponse.Id + dbcsInst.Status.KMSDetailsStatus.ManagementEndpoint = *vaultResponse.ManagementEndpoint + // Create the KMS key in the newly created vault + keyResponse, err := r.createKMSKey(ctx, &dbcsInst.Spec.KMSConfig, kmsClient, &dbcsInst.Status.KMSDetailsStatus) + if err != nil { + return ctrl.Result{}, err + } + + // Update the DbSystem with the encryption key ID + dbcsInst.Status.KMSDetailsStatus.KeyId = *keyResponse.Key.Id + dbcsInst.Status.KMSDetailsStatus.KeyName = *keyResponse.Key.DisplayName + + } } - assignDBCSID(dbcsInst, dbcsID) + } + //debugging + // lastSuccessfullSpec = nil + // r.ensureDBSystemSpec(&dbcsInst.Spec.DbSystem) + // Check if cloning is needed, debugging + // *dbcsInst.Status.DbCloneStatus.Id = "" + setupCloning := false + // Check if SetupDBCloning is true and ensure one of the required fields is provided + if dbcsInst.Spec.SetupDBCloning { + // If SetupDBCloning is true, at least one of Id, DbBackupId, or DatabaseId must be non-nil + if dbcsInst.Spec.Id == nil && dbcsInst.Spec.DbBackupId == nil && dbcsInst.Spec.DatabaseId == nil { + // If none of the required fields are set, log an error and exit the function + r.Logger.Error(err, "SetupDBCloning is defined but other necessary details (Id, DbBackupId, DatabaseId) are not present. Refer README.md file for instructions.") + return ctrl.Result{}, nil + } + // If the condition is met, proceed with cloning setup + setupCloning = true } else { - if lastSucSpec == nil { - if err := dbcsv1.GetDbSystemId(r.Logger, r.dbClient, dbcsInst); err != nil { - // Change the status to Failed - if statusErr := dbcsv1.SetLifecycleState(r.KubeClient, r.dbClient, dbcsInst, databasev1alpha1.Failed, r.nwClient, r.wrClient); statusErr != nil { + // If SetupDBCloning is false, continue as usual without cloning + setupCloning = false + } + + var dbSystemId string + // Executing DB Cloning Process, if defined. Do not repeat cloning again when Status has Id present. + if setupCloning && dbcsInst.Status.DbCloneStatus.Id == nil { + switch { + + case dbcsInst.Spec.SetupDBCloning && dbcsInst.Spec.DbBackupId != nil: + dbSystemId, err = dbcsv1.CloneFromBackupAndGetDbcsId(r.Logger, r.KubeClient, r.dbClient, dbcsInst, r.nwClient, r.wrClient) + if err != nil { + r.Logger.Error(err, "Fail to clone db system from backup and get DbcsSystem System ID") + if statusErr := dbcsv1.SetLifecycleState(r.KubeClient, r.dbClient, dbcsInst, databasev4.Failed, r.nwClient, r.wrClient); statusErr != nil { return ctrl.Result{}, statusErr } - return ctrl.Result{}, err + + return ctrl.Result{}, nil } - if err := dbcsv1.SetDBCSDatabaseLifecycleState(r.Logger, r.KubeClient, r.dbClient, dbcsInst, r.nwClient, r.wrClient); err != nil { - // Change the status to required state + r.Logger.Info("DB Cloning completed successfully from provided backup DB system") + + case dbcsInst.Spec.SetupDBCloning && dbcsInst.Spec.DatabaseId != nil: + dbSystemId, err = dbcsv1.CloneFromDatabaseAndGetDbcsId(r.Logger, r.KubeClient, r.dbClient, dbcsInst, r.nwClient, r.wrClient) + if err != nil { + r.Logger.Error(err, "Fail to clone db system from DatabaseID provided") + if statusErr := dbcsv1.SetLifecycleState(r.KubeClient, r.dbClient, dbcsInst, databasev4.Failed, r.nwClient, r.wrClient); statusErr != nil { + return ctrl.Result{}, statusErr + } + + return ctrl.Result{}, nil + } + r.Logger.Info("DB Cloning completed successfully from provided databaseId") + + case dbcsInst.Spec.SetupDBCloning && dbcsInst.Spec.DbBackupId == nil && dbcsInst.Spec.DatabaseId == nil: + dbSystemId, err = dbcsv1.CloneAndGetDbcsId(r.Logger, r.KubeClient, r.dbClient, dbcsInst, r.nwClient, r.wrClient) + if err != nil { + r.Logger.Error(err, "Fail to clone db system and get DbcsSystem System ID") + if statusErr := dbcsv1.SetLifecycleState(r.KubeClient, r.dbClient, dbcsInst, databasev4.Failed, r.nwClient, r.wrClient); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, nil + } + r.Logger.Info("DB Cloning completed successfully from provided db system") + } + } else if !setupCloning { + if dbcsInst.Spec.Id == nil && lastSuccessfullSpec == nil { + // If no DbcsSystem ID specified, create a new DB System + // ======================== Validate Specs ============== + err = dbcsv1.ValidateSpex(r.Logger, r.KubeClient, r.dbClient, dbcsInst, r.nwClient, r.Recorder) + if err != nil { return ctrl.Result{}, err } + r.Logger.Info("DbcsSystem DBSystem provisioning") + dbcsID, err := dbcsv1.CreateAndGetDbcsId(r.Logger, r.KubeClient, r.dbClient, dbcsInst, r.nwClient, r.wrClient, &dbcsInst.Status.KMSDetailsStatus) + if err != nil { + r.Logger.Error(err, "Fail to provision and get DbcsSystem System ID") + + // Change the status to Failed + if statusErr := dbcsv1.SetLifecycleState(r.KubeClient, r.dbClient, dbcsInst, databasev4.Failed, r.nwClient, r.wrClient); statusErr != nil { + return ctrl.Result{}, statusErr + } + // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue + return ctrl.Result{}, nil + } - dbcsInstId := *dbcsInst.Spec.Id + assignDBCSID(dbcsInst, dbcsID) + // Check if KMSConfig is specified + kmsConfig := dbcsInst.Spec.KMSConfig + if kmsConfig != (databasev4.KMSConfig{}) { + // Check if KMSDetailsStatus is uninitialized (zero value) + if dbcsInst.Spec.DbSystem.KMSConfig != dbcsInst.Spec.KMSConfig { + dbcsInst.Spec.DbSystem.KMSConfig = dbcsInst.Spec.KMSConfig + } + } if err := dbcsv1.UpdateDbcsSystemId(r.KubeClient, dbcsInst); err != nil { // Change the status to Failed - assignDBCSID(dbcsInst, dbcsInstId) - if statusErr := dbcsv1.SetLifecycleState(r.KubeClient, r.dbClient, dbcsInst, databasev1alpha1.Failed, r.nwClient, r.wrClient); statusErr != nil { + assignDBCSID(dbcsInst, dbcsID) + if statusErr := dbcsv1.SetLifecycleState(r.KubeClient, r.dbClient, dbcsInst, databasev4.Failed, r.nwClient, r.wrClient); statusErr != nil { return ctrl.Result{}, statusErr } return ctrl.Result{}, err } - r.Logger.Info("Sync information from remote DbcsSystem System successfully") - - dbcsInstId = *dbcsInst.Spec.Id + r.Logger.Info("DbcsSystem system provisioned succesfully") + assignDBCSID(dbcsInst, dbcsID) if err := dbcsInst.UpdateLastSuccessfulSpec(r.KubeClient); err != nil { return ctrl.Result{}, err } - assignDBCSID(dbcsInst, dbcsInstId) + assignDBCSID(dbcsInst, dbcsID) } else { - if dbcsInst.Spec.Id == nil { - dbcsInst.Spec.Id = lastSucSpec.Id - } + if lastSuccessfullSpec == nil { // first time after creation of DB + if err := dbcsv1.GetDbSystemId(r.Logger, r.dbClient, dbcsInst); err != nil { + // Change the status to Failed + if statusErr := dbcsv1.SetLifecycleState(r.KubeClient, r.dbClient, dbcsInst, databasev4.Failed, r.nwClient, r.wrClient); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, err + } + if err := dbcsv1.SetDBCSDatabaseLifecycleState(r.Logger, r.KubeClient, r.dbClient, dbcsInst, r.nwClient, r.wrClient); err != nil { + // Change the status to required state + return ctrl.Result{}, err + } - if err := dbcsv1.UpdateDbcsSystemIdInst(r.Logger, r.dbClient, dbcsInst, r.KubeClient, r.nwClient, r.wrClient); err != nil { - r.Logger.Error(err, "Fail to update DbcsSystem Id") + dbSystemId := *dbcsInst.Spec.Id + if err := dbcsv1.UpdateDbcsSystemId(r.KubeClient, dbcsInst); err != nil { + // Change the status to Failed + assignDBCSID(dbcsInst, dbSystemId) + if statusErr := dbcsv1.SetLifecycleState(r.KubeClient, r.dbClient, dbcsInst, databasev4.Failed, r.nwClient, r.wrClient); statusErr != nil { + return ctrl.Result{}, statusErr + } + return ctrl.Result{}, err + } - // Change the status to Failed - if statusErr := dbcsv1.SetLifecycleState(r.KubeClient, r.dbClient, dbcsInst, databasev1alpha1.Failed, r.nwClient, r.wrClient); statusErr != nil { - return ctrl.Result{}, statusErr + r.Logger.Info("Sync information from remote DbcsSystem System successfully") + + dbSystemId = *dbcsInst.Spec.Id + if err := dbcsInst.UpdateLastSuccessfulSpec(r.KubeClient); err != nil { + return ctrl.Result{}, err + } + assignDBCSID(dbcsInst, dbSystemId) + } else { + dbSystemId := "" + if dbcsInst.Spec.Id == nil { + dbcsInst.Spec.Id = lastSuccessfullSpec.Id + dbSystemId = *dbcsInst.Spec.Id + } else { + dbSystemId = *dbcsInst.Spec.Id + } + //debugging + // *dbcsInst.Spec.Id = "ocid1.dbsystem.oc1.iad.anuwcljsabf7htya55wz5vfil7ul3pkzpubnymp6zrp3fhgomv3fcdr2vtiq" + compartmentId, err := r.getCompartmentIDByDbSystemID(ctx, *dbcsInst.Spec.Id) + if err != nil { + fmt.Printf("Failed to get compartment ID: %v\n", err) + return ctrl.Result{}, err + } + dbHomeId, err := r.getDbHomeIdByDbSystemID(ctx, compartmentId, *dbcsInst.Spec.Id) + if err != nil { + fmt.Printf("Failed to get DB Home ID: %v\n", err) + return ctrl.Result{}, err + } + + databaseIds, err := r.getDatabaseIDByDbSystemID(ctx, *dbcsInst.Spec.Id, compartmentId, dbHomeId) + if err != nil { + fmt.Printf("Failed to get database IDs: %v\n", err) + return ctrl.Result{}, err + } + err = r.getPluggableDatabaseDetails(ctx, dbcsInst, *dbcsInst.Spec.Id, databaseIds) + if err != nil { + fmt.Printf("Failed to get pluggable database details: %v\n", err) + return ctrl.Result{}, err + } + + if err := dbcsv1.UpdateDbcsSystemIdInst(r.Logger, r.dbClient, dbcsInst, r.KubeClient, r.nwClient, r.wrClient, databaseIds[0]); err != nil { + r.Logger.Error(err, "Fail to update DbcsSystem Id") + + // Change the status to Failed + if statusErr := dbcsv1.SetLifecycleState(r.KubeClient, r.dbClient, dbcsInst, databasev4.Failed, r.nwClient, r.wrClient); statusErr != nil { + return ctrl.Result{}, statusErr + } + // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue + return ctrl.Result{}, nil + } + if err := dbcsv1.SetDBCSDatabaseLifecycleState(r.Logger, r.KubeClient, r.dbClient, dbcsInst, r.nwClient, r.wrClient); err != nil { + // Change the status to required state + return ctrl.Result{}, err + } + // Update Spec and Status + result, err := r.updateSpecsAndStatus(ctx, dbcsInst, dbSystemId) + if err != nil { + return result, err } - // The reconciler should not requeue since the error returned from OCI during update will not be solved by requeue - return ctrl.Result{}, nil - } - if err := dbcsv1.SetDBCSDatabaseLifecycleState(r.Logger, r.KubeClient, r.dbClient, dbcsInst, r.nwClient, r.wrClient); err != nil { - // Change the status to required state - return ctrl.Result{}, err } } } @@ -261,21 +539,914 @@ func (r *DbcsSystemReconciler) Reconcile(ctx context.Context, req ctrl.Request) //r.updateWalletSecret(dbcs) // Update the last succesful spec - dbcsInstId := *dbcsInst.Spec.Id - if err := dbcsInst.UpdateLastSuccessfulSpec(r.KubeClient); err != nil { - return ctrl.Result{}, err + if dbcsInst.Spec.Id != nil { + dbSystemId = *dbcsInst.Spec.Id + + if err := dbcsInst.UpdateLastSuccessfulSpec(r.KubeClient); err != nil { + return ctrl.Result{}, err + } + } else if dbcsInst.Status.DbCloneStatus.Id != nil { + dbSystemId = *dbcsInst.Status.DbCloneStatus.Id } //assignDBCSID(dbcsInst,dbcsI) // Change the phase to "Available" - assignDBCSID(dbcsInst, dbcsInstId) - if statusErr := dbcsv1.SetLifecycleState(r.KubeClient, r.dbClient, dbcsInst, databasev1alpha1.Available, r.nwClient, r.wrClient); statusErr != nil { + assignDBCSID(dbcsInst, dbSystemId) + if statusErr := dbcsv1.SetLifecycleState(r.KubeClient, r.dbClient, dbcsInst, databasev4.Available, r.nwClient, r.wrClient); statusErr != nil { return ctrl.Result{}, statusErr } - return ctrl.Result{}, nil + r.Logger.Info("DBInst after assignment", "dbcsInst:->", dbcsInst) + + // Check if specified PDB exists or needs to be created + exists, err := r.validatePDBExistence(dbcsInst) + if err != nil { + fmt.Printf("Failed to get PDB Details: %v\n", err) + return ctrl.Result{}, err + } + if dbcsInst.Spec.PdbConfigs != nil { + if !exists { + for _, pdbConfig := range dbcsInst.Spec.PdbConfigs { + if pdbConfig.PdbName != nil { + // Get database details + // Get DB Home ID by DB System ID + // Get Compartment ID by DB System ID + compartmentId, err := r.getCompartmentIDByDbSystemID(ctx, dbSystemId) + if err != nil { + fmt.Printf("Failed to get compartment ID: %v\n", err) + return ctrl.Result{}, err + } + dbHomeId, err := r.getDbHomeIdByDbSystemID(ctx, compartmentId, dbSystemId) + if err != nil { + fmt.Printf("Failed to get DB Home ID: %v\n", err) + return ctrl.Result{}, err + } + databaseIds, err := r.getDatabaseIDByDbSystemID(ctx, dbSystemId, compartmentId, dbHomeId) + if err != nil { + fmt.Printf("Failed to get database IDs: %v\n", err) + return ctrl.Result{}, err + } + + // Now you can use dbDetails to access database attributes + r.Logger.Info("Database details fetched successfully", "DatabaseId", databaseIds) + + // Check if deletion is requested + if pdbConfig.IsDelete != nil && *pdbConfig.IsDelete { + // Call deletePluggableDatabase function + if err := r.deletePluggableDatabase(ctx, pdbConfig, dbSystemId); err != nil { + return ctrl.Result{}, err + } + // Continue to the next pdbConfig + continue + } else { + // Call the method to create the pluggable database + r.Logger.Info("Calling createPluggableDatabase", "ctx:->", ctx, "dbcsInst:->", dbcsInst, "databaseIds:->", databaseIds[0], "compartmentId:->", compartmentId) + pdbId, err := r.createPluggableDatabase(ctx, dbcsInst, pdbConfig, databaseIds[0], compartmentId, dbSystemId) + if err != nil { + // Handle error if required + return ctrl.Result{}, err + } + + // Create or update the PDBConfigStatus in DbcsSystemStatus + pdbConfigStatus := databasev4.PDBConfigStatus{ + PdbName: pdbConfig.PdbName, + ShouldPdbAdminAccountBeLocked: pdbConfig.ShouldPdbAdminAccountBeLocked, + PdbLifecycleState: databasev4.Available, + FreeformTags: pdbConfig.FreeformTags, + PluggableDatabaseId: &pdbId, + } + + // Create a map to track existing PDBConfigStatus by PdbName + pdbDetailsMap := make(map[string]databasev4.PDBConfigStatus) + + // Populate the map with existing PDBConfigStatus from dbcsInst.Status.PdbDetailsStatus + for _, pdbDetails := range dbcsInst.Status.PdbDetailsStatus { + for _, existingPdbConfig := range pdbDetails.PDBConfigStatus { + pdbDetailsMap[*existingPdbConfig.PdbName] = existingPdbConfig + } + } + + // Update the map with the new or updated PDBConfigStatus + pdbDetailsMap[*pdbConfig.PdbName] = pdbConfigStatus + + // Convert the map back to a slice of PDBDetailsStatus + var updatedPdbDetailsStatus []databasev4.PDBDetailsStatus + for _, pdbConfigStatus := range pdbDetailsMap { + updatedPdbDetailsStatus = append(updatedPdbDetailsStatus, databasev4.PDBDetailsStatus{ + PDBConfigStatus: []databasev4.PDBConfigStatus{pdbConfigStatus}, + }) + } + + // Assign the updated slice to dbcsInst.Status.PdbDetailsStatus + dbcsInst.Status.PdbDetailsStatus = updatedPdbDetailsStatus + // Update the status in Kubernetes + // Update the status subresource + err = r.KubeClient.Status().Update(ctx, dbcsInst) + if err != nil { + r.Logger.Error(err, "Failed to update DB status") + return reconcile.Result{}, err + } + + } + } + } + } else { + r.Logger.Info("No change in PDB configurations or, already existed PDB Status.") + } + } + // } else { + // r.Logger.Info("No PDB configurations given.") + // } + // r.Logger.Info("DBInst after assignment", "dbcsInst:->", dbcsInst) + // // Check if PDBConfig is defined and needs to be created or deleted + pdbConfigs := dbcsInst.Spec.PdbConfigs + if pdbConfigs != nil { + for _, pdbConfig := range pdbConfigs { + if pdbConfig.PdbName != nil { + // Get database details + // Get DB Home ID by DB System ID + // Get Compartment ID by DB System ID + compartmentId, err := r.getCompartmentIDByDbSystemID(ctx, dbSystemId) + if err != nil { + fmt.Printf("Failed to get compartment ID: %v\n", err) + return ctrl.Result{}, err + } + dbHomeId, err := r.getDbHomeIdByDbSystemID(ctx, compartmentId, dbSystemId) + if err != nil { + fmt.Printf("Failed to get DB Home ID: %v\n", err) + return ctrl.Result{}, err + } + databaseIds, err := r.getDatabaseIDByDbSystemID(ctx, dbSystemId, compartmentId, dbHomeId) + if err != nil { + fmt.Printf("Failed to get database IDs: %v\n", err) + return ctrl.Result{}, err + } + + // Now you can use dbDetails to access database attributes + r.Logger.Info("Database details fetched successfully", "DatabaseId", databaseIds) + + // Check if deletion is requested + if pdbConfig.IsDelete != nil && *pdbConfig.IsDelete { + // Call deletePluggableDatabase function + if err := r.deletePluggableDatabase(ctx, pdbConfig, dbSystemId); err != nil { + return ctrl.Result{}, err + } + // Continue to the next pdbConfig + continue + } else { + // Call the method to create the pluggable database + r.Logger.Info("Calling createPluggableDatabase", "ctx:->", ctx, "dbcsInst:->", dbcsInst, "databaseIds:->", databaseIds[0], "compartmentId:->", compartmentId) + _, err := r.createPluggableDatabase(ctx, dbcsInst, pdbConfig, databaseIds[0], compartmentId, dbSystemId) + if err != nil { + // Handle error if required + return ctrl.Result{}, err + } + } + } + } + } + + return resultQ, nil + +} +func (r *DbcsSystemReconciler) updateSpecsAndStatus(ctx context.Context, dbcsInst *databasev4.DbcsSystem, dbSystemId string) (reconcile.Result, error) { + + // Retry mechanism for handling resource version conflicts + retryErr := retry.RetryOnConflict(retry.DefaultRetry, func() error { + // Fetch the latest version of the resource + latestDbcsInst := &databasev4.DbcsSystem{} + err := r.KubeClient.Get(ctx, types.NamespacedName{ + Name: dbcsInst.Name, + Namespace: dbcsInst.Namespace, + }, latestDbcsInst) + if err != nil { + r.Logger.Error(err, "Failed to fetch the latest DB resource") + return err + } + + // Update the Spec subresource + latestDbcsInst.Spec.Id = &dbSystemId + err = r.KubeClient.Update(ctx, latestDbcsInst) + if err != nil { + r.Logger.Error(err, "Failed to update DB Spec") + return err + } + + // Update the Status subresource + + // Update the Status subresource + originalStatus := reflect.ValueOf(&dbcsInst.Status).Elem() + latestStatus := reflect.ValueOf(&latestDbcsInst.Status).Elem() + + // Iterate over all fields in the Status struct and update them + for i := 0; i < originalStatus.NumField(); i++ { + fieldName := originalStatus.Type().Field(i).Name + latestStatus.FieldByName(fieldName).Set(originalStatus.Field(i)) + } + + err = r.KubeClient.Status().Update(ctx, latestDbcsInst) + if err != nil { + r.Logger.Error(err, "Failed to update DB status") + return err + } + + return nil + }) + + if retryErr != nil { + r.Logger.Error(retryErr, "Failed to update DB Spec and Status after retries") + return reconcile.Result{}, retryErr + } + + r.Logger.Info("Successfully updated Spec and Status") + return reconcile.Result{}, nil +} + +// getDbHomeIdByDbSystemID retrieves the DB Home ID associated with the given DB System ID +func (r *DbcsSystemReconciler) getDbHomeIdByDbSystemID(ctx context.Context, compartmentId, dbSystemId string) (string, error) { + listRequest := database.ListDbHomesRequest{ + CompartmentId: &compartmentId, + DbSystemId: &dbSystemId, + } + + listResponse, err := r.dbClient.ListDbHomes(ctx, listRequest) + if err != nil { + return "", fmt.Errorf("failed to list DB homes: %v", err) + } + + if len(listResponse.Items) == 0 { + return "", fmt.Errorf("no DB homes found for DB system ID: %s", dbSystemId) + } + + return *listResponse.Items[0].Id, nil +} +func (r *DbcsSystemReconciler) getCompartmentIDByDbSystemID(ctx context.Context, dbSystemId string) (string, error) { + // Construct the GetDbSystem request + getRequest := database.GetDbSystemRequest{ + DbSystemId: &dbSystemId, + } + + // Call GetDbSystem API using the existing dbClient + getResponse, err := r.dbClient.GetDbSystem(ctx, getRequest) + if err != nil { + return "", fmt.Errorf("failed to get DB system details: %v", err) + } + + // Extract the compartment ID from the DB system details + compartmentId := *getResponse.DbSystem.CompartmentId + + return compartmentId, nil +} +func (r *DbcsSystemReconciler) getDatabaseIDByDbSystemID(ctx context.Context, dbSystemId, compartmentId, dbHomeId string) ([]string, error) { + // Construct the ListDatabases request + request := database.ListDatabasesRequest{ + SystemId: &dbSystemId, + CompartmentId: &compartmentId, + DbHomeId: &dbHomeId, + } + + // Call ListDatabases API using the existing dbClient + response, err := r.dbClient.ListDatabases(ctx, request) + if err != nil { + return nil, fmt.Errorf("failed to list databases: %v", err) + } + + // Extract database IDs from the response + var databaseIds []string + for _, dbSummary := range response.Items { + databaseIds = append(databaseIds, *dbSummary.Id) + } + + return databaseIds, nil +} +func (r *DbcsSystemReconciler) validatePDBExistence(dbcs *databasev4.DbcsSystem) (bool, error) { + r.Logger.Info("Validating PDB existence for all provided PDBs") + + // Iterate over each PDBConfig in Spec.PdbConfigs + for _, pdbConfig := range dbcs.Spec.PdbConfigs { + pdbName := pdbConfig.PdbName + r.Logger.Info("Checking PDB existence in Status", "PDBName", *pdbName) + + found := false + + // Check if the PDB exists in Status.PdbDetailsStatus with a state of "Available" + for _, pdbDetailsStatus := range dbcs.Status.PdbDetailsStatus { + for _, pdbStatus := range pdbDetailsStatus.PDBConfigStatus { + if pdbStatus.PdbName != nil && *pdbStatus.PdbName == *pdbName && pdbStatus.PdbLifecycleState == "AVAILABLE" { + found = true + break + } + } + if found { + break + } + } + + if !found { + r.Logger.Info("Pluggable database does not exist or is not available in Status.PdbDetailsStatus", "PDBName", *pdbName) + return false, nil + } + } + + // If all PDBs are found and available + r.Logger.Info("All specified PDBs are available") + return true, nil +} +func (r *DbcsSystemReconciler) createPluggableDatabase(ctx context.Context, dbcs *databasev4.DbcsSystem, pdbConfig databasev4.PDBConfig, databaseId, compartmentId, dbSystemId string) (string, error) { + r.Logger.Info("Checking if the pluggable database exists", "PDBName", pdbConfig.PdbName) + + // Check if the pluggable database already exists + exists, pdbId, err := r.doesPluggableDatabaseExist(ctx, compartmentId, pdbConfig.PdbName, databaseId) + if err != nil { + r.Logger.Error(err, "Failed to check if pluggable database exists", "PDBName", pdbConfig.PdbName) + return "", err + } + if exists { + // Set the PluggableDatabaseId in PDBConfig + pdbConfig.PluggableDatabaseId = pdbId + r.Logger.Info("Pluggable database already exists", "PDBName", pdbConfig.PdbName, "PluggableDatabaseId", *pdbConfig.PluggableDatabaseId) + return *pdbId, nil + } + + // Define the DatabaseExists method locally + databaseExists := func(dbSystemID string) (bool, error) { + req := database.GetDbSystemRequest{ + DbSystemId: &dbSystemID, + } + _, err := r.dbClient.GetDbSystem(ctx, req) + if err != nil { + if ociErr, ok := err.(common.ServiceError); ok && ociErr.GetHTTPStatusCode() == 404 { + return false, nil + } + return false, err + } + return true, nil + } + + exists, err = databaseExists(dbSystemId) + if err != nil { + r.Logger.Error(err, "Failed to check database existence") + return "", err + } + + if !exists { + errMsg := fmt.Sprintf("Database does not exist: %s", dbSystemId) + r.Logger.Error(fmt.Errorf(errMsg), "Database not found") + return "", fmt.Errorf(errMsg) + } + + // Fetch secrets for TdeWalletPassword and PdbAdminPassword + tdeWalletPassword, err := r.getSecret(ctx, dbcs.Namespace, *pdbConfig.TdeWalletPassword) + // Trim newline character from the password + tdeWalletPassword = strings.TrimSpace(tdeWalletPassword) + r.Logger.Info("TDE wallet password retrieved successfully") + if err != nil { + r.Logger.Error(err, "Failed to get TDE wallet password secret") + return "", err + } + + pdbAdminPassword, err := r.getSecret(ctx, dbcs.Namespace, *pdbConfig.PdbAdminPassword) + // Trim newline character from the password + pdbAdminPassword = strings.TrimSpace(pdbAdminPassword) + r.Logger.Info("PDB admin password retrieved successfully") + if err != nil { + r.Logger.Error(err, "Failed to get PDB admin password secret") + return "", err + } + // Change the status to Provisioning + if statusErr := dbcsv1.SetLifecycleState(r.KubeClient, r.dbClient, dbcs, databasev4.Provision, r.nwClient, r.wrClient); statusErr != nil { + r.Logger.Error(err, "Failed to set DBCS LifeCycle State to Provisioning") + return "", statusErr + } + r.Logger.Info("Updated DBCS LifeCycle State to Provisioning") + // Proceed with creating the pluggable database + r.Logger.Info("Creating pluggable database", "PDBName", pdbConfig.PdbName) + createPdbReq := database.CreatePluggableDatabaseRequest{ + CreatePluggableDatabaseDetails: database.CreatePluggableDatabaseDetails{ + PdbName: pdbConfig.PdbName, + ContainerDatabaseId: &databaseId, + ShouldPdbAdminAccountBeLocked: pdbConfig.ShouldPdbAdminAccountBeLocked, + PdbAdminPassword: common.String(pdbAdminPassword), + TdeWalletPassword: common.String(tdeWalletPassword), + FreeformTags: pdbConfig.FreeformTags, + }, + } + response, err := r.dbClient.CreatePluggableDatabase(ctx, createPdbReq) + if err != nil { + r.Logger.Error(err, "Failed to create pluggable database", "PDBName", pdbConfig.PdbName) + return "", err + } + // Set the PluggableDatabaseId in PDBConfig + pdbConfig.PluggableDatabaseId = response.PluggableDatabase.Id + + r.Logger.Info("Pluggable database creation initiated", "PDBName", pdbConfig.PdbName, "PDBID", *pdbConfig.PluggableDatabaseId) + + // Polling mechanism to check PDB status + const maxRetries = 120 // total 1 hour wait for creation of PDB + const retryInterval = 30 // in seconds + + for i := 0; i < maxRetries; i++ { + getPdbReq := database.GetPluggableDatabaseRequest{ + PluggableDatabaseId: pdbConfig.PluggableDatabaseId, + } + + getPdbResp, err := r.dbClient.GetPluggableDatabase(ctx, getPdbReq) + if err != nil { + r.Logger.Error(err, "Failed to get pluggable database status", "PDBID", *pdbConfig.PluggableDatabaseId) + return "", err + } + + pdbStatus := getPdbResp.PluggableDatabase.LifecycleState + r.Logger.Info("Checking pluggable database status", "PDBID", *pdbConfig.PluggableDatabaseId, "Status", pdbStatus) + + if pdbStatus == database.PluggableDatabaseLifecycleStateAvailable { + r.Logger.Info("Pluggable database successfully created", "PDBName", pdbConfig.PdbName, "PDBID", *pdbConfig.PluggableDatabaseId) + // Change the status to Available + if statusErr := dbcsv1.SetLifecycleState(r.KubeClient, r.dbClient, dbcs, databasev4.Available, r.nwClient, r.wrClient); statusErr != nil { + return "", statusErr + } + return *response.PluggableDatabase.Id, nil + } + + if pdbStatus == database.PluggableDatabaseLifecycleStateFailed { + r.Logger.Error(fmt.Errorf("pluggable database creation failed"), "PDBName", pdbConfig.PdbName, "PDBID", *pdbConfig.PluggableDatabaseId) + // Change the status to Failed + if statusErr := dbcsv1.SetLifecycleState(r.KubeClient, r.dbClient, dbcs, databasev4.Failed, r.nwClient, r.wrClient); statusErr != nil { + return "", statusErr + } + return "", fmt.Errorf("pluggable database creation failed") + } + + time.Sleep(retryInterval * time.Second) + } + + r.Logger.Error(fmt.Errorf("timed out waiting for pluggable database to become available"), "PDBName", pdbConfig.PdbName, "PDBID", *pdbConfig.PluggableDatabaseId) + return "", fmt.Errorf("timed out waiting for pluggable database to become available") +} + +func (r *DbcsSystemReconciler) pluggableDatabaseExists(ctx context.Context, pluggableDatabaseId string) (bool, error) { + req := database.GetPluggableDatabaseRequest{ + PluggableDatabaseId: &pluggableDatabaseId, + } + _, err := r.dbClient.GetPluggableDatabase(ctx, req) + if err != nil { + if ociErr, ok := err.(common.ServiceError); ok && ociErr.GetHTTPStatusCode() == 404 { + // PDB does not exist + return false, nil + } + // Other error occurred + return false, err + } + // PDB exists + return true, nil +} + +func (r *DbcsSystemReconciler) deletePluggableDatabase(ctx context.Context, pdbConfig databasev4.PDBConfig, dbSystemId string) error { + if pdbConfig.PdbName == nil { + return fmt.Errorf("PDB name is not specified") + } + + r.Logger.Info("Deleting pluggable database", "PDBName", *pdbConfig.PdbName) + + if pdbConfig.PluggableDatabaseId == nil { + r.Logger.Info("PluggableDatabaseId is not specified, getting pluggable databaseID") + // Call a function to retrieve PluggableDatabaseId + pdbID, err := r.getPluggableDatabaseID(ctx, pdbConfig, dbSystemId) + if err != nil { + return fmt.Errorf("failed to get PluggableDatabaseId: %v", err) + } + pdbConfig.PluggableDatabaseId = &pdbID + } + + // Now pdbConfig.PluggableDatabaseId should not be nil + if pdbConfig.PluggableDatabaseId == nil { + return fmt.Errorf("PluggableDatabaseId is still nil after retrieval attempt. Nothing to delete") + } + + // Check if PluggableDatabaseId exists in the live system + exists, err := r.pluggableDatabaseExists(ctx, *pdbConfig.PluggableDatabaseId) + if err != nil { + r.Logger.Error(err, "Failed to check if pluggable database exists", "PluggableDatabaseId", *pdbConfig.PluggableDatabaseId) + return err + } + if !exists { + r.Logger.Info("PluggableDatabaseId does not exist in the live system, nothing to delete", "PluggableDatabaseId", *pdbConfig.PluggableDatabaseId) + return nil + } + + // Define the delete request + deleteReq := database.DeletePluggableDatabaseRequest{ + PluggableDatabaseId: pdbConfig.PluggableDatabaseId, + } + + // Call OCI SDK to delete the PDB + _, err = r.dbClient.DeletePluggableDatabase(ctx, deleteReq) + if err != nil { + r.Logger.Error(err, "Failed to delete pluggable database", "PDBName", *pdbConfig.PdbName) + return err + } + + r.Logger.Info("Successfully deleted pluggable database", "PDBName", *pdbConfig.PdbName) + return nil +} + +func (r *DbcsSystemReconciler) getPluggableDatabaseID(ctx context.Context, pdbConfig databasev4.PDBConfig, dbSystemId string) (string, error) { + compartmentId, err := r.getCompartmentIDByDbSystemID(ctx, dbSystemId) + if err != nil { + fmt.Printf("Failed to get compartment ID: %v\n", err) + return "", err + } + request := database.ListPluggableDatabasesRequest{ + CompartmentId: &compartmentId, + } + + response, err := r.dbClient.ListPluggableDatabases(ctx, request) + if err != nil { + return "", fmt.Errorf("failed to list Pluggable Databases: %v", err) + } + + var pdbID string + + for _, pdb := range response.Items { + if *pdb.PdbName == *pdbConfig.PdbName { + pdbID = *pdb.Id + break + } + } + + if pdbID == "" { + return "", fmt.Errorf("pluggable database '%s' not found", *pdbConfig.PdbName) + } + return pdbID, nil +} + +func (r *DbcsSystemReconciler) getPluggableDatabaseDetails(ctx context.Context, dbcsInst *databasev4.DbcsSystem, dbSystemId string, databaseIds []string) error { + compartmentId, err := r.getCompartmentIDByDbSystemID(ctx, dbSystemId) + if err != nil { + fmt.Printf("Failed to get compartment ID: %v\n", err) + return err + } + request := database.ListPluggableDatabasesRequest{ + CompartmentId: &compartmentId, + } + + response, err := r.dbClient.ListPluggableDatabases(ctx, request) + if err != nil { + return fmt.Errorf("failed to list Pluggable Databases: %v", err) + } + + // Create a map to track existing PDBDetailsStatus by PdbName + pdbDetailsMap := make(map[string]databasev4.PDBConfigStatus) + + // Populate the map with existing PDBDetailsStatus from dbcsInst.Status.PdbDetailsStatus + // for _, existingPdbDetails := range dbcsInst.Status.PdbDetailsStatus { + // for _, existingPdbConfig := range existingPdbDetails.PDBConfigStatus { + // pdbDetailsMap[*existingPdbConfig.PdbName] = existingPdbConfig + // } + // } + // Convert databaseIds array to a set for quick lookup + databaseIdsSet := make(map[string]struct{}) + for _, id := range databaseIds { + databaseIdsSet[id] = struct{}{} + } + // Update the map with new PDB details from the response + for _, pdb := range response.Items { + if pdb.ContainerDatabaseId != nil { + // Check if the ContainerDatabaseId is in the set of databaseIds + if _, exists := databaseIdsSet[*pdb.ContainerDatabaseId]; exists { + pdbConfigStatus := databasev4.PDBConfigStatus{ + PdbName: pdb.PdbName, + ShouldPdbAdminAccountBeLocked: pdb.IsRestricted, + FreeformTags: pdb.FreeformTags, + PluggableDatabaseId: pdb.Id, + PdbLifecycleState: convertLifecycleState(pdb.LifecycleState), + } + + // Update the map with the new or updated PDBConfigStatus + pdbDetailsMap[*pdb.PdbName] = pdbConfigStatus + } + } + } + + // Convert the map back to a slice of PDBDetailsStatus + var updatedPdbDetailsStatus []databasev4.PDBDetailsStatus + for _, pdbConfigStatus := range pdbDetailsMap { + updatedPdbDetailsStatus = append(updatedPdbDetailsStatus, databasev4.PDBDetailsStatus{ + PDBConfigStatus: []databasev4.PDBConfigStatus{pdbConfigStatus}, + }) + } + + // Assign the updated slice to dbcsInst.Status.PdbDetailsStatus + dbcsInst.Status.PdbDetailsStatus = updatedPdbDetailsStatus + + return nil +} + +func convertLifecycleState(state database.PluggableDatabaseSummaryLifecycleStateEnum) databasev4.LifecycleState { + switch state { + case database.PluggableDatabaseSummaryLifecycleStateProvisioning: + return databasev4.Provision + case database.PluggableDatabaseSummaryLifecycleStateAvailable: + return databasev4.Available + case database.PluggableDatabaseSummaryLifecycleStateTerminating: + return databasev4.Terminate + case database.PluggableDatabaseSummaryLifecycleStateTerminated: + return databasev4.LifecycleState(databasev4.Terminated) + case database.PluggableDatabaseSummaryLifecycleStateUpdating: + return databasev4.Update + case database.PluggableDatabaseSummaryLifecycleStateFailed: + return databasev4.Failed + default: + return databasev4.Failed + } +} + +// doesPluggableDatabaseExist checks if a pluggable database with the given name exists +func (r *DbcsSystemReconciler) doesPluggableDatabaseExist(ctx context.Context, compartmentId string, pdbName *string, databaseId string) (bool, *string, error) { + if pdbName == nil { + return false, nil, fmt.Errorf("pdbName is nil") + } + + listPdbsReq := database.ListPluggableDatabasesRequest{ + CompartmentId: &compartmentId, + } + + resp, err := r.dbClient.ListPluggableDatabases(ctx, listPdbsReq) + if err != nil { + return false, nil, err + } + + for _, pdb := range resp.Items { + if pdb.ContainerDatabaseId != nil { + if pdb.PdbName != nil && *pdb.PdbName == *pdbName && pdb.LifecycleState != "TERMINATED" && *pdb.ContainerDatabaseId == databaseId { + return true, pdb.Id, nil + } + } + } + + return false, nil, nil +} + +// Function to create KMS vault +func (r *DbcsSystemReconciler) createKMSVault(ctx context.Context, kmsConfig *databasev4.KMSConfig, kmsClient keymanagement.KmsManagementClient, kmsInst *databasev4.KMSDetailsStatus) (*keymanagement.CreateVaultResponse, error) { + // Dereference the ConfigurationProvider pointer + configProvider := *kmsClient.ConfigurationProvider() + + kmsVaultClient, err := keymanagement.NewKmsVaultClientWithConfigurationProvider(configProvider) + if err != nil { + r.Logger.Error(err, "Error creating KMS vault client") + return nil, err + } + var vaultType keymanagement.CreateVaultDetailsVaultTypeEnum + + if kmsConfig.VaultType != "" { + switch kmsConfig.VaultType { + case "VIRTUAL_PRIVATE": + vaultType = keymanagement.CreateVaultDetailsVaultTypeVirtualPrivate + case "EXTERNAL": + vaultType = keymanagement.CreateVaultDetailsVaultTypeExternal + case "DEFAULT": + vaultType = keymanagement.CreateVaultDetailsVaultTypeDefault + default: + err := fmt.Errorf("unsupported VaultType specified: %s", kmsConfig.VaultType) + r.Logger.Error(err, "unsupported VaultType specified") + return nil, err + } + } else { + // Default to DEFAULT if kmsConfig.VaultType is not defined + vaultType = keymanagement.CreateVaultDetailsVaultTypeDefault + } + + createVaultReq := keymanagement.CreateVaultRequest{ + CreateVaultDetails: keymanagement.CreateVaultDetails{ + CompartmentId: common.String(kmsConfig.CompartmentId), + DisplayName: common.String(kmsConfig.VaultName), + VaultType: vaultType, + }, + } + + resp, err := kmsVaultClient.CreateVault(ctx, createVaultReq) + if err != nil { + r.Logger.Error(err, "Error creating KMS vault") + return nil, err + } + // Wait until vault becomes active or timeout + timeout := time.After(5 * time.Minute) // Example timeout: 5 minutes + ticker := time.NewTicker(10 * time.Second) + defer ticker.Stop() + + for { + select { + case <-timeout: + r.Logger.Error(err, "timed out waiting for vault to become active") + case <-ticker.C: + getVaultReq := keymanagement.GetVaultRequest{ + VaultId: resp.Id, + } + + getResp, err := kmsVaultClient.GetVault(ctx, getVaultReq) + if err != nil { + r.Logger.Error(err, "Error getting vault status") + return nil, err + } + + if getResp.LifecycleState == keymanagement.VaultLifecycleStateActive { + r.Logger.Info("KMS vault created successfully and active") + // Save the vault details into KMSConfig + kmsInst.VaultId = *getResp.Vault.Id + kmsInst.ManagementEndpoint = *getResp.Vault.ManagementEndpoint + kmsInst.VaultName = *getResp.DisplayName + kmsInst.CompartmentId = *getResp.CompartmentId + kmsInst.VaultType = kmsConfig.VaultType + return &keymanagement.CreateVaultResponse{}, err + } + + r.Logger.Info(fmt.Sprintf("Vault state: %s, waiting for active state...", string(getResp.LifecycleState))) + } + } +} + +// Function to create KMS key +func (r *DbcsSystemReconciler) createKMSKey(ctx context.Context, kmsConfig *databasev4.KMSConfig, kmsClient keymanagement.KmsManagementClient, kmsInst *databasev4.KMSDetailsStatus) (*keymanagement.CreateKeyResponse, error) { + // Determine the KeyShape based on the encryption algorithm + var algorithm keymanagement.KeyShapeAlgorithmEnum + var keyLength int + switch kmsConfig.EncryptionAlgo { + case "AES": + algorithm = keymanagement.KeyShapeAlgorithmAes + keyLength = 32 + case "RSA": + algorithm = keymanagement.KeyShapeAlgorithmRsa + keyLength = 512 + default: + // Default to AES if the provided algorithm is unsupported + algorithm = keymanagement.KeyShapeAlgorithmAes + keyLength = 32 + r.Logger.Info("Unsupported encryption algorithm. Defaulting to AES.") + } + + // Create the key shape with the algorithm + keyShape := keymanagement.KeyShape{ + Algorithm: algorithm, + Length: common.Int(keyLength), + } + + createKeyReq := keymanagement.CreateKeyRequest{ + CreateKeyDetails: keymanagement.CreateKeyDetails{ + CompartmentId: common.String(kmsConfig.CompartmentId), + DisplayName: common.String(kmsConfig.KeyName), + KeyShape: &keyShape, + }, + RequestMetadata: common.RequestMetadata{}, + } + + // Call CreateKey without vaultID + resp, err := kmsClient.CreateKey(ctx, createKeyReq) + if err != nil { + r.Logger.Error(err, "Error creating KMS key:") + return nil, err + } + + r.Logger.Info("KMS key created successfully:", resp) + kmsInst.KeyId = *resp.Key.Id + kmsInst.EncryptionAlgo = string(algorithm) + return &resp, nil +} + +func (r *DbcsSystemReconciler) getSecret(ctx context.Context, namespace, secretName string) (string, error) { + secret := &corev1.Secret{} + err := r.KubeClient.Get(ctx, types.NamespacedName{Namespace: namespace, Name: secretName}, secret) + if err != nil { + return "", err + } + + // Assume the secret contains only one key-value pair + for _, value := range secret.Data { + return string(value), nil + } + + return "", fmt.Errorf("secret %s is empty", secretName) +} + +// func (r *DbcsSystemReconciler) cloneDbSystem(ctx context.Context, dbcsInst *databasev4.DbcsSystem, provider common.ConfigurationProvider) error { + +// // Initialize OCI clients +// dbClient, err := database.NewDatabaseClientWithConfigurationProvider(provider) +// if err != nil { +// return fmt.Errorf("failed to create OCI database client: %v", err) +// } + +// // Get DB System details +// compartmentId, err := r.getCompartmentIDByDbSystemID(ctx, *dbcsInst.Status.Id) +// if err != nil { +// fmt.Printf("Failed to get compartment ID: %v\n", err) +// return err +// } + +// dbHomeId, err := r.getDbHomeIdByDbSystemID(ctx, compartmentId, *dbcsInst.Status.Id) +// if err != nil { +// fmt.Printf("Failed to get DB Home ID: %v\n", err) +// return err +// } + +// databaseIds, err := r.getDatabaseIDByDbSystemID(ctx, *dbcsInst.Status.Id, compartmentId, dbHomeId) +// if err != nil { +// fmt.Printf("Failed to get database IDs: %v\n", err) +// return err +// } + +// // Use the first database ID for cloning +// if len(databaseIds) == 0 { +// return fmt.Errorf("no databases found in the DB system") +// } + +// // Retrieve details of the database to clone +// sourceDatabaseId := databaseIds[0] +// _, err = dbClient.GetDatabase(ctx, database.GetDatabaseRequest{ +// DatabaseId: common.String(sourceDatabaseId), +// }) +// if err != nil { +// return fmt.Errorf("failed to get source database details: %v", err) +// } + +// // adminPassword, err := dbcsv1.GetAdminPassword(kubeClient, dbcsInstance) +// // if err != nil { +// // log.Fatalf("Error getting admin password: %v", err) +// // } + +// // tdePassword, err := GetTdePassword(kubeClient, dbcsInstance) +// // if err != nil { +// // log.Fatalf("Error getting TDE password: %v", err) +// // } + +// // Define the details for creating the database from the existing DB system +// // createDatabaseDetails := CreateDatabaseBaseWrapper{ +// // CreateDatabaseFromDbSystemDetails: database.CreateDatabaseFromDbSystemDetails{ +// // AdminPassword: common.String(adminPassword), // Replace with actual admin password +// // DbName: common.String(dbcsInst.Spec.DbSystem.DbName), // Use the dbName from DbcsSystemSpec +// // DbDomain: common.String(dbcsInst.Spec.DbSystem.DbDomain), // Use the dbDomain from DbcsSystemSpec +// // DbUniqueName: common.String(dbcsInst.Spec.DbSystem.DbUniqueName), // Use the dbUniqueName from DbcsSystemSpec +// // DbBackupConfig: &database.DbBackupConfig{ +// // AutoBackupEnabled: dbcsInst.Spec.DbSystem.DbBackupConfig.AutoBackupEnabled, +// // RecoveryWindowInDays: dbcsInst.Spec.DbSystem.DbBackupConfig.RecoveryWindowsInDays, +// // }, +// // FreeformTags: dbcsInst.Spec.DbSystem.Tags, +// // DefinedTags: map[string]map[string]interface{}{ +// // "Namespace": { +// // "TagKey": "TagValue", // Replace with actual defined tags if needed +// // }, +// // }, +// // }, +// // } +// // createDatabaseRequest := database.CreateDatabaseRequest{ +// // CreateNewDatabaseDetails: &createDatabaseDetails, +// // } + +// // createDatabaseResponse, err := dbClient.CreateDatabase(ctx, createDatabaseRequest) +// // if err != nil { +// // return fmt.Errorf("failed to create database from DB system: %v", err) +// // } + +// // // Update instance status with the new database ID +// // dbcsInst.Status.DbInfo = append(dbcsInst.Status.DbInfo, databasev4.DbStatus{ +// // Id: createDatabaseResponse.Database.Id, +// // DbName: dbcsInst.Spec.DbSystem.DbName, +// // DbUniqueName: dbcsInst.Spec.DbSystem.DbUniqueName, +// // }) + +// // err = r.KubeClient.Status().Update(ctx, dbcsInst) +// // if err != nil { +// // return fmt.Errorf("failed to update instance status with database ID: %v", err) +// // } + +// return nil +// } + +// Convert DbBackupConfigAutoBackupWindowEnum to *string +func autoBackupWindowEnumToStringPtr(enum *database.DbBackupConfigAutoBackupWindowEnum) *string { + if enum == nil { + return nil + } + value := string(*enum) + return &value +} +func (r *DbcsSystemReconciler) stringToDbBackupConfigAutoBackupWindowEnum(value *string) (database.DbBackupConfigAutoBackupWindowEnum, error) { + // Define a default value + // Define a default value + const defaultAutoBackupWindow = database.DbBackupConfigAutoBackupWindowOne + + if value == nil { + return defaultAutoBackupWindow, nil // Return the default value + } + + // Convert to enum + enum, ok := database.GetMappingDbBackupConfigAutoBackupWindowEnum(*value) + if !ok { + return "", fmt.Errorf("invalid value for AutoBackupWindow: %s", *value) + } + return enum, nil } -func assignDBCSID(dbcsInst *databasev1alpha1.DbcsSystem, dbcsID string) { +func assignDBCSID(dbcsInst *databasev4.DbcsSystem, dbcsID string) { dbcsInst.Spec.Id = &dbcsID } @@ -286,8 +1457,8 @@ func (r *DbcsSystemReconciler) eventFilterPredicate() predicate.Predicate { }, UpdateFunc: func(e event.UpdateEvent) bool { // Get the dbName as old dbName when an update event happens - oldObject := e.ObjectOld.DeepCopyObject().(*databasev1alpha1.DbcsSystem) - newObject := e.ObjectNew.DeepCopyObject().(*databasev1alpha1.DbcsSystem) + oldObject := e.ObjectOld.DeepCopyObject().(*databasev4.DbcsSystem) + newObject := e.ObjectNew.DeepCopyObject().(*databasev4.DbcsSystem) specObject := !reflect.DeepEqual(oldObject.Spec, newObject.Spec) deletionTimeStamp := !reflect.DeepEqual(oldObject.GetDeletionTimestamp(), newObject.GetDeletionTimestamp()) @@ -307,7 +1478,7 @@ func (r *DbcsSystemReconciler) eventFilterPredicate() predicate.Predicate { // SetupWithManager sets up the controller with the Manager. func (r *DbcsSystemReconciler) SetupWithManager(mgr ctrl.Manager) error { return ctrl.NewControllerManagedBy(mgr). - For(&databasev1alpha1.DbcsSystem{}). + For(&databasev4.DbcsSystem{}). WithEventFilter(r.eventFilterPredicate()). WithOptions(controller.Options{MaxConcurrentReconciles: 50}). Complete(r) diff --git a/controllers/database/lrest_controller.go b/controllers/database/lrest_controller.go new file mode 100644 index 00000000..91c883e1 --- /dev/null +++ b/controllers/database/lrest_controller.go @@ -0,0 +1,1105 @@ +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package controllers + +import ( + "context" + "encoding/json" + "errors" + "fmt" + + //"fmt" + "strconv" + "strings" + "time" + + "github.com/go-logr/logr" + appsv1 "k8s.io/api/apps/v1" + corev1 "k8s.io/api/core/v1" + apierrors "k8s.io/apimachinery/pkg/api/errors" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/types" + "k8s.io/client-go/kubernetes" + "k8s.io/client-go/rest" + "k8s.io/client-go/tools/record" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/controller" + "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" + "sigs.k8s.io/controller-runtime/pkg/event" + "sigs.k8s.io/controller-runtime/pkg/predicate" + + dbapi "github.com/oracle/oracle-database-operator/apis/database/v4" + dbcommons "github.com/oracle/oracle-database-operator/commons/database" + //lrcommons "github.com/oracle/oracle-database-operator/commons/multitenant/lrest" +) + +// LRESTReconciler reconciles a LREST object +type LRESTReconciler struct { + client.Client + Scheme *runtime.Scheme + Config *rest.Config + Log logr.Logger + Interval time.Duration + Recorder record.EventRecorder +} + +var ( + lrestPhaseInit = "Initializing" + lrestPhasePod = "CreatingPod" + lrestPhaseValPod = "ValidatingPods" + lrestPhaseService = "CreatingService" + lrestPhaseSecrets = "DeletingSecrets" + lrestPhaseReady = "Ready" + lrestPhaseDelete = "Deleting" + lrestPhaseFail = "Failed" +) + +const LRESTFinalizer = "database.oracle.com/LRESTfinalizer" + +//+kubebuilder:rbac:groups=database.oracle.com,resources=lrests,verbs=get;list;watch;create;update;patch;delete +//+kubebuilder:rbac:groups=database.oracle.com,resources=lrests/status,verbs=get;update;patch +//+kubebuilder:rbac:groups=database.oracle.com,resources=lrests/finalizers,verbs=update +//+kubebuilder:rbac:groups="",resources=pods;pods/log;pods/exec;services;configmaps;events;replicasets,verbs=create;delete;get;list;patch;update;watch +//+kubebuilder:rbac:groups=core,resources=pods;secrets;services;configmaps;namespaces,verbs=get;list;watch;create;update;patch;delete +//+kubebuilder:rbac:groups=apps,resources=replicasets,verbs=get;list;watch;create;update;patch;delete + +// Reconcile is part of the main kubernetes reconciliation loop which aims to +// move the current state of the cluster closer to the desired state. +// TODO(user): Modify the Reconcile function to compare the state specified by +// the LREST object against the actual cluster state, and then +// perform operations to make the cluster state reflect the state specified by +// the user. +// +// For more details, check Reconcile and its Result here: +// - https://pkg.go.dev/sigs.k8s.io/controller-runtime@v0.9.2/pkg/reconcile +func (r *LRESTReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + + log := r.Log.WithValues("multitenantoperator", req.NamespacedName) + log.Info("Reconcile requested") + + reconcilePeriod := r.Interval * time.Second + requeueY := ctrl.Result{Requeue: true, RequeueAfter: reconcilePeriod} + requeueN := ctrl.Result{} + + var err error + lrest := &dbapi.LREST{} + + // Execute for every reconcile + defer func() { + log.Info("DEFER", "Name", lrest.Name, "Phase", lrest.Status.Phase, "Status", strconv.FormatBool(lrest.Status.Status)) + if !lrest.Status.Status { + if err := r.Status().Update(ctx, lrest); err != nil { + log.Error(err, "Failed to update status for :"+lrest.Name, "err", err.Error()) + } + } + }() + + err = r.Client.Get(context.TODO(), req.NamespacedName, lrest) + if err != nil { + if apierrors.IsNotFound(err) { + log.Info("LREST Resource Not found", "Name", lrest.Name) + // Request object not found, could have been deleted after reconcile req. + // Owned objects are automatically garbage collected. For additional cleanup logic use finalizers. + // Return and don't requeue + lrest.Status.Status = true + return requeueN, nil + } + // Error reading the object - requeue the req. + return requeueY, err + } + + log.Info("Res Status:", "Name", lrest.Name, "Phase", lrest.Status.Phase, "Status", strconv.FormatBool(lrest.Status.Status)) + + // Finalizer section + err = r.manageLRESTDeletion(ctx, req, lrest) + if err != nil { + log.Info("Reconcile queued") + return requeueY, nil + } + + // If post-creation, LREST spec is changed, check and take appropriate action + if (lrest.Status.Phase == lrestPhaseReady) && lrest.Status.Status { + r.evaluateSpecChange(ctx, req, lrest) + } + + if !lrest.Status.Status { + phase := lrest.Status.Phase + log.Info("Current Phase:"+phase, "Name", lrest.Name) + + switch phase { + case lrestPhaseInit: + err = r.verifySecrets(ctx, req, lrest) + if err != nil { + lrest.Status.Phase = lrestPhaseFail + return requeueN, nil + } + lrest.Status.Phase = lrestPhasePod + case lrestPhasePod: + // Create LREST PODs + err = r.createLRESTInstances(ctx, req, lrest) + if err != nil { + log.Info("Reconcile queued") + return requeueY, nil + } + lrest.Status.Phase = lrestPhaseValPod + case lrestPhaseValPod: + // Validate LREST PODs + err = r.validateLRESTPods(ctx, req, lrest) + if err != nil { + if lrest.Status.Phase == lrestPhaseFail { + return requeueN, nil + } + log.Info("Reconcile queued") + return requeueY, nil + } + lrest.Status.Phase = lrestPhaseService + case lrestPhaseService: + // Create LREST Service + err = r.createLRESTSVC(ctx, req, lrest) + if err != nil { + log.Info("Reconcile queued") + return requeueY, nil + } + //lrest.Status.Phase = lrestPhaseSecrets + lrest.Status.Phase = lrestPhaseReady + case lrestPhaseSecrets: + // Delete LREST Secrets + //r.deleteSecrets(ctx, req, lrest) + lrest.Status.Phase = lrestPhaseReady + lrest.Status.Msg = "Success" + case lrestPhaseReady: + lrest.Status.Status = true + r.Status().Update(ctx, lrest) + return requeueN, nil + default: + lrest.Status.Phase = lrestPhaseInit + log.Info("DEFAULT:", "Name", lrest.Name, "Phase", phase, "Status", strconv.FormatBool(lrest.Status.Status)) + } + + if err := r.Status().Update(ctx, lrest); err != nil { + log.Error(err, "Failed to update status for :"+lrest.Name, "err", err.Error()) + } + return requeueY, nil + } + + log.Info("Reconcile completed") + return requeueN, nil +} + +/* +********************************************************* + - Create a ReplicaSet for pods based on the LREST container + /******************************************************* +*/ +func (r *LRESTReconciler) createLRESTInstances(ctx context.Context, req ctrl.Request, lrest *dbapi.LREST) error { + + log := r.Log.WithValues("createLRESTInstances", req.NamespacedName) + + replicaSet := r.createReplicaSetSpec(lrest) + + foundRS := &appsv1.ReplicaSet{} + err := r.Get(context.TODO(), types.NamespacedName{Name: replicaSet.Name, Namespace: lrest.Namespace}, foundRS) + if err != nil && apierrors.IsNotFound(err) { + log.Info("Creating LREST Replicaset: " + replicaSet.Name) + err = r.Create(ctx, replicaSet) + if err != nil { + log.Error(err, "Failed to create ReplicaSet for :"+lrest.Name, "Namespace", replicaSet.Namespace, "Name", replicaSet.Name) + return err + } + } else if err != nil { + log.Error(err, "Replicaset : "+replicaSet.Name+" already exists.") + return err + } + + // Set LREST instance as the owner and controller + ctrl.SetControllerReference(lrest, replicaSet, r.Scheme) + + log.Info("Created LREST ReplicaSet successfully") + r.Recorder.Eventf(lrest, corev1.EventTypeNormal, "CreatedLRESTReplicaSet", "Created LREST Replicaset (Replicas - %s) for %s", strconv.Itoa(lrest.Spec.Replicas), lrest.Name) + return nil +} + +/* +************************************************ + - Validate LREST Pod. Check if there are any errors + /*********************************************** +*/ +func (r *LRESTReconciler) validateLRESTPods(ctx context.Context, req ctrl.Request, lrest *dbapi.LREST) error { + + log := r.Log.WithValues("validateLRESTPod", req.NamespacedName) + + log.Info("Validating Pod creation for :" + lrest.Name) + + podName := lrest.Name + "-lrest" + podList := &corev1.PodList{} + listOpts := []client.ListOption{client.InNamespace(req.Namespace), client.MatchingLabels{"name": podName}} + + // List retrieves list of objects for a given namespace and list options. + err := r.List(ctx, podList, listOpts...) + if err != nil { + log.Info("Failed to list pods of: "+podName, "Namespace", req.Namespace) + return err + } + + if len(podList.Items) == 0 { + log.Info("No pods found for: "+podName, "Namespace", req.Namespace) + lrest.Status.Msg = "Waiting for LREST Pod(s) to start" + return errors.New("Waiting for LREST pods to start") + } + + getLRESTStatus := " curl --cert /opt/oracle/lrest/certificates/tls.crt --cacert /opt/oracle/lrest/certificates/ca.crt --key /opt/oracle/lrest/certificates/tls.key -u `cat /opt/oracle/lrest/certificates/webserver_user`:`cat /opt/oracle/lrest/certificates/webserver_pwd` -sSkv -k -X GET https://localhost:" + strconv.Itoa(lrest.Spec.LRESTPort) + "/database/pdbs/" + readyPods := 0 + for _, pod := range podList.Items { + if pod.Status.Phase == corev1.PodRunning { + // Get LREST Status + out, err := dbcommons.ExecCommand(r, r.Config, pod.Name, pod.Namespace, "", ctx, req, false, "bash", "-c", getLRESTStatus) + if strings.Contains(out, "HTTP/1.1 200 OK") || strings.Contains(strings.ToUpper(err.Error()), "HTTP/1.1 200 OK") || + strings.Contains(out, "HTTP/2") || strings.Contains(strings.ToUpper(err.Error()), " HTTP/2") { + readyPods++ + } else if strings.Contains(out, "HTTP/1.1 404 Not Found") || strings.Contains(strings.ToUpper(err.Error()), "HTTP/1.1 404 NOT FOUND") || strings.Contains(strings.ToUpper(err.Error()), "HTTP/2 404") || strings.Contains(strings.ToUpper(err.Error()), "Failed to connect to localhost") { + // Check if DB connection parameters are correct + getLRESTInstallStatus := " grep -q 'Failed to' /tmp/lrest_install.log; echo $?;" + out, _ := dbcommons.ExecCommand(r, r.Config, pod.Name, pod.Namespace, "", ctx, req, false, "bash", "-c", getLRESTInstallStatus) + if strings.TrimSpace(out) == "0" { + lrest.Status.Msg = "Check DB connection parameters" + lrest.Status.Phase = lrestPhaseFail + // Delete existing ReplicaSet + r.deleteReplicaSet(ctx, req, lrest) + return errors.New("Check DB connection parameters") + } + } + } + } + + if readyPods != lrest.Spec.Replicas { + log.Info("Replicas: "+strconv.Itoa(lrest.Spec.Replicas), "Ready Pods: ", readyPods) + lrest.Status.Msg = "Waiting for LREST Pod(s) to be ready" + return errors.New("Waiting for LREST pods to be ready") + } + + lrest.Status.Msg = "" + return nil +} + +/* +*********************** + - Create Pod spec + +/*********************** +*/ +func (r *LRESTReconciler) createPodSpec(lrest *dbapi.LREST) corev1.PodSpec { + + podSpec := corev1.PodSpec{ + Volumes: []corev1.Volume{{ + Name: "secrets", + VolumeSource: corev1.VolumeSource{ + Projected: &corev1.ProjectedVolumeSource{ + DefaultMode: func() *int32 { i := int32(0666); return &i }(), + Sources: []corev1.VolumeProjection{ + { + Secret: &corev1.SecretProjection{ + LocalObjectReference: corev1.LocalObjectReference{ + Name: lrest.Spec.LRESTPubKey.Secret.SecretName, + }, + Items: []corev1.KeyToPath{ + { + Key: lrest.Spec.LRESTPubKey.Secret.Key, + Path: lrest.Spec.LRESTPubKey.Secret.Key, + }, + }, + }, + }, + { + Secret: &corev1.SecretProjection{ + LocalObjectReference: corev1.LocalObjectReference{ + Name: lrest.Spec.LRESTPriKey.Secret.SecretName, + }, + Items: []corev1.KeyToPath{ + { + Key: lrest.Spec.LRESTPriKey.Secret.Key, + Path: lrest.Spec.LRESTPriKey.Secret.Key, + }, + }, + }, + }, + + /***/ + { + Secret: &corev1.SecretProjection{ + LocalObjectReference: corev1.LocalObjectReference{ + Name: lrest.Spec.LRESTTlsKey.Secret.SecretName, + }, + Items: []corev1.KeyToPath{ + { + Key: lrest.Spec.LRESTTlsKey.Secret.Key, + Path: lrest.Spec.LRESTTlsKey.Secret.Key, + }, + }, + }, + }, + { + Secret: &corev1.SecretProjection{ + LocalObjectReference: corev1.LocalObjectReference{ + Name: lrest.Spec.LRESTTlsCrt.Secret.SecretName, + }, + Items: []corev1.KeyToPath{ + { + Key: lrest.Spec.LRESTTlsCrt.Secret.Key, + Path: lrest.Spec.LRESTTlsCrt.Secret.Key, + }, + }, + }, + }, + }, + }, + }, + }}, + SecurityContext: &corev1.PodSecurityContext{ + RunAsNonRoot: &[]bool{true}[0], + FSGroup: &[]int64{54321}[0], + SeccompProfile: &corev1.SeccompProfile{ + Type: corev1.SeccompProfileTypeRuntimeDefault, + }, + }, + /*InitContainers: []corev1.Container{{ + Image: lrest.Spec.LRESTImage, + Name: lrest.Name + "-init", + ImagePullPolicy: corev1.PullIfNotPresent, + SecurityContext: securityContextDefineLrest(), + Command: []string{"echo test > /opt/oracle/lrest/certificates/tests"}, + Env: func() []corev1.EnvVar { + return []corev1.EnvVar{ + { + Name: "ORACLE_HOST", + Value: lrest.Spec.DBTnsurl, + }} + }(), + VolumeMounts: []corev1.VolumeMount{ + { + MountPath: "/opt/oracle/lrest/certificates", + Name: "secrets", + ReadOnly: false, + }}, + }},*/ + Containers: []corev1.Container{{ + Image: lrest.Spec.LRESTImage, + Name: lrest.Name + "-lrest", + ImagePullPolicy: corev1.PullIfNotPresent, + SecurityContext: securityContextDefineLrest(), + VolumeMounts: []corev1.VolumeMount{ + { + MountPath: "/opt/oracle/lrest/certificates", + Name: "secrets", + ReadOnly: true, + }, + }, + Env: func() []corev1.EnvVar { + return []corev1.EnvVar{ + { + Name: "ORACLE_HOST", + Value: lrest.Spec.DBServer, + }, + { + Name: "DBTNSURL", + Value: lrest.Spec.DBTnsurl, + }, + { + Name: "TLSCRT", + Value: lrest.Spec.LRESTTlsCrt.Secret.Key, + }, + { + Name: "TLSKEY", + Value: lrest.Spec.LRESTTlsKey.Secret.Key, + }, + { + Name: "PUBKEY", + Value: lrest.Spec.LRESTPubKey.Secret.Key, + }, + { + Name: "PRVKEY", + Value: lrest.Spec.LRESTPriKey.Secret.Key, + }, + { + Name: "ORACLE_PORT", + Value: strconv.Itoa(lrest.Spec.DBPort), + }, + { + Name: "LREST_PORT", + Value: strconv.Itoa(lrest.Spec.LRESTPort), + }, + { + Name: "ORACLE_SERVICE", + Value: lrest.Spec.ServiceName, + }, + { + Name: "R1", + ValueFrom: &corev1.EnvVarSource{ + SecretKeyRef: &corev1.SecretKeySelector{ + LocalObjectReference: corev1.LocalObjectReference{ + Name: lrest.Spec.LRESTAdminUser.Secret.SecretName, + }, + Key: lrest.Spec.LRESTAdminUser.Secret.Key, + }, + }, + }, + { + Name: "R2", + ValueFrom: &corev1.EnvVarSource{ + SecretKeyRef: &corev1.SecretKeySelector{ + LocalObjectReference: corev1.LocalObjectReference{ + Name: lrest.Spec.LRESTAdminPwd.Secret.SecretName, + }, + Key: lrest.Spec.LRESTAdminPwd.Secret.Key, + }, + }, + }, + { + Name: "R3", + ValueFrom: &corev1.EnvVarSource{ + SecretKeyRef: &corev1.SecretKeySelector{ + LocalObjectReference: corev1.LocalObjectReference{ + Name: lrest.Spec.WebLrestServerUser.Secret.SecretName, + }, + Key: lrest.Spec.WebLrestServerUser.Secret.Key, + }, + }, + }, + { + Name: "R4", + ValueFrom: &corev1.EnvVarSource{ + SecretKeyRef: &corev1.SecretKeySelector{ + LocalObjectReference: corev1.LocalObjectReference{ + Name: lrest.Spec.WebLrestServerPwd.Secret.SecretName, + }, + Key: lrest.Spec.WebLrestServerPwd.Secret.Key, + }, + }, + }, + } + }(), + }}, + + NodeSelector: func() map[string]string { + ns := make(map[string]string) + if len(lrest.Spec.NodeSelector) != 0 { + for key, value := range lrest.Spec.NodeSelector { + ns[key] = value + } + } + return ns + }(), + } + + if len(lrest.Spec.LRESTImagePullSecret) > 0 { + podSpec.ImagePullSecrets = []corev1.LocalObjectReference{ + { + Name: lrest.Spec.LRESTImagePullSecret, + }, + } + } + + podSpec.Containers[0].ImagePullPolicy = corev1.PullAlways + + if len(lrest.Spec.LRESTImagePullPolicy) > 0 { + if strings.ToUpper(lrest.Spec.LRESTImagePullPolicy) == "NEVER" { + podSpec.Containers[0].ImagePullPolicy = corev1.PullNever + } + } + + return podSpec +} + +/* +*********************** + - Create ReplicaSet spec + +/*********************** +*/ +func (r *LRESTReconciler) createReplicaSetSpec(lrest *dbapi.LREST) *appsv1.ReplicaSet { + + replicas := int32(lrest.Spec.Replicas) + podSpec := r.createPodSpec(lrest) + + replicaSet := &appsv1.ReplicaSet{ + ObjectMeta: metav1.ObjectMeta{ + Name: lrest.Name + "-lrest-rs", + Namespace: lrest.Namespace, + Labels: map[string]string{ + "name": lrest.Name + "-lrest-rs", + }, + }, + Spec: appsv1.ReplicaSetSpec{ + Replicas: &replicas, + Template: corev1.PodTemplateSpec{ + ObjectMeta: metav1.ObjectMeta{ + Name: lrest.Name + "-lrest", + Namespace: lrest.Namespace, + Labels: map[string]string{ + "name": lrest.Name + "-lrest", + }, + }, + Spec: podSpec, + }, + Selector: &metav1.LabelSelector{ + MatchLabels: map[string]string{ + "name": lrest.Name + "-lrest", + }, + }, + }, + } + + return replicaSet +} + +/* +********************************************************* + - Evaluate change in Spec post creation and instantiation + /******************************************************* +*/ +func (r *LRESTReconciler) deleteReplicaSet(ctx context.Context, req ctrl.Request, lrest *dbapi.LREST) error { + log := r.Log.WithValues("deleteReplicaSet", req.NamespacedName) + + k_client, err := kubernetes.NewForConfig(r.Config) + if err != nil { + log.Error(err, "Kubernetes Config Error") + return err + } + + replicaSetName := lrest.Name + "-lrest-rs" + err = k_client.AppsV1().ReplicaSets(lrest.Namespace).Delete(context.TODO(), replicaSetName, metav1.DeleteOptions{}) + if err != nil { + log.Info("Could not delete ReplicaSet", "RS Name", replicaSetName, "err", err.Error()) + if !strings.Contains(strings.ToUpper(err.Error()), "NOT FOUND") { + return err + } + } else { + log.Info("Successfully deleted LREST ReplicaSet", "RS Name", replicaSetName) + } + + return nil +} + +/* +********************************************************* + - Evaluate change in Spec post creation and instantiation + /******************************************************* +*/ +func (r *LRESTReconciler) evaluateSpecChange(ctx context.Context, req ctrl.Request, lrest *dbapi.LREST) error { + log := r.Log.WithValues("evaluateSpecChange", req.NamespacedName) + + // List the Pods matching the PodTemplate Labels + podName := lrest.Name + "-lrest" + podList := &corev1.PodList{} + listOpts := []client.ListOption{client.InNamespace(req.Namespace), client.MatchingLabels{"name": podName}} + + // List retrieves list of objects for a given namespace and list options. + err := r.List(ctx, podList, listOpts...) + if err != nil { + log.Info("Failed to list pods of: "+podName, "Namespace", req.Namespace) + return err + } + + var foundPod corev1.Pod + for _, pod := range podList.Items { + foundPod = pod + break + } + + lrestSpecChange := false + for _, envVar := range foundPod.Spec.Containers[0].Env { + if envVar.Name == "ORACLE_HOST" && envVar.Value != lrest.Spec.DBServer { + lrestSpecChange = true + } else if envVar.Name == "ORACLE_PORT" && envVar.Value != strconv.Itoa(lrest.Spec.DBPort) { + lrestSpecChange = true + } else if envVar.Name == "LREST_PORT" && envVar.Value != strconv.Itoa(lrest.Spec.LRESTPort) { + lrestSpecChange = true + } else if envVar.Name == "ORACLE_SERVICE" && envVar.Value != lrest.Spec.ServiceName { + lrestSpecChange = true + } + } + + if lrestSpecChange { + // Delete existing ReplicaSet + err = r.deleteReplicaSet(ctx, req, lrest) + if err != nil { + return err + } + + lrest.Status.Phase = lrestPhaseInit + lrest.Status.Status = false + r.Status().Update(ctx, lrest) + } else { + // Update the RS if the value of "replicas" is changed + replicaSetName := lrest.Name + "-lrest-rs" + + foundRS := &appsv1.ReplicaSet{} + err := r.Get(context.TODO(), types.NamespacedName{Name: replicaSetName, Namespace: lrest.Namespace}, foundRS) + if err != nil { + log.Error(err, "Unable to get LREST Replicaset: "+replicaSetName) + return err + } + + // Check if number of replicas have changed + replicas := int32(lrest.Spec.Replicas) + if lrest.Spec.Replicas != int(*(foundRS.Spec.Replicas)) { + log.Info("Existing Replicas: " + strconv.Itoa(int(*(foundRS.Spec.Replicas))) + ", New Replicas: " + strconv.Itoa(lrest.Spec.Replicas)) + foundRS.Spec.Replicas = &replicas + err = r.Update(ctx, foundRS) + if err != nil { + log.Error(err, "Failed to update ReplicaSet for :"+lrest.Name, "Namespace", lrest.Namespace, "Name", replicaSetName) + return err + } + lrest.Status.Phase = lrestPhaseValPod + lrest.Status.Status = false + r.Status().Update(ctx, lrest) + } + } + + return nil +} + +/* +************************************************ + - Create a Cluster Service for LREST LREST Pod + /*********************************************** +*/ +func (r *LRESTReconciler) createLRESTSVC(ctx context.Context, req ctrl.Request, lrest *dbapi.LREST) error { + + log := r.Log.WithValues("createLRESTSVC", req.NamespacedName) + + foundSvc := &corev1.Service{} + err := r.Get(context.TODO(), types.NamespacedName{Name: lrest.Name + "-lrest", Namespace: lrest.Namespace}, foundSvc) + if err != nil && apierrors.IsNotFound(err) { + svc := r.createSvcSpec(lrest) + + log.Info("Creating a new Cluster Service for: "+lrest.Name, "Svc.Namespace", svc.Namespace, "Service.Name", svc.Name) + err := r.Create(ctx, svc) + if err != nil { + log.Error(err, "Failed to create new Cluster Service for: "+lrest.Name, "Svc.Namespace", svc.Namespace, "Service.Name", svc.Name) + return err + } + + log.Info("Created LREST Cluster Service successfully") + r.Recorder.Eventf(lrest, corev1.EventTypeNormal, "CreatedLRESTService", "Created LREST Service for %s", lrest.Name) + } else { + log.Info("LREST Cluster Service already exists") + } + + return nil +} + +/* +*********************** + - Create Service spec + /*********************** +*/ +func (r *LRESTReconciler) createSvcSpec(lrest *dbapi.LREST) *corev1.Service { + + svc := &corev1.Service{ + TypeMeta: metav1.TypeMeta{ + Kind: "Service", + }, + ObjectMeta: metav1.ObjectMeta{ + Name: lrest.Name + "-lrest", + Namespace: lrest.Namespace, + }, + Spec: corev1.ServiceSpec{ + Selector: map[string]string{ + "name": lrest.Name + "-lrest", + }, + ClusterIP: corev1.ClusterIPNone, + }, + } + // Set LREST instance as the owner and controller + ctrl.SetControllerReference(lrest, svc, r.Scheme) + return svc +} + +/* +************************************************ + - Check LREST deletion + /*********************************************** +*/ + +func (r *LRESTReconciler) manageLRESTDeletion(ctx context.Context, req ctrl.Request, lrest *dbapi.LREST) error { + log := r.Log.WithValues("manageLRESTDeletion", req.NamespacedName) + + /* REGISTER FINALIZER */ + if lrest.ObjectMeta.DeletionTimestamp.IsZero() { + if !controllerutil.ContainsFinalizer(lrest, LRESTFinalizer) { + controllerutil.AddFinalizer(lrest, LRESTFinalizer) + if err := r.Update(ctx, lrest); err != nil { + return err + } + } + + } else { + log.Info("lrest set to be deleted") + lrest.Status.Phase = lrestPhaseDelete + lrest.Status.Status = true + r.Status().Update(ctx, lrest) + + if controllerutil.ContainsFinalizer(lrest, LRESTFinalizer) { + + if err := r.DeletePDBS(ctx, req, lrest); err != nil { + log.Info("Cannot delete lrpdbs") + return err + } + + controllerutil.RemoveFinalizer(lrest, LRESTFinalizer) + if err := r.Update(ctx, lrest); err != nil { + return err + } + } + + err := r.deleteLRESTInstance(ctx, req, lrest) + if err != nil { + log.Info("Could not delete LREST Resource", "LREST Name", lrest.Spec.LRESTName, "err", err.Error()) + return err + } + + } + return nil +} + +/* +************************************************ + - Delete LREST Resource + +/*********************************************** +*/ +func (r *LRESTReconciler) deleteLRESTInstance(ctx context.Context, req ctrl.Request, lrest *dbapi.LREST) error { + + log := r.Log.WithValues("deleteLRESTInstance", req.NamespacedName) + + k_client, err := kubernetes.NewForConfig(r.Config) + if err != nil { + log.Error(err, "Kubernetes Config Error") + } + + replicaSetName := lrest.Name + "-lrest-rs" + + err = k_client.AppsV1().ReplicaSets(lrest.Namespace).Delete(context.TODO(), replicaSetName, metav1.DeleteOptions{}) + if err != nil { + log.Info("Could not delete ReplicaSet", "RS Name", replicaSetName, "err", err.Error()) + if !strings.Contains(strings.ToUpper(err.Error()), "NOT FOUND") { + return err + } + } else { + log.Info("Successfully deleted LREST ReplicaSet", "RS Name", replicaSetName) + } + + r.Recorder.Eventf(lrest, corev1.EventTypeNormal, "DeletedLRESTReplicaSet", "Deleted LREST ReplicaSet for %s", lrest.Name) + + svcName := lrest.Name + "-lrest" + + err = k_client.CoreV1().Services(lrest.Namespace).Delete(context.TODO(), svcName, metav1.DeleteOptions{}) + if err != nil { + log.Info("Could not delete Service", "Service Name", svcName, "err", err.Error()) + if !strings.Contains(strings.ToUpper(err.Error()), "NOT FOUND") { + return err + } + } else { + r.Recorder.Eventf(lrest, corev1.EventTypeNormal, "DeletedLRESTService", "Deleted LREST Service for %s", lrest.Name) + log.Info("Successfully deleted LREST Service", "Service Name", svcName) + } + + log.Info("Successfully deleted LREST resource", "LREST Name", lrest.Spec.LRESTName) + return nil +} + +/* +************************************************ + - Get Secret Key for a Secret Name + /*********************************************** +*/ +func (r *LRESTReconciler) verifySecrets(ctx context.Context, req ctrl.Request, lrest *dbapi.LREST) error { + + log := r.Log.WithValues("verifySecrets", req.NamespacedName) + /* + if err := r.checkSecret(ctx, req, lrest, lrest.Spec.SysAdminPwd.Secret.SecretName); err != nil { + return err + }*/ + if err := r.checkSecret(ctx, req, lrest, lrest.Spec.LRESTAdminUser.Secret.SecretName); err != nil { + return err + } + if err := r.checkSecret(ctx, req, lrest, lrest.Spec.LRESTAdminPwd.Secret.SecretName); err != nil { + return err + } + /* + if err := r.checkSecret(ctx, req, lrest, lrest.Spec.LRESTPwd.Secret.SecretName); err != nil { + return err + }*/ + if err := r.checkSecret(ctx, req, lrest, lrest.Spec.WebLrestServerUser.Secret.SecretName); err != nil { + return err + } + if err := r.checkSecret(ctx, req, lrest, lrest.Spec.WebLrestServerPwd.Secret.SecretName); err != nil { + return err + } + + lrest.Status.Msg = "" + log.Info("Verified secrets successfully") + return nil +} + +/* +************************************************ + - Get Secret Key for a Secret Name + /*********************************************** +*/ +func (r *LRESTReconciler) checkSecret(ctx context.Context, req ctrl.Request, lrest *dbapi.LREST, secretName string) error { + + log := r.Log.WithValues("checkSecret", req.NamespacedName) + + secret := &corev1.Secret{} + err := r.Get(ctx, types.NamespacedName{Name: secretName, Namespace: lrest.Namespace}, secret) + if err != nil { + if apierrors.IsNotFound(err) { + log.Info("Secret not found:" + secretName) + lrest.Status.Msg = "Secret not found:" + secretName + return err + } + log.Error(err, "Unable to get the secret.") + return err + } + + return nil +} + +/* +************************************************ + - Delete Secrets + /*********************************************** +*/ +func (r *LRESTReconciler) deleteSecrets(ctx context.Context, req ctrl.Request, lrest *dbapi.LREST) { + + log := r.Log.WithValues("deleteSecrets", req.NamespacedName) + + log.Info("Deleting LREST secrets") + secret := &corev1.Secret{} + /* + err := r.Get(ctx, types.NamespacedName{Name: lrest.Spec.SysAdminPwd.Secret.SecretName, Namespace: lrest.Namespace}, secret) + if err == nil { + err := r.Delete(ctx, secret) + if err == nil { + log.Info("Deleted the secret : " + lrest.Spec.SysAdminPwd.Secret.SecretName) + } + } + */ + + err := r.Get(ctx, types.NamespacedName{Name: lrest.Spec.LRESTAdminUser.Secret.SecretName, Namespace: lrest.Namespace}, secret) + if err == nil { + err := r.Delete(ctx, secret) + if err == nil { + log.Info("Deleted the secret : " + lrest.Spec.LRESTAdminUser.Secret.SecretName) + } + } + + err = r.Get(ctx, types.NamespacedName{Name: lrest.Spec.LRESTAdminPwd.Secret.SecretName, Namespace: lrest.Namespace}, secret) + if err == nil { + err := r.Delete(ctx, secret) + if err == nil { + log.Info("Deleted the secret : " + lrest.Spec.LRESTAdminPwd.Secret.SecretName) + } + } + /* + err = r.Get(ctx, types.NamespacedName{Name: lrest.Spec.LRESTPwd.Secret.SecretName, Namespace: lrest.Namespace}, secret) + if err == nil { + err := r.Delete(ctx, secret) + if err == nil { + log.Info("Deleted the secret : " + lrest.Spec.LRESTPwd.Secret.SecretName) + } + } + */ + + err = r.Get(ctx, types.NamespacedName{Name: lrest.Spec.WebLrestServerUser.Secret.SecretName, Namespace: lrest.Namespace}, secret) + if err == nil { + err := r.Delete(ctx, secret) + if err == nil { + log.Info("Deleted the secret : " + lrest.Spec.WebLrestServerUser.Secret.SecretName) + } + } + + err = r.Get(ctx, types.NamespacedName{Name: lrest.Spec.WebLrestServerPwd.Secret.SecretName, Namespace: lrest.Namespace}, secret) + if err == nil { + err := r.Delete(ctx, secret) + if err == nil { + log.Info("Deleted the secret : " + lrest.Spec.WebLrestServerPwd.Secret.SecretName) + } + } +} + +/* +************************************************************* + - SetupWithManager sets up the controller with the Manager. + /************************************************************ +*/ +func (r *LRESTReconciler) SetupWithManager(mgr ctrl.Manager) error { + return ctrl.NewControllerManagedBy(mgr). + For(&dbapi.LREST{}). + Owns(&appsv1.ReplicaSet{}). //Watch for deleted RS owned by this controller + WithEventFilter(predicate.Funcs{ + UpdateFunc: func(e event.UpdateEvent) bool { + // Ignore updates to CR status in which case metadata.Generation does not change + return e.ObjectOld.GetGeneration() != e.ObjectNew.GetGeneration() + }, + DeleteFunc: func(e event.DeleteEvent) bool { + // Evaluates to false if the object has been confirmed deleted. + //return !e.DeleteStateUnknown + return false + }, + }). + WithOptions(controller.Options{MaxConcurrentReconciles: 100}). + Complete(r) +} + +func securityContextDefineLrest() *corev1.SecurityContext { + return &corev1.SecurityContext{ + RunAsNonRoot: &[]bool{true}[0], + RunAsUser: &[]int64{54321}[0], + AllowPrivilegeEscalation: &[]bool{false}[0], + Capabilities: &corev1.Capabilities{ + Drop: []corev1.Capability{ + "ALL", + }, + }, + } +} + +func (r *LRESTReconciler) DeletePDBS(ctx context.Context, req ctrl.Request, lrest *dbapi.LREST) error { + log := r.Log.WithValues("DeletePDBS", req.NamespacedName) + + /* =================== DELETE CASCADE ================ */ + if lrest.Spec.DeletePDBCascade == true { + log.Info("DELETE PDB CASCADE OPTION") + lrpdbList := &dbapi.LRPDBList{} + listOpts := []client.ListOption{} + err := r.List(ctx, lrpdbList, listOpts...) + if err != nil { + log.Info("Failed to get the list of pdbs") + } + + if err == nil { + for _, pdbitem := range lrpdbList.Items { + log.Info("pdbitem.Spec.CDBName:" + pdbitem.Spec.CDBName) + log.Info("lrest.Spec.LRESTName:" + lrest.Spec.LRESTName) + if pdbitem.Spec.CDBName == lrest.Spec.LRESTName { + fmt.Printf("DEVPHASE: Call Delete function for %s %s\n", pdbitem.Name, pdbitem.Spec.LRPDBName) + + var objmap map[string]interface{} /* Used for the return payload */ + values := map[string]string{ + "state": "CLOSE", + "modifyOption": "ABORT", + } + + url := "https://" + pdbitem.Spec.CDBResName + "-lrest." + pdbitem.Spec.CDBNamespace + ":" + strconv.Itoa(lrest.Spec.LRESTPort) + "/database/pdbs/" + pdbitem.Spec.LRPDBName + + log.Info("callAPI(URL):" + url) + log.Info("pdbitem.Status.OpenMode" + pdbitem.Status.OpenMode) + + if pdbitem.Status.OpenMode != "MOUNTED" { + + log.Info("Force pdb closure") + respData, errapi := NewCallLAPI(r, ctx, req, &pdbitem, url, values, "POST") + + if err := json.Unmarshal([]byte(respData), &objmap); err != nil { + log.Error(err, "failed to get respData from callAPI", "err", err.Error()) + return err + } + + pdbitem.Status.SqlCode = int(objmap["sqlcode"].(float64)) + log.Info("pdb closure.......:", "sqlcode", pdbitem.Status.SqlCode) + + if errapi != nil { + log.Error(err, "callAPI cannot close pdb "+pdbitem.Spec.LRPDBName, "err", err.Error()) + return err + } + + r.Recorder.Eventf(lrest, corev1.EventTypeNormal, "close pdb", "pdbname=%s", pdbitem.Spec.LRPDBName) + } + + /* start dropping pdb */ + log.Info("Drop pluggable database") + values = map[string]string{ + "action": "INCLUDING", + } + respData, errapi := NewCallLAPI(r, ctx, req, &pdbitem, url, values, "DELETE") + + if err := json.Unmarshal([]byte(respData), &objmap); err != nil { + log.Error(err, "failed to get respData from callAPI", "err", err.Error()) + return err + } + + pdbitem.Status.SqlCode = int(objmap["sqlcode"].(float64)) + log.Info(".......:", "sqlcode", pdbitem.Status.SqlCode) + + if errapi != nil { + log.Error(err, "callAPI cannot drop pdb "+pdbitem.Spec.LRPDBName, "err", err.Error()) + return err + } + r.Recorder.Eventf(lrest, corev1.EventTypeNormal, "drop pdb", "pdbname=%s", pdbitem.Spec.LRPDBName) + + /* remove finalizer */ + + if controllerutil.ContainsFinalizer(&pdbitem, LRPDBFinalizer) { + log.Info("Removing finalizer") + controllerutil.RemoveFinalizer(&pdbitem, LRPDBFinalizer) + err = r.Update(ctx, &pdbitem) + if err != nil { + log.Info("Could not remove finalizer", "err", err.Error()) + return err + } + } + + err = r.Delete(context.Background(), &pdbitem, client.GracePeriodSeconds(1)) + if err != nil { + log.Info("Could not delete LRPDB resource", "err", err.Error()) + return err + } + + } /* check pdb name */ + } /* end of loop */ + } + + } + /* ================================================ */ + return nil +} diff --git a/controllers/database/lrpdb_controller.go b/controllers/database/lrpdb_controller.go new file mode 100644 index 00000000..1aadf65b --- /dev/null +++ b/controllers/database/lrpdb_controller.go @@ -0,0 +1,2381 @@ +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package controllers + +import ( + "bytes" + "context" + "crypto/rsa" + "crypto/tls" + "crypto/x509" + "encoding/base64" + "encoding/json" + "encoding/pem" + + //"encoding/pem" + "errors" + "fmt" + "io/ioutil" + "net/http" + "regexp" + "strconv" + "strings" + "time" + + dbapi "github.com/oracle/oracle-database-operator/apis/database/v4" + "github.com/oracle/oracle-database-operator/commons/k8s" + lrcommons "github.com/oracle/oracle-database-operator/commons/multitenant/lrest" + + "github.com/go-logr/logr" + corev1 "k8s.io/api/core/v1" + apierrors "k8s.io/apimachinery/pkg/api/errors" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/util/validation/field" + + //metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/types" + "k8s.io/client-go/tools/record" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/controller" + "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" + "sigs.k8s.io/controller-runtime/pkg/event" + "sigs.k8s.io/controller-runtime/pkg/predicate" +) + +// Bitmask functions +const ( + MPAPPL = 0x00000001 /* The map config has been applyed */ + MPSYNC = 0x00000002 /* The map config is in sync with v$parameters where is default=flase */ + MPEMPT = 0x00000004 /* The map is empty - not specify */ + MPWARN = 0x00000008 /* Map applied with warnings */ + MPINIT = 0x00000010 /* Config map init */ + SPARE3 = 0x00000020 +) + +func bis(bitmask int, bitval int) int { + bitmask = ((bitmask) | (bitval)) + return bitmask +} + +func bit(bitmask int, bitval int) bool { + if bitmask&bitval != 0 { + return true + } else { + return false + } +} + +func bid(bitmask int, bitval int) int { + bitmask ^= ((bitval) & (bitmask)) + return bitmask +} + +func bitmaskprint(bitmask int) string { + BitRead := "|" + if bit(bitmask, MPAPPL) { + BitRead = strings.Join([]string{BitRead, "MPAPPL|"}, "") + } + if bit(bitmask, MPSYNC) { + BitRead = strings.Join([]string{BitRead, "MPSYNC|"}, "") + } + if bit(bitmask, MPEMPT) { + BitRead = strings.Join([]string{BitRead, "MPEMPT|"}, "") + } + if bit(bitmask, MPWARN) { + BitRead = strings.Join([]string{BitRead, "MPWARN|"}, "") + } + if bit(bitmask, MPINIT) { + BitRead = strings.Join([]string{BitRead, "MPINIT|"}, "") + } + if bit(bitmask, SPARE3) { + BitRead = strings.Join([]string{BitRead, "SPARE3|"}, "") + } + + return BitRead +} + +// LRPDBReconciler reconciles a LRPDB object +type LRPDBReconciler struct { + client.Client + Log logr.Logger + Scheme *runtime.Scheme + Interval time.Duration + Recorder record.EventRecorder +} + +type restSQLCollection struct { + Env struct { + DefaultTimeZone string `json:"defaultTimeZone,omitempty"` + } `json:"env"` + Items []SQL_Item `json:"items"` +} + +type SQL_Item struct { + StatementId int `json:"statementId,omitempty"` + Response []string `json:"response"` + ErrorCode int `json:"errorCode,omitempty"` + ErrorLine int `json:"errorLine,omitempty"` + ErrorColumn int `json:"errorColumn,omitempty"` + ErrorDetails string `json:"errorDetails,omitempty"` + Result int `json:"result,omitempty"` +} + +type LRESTError struct { + Code string `json:"code,omitempty"` + Message string `json:"message,omitempty"` + Type string `json:"type,omitempty"` + Instance string `json:"instance,omitempty"` +} + +var ( + lrpdbPhaseCreate = "Creating" + lrpdbPhasePlug = "Plugging" + lrpdbPhaseUnplug = "Unplugging" + lrpdbPhaseClone = "Cloning" + lrpdbPhaseFinish = "Finishing" + lrpdbPhaseReady = "Ready" + lrpdbPhaseDelete = "Deleting" + lrpdbPhaseModify = "Modifying" + lrpdbPhaseMap = "Mapping" + lrpdbPhaseStatus = "CheckingState" + lrpdbPhaseFail = "Failed" + lrpdbPhaseAlterPlug = "AlterPlugDb" + lrpdbPhaseSpare = "NoAction" +) + +const LRPDBFinalizer = "database.oracle.com/LRPDBfinalizer" + +var tde_Password string +var tde_Secret string +var flood_control bool = false +var assertiveLpdbDeletion bool = false /* Global variable for assertive pdb deletion */ +/* + We need to record the config map name after pdb creation + in order to use it during open and clone op if config map + name is not set the open and clone yaml file +*/ +var globalconfigmap string +var globalsqlcode int + +/* mind https://github.com/kubernetes-sigs/kubebuilder/issues/549 */ +//+kubebuilder:rbac:groups=database.oracle.com,resources=lrpdbs,verbs=get;list;watch;create;update;patch;delete +//+kubebuilder:rbac:groups=database.oracle.com,resources=events,verbs=get;list;watch;create;update;patch;delete +//+kubebuilder:rbac:groups=database.oracle.com,resources=lrpdbs/status,verbs=get;update;patch +//+kubebuilder:rbac:groups=database.oracle.com,resources=lrpdbs/finalizers,verbs=get;create;update;patch;delete + +// Reconcile is part of the main kubernetes reconciliation loop which aims to +// move the current state of the cluster closer to the desired state. +// TODO(user): Modify the Reconcile function to compare the state specified by +// the LRPDB object against the actual cluster state, and then +// perform operations to make the cluster state reflect the state specified by +// the user. +// +// For more details, check Reconcile and its Result here: +// - https://pkg.go.dev/sigs.k8s.io/controller-runtime@v0.9.2/pkg/reconcile +func (r *LRPDBReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + log := r.Log.WithValues("multitenantoperator", req.NamespacedName) + log.Info("Reconcile requested") + + reconcilePeriod := r.Interval * time.Second + requeueY := ctrl.Result{Requeue: true, RequeueAfter: reconcilePeriod} + requeueN := ctrl.Result{} + + var err error + lrpdb := &dbapi.LRPDB{} + + // Execute for every reconcile + defer func() { + //log.Info("DEFER LRPDB", "Name", lrpdb.Name, "Phase", lrpdb.Status.Phase, "Status", strconv.FormatBool(lrpdb.Status.Status)) + if !lrpdb.Status.Status { + if lrpdb.Status.Phase == lrpdbPhaseReady { + lrpdb.Status.Status = true + } + if err := r.Status().Update(ctx, lrpdb); err != nil { + log.Error(err, "Failed to update status for :"+lrpdb.Name, "err", err.Error()) + } + } + }() + + err = r.Client.Get(context.TODO(), req.NamespacedName, lrpdb) + if err != nil { + if apierrors.IsNotFound(err) { + log.Info("LRPDB Resource Not found", "Name", lrpdb.Name) + // Request object not found, could have been deleted after reconcile req. + // Owned objects are automatically garbage collected. For additional cleanup logic use finalizers. + // Return and don't requeue + lrpdb.Status.Status = true + return requeueN, nil + } + // Error reading the object - requeue the req. + return requeueY, err + } + + // Finalizer section + err = r.manageLRPDBDeletion2(ctx, req, lrpdb) + if err != nil { + log.Info("Reconcile queued") + return requeueY, nil + } + + // Check for Duplicate LRPDB + if !lrpdb.Status.Status { + err = r.checkDuplicateLRPDB(ctx, req, lrpdb) + if err != nil { + return requeueN, nil + } + } + + action := strings.ToUpper(lrpdb.Spec.Action) + /* + Bug 36714702 - LREST OPERATOR - POST ALTER PDB OPTION LRPDB STATUS INTERMITTENTLY + SHOWS "WAITING FOR LRPDB PARAMETER TO BE MODIFIED" + introducing additional check to avoid alter system repetition during + reconciliation loop + */ + if lrpdb.Status.Phase == lrpdbPhaseReady { + if (lrpdb.Status.Action != "" || action != "NOACTION") && (action == "ALTER" || action == "MODIFY" || action == "STATUS" || lrpdb.Status.Action != action) { + lrpdb.Status.Status = false + } else { + err = r.getLRPDBState(ctx, req, lrpdb) + if err != nil { + lrpdb.Status.Phase = lrpdbPhaseFail + } else { + lrpdb.Status.Phase = lrpdbPhaseReady + lrpdb.Status.Msg = "Success" + } + r.Status().Update(ctx, lrpdb) + } + } + + if !lrpdb.Status.Status { + r.validatePhase(ctx, req, lrpdb) + phase := lrpdb.Status.Phase + log.Info("LRPDB:", "Name", lrpdb.Name, "Phase", phase, "Status", strconv.FormatBool(lrpdb.Status.Status)) + + switch phase { + case lrpdbPhaseCreate: + err = r.createLRPDB(ctx, req, lrpdb) + case lrpdbPhaseClone: + err = r.cloneLRPDB(ctx, req, lrpdb) + case lrpdbPhasePlug: + err = r.plugLRPDB(ctx, req, lrpdb) + case lrpdbPhaseUnplug: + err = r.unplugLRPDB(ctx, req, lrpdb) + case lrpdbPhaseModify: + err = r.modifyLRPDB(ctx, req, lrpdb) + case lrpdbPhaseDelete: + err = r.deleteLRPDB(ctx, req, lrpdb) + case lrpdbPhaseStatus: + err = r.getLRPDBState(ctx, req, lrpdb) + case lrpdbPhaseMap: + err = r.mapLRPDB(ctx, req, lrpdb) + case lrpdbPhaseFail: + err = r.mapLRPDB(ctx, req, lrpdb) + case lrpdbPhaseAlterPlug: + err = r.alterSystemLRPDB(ctx, req, lrpdb) + default: + log.Info("DEFAULT:", "Name", lrpdb.Name, "Phase", phase, "Status", strconv.FormatBool(lrpdb.Status.Status)) + return requeueN, nil + } + lrpdb.Status.Action = strings.ToUpper(lrpdb.Spec.Action) + if err != nil { + lrpdb.Status.Phase = lrpdbPhaseFail + lrpdb.Status.SqlCode = globalsqlcode + } else { + lrpdb.Status.Phase = lrpdbPhaseReady + lrpdb.Status.Msg = "Success" + } + } + + r.ManageConfigMapForCloningAndPlugin(ctx, req, lrpdb) + lrpdb.Status.BitStatStr = bitmaskprint(lrpdb.Status.Bitstat) + + log.Info("Reconcile completed") + return requeueY, nil +} + +/* +************************************************ + - Validate the LRPDB Spec + /*********************************************** +*/ +func (r *LRPDBReconciler) validatePhase(ctx context.Context, req ctrl.Request, lrpdb *dbapi.LRPDB) { + + log := r.Log.WithValues("validatePhase", req.NamespacedName) + + action := strings.ToUpper(lrpdb.Spec.Action) + + log.Info("Validating LRPDB phase for: "+lrpdb.Name, "Action", action) + + switch action { + case "CREATE": + lrpdb.Status.Phase = lrpdbPhaseCreate + case "CLONE": + lrpdb.Status.Phase = lrpdbPhaseClone + case "PLUG": + lrpdb.Status.Phase = lrpdbPhasePlug + case "UNPLUG": + lrpdb.Status.Phase = lrpdbPhaseUnplug + case "MODIFY": + lrpdb.Status.Phase = lrpdbPhaseModify + case "DELETE": + lrpdb.Status.Phase = lrpdbPhaseDelete + case "STATUS": + lrpdb.Status.Phase = lrpdbPhaseStatus + case "MAP": + lrpdb.Status.Phase = lrpdbPhaseMap + case "ALTER": + lrpdb.Status.Phase = lrpdbPhaseAlterPlug + case "NOACTION": + lrpdb.Status.Phase = lrpdbPhaseStatus + + } + + log.Info("Validation complete") +} + +/* + This function scans the list of crd + pdb to verify the existence of the + pdb (crd) that we want to clone. + Bug 36752925 - LREST OPERATOR - CLONE NON-EXISTENT + PDB CREATES A LRPDB WITH STATUS FAILED + + return 1 - CRD found + return 0 - CRD not found / Stop clone process + + Bug 36753107 - LREST OPERATOR - CLONE + CLOSED PDB SUCCESSFULLY CLONES + +*/ + +func (r *LRPDBReconciler) checkPDBforCloninig(ctx context.Context, req ctrl.Request, lrpdb *dbapi.LRPDB, targetPdbName string) (int, error) { + log := r.Log.WithValues("checkDuplicateLRPDB", req.NamespacedName) + var pdbCounter int + pdbCounter = 0 + + lrpdbList := &dbapi.LRPDBList{} + listOpts := []client.ListOption{client.InNamespace(req.Namespace), client.MatchingFields{"spec.pdbName": targetPdbName}} + err := r.List(ctx, lrpdbList, listOpts...) + if err != nil { + log.Info("Failed to list lrpdbs", "Namespace", req.Namespace, "Error", err) + return 0, err + } + if len(lrpdbList.Items) == 0 { + log.Info("No pdbs available") + return pdbCounter, err + } + + for _, p := range lrpdbList.Items { + fmt.Printf("DEBUGCLONE %s %s %i\n", p.Spec.LRPDBName, targetPdbName, pdbCounter) + if p.Spec.LRPDBName == targetPdbName { + log.Info("Found " + targetPdbName + " in the crd list") + if p.Status.OpenMode == "MOUNTED" { + log.Info("Cannot clone a mounted pdb") + return pdbCounter, err + } + pdbCounter++ + fmt.Printf("DEBUGCLONE %s %s %i\n", p.Spec.LRPDBName, targetPdbName, pdbCounter) + return pdbCounter, err + } + + } + return pdbCounter, err +} + +/* +*************************************************************** + - Check for Duplicate LRPDB. Same LRPDB name on the same LREST resource. + +/************************************************************** +*/ +func (r *LRPDBReconciler) checkDuplicateLRPDB(ctx context.Context, req ctrl.Request, lrpdb *dbapi.LRPDB) error { + + log := r.Log.WithValues("checkDuplicateLRPDB", req.NamespacedName) + + // Name of the LREST CR that holds the LREST container + lrestResName := lrpdb.Spec.CDBResName + //lrestame := lrpdb.Spec.LRESTName + + // Name of the LRPDB resource + lrpdbResName := lrpdb.Spec.LRPDBName + + lrpdbList := &dbapi.LRPDBList{} + + listOpts := []client.ListOption{client.InNamespace(req.Namespace), client.MatchingFields{"spec.pdbName": lrpdbResName}} + + // List retrieves list of objects for a given namespace and list options. + err := r.List(ctx, lrpdbList, listOpts...) + if err != nil { + log.Info("Failed to list lrpdbs", "Namespace", req.Namespace, "Error", err) + return err + } + + if len(lrpdbList.Items) == 0 { + log.Info("No lrpdbs found for LRPDBName: "+lrpdbResName, "CDBResName", lrestResName) + return nil + } + + for _, p := range lrpdbList.Items { + log.Info("Found LRPDB: " + p.Name) + if (p.Name != lrpdb.Name) && (p.Spec.CDBResName == lrestResName) { + log.Info("Duplicate LRPDB found") + lrpdb.Status.Msg = "LRPDB Resource already exists" + lrpdb.Status.Status = false + lrpdb.Status.Phase = lrpdbPhaseFail + return errors.New("Duplicate LRPDB found") + } + } + return nil +} + +/* +*************************************************************** + - Get the Custom Resource for the LREST mentioned in the LRPDB Spec + /************************************************************** +*/ +func (r *LRPDBReconciler) getLRESTResource(ctx context.Context, req ctrl.Request, lrpdb *dbapi.LRPDB) (dbapi.LREST, error) { + + log := r.Log.WithValues("getLRESTResource", req.NamespacedName) + + var lrest dbapi.LREST // LREST CR corresponding to the LREST name specified in the LRPDB spec + + // Name of the LREST CR that holds the LREST container + lrestResName := lrpdb.Spec.CDBResName + lrestNamespace := lrpdb.Spec.CDBNamespace + + log.Info("lrestResName...........:" + lrestResName) + log.Info("lrestNamespace.........:" + lrestNamespace) + + // Get LREST CR corresponding to the LREST name specified in the LRPDB spec + err := r.Get(context.Background(), client.ObjectKey{ + Namespace: lrestNamespace, + Name: lrestResName, + }, &lrest) + + if err != nil { + log.Info("Failed to get CRD for LREST", "Name", lrestResName, "Namespace", lrestNamespace, "Error", err.Error()) + lrpdb.Status.Msg = "Unable to get CRD for LREST : " + lrestResName + r.Status().Update(ctx, lrpdb) + return lrest, err + } + + log.Info("Found CR for LREST", "Name", lrestResName, "CR Name", lrest.Name) + return lrest, nil +} + +/* +*************************************************************** + - Get the LREST Pod for the LREST mentioned in the LRPDB Spec + /************************************************************** +*/ +func (r *LRPDBReconciler) getLRESTPod(ctx context.Context, req ctrl.Request, lrpdb *dbapi.LRPDB) (corev1.Pod, error) { + + log := r.Log.WithValues("getLRESTPod", req.NamespacedName) + + var lrestPod corev1.Pod // LREST Pod container with connection to the concerned LREST + + // Name of the LREST CR that holds the LREST container + lrestResName := lrpdb.Spec.CDBResName + + // Get LREST Pod associated with the LREST Name specified in the LRPDB Spec + err := r.Get(context.Background(), client.ObjectKey{ + Namespace: req.Namespace, + Name: lrestResName + "-lrest", + }, &lrestPod) + + if err != nil { + log.Info("Failed to get Pod for LREST", "Name", lrestResName, "Namespace", req.Namespace, "Error", err.Error()) + lrpdb.Status.Msg = "Unable to get LREST Pod for LREST : " + lrestResName + return lrestPod, err + } + + log.Info("Found LREST Pod for LREST", "Name", lrestResName, "Pod Name", lrestPod.Name, "LREST Container hostname", lrestPod.Spec.Hostname) + return lrestPod, nil +} + +/* +************************************************ + - Get Secret Key for a Secret Name + /*********************************************** +*/ +func (r *LRPDBReconciler) getSecret(ctx context.Context, req ctrl.Request, lrpdb *dbapi.LRPDB, secretName string, keyName string) (string, error) { + + log := r.Log.WithValues("getSecret", req.NamespacedName) + + secret := &corev1.Secret{} + err := r.Get(ctx, types.NamespacedName{Name: secretName, Namespace: lrpdb.Namespace}, secret) + if err != nil { + if apierrors.IsNotFound(err) { + log.Info("Secret not found:" + secretName) + lrpdb.Status.Msg = "Secret not found:" + secretName + return "", err + } + log.Error(err, "Unable to get the secret.") + return "", err + } + + return string(secret.Data[keyName]), nil +} + +/* +************************************************ + - Issue a REST API Call to the LREST container + /*********************************************** +*/ +func (r *LRPDBReconciler) callAPI(ctx context.Context, req ctrl.Request, lrpdb *dbapi.LRPDB, url string, payload map[string]string, action string) (string, error) { + log := r.Log.WithValues("callAPI", req.NamespacedName) + + var err error + + secret := &corev1.Secret{} + + err = r.Get(ctx, types.NamespacedName{Name: lrpdb.Spec.LRPDBTlsKey.Secret.SecretName, Namespace: lrpdb.Namespace}, secret) + + if err != nil { + if apierrors.IsNotFound(err) { + log.Info("Secret not found:" + lrpdb.Spec.LRPDBTlsKey.Secret.SecretName) + return "", err + } + log.Error(err, "Unable to get the secret.") + return "", err + } + rsaKeyPEM := secret.Data[lrpdb.Spec.LRPDBTlsKey.Secret.Key] + + err = r.Get(ctx, types.NamespacedName{Name: lrpdb.Spec.LRPDBTlsCrt.Secret.SecretName, Namespace: lrpdb.Namespace}, secret) + + if err != nil { + if apierrors.IsNotFound(err) { + log.Info("Secret not found:" + lrpdb.Spec.LRPDBTlsCrt.Secret.SecretName) + return "", err + } + log.Error(err, "Unable to get the secret.") + return "", err + } + + rsaCertPEM := secret.Data[lrpdb.Spec.LRPDBTlsCrt.Secret.Key] + + err = r.Get(ctx, types.NamespacedName{Name: lrpdb.Spec.LRPDBTlsCat.Secret.SecretName, Namespace: lrpdb.Namespace}, secret) + + if err != nil { + if apierrors.IsNotFound(err) { + log.Info("Secret not found:" + lrpdb.Spec.LRPDBTlsCat.Secret.SecretName) + return "", err + } + log.Error(err, "Unable to get the secret.") + return "", err + } + + caCert := secret.Data[lrpdb.Spec.LRPDBTlsCat.Secret.Key] + /* + r.Recorder.Eventf(lrpdb, corev1.EventTypeWarning, "LRESTINFO", string(rsaKeyPEM)) + r.Recorder.Eventf(lrpdb, corev1.EventTypeWarning, "LRESTINFO", string(rsaCertPEM)) + r.Recorder.Eventf(lrpdb, corev1.EventTypeWarning, "LRESTINFO", string(caCert)) + */ + + certificate, err := tls.X509KeyPair([]byte(rsaCertPEM), []byte(rsaKeyPEM)) + if err != nil { + lrpdb.Status.Msg = "Error tls.X509KeyPair" + return "", err + } + + caCertPool := x509.NewCertPool() + caCertPool.AppendCertsFromPEM(caCert) + /* + tlsConf := &tls.Config{Certificates: []tls.Certificate{certificate}, + RootCAs: caCertPool} + */ + tlsConf := &tls.Config{Certificates: []tls.Certificate{certificate}, + RootCAs: caCertPool, + //MinVersion: tls.VersionTLS12, + CurvePreferences: []tls.CurveID{tls.CurveP521, tls.CurveP384, tls.CurveP256}, + PreferServerCipherSuites: true, + CipherSuites: []uint16{ + tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, + tls.TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, + tls.TLS_RSA_WITH_AES_256_GCM_SHA384, + tls.TLS_RSA_WITH_AES_256_CBC_SHA, + }, + } + + tr := &http.Transport{TLSClientConfig: tlsConf} + + httpclient := &http.Client{Transport: tr} + + log.Info("Issuing REST call", "URL", url, "Action", action) + + webUser, err := r.getEncriptedSecret(ctx, req, lrpdb, lrpdb.Spec.WebLrpdbServerUser.Secret.SecretName, lrpdb.Spec.WebLrpdbServerUser.Secret.Key, lrpdb.Spec.LRPDBPriKey.Secret.SecretName, lrpdb.Spec.LRPDBPriKey.Secret.Key) + if err != nil { + log.Error(err, "Unable to get webuser account name ") + return "", err + } + + webUserPwd, err := r.getEncriptedSecret(ctx, req, lrpdb, lrpdb.Spec.WebLrpdbServerPwd.Secret.SecretName, lrpdb.Spec.WebLrpdbServerPwd.Secret.Key, lrpdb.Spec.LRPDBPriKey.Secret.SecretName, lrpdb.Spec.LRPDBPriKey.Secret.Key) + if err != nil { + log.Error(err, "Unable to get webuser account password ") + return "", err + } + + var httpreq *http.Request + if action == "GET" { + httpreq, err = http.NewRequest(action, url, nil) + } else { + jsonValue, _ := json.Marshal(payload) + httpreq, err = http.NewRequest(action, url, bytes.NewBuffer(jsonValue)) + } + + if err != nil { + log.Info("Unable to create HTTP Request for LRPDB : "+lrpdb.Name, "err", err.Error()) + return "", err + } + + httpreq.Header.Add("Accept", "application/json") + httpreq.Header.Add("Content-Type", "application/json") + httpreq.SetBasicAuth(webUser, webUserPwd) + + resp, err := httpclient.Do(httpreq) + if err != nil { + errmsg := err.Error() + log.Error(err, "Failed - Could not connect to LREST Pod", "err", err.Error()) + lrpdb.Status.Msg = "Error: Could not connect to LREST Pod" + r.Recorder.Eventf(lrpdb, corev1.EventTypeWarning, "LRESTError", errmsg) + return "", err + } + + r.Recorder.Eventf(lrpdb, corev1.EventTypeWarning, "Done", lrpdb.Spec.CDBResName) + if resp.StatusCode != http.StatusOK { + bb, _ := ioutil.ReadAll(resp.Body) + + if resp.StatusCode == 404 { + lrpdb.Status.ConnString = "" + lrpdb.Status.Msg = lrpdb.Spec.LRPDBName + " not found" + + } else { + if flood_control == false { + lrpdb.Status.Msg = "LREST Error - HTTP Status Code:" + strconv.Itoa(resp.StatusCode) + } + } + + if flood_control == false { + log.Info("LREST Error - HTTP Status Code :"+strconv.Itoa(resp.StatusCode), "Err", string(bb)) + } + + var apiErr LRESTError + json.Unmarshal([]byte(bb), &apiErr) + if flood_control == false { + r.Recorder.Eventf(lrpdb, corev1.EventTypeWarning, "LRESTError", "Failed: %s", apiErr.Message) + } + fmt.Printf("\n================== APIERR ======================\n") + fmt.Printf("%+v \n", apiErr) + fmt.Printf(string(bb)) + fmt.Printf("URL=%s\n", url) + fmt.Printf("resp.StatusCode=%s\n", strconv.Itoa(resp.StatusCode)) + fmt.Printf("\n================== APIERR ======================\n") + flood_control = true + return "", errors.New("LREST Error") + } + flood_control = false + + defer resp.Body.Close() + + bodyBytes, err := ioutil.ReadAll(resp.Body) + if err != nil { + fmt.Print(err.Error()) + } + respData := string(bodyBytes) + fmt.Print("CALL API return msg.....:") + fmt.Println(string(bodyBytes)) + + var apiResponse restSQLCollection + json.Unmarshal([]byte(bodyBytes), &apiResponse) + fmt.Printf("===> %#v\n", apiResponse) + fmt.Printf("===> %+v\n", apiResponse) + + errFound := false + for _, sqlItem := range apiResponse.Items { + if sqlItem.ErrorDetails != "" { + log.Info("LREST Error - Oracle Error Code :" + strconv.Itoa(sqlItem.ErrorCode)) + if !errFound { + lrpdb.Status.Msg = sqlItem.ErrorDetails + } + r.Recorder.Eventf(lrpdb, corev1.EventTypeWarning, "OraError", "%s", sqlItem.ErrorDetails) + errFound = true + } + } + + if errFound { + return "", errors.New("Oracle Error") + } + + return respData, nil +} + +/* +************************************************ + - Create a LRPDB + +*********************************************** +*/ +func (r *LRPDBReconciler) createLRPDB(ctx context.Context, req ctrl.Request, lrpdb *dbapi.LRPDB) error { + + log := r.Log.WithValues("createLRPDB", req.NamespacedName) + + var err error + var tde_Password string + var tde_Secret string + + log.Info("call getLRESTResource \n") + lrest, err := r.getLRESTResource(ctx, req, lrpdb) + if err != nil { + return err + } + + lrpdbAdminName, err := r.getEncriptedSecret(ctx, req, lrpdb, lrpdb.Spec.AdminpdbUser.Secret.SecretName, lrpdb.Spec.AdminpdbUser.Secret.Key, lrpdb.Spec.LRPDBPriKey.Secret.SecretName, lrpdb.Spec.LRPDBPriKey.Secret.Key) + if err != nil { + log.Error(err, "Unable to find pdb admin user ") + return err + } + + lrpdbAdminPwd, err := r.getEncriptedSecret(ctx, req, lrpdb, lrpdb.Spec.AdminpdbPass.Secret.SecretName, lrpdb.Spec.AdminpdbPass.Secret.Key, lrpdb.Spec.LRPDBPriKey.Secret.SecretName, lrpdb.Spec.LRPDBPriKey.Secret.Key) + + if err != nil { + log.Error(err, "Unable to find pdb admin password ") + return err + } + + err = r.getLRPDBState(ctx, req, lrpdb) + if err != nil { + if apierrors.IsNotFound(err) { + log.Info("Check LRPDB not existence completed", "LRPDB Name", lrpdb.Spec.LRPDBName) + } + + } else { + + lrpdb.Status.Phase = lrpdbPhaseFail + lrpdb.Status.Msg = "PDB " + lrpdb.Spec.LRPDBName + " already exists " + if err := r.Status().Update(ctx, lrpdb); err != nil { + log.Error(err, "Failed to update status for :"+lrpdb.Name, "err", err.Error()) + } + log.Info("Database already exists ", "LRPDB Name", lrpdb.Spec.LRPDBName) + err := fmt.Errorf("%v", 65012) + return err + } + + values := map[string]string{ + "method": "CREATE", + "pdb_name": lrpdb.Spec.LRPDBName, + "adminName": lrpdbAdminName, + "adminPwd": lrpdbAdminPwd, + "fileNameConversions": lrpdb.Spec.FileNameConversions, + "reuseTempFile": strconv.FormatBool(*(lrpdb.Spec.ReuseTempFile)), + "unlimitedStorage": strconv.FormatBool(*(lrpdb.Spec.UnlimitedStorage)), + "totalSize": lrpdb.Spec.TotalSize, + "tempSize": lrpdb.Spec.TempSize, + "getScript": strconv.FormatBool(*(lrpdb.Spec.GetScript))} + + fmt.Printf("===== PAYLOAD ===\n") + fmt.Print(" method ", values["method"], "\n") + fmt.Print(" pdb_name ", values["pdb_name"], "\n") + fmt.Print(" adminName ", values["adminName"], "\n") + fmt.Print(" adminPwd --------------\n") + fmt.Print(" fileNameConversions ", values["fileNameConversions"], "\n") + fmt.Print(" unlimitedStorage ", values["unlimitedStorage"], "\n") + fmt.Print(" reuseTempFile ", values["reuseTempFile"], "\n") + fmt.Print(" tempSize ", values["tempSize"], "\n") + fmt.Print(" totalSize ", values["totalSize"], "\n") + fmt.Print(" getScript ", values["getScript"], "\n") + + if *(lrpdb.Spec.LTDEImport) { + tde_Password, err = r.getSecret(ctx, req, lrpdb, lrpdb.Spec.LTDEPassword.Secret.SecretName, lrpdb.Spec.LTDEPassword.Secret.Key) + if err != nil { + return err + } + tde_Secret, err = r.getSecret(ctx, req, lrpdb, lrpdb.Spec.LTDESecret.Secret.SecretName, lrpdb.Spec.LTDESecret.Secret.Key) + if err != nil { + return err + } + + tde_Secret = tde_Secret[:len(tde_Secret)-1] + tde_Password = tde_Secret[:len(tde_Password)-1] + values["tde_Password"] = tde_Password + values["tdeKeystorePath"] = lrpdb.Spec.LTDEKeystorePath + values["tde_Secret"] = tde_Secret + } + + //url := "https://" + lrpdb.Spec.CDBResName + "-lrest:" + strconv.Itoa(lrest.Spec.LRESTPort) + "/database/pdbs/" + url := r.BaseUrl(ctx, req, lrpdb, lrest) + fmt.Print("============================================================\n") + fmt.Print(url) + fmt.Print("\n============================================================\n") + lrpdb.Status.TotalSize = lrpdb.Spec.TotalSize + lrpdb.Status.Phase = lrpdbPhaseCreate + lrpdb.Status.Msg = "Waiting for LRPDB to be created" + + if err := r.Status().Update(ctx, lrpdb); err != nil { + log.Error(err, "Failed to update status for :"+lrpdb.Name, "err", err.Error()) + } + + respData, err := NewCallLAPI(r, ctx, req, lrpdb, url, values, "POST") + if err != nil { + log.Error(err, "Failure NewCallLAPI( "+url+")", "err", err.Error()) + return err + } + + r.GetSqlCode(respData, &(lrpdb.Status.SqlCode)) + globalsqlcode = lrpdb.Status.SqlCode + if lrpdb.Status.SqlCode != 0 { + err := fmt.Errorf("%v", lrpdb.Status.SqlCode) + return err + } + + r.Recorder.Eventf(lrpdb, corev1.EventTypeNormal, + "Created", "LRPDB '%s' created successfully", lrpdb.Spec.LRPDBName) + + if lrest.Spec.DBServer != "" { + lrpdb.Status.ConnString = + lrest.Spec.DBServer + ":" + strconv.Itoa(lrest.Spec.DBPort) + "/" + lrpdb.Spec.LRPDBName + } else { + log.Info("Parsing connectstring") + lrpdb.Status.ConnString = lrest.Spec.DBTnsurl + parseTnsAlias(&(lrpdb.Status.ConnString), &(lrpdb.Spec.LRPDBName)) + } + + assertiveLpdbDeletion = lrpdb.Spec.AssertiveLrpdbDeletion + if lrpdb.Spec.AssertiveLrpdbDeletion == true { + r.Recorder.Eventf(lrpdb, corev1.EventTypeNormal, "Created", "PDB '%s' assertive pdb deletion turned on", lrpdb.Spec.LRPDBName) + } + + r.getLRPDBState(ctx, req, lrpdb) + log.Info("Created LRPDB Resource", "LRPDB Name", lrpdb.Spec.LRPDBName) + + if bit(lrpdb.Status.Bitstat, MPINIT) == false { + r.InitConfigMap(ctx, req, lrpdb) + Cardinality, _ := r.ApplyConfigMap(ctx, req, lrpdb) + log.Info("Config Map Cardinality " + strconv.Itoa(int(Cardinality))) + } + + if err := r.Status().Update(ctx, lrpdb); err != nil { + log.Error(err, "Failed to update status for :"+lrpdb.Name, "err", err.Error()) + } + + return nil +} + +/* +************************************************ + - Clone a LRPDB + /*********************************************** +*/ +func (r *LRPDBReconciler) cloneLRPDB(ctx context.Context, req ctrl.Request, lrpdb *dbapi.LRPDB) error { + + if lrpdb.Spec.LRPDBName == lrpdb.Spec.SrcLRPDBName { + return nil + } + + log := r.Log.WithValues("cloneLRPDB", req.NamespacedName) + + globalsqlcode = 0 + var err error + + lrest, err := r.getLRESTResource(ctx, req, lrpdb) + if err != nil { + return err + } + + /* Prevent cloning an existing lrpdb */ + err = r.getLRPDBState(ctx, req, lrpdb) + if err != nil { + if apierrors.IsNotFound(err) { + log.Info("Check LRPDB not existence completed", "LRPDB Name", lrpdb.Spec.LRPDBName) + } + + } else { + log.Info("Database already exists ", "LRPDB Name", lrpdb.Spec.LRPDBName) + return nil + } + + values := map[string]string{ + "method": "CLONE", + "pdb_name": lrpdb.Spec.LRPDBName, + "srcPdbName": lrpdb.Spec.SrcLRPDBName, + "reuseTempFile": strconv.FormatBool(*(lrpdb.Spec.ReuseTempFile)), + "unlimitedStorage": strconv.FormatBool(*(lrpdb.Spec.UnlimitedStorage)), + "getScript": strconv.FormatBool(*(lrpdb.Spec.GetScript))} + + //* check the existence of lrpdb.Spec.SrcLRPDBName // + var allErrs field.ErrorList + pdbCounter, _ := r.checkPDBforCloninig(ctx, req, lrpdb, lrpdb.Spec.SrcLRPDBName) + if pdbCounter == 0 { + log.Info("target pdb " + lrpdb.Spec.SrcLRPDBName + " does not exists or is not open") + allErrs = append(allErrs, field.NotFound(field.NewPath("Spec").Child("LRPDBName"), " "+lrpdb.Spec.LRPDBName+" does not exist : failure")) + r.Delete(context.Background(), lrpdb, client.GracePeriodSeconds(1)) + return nil + } + + if lrpdb.Spec.SparseClonePath != "" { + values["sparseClonePath"] = lrpdb.Spec.SparseClonePath + } + if lrpdb.Spec.FileNameConversions != "" { + values["fileNameConversions"] = lrpdb.Spec.FileNameConversions + } + if lrpdb.Spec.TotalSize != "" { + values["totalSize"] = lrpdb.Spec.TotalSize + } + if lrpdb.Spec.TempSize != "" { + values["tempSize"] = lrpdb.Spec.TempSize + } + + url := r.BaseUrl(ctx, req, lrpdb, lrest) + lrpdb.Spec.LRPDBName + "/" + + lrpdb.Status.Phase = lrpdbPhaseClone + lrpdb.Status.Msg = "Waiting for LRPDB to be cloned" + if err := r.Status().Update(ctx, lrpdb); err != nil { + log.Error(err, "Failed to update status for :"+lrpdb.Name, "err", err.Error()) + } + respData, err := NewCallLAPI(r, ctx, req, lrpdb, url, values, "POST") + if err != nil { + log.Error(err, "Failure NewCallLAPI( "+url+")", "err", err.Error()) + return err + } + + r.GetSqlCode(respData, &(lrpdb.Status.SqlCode)) + globalsqlcode = lrpdb.Status.SqlCode + + if lrpdb.Status.SqlCode != 0 { + errclone := errors.New("Cannot clone database: ora-" + strconv.Itoa(lrpdb.Status.SqlCode)) + log.Info("Cannot clone database ora-" + strconv.Itoa(lrpdb.Status.SqlCode)) + lrpdb.Status.Msg = lrpdb.Spec.SrcLRPDBName + " is open in mount cannot clone " + if err := r.Status().Update(ctx, lrpdb); err != nil { + log.Error(err, "Failed to update status for :"+lrpdb.Name, "err", err.Error()) + } + return errclone + } + + r.Recorder.Eventf(lrpdb, corev1.EventTypeNormal, "Created", "LRPDB '%s' cloned successfully", lrpdb.Spec.LRPDBName) + + if lrest.Spec.DBServer != "" { + lrpdb.Status.ConnString = lrest.Spec.DBServer + ":" + strconv.Itoa(lrest.Spec.DBPort) + "/" + lrpdb.Spec.LRPDBName + } else { + lrpdb.Status.ConnString = lrest.Spec.DBTnsurl + parseTnsAlias(&(lrpdb.Status.ConnString), &(lrpdb.Spec.LRPDBName)) + + } + assertiveLpdbDeletion = lrpdb.Spec.AssertiveLrpdbDeletion + if lrpdb.Spec.AssertiveLrpdbDeletion == true { + r.Recorder.Eventf(lrpdb, corev1.EventTypeNormal, "Clone", "PDB '%s' assertive pdb deletion turned on", lrpdb.Spec.LRPDBName) + } + + log.Info("Cloned LRPDB successfully", "Source LRPDB Name", lrpdb.Spec.SrcLRPDBName, "Clone LRPDB Name", lrpdb.Spec.LRPDBName) + r.getLRPDBState(ctx, req, lrpdb) + return nil +} + +/* +************************************************ + - Plug a LRPDB + +*********************************************** +*/ +func (r *LRPDBReconciler) plugLRPDB(ctx context.Context, req ctrl.Request, lrpdb *dbapi.LRPDB) error { + + log := r.Log.WithValues("plugLRPDB", req.NamespacedName) + globalsqlcode = 0 + + var err error + var tde_Password string + var tde_Secret string + + lrest, err := r.getLRESTResource(ctx, req, lrpdb) + if err != nil { + return err + } + + values := map[string]string{ + "method": "PLUG", + "xmlFileName": lrpdb.Spec.XMLFileName, + "pdb_name": lrpdb.Spec.LRPDBName, + "sourceFileNameConversions": lrpdb.Spec.SourceFileNameConversions, + "copyAction": lrpdb.Spec.CopyAction, + "fileNameConversions": lrpdb.Spec.FileNameConversions, + "unlimitedStorage": strconv.FormatBool(*(lrpdb.Spec.UnlimitedStorage)), + "reuseTempFile": strconv.FormatBool(*(lrpdb.Spec.ReuseTempFile)), + "totalSize": lrpdb.Spec.TotalSize, + "tempSize": lrpdb.Spec.TempSize, + "getScript": strconv.FormatBool(*(lrpdb.Spec.GetScript))} + + if *(lrpdb.Spec.LTDEImport) { + tde_Password, err = r.getSecret(ctx, req, lrpdb, lrpdb.Spec.LTDEPassword.Secret.SecretName, lrpdb.Spec.LTDEPassword.Secret.Key) + if err != nil { + return err + } + tde_Secret, err = r.getSecret(ctx, req, lrpdb, lrpdb.Spec.LTDESecret.Secret.SecretName, lrpdb.Spec.LTDESecret.Secret.Key) + if err != nil { + return err + } + + tde_Secret = tde_Secret[:len(tde_Secret)-1] + tde_Password = tde_Secret[:len(tde_Password)-1] + values["tde_Password"] = tde_Password + values["tdeKeystorePath"] = lrpdb.Spec.LTDEKeystorePath + values["tde_Secret"] = tde_Secret + values["tdeImport"] = strconv.FormatBool(*(lrpdb.Spec.LTDEImport)) + } + if *(lrpdb.Spec.AsClone) { + values["asClone"] = strconv.FormatBool(*(lrpdb.Spec.AsClone)) + } + + url := r.BaseUrl(ctx, req, lrpdb, lrest) + + lrpdb.Status.TotalSize = lrpdb.Spec.TotalSize + lrpdb.Status.Phase = lrpdbPhasePlug + lrpdb.Status.Msg = "Waiting for LRPDB to be plugged" + if err := r.Status().Update(ctx, lrpdb); err != nil { + log.Error(err, "Failed to update status for :"+lrpdb.Name, "err", err.Error()) + } + + respData, err := NewCallLAPI(r, ctx, req, lrpdb, url, values, "POST") + if err != nil { + log.Error(err, "Failure NewCallLAPI( "+url+")", "err", err.Error()) + return err + } + + r.GetSqlCode(respData, &(lrpdb.Status.SqlCode)) + globalsqlcode = lrpdb.Status.SqlCode + + if lrpdb.Status.SqlCode != 0 { + log.Info("Plug database failure........:" + strconv.Itoa(lrpdb.Status.SqlCode)) + err = fmt.Errorf("%v", lrpdb.Status.SqlCode) + return err + } + + r.Recorder.Eventf(lrpdb, corev1.EventTypeNormal, "Created", "LRPDB '%s' plugged successfully", lrpdb.Spec.LRPDBName) + + if lrest.Spec.DBServer != "" { + lrpdb.Status.ConnString = lrest.Spec.DBServer + ":" + strconv.Itoa(lrest.Spec.DBPort) + "/" + lrpdb.Spec.LRPDBName + } else { + log.Info("Parsing connectstring") + lrpdb.Status.ConnString = lrest.Spec.DBTnsurl + parseTnsAlias(&(lrpdb.Status.ConnString), &(lrpdb.Spec.LRPDBName)) + } + + assertiveLpdbDeletion = lrpdb.Spec.AssertiveLrpdbDeletion + if lrpdb.Spec.AssertiveLrpdbDeletion == true { + r.Recorder.Eventf(lrpdb, corev1.EventTypeNormal, "Plug", "PDB '%s' assertive pdb deletion turned on", lrpdb.Spec.LRPDBName) + } + + log.Info("Successfully plugged LRPDB", "LRPDB Name", lrpdb.Spec.LRPDBName) + r.getLRPDBState(ctx, req, lrpdb) + return nil +} + +/* +************************************************ + - Unplug a LRPDB + +*********************************************** +*/ +func (r *LRPDBReconciler) unplugLRPDB(ctx context.Context, req ctrl.Request, lrpdb *dbapi.LRPDB) error { + + log := r.Log.WithValues("unplugLRPDB", req.NamespacedName) + globalsqlcode = 0 + + var err error + var tde_Password string + var tde_Secret string + + lrest, err := r.getLRESTResource(ctx, req, lrpdb) + if err != nil { + return err + } + + values := map[string]string{ + "method": "UNPLUG", + "xmlFileName": lrpdb.Spec.XMLFileName, + "getScript": strconv.FormatBool(*(lrpdb.Spec.GetScript))} + + if *(lrpdb.Spec.LTDEExport) { + // Get the TDE Password + tde_Password, err = r.getSecret(ctx, req, lrpdb, lrpdb.Spec.LTDEPassword.Secret.SecretName, lrpdb.Spec.LTDEPassword.Secret.Key) + if err != nil { + return err + } + tde_Secret, err = r.getSecret(ctx, req, lrpdb, lrpdb.Spec.LTDESecret.Secret.SecretName, lrpdb.Spec.LTDESecret.Secret.Key) + if err != nil { + return err + } + + tde_Secret = tde_Secret[:len(tde_Secret)-1] + tde_Password = tde_Secret[:len(tde_Password)-1] + values["tde_Password"] = tde_Password + values["tdeKeystorePath"] = lrpdb.Spec.LTDEKeystorePath + values["tde_Secret"] = tde_Secret + values["tdeExport"] = strconv.FormatBool(*(lrpdb.Spec.LTDEExport)) + } + + url := r.BaseUrl(ctx, req, lrpdb, lrest) + lrpdb.Spec.LRPDBName + "/" + + log.Info("CallAPI(url)", "url", url) + lrpdb.Status.Phase = lrpdbPhaseUnplug + lrpdb.Status.Msg = "Waiting for LRPDB to be unplugged" + if err := r.Status().Update(ctx, lrpdb); err != nil { + log.Error(err, "Failed to update status for :"+lrpdb.Name, "err", err.Error()) + } + respData, err := NewCallLAPI(r, ctx, req, lrpdb, url, values, "POST") + if err != nil { + log.Error(err, "Failure NewCallLAPI( "+url+")", "err", err.Error()) + return err + } + + r.GetSqlCode(respData, &(lrpdb.Status.SqlCode)) + + if lrpdb.Status.SqlCode != 0 { + globalsqlcode = lrpdb.Status.SqlCode + + lrpdb.Status.Msg = lrpdb.Spec.LRPDBName + " database cannot be unplugged " + log.Info(lrpdb.Spec.LRPDBName + " database cannot be unplugged ") + if lrpdb.Status.SqlCode == 65170 { + log.Info(lrpdb.Spec.XMLFileName + " xml file already exists ") + } + + /* + err := r.Update(ctx, lrpdb) + if err != nil { + log.Info("Fail to update crd", "err", err.Error()) + return err + } + + if err := r.Status().Update(ctx, lrpdb); err != nil { + log.Error(err, "Failed to update status"+lrpdb.Name, "err", err.Error()) + return err + } + */ + + r.Recorder.Eventf(lrpdb, corev1.EventTypeNormal, "Unplugged", " ORA-%s ", strconv.Itoa(lrpdb.Status.SqlCode)) + err = fmt.Errorf("%v", lrpdb.Status.SqlCode) + return err + } + + if controllerutil.ContainsFinalizer(lrpdb, LRPDBFinalizer) { + log.Info("Removing finalizer") + controllerutil.RemoveFinalizer(lrpdb, LRPDBFinalizer) + err = r.Update(ctx, lrpdb) + if err != nil { + log.Info("Could not remove finalizer", "err", err.Error()) + return err + } + lrpdb.Status.Status = true + err = r.Delete(context.Background(), lrpdb, client.GracePeriodSeconds(1)) + if err != nil { + log.Info("Could not delete LRPDB resource", "err", err.Error()) + return err + } + } + + r.Recorder.Eventf(lrpdb, corev1.EventTypeNormal, "Unplugged", "LRPDB '%s' unplugged successfully", lrpdb.Spec.LRPDBName) + globalsqlcode = 0 + log.Info("Successfully unplugged LRPDB resource") + return nil +} + +/************************************************** +Alter system LRPDB +**************************************************/ + +/**just push the trasnsaction **/ +func (r *LRPDBReconciler) alterSystemLRPDB(ctx context.Context, req ctrl.Request, lrpdb *dbapi.LRPDB) error { + + log := r.Log.WithValues("alterSystemLRPDB", req.NamespacedName) + globalsqlcode = 0 + + var err error + err = r.getLRPDBState(ctx, req, lrpdb) + if err != nil { + if apierrors.IsNotFound(err) { + log.Info("Warning LRPDB does not exist", "LRPDB Name", lrpdb.Spec.LRPDBName) + return nil + } + return err + } + + lrest, err := r.getLRESTResource(ctx, req, lrpdb) + if err != nil { + log.Info("Cannot find LREST server") + return err + } + + /* alter system payload */ + + values := map[string]string{ + "state": "ALTER", + "alterSystemParameter": lrpdb.Spec.AlterSystemParameter, + "alterSystemValue": lrpdb.Spec.AlterSystemValue, + "parameterScope": lrpdb.Spec.ParameterScope, + } + + lrpdbName := lrpdb.Spec.LRPDBName + url := r.BaseUrl(ctx, req, lrpdb, lrest) + lrpdbName + log.Info("alter system payload...:", "lrpdb.Spec.AlterSystemValue=", lrpdb.Spec.AlterSystemValue) + log.Info("alter system payload...:", "lrpdb.Spec.AlterSystemParameter=", lrpdb.Spec.AlterSystemParameter) + log.Info("alter system payload...:", "lrpdb.Spec.ParameterScope=", lrpdb.Spec.ParameterScope) + log.Info("alter system path.......:", "url=", url) + + lrpdb.Status.Phase = lrpdbPhaseAlterPlug + lrpdb.Status.ModifyOption = lrpdb.Spec.AlterSystem + " " + lrpdb.Spec.ParameterScope + lrpdb.Status.Msg = "Waiting for LRPDB parameter to be modified" + if err := r.Status().Update(ctx, lrpdb); err != nil { + log.Error(err, "Failed to update lrpdb parameter :"+lrpdb.Name, "err", err.Error()) + return err + } + + respData, err := NewCallLAPI(r, ctx, req, lrpdb, url, values, "POST") + if err != nil { + log.Error(err, "Failure NewCallLAPI( "+url+")", "err", err.Error()) + return err + } + + r.GetSqlCode(respData, &(lrpdb.Status.SqlCode)) + globalsqlcode = lrpdb.Status.SqlCode + + if lrpdb.Status.SqlCode == 0 { + r.Recorder.Eventf(lrpdb, corev1.EventTypeNormal, "Altered", "LRPDB(name,cmd,sqlcode) '%s %s %d' ", lrpdb.Spec.LRPDBName, lrpdb.Spec.AlterSystem, lrpdb.Status.SqlCode) + lrpdb.Status.Phase = lrpdbPhaseReady + lrpdb.Spec.Action = "Noaction" + lrpdb.Status.Action = "Noaction" + lrpdb.Status.Status = true + + if err := r.Update(ctx, lrpdb); err != nil { + log.Error(err, "Cannot rest lrpdb Spec :"+lrpdb.Name, "err", err.Error()) + return err + } + + if err := r.Status().Update(ctx, lrpdb); err != nil { + log.Error(err, "Failed to update lrpdb parameter :"+lrpdb.Name, "err", err.Error()) + return err + } + return nil + + } + + if lrpdb.Status.SqlCode != 0 { + r.Recorder.Eventf(lrpdb, corev1.EventTypeWarning, "alter system failure", "LRPDB(name,cmd,sqlcode) '%s %s %d' ", lrpdb.Spec.LRPDBName, lrpdb.Spec.AlterSystem, lrpdb.Status.SqlCode) + erralter := errors.New("Error: cannot modify parameter") + + lrpdb.Status.ModifyOption = lrpdb.Spec.AlterSystem + " " + lrpdb.Spec.ParameterScope + lrpdb.Status.Msg = "Failed: cannot modify system parameter" + lrpdb.Status.Phase = lrpdbPhaseStatus + lrpdb.Spec.AlterSystem = "" + lrpdb.Spec.ParameterScope = "" + lrpdb.Spec.Action = "Noaction" + if err := r.Update(ctx, lrpdb); err != nil { + log.Error(err, "Cannot rest lrpdb Spec :"+lrpdb.Name, "err", err.Error()) + return err + } + + if err := r.Status().Update(ctx, lrpdb); err != nil { + log.Error(err, "Failed to update lrpdb parameter :"+lrpdb.Name, "err", err.Error()) + return err + } + return erralter + } + + lrpdb.Status.Status = false + + if err := r.Status().Update(ctx, lrpdb); err != nil { + log.Error(err, "Failed to update lrpdb parameter :"+lrpdb.Name, "err", err.Error()) + return err + } + return nil +} + +/************************************************* + * Modify a LRPDB state + ***********************************************/ +func (r *LRPDBReconciler) modifyLRPDB(ctx context.Context, req ctrl.Request, lrpdb *dbapi.LRPDB) error { + + log := r.Log.WithValues("modifyLRPDB", req.NamespacedName) + r.Recorder.Eventf(lrpdb, corev1.EventTypeNormal, "Modify", "Info:'%s %s %s' ", lrpdb.Spec.LRPDBName, lrpdb.Spec.LRPDBState, lrpdb.Status.ModifyOption) + + var err error + err = r.getLRPDBState(ctx, req, lrpdb) + if err != nil { + if lrpdb.Status.SqlCode == 1403 { + // BUG 36752465 + // We have to handle to verify a non existings results using both + log.Info("Database does not exists ") + r.Delete(context.Background(), lrpdb, client.GracePeriodSeconds(1)) + return nil + } + if apierrors.IsNotFound(err) { + log.Info("Warning LRPDB does not exist", "LRPDB Name", lrpdb.Spec.LRPDBName) + r.Delete(context.Background(), lrpdb, client.GracePeriodSeconds(1)) + return nil + } + return err + } + + /* This scenario is managed by webhook acceptance test ... leave it here anyway */ + if lrpdb.Status.OpenMode == "READ WRITE" && lrpdb.Spec.LRPDBState == "OPEN" && lrpdb.Spec.ModifyOption == "READ WRITE" { + /* Database is already open no action required */ + return nil + } + + if lrpdb.Status.OpenMode == "MOUNTED" && lrpdb.Spec.LRPDBState == "CLOSE" && lrpdb.Spec.ModifyOption == "IMMEDIATE" { + /* Database is already close no action required */ + return nil + } + + lrest, err := r.getLRESTResource(ctx, req, lrpdb) + if err != nil { + return err + } + + values := map[string]string{} + if lrpdb.Spec.LRPDBState == "OPEN" || lrpdb.Spec.LRPDBState == "CLOSE" { + values = map[string]string{ + "state": lrpdb.Spec.LRPDBState, + "modifyOption": lrpdb.Spec.ModifyOption, + "getScript": strconv.FormatBool(*(lrpdb.Spec.GetScript))} + if lrpdb.Spec.LRPDBState == "OPEN" || lrpdb.Spec.LRPDBState == "CLOSE" { + log.Info("MODIFY LRPDB", "lrpdb.Spec.LRPDBState=", lrpdb.Spec.LRPDBState, "lrpdb.Spec.ModifyOption=", lrpdb.Spec.ModifyOption) + log.Info("LRPDB STATUS OPENMODE", "lrpdb.Status.OpenMode=", lrpdb.Status.OpenMode) + } + } + + lrpdbName := lrpdb.Spec.LRPDBName + url := r.BaseUrl(ctx, req, lrpdb, lrest) + lrpdbName + "/status/" + + lrpdb.Status.Phase = lrpdbPhaseModify + if lrpdb.Spec.LRPDBState == "OPEN" || lrpdb.Spec.LRPDBState == "CLOSE" { + lrpdb.Status.ModifyOption = lrpdb.Spec.LRPDBState + "-" + lrpdb.Spec.ModifyOption + } + + lrpdb.Status.Msg = "Waiting for LRPDB to be modified" + if err := r.Status().Update(ctx, lrpdb); err != nil { + log.Error(err, "Failed to update status for :"+lrpdb.Name, "err", err.Error()) + } + + respData, err := NewCallLAPI(r, ctx, req, lrpdb, url, values, "POST") + if err != nil { + log.Error(err, "Failure NewCallLAPI( "+url+")", "err", err.Error()) + return err + } + + r.GetSqlCode(respData, &(lrpdb.Status.SqlCode)) + globalsqlcode = lrpdb.Status.SqlCode + + if lrpdb.Spec.LRPDBState == "OPEN" || lrpdb.Spec.LRPDBState == "CLOSE" { + r.Recorder.Eventf(lrpdb, corev1.EventTypeNormal, "Modified", " '%s' modified successfully '%s'", lrpdb.Spec.LRPDBName, lrpdb.Spec.LRPDBState) + } + + if lrest.Spec.DBServer != "" { + lrpdb.Status.ConnString = lrest.Spec.DBServer + ":" + strconv.Itoa(lrest.Spec.DBPort) + "/" + lrpdb.Spec.LRPDBName + } else { + lrpdb.Status.ConnString = lrest.Spec.DBTnsurl + parseTnsAlias(&(lrpdb.Status.ConnString), &(lrpdb.Spec.LRPDBName)) + + } + + lrpdb.Status.Msg = "alter lrpdb completed" + lrpdb.Status.Status = false + lrpdb.Status.Phase = lrpdbPhaseReady + + log.Info("Successfully modified LRPDB state", "LRPDB Name", lrpdb.Spec.LRPDBName) + + /* After database openining we reapply the config map if warning is present */ + if lrpdb.Spec.LRPDBState == "OPEN" { + if bit(lrpdb.Status.Bitstat, MPWARN|MPINIT) { + log.Info("re-apply config map") + r.ApplyConfigMap(ctx, req, lrpdb) + + } + } + if err := r.Status().Update(ctx, lrpdb); err != nil { + log.Error(err, "Failed to update status for :"+lrpdb.Name, "err", err.Error()) + } + + //r.getLRPDBState(ctx, req, lrpdb) + return nil +} + +/* +************************************************ + - Get LRPDB State + /*********************************************** +*/ +func (r *LRPDBReconciler) getLRPDBState(ctx context.Context, req ctrl.Request, lrpdb *dbapi.LRPDB) error { + + log := r.Log.WithValues("getLRPDBState", req.NamespacedName) + + var err error + + lrest, err := r.getLRESTResource(ctx, req, lrpdb) + if err != nil { + return err + } + + lrpdbName := lrpdb.Spec.LRPDBName + url := r.BaseUrl(ctx, req, lrpdb, lrest) + lrpdbName + "/status/" + + lrpdb.Status.Msg = "Getting LRPDB state" + fmt.Print("============================\n") + fmt.Println(lrpdb.Status) + fmt.Print("============================\n") + if err := r.Status().Update(ctx, lrpdb); err != nil { + log.Error(err, "Failed to update status for :"+lrpdb.Name, "err", err.Error()) + } + + respData, err := NewCallLAPI(r, ctx, req, lrpdb, url, nil, "GET") + if err != nil { + log.Info("Begin respData") + log.Info(respData) + log.Info("End respData") + lrpdb.Status.Msg = "getLRPDBState failure : check lrpdb status" + lrpdb.Status.Status = false + log.Error(err, "Failure NewCallLAPI( "+url+")", "err", err.Error()) + return err + } + + r.GetSqlCode(respData, &(lrpdb.Status.SqlCode)) + globalsqlcode = lrpdb.Status.SqlCode + + if lrpdb.Status.SqlCode == 1403 { + lrpdb.Status.OpenMode = "unknown" + lrpdb.Status.Msg = "check lrpdb status" + lrpdb.Status.Status = false + return errors.New("NO_DATA_FOUND") + } + + var objmap map[string]interface{} + if err := json.Unmarshal([]byte(respData), &objmap); err != nil { + log.Error(err, "Failed to get state of LRPDB :"+lrpdbName, "err", err.Error()) + } + lrpdb.Status.OpenMode = objmap["open_mode"].(string) + + /* if lrpdb.Status.Phase == lrpdbPhaseCreate && sqlcode == 1403 { + + if lrpdb.Status.OpenMode == "READ WRITE" { + err := r.mapLRPDB(ctx, req, lrpdb) + if err != nil { + log.Info("Fail to Map resource getting LRPDB state") + } + } + + if lrpdb.Status.OpenMode == "MOUNTED" { + err := r.mapLRPDB(ctx, req, lrpdb) + if err != nil { + log.Info("Fail to Map resource getting LRPDB state") + } + } + }*/ + + lrpdb.Status.Msg = "check lrpdb ok" + if err := r.Status().Update(ctx, lrpdb); err != nil { + log.Error(err, "Failed to update status for :"+lrpdb.Name, "err", err.Error()) + } + + log.Info("Successfully obtained LRPDB state", "LRPDB Name", lrpdb.Spec.LRPDBName, "State", objmap["open_mode"].(string)) + return nil +} + +/* +************************************************ + - Map Database LRPDB to Kubernetes LRPDB CR + +/*********************************************** +*/ +func (r *LRPDBReconciler) mapLRPDB(ctx context.Context, req ctrl.Request, lrpdb *dbapi.LRPDB) error { + + log := r.Log.WithValues("mapLRPDB", req.NamespacedName) + + var err error + + lrest, err := r.getLRESTResource(ctx, req, lrpdb) + if err != nil { + return err + } + + log.Info("callapi get to map lrpdb") + + lrpdbName := lrpdb.Spec.LRPDBName + url := r.BaseUrl(ctx, req, lrpdb, lrest) + lrpdbName + log.Info("DEBUG NEW URL " + url) + + respData, err := NewCallLAPI(r, ctx, req, lrpdb, url, nil, "GET") + if err != nil { + log.Error(err, "Failure NewCallLAPI( "+url+")", "err", err.Error()) + return err + } + + var objmap map[string]interface{} + if err := json.Unmarshal([]byte(respData), &objmap); err != nil { + log.Error(err, "Failed json.Unmarshal :"+lrpdbName, "err", err.Error()) + } + + //fmt.Printf("%+v\n", objmap) + totSizeInBytes := objmap["total_size"].(float64) + totSizeInGB := totSizeInBytes / 1024 / 1024 / 1024 + + lrpdb.Status.OpenMode = objmap["open_mode"].(string) + lrpdb.Status.TotalSize = fmt.Sprintf("%4.2f", totSizeInGB) + "G" + assertiveLpdbDeletion = lrpdb.Spec.AssertiveLrpdbDeletion + if lrpdb.Spec.AssertiveLrpdbDeletion == true { + r.Recorder.Eventf(lrpdb, corev1.EventTypeNormal, "Map", "PDB '%s' assertive pdb deletion turned on", lrpdb.Spec.LRPDBName) + } + + if lrest.Spec.DBServer != "" { + lrpdb.Status.ConnString = lrest.Spec.DBServer + ":" + strconv.Itoa(lrest.Spec.DBPort) + "/" + lrpdb.Spec.LRPDBName + } else { + lrpdb.Status.ConnString = lrest.Spec.DBTnsurl + parseTnsAlias(&(lrpdb.Status.ConnString), &(lrpdb.Spec.LRPDBName)) + } + + lrpdb.Status.Phase = lrpdbPhaseReady + + if err := r.Status().Update(ctx, lrpdb); err != nil { + log.Error(err, "Failed to update status for :"+lrpdb.Name, "err", err.Error()) + } + + log.Info("Successfully mapped LRPDB to Kubernetes resource", "LRPDB Name", lrpdb.Spec.LRPDBName) + lrpdb.Status.Status = true + return nil +} + +/* +************************************************ + - Delete a LRPDB + /*********************************************** +*/ +func (r *LRPDBReconciler) deleteLRPDB(ctx context.Context, req ctrl.Request, lrpdb *dbapi.LRPDB) error { + + log := r.Log.WithValues("deleteLRPDB", req.NamespacedName) + + errstate := r.getLRPDBState(ctx, req, lrpdb) + if errstate != nil { + if lrpdb.Status.SqlCode == 1403 { + // BUG 36752336: + log.Info("Database does not exists ") + r.Delete(context.Background(), lrpdb, client.GracePeriodSeconds(1)) + return nil + } + if apierrors.IsNotFound(errstate) { + log.Info("Warning LRPDB does not exist", "LRPDB Name", lrpdb.Spec.LRPDBName) + r.Delete(context.Background(), lrpdb, client.GracePeriodSeconds(1)) + return nil + } + log.Error(errstate, "Failed to update status for :"+lrpdb.Name, "err", errstate.Error()) + return errstate + //* if the pdb does not exists delete the crd *// + + } + + if lrpdb.Status.OpenMode == "READ WRITE" { + + errdel := errors.New("pdb is open cannot delete it") + log.Info("LRPDB is open in read write cannot drop ") + lrpdb.Status.Msg = "LRPDB is open in read write cannot drop " + if err := r.Status().Update(ctx, lrpdb); err != nil { + log.Error(err, "Failed to update status for :"+lrpdb.Name, "err", err.Error()) + } + + return errdel + } + + err := r.deleteLRPDBInstance(req, ctx, lrpdb) + if err != nil { + log.Info("Could not delete LRPDB", "LRPDB Name", lrpdb.Spec.LRPDBName, "err", err.Error()) + return err + } + + if controllerutil.ContainsFinalizer(lrpdb, LRPDBFinalizer) { + log.Info("Removing finalizer") + controllerutil.RemoveFinalizer(lrpdb, LRPDBFinalizer) + err := r.Update(ctx, lrpdb) + if err != nil { + log.Info("Could not remove finalizer", "err", err.Error()) + return err + } + lrpdb.Status.Status = true + err = r.Delete(context.Background(), lrpdb, client.GracePeriodSeconds(1)) + if err != nil { + log.Info("Could not delete LRPDB resource", "err", err.Error()) + return err + } + } + + r.Recorder.Eventf(lrpdb, corev1.EventTypeNormal, "Deleted", "LRPDB '%s' dropped successfully", lrpdb.Spec.LRPDBName) + + log.Info("Successfully deleted LRPDB resource") + return nil +} + +/* +************************************************ + - Check LRPDB deletion + /*********************************************** +*/ +func (r *LRPDBReconciler) manageLRPDBDeletion(ctx context.Context, req ctrl.Request, lrpdb *dbapi.LRPDB) error { + log := r.Log.WithValues("manageLRPDBDeletion", req.NamespacedName) + + // Check if the LRPDB instance is marked to be deleted, which is + // indicated by the deletion timestamp being set. + isLRPDBMarkedToBeDeleted := lrpdb.GetDeletionTimestamp() != nil + if isLRPDBMarkedToBeDeleted { + log.Info("Marked to be deleted") + lrpdb.Status.Phase = lrpdbPhaseDelete + lrpdb.Status.Status = true + r.Status().Update(ctx, lrpdb) + + if controllerutil.ContainsFinalizer(lrpdb, LRPDBFinalizer) { + // Remove LRPDBFinalizer. Once all finalizers have been + // removed, the object will be deleted. + log.Info("Removing finalizer") + controllerutil.RemoveFinalizer(lrpdb, LRPDBFinalizer) + err := r.Update(ctx, lrpdb) + if err != nil { + log.Info("Could not remove finalizer", "err", err.Error()) + return err + } + log.Info("Successfully removed LRPDB resource") + return nil + } + } + + // Add finalizer for this CR + if !controllerutil.ContainsFinalizer(lrpdb, LRPDBFinalizer) { + log.Info("Adding finalizer") + controllerutil.AddFinalizer(lrpdb, LRPDBFinalizer) + err := r.Update(ctx, lrpdb) + if err != nil { + log.Info("Could not add finalizer", "err", err.Error()) + return err + } + lrpdb.Status.Status = false + } + return nil +} + +/* +************************************************ + - Finalization logic for LRPDBFinalizer + +*********************************************** +*/ +func (r *LRPDBReconciler) deleteLRPDBInstance(req ctrl.Request, ctx context.Context, lrpdb *dbapi.LRPDB) error { + + log := r.Log.WithValues("deleteLRPDBInstance", req.NamespacedName) + + var err error + + lrest, err := r.getLRESTResource(ctx, req, lrpdb) + if err != nil { + return err + } + + values := map[string]string{ + "action": "KEEP", + "getScript": strconv.FormatBool(*(lrpdb.Spec.GetScript))} + + if lrpdb.Spec.DropAction != "" { + values["action"] = lrpdb.Spec.DropAction + } + + lrpdbName := lrpdb.Spec.LRPDBName + url := r.BaseUrl(ctx, req, lrpdb, lrest) + lrpdbName + "/" + + lrpdb.Status.Phase = lrpdbPhaseDelete + lrpdb.Status.Msg = "Waiting for LRPDB to be deleted" + if err := r.Status().Update(ctx, lrpdb); err != nil { + log.Error(err, "Failed to update status for :"+lrpdb.Name, "err", err.Error()) + } + + respData, err := NewCallLAPI(r, ctx, req, lrpdb, url, values, "DELETE") + if err != nil { + log.Error(err, "Failure NewCallLAPI( "+url+")", "err", err.Error()) + return err + } + + r.GetSqlCode(respData, &(lrpdb.Status.SqlCode)) + globalsqlcode = lrpdb.Status.SqlCode + + log.Info("Successfully dropped LRPDB", "LRPDB Name", lrpdbName) + return nil +} + +/* +*********************************************************** + - SetupWithManager sets up the controller with the Manager + +************************************************************ +*/ +func (r *LRPDBReconciler) SetupWithManager(mgr ctrl.Manager) error { + return ctrl.NewControllerManagedBy(mgr). + For(&dbapi.LRPDB{}). + WithEventFilter(predicate.Funcs{ + UpdateFunc: func(e event.UpdateEvent) bool { + // Ignore updates to CR status in which case metadata.Generation does not change + return e.ObjectOld.GetGeneration() != e.ObjectNew.GetGeneration() + }, + DeleteFunc: func(e event.DeleteEvent) bool { + // Evaluates to false if the object has been confirmed deleted. + //return !e.DeleteStateUnknown + return false + }, + }). + WithOptions(controller.Options{MaxConcurrentReconciles: 100}). + Complete(r) +} + +/************************************************************* +Enh 35357707 - PROVIDE THE LRPDB TNSALIAS INFORMATION +**************************************************************/ + +func parseTnsAlias(tns *string, lrpdbsrv *string) { + fmt.Printf("Analyzing string [%s]\n", *tns) + fmt.Printf("Relacing srv [%s]\n", *lrpdbsrv) + var swaptns string + + if strings.Contains(strings.ToUpper(*tns), "SERVICE_NAME") == false { + fmt.Print("Cannot generate tns alias for lrpdb") + return + } + + if strings.Contains(strings.ToUpper(*tns), "ORACLE_SID") == true { + fmt.Print("Cannot generate tns alias for lrpdb") + return + } + + swaptns = fmt.Sprintf("SERVICE_NAME=%s", *lrpdbsrv) + tnsreg := regexp.MustCompile(`SERVICE_NAME=\w+`) + *tns = tnsreg.ReplaceAllString(*tns, swaptns) + + fmt.Printf("Newstring [%s]\n", *tns) + +} + +// Compose url +func (r *LRPDBReconciler) BaseUrl(ctx context.Context, req ctrl.Request, lrpdb *dbapi.LRPDB, lrest dbapi.LREST) string { + log := r.Log.WithValues("BaseUrl", req.NamespacedName) + baseurl := "https://" + lrpdb.Spec.CDBResName + "-lrest." + lrpdb.Spec.CDBNamespace + ":" + strconv.Itoa(lrest.Spec.LRESTPort) + "/database/pdbs/" + log.Info("Baseurl:" + baseurl) + return baseurl +} + +func (r *LRPDBReconciler) DecryptWithPrivKey(Key string, Buffer string, req ctrl.Request) (string, error) { + log := r.Log.WithValues("DecryptWithPrivKey", req.NamespacedName) + Debug := 0 + block, _ := pem.Decode([]byte(Key)) + pkcs8PrivateKey, err := x509.ParsePKCS8PrivateKey(block.Bytes) + if err != nil { + log.Error(err, "Failed to parse private key - "+err.Error()) + return "", err + } + if Debug == 1 { + fmt.Printf("======================================\n") + fmt.Printf("%s\n", Key) + fmt.Printf("======================================\n") + } + + encString64, err := base64.StdEncoding.DecodeString(string(Buffer)) + if err != nil { + log.Error(err, "Failed to decode encrypted string to base64 - "+err.Error()) + return "", err + } + + decryptedB, err := rsa.DecryptPKCS1v15(nil, pkcs8PrivateKey.(*rsa.PrivateKey), encString64) + if err != nil { + log.Error(err, "Failed to decrypt string - "+err.Error()) + return "", err + } + if Debug == 1 { + fmt.Printf("[%s]\n", string(decryptedB)) + } + return strings.TrimSpace(string(decryptedB)), err + +} + +// New function to decrypt credential using private key +func (r *LRPDBReconciler) getEncriptedSecret(ctx context.Context, req ctrl.Request, lrpdb *dbapi.LRPDB, secretName string, keyName string, secretNamePk string, keyNamePk string) (string, error) { + + log := r.Log.WithValues("getEncriptedSecret", req.NamespacedName) + + log.Info("getEncriptedSecret :" + secretName) + secret1 := &corev1.Secret{} + err := r.Get(ctx, types.NamespacedName{Name: secretName, Namespace: lrpdb.Namespace}, secret1) + if err != nil { + if apierrors.IsNotFound(err) { + log.Info("Secret not found:" + secretName) + lrpdb.Status.Msg = "Secret not found:" + secretName + return "", err + } + log.Error(err, "Unable to get the secret.") + return "", err + } + + secret2 := &corev1.Secret{} + err = r.Get(ctx, types.NamespacedName{Name: secretNamePk, Namespace: lrpdb.Namespace}, secret2) + if err != nil { + if apierrors.IsNotFound(err) { + log.Info("Secret not found:" + secretNamePk) + lrpdb.Status.Msg = "Secret not found:" + secretNamePk + return "", err + } + log.Error(err, "Unable to get the secret.") + return "", err + } + + Encval := string(secret1.Data[keyName]) + Encval = strings.TrimSpace(Encval) + + privKey := string(secret2.Data[keyNamePk]) + privKey = strings.TrimSpace(privKey) + + /* Debuug info for dev phase + fmt.Printf("DEBUG Secretename:secretName :%s\n", secretName) + fmt.Printf("DEBUG privKey :%s\n", privKey) + fmt.Printf("DEBUG Encval :%s\n", Encval) + */ + + DecVal, err := r.DecryptWithPrivKey(privKey, Encval, req) + if err != nil { + log.Error(err, "Fail to decrypt secret:"+secretName) + lrpdb.Status.Msg = " Fail to decrypt secret:" + secretName + return "", err + } + return DecVal, nil +} + +func (r *LRPDBReconciler) manageLRPDBDeletion2(ctx context.Context, req ctrl.Request, lrpdb *dbapi.LRPDB) error { + log := r.Log.WithValues("manageLRPDBDeletion", req.NamespacedName) + if lrpdb.ObjectMeta.DeletionTimestamp.IsZero() { + if !controllerutil.ContainsFinalizer(lrpdb, LRPDBFinalizer) { + controllerutil.AddFinalizer(lrpdb, LRPDBFinalizer) + if err := r.Update(ctx, lrpdb); err != nil { + return err + } + } + } else { + log.Info("Pdb marked to be delted") + if controllerutil.ContainsFinalizer(lrpdb, LRPDBFinalizer) { + if assertiveLpdbDeletion == true { + log.Info("Deleting lrpdb CRD: Assertive approach is turned on ") + lrest, err := r.getLRESTResource(ctx, req, lrpdb) + if err != nil { + log.Error(err, "Cannont find cdb resource ", "err", err.Error()) + return err + } + + lrpdbName := lrpdb.Spec.LRPDBName + if lrpdb.Status.OpenMode == "READ WRITE" { + valuesclose := map[string]string{ + "state": "CLOSE", + "modifyOption": "IMMEDIATE", + "getScript": "FALSE"} + lrpdbName := lrpdb.Spec.LRPDBName + url := r.BaseUrl(ctx, req, lrpdb, lrest) + lrpdbName + "/status/" + _, errclose := r.callAPI(ctx, req, lrpdb, url, valuesclose, "POST") + if errclose != nil { + log.Info("Warning error closing lrpdb continue anyway") + } + } + + valuesdrop := map[string]string{ + "action": "INCLUDING", + "getScript": "FALSE"} + url := r.BaseUrl(ctx, req, lrpdb, lrest) + lrpdbName + "/" + + log.Info("Call Delete()") + _, errdelete := r.callAPI(ctx, req, lrpdb, url, valuesdrop, "DELETE") + if errdelete != nil { + log.Error(errdelete, "Fail to delete lrpdb :"+lrpdb.Name, "err", err.Error()) + return errdelete + } + } /* END OF ASSERTIVE SECTION */ + + log.Info("Marked to be deleted") + lrpdb.Status.Phase = lrpdbPhaseDelete + lrpdb.Status.Status = true + r.Status().Update(ctx, lrpdb) + + controllerutil.RemoveFinalizer(lrpdb, LRPDBFinalizer) + if err := r.Update(ctx, lrpdb); err != nil { + log.Info("Cannot remove finalizer") + return err + } + + } + + return nil + } + + return nil +} + +func (r *LRPDBReconciler) InitConfigMap(ctx context.Context, req ctrl.Request, lrpdb *dbapi.LRPDB) *corev1.ConfigMap { + log := r.Log.WithValues("InitConfigMap", req.NamespacedName) + log.Info("ConfigMap..............:" + "ConfigMap" + lrpdb.Name) + log.Info("ConfigMap nmsp.........:" + lrpdb.Namespace) + /* + * PDB SYSTEM PARAMETER + * record [name,value=[paramete_val|reset],level=[session|system]] + */ + + if lrpdb.Spec.PDBConfigMap == "" { + /* if users does not specify a config map + we generate an empty new one for possible + future pdb parameter modification */ + + var SystemParameters map[string]string + + log.Info("Generating an empty configmap") + globalconfigmap = "configmap-" + lrpdb.Spec.LRPDBName + "-default" + DbParameters := &corev1.ConfigMap{ + TypeMeta: metav1.TypeMeta{ + Kind: "configmap", + APIVersion: "v1", + }, + ObjectMeta: metav1.ObjectMeta{ + Name: globalconfigmap, + Namespace: lrpdb.Namespace, + }, + Data: SystemParameters, + } + + if err := ctrl.SetControllerReference(lrpdb, DbParameters, r.Scheme); err != nil { + log.Error(err, "Fail to set SetControllerReference", "err", err.Error()) + return nil + } + + /* Update Spec.PDBConfigMap */ + lrpdb.Spec.PDBConfigMap = "configmap" + lrpdb.Spec.LRPDBName + "default" + if err := r.Update(ctx, lrpdb); err != nil { + log.Error(err, "Failure updating Spec.PDBConfigMap ", "err", err.Error()) + return nil + } + lrpdb.Status.Bitstat = bis(lrpdb.Status.Bitstat, MPEMPT) + return DbParameters + + } else { + + lrpdb.Status.Bitstat = bis(lrpdb.Status.Bitstat, MPINIT) + globalconfigmap = lrpdb.Spec.PDBConfigMap + DbParameters, err := r.GetConfigMap(ctx, req, lrpdb) + if err != nil { + log.Error(err, "Fail to fetch configmap ", "err", err.Error()) + return nil + } + + //ParseConfigMapData(DbParameters) + + return DbParameters + } + + return nil +} + +func (r *LRPDBReconciler) GetConfigMap(ctx context.Context, req ctrl.Request, lrpdb *dbapi.LRPDB) (*corev1.ConfigMap, error) { + log := r.Log.WithValues("GetConfigMap", req.NamespacedName) + log.Info("ConfigMapGlobal.............:" + globalconfigmap) + DbParameters, err := k8s.FetchConfigMap(r.Client, lrpdb.Namespace, globalconfigmap) + if err != nil { + log.Error(err, "Fail to fetch configmap", "err", err.Error()) + return nil, err + } + + return DbParameters, nil +} + +func (r *LRPDBReconciler) ApplyConfigMap(ctx context.Context, req ctrl.Request, lrpdb *dbapi.LRPDB) (int32, error) { + log := r.Log.WithValues("ApplyConfigMap", req.NamespacedName) + /* We read the config map and apply the setting to the pdb */ + + log.Info("Starting Apply Config Map Process") + configmap, err := r.GetConfigMap(ctx, req, lrpdb) + if err != nil { + log.Info("Cannot get config map in the open yaml file") + return 0, nil + } + Cardinality := int32(len(configmap.Data)) + if Cardinality == 0 { + log.Info("Empty config map... nothing to do ") + return 0, nil + } + log.Info("GetConfigMap completed") + + lrest, err := r.getLRESTResource(ctx, req, lrpdb) + if err != nil { + log.Info("Cannot find lrest server") + return 0, nil + } + tokens := lrcommons.ParseConfigMapData(configmap) + for cnt := range tokens { + if len(tokens[cnt]) != 0 { + /* avoid null token and check malformed value */ + fmt.Printf("token=[%s]\n", tokens[cnt]) + Parameter := strings.Split(tokens[cnt], " ") + if len(Parameter) != 3 { + log.Info("WARNING malformed value in the configmap") + } else { + fmt.Printf("alter system set %s=%s scope=%s instances=all\n", Parameter[0], Parameter[1], Parameter[2]) + /* Preparing PayLoad + ----------------- + WARNING: event setting is not yet supported. It will be implemented in future release + */ + AlterSystemPayload := map[string]string{ + "state": "ALTER", + "alterSystemParameter": Parameter[0], + "alterSystemValue": Parameter[1], + "parameterScope": Parameter[2], + } + url := r.BaseUrl(ctx, req, lrpdb, lrest) + lrpdb.Spec.LRPDBName + respData, err := r.callAPI(ctx, req, lrpdb, url, AlterSystemPayload, "POST") + if err != nil { + log.Error(err, "callAPI failure durring Apply Config Map", "err", err.Error()) + return 0, err + } + /* check sql code execution */ + var retJson map[string]interface{} + if err := json.Unmarshal([]byte(respData), &retJson); err != nil { + log.Error(err, "failed to get Data from callAPI", "err", err.Error()) + return 0, err + } + /* We do not the execution if something goes wrong for a single parameter + just report the error in the event queue */ + SqlCode := strconv.Itoa(int(retJson["sqlcode"].(float64))) + AlterMsg := fmt.Sprintf("pdb=%s:%s:%s:%s:%s", lrpdb.Spec.LRPDBName, Parameter[0], Parameter[1], Parameter[2], SqlCode) + log.Info("Config Map Apply:......." + AlterMsg) + + if SqlCode != "0" { + r.Recorder.Eventf(lrpdb, corev1.EventTypeWarning, "LRESTINFO", AlterMsg) + lrpdb.Status.Bitstat = bis(lrpdb.Status.Bitstat, MPWARN) + } + + } + } + + } + + lrpdb.Status.Bitstat = bis(lrpdb.Status.Bitstat, MPAPPL) + + return Cardinality, nil +} + +func (r *LRPDBReconciler) ManageConfigMapForCloningAndPlugin(ctx context.Context, req ctrl.Request, lrpdb *dbapi.LRPDB) error { + log := r.Log.WithValues("ManageConfigMapForCloningAndPlugin", req.NamespacedName) + log.Info("Frame:") + /* + If configmap parameter is set and init flag is not set + then we need to iniialized the init mask. This is the case for + pdb generated by clone and plug action + */ + if lrpdb.Spec.Action != "CREATE" && lrpdb.Spec.PDBConfigMap != "" && bit(lrpdb.Status.Bitstat, MPINIT) == false { + if r.InitConfigMap(ctx, req, lrpdb) == nil { + log.Info("Cannot initialize config map for pdb.........:" + lrpdb.Spec.LRPDBName) + return nil + } + log.Info("Call...........:ApplyConfigMap(ctx, req, lrpdb)") + Cardinality, _ := r.ApplyConfigMap(ctx, req, lrpdb) + log.Info("Cardnality:....:" + strconv.Itoa(int(Cardinality))) + if Cardinality == 0 { + return nil + } + + } + return nil +} + +func NewCallLAPI(intr interface{}, ctx context.Context, req ctrl.Request, lrpdb *dbapi.LRPDB, url string, payload map[string]string, action string) (string, error) { + var c client.Client + var r logr.Logger + var e record.EventRecorder + var err error + + recpdb, ok1 := intr.(*LRPDBReconciler) + if ok1 { + fmt.Printf("func NewCallLApi ((*PDBReconciler),......)\n") + c = recpdb.Client + e = recpdb.Recorder + r = recpdb.Log + } + + reccdb, ok2 := intr.(*LRESTReconciler) + if ok2 { + fmt.Printf("func NewCallLApi ((*CDBReconciler),......)\n") + c = reccdb.Client + e = reccdb.Recorder + r = reccdb.Log + } + + log := r.WithValues("NewCallLAPI", req.NamespacedName) + + secret := &corev1.Secret{} + + err = c.Get(ctx, types.NamespacedName{Name: lrpdb.Spec.LRPDBTlsKey.Secret.SecretName, Namespace: lrpdb.Namespace}, secret) + + if err != nil { + if apierrors.IsNotFound(err) { + log.Info("Secret not found:" + lrpdb.Spec.LRPDBTlsKey.Secret.SecretName) + return "", err + } + log.Error(err, "Unable to get the secret.") + return "", err + } + rsaKeyPEM := secret.Data[lrpdb.Spec.LRPDBTlsKey.Secret.Key] + + err = c.Get(ctx, types.NamespacedName{Name: lrpdb.Spec.LRPDBTlsCrt.Secret.SecretName, Namespace: lrpdb.Namespace}, secret) + + if err != nil { + if apierrors.IsNotFound(err) { + log.Info("Secret not found:" + lrpdb.Spec.LRPDBTlsCrt.Secret.SecretName) + return "", err + } + log.Error(err, "Unable to get the secret.") + return "", err + } + + rsaCertPEM := secret.Data[lrpdb.Spec.LRPDBTlsCrt.Secret.Key] + + err = c.Get(ctx, types.NamespacedName{Name: lrpdb.Spec.LRPDBTlsCat.Secret.SecretName, Namespace: lrpdb.Namespace}, secret) + + if err != nil { + if apierrors.IsNotFound(err) { + log.Info("Secret not found:" + lrpdb.Spec.LRPDBTlsCat.Secret.SecretName) + return "", err + } + log.Error(err, "Unable to get the secret.") + return "", err + } + + caCert := secret.Data[lrpdb.Spec.LRPDBTlsCat.Secret.Key] + /* + r.Recorder.Eventf(lrpdb, corev1.EventTypeWarning, "LRESTINFO", string(rsaKeyPEM)) + r.Recorder.Eventf(lrpdb, corev1.EventTypeWarning, "LRESTINFO", string(rsaCertPEM)) + r.Recorder.Eventf(lrpdb, corev1.EventTypeWarning, "LRESTINFO", string(caCert)) + */ + + certificate, err := tls.X509KeyPair([]byte(rsaCertPEM), []byte(rsaKeyPEM)) + if err != nil { + lrpdb.Status.Msg = "Error tls.X509KeyPair" + return "", err + } + + caCertPool := x509.NewCertPool() + caCertPool.AppendCertsFromPEM(caCert) + /* + tlsConf := &tls.Config{Certificates: []tls.Certificate{certificate}, + RootCAs: caCertPool} + */ + tlsConf := &tls.Config{Certificates: []tls.Certificate{certificate}, + RootCAs: caCertPool, + //MinVersion: tls.VersionTLS12, + CurvePreferences: []tls.CurveID{tls.CurveP521, tls.CurveP384, tls.CurveP256}, + PreferServerCipherSuites: true, + CipherSuites: []uint16{ + tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, + tls.TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, + tls.TLS_RSA_WITH_AES_256_GCM_SHA384, + tls.TLS_RSA_WITH_AES_256_CBC_SHA, + }, + } + + tr := &http.Transport{TLSClientConfig: tlsConf} + + httpclient := &http.Client{Transport: tr} + + log.Info("Issuing REST call", "URL", url, "Action", action) + + // Get Web Server User + //secret := &corev1.Secret{} + err = c.Get(ctx, types.NamespacedName{Name: lrpdb.Spec.WebLrpdbServerUser.Secret.SecretName, Namespace: lrpdb.Namespace}, secret) + if err != nil { + if apierrors.IsNotFound(err) { + log.Info("Secret not found:" + lrpdb.Spec.WebLrpdbServerUser.Secret.SecretName) + return "", err + } + log.Error(err, "Unable to get the secret.") + return "", err + } + webUserEnc := string(secret.Data[lrpdb.Spec.WebLrpdbServerUser.Secret.Key]) + webUserEnc = strings.TrimSpace(webUserEnc) + + err = c.Get(ctx, types.NamespacedName{Name: lrpdb.Spec.LRPDBPriKey.Secret.SecretName, Namespace: lrpdb.Namespace}, secret) + if err != nil { + if apierrors.IsNotFound(err) { + log.Info("Secret not found:" + lrpdb.Spec.LRPDBPriKey.Secret.SecretName) + return "", err + } + log.Error(err, "Unable to get the secret.") + return "", err + } + privKey := string(secret.Data[lrpdb.Spec.LRPDBPriKey.Secret.Key]) + webUser, err := lrcommons.CommonDecryptWithPrivKey(privKey, webUserEnc, req) + + // Get Web Server User Password + secret = &corev1.Secret{} + err = c.Get(ctx, types.NamespacedName{Name: lrpdb.Spec.WebLrpdbServerPwd.Secret.SecretName, Namespace: lrpdb.Namespace}, secret) + if err != nil { + if apierrors.IsNotFound(err) { + log.Info("Secret not found:" + lrpdb.Spec.WebLrpdbServerPwd.Secret.SecretName) + return "", err + } + log.Error(err, "Unable to get the secret.") + return "", err + } + webUserPwdEnc := string(secret.Data[lrpdb.Spec.WebLrpdbServerPwd.Secret.Key]) + webUserPwdEnc = strings.TrimSpace(webUserPwdEnc) + webUserPwd, err := lrcommons.CommonDecryptWithPrivKey(privKey, webUserPwdEnc, req) + + var httpreq *http.Request + if action == "GET" { + httpreq, err = http.NewRequest(action, url, nil) + } else { + jsonValue, _ := json.Marshal(payload) + httpreq, err = http.NewRequest(action, url, bytes.NewBuffer(jsonValue)) + } + + if err != nil { + log.Info("Unable to create HTTP Request for LRPDB : "+lrpdb.Name, "err", err.Error()) + return "", err + } + + httpreq.Header.Add("Accept", "application/json") + httpreq.Header.Add("Content-Type", "application/json") + httpreq.SetBasicAuth(webUser, webUserPwd) + + resp, err := httpclient.Do(httpreq) + if err != nil { + errmsg := err.Error() + log.Error(err, "Failed - Could not connect to LREST Pod", "err", err.Error()) + lrpdb.Status.Msg = "Error: Could not connect to LREST Pod" + e.Eventf(lrpdb, corev1.EventTypeWarning, "LRESTError", errmsg) + return "", err + } + + e.Eventf(lrpdb, corev1.EventTypeWarning, "Done", lrpdb.Spec.CDBResName) + if resp.StatusCode != http.StatusOK { + bb, _ := ioutil.ReadAll(resp.Body) + + if resp.StatusCode == 404 { + lrpdb.Status.ConnString = "" + lrpdb.Status.Msg = lrpdb.Spec.LRPDBName + " not found" + + } else { + if flood_control == false { + lrpdb.Status.Msg = "LREST Error - HTTP Status Code:" + strconv.Itoa(resp.StatusCode) + } + } + + if flood_control == false { + log.Info("LREST Error - HTTP Status Code :"+strconv.Itoa(resp.StatusCode), "Err", string(bb)) + } + + var apiErr LRESTError + json.Unmarshal([]byte(bb), &apiErr) + if flood_control == false { + e.Eventf(lrpdb, corev1.EventTypeWarning, "LRESTError", "Failed: %s", apiErr.Message) + } + fmt.Printf("\n================== APIERR ======================\n") + fmt.Printf("%+v \n", apiErr) + fmt.Printf(string(bb)) + fmt.Printf("URL=%s\n", url) + fmt.Printf("resp.StatusCode=%s\n", strconv.Itoa(resp.StatusCode)) + fmt.Printf("\n================== APIERR ======================\n") + flood_control = true + return "", errors.New("LREST Error") + } + flood_control = false + + defer resp.Body.Close() + + bodyBytes, err := ioutil.ReadAll(resp.Body) + if err != nil { + fmt.Print(err.Error()) + } + respData := string(bodyBytes) + fmt.Print("CALL API return msg.....:") + fmt.Println(string(bodyBytes)) + + var apiResponse restSQLCollection + json.Unmarshal([]byte(bodyBytes), &apiResponse) + fmt.Printf("===> %#v\n", apiResponse) + fmt.Printf("===> %+v\n", apiResponse) + + errFound := false + for _, sqlItem := range apiResponse.Items { + if sqlItem.ErrorDetails != "" { + log.Info("LREST Error - Oracle Error Code :" + strconv.Itoa(sqlItem.ErrorCode)) + if !errFound { + lrpdb.Status.Msg = sqlItem.ErrorDetails + } + e.Eventf(lrpdb, corev1.EventTypeWarning, "OraError", "%s", sqlItem.ErrorDetails) + errFound = true + } + } + + if errFound { + return "", errors.New("Oracle Error") + } + + return respData, nil +} + +func (r *LRPDBReconciler) GetSqlCode(rsp string, sqlcode *int) error { + log := r.Log.WithValues("GetSqlCode", "callAPI(...)") + + var objmap map[string]interface{} + if err := json.Unmarshal([]byte(rsp), &objmap); err != nil { + log.Error(err, "failed to get respData from callAPI", "err", err.Error()) + return err + } + + *sqlcode = int(objmap["sqlcode"].(float64)) + log.Info("sqlcode.......:ora-" + strconv.Itoa(*sqlcode)) + if *sqlcode != 0 { + switch strconv.Itoa(*sqlcode) { + case "65019": /* already open */ + return nil + case "65020": /* already closed */ + return nil + } + err := fmt.Errorf("%v", sqlcode) + return err + } + return nil +} diff --git a/controllers/database/oraclerestdataservice_controller.go b/controllers/database/oraclerestdataservice_controller.go index 783ae70c..053f4a19 100644 --- a/controllers/database/oraclerestdataservice_controller.go +++ b/controllers/database/oraclerestdataservice_controller.go @@ -60,7 +60,7 @@ import ( "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" "sigs.k8s.io/controller-runtime/pkg/log" - dbapi "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + dbapi "github.com/oracle/oracle-database-operator/apis/database/v4" dbcommons "github.com/oracle/oracle-database-operator/commons/database" "github.com/go-logr/logr" @@ -210,6 +210,13 @@ func (r *OracleRestDataServiceReconciler) Reconcile(ctx context.Context, req ctr return result, nil } + // Configure MongoDB + result = r.enableMongoDB(oracleRestDataService, singleInstanceDatabase, sidbReadyPod, ordsReadyPod, ctx, req) + if result.Requeue { + r.Log.Info("Reconcile queued") + return result, nil + } + // Delete Secrets r.deleteSecrets(oracleRestDataService, ctx, req) @@ -263,37 +270,6 @@ func (r *OracleRestDataServiceReconciler) validate(m *dbapi.OracleRestDataServic eventMsgs = append(eventMsgs, "image patching is not available currently") } - // Validate the apex ADMIN password if it is specified - - if !m.Status.ApexConfigured && m.Spec.ApexPassword.SecretName != "" { - apexPasswordSecret := &corev1.Secret{} - err = r.Get(ctx, types.NamespacedName{Name: m.Spec.ApexPassword.SecretName, Namespace: m.Namespace}, apexPasswordSecret) - if err != nil { - if apierrors.IsNotFound(err) { - m.Status.Status = dbcommons.StatusError - eventReason := "Apex Password" - eventMsg := "password secret " + m.Spec.ApexPassword.SecretName + " not found, retrying..." - r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) - r.Log.Info(eventMsg) - return requeueY, nil - } - r.Log.Error(err, err.Error()) - return requeueY, err - } - // APEX_LISTENER , APEX_REST_PUBLIC_USER , APEX_PUBLIC_USER passwords - apexPassword := string(apexPasswordSecret.Data[m.Spec.ApexPassword.SecretKey]) - - // Validate apexPassword - if !dbcommons.ApexPasswordValidator(apexPassword) { - m.Status.Status = dbcommons.StatusError - eventReason := "Apex Password" - eventMsg := "password for Apex is invalid, it should contain at least 6 chars, at least one numeric character, at least one punctuation character (!\"#$%&()``*+,-/:;?_), at least one upper-case alphabet" - r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) - r.Log.Info("APEX password does not conform to the requirements") - return requeueY, nil - } - } - if len(eventMsgs) > 0 { m.Status.Status = dbcommons.StatusError r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, strings.Join(eventMsgs, ",")) @@ -406,7 +382,7 @@ func (r *OracleRestDataServiceReconciler) checkHealthStatus(m *dbapi.OracleRestD return requeueY, readyPod } if readyPod.Name == "" { - m.Status.Status = dbcommons.StatusNotReady + m.Status.Status = dbcommons.StatusPending return requeueY, readyPod } @@ -425,7 +401,7 @@ func (r *OracleRestDataServiceReconciler) checkHealthStatus(m *dbapi.OracleRestD } } - m.Status.Status = dbcommons.StatusNotReady + m.Status.Status = dbcommons.StatusUpdating if strings.Contains(out, "HTTP/1.1 200 OK") || strings.Contains(strings.ToUpper(err.Error()), "HTTP/1.1 200 OK") { if n.Status.Status == dbcommons.StatusReady || n.Status.Status == dbcommons.StatusUpdating || n.Status.Status == dbcommons.StatusPatching { m.Status.Status = dbcommons.StatusReady @@ -447,7 +423,7 @@ func (r *OracleRestDataServiceReconciler) checkHealthStatus(m *dbapi.OracleRestD } } } - if m.Status.Status == dbcommons.StatusNotReady { + if m.Status.Status == dbcommons.StatusUpdating { return requeueY, readyPod } return requeueN, readyPod @@ -480,13 +456,24 @@ func (r *OracleRestDataServiceReconciler) instantiateSVCSpec(m *dbapi.OracleRest }(), }, Spec: corev1.ServiceSpec{ - Ports: []corev1.ServicePort{ - { - Name: "client", - Port: 8443, - Protocol: corev1.ProtocolTCP, - }, - }, + Ports: func() []corev1.ServicePort { + ports := []corev1.ServicePort{ + { + Name: "client", + Port: 8181, + Protocol: corev1.ProtocolTCP, + }, + } + // Conditionally add MongoDB port if enabled + if m.Spec.MongoDbApi { + ports = append(ports, corev1.ServicePort{ + Name: "mongodb", + Port: 27017, + Protocol: corev1.ProtocolTCP, + }) + } + return ports + }(), Selector: map[string]string{ "app": m.Name, }, @@ -509,24 +496,7 @@ func (r *OracleRestDataServiceReconciler) instantiateSVCSpec(m *dbapi.OracleRest // // ############################################################################# func (r *OracleRestDataServiceReconciler) instantiatePodSpec(m *dbapi.OracleRestDataService, - n *dbapi.SingleInstanceDatabase) (*corev1.Pod, *corev1.Secret) { - - initSecret := &corev1.Secret{ - TypeMeta: metav1.TypeMeta{ - Kind: "Secret", - }, - ObjectMeta: metav1.ObjectMeta{ - Name: m.Name, - Namespace: m.Namespace, - Labels: map[string]string{ - "app": m.Name, - }, - }, - Type: corev1.SecretTypeOpaque, - StringData: map[string]string{ - "init-cmd": dbcommons.InitORDSCMD, - }, - } + n *dbapi.SingleInstanceDatabase, req ctrl.Request) *corev1.Pod { pod := &corev1.Pod{ TypeMeta: metav1.TypeMeta{ @@ -578,116 +548,92 @@ func (r *OracleRestDataServiceReconciler) instantiatePodSpec(m *dbapi.OracleRest }, }, { - Name: "init-ords-vol", + Name: "varmount", VolumeSource: corev1.VolumeSource{ - Secret: &corev1.SecretVolumeSource{ - SecretName: m.Name, - Optional: func() *bool { i := true; return &i }(), - Items: []corev1.KeyToPath{{ - Key: "init-cmd", - Path: "init-cmd", - }}, - }, + EmptyDir: &corev1.EmptyDirVolumeSource{}, }, }, }, - InitContainers: []corev1.Container{ - { - Name: "init-permissions", - Image: m.Spec.Image.PullFrom, - Command: []string{"/bin/sh", "-c", fmt.Sprintf("chown %d:%d /opt/oracle/ords/config/ords || true", int(dbcommons.ORACLE_UID), int(dbcommons.DBA_GUID))}, - SecurityContext: &corev1.SecurityContext{ - // User ID 0 means, root user - RunAsUser: func() *int64 { i := int64(0); return &i }(), - }, - VolumeMounts: []corev1.VolumeMount{{ - MountPath: "/opt/oracle/ords/config/ords", - Name: "datamount", - SubPath: strings.ToUpper(n.Spec.Sid) + "_ORDS", - }}, - }, - { + InitContainers: func() []corev1.Container { + initContainers := []corev1.Container{} + if m.Spec.Persistence.Size != "" && m.Spec.Persistence.SetWritePermissions != nil && *m.Spec.Persistence.SetWritePermissions { + initContainers = append(initContainers, corev1.Container{ + Name: "init-permissions", + Image: m.Spec.Image.PullFrom, + Command: []string{"/bin/sh", "-c", fmt.Sprintf("chown %d:%d /etc/ords/config/ || true", int(dbcommons.ORACLE_UID), int(dbcommons.DBA_GUID))}, + SecurityContext: &corev1.SecurityContext{ + // User ID 0 means, root user + RunAsUser: func() *int64 { i := int64(0); return &i }(), + }, + VolumeMounts: []corev1.VolumeMount{{ + MountPath: "/etc/ords/config/", + Name: "datamount", + }}, + }) + } + + initContainers = append(initContainers, corev1.Container{ Name: "init-ords", Image: m.Spec.Image.PullFrom, - Command: []string{"/bin/sh", "/run/secrets/init-cmd"}, - SecurityContext: &corev1.SecurityContext{ - RunAsUser: func() *int64 { i := int64(dbcommons.ORACLE_UID); return &i }(), - RunAsGroup: func() *int64 { i := int64(dbcommons.DBA_GUID); return &i }(), + Command: []string{"/bin/sh"}, + Args: []string{ + "-c", + fmt.Sprintf("while [ ! -f /opt/oracle/variables/%s ]; do sleep 0.5; done", "conn_string.txt"), }, VolumeMounts: []corev1.VolumeMount{ { - MountPath: "/opt/oracle/ords/config/ords", + MountPath: "/etc/ords/config/", Name: "datamount", - SubPath: strings.ToUpper(n.Spec.Sid) + "_ORDS", }, { - MountPath: "/run/secrets/init-cmd", - ReadOnly: true, - Name: "init-ords-vol", - SubPath: "init-cmd", + MountPath: "/opt/oracle/variables/", + Name: "varmount", }, }, - Env: []corev1.EnvVar{ - { - Name: "ORACLE_HOST", - Value: n.Name, - }, - { - Name: "ORACLE_PORT", - Value: "1521", - }, - { - Name: "ORACLE_SERVICE", - Value: func() string { - if m.Spec.OracleService != "" { - return m.Spec.OracleService - } - return n.Spec.Sid - }(), - }, - { - Name: "ORDS_USER", - Value: func() string { - if m.Spec.OrdsUser != "" { - return m.Spec.OrdsUser - } - return "ORDS_PUBLIC_USER" - }(), - }, + }) + return initContainers + }(), + Containers: []corev1.Container{{ + Name: m.Name, + Image: m.Spec.Image.PullFrom, + Ports: func() []corev1.ContainerPort { + ports := []corev1.ContainerPort{ { - Name: "ORDS_PWD", - ValueFrom: &corev1.EnvVarSource{ - SecretKeyRef: &corev1.SecretKeySelector{ - LocalObjectReference: corev1.LocalObjectReference{ - Name: m.Spec.OrdsPassword.SecretName, - }, - Key: m.Spec.OrdsPassword.SecretKey, - }, - }, + ContainerPort: 8181, // Default application port }, - { - Name: "ORACLE_PWD", - ValueFrom: &corev1.EnvVarSource{ - SecretKeyRef: &corev1.SecretKeySelector{ - LocalObjectReference: corev1.LocalObjectReference{ - Name: m.Spec.AdminPassword.SecretName, - }, - Key: m.Spec.AdminPassword.SecretKey, - }, - }, + } + if m.Spec.MongoDbApi { + ports = append(ports, corev1.ContainerPort{ + ContainerPort: 27017, // MongoDB port + }) + } + return ports + }(), + ReadinessProbe: &corev1.Probe{ + ProbeHandler: corev1.ProbeHandler{ + Exec: &corev1.ExecAction{ + Command: []string{"/bin/sh", "-c", dbcommons.ORDSReadinessProbe}, }, }, + InitialDelaySeconds: 20, + TimeoutSeconds: 20, + PeriodSeconds: func() int32 { + if m.Spec.ReadinessCheckPeriod > 0 { + return int32(m.Spec.ReadinessCheckPeriod) + } + return 60 + }(), + }, + VolumeMounts: []corev1.VolumeMount{ + { + MountPath: "/etc/ords/config/", + Name: "datamount", + }, + { + MountPath: "/opt/oracle/variables/", + Name: "varmount", + }, }, - }, - Containers: []corev1.Container{{ - Name: m.Name, - Image: m.Spec.Image.PullFrom, - Ports: []corev1.ContainerPort{{ContainerPort: 8443}}, - VolumeMounts: []corev1.VolumeMount{{ - MountPath: "/opt/oracle/ords/config/ords/", - Name: "datamount", - SubPath: strings.ToUpper(n.Spec.Sid) + "_ORDS", - }}, Env: func() []corev1.EnvVar { // After ORDS is Installed, we DELETE THE OLD ORDS Pod and create new ones ONLY USING BELOW ENV VARIABLES. return []corev1.EnvVar{ @@ -753,9 +699,9 @@ func (r *OracleRestDataServiceReconciler) instantiatePodSpec(m *dbapi.OracleRest } // Set oracleRestDataService instance as the owner and controller - ctrl.SetControllerReference(m, initSecret, r.Scheme) + // ctrl.SetControllerReference(m, initSecret, r.Scheme) ctrl.SetControllerReference(m, pod, r.Scheme) - return pod, initSecret + return pod } //############################################################################# @@ -878,14 +824,21 @@ func (r *OracleRestDataServiceReconciler) createSVC(ctx context.Context, req ctr if lbAddress == "" { lbAddress = svc.Status.LoadBalancer.Ingress[0].IP } - m.Status.DatabaseApiUrl = "https://" + lbAddress + ":" + - fmt.Sprint(svc.Spec.Ports[0].Port) + "/ords/" + n.Status.Pdbname + "/_/db-api/stable/" + m.Status.DatabaseApiUrl = "http://" + lbAddress + ":" + + fmt.Sprint(svc.Spec.Ports[0].Port) + "/ords/" + "{schema-name}" + "/_/db-api/stable/" m.Status.ServiceIP = lbAddress - m.Status.DatabaseActionsUrl = "https://" + lbAddress + ":" + + m.Status.DatabaseActionsUrl = "http://" + lbAddress + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/ords/sql-developer" if m.Status.ApexConfigured { - m.Status.ApxeUrl = "https://" + lbAddress + ":" + - fmt.Sprint(svc.Spec.Ports[0].Port) + "/ords/" + n.Status.Pdbname + "/apex" + m.Status.ApxeUrl = "http://" + lbAddress + ":" + + fmt.Sprint(svc.Spec.Ports[0].Port) + "/ords/apex" + } + if m.Status.MongoDbApi && len(svc.Spec.Ports) > 1 { + m.Status.MongoDbApiAccessUrl = "mongodb://[{user}:{password}@]" + lbAddress + ":" + + fmt.Sprint(svc.Spec.Ports[1].Port) + "/{user}?" + + "authMechanism=PLAIN&authSource=$external&ssl=true&retryWrites=false&loadBalanced=true" + } else { + m.Status.MongoDbApiAccessUrl = "" } } return requeueN @@ -893,13 +846,19 @@ func (r *OracleRestDataServiceReconciler) createSVC(ctx context.Context, req ctr nodeip := dbcommons.GetNodeIp(r, ctx, req) if nodeip != "" { m.Status.ServiceIP = nodeip - m.Status.DatabaseApiUrl = "https://" + nodeip + ":" + fmt.Sprint(svc.Spec.Ports[0].NodePort) + - "/ords/" + n.Status.Pdbname + "/_/db-api/stable/" - m.Status.DatabaseActionsUrl = "https://" + nodeip + ":" + fmt.Sprint(svc.Spec.Ports[0].NodePort) + + m.Status.DatabaseApiUrl = "http://" + nodeip + ":" + fmt.Sprint(svc.Spec.Ports[0].NodePort) + + "/ords/" + "{schema-name}" + "/_/db-api/stable/" + m.Status.DatabaseActionsUrl = "http://" + nodeip + ":" + fmt.Sprint(svc.Spec.Ports[0].NodePort) + "/ords/sql-developer" if m.Status.ApexConfigured { - m.Status.ApxeUrl = "https://" + nodeip + ":" + fmt.Sprint(svc.Spec.Ports[0].NodePort) + "/ords/" + - n.Status.Pdbname + "/apex" + m.Status.ApxeUrl = "http://" + nodeip + ":" + fmt.Sprint(svc.Spec.Ports[0].NodePort) + "/ords/apex" + } + if m.Status.MongoDbApi && len(svc.Spec.Ports) > 1 { + m.Status.MongoDbApiAccessUrl = "mongodb://[{user}:{password}@]" + nodeip + ":" + + fmt.Sprint(svc.Spec.Ports[1].NodePort) + "/{user}?" + + "authMechanism=PLAIN&authSource=$external&ssl=true&retryWrites=false&loadBalanced=true" + } else { + m.Status.MongoDbApiAccessUrl = "" } } return requeueN @@ -941,6 +900,97 @@ func (r *OracleRestDataServiceReconciler) createPVC(ctx context.Context, req ctr return requeueN, nil } +// ############################################################################# +// +// Function for creating connection sting file +// +// ############################################################################# +func (r *OracleRestDataServiceReconciler) createConnectionString(m *dbapi.OracleRestDataService, + n *dbapi.SingleInstanceDatabase, ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + + // Listing all the pods + readyPod, _, availableFinal, _, err := dbcommons.FindPods(r, m.Spec.Image.Version, + m.Spec.Image.PullFrom, m.Name, m.Namespace, ctx, req) + + if err != nil { + r.Log.Error(err, err.Error()) + return requeueY, nil + } + if readyPod.Name != "" { + return requeueN, nil + } + + if len(availableFinal) == 0 { + r.Log.Info("Pods are being created, currently no pods available") + return requeueY, nil + } + + // Iterate through the availableFinal (list of pods) to find out the pod whose status is updated about the init containers + // If no required pod found then requeue the reconcile request + var pod corev1.Pod + var podFound bool + for _, pod = range availableFinal { + // Check if pod status container is updated about init containers + if len(pod.Status.InitContainerStatuses) > 0 { + podFound = true + break + } + } + if !podFound { + r.Log.Info("No pod has its status updated about init containers. Requeueing...") + return requeueY, nil + } + + lastInitContIndex := len(pod.Status.InitContainerStatuses) - 1 + + // If InitContainerStatuses[].Ready is true, it means that the init container is successful + if pod.Status.InitContainerStatuses[lastInitContIndex].Ready { + // Init container named "init-ords" has completed it's execution, hence return and don't requeue + return requeueN, nil + } + + if pod.Status.InitContainerStatuses[lastInitContIndex].State.Running == nil { + // Init container named "init-ords" is not running, so waiting for it to come in running state requeueing the reconcile request + r.Log.Info("Waiting for init-ords to come in running state...") + return requeueY, nil + } + + r.Log.Info("Creating Connection String file...") + + // Querying the secret + r.Log.Info("Querying the database secret ...") + secret := &corev1.Secret{} + err = r.Get(ctx, types.NamespacedName{Name: m.Spec.AdminPassword.SecretName, Namespace: m.Namespace}, secret) + if err != nil { + if apierrors.IsNotFound(err) { + r.Log.Info("Secret not found") + m.Status.Status = dbcommons.StatusError + r.Status().Update(ctx, m) + return requeueY, nil + } + r.Log.Error(err, "Unable to get the secret. Requeueing..") + return requeueY, nil + } + + // Execing into the pods and creating the Connection String + adminPassword := string(secret.Data[m.Spec.AdminPassword.SecretKey]) + + _, err = dbcommons.ExecCommand(r, r.Config, pod.Name, pod.Namespace, "init-ords", + ctx, req, true, "bash", "-c", + fmt.Sprintf("mkdir -p /opt/oracle/variables && echo %[1]s > /opt/oracle/variables/%[2]s", + fmt.Sprintf(dbcommons.DbConnectString, adminPassword, n.Name, n.Status.Pdbname), + "conn_string.txt")) + + if err != nil { + r.Log.Error(err, err.Error()) + r.Log.Error(err, "Failed to create connection string in new "+m.Name+" POD", "pod.Namespace", pod.Namespace, "POD.Name", pod.Name) + return requeueY, nil + } + r.Log.Info("Succesfully Created connection string in new "+m.Name+" POD", "POD.NAME : ", pod.Name) + + return requeueN, nil +} + // ############################################################################# // // Create the requested POD replicas @@ -980,16 +1030,24 @@ func (r *OracleRestDataServiceReconciler) createPods(m *dbapi.OracleRestDataServ } else if replicasFound < replicasReq { // Create New Pods , Name of Pods are generated Randomly for i := replicasFound; i < replicasReq; i++ { - pod, initSecret := r.instantiatePodSpec(m, n) - // Check if init-secret is present - err := r.Get(ctx, types.NamespacedName{Name: m.Name, Namespace: m.Namespace}, &corev1.Secret{}) - if err != nil && apierrors.IsNotFound(err) { - log.Info("Creating a new secret", "name", m.Name) - if err = r.Create(ctx, initSecret); err != nil { - log.Error(err, "Failed to create secret ", "Namespace", initSecret.Namespace, "Name", initSecret.Name) + // Obtain admin password of the referred database + adminPasswordSecret := &corev1.Secret{} + err := r.Get(ctx, types.NamespacedName{Name: n.Spec.AdminPassword.SecretName, Namespace: n.Namespace}, adminPasswordSecret) + if err != nil { + if apierrors.IsNotFound(err) { + m.Status.Status = dbcommons.StatusError + eventReason := "Database Password" + eventMsg := "password secret " + m.Spec.AdminPassword.SecretName + " not found, retrying..." + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) + r.Log.Info(eventMsg) return requeueY } + log.Error(err, err.Error()) + return requeueY } + + pod := r.instantiatePodSpec(m, n, req) + log.Info("Creating a new "+m.Name+" POD", "POD.Namespace", pod.Namespace, "POD.Name", pod.Name) err = r.Create(ctx, pod) if err != nil { @@ -1024,6 +1082,17 @@ func (r *OracleRestDataServiceReconciler) createPods(m *dbapi.OracleRestDataServ } } + // Creating conn string in pods + result, err := r.createConnectionString(m, n, ctx, req) + + if err != nil { + return requeueY + } + if result.Requeue { + log.Info("Requeued at connection string creation") + return requeueY + } + m.Status.Replicas = m.Spec.Replicas return requeueN @@ -1241,97 +1310,14 @@ func (r *OracleRestDataServiceReconciler) cleanupOracleRestDataService(req ctrl. // ############################################################################# func (r *OracleRestDataServiceReconciler) configureApex(m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase, sidbReadyPod corev1.Pod, ordsReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) ctrl.Result { - log := r.Log.WithValues("configureApex", req.NamespacedName) + log := r.Log.WithValues("verifyApex", req.NamespacedName) - if m.Spec.ApexPassword.SecretName == "" { - m.Status.ApexConfigured = false - return requeueN - } if m.Status.ApexConfigured { return requeueN } - apexPasswordSecret := &corev1.Secret{} - err := r.Get(ctx, types.NamespacedName{Name: m.Spec.ApexPassword.SecretName, Namespace: m.Namespace}, apexPasswordSecret) - if err != nil { - if apierrors.IsNotFound(err) { - m.Status.Status = dbcommons.StatusError - eventReason := "Apex Password" - eventMsg := "password secret " + m.Spec.ApexPassword.SecretName + " not found, retrying..." - r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) - r.Log.Info(eventMsg) - return requeueY - } - log.Error(err, err.Error()) - return requeueY - } - // APEX_LISTENER , APEX_REST_PUBLIC_USER , APEX_PUBLIC_USER passwords - apexPassword := string(apexPasswordSecret.Data[m.Spec.ApexPassword.SecretKey]) - - if !n.Status.ApexInstalled { - m.Status.Status = dbcommons.StatusUpdating - result := r.installApex(m, n, ordsReadyPod, apexPassword, ctx, req) - if result.Requeue { - log.Info("Reconcile requeued because apex installation failed") - return result - } - } else { - // Alter Apex Users - log.Info("Alter APEX Users") - _, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", - ctx, req, true, "bash", "-c", fmt.Sprintf("echo -e \"%s\" | %s", - fmt.Sprintf(dbcommons.AlterApexUsers, apexPassword, n.Spec.Pdbname), dbcommons.SQLPlusCLI)) - if err != nil { - log.Error(err, err.Error()) - return requeueY - } - } - - // Set Apex users in apex_rt,apex_al,apex files - out, err := dbcommons.ExecCommand(r, r.Config, ordsReadyPod.Name, ordsReadyPod.Namespace, "", ctx, req, true, "bash", "-c", - fmt.Sprintf(dbcommons.SetApexUsers, apexPassword)) - log.Info("SetApexUsers Output: \n" + out) - if strings.Contains(strings.ToUpper(out), "ERROR") { - return requeueY - } - if err != nil { - log.Info(err.Error()) - if strings.Contains(strings.ToUpper(err.Error()), "ERROR") { - return requeueY - } - } - - // ORDS needs to be restarted to configure APEX - r.Log.Info("Restarting ORDS Pod to complete APEX configuration: " + ordsReadyPod.Name) - var gracePeriodSeconds int64 = 0 - policy := metav1.DeletePropagationForeground - err = r.Delete(ctx, &ordsReadyPod, &client.DeleteOptions{ - GracePeriodSeconds: &gracePeriodSeconds, PropagationPolicy: &policy}) - if err != nil { - r.Log.Error(err, err.Error()) - } - - m.Status.ApexConfigured = true - r.Status().Update(ctx, m) - eventReason := "Apex Configuration" - eventMsg := "configuration of Apex completed!" - r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) - log.Info(eventMsg) - - // Cannot return requeue as the secrets will be deleted if keepSecert is false, which cause problem in pod restart - return requeueY -} - -// ############################################################################# -// -// Install APEX in SIDB -// -// ############################################################################# -func (r *OracleRestDataServiceReconciler) installApex(m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase, - ordsReadyPod corev1.Pod, apexPassword string, ctx context.Context, req ctrl.Request) ctrl.Result { - log := r.Log.WithValues("installApex", req.NamespacedName) - // Obtain admin password of the referred database + adminPasswordSecret := &corev1.Secret{} err := r.Get(ctx, types.NamespacedName{Name: m.Spec.AdminPassword.SecretName, Namespace: m.Namespace}, adminPasswordSecret) if err != nil { @@ -1348,23 +1334,8 @@ func (r *OracleRestDataServiceReconciler) installApex(m *dbapi.OracleRestDataSer } sidbPassword := string(adminPasswordSecret.Data[m.Spec.AdminPassword.SecretKey]) - // Status Updation - m.Status.Status = dbcommons.StatusUpdating - r.Status().Update(ctx, m) - eventReason := "Apex Installation" - eventMsg := "performing install of Apex in database " + m.Spec.DatabaseRef - r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) - - //Install Apex in SIDB ready pod - out, err := dbcommons.ExecCommand(r, r.Config, ordsReadyPod.Name, ordsReadyPod.Namespace, "", ctx, req, true, "bash", "-c", - fmt.Sprintf(dbcommons.InstallApexInContainer, apexPassword, sidbPassword, n.Status.Pdbname)) - if err != nil { - log.Info(err.Error()) - } - log.Info("Apex installation output : \n" + out) - // Checking if Apex is installed successfully or not - out, err = dbcommons.ExecCommand(r, r.Config, ordsReadyPod.Name, ordsReadyPod.Namespace, "", ctx, req, true, "bash", "-c", + out, err := dbcommons.ExecCommand(r, r.Config, ordsReadyPod.Name, ordsReadyPod.Namespace, "", ctx, req, true, "bash", "-c", fmt.Sprintf(dbcommons.IsApexInstalled, sidbPassword, n.Status.Pdbname)) if err != nil { log.Error(err, err.Error()) @@ -1374,19 +1345,22 @@ func (r *OracleRestDataServiceReconciler) installApex(m *dbapi.OracleRestDataSer apexInstalled := "APEXVERSION:" if !strings.Contains(out, apexInstalled) { - eventReason = "Apex Installation" - eventMsg = "Unable to determine Apex version, retrying install..." + eventReason := "Apex Verification" + eventMsg := "Unable to determine Apex version, retrying..." r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) return requeueY } m.Status.Status = dbcommons.StatusReady - eventReason = "Apex Installation" + eventReason := "Apex Verification" outArr := strings.Split(out, apexInstalled) - eventMsg = "installation of Apex " + strings.TrimSpace(outArr[len(outArr)-1]) + " completed" + eventMsg := "Verification of Apex " + strings.TrimSpace(outArr[len(outArr)-1]) + " completed" r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) n.Status.ApexInstalled = true + m.Status.ApexConfigured = true r.Status().Update(ctx, n) + r.Status().Update(ctx, m) + return requeueN } @@ -1423,20 +1397,71 @@ func (r *OracleRestDataServiceReconciler) deleteSecrets(m *dbapi.OracleRestDataS } } } +} - if !*m.Spec.ApexPassword.KeepSecret { - // Fetch apexPassword Secret - apexPasswordSecret := &corev1.Secret{} - err := r.Get(ctx, types.NamespacedName{Name: m.Spec.ApexPassword.SecretName, Namespace: m.Namespace}, apexPasswordSecret) - if err == nil { - //Delete APEX Password Secret . - err := r.Delete(ctx, apexPasswordSecret, &client.DeleteOptions{}) - if err == nil { - log.Info("APEX password secret deleted : " + apexPasswordSecret.Name) +// ############################################################################# +// +// Enable MongoDB API Support +// +// ############################################################################# +func (r *OracleRestDataServiceReconciler) enableMongoDB(m *dbapi.OracleRestDataService, n *dbapi.SingleInstanceDatabase, + sidbReadyPod corev1.Pod, ordsReadyPod corev1.Pod, ctx context.Context, req ctrl.Request) ctrl.Result { + log := r.Log.WithValues("enableMongoDB", req.NamespacedName) + + if (m.Spec.MongoDbApi && !m.Status.MongoDbApi) || // setting MongoDbApi to true + (!m.Spec.MongoDbApi && m.Status.MongoDbApi) { // setting MongoDbApi to false + m.Status.Status = dbcommons.StatusUpdating + + out, err := dbcommons.ExecCommand(r, r.Config, ordsReadyPod.Name, ordsReadyPod.Namespace, "", ctx, req, true, "bash", "-c", + fmt.Sprintf(dbcommons.ConfigMongoDb, strconv.FormatBool(m.Spec.MongoDbApi))) + log.Info("configMongoDB Output: \n" + out) + + if strings.Contains(strings.ToUpper(out), "ERROR") { + return requeueY + } + if err != nil { + log.Info(err.Error()) + if strings.Contains(strings.ToUpper(err.Error()), "ERROR") { + return requeueY } } + + m.Status.MongoDbApi = m.Spec.MongoDbApi + m.Status.Status = dbcommons.StatusReady + r.Status().Update(ctx, m) + eventReason := "MongoDB-API Config" + eventMsg := "configuration of MongoDb API completed!" + r.Recorder.Eventf(m, corev1.EventTypeNormal, eventReason, eventMsg) + log.Info(eventMsg) + + // ORDS service is resatrted + r.Log.Info("Restarting ORDS Service : " + m.Name) + svc := &corev1.Service{ + ObjectMeta: metav1.ObjectMeta{Name: m.Name, Namespace: m.Namespace}, + } + var gracePeriodSeconds int64 = 0 + policy := metav1.DeletePropagationForeground + err = r.Delete(ctx, svc, &client.DeleteOptions{ + GracePeriodSeconds: &gracePeriodSeconds, PropagationPolicy: &policy}) + if err != nil { + r.Log.Error(err, "Failed to delete ORDS service", "Service Name", m.Name) + return requeueY + } + + // ORDS needs to be restarted to configure MongoDB API + r.Log.Info("Restarting ORDS Pod after configuring MongoDb API : " + ordsReadyPod.Name) + err = r.Delete(ctx, &ordsReadyPod, &client.DeleteOptions{ + GracePeriodSeconds: &gracePeriodSeconds, PropagationPolicy: &policy}) + if err != nil { + r.Log.Error(err, err.Error()) + } + return requeueY + + } else { + log.Info("MongoDB Already Configured") } + return requeueN } // ############################################################################# diff --git a/controllers/database/ordssrvs_controller.go b/controllers/database/ordssrvs_controller.go new file mode 100644 index 00000000..14c7f46e --- /dev/null +++ b/controllers/database/ordssrvs_controller.go @@ -0,0 +1,1116 @@ +/* +** Copyright (c) 2024 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package controllers + +import ( + "context" + "crypto/rsa" + "crypto/sha256" + "crypto/x509" + "encoding/base64" + "encoding/hex" + "encoding/json" + "encoding/pem" + "errors" + "fmt" + "reflect" + "strconv" + "strings" + "time" + + appsv1 "k8s.io/api/apps/v1" + corev1 "k8s.io/api/core/v1" + "k8s.io/apimachinery/pkg/api/equality" + apierrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/api/meta" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/types" + "k8s.io/apimachinery/pkg/util/intstr" + "k8s.io/client-go/tools/record" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/log" + + // dbapi "example.com/oracle-ords-operator/api/v1" + dbapi "github.com/oracle/oracle-database-operator/apis/database/v4" +) + +// Definitions of Standards +const ( + ordsSABase = "/opt/oracle/sa" + serviceHTTPPortName = "svc-http-port" + serviceHTTPSPortName = "svc-https-port" + serviceMongoPortName = "svc-mongo-port" + targetHTTPPortName = "pod-http-port" + targetHTTPSPortName = "pod-https-port" + targetMongoPortName = "pod-mongo-port" + globalConfigMapName = "settings-global" + poolConfigPreName = "settings-" // Append PoolName + controllerLabelKey = "oracle.com/ords-operator-filter" + controllerLabelVal = "oracle-database-operator" + specHashLabel = "oracle.com/ords-operator-spec-hash" +) + +// Definitions to manage status conditions +const ( + // typeAvailableORDS represents the status of the Workload reconciliation + typeAvailableORDS = "Available" + // typeUnsyncedORDS represents the status used when the configuration has changed but the Workload has not been restarted. + typeUnsyncedORDS = "Unsynced" +) + +// Trigger a restart of Pods on Config Changes +var RestartPods bool = false + +// OrdsSrvsReconciler reconciles a OrdsSrvs object +type OrdsSrvsReconciler struct { + client.Client + Scheme *runtime.Scheme + Recorder record.EventRecorder +} + +//+kubebuilder:rbac:groups=database.oracle.com,resources=ordssrvs,verbs=get;list;watch;create;update;patch;delete +//+kubebuilder:rbac:groups=database.oracle.com,resources=ordssrvs/status,verbs=get;update;patch +//+kubebuilder:rbac:groups=database.oracle.com,resources=ordssrvs/finalizers,verbs=update +//+kubebuilder:rbac:groups=core,resources=events,verbs=create;patch +//+kubebuilder:rbac:groups=core,resources=configmaps,verbs=get;list;watch;create;update;patch;delete +//+kubebuilder:rbac:groups=core,resources=configmaps/status,verbs=get;update;patch +//+kubebuilder:rbac:groups=core,resources=secrets,verbs=get;list;watch +//+kubebuilder:rbac:groups=core,resources=secrets/status,verbs=get +//+kubebuilder:rbac:groups=core,resources=services,verbs=get;list;watch;create;update;patch;delete +//+kubebuilder:rbac:groups=core,resources=services/status,verbs=get;update;patch +//+kubebuilder:rbac:groups=apps,resources=deployments,verbs=get;list;watch;create;update;patch;delete +//+kubebuilder:rbac:groups=core,resources=deployments/status,verbs=get;update;patch +//+kubebuilder:rbac:groups=apps,resources=daemonsets,verbs=get;list;watch;create;update;patch;delete +//+kubebuilder:rbac:groups=core,resources=daemonsets/status,verbs=get;update;patch +//+kubebuilder:rbac:groups=apps,resources=statefulsets,verbs=get;list;watch;create;update;patch;delete +//+kubebuilder:rbac:groups=core,resources=statefulsets/status,verbs=get;update;patch + +// SetupWithManager sets up the controller with the Manager. +func (r *OrdsSrvsReconciler) SetupWithManager(mgr ctrl.Manager) error { + return ctrl.NewControllerManagedBy(mgr). + For(&dbapi.OrdsSrvs{}). + Owns(&corev1.ConfigMap{}). + Owns(&corev1.Secret{}). + Owns(&appsv1.Deployment{}). + Owns(&appsv1.StatefulSet{}). + Owns(&appsv1.DaemonSet{}). + Owns(&corev1.Service{}). + Complete(r) +} + +func (r *OrdsSrvsReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + logr := log.FromContext(ctx) + ords := &dbapi.OrdsSrvs{} + + // Check if resource exists or was deleted + if err := r.Get(ctx, req.NamespacedName, ords); err != nil { + if apierrors.IsNotFound(err) { + logr.Info("Resource deleted") + return ctrl.Result{}, nil + } + logr.Error(err, "Error retrieving resource") + return ctrl.Result{Requeue: true, RequeueAfter: time.Minute}, err + } + + // Set the status as Unknown when no status are available + if ords.Status.Conditions == nil || len(ords.Status.Conditions) == 0 { + condition := metav1.Condition{Type: typeUnsyncedORDS, Status: metav1.ConditionUnknown, Reason: "Reconciling", Message: "Starting reconciliation"} + if err := r.SetStatus(ctx, req, ords, condition); err != nil { + return ctrl.Result{}, err + } + } + + // ConfigMap - Init Script + if err := r.ConfigMapReconcile(ctx, ords, ords.Name+"-"+"init-script", 0); err != nil { + logr.Error(err, "Error in ConfigMapReconcile (init-script)") + return ctrl.Result{}, err + } + + // ConfigMap - Global Settings + if err := r.ConfigMapReconcile(ctx, ords, ords.Name+"-"+globalConfigMapName, 0); err != nil { + logr.Error(err, "Error in ConfigMapReconcile (Global)") + return ctrl.Result{}, err + } + + // ConfigMap - Pool Settings + definedPools := make(map[string]bool) + for i := 0; i < len(ords.Spec.PoolSettings); i++ { + poolName := strings.ToLower(ords.Spec.PoolSettings[i].PoolName) + poolConfigMapName := ords.Name + "-" + poolConfigPreName + poolName + if definedPools[poolConfigMapName] { + return ctrl.Result{}, errors.New("poolName: " + poolName + " is not unique") + } + definedPools[poolConfigMapName] = true + if err := r.ConfigMapReconcile(ctx, ords, poolConfigMapName, i); err != nil { + logr.Error(err, "Error in ConfigMapReconcile (Pools)") + return ctrl.Result{}, err + } + } + if err := r.ConfigMapDelete(ctx, req, ords, definedPools); err != nil { + logr.Error(err, "Error in ConfigMapDelete (Pools)") + return ctrl.Result{}, err + } + if err := r.Get(ctx, req.NamespacedName, ords); err != nil { + logr.Error(err, "Failed to re-fetch") + return ctrl.Result{}, err + } + + // // Secrets - Pool Settings + // for i := 0; i < len(ords.Spec.PoolSettings); i++ { + // if err := r.SecretsReconcile(ctx, ords, i); err != nil { + // logr.Error(err, "Error in SecretsReconcile (Pools)") + // return ctrl.Result{}, err + // } + // } + + // Set the Type as Unsynced when a pod restart is required + if RestartPods { + condition := metav1.Condition{Type: typeUnsyncedORDS, Status: metav1.ConditionTrue, Reason: "Unsynced", Message: "Configurations have changed"} + if err := r.SetStatus(ctx, req, ords, condition); err != nil { + return ctrl.Result{}, err + } + } + + // Workloads + if err := r.WorkloadReconcile(ctx, req, ords, ords.Spec.WorkloadType); err != nil { + logr.Error(err, "Error in WorkloadReconcile") + return ctrl.Result{}, err + } + if err := r.WorkloadDelete(ctx, req, ords, ords.Spec.WorkloadType); err != nil { + logr.Error(err, "Error in WorkloadDelete") + return ctrl.Result{}, err + } + if err := r.Get(ctx, req.NamespacedName, ords); err != nil { + logr.Error(err, "Failed to re-fetch") + return ctrl.Result{}, err + } + + // Service + if err := r.ServiceReconcile(ctx, ords); err != nil { + logr.Error(err, "Error in ServiceReconcile") + return ctrl.Result{}, err + } + + // Set the Type as Available when a pod restart is not required + if !RestartPods { + condition := metav1.Condition{Type: typeAvailableORDS, Status: metav1.ConditionTrue, Reason: "Available", Message: "Workload in Sync"} + if err := r.SetStatus(ctx, req, ords, condition); err != nil { + return ctrl.Result{}, err + } + } + if err := r.Get(ctx, req.NamespacedName, ords); err != nil { + logr.Error(err, "Failed to re-fetch") + return ctrl.Result{}, err + } + + return ctrl.Result{}, nil +} + +/************************************************ + * Status + *************************************************/ +func (r *OrdsSrvsReconciler) SetStatus(ctx context.Context, req ctrl.Request, ords *dbapi.OrdsSrvs, statusCondition metav1.Condition) error { + logr := log.FromContext(ctx).WithName("SetStatus") + + // Fetch before Status Update + if err := r.Get(ctx, req.NamespacedName, ords); err != nil { + logr.Error(err, "Failed to re-fetch") + return err + } + var readyWorkload int32 + var desiredWorkload int32 + switch ords.Spec.WorkloadType { + //nolint:goconst + case "StatefulSet": + workload := &appsv1.StatefulSet{} + if err := r.Get(ctx, types.NamespacedName{Name: ords.Name, Namespace: ords.Namespace}, workload); err != nil { + logr.Info("StatefulSet not ready") + } + readyWorkload = workload.Status.ReadyReplicas + desiredWorkload = workload.Status.Replicas + //nolint:goconst + case "DaemonSet": + workload := &appsv1.DaemonSet{} + if err := r.Get(ctx, types.NamespacedName{Name: ords.Name, Namespace: ords.Namespace}, workload); err != nil { + logr.Info("DaemonSet not ready") + } + readyWorkload = workload.Status.NumberReady + desiredWorkload = workload.Status.DesiredNumberScheduled + default: + workload := &appsv1.Deployment{} + if err := r.Get(ctx, types.NamespacedName{Name: ords.Name, Namespace: ords.Namespace}, workload); err != nil { + logr.Info("Deployment not ready") + } + readyWorkload = workload.Status.ReadyReplicas + desiredWorkload = workload.Status.Replicas + } + + var workloadStatus string + if readyWorkload == 0 { + workloadStatus = "Preparing" + } else if readyWorkload == desiredWorkload { + workloadStatus = "Healthy" + ords.Status.OrdsInstalled = true + } else { + workloadStatus = "Progressing" + } + + mongoPort := int32(0) + if ords.Spec.GlobalSettings.MongoEnabled { + mongoPort = *ords.Spec.GlobalSettings.MongoPort + } + + meta.SetStatusCondition(&ords.Status.Conditions, statusCondition) + ords.Status.Status = workloadStatus + ords.Status.WorkloadType = ords.Spec.WorkloadType + ords.Status.ORDSVersion = strings.Split(ords.Spec.Image, ":")[1] + ords.Status.HTTPPort = ords.Spec.GlobalSettings.StandaloneHTTPPort + ords.Status.HTTPSPort = ords.Spec.GlobalSettings.StandaloneHTTPSPort + ords.Status.MongoPort = mongoPort + ords.Status.RestartRequired = RestartPods + if err := r.Status().Update(ctx, ords); err != nil { + logr.Error(err, "Failed to update Status") + return err + } + return nil +} + +/************************************************ + * ConfigMaps + *************************************************/ +func (r *OrdsSrvsReconciler) ConfigMapReconcile(ctx context.Context, ords *dbapi.OrdsSrvs, configMapName string, poolIndex int) (err error) { + logr := log.FromContext(ctx).WithName("ConfigMapReconcile") + desiredConfigMap := r.ConfigMapDefine(ctx, ords, configMapName, poolIndex) + + // Create if ConfigMap not found + definedConfigMap := &corev1.ConfigMap{} + if err = r.Get(ctx, types.NamespacedName{Name: configMapName, Namespace: ords.Namespace}, definedConfigMap); err != nil { + if apierrors.IsNotFound(err) { + if err := r.Create(ctx, desiredConfigMap); err != nil { + return err + } + logr.Info("Created: " + configMapName) + RestartPods = true + r.Recorder.Eventf(ords, corev1.EventTypeNormal, "Create", "ConfigMap %s Created", configMapName) + // Requery for comparison + if err := r.Get(ctx, types.NamespacedName{Name: configMapName, Namespace: ords.Namespace}, definedConfigMap); err != nil { + return err + } + } else { + return err + } + } + if !equality.Semantic.DeepEqual(definedConfigMap.Data, desiredConfigMap.Data) { + if err = r.Update(ctx, desiredConfigMap); err != nil { + return err + } + logr.Info("Updated: " + configMapName) + RestartPods = true + r.Recorder.Eventf(ords, corev1.EventTypeNormal, "Update", "ConfigMap %s Updated", configMapName) + } + return nil +} + +/************************************************ + * Secrets - TODO (Watch and set RestartPods) + *************************************************/ +// func (r *OrdsSrvsReconciler) SecretsReconcile(ctx context.Context, ords *dbapi.OrdsSrvs, poolIndex int) (err error) { +// logr := log.FromContext(ctx).WithName("SecretsReconcile") +// definedSecret := &corev1.Secret{} + +// // Want to set ownership on the Secret for watching; also detects if TNS_ADMIN is needed. +// if ords.Spec.PoolSettings[i].DBSecret != nil { +// } +// if ords.Spec.PoolSettings[i].DBAdminUserSecret != nil { +// } +// if ords.Spec.PoolSettings[i].DBCDBAdminUserSecret != nil { +// } +// if ords.Spec.PoolSettings[i].TNSAdminSecret != nil { +// } +// if ords.Spec.PoolSettings[i].DBWalletSecret != nil { +// } + +// if ords.Spec.PoolSettings[i].TNSAdminSecret != nil { +// tnsSecretName := ords.Spec.PoolSettings[i].TNSAdminSecret.SecretName +// definedSecret := &corev1.Secret{} +// if err = r.Get(ctx, types.NamespacedName{Name: tnsSecretName, Namespace: ords.Namespace}, definedSecret); err != nil { +// ojdbcPropertiesData, ok := secret.Data["ojdbc.properties"] +// if ok { +// if err = r.Update(ctx, desiredConfigMap); err != nil { +// return err +// } +// } +// } +// } + +// return nil +// } + +/************************************************ + * Workloads + *************************************************/ +func (r *OrdsSrvsReconciler) WorkloadReconcile(ctx context.Context, req ctrl.Request, ords *dbapi.OrdsSrvs, kind string) (err error) { + logr := log.FromContext(ctx).WithName("WorkloadReconcile") + objectMeta := objectMetaDefine(ords, ords.Name) + selector := selectorDefine(ords) + template := r.podTemplateSpecDefine(ords, ctx, req) + + var desiredWorkload client.Object + var desiredSpecHash string + var definedSpecHash string + + switch kind { + case "StatefulSet": + desiredWorkload = &appsv1.StatefulSet{ + ObjectMeta: objectMeta, + Spec: appsv1.StatefulSetSpec{ + Replicas: &ords.Spec.Replicas, + Selector: &selector, + Template: template, + }, + } + desiredSpecHash = generateSpecHash(desiredWorkload.(*appsv1.StatefulSet).Spec) + desiredWorkload.(*appsv1.StatefulSet).ObjectMeta.Labels[specHashLabel] = desiredSpecHash + case "DaemonSet": + desiredWorkload = &appsv1.DaemonSet{ + ObjectMeta: objectMeta, + Spec: appsv1.DaemonSetSpec{ + Selector: &selector, + Template: template, + }, + } + desiredSpecHash = generateSpecHash(desiredWorkload.(*appsv1.DaemonSet).Spec) + desiredWorkload.(*appsv1.DaemonSet).ObjectMeta.Labels[specHashLabel] = desiredSpecHash + default: + desiredWorkload = &appsv1.Deployment{ + ObjectMeta: objectMeta, + Spec: appsv1.DeploymentSpec{ + Replicas: &ords.Spec.Replicas, + Selector: &selector, + Template: template, + }, + } + desiredSpecHash = generateSpecHash(desiredWorkload.(*appsv1.Deployment).Spec) + desiredWorkload.(*appsv1.Deployment).ObjectMeta.Labels[specHashLabel] = desiredSpecHash + } + + if err := ctrl.SetControllerReference(ords, desiredWorkload, r.Scheme); err != nil { + return err + } + + definedWorkload := reflect.New(reflect.TypeOf(desiredWorkload).Elem()).Interface().(client.Object) + if err = r.Get(ctx, types.NamespacedName{Name: ords.Name, Namespace: ords.Namespace}, definedWorkload); err != nil { + if apierrors.IsNotFound(err) { + if err := r.Create(ctx, desiredWorkload); err != nil { + condition := metav1.Condition{ + Type: typeAvailableORDS, + Status: metav1.ConditionFalse, + Reason: "Reconciling", + Message: fmt.Sprintf("Failed to create %s for the custom resource (%s): (%s)", kind, ords.Name, err), + } + if statusErr := r.SetStatus(ctx, req, ords, condition); statusErr != nil { + return statusErr + } + return err + } + logr.Info("Created: " + kind) + RestartPods = false + r.Recorder.Eventf(ords, corev1.EventTypeNormal, "Create", "Created %s", kind) + + return nil + } else { + return err + } + } + + definedLabelsField := reflect.ValueOf(definedWorkload).Elem().FieldByName("ObjectMeta").FieldByName("Labels") + if definedLabelsField.IsValid() { + specHashValue := definedLabelsField.MapIndex(reflect.ValueOf(specHashLabel)) + if specHashValue.IsValid() { + definedSpecHash = specHashValue.Interface().(string) + } else { + return err + } + } + + if desiredSpecHash != definedSpecHash { + logr.Info("Syncing Workload " + kind + " with new configuration") + if err := r.Client.Update(ctx, desiredWorkload); err != nil { + return err + } + RestartPods = true + r.Recorder.Eventf(ords, corev1.EventTypeNormal, "Update", "Updated %s", kind) + } + + if RestartPods && ords.Spec.ForceRestart { + logr.Info("Cycling: " + kind) + labelsField := reflect.ValueOf(desiredWorkload).Elem().FieldByName("Spec").FieldByName("Template").FieldByName("ObjectMeta").FieldByName("Labels") + if labelsField.IsValid() { + labels := labelsField.Interface().(map[string]string) + labels["configMapChanged"] = time.Now().Format("20060102T150405Z") + labelsField.Set(reflect.ValueOf(labels)) + if err := r.Update(ctx, desiredWorkload); err != nil { + return err + } + r.Recorder.Eventf(ords, corev1.EventTypeNormal, "Restart", "Restarted %s", kind) + RestartPods = false + } + } + + return nil +} + +// Service +func (r *OrdsSrvsReconciler) ServiceReconcile(ctx context.Context, ords *dbapi.OrdsSrvs) (err error) { + logr := log.FromContext(ctx).WithName("ServiceReconcile") + + HTTPport := *ords.Spec.GlobalSettings.StandaloneHTTPPort + HTTPSport := *ords.Spec.GlobalSettings.StandaloneHTTPSPort + MongoPort := *ords.Spec.GlobalSettings.MongoPort + + desiredService := r.ServiceDefine(ctx, ords, HTTPport, HTTPSport, MongoPort) + + definedService := &corev1.Service{} + if err = r.Get(ctx, types.NamespacedName{Name: ords.Name, Namespace: ords.Namespace}, definedService); err != nil { + if apierrors.IsNotFound(err) { + if err := r.Create(ctx, desiredService); err != nil { + return err + } + logr.Info("Created: Service") + r.Recorder.Eventf(ords, corev1.EventTypeNormal, "Create", "Service %s Created", ords.Name) + // Requery for comparison + if err := r.Get(ctx, types.NamespacedName{Name: ords.Name, Namespace: ords.Namespace}, definedService); err != nil { + return err + } + } else { + return err + } + } + + deisredPortCount := len(desiredService.Spec.Ports) + definedPortCount := len(definedService.Spec.Ports) + + if deisredPortCount != definedPortCount { + if err := r.Update(ctx, desiredService); err != nil { + return err + } + } + + for _, existingPort := range definedService.Spec.Ports { + if existingPort.Name == serviceHTTPPortName { + if existingPort.Port != HTTPport { + if err := r.Update(ctx, desiredService); err != nil { + return err + } + logr.Info("Updated HTTP Service Port: " + existingPort.Name) + r.Recorder.Eventf(ords, corev1.EventTypeNormal, "Update", "Service HTTP Port %s Updated", existingPort.Name) + } + } + if existingPort.Name == serviceHTTPSPortName { + if existingPort.Port != HTTPSport { + if err := r.Update(ctx, desiredService); err != nil { + return err + } + logr.Info("Updated HTTPS Service Port: " + existingPort.Name) + r.Recorder.Eventf(ords, corev1.EventTypeNormal, "Update", "Service HTTPS Port %s Updated", existingPort.Name) + } + } + if existingPort.Name == serviceMongoPortName { + if existingPort.Port != MongoPort { + if err := r.Update(ctx, desiredService); err != nil { + return err + } + logr.Info("Updated Mongo Service Port: " + existingPort.Name) + r.Recorder.Eventf(ords, corev1.EventTypeNormal, "Update", "Service Mongo Port %s Updated", existingPort.Name) + } + } + } + return nil +} + +/* +************************************************ + - Definers + +************************************************* +*/ +func objectMetaDefine(ords *dbapi.OrdsSrvs, name string) metav1.ObjectMeta { + labels := getLabels(ords.Name) + return metav1.ObjectMeta{ + Name: name, + Namespace: ords.Namespace, + Labels: labels, + } +} + +func selectorDefine(ords *dbapi.OrdsSrvs) metav1.LabelSelector { + labels := getLabels(ords.Name) + return metav1.LabelSelector{ + MatchLabels: labels, + } +} + +func (r *OrdsSrvsReconciler) podTemplateSpecDefine(ords *dbapi.OrdsSrvs, ctx context.Context, req ctrl.Request) corev1.PodTemplateSpec { + labels := getLabels(ords.Name) + specVolumes, specVolumeMounts := VolumesDefine(ords) + + envPorts := []corev1.ContainerPort{ + { + ContainerPort: *ords.Spec.GlobalSettings.StandaloneHTTPPort, + Name: targetHTTPPortName, + }, + { + ContainerPort: *ords.Spec.GlobalSettings.StandaloneHTTPSPort, + Name: targetHTTPSPortName, + }, + } + + if ords.Spec.GlobalSettings.MongoEnabled { + mongoPort := corev1.ContainerPort{ + ContainerPort: *ords.Spec.GlobalSettings.MongoPort, + Name: targetMongoPortName, + } + envPorts = append(envPorts, mongoPort) + } + + // Environment From Source + podSpecTemplate := + corev1.PodTemplateSpec{ + ObjectMeta: metav1.ObjectMeta{ + Labels: labels, + }, + Spec: corev1.PodSpec{ + Volumes: specVolumes, + SecurityContext: &corev1.PodSecurityContext{ + RunAsNonRoot: &[]bool{true}[0], + FSGroup: &[]int64{54321}[0], + SeccompProfile: &corev1.SeccompProfile{ + Type: corev1.SeccompProfileTypeRuntimeDefault, + }, + }, + InitContainers: []corev1.Container{{ + Image: ords.Spec.Image, + Name: ords.Name + "-init", + ImagePullPolicy: corev1.PullIfNotPresent, + SecurityContext: securityContextDefine(), + Command: []string{"sh", "-c", ordsSABase + "/bin/init_script.sh"}, + Env: r.envDefine(ords, true, ctx), + VolumeMounts: specVolumeMounts, + }}, + Containers: []corev1.Container{{ + Image: ords.Spec.Image, + Name: ords.Name, + ImagePullPolicy: corev1.PullIfNotPresent, + SecurityContext: securityContextDefine(), + Ports: envPorts, + Command: []string{"/bin/bash", "-c", "ords --config $ORDS_CONFIG serve --apex-images /opt/oracle/apex/$APEX_VER/images --debug"}, + Env: r.envDefine(ords, false, ctx), + VolumeMounts: specVolumeMounts, + }}}, + } + + return podSpecTemplate +} + +// Volumes +func VolumesDefine(ords *dbapi.OrdsSrvs) ([]corev1.Volume, []corev1.VolumeMount) { + // Initialize the slice to hold specifications + var volumes []corev1.Volume + var volumeMounts []corev1.VolumeMount + + // SecretHelper + secretHelperVolume := volumeBuild(ords.Name+"-"+"init-script", "ConfigMap", 0770) + secretHelperVolumeMount := volumeMountBuild(ords.Name+"-"+"init-script", ordsSABase+"/bin", true) + + volumes = append(volumes, secretHelperVolume) + volumeMounts = append(volumeMounts, secretHelperVolumeMount) + + // Build volume specifications for globalSettings + standaloneVolume := volumeBuild("standalone", "EmptyDir") + standaloneVolumeMount := volumeMountBuild("standalone", ordsSABase+"/config/global/standalone/", false) + + globalWalletVolume := volumeBuild("sa-wallet-global", "EmptyDir") + globalWalletVolumeMount := volumeMountBuild("sa-wallet-global", ordsSABase+"/config/global/wallet/", false) + + globalLogVolume := volumeBuild("sa-log-global", "EmptyDir") + globalLogVolumeMount := volumeMountBuild("sa-log-global", ordsSABase+"/log/global/", false) + + globalConfigVolume := volumeBuild(ords.Name+"-"+globalConfigMapName, "ConfigMap") + globalConfigVolumeMount := volumeMountBuild(ords.Name+"-"+globalConfigMapName, ordsSABase+"/config/global/", true) + + globalDocRootVolume := volumeBuild("sa-doc-root", "EmptyDir") + globalDocRootVolumeMount := volumeMountBuild("sa-doc-root", ordsSABase+"/config/global/doc_root/", false) + + volumes = append(volumes, standaloneVolume, globalWalletVolume, globalLogVolume, globalConfigVolume, globalDocRootVolume) + volumeMounts = append(volumeMounts, standaloneVolumeMount, globalWalletVolumeMount, globalLogVolumeMount, globalConfigVolumeMount, globalDocRootVolumeMount) + + if ords.Spec.GlobalSettings.CertSecret != nil { + globalCertVolume := volumeBuild(ords.Spec.GlobalSettings.CertSecret.SecretName, "Secret") + globalCertVolumeMount := volumeMountBuild(ords.Spec.GlobalSettings.CertSecret.SecretName, ordsSABase+"/config/certficate/", true) + + volumes = append(volumes, globalCertVolume) + volumeMounts = append(volumeMounts, globalCertVolumeMount) + } + + // Build volume specifications for each pool in poolSettings + definedWalletSecret := make(map[string]bool) + definedTNSSecret := make(map[string]bool) + for i := 0; i < len(ords.Spec.PoolSettings); i++ { + poolName := strings.ToLower(ords.Spec.PoolSettings[i].PoolName) + + poolWalletName := "sa-wallet-" + poolName + poolWalletVolume := volumeBuild(poolWalletName, "EmptyDir") + poolWalletVolumeMount := volumeMountBuild(poolWalletName, ordsSABase+"/config/databases/"+poolName+"/wallet/", false) + + poolConfigName := ords.Name + "-" + poolConfigPreName + poolName + poolConfigVolume := volumeBuild(poolConfigName, "ConfigMap") + poolConfigVolumeMount := volumeMountBuild(poolConfigName, ordsSABase+"/config/databases/"+poolName+"/", true) + + volumes = append(volumes, poolWalletVolume, poolConfigVolume) + volumeMounts = append(volumeMounts, poolWalletVolumeMount, poolConfigVolumeMount) + + if ords.Spec.PoolSettings[i].DBWalletSecret != nil { + walletSecretName := ords.Spec.PoolSettings[i].DBWalletSecret.SecretName + if !definedWalletSecret[walletSecretName] { + // Only create the volume once + poolDBWalletVolume := volumeBuild(walletSecretName, "Secret") + volumes = append(volumes, poolDBWalletVolume) + definedWalletSecret[walletSecretName] = true + } + poolDBWalletVolumeMount := volumeMountBuild(walletSecretName, ordsSABase+"/config/databases/"+poolName+"/network/admin/", true) + volumeMounts = append(volumeMounts, poolDBWalletVolumeMount) + } + + if ords.Spec.PoolSettings[i].TNSAdminSecret != nil { + tnsSecretName := ords.Spec.PoolSettings[i].TNSAdminSecret.SecretName + if !definedTNSSecret[tnsSecretName] { + // Only create the volume once + poolTNSAdminVolume := volumeBuild(tnsSecretName, "Secret") + volumes = append(volumes, poolTNSAdminVolume) + definedTNSSecret[tnsSecretName] = true + } + poolTNSAdminVolumeMount := volumeMountBuild(tnsSecretName, ordsSABase+"/config/databases/"+poolName+"/network/admin/", true) + volumeMounts = append(volumeMounts, poolTNSAdminVolumeMount) + } + } + return volumes, volumeMounts +} + +func volumeMountBuild(name string, path string, readOnly bool) corev1.VolumeMount { + return corev1.VolumeMount{ + Name: name, + MountPath: path, + ReadOnly: readOnly, + } +} + +func volumeBuild(name string, source string, mode ...int32) corev1.Volume { + defaultMode := int32(0660) + if len(mode) > 0 { + defaultMode = mode[0] + } + switch source { + case "ConfigMap": + return corev1.Volume{ + Name: name, + VolumeSource: corev1.VolumeSource{ + ConfigMap: &corev1.ConfigMapVolumeSource{ + DefaultMode: &defaultMode, + LocalObjectReference: corev1.LocalObjectReference{ + Name: name, + }, + }, + }, + } + case "Secret": + return corev1.Volume{ + Name: name, + VolumeSource: corev1.VolumeSource{ + Secret: &corev1.SecretVolumeSource{ + SecretName: name, + }, + }, + } + case "EmptyDir": + return corev1.Volume{ + Name: name, + VolumeSource: corev1.VolumeSource{ + EmptyDir: &corev1.EmptyDirVolumeSource{}, + }, + } + default: + return corev1.Volume{} + } +} + +// Service +func (r *OrdsSrvsReconciler) ServiceDefine(ctx context.Context, ords *dbapi.OrdsSrvs, HTTPport int32, HTTPSport int32, MongoPort int32) *corev1.Service { + labels := getLabels(ords.Name) + + servicePorts := []corev1.ServicePort{ + { + Name: serviceHTTPPortName, + Protocol: corev1.ProtocolTCP, + Port: HTTPport, + TargetPort: intstr.FromString(targetHTTPPortName), + }, + { + Name: serviceHTTPSPortName, + Protocol: corev1.ProtocolTCP, + Port: HTTPSport, + TargetPort: intstr.FromString(targetHTTPSPortName), + }, + } + + if ords.Spec.GlobalSettings.MongoEnabled { + mongoServicePort := corev1.ServicePort{ + Name: serviceMongoPortName, + Protocol: corev1.ProtocolTCP, + Port: MongoPort, + TargetPort: intstr.FromString(targetMongoPortName), + } + servicePorts = append(servicePorts, mongoServicePort) + } + + objectMeta := objectMetaDefine(ords, ords.Name) + def := &corev1.Service{ + ObjectMeta: objectMeta, + Spec: corev1.ServiceSpec{ + Selector: labels, + Ports: servicePorts, + }, + } + + // Set the ownerRef + if err := ctrl.SetControllerReference(ords, def, r.Scheme); err != nil { + return nil + } + return def +} + +func securityContextDefine() *corev1.SecurityContext { + return &corev1.SecurityContext{ + RunAsNonRoot: &[]bool{true}[0], + RunAsUser: &[]int64{54321}[0], + AllowPrivilegeEscalation: &[]bool{false}[0], + Capabilities: &corev1.Capabilities{ + Drop: []corev1.Capability{ + "ALL", + }, + }, + } +} + +func (r *OrdsSrvsReconciler) envDefine(ords *dbapi.OrdsSrvs, initContainer bool, ctx context.Context) []corev1.EnvVar { + envVarSecrets := []corev1.EnvVar{ + { + Name: "ORDS_CONFIG", + Value: ordsSABase + "/config", + }, + { + Name: "JAVA_TOOL_OPTIONS", + Value: "-Doracle.ml.version_check=false", + }, + } + + // Limitation case for ADB/mTLS/OraOper edge + if len(ords.Spec.PoolSettings) == 1 { + poolName := strings.ToLower(ords.Spec.PoolSettings[0].PoolName) + tnsAdmin := corev1.EnvVar{ + Name: "TNS_ADMIN", + Value: ordsSABase + "/config/databases/" + poolName + "/network/admin/", + } + envVarSecrets = append(envVarSecrets, tnsAdmin) + } + if initContainer { + for i := 0; i < len(ords.Spec.PoolSettings); i++ { + poolName := strings.ReplaceAll(strings.ToLower(ords.Spec.PoolSettings[i].PoolName), "-", "_") + + dbSecret := corev1.EnvVar{ + Name: poolName + "_dbsecret", + Value: r.CommonDecryptWithPrivKey3(ords, ords.Spec.PoolSettings[i].DBSecret.SecretName, ords.Spec.PoolSettings[i].DBSecret.PasswordKey, ctx), + } + + envVarSecrets = append(envVarSecrets, dbSecret) + + if ords.Spec.PoolSettings[i].DBAdminUserSecret.SecretName != "" { + autoUpgradeORDSEnv := corev1.EnvVar{ + Name: poolName + "_autoupgrade_ords", + Value: strconv.FormatBool(ords.Spec.PoolSettings[i].AutoUpgradeORDS), + } + autoUpgradeAPEXEnv := corev1.EnvVar{ + Name: poolName + "_autoupgrade_apex", + Value: strconv.FormatBool(ords.Spec.PoolSettings[i].AutoUpgradeAPEX), + } + + dbAdminUserSecret := corev1.EnvVar{ + Name: poolName + "_dbadminusersecret", + Value: r.CommonDecryptWithPrivKey3(ords, ords.Spec.PoolSettings[i].DBAdminUserSecret.SecretName, ords.Spec.PoolSettings[i].DBAdminUserSecret.PasswordKey, ctx), + } + envVarSecrets = append(envVarSecrets, dbAdminUserSecret, autoUpgradeORDSEnv, autoUpgradeAPEXEnv) + } + + if ords.Spec.PoolSettings[i].DBCDBAdminUserSecret.SecretName != "" { + + dbCDBAdminUserSecret := corev1.EnvVar{ + Name: poolName + "_dbcdbadminusersecret", + Value: r.CommonDecryptWithPrivKey3(ords, ords.Spec.PoolSettings[i].DBCDBAdminUserSecret.SecretName, ords.Spec.PoolSettings[i].DBCDBAdminUserSecret.PasswordKey, ctx), + } + + envVarSecrets = append(envVarSecrets, dbCDBAdminUserSecret) + } + } + } + + return envVarSecrets +} + +/************************************************* + * Deletions + **************************************************/ +func (r *OrdsSrvsReconciler) ConfigMapDelete(ctx context.Context, req ctrl.Request, ords *dbapi.OrdsSrvs, definedPools map[string]bool) (err error) { + // Delete Undefined Pool ConfigMaps + configMapList := &corev1.ConfigMapList{} + if err := r.List(ctx, configMapList, client.InNamespace(req.Namespace), + client.MatchingLabels(map[string]string{ + controllerLabelKey: controllerLabelVal, + "app.kubernetes.io/instance": ords.Name}), + ); err != nil { + return err + } + + for _, configMap := range configMapList.Items { + if configMap.Name == ords.Name+"-"+globalConfigMapName || configMap.Name == ords.Name+"-init-script" { + continue + } + if _, exists := definedPools[configMap.Name]; !exists { + if err := r.Delete(ctx, &configMap); err != nil { + return err + } + RestartPods = ords.Spec.ForceRestart + r.Recorder.Eventf(ords, corev1.EventTypeNormal, "Delete", "ConfigMap %s Deleted", configMap.Name) + } + } + + return nil +} + +func (r *OrdsSrvsReconciler) WorkloadDelete(ctx context.Context, req ctrl.Request, ords *dbapi.OrdsSrvs, kind string) (err error) { + logr := log.FromContext(ctx).WithName("WorkloadDelete") + + // Get Workloads + deploymentList := &appsv1.DeploymentList{} + if err := r.List(ctx, deploymentList, client.InNamespace(req.Namespace), + client.MatchingLabels(map[string]string{ + controllerLabelKey: controllerLabelVal, + "app.kubernetes.io/instance": ords.Name}), + ); err != nil { + return err + } + + statefulSetList := &appsv1.StatefulSetList{} + if err := r.List(ctx, statefulSetList, client.InNamespace(req.Namespace), + client.MatchingLabels(map[string]string{ + controllerLabelKey: controllerLabelVal, + "app.kubernetes.io/instance": ords.Name}), + ); err != nil { + return err + } + + daemonSetList := &appsv1.DaemonSetList{} + if err := r.List(ctx, daemonSetList, client.InNamespace(req.Namespace), + client.MatchingLabels(map[string]string{ + controllerLabelKey: controllerLabelVal, + "app.kubernetes.io/instance": ords.Name}), + ); err != nil { + return err + } + + switch kind { + case "StatefulSet": + for _, deleteDaemonSet := range daemonSetList.Items { + if err := r.Delete(ctx, &deleteDaemonSet); err != nil { + return err + } + logr.Info("Deleted: " + kind) + r.Recorder.Eventf(ords, corev1.EventTypeNormal, "Delete", "Workload %s Deleted", kind) + } + for _, deleteDeployment := range deploymentList.Items { + if err := r.Delete(ctx, &deleteDeployment); err != nil { + return err + } + logr.Info("Deleted: " + kind) + r.Recorder.Eventf(ords, corev1.EventTypeNormal, "Delete", "Workload %s Deleted", kind) + } + case "DaemonSet": + for _, deleteDeployment := range deploymentList.Items { + if err := r.Delete(ctx, &deleteDeployment); err != nil { + return err + } + logr.Info("Deleted: " + kind) + r.Recorder.Eventf(ords, corev1.EventTypeNormal, "Delete", "Workload %s Deleted", kind) + } + for _, deleteStatefulSet := range statefulSetList.Items { + if err := r.Delete(ctx, &deleteStatefulSet); err != nil { + return err + } + logr.Info("Deleted StatefulSet: " + deleteStatefulSet.Name) + r.Recorder.Eventf(ords, corev1.EventTypeNormal, "Delete", "Workload %s Deleted", kind) + } + default: + for _, deleteStatefulSet := range statefulSetList.Items { + if err := r.Delete(ctx, &deleteStatefulSet); err != nil { + return err + } + logr.Info("Deleted: " + kind) + r.Recorder.Eventf(ords, corev1.EventTypeNormal, "Delete", "Workload %s Deleted", kind) + } + for _, deleteDaemonSet := range daemonSetList.Items { + if err := r.Delete(ctx, &deleteDaemonSet); err != nil { + return err + } + logr.Info("Deleted: " + kind) + r.Recorder.Eventf(ords, corev1.EventTypeNormal, "Delete", "Workload %s Deleted", kind) + } + } + return nil +} + +/************************************************* + * Helpers + **************************************************/ +func getLabels(name string) map[string]string { + return map[string]string{ + "app.kubernetes.io/instance": name, + controllerLabelKey: controllerLabelVal, + } +} + +func generateSpecHash(spec interface{}) string { + byteArray, err := json.Marshal(spec) + if err != nil { + return "" + } + + hash := sha256.New() + _, err = hash.Write(byteArray) + if err != nil { + return "" + } + + hashBytes := hash.Sum(nil) + hashString := hex.EncodeToString(hashBytes[:8]) + + return hashString +} + +func CommonDecryptWithPrivKey(Key string, Buffer string) (string, error) { + + Debug := 0 + block, _ := pem.Decode([]byte(Key)) + pkcs8PrivateKey, err := x509.ParsePKCS8PrivateKey(block.Bytes) + if err != nil { + fmt.Printf("Failed to parse private key %s \n", err.Error()) + return "", err + } + if Debug == 1 { + fmt.Printf("======================================\n") + fmt.Printf("%s\n", Key) + fmt.Printf("======================================\n") + } + + encString64, err := base64.StdEncoding.DecodeString(string(Buffer)) + if err != nil { + fmt.Printf("Failed to decode encrypted string to base64: %s\n", err.Error()) + return "", err + } + + if Debug == 1 { + fmt.Printf("======================================\n") + fmt.Printf("%s\n", encString64) + fmt.Printf("======================================\n") + } + + decryptedB, err := rsa.DecryptPKCS1v15(nil, pkcs8PrivateKey.(*rsa.PrivateKey), encString64) + if err != nil { + fmt.Printf("Failed to decrypt string %s\n", err.Error()) + return "", err + } + if Debug == 1 { + fmt.Printf("[%s]\n", string(decryptedB)) + } + return strings.TrimSpace(string(decryptedB)), err + +} + +func (r *OrdsSrvsReconciler) CommonDecryptWithPrivKey3(ords *dbapi.OrdsSrvs, sname string, skey string, ctx context.Context) string { + logr := log.FromContext(ctx).WithName("CommonDecryptWithPrivKey2") + secret_par := &corev1.Secret{} + fmt.Printf("sname: %s\n", sname) + fmt.Printf("skey: %s\n", skey) + err := r.Get(ctx, types.NamespacedName{Name: sname, Namespace: ords.Namespace}, secret_par) + if err != nil { + logr.Error(err, "Cannot read secret"+sname) + return "" + } + encVal := string(secret_par.Data[skey]) + encVal = strings.TrimSpace(encVal) + + secret_key := &corev1.Secret{} + /* get private key */ + if err := r.Get(ctx, types.NamespacedName{Name: ords.Spec.EncPrivKey.SecretName, + Namespace: ords.Namespace}, secret_key); err != nil { + logr.Error(err, "Cannot get privte key") + return "" + } + PrvKeyVal := string(secret_key.Data[ords.Spec.EncPrivKey.PasswordKey]) + PrvKeyVal = strings.TrimSpace(PrvKeyVal) + + decVal, err := CommonDecryptWithPrivKey(PrvKeyVal, encVal) + if err != nil { + logr.Error(err, "Fail to decrypt secret") + return "" + } + + logr.Info("Password decryption completed") + + return decVal +} diff --git a/controllers/database/ordssrvs_ordsconfig.go b/controllers/database/ordssrvs_ordsconfig.go new file mode 100644 index 00000000..edb2e0f6 --- /dev/null +++ b/controllers/database/ordssrvs_ordsconfig.go @@ -0,0 +1,258 @@ +/* +** Copyright (c) 2024 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package controllers + +import ( + "context" + "fmt" + "os" + "strings" + "time" + + dbapi "github.com/oracle/oracle-database-operator/apis/database/v4" + corev1 "k8s.io/api/core/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + ctrl "sigs.k8s.io/controller-runtime" +) + +func (r *OrdsSrvsReconciler) ConfigMapDefine(ctx context.Context, ords *dbapi.OrdsSrvs, configMapName string, poolIndex int) *corev1.ConfigMap { + var defData map[string]string + if configMapName == ords.Name+"-init-script" { + // Read the file from controller's filesystem + filePath := "/ords_init.sh" + scriptData, err := os.ReadFile(filePath) + if err != nil { + return nil + } + defData = map[string]string{ + "init_script.sh": string(scriptData)} + } else if configMapName == ords.Name+"-"+globalConfigMapName { + // GlobalConfigMap + var defStandaloneAccessLog string + if ords.Spec.GlobalSettings.EnableStandaloneAccessLog { + defStandaloneAccessLog = ` ` + ordsSABase + `/log/global` + "\n" + } + var defMongoAccessLog string + if ords.Spec.GlobalSettings.EnableMongoAccessLog { + defMongoAccessLog = ` ` + ordsSABase + `/log/global` + "\n" + } + var defCert string + if ords.Spec.GlobalSettings.CertSecret != nil { + defCert = ` ` + ordsSABase + `/config/certficate/` + ords.Spec.GlobalSettings.CertSecret.Certificate + `` + "\n" + + ` ` + ordsSABase + `/config/certficate/` + ords.Spec.GlobalSettings.CertSecret.CertificateKey + `` + "\n" + } + defData = map[string]string{ + "settings.xml": fmt.Sprintf(`` + "\n" + + `` + "\n" + + `` + "\n" + + conditionalEntry("cache.metadata.graphql.expireAfterAccess", ords.Spec.GlobalSettings.CacheMetadataGraphQLExpireAfterAccess) + + conditionalEntry("cache.metadata.jwks.enabled", ords.Spec.GlobalSettings.CacheMetadataJWKSEnabled) + + conditionalEntry("cache.metadata.jwks.initialCapacity", ords.Spec.GlobalSettings.CacheMetadataJWKSInitialCapacity) + + conditionalEntry("cache.metadata.jwks.maximumSize", ords.Spec.GlobalSettings.CacheMetadataJWKSMaximumSize) + + conditionalEntry("cache.metadata.jwks.expireAfterAccess", ords.Spec.GlobalSettings.CacheMetadataJWKSExpireAfterAccess) + + conditionalEntry("cache.metadata.jwks.expireAfterWrite", ords.Spec.GlobalSettings.CacheMetadataJWKSExpireAfterWrite) + + conditionalEntry("database.api.management.services.disabled", ords.Spec.GlobalSettings.DatabaseAPIManagementServicesDisabled) + + conditionalEntry("db.invalidPoolTimeout", ords.Spec.GlobalSettings.DBInvalidPoolTimeout) + + conditionalEntry("feature.graphql.max.nesting.depth", ords.Spec.GlobalSettings.FeatureGraphQLMaxNestingDepth) + + conditionalEntry("request.traceHeaderName", ords.Spec.GlobalSettings.RequestTraceHeaderName) + + conditionalEntry("security.credentials.attempts", ords.Spec.GlobalSettings.SecurityCredentialsAttempts) + + conditionalEntry("security.credentials.lock.time", ords.Spec.GlobalSettings.SecurityCredentialsLockTime) + + conditionalEntry("standalone.context.path", ords.Spec.GlobalSettings.StandaloneContextPath) + + conditionalEntry("standalone.http.port", ords.Spec.GlobalSettings.StandaloneHTTPPort) + + conditionalEntry("standalone.https.host", ords.Spec.GlobalSettings.StandaloneHTTPSHost) + + conditionalEntry("standalone.https.port", ords.Spec.GlobalSettings.StandaloneHTTPSPort) + + conditionalEntry("standalone.stop.timeout", ords.Spec.GlobalSettings.StandaloneStopTimeout) + + conditionalEntry("cache.metadata.timeout", ords.Spec.GlobalSettings.CacheMetadataTimeout) + + conditionalEntry("cache.metadata.enabled", ords.Spec.GlobalSettings.CacheMetadataEnabled) + + conditionalEntry("database.api.enabled", ords.Spec.GlobalSettings.DatabaseAPIEnabled) + + conditionalEntry("debug.printDebugToScreen", ords.Spec.GlobalSettings.DebugPrintDebugToScreen) + + conditionalEntry("error.responseFormat", ords.Spec.GlobalSettings.ErrorResponseFormat) + + conditionalEntry("icap.port", ords.Spec.GlobalSettings.ICAPPort) + + conditionalEntry("icap.secure.port", ords.Spec.GlobalSettings.ICAPSecurePort) + + conditionalEntry("icap.server", ords.Spec.GlobalSettings.ICAPServer) + + conditionalEntry("log.procedure", ords.Spec.GlobalSettings.LogProcedure) + + conditionalEntry("mongo.enabled", ords.Spec.GlobalSettings.MongoEnabled) + + conditionalEntry("mongo.port", ords.Spec.GlobalSettings.MongoPort) + + conditionalEntry("mongo.idle.timeout", ords.Spec.GlobalSettings.MongoIdleTimeout) + + conditionalEntry("mongo.op.timeout", ords.Spec.GlobalSettings.MongoOpTimeout) + + conditionalEntry("security.disableDefaultExclusionList", ords.Spec.GlobalSettings.SecurityDisableDefaultExclusionList) + + conditionalEntry("security.exclusionList", ords.Spec.GlobalSettings.SecurityExclusionList) + + conditionalEntry("security.inclusionList", ords.Spec.GlobalSettings.SecurityInclusionList) + + conditionalEntry("security.maxEntries", ords.Spec.GlobalSettings.SecurityMaxEntries) + + conditionalEntry("security.verifySSL", ords.Spec.GlobalSettings.SecurityVerifySSL) + + conditionalEntry("security.httpsHeaderCheck", ords.Spec.GlobalSettings.SecurityHTTPSHeaderCheck) + + conditionalEntry("security.forceHTTPS", ords.Spec.GlobalSettings.SecurityForceHTTPS) + + conditionalEntry("externalSessionTrustedOrigins", ords.Spec.GlobalSettings.SecuirtyExternalSessionTrustedOrigins) + + ` ` + ordsSABase + `/config/global/doc_root/` + "\n" + + // Dynamic + defStandaloneAccessLog + + defMongoAccessLog + + defCert + + // Disabled (but not forgotten) + // conditionalEntry("standalone.binds", ords.Spec.GlobalSettings.StandaloneBinds) + + // conditionalEntry("error.externalPath", ords.Spec.GlobalSettings.ErrorExternalPath) + + // conditionalEntry("security.credentials.file ", ords.Spec.GlobalSettings.SecurityCredentialsFile) + + // conditionalEntry("standalone.static.path", ords.Spec.GlobalSettings.StandaloneStaticPath) + + // conditionalEntry("standalone.doc.root", ords.Spec.GlobalSettings.StandaloneDocRoot) + + // conditionalEntry("standalone.static.context.path", ords.Spec.GlobalSettings.StandaloneStaticContextPath) + + ``), + "logging.properties": fmt.Sprintf(`handlers=java.util.logging.FileHandler` + "\n" + + `.level=SEVERE` + "\n" + + `java.util.logging.FileHandler.level=ALL` + "\n" + + `oracle.dbtools.level=FINEST` + "\n" + + `java.util.logging.FileHandler.pattern = ` + ordsSABase + `/log/global/debug.log` + "\n" + + `java.util.logging.FileHandler.formatter = java.util.logging.SimpleFormatter`), + } + } else { + // PoolConfigMap + poolName := strings.ToLower(ords.Spec.PoolSettings[poolIndex].PoolName) + var defDBNetworkPath string + if ords.Spec.PoolSettings[poolIndex].DBWalletSecret != nil { + defDBNetworkPath = ` ` + ordsSABase + `/config/databases/` + poolName + `/network/admin/` + ords.Spec.PoolSettings[poolIndex].DBWalletSecret.WalletName + `` + "\n" + + conditionalEntry("db.wallet.zip.service", strings.ToUpper(ords.Spec.PoolSettings[poolIndex].DBWalletZipService)) + "\n" + } else { + defDBNetworkPath = ` ` + ordsSABase + `/config/databases/` + poolName + `/network/admin/` + "\n" + } + defData = map[string]string{ + "pool.xml": fmt.Sprintf(`` + "\n" + + `` + "\n" + + `` + "\n" + + ` ` + ords.Spec.PoolSettings[poolIndex].DBUsername + `` + "\n" + + conditionalEntry("db.adminUser", ords.Spec.PoolSettings[poolIndex].DBAdminUser) + + conditionalEntry("db.cdb.adminUser", ords.Spec.PoolSettings[poolIndex].DBCDBAdminUser) + + conditionalEntry("apex.security.administrator.roles", ords.Spec.PoolSettings[poolIndex].ApexSecurityAdministratorRoles) + + conditionalEntry("apex.security.user.roles", ords.Spec.PoolSettings[poolIndex].ApexSecurityUserRoles) + + conditionalEntry("db.credentialsSource", ords.Spec.PoolSettings[poolIndex].DBCredentialsSource) + + conditionalEntry("db.poolDestroyTimeout", ords.Spec.PoolSettings[poolIndex].DBPoolDestroyTimeout) + + conditionalEntry("debug.trackResources", ords.Spec.PoolSettings[poolIndex].DebugTrackResources) + + conditionalEntry("feature.openservicebroker.exclude", ords.Spec.PoolSettings[poolIndex].FeatureOpenservicebrokerExclude) + + conditionalEntry("feature.sdw", ords.Spec.PoolSettings[poolIndex].FeatureSDW) + + conditionalEntry("http.cookie.filter", ords.Spec.PoolSettings[poolIndex].HttpCookieFilter) + + conditionalEntry("jdbc.auth.admin.role", ords.Spec.PoolSettings[poolIndex].JDBCAuthAdminRole) + + conditionalEntry("jdbc.cleanup.mode", ords.Spec.PoolSettings[poolIndex].JDBCCleanupMode) + + conditionalEntry("owa.trace.sql", ords.Spec.PoolSettings[poolIndex].OwaTraceSql) + + conditionalEntry("plsql.gateway.mode", ords.Spec.PoolSettings[poolIndex].PlsqlGatewayMode) + + conditionalEntry("security.jwt.profile.enabled", ords.Spec.PoolSettings[poolIndex].SecurityJWTProfileEnabled) + + conditionalEntry("security.jwks.size", ords.Spec.PoolSettings[poolIndex].SecurityJWKSSize) + + conditionalEntry("security.jwks.connection.timeout", ords.Spec.PoolSettings[poolIndex].SecurityJWKSConnectionTimeout) + + conditionalEntry("security.jwks.read.timeout", ords.Spec.PoolSettings[poolIndex].SecurityJWKSReadTimeout) + + conditionalEntry("security.jwks.refresh.interval", ords.Spec.PoolSettings[poolIndex].SecurityJWKSRefreshInterval) + + conditionalEntry("security.jwt.allowed.skew", ords.Spec.PoolSettings[poolIndex].SecurityJWTAllowedSkew) + + conditionalEntry("security.jwt.allowed.age", ords.Spec.PoolSettings[poolIndex].SecurityJWTAllowedAge) + + conditionalEntry("db.connectionType", ords.Spec.PoolSettings[poolIndex].DBConnectionType) + + conditionalEntry("db.customURL", ords.Spec.PoolSettings[poolIndex].DBCustomURL) + + conditionalEntry("db.hostname", ords.Spec.PoolSettings[poolIndex].DBHostname) + + conditionalEntry("db.port", ords.Spec.PoolSettings[poolIndex].DBPort) + + conditionalEntry("db.servicename", ords.Spec.PoolSettings[poolIndex].DBServicename) + + conditionalEntry("db.sid", ords.Spec.PoolSettings[poolIndex].DBSid) + + conditionalEntry("db.tnsAliasName", ords.Spec.PoolSettings[poolIndex].DBTnsAliasName) + + conditionalEntry("jdbc.DriverType", ords.Spec.PoolSettings[poolIndex].JDBCDriverType) + + conditionalEntry("jdbc.InactivityTimeout", ords.Spec.PoolSettings[poolIndex].JDBCInactivityTimeout) + + conditionalEntry("jdbc.InitialLimit", ords.Spec.PoolSettings[poolIndex].JDBCInitialLimit) + + conditionalEntry("jdbc.MaxConnectionReuseCount", ords.Spec.PoolSettings[poolIndex].JDBCMaxConnectionReuseCount) + + conditionalEntry("jdbc.MaxLimit", ords.Spec.PoolSettings[poolIndex].JDBCMaxLimit) + + conditionalEntry("jdbc.auth.enabled", ords.Spec.PoolSettings[poolIndex].JDBCAuthEnabled) + + conditionalEntry("jdbc.MaxStatementsLimit", ords.Spec.PoolSettings[poolIndex].JDBCMaxStatementsLimit) + + conditionalEntry("jdbc.MinLimit", ords.Spec.PoolSettings[poolIndex].JDBCMinLimit) + + conditionalEntry("jdbc.statementTimeout", ords.Spec.PoolSettings[poolIndex].JDBCStatementTimeout) + + conditionalEntry("jdbc.MaxConnectionReuseTime", ords.Spec.PoolSettings[poolIndex].JDBCMaxConnectionReuseTime) + + conditionalEntry("jdbc.SecondsToTrustIdleConnection", ords.Spec.PoolSettings[poolIndex].JDBCSecondsToTrustIdleConnection) + + conditionalEntry("misc.defaultPage", ords.Spec.PoolSettings[poolIndex].MiscDefaultPage) + + conditionalEntry("misc.pagination.maxRows", ords.Spec.PoolSettings[poolIndex].MiscPaginationMaxRows) + + conditionalEntry("procedure.postProcess", ords.Spec.PoolSettings[poolIndex].ProcedurePostProcess) + + conditionalEntry("procedure.preProcess", ords.Spec.PoolSettings[poolIndex].ProcedurePreProcess) + + conditionalEntry("procedure.rest.preHook", ords.Spec.PoolSettings[poolIndex].ProcedureRestPreHook) + + conditionalEntry("security.requestAuthenticationFunction", ords.Spec.PoolSettings[poolIndex].SecurityRequestAuthenticationFunction) + + conditionalEntry("security.requestValidationFunction", ords.Spec.PoolSettings[poolIndex].SecurityRequestValidationFunction) + + conditionalEntry("soda.defaultLimit", ords.Spec.PoolSettings[poolIndex].SODADefaultLimit) + + conditionalEntry("soda.maxLimit", ords.Spec.PoolSettings[poolIndex].SODAMaxLimit) + + conditionalEntry("restEnabledSql.active", ords.Spec.PoolSettings[poolIndex].RestEnabledSqlActive) + + defDBNetworkPath + + // Disabled (but not forgotten) + // conditionalEntry("autoupgrade.api.aulocation", ords.Spec.PoolSettings[poolIndex].AutoupgradeAPIAulocation) + + // conditionalEntry("autoupgrade.api.enabled", ords.Spec.PoolSettings[poolIndex].AutoupgradeAPIEnabled) + + // conditionalEntry("autoupgrade.api.jvmlocation", ords.Spec.PoolSettings[poolIndex].AutoupgradeAPIJvmlocation) + + // conditionalEntry("autoupgrade.api.loglocation", ords.Spec.PoolSettings[poolIndex].AutoupgradeAPILoglocation) + + // conditionalEntry("db.serviceNameSuffix", ords.Spec.PoolSettings[poolIndex].DBServiceNameSuffix) + + ``), + } + } + + objectMeta := objectMetaDefine(ords, configMapName) + def := &corev1.ConfigMap{ + TypeMeta: metav1.TypeMeta{ + Kind: "ConfigMap", + APIVersion: "v1", + }, + ObjectMeta: objectMeta, + Data: defData, + } + + // Set the ownerRef + if err := ctrl.SetControllerReference(ords, def, r.Scheme); err != nil { + return nil + } + return def +} + +func conditionalEntry(key string, value interface{}) string { + switch v := value.(type) { + case nil: + return "" + case string: + if v != "" { + return fmt.Sprintf(` %s`+"\n", key, v) + } + case *int32: + if v != nil { + return fmt.Sprintf(` %d`+"\n", key, *v) + } + case *bool: + if v != nil { + return fmt.Sprintf(` %v`+"\n", key, *v) + } + case *time.Duration: + if v != nil { + return fmt.Sprintf(` %v`+"\n", key, *v) + } + default: + return fmt.Sprintf(` %v`+"\n", key, v) + } + return "" +} diff --git a/controllers/database/pdb_controller.go b/controllers/database/pdb_controller.go index f0b4fd46..a2ca0f85 100644 --- a/controllers/database/pdb_controller.go +++ b/controllers/database/pdb_controller.go @@ -55,7 +55,8 @@ import ( "strings" "time" - dbapi "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + dbapi "github.com/oracle/oracle-database-operator/apis/database/v4" + lrcommons "github.com/oracle/oracle-database-operator/commons/multitenant/lrest" "github.com/go-logr/logr" corev1 "k8s.io/api/core/v1" @@ -82,6 +83,11 @@ type PDBReconciler struct { Recorder record.EventRecorder } +type controllers struct { + Pdbc PDBReconciler + Cdbc CDBReconciler +} + type RESTSQLCollection struct { Env struct { DefaultTimeZone string `json:"defaultTimeZone,omitempty"` @@ -426,7 +432,8 @@ func (r *PDBReconciler) getSecret(ctx context.Context, req ctrl.Request, pdb *db /* ************************************************ - Issue a REST API Call to the ORDS container - /*********************************************** + +*********************************************** */ func (r *PDBReconciler) callAPI(ctx context.Context, req ctrl.Request, pdb *dbapi.PDB, url string, payload map[string]string, action string) (string, error) { log := r.Log.WithValues("callAPI", req.NamespacedName) @@ -472,11 +479,6 @@ func (r *PDBReconciler) callAPI(ctx context.Context, req ctrl.Request, pdb *dbap } caCert := secret.Data[pdb.Spec.PDBTlsCat.Secret.Key] - /* - r.Recorder.Eventf(pdb, corev1.EventTypeWarning, "ORDSINFO", string(rsaKeyPEM)) - r.Recorder.Eventf(pdb, corev1.EventTypeWarning, "ORDSINFO", string(rsaCertPEM)) - r.Recorder.Eventf(pdb, corev1.EventTypeWarning, "ORDSINFO", string(caCert)) - */ certificate, err := tls.X509KeyPair([]byte(rsaCertPEM), []byte(rsaKeyPEM)) if err != nil { @@ -620,7 +622,8 @@ func (r *PDBReconciler) callAPI(ctx context.Context, req ctrl.Request, pdb *dbap /* ************************************************ - Create a PDB - /*********************************************** + +*********************************************** */ func (r *PDBReconciler) createPDB(ctx context.Context, req ctrl.Request, pdb *dbapi.PDB) error { @@ -635,15 +638,52 @@ func (r *PDBReconciler) createPDB(ctx context.Context, req ctrl.Request, pdb *db return err } - pdbAdminName, err := r.getSecret(ctx, req, pdb, pdb.Spec.AdminName.Secret.SecretName, pdb.Spec.AdminName.Secret.Key) + /*** BEGIN GET ENCPASS ***/ + secret := &corev1.Secret{} + + err = r.Get(ctx, types.NamespacedName{Name: pdb.Spec.AdminName.Secret.SecretName, Namespace: pdb.Namespace}, secret) if err != nil { + if apierrors.IsNotFound(err) { + log.Info("Secret not found:" + pdb.Spec.AdminName.Secret.SecretName) + return err + } + log.Error(err, "Unable to get the secret.") + return err + } + pdbAdminNameEnc := string(secret.Data[pdb.Spec.AdminName.Secret.Key]) + pdbAdminNameEnc = strings.TrimSpace(pdbAdminNameEnc) + + err = r.Get(ctx, types.NamespacedName{Name: pdb.Spec.PDBPriKey.Secret.SecretName, Namespace: pdb.Namespace}, secret) + if err != nil { + if apierrors.IsNotFound(err) { + log.Info("Secret not found:" + pdb.Spec.PDBPriKey.Secret.SecretName) + return err + } + log.Error(err, "Unable to get the secret.") return err } - pdbAdminPwd, err := r.getSecret(ctx, req, pdb, pdb.Spec.AdminPwd.Secret.SecretName, pdb.Spec.AdminPwd.Secret.Key) + privKey := string(secret.Data[pdb.Spec.PDBPriKey.Secret.Key]) + pdbAdminName, err := lrcommons.CommonDecryptWithPrivKey(privKey, pdbAdminNameEnc, req) + + // Get Web Server User Password + secret = &corev1.Secret{} + err = r.Get(ctx, types.NamespacedName{Name: pdb.Spec.AdminPwd.Secret.SecretName, Namespace: pdb.Namespace}, secret) if err != nil { + if apierrors.IsNotFound(err) { + log.Info("Secret not found:" + pdb.Spec.AdminPwd.Secret.SecretName) + return err + } + log.Error(err, "Unable to get the secret.") return err } + pdbAdminPwdEnc := string(secret.Data[pdb.Spec.AdminPwd.Secret.Key]) + pdbAdminPwdEnc = strings.TrimSpace(pdbAdminPwdEnc) + pdbAdminPwd, err := lrcommons.CommonDecryptWithPrivKey(privKey, pdbAdminPwdEnc, req) + pdbAdminName = strings.TrimSuffix(pdbAdminName, "\n") + pdbAdminPwd = strings.TrimSuffix(pdbAdminPwd, "\n") + /*** END GET ENCPASS ***/ + log.Info("====================> " + pdbAdminName + ":" + pdbAdminPwd) /* Prevent creating an existing pdb */ err = r.getPDBState(ctx, req, pdb) if err != nil { @@ -656,9 +696,6 @@ func (r *PDBReconciler) createPDB(ctx context.Context, req ctrl.Request, pdb *db return nil } - pdbAdminName = strings.TrimSuffix(pdbAdminName, "\n") - pdbAdminPwd = strings.TrimSuffix(pdbAdminPwd, "\n") - values := map[string]string{ "method": "CREATE", "pdb_name": pdb.Spec.PDBName, @@ -696,7 +733,7 @@ func (r *PDBReconciler) createPDB(ctx context.Context, req ctrl.Request, pdb *db if err := r.Status().Update(ctx, pdb); err != nil { log.Error(err, "Failed to update status for :"+pdb.Name, "err", err.Error()) } - _, err = r.callAPI(ctx, req, pdb, url, values, "POST") + _, err = NewCallApi(r, ctx, req, pdb, url, values, "POST") if err != nil { log.Error(err, "callAPI error", "err", err.Error()) return err @@ -724,7 +761,8 @@ func (r *PDBReconciler) createPDB(ctx context.Context, req ctrl.Request, pdb *db /* ************************************************ - Clone a PDB - /*********************************************** + +*********************************************** */ func (r *PDBReconciler) clonePDB(ctx context.Context, req ctrl.Request, pdb *dbapi.PDB) error { @@ -773,8 +811,6 @@ func (r *PDBReconciler) clonePDB(ctx context.Context, req ctrl.Request, pdb *dba values["tempSize"] = pdb.Spec.TempSize } - //url := "https://"+ pdb.Spec.CDBNamespace + "." + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdb.Spec.SrcPDBName + "/" - //url := "https://" + pdb.Spec.CDBResName + "-ords:" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdb.Spec.SrcPDBName + "/" url := "https://" + pdb.Spec.CDBResName + "-ords." + pdb.Spec.CDBNamespace + ":" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdb.Spec.SrcPDBName + "/" pdb.Status.Phase = pdbPhaseClone @@ -782,7 +818,7 @@ func (r *PDBReconciler) clonePDB(ctx context.Context, req ctrl.Request, pdb *dba if err := r.Status().Update(ctx, pdb); err != nil { log.Error(err, "Failed to update status for :"+pdb.Name, "err", err.Error()) } - _, err = r.callAPI(ctx, req, pdb, url, values, "POST") + _, err = NewCallApi(r, ctx, req, pdb, url, values, "POST") if err != nil { return err } @@ -809,7 +845,8 @@ func (r *PDBReconciler) clonePDB(ctx context.Context, req ctrl.Request, pdb *dba /* ************************************************ - Plug a PDB - /*********************************************** + +*********************************************** */ func (r *PDBReconciler) plugPDB(ctx context.Context, req ctrl.Request, pdb *dbapi.PDB) error { @@ -868,7 +905,7 @@ func (r *PDBReconciler) plugPDB(ctx context.Context, req ctrl.Request, pdb *dbap if err := r.Status().Update(ctx, pdb); err != nil { log.Error(err, "Failed to update status for :"+pdb.Name, "err", err.Error()) } - _, err = r.callAPI(ctx, req, pdb, url, values, "POST") + _, err = NewCallApi(r, ctx, req, pdb, url, values, "POST") if err != nil { return err } @@ -879,6 +916,7 @@ func (r *PDBReconciler) plugPDB(ctx context.Context, req ctrl.Request, pdb *dbap pdb.Status.ConnString = cdb.Spec.DBServer + ":" + strconv.Itoa(cdb.Spec.DBPort) + "/" + pdb.Spec.PDBName } else { pdb.Status.ConnString = cdb.Spec.DBTnsurl + ParseTnsAlias(&(pdb.Status.ConnString), &(pdb.Spec.PDBName)) } assertivePdbDeletion = pdb.Spec.AssertivePdbDeletion @@ -894,7 +932,8 @@ func (r *PDBReconciler) plugPDB(ctx context.Context, req ctrl.Request, pdb *dbap /* ************************************************ - Unplug a PDB - /*********************************************** + +*********************************************** */ func (r *PDBReconciler) unplugPDB(ctx context.Context, req ctrl.Request, pdb *dbapi.PDB) error { @@ -950,7 +989,7 @@ func (r *PDBReconciler) unplugPDB(ctx context.Context, req ctrl.Request, pdb *db if err := r.Status().Update(ctx, pdb); err != nil { log.Error(err, "Failed to update status for :"+pdb.Name, "err", err.Error()) } - _, err = r.callAPI(ctx, req, pdb, url, values, "POST") + _, err = NewCallApi(r, ctx, req, pdb, url, values, "POST") if err != nil { return err } @@ -1036,7 +1075,7 @@ func (r *PDBReconciler) modifyPDB(ctx context.Context, req ctrl.Request, pdb *db if err := r.Status().Update(ctx, pdb); err != nil { log.Error(err, "Failed to update status for :"+pdb.Name, "err", err.Error()) } - _, err = r.callAPI(ctx, req, pdb, url, values, "POST") + _, err = NewCallApi(r, ctx, req, pdb, url, values, "POST") if err != nil { return err } @@ -1078,7 +1117,7 @@ func (r *PDBReconciler) getPDBState(ctx context.Context, req ctrl.Request, pdb * log.Error(err, "Failed to update status for :"+pdb.Name, "err", err.Error()) } - respData, err := r.callAPI(ctx, req, pdb, url, nil, "GET") + respData, err := NewCallApi(r, ctx, req, pdb, url, nil, "GET") if err != nil { pdb.Status.OpenMode = "UNKNOWN" @@ -1129,7 +1168,7 @@ func (r *PDBReconciler) mapPDB(ctx context.Context, req ctrl.Request, pdb *dbapi log.Error(err, "Failed to update status for :"+pdb.Name, "err", err.Error()) } - respData, err := r.callAPI(ctx, req, pdb, url, nil, "GET") + respData, err := NewCallApi(r, ctx, req, pdb, url, nil, "GET") if err != nil { pdb.Status.OpenMode = "UNKNOWN" @@ -1227,6 +1266,7 @@ func (r *PDBReconciler) managePDBDeletion2(ctx context.Context, req ctrl.Request return err } + var errclose error pdbName := pdb.Spec.PDBName if pdb.Status.OpenMode == "READ WRITE" { valuesclose := map[string]string{ @@ -1234,23 +1274,26 @@ func (r *PDBReconciler) managePDBDeletion2(ctx context.Context, req ctrl.Request "modifyOption": "IMMEDIATE", "getScript": "FALSE"} url := "https://" + pdb.Spec.CDBResName + "-ords." + pdb.Spec.CDBNamespace + ":" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdbName + "/status" - _, errclose := r.callAPI(ctx, req, pdb, url, valuesclose, "POST") + _, errclose = NewCallApi(r, ctx, req, pdb, url, valuesclose, "POST") if errclose != nil { log.Info("Warning error closing pdb continue anyway") } } - valuesdrop := map[string]string{ - "action": "INCLUDING", - "getScript": "FALSE"} - url := "https://" + pdb.Spec.CDBResName + "-ords." + pdb.Spec.CDBNamespace + ":" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdbName + "/" - - log.Info("Call Delete()") - _, errdelete := r.callAPI(ctx, req, pdb, url, valuesdrop, "DELETE") - if errdelete != nil { - log.Error(errdelete, "Fail to delete pdb :"+pdb.Name, "err", err.Error()) - return errdelete + if errclose == nil { + valuesdrop := map[string]string{ + "action": "INCLUDING", + "getScript": "FALSE"} + url := "https://" + pdb.Spec.CDBResName + "-ords." + pdb.Spec.CDBNamespace + ":" + strconv.Itoa(cdb.Spec.ORDSPort) + "/ords/_/db-api/latest/database/pdbs/" + pdbName + "/" + + log.Info("Call Delete()") + _, errdelete := NewCallApi(r, ctx, req, pdb, url, valuesdrop, "DELETE") + if errdelete != nil { + log.Error(errdelete, "Fail to delete pdb :"+pdb.Name, "err", errdelete.Error()) + return errdelete + } } + } /* END OF ASSERTIVE SECTION */ log.Info("Marked to be deleted") @@ -1304,7 +1347,7 @@ func (r *PDBReconciler) deletePDBInstance(req ctrl.Request, ctx context.Context, if err := r.Status().Update(ctx, pdb); err != nil { log.Error(err, "Failed to update status for :"+pdb.Name, "err", err.Error()) } - _, err = r.callAPI(ctx, req, pdb, url, values, "DELETE") + _, err = NewCallApi(r, ctx, req, pdb, url, values, "DELETE") if err != nil { pdb.Status.ConnString = "" return err @@ -1363,3 +1406,226 @@ func ParseTnsAlias(tns *string, pdbsrv *string) { fmt.Printf("Newstring [%s]\n", *tns) } + +func NewCallApi(intr interface{}, ctx context.Context, req ctrl.Request, pdb *dbapi.PDB, url string, payload map[string]string, action string) (string, error) { + + var c client.Client + var r logr.Logger + var e record.EventRecorder + var err error + + recpdb, ok1 := intr.(*PDBReconciler) + if ok1 { + fmt.Printf("func NewCallApi ((*PDBReconciler),......)\n") + c = recpdb.Client + e = recpdb.Recorder + r = recpdb.Log + } + + reccdb, ok2 := intr.(*CDBReconciler) + if ok2 { + fmt.Printf("func NewCallApi ((*CDBReconciler),......)\n") + c = reccdb.Client + e = reccdb.Recorder + r = reccdb.Log + } + + secret := &corev1.Secret{} + + log := r.WithValues("NewCallApi", req.NamespacedName) + log.Info("Call c.Get") + err = c.Get(ctx, types.NamespacedName{Name: pdb.Spec.PDBTlsKey.Secret.SecretName, Namespace: pdb.Namespace}, secret) + + if err != nil { + if apierrors.IsNotFound(err) { + log.Info("Secret not found:" + pdb.Spec.PDBTlsKey.Secret.SecretName) + return "", err + } + log.Error(err, "Unable to get the secret.") + return "", err + } + rsaKeyPEM := secret.Data[pdb.Spec.PDBTlsKey.Secret.Key] + + err = c.Get(ctx, types.NamespacedName{Name: pdb.Spec.PDBTlsCrt.Secret.SecretName, Namespace: pdb.Namespace}, secret) + + if err != nil { + if apierrors.IsNotFound(err) { + log.Info("Secret not found:" + pdb.Spec.PDBTlsCrt.Secret.SecretName) + return "", err + } + log.Error(err, "Unable to get the secret.") + return "", err + } + + rsaCertPEM := secret.Data[pdb.Spec.PDBTlsCrt.Secret.Key] + + err = c.Get(ctx, types.NamespacedName{Name: pdb.Spec.PDBTlsCat.Secret.SecretName, Namespace: pdb.Namespace}, secret) + + if err != nil { + if apierrors.IsNotFound(err) { + log.Info("Secret not found:" + pdb.Spec.PDBTlsCat.Secret.SecretName) + return "", err + } + log.Error(err, "Unable to get the secret.") + return "", err + } + + caCert := secret.Data[pdb.Spec.PDBTlsCat.Secret.Key] + /* + r.Recorder.Eventf(pdb, corev1.EventTypeWarning, "ORDSINFO", string(rsaKeyPEM)) + r.Recorder.Eventf(pdb, corev1.EventTypeWarning, "ORDSINFO", string(rsaCertPEM)) + r.Recorder.Eventf(pdb, corev1.EventTypeWarning, "ORDSINFO", string(caCert)) + */ + + certificate, err := tls.X509KeyPair([]byte(rsaCertPEM), []byte(rsaKeyPEM)) + if err != nil { + pdb.Status.Msg = "Error tls.X509KeyPair" + return "", err + } + + caCertPool := x509.NewCertPool() + caCertPool.AppendCertsFromPEM(caCert) + + tlsConf := &tls.Config{Certificates: []tls.Certificate{certificate}, RootCAs: caCertPool} + + tr := &http.Transport{TLSClientConfig: tlsConf} + + httpclient := &http.Client{Transport: tr} + + log.Info("Issuing REST call", "URL", url, "Action", action) + + /* + cdb, err := r.getCDBResource(ctx, req, pdb) + if err != nil { + return "", err + } + */ + + err = c.Get(ctx, types.NamespacedName{Name: pdb.Spec.WebServerUsr.Secret.SecretName, Namespace: pdb.Namespace}, secret) + if err != nil { + if apierrors.IsNotFound(err) { + log.Info("Secret not found:" + pdb.Spec.WebServerUsr.Secret.SecretName) + return "", err + } + log.Error(err, "Unable to get the secret.") + return "", err + } + webUserEnc := string(secret.Data[pdb.Spec.WebServerUsr.Secret.Key]) + webUserEnc = strings.TrimSpace(webUserEnc) + + err = c.Get(ctx, types.NamespacedName{Name: pdb.Spec.PDBPriKey.Secret.SecretName, Namespace: pdb.Namespace}, secret) + if err != nil { + if apierrors.IsNotFound(err) { + log.Info("Secret not found:" + pdb.Spec.PDBPriKey.Secret.SecretName) + return "", err + } + log.Error(err, "Unable to get the secret.") + return "", err + } + privKey := string(secret.Data[pdb.Spec.PDBPriKey.Secret.Key]) + webUser, err := lrcommons.CommonDecryptWithPrivKey(privKey, webUserEnc, req) + + // Get Web Server User Password + secret = &corev1.Secret{} + err = c.Get(ctx, types.NamespacedName{Name: pdb.Spec.WebServerPwd.Secret.SecretName, Namespace: pdb.Namespace}, secret) + if err != nil { + if apierrors.IsNotFound(err) { + log.Info("Secret not found:" + pdb.Spec.WebServerPwd.Secret.SecretName) + return "", err + } + log.Error(err, "Unable to get the secret.") + return "", err + } + webUserPwdEnc := string(secret.Data[pdb.Spec.WebServerPwd.Secret.Key]) + webUserPwdEnc = strings.TrimSpace(webUserPwdEnc) + webUserPwd, err := lrcommons.CommonDecryptWithPrivKey(privKey, webUserPwdEnc, req) + /////////////////////////////////////////////////////////////////////////////////// + + var httpreq *http.Request + if action == "GET" { + httpreq, err = http.NewRequest(action, url, nil) + } else { + jsonValue, _ := json.Marshal(payload) + httpreq, err = http.NewRequest(action, url, bytes.NewBuffer(jsonValue)) + } + + if err != nil { + log.Info("Unable to create HTTP Request for PDB : "+pdb.Name, "err", err.Error()) + return "", err + } + + httpreq.Header.Add("Accept", "application/json") + httpreq.Header.Add("Content-Type", "application/json") + httpreq.SetBasicAuth(webUser, webUserPwd) + + resp, err := httpclient.Do(httpreq) + if err != nil { + errmsg := err.Error() + log.Error(err, "Failed - Could not connect to ORDS Pod", "err", err.Error()) + pdb.Status.Msg = "Error: Could not connect to ORDS Pod" + e.Eventf(pdb, corev1.EventTypeWarning, "ORDSError", errmsg) + return "", err + } + + e.Eventf(pdb, corev1.EventTypeWarning, "Done", pdb.Spec.CDBResName) + if resp.StatusCode != http.StatusOK { + bb, _ := ioutil.ReadAll(resp.Body) + + if resp.StatusCode == 404 { + pdb.Status.ConnString = "" + pdb.Status.Msg = pdb.Spec.PDBName + " not found" + + } else { + if floodcontrol == false { + pdb.Status.Msg = "ORDS Error - HTTP Status Code:" + strconv.Itoa(resp.StatusCode) + } + } + + if floodcontrol == false { + log.Info("ORDS Error - HTTP Status Code :"+strconv.Itoa(resp.StatusCode), "Err", string(bb)) + } + + var apiErr ORDSError + json.Unmarshal([]byte(bb), &apiErr) + if floodcontrol == false { + e.Eventf(pdb, corev1.EventTypeWarning, "ORDSError", "Failed: %s", apiErr.Message) + } + //fmt.Printf("%+v", apiErr) + //fmt.Println(string(bb)) + floodcontrol = true + return "", errors.New("ORDS Error") + } + floodcontrol = false + + defer resp.Body.Close() + + bodyBytes, err := ioutil.ReadAll(resp.Body) + if err != nil { + fmt.Print(err.Error()) + } + respData := string(bodyBytes) + //fmt.Println(string(bodyBytes)) + + var apiResponse RESTSQLCollection + json.Unmarshal([]byte(bodyBytes), &apiResponse) + //fmt.Printf("%#v", apiResponse) + //fmt.Printf("%+v", apiResponse) + + errFound := false + for _, sqlItem := range apiResponse.Items { + if sqlItem.ErrorDetails != "" { + log.Info("ORDS Error - Oracle Error Code :" + strconv.Itoa(sqlItem.ErrorCode)) + if !errFound { + pdb.Status.Msg = sqlItem.ErrorDetails + } + e.Eventf(pdb, corev1.EventTypeWarning, "OraError", "%s", sqlItem.ErrorDetails) + errFound = true + } + } + + if errFound { + return "", errors.New("Oracle Error") + } + + return respData, nil +} diff --git a/controllers/database/shardingdatabase_controller.go b/controllers/database/shardingdatabase_controller.go index 7fcaac2b..1ec77253 100644 --- a/controllers/database/shardingdatabase_controller.go +++ b/controllers/database/shardingdatabase_controller.go @@ -67,18 +67,17 @@ import ( "sigs.k8s.io/controller-runtime/pkg/event" "sigs.k8s.io/controller-runtime/pkg/predicate" - databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + databasev4 "github.com/oracle/oracle-database-operator/apis/database/v4" shardingv1 "github.com/oracle/oracle-database-operator/commons/sharding" ) -// Sharding Topology -type ShardingTopology struct { - topicid string - Instance *databasev1alpha1.ShardingDatabase - deltopology bool - onsProvider common.ConfigurationProvider - onsProviderFlag bool - rclient ons.NotificationDataPlaneClient +// Struct keeping Oracle Notification Server Info +type OnsStatus struct { + Topicid string `json:"topicid,omitempty"` + Instance *databasev4.ShardingDatabase `json:"instance,omitempty"` + OnsProvider common.ConfigurationProvider `json:"onsProvider,omitempty"` + OnsProviderFlag bool `json:"onsProviderFlag,omitempty"` + Rclient ons.NotificationDataPlaneClient `json:"rclient,omitempty"` } // ShardingDatabaseReconciler reconciles a ShardingDatabase object @@ -89,7 +88,6 @@ type ShardingDatabaseReconciler struct { kubeClient kubernetes.Interface kubeConfig clientcmd.ClientConfig Recorder record.EventRecorder - osh []*ShardingTopology InCluster bool Namespace string } @@ -97,6 +95,8 @@ type ShardingDatabaseReconciler struct { var sentFailMsg = make(map[string]bool) var sentCompleteMsg = make(map[string]bool) +var oshMap = make(map[string]*OnsStatus) + // +kubebuilder:rbac:groups=database.oracle.com,resources=shardingdatabases,verbs=get;list;watch;create;update;patch;delete // +kubebuilder:rbac:groups=database.oracle.com,resources=shardingdatabases/status,verbs=get;update;patch // +kubebuilder:rbac:groups=database.oracle.com,resources=shardingdatabases/finalizers,verbs=get;create;update;patch;delete @@ -120,19 +120,19 @@ func (r *ShardingDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.Req // your logic here var i int32 - //var ShardImageLatest []databasev1alpha1.ShardSpec - var OraCatalogSpex databasev1alpha1.CatalogSpec - var OraShardSpex databasev1alpha1.ShardSpec - var OraGsmSpex databasev1alpha1.GsmSpec + //var ShardImageLatest []databasev4.ShardSpec + var OraCatalogSpex databasev4.CatalogSpec + var OraShardSpex databasev4.ShardSpec + var OraGsmSpex databasev4.GsmSpec var result ctrl.Result var isShardTopologyDeleteTrue bool = false //var msg string var err error - var idx int var stateType string resultNq := ctrl.Result{Requeue: false} resultQ := ctrl.Result{Requeue: true, RequeueAfter: 30 * time.Second} var nilErr error = nil + var msg string // On every reconcile, we will call setCrdLifeCycleState // To understand this, please refer https://sdk.operatorframework.io/docs/building-operators/golang/advanced-topics/ @@ -146,7 +146,7 @@ func (r *ShardingDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.Req } } // Fetch the ProvShard instance - instance := &databasev1alpha1.ShardingDatabase{} + instance := &databasev4.ShardingDatabase{} err = r.Client.Get(context.TODO(), req.NamespacedName, instance) if err != nil { if errors.IsNotFound(err) { @@ -159,14 +159,10 @@ func (r *ShardingDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.Req return ctrl.Result{}, err } - _, instFlag := r.checkProvInstance(instance) - // assinging osh instance + instFlag := r.checkProvInstance(instance) if !instFlag { - // Sharding Topolgy Struct Assignment - // ====================================== - osh := &ShardingTopology{} - osh.Instance = instance - r.osh = append(r.osh, osh) + oshMap[instance.Name] = &OnsStatus{} + oshMap[instance.Name].Instance = instance } defer r.setCrdLifeCycleState(instance, &result, &err, &stateType) defer r.updateShardTopologyStatus(instance) @@ -187,30 +183,20 @@ func (r *ShardingDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.Req } // ======== Setting the flag and Index to be used later in this function ======== - idx, instFlag = r.checkProvInstance(instance) - if !instFlag { - //r.setCrdLifeCycleState(instance, &result, &err, stateType) - result = resultNq - return result, fmt.Errorf("DId not find the instance in checkProvInstance") - } + // instFlag = r.checkProvInstance(instance) + // if !instFlag { + //r.setCrdLifeCycleState(instance, &result, &err, stateType) + //// result = resultNq + // return result, fmt.Errorf("DId not find the instance in checkProvInstance") + // } // ================================ OCI Notification Provider =========== - r.getOnsConfigProvider(instance, idx) + r.getOnsConfigProvider(instance) // =============================== Checking Namespace ============== - if instance.Spec.Namespace == "" { - ///err = shardingv1.AddNamespace(instance, r.Client, r.Log) - //if err != nil { - // //r.setCrdLifeCycleState(instance, &result, &err, stateType) - // result = resultNq - // return result, err - // } - // } else { - instance.Spec.Namespace = "default" - } // ======================== Validate Specs ============== - err = r.validateSpex(instance, idx) + err = r.validateSpex(instance) if err != nil { //r.setCrdLifeCycleState(instance, &result, &err, stateType) result = resultNq @@ -239,6 +225,12 @@ func (r *ShardingDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.Req if len(instance.Spec.Catalog) > 0 { for i = 0; i < int32(len(instance.Spec.Catalog)); i++ { OraCatalogSpex = instance.Spec.Catalog[i] + if len(OraCatalogSpex.Name) > 9 { + msg = "Catalog Name cannot be greater than 9 characters." + err = fmt.Errorf(msg) + result = resultNq + return result, err + } // See if StatefulSets already exists and create if it doesn't result, err = r.deployStatefulSet(instance, shardingv1.BuildStatefulSetForCatalog(instance, OraCatalogSpex), "CATALOG") if err != nil { @@ -290,6 +282,12 @@ func (r *ShardingDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.Req // if user set replicasize greater than 1 but also set instance.Spec.OraDbPvcName then only one service will be created and one pod for i = 0; i < int32(len(instance.Spec.Shard)); i++ { OraShardSpex = instance.Spec.Shard[i] + if len(OraShardSpex.Name) > 9 { + msg = "Shard Name cannot be greater than 9 characters." + err = fmt.Errorf(msg) + result = resultNq + return result, err + } if !shardingv1.CheckIsDeleteFlag(OraShardSpex.IsDelete, instance, r.Log) { result, err = r.createService(instance, shardingv1.BuildServiceDefForShard(instance, 0, OraShardSpex, "local")) if err != nil { @@ -342,7 +340,7 @@ func (r *ShardingDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.Req // Loop will be requeued only if Shard Statefulset is not ready or not configured. // Till that time Reconcilation loop will remain in blocked state // if the err is return because of Shard is not ready then blocked state is rmeoved and reconcilation state is set - err = r.addPrimaryShards(instance, idx) + err = r.addPrimaryShards(instance) if err != nil { // time.Sleep(30 * time.Second) err = nilErr @@ -353,7 +351,7 @@ func (r *ShardingDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.Req // Loop will be requeued only if Standby Shard Statefulset is not ready or not configured. // Till that time Reconcilation loop will remain in blocked state // if the err is return because of Shard is not ready then blocked state is rmeoved and reconcilation state is - err = r.addStandbyShards(instance, idx) + err = r.addStandbyShards(instance) if err != nil { // time.Sleep(30 * time.Second) err = nilErr @@ -363,7 +361,7 @@ func (r *ShardingDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.Req // we don't need to run the requeue loop but still putting this condition to address any unkown situation // delShard function set the state to blocked and we do not allow any other operationn while delete is going on - err = r.delGsmShard(instance, idx) + err = r.delGsmShard(instance) if err != nil { // time.Sleep(30 * time.Second) err = nilErr @@ -376,13 +374,13 @@ func (r *ShardingDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.Req OraCatalogSpex = instance.Spec.Catalog[i] sfSet, catalogPod, err := r.validateInvidualCatalog(instance, OraCatalogSpex, int(i)) if err != nil { - shardingv1.LogMessages("INFO", "Catalog "+sfSet.Name+" is not in available state.", nil, instance, r.Log) + shardingv1.LogMessages("Error", "Catalog "+sfSet.Name+" is not in available state.", nil, instance, r.Log) result = resultNq return result, err } result, err = shardingv1.UpdateProvForCatalog(instance, OraCatalogSpex, r.Client, sfSet, catalogPod, r.Log) if err != nil { - shardingv1.LogMessages("INFO", "Error Occurred during catalog update operation.", nil, instance, r.Log) + shardingv1.LogMessages("Error", "Error Occurred during catalog update operation.", nil, instance, r.Log) result = resultNq return result, err } @@ -394,13 +392,13 @@ func (r *ShardingDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.Req if !shardingv1.CheckIsDeleteFlag(OraShardSpex.IsDelete, instance, r.Log) { sfSet, shardPod, err := r.validateShard(instance, OraShardSpex, int(i)) if err != nil { - shardingv1.LogMessages("INFO", "Shard "+sfSet.Name+" is not in available state.", nil, instance, r.Log) + shardingv1.LogMessages("Error", "Shard "+sfSet.Name+" is not in available state.", nil, instance, r.Log) result = resultNq return result, err } result, err = shardingv1.UpdateProvForShard(instance, OraShardSpex, r.Client, sfSet, shardPod, r.Log) if err != nil { - shardingv1.LogMessages("INFO", "Error Occurred during shard update operation..", nil, instance, r.Log) + shardingv1.LogMessages("Error", "Error Occurred during shard update operation..", nil, instance, r.Log) result = resultNq return result, err } @@ -412,19 +410,19 @@ func (r *ShardingDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.Req OraGsmSpex = instance.Spec.Gsm[i] sfSet, gsmPod, err := r.validateInvidualGsm(instance, OraGsmSpex, int(i)) if err != nil { - shardingv1.LogMessages("INFO", "Gsm "+sfSet.Name+" is not in available state.", nil, instance, r.Log) + shardingv1.LogMessages("Error", "Gsm "+sfSet.Name+" is not in available state.", nil, instance, r.Log) result = resultNq return result, err } result, err = shardingv1.UpdateProvForGsm(instance, OraGsmSpex, r.Client, sfSet, gsmPod, r.Log) if err != nil { - shardingv1.LogMessages("INFO", "Error Occurred during GSM update operation.", nil, instance, r.Log) + shardingv1.LogMessages("Error", "Error Occurred during GSM update operation.", nil, instance, r.Log) result = resultNq return result, err } } - stateType = string(databasev1alpha1.CrdReconcileCompeleteState) + stateType = string(databasev4.CrdReconcileCompeleteState) // r.setCrdLifeCycleState(instance, &result, &err, stateType) // Set error to ni to avoid reconcilation state reconcilation error as we are passing err to setCrdLifeCycleState @@ -439,7 +437,7 @@ func (r *ShardingDatabaseReconciler) Reconcile(ctx context.Context, req ctrl.Req // Check https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/controller#Options to under MaxConcurrentReconciles func (r *ShardingDatabaseReconciler) SetupWithManager(mgr ctrl.Manager) error { return ctrl.NewControllerManagedBy(mgr). - For(&databasev1alpha1.ShardingDatabase{}). + For(&databasev4.ShardingDatabase{}). Owns(&appsv1.StatefulSet{}). Owns(&corev1.Service{}). Owns(&corev1.Pod{}). @@ -457,65 +455,58 @@ func (r *ShardingDatabaseReconciler) eventFilterPredicate() predicate.Predicate return true }, UpdateFunc: func(e event.UpdateEvent) bool { + instance := &databasev4.ShardingDatabase{} if old, ok := e.ObjectOld.(*corev1.Secret); ok { if new, ok := e.ObjectNew.(*corev1.Secret); ok { - for i := 0; i < len(r.osh); i++ { - oshInst := r.osh[i] - if (new.Name == oshInst.Instance.Spec.DbSecret.Name) && (new.Name == old.Name) { - _, ok := old.Data[oshInst.Instance.Spec.DbSecret.PwdFileName] - if ok { - if !reflect.DeepEqual(old.Data[oshInst.Instance.Spec.DbSecret.PwdFileName], new.Data[oshInst.Instance.Spec.DbSecret.PwdFileName]) { - shardingv1.LogMessages("INFO", "Secret Changed", nil, oshInst.Instance, r.Log) - } + oshInst := instance + if (new.Name == oshInst.Spec.DbSecret.Name) && (new.Name == old.Name) { + _, ok := old.Data[oshInst.Spec.DbSecret.PwdFileName] + if ok { + if !reflect.DeepEqual(old.Data[oshInst.Spec.DbSecret.PwdFileName], new.Data[oshInst.Spec.DbSecret.PwdFileName]) { + shardingv1.LogMessages("INFO", "Secret Changed", nil, oshInst, r.Log) } - shardingv1.LogMessages("INFO", "Secret update block", nil, oshInst.Instance, r.Log) } + shardingv1.LogMessages("INFO", "Secret update block", nil, oshInst, r.Log) } } } return true }, DeleteFunc: func(e event.DeleteEvent) bool { + instance := &databasev4.ShardingDatabase{} _, podOk := e.Object.GetLabels()["statefulset.kubernetes.io/pod-name"] - for i := 0; i < len(r.osh); i++ { - if r.osh[i] != nil { - oshInst := r.osh[i] - if oshInst.deltopology == true { - break + if oshMap[instance.Name] != nil { + oshInst := instance + if instance.DeletionTimestamp == nil { - } - if e.Object.GetLabels()[string(databasev1alpha1.ShardingDelLabelKey)] == string(databasev1alpha1.ShardingDelLabelTrueValue) { - break + if e.Object.GetLabels()[string(databasev4.ShardingDelLabelKey)] == string(databasev4.ShardingDelLabelTrueValue) { } if podOk { delObj := e.Object.(*corev1.Pod) - if e.Object.GetLabels()["type"] == "Shard" && e.Object.GetLabels()["app"] == "OracleSharding" && e.Object.GetLabels()["oralabel"] == oshInst.Instance.Name { + if e.Object.GetLabels()["type"] == "Shard" && e.Object.GetLabels()["app"] == "OracleSharding" && e.Object.GetLabels()["oralabel"] == oshInst.Name { if delObj.DeletionTimestamp != nil { - go r.gsmInvitedNodeOp(oshInst.Instance, delObj.Name) + go r.gsmInvitedNodeOp(oshInst, delObj.Name) } } - if e.Object.GetLabels()["type"] == "Catalog" && e.Object.GetLabels()["app"] == "OracleSharding" && e.Object.GetLabels()["oralabel"] == oshInst.Instance.Name { + if e.Object.GetLabels()["type"] == "Catalog" && e.Object.GetLabels()["app"] == "OracleSharding" && e.Object.GetLabels()["oralabel"] == oshInst.Name { if delObj.DeletionTimestamp != nil { - go r.gsmInvitedNodeOp(oshInst.Instance, delObj.Name) + go r.gsmInvitedNodeOp(oshInst, delObj.Name) } } - } - } } - return true }, } } // ================ Function to check secret update============= -func (r *ShardingDatabaseReconciler) UpdateSecret(instance *databasev1alpha1.ShardingDatabase, kClient client.Client, logger logr.Logger) (ctrl.Result, error) { +func (r *ShardingDatabaseReconciler) UpdateSecret(instance *databasev4.ShardingDatabase, kClient client.Client, logger logr.Logger) (ctrl.Result, error) { sc := &corev1.Secret{} //var err error @@ -523,7 +514,7 @@ func (r *ShardingDatabaseReconciler) UpdateSecret(instance *databasev1alpha1.Sha // Reading a Secret var err error = kClient.Get(context.TODO(), types.NamespacedName{ Name: instance.Spec.DbSecret.Name, - Namespace: instance.Spec.Namespace, + Namespace: instance.Namespace, }, sc) if err != nil { @@ -534,43 +525,61 @@ func (r *ShardingDatabaseReconciler) UpdateSecret(instance *databasev1alpha1.Sha } // ================== Function to get the Notification controller ============== -func (r *ShardingDatabaseReconciler) getOnsConfigProvider(instance *databasev1alpha1.ShardingDatabase, idx int, -) { +func (r *ShardingDatabaseReconciler) getOnsConfigProvider(instance *databasev4.ShardingDatabase) { var err error - if instance.Spec.DbSecret.NsConfigMap != "" && instance.Spec.DbSecret.NsSecret != "" && r.osh[idx].onsProviderFlag != true { + if instance.Spec.DbSecret.NsConfigMap != "" && instance.Spec.DbSecret.NsSecret != "" && oshMap[instance.Name].OnsProviderFlag != true { cmName := instance.Spec.DbSecret.NsConfigMap secName := instance.Spec.DbSecret.NsSecret shardingv1.LogMessages("DEBUG", "Received parameters are "+shardingv1.GetFmtStr(cmName)+","+shardingv1.GetFmtStr(secName), nil, instance, r.Log) region, user, tenancy, passphrase, fingerprint, topicid := shardingv1.ReadConfigMap(cmName, instance, r.Client, r.Log) privatekey := shardingv1.ReadSecret(secName, instance, r.Client, r.Log) - r.osh[idx].topicid = topicid - r.osh[idx].onsProvider = common.NewRawConfigurationProvider(tenancy, user, region, fingerprint, privatekey, &passphrase) - r.osh[idx].rclient, err = ons.NewNotificationDataPlaneClientWithConfigurationProvider(r.osh[idx].onsProvider) + + oshMap[instance.Name].Topicid = topicid + oshMap[instance.Name].OnsProvider = common.NewRawConfigurationProvider(tenancy, user, region, fingerprint, privatekey, &passphrase) + //VV instance.Spec.TopicId = topicid + oshMap[instance.Name].Rclient, err = ons.NewNotificationDataPlaneClientWithConfigurationProvider(oshMap[instance.Name].OnsProvider) if err != nil { msg := "Error occurred in getting the OCI notification service based client." - r.osh[idx].onsProviderFlag = false + oshMap[instance.Name].OnsProviderFlag = false r.Log.Error(err, msg) shardingv1.LogMessages("Error", msg, nil, instance, r.Log) } else { - r.osh[idx].onsProviderFlag = true + oshMap[instance.Name].OnsProviderFlag = true } + } +} +func (r ShardingDatabaseReconciler) marshalOnsInfo(instance *databasev4.ShardingDatabase) (OnsStatus, error) { + onsData := OnsStatus{} + specBytes, err := instance.GetLastSuccessfulOnsInfo() + if err != nil { + shardingv1.LogMessages("Error", "error occurred while getting the data from getLastSuccessfulOnsInfo", nil, instance, r.Log) + return onsData, err + } else { + shardingv1.LogMessages("Error", "error occurred while getting the data from getLastSuccessfulOnsInfo and unmarshaling the object", nil, instance, r.Log) + err := json.Unmarshal(specBytes, &onsData) + if err != nil { + return onsData, err + } } + return onsData, nil } // ================== Function the Message ============== -func (r *ShardingDatabaseReconciler) sendMessage(instance *databasev1alpha1.ShardingDatabase, title string, body string) { - idx, instFlag := r.checkProvInstance(instance) +func (r *ShardingDatabaseReconciler) sendMessage(instance *databasev4.ShardingDatabase, title string, body string) { + instFlag := r.checkProvInstance(instance) if instFlag { - if r.osh[idx].onsProviderFlag { - shardingv1.SendNotification(title, body, instance, r.osh[idx].topicid, r.osh[idx].rclient, r.Log) + shardingv1.LogMessages("INFO", "sendMessage():instFlag true", nil, instance, r.Log) + if oshMap[instance.Name].OnsProviderFlag { + shardingv1.LogMessages("INFO", "sendMessage():OnsProviderFlag true", nil, instance, r.Log) + shardingv1.SendNotification(title, body, instance, oshMap[instance.Name].Topicid, oshMap[instance.Name].Rclient, r.Log) } } } -func (r *ShardingDatabaseReconciler) publishEvents(instance *databasev1alpha1.ShardingDatabase, eventMsg string, state string) { +func (r *ShardingDatabaseReconciler) publishEvents(instance *databasev4.ShardingDatabase, eventMsg string, state string) { - if state == string(databasev1alpha1.AvailableState) || state == string(databasev1alpha1.AddingShardState) || state == string(databasev1alpha1.ShardOnlineState) || state == string(databasev1alpha1.ProvisionState) || state == string(databasev1alpha1.DeletingState) || state == string(databasev1alpha1.Terminated) { + if state == string(databasev4.AvailableState) || state == string(databasev4.AddingShardState) || state == string(databasev4.ShardOnlineState) || state == string(databasev4.ProvisionState) || state == string(databasev4.DeletingState) || state == string(databasev4.Terminated) { r.Recorder.Eventf(instance, corev1.EventTypeNormal, "State Change", eventMsg) } else { r.Recorder.Eventf(instance, corev1.EventTypeWarning, "State Change", eventMsg) @@ -580,7 +589,7 @@ func (r *ShardingDatabaseReconciler) publishEvents(instance *databasev1alpha1.Sh } // ================== Function to check insytance deletion timestamp and activate the finalizer code ======== -func (r *ShardingDatabaseReconciler) finalizerShardingDatabaseInstance(instance *databasev1alpha1.ShardingDatabase, +func (r *ShardingDatabaseReconciler) finalizerShardingDatabaseInstance(instance *databasev4.ShardingDatabase, ) (error, bool) { isProvOShardToBeDeleted := instance.GetDeletionTimestamp() != nil @@ -619,8 +628,8 @@ func (r *ShardingDatabaseReconciler) finalizerShardingDatabaseInstance(instance } // ========================== FInalizer Section =================== -func (r *ShardingDatabaseReconciler) addFinalizer(instance *databasev1alpha1.ShardingDatabase) error { - reqLogger := r.Log.WithValues("instance.Spec.Namespace", instance.Spec.Namespace, "instance.Name", instance.Name) +func (r *ShardingDatabaseReconciler) addFinalizer(instance *databasev4.ShardingDatabase) error { + reqLogger := r.Log.WithValues("instance.Namespace", instance.Namespace, "instance.Name", instance.Name) controllerutil.AddFinalizer(instance, shardingv1.ShardingDatabaseFinalizer) // Update CR @@ -632,7 +641,7 @@ func (r *ShardingDatabaseReconciler) addFinalizer(instance *databasev1alpha1.Sha return nil } -func (r *ShardingDatabaseReconciler) finalizeShardingDatabase(instance *databasev1alpha1.ShardingDatabase) error { +func (r *ShardingDatabaseReconciler) finalizeShardingDatabase(instance *databasev4.ShardingDatabase) error { // TODO(user): Add the cleanup steps that the operator needs to do before the CR // can be deleted. Examples of finalizers include performing backups and deleting // resources that are not owned by this CR, like a PVC. @@ -641,10 +650,9 @@ func (r *ShardingDatabaseReconciler) finalizeShardingDatabase(instance *database var err error var pvcName string - idx, _ := r.checkProvInstance(instance) + r.checkProvInstance(instance) sfSetFound := &appsv1.StatefulSet{} svcFound := &corev1.Service{} - r.osh[idx].deltopology = true if len(instance.Spec.Shard) > 0 { for i = 0; i < int32(len(instance.Spec.Shard)); i++ { OraShardSpex := instance.Spec.Shard[i] @@ -831,41 +839,29 @@ func (r *ShardingDatabaseReconciler) finalizeShardingDatabase(instance *database } } - r.osh[idx].deltopology = false - //r.osh[idx].addSem.Release(1) - //r.osh[idx].delSem.Release(1) - //instance1 := &shardingv1alpha1.ProvShard{} - r.osh[idx].Instance = &databasev1alpha1.ShardingDatabase{} - - //r.osh[idx] = nil + oshMap[instance.Name].Instance = &databasev4.ShardingDatabase{} return nil } -//============== - // Get the current instance -func (r *ShardingDatabaseReconciler) checkProvInstance(instance *databasev1alpha1.ShardingDatabase, -) (int, bool) { +func (r *ShardingDatabaseReconciler) checkProvInstance(instance *databasev4.ShardingDatabase, +) bool { var status bool = false - var idx int - for i := 0; i < len(r.osh); i++ { - idx = i - if r.osh[i] != nil { - if !r.osh[i].deltopology { - if r.osh[i].Instance.Name == instance.Name { - status = true - break - } - } + if oshMap[instance.Name] != nil { + title := "checkProvInstance()" + message := "oshMap.Instance.Name=[" + oshMap[instance.Name].Instance.Name + "]. instance.Name=[" + instance.Name + "]." + shardingv1.LogMessages("INFO", title+":"+message, nil, instance, r.Log) + if oshMap[instance.Name].Instance.Name == instance.Name { + status = true } } - return idx, status + return status } // =========== validate Specs ============ -func (r *ShardingDatabaseReconciler) validateSpex(instance *databasev1alpha1.ShardingDatabase, idx int) error { +func (r *ShardingDatabaseReconciler) validateSpex(instance *databasev4.ShardingDatabase) error { var eventMsg string var eventErr string = "Spec Error" @@ -880,7 +876,7 @@ func (r *ShardingDatabaseReconciler) validateSpex(instance *databasev1alpha1.Sha if lastSuccSpec == nil { // Logic to check if inital Spec is good or not - err = r.checkShardingType(instance, idx) + err = r.checkShardingType(instance) if err != nil { return err } @@ -909,11 +905,6 @@ func (r *ShardingDatabaseReconciler) validateSpex(instance *databasev1alpha1.Sha } else { // if the last sucessful spec is not nil // check the parameters which cannot be changed - if lastSuccSpec.Namespace != instance.Spec.Namespace { - eventMsg = "ShardingDatabase CRD resource " + shardingv1.GetFmtStr(instance.Name) + " namespace changed from " + shardingv1.GetFmtStr(lastSuccSpec.Namespace) + " to " + shardingv1.GetFmtStr(instance.Spec.Namespace) + ". This change is not allowed." - r.Recorder.Eventf(instance, corev1.EventTypeWarning, eventErr, eventMsg) - return fmt.Errorf("instance spec has changed and namespace change is not supported") - } if lastSuccSpec.DbImage != instance.Spec.DbImage { eventMsg = "ShardingDatabase CRD resource " + shardingv1.GetFmtStr(instance.Name) + " DBImage changed from " + shardingv1.GetFmtStr(lastSuccSpec.DbImage) + " to " + shardingv1.GetFmtStr(instance.Spec.DbImage) + ". This change is not allowed." @@ -950,7 +941,7 @@ func (r *ShardingDatabaseReconciler) validateSpex(instance *databasev1alpha1.Sha return nil } -func (r *ShardingDatabaseReconciler) checkShardingType(instance *databasev1alpha1.ShardingDatabase, idx int) error { +func (r *ShardingDatabaseReconciler) checkShardingType(instance *databasev4.ShardingDatabase) error { var i, k int32 var regionFlag bool @@ -973,7 +964,7 @@ func (r *ShardingDatabaseReconciler) checkShardingType(instance *databasev1alpha // Check the ShardGroups/ Shard Space and Shard group Name // checkShrdGSR is Shardgroup/ShardSpace/ShardRegion -func (r *ShardingDatabaseReconciler) checkShardSpace(instance *databasev1alpha1.ShardingDatabase, OraShardSpex databasev1alpha1.ShardSpec) error { +func (r *ShardingDatabaseReconciler) checkShardSpace(instance *databasev4.ShardingDatabase, OraShardSpex databasev4.ShardSpec) error { if instance.Spec.ShardingType != "" { // Check for the Sharding Type and if it is USER do following @@ -992,7 +983,7 @@ func (r *ShardingDatabaseReconciler) checkShardSpace(instance *databasev1alpha1. // Check the ShardGroups/ Shard Space and Shard group Name // checkShrdGSR is Shardgroup/ShardSpace/ShardRegion -func (r *ShardingDatabaseReconciler) checkShardGroup(instance *databasev1alpha1.ShardingDatabase, OraShardSpex databasev1alpha1.ShardSpec) error { +func (r *ShardingDatabaseReconciler) checkShardGroup(instance *databasev4.ShardingDatabase, OraShardSpex databasev4.ShardSpec) error { // We need to check Shard Region and Shard Group for ShardingType='SYSTEM' and 'NATIVE' if strings.TrimSpace(strings.ToUpper(instance.Spec.ShardingType)) != "USER" { @@ -1011,7 +1002,7 @@ func (r *ShardingDatabaseReconciler) checkShardGroup(instance *databasev1alpha1. // Compare GSM Env Variables -func (r *ShardingDatabaseReconciler) comapreGsmEnvVariables(instance *databasev1alpha1.ShardingDatabase, lastSuccSpec *databasev1alpha1.ShardingDatabaseSpec) bool { +func (r *ShardingDatabaseReconciler) comapreGsmEnvVariables(instance *databasev4.ShardingDatabase, lastSuccSpec *databasev4.ShardingDatabaseSpec) bool { var eventMsg string var eventErr string = "Spec Error" var i, j int32 @@ -1036,7 +1027,7 @@ func (r *ShardingDatabaseReconciler) comapreGsmEnvVariables(instance *databasev1 return true } -func (r *ShardingDatabaseReconciler) comapreCatalogEnvVariables(instance *databasev1alpha1.ShardingDatabase, lastSuccSpec *databasev1alpha1.ShardingDatabaseSpec) bool { +func (r *ShardingDatabaseReconciler) comapreCatalogEnvVariables(instance *databasev4.ShardingDatabase, lastSuccSpec *databasev4.ShardingDatabaseSpec) bool { var eventMsg string var eventErr string = "Spec Error" var i, j int32 @@ -1061,7 +1052,7 @@ func (r *ShardingDatabaseReconciler) comapreCatalogEnvVariables(instance *databa return true } -func (r *ShardingDatabaseReconciler) comapreShardEnvVariables(instance *databasev1alpha1.ShardingDatabase, lastSuccSpec *databasev1alpha1.ShardingDatabaseSpec) bool { +func (r *ShardingDatabaseReconciler) comapreShardEnvVariables(instance *databasev4.ShardingDatabase, lastSuccSpec *databasev4.ShardingDatabaseSpec) bool { var eventMsg string var eventErr string = "Spec Error" var i, j int32 @@ -1088,21 +1079,21 @@ func (r *ShardingDatabaseReconciler) comapreShardEnvVariables(instance *database //===== Set the CRD resource life cycle state ======== -func (r *ShardingDatabaseReconciler) setCrdLifeCycleState(instance *databasev1alpha1.ShardingDatabase, result *ctrl.Result, err *error, stateType *string) { +func (r *ShardingDatabaseReconciler) setCrdLifeCycleState(instance *databasev4.ShardingDatabase, result *ctrl.Result, err *error, stateType *string) { var metaCondition metav1.Condition var updateFlag = false if *stateType == "ReconcileWaiting" { - metaCondition = shardingv1.GetMetaCondition(instance, result, err, *stateType, string(databasev1alpha1.CrdReconcileWaitingReason)) + metaCondition = shardingv1.GetMetaCondition(instance, result, err, *stateType, string(databasev4.CrdReconcileWaitingReason)) updateFlag = true } else if *stateType == "ReconcileComplete" { - metaCondition = shardingv1.GetMetaCondition(instance, result, err, *stateType, string(databasev1alpha1.CrdReconcileCompleteReason)) + metaCondition = shardingv1.GetMetaCondition(instance, result, err, *stateType, string(databasev4.CrdReconcileCompleteReason)) updateFlag = true } else if result.Requeue { - metaCondition = shardingv1.GetMetaCondition(instance, result, err, string(databasev1alpha1.CrdReconcileQueuedState), string(databasev1alpha1.CrdReconcileQueuedReason)) + metaCondition = shardingv1.GetMetaCondition(instance, result, err, string(databasev4.CrdReconcileQueuedState), string(databasev4.CrdReconcileQueuedReason)) updateFlag = true } else if *err != nil { - metaCondition = shardingv1.GetMetaCondition(instance, result, err, string(databasev1alpha1.CrdReconcileErrorState), string(databasev1alpha1.CrdReconcileErrorReason)) + metaCondition = shardingv1.GetMetaCondition(instance, result, err, string(databasev4.CrdReconcileErrorState), string(databasev4.CrdReconcileErrorReason)) updateFlag = true } else { @@ -1119,7 +1110,7 @@ func (r *ShardingDatabaseReconciler) setCrdLifeCycleState(instance *databasev1al } -func (r *ShardingDatabaseReconciler) validateGsmnCatalog(instance *databasev1alpha1.ShardingDatabase) error { +func (r *ShardingDatabaseReconciler) validateGsmnCatalog(instance *databasev4.ShardingDatabase) error { var err error _, _, err = r.validateCatalog(instance) if err != nil { @@ -1132,7 +1123,7 @@ func (r *ShardingDatabaseReconciler) validateGsmnCatalog(instance *databasev1alp return nil } -func (r *ShardingDatabaseReconciler) validateGsm(instance *databasev1alpha1.ShardingDatabase, +func (r *ShardingDatabaseReconciler) validateGsm(instance *databasev4.ShardingDatabase, ) (*appsv1.StatefulSet, *corev1.Pod, error) { //var err error var i int32 @@ -1160,7 +1151,7 @@ func (r *ShardingDatabaseReconciler) validateGsm(instance *databasev1alpha1.Shar return gsmSfSet, gsmPod, fmt.Errorf("GSM is not ready") } -func (r *ShardingDatabaseReconciler) validateInvidualGsm(instance *databasev1alpha1.ShardingDatabase, OraGsmSpex databasev1alpha1.GsmSpec, specId int, +func (r *ShardingDatabaseReconciler) validateInvidualGsm(instance *databasev4.ShardingDatabase, OraGsmSpex databasev4.GsmSpec, specId int, ) (*appsv1.StatefulSet, *corev1.Pod, error) { //var err error var i int32 @@ -1172,42 +1163,44 @@ func (r *ShardingDatabaseReconciler) validateInvidualGsm(instance *databasev1alp podList := &corev1.PodList{} var isPodExist bool + // VV : uninitialised variable 'i' being used. + i = int32(specId) gsmSfSet, err = shardingv1.CheckSfset(OraGsmSpex.Name, instance, r.Client) if err != nil { msg = "Unable to find GSM statefulset " + shardingv1.GetFmtStr(OraGsmSpex.Name) + "." - shardingv1.LogMessages("INFO", msg, nil, instance, r.Log) - r.updateGsmStatus(instance, int(i), string(databasev1alpha1.StatefulSetNotFound)) + shardingv1.LogMessages("Error", msg, nil, instance, r.Log) + r.updateGsmStatus(instance, int(i), string(databasev4.StatefulSetNotFound)) return gsmSfSet, gsmPod, err } podList, err = shardingv1.GetPodList(gsmSfSet.Name, "GSM", instance, r.Client) if err != nil { msg = "Unable to find any pod in statefulset " + shardingv1.GetFmtStr(gsmSfSet.Name) + "." - shardingv1.LogMessages("INFO", msg, nil, instance, r.Log) - r.updateGsmStatus(instance, int(i), string(databasev1alpha1.PodNotFound)) + shardingv1.LogMessages("Error", msg, nil, instance, r.Log) + r.updateGsmStatus(instance, int(i), string(databasev4.PodNotFound)) return gsmSfSet, gsmPod, err } isPodExist, gsmPod = shardingv1.PodListValidation(podList, gsmSfSet.Name, instance, r.Client) if !isPodExist { msg = "Unable to validate GSM " + shardingv1.GetFmtStr(gsmPod.Name) + " pod. GSM pod doesn't seems to be ready to accept the commands." - shardingv1.LogMessages("INFO", msg, nil, instance, r.Log) - r.updateGsmStatus(instance, int(i), string(databasev1alpha1.PodNotReadyState)) + shardingv1.LogMessages("Error", msg, nil, instance, r.Log) + r.updateGsmStatus(instance, int(i), string(databasev4.PodNotReadyState)) return gsmSfSet, gsmPod, fmt.Errorf("pod doesn't exist") } err = shardingv1.CheckGsmStatus(gsmPod.Name, instance, r.kubeClient, r.kubeConfig, r.Log) if err != nil { msg = "Unable to validate GSM director. GSM director doesn't seems to be ready to accept the commands." - shardingv1.LogMessages("INFO", msg, nil, instance, r.Log) - r.updateGsmStatus(instance, int(i), string(databasev1alpha1.ProvisionState)) + shardingv1.LogMessages("Error", msg, nil, instance, r.Log) + r.updateGsmStatus(instance, int(i), string(databasev4.ProvisionState)) return gsmSfSet, gsmPod, err } - r.updateGsmStatus(instance, specId, string(databasev1alpha1.AvailableState)) + r.updateGsmStatus(instance, specId, string(databasev4.AvailableState)) return gsmSfSet, gsmPod, nil } -func (r *ShardingDatabaseReconciler) validateCatalog(instance *databasev1alpha1.ShardingDatabase, +func (r *ShardingDatabaseReconciler) validateCatalog(instance *databasev4.ShardingDatabase, ) (*appsv1.StatefulSet, *corev1.Pod, error) { catalogSfSet := &appsv1.StatefulSet{} @@ -1236,7 +1229,7 @@ func (r *ShardingDatabaseReconciler) validateCatalog(instance *databasev1alpha1. } // === Validate Individual Catalog -func (r *ShardingDatabaseReconciler) validateInvidualCatalog(instance *databasev1alpha1.ShardingDatabase, OraCatalogSpex databasev1alpha1.CatalogSpec, specId int, +func (r *ShardingDatabaseReconciler) validateInvidualCatalog(instance *databasev4.ShardingDatabase, OraCatalogSpex databasev4.CatalogSpec, specId int, ) (*appsv1.StatefulSet, *corev1.Pod, error) { var err error @@ -1248,40 +1241,40 @@ func (r *ShardingDatabaseReconciler) validateInvidualCatalog(instance *databasev catalogSfSet, err = shardingv1.CheckSfset(OraCatalogSpex.Name, instance, r.Client) if err != nil { msg := "Unable to find Catalog statefulset " + shardingv1.GetFmtStr(OraCatalogSpex.Name) + "." - shardingv1.LogMessages("INFO", msg, nil, instance, r.Log) - r.updateCatalogStatus(instance, specId, string(databasev1alpha1.StatefulSetNotFound)) + shardingv1.LogMessages("Error", msg, nil, instance, r.Log) + r.updateCatalogStatus(instance, specId, string(databasev4.StatefulSetNotFound)) return catalogSfSet, catalogPod, err } podList, err = shardingv1.GetPodList(catalogSfSet.Name, "CATALOG", instance, r.Client) if err != nil { msg := "Unable to find any pod in statefulset " + shardingv1.GetFmtStr(catalogSfSet.Name) + "." - shardingv1.LogMessages("INFO", msg, nil, instance, r.Log) - r.updateCatalogStatus(instance, specId, string(databasev1alpha1.PodNotFound)) + shardingv1.LogMessages("Error", msg, nil, instance, r.Log) + r.updateCatalogStatus(instance, specId, string(databasev4.PodNotFound)) return catalogSfSet, catalogPod, err } isPodExist, catalogPod = shardingv1.PodListValidation(podList, catalogSfSet.Name, instance, r.Client) if !isPodExist { msg := "Unable to validate Catalog " + shardingv1.GetFmtStr(catalogSfSet.Name) + " pod. Catalog pod doesn't seems to be ready to accept the commands." - shardingv1.LogMessages("INFO", msg, nil, instance, r.Log) - r.updateCatalogStatus(instance, specId, string(databasev1alpha1.PodNotReadyState)) + shardingv1.LogMessages("Error", msg, nil, instance, r.Log) + r.updateCatalogStatus(instance, specId, string(databasev4.PodNotReadyState)) return catalogSfSet, catalogPod, fmt.Errorf("Pod doesn't exist") } err = shardingv1.ValidateDbSetup(catalogPod.Name, instance, r.kubeClient, r.kubeConfig, r.Log) if err != nil { msg := "Unable to validate Catalog. Catalog doesn't seems to be ready to accept the commands." - shardingv1.LogMessages("INFO", msg, nil, instance, r.Log) - r.updateCatalogStatus(instance, specId, string(databasev1alpha1.ProvisionState)) + shardingv1.LogMessages("Error", msg, nil, instance, r.Log) + r.updateCatalogStatus(instance, specId, string(databasev4.ProvisionState)) return catalogSfSet, catalogPod, err } - r.updateCatalogStatus(instance, specId, string(databasev1alpha1.AvailableState)) + r.updateCatalogStatus(instance, specId, string(databasev4.AvailableState)) return catalogSfSet, catalogPod, nil } // ======= Function to validate Shard -func (r *ShardingDatabaseReconciler) validateShard(instance *databasev1alpha1.ShardingDatabase, OraShardSpex databasev1alpha1.ShardSpec, specId int, +func (r *ShardingDatabaseReconciler) validateShard(instance *databasev4.ShardingDatabase, OraShardSpex databasev4.ShardSpec, specId int, ) (*appsv1.StatefulSet, *corev1.Pod, error) { var err error @@ -1291,39 +1284,39 @@ func (r *ShardingDatabaseReconciler) validateShard(instance *databasev1alpha1.Sh shardSfSet, err = shardingv1.CheckSfset(OraShardSpex.Name, instance, r.Client) if err != nil { msg := "Unable to find Shard statefulset " + shardingv1.GetFmtStr(OraShardSpex.Name) + "." - shardingv1.LogMessages("INFO", msg, nil, instance, r.Log) - r.updateShardStatus(instance, specId, string(databasev1alpha1.StatefulSetNotFound)) + shardingv1.LogMessages("Error", msg, nil, instance, r.Log) + r.updateShardStatus(instance, specId, string(databasev4.StatefulSetNotFound)) return shardSfSet, shardPod, err } podList, err := shardingv1.GetPodList(shardSfSet.Name, "SHARD", instance, r.Client) if err != nil { msg := "Unable to find any pod in statefulset " + shardingv1.GetFmtStr(shardSfSet.Name) + "." - shardingv1.LogMessages("INFO", msg, nil, instance, r.Log) - r.updateShardStatus(instance, specId, string(databasev1alpha1.PodNotFound)) + shardingv1.LogMessages("Error", msg, nil, instance, r.Log) + r.updateShardStatus(instance, specId, string(databasev4.PodNotFound)) return shardSfSet, shardPod, err } isPodExist, shardPod := shardingv1.PodListValidation(podList, shardSfSet.Name, instance, r.Client) if !isPodExist { msg := "Unable to validate Shard " + shardingv1.GetFmtStr(shardPod.Name) + " pod. Shard pod doesn't seems to be ready to accept the commands." - shardingv1.LogMessages("INFO", msg, nil, instance, r.Log) - r.updateShardStatus(instance, specId, string(databasev1alpha1.PodNotReadyState)) + shardingv1.LogMessages("Error", msg, nil, instance, r.Log) + r.updateShardStatus(instance, specId, string(databasev4.PodNotReadyState)) return shardSfSet, shardPod, err } err = shardingv1.ValidateDbSetup(shardPod.Name, instance, r.kubeClient, r.kubeConfig, r.Log) if err != nil { msg := "Unable to validate shard. Shard doesn't seems to be ready to accept the commands." - shardingv1.LogMessages("INFO", msg, nil, instance, r.Log) - r.updateShardStatus(instance, specId, string(databasev1alpha1.ProvisionState)) + shardingv1.LogMessages("Error", msg, nil, instance, r.Log) + r.updateShardStatus(instance, specId, string(databasev4.ProvisionState)) return shardSfSet, shardPod, err } - r.updateShardStatus(instance, specId, string(databasev1alpha1.AvailableState)) + r.updateShardStatus(instance, specId, string(databasev4.AvailableState)) return shardSfSet, shardPod, nil } // This function updates the shard topology over all -func (r *ShardingDatabaseReconciler) updateShardTopologyStatus(instance *databasev1alpha1.ShardingDatabase) { +func (r *ShardingDatabaseReconciler) updateShardTopologyStatus(instance *databasev4.ShardingDatabase) { //shardPod := &corev1.Pod{} //gsmSfSet := &appsv1.StatefulSet{} gsmPod := &corev1.Pod{} @@ -1340,7 +1333,7 @@ func (r *ShardingDatabaseReconciler) updateShardTopologyStatus(instance *databas } -func (r *ShardingDatabaseReconciler) updateShardTopologyShardsInGsm(instance *databasev1alpha1.ShardingDatabase, gsmPod *corev1.Pod) { +func (r *ShardingDatabaseReconciler) updateShardTopologyShardsInGsm(instance *databasev4.ShardingDatabase, gsmPod *corev1.Pod) { shardSfSet := &appsv1.StatefulSet{} //shardPod := &corev1.Pod{} //gsmSfSet := &appsv1.StatefulSet{} @@ -1366,7 +1359,7 @@ func (r *ShardingDatabaseReconciler) updateShardTopologyShardsInGsm(instance *da } } -func (r *ShardingDatabaseReconciler) updateGsmStatus(instance *databasev1alpha1.ShardingDatabase, specIdx int, state string) { +func (r *ShardingDatabaseReconciler) updateGsmStatus(instance *databasev4.ShardingDatabase, specIdx int, state string) { var currState string var eventMsg string @@ -1396,7 +1389,7 @@ func (r *ShardingDatabaseReconciler) updateGsmStatus(instance *databasev1alpha1. } } -func (r *ShardingDatabaseReconciler) updateCatalogStatus(instance *databasev1alpha1.ShardingDatabase, specIdx int, state string) { +func (r *ShardingDatabaseReconciler) updateCatalogStatus(instance *databasev4.ShardingDatabase, specIdx int, state string) { var eventMsg string var currState string var eventMsgFlag = true @@ -1404,7 +1397,7 @@ func (r *ShardingDatabaseReconciler) updateCatalogStatus(instance *databasev1alp name := instance.Spec.Catalog[specIdx].Name if len(instance.Status.Catalog) > 0 { - currState = shardingv1.GetGsmCatalogStatusKey(instance, name+"_"+string(databasev1alpha1.State)) + currState = shardingv1.GetGsmCatalogStatusKey(instance, name+"_"+string(databasev4.State)) if currState == state { eventMsgFlag = false } @@ -1423,14 +1416,14 @@ func (r *ShardingDatabaseReconciler) updateCatalogStatus(instance *databasev1alp } } -func (r *ShardingDatabaseReconciler) updateShardStatus(instance *databasev1alpha1.ShardingDatabase, specIdx int, state string) { +func (r *ShardingDatabaseReconciler) updateShardStatus(instance *databasev4.ShardingDatabase, specIdx int, state string) { var eventMsg string var currState string var eventMsgFlag = true name := instance.Spec.Shard[specIdx].Name if len(instance.Status.Shard) > 0 { - currState = shardingv1.GetGsmShardStatusKey(instance, name+"_"+string(databasev1alpha1.State)) + currState = shardingv1.GetGsmShardStatusKey(instance, name+"_"+string(databasev4.State)) if currState == state { eventMsgFlag = false } @@ -1449,7 +1442,7 @@ func (r *ShardingDatabaseReconciler) updateShardStatus(instance *databasev1alpha } } -func (r *ShardingDatabaseReconciler) updateGsmShardStatus(instance *databasev1alpha1.ShardingDatabase, name string, state string) { +func (r *ShardingDatabaseReconciler) updateGsmShardStatus(instance *databasev4.ShardingDatabase, name string, state string) { var eventMsg string var currState string var eventMsgFlag = true @@ -1480,7 +1473,7 @@ func (r *ShardingDatabaseReconciler) updateGsmShardStatus(instance *databasev1al } // This function add the Primary Shards in GSM -func (r *ShardingDatabaseReconciler) addPrimaryShards(instance *databasev1alpha1.ShardingDatabase, idx int) error { +func (r *ShardingDatabaseReconciler) addPrimaryShards(instance *databasev4.ShardingDatabase) error { //var result ctrl.Result var result ctrl.Result var i int32 @@ -1509,7 +1502,7 @@ func (r *ShardingDatabaseReconciler) addPrimaryShards(instance *databasev1alpha1 if !shardingv1.CheckIsDeleteFlag(OraShardSpex.IsDelete, instance, r.Log) { if setLifeCycleFlag != true { setLifeCycleFlag = true - stateType := string(databasev1alpha1.CrdReconcileWaitingState) + stateType := string(databasev4.CrdReconcileWaitingState) r.setCrdLifeCycleState(instance, &result, &err, &stateType) } // 1st Step is to check if Shard is in good state if not then just continue @@ -1551,14 +1544,14 @@ func (r *ShardingDatabaseReconciler) addPrimaryShards(instance *databasev1alpha1 last := fileName[strings.LastIndex(fileName, "/")+1:] fileName1 := last fsLoc := shardingv1.TmpLoc + "/" + fileName1 - _, _, _, err = shardingv1.KctlCopyFile(r.kubeClient, r.kubeConfig, instance, configrest, kclientset, r.Log, fmt.Sprintf("%s/%s:/%s", instance.Spec.Namespace, gsmPod.Name, fileName), fsLoc, "") + _, _, _, err = shardingv1.KctlCopyFile(r.kubeClient, r.kubeConfig, instance, configrest, kclientset, r.Log, fmt.Sprintf("%s/%s:/%s", instance.Namespace, gsmPod.Name, fileName), fsLoc, "") if err != nil { fmt.Printf("failed to copy file") //return err } // Copying it to Shard Pod - _, _, _, err = shardingv1.KctlCopyFile(r.kubeClient, r.kubeConfig, instance, configrest, kclientset, r.Log, fsLoc, fmt.Sprintf("%s/%s:/%s", instance.Spec.Namespace, OraShardSpex.Name+"-0", fsLoc), "") + _, _, _, err = shardingv1.KctlCopyFile(r.kubeClient, r.kubeConfig, instance, configrest, kclientset, r.Log, fsLoc, fmt.Sprintf("%s/%s:/%s", instance.Namespace, OraShardSpex.Name+"-0", fsLoc), "") if err != nil { fmt.Printf("failed to copy file") //return err @@ -1569,18 +1562,19 @@ func (r *ShardingDatabaseReconciler) addPrimaryShards(instance *databasev1alpha1 // If the shard doesn't exist in GSM then just add the shard statefulset and update GSM shard status // ADD Shard in GSM - r.updateGsmShardStatus(instance, OraShardSpex.Name, string(databasev1alpha1.AddingShardState)) + r.updateGsmShardStatus(instance, OraShardSpex.Name, string(databasev4.AddingShardState)) err = shardingv1.AddShardInGsm(gsmPod.Name, sparams, instance, r.kubeClient, r.kubeConfig, r.Log) if err != nil { - r.updateGsmShardStatus(instance, OraShardSpex.Name, string(databasev1alpha1.AddingShardErrorState)) - title = "Shard Addition Failure" - message = "Error occurred during shard " + shardingv1.GetFmtStr(OraShardSpex.Name) + " addition." - shardingv1.LogMessages("INFO", title+":"+message, nil, instance, r.Log) - if sentFailMsg[OraShardSpex.Name] != true { + r.updateGsmShardStatus(instance, OraShardSpex.Name, string(databasev4.AddingShardErrorState)) + title = instance.Namespace + ":Shard Addition Failure" + message = "TopicId:" + oshMap[instance.Name].Topicid + ":Error occurred during shard " + shardingv1.GetFmtStr(OraShardSpex.Name) + " addition." + shardingv1.LogMessages("Error", title+":"+message, nil, instance, r.Log) + msgKey := instance.Namespace + "-" + OraShardSpex.Name + if sentFailMsg[msgKey] != true { r.sendMessage(instance, title, message) } - sentFailMsg[OraShardSpex.Name] = true - sentCompleteMsg[OraShardSpex.Name] = false + sentFailMsg[msgKey] = true + sentCompleteMsg[msgKey] = false deployFlag = false } } @@ -1606,7 +1600,7 @@ func (r *ShardingDatabaseReconciler) addPrimaryShards(instance *databasev1alpha1 } // This function Check the online shard -func (r *ShardingDatabaseReconciler) verifyShards(instance *databasev1alpha1.ShardingDatabase, gsmPod *corev1.Pod, shardSfSet *appsv1.StatefulSet, OraShardSpex databasev1alpha1.ShardSpec) error { +func (r *ShardingDatabaseReconciler) verifyShards(instance *databasev4.ShardingDatabase, gsmPod *corev1.Pod, shardSfSet *appsv1.StatefulSet, OraShardSpex databasev4.ShardSpec) error { //var result ctrl.Result //var i int32 var err error @@ -1619,37 +1613,38 @@ func (r *ShardingDatabaseReconciler) verifyShards(instance *databasev1alpha1.Sha if err != nil { // If the shard doesn't exist in GSM then just delete the shard statefulset and update GSM shard status /// Terminate state means we will remove teh shard entry from GSM shard status - r.updateGsmShardStatus(instance, shardSfSet.Name, string(databasev1alpha1.ShardOnlineErrorState)) + r.updateGsmShardStatus(instance, shardSfSet.Name, string(databasev4.ShardOnlineErrorState)) if strings.ToUpper(instance.Spec.ReplicationType) != "NATIVE" { shardingv1.CancelChunksInGsm(gsmPod.Name, sparams, instance, r.kubeClient, r.kubeConfig, r.Log) } return err } oldStateStr := shardingv1.GetGsmShardStatus(instance, shardSfSet.Name) - r.updateGsmShardStatus(instance, shardSfSet.Name, string(databasev1alpha1.ShardOnlineState)) + r.updateGsmShardStatus(instance, shardSfSet.Name, string(databasev4.ShardOnlineState)) // Following logic will sent a email only once - if oldStateStr != string(databasev1alpha1.ShardOnlineState) { - title = "Shard Addition Completed" - message = "Shard addition completed for shard " + shardingv1.GetFmtStr(shardSfSet.Name) + " in GSM." + if oldStateStr != string(databasev4.ShardOnlineState) { + title = instance.Namespace + ":Shard Addition Completed" + message = "TopicId:" + oshMap[instance.Name].Topicid + ":Shard addition completed for shard " + shardingv1.GetFmtStr(shardSfSet.Name) + " in GSM." shardingv1.LogMessages("INFO", title+":"+message, nil, instance, r.Log) - if sentCompleteMsg[shardSfSet.Name] != true { + msgKey := instance.Namespace + "-" + shardSfSet.Name + if sentCompleteMsg[msgKey] != true { r.sendMessage(instance, title, message) } - sentCompleteMsg[shardSfSet.Name] = true - sentFailMsg[shardSfSet.Name] = false + sentCompleteMsg[msgKey] = true + sentFailMsg[msgKey] = false } return nil } -func (r *ShardingDatabaseReconciler) addStandbyShards(instance *databasev1alpha1.ShardingDatabase, idx int) error { +func (r *ShardingDatabaseReconciler) addStandbyShards(instance *databasev4.ShardingDatabase) error { //var result ctrl.Result return nil } // ========== Delete Shard Section==================== -func (r *ShardingDatabaseReconciler) delGsmShard(instance *databasev1alpha1.ShardingDatabase, idx int) error { +func (r *ShardingDatabaseReconciler) delGsmShard(instance *databasev4.ShardingDatabase) error { var result ctrl.Result var i int32 var err error @@ -1671,7 +1666,7 @@ func (r *ShardingDatabaseReconciler) delGsmShard(instance *databasev1alpha1.Shar if shardingv1.CheckIsDeleteFlag(OraShardSpex.IsDelete, instance, r.Log) { if setLifeCycleFlag != true { setLifeCycleFlag = true - stateType := string(databasev1alpha1.CrdReconcileWaitingState) + stateType := string(databasev4.CrdReconcileWaitingState) r.setCrdLifeCycleState(instance, &result, &err, &stateType) } // Step 1st to check if GSM is in good state if not then just return because you can't do anything @@ -1699,18 +1694,18 @@ func (r *ShardingDatabaseReconciler) delGsmShard(instance *databasev1alpha1.Shar // If the shard doesn't exist in GSM then just delete the shard statefulset and update GSM shard status /// Terminate state means we will remove teh shard entry from GSM shard status r.delShard(instance, shardSfSet.Name, shardSfSet, shardPod, int(i)) - r.updateGsmShardStatus(instance, OraShardSpex.Name, string(databasev1alpha1.Terminated)) - r.updateShardStatus(instance, int(i), string(databasev1alpha1.Terminated)) + r.updateGsmShardStatus(instance, OraShardSpex.Name, string(databasev4.Terminated)) + r.updateShardStatus(instance, int(i), string(databasev4.Terminated)) continue } // 4th step to check if shard is in GSM and shard is online if not then continue // CHeck before deletion if GSM is not ready set the Shard State to Delete Error - r.updateGsmShardStatus(instance, OraShardSpex.Name, string(databasev1alpha1.DeletingState)) + r.updateGsmShardStatus(instance, OraShardSpex.Name, string(databasev4.DeletingState)) err = shardingv1.CheckOnlineShardInGsm(gsmPod.Name, sparams, instance, r.kubeClient, r.kubeConfig, r.Log) if err != nil { // If the shard doesn't exist in GSM then just delete the shard statefulset and update GSM shard status /// Terminate state means we will remove teh shard entry from GSM shard status - r.updateGsmShardStatus(instance, OraShardSpex.Name, string(databasev1alpha1.DeleteErrorState)) + r.updateGsmShardStatus(instance, OraShardSpex.Name, string(databasev4.DeleteErrorState)) continue } // 5th Step @@ -1720,7 +1715,7 @@ func (r *ShardingDatabaseReconciler) delGsmShard(instance *databasev1alpha1.Shar if len(instance.Spec.ReplicationType) == 0 { err = shardingv1.MoveChunks(gsmPod.Name, sparams, instance, r.kubeClient, r.kubeConfig, r.Log) if err != nil { - r.updateGsmShardStatus(instance, OraShardSpex.Name, string(databasev1alpha1.ChunkMoveError)) + r.updateGsmShardStatus(instance, OraShardSpex.Name, string(databasev4.ChunkMoveError)) title = "Chunk Movement Failure" message = "Error occurred during chunk movement in shard " + shardingv1.GetFmtStr(OraShardSpex.Name) + " deletion." r.sendMessage(instance, title, message) @@ -1728,7 +1723,7 @@ func (r *ShardingDatabaseReconciler) delGsmShard(instance *databasev1alpha1.Shar err = shardingv1.InstanceShardPatch(instance, instance, r.Client, i, "isDelete", "failed") if err != nil { msg = "Error occurred while changing the isDelete value to failed in Spec struct" - shardingv1.LogMessages("INFO", msg, nil, instance, r.Log) + shardingv1.LogMessages("Error", msg, nil, instance, r.Log) return err } continue @@ -1752,9 +1747,9 @@ func (r *ShardingDatabaseReconciler) delGsmShard(instance *databasev1alpha1.Shar instance.Spec.Shard[i].IsDelete = "failed" err = shardingv1.InstanceShardPatch(instance, instance, r.Client, i, "isDelete", "failed") if err != nil { - // r.updateGsmShardStatus(instance, OraShardSpex.Name, string(databasev1alpha1.ChunkMoveError)) + // r.updateGsmShardStatus(instance, OraShardSpex.Name, string(databasev4.ChunkMoveError)) msg = "Error occurred while changing the isDelete value to failed in Spec struct" - shardingv1.LogMessages("INFO", msg, nil, instance, r.Log) + shardingv1.LogMessages("Error", msg, nil, instance, r.Log) // return err } return err @@ -1771,7 +1766,7 @@ func (r *ShardingDatabaseReconciler) delGsmShard(instance *databasev1alpha1.Shar if err != nil { msg = "Error occurred during shard" + shardingv1.GetFmtStr(OraShardSpex.Name) + "removal from Gsm" shardingv1.LogMessages("Error", msg, nil, instance, r.Log) - r.updateShardStatus(instance, int(i), string(databasev1alpha1.ShardRemoveError)) + r.updateShardStatus(instance, int(i), string(databasev4.ShardRemoveError)) instance.Spec.Shard[i].IsDelete = "failed" continue } @@ -1779,8 +1774,8 @@ func (r *ShardingDatabaseReconciler) delGsmShard(instance *databasev1alpha1.Shar // 8th Step // Delete the Statefulset as all the chunks has moved and Shard can be phyiscally deleted r.delShard(instance, shardSfSet.Name, shardSfSet, shardPod, int(i)) - r.updateGsmShardStatus(instance, OraShardSpex.Name, string(databasev1alpha1.Terminated)) - r.updateShardStatus(instance, int(i), string(databasev1alpha1.Terminated)) + r.updateGsmShardStatus(instance, OraShardSpex.Name, string(databasev4.Terminated)) + r.updateShardStatus(instance, int(i), string(databasev4.Terminated)) title = "Shard Deletion Completed" message = "Shard deletion completed for shard " + shardingv1.GetFmtStr(OraShardSpex.Name) + " in GSM." r.sendMessage(instance, title, message) @@ -1792,7 +1787,7 @@ func (r *ShardingDatabaseReconciler) delGsmShard(instance *databasev1alpha1.Shar } // This function delete the physical shard -func (r *ShardingDatabaseReconciler) delShard(instance *databasev1alpha1.ShardingDatabase, sfSetName string, sfSetFound *appsv1.StatefulSet, sfsetPod *corev1.Pod, specIdx int) { +func (r *ShardingDatabaseReconciler) delShard(instance *databasev4.ShardingDatabase, sfSetName string, sfSetFound *appsv1.StatefulSet, sfsetPod *corev1.Pod, specIdx int) { //var status bool var err error @@ -1803,7 +1798,7 @@ func (r *ShardingDatabaseReconciler) delShard(instance *databasev1alpha1.Shardin if err != nil { msg := "Failed to patch the Shard StatefulSet: " + sfSetFound.Name shardingv1.LogMessages("DEBUG", msg, err, instance, r.Log) - r.updateShardStatus(instance, specIdx, string(databasev1alpha1.LabelPatchingError)) + r.updateShardStatus(instance, specIdx, string(databasev4.LabelPatchingError)) return } @@ -1811,7 +1806,7 @@ func (r *ShardingDatabaseReconciler) delShard(instance *databasev1alpha1.Shardin if err != nil { msg = "Failed to delete Shard StatefulSet: " + shardingv1.GetFmtStr(sfSetFound.Name) shardingv1.LogMessages("DEBUG", msg, err, instance, r.Log) - r.updateShardStatus(instance, specIdx, string(databasev1alpha1.DeleteErrorState)) + r.updateShardStatus(instance, specIdx, string(databasev4.DeleteErrorState)) return } /// Delete External Service @@ -1842,14 +1837,14 @@ func (r *ShardingDatabaseReconciler) delShard(instance *databasev1alpha1.Shardin if err != nil { msg = "Failed to delete Shard pvc claim " + shardingv1.GetFmtStr(pvcName) shardingv1.LogMessages("DEBUG", msg, err, instance, r.Log) - r.updateShardStatus(instance, specIdx, string(databasev1alpha1.DeletePVCError)) + r.updateShardStatus(instance, specIdx, string(databasev4.DeletePVCError)) } } } // ======== GSM Invited Node ========== // Remove and add GSM invited node -func (r *ShardingDatabaseReconciler) gsmInvitedNodeOp(instance *databasev1alpha1.ShardingDatabase, objName string, +func (r *ShardingDatabaseReconciler) gsmInvitedNodeOp(instance *databasev4.ShardingDatabase, objName string, ) { var msg string @@ -1892,10 +1887,10 @@ func (r *ShardingDatabaseReconciler) gsmInvitedNodeOp(instance *databasev1alpha1 // ================================== CREATE FUNCTIONS ============================= // This function create a service based isExtern parameter set in the yaml file -func (r *ShardingDatabaseReconciler) createService(instance *databasev1alpha1.ShardingDatabase, +func (r *ShardingDatabaseReconciler) createService(instance *databasev4.ShardingDatabase, dep *corev1.Service, ) (ctrl.Result, error) { - reqLogger := r.Log.WithValues("Instance.Namespace", instance.Spec.Namespace, "Instance.Name", instance.Name) + reqLogger := r.Log.WithValues("Instance.Namespace", instance.Namespace, "Instance.Name", instance.Name) // See if Service already exists and create if it doesn't // We are getting error on nil pointer segment when r.scheme is null // Error : invalid memory address or nil pointer dereference" (runtime error: invalid memory address or nil pointer dereference) @@ -1912,7 +1907,7 @@ func (r *ShardingDatabaseReconciler) createService(instance *databasev1alpha1.Sh err := r.Client.Get(context.TODO(), types.NamespacedName{ Name: dep.Name, - Namespace: instance.Spec.Namespace, + Namespace: instance.Namespace, }, found) jsn, _ := json.Marshal(dep) @@ -1939,12 +1934,12 @@ func (r *ShardingDatabaseReconciler) createService(instance *databasev1alpha1.Sh } // This function deploy the statefulset -func (r *ShardingDatabaseReconciler) deployStatefulSet(instance *databasev1alpha1.ShardingDatabase, +func (r *ShardingDatabaseReconciler) deployStatefulSet(instance *databasev4.ShardingDatabase, dep *appsv1.StatefulSet, resType string, ) (ctrl.Result, error) { - reqLogger := r.Log.WithValues("Instance.Namespace", instance.Spec.Namespace, "Instance.Name", instance.Name) + reqLogger := r.Log.WithValues("Instance.Namespace", instance.Namespace, "Instance.Name", instance.Name) message := "Inside the deployStatefulSet function" shardingv1.LogMessages("DEBUG", message, nil, instance, r.Log) // See if StatefulSets already exists and create if it doesn't @@ -1961,7 +1956,7 @@ func (r *ShardingDatabaseReconciler) deployStatefulSet(instance *databasev1alpha found := &appsv1.StatefulSet{} err := r.Client.Get(context.TODO(), types.NamespacedName{ Name: dep.Name, - Namespace: instance.Spec.Namespace, + Namespace: instance.Namespace, }, found) jsn, _ := json.Marshal(dep) shardingv1.LogMessages("DEBUG", string(jsn), nil, instance, r.Log) @@ -1993,11 +1988,11 @@ func (r *ShardingDatabaseReconciler) deployStatefulSet(instance *databasev1alpha return ctrl.Result{}, nil } -func (r *ShardingDatabaseReconciler) checkShardState(instance *databasev1alpha1.ShardingDatabase) error { +func (r *ShardingDatabaseReconciler) checkShardState(instance *databasev4.ShardingDatabase) error { var i int32 var err error = nil - var OraShardSpex databasev1alpha1.ShardSpec + var OraShardSpex databasev4.ShardSpec var currState string var eventMsg string var msg string @@ -2016,34 +2011,29 @@ func (r *ShardingDatabaseReconciler) checkShardState(instance *databasev1alpha1. for i = 0; i < int32(len(instance.Spec.Shard)); i++ { OraShardSpex = instance.Spec.Shard[i] currState = shardingv1.GetGsmShardStatus(instance, OraShardSpex.Name) - if currState == string(databasev1alpha1.AddingShardState) { - eventMsg = "Shard Addition in progress. Requeuing" + if OraShardSpex.IsDelete == "failed" { + eventMsg = "Shard Deletion failed for [" + OraShardSpex.Name + "]. Retry shard deletion after manually moving the chunks. Requeuing" err = fmt.Errorf(eventMsg) - break - } else if currState == string(databasev1alpha1.DeletingState) { - eventMsg = "Shard Deletion in progress. Requeuing" + } else if currState == string(databasev4.AddingShardState) { + eventMsg = "Shard Addition in progress for [" + OraShardSpex.Name + "]. Requeuing" err = fmt.Errorf(eventMsg) - err = nil - break - } else if OraShardSpex.IsDelete == "failed" { - eventMsg = "Shard Deletion failed. Manual intervention required. Requeuing" + } else if currState == string(databasev4.DeletingState) { + eventMsg = "Shard Deletion in progress for [" + OraShardSpex.Name + "]. Requeuing" err = fmt.Errorf(eventMsg) - break - } else if currState == string(databasev1alpha1.DeleteErrorState) { - eventMsg = "Shard Deletion Error. Manual intervention required. Requeuing" + err = nil + } else if currState == string(databasev4.DeleteErrorState) { + eventMsg = "Shard Deletion Error for [" + OraShardSpex.Name + "]. Manual intervention required. Requeuing" err = fmt.Errorf(eventMsg) - break - } else if currState == string(databasev1alpha1.ShardRemoveError) { - eventMsg = "Shard Deletion Error. Manual intervention required. Requeuing" + } else if currState == string(databasev4.ShardRemoveError) { + eventMsg = "Shard Deletion Error for [" + OraShardSpex.Name + "]. Manual intervention required. Requeuing" err = fmt.Errorf(eventMsg) - break } else { - eventMsg = "checkShardState() : Shard State=[" + currState + "]" + eventMsg = "checkShardState() : Shard State[" + OraShardSpex.Name + "]=[" + currState + "]" shardingv1.LogMessages("INFO", eventMsg, nil, instance, r.Log) err = nil } + r.publishEvents(instance, eventMsg, currState) } - r.publishEvents(instance, eventMsg, currState) } return err } diff --git a/controllers/database/singleinstancedatabase_controller.go b/controllers/database/singleinstancedatabase_controller.go index a20fa1fd..13f2ec6f 100644 --- a/controllers/database/singleinstancedatabase_controller.go +++ b/controllers/database/singleinstancedatabase_controller.go @@ -46,7 +46,7 @@ import ( "strings" "time" - dbapi "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + dbapi "github.com/oracle/oracle-database-operator/apis/database/v4" dbcommons "github.com/oracle/oracle-database-operator/commons/database" "golang.org/x/text/cases" "golang.org/x/text/language" @@ -90,6 +90,11 @@ const singleInstanceDatabaseFinalizer = "database.oracle.com/singleinstancedatab var oemExpressUrl string +var ErrNotPhysicalStandby error = errors.New("database not in PHYSICAL_STANDBY role") +var ErrDBNotConfiguredWithDG error = errors.New("database is not configured with a dataguard configuration") +var ErrFSFOEnabledForDGConfig error = errors.New("database is configured with dataguard and FSFO enabled") +var ErrAdminPasswordSecretNotFound error = errors.New("Admin password secret for the database not found") + //+kubebuilder:rbac:groups=database.oracle.com,resources=singleinstancedatabases,verbs=get;list;watch;create;update;patch;delete //+kubebuilder:rbac:groups=database.oracle.com,resources=singleinstancedatabases/status,verbs=get;update;patch //+kubebuilder:rbac:groups=database.oracle.com,resources=singleinstancedatabases/finalizers,verbs=update @@ -220,7 +225,6 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct sidbRole, err := dbcommons.GetDatabaseRole(readyPod, r, r.Config, ctx, req) if sidbRole == "PRIMARY" { - // Update DB config result, err = r.updateDBConfig(singleInstanceDatabase, readyPod, ctx, req) if result.Requeue { @@ -243,8 +247,7 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct } } else { - // Database is in role of standby - if !singleInstanceDatabase.Status.DgBrokerConfigured { + if singleInstanceDatabase.Status.DgBroker == nil { err = SetupStandbyDatabase(r, singleInstanceDatabase, referredPrimaryDatabase, ctx, req) if err != nil { return requeueY, err @@ -280,6 +283,17 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct } + // manage snapshot database creation + if singleInstanceDatabase.Spec.ConvertToSnapshotStandby != singleInstanceDatabase.Status.ConvertToSnapshotStandby { + result, err := r.manageConvPhysicalToSnapshot(ctx, req) + if err != nil { + return requeueN, err + } + if result.Requeue { + return requeueY, nil + } + } + // Run Datapatch if strings.ToUpper(singleInstanceDatabase.Status.Role) == "PRIMARY" && singleInstanceDatabase.Status.DatafilesPatched != "true" { // add a blocking reconcile condition @@ -293,7 +307,7 @@ func (r *SingleInstanceDatabaseReconciler) Reconcile(ctx context.Context, req ct } } - // If LoadBalancer = true , ensure Connect String is updated + // This is to ensure that in case of LoadBalancer services the, the Load Balancer is ready to serve the requests if singleInstanceDatabase.Status.ConnectString == dbcommons.ValueUnavailable { r.Log.Info("Connect string not available for the database " + singleInstanceDatabase.Name) return requeueY, nil @@ -466,8 +480,20 @@ func (r *SingleInstanceDatabaseReconciler) validate(m *dbapi.SingleInstanceDatab m.Status.Pdbname = m.Spec.Pdbname m.Status.Persistence = m.Spec.Persistence m.Status.PrebuiltDB = m.Spec.Image.PrebuiltDB - + if m.Spec.CreateAs == "truecache" { + // Fetch the Primary database reference, required for all iterations + err = r.Get(ctx, types.NamespacedName{Namespace: req.Namespace, Name: m.Spec.PrimaryDatabaseRef}, rp) + if err != nil { + if apierrors.IsNotFound(err) { + r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, err.Error()) + r.Log.Info(err.Error()) + return requeueN, err + } + return requeueY, err + } + } if m.Spec.CreateAs == "clone" { + // Once a clone database has created , it has no link with its reference if m.Status.DatafilesCreated == "true" || !dbcommons.IsSourceDatabaseOnCluster(m.Spec.PrimaryDatabaseRef) { @@ -904,6 +930,49 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns return mounts }(), Env: func() []corev1.EnvVar { + if m.Spec.CreateAs == "truecache" { + return []corev1.EnvVar{ + { + Name: "SVC_HOST", + Value: m.Name, + }, + { + Name: "SVC_PORT", + Value: strconv.Itoa(int(dbcommons.CONTAINER_LISTENER_PORT)), + }, + { + Name: "ORACLE_CHARACTERSET", + Value: m.Spec.Charset, + }, + { + Name: "ORACLE_EDITION", + Value: m.Spec.Edition, + }, + { + Name: "TRUE_CACHE", + Value: "true", + }, + { + Name: "PRIMARY_DB_CONN_STR", + Value: func() string { + if dbcommons.IsSourceDatabaseOnCluster(m.Spec.PrimaryDatabaseRef) { + return rp.Name + ":" + strconv.Itoa(int(dbcommons.CONTAINER_LISTENER_PORT)) + "/" + rp.Spec.Sid + } + return m.Spec.PrimaryDatabaseRef + }(), + }, + { + Name: "PDB_TC_SVCS", + Value: func() string { + return strings.Join(m.Spec.TrueCacheServices, ";") + }(), + }, + { + Name: "ORACLE_HOSTNAME", + Value: m.Name, + }, + } + } // adding XE support, useful for dev/test/CI-CD if m.Spec.Edition == "express" || m.Spec.Edition == "free" { return []corev1.EnvVar{ @@ -1098,35 +1167,27 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns }(), Resources: func() corev1.ResourceRequirements { - if m.Spec.Resources.Requests != nil && m.Spec.Resources.Limits != nil { - return corev1.ResourceRequirements{ - Requests: corev1.ResourceList{ - "cpu": resource.MustParse(m.Spec.Resources.Requests.Cpu), - "memory": resource.MustParse(m.Spec.Resources.Requests.Memory), - }, - Limits: corev1.ResourceList{ - "cpu": resource.MustParse(m.Spec.Resources.Limits.Cpu), - "memory": resource.MustParse(m.Spec.Resources.Requests.Memory), - }, - } - } else if m.Spec.Resources.Requests != nil { - return corev1.ResourceRequirements{ - Requests: corev1.ResourceList{ - "cpu": resource.MustParse(m.Spec.Resources.Requests.Cpu), - "memory": resource.MustParse(m.Spec.Resources.Requests.Memory), - }, - } - } else if m.Spec.Resources.Limits != nil { - return corev1.ResourceRequirements{ - Limits: corev1.ResourceList{ - "cpu": resource.MustParse(m.Spec.Resources.Limits.Cpu), - "memory": resource.MustParse(m.Spec.Resources.Requests.Memory), - }, - } - } else { - return corev1.ResourceRequirements{} + var resourceReqRequests corev1.ResourceList = corev1.ResourceList{} + var resourceReqLimits corev1.ResourceList = corev1.ResourceList{} + + if m.Spec.Resources.Requests != nil && m.Spec.Resources.Requests.Cpu != "" { + resourceReqRequests["cpu"] = resource.MustParse(m.Spec.Resources.Requests.Cpu) + } + if m.Spec.Resources.Requests != nil && m.Spec.Resources.Requests.Memory != "" { + resourceReqRequests["memory"] = resource.MustParse(m.Spec.Resources.Requests.Memory) + } + + if m.Spec.Resources.Limits != nil && m.Spec.Resources.Limits.Cpu != "" { + resourceReqLimits["cpu"] = resource.MustParse(m.Spec.Resources.Limits.Cpu) + } + if m.Spec.Resources.Limits != nil && m.Spec.Resources.Limits.Memory != "" { + resourceReqLimits["memory"] = resource.MustParse(m.Spec.Resources.Limits.Memory) } + return corev1.ResourceRequirements{ + Requests: resourceReqRequests, + Limits: resourceReqLimits, + } }(), }}, @@ -1205,39 +1266,35 @@ func (r *SingleInstanceDatabaseReconciler) instantiatePodSpec(m *dbapi.SingleIns // // ############################################################################# func (r *SingleInstanceDatabaseReconciler) instantiateSVCSpec(m *dbapi.SingleInstanceDatabase, - svcName string, ports []corev1.ServicePort, svcType corev1.ServiceType) *corev1.Service { - svc := &corev1.Service{ - TypeMeta: metav1.TypeMeta{ - Kind: "Service", - }, - ObjectMeta: metav1.ObjectMeta{ - Name: svcName, - Namespace: m.Namespace, - Labels: map[string]string{ + svcName string, ports []corev1.ServicePort, svcType corev1.ServiceType, publishNotReadyAddress bool) *corev1.Service { + svc := dbcommons.NewRealServiceBuilder(). + SetName(svcName). + SetNamespace(m.Namespace). + SetLabels(func() map[string]string { + return map[string]string{ "app": m.Name, - }, - Annotations: func() map[string]string { - annotations := make(map[string]string) - if len(m.Spec.ServiceAnnotations) != 0 { - for key, value := range m.Spec.ServiceAnnotations { - annotations[key] = value - } + } + }()). + SetAnnotation(func() map[string]string { + annotations := make(map[string]string) + if len(m.Spec.ServiceAnnotations) != 0 { + for key, value := range m.Spec.ServiceAnnotations { + annotations[key] = value } - return annotations - }(), - }, - Spec: corev1.ServiceSpec{ - Ports: []corev1.ServicePort{}, - Selector: map[string]string{ + } + return annotations + }()). + SetPorts(ports). + SetSelector(func() map[string]string { + return map[string]string{ "app": m.Name, - }, - Type: svcType, - }, - } - svc.Spec.Ports = ports - // Set SingleInstanceDatabase instance as the owner and controller - ctrl.SetControllerReference(m, svc, r.Scheme) - return svc + } + }()). + SetPublishNotReadyAddresses(publishNotReadyAddress). + SetType(svcType). + Build() + ctrl.SetControllerReference(m, &svc, r.Scheme) + return &svc } // ############################################################################# @@ -1572,7 +1629,7 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex if getClusterSvcErr != nil && apierrors.IsNotFound(getClusterSvcErr) { // Create a new ClusterIP service ports := []corev1.ServicePort{{Name: "listener", Port: dbcommons.CONTAINER_LISTENER_PORT, Protocol: corev1.ProtocolTCP}} - svc := r.instantiateSVCSpec(m, clusterSvcName, ports, corev1.ServiceType("ClusterIP")) + svc := r.instantiateSVCSpec(m, clusterSvcName, ports, corev1.ServiceType("ClusterIP"), true) log.Info("Creating a new service", "Service.Namespace", svc.Namespace, "Service.Name", svc.Name) err := r.Create(ctx, svc) if err != nil { @@ -1774,7 +1831,7 @@ func (r *SingleInstanceDatabaseReconciler) createOrReplaceSVC(ctx context.Contex } // Create the service - svc := r.instantiateSVCSpec(m, extSvcName, ports, extSvcType) + svc := r.instantiateSVCSpec(m, extSvcName, ports, extSvcType, false) log.Info("Creating a new service", "Service.Namespace", svc.Namespace, "Service.Name", svc.Name) err := r.Create(ctx, svc) if err != nil { @@ -3129,7 +3186,7 @@ func (r *SingleInstanceDatabaseReconciler) cleanupSingleInstanceDatabase(req ctr return requeueY, nil } - if m.Status.DgBrokerConfigured { + if m.Status.DgBroker != nil { eventReason := "Cannot Delete" eventMsg := "database cannot be deleted as it is present in a DataGuard Broker configuration" r.Recorder.Eventf(m, corev1.EventTypeWarning, eventReason, eventMsg) @@ -3163,6 +3220,198 @@ func (r *SingleInstanceDatabaseReconciler) cleanupSingleInstanceDatabase(req ctr return requeueN, nil } +// ############################################################################################# +// +// Manage conversion of singleinstancedatabase from PHYSICAL_STANDBY To SNAPSHOT_STANDBY +// +// ############################################################################################# +func (r *SingleInstanceDatabaseReconciler) manageConvPhysicalToSnapshot(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + log := r.Log.WithValues("manageConvPhysicalToSnapshot", req.NamespacedName) + var singleInstanceDatabase dbapi.SingleInstanceDatabase + if err := r.Get(ctx, types.NamespacedName{Namespace: req.Namespace, Name: req.Name}, &singleInstanceDatabase); err != nil { + if apierrors.IsNotFound(err) { + log.Info("requested resource not found") + return requeueY, nil + } + log.Error(err, err.Error()) + return requeueY, err + } + + sidbReadyPod, err := GetDatabaseReadyPod(r, &singleInstanceDatabase, ctx, req) + if err != nil { + return requeueY, err + } + if sidbReadyPod.Name == "" { + log.Info("No ready Pod for the requested singleinstancedatabase") + return requeueY, nil + } + + if singleInstanceDatabase.Spec.ConvertToSnapshotStandby { + // Convert a PHYSICAL_STANDBY -> SNAPSHOT_STANDBY + singleInstanceDatabase.Status.Status = dbcommons.StatusUpdating + r.Status().Update(ctx, &singleInstanceDatabase) + if err := convertPhysicalStdToSnapshotStdDB(r, &singleInstanceDatabase, &sidbReadyPod, ctx, req); err != nil { + switch err { + case ErrNotPhysicalStandby: + r.Recorder.Event(&singleInstanceDatabase, corev1.EventTypeWarning, "Conversion to Snapshot Standby Not allowed", "Database not in physical standby role") + log.Info("Conversion to Snapshot Standby not allowed as database not in physical standby role") + return requeueY, nil + case ErrDBNotConfiguredWithDG: + // cannot convert to snapshot database + r.Recorder.Event(&singleInstanceDatabase, corev1.EventTypeWarning, "Conversion to Snapshot Standby Not allowed", "Database is not configured with dataguard") + log.Info("Conversion to Snapshot Standby not allowed as requested database is not configured with dataguard") + return requeueY, nil + case ErrFSFOEnabledForDGConfig: + r.Recorder.Event(&singleInstanceDatabase, corev1.EventTypeWarning, "Conversion to Snapshot Standby Not allowed", "Database is a FastStartFailover target") + log.Info("Conversion to Snapshot Standby Not allowed as database is a FastStartFailover target") + return requeueY, nil + case ErrAdminPasswordSecretNotFound: + r.Recorder.Event(&singleInstanceDatabase, corev1.EventTypeWarning, "Admin Password", "Database admin password secret not found") + log.Info("Database admin password secret not found") + return requeueY, nil + default: + log.Error(err, err.Error()) + return requeueY, nil + } + } + log.Info(fmt.Sprintf("Database %s converted to snapshot standby", singleInstanceDatabase.Name)) + singleInstanceDatabase.Status.ConvertToSnapshotStandby = true + singleInstanceDatabase.Status.Status = dbcommons.StatusReady + // Get database role and update the status + sidbRole, err := dbcommons.GetDatabaseRole(sidbReadyPod, r, r.Config, ctx, req) + if err != nil { + return requeueN, err + } + log.Info("Database "+singleInstanceDatabase.Name, "Database Role : ", sidbRole) + singleInstanceDatabase.Status.Role = sidbRole + r.Status().Update(ctx, &singleInstanceDatabase) + } else { + // Convert a SNAPSHOT_STANDBY -> PHYSICAL_STANDBY + singleInstanceDatabase.Status.Status = dbcommons.StatusUpdating + r.Status().Update(ctx, &singleInstanceDatabase) + if err := convertSnapshotStdToPhysicalStdDB(r, &singleInstanceDatabase, &sidbReadyPod, ctx, req); err != nil { + switch err { + default: + r.Log.Error(err, err.Error()) + return requeueY, nil + } + } + singleInstanceDatabase.Status.ConvertToSnapshotStandby = false + singleInstanceDatabase.Status.Status = dbcommons.StatusReady + // Get database role and update the status + sidbRole, err := dbcommons.GetDatabaseRole(sidbReadyPod, r, r.Config, ctx, req) + if err != nil { + return requeueN, err + } + log.Info("Database "+singleInstanceDatabase.Name, "Database Role : ", sidbRole) + singleInstanceDatabase.Status.Role = sidbRole + r.Status().Update(ctx, &singleInstanceDatabase) + } + + return requeueN, nil +} + +func convertPhysicalStdToSnapshotStdDB(r *SingleInstanceDatabaseReconciler, singleInstanceDatabase *dbapi.SingleInstanceDatabase, sidbReadyPod *corev1.Pod, ctx context.Context, req ctrl.Request) error { + log := r.Log.WithValues("convertPhysicalStdToSnapshotStdDB", req.NamespacedName) + log.Info(fmt.Sprintf("Checking the role %s database i.e %s", singleInstanceDatabase.Name, singleInstanceDatabase.Status.Role)) + if singleInstanceDatabase.Status.Role != "PHYSICAL_STANDBY" { + return ErrNotPhysicalStandby + } + + var dataguardBroker dbapi.DataguardBroker + log.Info(fmt.Sprintf("Checking if the database %s is configured with dgbroker or not ?", singleInstanceDatabase.Name)) + if singleInstanceDatabase.Status.DgBroker != nil { + if err := r.Get(ctx, types.NamespacedName{Namespace: singleInstanceDatabase.Namespace, Name: *singleInstanceDatabase.Status.DgBroker}, &dataguardBroker); err != nil { + if apierrors.IsNotFound(err) { + log.Info("Resource not found") + return errors.New("Dataguardbroker resource not found") + } + return err + } + log.Info(fmt.Sprintf("database %s is configured with dgbroker %s", singleInstanceDatabase.Name, *singleInstanceDatabase.Status.DgBroker)) + if fastStartFailoverStatus, _ := strconv.ParseBool(dataguardBroker.Status.FastStartFailover); fastStartFailoverStatus { + // not allowed to convert to snapshot standby + return ErrFSFOEnabledForDGConfig + } + } else { + // cannot convert to snapshot database + return ErrDBNotConfiguredWithDG + } + + // get singleinstancedatabase ready pod + // execute the dgmgrl command for conversion to snapshot database + // Exception handling + // Get Admin password for current primary database + var adminPasswordSecret corev1.Secret + if err := r.Get(context.TODO(), types.NamespacedName{Name: singleInstanceDatabase.Spec.AdminPassword.SecretName, Namespace: singleInstanceDatabase.Namespace}, &adminPasswordSecret); err != nil { + return err + } + var adminPassword string = string(adminPasswordSecret.Data[singleInstanceDatabase.Spec.AdminPassword.SecretKey]) + + // Connect to 'primarySid' db using dgmgrl and switchover to 'targetSidbSid' db to make 'targetSidbSid' db primary + if _, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", fmt.Sprintf(dbcommons.CreateAdminPasswordFile, adminPassword)); err != nil { + return err + } + + out, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", fmt.Sprintf("dgmgrl sys@%s \"convert database %s to snapshot standby;\" < admin.pwd", dataguardBroker.Status.PrimaryDatabase, singleInstanceDatabase.Status.Sid)) + if err != nil { + return err + } + log.Info(fmt.Sprintf("Convert to snapshot standby command output \n %s", out)) + + out, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", fmt.Sprintf("echo -e \"alter pluggable database %s open;\" | %s", singleInstanceDatabase.Status.Pdbname, dbcommons.SQLPlusCLI)) + if err != nil { + return err + } + log.Info(fmt.Sprintf("Open pluggable databases output \n %s", out)) + + return nil +} + +func convertSnapshotStdToPhysicalStdDB(r *SingleInstanceDatabaseReconciler, singleInstanceDatabase *dbapi.SingleInstanceDatabase, sidbReadyPod *corev1.Pod, ctx context.Context, req ctrl.Request) error { + log := r.Log.WithValues("convertSnapshotStdToPhysicalStdDB", req.NamespacedName) + + var dataguardBroker dbapi.DataguardBroker + if err := r.Get(ctx, types.NamespacedName{Namespace: singleInstanceDatabase.Namespace, Name: *singleInstanceDatabase.Status.DgBroker}, &dataguardBroker); err != nil { + if apierrors.IsNotFound(err) { + return errors.New("dataguardbroker resource not found") + } + return err + } + + var adminPasswordSecret corev1.Secret + if err := r.Get(context.TODO(), types.NamespacedName{Name: singleInstanceDatabase.Spec.AdminPassword.SecretName, Namespace: singleInstanceDatabase.Namespace}, &adminPasswordSecret); err != nil { + if apierrors.IsNotFound(err) { + return ErrAdminPasswordSecretNotFound + } + return err + } + var adminPassword string = string(adminPasswordSecret.Data[singleInstanceDatabase.Spec.AdminPassword.SecretKey]) + + // Connect to 'primarySid' db using dgmgrl and switchover to 'targetSidbSid' db to make 'targetSidbSid' db primary + _, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", + fmt.Sprintf(dbcommons.CreateAdminPasswordFile, adminPassword)) + if err != nil { + return err + } + log.Info("Converting snapshot standby to physical standby") + out, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", fmt.Sprintf("dgmgrl sys@%s \"convert database %s to physical standby;\" < admin.pwd", dataguardBroker.Status.PrimaryDatabase, singleInstanceDatabase.Status.Sid)) + if err != nil { + log.Error(err, err.Error()) + return err + } + log.Info(fmt.Sprintf("Database %s converted to physical standby \n %s", singleInstanceDatabase.Name, out)) + log.Info("opening the PDB for the database") + out, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", fmt.Sprintf("echo -e \"alter pluggable database %s open;\" | %s", singleInstanceDatabase.Status.Pdbname, dbcommons.SQLPlusCLI)) + if err != nil { + r.Log.Error(err, err.Error()) + return err + } + log.Info(fmt.Sprintf("PDB open command output %s", out)) + + return nil +} + // ############################################################################# // // SetupWithManager sets up the controller with the Manager diff --git a/controllers/dataguard/datagauard_errors.go b/controllers/dataguard/datagauard_errors.go new file mode 100644 index 00000000..94b2b0ea --- /dev/null +++ b/controllers/dataguard/datagauard_errors.go @@ -0,0 +1,47 @@ +/* +** Copyright (c) 2024 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package controllers + +import ( + "errors" +) + +var ErrSidbWithMutipleReplicas error = errors.New("SingleInstanceDatabase with multiple replicas is not supported") +var ErrCurrentPrimaryDatabaseNotReady error = errors.New("current primary database not ready") +var ErrCurrentPrimaryDatabaseNotFound error = errors.New("current primary database not found") diff --git a/controllers/dataguard/dataguard_utils.go b/controllers/dataguard/dataguard_utils.go new file mode 100644 index 00000000..4c16f82b --- /dev/null +++ b/controllers/dataguard/dataguard_utils.go @@ -0,0 +1,1061 @@ +/* +** Copyright (c) 2024 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package controllers + +import ( + "context" + "errors" + "fmt" + "strconv" + "strings" + "time" + + dbapi "github.com/oracle/oracle-database-operator/apis/database/v4" + dbcommons "github.com/oracle/oracle-database-operator/commons/database" + corev1 "k8s.io/api/core/v1" + apierrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/types" + ctrl "sigs.k8s.io/controller-runtime" + ctrllog "sigs.k8s.io/controller-runtime/pkg/log" +) + +// ############################################################################################################### +// +// Clean up necessary resources required prior to dataguardbroker resource deletion +// +// ############################################################################################################### +func cleanupDataguardBroker(r *DataguardBrokerReconciler, broker *dbapi.DataguardBroker, req ctrl.Request, ctx context.Context) error { + log := ctrllog.FromContext(ctx).WithValues("cleanupDataguardBroker", req.NamespacedName) + + log.Info(fmt.Sprintf("Cleaning for dataguard broker %v deletion", broker.Name)) + + // Fetch Primary Database Reference + var sidb dbapi.SingleInstanceDatabase + if err := r.Get(ctx, types.NamespacedName{Namespace: req.Namespace, Name: broker.GetCurrentPrimaryDatabase()}, &sidb); err != nil { + if apierrors.IsNotFound(err) { + log.Info(fmt.Sprintf("SingleInstanceDatabase %s deleted.", broker.GetCurrentPrimaryDatabase())) + return err + } + return err + } + + log.Info(fmt.Sprintf("The current primary database is %v", sidb.Name)) + + // Validate if Primary Database Reference is ready + if err := validateSidbReadiness(r, broker, &sidb, ctx, req); err != nil { + log.Info("Reconcile queued") + return err + } + + log.Info(fmt.Sprintf("The current primary database %v is ready and healthy", sidb.Name)) + + sidbReadyPod, _, _, _, err := dbcommons.FindPods(r, sidb.Spec.Image.Version, + sidb.Spec.Image.PullFrom, sidb.Name, sidb.Namespace, ctx, req) + if err != nil { + log.Error(err, err.Error()) + return err + } + + log.Info(fmt.Sprintf("Ready pod for the sidb %v is %v", sidb.Name, sidbReadyPod.Name)) + + out, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | dgmgrl / as sysdba ", dbcommons.RemoveDataguardConfiguration)) + if err != nil { + log.Error(err, err.Error()) + return err + } + log.Info("RemoveDataguardConfiguration Output") + log.Info(out) + + for _, databaseRef := range broker.Status.DatabasesInDataguardConfig { + + var standbyDatabase dbapi.SingleInstanceDatabase + if err := r.Get(ctx, types.NamespacedName{Namespace: req.Namespace, Name: databaseRef}, &standbyDatabase); err != nil { + if apierrors.IsNotFound(err) { + continue + } + log.Error(err, err.Error()) + return err + } + + // Set DgBrokerConfigured to false + standbyDatabase.Status.DgBroker = nil + if err := r.Status().Update(ctx, &standbyDatabase); err != nil { + r.Recorder.Eventf(&standbyDatabase, corev1.EventTypeWarning, "Updating Status", "DgBrokerConfigured status updation failed") + log.Info(fmt.Sprintf("Status updation for sidb %s failed", standbyDatabase.Name)) + return err + } + } + + log.Info("Successfully cleaned up Dataguard Broker") + return nil +} + +// ##################################################################################################### +// +// Validate readiness of the primary singleinstancedatabase specified +// +// ##################################################################################################### +func validateSidbReadiness(r *DataguardBrokerReconciler, broker *dbapi.DataguardBroker, sidb *dbapi.SingleInstanceDatabase, ctx context.Context, req ctrl.Request) error { + + log := r.Log.WithValues("validateSidbReadiness", req.NamespacedName) + + var adminPassword string + var sidbReadyPod corev1.Pod + + // Check if current primary singleinstancedatabase is "ready" + if sidb.Status.Status != dbcommons.StatusReady { + return ErrCurrentPrimaryDatabaseNotReady + } + + // ## FETCH THE SIDB REPLICAS . + sidbReadyPod, _, _, _, err := dbcommons.FindPods(r, sidb.Spec.Image.Version, + sidb.Spec.Image.PullFrom, sidb.Name, sidb.Namespace, ctx, req) + if err != nil { + log.Error(err, err.Error()) + return err + } + if sidbReadyPod.Name == "" { + log.Info("No ready pod avail for the singleinstancedatabase") + return ErrCurrentPrimaryDatabaseNotReady + } + + log.Info(fmt.Sprintf("Ready pod for the singleInstanceDatabase %s is %s", sidb.Name, sidbReadyPod.Name)) + + // Validate databaseRef Admin Password + var adminPasswordSecret corev1.Secret + err = r.Get(ctx, types.NamespacedName{Name: sidb.Spec.AdminPassword.SecretName, Namespace: sidb.Namespace}, &adminPasswordSecret) + if err != nil { + if apierrors.IsNotFound(err) { + //m.Status.Status = dbcommons.StatusError + eventReason := "Waiting" + eventMsg := "waiting for : " + sidb.Spec.AdminPassword.SecretName + " to get created" + r.Recorder.Eventf(broker, corev1.EventTypeNormal, eventReason, eventMsg) + r.Log.Info("Secret " + sidb.Spec.AdminPassword.SecretName + " Not Found") + return fmt.Errorf("adminPassword secret for singleinstancedatabase %v not found", sidb.Name) + } + log.Error(err, err.Error()) + return err + } + adminPassword = string(adminPasswordSecret.Data[sidb.Spec.AdminPassword.SecretKey]) + + out, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | %s", fmt.Sprintf(dbcommons.ValidateAdminPassword, adminPassword), dbcommons.GetSqlClient(sidb.Spec.Edition))) + if err != nil { + fastStartFailoverStatus, _ := strconv.ParseBool(broker.Status.FastStartFailover) + if strings.Contains(err.Error(), "dialing backend") && broker.Status.Status == dbcommons.StatusReady && fastStartFailoverStatus { + // Connection to the pod is failing after broker came up and running + // Might suggest disconnect or pod/vm going down + log.Info("Dialing connection error") + if err := updateReconcileStatus(r, broker, ctx, req); err != nil { + return err + } + } + log.Error(err, err.Error()) + return err + } + + if strings.Contains(out, "USER is \"SYS\"") { + log.Info("validated Admin password successfully") + } else if strings.Contains(out, "ORA-01017") { + //m.Status.Status = dbcommons.StatusError + eventReason := "Logon denied" + eventMsg := "invalid databaseRef admin password. secret: " + sidb.Spec.AdminPassword.SecretName + r.Recorder.Eventf(broker, corev1.EventTypeWarning, eventReason, eventMsg) + return fmt.Errorf("logon denied for singleinstancedatabase %v", sidb.Name) + } else { + return fmt.Errorf("%v", out) + } + + return nil +} + +// ############################################################################# +// +// Setup the requested dataguard Configuration +// +// ############################################################################# +func setupDataguardBrokerConfiguration(r *DataguardBrokerReconciler, broker *dbapi.DataguardBroker, sidb *dbapi.SingleInstanceDatabase, + ctx context.Context, req ctrl.Request) error { + + log := r.Log.WithValues("setupDataguardBrokerConfiguration", req.NamespacedName) + + // Get sidb ready pod for current primary database + sidbReadyPod, _, _, _, err := dbcommons.FindPods(r, sidb.Spec.Image.Version, + sidb.Spec.Image.PullFrom, sidb.Name, sidb.Namespace, ctx, req) + if err != nil { + log.Error(err, err.Error()) + return err + } + + log.Info(fmt.Sprintf("broker.Spec.StandbyDatabaseRefs are %v", broker.Spec.StandbyDatabaseRefs)) + + for _, database := range broker.Spec.StandbyDatabaseRefs { + + log.Info(fmt.Sprintf("adding database %v", database)) + + // Get the standby database resource + var standbyDatabase dbapi.SingleInstanceDatabase + err := r.Get(ctx, types.NamespacedName{Namespace: req.Namespace, Name: database}, &standbyDatabase) + if err != nil { + if apierrors.IsNotFound(err) { + eventReason := "Warning" + eventMsg := database + "not found" + r.Recorder.Eventf(broker, corev1.EventTypeNormal, eventReason, eventMsg) + continue + } + log.Error(err, err.Error()) + return err + } + + // validate standby database status + if standbyDatabase.Status.Status != dbcommons.StatusReady { + eventReason := "Waiting" + eventMsg := "Waiting for " + standbyDatabase.Name + " to be Ready" + r.Recorder.Eventf(broker, corev1.EventTypeNormal, eventReason, eventMsg) + log.Info(fmt.Sprintf("single instance database %s not ready yet", standbyDatabase.Name)) + continue + } + + // Check if dataguard broker is already configured for the standby database + if standbyDatabase.Status.DgBroker != nil { + log.Info("Dataguard broker for standbyDatabase : " + standbyDatabase.Name + " is already configured") + continue + } + + // Check if dataguard broker already has a database with the same SID + _, ok := broker.Status.DatabasesInDataguardConfig[strings.ToUpper(standbyDatabase.Status.Sid)] + if ok { + log.Info("A database with the same SID is already configured in the DG") + r.Recorder.Eventf(broker, corev1.EventTypeWarning, "Spec Error", "A database with the same SID "+standbyDatabase.Status.Sid+" is already configured in the DG") + continue + } + + broker.Status.Status = dbcommons.StatusCreating + r.Status().Update(ctx, broker) + + // ## FETCH THE STANDBY REPLICAS . + standbyDatabaseReadyPod, _, _, _, err := dbcommons.FindPods(r, sidb.Spec.Image.Version, + sidb.Spec.Image.PullFrom, standbyDatabase.Name, standbyDatabase.Namespace, ctx, req) + if err != nil { + log.Error(err, err.Error()) + return err + } + + var adminPasswordSecret corev1.Secret + if err := r.Get(ctx, types.NamespacedName{Name: sidb.Spec.AdminPassword.SecretName, Namespace: sidb.Namespace}, &adminPasswordSecret); err != nil { + return err + } + var adminPassword string = string(adminPasswordSecret.Data[sidb.Spec.AdminPassword.SecretKey]) + if err := setupDataguardBrokerConfigurationForGivenDB(r, broker, sidb, &standbyDatabase, standbyDatabaseReadyPod, sidbReadyPod, ctx, req, adminPassword); err != nil { + log.Error(err, fmt.Sprintf(" Error while setting up DG broker for the Database %v:%v", standbyDatabase.Status.Sid, standbyDatabase.Name)) + return err + } + if len(broker.Status.DatabasesInDataguardConfig) == 0 { + log.Info("DatabasesInDataguardConfig is nil") + broker.Status.DatabasesInDataguardConfig = make(map[string]string) + } + log.Info(fmt.Sprintf("adding %v:%v to the map", standbyDatabase.Status.Sid, standbyDatabase.Name)) + broker.Status.DatabasesInDataguardConfig[standbyDatabase.Status.Sid] = standbyDatabase.Name + r.Status().Update(ctx, broker) + // Update Databases + } + if len(broker.Status.DatabasesInDataguardConfig) == 0 { + broker.Status.DatabasesInDataguardConfig = make(map[string]string) + } + log.Info(fmt.Sprintf("adding primary database %v:%v to the map", sidb.Status.Sid, sidb.Name)) + broker.Status.DatabasesInDataguardConfig[sidb.Status.Sid] = sidb.Name + + eventReason := "DG Configuration up to date" + eventMsg := "" + + // Patch DataguardBroker Service to point selector to Current Primary Name + if err := patchService(r, broker, ctx, req); err != nil { + log.Error(err, err.Error()) + return err + } + + r.Recorder.Eventf(broker, corev1.EventTypeNormal, eventReason, eventMsg) + + return nil +} + +// ############################################################################# +// +// Set up dataguard Configuration for a given StandbyDatabase +// +// ############################################################################# +func setupDataguardBrokerConfigurationForGivenDB(r *DataguardBrokerReconciler, m *dbapi.DataguardBroker, n *dbapi.SingleInstanceDatabase, standbyDatabase *dbapi.SingleInstanceDatabase, + standbyDatabaseReadyPod corev1.Pod, sidbReadyPod corev1.Pod, ctx context.Context, req ctrl.Request, adminPassword string) error { + + log := r.Log.WithValues("setupDataguardBrokerConfigurationForGivenDB", req.NamespacedName) + + if standbyDatabaseReadyPod.Name == "" || sidbReadyPod.Name == "" { + return errors.New("no ready Pod for the singleinstancedatabase") + } + + // ## CHECK IF DG CONFIGURATION AVAILABLE IN PRIMARY DATABSE## + out, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | dgmgrl / as sysdba ", dbcommons.DBShowConfigCMD)) + if err != nil { + log.Error(err, err.Error()) + return err + } + log.Info("ShowConfiguration Output") + log.Info(out) + + if strings.Contains(out, "ORA-16525") { + log.Info("ORA-16525: The Oracle Data Guard broker is not yet available on Primary") + return fmt.Errorf("ORA-16525: The Oracle Data Guard broker is not yet available on Primary database %v", n.Name) + } + + _, err = dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, true, "bash", "-c", + fmt.Sprintf(dbcommons.CreateAdminPasswordFile, adminPassword)) + if err != nil { + log.Error(err, err.Error()) + return err + } + log.Info("DB Admin pwd file created") + + // ORA-16532: Oracle Data Guard broker configuration does not exist , so create one + if strings.Contains(out, "ORA-16532") { + if m.Spec.ProtectionMode == "MaxPerformance" { + // Construct the password file and dgbroker command file + out, err := dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf(dbcommons.CreateDGMGRLScriptFile, dbcommons.DataguardBrokerMaxPerformanceCMD)) + if err != nil { + log.Error(err, err.Error()) + return err + } + log.Info("DGMGRL command file creation output") + log.Info(out) + + // ## DG CONFIGURATION FOR PRIMARY DB || MODE : MAXPERFORMANCE ## + out, err = dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + "dgmgrl sys@${PRIMARY_DB_CONN_STR} @dgmgrl.cmd < admin.pwd && rm -rf dgmgrl.cmd") + if err != nil { + log.Error(err, err.Error()) + return err + } + log.Info("DgConfigurationMaxPerformance Output") + log.Info(out) + } else if m.Spec.ProtectionMode == "MaxAvailability" { + // ## DG CONFIGURATION FOR PRIMARY DB || MODE : MAX AVAILABILITY ## + out, err := dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf(dbcommons.CreateDGMGRLScriptFile, dbcommons.DataguardBrokerMaxAvailabilityCMD)) + if err != nil { + log.Error(err, err.Error()) + return err + } + log.Info("DGMGRL command file creation output") + log.Info(out) + + // ## DG CONFIGURATION FOR PRIMARY DB || MODE : MAXPERFORMANCE ## + out, err = dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + "dgmgrl sys@${PRIMARY_DB_CONN_STR} @dgmgrl.cmd < admin.pwd && rm -rf dgmgrl.cmd") + if err != nil { + log.Error(err, err.Error()) + return err + } + log.Info("DgConfigurationMaxAvailability Output") + log.Info(out) + } else { + log.Info("SPECIFY correct Protection Mode . Either MaxAvailability or MaxPerformance") + return err + } + + // ## SHOW CONFIGURATION DG + out, err := dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | dgmgrl / as sysdba ", dbcommons.DBShowConfigCMD)) + if err != nil { + log.Error(err, err.Error()) + return err + } else { + log.Info("ShowConfiguration Output") + log.Info(out) + } + // Set DG Configured status to true for this standbyDatabase and primary Database. so that in next reconcilation, we dont configure this again + n.Status.DgBroker = &m.Name + standbyDatabase.Status.DgBroker = &m.Name + r.Status().Update(ctx, standbyDatabase) + r.Status().Update(ctx, n) + // Remove admin pwd file + _, err = dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, true, "bash", "-c", + dbcommons.RemoveAdminPasswordFile) + if err != nil { + log.Error(err, err.Error()) + return err + } + log.Info("DB Admin pwd file removed") + + return err + } + + // DG Configuration Exists . So add the standbyDatabase to the existing DG Configuration + databases, err := GetDatabasesInDataGuardConfigurationWithRole(r, m, ctx, req) + if err != nil { + log.Info("Error while setting up the dataguard configuration") + log.Error(err, err.Error()) + return err + } + + // ## ADD DATABASE TO DG CONFIG , IF NOT PRESENT + found, _ := dbcommons.IsDatabaseFound(standbyDatabase.Spec.Sid, databases, "") + if found { + return err + } + primarySid := dbcommons.GetPrimaryDatabase(databases) + + // If user adds a new standby to a dg config when failover happened to one ot the standbys, we need to have current primary connect string + primaryConnectString := n.Name + ":1521/" + primarySid + if !strings.EqualFold(primarySid, n.Spec.Sid) { + primaryConnectString = m.Status.DatabasesInDataguardConfig[strings.ToUpper(primarySid)] + ":1521/" + primarySid + } + + if m.Spec.ProtectionMode == "MaxPerformance" { + // ## DG CONFIGURATION FOR PRIMARY DB || MODE : MAXPERFORMANCE ## + out, err := dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf(dbcommons.CreateDGMGRLScriptFile, dbcommons.DataguardBrokerAddDBMaxPerformanceCMD)) + if err != nil { + log.Error(err, err.Error()) + return err + } + log.Info("DGMGRL command file creation output") + log.Info(out) + + out, err = dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("dgmgrl sys@%s @dgmgrl.cmd < admin.pwd && rm -rf dgmgrl.cmd ", primaryConnectString)) + if err != nil { + log.Error(err, err.Error()) + return err + } + log.Info("DgConfigurationMaxPerformance Output") + log.Info(out) + + } else if m.Spec.ProtectionMode == "MaxAvailability" { + // ## DG CONFIGURATION FOR PRIMARY DB || MODE : MAX AVAILABILITY ## + out, err := dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf(dbcommons.CreateDGMGRLScriptFile, dbcommons.DataguardBrokerAddDBMaxAvailabilityCMD)) + if err != nil { + log.Error(err, err.Error()) + return err + } + log.Info("DGMGRL command file creation output") + log.Info(out) + + out, err = dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("dgmgrl sys@%s @dgmgrl.cmd < admin.pwd && rm -rf dgmgrl.cmd ", primaryConnectString)) + if err != nil { + log.Error(err, err.Error()) + return err + } + log.Info("DgConfigurationMaxAvailability Output") + log.Info(out) + + } else { + log.Info("SPECIFY correct Protection Mode . Either MaxAvailability or MaxPerformance") + log.Error(err, err.Error()) + return err + } + + // Remove admin pwd file + _, err = dbcommons.ExecCommand(r, r.Config, standbyDatabaseReadyPod.Name, standbyDatabaseReadyPod.Namespace, "", ctx, req, true, "bash", "-c", + dbcommons.RemoveAdminPasswordFile) + if err != nil { + log.Error(err, err.Error()) + return err + } + log.Info("DB Admin pwd file removed") + + // Set DG Configured status to true for this standbyDatabase. so that in next reconcilation, we dont configure this again + standbyDatabase.Status.DgBroker = &m.Name + r.Status().Update(ctx, standbyDatabase) + + return nil +} + +// ########################################################################################################### +// +// Patch the service for dataguardbroker resource to point selector to current Primary Name +// +// ########################################################################################################### +func patchService(r *DataguardBrokerReconciler, broker *dbapi.DataguardBroker, ctx context.Context, req ctrl.Request) error { + log := r.Log.WithValues("patchService", req.NamespacedName) + + primaryDatabaseRef := broker.Status.DatabasesInDataguardConfig[broker.Status.PrimaryDatabase] + var svc *corev1.Service = &corev1.Service{} + + // fetch the k8s service for the dataguardbroker resource + err := r.Get(ctx, types.NamespacedName{Name: req.Name, Namespace: req.Namespace}, svc) + if err != nil { + return err + } + + log.Info(fmt.Sprintf("Patching Service %s to point to the currPrimaryDatabase %s", svc.Name, primaryDatabaseRef)) + + // updating service selector for the primary database pod to attach itself to the service + svc.Spec.Selector["app"] = primaryDatabaseRef + if err = r.Update(ctx, svc); err != nil { + return err + } + log.Info(fmt.Sprintf("Patching service %s successful ", svc.Name)) + + // updating the dataguardbroker resource connect strings + broker.Status.ClusterConnectString = svc.Name + "." + svc.Namespace + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/DATAGUARD" + if broker.Spec.LoadBalancer { + if len(svc.Status.LoadBalancer.Ingress) > 0 { + lbAddress := svc.Status.LoadBalancer.Ingress[0].Hostname + if lbAddress == "" { + lbAddress = svc.Status.LoadBalancer.Ingress[0].IP + } + broker.Status.ExternalConnectString = lbAddress + ":" + fmt.Sprint(svc.Spec.Ports[0].Port) + "/DATAGUARD" + } + } else { + nodeip := dbcommons.GetNodeIp(r, ctx, req) + if nodeip != "" { + broker.Status.ExternalConnectString = nodeip + ":" + fmt.Sprint(svc.Spec.Ports[0].NodePort) + "/DATAGUARD" + } + } + log.Info("Updated connect strings to the dataguard broker") + return nil +} + +// ########################################################################################################### +// +// Update Reconcile Status +// +// ########################################################################################################### +func updateReconcileStatus(r *DataguardBrokerReconciler, broker *dbapi.DataguardBroker, ctx context.Context, req ctrl.Request) (err error) { + + log := r.Log.WithValues("updateReconcileStatus", req.NamespacedName) + + // fetch the singleinstancedatabase (database sid) and their role in the dataguard configuration + var databases []string + databases, err = GetDatabasesInDataGuardConfigurationWithRole(r, broker, ctx, req) + if err != nil { + log.Info("Problem when retrieving the databases in dg config") + broker.Status.Status = dbcommons.StatusNotReady + r.Status().Update(ctx, broker) + return nil + } + + // loop over all the databases to update the status of the dataguardbroker and the singleinstancedatabase + var standbyDatabases string = "" + for i := 0; i < len(databases); i++ { + splitstr := strings.Split(databases[i], ":") + database := strings.ToUpper(splitstr[0]) + var singleInstanceDatabase dbapi.SingleInstanceDatabase + err := r.Get(ctx, types.NamespacedName{Name: broker.Status.DatabasesInDataguardConfig[database], Namespace: req.Namespace}, &singleInstanceDatabase) + if err != nil { + return err + } + log.Info(fmt.Sprintf("Checking current role of %v is %v and its status is %v", broker.Status.DatabasesInDataguardConfig[database], strings.ToUpper(splitstr[1]), singleInstanceDatabase.Status.Role)) + if singleInstanceDatabase.Status.Role != strings.ToUpper(splitstr[1]) { + singleInstanceDatabase.Status.Role = strings.ToUpper(splitstr[1]) + r.Status().Update(ctx, &singleInstanceDatabase) + } + if strings.ToUpper(splitstr[1]) == "PRIMARY" && strings.ToUpper(database) != strings.ToUpper(broker.Status.PrimaryDatabase) { + log.Info("primary Database is " + strings.ToUpper(database)) + broker.Status.PrimaryDatabase = strings.ToUpper(database) + // patch the service with the current primary + } + if strings.ToUpper(splitstr[1]) == "PHYSICAL_STANDBY" { + if standbyDatabases != "" { + standbyDatabases += "," + strings.ToUpper(splitstr[0]) + } else { + standbyDatabases = strings.ToUpper(splitstr[0]) + } + } + } + + broker.Status.StandbyDatabases = standbyDatabases + broker.Status.ProtectionMode = broker.Spec.ProtectionMode + r.Status().Update(ctx, broker) + + // patch the dataguardbroker resource service + if err := patchService(r, broker, ctx, req); err != nil { + return err + } + + return nil +} + +// ##################################################################################################### +// +// Get the avail FSFO targets for a given singleinstancedatabase sid +// +// ##################################################################################################### +func GetFSFOTargets(databaseSid string, databasesInDgConfig map[string]string) (string, error) { + if _, ok := databasesInDgConfig[databaseSid]; !ok { + return "", fmt.Errorf("database %s not in dataguard config", databasesInDgConfig[databaseSid]) + } + var fsfoTarget []string + for dbSid, _ := range databasesInDgConfig { + if strings.Compare(databaseSid, dbSid) != 0 { + fsfoTarget = append(fsfoTarget, dbSid) + } + } + return strings.Join(fsfoTarget, ","), nil +} + +// ##################################################################################################### +// +// Set faststartfailover targets accordingly to dataguard configuration +// +// ##################################################################################################### +func setFSFOTargets(r *DataguardBrokerReconciler, broker *dbapi.DataguardBroker, ctx context.Context, req ctrl.Request) error { + + log := r.Log.WithValues("setFSFOTargets", req.NamespacedName) + + // fetch the current primary singleinstancedatabase + var currentPrimaryDatabase dbapi.SingleInstanceDatabase + err := r.Get(ctx, types.NamespacedName{Namespace: req.Namespace, Name: broker.GetCurrentPrimaryDatabase()}, ¤tPrimaryDatabase) + if err != nil { + if apierrors.IsNotFound(err) { + r.Log.Info("Resource not found") + return nil + } + r.Log.Error(err, err.Error()) + return err + } + + log.Info(fmt.Sprintf("current primary database for the dg config is %s", currentPrimaryDatabase.Name)) + + // fetch the singleinstancedatabase ready pod + sidbReadyPod, _, _, _, err := dbcommons.FindPods(r, currentPrimaryDatabase.Spec.Image.Version, + currentPrimaryDatabase.Spec.Image.PullFrom, currentPrimaryDatabase.Name, currentPrimaryDatabase.Namespace, ctx, req) + if err != nil { + log.Error(err, err.Error()) + return fmt.Errorf("error while fetching ready pod for %s", currentPrimaryDatabase.Name) + } + + log.Info(fmt.Sprintf("current primary database ready pod is %s", sidbReadyPod.Name)) + + // fetch singleinstancedatabase admin password + var adminPasswordSecret corev1.Secret + if err = r.Get(ctx, types.NamespacedName{Name: currentPrimaryDatabase.Spec.AdminPassword.SecretName, Namespace: currentPrimaryDatabase.Namespace}, &adminPasswordSecret); err != nil { + if apierrors.IsNotFound(err) { + //m.Status.Status = dbcommons.StatusError + eventReason := "Waiting" + eventMsg := "waiting for : " + currentPrimaryDatabase.Spec.AdminPassword.SecretName + " to get created" + r.Recorder.Eventf(broker, corev1.EventTypeNormal, eventReason, eventMsg) + r.Log.Info("Secret " + currentPrimaryDatabase.Spec.AdminPassword.SecretName + " Not Found") + return errors.New("admin password secret not found") + } + log.Error(err, err.Error()) + return err + } + adminPassword := string(adminPasswordSecret.Data[currentPrimaryDatabase.Spec.AdminPassword.SecretKey]) + + for databaseSid, databaseRef := range broker.Status.DatabasesInDataguardConfig { + // construct FSFO target for this database + fsfoTargets, err := GetFSFOTargets(databaseSid, broker.Status.DatabasesInDataguardConfig) + if err != nil { + return err + } + log.Info(fmt.Sprintf("Setting fast start failover target for the database %s to %s", databaseRef, fsfoTargets)) + out, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("echo -e \"EDIT DATABASE %s SET PROPERTY FASTSTARTFAILOVERTARGET=%s \" | dgmgrl sys/%s@%s ", + databaseSid, fsfoTargets, adminPassword, currentPrimaryDatabase.Status.Sid)) + if err != nil { + log.Error(err, err.Error()) + return err + } + log.Info("SETTING FSFO TARGET OUTPUT") + log.Info(out) + + out, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("echo -e \"SHOW DATABASE %s FASTSTARTFAILOVERTARGET \" | dgmgrl sys/%s@%s ", databaseSid, adminPassword, currentPrimaryDatabase.Status.Sid)) + if err != nil { + log.Error(err, err.Error()) + return err + } + log.Info("FSFO TARGETS OF " + databaseSid) + log.Info(out) + } + + // Set FSFO Targets according to the input yaml of broker + return nil +} + +// ############################################################################# +// +// Setup the requested dataguard configuration +// +// ############################################################################# +func createObserverPods(r *DataguardBrokerReconciler, broker *dbapi.DataguardBroker, ctx context.Context, req ctrl.Request) error { + + log := r.Log.WithValues("createObserverPods", req.NamespacedName) + + // fetch the current primary singleinstancedatabase resourcce + var currPrimaryDatabase dbapi.SingleInstanceDatabase + namespacedName := types.NamespacedName{ + Namespace: broker.Namespace, + Name: broker.GetCurrentPrimaryDatabase(), + } + if err := r.Get(ctx, namespacedName, &currPrimaryDatabase); err != nil { + if apierrors.IsNotFound(err) { + broker.Status.Status = dbcommons.StatusError + r.Recorder.Eventf(broker, corev1.EventTypeWarning, "SingleInstanceDatabase Not Found", fmt.Sprintf("SingleInstanceDatabase %s not found", namespacedName.Name)) + r.Log.Info(fmt.Sprintf("singleinstancedatabase %s not found", namespacedName.Name)) + return ErrCurrentPrimaryDatabaseNotFound + } + return err + } + + // fetch the dataguardbroker observer replicas + _, brokerReplicasFound, _, _, err := dbcommons.FindPods(r, "", "", broker.Name, broker.Namespace, ctx, req) + if err != nil { + log.Error(err, err.Error()) + return err + } + + if brokerReplicasFound > 0 { + return nil + } + + // Stop the already running observer + // find the avail pods for the currPrimaryDatabase + log.Info("Need to stop the observer if already running") + currPrimaryDatabaseReadyPod, _, _, _, err := dbcommons.FindPods(r, "", "", currPrimaryDatabase.Name, currPrimaryDatabase.Namespace, ctx, req) + if err != nil { + log.Error(err, err.Error()) + return err + } + if currPrimaryDatabaseReadyPod.Name == "" { + return errors.New("No ready pods avail ") + } + + // fetch singleinstancedatabase admin password + var adminPasswordSecret corev1.Secret + if err = r.Get(ctx, types.NamespacedName{Name: currPrimaryDatabase.Spec.AdminPassword.SecretName, Namespace: currPrimaryDatabase.Namespace}, &adminPasswordSecret); err != nil { + if apierrors.IsNotFound(err) { + //m.Status.Status = dbcommons.StatusError + eventReason := "Waiting" + eventMsg := "waiting for : " + currPrimaryDatabase.Spec.AdminPassword.SecretName + " to get created" + r.Recorder.Eventf(broker, corev1.EventTypeNormal, eventReason, eventMsg) + r.Log.Info("Secret " + currPrimaryDatabase.Spec.AdminPassword.SecretName + " Not Found") + return errors.New("admin password secret not found") + } + log.Error(err, err.Error()) + return err + } + adminPassword := string(adminPasswordSecret.Data[currPrimaryDatabase.Spec.AdminPassword.SecretKey]) + + out, err := dbcommons.ExecCommand(r, r.Config, currPrimaryDatabaseReadyPod.Name, currPrimaryDatabaseReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("echo -e \" STOP OBSERVER %s \" | dgmgrl sys/%s@%s ", broker.Name, adminPassword, currPrimaryDatabase.Status.Sid)) + if err != nil { + log.Error(err, err.Error()) + return err + } + log.Info(out) + // instantiate observer pod specification + pod := dbcommons.NewRealPodBuilder(). + SetNamespacedName(types.NamespacedName{ + Name: broker.Name + "-" + dbcommons.GenerateRandomString(5), + Namespace: broker.Namespace, + }). + SetLabels(map[string]string{ + "app": broker.Name, + "version": currPrimaryDatabase.Spec.Image.PullSecrets, + }). + SetTerminationGracePeriodSeconds(int64(30)). + SetNodeSelector(func() map[string]string { + var nsRule map[string]string = map[string]string{} + if len(broker.Spec.NodeSelector) != 0 { + for key, value := range broker.Spec.NodeSelector { + nsRule[key] = value + } + } + return nsRule + }()). + SetSecurityContext(corev1.PodSecurityContext{ + RunAsUser: func() *int64 { i := int64(54321); return &i }(), + FSGroup: func() *int64 { i := int64(54321); return &i }(), + }). + SetImagePullSecrets(currPrimaryDatabase.Spec.Image.PullSecrets). + AppendContainers(corev1.Container{ + Name: broker.Name, + Image: currPrimaryDatabase.Spec.Image.PullFrom, + Lifecycle: &corev1.Lifecycle{ + PreStop: &corev1.LifecycleHandler{ + Exec: &corev1.ExecAction{ + Command: []string{"/bin/sh", "-c", "/bin/echo -en 'shutdown abort;\n' | env ORACLE_SID=${ORACLE_SID^^} sqlplus -S / as sysdba"}, + }, + }, + }, + ImagePullPolicy: corev1.PullAlways, + Ports: []corev1.ContainerPort{{ContainerPort: 1521}, {ContainerPort: 5500}}, + + ReadinessProbe: &corev1.Probe{ + ProbeHandler: corev1.ProbeHandler{ + Exec: &corev1.ExecAction{ + Command: []string{"/bin/sh", "-c", "$ORACLE_BASE/checkDBLockStatus.sh"}, + }, + }, + InitialDelaySeconds: 20, + TimeoutSeconds: 20, + PeriodSeconds: 40, + }, + Env: []corev1.EnvVar{ + { + Name: "SVC_HOST", + Value: broker.Name, + }, + { + Name: "SVC_PORT", + Value: "1521", + }, + { + Name: "PRIMARY_DB_CONN_STR", + Value: currPrimaryDatabase.Name + ":1521/" + currPrimaryDatabase.Spec.Sid, + }, + { + Name: "DG_OBSERVER_ONLY", + Value: "true", + }, + { + Name: "DG_OBSERVER_NAME", + Value: broker.Name, + }, + { + // Sid used here only for Locking mechanism to work . + Name: "ORACLE_SID", + Value: "OBSRVR" + strings.ToUpper(currPrimaryDatabase.Spec.Sid), + }, + { + Name: "ORACLE_PWD", + ValueFrom: &corev1.EnvVarSource{ + SecretKeyRef: &corev1.SecretKeySelector{ + LocalObjectReference: corev1.LocalObjectReference{ + Name: currPrimaryDatabase.Spec.AdminPassword.SecretName, + }, + Key: currPrimaryDatabase.Spec.AdminPassword.SecretKey, + }, + }, + }, + }, + }). + Build() + + // set the ownership and lifecyle of the observer pod to the dataguardbroker resource + ctrl.SetControllerReference(broker, &pod, r.Scheme) + + log.Info("Creating a new POD", "POD.Namespace", pod.Namespace, "POD.Name", pod.Name) + if err = r.Create(ctx, &pod); err != nil { + log.Error(err, "Failed to create new POD", "pod.Namespace", pod.Namespace, "POD.Name", pod.Name) + return err + } + + // Waiting for Pod to get created as sometimes it takes some time to create a Pod . 30 seconds TImeout + timeout := 30 + err = dbcommons.WaitForStatusChange(r, pod.Name, broker.Namespace, ctx, req, time.Duration(timeout)*time.Second, "pod", "creation") + if err != nil { + log.Error(err, "Error in Waiting for Pod status for Creation", "pod.Namespace", pod.Namespace, "POD.Name", pod.Name) + return err + } + log.Info("Succesfully Created New Pod ", "POD.NAME : ", pod.Name) + + eventReason := "SUCCESS" + eventMsg := "" + r.Recorder.Eventf(broker, corev1.EventTypeNormal, eventReason, eventMsg) + + return nil +} + +// ############################################################################# +// +// Enable faststartfailover for the dataguard configuration +// +// ############################################################################# +func enableFSFOForDgConfig(r *DataguardBrokerReconciler, broker *dbapi.DataguardBroker, ctx context.Context, req ctrl.Request) error { + + log := r.Log.WithValues("enableFSFOForDgConfig", req.NamespacedName) + + // Get the current primary singleinstancedatabase resourcce + var sidb dbapi.SingleInstanceDatabase + namespacedName := types.NamespacedName{ + Namespace: broker.Namespace, + Name: broker.GetCurrentPrimaryDatabase(), + } + if err := r.Get(ctx, namespacedName, &sidb); err != nil { + if apierrors.IsNotFound(err) { + broker.Status.Status = dbcommons.StatusError + r.Recorder.Eventf(broker, corev1.EventTypeWarning, "SingleInstanceDatabase Not Found", fmt.Sprintf("SingleInstanceDatabase %s not found", sidb.Name)) + log.Info(fmt.Sprintf("singleinstancedatabase %s not found", namespacedName.Name)) + return ErrCurrentPrimaryDatabaseNotFound + } + return err + } + + // fetch singleinstancedatabase ready pod + sidbReadyPod, _, _, _, err := dbcommons.FindPods(r, sidb.Spec.Image.Version, + sidb.Spec.Image.PullFrom, sidb.Name, sidb.Namespace, ctx, req) + if err != nil { + log.Error(err, err.Error()) + return err + } + + // fetch singleinstancedatabase adminpassword secret + var adminPasswordSecret corev1.Secret + if err := r.Get(ctx, types.NamespacedName{Name: sidb.Spec.AdminPassword.SecretName, Namespace: sidb.Namespace}, &adminPasswordSecret); err != nil { + return err + } + var adminPassword string = string(adminPasswordSecret.Data[sidb.Spec.AdminPassword.SecretKey]) + + r.Recorder.Eventf(broker, corev1.EventTypeNormal, "Enabling FastStartFailover", fmt.Sprintf("Enabling FastStartFailover for the dataguard broker %s", broker.Name)) + log.Info(fmt.Sprintf("Enabling FastStartFailover for the dataguard broker %s", broker.Name)) + + // enable faststartfailover for the dataguard configuration + out, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | dgmgrl sys/%s@%s ", dbcommons.EnableFSFOCMD, adminPassword, sidb.Status.Sid)) + if err != nil { + r.Recorder.Eventf(broker, corev1.EventTypeWarning, "Enabling FastStartFailover failed", fmt.Sprintf("Enabling FastStartFailover for the dataguard broker %s failed", broker.Name)) + log.Error(err, err.Error()) + return err + } + log.Info("EnableFastStartFailover Output") + log.Info(out) + + r.Recorder.Eventf(broker, corev1.EventTypeNormal, "Enabling FastStartFailover successful", fmt.Sprintf("Enabling FastStartFailover for the dataguard broker %s successful", broker.Name)) + + return nil +} + +// ############################################################################# +// +// Disable faststartfailover for the dataguard configuration +// +// ############################################################################# +func disableFSFOForDGConfig(r *DataguardBrokerReconciler, broker *dbapi.DataguardBroker, ctx context.Context, req ctrl.Request) error { + + log := r.Log.WithValues("disableFSFOForDGConfig", req.NamespacedName) + + // Get the current primary singleinstancedatabase resource + var sidb dbapi.SingleInstanceDatabase + namespacedName := types.NamespacedName{ + Namespace: broker.Namespace, + Name: broker.GetCurrentPrimaryDatabase(), + } + if err := r.Get(ctx, namespacedName, &sidb); err != nil { + if apierrors.IsNotFound(err) { + broker.Status.Status = dbcommons.StatusError + r.Recorder.Eventf(broker, corev1.EventTypeWarning, "SingleInstanceDatabase Not Found", fmt.Sprintf("SingleInstanceDatabase %s not found", sidb.Name)) + log.Info(fmt.Sprintf("singleinstancedatabase %s not found", namespacedName.Name)) + return ErrCurrentPrimaryDatabaseNotFound + } + return err + } + + // fetch singleinstancedatabase ready pod + sidbReadyPod, _, _, _, err := dbcommons.FindPods(r, sidb.Spec.Image.Version, + sidb.Spec.Image.PullFrom, sidb.Name, sidb.Namespace, ctx, req) + if err != nil { + log.Error(err, err.Error()) + return err + } + + // fetch admin password for the singleinstancedatabase + var adminPasswordSecret corev1.Secret + if err := r.Get(ctx, types.NamespacedName{Name: sidb.Spec.AdminPassword.SecretName, Namespace: sidb.Namespace}, &adminPasswordSecret); err != nil { + return err + } + var adminPassword string = string(adminPasswordSecret.Data[sidb.Spec.AdminPassword.SecretKey]) + + r.Recorder.Eventf(broker, corev1.EventTypeNormal, "Disabling FastStartFailover", fmt.Sprintf("Disabling FastStartFailover for the dataguard broker %s", broker.Name)) + log.Info(fmt.Sprintf("Disabling FastStartFailover for the dataguard broker %s", broker.Name)) + + out, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | dgmgrl sys/%s@%s ", fmt.Sprintf(dbcommons.DisableFSFOCMD, broker.Name), adminPassword, sidb.Status.Sid)) + if err != nil { + r.Recorder.Eventf(broker, corev1.EventTypeWarning, "Disabling FastStartFailover failed", fmt.Sprintf("Disabling FastStartFailover for the dataguard broker %s failed", broker.Name)) + log.Error(err, err.Error()) + return err + } + log.Info("DisableFastStartFailover Output") + log.Info(out) + + r.Recorder.Eventf(broker, corev1.EventTypeNormal, "Disabling FastStartFailover", "faststartfailover disabled successfully") + log.Info("faststartfailover disabled successfully") + + return nil +} + +// ############################################################################# +// +// Get databases in dataguard configuration along with their roles +// +// ############################################################################# +func GetDatabasesInDataGuardConfigurationWithRole(r *DataguardBrokerReconciler, broker *dbapi.DataguardBroker, ctx context.Context, req ctrl.Request) ([]string, error) { + r.Log.Info(fmt.Sprintf("GetDatabasesInDataGuardConfiguration are %v", broker.GetDatabasesInDataGuardConfiguration())) + for _, database := range broker.GetDatabasesInDataGuardConfiguration() { + + var singleInstanceDatabase dbapi.SingleInstanceDatabase + if err := r.Get(context.TODO(), types.NamespacedName{Namespace: broker.Namespace, Name: database}, &singleInstanceDatabase); err != nil { + // log about the error while fetching the database + continue + } + + // Fetch the primary database ready pod + sidbReadyPod, _, _, _, err := dbcommons.FindPods(r, singleInstanceDatabase.Spec.Image.Version, + singleInstanceDatabase.Spec.Image.PullFrom, singleInstanceDatabase.Name, singleInstanceDatabase.Namespace, ctx, req) + if err != nil || sidbReadyPod.Name == "" { + continue + } + + // try out + out, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("echo -e \"%s\" | sqlplus -s / as sysdba ", dbcommons.DataguardBrokerGetDatabaseCMD)) + if err != nil || strings.Contains(out, "no rows selected") && strings.Contains(out, "ORA-") { + continue + } + + r.Log.Info(fmt.Sprintf("sidbReadyPod is %v \n output of the exec is %v \n and output contains ORA- is %v", sidbReadyPod.Name, out, strings.Contains(out, "ORA-"))) + + out1 := strings.Replace(out, " ", "_", -1) + // filtering output and storing databses in dg configuration in "databases" slice + databases := strings.Fields(out1) + + // first 2 values in the slice will be column name(DATABASES) and a seperator(--------------) . so take the slice from position [2:] + databases = databases[2:] + return databases, nil + } + + return []string{}, errors.New("cannot get databases in dataguard configuration") +} diff --git a/controllers/dataguard/dataguardbroker_controller.go b/controllers/dataguard/dataguardbroker_controller.go new file mode 100644 index 00000000..4d7ae044 --- /dev/null +++ b/controllers/dataguard/dataguardbroker_controller.go @@ -0,0 +1,513 @@ +/* +** Copyright (c) 2024 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package controllers + +import ( + "context" + "errors" + "fmt" + "strconv" + "time" + + "github.com/go-logr/logr" + dbapi "github.com/oracle/oracle-database-operator/apis/database/v4" + dbcommons "github.com/oracle/oracle-database-operator/commons/database" + corev1 "k8s.io/api/core/v1" + apierrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/types" + "k8s.io/client-go/rest" + "k8s.io/client-go/tools/record" + ctrl "sigs.k8s.io/controller-runtime" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/controller" + "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" +) + +// DataguardBrokerReconciler reconciles a DataguardBroker object +type DataguardBrokerReconciler struct { + client.Client + Log logr.Logger + Scheme *runtime.Scheme + Config *rest.Config + Recorder record.EventRecorder +} + +const dataguardBrokerFinalizer = "database.oracle.com/dataguardbrokerfinalizer" + +//+kubebuilder:rbac:groups=database.oracle.com,resources=dataguardbrokers,verbs=get;list;watch;create;update;patch;delete +//+kubebuilder:rbac:groups=database.oracle.com,resources=dataguardbrokers/status,verbs=get;update;patch +//+kubebuilder:rbac:groups=database.oracle.com,resources=dataguardbrokers/finalizers,verbs=update +//+kubebuilder:rbac:groups="",resources=pods;pods/log;pods/exec;persistentvolumeclaims;services,verbs=create;delete;get;list;patch;update;watch +//+kubebuilder:rbac:groups="",resources=events,verbs=create;patch + +func (r *DataguardBrokerReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + + log := r.Log.WithValues("reconciler", req.NamespacedName) + + log.Info("Reconcile requested") + + // Get the dataguardbroker resource if already exists + var dataguardBroker dbapi.DataguardBroker + if err := r.Get(ctx, types.NamespacedName{Namespace: req.Namespace, Name: req.Name}, &dataguardBroker); err != nil { + if apierrors.IsNotFound(err) { + log.Info("Resource deleted") + return ctrl.Result{Requeue: false}, nil + } + return ctrl.Result{Requeue: false}, err + } + + // Manage dataguardbroker deletion + if !dataguardBroker.DeletionTimestamp.IsZero() { + return r.manageDataguardBrokerDeletion(&dataguardBroker, ctx, req) + } + + // initialize the dataguardbroker resource status + if dataguardBroker.Status.Status == "" { + r.Recorder.Eventf(&dataguardBroker, corev1.EventTypeNormal, "Status Initialization", "initializing status fields for the resource") + log.Info("Initializing status fields") + dataguardBroker.Status.Status = dbcommons.StatusCreating + dataguardBroker.Status.ExternalConnectString = dbcommons.ValueUnavailable + dataguardBroker.Status.ClusterConnectString = dbcommons.ValueUnavailable + dataguardBroker.Status.FastStartFailover = "false" + if len(dataguardBroker.Status.DatabasesInDataguardConfig) == 0 { + dataguardBroker.Status.DatabasesInDataguardConfig = map[string]string{} + } + } + + // Always refresh status before a reconcile + defer r.Status().Update(ctx, &dataguardBroker) + + // Mange DataguardBroker Creation + result, err := r.manageDataguardBrokerCreation(&dataguardBroker, ctx, req) + if err != nil { + return ctrl.Result{Requeue: false}, err + } + if result.Requeue { + return result, nil + } + + // manage enabling and disabling faststartfailover + if dataguardBroker.Spec.FastStartFailover { + + for _, DbResource := range dataguardBroker.Status.DatabasesInDataguardConfig { + var singleInstanceDatabase dbapi.SingleInstanceDatabase + if err := r.Get(ctx, types.NamespacedName{Namespace: req.Namespace, Name: DbResource}, &singleInstanceDatabase); err != nil { + return ctrl.Result{Requeue: false}, err + } + r.Log.Info("Check the role for database", "database", singleInstanceDatabase.Name, "role", singleInstanceDatabase.Status.Role) + if singleInstanceDatabase.Status.Role == "SNAPSHOT_STANDBY" { + r.Recorder.Eventf(&dataguardBroker, corev1.EventTypeWarning, "Enabling FSFO failed", "database %s is a snapshot database", singleInstanceDatabase.Name) + r.Log.Info("Enabling FSFO failed, one of the database is a snapshot database", "snapshot database", singleInstanceDatabase.Name) + return ctrl.Result{Requeue: true}, nil + } + } + + // set faststartfailover targets for all the singleinstancedatabases in the dataguard configuration + if err := setFSFOTargets(r, &dataguardBroker, ctx, req); err != nil { + return ctrl.Result{Requeue: false}, err + } + + // enable faststartfailover in the dataguard configuration + if err := enableFSFOForDgConfig(r, &dataguardBroker, ctx, req); err != nil { + return ctrl.Result{Requeue: false}, err + } + + // create Observer Pod + if err := createObserverPods(r, &dataguardBroker, ctx, req); err != nil { + return ctrl.Result{Requeue: false}, err + } + + // set faststartfailover status to true + dataguardBroker.Status.FastStartFailover = "true" + + } else { + + // disable faststartfailover + if err := disableFSFOForDGConfig(r, &dataguardBroker, ctx, req); err != nil { + return ctrl.Result{Requeue: false}, err + } + + // delete Observer Pod + observerReadyPod, _, _, _, err := dbcommons.FindPods(r, "", "", dataguardBroker.Name, dataguardBroker.Namespace, ctx, req) + if err != nil { + return ctrl.Result{Requeue: false}, err + } + if observerReadyPod.Name != "" { + if err := r.Delete(ctx, &observerReadyPod); err != nil { + return ctrl.Result{Requeue: false}, err + } + } + + r.Recorder.Eventf(&dataguardBroker, corev1.EventTypeNormal, "Observer Deleted", "database observer pod deleted") + log.Info("database observer deleted") + + // set faststartfailover status to false + dataguardBroker.Status.FastStartFailover = "false" + } + + // manage manual switchover + if dataguardBroker.Spec.SetAsPrimaryDatabase != "" && dataguardBroker.Spec.SetAsPrimaryDatabase != dataguardBroker.Status.PrimaryDatabase { + if _, ok := dataguardBroker.Status.DatabasesInDataguardConfig[dataguardBroker.Spec.SetAsPrimaryDatabase]; !ok { + r.Recorder.Eventf(&dataguardBroker, corev1.EventTypeWarning, "Cannot Switchover", fmt.Sprintf("database with SID %v not found in dataguardbroker configuration", dataguardBroker.Spec.SetAsPrimaryDatabase)) + log.Info(fmt.Sprintf("cannot perform switchover, database with SID %v not found in dataguardbroker configuration", dataguardBroker.Spec.SetAsPrimaryDatabase)) + return ctrl.Result{Requeue: false}, nil + } + r.Recorder.Eventf(&dataguardBroker, corev1.EventTypeWarning, "Manual Switchover", fmt.Sprintf("Switching over to %s database", dataguardBroker.Status.DatabasesInDataguardConfig[dataguardBroker.Spec.SetAsPrimaryDatabase])) + log.Info(fmt.Sprintf("switching over to %s database", dataguardBroker.Status.DatabasesInDataguardConfig[dataguardBroker.Spec.SetAsPrimaryDatabase])) + result, err := r.manageManualSwitchOver(dataguardBroker.Spec.SetAsPrimaryDatabase, &dataguardBroker, ctx, req) + if err != nil { + return ctrl.Result{Requeue: false}, err + } + if result.Requeue { + return result, nil + } + } + + // Update Status for broker and sidb resources + if err := updateReconcileStatus(r, &dataguardBroker, ctx, req); err != nil { + return ctrl.Result{Requeue: false}, err + } + + dataguardBroker.Status.Status = dbcommons.StatusReady + log.Info("Reconcile Completed") + + if dataguardBroker.Spec.FastStartFailover { + return ctrl.Result{Requeue: true, RequeueAfter: 30 * time.Second}, nil + } else { + return ctrl.Result{Requeue: false}, nil + } +} + +// ############################################################################################################################# +// +// Manage deletion and clean up of the dataguardBroker resource +// +// ############################################################################################################################# +func (r *DataguardBrokerReconciler) manageDataguardBrokerDeletion(broker *dbapi.DataguardBroker, ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + + log := r.Log.WithValues("manageDataguardBrokerDeletion", req.NamespacedName) + + log.Info(fmt.Sprintf("Deleting dataguard broker %v", broker.Name)) + // Check if the DataguardBroker instance is marked to be deleted, which is + // indicated by the deletion timestamp being set. + if controllerutil.ContainsFinalizer(broker, dataguardBrokerFinalizer) { + // Run finalization logic for dataguardBrokerFinalizer. If the + // finalization logic fails, don't remove the finalizer so + // that we can retry during the next reconciliation. + if err := cleanupDataguardBroker(r, broker, req, ctx); err != nil { + // handle the errors + return ctrl.Result{Requeue: false}, err + } + + // Remove dataguardBrokerFinalizer. Once all finalizers have been + // removed, the object will be deleted. + controllerutil.RemoveFinalizer(broker, dataguardBrokerFinalizer) + if err := r.Update(ctx, broker); err != nil { + r.Recorder.Eventf(broker, corev1.EventTypeWarning, "Updating Resource", "Error while removing resource finalizers") + log.Info("Error while removing resource finalizers") + return ctrl.Result{Requeue: false}, err + } + } + return ctrl.Result{Requeue: false}, nil +} + +// ############################################################################################################################# +// +// Manage validation of singleinstancedatabases and creation of the dataguard configuration +// +// ############################################################################################################################# +func (r *DataguardBrokerReconciler) manageDataguardBrokerCreation(broker *dbapi.DataguardBroker, ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + + log := r.Log.WithValues("manageDataguardBrokerCreation", req.NamespacedName) + + // Add finalizer for this dataguardbroker resource + if !controllerutil.ContainsFinalizer(broker, dataguardBrokerFinalizer) { + r.Recorder.Eventf(broker, corev1.EventTypeNormal, "Updating Resource", "Adding finalizers") + log.Info("Adding finalizer") + controllerutil.AddFinalizer(broker, dataguardBrokerFinalizer) + if err := r.Update(ctx, broker); err != nil { + return ctrl.Result{Requeue: false}, err + } + } + + // Check if a service for the dataguardbroker resources exists + var service corev1.Service + if err := r.Get(context.TODO(), types.NamespacedName{Name: broker.Name, Namespace: broker.Namespace}, &service); err != nil { + // check if the required service is not found then create the service + if apierrors.IsNotFound(err) { + r.Recorder.Eventf(broker, corev1.EventTypeNormal, "Creating Service", "creating service for the resource") + log.Info("creating service for the dataguardbroker resource") + + // instantiate the service specification + svc := dbcommons.NewRealServiceBuilder(). + SetName(broker.Name). + SetNamespace(broker.Namespace). + SetLabels(map[string]string{ + "app": broker.Name, + }). + SetAnnotation(func() map[string]string { + annotations := make(map[string]string) + if len(broker.Spec.ServiceAnnotations) != 0 { + for key, value := range broker.Spec.ServiceAnnotations { + annotations[key] = value + } + } + return annotations + }()). + SetPorts([]corev1.ServicePort{ + { + Name: "listener", + Port: 1521, + Protocol: corev1.ProtocolTCP, + }, + { + Name: "xmldb", + Port: 5500, + Protocol: corev1.ProtocolTCP, + }, + }). + SetSelector(map[string]string{ + "app": broker.Name, + }). + SetType(func() corev1.ServiceType { + if broker.Spec.LoadBalancer { + return corev1.ServiceType("LoadBalancer") + } + return corev1.ServiceType("NodePort") + }()). + Build() + + // Set the ownership of the service object to the dataguard broker resource object + ctrl.SetControllerReference(broker, &svc, r.Scheme) + + // create the service for dataguardbroker resource + if err = r.Create(ctx, &svc); err != nil { + r.Recorder.Eventf(broker, corev1.EventTypeWarning, "Service Creation", "service creation failed") + log.Info("service creation failed") + return ctrl.Result{Requeue: false}, err + } else { + timeout := 30 + // Waiting for Service to get created as sometimes it takes some time to create a service . 30 seconds TImeout + err = dbcommons.WaitForStatusChange(r, svc.Name, broker.Namespace, ctx, req, time.Duration(timeout)*time.Second, "svc", "creation") + if err != nil { + log.Error(err, "Error in Waiting for svc status for Creation", "svc.Namespace", svc.Namespace, "SVC.Name", svc.Name) + return ctrl.Result{Requeue: false}, err + } + r.Recorder.Eventf(broker, corev1.EventTypeNormal, "Service Created", fmt.Sprintf("Succesfully Created New Service %v", svc.Name)) + log.Info("Succesfully Created New Service ", "Service.Name : ", svc.Name) + } + time.Sleep(10 * time.Second) + } else { + return ctrl.Result{Requeue: false}, err + } + } + + log.Info(" ", "Found Existing Service ", service.Name) + + // validate if all the databases have only one replicas + for _, databaseRef := range broker.GetDatabasesInDataGuardConfiguration() { + var singleinstancedatabase dbapi.SingleInstanceDatabase + if err := r.Get(ctx, types.NamespacedName{Name: databaseRef, Namespace: broker.Namespace}, &singleinstancedatabase); err != nil { + if apierrors.IsNotFound(err) { + broker.Status.Status = dbcommons.StatusError + r.Recorder.Eventf(broker, corev1.EventTypeWarning, "SingleInstanceDatabase Not Found", fmt.Sprintf("SingleInstanceDatabase %s not found", singleinstancedatabase.Name)) + log.Info(fmt.Sprintf("singleinstancedatabase %s not found", databaseRef)) + return ctrl.Result{Requeue: false}, nil + } + return ctrl.Result{Requeue: false}, err + } + if broker.Spec.FastStartFailover && singleinstancedatabase.Status.Replicas > 1 { + r.Recorder.Eventf(broker, corev1.EventTypeWarning, "SIDB Not supported", "dataguardbroker doesn't support multiple replicas sidb in FastStartFailover mode") + log.Info("dataguardbroker doesn't support multiple replicas sidb in FastStartFailover mode") + broker.Status.Status = dbcommons.StatusError + return ctrl.Result{Requeue: false}, nil + } + } + + // Get the current primary singleinstancedatabase resourcce + var sidb dbapi.SingleInstanceDatabase + namespacedName := types.NamespacedName{ + Namespace: broker.Namespace, + Name: broker.GetCurrentPrimaryDatabase(), + } + if err := r.Get(ctx, namespacedName, &sidb); err != nil { + if apierrors.IsNotFound(err) { + broker.Status.Status = dbcommons.StatusError + r.Recorder.Eventf(broker, corev1.EventTypeWarning, "SingleInstanceDatabase Not Found", fmt.Sprintf("SingleInstanceDatabase %s not found", sidb.Name)) + log.Info(fmt.Sprintf("singleinstancedatabase %s not found", namespacedName.Name)) + return ctrl.Result{Requeue: false}, nil + } + return ctrl.Result{Requeue: false}, err + } + if sidb.Status.Role != "PRIMARY" { + r.Recorder.Eventf(broker, corev1.EventTypeWarning, "Spec Validation", fmt.Sprintf("singleInstanceDatabase %v not in primary role", sidb.Name)) + log.Info(fmt.Sprintf("singleinstancedatabase %s expected to be in primary role", sidb.Name)) + log.Info("updating database status to check for possible FSFO") + if err := updateReconcileStatus(r, broker, ctx, req); err != nil { + return ctrl.Result{Requeue: false}, err + } + return ctrl.Result{Requeue: true, RequeueAfter: 60 * time.Second}, nil + } + + // validate current primary singleinstancedatabase readiness + log.Info(fmt.Sprintf("Validating readiness for singleinstancedatabase %v", sidb.Name)) + if err := validateSidbReadiness(r, broker, &sidb, ctx, req); err != nil { + if errors.Is(err, ErrCurrentPrimaryDatabaseNotReady) { + fastStartFailoverStatus, _ := strconv.ParseBool(broker.Status.FastStartFailover) + if broker.Status.Status != "" && fastStartFailoverStatus { + r.Recorder.Eventf(broker, corev1.EventTypeNormal, "Possible Failover", "Primary db not in ready state after setting up DG configuration") + } + if err := updateReconcileStatus(r, broker, ctx, req); err != nil { + log.Info("Error updating Dgbroker status") + } + r.Recorder.Eventf(broker, corev1.EventTypeWarning, "Waiting", err.Error()) + return ctrl.Result{Requeue: true, RequeueAfter: 60 * time.Second}, nil + } + return ctrl.Result{Requeue: false}, err + } + + // setup dataguard configuration + log.Info(fmt.Sprintf("setup Dataguard configuration for primary database %v", sidb.Name)) + if err := setupDataguardBrokerConfiguration(r, broker, &sidb, ctx, req); err != nil { + return ctrl.Result{Requeue: false}, err + } + + return ctrl.Result{Requeue: false}, nil +} + +// ############################################################################################################################# +// +// Manange manual switchover to the target database +// +// ############################################################################################################################# +func (r *DataguardBrokerReconciler) manageManualSwitchOver(targetSidbSid string, broker *dbapi.DataguardBroker, ctx context.Context, req ctrl.Request) (ctrl.Result, error) { + + log := r.Log.WithValues("SetAsPrimaryDatabase", req.NamespacedName) + + if _, ok := broker.Status.DatabasesInDataguardConfig[targetSidbSid]; !ok { + eventReason := "Cannot Switchover" + eventMsg := fmt.Sprintf("Database %s not a part of the dataguard configuration", targetSidbSid) + r.Recorder.Eventf(broker, corev1.EventTypeWarning, eventReason, eventMsg) + return ctrl.Result{Requeue: false}, nil + } + + // change broker status to updating to indicate manual switchover start + broker.Status.Status = dbcommons.StatusUpdating + r.Status().Update(ctx, broker) + + var sidb dbapi.SingleInstanceDatabase + if err := r.Get(context.TODO(), types.NamespacedName{Name: broker.GetCurrentPrimaryDatabase(), Namespace: broker.Namespace}, &sidb); err != nil { + return ctrl.Result{Requeue: false}, err + } + + // Fetch the primary database ready pod to create chk file + sidbReadyPod, _, _, _, err := dbcommons.FindPods(r, sidb.Spec.Image.Version, + sidb.Spec.Image.PullFrom, sidb.Name, sidb.Namespace, ctx, req) + if err != nil { + return ctrl.Result{Requeue: false}, err + } + + // Fetch the target database ready pod to create chk file + targetReadyPod, _, _, _, err := dbcommons.FindPods(r, "", "", broker.Status.DatabasesInDataguardConfig[targetSidbSid], req.Namespace, + ctx, ctrl.Request{NamespacedName: types.NamespacedName{Name: broker.Status.DatabasesInDataguardConfig[targetSidbSid], Namespace: req.Namespace}}) + if err != nil { + return ctrl.Result{Requeue: false}, err + } + + // Create a chk File so that no other pods take the lock during Switchover . + out, err := dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", dbcommons.CreateChkFileCMD) + if err != nil { + log.Error(err, err.Error()) + return ctrl.Result{Requeue: false}, err + } + log.Info("Successfully Created chk file " + out) + + out, err = dbcommons.ExecCommand(r, r.Config, targetReadyPod.Name, targetReadyPod.Namespace, "", ctx, req, false, "bash", "-c", dbcommons.CreateChkFileCMD) + if err != nil { + log.Error(err, err.Error()) + return ctrl.Result{Requeue: false}, err + } + log.Info("Successfully Created chk file " + out) + + eventReason := "Waiting" + eventMsg := "Switchover In Progress" + r.Recorder.Eventf(broker, corev1.EventTypeNormal, eventReason, eventMsg) + + // Get Admin password for current primary database + var adminPasswordSecret corev1.Secret + if err := r.Get(context.TODO(), types.NamespacedName{Name: sidb.Spec.AdminPassword.SecretName, Namespace: sidb.Namespace}, &adminPasswordSecret); err != nil { + return ctrl.Result{Requeue: false}, err + } + var adminPassword string = string(adminPasswordSecret.Data[sidb.Spec.AdminPassword.SecretKey]) + + // Connect to 'primarySid' db using dgmgrl and switchover to 'targetSidbSid' db to make 'targetSidbSid' db primary + _, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, true, "bash", "-c", + fmt.Sprintf(dbcommons.CreateAdminPasswordFile, adminPassword)) + if err != nil { + log.Error(err, err.Error()) + return ctrl.Result{Requeue: false}, err + } + log.Info("DB Admin pwd file created") + + out, err = dbcommons.ExecCommand(r, r.Config, sidbReadyPod.Name, sidbReadyPod.Namespace, "", ctx, req, false, "bash", "-c", + fmt.Sprintf("dgmgrl sys@%s \"SWITCHOVER TO %s\" < admin.pwd", broker.Status.PrimaryDatabase, targetSidbSid)) + if err != nil { + log.Error(err, err.Error()) + return ctrl.Result{Requeue: false}, err + } + log.Info("SWITCHOVER TO " + targetSidbSid + " Output") + log.Info(out) + + return ctrl.Result{Requeue: false}, nil +} + +// ############################################################################################################################# +// +// Setup the controller with the Manager +// +// ############################################################################################################################# +func (r *DataguardBrokerReconciler) SetupWithManager(mgr ctrl.Manager) error { + return ctrl.NewControllerManagedBy(mgr). + For(&dbapi.DataguardBroker{}). + Owns(&corev1.Pod{}). //Watch for deleted pods of DataguardBroker Owner + WithEventFilter(dbcommons.ResourceEventHandler()). + WithOptions(controller.Options{MaxConcurrentReconciles: 100}). //ReconcileHandler is never invoked concurrently with the same object. + Complete(r) +} diff --git a/controllers/observability/databaseobserver_controller.go b/controllers/observability/databaseobserver_controller.go index bd58e71e..e17ee0b3 100644 --- a/controllers/observability/databaseobserver_controller.go +++ b/controllers/observability/databaseobserver_controller.go @@ -44,6 +44,7 @@ import ( "github.com/go-logr/logr" appsv1 "k8s.io/api/apps/v1" corev1 "k8s.io/api/core/v1" + "k8s.io/apimachinery/pkg/api/equality" apiError "k8s.io/apimachinery/pkg/api/errors" "k8s.io/apimachinery/pkg/api/meta" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" @@ -55,7 +56,7 @@ import ( "sigs.k8s.io/controller-runtime/pkg/client" "time" - apiv1 "github.com/oracle/oracle-database-operator/apis/observability/v1alpha1" + api "github.com/oracle/oracle-database-operator/apis/observability/v4" constants "github.com/oracle/oracle-database-operator/commons/observability" ) @@ -90,8 +91,8 @@ func (r *DatabaseObserverReconciler) Reconcile(ctx context.Context, req ctrl.Req r.Log.WithName(constants.LogReconcile).Info(constants.LogCRStart, "NamespacedName", req.NamespacedName) // fetch databaseObserver - api := &apiv1.DatabaseObserver{} - if e := r.Get(context.TODO(), req.NamespacedName, api); e != nil { + a := &api.DatabaseObserver{} + if e := r.Get(context.TODO(), req.NamespacedName, a); e != nil { // if CR is not found or does not exist then // consider either CR has been deleted @@ -101,7 +102,7 @@ func (r *DatabaseObserverReconciler) Reconcile(ctx context.Context, req ctrl.Req } r.Log.WithName(constants.LogReconcile).Error(e, constants.ErrorCRRetrieve) - r.Recorder.Event(api, corev1.EventTypeWarning, constants.EventReasonFailedCRRetrieval, constants.EventMessageFailedCRRetrieval) + r.Recorder.Event(a, corev1.EventTypeWarning, constants.EventReasonFailedCRRetrieval, constants.EventMessageFailedCRRetrieval) return ctrl.Result{}, e } @@ -110,19 +111,19 @@ func (r *DatabaseObserverReconciler) Reconcile(ctx context.Context, req ctrl.Req defer r.validateCustomResourceReadiness(ctx, req) // initialize databaseObserver custom resource - if e := r.initialize(ctx, api, req); e != nil { + if e := r.initialize(ctx, a, req); e != nil { return ctrl.Result{}, e } // validate specs - if e := r.validateSpecs(api); e != nil { - meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + if e := r.validateSpecs(a); e != nil { + meta.SetStatusCondition(&a.Status.Conditions, metav1.Condition{ Type: constants.IsExporterDeploymentReady, Status: metav1.ConditionFalse, Reason: constants.ReasonDeploymentSpecValidationFailed, Message: constants.MessageExporterDeploymentSpecValidationFailed, }) - if e := r.Status().Update(ctx, api); e != nil { + if e := r.Status().Update(ctx, a); e != nil { r.Log.WithName(constants.LogReconcile).Error(e, constants.ErrorStatusUpdate) } r.Log.WithName(constants.LogExportersDeploy).Error(e, constants.ErrorSpecValidationFailedDueToAnError) @@ -131,50 +132,74 @@ func (r *DatabaseObserverReconciler) Reconcile(ctx context.Context, req ctrl.Req // create resource if they do not exist exporterDeployment := &ObservabilityDeploymentResource{} - if res, e := r.createResourceIfNotExists(exporterDeployment, api, ctx, req); e != nil { + if res, e := r.createResourceIfNotExists(exporterDeployment, a, ctx, req); e != nil { return res, e } - if res, e := r.checkDeploymentForUpdates(exporterDeployment, api, ctx, req); e != nil { - meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + // otherwise, check for updates on resource for any changes + if res, e := r.checkResourceForUpdates(exporterDeployment, a, ctx, req); e != nil { + meta.SetStatusCondition(&a.Status.Conditions, metav1.Condition{ Type: constants.IsExporterDeploymentReady, Status: metav1.ConditionFalse, - Reason: constants.ReasonDeploymentUpdateFailed, - Message: constants.MessageExporterDeploymentUpdateFailed, + Reason: constants.ReasonResourceUpdateFailed, + Message: constants.MessageExporterResourceUpdateFailed, }) return res, e } exporterService := &ObservabilityServiceResource{} - if res, e := r.createResourceIfNotExists(exporterService, api, ctx, req); e != nil { + if res, e := r.createResourceIfNotExists(exporterService, a, ctx, req); e != nil { + return res, e + } + + // otherwise, check for updates on resource for any changes + if res, e := r.checkResourceForUpdates(exporterService, a, ctx, req); e != nil { + meta.SetStatusCondition(&a.Status.Conditions, metav1.Condition{ + Type: constants.IsExporterServiceReady, + Status: metav1.ConditionFalse, + Reason: constants.ReasonResourceUpdateFailed, + Message: constants.MessageExporterResourceUpdateFailed, + }) return res, e } exporterServiceMonitor := &ObservabilityServiceMonitorResource{} - if res, e := r.createResourceIfNotExists(exporterServiceMonitor, api, ctx, req); e != nil { + if res, e := r.createResourceIfNotExists(exporterServiceMonitor, a, ctx, req); e != nil { + return res, e + } + + // otherwise, check for updates on resource for any changes + if res, e := r.checkResourceForUpdates(exporterServiceMonitor, a, ctx, req); e != nil { + meta.SetStatusCondition(&a.Status.Conditions, metav1.Condition{ + Type: constants.IsExporterServiceMonitorReady, + Status: metav1.ConditionFalse, + Reason: constants.ReasonResourceUpdateFailed, + Message: constants.MessageExporterResourceUpdateFailed, + }) return res, e } // check if deployment pods are ready - return r.validateDeploymentReadiness(api, ctx, req) + return r.validateDeploymentReadiness(a, ctx, req) } // initialize method sets the initial status to PENDING, exporterConfig and sets the base condition -func (r *DatabaseObserverReconciler) initialize(ctx context.Context, api *apiv1.DatabaseObserver, req ctrl.Request) error { +func (r *DatabaseObserverReconciler) initialize(ctx context.Context, a *api.DatabaseObserver, req ctrl.Request) error { - if api.Status.Conditions == nil || len(api.Status.Conditions) == 0 { + if a.Status.Conditions == nil || len(a.Status.Conditions) == 0 { // set condition - meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + meta.SetStatusCondition(&a.Status.Conditions, metav1.Condition{ Type: constants.IsCRAvailable, Status: metav1.ConditionFalse, Reason: constants.ReasonInitStart, Message: constants.MessageCRInitializationStarted, }) - api.Status.Status = string(constants.StatusObservabilityPending) - api.Status.ExporterConfig = constants.UnknownValue - if e := r.Status().Update(ctx, api); e != nil { + a.Status.Status = string(constants.StatusObservabilityPending) + a.Status.ExporterConfig = constants.UnknownValue + a.Status.Version = constants.UnknownValue + if e := r.Status().Update(ctx, a); e != nil { r.Log.WithName(constants.LogReconcile).Error(e, constants.ErrorStatusUpdate) return e } @@ -185,45 +210,45 @@ func (r *DatabaseObserverReconciler) initialize(ctx context.Context, api *apiv1. } // validateSpecs method checks the values and secrets passed in the spec -func (r *DatabaseObserverReconciler) validateSpecs(api *apiv1.DatabaseObserver) error { +func (r *DatabaseObserverReconciler) validateSpecs(a *api.DatabaseObserver) error { // If either Vault Fields are empty, then assume a DBPassword secret is supplied. If the DBPassword secret not found, then error out - if api.Spec.Database.DBPassword.VaultOCID == "" || api.Spec.Database.DBPassword.VaultSecretName == "" { + if a.Spec.Database.DBPassword.VaultOCID == "" || a.Spec.Database.DBPassword.VaultSecretName == "" { dbSecret := &corev1.Secret{} - if e := r.Get(context.TODO(), types.NamespacedName{Name: api.Spec.Database.DBPassword.SecretName, Namespace: api.Namespace}, dbSecret); e != nil { - r.Recorder.Event(api, corev1.EventTypeWarning, constants.EventReasonSpecError, constants.EventMessageSpecErrorDBPasswordSecretMissing) + if e := r.Get(context.TODO(), types.NamespacedName{Name: a.Spec.Database.DBPassword.SecretName, Namespace: a.Namespace}, dbSecret); e != nil { + r.Recorder.Event(a, corev1.EventTypeWarning, constants.EventReasonSpecError, constants.EventMessageSpecErrorDBPasswordSecretMissing) return e } } // Does DB Connection String Secret Name actually exist dbConnectSecret := &corev1.Secret{} - if e := r.Get(context.TODO(), types.NamespacedName{Name: api.Spec.Database.DBConnectionString.SecretName, Namespace: api.Namespace}, dbConnectSecret); e != nil { - r.Recorder.Event(api, corev1.EventTypeWarning, constants.EventReasonSpecError, constants.EventMessageSpecErrorDBConnectionStringSecretMissing) + if e := r.Get(context.TODO(), types.NamespacedName{Name: a.Spec.Database.DBConnectionString.SecretName, Namespace: a.Namespace}, dbConnectSecret); e != nil { + r.Recorder.Event(a, corev1.EventTypeWarning, constants.EventReasonSpecError, constants.EventMessageSpecErrorDBConnectionStringSecretMissing) return e } // Does DB User String Secret Name actually exist dbUserSecret := &corev1.Secret{} - if e := r.Get(context.TODO(), types.NamespacedName{Name: api.Spec.Database.DBUser.SecretName, Namespace: api.Namespace}, dbUserSecret); e != nil { - r.Recorder.Event(api, corev1.EventTypeWarning, constants.EventReasonSpecError, constants.EventMessageSpecErrorDBPUserSecretMissing) + if e := r.Get(context.TODO(), types.NamespacedName{Name: a.Spec.Database.DBUser.SecretName, Namespace: a.Namespace}, dbUserSecret); e != nil { + r.Recorder.Event(a, corev1.EventTypeWarning, constants.EventReasonSpecError, constants.EventMessageSpecErrorDBPUserSecretMissing) return e } // Does a custom configuration configmap actually exist, if provided - if configurationCMName := api.Spec.Exporter.ExporterConfig.Configmap.Name; configurationCMName != "" { + if configurationCMName := a.Spec.ExporterConfig.Configmap.Name; configurationCMName != "" { configurationCM := &corev1.ConfigMap{} - if e := r.Get(context.TODO(), types.NamespacedName{Name: configurationCMName, Namespace: api.Namespace}, configurationCM); e != nil { - r.Recorder.Event(api, corev1.EventTypeWarning, constants.EventReasonSpecError, constants.EventMessageSpecErrorConfigmapMissing) + if e := r.Get(context.TODO(), types.NamespacedName{Name: configurationCMName, Namespace: a.Namespace}, configurationCM); e != nil { + r.Recorder.Event(a, corev1.EventTypeWarning, constants.EventReasonSpecError, constants.EventMessageSpecErrorConfigmapMissing) return e } } // Does DBWallet actually exist, if provided - if dbWalletSecretName := api.Spec.Database.DBWallet.SecretName; dbWalletSecretName != "" { + if dbWalletSecretName := a.Spec.Database.DBWallet.SecretName; dbWalletSecretName != "" { dbWalletSecret := &corev1.Secret{} - if e := r.Get(context.TODO(), types.NamespacedName{Name: dbWalletSecretName, Namespace: api.Namespace}, dbWalletSecret); e != nil { - r.Recorder.Event(api, corev1.EventTypeWarning, constants.EventReasonSpecError, constants.EventMessageSpecErrorDBWalletSecretMissing) + if e := r.Get(context.TODO(), types.NamespacedName{Name: dbWalletSecretName, Namespace: a.Namespace}, dbWalletSecret); e != nil { + r.Recorder.Event(a, corev1.EventTypeWarning, constants.EventReasonSpecError, constants.EventMessageSpecErrorDBWalletSecretMissing) return e } } @@ -232,17 +257,17 @@ func (r *DatabaseObserverReconciler) validateSpecs(api *apiv1.DatabaseObserver) } // createResourceIfNotExists method creates an ObserverResource if they have not yet been created -func (r *DatabaseObserverReconciler) createResourceIfNotExists(or ObserverResource, api *apiv1.DatabaseObserver, ctx context.Context, req ctrl.Request) (ctrl.Result, error) { +func (r *DatabaseObserverReconciler) createResourceIfNotExists(or ObserverResource, a *api.DatabaseObserver, ctx context.Context, req ctrl.Request) (ctrl.Result, error) { conditionType, logger, groupVersionKind := or.identify() // update after - defer r.Status().Update(ctx, api) + defer r.Status().Update(ctx, a) - // generate desired object based on api.Spec - desiredObj, genErr := or.generate(api, r.Scheme) + // generate desired object based on a.Spec + desiredObj, genErr := or.generate(a, r.Scheme) if genErr != nil { - meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + meta.SetStatusCondition(&a.Status.Conditions, metav1.Condition{ Type: conditionType, Status: metav1.ConditionFalse, Reason: constants.ReasonGeneralResourceGenerationFailed, @@ -260,7 +285,7 @@ func (r *DatabaseObserverReconciler) createResourceIfNotExists(or ObserverResour if getErr != nil && apiError.IsNotFound(getErr) { if e := r.Create(context.TODO(), desiredObj); e != nil { // create - meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + meta.SetStatusCondition(&a.Status.Conditions, metav1.Condition{ Type: conditionType, Status: metav1.ConditionFalse, Reason: constants.ReasonGeneralResourceCreationFailed, @@ -271,7 +296,7 @@ func (r *DatabaseObserverReconciler) createResourceIfNotExists(or ObserverResour } // mark ready if created - meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + meta.SetStatusCondition(&a.Status.Conditions, metav1.Condition{ Type: conditionType, Status: metav1.ConditionTrue, Reason: constants.ReasonGeneralResourceCreated, @@ -280,7 +305,7 @@ func (r *DatabaseObserverReconciler) createResourceIfNotExists(or ObserverResour r.Log.WithName(logger).Info(constants.LogResourceCreated, "ResourceName", desiredObj.GetName(), "Kind", groupVersionKind, "Namespace", req.Namespace) } else if getErr != nil { // if an error occurred - meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + meta.SetStatusCondition(&a.Status.Conditions, metav1.Condition{ Type: conditionType, Status: metav1.ConditionFalse, Reason: constants.ReasonGeneralResourceValidationFailureDueToError, @@ -290,7 +315,7 @@ func (r *DatabaseObserverReconciler) createResourceIfNotExists(or ObserverResour return ctrl.Result{}, getErr } else if getErr == nil && conditionType != constants.IsExporterDeploymentReady { // exclude deployment - meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + meta.SetStatusCondition(&a.Status.Conditions, metav1.Condition{ Type: conditionType, Status: metav1.ConditionTrue, Reason: constants.ReasonGeneralResourceValidationCompleted, @@ -304,138 +329,68 @@ func (r *DatabaseObserverReconciler) createResourceIfNotExists(or ObserverResour return ctrl.Result{}, nil } -// checkDeploymentForUpdates method checks the deployment if it needs to be updated -func (r *DatabaseObserverReconciler) checkDeploymentForUpdates(or ObserverResource, api *apiv1.DatabaseObserver, ctx context.Context, req ctrl.Request) (ctrl.Result, error) { +// checkResourceForUpdates method checks the resource if it needs to be updated, updates if changes are found +func (r *DatabaseObserverReconciler) checkResourceForUpdates(or ObserverResource, a *api.DatabaseObserver, ctx context.Context, req ctrl.Request) (ctrl.Result, error) { - // declare - foundDeployment := &appsv1.Deployment{} + conditionType, logName, groupVersionKind := or.identify() - // generate object - desiredObj, genErr := or.generate(api, r.Scheme) + // generate desired object + dO, genErr := or.generate(a, r.Scheme) if genErr != nil { return ctrl.Result{}, genErr } - // convert - desiredDeployment := &appsv1.Deployment{} - if e := r.Scheme.Convert(desiredObj, desiredDeployment, nil); e != nil { - return ctrl.Result{}, e - } - - // retrieve latest deployment - if e := r.Get(context.TODO(), types.NamespacedName{Name: desiredObj.GetName(), Namespace: req.Namespace}, foundDeployment); e != nil { - return ctrl.Result{}, e - } - // check for containerImage - if constants.IsUpdateRequiredForContainerImage(desiredDeployment, foundDeployment) { - foundDeployment.Spec.Template.Spec.Containers[0].Image = constants.GetExporterImage(api) - - if e := r.updateDeployment(api, ctx, req, foundDeployment, constants.MessageExporterDeploymentImageUpdated, constants.EventMessageUpdatedImageSucceeded); e != nil { - return ctrl.Result{}, e - } - } - - // retrieve latest deployment - if e := r.Get(context.TODO(), types.NamespacedName{Name: desiredObj.GetName(), Namespace: req.Namespace}, foundDeployment); e != nil { + // convert dO -> d + d := &unstructured.Unstructured{} + d.SetGroupVersionKind(groupVersionKind) + if e := r.Scheme.Convert(dO, d, nil); e != nil { return ctrl.Result{}, e } - // check environment variables - if constants.IsUpdateRequiredForEnvironmentVars(desiredDeployment, foundDeployment) { - foundDeployment.Spec.Template.Spec.Containers[0].Env = constants.GetExporterEnvs(api) - - if e := r.updateDeployment(api, ctx, req, foundDeployment, constants.MessageExporterDeploymentEnvironmentUpdated, constants.EventMessageUpdatedEnvironmentSucceeded); e != nil { - return ctrl.Result{}, e - } - } - // retrieve latest deployment - foundDeployment = &appsv1.Deployment{} - if e := r.Get(context.TODO(), types.NamespacedName{Name: desiredObj.GetName(), Namespace: req.Namespace}, foundDeployment); e != nil { + // declare found + // retrieve latest into f + f := &unstructured.Unstructured{} + f.SetGroupVersionKind(groupVersionKind) + if e := r.Get(context.TODO(), types.NamespacedName{Name: dO.GetName(), Namespace: req.Namespace}, f); e != nil { return ctrl.Result{}, e } - // check config-volume, creds and ocikey - if constants.IsUpdateRequiredForVolumes(desiredDeployment, foundDeployment) { - foundDeployment.Spec.Template.Spec.Volumes = constants.GetExporterDeploymentVolumes(api) - foundDeployment.Spec.Template.Spec.Containers[0].VolumeMounts = constants.GetExporterDeploymentVolumeMounts(api) - if e := r.updateDeployment(api, ctx, req, foundDeployment, constants.MessageExporterDeploymentVolumesUpdated, constants.EventMessageUpdatedVolumesSucceeded); e != nil { - return ctrl.Result{}, e - } - } - - // update status for exporter config - var setConfigmapNameStatus string - for _, v := range desiredDeployment.Spec.Template.Spec.Volumes { - if v.Name == constants.DefaultConfigVolumeString { - setConfigmapNameStatus = v.ConfigMap.Name - api.Status.ExporterConfig = setConfigmapNameStatus - } - } - if api.Status.ExporterConfig != setConfigmapNameStatus { - api.Status.ExporterConfig = constants.DefaultValue - } - r.Status().Update(ctx, api) - - // retrieve latest deployment - foundDeployment = &appsv1.Deployment{} - if e := r.Get(context.TODO(), types.NamespacedName{Name: desiredObj.GetName(), Namespace: req.Namespace}, foundDeployment); e != nil { - return ctrl.Result{}, e - } - // check replicateCount - if constants.IsUpdateRequiredForReplicas(desiredDeployment, foundDeployment) { - desiredReplicaCount := constants.GetExporterReplicas(api) - foundDeployment.Spec.Replicas = &desiredReplicaCount + // check if something changed + if !equality.Semantic.DeepDerivative(d.Object, f.Object) { - if e := r.updateDeployment(api, ctx, req, foundDeployment, constants.MessageExporterDeploymentReplicaUpdated, constants.EventMessageUpdatedReplicaSucceeded); e != nil { + if e := r.Update(context.TODO(), d); e != nil { + r.Log.WithName(logName).Error(e, constants.LogErrorWithResourceUpdate, "ResourceName", f.GetName(), "Kind", groupVersionKind.Kind, "Namespace", req.Namespace) return ctrl.Result{}, e } - } - - return ctrl.Result{}, nil -} - -// updateDeployment method updates the deployment and sets the condition -func (r *DatabaseObserverReconciler) updateDeployment(api *apiv1.DatabaseObserver, ctx context.Context, req ctrl.Request, d *appsv1.Deployment, updateMessage string, recorderMessage string) error { - // make update - defer r.Status().Update(ctx, api) - - if e := r.Update(context.TODO(), d); e != nil { - meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ - Type: constants.IsExporterDeploymentReady, + // update completed, however the pods needs to be validated for readiness + meta.SetStatusCondition(&a.Status.Conditions, metav1.Condition{ + Type: conditionType, Status: metav1.ConditionFalse, - Reason: constants.ReasonDeploymentUpdateFailed, - Message: constants.MessageExporterDeploymentUpdateFailed, + Reason: constants.ReasonResourceUpdated, + Message: constants.MessageExporterResourceUpdated, }) - r.Log.WithName(constants.LogExportersDeploy).Error(e, constants.ErrorDeploymentUpdate, "ResourceName", d.GetName(), "Kind", "Deployment", "Namespace", req.Namespace) - return e + r.Log.WithName(logName).Info(constants.LogSuccessWithResourceUpdate, "ResourceName", f.GetName(), "Kind", groupVersionKind.Kind, "Namespace", req.Namespace) + r.Recorder.Event(a, corev1.EventTypeNormal, constants.EventReasonUpdateSucceeded, groupVersionKind.Kind+" is updated.") + r.Status().Update(ctx, a) } - // update completed, however the pods needs to be validated for readiness - meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ - Type: constants.IsExporterDeploymentReady, - Status: metav1.ConditionFalse, - Reason: constants.ReasonDeploymentUpdated, - Message: updateMessage, - }) - r.Log.WithName(constants.LogExportersDeploy).Info(constants.LogResourceUpdated, "ResourceName", d.GetName(), "Kind", "Deployment", "Namespace", req.Namespace) - r.Recorder.Event(api, corev1.EventTypeNormal, constants.EventReasonUpdateSucceeded, recorderMessage) + return ctrl.Result{}, nil - return nil } // validateDeploymentReadiness method evaluates deployment readiness by checking the status of all deployment pods -func (r *DatabaseObserverReconciler) validateDeploymentReadiness(api *apiv1.DatabaseObserver, ctx context.Context, req ctrl.Request) (ctrl.Result, error) { +func (r *DatabaseObserverReconciler) validateDeploymentReadiness(a *api.DatabaseObserver, ctx context.Context, req ctrl.Request) (ctrl.Result, error) { d := &appsv1.Deployment{} - rName := constants.DefaultExporterDeploymentPrefix + api.Name + rName := a.Name // update after - defer r.Status().Update(ctx, api) + defer r.Status().Update(ctx, a) // get latest deployment - if e := r.Get(context.TODO(), types.NamespacedName{Name: rName, Namespace: api.Namespace}, d); e != nil { - meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + if e := r.Get(context.TODO(), types.NamespacedName{Name: rName, Namespace: a.Namespace}, d); e != nil { + meta.SetStatusCondition(&a.Status.Conditions, metav1.Condition{ Type: constants.IsExporterDeploymentReady, Status: metav1.ConditionFalse, Reason: constants.ReasonGeneralResourceValidationFailureDueToError, @@ -445,16 +400,14 @@ func (r *DatabaseObserverReconciler) validateDeploymentReadiness(api *apiv1.Data } // get deployment labels - labels := d.Spec.Template.Labels - cLabels := client.MatchingLabels{} - for k, v := range labels { - cLabels[k] = v + cLabels := client.MatchingLabels{ + "app": a.Name, } // list pods pods := &corev1.PodList{} if e := r.List(context.TODO(), pods, []client.ListOption{cLabels}...); e != nil { - meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + meta.SetStatusCondition(&a.Status.Conditions, metav1.Condition{ Type: constants.IsExporterDeploymentReady, Status: metav1.ConditionFalse, Reason: constants.ReasonDeploymentFailed, @@ -466,7 +419,7 @@ func (r *DatabaseObserverReconciler) validateDeploymentReadiness(api *apiv1.Data // check each pod phase for _, pod := range pods.Items { if pod.Status.Phase == corev1.PodFailed { - meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + meta.SetStatusCondition(&a.Status.Conditions, metav1.Condition{ Type: constants.IsExporterDeploymentReady, Status: metav1.ConditionFalse, Reason: constants.ReasonDeploymentFailed, @@ -475,7 +428,7 @@ func (r *DatabaseObserverReconciler) validateDeploymentReadiness(api *apiv1.Data return ctrl.Result{}, errors.New(constants.ErrorDeploymentPodsFailure) } else if pod.Status.Phase != corev1.PodRunning { // pod could be creating, - meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + meta.SetStatusCondition(&a.Status.Conditions, metav1.Condition{ Type: constants.IsExporterDeploymentReady, Status: metav1.ConditionUnknown, Reason: constants.ReasonDeploymentPending, @@ -486,12 +439,14 @@ func (r *DatabaseObserverReconciler) validateDeploymentReadiness(api *apiv1.Data } // once all pods are found to be running, mark deployment as ready and the exporter as ready - meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + meta.SetStatusCondition(&a.Status.Conditions, metav1.Condition{ Type: constants.IsExporterDeploymentReady, Status: metav1.ConditionTrue, Reason: constants.ReasonDeploymentSuccessful, Message: constants.MessageExporterDeploymentSuccessful, }) + a.Status.Version = constants.GetExporterVersion(a) + a.Status.ExporterConfig = constants.GetExporterConfig(a) return ctrl.Result{}, nil } @@ -499,48 +454,48 @@ func (r *DatabaseObserverReconciler) validateDeploymentReadiness(api *apiv1.Data func (r *DatabaseObserverReconciler) validateCustomResourceReadiness(ctx context.Context, req ctrl.Request) { // get latest object - api := &apiv1.DatabaseObserver{} - if e := r.Get(context.TODO(), req.NamespacedName, api); e != nil { + a := &api.DatabaseObserver{} + if e := r.Get(context.TODO(), req.NamespacedName, a); e != nil { r.Log.WithName(constants.LogReconcile).Error(e, constants.ErrorCRRetrieve) return } // make update - defer r.Status().Update(ctx, api) + defer r.Status().Update(ctx, a) - if meta.IsStatusConditionPresentAndEqual(api.Status.Conditions, constants.IsExporterDeploymentReady, metav1.ConditionUnknown) { - meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + if meta.IsStatusConditionPresentAndEqual(a.Status.Conditions, constants.IsExporterDeploymentReady, metav1.ConditionUnknown) { + meta.SetStatusCondition(&a.Status.Conditions, metav1.Condition{ Type: constants.IsCRAvailable, Status: metav1.ConditionFalse, Reason: constants.ReasonValidationInProgress, Message: constants.MessageCRValidationWaiting, }) - api.Status.Status = string(constants.StatusObservabilityPending) - } else if meta.IsStatusConditionFalse(api.Status.Conditions, constants.IsExporterDeploymentReady) || - meta.IsStatusConditionFalse(api.Status.Conditions, constants.IsExporterServiceReady) || - meta.IsStatusConditionFalse(api.Status.Conditions, constants.IsExporterServiceMonitorReady) { - meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + a.Status.Status = string(constants.StatusObservabilityPending) + } else if meta.IsStatusConditionFalse(a.Status.Conditions, constants.IsExporterDeploymentReady) || + meta.IsStatusConditionFalse(a.Status.Conditions, constants.IsExporterServiceReady) || + meta.IsStatusConditionFalse(a.Status.Conditions, constants.IsExporterServiceMonitorReady) { + meta.SetStatusCondition(&a.Status.Conditions, metav1.Condition{ Type: constants.IsCRAvailable, Status: metav1.ConditionFalse, Reason: constants.ReasonReadyFailed, Message: constants.MessageCRValidationFailed, }) - api.Status.Status = string(constants.StatusObservabilityError) + a.Status.Status = string(constants.StatusObservabilityError) } else { - meta.SetStatusCondition(&api.Status.Conditions, metav1.Condition{ + meta.SetStatusCondition(&a.Status.Conditions, metav1.Condition{ Type: constants.IsCRAvailable, Status: metav1.ConditionTrue, Reason: constants.ReasonReadyValidated, Message: constants.MessageCRValidated, }) - api.Status.Status = string(constants.StatusObservabilityReady) + a.Status.Status = string(constants.StatusObservabilityReady) } } // SetupWithManager sets up the controller with the Manager. func (r *DatabaseObserverReconciler) SetupWithManager(mgr ctrl.Manager) error { return ctrl.NewControllerManagedBy(mgr). - For(&apiv1.DatabaseObserver{}). + For(&api.DatabaseObserver{}). Owns(&appsv1.Deployment{}). Owns(&corev1.Service{}). Complete(r) diff --git a/controllers/observability/databaseobserver_resource.go b/controllers/observability/databaseobserver_resource.go index 8c20ebe5..6be6f693 100644 --- a/controllers/observability/databaseobserver_resource.go +++ b/controllers/observability/databaseobserver_resource.go @@ -1,7 +1,7 @@ package controllers import ( - apiv1 "github.com/oracle/oracle-database-operator/apis/observability/v1alpha1" + api "github.com/oracle/oracle-database-operator/apis/observability/v4" constants "github.com/oracle/oracle-database-operator/commons/observability" monitorv1 "github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring/v1" appsv1 "k8s.io/api/apps/v1" @@ -10,7 +10,6 @@ import ( "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" "k8s.io/apimachinery/pkg/runtime" "k8s.io/apimachinery/pkg/runtime/schema" - "k8s.io/apimachinery/pkg/util/intstr" "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" ) @@ -25,55 +24,78 @@ type ObservabilityServiceResource struct{} type ObservabilityServiceMonitorResource struct{} type ObserverResource interface { - generate(*apiv1.DatabaseObserver, *runtime.Scheme) (*unstructured.Unstructured, error) + generate(*api.DatabaseObserver, *runtime.Scheme) (*unstructured.Unstructured, error) identify() (string, string, schema.GroupVersionKind) } -func (resource *ObservabilityDeploymentResource) generate(api *apiv1.DatabaseObserver, scheme *runtime.Scheme) (*unstructured.Unstructured, error) { - rName := constants.DefaultExporterDeploymentPrefix + api.Name +func (resource *ObservabilityDeploymentResource) generate(a *api.DatabaseObserver, scheme *runtime.Scheme) (*unstructured.Unstructured, error) { + rName := a.Name rContainerName := constants.DefaultExporterContainerName - rContainerImage := constants.GetExporterImage(api) - rVolumes := constants.GetExporterDeploymentVolumes(api) - rVolumeMounts := constants.GetExporterDeploymentVolumeMounts(api) - rSelectors := constants.GetExporterSelector(api) - rReplicas := constants.GetExporterReplicas(api) - rEnvs := constants.GetExporterEnvs(api) + rContainerImage := constants.GetExporterImage(a) + rArgs := constants.GetExporterArgs(a) + rCommands := constants.GetExporterCommands(a) + rVolumes := constants.GetExporterDeploymentVolumes(a) + rVolumeMounts := constants.GetExporterDeploymentVolumeMounts(a) + + rReplicas := constants.GetExporterReplicas(a) + rEnvs := constants.GetExporterEnvs(a) + + rLabels := constants.GetLabels(a, a.Spec.Exporter.Deployment.Labels) + rPodLabels := constants.GetLabels(a, a.Spec.Exporter.Deployment.DeploymentPodTemplate.Labels) + rSelector := constants.GetSelectorLabel(a) + + rDeploymentSecurityContext := constants.GetExporterDeploymentSecurityContext(a) + rPodSecurityContext := constants.GetExporterPodSecurityContext(a) rPort := []corev1.ContainerPort{ - {ContainerPort: 8080}, + {ContainerPort: constants.DefaultAppPort}, + } + + // exporterContainer + rContainers := make([]corev1.Container, 1) + rContainers[0] = corev1.Container{ + Image: rContainerImage, + ImagePullPolicy: corev1.PullAlways, + Name: rContainerName, + Env: rEnvs, + VolumeMounts: rVolumeMounts, + Ports: rPort, + Args: rArgs, + Command: rCommands, + SecurityContext: rDeploymentSecurityContext, } + constants.AddSidecarContainers(a, &rContainers) + constants.AddSidecarVolumes(a, &rVolumes) + + // additionalContainers + obj := &appsv1.Deployment{ ObjectMeta: metav1.ObjectMeta{ Name: rName, - Namespace: api.Namespace, + Namespace: a.Namespace, + Labels: rLabels, }, Spec: appsv1.DeploymentSpec{ Replicas: &rReplicas, Selector: &metav1.LabelSelector{ - MatchLabels: rSelectors, + MatchLabels: rSelector, }, Template: corev1.PodTemplateSpec{ ObjectMeta: metav1.ObjectMeta{ - Labels: rSelectors, + Labels: rPodLabels, }, Spec: corev1.PodSpec{ - Containers: []corev1.Container{{ - Image: rContainerImage, - ImagePullPolicy: corev1.PullAlways, - Name: rContainerName, - Env: rEnvs, - VolumeMounts: rVolumeMounts, - Ports: rPort, - }}, - RestartPolicy: corev1.RestartPolicyAlways, - Volumes: rVolumes, + Containers: rContainers, + RestartPolicy: corev1.RestartPolicyAlways, + Volumes: rVolumes, + SecurityContext: rPodSecurityContext, }, }, }, } - if err := controllerutil.SetControllerReference(api, obj, scheme); err != nil { + if err := controllerutil.SetControllerReference(a, obj, scheme); err != nil { return nil, err } @@ -84,32 +106,26 @@ func (resource *ObservabilityDeploymentResource) generate(api *apiv1.DatabaseObs return u, nil } -func (resource *ObservabilityServiceResource) generate(api *apiv1.DatabaseObserver, scheme *runtime.Scheme) (*unstructured.Unstructured, error) { - rServiceName := "obs-svc-" + api.Name - rLabels := constants.GetExporterLabels(api) - rPort := constants.GetExporterServicePort(api) - rSelector := constants.GetExporterSelector(api) +func (resource *ObservabilityServiceResource) generate(a *api.DatabaseObserver, scheme *runtime.Scheme) (*unstructured.Unstructured, error) { + rServiceName := a.Name + rLabels := constants.GetLabels(a, a.Spec.Exporter.Service.Labels) + rSelector := constants.GetSelectorLabel(a) + rPorts := constants.GetExporterServicePort(a) obj := &corev1.Service{ ObjectMeta: metav1.ObjectMeta{ Name: rServiceName, Labels: rLabels, - Namespace: api.Namespace, + Namespace: a.Namespace, }, Spec: corev1.ServiceSpec{ - Type: "ClusterIP", + Type: constants.DefaultServiceType, Selector: rSelector, - Ports: []corev1.ServicePort{ - { - Name: "metrics", - Port: rPort, - TargetPort: intstr.FromInt32(constants.DefaultServiceTargetPort), - }, - }, + Ports: rPorts, }, } - if err := controllerutil.SetControllerReference(api, obj, scheme); err != nil { + if err := controllerutil.SetControllerReference(a, obj, scheme); err != nil { return nil, err } @@ -120,32 +136,32 @@ func (resource *ObservabilityServiceResource) generate(api *apiv1.DatabaseObserv return u, nil } -func (resource *ObservabilityServiceMonitorResource) generate(api *apiv1.DatabaseObserver, scheme *runtime.Scheme) (*unstructured.Unstructured, error) { - rName := constants.DefaultServiceMonitorPrefix + api.Name - rLabels := constants.GetExporterLabels(api) - rSelector := constants.GetExporterSelector(api) - rPort := constants.GetExporterServiceMonitorPort(api) - rInterval := "20s" +func (resource *ObservabilityServiceMonitorResource) generate(a *api.DatabaseObserver, scheme *runtime.Scheme) (*unstructured.Unstructured, error) { + rName := a.Name + rEndpoints := constants.GetEndpoints(a) + + rSelector := constants.GetSelectorLabel(a) + rLabels := constants.GetLabels(a, a.Spec.Prometheus.ServiceMonitor.Labels) + + smSpec := monitorv1.ServiceMonitorSpec{ + Endpoints: rEndpoints, + Selector: metav1.LabelSelector{ + MatchLabels: rSelector, + }, + } + constants.AddNamespaceSelector(a, &smSpec) obj := &monitorv1.ServiceMonitor{ ObjectMeta: metav1.ObjectMeta{ Name: rName, Labels: rLabels, - Namespace: api.Namespace, - }, - Spec: monitorv1.ServiceMonitorSpec{ - Endpoints: []monitorv1.Endpoint{{ - Interval: monitorv1.Duration(rInterval), - Port: rPort, - }}, - Selector: metav1.LabelSelector{ - MatchLabels: rSelector, - }, + Namespace: a.Namespace, }, + Spec: smSpec, } // set reference - if e := controllerutil.SetControllerReference(api, obj, scheme); e != nil { + if e := controllerutil.SetControllerReference(a, obj, scheme); e != nil { return nil, e } diff --git a/docs/adb/ADB_LONG_TERM_BACKUP.md b/docs/adb/ADB_LONG_TERM_BACKUP.md index 4720697d..312dac0d 100644 --- a/docs/adb/ADB_LONG_TERM_BACKUP.md +++ b/docs/adb/ADB_LONG_TERM_BACKUP.md @@ -2,13 +2,13 @@ To create long-term backups of Autonomous Databases, use this procedure. -Oracle Cloud Infrastructure (OCI) automatically backs up your Autonomous Databases, and retains these backups for 60 days. You can restore and recover your database to any point-in-time in this retention period. Automatic backups are full backups taken every 60 days, with daily incremental backups. You can also create long-term backups for your database with a retention period between 3 months and up to 10 years. For more information, please visit [Create Long-Term Backups on Autonomous Database](https://docs.oracle.com/en/cloud/paas/autonomous-database/serverless/adbsb/backup-long-term.html) and [Backup and Restore Notes](https://docs.oracle.com/en-us/iaas/autonomous-database-serverless/doc/backup-restore-notes.html). +Oracle Cloud Infrastructure (OCI) automatically backs up your Autonomous Databases, and retains these backups for 60 days. You can restore and recover your database to any point-in-time in this retention period. Automatic backups are full backups taken every 60 days, with daily incremental backups. You can also create long-term backups for your database with a retention period ranging from 3 months to 10 years. For more information, see: [Create Long-Term Backups on Autonomous Database](https://docs.oracle.com/en/cloud/paas/autonomous-database/serverless/adbsb/backup-long-term.html) and [Backup and Restore Notes](https://docs.oracle.com/en-us/iaas/autonomous-database-serverless/doc/backup-restore-notes.html). ## Create Long-Term Backup To back up an Autonomous Database, complete this procedure. -1. Add the following fields to the AutonomousDatabaseBackup resource definition. An example `.yaml` file is available here: [`config/samples/adb/autonomousdatabase_backup.yaml`](./../../config/samples/adb/autonomousdatabase_backup.yaml) +1. Add the following fields to the `AutonomousDatabaseBackup` resource definition. An example `.yaml` file is available here: [`config/samples/adb/autonomousdatabase_backup.yaml`](./../../config/samples/adb/autonomousdatabase_backup.yaml) | Attribute | Type | Description | Required? | |----|----|----|----| | `spec.displayName` | string | The user-friendly name for the backup. This name does not have to be unique. | Yes | diff --git a/docs/adb/ADB_PREREQUISITES.md b/docs/adb/ADB_PREREQUISITES.md index f8c04c4b..a730f4fe 100644 --- a/docs/adb/ADB_PREREQUISITES.md +++ b/docs/adb/ADB_PREREQUISITES.md @@ -48,7 +48,7 @@ After creating the ConfigMap and the Secret, use their names as the values of `o Instance principal authorization enables the operator to make API calls from an instance (that is, a node) without requiring the `ociConfigMap`, and `ociSecret` attributes in the `.yaml` file. This approach applies only to instances that are running in the Oracle Cloud Infrastructure (OCI). In addition, this approach grants permissions to the nodes that match the rules, which means that all the pods in the nodes can make the service calls. -To set up the instance principals, you will have to: +To set up the instance principals, complete the following tasks: * [Define dynamic group that includes the nodes in which the operator runs](#define-dynamic-group) * [Define policies that grant to the dynamic group the required permissions for the operator to its OCI interactions](#define-policies) @@ -147,4 +147,4 @@ kubectl create configmap oci-cred \ Allow any-user to manage all-resources in compartment where all {request.principal.namespace='oracle-database-operator-system',request.principal.type='workload',request.principal.cluster_id='',request.principal.service_account='default'} ``` -After creating the policy, operator pods will be granted sufficient permissions to call OCI services. You can now proceed to the installation. \ No newline at end of file +After creating the policy, operator pods will be granted sufficient permissions to call OCI services. You can now proceed to the installation. diff --git a/docs/adb/NETWORK_ACCESS_OPTIONS.md b/docs/adb/NETWORK_ACCESS_OPTIONS.md index e029b52d..e7eb0a56 100644 --- a/docs/adb/NETWORK_ACCESS_OPTIONS.md +++ b/docs/adb/NETWORK_ACCESS_OPTIONS.md @@ -7,77 +7,58 @@ Network access for Autonomous Database includes public access, and configuring s For more information about these options, see: [Configuring Network Access with Access Control Rules (ACLs) and Private Endpoints ](https://docs.oracle.com/en/cloud/paas/autonomous-database/adbsa/autonomous-network-access.html#GUID-D2D468C3-CA2D-411E-92BC-E122F795A413). ## Supported Features -Review the network access configuration options available to you with Autonomous Database. +Review the following options available to you with Autonomous Database. -### Types of Network Access +* [Configuring Network Access with Allowing Secure Access from Anywhere](#configuring-network-access-with-allowing-secure-access-from-anywhere) on shared Exadata infrastructure +* [Configuring Network Access with Access Control Rules (ACLs)](#configuring-network-access-with-access-control-rules-acls) on shared Exadata infrastructure +* [Configure Network Access with Private Endpoint Access Only](#configure-network-access-with-private-endpoint-access-only) on shared Exadata infrastructure +* [Allowing TLS or Require Only Mutual TLS (mTLS) Authentication](#allowing-tls-or-require-only-mutual-tls-mtls-authentication) on shared Exadata infrastructure +* [Autonomous Database with access control list enabled](#autonomous-database-with-access-control-list-enabled-on-dedicated-exadata-infrastructure) on dedicated Exadata infrastructure -There are three types of network access supported by Autonomous Database: +## Configuring Network Access with Allowing Secure Access from Anywhere -* **PUBLIC** +Before changing the Network Access to Allowing Secure Access from Anywhere, ensure that your network security protocol requries only mTLS (Mutual TLS) authentication. For more details, see: [Allow both TLS and mutual TLS (mTLS) authentication](#allow-both-tls-and-mutual-tls-mtls-authentication). If mTLS enforcement is already enabled on your Autonomous Database, then you can skip this step. - The Public option permits secure access from anywhere. The network access type is PUBLIC if no option is specified in the specification. With this option, mutual TLS (mTLS) authentication is always required to connect to the database. This option is available only for databases on shared Exadata infrastructure. +To specify that Autonomous Database can be connected from any location with a valid credential, complete one of the following procedures, based on your network access configuration. -* **RESTRICTED** - - The Restricted option permits connections to the database only as specified by the access control lists (ACLs) that you create. This option is available only for databases on shared Exadata infrastructure. - - You can add the following to your ACL: - * **IP Address**: Specify one or more individual public IP addresses. Use commas to delimit your addresses in the input field. - * **CIDR Block**: Specify one or more ranges of public IP addresses using CIDR notation. Use commas to separate your CIDR block entries in the input field. - * **Virtual Cloud Network (OCID)** (applies to Autonomous Databases on shared Exadata infrastructure): Specify the Oracle Cloud Identifier (OCID) of a virtual cloud network (VCN). If you want to specify multiple IP addresses or CIDR ranges within the same VCN, then do not create multiple access control list entries. Instead, use one access control list entry with the values for the multiple IP addresses or CIDR ranges, separated by commas. - -* **PRIVATE** - - The Private option creates a private endpoint for your database within a specified VCN. This option is available for databases on shared Exadata infrastructure, and is the only available option for databases on dedicated Exadata infrastructure. Review the private options for your configuration: - - * **Autonomous Databases on shared Exadata infrastructure**: - - This option permits access through private enpoints by specifying the OCIDs of a subnet and the network security groups (NSGs) under the same VCN in the specification. - - * **Autonomous Databases on dedicated Exadata infrastructure**: - - The network path to a dedicated Autonomous Database is through a VCN and subnet defined by the dedicated infrastucture hosting the database. Usually, the subnet is defined as private, which means that there is no public Internet access to the databases. - - Autonomous Database supports restricted access using an ACL. You have the option to enable an ACL by setting the `isAccessControlEnabled` parameter. If access is disabled, then database access is defined by the network security rules. If enabled, then database access is restricted to the IP addresses and CIDR blocks defined in the ACL. Note that enabling an ACL with an empty list of IP addresses makes the database inaccessible. See [Autonomous Database with Private Endpoint](https://docs.oracle.com/en-us/iaas/Content/Database/Concepts/adbsprivateaccess.htm) for overview and examples for private endpoint. - -### Allowing TLS or Require Only Mutual TLS (mTLS) Authentication - -If your Autonomous Database instance is configured to allow only mTLS connections, then you can reconfigure the instance to permit both mTLS and TLS connections. When you reconfigure the instance to permit both mTLS and TLS, you can use both authentication types at the same time, so that connections are no longer restricted to require mTLS authentication. - -This option only applies to Autonomous Databases on shared Exadata infrastructure. You can permit TLS connections when network access type is configured by using one of the following options: - -* **RESTRICTED**: with ACLs defined. -* **PRIVATE**: with a private endpoint defined. - -## Example YAML - -You can always configure the network access options when you create an Autonomous Database, or update the settings after you create the database. Following are some example YAMLs that show how to configure the networking with different network access options. - -For Autonomous Databases on shared Exadata infrastructure, review the following examples: - -* Configure network access [with PUBLIC access type](#autonomous-database-with-public-access-type-on-shared-exadata-infrastructure) -* Configure network access [with RESTRICTED access type](#autonomous-database-with-restricted-access-type-on-shared-exadata-infrastructure) -* Configure network access [with PRIVATE access type](#autonomous-database-with-private-access-type-on-shared-exadata-infrastructure) -* [Change the mutual TLS (mTLS) authentication setting](#allow-both-tls-and-mutual-tls-mtls-authentication-of-autonomous-database-on-shared-exadata-infrastructure) +### Option 1 - Change the Network Access from "Secure Access from Allowed IPs and VCNs Only" to "Allowing Secure Access from Anywhere" +1. Add the following parameters to the specification. An example file is availble here: [`config/samples/adb/autonomousdatabase_update_network_access.yaml`](./../../config/samples/adb/autonomousdatabase_update_network_access.yaml): -For Autonomous Databases on dedicated Exadata infrastructure, refiew the following examples: + | Attribute | Type | Description | + |----|----|----| + | `whitelistedIps` | []string | The client IP access control list (ACL). This feature is available for Autonomous Databases on [shared Exadata infrastructure](https://docs.cloud.oracle.com/Content/Database/Concepts/adboverview.htm#AEI) and on Exadata Cloud@Customer.
Only clients connecting from an IP address included in the ACL can access the Autonomous Database instance.

For shared Exadata infrastructure, this is an array of CIDR (Classless Inter-Domain Routing) notations for a subnet or VCN OCID.
Use a semicolon (;) as a deliminator between the VCN-specific subnets or IPs.
Example: `["1.1.1.1","1.1.1.0/24","ocid1.vcn.oc1.sea.","ocid1.vcn.oc1.sea.;1.1.1.1","ocid1.vcn.oc1.sea.;1.1.0.0/16"]`

For Exadata Cloud@Customer, this is an array of IP addresses or CIDR (Classless Inter-Domain Routing) notations.
Example: `["1.1.1.1","1.1.1.0/24","1.1.2.25"]`

For an update operation, if you want to delete all the IPs in the ACL, then use an array with a single empty string entry. | -* Configure network access [with access control list enabled](#autonomous-database-with-access-control-list-enabled-on-dedicated-exadata-infrastructure) + ```yaml + --- + apiVersion: database.oracle.com/v1alpha1 + kind: AutonomousDatabase + metadata: + name: autonomousdatabase-sample + spec: + action: Update + details: + autonomousDatabaseOCID: ocid1.autonomousdatabase... + whitelistedIps: + - + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey + ``` -> Note: -> -> * Operations on Exadata infrastructure require an `AutonomousDatabase` object to be in your cluster. These examples assume either the provision operation or the bind operation has been done before you begin, and the operator is authorized with API Key Authentication. -> * If you are creating an Autonomous Database, then see step 4 of [Provision an Autonomous Database](./README.md#provision-an-autonomous-database) in [Managing Oracle Autonomous Databases with Oracle Database Operator for Kubernetes](./README.md) topic to return to provisioning instructions. +2. Apply the yaml: -### Autonomous Database with PUBLIC access type on shared Exadata infrastructure + ```sh + $ kubectl apply -f config/samples/adb/autonomousdatabase_update_network_access.yaml + autonomousdatabase.database.oracle.com/autonomousdatabase-sample configured + ``` -To configure the network with PUBLIC access type, complete this procedure. +### Option 2 - Change the Network Access from "Private Endpoint Access Only" to "Allowing Secure Access from Anywhere" 1. Add the following parameters to the specification. An example file is availble here: [`config/samples/adb/autonomousdatabase_update_network_access.yaml`](./../../config/samples/adb/autonomousdatabase_update_network_access.yaml): - | Attribute | Type | Description | Required? | - |----|----|----|----| - | `networkAccess.accessType` | string | An enumeration (enum) value that defines how the database can be accessed. The value can be PUBLIC, RESTRICTED or PRIVATE. See [Types of Network Access](#types-of-network-access) for more descriptions. | Yes | + | Attribute | Type | Description | + |----|----|----| + | `privateEndpointLabel` | string | The hostname prefix for the resource. | ```yaml --- @@ -86,11 +67,10 @@ To configure the network with PUBLIC access type, complete this procedure. metadata: name: autonomousdatabase-sample spec: + action: Update details: autonomousDatabaseOCID: ocid1.autonomousdatabase... - networkAccess: - # Allow secure access from everywhere. - accessType: PUBLIC + privateEndpointLabel: "" ociConfig: configMapName: oci-cred secretName: oci-privatekey @@ -99,21 +79,20 @@ To configure the network with PUBLIC access type, complete this procedure. 2. Apply the yaml: ```sh - kubectl apply -f config/samples/adb/autonomousdatabase_update_network_access.yaml + $ kubectl apply -f config/samples/adb/autonomousdatabase_update_network_access.yaml autonomousdatabase.database.oracle.com/autonomousdatabase-sample configured ``` -### Autonomous Database with RESTRICTED access type on shared Exadata infrastructure +## Configuring Network Access with Access Control Rules (ACLs) -To configure the network with RESTRICTED access type, complete this procedure. +To configure Network Access with ACLs, complete this procedure. 1. Add the following parameters to the specification. An example file is availble here: [`config/samples/adb/autonomousdatabase_update_network_access.yaml`](./../../config/samples/adb/autonomousdatabase_update_network_access.yaml): - | Attribute | Type | Description | Required? | - |----|----|----|----| - | `networkAccess.accessType` | string | An enumerated (enum) that defines how the database can be accessed. The value can be PUBLIC, RESTRICTED or PRIVATE. See [Types of Network Access](#types-of-network-access) for more descriptions. | Yes | - | `networkAccess.accessControlList` | []string | The client IP access control list (ACL). This feature is available for Autonomous Databases on [shared Exadata infrastructure](https://docs.cloud.oracle.com/Content/Database/Concepts/adboverview.htm#AEI) and on Exadata Cloud@Customer.
Only clients connecting from an IP address included in the ACL may access the Autonomous Database instance.

For shared Exadata infrastructure, this is an array of CIDR (Classless Inter-Domain Routing) notations for a subnet or VCN OCID.
Use a semicolon (;) as a deliminator between the VCN-specific subnets or IPs.
Example: `["1.1.1.1","1.1.1.0/24","ocid1.vcn.oc1.sea.","ocid1.vcn.oc1.sea.;1.1.1.1","ocid1.vcn.oc1.sea.;1.1.0.0/16"]`

For Exadata Cloud@Customer, this is an array of IP addresses or CIDR (Classless Inter-Domain Routing) notations.
Example: `["1.1.1.1","1.1.1.0/24","1.1.2.25"]`

For an update operation, if you want to delete all the IPs in the ACL, use an array with a single empty string entry. | Yes | + | Attribute | Type | Description | + |----|----|----| + | `whitelistedIps` | []string | The client IP access control list (ACL). This feature is available for Autonomous Databases on [shared Exadata infrastructure](https://docs.cloud.oracle.com/Content/Database/Concepts/adboverview.htm#AEI) and on Exadata Cloud@Customer.
Only clients connecting from an IP address included in the ACL can access the Autonomous Database instance.

For shared Exadata infrastructure, this is an array of CIDR (Classless Inter-Domain Routing) notations for a subnet or VCN OCID.
Use a semicolon (;) as a deliminator between the VCN-specific subnets or IPs.
Example: `["1.1.1.1","1.1.1.0/24","ocid1.vcn.oc1.sea.","ocid1.vcn.oc1.sea.;1.1.1.1","ocid1.vcn.oc1.sea.;1.1.0.0/16"]`

For Exadata Cloud@Customer, this is an array of IP addresses or CIDR (Classless Inter-Domain Routing) notations.
Example: `["1.1.1.1","1.1.1.0/24","1.1.2.25"]`

For an update operation, if you want to delete all the IPs in the ACL, use an array with a single empty string entry. | ```yaml --- @@ -122,12 +101,12 @@ To configure the network with RESTRICTED access type, complete this procedure. metadata: name: autonomousdatabase-sample spec: + action: Update details: autonomousDatabaseOCID: ocid1.autonomousdatabase... networkAccess: # Restrict access by defining access control rules in an Access Control List (ACL). - accessType: RESTRICTED - accessControlList: + whitelistedIps: - 1.1.1.1 - 1.1.0.0/16 - ocid1.vcn... @@ -141,23 +120,22 @@ To configure the network with RESTRICTED access type, complete this procedure. 2. Apply the yaml: ```sh - kubectl apply -f config/samples/adb/autonomousdatabase_update_network_access.yaml + $ kubectl apply -f config/samples/adb/autonomousdatabase_update_network_access.yaml autonomousdatabase.database.oracle.com/autonomousdatabase-sample configured -### Autonomous Database with PRIVATE access type on shared Exadata infrastructure +## Configure Network Access with Private Endpoint Access Only -To configure the network with PRIVATE access type, complete this procedure +To change the Network Access to Private Endpoint Access Only, complete this procedure -1. Visit [Overview of VCNs and Subnets](https://docs.oracle.com/en-us/iaas/Content/Network/Tasks/managingVCNs_topic-Overview_of_VCNs_and_Subnets.htm#console) and [Network Security Groups](https://docs.oracle.com/en-us/iaas/Content/Network/Concepts/networksecuritygroups.htm#working) to see how to create VCNs, subnets, and network security groups (NSGs) if you haven't created them yet. The subnet and the NSG has to be in the same VCN. +1. Visit [Overview of VCNs and Subnets](https://docs.oracle.com/en-us/iaas/Content/Network/Tasks/managingVCNs_topic-Overview_of_VCNs_and_Subnets.htm#console) and [Network Security Groups](https://docs.oracle.com/en-us/iaas/Content/Network/Concepts/networksecuritygroups.htm#working) to see how to create VCNs, subnets, and network security groups (NSGs) if you haven't already created them. The subnet and the NSG must be in the same VCN. 2. Copy and paste the OCIDs of the subnet and NSG to the corresponding parameters. An example file is availble here: [`config/samples/adb/autonomousdatabase_update_network_access.yaml`](./../../config/samples/adb/autonomousdatabase_update_network_access.yaml): | Attribute | Type | Description | Required? | |----|----|----|----| - | `networkAccess.accessType` | string | An enumeration (enum) value that defines how the database can be accessed. The value can be PUBLIC, RESTRICTED or PRIVATE. See [Types of Network Access](#types-of-network-access) for more descriptions. | Yes | - | `networkAccess.privateEndpoint.subnetOCID` | string | The [OCID](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm) of the subnet the resource is associated with.

**Subnet Restrictions:**
- For bare metal DB systems and for single node virtual machine DB systems, do not use a subnet that overlaps with 192.168.16.16/28.
- For Exadata and virtual machine 2-node RAC systems, do not use a subnet that overlaps with 192.168.128.0/20.
- For Autonomous Database, setting this will disable public secure access to the database.
These subnets are used by the Oracle Clusterware private interconnect on the database instance.
Specifying an overlapping subnet will cause the private interconnect to malfunction.
This restriction applies to both the client subnet and the backup subnet. | Yes | - | `networkAccess.privateEndpoint.nsgOCIDs` | string[] | A list of the [OCIDs](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm) of the network security groups (NSGs) that this resource belongs to. Setting this to an empty array after the list is created removes the resource from all NSGs. For more information about NSGs, see [Security Rules](https://docs.cloud.oracle.com/Content/Network/Concepts/securityrules.htm).

**NsgOCIDs restrictions:**
- Autonomous Databases with private access require at least 1 Network Security Group (NSG). The nsgOCIDs array cannot be empty. | Yes | - | `networkAccess.privateEndpoint.hostnamePrefix` | string | The hostname prefix for the resource. | No | + | `subnetId` | string | The [OCID](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm) of the subnet the resource is associated with.

**Subnet Restrictions:**
- For bare metal DB systems and for single-node virtual machine DB systems, do not use a subnet that overlaps with 192.168.16.16/28.
- For Exadata and virtual machine 2-node Oracle RAC systems, do not use a subnet that overlaps with 192.168.128.0/20.
- For Autonomous Database, setting `subnetID` disables public secure access to the database.
These subnets are used by the Oracle Clusterware private interconnect on the database instance.
Specifying an overlapping subnet will cause the private interconnect to malfunction.
This restriction applies to both the client subnet and the backup subnet. | Yes | + | `nsgIds` | string[] | The list of [OCIDs](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm) for the network security groups (NSGs) to which this resource belongs. Setting `nsgIds` to an empty list removes all resources from all NSGs. For more information about NSGs, see [Security Rule](https://docs.cloud.oracle.com/Content/Network/Concepts/securityrules.htm).

**NsgIds restrictions:**
- A network security group (NSG) is optional for Autonomous Databases with private access. The nsgIds list can be empty. | No | + | `privateEndpointLabel` | string | The resource's private endpoint label.
- Setting the endpoint label to a non-empty string creates a private endpoint database.
- Resetting the endpoint label to an empty string, after the creation of the private endpoint database, changes the private endpoint database to a public endpoint database.
- Setting the endpoint label to a non-empty string value, updates to a new private endpoint database, when the database is disabled and re-enabled.
This setting cannot be updated in parallel with any of the following: licenseModel, cpuCoreCount, computeCount, computeModel, adminPassword, whitelistedIps, isMTLSConnectionRequired, dbWorkload, dbVersion, dbName, or isFreeTier. | No | ```yaml --- @@ -166,29 +144,29 @@ To configure the network with PRIVATE access type, complete this procedure metadata: name: autonomousdatabase-sample spec: + action: Update details: autonomousDatabaseOCID: ocid1.autonomousdatabase... - networkAccess: - # Assigns a private endpoint, private IP, and hostname to your database. - accessType: PRIVATE - privateEndpoint: - subnetOCID: ocid1.subnet... - nsgOCIDs: - - ocid1.networksecuritygroup... + subnetId: ocid1.subnet... + nsgIds: + - ocid1.networksecuritygroup... ociConfig: configMapName: oci-cred secretName: oci-privatekey ``` -### Allow both TLS and mutual TLS (mTLS) authentication of Autonomous Database on shared Exadata infrastructure +## Allowing TLS or Require Only Mutual TLS (mTLS) Authentication +You can choose either to require mTLS authentication and disallow TLS authentication, or allow both mTLS and TLS authentication. + +### Require mutual TLS (mTLS) authentication and Disallow TLS Authentication -If you are using either the RESTRICTED or the PRIVATE network access option, then you can choose whether to permit both TLS and mutual TLS (mTLS) authentication, or to permit only mTLS authentication. To change the mTLS authentication setting, complete the following steps: +To configure your Autonomous Database instance to require mTLS connections and disallow TLS connections, complete this procedure. 1. Add the following parameters to the specification. An example file is availble here: [`config/samples/adb/autonomousdatabase_update_mtls.yaml`](./../../config/samples/adb/autonomousdatabase_update_mtls.yaml): | Attribute | Type | Description | Required? | |----|----|----|----| - | `networkAccess.isMTLSConnectionRequired` | boolean| Indicates whether the Autonomous Database requires mTLS connections. | Yes | + | `isMtlsConnectionRequired` | boolean| Indicates whether the Autonomous Database requires mTLS connections. | Yes | ```yaml --- @@ -197,10 +175,10 @@ If you are using either the RESTRICTED or the PRIVATE network access option, the metadata: name: autonomousdatabase-sample spec: + action: Update details: autonomousDatabaseOCID: ocid1.autonomousdatabase... - networkAccess: - isMTLSConnectionRequired: false + isMtlsConnectionRequired: true ociConfig: configMapName: oci-cred secretName: oci-privatekey @@ -209,20 +187,26 @@ If you are using either the RESTRICTED or the PRIVATE network access option, the 2. Apply the yaml: ```sh - kubectl apply -f config/samples/adb/autonomousdatabase_update_mtls.yaml + $ kubectl apply -f config/samples/adb/autonomousdatabase_update_mtls.yaml autonomousdatabase.database.oracle.com/autonomousdatabase-sample configured ``` -### Autonomous Database with access control list enabled on dedicated Exadata infrastructure +### Allow both TLS and mutual TLS (mTLS) authentication -To configure the network with RESTRICTED access type using an access control list (ACL), complete this procedure. +If your Autonomous Database instance is configured to allow only mTLS connections, then you can reconfigure the instance to permit both mTLS and TLS connections. When you reconfigure the instance to permit both mTLS and TLS, you can use both authentication types at the same time, so that connections are no longer restricted to require mTLS authentication. -1. Add the following parameters to the specification. An example file is availble here: [`config/samples/adb/autonomousdatabase_update_network_access.yaml`](./../../config/samples/adb/autonomousdatabase_update_network_access.yaml): +This option only applies to Autonomous Databases on shared Exadata infrastructure. You can permit TLS connections when network access type is configured by using one of the following options: + +* **Access Control Rules (ACLs)**: with ACLs defined. +* **Private Endpoint Access Only**: with a private endpoint defined. + +Complete this procedure to allow both TLS and mTLS authentication. + +1. Add the following parameters to the specification. An example file is availble here: [`config/samples/adb/autonomousdatabase_update_mtls.yaml`](./../../config/samples/adb/autonomousdatabase_update_mtls.yaml): | Attribute | Type | Description | Required? | |----|----|----|----| - | `networkAccess.isAccessControlEnabled` | boolean | Indicates if the database-level access control is enabled.

If disabled, then database access is defined by the network security rules.

If enabled, then database access is restricted to the IP addresses defined by the rules specified with the `accessControlList` property. While specifying `accessControlList` rules is optional, if database-level access control is enabled, and no rules are specified, then the database will become inaccessible. The rules can be added later by using the `UpdateAutonomousDatabase` API operation, or by using the edit option in console.

When creating a database clone, you should specify the access control setting that you want the clone database to use. By default, database-level access control will be disabled for the clone.
This property is applicable only to Autonomous Databases on the Exadata Cloud@Customer platform. | Yes | - | `networkAccess.accessControlList` | []string | The client IP access control list (ACL). This feature is available for autonomous databases on [shared Exadata infrastructure](https://docs.cloud.oracle.com/Content/Database/Concepts/adboverview.htm#AEI) and on Exadata Cloud@Customer.
Only clients connecting from an IP address included in the ACL may access the Autonomous Database instance.

For shared Exadata infrastructure, this is an array of CIDR (Classless Inter-Domain Routing) notations for a subnet or VCN OCID.
Use a semicolon (;) as a deliminator between the VCN-specific subnets or IPs.
Example: `["1.1.1.1","1.1.1.0/24","ocid1.vcn.oc1.sea.","ocid1.vcn.oc1.sea.;1.1.1.1","ocid1.vcn.oc1.sea.;1.1.0.0/16"]`

For Exadata Cloud@Customer, this is an array of IP addresses or CIDR (Classless Inter-Domain Routing) notations.
Example: `["1.1.1.1","1.1.1.0/24","1.1.2.25"]`

For an update operation, if you want to delete all the IPs in the ACL, use an array with a single empty string entry. | Yes | + | `isMtlsConnectionRequired` | boolean| Indicates whether the Autonomous Database requires mTLS connections. | Yes | ```yaml --- @@ -231,13 +215,47 @@ To configure the network with RESTRICTED access type using an access control lis metadata: name: autonomousdatabase-sample spec: + action: Update details: autonomousDatabaseOCID: ocid1.autonomousdatabase... - networkAccess: - isAccessControlEnabled: true - accessControlList: - - 1.1.1.1 - - 1.1.0.0/16 + isMtlsConnectionRequired: false + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey + ``` + +2. Apply the yaml: + + ```sh + $ kubectl apply -f config/samples/adb/autonomousdatabase_update_mtls.yaml + autonomousdatabase.database.oracle.com/autonomousdatabase-sample configured + ``` + +## Autonomous Database with access control list enabled on dedicated Exadata infrastructure + +To configure the network access of Autonomous Database with access control list (ACL) on dedicated Exadata infrastructure, complete this procedure. + +1. Add the following parameters to the specification. An example file is available here: [`config/samples/adb/autonomousdatabase_update_network_access.yaml`](./../../config/samples/adb/autonomousdatabase_update_network_access.yaml): + + | Attribute | Type | Description | Required? | + |----|----|----|----| + | `isAccessControlEnabled` | boolean | Indicates if the database-level access control is enabled.

If disabled, then database access is defined by the network security rules.

If enabled, then database access is restricted to the IP addresses defined by the rules specified with the `accessControlList` property. While specifying `accessControlList` rules is optional, if database-level access control is enabled, and no rules are specified, then the database will become inaccessible. The rules can be added later by using the `UpdateAutonomousDatabase` API operation, or by using the edit option in console.

When creating a database clone, you should specify the access control setting that you want the clone database to use. By default, database-level access control is disabled for the clone.
This property is applicable only to Autonomous Databases on the Exadata Cloud@Customer platform. | Yes | + | `accessControlList` | []string | The client IP access control list (ACL). This feature is available for Autonomous Databases on [shared Exadata infrastructure](https://docs.cloud.oracle.com/Content/Database/Concepts/adboverview.htm#AEI) and on Exadata Cloud@Customer.
Only clients connecting from an IP address included in the ACL can access the Autonomous Database instance.

For shared Exadata infrastructure, the access control list is an array of CIDR (Classless Inter-Domain Routing) notations for a subnet or VCN OCID.
Use a semicolon (;) as a deliminator between the VCN-specific subnets or IPs.
Example: `["1.1.1.1","1.1.1.0/24","ocid1.vcn.oc1.sea.","ocid1.vcn.oc1.sea.;1.1.1.1","ocid1.vcn.oc1.sea.;1.1.0.0/16"]`

For Exadata Cloud@Customer, this is an array of IP addresses or CIDR (Classless Inter-Domain Routing) notations.
Example: `["1.1.1.1","1.1.1.0/24","1.1.2.25"]`

For an update operation, if you want to delete all the IPs in the ACL, use an array with a single empty string entry. | Yes | + + ```yaml + --- + apiVersion: database.oracle.com/v1alpha1 + kind: AutonomousDatabase + metadata: + name: autonomousdatabase-sample + spec: + action: Update + details: + autonomousDatabaseOCID: ocid1.autonomousdatabase... + isAccessControlEnabled: true + accessControlList: + - 1.1.1.1 + - 1.1.0.0/16 ociConfig: configMapName: oci-cred secretName: oci-privatekey @@ -246,5 +264,5 @@ To configure the network with RESTRICTED access type using an access control lis 2. Apply the yaml: ```sh - kubectl apply -f config/samples/adb/autonomousdatabase_update_network_access.yaml + $ kubectl apply -f config/samples/adb/autonomousdatabase_update_network_access.yaml autonomousdatabase.database.oracle.com/autonomousdatabase-sample configured diff --git a/docs/adb/README.md b/docs/adb/README.md index 1b59c4d6..e8164697 100644 --- a/docs/adb/README.md +++ b/docs/adb/README.md @@ -1,14 +1,14 @@ # Managing Oracle Autonomous Databases with Oracle Database Operator for Kubernetes -Before you use the Oracle Database Operator for Kubernetes (the operator), ensure your system meets all of the Oracle Autonomous Database (ADB) Prerequisites [ADB_PREREQUISITES](./ADB_PREREQUISITES.md). +Before you use the Oracle Database Operator for Kubernetes (the operator), ensure that your system meets all of the Oracle Autonomous Database (ADB) Prerequisites [ADB_PREREQUISITES](./ADB_PREREQUISITES.md). -As indicated in the prerequisites (see above), to interact with OCI services, either the cluster has to be authorized using Principal Instance, or using the API Key Authentication by specifying the configMap and the secret under the `ociConfig` field. +As indicated in the prerequisites (see above), to interact with OCI services, either the cluster must be authorized using Principal Instance, or the cluster must be authorized using the API Key Authentication by specifying the configMap and the secret under the `ociConfig` field. ## Required Permissions -The operator must be given the required type of access in a policy written by an administrator to manage the Autonomous Databases. See [Let database and fleet admins manage Autonomous Databases](https://docs.oracle.com/en-us/iaas/Content/Identity/Concepts/commonpolicies.htm#db-admins-manage-adb) for sample Autonomous Database policies. +The operator must be given the required type of access in a policy written by an administrator to manage the Autonomous Databases. For examples of Autonomous Database policies, see: [Let database and fleet admins manage Autonomous Databases](https://docs.oracle.com/en-us/iaas/Content/Identity/Concepts/commonpolicies.htm#db-admins-manage-adb) -The permission to view the workrequests is also required, so that the operator will update the resources when the work is done. See [Viewing Work Requests](https://docs.oracle.com/en-us/iaas/Content/ContEng/Tasks/contengviewingworkrequests.htm#contengviewingworkrequests) for sample work request policies. +Permissions to view the work requests are also required, so that the operator can update the resources when the work is done. For example work request policies, see: [Viewing Work Requests](https://docs.oracle.com/en-us/iaas/Content/ContEng/Tasks/contengviewingworkrequests.htm#contengviewingworkrequests) ## Supported Features @@ -25,14 +25,15 @@ After you create the resource, you can use the operator to perform the following * [Download instance credentials (wallets)](#download-wallets) of an Autonomous Database * [Stop/Start/Terminate](#stopstartterminate) an Autonomous Database * [Delete the resource](#delete-the-resource) from the cluster +* [Clone](#clone-an-existing-autonomous-database) an existing Autonomous Database -To debug the Oracle Autonomous Databases with Oracle Database operator, see [Debugging and troubleshooting](#debugging-and-troubleshooting) +To debug the Oracle Autonomous Databases with Oracle Database Operator, see [Debugging and troubleshooting](#debugging-and-troubleshooting) ## Provision an Autonomous Database -Follow these steps to provision an Autonomous Database that will map objects in your cluster. +To provision an Autonomous Database that will map objects in your cluster, complete the following steps: -1. Get the `Compartment OCID`. +1. Obtain the `Compartment OCID`. Log in to the Cloud Console and click `Compartment`. @@ -66,24 +67,36 @@ Follow these steps to provision an Autonomous Database that will map objects in kubectl create secret generic admin-password --from-literal=admin-password='password_here' ``` -4. Add the following fields to the AutonomousDatabase resource definition. An example `.yaml` file is available here: [`config/samples/adb/autonomousdatabase_create.yaml`](./../../config/samples/adb/autonomousdatabase_create.yaml) +4. Add the following fields to the Autonomous Database resource definition. An example `.yaml` file is available here: [`config/samples/adb/autonomousdatabase_create.yaml`](./../../config/samples/adb/autonomousdatabase_create.yaml) | Attribute | Type | Description | Required? | |----|----|----|----| - | `spec.details.compartmentOCID` | string | The [OCID](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm) of the compartment of the Autonomous Database. | Yes | + | `spec.details.compartmentId` | string | The [OCID](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm) of the compartment of the Autonomous Database. | Yes | | `spec.details.dbName` | string | The database name. The name must begin with an alphabetic character and can contain a maximum of 14 alphanumeric characters. Special characters are not permitted. The database name must be unique in the tenancy. | Yes | | `spec.details.displayName` | string | The user-friendly name for the Autonomous Database. The name does not have to be unique. | Yes | - | `spec.details.cpuCoreCount` | int | The number of OCPU cores to be made available to the database. | Yes | - | `spec.details.adminPassword` | dictionary | The password for the ADMIN user. The password must be between 12 and 30 characters long, and must contain at least 1 uppercase, 1 lowercase, and 1 numeric character. It cannot contain the double quote symbol (") or the username "admin", regardless of casing.

Either `k8sSecret.name` or `ociSecret.ocid` must be provided. If both `k8sSecret.name` and `ociSecret.ocid` appear, the Operator reads the password from the K8s secret that `k8sSecret.name` refers to. | Yes | + | `spec.details.dbWorkload` | string | The Autonomous Database workload type. The following values are valid:
`OLTP` - indicates an Autonomous Transaction Processing database
`DW` - indicates an Autonomous Data Warehouse database
`AJD` - indicates an Autonomous JSON Database
`APEX` - indicates an Autonomous Database with the Oracle APEX Application Development workload type.
This cannot be updated in parallel with any of the following: licenseModel, cpuCoreCount, computeCount, computeModel, adminPassword, whitelistedIps, isMtlsConnectionRequired, privateEndpointLabel, nsgIds, dbVersion, dbName, or isFreeTier. | No | + | `spec.details.licenseModel` | string | The Oracle license model that applies to the Oracle Autonomous Database. Bring your own license (BYOL) allows you to apply your current on-premises Oracle software licenses to equivalent, highly automated Oracle services in the cloud.License Included allows you to subscribe to new Oracle Database software licenses and the Oracle Database service. Note that when provisioning an [Autonomous Database on dedicated Exadata infrastructure](https://docs.oracle.com/en/cloud/paas/autonomous-database/index.html), this attribute must be null. It is already set at the Autonomous Exadata Infrastructure level. When provisioning an [Autonomous Database Serverless ](https://docs.oracle.com/en/cloud/paas/autonomous-database/index.html) database, if a value is not specified, the system defaults the value to `BRING_YOUR_OWN_LICENSE`. Bring your own license (BYOL) also allows you to select the DB edition using the optional parameter.
This cannot be updated in parallel with any of the following: cpuCoreCount, computeCount, dataStorageSizeInTBs, adminPassword, isMtlsConnectionRequired, dbWorkload, privateEndpointLabel, nsgIds, dbVersion, dbName, or isFreeTier. | No | + | `spec.details.dbVersion` | string | A valid Oracle Database version for Autonomous Database. | No | + | `spec.details.dataStorageSizeInTBs` | int | The size, in terabytes, of the data volume that will be created and attached to the database. This storage can later be scaled up if needed. For Autonomous Databases on dedicated Exadata infrastructure, the maximum storage value is determined by the infrastructure shape. See Characteristics of [Infrastructure Shapes](https://www.oracle.com/pls/topic/lookup?ctx=en/cloud/paas/autonomous-database&id=ATPFG-GUID-B0F033C1-CC5A-42F0-B2E7-3CECFEDA1FD1) for shape details. A full Exadata service is allocated when the Autonomous Database size is set to the upper limit (384 TB). | No | + | `spec.details.cpuCoreCount` | int | The number of CPU cores to be made available to the database. For Autonomous Databases on dedicated Exadata infrastructure, the maximum number of cores is determined by the infrastructure shape. See [Characteristics of Infrastructure Shapes](https://www.oracle.com/pls/topic/lookup?ctx=en/cloud/paas/autonomous-database&id=ATPFG-GUID-B0F033C1-CC5A-42F0-B2E7-3CECFEDA1FD1) for shape details.
**Note:** This parameter cannot be used with the `ocpuCount` parameter. | Conditional | + | `spec.details.computeModel` | string | The compute model of the Autonomous Database. This is required if using the `computeCount` parameter. If using `cpuCoreCount` then it is an error to specify `computeModel` to a non-null value. ECPU compute model is the recommended model and OCPU compute model is legacy. | Conditional | + | `spec.details.computeCount` | float32 | The compute amount (CPUs) available to the database. Minimum and maximum values depend on the compute model and whether the database is an Autonomous Database Serverless instance or an Autonomous Database on Dedicated Exadata Infrastructure.
For an Autonomous Database Serverless instance, the 'ECPU' compute model requires a minimum value of one, for databases in the elastic resource pool and minimum value of two, otherwise. Required when using the `computeModel` parameter. When using `cpuCoreCount` parameter, it is an error to specify computeCount to a non-null value. Providing `computeModel` and `computeCount` is the preferred method for both OCPU and ECPU. | Conditional | + | `spec.details.ocpuCount` | float32 | The number of OCPU cores to be made available to the database.
The following points apply:
- For Autonomous Databases on Dedicated Exadata infrastructure, to provision less than 1 core, enter a fractional value in an increment of 0.1. For example, you can provision 0.3 or 0.4 cores, but not 0.35 cores. (Note that fractional OCPU values are not supported for Autonomous Database Serverless instances.)
- To provision 1 or more cores, you must enter an integer between 1 and the maximum number of cores available for the infrastructure shape. For example, you can provision 2 cores or 3 cores, but not 2.5 cores. This applies to an Autonomous Database Serverless instance or an Autonomous Database on Dedicated Exadata Infrastructure.
- For Autonomous Database Serverless instances, this parameter is not used.
For Autonomous Databases on Dedicated Exadata infrastructure, the maximum number of cores is determined by the infrastructure shape. See [Characteristics of Infrastructure Shapes](https://www.oracle.com/pls/topic/lookup?ctx=en/cloud/paas/autonomous-database&id=ATPFG-GUID-B0F033C1-CC5A-42F0-B2E7-3CECFEDA1FD1) for shape details.
**Note:** This parameter cannot be used with the `cpuCoreCount` parameter. | Conditional | + | `spec.details.adminPassword` | dictionary | The password for the ADMIN user. The password must be between 12 and 30 characters long, and must contain at least 1 uppercase, 1 lowercase, and 1 numeric character. It cannot contain the double quote symbol (") or the username "admin", regardless of casing.

Either `k8sSecret.name` or `ociSecret.id` must be provided. If both `k8sSecret.name` and `ociSecret.id` appear, the Operator reads the password from the K8s secret that `k8sSecret.name` refers to. | Yes | | `spec.details.adminPassword.k8sSecret.name` | string | The **name** of the K8s Secret where you want to hold the password for the ADMIN user. | Conditional | - |`spec.details.adminPassword.ociSecret.ocid` | string | The **[OCID](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm)** of the [OCI Secret](https://docs.oracle.com/en-us/iaas/Content/KeyManagement/Tasks/managingsecrets.htm) where you want to hold the password for the ADMIN user. | Conditional | + |`spec.details.adminPassword.ociSecret.id` | string | The **[OCID](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm)** of the [OCI Secret](https://docs.oracle.com/en-us/iaas/Content/KeyManagement/Tasks/managingsecrets.htm) where you want to hold the password for the ADMIN user. | Conditional | | `spec.details.dataStorageSizeInTBs` | int | The size, in terabytes, of the data volume that will be created and attached to the database. This storage can later be scaled up if needed. | Yes | | `spec.details.isAutoScalingEnabled` | boolean | Indicates if auto scaling is enabled for the Autonomous Database OCPU core count. The default value is `FALSE` | No | - | `spec.details.isDedicated` | boolean | True if the database is on dedicated [Exadata infrastructure](https://docs.cloud.oracle.com/Content/Database/Concepts/adbddoverview.htm). `spec.details.autonomousContainerDatabase.k8sACD.name` or `spec.details.autonomousContainerDatabase.ociACD.ocid` has to be provided if the value is true. | No | + | `spec.details.isDedicated` | boolean | True if the database is on dedicated [Exadata infrastructure](https://docs.cloud.oracle.com/Content/Database/Concepts/adbddoverview.htm). `spec.details.autonomousContainerDatabase.k8sACD.name` or `spec.details.autonomousContainerDatabase.ociACD.id` has to be provided if the value is true. | No | + | `spec.details.isFreeTier` | boolean | Indicates if this is an Always Free resource. The default value is false. Note that Always Free Autonomous Databases have 1 CPU and 20GB of memory. For Always Free databases, memory and CPU cannot be scaled.
This cannot be updated in parallel with any of the following: licenseModel, cpuCoreCount, computeCount, computeModel, adminPassword, whitelistedIps, isMtlsConnectionRequired, privateEndpointLabel, nsgIds, dbVersion, or dbName. | No | + | `spec.details.isAccessControlEnabled` | boolean | Indicates if the database-level access control is enabled.
If disabled, database access is defined by the network security rules.
If enabled, database access is restricted to the IP addresses defined by the rules specified with the `whitelistedIps` property. While specifying `whitelistedIps` rules is optional, if database-level access control is enabled and no rules are specified, the database will become inaccessible.
When creating a database clone, the desired access control setting should be specified. By default, database-level access control will be disabled for the clone.
This property is applicable only to Autonomous Databases on the Exadata Cloud@Customer platform. For Autonomous Database Serverless instances, `whitelistedIps` is used. | No | + | `spec.details.whitelistedIps` | []string | The client IP access control list (ACL). This feature is available for [Autonomous Database Serverless](https://docs.oracle.com/en/cloud/paas/autonomous-database/index.html) and on Exadata Cloud@Customer.
Only clients connecting from an IP address included in the ACL may access the Autonomous Database instance.
If `arePrimaryWhitelistedIpsUsed` is 'TRUE' then Autonomous Database uses this primary's IP access control list (ACL) for the disaster recovery peer called `standbywhitelistedips`.
For Autonomous Database Serverless, this is an array of CIDR (classless inter-domain routing) notations for a subnet or VCN OCID (virtual cloud network Oracle Cloud ID).
Multiple IPs and VCN OCIDs should be separate strings separated by commas. However, if other configurations require multiple pieces of information, then each piece is connected with semicolon (;) as a delimiter.
Example: `["1.1.1.1","1.1.1.0/24","ocid1.vcn.oc1.sea.","ocid1.vcn.oc1.sea.;1.1.1.1","ocid1.vcn.oc1.sea.;1.1.0.0/16"]`
For Exadata Cloud@Customer, this is an array of IP addresses or CIDR notations.
Example: `["1.1.1.1","1.1.1.0/24","1.1.2.25"]`
For an update operation, if you want to delete all the IPs in the ACL, use an array with a single empty string entry.
This cannot be updated in parallel with any of the following: licenseModel, cpuCoreCount, computeCount, computeModel, adminPassword, isMtlsConnectionRequired, dbWorkload, dbVersion, dbName, or isFreeTier. | No | + | `spec.details.subnetId` | string | The [OCID](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm) of the subnet the resource is associated with.
**Subnet Restrictions:**
- For Autonomous Database, setting this will disable public secure access to the database.
These subnets are used by the Oracle Clusterware private interconnect on the database instance.
Specifying an overlapping subnet will cause the private interconnect to malfunction.
This restriction applies to both the client subnet and the backup subnet. | No | + | `spec.details.nsgIds` | []string | The list of [OCIDs](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm) for the network security groups (NSGs) to which this resource belongs. Setting this to an empty list removes all resources from all NSGs. For more information about NSGs, see [Security Rules](https://docs.cloud.oracle.com/Content/Network/Concepts/securityrules.htm).
**NsgIds restrictions:**
- A network security group (NSG) is optional for Autonomous Databases with private access. The nsgIds list can be empty. | No | + | `spec.details.privateEndpointLabel` | string | The resource's private endpoint label.
- Setting the endpoint label to a non-empty string creates a private endpoint database.
- Resetting the endpoint label to an empty string, after the creation of the private endpoint database, changes the private endpoint database to a public endpoint database.
- Setting the endpoint label to a non-empty string value, updates to a new private endpoint database, when the database is disabled and re-enabled.
This setting cannot be updated in parallel with any of the following: licenseModel, cpuCoreCount, computeCount, computeModel, adminPassword, whitelistedIps, isMTLSConnectionRequired, dbWorkload, dbVersion, dbName, or isFreeTier. | No | + | `spec.details.isMtlsConnectionRequired` | boolean | Specifies if the Autonomous Database requires mTLS connections. | No | | `spec.details.autonomousContainerDatabase.k8sACD.name` | string | The **name** of the K8s Autonomous Container Database resource | No | - | `spec.details.autonomousContainerDatabase.ociACD.ocid` | string | The Autonomous Container Database [OCID](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm). | No | + | `spec.details.autonomousContainerDatabase.ociACD.id` | string | The Autonomous Container Database [OCID](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm). | No | | `spec.details.freeformTags` | dictionary | Free-form tags for this resource. Each tag is a simple key-value pair with no predefined name, type, or namespace. For more information, see [Resource Tag](https://docs.cloud.oracle.com/Content/General/Concepts/resourcetags.htm).

Example:
`freeformTags:`
    `key1: value1`
    `key2: value2`| No | - | `spec.details.dbWorkload` | string | The Oracle Autonomous Database workload type. The following values are valid:
- OLTP - indicates an Autonomous Transaction Processing database
- DW - indicates an Autonomous Data Warehouse database
- AJD - indicates an Autonomous JSON Database
- APEX - indicates an Autonomous Database with the Oracle APEX Application Development workload type. | No | - | `spec.details.dbVersion` | string | A valid Oracle Database release for Oracle Autonomous Database. | No | | `spec.ociConfig` | dictionary | Not required when the Operator is authorized with [Instance Principal](./ADB_PREREQUISITES.md#authorized-with-instance-principal). Otherwise, you will need the values from the [Authorized with API Key Authentication](./ADB_PREREQUISITES.md#authorized-with-api-key-authentication) section. | Conditional | | `spec.ociConfig.configMapName` | string | Name of the ConfigMap that holds the local OCI configuration | Conditional | | `spec.ociConfig.secretName`| string | Name of the K8s Secret that holds the private key value | Conditional | @@ -95,8 +108,9 @@ Follow these steps to provision an Autonomous Database that will map objects in metadata: name: autonomousdatabase-sample spec: + action: Create details: - compartmentOCID: ocid1.compartment... + compartmentId: ocid1.compartment... dbName: NewADB displayName: NewADB cpuCoreCount: 1 @@ -111,7 +125,7 @@ Follow these steps to provision an Autonomous Database that will map objects in 5. Choose the type of network access (optional): - By default, the network access type is set to PUBLIC, which allows secure connections from anywhere. Uncomment the code block if you want configure the network access. See [Configuring Network Access of Autonomous Database](./NETWORK_ACCESS_OPTIONS.md) for more information. + By default, the network access type is set to PUBLIC, which allows secure connections from anywhere. Uncomment the code block if you want configure the network access. For more information, see: [Configuring Network Access of Autonomous Database](./NETWORK_ACCESS_OPTIONS.md) 6. Apply the YAML: @@ -142,7 +156,7 @@ The operator also generates the `AutonomousBackup` custom resources if a databas 3. Add the following fields to the AutonomousDatabase resource definition. An example `.yaml` file is available here: [`config/samples/adb/autonomousdatabase_bind.yaml`](./../../config/samples/adb/autonomousdatabase_bind.yaml) | Attribute | Type | Description | Required? | |----|----|----|----| - | `spec.details.autonomousDatabaseOCID` | string | The [OCID](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm) of the Autonomous Database you want to bind (create a reference) in your cluster. | Yes | + | `spec.details.id` | string | The [OCID](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm) of the Autonomous Database that you want to bind (create a reference) in your cluster. | Yes | | `spec.ociConfig` | dictionary | Not required when the Operator is authorized with [Instance Principal](./ADB_PREREQUISITES.md#authorized-with-instance-principal). Otherwise, you will need the values from the [Authorized with API Key Authentication](./ADB_PREREQUISITES.md#authorized-with-api-key-authentication) section. | Conditional | | `spec.ociConfig.configMapName` | string | Name of the ConfigMap that holds the local OCI configuration | Conditional | | `spec.ociConfig.secretName`| string | Name of the K8s Secret that holds the private key value | Conditional | @@ -154,8 +168,9 @@ The operator also generates the `AutonomousBackup` custom resources if a databas metadata: name: autonomousdatabase-sample spec: + action: Sync details: - autonomousDatabaseOCID: ocid1.autonomousdatabase... + id: ocid1.autonomousdatabase... ociConfig: configMapName: oci-cred secretName: oci-privatekey @@ -170,7 +185,7 @@ The operator also generates the `AutonomousBackup` custom resources if a databas ## Scale the OCPU core count or storage -> Note: this operation requires an `AutonomousDatabase` object to be in your cluster. To use this example, either the provision operation or the bind operation must be done, and the operator is authorized with API Key Authentication. +> Note: this operation requires an `AutonomousDatabase` object to be in your cluster. To use this example, either the provision operation or the bind operation must be completed, and the operator must be authorized with API Key Authentication. You can scale up or scale down the Oracle Autonomous Database OCPU core count or storage by updating the `cpuCoreCount` and `dataStorageSizeInTBs` parameters. The `isAutoScalingEnabled` indicates whether auto scaling is enabled. In this example, the CPU count and storage size (TB) are scaled up to 2 and the auto-scaling is turned off by updating the `autonomousdatabase-sample` custom resource. @@ -183,8 +198,9 @@ You can scale up or scale down the Oracle Autonomous Database OCPU core count or metadata: name: autonomousdatabase-sample spec: + action: Update details: - autonomousDatabaseOCID: ocid1.autonomousdatabase... + id: ocid1.autonomousdatabase... cpuCoreCount: 2 dataStorageSizeInTBs: 2 isAutoScalingEnabled: false @@ -215,8 +231,9 @@ You can rename the database by changing the values of the `dbName` and `displayN metadata: name: autonomousdatabase-sample spec: + action: Update details: - autonomousDatabaseOCID: ocid1.autonomousdatabase... + id: ocid1.autonomousdatabase... dbName: RenamedADB displayName: RenamedADB ociConfig: @@ -257,8 +274,9 @@ You can rename the database by changing the values of the `dbName` and `displayN metadata: name: autonomousdatabase-sample spec: + action: Update details: - autonomousDatabaseOCID: ocid1.autonomousdatabase... + id: ocid1.autonomousdatabase... adminPassword: k8sSecret: name: new-admin-password @@ -301,13 +319,14 @@ A client Wallet is required to connect to a shared Oracle Autonomous Database. U metadata: name: autonomousdatabase-sample spec: + action: Update details: - autonomousDatabaseOCID: ocid1.autonomousdatabase... - wallet: - name: instance-wallet - password: - k8sSecret: - name: instance-wallet-password + id: ocid1.autonomousdatabase... + wallet: + name: instance-wallet + password: + k8sSecret: + name: instance-wallet-password ociConfig: configMapName: oci-cred secretName: oci-privatekey @@ -339,12 +358,12 @@ To use the secret in a deployment, refer to [Using Secrets](https://kubernetes.i > Note: this operation requires an `AutonomousDatabase` object to be in your cluster. This example assumes the provision operation or the bind operation has been done by the users and the operator is authorized with API Key Authentication. -To start, stop, or terminate a database, use the `lifecycleState` attribute. -Here's a list of the values you can set for `lifecycleState`: +To start, stop, or terminate a database, use the `action` attribute. +Here's a list of the values you can set for `action`: -* `AVAILABLE`: to start the database -* `STOPPED`: to stop the database -* `TERMINATED`: to terminate the database +* `Start`: to start the database +* `Stop`: to stop the database +* `Terminate`: to terminate the database 1. An example .yaml file is available here: [config/samples/adb/autonomousdatabase_stop_start_terminate.yaml](./../../config/samples/adb/autonomousdatabase_stop_start_terminate.yaml) @@ -355,9 +374,9 @@ Here's a list of the values you can set for `lifecycleState`: metadata: name: autonomousdatabase-sample spec: + action: Stop details: - autonomousDatabaseOCID: ocid1.autonomousdatabase... - lifecycleState: STOPPED + id: ocid1.autonomousdatabase... ociConfig: configMapName: oci-cred secretName: oci-privatekey @@ -387,8 +406,9 @@ To delete the resource and terminate the Autonomous Database, complete these ste metadata: name: autonomousdatabase-sample spec: + action: Update details: - autonomousDatabaseOCID: ocid1.autonomousdatabase... + id: ocid1.autonomousdatabase... hardLink: true ociConfig: configMapName: oci-cred @@ -411,6 +431,83 @@ To delete the resource and terminate the Autonomous Database, complete these ste Now, you can verify that the database is in TERMINATING state on the Cloud Console. +## Clone an existing Autonomous Database + +> Note: this operation requires an `AutonomousDatabase` object to be in your cluster. This example assumes the provision operation or the bind operation has been done by the users and the operator is authorized with API Key Authentication. + +To clone an existing Autonomous Database, complete these steps: + +1. Add the following fields to the AutonomousDatabase resource definition. An example YAML file is available here: [config/samples/adb/autonomousdatabase_clone.yaml](./../../config/samples/adb/autonomousdatabase_clone.yaml) + | Attribute | Type | Description | Required? | + |----|----|----|----| + | `spec.details.id` | string | The [OCID](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm) of the source Autonomous Database that you will clone to create a new Autonomous Database. | Yes | + | `spec.clone.cloneType` | string | The Autonomous Database clone type. Accepted values are: `FULL` and `METADATA`. | No | + | `spec.clone.compartmentId` | string | The [OCID](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm) of the compartment of the Autonomous Database. | Yes | + | `spec.clone.dbName` | string | The database name. The name must begin with an alphabetic character and can contain a maximum of 14 alphanumeric characters. Special characters are not permitted. The database name must be unique in the tenancy. | Yes | + | `spec.clone.displayName` | string | The user-friendly name for the Autonomous Database. The name does not have to be unique. | Yes | + | `spec.clone.dbWorkload` | string | The Autonomous Database workload type. The following values are valid:
`OLTP` - indicates an Autonomous Transaction Processing database
`DW` - indicates an Autonomous Data Warehouse database
`AJD` - indicates an Autonomous JSON Database
`APEX` - indicates an Autonomous Database with the Oracle APEX Application Development workload type.
This cannot be updated in parallel with any of the following: licenseModel, cpuCoreCount, computeCount, computeModel, adminPassword, whitelistedIps, isMtlsConnectionRequired, privateEndpointLabel, nsgIds, dbVersion, dbName, or isFreeTier. | No | + | `spec.clone.licenseModel` | string | The Oracle license model that applies to the Oracle Autonomous Database. Bring your own license (BYOL) allows you to apply your current on-premises Oracle software licenses to equivalent, highly automated Oracle services in the cloud.License Included allows you to subscribe to new Oracle Database software licenses and the Oracle Database service. Note that when provisioning an [Autonomous Database on dedicated Exadata infrastructure](https://docs.oracle.com/en/cloud/paas/autonomous-database/index.html), this attribute must be null. It is already set at the Autonomous Exadata Infrastructure level. When provisioning an [Autonomous Database Serverless ](https://docs.oracle.com/en/cloud/paas/autonomous-database/index.html) database, if a value is not specified, the system defaults the value to `BRING_YOUR_OWN_LICENSE`. Bring your own license (BYOL) also allows you to select the DB edition using the optional parameter.
This cannot be updated in parallel with any of the following: cpuCoreCount, computeCount, dataStorageSizeInTBs, adminPassword, isMtlsConnectionRequired, dbWorkload, privateEndpointLabel, nsgIds, dbVersion, dbName, or isFreeTier. | No | + | `spec.clone.dbVersion` | string | A valid Oracle Database version for Autonomous Database. | No | + | `spec.clone.dataStorageSizeInTBs` | int | The size, in terabytes, of the data volume that will be created and attached to the database. This storage can later be scaled up if needed. For Autonomous Databases on dedicated Exadata infrastructure, the maximum storage value is determined by the infrastructure shape. See Characteristics of [Infrastructure Shapes](https://www.oracle.com/pls/topic/lookup?ctx=en/cloud/paas/autonomous-database&id=ATPFG-GUID-B0F033C1-CC5A-42F0-B2E7-3CECFEDA1FD1) for shape details. A full Exadata service is allocated when the Autonomous Database size is set to the upper limit (384 TB). | No | + | `spec.clone.cpuCoreCount` | int | The number of CPU cores to be made available to the database. For Autonomous Databases on dedicated Exadata infrastructure, the maximum number of cores is determined by the infrastructure shape. See [Characteristics of Infrastructure Shapes](https://www.oracle.com/pls/topic/lookup?ctx=en/cloud/paas/autonomous-database&id=ATPFG-GUID-B0F033C1-CC5A-42F0-B2E7-3CECFEDA1FD1) for shape details.
**Note:** This parameter cannot be used with the `ocpuCount` parameter. | Conditional | + | `spec.clone.computeModel` | string | The compute model of the Autonomous Database. This is required if using the `computeCount` parameter. If using `cpuCoreCount` then it is an error to specify `computeModel` to a non-null value. ECPU compute model is the recommended model and OCPU compute model is legacy. | Conditional | + | `spec.clone.computeCount` | float32 | The compute amount (CPUs) available to the database. Minimum and maximum values depend on the compute model and whether the database is an Autonomous Database Serverless instance or an Autonomous Database on Dedicated Exadata Infrastructure.
For an Autonomous Database Serverless instance, the 'ECPU' compute model requires a minimum value of one, for databases in the elastic resource pool and minimum value of two, otherwise. Required when using the `computeModel` parameter. When using `cpuCoreCount` parameter, it is an error to specify computeCount to a non-null value. Providing `computeModel` and `computeCount` is the preferred method for both OCPU and ECPU. | Conditional | + | `spec.clone.ocpuCount` | float32 | The number of OCPU cores to be made available to the database.
The following points apply:
- For Autonomous Databases on Dedicated Exadata infrastructure, to provision less than 1 core, enter a fractional value in an increment of 0.1. For example, you can provision 0.3 or 0.4 cores, but not 0.35 cores. (Note that fractional OCPU values are not supported for Autonomous Database Serverless instances.)
- To provision 1 or more cores, you must enter an integer between 1 and the maximum number of cores available for the infrastructure shape. For example, you can provision 2 cores or 3 cores, but not 2.5 cores. This applies to an Autonomous Database Serverless instance or an Autonomous Database on Dedicated Exadata Infrastructure.
- For Autonomous Database Serverless instances, this parameter is not used.
For Autonomous Databases on Dedicated Exadata infrastructure, the maximum number of cores is determined by the infrastructure shape. See [Characteristics of Infrastructure Shapes](https://www.oracle.com/pls/topic/lookup?ctx=en/cloud/paas/autonomous-database&id=ATPFG-GUID-B0F033C1-CC5A-42F0-B2E7-3CECFEDA1FD1) for shape details.
**Note:** This parameter cannot be used with the `cpuCoreCount` parameter. | Conditional | + | `spec.clone.adminPassword` | dictionary | The password for the ADMIN user. The password must be between 12 and 30 characters long, and must contain at least 1 uppercase, 1 lowercase, and 1 numeric character. It cannot contain the double quote symbol (") or the username "admin", regardless of casing.

Either `k8sSecret.name` or `ociSecret.id` must be provided. If both `k8sSecret.name` and `ociSecret.id` appear, the Operator reads the password from the K8s secret that `k8sSecret.name` refers to. | Yes | + | `spec.clone.adminPassword.k8sSecret.name` | string | The **name** of the K8s Secret where you want to hold the password for the ADMIN user. | Conditional | + |`spec.clone.adminPassword.ociSecret.id` | string | The **[OCID](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm)** of the [OCI Secret](https://docs.oracle.com/en-us/iaas/Content/KeyManagement/Tasks/managingsecrets.htm) where you want to hold the password for the ADMIN user. | Conditional | + | `spec.clone.dataStorageSizeInTBs` | int | The size, in terabytes, of the data volume that will be created and attached to the database. This storage can later be scaled up if needed. | Yes | + | `spec.clone.isAutoScalingEnabled` | boolean | Indicates if auto scaling is enabled for the Autonomous Database OCPU core count. The default value is `FALSE` | No | + | `spec.clone.isDedicated` | boolean | True if the database is on dedicated [Exadata infrastructure](https://docs.cloud.oracle.com/Content/Database/Concepts/adbddoverview.htm). `spec.clone.autonomousContainerDatabase.k8sACD.name` or `spec.clone.autonomousContainerDatabase.ociACD.id` has to be provided if the value is true. | No | + | `spec.clone.isFreeTier` | boolean | Indicates if this is an Always Free resource. The default value is false. Note that Always Free Autonomous Databases have 1 CPU and 20GB of memory. For Always Free databases, memory and CPU cannot be scaled.
This cannot be updated in parallel with any of the following: licenseModel, cpuCoreCount, computeCount, computeModel, adminPassword, whitelistedIps, isMtlsConnectionRequired, privateEndpointLabel, nsgIds, dbVersion, or dbName. | No | + | `spec.clone.isAccessControlEnabled` | boolean | Indicates if the database-level access control is enabled.
If disabled, database access is defined by the network security rules.
If enabled, database access is restricted to the IP addresses defined by the rules specified with the `whitelistedIps` property. While specifying `whitelistedIps` rules is optional, if database-level access control is enabled and no rules are specified, the database will become inaccessible.
When creating a database clone, the desired access control setting should be specified. By default, database-level access control will be disabled for the clone.
This property is applicable only to Autonomous Databases on the Exadata Cloud@Customer platform. For Autonomous Database Serverless instances, `whitelistedIps` is used. | No | + | `spec.clone.whitelistedIps` | []string | The client IP access control list (ACL). This feature is available for [Autonomous Database Serverless](https://docs.oracle.com/en/cloud/paas/autonomous-database/index.html) and on Exadata Cloud@Customer.
Only clients connecting from an IP address included in the ACL may access the Autonomous Database instance.
If `arePrimaryWhitelistedIpsUsed` is 'TRUE' then Autonomous Database uses this primary's IP access control list (ACL) for the disaster recovery peer called `standbywhitelistedips`.
For Autonomous Database Serverless, this is an array of CIDR (classless inter-domain routing) notations for a subnet or VCN OCID (virtual cloud network Oracle Cloud ID).
Multiple IPs and VCN OCIDs should be separate strings separated by commas, but if it’s other configurations that need multiple pieces of information then its each piece is connected with semicolon (;) as a delimiter.
Example: `["1.1.1.1","1.1.1.0/24","ocid1.vcn.oc1.sea.","ocid1.vcn.oc1.sea.;1.1.1.1","ocid1.vcn.oc1.sea.;1.1.0.0/16"]`
For Exadata Cloud@Customer, this is an array of IP addresses or CIDR notations.
Example: `["1.1.1.1","1.1.1.0/24","1.1.2.25"]`
For an update operation, if you want to delete all the IPs in the ACL, use an array with a single empty string entry.
This cannot be updated in parallel with any of the following: licenseModel, cpuCoreCount, computeCount, computeModel, adminPassword, isMtlsConnectionRequired, dbWorkload, dbVersion, dbName, or isFreeTier. | No | + | `spec.clone.subnetId` | string | The [OCID](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm) of the subnet the resource is associated with.
**Subnet Restrictions:**
- For Autonomous Database, setting this will disable public secure access to the database.
These subnets are used by the Oracle Clusterware private interconnect on the database instance.
Specifying an overlapping subnet will cause the private interconnect to malfunction.
This restriction applies to both the client subnet and the backup subnet. | No | + | `spec.clone.nsgIds` | []string | The list of [OCIDs](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm) for the network security groups (NSGs) to which this resource belongs. Setting this to an empty list removes all resources from all NSGs. For more information about NSGs, see [Security Rules](https://docs.cloud.oracle.com/Content/Network/Concepts/securityrules.htm).
**NsgIds restrictions:**
- A network security group (NSG) is optional for Autonomous Databases with private access. The nsgIds list can be empty. | No | + | `spec.clone.privateEndpointLabel` | string | The resource's private endpoint label.
- Setting the endpoint label to a non-empty string creates a private endpoint database.
- Resetting the endpoint label to an empty string, after the creation of the private endpoint database, changes the private endpoint database to a public endpoint database.
- Setting the endpoint label to a non-empty string value, updates to a new private endpoint database, when the database is disabled and re-enabled.
This setting cannot be updated in parallel with any of the following: licenseModel, cpuCoreCount, computeCount, computeModel, adminPassword, whitelistedIps, isMTLSConnectionRequired, dbWorkload, dbVersion, dbName, or isFreeTier. | No | + | `spec.clone.isMtlsConnectionRequired` | boolean | Specifies if the Autonomous Database requires mTLS connections. | No | + | `spec.clone.autonomousContainerDatabase.k8sACD.name` | string | The **name** of the K8s Autonomous Container Database resource | No | + | `spec.clone.autonomousContainerDatabase.ociACD.id` | string | The Autonomous Container Database [OCID](https://docs.cloud.oracle.com/Content/General/Concepts/identifiers.htm). | No | + | `spec.clone.freeformTags` | dictionary | Free-form tags for this resource. Each tag is a simple key-value pair with no predefined name, type, or namespace. For more information, see [Resource Tag](https://docs.cloud.oracle.com/Content/General/Concepts/resourcetags.htm).

Example:
`freeformTags:`
    `key1: value1`
    `key2: value2`| No | + | `spec.ociConfig` | dictionary | Not required when the Operator is authorized with [Instance Principal](./ADB_PREREQUISITES.md#authorized-with-instance-principal). Otherwise, you will need the values from the [Authorized with API Key Authentication](./ADB_PREREQUISITES.md#authorized-with-api-key-authentication) section. | Conditional | + | `spec.ociConfig.configMapName` | string | Name of the ConfigMap that holds the local OCI configuration | Conditional | + | `spec.ociConfig.secretName`| string | Name of the K8s Secret that holds the private key value | Conditional | + + ```yaml + --- + apiVersion: database.oracle.com/v1alpha1 + kind: AutonomousDatabase + metadata: + name: autonomousdatabase-sample + spec: + action: Clone + details: + id: ocid1.autonomousdatabase... + clone: + compartmentId: ocid1.compartment... OR ocid1.tenancy... + dbName: ClonedADB + displayName: ClonedADB + cpuCoreCount: 1 + adminPassword: + k8sSecret: + name: admin-password + dataStorageSizeInTBs: 1 + dbWorkload: OLTP + cloneType: METADATA + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey + ``` + +2. Apply the yaml + + ```sh + kubectl apply -f config/samples/adb/autonomousdatabase_clone.yaml + autonomousdatabase.database.oracle.com/autonomousdatabase-sample configured + ``` + +Now, you can verify that a cloned database with name "ClonedADB" is being provisioned on the Cloud Console. + ## Roles and Privileges requirements for Oracle Autonomous Database Controller Autonomous Database controller uses Kubernetes objects such as: diff --git a/docs/dbcs/README.md b/docs/dbcs/README.md index c8b8d5d9..2c06511c 100644 --- a/docs/dbcs/README.md +++ b/docs/dbcs/README.md @@ -1,12 +1,12 @@ -# Using the DB Operator DBCS Controller +# Using the DB Operator Oracle Base Database Service (OBDS) Controller -Oracle Cloud Infastructure (OCI) Oracle Base Database Cloud Service (BDBCS) provides single-node Database (DB) systems, deployed on virtual machines, and provides two-node Oracle Real Appliation Clusters (Oracle RAC) database systems on virtual machines. +Oracle Cloud Infastructure (OCI) Oracle Base Database Service (OBDS) provides single-node Database (DB) systems, deployed on virtual machines, and provides two-node Oracle Real Application Clusters (Oracle RAC) database systems on virtual machines. -The single-node DB systems and Oracle RAC systems on virtual machines are [co-managed Oracle Database cloud solutions](https://docs.oracle.com/en-us/iaas/Content/Database/Concepts/overview.htm). To manage the lifecycle of an OCI DBCS system, you can use the OCI Console, the REST API, or the Oracle Cloud Infrastructure command-line interface (CLI). At the granular level, you can use the Oracle Database CLI (DBCLI), Oracle Enterprise Manager, or Oracle SQL Developer. +The single-node DB systems and Oracle RAC systems on virtual machines are [co-managed Oracle Database cloud solutions](https://docs.oracle.com/en-us/iaas/Content/Database/Concepts/overview.htm). To manage the lifecycle of an OCI OBDS system, you can use the OCI Console, the REST API, or the Oracle Cloud Infrastructure command-line interface (CLI). At the granular level, you can use the Oracle Database CLI (DBCLI), Oracle Enterprise Manager, or Oracle SQL Developer. -The Oracle DB Operator DBCS Controller is a feature of the Oracle DB Operator for Kubernetes (OraOperator) which uses OCI's BDBCS service to support lifecycle management of the database systems. +The Oracle DB Operator Oracle Base Database Service (OBDS) Controller is a feature of the Oracle DB Operator for Kubernetes (OraOperator) which uses OCI's Oracle Base Database Service OBDS service to support lifecycle management of the database systems. -Note: Oracle Base Database Cloud Service (BDBCS) was previously known as Database Cloud Service (DBCS). +Note: Oracle Base Database Cloud Service (OBDS) was previously known as Database Cloud Service (DBCS). # Supported Database Editions and Versions @@ -22,55 +22,47 @@ Two-node Oracle RAC DB systems require Oracle Enterprise Edition - Extreme Perfo For standard provisioning of DB systems (using Oracle Automatic Storage Management (ASM) as your storage management software), the following database releases are supported: -- Oracle Database 21c +- Oracle Database 23ai - Oracle Database 19c -- Oracle Database 18c (18.0) -- Oracle Database 12c Release 2 (12.2) -- Oracle Database 12c Release 1 (12.1) -- Oracle Database 11g Release 2 (11.2) - For fast provisioning of single-node virtual machine database systems (using Logical Volume Manager as your storage management software), the following database releases are supported: -- Oracle Database 21c +- Oracle Database 23ai - Oracle Database 19c -- Oracle Database 18c -- Oracle Database 12c Release 2 (12.2) -# Oracle DB Operator DBCS Controller Deployment +# Oracle DB Operator Oracle Base Database Service (OBDS) Controller Deployment -To deploy OraOperator, use this [Oracle Database Operator for Kubernetes](https://github.com/oracle/oracle-database-operator/blob/main/README.md) step-by-step procedure. +To deploy Oracle Database Operator (`OraOperator`), use the [Oracle Database Operator for Kubernetes](https://github.com/oracle/oracle-database-operator/blob/main/README.md) step-by-step procedure. -After the Oracle Database Operator is deployed, you can see the DB operator pods running in the Kubernetes Cluster. As part of the OraOperator deployment, the DBCS Controller is deployed as a CRD (Custom Resource Definition). The following screen output is an example of such a deployment: -``` +After the Oracle Database Operator is deployed, you can see the DB operator pods running in the Kubernetes Cluster. As part of the `OraOperator` deployment, the OBDS Controller is deployed as a CRD (Custom Resource Definition). The following screen output is an example of such a deployment: +```bash [root@test-server oracle-database-operator]# kubectl get ns NAME STATUS AGE -cert-manager Active 2m5s -default Active 125d -kube-node-lease Active 125d -kube-public Active 125d -kube-system Active 125d -oracle-database-operator-system Active 17s <<<< namespace to deploy the Oracle Database Operator +cert-manager Active 33d +default Active 118d +kube-node-lease Active 118d +kube-public Active 118d +kube-system Active 118d +oracle-database-operator-system Active 10m <<<< namespace to deploy the Oracle Database Operator [root@test-server oracle-database-operator]# kubectl get all -n oracle-database-operator-system NAME READY STATUS RESTARTS AGE -pod/oracle-database-operator-controller-manager-665874bd57-dlhls 1/1 Running 0 28s -pod/oracle-database-operator-controller-manager-665874bd57-g2cgw 1/1 Running 0 28s -pod/oracle-database-operator-controller-manager-665874bd57-q42f8 1/1 Running 0 28s - +pod/oracle-database-operator-controller-manager-678f96f5f4-f4rhq 1/1 Running 0 10m +pod/oracle-database-operator-controller-manager-678f96f5f4-plxcp 1/1 Running 0 10m +pod/oracle-database-operator-controller-manager-678f96f5f4-qgcg8 1/1 Running 0 10m + NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE -service/oracle-database-operator-controller-manager-metrics-service ClusterIP 10.96.130.124 8443/TCP 29s -service/oracle-database-operator-webhook-service ClusterIP 10.96.4.104 443/TCP 29s - +service/oracle-database-operator-controller-manager-metrics-service ClusterIP 10.96.197.164 8443/TCP 11m +service/oracle-database-operator-webhook-service ClusterIP 10.96.35.62 443/TCP 11m + NAME READY UP-TO-DATE AVAILABLE AGE -deployment.apps/oracle-database-operator-controller-manager 3/3 3 3 29s - +deployment.apps/oracle-database-operator-controller-manager 3/3 3 3 11m + NAME DESIRED CURRENT READY AGE -replicaset.apps/oracle-database-operator-controller-manager-665874bd57 3 3 3 29s -[root@docker-test-server oracle-database-operator]# - +replicaset.apps/oracle-database-operator-controller-manager-6657bfc664 0 0 0 11m +replicaset.apps/oracle-database-operator-controller-manager-678f96f5f4 3 3 3 10m [root@test-server oracle-database-operator]# kubectl get crd NAME CREATED AT @@ -81,7 +73,7 @@ certificaterequests.cert-manager.io 2022-02-22T23:21:35Z certificates.cert-manager.io 2022-02-22T23:21:36Z challenges.acme.cert-manager.io 2022-02-22T23:21:36Z clusterissuers.cert-manager.io 2022-02-22T23:21:36Z -dbcssystems.database.oracle.com 2022-02-22T23:23:25Z <<<< CRD for DBCS Controller +dbcssystems.database.oracle.com 2022-02-22T23:23:25Z <<<< CRD for OBDS Controller issuers.cert-manager.io 2022-02-22T23:21:36Z orders.acme.cert-manager.io 2022-02-22T23:21:37Z shardingdatabases.database.oracle.com 2022-02-22T23:23:25Z @@ -89,56 +81,56 @@ singleinstancedatabases.database.oracle.com 2022-02-22T23:23:25Z ``` -# Prerequisites to deploy a DBCS system using Oracle DB Operator DBCS Controller +# Prerequisites to deploy a OBDS system using Oracle DB Operator OBDS Controller -Before you deploy a DBCS system in OCI using the Oracle DB Operator DBCS Controller, complete the following procedure. +Before you deploy a OBDS system in OCI using the Oracle DB Operator OBDS Controller, complete the following procedure. **CAUTION :** You must make the changes specified in this section before you proceed to the next section. -## 1. Create a Kubernetes Configmap. For example: We are creating a Kubernetes Configmap named `oci-cred` using the OCI account we are using as below: +## 1. Create a Kubernetes Configmap. In this example. we create a Kubernetes Configmap named `oci-cred` with the OCI account we are using: -``` +```bash kubectl create configmap oci-cred \ ---from-literal=tenancy=ocid1.tenancy.oc1..................67iypsmea \ ---from-literal=user=ocid1.user.oc1..aaaaaaaaxw3i...............ce6qzdrnmq \ ---from-literal=fingerprint=b2:7c:a8:d5:44:f5.....................:9a:55 \ +--from-literal=tenancy= \ +--from-literal=user= \ +--from-literal=fingerprint= \ --from-literal=region=us-phoenix-1 ``` ## 2. Create a Kubernetes secret `oci-privatekey` using the OCI Pem key taken from OCI console for the account you are using: -``` --- assuming the OCI Pem key to be "/root/.oci/oci_api_key.pem" +```bash +#---assuming the OCI Pem key to be "/root/.oci/oci_api_key.pem" kubectl create secret generic oci-privatekey --from-file=privatekey=/root/.oci/oci_api_key.pem ``` -## 3. Create a Kubernetes secret named `admin-password`; This passward must meet the minimum passward requirements for the OCI BDBCS Service. +## 3. Create a Kubernetes secret named `admin-password`; This passward must meet the minimum passward requirements for the OCI OBDS Service. For example: -``` --- assuming the passward has been added to a text file named "admin-password": +```bash +#-- assuming the passward has been added to a text file named "admin-password": kubectl create secret generic admin-password --from-file=./admin-password -n default ``` -## 4. Create a Kubernetes secret named `tde-password`; this passward must meet the minimum passward requirements for the OCI BDBCS Service. +## 4. Create a Kubernetes secret named `tde-password`; this passward must meet the minimum passward requirements for the OCI OBDS Service. For example: -``` --- assuming the passward has been added to a text file named "tde-password": +```bash +# -- assuming the passward has been added to a text file named "tde-password": kubectl create secret generic tde-password --from-file=./tde-password -n default ``` -## 5. Create an ssh key pair, and use its public key to create a Kubernetes secret named `oci-publickey`; the private key for this public key can be used later to access the DBCS system's host machine using ssh: +## 5. Create an SSH key pair, and use its public key to create a Kubernetes secret named `oci-publickey`; the private key for this public key can be used later to access the OBDS system's host machine using SSH: -``` -[root@test-server DBCS]# ssh-keygen -N "" -C "DBCS_System"-`date +%Y%m` -P "" +```bash +[root@test-server OBDS]# ssh-keygen -N "" -C "DBCS_System"-`date +%Y%m` -P "" Generating public/private rsa key pair. Enter file in which to save the key (/root/.ssh/id_rsa): Your identification has been saved in /root/.ssh/id_rsa. @@ -159,31 +151,35 @@ The key's randomart image is: +----[SHA256]-----+ -[root@test-server DBCS]# kubectl create secret generic oci-publickey --from-file=publickey=/root/DBCS/id_rsa.pub +[root@test-server OBDS]# kubectl create secret generic oci-publickey --from-file=publickey=/root/DBCS/id_rsa.pub ``` +# Use Cases to manage the lifecycle of an OCI OBDS System with Oracle DB Operator OBDS Controller +For more informatoin about the multiple use cases available to you to deploy and manage the OCI OBDS Service-based database using the Oracle DB Operator OBDS Controller, review this list: +[1. Deploy a DB System using OCI OBDS Service with minimal parameters](./provisioning/dbcs_service_with_minimal_parameters.md) +[2. Binding to an existing OBDS System already deployed in OCI Oracle Base Database Service](./provisioning/bind_to_existing_dbcs_system.md) +[3. Scale UP the shape of an existing OBDS System](./provisioning/scale_up_dbcs_system_shape.md) +[4. Scale DOWN the shape of an existing OBDS System](./provisioning/scale_down_dbcs_system_shape.md) +[5. Scale UP the storage of an existing OBDS System](./provisioning/scale_up_storage.md) +[6. Update License type of an existing OBDS System](./provisioning/update_license.md) +[7. Terminate an existing OBDS System](./provisioning/terminate_dbcs_system.md) +[8. Create OBDS with All Parameters with Storage Management as LVM](./provisioning/dbcs_service_with_all_parameters_lvm.md) +[9. Create OBDS with All Parameters with Storage Management as ASM](./provisioning/dbcs_service_with_all_parameters_asm.md) +[10. Deploy a 2 Node RAC DB System using OCI OBDS Service](./provisioning/dbcs_service_with_2_node_rac.md) +[11. Create PDB to an existing OBDS System already deployed in OCI OBDS Service](./provisioning/create_pdb_to_existing_dbcs_system.md) +[12. Create OBDS with PDB in OCI](./provisioning/create_dbcs_with_pdb.md) +[13. Create OBDS with KMS Vault Encryption in OCI](./provisioning/create_dbcs_with_kms.md) +[14. Migrate to KMS vault from TDE Wallet password encryption of an existing OBDS System already deployed in OCI Base OBDS Service](./provisioning/migrate_to_kms.md) +[15. Clone DB System from Existing DB System in OCI OBDS Service](./provisioning/clone_from_existing_dbcs.md) +[16. Clone DB System from Backup of Existing DB System in OCI OBDS Service](./provisioning/clone_from_backup_dbcs.md) +[17. Clone DB System from Existing Database of DB System in OCI OBDS Service](./provisioning/clone_from_database.md) -# Use Cases to manage the lifecycle of an OCI DBCS System with Oracle DB Operator DBCS Controller - -For more informatoin about the multiple use cases available to you to deploy and manage the OCI BDBCS Service-based database using the Oracle DB Operator DBCS Controller, review this list: - -[1. Deploy a DB System using OCI BDBCS Service with minimal parameters](./provisioning/dbcs_service_with_minimal_parameters.md) -[2. Binding to an existing DBCS System already deployed in OCI BDBCS Service](./provisioning/bind_to_existing_dbcs_system.md) -[3. Scale UP the shape of an existing BDBCS System](./provisioning/scale_up_dbcs_system_shape.md) -[4. Scale DOWN the shape of an existing BDBCS System](./provisioning/scale_down_dbcs_system_shape.md) -[5. Scale UP the storage of an existing BDBCS System](./provisioning/scale_up_storage.md) -[6. Update License type of an existing BDBCS System](./provisioning/update_license.md) -[7. Terminate an existing BDBCS System](./provisioning/terminate_dbcs_system.md) -[8. Create BDBCS with All Parameters with Storage Management as LVM](./provisioning/dbcs_service_with_all_parameters_lvm.md) -[9. Create BDBCS with All Parameters with Storage Management as ASM](./provisioning/dbcs_service_with_all_parameters_asm.md) -[10. Deploy a 2 Node RAC DB System using OCI BDBCS Service](./provisioning/dbcs_service_with_2_node_rac.md) - -## Connecting to OCI DBCS database deployed using Oracle DB Operator DBCS Controller +## Connecting to OCI OBDS database deployed using Oracle DB Operator OBDS Controller -After you have deployed the OCI BDBCS database with the Oracle DB Operator DBCS Controller, you can connect to the database. To see how to connect and use the database, refer to the steps in [Database Connectivity](./provisioning/database_connection.md). +After you have deployed the OCI OBDS database with the Oracle DB Operator OBDS Controller, you can connect to the database. To see how to connect and use the database, refer to the steps in [Database Connectivity](./provisioning/database_connection.md). ## Known Issues -If you encounter any issues with deployment, refer to the list of [Known Issues](./provisioning/known_issues.md) for an OCI DBCS System deployed using Oracle DB Operator DBCS Controller. +If you encounter any issues with deployment, refer to the list of [Known Issues](./provisioning/known_issues.md) for an OCI OBDS System deployed using Oracle DB Operator OBDS Controller. diff --git a/docs/dbcs/provisioning/bind_to_existing_dbcs_system.md b/docs/dbcs/provisioning/bind_to_existing_dbcs_system.md index 6fcff5de..eced7538 100644 --- a/docs/dbcs/provisioning/bind_to_existing_dbcs_system.md +++ b/docs/dbcs/provisioning/bind_to_existing_dbcs_system.md @@ -1,32 +1,32 @@ -# Binding to an existing DBCS System already deployed in OCI DBCS Service +# Binding to an existing OBDS System already deployed in OCI Oracle Base Database Service -In this use case, we bind the Oracle DB Operator DBCS Controller to an existing OCI DBCS System which has already been deployed earlier. This will help to manage the life cycle of that DBCS System using the Oracle DB Operator DBCS Controller. +In this use case, we bind the Oracle DB Operator OBDS Controller to an existing OCI OBDS System which has already been deployed earlier. This will help to manage the life cycle of that OBDS System using the Oracle DB Operator OBDS Controller. **NOTE** We are assuming that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-deploy-a-dbcs-system-using-oracle-db-operator-dbcs-controller) steps to create the configmap and the secrets required during the deployment. -This example uses `bind_to_existing_dbcs_system.yaml` to bind to an existing DBCS VMDB using Oracle DB Operator DBCS Controller with: +This example uses `bind_to_existing_dbcs_system.yaml` to bind to an existing OBDS VMDB using Oracle DB Operator OBDS Controller with: -- OCI Configmap as `oci-cred` +- OCI Configmap as `oci-cred-mumbai` - OCI Secret as `oci-privatekey` -- OCID of the existing DBCS System as `ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa` +- OCID of the existing OBDS System as `ocid1.dbsystem.oc1.ap-mumbai-1.anrg6ljrabf7htyadgsso7aessztysrwaj5gcl3tp7ce6asijm2japyvmroa` Use the file: [bind_to_existing_dbcs_system.yaml](./bind_to_existing_dbcs_system.yaml) for this use case as below: 1. Deploy the .yaml file: -```sh -[root@docker-test-server DBCS]# kubectl apply -f bind_dbcs.yaml +```bash +kubectl apply -f bind_to_existing_dbcs_system.yaml dbcssystem.database.oracle.com/dbcssystem-existing created ``` -2. Monitor the Oracle DB Operator Pod `pod/oracle-database-operator-controller-manager-665874bd57-g2cgw` for the progress of the DBCS VMDB deployment. +2. Monitor the Oracle DB Leader Operator Pod `pod/oracle-database-operator-controller-manager-665874bd57-g2cgw` for the progress of the OBDS VMDB deployment. NOTE: Check the DB Operator Pod name in your environment. -``` -[root@docker-test-server DBCS]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-g2cgw -n oracle-database-operator-system +```bash +[root@docker-test-server OBDS]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-g2cgw -n oracle-database-operator-system ``` ## Sample Output -[Here](./bind_to_existing_dbcs_system_sample_output.log) is the sample output for binding to an existing DBCS System already deployed in OCI using Oracle DB Operator DBCS Controller. +[Here](./bind_to_existing_dbcs_system_sample_output.log) is the sample output for binding to an existing OBDS System already deployed in OCI using Oracle DB Operator OBDS Controller. diff --git a/docs/dbcs/provisioning/bind_to_existing_dbcs_system.yaml b/docs/dbcs/provisioning/bind_to_existing_dbcs_system.yaml index 49647229..6ff24bc8 100644 --- a/docs/dbcs/provisioning/bind_to_existing_dbcs_system.yaml +++ b/docs/dbcs/provisioning/bind_to_existing_dbcs_system.yaml @@ -1,8 +1,8 @@ -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: DbcsSystem metadata: name: dbcssystem-existing spec: - id: "ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa" + id: "ocid1.dbsystem.oc1.ap-mumbai-1.anrg6ljrabf7htyadgsso7aessztysrwaj5gcl3tp7ce6asijm2japyvmroa" ociConfigMap: "oci-cred" ociSecret: "oci-privatekey" diff --git a/docs/dbcs/provisioning/bind_to_existing_dbcs_system_sample_output.log b/docs/dbcs/provisioning/bind_to_existing_dbcs_system_sample_output.log index 454a4452..f6505337 100644 --- a/docs/dbcs/provisioning/bind_to_existing_dbcs_system_sample_output.log +++ b/docs/dbcs/provisioning/bind_to_existing_dbcs_system_sample_output.log @@ -48,130 +48,61 @@ replicaset.apps/oracle-database-operator-controller-manager-665874bd57 3 2022-03-08T23:27:48.625Z INFO controller-runtime.manager.controller.dbcssystem OCI provider configured succesfully {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} 2022-03-08T23:27:52.513Z INFO controller-runtime.manager.controller.dbcssystem Sync information from remote DbcsSystem System successfully {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} -[root@docker-test-server test]# kubectl describe dbcssystems.database.oracle.com dbcssystem-existing +[root@docker-test-server test]# kubectl describe dbcssystems.database.oracle.com dbcssystem-existing Name: dbcssystem-existing Namespace: default Labels: Annotations: lastSuccessfulSpec: - {"dbSystem":{"compartmentId":"ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya","availabilityDomain":"O... -API Version: database.oracle.com/v1alpha1 + {"dbSystem":{"compartmentId":"","availabilityDomain":"","subnetId":"","shape":"","hostName":"","dbAdminPaswordSecret":"","dbBackupConfig":... +API Version: database.oracle.com/v4 Kind: DbcsSystem Metadata: - Creation Timestamp: 2022-03-08T23:27:48Z + Creation Timestamp: 2024-12-06T15:16:07Z Generation: 1 - Managed Fields: - API Version: database.oracle.com/v1alpha1 - Fields Type: FieldsV1 - fieldsV1: - f:metadata: - f:annotations: - .: - f:kubectl.kubernetes.io/last-applied-configuration: - f:spec: - .: - f:id: - f:ociConfigMap: - f:ociSecret: - Manager: kubectl-client-side-apply - Operation: Update - Time: 2022-03-08T23:27:48Z - API Version: database.oracle.com/v1alpha1 - Fields Type: FieldsV1 - fieldsV1: - f:metadata: - f:annotations: - f:lastSuccessfulSpec: - f:spec: - f:dbSystem: - .: - f:availabilityDomain: - f:compartmentId: - f:cpuCoreCount: - f:dbAdminPaswordSecret: - f:dbBackupConfig: - f:dbEdition: - f:dbName: - f:dbUniqueName: - f:dbVersion: - f:diskRedundancy: - f:displayName: - f:faultDomains: - f:hostName: - f:nodeCount: - f:shape: - f:sshPublicKeys: - f:subnetId: - f:status: - .: - f:availabilityDomain: - f:cpuCoreCount: - f:dataStoragePercentage: - f:dataStorageSizeInGBs: - f:dbEdition: - f:dbInfo: - f:displayName: - f:id: - f:licenseModel: - f:network: - .: - f:clientSubnet: - f:domainName: - f:hostName: - f:listenerPort: - f:scanDnsName: - f:vcnName: - f:nodeCount: - f:recoStorageSizeInGB: - f:shape: - f:state: - f:storageManagement: - f:subnetId: - f:timeZone: - f:workRequests: - Manager: manager - Operation: Update - Time: 2022-03-08T23:27:52Z - Resource Version: 55191827 - UID: 96d7bc49-33e9-42cc-8dd0-ada9a5a4c7e5 + Resource Version: 116146012 + UID: 375b1bea-9b69-4b86-a2b1-fe7750608913 Spec: - Id: ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa - Oci Config Map: oci-cred + Db System: + Availability Domain: + Compartment Id: + Db Admin Pasword Secret: + Db Backup Config: + Host Name: + Kms Config: + Shape: + Subnet Id: + Id: ocid1.dbsystem.oc1.ap-mumbai-1.anrg6ljrabf7htya6crmqdnyz5h7ngpi4azbhndm6ssdmyn7yxk2uhbvxala + Kms Config: + Oci Config Map: oci-cred-mumbai Oci Secret: oci-privatekey Status: - Availability Domain: OLou:PHX-AD-1 - Cpu Core Count: 1 + Availability Domain: OLou:AP-MUMBAI-1-AD-1 + Cpu Core Count: 2 Data Storage Percentage: 80 Data Storage Size In G Bs: 256 - Db Edition: ENTERPRISE_EDITION - Db Info: - Db Home Id: ocid1.dbhome.oc1.phx.anyhqljr5gy3jhqat52milqwt3gq6lwohhacwg5yi4mtzq7c7hag53lrkugq - Db Name: db0130 - Db Unique Name: db0130_phx1zn - Db Workload: OLTP - Id: ocid1.database.oc1.phx.anyhqljrabf7htyackgmsaqjfexoqgrzuuk33ju2q25z2al43tnd5mhhvkra - Display Name: dbsystem20220308221302 - Id: ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa - License Model: LICENSE_INCLUDED + Db Edition: ENTERPRISE_EDITION_HIGH_PERFORMANCE + Display Name: dbsystem1234 + Id: ocid1.dbsystem.oc1.ap-mumbai-1.anrg6ljrabf7htya6crmqdnyz5h7ngpi4azbhndm6ssdmyn7yxk2uhbvxala + License Model: BRING_YOUR_OWN_LICENSE Network: - Client Subnet: k8test-pubvcn - Domain Name: k8testpubvcn.k8test.oraclevcn.com - Host Name: host0130 + Client Subnet: oke-nodesubnet-quick-cluster1-2bebe95db-regional + Domain Name: subdda0b5eaa.cluster1.oraclevcn.com + Host Name: host1234 Listener Port: 1521 - Scan Dns Name: host0130-scan.k8testpubvcn.k8test.oraclevcn.com - Vcn Name: k8test + Scan Dns Name: host1234-scan.subdda0b5eaa.cluster1.oraclevcn.com + Vcn Name: oke-vcn-quick-cluster1-2bebe95db Node Count: 1 Reco Storage Size In GB: 256 - Shape: VM.Standard2.1 + Shape: VM.Standard.E5.Flex State: AVAILABLE Storage Management: ASM - Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a + Subnet Id: ocid1.subnet.oc1.ap-mumbai-1.aaaaaaaa5zpzfax66omtbmjwlv4thruyru7focnu7fjcjksujmgwmr6vpbvq Time Zone: UTC Work Requests: - Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrxivzvgzel47zuoyke5yk36o7mrgjl27vscd5z3bqptmyh3rxwbqq + Operation Id: ocid1.coreservicesworkrequest.oc1.ap-mumbai-1.abrg6ljro3fhuxevjwxlue5gqq63q7rd7uhub2ru6gd6ay6k35f4hdeqqxkq Operation Type: Create DB System Percent Complete: 100 - Time Accepted: 2022-03-08 22:13:02.999 +0000 UTC - Time Finished: 2022-03-08 23:11:50.46 +0000 UTC - Time Started: 2022-03-08 22:13:16.995 +0000 UTC -Events: -[root@docker-test-server test]# \ No newline at end of file + Time Accepted: 2024-12-06 12:12:04.031 +0000 UTC + Time Finished: 2024-12-06 13:01:20.457 +0000 UTC + Time Started: 2024-12-06 12:12:11.041 +0000 UTC +Events: \ No newline at end of file diff --git a/docs/dbcs/provisioning/clone_dbcs_system.yaml b/docs/dbcs/provisioning/clone_dbcs_system.yaml new file mode 100644 index 00000000..fd6cc1d4 --- /dev/null +++ b/docs/dbcs/provisioning/clone_dbcs_system.yaml @@ -0,0 +1,20 @@ +apiVersion: database.oracle.com/v4 +kind: DbcsSystem +metadata: + name: dbcssystem-clone + namespace: default +spec: + id: "ocid1.dbsystem.oc1.ap-mumbai-1.anrg6ljrabf7htyaqui4hoqdyzmzl65jwkncyp3bnohengniqienetsdzw2q" + ociConfigMap: "oci-cred" + ociSecret: "oci-privatekey" + setupDBCloning: true + dbClone: + dbAdminPasswordSecret: "admin-password" + dbName: "db1212" + hostName: "host1213" + displayName: "dbsystem01312" + licenseModel: "BRING_YOUR_OWN_LICENSE" + domain: "subdda0b5eaa.cluster1.oraclevcn.com" + sshPublicKeys: + - "oci-publickey" + subnetId: "ocid1.subnet.oc1.ap-mumbai-1.aaaaaaaa5zpzfax66omtbmjwlv4thruyru7focnu7fjcjksujmgwmr6vpbvq" \ No newline at end of file diff --git a/docs/dbcs/provisioning/clone_dbcs_system_from_backup.yaml b/docs/dbcs/provisioning/clone_dbcs_system_from_backup.yaml new file mode 100644 index 00000000..54280af9 --- /dev/null +++ b/docs/dbcs/provisioning/clone_dbcs_system_from_backup.yaml @@ -0,0 +1,22 @@ +apiVersion: database.oracle.com/v4 +kind: DbcsSystem +metadata: + name: dbcssystem-clone + namespace: default +spec: + ociConfigMap: "oci-cred" + ociSecret: "oci-privatekey" + setupDBCloning: true + dbBackupId: "ocid1.dbbackup.oc1.ap-mumbai-1.anrg6ljrabf7htyaae3fmnpacavkuwt2zqaj5q3gol2g6m6tirriveytoarq" + dbClone: + dbAdminPasswordSecret: "admin-password" + tdeWalletPasswordSecret: "tde-password" + dbName: "db1212" + hostName: "host1213" + displayName: "dbsystem01312" + licenseModel: "BRING_YOUR_OWN_LICENSE" + domain: "subdda0b5eaa.cluster1.oraclevcn.com" + sshPublicKeys: + - "oci-publickey" + subnetId: "ocid1.subnet.oc1.ap-mumbai-1.aaaaaaaa5zpzfax66omtbmjwlv4thruyru7focnu7fjcjksujmgwmr6vpbvq" + initialDataStorageSizeInGB: 256 \ No newline at end of file diff --git a/docs/dbcs/provisioning/clone_dbcs_system_from_backup_sample_output.log b/docs/dbcs/provisioning/clone_dbcs_system_from_backup_sample_output.log new file mode 100644 index 00000000..82531993 --- /dev/null +++ b/docs/dbcs/provisioning/clone_dbcs_system_from_backup_sample_output.log @@ -0,0 +1,75 @@ +2024-09-18T12:55:33Z INFO Starting the clone process for DBCS from backup {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df", "dbcs": {"apiVersion": "database.oracle.com/v4", "kind": "DbcsSystem", "namespace": "default", "name": "dbcssystem-clone"}} +2024-09-18T12:55:33Z INFO Retrieved existing Db System Details from OCI using Spec.Id {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T12:55:41Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T12:56:42Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T12:57:43Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T12:58:44Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T12:59:45Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:00:46Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:01:47Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:02:47Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:03:48Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:04:49Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:05:50Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:06:51Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:07:52Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:08:53Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:09:53Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:10:54Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:11:55Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:12:56Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:13:57Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:14:58Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:15:59Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:17:00Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:18:01Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:19:02Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:20:02Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:21:03Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:22:05Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:23:05Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:24:06Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:25:08Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:26:08Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:27:09Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:28:10Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:29:11Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:30:12Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:31:13Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:32:14Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:33:15Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:34:16Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:35:16Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:36:17Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:37:18Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:38:19Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:39:20Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:40:21Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:41:22Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:42:23Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:43:23Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:44:24Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:45:25Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:46:26Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:47:27Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:48:28Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:49:29Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:50:30Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:51:31Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:52:32Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:53:32Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:54:33Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:55:34Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:56:35Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:57:36Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:58:37Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T13:59:38Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T14:00:39Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T14:01:40Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T14:02:41Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T14:03:42Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T14:04:42Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T14:05:43Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T14:06:44Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T14:07:45Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} +2024-09-18T14:08:46Z INFO DB Cloning completed successfully from provided backup DB system. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "a299bb7c-22eb-4db4-9037-ce81897345df"} \ No newline at end of file diff --git a/docs/dbcs/provisioning/clone_dbcs_system_from_database.yaml b/docs/dbcs/provisioning/clone_dbcs_system_from_database.yaml new file mode 100644 index 00000000..40767739 --- /dev/null +++ b/docs/dbcs/provisioning/clone_dbcs_system_from_database.yaml @@ -0,0 +1,22 @@ +apiVersion: database.oracle.com/v4 +kind: DbcsSystem +metadata: + name: dbcssystem-clone + namespace: default +spec: + databaseId: "ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyapxtsgw6hy3kyosmrawefq2csm4kjv4d5au7biuiaabsq" + ociConfigMap: "oci-cred" + ociSecret: "oci-privatekey" + setupDBCloning: true + dbClone: + dbAdminPasswordSecret: "admin-password" + tdeWalletPasswordSecret: "tde-password" + dbName: "db1212" + hostName: "host1213" + displayName: "dbsystem01312" + licenseModel: "BRING_YOUR_OWN_LICENSE" + domain: "subdda0b5eaa.cluster1.oraclevcn.com" + sshPublicKeys: + - "oci-publickey" + subnetId: "ocid1.subnet.oc1.ap-mumbai-1.aaaaaaaa5zpzfax66omtbmjwlv4thruyru7focnu7fjcjksujmgwmr6vpbvq" + initialDataStorageSizeInGB: 256 \ No newline at end of file diff --git a/docs/dbcs/provisioning/clone_dbcs_system_from_database_sample_output.log b/docs/dbcs/provisioning/clone_dbcs_system_from_database_sample_output.log new file mode 100644 index 00000000..2881051d --- /dev/null +++ b/docs/dbcs/provisioning/clone_dbcs_system_from_database_sample_output.log @@ -0,0 +1,39 @@ +2024-09-19T19:23:08Z INFO Starting the clone process for Database {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "4c2b2567-052a-4a27-ae96-e18f655577d1", "dbcs": {"apiVersion": "database.oracle.com/v4", "kind": "DbcsSystem", "namespace": "default", "name": "dbcssystem-clone"}} +2024-09-19T19:23:08Z INFO Retrieved passwords from Kubernetes secrets {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "4c2b2567-052a-4a27-ae96-e18f655577d1"} +2024-09-19T19:23:09Z INFO Retrieved existing Database details from OCI {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "4c2b2567-052a-4a27-ae96-e18f655577d1", "DatabaseId": "ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyapxtsgw6hy3kyosmrawefq2csm4kjv4d5au7biuiaabsq"} +2024-09-20T08:51:45Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "3ea6b5b1-1196-4279-bf8f-9e663f9a5543"} +2024-09-20T08:52:46Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "3ea6b5b1-1196-4279-bf8f-9e663f9a5543"} +2024-09-20T08:53:46Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "3ea6b5b1-1196-4279-bf8f-9e663f9a5543"} +2024-09-20T08:54:47Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "3ea6b5b1-1196-4279-bf8f-9e663f9a5543"} +2024-09-20T08:55:48Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "3ea6b5b1-1196-4279-bf8f-9e663f9a5543"} +2024-09-20T08:56:49Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "3ea6b5b1-1196-4279-bf8f-9e663f9a5543"} +2024-09-20T08:57:50Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "3ea6b5b1-1196-4279-bf8f-9e663f9a5543"} +2024-09-20T08:58:51Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database +.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone" +, "reconcileID": "3ea6b5b1-1196-4279-bf8f-9e663f9a5543"} +2024-09-20T08:59:52Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "3ea6b5b1-1196-4279-bf8f-9e663f9a5543"} +2024-09-20T09:00:53Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database +.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone" +, "reconcileID": "3ea6b5b1-1196-4279-bf8f-9e663f9a5543"} +2024-09-20T09:01:53Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "3ea6b5b1-1196-4279-bf8f-9e663f9a5543"} +2024-09-20T09:02:54Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "3ea6b5b1-1196-4279-bf8f-9e663f9a5543"} +2024-09-20T09:03:55Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "3ea6b5b1-1196-4279-bf8f-9e663f9a5543"} +2024-09-20T09:04:56Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "3ea6b5b1-1196-4279-bf8f-9e663f9a5543"} +2024-09-20T09:52:39Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "3ea6b5b1-1196-4279-bf8f-9e663f9a5543"} +2024-09-20T09:53:40Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database +.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone" +, "reconcileID": "3ea6b5b1-1196-4279-bf8f-9e663f9a5543"} +2024-09-20T09:54:41Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "3ea6b5b1-1196-4279-bf8f-9e663f9a5543"} +2024-09-20T09:55:41Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database +.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone" +, "reconcileID": "3ea6b5b1-1196-4279-bf8f-9e663f9a5543"} +2024-09-20T09:56:42Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "3ea6b5b1-1196-4279-bf8f-9e663f9a5543"} +2024-09-20T09:57:43Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database +.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone" +, "reconcileID": "3ea6b5b1-1196-4279-bf8f-9e663f9a5543"} +2024-09-20T09:58:44Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "3ea6b5b1-1196-4279-bf8f-9e663f9a5543"} +2024-09-20T09:59:45Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database +.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone" +, "reconcileID": "3ea6b5b1-1196-4279-bf8f-9e663f9a5543"} +2024-09-20T10:00:46Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "3ea6b5b1-1196-4279-bf8f-9e663f9a5543"} +2024-09-20T10:01:47Z INFO DB Cloning completed successfully from provided backup DB system {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone", "reconcileID": "3ea6b5b1-1196-4279-bf8f-9e663f9a5543"} \ No newline at end of file diff --git a/docs/dbcs/provisioning/clone_dbcs_system_sample_output.log b/docs/dbcs/provisioning/clone_dbcs_system_sample_output.log new file mode 100644 index 00000000..22d86e1e --- /dev/null +++ b/docs/dbcs/provisioning/clone_dbcs_system_sample_output.log @@ -0,0 +1,60 @@ +2024-09-17T11:40:26Z INFO Starting the clone process for DBCS {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3", "dbcs": {"apiVersion": "database.oracle.com/v4", "kind": "DbcsSystem", "namespace": "default", "name": "dbcssystem-clone.yaml"}} +2024-09-17T11:40:26Z INFO Retrieved passwords from Kubernetes secrets {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T11:40:26Z INFO Retrieved existing Db System Details from OCI using Spec.Id {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T11:40:33Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T11:41:33Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T11:42:34Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T11:43:35Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T11:44:36Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T11:45:37Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T11:46:38Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T11:47:39Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T11:48:40Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T11:49:41Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T11:50:42Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T11:51:43Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T11:52:44Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T11:53:45Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T11:54:45Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T11:55:46Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T11:56:47Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T11:57:48Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T11:58:49Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T11:59:50Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T12:00:51Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T12:01:51Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T12:02:52Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T12:03:53Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T12:04:54Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T12:05:55Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T12:06:56Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T12:07:57Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T12:08:58Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T12:09:59Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T12:11:00Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T12:12:01Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T12:13:01Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T12:14:02Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T12:15:03Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T12:16:04Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T12:17:05Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T12:18:06Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T12:19:07Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T12:20:08Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T12:21:08Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T12:22:09Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T12:23:10Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T12:24:11Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T12:25:12Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T12:26:13Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T12:27:14Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T12:28:15Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T12:29:16Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T12:30:16Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T12:31:17Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T12:32:18Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T12:33:19Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T12:34:20Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T12:35:21Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T12:36:22Z INFO DB System current state is still:PROVISIONING. Sleeping for 60 seconds. {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} +2024-09-17T12:36:22Z INFO DB Cloning completed successfully from provided db system {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-clone.yaml","namespace":"default"}, "namespace": "default", "name": "dbcssystem-clone.yaml", "reconcileID": "bf0c98c6-13b6-4c15-938a-3594cd4cf1f3"} diff --git a/docs/dbcs/provisioning/clone_from_backup_dbcs.md b/docs/dbcs/provisioning/clone_from_backup_dbcs.md new file mode 100644 index 00000000..4597cff7 --- /dev/null +++ b/docs/dbcs/provisioning/clone_from_backup_dbcs.md @@ -0,0 +1,36 @@ +# Clone DB System from Backup of Existing DB System in OCI Oracle Base Database System (OBDS) + +In this use case, an existing OCI OBDS system deployed earlier with the Backup is going to be cloned. + +In order to clone OBDS to an existing OBDS system using Backup, get the details of OCID of backup in OCI OBDS. + +**NOTE:** We are assuming that before this step, you have followed the [prerequisite](./../README.md#prerequisites-to-deploy-a-dbcs-system-using-oracle-db-operator-dbcs-controller) steps to create the configmap and the secrets required during the deployment. + +This example uses `clone_dbcs_system_from_backup.yaml` to clone a Single Instance OBDS VMDB using Oracle DB Operator OBDS Controller with: + +- OCI Configmap as `oci-cred` +- OCI Secret as `oci-privatekey` +- setupDBCloning: as `true` +- OCID of Backup DB as `dbBackupId` of existing OBDS system. +- Specification for DB Cloning as `dbClone`-> `dbAdminPasswordSecret`,`tdeWalletPasswordSecret`, `dbName`,`hostName`,`displayName`,`licenseModel`,`domain`,`sshPublicKeys`,`subnetId`, `initialDataStorageSizeInGB` +**NOTE:** For the details of the parameters to be used in the .yaml file, please refer [here](./dbcs_controller_parameters.md). + +Use the file: [clone_dbcs_system_from_backup.yaml](./clone_dbcs_system_from_backup.yaml) for this use case as below: + +1. Deploy the .yaml file: +```sh +[root@docker-test-server OBDS]# kubectl apply -f clone_dbcs_system_from_backup.yaml +dbcssystem.database.oracle.com/dbcssystem-clone created +``` + +2. Monitor the Oracle DB Operator Pod `pod/oracle-database-operator-controller-manager-665874bd57-g2cgw` for the progress of the OBDS VMDB creation of PDBs. + +NOTE: Check the DB Operator Pod name in your environment. + +``` +[root@docker-test-server OBDS]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-g2cgw -n oracle-database-operator-system +``` + +## Sample Output + +[Here](./clone_dbcs_system_from_backup_sample_output.log) is the sample output for cloning an existing OBDS System deployed in OCI using Oracle DB Operator OBDS Controller. diff --git a/docs/dbcs/provisioning/clone_from_database.md b/docs/dbcs/provisioning/clone_from_database.md new file mode 100644 index 00000000..05b294b5 --- /dev/null +++ b/docs/dbcs/provisioning/clone_from_database.md @@ -0,0 +1,35 @@ +# Clone DB System from Existing Database of DB System in OCI Oracle Base Database System (OBDS) + +In this use case, an existing OCI OBDS system deployed earlier with existing Database is going to be cloned in OCI Base OBDS Service using existing Database ID. + +As an pre-requisite, get the details of OCID of database of an existing OBDS System which you want to clone. + +**NOTE:** We are assuming that before this step, you have followed the [prerequisite](./../README.md#prerequisites-to-deploy-a-dbcs-system-using-oracle-db-operator-dbcs-controller) steps to create the configmap and the secrets required during the deployment. + +This example uses `clone_dbcs_system_from_database.yaml` to clone a Single Instance OBDS VMDB using Oracle DB Operator OBDS Controller with: +- OCID of existing as `databaseId` +- OCI Configmap as `oci-cred` +- OCI Secret as `oci-privatekey` +- setupDBCloning: as `true` +- Specification of dbClone as - Details of new DB system for cloning `dbAdminPasswordSecret`,`tdeWalletPasswordSecret`, `dbName`,`hostName`,`displayName`,`licenseModel`,`domain`,`sshPublicKeys`,`subnetId`, `initialDataStorageSizeInGB` +**NOTE:** For the details of the parameters to be used in the .yaml file, please refer [here](./dbcs_controller_parameters.md). + +Use the file: [clone_dbcs_system_from_database.yaml](./clone_dbcs_system_from_database.yaml) for this use case as below: + +1. Deploy the .yaml file: +```sh +[root@docker-test-server OBDS]# kubectl apply -f clone_dbcs_system_from_database.yaml +dbcssystem.database.oracle.com/dbcssystem-clone created +``` + +2. Monitor the Oracle DB Operator Pod `pod/oracle-database-operator-controller-manager-665874bd57-g2cgw` for the progress of the OBDS VMDB creation of PDBs. + +NOTE: Check the DB Operator Pod name in your environment. + +``` +[root@docker-test-server OBDS]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-g2cgw -n oracle-database-operator-system +``` + +## Sample Output + +[Here](./clone_dbcs_system_from_database_sample_output.log) is the sample output for cloning an existing OBDS System deployed in OCI using Oracle DB Operator OBDS Controller. diff --git a/docs/dbcs/provisioning/clone_from_existing_dbcs.md b/docs/dbcs/provisioning/clone_from_existing_dbcs.md new file mode 100644 index 00000000..61665188 --- /dev/null +++ b/docs/dbcs/provisioning/clone_from_existing_dbcs.md @@ -0,0 +1,36 @@ +# Clone DB System from Existing DB System in OCI Oracle Base Database System (OBDS) + +In this use case, an existing OCI OBDS system deployed earlier is going to be cloned in OCI Oracle Base Database System (OBDS). Its a 2 Step operation. + +In order to clone OBDS to an existing OBDS system, get the OCID of DB System ID you want to clone. + +**NOTE:** We are assuming that before this step, you have followed the [prerequisite](./../README.md#prerequisites-to-deploy-a-dbcs-system-using-oracle-db-operator-dbcs-controller) steps to create the configmap and the secrets required during the deployment. + +This example uses `clone_dbcs_system.yaml` to clone a Single Instance OBDS VMDB using Oracle DB Operator OBDS Controller with: + +- OCID of existing VMDB as `id` to be cloned. +- OCI Configmap as `oci-cred` +- OCI Secret as `oci-privatekey` +- setupDBCloning: as `true` +- Specification of DB System been cloned as `dbClone` -> `dbAdminPaswordSecret`, `dbName`,`hostName`,`displayName`,`licenseModel`,`domain`,`sshPublicKeys`,`subnetId`. These must be unique and new details for new cloned DB system to be created. +**NOTE:** For the details of the parameters to be used in the .yaml file, please refer [here](./dbcs_controller_parameters.md). + +Use the file: [clone_dbcs_system.yaml](./clone_dbcs_system.yaml) for this use case as below: + +1. Deploy the .yaml file: +```sh +[root@docker-test-server DBCS]# kubectl apply -f clone_dbcs_system.yaml +dbcssystem.database.oracle.com/dbcssystem-clone created +``` + +2. Monitor the Oracle DB Operator Pod `pod/oracle-database-operator-controller-manager-665874bd57-g2cgw` for the progress of the DBCS VMDB creation of PDBs. + +NOTE: Check the DB Operator Pod name in your environment. + +``` +[root@docker-test-server DBCS]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-g2cgw -n oracle-database-operator-system +``` + +## Sample Output + +[Here](./clone_dbcs_system_sample_output.log) is the sample output for cloning an existing DBCS System deployed in OCI using Oracle DB Operator DBCS Controller. diff --git a/docs/dbcs/provisioning/create_dbcs_with_kms.md b/docs/dbcs/provisioning/create_dbcs_with_kms.md new file mode 100644 index 00000000..97d912d4 --- /dev/null +++ b/docs/dbcs/provisioning/create_dbcs_with_kms.md @@ -0,0 +1,73 @@ +# Deploy a OBDS DB System alongwith KMS Vault Encryption in OCI + +In this use case, an OCI OBDS system is deployed using Oracle DB Operator OBDS controller along with KMS Vault configuration + +**NOTE** It is assumed that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-deploy-a-dbcs-system-using-oracle-db-operator-dbcs-controller) steps to create the configmap and the secrets required during the deployment. + +## Pre-requisites for KMS Vaults related to OBDS System +There is also other set of pre-requisites for KMS Vaults related to dynamic group and policies. Please follow instructions for same. +1. Create Dynamic group with rule `ALL {resource.compartment.id =` and give it some name. +2. Create policy in your compartment for this dynamic group to access to key/vaults by database. + +```txt +Allow dynamic-group <> to manage secret-family in compartment <> +Allow dynamic-group <> to manage instance-family in compartment <> +Allow dynamic-group <> to manage database-family in compartment <> +Allow dynamic-group <> to manage keys in compartment <> +Allow dynamic-group <> to manage vaults in compartment <> +``` + +E.g + +```txt +ALL {resource.compartment.id = 'ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a'} +``` +```txt +Allow dynamic-group db_dynamic_group to manage secret-family in compartment sauahuja +Allow dynamic-group db_dynamic_group to manage instance-family in compartment sauahuja +Allow dynamic-group db_dynamic_group to manage database-family in compartment sauahuja +Allow dynamic-group db_dynamic_group to manage keys in compartment sauahuja +Allow dynamic-group db_dynamic_group to manage vaults in compartment sauahuja +``` +3. Do also create KMS Vault and KMS Key in order to use it during OBDS provisioning. We are going to refer those variables (`vaultName`, `keyName`) in the yaml file. + +This example uses `dbcs_service_with_kms.yaml` to deploy a Single Instance OBDS VMDB using Oracle DB Operator OBDS Controller with: + +- OCI Configmap as `oci-cred` +- OCI Secret as `oci-privatekey` +- Availability Domain for the OBDS VMDB as `OLou:AP-MUMBAI-1-AD-1` +- Compartment OCID as `ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a` +- Database Admin Credential as `admin-password` +- Database Name as `kmsdb` +- Oracle Database Software Image Version as `19c` +- Database Workload Type as Transaction Processing i.e. `OLTP` +- Database Hostname Prefix as `kmshost` +- Oracle VMDB Shape as `VM.Standard2.2` +- SSH Public key for the OBDS system being deployed as `oci-publickey` +- domain `subdda0b5eaa.cluster1.oraclevcn.com` +- OCID of the Subnet as `ocid1.subnet.oc1.ap-mumbai-1.aaaaaaaa5zpzfax66omtbmjwlv4thruyru7focnu7fjcjksujmgwmr6vpbvq` +- KMS Vault Name as `dbvault` +- KMS Compartment Id as `ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a` +- KMS Key Name as `dbkey` + +**NOTE:** For the details of the parameters to be used in the .yaml file, please refer [here](./dbcs_controller_parameters.md). While giving KMS Vault make sure not to pass TDE wallet password in DB creation as either of them can be only used for encryption. + +Use the file: [dbcs_service_with_kms.yaml](./dbcs_service_with_kms.yaml) for this use case as below: + +1. Deploy the .yaml file: +```bash +[root@docker-test-server OBDS]# kubectl apply -f dbcs_service_with_kms.yaml +dbcssystem.database.oracle.com/dbcssystem-create created +``` + +2. Monitor the Oracle DB Operator Pod `pod/oracle-database-operator-controller-manager-665874bd57-g2cgw` for the progress of the OBDS VMDB deployment. + +NOTE: Check the DB Operator Pod name in your environment. + +```bash +[root@docker-test-server OBDS]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-g2cgw -n oracle-database-operator-system +``` + +## Sample Output + +[Here](./dbcs_service_with_kms_sample_output.log) is the sample output for a OBDS System deployed in OCI using Oracle DB Operator OBDS Controller with KMS configurations. diff --git a/docs/dbcs/provisioning/create_dbcs_with_pdb.md b/docs/dbcs/provisioning/create_dbcs_with_pdb.md new file mode 100644 index 00000000..d68a1991 --- /dev/null +++ b/docs/dbcs/provisioning/create_dbcs_with_pdb.md @@ -0,0 +1,55 @@ +# Deploy a OBDS DB System using OCI Oracle Base Database System (OBDS) alongwith PDB + +In this use case, an OCI OBDS system is deployed using Oracle DB Operator OBDS controller along with PDB configuration + +**NOTE** It is assumed that before this step, you have followed the [prerequisite](./../README.md#prerequisites-to-deploy-a-dbcs-system-using-oracle-db-operator-dbcs-controller) steps to create the configmap and the secrets required during the deployment. + +Also, create a Kubernetes secret `pdb-password` using the file: + +```bash +#---assuming the PDB password is in ./pdb-password file" + +kubectl create secret generic pdb-password --from-file=./pdb-password -n default +``` + +This example uses `dbcs_service_with_pdb.yaml` to deploy a Single Instance OBDS VMDB using Oracle DB Operator OBDS Controller with: + +- OCI Configmap as `oci-cred` +- OCI Secret as `oci-privatekey` +- Availability Domain for the OBDS VMDB as `OLou:US-ASHBURN-AD-1` +- Compartment OCID as `ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a` +- Database Admin Credential as `admin-password` +- Database Name as `dbsystem24` +- Oracle Database Software Image Version as `21c` +- Database Workload Type as Transaction Processing i.e. `OLTP` +- Database Hostname Prefix as `host24` +- Cpu Core Count as `1` +- Oracle VMDB Shape as `VM.Standard2.1` +- SSH Public key for the OBDS system being deployed as `oci-publickey` +- domain `subd215df3e6.k8stest.oraclevcn.com` +- OCID of the Subnet as `ocid1.subnet.oc1.iad.aaaaaaaa3lmmxwsykn2jc2vphzpq6eoyoqtte3dpwg6s5fzfkti22ibol2ua` +- PDB Name as `pdb_sauahuja_11` +- TDE Wallet Password as `tde-password` +- PDB Admin Password as `pdb-password` + +**NOTE:** For the details of the parameters to be used in the .yaml file, please refer [here](./dbcs_controller_parameters.md). + +Use the file: [dbcs_service_with_pdb.yaml](./dbcs_service_with_pdb.yaml) for this use case as below: + +1. Deploy the .yaml file: +```bash +[root@docker-test-server OBDS]# kubectl apply -f dbcs_service_with_pdb.yaml +dbcssystem.database.oracle.com/dbcssystem-create-with-pdb created +``` + +2. Monitor the Oracle DB Operator Pod `pod/oracle-database-operator-controller-manager-665874bd57-g2cgw` for the progress of the OBDS VMDB deployment. + +NOTE: Check the DB Operator Pod name in your environment. + +```bash +[root@docker-test-server OBDS]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-g2cgw -n oracle-database-operator-system +``` + +## Sample Output + +[Here](./dbcs_service_with_pdb_sample_output.log) is the sample output for a OBDS System deployed in OCI using Oracle DB Operator OBDS Controller with PDB configurations. diff --git a/docs/dbcs/provisioning/create_kms.md b/docs/dbcs/provisioning/create_kms.md new file mode 100644 index 00000000..43db7037 --- /dev/null +++ b/docs/dbcs/provisioning/create_kms.md @@ -0,0 +1,50 @@ +# Create and update KMS vault to an existing DBCS System already deployed in OCI Base DBCS Service + +In this use case, an existing OCI DBCS system deployed earlier is going to have KMS Vault created and update DBCS System in OCI. Its a 2 Step operation. + +In order to create KMS Vaults to an existing DBCS system, the steps will be: + +1. Bind the existing DBCS System to DBCS Controller. +2. Apply the change to create KMS Vaults. + +**NOTE:** We are assuming that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-deploy-a-dbcs-system-using-oracle-db-operator-dbcs-controller) steps to create the configmap and the secrets required during the deployment. + +As step 1, first bind the existing DBCS System to DBCS Controller following [documentation](./../provisioning/bind_to_existing_dbcs_system.md). After successful binding, it will show as below- +```bash +kubectl get dbcssystems +NAME AGE +dbcssystem-existing 3m33s +``` +Below proceeding further create PDB Admin Password which is going to used as name suggests. + + +This example uses `dbcs_service_with_kms.yaml` to create KMS Vault to existing DBCS VMDB using Oracle DB Operator DBCS Controller with: + +- OCID of existing VMDB as `ocid1.dbsystem.oc1.iad.anuwcljsabf7htyag4akvoakzw4qk7cae55qyp7hlffbouozvyl5ngoputza` +- OCI Configmap as `oci-cred` +- OCI Secret as `oci-privatekey` +- Existing `dbSystem` used before to create DBCS system. +- kmsConfig - vaultName as "basdbvault" as an example. +- kmsConfig - keyName as "dbvaultkey" as an example. +- kmsConfig - compartmentId as "ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a" as an example. +**NOTE:** For the details of the parameters to be used in the .yaml file, please refer [here](./dbcs_controller_parameters.md). + +Use the file: [dbcs_service_with_kms.yaml](./dbcs_service_with_kms.yaml) for this use case as below: + +1. Deploy the .yaml file: +```sh +[root@docker-test-server DBCS]# kubectl apply -f createpdb_in_existing_dbcs_system_list.yaml +dbcssystem.database.oracle.com/dbcssystem-existing configured +``` + +2. Monitor the Oracle DB Operator Pod `pod/oracle-database-operator-controller-manager-665874bd57-g2cgw` for the progress of the DBCS VMDB creation of KMS Vaults. + +NOTE: Check the DB Operator Pod name in your environment. + +``` +[root@docker-test-server DBCS]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-g2cgw -n oracle-database-operator-system +``` + +## Sample Output + +[Here](./createkms_in_existing_dbcs_system_sample_output.log) is the sample output for creation of KMS Vaults on an existing DBCS System deployed in OCI using Oracle DB Operator DBCS Controller. diff --git a/docs/dbcs/provisioning/create_pdb.md b/docs/dbcs/provisioning/create_pdb.md new file mode 100644 index 00000000..610ccd41 --- /dev/null +++ b/docs/dbcs/provisioning/create_pdb.md @@ -0,0 +1,55 @@ +# Create PDB to an existing DBCS System + +In this use case, an existing OCI DBCS system deployed earlier is going to have PDB/PDBs created. Its a 2 Step operation. + +In order to create PDBs to an existing DBCS system, the steps will be: + +1. Bind the existing DBCS System to DBCS Controller. +2. Apply the change to create PDBs. + +**NOTE:** We are assuming that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-deploy-a-dbcs-system-using-oracle-db-operator-dbcs-controller) steps to create the configmap and the secrets required during the deployment. + +As step 1, first bind the existing DBCS System to DBCS Controller following [documentation](./../provisioning/bind_to_existing_dbcs_system.md). After successful binding, it will show as below- +```bash +kubectl get dbcssystems +NAME AGE +dbcssystem-existing 3m33s +``` +Below proceeding further create PDB Admin Password which is going to used as name suggests. + +Create a Kubernetes secret `pdb-password` using the file: + +```bash +#---assuming the PDB password is in ./pdb-password file" + +kubectl create secret generic pdb-password --from-file=./pdb-password -n default +``` + +This example uses `createpdb_in_existing_dbcs_system_list.yaml` to scale up a Single Instance DBCS VMDB using Oracle DB Operator DBCS Controller with: + +- OCID of existing VMDB as `ocid1.dbsystem.oc1.iad.anuwcljsabf7htyag4akvoakzw4qk7cae55qyp7hlffbouozvyl5ngoputza` +- OCI Configmap as `oci-cred` +- OCI Secret as `oci-privatekey` +- TDE Wallet Password as `tde-password` +- PDB Admin Password as `pdb-password` +**NOTE:** For the details of the parameters to be used in the .yaml file, please refer [here](./dbcs_controller_parameters.md). + +Use the file: [createpdb_in_existing_dbcs_system_list.yaml](./createpdb_in_existing_dbcs_system_list.yaml) for this use case as below: + +1. Deploy the .yaml file: +```sh +[root@docker-test-server DBCS]# kubectl apply -f createpdb_in_existing_dbcs_system_list.yaml +dbcssystem.database.oracle.com/dbcssystem-existing configured +``` + +2. Monitor the Oracle DB Operator Pod `pod/oracle-database-operator-controller-manager-665874bd57-g2cgw` for the progress of the DBCS VMDB creation of PDBs. + +NOTE: Check the DB Operator Pod name in your environment. + +``` +[root@docker-test-server DBCS]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-g2cgw -n oracle-database-operator-system +``` + +## Sample Output + +[Here](./createpdb_in_existing_dbcs_system_list_sample_output.log) is the sample output for creation of PDBs on an existing DBCS System deployed in OCI using Oracle DB Operator DBCS Controller. diff --git a/docs/dbcs/provisioning/create_pdb_to_existing_dbcs_system.md b/docs/dbcs/provisioning/create_pdb_to_existing_dbcs_system.md new file mode 100644 index 00000000..d1c4ed5b --- /dev/null +++ b/docs/dbcs/provisioning/create_pdb_to_existing_dbcs_system.md @@ -0,0 +1,55 @@ +# Create PDB to an existing OBDS System + +In this use case, an existing OCI OBDS system deployed earlier is going to have a PDB/many PDBs created. Its a 2 Step operation. + +In order to create PDBs to an existing OBDS system, the steps will be: + +1. Bind the existing OBDS System to OBDS Controller. +2. Apply the change to create PDBs. + +**NOTE:** We are assuming that before this step, you have followed the [prerequisite](./../README.md#prerequisites-to-deploy-a-dbcs-system-using-oracle-db-operator-dbcs-controller) steps to create the configmap and the secrets required during the deployment. + +As step 1, first bind the existing OBDS System to OBDS Controller following [documentation](./../provisioning/bind_to_existing_dbcs_system.md). After successful binding, it will show as below- +```bash +kubectl get dbcssystems +NAME AGE +dbcssystem-existing 3m33s +``` +Below proceeding further create PDB Admin Password which is going to used as name suggests. + +Create a Kubernetes secret `pdb-password` using the file: + +```bash +#---assuming the PDB password is in ./pdb-password file" + +kubectl create secret generic pdb-password --from-file=./pdb-password -n default +``` + +This example uses `createpdb_in_existing_dbcs_system_list.yaml` to scale up a Single Instance OBDS VMDB using Oracle DB Operator OBDS Controller with: + +- OCID of existing VMDB as `ocid1.dbsystem.oc1.iad.anuwcljsabf7htya55wz5vfil7ul3pkzpubnymp6zrp3fhgomv3fcdr2vtiq` +- OCI Configmap as `oci-cred` +- OCI Secret as `oci-privatekey` +- TDE Wallet Password as `tde-password` +- PDB Admin Password as `pdb-password` +**NOTE:** For the details of the parameters to be used in the .yaml file, please refer [here](./dbcs_controller_parameters.md). + +Use the file: [createpdb_in_existing_dbcs_system_list.yaml](./createpdb_in_existing_dbcs_system_list.yaml) for this use case as below: + +1. Deploy the .yaml file: +```sh +[root@docker-test-server OBDS]# kubectl apply -f createpdb_in_existing_dbcs_system_list.yaml +dbcssystem.database.oracle.com/dbcssystem-existing configured +``` + +2. Monitor the Oracle DB Operator Pod `pod/oracle-database-operator-controller-manager-665874bd57-g2cgw` for the progress of the OBDS VMDB creation of PDBs. + +NOTE: Check the DB Operator Pod name in your environment. + +``` +[root@docker-test-server OBDS]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-g2cgw -n oracle-database-operator-system +``` + +## Sample Output + +[Here](./createpdb_in_existing_dbcs_system_list_sample_output.log) is the sample output for creation of PDBs on an existing OBDS System deployed in OCI using Oracle DB Operator OBDS Controller. diff --git a/docs/dbcs/provisioning/createkms_in_existing_dbcs_system_sample_output.log b/docs/dbcs/provisioning/createkms_in_existing_dbcs_system_sample_output.log new file mode 100644 index 00000000..18ac916e --- /dev/null +++ b/docs/dbcs/provisioning/createkms_in_existing_dbcs_system_sample_output.log @@ -0,0 +1 @@ +# To be added \ No newline at end of file diff --git a/docs/dbcs/provisioning/createpdb_in_existing_dbcs_system_list.yaml b/docs/dbcs/provisioning/createpdb_in_existing_dbcs_system_list.yaml new file mode 100644 index 00000000..589ce0cf --- /dev/null +++ b/docs/dbcs/provisioning/createpdb_in_existing_dbcs_system_list.yaml @@ -0,0 +1,27 @@ +apiVersion: database.oracle.com/v4 +kind: DbcsSystem +metadata: + name: dbcssystem-existing +spec: + id: "ocid1.dbsystem.oc1.ap-mumbai-1.anrg6ljrabf7htyadgsso7aessztysrwaj5gcl3tp7ce6asijm2japyvmroa" + ociConfigMap: "oci-cred" + ociSecret: "oci-privatekey" + pdbConfigs: + - pdbName: "pdb_sauahuja_sdk_13" + tdeWalletPassword: "tde-password" + pdbAdminPassword: "pdb-password" + shouldPdbAdminAccountBeLocked: false + freeformTags: + Department: "Finance" + - pdbName: "pdb_sauahuja_sdk_14" + tdeWalletPassword: "tde-password" + pdbAdminPassword: "pdb-password" + shouldPdbAdminAccountBeLocked: false + freeformTags: + Department: "HR" + - pdbName: "pdb_sauahuja_sdk_15" + tdeWalletPassword: "tde-password" + pdbAdminPassword: "pdb-password" + shouldPdbAdminAccountBeLocked: false + freeformTags: + Department: "IT" \ No newline at end of file diff --git a/docs/dbcs/provisioning/createpdb_in_existing_dbcs_system_list_sample_output.log b/docs/dbcs/provisioning/createpdb_in_existing_dbcs_system_list_sample_output.log new file mode 100644 index 00000000..9bee73c8 --- /dev/null +++ b/docs/dbcs/provisioning/createpdb_in_existing_dbcs_system_list_sample_output.log @@ -0,0 +1,185 @@ +2024-08-15T14:14:55Z INFO Database details fetched successfully {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "3ed0d8d1-0d9c-4163-8c71-347c441420b4", "DatabaseId": ["ocid1.database.oc1.iad.anuwcljsabf7htya5c2ttar7axxqq6qej3allfz23nvrtx6ilka4stdmrpga"]} +2024-08-15T14:14:55Z INFO Calling createPluggableDatabase {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "3ed0d8d1-0d9c-4163-8c71-347c441420b4", "ctx:->": "context.Background.WithCancel.WithValue(type logr.contextKey, val ).WithValue(type controller.reconcileIDKey, val )", "dbcsInst:->": {"apiVersion": "database.oracle.com/v4", "kind": "DbcsSystem", "namespace": "default", "name": "dbcssystem-existing"}, "databaseIds:->": "ocid1.database.oc1.iad.anuwcljsabf7htya5c2ttar7axxqq6qej3allfz23nvrtx6ilka4stdmrpga", "compartmentId:->": "ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a"} +2024-08-15T14:14:55Z INFO Checking if the pluggable database exists {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "3ed0d8d1-0d9c-4163-8c71-347c441420b4", "PDBName": "pdb_sauahuja_sdk_13"} +2024-08-15T14:14:55Z INFO TDE wallet password retrieved successfully {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "3ed0d8d1-0d9c-4163-8c71-347c441420b4"} +2024-08-15T14:14:55Z INFO PDB admin password retrieved successfully {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "3ed0d8d1-0d9c-4163-8c71-347c441420b4"} +2024-08-15T14:14:55Z INFO Creating pluggable database {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "3ed0d8d1-0d9c-4163-8c71-347c441420b4", "PDBName": "pdb_sauahuja_sdk_13"} +2024-08-15T14:14:56Z INFO Pluggable database creation initiated {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "3ed0d8d1-0d9c-4163-8c71-347c441420b4", "PDBName": "pdb_sauahuja_sdk_13", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htyahtowm4kb7rwjemwjnyyxy2nv525qqqpmjue2lua3rihq"} +2024-08-15T14:14:56Z INFO Checking pluggable database status {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "3ed0d8d1-0d9c-4163-8c71-347c441420b4", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htyahtowm4kb7rwjemwjnyyxy2nv525qqqpmjue2lua3rihq", "Status": "PROVISIONING"} +2024-08-15T14:15:26Z INFO Checking pluggable database status {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "3ed0d8d1-0d9c-4163-8c71-347c441420b4", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htyahtowm4kb7rwjemwjnyyxy2nv525qqqpmjue2lua3rihq", "Status": "PROVISIONING"} +2024-08-15T14:15:57Z INFO Checking pluggable database status {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "3ed0d8d1-0d9c-4163-8c71-347c441420b4", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htyahtowm4kb7rwjemwjnyyxy2nv525qqqpmjue2lua3rihq", "Status": "PROVISIONING"} +2024-08-15T14:16:27Z INFO Checking pluggable database status {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "3ed0d8d1-0d9c-4163-8c71-347c441420b4", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htyahtowm4kb7rwjemwjnyyxy2nv525qqqpmjue2lua3rihq", "Status": "PROVISIONING"} +2024-08-15T14:16:57Z INFO Checking pluggable database status {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "3ed0d8d1-0d9c-4163-8c71-347c441420b4", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htyahtowm4kb7rwjemwjnyyxy2nv525qqqpmjue2lua3rihq", "Status": "PROVISIONING"} +2024-08-15T14:17:27Z INFO Checking pluggable database status {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "3ed0d8d1-0d9c-4163-8c71-347c441420b4", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htyahtowm4kb7rwjemwjnyyxy2nv525qqqpmjue2lua3rihq", "Status": "PROVISIONING"} +2024-08-15T14:17:57Z INFO Checking pluggable database status {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "3ed0d8d1-0d9c-4163-8c71-347c441420b4", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htyahtowm4kb7rwjemwjnyyxy2nv525qqqpmjue2lua3rihq", "Status": "AVAILABLE"} +2024-08-15T14:17:57Z INFO Pluggable database successfully created {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "3ed0d8d1-0d9c-4163-8c71-347c441420b4", "PDBName": "pdb_sauahuja_sdk_13", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htyahtowm4kb7rwjemwjnyyxy2nv525qqqpmjue2lua3rihq"} +2024-08-15T14:17:59Z INFO Database details fetched successfully {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "3ed0d8d1-0d9c-4163-8c71-347c441420b4", "DatabaseId": ["ocid1.database.oc1.iad.anuwcljsabf7htya5c2ttar7axxqq6qej3allfz23nvrtx6ilka4stdmrpga"]} +2024-08-15T14:17:59Z INFO Calling createPluggableDatabase {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "3ed0d8d1-0d9c-4163-8c71-347c441420b4", "ctx:->": "context.Background.WithCancel.WithValue(type logr.contextKey, val ).WithValue(type controller.reconcileIDKey, val )", "dbcsInst:->": {"apiVersion": "database.oracle.com/v4", "kind": "DbcsSystem", "namespace": "default", "name": "dbcssystem-existing"}, "databaseIds:->": "ocid1.database.oc1.iad.anuwcljsabf7htya5c2ttar7axxqq6qej3allfz23nvrtx6ilka4stdmrpga", "compartmentId:->": "ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a"} +2024-08-15T14:17:59Z INFO Checking if the pluggable database exists {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "3ed0d8d1-0d9c-4163-8c71-347c441420b4", "PDBName": "pdb_sauahuja_sdk_14"} +2024-08-15T14:17:59Z INFO TDE wallet password retrieved successfully {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "3ed0d8d1-0d9c-4163-8c71-347c441420b4"} +2024-08-15T14:17:59Z INFO PDB admin password retrieved successfully {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "3ed0d8d1-0d9c-4163-8c71-347c441420b4"} +2024-08-15T14:18:00Z INFO Creating pluggable database {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "3ed0d8d1-0d9c-4163-8c71-347c441420b4", "PDBName": "pdb_sauahuja_sdk_14"} +2024-08-15T14:18:00Z INFO Pluggable database creation initiated {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "3ed0d8d1-0d9c-4163-8c71-347c441420b4", "PDBName": "pdb_sauahuja_sdk_14", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htyajgpwlaeyxj72m773xrqm6a4masvm5symlmine6v47llq"} +2024-08-15T14:18:01Z INFO Checking pluggable database status {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "3ed0d8d1-0d9c-4163-8c71-347c441420b4", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htyajgpwlaeyxj72m773xrqm6a4masvm5symlmine6v47llq", "Status": "PROVISIONING"} +2024-08-15T14:18:31Z INFO Checking pluggable database status {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "3ed0d8d1-0d9c-4163-8c71-347c441420b4", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htyajgpwlaeyxj72m773xrqm6a4masvm5symlmine6v47llq", "Status": "PROVISIONING"} +2024-08-15T14:19:01Z INFO Checking pluggable database status {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "3ed0d8d1-0d9c-4163-8c71-347c441420b4", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htyajgpwlaeyxj72m773xrqm6a4masvm5symlmine6v47llq", "Status": "PROVISIONING"} +2024-08-15T14:19:31Z INFO Checking pluggable database status {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "3ed0d8d1-0d9c-4163-8c71-347c441420b4", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htyajgpwlaeyxj72m773xrqm6a4masvm5symlmine6v47llq", "Status": "PROVISIONING"} +2024-08-15T14:20:01Z INFO Checking pluggable database status {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "3ed0d8d1-0d9c-4163-8c71-347c441420b4", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htyajgpwlaeyxj72m773xrqm6a4masvm5symlmine6v47llq", "Status": "PROVISIONING"} +2024-08-15T14:20:32Z INFO Checking pluggable database status {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "3ed0d8d1-0d9c-4163-8c71-347c441420b4", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htyajgpwlaeyxj72m773xrqm6a4masvm5symlmine6v47llq", "Status": "PROVISIONING"} +2024-08-15T14:21:02Z INFO Checking pluggable database status {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "3ed0d8d1-0d9c-4163-8c71-347c441420b4", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htyajgpwlaeyxj72m773xrqm6a4masvm5symlmine6v47llq", "Status": "AVAILABLE"} +2024-08-15T14:21:02Z INFO Pluggable database successfully created {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "3ed0d8d1-0d9c-4163-8c71-347c441420b4", "PDBName": "pdb_sauahuja_sdk_14", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htyajgpwlaeyxj72m773xrqm6a4masvm5symlmine6v47llq"} +2024-08-15T14:21:03Z INFO Database details fetched successfully {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "3ed0d8d1-0d9c-4163-8c71-347c441420b4", "DatabaseId": ["ocid1.database.oc1.iad.anuwcljsabf7htya5c2ttar7axxqq6qej3allfz23nvrtx6ilka4stdmrpga"]} +2024-08-15T14:21:03Z INFO Calling createPluggableDatabase {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "3ed0d8d1-0d9c-4163-8c71-347c441420b4", "ctx:->": "context.Background.WithCancel.WithValue(type logr.contextKey, val ).WithValue(type controller.reconcileIDKey, val )", "dbcsInst:->": {"apiVersion": "database.oracle.com/v4", "kind": "DbcsSystem", "namespace": "default", "name": "dbcssystem-existing"}, "databaseIds:->": "ocid1.database.oc1.iad.anuwcljsabf7htya5c2ttar7axxqq6qej3allfz23nvrtx6ilka4stdmrpga", "compartmentId:->": "ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a"} +2024-08-15T14:21:03Z INFO Checking if the pluggable database exists {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "3ed0d8d1-0d9c-4163-8c71-347c441420b4", "PDBName": "pdb_sauahuja_sdk_15"} +2024-08-15T14:21:03Z INFO TDE wallet password retrieved successfully {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "3ed0d8d1-0d9c-4163-8c71-347c441420b4"} +2024-08-15T14:21:03Z INFO PDB admin password retrieved successfully {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "3ed0d8d1-0d9c-4163-8c71-347c441420b4"} +2024-08-15T14:21:04Z INFO Creating pluggable database {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "3ed0d8d1-0d9c-4163-8c71-347c441420b4", "PDBName": "pdb_sauahuja_sdk_15"} +2024-08-15T14:21:05Z INFO Pluggable database creation initiated {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "3ed0d8d1-0d9c-4163-8c71-347c441420b4", "PDBName": "pdb_sauahuja_sdk_15", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htyaq2s5xfn6jpkehpcoclcrzgksbxsj426dynsqyq7ajhla"} +2024-08-15T14:21:05Z INFO Checking pluggable database status {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "3ed0d8d1-0d9c-4163-8c71-347c441420b4", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htyaq2s5xfn6jpkehpcoclcrzgksbxsj426dynsqyq7ajhla", "Status": "PROVISIONING"} +2024-08-15T14:21:35Z INFO Checking pluggable database status {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "3ed0d8d1-0d9c-4163-8c71-347c441420b4", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htyaq2s5xfn6jpkehpcoclcrzgksbxsj426dynsqyq7ajhla", "Status": "PROVISIONING"} +2024-08-15T14:22:05Z INFO Checking pluggable database status {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "3ed0d8d1-0d9c-4163-8c71-347c441420b4", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htyaq2s5xfn6jpkehpcoclcrzgksbxsj426dynsqyq7ajhla", "Status": "PROVISIONING"} +2024-08-15T14:22:36Z INFO Checking pluggable database status {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "3ed0d8d1-0d9c-4163-8c71-347c441420b4", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htyaq2s5xfn6jpkehpcoclcrzgksbxsj426dynsqyq7ajhla", "Status": "PROVISIONING"} +2024-08-15T14:23:06Z INFO Checking pluggable database status {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "3ed0d8d1-0d9c-4163-8c71-347c441420b4", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htyaq2s5xfn6jpkehpcoclcrzgksbxsj426dynsqyq7ajhla", "Status": "AVAILABLE"} +2024-08-15T14:23:06Z INFO Pluggable database successfully created {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "3ed0d8d1-0d9c-4163-8c71-347c441420b4", "PDBName": "pdb_sauahuja_sdk_15", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htyaq2s5xfn6jpkehpcoclcrzgksbxsj426dynsqyq7ajhla"} + + +# kubectl describe dbcssystems.database.oracle.com +Name: dbcssystem-existing +Namespace: default +Labels: +Annotations: lastSuccessfulSpec: + {"dbSystem":{"compartmentId":"","availabilityDomain":"","subnetId":"","shape":"","hostName":"","dbAdminPaswordSecret":"","dbBackupConfig":... +API Version: database.oracle.com/v4 +Kind: DbcsSystem +Metadata: + Creation Timestamp: 2024-12-10T10:54:17Z + Generation: 4 + Resource Version: 117823935 + UID: c9da1245-3582-4926-b311-c24d75e75003 +Spec: + Db System: + Availability Domain: + Compartment Id: + Db Admin Pasword Secret: + Db Backup Config: + Host Name: + Kms Config: + Shape: + Subnet Id: + Id: ocid1.dbsystem.oc1.ap-mumbai-1.anrg6ljrabf7htyadgsso7aessztysrwaj5gcl3tp7ce6asijm2japyvmroa + Kms Config: + Oci Config Map: oci-cred-mumbai + Oci Secret: oci-privatekey + Pdb Configs: + Freeform Tags: + Department: Finance + Pdb Admin Password: pdb-password + Pdb Name: pdb_sauahuja_sdk_13 + Should Pdb Admin Account Be Locked: false + Tde Wallet Password: tde-password + Freeform Tags: + Department: HR + Pdb Admin Password: pdb-password + Pdb Name: pdb_sauahuja_sdk_14 + Should Pdb Admin Account Be Locked: false + Tde Wallet Password: tde-password + Freeform Tags: + Department: IT + Pdb Admin Password: pdb-password + Pdb Name: pdb_sauahuja_sdk_15 + Should Pdb Admin Account Be Locked: false + Tde Wallet Password: tde-password +Status: + Availability Domain: OLou:AP-MUMBAI-1-AD-1 + Cpu Core Count: 1 + Data Storage Percentage: 80 + Data Storage Size In G Bs: 512 + Db Clone Status: + Db Db Unique Name: + Host Name: + Db Edition: ENTERPRISE_EDITION_HIGH_PERFORMANCE + Display Name: dbsystem1234 + Id: ocid1.dbsystem.oc1.ap-mumbai-1.anrg6ljrabf7htyadgsso7aessztysrwaj5gcl3tp7ce6asijm2japyvmroa + Kms Details Status: + License Model: BRING_YOUR_OWN_LICENSE + Network: + Client Subnet: oke-nodesubnet-quick-cluster1-2bebe95db-regional + Domain Name: subdda0b5eaa.cluster1.oraclevcn.com + Host Name: host1234 + Listener Port: 1521 + Scan Dns Name: host1234-scan.subdda0b5eaa.cluster1.oraclevcn.com + Vcn Name: oke-vcn-quick-cluster1-2bebe95db + Node Count: 1 + Pdb Details Status: + Pdb Config Status: + Freeform Tags: + Department: IT + Pdb Name: pdb_sauahuja_sdk_15 + Pdb State: AVAILABLE + Pluggable Database Id: ocid1.pluggabledatabase.oc1.ap-mumbai-1.anrg6ljrabf7htyazfddwgjlmpm3tctcnmqe7zwefzghr4wttij6u4lhh7bq + Pdb Config Status: + Pdb Name: cdb1_pdb1 + Pdb State: AVAILABLE + Pluggable Database Id: ocid1.pluggabledatabase.oc1.ap-mumbai-1.anrg6ljrabf7htyakgj4wuabus6z5kmalvob6r6b7vivkbsmmh7bjprzbuwa + Pdb Config Status: + Freeform Tags: + Department: Finance + Pdb Name: pdb_sauahuja_sdk_13 + Pdb State: AVAILABLE + Pluggable Database Id: ocid1.pluggabledatabase.oc1.ap-mumbai-1.anrg6ljrabf7htyakkcrbhf6cit3z2hbcvded5g2rc7r5obbxeax7dv527xq + Pdb Config Status: + Freeform Tags: + Department: HR + Pdb Name: pdb_sauahuja_sdk_14 + Pdb State: AVAILABLE + Pluggable Database Id: ocid1.pluggabledatabase.oc1.ap-mumbai-1.anrg6ljrabf7htyaqnht5ctcopuntaj74ptum27tbdk5rouvnfq5f2y3eyna + Reco Storage Size In GB: 256 + Shape: VM.Standard2.1 + State: AVAILABLE + Storage Management: ASM + Subnet Id: ocid1.subnet.oc1.ap-mumbai-1.aaaaaaaa5zpzfax66omtbmjwlv4thruyru7focnu7fjcjksujmgwmr6vpbvq + Time Zone: UTC + Work Requests: + Operation Id: ocid1.coreservicesworkrequest.oc1.ap-mumbai-1.abrg6ljrtpnjzjidageolva6ytlzjfb2lqhbbrivm4lsb67xyjzyyke6bt4a + Operation Type: Update Shape + Percent Complete: 100 + Time Accepted: 2024-12-10 08:57:53.547 +0000 UTC + Time Finished: 2024-12-10 09:14:04.572 +0000 UTC + Time Started: 2024-12-10 08:57:57.588 +0000 UTC + Operation Id: ocid1.coreservicesworkrequest.oc1.ap-mumbai-1.abrg6ljrxg7gov22vlcbqbnxrkl7t7xkcfya6w6gvck344jdf5vtqgw5wzgq + Operation Type: Update DB System + Percent Complete: 100 + Time Accepted: 2024-12-10 08:57:43.701 +0000 UTC + Time Finished: 2024-12-10 09:14:22.705 +0000 UTC + Time Started: 2024-12-10 08:57:53.873 +0000 UTC + Operation Id: ocid1.coreservicesworkrequest.oc1.ap-mumbai-1.abrg6ljrk2efvqjda2t7k5iaerahw7wcyz5dq2zev2k55gmq2gvsjkui7hxq + Operation Type: Create DB System + Percent Complete: 100 + Time Accepted: 2024-12-10 05:19:52.499 +0000 UTC + Time Finished: 2024-12-10 07:59:19.083 +0000 UTC + Time Started: 2024-12-10 05:19:55.747 +0000 UTC + Operation Id: ocid1.coreservicesworkrequest.oc1.ap-mumbai-1.abrg6ljr4qmf6rdtcbrc5p2q7bev3igugtpgfbwc2laht22yyjzr2srrg7vq + Operation Type: Update DB System + Percent Complete: 100 + Time Accepted: 2024-12-10 10:57:27.313 +0000 UTC + Time Finished: 2024-12-10 11:15:50.597 +0000 UTC + Time Started: 2024-12-10 10:57:45.242 +0000 UTC + Operation Id: ocid1.coreservicesworkrequest.oc1.ap-mumbai-1.abrg6ljr2vehqv3vgrxr5mrmd6hoqxg2zr6m5eaunv3ip6bcrubcpvhudmia + Operation Type: Update Shape + Percent Complete: 100 + Time Accepted: 2024-12-10 10:57:44.95 +0000 UTC + Time Finished: 2024-12-10 11:15:40.364 +0000 UTC + Time Started: 2024-12-10 10:57:54.082 +0000 UTC + Operation Id: ocid1.coreservicesworkrequest.oc1.ap-mumbai-1.abrg6ljr36bt7ot5oq3otch4bu2axn3azkicot4zuwgwmxeupxr4siisydja + Operation Type: Scale Storage + Percent Complete: 100 + Time Accepted: 2024-12-10 11:44:49.369 +0000 UTC + Time Finished: 2024-12-10 11:58:45.01 +0000 UTC + Time Started: 2024-12-10 11:44:55.544 +0000 UTC + Operation Id: ocid1.coreservicesworkrequest.oc1.ap-mumbai-1.abrg6ljrxdpmmaipuqke5yx3szyfnf2zwkfptz3jevlq3coicecfjihnm4kq + Operation Type: Scale Storage + Percent Complete: 100 + Time Accepted: 2024-12-10 11:44:55.255 +0000 UTC + Time Finished: 2024-12-10 11:58:25.229 +0000 UTC + Time Started: 2024-12-10 11:44:57.743 +0000 UTC +Events: \ No newline at end of file diff --git a/docs/dbcs/provisioning/dbcs_controller_parameters.md b/docs/dbcs/provisioning/dbcs_controller_parameters.md index 82fc3dc2..96bedf30 100644 --- a/docs/dbcs/provisioning/dbcs_controller_parameters.md +++ b/docs/dbcs/provisioning/dbcs_controller_parameters.md @@ -8,7 +8,7 @@ This page has the details of the parameters to define the specs related to an op | ociSecret | Kubernetes Secret created using PEM Key for OCI account in the prerequisites steps. | Y | String | | | | availabilityDomain | Availability Domain of the OCI region where you want to provision the DBCS System. | Y | String | | Please refer to this link: https://docs.oracle.com/en-us/iaas/Content/General/Concepts/regions.htm | | compartmentId | OCID of the OCI Compartment. | Y | String | | | -| dbAdminPaswordSecret | Kubernetes Secret created for DB Admin Account in prerequisites steps. | Y | String | | A strong password for SYS, SYSTEM, and PDB Admin. The password must be at least nine characters and contain at least two uppercase, two lowercase, two numbers, and two special characters. The special characters must be _, #, or -.| +| dbAdminPasswordSecret | Kubernetes Secret created for DB Admin Account in prerequisites steps. | Y | String | | A strong password for SYS, SYSTEM, and PDB Admin. The password must be at least nine characters and contain at least two uppercase, two lowercase, two numbers, and two special characters. The special characters must be _, #, or -.| | autoBackupEnabled | Whether to enable automatic backup or not. | N | Boolean | | True or False | | autoBackupWindow | Time window selected for initiating automatic backup for the database system. There are twelve available two-hour time windows. | N | String | | Please refer to this link: https://docs.oracle.com/en-us/iaas/api/#/en/database/20160918/datatypes/DbBackupConfig | | recoveryWindowsInDays | Number of days between the current and the earliest point of recoverability covered by automatic backups. | N | Integer | | Minimum: 1 and Maximum: 60 | diff --git a/docs/dbcs/provisioning/dbcs_service_migrate_to_kms.log b/docs/dbcs/provisioning/dbcs_service_migrate_to_kms.log new file mode 100644 index 00000000..2405a90a --- /dev/null +++ b/docs/dbcs/provisioning/dbcs_service_migrate_to_kms.log @@ -0,0 +1,132 @@ +2025-01-10T14:30:21Z INFO Updating KMS details in Existing Database {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc"} +2025-01-10T14:30:27Z INFO MigrateVaultKey request succeeded, waiting for database to reach the desired state {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc"} +2025-01-10T14:30:27Z INFO Starting to wait for the database to reach the desired state {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc", "DatabaseID": "ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyavlxjvktbs6xko5fbutwdwb3wgr2mnisovycrisj7abxq", "DesiredState": "AVAILABLE", "Timeout": "2h0m0s"} +2025-01-10T14:30:27Z INFO Database State {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc", "DatabaseID": "ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyavlxjvktbs6xko5fbutwdwb3wgr2mnisovycrisj7abxq", "CurrentState": "UPDATING"} +2025-01-10T14:30:27Z INFO Database not in the desired state yet, waiting... {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc", "DatabaseID": "ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyavlxjvktbs6xko5fbutwdwb3wgr2mnisovycrisj7abxq", "CurrentState": "UPDATING", "DesiredState": "AVAILABLE", "NextCheckIn": "1m0s"} +2025-01-10T14:31:28Z INFO Database State {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc", "DatabaseID": "ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyavlxjvktbs6xko5fbutwdwb3wgr2mnisovycrisj7abxq", "CurrentState": "UPDATING"} +2025-01-10T14:31:28Z INFO Database not in the desired state yet, waiting... {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc", "DatabaseID": "ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyavlxjvktbs6xko5fbutwdwb3wgr2mnisovycrisj7abxq", "CurrentState": "UPDATING", "DesiredState": "AVAILABLE", "NextCheckIn": "1m0s"} +2025-01-10T14:32:29Z INFO Database State {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc", "DatabaseID": "ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyavlxjvktbs6xko5fbutwdwb3wgr2mnisovycrisj7abxq", "CurrentState": "UPDATING"} +2025-01-10T14:32:29Z INFO Database not in the desired state yet, waiting... {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc", "DatabaseID": "ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyavlxjvktbs6xko5fbutwdwb3wgr2mnisovycrisj7abxq", "CurrentState": "UPDATING", "DesiredState": "AVAILABLE", "NextCheckIn": "1m0s"} +2025-01-10T14:33:30Z INFO Database State {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc", "DatabaseID": "ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyavlxjvktbs6xko5fbutwdwb3wgr2mnisovycrisj7abxq", "CurrentState": "UPDATING"} +2025-01-10T14:33:30Z INFO Database not in the desired state yet, waiting... {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc", "DatabaseID": "ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyavlxjvktbs6xko5fbutwdwb3wgr2mnisovycrisj7abxq", "CurrentState": "UPDATING", "DesiredState": "AVAILABLE", "NextCheckIn": "1m0s"} +2025-01-10T14:34:31Z INFO Database State {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc", "DatabaseID": "ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyavlxjvktbs6xko5fbutwdwb3wgr2mnisovycrisj7abxq", "CurrentState": "UPDATING"} +2025-01-10T14:34:31Z INFO Database not in the desired state yet, waiting... {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc", "DatabaseID": "ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyavlxjvktbs6xko5fbutwdwb3wgr2mnisovycrisj7abxq", "CurrentState": "UPDATING", "DesiredState": "AVAILABLE", "NextCheckIn": "1m0s"} +2025-01-10T14:35:32Z INFO Database State {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc", "DatabaseID": "ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyavlxjvktbs6xko5fbutwdwb3wgr2mnisovycrisj7abxq", "CurrentState": "UPDATING"} +2025-01-10T14:35:32Z INFO Database not in the desired state yet, waiting... {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc", "DatabaseID": "ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyavlxjvktbs6xko5fbutwdwb3wgr2mnisovycrisj7abxq", "CurrentState": "UPDATING", "DesiredState": "AVAILABLE", "NextCheckIn": "1m0s"} +2025-01-10T14:36:33Z INFO Database State {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc", "DatabaseID": "ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyavlxjvktbs6xko5fbutwdwb3wgr2mnisovycrisj7abxq", "CurrentState": "UPDATING"} +2025-01-10T14:36:33Z INFO Database not in the desired state yet, waiting... {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc", "DatabaseID": "ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyavlxjvktbs6xko5fbutwdwb3wgr2mnisovycrisj7abxq", "CurrentState": "UPDATING", "DesiredState": "AVAILABLE", "NextCheckIn": "1m0s"} +2025-01-10T14:37:34Z INFO Database State {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc", "DatabaseID": "ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyavlxjvktbs6xko5fbutwdwb3wgr2mnisovycrisj7abxq", "CurrentState": "UPDATING"} +2025-01-10T14:37:34Z INFO Database not in the desired state yet, waiting... {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc", "DatabaseID": "ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyavlxjvktbs6xko5fbutwdwb3wgr2mnisovycrisj7abxq", "CurrentState": "UPDATING", "DesiredState": "AVAILABLE", "NextCheckIn": "1m0s"} +2025-01-10T14:38:35Z INFO Database State {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc", "DatabaseID": "ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyavlxjvktbs6xko5fbutwdwb3wgr2mnisovycrisj7abxq", "CurrentState": "UPDATING"} +2025-01-10T14:38:35Z INFO Database not in the desired state yet, waiting... {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc", "DatabaseID": "ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyavlxjvktbs6xko5fbutwdwb3wgr2mnisovycrisj7abxq", "CurrentState": "UPDATING", "DesiredState": "AVAILABLE", "NextCheckIn": "1m0s"} +2025-01-10T14:30:21Z INFO Updating KMS details in Existing Database {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc"} +2025-01-10T14:30:27Z INFO MigrateVaultKey request succeeded, waiting for database to reach the desired state {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc"} +2025-01-10T14:30:27Z INFO Starting to wait for the database to reach the desired state {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc", "DatabaseID": "ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyavlxjvktbs6xko5fbutwdwb3wgr2mnisovycrisj7abxq", "DesiredState": "AVAILABLE", "Timeout": "2h0m0s"} +2025-01-10T14:30:27Z INFO Database State {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc", "DatabaseID": "ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyavlxjvktbs6xko5fbutwdwb3wgr2mnisovycrisj7abxq", "CurrentState": "UPDATING"} +2025-01-10T14:30:27Z INFO Database not in the desired state yet, waiting... {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc", "DatabaseID": "ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyavlxjvktbs6xko5fbutwdwb3wgr2mnisovycrisj7abxq", "CurrentState": "UPDATING", "DesiredState": "AVAILABLE", "NextCheckIn": "1m0s"} +2025-01-10T14:31:28Z INFO Database State {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc", "DatabaseID": "ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyavlxjvktbs6xko5fbutwdwb3wgr2mnisovycrisj7abxq", "CurrentState": "UPDATING"} +2025-01-10T14:31:28Z INFO Database not in the desired state yet, waiting... {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc", "DatabaseID": "ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyavlxjvktbs6xko5fbutwdwb3wgr2mnisovycrisj7abxq", "CurrentState": "UPDATING", "DesiredState": "AVAILABLE", "NextCheckIn": "1m0s"} +2025-01-10T14:32:29Z INFO Database State {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc", "DatabaseID": "ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyavlxjvktbs6xko5fbutwdwb3wgr2mnisovycrisj7abxq", "CurrentState": "UPDATING"} +2025-01-10T14:32:29Z INFO Database not in the desired state yet, waiting... {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc", "DatabaseID": "ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyavlxjvktbs6xko5fbutwdwb3wgr2mnisovycrisj7abxq", "CurrentState": "UPDATING", "DesiredState": "AVAILABLE", "NextCheckIn": "1m0s"} +2025-01-10T14:33:30Z INFO Database State {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc", "DatabaseID": "ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyavlxjvktbs6xko5fbutwdwb3wgr2mnisovycrisj7abxq", "CurrentState": "UPDATING"} +2025-01-10T14:33:30Z INFO Database not in the desired state yet, waiting... {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc", "DatabaseID": "ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyavlxjvktbs6xko5fbutwdwb3wgr2mnisovycrisj7abxq", "CurrentState": "UPDATING", "DesiredState": "AVAILABLE", "NextCheckIn": "1m0s"} +2025-01-10T14:34:31Z INFO Database State {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc", "DatabaseID": "ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyavlxjvktbs6xko5fbutwdwb3wgr2mnisovycrisj7abxq", "CurrentState": "UPDATING"} +2025-01-10T14:34:31Z INFO Database not in the desired state yet, waiting... {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc", "DatabaseID": "ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyavlxjvktbs6xko5fbutwdwb3wgr2mnisovycrisj7abxq", "CurrentState": "UPDATING", "DesiredState": "AVAILABLE", "NextCheckIn": "1m0s"} +2025-01-10T14:35:32Z INFO Database State {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc", "DatabaseID": "ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyavlxjvktbs6xko5fbutwdwb3wgr2mnisovycrisj7abxq", "CurrentState": "UPDATING"} +2025-01-10T14:35:32Z INFO Database not in the desired state yet, waiting... {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc", "DatabaseID": "ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyavlxjvktbs6xko5fbutwdwb3wgr2mnisovycrisj7abxq", "CurrentState": "UPDATING", "DesiredState": "AVAILABLE", "NextCheckIn": "1m0s"} +2025-01-10T14:36:33Z INFO Database State {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc", "DatabaseID": "ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyavlxjvktbs6xko5fbutwdwb3wgr2mnisovycrisj7abxq", "CurrentState": "UPDATING"} +2025-01-10T14:36:33Z INFO Database not in the desired state yet, waiting... {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc", "DatabaseID": "ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyavlxjvktbs6xko5fbutwdwb3wgr2mnisovycrisj7abxq", "CurrentState": "UPDATING", "DesiredState": "AVAILABLE", "NextCheckIn": "1m0s"} +2025-01-10T14:37:34Z INFO Database State {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc", "DatabaseID": "ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyavlxjvktbs6xko5fbutwdwb3wgr2mnisovycrisj7abxq", "CurrentState": "UPDATING"} +2025-01-10T14:37:34Z INFO Database not in the desired state yet, waiting... {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc", "DatabaseID": "ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyavlxjvktbs6xko5fbutwdwb3wgr2mnisovycrisj7abxq", "CurrentState": "UPDATING", "DesiredState": "AVAILABLE", "NextCheckIn": "1m0s"} +2025-01-10T14:38:35Z INFO Database State {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc", "DatabaseID": "ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyavlxjvktbs6xko5fbutwdwb3wgr2mnisovycrisj7abxq", "CurrentState": "UPDATING"} +2025-01-10T14:38:35Z INFO Database not in the desired state yet, waiting... {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc", "DatabaseID": "ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyavlxjvktbs6xko5fbutwdwb3wgr2mnisovycrisj7abxq", "CurrentState": "UPDATING", "DesiredState": "AVAILABLE", "NextCheckIn": "1m0s"} +2025-01-10T14:40:37Z INFO Database reached the desired state {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc", "DatabaseID": "ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyavlxjvktbs6xko5fbutwdwb3wgr2mnisovycrisj7abxq", "State": "AVAILABLE"} +2025-01-10T14:40:39Z INFO KMS migration process completed successfully {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "890fac49-f8c0-43a8-817b-fb94e96627cc"} + + +basedb/ $ kubectl describe dbcssystems.database.oracle.com/dbcssystem-existing +Name: dbcssystem-existing +Namespace: default +Labels: +Annotations: lastSuccessfulSpec: + {"dbSystem":{"compartmentId":"ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a","availabilityDomain":""... +API Version: database.oracle.com/v4 +Kind: DbcsSystem +Metadata: + Creation Timestamp: 2025-01-10T14:29:37Z + Generation: 2 + Resource Version: 130979222 + UID: f7535120-dd4a-4cbc-9e29-b9f104904773 +Spec: + Db System: + Availability Domain: + Compartment Id: ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a + Db Admin Password Secret: admin-password + Db Backup Config: + Host Name: + Kms Config: + Shape: + Subnet Id: + Tde Wallet Password Secret: tde-password + Id: ocid1.dbsystem.oc1.ap-mumbai-1.anrg6ljrabf7htyafdoaarkuhhxjfgjrzjtxpbcaycib3woadfmcz545mwua + Kms Config: + Compartment Id: ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a + Key Name: dbkey + Vault Name: dbvault + Oci Config Map: oci-cred-mumbai + Oci Secret: oci-privatekey +Status: + Availability Domain: OLou:AP-MUMBAI-1-AD-1 + Cpu Core Count: 2 + Data Storage Percentage: 80 + Data Storage Size In G Bs: 256 + Db Clone Status: + Db Db Unique Name: + Host Name: + Db Edition: ENTERPRISE_EDITION_HIGH_PERFORMANCE + Db Info: + Db Home Id: ocid1.dbhome.oc1.ap-mumbai-1.anrg6ljrqlb5nxia3s627hwjr36bix3dnh4dlbny22tzcmb2a3b4rcp74clq + Db Name: cdb12 + Db Unique Name: cdb12_hf8_bom + Db Workload: OLTP + Id: ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyavlxjvktbs6xko5fbutwdwb3wgr2mnisovycrisj7abxq + Display Name: dbsys123 + Id: ocid1.dbsystem.oc1.ap-mumbai-1.anrg6ljrabf7htyafdoaarkuhhxjfgjrzjtxpbcaycib3woadfmcz545mwua + Kms Details Status: + Compartment Id: ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a + Key Id: ocid1.key.oc1.ap-mumbai-1.fbtxxaolaaavw.abrg6ljr63rcu5h6lmaeux752pzmp334zihovh3n2acags6zt37emab34yba + Key Name: dbkey + Management Endpoint: https://fbtxxaolaaavw-management.kms.ap-mumbai-1.oraclecloud.com + Vault Id: ocid1.vault.oc1.ap-mumbai-1.fbtxxaolaaavw.abrg6ljrbjokn2fwhh36tqzyog4yjrth3mj2emxea4fxmzw6z35zlmh65p2a + Vault Name: dbvault + License Model: BRING_YOUR_OWN_LICENSE + Network: + Client Subnet: oke-nodesubnet-quick-cluster1-2bebe95db-regional + Domain Name: subdda0b5eaa.cluster1.oraclevcn.com + Host Name: host01234 + Listener Port: 1521 + Scan Dns Name: host01234-scan.subdda0b5eaa.cluster1.oraclevcn.com + Vcn Name: oke-vcn-quick-cluster1-2bebe95db + Node Count: 1 + Pdb Details Status: + Pdb Config Status: + Freeform Tags: + Created By: MAA_TEAM + TEST: test_case_provision + Pdb Name: PDB0123 + Pdb State: AVAILABLE + Pluggable Database Id: ocid1.pluggabledatabase.oc1.ap-mumbai-1.anrg6ljrabf7htyafnt7gvokjw7cvzs6xjxw5nmlz6awzycqcnf57blcuefa + Reco Storage Size In GB: 256 + Shape: VM.Standard2.2 + State: AVAILABLE + Storage Management: ASM + Subnet Id: ocid1.subnet.oc1.ap-mumbai-1.aaaaaaaa5zpzfax66omtbmjwlv4thruyru7focnu7fjcjksujmgwmr6vpbvq + Time Zone: UTC + Work Requests: + Operation Id: ocid1.coreservicesworkrequest.oc1.ap-mumbai-1.abrg6ljroc46ic555q2rfcwxg3srsbq4indueiuvj7tlziyy63uz3pvpe4ra + Operation Type: Create DB System + Percent Complete: 100 + Time Accepted: 2025-01-10 06:30:12.411 +0000 UTC + Time Finished: 2025-01-10 07:51:04.59 +0000 UTC + Time Started: 2025-01-10 06:30:20.62 +0000 UTC +Events: \ No newline at end of file diff --git a/docs/dbcs/provisioning/dbcs_service_migrate_to_kms.yaml b/docs/dbcs/provisioning/dbcs_service_migrate_to_kms.yaml new file mode 100644 index 00000000..922f7eeb --- /dev/null +++ b/docs/dbcs/provisioning/dbcs_service_migrate_to_kms.yaml @@ -0,0 +1,16 @@ +apiVersion: database.oracle.com/v4 +kind: DbcsSystem +metadata: + name: dbcssystem-existing +spec: + id: "ocid1.dbsystem.oc1.ap-mumbai-1.anrg6ljrabf7htyafdoaarkuhhxjfgjrzjtxpbcaycib3woadfmcz545mwua" + ociConfigMap: "oci-cred" + ociSecret: "oci-privatekey" + dbSystem: + compartmentId: "ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a" + dbAdminPasswordSecret: "admin-password" + tdeWalletPasswordSecret: "tde-password" + kmsConfig: + vaultName: "dbvault" + compartmentId: "ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a" + keyName: "dbkey" \ No newline at end of file diff --git a/docs/dbcs/provisioning/dbcs_service_with_2_node_rac.md b/docs/dbcs/provisioning/dbcs_service_with_2_node_rac.md index 1cfbe006..b9ce6931 100644 --- a/docs/dbcs/provisioning/dbcs_service_with_2_node_rac.md +++ b/docs/dbcs/provisioning/dbcs_service_with_2_node_rac.md @@ -1,39 +1,39 @@ -# Deploy a 2 Node RAC DB System using OCI DBCS Service +# Deploy a 2 Node RAC DB System using OCI OBDS Service -In this use case, a 2 Node RAC OCI DBCS system is deployed using Oracle DB Operator DBCS controller using all the available parameters in the .yaml file being used during the deployment. The type of the Storage Management in this case is ASM. +In this use case, a 2 Node RAC OCI OBDS system is deployed using Oracle DB Operator OBDS controller using all the available parameters in the .yaml file being used during the deployment. The type of the Storage Management in this case is ASM. **NOTE** We are assuming that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-deploy-a-dbcs-system-using-oracle-db-operator-dbcs-controller) steps to create the configmap and the secrets required during the deployment. -This example uses `dbcs_service_with_2_node_rac.yaml` to deploy a 2 Node RAC VMDB using Oracle DB Operator DBCS Controller with: +This example uses `dbcs_service_with_2_node_rac.yaml` to deploy a 2 Node RAC VMDB using Oracle DB Operator OBDS Controller with: - OCI Configmap as `oci-cred` - OCI Secret as `oci-privatekey` -- Availability Domain for the DBCS VMDB as `OLou:PHX-AD-1` - Cluster Name as `maa-cluster` -- Compartment OCID as `ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya` +- Availability Domain for the OBDS VMDB as `OLou:AP-MUMBAI-1-AD-1` +- Compartment OCID as `ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a` - Database Admin Credential as `admin-password` -- Enable flag for Automatic Backup for DBCS Database as `True` -- Auto Backup Window for DBCS Database as `SLOT_FOUR` +- Enable flag for Automatic Backup for OBDS Database as `True` +- Auto Backup Window for OBDS Database as `SLOT_FOUR` - Recovery Windows for Backup retention in days as `15` -- Oracle Database Edition as `ENTERPRISE_EDITION_EXTREME_PERFORMANCE` +- Oracle Database Edition as `STANDARD_EDITION` - Database Name as `db0130` -- Oracle Database Software Image Version as `21c` +- Oracle Database Software Image Version as `19c` - Database Workload Type as Transaction Processing i.e. `OLTP` - Redundancy of the ASM Disks as `EXTERNAL` -- Display Name for the DBCS System as `dbsystem0130` -- Database Hostname Prefix as `host0130` +- Display Name for the OBDS System as `dbsys123` +- Database Hostname Prefix as `host01234` - Initial Size of the DATA Storage in GB as `256` - License Model as `BRING_YOUR_OWN_LICENSE` -- Node count as `2` -- Name of the PDB to be created as `PDB0130` +- Name of the PDB to be created as `PDB0123` - Private IP explicitly assigned to be `10.0.1.99` -- Oracle VMDB Shape as `VM.Standard2.2` -- SSH Public key for the DBCS system being deployed as `oci-publickey` +- Node count as `2` +- Oracle VMDB Shape as `VM.Standard2.1` +- SSH Public key for the OBDS system being deployed as `oci-publickey` - Storage Management type as `ASM` -- OCID of the Subnet as `ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a` -- Tag the DBCS system with two key value pairs as `"TEST": "test_case_provision"` and `"CreatedBy": "MAA_TEAM"` +- OCID of the Subnet as `ocid1.subnet.oc1.ap-mumbai-1.aaaaaaaa5zpzfax66omtbmjwlv4thruyru7focnu7fjcjksujmgwmr6vpbv` +- Tag the OBDS system with two key value pairs as `"TEST": "test_case_provision"` and `"CreatedBy": "MAA_TEAM"` - TDE Wallet Secret as `tde-password` -- Time Zone for the DBCS System as `Europe/Berlin` +- Time Zone for the OBDS System as `Europe/Berlin` **NOTE:** For the details of the parameters to be used in the .yaml file, please refer [here](./dbcs_controller_parameters.md). @@ -46,14 +46,14 @@ Use the file: [dbcs_service_with_all_parameters_asm.yaml](./dbcs_service_with_2_ dbcssystem.database.oracle.com/dbcssystem-create configured ``` -2. Monitor the Oracle DB Operator Pod `pod/oracle-database-operator-controller-manager-665874bd57-g2cgw` for the progress of the DBCS VMDB deployment. +2. Monitor the Oracle DB Operator Pod `pod/oracle-database-operator-controller-manager-665874bd57-g2cgw` for the progress of the OBDS VMDB deployment. NOTE: Check the DB Operator Pod name in your environment. ``` -[root@docker-test-server DBCS]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-g2cgw -n oracle-database-operator-system +[root@docker-test-server OBDS]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-g2cgw -n oracle-database-operator-system ``` ## Sample Output -[Here](./dbcs_service_with_2_node_rac_sample_output.log) is the sample output for a 2 Node RAC DBCS System deployed in OCI using Oracle DB Operator DBCS Controller with all parameters and with Storage Management as ASM. +[Here](./dbcs_service_with_2_node_rac_sample_output.log) is the sample output for a 2 Node RAC OBDS System deployed in OCI using Oracle DB Operator OBDS Controller with all parameters and with Storage Management as ASM. diff --git a/docs/dbcs/provisioning/dbcs_service_with_2_node_rac.yaml b/docs/dbcs/provisioning/dbcs_service_with_2_node_rac.yaml index 168cb427..3b4f35e3 100644 --- a/docs/dbcs/provisioning/dbcs_service_with_2_node_rac.yaml +++ b/docs/dbcs/provisioning/dbcs_service_with_2_node_rac.yaml @@ -1,4 +1,4 @@ -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: DbcsSystem metadata: name: dbcssystem-create @@ -6,21 +6,22 @@ spec: ociConfigMap: "oci-cred" ociSecret: "oci-privatekey" dbSystem: - availabilityDomain: "OLou:PHX-AD-1" + availabilityDomain: "OLou:AP-MUMBAI-1-AD-1" clusterName: "maa-cluster" - compartmentId: "ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya" - dbAdminPaswordSecret: "admin-password" + compartmentId: "ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a" + dbAdminPasswordSecret: "admin-password" dbBackupConfig: autoBackupEnabled: True autoBackupWindow: "SLOT_FOUR" recoveryWindowsInDays: 15 - dbEdition: "ENTERPRISE_EDITION_EXTREME_PERFORMANCE" - dbName: "db0130" - dbVersion: "21c" + dbEdition: "ENTERPRISE_EDITION_HIGH_PERFORMANCE" + dbName: "cdb12" + displayName: "dbsys123" + licenseModel: "BRING_YOUR_OWN_LICENSE" + dbVersion: "19c" dbWorkload: "OLTP" diskRedundancy: "EXTERNAL" - displayName: "dbsystem0130" - hostName: "host0130" + hostName: "host01234" initialDataStorageSizeInGB: 256 licenseModel: "BRING_YOUR_OWN_LICENSE" nodeCount: 2 @@ -30,7 +31,7 @@ spec: sshPublicKeys: - "oci-publickey" storageManagement: "ASM" - subnetId: "ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a" + subnetId: "ocid1.subnet.oc1.ap-mumbai-1.aaaaaaaa5zpzfax66omtbmjwlv4thruyru7focnu7fjcjksujmgwmr6vpbvq" tags: "TEST": "test_case_provision" "CreatedBy": "MAA_TEAM" diff --git a/docs/dbcs/provisioning/dbcs_service_with_all_parameters_asm.md b/docs/dbcs/provisioning/dbcs_service_with_all_parameters_asm.md index 5ef94c30..7bd9abea 100644 --- a/docs/dbcs/provisioning/dbcs_service_with_all_parameters_asm.md +++ b/docs/dbcs/provisioning/dbcs_service_with_all_parameters_asm.md @@ -1,38 +1,37 @@ -# Create DBCS with All Parameters with Storage Management as ASM +# Create OBDS with All Parameters with Storage Management as ASM -In this use case, the an OCI DBCS system is deployed using Oracle DB Operator DBCS controller using all the available parameters in the .yaml file being used during the deployment. The type of the Storage Management in this case is ASM. +In this use case, the an OCI OBDS system is deployed using Oracle DB Operator OBDS controller using all the available parameters in the .yaml file being used during the deployment. The type of the Storage Management in this case is ASM. **NOTE** We are assuming that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-deploy-a-dbcs-system-using-oracle-db-operator-dbcs-controller) steps to create the configmap and the secrets required during the deployment. -This example uses `dbcs_service_with_all_parameters_asm.yaml` to deploy a Single Instance DBCS VMDB using Oracle DB Operator DBCS Controller with: +This example uses `dbcs_service_with_all_parameters_asm.yaml` to deploy a Single Instance OBDS VMDB using Oracle DB Operator OBDS Controller with: - OCI Configmap as `oci-cred` - OCI Secret as `oci-privatekey` -- Availability Domain for the DBCS VMDB as `OLou:PHX-AD-1` -- Cluster Name as `maa-cluster` -- Compartment OCID as `ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya` +- Availability Domain for the OBDS VMDB as `OLou:AP-MUMBAI-1-AD-1` +- Compartment OCID as `ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a` - Database Admin Credential as `admin-password` -- Enable flag for Automatic Backup for DBCS Database as `True` -- Auto Backup Window for DBCS Database as `SLOT_FOUR` +- Enable flag for Automatic Backup for OBDS Database as `True` +- Auto Backup Window for OBDS Database as `SLOT_FOUR` - Recovery Windows for Backup retention in days as `15` - Oracle Database Edition as `STANDARD_EDITION` - Database Name as `db0130` -- Oracle Database Software Image Version as `21c` +- Oracle Database Software Image Version as `19c` - Database Workload Type as Transaction Processing i.e. `OLTP` - Redundancy of the ASM Disks as `EXTERNAL` -- Display Name for the DBCS System as `dbsystem0130` -- Database Hostname Prefix as `host0130` +- Display Name for the OBDS System as `dbsys123` +- Database Hostname Prefix as `host01234` - Initial Size of the DATA Storage in GB as `256` - License Model as `BRING_YOUR_OWN_LICENSE` -- Name of the PDB to be created as `PDB0130` +- Name of the PDB to be created as `PDB0123` - Private IP explicitly assigned to be `10.0.1.99` -- Oracle VMDB Shape as `VM.Standard2.1` -- SSH Public key for the DBCS system being deployed as `oci-publickey` +- Oracle VMDB Shape as `VM.Standard2.1` +- SSH Public key for the OBDS system being deployed as `oci-publickey` - Storage Management type as `ASM` -- OCID of the Subnet as `ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a` -- Tag the DBCS system with two key value pairs as `"TEST": "test_case_provision"` and `"CreatedBy": "MAA_TEAM"` +- OCID of the Subnet as `ocid1.subnet.oc1.ap-mumbai-1.aaaaaaaa5zpzfax66omtbmjwlv4thruyru7focnu7fjcjksujmgwmr6vpbv` +- Tag the OBDS system with two key value pairs as `"TEST": "test_case_provision"` and `"CreatedBy": "MAA_TEAM"` - TDE Wallet Secret as `tde-password` -- Time Zone for the DBCS System as `Europe/Berlin` +- Time Zone for the OBDS System as `Europe/Berlin` **NOTE:** For the details of the parameters to be used in the .yaml file, please refer [here](./dbcs_controller_parameters.md). @@ -41,18 +40,18 @@ Use the file: [dbcs_service_with_all_parameters_asm.yaml](./dbcs_service_with_al 1. Deploy the .yaml file: ```sh -[root@docker-test-server DBCS]# kubectl apply -f dbcs_service_with_all_parameters_asm.yaml +[root@docker-test-server OBDS]# kubectl apply -f dbcs_service_with_all_parameters_asm.yaml dbcssystem.database.oracle.com/dbcssystem-create created ``` -2. Monitor the Oracle DB Operator Pod `pod/oracle-database-operator-controller-manager-665874bd57-g2cgw` for the progress of the DBCS VMDB deployment. +2. Monitor the Oracle DB Operator Pod `pod/oracle-database-operator-controller-manager-665874bd57-g2cgw` for the progress of the OBDS VMDB deployment. NOTE: Check the DB Operator Pod name in your environment. ``` -[root@docker-test-server DBCS]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-g2cgw -n oracle-database-operator-system +[root@docker-test-server OBDS]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-g2cgw -n oracle-database-operator-system ``` ## Sample Output -[Here](./dbcs_service_with_all_parameters_asm_sample_output.log) is the sample output for a DBCS System deployed in OCI using Oracle DB Operator DBCS Controller with all parameters and with Storage Management as ASM. +[Here](./dbcs_service_with_all_parameters_asm_sample_output.log) is the sample output for a OBDS System deployed in OCI using Oracle DB Operator OBDS Controller with all parameters and with Storage Management as ASM. diff --git a/docs/dbcs/provisioning/dbcs_service_with_all_parameters_asm.yaml b/docs/dbcs/provisioning/dbcs_service_with_all_parameters_asm.yaml index 1dcec54c..34811df7 100644 --- a/docs/dbcs/provisioning/dbcs_service_with_all_parameters_asm.yaml +++ b/docs/dbcs/provisioning/dbcs_service_with_all_parameters_asm.yaml @@ -1,4 +1,4 @@ -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: DbcsSystem metadata: name: dbcssystem-create @@ -6,31 +6,32 @@ spec: ociConfigMap: "oci-cred" ociSecret: "oci-privatekey" dbSystem: - availabilityDomain: "OLou:PHX-AD-1" - compartmentId: "ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya" - dbAdminPaswordSecret: "admin-password" + availabilityDomain: "OLou:AP-MUMBAI-1-AD-1" + clusterName: "maa-cluster" + compartmentId: "ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a" + dbAdminPasswordSecret: "admin-password" dbBackupConfig: autoBackupEnabled: True autoBackupWindow: "SLOT_FOUR" recoveryWindowsInDays: 15 - dbEdition: "STANDARD_EDITION" - dbName: "db0130" - dbVersion: "21c" + dbEdition: "ENTERPRISE_EDITION_HIGH_PERFORMANCE" + dbName: "cdb12" + displayName: "dbsys123" + licenseModel: "BRING_YOUR_OWN_LICENSE" + dbVersion: "19c" dbWorkload: "OLTP" diskRedundancy: "EXTERNAL" - displayName: "dbsystem0130" - hostName: "host0130" + hostName: "host01234" initialDataStorageSizeInGB: 256 - licenseModel: "BRING_YOUR_OWN_LICENSE" - pdbName: "PDB0130" + pdbName: "PDB0123" privateIp: "10.0.1.99" - shape: "VM.Standard2.1" + shape: "VM.Standard2.2" sshPublicKeys: - "oci-publickey" storageManagement: "ASM" - subnetId: "ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a" + subnetId: "ocid1.subnet.oc1.ap-mumbai-1.aaaaaaaa5zpzfax66omtbmjwlv4thruyru7focnu7fjcjksujmgwmr6vpbvq" tags: "TEST": "test_case_provision" "CreatedBy": "MAA_TEAM" tdeWalletPasswordSecret: "tde-password" - timeZone: "Europe/Berlin" + timeZone: "Europe/Berlin" \ No newline at end of file diff --git a/docs/dbcs/provisioning/dbcs_service_with_all_parameters_asm_sample_output.log b/docs/dbcs/provisioning/dbcs_service_with_all_parameters_asm_sample_output.log index a2fc6690..eec26016 100644 --- a/docs/dbcs/provisioning/dbcs_service_with_all_parameters_asm_sample_output.log +++ b/docs/dbcs/provisioning/dbcs_service_with_all_parameters_asm_sample_output.log @@ -1,35 +1,36 @@ [root@docker-test-server test]# cat dbcs_service_with_all_parameters_asm.yaml -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: DbcsSystem metadata: name: dbcssystem-create spec: - ociConfigMap: "oci-cred" + ociConfigMap: "oci-cred-mumbai" ociSecret: "oci-privatekey" dbSystem: - availabilityDomain: "OLou:PHX-AD-1" - compartmentId: "ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya" + availabilityDomain: "OLou:AP-MUMBAI-1-AD-1" + compartmentId: "ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a" dbAdminPaswordSecret: "admin-password" dbBackupConfig: autoBackupEnabled: True autoBackupWindow: "SLOT_FOUR" recoveryWindowsInDays: 15 - dbEdition: "STANDARD_EDITION" - dbName: "db0130" - dbVersion: "21c" + dbEdition: "ENTERPRISE_EDITION_HIGH_PERFORMANCE" + dbName: "cdb12" + displayName: "dbsys123" + licenseModel: "BRING_YOUR_OWN_LICENSE" + dbVersion: "19c" dbWorkload: "OLTP" diskRedundancy: "EXTERNAL" - displayName: "dbsystem0130" - hostName: "host0130" + hostName: "host01234" initialDataStorageSizeInGB: 256 licenseModel: "BRING_YOUR_OWN_LICENSE" - pdbName: "PDB0130" + pdbName: "PDB0123" privateIp: "10.0.1.99" - shape: "VM.Standard2.1" + shape: "VM.Standard.E5.Flex" sshPublicKeys: - "oci-publickey" storageManagement: "ASM" - subnetId: "ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a" + subnetId: "ocid1.subnet.oc1.ap-mumbai-1.aaaaaaaa5zpzfax66omtbmjwlv4thruyru7focnu7fjcjksujmgwmr6vpbvq" tags: "TEST": "test_case_provision" "CreatedBy": "MAA_TEAM" @@ -41,9 +42,6 @@ spec: dbcssystem.database.oracle.com/dbcssystem-create created - - - [root@docker-test-server test]# kubectl get ns kubectl get allNAME STATUS AGE @@ -145,169 +143,100 @@ replicaset.apps/oracle-database-operator-controller-manager-665874bd57 3 [root@docker-test-server test]# kubectl describe dbcssystems.database.oracle.com dbcssystem-create +user/ $ k describe dbcssystems/dbcssystem-create Name: dbcssystem-create Namespace: default Labels: Annotations: lastSuccessfulSpec: - {"dbSystem":{"compartmentId":"ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya","availabilityDomain":"O... -API Version: database.oracle.com/v1alpha1 + {"dbSystem":{"compartmentId":"ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a","availabilityDomain":"O... +API Version: database.oracle.com/v4 Kind: DbcsSystem Metadata: - Creation Timestamp: 2022-03-09T02:59:43Z - Generation: 1 - Managed Fields: - API Version: database.oracle.com/v1alpha1 - Fields Type: FieldsV1 - fieldsV1: - f:metadata: - f:annotations: - .: - f:kubectl.kubernetes.io/last-applied-configuration: - f:spec: - .: - f:dbSystem: - .: - f:availabilityDomain: - f:compartmentId: - f:dbAdminPaswordSecret: - f:dbBackupConfig: - .: - f:autoBackupEnabled: - f:autoBackupWindow: - f:recoveryWindowsInDays: - f:dbEdition: - f:dbName: - f:dbVersion: - f:dbWorkload: - f:diskRedundancy: - f:displayName: - f:hostName: - f:initialDataStorageSizeInGB: - f:licenseModel: - f:pdbName: - f:privateIp: - f:shape: - f:sshPublicKeys: - f:storageManagement: - f:subnetId: - f:tags: - .: - f:CreatedBy: - f:TEST: - f:tdeWalletPasswordSecret: - f:timeZone: - f:ociConfigMap: - f:ociSecret: - Manager: kubectl-client-side-apply - Operation: Update - Time: 2022-03-09T02:59:43Z - API Version: database.oracle.com/v1alpha1 - Fields Type: FieldsV1 - fieldsV1: - f:metadata: - f:annotations: - f:lastSuccessfulSpec: - f:spec: - f:id: - f:status: - .: - f:availabilityDomain: - f:cpuCoreCount: - f:dataStoragePercentage: - f:dataStorageSizeInGBs: - f:dbEdition: - f:dbInfo: - f:displayName: - f:id: - f:licenseModel: - f:network: - .: - f:clientSubnet: - f:domainName: - f:hostName: - f:listenerPort: - f:scanDnsName: - f:vcnName: - f:nodeCount: - f:recoStorageSizeInGB: - f:shape: - f:state: - f:storageManagement: - f:subnetId: - f:timeZone: - f:workRequests: - Manager: manager - Operation: Update - Time: 2022-03-09T03:59:22Z - Resource Version: 55276756 - UID: e7d874e7-3cd7-4b8b-8cd1-32d68795a38c + Creation Timestamp: 2024-12-09T09:42:08Z + Generation: 2 + Resource Version: 117337682 + UID: cc31eb51-56bc-48f5-926b-2453710b1592 Spec: Db System: - Availability Domain: OLou:PHX-AD-1 - Compartment Id: ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya + Availability Domain: OLou:AP-MUMBAI-1-AD-1 + Compartment Id: ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a Db Admin Pasword Secret: admin-password Db Backup Config: Auto Backup Enabled: true Auto Backup Window: SLOT_FOUR Recovery Windows In Days: 15 - Db Edition: STANDARD_EDITION - Db Name: db0130 - Db Version: 21c + Db Edition: ENTERPRISE_EDITION_HIGH_PERFORMANCE + Db Name: cdb12 + Db Version: 19c Db Workload: OLTP Disk Redundancy: EXTERNAL - Display Name: dbsystem0130 - Host Name: host0130 + Display Name: dbsys123 + Host Name: host01234 Initial Data Storage Size In GB: 256 - License Model: BRING_YOUR_OWN_LICENSE - Pdb Name: PDB0130 - Private Ip: 10.0.1.99 - Shape: VM.Standard2.1 + Kms Config: + License Model: BRING_YOUR_OWN_LICENSE + Pdb Name: PDB0123 + Private Ip: 10.0.1.99 + Shape: VM.Standard.E5.Flex Ssh Public Keys: oci-publickey Storage Management: ASM - Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a + Subnet Id: ocid1.subnet.oc1.ap-mumbai-1.aaaaaaaa5zpzfax66omtbmjwlv4thruyru7focnu7fjcjksujmgwmr6vpbvq Tags: Created By: MAA_TEAM TEST: test_case_provision Tde Wallet Password Secret: tde-password Time Zone: Europe/Berlin - Oci Config Map: oci-cred - Oci Secret: oci-privatekey + Id: ocid1.dbsystem.oc1.ap-mumbai-1.anrg6ljrabf7htyap33onviyojoimevpazf4wtbnfsi5v5izah2s365wmyka + Kms Config: + Oci Config Map: oci-cred-mumbai + Oci Secret: oci-privatekey Status: - Availability Domain: OLou:PHX-AD-1 - Cpu Core Count: 1 + Availability Domain: OLou:AP-MUMBAI-1-AD-1 + Cpu Core Count: 2 Data Storage Percentage: 80 Data Storage Size In G Bs: 256 - Db Edition: STANDARD_EDITION + Db Clone Status: + Db Db Unique Name: + Host Name: + Db Edition: ENTERPRISE_EDITION_HIGH_PERFORMANCE Db Info: - Db Home Id: ocid1.dbhome.oc1.phx.anyhqljr5gy3jhqaubltt77vlwmsx7w5d5dvq6be7isglwbpqijfi5gflh5a - Db Name: db0130 - Db Unique Name: db0130_phx1sw + Db Home Id: ocid1.dbhome.oc1.ap-mumbai-1.anrg6ljrqlb5nxiav4nm27oy6tfbqqyukvcgba7nalyozrgwfvkt5f25fazq + Db Name: cdb12 + Db Unique Name: cdb12_z4b_bom Db Workload: OLTP - Id: ocid1.database.oc1.phx.anyhqljrabf7htya5bzvoxrrc2qu6yjw6c27hcsx32bp7c76vzy35kesa2nq - Display Name: dbsystem0130 - Id: ocid1.dbsystem.oc1.phx.anyhqljrabf7htyaz42sxinatef6xieeppxmwg3bwlw5chpefc52s4joraxq - License Model: BRING_YOUR_OWN_LICENSE + Id: ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyanuajzgrh6u5qtvlui4e7jtfwbcnx7lcplw36dy4u4fza + Display Name: dbsys123 + Id: ocid1.dbsystem.oc1.ap-mumbai-1.anrg6ljrabf7htyap33onviyojoimevpazf4wtbnfsi5v5izah2s365wmyka + Kms Details Status: + License Model: BRING_YOUR_OWN_LICENSE Network: - Client Subnet: k8test-pubvcn - Domain Name: k8testpubvcn.k8test.oraclevcn.com - Host Name: host0130 - Listener Port: 1521 - Scan Dns Name: host0130-scan.k8testpubvcn.k8test.oraclevcn.com - Vcn Name: k8test - Node Count: 1 - Reco Storage Size In GB: 256 - Shape: VM.Standard2.1 - State: AVAILABLE - Storage Management: ASM - Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a - Time Zone: Europe/Berlin + Client Subnet: oke-nodesubnet-quick-cluster1-2bebe95db-regional + Domain Name: subdda0b5eaa.cluster1.oraclevcn.com + Host Name: host01234 + Listener Port: 1521 + Scan Dns Name: host01234-scan.subdda0b5eaa.cluster1.oraclevcn.com + Vcn Name: oke-vcn-quick-cluster1-2bebe95db + Node Count: 1 + Pdb Details Status: + Pdb Config Status: + Freeform Tags: + Created By: MAA_TEAM + TEST: test_case_provision + Pdb Name: PDB0123 + Pdb State: AVAILABLE + Pluggable Database Id: ocid1.pluggabledatabase.oc1.ap-mumbai-1.anrg6ljrabf7htya6wvrskrlk2fazy4pa25jcbinks7vsjdv4kxf5t6nxcxq + Reco Storage Size In GB: 256 + Shape: VM.Standard.E5.Flex + State: AVAILABLE + Storage Management: ASM + Subnet Id: ocid1.subnet.oc1.ap-mumbai-1.aaaaaaaa5zpzfax66omtbmjwlv4thruyru7focnu7fjcjksujmgwmr6vpbvq + Time Zone: UTC Work Requests: - Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrlpxe723pq3z5fkeyfgbu4ewsysjcdrxiyxigponwosy44uhcpcsq + Operation Id: ocid1.coreservicesworkrequest.oc1.ap-mumbai-1.abrg6ljrq63rk37tfqyu64lwason4rczllxmd5nk5iovdzbqkkk2d4nwp5ka Operation Type: Create DB System Percent Complete: 100 - Time Accepted: 2022-03-09 02:59:48.969 +0000 UTC - Time Finished: 2022-03-09 03:56:52.77 +0000 UTC - Time Started: 2022-03-09 02:59:56.287 +0000 UTC -Events: -[root@docker-test-server test]# \ No newline at end of file + Time Accepted: 2024-12-09 09:42:14.521 +0000 UTC + Time Finished: 2024-12-09 10:32:30.77 +0000 UTC + Time Started: 2024-12-09 09:42:21.084 +0000 UTC +Events: \ No newline at end of file diff --git a/docs/dbcs/provisioning/dbcs_service_with_all_parameters_lvm.md b/docs/dbcs/provisioning/dbcs_service_with_all_parameters_lvm.md index d6beeb16..1bc560f4 100644 --- a/docs/dbcs/provisioning/dbcs_service_with_all_parameters_lvm.md +++ b/docs/dbcs/provisioning/dbcs_service_with_all_parameters_lvm.md @@ -1,35 +1,37 @@ -# Create DBCS with All Parameters with Storage Management as LVM +# Create OBDS with All Parameters with Storage Management as LVM -In this use case, the an OCI DBCS system is deployed using Oracle DB Operator DBCS controller using all the available parameters in the .yaml file being used during the deployment. +In this use case, the an OCI OBDS system is deployed using Oracle DB Operator OBDS controller using all the available parameters in the .yaml file being used during the deployment. **NOTE** We are assuming that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-deploy-a-dbcs-system-using-oracle-db-operator-dbcs-controller) steps to create the configmap and the secrets required during the deployment. -This example uses `dbcs_service_with_all_parameters_lvm.yaml` to deploy a Single Instance DBCS VMDB using Oracle DB Operator DBCS Controller with: +This example uses `dbcs_service_with_all_parameters_lvm.yaml` to deploy a Single Instance OBDS VMDB using Oracle DB Operator OBDS Controller with: - OCI Configmap as `oci-cred` - OCI Secret as `oci-privatekey` -- Availability Domain for the DBCS VMDB as `OLou:PHX-AD-1` -- Compartment OCID as `ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya` +- Availability Domain for the OBDS VMDB as `OLou:AP-MUMBAI-1-AD-1` +- Compartment OCID as `ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a` - Database Admin Credential as `admin-password` -- Enable flag for Automatic Backup for DBCS Database as `True` -- Auto Backup Window for DBCS Database as `SLOT_FOUR` +- Enable flag for Automatic Backup for OBDS Database as `True` +- Auto Backup Window for OBDS Database as `SLOT_FOUR` - Recovery Windows for Backup retention in days as `15` - Oracle Database Edition as `STANDARD_EDITION` - Database Name as `db0130` -- Oracle Database Software Image Version as `21c` +- Oracle Database Software Image Version as `19c` - Database Workload Type as Transaction Processing i.e. `OLTP` -- Display Name for the DBCS System as `dbsystem0130` -- Database Hostname Prefix as `host0130` +- Redundancy of the ASM Disks as `EXTERNAL` +- Display Name for the OBDS System as `dbsys123` +- Database Hostname Prefix as `host01234` - Initial Size of the DATA Storage in GB as `256` - License Model as `BRING_YOUR_OWN_LICENSE` -- Name of the PDB to be created as `PDB0130` -- Oracle VMDB Shape as `VM.Standard2.1` -- SSH Public key for the DBCS system being deployed as `oci-publickey` +- Name of the PDB to be created as `PDB0123` +- Private IP explicitly assigned to be `10.0.1.99` +- Oracle VMDB Shape as `VM.Standard2.1` +- SSH Public key for the OBDS system being deployed as `oci-publickey` - Storage Management type as `LVM` -- OCID of the Subnet as `ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a` -- Tag the DBCS system with two key value pairs as `"TEST": "test_case_provision"` and `"CreatedBy": "MAA_TEAM"` +- OCID of the Subnet as `ocid1.subnet.oc1.ap-mumbai-1.aaaaaaaa5zpzfax66omtbmjwlv4thruyru7focnu7fjcjksujmgwmr6vpbv` +- Tag the OBDS system with two key value pairs as `"TEST": "test_case_provision"` and `"CreatedBy": "MAA_TEAM"` - TDE Wallet Secret as `tde-password` -- Time Zone for the DBCS System as `Europe/Berlin` +- Time Zone for the OBDS System as `Europe/Berlin` **NOTE:** For the details of the parameters to be used in the .yaml file, please refer [here](./dbcs_controller_parameters.md). @@ -38,18 +40,18 @@ Use the file: [dbcs_service_with_all_parameters_lvm.yaml](./dbcs_service_with_al 1. Deploy the .yaml file: ```sh -[root@docker-test-server DBCS]# kubectl apply -f dbcs_service_with_all_parameters_lvm.yaml +[root@docker-test-server OBDS]# kubectl apply -f dbcs_service_with_all_parameters_lvm.yaml dbcssystem.database.oracle.com/dbcssystem-create created ``` -2. Monitor the Oracle DB Operator Pod `pod/oracle-database-operator-controller-manager-665874bd57-g2cgw` for the progress of the DBCS VMDB deployment. +2. Monitor the Oracle DB Operator Pod `pod/oracle-database-operator-controller-manager-665874bd57-g2cgw` for the progress of the OBDS VMDB deployment. NOTE: Check the DB Operator Pod name in your environment. ``` -[root@docker-test-server DBCS]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-g2cgw -n oracle-database-operator-system +[root@docker-test-server OBDS]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-g2cgw -n oracle-database-operator-system ``` ## Sample Output -[Here](./dbcs_service_with_all_parameters_lvm_sample_output.log) is the sample output for a DBCS System deployed in OCI using Oracle DB Operator DBCS Controller with all parameters and with Storage Management as LVM. +[Here](./dbcs_service_with_all_parameters_lvm_sample_output.log) is the sample output for a OBDS System deployed in OCI using Oracle DB Operator OBDS Controller with all parameters and with Storage Management as LVM. diff --git a/docs/dbcs/provisioning/dbcs_service_with_all_parameters_lvm.yaml b/docs/dbcs/provisioning/dbcs_service_with_all_parameters_lvm.yaml index 73208317..f76962d1 100644 --- a/docs/dbcs/provisioning/dbcs_service_with_all_parameters_lvm.yaml +++ b/docs/dbcs/provisioning/dbcs_service_with_all_parameters_lvm.yaml @@ -1,4 +1,4 @@ -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: DbcsSystem metadata: name: dbcssystem-create @@ -6,27 +6,30 @@ spec: ociConfigMap: "oci-cred" ociSecret: "oci-privatekey" dbSystem: - availabilityDomain: "OLou:PHX-AD-1" - compartmentId: "ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya" - dbAdminPaswordSecret: "admin-password" + availabilityDomain: "OLou:AP-MUMBAI-1-AD-1" + compartmentId: "ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a" + dbAdminPasswordSecret: "admin-password" dbBackupConfig: autoBackupEnabled: True autoBackupWindow: "SLOT_FOUR" recoveryWindowsInDays: 15 - dbEdition: "STANDARD_EDITION" - dbName: "db0130" - dbVersion: "21c" + dbEdition: "ENTERPRISE_EDITION_HIGH_PERFORMANCE" + dbName: "cdb12" + displayName: "dbsys123" + licenseModel: "BRING_YOUR_OWN_LICENSE" + dbVersion: "19c" dbWorkload: "OLTP" - displayName: "dbsystem0130" - hostName: "host0130" + diskRedundancy: "EXTERNAL" + hostName: "host01234" initialDataStorageSizeInGB: 256 licenseModel: "BRING_YOUR_OWN_LICENSE" - pdbName: "PDB0130" - shape: "VM.Standard2.1" + pdbName: "PDB0123" + privateIp: "10.0.1.99" + shape: "VM.Standard.E5.Flex" sshPublicKeys: - "oci-publickey" storageManagement: "LVM" - subnetId: "ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a" + subnetId: "ocid1.subnet.oc1.ap-mumbai-1.aaaaaaaa5zpzfax66omtbmjwlv4thruyru7focnu7fjcjksujmgwmr6vpbvq" tags: "TEST": "test_case_provision" "CreatedBy": "MAA_TEAM" diff --git a/docs/dbcs/provisioning/dbcs_service_with_kms.yaml b/docs/dbcs/provisioning/dbcs_service_with_kms.yaml new file mode 100644 index 00000000..691b17a1 --- /dev/null +++ b/docs/dbcs/provisioning/dbcs_service_with_kms.yaml @@ -0,0 +1,27 @@ +apiVersion: database.oracle.com/v4 +kind: DbcsSystem +metadata: + name: dbcssystem-create +spec: + ociConfigMap: "oci-cred" + ociSecret: "oci-privatekey" + dbSystem: + availabilityDomain: "OLou:AP-MUMBAI-1-AD-1" + compartmentId: "ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a" + dbAdminPasswordSecret: "admin-password" + dbEdition: "ENTERPRISE_EDITION_HIGH_PERFORMANCE" + dbName: "kmsdb" + displayName: "kmsdbsystem" + licenseModel: "BRING_YOUR_OWN_LICENSE" + dbVersion: "19c" + dbWorkload: "OLTP" + hostName: "kmshost" + shape: "VM.Standard2.2" + domain: "subdda0b5eaa.cluster1.oraclevcn.com" + sshPublicKeys: + - "oci-publickey" + subnetId: "ocid1.subnet.oc1.ap-mumbai-1.aaaaaaaa5zpzfax66omtbmjwlv4thruyru7focnu7fjcjksujmgwmr6vpbvq" + kmsConfig: + vaultName: "dbvault" + compartmentId: "ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a" + keyName: "dbkey" \ No newline at end of file diff --git a/docs/dbcs/provisioning/dbcs_service_with_kms_sample_output.log b/docs/dbcs/provisioning/dbcs_service_with_kms_sample_output.log new file mode 100644 index 00000000..7ddf7d2f --- /dev/null +++ b/docs/dbcs/provisioning/dbcs_service_with_kms_sample_output.log @@ -0,0 +1,91 @@ +kubectl describe dbcssystems.database.oracle.com/dbcssystem-create +Name: dbcssystem-create +Namespace: default +Labels: +Annotations: lastSuccessfulSpec: + {"dbSystem":{"compartmentId":"ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a","availabilityDomain":"O... +API Version: database.oracle.com/v4 +Kind: DbcsSystem +Metadata: + Creation Timestamp: 2025-01-09T18:10:30Z + Generation: 2 + Resource Version: 130640272 + UID: 85e39113-0a02-4cf6-84d8-2270c543b0bf +Spec: + Db System: + Availability Domain: OLou:AP-MUMBAI-1-AD-1 + Compartment Id: ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a + Db Admin Password Secret: admin-password + Db Backup Config: + Db Edition: ENTERPRISE_EDITION_HIGH_PERFORMANCE + Db Name: kmsdb + Db Version: 19c + Db Workload: OLTP + Display Name: kmsdbsystem + Domain: subdda0b5eaa.cluster1.oraclevcn.com + Host Name: kmshost + Kms Config: + License Model: BRING_YOUR_OWN_LICENSE + Shape: VM.Standard2.2 + Ssh Public Keys: + oci-publickey + Subnet Id: ocid1.subnet.oc1.ap-mumbai-1.aaaaaaaa5zpzfax66omtbmjwlv4thruyru7focnu7fjcjksujmgwmr6vpbvq + Id: ocid1.dbsystem.oc1.ap-mumbai-1.anrg6ljrabf7htyai52ll4ifn52jcwdwpvv2exqqfa2wptypvi46wibx5sea + Kms Config: + Compartment Id: ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a + Key Name: dbkey + Vault Name: dbvault + Oci Config Map: oci-cred-mumbai + Oci Secret: oci-privatekey +Status: + Availability Domain: OLou:AP-MUMBAI-1-AD-1 + Cpu Core Count: 2 + Data Storage Percentage: 80 + Data Storage Size In G Bs: 256 + Db Clone Status: + Db Db Unique Name: + Host Name: + Db Edition: ENTERPRISE_EDITION_HIGH_PERFORMANCE + Db Info: + Db Home Id: ocid1.dbhome.oc1.ap-mumbai-1.anrg6ljrqlb5nxiasjpqkykdchfksgas4k62cqsf6p5gkvubsj53fdokovnq + Db Name: kmsdb + Db Unique Name: kmsdb_7cb_bom + Db Workload: OLTP + Id: ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyarhrmiqzf4pari5sshhdglj6bpuijy3fupxvveblr2l6q + Display Name: kmsdbsystem + Id: ocid1.dbsystem.oc1.ap-mumbai-1.anrg6ljrabf7htyai52ll4ifn52jcwdwpvv2exqqfa2wptypvi46wibx5sea + Kms Details Status: + Compartment Id: ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a + Key Id: ocid1.key.oc1.ap-mumbai-1.fbtxxaolaaavw.abrg6ljr63rcu5h6lmaeux752pzmp334zihovh3n2acags6zt37emab34yba + Key Name: dbkey + Management Endpoint: https://fbtxxaolaaavw-management.kms.ap-mumbai-1.oraclecloud.com + Vault Id: ocid1.vault.oc1.ap-mumbai-1.fbtxxaolaaavw.abrg6ljrbjokn2fwhh36tqzyog4yjrth3mj2emxea4fxmzw6z35zlmh65p2a + Vault Name: dbvault + License Model: BRING_YOUR_OWN_LICENSE + Network: + Client Subnet: oke-nodesubnet-quick-cluster1-2bebe95db-regional + Domain Name: subdda0b5eaa.cluster1.oraclevcn.com + Host Name: kmshost + Listener Port: 1521 + Scan Dns Name: kmshost-scan.subdda0b5eaa.cluster1.oraclevcn.com + Vcn Name: oke-vcn-quick-cluster1-2bebe95db + Node Count: 1 + Pdb Details Status: + Pdb Config Status: + Pdb Name: kmsdb_pdb1 + Pdb State: AVAILABLE + Pluggable Database Id: ocid1.pluggabledatabase.oc1.ap-mumbai-1.anrg6ljrabf7htyaqbjqveoqzvn5dklbuc575xdrclsrkjt5juzzcelmuqla + Reco Storage Size In GB: 256 + Shape: VM.Standard2.2 + State: AVAILABLE + Storage Management: ASM + Subnet Id: ocid1.subnet.oc1.ap-mumbai-1.aaaaaaaa5zpzfax66omtbmjwlv4thruyru7focnu7fjcjksujmgwmr6vpbvq + Time Zone: UTC + Work Requests: + Operation Id: ocid1.coreservicesworkrequest.oc1.ap-mumbai-1.abrg6ljrcik5rtyygbv7qzzxqsmv6dvdwlfb7i2k3pitfqr2zomspcnkx7oa + Operation Type: Create DB System + Percent Complete: 100 + Time Accepted: 2025-01-09 18:10:41.171 +0000 UTC + Time Finished: 2025-01-09 19:31:17.126 +0000 UTC + Time Started: 2025-01-09 18:10:49.668 +0000 UTC +Events: \ No newline at end of file diff --git a/docs/dbcs/provisioning/dbcs_service_with_minimal_parameters.md b/docs/dbcs/provisioning/dbcs_service_with_minimal_parameters.md index f6bc8185..0d75297b 100644 --- a/docs/dbcs/provisioning/dbcs_service_with_minimal_parameters.md +++ b/docs/dbcs/provisioning/dbcs_service_with_minimal_parameters.md @@ -1,24 +1,24 @@ -# Deploy a DBCS DB System using OCI DBCS Service with minimal parameters +# Deploy a DB System using OCI Oracle Base Database System (OBDS) with minimal parameters -In this use case, an OCI DBCS system is deployed using Oracle DB Operator DBCS controller using minimal required parameters in the .yaml file being used during the deployment. +In this use case, an OCI Oracle Base Database System (OBDS) system is deployed using Oracle DB Operator OBDS controller using minimal required parameters in the .yaml file being used during the deployment. **NOTE** It is assumed that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-deploy-a-dbcs-system-using-oracle-db-operator-dbcs-controller) steps to create the configmap and the secrets required during the deployment. -This example uses `dbcs_service_with_minimal_parameters.yaml` to deploy a Single Instance DBCS VMDB using Oracle DB Operator DBCS Controller with: +This example uses `dbcs_service_with_minimal_parameters.yaml` to deploy a Single Instance OBDS VMDB using Oracle DB Operator OBDS Controller with: - OCI Configmap as `oci-cred` - OCI Secret as `oci-privatekey` -- Availability Domain for the DBCS VMDB as `OLou:EU-MILAN-1-AD-1` -- Compartment OCID as `ocid1.compartment.oc1..aaaaaaaaks5baeqlvv4kyj2jiwnrbxgzm3gsumcfy4c6ntj2ro5i3a5gzhhq` +- Availability Domain for the OBDS VMDB as `OLou:AP-MUMBAI-1-AD-1` +- Compartment OCID as `cid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a` - Database Admin Credential as `admin-password` -- Database Name as `dbsystem0130` +- Database Name as `dbsystem1234` - Oracle Database Software Image Version as `19c` - Database Workload Type as Transaction Processing i.e. `OLTP` -- Database Hostname Prefix as `host1205` +- Database Hostname Prefix as `host1234` - Oracle VMDB Shape as `VM.Standard2.1` -- SSH Public key for the DBCS system being deployed as `oci-publickey` +- SSH Public key for the OBDS system being deployed as `oci-publickey` - domain `vcndns.oraclevcn.com` -- OCID of the Subnet as `ocid1.subnet.oc1.eu-milan-1.aaaaaaaaeiy3tvcsnyg6upfp3ydtu7jmfnmoyifq2ax6y45b5qpdbpide5xa` +- OCID of the Subnet as `ocid1.subnet.oc1.ap-mumbai-1.aaaaaaaa5zpzfax66omtbmjwlv4thruyru7focnu7fjcjksujmgwmr6vpbvq` **NOTE:** For the details of the parameters to be used in the .yaml file, please refer [here](./dbcs_controller_parameters.md). @@ -36,9 +36,9 @@ dbcssystem.database.oracle.com/dbcssystem-create created NOTE: Check the DB Operator Pod name in your environment. ``` -[root@docker-test-server DBCS]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-g2cgw -n oracle-database-operator-system +[root@docker-test-server OBDS]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-g2cgw -n oracle-database-operator-system ``` ## Sample Output -[Here](./dbcs_service_with_minimal_parameters_sample_output.log) is the sample output for a DBCS System deployed in OCI using Oracle DB Operator DBCS Controller with minimal parameters. +[Here](./dbcs_service_with_minimal_parameters_sample_output.log) is the sample output for a OBDS System deployed in OCI using Oracle DB Operator OBDS Controller with minimal parameters. diff --git a/docs/dbcs/provisioning/dbcs_service_with_minimal_parameters.yaml b/docs/dbcs/provisioning/dbcs_service_with_minimal_parameters.yaml index dc02cfa3..66e1c229 100644 --- a/docs/dbcs/provisioning/dbcs_service_with_minimal_parameters.yaml +++ b/docs/dbcs/provisioning/dbcs_service_with_minimal_parameters.yaml @@ -1,4 +1,4 @@ -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: DbcsSystem metadata: name: dbcssystem-create @@ -6,20 +6,18 @@ spec: ociConfigMap: "oci-cred" ociSecret: "oci-privatekey" dbSystem: - availabilityDomain: "OLou:EU-MILAN-1-AD-1" - compartmentId: "ocid1.compartment.oc1..aaaaaaaaks5baeqlvv4kyj2jiwnrbxgzm3gsumcfy4c6ntj2ro5i3a5gzhhq" - dbAdminPaswordSecret: "admin-password" + availabilityDomain: "OLou:AP-MUMBAI-1-AD-1" + compartmentId: "ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a" + dbAdminPasswordSecret: "admin-password" dbEdition: "ENTERPRISE_EDITION_HIGH_PERFORMANCE" - dbName: "testdb" - displayName: "dbsystem0130" + dbName: "cdb1" + displayName: "dbsystem1234" licenseModel: "BRING_YOUR_OWN_LICENSE" dbVersion: "19c" dbWorkload: "OLTP" - hostName: "host1205" + hostName: "host1234" shape: "VM.Standard2.1" - domain: "vcndns.oraclevcn.com" - sshPublicKeys: + domain: "subdda0b5eaa.cluster1.oraclevcn.com" + sshPublicKeys: - "oci-publickey" - subnetId: "ocid1.subnet.oc1.eu-milan-1.aaaaaaaaeiy3tvcsnyg6upfp3ydtu7jmfnmoyifq2ax6y45b5qpdbpide5xa" - - + subnetId: "ocid1.subnet.oc1.ap-mumbai-1.aaaaaaaa5zpzfax66omtbmjwlv4thruyru7focnu7fjcjksujmgwmr6vpbvq" diff --git a/docs/dbcs/provisioning/dbcs_service_with_minimal_parameters_sample_output.log b/docs/dbcs/provisioning/dbcs_service_with_minimal_parameters_sample_output.log index 133f109c..80860c51 100644 --- a/docs/dbcs/provisioning/dbcs_service_with_minimal_parameters_sample_output.log +++ b/docs/dbcs/provisioning/dbcs_service_with_minimal_parameters_sample_output.log @@ -1,135 +1,82 @@ -/usr/bin/kubectl describe dbcssystems.database.oracle.com dbcssystem-create +kubectl describe dbcssystems/dbcssystem-create Name: dbcssystem-create Namespace: default Labels: -Annotations: kubectl.kubernetes.io/last-applied-configuration: - {"apiVersion":"database.oracle.com/v1alpha1","kind":"DbcsSystem","metadata":{"annotations":{},"name":"dbcssystem-create","namespace":"defa... -API Version: database.oracle.com/v1alpha1 +Annotations: lastSuccessfulSpec: + {"dbSystem":{"compartmentId":"ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a","availabilityDomain":"O... +API Version: database.oracle.com/v4 Kind: DbcsSystem Metadata: - Creation Timestamp: 2023-12-12T12:59:58Z - Generation: 1 - Managed Fields: - API Version: database.oracle.com/v1alpha1 - Fields Type: FieldsV1 - Fields V 1: - F : Metadata: - F : Annotations: - .: - F : Kubectl . Kubernetes . Io / Last - Applied - Configuration: - F : Spec: - .: - F : Db System: - .: - F : Availability Domain: - F : Compartment Id: - F : Db Admin Pasword Secret: - F : Db Edition: - F : Db Name: - F : Db Version: - F : Db Workload: - F : Display Name: - F : Domain: - F : Host Name: - F : License Model: - F : Shape: - F : Ssh Public Keys: - F : Subnet Id: - F : Oci Config Map: - F : Oci Secret: - Manager: kubectl - Operation: Update - Time: 2023-12-12T12:59:58Z - API Version: database.oracle.com/v1alpha1 - Fields Type: FieldsV1 - Fields V 1: - F : Status: - .: - F : Availability Domain: - F : Cpu Core Count: - F : Data Storage Percentage: - F : Data Storage Size In G Bs: - F : Db Edition: - F : Db Info: - F : Display Name: - F : Id: - F : License Model: - F : Network: - .: - F : Client Subnet: - F : Domain Name: - F : Host Name: - F : Listener Port: - F : Vcn Name: - F : Node Count: - F : Reco Storage Size In GB: - F : Shape: - F : State: - F : Storage Management: - F : Subnet Id: - F : Time Zone: - F : Work Requests: - Manager: manager - Operation: Update - Subresource: status - Time: 2023-12-12T14:12:36Z - Resource Version: 1571919 - UID: e11353f3-1334-4ca8-af31-4b638442f429 + Creation Timestamp: 2024-12-10T05:19:46Z + Generation: 2 + Resource Version: 117717259 + UID: 3ff13686-50ec-41e3-81c8-77bb6b5a8afa Spec: Db System: - Availability Domain: OLou:EU-MILAN-1-AD-1 - Compartment Id: ocid1.compartment.oc1..aaaaaaaaks5baeqlvv4kyj2jiwnrbxgzm3gsumcfy4c6ntj2ro5i3a5gzhhq + Availability Domain: OLou:AP-MUMBAI-1-AD-1 + Compartment Id: ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a Db Admin Pasword Secret: admin-password - Db Edition: ENTERPRISE_EDITION_HIGH_PERFORMANCE - Db Name: testdb - Db Version: 19c - Db Workload: OLTP - Display Name: dbsystem0130 - Domain: vcndns.oraclevcn.com - Host Name: host1205 - License Model: BRING_YOUR_OWN_LICENSE - Shape: VM.Standard2.1 + Db Backup Config: + Db Edition: ENTERPRISE_EDITION_HIGH_PERFORMANCE + Db Name: cdb1 + Db Version: 19c + Db Workload: OLTP + Display Name: dbsystem1234 + Domain: subdda0b5eaa.cluster1.oraclevcn.com + Host Name: host1234 + Kms Config: + License Model: BRING_YOUR_OWN_LICENSE + Shape: VM.Standard2.1 Ssh Public Keys: oci-publickey - Subnet Id: ocid1.subnet.oc1.eu-milan-1.aaaaaaaaeiy3tvcsnyg6upfp3ydtu7jmfnmoyifq2ax6y45b5qpdbpide5xa - Oci Config Map: oci-cred + Subnet Id: ocid1.subnet.oc1.ap-mumbai-1.aaaaaaaa5zpzfax66omtbmjwlv4thruyru7focnu7fjcjksujmgwmr6vpbvq + Id: ocid1.dbsystem.oc1.ap-mumbai-1.anrg6ljrabf7htyadgsso7aessztysrwaj5gcl3tp7ce6asijm2japyvmroa + Kms Config: + Oci Config Map: oci-cred-mumbai Oci Secret: oci-privatekey Status: - Availability Domain: OLou:EU-MILAN-1-AD-1 + Availability Domain: OLou:AP-MUMBAI-1-AD-1 Cpu Core Count: 1 Data Storage Percentage: 80 Data Storage Size In G Bs: 256 - Db Edition: ENTERPRISE_EDITION_HIGH_PERFORMANCE + Db Clone Status: + Db Db Unique Name: + Host Name: + Db Edition: ENTERPRISE_EDITION_HIGH_PERFORMANCE Db Info: - Db Home Id: ocid1.dbhome.oc1.eu-milan-1.anwgsljrx2vveliazumnbyvudq3rwkbc4brtamgqzyrjuwfbtx5k7hlqwx2a - Db Name: testdb - Db Unique Name: testdb_fg4_lin + Db Home Id: ocid1.dbhome.oc1.ap-mumbai-1.anrg6ljrqlb5nxiaoqqlaxhx4urdwmefw4il5efzekneuru4bpfv57i7iy6a + Db Name: cdb1 + Db Unique Name: cdb1_tkf_bom Db Workload: OLTP - Id: ocid1.database.oc1.eu-milan-1.anwgsljrabf7htyasoe7b7mtfecc7tdfkp6w5knvvufxmk3phztxfktf6naq - Display Name: dbsystem0130 - Id: ocid1.dbsystem.oc1.eu-milan-1.anwgsljrabf7htyat3fsgcftfilt45bgbrfgawroa2oasamavsluwqyr5aya - License Model: BRING_YOUR_OWN_LICENSE + Id: ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyalxin4xpiggjh4nxlta6o6iq56hjrlh4of2cq6c4qgrqa + Display Name: dbsystem1234 + Id: ocid1.dbsystem.oc1.ap-mumbai-1.anrg6ljrabf7htyadgsso7aessztysrwaj5gcl3tp7ce6asijm2japyvmroa + Kms Details Status: + License Model: BRING_YOUR_OWN_LICENSE Network: - Client Subnet: vcnsbn - Domain Name: vcndns.oraclevcn.com - Host Name: host1205 - Listener Port: 1521 - Vcn Name: vcnnet - Node Count: 1 - Reco Storage Size In GB: 256 - Shape: VM.Standard2.1 - State: PROVISIONING - Storage Management: ASM - Subnet Id: ocid1.subnet.oc1.eu-milan-1.aaaaaaaaeiy3tvcsnyg6upfp3ydtu7jmfnmoyifq2ax6y45b5qpdbpide5xa - Time Zone: UTC + Client Subnet: oke-nodesubnet-quick-cluster1-2bebe95db-regional + Domain Name: subdda0b5eaa.cluster1.oraclevcn.com + Host Name: host1234 + Listener Port: 1521 + Scan Dns Name: host1234-scan.subdda0b5eaa.cluster1.oraclevcn.com + Vcn Name: oke-vcn-quick-cluster1-2bebe95db + Node Count: 1 + Pdb Details Status: + Pdb Config Status: + Pdb Name: cdb1_pdb1 + Pdb State: AVAILABLE + Pluggable Database Id: ocid1.pluggabledatabase.oc1.ap-mumbai-1.anrg6ljrabf7htyakgj4wuabus6z5kmalvob6r6b7vivkbsmmh7bjprzbuwa + Reco Storage Size In GB: 256 + Shape: VM.Standard2.1 + State: AVAILABLE + Storage Management: ASM + Subnet Id: ocid1.subnet.oc1.ap-mumbai-1.aaaaaaaa5zpzfax66omtbmjwlv4thruyru7focnu7fjcjksujmgwmr6vpbvq + Time Zone: UTC Work Requests: - Operation Id: ocid1.coreservicesworkrequest.oc1.eu-milan-1.abwgsljrkv6jwqtepnxyhnxgtzolw74bqfh5oqlwskq72dqgjpfs5rxu66wa + Operation Id: ocid1.coreservicesworkrequest.oc1.ap-mumbai-1.abrg6ljrk2efvqjda2t7k5iaerahw7wcyz5dq2zev2k55gmq2gvsjkui7hxq Operation Type: Create DB System - Percent Complete: 0 - Time Accepted: 2023-12-12 13:00:03.156 +0000 UTC - Operation Id: ocid1.coreservicesworkrequest.oc1.eu-milan-1.abwgsljrxudjz2qivun6ypupqhytam4axv4wago7nuauceqyapbceysjukfq - Operation Type: Create DB System - Percent Complete: 0 - Time Accepted: 2023-12-12 14:12:36.047 +0000 UTC + Percent Complete: 100 + Time Accepted: 2024-12-10 05:19:52.499 +0000 UTC + Time Finished: 2024-12-10 07:59:19.083 +0000 UTC + Time Started: 2024-12-10 05:19:55.747 +0000 UTC Events: - diff --git a/docs/dbcs/provisioning/dbcs_service_with_pdb.yaml b/docs/dbcs/provisioning/dbcs_service_with_pdb.yaml new file mode 100644 index 00000000..7da5f729 --- /dev/null +++ b/docs/dbcs/provisioning/dbcs_service_with_pdb.yaml @@ -0,0 +1,38 @@ +apiVersion: database.oracle.com/v4 +kind: DbcsSystem +metadata: + name: dbcssystem-create-with-pdb + namespace: default +spec: + ociConfigMap: "oci-cred" + ociSecret: "oci-privatekey" + dbSystem: + availabilityDomain: "OLou:US-ASHBURN-AD-1" + compartmentId: "ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a" + dbAdminPasswordSecret: "admin-password" + dbEdition: "ENTERPRISE_EDITION_HIGH_PERFORMANCE" + dbName: "dbsys" + displayName: "dbsystem24" + licenseModel: "BRING_YOUR_OWN_LICENSE" + dbVersion: "21c" + dbWorkload: "OLTP" + hostName: "host24" + shape: "VM.Standard3.Flex" + cpuCoreCount: 1 + domain: "subd215df3e6.k8stest.oraclevcn.com" + sshPublicKeys: + - "oci-publickey" + subnetId: "ocid1.subnet.oc1.iad.aaaaaaaa3lmmxwsykn2jc2vphzpq6eoyoqtte3dpwg6s5fzfkti22ibol2ua" + pdbConfigs: + - pdbName: "pdb_sauahuja_11" + tdeWalletPassword: "tde-password" + pdbAdminPassword: "pdb-password" + shouldPdbAdminAccountBeLocked: false + freeformTags: + Department: "Finance" + - pdbName: "pdb_sauahuja_12" + tdeWalletPassword: "tde-password" + pdbAdminPassword: "pdb-password" + shouldPdbAdminAccountBeLocked: false + freeformTags: + Department: "HR" \ No newline at end of file diff --git a/docs/dbcs/provisioning/dbcs_service_with_pdb_sample_output.log b/docs/dbcs/provisioning/dbcs_service_with_pdb_sample_output.log new file mode 100644 index 00000000..15946a43 --- /dev/null +++ b/docs/dbcs/provisioning/dbcs_service_with_pdb_sample_output.log @@ -0,0 +1,137 @@ +2024-08-14T13:59:34Z INFO DbcsSystem system provisioned succesfully {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-create-with-pdb","namespace":"default"}, "namespace": "default", "name": "dbcssystem-create-with-pdb", "reconcileID": "4a257c84-d61d-4373-aec5-1bca6abf8267"} +2024-08-14T13:59:35Z INFO DBInst after assignment {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-create-with-pdb","namespace":"default"}, "namespace": "default", "name": "dbcssystem-create-with-pdb", "reconcileID": "4a257c84-d61d-4373-aec5-1bca6abf8267", "dbcsInst:->": {"apiVersion": "database.oracle.com/v1alpha1", "kind": "DbcsSystem", "namespace": "default", "name": "dbcssystem-create-with-pdb"}} +2024-08-14T13:59:36Z INFO Database details fetched successfully {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-create-with-pdb","namespace":"default"}, "namespace": "default", "name": "dbcssystem-create-with-pdb", "reconcileID": "4a257c84-d61d-4373-aec5-1bca6abf8267", "DatabaseId": ["ocid1.database.oc1.iad.anuwcljsabf7htyarsik3zmfezgl5tvvnmtf7wqm2n3cnvhyx5oo3nk5f6lq"]} +2024-08-14T13:59:36Z INFO Calling createPluggableDatabase {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-create-with-pdb","namespace":"default"}, "namespace": "default", "name": "dbcssystem-create-with-pdb", "reconcileID": "4a257c84-d61d-4373-aec5-1bca6abf8267", "ctx:->": "context.Background.WithCancel.WithValue(type logr.contextKey, val ).WithValue(type controller.reconcileIDKey, val )", "dbcsInst:->": {"apiVersion": "database.oracle.com/v1alpha1", "kind": "DbcsSystem", "namespace": "default", "name": "dbcssystem-create-with-pdb"}, "databaseIds:->": "ocid1.database.oc1.iad.anuwcljsabf7htyarsik3zmfezgl5tvvnmtf7wqm2n3cnvhyx5oo3nk5f6lq", "compartmentId:->": "ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a"} +2024-08-14T13:59:36Z INFO Checking if the pluggable database exists {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-create-with-pdb","namespace":"default"}, "namespace": "default", "name": "dbcssystem-create-with-pdb", "reconcileID": "4a257c84-d61d-4373-aec5-1bca6abf8267", "PDBName": "pdb_sauahuja_11"} +2024-08-14T13:59:36Z INFO TDE wallet password retrieved successfully {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-create-with-pdb","namespace":"default"}, "namespace": "default", "name": "dbcssystem-create-with-pdb", "reconcileID": "4a257c84-d61d-4373-aec5-1bca6abf8267"} +2024-08-14T13:59:36Z INFO PDB admin password retrieved successfully {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-create-with-pdb","namespace":"default"}, "namespace": "default", "name": "dbcssystem-create-with-pdb", "reconcileID": "4a257c84-d61d-4373-aec5-1bca6abf8267"} +2024-08-14T13:59:36Z INFO Creating pluggable database {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-create-with-pdb","namespace":"default"}, "namespace": "default", "name": "dbcssystem-create-with-pdb", "reconcileID": "4a257c84-d61d-4373-aec5-1bca6abf8267", "PDBName": "pdb_sauahuja_11"} +2024-08-14T13:59:37Z INFO Pluggable database creation initiated {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-create-with-pdb","namespace":"default"}, "namespace": "default", "name": "dbcssystem-create-with-pdb", "reconcileID": "4a257c84-d61d-4373-aec5-1bca6abf8267", "PDBName": "pdb_sauahuja_11", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htya4ydcdycxirop2jjf4htjcq6mnzavf6yyqfsuo74miviq"} +2024-08-14T13:59:37Z INFO Checking pluggable database status {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-create-with-pdb","namespace":"default"}, "namespace": "default", "name": "dbcssystem-create-with-pdb", "reconcileID": "4a257c84-d61d-4373-aec5-1bca6abf8267", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htya4ydcdycxirop2jjf4htjcq6mnzavf6yyqfsuo74miviq", "Status": "PROVISIONING"} +2024-08-14T14:00:07Z INFO Checking pluggable database status {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-create-with-pdb","namespace":"default"}, "namespace": "default", "name": "dbcssystem-create-with-pdb", "reconcileID": "4a257c84-d61d-4373-aec5-1bca6abf8267", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htya4ydcdycxirop2jjf4htjcq6mnzavf6yyqfsuo74miviq", "Status": "PROVISIONING"} +2024-08-14T14:00:38Z INFO Checking pluggable database status {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-create-with-pdb","namespace":"default"}, "namespace": "default", "name": "dbcssystem-create-with-pdb", "reconcileID": "4a257c84-d61d-4373-aec5-1bca6abf8267", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htya4ydcdycxirop2jjf4htjcq6mnzavf6yyqfsuo74miviq", "Status": "PROVISIONING"} +2024-08-14T14:01:08Z INFO Checking pluggable database status {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-create-with-pdb","namespace":"default"}, "namespace": "default", "name": "dbcssystem-create-with-pdb", "reconcileID": "4a257c84-d61d-4373-aec5-1bca6abf8267", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htya4ydcdycxirop2jjf4htjcq6mnzavf6yyqfsuo74miviq", "Status": "PROVISIONING"} +2024-08-14T14:01:38Z INFO Checking pluggable database status {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-create-with-pdb","namespace":"default"}, "namespace": "default", "name": "dbcssystem-create-with-pdb", "reconcileID": "4a257c84-d61d-4373-aec5-1bca6abf8267", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htya4ydcdycxirop2jjf4htjcq6mnzavf6yyqfsuo74miviq", "Status": "PROVISIONING"} +2024-08-14T14:02:08Z INFO Checking pluggable database status {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-create-with-pdb","namespace":"default"}, "namespace": "default", "name": "dbcssystem-create-with-pdb", "reconcileID": "4a257c84-d61d-4373-aec5-1bca6abf8267", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htya4ydcdycxirop2jjf4htjcq6mnzavf6yyqfsuo74miviq", "Status": "PROVISIONING"} +2024-08-14T14:02:38Z INFO Checking pluggable database status {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-create-with-pdb","namespace":"default"}, "namespace": "default", "name": "dbcssystem-create-with-pdb", "reconcileID": "4a257c84-d61d-4373-aec5-1bca6abf8267", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htya4ydcdycxirop2jjf4htjcq6mnzavf6yyqfsuo74miviq", "Status": "AVAILABLE"} +2024-08-14T14:02:38Z INFO Pluggable database successfully created {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-create-with-pdb","namespace":"default"}, "namespace": "default", "name": "dbcssystem-create-with-pdb", "reconcileID": "4a257c84-d61d-4373-aec5-1bca6abf8267", "PDBName": "pdb_sauahuja_11", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htya4ydcdycxirop2jjf4htjcq6mnzavf6yyqfsuo74miviq"} +2024-08-14T14:02:39Z INFO Database details fetched successfully {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-create-with-pdb","namespace":"default"}, "namespace": "default", "name": "dbcssystem-create-with-pdb", "reconcileID": "4a257c84-d61d-4373-aec5-1bca6abf8267", "DatabaseId": ["ocid1.database.oc1.iad.anuwcljsabf7htyarsik3zmfezgl5tvvnmtf7wqm2n3cnvhyx5oo3nk5f6lq"]} +2024-08-14T14:02:39Z INFO Calling createPluggableDatabase {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-create-with-pdb","namespace":"default"}, "namespace": "default", "name": "dbcssystem-create-with-pdb", "reconcileID": "4a257c84-d61d-4373-aec5-1bca6abf8267", "ctx:->": "context.Background.WithCancel.WithValue(type logr.contextKey, val ).WithValue(type controller.reconcileIDKey, val )", "dbcsInst:->": {"apiVersion": "database.oracle.com/v1alpha1", "kind": "DbcsSystem", "namespace": "default", "name": "dbcssystem-create-with-pdb"}, "databaseIds:->": "ocid1.database.oc1.iad.anuwcljsabf7htyarsik3zmfezgl5tvvnmtf7wqm2n3cnvhyx5oo3nk5f6lq", "compartmentId:->": "ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a"} +2024-08-14T14:02:39Z INFO Checking if the pluggable database exists {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-create-with-pdb","namespace":"default"}, "namespace": "default", "name": "dbcssystem-create-with-pdb", "reconcileID": "4a257c84-d61d-4373-aec5-1bca6abf8267", "PDBName": "pdb_sauahuja_12"} +2024-08-14T14:02:39Z INFO TDE wallet password retrieved successfully {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-create-with-pdb","namespace":"default"}, "namespace": "default", "name": "dbcssystem-create-with-pdb", "reconcileID": "4a257c84-d61d-4373-aec5-1bca6abf8267"} +2024-08-14T14:02:39Z INFO PDB admin password retrieved successfully {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-create-with-pdb","namespace":"default"}, "namespace": "default", "name": "dbcssystem-create-with-pdb", "reconcileID": "4a257c84-d61d-4373-aec5-1bca6abf8267"} +2024-08-14T14:02:39Z INFO Creating pluggable database {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-create-with-pdb","namespace":"default"}, "namespace": "default", "name": "dbcssystem-create-with-pdb", "reconcileID": "4a257c84-d61d-4373-aec5-1bca6abf8267", "PDBName": "pdb_sauahuja_12"} +2024-08-14T14:02:40Z INFO Pluggable database creation initiated {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-create-with-pdb","namespace":"default"}, "namespace": "default", "name": "dbcssystem-create-with-pdb", "reconcileID": "4a257c84-d61d-4373-aec5-1bca6abf8267", "PDBName": "pdb_sauahuja_12", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htyahbbg3hf56qhw55cou7465zmuukgv7hh46niu3dsoug3q"} +2024-08-14T14:02:40Z INFO Checking pluggable database status {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-create-with-pdb","namespace":"default"}, "namespace": "default", "name": "dbcssystem-create-with-pdb", "reconcileID": "4a257c84-d61d-4373-aec5-1bca6abf8267", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htyahbbg3hf56qhw55cou7465zmuukgv7hh46niu3dsoug3q", "Status": "PROVISIONING"} +2024-08-14T14:03:11Z INFO Checking pluggable database status {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-create-with-pdb","namespace":"default"}, "namespace": "default", "name": "dbcssystem-create-with-pdb", "reconcileID": "4a257c84-d61d-4373-aec5-1bca6abf8267", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htyahbbg3hf56qhw55cou7465zmuukgv7hh46niu3dsoug3q", "Status": "PROVISIONING"} +2024-08-14T14:03:41Z INFO Checking pluggable database status {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-create-with-pdb","namespace":"default"}, "namespace": "default", "name": "dbcssystem-create-with-pdb", "reconcileID": "4a257c84-d61d-4373-aec5-1bca6abf8267", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htyahbbg3hf56qhw55cou7465zmuukgv7hh46niu3dsoug3q", "Status": "PROVISIONING"} +2024-08-14T14:04:11Z INFO Checking pluggable database status {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-create-with-pdb","namespace":"default"}, "namespace": "default", "name": "dbcssystem-create-with-pdb", "reconcileID": "4a257c84-d61d-4373-aec5-1bca6abf8267", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htyahbbg3hf56qhw55cou7465zmuukgv7hh46niu3dsoug3q", "Status": "PROVISIONING"} +2024-08-14T14:04:41Z INFO Checking pluggable database status {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-create-with-pdb","namespace":"default"}, "namespace": "default", "name": "dbcssystem-create-with-pdb", "reconcileID": "4a257c84-d61d-4373-aec5-1bca6abf8267", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htyahbbg3hf56qhw55cou7465zmuukgv7hh46niu3dsoug3q", "Status": "PROVISIONING"} +2024-08-14T14:05:11Z INFO Checking pluggable database status {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-create-with-pdb","namespace":"default"}, "namespace": "default", "name": "dbcssystem-create-with-pdb", "reconcileID": "4a257c84-d61d-4373-aec5-1bca6abf8267", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htyahbbg3hf56qhw55cou7465zmuukgv7hh46niu3dsoug3q", "Status": "PROVISIONING"} +2024-08-14T14:05:42Z INFO Checking pluggable database status {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-create-with-pdb","namespace":"default"}, "namespace": "default", "name": "dbcssystem-create-with-pdb", "reconcileID": "4a257c84-d61d-4373-aec5-1bca6abf8267", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htyahbbg3hf56qhw55cou7465zmuukgv7hh46niu3dsoug3q", "Status": "PROVISIONING"} +2024-08-14T14:06:12Z INFO Checking pluggable database status {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-create-with-pdb","namespace":"default"}, "namespace": "default", "name": "dbcssystem-create-with-pdb", "reconcileID": "4a257c84-d61d-4373-aec5-1bca6abf8267", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htyahbbg3hf56qhw55cou7465zmuukgv7hh46niu3dsoug3q", "Status": "AVAILABLE"} +2024-08-14T14:06:12Z INFO Pluggable database successfully created {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-create-with-pdb","namespace":"default"}, "namespace": "default", "name": "dbcssystem-create-with-pdb", "reconcileID": "4a257c84-d61d-4373-aec5-1bca6abf8267", "PDBName": "pdb_sauahuja_12", "PDBID": "ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htyahbbg3hf56qhw55cou7465zmuukgv7hh46niu3dsoug3q"} + + +#kubectl describe dbcssystems.database.oracle.com +Name: dbcssystem-create-with-pdb +Namespace: default +Labels: +Annotations: lastSuccessfulSpec: + {"dbSystem":{"compartmentId":"ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a","availabilityDomain":"O... +API Version: database.oracle.com/v1alpha1 +Kind: DbcsSystem +Metadata: + Creation Timestamp: 2024-08-16T09:26:08Z + Generation: 1 + Resource Version: 68483815 + UID: 9dd15628-e47b-4d9c-8bc6-2388e51cba30 +Spec: + Db System: + Availability Domain: OLou:US-ASHBURN-AD-1 + Compartment Id: ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a + Cpu Core Count: 1 + Db Admin Pasword Secret: admin-password + Db Backup Config: + Db Edition: ENTERPRISE_EDITION_HIGH_PERFORMANCE + Db Name: dbsys + Db Version: 21c + Db Workload: OLTP + Display Name: dbsystem24 + Domain: subd215df3e6.k8stest.oraclevcn.com + Host Name: host24 + Kms Config: + License Model: BRING_YOUR_OWN_LICENSE + Shape: VM.Standard3.Flex + Ssh Public Keys: + oci-publickey + Subnet Id: ocid1.subnet.oc1.iad.aaaaaaaa3lmmxwsykn2jc2vphzpq6eoyoqtte3dpwg6s5fzfkti22ibol2ua + Kms Config: + Oci Config Map: oci-cred + Oci Secret: oci-privatekey + Pdb Configs: + Freeform Tags: + Department: Finance + Pdb Admin Password: pdb-password + Pdb Name: pdb_sauahuja_11 + Should Pdb Admin Account Be Locked: false + Tde Wallet Password: tde-password + Freeform Tags: + Department: HR + Pdb Admin Password: pdb-password + Pdb Name: pdb_sauahuja_12 + Should Pdb Admin Account Be Locked: false + Tde Wallet Password: tde-password +Status: + Availability Domain: OLou:US-ASHBURN-AD-1 + Cpu Core Count: 1 + Data Storage Percentage: 80 + Data Storage Size In G Bs: 256 + Db Edition: ENTERPRISE_EDITION_HIGH_PERFORMANCE + Db Info: + Db Home Id: ocid1.dbhome.oc1.iad.anuwcljsqlb5nxiaqfh3twuegmxxci5boocmowxd6kcczeq6e7jwqezfmbwq + Db Name: dbsys + Db Unique Name: dbsys_dss_iad + Db Workload: OLTP + Id: ocid1.database.oc1.iad.anuwcljsabf7htya5c2ttar7axxqq6qej3allfz23nvrtx6ilka4stdmrpga + Display Name: dbsystem24 + Id: ocid1.dbsystem.oc1.iad.anuwcljsabf7htya55wz5vfil7ul3pkzpubnymp6zrp3fhgomv3fcdr2vtiq + Kms Details Status: + License Model: BRING_YOUR_OWN_LICENSE + Network: + Client Subnet: oke-nodesubnet-quick-k8s-test-ae2addeb0-regional + Domain Name: subd215df3e6.k8stest.oraclevcn.com + Host Name: host24 + Listener Port: 1521 + Scan Dns Name: host24-scan.subd215df3e6.k8stest.oraclevcn.com + Vcn Name: oke-vcn-quick-k8s-test-ae2addeb0 + Node Count: 1 + Pdb Details Status: + Pdb Config Status: + Freeform Tags: + Department: Finance + Pdb Name: pdb_sauahuja_11 + Pdb State: AVAILABLE + Pluggable Database Id: ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htya4arzakcgum6mv7h6cqmxhepyrjzfs77mxhqt4f3gylxq + Should Pdb Admin Account Be Locked: false + Pdb Config Status: + Freeform Tags: + Department: HR + Pdb Name: pdb_sauahuja_12 + Pdb State: AVAILABLE + Pluggable Database Id: ocid1.pluggabledatabase.oc1.iad.anuwcljsabf7htyaiq6cyhxwqr4ad3pfn7g6e6nd2myiibj54tbg7vc27hfa + Should Pdb Admin Account Be Locked: false + Reco Storage Size In GB: 256 + Shape: VM.Standard3.Flex + State: AVAILABLE + Storage Management: ASM + Subnet Id: ocid1.subnet.oc1.iad.aaaaaaaa3lmmxwsykn2jc2vphzpq6eoyoqtte3dpwg6s5fzfkti22ibol2ua + Time Zone: UTC + Work Requests: + Operation Id: ocid1.coreservicesworkrequest.oc1.iad.abuwcljscvomyvuthyc5bnmgi4myo565mbaghtjbhscgvabiy4tyzahjtiba + Operation Type: Create DB System + Percent Complete: 100 + Time Accepted: 2024-08-14 18:28:36.996 +0000 UTC + Time Finished: 2024-08-14 19:44:28.607 +0000 UTC + Time Started: 2024-08-14 18:28:45.134 +0000 UTC +Events: diff --git a/docs/dbcs/provisioning/delete_pdb.md b/docs/dbcs/provisioning/delete_pdb.md new file mode 100644 index 00000000..84d676bc --- /dev/null +++ b/docs/dbcs/provisioning/delete_pdb.md @@ -0,0 +1,50 @@ +# Delete PDB of an existing DBCS System + +In this use case, an existing OCI DBCS system deployed earlier is going to have PDB/PDBs deleted. Its a 2 Step operation. + +In order to create PDBs to an existing DBCS system, the steps will be: + +1. Bind the existing DBCS System to DBCS Controller. +2. Apply the change to delete PDBs. + +**NOTE:** We are assuming that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-deploy-a-dbcs-system-using-oracle-db-operator-dbcs-controller) steps to create the configmap and the secrets required during the deployment. + +As step 1, first bind the existing DBCS System to DBCS Controller following [documentation](./../provisioning/bind_to_existing_dbcs_system.md). After successful binding, it will show as below- +```bash +kubectl get dbcssystems +NAME AGE +dbcssystem-existing 3m33s +``` + +This example uses `deletepdb_in_existing_dbcs_system_list.yaml` to delete PDBs of a Single Instance DBCS VMDB using Oracle DB Operator DBCS Controller with: + +- OCID of existing VMDB as `ocid1.dbsystem.oc1.iad.anuwcljsabf7htyag4akvoakzw4qk7cae55qyp7hlffbouozvyl5ngoputza` +- OCI Configmap as `oci-cred` +- OCI Secret as `oci-privatekey` +- PDB Name to be deleted e.g `pdb_sauahuja_11` and `pdb_sauahuja_12` +**NOTE:** For the details of the parameters to be used in the .yaml file, please refer [here](./dbcs_controller_parameters.md). + +Use the file: [deletepdb_in_existing_dbcs_system_list.yaml](./deletepdb_in_existing_dbcs_system_list.yaml) for this use case as below: + +1. Deploy the .yaml file: +```sh +[root@docker-test-server DBCS]# kubectl apply -f deletepdb_in_existing_dbcs_system_list.yaml +dbcssystem.database.oracle.com/dbcssystem-existing configured +``` + +2. Monitor the Oracle DB Operator Pod `pod/oracle-database-operator-controller-manager-665874bd57-g2cgw` for the progress of the DBCS VMDB deletion of PDBs. + +NOTE: Check the DB Operator Pod name in your environment. + +```bash +[root@docker-test-server DBCS]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-g2cgw -n oracle-database-operator-system +``` + +3. Remove DBCS Systems resource- +```bash +kubectl delete -f deletepdb_in_existing_dbcs_system_list.yaml +``` + +## Sample Output + +[Here](./deletepdb_in_existing_dbcs_system_list_sample_output.log) is the sample output for deletion of PDBs from an existing DBCS System deployed in OCI using Oracle DB Operator DBCS Controller. \ No newline at end of file diff --git a/docs/dbcs/provisioning/deletepdb_in_existing_dbcs_system_list.yaml b/docs/dbcs/provisioning/deletepdb_in_existing_dbcs_system_list.yaml new file mode 100644 index 00000000..fed3ec6c --- /dev/null +++ b/docs/dbcs/provisioning/deletepdb_in_existing_dbcs_system_list.yaml @@ -0,0 +1,13 @@ +kind: DbcsSystem +metadata: + name: dbcssystem-existing + namespace: default +spec: + id: "ocid1.dbsystem.oc1.iad.anuwcljsabf7htyag4akvoakzw4qk7cae55qyp7hlffbouozvyl5ngoputza" + ociConfigMap: "oci-cred" + ociSecret: "oci-privatekey" + pdbConfigs: + - pdbName: "pdb_sauahuja_11" + isDelete: true + - pdbName: "pdb_sauahuja_12" + isDelete: true \ No newline at end of file diff --git a/docs/dbcs/provisioning/deletepdb_in_existing_dbcs_system_list_sample_output.log b/docs/dbcs/provisioning/deletepdb_in_existing_dbcs_system_list_sample_output.log new file mode 100644 index 00000000..a4f75fa5 --- /dev/null +++ b/docs/dbcs/provisioning/deletepdb_in_existing_dbcs_system_list_sample_output.log @@ -0,0 +1,8 @@ +2024-07-01T12:34:44Z INFO Database details fetched successfully {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "e9cdd6c8-a381-40c2-b621-5c97467f6808", "DatabaseId": ["ocid1.database.oc1.iad.anuwcljsabf7htyaxx3o46rynl5vyduxilwxeeafndy4cwqtkywkhcws435a"]} +2024-07-01T12:34:44Z INFO Deleting pluggable database {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "e9cdd6c8-a381-40c2-b621-5c97467f6808", "PDBName": "pdb_sauahuja_11"} +2024-07-01T12:34:44Z INFO PluggableDatabaseId is not specified, getting pluggable databaseID {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "e9cdd6c8-a381-40c2-b621-5c97467f6808"} +2024-07-01T12:34:45Z INFO Successfully deleted pluggable database {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "e9cdd6c8-a381-40c2-b621-5c97467f6808", "PDBName": "pdb_sauahuja_11"} +2024-07-01T12:34:46Z INFO Database details fetched successfully {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "e9cdd6c8-a381-40c2-b621-5c97467f6808", "DatabaseId": ["ocid1.database.oc1.iad.anuwcljsabf7htyaxx3o46rynl5vyduxilwxeeafndy4cwqtkywkhcws435a"]} +2024-07-01T12:34:46Z INFO Deleting pluggable database {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "e9cdd6c8-a381-40c2-b621-5c97467f6808", "PDBName": "pdb_sauahuja_12"} +2024-07-01T12:34:46Z INFO PluggableDatabaseId is not specified, getting pluggable databaseID {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "e9cdd6c8-a381-40c2-b621-5c97467f6808"} +2024-07-01T12:34:47Z INFO Successfully deleted pluggable database {"controller": "dbcssystem", "controllerGroup": "database.oracle.com", "controllerKind": "DbcsSystem", "DbcsSystem": {"name":"dbcssystem-existing","namespace":"default"}, "namespace": "default", "name": "dbcssystem-existing", "reconcileID": "e9cdd6c8-a381-40c2-b621-5c97467f6808", "PDBName": "pdb_sauahuja_12"} \ No newline at end of file diff --git a/docs/dbcs/provisioning/migrate_to_kms.md b/docs/dbcs/provisioning/migrate_to_kms.md new file mode 100644 index 00000000..0c5ee10c --- /dev/null +++ b/docs/dbcs/provisioning/migrate_to_kms.md @@ -0,0 +1,49 @@ +# Create and update KMS vault to an existing OBDS System already deployed in OCI Oracle Base Database System (OBDS) + +In this use case, an existing OCI OBDS system deployed earlier having encryption with TDE Wallet Password, will be migrated to have KMS Vault created and update OBDS System in OCI. Its a 2 Step operation. + +In order to create KMS Vaults to an existing OBDS system, the steps will be: + +1. Bind the existing OBDS System (having encryption enabled with TDE Wallet password) to the OBDS Controller. +2. Apply the change to create KMS Vaults. + +**NOTE:** We are assuming that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-deploy-a-dbcs-system-using-oracle-db-operator-dbcs-controller) steps to create the configmap and the secrets required during the deployment. It is also assumed that OBDS System you created earlier is using TDE Wallet password. + +As step 1, first bind the existing OBDS System to OBDS Controller following [documentation](./../provisioning/bind_to_existing_dbcs_system.md). After successful binding, it will show as below- +```bash +kubectl get dbcssystems +NAME AGE +dbcssystem-create 3m33s +``` +Below proceeding further create PDB Admin Password which is going to used as name suggests. + +This example uses `dbcs_service_migrate_to_kms.yaml` to create KMS Vault to existing OBDS VMDB having encryption already enabled earlier with TDE Wallet using Oracle DB Operator OBDS Controller with: + +- OCID of existing VMDB as `ocid1.dbsystem.oc1.iad.anuwcljsabf7htyaoja4v2kx5rcfe5w2onndjfpqjhjoakxgwxo2sbgei5iq` +- OCI Configmap as `oci-cred` +- OCI Secret as `oci-privatekey` +- Existing `dbSystem` details (`compartmentId`,`dbAdminPasswordSecret`,`tdeWalletPasswordSecret`)used before to create OBDS system. +- kmsConfig - vaultName as `dbvault` as an example. +- kmsConfig - keyName as `dbkey` as an example. +- kmsConfig - compartmentId as `ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a` as an example. +**NOTE:** For the details of the parameters to be used in the .yaml file, please refer [here](./dbcs_controller_parameters.md). + +Use the file: [dbcs_service_migrate_to_kms.yaml](./dbcs_service_migrate_to_kms.yaml) for this use case as below: + +1. Deploy the .yaml file: +```sh +[root@docker-test-server OBDS]# kubectl apply -f dbcs_service_migrate_to_kms.yaml +dbcssystem.database.oracle.com/dbcssystem-existing configured +``` + +2. Monitor the Oracle DB Operator Pod `pod/oracle-database-operator-controller-manager-665874bd57-g2cgw` for the progress of the OBDS VMDB creation of KMS Vaults. + +NOTE: Check the DB Operator Pod name in your environment. + +``` +[root@docker-test-server OBDS]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-g2cgw -n oracle-database-operator-system +``` + +## Sample Output + +[Here](./dbcs_service_migrate_to_kms.log) is the sample output for creation of KMS Vaults on an existing OBDS System deployed in OCI using Oracle DB Operator OBDS Controller. diff --git a/docs/dbcs/provisioning/scale_down_dbcs_system_shape.md b/docs/dbcs/provisioning/scale_down_dbcs_system_shape.md index abe98eea..1f03ff9f 100644 --- a/docs/dbcs/provisioning/scale_down_dbcs_system_shape.md +++ b/docs/dbcs/provisioning/scale_down_dbcs_system_shape.md @@ -1,26 +1,26 @@ -# Scale Down the shape of an existing DBCS System +# Scale Down the shape of an existing OBDS System -In this use case, an existing OCI DBCS system deployed earlier is scaled down for its shape using Oracle DB Operator DBCS controller. Its a 2 Step operation. +In this use case, an existing OCI OBDS system deployed earlier is scaled down for its shape using Oracle DB Operator OBDS controller. Its a 2 Step operation. -In order to scale down an existing DBCS system, the steps will be: +In order to scale down an existing OBDS system, the steps will be: -1. Bind the existing DBCS System to DBCS Controller. +1. Bind the existing OBDS System to OBDS Controller. 2. Apply the change to scale down its shape. **NOTE** We are assuming that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-deploy-a-dbcs-system-using-oracle-db-operator-dbcs-controller) steps to create the configmap and the secrets required during the deployment. -This example uses `scale_down_dbcs_system_shape.yaml` to scale down a Single Instance DBCS VMDB using Oracle DB Operator DBCS Controller with: +This example uses `scale_down_dbcs_system_shape.yaml` to scale down a Single Instance OBDS VMDB using Oracle DB Operator OBDS Controller with: -- OCID of existing VMDB as `ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa` +- OCID of existing VMDB as `ocid1.dbsystem.oc1.ap-mumbai-1.anrg6ljrabf7htyadgsso7aessztysrwaj5gcl3tp7ce6asijm2japyvmroa` - OCI Configmap as `oci-cred` - OCI Secret as `oci-privatekey` -- Availability Domain for the DBCS VMDB as `OLou:PHX-AD-1` -- Compartment OCID as `ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya` +- Availability Domain for the OBDS VMDB as `OLou:AP-MUMBAI-1-AD-1` +- Compartment OCID as `ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72` - Database Admin Credential as `admin-password` -- Database Hostname Prefix as `host0130` +- Database Hostname Prefix as `host1234` - Oracle VMDB target Shape as `VM.Standard2.1` -- SSH Public key for the DBCS system being deployed as `oci-publickey` -- OCID of the Subnet as `ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a` +- SSH Public key for the OBDS system being deployed as `oci-publickey` +- OCID of the Subnet as `ocid1.subnet.oc1.ap-mumbai-1.aaaaaaaa5zpzfax66omtbmjwlv4thruyru7focnu7fjcjksujmgwmr6vpbvq` **NOTE:** For the details of the parameters to be used in the .yaml file, please refer [here](./dbcs_controller_parameters.md). @@ -28,18 +28,18 @@ Use the file: [scale_down_dbcs_system_shape.yaml](./scale_down_dbcs_system_shape 1. Deploy the .yaml file: ```sh -[root@docker-test-server DBCS]# kubectl apply -f scale_down_dbcs_system_shape.yaml +[root@docker-test-server OBDS]# kubectl apply -f scale_down_dbcs_system_shape.yaml dbcssystem.database.oracle.com/dbcssystem-existing configured ``` -2. Monitor the Oracle DB Operator Pod `pod/oracle-database-operator-controller-manager-665874bd57-g2cgw` for the progress of the DBCS VMDB Scale down. +2. Monitor the Oracle DB Operator Pod `pod/oracle-database-operator-controller-manager-665874bd57-g2cgw` for the progress of the OBDS VMDB Scale down. NOTE: Check the DB Operator Pod name in your environment. ``` -[root@docker-test-server DBCS]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-g2cgw -n oracle-database-operator-system +[root@docker-test-server OBDS]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-g2cgw -n oracle-database-operator-system ``` ## Sample Output -[Here](./scale_down_dbcs_system_shape_sample_output.log) is the sample output for scaling down the shape of an existing DBCS System deployed in OCI using Oracle DB Operator DBCS Controller. +[Here](./scale_down_dbcs_system_shape_sample_output.log) is the sample output for scaling down the shape of an existing OBDS System deployed in OCI using Oracle DB Operator OBDS Controller. diff --git a/docs/dbcs/provisioning/scale_down_dbcs_system_shape.yaml b/docs/dbcs/provisioning/scale_down_dbcs_system_shape.yaml index 5e2cfb3f..f4394ddc 100644 --- a/docs/dbcs/provisioning/scale_down_dbcs_system_shape.yaml +++ b/docs/dbcs/provisioning/scale_down_dbcs_system_shape.yaml @@ -1,17 +1,18 @@ -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: DbcsSystem metadata: name: dbcssystem-existing spec: - id: "ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa" + id: "ocid1.dbsystem.oc1.ap-mumbai-1.anrg6ljrabf7htyadgsso7aessztysrwaj5gcl3tp7ce6asijm2japyvmroa" ociConfigMap: "oci-cred" ociSecret: "oci-privatekey" dbSystem: - availabilityDomain: "OLou:PHX-AD-1" - compartmentId: "ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya" - dbAdminPaswordSecret: "admin-password" - hostName: "host0130" + availabilityDomain: "OLou:AP-MUMBAI-1-AD-1" + compartmentId: "ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a" + dbAdminPasswordSecret: "admin-password" + hostName: "host1234" shape: "VM.Standard2.1" + domain: "subdda0b5eaa.cluster1.oraclevcn.com" sshPublicKeys: - - "oci-publickey" - subnetId: "ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a" + - "oci-publickey" + subnetId: "ocid1.subnet.oc1.ap-mumbai-1.aaaaaaaa5zpzfax66omtbmjwlv4thruyru7focnu7fjcjksujmgwmr6vpbvq" diff --git a/docs/dbcs/provisioning/scale_down_dbcs_system_shape_sample_output.log b/docs/dbcs/provisioning/scale_down_dbcs_system_shape_sample_output.log index acc45208..32e0a318 100644 --- a/docs/dbcs/provisioning/scale_down_dbcs_system_shape_sample_output.log +++ b/docs/dbcs/provisioning/scale_down_dbcs_system_shape_sample_output.log @@ -1,176 +1,6 @@ -[root@docker-test-server test]# kubectl describe dbcssystems.database.oracle.com dbcssystem-existing -Name: dbcssystem-existing -Namespace: default -Labels: -Annotations: lastSuccessfulSpec: - {"dbSystem":{"compartmentId":"ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya","availabilityDomain":"O... -API Version: database.oracle.com/v1alpha1 -Kind: DbcsSystem -Metadata: - Creation Timestamp: 2022-03-08T23:27:48Z - Generation: 2 - Managed Fields: - API Version: database.oracle.com/v1alpha1 - Fields Type: FieldsV1 - fieldsV1: - f:metadata: - f:annotations: - .: - f:kubectl.kubernetes.io/last-applied-configuration: - f:spec: - .: - f:dbSystem: - .: - f:availabilityDomain: - f:compartmentId: - f:dbAdminPaswordSecret: - f:hostName: - f:shape: - f:sshPublicKeys: - f:subnetId: - f:id: - f:ociConfigMap: - f:ociSecret: - Manager: kubectl-client-side-apply - Operation: Update - Time: 2022-03-08T23:32:50Z - API Version: database.oracle.com/v1alpha1 - Fields Type: FieldsV1 - fieldsV1: - f:metadata: - f:annotations: - f:lastSuccessfulSpec: - f:spec: - f:dbSystem: - f:cpuCoreCount: - f:dbBackupConfig: - f:dbEdition: - f:dbName: - f:dbUniqueName: - f:dbVersion: - f:diskRedundancy: - f:displayName: - f:faultDomains: - f:nodeCount: - f:status: - .: - f:availabilityDomain: - f:cpuCoreCount: - f:dataStoragePercentage: - f:dataStorageSizeInGBs: - f:dbEdition: - f:dbInfo: - f:displayName: - f:id: - f:licenseModel: - f:network: - .: - f:clientSubnet: - f:domainName: - f:hostName: - f:listenerPort: - f:scanDnsName: - f:vcnName: - f:nodeCount: - f:recoStorageSizeInGB: - f:shape: - f:state: - f:storageManagement: - f:subnetId: - f:timeZone: - f:workRequests: - Manager: manager - Operation: Update - Time: 2022-03-08T23:32:55Z - Resource Version: 55197836 - UID: 96d7bc49-33e9-42cc-8dd0-ada9a5a4c7e5 -Spec: - Db System: - Availability Domain: OLou:PHX-AD-1 - Compartment Id: ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya - Db Admin Pasword Secret: admin-password - Host Name: host0130 - Shape: VM.Standard2.2 - Ssh Public Keys: - oci-publickey - Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a - Id: ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa - Oci Config Map: oci-cred - Oci Secret: oci-privatekey -Status: - Availability Domain: OLou:PHX-AD-1 - Cpu Core Count: 2 - Data Storage Percentage: 80 - Data Storage Size In G Bs: 256 - Db Edition: ENTERPRISE_EDITION - Db Info: - Db Home Id: ocid1.dbhome.oc1.phx.anyhqljr5gy3jhqat52milqwt3gq6lwohhacwg5yi4mtzq7c7hag53lrkugq - Db Name: db0130 - Db Unique Name: db0130_phx1zn - Db Workload: OLTP - Id: ocid1.database.oc1.phx.anyhqljrabf7htyackgmsaqjfexoqgrzuuk33ju2q25z2al43tnd5mhhvkra - Display Name: dbsystem20220308221302 - Id: ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa - License Model: LICENSE_INCLUDED - Network: - Client Subnet: k8test-pubvcn - Domain Name: k8testpubvcn.k8test.oraclevcn.com - Host Name: host0130 - Listener Port: 1521 - Scan Dns Name: host0130-scan.k8testpubvcn.k8test.oraclevcn.com - Vcn Name: k8test - Node Count: 1 - Reco Storage Size In GB: 256 - Shape: VM.Standard2.2 - State: AVAILABLE - Storage Management: ASM - Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a - Time Zone: UTC - Work Requests: - Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrxivzvgzel47zuoyke5yk36o7mrgjl27vscd5z3bqptmyh3rxwbqq - Operation Type: Create DB System - Percent Complete: 100 - Time Accepted: 2022-03-08 22:13:02.999 +0000 UTC - Time Finished: 2022-03-08 23:11:50.46 +0000 UTC - Time Started: 2022-03-08 22:13:16.995 +0000 UTC - Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrc3fx6kwq4yxerk3ngztdbbngm7w4dnlddcdhxqxjn6e4kcyux5ca - Operation Type: Update Shape - Percent Complete: 100 - Time Accepted: 2022-03-08 23:33:42.807 +0000 UTC - Time Finished: 2022-03-08 23:46:21.126 +0000 UTC - Time Started: 2022-03-08 23:33:52.109 +0000 UTC -Events: -[root@docker-test-server test]# - - - - -[root@docker-test-server test]# cat scale_down_dbcs_system_shape.yaml -apiVersion: database.oracle.com/v1alpha1 -kind: DbcsSystem -metadata: - name: dbcssystem-existing -spec: - id: "ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa" - ociConfigMap: "oci-cred" - ociSecret: "oci-privatekey" - dbSystem: - availabilityDomain: "OLou:PHX-AD-1" - compartmentId: "ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya" - dbAdminPaswordSecret: "admin-password" - hostName: "host0130" - shape: "VM.Standard2.1" - sshPublicKeys: - - "oci-publickey" - subnetId: "ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a" -[root@docker-test-server test]# -[root@docker-test-server test]# [root@docker-test-server test]# kubectl apply -f scale_down_dbcs_system_shape.yaml dbcssystem.database.oracle.com/dbcssystem-existing configured - - - [root@docker-test-server test]# kubectl get ns kubectl get allNAME STATUS AGE @@ -221,158 +51,103 @@ replicaset.apps/oracle-database-operator-controller-manager-665874bd57 3 2022-03-09T00:38:18.344Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} - - - - - - - [root@docker-test-server test]# kubectl describe dbcssystems.database.oracle.com dbcssystem-existing Name: dbcssystem-existing Namespace: default Labels: Annotations: lastSuccessfulSpec: - {"dbSystem":{"compartmentId":"ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya","availabilityDomain":"O... -API Version: database.oracle.com/v1alpha1 + {"dbSystem":{"compartmentId":"ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a","availabilityDomain":"O... +API Version: database.oracle.com/v4 Kind: DbcsSystem Metadata: - Creation Timestamp: 2022-03-08T23:27:48Z - Generation: 3 - Managed Fields: - API Version: database.oracle.com/v1alpha1 - Fields Type: FieldsV1 - fieldsV1: - f:metadata: - f:annotations: - .: - f:kubectl.kubernetes.io/last-applied-configuration: - f:spec: - .: - f:dbSystem: - .: - f:availabilityDomain: - f:compartmentId: - f:dbAdminPaswordSecret: - f:hostName: - f:shape: - f:sshPublicKeys: - f:subnetId: - f:id: - f:ociConfigMap: - f:ociSecret: - Manager: kubectl-client-side-apply - Operation: Update - Time: 2022-03-08T23:32:50Z - API Version: database.oracle.com/v1alpha1 - Fields Type: FieldsV1 - fieldsV1: - f:metadata: - f:annotations: - f:lastSuccessfulSpec: - f:spec: - f:dbSystem: - f:cpuCoreCount: - f:dbBackupConfig: - f:dbEdition: - f:dbName: - f:dbUniqueName: - f:dbVersion: - f:diskRedundancy: - f:displayName: - f:faultDomains: - f:nodeCount: - f:status: - .: - f:availabilityDomain: - f:cpuCoreCount: - f:dataStoragePercentage: - f:dataStorageSizeInGBs: - f:dbEdition: - f:dbInfo: - f:displayName: - f:id: - f:licenseModel: - f:network: - .: - f:clientSubnet: - f:domainName: - f:hostName: - f:listenerPort: - f:scanDnsName: - f:vcnName: - f:nodeCount: - f:recoStorageSizeInGB: - f:shape: - f:state: - f:storageManagement: - f:subnetId: - f:timeZone: - f:workRequests: - Manager: manager - Operation: Update - Time: 2022-03-08T23:32:55Z - Resource Version: 55214174 - UID: 96d7bc49-33e9-42cc-8dd0-ada9a5a4c7e5 + Creation Timestamp: 2024-12-10T10:54:17Z + Generation: 2 + Resource Version: 117775637 + UID: c9da1245-3582-4926-b311-c24d75e75003 Spec: Db System: - Availability Domain: OLou:PHX-AD-1 - Compartment Id: ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya + Availability Domain: OLou:AP-MUMBAI-1-AD-1 + Compartment Id: ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a Db Admin Pasword Secret: admin-password - Host Name: host0130 - Shape: VM.Standard2.1 + Db Backup Config: + Domain: subdda0b5eaa.cluster1.oraclevcn.com + Host Name: host1234 + Kms Config: + Shape: VM.Standard2.1 Ssh Public Keys: oci-publickey - Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a - Id: ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa - Oci Config Map: oci-cred + Subnet Id: ocid1.subnet.oc1.ap-mumbai-1.aaaaaaaa5zpzfax66omtbmjwlv4thruyru7focnu7fjcjksujmgwmr6vpbvq + Id: ocid1.dbsystem.oc1.ap-mumbai-1.anrg6ljrabf7htyadgsso7aessztysrwaj5gcl3tp7ce6asijm2japyvmroa + Kms Config: + Oci Config Map: oci-cred-mumbai Oci Secret: oci-privatekey Status: - Availability Domain: OLou:PHX-AD-1 + Availability Domain: OLou:AP-MUMBAI-1-AD-1 Cpu Core Count: 1 Data Storage Percentage: 80 Data Storage Size In G Bs: 256 - Db Edition: ENTERPRISE_EDITION + Db Clone Status: + Db Db Unique Name: + Host Name: + Db Edition: ENTERPRISE_EDITION_HIGH_PERFORMANCE Db Info: - Db Home Id: ocid1.dbhome.oc1.phx.anyhqljr5gy3jhqat52milqwt3gq6lwohhacwg5yi4mtzq7c7hag53lrkugq - Db Name: db0130 - Db Unique Name: db0130_phx1zn + Db Home Id: ocid1.dbhome.oc1.ap-mumbai-1.anrg6ljrqlb5nxiaoqqlaxhx4urdwmefw4il5efzekneuru4bpfv57i7iy6a + Db Name: cdb1 + Db Unique Name: cdb1_tkf_bom Db Workload: OLTP - Id: ocid1.database.oc1.phx.anyhqljrabf7htyackgmsaqjfexoqgrzuuk33ju2q25z2al43tnd5mhhvkra - Display Name: dbsystem20220308221302 - Id: ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa - License Model: LICENSE_INCLUDED + Id: ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyalxin4xpiggjh4nxlta6o6iq56hjrlh4of2cq6c4qgrqa + Display Name: dbsystem1234 + Id: ocid1.dbsystem.oc1.ap-mumbai-1.anrg6ljrabf7htyadgsso7aessztysrwaj5gcl3tp7ce6asijm2japyvmroa + Kms Details Status: + License Model: BRING_YOUR_OWN_LICENSE Network: - Client Subnet: k8test-pubvcn - Domain Name: k8testpubvcn.k8test.oraclevcn.com - Host Name: host0130 - Listener Port: 1521 - Scan Dns Name: host0130-scan.k8testpubvcn.k8test.oraclevcn.com - Vcn Name: k8test - Node Count: 1 - Reco Storage Size In GB: 256 - Shape: VM.Standard2.1 - State: AVAILABLE - Storage Management: ASM - Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a - Time Zone: UTC + Client Subnet: oke-nodesubnet-quick-cluster1-2bebe95db-regional + Domain Name: subdda0b5eaa.cluster1.oraclevcn.com + Host Name: host1234 + Listener Port: 1521 + Scan Dns Name: host1234-scan.subdda0b5eaa.cluster1.oraclevcn.com + Vcn Name: oke-vcn-quick-cluster1-2bebe95db + Node Count: 1 + Pdb Details Status: + Pdb Config Status: + Pdb Name: cdb1_pdb1 + Pdb State: AVAILABLE + Pluggable Database Id: ocid1.pluggabledatabase.oc1.ap-mumbai-1.anrg6ljrabf7htyakgj4wuabus6z5kmalvob6r6b7vivkbsmmh7bjprzbuwa + Reco Storage Size In GB: 256 + Shape: VM.Standard2.1 + State: AVAILABLE + Storage Management: ASM + Subnet Id: ocid1.subnet.oc1.ap-mumbai-1.aaaaaaaa5zpzfax66omtbmjwlv4thruyru7focnu7fjcjksujmgwmr6vpbvq + Time Zone: UTC Work Requests: - Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrxivzvgzel47zuoyke5yk36o7mrgjl27vscd5z3bqptmyh3rxwbqq + Operation Id: ocid1.coreservicesworkrequest.oc1.ap-mumbai-1.abrg6ljrtpnjzjidageolva6ytlzjfb2lqhbbrivm4lsb67xyjzyyke6bt4a + Operation Type: Update Shape + Percent Complete: 100 + Time Accepted: 2024-12-10 08:57:53.547 +0000 UTC + Time Finished: 2024-12-10 09:14:04.572 +0000 UTC + Time Started: 2024-12-10 08:57:57.588 +0000 UTC + Operation Id: ocid1.coreservicesworkrequest.oc1.ap-mumbai-1.abrg6ljrxg7gov22vlcbqbnxrkl7t7xkcfya6w6gvck344jdf5vtqgw5wzgq + Operation Type: Update DB System + Percent Complete: 100 + Time Accepted: 2024-12-10 08:57:43.701 +0000 UTC + Time Finished: 2024-12-10 09:14:22.705 +0000 UTC + Time Started: 2024-12-10 08:57:53.873 +0000 UTC + Operation Id: ocid1.coreservicesworkrequest.oc1.ap-mumbai-1.abrg6ljrk2efvqjda2t7k5iaerahw7wcyz5dq2zev2k55gmq2gvsjkui7hxq Operation Type: Create DB System Percent Complete: 100 - Time Accepted: 2022-03-08 22:13:02.999 +0000 UTC - Time Finished: 2022-03-08 23:11:50.46 +0000 UTC - Time Started: 2022-03-08 22:13:16.995 +0000 UTC - Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrc3fx6kwq4yxerk3ngztdbbngm7w4dnlddcdhxqxjn6e4kcyux5ca - Operation Type: Update Shape + Time Accepted: 2024-12-10 05:19:52.499 +0000 UTC + Time Finished: 2024-12-10 07:59:19.083 +0000 UTC + Time Started: 2024-12-10 05:19:55.747 +0000 UTC + Operation Id: ocid1.coreservicesworkrequest.oc1.ap-mumbai-1.abrg6ljr4qmf6rdtcbrc5p2q7bev3igugtpgfbwc2laht22yyjzr2srrg7vq + Operation Type: Update DB System Percent Complete: 100 - Time Accepted: 2022-03-08 23:33:42.807 +0000 UTC - Time Finished: 2022-03-08 23:46:21.126 +0000 UTC - Time Started: 2022-03-08 23:33:52.109 +0000 UTC - Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljr5sveun3f6k3zuz23py7mm7jncmpq5vwyajbo5ezhc765347defwq + Time Accepted: 2024-12-10 10:57:27.313 +0000 UTC + Time Finished: 2024-12-10 11:15:50.597 +0000 UTC + Time Started: 2024-12-10 10:57:45.242 +0000 UTC + Operation Id: ocid1.coreservicesworkrequest.oc1.ap-mumbai-1.abrg6ljr2vehqv3vgrxr5mrmd6hoqxg2zr6m5eaunv3ip6bcrubcpvhudmia Operation Type: Update Shape Percent Complete: 100 - Time Accepted: 2022-03-09 00:25:03.644 +0000 UTC - Time Finished: 2022-03-09 00:38:59.526 +0000 UTC - Time Started: 2022-03-09 00:25:15.578 +0000 UTC + Time Accepted: 2024-12-10 10:57:44.95 +0000 UTC + Time Finished: 2024-12-10 11:15:40.364 +0000 UTC + Time Started: 2024-12-10 10:57:54.082 +0000 UTC Events: \ No newline at end of file diff --git a/docs/dbcs/provisioning/scale_up_dbcs_system_shape.md b/docs/dbcs/provisioning/scale_up_dbcs_system_shape.md index 8efccb5f..924a8517 100644 --- a/docs/dbcs/provisioning/scale_up_dbcs_system_shape.md +++ b/docs/dbcs/provisioning/scale_up_dbcs_system_shape.md @@ -1,26 +1,26 @@ -# Scale UP the shape of an existing DBCS System +# Scale UP the shape of an existing OBDS System -In this use case, an existing OCI DBCS system deployed earlier is scaled up for its shape using Oracle DB Operator DBCS controller. Its a 2 Step operation. +In this use case, an existing OCI OBDS system deployed earlier is scaled up for its shape using Oracle DB Operator OBDS controller. Its a 2 Step operation. -In order to scale up an existing DBCS system, the steps will be: +In order to scale up an existing OBDS system, the steps will be: -1. Bind the existing DBCS System to DBCS Controller. +1. Bind the existing OBDS System to OBDS Controller. 2. Apply the change to scale up its shape. **NOTE:** We are assuming that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-deploy-a-dbcs-system-using-oracle-db-operator-dbcs-controller) steps to create the configmap and the secrets required during the deployment. -This example uses `scale_up_dbcs_system_shape.yaml` to scale up a Single Instance DBCS VMDB using Oracle DB Operator DBCS Controller with: +This example uses `scale_up_dbcs_system_shape.yaml` to scale up a Single Instance OBDS VMDB using Oracle DB Operator OBDS Controller with: -- OCID of existing VMDB as `ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa` +- OCID of existing VMDB as `ocid1.dbsystem.oc1.ap-mumbai-1.anrg6ljrabf7htyadgsso7aessztysrwaj5gcl3tp7ce6asijm2japyvmroa` - OCI Configmap as `oci-cred` - OCI Secret as `oci-privatekey` -- Availability Domain for the DBCS VMDB as `OLou:PHX-AD-1` -- Compartment OCID as `ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya` +- Availability Domain for the OBDS VMDB as `OLou:AP-MUMBAI-1-AD-1` +- Compartment OCID as `ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72` - Database Admin Credential as `admin-password` -- Database Hostname Prefix as `host0130` -- Oracle VMDB Shape as `VM.Standard2.2` -- SSH Public key for the DBCS system being deployed as `oci-publickey` -- OCID of the Subnet as `ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a` +- Database Hostname Prefix as `host1234` +- Oracle VMDB target Shape as `VM.Standard2.2` +- SSH Public key for the OBDS system being deployed as `oci-publickey` +- OCID of the Subnet as `ocid1.subnet.oc1.ap-mumbai-1.aaaaaaaa5zpzfax66omtbmjwlv4thruyru7focnu7fjcjksujmgwmr6vpbvq` **NOTE:** For the details of the parameters to be used in the .yaml file, please refer [here](./dbcs_controller_parameters.md). @@ -28,18 +28,18 @@ Use the file: [scale_up_dbcs_system_shape.yaml](./scale_up_dbcs_system_shape.yam 1. Deploy the .yaml file: ```sh -[root@docker-test-server DBCS]# kubectl apply -f scale_up_dbcs_system_shape.yaml +[root@docker-test-server OBDS]# kubectl apply -f scale_up_dbcs_system_shape.yaml dbcssystem.database.oracle.com/dbcssystem-existing configured ``` -2. Monitor the Oracle DB Operator Pod `pod/oracle-database-operator-controller-manager-665874bd57-g2cgw` for the progress of the DBCS VMDB Scale up. +2. Monitor the Oracle DB Operator Pod `pod/oracle-database-operator-controller-manager-665874bd57-g2cgw` for the progress of the OBDS VMDB Scale up. NOTE: Check the DB Operator Pod name in your environment. ``` -[root@docker-test-server DBCS]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-g2cgw -n oracle-database-operator-system +[root@docker-test-server OBDS]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-g2cgw -n oracle-database-operator-system ``` ## Sample Output -[Here](./scale_up_dbcs_system_shape_sample_output.log) is the sample output for scaling up the shape of an existing DBCS System deployed in OCI using Oracle DB Operator DBCS Controller. +[Here](./scale_up_dbcs_system_shape_sample_output.log) is the sample output for scaling up the shape of an existing OBDS System deployed in OCI using Oracle DB Operator OBDS Controller. diff --git a/docs/dbcs/provisioning/scale_up_dbcs_system_shape.yaml b/docs/dbcs/provisioning/scale_up_dbcs_system_shape.yaml index d1c2b95d..0be84c53 100644 --- a/docs/dbcs/provisioning/scale_up_dbcs_system_shape.yaml +++ b/docs/dbcs/provisioning/scale_up_dbcs_system_shape.yaml @@ -1,17 +1,18 @@ -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: DbcsSystem metadata: name: dbcssystem-existing spec: - id: "ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa" + id: "ocid1.dbsystem.oc1.ap-mumbai-1.anrg6ljrabf7htyadgsso7aessztysrwaj5gcl3tp7ce6asijm2japyvmroa" ociConfigMap: "oci-cred" ociSecret: "oci-privatekey" dbSystem: - availabilityDomain: "OLou:PHX-AD-1" - compartmentId: "ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya" - dbAdminPaswordSecret: "admin-password" - hostName: "host0130" + availabilityDomain: "OLou:AP-MUMBAI-1-AD-1" + compartmentId: "ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a" + dbAdminPasswordSecret: "admin-password" + hostName: "host1234" shape: "VM.Standard2.2" + domain: "subdda0b5eaa.cluster1.oraclevcn.com" sshPublicKeys: - - "oci-publickey" - subnetId: "ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a" + - "oci-publickey" + subnetId: "ocid1.subnet.oc1.ap-mumbai-1.aaaaaaaa5zpzfax66omtbmjwlv4thruyru7focnu7fjcjksujmgwmr6vpbvq" diff --git a/docs/dbcs/provisioning/scale_up_storage.md b/docs/dbcs/provisioning/scale_up_storage.md index 64514025..ff16cbf9 100644 --- a/docs/dbcs/provisioning/scale_up_storage.md +++ b/docs/dbcs/provisioning/scale_up_storage.md @@ -1,45 +1,45 @@ -# Scale UP the storage of an existing DBCS System +# Scale UP the storage of an existing OBDS System -In this use case, an existing OCI DBCS system deployed earlier is scaled up for its storage using Oracle DB Operator DBCS controller. Its a 2 Step operation. +In this use case, an existing OCI OBDS system deployed earlier is scaled up for its storage using Oracle DB Operator OBDS controller. Its a 2 Step operation. -In order to scale up storage of an existing DBCS system, the steps will be: +In order to scale up storage of an existing OBDS system, the steps will be: -1. Bind the existing DBCS System to DBCS Controller. +1. Bind the existing OBDS System to OBDS Controller. 2. Apply the change to scale up its storage. **NOTE** We are assuming that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-deploy-a-dbcs-system-using-oracle-db-operator-dbcs-controller) steps to create the configmap and the secrets required during the deployment. -This example uses `scale_up_storage.yaml` to scale up storage of an existing Single Instance DBCS VMDB using Oracle DB Operator DBCS Controller with: +This example uses `scale_up_storage.yaml` to scale up storage of an existing Single Instance OBDS VMDB using Oracle DB Operator OBDS Controller with: -- OCID of existing VMDB as `ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa` +- OCID of existing VMDB as `ocid1.dbsystem.oc1.ap-mumbai-1.anrg6ljrabf7htyadgsso7aessztysrwaj5gcl3tp7ce6asijm2japyvmroa` - OCI Configmap as `oci-cred` - OCI Secret as `oci-privatekey` -- Availability Domain for the DBCS VMDB as `OLou:PHX-AD-1` -- Compartment OCID as `ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya` +- Availability Domain for the OBDS VMDB as `OLou:AP-MUMBAI-1-AD-1` +- Compartment OCID as `ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a` - Database Admin Credential as `admin-password` -- Database Hostname Prefix as `host0130` +- Database Hostname Prefix as `host1234` - Target Data Storage Size in GBs as `512` - Oracle VMDB Shape as `VM.Standard2.1` -- SSH Public key for the DBCS system being deployed as `oci-publickey` -- OCID of the Subnet as `ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a` +- SSH Public key for the OBDS system being deployed as `oci-publickey` +- OCID of the Subnet as `ocid1.subnet.oc1.ap-mumbai-1.aaaaaaaa5zpzfax66omtbmjwlv4thruyru7focnu7fjcjksujmgwmr6vpbvq` Use the file: [scale_up_storage.yaml](./scale_up_storage.yaml) for this use case as below: 1. Deploy the .yaml file: ```sh -[root@test-server DBCS]# kubectl apply -f scale_storage.yaml +[root@test-server OBDS]# kubectl apply -f scale_storage.yaml dbcssystem.database.oracle.com/dbcssystem-existing configured ``` -2. Monitor the Oracle DB Operator Pod `pod/oracle-database-operator-controller-manager-665874bd57-g2cgw` for the progress of the DBCS VMDB Scale up. +2. Monitor the Oracle DB Operator Pod `pod/oracle-database-operator-controller-manager-665874bd57-g2cgw` for the progress of the OBDS VMDB Scale up. NOTE: Check the DB Operator Pod name in your environment. ``` -[root@docker-test-server DBCS]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-g2cgw -n oracle-database-operator-system +[root@docker-test-server OBDS]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-g2cgw -n oracle-database-operator-system ``` ## Sample Output -[Here](./scale_up_storage_sample_output.log) is the sample output for scaling up the storage of an existing DBCS System deployed in OCI using Oracle DB Operator DBCS Controller with minimal parameters. +[Here](./scale_up_storage_sample_output.log) is the sample output for scaling up the storage of an existing OBDS System deployed in OCI using Oracle DB Operator OBDS Controller with minimal parameters. diff --git a/docs/dbcs/provisioning/scale_up_storage.yaml b/docs/dbcs/provisioning/scale_up_storage.yaml index 34e64b5e..a2977157 100644 --- a/docs/dbcs/provisioning/scale_up_storage.yaml +++ b/docs/dbcs/provisioning/scale_up_storage.yaml @@ -1,18 +1,19 @@ -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: DbcsSystem metadata: name: dbcssystem-existing spec: - id: "ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa" + id: "ocid1.dbsystem.oc1.ap-mumbai-1.anrg6ljrabf7htyadgsso7aessztysrwaj5gcl3tp7ce6asijm2japyvmroa" ociConfigMap: "oci-cred" ociSecret: "oci-privatekey" dbSystem: - availabilityDomain: "OLou:PHX-AD-1" - compartmentId: "ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya" - dbAdminPaswordSecret: "admin-password" - hostName: "host0130" - initialDataStorageSizeInGB: 512 + availabilityDomain: "OLou:AP-MUMBAI-1-AD-1" + compartmentId: "ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a" + dbAdminPasswordSecret: "admin-password" + hostName: "host1234" shape: "VM.Standard2.1" + domain: "subdda0b5eaa.cluster1.oraclevcn.com" + initialDataStorageSizeInGB: 512 sshPublicKeys: - - "oci-publickey" - subnetId: "ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a" + - "oci-publickey" + subnetId: "ocid1.subnet.oc1.ap-mumbai-1.aaaaaaaa5zpzfax66omtbmjwlv4thruyru7focnu7fjcjksujmgwmr6vpbvq" \ No newline at end of file diff --git a/docs/dbcs/provisioning/scale_up_storage_sample_output.log b/docs/dbcs/provisioning/scale_up_storage_sample_output.log index 667667b8..e703391e 100644 --- a/docs/dbcs/provisioning/scale_up_storage_sample_output.log +++ b/docs/dbcs/provisioning/scale_up_storage_sample_output.log @@ -3,387 +3,111 @@ Name: dbcssystem-existing Namespace: default Labels: Annotations: lastSuccessfulSpec: - {"dbSystem":{"compartmentId":"ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya","availabilityDomain":"O... -API Version: database.oracle.com/v1alpha1 + {"dbSystem":{"compartmentId":"ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a","availabilityDomain":"O... +API Version: database.oracle.com/v4 Kind: DbcsSystem Metadata: - Creation Timestamp: 2022-03-08T23:27:48Z + Creation Timestamp: 2024-12-10T10:54:17Z Generation: 3 - Managed Fields: - API Version: database.oracle.com/v1alpha1 - Fields Type: FieldsV1 - fieldsV1: - f:metadata: - f:annotations: - .: - f:kubectl.kubernetes.io/last-applied-configuration: - f:spec: - .: - f:dbSystem: - .: - f:availabilityDomain: - f:compartmentId: - f:dbAdminPaswordSecret: - f:hostName: - f:shape: - f:sshPublicKeys: - f:subnetId: - f:id: - f:ociConfigMap: - f:ociSecret: - Manager: kubectl-client-side-apply - Operation: Update - Time: 2022-03-08T23:32:50Z - API Version: database.oracle.com/v1alpha1 - Fields Type: FieldsV1 - fieldsV1: - f:metadata: - f:annotations: - f:lastSuccessfulSpec: - f:spec: - f:dbSystem: - f:cpuCoreCount: - f:dbBackupConfig: - f:dbEdition: - f:dbName: - f:dbUniqueName: - f:dbVersion: - f:diskRedundancy: - f:displayName: - f:faultDomains: - f:nodeCount: - f:status: - .: - f:availabilityDomain: - f:cpuCoreCount: - f:dataStoragePercentage: - f:dataStorageSizeInGBs: - f:dbEdition: - f:dbInfo: - f:displayName: - f:id: - f:licenseModel: - f:network: - .: - f:clientSubnet: - f:domainName: - f:hostName: - f:listenerPort: - f:scanDnsName: - f:vcnName: - f:nodeCount: - f:recoStorageSizeInGB: - f:shape: - f:state: - f:storageManagement: - f:subnetId: - f:timeZone: - f:workRequests: - Manager: manager - Operation: Update - Time: 2022-03-08T23:32:55Z - Resource Version: 55214174 - UID: 96d7bc49-33e9-42cc-8dd0-ada9a5a4c7e5 + Resource Version: 117788129 + UID: c9da1245-3582-4926-b311-c24d75e75003 Spec: Db System: - Availability Domain: OLou:PHX-AD-1 - Compartment Id: ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya + Availability Domain: OLou:AP-MUMBAI-1-AD-1 + Compartment Id: ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a Db Admin Pasword Secret: admin-password - Host Name: host0130 - Shape: VM.Standard2.1 + Db Backup Config: + Domain: subdda0b5eaa.cluster1.oraclevcn.com + Host Name: host1234 + Initial Data Storage Size In GB: 512 + Kms Config: + Shape: VM.Standard2.1 Ssh Public Keys: oci-publickey - Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a - Id: ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa - Oci Config Map: oci-cred + Subnet Id: ocid1.subnet.oc1.ap-mumbai-1.aaaaaaaa5zpzfax66omtbmjwlv4thruyru7focnu7fjcjksujmgwmr6vpbvq + Id: ocid1.dbsystem.oc1.ap-mumbai-1.anrg6ljrabf7htyadgsso7aessztysrwaj5gcl3tp7ce6asijm2japyvmroa + Kms Config: + Oci Config Map: oci-cred-mumbai Oci Secret: oci-privatekey Status: - Availability Domain: OLou:PHX-AD-1 + Availability Domain: OLou:AP-MUMBAI-1-AD-1 Cpu Core Count: 1 Data Storage Percentage: 80 - Data Storage Size In G Bs: 256 - Db Edition: ENTERPRISE_EDITION + Data Storage Size In G Bs: 512 + Db Clone Status: + Db Db Unique Name: + Host Name: + Db Edition: ENTERPRISE_EDITION_HIGH_PERFORMANCE Db Info: - Db Home Id: ocid1.dbhome.oc1.phx.anyhqljr5gy3jhqat52milqwt3gq6lwohhacwg5yi4mtzq7c7hag53lrkugq - Db Name: db0130 - Db Unique Name: db0130_phx1zn + Db Home Id: ocid1.dbhome.oc1.ap-mumbai-1.anrg6ljrqlb5nxiaoqqlaxhx4urdwmefw4il5efzekneuru4bpfv57i7iy6a + Db Name: cdb1 + Db Unique Name: cdb1_tkf_bom Db Workload: OLTP - Id: ocid1.database.oc1.phx.anyhqljrabf7htyackgmsaqjfexoqgrzuuk33ju2q25z2al43tnd5mhhvkra - Display Name: dbsystem20220308221302 - Id: ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa - License Model: LICENSE_INCLUDED + Id: ocid1.database.oc1.ap-mumbai-1.anrg6ljrabf7htyalxin4xpiggjh4nxlta6o6iq56hjrlh4of2cq6c4qgrqa + Display Name: dbsystem1234 + Id: ocid1.dbsystem.oc1.ap-mumbai-1.anrg6ljrabf7htyadgsso7aessztysrwaj5gcl3tp7ce6asijm2japyvmroa + Kms Details Status: + License Model: BRING_YOUR_OWN_LICENSE Network: - Client Subnet: k8test-pubvcn - Domain Name: k8testpubvcn.k8test.oraclevcn.com - Host Name: host0130 - Listener Port: 1521 - Scan Dns Name: host0130-scan.k8testpubvcn.k8test.oraclevcn.com - Vcn Name: k8test - Node Count: 1 - Reco Storage Size In GB: 256 - Shape: VM.Standard2.1 - State: AVAILABLE - Storage Management: ASM - Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a - Time Zone: UTC + Client Subnet: oke-nodesubnet-quick-cluster1-2bebe95db-regional + Domain Name: subdda0b5eaa.cluster1.oraclevcn.com + Host Name: host1234 + Listener Port: 1521 + Scan Dns Name: host1234-scan.subdda0b5eaa.cluster1.oraclevcn.com + Vcn Name: oke-vcn-quick-cluster1-2bebe95db + Node Count: 1 + Pdb Details Status: + Pdb Config Status: + Pdb Name: cdb1_pdb1 + Pdb State: AVAILABLE + Pluggable Database Id: ocid1.pluggabledatabase.oc1.ap-mumbai-1.anrg6ljrabf7htyakgj4wuabus6z5kmalvob6r6b7vivkbsmmh7bjprzbuwa + Reco Storage Size In GB: 256 + Shape: VM.Standard2.1 + State: AVAILABLE + Storage Management: ASM + Subnet Id: ocid1.subnet.oc1.ap-mumbai-1.aaaaaaaa5zpzfax66omtbmjwlv4thruyru7focnu7fjcjksujmgwmr6vpbvq + Time Zone: UTC Work Requests: - Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrxivzvgzel47zuoyke5yk36o7mrgjl27vscd5z3bqptmyh3rxwbqq - Operation Type: Create DB System - Percent Complete: 100 - Time Accepted: 2022-03-08 22:13:02.999 +0000 UTC - Time Finished: 2022-03-08 23:11:50.46 +0000 UTC - Time Started: 2022-03-08 22:13:16.995 +0000 UTC - Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrc3fx6kwq4yxerk3ngztdbbngm7w4dnlddcdhxqxjn6e4kcyux5ca + Operation Id: ocid1.coreservicesworkrequest.oc1.ap-mumbai-1.abrg6ljrtpnjzjidageolva6ytlzjfb2lqhbbrivm4lsb67xyjzyyke6bt4a Operation Type: Update Shape Percent Complete: 100 - Time Accepted: 2022-03-08 23:33:42.807 +0000 UTC - Time Finished: 2022-03-08 23:46:21.126 +0000 UTC - Time Started: 2022-03-08 23:33:52.109 +0000 UTC - Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljr5sveun3f6k3zuz23py7mm7jncmpq5vwyajbo5ezhc765347defwq - Operation Type: Update Shape + Time Accepted: 2024-12-10 08:57:53.547 +0000 UTC + Time Finished: 2024-12-10 09:14:04.572 +0000 UTC + Time Started: 2024-12-10 08:57:57.588 +0000 UTC + Operation Id: ocid1.coreservicesworkrequest.oc1.ap-mumbai-1.abrg6ljrxg7gov22vlcbqbnxrkl7t7xkcfya6w6gvck344jdf5vtqgw5wzgq + Operation Type: Update DB System Percent Complete: 100 - Time Accepted: 2022-03-09 00:25:03.644 +0000 UTC - Time Finished: 2022-03-09 00:38:59.526 +0000 UTC - Time Started: 2022-03-09 00:25:15.578 +0000 UTC -Events: -[root@docker-test-server test]# - - - -[root@docker-test-server test]# cat scale_up_storage.yaml -apiVersion: database.oracle.com/v1alpha1 -kind: DbcsSystem -metadata: - name: dbcssystem-existing -spec: - id: "ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa" - ociConfigMap: "oci-cred" - ociSecret: "oci-privatekey" - dbSystem: - availabilityDomain: "OLou:PHX-AD-1" - compartmentId: "ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya" - dbAdminPaswordSecret: "admin-password" - hostName: "host0130" - initialDataStorageSizeInGB: 512 - shape: "VM.Standard2.1" - sshPublicKeys: - - "oci-publickey" - subnetId: "ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a" -[root@docker-test-server test]# -[root@docker-test-server test]# kubectl apply -f scale_up_storage.yaml -dbcssystem.database.oracle.com/dbcssystem-existing configured -[root@docker-test-server test]# - - - -[root@docker-test-server test]# kubectl get ns - -kubectl get allNAME STATUS AGE -cert-manager Active 13d -default Active 139d -kube-node-lease Active 139d -kube-public Active 139d -kube-system Active 139d -oracle-database-operator-system Active 13d -shns Active 88d -[root@docker-test-server test]# -[root@docker-test-server test]# kubectl get all -n oracle-database-operator-system -NAME READY STATUS RESTARTS AGE -pod/oracle-database-operator-controller-manager-665874bd57-dlhls 1/1 Running 3 13d -pod/oracle-database-operator-controller-manager-665874bd57-g2cgw 1/1 Running 3 13d -pod/oracle-database-operator-controller-manager-665874bd57-q42f8 1/1 Running 4 13d - -NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE -service/oracle-database-operator-controller-manager-metrics-service ClusterIP 10.96.130.124 8443/TCP 13d -service/oracle-database-operator-webhook-service ClusterIP 10.96.4.104 443/TCP 13d - -NAME READY UP-TO-DATE AVAILABLE AGE -deployment.apps/oracle-database-operator-controller-manager 3/3 3 3 13d - -NAME DESIRED CURRENT READY AGE -replicaset.apps/oracle-database-operator-controller-manager-665874bd57 3 3 3 13d -[root@docker-test-server test]# - - -[root@docker-test-server test]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-dlhls -n oracle-database-operator-system -. -. -2022-03-09T00:48:11.373Z INFO controller-runtime.manager.controller.dbcssystem OCI provider configured succesfully {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} -2022-03-09T00:48:15.961Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} -2022-03-09T00:49:16.273Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} -2022-03-09T00:50:16.557Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} -2022-03-09T00:51:16.910Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} -2022-03-09T00:52:17.277Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} -2022-03-09T00:53:17.600Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} -2022-03-09T00:54:18.189Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} -2022-03-09T00:55:18.506Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} -2022-03-09T00:56:18.862Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} -2022-03-09T00:57:19.180Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} -2022-03-09T00:58:19.544Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} -2022-03-09T00:59:19.870Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} -2022-03-09T01:00:20.230Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} -2022-03-09T01:01:20.663Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} -2022-03-09T01:02:21.303Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} -2022-03-09T01:03:21.690Z INFO controller-runtime.manager.controller.dbcssystem DB System current state is still:UPDATING. Sleeping for 60 seconds. {"reconciler group": "database.oracle.com", "reconciler kind": "DbcsSystem", "name": "dbcssystem-existing", "namespace": "default"} - - - - - -[root@docker-test-server test]# kubectl describe dbcssystems.database.oracle.com dbcssystem-existing -Name: dbcssystem-existing -Namespace: default -Labels: -Annotations: lastSuccessfulSpec: - {"dbSystem":{"compartmentId":"ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya","availabilityDomain":"O... -API Version: database.oracle.com/v1alpha1 -Kind: DbcsSystem -Metadata: - Creation Timestamp: 2022-03-08T23:27:48Z - Generation: 4 - Managed Fields: - API Version: database.oracle.com/v1alpha1 - Fields Type: FieldsV1 - fieldsV1: - f:metadata: - f:annotations: - f:lastSuccessfulSpec: - f:spec: - f:dbSystem: - f:cpuCoreCount: - f:dbBackupConfig: - f:dbEdition: - f:dbName: - f:dbUniqueName: - f:dbVersion: - f:diskRedundancy: - f:displayName: - f:faultDomains: - f:nodeCount: - f:status: - .: - f:availabilityDomain: - f:cpuCoreCount: - f:dataStoragePercentage: - f:dataStorageSizeInGBs: - f:dbEdition: - f:dbInfo: - f:displayName: - f:id: - f:licenseModel: - f:network: - .: - f:clientSubnet: - f:domainName: - f:hostName: - f:listenerPort: - f:scanDnsName: - f:vcnName: - f:nodeCount: - f:recoStorageSizeInGB: - f:shape: - f:state: - f:storageManagement: - f:subnetId: - f:timeZone: - f:workRequests: - Manager: manager - Operation: Update - Time: 2022-03-08T23:32:55Z - API Version: database.oracle.com/v1alpha1 - Fields Type: FieldsV1 - fieldsV1: - f:metadata: - f:annotations: - .: - f:kubectl.kubernetes.io/last-applied-configuration: - f:spec: - .: - f:dbSystem: - .: - f:availabilityDomain: - f:compartmentId: - f:dbAdminPaswordSecret: - f:hostName: - f:initialDataStorageSizeInGB: - f:shape: - f:sshPublicKeys: - f:subnetId: - f:id: - f:ociConfigMap: - f:ociSecret: - Manager: kubectl-client-side-apply - Operation: Update - Time: 2022-03-09T00:48:11Z - Resource Version: 55222013 - UID: 96d7bc49-33e9-42cc-8dd0-ada9a5a4c7e5 -Spec: - Db System: - Availability Domain: OLou:PHX-AD-1 - Compartment Id: ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya - Db Admin Pasword Secret: admin-password - Host Name: host0130 - Initial Data Storage Size In GB: 512 - Shape: VM.Standard2.1 - Ssh Public Keys: - oci-publickey - Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a - Id: ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa - Oci Config Map: oci-cred - Oci Secret: oci-privatekey -Status: - Availability Domain: OLou:PHX-AD-1 - Cpu Core Count: 1 - Data Storage Percentage: 80 - Data Storage Size In G Bs: 512 - Db Edition: ENTERPRISE_EDITION - Db Info: - Db Home Id: ocid1.dbhome.oc1.phx.anyhqljr5gy3jhqat52milqwt3gq6lwohhacwg5yi4mtzq7c7hag53lrkugq - Db Name: db0130 - Db Unique Name: db0130_phx1zn - Db Workload: OLTP - Id: ocid1.database.oc1.phx.anyhqljrabf7htyackgmsaqjfexoqgrzuuk33ju2q25z2al43tnd5mhhvkra - Display Name: dbsystem20220308221302 - Id: ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa - License Model: LICENSE_INCLUDED - Network: - Client Subnet: k8test-pubvcn - Domain Name: k8testpubvcn.k8test.oraclevcn.com - Host Name: host0130 - Listener Port: 1521 - Scan Dns Name: host0130-scan.k8testpubvcn.k8test.oraclevcn.com - Vcn Name: k8test - Node Count: 1 - Reco Storage Size In GB: 256 - Shape: VM.Standard2.1 - State: AVAILABLE - Storage Management: ASM - Subnet Id: ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a - Time Zone: UTC - Work Requests: - Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrxivzvgzel47zuoyke5yk36o7mrgjl27vscd5z3bqptmyh3rxwbqq + Time Accepted: 2024-12-10 08:57:43.701 +0000 UTC + Time Finished: 2024-12-10 09:14:22.705 +0000 UTC + Time Started: 2024-12-10 08:57:53.873 +0000 UTC + Operation Id: ocid1.coreservicesworkrequest.oc1.ap-mumbai-1.abrg6ljrk2efvqjda2t7k5iaerahw7wcyz5dq2zev2k55gmq2gvsjkui7hxq Operation Type: Create DB System Percent Complete: 100 - Time Accepted: 2022-03-08 22:13:02.999 +0000 UTC - Time Finished: 2022-03-08 23:11:50.46 +0000 UTC - Time Started: 2022-03-08 22:13:16.995 +0000 UTC - Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrc3fx6kwq4yxerk3ngztdbbngm7w4dnlddcdhxqxjn6e4kcyux5ca - Operation Type: Update Shape + Time Accepted: 2024-12-10 05:19:52.499 +0000 UTC + Time Finished: 2024-12-10 07:59:19.083 +0000 UTC + Time Started: 2024-12-10 05:19:55.747 +0000 UTC + Operation Id: ocid1.coreservicesworkrequest.oc1.ap-mumbai-1.abrg6ljr4qmf6rdtcbrc5p2q7bev3igugtpgfbwc2laht22yyjzr2srrg7vq + Operation Type: Update DB System Percent Complete: 100 - Time Accepted: 2022-03-08 23:33:42.807 +0000 UTC - Time Finished: 2022-03-08 23:46:21.126 +0000 UTC - Time Started: 2022-03-08 23:33:52.109 +0000 UTC - Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljr5sveun3f6k3zuz23py7mm7jncmpq5vwyajbo5ezhc765347defwq + Time Accepted: 2024-12-10 10:57:27.313 +0000 UTC + Time Finished: 2024-12-10 11:15:50.597 +0000 UTC + Time Started: 2024-12-10 10:57:45.242 +0000 UTC + Operation Id: ocid1.coreservicesworkrequest.oc1.ap-mumbai-1.abrg6ljr2vehqv3vgrxr5mrmd6hoqxg2zr6m5eaunv3ip6bcrubcpvhudmia Operation Type: Update Shape Percent Complete: 100 - Time Accepted: 2022-03-09 00:25:03.644 +0000 UTC - Time Finished: 2022-03-09 00:38:59.526 +0000 UTC - Time Started: 2022-03-09 00:25:15.578 +0000 UTC - Operation Id: ocid1.coreservicesworkrequest.oc1.phx.abyhqljrbaqah6qktukvdlnx66fp2hlevegryfuppsshkqemfcdjtwfwaq3q + Time Accepted: 2024-12-10 10:57:44.95 +0000 UTC + Time Finished: 2024-12-10 11:15:40.364 +0000 UTC + Time Started: 2024-12-10 10:57:54.082 +0000 UTC + Operation Id: ocid1.coreservicesworkrequest.oc1.ap-mumbai-1.abrg6ljr36bt7ot5oq3otch4bu2axn3azkicot4zuwgwmxeupxr4siisydja + Operation Type: Scale Storage + Percent Complete: 100 + Time Accepted: 2024-12-10 11:44:49.369 +0000 UTC + Time Finished: 2024-12-10 11:58:45.01 +0000 UTC + Time Started: 2024-12-10 11:44:55.544 +0000 UTC + Operation Id: ocid1.coreservicesworkrequest.oc1.ap-mumbai-1.abrg6ljrxdpmmaipuqke5yx3szyfnf2zwkfptz3jevlq3coicecfjihnm4kq Operation Type: Scale Storage Percent Complete: 100 - Time Accepted: 2022-03-09 00:48:54.849 +0000 UTC - Time Finished: 2022-03-09 01:03:10.885 +0000 UTC - Time Started: 2022-03-09 00:49:05.911 +0000 UTC -Events: -[root@docker-test-server test]# \ No newline at end of file + Time Accepted: 2024-12-10 11:44:55.255 +0000 UTC + Time Finished: 2024-12-10 11:58:25.229 +0000 UTC + Time Started: 2024-12-10 11:44:57.743 +0000 UTC +Events: \ No newline at end of file diff --git a/docs/dbcs/provisioning/terminate_dbcs_system.md b/docs/dbcs/provisioning/terminate_dbcs_system.md index 071cda30..f3b19cbc 100644 --- a/docs/dbcs/provisioning/terminate_dbcs_system.md +++ b/docs/dbcs/provisioning/terminate_dbcs_system.md @@ -1,15 +1,15 @@ -# Terminate an existing DBCS System +# Terminate an existing Oracle Base Database System (OBDS) -In this use case, an existing OCI DBCS system deployed earlier is terminated using Oracle DB Operator DBCS controller. Its a 2 Step operation. +In this use case, an existing OCI OBDS system deployed earlier is terminated using Oracle DB Operator OBDS controller. Its a 2 Step operation. -In order to terminate an existing DBCS system, the steps will be: +In order to terminate an existing OBDS system, the steps will be: -1. Bind the existing DBCS System to DBCS Controller. -2. Apply the change to terminate this DBCS System. +1. Bind the existing OBDS System to OBDS Controller. +2. Apply the change to terminate this OBDS System. **NOTE** We are assuming that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-deploy-a-dbcs-system-using-oracle-db-operator-dbcs-controller) steps to create the configmap and the secrets required during the deployment. -This example uses `terminate_dbcs_system.yaml` to terminated a Single Instance DBCS VMDB using Oracle DB Operator DBCS Controller with: +This example uses `terminate_dbcs_system.yaml` to terminated a Single Instance OBDS VMDB using Oracle DB Operator OBDS Controller with: - OCID of existing VMDB as `ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa` - OCI Configmap as `oci-cred` @@ -21,26 +21,26 @@ Use the file: [terminate_dbcs_system.yaml](./terminate_dbcs_system.yaml) for thi 1. Deploy the .yaml file: ```sh -[root@test-server DBCS]# kubectl apply -f terminate_dbcs_system.yaml +[root@test-server OBDS]# kubectl apply -f terminate_dbcs_system.yaml dbcssystem.database.oracle.com/dbcssystem-terminate created -[root@test-server DBCS]# kubectl delete -f terminate_dbcs_system.yaml +[root@test-server OBDS]# kubectl delete -f terminate_dbcs_system.yaml dbcssystem.database.oracle.com "dbcssystem-terminate" deleted ``` 2. Check the logs of Oracle DB Operator Pod `pod/oracle-database-operator-controller-manager-665874bd57-g2cgw` for an update on the terminate operation been accepted. ``` -[root@test-server DBCS]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-g2cgw -n oracle-database-operator-system +[root@test-server OBDS]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-g2cgw -n oracle-database-operator-system ``` -3. Check and confirm if the existing OCI DBCS system is NO longer available after sometime because of termination: +3. Check and confirm if the existing OCI OBDS system is NO longer available after sometime because of termination: ``` -[root@test-server DBCS]# kubectl describe dbcssystems.database.oracle.com dbcssystem-terminate +[root@test-server OBDS]# kubectl describe dbcssystems.database.oracle.com dbcssystem-terminate ``` ## Sample Output -[Here](./terminate_dbcs_system_sample_output.log) is the sample output for terminating an existing DBCS System deployed in OCI using Oracle DB Operator DBCS Controller with minimal parameters. +[Here](./terminate_dbcs_system_sample_output.log) is the sample output for terminating an existing OBDS System deployed in OCI using Oracle DB Operator OBDS Controller with minimal parameters. diff --git a/docs/dbcs/provisioning/terminate_dbcs_system.yaml b/docs/dbcs/provisioning/terminate_dbcs_system.yaml index 075ce54e..a4a2f105 100644 --- a/docs/dbcs/provisioning/terminate_dbcs_system.yaml +++ b/docs/dbcs/provisioning/terminate_dbcs_system.yaml @@ -1,4 +1,4 @@ -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: DbcsSystem metadata: name: dbcssystem-terminate diff --git a/docs/dbcs/provisioning/terminate_dbcs_system_sample_output.log b/docs/dbcs/provisioning/terminate_dbcs_system_sample_output.log index ff8afc96..383f823c 100644 --- a/docs/dbcs/provisioning/terminate_dbcs_system_sample_output.log +++ b/docs/dbcs/provisioning/terminate_dbcs_system_sample_output.log @@ -4,13 +4,13 @@ Namespace: default Labels: Annotations: lastSuccessfulSpec: {"dbSystem":{"compartmentId":"ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya","availabilityDomain":"O... -API Version: database.oracle.com/v1alpha1 +API Version: database.oracle.com/v4 Kind: DbcsSystem Metadata: Creation Timestamp: 2022-03-08T23:27:48Z Generation: 5 Managed Fields: - API Version: database.oracle.com/v1alpha1 + API Version: database.oracle.com/v4 Fields Type: FieldsV1 fieldsV1: f:metadata: @@ -58,7 +58,7 @@ Metadata: Manager: manager Operation: Update Time: 2022-03-08T23:32:55Z - API Version: database.oracle.com/v1alpha1 + API Version: database.oracle.com/v4 Fields Type: FieldsV1 fieldsV1: f:metadata: @@ -164,7 +164,7 @@ Events: [root@docker-test-server test]# cat terminate_dbcs_system.yaml -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: DbcsSystem metadata: name: dbcssystem-terminate diff --git a/docs/dbcs/provisioning/update_license.md b/docs/dbcs/provisioning/update_license.md index 7c7c43b7..6f32c31b 100644 --- a/docs/dbcs/provisioning/update_license.md +++ b/docs/dbcs/provisioning/update_license.md @@ -1,27 +1,27 @@ -# Update License type of an existing DBCS System +# Update License type of an existing OBDS System -In this use case, the license type of an existing OCI DBCS system deployed earlier is changed from `License Included` to `Bring your own license` using Oracle DB Operator DBCS controller. Its a 2 Step operation. +In this use case, the license type of an existing OCI OBDS system deployed earlier is changed from `License Included` to `Bring your own license` using Oracle DB Operator OBDS controller. Its a 2 Step operation. -In order to update the license type an existing DBCS system, the steps will be: +In order to update the license type an existing OBDS system, the steps will be: -1. Bind the existing DBCS System to DBCS Controller. +1. Bind the existing OBDS System to OBDS Controller. 2. Apply the change to change its license type. **NOTE** We are assuming that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-deploy-a-dbcs-system-using-oracle-db-operator-dbcs-controller) steps to create the configmap and the secrets required during the deployment. -This example uses `update_license.yaml` to change the license type of a Single Instance DBCS VMDB using Oracle DB Operator DBCS Controller with: +This example uses `update_license.yaml` to change the license type of a Single Instance OBDS VMDB using Oracle DB Operator OBDS Controller with: -- OCID of existing VMDB as `ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa` +- OCID of existing VMDB as `ocid1.dbsystem.oc1.ap-mumbai-1.anrg6ljrabf7htyadgsso7aessztysrwaj5gcl3tp7ce6asijm2japyvmroa` - OCI Configmap as `oci-cred` - OCI Secret as `oci-privatekey` -- Availability Domain for the DBCS VMDB as `OLou:PHX-AD-1` -- Compartment OCID as `ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya` -- Database Admin Credential as `admin-password` -- Database Hostname Prefix as `host0130` +- Availability Domain for the OBDS VMDB as `OLou:AP-MUMBAI-1-AD-1` - Target license model as `BRING_YOUR_OWN_LICENSE` +- Compartment OCID as `ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a` +- Database Admin Credential as `admin-password` +- Database Hostname Prefix as `host1234` - Oracle VMDB Shape as `VM.Standard2.1` -- SSH Public key for the DBCS system being deployed as `oci-publickey` -- OCID of the Subnet as `ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a` +- SSH Public key for the OBDS system being deployed as `oci-publickey` +- OCID of the Subnet as `ocid1.subnet.oc1.ap-mumbai-1.aaaaaaaa5zpzfax66omtbmjwlv4thruyru7focnu7fjcjksujmgwmr6vpbvq` **NOTE:** For the details of the parameters to be used in the .yaml file, please refer [here](./dbcs_controller_parameters.md). @@ -29,18 +29,18 @@ Use the file: [update_license.yaml](./update_license.yaml) for this use case as 1. Deploy the .yaml file: ```sh -[root@test-server DBCS]# kubectl apply -f update_license.yaml +[root@test-server OBDS]# kubectl apply -f update_license.yaml dbcssystem.database.oracle.com/dbcssystem-existing configured ``` -2. Monitor the Oracle DB Operator Pod `pod/oracle-database-operator-controller-manager-665874bd57-g2cgw` for the progress of the DBCS VMDB Scale up. +2. Monitor the Oracle DB Operator Pod `pod/oracle-database-operator-controller-manager-665874bd57-g2cgw` for the progress of the OBDS VMDB Scale up. NOTE: Check the DB Operator Pod name in your environment. ``` -[root@docker-test-server DBCS]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-g2cgw -n oracle-database-operator-system +[root@docker-test-server OBDS]# kubectl logs -f pod/oracle-database-operator-controller-manager-665874bd57-g2cgw -n oracle-database-operator-system ``` ## Sample Output -[Here](./update_license_sample_output.log) is the sample output for updating the license type an existing DBCS System deployed in OCI using Oracle DB Operator DBCS Controller. +[Here](./update_license_sample_output.log) is the sample output for updating the license type an existing OBDS System deployed in OCI using Oracle DB Operator OBDS Controller. diff --git a/docs/dbcs/provisioning/update_license.yaml b/docs/dbcs/provisioning/update_license.yaml index 1fb54a64..7c192b6b 100644 --- a/docs/dbcs/provisioning/update_license.yaml +++ b/docs/dbcs/provisioning/update_license.yaml @@ -1,18 +1,20 @@ -apiVersion: database.oracle.com/v1alpha1 -kind: DbcsSystem -metadata: - name: dbcssystem-existing -spec: - id: "ocid1.dbsystem.oc1.phx.anyhqljrabf7htyanr3lnp6wtu5ld7qwszohiteodvwahonr2yymrftarkqa" - dbSystem: - availabilityDomain: "OLou:PHX-AD-1" - compartmentId: "ocid1.compartment.oc1..aaaaaaaa4hecw2shffuuc4fcatpin4x3rdkesmmf4he67osupo7g6f7i6eya" - dbAdminPaswordSecret: "admin-password" - hostName: "host0130" - licenseModel: "BRING_YOUR_OWN_LICENSE" - shape: "VM.Standard2.1" - sshPublicKeys: - - "oci-publickey" - subnetId: "ocid1.subnet.oc1.phx.aaaaaaaauso243tymnzeh6zbz5vkejgyu4ugujul5okpa5xbaq3275izbc7a" - ociConfigMap: "oci-cred" - ociSecret: "oci-privatekey" + apiVersion: database.oracle.com/v4 + kind: DbcsSystem + metadata: + name: dbcssystem-existing + spec: + id: "ocid1.dbsystem.oc1.ap-mumbai-1.anrg6ljrabf7htyadgsso7aessztysrwaj5gcl3tp7ce6asijm2japyvmroa" + ociConfigMap: "oci-cred" + ociSecret: "oci-privatekey" + dbSystem: + availabilityDomain: "OLou:AP-MUMBAI-1-AD-1" + compartmentId: "ocid1.compartment.oc1..aaaaaaaa63yqilqhgxv3dszur3a2fgwc64ohpfy43vpqjm7q5zq4q4yaw72a" + dbAdminPasswordSecret: "admin-password" + hostName: "host1234" + licenseModel: "BRING_YOUR_OWN_LICENSE" + shape: "VM.Standard2.1" + domain: "subdda0b5eaa.cluster1.oraclevcn.com" + initialDataStorageSizeInGB: 512 + sshPublicKeys: + - "oci-publickey" + subnetId: "ocid1.subnet.oc1.ap-mumbai-1.aaaaaaaa5zpzfax66omtbmjwlv4thruyru7focnu7fjcjksujmgwmr6vpbvq" diff --git a/docs/multitenant/README.md b/docs/multitenant/README.md index 6c9a6756..0d3057fc 100644 --- a/docs/multitenant/README.md +++ b/docs/multitenant/README.md @@ -1,254 +1,11 @@ - +# Multitenant Controllers -# Oracle Multitenant Database Controllers +Starting from OraOperator version 1.2.0, there are two classes of multitenant controllers: one based on [ORDS](https://www.oracle.com/uk/database/technologies/appdev/rest.html) and another based on a dedicated REST server for the operator, called LREST. In both cases, the features remains unchanged (a part from CRD name changes). A pod running a REST server (either LREST or ORDS) acts as the proxy server connected to the container database (CDB) for all incoming kubectl requests. We plan to discontinue the ORDS based controller, in the next release; no regression (a part form CRD name changes). -The Oracle Database Operator for kubernetes uses two controllers to manage [Pluggable Database life cycle][oradocpdb] +## What are the differences -- CDB controller -- PDB controller - -By usigng CDB/PDB controllers you can perform the following actions **CREATE**,**MODIFY(OPEN/COSE)**,**DELETE**,**CLONE**,**PLUG** and **UNPLUG** - -This file examplains how to setup CDB and PDB controllers, additional details can be found in the README files under usecases directories.. - -- [Usecase01][uc01] pdb crd and cdb pod are running in the same namesaoce -- [Usecase02][uc02] unplug and plug operation examples -- [Usecase03][uc03] multiple namespace example cdb pod ,pdb crd and pod operator are running in different namespaces. - -> **NOTE** that there is no controller for Container Database Operations - -## Macro steps for setup - -- Deply the Oracle Database Operator -- Create Ords based image for CDB pod -- Container DB user creation -- Create secrets for credentials -- Create certificates for https connection -- Create CDB pod - -## Oracle DB Operator Multitenant Database Controller Deployment - -To deploy OraOperator, use this [Oracle Database Operator for Kubernetes](https://github.com/oracle/oracle-database-operator/blob/main/README.md) step-by-step procedure. - -After the Oracle Database Operator is deployed, you can see the DB Operator Pods running in the Kubernetes Cluster. As part of the `OraOperator` deployment, the multitenant Database Controller is deployed. You can see the CRDs (Custom Resource Definition) for the CDB and PDBs in the list of CRDs. The following output is an example of such a deployment: - -```bash -[root@test-server oracle-database-operator]# kubectl get ns -NAME STATUS AGE -cert-manager Active 32h -default Active 245d -kube-node-lease Active 245d -kube-public Active 245d -kube-system Active 245d -oracle-database-operator-system Active 24h <<<< namespace to deploy the Oracle Database Operator - -[root@test-server oracle-database-operator]# kubectl get all -n oracle-database-operator-system -NAME READY STATUS RESTARTS AGE -pod/oracle-database-operator-controller-manager-665874bd57-dlhls 1/1 Running 0 28s -pod/oracle-database-operator-controller-manager-665874bd57-g2cgw 1/1 Running 0 28s -pod/oracle-database-operator-controller-manager-665874bd57-q42f8 1/1 Running 0 28s - -NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE -service/oracle-database-operator-controller-manager-metrics-service ClusterIP 10.96.130.124 8443/TCP 29s -service/oracle-database-operator-webhook-service ClusterIP 10.96.4.104 443/TCP 29s - -NAME READY UP-TO-DATE AVAILABLE AGE -deployment.apps/oracle-database-operator-controller-manager 3/3 3 3 29s - -NAME DESIRED CURRENT READY AGE -replicaset.apps/oracle-database-operator-controller-manager-665874bd57 3 3 3 29s -[root@docker-test-server oracle-database-operator]# - -[root@test-server oracle-database-operator]# kubectl get crd -NAME CREATED AT -autonomouscontainerdatabases.database.oracle.com 2022-06-22T01:21:36Z -autonomousdatabasebackups.database.oracle.com 2022-06-22T01:21:36Z -autonomousdatabaserestores.database.oracle.com 2022-06-22T01:21:37Z -autonomousdatabases.database.oracle.com 2022-06-22T01:21:37Z -cdbs.database.oracle.com 2022-06-22T01:21:37Z <<<< -certificaterequests.cert-manager.io 2022-06-21T17:03:46Z -certificates.cert-manager.io 2022-06-21T17:03:47Z -challenges.acme.cert-manager.io 2022-06-21T17:03:47Z -clusterissuers.cert-manager.io 2022-06-21T17:03:48Z -dbcssystems.database.oracle.com 2022-06-22T01:21:38Z -issuers.cert-manager.io 2022-06-21T17:03:49Z -oraclerestdataservices.database.oracle.com 2022-06-22T01:21:38Z -orders.acme.cert-manager.io 2022-06-21T17:03:49Z -pdbs.database.oracle.com 2022-06-22T01:21:39Z <<<< -shardingdatabases.database.oracle.com 2022-06-22T01:21:39Z -singleinstancedatabases.database.oracle.com 2022-06-22T01:21:40Z -``` - - -## Prerequsites to manage PDB Life Cycle using Oracle DB Operator Multitenant Database Controller - -* [Prepare the container database for PDB Lifecycle Management or PDB-LM](#prepare-cdb-for-pdb-lifecycle-management-pdb-lm) -* [Oracle REST Data Service or ORDS Image](#oracle-rest-data-service-ords-image) -* [Kubernetes Secrets](#kubernetes-secrets) -* [Kubernetes CRD for CDB](#kubernetes-crd-for-cdb) -* [Kubernetes CRD for PDB](#kubernetes-crd-for-pdb) - -## Prepare the container database for PDB Lifecycle Management (PDB-LM) - -Pluggable Database (PDB) management operations are performed in the Container Database (CDB). These operations include create, clone, plug, unplug, delete, modify and map pdb. - -You cannot have an ORDS-enabled schema in the container database. To perform the PDB lifecycle management operations, you must first use the following steps to define the default CDB administrator credentials on target CDBs: - -Create the CDB administrator user, and grant the required privileges. In this example, the user is `C##DBAPI_CDB_ADMIN`. However, any suitable common user name can be used. - -```SQL -SQL> conn /as sysdba - --- Create following users at the database level: - -ALTER SESSION SET "_oracle_script"=true; -DROP USER C##DBAPI_CDB_ADMIN cascade; -CREATE USER C##DBAPI_CDB_ADMIN IDENTIFIED BY CONTAINER=ALL ACCOUNT UNLOCK; -GRANT SYSOPER TO C##DBAPI_CDB_ADMIN CONTAINER = ALL; -GRANT SYSDBA TO C##DBAPI_CDB_ADMIN CONTAINER = ALL; -GRANT CREATE SESSION TO C##DBAPI_CDB_ADMIN CONTAINER = ALL; - - --- Verify the account status of the following usernames. They should not be in locked status: - -col username for a30 -col account_status for a30 -select username, account_status from dba_users where username in ('ORDS_PUBLIC_USER','C##DBAPI_CDB_ADMIN','APEX_PUBLIC_USER','APEX_REST_PUBLIC_USER'); -``` - -## OCI OKE(Kubernetes Cluster) - -You can use an [OKE in Oracle Cloud Infrastructure][okelink] to configure the operator for PDB lifecycle management. **Note that there is no restriction about container database location; it can be anywhere (on cloud or premises , on any supported platform).** -To quickly create an OKE cluster in your OCI cloud environment you can use the following [link](./provisioning/quickOKEcreation.md). -In this setup example [provisioning example setup](./provisioning/example_setup_using_oci_oke_cluster.md), the Container database is running on a OCI Exadata Database Cluster. - - -## Oracle REST Data Service (ORDS) Image - - The PDB Database controllers require a pod running a dedicated rest server image based on [ORDS][ordsdoc]. Read the following [link](./provisioning/ords_image.md) to build the ords images. - - -## Kubernetes Secrets - - Multitenant Controllers use Kubernetes Secrets to store the required credential. The https certificates are stored in Kubernetes Secrets as well. - - **Note** In multi namespace enviroment you have to create specific secrets for each namespaces - - **Note** In multi namespace enviroment you have to create specific secrets for each namespaces - - **Note** In multi namespace enviroment you have to create specific secrets for each namespaces - -### Secrets for CDB CRD - - Create a secret file as shown here: [config/samples/multitenant/cdb_secret.yaml](../multitenant/provisioning/singlenamespace/cdb_create.yaml). Modify this file with the `base64` encoded values of the required passwords for CDB, and use this file to create the required secrets. - - ```bash - kubectl apply -f cdb_secret.yaml - ``` - - **Note:** To obtain the `base64` encoded value for a password, use the following command: - - ```bash - echo -n "" | base64 - ``` - - **Note:** After successful creation of the CDB Resource, the CDB secrets are deleted from the Kubernetes system . - -### Secrets for PDB CRD - - Create a secret file as shown here: [pdb_secret.yaml](../multitenant/provisioning/singlenamespace/pdb_secret.yaml). Edit the file using your base64 credential and apply it. - - ```bash - kubectl apply -f pdb_secret.yaml - ``` - - **NOTE:** Don't leave plaintext files containing sensitive data on disk. After loading the Secret, remove the plaintext file or move it to secure storage. - -### Secrets for CERTIFICATES - -Create the certificates and key on your local host, and use them to create the Kubernetes secret. - -```bash -openssl genrsa -out ca.key 2048 -openssl req -new -x509 -days 365 -key ca.key -subj "/C=US/ST=California/L=SanFrancisco/O=oracle /CN=cdb-dev-ords /CN=localhost Root CA " -out ca.crt -openssl req -newkey rsa:2048 -nodes -keyout tls.key -subj "/C=US/ST=California/L=SanFrancisco/O=oracle /CN=cdb-dev-ords /CN=localhost" -out server.csr -echo "subjectAltName=DNS:cdb-dev-ords,DNS:www.example.com" > extfile.txt -openssl x509 -req -extfile extfile.txt -days 365 -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out tls.crt -``` - -```bash -kubectl create secret tls db-tls --key="tls.key" --cert="tls.crt" -n oracle-database-operator-system -kubectl create secret generic db-ca --from-file=ca.crt -n oracle-database-operator-system -``` - -image_not_found - -**Note:** On successful creation of the certificates secret creation remove files or move to secure storage . - -## Kubernetes CRD for CDB - -The Oracle Database Operator Multitenant Controller creates the CDB kind as a custom resource that models a target CDB as a native Kubernetes object. This kind is used only to create Pods to connect to the target CDB to perform PDB-LM operations. Each CDB resource follows the CDB CRD as defined here: [config/crd/bases/database.oracle.com_cdbs.yaml](../../config/crd/bases/database.oracle.com_cdbs.yaml) - -To create a CDB CRD, see this example `.yaml` file: [cdb_create.yaml](../multitenant/provisioning/singlenamespace/cdb_create.yaml) - -**Note:** The password and username fields in this *cdb.yaml* Yaml are the Kubernetes Secrets created earlier in this procedure. For more information, see the section [Kubernetes Secrets](https://kubernetes.io/docs/concepts/configuration/secret/). To understand more about creating secrets for pulling images from a Docker private registry, see [Kubernetes Private Registry Documenation]( https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/). - -Create a CDB CRD Resource example - -```bash -kubectl apply -f cdb_create.yaml -``` - -see [usecase01][uc01] and usecase03[uc03] for more information about file configuration - -## Kubernetes CRD for PDB - -The Oracle Database Operator Multitenant Controller creates the PDB kind as a custom resource that models a PDB as a native Kubernetes object. There is a one-to-one mapping between the actual PDB and the Kubernetes PDB Custom Resource. You cannot have more than one Kubernetes resource for a target PDB. This PDB resource can be used to perform PDB-LM operations by specifying the action attribute in the PDB Specs. Each PDB resource follows the PDB CRD as defined here: [config/crd/bases/database.oracle.com_pdbs.yaml](../../../config/crd/bases/database.oracle.com_pdbs.yaml) - -To create a PDB CRD Resource, a sample .yaml file is available here: [pdb_create.yaml](../multitenant/provisioning/singlenamespace/pdb_create.yaml) - -```bash -kubectl apply -f cdb_create.yaml -``` - -## Usecases files list - -### Single Namespace - -1. [Create CDB](./provisioning/singlenamespace/cdb_create.yaml) -2. [Create PDB](./provisioning/singlenamespace/pdb_create.yaml) -3. [Clone PDB](./provisioning/singlenamespace/pdb_clone.yaml) -4. [Open PDB](./provisioning/singlenamespace/pdb_open.yaml) -4. [Close PDB](./provisioning/singlenamespace/pdb_close.yaml) -5. [Delete PDB](./provisioning/singlenamespace/pdb_delete.yaml) -6. [Unplug PDB](./provisioning/singlenamespace/pdb_unplug.yaml) -7. [Plug PDB](./provisioning/singlenamespace/pdb_plug.yaml) - -### Multiple namespace (cdbnamespace,dbnamespace) - -1. [Create CDB](./provisioning/multinamespace/cdb_create.yaml) -2. [Create PDB](./provisioning/multinamespace/pdb_create.yaml) -3. [Clone PDB](./provisioning/multinamespace/pdb_clone.yaml) -4. [Open PDB](./provisioning/multinamespace/pdb_open.yaml) -4. [Close PDB](./provisioning/multinamespace/pdb_close.yaml) -5. [Delete PDB](./provisioning/multinamespace/pdb_delete.yaml) -6. [Unplug PDB](./provisioning/multinamespace/pdb_unplug.yaml) - -## Known issues - - - Ords installatian failure if pluaggable databases in the container db are not opened - - - Version 1.1.0: encoded password for https authentication may include carriege return as consequence the https request fails with http 404 error. W/A generate encoded password using **printf** instead of **echo**. - - - pdb controller authentication suddenly failes without any system change. Check the certificate expiration date **openssl .... -days 365** - - - Nothing happens after cdb yaml file applying: Make sure to have properly configure the WHATCH_NAMESPACE list in the operator yaml file - - [okelink]:https://docs.oracle.com/en-us/iaas/Content/ContEng/Concepts/contengoverview.htm - [ordsdoc]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/23.1/index.html - [uc01]:../multitenant/usecase01/README.md - [uc02]:../multitenant/usecase02/README.md - [uc03]:../multitenant/usecase03/README.md - [oradocpdb]:https://docs.oracle.com/en/database/oracle/oracle-database/21/multi/introduction-to-the-multitenant-architecture.html#GUID-AB84D6C9-4BBE-4D36-992F-2BB85739329F - - \ No newline at end of file +- Regarding the YAML file, the parameters for the existing functionalities are unchanged. +- The **CRD** names are different: for controllers based on [ORDS](./ords-based/README.md), we have **PDB** and **CDB**, while for controllers based on [LREST](./lrest-based/README.md), we have **LRPDB** and **LREST**. +- If you use an LREST-based controller, there is no need to manually create the REST server pod. The image is available for download on OCR. +- Controllers based on **LREST** allow you to manage PDB parameters using kubectl. +- ORDS controllers currently do not support ORDS version 24.1. diff --git a/docs/multitenant/lrest-based/README.md b/docs/multitenant/lrest-based/README.md new file mode 100644 index 00000000..f17abd41 --- /dev/null +++ b/docs/multitenant/lrest-based/README.md @@ -0,0 +1,500 @@ + + + +# LREST BASED MULTITENANT CONTROLLERS FOR PDB LIFE CYCLE MANAGEMENT + + +- [LREST BASED MULTITENANT CONTROLLERS FOR PDB LIFE CYCLE MANAGEMENT](#lrest-based-multitenant-controllers-for-pdb-life-cycle-management) + - [STEP BY STEP CONFIGURATION](#step-by-step-configuration) + - [Multiple namespace setup](#multiple-namespace-setup) + - [Create the operator](#create-the-operator) + - [Container database setup](#container-database-setup) + - [Apply rolebinding](#apply-rolebinding) + - [Certificate and credentials](#certificate-and-credentials) + - [Private key 🔑](#private-key-) + - [Public Key 🔑](#public-key-) + - [Certificates](#certificates) + - [Create secrets for certificate and keys](#create-secrets-for-certificate-and-keys) + - [Create secrets with encrypted password](#create-secrets-with-encrypted-password) + - [Create lrest pod](#create-lrest-pod) + - [Create PDB](#create-pdb) + - [pdb config map ](#pdb-config-map) + - [Open PDB](#open-pdb) + - [Close PDB](#close-pdb) + - [Clone PDB](#clone-pdb) + - [Unplug PDB](#unplug-pdb) + - [Plug PDB](#plug-pdb) + - [Delete PDB](#delete-pdb) + - [Map PDB](#map-pdb) + + + + + +**Lrpdb** and **lrest** are two controllers for PDB lifecycle management (**PDBLCM**). They rely on a dedicated REST server (Lite Rest Server) Container image to run. The `lrest` controller is available on the Oracle Container Registry (OCR). The container database can be anywhere (on-premises or in the Cloud). + +![generaleschema](./images/Generalschema2.jpg) + +## STEP BY STEP CONFIGURATION +Complete each of these steps in the order given. + +### Multiple namespace setup + +Before proceeding with controllers setup, ensure that the Oracle Database Operator (operator) is configured to work with multiple namespaces, as specified in the [README](../../../README.md). +In this document, each controller is running in a dedicated namespace: lrest controller is running in **cdbnamespace** , lrpdb controller is running in **pdbnamespace**. The [usecase directory](./usecase/README.md) contains all the files reported in this document. + +Configure the **WACTH_NAMESPACE** list of the operator `yaml` file + +```bash +sed -i 's/value: ""/value: "oracle-database-operator-system,pdbnamespace,cdbnamespace"/g' oracle-database-operator.yaml +``` + +### Create the operator +Run the following command: + +```bash +kubectl apply -f oracle-database-operator.yaml +``` +Check the controller: +```bash +kubectl get pods -n oracle-database-operator-system +NAME READY STATUS RESTARTS AGE +oracle-database-operator-controller-manager-796c9b87df-6xn7c 1/1 Running 0 22m +oracle-database-operator-controller-manager-796c9b87df-sckf2 1/1 Running 0 22m +oracle-database-operator-controller-manager-796c9b87df-t4qns 1/1 Running 0 22m +``` +### Container database setup + +On the container database, use the following commands to configure the account for PDB administration: + +```sql +alter session set "_oracle_script"=true; +create user identified by ; +grant create session to container=all; +grant sysdba to container=all; +``` + + +### Apply rolebinding + + +Apply the following files : [`pdbnamespace_binding.yaml`](./usecase/pdbnamespace_binding.yaml) [`cdbnamespace_binding.yaml`](./usecase/cdbnamespace_binding.yaml) +```bash +kubectl apply -f pdbnamespace_binding.yaml +kubectl apply -f cdbnamespace_binding.yaml +``` + +### Certificate and credentials +You must create the public key, private key, certificates and Kubernetes Secrets for the security configuration. + +#### Private key 🔑 +> Note: Only private key **PCKS8** format is supported by LREST controllers. Before you start configuration, ensure that you can use it. If you are using [`openssl3`](https://docs.openssl.org/master/) then `pcks8` is generated by default. If it is not already generated, then use the following command to create a `pcks8` private key + +```bash +openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -pkeyopt rsa_keygen_pubexp:65537 -out private.key +``` +#### Public Key 🔑 +Create the public key. + +```bash +/usr/bin/openssl rsa -in private.key -outform PEM -pubout -out public.pem +``` +#### Certificates +Create certificates. +```bash +openssl req -new -x509 -days 365 -key private.key -subj "/C=CN/ST=GD/L=SZ/O=oracle, Inc./CN=oracle Root CA" -out ca.crt +``` +```bash +openssl req -newkey rsa:2048 -nodes -keyout tls.key -subj "/C=CN/ST=GD/L=SZ/O=oracle, Inc./CN=cdb-dev-lrest.cdbnamespace" -out server.csr +``` +```bash +/usr/bin/echo "subjectAltName=DNS:cdb-dev-lrest.cdbnamespace,DNS:www.example.com" > extfile.txt +``` +```bash +/usr/bin/openssl x509 -req -extfile extfile.txt -days 365 -in server.csr -CA ca.crt -CAkey private.key -CAcreateserial -out tls.crt +``` + +### Create secrets for certificate and keys +Create the Kubernetes Secrets. + +```bash +kubectl create secret tls db-tls --key="tls.key" --cert="tls.crt" -n oracle-database-operator-system +kubectl create secret generic db-ca --from-file="ca.crt" -n oracle-database-operator-system +kubectl create secret tls db-tls --key="tls.key" --cert="tls.crt" -n cdbnamespace +kubectl create secret generic db-ca --from-file="ca.crt" -n cdbnamespace +kubectl create secret tls db-tls --key="tls.key" --cert="tls.crt" -n pdbnamespace +kubectl create secret generic db-ca --from-file="ca.crt" -n pdbnamespace +``` + +```bash +kubectl create secret tls prvkey --key="private.key" --cert=ca.crt -n cdbnamespace +kubectl create secret generic pubkey --from-file=publicKey=public.pem -n cdbnamespace +kubectl create secret generic prvkey --from-file=privateKey="private.key" -n pdbnamespace +``` + +### Create secrets with encrypted password + +In this example, we create the Secrets for each credential (username and password) + +| secret usr | secrets pwd | credential description | +| -----------|-------------|-----------------------------------------------------------| +| **dbuser** |**dbpass** | the administrative user created on the container database | +| **wbuser** |**wbpass** | the user for https authentication | +| **pdbusr** |**pdbpwd** | the administrative user of the pdbs | + + +```bash +echo "[ADMINUSERNAME]" > dbuser.txt +echo "[ADMINUSERNAME PASSWORD]" > dbpass.txt +echo "[WEBUSER]" > wbuser.txt +echo "[WEBUSER PASSWORD]" > wbpass.txt +echo "[PDBUSERNAME]" > pdbusr.txt +echo "[PDBUSERNAME PASSWORD]" > pdbpwd.txt + +## Encrypt the credentials +openssl rsautl -encrypt -pubin -inkey public.pem -in dbuser.txt |base64 > e_dbuser.txt +openssl rsautl -encrypt -pubin -inkey public.pem -in dbpass.txt |base64 > e_dbpass.txt +openssl rsautl -encrypt -pubin -inkey public.pem -in wbuser.txt |base64 > e_wbuser.txt +openssl rsautl -encrypt -pubin -inkey public.pem -in wbpass.txt |base64 > e_wbpass.txt +openssl rsautl -encrypt -pubin -inkey public.pem -in pdbusr.txt |base64 > e_pdbusr.txt +openssl rsautl -encrypt -pubin -inkey public.pem -in pdbpwd.txt |base64 > e_pdbpwd.txt + +kubectl create secret generic dbuser --from-file=e_dbuser.txt -n cdbnamespace +kubectl create secret generic dbpass --from-file=e_dbpass.txt -n cdbnamespace +kubectl create secret generic wbuser --from-file=e_wbuser.txt -n cdbnamespace +kubectl create secret generic wbpass --from-file=e_wbpass.txt -n cdbnamespace +kubectl create secret generic wbuser --from-file=e_wbuser.txt -n pdbnamespace +kubectl create secret generic wbpass --from-file=e_wbpass.txt -n pdbnamespace +kubectl create secret generic pdbusr --from-file=e_pdbusr.txt -n pdbnamespace +kubectl create secret generic pdbpwd --from-file=e_pdbpwd.txt -n pdbnamespace + +rm dbuser.txt dbpass.txt wbuser.txt wbpass.txt pdbusr.txt pdbpwd.txt \ + e_dbuser.txt e_dbpass.txt e_wbuser.txt e_wbpass.txt e_pdbusr.txt e_pdbpwd.txt +``` + +### Create lrest pod + +To create the REST pod and monitor its processing, use the `yaml` file [`create_lrest_pod.yaml`](./usecase/create_lrest_pod.yaml) + +Ensure that you update the **lrestImage** with the latest version available on the [Oracle Container Registry (OCR)](https://container-registry.oracle.com/ords/f?p=113:4:104288359787984:::4:P4_REPOSITORY,AI_REPOSITORY,AI_REPOSITORY_NAME,P4_REPOSITORY_NAME,P4_EULA_ID,P4_BUSINESS_AREA_ID:1283,1283,This%20image%20is%20part%20of%20and%20for%20use%20with%20the%20Oracle%20Database%20Operator%20for%20Kubernetes,This%20image%20is%20part%20of%20and%20for%20use%20with%20the%20Oracle%20Database%20Operator%20for%20Kubernetes,1,0&cs=3076h-hg1qX3eJANBcUHBNBCmYWjMvxLkZyTAhDn2e8VR8Gxb_a-I8jZLhf9j6gmnimHwlP_a0OQjX6vjBfSAqQ) + +```bash +--> for amd64 +lrestImage: container-registry.oracle.com/database/operator:lrest-241210-amd64 + +--> for arm64 +lrestImage: container-registry.oracle.com/database/operator:lrest-241210-arm64 +``` + +```bash +kubectl apply -f create_lrest_pod.yaml +``` + +monitor the file processing: + +```bash +kubectl get pods -n cdbnamespace --watch +NAME READY STATUS RESTARTS AGE +cdb-dev-lrest-rs-9gvx2 0/1 Pending 0 0s +cdb-dev-lrest-rs-9gvx2 0/1 Pending 0 0s +cdb-dev-lrest-rs-9gvx2 0/1 ContainerCreating 0 0s +cdb-dev-lrest-rs-9gvx2 1/1 Running 0 2s + +kubectl get lrest -n cdbnamespace +NAME CDB NAME DB SERVER DB PORT TNS STRING REPLICAS STATUS MESSAGE +cdb-dev DB12 (DESCRIPTION=(CONNECT_TIMEOUT=90)(RETRY_COUNT=30)(RETRY_DELAY=10)(TRANSPORT_CONNECT_TIMEOUT=70)(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan12.testrac.com)(PORT=1521)(IP=V4_ONLY))(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan34.testrac.com)(PORT=1521)(IP=V4_ONLY))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=TESTORDS))) 1 Ready +``` + +Check the Pod logs: + +```bash +/usr/local/go/bin/kubectl logs -f `/usr/local/go/bin/kubectl get pods -n cdbnamespace|grep lrest|cut -d ' ' -f 1` -n cdbnamespace +``` + +Output example: + +```text +... +... +2024/09/05 12:44:09 wallet file /opt/oracle/lrest/walletfile exists completed +2024/09/05 12:44:09 call: C.ReadWallet +LENCHECK: 7 11 7 8 +2024/09/05 12:44:09 ===== DUMP INFO ==== +00000000 28 44 45 53 43 52 49 50 54 49 4f 4e 3d 28 43 4f |(DESCRIPTION=(CO| +00000010 4e 4e 45 43 54 5f 54 49 4d 45 4f 55 54 3d 39 30 |NNECT_TIMEOUT=90| +00000020 29 28 52 45 54 52 59 5f 43 4f 55 4e 54 3d 33 30 |)(RETRY_COUNT=30| +00000030 29 28 52 45 54 52 59 5f 44 45 4c 41 59 3d 31 30 |)(RETRY_DELAY=10| +00000040 29 28 54 52 41 4e 53 50 4f 52 54 5f 43 4f 4e 4e |)(TRANSPORT_CONN| +00000050 45 43 54 5f 54 49 4d 45 4f 55 54 3d 37 30 29 28 |ECT_TIMEOUT=70)(| +00000060 4c 4f 41 44 5f 42 41 4c 4c 41 4e 43 45 3d 4f 4e |LOAD_BALLANCE=ON| +00000070 29 28 41 44 44 52 45 53 53 3d 28 50 52 4f 54 4f |)(ADDRESS=(PROTO| +00000080 43 4f 4c 3d 54 43 50 29 28 48 4f 53 54 3d 73 63 |COL=TCP)(HOST=sc| +00000090 61 6e 31 32 2e 74 65 73 74 72 61 63 2e 63 6f 6d |an12.testrac.com| +000000a0 29 28 50 4f 52 54 3d 31 35 32 31 29 28 49 50 3d |)(PORT=1521)(IP=| +000000b0 56 34 5f 4f 4e 4c 59 29 29 28 4c 4f 41 44 5f 42 |V4_ONLY))(LOAD_B| +000000c0 41 4c 4c 41 4e 43 45 3d 4f 4e 29 28 41 44 44 52 |ALLANCE=ON)(ADDR| +000000d0 45 53 53 3d 28 50 52 4f 54 4f 43 4f 4c 3d 54 43 |ESS=(PROTOCOL=TC| +000000e0 50 29 28 48 4f 53 54 3d 73 63 61 6e 33 34 2e 74 |P)(HOST=scan34.t| +000000f0 65 73 74 72 61 63 2e 63 6f 6d 29 28 50 4f 52 54 |estrac.com)(PORT| +00000100 3d 31 35 32 31 29 28 49 50 3d 56 34 5f 4f 4e 4c |=1521)(IP=V4_ONL| +00000110 59 29 29 28 43 4f 4e 4e 45 43 54 5f 44 41 54 41 |Y))(CONNECT_DATA| +00000120 3d 28 53 45 52 56 45 52 3d 44 45 44 49 43 41 54 |=(SERVER=DEDICAT| +00000130 45 44 29 28 53 45 52 56 49 43 45 5f 4e 41 4d 45 |ED)(SERVICE_NAME| +00000140 3d 54 45 53 54 4f 52 44 53 29 29 29 |=TESTORDS)))| +00000000 2f 6f 70 74 2f 6f 72 61 63 6c 65 2f 6c 72 65 73 |/opt/oracle/lres| +00000010 74 2f 77 61 6c 6c 65 74 66 69 6c 65 |t/walletfile| +2024/09/05 12:44:09 Get credential from wallet +7 +8 +2024/09/05 12:44:09 dbuser: restdba webuser :welcome +2024/09/05 12:44:09 Connections Handle +2024/09/05 12:44:09 Working Session Aarry dbhanlde=0x1944120 +2024/09/05 12:44:09 Monitor Session Array dbhanlde=0x1a4af70 +2024/09/05 12:44:09 Open cursors +Parsing sqltext=select inst_id,con_id,open_mode,nvl(restricted,'NONE'),total_size from gv$pdbs where inst_id = SYS_CONTEXT('USERENV','INSTANCE') and name =upper(:b1) +Parsing sqltext=select count(*) from pdb_plug_in_violations where name =:b1 +2024/09/05 12:44:11 Protocol=https +2024/09/05 12:44:11 starting HTTPS/SSL server +2024/09/05 12:44:11 ==== TLS CONFIGURATION === +2024/09/05 12:44:11 srv=0xc000106000 +2024/09/05 12:44:11 cfg=0xc0000a2058 +2024/09/05 12:44:11 mux=0xc0000a2050 +2024/09/05 12:44:11 tls.minversion=771 +2024/09/05 12:44:11 CipherSuites=[49200 49172 157 53] +2024/09/05 12:44:11 cer=/opt/oracle/lrest/certificates/tls.crt +2024/09/05 12:44:11 key=/opt/oracle/lrest/certificates/tls.key +2024/09/05 12:44:11 ========================== +2024/09/05 12:44:11 HTTPS: Listening port=8888 +2024/09/05 12:44:23 call BasicAuth Succeded +2024/09/05 12:44:23 HTTP: [1:0] Invalid credential <-- This message can be ignored + +``` + +**lrest Pod creation** - parameters list +| Name | Dcription | +--------------------------|-------------------------------------------------------------------------------| +|cdbName | Name of the container database (db) | +|lrestImage (DO NOT EDIT) | **container-registry.oracle.com/database/lrest-dboper:latest** | +|dbTnsurl | TNS alias of the container db | +|deletePdbCascade | Delete all of the PDBs associated to a CDB resource when the CDB resource is dropped using [imperative approach](https://kubernetes.io/docs/tasks/manage-kubernetes-objects/imperative-command/) | +|cdbAdminUser | Secret: the administrative (admin) user | +|cdbAdminPwd | Secret: the admin user password | +|webServerUser | Secret: the HTTPS user | +|webServerPwd | Secret: the HTTPS user password | +|cdbTlsCrt | Secret: the `tls.crt ` | +|cdbPubKey | Secret: the public key | +|cdbPrvKey | Secret: the private key | + + + + +### Create PDB + +To create a pluggable database (PDB), apply the yaml file [`create_lrpdb1_resource.yaml`](./usecase/clone_lrpdb1_resource.yaml) + +```bash +kubectl apply -f create_lrpdb1_resource.yaml +``` +Check the status of the resource and the PDB existence on the container db: + +```bash +kubectl get lrpdb -n pdbnamespace +NAME CONNECT_STRING CDB NAME LRPDB NAME LRPDB STATE LRPDB SIZE STATUS MESSAGE LAST SQLCODE +lrpdb1 (DESCRIPTION=(CONNECT_TIMEOUT=90)(RETRY_COUNT=30)(RETRY_DELAY=10)(TRANSPORT_CONNECT_TIMEOUT=70)(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan12.testrac.com)(PORT=1521)(IP=V4_ONLY))(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan34.testrac.com)(PORT=1521)(IP=V4_ONLY))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=pdbdev))) DB12 pdbdev MOUNTED 2G Ready Success +``` + +```bash +SQL> show pdbs + + CON_ID CON_NAME OPEN MODE RESTRICTED +---------- ------------------------------ ---------- ---------- + 2 PDB$SEED READ ONLY NO + 3 PDBDEV MOUNTED +SQL> +``` +``Note that after creation, the PDB is not open. You must explicitly open it using a dedicated `yaml` file. + +**pdb creation** - parameters list + +| Name | Dcription | +|-------------------------|-------------------------------------------------------------------------------| +|cdbResName | REST server resource name | +|cdbNamespace | Namespace of the REST server | +|cdbName | Name of the container database | +|pdbName | Name of the PDB that you want to create | +|assertiveLrpdbDeletion | Boolean: Turn on the imperative approach on PDB resource deletion | +|adminpdbUser | Secret: PDB admin user | +|adminpdbPass | Secret: password of PDB admin user | +|lrpdbTlsKey | Secret: `tls.key ` | +|lrpdbTlsCrt | Secret: `tls.crt` | +|lrpdbTlsCat | Secret: `ca.crt` | +|webServerUser | Secret: the HTTPS user | +|webServerPwd | Secret: the HTTPS user password | +|cdbPrvKey | Secret: private key | +|cdbPubKey | Secret: public key | +|pdbconfigmap | kubernetes config map that contains the PDB initialization (init) parameters | + +> NOTE: **assertiveLrpdbDeletion** must be specified for the following PDB actions **CLONE** **CREATE** **PLUG** **MAP**. + +🔥 **assertiveLrpdbDeletion** drops pluggable database using **INCLUDE DATAFILES** option + +All of the parameters **adminpdbUser** **adminpdbPass** **lrpdbTlsKey** **lrpdbTlsCrt** **lrpdbTlsCat** **webServerUser** **webServerPwd** **cdbPrvKey** **cdbPubKey** must be specified in all PDB lifecycle management `yaml` files. To simplify presentation of requirements, we will not include them in the subsequent tables. + + +#### pdb config map + +By using **pdbconfigmap** it is possible to specify a kubernetes `configmap` with init PDB parameters. The config map payload has the following format: + + +``` +;; +;; +;; +.... +.... +;; +``` + +Example of `configmap` creation: + +```bash +cat < parameters.txt +session_cached_cursors;100;spfile +open_cursors;100;spfile +db_file_multiblock_read_count;16;spfile +EOF + +kubectl create configmap config-map-pdb -n pdbnamespace --from-file=./parameters.txt + +kubectl describe configmap config-map-pdb -n pdbnamespace +Name: config-map-pdb +Namespace: pdbnamespace +Labels: +Annotations: + +Data +==== +parameters.txt: +---- +session_cached_cursors;100;spfile +open_cursors;100;spfile +db_file_multiblock_read_count;16;spfile +test_invalid_parameter;16;spfile +``` + +- If specified, the `configmap` is applied during PDB **cloning**, **opening** and **plugging** +- The `configmap` is not monitored by the reconciliation loop; this feature will be available in future releases. This means that if someone decides to manually alter an init parameter, then the operator does not take any actions to syncronize PDB configuration with the `configmap`. +- **Alter system parameter feature** will be available in future releases. +- An application error with the `configmap` (for whatever reason) does not stop processes from completing. A warning with the associated SQL code is reported in the log file. + + + +### Open PDB + +To open the PDB, use the file [`open_lrpdb1_resource.yaml`](./usecase/open_lrpdb1_resource.yaml): + +```bash +kubectl apply -f open_lrpdb1_resource.yaml +``` + + **pdb opening** - parameters list + +| Name | Description/Value | +|-------------------------|-------------------------------------------------------------------------------| +|cdbResName | REST server resource name | +|cdbNamespace | Namespace of the REST server | +|cdbName | Name of the container database (CDB) | +|pdbName | Name of the pluggable database (PDB) that you are creating | +|action | Use **Modify** to open the PDB | +|pdbState | Use **OPEN** to open the PDB | +|modifyOption | Use **READ WRITE** to open the PDB | + +### Close PDB + +To close the PDB, use the file [`close_lrpdb1_resource.yaml`](./usecase/close_lrpdb1_resource.yaml): + +```bash +kubectl apply -f close_lrpdb1_resource.yaml +``` +**pdb closing** - parameters list +| Name | Description/Value | +|-------------------------|-------------------------------------------------------------------------------| +|cdbResName | REST server resource name | +|cdbNamespace | Namespace of the REST server | +|cdbName | Name of the container database (CDB) | +|pdbName | Name of the pluggable database (PDB) that you want to create | +|action | Use **Modify** to close the PDB | +|pdbState | Use **CLOSE** to close the PDB | +|modifyOption | Use **IMMEDIATE** to close the PDB | + +### Clone PDB ### + +To clone the PDB, use the file [`clone_lrpdb1_resource.yaml`](./usecase/clone_lrpdb1_resource.yaml): + +```bash +kubeclt apply -f clone_lrpdb1_resource.yaml +``` +**pdb cloning** - parameters list +| Name | Description/Value | +|-------------------------|-------------------------------------------------------------------------------| +|cdbResName | REST server resource name | +|cdbNamespace | Namespace of the REST server | +|cdbName | Name of the container database (CDB) | +|pdbName | The name of the new pluggable database (PDB) | +|srcPdbName | The name of the source PDB | +|fileNameConversions | File name convert pattern **("path1","path2")** or **NONE** | +|totalSize | Set **unlimited** for cloning | +|tempSize | Set **unlimited** for cloning | +|pdbconfigmap | kubernetes `configmap` which contains the PDB init parameters | +|action | Use **clone** to clone the PDB | + +### Unplug PDB + +To unplug the PDB, use the file [`unplug_lrpdb1_resource.yaml`](./usecase/unplug_lrpdb1_resource.yaml): + +**pdb unplugging** +| Name | Description/Value | +|-------------------------|-------------------------------------------------------------------------------| +|cdbResName | REST server resource name | +|cdbNamespace | Namespace of the REST server | +|cdbName | Name of the container database (CDB) | +|pdbName | Name of the pluggable database (PDB)| +### Plug PDB + +To plug in the PDB, use the file [`plug_lrpdb1_resource.yaml`](./usecase/plug_lrpdb1_resource.yaml). In this example, we plug in the PDB that was unpluged in the previous step: + +**pdb plugging** +| Name | Description/Value | +|-------------------------|-------------------------------------------------------------------------------| +|cdbResName | REST server resource name | +|cdbNamespace | Namespace of the REST server | +|cdbName | Name of the container database (CDB)| | +|pdbName | Name of the pluggable database (PDB) | +|**xmlFileName** | Path of the XML file | +|action | **plug** | +|fileNameConversions | File name convert pattern **("path1","path2")** or **NONE** | +|sourceFileNameConversion | See parameter [SOURCE_FILE_NAME_CONVERT](https://docs.oracle.com/en/database/oracle/oracle-database/23/sqlrf/CREATE-PLUGGABLE-DATABASE.html#GUID-F2DBA8DD-EEA8-4BB7-A07F-78DC04DB1FFC__CCHEJFID) documentation | +|pdbconfigmap | Kubernetes `configmap` that contains the PDB init parameters | + +### Delete PDB + +To delete the PDB, use the file [`delete_lrpdb1_resource.yaml`](./usecase/delete_lrpdb1_resource.yaml) + +**pdb deletion** + +| Name | Dcription/Value | +|-------------------------|-------------------------------------------------------------------------------| +|cdbResName | REST server resource name | +|cdbNamespace | Namespace of the REST server | +|cdbName | Name of the container database (CDB) | +|action | **Delete** | +|dropAction | **INCLUDING** - Including datafiles or **NONE** | + + +### Map PDB + +If you need to create a CRD for an existing PDB, then you can use the map option by applying the file [`map_lrpdb1_resource.yaml`](./usecase/map_lrpdb1_resource.yaml) + + + + diff --git a/docs/multitenant/lrest-based/images/Generalschema2.jpg b/docs/multitenant/lrest-based/images/Generalschema2.jpg new file mode 100644 index 0000000000000000000000000000000000000000..7e7c20c045cf40599983304a17c99bd187e9ebec GIT binary patch literal 96239 zcmd421yq~S@-G_PN?RyW+$B(;xR>G(Jh-=bfIxAVQlz*Aw<5u%NN`GV2=2v;7N>ae zOV2s?f6w{fx7L%j-dk_-$$a15duCfEvwwT?@Z;ep;5kT6Q4WBD0sx>qegF@vq~Byc zAyxo@q9Pjr8vp>{0`LH+fTxdK=;H}M!TFmFppEv({)-Ps1po;D#s5ts{}-?D$W#0q z&-yR^F*|?<1rD%k{*6#k z(NO;qIvN@(8U{KB1_nAhItC^dCI$u;COSGM4ki}%U&6q|!NtMG{rm9nDCDn%zoi}z z?8j37$nt;d^6(u%fDL#L2t-4n2cQz5pb?-vbOWd#n}LGyDEpsH_;+L00j*V4Fw$?2MZhTZ?4C_eylD50E>_g`vpBW5izd@1CQi;Cui5{T0XFrOMFrd z3BQ!|$DrJCQbtWPw}iyJx{is-M_E*lS{_^f@47It&@oWZuyIfyllcf9Wk2=>8YbrB zQ2iwoR06=GGD13TNldI48rVEe^zT2$=XT%_OI45mCL&>Ym@;ed@jf(@7qxC){TDsob;p5#Q#Wp=wUM&?3D= z<+9&KPE@M7O4vtI33WwSf$OmZS0{Y+7Xd%0#^zh-+~G6lxdy4sO-v~zk*Ny`QWJ6D zDwQOTLw(%)+Rs>kr_D?JMY|OG@;R=`=dPtK!x4TW#W@BZhIE@uQV~nK@J_C6(x=y@r6jhs-|6uFH#!nFj-IiliK*iv^YJkz z{)2SWvepuJBomk3j|usib1@wI{EaOL^wgJ0d@3c$3_BxLv={#-Wulqc4KL)`giC33a<_^?!GV6>Ytv($QOFJJ^F z&~-@&8&18Rp52-szrZsX9Ho06}aup1Ix5PWpq;SLV42WJ~hd-b6#p> zsqs@u#&eQrUcEucuhT3YgZj7|zSV5g#92ixrUL;7bV>MZ8Eb@2uhqEn8E=aQ+9sg>x;Sa%d+U{|m z_w-8^0B2blOt>)JiJmYH6Q~?UVpv8&F9ZK_Ld%=+B+8 zrwz8AS&HxH*ctJ)eQhifWgaUQVwiyK*6xw&SB*GOijEh{^oRcGDgM4 zd#-ne&!i$IP7yTb+plM>cvMV#n1F{C2ltHV5Lp02imV$&307`e$-|c=hR@pnARY%2 z*rvq)vc+sTNjZA=kHZv=m8&pQ@=S1v4fKq|2!zzkWXvwGorxw=Ox$=A&1Rfy#jzJN zTu5!)#m%hDAdVLT<$E?kXhgFj@;=1B7(mlsmUCY&@c{7SpvpBVyG)KA$>7|Ou}VJY zKPRR4^f|<}&y*N*u@0M9^Sl1uRFH;?%D70yjng#1%NJ0B{;W(Yq~&ZdqCq|n3~h1N zK3Of{XNEge>ztb-=Iw0sygt5;3zcVislO#fwYr#q)6xR z5ZfZzeJC@Omd0L>Plr;!_r(|;$ADrDXTKoh42tGdk!>^x9H>i3_>-wODZXzNH-50w z^D1)8O0Kxa%$zcXs^eHEo9(3#SSxYki=-k}e-J-IKGg4%rLL`8i;u(p4QLa#;>Ke{ zu1Qd!XkV3v=TsL|m-MG~L_5LzmxjJWwNj;tosNt^j;(D%d?wIo3=t^4DJ^sby7!a$ z(>baDNi>xkQrkI+^T1C*zM|GT_H#ud7T950-{Z8m(0PMY?fd-5sg_`=w@X!ZQ6VWL zU3kGbr$Ec;n#>1)TixD@j>#;yX8N!hQ?~7@Bv|xEMac*V5LDZp87H|ni*CzXf4*lm zUc0SRem$jO97gHw!-}hKCN%{YO6;a#!7I52`}LV*SUXjO6IXOW`q*EChmGsU_Dfy% z5wE?`Wj*3&3Rbr|G$akYF><@FzU)ye&;h6n=*|bFiYmaj1<7Z=xz7(=^c$E=bfR&m zt)hl`IQZu(KP$t(He`9UDC$1vks{NhKD52uQ3FlrtUa$wA&7a$UdqG6^F}!)|BbiH zjTx?rcdeDl6z(T}1sljN<62aXyib~2+?Nk>K24BojaYG3QMVewI;GN#fQ6Yi6At`e zAk!~3sZ%tFyfdCErKfR`2L}u6r0Q`5l$99Ix-OI|Y&}y;Fk0F$y~*LNf+bC_CHpq< z6i&hwz==ZWN|LIyasa&MH2rV-q%Gx?if@7@QZkU^kHHlI zo$fFQQO_Gbmo+|K=HGTKm6|Wq{!`(UxLo113^w@CZ>dKo=2Ch+`txK^t3_%$K+3|1 zF+qD=gJ)NYq?Q5Ov_4KFtQ`C0=a()7HKpql9T-ZDDWNyI(X+_026164c+hTaggBV3T$gsfHK5R)u*0sPuk&J&NuitnDar%` zYRx#!C z+^Of-Co|@9jdbg<;-GcCUm=r{mHP+%&Cx?sY3`Z9%4 zF78HDFf1Iyo%)Xv(T$Y@a%DlqVc}!_G*H`{;hcX+m+3NVGV0qrVG?7mqG@EYFgiQk zHY^nx^Cvl3m*V3QS|AZ@y;5=~3Sv;Aeqj}>R$}6gQKa`#J#JmW;9#C#5tyK~BQ+39 z_6<*5JLv=Clgy|W6<7rGRyrDa$p(Qt3E*q%VhQtlbKm-bCJb%1%6Au201b-zutlGJ zXJIPWy902${Q_vr`DPIh=ca>q(b=VXo0E>1wNxs|Oe%T@N6)vag{}u^j$;Dm9pK@} zo;&EP7;(64gR4Whb-89#B)wrr!d!u(HFYT+N*w3{&y(IOfl>Pv0PMc=Gh0K3mv+vq zGm8p>-OH9QtR^?essKD+82c6VUfzFJrx9p~spTNTEy_)iLkKCdT?*gee`E1&4jAna zu-K7=RR?GNzMyW&9zwI)9ctffy*Tu?jBVD@4)YY`$8)!oN@jUouPNf>l>4$Ds*;47 z$)2_zwW>dx-eR&OiW)+H-gJ3syQ_mB$vy`)f3k-O(}hYcd44ra`)wf*M2M}=QgSud zXH(~~WH<^gscg+p0xHuc(PYNY>QFXtTpAjxj%^TwH%?vRr3fX6BYM~5MPbSJ%ve1ZDyI-sqTcr@G~zJ zp&Q0JuP>BNP91}|7l-@j(})J`cJh;0;nzQ4*CpqARU%WkahiCJy@X$azOhLFE;-qb z8IlRx+;lB`*e|^Im+vN7%-__m)3tx><<>RfOWpST9H(8i`lgYSW@Vjmi4hF?H`>{uzOlDW94<31^< z5Q?_qP8xj{(#&uScBifFL)~+|_{FUvy_-Ge zkV^={BQ$=ORKE?Dg3TxBM14%2hOxS;tMBBsp)_Yvmqlq3RR>F z8hD2BGn1rW{S=+Ld0%B_!qTlb&a2=VQU5j7gs@1Xc?2IrAY4pAO)i6FrnvK#a~0TKK(eM`kj5#8yc_GT>)7LZR#Vrf z^?El%Yl>9k3gw4S*rgDk&3MU>9qo9&TwLs{En+sx#(prkh};0>&0ARf1E7s@w*E(l zraId!bl4fpBK^79eB7obku!x{TDAEn@`sdqT=fptDkkx`xEj&A{~`alT$ws`?nK$$ z)Gu3twc?r(R(3+d6bcqf1u?CpCs9#RJD{b=Dkuk1vBrbN?V`#qMIDl+T;aB%<AF zY^ZL?4u(nLMKd7?#wC^H&+3A?GhE(FIJVnN+8|iik~cX9R7-Y4X6X^UtMWZZ*^&kc z#2ZBI*g{mV`TBOK%o@~ME0R{+;luB>iX)(NPHZy|05U~=sTd_fa9$<`o#O8hFM`7R z4EKd;{qUlWlA5z=I#s5cBqk6@VSqAlMEKc40lj>_Tr?f#*K?Y|?SM*`-znw2|BEkR zII#X9ePa>Oty)^~WSp(t(&Nw5lJXnu0Q=ZK;nUjH=}fx&l@9=v34(a*IsS9`rLy$; z(Hgtm*cM&BU~)d2iRU)s70T@v}i$#W@^^>W3p~m*kqb z{4v58lj6LwT;ePlv2)*I-Da7MNwWg#h)|2L9e7_X7zu$4!`G52@I?qwKjV-wbbC|o zLgz+(t3Y=KAvVTew~rLE=nLSV-wAIU8T-x56fJato(jOSU*am7JoPNo8yWXm^61KN zo4eS+B{cddfJ<)XOc<-yRh?z{@sei@L6*FT`rQ}%R){N1vyKI+Dd6rw3QBxYGR~S{E&?zL?CBDP97Npxl zb~2LDTrWEZd6Mbj;%|CMP+?csQY~^(@6ZgFqu^=-xx%@=B2#6yZsLA|)oT>qKv={S zclnC7L}d>-RCnt*vE9n3Q$Us_pbVu%#ki)ojgx*z^3Kk1TwFYs^(P?$<%f()#U+gx}!tOm)U6F+tD^ z?#OxzuCqYtNd+7tlQf(A z1ni)l^|t%xoQ&@P@3?bTif$1$nghyBkmM`?+9m}f{1f&MGV0@UahXr*xj#Ss{ySqx z&puw8o3^W6+1A4aCJz)iJN$Z2yV@#V)uSfJD9b1A@m;o8g|kD0Qj7xZ7^EG z)_6y}@>8Tkp){?KUA0B}eciW(QuSgc8T4Y-pH5`r+*MGk3T0#J;(4~$M73mN058VG z?`NkI1a$vpm}dW*VaivKF;LrldI!}T&sUKAhZ&3}|CiaaE%0@{V|cWTgi}|~@=aZB zCI}wp{sP1QKSGyL!_l<=16`(h#Mb{Cyo`;5g@THKiHnAg_lWs=JqG|V9`W!;{2L1o zALq#ke~;u)Wul3Rmlz-3~V`IJ~)TgNY;W#I-($}cGDoMe$zaPR0Im(#I!omdSH4J+)L z{{5($`Y#TR^&iML%3ov~3mX;ZFY>MS2uD96^k|s4Sm@a3|A3_b1$`4@(Q&_^=aJNK z!X}ae|3##$$B7wu&73=a2jxwyl8`d;Nhj1CUrD5z z3Pch^OJ(<2nwpw6P~)t^N1S;550+t8Q$@>rxQUVG@3B{Hc4E#linr$EpNNI0+{wAf z<&Arrfn7CUm8n1z?XYQn)>nd0)Sz(ph>{;YXQeDXAf%F|@`@#SOB1p3QN!=aTY&9k zZL|;M54w}7`mh$i5?Ulr(sz*evuO$*c)-WC?Tp*5W3F^((Nf(3A!v=MEE6Z=!t4XU zTJif=rMe6yUFJ63rWmb@d~Lh}A=S29vI8x(hy<69cD=CGZr@$CE>$uAkHka+R|A`G zKtFn)yPY2N(`17YxpHT)Gr{SZ*9MEUDL;24;Gl#M9BM(nC|K% zDR&^U!Fk^I`UwlMWBwhLyEkVK0Kn&>3-qLmL zPTta!X@_c`+u|{5Uar4;0DO6tOJV8k&5QVrkf@iH3HV*h$~igr0H{m6l!%J_AhvhU zF5&Y>F~0ZPpQYyPT5$V!x;gD3nSC`LMUJ~=;`sJ)<@|F(YhgN7?2R(*fX>~8l?Q;% zDz}JuVfD(DzJ$x4QLgW&=b+K$dqNXGv@K{Wq$>=CP|4kTW-7PL)hyikj{2?p?A4N+ z;8NHZk0TKI1Q!APT!ud=IrcuM3%$)EtOex-;T5fx8lUN59X~U_L}ocB=QRZ zr^J*_#MMyhvCTn)hGYI=HQPKo6;6^WvyJLsi^c62jQSorh*eFUPdm#+kKW$v4yYS( z$aD$o#Z7br5qCW-5yDPlo3Wj1oFsBg=FaBK?3SmNyU742BhQ-7sMJ<)T;}#1vmknz zyrR4c&0Ce%)RJ}Hr!m?O8voGF9q>d5=aU;`#!IH9Wm(4CP=6#)Tg(cPkWIkO;m|% z3X_yNoHu{jTI<>1QJ*Z=IV`DK2hzbbd1EVVGBs<{q^ik$-M8@UT@=LRj~ZGI()Pgoe9zcWbG!!*i<2nt0B#97IWfIH~T?&rE(iLHMdA zzbz9HoHtfMQk+DIj*^qsLfK7C-z|K-^*maaJcA2NASz_9ak1%@^UO8A1@~uOQ63x` z4KD)E)NKK5<2*nAV@Bf(O=PD}1meFHn%w!~R5U3`8AuN08ka1djAq}iU=u5)KU5E$ zrYLaE3HQEI_UwC6P`5^_NfI~S*{nlH_FZt`Xk83gBj zq6C=pg}J!34H&8s6ib{(1m8;Jh*~RmZYq?>Jn{TCpxM!@(5&vL1ZGLy7Wh1@z~dQ*}iUNN2vKjf42wF%>DD z#lGF=g9eXe-G28sT(xnlqsfhy8GU@x2bE0`X#r8R!6~3*n8~!(S7cg-<{`ucrzCnG zqUhOkA36}=^_QpCQm&4Of5cey5UV&3pu>9KfqfT(LJ*5UnFG$-&QJH z9j(E@-&S$GWOtu1IR(fuaa=nfM>R{j;W#)5Xl_CxqASxiX(1vcP-wL z>#XPWS-xsGZ?e)?KR&c9WZHCXOHAjEun4l!@K7}>0ced&@=c0_s`mmx23pfpvFA#B z2X_wqcejZrIosQ)-vPRhK0`8O;d;^T=89!gVtnilJS-Yu7#$G6cw2NW0{99begIs< z)7|eI$nWnJuhh<5CA4Z)S7z2tR|X`2_c4L502l%HC01GO6}|243I3XQRL`bgO*Wi2 z&R(9VAp`Qi-~muf9{{ELMUHz;RYw~+k6@R)yitpYmTUfdLsIc0-$(7WuZle{c_s-@ znoS0VU-s}w^0KxRacTv!*?dWdWJ$A{6TvyyYy)3K{dZl23&fV9FNDm7?9!d=Hy!M) zJr|BIWt&)Z$^=iM3}<#4?bqX~HTA`}%i}Lt!NtZ4%HWLfw`8nul_M`T&OH7E4M}9` zZL3wI@Ng~3kFTuZy}oxc|AKz=UxS1e4cw`Ed=i=*@2dj<) zSjLsZ(ld{=N@Iz+^s@>0c)&ba5OVjzZ9Amy*frBAxvZ~bx|&#pli4KN;%Rt97M{so zkCxyNNzlGG)0sJUt_q>~6P0)tI2Yv&hb$Lm)|_~O^aG%lSyJ;J$_}V=DgYO!lkpj-z6m=r#yse z``dCO;TO8Q)EE8}RZT8xPV0Tdnwe9kx`SSkDqa?tS`Pr<<5z>}*UC0ca!QF00HZwS zdz+~=jau7T6Y59AT&D{c6H8t(Zbe^SPuI4d;^Z zdkSH>#8p0+)hoB(tqq$w?3ALC=K;`8xHBCKeHq5NE$L$IYTOjRE(4+-i_wOnZ|+VWo#Vf*sWy-uXs8 ze^l*yq1)TJsfPLOKrLi}B%9YoB!fi_uCTi1_+oD-?`NSbQJ#env00v3ak}EA($L|C za?8YfEc{4U72%>GcK#CIo?-#Z_&xzUNd2)w`LNK&N~`m|qFI_iF+x)7D)dKQW)~4$ zt*72VpUdu^K?2!kTJK}9E349fDvE<$cyX%zCj1W)Px=Y>V7n{n4|+K^;VE@6Mk5Z?KMZUT`%XIaB3Q#!O5oK zf#U#=po;=ecrz z;%iS$2T%l*bK)|>l;m(9yyPX%->kQPe^N9Q8x7vfKAFTm&n;QhZ`%9wO!8tm%a2mX zkN$}nQ<~y?_I8flE+st*)!1CIFIT58Au}mkjjJ ziiHPy#`D2mpPcEJo=*rQC!9!6jUkR^PPhpZpn8?EMAkrVK%{eVM^F!)$_U$@GcwPo zRd#7ji1YTS)LQV-%Wm(KJe@4e+G`)b9qcX*2e<7Ony%7dP**d((dd%8%ZGW*`c2qb zGDQDE_&pmBUbfD67FaVN>ot_qp--^ugVSxCTV)t(JC1jn^Wn-6E-R`Cg^KZj*Ma421fdY)zKZdHMkHhSge1xo<{I$to~{ z4IFWv48fDQBHd5~&8xC)UIg%5CO30zE!u4PQ?UiuJEsNw3B%lDF}KpX^YPU$e*iRt zzL=gxv94Y30H(zAVSJG>b{Ff0@+^-i zQY{8om+ERjvpe`N?OxI8>VZRwAyb7Yr3zUH*_rDo=jKH^Dz)O8;}e2SJ||Jbm)MzA3b0 zwW&0PFK2w}3r+L-13(}=gn7$9tMLhquv5M(Btd^O3#+&+5^H4OYqKy?u17|;F44>R z+?MC4FvFU?)XJv?Zfoum-2!ngEVwp?zivB9bJ`M`^nLn1yP>9Dut(>*bfqW4+PSfX z9zr{}nt6_eVH?COqIi5|R;+m{$g=uEBEMm2#bEd`d>|m|Bn9-U#_06c^iIiCK%G7I zCjOm8nxn_v-fsIvKD_|RVgC`ZmzP-B(qOVh(JS<^Q1tZm_cEi=3E8q?u{Auan?Zx8 z-Qx|z!!QnebI4m+YpntbkVuMX@YP{~(i3nk8yn+C_cSI!Vsi>7FTefc{9!Lsk)Toi zS*5fVHK$ZGp&w(_q!a~oT*;UA(m#G}X_dUJ8h2POZUkmzM6fYxLb;WVdk9TK4h@7A z*+{vhcu(cIS~I9da&$ub4yJFz^R!N7;%`jM%hZ#O)ju~SmiP}0S)Dx^&_5?@34#Xt5EONqJ1H7)t#~wG}iI38p49ZT&2N_JjHYvi6B8a+P3%g{Q++T8_ zmKe;^TTIEEv-E{e&~j%bIf#p88n;kJb+i=vS1Y%4-4)?FsITRH-y*I9DTJU!#rY*F3jA}5~kJZG0A4#={~@E}`wBEK)wogMYm<`L9rA;N6S z&}6#a{xfzNc(8#MN}(UbBnh)O1*7bp6OvV-z7 zCl#=+rbhi8fSCjr=Px2C21w8k$GHa`80UQ|L=p`Yr-v+R5-roG=CMeObLNn%DVpe2 zE@w?z$+-KwL`IS*H6o}*buaoLeQ5$mH*HH*1bS@~mp*X9t*|2M`41G3-c~yUaGr}X zqaR^O?onndOn)Xb|w>ITt^fRWhoP9(PpZ<@=>tW#49jM zQZ|ogc?uEQ@ci?RGo&e!6ygcBW14v(hzhGTAZ>jms!HD7C6cM?%sdpAg>d%ec)>sT ze(A|=mh*z>EvEbTUO}Q~3xn?AF#CpQ3EHZ}WK8__&Sxs}o?Qr6&PiSm_jAH-%uG*k zrkyH-hK~ND+xtk^C=>cOR8FIp-{0k26xoH}(_$%F@4NL$te^x1%x6^sCgHJ%dS}W0 zv^=C+d#mqI+pqQRH)9p8*U;$v9lrf5*!-`g|1E|60r2l-e_{mUJ^;2eZ<58y13o|T zxoYm#1619Ik2aZ)7ySwBYtJZuO9G%}mc1#y7QKCRxJF*Nvit~4BR$ODK03xo4G^jx zdxWLGvcHYzbED{b-P!q`~zJn@v`W_dJCsM0@~&pM*kn0rS_cq>7tZc}Y>C%HdR zC+SSrH(Z6;Y__LK=rqLmyUQ_D+mwqu@J zxqftS=4i~~;73Gt`{(#ep%tFGAHRLT;vY(5q-ZF`V{fd2aN=%1!FBIwD{bvGZ#OHBkSM&pwN zKVMqTa*3GxzFZDr zxu`zR!}-SEgbk~5@`V-l^5Uc1(Za&1DcwYSM)5&e^-5J z|3y9lFRJa$XRkT9BwJX@;vcbZA>Q4%5Y`s&!H^@VADsLs9>(Fg=Taa~MIO?;-^O7Z zM8|V&9?M{ke3^ZTB-be24cO`tyr`z<;7p+sr_FoiMrFR$MyEJa#}@0T0B#CgI=k}b zmREkRG%GEl-SjO1wg_Utub3D?BaDpslfJ+-MQIx8;=sh zCJAIAh0~nuH^;+6y`=-SfT7kwo|{@8@djdV^FicPyZx`7)6_w!e)_A7PM-nJ@Tj!G zKOozfNBdv5+MF5S>Ge@f)g4~sqG0g~-`V8a82NN6@6ft>Z&yx7)D0U^v9?&mjw!#IB2T3w8FmHu)ttBLs9nAPf!=UO~E&heEt zh3zidq)Y$++u#2f{Rg3-y>DI`pQuxjF-?+cP_&qI(yA{8r?eU`-RmK%DLZ}OI)acTI0UhkFzFtV;Cdiy?OudCs!fZu!fm@5%i zC7*-iD#8$s?8r8G%M%;I@fl3n>H{N>VD72bH@93KLkknUq+q0=#q>2>z3tGjp%;+2 zDMUr;zR`V^r_iFMtI9x}Sfs2oSv76tXJ%Q*1Tx)cfOI`dWO>(x{+`E?YBZzm#hXn& z&??qWTn|u3rV`hrJSWn9>JC#2acyTX1?--N`}p{v`=@4v>F#YwLdUE2S~k>YJjdHX zm5z2rtxay5;^a$d6TQRsCt51mmIZ4SqRw>KVXq=7=VgV?3us>DReESsn@vG_6PGHTyHic{1@sjBww5G(@_KvNO?&KQ zZ**U^CU3EYaz}w@O&oTFXZYJ-h&ck=y#Rj6%{@~wkh!u%gxE`=4+w+EOo(IiQSyaD^1A-p+U9%6EvL> zngtaTMPh4ssBevradAoFFW@p;=)w<^gDcoM{zS!O%du}86XN`JlXlGL()BK;Bb+X>t|Y@L%ABZr zMe+Lmj_UWX>DH6ReZ?HCh>3b0?MzEf1y2{*b1CqI zY4bo}om_N7T+xn&!%D^et)6#(_9V%R$w3NLoT(@Qaz`&CFI)!@H>qvEc;p6a4F}R1 z8&)w#Z;;yB<)93w^6%Lpn4WP&>I4gNtC3CgHk zL;3}kk?|JZd8LIpK5Y%nxi;W*H+fgD6|Q)RlxXMmmM#D0Gp)UPQKbeN+3B|Qv);-4 z@v|;iTr-h1v;9Xh+eR2)LN!j^!p1qtVqP@zH}ZC_!MOpt(4et|muAPdI&9UoE_z91 zzQ5qK*V_#fmy!4kH_RwT)8}%+kaM zQ+qhq3eBc_))n^Shest;Z|`J9lZmaobyZDbs`OoJt|^g6HM7#^_t9N3kpxBCCH15U zR-SuV%w;X$c%1ZCZKk$8Y-fuOPVrYFhmXJgbQH^>z9Sc%8Kb3JjN?0GhWx$)wLq?& z(N6^hiCKB!v^IxoXaTJy$r9ymCe}WVUObo8Bsu4T6Zj?e`K3`f1r_$2Pc}N2d*G>3 zSN)z~&XE8M(@@Wd@i8Q~{wL=~JKC`rk2JZKMCHZqX<+?fP8QjG`;?;s|3Lyji#VxZ z@L4+!9?kpSAC7Aa#8X*^@u0e!^UO3}5jGA|!9)$97Mrbeiu%AQo=V*WTp545q;=); za!bXUfo(J4=mFs9E=N`;!hEvpo#2w-{Hl_u*@@y#$s!ZMfookwqJLNYLBIQxVtfKG zxX{{YX>sE!^P6HzDXU}R^GdZZE;>=%%;r6=-_FJ}-{}nN-5=JGnexUoj-KciF{7Pj<|mM`6|88#{01yYR=Ih5iQ%6vsTu^oUQno^;c9w8riIL& z)ao1;(z+FL#}}H)j<|RLj4_USrX9FCY$uxKnNyVr<){)}0{{ld^Fh7#cY>BIa#Z#xPi z_=nPRqMrd&px><=Mt-)7Dbq*J()t$`(!>RpmKIsWEqWIvBt;^FZ2r3t7@WJ*9H-!g z*HD}Hgm_{a)?r5VyCq(>nnCLD!f-s5UhZg3hBD+Fv?y5x)jw&$i83 zXY2&0=}UL5;QYAxKKyw-Ja(jYJ$!>fY1IL11I;U>}|FeTqUY4Ot>&I)oYg zchPwX0;kNqJnua!+FcjOCj6dRnI16`n{mn=L)YoKhq*8Uz4gO(w-x(fJxMrfl(moO z{<9)3fykaggTBaLujIH*Inn6L3&aVxl@r%=sfpyg_|2nj-j!e1@)s|MbQ;LS;HQ=YLenvtwOV3g`6sY z2D)RwL*0S^_1TzW1b&f<;PzGra;SRIuZ)RF)WtGy9x0~k%v(hZuF(N@L|9x@rhQXi zCd_DQfq03vPpH6-m!UoL2K9A=?~T$EQ~L0oDO+ES=viZs43#!&G$ficCR>?5c0|5g!2~kc>tGFktUPNnp#Ygsc%#RHkj}i zv9Ksic-EpeXZe~XX31M#yNW@KA24Pv?TbslEGOf|p(e zIQJFQ9;3vy3z|NahtnKdokjZ7JFX00e!;;#Rc|B|9^rw|Wqv&+6)g8YsW($HDo^B} z?;+WAEi3L!Q{^I1K}#}cjaf_LCF-qQ{(aRjWs+k&=Q+qIIMN916>npllh&UsS=^7N zaPJr8j+ibmYz7W`HuusY(`_C@?jiNb`N&JzTk(UU{fwvz6=-w)3AMo#a~8DtPSg@N zFW+W6wf^Fo-;^dVB9zBIlc7nlwfo{q+Bt&N!4fy?wSSh2ct#puz?l5_v>9I;BY#df z0?k6zVgbj{?Z%^?$g*|!?Fr{bR4J1RI>?}CkURDybM0Vhc^<1=Nz~+;tUrxBHR$_H zXkt}Vy>(F(yy{u}`(>r+)hA|I?O?@@?WU(b7_@{HJoS6b|{9L6g#wlR%6X841Uh&MK5Us0kN*Kv(yABub$t|HYt<6dz!qrO)&Uzjf z?Pn@xE9#>uQ7Iv*$gj#dGA%N_^c9N@`r}%%7Q@m+1F3LsWfRPwVsk$Pi{u5Kculx` zYzFgP{9@lD{5>jYv`zt4!k9u#!wxPh6ZudotV5IwMfl{C(`99m;@UD(>jTGl&!&OGD7$uM(%?l6CMMiMlCJ8#=UBxA1YKvjJaLCm!?7; z1`UozuZxX`5pa5yoj$c+;@I_|lqe2P!={pwwT5-d6nkgPqL9-GGINr4=D9a(>(9-_ z_}{H*{USn`EtWYAHnQ(r&9T@iC@ne)eb@u@a6YkqI0QiUbrselUj0XQP0U)o>_dvIgw~^%d-NEL@vS`3X~NS3OMSr*2Aw(*_Dlx=#zK|< z0}BA)P9*JD2#ow#I03$h1=f}Q56aFvpoyjL z_wm>fRHS#68me>wfg_zjXrV|~LXj4F=x{6`9f1%?Xi^e7NH3u%5IRUl5(3hj^xpL5 zJkNdaeb4jWfA0RVncZwAvpc&p^ZU)`_gy;*`#qOlDC7`_7z6sjcm;`ug0o-PxWB!` zmR=~9okT5sP3y>Qr!w6fMx-d61NT+yHN_?!N{{`ubAI*65(H^ zGn^OpC%b6nR#Ea80cC1att47$p{Fogr!6yMCi$A_3EXsz?uUncJW6%k=P#r;Q<4RX z%{ifgwX-cHnzfpmD*YWePHZ9=Aw16?)w2uibqZKj82wr7JYn(5)wS&#kK>Z0uT1p! z#dT}XL;mc6$!U3rTS{VqZsIZ9a%L3ZU+e-wHGS+KSN%Tk^~rEXyxo~=DtX*Xh7R(w znCTJacO@CK7#bbi+S_bSuT6Yt{KPpF=fVDSmfw5$5k%nqKsCmcj?W;qh!z(}1XmNI zEUL^r48lb2a{T2vNI!x+fm)#@se>sZ6LXIEMQG>4FN)xWbeFh+g9FF};ExL}av2C# z1p1m%wUINh86k}V zG#{PPlPlbd)t=SeFHUaVfR*`mu5Vp-*g@$3JXUsBhmIvyDaTKJ50CWR89X|i z7gv>;C|Vl50|Q% z>gJG@G80Xd{TKmlPpT%3x(tu0fw0zH1pVHobl3f>G&{zv{1hx{!w82UummVqZhxD zytd*ptz?n~9TF<9rpHU2%G}^aIP)|u_$TEIL&O)8>rCwvKt>PrbxdQd?6(nulB*sa z(li_2Oi%ew#+Iupr(-{4m0y@%Veq#zMHM8riCWSUjo^2Y`gE*_eCa+$T-Yn`u;pe) zO*^pr!t|9w%Kg{YR5hkH=C$S;X@zIECBWK&TxSWIJZL=7w9#+sD*tpVPd2U-81t~# z(H1pas}q+fQV^TKncSpnKqZk9mn#_xEY%8tpVo?bW6-fl9{39H>Cbh_60A(dWgN9) zJbK^EQ+s(5*Gf=PsD4x(L!+X7a&>8Z^}9G>%2(t;l1wU-K$+)`d-cY9V|SAgH$!*> zg{{6yQ(^{`PHc19U}qZ2()8@}Of!%-GQ%Szk0|t z%;$?vFE)Rfp2psY@t%|9q(Ent-O9a2)u$?k=+x_cvM*n0YmZ_(Qy!DPmEKTu5Bvon znsX5KcN0Z62S@+B*^4P0o!c~oUZgtFKbYXfw*1>XSAK-7ew;3#A7?*Z)dY+7{PG^` z-;-xQkKv#z#8rR^&8m~4y=mHp$3C8j!V?6EbQEK5IbmEP{3bwGRpp>)OGU{rt|u;F zN2Nb~afji!K-#zoXhtTHuL~&x#=JX#Nz@FVhl}#>@N%`$OI|cpO;jWZ>Fc+r2-D^` zN@;{m+tJG4FpoeGW303tACP?$XL%4sa-Zy|)FG^zQSG_?>r?>LtK1mTcZ@R^rlcx3U^x8Ta)TmvZZs4Rw&Sn z2f|aG(%gK*ItXsKKWe`!0gIL@csNz0%I}CJH$Pz_?=gH8mRn{aW#UdRwIB6{x)i!e z_qZ*GpbZdnQ#zb_S7KOVX&wRqRAl3JSbx7ztLp>%)liBD$>n**SkK8gt+!)MnTZnl zn65KS-;OcSxoB4ohcB(5cByv9fX3Dk&z$-DyPz zy<~uFvz+}3*#WEZ_peMW>Rayv!Jfif>nB>!oWJF1yEO-K@JR+AmW=! z_sf1h*;JQFXM+O0}(??o1@GDWyB9sRtuxk<1NR+s2npV%!zKjzsQBA%*gFB$9 zry)@Pq}J`v`mJw{5z}>Up&Hz1*|k)!1?@{tN%iiY!>l9%#5c0KlP)&s@-e!gbQRa_dL1GOxf1J{D7#-p$ zrRHP2tWv<+r_Zx2BGPDc{&behDk3$kuhf3ov_8Sz zelv?dG&sqd*#1}2z$>0JAwd-pzC7#F z&73b@CnxBuujS4VpnUO9@r29;UzM)C)65shW63F*>CFrI;B@&egN)VoL5J1xk%?&L zC8FAK_mTXe8%Ye}{@m zT~ieNkSunyd|g2G$$A5HFrNPe4$fy9lTG10caX~%Jc93CW^0OpaOqYBTzOER3R3Km zJCD4;TIo|r{MQAR z3d$;8iYr0B*SIJqdK@mVcQ<4YXDa!;vF3^r@L~c<^E=k-I^jvv+IXq?)E2hUvX-db zT?&UPt?H)I`~sK$CC=3>Jpj8VHk};75F+!mdo5yRSFGO@gqMA;wuP?OE_&R^BRtAe z2quYY|4l_Hz8-0bdVAkcbvh6Br9pM(jj*#Xh4tPy>s55XbH-w3pM`gPE?}<4^y-IM ze6ZsL6Pc(qULNTUBYxu|7DTQWug`h$+Rq>Sgu}{B!6b+2qwA+`DwY$5qujaNP>wvK zv=)~ld+pT%h+mXVDxTFU!V5ecEixN{bV)}y*mBOt@?cO@#l$%k$;$uI%5-|M=>9 z=)r$Cd;y=YSYOg)d3NRM?p}!xm3|TZH$6Tqb1WDj@f&cT@p#`S;*T%au64KiaHxky z2)3W%hRVb#4AdRc((XFz#QQ!wCOWRcTE@*D`EDsS#I%CW}4D%B*6 zOq90U5ZwDX;gij1bE>n06m_zZaDu>TF+lB{;bm#ui3>|BM4K{aDRb(S9S}A2Q;knp;`Fpk5*Y4aHCysAL$#jn$ zoQ}##I;{r0G=go62IeJa=8aOj){~}r*Rw-r{e_hc8WE_tW}ll6$9|#{unQ{RJ)#(~ zdn)`O-z0SptpRtNWCsv!5wJE)ldDQSH`dWg;xJ6=q|j>i6Y$IU=GIUUU1$BoQUpxQ z5;jtjB=1{v7(gwXAe$ilXq=wtBJr`70>0)9{sa&e`~2_f`n%7Jrx?j~6O}ay(*cV5+XeZMa}{@wt`SBt?U8nd^9LS?@KH6mY+ z7gRZHBCzv(s5Np_0XCz$oMOoL-B69_ni+Fbu*8{q+654$H9Vf+P}Ws$tgK?pBUFaF1HSrJ&@WdoU)oHBGX~FG&4>_RPc?U&z0s#r+`qI5%kKt{l}E6>qAl_6?)F zvs;WtdGr;19cYh)=Q)9}6~=eJnk!x98fxV4sPr>kk3R{(A$pa51HLi3-*p2P#9eO? z4L7K(hs3^q;iWOuiuf`}!P8LWbf)#oz0fyhC&s`PUF<)0D6(bfob)659RG{bm7m8z zZ8A_l#A=>OD5V0Ya4i{^TUrAqO$jMKkpv%E=!;G*2=|%6!uI69T@_IF3#h6dBZu>} zS)v*pFd$E?=R`&|S$ic+GO{8W-&j?@*NrC4Z+K`3be-S}>Tp*`rp>%ia*Uz*_Sp6t zFPLP$FJp3Rb@b_@2(8-8OpewL@BZX?m-N;+kqga)>}ZdLjU4L32HEzKk`GpKaFW^< z_3~qzg3QHaJzNqQYbB$PsrFFE_FE%qW(#Sa{D|LxFErvh9Q! zG9&tqoHT zrJA8o+Oed2w!j!-B(XJUB@IW_Fm67jNSA%by)`-`ulD;ky0& zOr-0zS9Kje(0P_|HN!Ap7(0&_Cy6HEPN$5My>qp(n3`(4Dzz8|lRQ>~s5M8d&TU$o z&2?4{Flr$#PP4pEs1@lPc2er|~TKkZ#>wqXWUM(8KiCG6_3KTdm# z?GYcRVS7M31{{?l5w)%F@2m?DCo`_L4$jWYiygb_MCDh6sJplOU8VVPtqU&s1Mhe%I;jo3LiUq>Y~YkMmw zZ(XaeQgzv4x1v{FEVxG?&eY%4L|;Kut4C_HQ@fM!N~}ess-&hh9-+TPIT};~BdRgGHF@Jg`qAPv<4d`>>7Z?FADXP8dw*8-AM>TaSe$1_o#3^l9e9lS^d>A-#iS)xmvlvh_8@ zcgZyxo9DJp%;i%&H{r|+UDtUoUt7F?Cc1W^d-^yrxBaA(Em*KQ-Vzxep27 zIgxY}P+>UF9V9H-RlB6Q$ihAxMx0wu=+13F!F<{p2QwQWvK=pI=|29rlY2U%(|IPr ze|{+a10pQ>rnr&Yn$Wjrz#!GI6&YJySHYu`CKUMECpDkdio?8|FS0eQ-YbH@G-ylI z`|ERwJ$5r^+m`O5yKeOq*C_7^!-IK5{wugaRvYac=#ryV-|K^YjP*9%JztV`f&#_j zZ-6K)jk&+owzH1VnrSsJS@ngYr$n{#t}+hhUAO>cx50~-hWg)|q!SqzTNW8Vh#7Sy z7>}pXrVEwcWPst3xF{0(x<*>D)u_8j-uPkKsI|r>cXqu=U99u@*Xn7x&Qjp{7?V#L zh54$9<+GsQ02wb6Unh%65jA@`OaH<`lEA*sV7ZNL_%DeCP=ND6#Jw_->de!`p6Xq- z``+E?%pMVjHh^5It@C_qtY+mG$4Su;-_+ChH*FUv1gCQlbDxB2AUfh zs}ns;ZCea_dEje(;7*x-*c!;px5()TP9gVYROFB)!IP)43}KnjycqdNw3y+Vn0@%? zG(M=fqTsZQvqp&>*SK^?BYwTegB_sd1Pva-c)x>(wL zx}~9L0Y2KS7zxDb1ZI4E*SNBYEV-^fErZK z=?uTy_RJ8Vb%{hX1_J>8XD8mnjUV+g$Cfs-uKP!4E#I;`O17 z;bLK;v81p5RXIfCv&@>qP^36gWpu=>s2hG*3?(KQuO|Wt6S2*7&nAov2M_I+{npyU z?&}d7t^>iO#Y9% z*Ni{=M8|BKKakd11A36erMxidypP!6TKM62D+xWxG>>g9eO@djFY`mZKowC|JA>ap zsG$_T4p{QZFq?UHc-={3>lAT*JU;sFsg3cF(aTUivWZoAQR>T(XoqWA!{zi9f_69a zUy?ei=kyo4Sm(XruU|CO)hWW&)0==p3B1@1j@$*lS83MTyRxHrF^uHl;|dFZ=F1qX zD6Rvn$2euHrFOWc+92gL7Oa#u=Gy+GawuOz;Y!`1Xg-PCPDsE6jAEY8sOMbVHXgy( z?_S#TIW@tC=?hWCd-)K)oExSp;QCo$ee-Yt9Z~0-^*xPQv)%md_z?NW2a&GrW1BF2 zvd2@erggZhn2KwiL03juB~D^E#I)8vgE)6ns8ai&+B~HUD89s<2QsE)V_2d@2b1vI zeA)0#ps%k04xGV*nXrF0$(D3I+W~dI9_(V|Wj>M;bS7F*<{YN^p#J6D(_FrypgaP@ z5+AA|+#FYp5iOhDfrkYfmR<|;A-z|48{0I|U^Y4k&pnf@c~?L(GB+#feZi2c`?R|- z+U&(1GI`1UTQLJ2&wJb?w?AvP4P{Q)qTt&=L^f4}YIL0=2x_O!M*D!NN@%rnWo&|K zxLBJvUNA6o0@;NeMcv$`n)F^QdSW0-hR6z|P`8!&>}2fu!=>5^tah3S9;JSxVU8Qj z6Un8{I?pIL58l5!le?Qwb_=Nb9Jo5NJbLYR-#Wl8%|DK(f>M>lbIhut!lcxbd8Mt> zNP;KIehu+WUoNEhC}x zzzhZ;hg;~eYkne2*~`lnt}==!&Mj53#SLRx-G!ibe7AXyC=g<%bij!9qvuc&4O7B0S(GTRjI8muGSBl$&L9-%7iU! z%e+?kIh{>=I#3=$^fr$2ukn&3n8fsA6g;T9z_#?~`*y_QHbXuLxM>Mh70$0tn@|(2t>VLt7^llUJyY{n zh4gk6EL$i$_)wW=``+WNpZSki1KyK30(x{>A%B;qUrRiA_M|P-otUEH)9yT;f|$cR z!lJY*+~3$TOew`MV)%R)6LZX zSZ7<-;IP(NY_ijsZiRgj*&>>|#NMxE=`x*Q_iW_5Ek#CsM-MU*fft>IiKRsuerXuG zfAybbQ2ys8xk&$Ey0=?Lw&b+>M3V!v=a&U9sThy>y?t!^XQMrJ7<%P-(I*YN8FN=2 zxmb8)s&=|wTvD`6mj)iZ*He<^&!BrZ z$77YH(G3Z&plU#c3q*SuaU+S66U|eqcZ_;l-d{|<D(tpOX54e_NJRSeZNXqFU#N)XqjG zToes%5ci>(s9!sy{!d{umaV$?XFwy(dpnoAn-#b?^<+=l-A_SD@jv@VM-1C`S_IEc z38)rT!;!=!&B|!ru`~t`PA3>#BI*K#fb$6#{JYp~Xb+!N}SamGt zk!0myJ(WCi@6(uU8bl)s!3$fPzz+q1S!x7@X4q}#mc5a!jIzF*=U+1v6O$?zEaieK znm~dTLt;^kGrKWjX{;tuP?Xhf#fz;r``#5J6KMhO93nR3OHtkuf9^a2XMs{;Kth2y zk3j5JT-|5aix-k)coc*3^{q`IFc|J+jxFZB z$YKT3$GUz?R$mR2tY}P>;*~ejFF-A#$`-lgZtzPZ2Fl=N@kz+nxUu=S;{&@-KdmZR$rr0i!n88}J?cuMIcp`h$!?Kr7IQsIR;JX(0QiOcxe z_(b`xcp!WfpBH$?%_0G=n&ff%)mceOCrZl5duBIZXs;+0D;EN?=2c>8ofm2$p+mmZ zEu~$F+l~>B+!hR__Y`2dDFCbX;b0E9-vLakF%suVh*qTqY8#|9LKz6GSc$e&H%hUO zvF#_tEb~a+yO0JYQkW~VI*-WvUb;VMv7){#9ekp7?~NKaKdcK0VFNiH^Yi?Ioibj+ zW)qve_fwJvj7)fQ__JlF-Y>l^tf4sC4*{Wk11{CH5B?%TKDHzwvs(3*X{uUuX4T#D zJSkL`euR4zQhkxgj&kw<7C41a#HuH&f_np00f2|(2S`y65y)ihma{lj^D*8tA;Fzw z+neKzwb>;5fPJ+g3{y-3xoI*zbI;x)-|X4;rA(lIPA0X9#gXxZ7vXC|gQdwx(O9#R zFRgq;k0Lup-_UswBk3m2EZB)s6}8(wo$&hc46&Yd%AcM6c}>@A-MZ7&+Y?1z=!UbN zWTj0bMT+i%N}jEPS0n6{M7ON#HuQ$_ee;QnQ;^}Y#e=$djJQtK{m$>N>SWa=TVKy` zh?uU~)GS!Bmw$R~w)1n2@Yp`OMx3MQ;-<6R)y@Z#EeWQ6EWLl^7sk3yyA1RM^~DBu zY8N4TA7{7tHx`g(EyErRA{N%l{X+iYE4C?!thtYqk`Y(cp@{yfqo2qt+B?bdCxGGV z7xIPquVqg6(|;w&pa0Q~w0-#>9^n6WFgbZ6c@y(m5^&x8Ak|6-hI6KWjU{|^ADQdF zM-s0ed|LxHrWbgCtzI7!4aDaxS;iP|+m0+sSIQt=raHx4IoQXmZXM9@^W{X@fhCfF za0CVzhyiN-jF__pXFYSm84{3I7++_6&&A#Ey(ZNy+F_<3P`yYcL`bY8*NYKX$ICcI zY)}Ufe+j9<$i<~N6ZqoTnFYR) zT>8``?DZ9@CqNv(E&71j`Wx- zYjS<;8u66ujeQb5F|u>nSSCxcUcZ3UiMCkIjkI_2$OmAa8X!0vz@*Ra@N#qh;eltG z1(w(!3eOx5hHUOa(GC2zSP$0SX6oIt*G>Rd$AfsTzw3Z(YMu!^FZet8UkhD6v+-X5 zGA6#;T%77Lz8&ROu+48;cMV$L&joVD%K1RV?8WSl49V&g(h*pow)L$+d3IFGTHgr( zIcSi~)#={_xmz)CHs3s|m~3OXj!4$}nS9Q@u-I#DTOMU?>)kSHkoEqh3sg3UX5@}j z#@v2ON}Z<;vXGJ95$< zpnq{E18^k;&G+v@|CPG2mAgT+d;LS}(@MT9o5J#-RvK(e!n5Cigi0&F5`)VJaIrz{OQ zob->@-++Gz+;u}FaU19>sv^wnCMEzw$49u12%ivBAgX zDzj%$!byU>owzJ{qn$4JD*7i%s+Xw^lYe2i(dwt$f=oNAyPgzL?R>xSeQ5Hsq2!RS}9s!GGVOZb+8zh0Ur?O*!-1>gXJ7ynhMH)`gc465`07}BSAs@;LTUX$qk z-e>8#n|1xk_oK&JOwTPKAhq`9qN2j3n~{)Iv{U2bu%#`j1v;_Gqsr7e98rfZ-+`>@ z0=0+_`@Ci6$lN77-|v=05SK}Q9frR}(1?w5qfX*qOT|cgH_yi=RB78$AR&%EL#2ASk zPaUCgR_hGrWaALbJqZk0H(RntrF}N+otL;i4k!&4qn9ke~L_9O||oEW?D;o|WspE2*AK{F>jfc2=@@-jDNDI>`|{nSgm zhsIObpC_68DvFp&&LoKcOdM-}bv|k1(%%2HYK>R4ltjNuqaT|-3wgR0u2P$hOVFv& z*{ibt`tU`allv~Zbl9}=+}D@L;Zes3FO^ zX=r$9-8@x)SlBXjK(=`pppNG2_kg|74Ssc=P6@eN#4x{V+QfL5;4xW^Mp6GBEX*d2 zjKg84-TZ=IELa3G@CNqdza!rV)eIPp)u(4OzjgY?7`YeW5JihiA?N^8P&gcECtChn z8@Fxb?r_Bp*GDO7$+(_qkr>Yrd1^W_^_=E{42@0s2i}bO)SISsWg`tc>%&Qq@%r6E zEUVft`fsakh!ct~Sr@CV#wq?uDg5*|R*l}0>2-m{I#(J^iDA>jJH~6t%0&Z^aT)GhhoM_I6q4vminYd<5vRqb#9ynUbfDN5gzqZ=YTsdT$xSr zy7TL#?M32Vi<#$HNk68(hqyf5dQYe=5(RQSf~VRYTT}3xost9-v>p^E`=}jECW0zC zedH}RMVxZ2`VSu0uud!%jf7J&o}iK}CPTQEclwH}3jMbE&efUJ*0D7Qb-_+nYj$qd z24-@uro2^U5kuQZ`Ce3#Sf?KT(mjX!akxk{e2zCyOh*AaN^{*uvBl*Gnbs^`l4Q!A zHzMG`??0lm?#xKkc1waZLIQb?RihU(BpknqX^YoaZe4GeF_?GxUh?r{{bsnBz^6Cb zb`%MQmNbloTl&hbBEBB6+jr|ka;YBaR@LR^dspSlbW#gZanME=^GF(~)voZBKWfkp zA4^&;sZ6ij6kej0j_71;{^sHr&{#M=`kYHA@#de)q9sEeo((?Y>8K?%4=55>Qu_9* z32jiC#iXN`<={oH)P$jYSc)#z-|jny1TV61==y>!YSJwV-^51cU7F@>jiMzM52 zDc?B!5M+8O;wn?|WJ0it_3Nj#60rp1bK~&H=U(aX2wRhaAjI?f9{KFHfS>1E%;kRH{@Jn6A=G%8GkR@bPMXSq%D~p6O{a>ITiFc4_KMOmQ^w&anfgKtA|-rp3sG0YMW8 zjI(u{OxNr0WpbqTz!F#ES|imS-y)bwpF&f;la*IJE}$^QbTtF4{ZX+;0&wLW-4Al} ze*lLJ0j}xIOUxzcVuf73)WaLluYOM3A2m&ByCuXnr9HCdRw&0!5P3>o4Q%qgf2E#n zwds7Kj!s{QrR(#{*{NXem#NLSlom*JgdhH>7A&HmYiC%SY~G8IQ^mbK4$?qa!eeUw zu+(3RzHh(cv8rsbSAli)P|G!&__)7Gub-YKJTv<-V2U8$TC>aH{Ol^ctS(*ZoXxxn z9ji|>%6y>MTTpuN_|d>Q3yfjx{!jWGtCViLb^bN&neusuaoF zhLbwOTY9JV7%aHv$t!XNuUVm-#MF$Mg9{37Ybg2u^HB4kR4aO5UKfg$C#(XA;b~I$jDQYm*?c#3+nbfmh;PQ#;^+8O^7xxQHpD> zS?d-!E7Kz|Ph$^C1&Ghfxh=I&v#tvacM4@yvzy@U|F+|TXxSr_JQ8=y& zLVFi}gSZ0KGaXExu{*D|MZV&hkXow#KEqX=^m>i?sZ%BOW&kR>8c(o6mySs{y0#ZQ zqyH>HtU02Dexol->gpy>0~B#X(@Q>LtW zzN)R7nYFyoJ*gZwjbgiZdA*f)roB`xdZZKs->HFRBI`r~m*HexXK~l#Un^<{F?Gae zooF%*zcrGyWUk#-%>KP|ZjFDnpdtE#tDt!jP228q!{`@7=*McJ1e6IR@M;jq+TU5; zOb>%z4S@ti97mQ?DLdRb)_Xe1=QPXD?f9bZG5xxNi&~dbb2@N=(6}N8PrRWQo^M_h zXuhXK#yfqX&$udBOTBBO>I(}$sSI+3F{Ai%Y$GCh0>`C`(}=jYo`%FL6_TqdW~Ti7 zJih^Sb|%6lRWm<{DR=V>K3HfUd^)fab(-Z-QJ<}U62ACp5ParE9T>U`rB-T$o<7St zR|AI4|FGzCTW6qv29aFU8s6JI8-SKGH)384501F!2;}w?Eb-Hs^J$LSwqv^PC(c!x zvH^Is;Mn+*^MXc-^psF?=p5_FhL%=Vi|((_em<;_{PaMC_ULEvAVtfPNX$o>}D>} ztjf`(4P#ifaQsW_^k`XRiB_Pni7y&KPdL!RyXwt#hygGY(ma`gR$6W}lE za*nURJiLCD^93zk21Gx0MQRw*I&qzcB!G3gm$vt+|WA!!9Z(YwAmZ$Capg ze6o|ZEe3W{fQ1D0L>VA_Fg34yUVU)QmMo5(eyL`!S32A}pT%sC_k;R^>@XB)XI0n& z?sE5eBlomHa$6@On0RZ#pbP`UUXH$=BjuYrrzDwp!uBoqc~^Jfe8WJ$++0(w zL8XkP9+qtgZnGDjFMu#K+N3sbwjcg2L>$AQ9l~spA`z7+9V1O6Ifo?rHngx>J|lZ||D2_*U-Vw@%(Sb0qh{w^_=@ zN4+g-^}@j_W5V<4*j?HVn!>!HOI68W0f#rka=n5N#v4AlnOhN>TqEiCrTVMr_Y%HQ zm(!A@OvMTk!_LU@9&U4N5r{d62VLK=xkSamt5?w5yjo9@ot{;xg=XpJi1i^)80Yu& z&I7XB{U`l4eA5sY7RQYE;S_Twy*xb#$z%{y7bU6v0MOj~kG|+i?A{Tm?sF$^&uP@Q zaRC9Df#de<$xNQB$d&4SYB<}W(cTfSR%6k0KZ?gWL1}Vx2J0~?Tu|AA6zH=bl+%9h z!V>Qp7N2jPY|*k7|M1c{|2LpV^E|h~pb6NPi9S;a8Nt_i5M@KVt~cx0xt$tESsaVi z2op+Wcq$Xc_(LH1WmKn1&)u;ElgSN^bhf4i6B|lPOK+orvIPCsH%$}g2@Fj?1$kX| zwSx2x9goXtUIVE9wEpvdp9bZZ|9d`Eev=>WO8$`ezkA3j*c^l*_UUf^%We`=O0dUF zu8$@5Lp1d*)3zslN0pM&ulVP~;!>ZJnGkTF#<5!dW+IkxIbCb&fD%Snk^+wZiu96X zFe)+K>U!GivUIetGPJ(R3ki0F*OI5$gEw=FFDN}mOX}Cr(sF$3UXimk&8mMciKO6rcVf8=cuK+Lp&!PWHNCo;_l4pr_3S6}n zF|#A;5w80NSl3r`4Gr&7rZ@}Nu;3xX*%yUG-CpaW%C|GYxrPv$I-`17mBR|D0KLx0 znu^0z<-oA#Os4X!o%NJ%z4XN5gD+V9H=icGVihfJ|I4`rc#@Py{ao+7?UUTW-sS9%D+08u2jYG1IevGROsjJp&_~%lDnV|U z@H@}ifypdFgm!C?J6hwZDQBRZ4hn}Ao@v+t6JlGwnH$;Q_8p6fJq^Dg3;~aig`KnP zmwtFSgd6w3Oo}Kho;$#cqHbq5v}w?tbkIQhXTYSwi++u?d{X|1-q*&ad3p15_;A>=1dwe-X}CIYVS{{ zn|D>*6|Fn5CSc=$W$ukK^IkQgGTmD?`O;MZr=|hlK*7SYwO?M$YBMF>{Ij1;7$GAl!^r>Yg@x4JV{&n2Ld=<_yrRz zG%;jKJ9-?0=JtdJVoVi@6#S!_XSO})iGiOkt99{3{k>?ll?HahJ_GK|?PN=3mZqF= z(b|vczI@pQR@;8~qm0;HeA0)X0w(ke5jN&W+TvD4J*`xoiq$#2ECf?!n7(qL+LPig z<6`)gc7vCVGEFwv!XB}s*Bm=(BfMl#WX(3mz+ zcB&{$Qy%M-0?*`E7Tv;6fv<{Il(*e{yZgt>nN3D$$<)h{d;WGrYksNHv!1PON=t+g zLKKGCysIgE4#^tSjx5F{0GY}7M;+}3GCR?u8ar)7tyfqS^CE`fepzP!KGIlVH$<89_lVrK9gYr5H3R_;!zuaEt5)ElBT% z;VyqV7$2{P6?q8@d{AvOzFb%Z{89h;uYc9of31SXM3{}6RBgQ*2id5oW{<8K9X)H! zcP;TTfYhN!4D52%mww&gSCqb2B)Ac)*alp{gg*z$T>h~zDBtD(iqy8Ip!k%aFo_~| z{cMDPnf)laHSX5u(k^+pxj<_mBR5=BxGfB)&@pS|OX>CGZTb>9>Udi>2f^~?=5p-T zfRW+PPORofz?H>_|0>Y``}2=Xbifw?^F&&m$q=V-C3UJWY7gJ#lHamXo7>~xMzfX_ zX(}VtYCC;&>-ar-VahC+Ga@VEmHYGlFCfkdXqLDUdhL9zfE=##3mC5Ye*0b>3AnMa zrg9{&XxH7EpzD$EGNi!0mzEGRfydMMnj;UVuC2yDvi)9{U|*wf_O>utiRA=j6X%B~ zwb%XirkRmJxwt*ISfw&i3pL#rIejYboAmy0h=I^T(He{OZc*nM4Xm z*LW@odXO$(&YVG+@1{EXGe6!{o#O!aq_I6f1E|YJ8+-U>#BZWzyBt}1Jhrfv#zxQ} z=li{FIN+Q{dTPd6qpm@d9dPxo>@{-kbmbJrpClG^W`=Gm^_=p2YoDN+MTpV1J&|&u zq2uCWBTIy+^!>yd93%8P324U@4}=~b2!b+V7@?r=o?8vVx^YrDLMg_-g3@Ef1w5y` zZ3=fx-1uw?UQ{2Y;`b7X5Yvssw5AG(3RX;LRJ2i++Gsqk5D^A9Xc;YIL^z%9@>zCI zpWZ~8P$iJ&^qUOsaQ(pV9Ik*bcx`N=c^5!O23ajqN7vg+Z9}A`HK*^sCQL|rj|Q`7 zZ#f58`-dhl?cI&8PELmT3rzePol(fbu^0Gc$zvE?a`CbA`bwWF?Zm>5V(UDbvxi4l zmxi(h%u=-B)B-1L-J`z&Z7M?;M;-Z@p*geB_`6>l3oC+W7>Tr!Ns_U9=A6Uw=s356 zHhL6CpB9*9_BX(_m6LhCUo_&gEo)fwhYEag+&r70Q6{}4jTbu~S((0Zov{~KM;31E zBv^J5=b3nGTe8Cl>n!&g$5=CwETnM-e+NfQub8C7=?Zf9XrUj&^k>Q_lF?mql+gJ#HRM<8n zcI(n+tHN-aO)xIgVcn}y$8+7c)UefHs8UMm7E5-fvzg8y_i^YTtJcXnPSF-#{#ppuZ+M z?D%=~;`~uEj0MWX>6!Z)yX@JM6&3hMtjNxkM?06T0=zL!iK?S59f82d6`iFBZcGm8 zx{G%EnoDMoajQoMaCp>?F2-8PdBL%yvg?UO}<1N)B zZMX}a* z$%`c?)d;OH*e<%nT&p&W_Ok^+`7WjIndRWHOTZ8h+A$G7%q}CYUXW&_efYdkYo@RB zBunr1v!*gz%4LFsoO3^M#`8XA+Km^;GtdYdsB5mFVI(Sdny%cVO^QqG=&U{%di{sf zgZ;-k1qZvpq}wjblxpk3_yQ)Fs~9#*0r zb_D?9<>$vW}H^&R@dFxcurT!&(oMwsRulwX6+Us>twa#h4jMB$5oP4)g_%* zV|Nzp<2#{i(m!Bq$qOd>W8AERXAO!z9b@@9>3=$cM#BWmV%3smGydJ&wWWR^a~uhK z7h&d6d~rLWD719EOhDT0vU;|Y;Xx>ss@%fc!%W1pYn~H0)DP|C{@;L<`jU*+Bjr00 zis89Z)uFzQgQ?rU9C(X=g#Q3Aa6b5Vwtt|jeQ%FkDU9WGbTFVHoY^_+EDCzLTHoVj zn)geljx_@b;qWYZP+)z>qZCmP)VCI)sTkdYYxO2WR^EEE7hkTw0hG`FW6ue019zj^&8L+cyXWN@}Ilz{p0ZKZz$B)giO`Z?L#U;&U<_W6pyppgsPTL4b`vPS4U4& zYRfWeNas1T=Q-iO0m~HK;|lVF-+u!v8r7s+#*DlU*=;UT(=CIB9d2r%k;a2!S!2u6 za;cUrBKLT4^G}9CXtECIMt_aA+|YfSmN9X!!}lYABcrP9Prwz{SU6%F6HG_3BW>s0 zb#f!TZ07JDh(0q@lGSp+gHWri(Zlj|aEGxVmG&1tmnVhNaDbP62C3UY~=cRK6YXt}z_eSvTt@ zv6fgQm#M0%{xq~)#n`dgLOQ2x;Mn>h!1elc0N_d~6=|+LOs(c;{vXHA`o~V=qlXAL z_Nn$I5QNW$7fhrPiRe5AS-);CBTlNO1!yQnRJip$xI{T9|1ZwoI;hQUZ~tcR?iy0V zt+aT8yA|4E#R7yNC0L6SAQT8L-4&pCaS2ia2^uI434s<5?i5KV?ocRFv~SLN=9xMB zyz`rPe(yhlMZ&!(XW+T`53`qpSVvYmMD;)=QfTAK4f>WaN{_WGFW6 zm!dG#Qj;!DG??l-^{Kq?-;^T?Y06>z^h9#u{9SdE**BoGpfSo1C|fsnPgk8WPcxAY z*tcySo4MoI-1zk7%RFeHHk0z}vp2@F%QU-!M%!XFws9|rtg?GMo5m*g>nh#?f`TSU zg27Qcvw@eZz+sdfgX1wL(w0rh%`X$jnqQEV&yydhx!bx7652582SvJ98{^_wGuM$F zSZ{CrgN5KPR40!xZ-lA6-bfLw7HyLyV!{LJoxyofu0L>(}eC*nFmMUw7 z(AKn#gzS0AG4$o1^&(fZ%wqk`Cfh!gg?{j6!tun(C{YKaQ}qq1+`?Y5<*qP!bk10^E?T|Rdk;hQEADS7`R#q8wIMRAZ5tb3`;=bna*#Sre{CT;=2`FU+dF5rg9Sy=YdaG z@@*q4tJU<`GR=Ag1b@gBVsa`!-Hb>!dS7k*1TQ@}s^+u7wlL~jE{Wkz8GUb+GwsT` zsNkCMXV1vutF>;yd|qRqBTyX>1EFLNM4ol9Bwn^bOU5g-mwR&$<^y5T>f(twsN(?l zR(jXXdQd>(AVK2c(r~JC9@r3{@G2j1q_CWf*-362F}O9rqtBV8L9^x)V4c_~RJCtF zuRR}p5{JF@;jEeF#=(adO}P%FlQ+Tr-qYzs(wt`{j_X;yh6_bq@YS*0T)TuqlLTr$ zOGIT=vudsw);&$W^_%L^I(Iw2NoR;CTS{WBWUs=QnBDW6-x}GuL`@xm_df4wSHaC? z*M2hSb%3sdPQTAY!OeY7HAyzl*untNQ$V|}J6>cu9^Sb670q{m(1q=+#!s?8A~I+1 z#EA67sj*s_YkqZw*)S`_vQ^sfa1(KYMf1}7rjZ5n_BT+-80#YvdLWyI<@CHGn|b1juCs`!Soh7lI2ek(9;2|85i3= z8@K7^K^qh8i^kj0p=lf@Mdv9gffDvLgC0d2hky5lCtLe0SDnJyCUl8#dLg3ybhZ>f zMS^D4X(G8Jqtwa)!=rW~;;D2eB^d$fv5p&Xn@4srZLiHI`tH8S7Yz!W((sLJ)u^)a zX)M|2$WUW-&~u2%(5>kN1mt81P3gR`yMOaCBv} zZwFL;?DFns34P8Uw^cqb&di#>YL_1>Q04QBdv4us>>YsOXW;<7)QvF;hBmtI>$ia` zim)Ud&bv@P3cjdh%RIr;YR=B8zT+_To0v#?sYDsKk5u`m?RAJ!z0U3uCX=sxhkwB~ zIdBB~nC@E68$S0FJS6+{;vr*2FOstLZOBu-u1jP&8tnBy{ngRNGbfJ5_GG9=?;@)J4tEd{=nWU=ICw?e6u5wZFJ0J0Iabidaksgpc^ZbY2zu z4Lnz|`}K3ErB3AjihqP$w41lrag@<^I^*`dzRW@3Go=M3kEjWp^jdbs-@Cb57Cae& zR?L2RO;lI^?xW0Q;+HuyY#s#zKLVj1=<2sq1%*c?y_PiYmvYF?-&9DZpKZg|_w$TT zX>EhzjR_A~rEAIW6lkPac?-FilTdV$wxe5_KjfC-r>&EZWX`WB)yo($WerASmPMtM zMt{gbs**pAQJJRy`uDU+x$=LSu>apa?f;oxuZ{Zq`$ieA%iHFQn52~E?Z=qjTdOvN)(=&0K!$53LcZu=1C6ywd^ zKIa_97UXf1g8eF|gIM_7T}+NOQ9mqUJ{J-b@P6}~DnNzTmuj)A=KAV&p3l!JVwHgG zo6w1tCuY?@;-~I^?Q4p0lj1fWv~J{4*S)AzV?a|}cqZUcl>iGGdKoX2K-o`etUc`5 z=b_bU&A6;nwPcxNCgST0A;fan0vi$k(S@a;Jwb1|1qu{qW5DR)Ed(QD@{+)mGdl|U zGVYB%!e~PYtO)PN<@8O;db})>HSuq9FMZXjWU3Pw`w;{`G?RM{*6?HHjWR+u8Myk6 zd>8hv==A3)kP?QKt8|JQ)swVzJ3G6WbcdU49cHr7H>9R z=J1H$*9*3c5d!=&g*Z$5TSeFMyr-^&80Ykap^zX0ObS=EF4G;W4=*j03u!G-nBb4C z>m_%w=KL8&(cDUpCizVC3l?r1nA z1?mW?29d0~j50e3TjqTB!7H^kg*p{7mXlK1sc`lA z4%&Aw%#dWNX^!VG6iS=oGWO$-_jN8;A1h5gPhs|>TOz_vlG;lg2Qw84@3-;i={*KJ zdw}FGe^u%t?<~5u1tys$E5?`3`$s*BP(KqAsHIq+S=7!m!ge@bC1rH`8Ho1sDfcKy zhF$ovP_TdK6Z_t67QCK~GiH}1)WtXLdv0hY1Xq_q1k?DHClkA~aeg~HCMvZSdz{Lu z`!4-nCv#Ga6SYO_jlj!@4*5ymW$jLC-S(M}tBFe7BE-$8ar>E1k;z9o4zqc9k01)B zbmzzRI#5F^lgJWvhqRDhyZ3>9#34`sw(gVtIdNQ5bqfb}HrU3ppd7N8FMVcRS{|7G zqH(k32&#=1WiE@iyw?|(K+xrIt_T#%1DE7fTl{)z7b!k*yP`>bTT)WS6u&FFayAxY zdgdN<5N73nG+{l}^ZYjzs9t)6cFf0pdN&d-;X0w<^a_XGX8(@-%R^47<@SN^h z>m6hI=&j`Vi=SC;xmTeH;`)X#N(XPE+@<4r$$(LlUfT>w!ED17Pf8L?6#q*fD{9=P zQ)bx*D*-d69*aa-WsJ#au5bj@T~Jj4sEK;NEMUwq`#lBVU)+F(5I}^@ABOME`REP9 zwhO6`Jik9SV~+i7_MN8f^K$vjRS`Nxg-HEQ36`UhpVrGTL`tL2F~+I)k#gl`s906q z9@t9w&}jX&aLJOlWfpYymz!eiJo=uW! zbeZ*LRR)9~;vMB6i^S#mMnhe^EC%)Tu?lyVRtnknXBlQ6X9-4FoGRt4Esouniiqr$ zV7;epY}mMH&t56TOlJ%l*VXq9rR2L3MAsX`Ni}57{3%}VOBhqY^F(L)8#5UoO z2{!_wOZ@~>?9VlhjRj9A90G>Ne+u(_0J{6n{Uwu}Fd?%pAi$JLSSu=8hPxB}%3V`k zClCg2u8n)dxVO#&W<5H5s+2CsiFN&HJO;^>Tp*gSrR4)flKJ+ z4BOrv-f^_gpJ%&=@!un|Jv7*kVnK_Ys>bY*o6}!dCveo3q?&3-7F?;{AKpvoNZc?H z4u~hZLq*h0vA&-YLr8?(5|!-NUy)x&=Au0r7w?ipwkq7~{(6o|xchvaTvBE-=(}XQ zx81k!`W79g-?82hy1qfuT{J?eXdYy?>-(B}^n0uK%R6e(2%wU{fzhL(ZsXTna8d~P zrzzH%51U5wZ+**>5834?Q@bTwIRn_5AH!8AOGd zt$?{Xhjv$(YiG(~YzQOvruA0=kA7+HDL(r>B$|hVxyd;KmZ8~uVLT?g$nCKxu(d7r zL8B(cs08P@UHakVS`VeFSWZ)`xdEAyc-Di(Ya6aVdIO2-~rP=CrHJ=b%r~-e<;RR9BZ0|7V@{kJtapI>4WVWb6}m zA#2`q?y^%YyJ9yIj%!dGV_W|>@t}sEsnAJO^!tki8Ofc~5lnqcbX!La^cmjZEBIdR zWLRPC^#b9OD7TFS;qcQr^FvH=Z#%YJ)=}siI(9~X!y~gA(qVA>LvsSq7iL?7tuyhG zp+)u^i+4~%N3!4fy>vM+v^^I{o|g)L&~Xwq`|#si+zX!7X%yPOXtvxU1+J5`5xFyM zi)^CQT*QehKgL|g_0o7sP(jx1-W0Ij1ND<5)7pyre*&GSZY%9KH%NKa6qgEo9IoPy zhaTONyrztY7M@7D4zgcDwr`q2M+v{w?bfCM*=vieRwa{B$HuW;K}ChznDm^FNfIqb z!=HTv3`#0T0&l4c8&JOuifrNRMjk=U&)89wDaqa#9=C{BFABtET;?aXAXSQS8q_N z{ZA*w`L@dSzxtc;${Ur+H;BjwR3ph!sV-PNB|J}tX?Pvc&k>>JCR=YQ1vzPy@BL0C zlh;fK_)@c7G-b_D=)_}5Wm)m)7 z!6Hj>4PlW{hK#AKyV#bYYe0)GoJD5l79?00enTPR_O%s;E!fR64Rs|c>4kjkv>{%e$>uR69{PGN_QN` zxetr|xXV3ce~V#JC-xn?biH5?#>=jSg;u0g^_>38S35@28OpV!+ zxEEY<$cOZ&on=m99<_Ln8ev0Ht%$R{dBbyj_Qt!jGZj zOhf$Qpn`e?CF=rCCKN_ShKoAs67Ui=?HK7qaNsKw;)1mx@j8Fu(NVWF0urQQiMeN1 z?*mxp$|!S+K-Z8;+>;P_LFp}3B@r&rjYL$A0&-bS%E4u3X8qFzU6xXef2Y2}*Gz3$ z3udfYwTmtwoA_6htK7@71BED|Q8l9MK+Stk zs_(jvb!ck05DR!a`9djG;q9=j9dL6KOEYonzhtx>xmsb2VijOA4C# zM(+9BI|$1*q3LTfWvCiwmBL6Hb@fDp)|OIfNvp@>Jzqm|u8^5og4kXP3i;-7RGWS2 zz_?C-SRq6XCqSEc_4danic9e;np5Xo63VDX^EdWUKy$8MArs5EXHLirq<`gaVS~f-oh+eHkvIK1C@W+|M44zOot0;xw z&57GfLPnjqy=&S-$mv(7e21pq%vmVPf@vPbwikfy{Pifj`eO(n7R=hqxEvQ1Ba-X$ zX3uC8IN1?R5AQZNL!~8Wk($Vwsv$9UnBA8X8}KOH4JB}oeouv1D|{nt+WNc;^B5xD zFB8Vm^K(eH;^XI6kH}^*&ElH@ChlthUl^k_w*Yni>Q#v>h807!Mi0zgTYrbV=chj{ zyR8NZM}2L0cz^m@Nm*PdbgtOPzA-?sG`G;p$D{DuW!q-)9&LIxi;#^)t8Be8tzK2X z)P9;+A0-4v1`Bv%jFt@2gBs_=KgrM1t)TONo?)gVziBE@k(aP3DVbMQ`;*PsO*L8i zR$1Gb%!v28yBiP`u^nSEJt7mYlSkKp3~J+cX12)03}j{vWV((C-^$pL^9{jrN=Yb0 z&T7fOv^OZ(_T8NJ>cxT8Cz7udSw!3Qoh^KN;d{q5dE9Xo~&F&t*jUSsw*)f+5Gd+~Zh&l+Ny&79Hr zW1S=8fhv&&4QR~$Rs9x|=^E5jGbXxYJ!)=Uw4k%l9+Rce2-DQ9HcmO!DaK)Se^b?r z{zBiG*WdFqrT7GO_PBtalyiBvLYL1AdH?|)Y1mYLZ}|CezwxGPJ*v=l$tf*MMk8`d zaz->}nI{3qncGE>6AdZw?^5Pxd94{JlQ=q~Ya7Jj%0ClkeYeezY46sXWe8XmGp`YC zRg(0kF-)SKd7HUo!1-sPeoxfbYHe5UmByWz_G!>m2H-9nqoyss%x@o%m1E(B?UN)P z8HP<8Z+JMj8np~DrPnGuEOJC=xSbUDejAEp)Yg=Yk}+TOQ33N#&NP&GY2_K|k-0HO zg?JYTz$IKtOI+bM74Igu(6db4&!Uzg5u%f_R}BIkSefq0ky?jQx2&Zcjm-?g3p8>S{2P%af<@j#`5g_1C6X-EN+>R?XkkKrn{>@LdJ8nrX zsrbw`$IPqfFis9TSaf(S4dWuVxp9|$18nN<2daU>0!NQBE>+7KI7>Dkw!r2 z(l3*J?9i6xz6hta(=TlwWyhL8@yTPUe%|CNNz=suv)DVK10d-+1)&t$nXG6#e`GRx z+9>jkhwEZN8#rTUsO=Qi#irMu80E4u+E;aS3Hg9DKQ;1l70s+P_U&Kbds3WnFBt=_ zq#VO8obip4>DisJU2?Lxzn0b>=%HYT0$D8qq>xB_i;ut3R%qI_8B3y>m3?oC*-?dE z=w=|^Kj+PD=unMV!ArOK*yz{O1Y4QLN4wF$IF!kcU5H1(mETmZhJ?A=8ge(9BWwg; zs#N;aRHgZIsOYFHPiMkW@Ol4*Wwz-0aMN<^MQMMK74rj{Hn=D4yzhEZzDyAML|xC0AHE*c!QmJ43c;-8 zmMpV1&hcTTBrw#aLrFdx;DPh@c{TT~!29JvyKf^UBw(|yJ0z7lJ+o0LBeV8CnaJ8fz)v@6ZI{|h zf5W|1c%X?k<}#R@$%-RxK84s#Q(IguUsl_n#p7m@Pnkdg7i z7fWmVGYaLB+h9Ckw4;xVg8zuM62U`Ur>>;T91m7Z zEcipwm|L8;WkxpvPR5@Sp7p!twAUdKai~e(Ofjb7Ad{Y`Wm6f(#sZfq{2MoAHnuyu z&%P1H+X7|ab^eVrh08r7VG=J{cYyEXw8+B^$t6IKo5=tj>kS8qvhL0qwc@F|EWp!c zdOpPbMtOUqPxz(i<=T9jU#z$C*EsAJaK5HTf$cor0ciR`u;)W}8ARS(MyRI4(f;H^ zF$oZIk9{XTF;yoKP_D0pDM^GEJr|H}XfZ^2q$QCDNd-Dw7Te)H#OKWE*?o7XPjZc)n%__8 z+eSDwN009{i+5Bis*TZ0&yGMH77NV)W2Pi)$H-44dVW#{cHI zjn0a7P-YgoaiJ*MF+we8nR9Z>3M00sr?3gbKYn{?pGkU7Mp|MI6y(JM0aay_CQ-gU z1Lb_si5jdk%y&qBuYxU3^jh4bL?Uy_>>f9mG4|p!Vf3|peiGLW829r61bhJ*JIo`E z*KQ)`FdTm9DE$y2SI+7a&H^lvRABSH&6=<3UkB2vkF)H!TzOYd%|a$ZBnS!ozkXnW zip_#`gK3ISr7P|3N-F$a`E6i105#Sk(63#e$dd@!= zq#Bv&_2F>GNjZd=5WPqfA2s8UwzIrF8+c!X0a>7aPcM>Jbz6km`YIsw&8cd3GMXwh z20Erry!DidO8I{`EC2cR#+!;hgCM6^YFEg$#pg?WRU)gAqj84*cS?%IGD^ZjosDhe-W2_|$b;6vFPNh}PGKKN(z z;p6|m|B36=Y0~s1`tezQM|LjdEc50i)ooFKC*_qs=H%Mz7>CnGwf<3_$2?7MV5*6( zTq%9-SjF`=6o1`ja#4CQQ9yA@vfkVlL&958pn*%NI>9ihY(Uqc>|vqJ*X3ECQzPC% zt;0%78?P5wuV;7GK37?ZINM9{yVn@?rJ3Vkc(aJ8gx#T-Ksy%9a-DEEsj2POaVN!B zMXR~W_DtYefkzpgCdj*6$Eey{0D3SIb`5XiwhFmvvP17c#cP^Ceu^#4z za&~L?X7{?E$u(|wA6Ro%a_Bb3n`=2;6`=>{fFByEG+lDag8?gzpL3VvK$= z%GKo6vyg{lB<3{Vj)}gl!O}p_3pi0DlZMR;3FFcMy(j$6wz25H1!8|{qj3R6-kp=~ zhj$=f5?jm^UUPd+j)Fg#jT*8)8h$`gXyC(Ca=ys8cGZu_uLGD8=mKY|xhO*8Oqb*y z!Fe@*h!%R6rSn)r#sOixS2n7oYtdnhG|)fcBx;iM6{tNA4PV8>wPZzcZ%25clvUvs zct)1|Z>n@X4CEv~JF?OVu9~nsz1~5%*`3-HLs0)Klq36?pO>77-Y%ClDCs!bwDh$MW6aA3x<(UCvO_Lb z=&gC3nA@k{nqlKT_1`jrFIF@zZr1-8X&zCm$2~|!Xx`$$GV=!dLA&L^nO$eR&(RVs zyuO@GL9X8)`AsGNb}dDH@F?nKRKZ@`H`~L~yDMY56xVKqb%!}X z{3|X_bKM*CiAqSDG68WVoEh6x`F3<2N)lNaRfy3wtxcy2oY;rM+j-38bZQDVv{O38 zZDeU4DHf1kix1k?)-5ln(eOikc)dM3S%UEP)3^q$$I6E`)QpR()HA4IHAz6-EV8q% z9Ts-1mu!JJ`b{MbyFiP{ikUYbF%6a%*{EqtQbaWVkPl<4nz@R5g|{9B{T1})4Mb^k z@upqXy|^+r-CND3jX{3Qz412|fLBXu4v*ULVpOCXGr@vY%zO zypSoka(hA2a(^uo$VjgTSQUP(lc8r9k}wr0BawX0&(d&s_2UB9SBIDmX}5QR9i+#W zs+xjt-RDsQHiV2CS6>aVR?d&iYZ z*J9|QDP!dYKd$A(DowcD)Lq$+d$Zqfxcu`JeP1|NDQ&`Vu9sAwm9u+S^V)@}XtWqf zc}>Y1OtfZIVoNu`MSolrHeTt;Pxc~k@SEhPYr#oU3(9l&sa4lEU~k$any zYX@F#9UB@Kd!r_;FYsb0tg^T-d0r`M*X>x$!ajhyAIK`dmI9bmYD$hnt?13tW)7z5 z&B=n-?EcvqRJq?DT>s~_{Ifs*zIpj4RcI-*M%qrQ^Of@|i`>!w>QAcN`POXIe|_ef zdF$!SaB9I$BRjMHbi=4@FacU=WHPIf@>;#W*{wr{UT??Pt-m#j@m>+`=D1FpJ8hK% zHmHNjm|VEEF@NhL67M2N9_%}6l?!6q$%CMKZ>jSXjG~pVO2!!1kBL0eTvCt=a*45- zvX-I9ohR4nbz=9dZ+brvcxe#l7??ag{AAjO;loB>CACdk+$5nOTQ~vZkt5 z@SX+=@u(IfHe_qYF&iwX0QR$|o^g>C-r89-(xJ&cU2pMBzdB8k>>(FaJ9>Q0fhd(P z3ZVoVCF6B_Ie$~BAWSsvXC|LRp2v$n`|ipvzo1b`_~}|(6Lpxmzn-A}d{)pMF_$`z zqhW0vo~gC}<(DKQN8>?Zm3JN(!{CoVZSh5ZftD-qBmDg7hPVtW(kw8vMF-N%w^rY? z?a^fLu~rRR8lj+Xr^{{B*P#tPQnS#YVHwQty}x1{wrRlN%`+ZlJPKCZAKhh7YiMg) z`Kw(CJe4GYX0C62SIj z(TI6o$Sh|3sK|%{Oal1!h{)O8_&V5QH^tYDH0RU1e9*%_vgpzK$%ZH<-m>2cOHBck zyatCnejg(9jbABhFWjZ1m^9#eH(tL>h5{4WCFg{xSuO}cyNs6t;R`t}6B^2vgj?UL z&LKA4_Q7c~AE7HVp@ZS^A5-Ht>FXdxEb5C$odB>#O#Ujy3h&ejfjt(?8TFQfb9%kb zH-Iws@8P1mLU;`+s}phMc0>Q1-7FP0P3`t$u$50Na4uV@*Lm4ao>bFLdFboG_dqrE z7M)r0ds}RKhp{~~WkkGW+rr%d^mQ^P4RZ{TO`s03SlXTC86*B4Qu_egyUKoV-$Td; zhp3K_Q>r@A$b@O74;V`ZB}rIh7D`&^75eVEeb_7q_SK*&O0#1(gJhy%WJiY@kEG$$ zV09f668~s=p&2lWp@;FlZ5Cn1j~zZg?F?lh$MLW%_l3`JOpEd98mDHM9hOpJZ2LO?cq`k5#_qr79_Jy+esr6ffu#6G z=#>=Yxh?>r8!!wr?g-PBGTYkTs2?_3rk1TxvE-&%YoOhll0%92t4^=u3S4YVV^5`} zv~AUj;Y`3a!{Va6fb2zp>AypV15iByQ~C|NM^ zJRl-3+WN1;f{@Os0okZM$M9v%ZM`69>6#zZ5494WhMP$RkQj0dHpxay$7;2WB2hB* z?j4TFB3Y~6FlqzC36*Cq&V}Z*o|5>3*-x{@8565Sj8S>yv0$Vr_eOB0T?HjAjZ@1y zKnY6kTcO1!pUdD>e|N*4<%Zt#NHhaHSrr@yj!X08E;L=`&CK2@&r4eM)3amttYD9K zSGI$1_<9ynI9NkcbylkeLX`PNr2JECaAJ0KmnPf`zke-mxn{Zakw@XI*_-M5XB^Wb z&$EuHV+OR|b+BWV_wm-Bb4?%nsF+jOc)p7?K@=@PpP3)`yd-ZFKiCImg~ke6^UIBM z)wjO>!lQ|ojU3l7yT$~W zo}4tXzjjSXo1(s@9zs$tosw0ZWs*Jb3{S6*GtPRxvONno{MVR}_iA!6`f>h9bA+LQ z?}qgwK^d-V8-?Z$i0BTp$1b>~BuKR(><1sn;92=Q#REh09G^*}(}vK5xUfxcqrqB} z;)bCkQnSxAqpIftql-*oDxmlq>KPoq=R5Y)UDd!ETjp!Z#Bah|?XU}+=&IWAVp&!- z-;2rSOh=>TxQDqLf-77?wNd#)Wcv2H&@1ifT{jkwob2qOhI;O6HDl$=z;_6p1pVR* zG^pA$c3mRI&TK>l`J1W@?+8a6%9rFdO ziyD9b=g$5A{5|#2NDUR04}0f8UDefm#5cbJDVn$eWUS9^5so!{LMRc@msnI}V}2`R z0D4Is1szpG#0Vf|BnO}=ta$>q+?>l<+txm1>_nL|4W>k z@fr0z7~TL8uDn@ z8x`GmuUAxTBU!FGRF8#Lw^38_3BmPznScxpS|z?PR|Sd0Cb(m#=6A4^k7mSf03UtG z?pbB@&+zlaV_ovbUP36s8f3>w4=J`}Q_#(l+Og!RimWzegwsJ=gNV8EX_KqmYi- zsE#p(t!`G8QZ)9LAP>lalHsa$*z{QVMk%uOdoz!m4OwfhU(Ub9f=jI@U$jFm z`Ke=^pdB~fN5!+GV<($3XQexB*Fya8Ps7AmMrJ1MZgws&cj>VgKLWUtgjx$586N{A zIi*69JNu^M#1l!D(=MEclI6O`CVt$Py4xG#vsQNQAf*DQX}S4t&R3wPt{ZH(@X~Ql zC~4A31%coNHr}zT6-<>0$R^u_&b@JiRqCUV5%CXRmvtwS=`HqJrou=~N&BzSUNSG< zI%qKT&TC7ix#-vs@0yv!MSnbG&9|hQ4!1&5BvO^0A8Y4z#KYmHTS;b9fLMSCOI7Hx zk%hSsF2ew)z8&i6q#U0f2+NzYm%XAgOSP8E$M8rUbiR;kVZDd*QI%e|G@!m#HJw>b?Rn@mgfAJHIUp8JFEBu&oOSm0r0(#*EhOKuxy)Pc*F zF*NKUQ$)?G*Nb2qk;f?}u_d|J}V&$nmMid3=Q1P-D2B`AW^J`Lx$+ySNh^iZvCcDyO*f>1Fzv!-MHVkAV*>c2gCh zvR;E}{5u%xR~FjM?6A7&QMI$c-A6)Y!|hV4EV6( zqsq#6%f=@UL>AqhiLLjjvbh<}%5(dsmuw84m7u?>(Plp@m1siD@lH_iV^p7zproCM zj=}->wrn&_T{c;oO$v6O`8cTH_NThGg?_66-+-1zd7hBXb+L&pDT90aQ9EGr;?G(b zDe;DkTf&{ElI{Q8IPfjVoh~(8$QargbUkW>b6)H=$_msy1Lq(v@n&ZSaj8P z1?fk&(I7n3%}+ae*pCk;cE$v^JbY%?HJtCSN8`COFt^zQ@1r;4q`WtfPJcAN7Fiv1B@Zd3QERsJNF+6^z2Qa zPj9~XfhwzLOOcPP>+&)#=_G*HQ$6Neg!e(y_3*K5%vG>GCazpj0~wsL+`o00gZ5}z zw0n+$DpsjyH@c97Ev_&+vA5I(38rlV>i6HXHkKo!9~g|R;l|a=yCM7?iHs7+PL#)8 z2x&nAk54|TJL3cB$jj)D^7QaLVTno3k8oyQhurpmFTTr$cbzgY(Hy_`6voL2 z2HCQ=Lm1!}iSE-s-`ksx0JX2wUSLbqLP_uEM>HlXM-0e>7S=OG!<*9+waU*hbwtPWyFgdf%oOAyXj? zg+M-F94*BE!<*i1x@-<;t$X=4kKH5ut3f@>;}XFWwk_Dn=>T<1uUB#@d#W8v0kA_7 zE@<>itC40ajuIugA6j@SasfP*nT-Gt59(pPkAC(lE1Mtfh{xvvnu6LJ;i1`D0q>`|~Po05+RfRBd)fGcbN*|z&?sbkuv zBtv%#V`HDckm<~)${htjss7S2ian>(*)9xP=isX);H^CoA5vfmUr5`rO-kc|lLsHd zJ>a!%IvE;W0~|1jnMZiuRLwZnMJMn#)eX}o`LyAK7@icax%v*`3zz)sP4H1Ad&X2J zj{TZh3+O{po8;ATG2Ad8O)n**M!^L2ICNHqOw-C(zgfT(9pnR&N+>=lIlU`aa>#wZ z+)S|8g#;G6mS~Va!(nE~mP$B$$v;~m0aq*tdlP>f>Zz|8ptHZ~kW_K&*lanKPbvt= zWZX>8U`+uQ*J52d^BG_2J+}Q$YY;D9E*T}w7vZKM7ZLNM$HS_vp25xUlhrj`~V_2JmEHq1;BN9OikkvCZcW+2CC$ z9LcUhKcR9{7y0|Y9*+O#=4~<+mC-fxD#oGESFb%$j>DNeMEWpEUVuDLbcJ6uGJw{7 z1{|4c1?rFst9ctP7Pjx81KLBo%ygY|doKlh+dFLD+U}&VTOtUaOXpsDQWEzBQyfQx z$N3;&Mrx0{dJtpCd#ImpoTM@qy*2(=0#Qvt51dRe3 zh1gFCxZuHD&{s`4-+HkbO*rk9Xhmrm(euY*2|ytm)LV8}Q&ISzt<|-x&d(mD5Gom4 zW6zPMHF;S&>USA4`zrNnb~$=A@Cn~?gS_9*AYEdNB9p8+6dFQCO!KQ}7^RlB~QR7;R z>`i%Lkw(adxP;otI?T0xuH;waE8~2Oy2Xy`($f^L;=8BCA<`FnCQ?Sz z5-nLTKhjVb#;%m&<*Sp^o;3aCj6eg+NrGST$X@5SVHPsBEp9*>8>IZi_Fv-={EYC& zlq=9t8Q`=TFKXWxmLkezzHt1(@pLi6lYhSSFQz9&GN5PF934*;S!N(&2s|Fmpq+Lo zKh0wiB9b_rH<&FSQ^~K=a!3EB+aEr9Ofyx_SqU)25}gJviL=M6(OzY&pUg@pdmoXc zeXBfDf1@)Wd{NkVd2LZ=d+~ryej*!rJ)-Oosi?ZWed9oQ6l|Vru3r)JCaD0NZg7iq zVfw2Hdvz358QZrW{T|sAnr~N-CkLo+$`zj-cu`GK`i6^3|1v#iq2+=YumgciJEeUR zG3BD|OWdXj#$6H&;UK5-1kf}>%_Y?k4G18@XCdrvQ{p?}FhuNys8wP31a~>5NMEcr zQA0V)V;l8zcb)llJ<*-dMotqr7MkSQT;#((%7^AyM__7;mH)^naP|H+rQ`0KmH+Un zbYib;{3{bb!MDX;vAE(ikDz7g|ugDmZ^qgkSU%=~CM)Im(br=JXlxKl#IDUaT^1jJkT z9h0j;Mhdi$kc8+)`Qy-F{PfEc7}3 z_4DcJR&PA(6U`cM)#A@CZ77aG^jj|2G4wPtrPh4Cl*8HX zdBdjsQg{W=d~T`S3&9mhP9ARa0NL5-1X~KmZa|*biK9ku-I17~nxUFJNnLWJ#$J$w z8nJ4g;#0?lt0cv2W6X+If-8!$t+3YB@L40lZ>el4yU7#nz~20cqzc@=c3)d!dW(Dn zM9v=#ct3eyN+6^Z7CfP0mtV^AaDdA1cdfNgXb>E}bjpN(i1(0-r$#b4^BNnexBMt= zHSVUEv8c9l*@9PKo)wxaHS50~;p19#u0NV}OhozVG@BJ{2VH$6E$ul{t>PLyenQW; zxdO&=cDEo%?}HG1Ri(mt5D;&VRi{J@CH>6kOpVUv3TN|pxyjp!?5%IBqh(haFn7Wb z2_Ib2bQskwt%V9a9MqG+HJ*Cbn+_a`zp3JQdMYrDK{AN9B3!p`56_ha;nit*$)B5b zS(RU479VO=KQGRt@l2r^|J9^9DILUUoE4oaoj-Nt8W2VZd@It`l~{UIITDY7+2D)^Gwt?6 z|LnOi7|c*IpG;FoyY+a*N9U=Y_V&^~v2T+jH^(5Y??m5<@b~|h>Y5{7=kaX+I@7uO zTnlpd34PP`_oDHAe;%`LROFk)cOYpIp~^|4eWXj>xWOiyUM}z&z=Qwoa0OV(cY8Q6 z)-qqTN&vW`R|EJ^=;s*}|HPX9>mEy?Mn=@qaQGWx*Eh8l4=x)MQlXpXbhr!iDTvR? zpy6e4agn-S+M^jraV&YJSN#4WmA}_e`J++a`va$ZpCz*i7?5-4iUu4Bl#^eGDdu;5 z$1VFYIygUZiN<8EM**a+&|x@kqCQ<_goHWf=?F8Ukompp>6-gdRw8g*-1o#7QyXJ;oxr}24b;Ze`u%IZ}6n?)A4pG_yOM~#OHu7h#z@ApK| zkyzGX9(QZAz#(y=DleyI>vRv*hNm+3QOitgKa1eDE@@NC$sLiWM`@kYwO=!@to<6U zrn7L}ydn?>&nj{eEOZ8V7Y+Y)|B1yteU4r%XCIsl-q^We=a*cd>&;M-woG+@;O_v9 zj?C`5LmF)P8!UKPDD}?{WXjh@9mOrvmkYGuxJI@Uk(hkD42t7b!bS>_q5JEv`f``H z%9fqCX3cXBZm`bogkf*lEl0C#7?Qb!={yj^ADsl~HSCbo8Fc||AxTt$SN>>i?Q6I6 zl=zu?P=J^fs?nLR?sQEt=D@&5vSn!6F#y6sob!LBHOH6#=a^HmqR-A|vSLG2ID}ID zTd{xG{jc0+pTpok{yiwEa$l}38d6=o={iZvuHC5b2W5I^mdjk)IOBL?%UALUQp*Rb zWe}~pQ?pZN6!x?!qRI+OLcW3c<~6)e=3e67?$e-`^T&t;tDd0e7vU@!TCCj@^shXS zz~LD==M?+<54?;ELB_E@;i~wwuSjQ_kg-VgnRB0+x%YSP%zf@Jf3OMcXRnpLcGg-Sd%crv9oG$yR0o`zOnX0+&CETSOX}|Z zB8ajEG&dY>RP4*7vsKG)V~AuMwpYij&42Y7)kCQAM%dH z{(Pn`O9JeLK_1iq2sgLo7Xi?u(->W))j$J<>g0tQLD_at?OgKGl0+{fL4uBWZ*g(c zxRyt8aRr4&<1-wcD;glDFavvG61CLN?p$NCA2w(#m0l-h1kaDkN981VHj>UVG@`Uv zN;Nh4MEElV2l`;nmH?I8F<&a^1id7-3475gZ{TepXHYl+`ousi1oVq4zbdR$I_~6;Y5Vc~{?Y z)K-h+gQaFMHJ1N{2oSuET%hUTXsn$!vk%_gjq+l*&b@J>s9iQtW5q-KA$@Fgo2uCq z4)OtNO-PK_ac+^7i#TDCC8HcP9R*9;GvVbl!4lUK#iCh=XWldna0o>RjCUTXeQK7* zKHhQAvmRPM->m;R^3$eI)tB(3t@FoDrh$?hkl!5Qc~@vbsDkl1m*iK&BVZxMjY71> z!3=Pm(}1S;#aRskL3*FrgF%)Y;tELWG`($q{eGl{HB!KtAS1QRI$TuLW!S4HJl1je zm>}6~DE?T0ylF2zG_LIjbjlFE|R`|NF1zRJVDN~k9_zj@=w=>4j(^6`_&LgmdK z!5Y4pTOWqLYw2?v5R-~yt>ATi zGLmuy6<%gGSD(4|G*F!j^XV*@a8-xEeaF#;mNg@uH3F@u)`Jr!$62B)_~5GlZ>#RV z{11s@>eE)iF;SRA^L+W?0o>IfH%Zrq=1O#K=c3b!p{G=S`gE}HST>xM9Hmk^S(O&G z;c@x+#Et-eBKu4m2~Zu^&wq06|Tt7OGHtPh< zi->g!OT z+5~gW!*IO109%eFzb%`Yuf5n|Z?CXhTMQU8dEMvx?m#a=SQ|WSDC`;5k_Y6g&kp++ zAxuMyQcLNxL!I9oIOKP0l{ckt4O_)IopMs}FL~zOsezD^m{AD+q)GB%mZnC%&0Hx? zMlBTw;clAB<|f6YZQ{@JgwsA~EZ>Y8@yiYHLdC;E7^j_RM}ybj77#m8*O6MecfD#khkt6y!b)SD)2V@`?&QeRdh&9W$f=nviuR@s?*=*R;VpIeH7Ai=IgWx z!Z9F=6oCE(f~Ep>;X)KI0KC4?bD;u8){o`dlj>TE3fs(6KF@q$PYyDerM0K#?FV)d z&p&&U+8aL4SY!%pl<`)7no%a=q<~j@v}Je6knUy92!RG??gD{@p-n} z&PPSl-wuXn*ch%gwes7FtevVk9+#B$=_sd@mr+>fx{}E@T5@teo;QA@$sa7d{Xk~> zyU8aTXxs+@V_T*XCn5Bv>MafXA<3w$ykB>d(7!rh)6+FS-r~njH@jfNbZDh_+(>*eLU9u?>#cL9{n)7*9@5G1Js^zuakgjh2K`mwN69?ZcUm@SG zH6-&*fpdcE_O!*}V`KKl`ZI%sWwHE22bhyKbDp!U$q|^XhmTsN$M# zkiNOJnN}3(Y@3`s0@PsLo#~mnBBE<&smQV=Ijz7lPC=pf@O&!a)tO9-PcDh}kx{G> zGx5;e@&FImsglzv$aSteICM8P?)f6?_-U8^N}(y?=k7^>4}SLH&)4>aadjdYBLR5v zD$2U|`zQ#z@w5|z>24A?L#}h5cDnM^$DqE02+aYFq;uA)#3mJu2&MQ;in$nvYemi5 zMiiq}XA$4Pn!?-Y0_GQ0X2kloBAk!yA2|6hZ5wqi3Q1BzMU?_89Dz^_|KCS7%ApM6CP9*ch7EkHE*uj>Q zd^oLu6?i6ZQR|ljAg9D-kdn;nl6xS}`u1-ih$|n5s%(O4U9NF3O$ARb@e& z($p}CIcdtbsmfRyK+l!Z6dDcm!ao;zt+UCE)bUu-+kJGnUQr4t!l27~pUK9tsS8oQ zy;#Vfju(!M1tkKJU=IcS!xULkJV;qc4&0gfbep1J^;|V>%KsLWqcy&4rKxA1Io7a2 zvY5EhwUNhw4bUk@(AhvWjw8%&Raj5!m?%_Z7VK?`Hdv*4xGg8IFMYM3ZR-;P)fUka zHMkc>sD(iuqNXjHy3xY;LD<#=8`2qD6KN2lD;$x}I5MEgxkDW@^kOc2oN+C*W;Tay z`lJ4-Z)j{}Q*~J(d{2-d%r6ol=yEGGnAD7UIw#G&EUiFh1Ri?_U6e0%^hXsGkhI^c z@!=~Ry>{9|Txt+4Vx&3d$0WAo{dJR_{{&K3eym!h!$E^b>}Mn@+cGl|=QlIMj3X4kVi{@EWt| zrNXD6{JOy^+JyVB@>#kIQwF8kY&G)%wwJr66g+IG!ECtT_+3|9S}Bm2@N_0K>z549 z-f+b1wyC1FyEuoz6NolWNXaUxWVrTwRmzJA6Xx9)taVFgt&FSflvF^LdpS?@Yko@b zH~+{=4B&-Zi8lTDy3+TtZ=e~%n-acE2`r=H3=7Hmn`FvF4<u3(TzL(-nOH)*`x0r3+ZnLRp)wQBG9HD zk)>Gk%3eD7;wK$NP@j-{QnNAR!>A!gz#y~16Sg-@sLgQB=f*S{jSJ2a6xCigAE?;c z)|z@K=4+8mk#V44(jGkG*l7&{jUdr|!u~3dU=yrt-40>3PEt79pmgTv-yBZEeyfPL zJ=w+R?d2g#TwawlXri<%?#yapK+!Pjw*r2&E%Zn97O2~ASMSN}IZ>W_Exgum!Ms&s zz0p0G=xI4XCSX*^V#S;Btk`CYPsmMkhr>+UiRdbw)JwBcfj6BvZM19x!pW_TQ7*#l zdyOLv%?%987K;K)ibC8iHvLyWEoMflwEH~STyx(e zW3tAdVf3v`KB(xZ9g(`WG-ibJuuZLJh@9fh2(j&`%q`5Lr;bdBHLP*c(o)SB1@-DR z_{=h(aUD*<3adG*sP@xPjMHhtm~#5q0D8r!Lm|!SiS2MnJFr7qdG;3OLJmq?VV42E+1=3$}S))zT)2UwUO^Ty=W&=f>QN_`TLj5mp6}8 z$W&4W8hVtGBw-OHE*z=8RY)ud$-}kRcWU(wYmpKD1?KJWxlBwsi8hg2d>-CnMR^dI z>J;p!*XDT$Si?g9Uv|1=ef103P8Vh5T93l#z-y$ zhJ_NiJO>pGlTfV}%QcTr77n!94`eW+Y_G<*sb_+4Nlr~zTw_ZOWg50d$W!y}C3i=> z9>HaLUG#0kfQf3t;>LozDoy88yu+^BMqSh$*?Y8(&-UWk-q({&d-}v-b zYkB$>X424nV%~Q*(B?P}ERsM&3puI7s8A5*XRO`w4Gca<1 z2bB-1P0r0*I_~p2*egg9&?#xs*2|b(?ic$_8_k2={PwUYB{fSC`XMUgBHxvXe+%g> zt$l0-PZFxKjIu4x$z@pDkyRaEZd{Iyj;wIRAEzV}-FT+<7t7|Vu~4=O4%?>C z<=HfUH?ir|eEZ2s&+^G^#n|EIu1*5|3B-M_)$MLk+s57vsWQ;#vBma}q(2J#oO!!a z9qf#wDpsrUM`j5oCHiCCP1=4tc!8}${qmdv(18KD@!IfnDcLjNIuiGG>)GcQ2%upN z!z}CZ?emCUqTjle|JsEzV_rC=aKgyp!0SPM0#u|o*1=H6me(jjz&Re)Y`hkY&flim zBIo|8!__A@>E|Uch-lJ^w$ID8T$u^CvQ%yc>}9$}EeLtbqF?U4p%~hfgAi}QvY{0o z$}sn=#FO+kR)P#>L#&RWLM0v%}cKyn`LT8FhD5Hud|=7qQ-z>WWo@hY@B0Ks^P1c-A#L zPpMBk>3U_MpN5`*wp=FbOfJ{!MVkC^#e?1-!8xHShGmhnD*UNI>S+B8l<;_ZQmV$| zMC*}BdCWZ6-Qe`>Ce+~Mmr5itmDrAE{z?(0yKcvk=uU|-tv;@3y2nD29Am>J_L zpTHuhzIM9jy|UIj#y4Z%;HfqhCR(e*X)rANhg=uW>Bh6jB+Vjo8mmS1dF@0e2QTF( zgCyS3!iFUaPj7_Uej@tq(O(AY-~7P^6W$M~aq^s*>QFe!^@IGp4WaOp=qnt@7x-7~ z_se|iTyB8mRB`u*m(L*uJ|rPoI{cJcu^!y2PbVZKTu{xaTc*{o5B3J@tno@0_v`MC z4Zp%3>Nj%fWF5E8{%T&IjC`iiOe|WqXsIu9V=B3-qR)7`WpnK@=d>K#8-ycGtIX`< z2}zA)1l2GeY$pjXs_Wug;7_CEEzVvk5XlH)GSQ_x#HLU#jwQxXh6MIh6{y$^_WC!Z zeK8-@gK88mVL*)$l`s%YRYhhZ_NGo#OnNqh>~sy_GAshKsmk)*md+TTeK zoF2jvgXTAQ5M~r|*llNg9e?BmAUx94u|I>S_h94UkE3N6tV&jVf3Laza2sATWJ%vv zgO-(z4HxDRK|#dgC-G;jN$3%qUpRl_e|*@LO4w3j+o`V-T>^v9k3dM|Nb>L7bS( zMn`IJr%0v9oefnzx!yLDnj(V%EX1SlZAv&G^%-mJP)=RXS3Rg)Ox13>MNJrf+Nq;? z3Y}`zE)Z$4%@RDyC**FBk@#jK0dd{r8R&Yzesu2TE1uDc_oi$`ktoA zpakkCjV4L`hy8NS@&QE2EJJ(3qAF`ftj_bKuu64xPe9sfUuFcoc+9mQ$zjaUni@G? z23jlQ5XEC$+x5%HJf7~wxvBKQ!<72Hd*?tc_NowS0xDBp>2EH(2}k9Hok z?m9jsPmz9=bO7k4$RYeI0p1B^(c`)*3-VVuZFk4p8fxxyw>{{Zxk3b86F6(AI~H6W zo-;Wpj^&vJERRh@38s*CwZQeo;*-)0>YSiC4i8?nEO6A~bNx4QxA=n$+#BH@F@r4^ zwiyj@nRQjM0iRH=f^w6G*>Ia>u@YPdpQ5Q=+bim7HOdaL=qEGm=G8}xYJs7qc=@;^ zu|2A#@0ic>&O`Kc)k~uwAfGW3jRJ=glVK-U3YU4#he|^Qoocy18j0Y{?$`T<2Upsn zcEgm6?!v%nt^O(D{VLhYmiFsFN&nj-=)6S8`=^7S$~THulsafVdQzCU6zi6r4|+$w znwW>q%nCC9AdW_+3T_rPBJ6Y1iW&rHEbcW(zHtwb^Q&=fifX8-K&~7Fz5SE}e^`s} zN~FcSuC%Mqmz$B5CEjrGa1c!0jxZJBN@&;B&MmT7d$6Wi+pXqst7MlWo-Vqkh<=9MX1MaC}7!Y6;hA;|?;c!Li2uShO0 zZ%REf9}=7a?>N#hiD2^1YJk z>HyPCBaYvRrouisF5#T~)8DawzjgbU+eq|9a_Wssl{%`mZeIr1+~0{T+1|IsJ77+o ziLzclV*fe`eQ{`NO?M(ZV^YBUkLUm9_wHTRKVCKDAI^Vt`%sv}p(uS$mP`H0<=sa0 z--)WpUe5K@E?kJvJ;b)K-QfkWiw3KT=gecw43FkilZc_^vu&bMg?2N`%#qO4AaSTI z6&~`*+fpyz_a5I6bh&fqN@+yC{dh6S5t7W=%HDzyvRj zh$=QzR9wbQBU+aJsNC-b%sjybGY{lHjjIqRyGR#K~^tLtPKss%| zBAEp~!zD(+1m>!x!HI~%qdZgdK}tGLW{lTVY5)eU`__?%)7g{oU-r~>SBLw8tpLqU zBgXRZgtCz`56jKbZ%s+=*1qxW-^+9FEQ%KHkSHAxT#U=s*a~drAvF@94u5$RP68?= z2B-jB=i1=yxBKc-T066hUSUtv$1O7}o6_X;I>Y$lM&y)>4r+*XlsHTY{K3lFH`U+< zoheMPiJ%%YaVUJRhB^9Z2}&0C`9 zw9wLn!L{5-r_HFxcc|Y;+&iPGuvn{)S}E*&UyQ9mgr_z@@>i|lXWa0RUV4;E`b?Evh4;#VhoZ;ZAKviaiCnT1yfkPL?T+bjHCmLA=eG2|f}jX| zPuoGn??hk3p@|7kfW3nC^9xtzRnGaZ?$sCS=AFi}CRFAs)Y7s*A(+KoH!_3Nh5&or z?i|yk@5-n{Bv@&cF~7iy_%Mp*S85ZkWT(bKt8zM5OGay5N2tuM>Y$)bpsJa{L5G;0 zSy9Oup7MZ-ue~JAuj~77nFy!-ci&cauBwusatIM!H?q~qwH@tFa_dNUopM9vx5#$$ z$yeWlOa zYY!o6H@QadDgu$QWrdRWsl3B}_O!nlB&J$(!ziRDbh-_~J60NI2j>TU41Za6=vErR z1Pv36lh`@?r<+%bU8-i5=Dxb%n?9ttGhoNjOxj(x(Uk$b?K#Be8@R^wXHNS&bWKXe z%Nzp1lB;=g1EPESto^&k#AqR{y@w0-s1D1rH6fTiUKZv<(O6StGd44P?Z|1p3x}3g z8nIJv^ap<%XH2R%Gkw7LL}T}Jy#B9Rtpap*#K6PuYNgTo!Ks`S&!lD2S=?@x&E@su zzI&W~DpZkQf6BMa*1A@oi#81`n@+2=RwN%;dldY<#2ONSTx?@j=-V=X1c_Tz)gW)* z4tu?m`^##_V{h%S!WBkEDsx4(>9Ox-&I5ReE1s|xEIZfO%+=m3@xE86_GB$j*Ot1} z+eubAK^$te1MQ(JLRb_178r)IQwgrPD@g5lT@e?T_lKh4BvD(>cHu@P`b zgV|$87yhqXMvFZI!;Yqwd3~9dewkn}&189bb^ExD^N8R-OeKhplV?D(4fGX2Yz_#t zfxSvsf+eEQg{bu}$F!-jFnb4s^7C-i;d(QTfm2b@6A@;krZxkN0B00{*oaqX-?FKk z(CJ5xfkpMGJ_r>rn`k&b(bDR=Ku~Q~-GW3T-=x&+u4oU2c75!LI+>hIiu|BEkFKpCdl`3w2(~+bQ;j|3~!r?h02Wgikq!?CZ ze#z7|RNBH1sa;|mT-vDR0f1!@T(Fd{4)rb^`2JKJ2izbiVmP+`Tc=ULt$O2V2YQ=o zRZn+p+tV?jac84Rm6C_m8?!sQ(Y#)X6!6qYT!Ud78~$3ase|Q2SgkTHLIVztu&Id2 zGzL$Q{`#GW9wR@rMEPr*^26)ReNMi|(6Dtpg$k?y9jp zLdz?0^-+C@C3@pV2uj`Rk-3u2?Z>tZs*gF-+mqf8O{qO-mR`^7b|-!P)ismPE*esP z5R02B!DU?|$jAs9osnzJ*eQ`IUsH7)lc*buv~! z+PUe`CXWdvcnk9J@xtu(nw$_Y%Dp0^7IgW#H+E|b zr}paCL=JBrTWyYv3P)4)vpcSSuEPuD-I>gja>;I-!<8GlD-9Ym@u7KM;$w6C0zCqB zA#;@86^_+)jgJ{3V?|$Awhyoh@Pxbn+NQRQYj7K!4Ms>8V)~ES<>V?>4JW)x#_$@&m19eNHsd4)t#ua>>!J;JpeJ^!g#1} zr{=F!M(ErN?*qr_iDMu(vLCkjMjqXhy>h&-?`j9Hxav!^IQwE@<3VJAHgfKC=wfc3 z*JMPm8GCkV?%%X~4=Q_Dl>2lR;grc{Z(0abT-r>QhZ&p=6?;x{ycdAK@fMBaA$73Ce8&$-d1LQyF*qeNNQXu9|kbnG>{;b4E*&L58 zju)6jL>T2tyJxWNawF1Fhp5)|%Q$b@6!zH)Z6Dlm8!6I!o=-jny z_-HlK#f@WIYQ^X8`F@SyaIFUS`ME z2g52m*S&R=t7u}bC`r5LI*>&Zzwg=m6RTFSR>U1~lF-B%3!&_Z;X%DE^^z4I$;i34 zSm!e$Q-CIldv83gNknnwu0ZeyL65?ocq1=IwFwgZsp%$alyPPULhO13OEEP!VKeUa z)Gi4Az&ImLMwo9PJmsxEV2TQFTFQavbj^ZoVB?)b>fEtypd)}CWyoxUMQ;`y%MlnG zCy+n|Z#$TG&>oWL?4hr)e6B2eXqna`>&CE9oH6|=c7UJ}g4&lJf!LHv86>ra@0Dnr zIfGQDIE#)w+4ORd9iMs@qGGMDYKf4N8$c|S7Q_pXx#=;#6Pfjby3&p@9Q)#5nz@X7 zLx_LPOZ}qdJo;f&-HXvdQHJ<+wX|;O`(d>d(RR&-7UQ#_UOjni4P>ZWKXVN&d!k>BmSf?ift(a*k*uwZur(#GtxZ<@+H|o>$iM zd&qO!;P-_Yi|laFNsrf&(|XC(R$x^BDF(||Nzd;^^{C1i^i?=z>;si+6<^+E|3`Fs zcVcT*GmmUa>(%cMJ~w8t_$P7c@T$+~Bf=)LkSLvlG}rMG4Ig6v@UMQ4lR(gP%Y?To z;reGD?gsqN+q%Oe{F*ii=Q+p0vLUNylP%w4efP(rQa<;2Po$6Wq;WzQU8+}A7Q0GE zK&_RjOq<~Yi(OU78KHS>25NraJtD5>hg)^;^@-_gKCNe&+s+NJ??DXO!|>gmB(-@N zDU`!y=;jL+?~=^DiJ}^6vr$CpW8OxCN}rZ#!Mfq-753E=p}`u`EjU@%9h(9*ea9fD zo9{kn@^$Ql>xAdvVjRBH>|EGBQ)eOH0O^m00n(4)k_90-cOE-q!TGMEb%`_^HCYbG z_{W>mJjzQB$o`;~2vZ^P?qGH;r@{<3)dS~K9f7w*Bwn`mO7q?{>mL_*ga|H1z`?L$3rH_rv&nz98 z{>5p5XOXFT!PTECGy1WZo^9tD{uHW5E&5tLM>M4$j$EsTaxVLQY7Dz5GjpnWh#vkm zd;qY)hMmd!21UQNGMOdklbzgd7Q=i>78Bn!zK&V0m}Mu& z?p∨N8p&UJBswfyJA%GVuLsj(or)Vd^dviK%ag3^iK(R#&$eTh^!5Fzd7hPw_qy zNX{pC3=&pAO(_eI&@2BHt^HpgF5t{7uXNx@*Sexvq=T}KMx(vpqS6)({{uZ)57%>}ph-aM!EZO}4>x`O&CxN@@ zreoI~&D**P5_V#5*zhA;jpZ0Bndr>?)O_Bh-nFkv8ha{@Z!hz<0mzZf>ecf|T-LWQ zWarn2E)-|O`DR7|;1pfmi|mkf)qOqxmCSc9y73JcBv+>0mMg@cyS`mp_kqzSWhRGL zrnX4&q&>r;Tv$a|YRLJO^;pcvi+N4+D(k72d5BYk0pXF*JC{EvjrBbykGx zjMb`(0db;$*d~`#YXAS~BZZ3lyISr$)~+#*ltoSEp2=*L7ICvDpyoOX5TBs!l6!i% z+roq-hjDj~#>c(~g5zKhHXT)B{iIZfFD699CVl!W!|>Dtvtc{mwMB(fvoDb9QoPW^ z1>@e&rl00S%=5u5NHMc|OS}+CSkwnj8UZ>7WrXE^OwQ`R@j%Em;{~j+<`<0o4Lk80 zr?yJ05*F~@JQYlp`5hZWE=F}V*8=O0$6K@ev7oKn98a;z$L4!OY9=>~CHI(ysF-fG zcsPm;Y(th$#9`5M&1FlxV*Y!B2XG=wU~lo)m)rn%{}f=0ulSh)Fy=U#=ixDWJmgDh z%IlJM8uh3~vMeR1Rn3)5^o%KU6eGYNTdA7|Q;yws?`2+k8nwvxGR-PlbjS|Lvw$ef?m{h zUY{*qQ^5$UE}hm0BoTp&&Agx8g2ei-O8{A5_w_>d!Wr%jk2u;chW`W{wK|mU#x}`D z3nxk}l7ZXFU~Itij{@&$9>?4wcP&cvs7AcKIiK}D6q=Sz` z!Ad*^qMOkvyLypb4)h0FnoERfI9=_r@(W7-asR}<=QUIC`4)vA&uiYkd~8ubV?Y(* zn)9m&iBtBlsnf;2L?p(o9m7`v_k90O(2#N`f3RRLne5+iHIa4Qf*=d03TJ zB&u6HnkR4Y@iiJ@!pt);P9Rs;zxme*krk-MU_IM41|vp-nX8z$rf>m$cM0;!W?2yT z384ms>@oL(YnP5T5f;P_v)`^x+|Zv0j{e0ke%s8-NO!G(*m&1I78x1b#MfQ#F+wuv zc1!*i#I@0u1DHcFoI$*!Gy+M{t{tE>-UVbij-?aEVFpyUE4ze+TT+jzSf4JMzK8J- zCfBaG(0@wNd5Kl$PeyTu5riP*p02|!%LNxSwsi8slgccKvZddr?^ux_*_1iSjz30>gVI`*}~%)q7}T1`@RY5ufn=h`x=r)F6qj0GmZ3Bw#!|t zgfZyLH#y&i3DEpyGKh{k8Cyy-D24yX-GPH~EAZN1sz(AsxjRlUQGuynMR50R%_ycD zI(}QC^Wj*qnSLfHvk$FNFS%3-qATeW@)OgNG~YM~eM95ShnF9oTe$P|(#h=8%^^Le zDy|%lvmb0PrYw%xkKfUDUjWtqZF5(^mwkL+Uwrx3zoEcr~{n?QR)k;q%Tqy+_lD{{I6AXf3EK5`+&<2S` zp%`OAqTasARJ40Z^1|qtw#BD7Q(+rR%fSq+hrk9EwRmSt&ywMX9_xa~(^eX{j7GX! zk!SpOM_gF+PkVpW5Gg+WTN^hZ|M%M$r~e;k>MMya|GYc@yW^EB;7Ppv1pC6Bd;U)5 z>JpvTx=h@dbhARa#ZNG?FG<3W)b~A(KlHyy*IiYsyu$x?k5IP6mn*+d4|NH7GqLU5QLs?hV--&#$b)9jy0c_3tSO4kB{ntM^2{>MY=K0>}yAOS392(Zg9_M;KW}|bMa`K%j-_mT=QOwL9o?>3W&7cuGb6sXt;0c>DScA?x9Rv zr6QA!qs>))9b)!y*er;a_fp=mg%*1D=r+fnogn#F*YaF}&?!{1o#-8!nSOi2%p;x- zP^|Nvag2Ty9tmxZ&0!aC)e=(89HCLwQ|VmZtO#v=)K$SJLU*IfFE%Inr8)ck&54ZH-}Z~S zmu$mG=E)HKZ zCI53mBo7~$3-QUgHNn4`Oo-BXzGpglgFAcOf}|)CMvW~h=6m-~P@zYp0U90vXI60k z%B9@a(CVAh_!df&h5~>y3xNkn|AqK{!0yf6=A!nJX$d}a8O&5925@Fmy#dZFtNYJ= zFo5&E3l5Q=`0zC7=z7A_+EBru&|P1fq&&ASyF~@vB8By0L}*F3C%(~+y7zkysu-3s3zpwx`&;$EJItR`ZkH5cf>IcY8Q>Uz`Xn^Ep9WY7z-RI;HK6_Pz8AErylH)W!4p zU0dsN_(JUhjrFQ@%^via6^8uMJ@7H+v8Lo5XmJOx*rvqDN^8kNsEfw=(|`~g zU+66J&=Px6y4y!zi7v1*`!1QMN4VrucW-s7I6;{eI3mAjbXpqJ2dYnRN!X$4q&`^w z$@=>Hc9kIP7D?a&L+WTLEn|XgdZapYI!kP!8{~em3$gJ{h{bcc9U`%B|He4|=Tq-} z*}~}_mNSVPQ93+ePSCO-MTgx-F8~qbeex@3Uvzvjv@Ga12GkU^Ybuppy>~}@As_5$ zpZs*Z3L>Qpi+AG4YOF-s;?*A7ajQ=<*aX&HaBjSsDiO5mV&CSXCD2cM!q#bB1klMa4z z6gQWc293-t0(3jViWrbVJs35o&Dzyss$iHuR|=g1QH6uu#jz2bM)`6#Xqp~oZS6vh zI#`In4^zMNYG|Sy{7BXnU0bd@34v_^x=#0QRn(mm1*AXix|j}cJ*|Lzq5!|nuwF!A znv1`VPNd5h%g;n_G=3U66}1F^}&yF&V@ZUcjv(y zf5%E91(tPb?EyP1Og_<|^P+-v`SA?hu6L);n={2yZWJB#WH>uR=}}Ty=kuDf$6*2N zy*>4irMx#uFRw&Am>i^jBdvW_?ZzG+xdnnmu#=4Qp$|&5rjvFPXk6&lD^x^cw_|tQ z{X2E~FSih~U52EZ{$VZS`Ys|p0Kpga^jw7;_bBKSpH5T3XZ86(%|GT>ExHRWMaJZ;26{5PP?v`K}EXM#-Sg->xxLu)OduxekU zhWzX_EUj}`R&3}49Tu>l6Bi1p*#i6zo~dk;H4>S6t8 zt+?kIJR6!#u8c!u+u@|i$M#3j{$Kv{v-w~C<(l2Vw+Y`&wc$lv7F+9Q%UoxHpS7DB z^3w*>FDDY(!xqMl5OU#dvAAuyxYq8t4?-(zX%Yf}2R~sbBFkh{Lyu;U%Uz2F&J+L0 z06MR`i||2-$uM=nkMhFQtgk%zv&}!I8&WGtGqIG&@c7x~dJBGDNpEyXJV8M3I$_Ns zOZFo9$hIMhM_krY|27+|Wum41+KiQpM5WWm8AXyz*>3JcgtJP%+oK>nclerG^`k9z z{wEcs6?a};i~q9?GvEL7wKRk-4L?;`Z^o;zWF4|se{jcG-DZV{w9;dE*)}^#RzO|m zFu7+~Drx!7EF`p*x%`2bWzU-k`6p31Okd5$w>)b;={5-;eD6vrwv?nG)gqrUEzsHPt5i*O)>|wbLpG5I72;r@wOr=rFGm7LH~iR%EhRK+Eb#!n zXmCL}$2VaNGALqMt%W?BM=}LA)V(_Xo#>#|bd(iO#i%Nw_Bct7b znPHWQInP<J*U=5*K+F+w{&F1e34CGPw>YaN0bS*Qd^KmD|InDx znQ}2oPJ1pHk%rJ&!B;zL#XzROKA-L?5D}4^T&_lwY|x3h!i^|}!&7Em>XOo(ws{Mc zT(P?e7zYX}S?A#@oAVs(&y)|rvJ$&tIoUpbH-b$FI<9l&ThG=iBs$j@#(IvMXnIuS zilQC3T5|eLIys~ocM(hGKl1d7?`SDppukG?-90Ki@{e)5Ne33W1DEIFdaEoe zCHrJAYF31kgK7)No~C_V(}2pan32Yvc$0A$g6I-Gn`oNCi*}*@6PClE26X|y9OM;F zc~~3VfOD{_yrh7&|I1fh66(_CjvgUmZEFFL&x&EM2as+R=qT(kCh_vZF~qfx&-SUf zaN>`ikGk8Z~G}NK~ff0UjL}bWrlY3)Rwa5 z9L4M9DQd#f8!b=j_$Xbg0(|&ilY|dvd&`@63u0aFt9zI;RLvT%1qMf_gNqM&c_J|^ zm|QKW`6$0{;?{kIqU5G{P77r$I2=an8w&fx_x;StUzoq{=jN}}FPBxMKY)<^yFYHp z|M0p&-IGO<>9w};qpw@S>||kao~nH7r&r6$c^iAN)7eo%b5NJ#tQSBq5(u44%Pu{6 zo|W5GC!yl^dO&W;YRKVxUNpK6{l1JVhzwJ-O&&=Vz*;vO`NS{g8uHJUZ%}`oBQds8 zJ{LW?ITY}mcQ$tk-<$<${t>Vwj`P1*%6hWr=R0e~IGoaHKhiIhaXo@yDa@TG{YCKJVPH)q*xQhz5ZJRcf)~n3nw7m;*1{iG^<->ewOFj z*QqrbV})U|lt#|f0yP_w0`B|;=R+O9>uxrglpgufthIYs`4K-^sJor&XKQaQN@pRN z;I|05?6DcCqJEGkJs!D3*ZJb^|9RNxB}vSm_vL?c6prtZBtAGhV{(|oVnOE@@5x?e z(%#aknEQSPzZ3?m{oK5@Kgc0DLlQxBwTEn1QBH0E>_P!%V&bB%bT7q^JW&@?Isl{> z18tIr*<$nO%DD40z=F@$GexL9%}wX%wnQv7Cm=h4*+kcrkCRPTEavUz^wX;@Iv1<( zz2-`X2HYnyN++O5^vHPo+~tz(qIS&Ep^{!`UD}Ji?t{P@qRBPZxLD!Tw|yXY$fSFz zLHzGTY*WyHiI>z|1te7!W|1DYn6EE(9gtZUhF4^&04dhuHQRZo--+A~oJtfw_{ipU z8i@Z+bdv7P>e1Oj^_=fc>0l+I9hJB0(d^5j{oKljGQ{tn`%BKTZThmA$W+XDz?l z$6lbU0v65sAZJQLBab=bHKvdaKY#Ni*NpXzZY?4tIvNon)1(d`f0&;tdy3+7%{_Q` zAGrTrqQ6~4*HJB+aryfX-7aNTeb;8Tg*pD-JC{}6Zg=&x8}g2bh~-}yD8O7+p4v%y zAsf~gqy_>_l}6FNbKab_g1w}bfv^8Fcn zKmKgXq;gc>h0m&`&AxrH%ed9%zpEB^(%R}M5N&-`0QiW1d4YKyJuI>F5e=-dIZT6# zV?1861<7b(%$$5eaj+5jfGpXx8M`d836jrqmgP9OUUoU{Nidi`Q8YB*dbCc>!o+*i zO@RE%XjznUO;bn{sJqI5fm&bT7mgzr=r5(byt;HD9h)xX?E%b?5KlDOO)Jy`FIE{< z%hh7r4Dmev^s2P5zA2LjzLL?##?Hc|QD>~OPv3Q`GEbF>e$8If)axF9NYf6Y4S_ik z7d%D%aB+uqI^Mt9^0)5)yN6p}i&PY=M?V@$;kgwPToU&DjDo9q!Yu~`p^i)DcgR?| ziz3j~D5`YqqWGqM~>z_O@M83{K6ld{`zXOIT^^VnA%s;=+`ax82 zYR9Usi2|vHT?Ei(sqjA6J2hujEfIQ>QnBgsT3^RXezyEqWa}Qc+1*N4!#!cT(C${F zE#hpiHH=|}FNFI;xQlypZH}gm?3;X#38_4k@6T4k(+2@hZ4NkrHcb^ywiQFvH1Z&E- z;sBw)end3FqWKAf!A+wuYn+a#S%0GO2T`dz1*fYyH@-&HWmjr0^`q(6J-F()WT5n` zd74H*r~=84RW4XzD{vDcetmt^JToEkx@oWJy=Ma9eGzpRjWsx{BI{{^I!P-Q7=YPd zEeN$%TCcOHdumGG71vz@n}QJyR2W-Ww@M|EmMFeYWK$05EW^O6VyOePg%^k z6vXa0G_yL^N69a{RIGT82v%;22X3wpn#eo5!CS`F$kM_{a^9fsIWBQl79%{O<99i{ zM-^IEyO5wlajPm?Ia=!#mJ5qhSCIzoVtDZ#Nu!p{=#rAO58LRc!fH;=vu~BBdlWmM zxZSrJ90G<|PF2K+N{Jg14hW@k0)u|+gI|aS>rO)T2G8pcj52G4vVBjG##TJy-9|?> zQD1M~-zGzP`DVy{i)3uzAxl~FN8l{?t=`$Vh-%j5B@kPDvYon&2 zm-@8}a)GjS^;_Q)#`W)F)z*(>D+S)vMI@{kk3#8U(#v{#=DjSzF%Xfe0FIPef|_k` zyx@dfPKil_c*SYJ`5!uPA=clhHvQ_`=x7Y~4n0*~2k701`JbiYSack_+7I3}?TFVU z3EugC?VWX0RDZwku|N?l8Uz&SbZDduLOO;V5R`OC=>|msi2-SbA*6-|=@>#fhLDB{ zkd%<_^ls02e&=_ebDndqzq`(}*1hYv|8TpuuKk_;?RbA)?}9K!Qg_4etCx@ceN|gL z7zSl~wdP{G9dx-mv^Z)k*OC4Rch}Hc6vxGlXKm6`7hd`usDHZi^F#jY>-WLHf2C{q zfAV19BEhKq_mIWpvx6%Ud-}dt&h8MFN}K7*tnYtT_A5nioC6Mq&XxcBNmt^#jW3Vu zea||uZkr_`e$vn7kq(cqeKmVd!0UqDdz%j)zYnnSO21^H-tXi$EU`E>2 zRLOx1Sd@#JrBe_q7Cz=%cH1yST5e!RHvk$*T9qsLTDE{)FQlqHBhE8R=pwIyA8j;pc~w^xtCc7 zH^ofys6RsMg4$e7;KD^D(L*#WF?`&4Y?GUplXJhz4=N1a%=Rv(a$Q=v36-MFi#VBJqZ;m^1!DF#l*lC&;YDQ1JaIM!IY_Detntg8|J(Xh~{vGQ&7l0PH4kv0`yrAv_Mv|Dh=N) zs0jBg7nkrHvjFE9iBI7LdfN6fk}M{ssi)$hA7s|aRH*#cA-zLDSUgg%Yd-{D)7-Et zj`Z={2{DXUHSB(%6gJrOo%=$}cND|0gKuk};#8G(|CPBjV||-!9jhk(kI(W%wFLlb zKp2F|A$ldDB|}_$T-w-&{_xu8QW~Bt$|V*bvpwxZGkC$(wK#AN+sd6F@zBgU=Ys;f0F(qm;FLFawkCRY-YZVzmW|%8$6u zxKF*yeV5+cw9_%;=e+Mef2(HP7Ws%jFHH%Jxs+&T7g| zJDg|oz3!)D`l~+16=;z(3gpGEOFg46A~x&MCQbzb=eapA40Ok2<37!0xR}EWZo3ao z49_v?VN7o%>$j7ue&-Uu6hqN1lz&Saq>yh4sEjeQirbR@c4@_f$c z{xdh>2qELB7cZRiBOJnNKvEx>3M?ZJq@b<|NDVS-Y>25V0WW4} z>1EZba>aFjrgFZ)3z1ug->K_hAbYGUjw~py&bs3o9Wv&PR=k?Ee{yXA|I6*rX^iQW zfvkEZlhqf`l^t?#^Y6Xb6RSxqn1(Ua!1G(l6`EnXcJmj!F`Y#dX39pJZmhOQkXLhn zU3h5kBdZpTF{Ba-z8oRZY;ZNj5*p|%s&2V%eeC_^xS_vs{Q=e-}?st zzw8_QyYWkHlRzl)n5mY{ui?DuX`C@&M+g*WAnkg@hp46-m*vV_51aT?_+FU}D0frm zvUoS;r0vC5L7zUMZr@V`(+#t_x747oH-EFyb(=x^E9y{WCMqOfA*A4VCe!f93@7Lr zpz?$&3EzB~H)WX`6%E8$e<58a(l~c3s&6cuP_{n5U{PoEL8n#@TT})l$6DA6ukUi) z9%P7hV-M}kiNO!!xLG75T2-1To2>|Sgzf0N&GiA~D217>@+28^E?^uwLE!uKXBGVO z@}&uWkMQL=8Qv+dVHP**DT|MwHv8GDE1XSCZ8*;cH`Aj;O=Y0iCu638Wwwa z8=2IQ>ni9Pn0#_T8SL5GPwhrHePBoyvPz|565x;(mq@=?3C20Q+tH(;rHVKfmrj{Oz}V@Bcgb z-Y+#m0s?8y&dlk1>7{1+_u{g|l#N(GzHhFrkrFM9>>yS`8JqC5KDD?cwCINp0ac!v z8m-CL+og6UWzK#D)0Zyx0shwx*(!P%ulxuXW1Fd-E8}u-h8EeOETtWnMVE{actZ(- zW6vk8t-#0Gz&zJ{=lG2you{X6r6X5hAAtEekRQlJrk**zFnLZ;I3gftACi4FX zyO_&?PHY*0p{}*nMleXY_^)eLnFU|1k)=>CVg1P;1BlzUSsahq^sAB6A+V%M8%aH! z`Z@x+iD<9p-4$@lNkTDRW4 zIxBp5jvSr4LqC`mIMnxvmDKo(O6WWQ>c^7F_zrK&*~eyIH3mN8L!tHOBIXqA&6n}3 z6s=7I3Bkw?Vq1*-z_!`MKD!6PV-H@Y!{Ll{z1^EC?nPJRq|F86v${M|tr-bOrz6!R5cQNgv4qoHznbvKGLJBUdVoR3kF8 zRA52pd!7u=fc0TzY;(yx`M#p-d+Biey27oLFXt8iNbfwLE~D-92h?SBRBZL=SKVfY z{<~0JHk?K2Rs0}YD)bn%HsW*};u+FYeoOf;h=D&|Vf`bG_QMx2vl2Ts*!EgNT@lYA zIn;5$P7Q%T8p0b$j*W5-yD`H>(YmMAao|{8QmjKctb4GgI7V(=>dG* zRCV?@y`LB|ZS_XAP=_ng%SiioGc`K!yg1164%1kYnld5528vM zhm|{CjZu7R-yN_hwmD?u#kQemj`M^DVLib1i%y6mJP#fQ*ewp}2HhGK6t@ADmg{G7 zNeQ8=ZaPpmjKHii-jbtf$W0KekQ{)=i7zjo{aj9<`1yZJoPa=gt>Mf~W@_r(%O3>k z8+td2SG~O#f*Tvo4+6v2*H1rp)&CFIi!k^`*SD0E29mnB3(_gPQa3%nZJEwT1*FW%KEP z+m^V`;-KyXfx^Q-QhpIeztOhf+}~3C!FzwZd39Mz1>3dl)kk#p1Yfm2sjy`%GS<7` zuu78zRLvIsFM53orY3XyHFh-0p&yJ2Ip>bUMjv_A>RS%@Sx91vOheT)^y6OtSv41H z{z(GyCmZvDLQ!osQb1IQ$yRmMoJJN}wnoa%&Mxg|R7lFC8W0u2RuvD3bU@;??K8L+ zbl>p&Zw#=Toul)4xOLIJD&=}Fcn{c3{F@t4^?QI*EP}2-vgI$_A0)4OLJk&sYfDMj zg&1x1KRNWVWMw`}wc^(Q2#*1zG|L=-clW_X3P z2jUF|pedVvt>mV05HBwqiS}vXOd+wwceTZ{JZ;9CJ~Qt%+?3{EMk(=>dR{K3J;Zad z1yTYQi@g*vFRkoU+B$1lIYT+qye3tZ+w9;fXoCW}I>UH-gSg!cl6GAA*B3n2K6&1i z1*TY5JWBI8mlX~;V0{nwZKrd~N{0^*lgt}VHnJTaqtK+!=3xtE!Pv=_2m!Quw_Yt@ zJw#Z97-`+N85|4>UAK7-C8JDzjbC@q+PN&AC#k+%Yb?6+N>O16aY0rb*0U91ipYq| zq|DeS?1=ZLkahB?Gb%Cy2aGIGC2ze)rr`5v%ogki6g6Xu*DqV<9%;fGGcsT=jl|;A z%61e}8;IMA#gs#7 zi0=2ehb37en>qZXWrO#SRsHamIVhq^jp}LudkD`KGv^nWw)zN0YcNJ?aXuy&RacHn zz*g>iwc$hZCtAV>4pU3a*?W7;5IUvGw^_|}?WXy25_1uNyA^(j{PDyzsblaD0?dpl zcCnXgTg_n^h+J4ovs_IU_QX@EP|2dr$vEEuuGpLb#ZX|DyC|)14H>HnVCN9VmapT5 zpC+yiqPjm-k%8i3+U$}YS0`o?urbSYRUY(fcr3YnMGb7b6X6CDf7T-A+1FRyJ={Hd zFDM_&-+8-D1tSr!oWzX0;dGymYC-(~X?3MD) zTrPb9Cukf2fyKOjS!qnRx#%m}!Fr5N3@w0QcGXPvPtPqA_UP03PO?k6lbBcEroI=9wfEsz026z*Ubi_h{1)^srDylla{*AvSm>ke7$g$_3!5 zo*{v)cvV(alaaWTS6Yd}3s1XBZtmwTIYF=vxjbACHd2mma!2dW1OS9}tuQ0Z<(Xq+ zU9EaGWz$UV6^Z_cin#U=dcNz)Ok`7IK*TwA^6 zK}*$|!zhe?lf(TrZg{z_zIWYDA05(yi@cqQ<1=+2NC~np7Jy%;YL!+^*bT_#@=~q! zd%=7u ze%H!wZ?MF`?MRkXCSxlD+X%}1FVsbPK9!<8quN^dMnS>dH7CEA9cmhV*}YFU$mBe`#9 zSa%pz5AKz3Q6z345*3>jm<|z9ryNDWVNd)n$$Ds(^GNz+TvnRaiPU|P6#uSBpZh{W zte6hOaq4NkktMXsaXzE>Xi`DIU8RRlH0v-Hh@3W0uy3YdWRiCbRrKgN`Prabf# z*Q5X&i-IizwUiUJh;WDm!MK;Os*@D-_vFF!GAfB|R@PPNa5Cm$yowOPIYTdD%_I(^ zZk_VmB)csatR8n7>MHk0R@nxePUzkuGF*+A4Sj{>V$C+E!n1nHqrM54Lc{;+b~;1v$Rc_7__)Qyl>G;b^}d9`y}^iT1Uk9U17k z?FBiob}_5`PM(e7B2+`4fT?|%hF-Xnmn#QcBbrS#!gTbuwJPx$-lyE9{LuY3Zoi0j4{gF|rJR4W4 z2ur^kLphq4AKWh1w1}x z<#TW7oaQ#{@P3JJR^>Aoo$pFi=Fk0juwB%%WkADfn;Sd8$g$O*9%7%&^_G3Q#*JN@ zTt$G@u<_DrL-;%w)M}#Z=^FjZM!wOl%J7W|`ZSzU__=aqT5fyFn`(PAO(5C~I1m0$ zuD>?-k9tdD)*q-Qv9r5jKF0_;*>u%zbXPz&f@ekbW=8*hn4BDpt~A@ ziQscP`BkstjP7jK$J|pDC^O_3Y40STi&?kb#$ltbO=Jz4Q7pMps%?GO;<@#CxX6eR zW@wS&k*9gBRE7DyVq58F7{La|ESSXtI+9WZbjK`3B%;dBIU~-jt|bPMl%<^iq-ZY? zeni*6|s6k{lZ(; z>?`*Z*hY%jb9XH5=sPG8btJ4!@uR=FYM|ESU?8@-9iXq$VQBbM8Zeu*x}UEpGP`wAH{|=^C*Xl-f_Fyr0>qH?v+vPEOGVJ&P zw&VQ@Z&w);-6MmW9xpx$expX!@70qr}Wc2 z>JdW`*#jeu{D&5HaDJYP1Vo1LN1KIQiit_?d#=}|F$8sA*H@{b&CC8D zf$jK9+jz&?MfO_6xuWS-s6Ef^sGo2x=WLT^*?2`dAQQ@aV_uQ5d%cdPUKn<99^blF zi|~jIy*7;e5aCqbX7z(0PBW$rtx3GKG6kJ-~W6Y0w4J6*!VPU5-ky*~H{VixOXPBOsUsqXdD zsqUKP+1CVb`)SAuTE`T2w)5-cD&ij;>DD_{h5@t101G=MU3Kb0qA>}saW^Yr*DdDW zL&xtTthRuEM_YPQ!~Xr=r>RpU^p|M0>tD%**`CxVE)*$ctBYw+ipXFmf~>?`2iXe` z9Y{oOV*wW%SkFAdo=09@ZQYy-3>l4G1>mFZn}w1T#RX=;q1i|G7L#gf%SOhsd)NB? zJjC0RyS6MQC1a))&(0NcEf*5e@CjVEm#srovt&HKgJtV^wK?mb9_o19p$?cW$x(TR z$8^0b5RNU>oVQe@%@X>0oG>~J<6YYd@Tzi-3^CdSTlN5^_xg-CK51et;=K%dSC`fF z@0M_dP5T#H>m)O)>n#hRLnb?#!GnWy3IiU_77B~+CbkEF8a`Pb&y5W&N9xwvSz6u(RF)Tp9^)uY8gk;YH65gzoU5k#G-(b@wnhH0bw^Ui;{KV_FMna@BmwD*b!;2 zfc2cB%$>B#0ev$VfeCdZmiWt{>@g>`>*K-L;tZIi>BpV7rb=F1R(?u*KoH|9*@1AU znx3>-70Y+=xvcN=H(Upd>x&WtI6~~Nmjc^^AeA>03t>Tl?3uiotMO5;7}oiU&bVRE z@C+j+;|4{386-pqIEm$i`4l<&2^$GuV~N!~jO`$-mauYC7%#6=n$eD?6_49#+J) z8@sMs##g&Ygyu-dhcP3t@hrHB`HiorL<10eE(-uwEL$g`i|HdV+Vln6bE875+o*dB zGooCs@X&$aJc*{t{9QAAvo^6*Dy;#{B_!InAkJM0oybHh@;nNER&^vu(L6?YQo&wl z#Vc`7OUoVad{FYxEZ zWTl5Vh(G~4EXp-TG2_7M?l*bA?*w1^XB^Q#2sF#dNX}>@GQaGQRrgOck$xs%WqAFc zjeq}SIMKhQ`xodT7o-1*0Q0=UA8Gs#V9!XI?@7OV#2eW--afXO8rmrJv2~TN&p zGwiHgxDiJ3>~NQmN2GrZ>`Uc+#c@Zg4asATReRc6Lq|CMy);8!pYb+?$}HM=S%F9y zJ(l2O(TCpZMAX@BS+JvpvO@rdqR@9bKO$N!@8Q6_rzyq37(?r}XLIJfaWga4&7HlN zxK9H-;yD?fqOmG`d75Y!eAZFJEtu~)M<}#}$I9x}`{M?+a?+F8hihlAqUlqgj>Z({ z#pDz#;T1P^Q1v%`*;VF;!(4_ynXuB0Vd3oJi=;DZed#Sj@o0 zm58vFh%k8Q(&dhOs=M|ABGD;Al=N*CKhUbi@Y#X5#cRLyx6e|;SyL1Ca_~miRshfy z&|Yl)h6mS8k7c=^6A)R@K(~th8-e|4LGUrDV8lU5MdJuh+v-@8ahZlpBFAZ9qWp;- zW@X<5i$)?MCx9=N95HWH zNx1@4+0VtIM>Vx{a;iQv+9ZBo_$q)z_(8yOEBJ|~PQ}j9$pw`u|B8?E54@5#S>x&< zlYPZ6UQ6}RXRl4;8XK@t(W(!QoHR4=9s?Y^Q6)b0xeZ+U(&cd0DU?1_J@vN#? z2O8@){E(R3XV&82qMQ3Ky9d<{yP9IA`WR`KR0m>b4E9-<8JR_jgX;n|4l}gN zoVEByGz6%YgJoNXcYhG1yn&2GFsu?))DLGra^4pr`}$r?mwKIH#^yzEMqF^a-rko_ z3EXz3?`ARvtVgv@OTetp01}nmW)-y=;E=akygs%;S}}`N75s>A3Aox`pAKVLE6I~%Z@Ci3bP@7}@8~8Jw zkNIVA&)>kG;5e{@m>1X29sCt^pIlSPl-Wz<4Vo*h;zHSaf|8t}VzlW`L}l-(FWcSK zS|G2xX4ef=AjTkWP(d6|*Cqfaql#^7WPiOXD zQ4z}7%TrQIF~pNo;c`_xI^R4xDr(JFaD6~j`jj{dpi5c4pS(i|1Zw@oi;@4BvH4E? z)2Z*K)(2nOrc%{ST!+TbZeFaF1Gsr~#_hg*Cr5875#RPX8m~57tf}QRX`vo2E{eJN zR!e_!x2Yt-rBb3pxLvWn*Z`j#d%N3e4ni8OO<}P*rl*hLPT?poe{QnMNr>;+lSu(~ z?)KDWB#oh^PrxM3KM2mpv{Z|VTi+`|SqxjgdfMEmBguLl3l~;14MV&$Tz8EK(DKRx z&r3#o&z2R6z}R3r($A*I2rQBfX9)};o9NM<$P&$?SD!+2FP2L1yq=*xByF;7$!58+ ztWxeTAp2h6En7)c7n}^F@;JxX4GQZr^4#haUBh&{j;v?rJf&AO(2P@JwskUUm$O~wNJq0N?b&x_L*#iv;BlM*oKRcfAUNZeCH*bSRayPRzJqQM#5uV~P*~l(wOnR~>*Q#|6r;vX^OUz=%C&$LeacN6TdHT5~$;i8Md-C~NR&S*CgDPGX ztOct~*VZuqy#9H(plgn1VUIGhx7Mmhn%YPFYu&z*W2FzDk|OKUmaf5yuks3v%s79$ z2DT~TM>O%cVS@x+cJs{a9H+hP1=C=j-S!(RR;WTJn7hGjjE%+#_6;UKa0EobYN^xiR}o1bH!XC{e7&skVH(%F3t?<<)f^mj8vN zV~WV;%k|R@>$1((p>2i{dNu5k{J%b_+H4&Ip3qS>Vb7e{PAh&8fR(Z$R>@0V6VNA^ zXLmOcpC*Pd52$??9F(4E9|bz}Dyk}@X59!&bJ<%jGpghg<)>ApPpSXWh|489mpC29 zm*8|gGS}fS+gcIVpz>|kw|;mcXEwSYV`sExgvY8>xvnBN-t&kTb2of+c&9-G_hq28P92vHgOhVUs^;4%{ zfq0tPTJ8yyhkt(zzipKBQb=#7c@JFK1PNw;2Eaya8@k>pIt{EoZ&_Nur@xoX%aOFg z7h=A7eC4?PIJM?Rh{l{B^HRB{N=pX6ReEl^X>1;JjZ@(SUKx{-%*|Js$%r+`U0zqa zzRhaXLCJo2dp|kx^GKYb+P3=LR{+hDEqaXeHv? zB$q&(m8U55&y7IBZF;Tpj=9Y ztVRZ_gDI(DC3}#P=X>zlQ>+bw=x}ngIm>i7yX^C!h;`n@$@rB%v7Gry%OJ-VNo2Uw z+)YUw}oU_xz^)TGt-_Ab6(VJP~ATV|td9eOkQBkpMYs`ba=H z#4G#UYJJaq3B36%N_w>N)nV$OvEI?&KVm5VKCRJ8=kbFe)fcTSe7HBq)lD1JS5gb} z$S$CJ>tLr4X>L+hKgVomFsL*)+cSrOcMK;$tp^vva;QDL>%koS*18R}aWJbab=1KB60AmA$y}MZtB7G6p zm9L;Y?W;D~Q?gmo>XI^%T$Ed6;+g%$z=Fc#!XA^1(`mR=zz#&)g*Qd=#uhqFr07!qtz1y3xR`q2Gc{jBU?+L z{hEhLE#R{)j|Yyubd#vA+}j=-++`GIm%mqpff~T?!{Y;jYd+pvV=ZLNDnXk)6H#d* z)0Z{}enL#!e#na{AHwg&H$yjkBAsWILBXJrO|MQf^dR9xSYUj0^4bd7+83)tP848k zXN$g#RL6#yx}n-d-kbCG>t+lK;d041wV4(z4`Y65UK#T%^GaYZ;V)nk6dncgTeh&J zczPc2Z(UC~bFV&-T|s$F+IuwebuB0Pp3d9fb-?bYxaGh4Gj#H&9O_qLTep;dRh{xD zPLYu1p9h4qb3Tr| z+D8NSo=~>i1Y=RnMiR;po#N1Vu&K_*$8zCY0(HL4!E{INY29Y?b zC|OFa%|udNOpZZo*+ty{+)RKcT+#HOFiQ32Vin zB}Oq2)#p)7N^(AzRnVyR!e`Y zPV1@*lVoClLmFGCVnjZzfpaGJztX+(VjD_nAMFx1ASq@Jo+C+4wiV$A0u(h9updNv zhue^=s4DU{ZnL;pA-Z=}yJ%Q#j-vpiHImD{%@0dQlOYQ35RB&LC>YW;yhbE3sM zb)|8(1QktolJ3n2z1|143s%#^<#V%9wh&kS!H#FxjQHRhbN&#zxVYs3!No-LHKfKc z-MMA}7R}Oav1E$K5?JZga9YEe#X&5s=C%5N5Hy@S{`w%P>w5695@cibD|?4LPo#M! z?7lqHX;Qi^Xt%Djk1p~tGVlE-Spi#Mf*Y{@Q+4|WhD#y_l1_(s zlSp>EG7mlE!76k2Xmd74MctQ7x#v-pT`}FX%xKQM;A&+gmu8OHFej8X@$J*qs;k<$eYX^mKn&zj+wFQ$+d}tn_9(WJBL+l|toqSL_7}lIj^_Fp+5mUY4u2waWeO*F zgH&9U)EZS*nL$0r-Do}=q;VsPHMWMY3+y;;`%)O`BP5JmZ7fV=n*=G^-)hH%__B~I z*H7y|vQ0tiG^yfM-89KegCN`2zP-Qk$kuJ9O4j=eTBy~n)=CJIeZS4fY7~y_Y>el?+aW#XoC`IBKW!S*eZtmJZOrk=3ohfzu zM$Gd?Jt)jxQ;Bva)_elG+SJp&)tRdyL)XQ>QA}sg2Y7ero2)7KYnuDQmc z#i6P>X8*B`qakYO3(NS7iOKB8nb|utG{WKUE%3sHH)IgtN4Kb;U=E+Cm@E!?ZmF<# zT;h96KuqfOpOXjv5}l7p5l{sqZ4P=;Gum~f_yDVUw2^s7h*kxqw5c1eU{Xp#QKMS zqPSsOEwo)GxKu#M2=XvA*kT`?PsgQ-h={2F6f&yWvjIXzGA#NhJ1iMy>R@*t&;!FC z7Oo!5Sv!u{xg~iSAb;wCIZ(-f9yr8q0oW|LZTHV384#HN-vB&_45|xJu3K+nEA0+0 zTO%>vFoTAOC@TcA_*QYS&`r%i&VVE2M}M+s^m{V2A!0*)dfi8^+3!CU(6fQ;9rl%| zn6zv2{4nD=!;S62D>s3ikym((ux*;VL-F$pT%q%b?cB1dHFi7~&iOR)EK$|0iyF56zFr>(iGN#XIP_p0#WV)c=4drP?`?^Qg2OXj6sVjmqL zN_?~caUV0B-g~MB<)cE{H{Q_?&4>qRAtvqWQ|;{QH%=KVs+Xe@-wn$dqItN+3rJ|9 zEADC^x!sa}JAc_8DwL7~55s1R=bH7`4+h=Wx>va-Y2C+2Kn-(*k)9zYF=P znweHSg0>QzBeY*V_Z6M!s z>NSFx1>pr;R(X`e{RdCj>KA9O5Dv<4tdBZVNa)*8XAUN8nIi@n!$J5{ug6PM%7^}nx9I1}Ua~7Eb7K^JWNz{Y zSah*GC%>CiSQQ7?(#qlwi-6Pxc>5vwp&#m;{gH z<``7Y4h4Sv3R+No5#V@rg~=s`Vs6z4A{_H(oLh@|Sh`ehnMSC`UIR>nGlh-s(aT$H z`gt{&OAn}fWLdN?I`Kaf;&HlG@QqF`0jEY2&f~hbYz4+?W80YN{koYloP1I*$BkP8 z-e_v8%nBi(6PJ3yLQ#i8uE#_BfuxQT72$;# zWVW3zY>Fg=2>aZPGA1GdYn^}Z8N9jA^mX}&b!|z}J09I)Gk%+<4+;8uPrHmtTh6Y< z<%@7ST@e9=nVePIeQoHQwWy>u7|rTv;(B`@+DK7V=(Q76F~>_};8#&JJtR0JGRjX3 z?A7;&tt6am5A!wKfAH?yZfa~A1OE$~^&rK~U>e)ERTb}Uw>t0WV<2aNH|k@VK4A(> zHLLcdva)^PIifq09SHfTO3HOMzPQuMS!XSfZ08-E$C0VUxl~z~!ZN3Iy*3azE(iVC zq=Hf^N2iwtOm8R8Y7I4I$VK)VfS&u4(lEg_WyymdNAgo56gA^g{D&5IuTMa1i;H4s zL^i)dbYXH1kL>;7Ny&mQNB5Y_bmhq)aKTA}^D}?gD*VIQ{x8$x0Wtnx;V8dm{BKtvv|Hb)$8(o0HkL800wMC*IR)s4f70xn z4c~2ThkvpU8eiaeclOx++U85vYyFB$TXOG{dIz*d9(X-97c&gH_d z==bD8UYz@o^rQ^p^7hGKPFZ9gv%ot|p=-z)3e^`%pBKJo1MtiP$N%=xkHGBpKP-s< z=bE1Im#c=kj89psa$kLJ8IIJBY>wSE0{Ib=WX&JiD{KrDH~roLEq=UOFh&Re?WVX zV6{C<^Np%)|7*k2_Ij6i?LMMG(P92tVO4$KGVT=NQ<$zvax*tKzZM5{UEekrKra$Y zuhKEOe5X%bZ{w2DJ zs)}4Q`(LA*>VlbuenvOl2Y8AK3=LnR-d@8U+#@pPc663q3fNS^lX{ z<_4bTq$@mThlgzBBH?5_lxdYQ0UTb$548Hi%OhUtS2xldK7b6xtVxv`YK*`y?sbMj zq`p9s%QbmQBInD5Bk#4Y8KfFGFdDgfEW&P@@1+EB^zRre3#qO93fI5+vID=OTbZXQ%qdY7SX@`4 zqJA;{94)J}OQx59Z`;B0^x$Yxw!?1JOn5;jTcr$SV9$SeH9gWcEP@A~yUwIJ?8X?{ zcj3VFpw0WT31b*z^hxx$ZwTd+nPqX;U1LtiqkcAfxA-1x%P>LD=X)~3!1vit)I6a# zzyRMzDOI1OJ@V~hcW0=a)Hvg-68p^t``1x(&ja@?4p}zBssAIgf*)dB7Qj}G-OtnFl?sZEKzK_1Bh8!K~+k|!G|&xmN(E$_+g zH$MnI?AspiM|sEh?rwV|c|H6$y4|^8Cohz+2#I;uZP<-7H-K7DwV|~ih5f}xsxd^FC7u4 zwG3C;3Isx9>~DPA`X{+Py{2FMBZL0p{drRH6?a8rjD?@A+4wQ#tp|I~dtXEJp9R+I z>0lFOB7?zLF?!y1d3)4tC$D?rm#b2JiOwi?R5p^oF{bztUvIcpo3f^sBxR1PTdQhW zXHeW)C!O-0JKB5Aa_8@U%6&Oxn%zo-D(&0X9dNNgicY?Q;8AtA=n9XQ7=EJXCF;>y zXI*qTW@DgwS!GLy;B8l?GM96K{A)y=u&D>1_^eGdza*=qpTx#Sc#p{C#|nw+zBshz zD4*;KdK6f*>$uR{wVu5Zmo&3=ReRZh9~7<`?ZhY%pR*OwZ_k_wZOG$E1+QqGV|gtH zN2KcV6yTm8UAO}61K+7N3aFc9H0KZ=*x>N&r#y+<%2S|QE|`ExRS-FBjrtszZLu{q zFdsDTUb9S^^s3tqDz-gLz3ai2@95@p+$~=3-Dq|W{!lK#I#rj~2A6W=2fUW=PK8lc z**)rc4iV6HmuCd~$=;Tt9AY_rVppo-vBkF$w6zGe)az!5Q%omse44#z$AqFvxi_&< zSz9AZU0#&jQNFOLTH(WsapwZ-DpGxEB!V~WgQfURpTNI~q^wI!uV8&GJB<{UamiD@ zTjA@*d=vB!Xo1%vMA#Zqf!Bt1^62K;bhe8rs`Pko8D(@!OV&k2B9C$^ebfrof!ESn z*!dV%mN~p#Bw@)GQ}$o-q}&ekiOIyuf6iKwFxWEa&SkdGE3*(h8m8|+>+erTccu~^ zM9=&n7#2%$@>tvMW2moM@D7+uDeS`Uud}c1Zk>K0G(SiCqR7Xvrp(c6>4YU)w9bFT zGp;AhG$s?*(a*9hk+F^K;-Iz93$+|R8oBbz1wn92kC65k%ffl_mkZ+IExn6SI8b>o zz3taO-j7R~=f9e@XwV1R(~P!dJRe`MY2RhfnHkxT*PAf3qVC4XV>t!jF1_hy)Ixs{ zs4sZ*ena((dhSiZx2U2#tJvJ8>M~Gc=)H$DF&Ck~UMX3Bxg-b_o>N=@VqruMrTubA z0Pnqt{N-Bt2Rl_S3(+hre~y z|FrHw<1aV>kbW4GDl(j?k=!}EHI!uZ$f{n+LJrieX=nN^YL`LdaCI&QCNd&hw(MgS zj`g=0i<1%z35B=sQF7Ewb}HfoKIqg91;4M{zQupFQ%awxkG4#sLk!z-jU)|muXTm7 zTAdL3$Asm7jA;Nc`x9}Oox)hWur*~rUcM!_=iL1@R-b2~G(i)as22s*9T&sjX%}}x zF*zPT5ho#wo#wlubgbOJT7Y0QX_hzYOTpb#;`%g}PmFbX8U^C?vb>iy4p=pthTEH+ zOMd-FhW+^!jiz`p}FCjuVM_RH9#Z+fkSla0IKd9NfugF-vl#7*R!qPAp z2!JPLvU1j>sgj5`IyOTNZnR2MOS2ua_4fK=66$6C)fGlRY5rx&k5w- z+tNcBrpg=xA5@iBWV3Voc!}oc_g#sNqzv_@i{E|c z?{Vb!nEHEu@q4cIe>v}@*!h4pn;YwAUWA2fhILMBRNWwg8>C3=g>$-XmTSt2NlU#at6sr zR8dfj7tcT5+n)FD{r-LK_{Y6(ynCuguQh8`m{qgpSF2Xltopg|a~+^S+|;-UfIt8M zy8Hn@H$e*;%F4Hp`noqYv{e5iU=X-;5McoDK>7LTt0}RYnwhiXum4AiUtzZPzFxoX zf8j3kJ)irPI{-`z{zaVsx5tDIj=uJn5#C+?xqL2}Ulx|;lBRL~51Qu}ZTBCv+%G!N z&&%&J4)Pc6W2mouNjqH9JkGz-cE8c~UOvC#$6v;g^SJH*i`OsrtHdOZC?kW*JHh3j z1@Hy*0X0DBSN$*jm(CLd08(cFfRp^kGrKGRXo&y-hNVBAaTNdnc{l(xzxw0ZA2RW> z^|Ad&c96>}*vSb1PRjs*)C>TqCjfxR;vZp`m%k_*`(+TzWw|^rf6l;dz!6{vGyoJ} z4+vdSqJS_U0!aK^29yCD@Gtl4igW28c#vO?01porkC1?fh>(Dgkcfnwgov1on2?Z^ zl9Y^`f`XEQh=huoih}x*rudZ!=vPV{2*G7S3SvUy%cuXM``HCh5k%FXK>B zQ4`S6(s7tPo~4H>&M=7R8QY@MGp0F3l?;tc?d&C`^aBDLo0_?9JsxNMdvRZUt}x}FO907b z(#wRDzzyI_8A3lqA6-CG!h_3;0R4*|U)7?Yz!~x2lMOWe#hxL)=2TlH>?d$bj{Ezo z_MgB{z=JQ2_$u>a3+^V$;(q?yd(F1TFQM;Q!qE0@gLQFw28`ugPLVQoUD*~IJ&>VB z&2oan0?pkb z(S__)3H-wxJUG@X{UQ~!Oxbg2xB8H0mNT+gW}n@+6p#Gq1bq0>-7w(lwpC=!eOpiK zJNbggUxO_twM$A)T&2~N-sZNdV>6j@W9f5XiV8Es$rM1zwU*|PgC!%%pjR^6kA_ev zYx9p;&+^SjA0rrN2GdPyET7hpJ~ARf&g$XPs0waXgcp_{i0hmhG;+7`It57S?B(gJ zl00)S@%vC>vI(l*b>Pv0DbBaVW=&h%O5wQRAzLi*dXA`P-)Lz13HY93yXf=vns!+w z%qMf>l8k+1=)|f94Vq+WbbH-=u=pbBd^cZrSiL-+aT47~{RwPLXZ7H0SGZ%QMLU*j zQ~WTcEr~)oo=IA9$<-8EMTcNbH=8vm>mzvJcm8mD!K82E*2Fg>wl|>sH9cC8&&}(+ zD!UW1T7qK0%mWct{>JKM8LHEICjmmIbrEWB6qKbHd9-`TS$AAzuvVT9%}HQX|~7Va@jV$W$IfgFnG%!n(LHwYa04eVWZ^i!T}= zB#`0ZWPTnfYeX=km8OEWT6$Az7k0p41Wwj1rwJ_p&Iz348=x1wr>T7+QZXiV0SX4* z+#qqcxV{X#M~&uLBY|Vo$o_RU!%^;K7_H|?Sb4~mf`)T)$9cZ-EO4UTEWgt`+e)0tVQT3*_-TxH?~)IH zjmu!q`(!%J786n_FKH$&?7Ed%4d^Xi5VegeZ6 zIC2HMyD?4IvSgu4%`6>;Dc)m6y)k)REgZ2%ECPk`_Ka}h5D;996p$;a3#Vh*!SOr| zYO++Pr*p2<-%bd2uYPYLh_Swo%1YcRqv@wXV1-DOVS0l!pOaAwGu$(*__wys2oQJ6)x)v^$SooP8ifISB!3yiG zjD1sXl#Ig@wx2ryNc4opdO&b=meWtKJ^8Txy}t3t588b80{1qhcTzm6 zzC^Ek&#H^oXdbDqbXXg~t62as!HNh zx}Kn?$K59^Ohcll>5X6tF;UhJ*;*V|la{znsAzuf@ydPV7HNIu{R)A`Cs$;6okOYb!U3^M>EJn3pvRntqI8Z8F~+ z!8`5yWYj0%2k@kB&7!#e?U~usvix~KJ+*`kJ8l-!Eg!Yn-~W#d&>@d za*VobVUY9++LylG&`h=w7z!WLH^cCyJMB`Rao}?$9~xiS z;yxNxu49^dr}0DbT2f-eNUQ|w^~zUNt!pB;`?~ko)1rE4^h~>Kq;h;P%znq@NA%pX zSjoryD{H$xEZ$!b&s*&Zr)4_d^T!<0#amYU5K7$_QLT@}0rf7=vY9B($lOKa->_WJ z6*9f>*%-Z_hhjD@dp6}U7iYK687AVl-+o^#zr|~XB;ep(7lF^aafAKxFG~dX%oa>| zSIq+GC4C}W1xvO)@Di7nnvd(Vx7e79Frh4RMuC74i;zQ4qTwkyyJJ^>@0 zSvJ-DckUUxR8tSQJquc=k8{}$in4pFqK#26hd~mRd2BfvDTF474w;ZNI=0(q(m~4s z5z~o#naf$uS0ptnKSbfg8HEt&SHE1Rkt`qc0^#80#LBP~mJ@0}?xDK%>W=pE#p$~I z^Vv#E)U!(H%bdOpWGdph>QDrw3>QudptX)K(G$IYfRDj)uJGr%s#D>!Q%&J5?LP|tJLhM7LLsk6d;cT&37Bl=2E7$#H)^pC-B;X&v|Ege7+1VX9?65L3+eKf*in(#Ut(JjLIfIm3YfG#8{L-L`j{T<)u&wQe`nK# zdQOm`&2pQ{>$6>)BMBZ(Obb?_`cb*x+5gP&^#4z3(n(_4L{%OuW=D7@6)`MPYEsry zqhK@mH8XVO0({UxrdYcWR(t5x>&X%y4bQ};kubmVd%Rkm#bQgR67u=s^+W_pO-cuT zm4U_QjKf>7U=jSlENtwpayk=&#Szhv@a z9>NH|8?QWnwE13ny8PW>_iMXB8QuOBg(q2~ZDqtxFUrRKf3P{aFCrXwI7;VJ>W)5g zCnJ?;vGfKd7SFEMj+-!SX4QU&Xn1|O8Iq}A_|{SM)N3Mc!1Vm?=}0+9Z1!Hol=dR^ zlG8}Icl`E)N5vt*92R8v9-fV4o;kffcp|wvl_%Ar{{G{)amO(!3-Rv?zBTW@@(|+k zT^TmG#!D*EnULgMC=eD#QH5;J<6GUAkMwCBpc#vL1J#V061&Hnkro+lEi=$C|JVvn zSX#NNlZqP5pO&!XjCbbz$jXJYPXx9_Bz;)^2~=ASR7(5=3@@Iap!fUQ)so6~eT7`R z7i;!E8BQM8+GHvbBGaJCg0y?EdBc-mQ&u<7ennYkekKi*qw;XxQzYcQu;ebyZ*7uBt&_vG(5&H&kN&t~>vM={6O z9lNFaeQqdD!4;B!iT)=b|KiKe=-md-Gu@n%eVts3;-D`NoJDtY1+#xVk)Hn?oL=L| zn}LqZvA#LbF!5MXk-)gzN+%ulF~3LJ(mvLy=OZO|&py+oN<=+Fj|}xFU1POhJRA`! zfJ_^mlxavh=(^?zQTAU?`9hWo`WSIr`uHa>FJwQP^ALOgbs)lTbCq*o`h#?a*v9So zj~C-FdX5UXE;ZucSo}8@|Bc1}jKyZua=ONcrA0ixrHIEtIb{=L&@z)0**Vhj7&#VY@OY#iV zds?NL&PeOHnIthvrHaI5OtDefqb7tC*kNT!vcky>fDp78D|=QQqii707E86n(ab1q z^L3Haik|)|_Wd$>MRwH*33#cQK6u65A+m~jCS%*rdtx0YtQH#Wz?cGN%KKc;kE{aMF85^}iiZOJhKINs{KnjjK=Rv;Ixlsf zIzGYndOHU}VcJ=Uq>GWdY+OaKXA+W?-ha_^;qgkuYyTU&B9!WJQHeonCgGU`Kqewf z%NCe#fwL;e?8fnpKF8cq(^?J8tK+`v*!j$11TW0H^5IbyH-i{tC!K7wk)Kbhjhh4_Gs|Ve6A@_EjuJ1MMb8w2Go)F$NG z)RDg`XmOQgUz`Oy+1!WLZV(T;C+}e-MHm$@A;9y%pcUmaz>+-f(X#w)&|!}>y* zeB8Whvy@AK72TZWK+Ao!rD}(lZ;P)s*XugnY@sK#1fN{Rf`WD6;CJ&@P#0T}#4!So zhaeL7BKQ5dMBApo*UG0Gyz+}ox&B4hU};6i$jvM|mqnEbr4(8nhfkioL|27De&W87 z+yc9-xqZHY$CVFS-TIeaZ05_lI;#%}ndqP-h+AkUZ5B3@pQaLF^{6Pq0NHyG@YCF( z2Lp+E-PEL2?q^#(7itDICXRMvGW<6a3yYbd{9f5P;Wdm7!w%$_iW87Df%Ou2AZL7` zeYJz!ab$%|KSg!RbdN8Uck^kstPh0lctswnwAhXXS9oX}5!rhiOvw8jR(lTLU}%5d z!&Dscc8=08CeD6=x>u`MraQ(&bk~U>y|vr`Uy3?RhDPl{$mIu{3UM~hRb36ca)Bhu zK!k2TPp6GizS3+v6}ULaRu+5=v-(In$H}sgSjyz9a~f4>3EU z(;EJal$5+4HrOVeiYvBs6`Wa0byyAx@tbfV=F?F{%elARp=}jpC@4ywS%NKO5y3${xa?NBWugiU7rYBePQlAjXZvp#Ue)WiCi|u zo31f^;DU?-9Q&?q8dIv)N6#e{LbM-6BP$WQQZGtk$GM6aYS*ZgP|xd$n3b5~O1UiF zs6rP2qGZDqo#UxDo!-_c6NHX2MiT}Ia)W;e)APpS@kM`V)Td3mTu;)I?`3@;v#V)O zC6U>Cku8h$c%r1WtijE8yCnXKMLbv!?VH!o0-1-yNfxnTg(EMKL_k_Ji=uD%|#6c_t`#ItR3oS6T6yOBG*Ufdfp`2rfWyx<#M9T7|_XUVC zn+fD@xSRK75cZtn23OIn4Pzo;b+vVf)8(#9 zdFa}jEN$mNPItjbkl|g;+|K8U<;}x)g_YlpgW0gQ6R?O;R`m*O5r<-G?C;DI((5qh za)Aqz$PciCQ5t+UnRVPobQK_xU!dux<0lmUW2PC@@*8A z`&x{J8n6%14CG1@ia=vwDP&;3IZB@BD^IP}ju)D1QH^#OE7e-TuiiKksAz{7)a(3Q zYAtotOjpSSNMu4CfKY&hLU0fNX+;ltp;kMzbLb>}&&p^gqj`_sT@3e(QIfN~)~Y)<#>B*ufL!p^v4q*1$eb92Tu2Vx%~M4*H;Vw z_30Q|#>TbL_WWlQ=%%UI+vDC`oO#LZBn{pP4!$yy!`!BaE%V#Ak>`flMbYLy9w&s0 z^|wAT=8Qp=IO)8c0#ahrMN9d`S5}O0$hw46RB1H5GU1`E{$Mpkgg^N3z|-ivhspSzEJ`U%iCH>@rBj+feUQw+`XOH68nvJSAXiCK&gJjh97 zEllT0I;BxjfD?hW5%Kknd)cF7!`ITPO%h5BsKgSjI$b1VD8Ebeb1GqN$@j`@?-z)N z&JfpBJTv+D`tHXJsV)DcY^9HhWB%($Z+^HoEY?WA{&x8~a7V;HYuwAz>PB#+RU3O? z(!jEbik&Vkk14)O!eo{C^w_p}qVI>$IJ&+P&&omHx@uX&j40)DW$6f-@TB7W;t_u{ z!myj|m7dK7+=Wn_kM_Uj`RbnQE19ftwDMKUr^~7^NFd9jph-^wjxQ*!fJj5kjX^z% z3xMY`FVW%YeD~{;JkNF7tR>R5)Tg8a-WRNKseN2#NFb^Rzmbo5c^Wc@j}&@ddUk6% z_;AW{D`8H-m3T^4eEv~G(M9Pir_R?8-`{f~tl|?rpU(HcPRJ9f(`MEjt3_}kjrSh2 zW~%&gCG(i1LgisHhT{~_OD~g~b3*Ce&-3Z@;LA)Y_Bq~C>{L(MZx&&qH6VB>uli^Q zRKPV;Zt`F9beBIalRI#t(j&?9Y4D)Qq+IbK-BRtWjoNP$Sz^?`T!7?IQQiiAWj4aE zbLZ*k3g$v)Cuqq?gj0QIx&Kj63fuH;J2AdeiXV65@P~0sS>f$pa%LZDf2z^&O_Vgf zD88nCG;d8dp0t}IpZy{DC(w6%Iq)wZS=PBTGs=Zwl-Y4w;uYp>-Dd8OyJJf}?PK34 zT0;3lDLQHpbkjKZgIQLa>Z$+m`$nwY&6JUr1ibgq8>I`qE=F1$-_GdB(Rd+g`h`~i zHPfe`nsYx?WNN60`VWsa>f5M!4@xG`op!U8ZV$LPA~?TAP?9HuFEc9=sFL1oxb73T zeVD%GYS^2=nc>Mqx~cL|yDoYHo2h(pAi%~$YV$W>5|TYt`Le0rx3f4mY!Pako<&K_ zWPD`7!6Yxj*&jr%x}H~&&Km^p(i|I)Mu|Sl?^^pia1n!KrxP9E&d;n>e(`*ES2NeU zePE2^u9{Q(y8yq3Vv9%fId6Yl^KkxKaJkHj@$W#!9jBcBmciYc{pV-J2zN+zWDsC5i*y~__rYQci=L?c)n8xa^+@Ec;1surLQ{LP`jC@!&!#E zhRttH%!!vdCMCMkMP&Xs7dHhG(d1sct5=0KGfdgc(=IhK;43(hiit8P_ddQd_G{Uc zU-|!wa3>wxakL2h#(2D35?gJ?=wO+w=htAAVns`YK@``+pcOpI=u+N6yG)2I-V5X4 z9F!@NU|Hg6P(zZU>SJ)8KfRu+5po!djm6sYV55+G(uuwd{UYHiDmQEYRtGKmH#+D4 zL$r;`&5-}U)bOVKztcG*sobK5MI#NaS%~tl5)}$Wv`9TNMM{>RwXLego`lO|Leo%i zwv@%^=EHvrdBbzH)Q`-q`k4fBeQJ*8%`9-DNh=?cpIw-1lwIA4#lmnTnvm$)I3;wiVY| z&7S9RJqO7@w)C?T95e?7n2juQC)Ynx1@x6;_`P0yAWQY^2$+80Ngt3Ma6H{$1t(X) zTC;w%W;2ZX#N@C4?k(osm*N|(bM0%x@2`81%k9*Nw7k~jrUbFu=P71L57_V-4(kqK z9L4#ZsR>vLU(E8LwG;eyN6Z&fqg^0Yr zL4G23P_pq_b>mF^p?8&ie2-_#5RZLGLZ(+k<~CdDMX~09U{ri{Z2sAB~kvn`xOXR~FJdHYElFklBnqo4{Fqp4JLuD4O$7^twdlzG*TTZP9P)-16 ze=sRCx<>Zp8aY<^YXZWJmNu)dcS=-;b8kjQ=E+%pEjoX)EBkmjRw^%-QZmLbXR_gsoATHl%6nj8e~NMq?U%A;fb~r znWCpSYD{AxAvHBETo~4QzePGmguD?;Xtc@hXfVs^$@4CpwA@98AQL$M)51Owyq`cS zkME@Zf-Ldgy_==q<3!fVQFMJ?exXPn8ZCW-X!k}fh;T>%Sxzf|?3G=&js?lw9lTd8 zwR&)^oolS^r5bjT>!izu93!pbqzD;;38n{s*4#6t*xdA7E3=HJj-jYUeQKW?^Xdyn z8fVze@`9`sp%SA?dT4ocHIlVtL0e3E5XQqm==q08{}7U!0n+sLM(hc3?;MHhN?c5> zS&to*bdFC`AjGX*L&y${$P=FTCh%owCo5ng1=am&rCp8{keOkh13EOVPxitl+fVZN zL{3r_>nMf83hIYphEpPblncI~<~KH^7#!>7-Fv-(mIAYVq<0EL&ObP;4GHYGA4`#< z+4IDkpDHm7<(3GecR;+2XDTkFFaEU9a`#sWy-h#i&c_nW?cPJ7$xg&iD|>Sz7KjL* z#D=0_6>dHRWsTpn$?^)n#*9Bs!i;EMT|FRS3iOqT6SGO7z2BB$CB)92LaAkF*~R=q zT;7i-F@y$`&p_~XN)swN*g_>K&;fh@s(6Tvgt&T%P@5>eq+5MuX6mozolymY& zn=CEU%i-<;!*80lr;_86b)#3i?}o65Q&3F6A(%@eYdzqXX*38#6z(MV)o!D8h$8c@ z$@Q@ftFi0R97Zb}Vh9;e!_ zYPEw6lM6THWcaFESah=D3EoX|AW|UM+;t;dMet#*KUn*ZeoyC*n$7j?k=V-m%hz)V zh3Acn?y1tPuiC+|3E?2WDa&3+x~QEE4mwge!+L@!!FJr>^P#}<K?FSS?d=(9PQQBB$(Q^zfY z=0g^h2&XOFdU+GGhV`ge3b-Zm%M&PGEu`NIKJY5DQZbwgkv^xufbEZ$kQ6h@<~Y-T z+GMP)cQkBeEysvOf}zM@3iMas7N?ydnndKLQY%hzR$PO(n%?{@*|ZC5Q(fM;DRwE> z6*aO(I(_#&Y$PK*Yy_H$BCo@FTJIpLPOIz!5iR9mMoydK*#BvZ&D7+JnA3M>F3j^` zEz6%Fw^RY^B6CLW1Q7_Tp}H8AwFE+(?9_bi3)J{}3|^e_g#XS7K9E(oY?fQI)2J(h02N>tB=M+4 z4^$xhr@Ztobo4UauCVgXe~Nx|mHd{l%8;6@;Q@9~g*7uAs^rU+9CBg^7yzV!t{NwL ztxii1!X27+^{e-b0tdYqWV}>Uyi#KexMVakOKAyWF-jpP5~~121%N02F--VPWiZo> z71PWgt|+Qn+;$YaAFmwmcj{9LEunBJO1|R8bGKL20T#I#@@4BFGOsxuME6I7)JRdU zBHM&SU#Gl)UCmT~snIDTN}8D#Sx?509LbT>qW(I21Ae)MgH^eqzNCrq#>XL#pFn`~ z!!cDP9qIp)X+*1~CtrG?KY=^UKY^kFqZp&ril4w!(QBVlUVM@B2l19$+t_|E+tOQ0oc1d?WWA>cS7-h8Psw~TyIKFp^2)x@G_7XNlqed%dgk~C2v*I~3bG>QO@U{48{Ol!j zR!4Ln6BwvvhvA?Nzx}uphPzsZ>gwc$Bi(H@>mVK%GpyowB!)C2%^ITr zj`Tu%d?Ce!gZ3be3;5+)78*C0)BUwpy*=>~Ap((4NbHmg#~0-(#UyyO;j5uK9v{ShV|U=fL01i&T53bI z*wCcnl@e(D4gGSzL_;)WoST9-y#1I}7GKxHS93)#O)y#`uKO+8o~!8gJxM~ow!PQ# zMm-T!8rDIn?l3|qz6>4+l}S;2xx=NRfJVj%jW4v;9DLF&Y<}8G3Yk@@S|H$QT*P8` zOnw3`lE`s--f7*a@-Q1h8r{arohE2jrKjfsr8-WM3F9Jk?b9v#2|1~9%xn|pRtUd@O6n={seO{HnP+*YZ2AN&SW z40u=}I^cS{jqAxJ!j#uDyFGw_mB)hEwZVpsV~N8ouXvYiA|aczTs{ysj`?$ZB_96+hskwkl%Y-xFCsFi?l%ct8^ zk0T;(T)RQn;ce=$#=2h3rRtMekOhQof5|PAc*m>k>W4 zOir^Rq|+n{SJ+dXd4q+*MBwU>6)s+gyzQYIl9ssj3R09VVHlledAD)qJ)#fKMb>Y% z(ztVFzq}&P*B4tK<_m<`r6BarQEc(uCgFqt7F|bHzX)%FBIDm!{R_RnK+lFbi0I)?1SyZRQHrW)7Ir?I|r*ET&XC z)Z3DUhwZUK$5>%J)T87GNN|eBPZkaFekZDBS1Xn6_vz zmHG*!NjQZ>H=Am%&UN~BTp7y<3k{|oFj9Q*4M%m3F9r?7cnD>?!6;VPtper^r$TnV zUQc^q6s7;xdiS224%6F4b7fhCwXwco2(I!D;*DNOm#!BU8dp;Uh-fo?t zxNhcIRtjgDA&RgB_N0W_3&967RTNFuIx|9SC?Bim#IuMdohyj2@gI^|-9E8UIO*l< zPVcLPoGu9jckK_QTo;c`ri7aoV8RM&nBq>A;L@{lise5`OUu?P9j(cIKOGoKt&oC#4 zI8Tp_8pcpEF2I?WHFTPXfuLfj_DeLgBE^-85|E{-)RfN5@P*8e`;|D&&#YAWM#Aig zwzVsv>~>WZaGZIZn1IaHN9Awi(@mFJ@x^ZLjJR{HO5QizgKKIrDqj=+w(Irsy_SBg z_P8EuRwb_I8X+`Pw;%?6#AAY(3c=WTMSNUa8T$O(@gs$K%XHS>)9s~JQwIutm2URz zgwf2aO~TUOZ43ilkV{O7*)-B}4j1J)6@9Dx?wrCO7};2QN|-7iNr7=#M=wHDD^`+#fSu$z#SJ)q%~S8a+*WfB03AJL-3Z5 z=CBHex;ys5vn{*==h6Zh%vWhSAZt>wJZVF{qx#s@IuaITm`|P$-Z8#ylVq>>1iC=F zd2@R?S=)9YbN5vzr87P$_(aIRt$0n8{u&F+>Q$il#JCvJSgN3KQ2FigmB}hGN$%0y zH23@5+f{Z|Wc>>^1(E^lJJNR9Lz2>6TA^wNkNS6k=`CD9ZI&koyASs@Z@nk4mPRff zkyLA06kD`h^lavEM(g{7c)Zxi2nR0?VAGyvDLS)&35|h*rZkAcvT*ItSP#;5y%`|5 zo!qJLa;wn`!vfmhhYXi%eOG^E3BEWPeS1-k1;6}XV-H3x^@7P#UsG#^;dC+W!ImfE z^kC$nK*4)Xm3j=Je|@&Od@n|@?Dl@NwX7kHqBvUpZ8k<#f_2(Um`3GgAJ2Z7hbhg!yrGlU zsO`J!Dck3VPCtS26@Rx`Lf!yf^ZRAjGAN{mcRFXRdm5rK6ev!8SuaB3c=#Uu$ubT6 z!8K$*>jPonxI(g2EADHVu0RTt=1CMqSgzaadT1Qh70D!uoI4Lovhq?sij((Qg5w(q35rYtRUjx#1i81 z-;Z1{;rHmzDxu6JD6p}VlNJ8s=tljqwNi2Ky*^%*$dDDm}=xU)0km9Ud9#Jqpj|AR4MiZWozdo;zaU_izv$2WJVfQ+o|q8vCjtawu% zMUKQt40}B)9Y8eJ?o?y8xLa4+urS;|I@%Btbj)~_FP4ig9+O8PuxP-OL7d1@7;Ph3 z24XGnkKm7cYq7_^8pM3VT#|LwPuSP7!)5Vmsz!Hsg?aT@{cyoyX39-FD?^hx4uls+ z2no9lC9|f;%Hma4(SZQ&2H!P$&SgySH2$;)GK}yMEJu5w^V&~Gv>`ccv(EV>6x-cFe1pco;cEZgWW)qVhQ6$ zkfnS#)WJeBVZ+Bd&4i}ZR%zLm`~kl!r+b^t=$JKBWstD+stDpRujWN!R%~+@M>Gp{ zB!>N#7*>-?-C?t-Dp~yUE92B}fbummPY1?(Z0|^Ji|;!izV`GhI?3 z-xNi9ZB}wcA9xQTmZnu(i{!_{0z4EvR-bgiD(%H z9#P8J;T!0;#eu0lw_I>m`KUVjb2;b3+pT#%Nf%8d#oaZi){icK= z)-}cJ?V9)K5q5vP&{=#^;=Il(Nv~CHoDv>1E1rS_h<<=8Yz&_kU z3|lF;IxVcyB-mlM-paXiB5kCnLHt1aQ5L-mbkZ@M522pP*aAM{7{wF>BMc}?R=h;q zp58(GG~e=!G%ZZYvv?U`-lBO?mNNPNpn9kG-r)K1>{VLv6s^~fk^Nv$+7*s2oDU>$ z8EkrQi?P+<9c992-q(@|-AfrLndjhDS>iNKY;vl@?kPGVWIpUVpkPC-tTVq`w`vRI z1CxFNA6YLqnRPS%I&Opt$^6BL^!A!$#L!)BOZ}_}Vvlh7&sD_<)jN-_iwDI}v%@lT z`}nH^p8t?>;#&qH*}Zg_qWeV1EXF>i{{)D6Vcjmx-)6_Wx(CJY2hrG@NjN!&D`$b- zKV~OJ2b8DiW0irhxLl$vb#t5wUc;!XO`^dUZnDp7Eb9c?^5t@)T$#pYr0K>CJfIy8 zRw%meij`30>e2)9nu1~vNwOEIsxQ0*hYonvvg~$}eI;26*s+8Cp3x?;ZCXVh2}F1j z?^op3=}3ElY3)9WYhO&*b<4c@FHbc&hwCLA^3wd^;9n?Bj(2&9eMPXQqanf9@8fD$R+!DwAor#+fUl!h|5)^w%AZ{ZgAbr$&tQ|_GZGy zyc7iugb`)p=)m&O_?r3^9Y5APihTr=;tJko7+dgGqOARU8(>doz3NBD`^1*Y#;1&^ zGTi%&Zfy5<8!Ee?B}t9UUdvgZmn0S+$Dvur)t5eRU%QI(7&NW#Q)l@GzkNJalOg;_ zb$rn-9na431P213#vBY8jqRhr4Ckw(m;DH7msX;zuvI|i$w8MyI zp=|q8jE+I%)RBnzaiZEz=j-bkTwkQa=N0=LsFvJC4d3x7Kb9iO(kYC`?xbCR$(Oqw zq7*}770%}bdJXsI=6`au{E5AJbg1XGi3+Apl4fQm_{FGxTp(~P@y(wKm0cck*N_e=;NhV_Sa$RSI&K-x_bPoiY1VnPaPO9O@?B==nPG{pL!g z@$dcjZ?p7`|7Y}sf0x&$*>B^sv{9WBQkFfZ0kN~uXZJF zk|zE(IT?^|R(srRJSgtob2l}~_DbBNa)*lnnqg@}%{sVlB;wv_yq86fZ3S!Sj`B4#)C^Hb{uVRhoX?HBs1GX7h zl&>oAX`3cGjXmtxb)mThjrmNeN)-xJ5I3$Ae>io7QLy|c2xha2B>eiV7nR{o2Jixf>}e(8_saJu%U62aXx;ssVkXJ}V| zw(5(ssI{Ujv^SMfL;-rM!qRp(s5nOn$jJ7^KE3S2Xp*16Qmwe#EYA{-!ongBbY#)^ z=8aYkPubAML&uV9^mVA@MRJzjeO8*ym9!5F4C^)~jb)^82wwB5{f_R+8sFEElvBQ6 zgHNm9{QgPlH++LC-wLS>mj4>G{^qZkQ!xBKwpr8J#s>Yz*z-4kndybWC&}37r`z{q zLz>(arrAtIKG|2EMWg^ryZ4jX4_p~~)xZ0Fl~-;BzbCn6C=2Ok4LvpFe|o>-Yfj&- zJ3b1kI(4UIC!VyyJA)UrAcwvl6~#J&LDN$2qnD~ic*jUS4Nv5*p0WoI;AWq@E4;sC zPmtu2ptkv5i0@HcMtDSjK=FP5E-=;TQg^ck;pKw(6)&Y^q8gj%yX!X?W-R`Skzhh! z>o9B493{is@nwo-ePY-(RiNl9TiDbx3lcEqg?>>gb_t@gf{skb5b00LakkNhwEOX!=Jg_kL)gt{puA~7^KfBsUvfoUZC2_Ct-I60} z@?JLXzib{lEkijiYW9TiIM*u^SiV~JS#WVCt8$hqX=4GXBGH$n=Y32(R30PFq)BQA zp~$}-$=fh}b6@z1zo?=(csP^fD##Derf2Pj&3x@pQQIni{Z|rG28F*HtDwI|=)Nf{ zbVVlkiif~hIL^&C>fD8oBG<#Uh${S$-=jd}malo{hCdESec}>Tl7yND98XQwhha%~ zF%$Q*)hL!=yvDpC7o9&&mJ?@W-D}A>c>L!q4>OZT* z(GY2nxG~ekbe=R~_Q7yl+oxt`v#YWg<^WW4qv=Tjq5BJF7}XULS;C` zxuNV|frua7$I$Fe0BZ=En@zqs4~8(I3Ujcgwd^!Rr0`w}WjF^Q;W2Id0RA++ZYN~( z1o0EF?r)iCr_>EcFK$uLZ^I^qrdp}q0}|1ZAs%9^eDr_S7iSIF$KN6M?YP`)ChMQx z`;CNbXIZr%XCCm7^rq)kR6ZI)1xr-QrkaLbm*S8Eg{SD{dp&q$lq*Vd+3ds>&> z60L9n?=BS*TT0}!(1HZZf0_oueqv;U4Oz8l;btUxM2%M6;j)_v49>5^bVDIYPtrmB7shqv#qSpgLZbQV5co)O zfX!5`5!{edG`>Fe^2kYjGysSJz!6YB(=5_I{fQ{>_KAFRv9lf@ zmyY&|rhi^?1&@(FVA@L^LaBt_IX=@u3cIl5=vDtADE~_NPq%D>cHW|=g`{FdjTuh6 zh+j+*n9hB+l2ahp{%FU_(STa*-14N>eqNo^I9=bcXc2I9do;a|6^CWc*nyJp>Yz9W zi#!CutwM*y@yQ{w1)oqZf!Uedk=B37W~9XUxB46!-#dBU^?u|vZ>gm>jbjF)O6ISm zJR+2C{#%j%5KoqOuij%TzQ#ZDDl>0*?SGD+XnmnAx)og{@ITml>#(-gZC^OSy##j& z7DA!8mSVxB1cFQP;u5q_!3ho_IJ7`R{Xzvc{Y<{a-B<9+AMZ;bI99Y)tIcFk2Bhe2v){@bVLX|DCl1vyh%*-7Yns1i$V?Vh zM#tHzYns1i$Ul7VzrrbAXuEOV8{egqFT&JM-zftwcX+GzzIjbKkEl`x7y_b+%fG z1DV!LgppYdm*l;e=T3sl^p}g{q?*HK_7)QQh&($px-xSS+Bgn;BUqg8Z=I@krToDq zSTM z5iJ&NAba4b;3JpuI6eFe*O-{8RfL^v9(=mRZ%vg6g61#f(}8Q`QsY241$W=vH`iI| z9hlf!i%LAT(4YCVJCR^8nWL#7g;%S!=NM1|^-sr7QbmzsVlQxmZ3-wal%e-TmeRXL zT**%;jvf#tc~+sJGdbhX3hIr{=vX3Xxrc@gPD+tIMW{Zj%Du_75Q_&?s~&;ocj3W@|+X*K*zIB9hT;bIuYLEwC|I?~+5&Q3?8>w%9>_X|0KSWtUZKI#L{ zvr5$kuyS^)M>Pt-03!>n#OMDo4^S)qp3Xb=j#WC`2k1L^$5u-G3{hG$O4L0nml51S zpkq~(B~YoxA4$YZh>4&S{xUXw>!%VzF2ncGF_Yg0ub9L;mnWUr*^a$8lnJ>x)aulQp3?kY~j2Z9xm! zS`|w$;y$PoA5B=L6E2EAh^o9|eStB3q@|eWj5OJiMT1k!J5^;UMpaJCXi$7X|LCet z$0A)(m-a@#Xy~I7;95a+m&3;uc)RV{#EWDh=8PR}f+mNBBDmXFy9-2!g%;;oq+y&M z>9W3`2T69E53P2mp=DTqGG_)bQ?$CPSz0K-Fdf+`%}=S$(7q4oj!OKyT5&YSxJn{Ba!Um}xBq zj-_*1g3DCvt;CTA3cB1=4H3j&*>jy!9x z_Y;!~rZPuzHgAw!p2o!TPn~N4r2w~>OSADPY76UBML%Yh2#2_4j+myGyoh40#XTxUd}Y|a zYBAq*_qn_*7VPC>+P7`w(AleaA3s zP5Mn16uFJctkYUiS8ZIE_Zxdyy9!-Vccv9Fs*Kd20l<|r?^iQLM-(ePy*Z&WHl&3N zmUJXm8m*X6v(&92^dbv!pyt38$vlW{@|=Kga|iULV(7v9F9KhO8{WH)PfSg)8DYa$%y4fp?Sky%e%$_wBonQbO=uU$Sb{0uQ2>6fr(<_$f ztSe_l@eknbluolM7edhialDQU zKM{knAr>hkr70e(0DW@8mZnsOxUImbhTA^2FPtqrtz9>*Fx+(FeuGs;xkGkNky3gkABy@a!4-Q}RrGS|voUD*7)ue9sA*$VPRd8kAssI|h zPGe55xJt09$p5}}WpJQruFe)+@4;t0K;oR0U}rhNPxli*OJF z{;U#(6cAda(Cu&_R>@x>j2>Cvr9m9%tDuws zK&b)IB2~SwT@B=g$zN~O%zp2$P7|#8{Q9%M(E$~OlsEi)wr*oUfAG+BVVQH3M%97N zPobAIQxzk3?JQ>X-ra&Qtn64phGdKtuQ>#}gN}7T15N2xYw1$+T>W2fSTx~@JkXl)14%&MwZR6Y-6lYE9 zl6nq9u6%0}^`wX^~E8Hd^*7n1PeoWE_+RfQGS7IDCFqKjk!M2o7=0xgz4RDeQO4OqU? zJwS_!oVTfKCu|r1=k5%W7-kxvD53$BA45$DZJAy7`W7cI&D<@XNO|~}Y7FvjMAd*l zsO~E3JrE~xnAI2}Pn5|9z{J7I>vAOAOJy=lnlgrIQL!GHYf?99N@{RfNx`!Ls|+szKToMxh2SxH zh7ya5kp+SRQrFW{#z3{ELg1I&O>32$!h*ptgF|C8Yc{k1)eJBc5~k zkJb^|wlY@YR)oSN1BhHb{9a!B?zKy1XWNOUb_iDCXxCV*SBvME-%eHcA5?HNfH-PB zoeT(I*=Jq#Je#u;Cs{)%T$4`zn$Ti5)eI!P7j;{K-Z~rbiNzzA3+-o}-WT@?WvDj# zHS!Fzs5FX-!Vzh!ilQ-;LZ!k99rOl^>b2T)XL!h@P& zI|W>Gk856|Gf~f|@$dQB^Q+HadpkC%)Q%cBcv(wRz^c1kAn!s$AP5JT1gC@$Ct&m4 zP~V$hT)Xg}+;JR)iR9%RYFXH|_`1==FgS^3FCeMn5GAIKk*3E<;ZVX;F-owH?a3Wb z*Y!KamT>=-Yl;pyf^F}mQ$s7VxlWqi^04A5`mq$D{mbgcR`qel;awJ+b27X~?Qg^q0%;ax5Ibp_czKl@sycv~ zA{<_7ahQZK3II4{?N}&u+CCmZR`NBK2-p?e1od}BZy-*ty;cSWG(D74|?tnN$t3+@E- zq&=ZbvUDrMde>7Jm2UW3gD@*h#|0iti>n~Oy_Y>kk#J-t=mvvQ@xqN z89I0+=QSYX7sgVN4{cweMx*Zk`u;CW!}Y$_eFsrE*{|W(pbwoBgV`hOPO52^c+V>f zC(ok0`vlTLobH_izybNXMS3e=h52(a4=6(i^ zuy)!(e@l9{x7r5p*qwfhdt4nCwWLevSpdetxp^9`C2sW#CHy5+Mg9XBWQ62cmtTc} zDh{J6g%_#>;tE#y7RsNWG73}wM_7(T{evfKH%}cd6_}LPNhVTKG>t)ePlcZQD;BAw zHxTFh2C$S;;<)P1>j3aUK+>r%h+sS$@F#$TxFHKZte-tnbsGs~NEkuVkwJ|9UN7ZIhP zGUtLSgKTg%%rakC=~1Z_IEz}p4=x!F;NYOf&P$E{FHDwzRn%V?J2k<8M$Z=;FPw*| zD+0=&-X06^G_-4~L3cH_y&plxcKZ{2iNEhsj>8Ran1bshrBMoX&WMZ`VEm^C7yp>s z#V@U2cZDPPABqeikjm`s)sn^?5U22XUJ#URGBQDZn zDHggs_65*LP^{*iW{s-d{YMjX3n*D4%b#AIlG*gjZGmcX>)W&y_Hd4=5a$7ql@qmJaxSuP-%b)50lV80*9Aso@Lx5{fabLYbyKlK0%CVMQY^AXm(37 zMd!Hb&K{9LE($g!O9Rs`jmANRB09?CIRAQS{u_4Ye!oWQN&BIff2uvt*SGy%PLt8> z8W9ffavYCn9JAsnKZ*T~0RWe|yIY!jb+>cN%Zv9NX#HebE4)0&WI)+AxB`tR)kK1n zV-cKV9Ck5-mxu%GMiBdtop<*6za{3sXA3ahqVlWEk(=(?+rzEf9eRx48%^LQRI}_~ zR9A}qrc}+O)?Ua%!a|3>_W~)Vnd*=0uRm<6k2h~X`5Ru?)0995{;7=^sH`PCdh3hc zJ?^0;$7a0U4Awj$L|$v2Hb~hoYPe7XX5CCh(Om?#2gEpqUCifJ^d7-+41YV>G%Np z_=||iMsf1fg(@wa%4_Fy*RF3|10kupcCL5r^5#wdy4~ND`)?_83D0RlemweMukA7< z?Q|P-Rb8}`Yx&6!L7f|b@7NRWhkBc2b^B%*Dr^lHnv1R5Q{|u{jyZOOkf`7^c5`hw zNSaZ(N?Px!?Kx>m%}kOlG@N{R%;kehO|XSLwBDSVS0ur05)6(o{nL}yvgKK9j=FaV zKY|+@yrLavL{4>YAW>;uv@H%!7dsl`>SfrP!kd-o?|i<@I;pYc(gd#68+|oeh75aD zH|i>?=HD0j5kkk**AsO-~yn2Gv>kdb5&{GOj-I_EMYfPcG602@G*ylyEDA9f(U$fe8YNWEvveYp;{4OQzP)(H4^Xw7%tx5noaXn1yXjDfwf0-AV zN2b5Su8aHJMSl;jlJA3;p)kCjXWkGon916-CtX_2%OEn0^w_v{^JbhXjVMvmw)BVC zrYRD`QZVp~(mXV9G)IT+Z53G1Id*xtP8gJWV|7CLc7hq7MkB0}hT0G9B%`>Irwl}a zaR9cItT(^H>wJfyD85gY&CQ-kGG2EEO;`M>gFkMD$Sx3Wlg}x=68Egs1UH2Glc^_> z8F2MU1LFsT7XJEv?cN2U9ECZ?9I3vIk#+yM$R}ZF*96muZ$q=5e7|NCp-x{PX zO3@T}QwK)5YPruxE##HuOoNNy9Q%bFPhEzorZ9R`EfiT!xCZm3K*Pg+3(IrDV z7JkiHNj;^<{F6fh$gJ|{84!O!*yNW2rjxZ7jvf^lh_B@B<~Iv@swio-0(O3ji+yUGpC~C;FJ~gZN#~k)@LFSfx)08zopfHX3o#*u_06L|(pbd; zUbubUH!TB9F$YB-4WI-k)Wu@8${&x+L!k0)V=ZZ*aFw&wqyx1K+-YnBo6o=vEC)GQ z${&%22%AVRghTC;Vow_rIlm_G;U z_M{B!6nH#>ugZn+ukEm3p(Zq%Ux!`Z&frZ7Z?pIyqb!|;PD(XWP6)0|)j2TysSiQ4 z61TTpV*0XT?gz~NVY12Ayr-gx;{;Xiuk6;vuIgEG^kzEx;$wh|-1!~wB0y1a`u$Ow z(KDgaJ6r1j+0ca|YrWsh2xYh8pYv|{m5yC8-juR9auz|9X6W+b!(PzHh!f%iOiQ2< zT>uIIYlKyqR$hxLg?!NEX2jn4?C*ex{!imhwP!Jl>!(-RBzJQ=F>;GhipMenkK&4n z!3r15QNo-mi-6gRPUr%wj77R~gdL2q?btY)h7+QlG}WtW9ha;E)=i?8&y*$f_dT;0Z&enK zWb1fiYpFvMwNeVh#?I-GJG+BvHZ@t9v*;H!t*Az)LEPFce^^9QY^fqOL!SaAfM|)8 zFCHl*ku~O3UCr)0CgG3&P+cNXlm3hQXoZ<1yKrct_ylWe3Zo@5BIRv>$1Pae3g zCp6?ON;KZPQ)MS}V*x( zu&wKOWVWM+GeVQ?J>(-@kBUm5+NFc|i-^f3nK{-`#>Wtb9(x=+Y;kw3s^q|4y~E~_ zbhDF3yzP%q(66tfGF+dVkQF8og*y@)I|m$P+tnhAZGeC~9nPF4f=9_UWYLi-Zhc(E z)M458z11`GpqcpJRHYLazh}pHx^xEQ8YDO232biFpLVt#3Iv!_rm4FVKzUP>HAjjG z=JX-u@U_~o+GiZ_v2q?2bh~R^^KQGc9dBlSj9gH^fZHaIKZs<;M33dMZVKNQyqMqu z5@Kxg)PP)<0RRy%iSmd3FX=k|6#YWmC)-yTGNLZQ4$vfhuybxasvZOVF2Y8&JwG|Z zE9F{{Jwv|F8vT`Sj?8Xkt(5M4Yy%gUW6kc<=PcC^@GG6uR}2KgXbh-|eOV}BQY@vy zKwKq?)f_n1sV8n{qP#|B1a=dI2S1;mr8Zsc$gw#VTNF1JT0rSnVT*x*SF^)$=AVe?Z*!LFN}FCBhEmlcRLk2omUjbvfPxBpp}E#$XoJGB z;_0p1hPHPb_jqfGh$7mWymgv3K9QjqfcPC&U?!e&qMt6Qa@jUO_V`GpO4X>uY-PG{ z$TfnEX_tFJMZppS?*26*ZA&0FP@h9UXZkwyVi<2M+;f6BUH7~ACt|-FB`s`_I-lIw zr4|RGhCtstjEwiS<*epn+>Q!}7c39%%*NPi;NWq%;sbyc9u_y_SOmjwj#@J2pSulC zNpxke5lM8dh1Rt`yS-f`HnX3sOM})K((k14D^lgSQSLaUKNqx6G9s#`RLkk(_}VNm zv0H#yN{pk@cX92okCh(nnW1PP7l|^08H&ILIZ&D2z)*?W%o3xqA`JGGllXrS>)}wn z``WYeJ79e+_xmZ2*@EeHu-Dk%-|Q!i)2BFtbZoQ=)lAAG*SY@6VgKt;jc;c0 z^%tI8b`553;MYkHtrB?hj{!krRep3Gt)tg9X!08R3mqvDHO8r~`3ro`jHXnp$0+^P zwYyydwX49GQB#tJ)_;r=raKs93jmFcfBs`6uR6|<#|Ap~A2sFl3g7rkkZ1Y6^JRLc zCR26D0*eGZf&PvWvV55e4mv#RqM|v$$-$^2QO-Qmii;nVOFD9oiWUym-__)sJUi^} z_)S(Wf8c84=j3;5+t$PFN8;GIHJbe&t~Sgy+x_Qz3ckBtdch02Pfb&r9iD*`JFJ5e zdPdT_vr2r>R=6QR+&MHhauP(Y0|o$O2|b(TOQ4FsP{#sa#)kf=r)e#jJX06nWk*{d zf&vnn;xwRshL**&xsq`Cbsn^2s4XRKbO++{&*2y}@B*5LI!pmRK8z2fM6;78h%ADfnJdj_k9^6R1+ znPr{SA9a5Re0ty=k}iL3Y^8xJI%#H&X;dL zL9Y)+v|wHblA(F7P3i9Su^F@3=Ue~F)4zJnfOHvLq!+5n#3=S|syKzKWwkQUv)_$P z$?|Wj1!zt*;mxw~k1zoUSy{U)$OC={{NJaf|4JV*9$Ol3F?!=ow-|SV>!Qk7!;wSP zGLN}O+o7%zmlqun?s#WT2t`B#Qwa)ivFP|y_|-V7)Ubrer|6FWYz6$E;tHl^eEiYz zb{alQR4w{Kpm$v8*jf9e!CM{lHbS4>M#zYTMP*fx^b(QNMv5)?+17VszAc!r=VP=? zUrmAz=#aFRf>cgD`qrwW#=m9FK;ke4DQ*dc5Jx;sQZi^S`I3bwqcg06>& zL;SoAk~CF(wd2Xs zpz(SkNK((B4$W4J=yvMZE(=lKs4F+6$PKr~;YAw%+5OMS4IIpt zUK;%ES!xngcPt}HOJv_#b4D!OvdLX$=XqjN%}lqLk@~d7y__LdCgOtIx-L<*;~jZ< zF7j_RGH}p*AGU;o796EMD!FR*6A-U;-sRv$XMz_TE1j6_-gB0!V`Li7jj!6_q+jVN z%tA-2Af-4h1$g&>U)Y{D%1lj7R4G@|cL_|sR*=Pu=BM(w&{C%90BQlSe@xhlBf+~A z+uSL6t{FF*h!Ur~4SYmAC}k5r#%Dp948cp-#T z6w|7e<4^=9MOfSue;HdSqWBe4%BkJ<)^2`L&2nT#<~DTDGN+`8KE2tmGd*B5@4-^= zzDz%Ns(yH-8(8H`c37B_EGrB%Esn#`&F8z)5_O6#+L@Yf zf<{r|0vLMRD$dOoI z@wL6sQE+%YR%w70Met)%9s`M02@nn&=mS{yoSjmP!Z7Ztq&rJD-5 z?;m&|qp`k){-| z?cAbCbF4@X5Kvh$!Vc2HrX=`9*{Q-*l|;B5xa~DQx?~-21(c_A$JuA;`QTs+2(XNP z{VS_ru>5na@PfU2X+rCcb{3T@my=s8>z@)N>h}HnkvBDrU(hdC^@5+Jfgl9a+rikT z%85p10?>U3nTzwV{lpJ^U&~vUCl<7WLl`c5zR5hCVj3&3{uZO3j>Cj6XCE$!DqyXm zC%6wuKyeW3i;L_R$w&y+W6rpno9ai$+i@m)AIKk}Pti-VC^gJ*Q7o?7w%Feaj|~mT z{hUEOSYUi{ZU_=}lhPq=<{{w~#_R$psfj{A&X=ah#K=2TAH;gzc8l4$aKmR~ktaEYYcBTH&1HeqIB57$ zOJ07^nWDt7Q(z+=K_REi*hdHg z_rNCX0T(LK%w&r51PaV!+P5c=5gD=0LWU7BV|WcTQTGa&)tMj~%JEO3=fvQ^QrxR> zkSL=VWv7-QQPO5z+CWyR2Zq~WO0?-sV@sTJU#maK3StLPpbwG6mYkpflsWEmx`mhc z4(-DXKNc1(K3IU?Fqeq8@#`g6_-b(LcKQiKQ0f7IE2SMSE2Wr98g69{eQE|pfE-MH z#WFF>zQ%%C0$C9`%&*!j;kivdfdq{fBhZLxRU=jEyGoE`nqiqvJiI>H@Ag)7QSq{* zyrPW8hiF#<3OOlVuqr+O$q9Gqkg8-qL4lkOZb4EwJFAU_<+I-;vLR;U3dE+u1~6Ti z)}`c{(}K6Ia;%;rT)byZ2Tp}@D+ooAEBkTpGq4PfDKGCTagJ5)jT~*z<3*J&tP$Du ziYsA)W6Z?c53@znRrQFo#vz6bRKRt7P7*x2xp?0;I>k%ea-z0@y9|<78XbnA8VE8M9T@5?E0)@9C%xK{t${HI9tyb zC^7?2f9G~t>iPHFQTfgT8H2t1oWz)Ik1q>vl+_vnP-u90I^S&}F(Z7SEEH$NT~~=y zabWHNVVnEMS~CkBCi%`hE7GE@RLwv2xF{;Gy}6DQd^kZ*%&N zyqC>dtoazF;cZ56%FPFHyz4x4Z><@k&o?B$KWdKDao^-ggIJL4JaSN0FYNKS;WexV z9=&BbL`^GUUQq|rkqv{9uGHqBQ2qYg`%jHzTA?duaR<$UEMsG1@!|dFUeSr`YNFn^ z9GT@!3s8(mVr<1eKbZk4DXaoOUQtP#^1y6=d)T^l!hgMQ*1V_rbY;jGU0<_6nmW+H zZ8UaYv_K$C%Qqg0LjVaIFGnTYywBa#k=z^1nj6ce(5?AxMF1}6T^~rsEXAIo`(-0_ zb33I17~f^cuS7{nv9S&2G=@#d4S%9J^`G%JZF!$JaVMV@d?|vzAvWz#K~e_EmnUpp zpggOngu(N;QbiLl(4vW5A(cO!BeX*>c~&*dPQH@6@J{2%b@g>c3Ymm5%09Ws(L?0f zv`TAt-Xv^7HnVZ7c2W17AYsgZ*0TZHnT#K2qv~7W;RmG3((dZit$QpCtO;2+ir=kO z>MHi>u0DQf;(6$39OD_7=b*65KSbrV{_K$1O8a!SrNTm(oYz6f`A~YX5sl*BSVy~$ zP+0P2oP&(?0ThLrU29sl(^3>Ut`g*>TI#GjcMP-5w-A$F%+1n`ww+9mKEl0CF_xf# z@jwA_w{B6IQi!|V-T^r-PZx#qPv_{As{84Bz9GcHJxOXQ=1LXrgcnq^b<-nVy8I7d z7M%e1iIW1Nd>#@z5S(@nUtIcp)%C>vRyb(L;~hV9%Y&K?qyxXo=sJ>%N=6fVOC z(}tIY740TB4!ts=+NCsdW0=QM2SKl^q&0=oRpbl}U|`}ar+7yBgBF_BYpUjvsiw0v z&iYNB$lCLG)44Jjq>AD25Vd2gOtNITqasuu<1)e3)^M?SXK3bRV5f?1vpK8XB2Q?Q zPK1CTT04D%j}}XgxyB4CBe+Pcp5HRMw)+MmdaMY15B9}zUBHp$&ajP{?lpI z%42R=E1f^T*0I?Cz&HP(zKGVa%;uU^mh7Utli=eH9;_NA4g0>Ds7Gj~jwecP0jez6 zf{leyJUT7{P*6{+x9Tk3z0^t*bJkc5m<2(2xk!#kqxx#uo-La|NWtlKkGPF|ZlpN( zV#Xfu2B?M<@@$(##41%on3o*hCUN5jDzo)iR!VW)XfkKst&jkc9J$ZTZ$6qQ6{yME z@lyTJ{YKqe;``5#nJ%lZk7Fd~K6f`i{qR#$&k00yqPJ?zwY%FN=NB<}jwj#Ic@tpJVt!MX)=}uBto4?6TD1Dd^ajAl zAd+2kK~a5|MDz(7k)5loHrJ98t0cl;qG2gS>;3*`U%R3iO-;lRxC-?!0xK6~?Z2y4 zvv>D)G5fZZNlj-Z+a@{$F0-FVBlAQRnrc!A)C5w#>Q&+>a3FH9a;1Go7f-BGN#-|2 zY7wLQI+VA(t76JN(vIPbl(@5N(pswQGWGoXZX;0wWDNctz+EOr6zN~)af_<^0h*Ld293*PaM$%O`F-ZNEnwfHYLw`)v? zjF_HPHhO8#f+xxSjpky%#B)sodi$p_NH_?O4f5%^bWsNkZi90L2ZFShC_w&4$NxP& z++x&BjWX4Xx5P5Nlb3wnZ_gfp;>C7um5I^Y>;YoC#|(`$W+u4_vb54x%@b|mOU4G4 zBE#Zji2%_BoXeawz#<2%!7IchBNNL!IH|(zglkjUETLORKcbMK4g3~_v#KVPUkK`C z|J?!PS;{c4ndqK}i+WlejaHj8Yo*=7Fw~$Zzo@<~lN)zW1u*!jTFZCP(6*Qn3Eq-_ZFB$H=KgM*lNV^&OYG7nelTbO zhl|}HxU=fF>K}N^VVxG=l2L-0vB*JDiQuH-xb}sey&*}=n;W({>z_XmYg?I_+!9iq zP6|*y1_|MSVf%hwDP*zqUN6RW3(qQ|0KdWwmDaUR%&s50c|6?gCNtyP_bJnrG)dFv z2#Gxs9i~UK!NQiW@}`GyaqS^mjZ` z|MzL{Kgw6WTLf&vQ^rIBo6m6FU?QkcOq{NC#Q_zxx~i3o2A2pj+D2I-z(vrJ1=C~2 zQiPa`SLnz2YZ%pQ8C3rY+~Gm_DqiLY)!I*eU0DdCz-+w0ecry_D!Ov|X7J$K%Ov(0 zL-T~X5YbZ2{3PooO(G1(w#gsul{eeuVgcCp%1x9|k160Oz0((0-M=6K1DYa>e^$2# zev4K3`Lx@2K0TL@$Ln64-Nb!gl{G!VT-~M#n|^7#uJK~-d1KQGZX{-EQ9LM{hyJd& z5T8zoY`FDQPlrdA_Uv+IW37|rCEJ4t)F{EaO4*Qmhm)>>7e`op1)0r!5f0Na4(^%- z4P3W_zZKSLs8OIC-EoKoef3$S1Z{@plrPMe`!Wr+Y4>%TxbDoCdY1B$hdadr>+)|$Xp`=x*;?Y0kG6SKmj99ASV2AH!n zF?5H1Y*yrSC1E9igVuXC)P(sm4HcO)X$F?rRLE&RV6|gtS)3jksQ!Gqm-Nw@8Dhw7 z=&Kr(3rkAkhFVa73ne~uf}4rVy~%5bTgPrnPL2#q#3E1Pe6}S#E02qw_OgaGooBN4 z=nKD_hAEN&2sxZkAs3ctDN1{FH@(#>W7pz5N0-sDS0QW>(eWC!qnEDUjK%nJO$aBn z7PO64Nd%7umu{>6g1npSSwn_WXSKtoH1~V$PXFEKY+z@Wl+!u*IH!J*0fbkhA|LTy z_%{+byP;yYi=sEb$H|^Ww3bg27!zr}+j0vv+A{O;VY#f>0seNA>if811Dcg0b@+dXmh?2#_PXOTIYT!0vU_C55 zNnWqF$pgC~6!LZTOmn861eJ%@oFgrNpIWx2Cd6L#zXwmBVB1De->^`*g39A!^_ zkcc8;&gR1ggyN2U%o{z^<*&f+mc{8WYmJQ&rd%$ap&rasLBkc$P+>F249`(jib7Zw zF+l6kaM^&NKy(V;);D{f{l1#TM=i*Z#TgG(Qj|KIppQjoIt^iTBb}U)ds{3Fj#T6~ z#FYlJ>fuXayCzI8YmL^}t;Opqu~DgNWH<#fJWoYB36St?hE{{Y1u3hUL8#c4le3x+ z{C~At*xNhy=8T^ieJN9W-z!McJr+rw}w;c7sc}YK&FQ^ad*bD{>d;K_6hqRTsg$A=xo$-@9CA+ zsn+{^)k1~{o&pWQM3(l@VG$EyR*+>}45bQUfl~*cgQ0Yl$<564lboqbTk{pq+lz_H z@Bp+xGl~$zp-`|~Y==kyet*BnUhHbCg~Rr7xP3!KS+y|x7J;W72DdX+samlu1Mq{8 zM~<3XpM94X^@G!bqI%MQyoN2MmLhTgJ6o+c*H^j6IX>iRoFV1bwi2luKpI=d~ zorYc*)elO^7e4ktM~W-N8x$vQ5o|S$%Bv| z9lnXr`pX>MR{k}~e_8Th?)tAa`LB)V|B&7F4m~TWNx8;GCT3%IUsnHe`dFu+S0vVS za?z9HO?Hcl{@ZrboT>rI77PJKsg_+7rdHGX{Mj2;JZwd-B;Y zjTk^h9A$W>SR#^SPl6$+;yAckNmUM)-aNG4W%k$`+lN#%nErMK{}A*7FR6t^Dk+zl z)sejE8;P9%i*-Ioh+oF!%ge*??i&I;r0m>>wl^7%=oFtvk5+P^&vGhLL#2)iAhtp| z%uNKZ9vG5Z(hm-GPdeW_v=DOiD3i>4#m4pQBCGPz9{k2QPp_32qg9zOL_{qZhOD=OnSl91O$WMn;<|r5r@1h4BL%<#@owX5Y`RZjS9c>l6+1 ztm>L7iln04=xPe~91}(_CyXp9cDdu=rUdfB;LsErsaym0Moo_>o2XQt2O4-U{LLeHf2rZ4|Wmd?YEjFFEb>C?cLI-%nooSR^@ z7SOz1+Iu1=h7HtGYX}Ua=?G=M3jZR%ezm+gA!PK_vc6vOdzYIuu5y0-esn4>cU`;b zF*Ax=Lk~7aR8okT-!2Fxci(-$XKliL6JxcO0}rUm<(K13i#A@Cq??@cMi2PeWsikE z58HC~Auk54@^Au*41u=w6PK-|+DVctgMsuGXA4ZJ7QtrYym8(eG&&nInc(AM^2vH# zSgCBuR-Ux7&8pmO&_li5E;;9Wt#VG}du4X1K_7VWWh_$d39E_2ChUR!pPNBUvwe1_ z=>%U-x@JZecj6Zh5BIx%gjjhdxesRX_rVy`FAyRFCf>G&c>qttx&nNh#c@3Q+*WG) zwuwi6)*yN&ZxruLL@8z5P!=WRp%&&m&+AS{(snGa-sHnq4CMQr);wu+JnNRE2r4^ zLvN7U-l2N^Cyx?`eihy&RO4`RGT&`$Pvc0AHhdBx%1uLF94wdeooNwR@ai>+cEr%# z*MWkufmV{Ibdt*cCAm&g-42@qQj-);WoAbd;r!k-xYhV}_LO8u%C$%`hmWB-2bF9; zdeH4+;#TNJ>Q*wIGCA8p3q2ZCz+1)d z=Ox*|gS%bnKrf}!5kyq-^7j(i+&Qf_3CMpp4)Bq#v9+A6sy)Dhcw`(#4bE3YaT&#E;W|MjD z@YE6bvi$I^_f*?A!`FZmBNl%L#H%<(OEY^Pb5qV18xQl08_cD(z8QK$|6sZ~o_mug zuBwslY*k_2Qj$`@Ml_1rT^i!c=wWzk4-fCHN6zg$k!8hR3o3#%bbNSSzCF6duU)bu z&RZ=KVN69?BtUG&ja^@W^(Q7xor*8)+8qjoW|b1GUO7*$3tFw!OnZ1)k7}vsHC$pF zw-{|>6Q8*&p}}KieZuY{c+q#xC!8!>%+XT@I~_MsYY%op!wz?!trzR2e$`diOqsG~ zrAn`-79&&>35_F4DW(L(&m z9Ekvh6L7@dDXkm{&z#6^z0HuCFT>GEYA43LU&@|C9paloG9+~5G(`ncx2b499+YM& z)7hOa(#2Z~y{iABF7v8J)ss|#LDO2hE;Epro9^MM_P&Nttbq(dbFEd7q(h6h!HFMb z)gu>&Oi^hC-k=cNlT}5#giQN3CV`3Dg~Q3ei~gn+hi%`A>#CX3K_e$|RxU9< zdTKNlko^25vI1Fuk!keiSwGuKm=Z1IWN>IhJ)Ehj|YTv3;o4t zgX;6+gHG@KawCd}+=;@9(;H1>dmX92E153u~WFS0$_HRW+P00w@W84Fh+y^Vpxn zZ5qtRww+g*({AK!_NO=!owXDFWNoMAa!40Ftn?OD-0~>nMDgO28NP#@)g*l4tulVw zUg}9`615nwojPHW-G;D~+!y6!L-cms$K`;g=am>GeiwlB_2n4TRwa!o~1M#~mw9$RY8yJq%d)1u%OX-y;8 z^3a+mOljOEiP*2F!10?t?EnQn>u6lj(;bFE{=0<{{`R(urauvMkuQaw_qK{XuWMQK ze)HTkJ_=vuw~t_>c?iSMvQfHzMOYIDz49T!bJl}J8A+$t39TLr(t+d!QVvCN?W@i< zq7_B)M#POvh~uk<8!nPp5|2fuq08k2qJ0?wd|92mXHncVK0%sKb*JSJFqAPk5FHyu(Sm@3(YEI!3A4@=5wli_@ZNn6C0EMhvhyl~t z?|}dK+KvC_O8U<(UAaE}n)DZ$?o}_}3f9mr63oVlnJbC&cvL`M3iu_Z(#nvX;f-N= zXDBHlsOaBLXIuUMMDtWbs#B StsFA*^K#nnXn{ybpV_(Ly`TetC!4<|*X^PQXi z6#^V+XgPS0DXS`T>$u77d7G#tv;J@t&J+U;xe#arTczAu=JJ8D^UUYov- zVte!JRFVNxg=_*OiU&l3oD_VQ><38e;5#w@+tLHzf%N`QkL_jL)~B>Ruhe?dB9Ls%-((Eihr+7Si*UTPS0LWF7G;s%bUE_4{JZ1Pml2Qf z;nW}fCfgHzCZBP4O+L4=@J3VU3pj2Jt8%P3!i91TMX-It2gz}jQHMIqx{%9%r1r_c`PJZ?2JyJ8R9A zHRrtNx_{RNvuRIF*6gXlE@LPrE_8Do$S=$=C&hBhFF>AMRO-^aUR#k2)Y-TJMv(l# z1IiP@m;O3fgM>009~8Tz(xc`<oDy&RGN5>2wjP>IeOjVLIe3 zBrC!r$gtT+1b}?AMhZS0LACpm|Kxh&7Z%J&sFcEJp?5s$gd#*wn%q=w5hWm#-;SYu+2d+_Rl#;zeUy7J~8vcB6_x}ntrcqC z8gF|Jhdm|nY0Hhyg2HEmey-ng-MrVNSS@$>%Zs8xCQ8u*O3{Pnn>_kWF?gbG?g*EV zxjC-Dm5Jzt%Ec&R@2Qnzy>IGZ@5`IIRmC25w%>f0K8$>8Jz#M(mZX3@u?vudmJ9so%vfm+PzxBrI?iN@u&qet<1<%n=O_*oJ~kXJQ5d zkC?clX&W=(kk&vE*%TlDHn#6xRk|GFmu2L0`ZW~z88_UP7{z|NzHo#@L6q~om!HGO z^BOT3S~!s?5K~2Qx;t5Fv>2LFjN0iRlp5t7la5C@5r`fAvshnOd||_Cm}IPaZ+e$` z+hy&R5N%9qP%dJzdz+##-A%f(P?Q*Das+#zJip4396V%Cl&rRM+MS=Ddm)jK=XlA@ z)yY>8OQU+3)TKxbZ*~# z()Ct4_RRf#IxyWO88%J&vI^F#LbU4)^$-XpYRAaiNd>u_9k!Y*8|yiITafC@(X*WJ zh{FpryNN5Gc|vV(%Je}q(9w7H?8lNd4VsR1Xu(5hS71_;vitSWl(a7Jkj`SuiBw7Y zI#kiRj|}miM0Ui%6CUMI2rMPZ0cg9O@xE1lU8$~ZVxxRV9VIqLF`X>^opIE_1q%g8 zkeFb3Y?)XJdY@mlsnR8O`Lwl$r*`=+=2n_XgR#+&`4hQx0me0#V<1_6*%?v+aLK=k ztj(XtE#VX2AHYUE9z$;wD4WtYEX>SH))|st?4tY_jKcRxgCglIRFIv9z;C5u=L@b~ z{V*#Y4yGIthr7R-hxK1Vb`lZ-hRmTOF5=MA$=rOR<2Vu_&phKm!L+DAvyzr9;CH0z zra9OZw00PqSaXhpG)6PLpfpq$p@hvSRBM2|O-Mi>(Tm0SoA<`JV#iT0=hQnWo05p%xec>y}VoTsk)$xYr+Cg{1`vr!xnm+|LbZlGUJmti(o(|}uu&~WC_ zBWYbgLMHq`BrSl9?sA1nV3E=ca9}iD z=`-t$ua_tttXLNAPsLO##V=4ps`PHQa1(kc;INncyNN;1Ui>$S%5G ztJ#114?s+}E%8N`r;N?c=1KE#cyI4APFW};>TL|y0#Okmdaz~>2&z{aK&!XF6eE`OjuGUIUjreqyA!t z%nVb0M!sslk=jBg@;h+~UdhVYy~_}J^L^cZPCJ69m`c{<)9kE{XP4fbMAj~T)352H zqe^Rp2$PYyxvh_U4s3gYSB4sz@$&}*vh_I2gNS2U^U#;eh7PCEFN)l1!Z7ysOjFv1 z@BGF0u($hyc?+Au<0^|XR*_?BfOmbs+R>=~#gPR=8wnpiJ7%FL?=&yNOl)@zh1hbv;C(Jr^mkh?Q-;Wa~uMa8|QQkJgOME zAOq9231N1vQf}3QIZ9ne_UyiqjrnUHDQ>B{{i=y3JtWJGQV=t89z5jEBJS~X@`#x) z&}xUS;5bj}-Cfu1UG(*K*`tfa0`kOwx6m3!jl~kM2y(opt~|&OG01}D9RTv(`3Zwd z&}mG4bX75ybT($&zY*&<^`m?p7IZ%)Pa>=iQbC9ycwFz)_o_|sfQW+uS_SeIM|g2TkjD<#+RH|8iSa2-|-%3 zNm5J5!aEM)XT@@0vw%>sxPIhhQFg%;l`d!zO5;Iax?@|gf-Z3soybIC9ig}O8Wk8N0lpc_H~^oV|^#iK^qa53O%Y> zq6)SL!e_g?(&}i~*umGWeNC1ZUz0ObJ}0Z}o8B2-QCc2*3jU$r2Ka|kL2aZ3DKFY8 zdc4&!tW&Dkv&>^_1{)87xqIkjyDgXmwahXGOBLp?i<2xb*kQ|#aPpQ05um5qsYl3iR;Nn&8Es@Ho#wdw{7R8xGnED7l=ixwC+9i9Rmjx-_>4blf~vC3=Y(u zHz8)JTT4+1Kg!&1HPzfwD*H?O`Cc)K^rPe9_tiIl083?mJt(~SGDqgxaY5#T>E?U8 zZ7Hfjr#r%rB5a;{t%MJ3=HohCm#))4RMh#x!jLsN%*j~;?tI3I1F;!Y#t#P0GU;5r zyg7gO*MQ|eoj)wvwYdDoZ>NBlciMoE5&g0tY3k&WbTFA{s#C)tWEP2L zV1%}nV3zWJ@8mbR)p(vNkh8673>C;KxdmeJbFz33Sm^oBJNp9V;7~DZdFsJ*+35uj zgVgq#iO6)z1OLI4z39-_(=1o%zL^5RD|ku;&;Cv$cSDKyn5X;NtM|?he|g=#NYVGD zmCCr@TG0fP=A3Cq)zlR`-Gx9OhlL6aI}9#HozG)_tNaGCf#Zv2`E&=2v-8j4?UoubtvjM zZA2iKu${n(j)xN~2PZdG497#Nslpd{2VaGX_jI?csNR#HD--MI*R)~_-He~0oLxEN zTgDyFhR)ZI5BpZM1j*MhdPa^!T;kj;O?El%?wH}dUa&YWoC_B>F*Koi&jf<9!Bhgp z2jL041AhQ9H+RTQrctGOV`1A21M?1Jo#rv5+VSY|K56fpE-a?JQ_F)|Ftei9x-dw2 zf$3v2&UL>X{v^kboWFaTfgf$>Q51^0k$PXAZJGE_U<>&HwLP)F07&w%7g6$G;|4iT z-^Sea>$VO%azi~izVkZY7Ss~lcHwk{iXnSye8ktS9OtgykV_I?);tH#qtzp>ibkkc zI=!m{!y~QJ5GA&GPf=n2F>d$uFuM1kv?%N9a*^J8}3}l$XDhCaa|| zXxq%K$mrXS45b(oVmY}`VCA`=x8sJrk|0ITLTX=&YdN}6%mX%xXjZW-H4Zg4)iGOC zlRi(LWVqifHQw}N4rCcGzdBqH{8WtoRoIf*Cq@z--t&={TwK7f1oZxT%EkMn{0>!z zsn?-E^ASn`Zg43IfpWfaN@!hs;kVL~sLuYY0?0?lzVHy37;L_xpdT2$yPWGpDOCzf zRHjDbQU>Io7Ytf|-%pim9(P%p{*XL7hT8dQ%gDw)Kn_e}%_m-mk>^u%?DM;Zg(W!# z5}qOVN7PLcE9dmI4u&Jhq)vvW>7Rchw^KfC-dMEj9xqJm^`dZ9GoMu~mB8gsJO(G4 z5fJOjAH*(tH2oB?T5$EnZ%@*o`1OVk3a%z`VOL_51IqmM17&xB)ooUEsgX4iB3cE1 z*JEj~dUEsr%SX{^4@Je^YDsFkm~LrgitSc^)e%?DF63W$P_k6|tAlY(`-|&Fp5B}M z%#H;-F0D;SM93}e@pW5d_#B5`_U398;}%J3gRvLD`0OCI!!m-k#^;L}s?98rrG;cU z7JS1!xq;QwPX}c;iHZ0mqJzy+bJg?yxXlvJX7*qPl+Vznf9T`*HCW7n%W@M_KQAF- zNqU*5pP9zOPrzuhvq-%QkKIgg^e%G(0V9_#3>+95zK%)8*R-3oljv>SdRC!P#AS=- zt|j?t<82snWd_Q4vD^Y;8BLgAkBsLV0!yM$59z-(#_bR38|W$LX5ULdcxxM3!AoJL zW8(l0$7BG@ufuZPsyg}V)yhJ z!R%MuYA$!JTM9aY3M#Bj2G*0D2F)&XTMQCHm>23)F_$;aakDLiDxq|uK+BrV&^=<~ zym8a>>$#b&shrD6^_h*VN<8elsvW;PY^L!iw87$Jg(74*CIF<4-`*@rHPU4C9U;sm z`qlF0F^|kTV z-&V3lqq5TD;pdmGXhS+k47DA_MI$1pP|_@suNl-5abr2)o9W3&V~vYvF-F%isGfK% zASg*oq@VV^&Xyg}5f&E@Mw!J)nR zeD%x$Oi4i+EAru?-}xFTwoJ*Hszith(NQV#WY?NNZe!Wmw2s}}zn!WV^7f>z!-T`P zAqJY;Wnb>u4b33s5j@e(t2rJ0CJ^vu*-u5nS|kebf(fx9H#F!+<@Z#Q)11tM^Sqp)uU=@Y69rbwT9Jf4H6oe`PyMnESFRhVz1Z#Da6D)DP5?X5JQn{#_E(Y$Dn_5 z$s-CNKd-k%9BFlwI_)T~UG((I5e;NT39@-?KRY3d7F4IfQmX+L3=s0bYIHSQw9|#2 zzq<42W!3xay76T+LJ25S^JXh4iRCx?^>GcFgJzHr8m)f^C zJ`k?2rj}{%7u0u*AiFH0rHR&p$|t$KXKNdu;?HQ7$uEH!7R4BuD_l`yDp!3q!Xx!Y z$5!#|*uoY3DDRxqTJTPg`;9JDpcq+Aob%{7UV^4CmIdGq%5%adH%~8RT&ONHNP@mh zi^+NVtX-`Y&J*wBE&e9NY98zD5|5{ZzIFymM`8hT%gXb=?46U?TdkKRTu0q?#G3IN zi8e&L1g%N}Qje$;4`=EYk;W1Lp%;2V$s-uU6wen~mZ>JoEk9E77GpP_M8?)j@%Rfj zHoE0H=hd9Dh|8%3wJFOUNP(D=0s2KT(#%J!zR+$tW^&4%w37U_>FbX_32#Jn_z21? zk-fhga7au5Z7C)MG#p0VRa1i{Q;KnE@OXtn5uHiTFQ|&ez!h`O&c1o?@!Ud>a!4_q z8M6?om3NpxG@w{7N`bQ!H;ik;27+yfe*hEh;;Orr zDm+>}9ZcTSP>_d*$$?&@nyZ3EuxMKalfglDt{9}Kl@~z^^uIiqBQU0OloO;aCx+u6 zuhOa0Y_ob{%smY99qjRS6`iln z?gQ+b?_yu+Nc4jBw0YeI5ypl*ddAEP>OlqL@Z{@5myX}>$==RaO|&a|By6pvl%N&zJXqzR0sPn&v_ zIpGQmV)jk!zpXRsK9q_&E{`x}t#VBaH;m(^9-uYz@)5cDQN$4pybD1h%kQ8n(Cg)J zRjdjAdAVaTf=C?23Pq|iZTzL@#O~=qTC-1->KqK-j zB@%XBQ0Z1xZRpPgbY7U2gGdDg`pE?!$56sb7?`Axmp-2hOpQS9f}bvjl@z3=Wf}Oe z{p#}}uBG60n|inTa@uc46Ss9m`pAIu>v{2q2L>f1#@==kkLh`W9N(TV@{LyZj`7&` zlE{0Y;yo?!tk`4Y*$_Sn+F0pl0RVC| zwG;KQnFO?Bdvw*!#E-^Rj^Q*}^>fZQSfQ8Ts2- z7x>Ir0ff$GCQ|Lgq4r18D-quQWZG*~hC#>e{Jb{lysztLJ*n2g*HeL%FbH;)Q<-4| zJTMhud1Ds2fGC`>9|6i8O)gZI8a=_vAT2f zJI|oq(I>llC>NK^_?FF?Byk|8<~dyq9DB}sp+UF#!u^%USqI?W_KcAePScWNcCjY zJ?`ghp3VJa7+6!;lu$M_TuajN*tN039rgsc#w&1{V&F5lW2t9_u}vDRn*+l_Yc2B4 zrn+RsNSuyGzqq>r8xx8aVAw4#aWw~V10o(2=NwIylc%>J(b_em;;H9$4t^^1_3n+? z;cssj8m;>WG+R2k=tQmp;)~VkU4wa?>6S`2q?4aaaliMDwQaLvSZwHj(g2gp@z>3i z6|$-~mb8_kP}l>5b!GC@W}4(Dy*=%0G<)Pc;OSb)wKlL$F^FO@^Niog$WOxTswQ52d06rY+sR>xsj#((`K(e>tkRn1WUYrO5%M6bot zE!Bs=h_^1KeXxHunKHO*;rN#%^~qn_HMUkq;JrW0-kywEj{S zRi3S|TR|(j(L5iBdVQD%#e3HLV5FR7R0fgwuHq|e|Kea@C~13Phx~6s`+p(Y{BJ`0 ze+R7o{}G{GS;g=Npwqtc6XE%LF0#t9vRdI6WY7Nu`1FS*6}H*y!k*KVKq)Nhyvtd2 z`YEru(056(Gs~hYmpZkrKqeq-Xx9&=A)-ac@hLaEoBzS2bQmD!;T<}{r7`=vTiP!z ztlT9kIY!Z%TGw2TwXYv%$z~9;-o@*F<`BD*-S%Z5d)sA=B)iAuYm8bMJ2@q;s>yU4on5|L4V&{WE0c zQHk7-K@K4~3C7F5#ozyVEr$OLX|bf|^(9$K2W`oFed%BJ4`Jp5uYKyjyXT*``fq=q z|8IAXIlF2KL7!mlxX`3o!#N;BW(bwrEaSat;qIX5Qpl;Mg$B}^HcIdo5(z4ZraO(?-up?Vf{f1e~T9_Xr+?=xF(KK?$vaO7$gqqF=^mgN5y*#=tt zcOn}99kTr|$nw8YXa6`5uk3v|Ep;LD&BXNw(Cp+r_xCa9Kj!iO@>%_#!vAYmVDZYf zKY*GW*Z%-6eYkeU6I5gSf3bkLAmbdj=`#Wm4wK=hy-l2}3PQ8N->`b>)>Ab+^%u4% z<-yhF9MbtCDu`l!5q{--WIx&Y{;~b|!@<>ag9QIgmTQ4;!+v~w?IU?Z3*DEKvy>U) za5YK_fS`O&ORZDs9RQf|iryjIYHKd|0Ameq=-4CqM6q?Y7`PYFucVCg;?+BY*+wLU zTzJPP6}%L{!(cs^^TQtr{g$5%2EA8F+am0@(ROcYO6^Je?QD6c^!f$GS zKkz2zOaT~;(4#aeXq?w>b^Z|k^P!Kd!ClFG@St*> z4x*B3I{$S`4W*3aTm7=Co$iIH#o%?XnV1O5u{QPlU3=)EM^BijpO;a`H>t3)-f6xv z^fS-TH$zg1|MX+WMxOn?j8>2`bkuD{C#NWSJX5hU3eU8C1l-7vQPMHZF9S>$(>nQb z*UT)$1oY$YCSIX`tE3*jLp#`UC1N z3@;k}-weuzuN9Fgd8P^LG#~}++wly1Zfb@JgG*w;_36dS+6@}h+k`OC@;3! z7!=4#1aiJ{>`JmQPgdt=2xBJCp+vCDbe@SlK(ry@2uK1Qe=TbRkdlQAmJX-ST_ zOaeK1G{Q%eEjq|(q#-5Zo_E?J@rAsh09Yabc&nnz-0mUS+Sh?0>L5B@%{QMQ$~Mv_ zRz_nx&LY1+2u6uWC^BTS!&|U08-bG;i87>R(bM}p(4QP`XOX!;KTH@>k=-tkVwmx* z7{fb7?bsk`{S>1CR>n##ICGi337bt3($ly=rq_hT`%4}du%^d1Q3EzTgvA!&IoMY- z+#8L6jp6uI;}%UO72edmM8Qwb?XvTZH`;fv=r14nH5MEW{Im`$e_)JjQl?>@hxAZy zmW!9WiagoMre!NHmz2Du*fmm{!#wP$h1rc0F!MGC8Lv9@QMBYa zd0(#o0^Hq$tXcJE%AA}nzG0PcU1fBAZnZBJ9S6reGPbA8i$jwsi&-pR!8P7TL=}Yr zo1{^7<~#3$T(Y^wu(_{gKGrgd2265GbvgzV%GtO90L(U8-BEbaLSLmZhmsit9u;&e z>Tzq1`ggq-Jj&{O@^8uGTR?haa=4r@%@VAzpy5~9ptC6R-*Ys&v=SHp&K z`}rkk@p0%3=C!MvsI5(l>)m;S&Bl9^yWXKsQuiT`fe7VpOA#1gP=KKgF<^wIsl8M5 zVUwAr+52&u*{@2Hv&2|rk3L641OB21(yEsn^+Kj?XU%*aQUT$ajjDh2EUo<^iJ-?TdSepdRypn{ceuhtZ z83kHX(^-HPQkR*o))s|e7(BbH-$Z*Zk)_8%65{M%IZs^cnOo~vqnbZEA{0lAUgM`X zqjVH{cdAD$tFF8!NY#6>wQ+nj;6CbJadk&Zg*cSd`a}_QIa&AD2y*F4@oY#A{m#Y| z)Q(wJI>eO|R2WaI24H;}%bp&t$^k1*>Im{91(V6tP1np3f@XrKlESogL`)0UD@eG$xqsO+BK_X!(=Fe;*^}KU5>MSHw3_8w0Tde%_2ua zCJbnWCJovysazoh=Y5~GTyH#y&fM;#xgC(M<~h;#(k*H~Q!o=*&>1vYRZR2|amkKZ zC2uQ(Qm6!p1Y05`=P)+#T{ip1jPsve-xgO{NjZFJobBFm3>u9ReTnW7OY~|29;RKe^=TIKc0zd+}n&CCu-40R>h?}o1*}&Aqo6!2;Gq=my*RGU@LgjgbDKPTBgI) zwYcNEtu)Iq#z!zc8%i7P=R^1@w=^Nn%h z@GzH`;1; zB*pTQ51NY{mTHcasEm_WQd2fo(oEsHW>WYvav?_c!INe=%#kw{`R~W&0dTvgn0+WQyL&cRzXvV5Zw?{yFoMqWvFAhXTND0vhSt(e&XYg zspq;c=qxEQqwc8{)&>DdX>~CWag{sZ@yc!&TsEQ*Pu z2h~JK70=$5=YrU(`sGlg1ig?mir1zbK}c8zGl|D?W-ygK8ac)zs}w$c$5zi_=`v^8 zoYiQ7dj>Ag4jrnbd5Jyme)c~$v3 z#(hFe#Ldb;lomKGko_k(xyMh z@_?BoN=L*sp0WY}HXbnUXQrgT4-#(vZv(D{CC~4);MT9$H@~XLEo&^S{l9BYyxSzd zd0x!`^q&5Y>i+)$A2HLvaL)hQ+{D6S@R!%g-q(i_56f3Fz2__f{{U?7Mf|?<_SOM> z=fC|f8+rD47Vp*11x@Wzw^KIe4La8on%s*`)y#;wJN(s44e`etSJRpWLv@5}>$GBI)?F{`$AG(|?*^{l8#r)nfV5p`rE%wx!9+ znnzD;+>2r|v!$5$wPsowX}&Gn#0ovn2|*x;5T4#fy|r24cW04}e*o>R3f~s>nj~ck z#59taJxvKUdict^cb^jqUiDEuBw2Ged;dMzME_^LYcPto0cX|t* z!@2tXueM7SE$2_%Yrb0s2j~ZqKH>*nRo%J_c=AR0FBo2zm+e7YpscFv4e-CD^UN%I z^_;%m;$1D8mUJO5W^(ZA1& zQ-62kvRVGk;@XuzO{jI%$RfNr)_GvbA(=HES}1^_DL0M?z>8JpR+Vv@Jp&_dtQ%+p zH090vCuy@&AN8BZihQH*jEFihuwSjn^vp>+D~0BC09*%3t+WLL7@qWqVf|I2#N@{aD@ZPuVhvPv)(g(sU1PueSdWNKvrDKh^ z+I2O(M&%Xk*US6)uhAFLQVR6@b@ug(0RV^P7nqHKv62z!)|a<82Nh3N9p>*=B|Nok<%y+X4W`LV+&1yVSQNC9iWdpb$)QQh(g=73b*a2aNfJ~vCl;Z zK&+Ub7X|uDpg}70Be;0zO(~uchRX zhac*~`PmgL$3QDsh-UlVT0f!Mlq7W5w71pSK*n!c;|lvD=R2w|sscuM)y|T>E&p^q zotey=R>SGynUXHe~=w&n6p!=1yG;FN=uPF2kjKyQbB#5!n zsl%gEcOmkB0HXT%cP~UThZs^QHr6EqWSnYaB`^>K1!hesL;!D=+M_1T_^2&};VND5 zl|{>Y8s&6aL)hqqJFW$zV*9gp3ibn1Gc+@(LNgEnK!Kz-W{^NB%F-m&vLiND`2D6| zyl2AkM3l);--QQ$<-Ku*!zupu`%bkP2XS%TjJQ7lqQ2mdcAnoh$`ui&?)n|W?%g~ypj+Y!AgcJE)NC2LWsfMeaWMWR$D(WmN zNiwDOHhtQ~<3Mu+z^GTJL~4VKKoPQlxRm}%_M9y#^p(3rh)t^5C0A^}$F8~(2W<`%ZcZ*S28k{OF4sMf4BP99b5rd0kMQjj$nx~1Co}#Q>ni8@|xZLMp!Oue))j?R=SSWy&%$XSgxP)(NTM^#o z$^H7M?^JeGw-Y+s_D(9YQdqW+c}lDshQOM~EMrn5zDbMZuWkd$aM&fG*KH;$&z{q4 z{NQip?BZg~rn_ILx~G1B^VOBOzD=4wgI5mA=Z7^9eR+eLD*_yD=Wkc(+QlA+MWygH zr|f#ARfV(}*b$=^{IJV#$t!Hk1Fu%;};|%$rGk0=0-=>&u={t$Dd_ z^}e)5$i6&$Xx@{Jeif?{li{ZL>?HXt91p@B1Xg)>t-vwk9R}R|xgdV^(sW${tvXXt zA-Iqa`wudY?Y-6kqx%kd=CygQ`egB_(ZIzPE;>fN6VvsG%S%&y9cm+9{+&PKQ~e z!%9tJlWBSy_9ruTD?ZhEy^0_Iy6nH<4}gz{9g}8kbBi>R{4u6V?_0$rUaqtveNo|{f& zzx41OylUU8)!?#tl#|_~bwIY4Y7w#u1{{j1^9Pt+lx!5H*bKW?-t z4X`vlcA2y|bOJI141&St`~ci`Q^Z3HQfOgA+)?ZfiHyEpC@Z`?}BFNhT&_ zhzYH_$G5GRGp{;mt1))`#ydPh{@M-;>PPsnwOh92S_osD0l_mSx~Eb#-jb zH6T?23-80>*km(RE3hP1Z|9XD#R;yb!mEiq?;4WbH})EpR{A=!^s&Vou1qxKWleN-_I{f*VOXlX zrb&PVFc#WS({ieoy#DO!?S_L)&j$7*efDEsUTO#P)%c;pWFmJyQT16N_I$bm&*L-l zwh7beMQVSbsWq&2Q@}KQS|?g1G@*@xHt6Xb(<(sqNgh8>KU(sSrxPWS%YL;tONCFiJ~9Vkw!#0HCmY7UH_h z8hc!)Y+DKHcYf6exzgEMuD!MdH(V`{!EBXk3QahA^S2Nw6j;$8Kbr-`?>hmQzv)zc7e1ow4O?w!B*sf-d8L*aaz#O0BC|pq!lTLJ0gNIyHlzWeD~Ymk zYwhThd9lW4g;-v5or;+HjL&q=ub%( z-8a?syioDeRgcErN&^6%_5#flk(%CDKH$+X!sqB1Ke2*rcK!hngjTg2M`d%ao}Pp+ zZ+AIwkdOjgslpj4Y~NZH1I%9=D{`6%s0oM^rqc9n2G%ATjT^S;@nJ76TOW+$t8;$8Zn8fXU?OyP`S2Djz`ABCejU^H&BNO0$M{(7$pATFH zO=VKTqJ-(ql}tSJeZcShM5Vr|J5DxOb}C3ofzZ#&{97}FyHz%ol8tSzYs?uBA45F& zvHqKY1^3r3NA5mlS2LTFYjEjCz$o)-7oNgP-3UbZx;iz7F${bA@FZpQ7Vf%9f*+3K zyrwnD&pO-2VfD9sqHG9v=d9diaU?&pfkxeJz`6O(RZ!8e;E0Ok%3~Qz*RlM!VN!H( z#L@(&QM*A$X!FT{*hW}YzzhY5=tJu!6VE35038NOZmT+tnfPMx*#AZSX|UX7`5~~g`GZ~I11t|mV6|a4R1Lbl1PeQ7G!)Y9&tO$RjjeOgjb~DYEomgp)aHe z<;@Bko~*ol^Hto%nf1hmSUAW*)Ub!MRjm~=Y3QOyf=&*jaH4km5{)@wenI= z+2g2=wA2m?_D>EKWPg;IljjhMJOHDz4*<#pjZGrjc+7DBkx;ZRIJ%L_EU3ni)MWac zbx*GpaShyO|jxk8@nu| zCMH}*?^>M!GcO_cHsR}bW@fAj_FH)_YO}_;F8DYnS<`ZC+uS>+0IuuMWj2 zp^RvuKtJMpg_r;<3Prc@1GbP$CyYnlFxWO|g?r@%{3 z!`13_<)Qt43wlwHLap5`Q`{Y@vB?hxkS`Tl>Cb8n_i;YvRtoyUyLqTJ7jPz1yo1)! z8Z&GMgp%bCr4pThBmnnga|J8Y3lCzl-Q4D2zf1wj6bxN|Pgc!Pe0I;Z%jxe^9eP34 z&i8gY>;QU{Oyu!rN+Dvg8ST+;x1J5mh0rmXGAoA%``k;wtSvH^OyE~FJy-YKl4Izg=%uJa!qD|!oGN5 zlAxK!)9O+sL|~S|szeW@>LV^W8`xgdRpqRF3Hbv!Yk&F&(1`deEf{$D&d$z=KGkXC zg)i7iQoj;&$Y4V_=kI{DV<8^NqI7APE%dgQC=-)-o&{_2Ryltv5bsQS^|yc~Uwhxx zwfgq|VjI4EAqWsarb&2gxb1X_FH>kJ*G+jvA)XdF1aJt8&3#{vo8;y-3)EX{RQNTj zgdVnivyGvWXp2iz_6Qo<0EbSac=u)57z!k`EVoeqr}+zGsBJ@S9eP zXbDM5&?bqHD%dNO9;y5uj_P)ufwcxi%8d^yI2#6BD_1eI?ONWkd%51=E0V#%g5h<5 z$4nkO+#nN{jo~g+(VvvfPVW#ev;9Z@U4 z{{v9*uj>0rQjwpo_v~Y}RQ@w?B%8g(0$x;+!_0s7wzpuX$%5^m zf^b9W($IX9(i3Xv_H|c$u2MoL*&!<6^tP*97J9KlQwhmajNmUe?<4h9@h)Et=J^f2 zm@yf&b!BoSU@NH#JT|vwNTwVuop4qKrq|`aaJo9d+CF@o{f2RrLVb8*sGbAr;*BIB zO8#!Q>kryl>eoIt(o7p7loD@Kk~Ci_lVP!BowPikflLg`=b(knE4;3wci2d+KtO8c?QnsOP&*gyePt~=8uG!B?5<@a5anR^IZ{k$onQ$@0u(JWVLfjPmPy0?=XAfS9A|g2O$jUU^7k6Y!<)*ZFrw z7R{ge&muUR+2Zw+7|oyMHg52jAGly2IedTYk$1kY5_)AVCDu>kMv qc}usJvBQm zUCaK&i+Ri3LEJSVs9GhZsWV!I9-y33$^G7D+pzihnt{xQL7(1AvmI>gTbA;WK|!+s z%wMg2Z!8BwWFnOjMX1i ztcZ;qqXH-)5d#bww!;HiG_aqtyAxNQ2}sI|9e6vRN&+kRRPC3u`*rC3VxW4t?m$>l zv&=CpsUmhNIzzwF%f?XhW~5+dcvpyPJJnQjah7l7yg{A`iap4MfMi9P!)V%ixT!u7 zI&j`jjLRk8G_SWw{Wh?Y!?C}wTg|C|-78NSSx}D4HkA?2j@pq%?c5-lDPE~7YI|9) zby{tsSU%iBr->S9elR>7R=qSWY#~fn+rb%ofnXwTNLNiL=@Qej^ijz!KfNg9NiNG_ z??=K)9JG0&5LXE~mF5`ECIa$0OQtveFPEYSMYMEaak``QDOB|OXTS8rv8l5C66 z4p|9Q?6y-i^qsq>X!ssOzZ$<}IoHLuD)=^lunrNPEIER%&<=D6yY6A^q-bsRgjN=> z-{-|2HeHApq{m%|7c8`zUZ^QUmbQO*_3r2oicd7hKS^sBwiy{nXp2s4635Bv)_t^; z&R$9BA*aDpFdZ?j$=7-<*(x+RynJOuPK0N9`Q5sEql5!}8Z*AY7!cN{*{v==e93oi zV_`pRj5SGzUcEXOFU3N8eHQd`QXoUd#R+;B;PWl2(a}G@?PFi&R%*OFNBYyX6MsPi zra1tHIWEZKkPV45!$1AFX%=?G5`W*JTU5a!Cenp+USZM3SRg1~QA2jB{1YiJjI>4C z^lEK1c*o)H%EP^6u%z7_*|O<07bBbLyr$jX1gY0+Tbls;QtR~TjQHO;A^W~E-szx) zAMu(SJ+3;?Xntfq{9_s8DY#KmbgOUGz_vZ%-jddH;}C#YbyN&XejIri4a$Ux>GEq} z?bUjJ8hV4F3<;yn_wdwdc=J@C1jqQ4rYT$}ooEVXQ9>C+8JWvp>r>4I0Tyf|k`u(& z!aGL}(mS0CO5F_s`2W53iZPC3ken9PE`T))Q zO3T~Z_Y!?ng}C|(XuEE8sMOip?faMUsC^F>*eGioKptM|*;5(hdseo1WG(`^;05F3-v^xuom9t%QMc1TAX7Ur~C1PP7@fb<&N)u-B zc!qcwbx5at zigILq|4n%tzq%q&Di++WU1v~N2AD+Bvxhm~_S;JxAX(T8p3bd!_Kdd^BBVV6&t(N^ zI4m|P-ro9JV(7{gkBuJ#kU2x?Ru+$?TX@6fsH9Hha~nO%xhg>wWV3y`tdfH=?qV70 zsg^&>2`Uu7TL)KeCFFrN?#>%~e7LINZlr^ELpthq%Z0q6k1W!Cf$UpBr~u?A#qlIz z-rRNc2j-xWB3Tq-n9y$)4gy3rU$T3DRD1h>7~i{ayq@54rymtv9pj0QHuYYszC}ak zb5Lqr08rr6Mlf7dNVLNK%lKZYNsv!NoSZHGezW2Ss?*eo+vDS`ve(}W35pJE z8+_sl?rlnmWw&?6NPhMC9MuIw76%TXLMSOVnl41+LPVjfGw7``?C;|z!3q(*i~Sv( zG8WOU*^O}*0vo$Vu7oMeqCRMwL`Hm5;dM2JZoZT&Db-?mF=uORZV1_e46)3kyzp7G z|AW2vj*4R2_JzC2IfF!jCN)hEkc=pyNlk;KCg&U*Xp#yjIcJ&-3P{d5BS?}gIW>r6 zBq^vU#;^Cj_dDBj-ahA^``#OOj5o&S4+b@=sH!z<)mk;@`UU^PV%Zo-BaBd*DCD76 z;yq0E1k37P3QuWrZH3Z!Ncg*?Sj7f8;IfWUl)`q9-ERv9#?>s)KI-PyX3ux|$$l4M z6Pa9^XXJGjZvK*O8BLqzel!ws)*Jj%!x|U|p!jDNe35l!Vud~auW+uSaJyhshlndjA%bN)ZbK9^{1b&P3pJwtH{;r^OwR9} ztvKQG^4`SCTanK@AF64o7{C-bKh8+3t8K?KNA5A6 z2i;N08mh7F_6~EdX`m;P;0mubT^YCsKp6{XGh!k{YujjiFLyOrC1$c;YP4Ec?Gjq~ zT#^UBt+<+(bq^*CO>-&0+q)ARQfbv^5@}&>qT&nYj<_O zZOHnawj;-S6H?PclKju;l?9l)PW5$F)vyxqV!Tv~qBKNkXctjPOb`N(m&M{TK-j(W z%)VrVm)|%KDI`5tNF8Bi-Jx^*Ry0dzTMOXJsjkl`855IR>%!EM_UOy00I{Ov8rc3y z)3OSc8Q-@3)+o&|zjQ~`xo&C*>Z7WKcAOlc;dm&zS=D$Aj6^6=>R7Z0AgkO$0NH^X> zqh9=gd97N#D<~oNY#|EikZ~`mhmSxy-8+MK$3r@4`$(1u>vnis&^=8HV`qloPJuwx zBcj~($_!?8@%jt#PH>8w#?ZLipmMN3qk>CLOD(E9aO!58Z`oZNjf=a{&8P4u1*d@$ z3Fh`@ZjC8Tw*5r8P>vBP(Kf2yTqG^gn^Wq#%nFC4Co`(?frfObc%Q zn!py5e(A@suVdoBok&<@ykEwqyFBl%u78~PPW0`e1L93O=*OE?qf-H+X;)7hOI;@B zk#-dQGb8-?&TUh8qd*vtp5ROuQW-jq!$Gj-wE$34TyAj-Mi_CwJ!-_7#tnWA{4PVX zBOCt@!~@7;Rn^Xqme23M-RbFD@EFHg{EZn_0LK!vqgGFuKFP`Wnc{!m;srj{t(5+8fCP@-3~;mvTk;8n*~Sn7LLz}V{|f@W#Xs@A{Vvfk zZ;ikR#cr~SkA&_zH0%a7Xc$1t%cnj9W(5pW3wMPu-qxwz1C$DA1XwXxgfh@(ZEO;I zjO6GQQu6H5IEGYs_@~EI##sy&9x@-vk^{R-@3r_Zel@TXMWts>>q9DPQu}vxqR3&k z*>ZTl{m9`F^j6^iL$}1zHq!z=8V^3&`ri8^KN9$lIC#@mg!!YTC)XcEf6Drj*r{OJ zf>ql8=Ry3>Jrx*Afs4d32KnBU(;lJ3`PeQ{F1&&3B;h+B4{{4sl?_o+VlCAn@mkl* z@5ikGCa^v2J!XeI+&?#ZWuBa>!%z#U`ej?cCaCgj-muku)NK{)9aL5SvaQ{CQGVu0 zAG|kuMUMsTSJnNpE&F2pNoNn@Qx?~*0e&^Cs_U2SUrztm<^KQR{(UeP5!g@v67$m^ z95;UgT>IyLiMmM#a`u_Wr>xVdlQnDqExQ99CzBi-V1c4x!vl4xK8pu zn7(zi zSu!SO_)R^>*D?Dcav+jIbGw+E3MYGrc{$_OiS3OhkH!G2=G5e;J080yqk2=)S~3jL zIx=hP%MV`+4&jqj#nCHsL&t;l*^L8{XneZ9(w_hr=irCq84Jm~SHY@>+HND{kK}#= zTzo9Lj(-B)%1O*G&$YH8t%l}r)z?dRBosM6d+=qv&5pUvLvj^ibnlhYz2kOH5^q{a zjsg4y2Ozr;6%zu8^pyuw7&kn1oZH=bxK}Cg<07>)p6$N4^Vaat2OaxxCr=tAu@wG2 zj!KeMQT(B>wqg$oENVqDZFIh*Nx0m&nsT>!U2eBg5_@uXXpue&!)uXy#8UJ89m z4&j@))%~6ugXq|YsVEodmsyRw_R$TboQuocv&L`X*zashk zJV<-UnCIc5<6iy;ch=8jnS&4@(0Hz0h5;)lIH)ly=Yo%K)~E*}>+%&NQ%_V@>Bg!e#;l@SV&BKbuF{L!~nAMu-Xe4}r=Ay8*&8$jOF@tB`ec<(6W1FIxg zbPP^Nz0@7x!2T@V<&hE*uMHS*iM6~3Imr2GkMOY5XV6+MruEupOG(tGI`CyEXt92H zNyX%ss;E!3t@>h%UKtCxl1fB|<9Le|YS`3b9uDz^_m8tIC)nF;UTofo( zxNzaEnmt`i!Na?!!VegKHyDM$Q$C~zQ&_e=ag-56xc5YLqUb(@FYUs90a94JT0O7kw;J9j})yp zu=r?PGL(eqs4@{9Ej0&v_!R*an`|ALkl0*CKv?T#BVywdEw0Fyl~yZLzVx2WA>PyZ z2t_qMm{7G!dM0fLEpM?CH4PPUOXV4ElE-Q;Y60H58lBts!1?2iIaB;xLJC=p>NjKh za?a3v;72Wn^8-v-E}9;RFRGBI*1@kJkS!x;xzt=3mDi2>K2oN{DE0D22cM6Cv<<`7 z%@m^pkBoJ+2^NACbMhJ|Vx4OkuzoEZZsOkT8$V=!_8uG-sZ&_HBV36GlMP{vM)8RF z`=k3gI9E70^}gUH89N4|v?upyq{DfY#?}eC?h0-ea?d=tlCHC%i7!8$hilt~+3$GB zz&M9E2%RsD_AU3lPHSI~$BUJ|;IpZ|zw`O?o26xe9DI`QG3Z=|YeZZexH#feQfHAl<$#4pTwi|Gl!=>mQC zoftl4qky)jL@jh4IfBW8hll*Qn8oO|3A}pJhi2p=QalIM z4RO3#AHs?3NCry%PQgUNWMQoyL3f-nnOaM8`w9R6+&CaQ^ttO@M(c9 z{CF^{DT@Be#ot6H{?#`&RtDqmbus?5JoR6FNaG{M)SA;7c=;{`J8U0)XjZ@T6`PkE z@!X!)x;Oe{rFCciC*TdV2wP#RHT`c}`;q)zxeUu|`bRB`-_BT4@^*4i;Gv~gX3NJs zrfVLs_5_7AhU4-&`c2lU*(3*|?=(Mrp3(t`SMVujsq5crUxZ$LOb%lBH08!{?F(%D zC*W1tlS5GiS3T;jZ({-40et%K{YZPSnbMqY{OUXL4Y_f`gC+TQW}6b_zx=O{slfPo zLgNlIY=YyBf#rniAg|#be*&qzf4V_}TmrCx&N$3wSqDrMLUf4R+-Wgaoh)fzpZV4{ zYxKjLS2}lYH@*#fbyX`HfTny|6ZpFQnVV&dMP_vtf3pW_7}E99FhL6>4}T#y6k}Jl zPJ&xer9`zaio3)Qv!Mb2E2f_3H5Sy0(zD3iXdo)cPg;nR~XzzGThR4#9w{zP26nks@7z>;e=wA~k{Vjx*7N z!aMZh0_~JiN8;Thw^G0Ud23AxWa=63jL}HvuuOBb-}0uS6}c4#XnLVj#j2+v|4lhR zLeqPqeUX3k_UybEE+?nVLv!PgZmzb%O0S%!6%6kQMH1FpK=!yZ6?9FMop~J_h!GG+ zN~6Wo;x3EFdsB&!ckR#AS3kT3vp+HZ^xQLOso;+`*FI0s3&|b;0GMNv8z53F>5VNz znq!9vksh}bl8kF#$$JQH7_d~?;GpBc`myRZv~&PG1t|2kX%?!x_I&@lx>5Jw`W4sB z^7{(kzPxxw8o1x`OCxQ0(#{9oId;yBGOVTSdQXvqGz{%fDN1O{ZqTxqdcgn7=BE z*t)!chX46R9W!mgbNqw8={l*1h8`|a;~sAXGJJaJ2WKk;qA=chEhw(#+ILmh@KmF= zHnT?crE6@c6uBX!k`-pdA3~-}#YTpMi@^?^CzUgG-)^lkJM>-Sw)Xw=@J$)aeJg83 z`y07Mt=xfG1|wLCPnb1Txm6*8mGcl+`oWS6T^+hfUJV>bkutJmVHX!#7vTIhSrXR! zil_5iJno;j0qG0^;D9&)Y{guLGsyqEiP8Cgaub2KvYcKHOlq#^3wk0rVP0P2uTos! zcpC$tRFvZxVqgzE0G$l3Rf6(Pt)%R9+$EeGeMT;dfo&tM zwV}8Se3kGWxD>%zD2Vas9eT6nE0JxwniLmQewJj503sJ%ta0?Kbgy-8EP9cgl3dkSFSwTb;WX-8VT8-7vxPr|OH!Wq1Kwdc^7{}1*?#^ej>Q#-i(s8dF z;dtt5jhr`XEH{&tJ2##_yCYH`5bJ(-_n)D4yfWS>E7OiiT&IrdHMo5WkxCmGrS(0 zYIX}rL9<=x%};?M*4BcGOZJ9W;A zw#lm!Zkruj$G74GjI*Y~TD5_r8{CAqM^`Ri5}kK@PWaQ7Mys-Mfy0sh@&&QpWUP0m zDW{fqC^K`}Fch8(E&&PmZz%Kqd3*1}t(JJY#Ro7QH;fs_?&|`qpYNuDI|C{XS-F#} z#d4FKs&Fw@&qo=k7}ceE0E|2Xn^bSO{fHF-RCZUJSBZFp;a6WMhqWk9qyKZTKmDb{ z{?g6{GY3}3%U~AW$0}{t@=#k7P&>UWq!UbdGk^_LLimdBu1z}{SMQX~cg79E@V&84 zLxWM%|9u!bHCiN14JRpM*;<_C%RYBfjvTRcU`*v^ps2c86E_02d7fY%+7-nezFp-c;`n7)lb0bZw2X0Ihot5 z&>uthV}AnXDhIK;ZP*fTXFKU@GRj97BmC9iEjg0i{QHkAhkW>Mcr@QTwJk-Bac8qR z=!;I`!6P{Rt`DSOQ7{03lB%lR`R{Dl<@cCtn_sK+LV{>J1s|SDS3hsu48I!|z>~aD zPJd5Q?w#vmDG>3`cDh-ipY9_k4{VLWR@a1jY0l<5_t_Z8KS{_%lRsAC4#~%;>F`@h zpIqJjVECwz{mWuTkj75{GUUg%e{waqt3$#ctqxW+k1*wb9sj>Rjlbgad;dvYl{TZW zlfmC>9N}#iQkzcQKkkeObsa|NtrKizAEp%Bp7E@s4D+s^RB|3kJXUR=|47r?7s?298o57$kzMYuL$R5WbH5OKl(go zbZcv_y9hLOTv>>m_n$dA0RsXd-d37eMuF1por8L-W6f8P_LrAh!zu0rmy2~<7^m`% z*K9fSDtcP}F;W@sO=q<9e!L&%;70%=7F{raZh%9#iQjBoMx)H`TG~<4!?;EieIwHL z!-J?B`F5I#YuYIZsKuKTVz+#jN>ViwuqZH{Gd+YJ1oh-z9cQ0X><9nF@bps?O`Oy8 zp?EC;0;!`k>QnA)CdYDgmX?25YNm1QlLs8ZYF-{3`Q$|{oL&AIj~(njHP6h;zWa(f zLWzBQauLMJ*3IfG%*1fCW`kfeDE5#~*MXCXR!0dM(;W zSt!n0kZwubE0bA69&`Knju(12)=0?G9T%Gj1g3@K8OI7b*UP|#USjc--x?Q`K9|xb z7`{F&`IHglM*0jh9Uxd?B1R)(WCu>;6P}${P>IFRKSu~eEE>l@%(d0kkN^ZwbobiC111eScocmWb10Md5Jb+zM`;Vsb#3zFt%}Q!m6z?gNx#jZMZ~x4zm3Kd}L|Ufc(k8x%Par2T13p%6bkhjz$gcc^Q_}_56D(^RCjKKNPRX`8xAJ|E z>dEL*Tnz_Xx3iPLd-+JQGBH>IPK6J_9H8=?`|&yNv$~_hRx#HOHQ|BAVIN^fvAFlW zY*ZWHtgj2!Jsn--l9f%0N%7PbHCU&u;(#QN$Sy12iA%_U;G+{=-!a$al>i5LbZCsj zE9rz^vz3D{L&IALw`;2E91y2`vX|~)G7RT~w53**L$!NPyQg72y3Is&yl!ElN<)F5 z5JxBzag#0ynAZr9P9`t!prUwf%QRf zj!CqaZ!hF7eR@!0y|bY)gj00=qvMe5$#Wf?>H!^PRA7H8t_POr%6wev4mpqHUNwT0 zB;rkV%kbS{5gz!F6Kg-DV6Gy896jBPI`uV};P8vD@PDYGU_Wn3*6=Iu-9a>*VCC0P#vq2k8r6{)4F*g_;C+oRv7Pci}RCvDnmN(6gi3XV$b zu3`p@yl#r8mlHVVqYI zFL%z5CNRUr4XH^*NnPajPuL76K$PRvTG5(|I=yjp<>a^)!*%DSMygRa&K^g+q51mx zONO>UYLm#)%~tQk#bBt)nAy2dE@ay9wR?5hYdrGG0d4F1Y7=p$#4Em~f8My=TrUFv*t%$j@h z0bI*3Smgokv5U$|+#}Dr=v}sN=0qBKIkj(g>>pPTD9M0sG?3UaQR9zu$c91(unP77 z@Uo1xxlypyoZ(PGuaiVXLi>&GHLbnZc|+32IQzacwV@nT#&>%(yLDC@#fTWSj-qFn z`y<+4dz3jQwR1A6h|7+5ijn0};|(Jy_;eNd0l3~tc8nvRhoI0v)ru2ZH}akh&1(Pj z&p{Pl1(jpZ+~*ByBe~U=No@zmUPs~b=72pws@QZ{W2D(0v$HsgMmg?1NKj6`?nxPev;(h}i_owyLbnPD161$E}^ zeh>f`WAq(zRhpW*qAz>V^c8E^$QN;=vij9v&v}FZR!4C+Xvi)*#w%9CK5|Qc)*w9% zehMvHLGiB;u36KZeGIX_B2AA{Pin4-nV@*_I9MLvF0oB*P@d|0r=K)q5|%BZOURKU}k=+A1>Qhk(Ua|2u}l-dbR>M!w{_0Y2;3b7Rm z3NP@bW3!8SPso+ELV^S3#4?Qhkf|>b<&fw6I+tpo%Q?iEt&jq*kCEJ0!nICLd#T4bP57*>xSWZ*lQp#^U3|R+ z5qlK}qyqv({n*IMis~J18bm4C$(ii6p!8YIvaD>5akkoPK3HjPw3|&_o&Z!uF5KxK z=9Vi=MQKT}l-bAlQp>b4DRANuStUTOxnG!0Rje`ib>2xzm&-Hc%!gDnYsAIY5L--c z=D&HUuLCzJEBYV`lcmyVA>#pm`Ut5zz1G`w^q!}aD}kqR$tPy`UOKy~^;M3Oq{GS5 zS&B)`B+NMj&witNDt6dknWQ*%PQ;l-TpEy#M+RI;rPvr%5qtQ9|K1C2v$Cd1b8yb0 zYf&7IeUg2G{iJP7EL7c&4oavw%s*&x=oEGE*kuuV4GA zblbzyY<&O#;q$rhmh{Sz+B)|?Y6Im*6Ka7k%D3uC;S^8 zU1IabI?di}CA=pqKD=0>Dm*;o_aqQdRh@qKgWruG99ECcDq+|ah`&$de)9&P|4kXO z@!%8A@8aVBxrnmz{%h!eC?);|O8grz=)cw*x6*!{@8`8nW2C%LCwW0jQr5krW`Sya zBUnkTMU$`Rz8f3bpn&PtOo}u&$Qel|0DSwbtUM&$2sK zOP*N5YT({&ijL)vc{?xOzq&H?;4gSAn6NIQ;_%LIq_1Z4`klw0#)H_r>yM?d?v%Y3 zHEa5Ew~VI-@eHj9&E39Co$p+|4Jv=W%>1J{=-!XOO~_Q~&F101PViR@{>S=(l8~T; z&8if@LL^380Bq={mU>?>Fdn4*hr z@Ps*K!4CWrpc;UckHLBL!~6qbX?mLID|oV{8LL&>-qLW@`bL0ATFh+?9{LMzQMS%vv z!If&FB6+AF4P34oQ{(vFWBL+P_HoS>veB3UhE&qhghg@U`H9eBtJk;+uPh#l)a34- zBz(OtH`XD=y+>NN*SxhS7;8(KV^+IMoqo?d&Z651wBf5Kkd8Kg3kOISXND2r#o!+^ z1C8%GZrQQ5*zgFCJ$vT3yx@=E3&0xK?!INSQn2@G?r2l>n? zJbjy4?v*iYrzer6U`DjsZ{al1zDrt=kTs(6bQO{PLC1BIa&i*X2mRM?-Is-1@_?TW?A2MR5AsVJ3ucxTeJL+@;uctS+02wBCH?l{3>hdCb{QOz9bufh~0s{Knt3HfQ z=@`UC^Sy6DXb^RleT&4nAgv2;geA` zrR2rs_9BN~7oG$fIuKP{eM-=|6&{^7BzC}XL^VR5FsjeXyH#0N&OcN-*X4H4zCFL# zO1Y;OWLuWEIpn^?e|pTJvW^MKeUQqCx5m^}Spbv`d1nBGteqKzD8d0Ju-_+JAOdB5 zGALT|W2k29&KHLQtSX-CZ*%wXYD_BMr7#Qn$@iO{;r~(4t*X%Wzl)4$Byy*$JJ4-&%fSiJwn9ACwC`{mFJdoxKs zZmTp&q+9@{Sa&6F^Ta?|&48;IfivVIKCO&|irNd1SwEeaT?LCZokH(1jj1IsfsD)j zG1b&bGXRs5_A&%qhQk5im5BfpRfPB_w{`BT%4~c}YxrIaPfd|+7D#sDu_r9CP`;DJ zFfmBoRM*+TQAPS_gms?=M{NNb2GC~=H@zsw8XsS3wFw_zqC`IgdGr+IkeHMcmxE@G z2<6Oj->!VV7gB`KRjRO$$+=r)cTB|AZYr(0Kh|01aEd{mhi7E8FM#w;j7LG;G4HeC z_yQ55HeyPkoPNM-969m}a)XI4Rv(I4?`Zep&)fe5WGx^g_Is?C-}hZ+Nnq;we!OqC zkPj9xbKLu;b#wg7`}mk~hWLZ}nrcVUYR7qx3`9WUE?-hNimIGitTF>Si2@ME5rSi| zYUXT|u)TN^Ne^LdsWa2GzPJ1=*^SHTs!6QocU1_~CH>ENU#EK+g^{Gxrc;@#3%K&m zyw=5uU=mD})dU^+#Wi_Md9^BIDVrUe3C*#5)MDc@Yj2di$-JEpnd37k)bTtUZgrA0N9Hr1v(;N}% z4UZ2`t4U@@rnzxEKT@%1E~}bQZz9xsG>!l#{3`T- z<4$Ud$XKW76DjhDwbkblM$kM3lC(-RUO&L53o}57 zRTh%^*j5x7e#>IB#-&xnd0xc6^SsmJg}w|(H!(Y?G63-M@qK?M-Et^xnB5q9&!Cd= zp0yFHj#voyqOvz0oNxwWM6w|xcHkXXQ&GuMd_Wv6xi`T@@^NbW2)9&E<&cZ9f~^T6 zllIW@8$J={JsrNb1Bnk~7>iuf$IcshrXNS)D2IWxl>8@KZy9GJ^G&LirYPZHC#76M z9I(e}j6Sb5-84QW7p>LJzbCZYQ1% zrUk*Tpz9;2qRcvAKdv($fsaohSvnOg@oarEo%Ffn!{Al@#O#GDa9>-;KB9e_P~to2?X9JSMX6Qn#!t;ep|c6|Pqf$~YV9 zMrr6EiNTNz30ijJ3C|5Z(h+^}=i9d`B}BRMcSdgaKk=Q~7;nu@_diRwFG|zp8X%Gb z?NKd60dWYzxG-%YM_-IMcJI3QE3%g zS-3qqS%8e+hf*Vzk-}a#*TGXYJKBo z_PQA9&Mm+xHC6%>s@mYQH%Q8!F7!F!Ny=VLHL)8?YYs)`locfqnbp^r^sbOHkz4HNl;ldd!Ql_?s%d9Jl<}zi~q0-T;v}dob&O1%6JOf12@m4)en>44tha;oco|~OstfGR$GD=RVJpm7m2b3VHgcU2@ZW{2C0Is0_ zBTxQ!GU}=u>>I3wp4AkX-Y=)g#vh|N^Xny!+!pV#`l{DcK7wWWZ+t<{NX%Q{ZrA>J zr6a$+vxz7F-u0^b?w7$L_p&Xy;A*??CO`BY0$KuHE`&@Gh;}sPy^c@MDTCJX?kjx4 zy3SF3nyM=BIlSY!_hgS*zK$KVSc)+F+pfw#j{m!ZoSar>CH0nZX0_R>+|nHGzc7Ej zCqnI~o*P{pIP=8)TW_0^hID)og0P@Lz26C{t}hC$3NxG3 zr4n(qkj&GmC;wX<7Qe}wd{<(U5M#I{N`|9Pw-EgfaXKIeN*h`Z{}|VsQ-8au zzkfJPLB($c(~}xbhaYWcGj@(mc;k#c7E_4Ju4gK-zF9I)H>po=Nd&f~h*}s1ta-R) z`~l$99Lepl&vpm7NE^bN150zOLg1-;Q^UiL zyaBl7c=?6c5ZJ@-ama_^+V#G74Ydin7KD1#`q-qS>)?gAu4kzQTrkDtZUV1`4;H~K zbXCGZEk@W7n<7CCr1pDR_`NUl5!(5(BPc2T-{`_zF=7WoNS!4kK}qR^4$DpSD0Ri8 zssBXFy0ogqSC{D2_@;YRTpeF{(*gJj)c~ZqlN<@O)_Uc2)Hd;7)9i^S{8l&M&XQ0*TNlYUFLI>LmhMkI?2nna=Z7T_ zeH~+E1&pQqrYXPRvi0-=_H8oyZb)mIKA1{~Z!0w$;m{!Un_nh*o8)--LM-B8>?4`k zH~?0!A=)`+`O_cZrPf8#f|l)4`o>RB9|v?ZYK+|V9O~vbc=G1}#J13TTl;Zu+E~qV z{Pikt|AB{`|Kp4;GdEao&CI(?cZw}_d$dVqRNr)lLAu}vT<**7X`E&f#yh4xHdu6SL?cIr{42=(e3V1F1t>sa9Hp63C4v_R; zpuWFA&$MvP2dBs-2jkO)#}Ryg4ui|Z--*s;_K3d5Yk{u1`%Bb*Ii)!13rcZu423ZK zpFaubYf0CgUH;t%@qaBmPdjl&Vn;8l^OU>?TbQ~ms>2J4#t{mm(ptYIX4W&>sS1I& zIoZUo&(uEnQ()Qu+p%^sH#C=@=@RCCtX*tR#0^%v`z#q8lJu<6OfZ}AVLBkuQ9(hp zZ6(1Mb)m@(A<26(CKFWmv>Yp$=;iZ7`(?ZZZo9%MbLIbFXcX->nV_pGk?Db&Yjf`Q zueta1^&sSf__~PdDE?5W!MzUFGJ!#gTsj4HwmwEgpZQWIsworNo;Tp#l4KBo6)x`uW%;lfod+@3YuK3vb3n2KLHzzk1kvcF8gvlH(Rk}aM!)hOHyKOB&JHPgW{u3EuG0HE2WaA z4z4p&6zj9%l~PmHvup~jT#|5G`da|`c}#|Gz3x!^32*`*v`Vm^$=u^V`8YI)X>S22 zbso+2UpdYXNKr`m5x8`!od{hmUBvge8VU5cP2D>uWhBF2yzbJ74**_gjs$1pMaf(m zA?3%j40?wH@@S>$Ha6Qz`{y`!(}7;i;17EElQxhMXcaI`a*iYn_vpd%%C@hNE#K+B zud&0o^y;@>y!s}Yvj@wI|8^zC!sk-{urkG?7z0UK*pD*l5RpEkai)-ZHxvNKdjCvh zey4tYN&SWYI<8#ZcAv*+hX#56w8lv>k%V_ockH0SJ`THgpt2&Ha*II0o;QK<10fV$ zko~#b@ zCwJ$&kPGzTFq0%bwshX|%kedVa|@}&vT}z_Q-p;G;D+2G?o7ZTij}vGI`sq8*O8(w z+qu0ZHaxeLWVII)gW0$|FgHwijbEWwX;Nw@4^Pu%`E%F`!gOa8V0Wz_>}BN;2jwM~ zdYlOO-&HdeLzYYt7BcNrpDH5WAaNv3L zgjxS7x$pd8okepE*%J27J|PC3dX*%vqjs6_m~!cR?0lc(*m1upqNeh#)aJ-prhhq1 zZ-9%V`Y|H^lUZ*;i{k-zMH>PODfuB+dHz{WqZM{-;kLoZHIlp6LdF}FLiv#~&PryG%a`)8}i#8lwH z57sy=IY%uX-UI+*y!U0>g~yV#8T^I<+0=Z{=uFziFK{-gL0KqSO#>NltzbJidI=Zg)-Ao1xR}whDcmPpk1* zs$N;8f*)t31oo9%qnJlO6LuXXU{6ma!nB6+YS)xXt!`OMFas~2g2Cs8bGa;Y9B$e>sF7ovCW&o9DIkMY#zO8+6 z+&}aBXj_|Yr(VXu()W2bFZ;%4%$9RB6`nEHZ)`6cHr>H93qM^QmQCgzQU-- z4fsC0+=i~Hdv^jhZ+Hby0X;VitK|^H;giZ0f(S(?wQG)Ad{GAaRK{UC{LnAo97sZK zerPcniVo+|`)oT`NM?Uxf4PPdD3uJ=xGxA#~FWI2k|-{bZ?#RI<7K(1!<0Q%p8HBp$I78 z0C1g&j6gahhE5LZdsq1C_6^P`!rm5>yCk% z<#^yCVc1&htDocp9ri!P=KdSxKE`{=fh*5qkmfG);jLC|5hl;7cKkO0l>W{%&66-rh}b6G;Z_iiM@7bpTaOPR^iVMB>dHiFvN# z^N13hG8-TjM_4Xp748~+;dl@FW~0pwj5Zf*<1?%$)?&(3WdD7`m|Te=Q5vl^1tS7T zZWVuyQmildoV;W0%hOH&!@PKZ{kGQP3>K|nccehDzpmuv+p>YQmdrJ-2EoTV9Rr`c z%1@%2r?wAXBl$9CRL8oI;gy8ssVWEq=M*p>f&;Smu?4t5Ua0`U^FGnmlOE*R3yh~) z6{Fy>uk|PH3+4zVaz0;gOcUd;AzV};LdCEsVbx%Sw?yGO`>irEjxuDGe031pvGkK8 z3@1;(W3B9}1UW1jFpI35zSRJRqUw5drGl@(;mq4mNSX=iG=cBAtW)xN8lMSq3uQR0 z-&GDHCs{qt;(JhR?+d}<1(mW%zNMHIRc>o)J^Y%MP=KDlPp>H~$J5;B;eYa~@>Q&U zmdpI~MyvIr)64vljb`7;stJe}CoQM4KI6#seyrCvHTHMKg0S&$7HB#T(+OEbBjWM) zEU3{F4wKWh0 z4s$&y1+(j7#bz%S+#Z9j_|_}2VC-+w5N1q5TN7SnTbvj(8{4rwWY@v~;gxxjPq)G! z+bXI+2D~b=3|@Ql8#v&rm={vla`)Io_O!U%QEp=V?)P#{(eafo?FGN!k=;C#jh3P9 zyh=1LOT{oQAcR=EgH<0Kl3mEbOPO%n^hY)NJd7diK7LTLTe%xVPCxI~pjWc=5zh=O z@n^d=9QK0qkVurBrM3)#>0lo>GRW_NTvkWv>9TW!-a&{BD0K(w-|K8B1nZCE>xkrM z5D6QGc1Lo80bf*X$Ue^zvRizTyk&!uX?nXFJ6vmXn0cdQXQNq`NY>vTFM%#aLpeuL zgb~osA=mC!?{nDwc=PLoUHNv>=3?E;F!V5lcm;D>_lB?B&W?Vc2--IC$8;Z1-Th;3yAikM7x#WB30o&EJMT&AMepxH!xUsX zo)#HWi2{bfQC%T#G(LEYOv%XkE*~SktIMn$0#SB%k5(;0weL2+wOR0Na+HowiH=Pl zWx%K{QxlcQIqd0Znb16WPqrQd`JE*z^w3%_XLrcdiZ<3 zcK>&t-7!2EOx=d~X`NSJ!-#@g-rLU4CtOH-W7d!nj1y~1^ZucD`-58t`07ng-`Xkl z=}|tYZ0H_5hQ>66aQR?Q4%^T|gQG+9+!90${ueOW|Q$x6X$^Gc^R8&3J^O#3tEM+Q6_JTdoxVeTC=4mFW?(T2FPJr~~BF4iRk?THj(kIGt>5i`3qz z!K8j{Yy}XG!&P`}Nw#+u{VC$KThunAwrMucfpO}w3cV(GamYhi&yard}s-vJ9AIarz;<$f!N6a#R^MmiiBUO^_ zTgsud8Y&83)qPlpv4nUaEh7L|x7PlZZ^PQ^d_W@lN-#__xQUWoq-EE*$2ZSUvnMWw z4aJL3*FO%(-s*^?7C!60F5WI9ei4JY6|pbwC3JT0D;b@JT8hxTbIGl7h#XRA<()3X zI-2e<6jt~y248eo5pTdOp<%>*NGCvtGjemh`i=vBSDy5kaZ7@cQ)a%ofppbw#F~Mn zgh&ig%sF;|g?Zg5y&c=J$_K`RUL;g9n8oMn)@m1ByQSz1(}JU|?Bb8O_*NLsR6~&G zJswkxYlz~zM0&zdL-_y4-djht`ThHz!JQ((p@raD+$jM<(IA1~P}~U+tdvTiXmAJ) zEgGESPK!GfhhW96K)=wIdVBM`XJ&ribLX5pb7sz2>&{y9Usl%6UfDa(v!Cq!dB0z8 zTFojLD}Hrgl>{cMgIUE7vQ(?5P05$`96w2fKmH5-*^82(5ex}V|OF)u@;L$!un;r zywdW9<2VgsnZ9E$9UDs-E5(x&c{&dWy)388BIi-W?Oih=Q8n9+c#?HufRHDM+ab87 zc5Yky*E#YPuKiV}+23COca+p8h4g0;oV7Q8wFgUoe@*=8U(p$=?y1J6cU(>v(2Ixa zZ_aLPLBD2XB(y;nljqXiX%0QnLa}pn+`hW6tF`Q*)v^DBM@UpV+(9Z^)kV+Rl}jWM z=eg!Oxmh6<>Cv#@ogNp6k|#{0GgLk!A)q7R0)(~M9{&*78ggaC+XsxSabCpw`E`6XN1=~BGvJFg_vroj$7(>4XHd9P+8 zB^&=)hVughPfBfo5N&BS-Wfi8%c?>+8Vn#UQy@i*hPDoy`(EH!XUco56@;h6esu?) zB$Cl=bn0nO7Zboq>@X;k)fFh1tJI#f>%ZEn)Et^Q^uc3kUQ_Yc#dnsU%fFmATvOCW z9`%o8JA#`AZ}fl!J1O~uD$|_MFqV%YKX<_Tdrph5z z@njUdth7((=sts=iX8EOQ$P5ss1mApVka%WJbK|fA;7$mYa&9Ku5DPOf7_H7&&*}5 z%~Crd{l$UNmU-0U%Ph~e<+-`asn?WuH-4jHnsVl#{6Q;_O<<?kWC8*2L z6ujfceCL}>RdlUh--(K1sC2(e{?+rJ>AVT)y?azaWAp?vMC064Vj+|zBYc_fhLy%4 z9h^N+V1>^wxMaAc85na8M^Y`?v7iI-qoe~8kEk`1N6QNhFg;fiCX9TPyAL1>hIRb> z#9=*ow6W)+_ZT@DlgYzN5-OblLSAsm5?d8D*)hR7q44;QQ!GCaPumE1-=V*{|SgNgo0-W?L7hC~C1NUI8kP!d@ z9X}LP}8WWbVm*;L5m7?$m537p*WUz(Mc{)WH(OB50cJ+W^!E3qFT5>YPI@{#W zw@!xDNg0*k8qR@Ml*nT}!Ct*rwcXv~hWQQjQ;l_jc2jvtrM(|?APUb|$r44YuH|%R zQ_4#DIF=0I3dCpel=z^@HcKz@nwXZrwX#aVU*D#zLhe(f^L^VHv9Yx-5Z9WZCBt;AZ_EhELmZ&x=-g?-~hiX^S zxDV0|JCP2M-}E~z4dz1#0l81GbTp7Br69R-*Uc*TJ&ZR(1e>XJ*pC#aDwyVV$IP>EV>)CUS-rQTQ;l!1&el?NEc8&@ zW|gqr9(yL~aW3aGu9q1mMK6v0Jd4HiMy(q@^%P0t7{&ejrQG+?8hZ^mFJ3YGVwJ>J zPCV}(K>K@uiwx3SphsSH;rqe_g3q1>15_aQvzRBddn>MzBz{ORKv;F(4yY%2olP&=3$ynUZ=1&tEMt3PM>6}yRAf7}`Yks!eIO4@Fe#qkBpwEgUZ z8$;hFj;K-1?vfiTFlGmc(Px;{&ta#zPPTl##pg1H6N)AkO1jPNwM)P<0iY1|86{60Tt4Y^nY@`nhMaI(zJg?;m8Q+_yMy zZSaX5a9@xbwan|=9%S2qCyps0ylsrmWDMHrrQ8Fxf@n*u^a)_eUe+w`Z^n_ zW3_kvnbCvym1C7p`Av4 zq@WLG0woBUDA8&GFe7wJc)W=uT@}f4Ug`PI3a>q!kN!TBzf4V%>!e=0Y#0|G-0GxI zJHQG{$j7^uma}r#e+>JIX&lGu!?xZzdiJ^$bWr~_dxptFG(aA{7(ht6G>ctA{m|z& zkrXhY1I)=*(Rw`K+sgB2*?#!p*3?$}RyyCMsh;O)Cz6M|AonvJ070d~GprvXOE0B= zyXu4&*N|CY3iZdm`yi9Vm}wlka1n+HETCw;sKha^`AgL9jSAkb>p?213;|<}Sx1$KAt-h6#7g&4!(~{`mN~RmoR?lPp-KRI$cp zDXbnFSBJQf7Bqa&s+Mpt&&}U13@4nmLNJx>2uP554>aJ`p^Hhv)VtkLl&JJkG>aE8 zK)sc}zK7OCvE7VIeIhW2;co8mNSP>TAS0nC%@qzKVbpM?mzvg&CiA3wX5g@;;*(Kt z1P$*>;&bIlk{#vo)NA{)GY=MW6{@8m~lLDNxGJ;V`g7Sn=@- zw8b<2$me3(GA zL6zp@CIMt0Z-?`hnK3=wE2j7)1&O6&AjRUv(}0_mrq*mkUnd*h+FKM;-tI7bQJ zq$ktHP~bFP>3GKrWze%i@ccl+hz+DX+srGbay$IfC#Dp6+#TvqS~rJFJI-ShZE4^H zC2xC5PICY)H=%t#0mMhtdbCzB&=WVPr&@n(`=3K;vCGO)Qg*k^*5MhEZQ^lfnwVSZ z5m-D?(;&TRKBt6r5tbK6ZHo7c3XlT_LEMA@wYIcB0PR*(pS)A?*Yww-Km zSia&oy&O3?>rN%T3=pyvS>@Gu>y@jD58g~gZedcnr<~PpI_mE+-^O-e>lEH0!P$CW zPB`=^xb)&n;DQMF$zJdcM>tqcVty4$+Fatl!G3^AkM8hS6xL%6qTsWUPS*~>71+yK zMN8&MEfK3Z-a^-i(LCR>VZb5wyTn7&Gdt`*OYt*`ej_PN@-F9AFp1;Norv9+P$AZL zWaKcRS+?01Bg)*^i+dPl$*|Gh+8UA|Fz9f{iM4ikdg?fmAzXzl8;=Fpu8iU!VvpfC1 zGFkB`Dc#{LuWfF`8v}F`in=5z&}gs|EhvgaGLot)9;jn9&=7b~65C9Z5Lg}fE-ffs zf^9wUrl0;GTB)Fum{x$6QfizIl5iFPs(p#(HwWwC>JdsB)#@Qghs?fL@ub;II3LlK z+?ABW^kC|Dqg~Q7tpQMSCwp12EbSZ_;e#a73L67n;renFxtxp_gS(j%o!d<1h0(Jb z?YVL>!4}d!-Psy^I3+3Yh_(ys4n-9{U7ga87z_@+cr_erHY!EL$$-1P>MDJbKKxKI z7x7Y6_xriBN(!fDfySIc6>U-0R9zs#{z+UNRqS@l{FA$(TmE-98VOj3CK}=2y^DzC z*<_axc^N!&kfSv>Kr|iCQ^_j(mI)vEGEeK3gWt-Amwr~U0X21A0wMy9{9!UrgDSp3 zHKg40av6}&nGv|?lKAW-rlfzj>Cvw^Iy7+g-K@5qQ9P50$*El|njB|sU9lM33x zn-m+yoy=j%_i2dSTB zVDnxhvVtX6Z=UvoIY!xaUnq@skIt~hk6z4?Ex-_ zoL7lXGcUL`f~H!>o|@#2CpP%)8apSE3A;*q7|6k7^X2u-50ZxmR4HslgA;=1A5_ep zxk`Udh&if+5eQ5SMQ1B0il9h@$r0@NdC&@mpZ}x5@)IKKUwJJ5?zj9u$C3!2_=A4m zC|RCwSWeW+&ky;F|2S;d>}uXRel)2w+!yO;-HL}ql!f`o!TT|FZ1Rk(qpt5-_xu3Q zi%EBvs6dzs0wd(J-;q~uzTCg_Gas&2#Ul6zaB>F^a}Rd<1DL2VTmS9#=IOG}FW9#h zT->f^6aPV4EILB+1kJ`QzJx5nBY-G$P4Qcvc+ze%4^Inw?}inL@(aDH72!a;o>XKM z-zLdskbpBrjtd}1Pdk6!@R08ybJIXhvP*v?50%<^D{yQyLz6#xRX*(@T7si3yjVfH zd|O4K@S&q}f?)sy2BO*P|G2|tO(;Da7ff89Q|neQQ+A^4Z*X`u98aiWT1t(Qgty znEg6robo2fnwM)Mfcmlr{i_4C%O@YCB-k9!y*zq-DM(|MOa5f!f`&MRcCH~TYnaXH zq;1^vuP?+L83)#)EyWI~=eKUP{HRMeK`KgM9+^68=4&Of4JeTk(SAES6nX-1>6Rc}!D20v&fv{TS`)mhS z{%gd&td7QdoEjItYMcJ(D)Yp3(L8aa3qh>gaEGrrw?>hhBT%0mIB2Ti-ZgsRB$mb8 zJ{nS&{W;5M*=Tmv=A6y2`@Ig=Q>#T5j`zv}AEP4FSxr^eunR&}SJhy!9GoH72)s0M zAxSoRN8BHum%M0P+7&w6C}Zo}=doo;^Wro4TKH6&E1kQi#Z~gcQ@5l{Y&1k;p+Ygr z!w&CihnrASaI7!{Z6R(#=L-rmN^urh_8ea?E5~*&@D)>gI+j?P>~^~i`L;#p-j34U z{#lp(cQU4#RwV|FM(NaiyiUf40D*UyOokD;4;!tquIXD!^O+`7KL@_iApb>WBW?g;}XZJdie~=a67ZDDc?3 zDHBuyg`v*0D7^&&@DWj!LO(T$>}D63uTu|JktJbk$WXiGbV1&T;ojUQoS|n8dX_L< zAkg#Bi6_)o&u_bv)RK>kkkixCCa<{COmM_}|T zI%`Rx1k`#YcpW1vpN|^5YhbjKHIX~{kj zoi=m>EY()zmAoxB;QY<4n+5RTiwJo?agjwQh4^$MXMRk#Fxu-iq9B9L9I7G2S?ow4 z?Jh-g_0v@feQy9y2e86|8|Qn>8sUA#6hSpPY>H-jpj)#qwl)S{UvlRdZH1mz`uW#l zF2vN*{d`A|6{0cqKd{O>6E2Ts`^xz^P$SJUuH)vqIX(@@__PTuz=3(07&1Zw_6uoj z?l23@m?*7*>%SI93`LxSpfx64_0(I8p@$J=wACJVjV%@}kaD1no_sOxqqc=9Uk<|6 z(qzyKtwTpbspW;OAULlUW7Zy9g%o1=Tb^tzTuTn}XLyd9l2J+HVUo-|)B?sDG-{5N z$K$$n%e<3MKfHF>`H@ffuDjkMw1$x^H_pVWCTccCVmjZLx(dtPL|hC@`22N}iE!v= zzI2^SqP)oQ@Vtqi)TR2%)&om@Kf;XbfG@G;lW>m6Y;F48JTsvtz@c(k2@SUG)VQo6 zSxH6v&T?WY#$lEDoj*f`il}81a4MV3ej`tveVe+x+)pgdccwLVwcIi#8*wQA2oQgj zlsC*cN9pvr^_<|FsQhA|@5YUo`?*g$^InPw7VH3dqqhzy3EqUBvW{%B$CfmdR9W8A zxwd^9)S1e^FW>$nF;w1SE+syaDr-q4&oKb^7C5c{TQR?;A9k0z^gsbs#oG#i){jkW zZ-vG4`KugDsfBxX_ADnO(3X%*hR1>KkrREgF7(>XPpc=PVVuVz5Jm!d6pg{XOM2N_wvV7e>9DktDrmhZrkpFJdl4yr?HKlrpNr7wCD zF>qehH#Ba=PO8Y<-lHG+(}`Y=xLsA0MMz`9GQ z&)Z2PavI47Gqy7y$rQvbw`=NuP%V&xvxZhze3M`^67{aa9Pr@DY}aT?6nG9QKGuY5 zdMlJ?_vYnag=L}X@87ujO^XLXlGLin8bj>pL?g{-S%is_8gopD2*Bi;@o>TURn}2h z^x`l5iX8SsGdKB*jXr-pwX8rv2_5SmJa`TM@*XlXbY@YyR;;=|+AxO3(1kSufDf60 zL;wiOOuAf{?Gly!5eKyvrWRt1S;6G96FKrTvdJb#nMaoklk zENoQ0-LLlhT2^L;`)-c~)NMng2A@cKhADocv+)NoW!msC@!%yAFX4^*Yp2QwA-z7% zz`QOr?in0b_o+5L#1uYgI~ic+h!wzlP9({WV#0HLDw6Onne&N(F71fSde`^u=%U;Y zcZJn&ip0i5oljLvm%ngi_<4jbGNKV!lUzg@keU#{ue=jzSW4)OLgn6*{O0q$!EBV= zhT?$fTCJg|uIj?b_an`q>V8!YI6!bW)sy^b*eD^>L~^*|0hsghDU&faO+$R7?z)!02M4Q>JK{eO=BcVVi4LG2=&5Z`=i$ z+u};CUT3C;PM>g};?fu-VH*6jZp8hLcyYqzRzu(0(#wcvAH-8$qE((*^xRXg#hmnb zy~g0zJ-do$Yl$zEWc{s}Zb?p9v+iZ<(X%8o5TX-H0sfPGU@6F$Lty?ONZLsL+u&jGXt=t_m-abv1c%qKLN%k`yyHwxnz_$-iUnmry z8`q|KJV$R_cqzA?x`n~fi%|`8sM07{RfZ;q`7S{;G2YV@n?K(4KtQFB>Um~CU3w~8 z!&|QXZ{%H3D)waxtYw3C-IA#@x1kckCdbw2Pc94LMi+XVWrB)*Vs689N0aPfB8fQs3XuzM z3yK%EPCgbgfhJyFZ>kx>CuSy{7%>2R95=e@U@UtbiNa)HJd|%XpI{irY4~w^TE+7~ zG>pQ`7QK0W-y}x3;4d>NQ)Ty`d)`j$)}KOV11Cc#p4Nxee=;g~54Wf~*%E0-aZ{)wqzw!GoSD%B6tTt4*6<7^r5du9#jn-u5Zq zKbBTXeAGbv4T+xAYnw6ar}FoG*tUExh1R-AK&y|Z0yo$u$h8WqYq29}K*^Kf;gO}_ zRTK6DyQ}emV^R3^bH(!!Mx~HIkFO{9VWTPChb{}&Fi<|fVmWStktiTtw>uIHA!mPY_3WQerB@bX)&>6MkWYLYC_sP$!U#mV-`+)fy!O5qdXK?y+@YnQx{hPnN{Wq`^)Qgr` z?mwr2Rdb+ekl|Y{DfZVI`f)nt^E2nE-609RrJy})os({H?{{N&`nf-0# zD*?MMmd5c*egAh+m;c_p$rZh%V()&e^Y2cqT)Cpv^5WkA8VwvZZU2&Al+m$GO9JI{ zx!L8BLgS;HYvLHIiZLjH&M~PZm&eW^kXo7TgMVkUrXfkWBamt{_P81XCc)8G5)uE zaH;C$@a~_t)!$fU!uY#pwBaSg|N92AbPrZY*M1*uJ~(!&Ij_>Lh^p54wPPWdslJlSAbyP%7 z{}OU2JZt<1Fn3Sk_sP&7K&F0W8{xqpz%QcxXO~Ip-(+hd9?U3wSg(Ip^FTgA@~4mm z^`}Zrjq#ZP#m#}z*-qQ`4{I9oTftm%i_l>9*?PYj_THHeQQD=`ERShR-#iRGVPr}k znh*rw#g=WX$-Sj?6RRI?rYCsmOJQKzHzCS9f0iL|+sL%rv*93pG=unCH<@|Dg>;k~ z#n}xfrE}qkFg2liQtFF(u}5UWU(~H*2AGIM(kYvarr)BlhKB&d!LD}JQVJ}g06ifp zGETp){;8otUtXMhAhFni=J}nb1dMlB%BD~$BmRJ|NWhIcxA1~l#dX#!^^kG91RMqC zp?Ikh69oK;Ib5s4)w`A76%9T1?&sqY!^yh94CBD>6P?HCo=xNE>hCQ(4q4`tBHL`bv^G ze3iAr@K}mOyk9e67*JCG^yM+9+Z_^`z~GOS^$5bchf8J7FVsW3OwaAYYlBm|GSUqhjr3|xYX|O((|f;+|ksv2X{oe?$i~;iHN{AP`Q(g z*S%(`2d%=IDHj2Ht}`7SZDv8Z2T#;(Fd+ghd^rk28@IQMn#*k3@{61v>vUpVcqu4< zLwS55<48`_e0MK&Q}YIZ!VpvY0krH7m(v963v=9XL%w5O(Y?&balY>+2f@n=;+)D1 z9~}T`rwJS=|NY1g&L>c+s+(fuXY#X8d*i9I7A{IJr+XjagWhQr57D{6 zS>k@bqfO$kiLcxPn|cfc1zyI6)VDsol7CQK zabzCa{Y4C2dX+h6RaL}%-ypw5nGdKpXh=-U5e*m0(fWut?fCI3_=w21`(2d%J{1ey zwW!CRry5+?%}-qJGv*Bx%4-7m^-L;jP?KZ;{B@EbY`Lu~MP5^ZBS^K4;TtxR;(+rbPc9xX$&K0?dRG@F_3{jBafmy^&5th{ML6A_XWq04y-*;z z)(v_5qhX#N6@AksLdt#2J$9dT12oHPj_f0qL7kwdzMaLEamMKTSS0Zd(?G&|LQt^~ z&lPi>&2g;?#gXB&3fYW+c>z~+0_&2`aA}iy*TjJ~!!}*o)?u$8iKG|9ehS@3x zI>;yK>Bb;6d3(O*k{5`K3Ynxlnh1kPsQDwdvOh+^*8{)s9^P*EM5oZwG77KJ#hUkb zk`5?=yPpOuMX(`FCetK*Z^C+)Q_f^qaf{4F?KEk~PJ^I5Y_2#cazGN~1_cqML$xj$ zSs%unztRtLNcEaDGYyw2HR?gy<2j{#)o}e)q;QrhRmSF_!gEBf-V|E;2bA?aQccN= zsobhNb+^fcW+>PMnc-X`xaKI55Q%o8!5X)EhWa)?7@Hs(cN4hl~3C~ob!6Gw*R@auK%3TdVQ-;r=_G8YQ$1zU=?R~fe5~_%q zHkC<$w9A-v6MQr%olu^}oMuPuOP$eK#0Sqq<+`k<^I!I#Y{RBPV3pxgn6sLMcS5Qq zNT(#YTxR{0c-AcQ8Kruf>zU*Hx(rmNNDEuc-ToP%Z$e0+Pt>Us;8px>tG86jT`{W7 zT^c@Y`*1bF)Ip9>KHxa1#8jjbg4L7penJa?#S>s|4-&*Rmnr><`B4<`YbyDQ+RzKw zS++SmuGEv68ADoKmxig~Hj`Ft;8s4nT}9?mzhN`vWLFjnb)5LIRq&b`ztsR)klLnaqK6oov1tPLyTF1qghcq{mg2he zLsJ0UE{|eaX+X$WKgH10Kn<+)gr@49jaX zFEFytN|~%W*Hcopmp3WH+|09x=S+U&*kmK!9aYdAZa;Rpt0V7gqtE8Hm}x4@0GA_i zgfJjNAZNh%!XHUO1)c@1M3cOjzPZP%BGZ%Bh!h8w}kv zG$|d;#~#vFEQx;F$Gm3racg^%-A4?1G4X<_+llqBx0p16iwli-{@Kovbtm9gTNIHg zfOZ{gjWq6CZTbK~s5~0W>O^!jj#>BCF9+&lcYBCfb$R)^iw~Q3M5=^^&<`@si3s1Y zK7t_KXa7oD;h$ERRz2KJz3kG<&Bfuu-dgc9e=f;SdN?kW)@1?{jy|wDB}e5NbbtvO zj)PA6T3aK2*zDHJ3;w>yv(4DpG!uEq;;JKLsHDF9JG!snH5FF;=d@{>Z`b5*8IQ0% zCxoKZ-EfpJPv+v@g_dVLIcbJv)*Hcbs|}>Z{ohJ-J$sLkd0P$`RYzVBEv<`v#wS#|R`1cNY9eB5jO;B*GG_3m4T= zCjQoCN9ia)tZl>{cd{H2w5vM7k>_vwgL&3ECTRk?GnX&dA2rpP$olJmu^n6i+>BgM z{Ep_56#nI>v&Z`UwvMG?s82>W4KJIZr{256acpEo5_Mh%nekV_0T zh?om&xC3C)wSOcx*xc&#*{AQRh)9hoETtx^t{M)_$`RfPv8!DPoKNNB0}syEP>OXj zF^Z^4V<78v#R7^aL}w`S)L{TGJSHovOW8K`R}sCV?trF-{ncFr)st3N)4UcLQv5zo zf)>UHlNZ*|Oz7N{+5|Z)0%D+6)zA3DjYJjYPkk3;ygi>QFnxz`lKu{v4SQ;Ur*qtm zYw7s-B|K@kbSByAh%UZcC!>@*EZM0b89i{T|>8{_Upc+KuwE$a#dB*Xwm*vq0%gC|@wdQuLnjqQ!LEF4waUiP116 zg~{8(*WLoRmPHIPB7zX0vH|BIp@=5hnLXtK0FiplxZ=0(y4fK>4NrbA>^oW}(d%<) zuSIu3D{>ZKZBa>2cXUa~4dc)@mG)S34(^ zWZ|USd<4F(BQ@TvZPjnGn$KcrFV8+z ziqjV(<&`5|XVC5>2#v*xbjK6u3w|gXD=jW)X`Rf1GiH?8irVmRU~2Nuzs~E=f6%$V zhgUN#w2QVPJs_VR*f?2FeUwnYQ-Ieq7TyzB_^{Cf-7si)NoU?&vkhf-*BxC8-5g^R zuE5KNEtJYt;5!GFp_%1$;Aj%#QA9j8PzrhmU){8y zz3Y+N|BTkD{~wOv{kM9nR8s!~0Dpb=qwLmisK|?z)SZioADz7syRi>qjGuQ1UhZ9e zh-G`q^wh`y4`3g^4-&N($b-HaNc`Q>C z=a{tk_r8Ss{i!dn1ivNAd=39ab116wSLng7O$W8VhVBkl{=Mfiw;!vSKS%R^s{W3> z&MNz=f_!>;_x4w(oL#fO4{yHt*ZYazE{4BvKhp}=pa03#r5?tT`dD)6uj;?|{J#xC z3j6GqdP#4E=kI@x(ERURfAW3v==xdiqI%rRxoueMv%ih+@!z`sF8`n}Tz%T><_T zV-e_{u)d`!A*KRQA^aajWB+G==3leC|IPG{|4=%|fANqkGgV8!-LuV{759l#aT)Kz zCI)%Xze0oxDQn{wQu*MNDo*UhEpsi0^~ST}-e>g|;=hj^9vl_dz3ms5xz>FA3#z1hDFJbAyNs)!&$s!X`%Dgfn-q55BG~UuM`{nv-7X#ECR8^1T6C zh$1p}%H1OUP2g}zC2)^EnYBXCg@Q4Cs!uRP>~yG-+EhcKnoXOtgMevX7`%>7K#>=g ztlN{BP;RUOHa^td6OTRH9Y;2`N9Ae$+)^!HI0u?mQj8Iw=>iYn(s!r-r+Sp%PrpVD`zQtEmnY`8+#T}LR{;h-_W`&Ns<1+ zB`nYMMm@YJSLMwEjqboC7p`?(W^IVM%$#hOo!D@(<85JLc66n#0to}fUC+;-r5+lQYf)s}D_8VW1e|F-er{Lc^R9t;l5kK( z(C*d;jcHh*mcO{-E@TS9k~}hxdKov9DMW$Z_oyn-&*hXQG%w|q*iXz+b{9(I5Vn;z z%UJQ1v2wo(N?r~=-A5V}K{m2v5>iUR{8Fg=+rrrW!!6;z3mZa73=#+ZalIr$#V zQXt>Uyn?9b0lvjR2NeRz$Sr!AD}}V;=9hGS*Q9qk#O<_vd1#VnKR?eq5vKapB9y6; zUgVwo+ghg@U!nimUmaf>Eh!WPtRY?W=BK;+{^@&Ex3NeeAl8Yr+$j_A}rGtj+1rt8qsF1(mx%11zy#>yQ+n}#B1VxlzB zCGsRNG!=U4->a8FqwX@Pvu?E5>01js&P$Y%@DfdCm=qKy?+JC!e458hAB>hrTSLYs z6gdhMYtRw?em;{sy6@ey=2K7zTHo>y6+yV}6gX!~AL@<98WM*v*T(jL9`f+gg1v8T zN2=vLoT$F#H~n!eerWV<(LF@XBgvroLk~m)CK5mdXY@VIIHj1X)}`vEzhe@dF=}nbKj~-5LF=2xr268!>EV)R%s|(L z8C#&XM8?LTGUxQn#&fWVAr(s)wR}Q9ZFO#TDk}boir7ACRIa&UOSExxw>%FrCcl7+ zKQW=71Lz!7g{safZV2_s`jrU_zSTe@z1p&(o-uTvr*M-X(f+2oQy>sGrzQa!FIh|V z^l6=}+Zx~dF?dg{X0}Kfm8MNh&t0T#E)x`~3ybqg0&xQf)W}c$K{g$C&!N_GIt3;qYf+Sn?9vZ|4cPjwEw(=;J$dT>sN($#bQ}MzWThlq zAGb}@ZKk?w^u42cqkIotGh;L1vrLU;)6Yhf?7Fdc??1s=ZFG)WsB`pAr6cL*62YCR z+5&tryM0zxRe97~r75UGDWg2{Aj(*V1o+ZUsGMjr)^|ejz4yI=h)r>;N(P?l228z!Y=-u~xnm;_aqd3kwx%cprmSH^Vsh6jR zdM4?0i?A-p$%GiF%TdH_62=*;?Hr#wY(m`s1W@t#hSNHW@1ho7gu5d$vbQ$6n%Ln` zpX%ySzH`W+4eMflFK|kztqLlQoUTn1HZFB^iRVL-%^?Z;V~BF89RW*VS%N#iwkL@` z|4X^~Z?R}u5D(nf-19{D;e*8M`!i#R-6P8ozLoHY=F7Rz``{T8hvHHXfvT#n2L=xb zV+})ydpk2 z*b5o)q=<77nIcOz(kO<2X1CdNpy2Zyg(ujFZ+qMJ^yF%Kp)*D}%W5WA-M#~%vxa7xCy4+%_o$5CKN;tj1dMy3_$<@A+H@x zknaA-B~{<@s|+_jvFmpazY;!fnR+7)ZL>{rEjL$MR(AV|t-Uf7zQw4O6AQ5R`yiF* z{!PlwShtI!x2R^)XvZ)0f$^4zP+^lEr&&2ntohv!gV=MsEFKe!I}>#JAbuhfE^O3E zm$rgM8&ng385ceg}(}S>13@x&j~anBxFA$WT!Ex`nOIXV3^{6Ud|S!_{V44N`ht}{!cP26q0VbJE-(^Hbtw- z{Fez=cNTBTzUKqW{Ut32%^t_~8TY#OZPJn4XOB~KT9%^<{pgy<$M`UhMN+2kQkp_b zG&)5@B4>@H-HH2wKj+4&_{R@V6z*QRjrLt`NZ!lN=rSPQNBKNFRU-v?(G>EVfJ9_< z>4{RQ59k?kpb*-&Hi>bnZ3?RF)n{*A?5dje#0Qllk}|M*;Pn6^(m2E2b*^r2{5-5| zcRpjlHQ#I$Sm-Yeg=vY-av6+lE$vR5RW0f_nmbT2Kj%gQ6Dbq9($jfpT|h9_U}KFH z>B-67xkh9!Nng^T3%k8lG@a)p^H>%d>V*mK7s)^!oQWL%#Wd)4EJ8rL57j3FPV*C; z$cd&Su>xJY##tA2dLFe*fL~cDKQF9@Yig|tD}~-;De*Q*%?q<;Jy2ab{!UBN)sSS< zOxBd}RgzP3WO~kF|8tnJHY4PRmHWeA7q3T9G3DQYBfUW}+19|6k5SGai2Lnw4K>e@ zx&2|uqS5d6e(Wh_-WSCk8%?D4v4nn?hD|&Plr>96j0J4I5bsV9IVt?>0YQrWjwL0< zw5gP#vvt7N=K`-%&yNh94+JBVtQVNx9ZY2m+w)Byy3vE@rzuam1rinvX`4rFy#SSS z1{DNLa#h_DomA37nPH;8-6wSF%FwUK&%AdxF!>z<(FAww-7*BaJvrJW@wz+3Aqktq zutx_QZoXkMQcpOTY_4uH<_{y(<{39CCeeMM(#Mc#?MTLt$AH}2s@yRt2&AlY%0dfA z{XFEh4~v~yM`CZf>x6B~`Hr2UMqBsRL#F`EULF!z7}V9(s4P&C74p+mC}&lO%O7E% z#^Md5vim640p7`01(dOxOw9D=sPO zC~%HrMXESZ#1(TW-uL7_SEOY)yBuGu#Ime!2ceRkYYyAF{QSv*cS#RrAGIUYS#*vB zsR~3)8M59mnpFjuFw)V%TC*F}yA1+y2OUyvBlH8U` zL380Ash(z$CRPJiP28_R)KzT~t`BB(d}+k0PsXWadrvG+aSIpf}QzW-)M)|hLq zx6HZLoX`6_cam3KR{WFV+083*n#d@?tL!1a*0^Bq+|wZ~xq9 z`fwtdfBDR|n02r))!1s;Xe8;1+2W0tTX?}J1V7_aEmlwP!BaMk3v;%fg>(M=+O;pw z`O5tH5$(-g%f@Y~!>q0!oJyQOQu@6RMn0II_wrsl88XR_{ybm#4bYGKCgf9;x-RnW zD`~2C^>oEx*AK>zRH-e`b3Zd-g(CiRrRS4=wcGWpl=j5%Z&y*jT)dLMd@Am?-Y%u5 z-VyIM+36o?qdlw>zFbsckT7(9gHE(kFw(JlCCq9T4$cGUC@}LORsP(p>MuiZL~L37 zBr$OfExLaF_lH**Ez19H+An5g>D~Bc;zjm=zXB8s5~I0Pm8#)uQtmdnf9XMVHCi446?*oVP`L zhBJYf2uEZ%M9?6-~_~x7SX=}F~p9$WN5tkcRrRnL#o@If@FRwQLceN zzP(*LqjYm*HUkh}mP7!+O?e+_7vMo7mJ0b34h>ZK4!X9T6(z?Z(bD8^2R=v$bvUt5 zt8$D-6G04V_8zh%S^oe?fgJ$=s=peBLsqAxYfAGpXa7f(W|u+H&XY^`r`77g*;0aD zqE900W$sQrc{uT~wU*+~U-EyhSIwo{KTsI3J>K8F9=iHxSO0KZt&{6nWMD;e0z?Ln zoV4OM?3Tu}*ibJBO(Y*WMvzsiOnVIJ)snPirhRB=)@%~p$kkY#f$Ed!H&Fzg%v zaC*SL&--oM=cg*~N?n7+&if!;kJr2#`V zreRV^Sdmsfpaet%pz;NLP$nG1aRGr^IBNCQzjvX`3H{)yQmh%nMqEbVl++yC<6*CYif1=9;ib~v1NX)k_Dlei5{ zT&OqL#6fA*xf2|)?`8Ko^$7{p71dgr8z1ktHTxsRT)&DMk3OtlMbR7jV(uR=RtTCF zT3C% zUG8O0dS6FyCZ(G;IQ*j&x~|b~jGtNs4k3LO-V+lVO+&zGeSz1`rpwy(8^G@E+f^@Y z=0C-UA>uuilXa_{ctmll+({BR#@FK|?lP#K)5Nz+84R5JTS+! z9Nm|oc5gj)@mrZ;M6Zohp{qPJ2qlIh)2mrZewN)=;EW5WkGqi(Ml{T?) zj>J)rkGa}j`S*k~>r3(pN1V{CcmQcUD#Ct(YFBp)w&1nF;;t&Agw*Gu_vl5Z?#j3w z!NM8A3Cii~d<^y>}NDgrS8JLu5|XAJH(Ag^$DP zeZIaBK4XQ#3gJTDZ^;f{U`69zCP^m>g+uSw!Pe&_ThBv2&MH#%OrXB zR3)_dPfXq0WoBa`@c1b*B>!Gq9Uw*fL3;=(uR(Bo71BoLVQ%DIOv}qD_x+u_EbKR6 zE1jM^`8~dvfp=(Wn*o`6upd+k+seXmlvMZ((hXLWP-l3rVvUrVo0bK}(q-xCb0|~t zgX8=UI2lZ3jqK$rfHO>#P1aN!j9f6`_x%SZB-a!v&Z2Lr<=ROs?vijufBe*0cyWP! zLGe_=wyw-g@d=%uUvcB%HV~q~2A zaWxIfy}4u1rZG{7JjxK?%d!$4>2@p}9-;%HdHgcKU_y!btSebE!VO{o?faJ^uHBk4 zy}JWxwk()P54~qZ84ZhF_`^4&eF?<}y8ScAQ=tR#bM>TEad-E6H zh1I>B$(Zs$LxU$>z%uSwL!9~D{;1iCs(`HUOtvp zd_@s^T4V9ELV_ijIobMW4KrF>Dp&2)WPhUs%N53~E77{U{tg3gL3owkJpAZ=t4@FWAU;YOSK3XJAWZUW zmkzovs9N3;9d=>amom(-;VELe;7$HaWx4KFov%Lf_GQ(YAMz$n(bLD-!$f`g+9yG1 zeomK{o3A|~qEOCetJ*GR|7)T@;|R^3o+1c$bmxLYSY3heww{jv5-K(p9rbob@4ZT? z_g$@a(Oin{BshG-bx{o=E1cI1tH2fTzc$hF!6_VV!`_ihSsP?$U>Cmmymndz0sS^=hY! zkBA|M7TI#%Lzs|LL%d6n@@zw9nyS1sxx|HJCo{=Aq%S$aJ3pr_W%yn5E-MB`-V=ok z!5)PvD2ee|L3ljh^ki<49tkWH9L)pw4GXRoeS0WgK`U6?Pq9DUPnK`foixj9j@*6e zk$5Vk+TOyp$i!CXZ(gWmgyd4-3L{5}v!$y6iao&qffo+1*2W+s#Z3@Y!&LNue|kZ? zcM@$L)J#r19ex-f4Glw(Hu@2437L;9auFDMTNHhcew*}}TBe71I9Znu{i=(lFD2Li zc=F*b|6y2nJdW}Js8*O#8y>YQmZ5kyO_>-b}Sx><(Ggf}(M zSFO=*aT*4#@Q$p&H*f%!9uFWSv6pk&_gWPh9kr`-&GZ}a?sf0co#3on7ImDwg4TMz zg^u|ILrP+8rg)GgdYBrD-!NKcD;wcb;!hfP!;zHG=iY8zB>7U*t8Lfz07{pD>oO@#>lO6V+}}#&;Y70e0F!G1MEIoOc`! z@gTK0B)c{*DM|=bs^jC-O9}#ff5XpSiY-cgCSIP%%cRlIGj>&(pcLTqOheGu(D|&V z3}1^SKU(#9bgLeZKHkmAetbNf3M;broh0)Ep#683_TGzQ0TvR4tqD`nDf}N@lbrlq z)2F6we8E*XvAzUaNqh)IG9i|L{dJ~j)vSTrGMr|9(!22O#qkn$3A4idT0Q;fR>4^# z+K}{%w5y+3{(2w8H_DL;1HTyZmK3SnAxA$2DI&j2%q{x)*bZ zvJ@5_2=w+!O%aQSMc{`52o;@ENAre%OazT6ZM(p}5P%QA7b>B~Yi@!Yaa9(;Eg5{E z@{wcy_{%MQT5(NJw`m(h#_G41g~Dts$C)d&IK2is6YJJ)k4D3GqPm~~+XHD`C50cy zsck44s1Zbl#2X)Uj~OAMQ=R8Z&tq^xwOO6>nz=mNNNMbT#7an$wJ6fgKCeAdmc`SP zl2JdF?smVBgT4Wm5I8IzZ2&mnE{!2P&Ae%-dd>rv^smi{mlzKz9Q3XW%ivJP^T()2 z3LO3DS970R4O0XtKJEORDPt7Tg{8Nomr281M9(Yvmxv$9NzQ)=v46|Bzm~5>2o~wx z$5r2d1#~k7l?%P^N$gQ<^mwvti`&QQlkvHAEd88jBEavndu#J<$?`I}W_I^^lmay; z>X5z~37uC39x$mow{mkd}tlZ;udV1j81csa}LhCb6LW!3cS&y;~ zYf5AV>|A1Wcfu)X+uWW-BO{l;ho5=g$x)CPrd2yRl@Ea!;-Fu|axRsXvzCA-LcQi= znBM;PQ!GQ7-~EjWgBkplXTrhS#mB*@y`-mHfjK)m(#xsT-wQk1_w2PSB8E1Lpm|lDC2(3C z#yV!_x~A0k=yhIzU!Big$m)n3h-X+W0+NS7IgNWK4v6K<@vr>&!d>nz(HPj|Iy9cWlFbpgM@R=nXVFcosE7kdm135qGxn8i zG-?zqzGaY?P0fF7t}R)$&pfakxvd~Lsv>v3FUuY#l%=R136H<2X)21psJb}v4cUGuYf=*u z0BU{HsWoIcF4QTfPS5>SFr_<+Ml39?%z*fes+`5(O>Uee#PJxi?mx7MFB-n>UcySH-B|aG4&ADT>#q+| zg{tQDJNGh~xe<4iU20vR{U6)OFL4lv^z!1pYJ_~FpTk=nynbiwP;l|lfC=>4q@tlX z%OSh4_QwTtvwRQ3pqyfm@Y@zI9n44X1Qz~4^*H))ys<3IOTaJ(j(KzsJ6&F~5U*#L zC&K2@kocgs>L#5=oZLCAoAy^|)xUI<10~!T@Gyf=MgmdA1WRQ~wJ_>%(*tkFxUC2nJ2BMi@P4-D zejdBt*genie%j-pbcR)s9}_?^3QErYhDZNy6c;e6)_xN8UcooG*P>j|{YOCPbjOPa z2+v6`k&zs6sbIF%3Y1TQY4>SP;$pZtK{%Y3qiYB*;ml*>sLzeA+ntIsqqI57QCK(i>Ro+z0x!~LURH@KBH*5*2vs;&0v`qE?nY(CG@u=^%O2gvY%Xt zz7T&Q`uSJg_mI?%v&&|jMEOJFBEQe6LO+i!AK2di%cb=`SbhaHKHGMuxC@eVDpa_pQkcdPxWn!25iPF5W;a59i`Arv{t9_T=r36QUcy z`v9iakL_-BIWnWTsXEe@Y(>7}E&18o3LsSPE|pLlDS@8|Rt;lO{RM>$;K}7FA5`6Y ze^(6$*KTdPl*YypmaW^YLH=@?!41^*OUbcC1AEtA& z6L4_>IMna4CAVl#vB*O=A)r9B&3tcDP+gbF;2qUC-hs`qG!+4|#<)T!Wztfh82fTK zVL>32HzU8){q0>ZH#*U|=x4Fc-M5((PJ<403>oB!@UuBel-KkniYsaHj;DRq@> ztQuUE|N3&ZD&H)*cc-_ZwDX3@jOqdzTlf!;G)uy12j6cjb_=EwwEg@v;t;zIdj0PM z`s7P36M@(r_J%i-F0C$PZ})eevK|&@1ulC&-CTesvc>TLp1r?8vsM#u!yXCQ`TAs) zF~6a3SqES0?EAHA-tHEArv;OKe|XX%B3~++C%m8xkWOhmjefjYZS7qL7V`e-0|!ILVQ;dzy5d|?U8;^=^jpFc0?Fy;4_ zJdFs8f3ip6gd=ZHC;7hNc~EvnB9t#C$mnhtIQpYDZ9@PGP#&DM-2HHy%U*V2N;oJB zYi{wrp*$VFonrvODH10Ks;5*%)Nl1PcM3wrPDsp?vF5|JW>3 zNyF&JLJ7eE^{IleH+r213VDEolJ(On!aH%OD(HwX7jXN%|p?Truf}g zBy~C-m$KECfN(uX!~7t8)SGs*5^iy2(cOTsj6+7j{1)ua*KuNY4%!mU%-fMy5OjE zJP?GyQP!I)!b-m`bM}zG>pi)wkpgh*x0gv2q0Sq-`g^~RsMrG<(5W}9!m9bjYK6kE zbG_o%N0k<}?#HfK&g(YVA<+}avjMQ8hxfW?M-shByOBBZ`;D)OL(lWa8M>y!C*owe zM{9KE`UNu+tMdSrGR4hC-?Cdz9>|cPYM6p=v!Z$=_|%GOCAl*Mv)AxiOKT4S?lkny zipu_lT10JBS!L(RVyBj(lrFrHoQFb#mJ-a@t_LN zw;Jw&ge@lXpsT!GMNx1gHuqWYXJu;U^*QDR%w_kAtLsYxVr`Rk0AjOcS|9ryd(svYF`xeWY^+1AlD0zb zcA}0hAO2eyvXm<{7oPLFa$T6ekNE z(MQ>NgSvJMF3bG&&8X|AU6}ml1Z1#T@VcK7$E)*UT{)ZG1r@@^P_%rvadQT> z-G(f*jD&&h>G<(9{k>h1bK5HGSp9m^2k^xQ+kIx+qq=h``_JzNvCnrgcWrih&P!Hr z)YJ=pN+HXZbQjm-#{-D=L;;-bx}@#>JGM2!lgkNa_02s=Nh%U4S0$zlD@cQ@jOW;^O~IhdHZoQxV)P}^#El@G!*#p*X| zGGj=H)E4k)Zo4XKgK85su(om?jH?VmcTfGNKKJq)Wq2^21v-G0(qP@O&PxIK5iYlg zXt}9NfLr-cHvp7KCQJdR{!?2!u**Ot84qe+&`8vowsjxB8=u9ahs=P{%lJ%Al(5I8 z?SY?PhK4DmT9MB6&0qA^$xx``lpSW(xddEa8H;l%}H!I3N+-r2R=#ht_*tYis|A zgw@P-x$g`1s?_ay)&iX#o??aWn5z11(IWbQnE1zJa*xX#;1WDMsk>S>Y5hpDS{p6^ zF)a<=mwSS@z`72g(~n&@IOKks40CCSrtE5JyKnP}go~R*P}I;MJASxfJ+$?ey@~(z$AfC)l*tI(p*c8MuP@gl&dQ{aA&z8_({^jP< zp>JSBpIcu~)+4^rYq`kh^EyK@%v7n`8K6D^pH^!fqH}5u443GA>F=S+M!U55rCo6` z1W|7=XE}lHb>01eWvN~qlyH(ZL~)11-f(%{oRq3YwBpq3mR519pzvc1=CtYptKd-A z8lC{2w+W@N8ocvCs=jO zhqCX#L3>>o=H#X$8VgjsF<{v6MZX~I!0MQ0T5Y>}PiUAf2lg}bRG!~h_3>)d;w#XW|Z6mCgPfa`Rl&usF z4#hq5@eD~E56XfQ?%j4o%E!y|Su!7OBxz`WxmB;!C@B!>f9_zqH25H5ND#(qPdW*} z7xk{?qd&VVhR+;e1k~iSNH`zwYr1Y5XK=;ja*+b_1@fj>J*xkOBuom6wN!A+DrDc~X-}%^$ zMn6Ib@M-)-z16gDrD!9x%ZE$6uQ9Q}eLA)G5xhe`(dA}&HZL7%mGTZ{cMPrVQ(h zkN2hWd;eOJ9pzcXE+z8j{K~zKB0M7lek9?zxlvs9*o&P#4&=j^j^iMj!_O+OW>!{q zp_kZJoI2=nWzS5tR89ICbM~V~JRfP8-wWA>tLRI_)9gcc@o+ewn2WG$KNCA@sHx6T zURYnmW>}~v%cCUJH++)Dk1Qgw`C zzLFPIKP?q96U(&X%`53Dwt%pdMN0H>YgNe zDg7|&5h;qiHr&E>Bi{zZWy@y}kLF}1a;~2;Sd`_0YA7!ozg7wrvq{fzND2%C+saBt z0P+nJ5GPs*%so-Eo^$v|HthIy!DHj{uO`GVc@SUXa@{B%_Y~MB5kDb zDeM@OW4}UG^NeV}&T)}6ycl86JP(b-5fdN~^WJ@}DDoqaW(qknY4|3jAez8adV2hh zma_UiF=`$F7m>7gmF4hKVY_aSxl^h~Y-hLUo>`}a)X+z_RT*8c7+!bF(&#)$RLNY@JN?ihz~F_&vUd#*nWJcKj?MT2(?X)F*qqdmsqs3?e8B zSpbpOjO~PiaJs3E(m}`#jJckRAM?AfvaYKqm(D66rkqLDRoFz4XvZ>RRFP zpQe+4=x@MN+Ugf_BSSxEgbcq;$1tA6ps(<%HfR6R)NMJv`ko948oqmbIvi2;DoU)w zSH^ge>K^s8Kh1)_jmuehO8o5akKXtF@zt+iyzmEJsfk~>4F8TrDsvyLe>-DM_?#5- zuoS=IekiH6`+@6=m#j}Xl>RiOdhY%PypEeQF0$SFNy;D;e4s`Vpk^N$Hv8%DFU^Xj zZ$Ga!{@l;3l@O^#hE`b`^B^rgD8s2z^%}xW zu0IXH|EnQl-2P_`>z`iyhXg(A9eP{={?A`u=zoF~8ieLC?F~(i&3D|Y+#&s~NVo-d zJHK*C63j(JA8!D_Rln)5UOxuj)mb)XXq`u`q<@iR>uu}^T+*Je{p^Ngnay9o6+-@y-vADmU+J8tNc%~f30rUN25B$b#0LnK;Wf#Qm)H-cunPQKuPGI=14I~c z`62-CxSsP*7C(sc@PunuHQi1>U+5zV%tq}qvC3SptZv=M-}ryx7XHC65W#=t9jOZ$%Fh|7 zU)Jd!GLD{P-5rq82qvyK0Y&EOWje3YXJh7o^c?mXqFZT$0GUd0WfE9db43jmJ>?be zU8mIbDfThx_%3@Bu48m~`=>Vu&oUimKkz|D?6JJxbTEJ{S?Hc@T0NVyx58rQY2p(h z2MZnAvv+ngD&ueI-2f2YJnG8IW2zFRzHH9acP2TSg4&y?+tsS8O?K`ADG2Q@BTI-* z%@8nS%Zs|qPB#cm^i{|EMyoIWn+hQ%g0yU${Ub&NwvB?*geZ`PvRn0r9wZfZp>fDI z!=53^OVgVUFvlm;udp+b>Gat}&vm#b+2gf*^61U|2M*M;#SVB&KwjSy6dqxWA#FkZ zGF`4$2@?L?CueTltXNrC&axMGyHH*{tTe$UC32CU=6iqG()+(wQ~g_M71E;MX?e}l zGyJRH06!gM9nu)|{NU)}weqXqfK@~mQQaRT^S^obKj`Ft(LXq1MWS-gweN>uwg>l~ zy%L#p9VNFw;55jn-+Luik*xRt9#@yueE9&!lqy5H2u!>q2jz+P#}G{F6fVsO>aFI- zsKey$Ei%chl}$e&3Gc1+Hw_EC5#dSeNPs?-)>8wwum0;s5ZrgH?$KFf2a9MmS7y8u zp(zR{Wx8lkYZF$eOgl|t32+f?n{>b9y<4aoAss-u%=5Efy9YV~X~ujPW-n35mi&~x z>~$lMEcCNrl%~WFxw?k)y9YXvM*(f)d48 z%WlKTVdM~&qhHJ~L&VNvWfZkV%K&1_)|Seijqx9%KWe&P?MCNJQ0>&<%=>wPKOKz- zbfP$->2Nh=i{H-5a7X6C5w%_rzRn`G>U_F2@9=*W%?3QXJwgcFd!u4*_36zPm*q09 zeA2z(19;P%Q{ME$&$i$P%tuZdGo8P5sCHaf;+B&v{esS$doKdd^Xc{z3j=k`q6TWTg`j04*EC>!{Qx0L0}12=p&qhSzW! zyC>g@7#iyB z&#WA3b*UOwmiE>d*pLla(8D}`+%i%5{yfjg=T1Bltwpa4N0PpYZxt=eH^j|Lr1@%b zdHuEkU#w;y&-Td+iJr$_J$!*VC-wsG<9sv~wR{R=+Oe_I@j}+i|27R+$xY-%S2Xs2 z3&_6pMPvyHjU@9EV9m~KBew+(YJ1(IQUj!IaN&+x1tS-w2>Y`co}(AL=R!=z%lVe4 zPpL7_7>+C}Z;LW-dEHc(&46e&Brbgf zheHLp9uHw@ZcN?C30p^$i!I5rSs1Jub(}M_2}n;uugXowWlcZG5QG!C(^3m;hDVWu zW?||Cmd>!hFwy+~h!h4-f4>K@Ss3J|x195JWy_57S$1zj121#t0=o?eU#%mP#(9%S z`PHjU`YZ#`G)KS9eEtLUC{?MKoi<&3YR~QOd}pC^3+yW0q*6$L_wFnrc94qR{W~50 z0nxtNnLnx1Etk*ID+{pK-uI?D6x{903htW34fW&O!wq$CrFbT!;eKL=)p3Vd)-eU+ zqg=Xx$N3J9v(>GPWN236btXGk&>A>mm7|xQ{xL>+_Q9%j!|h(&47+2Meuz4Q{vbS$JhE3YR9QsV zX4}KuJd0yq%$!&cvDTY60Lq$`O$sOd%9=ff6wb=FNW3_sr%TYTCFu0SR3JmGCzP&N zpF5~rhAIr?1z+~uorpaD;rRLX{LRd_fq9H00xN}_JYt#E4?N=W1pLN_`07pK zzf8x{(vZ02+*iiCo?LdN%M0oLb5Flmv2UOKL-R>?zs&3@l)n$WO*gR{w=&fwvBOy5 zniy^-979^K04hm9QB`t~sHP4oMJ^pz6OG$Dm>$+O;>ym3sksv+Z@chlRU}iMAVD)_ zT?^^X^Tx9A3`mZIEy?7eqR>^tpDiGKo!diOiJxitWSTup=uI5}1xB=#^rogM|mpdra}8#~j(?kIcm0 z3o4Id4$IlMlxpZKWnGjY#zchK;QE53Xc*Kd4i5_O(fvyh{09!1OHA%_bJq_reCx_0 zulxAima@PAx+X!$JLhc7Z`g=Hdf&zYXmCe#)zq6G+_TNLploSwMbU)O%S?qiMirBu z?^@HCUh>75KhNI)4hd=b7Sw_@3#to-1@TnlJBsL`BuxKZ6Nui_Gx>TVGh7*w8=)Zz zo4ejAETcIrLVu2z6)G!D$%|u#5a0;=YjlXfhaJi-A2Waa_u`AbUeMzC0{g)F@Y&0D*%kBXw$-xt2)$redqI|tP zGWALV<}zX`;x3Bc07}XkG<_C9-Lt*!?1bnMPXygjB6g7V-uAZb|aCx8_w1ppBL`*Y%m zmHr2xQ@{k#b??t8{)6~c=+7Se`Nx8Pi8RO09Pq>HjHX=%0Ibi(aJ&ow)?Aik=W{iatnn#$xPyT{Y(qMVN%p?MYE_{WN5LIA*s2vY(tN=hIr~N?Wn!& z3f8)E>%&BD&*4tUmfjyVQNoesL9a>b zib#K@tXCGfU1N$*FAk?lxkGpZ43#>1}f2fe;$jPxsvH~B?_78pq zPg<2`Zg>R-cUGu~1UR}$t-lz;R2Dzy&@mWo_eL{FRmw(W;;_x=RX9v@!+?2==E}d& zeIRQXk1lIa9H;3<7sZw1vJyU4yWYyXQn?yL@q`&qt=>1Rg4?{>jar!=h|y9Y#gXqh zhWLHI4_u>Js8?zt-Qs@FXg`>g+-t0d)G|sH)B~)Jq0aE`IH_e!!L9i zI9pAYA8?b}361dFRqB0pQJ88!FsD-O`Sub<`v8nQ2t^6oz9wjbEy(-`BkXXLb4 zjwevYC+kszQ)R_ikHDMKGs#zoEm1wZ+O3KpJ&w2*Fi>w}7b3~Rm)P5r;wh-ZTM|d5 zT<+y=5M7keQ;;(s#D%um$h6FOvBDD{&t*3xHTsgv2axG_B36ZIl>)3N`W>AIca`U3 zK92Obfi6+xGjv*hDLS-xa7|)Qp|s<#3N_*LhQ0g%Y0<{AU?60MKa9M+_r8Do#WXk zE7DH6vuU#hJmfFJDay{+A?bzUK}2$0T$Lq``SVaeP+7?dhwS}^p2-vwX)IuYVaK9g zjX9pKH>56rN@sOEIdGd2d&OA~EySO&l*M?bY6DjU% zediPeWdkX`wBs@@n`=@L8v7wZjD*Aw2|4h@CGZnQQSw&K->3$Jm>tknAhiiox{ny7 zX2C`KatDYk0yr)0_lP>rD741-Zf~mlaw)0FXEk?<@X>C@tsNlIcZ+d{pQXP6!ts^m zRrGqx&tdHuL6kpHlIP7&dXNUrJA(uf6l8>?C0S49$gS~aF%>!v7m|etbmQ^yf ze(OW>Lsi`s?fdnaq+lFg2sz<{BCLm!(8)s+KnAzs!zHnBF>zR{FYSxvu2c@$_>95; zg%+bG@)q3pN{VknyiWyrrq8FTJod3_9{_`DmX-o@fTIB^{M&-)YqyYqbWxY|S^mu6 zlofRadW##V4>-X?wX=jrf>|$ zf5CGF<)Q!~Z?05uDm1A6w44U;&k6(BlE%$Tpn{g&j>K~ekq8f;enaeux#E(emR-~X z)7cJ@q)8TIPyA9#u}sOh@=D)Xp(>uRGGe5pXOck{kq9L**Ac*NzA5IF^-dv%FYw7D zZr>5#M42)YIZ_6xsZu;xiRBQH?xF%&7$p}ga9Q;}j>u;7MLb~{HoEFXfU-f0)OgfV zld}<@s{#JT^oS|UV1>J%%P~~;aTq9dJ-PAfDZN?K)Fd8Dn$>!fn@ic0ie}W~4V19$ zVIMHHG|hDQ6F!;2?M7Qtpl?xSdm;L>+auF4SDn_%$<&s<5pR!BiZbHO*skqKVdB$b zDdAYEz+TBT$};j|20%}kEPz8Yj9FT+b*_wyOS-86Qtl?+;^tV05Q93*P`-D_e|I^8 z52WZFz>4Dpd_&84r1zzrG^TLz)ALj+Va6H1x)N=TQ3uk9u|d)ys+C=S;K#MVw3|TWo;7NIeR>(a2&&+vM|+zM=BY+#@k}? zgIZ~BGzSV*VHaxR~RaooLt7ess&iRy1m- z$VJ(=P?OLy-k2b~lbE13r!0O#g<724``U)>6?e9U>n@+$q1JbKuGDnjF<;XjIuGKH zc@cyLj=%h8NvWT$3WW^?8pzrfHcWo`p|ScrJJ`7++SB+Od%;%uAOn5Giv5iK5Da$2 ztfiV&6Bn+d=AUN+tE{#SYVT-ii*nsI>E2a9)(idyRO*(>tD)K%x|)B^IMpz_SWaZh z!Y5X$aWycL3=(DxsyqgGQ14%Xl|;D0W^4SgIFGWHwun=X<}tc=H`O(J;KaQGUMaIC zQ}crA)ja7lLXCv#KY#|@VLxytmHc)PA;gum67VJCOPEqj{)PWij~a?fspB%F1&{3C<>KlZw+#Ds|A zb(1XCqDx56;98Z_gh#*%EZzs5aG9NtcXS_*+IU7mR%F!pv0^IeA)jgipe9Y5laAWv zSn`6YY$^Y5U+o2VYEjkI7Yo_gZ^u=AnX*BJEH8C-JWr0)fl*C*&tennnBZ0T*NlU)gfJ9I`2I>G*F20o@dIcxfWlZpsP z0Mvm{8c>%mVGJJ_HD!1EFv^c;uTq@lizCrI<&D3;P=ujvPkm@Ke zMq)(;J1yc*b2#64m=k%R7|0lWvn4)9!UOZlw3ZD|<0L=V$G9-TfTl_;O$QEkR1CHi zvW%7*-zmy2q94n6D03Bn+Uj`UcFdo%_qbj3PKTF`6t8_oD84VPaA{H?xV|Y94k5Sw zq#~t*sASXg2}iCTxh%bS-OomSR!K`SDSjaH;(?+ei#!mTDKNUH3~uT+!Qe zBcOET6AVlL(JPx|!1+oa1?%nbf$WTsQ7l_N2OYE~Us`!N%TIi)%EM}2Y9xJio6|#k znLjHoaHh!e&D46xiNE2#PzGsZMsR-IXQMYno>uxhLJ^_T_~AW{E7Qf>s9W>A3;jo#9Ko zTaj|ujI)LnJ@$j)vO`Qd$xyP80$%|SZ%MPUe{mQwR}7B8RFIM9tXU>SruAy((Fh&$g4Jl55BR$N=lO#RtIwhr(0d+2 zPI8ZY#Hk9?t@q=(;B$I5mG17A8)WhG_~B3&6 zwlR)?%isp(M!~&I9veb%y%#=dQ8#uDU{x;<-uY;T*JAFl-L|SVwq(GVwfK6wG6EEG z_Nd6AM3oszq_7W~4fg{BSIdt+)UShXb)Lu|$VDCA!HNfLm6w{yJK6z?1l>9-58gp5KEq+3EP?r=cgWh)3=R${|=CuJ=tRN)9E;aD8 zaHT&M0Vm#gO~o-dGa3)I>{r9n?(D1N&|yr52Q(^9dNj9X`YqSHW^)}&D7ugGIdD{B zm)*@O6Cz5r<_5VF_{3sjXk;C_U6vd(enrB)JA^5csBOamFt^cgzYlt|=g7k2-VSZLST1)PMI!~mra^}?*W39+#(OJ@O5LhXOJgmvs za4f%cV4`9tGKmYmK&jpUUXH6v9*;dx{vTo6wli#%E)HLljCSoIt|kfqI)F$=#GhJ7QXX1@%Gv6mNmTfxa0xX=NpO%Hb&891*BU0 zvHsN9hnn6(D_yabd-3hlw@H)E{7^pJj|G}_7|dc-gECZnGST&Yh3T2=GBFoNa^^cI6##=n~C zS}Tste`N|J3^MkbiPp}9I~!fQa{{4u6Sm|grztPpZZLemVL^23y@~jSi4|s zG-1!=;n{!{_uE+nnbi90z?17$r8c`=V}r_5x1g-OhE4T?3fa@(D_QvQinGDQ;*tYw zy~-;Nb+p9>Mg@pbG8P60q!3>+^kx6@!}q8L5HP!GvR@ZFK+hE zl&3J4P5XM=8EzhX4cr{SSM|a>eqXDoI&bak-PyTro2BWh-c8W+B;*KrU2diX2~kEmbR%I{dop= zx_TH~P`w{V`9i*{`*L&kMYE}qC(K^DNvYMz`04An*c!h#`@Md_U1m2p^~%NX_@@+! z+ia3RGrfIv)X0eJX{QU${xA04GAgbv+xITq-Q8V6;SxyUPM~lN9$X3!kO0A53N1)* zhu{(7BMtsg%Wouy~JxmO)?Tz0UDAoETU_$GFBRzJH5LP zL#E9Ik6P;wQ+b)|(Q3Nh&b~x-|C2yDX7ii}+8OgP{R{(1b=#HE)2W!lN zvN!2v#-~^3NEFD*z1v)&GR3W(E}L6XcHFD%aB~nhZi_S(v{R6r_*%+KmAR(jj$*oy zXw00jWZ?6}tB~t=$AX?ya0Z*yqw_rHSE`pC<;X_pZO`k|I^B2d$d?EF#%@B1s7W zv4PK0{!SO#jlUi$Z@9M7UAF9%lhWAyyo16F>qDV$pB!>3!>S;a?ZKSwZMrcchZ8drv|&tLqgd*#LAEp>l$_(V$C zK%HYA9b3tygP<7iVh1M-q*>m-qKg(P`=T+a%1g4P-t@EC#K?2TbZzXBWb=kY$5q4x zT$@Cld^%LakQ|6trK(4Oj>a+UKJ4xZYA}Wy@i)9?H_2x!z+;t@wyvV3IpUg%e>X^) zLqZ`hkFdmBr|rLY%MfU}H^D0!guCf_CrAMU==muT8y4~m?c|Uy;#IwIg5tAnR8qFJ zmBnPaC3Mv$$mjx#>ar~yO- z&Ds|c`WB#ifVB>l6-q196vzi3;P)XbmT*4_LWcX@Uw0O0s5d&(Ut)ireTMp@`bQ<~&LavP7tz^3^xn?jUYFS)#0TFZ#LYoo>bU22-$ zO;y#&N8CCd@xaNpBNQxg)4otI?S0H}RD(4YDE8*EF{X6b3F19PZ|{DyXohA>ols1M zsm(Q-Xxr;tNHN8Q!@}9IB)QB549SNpIR9O1zi z!8VGQ>YCn{nC02Tu3@0O{ANe&O^h`x{o8$rLZ$0 z&0-f>ZCy4UOtX~mSWHH+`eyLC$F_3Evr8A+RkC##g3xTWu0kietvi0iGoFhv* zx=I&J3JRrhVriozkP-(=>S+z`OMw+Rnhbp8h?P(G%_Y-J#>gJ~pck!tPa(YnV+$U>kR!4?7_5vM) zv;*Nis&&&R{Kmd4=j8Y>2;eq(>S}Qy?62YyXt9?}pY$Dm0s=~UC%C}lU9bPXilebv zcWFD5pWRP@*+K2@Fit@5@^amCS(Ol^%Z+kt~Nc zFa+H_nz`d()A&c1*4Hp0>_$Qd50MPCfAex!R?o6O{shFS&DX#G`^uXJ^I2%G9!%W) z1Pqos8wdQZjrrRqoHbEaUhdx_0iR&eny%lKF@Kv?v@Gm&=IZX*B@eCP=5IUtU2roG z+~;?A^X$P_XcF(m?~<6m&8f{}{U5fHq3PAren|zR2^iR%V0FgWGsN;-tt26+IEI&J z<~1)*ST2y5g$$Lr>}xz2T2u(Y`0JwCTK=!z*x%@f{=aei)n0PEy`f?IzW4BxNoQA$ zwyUYXUjk#Pc82pC^GPEUi^^r52yIg7VldQzloKgLuAvZpE;d`p*jG2@qhShm`wH89 zof%lr!vozYr|=^$=t|?0*M}7$aP)@wG&14T5WFl6>_#zP9W@(O3Q5orRam*rL)>H&}|;Q zB1K%VVKb4KbF(92u}0+qCH!_tC6104l4rs0Vp$#fDM#k$p3eM}CRl?5!6z`vai9U$Z=e*_1(FE?-P(z|nr;CPSjZa6Qt zw&bwk{s_!E^ri-VjbD@J+ylueU}K<2RiJ$93kV@zE$fhx}5NCZZAw4PsPj)!Sa0B&PQ za8b22qW+mZU1cwT!}Iz=+4RY>Z3UX<>JdV(R^mJYNFs@1>!73_N@#n`VY;L)1`b)k z;kjm!0D6Dfw!8_8s(6uyvvD4(q{swOy#cC+3D%xutP$2fJ1k*&^CkDZlf~0>BjJ1A z=Pf5@>slmHPrEgJ;EBv3%5H@j5N0M8bkxMgWbve27<{+!yk*&JUrR6VX{07ZGK2Zn z0a8c?>3hI3s1TL`F0%S(_M8b$^lpJpa*VL0xL1Ie=dBa@KJS+CesNp5d5>*sC^pt17J*>!!oRa&e?V+>cqL|0C7m?5mGAujef{(D!xr0Shrr#K_&NrAZq=w`x`}U z?DuCVNP^MEd}dW=ePEaP+aMq9bkLazF(F)axFgj9mlO5GfR3}cV!=|?84H9WbOc}p zxL28Ncx=aX)vb0lZ3Rv5l;^Ur4vpZCUBYT}xO1=b0Yi)BD6@U2wAWz-Zb*?jvmrq> zWX?eVt(RL%%C|~j+1lp z5N%q1?hmmL;`MrrZSwC>!MM1v9lA(dW_@h8-wM9|kh?RqC-Ut=AV|Jr`e8YxXh z%^BN0yLUS;)H?Cn<{{jopb2j560ec!CYT1Qj9M1H{e$_c6sUObzL$#1*j?rg=C=tGi-md>#=6bs23HN5&J9GBB&CXFbkg} zQV46l60SR98CjzkXJc_8mDrdmRBp{{8O@DTtxsUTnzh1b-gKBh$QtAtS1Xh7bl3B7 z^||N@uTHwZof><1?3}iwwFhrp+_R5{BZ=mCB2!F9a$L}U5Z@xo2 z&0Mdwo!_QMZ2N9hzuQM3UtYyASG5X-5pDwfh*8!ny|hKRt&utcTFl_MnC3meIn$4R zkT*eD;(KW0&{s-!w?vm|ceQ0A>s@}T>x%o9VqZGeOWNaUHg?4iv2zDN7?PDQs&hb{ z7XFJb_#4kArWKiNzR#~be`>#;)+{5g$13aZIE*3_2%V!y<#qyBwo7Qmh(~bBgAQ1n7whV-!;7zx*-etxyqK$oH(2hYY zhV;&izD!wlEWR9lyzs(9SyXOY+F@aQY}hAYDrM|$X-xe7&XM_oPudqc9O@AqR+TEz zC)lj-DiHdjz}Q*5s_FDFgNY6h<5qiquARp~FP!~mMkL9trk#GRy54BJTo2gB8mCnp zJ&UD4Tn>zB04Ra@gkmtT^|LMU0W6i+$RxnKr@Uy%K~^S^nbc56yGc2O0vRJ*W5$1a zIhfsO9@RkuX7bLhTHBVM6}w%xUWYD{P=UB&I}Xz(z}$%LLU&otwEM$u$MAexAoIM| zGG81P{=UX#Q{7O%2I8biw-h*7VM2zERNomQCa%vZLzxyVJzITh;wBiI@8cuxb-k7z z1}k`+vM)~s!3w4(KnWOheq$1rj^_lgw2xIFtl)MwgQ3xHRqiidW^9Ch)MTU7)Uu>J zi$71s-x6TgwnK_F&>3KcTp5ZGqbuc$m#kH(bf!bGn66YtvdcRBKyn@F-ZZjwuJz;Z zdHwN@8ldUYNrU4(nOvjn|BKoSvtQ3RtOs{cAGDu#WLzRer>P+30mU;x~m<-Y$5}44cmrSBLiVANg)Vs%@0E&!MSt z-n`wGLnSYOgw@KDqP{!@;X(bAx+L--4D?{kL05(+R$rdyCw~+`oN-t?dcd9{o7|MW zOdF2vSnV|oY4RUd*{z>ApdTzgFO1#=jKOJCd%rK79We>MVqB^4@E1>S&hXQ&XhvIV zJ~Cgbr6>iL7`~0ueY<-3eXVQJU`Rq{VnAV~#muA`JV)R>%g{i@bFf9a81*J8Ad;Cecri)eEhg9DXK?0&;V2!r{kqmj6;O2g)Y&{0TVuJ zbY#7d8EQ!k>BcBL=bPe`+)^)Yn)$+PSGj+Ks5Z?86>BnZ&=z|G4GhqtaVdZefq&g* z1seAkWI#aK}UYaytc$`K*M)v=f>J z`pU|E&VAu{cEebM_$qRSXR05c>zoq7U-U`Ei_6XFn2Bp+>af;j!Izd=W>e_MwVG;o zUkX;28!CM0(*kqqGcd1p4+|~9`o?9EvcWGOgKqpJ|6GuXhh#5G<*S@~Wcn}c8qwdf z2bV82e}Lw$jw(*Fxb}$+3$x1v`G{GNm?X;QFcb|LqHA<4Xh~2j5pC4j+ZzgWUFK#rNp6DSkXWAKZK|gq7-z-L{Z`t|O1}QN)xT?dUuC|6|88R}f_TM0XEz99BfPyy zF^{ceZQqPmZiAZE zxqR^$7ez6K6wh&gTFpPnl~wm|i@b6nQZ{9$2d-WRX z`&Pk(3-Ez66A>Qs>=l72doix0m9iKJqZ&Iro_UFB8ikmc0jsUv?DLsGNno>417b#S zecXsLc{+HujK?hZH#yw@LB{$+ob9(x$p6hhP^QGO?i8MV-}ie~FIy$1;zo2E`4jME z{NyLVA{gv)`=<1PYwKNA$l5otk!$mN$0=jD%NiD`4NXFG!k&SG7g&$&=wIR=?AEKC zz=3|;*i=fX2J1N>Sxd1+5Y}UZULMUytd^Eau8Eepk>g}3Q2y99=_+BKB6fYW{Gp$K z4EQ(&YqfMZVHK7lVWc>vkWt212C}Fkt+jSDU%0rY>%fwtR%I{BYCqs{=@;KFN`bw6 ziS2xGaZBY4weQFvFL~2?7F|i)g&Dw7N52t+0<=NQu1+7sM+qcIpsSVI(hF`s|4iQ? zMJI^P6IpB{|Lp7Y6#N~_o$?r-B*WJY7PPC5>jy)|_o1(f@F0xX$MPK}CNt&6eP9nB zblMKBL$$yWQZJCI_gJ@tpDAKj;OcuFP^uyr8lZZS*>@gq%wv^B@q;b1G`m3e-H(M> zvW*3Tvx@X-S`zZK-RD8*i1O#CA0s4%OAQLKSbMrb#LSwt8&6nXsQLG7B-LZJ&($AG zxL>f_t+1*O(Hl}mI1?GNN$}aR$F}ZJ2Zq1G_T3`STm~kCGyRPNwx)+C_$Kp_6{T&k zyX))_gHKxP^8!0t6a#7ZFrN3J=16Kr*z2lSh5?QY)uf|~Hp)B0!d&a{uGU{i7zixA zrBRdEq+?E4UEw$K*p0~Rlh+=wmkcXyQbhycC=sHfiW%C{IgU39tH@&jpET-ES1^u> zXSqAq;=H0QCYWiJ7%S$1LBgJmM!*L5ne?sJ*KA;AwOLAIeZ*9Dqz^3w_Sv#BT1$85 zEStRM>JqID$I)3-6C=t{sX=cVlmLqIm6?&?j3i6aXZL!;oi^Gt%-r2WCeky}rY&w) z>w~B_oSi>gkYFBKr%DPHP*7t?h$ca;65E^+60!>|3|;10!H%PSqnEC|-3hM4j;5kj z>wb(~o)a#^)cqaCA)8VGqbhBt)GLB|d{N8VZ*6vsPCfo5S)vn4KdkLg$9$pt(MI0io2#96l|E=rB7~J*~z_p>V zklE^q!5+OIlj$~8*3SGx#arM)2^q5tY*L`Gu?`8}Y3*R$zL-VI?JOv2=xvzwUaUMl zapx&(D{F#2S_^Jpu~}%T513TW0zZ-6hpsD=i`=(0axEz+uCov^DHksFw>RW5P}{RD zEsF65ZKQ#UnJ5cT82j_}G3U~zVmC`f)grJ{MR}c53z3~SdjJ6LK}*xrkVeB7wzPr40RTQ7)dgbLLyjB`Gza)ep)P!yIfoyZslU8oFZ{jo`@_MZ=CRD5Q;2Z9X&H zmWq6z6AYNZ7`(#R0rrJo^ExD^h2iFQiRW(?>Me?r5YB!*O;n_Qq~m^8nngVnkXag; zl;kGnwD!p-+C@fMaMGEb1TCjzP&mXI2pAHMun~GT#WPZT>KoNfU&ib2WBw?G4R7ke zUsx~Ebz|(zihlaC`#m=vDk8w&;`F`-FqV9p+?8?rB$7K8^MVY*AP-p4BjygnES3gf zue8?q^lvRMTE2CYJ8UG=CJh-H!9Kp!)h)DTa_+;o)L@pT1}LsmOymr;)ZZVH0*+q~ z&5}@&cF$8{vN5SOYBy&z0?B3kQERe!ihamo(~xo<&TWE#Q{5mZnd`&;udEz0ggc@f zPz2jOYV|H&^*ODh#z8heB6@x1=0Sb^Sh@^gynaqO!-f){viI|~%nglekNPv$=BFPe zLen=g*1C4Jzv_FOkGM^5wta58po!c|GlRqJQ3b_-ER%u^xEca)2nuDMamGh4u4C1d zcg6zFPvZsiB|93+F`%P4*B)QpT_&Y<+_@;{5XefACY?YW?$LNj1AQ&!My0N4>+MwS zqE^6%eWn!8gc7zQHX7H-q}uB-$MGc*bM?K9SgKonoGA1(2!gEuih+hY@vRVjl@*{* zt_NS6*5D0XT5ufEhIwsVaJ&^@qvFCBH>R-cWr*=KkbPm+$YdWQnMNZK)tT2U8opLX zdCoq)TAV|pgkWG+)dC;U(ae>ANV+QaOct05uV0%F zm`Nd%0>$tlyr}4Fgmf&xS$T(#ibZ=j`hCMtfv>`Szo^ZItTZoWJr!GZc`Z|lrgHHm zL-cSE%jko2miU}FbnzGi58@Yw0up+Lb+D%o+zv9Yy~k=Lmgc+sLkJJLM4|;73#J=7 zJ)YY4v`l5(DEM&U`uH-u>hlOy%MJU!M9@j4N0iuyvH`A%-+A6r_|{^Dza;PoK1jH~ zQGvCdr%o;gjC}S5Yv?ki>GhL`JI5ngMuzUmUP+&cn7AC*&e?WXVtgYmEK4}`rFq$f zaN;4!yG%Rhh{|sjXnBjR=aQ06-ThoVkr#~(_MCRhEp;p6Zz}4+8Ej}I6N78+(vjTt z%*<^tr9MuSbCjBO3;q(?D%l?!&xoFmwC;Anr>Ra&r6ud14Ncg-c^0%Rp0wuq)_>4p z-$l*mC@5<%aBK$q*jrE1Mc+*q*YFsQE%5tt^ z#5=M&($3^sJejso11=BPM3oXq(W<|WiaXIrs{|(0#ghQh&{^{c4r@jd$qEsbxy7pL zT)bSX28*WbZ=53m-~F#oSS)g55BKdiN$WSiXM}DX`5o@(Bo!Naz+}$_e~c)A;R(y?cF1ME>O`{_MiS|X4{zQ&@d5Wwz$D9a%M$NMs9DQa zP1okzhfm3YnOENKk{-Tq-(P$fJjXRAdp**DqP5!G-&pVSmUBW)v$eCo3DGYjiK~qS zQFJqNp%dn`t$5}8Evz$23a|9qz7L^snY1&ne^4n*HBKf>Sd8S9<2Bco*m1sw=_o?` zi{31e5OZUvSZl(S6?wEhUU6P-8f1@n8Lvs%!~GZXOafNk`G4k1_KM-Ufjv$S8Qg4j zQB&MM)_ejHsHu2Y6NHW}NP#((fuV$f>ZXpu3?K(65uKCNnO&22O+O6SW5gRYUMk=} zp(Wu#RZH^^<=*DNzL(*#rU*SW(8bUw9$d$Z>nrpjswF8h?U1ax$mCN_;_LON{z4$^ z!mC5K6sv79+i)yCU40oUevcu#Q=7lJ$?`nF*!9D@Zzf;FRTkB2TcCSXJj{+u0zKQ{ zfSrC4-)D`q0@5|7jPpi*fVa8`L?=r=nw3{=GGD+c1 z=f<oPu zSAUlw+)SM?VmDGr7$Oz+q(dw?^VSC24RxVgWQrzM!{|T2QEgKrSgkmVR%urK9TgN8wCxBSF zQN%yDtSK&IF#%=#Y_0@_tKvLx0WwRrtcB`-z(1j&kJCFX;o_@bAqWa?`|v~ou;^G z>%ow@K$ZtTt~z^H7#H(;g^r2V&!1I45`UWH{UdAQo86T{YkD?2KYyaA7HkWzD|~Hr zgzJqI{4}kz9{G)oiZRDdAzn^QWUP(MfUhmx8wan9W*h6#?ov702uJSjg0NYhne7jG}BtTSLq@%b<({`$#l!rP7 z4{rTKm7h>$A{^I-J4g4?9rNxR$?U+YqH#Mz&sr+XHk!7MM#C8nz9R)hOoX(qhY_X7 zapnhdSiC%Dy}r~uBXBUEm;?dwB-k_xkemp4eX6D7ytU6T_r1(Z#x`@@xGd_K+P*W# zp@HNB-yQVpPj7;G_L`$3mDvN|ZOKdU6!vw$OUAFF;6@pJXs}iFVL3%goI$&5cdt zAG!GR^W2d!UQgLu3RyDomV`iL|7xjbzx4+|IU(u0Tyj!mG$%~~CM>G-Kdvuirae`- zZF^OiiRTwE3?}%ZGRk@_HbDV{zbkJb*7wRKWadF~IQvlb(89u!3{9ffnrgMuQsuR* zY7IVCjWiLI!OUDyU56{5`lyoh3gRyXA|z+EG(L!m;&snX5q2#6o8z0ODB}4jb~)U1Oz}6Yt-diJ$K+!^|HE z@738awc~m7P}jZ@^a>0ZOt-(Um=U=bC58LUK+!)+ce|*S@NGf^uRwmsO{2vXux)N8AAv+Mr#_JGD4cVQNukEe8xNuUQWjW+Nita{ z;a$6u1ZKvq5V!T#-N3N7$W<8vCbM!Oa&QafVLcl%^sT(1^>XUQ|SZcVwP2P6O1l?DMJB__k0Bid(u?b zT3nn%+!5KF_PrmO3eOqYq#;50fCHNi%Z@(c)v$ ztv&g9=URa|T~c9w{d;@U%+JQo0VHR2qRJ-a8`=xo#_u;ZdV^j-k2%w|d9SR_Kd)U# zlMEGkc6F|A)#vfHG*ebME-5qc+QZ7dNu)AB+5@c7D^AtuXmP&WIT`?pj{n>Msnw+o`i9sXw+rn5H6-x-9BzRZFO~-?XH)uDc^0muSOT%yyMzqv~Ms zv`x{Ru<7YOU1kDF=p+&8F?v*$qyt%L$2>YYkFvqUjR;JhK~9JmM^yUZT&!(kO}N)* z@iB_=Vh|UJF1M%nd~CQMgDQvwaHG<(?EhYWI}Srmlkx|cQ+hSE%|flhNu8!p*6_Oz znQ}Now3XLk|dE{)CN-5At_1LUdD)GLE!5YwgNV=O{jTZUSlM~ni_&Uk@K-))$ zw*N!JT!$UYrjgyCpi+HTH}j@CdlmjVP`|D@FZCOFWhHKP=nw(XAju*KP2(PkNe8Yc zy+mj<|7kJFZmr;XWJ^K>Z@dVp2CnUBT_&)tx{@6ig;4%B4c}l;jip7p1d<3Xt`p~M zugl!n@)^J3hL5eCJ71rdPw#frPi`mFKK7{GbrdNR8-5uWL8B|)l1EY>B@JT%+k($I{!Bzm|9|9u*xVj`v*2XCR zi*eeZ{lV3c(VayX@C8oCN(>afJ>~#<(2l70e>lE~d#z=LqEiLtE$R(YT#-^##5`A2 z_YKXY)M{IDTSdy5GOi>KAs*BHD6=x~*YEw>9l$T~r9j40%?5Ff9kortgyAQwu56kX z&=}@mUa9C*%*nckK`KNvtVmw6j#lBW$x_H%H*Ck`snC$%!bO3BsT~^;Jff}L5{6QF z#L%m3*qvp|%5vEZ%@OI=WWKOjI38IsV4ZvDz`l&i-BlP_WXLWepyxZK)h`;f#GZCwuotvR;Rj7+jKUKjp!o0b7UXjiZwalM2CmD?F4 zFa8z#);a7B3*3_4Xw<ma;{=&jo_71;6xX z)eehw?tXjD5>h0>e8K|;GIvRjiwnCRx9mx2(bCK6ks!Gp*XUi52Ex$glPgb#CH0U1 zjL?iM%@}v7vY61PSe{_`AvWY$Zd(x7r1pkUX{4MLw7r6Lz4Iwz;;2Y)3?akade||{0V?lhUci@Ph;PS zzkTc^8O1}Gp}U*;oJ-KqBCr_jPZzDv2iHKwTo#Zfou8!gS{I5w>8blp_V&kg zY?&9K!XOO_y|Ml3$k6m-$8v#=iSSSGVcAq(3I4E%awUD`0q!;v{K9ZX$t|1@&z-PO zF#F(_N$06An>#s?gon#r_$p=cQYije0^x?ELGq{<5a3JY;G%!JYWD%)x0v@ywWM~h zsTn4IZGF@gtP}7o&v)v4Z6m&(A;x3Wp@x)?O?fAdMk49Lxdkap-?+0GA*hk3VyK-Q zXn_J$lP9@Hrh7&pMvRhU%%owd4_tZ6^>}{uCxES6HV$`BP*vb;O;uU>v2GkEXe$pB zAH~0J1)bmwGAob#-9NDJHJQBs;>^6j0M7< z0Iij`Uu@TrQcfbq4&cCmZ;r2j0=g(yZaVV1_&&Xk`w>5bcp5tYtj#QVh)3fS;eTl}NSR*TVf6h~s`9n3Dr23=?CWOuAt zZsR<3QI}02^3K#4Itk!r%FX#Qd@rQFKxZHAqX^^Iq4$|9;zh?qkEi7-R6608SJ_4X zaNvUOr;x;jwM`@V~^EN3?(i>@^( zOgv9$3~H_ZT<7uPh3*3p;QK_Y8%r@FwO~2#&$K7IBjZ-lhOV4-aPv5|Ax9dY=WkxK zXRz$h=X0_1jOu?YrfcDiAt0#qzqJfEmHl(!3ss21r;c`Sse7(EGiNPX+bV-fP85$} z&sPt}zERJMBlv3vwZ30l7=L(ds)Bn-XoZuhrLXHB7Oq`|juggR*d@SfYx7I^ZAsF9 zZm0y=&=|aZkWy(W;$>ceoR3fdFLZNx`LpkwVOdVAjCeKou|E5zNNn6>9;yckxpb0@XpW8qBMpSC`6}d_(Ed=H9)zfH0KC0g~;rHi< z^eUeyu6<0YlK%g#^N+@9E6y{{(j2x=^`0 zcGu<5*jcDq*|+7?iaAk}ukuA_0mCkgsn;R9Ct^z@DbOAwWB3f+ShtyiCX>0&rle$L z)9GMgaXh`M9x#!8QR%5(6{$LVgpxe^Xr40^TY(sWERb>n0GNm|0)N}>ZzejKDUwz( zr@ueHjhSbq-M#X;|`dY4R@C9XSDGv^}dXn*Wol8}} zf=Vq4*QIW}y3Y&%(0xQ#Qf#Vw@;y1FsUT{Z(my^)pkm}vosD}?K`Dd2Ews#^J_1_y zpIhDVADXSYhTtbatz+XB^!RQmyvC@yR^kWiq30#a%kM^*VXe;pHH8@ek)@)jUlq&# zz;bHp=C|m8th;haq2QOO9q4JcTS|N+psB99*%e#Fz|F`wTZcgLG04a_3#ISB9M^e} zfry|!aokSqC}%xiut+DR@|$2nKmTYaSVg8@Me3$x(J55isFAC$6o!-G-KT>p!3k<) z6;_68+Imk6(x&SO8xYR!^X5C=OAUfSU zuJb$7iU>b1;!ec|)XFPzD;ew-)^Em07b}ylRs~k9do72>KD^41kGk<#*PTrrdF-t+ ztRWmACP%_M#(Alj9YvoPOC#Z0s(><`mZjE*%ZXBd?kc#_3ps1&c3mGed)qei7}ORA zq@pR)xCB^Aa8YQ43D4>053*<}3W8Is!k}85LPl421Kw*SK1CuSn@;_m>eXOk?hcO< zs=iI_Ju=)Q?j{wIS$um6wc&L=l0|?kPzxLpBs;7d+v6dbxo6(uECucI^|-ef8`Gwyz?2$u*di%FY-)Sh=>t<3 z*mkE?;UHt@NoIX_0N^#D9uOa#KS+WC0QxS4dql$)X~I?szF-db2K`?fU9yjDU~PpnAaOp+wlz|ML3UZAkaol(uWCs zmXA*gUrMk@)x74cqE>uH#9; z;OytgBgU3_qY&4MU_~YU2!XU)vhbf+eeT@5pfSb1VYTp}bTY-%Xk5ud1!O=ZP`P>@ z8I0y2`}N{_17}GbyK70a8*(;{T8o$8WdZWVh(X* z)8j+J$Vbv&k^n=AFlfB3j73j#cs_XutMz(?!M%=wLeLoWIaVdj<%Q^U=P&?f9%1o= ziSL98Z@=8vM)VBH5V29v-cdiKuEsdzXQV-s^zb(1K!-&&J%TFf8+0N`0xcNNWb>MQ z&u3ybZCxEP&L|byH)j~S5%Xy$)ZJ?s`Ye+$30CbWi}J;iQ#x>R7=Sb9QFhrON{)A3 z!)Au>IV|_%vTLV?TH&@a&u}*chj;Ou;D#7;Ply`^o#_Ul7lVl9h#|i5&aNinXOybl zHd}Vls)^BgE>0hNC>@n5}=q7Sy`n$doB`quqmKqPOAs_p7t#T|J_YbT!Rg za}nkygEooJ1F&3b3@MP^+}71GWTl;j%sx4VoQs8)Xdtobt4L`u-Ufrt^5H{f4!a7f zd8Ut~(#nBEUp};=qB|ALW;jLfO=jIqeP(F5u`x4V)<|m1&DuElxImif9kzU=JlSDv zYm6zJ!lsc!Q;kbeJdpb}hH(+J&s+(hXs)X;(5Faez zld5&7`Se`Uk@Wm)XxSyCS?67^vZIp1!=*Z=THy`u_wF)BFUqTDvU$?W4O;zzhuiuYddW?|heKbVJ6q zX*3LK1rEXNfG^WKZ4cPTn~yV8+g*$>9rbu7jZUdSw;urLxYE>RC@5$+aVDF$RUXa= zw{JeArpzUKmyPe+YaXPt;tq0F(UYLhYerOiWb@WpxG3g|jy%8Fg`;*i3rr;o@}E1( zadDgAf!az1W@tE=2aaQvPq8LmQG+F<3BQK1+P3?mVk&Lm=U~SXG`N~exm|(M+t=1V zbe9(kRPNO=H*Qss1r1$f#BkHu3KGJ(-G?YBgm`nx%d4xZBJo-DRqz+C;>L`i`@29M z&mymNqk;~1ZH?N1u_n`aoiu~#Ot)NO32FUB3jDknwL&m}cpc{MhXENxNU+~@@Nj8d zl_z)%_(d96Aiz|Ss-jS5MpW$L`W+etl&}%BcvY$PW=Dg2QNRTW*5s{{Nk9y*Lhrs%Xy z3(*hnX7R{ER2D5eyH9FpEA|Tn3?}+el;5L9sHe3j{3g9q6( zb=tmCHV$B3);VmS+wL*Y7wS@0ai7%_)8HZ2JjKCfsid;RY)?VT&G@uYOjvK_*VzgB zdPFLBWnWQ+D0rBYS(E1@>SEJzaFhUj_gTp3TE5@$>O~mm-b-wEfCP8GUp$YcG-hWP zg(kF6Tf|7aZ4_=gNS<5VqlMJMQLs6*jS6#el<;l1pKO@Va3Q?DaCF$1EI*ft-+K`k zEY35kiSJ`{mPga%LVrqvk;qG<>I98M|HPofY8xKx>N*^N_10wU;vQVev#>eW|NWp1ts$dtWi)|UJ(Wjn8gSU2or zlP?eMlmUzRbA3+MO%#-OD1Bh=@Gw6Tz2`h{d5=x0BM;r&tkhK;M)pPthgICDE1!Ew zml@`e$MR!(=_&PD&1wx`F75$P#BNA9*IhWE69TuJ-+ebZx<$p`K=CaVSEaFy?~8q9 z98rNs$4_4;aPqCOH^fI6hXjRl?_ByyY57nO^zJ8BO>?3!plh&ICdJ?tNJ;lA3Pm~> zoRJkVu5nh;XcoohtK|X;If-@AECFJ(s8H+XW5f3YTp3@U=6o4E>!o~&{o>)~H@mlF zO}A!x5d&q$y}$h<Wr#pVu(>w|`Xpb_J+)L)E`= z$;Yw}&|DR};2uG3BqUm76a}xJ?j>!sYsral7Y*Vn{9t9K(6vpNo=F-r^`oJyXQvG1xomI3ch!nbd=Cd5kKiWwc) zvs)sMGq>VX;~ml*#h-wJkZ*f9;SJYV`(&xYl!{YExMy{$8iS^U#vQ9fvg(kN$L|0Y z)z)sd9BY*xhLBt5`9vloo_}K-{|7Ah-|LbFPv%GA-|h}Wm0s7KrsR0ttSF5YehlyR zpXS0I-srOVw4&{w{E~Pvy6|tS(+wbx*zLNW9w#+cyxM?>@8xnciZkpLW50f)M*<+% z1%@l5fpcWRDbA{Uuio$%Ah8yX=c8v)zpxe*0=WG_j=xY8iEBVP zB#HtIi28-Bus=hhEhIct?fHJ8E$T%3aU6f4EhhJcC=oc{$`!DZ*PmZJ$#WikH)N73(Ogf}7a2aaOUaV+i&5=W7W z&OO0p=ggDa1(9Jf{{w3=uuJsU!qJW06{6bck z${^7eQyXb(xWCXAeA+?^roYe@B2rpJNVEm?9f}Df>;n>6!S=-J7qX(_q3`!UkQEhp z`InOBpU*bB{(`I+TSek1ND*nHMt|Tagv=-+uBLwBC}^5&LyEJ3!^;i$mU;I3r9!} z6B28|(}(&CS@F;8ITM`Gh(u9vbg-q?{ehyeZsEh&hy8(~*b%WG{EkFXsGbL|fX|(sOL4|9C!4Yj4hU z7g@KkTdasPe||TMq4pag;eUSNR5|rG)rXIAd}Gf2dzbFP9;m?YCLDeMZ&);Jch~e53tu#*Nc?bSg;C(!)>H{PU$+i;` z;rtUxiQr0MuY!nyedRaGP>&9V+9Q3SMkTdBSCJ^;l%n1_Q+yZ0_?(&Jmgx6ODZ>jw z8dux`5{)9k#OHLw^_~oYk!)S2| zHN|H7e9gyZUqgZuoQ?d(&c7`jez+%X$d$hD!)6kU(i~j89t#T&NEerPthMNL+Lx`z zVDq{eP-W3F7gZ%yHYPqCQ8=*O+Ldbgx>R zbwHU+DgZ!YV{A!`T4Z2z|k_u%eYKyZhU6b=PNDBOd)J0TEUgHt#O z?(P~0?!n#N6EuOy`PJ#ZqjUO>b6)owqu(2Mym#yGHEQp<=3H~Fz4x4-Zv|Fs`6rfD zm*>rFM5Gfqx^fzSKrj&wEd`=E<37rpmLY%0R?Vi$n*(%&jEp$n*$ z=+zRdgNPNeiBS&ta9}H|*U09^Vo@hG2Ycr9ne`datTu&5qS96skL&Pvv%fQ@D=$gg zfX@)vXj^^Asi>B+Kj&l^$qr$G!FxD2htAU#v1E|Z-OX?nt#%BjE&G!}O^17%JJ*Xq zWbFvDaQY`Scp3znanK%_$b`J&He9!-QbV0@tL%$+;ij%oXew?D8}2r3HVoY)N;Hmu z@wEC}`a*0doS6ig2gYD(FIs1P``po1n&bpmrykBzu1Q2-+`@HC@VICiZ@(h4O{Pw2 zC&Af_C`l$bsmLW+CumTmz1$)chZXlI8{CUTQn62)bi5NKR?I(h`7vM9>A3u|K2DGP z(I5lLqaHDzLc;7G=BGQ4Od{=o#dZUSS}ApT1x{-*&)F(sR~9i+*;slf#?H8we6Q9C#KJwMt_=*Mt0@k9c!cCn=M7G8?4C?Ga0X4M(FyWe>I@8G3Qcf>h9_gtcyzEG*~!%Ts0w(vqLI zPaWB*mv(r)wE9ph;1T*H`IyCFSxG2wOoH)5kq9MkrIk@0W~ow_w3FV;vV(p|C51IO zIkKXib?5FOoKx%s|8&a((UNf4%y3#6O8jC!yJDxbrs49KoR1S7RbG-Ey;MYuR3{sw zO$_UOt{a@5fad_OT^qB0Lq@QrpfG$LyP$g4&ZDkik^CoybZ9qS=5-wXyHNW0NAdSR z-*@=yPaBo6XfBLiZTuv;5%7*Ik$?B|3t1RvY87??H)MPnHqLz)rTnz@3R$Q8JsA%< zP=6F88%PyAlA+Fux?L!x*2y1la%5CcG4Z`1C9TI!Q2IWuu6l%1Ro->9RGFk85LWv* zNs*T*y`1{AT^5{gq=HGHIwh7&FmK3uko)$_bFYPN1XXMaBqr4M+Tu4ryZ8Y-mWqW& zfOf^0R;uBE2gVdWnZ=={Zo~w@MD!cT;=q82-xSzhv+ERN<=oK{6xuT)uBffD7xvQ2 zM2$W66j4FB6q1KN&mSm$jw{3m_L(zSN?h1WU6{D#NNm13knwZrH0_39nzK53>(?;4m8d-m2yB7%AU#~ zfEN&WT zo2=^g5SwUSX@$ZiDBz>TEaCjVa!)ueNdrX-(D%~kYZu|cqz*0 zsY*GqLG&mEBM5z|3@GD@MN@FBbx*rwU5Gt8p-ZpHZX3Q^JG%!ImR1&C%x6BjGyr=MB4A!_@U^dvwppO;uQJPeRh5z~NvJ&&^#XfGW2)1+dqS9$^o6crAxt-zlNB@FD-y}L z*4Emi5z5we@rk2G+flefB+ZDUrC;ZK*}MI1Y&+xHd9;9eBb>@i9+RwJH+)UH*}pt% zhTja0Fna)B+b1%7Tx-Bsv~BF;y&V-P-=M7*F7ur<;PNU_r&=8vf*VM_t$is z-vF-Q_qM_n8LMs^N@dGZT|Vo4OlAIZeW8TnP?<$soVHnH8^?SJXi@JN1GqaQD@ZDA zR0q+dRXy2N#~kAckc`cy0Vp9Vfw0ik6qZOHUda9&u)cWq^!V8-m*ie19P6gb&Qqku zpXlX{v*^JZULn*Y?uW->y1TJ=!`H6{lI;tAYhg&eR)nZtWbD}=ixHfB(*C50Ss;Rw zFN?|=@|`n>?~#p8uFK!>g3>{+1w@cSuv4l{h)9fb{LGX{PXK;YqT+z$Is}M6H@=n= zdFHlkZlmi4dOF>?o8=;&C^RE+J8Nt@;k)s1ciZ9a6b!Y6rL|$9hK@61opEc3y;C8v z@!*PL$tp%ZbK8ou>T%FHuLWw z8Yw!H-=82k|J4NzyT@eGpsOQftV>Q0%$d#>|nR zEb{z3RQmNkR4hhx^?|N%4IabkVf~Z5$6+l9-gO>N>22FO1PK@HpC6R$;`%6YIY2*M zOD0f2rdkb`WK?*~A%^-jCoVPb@R&kI!On2-C*Qj*8c4n&L;RA*xJSQFCb=vQ&{^%D zU#@6bX1P)CC&0Vt9HG5K!;_L)_=mM_0&#drHZ(HsHV0JNhTxvJ*I1z(76iN(G+08b z)oN}0FNn-vWgt5@#h7}?a4|@-&9`CRn60Ihxy+Ep5-W|s)Ks>5TZ4k=WdVD)Ux@J% zi`4WaL^rkTy>18ai{4L8CbZ0^ySmd-d>>b@$dYl1+s+%c!u@K7jvm9D@AtekMHquh z&+jVneSSTB#BPPOB14<>Rj3U5V|~4ALi^F{LN%B6C`BPqqjj&;2uh)D2~m3ZoCYXm z4qZ{boa*K6n75ZzR~0GtN@8kLA{Xu2Ne$)p*fFu80>XI^vo@7T>Im5L(I`zI`z#L* zO1RkVS3DUm6=MpniS1X?5UC-fPX+pQ5RzVQ{H68WV$5pqX#6OktfwNuoT@pUixc4# znI}g`laoXL2G5h+>2a1m-<4#rq8UlUp00#OuK7B1ps?0XZy@=YOFw~HArVGdgX_l* zsLNITtEMIo3r;kK?4l}MbVqj~)%o7jDj~soL*_TK2-zAN5swj#Vri`6Lx6fDh7G+( z6=&)NPFIq3r6nducBgsdc(VCWbC$CcWR-oa)?0{HYnD@|ZxCeTIi=Y#DUKTAbaUddxE7K?MSjC%@iHPz8IZL8yGb^_L-m#hppk359;;}}|89v$H-C$`D$ z4g;IP6qOeeK2LriZcTM1&P#Xq9FUW z*=4DZV|F>#`DUh_&;6kLwbrg~mTnzkFt_*u%|BzC9Ad2^h@rU}9lS2Lx5y*lg#~8-1%sjGAn6oMT8`D#U-#VF}M*|fW(b*ATz`U50#DDD>pP~I6s;h6MsNA8t{lbpBa<@u}51{=HUuT zGM4|=yta11sOGltyy9Xs*aizT9a1mJFrZ}$(6|~fdZxQC{81C!qkmH!(993NXNu?5 z>bB#=pV=DL?j>}xI8gUw4$(}Am9y-Ua*_n#pmUP`#R&LhNOfai3C4JYlXhcVZZcCQJs9zLL_Rpn+jyBCZN2A81Le=jLy8 z!(2udF#N2-DpyP8!pLoqXcYK#UF5iCTF))W&$ArI6pBW=h`FhXoNMq7D@$p@w>HJc z#AwZ!_2;^<O+7A{@*CJc#E{F}{65m~dX$h{z{)tFj||-V zKG2>iKuKJd#E<+^F3gMu#__)0g3lYhoCS!5+mI7$GOfY9iH#hDZ=G6nna+&=6%5ata=JW!?r47l;^P}XbC5?6AjX_ij1QXPbVVi zup6o=wcC~w**-#Ej*e@YM8F=0geImSgAoS2c@I}|GY<78!4&OS!WqHKzeu7N8PNf$ zi1HE#ORSPpS7Ro4V~`7hB2y$J0CwlO|AZ}M5;12_(^$~I^p=J}GbTAYV`Q?VQl0o} z6t#=)Wh-dVDPP6Dc{-Y`$?+TD#99BcM(=)8$ulpY+QoBuMiQH?JEr8w{i$DZcS*xh zS=FyH9dyAZx_ivu0H5IVKU?n#=gFeNMpN6b>q5eue3|FltSzWIEt&8zs@LbT(lD}f zD|SGX#dSi+`1#1Cwgz}lVHZd@-{jH9?XSOmlH26s5l)WHvK?vK@gMkDSl#UHCkn9@ zBDoSFu6jlX`T~6EUx0idxVulg#q+$;BKmrV zA4w+MB)k+I^FGz;YAC%}2`4XCpSNN?%S5e$7`}7maGl{(Hc&NchQ~limW{`{JS@ps zUBw;0l-Iq{95toELIO}mqMJl&w#29S1$I514!Wjvy^41sD_P-vYOc>6cg)hxfBBGc z{Dby4z&ahW>?ZDR^XHjb`7pQr&XuFKR~2m{PDY?|Mi_W&UrVJ9m4N|OvILJj1tm+N zxm%&1l(6~O5nk}3d?`Lyty(kvrS-lKu#p;hS|>X=@P&pCd-X>hqb_*IFH5su6|A8i z-@SR1KC9!AhSC{O@cif}C{sr_f=;55N_QyC2vn@vE7dKM=n+!5eu?612|Y#8^oO96AbN@ zJ|iHEOtGN;W$iBqRK64_q(3DWRB4yrWl@3(XbzDqUA8FIkn%af5(MCH&oJ2x$E&qK zaIMiP&4Yc-EpsO|BILMhJMnbS4=uboVRj$e+IE8Xw53i=k{c9_)N5uu)aVm(%ekp( zguGG)kv@UW7aJgnm;Z3I>OaEMZl$BJ#X=cT5vMNrGkz0Jq+z#{MLNcrvSpSHXXAqO zt9vsa@7OePw~NTel)B|cB0kY`@v=)GH`EFa#`cllaZDTcJ5w<>B2*Nj=l50*rI%oY z(NEUpQu$Vq%%sT!6QgP&Fz#T0Q1@T>JoS5&&ky-`)8FdzUvt?*@FwD(SP-g4Cyh~z zj!L}h7+BEk0kFX=Cq0RU!$5WjET|gb41l%x;_K+l$T(qiCnj%$U<;tdT>L>&$Qx7P zs>rV1q&iqS(s{kna~7&#l=f%-E^ikoB(9&MaXn|bi5(bXn9x8befpu<+(^0`iau#< zaK0`S@9_Yg9hg-biL?wnd7L*{y{+_g9;H_e1e8Ut&#V4yNHdV^WWVs)PN&jRBZ^ag zcvxJv99xwkw0c@)K*XKO`#v>aHTq%ZH zdC?GQ{|ba)W(}Il{K#UX~Xo8jUEAaHw?R#%p znXU89cs-A{*Bj^++hp(e*u^>unxvLB=4Jrs^XQHr%Vd>LkE?gmk(j1)7*G{^6*D|y z2c7COtk`|f=!Q}9e(q2f9p4J_{1or;Wd%b+meGj&4^#?@Ryggjc8^#I5_aK zMmv6d7-OF>k&^`MZ=gG-nHr;jr5NALb zJ_Xj-y9sxbs+f4RL`QD|>o9dm7Q76(j9KFGbn0T{@InII9>?|e(;7$1UzhwtV9+Ue zq_p(vetJl%`GVRG`>Q>oVV>-h)r3&d#}y{j_agNx?s(3)4FznphOw`ANNA~Js$GW+ z8rM78cH`AN>0KqTsFaH*58oKut)zzAc?>t)aW<=rVg!MQR0e%BrN z_Ut)19bd^mmW`03&TFUY3Aho9D!R^w+HLjFg4Nu%*WrBVHN)Euyp zx(MB&m^5|&_6ug}R>K)|Dft7J^3hUDQr`Tsk$V|(K!?X8&35z7KJx9OX7O4lA_<)& zo@7{ORv{k8GlRI##r$@}s!8QG=FFzVL$-+SJOCmM{=@7)?8X%p?U1J7wD`H^chfh; z2riTGqd|O>cvX&77~=rz0I7*DI>d#@s{OQy!MpTnzMYtXZ2&!cV5TEG-VXcY`I?_bY|M<$}5fm!Lo&&EH^c#}pXuahe)SFK_SFJPJ)@;mU7Kiiw;>|(w&Sez=q zYm~qKZ$#`h+b}RdY+_iwP8>Sw-z~Q&gx0Z}3;OwLlmi8swVw=VSC6hg!B;Rjq>5 zy%;~C$rkGnEWBvi4~C9OiY?JGB_B*iSZ^>a^7<;IKw$d*PFtjf%m zL4!qPSk&Q+3ZTnLG(NAppL#FIBuZ`Dy0?dCrM$r1z*!prG22pilm@vz z20l2xQ5aOrefAMX@PLFMOyyJxGUt;!efBxf0yDMl(a0i4CeY2gnZM_d2mF+dc{C-6 z(hH|M6gKjN$==US@+W*}^iKG4U|Jz2AFBS_lxFJWj zq)DLLiHC`onC+(|7jqV3C7TuQ2*U(21X2Jv8KH`f*rj+ug+AFibpk#ZSysZ7`utBZ z;u7PSEr*y#_fNTc)F}(Zt$yB2P@{!jUnqPI2uTh4z8^k3CpSHo`yfm8f3UT0?-7KwEU-BgF6l_ z0ip|txyaRSDH_HRc(S$-9~@#82H~r$sxYeCB*}7z0f4KeCyb2Z^5oHj(H%vfe41y@ zir7LA6F6-Tj|KHKm}TE|3v%?VRbUdShz_U_-^wg3Hg`Y&H=<0a_Fvlpq^ol=emX>k z6EX*UCS@!yC$U__Cwk}lhLtr-sTPYKGYA7^1r>y5z`1kii7NtnPt{4$7Okc@ zBjyc5+R;V6StXBulL&tOHlOJRm3EU%Lf1w~mUn&Ya=biH!F5CS%Q?loynQ;p>-!;8G)eX>695D`8ICZz)R z3B92*ARx<};4%sH=o--^7nsYqOoz%hvTTfW(T0dsIWE0oa>G#U;+%%5`A8{K){N+HJA^H<~YT3Y_S;;h)>b&Sz#JT}NRS|l{ z?fdhu64y49h<%jUJX_2<`^hC?gRxe_b_NylsPR=YaMDcn*^Vj6 zvV5D^F+c$X=C(R$NuhpVPzh(yWQf)Pq$9gT#$P>4zF>RvUTe72;G-z zUj7et2iVMGxb8%vE`-sT(KclTYvY7I!i;3d+Rir3`6KM9$4} z$2xlARt}*Gj#hH!lkJfPg256T8Vk17ZJsZt&p%9eZpJP=eezntdDa+j(tYyFH=%=X z&tg%pY-F&kH?tiV8gd zJ}n|xKc#xn_n%wr=QWPf%c*nFcaHoAuJ|fyu_l|5o}yfdDrJSxAp*We1_))ktO7#u z*`88;wAsq3RAL1ps+HM1J-nqopX`)FGvTu7t=BPAc95iam70Fw+OLW*V!^ z(=@Fgv}-g}6JOWOcZY7geh6)y9X|Q=y204F;cMf0E+dYZfXUHhO^5_0QtDFkn%?D} z#1IEjo(_maTbXziLrf8jTxTL0ctTIl$fPe!{iaQDVE0=*S?!>0k<)syl>i8(Vq0%Y zP_0~uzKx`Hk5QgMHk2`pT}BNsC5z_upWC0&Nw(IU!+KTtZeDe$rEguoPiVEU6me-e zTm49;6Y(2>_W@DrskUg8SNKry<6qic<@fJ|7Vvi^?SEr0u{ZtK3m!=d67lb>dDy?> z*}!!5|B*%_51#oU)C9+*xfg#|6C6-W^m04-T}|+K^r#!5CP=^eB5@=Hp(a=}<^S4v zR_(b)^942qyv>E9G1=ErmveR;O>&2Y!*F8sn<5^mUCX|Y)%W;=`Jn%sXz+jPZAIt< z)o$LzKlq_81e8mhbyJalQmfV%MoQ6?4d%SyT#pYmYD9kmp*uYYo{WP zRyovZQTcU^cI7-hjqF;ldzHyvo85e-{?a!4Yut&zouTpoQAX2t0BRnj;wPbQ$2YQ) zz{9S|cbJcje#Ff7xNjV){756blo7D+{f;dS*ZIy|=Y7ZX9Qe^M=jpHE8Q-{BcEYU| z!@iP+anyWOGW&F9{1*A`7x7oWhX3taI1@G4pSiHKUVKg(*j(y*f-ZUC@rU8oXRT?` z6j{b(Tl&Q1-M?=1_z%NBF1zv&3kY}qU)Xj0u71ZDWaa4dBFIL){5L>oD)-NU<1MjI z?j?Vihi8`kooeGNpSzK!inI`ILjSP%`LFTC0b@}c-@r|Q`Fydz70+F8K>pU^JP;aHx;NCL`AOlG&D{X!`wRBua zjLtZzQD>>Z?J_D23S%DST4Ybru!UG?f+ETDftFqS*pY=9H8k{-pWNnWqYA3Y_Nn402<0uf$HX<)#R|o$}%W#7DQ7@0GZ!EFqn(6LBIA$gE z7P&7nLbOzZRe>aQ!B3{jqdD^f$Sb<#uuty70`x4EO}`7ST@iQ(8Gu3T5{!2a?cBup zcg;>-Jr%V-nK5!7je%9z)xNZ;FPY+EKIt^tvYI_Xf?|YaaW{na`&Szz)|O8_i*jQH zUM=+1;CH#5u<^HwD`#E5eWp_=E@72HHU90rO+Sxed(?O}Uedw$!}$4Qytw7!c8>`3Oy{3?+=zo@=0aDhW;0zM z^dkeA4@o0ZaB@A5)mZ+v)FXSVLkFymM?2tX!1v$(>X9FuD;$3+d2C<<&GBh2^Hii> z{i&#m!peRd6;>7U*EB{GIxYwCOYBANe0R=}!nSU)-P@Nk1+x0gRqBmR? zo49?V4b8j)UiwAd%mx1v2Sm>NZ+b;<{<$XSK0+~I(f)_}eZ!4?{8{shS-K?RBF;A_ z#MRLLHo`0U(M~3+C;FQwl_Y&^7*prdnzJ0(hrv8OgUBxV zD{+?Nj2bD}m5o}PFNHtw2{fC2*6MwicJII{BxYaVha#gZ+GEUc zhmp`$eygQ2#6zD9;r(DIoM$(w?nLT3&STXCJ1^%qgz z>TT6Soo=_&_M@R^l@B;YXBrcEptC6!BlLVVd;#K&($X07lmo(;&~dZRJtzl6HWavr zpKu&Uk|q)Y+pR9pXIYw;nYU#NM5ia#uRTJmdNib3kI)E zP0Hs9HAO_F3M~#EH15+LZj~vY^ zUtd(PereZo+^{=;eJ&x}AtD)82dQwg2)jB2>E6jz`PMiqs18ASSR1SyQ(b=p;8~d^ zPLs^4g^6NC@KkCLL&|PJ=@@1`SEN$I+T$A#bqQu}%-tnjF*Lpii{U=z`ZYDRINT2# z^mR$>*uBIzVnfbWVc5&-MYa&37H22HKEmWq`HA+DhV?Mlo}&{TJ`4M}-PS`LW5|tw zHt(|Aa!cmw_w`n0kI@cvKyJ3u5Xl=$oZLr5))<1zH8n5BtzC>r_hwlNJ_(4&<&1e7 zy+7DIm>3Y8A>&t-QxheM!zLafk(yVf!&q?uPTdxVU-4fxe|M?bwDyH^S?{6iPV;WG=`8MGgORVweL)0VG2UeRGljxYienk zQ$@R;y^dSOCE!uM+>%s==nE6RNgrNt;*4p_-X~+?aeZ#!xaih^CTB|;Gju-flI7+C z_VFVX1yp+CQ35{UO;?%LoPNE2AB!Cz!JxntTTTWRu$;Wl1>d~G!2(@7iO^}Uwu(Cf zZ@upp65B_W8W)`;Q);YVcTVc04@{RVM>t)~=Ghwhn-;L~iNr1O4nq{${X8s^JZ71x zD6GXH9H!zdy3veC{#g;~TH&5+Shyk>J~j=Jezs{UbxLL0vwJ?%3Y8j4g~VrN#+Y3b&Y^xS5{V7VxKgy%#b`1 z&se@jLAQ-C?HLP+T}i<{$v2tXKc9u+iJ)uOiqFjx^>3=TJuJSj=c}fAX{>&$ck^lk z(#AzJKoO%(pGDdS;u=TiFsUI99$>s6I_E(R^6Irji>uk$+yL+G#Su78=N1#fV zRPEy>l^;aFR1Rh+0iVbPE{57FqK?`&qF4k|M1kcaad+EC&NKD$=c$N75c67g!!!F; zmYZ3JfK`SLH3N&E5V-jAv7D?T_{%k(J6cP_2+LO z%qkB4yR*0U0+^)Xd~L?Ua231@Idp>D4Q7QK>XeJove0Jb#$#vF790{}2WT;0Xz5)r#?BoB94e~E9${Eq7?l$wWGfV7+f}R(%^gSNo zRY)9MeNeBB*pKpTO;&Zjyvn;&3rBlT6TD_-R~A}?k1qt=AkA7%(^$+WOl|+YVw^eD zy`Jp%it)-gYx&CA|DPn zt1UDq#^GdbNWcYqwR0ns=<-ab{@B!WZDf(fSP(Xe0%eSvG`%B6hEXEVz(>>VMPGbD z{hVSEw#~pmOW05fbYp7g?V zd<_~}k_X#Rn;J)9fomV|bf>i3OB2M*Zlk1_nqksGL`;QW!+42R(AXPu#RK4@?!F}~ zVDm3(RkX&Z>lW`B9ZUA$rYL%mg|w`hLA_c`NnGyi)pMFpr+|iGVmM~!!UmloZ-aZo z%qmwSDO&XSur=fNk|KTXb&HaPOFWW^%I8Jc(+{Q!3GJ+oqasnoBI!MGIo}H8lz0mJ zhUCU%a^FzL-c=|N0XFDq#7Gz|=2cFJ_1yjUwu5Dn3e}@DX$HjZs5?4Odi9c8tD#gM zt_$@y)GD)uTn5EFEO7F(d`E+=dOa2Za}cXTMS$wl>P!I8OTBG~XWt>&@2;;~Vs3Uc*}9dv1AF2 zrrpNJH&B?@EM-}rXk6XYOf zn&p!O8PLKo#Qp>7i9!A*iLnT$H{-HgAEIio*Gdh@f>IPQsK*BvM`znQYFo|XNtXCay|GF8)i`??4iAmgQ8w&qjReIWGM?uP6w8A z+r$lA_wxOSLyvt6Djk%VlC>U#H9s;flnSdZg-`a7W}|kB49b~cdabSNY`VIbxlC-8 zv)Nky@)nk+(7@<*dRLWMo*=xGKn;T`$b$#mIMP+6y&|e%4WG~F6VT~XXFHvB$QTK| z9z&iJm>)sxdQn_ketq4b{a^hB6aArVw?Z=#pGI#M%KgI$x( z5D?4)5V5vOLEC}%ueDb;?BT((-#Ug~hIJ*VY{Pc~g5M?yklLR~1+`?D6_%zb6>s4p zzI3Bo4{4@xwG~x;(Oo6$S!HY3<2^4F!)z34TwZG+)nM^S;J33(RG3pCa=Ka5f@FXn zl}6eTqtDst(C@y=u^4xoZLOJi0v&y=$RaS{hn;kJWzv@`i@B8v5@7)1W+MhP&w874 z;zwOyWw9_h?P*_SSYixb6NoUSia0jkY9zMGJs$sAQF=2pPaMh68f=b%feO3{hF`}g zgk{)q`IDX}^pnqBu^W?#U|r}RuDT?udK`tan}3zeTkQkx{4|MbywjN;{U?e%5*=ui zLZ!=LxW3E27^3o17ItT+fv%G>6C!seH%B z6te;b63i|F%(N;ZL?i}BGuV=_+Zj|7%~O!liwv>o(RuA)Q}#T6z>t4cK9t_9au!Za z2p?iySh{fD@GqJzy71ERk=9sAdT!pg_LuB;&x!xymQVj35^tJkt6!xueqNPc7)v)P zmR>x<9}^&Y()|d0D;l!mFSz!Mwy8RP;OityRysnJ!$H9A@+o#GRlf*SD0vNS$t@)i zje83?w>crY_I(&B#T}29lVamK!XhlN6c3NC}|`@L~}F@xuimgR7L+ z{7sdfhb=N$fW@}IP8_$w8U>jcVuDIUPc8(+r`?z$Kxpg!;@$s|0Po48G?RfaSHAh^ zWRY|92d-ncxt?&+M??MQSIqqN);p3Q%4gMY($F(r{+CnDSMSz4YzIm( z?ZHDb3XH7$j!Y}X>qr=J`Sj8NVj3+*vHkQ0^5P100kosVM9qf@!|Sq((Rb246o~S$|p5UL|Tdi z2E+)kvc?5IU+4f01QmaYrr<2&FD7{`uN|sY>iiS*?ULc9MWJ2WpyOR$xnF!GJNxJM zJ%>ni4a==#=0-a<4{D~A;{Z+N?m00Gv%8045Ta|C*fjo3v}|^rz)J;L3@+O1OL|#a ze6(hXliyyh9yH`TZl2Y$z!$0jBlxS(Dp0?HvnL)H7Z}|5+r#$*A_@i_ne}<-=;z6} z42vyvq|AV`S^{X{4AGL|pg=@xj!EF0IuWiJw#_Ha7CvYWu`g>mpO2LqRE>-)Wi~Q8 zSQ_nSqOm=}6F4P=H=2C=E3#nUEDcvZp+*U?JC=m37;#fkg+cvKUMMj4p~hEeiqgEY zO1-ZVWmQmV_C;-Tfp3g3XJDFYh>Vgp^Mj5}=NV$TQ4rDAb)OFk_m1YyXNmc=F%??n z)R+M90=_Ao*gAFD9z<`w_M?vZWa|aK5^D@?2#}6&um*Vth2?qvWoYoO(WzoB; z`W>C~kipwO7bAciIVCH}$(XHvE$Hr#b(}um7js;1d!S@%?VuQuXTfHQJCA@3QGMr_`gTV_0AQPV zAOAU+U1ol^7J2Wlq7vbfcfS^aMM@C^I21%&gbM%&0v;uGH~ocBM)}F#^&npST^M0d zb*VY=;5*ez-t6hCnMg;QbeFfB{X4B}*5zei$DBl84(_~z(e%|6et)4Vr6le8m4<%W zs?(^W*%gCjy*RHr^Rzh6_%Cz}aujBSsue^Z605n1i(jxE$yFLcEtZ<=xCVvCDdKj- z&JJ198U+~H@YJf@{0W;jOfI1 zY>INctkZ<#vIBDx2+Y4!)t5idg_RFfg?p>6o8MswVG4tSr2I7a1Le^e<5;LU9R;pU zZ>=@q)<>q*rg$ahOm4I6VlBp$T+D1pc^DBwN|spEJ}=^h$kIb^GPGS?8_{Ja__8v_ zt%fo>Fu6&UP0I}!EukegZce^^iWw@4s86*2^Ubbwu8T!~51}8T+H&-u@q232!0|Vg zLz0WBr>6);%1A=HEG_9Ho`+fVcR(mHwe&2pOZq)kP0o2E@3~@K$KZKuqP1L2S}n

+p?`Me)>`WtWPG- zyT_+lm;UBaN`uSyJ!q@dQ;XogN>iR9O8+I%Veb@It*yQ3ej`|yzDgsl$ijtdvigRO zzP&ohZGjyH4z+}^z@)Q_fXLbTEPR2U_yoX(uDnVr(lu2%&X=P>G0Z)Y!|)V45b_6n zd;(nzwH-S&kwg2-_%VY&Y(9UK2a!24PC`h)80UF(D^$%~b<1EaH=J{FMeUrO8Z)A1 zMq>>*laC+(xKaS1Xd-_gW+*_yW3iUg`LAR+Dvq8eCRZS~)1ow$HNyONY=5nx<4O5< zEso^BAET^IH;*&F{X8>xad2RFR$hR=ncJ# zT&teS^DqMA6Bn>TX{`DelR_HKrmtHN!j3mi^nZyA{D zooU6kSY-%y43{8aWFoL`IY$n zf>z3(uO*K*xPLs8$+G=!yz=qoZvf^J0gO=35=S1wb`t^Bv{pmr?Zh8N4uSl9h7mKK zY6SPc^E3+{zuSYeT1_Q3o)Eij@DhTlQc3yrNw8P)1r)cDN2;rxCvc`f6g?6Dafm3j>Stc zz6XKHcX4dV?8WN}`1b2(R$s9SlzK246c4xxXN4XS2WEtNr5QoQkLXw|XHUu1oJ2CmAkRfEi`cL9dVU2-7O+uV@tPq3u`*#D zV~6!t)4p5M0bw)3YyFwETpuOaBNsI&X!w3ndtBY@9@|B>@Wr2|XYu;}ll-Ok@dNeh z^Bn|cVyt?ik!RR_wPm90k*gesTd5cSxVqQc5a>h$%PwVr0HFmxTVL=653`rdWhizz z{LD5l$|}XfXLxmd?OMLng(Y#NkJ#L0fI=moPd*bwaQua6_!9q@BC*(MALfm-U zB{^K-4qgu!9&n>mng=WFVe!9C(#*(`mY;1-h?3$X*HJX&CULp>+1m!zBd!WLW~_*O z6#}**9Bt4r#+>m=;nQxK5y!VfMhTp@&7oK9x0CP$=cfZvFmo}%*q2^9!An;?uHG!K zYs?K!j4(=iAr6<3*jGe0X|{%#K7mxSGGO6MJb1;i!}Q#S{~R`FRrap4+-=vUf{W1i zpMKMwMh^iI!n0qyn%>_2R8qAfz&KT1F=p}TvC83bX;tcD`UY7BSeUgvDm=+$x@cjRKh_SYfg=i>cj4oilmvjfc>A8uW_@|Ij>1Qtx@ zM6VK~oQCbnpabODZum@qXM;zI-BxprOZgC;QZL#nv*iYoa<|W2`EhUxxhJ}ZcZaF zQrVppOGgtO7~Q*We(mmo2$m>SZ{b#&aO;yH8Dh$(bg!k&W*_B9V-8x*F7*cdIB0i( zMU_j!%1uf-Ig;G2Yy-1;fLpbf&#hS_SvNcC$x5mM7t9XEq^a_!mDI&ZsH_jfr+udt zp;1eQlQMuwWrC*8zm!VD+TY*+C;QC++&e#7n(X7H;bKG0B+J7euh%VuLLClq^J+>9 z=0rsaX7>_N0YI;DsJVa@%LM1A>O(r4dA2hZ?lM!+wg}V*BT1RB>lt-XxkW^iG6V=9 z&fTu`Kjp8wq7RaHyh|BpI}?3gwQBj&$XI;{!{4&$dORjs5fK4 zU3b_xZk_UicX+w41vagD>+t=So+>?hdAy9nHki7LX>PHqi=h9QcTru!`&iC7nI|d_ z9G6iJf?f46V2N;{p&{^W1VL4WWv3UPm%qFyV=%W?99WCr4}F#N-t+wO+w=P!Uw={~ zUmS;es{EaRLl}0NuE_5>sMDx9qpizEG-)fLOF}wA`g< zImW13XOU=$KUg%b6;eSUjPZs+2q4r%;1wbOTOj!tv1jhIMpnL=jcU$rdBxMF?WGb$1tQBv*xd3uUJ|pqJ;akF=GU$RFq)uL4oeXPqR@ni`KqU(x0N>z zD#bI$+4MiGs@AtJPRM<=vtIogOlbG!LnmwhyDJ1AldQ-uX8<6wnXdR--sZ8 zD+YqD0R8^bK>lVr;xEy%yge)o8!zN>U)F)*vJ*s##Tm(4LmoTW;6&rhdcLNb_T5se zr2b3k{}a)#=0q<*0;6q(nm!gOC6w6>jqWvI=2$WtBIKbTdL&>9w- zSW@Q$^*Pvb`L>$Ap1C03209xxUn`EQ<~pGQ9(=2|eH^oMa|ZwDn!t2k6q~&FG_-RS^o}&z;26F0Mw_;2o)?E+VC6BnLyoy9#X~OB4*fC_Y z+|iPYtRvyBQD6abD3)& zJI%?)<6#_ALfwOdREzfKyp~aJuM%K%g<0u|t9};gbiH3H#2>h2lqVcb#~*6~fH|=W zstKWkne8RAOS#ChJWnEqlL#KDug*-1xg5(3s761!K}S1Qq9g!iX?0wzo3%(x(pHM_ zX%RY?I!msGkT@H)?(L7aS&a>TgQVZA%%5rU3q&5RnT`dBLyY_LeaCo(b-|@xh0p;m z(=}hv-E+11`thXDkO{r*hwx8AV=7wYQv7Ne-FZwCA?Oa>fXc~!hN9W+rG=Q&Njl%v zFjNN&U@y(IZw(K$P@{rJ+k{RA>*;x~5lJM3fZ%b(f+EhzBxs~v~FV)*XSa(3XnD4abir>@(3<>47sYdoK z>1Zc3(m6J*(wxWM$}D8!h;`0^wO!Uhreu!vR*IU=3wyKOa1gHTI?w?B-gya-Le48s zN&T(F?=BxJYPOzgxJT5FWq@D(us@RLfph|wk^Y{hYxhuBV5*E@)r1Lch7<9Z;|!{S z!DS)8bnIU7~~ z150K*&Sk?K#nEpwj4ByO#kK-z=NmjJ{;L7mFzA$sp$$|o%~9|WXe-vIfIeD;?Z945 zj@2F5R9njc~7^n0CZ;oz~|9C+r7CLJ{MT+dhc`_ zD3&bRG_t2g7SNu@J9FpJ_R9y~s#p}2#PW-@Nvv-6g4UU*b@M@SS1tH@a9AH3&j8s@ zzty_n`b=ghf8k=6{^v%9qAhypi;d4vsL^V5hb&wBZrN@XkoW4+!~aw5z7R+IEW<-I zE>T@xAC=P-;xhajvywG8h)miK6=;mIarU!;Y*!mHlFDI30%Ik#QTE!YRrL15J=S1) zueWM26HRINaykhdIy)K+grv`^80}hYor@g*a;rUlTlH9_X;VOKIZaE`Rn)Tj{a&M) zXkuTCe_9DA)>fnj&*_-m%~d@^Ta-AysCQAf@!@W&af6MvL)m4_`fFDzYHV>_cNY^q ziBNeHUVS0LpI7MFFVQZncz79`h@aDSWbi20ZPeznhqpeS*dub0Df7r&m(GJb#{?eX zhz(-XBYc6F&26iz^RAf3B9r~vhZxza-blWH)sVjxJchoIo^$MQ3W&iAf+S+e{e-&L z8#`bIt04yH|Rv57?MBj zaa%EO_mpb1RB&TI?chyUnKQhPR}m+0Mx#l|0W19)Cys5I3Ry9(?>X;mpDS5xY0SWM{ed<87heS z7cC@CR2OjyK!U(Cb{OK)m)S)e(D%zC&+KShZrp4sTIGH~Aaz4V=U{p%!saaqZA>K# zG=4B>IJ>#1s#le8H{L7r(8@A<4A1KIRNs)R2m%v7A75T zLpBn>MzDPzS#Ol`i}=$xG@1+^+NmFP8_&p=JBb7@7wz z1~O3(8`rPVj?4_D>pPY(=uew%YVteGB4V#&KISaTSLl5j#iPoC@`v^zE$NVpDlH9r?p@ ztw_B@!f${na2N#fj2g=sU%D&*VkGY&lTk}tNO2x0JVI3~nOc}^<*K6iSa)OIA7#+xS z(7|&T9NQE-!B|kW5ccJi75Q=y?{;yD6zoasAT(-fxm7hsOTp&Q#1O1Fn75~tI*gz| z4fWrD&CI_m>XRfE3QvX>ZPof7E<|kDqGEe!8Bw`OxtCV6u@}Trl{Q+s^=g~)?rbDf z^Dv06^B4g!*9cOz|Elr}qyiC_-*l9g`}rw9x4nHjHQ@Tsc&rP*m6R9nDs5c!2VogC zu#)Z>gwmHT!UaG_ojdC|(ot_xmGj85WqTJT-7(oPldZ40tRW2jmg*nhIuA>SWOxO< zDAd^&mqhI;g-5E2FI;NIadX~VsE?3?@7J9@{92;y6BpAMWgVnq!K zI9N{D+Q+RBwuQYqTeJc=R0xi(Amge#K!@C}3?c%-8BU(7%MTYKRgXa%p;7->=K)NV zscatw4NNftOWqAvI`~h$jwKe74w_(XH6+5+>yj3m>J({nMN6({_tDX*V~F$-L_&H% zQ#00IsR1qD(QkfM5Smq)TKN3XoOriEc>Ao93g|KYuR8SAxPPHW{a-++|GN)_REYao zv{i1`H5<)z<3!{CoqFVW?`6T@Os)Z!Ww{ekyrau5>N^-r^W@edKKcq_9 zTFV);GD1KvvvVq0^5@Z!?9W{%>n}%zri?TddRm;jT6!2P^ka@XrWC1XINN>=&%tRD zQx^8;D~0CP;I~yIKSkaNukb6ALIH^iC^qLrDxZx=4f$v9qBWXV&CP4Ujlg*1wE zT(E2aq+1=0MiZI*L~av)nPLAf`zhDB>Lo5L#OWPz=NG-8^rD|NnfD@9DDSXOytasB z>+%yyKGrYrQbA!_e0NyLD)>xwpp?9d(_GMg8X;{pHd(=I_04i&Dz$!_=AxAp?C+Yb zJFW8-wb&&YptiO$8`sMrI4Ac*_NDS4%OfNoN>x64zLR-j{Xq5d>)if)faZmr zXeV(9QZB9^3*_gTNR|B0AJWq$j4wn)UZ2{7^~s00I{FFb8`0}Hz+gh@w(Xv^?U!n%kAI)dN08%b_P3sz zhtmblAG4E2?JB5(%y82Hgfnn-M z+QWZ@eL}=P!~S&VbiG#}*#0er7QKvPe&un5vwaHv^XPC@Qxh~rk(+{OEmQxbe*di^ z3jD`2%JKGpS+$D{zI;i<6s z(Hm+{V+5>?{#|7Ga}eLLUg2+Z{Tq+$7k=`W3b2FVIsVcXAO8PO!oQ$-XGt9LwOx$j z@bu+H;l_q@Zfk6uZPLqK^WpqBdUKJZnVb|=)jQ`K1cZxbO(ACtXTf>0p3eJ>ZGQj` zreyg{g!Igca=zvy~pZvJC`FMZaDvZ{J4t{s7oMG77o$DY@<4 z{HyKZ;)lD(SBR3Wo2j2<-z@c1YcA`Rd8YU|_Evd&Lh%pyg_%0x9wICJD^6JU1 zqHc#4U*NMg)B>ZrT7!wvO{o~JMo2oOi?dMG7+|Qsv$|ith^nh^ibSP7$A1pJl+V;v z`JlFDDK}M@n={*&=JtL_Yfwp_D2<*u=Su`-c9!tnG-|WZGQZS)=i3=U~=C)XP z=gh0aO=}$Txi~F`f|G4oX7gx$zvJ!qU$@>Ql`YTp4QD-HuHT@%*G8B6wEkUV+#}Ax zgzd(FjmiVOm&GnvwL~ubWw$-;r6V`|(!g;nM=F?cnbfk3es-t_MN1Z+-7w5Q$wk51 zTJ+x3X^=inTrRuUwv>(1R0g~ze}{zLviRpdXm?|6btG!5^3K{F&IS0~7tdzLZ)hjb z7dQFz0Nyx#hBcNf#{miO5}Gt7a=FCaM|3q_l<{nzqWN2NBE>U|EA{oE7EPcICdWWB zsc_`FJ|V_@I1rN=f>C>4qb+&v8~;rk6bBq;;v9?wT_e)q1=P2k34g zJTbQ`Ku}o$GJ;TUX_z~F2Zq@O^f>k<$hBAhP+x9YlM#^OiP7UX!nu)n$Bt-Z7DXCtx@i=aHcIJe@|H)SW-7FWc+kTwKug&W~LL)Cdqku9!!R7e| zri_BDM%Z>puc5qa3m6sC?rt+TnczfR-h^DXZ3<%*G<~i&chxbIO+)Xp>!G?-9BuSWcy@<5ejq9b;hC~T*z!%#FL=v&O zOYF60sLLQ7O^d~cZH^0VC1Tp{1FzTUlGeC~9Kp#V^07B{Nf3}+C6f(u&R~MN?u`H@ zot&>!bub5?RXV2LiFCf)`v1pLt`Dp#c{uAn-Nf8C(|i z(afaUDzrtm>$H<$;@8Hx;BMJLxVw!$Z)W|6=i*tCm}65HD(=Ogs9s|-x z3VO$`ZX0>+A7uAxHc$k!N{4IaX(td_MrTO7kfYvlI*0P*(yeH!mo z{K=y>T#d0H_O!Y2PRY&nLT{L=ypQ#XPAta?NHqGs#1 z*shQ1veiNL+F-+qM%<0F>D1(VHb`SRKu`%QV*@KZ~ zYM@cs4JSAPUjzpNVS2mt#!E_Q%;D-@Oaxo|vB9|3RBuPK`G=0$3Tp&Z>MU&Vz6B4< zsz0J!NTQZ|L`;!bKVQ%`j>%NApgK;s7?2u?2_m;$)+6AT08HRYQlCX86?mnTV$-3v zrD~>p&=NS>9;hUsCuD;Wx%bQnJ zrz{O?CK}ngYUouY^l62hJW3MK?!hk@lZ%dalYLd0H7}hfmbgj0vF>Uhl8r5~ze$8@@_D2qC#99ot2sPtuYV%GihZPN*Xwx2j$ffb+uJ!m=zb^DvevH_{?$m|`-0kfxNQSK9zvu^ zn)G0Gv14Pt!D?Awr-lxEV%Vs$Xllh7gz3+^maizO0oT99HKhOgn0OO=b$pk6JxRyR zrMuCUb=crgr{msiJ&(AHbG4~6Yf1ybX-~R74+I3cLD^5oWC{>3p(;|^wT<`#AYz5D z^P7DjI6C1RwYJ8n0_N;*aCdI3|371f8gD$>e4$3{E{Y z-Rv^DkbiJZ5cbTSInBo;Q8^{oONf5SJ?@3Kz&o6UUnfMJo(`nURVQ#TMtCk5%EBHPTTM3CGwHF6*YaF% zc28X>Al2H^tT!Oyr*2M>2Y{~?k`vK=WWly$NDy8}<`1kDw5+#z%VsQv_zOb6J!DoC zZoJW=c!NzkGN~>9UFM#jG&w!f`~X&!wG+M+t`ir5#4|xb0P-tv=(hpxC+&TVYE?5G z4|kS51FVH@#qPbqzs(C=reA0A#lp?NL}k^h-e6_xI7@71p~Y#%iB&4k zvZ8;cLrs?-hn12h1|kc`Zt`Ssaz>z^t^-U^z$fssuQvp27JZh^BFz^JtNLDs@n$r# z$@qz}-*e;NeV-#KSYRmB2!p~LlkzBifpiG9HAadweP!sX-<|lbHl?HG+WhQ|z%}h{ zS*K0rQ%C;8{i;o3UgzS*t%r6jQi9d93+U?F{37Kj)yMp%GC-=pxvG)PhgnY))rS`t zwD=P0m&cw?$42N0Z|JS-%aPsoHUSP7FWgu-mdc?TM)Tu7D*Y&>Vc8=(c`Aii*YZA5 zsg92_5W=UW9gg^4Cs7l`-Yp}gF2v1K?HszXDk!tnAQAh~Yz%vc3qMygg3?SD!iLl9 z>eJn=g|5t+n6hA`0OmCo2|)oT~w;7B5XTv?qg$4Q)*ic(KDz{9=iTR^3G6% z8&@2^lpU7-r!i8lvxtS%RCD+bK)M;c-o=5ZCfnT{+Q{w3la6|TXykG3*?lrAAm@E! z!c^NaMmil`>c7YOOSm~6Y?X~lvC_m_yYkpn_b--$aiyF_3&;U zZj@A+xS1+|`=$felP3$tVW?-|%Z%?1>Bq&D>cOCU8qN1KD^C?>*9yf~wLYK%fI4VF z<5*2&rfz_KFewH=g7n*G*Jp0O%eJ=gLuuZKm~XXsTAYs}am%=+-#@Z%Zbz`oQ5Akb zndTwz2Y?oUG%#FZZ}JfrWz0H~wqs@yU~Fz#)mf1XY_xq!>(ygAN?lr!Z>rO~DqTjh zbk|0imTHLxgV^)J^ ziQ6LX09=%(C)%CFEB(`K-Ns?!Z=LS)zJ-5OsUmNius24;>g1M463)>G9ufkuT5rZ% z+e53rPk*oTYoI7)A7jSaS$)L6%@)34O(Qc8W2D7#t8t-KdkLEimunEz;=3p#_yjWO z&}5;atMlNl@32MY%8^E`7|QR2QTB^ZpKK;=rb_TMLSBC%8=yzVv;#1PgTc!+0~}&H z096e*o2qA`l(|4<4Hu7x4H%`4T!@a^n@(x23Qo1hI>z7Vrv*=A3T^>0-pmuJYVh@# zNl9(Bj~5P|?;QI#D~J;D8;-K1^@IAb8i|y0=J8wzt>OqTS3!;`MJ-G{U5#x6PCYWR zw(sIbIx^wjVui25);#`oLF%XL<$7up?oi17!_spJC@1O9MteS7oC?Z)wPqCHV!%Hc z@1scZS|YqjQ9~ap_*Auau9e*iu{KGPp9HnJnIM|ReOP~GTR}9dFQWlqHYh;ZGuvC4 z*)q(Mcd+)zTA`_=U|?(aYzR*v#xYphU{vZ)Voxv{KolBCM^19UM%-B1yHdz7HZ{Kz zb6X=v+m=yiPEXRpHPOtCcgmUOem1z+zdX`pthLp|gofY%7Dz*#F?B{|W`?cwZ=OPV z-TP7cl39_0uEL?dTB^;Kw|D*uiDG7+2tK|vc&FL7{!$9^fdKM?MlMQ`@;2x7A|(KN zu7P%kw6Zn2R}}f+;?Z`E_xM}o-ztLsJ*xy?^k5Fk%~o!i>&*WmBkNqmVh8gAb483xatX|;22F0UEq zGbEI(aK?o`t8_wgQ4>+Y9B3w$q0mQ7=6k-UWZpIKQf_7Jg~zu-yKHeR^R1 zNx_AD_?eBn9Z#pcYuGm*yOz~od@6sE{vNl_6(8vg-D;hFr5)lu_VZ}1%ctW~?>F+F zBec{`2Qp5SXHagRP^TA<-fbBFMjHMB7#`h|x>hYyle(7Ia67c6X8GtUSUvrX&Q1Ph zU}u=}$gkvK(x0D;{f(S`eB%EjA#y*_K_)WVf!2ap;vi3po`=&+SDp(D8no3RLnD`z zk_t-;p4;*N{F8;>xG^Zc>Ui{Z`;?hvuGwozTSons3SKO?8CHgi;3eQsbD<(v$10P< zuL}Ou-GEB>(Je|>92m{;&+bX z3$238@*chWpP0&E`TJ~(Yz3DdyjeTkKfCDad+{e(`ur+!Wz9NsSm1D+RriZItIEu1 zB&$@rBjv;}5oEvwezx)yK*{U?4IAMm4F}R~e(00_;`NMCIWAFY+8vUSYF?au` zqGZDbfJAZ)By(>~+eeeJ0~JKp5PTOM*1zy)lzKquOLI%UR;Xg5Om18)-{A z_x4}!`X7}3{6B0Ne<`AWZ|Jfdm-(-rbN<-WJ7?SNkp9*Y^7GZFW1-*3vOfS@|1h1m z3!`QW(b%;7nC6WIow0np&P%UxnXC~1m}sd%K_C`P-AzaOk-qWY+j_}Yk<9u3ZQ*2c z{mS`NbJW@N%iFaY|Gwr@+&k~-Q7G2Ax+BTReaT&oQ=9e%#J|uOm;&ZxR-++6XTN^Y me|5mT`m|;$M0`{-prN4(ku<83s*WuQ&ib!!P!ayI_ + + +# Use case directory + +The use case directory contains the `yaml` files to test the multitenant controller functionalities: create `lrest` pod, and create PDB operations *create / open / close / unplug / plug / delete / clone /map / parameter session* + +## Makefile helper + +Customizing `yaml` files (tns alias / credential / namespaces name, and so on) is a long procedure that is prone to human error. A simple [`makefile`](../usecase/makefile) is available to quickly and safely configure `yaml` files with your system environment information. Just edit the [parameter file](../usecase/parameters.txt) before proceding. + +```text +TNSALIAS...............:[Tnsalias do not use quotes and avoid space in the string --> (DESCRIPTION=(CONNECT_TIMEOUT=90)(RETRY_COUNT=30)(RETRY_DELA....] +DBUSER.................:[CDB admin user] +DBPASS.................:[CDB admin user password] +WBUSER.................:[HTTPS user] +WBPASS.................:[HTTPS user password] +PDBUSR.................:[PDB admin user] +PDBPWD.................:[PDB admin user password] +PDBNAMESPACE...........:[pdb namespace] +LRSNAMESPACE...........:[cdb namespace] +COMPANY................:[your company name] +APIVERSION.............:v4 --> do not edit +``` + +âš  **WARNING: The makefile is only intended to speed up the usecase directory configuration. Use of this file for production purposes is not supported. The editing and configuration of yaml files for production system is left up to the end user** + +### Prerequisistes: + +- Ensure that **kubectl** is properly configured. +- Ensure that all requirements listed in the [operator installation page](../../../../docs/installation/OPERATOR_INSTALLATION_README.md) are implemented. (role binding,webcert,etc) +- Ensure that the administrative user (admin) on the container database is configured as documented. + +```bash +make operator +``` +This command creates the `operator-database-operator.yaml` file in the local directory, and set up the `watchnamespace` list. Note that the `yaml` file is not applied. + +```bash +make secrets +``` +This command creates all of the Secrets with the encrypted credentials. + +```bash +make genyaml +``` +*make genyaml* generates the required `yaml` files to work with multitenant controllers. + + +![image](../images/UsecaseSchema.jpg) + +## Diag commands and troubleshooting + +### Connect to rest server pod + +```bash +/usr/bin/kubectl exec -n -it -- /bin/bash +``` + + +```bash +## example ## + +kubectl get pods -n cdbnamespace +NAME READY STATUS RESTARTS AGE +cdb-dev-lrest-rs-fnw99 1/1 Running 1 (17h ago) 18h + +kubectl exec cdb-dev-lrest-rs-fnw99 -n cdbnamespace -it -- /bin/bash +[oracle@cdb-dev-lrest-rs-fnw99 ~]$ +``` + +### Monitor control plane + +```bash +kubectl logs -f -l control-plane=controller-manager -n oracle-database-operator-system +``` +```bash +## output example: ## +2024-10-28T23:54:25Z INFO lrpdb-webhook ValidateUpdate-Validating LRPDB spec for : lrpdb2 +2024-10-28T23:54:25Z INFO lrpdb-webhook validateCommon {"name": "lrpdb2"} +2024-10-28T23:54:25Z INFO lrpdb-webhook Valdiating LRPDB Resource Action : MODIFY +2024-10-29T10:07:34Z INFO lrpdb-webhook ValidateUpdate-Validating LRPDB spec for : lrpdb2 +2024-10-29T10:07:34Z INFO lrpdb-webhook ValidateUpdate-Validating LRPDB spec for : lrpdb1 +2024-10-29T16:49:15Z INFO lrpdb-webhook ValidateUpdate-Validating LRPDB spec for : lrpdb1 +2024-10-29T16:49:15Z INFO lrpdb-webhook validateCommon {"name": "lrpdb1"} +2024-10-29T16:49:15Z INFO lrpdb-webhook Valdiating LRPDB Resource Action : CREATE +2024-10-29T10:07:20Z INFO controller-runtime.certwatcher Updated current TLS certificate +2024-10-29T10:07:20Z INFO controller-runtime.webhook Serving webhook server {"host": "", "port": 9443} +2024-10-29T10:07:20Z INFO controller-runtime.certwatcher Starting certificate watcher +I1029 10:07:20.189724 1 leaderelection.go:250] attempting to acquire leader lease oracle-database-operator-system/a9d608ea.oracle.com... +2024-10-29T16:49:15Z INFO lrpdb-webhook Setting default values in LRPDB spec for : lrpdb1 + +``` + +### Error decrypting credential + +The following is an example of a resource creation failure due to decription error: + +```text +2024-10-30T10:09:08Z INFO controllers.LRPDB getEncriptedSecret :pdbusr {"getEncriptedSecret": {"name":"lrpdb1","namespace":"pdbnamespace"}} +2024-10-30T10:09:08Z ERROR controllers.LRPDB Failed to parse private key - x509: failed to parse private key (use ParsePKCS1PrivateKey instead for this key format) {"DecryptWithPrivKey": {"name":"lrpdb1","namespace":"pdbnamespace"}, "error": "x509: failed to parse private key (use ParsePKCS1PrivateKey instead for this key format)"} +``` + + +**Solution**: Ensure you use **PCKS8** format during private key generation. If you are not using `openssl3`, then run this command: + +```bash +openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -pkeyopt rsa_keygen_pubexp:65537 > mykey +``` + +### Crd details + +Use the **describe** option to obtain `crd` information + +```bash +kubectl describe lrpdb lrpdb1 -n pdbnamespace +[...] + Secret: + Key: e_wbuser.txt + Secret Name: wbuser +Status: + Action: CREATE + Bitstat: 25 + Bitstatstr: |MPAPPL|MPWARN|MPINIT| + Conn String: (DESCRIPTION=(CONNECT_TIMEOUT=90)(RETRY_COUNT=30)(RETRY_DELAY=10)(TRANSPORT_CONNECT_TIMEOUT=70)(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan12.testrac.com)(PORT=1521)(IP=V4_ONLY))(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan34.testrac.com)(PORT=1521)(IP=V4_ONLY))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=pdbdev))) + Msg: Success + Open Mode: MOUNTED + Phase: Ready + Status: true + Total Size: 2G +Events: + Type Reason Age From Message + ---- ------ ---- ---- ------- + Normal Created 108s LRPDB LRPDB 'pdbdev' created successfully + Normal Created 108s LRPDB PDB 'pdbdev' assertive pdb deletion turned on + Warning LRESTINFO 95s LRPDB pdb=pdbdev:test_invalid_parameter:16:spfile:2065 + Warning Done 15s (x12 over 2m25s) LRPDB cdb-dev + +``` diff --git a/docs/multitenant/lrest-based/usecase/altersystem_pdb1_resource.yaml b/docs/multitenant/lrest-based/usecase/altersystem_pdb1_resource.yaml new file mode 100644 index 00000000..0467a948 --- /dev/null +++ b/docs/multitenant/lrest-based/usecase/altersystem_pdb1_resource.yaml @@ -0,0 +1,50 @@ +apiVersion: database.oracle.com/v4 +kind: LRPDB +metadata: + name: pdb1 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + cdbName: "DB12" + pdbName: "pdbdev" + action: "Alter" + alterSystemParameter : "cpu_count" + alterSystemValue : "3" + parameterScope : "memory" + + + adminpdbUser: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminpdbPass: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + lrpdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + lrpdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + lrpdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + cdbPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/lrest-based/usecase/cdbnamespace_binding.yaml b/docs/multitenant/lrest-based/usecase/cdbnamespace_binding.yaml new file mode 100644 index 00000000..5fd355f4 --- /dev/null +++ b/docs/multitenant/lrest-based/usecase/cdbnamespace_binding.yaml @@ -0,0 +1,13 @@ +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: oracle-database-operator-oracle-database-operator-manager-rolebinding2 + namespace: cdbnamespace +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: oracle-database-operator-manager-role +subjects: +- kind: ServiceAccount + name: default + namespace: oracle-database-operator-system diff --git a/docs/multitenant/lrest-based/usecase/clone_pdb1_resource.yaml b/docs/multitenant/lrest-based/usecase/clone_pdb1_resource.yaml new file mode 100644 index 00000000..2c4afe13 --- /dev/null +++ b/docs/multitenant/lrest-based/usecase/clone_pdb1_resource.yaml @@ -0,0 +1,51 @@ +apiVersion: database.oracle.com/v4 +kind: LRPDB +metadata: + name: pdb3 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + cdbName: "DB12" + pdbName: "new_clone" + srcPdbName: "pdbdev" + fileNameConversions: "NONE" + totalSize: "UNLIMITED" + tempSize: "UNLIMITED" + pdbconfigmap: "config-map-pdb" + assertiveLrpdbDeletion: true + action: "Clone" + adminpdbUser: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminpdbPass: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + lrpdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + lrpdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + lrpdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + cdbPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/lrest-based/usecase/clone_pdb2_resource.yaml b/docs/multitenant/lrest-based/usecase/clone_pdb2_resource.yaml new file mode 100644 index 00000000..16255a87 --- /dev/null +++ b/docs/multitenant/lrest-based/usecase/clone_pdb2_resource.yaml @@ -0,0 +1,51 @@ +apiVersion: database.oracle.com/v4 +kind: LRPDB +metadata: + name: pdb4 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + cdbName: "DB12" + pdbName: "new_clone2" + srcPdbName: "pdbprd" + fileNameConversions: "NONE" + totalSize: "UNLIMITED" + tempSize: "UNLIMITED" + pdbconfigmap: "config-map-pdb" + assertiveLrpdbDeletion: true + action: "Clone" + adminpdbUser: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminpdbPass: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + lrpdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + lrpdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + lrpdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + cdbPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/lrest-based/usecase/close_pdb1_resource.yaml b/docs/multitenant/lrest-based/usecase/close_pdb1_resource.yaml new file mode 100644 index 00000000..87f7383d --- /dev/null +++ b/docs/multitenant/lrest-based/usecase/close_pdb1_resource.yaml @@ -0,0 +1,47 @@ +apiVersion: database.oracle.com/v4 +kind: LRPDB +metadata: + name: pdb1 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + cdbName: "DB12" + pdbName: "pdbdev" + pdbState: "CLOSE" + modifyOption: "IMMEDIATE" + action: "Modify" + adminpdbUser: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminpdbPass: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + lrpdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + lrpdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + lrpdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + cdbPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/lrest-based/usecase/close_pdb2_resource.yaml b/docs/multitenant/lrest-based/usecase/close_pdb2_resource.yaml new file mode 100644 index 00000000..0743bd8c --- /dev/null +++ b/docs/multitenant/lrest-based/usecase/close_pdb2_resource.yaml @@ -0,0 +1,47 @@ +apiVersion: database.oracle.com/v4 +kind: LRPDB +metadata: + name: pdb2 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + cdbName: "DB12" + pdbName: "pdbprd" + pdbState: "CLOSE" + modifyOption: "IMMEDIATE" + action: "Modify" + adminpdbUser: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminpdbPass: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + lrpdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + lrpdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + lrpdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + cdbPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/lrest-based/usecase/close_pdb3_resource.yaml b/docs/multitenant/lrest-based/usecase/close_pdb3_resource.yaml new file mode 100644 index 00000000..6c6ca519 --- /dev/null +++ b/docs/multitenant/lrest-based/usecase/close_pdb3_resource.yaml @@ -0,0 +1,47 @@ +apiVersion: database.oracle.com/v4 +kind: LRPDB +metadata: + name: pdb3 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + cdbName: "DB12" + pdbName: ""new_clone" + pdbState: "CLOSE" + modifyOption: "IMMEDIATE" + action: "Modify" + adminpdbUser: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminpdbPass: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + lrpdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + lrpdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + lrpdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + cdbPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/lrest-based/usecase/config-map-pdb.yaml b/docs/multitenant/lrest-based/usecase/config-map-pdb.yaml new file mode 100644 index 00000000..2769b498 --- /dev/null +++ b/docs/multitenant/lrest-based/usecase/config-map-pdb.yaml @@ -0,0 +1,11 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: config-map-pdb + namespace: pdbnamespace +data: + rdbmsparameters.txt: | + session_cached_cursors;100;spfile + open_cursors;100;spfile + db_file_multiblock_read_count;16;spfile + test_invalid_parameter;16;spfile diff --git a/docs/multitenant/lrest-based/usecase/config_map_pdb.yaml b/docs/multitenant/lrest-based/usecase/config_map_pdb.yaml new file mode 100644 index 00000000..2769b498 --- /dev/null +++ b/docs/multitenant/lrest-based/usecase/config_map_pdb.yaml @@ -0,0 +1,11 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: config-map-pdb + namespace: pdbnamespace +data: + rdbmsparameters.txt: | + session_cached_cursors;100;spfile + open_cursors;100;spfile + db_file_multiblock_read_count;16;spfile + test_invalid_parameter;16;spfile diff --git a/docs/multitenant/lrest-based/usecase/create_lrest_pod.yaml b/docs/multitenant/lrest-based/usecase/create_lrest_pod.yaml new file mode 100644 index 00000000..b80c1c56 --- /dev/null +++ b/docs/multitenant/lrest-based/usecase/create_lrest_pod.yaml @@ -0,0 +1,44 @@ +apiVersion: database.oracle.com/v4 +kind: LREST +metadata: + name: cdb-dev + namespace: cdbnamespace +spec: + cdbName: "DB12" + lrestImage: container-registry.oracle.com/database/operator:lrest-241210-amd64 + lrestImagePullPolicy: "Always" + dbTnsurl : "(DESCRIPTION=(CONNECT_TIMEOUT=90)(RETRY_COUNT=30)(RETRY_DELAY=10)(TRANSPORT_CONNECT_TIMEOUT=70)(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan12.testrac.com)(PORT=1521)(IP=V4_ONLY))(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan34.testrac.com)(PORT=1521)(IP=V4_ONLY))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=TESTORDS)))" + replicas: 1 + deletePdbCascade: true + cdbAdminUser: + secret: + secretName: "dbuser" + key: "e_dbuser.txt" + cdbAdminPwd: + secret: + secretName: "dbpass" + key: "e_dbpass.txt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + cdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + cdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + cdbPubKey: + secret: + secretName: "pubkey" + key: "publicKey" + cdbPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/lrest-based/usecase/create_pdb1_resource.yaml b/docs/multitenant/lrest-based/usecase/create_pdb1_resource.yaml new file mode 100644 index 00000000..fa58d36a --- /dev/null +++ b/docs/multitenant/lrest-based/usecase/create_pdb1_resource.yaml @@ -0,0 +1,52 @@ +apiVersion: database.oracle.com/v4 +kind: LRPDB +metadata: + name: pdb1 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + cdbName: "DB12" + pdbName: "pdbdev" + assertiveLrpdbDeletion: true + fileNameConversions: "NONE" + unlimitedStorage: false + pdbconfigmap: "config-map-pdb" + tdeImport: false + totalSize: "2G" + tempSize: "800M" + action: "Create" + adminpdbUser: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminpdbPass: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + lrpdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + lrpdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + lrpdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + cdbPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/lrest-based/usecase/create_pdb2_resource.yaml b/docs/multitenant/lrest-based/usecase/create_pdb2_resource.yaml new file mode 100644 index 00000000..02d5763b --- /dev/null +++ b/docs/multitenant/lrest-based/usecase/create_pdb2_resource.yaml @@ -0,0 +1,52 @@ +apiVersion: database.oracle.com/v4 +kind: LRPDB +metadata: + name: pdb2 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + cdbName: "DB12" + pdbName: "pdbprd" + assertiveLrpdbDeletion: true + fileNameConversions: "NONE" + unlimitedStorage: false + pdbconfigmap: "config-map-pdb" + tdeImport: false + totalSize: "2G" + tempSize: "800M" + action: "Create" + adminpdbUser: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminpdbPass: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + lrpdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + lrpdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + lrpdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + cdbPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/lrest-based/usecase/delete_pdb1_resource.yaml b/docs/multitenant/lrest-based/usecase/delete_pdb1_resource.yaml new file mode 100644 index 00000000..1a3c328a --- /dev/null +++ b/docs/multitenant/lrest-based/usecase/delete_pdb1_resource.yaml @@ -0,0 +1,45 @@ +apiVersion: database.oracle.com/v4 +kind: LRPDB +metadata: + name: pdb1 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + pdbName: "pdbdev" + action: "Delete" + dropAction: "INCLUDING" + adminpdbUser: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminpdbPass: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + lrpdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + lrpdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + lrpdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + cdbPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/lrest-based/usecase/delete_pdb2_resource.yaml b/docs/multitenant/lrest-based/usecase/delete_pdb2_resource.yaml new file mode 100644 index 00000000..747641d4 --- /dev/null +++ b/docs/multitenant/lrest-based/usecase/delete_pdb2_resource.yaml @@ -0,0 +1,45 @@ +apiVersion: database.oracle.com/v4 +kind: LRPDB +metadata: + name: pdb2 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + pdbName: "pdbprd" + action: "Delete" + dropAction: "INCLUDING" + adminpdbUser: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminpdbPass: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + lrpdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + lrpdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + lrpdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + cdbPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/lrest-based/usecase/makefile b/docs/multitenant/lrest-based/usecase/makefile new file mode 100644 index 00000000..4203baa4 --- /dev/null +++ b/docs/multitenant/lrest-based/usecase/makefile @@ -0,0 +1,911 @@ +# Copyright (c) 2022, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# __ __ _ __ _ _ +# | \/ | __ _| | _____ / _(_) | ___ +# | |\/| |/ _` | |/ / _ \ |_| | |/ _ \ +# | | | | (_| | < __/ _| | | __/ +# |_| |_|\__,_|_|\_\___|_| |_|_|\___| +# | | | | ___| |_ __ ___ _ __ +# | |_| |/ _ \ | '_ \ / _ \ '__| +# | _ | __/ | |_) | __/ | +# |_| |_|\___|_| .__/ \___|_| +# |_| +# +# WARNING: Using this makefile helps you to customize yaml +# files. Edit parameters.txt with your enviroment +# informartion and execute the following steps +# +# 1) make operator +# it configures the operator yaml files with the +# watch namelist required by the multitenant controllers +# +# 2) make secrets +# It configure the required secrets necessary to operate +# with pdbs multitenant controllers +# +# 3) make genyaml +# It automatically creates all the yaml files based on the +# information available in the parameters file +# +# LIST OF GENERAED YAML FILE +# +# ----------------------------- ---------------------------------- +# oracle-database-operator.yaml : oracle database operator +# cdbnamespace_binding.yaml : role binding for cdbnamespace +# pdbnamespace_binding.yaml : role binding for pdbnamespace +# create_lrest_secret.yaml : create secrets for rest server pod +# create_lrpdb_secret.yaml : create secrets for pluggable database +# create_lrest_pod.yaml : create rest server pod +# create_pdb1_resource.yaml : create first pluggable database +# create_pdb2_resource.yaml : create second pluggable database +# open_pdb1_resource.yaml : open first pluggable database +# open_pdb2_resource.yaml : open second pluggable database +# close_pdb1_resource.yaml : close first pluggable database +# close_pdb2_resource.yaml : close second pluggable database +# clone_lrpdb_resource.yaml : clone thrid pluggable database +# clone_pdb2_resource.yaml : clone 4th pluggable database +# delete_pdb1_resource.yaml : delete first pluggable database +# delete_pdb2_resource.yaml : delete sencond pluggable database +# delete_pdb3_resource.yaml : delete thrid pluggable database +# unplug_pdb1_resource.yaml : unplug first pluggable database +# plug_pdb1_resource.yaml : plug first pluggable database +# map_pdb1_resource.yaml : map the first pluggable database +# config_map.yam : pdb parameters array +# altersystem_pdb1_resource.yaml : chage cpu_count count parameter for the first pdb +# +DATE := `date "+%y%m%d%H%M%S"` +###################### +# PARAMETER SECTIONS # +###################### + +export PARAMETERS=parameters.txt +export TNSALIAS=$(shell cat $(PARAMETERS) |grep -v ^\#|grep TNSALIAS|cut -d : -f 2) +export DBUSER=$(shell cat $(PARAMETERS)|grep -v ^\#|grep DBUSER|cut -d : -f 2) +export DBPASS=$(shell cat $(PARAMETERS)|grep -v ^\#|grep DBPASS|cut -d : -f 2) +export WBUSER=$(shell cat $(PARAMETERS)|grep -v ^\#|grep WBUSER|cut -d : -f 2) +export WBPASS=$(shell cat $(PARAMETERS)|grep -v ^\#|grep WBPASS|cut -d : -f 2) +export PDBUSR=$(shell cat $(PARAMETERS)|grep -v ^\#|grep PDBUSR|cut -d : -f 2) +export PDBPWD=$(shell cat $(PARAMETERS)|grep -v ^\#|grep PDBPWD|cut -d : -f 2) +export PDBNAMESPACE=$(shell cat $(PARAMETERS)|grep -v ^\#|grep PDBNAMESPACE|cut -d : -f 2) +export LRSNAMESPACE=$(shell cat $(PARAMETERS)|grep -v ^\#|grep LRSNAMESPACE|cut -d : -f 2) +export LRESTIMG=$(shell cat $(PARAMETERS)|grep -v ^\#|grep LRESTIMG|cut -d : -f 2,3) +export COMPANY=$(shell cat $(PARAMETERS)|grep -v ^\#|grep COMPANY|cut -d : -f 2) +export APIVERSION=$(shell cat $(PARAMETERS)|grep -v ^\#|grep APIVERSION|cut -d : -f 2) +export OPRNAMESPACE=oracle-database-operator-system +export ORACLE_OPERATOR_YAML=../../../../oracle-database-operator.yaml +export TEST_EXEC_TIMEOUT=3m + +REST_SERVER=lrest +SKEY=tls.key +SCRT=tls.crt +CART=ca.crt +PRVKEY=ca.key +PUBKEY=public.pem +COMPANY=oracle +DBUSERFILE=dbuser.txt +DBPASSFILE=dbpass.txt +WBUSERFILE=wbuser.txt +WBPASSFILE=wbpass.txt +PDBUSRFILE=pdbusr.txt +PDBPWDFILE=pdbpwd.txt + +################# +### FILE LIST ### +################# + +export LREST_POD=create_lrest_pod.yaml + +export LRPDBCRE1=create_pdb1_resource.yaml +export LRPDBCRE2=create_pdb2_resource.yaml + +export LRPDBCLOSE1=close_pdb1_resource.yaml +export LRPDBCLOSE2=close_pdb2_resource.yaml +export LRPDBCLOSE3=close_pdb3_resource.yaml + +export LRPDBOPEN1=open_pdb1_resource.yaml +export LRPDBOPEN2=open_pdb2_resource.yaml +export LRPDBOPEN3=open_pdb3_resource.yaml + +export LRPDBCLONE1=clone_pdb1_resource.yaml +export LRPDBCLONE2=clone_pdb2_resource.yaml + +export LRPDBDELETE1=delete_pdb1_resource.yaml +export LRPDBDELETE2=delete_pdb2_resource.yaml +export LRPDBDELETE3=delete_pdb3_resource.yaml + +export LRPDBUNPLUG1=unplug_pdb1_resource.yaml +export LRPDBPLUG1=plug_pdb1_resource.yaml + +export LRPDBMAP1=map_pdb1_resource.yaml +export LRPDBMAP2=map_pdb2_resource.yaml +export LRPDBMAP3=map_pdb3_resource.yaml + +export LRPDBMAP1=map_pdb1_resource.yaml +export LRPDBMAP2=map_pdb2_resource.yaml +export LRPDBMAP3=map_pdb3_resource.yaml + +export ALTERSYSTEMYAML=altersystem_pdb1_resource.yaml +export CONFIG_MAP=config_map_pdb.yaml + + + + +##BINARIES +export KUBECTL=/usr/bin/kubectl +OPENSSL=/usr/bin/openssl +ECHO=/usr/bin/echo +RM=/usr/bin/rm +CP=/usr/bin/cp +TAR=/usr/bin/tar +MKDIR=/usr/bin/mkdir +SED=/usr/bin/sed + +check: + @printf "TNSALIAS...............:%.60s....\n" $(TNSALIAS) + @printf "DBUSER.................:%s\n" $(DBUSER) + @printf "DBPASS.................:%s\n" $(DBPASS) + @printf "WBUSER.................:%s\n" $(WBUSER) + @printf "WBPASS.................:%s\n" $(WBPASS) + @printf "PDBUSR.................:%s\n" $(PDBUSR) + @printf "PDBPWD.................:%s\n" $(PDBPWD) + @printf "PDBNAMESPACE...........:%s\n" $(PDBNAMESPACE) + @printf "LRSNAMESPACE...........:%s\n" $(LRSNAMESPACE) + @printf "COMPANY................:%s\n" $(COMPANY) + @printf "APIVERSION.............:%s\n" $(APIVERSION) + +define msg +@printf "\033[31;7m%s\033[0m\r" "......................................]" +@printf "\033[31;7m[\xF0\x9F\x91\x89 %s\033[0m\n" $(1) +endef + +tls: + $(call msg,"TLS GENERATION") + #$(OPENSSL) genrsa -out $(PRVKEY) 2048 + $(OPENSSL) genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -pkeyopt rsa_keygen_pubexp:65537 > $(PRVKEY) + $(OPENSSL) req -new -x509 -days 365 -key $(PRVKEY) \ + -subj "/C=CN/ST=GD/L=SZ/O=$(COMPANY), Inc./CN=$(COMPANY) Root CA" -out ca.crt + $(OPENSSL) req -newkey rsa:2048 -nodes -keyout $(SKEY) -subj \ + "/C=CN/ST=GD/L=SZ/O=$(COMPANY), Inc./CN=cdb-dev-$(REST_SERVER).$(LRSNAMESPACE)" -out server.csr + $(ECHO) "subjectAltName=DNS:cdb-dev-$(REST_SERVER).$(LRSNAMESPACE)" > extfile.txt + $(OPENSSL) x509 -req -extfile extfile.txt -days 365 -in server.csr -CA ca.crt -CAkey $(PRVKEY) -CAcreateserial -out $(SCRT) + $(OPENSSL) rsa -in $(PRVKEY) -outform PEM -pubout -out $(PUBKEY) + +secrets: tls delsecrets + $(call msg,"CREATING NEW TLS/PRVKEY/PUBKEY SECRETS") + $(KUBECTL) create secret tls db-tls --key="$(SKEY)" --cert="$(SCRT)" -n $(LRSNAMESPACE) + $(KUBECTL) create secret generic db-ca --from-file="$(CART)" -n $(LRSNAMESPACE) + $(KUBECTL) create secret tls db-tls --key="$(SKEY)" --cert="$(SCRT)" -n $(PDBNAMESPACE) + $(KUBECTL) create secret generic db-ca --from-file="$(CART)" -n $(PDBNAMESPACE) + #$(KUBECTL) create secret tls prvkey --key="$(PRVKEY)" --cert=ca.crt -n $(LRSNAMESPACE) + $(KUBECTL) create secret generic pubkey --from-file=publicKey=$(PUBKEY) -n $(LRSNAMESPACE) + $(KUBECTL) create secret generic prvkey --from-file=privateKey=$(PRVKEY) -n $(LRSNAMESPACE) + $(KUBECTL) create secret generic prvkey --from-file=privateKey="$(PRVKEY)" -n $(PDBNAMESPACE) + $(call msg,"CREATING NEW CREDENTIAL SECRETS") + @$(ECHO) $(DBUSER) > $(DBUSERFILE) + @$(ECHO) $(DBPASS) > $(DBPASSFILE) + @$(ECHO) $(WBUSER) > $(WBUSERFILE) + @$(ECHO) $(WBPASS) > $(WBPASSFILE) + @$(ECHO) $(PDBUSR) > $(PDBUSRFILE) + @$(ECHO) $(PDBPWD) > $(PDBPWDFILE) + $(OPENSSL) rsautl -encrypt -pubin -inkey $(PUBKEY) -in $(DBUSERFILE) |base64 > e_$(DBUSERFILE) + $(OPENSSL) rsautl -encrypt -pubin -inkey $(PUBKEY) -in $(DBPASSFILE) |base64 > e_$(DBPASSFILE) + $(OPENSSL) rsautl -encrypt -pubin -inkey $(PUBKEY) -in $(WBUSERFILE) |base64 > e_$(WBUSERFILE) + $(OPENSSL) rsautl -encrypt -pubin -inkey $(PUBKEY) -in $(WBPASSFILE) |base64 > e_$(WBPASSFILE) + $(OPENSSL) rsautl -encrypt -pubin -inkey $(PUBKEY) -in $(PDBUSRFILE) |base64 > e_$(PDBUSRFILE) + $(OPENSSL) rsautl -encrypt -pubin -inkey $(PUBKEY) -in $(PDBPWDFILE) |base64 > e_$(PDBPWDFILE) + $(KUBECTL) create secret generic dbuser --from-file=e_$(DBUSERFILE) -n $(LRSNAMESPACE) + $(KUBECTL) create secret generic dbpass --from-file=e_$(DBPASSFILE) -n $(LRSNAMESPACE) + $(KUBECTL) create secret generic wbuser --from-file=e_$(WBUSERFILE) -n $(LRSNAMESPACE) + $(KUBECTL) create secret generic wbpass --from-file=e_$(WBPASSFILE) -n $(LRSNAMESPACE) + $(KUBECTL) create secret generic wbuser --from-file=e_$(WBUSERFILE) -n $(PDBNAMESPACE) + $(KUBECTL) create secret generic wbpass --from-file=e_$(WBPASSFILE) -n $(PDBNAMESPACE) + $(KUBECTL) create secret generic pdbusr --from-file=e_$(PDBUSRFILE) -n $(PDBNAMESPACE) + $(KUBECTL) create secret generic pdbpwd --from-file=e_$(PDBPWDFILE) -n $(PDBNAMESPACE) + $(RM) $(SKEY) $(SCRT) $(CART) $(PRVKEY) $(PUBKEY) server.csr extfile.txt ca.srl \ + $(DBUSERFILE) $(DBPASSFILE) $(WBUSERFILE) $(WBPASSFILE) $(PDBUSRFILE) $(PDBPWDFILE)\ + e_$(DBUSERFILE) e_$(DBPASSFILE) e_$(WBUSERFILE) e_$(WBPASSFILE) e_$(PDBUSRFILE) e_$(PDBPWDFILE) + $(KUBECTL) get secrets -n $(LRSNAMESPACE) + $(KUBECTL) get secrets -n $(PDBNAMESPACE) + +delsecrets: + $(call msg,"CLEAN OLD SECRETS") + $(eval SECRETSP:=$(shell kubectl get secrets -n $(PDBNAMESPACE) -o custom-columns=":metadata.name" --no-headers) ) + $(eval SECRETSL:=$(shell kubectl get secrets -n $(LRSNAMESPACE) -o custom-columns=":metadata.name" --no-headers) ) + @[ "${SECRETSP}" ] && ( \ + printf "Deleteing secrets in namespace -n $(PDBNAMESPACE)\n") &&\ + ($(KUBECTL) delete secret $(SECRETSP) -n $(PDBNAMESPACE))\ + || ( echo "No screts in namespace $(PDBNAMESPACE)") + @[ "${SECRETSL}" ] && ( \ + printf "Deleteing secrets in namespace -n $(LRSNAMESPACE)\n") &&\ + ($(KUBECTL) delete secret $(SECRETSL) -n $(LRSNAMESPACE))\ + || ( echo "No screts in namespace $(PDBNAMESPACE)") + +cleanCert: + $(RM) $(SKEY) $(SCRT) $(CART) $(PRVKEY) $(PUBKEY) server.csr extfile.txt ca.srl \ + $(DBUSERFILE) $(DBPASSFILE) $(WBUSERFILE) $(WBPASSFILE) $(PDBUSRFILE) $(PDBPWDFILE)\ + e_$(DBUSERFILE) e_$(DBPASSFILE) e_$(WBUSERFILE) e_$(WBPASSFILE) e_$(PDBUSRFILE) e_$(PDBPWDFILE) + +### YAML FILE SECTION ### +define _opr +cp ${ORACLE_OPERATOR_YAML} . +export OPBASENAME=`basename ${ORACLE_OPERATOR_YAML}` +#export PDBNAMESPACE=$(cat ${PARAMETERS}|grep -v ^\#|grep PDBNAMESPACE|cut -d : -f 2) + +cp ${OPBASENAME} ${OPBASENAME}.ORIGNINAL +printf "\n\t\xF0\x9F\x91\x89 ${OPBASENAME}\n\n" +printf "\n\t\xF0\x9F\x91\x89 ${PDBNAMESPACE}\n\n" +sed -i 's/value: ""/value: ${OPRNAMESPACE},$PDBNAMESPACE,${LRSNAMESPACE}/g' ${OPBASENAME} +endef + +export opr = $(value _opr) + +operator: +# @ eval "$$opr" + $(CP) ${ORACLE_OPERATOR_YAML} . + ${CP} `basename ${ORACLE_OPERATOR_YAML}` `basename ${ORACLE_OPERATOR_YAML}`.ORG + $(SED) -i 's/value: ""/value: $(OPRNAMESPACE),$(PDBNAMESPACE),$(LRSNAMESPACE)/g' `basename ${ORACLE_OPERATOR_YAML}` + + +define _script00 +cat < authsection.yaml + adminpdbUser: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminpdbPass: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + lrpdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + lrpdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + lrpdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + cdbPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" +EOF + + +cat < ${PDBNAMESPACE}_binding.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: oracle-database-operator-oracle-database-operator-manager-rolebinding1 + namespace: ${PDBNAMESPACE} +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: oracle-database-operator-manager-role +subjects: +- kind: ServiceAccount + name: default + namespace: oracle-database-operator-system +EOF + +cat < ${LRSNAMESPACE}_binding.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: oracle-database-operator-oracle-database-operator-manager-rolebinding2 + namespace: ${LRSNAMESPACE} +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: oracle-database-operator-manager-role +subjects: +- kind: ServiceAccount + name: default + namespace: oracle-database-operator-system +EOF + +endef +export script00 = $(value _script00) +secyaml: + @ eval "$$script00" + + +#echo lrest pod creation +define _script01 +cat < ${LREST_POD} +apiVersion: database.oracle.com/${APIVERSION} +kind: LREST +metadata: + name: cdb-dev + namespace: cdbnamespace +spec: + cdbName: "DB12" + lrestImage: ${LRESTIMG} + lrestImagePullPolicy: "Always" + dbTnsurl : ${TNSALIAS} + replicas: 1 + deletePdbCascade: true + cdbAdminUser: + secret: + secretName: "dbuser" + key: "e_dbuser.txt" + cdbAdminPwd: + secret: + secretName: "dbpass" + key: "e_dbpass.txt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + cdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + cdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + cdbPubKey: + secret: + secretName: "pubkey" + key: "publicKey" + cdbPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" +EOF + +endef +export script01 = $(value _script01) + + +define _script02 + +cat <${LRPDBCRE1} +apiVersion: database.oracle.com/${APIVERSION} +kind: LRPDB +metadata: + name: pdb1 + namespace: ${PDBNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${LRSNAMESPACE}" + cdbName: "DB12" + pdbName: "pdbdev" + assertiveLrpdbDeletion: true + fileNameConversions: "NONE" + unlimitedStorage: false + pdbconfigmap: "config-map-pdb" + tdeImport: false + totalSize: "2G" + tempSize: "800M" + action: "Create" +EOF + +cat < ${LRPDBCRE2} +apiVersion: database.oracle.com/${APIVERSION} +kind: LRPDB +metadata: + name: pdb2 + namespace: ${PDBNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${LRSNAMESPACE}" + cdbName: "DB12" + pdbName: "pdbprd" + assertiveLrpdbDeletion: true + fileNameConversions: "NONE" + unlimitedStorage: false + pdbconfigmap: "config-map-pdb" + tdeImport: false + totalSize: "2G" + tempSize: "800M" + action: "Create" +EOF + +cat <${LRPDBOPEN1} +apiVersion: database.oracle.com/${APIVERSION} +kind: LRPDB +metadata: + name: pdb1 + namespace: ${PDBNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${LRSNAMESPACE}" + cdbName: "DB12" + pdbName: "pdbdev" + action: "Modify" + pdbState: "OPEN" + modifyOption: "READ WRITE" +EOF + +cat <${LRPDBOPEN2} +apiVersion: database.oracle.com/${APIVERSION} +kind: LRPDB +metadata: + name: pdb2 + namespace: ${PDBNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${LRSNAMESPACE}" + cdbName: "DB12" + pdbName: "pdbprd" + action: "Modify" + pdbState: "OPEN" + modifyOption: "READ WRITE" +EOF + +cat <${LRPDBOPEN3} +apiVersion: database.oracle.com/${APIVERSION} +kind: LRPDB +metadata: + name: pdb3 + namespace: ${PDBNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${LRSNAMESPACE}" + cdbName: "DB12" + pdbName: "new_clone" + action: "Modify" + pdbState: "OPEN" + modifyOption: "READ WRITE" +EOF + +cat <${LRPDBCLOSE1} +apiVersion: database.oracle.com/${APIVERSION} +kind: LRPDB +metadata: + name: pdb1 + namespace: ${PDBNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${LRSNAMESPACE}" + cdbName: "DB12" + pdbName: "pdbdev" + pdbState: "CLOSE" + modifyOption: "IMMEDIATE" + action: "Modify" +EOF + +cat <${LRPDBCLOSE2} +apiVersion: database.oracle.com/${APIVERSION} +kind: LRPDB +metadata: + name: pdb2 + namespace: ${PDBNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${LRSNAMESPACE}" + cdbName: "DB12" + pdbName: "pdbprd" + pdbState: "CLOSE" + modifyOption: "IMMEDIATE" + action: "Modify" +EOF + +cat <${LRPDBCLOSE3} +apiVersion: database.oracle.com/${APIVERSION} +kind: LRPDB +metadata: + name: pdb3 + namespace: ${PDBNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${LRSNAMESPACE}" + cdbName: "DB12" + pdbName: ""new_clone" + pdbState: "CLOSE" + modifyOption: "IMMEDIATE" + action: "Modify" +EOF + +cat < ${LRPDBCLONE1} +apiVersion: database.oracle.com/${APIVERSION} +kind: LRPDB +metadata: + name: pdb3 + namespace: ${PDBNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${LRSNAMESPACE}" + cdbName: "DB12" + pdbName: "new_clone" + srcPdbName: "pdbdev" + fileNameConversions: "NONE" + totalSize: "UNLIMITED" + tempSize: "UNLIMITED" + pdbconfigmap: "config-map-pdb" + assertiveLrpdbDeletion: true + action: "Clone" +EOF + +cat < ${LRPDBCLONE2} +apiVersion: database.oracle.com/${APIVERSION} +kind: LRPDB +metadata: + name: pdb4 + namespace: ${PDBNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${LRSNAMESPACE}" + cdbName: "DB12" + pdbName: "new_clone2" + srcPdbName: "pdbprd" + fileNameConversions: "NONE" + totalSize: "UNLIMITED" + tempSize: "UNLIMITED" + pdbconfigmap: "config-map-pdb" + assertiveLrpdbDeletion: true + action: "Clone" +EOF + +cat < ${LRPDBDELETE1} +apiVersion: database.oracle.com/${APIVERSION} +kind: LRPDB +metadata: + name: pdb1 + namespace: ${PDBNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${LRSNAMESPACE}" + pdbName: "pdbdev" + action: "Delete" + dropAction: "INCLUDING" +EOF + +cat < ${LRPDBDELETE2} +apiVersion: database.oracle.com/${APIVERSION} +kind: LRPDB +metadata: + name: pdb2 + namespace: ${PDBNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${LRSNAMESPACE}" + pdbName: "pdbprd" + action: "Delete" + dropAction: "INCLUDING" +EOF + +cat < ${LRPDBUNPLUG1} +apiVersion: database.oracle.com/${APIVERSION} +kind: LRPDB +metadata: + name: pdb1 + namespace: ${PDBNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${LRSNAMESPACE}" + cdbName: "DB12" + pdbName: "pdbdev" + xmlFileName: "/tmp/pdb.xml" + action: "Unplug" +EOF + +cat <${LRPDBPLUG1} +apiVersion: database.oracle.com/${APIVERSION} +kind: LRPDB +metadata: + name: pdb1 + namespace: ${PDBNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${LRSNAMESPACE}" + cdbName: "DB12" + pdbName: "pdbdev" + xmlFileName: "/tmp/pdb.xml" + action: "plug" + fileNameConversions: "NONE" + sourceFileNameConversions: "NONE" + copyAction: "MOVE" + totalSize: "1G" + tempSize: "100M" + assertiveLrpdbDeletion: true + pdbconfigmap: "config-map-pdb" + action: "Plug" +EOF + +cat <${LRPDBMAP1} +apiVersion: database.oracle.com/${APIVERSION} +kind: LRPDB +metadata: + name: pdb1 + namespace: ${PDBNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${LRSNAMESPACE}" + cdbName: "DB12" + pdbName: "pdbdev" + assertiveLrpdbDeletion: true + fileNameConversions: "NONE" + totalSize: "1G" + tempSize: "100M" + action: "Map" +EOF + +cat <${LRPDBMAP2} +apiVersion: database.oracle.com/${APIVERSION} +kind: LRPDB +metadata: + name: pdb2 + namespace: ${PDBNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${LRSNAMESPACE}" + cdbName: "DB12" + pdbName: "pdbprd" + assertiveLrpdbDeletion: true + fileNameConversions: "NONE" + totalSize: "1G" + tempSize: "100M" + action: "Map" +EOF + + +cat <${LRPDBMAP3} +apiVersion: database.oracle.com/${APIVERSION} +kind: LRPDB +metadata: + name: pdb3 + namespace: ${PDBNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${LRSNAMESPACE}" + cdbName: "DB12" + pdbName: "new_clone" + assertiveLrpdbDeletion: true + fileNameConversions: "NONE" + totalSize: "1G" + tempSize: "100M" + action: "Map" +EOF + +cat <${CONFIG_MAP} +apiVersion: v1 +kind: ConfigMap +metadata: + name: config-map-pdb + namespace: ${PDBNAMESPACE} +data: + rdbmsparameters.txt: | + session_cached_cursors;100;spfile + open_cursors;100;spfile + db_file_multiblock_read_count;16;spfile + test_invalid_parameter;16;spfile +EOF + + +cat < ${ALTERSYSTEMYAML} +apiVersion: database.oracle.com/${APIVERSION} +kind: LRPDB +metadata: + name: pdb1 + namespace: ${PDBNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${LRSNAMESPACE}" + cdbName: "DB12" + pdbName: "pdbdev" + action: "Alter" + alterSystemParameter : "cpu_count" + alterSystemValue : "3" + parameterScope : "memory" + + +EOF + +## Auth information +for _file in ${LRPDBCRE1} ${LRPDBCRE2} ${LRPDBOPEN1} ${LRPDBOPEN2} ${LRPDBOPEN3} ${LRPDBCLOSE1} ${LRPDBCLOSE2} ${LRPDBCLOSE3} ${LRPDBCLONE1} ${LRPDBCLONE2} ${LRPDBDELETE1} ${LRPDBDELETE2} ${LRPDBUNPLUG1} ${LRPDBPLUG1} ${LRPDBMAP1} ${LRPDBMAP2} ${LRPDBMAP3} ${ALTERSYSTEMYAML} +do +ls -ltr ${_file} + cat authsection.yaml >> ${_file} +done +rm authsection.yaml +endef + +export script02 = $(value _script02) + +genyaml: secyaml + @ eval "$$script01" + @ eval "$$script02" + +cleanyaml: + - $(RM) $(LRPDBMAP3) $(LRPDBMAP2) $(LRPDBMAP1) $(LRPDBPLUG1) $(LRPDBUNPLUG1) $(LRPDBDELETE2) $(LRPDBDELETE1) $(LRPDBCLONE2) $(LRPDBCLONE1) $(LRPDBCLOSE3) $(LRPDBCLOSE2) $(LRPDBCLOSE1) $(LRPDBOPEN3) $(LRPDBOPEN2) $(LRPDBOPEN1) $(LRPDBCRE2) $(LRPDBCRE1) $(LREST_POD) ${ALTERSYSTEMYAML} + - $(RM) ${CONFIG_MAP} ${PDBNAMESPACE}_binding.yaml ${LRSNAMESPACE}_binding.yaml + + + + +################# +### PACKAGING ### +################# + +pkg: + - $(RM) -rf /tmp/pkgtestplan + $(MKDIR) /tmp/pkgtestplan + $(CP) -R * /tmp/pkgtestplan + $(CP) ../../../../oracle-database-operator.yaml /tmp/pkgtestplan/ + $(TAR) -C /tmp -cvf ~/pkgtestplan_$(DATE).tar pkgtestplan + +################ +### diag ### +################ + +login: + $(KUBECTL) exec `$(KUBECTL) get pods -n $(LRSNAMESPACE)|grep rest|cut -d ' ' -f 1` -n $(LRSNAMESPACE) -it -- /bin/bash + + +reloadop: + echo "RESTARTING OPERATOR" + $(eval OP1 := $(shell $(KUBECTL) get pods -n $(OPRNAMESPACE)|grep oracle-database-operator-controller|head -1|cut -d ' ' -f 1 )) + $(eval OP2 := $(shell $(KUBECTL) get pods -n $(OPRNAMESPACE)|grep oracle-database-operator-controller|head -2|tail -1|cut -d ' ' -f 1 )) + $(eval OP3 := $(shell $(KUBECTL) get pods -n $(OPRNAMESPACE)|grep oracle-database-operator-controller|tail -1|cut -d ' ' -f 1 )) + $(KUBECTL) get pod $(OP1) -n $(OPRNAMESPACE) -o yaml | kubectl replace --force -f - + $(KUBECTL) get pod $(OP2) -n $(OPRNAMESPACE) -o yaml | kubectl replace --force -f - + $(KUBECTL) get pod $(OP3) -n $(OPRNAMESPACE) -o yaml | kubectl replace --force -f - + + +dump: + @$(eval TMPSP := $(shell date "+%y%m%d%H%M%S" )) + @$(eval DIAGFILE := ./opdmp.$(TMPSP)) + @>$(DIAGFILE) + @echo "OPERATOR DUMP" >> $(DIAGFILE) + @echo "~~~~~~~~~~~~~" >> $(DIAGFILE) + $(KUBECTL) logs pod/`$(KUBECTL) get pods -n $(OPRNAMESPACE)|grep oracle-database-operator-controller|head -1|cut -d ' ' -f 1` -n $(OPRNAMESPACE) >>$(DIAGFILE) + $(KUBECTL) logs pod/`$(KUBECTL) get pods -n $(OPRNAMESPACE)|grep oracle-database-operator-controller|head -2|tail -1 | cut -d ' ' -f 1` -n $(OPRNAMESPACE) >>$(DIAGFILE) + $(KUBECTL) logs pod/`$(KUBECTL) get pods -n $(OPRNAMESPACE)|grep oracle-database-operator-controller|tail -1|cut -d ' ' -f 1` -n $(OPRNAMESPACE) >>$(DIAGFILE) + +####################################################### +#### TEST SECTION #### +####################################################### + +run00: + @$(call msg,"lrest pod creation") + - $(KUBECTL) delete lrest cdb-dev -n $(LRSNAMESPACE) + $(KUBECTL) apply -f $(LREST_POD) + $(KUBECTL) wait --for jsonpath='{.status.phase'}="Ready" lrest cdb-dev -n $(LRSNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) + @$(call msg,"lrest pod completed") + $(KUBECTL) get lrest -n $(LRSNAMESPACE) + $(KUBECTL) get pod -n $(LRSNAMESPACE) + +run01.1: + @$(call msg,"lrpdb pdb1 creation") + $(KUBECTL) apply -f $(LRPDBCRE1) + $(KUBECTL) wait --for jsonpath='{.status.phase'}="Ready" lrpdb pdb1 -n $(PDBNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) + @$(call msg, "lrpdb pdb1 creation completed") + $(KUBECTL) get lrpdb pdb1 -n $(PDBNAMESPACE) + +run01.2: + @$(call msg, "lrpdb pdb2 creation") + $(KUBECTL) apply -f $(LRPDBCRE2) + $(KUBECTL) wait --for jsonpath='{.status.phase'}="Ready" lrpdb pdb2 -n $(PDBNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) + @$(call msg, "lrpdb pdb2 creation completed") + $(KUBECTL) get lrpdb pdb2 -n $(PDBNAMESPACE) + +run02.1: + @$(call msg, "lrpdb pdb1 open") + $(KUBECTL) apply -f $(LRPDBOPEN1) + $(KUBECTL) wait --for jsonpath='{.status.openMode'}="READ WRITE" lrpdb pdb1 -n $(PDBNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) + @$(call msg, "lrpdb pdb1 open completed") + $(KUBECTL) get lrpdb pdb1 -n $(PDBNAMESPACE) + +run02.2: + @$(call msg,"lrpdb pdb2 open") + $(KUBECTL) apply -f $(LRPDBOPEN2) + $(KUBECTL) wait --for jsonpath='{.status.openMode'}="READ WRITE" lrpdb pdb2 -n $(PDBNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) + @$(call msg,"lrpdb pdb2 open completed") + $(KUBECTL) get lrpdb pdb2 -n $(PDBNAMESPACE) + + +run03.1: + @$(call msg,"clone pdb1-->pdb3") + $(KUBECTL) apply -f $(LRPDBCLONE1) + $(KUBECTL) wait --for jsonpath='{.status.phase'}="Ready" lrpdb pdb3 -n $(PDBNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) + @$(call msg,"clone pdb1-->pdb3 completed") + $(KUBECTL) get lrpdb pdb3 -n $(PDBNAMESPACE) + + +run03.2: + @$(call msg,"clone pdb2-->pdb4") + $(KUBECTL) apply -f $(LRPDBCLONE2) + $(KUBECTL) wait --for jsonpath='{.status.phase'}="Ready" lrpdb pdb4 -n $(PDBNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) + @$(call msg,"clone pdb2-->pdb4 completed") + $(KUBECTL) get lrpdb pdb3 -n $(PDBNAMESPACE) + + +run04.1: + @$(call msg,"lrpdb pdb1 close") + $(KUBECTL) apply -f $(LRPDBCLOSE1) + $(KUBECTL) wait --for jsonpath='{.status.openMode'}="MOUNTED" lrpdb pdb1 -n $(PDBNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) + @$(call msg, "lrpdb pdb1 close completed") + $(KUBECTL) get lrpdb pdb1 -n $(PDBNAMESPACE) + +run04.2: + @$(call msg,"lrpdb pdb2 close") + $(KUBECTL) apply -f $(LRPDBCLOSE2) + $(KUBECTL) wait --for jsonpath='{.status.openMode'}="MOUNTED" lrpdb pdb2 -n $(PDBNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) + @$(call msg,"lrpdb pdb2 close completed") + $(KUBECTL) get lrpdb pdb2 -n $(PDBNAMESPACE) + +run05.1: + @$(call msg,"lrpdb pdb1 unplug") + $(KUBECTL) apply -f $(LRPDBUNPLUG1) + $(KUBECTL) wait --for=delete lrpdb pdb1 -n $(PDBNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) + @$(call msg,"lrpdb pdb1 unplug completed") + +run06.1: + @$(call msg, "lrpdb pdb1 plug") + $(KUBECTL) apply -f $(LRPDBPLUG1) + $(KUBECTL) wait --for jsonpath='{.status.phase'}="Ready" lrpdb pdb1 -n $(PDBNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) + @$(call msg, "lrpdb pdb1 plug completed") + $(KUBECTL) get lrpdb pdb1 -n $(PDBNAMESPACE) + +run07.1: + @$(call msg,"lrpdb pdb1 delete ") + - $(KUBECTL) apply -f $(LRPDBCLOSE1) + $(KUBECTL) wait --for jsonpath='{.status.openMode'}="MOUNTED" lrpdb pdb1 -n $(PDBNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) + $(KUBECTL) apply -f $(LRPDBDELETE1) + $(KUBECTL) wait --for=delete lrpdb pdb1 -n $(PDBNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) + @$(call msg,"lrpdb pdb1 delete") + $(KUBECTL) get lrpdb -n $(PDBNAMESPACE) + +run99.1: + $(KUBECTL) delete lrest cdb-dev -n cdbnamespace + $(KUBECTL) wait --for=delete lrest cdb-dev -n $(LRSNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) + $(KUBECTL) get lrest -n cdbnamespaace + $(KUBECTL) get lrpdb -n pdbnamespaace + +runall01: run00 run01.1 run01.2 run02.1 run02.2 run03.1 run03.2 run04.1 run04.2 run05.1 run06.1 run07.1 + + diff --git a/docs/multitenant/lrest-based/usecase/map_pdb1_resource.yaml b/docs/multitenant/lrest-based/usecase/map_pdb1_resource.yaml new file mode 100644 index 00000000..2cd57b87 --- /dev/null +++ b/docs/multitenant/lrest-based/usecase/map_pdb1_resource.yaml @@ -0,0 +1,49 @@ +apiVersion: database.oracle.com/v4 +kind: LRPDB +metadata: + name: pdb1 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + cdbName: "DB12" + pdbName: "pdbdev" + assertiveLrpdbDeletion: true + fileNameConversions: "NONE" + totalSize: "1G" + tempSize: "100M" + action: "Map" + adminpdbUser: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminpdbPass: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + lrpdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + lrpdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + lrpdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + cdbPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/lrest-based/usecase/map_pdb2_resource.yaml b/docs/multitenant/lrest-based/usecase/map_pdb2_resource.yaml new file mode 100644 index 00000000..bab614cf --- /dev/null +++ b/docs/multitenant/lrest-based/usecase/map_pdb2_resource.yaml @@ -0,0 +1,49 @@ +apiVersion: database.oracle.com/v4 +kind: LRPDB +metadata: + name: pdb2 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + cdbName: "DB12" + pdbName: "pdbprd" + assertiveLrpdbDeletion: true + fileNameConversions: "NONE" + totalSize: "1G" + tempSize: "100M" + action: "Map" + adminpdbUser: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminpdbPass: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + lrpdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + lrpdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + lrpdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + cdbPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/lrest-based/usecase/map_pdb3_resource.yaml b/docs/multitenant/lrest-based/usecase/map_pdb3_resource.yaml new file mode 100644 index 00000000..7bbae48d --- /dev/null +++ b/docs/multitenant/lrest-based/usecase/map_pdb3_resource.yaml @@ -0,0 +1,49 @@ +apiVersion: database.oracle.com/v4 +kind: LRPDB +metadata: + name: pdb3 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + cdbName: "DB12" + pdbName: "new_clone" + assertiveLrpdbDeletion: true + fileNameConversions: "NONE" + totalSize: "1G" + tempSize: "100M" + action: "Map" + adminpdbUser: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminpdbPass: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + lrpdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + lrpdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + lrpdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + cdbPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/lrest-based/usecase/open_pdb1_resource.yaml b/docs/multitenant/lrest-based/usecase/open_pdb1_resource.yaml new file mode 100644 index 00000000..a845a0bd --- /dev/null +++ b/docs/multitenant/lrest-based/usecase/open_pdb1_resource.yaml @@ -0,0 +1,47 @@ +apiVersion: database.oracle.com/v4 +kind: LRPDB +metadata: + name: pdb1 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + cdbName: "DB12" + pdbName: "pdbdev" + action: "Modify" + pdbState: "OPEN" + modifyOption: "READ WRITE" + adminpdbUser: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminpdbPass: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + lrpdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + lrpdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + lrpdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + cdbPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/lrest-based/usecase/open_pdb2_resource.yaml b/docs/multitenant/lrest-based/usecase/open_pdb2_resource.yaml new file mode 100644 index 00000000..9356184f --- /dev/null +++ b/docs/multitenant/lrest-based/usecase/open_pdb2_resource.yaml @@ -0,0 +1,47 @@ +apiVersion: database.oracle.com/v4 +kind: LRPDB +metadata: + name: pdb2 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + cdbName: "DB12" + pdbName: "pdbprd" + action: "Modify" + pdbState: "OPEN" + modifyOption: "READ WRITE" + adminpdbUser: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminpdbPass: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + lrpdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + lrpdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + lrpdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + cdbPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/lrest-based/usecase/open_pdb3_resource.yaml b/docs/multitenant/lrest-based/usecase/open_pdb3_resource.yaml new file mode 100644 index 00000000..1b8024ba --- /dev/null +++ b/docs/multitenant/lrest-based/usecase/open_pdb3_resource.yaml @@ -0,0 +1,47 @@ +apiVersion: database.oracle.com/v4 +kind: LRPDB +metadata: + name: pdb3 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + cdbName: "DB12" + pdbName: "new_clone" + action: "Modify" + pdbState: "OPEN" + modifyOption: "READ WRITE" + adminpdbUser: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminpdbPass: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + lrpdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + lrpdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + lrpdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + cdbPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/lrest-based/usecase/parameters.txt b/docs/multitenant/lrest-based/usecase/parameters.txt new file mode 100644 index 00000000..1f21ed38 --- /dev/null +++ b/docs/multitenant/lrest-based/usecase/parameters.txt @@ -0,0 +1,52 @@ + +######################## +## REST SERVER IMAGE ### +######################## + +LRESTIMG:container-registry.oracle.com/database/operator:lrest-241210-amd64 + +############################## +## TNS URL FOR CDB CREATION ## +############################## +TNSALIAS:"(DESCRIPTION=(CONNECT_TIMEOUT=90)(RETRY_COUNT=30)(RETRY_DELAY=10)(TRANSPORT_CONNECT_TIMEOUT=70)(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan12.testrac.com)(PORT=1521)(IP=V4_ONLY))(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan34.testrac.com)(PORT=1521)(IP=V4_ONLY))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=TESTORDS)))" + +########################################### +## CDB USER FOR PDB LIFECYCLE MANAGMENT ### +########################################### + +DBUSER:restdba +DBPASS:CLWKO655321 + +####################### +## HTTPS CREDENTIAL ### +####################### + +WBUSER:welcome +WBPASS:welcome1 + +##################### +## PDB ADMIN USER ### +##################### + +PDBUSR:Citizenkane +PDBPWD:Rosebud + +################### +### NAMESPACES #### +################### + +PDBNAMESPACE:pdbnamespace +LRSNAMESPACE:cdbnamespace + + +#################### +### COMPANY NAME ### +#################### + +COMPANY:oracle + +#################### +### APIVERSION ### +#################### + +APIVERSION:v4 diff --git a/docs/multitenant/lrest-based/usecase/pdbnamespace_binding.yaml b/docs/multitenant/lrest-based/usecase/pdbnamespace_binding.yaml new file mode 100644 index 00000000..5af79ed6 --- /dev/null +++ b/docs/multitenant/lrest-based/usecase/pdbnamespace_binding.yaml @@ -0,0 +1,13 @@ +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: oracle-database-operator-oracle-database-operator-manager-rolebinding1 + namespace: pdbnamespace +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: oracle-database-operator-manager-role +subjects: +- kind: ServiceAccount + name: default + namespace: oracle-database-operator-system diff --git a/docs/multitenant/lrest-based/usecase/plug_pdb1_resource.yaml b/docs/multitenant/lrest-based/usecase/plug_pdb1_resource.yaml new file mode 100644 index 00000000..d7d310db --- /dev/null +++ b/docs/multitenant/lrest-based/usecase/plug_pdb1_resource.yaml @@ -0,0 +1,54 @@ +apiVersion: database.oracle.com/v4 +kind: LRPDB +metadata: + name: pdb1 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + cdbName: "DB12" + pdbName: "pdbdev" + xmlFileName: "/tmp/pdb.xml" + action: "plug" + fileNameConversions: "NONE" + sourceFileNameConversions: "NONE" + copyAction: "MOVE" + totalSize: "1G" + tempSize: "100M" + assertiveLrpdbDeletion: true + pdbconfigmap: "config-map-pdb" + action: "Plug" + adminpdbUser: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminpdbPass: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + lrpdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + lrpdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + lrpdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + cdbPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/lrest-based/usecase/unplug_pdb1_resource.yaml b/docs/multitenant/lrest-based/usecase/unplug_pdb1_resource.yaml new file mode 100644 index 00000000..a5da5a57 --- /dev/null +++ b/docs/multitenant/lrest-based/usecase/unplug_pdb1_resource.yaml @@ -0,0 +1,46 @@ +apiVersion: database.oracle.com/v4 +kind: LRPDB +metadata: + name: pdb1 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + cdbName: "DB12" + pdbName: "pdbdev" + xmlFileName: "/tmp/pdb.xml" + action: "Unplug" + adminpdbUser: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminpdbPass: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + lrpdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + lrpdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + lrpdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + cdbPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/ords-based/NamespaceSeg.md b/docs/multitenant/ords-based/NamespaceSeg.md new file mode 100644 index 00000000..6738fe56 --- /dev/null +++ b/docs/multitenant/ords-based/NamespaceSeg.md @@ -0,0 +1,14 @@ + + +# Namespace segregation + +With the namespace segregation pdb controller and cdb controller run in different namespaces. The new functionality introduces a new parameter (the cdb namespace) in pdb crd definition. In case you don't need the namespace segregation you have to sepcify the namespace name that you are using for yours crd and pods anyway. Refer to usercase01 and usecase02 to see single namespace configuration. Refer to usecase03 to see examples of namespace segregation. + +# Secrets + +In order to use multiple namespace we need to create approriate secrets in each namespace. Tls certificate secrets must be created in all namespaces (db-ca db-tls). + +![general_schema](./images/K8S_NAMESPACE_SEG.png) + + + diff --git a/docs/multitenant/ords-based/README.md b/docs/multitenant/ords-based/README.md new file mode 100644 index 00000000..edfd0208 --- /dev/null +++ b/docs/multitenant/ords-based/README.md @@ -0,0 +1,411 @@ + + +# Oracle Multitenant Database Controllers + +The Oracle Database Operator for Kubernetes uses two controllers to manage the [Pluggable Database lifecycle][oradocpdb] + +- CDB controller +- PDB controller + +By using CDB/PDB controllers, you can perform the following actions **CREATE**, **MODIFY(OPEN/COSE)**, **DELETE**, **CLONE**, **PLUG** and **UNPLUG** against pluggable database + +Examples are located under the following directories: + +- the directories [`Usecase`](./usecase/) and [`usecase01`](./usecase01/) contain a [configuration file](./usecase/parameters.txt) where you can specify all the details of your environment. A [`makefile`](./usecase/makefile) takes this file as input to generate all of the `yaml` files. There is no need to edit `yaml` files one by one. +- [Singlenamespace provisioning](./provisioning/singlenamespace/) This file contains base example files that you can use to manage the PDB and CDB within a single namespace. +- [Multinamespace provisioning](./provisioning/multinamespace/) This file contains base example files that you can use to manage the PDB and CDB in different namespaces. +- [Usecase01](./usecase01/README.md) [Usecase02](./usecase02/README.md) This file contains other step-by-step examples; + +Automatic `yaml` generation is not available for the directory `usecase02` and provisioning directories. + +**NOTE** the CDB controller is not intended to manage the container database. The CDB controller is meant to provide a pod with a REST server connected to the container database that you can use to manage PDBs. + + +## Macro steps for setup + +- Deploy the Oracle Database Operator (operator, or `OraOperator`) +- [Create Ords based image for CDB pod](./provisioning/ords_image.md) +- [Container RDBMB user creation](#prepare-the-container-database-for-pdb-lifecycle-management-pdb-lm) +- Create certificates for https connection +- Create secrets for credentials and certificates +- Create CDB pod using the Ords based image + +## Oracle DB Operator Multitenant Database Controller Deployment + +To deploy `OraOperator`, use this [Oracle Database Operator for Kubernetes](https://github.com/oracle/oracle-database-operator/blob/main/README.md) step-by-step procedure. + +After the **Oracle Database Operator** is deployed, you can see the Oracle Database (DB) Operator Pods running in the Kubernetes Cluster. The multitenant controllers are deployed as part of the `OraOperator` deployment. You can see the CRDs (Custom Resource Definition) for the CDB and PDBs in the list of CRDs. The following output is an example of such a deployment: + +```bash +[root@test-server oracle-database-operator]# kubectl get ns +NAME STATUS AGE +cert-manager Active 32h +default Active 245d +kube-node-lease Active 245d +kube-public Active 245d +kube-system Active 245d +oracle-database-operator-system Active 24h <---- namespace to deploy the Oracle Database Operator + +[root@test-server oracle-database-operator]# kubectl get all -n oracle-database-operator-system +NAME READY STATUS RESTARTS AGE +pod/oracle-database-operator-controller-manager-665874bd57-dlhls 1/1 Running 0 28s +pod/oracle-database-operator-controller-manager-665874bd57-g2cgw 1/1 Running 0 28s +pod/oracle-database-operator-controller-manager-665874bd57-q42f8 1/1 Running 0 28s + +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +service/oracle-database-operator-controller-manager-metrics-service ClusterIP 10.96.130.124 8443/TCP 29s +service/oracle-database-operator-webhook-service ClusterIP 10.96.4.104 443/TCP 29s + +NAME READY UP-TO-DATE AVAILABLE AGE +deployment.apps/oracle-database-operator-controller-manager 3/3 3 3 29s + +NAME DESIRED CURRENT READY AGE +replicaset.apps/oracle-database-operator-controller-manager-665874bd57 3 3 3 29s +[root@docker-test-server oracle-database-operator]# + +[root@test-server oracle-database-operator]# kubectl get crd +NAME CREATED AT +autonomouscontainerdatabases.database.oracle.com 2022-06-22T01:21:36Z +autonomousdatabasebackups.database.oracle.com 2022-06-22T01:21:36Z +autonomousdatabaserestores.database.oracle.com 2022-06-22T01:21:37Z +autonomousdatabases.database.oracle.com 2022-06-22T01:21:37Z +cdbs.database.oracle.com 2022-06-22T01:21:37Z <---- +certificaterequests.cert-manager.io 2022-06-21T17:03:46Z +certificates.cert-manager.io 2022-06-21T17:03:47Z +challenges.acme.cert-manager.io 2022-06-21T17:03:47Z +clusterissuers.cert-manager.io 2022-06-21T17:03:48Z +dbcssystems.database.oracle.com 2022-06-22T01:21:38Z +issuers.cert-manager.io 2022-06-21T17:03:49Z +oraclerestdataservices.database.oracle.com 2022-06-22T01:21:38Z +orders.acme.cert-manager.io 2022-06-21T17:03:49Z +pdbs.database.oracle.com 2022-06-22T01:21:39Z <--- +shardingdatabases.database.oracle.com 2022-06-22T01:21:39Z +singleinstancedatabases.database.oracle.com 2022-06-22T01:21:40Z +``` + + +## Prerequisites to manage PDB Life Cycle using Oracle DB Operator Multitenant Database Controller + +* [Prepare the container database (CDB) for PDB Lifecycle Management or PDB-LM](#prepare-cdb-for-pdb-lifecycle-management-pdb-lm) +* [Oracle REST Data Service or ORDS Image](#oracle-rest-data-service-ords-image) +* [Kubernetes Secrets](#kubernetes-secrets) +* [Kubernetes CRD for CDB](#cdb-crd) +* [Kubernetes CRD for PDB](#pdb-crd) + +## Prepare the container database for PDB Lifecycle Management (PDB-LM) + +Pluggable Database (PDB) management operations are performed in the Container Database (CDB). These operations include **create**, **clone**, **plug**, **unplug**, **delete**, **modify** and **map pdb**. + +To perform PDB lifecycle management operations, you must first use the following steps to define the default CDB administrator credentials on target CDBs: + +Create the CDB administrator user and grant the required privileges. In this example, the user is `C##DBAPI_CDB_ADMIN`. However, any suitable common username can be used. + +```SQL +SQL> conn /as sysdba + +-- Create following users at the database level: + +ALTER SESSION SET "_oracle_script"=true; +DROP USER C##DBAPI_CDB_ADMIN cascade; +CREATE USER C##DBAPI_CDB_ADMIN IDENTIFIED BY CONTAINER=ALL ACCOUNT UNLOCK; +GRANT SYSOPER TO C##DBAPI_CDB_ADMIN CONTAINER = ALL; +GRANT SYSDBA TO C##DBAPI_CDB_ADMIN CONTAINER = ALL; +GRANT CREATE SESSION TO C##DBAPI_CDB_ADMIN CONTAINER = ALL; + + +-- Verify the account status of the following usernames. They should not be in locked status: + +col username for a30 +col account_status for a30 +select username, account_status from dba_users where username in ('ORDS_PUBLIC_USER','C##DBAPI_CDB_ADMIN','APEX_PUBLIC_USER','APEX_REST_PUBLIC_USER'); +``` + +## OCI OKE (Kubernetes Cluster) + +You can use an [OKE in Oracle Cloud Infrastructure][okelink] to configure the controllers for PDB lifecycle management. **Note that there is no restriction about container database location; it can be anywhere (on Cloud or on-premises).** +To quickly create an OKE cluster in your OCI cloud environment you can use the following [link](./provisioning/quickOKEcreation.md). +In this setup example [provisioning example setup](./provisioning/example_setup_using_oci_oke_cluster.md), the Container Database is running on an OCI Exadata Database Cluster. + + +## Oracle REST Data Service (ORDS) Image + +The PDB Database controllers require a pod running a dedicated REST server image based on [ORDS][ordsdoc]. Read the following [document on ORDS images](./provisioning/ords_image.md) to build the ORDS images. + + +## Kubernetes Secrets + + Multitenant Controllers use Kubernetes Secrets to store the required credential and HTTPS certificates. + + **Note** In multi-namespace environments you must create specific Secrets for each namespaces. + +### Secrets for CERTIFICATES + +Create the certificates and key on your local host, and then use them to create the Kubernetes Secret. + +```bash +openssl genrsa -out ca.key 2048 +openssl req -new -x509 -days 365 -key ca.key -subj "/C=US/ST=California/L=SanFrancisco/O=oracle /CN=cdb-dev-ords /CN=localhost Root CA " -out ca.crt +openssl req -newkey rsa:2048 -nodes -keyout tls.key -subj "/C=US/ST=California/L=SanFrancisco/O=oracle /CN=cdb-dev-ords /CN=localhost" -out server.csr +echo "subjectAltName=DNS:cdb-dev-ords,DNS:www.example.com" > extfile.txt +openssl x509 -req -extfile extfile.txt -days 365 -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out tls.crt +``` + +```bash +kubectl create secret tls db-tls --key="tls.key" --cert="tls.crt" -n oracle-database-operator-system +kubectl create secret generic db-ca --from-file=ca.crt -n oracle-database-operator-system +``` + +image_not_found + +**Note:** Remove temporary files after successfful Secret creation. + +### Secrets for CDB CRD + + **Note:** base64 encoded secrets are no longer supported; use OpenSSL secrets as documented in the following section. After successful creation of the CDB Resource, the CDB and PDB Secrets can be deleted from the Kubernetes system. Don't leave plaintext files containing sensitive data on disk. After loading the Secret, remove the plaintext file or move it to secure storage. + + ```bash + +export PRVKEY=ca.key +export PUBKEY=public.pem +WBUSERFILE=wbuser.txt +WBPASSFILE=wbpass.txt +CDBUSRFILE=cdbusr.txt +CDBPWDFILE=cdbpwd.txt +SYSPWDFILE=syspwd.txt +ORDPWDFILE=ordpwd.txt +PDBUSRFILE=pdbusr.txt +PDBPWDFILE=pdbpwd.txt + +# Webuser credential +echo [WBUSER] > ${WBUSERFILE} +echo [WBPASS] > ${WBPASSFILE} + +# CDB admin user credentioan +echo [CDBPWD] > ${CDBPWDFILE} +echo [CDBUSR] > ${CDBUSRFILE} + +# SYS Password +echo [SYSPWD] > ${SYSPWDFILE} + +# Ords Password +echo [ORDPWD] > ${ORDPWDFILE} + +## PDB admin credential +echo [PDBUSR] > ${PDBUSRFILE} +echo [PDBPWD] > ${PDBPWDFILE} + +#Secrets creation for pub and priv keys +openssl rsa -in ${PRVKEY} -outform PEM -pubout -out ${PUBKEY} +kubectl create secret generic pubkey --from-file=publicKey=${PUBKEY} -n ${CDBNAMESPACE} +kubectl create secret generic prvkey --from-file=privateKey=${PRVKEY} -n ${CDBNAMESPACE} +kubectl create secret generic prvkey --from-file=privateKey="${PRVKEY}" -n ${PDBNAMESPACE} + +#Password encryption +openssl rsautl -encrypt -pubin -inkey ${PUBKEY} -in ${WBUSERFILE} |base64 > e_${WBUSERFILE} +openssl rsautl -encrypt -pubin -inkey ${PUBKEY} -in ${WBPASSFILE} |base64 > e_${WBPASSFILE} +openssl rsautl -encrypt -pubin -inkey ${PUBKEY} -in ${CDBPWDFILE} |base64 > e_${CDBPWDFILE} +openssl rsautl -encrypt -pubin -inkey ${PUBKEY} -in ${CDBUSRFILE} |base64 > e_${CDBUSRFILE} +openssl rsautl -encrypt -pubin -inkey ${PUBKEY} -in ${SYSPWDFILE} |base64 > e_${SYSPWDFILE} +openssl rsautl -encrypt -pubin -inkey ${PUBKEY} -in ${ORDPWDFILE} |base64 > e_${ORDPWDFILE} +openssl rsautl -encrypt -pubin -inkey ${PUBKEY} -in ${PDBUSRFILE} |base64 > e_${PDBUSRFILE} +openssl rsautl -encrypt -pubin -inkey ${PUBKEY} -in ${PDBPWDFILE} |base64 > e_${PDBPWDFILE} + +#Ecrypted secrets creation +kubectl create secret generic wbuser --from-file=e_${WBUSERFILE} -n ${CDBNAMESPACE} +kubectl create secret generic wbpass --from-file=e_${WBPASSFILE} -n ${CDBNAMESPACE} +kubectl create secret generic wbuser --from-file=e_${WBUSERFILE} -n ${PDBNAMESPACE} +kubectl create secret generic wbpass --from-file=e_${WBPASSFILE} -n ${PDBNAMESPACE} +kubectl create secret generic cdbpwd --from-file=e_${CDBPWDFILE} -n ${CDBNAMESPACE} +kubectl create secret generic cdbusr --from-file=e_${CDBUSRFILE} -n ${CDBNAMESPACE} +kubectl create secret generic syspwd --from-file=e_${SYSPWDFILE} -n ${CDBNAMESPACE} +kubectl create secret generic ordpwd --from-file=e_${ORDPWDFILE} -n ${CDBNAMESPACE} +kubectl create secret generic pdbusr --from-file=e_${PDBUSRFILE} -n ${PDBNAMESPACE} +kubectl create secret generic pdbpwd --from-file=e_${PDBPWDFILE} -n ${PDBNAMESPACE} + +#Get rid of the swap files +rm ${WBUSERFILE} ${WBPASSFILE} ${CDBPWDFILE} ${CDBUSRFILE} \ + ${SYSPWDFILE} ${ORDPWDFILE} ${PDBUSRFILE} ${PDBPWDFILE} \ + e_${WBUSERFILE} e_${WBPASSFILE} e_${CDBPWDFILE} e_${CDBUSRFILE} \ + e_${SYSPWDFILE} e_${ORDPWDFILE} e_${PDBUSRFILE} e_${PDBPWDFILE} +``` + +Check Secrets details + +```bash +kubectl describe secrets syspwd -n cdbnamespace +Name: syspwd +Namespace: cdbnamespace +Labels: +Annotations: + +Type: Opaque + +Data +==== +e_syspwd.txt: 349 bytes +``` +Example of `yaml` file Secret section: + +```yaml +[...] + sysAdminPwd: + secret: + secretName: "syspwd" + key: "e_syspwd.txt" + ordsPwd: + secret: + secretName: "ordpwd" + key: "e_ordpwd.txt" +[...] +``` + +## CDB CRD + +The Oracle Database Operator Multitenant Controller creates the CDB as a custom resource object kind that models a target CDB as a native Kubernetes object. This object kind is used only to create Pods to connect to the target CDB to perform PDB-LM operations. Each CDB resource follows the CDB CRD as defined here: [`config/crd/bases/database.oracle.com_cdbs.yaml`](../../../config/crd/bases/database.oracle.com_cdbs.yaml) + +To create a CDB CRD, use this example`.yaml` file: [`cdb_create.yaml`](../multitenant/provisioning/singlenamespace/cdb_create.yaml) + +**Note:** The password and username fields in this *cdb.yaml* Yaml are the Kubernetes Secrets created earlier in this procedure. For more information, see the section [Kubernetes Secrets](https://kubernetes.io/docs/concepts/configuration/secret/). To understand more about creating secrets for pulling images from a Docker private registry, see [Kubernetes Private Registry Documenation]( https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/). + +Create a CDB CRD Resource example + +```bash +kubectl apply -f cdb_create.yaml +``` + +see [usecase01][uc01] and usecase02[uc02] for more information about file configuration + +## PDB CRD + +The Oracle Database Operator Multitenant Controller creates the PDB object kind as a custom resource that models a PDB as a native Kubernetes object. There is a one-to-one mapping between the actual PDB and the Kubernetes PDB Custom Resource. You cannot have more than one Kubernetes resource for a target PDB. This PDB resource can be used to perform PDB-LM operations by specifying the action attribute in the PDB Specs. Each PDB resource follows the PDB CRD as defined here: [config/crd/bases/database.oracle.com_pdbs.yaml](../../../config/crd/bases/database.oracle.com_pdbs.yaml) + +Yaml file [pdb_create.yaml](../multitenant/provisioning/singlenamespace/pdb_create.yaml) to create a pdb + +```bash +kubectl apply -f pdb_create.yaml +``` + +## CRD TABLE PARAMETERS + +| yaml file parameters | value | description /ords parameter | CRD | +|------------------ |--------------------------- |-------------------------------------------------------------------------------|-----------| +| dbserver | or | [--db-hostname][1] | CDB | +| dbTnsurl | | [--db-custom-url/db.customURL][dbtnsurl] | CDB | +| port | | [--db-port][2] | CDB | +| cdbName | | Container Name | CDB | +| name | | ORDS podname prefix in `cdb.yaml` | CDB | +| name | | Pdb resource in `pdb.yaml` | PDB | +| ordsImage | ords-dboper:latest | ORDS pod public container registry | CDB | +| pdbName | | Pluggable database (PDB) name | Container database (CDB) | +| servicename | | [--db-servicename][3] | CDB | +| sysadmin_user | | [--admin-user][adminuser] | CDB | +| sysadmin_pwd | | [--password-stdin][pwdstdin] | CDB | +| cdbadmin_user | | [db.cdb.adminUser][1] | CDB | +| cdbadmin_pwd | | [db.cdb.adminUser.password][cdbadminpwd] | CDB | +| webserver_user | | [https user][http] NOT A DB USER | CDB PDB | +| webserver_pwd | | [http user password][http] | CDB PDB | +| ords_pwd | | [ORDS_PUBLIC_USER password][public_user] | CDB | +| pdbTlsKey | | [standalone.https.cert.key][key] | PDB | +| pdbTlsCrt | | [standalone.https.cert][cr] | PDB | +| pdbTlsCat | | certificate authority | PDB | +| cdbTlsKey | | [standalone.https.cert.key][key] | CDB | +| cdbTlsCrt | | [standalone.https.cert][cr] | CDB | +| cdbTlsCat | | Certificate authority | CDB | +| cdbOrdsPrvKey | | Private key | CDB | +| pdbOrdsPrvKey | | Private key | PDB | +| xmlFileName | | Path for the unplug and plug operation | PDB | +| srcPdbName | | Name of the database that you want to be cloned | PDB | +| action | | Create open close delete clone plug unplug and map | PDB | +| deletePdbCascade | boolean | Delete PDBs cascade during CDB deletion | CDB | +| assertivePdbDeletion | boolean | Deleting the PDB crd means deleting the PDB as well | PDB | +| fileNameConversions | | Used for database cloning | PDB | +| totalSize | | `dbsize` | PDB | +| pdbState | | Change PDB state | PDB | +| modifyOption | | To be used along with `pdbState` | PDB | +| dropAction | | Delete datafiles during PDB deletion | PDB | +| sourceFileNameConversions | | [sourceFileNameConversions(optional): string][4] | PDB | +| tdeKeystorePath | | [tdeKeystorePath][tdeKeystorePath] | N/A | +| tdeExport | | [tdeExport] | N/A ] +| tdeSecret | | [tdeSecret][tdeSecret] | N/A | +| tdePassword | | [tdeSecret][tdeSecret] | N/A | + + + + +## Usecases files list + +### Single Namespace + +1. [Create CDB](./provisioning/singlenamespace/cdb_create.yaml) +2. [Create PDB](./provisioning/singlenamespace/pdb_create.yaml) +3. [Clone PDB](./provisioning/singlenamespace/pdb_clone.yaml) +4. [Open PDB](./provisioning/singlenamespace/pdb_open.yaml) +4. [Close PDB](./provisioning/singlenamespace/pdb_close.yaml) +5. [Delete PDB](./provisioning/singlenamespace/pdb_delete.yaml) +6. [Unplug PDB](./provisioning/singlenamespace/pdb_unplug.yaml) +7. [Plug PDB](./provisioning/singlenamespace/pdb_plug.yaml) + +### Multiple namespace (cdbnamespace,dbnamespace) + +1. [Create CDB](./provisioning/multinamespace/cdb_create.yaml) +2. [Create PDB](./provisioning/multinamespace/pdb_create.yaml) +3. [Clone PDB](./provisioning/multinamespace/pdb_clone.yaml) +4. [Open PDB](./provisioning/multinamespace/pdb_open.yaml) +4. [Close PDB](./provisioning/multinamespace/pdb_close.yaml) +5. [Delete PDB](./provisioning/multinamespace/pdb_delete.yaml) +6. [Unplug PDB](./provisioning/multinamespace/pdb_unplug.yaml) + +## Known issues + + - ORDS installation failure if pluaggable databases in the container db are not openedS + + - Version 1.1.0: encoded password for https authentication may include carriage return as consequence the https request fails with http 404 error. W/A generate encoded password using **printf** instead of **echo**. + + - pdb controller authentication suddenly fails without any system change. Check the certificate expiration date **openssl .... -days 365** + + - Nothing happens after applying cdb yaml files: Make sure to have properly configured the WHATCH_NAMESPACE list in the operator yaml file + + [okelink]:https://docs.oracle.com/en-us/iaas/Content/ContEng/Concepts/contengoverview.htm + + [ordsdoc]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/23.1/index.html + + [uc01]:../multitenant/usecase01/README.md + + [uc02]:../multitenant/usecase02/README.md + + [oradocpdb]:https://docs.oracle.com/en/database/oracle/oracle-database/21/multi/introduction-to-the-multitenant-architecture.html#GUID-AB84D6C9-4BBE-4D36-992F-2BB85739329F + + [1]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/installing-and-configuring-oracle-rest-data-services.html#GUID-E9625FAB-9BC8-468B-9FF9-443C88D76FA1:~:text=Table%202%2D2%20Command%20Options%20for%20Command%2DLine%20Interface%20Installation + + [2]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/installing-and-configuring-oracle-rest-data-services.html#GUID-E9625FAB-9BC8-468B-9FF9-443C88D76FA1:~:text=Table%202%2D2%20Command%20Options%20for%20Command%2DLine%20Interface%20Installation + + [3]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/installing-and-configuring-oracle-rest-data-services.html#GUID-DAA027FA-A4A6-43E1-B8DD-C92B330C2341:~:text=%2D%2Ddb%2Dservicename%20%3Cstring%3E + + [4]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.3/orrst/op-database-pdbs-post.html + +[adminuser]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/installing-and-configuring-oracle-rest-data-services.html#GUID-A9AED253-4EEC-4E13-A0C4-B7CE82EC1C22:~:text=Table%202%2D6%20Command%20Options%20for%20Uninstall%20CLI + +[public_user]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/using-multitenant-architecture-oracle-rest-data-services.html#GUID-E64A141A-A71F-4979-8D33-C5F8496D3C19:~:text=Preinstallation%20Tasks%20for%20Oracle%20REST%20Data%20Services%20CDB%20Installation + +[key]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/about-REST-configuration-files.html#GUID-006F916B-8594-4A78-B500-BB85F35C12A0:~:text=standalone.https.cert.key + +[cr]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/about-REST-configuration-files.html#GUID-006F916B-8594-4A78-B500-BB85F35C12A0 + +[cdbadminpwd]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/about-REST-configuration-files.html#GUID-006F916B-8594-4A78-B500-BB85F35C12A0:~:text=Table%20C%2D1%20Oracle%20REST%20Data%20Services%20Configuration%20Settings + + +[pwdstdin]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/installing-and-configuring-oracle-rest-data-services.html#GUID-88479C84-CAC1-4133-A33E-7995A645EC05:~:text=default%20database%20pool.-,2.1.4.1%20Understanding%20Command%20Options%20for%20Command%2DLine%20Interface%20Installation,-Table%202%2D2 + +[http]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/installing-and-configuring-oracle-rest-data-services.html#GUID-BEECC057-A8F5-4EAB-B88E-9828C2809CD8:~:text=Example%3A%20delete%20%5B%2D%2Dglobal%5D-,user%20add,-Add%20a%20user + +[dbtnsurl]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/installing-and-configuring-oracle-rest-data-services.html#GUID-A9AED253-4EEC-4E13-A0C4-B7CE82EC1C22 + +[tdeKeystorePath]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/21.4/orrst/op-database-pdbs-pdb_name-post.html + +[tdeSecret]:https://docs.oracle.com/en/database/oracle/oracle-database/19/sqlrf/ADMINISTER-KEY-MANAGEMENT.html#GUID-E5B2746F-19DC-4E94-83EC-A6A5C84A3EA9 +~ + + + + + diff --git a/docs/multitenant/images/K8S_NAMESPACE_SEG.png b/docs/multitenant/ords-based/images/K8S_NAMESPACE_SEG.png similarity index 100% rename from docs/multitenant/images/K8S_NAMESPACE_SEG.png rename to docs/multitenant/ords-based/images/K8S_NAMESPACE_SEG.png diff --git a/docs/multitenant/images/K8S_SECURE1.png b/docs/multitenant/ords-based/images/K8S_SECURE1.png similarity index 100% rename from docs/multitenant/images/K8S_SECURE1.png rename to docs/multitenant/ords-based/images/K8S_SECURE1.png diff --git a/docs/multitenant/images/K8S_SECURE2.png b/docs/multitenant/ords-based/images/K8S_SECURE2.png similarity index 100% rename from docs/multitenant/images/K8S_SECURE2.png rename to docs/multitenant/ords-based/images/K8S_SECURE2.png diff --git a/docs/multitenant/images/K8S_SECURE3.png b/docs/multitenant/ords-based/images/K8S_SECURE3.png similarity index 100% rename from docs/multitenant/images/K8S_SECURE3.png rename to docs/multitenant/ords-based/images/K8S_SECURE3.png diff --git a/docs/multitenant/images/K8S_SECURE4.png b/docs/multitenant/ords-based/images/K8S_SECURE4.png similarity index 100% rename from docs/multitenant/images/K8S_SECURE4.png rename to docs/multitenant/ords-based/images/K8S_SECURE4.png diff --git a/docs/multitenant/ords-based/images/makerunall.png b/docs/multitenant/ords-based/images/makerunall.png new file mode 100644 index 0000000000000000000000000000000000000000..ab856f90ca3a29bbaece7f4d80b0056801243613 GIT binary patch literal 211874 zcmdSB2T+r17cLq_K|w^6CJG3sAfSk10fZ=31Zg52A|TR1sY*#yq)11kC?y~u(xijb zNLNvMM|zds2_?x{5%>P=5$3-}|ohtfzc0Rg~mtsadHJ2n4PC zO=&d*V$WFwg5v8wN_gcmv4tM~wZ~pkUVR_@&u!m*ANcvGgN&BL9oq*E&W3g-2vZwd zYm>9~#&#wqHuh$=4)YW^2?XKE~ueVPCp~pW2LVZXQ=XbW~iAM?&MEi|sWpe5AC9d;|Wf#5+x!;1?ehh5ZwD zSMbsm9pU1lq-VJyqiTllLmsxUDlSQVYjgd_d@hrvW9P@Rbz5X=YRQLGjYIS?8oZxb zj2jakzAHcYOsJN{*k7pQwnjK_b?q({-m|KQ;pg8k)^;hcDCxW~&)>%*Ifx>1z3FbC z#kcF;@<)#D#yS>=AC42MiV&1*@}#NUdV0}xu*k(Rv-ZS$OSeWpj7|v?F$hs&#W@s`Ec#070PQHH*S2=)G`we?K#H6-pk=}TBM?*pQ)|Gx1e^M zo#xf*!Me}a_b@YDpK*#GA2w2PbrqDYiM%$T6h-O%XylOHFEx~RCHdRevg95LC9#P( z1YR;wWT7^}5ht4ph77CNes1HORktu=aSoexE^WJ1#F~gruZD8UJAz{Zyk=&~4|@D{ zrGigqOm$Y4_ZVmy(#%&6sjN-YA>sm^c(;0oHJA0zqj48cA6GZ7+$E;?)6(OmYsUFU zSVpggR~-Jv56|yzh^?tVdp1(-jt4WTlXSS{H3ijhC(7+v1c$ifLqpCP)Z-h1IJ?2? zdkCwUk2jjR@mLAku&Ah+VhYj{Gk&JWGx!~sqNPP;roG)gF|*)co4!mAgH60K`8C_Q zK96}UanCf7aND;rdX3`g0FkiJpG_fX)8}g*5D@9Ke$K0G<22&Bp>+WLe7Rprp^afr zUkOXpo0L{`r-=QbXl&4c@D=Wr2DZ|b#kwqwv|R{7Y3U)Q7lY4tBMzO*COIlKAeZFi zXzFSE*tyz~aNnw2x19EDZ#3e6l)3KvUVqWtZD*N{{RvB7#LNx%`u7SU1gpysmSHhj zT{`A8N$yDA+1YsYGls^-w3e28)6*HV3MkX9N0D)@PE9B9SW!>7gL(%JRF7%r-ZNPb zDwZuLNxc5}@eAES*>=}?LoK&bgQKvU>H=naLpWIW@rP0Oi#dNw+cuRlGFCmz?49dD z8DCa(RM4jPbYfW+T3qk_?W&|-t*Q+4yOQ@GS*WZHI9+3^v?WFxi{xMn;uwjJU_KXk z%-efx4Dl3m>vm1t0Fg;^yg>%7awyMvzH0}mlAkw~>trw(22am;T&7@YOA7)) z$ItgrC;mFT5iyu2`K*QNn{TtJP>^0_|yBSGqwv;0gQyd7E(tEqU9Xb3EuFoN_ zU>K`*=k^r$&cR_Z#tG!u(5dnwJk{LpPMvUYaEQyur%0MFh@U<8^zz`4~#Xeh%-#EW!liL_I<&WqT=SxyjPM2*sME4**;0rNQ zpSyFj9@#x#o72}8YGV`kr7|%=JWXBim%3-b{*f3&dKM@bbbbXzVjM_L9GpXus1YB2 z=I!qna~j-gPcy0O>{~;otgcyaFCa-gh`8B#jGThWDTa;?eDVN^J%YG_ZC+WXP_Ku3 zVLR9F`Rxa7kZoUOP>J*O>pNw}ROnTEk?<28$e5`>wknU77OVk>ICY^X%~=iPpSqDf z5wsE1McQ~1K+VsurL8++c$;btO{EfLUbC{kD%0ACdR&roOyW3o(%~OS%+TJaRHsh4 z@az>;zKZzlk)|1+mGeBto9eBfpGnQaf;Y#D&a<5#Z^WxtHuSLt*$#NYO^dO6t8nYy z$jpHxKN>h|-+Yz*wYIPh^AoyPWA9J=+HaZ9`s~RQ%2xJ);=QYpLXQ{A)(2S__Q1Ol zgQ|}CGx%yPv8z67%DTqfh{44ptN8;sx+v_(y}#<-fe=U6eY;e&j*?{dBPPvky$f#p z^ZV^Yq21yqN1MN@*yr3FEgKy)jK`(6)VqR;J5v%R=I|h;Pw?SaNTu2wmI*b zv-40#U<0lMNt&7Z)QFT*klEX?g(vT+;Yb$V^!vz`#9oKZlXST7u0daFHh%M@slI7>+=b4pF8C>w>EC{w(YUL6AEkKlaKnA zkjj^L(_$zmQcRc+v8#^IV=~ujB-aU(R;EUZ5bKZ11OtmCvFnv_~TzbmzC7ajj>hdfQ&iYi8DdSs$pI zl=5C!&n!QTzH#5^iOA*?SDI-rlG_rWE&k!P@+Wt4Fn_fHp*v@Me3X!n6eJ@x{GKn4 zJj%>cxt4G^OM|ddoWM5uc+ZeVh^Z=ohUdiza*w#h%QB@5{T)NAK0Vr3q1 zn}~3;DoEZ0gka+oe?{+Injwbfk5hnj^Q!MFg=)U5IyW09~ zS|}d>*?r1_WxY~FE7MNqN9SuJwS5hqBxOqdImFtL{j;3@LTlHCdZqsS4W~0c_#@(f zgPT~1CG^tkKmDJ@I6XrABG1|Px1+o=bPI2NEHtTSdg1M@TS`2uTX1h_^lM*(;1y!* zJB_q>72Y@%UKIla8>c5?&U4lH#gVOaVtGeCp-jA7TpyP*vQ_$PdeLO(JQJ!X<^8OC z3pjasKU^`+zDY=s3zEHm|30>TAcC5PCjR`A+pF>M2YGpUD|XW18K`vkzv@6N(gnx9 z8!v^dU#;f*g#!lWk7l}r@(K%$=YAGd<}55c zwCc{0d5pu5(0Dw4-@LA_E( zt+9wY@WM+Jhz}n=Je7<;^zp#28Sf7tcoP<*Qsd)ywI(Zj$HcJhrl5E<+jA+#L1fX7 zzFsideY}H?WgaCmJFi#Qg_oFJjk;JKA&BzI$T<7FQ!BJyaDZ?z(p3$`xjm`zqrt1%;64X#ce}r?S<991;iM%lZ0JyRG**yP#%j zrUr`ac{I|so>f*#UPEuQK*wcnd6$rIL%OT9h>M2HZiBT@^bcqf&$sh#9#}O z7wxg&urSX%$x1g8nprY3`Lpq@EiL4i8YQ+4?A^Qfp`D$il~sDM0SXo5QS-I}bMN<_G4KEtzbBN&B0lPIE)xqPR ziG(dx6pAiiC*W?H#@9gQRMPi8;?AkP=v#c%dG2^hyM=|7Rhu?D6O(Mz6-=B_Iy=>dB zXlQ8YGUny!*{o|@cB+&#!a=%ulVO4#NTWr| zd>ys4_SX>+Jc$D(&Or>+d)wRE4$(KmoiO_zDde&!krEg8^uU3XxTGXcL{od3rcAxi zYvmMmH6^9MEP>Ofy(A?ieSGfM2x{PdZrAqkT*0wq6T>W!A=?`p!X36R7jMhU$9LzN z()?N;|B+B#E&a84>{SVYwY8OCY2miDKA)_bAWKiLs-eNOvAKDezPTsg(!BYdjI=ZZ zJG&ArUvN7~ZEHY2^mH?vYR`4%b9yI|)a1{eJ^N~r{^dIMQ~`3QQusHQv>5dkSZn3n z-w*rww&;cshUgZJ#IO^I#G7z@;Y-kdnugA#>88NR_{u?sp%+eVwB$P@iIwb}Xm4cjvTn-kyP zP#imUtgEZ*-uU-7IrR~(xLnglStX@|IyyQF_(fy4jj@Qxi>86;1qwBt@(pX2inc46 zT5g&j^={n1e^Qv07VgmS@UVjoE-N<|3XD2mOed^G&L*mX=v?hXUpdWe@OYQCwe_bb z%kR_i)tilndpUNRLhWR9-X}CcRD!f^pRQM=W?SX>zbwJayDuy(?B=aolGfH| zD=RCrYzHLD?2fas1yFKpU^-cRzTDF@N>6uaY<39eEJVXW_hS}(7axCP{xU-fG2{s8 zN7NNhT@F!F?0amC+mh4KB(;qnPr zHm{sKxra4?l1DjK8V=Ov`n>Iu{N)sUVgbtjO)t)9v23l!a$#vn^2rkspM6wx?82Xd zCa1n$s3u>lW?K7H*_;5bWKsN=`2gm3pUXB4ZAhM3S#c=coJtcVg-nl*jwUW+3#XP*5El{BdPK+R3RvzQU)S4q;+qQaQb}WT&O8Th-dynkf1`WuUH(YS*q^ zUAd-o{qF1fdPR0JH$z*IwZrQxD=&01WDN{ro4efH-5<^U6loT;fa-;Cz(qUU*5nI2 zOnbm^p3W_4ZTr*627hcgH{Wf;a6kN)8SV=wP4w`t+SP z>^3#yRqjFGxQgDsdg0>5#m(u=Y4?{AMgkHFll6-k85uHn?=q_0xid61m8u!eP(-fc zx_Wx)txT|`K9!Wz`zoiC1hNS`P5*)aLdNi1UkQFRR5|$d>-!I34#eCyH60nM^8S#K zQPW@M>e%8u-ygs&7Z{h5b2#Fhe&v+`w*>p5e1??GT_^bYX}sw;WaZ`e!=I#QW21$` z(d`(ii%Et~R$X0vZ*jP$lG!YUNbfh$)5Ch^%o(kG3r6zyuA)Pp5EDDr+}teLUpRn1 zcK7aGt#UV!Z{NPDsH?viAQ9I^$6Oyj{#;v2SqD>Al{cLyihc6FiHYZk&Fnas3 zPN7XqgU9LhxjqHsT7OPgS64en$0R)dahZ2%O>J$(Cr_$8e0V-Z zozK_DC$6VQhqJG@HwrOHxT~poBxJ?oLekW{+ac3rKwQC5QOB%*ep1yS-K@!MxA+y~ z+|TLk?5vieE^Ie?Q>Vl!L+Jpsph05Fo#a=5B503Zcmb_#Wmr*JnFrv=?t2RrZW|jL zqa!0y)Si`t6bb<;X=w>?&sfjjL+quZYRxgJcVJiN{f}i#swMZq!y{?!7UoTHHxir8 zV4M6%EKmsPL61#-C5HJ5^6+>=SNhu57js%M^18XXxv*8&d9KZ0H=^OZNK5CT2`;`t zK|EaVj$VAAA}dQtFTaj!BRZ2ikKNcU>V5l8gx!+KEM21RD|gq`aI^cT>g2`iq#r%%7{RF2+mnKt#$jui%|A}uG!EhLmp2HNEP{r#i{;^B11+0w<@W07Lc zHrPFj3Yply;qE4bG!uFcjO^LT)F|_c-_17mu3O9Vzu%#Jsudj?{P&OP5exD(e?d-v z@vvg|7Nd!V7#1YDbOGIyXWmupm?rc5f;w6`*I~QySiTOer{peA$!AMTOXUYAQr1hY zIo@BmRGw^&7M~^5|NeVjf;IMzNiv3poD9@5DDOo*3mY3*1qB6^cXI+6`>jl>-hc3* zvavBUv+9?ldpBC)yI?_wC#TuB28Oh>w2@aIam~!k90Dw>TWIt3pmgfCaa|zi)vGKP z`K(ZJ6(Y{b*xO%3#JzjBo7!{tjT<)*C-fe$Gcumy$3^3lmI@0BxFjUlKYskElGeWO z;KAA?g@_j`fb-II3lG}Z*Z>NvOxMm;oWeqdF#6S!^w7zPjFy?s>b!tDJ(+%lnb{j4 zW4b{ZE1VVb+hfID?-!!1wv!@zB~mtPdN6>p9=p04Z{ruqB_8&Q*FiQo8~gI|^WQK? zIgdp7iJ8@nPbd_qWwUPead%a0#2=H_Rh`tAA)ShD;9mOINd-S_%`Z3>U%q{-G{w4m0al0cfb;TY4JVS-&rkHG4bc_Mz?=fW9opVl*21C7 z`dwfRy>2B*%E;_3-c{+o*Eh&dUk|IY2UP|sj2b8U_~Xfd&=7SeK$J* zzPSX2c0tqaKYPplDZ0S0ijwu>g9Arl2$Q^(^vcf8?uChvkPyS+!(63ww6sZ2mdE$c zFE1^@Rr$gsav&U|DV+9x6b7kGJtc<=aaN-``wJ zJ|=GH0%(gvJO(v~|5Msru<~@ndD+?Y2K-^S%2s%=Kb56j>c5{M<#!r*L_{!utFONy zef9Ce3Ao<-5odXDTH9(H9?zevXluVrOg!1v(ZMAkaA0d~w)G5oihv7>!Hg5rjXH^r z4i3VfVv7hoa>{wizHQr6$Lsp<&8=m-Z#lhw{W>l&aZicU?1dt?Y$DzM{m*0*D!zvD zghWO8!8qi+buvt7yc&LUBEt7i)$Z7O5z16vUJmn6bcNs(Kn31vDbgk;Nm<{UJf;9V zyyzSo8_Oyv@JdcjZV

C%Uk6rq%DXr{2boR=;!LcMZvbj-=m--|HI*NGXkTR!H8 zvOx#PdU(<>4+9W*y!_)zNK6bZV&QKfXAt}Gz-o|(2L7Kbxn;btZsgOe)IYOMwvB0c0QJXCTuptk^k`NQzXz5lFJi~oh2M5GhNx0%T%!^ zHPE`c$BI6EYJy|s<3oNEB^|qBl^1R6nS|`@5ZFA>G^ADrD<~|it;1qa0%(y9|H)~#DD+B}yphiiP)ONr1_+EyDGIK#{96E28SLK5-ZeMR3|L)WFHr#mx~0Vi>By#>C<gyyEf3Xfka zzgi!x$Tn8X-S6MNP(Y~vD#-YUhflYiyp}&VT;nI~IHjJGo14HXOTNz8vgHP` z(YZM#zA#_ez~l0kogc&ckUxI>$a}nKWcTpl2`GV&9zAk^lON;gACni{i!p-bK_ank zZf-Z@RaCUJB5Qhh?&7qzTt^Z;)!o8-B>Y48P)Z~i%vpua4w(zHT3A|MOyF@jEy1-s z5s^KMB7dbFdM3L({=`&G)E~@1hOGBjQAfz_sx*P{TjK z9;|C0uf2A5F)Sj2`@{*vt)#;+XU*hZjdlKuLKHoa5L3RlyV}D@X%%HANkve5_dfnrK* zUN++(gxTsHxCc`goJN{@z}}R!tS?6Df^-wzVG-aCkeR9Qv$~X?l7>sOoVB&dp`P{6Ogf7QkJ{Vy+oGWH;Ft$Qdbr}S&MOi7%w7HIhzK}IZ# z$nqVl?f)55*$vD$8K|sXyYRSWn6N$|2WxI+b-=*Kun7<)j7ol)<(t$=Nl8HP$hx|g z{5zT|6I?S|o&Q;M1eX0eAaQmi+8?+b8gm#Y%)g^EOl$kU!n&K8nVE9Z(tBis*aLv)B-ee|$Owd>px_~TYHBc7 zNblj21gdPND-l}%e;`Z84v~k0h_(5FAi%9~pwe~nza=^tLn(Rr`SbPTq7FB;!lR;S zGBo@E=0^=&MIK$!8yc{hVdLfW}b&ADEOy=4p(`h~VAw*8#__3LNClbj4^LkgAYoU@v(H5PM(Xs*`It5?-R@Ba%~%MfEY(ZUvOyhDUU`A{>MS}M1I zrwvw1NemBX(*C8k=)O=vCG0e#HAHxIN?|hKO8@y1Hhs?*FPgkL&}McLA|u9D#QLM8 zkHu@qkj2G$r44=Be+RP~TfpPp{8x7L#k$=Zhf(5qgx0p*aH!GLwwv8Hu(srG>*Le$ zQD4E2=`0u5J5V)59}YeZu6xxhxKo`Psmu7!LXGq~r4atPySqCvlmnW6-Ny37(ENN5 z0PSdT*N#CKHM&BEG|)#44jTlqUZQ4WWBc@I#A}8Hj_0{jr-u)(8yhFcucFG`j#_r4 z0}gu@1>Ou7v7>&t)KqF{Rgo7=d>@+QV1D!22UHE>X-kV5l(6Pe(X(SfVfgs_zkpwo zjK8j@7u_(+i8yamy>DohMzHiLXbC?{pWJaOj@(Pb6bF!6Gt)rE(vlxYpxeb@C<2P} zpV-dRV_{s*ZU=sl_h>RHAmmPYB+$PJz^X%mWC{n)B0eQW-uBpqwsK(XU=FP4?9}8T zg^tc4t68E@YK!$R$uRRZD-x;+;_0oVmM1^x2JZjvXgCK^p0giOxWGmv={UdU$2Q&U z>CD66OPDOBqo$TLF*)_>)hiWUUB9fX^E17L%9+QZ{w4qcIBjFyz3qOCKy>N`l zmkd;r+7O@$Gc&WM5?Yy5%xE=cdHnWJ3$X#0JZ8#_a-F^O_{o#_($eG7FFdH|rt($E zOe8udD5<69Q*kjD7uRlbK>@7WoQXmz#)!5jUbY%2XFGN36z`cc6(99W6u()XF8*hrNAM48rBY-!)E6!odZuYDMwUk%$X7tq=G#G5}F=K`@m|LU0&VGV`&zv<2F znH_qw;{PnYDYQdxmOAdxo9pl+6YP?kw{Lp_)-uo)Wn^Zi$*lkO?VhnQ1@KheaFA+h zYK8!8UIl1rWMs6WBq@LNvK0#;p23lkggcq1z8vK+POL@Qkd7V0h%77k@4)Z4i)dIR zpc29zWjg+eM7_8Yr(nO%Q9B-?zHK)`@f5~LKmja@7cV2n0Y;ZfEM!f!otve8! zH9T0i=TPW8_&2wFDNjl*d^RP ztk+X$XlSrpu{kcWF?xi2J*DEc!OtkDDr;)?qA-L@PqF0-M>pFeU`ibXB+Ps6Ty3%A z^m*If!`IJZj7J1b(em1$gpdhra`^|w zuo~3gtovUNYd`s~#A<`+_#6_kxN(XJI3Us6SP3>D>U=ANjhFAt;pKnLt^HxOE@mtb zG0m5)v+t#(WOc{O7#nlT$jZI~i8X)&84IFl@^akQGS~GMkW0Vx_s8n#>z|R3Fqq)< zlSYbvu!WLBSs#c&Dc`0!!gt5+k{`4~a%QL(np2%ejry^UBQf{^m^=H`tzYOBr? zKDn(iV$Q(D1YEZH$(x2r9PYHpUuh7V4=yPY13r~i#O`E2>=e| zbfWTG;g>MT$OE65sFpGrrfCr0TWAaGeGH|icRYYaNYd%&`>8pPt1lDlC@F>45@r(t-ZZnRb8EdmgW3Y&}qrTo+w@M>s-uEL^OQ4+!EnjvhTKI_BWKwWGVw zXeXWEYO%;?5i;6SVAcIRGBVNt`DjTYll0{LKe$reUv{3}-p0e#FT3*0L$$}UCj64d zBZ?lQFZ>2o5411;7YbG0(!!z=WXy*^rr7A~G-@diZ+!jT_O%+bcA;HV`ZZA|||(#}V(xjp{)F%KBP7n1Rl)8}IqpwF3^oBx_VFYo}q z>^^Ghsp<1wM;`xKPk+Z%y_mdvih+QTOX{0wUjy2oyK`%HJPoYPiKEy}lOXAFmV{wZvISXes1 zso>VQ2h@O%kI!{o-6#WbkYuD_j2dr<;V&VU$C8W)t{Y1xT|GVTpgU`kKsHiIQRn65 z4QBIb!~6XZOj*3*ycNcWYQX=wd9mXEr)gN$|Nqdiahjcf{1GwOuWmgPI4u&)5^yfq zuRa=nk%e}0x(qywz!nor(TJ@TT0Q_*)SIckVOQ(@XQLi?hqdm?dRFUyRLTp+Rx%uo zjy`Uvt*Ceapy$5%|DyGKn7q|NYyOeWe+lsO^2YJJJ(+5WzB4la7AYd~2B=Efk09g7 zXK363u`+80Zd#%W?}zvAWi?7+GEY=S64W2#K;*S^a_aa^-fVAg-w*4&gw9@^Zvz?_ zF$w!ewEk@KZ?Ya67A6DC90)v4vpPC?X5b@%!qN)e%Hf;H-}Z&q`WRFrMuZUc>Xjsj z+Dt7IKceKoF!9RK`PD^Y?YO3M?ZE3<&D8C=8RVa%)kryX%kK}|9DEt@57?(QJ~5G8 z*#EzqjGg~~BVz*yruuxa2oXj$Ks`saqW^}PeXB$9SXauyK~S*B3U7)Utw-+qe}@ZLFS+<97+J?aHl7 zqEB@{_-3S>y6u+yY6QcO|4prrS+eafJ)^hTzhOGp zc)@8F1^Cuu`b{s!ZUsp6n?Eue9_@JwGvLT_BeD)qeaYR3qGb%wx&TUV0Obknh>6{X z5#iY3!@Gey2<1`c0+$Z~HrbjyuMin~xrIk!)Rq-eNq^lt_Mt@2%uGQbEs~c{Ww!n+ zVk^nXxd1lcCnD(?8AHE*rQ|BzzD;XkVWG%}`&R&|A7f>0y$hj}p3Hd|sz$;l(_ny4C}E)`P;)Rr+_S%zfdHYkYhXJ4O5 zgkUm;k$UfgA8$Q@{@}iN@d&8JqUq_Q*wELoac|a`&aLs^E&sXr$J{UOAv%Ag$bIXw zh^S}_sHq?F@?Hb%)4e4xPd~@~($6mez$37B3D6p2g;@jo#R-JX)dVT;gQ|}n3Eh5k z!E?M3o9T@FM(+C)bl2q5w_%d9d-N#YeQOP@Fz_Z?_G^)VN8aC&1$fon)dgtUdlJOz zww@k7F`($}mM|;Lat`P0iwmL%Yj>9RQv_F;~gTN+1LLAcC2o{ek|XqH^dgsyrIKy(m{~ zKN0wcp3Ke3;XQHUDNL-HDCNb)MQLeai9kiY zUr##A$@vUmlzft!D)27t9UW75&1MwR)Hg&#L`DHV&dtpU8&>WCdsgdKi1*jOb{ zEcaZWsEAbCNCLqI=R9vfmb16;W`EOWx1Kd|kmRNOy(QJf$%W&Cxe*$ng9;@F{V9O1#q(FJd z)JRLZ;XxT9n-C9oyxXzf92tV&e&yTv6Y8Q{3ULQG`HM#}NcVkWkIpvr(MTij+AOeZ zg5_ex@#@Z~14Qoy5Z(R7{cQS*J+JgXrm&xAnEeN0dGjI*L)3X+bs))3XJ-7f8&)#{}I1tBTRDp(QaWn{9EKe@0%7!;>kC*{{Js zvuDqq>nZ9+gP)(38DL=C0;7@*@N3hK>xnyt&7`_%a3WOYnYT8qskY01-=o-JJR>MA}bb}|D7Eg=H$D}xsGz7QZDjPWc?Dh>spG(a1a(ScdqK*t!`fym)@y^1ckyyLoNNg) z-2j?0&C->FDTAo<2PcXSdHrPWe0&(s*Wr zX991S(ufnUh#o;%Sn!cm6d3Bjesl!*b^u=`p4|8S#b>P!oV5br2Y4blJiMcKF;bu8 zfAr#c@R~@ZVPED}<5vTj$^d~YL0bv}v+MP=cCyz&T#eb-U%RK3xRF`{ZYl5+`onCQ z0M;NNPl0Z!Bm@8Qg8@Pve*f;s{TM+vKY-%^J*;x8qeG**uC5CHAeap*Dk@;iex+3O zUs0(B0nr2whL=ZpD#5`3q_Zf8K@ht*-*(~mrm;uxGg+TEKyR3k0WPSivdC@LG?F}c zB3a-7gBO^aJP-Z{I93oWheHXQsj|PH9jb24+e?-R1Qbc&v@63nV+e;B8BK)IcgPYR zR1KGU()I+B1~nJNCguf*8)#VY{R!w7Gr-VLS!^`Y+PYXz&(3}j>>hB;3}JwS4-zf( zzUGEM1Xc>q+}w+x;$@pPbCr>*zr_rAfgnONd)dL&r)}*NBere(zccW`=PsAq)y;!N z3(&)9u)hfw6W$~wBv1gkaC+hh*_(Q?5F?O{VdTD4{uGbDX+5B}__Z7N)zI2HATaP? z=<-)sqGw>A90D!33fz&4;990OS6ftARVH)r|KQ#AYE}zz1X^GmzyMgBr3g11WDxL~ zeX8j(oxyW54unp#h=FxwRUu69^8d_rEP-Wd4JGh1V!O7HkB0Yv8wif+V_?KM!2}Cy z4U_zyo}M1WOC-5 z70>tj`1w^~N!zB~v&R1790y|a9+L>`A;H0jW5+z0jac%)yFEHSKAe_WE(xZLSD~S_ z2IY>>|G)vMuC6{o5PWAv!AxLYS8RUzTr_PM^b;)q3<|_U;`Wnt z6wV;?xX5zvA<)1Mg*4Mis`W-h;}ge80K&7t2jAA&DV3pN+@5+@@@*IX8ygNhBj2$| z4)9-ma$WzZLS7d~rLJomovRX-0kYi1Kg3;D_Cc|gZ4IC&Z?#-$lEW2=qUu`a^AZwj zv!O?@ZN!YwHay%+>a8**yNO1?Ak8=oW@WmQ4&;%GI(0!vq7Td}a0j}{TTV^w@YK{) zFu0Unzh;1uL-*z};AY59Cs=etZLoNP#sr(v#%|lohYguHJblt>+ofkZf)8co233v* zN@eubN0M5=?;ZncwP5mWq8$>;(sGQz&jhC^^W#TZ@CkuMJ2*Br5EdvdJ^cXKfe|Ol z+`o2{9M|!ul1Za;IG5+qBi?NjDElqc+yr*6+V1}lkxpa zut!>wv1q>1fkImj5iv1Mj7DN;xVLkOhim_Xfz?> z%yO<6!FY~%7Z^$3fFMr&ULGcFG$Be4phXgRph8kpQ(u^jH((!ru){`w-H}@ zdfpPZaHNvnMPxS;>@A7vd}nxgIdCX^23@;dUL&Y>fSa+Bq-!ACJ(!DdV~g zC5%U+sBR!R8x1ha-}crm--icIa=pD`%YN$g>02FVBYrmW)UG&?-m(saE+hH*{lLEa z(#LGJCqEC$HWwE)9bGaE#$To|DFkd$onSUGca&Iv+%6F5C4tg@M;dc0KO}~JStIC7 zDzof72l5RUf-Lcqy%g=RotCBMZ;^8_hF@Qi8SKlyM(e=e^$1Qb9yn+bk)x)Kv6nu@ zwqZ_JPhox~MZD-tkPVQOlcTcv`H54k7WynrCfo7jO1K$T%w4sxO!yfv05N{A<+ZlAvw|dE^pz46hbKhrW(jufi5OJF0Z+my2v7c8Lo*3C;Uf=~&Jq0A^5+C$wX1P5o13>QC&waD^DG^s)s{!F`<^wLnjAtNEquDJ9KS#Wqse? z^^~;TNRqyBV;2}l0+HK8bUrU%hOhyjqOPf#kdZ+P1cao%K1X5^klnAt!-s*OFtfIn zdGO%93@Mb~X*>7`$G!f=!Yfl(3NTPo8t@IV;>X~`1MODp&fNQ-POI+d6Cig3(gOZLO4$X$e5Q{442>RH~-qrHqxVRJQ8X8<@&g=&x&psNO zi}=^h|0}Z70p)+lPDLO)4ViX72SXMZ0l$5_1<`>x6b=m%12_gf7fuoW&Bt{wsL$2r z`@fm(SLP-~=ZuznkJ}Xrg03`<(@1M~?JEjARa-bcs}W-|-;+-RdfMOsX1fmag-;Be zhMk!xIdBve*bdmJ>0MvnDhw?m7^e`@Sh5%Q`UlaS=#vmx)HWg&z-hSlcmV-#uao0& zKPhn0nTYtDiC*R3-t5HH!uNPIdJ^;V%3of9FZ3~{IyY;;fQX#|sVyaRd}@kp`vj?b zA2|qsLle$}p(k{+R?i4*QyduAVsr|jM18|(W4~Wo6~=u_#>}Pzt+u?din8}5K6VTOS1r^8O72F0buq2Dbt@*0@dK$3Q zf;0P{!T_BbMIa<_IbsV_)}J}UDE=8nW)<+YKYfZoT;DOY-7e?)dc3wkP@zUsPfwuq z96!GYcnZn%N06;^oh=|RaQ5U>2;Zsw4s!?@i_cD`zJE{oS-FvF@|Fe3B;7rHl0#t@ z1&GngE$NFjHX9Ge_td(r9cXDoaJo9W=V9`9P~&B|31|oA8pv9cv@f7jh({&duD-r? zijSXP=>C^Oi1YXp)liPehS<)J`cjID{{0cs_5cY+YfT$|pWw?Kz>&Sdf;Tz$!H-J7 zafX(bew`>x-{|76xkb6a^5ZD`Ad=lWo!S->Fs2c~G=&gRgjN1daJ%oXzBTGr4tVnd z6UKRM0Oat;_=WiIOc!cnnJZPn$@Tv|5h3#Ke?1X#)ct=;B4RZ*+Uj2e5%LjhKnM?x z1R-SsV?#)1VX=;brqJua@vDbmN8o{L*|qI(J#;JW==7QqmOJ91S*i z>VpSQL#_c>adll?{zS-u96^-f8mL%riSyn}B{on!asndlAeoh`4LD}98l=9hlMzn- zI(E*^Jn);4z)8%v?3}88xHv5HR@9LX#N_5+PAT3~r(^^hK=FsUfg3@|p%;e5D#MTs zT)w}=q1NCaiA^7v8flpT!$n8A)2_;VP|FAWZV;Y?L7H*@p1FinzKDk9jZnLfI)Uz05hFYNGA7u65*i}SfEqDhx!hvU2<9mav+KmH+*6s z?`8(kHYRHnPxgp60M)w8vd!p#a-(Ns3li_fjn8nvuK6!~ z209mPg(uD~4$gT-~g%w=e1Wn2acnpl@`pXo^VMW6#uMqJ5f<$}= zY-fECkMtS+AqZga#trGjb_*f!!Tio4zN(Sy(&|4ZV}uPLA0A z$*puw&Flv|v?Mec;!*_e9+>jo*&6PQJYgAtiw3S@IHRzCs-sE+OSW2)102ww+B)OU zuYG^azGe%wU$yqG&p_WS)E=|F2h@lA;cCG$bS)9g1>^`bIpGULq08!vn&nXxaSV0d zsw<1s*dOC9bGHD4ARVkqtvdK|A7A8$$oJ1_-qNt}hQIxEs%SLrb!Mc`>$hR=Q>+Km z9%O12d?+$Zxfe2~{PO$WBexlDsqMd`uBLw3&}(bmdp?@w!FV*q#eGwRqH`bpL>fgJ z?UD7`6{Ezbwr~0Zc6X&lOuj&FL1@yCJ zOlC^`pNKkZxcAg8KY*B^N4Gh%&?Nyv1c2J%%<=$oFjN=}Y4f1R-PR&|Q=mFn6hm{- zerE@o7T5fC6hQazRW^k2dK4Js0n%o}j@mrvdG`ntlUG(&mWyueAFVuVQtx*Per0}u z1I(pn&F}V-(F$iqTpR*;7f;xzE-RDIz#uklqM@SlRKk7pb4LdgnSiq!^PRb#xJ1rl z12&0MOBsYf91sidU|v8#P8!Ao&TYGoWdWSR!s;DMcEb@m<=}4$2?;T4N!$YY-FKoA+4<@g?2d98xC3l2+s~>MKMP9OG3gkcrLWanQ9O0!1`G$jA0D@;N^i(nxLv~EkFbN**E5%;h520giAYpI}ccyvn zma7vWu9F36yvHssMo_}Z`3@L|{GicHhB0z*1VJ27ONVG5uPVhsGMep8QecL6a*2p& zM`Gst%glQ6Qf74TDOzI?kbC1N6LlaRRzcEY;ucfNCTPDNK3Q%Xc4b|o=>tafQQYV= zY9o+69c|ljc4W&j5gOmg_)P*r;gJF<<|`zJAf))h98O6x7A9DYiHY@l?)weRLI5gLP4ONLdC>P#)lR9lM`W6Bw@3UT06dmc+btjq2?y-7VhLz) zXBeo9-8OR4ANN6Ep;Z~lC}9if1u}pie{;(HX>x+g@^@$m!Dt3v{+;c z7DB#{}Uwj$fxW00V${DkV7>VObbc3L1U*;4 ztK9@b@2UEZ3rU2RJ}{h3)-aj? z&Y-texSa~+s=iNChrCtXH&=un8*vK?(g8y~*_IM|{zqe@vTZrg{?sJr$);TZukoPT zo>Gqf@|6T2Smx%aE2@AZJVuj#ycYsQdZkkKaBWMF><44=ZcxX`;YgUb zVWv>!Apt25mupE)K2OIPvw;aunG} z;P{b*upl|U44}2+DkghvwmX*x)|s4zClCDKss6Ys|KaTi9V87+-w*lr0;(rZRN$pNEApk^_5+7gtVHLY<%{fm|(>qFz~F zuV5MrkHr`Pui&L`NHApsM1>%lYu#lCL z(*nvkxdz)>5c0w5_7Y%DMQ?A3>{yiLHkh;y1JMF$I5_0qGuvZ%nUHD%V6gyb%^0RN z7NCCmBOVGlb`$M-m3^~Ub$yF!m94+(BQ13QsJZM9DhuC67`W_*t05~J>_otUlSw2v23H>hCu&jRnzrI3+o@<<03Z z9Ju%(mi1`x^De}C`9re-U)8YWWFCRDXR`tFf{wjUd2VN!1aLNeqGZP{P{Y@OFDC2Q zLw1jo)Lz2*1yei~RQO5`%=R` zkOWQyRq|6#$dMIlTG~@UL_l^r4w5XB;y&m3_90f3pr3}>6P&)TQzu=1$c;nQwEdcOP@)y zRt`B`76DDAjd4~|?aeWG98SB%C4kuKwkmE_C!-)Q&kp4`2FdMAl-UyliGO z-*zAtg7lCL{8epcMF`jFZ~NWn?KEBIeLUp=j(D69?g zBr`J_7WB>Zd3Le3T8t~=IYhYJ=Fkf#O2-?@GKG9n*ilkfvOesqkluG%(ik{H$r(~`4hC5S5<9}b_OjXNV``swehbXEU?elNo<7&j|m znTPG8hTnsSm8emq)pnR&>^NF>StNd)iP6QjKn5!)&^$pyv(jz$jEQ@cYUI$Nr*kPO zEb2PG=m^1tKC$_I<{CCOj{yLk|B!xQS6xumM{C`(v5cQT2T99~Eb1CwHpp#6)c6mR zh`y0z?BLdr_vDk|J(LVg!!KVmXqA-E{Q4#v(6kt;d_FGg5QXi4Y;q&gJLk~|YmMnG zLd`d45P{v%TeL*Ez!T?rsLm$emcrFXG|z6$!;|`6-9?^(=lJz2L=1oB*E~Pian_+b zU%_lZRzd=9#B~=@#J-40WXLNn%P02ve;%{qpY82e8p_0Uy|ewJ*|BD(TgP+_b6UQ! zYB&YlGcl#&KgO@6!v_xtxfic~J!Q}9@Fk8W^VzvT8vu9jT9Y5;$R;$WA_hqF|wD+;S zefK{YdlPV|_cm^P)G3vOR7551$&ye}QB+!pXhAfVN=Ootbt)7gZ73}$`;wHsn35$C z$ySIYd&n}D88h?$e5-Sw^ZeiEy{`AV&ULPH$};nt?{eRt?Ou}R?=SqZK5u-*ecqW{ zi#|kdls~_Af!pZLhC+XvIpM1rp;FBm2iF`!7T7a2u6M_e&tu15Q55?=30=!|ElCK! zUCTG`6}u~Tpr^BPV|%{1(X8#2oG~Th>Fn@esp%+O;#Q>;+>M?amo+6~(yHuLa)Lqg6?D%HEb+XjtbgYm z>ytd?0@h(z;6ndfI%C`9}H*Ym$%fP?U!{9=lmfNiSR&ndh|k z(9F8o!u#Dvq$!I>b4NccmEkw6iiv!dG*bq_Hy!2{_B|cpJQR^8SH51vzGY`S94|iH zz?>{7BSG0y5w2O;Rh8v5yy{fg2F;Rw`s1-5hMzL5mnV9f&)Kuwh(0)D-Q}6vTDXfo zgl%zHF|{dY3Raf==)1YMLQ=9^s=kl>QRv8z{FG-4vAAySkMi>;&D*nlkL=-ZwYx7! zS%+x1?t7p+Mwk0u^EtzMicj#|tL68FV&+<&==O-O@s#Af@GVIi!erB)EC6aY=-^;Kl*w`qSX60lbyND#1lO44D$BBI7jJ@W&hoDD;Ym~XcehTnHRD% zuaVQg>G?;!>DMO}(o5BL>mD4qt8iw+#@{@gy+X|a$JyD4h6A`k~-Z@Wo&$*iybi?Ep&!~Dd zcvg3ZPk9a2X2?B0?fpXcnGE6g^k**@T<9wiQZB8FVxKZfyP)kqZ>PoAVa-i{ha8@{ zF}k>X&)51OpSV@B>s;F|?M&Xr30j`zlyL5%F+96R! zW{x*xytXM{uu58{@@+A%c5y&u>6_lJ*hKK zFS&JcAfNB4Ri%0s!&^~|Z2m?5{$4C!p{GM5myZ~TF4EDeE}{RZP}^p$6c!*h<@n>B z`Sdyp2fXR(|8>TCP5ypzV%bk7XW*kd>6 z(aX+R@BIDkGvh>^`T8Gy)jgg(c{TkU|H`QXJ1x}Ih9}xYHm5qL>|lZyz5hC2RIqsQmWAC7nFDT)Y1hwr zRxWAO+N+V-pQn*2;rS~0vF1RrxLx%sRsBDo;yXZCEwdK{c!tVG4qaQAKg?%G52V+o zTGWuyiMOs%lI@^Un!v|rf4a|j^E`>5i-m=qVYipLx#&0i>+spOeOM#-$m~ni z)={I~J`Ke|{&FX$tg`K6Q{#E0N*6au4KD5;sM5&_x#O#L_(@W~Zg;_(qiRKzS9vyv zhs)l+@?5i~W&Y~dO6Q;O|K8_GW(XC-k0V#LV*{P)xGjNFyEn_*o z4ZPf?np?%R0~fve?T`_*$r^52G)T+HU=Yr1%75|>lUiw=Y$-s@ao)7 z;2_DNq2e@?|H7*>H}-`p+0;hBj+fuTJk4eG?oat+oBz>bIvlkFR+b4iyWdj!#A-Sq znrYLg$7c@qRDo#umDudsM34eZ#vs>K31#IudaHc)1pIW~ zK0Y-^#{aqc6$HEv8~uRg7E~KC_Qjy3w!mx6@6f)to>=AMXQg0AEqU>U-z&N?HBDs~nK*Z{bA-X>8i2VV*Y=Xq3dYy9SpO54P7b+>= zpdd|{i=<`v;fdD$pV(59JD?&$_Tk-~8T`JRc%s`<;2{6Qt=W(EnSiX+-vt2J6e3;{Lc9XcFIzM)G1#J^Yh8@iSM%C}*N{x4qIrAZm^$`eDBOU}gHRGNG@WmV zuU%X`QAA>K6Q_N#vwKYuI}R#nL;>B|;O2p(7Kh*Fi~|+>is2o53%JH{kwx2Re{(o&982Bz?cKW+{i2k#boE_3JkB|D z=0t_sp>rXU>--KdJ$eAyj)&3#q0N{%GYYs~I802P@UB*uA|kcm8`ACIK7%Pudl&s5 zjBA~ckc?59?;_fSeeU_hJ&607Y$AoPSx$)Pun!lF$)SIqi?oTceS6d zFJLguTPw6-O@=749+{m<1Cr_NX?VdvVrPU0El^g6YzM(5Qu8HloM)9lbM~vrNoDCk z3jFCF9KA)ImfyteR+!)Zm3` z;u{{u?S&SAWxTn_h_weFBNj}9mz`w$VK;5V9~=)&JTA>?Y&4!TcWxw>Tz;}Jz~9A7 zpc9-#!Kdm447@r^%i(QlsRo<~Do)+Yv=WU(ln3tZN#cNpCqT1b1p(Vs^@}Tz25D+# zH5~z6GbL1w7dy=#yO@}^4!N86I;$1^@p31fBNCDSpiW8P5;9fsM!VfdMn{XGw-Q9!B4n!& zGn-+ZxsJnqD$2hzX`&knf!A3D1_qrEA6K?ZX1YVVLm&z)e%H*%UFkvh$K*_MmNlwG zvJ;t&zL%q18+aHJw3-H&q@<+bkJ<~>{u|rsj@}!Xl5@hEUHFaGe%BI~TrY&33JSqT zXnrQ(@L?r^)B#CL7VH)bPF*{-VJsn`f8fu}RuwxWa~wc|==xK4%H_h0z&NVVjoZ4#SmhTZ2Lv&}&q~*(yC>spCn>#f z+|0|nus0p%Vmv5>y&?s4762e0Wavx=aI)k4`K+fuV2JX@D<|3#D9Q@KIQ)FKCZ+Rx ze^(b-x#Ze~hdTeBc$&1d^oMkt!(ah-n6L|Axe}BWo24kc5L~b zYSso5w~80axciY@2T&|ud|p&NvQ+Ahy75CEsewxp2Y@#$TM>vmQg_A4Ms70V_5ri2 zi5#Az92{0eb&RmfKoIqHS#RG+9Yc^{3af?%>c@Yj{+!TC6I8faMdrJG6#xw z1qvuP1Ys#7v0U{6vX~bvSOAf)+$Podky{F$f3mAU*{(T{#Lj@6_y@fD!vz4Qn(LZaQ|)c6 zEPOz7Vh$cU^aue&XB-^7ko_5xVw^o2e?>B+V{+jf0WnKHDIQ8>B))r&D-93b$)uRo z4t#tkG-My*_Ae#%C3e+R3OSAAQUF>4Y`g;6@xgI$McyvJ_65PRqW&$B8a3;u=bTof zkbK+~+F{;6U|&P}0&+6{Fx6vulz$EV=~LaYqythgv-u%??QvF?D21eLr`go)OVI*& z|B}P(S(gW%PNqY>MHX)7ZG#}g<*kV}?Vs}Y7RtqXnz!;&R9Qb19J{NhqeKOyU*odrphq7qisa|k zRdY5RcQ5_S@wJuszKH6}An4TzsRwIht}WVf6+8T~f&?yXa(aTD1-;<-5peKz5UTD? z;F0m)j_uP01T=Te!4Ajvf^8vA4>UtlC`1xZ zcpsctcrU&Xs*{pl+AhfhrfZ)Os1R47rSPeMuSO|yhxN_`E}mABN7TuN#mkm$kK=-& z_ilZAS$qlTjSkM}q*HB#D`vM6S-7D97l1v9|@P&j`fVTISh!0QT zS!?3nvn=WfQ$nt@GrG`<5$`6&6a}j&)Okn&go;sS&E+e(-0lQ6C|i=^`<3h_$vWjEp1susO&nr}g2d(SVR?k|r*|+1N+bjyAGiJbZ9=Tb6kWTG( zolsF$o?c2DC=?+zn~{+iNj4%`?x5)A?i-Xo`cf20OlV?6LJ;T+1EWy8{kd~HajEA9 zSj2N~b<-?*g%>T7HO$E12k{tzqU7C;=WIm5U4Ox;o(m1o2tAd#TSksyL-WWKFs&bf zeK(t-P@4y!KP7iWjHs5%9&8e!{45aGSu8EBb;#XGJApeiG{hs?%nhsO#$2=Otx+{^ zkcjW~GGKon`Lz#bU?0G#a1DHPeVB(LCbSa!K--@ciX8yN!WXY`*uL5H|^pR-Y;n^{_3f!O@c z@(enM$p--iczhlTg49rDE>==HcxNTT40X0J<{)*jWW4T5jwt;g6k)G{G?I7|e8l?= z4B{LjAWw5f={x`BlxWGcAW?^c!|5Jc%aIXrie=Wvh>a;bm{&cLn$i8jBhC7SN9a}R zX?8$*KL{LqMt6-E^={1v(FXDeoN{S5OAXkILWuvc_G#DsG(xx{586?7X-&>~ptW4K zGqHUjV-`UiHLU!#uEQrQy&j?7>1tHyYHhV>@2(DlFarr{yf`v$=W5}~=7T2AGD=3Ti1Ff28&V z$Z-(|HWtqv87?bXw#W`J0EkgDJG&sV)zIc{*f{H=+hN`Vb=QAAmMA8DMX$way#@hg zJd<_l?#d0t%z%moTA#_)5iA?)kxrk`fejSeD`Y=a8divZEnY@1$YuTAb&;6cd6RHVzJPVCy!MlE~%k>*tCNy@Um?2ey}4i#q5 z0pbo>MvNaVW_EfEyQ&!ehv&adM{wE<>@$q26MPbbv*scyDcm8L*X1Q^l7k^RCsa!B z-Q=%$YM6z>-Q!_dU>ea&nqNY0R?MxHF2Ph|-svzCBSgJxCJrat1wh zL_YX_Z|MErE7eT>I<3{zzL847HEY%wS~0$;Ua)F#jvp)hSXiHl=?;{>FBxBZdiW58 z;6M3kK-{8YVwNCyn5++M zQ6L_(&<3^O0#hj^E(-u0DXh2Rg0ON#L_~$M&y&i0CjfhhYswItRZn?@S$CNcAC6aBF{Cuz7G$HvEBS5@hE@VQ_Y zeg*CZvVK~X=Xje4S~0Jbo3_(>m&Rla%(sznC|H`oPAD1CD80=nUSO@~Y1@4`7;Odk zW5;*IEB;e@l76egN(bBK=x-@uvNd@y2_YWU&FkgaM=>*dY5el(+#Yl8vMZ6yEm~wd zCAEBf@o`d?^K?@PpL{CNa(F@Y@X)1we#`5NaOZd-;hdx*fA(RcSl{R}7*(zasZ&c` zDDP)TqWo|Kwj%i~ALe9|-V`TK{$@xIp!Cr=dv+Z{hMvMB{9yBwqq$eHK9RX9 z4!ZsVO0e=5m2IvDtQ6S>xwUHrNt1vx?+k0imNYUtn+Lqh<;IJ;?cSo=77`Zr!cN}I z95Ir#!J$N4#URz^vx8r#*2k`x zS!mhKbi(+U+_od>)>A;~8)u#2K})F)`3>y0M8t*auqBztyg@F$w-}J(@nkiplG}rA z)jIFcccl_z*5P(!!9@VJ%rA*%o|aHX1||fnU6{rae2it?JoOmXJFLvD@aj<fJD#1Ku(c)D$uv!NTbjuM@6%%*JLfG2b75H)qKb)Dyzp z4O#XU1mD8Qm`>@Q?ry2F*3Qmd*xS)q%%y+V+)ExI&`uwjX;4~TgCAk_=FRtsTn05a z2^{|G0urMmZgoJbunPrbeh1`1){rH_#-?9xAZ5CK_kc*js&RuxC{O7a6X^h zbyVQ6_*Y?N#3KyXH*RMy(J6>2S>OJVRkm%PyIxP|0oL}NJH?DSkAqZ!a-mS`iH0Vj z>$}-0*79`r;+scl-C>P`{BV>o{bY2uJ2JWty5b;u3zayUhm$xj@brSkiVusFrIPzq z+T=9r$&t)!^_g}$fFY`LoNSn1y#(T(J<~hnKI$LI39aYOzT3o&39wUckVx(Iy3fQ2 zli!v16d|?Met(MuEck=QPY0a%52n8KzuG3he>wOeM)+5%tIhTQ>b9bc!qX5G&yNdo zx%(pI|2NUN9?$H*6xrSo_K!7LCB#1*Fqh&6F<=7O3|WqfZasa z!LY^P3=tbf@%^)@|fLg9&GmHj9ZdIB~VOB z9SuIo!lDcoaiL!tezypi)$1{I*B=PDK3t`j!r0aNvKKL z!!1WvE~JpF6{TnkFi}OLYmoyhE!lHy)Z&?Y+vubNh1eQ(;oNs?qKTvOzGQSKc)feP16d-~6C)_B6 z^f`}VP(5j?0rErFjp%o~$mkh3GR&c)C4z7Wm}{7`{I(n;pEGhXUf0wdl+(86N+4DY z{^bXBs0Vo{uxAnrG?8@0$HxS8}k15WGtUHg>~zmBHhSvyyAbhh_?|gLB=)IMr;_K-vdj7 zf~LD(7v#(^wT_5bT;mM={J9mh)K3?19zTEnyl$eV+-tLELNBej9zmQ%*aA1uZ~|SE z^uU=hh7K;(W#|NXQ&jhXDXL>%CqLK>-b?=cPgA0P-&rcupt`F!JRm*1kttrPhFwZ@ zplCFFT)1F{$h$o!amRvv!~7ImUX%MGE6WsA0}=WXV+lc4QO~NYjM5eLN9@iL@clo% zymy!Dfc{*~pheM$O1ueYSv*jrVE~EKte}RDMUwV)-0DvKO%I@~j0e!D?g|X33lTce zxNvXb?*KkmqVwJ@8$SV=78%r2on)v$LSTrMbsT{QH-SEP;{48^z#X>)2(HSxA9sF; zT4Rw0Q9j_Se5@2sl!5tCYk7_s77xVPM`hQ2Yk14RsDO_e0^sN3zsxEhocg|}yuCk-Ech2y+L~fQcDniW>3Nws1RAW;3*`uSQLlcU?)N(~-78fpKGQi-DC46&+ z1_mE)EY%DM*@V$6uJ~B^V@l?8jj)!Yah`Q)rz~m)a5jyC8Fm_$dDDegh#DW2rXlki zC2Jbc3bT4N^CYSU^n*$ESq#vEtA`k7wr>Y#vxDMTyIOfBLjdf_U9oV4%`>S|c?nVV z&DCxE463{PW<)INVN<~YeFY0h6PEo5JPB8%FZ%9!>8aQ2hI(SsR-8D=*FEt#CPg@C z$*uI%0H!vbg3u$rDoD4rq36u+Kpj~H)Wy33EyhZ0k=&PL2m=vXS3a8=sHWT#TRrw+ z-2s~NIFU~yvoZeZ)9W~sG;pO4f3^(Y9TI!|9dl_7UCEHEy^w2!l3o#oRA>7B6y}i4Wlwjz%9VNZ79!B|~4O(UX<&!*jK?bOCY`KO`U7 zP~uWJ!mTmHL@C6YV1(J)JNvd1FWg#4rVZ`I#JEbw;W@@jc(C`XO%G6H=z*%OGVxwv z;oj9T?=#?F3)G`3#SdQPw(hYA)jaHREiCoV&2#UxmNqLh4=P!#!-eE&07gG9w=&bi zX0NsTNjaqy*Cbo;)mRq%0DVYuC>DRrP{5<3xvr8cSMuV}u3cS?27MYE_G%_B$p!j<{F-s&nXgiU#o$N@DyZzto-8n+c_Z+y%GEHIATW*Zd@=38{YaObGr5p1~#q zgEoI`n>T09Ul`>BlROl{*vylzXL50gRhO)ReFe{LJ;jAZhe4STDfq3EDP$ZKs3fOz zrultz4B!(Kf~-JRIVF@sntd?sRLBhu{+MU25yKrAoYImDZJj3D5J8KwI4Z9yDpyS2SJBYibRKF=?S(k zgg5hP)1kxZ+5vD9l7l1UKn^n58|VyM1R#-Q9N%f6Qn!NL*_Q)wEIkDSGj`kW4io^1 zgY>1T-0=z;x$98BPM$t}`r;+zKVUwf^lfoIjYWXYNw|+8hu{-0FR$z2=S~rAE&JrE zbdT#sYJutD#^&Q|=FGs1iWZ2oFJVXr+N?Pkwzap^QNI_vV(0@Oi-m?rNY6lp#<4$STt5Hqm_ZNne1KzTOIRGPT>-p^kLkN;Cu&X~m? z4fqA9(Mf#f)+SCXab&q<53eQ?KKE{X8uI4TSm+vzFvZ;@j}{T- zK`Qa|xVhD>63F&_OK7}U3v4zOsu50X@C(Pzct-23o}k|1$XmLfttw=4w|(vHC9wgh zNWEGgtsux~x}=gSN>ZYLz`8~v`ZScw`LLoOR%I5SKceDe4<%NnS?MEJXWGhj2`R;> z8!3x7AA2ShKd>_0GfYJ_YEWS2%n4UV!36YZg_Nz%-&z1Xa!Q{`*UkI)lZLkH+fIZO zt&=bKI2ispnUL6thqiAznYa&)bMVie;62d^_8he8>s&KZP@E8L4N6Wj$p8n5_?GDo zYMAph`9is2CbO?5yPzIPmlD|7vH<-g?x)xIuW&&0Lb(S;kbs`{@_W(I101}%b zL+9tvkjB=n6vPNGi9%sP@Cc}!2ro|@aR{QliuKxEwlsrAT2Uh1fin}=js%`$!q8-m zb2I=DJXxcqkDot(-ud$K2p9h0&ieGk%Hu$>36Ry2-SO}Z`jWLMEXWKq%*eWgh%MYj z%Z*~H$3oZr7f5xbTtr3xB+c&yg5B3@yP4}Y#2Ebxb2F*Pr;MuWDSNr6rpHWtC_2D1}w2;J3BemxF~onh$3DV1Cm7Et6~sM) zQ5acSSrGvDafZCX{u;ofn7y4F$Or$NCVUHXF2Lo?2OR|zWviZ4u&dog%*iA?#6t` z=YL3`zeNW+@8NFAbS_jqnusi#hr|yMf|L(mLQB4IK~R~(2}ryu#6FP1k~rasUyctC zjZw(=mwGIk@^Or8fgc6Q#EVz0+BFiqsiv52jA|5X$_odvVe#qz0kgFXQe{vhlgS}~ zYeiykdhF|uAFihZL7G1FyWvpE3yTuhtKuZOV>n~r47cunJS#H+yM_d#V}$Gd%`USEPWd~TE@o6qh{83t z8u++ZP7?e#fvM?SwjC;5GCzj2fF!=*kL`z0;3SU#Q=)SDT~QI>`CkU$ZiWO3P}f&- zb90Y<&>P3FDl)7E#TODWcHo%D+@?ShTicS<%O#`rR z{KS}Bo&amev`ZAeU{NOnBK1k!M&1I_S(|Cxg|BkG+Huo5{`e#WTa@S)S4EQN=iKDN z3sPpy1-CWlYmS^1wcx*1&3s>GgL?sSG~4iUA9^yaD%n^5V@D!RSBMC3k>&@_Jv&MR zsk;^S7V;t-C}^(&@0xXM1R#|`~|!y_l=o`alHrC13a&{W#zq)y)mOOas$EtTqqZRV^25CiA|W9m z1|%fm@Ch!iHqnh|BFb%fjr1C>BqXGSUpUXIxg9y7ZPv}yzr_hQq9@$y<|3Zd9QhFE zAxZiGo6^KcgB_`fewHazOdC4G-K%h%JGF7s{)1a`0kgx~=<^3Bjx=dVx8cPmAakel zd#+A7NL{1H2#y8B@EGzDNnDRM=mhqV00#sapy&MH)@B7Y2tw9LmrIgX(J$?w+Uo@$ z;8>@J5o%%Z=g-BES-Jj5Ek22cM#Tk^7?Rb1^DQAR?z%}vd|HY(&sOZx81!TH&n<(> zH{%!NZ-yYM8t_s*okqX@#AAvsgAQy>7dYkXG*He>nJVjwtaVPYPsGR%zmr??h~!|w z%jLVvm3srJ$-NDkD0B$gQYLkun!yzz2C%^l{7XK(WJ z9ls^^t{MIQovNp&F8A{1BobQ>jfD2Y&3CcT{CvF+FS2g9tV6th-4ACZHGqrF)-}&NTY!9=@75)G%((8 zNGBBg&^00NjakhZ?$exSaCf6L;{#g0-gU~fAo>-NQQDhV8@`Z0{S|293g8$gXCw%m zI_Hf^*HCoO=2E5m4AtF z5*PlB`hJ=m!%V!Ji0}cQ@}98yiC2p&9`|nRq5b{$zr-|t;s2J_bVaII{Xzmqg^Mw) z7lF#7l8-d~53n4#YxRGQ$qSqgzwB4M!J=q@xj!?@qenzSP2mDdXZDR2XGI$>hI${% zEoC6n8K=um++M%1nBk@rqb>dD0FUmX!a)a>6;9=a>8HPhHM#c4d_1Z6Y-r^YqL_xU zbN}hnPbxP;nYbOF?*U!i2nCaJx3csqh&flUU3(phl4I%b5vKjNxLDJU7-mo6C8|`R z%w_a8gi=J^b<*X*(J0#1E1q?9T*k}G>xu}nxTq*^Uyh5wSxE1E#*sb2~g96fBwwYlAHz6Yh| zlHUhib3n*(UraQ)eg$!!{0miX6VltTtu#LTK#l?DEHm7f%+Dj68^h+7g5QHl_vZBz zKem*Si#Kq0Y4yPFWA$S~HWKli&^>i|E5#{5k95%ubOM}=0c`!b*BO3J!lx+N*53yG z$@ua4*15jM)zY9`k*OopkXI6q>BH|n)887>3X5W^2hY|@SbXo3XnUeoG$N6{u(^H! zssCnXaDrVf2NI=ZvBG0dtjfx9`LC0p_~oO555>7Lr^ zmL0$duO0|DVC}J`e@~*ue3#p?k9$C-c@hPR8AL=}1lBqeDtLGjeo33vufdb>82Xv> zz3FuUKs{vg%Qx z9~NVrPPbNF#A-eDl46M1^fqjQQCCR=Cy;@vSQUMC- zRa&fm8rL6@#t+s3i@VLau_TrQ40k6Etrj%nX=#eNn@JAgh**z6+2Wr&{uO(1z_gbgEr6 zJm#8&K8q^Fh8#c>LK0voUeNZm5U6suhz&s^$T$}~W09lxTo4yzW*G07T~M!S&02vo zm=pY*c0)`;uT>xNWf|pxQbSr)eWq$=(4Up^9wKHSfW>&$1DK(*7|kW#Rn@(e3>Nj! zsFsOZL9Lo{gMh{F)nO^m-!k@j0ZRu|L$v=wu5Y2`6LwJbz`fK~jqJ92tz%6F+!AHZ zEfv)b8SI-cJZsj|vZI)qNAjAiSb*jr!h3w|fu#I)md%Aog+e^xXG4C!!Q9W|HXh8b z*lLKqACEJ~$18Cvo>KIHqGoE31h_>kto#n~4W|Vu^{EzD43sd@^4YVSVe4)8!2CWh z2BD5O(OH$JY(ZtX6ofDn=`A*nF$T$pZh^%~HhjMGP6k?K1VtP;dQ|rQ^o1b4Q4#q2 zTAz!#<9iZ~-1qM8J)oW(fuNGx5#5xQ9VC}2ws@B-57Lr>h*v_+Vuh`xXj#6%EGLpG z1=T797oIy?lgevqf>q%ZYZ(IYFrD%x*X{5#_jqQ>Zjn;VZiF=cCh>AVb~!A;RWiFa z5D)Csq=B3Z53N+mkS`8W5BH<;_Ng%MJ&X$&MWQ1}7Nh7o?_USO?i!swrslEp+$EMS zsROGmUG$%%UB60|Qe8663aSsVi(exX1yUJClVV;VT+sjmZebpd(&@%KF!po-x>Om0 zNco^x{jR$diLKf_x$1gf1VEqaVhDWihVv(W6Hvk^Dunt_VYmMIs)!Qbi>U zb5SD{k~3J%&`Iv*%m6gdg~Y+fz-MoOVdaNZex+*a$cmlS_bcqUC~(?ciEUuML~nOU>6Pm0GqyaXdwN5~Lc%F; zWHOCsJ4{)=Lr6Y)@s0)ad|Daq*dfM(yN zqCgkaF4ZX;xtrqQQzR$f6RJASC-mICJ4CMg;XKsfm$omd95cdwU*Y_GbT@-i~$k`Pd%2+o(}Yi zsGv@BV5jN<>)H|2}N?(p-q*gn{?7uwwB z8-SB6EG(MVS%ZXoU0po~XXt6&oKFLWp^U`7u@wi@QcDkuM~?e2WAoa2UU=BNWj;>! zfG%=QH&wkL4Rz^kG)>-Ejgo8D@Zt18`)|AmdzJ_mi82g|Pap%D#}}+ZS%tCBXZjkI zNsc67ZKO%-`~8B)F%AK^A6|9TfYyv6Q4;BcV42{N50`IKq`TB9ZUQ>&cj;ne)-VdblUx93#adMhl&xBVmsMccQx zHs&qvO%NHv+&L8pVkVfD-o~XmUbm z!lJG3cRuArJO~d*a>)j54Fu`Dgtp`Qo@h}B1z;cetHBZrz#zwQ^(Y%;uRMhIeM7jK zk(Y`I_L_;daXwHV^28}m$vD)@LV?qli_!`CH-P{m5mQBtU5CXB{Y}=j`%s1YZGW_W zA(h1!y}@yOT8%{9{5y!tnM$M4L`6jxBmIU1j)E#wZ2yb^Fv=@vgSXpFVC51O9GRs8 zDbZ`NkJ*YQrFFq~CemfNkXLSY9o}`uWZ$pI{w1TIvAj_QTxxv-T!2S$ANSNrZl&co zVqFJg8X6iVQ_yQIzOf59yJYYlGC3gFB@G9Z)USzgV4WmU86e9BW&lyV D4UFOA> zsXd3;1IZy7Og}^J*tGX@)oKhGYIEh z=;+;u3M1~xlOobppKaSCs~z5(A_$HG?QaWy+V^_Mk~xa-#6)8h}pKno(9 z4vITsdO#53C8wZUE}W*d{^&69}Q6 zKSs5a$Ud>y78_cmqerNY38N?c=FLfXEM)Ug)`H|D$~+Ynqt=PA^{doLjR(t4Bb_Ee z!eDUI5_2JR23^K^PYT*oDhAP*@6TS5aZY)9pg=mWOtK|yy$`1tt?aqknu0)SO>1X)TojG{d4 z<4s-x^jsaJ1#u~1A~=O`EC*uZS)a?w0`5IIg!PZdLmuRYW_MwT%jRMP-N1nix)Ye7 zZIQaKK(JaE0aKjPpeYT@R$wz@dWbbrI|iOJ{C&h%S5l#4pq9YT()=@bdSzwn${);k=%*_=yLZRMghyLoQj6 zs>B!r=!loH(V=}NZV=+$5Ed3bZ9#(k2(o&~zGjhMLh>fcY0C)DO@#9RLJbYuX%Bv^T7stiu?QL76tnE&378->VjwJE0Wq-1Tt^?#V)& zT2BFcr_*xM(y1~qg#C)Lfr|ecWDrDkL#RD1kxi<6$<+Fy4Ux>QXYNWuVbnNU_mRB3 zh7GAbLAorAAkio)#t~-kQ3|CS>*y)gG=`dOL0UMnreFdh_9!5ys1$rTv0@ZB2qxpK z1J>##f7vc?d@AN0N>!n(>MO5q9zaXF6TLcIlNXPdhS;-2nWt%mBn{QsO0zd{L`&SG zAvOeLNl|}c8r4BiWuq}|e)n_FlSjA(PbAb^U#w5#8hed|-?QXq%V&nGGdlLxWpJ0C zV|~8U-B0g&K`Y+?hd}sr+i;%|h&AQ%7<-#1#Mg1G>gi8p=q0iO@p1|Z`fYRG!j$=P zVz88OFz;4h&VdB&(%rlBF+3?Dq`k9q8XlE#u50inTRjm7_s%#wA0}^`0IwTehgSsz z1|IBtC$?Zg0NFl6fUkvc^P==gnAjAljlE=QW=2MX*vC#I!|Sh?wsqDf-+Sk@4f6^} zWE##YY2%iuUvb0?UzU}KWP&~=7EZuWU%Z1+fxovrPclkDe`Y|Z928oHULVW#+e zzwMXMzyOWk*W1@_jyY8zJsu?`X|)O)b;tNGqVDzj^<4phW1#oA?_S(RH z;YiNm&`%@JVY39VjM&a&1_lO;F(s45`x)01(hsLvX!Tw>g)G|JWZosvzc9SIqn5Frk|7jVInY{b|_S~p@W$gNUg7xGpVVdONL(g$>z*XZ+o7e zK6j#c4@eqN^|%Tr>ptI~lzVgp*Dy&SfD@7rZ)m}$b2Bl)ZVVW9_V`K|`k>u&$CPL? zI`%@?+z2!-m&WUIKNDqM01jE}&AsKXUQwWdd;@7W;r4>lz($inF%Y1jekImWlvdmM z%gyU~v2}Orhe5M^uTOsh6&HXrKB;Dyws*%$^tt!S-7e2`_x{$JNZYe#1}?<4C(1;h zG-5xK8XSjq>(p+@7C`R8YciMUX|f$+J6$KgLTPvrI!BO6roaT&Dk>7+ETSqaj(GMy z-tSXSJ;KE4?lP1@GkO-&Mu*Cf*or`ykrIVWA^^5ROrwp8kZy(=jRO#W#H3ja>qMBb zHRITYvNRK>=K1p*A#c2nkU;#b$sd|LaUg;jdxNK2Nlw+gUJm6Ob}uB3>S#AX zPF==DE$ac;hYZ97!A(?&s3<2v9q|gIdlBlu2N^%n#z8Df;{N~xZs(riwg#z@%y$f1 znu_kJ7%GsE3scS zmuDJxA~QCf+bG(g6UpR@9_r;P){fi?%3yCuWp&j1-laDGx*~0R6a8Wa_BJqUM8}6} z=p#DYBH1=iCYV7IR-H@l&5}?}8b~$9A)EB_<$a8>y@Y{sYthNnKGqZnF4n*!v%a5+ zX6eyjN-1y+4>WH^tyW^n*t9x6q4PC$J;1-v?e}5NeE1F=jH8L{C1K4aB@I&Psss!e*+unOaJFGsl@2t5N zYiY9e^mLj=XppYEJ41wZUXgdyvO&uLaE;gG0SlclH_r#P_6E&db{dMpz_z;g zw}SH6=;fI~d8MdhtCog?`EBNMHVlo7oCuhn*^JF(h#U3}>Q5U8mibW4#lO(aF(4<1 zgw;X1r+aCptZjpc{Lm0XCvv>-nw|PsMv!_>(4AgQ4QA$tr8@J4PfMh-6C+u?sLhVU zLuwuq6H^9tJ3*j=Lqij_^ha;p1u$##kSNHRBjGug7q)lwr0qghQz}Z_W-(9kivPn10{b*qB!90O}$M)qr)- zvgCvWb26&=d1j_%QBhG6zUBvd+uC@Xx|T^NJ`?4ESnF(SwW6^Gsup{^{qU+epck!h z7zLEHom~%&b>EFdfTfr}L}H|nMq8G=5>W3cNTH;TD>`zwon*HjQ7Z_brJv;99SJ}3 zjiX?}-;6Gns>SmCO#AMi5~3Qkxh8|V;sp18in~d>$RpcvT-W3EhkuL}W_l{^Q|$oX0&>k-|ov`P;wv4jIA?{1bA;_%GKJWpN1tVOllk>i^TXFAL}zKc%cbP10MC z{Ofw`)(M%o-OmkSF|%VrQs$I%Z(}8Yx*GZWx6ge>njhMwxpNIWwfO$|p0r&RHCtr< z{t1GE6X#FNGJ+}fFPqvVq5G$y`<*s(=BeR9)OddA+m+5^(x{pYF2mV=f8ih0kMHMc z?j=$Ekfuarzd3K+U6+#0lKAtI6uoP`<^d$b0cZWrKrGrFTp^@)N=L@ucL4~wzH6CS z6@)$TUfkQbY2(BChNVfAv8pI&?cb~^9sO2TUi zUk?jT=uu2-2aNg#f7IUIPOe$FV6%sTYY-_It|>CGhUDbh-HQ7VO;fs^(I()VC_&{4 zU!~!pK{@U!!k^N@kx}%VCPT{`Ls@-? z5%g=x34R<_T$E<;;+zb?SVvbat^b99Sy%PKQi{z?%+F8&RC|6J3Z`Sp5SJ^VJ)oME z2VgBLvuxR40rE)(sp>WX+G!?-31_}kNplJXZ`$l;sI@w|-O4mmg!cbs)p*gs(>LG?+o(xjl zqe7hsoZAeUTp)oWpFS1Df`_uM<0Gb5Uqz#4ivEE3#UTRmL22#K1#K}2JhpEyn-2|0 zLSeIL!AQ#GuSYl+vX}!5wa76cGGGicPcOW- zK=~4U8AK#U@n&V-8St_j=PO)U?Wz9v#t569*n$3COE&a5jE-^c5`u!S4ka19_Gp5> z2S;*?Dwxx!aN99I#{&m6g%tW_)yXw9w9Kb4RK2kSk6HmhEqe2$bziQ|U1P9y>jk4c z9NoVzl+kh25htLcm}A1hU)~81mpB4tvxB4KF(_~km4>7CC4-&J&t~6WsOEWpuiWa@ zW>~}^6GG*SK7kuGNI9BY{VjP7=6qQhMBN8tH1&cB+LR3v+C#upp`jVEZzdPy_!x$7 z--#P5#N@?_iXiQ9)dwth;EXG7l^8&J4mKGgfDu5)XSzT-SU{a&JhAR>03_lLT>ycM zCjz&f;##lDYm0>heB)y5z7YK&R|c z2wS&0IVnN@BBN4-bztBBzO0NO6A#{@&Rjy+WrHI}NPNhn(&T6v4(eA>>lm%&5V{Yi z2O3ZO2?bNc=0QB*B>P+*0>D;~@Q9CS%}YEYB0sQ!3{LO!K&8Lm(9jRS|JUD0#U(%| zq4;305ik*RjKFDw20Y-`{N;;D#~_*%0Sw?qjrWaIL3>52=wesx#{HASKQP~jaDn>@ zLOu-P)~;o<;r`p*Hpqv%k8**O^elW_``H?0W0AJw{(eCz?63sx?B8dtW9-=v-=HBU zns74DAx_(X8!tfm=J4E?FOksg57QG;qX@f*&r83w7)^xDTKBfPpo{V`^0KN^rzRAi zv>p;Xl9!q)Pn?D$-y*P80>3eQh#@k10*{dN{?Nb z$-7!KZ;*#5v>RFIHFiboZB$X|K5$vJ?L|(`BNQ13%29{$0fOg(Xg@0n+ks8!U1dXC z;lVVrz&a#-)p^=9@%i(4blLWY)lBeaCOY*=AwGY83k+C|&Io#FYx4J_YPqX-htObe z#K%V_fu1;V0!I7&yDKoG6((;i%*uHEm2Z!&LtfO`F8*vGOUUV4y1QqGUhgu%pd^1t)3V7&`3<6HzXNBe^_lWrRdH1;R$biNj!9LwBvbuk>d|{EQx(M zIOq(!b7Z9Cx&Ed=%;xSZ;ZH~3L2_3xabgPbk9mw=!fJxs3f=q_6zofJgQ4^@&bfng zW#uybU$grVzbrK`05`H34#2faN>eE4T9zPj2LNI|;Hygjc_cwgZG9ijhfou81{^#% z7r}D`mU{5+^lA4}>M6EZ4hQIG{=0WGKnYFNb=bPAN}rY{R7KWTeF*)|*jT;JDem&$ z3rulKU&JpQ6P7rz1%bkOa2pZO3;8`-oE0O9#&U!wppH3w1i1(X>p)kNVlRCG0 zzvI1tTYV{rZpc}$`1wu8q=|}XNI=w=+#sqLL`(@f+q(8D^#|_vGyy8U1E2M%_ms#H zp{v{n&P}gLX9_PtR(3X-Uj$%BJoNPM00$!b1KXx?=FC$g4pF?}bcbQ85}gW}re_<5 z`8?Bzg&wl1{-q{8M-ZGyAVnaiWV$S|9O5(V94jm;w#3Ip%v}H=r(l&QVUWPyBhQXa z2bn80XO1^SW+W;QjxJ+Y2pm!5;pbd|767W`t^?;BgpD}n-Nkj>F>+EtN4^bPh@>6C zk7Awcy+=HfBY9%tL&gL?$7)Y=rrhs>GOBAgT%edE2!#&Rddne3xVPDt<~m;Sh~zLq zr}(0X6|$M1JWR|S0G3?0tnEgKPspztc$y?v>oVm`f72Em7aFm&Ch;n@+zHFqc-jLv zxd}D^pH=~MkVsnj)M)m1HR^J0{v@RhgKm)P-IMwC+sM3tvU?i@p!X-?B1Ccvca@T} z!S4kisQ%Dmc5QXV@nwJi|1_c3ThISFOG&97qsjZ9s{7-T%5DEH$)^%S55;f$B=m!_ zBw`4aHh89Qd5M@Oiqn;7Q*l7{v0y&hhBZODE@$A(svxfjPf8J$H~(wdY`^-wk-D3E zRc7ZW0@dyb!Uja4K?Y}$GSRAFm}=Wo43ssi3s=|I{(d5;mfeD#fj=ukp^)AGaI?!L z(2wRwJk!Y@Q39q+tfs`5iIa)ciNxqmAvLa2tqEQT$T#Cf_1$ucboIy!+m~!ZpSYm{ zZZt4hnC&JlN_DUa;0!4?P*|i_SjMost zJsvax*DMyj#JjTxTjCP1U?P}>4*n5V5SsmOAAOi|%}B6Cv3UUi0dfX@ZBHmPFiva% zL4Bb6CtCu=A1}on@{jX_->0K64>u3wl2(%lhH*gyJ|&pD^Q*IHkqxj3C|}7w1c3h3 zm0?Y}ke#g@dL@JD{{R(OcN4*%yRO_#!oEM zX^l>sd#u8;uv-xpLunH_xf zNE6)bV=efIpgu6#OeRJ!)T?B$WI_}lgPO;f9A~Hry~G6|fvDgo`HRREW!blyT3cU* zSJDDP(mSdiDp7i?p`1#EC40sA{)31@MbrBY7%MVYcqr&e&US(6Y-u`TXc!6F-U7Y{ zfJN%ub9{w$dWvb>&AlTcS9Ml#Lq2lqyAK~c=tCsSfZbPM+fXRPCWQc_gYB`s0>2y8 z-E9x&5k)q1(BSpA37_|)gL}1`e>B%YuB%@wNi~_ zJbps%9~^eXl1*6*iWvvS7>w;tVR1xEp(#dl^FQ&D#Zn^g6+KQ*aF8scZihu--2um5 zHSI5)_jAMZ(rWsFt&r%VwPyL=CjzehDi$}(Nb$;`Ln1=wWh+-EYM+Os+4#jde$4hD zHVj;4SZ7bxy$5G=UchMCiWLdQItZL-d|M+K)q)R>;N%!80&riBNX-v^9A~E7!E46B zGYN4VFL)_vj0pe;JQ+iXE+AEQ*Rq?jx_h0lg~Qzkr$d-~8TE!KIfB8PJHFGXV{UMA z*N%*2$Hv!Vci9@LvxaKq)YK#>ACO-|AwF+2{8m(kaD?9|^~zBDbuv|~NRPcjqWJ)- zS_{WSMZGF6=7lljGN2?PqZzS#2xScLv+Pj@8RdtltHw3Umq({|Z}V*;QC-K59W#aE zoxBaq8-H@MypC3wB;jPGvbd$AgBRx)@wP*&tKNE0%Nyk_aYvFEG=zW1!&>+f!Dx>< z>e4yd3#s}apv2h0;wvnU%_h)}b;SY~M$KV+4BcB5yk~7umSa8V_w&Cuvxg=Rp)Tnk zfj~$#$Hv5v^@zrqN7rFtDj-daws_}=tFeYDAaA0tW%LY>D*`P75!OQz(gKSS=6(3> z?dYMK*995D989@@^_J+4ZS#1bWSaJCG}Xic^b>XvK45c{leRj1lR$uXH`vhJ?d zfiTqpr70dvfuFlP9NvOPXMlp=`l2K3+_kmqIFEN*WY8Rn zW91^57_%XcV-|)2* z<<5^ZpfeAxSpXM}54gLBJ!cmVr=YI^;VlM`fqW_Ab$PZj)=B^8(f3*59lUV90wLL` zGU`HT;u>cy)phv)n0xbRs`oa0cxzOYN<}1TP^k=w24jYFqKrj~T~tJ(BJ)%zA$1yP zK$NMHlv(V|6`2w;WJ*ZpOxw2idwp=u^PJ~-e(zuJZ>@K&v(7pvV(;(%4EKHA*Kko9 z1zO0Xl`=QLh-N9`tewu#r7a;LJDhr z7g_7X?gmaUO*mUjLGuNkh}ZNBF(kniSl--U+%`YEaweS7?1)o=o=`TBMU>5l{R4Ons~ ziV-GN>csHceiak4HR9s0(sEZ%{rJR1wo$X$C@ADme8jwkky?zj5qe=%enPnDwY17O zZ^;XQab7?qFG!jNs);V4?_k)?n-npz_&KGGe6V$m^?yE)m}M5C$HR~npK+*fbkz7zvD;-`wc$_8H0qDH?)RVUwH zsGQFy0m_+ONW!^{jzSxv&u>_2aj?d9taIv@!g>>J+=XxRu$Yt3= z&3e0MFh!3L2MEP&f^J6+7YqPvpenpwun2?Hrluz0t#2dD3jQpL)@-Fb!|mWghQN|P!e4@Zm@HF60JOwpO)ir%%d{|Dk2J7;OwM=BzS>n z947lfm%LE&6In5Mqt8wqhW|E>(cs|V)o}a)YR4^bO^XSoE}1!DbcKO}CuRp1#;9=W z1;!4M%a16)GM;FRz?D$BOZ`uf@L8?GCo(*OpoCn&7$=cA>84G8=rR5$SZX`Yez%c# z3@K+M_^a9YLfWzQ$m^VoDI-c9-;X0Eb=0E_K>R9BPU(iS3KPG%fyY1572x{`_(lbF zJ)qAmJG*i6lT#lT>6}lEI$&Eq#vqVre~E1!CV98ifzjll&rU1F1H1}i`+oX=z^xNJ z`>|1fFomjZ9j}!!U|+3Bv(33zN%CAC+JJ{I80|)g zR)R2Y782z$-Y9d@%b4n-&^EJYX`~*9UoB{7zA~z?pUAV-1ASn(FDun{h zlMm>ps4>$zKUY+c%*eaCqhqA4A~ZWp5=m0jquqA%QG^|Scn(Dk4>lI2LoD93@tiSQ2wW8FIm9xM+cx4kPVf-8IRiWP);6)R9&7KxLl z7mo-4*rBbVID)ewVTj(n-5bW{K$?nn0|15NM=bfKSJ{)xZdmDKjCP*zfU z$D0dWknawTU{1gT(-yi~t)ssaVL;P@vP^f^zHJ~NfpYlx>C>klS{y>h==3$Pf&;|` zluSSix%gy;~4C;W?TM))o$6XnPfpb$tH zdPuQ!A(%*C22($mh;IAfl#8*z{_Y}pu!t?uV9r$hbV>+1WHM(x7$zSlO{*EMUdlodlqR{2 z{8mhh@NG|2S0%zFY#rP}50xl67=xsuxaO8(tez64=V<3^U zTVSPb?9#7#*XnNBKR$x{%fUje=NhI%TL#4O!^Ck41)`CpPS~=4$m~U402bAG2$gy! znSW(vi+p^196&2lZ3oR}uD4Ql14U=et#|eHEBz(FQ(}u5D}NLkdIO*$Aw$9x`bllz zX{$Dv*|B2^h#jW2i3U$vcnoQMQ2uGvGU)&NVv@ETr; zXM42x8qC66eIQ^U$!egbh!RD^0~(HldS}2_Np@-*GsDMVutMNikuGXOLQJgQp+n(V z5#*#sBsd#oiSN@7tq!XQiHm5cZ~)sNL5#4XiH0QAHmjdw(GK~^e*fmZKcWZgx3^hJCGG zmhh0fZQq3DM;_V~o$1!SdxQ~;mC}R~;~gqryX%@q#{%xU$Ke=RyKC19#ZQ?Brx{>P zy{GC3{|n7)u(Z{KnBnT>S zC$R;_>wVYIAn*-5&`oeZzkzCik=j2l}^f!ZX_T(Rvu?KP~9UqO5) zUQp@wm{ zP&4ED?FXSd_x*eJ__qVsYXetWwLQFlKN|_GIG>5%GKGSXKUT!jp}_PSc#i%DYv{nK z=p#p6Or%aT@on|5ITkM53#>lq67(xLeVW1etpB>nq?XE$C6k)^yH;KMFD^hT$;aS@ zEial&;#sj%MkW__2Da$uAuk|?Dm_h~2KNIlL3<6>ck4n!T)+@t+kD0hV28eg1suPQ zpFHVB7znO!H*sx&T#XNdxt(@DmyUeDO=x*vzup&JaAp<`)cX6IlhB{@nu?4VJp)Jm zkh;0c6za55PdA*8g`vKHS03psfg%y#bU+vu_amP^J&sEU z^m!HdmP1Z=9I!60mlpH?1l)>@5f^yitz;-*n;!OmKvs1s-(qKj=t6R8p-(5?-Ds#{ zAC^U@U`tIx3BW-T$>4`L0I}^MF zfd?xwwHjp%mOlJlK`bWGk2C>dq*yF`X9Sxh{RwM%V0omMA>9j!5gYiS}0sh zq1!NgGdaw|{Y??US2x*qs#V`hjv`DS!PP;seZ{UNy_c0f!I*eI!0 z;kJw_wQWm)z z0mr^}HBHeYuV>+KWuHHPhAAg$TVJ0l;@iqR9B0^!?NA7b#fj6WeF$$EIPUKJ_iAB` zQ`-pAPrPg~pdxM|M9%M87Ija)n;5x3%5?%H4-~WF;^Ix`X4Nev>6qBs7s4<$rBJXn zJL=OU3Aq`h54@Ni=1jAIW1?y>xdCb z?$HtFWPRuk@YUJvQ~=JxnxUKkoH4vT4%s(=HH|0v8`hz_BgQyjWAU2eAmhUY@c>0# z1GwmcsPxH#pb!b33orvpUI10onK1>&YX*8rwdFT@$zFN(>=}us!2$Bq<>LMd*>CW@ zV?c(rTOoY9ajJOTVY~LA64MvO>Q#hS9*4KgCxQ9pE$I3j`kL70EL?dFWjVQ*_i1RH z?RhobR5?XR0?sXci%y^!S|b7h5yNSK@Npx5aUo=7{@Ic#Q+68kE{^tEH1L-FY3u8$ zS0MX_(`&=x)8=#boea3eJ=rcymZl4mt}JIebcp>V=Y%`+h|THS!-_%fMdHV=E<47v zLhddlsdKy8!nX&*e#no=T$pu(V>!e0qx)eyyg`RmvVgL0wrkPWsx5lYJ7dL_yqi4| z8j~;l{h(rN=)?*!u{)3Cp4$nhag=R}kg6R$^6q^l`-_2jB7z%34R4mt^kbV@cB*`h z!6;qn^eOMwkloGwAu^u?4nNt3Wbbb)R=f@3-y}+%qbu@ITx3AAzw4%uc-QB-^6jNN z>^Dxmt($uz1{-e)6sVVPeKzxCoZ*x7XG>10j+}3&acTE8TD;%F483t^rpCODMSPO= zW}-fVSSCHnYx&2h7u(HQ5`*I+?uw3^c)e)t#@&u_&`LDyMf7u&_3 zrt-R?;H`{W5G490NwqQ4S>i>+jBjS8If<6~NwO#5LBHE^Nv?=UlVjg%|GE^OqnXjh zOe%tV3L;b;Y<|ad1lykK=?-vH3p6os+{M89OyQE(`okU`d7}I~BJkpQ2b$P?WL>V^ zkTjb?S*Nqje~(8+v*g_Ov5#a%6)8WBGYoHU4>R3&Uc%?zop;^O<5ycXJWwBzbyy_I zFa5YtQ{ja9AM@yf3NJqoOb?_M4g6YcUzS7ZcvC*dXoH=1WsTRkp@`(=;>2YqqoQuE zC^~Mw(OXZ%(^GFo?9!WjKQCP=xwLrk`$ya7snsQV>Vzw%c+C=x8?Q~ei6hKEWyiqb z_`QWk-c~c(p1_yav5B%2?$ANZiSy@w8Sn-N z#~i4Z-oErppkS-l8%Z~qxt#j?mEuuy{=B)Q({|AV!5)&v#xYK#i+ZT;pFVAPla$|W z*VX42y`g0Gtj9?R(+v`vOS#kbz3H*aWqc>aB6ROXtu@(Wi`S-(u`~`A?D3toi-X6) zp9X)jSA5%MwS!Tg)|`7n!e({4sL?Y%X9M2XPajHP61C;m3&6~Kf$~)Y9_A|^_g+Eqbov@4#Z>ieK zkTzR5V)Ws@q{U~su~x2nhjsyqVjI;xTy_V~lGvpo&@i-9o#y4UGT1DKAMN6$_X%P7 z6}CE$^f`l{B0q-3Rm2)N!zsY*W!cC{FXz$>okjUBdUQ@u0RQ>Sl1~R%^_l#UM~+DK z-o9P#Ht>D#=CEnVXJ641*4AN`eW|LiCmZKeojVP21-TrQ=aq8LU-R!s9ckNPt+eKL zcHXyDH}|c+_2KboBiCWd#!tn1GUi&&@7O5&n@o;svkn{ld|3INm2Mi09>OJmz1i;Z3hyYK%|NwaKj+u{A}&n0nZ3cI`0)MYHF zT&xj;YI(@X1|F?^bipnkizO&GS4d)!tNNWergpV~hxDI6{lg(tXY#(rzrKIn@KZlp zWd<*=#hB+Pt?`bI@}_s|HfC1!33guK;_7kj3YE)#_Dt)1`$Ca{!EBMuCu$=(f?js+ z4UTgjW{hV(-yE#tJMi|cS(J6Z3{Fp6=%;hdB9*I=RPS_^7Bfpf8J)J+*MVLap|K}DPOJ1>**h;qe`A zvN+ZDtl@U&smh?bc#8Th&o*gNG#DLyT;$x_+`~h?7D_uDmg(G_=B^OQQvN)N*WkkC z(RTNJW3k%#%&v1*Su-E*yA)bX3%CE}>JjHi`; zd3HzErLD4HzfnPz(p=R4sbueF9-ecx?^pti-QQxlS2d@Nvz3)7ynFTnDf{$|gg^KF8zZ&zW3 zxfg_e_>h`;&L~&1T=`;CxE2I8OQLYn*4s{^G zxKGyhn*H9x^Yk`Z7=LLi>+%55+);Fq<>n9~zaUC4IhDU@deY6ew|n4bn1;rKwF{Nl z%Zt)@sAqc?O_xjo8r=hU!H%J2ZPPnlHe;QC`6{{T|68PFxaxeM>|ZyS68jmG(&9aS z(*X75&v9t_Vc8a7%mid8VP5ulPN|qQGg?(NNEz_Snp@zpXMAlbi;Bz(f)A711Rb&~ zq&g&jZE>hV;XV(JR88_b>{%N$_0^BKPcEqUgKAxS{Ls2OI66w5+7h6Xb@l38NSPnR zF@D%WGPf_WK1?NC3lhJ#zH67nN9Ls<4iAQ=1wHeZjkLtnBYFas!Tux$?s}mw~exj)TJBhrTwvF}}ac}40N_^L41LA^EnvyUvFzErF5#z;}0k+2gkASZbBoRDXys5&-^&2)^ zjNV|GE{p1a5kQrE7+4%XcWxo3CU?j3?@;vt1d=QZ=+*oz1j9$~PPxcqN(8^28E8Nb z`Sk*4AwwC^^UGnL@OG%`D8=K^qm>|_^%UQ%y#%s?SJkLp5?X%v1S*RL<<{2n5vGY! zcvHn(JO}VF1<FbjG~n1k95ORK(n&* ze9J6j=WZ$dmJl(n@?;l_{vr49w$f^*3Ftb^d^Phi_(z_AFoL{Y%(|pPBXg=*P(G-F zk!T7gUxev1&9LERd^9MpNOuGuwj5wvz=>25O&BD4dV+_KHA)YZhp3*OTTKJ9OS&7j znRC#su0WTt7htV8Sg0^~xqjVqa&zAFsc{h) z4iNu={yt7NE!Ahj8BiQHwU6YSmj?b^9m5 z7O4~3R&H6$MgzXuETz_Jcz%&FF>x^=UL|0BzH;bq+3Tkn!|Hg$$a|rn)Mbwj^M5Pb zL+U)H4$hEQP8b|!W2ZRxHF$Dx}N`Haxp%kb-pY2yb)mpRevelZLPsHrp}k;>cR@kP)!6Oec-js%5E z09BJ1W3^Ys2=7risOT#+U1`4|lMueOuLe6Dk!o9yy!HYlr@r-DI>tJ|KB^x#iHVs} zY5=7<4lXvuj(>r1O1&8dtI}AkJ%b=M-PaH{h5R+5!n@@t%}uTg!`6{q+qV7r3GAm{ zr^@!$)?5<%6w=Q9#wBscU80DVMei;R9&T&A$>?f5>W06oA$&UlP@||EA`U>=wF%3A zNlUVW(;&|>eIt5eeA(zJOUojXCD=0v`a&>RiX%tNC))i2C78_5yhWPTL(%70`~Udk z1iafI#~>}YG&n>MhhU%n!nEwik2A&n-c!57(8A0R?o+El#{)TSw*Z43f?#}_O%MbJ z2lGOJsMl(%v?JyFW*T_o_2ZbEKTc2Q12=iOfGNWyvc>_ z7C`WY1i2=>Mh>yJk97rzM?T0~@Stx&UPZ;3Zv{AE$8h`(^F+XPz|hT3Ec(w6CoKlM zGMLxP%Z8W9xP8&*rv(oYHU)i)LikX0_=Nf|y82|vuunObah>`%ZP#Low)+i0_WwoN z?m93w2yOSs6m7TH{GYVi^vZO|K1WCiLez(Fg6yyjAH{^p3-eHdrotamUxt1CQ=ERd zk-f=KGc90G@j&@r^D%^}Srt(=SWA@a1_*8BXy4>) z3)olyx09388nA6a3fZ`EBk=-1Q~}mE@f!t`xCN{pz;nSLJ6n_p`4q!Tk^u(X8|bt$ zXq3=asRV^vk0n8;;t8idjfc_!b_Va}+r1maVi4YPN4P9_wktGgq$`{aC$Uhy*`lqN5xPCdjzxk#VT1!-HiwJ zS$-?1E?)Hbs3;$}PP{=UAc5*N8Dii7-<@LxEZi@N=V;`fG_8zw^&Y0~kNgux9Dzp+ z4Us974SMuf&@t0&!@F@T6^s@J)-d@o(%o?;N;*9mzXy`id>0oN7#hOF%LjpJHz9=q zRrLs=L&!HY(59-32tO~00$AuFhUXO+Cm^8@)4gxEgMUnl?I{j>b)0Pi1~1?i?Sy>M zINoJnHSaUlwe#h}%99&ARQfc9q&>YI`|%Le5+Ci$4&RD47s$eKgIXb zj;OsKBs$Kwov0@D#&>=GHvF_uZR%X<++|ySdFt%|A<G3t|QpqXy=*{0@anYZ5lWDex}eU5yHkij3E|46NuqG5i2AmCDmU> zt{O2wKotmK&Up}l0*ZhB{7F8N_iYz&vNfT%iI}{o{xC?-eaAH^U<(+qLR*tNXzB z4}=+sUN^l`*raZ5Q?GkCv)q1s$w!D{r)lu}%q;8;NuL5$q$-j(;h$eijc9Fc)z;O$ zf)@gws|jj+zWb;&7QoGis1i`1dBKMF+Cr6$tn~EV#DE`!T0(b#Ve73scV|0Ugb-dl4FMQ^TZOF@E$rp@cj>BXn=~E=rZt%gF-=*o|Vr8bT)wAZhf5KjvjWE&dtfw&QXIvls8#hoNr;NY^eU}!{16w($YOgwqc z@}{dbcB^l6m*}3}B0_vM)*zOYVmwvb_!=T8;k)~fn3}G&JbU(}PV60|Xk6?B72zx@ zs`_!1n$j3=^|*rNBPk(qV!?t18rOmuGIg-!;+JgqrxI0*Bzalu78Zb z$N(Ys6o82qqv(W$TWrT6>eU{vUo^xm5*E)y_gS2q3;A3Pd5dBZk2dugLa%7c}PzWRGCTmBW?&g=D0thpos7vKakveDIp7uWrChju;H;nq z5+$hcgmjj~ux?1tZ!m0RYBq+s{;JQydov9BoHCKu;Y**ln2#z@*b6V`KB7~D#>Oy6 zYIt^;&AuLQOj_px6e>FF&z8Cigqh;9q3 zgxH}I>lq4xKfv|^la{Z`HY(1Q$kyVi3Q0_O2_w=yph{-WZf8tW`wN=yd&z8nyOl+1T#ZjgVnL7Qh|Pss)VvxVH%_F)aZ`L$2s>3 zCdbFJTvou4y8vJlhUjc*$8HdJG%$UU;)=2tD8>r}80m#GV!0=9*&|kb8QZQn?LyV_zIQImt&3hTY*-fIW7k~?aGw={a zjo)5_{w6d-;2WsdgczCSLgJqhD55Pd3&B7RllDR^HmQ4` z_i8Xmw+fXq@goNZyb0%|0|x$;_rR+51YrjZ0dFGuDd$wK)QoX){mDfk^*>5iWUt3x zydS!f1y(4|xKub=dld4JaP9K?`Yj@E5))7Sv{)S(uJhW&k|(BaTOk$Y@$bta@-ai% zZ3=sRQm~<{e1m(%<~Fsmvho!QeF)DKNFfLlg%=WwK@8k7I5J`e87@NTUZc$n_+$It zz*V0PJgB*)#RMe>Ya;;8J+ILUzJ{ZoG?KvrT;Q8r1KPt1N|LXHcNNcdY-BVoZ@{IN zfbjB1Q`0+;Cf@V$^QYO~-Jefl#b9rQ?YIJzE?$zK(J`zoau?zk$Kr~LKXK|U7ZCW= z(@2X?)&PYS^p5)l=?I`iNLS{-$IV0<4G8WPI3xis=nbdq$}Ky0)*S!WL04D^6YZJ@ zQkFG%%sA){gzUZxX|ZGhZ(ZUU?m+~P8;Y9BDKKGzn@=%kAOWm;g|2?>Re%~iKLuS= za1?mgPD|jG5^sqQkGc&)@kWgGMMX=@d9>I&=V9T2v_n8%@LQ5Zm0SOsz=g^R=lCre zu+4BCm;UB5cmP*HZcX@KWmV@4NmNyi>BJuxYbq<@FWU8@3f217zrk^oUrU^1r%&6- zVl4gNDBc??bT5{hKxPP~!JDQF1Q&!QaduH7Ud@b|Gcz)?YDS%}Pl~iWq0<5trM#sh zmML)ux;DUZiXdix`<7K*6#w$257<;-qB&*$3PptGWgI%g4q(tr1;Z5@^o%Ph-<+No z7C9gOeVApd53(=$Gu&%1`{b8dnMrB^)Pn^Jm7`1ghlcx$YTEeg28~%}Uv9fGQPgT~ z3*p&Hkkj_|>W2xuPZ`+GYqo4DHgDtCMg0H~Th(^3Istna-`K}ZW%cDZ9)&Xv9R6=$ zGaLgmrn)Ev98X*e<$!^V#g0-l%iR&k_*Ow3ZJ{v(1fh*$Vuhe)T9myWifg%ZaIep9h3(w8c-%UuXCY zLAmg<;O9U~P!^hlO#>pmSbdgjhS~rNG9o;_t6^^RdAK>fk;wui>nE69e#pui)q) zc*L^Jhk0dIDkVBql)9%EA><3ZEnZj*!+L|nv|_)}zep=x+jA!KGnVU2)TV^HK60eJ z?kau?NeAw6y?9X)Ta|t8+__Uy_EzM8S&V@iy;fR!IpUH$;^MZVAWOK|eo(t@rlOIN z02~}(yKw_Bb<*u>Jl>1$TR&eKHv*t^WdDAztt%|_2Gt)g4nqP0tD`E%YS(s_{f>%+ zd>6tNi2MMkdjKu(xA3QtpE(m$-D*jd82X&z%9-Ha23U{c&SJM}gc|TG&G!VfXwblj z#&7U}?21IaR2w@lzy&ZG#u^h8J+RR&d^q;`^IQZ2tPU!I|K9UUJk>yePr$1ed)y{0 zR&UZ9aY9%%5j~ z6c0Y>I}|A>eo6WVD&+EtiW!Oy3=S(6eI>QFb2n=-H&t`V8~1h@796tub}_qlXYsZ$ zEYU53IQI?C=YRd1V#=YDoQ0yslti0JPUU zQ|=e&ls7+|gp5h~#EI|%9_Orq6lAPG)Dr<`bVz7&U@eiFhznc107hI9k_GUnAh9+v zadF;g3QJ@*dHtk;YYC_IX6WwGk$pa5Z06Dr2Zgo3YKg1^v_memS=e6ZJJIM7WElpy z+L-+UqTDVe#YO=eEt@wN(Y|aHe+lt|L}T*~GUplXdSNUm8hqx-7z3{moh2zFf(oi>CgWmg;4sNkLZbkEXwh97 ztYDKkqy<=ZZa3=S(`5h9(!%00c@(xrqqX`P6P)Gp4FQ^4vY4}NM*_7b^71w$x?66j z1l<0k-Yq_)re6|VIik6OrXw4V-|k*N3{2KSYz$?aEkmiB8=<5R9`J#Zx)$6`$N`!v zh{G(jC@Xi`c#Qu7>lg_}XqEQ|wN`t_W2IRZ+B2N^K2`_S49+!jN@u$C&kHC8^Yv+T zbS}7!0DQ<}fZPiQC=HPgA>^>38=OB~B| zT86R$1^@EN)#PmlLhHX85qRYwQrz1a!+=$IUj><(`}V=t@I=TPUG8Pe@@3`Z9D~Xa zxoK%FdI+2E5qV@%-VY9bg?1pR4a|81sNnZz5o@UO2-W&=P5Cje00Ei9wqrON&%z;^ zI7StNY1{DDQFpC-)^M{QHBv_ckPs@u)ZgfWC45zKP$m=P2kLhOY3*P8?FQCaX`=0> zzdNkvPtk9Hs18RfOdu?|2Hf)kh%8WZsP5hS8hlJkBQv* zibCAS#>X33-?~R-k~Pus-rgU%1xTJc@kYZtYX0U9*N(H0A=i&%kG+9UtM19??5&V; z_WQ~)`7#*8Rxbvbx>S==L0g1_h|2=uX@lU;{-OYYOV1L4%7I-^9=tD_t|-vp0Mi z6-7{MoZ6ig`GNn{Sw6Ab75-uC*hfJq=|`OK;o$ft7R{=KJ5Rn})RTlc*Uzr53_T`f zE7w3M&B2X2>6>#-#;9NaUrp(+lm8)vWlg;9y{BE z9|3i?-9`(1wU_iq(txNxediN0e5hD1ubKpd!!H$uK7*Z7m z2AMtfmd4|>4b5i#uo>3E+k~nGy>vJdcw%B=6ycUKvVH>0-zx|P2EK`V1S4UdY;dFF4uT_Sk3j$8japAxlZA6=%wsx96w@ize#E&OaZ1iz;?utlqPs zMkuHHsUAHk)-tYjg<9kK@-7n`9X*1Isct4Ml6g!k41ozr znM`456W!MJX7n3^6o>wNo_;)?R%lBL!j-R9{zzPO9aAU!vsC<=dnhG6pgP8xPJx7I zE2-YW4v$o~TKjbKMB^@bMOJHkLJ`jzTbD((?(-)uVt{BXn}9d1m@4LQ3xk4|H#l!XOH^FuECQzLnrK z>i%et(Q3gq6lP>r1TcX}U`W?|@kjAu0Z!@|MN3B~CsP!0XgFdXg@mZK$6JrV015U3 z*#HUqu{m0SgA`(}LAhorXw*LfYUvaJ5iIANT=g2m>zlW3u@wZDa3>9$jU5uXTQ#;5 zy(%o#GhFQIp!CLYdbHz7pg1fYV9sQU=73D=P|gx2D8yDU<%4U0dF7!k;iKN{a_tQv z?e9DbhkC2y1)&w-g!MK>5wYoK&z?V`1t2{* zRM*RI!AQiggpB84xaNj*ZH@x5O8I}GI?(P@D25vh{cqeTfarwiSut?1aGCcd=p{2~ z-@1sF&m)r?&~{{W^}U3j@xA>=ehqIL?waA(WS?cfK?}NWe;JBZs)_7mO zHR@tC^ib)p#8fi%Qa9g?RdR%bajpUFCkOB;V!DCKf>>m~9W0Tt7?atxD;vhV@Emg( z?F%m`E+($6NRkI$qIKxdm&l`Y3;kDIL52z^XgD|#3dqGQ$7f<~&4q~>kzu}Tof%a_ z{nW9aQLr5&IFhD|(v2_%0E-0T3L7yTQ>qIRYC+OAJ4|N-q1Q1*w-St6gBa$HSlf@6 z!o&%acte)mP_)_BEn9F5zL0c<0|3c{FK9Ozrl$WiyEcsT?U4QirD0;TipIof3pau% zpQokW&&W(Qb@Aa-=tf6Ipd>u9dE|438y%?yp{U0^2}4x!gafuga7}t*aB8%A{bXu8 zej-`E)0xrzVF08ofI;Ws+Dr+g{YFH@Ko5u0!-QIDW9ysXZvA5BA7QN=jbnjABVzQ9 z*g+nNA@1Ewzg-G#GFkoI)ZY5ED+&nUCBh)QdE&;zB|l0*WYdE^KQ-Bf9u0rk3aVNj z@i9B-rcI@m=H%oAWAAXVvmXZ}clhCZGqjLrD`T{ab8;?2`1=Y6w}r-QoQnByvWSv$ z4%^W! zsYpEPG{UuBVc>p=#dJ+NHCm8jkS^S^^xsOf{{gna#;guQO5gkJ#fyk_CDhO_C0$9G z^z^uZ?^kz;s0=rHa-h&~yvq>xPbz*ZEYBuyW@w}yu3xv#)=^HroBMp|ZoB>6gYdZ` z)6Ifl8rozbu zoTA>-ez6Gg`0{zE+HCyO$B!le2SMC{uTU<*>3B`g{;Z6xe_Dr|%Y1xC-emcuh4QPj zmNpo|;*FT_zQIx|FVsaKAWCQ1H`vx-MXiBV?Q(Gr|`z_V*?ruXV5Fdl*;6nc<3KMnn^gu!rbH2*qvn? zVyj6eC|d6;(}-c*S!KRm7uMl4NR0CR${ek7uPC49D%|{LcKR2Ez0(RfQ%$N=7$pEnc_qF85T{!d0S)Zp19R<3u|bB1_|Q2;E&|zC4u(W$Oo}Qh2fIt&z04`Im&GcLcSRqKJ4%wP!%bt zE32xMkV*_&=VOLIbkQl^3=0dy1&pvW<+X2s>7IbDoG^)yJVE9a<(}+qR6PFF)Sbtm zy~K-BIe%UPSSVa=UcNl;-;5y#xNE$;rU2!ZCuIeXdRKv9wt*lhU`8PYiWiejb1SR1 z3WTBEg1JF_7JWV%3}Q|TUn-T9Qc}f$nWz3015aKCYLpyrjl2|+{V#my3>=XeOxDK zE5Hc!^YuOBUIbFteLay=*f1|JcKst#=Uzz2r${xl{}QGJHods|YB04*8^OS=I4{pj zYWsF^A(MgLv(vDEMOcV}d#GK^lK7G?dnsl~*M@cLz}Xg$D|)6ULWZ)yghsx?ULsg! z4H)`|B3hE*osHDwT_eP*w78&Puho6fy?=CeZW5uD^drNFU=`QxL+};2v!-28?Y=lU zQ_7;7ojN4}i)OGh;hbiIDb*=|iRDX{u8SZ3tTNOWbOuB@Rwrdm!-H;j=#StRBwr74$%W=C!DTl_4{S- zkmro^X)p8w=&qmKV_`OvdbFsH)j9s=Wk_8$@V2fC^yDyJ zIc)_YrPk-IqnQ|UWHITMTR`yYdR?%JB^PcwCM^6@VR>c3IS-o&|M1QI_G*Q){(rwt zMrGD)%v0|>8|ml}M^7R(#yf-H9;1}@ilhT72LJ5HzTAs__6)r4yR~LYV~}iv^Y7 zyBQt3_!k%8G~8g`VFr>$owFz->*Z+v3mO=+Bs~e`4!KvdSWM@hOU|g;2$cq>FR3V? zb09t#m@Jy-J&wZ?BUDRJb;tn-lmaxdRj|9MHaN&H(NBZ40%*iLpfYHgFB3&PYzVcF z7Ubp4LKjk#WXS`#GPQ1Sfwn^Uz)-9L1tr>F{aeW?rwa^3aKiwn<>!~@lvUIwSX-zN z%xu*8wQF9L=CzmInw+lV`q;nm@(&2%pCemvi~Kx_hd1^!?Eg&r z+YOKsdK41p5ZAV?YOs|BIv}S<0S%y0h{TF#uDs5&w@$%)ZR&vj^yz@@1j#`tXmnK4 zHmU>Nn_f_=Ry?{$#?6jkTBN?KkX+|4m(?f>84lP4=={4Ho(p^Z7QefHHbB3#^{I=& zDl!ZT??-t%`N?RmBJ4cJbU>i|#B32cVey>}!`l4a*kr_*3g)aPEkFq;zv_`1D z+V8!nplLjp>+|6J4Lv9(_R;mAIZbWZpaxTU4lJBNLaZhmW$e^Hlt`=|Z$ z!`b6`^GvUncL{I0%YK4W$Ui(jBO@WVoHwj=CcZOG{$_Bwt=!a#ljTVKcIH-Z`_;?u z6hk&|s`qH_8qaa>U7gBkKmLB{a`V{eqp!vc=FWQ8^?vcJ3HoLw-YdoB)!5LHJeq-ftpH0Ts z1!kQI%QDwZYC6#npE|_qiJ-HR_E=2xwi)ohGxV&w>cctGc?#nAac}P!1-toTR^QJ4 zvT@RAD1Q z-_^KY@UOG-IiGPr{9NF?IR|)xXuVtt?_9?(`KQ^iar5)_UAQD&Om*;752~$eZL`kG zetzIUpV&r^{IYO9$9=F_7%6pc`4T1U5Vn);>pM@A=hdxktby*7)MO{ApQVo$3+S31 zhFRsVi*w?orA?F$Ei&$b7XA6b0Pvb!I>k~7L z(_HzvG1crS)<_y%`ORS_b8Ia5sot87YN|xbo6_UYv?vsMYWf;4%qtbRDD_)M#mvlD z&J)+zdoEf9UJV}Zys{Y9JH87R1l#tmlFec)iExkNJyj&TXPGW^=>C!IiOdT}9#TB=!IfE8ntcLNJ4DoN*zD>n}!C8BDZsjiMN@@^}!u5Pt=`3XX~=6mdA!ZFs7H)X>ei%Y=1EG^>TQdeiA z(A4+maJY}o6#W$J>7t+z)AP-{Jb}r_L*2wlQ(=rGY@C)y% ziY=6Pmfmk_dj5wuPgzyHf8+Y~D_*#IeGKASae1@*7{`Ni-@J5>tt9K6euI~-%Y$-m zMOMF%7xY$%@meh36=Bo0*+AmrA%|-!Gxj}Y_`R0)Nlund+@2Edpv+;|GsB#1)we~0 z8lReT9kg_2P-1El=Vc+zdLW2Kq0A|M;GLtuVEd>Z^!TmqJh8XT^aASbt{bcJLuSsU zP)_w`afC^l9Y1Zg;7db>rs|l>N*=Mjp9AL=P2*eGof+d}8ds5~)H@xY%`cB7~pgWh+7^O2;QEmNs`t)5%!P?wf*mVbt$ zkTW)+ceXzB{BPy5mdfTZe`s3N}m~uwJrOzr#$Z19k4g z!r~HA`MJ-e>$lD_P_KUD| znvkptn`y6sY5wny!PnOU&N(!Nh0${!4O3|nnP#G?8` z>FkfCTslXOQWSMhMSeR!(<3%={nO!Li|)r;L@wVCSnxDC**rFWE=BgImT}j_xcKp<+>+o^{UsB)dG)KoVaknsI~RPkrmcwVPh0FbYR(h?bh4VCx57P zgs7o7WeNAZ-fig7&8@VXOYEPWGkQgNJS^#5Z~1{%bV}q4t;n)mXI~-J*qj>AK`S`^ zYhb}1yWY@}#Fc7lvh*>txHHCx&`5uxrY&o>@PVa8nfvfr&MxnH%15papRlsBIJ0CP zD_GKYtk%|lkGTsw@+@7edIu}iDJuS&(-c3)kJlc2R$9ZRQEcn7YrI<~EcQNB_zpHn z4bMyZQXw8@a3e$cZeHKv$b+p?U6Cbqfs+5t|TQ#Bt*2tZVD^C z(qdBMFW49<_M!WMh)CHpy`0lIv*J2`%$%dx^UHtW`^oLql@!WSfu9vG1SierOxzm>}X=WoO zAF4inX6%B)uWteR&z_fK3-+wsNl_OROd5I)l)t1XNLj*ZJ6{%S6)WQ`+#j><#*4We z3~BjBbNR(fY5L_S7SUg@AiD#?vTceVVeA~+b6 z4{{KzED|z>z=jNo@7*c=UG|O5O8eL;%wg-o>MW%$zkfTeZ;2Pw_%2MTocmmFr{E$F z=h?FvoAI`ic>twovNmqo;z|?Q0wBT__?S^I|Xhz-VB$4lTZ5lZD0K!$O-o2 zIj{Y@XbzsZIX=vw9A1 zNcrEXPTxPQ?b)C?-7{lhIDP#uGUIz|1<5n}2(1w!@RVvE0P)%zmqPZ9e>QI2DrLAb z^As*MlOx>XSkR;ef`-9I*faB6<`?60ZE0$wE?(V+{z65a@1*9fI*lZ_qW+e^f-)YnahaL$i-ozAC zZ=5~9@xM?_sg|PugejL$v<@Gh1Kvx+ii&Rwo12?=xAs26Pgn-Vv^T<}FlJc7&Hd#Z ztKF}0QUB~Yb6ODI<^v!GxF}%N8yNVCW&)MfjneSMwsE>v2)YwNi05sw zjJ6zK#yw4iFWrpC{x`~Z-?zm4zsP8hladbohh~J2`fKcCBVcqwSp=Yn0B?5nds~_- zFp_{^W&i%`CA?h7PzMkbk(|67M5KKO4_2i=(wcy+o*%R-g!F~!_U@uCe=#O#(PZ+8 zJc1mI5OE6z0Gboc#9zc55-`*{$aP7|O;nUHh2X!urbOmK@sOBP!29~m8<>u+gPA4f zt5EmmgJ$Okp=nIyV7_(CC_uWaSU==NAcM90U@$BfLF8Ty7kE(8`5-1n7~`FIv4mD? zQ0PkqstbTGG?KXhxNRV`fhF_WO`Ckx;{m?G><+@`oA8;0riLV3f>}nMs>HZh7U2x0 zOkDS!YD{moFYo#e2V64wqSKv{N}&=s@^7tefHPd5-(d!@+bnuoiZUSjMiNM9_9 zU_tl_Fk*lUe#qAdRln2`;&#t3bLmCLfUb^C4XAxzF~hhIE(%OCqV_Ufx^sDG`y)-} zdtYw*{@tb%yUi5(E1;kfpbR0Fj9<#)&fd2I1VKI1iI{NHwKLyAj+*l@FnI_JfUjVx zr+jX{$E#~r)1LQt zH)I4Y5R`v}ASyUU4)*mO_m@bv_#z?=8YzHPL>8Qd(|ANdbQ#kx9x)IS1+M;_c)Y9gC=f&nNNhG-aRHLU4$r@;%}W>>c=!Jl&j_?4KsvP=bI6$Z z_+Fhc7!It(NO2y<9LP4&vQ6u@&4+a?8wl0&xP@&34;maf;wj;S=m?4;@Oscps@t|L z2!{X)f;ITFHpzDxAc%_pG7>*WoG`Q^EEFtr!hgYpG-(tJJE^K_dj@V1A?vsOb3j#s zX`~M(v_@-AD+O4|9Om+Q<{5-6w z;i_q!w$r&L5j~I*mJ5EdOlCC;JC^6B&6Fk(3 z41TJb=I7z5>t>&|_V_)rlRz1dols2(0kyN1x)8K5se42@|4%HmTL+n0GZp{%W8O$Q zP+$%CLli_vQlx)iyE;z?Ys!{>lE{i8w=0Y=toY z#lns#DobI&(yIBIzkeP$eW_wiKni$hU${6j-<;4FtN*7QloQ54_(v+U}=*!Tp;~gAT+rX=$x<%Re_2*x@A`2y}z!6A>(i zz=1?jAjc-Z;ll{g^=PXg>l#MT&msu}O3-ao2on@?>tbLENcs9g74(7NweP{~0&I={ zhpjJxt8rc1U&atBq`{ENkkFuv(MltU(jb%;g-S?7q*~FaVHb)_jha3~H@Tn)eqlJCO#2n;X%z^!iHE9&^E*R5fqPF(hw|Ki_o z2!06bhQ=LSo_O^}#S;lvuDAD>kQn|fg4Al$!o zCj4L$KL(x)#TCjhum=3;+R0asJJqgPAX{zH>So`9p-D2Jfr$-PLO+;U%O6RHl9-u{y$0I}d!HNnD@mt!qie zL(Db+%SYmvc~8B?T%RdRAzKo(z9cn?7vwhBcg%z3`1i_q=Mjq!qd1id!9%wLi414a zOx)qt{Zg4{zP!yS0>p(-zqh&EwmZ66WI5ZVMrf$r5~feux`olw?9|fiB(6~c1YZP4K@JLXHS9(qp@et&CYSh2jvrQ zNHGgX2~qOEVn@CO*EUcNGIqHCMOE!`LLre7Ta@y75z?LPxaTaeodS{czkEhk=*pyVJN9^U zG=aZHM%tCP9AquIHaVM}0j2(!qVp(CbpTF=QaQ?Q>XyZ*UwWcwuEH zkS_+bobYKyuXfgRDLG%YddIPU4xTZ1S!E6d@MlQZ@!X`#JOYaYmz|{ih-Pk?7Qog2 z1@bCsd{hBumZ>+)^OGm10qRE2&P#;EgE9+Ya2k?**!@UyMr>?sr)>4>=U?;cVUYeB zASRSRl@Ip!C5APgp+Pf@ziq~m=~%G7np1W=HkDqV1UM8#PXrPt?Ha7AyQ9pR`;GIh z<{O&9N%NlAL}whkc6b71nzq%MxPSU>SVQC>vV&l`!IJNE!1>wB=CR@T< z;aX)n5Z(++(W6I?&NE%shmGPqn1P^^9U9(_;(><(1>Zx~dWfSy>HNc39tUwJXHn7f z`n<<$gSAokys54ZVy(MqS)KQ-?rR@ZK0YM(fGCII{qKzsn`S&XJ^CLo#4Wy4iQTY} zW9`sYxrzuJ{zTKAwk|x)U2}4BiIJ)?@Q+;Pd(M~&GpFLEjcJw+KLqxw(DI5;>+`2P;p&PV$sFHhol zy86O;(y`0NdC|68c_0LoW~2&ol6Z^(lx*&R{{D9N$%Xe)N^JZkfZl40v0{M@+% z>T}RuonS&w)->ic&(LgDr21OZ#eY9R7rB&xHTr}&lnQ!E#-4$Ck>~6%_kp@Q10pw+ zS+ZbsS8}P>ESHN_)f5gio!EGI&9xiV3vLA>?r3GIS7BcD0#@y987WnhC#S9ZqBRm# zt@oNp8?I6`)b14we&s!Vt@)Z=>YFzzl(D=V`X82jXt-v1(>?TOheBA{hpV3kA9ZZ| z+HkEud$U8-tbe!w=Dmq|L-32cLg0TChUzP|83~$)bKghWas(vl_gO*|!Z!xTh{$Y} zAt6B8VfC?{lVO#PsUs{)@;=qTu;jBXn@|{t#MmoPD61&{`L{h?{A!5=5Z? zBX0`gM|QZqS)Q}~qT_ojNZ2RSyF0g%uwEGHH4l(H05 z-bKDwHvh`!_U4$J+1YfNBWP@ogKvBYu6IHQA@Rf11(KHtI1*7@!Jq>dUV=)cNjk%? zWZgjvNz@21>M8=KLLQ8c1J-CI1mKyp;ZUiCwe{?H!_&XkeIeHc1k_y!%#O?YkmqS2 z*qc}gwj%BxSSmMskBGAn{3{x*X4HCegpY!4y6rZ<{$bFi3AA0Z1FHU%p%j6#wCSf&p(mt*>(cIwn>D0E4t7Q7X@8#Fh( zOSGcN%drlw7qo5yYmG!k!1)gjf?8lpKG_7J}ng>u5?$t47Yw_r&eKmmmPVUJ-pcP2|y zT^&=3p#37xQ@RXeoNs!gAKinaeI?4h!C$|^uTSHt#OKLqA%iIr>?pAM1U+I{J=m5m zo{&RFz)RxEm!UhaUAO>EP*on`ix@NhTwRcIE5NBQM$jJyt=8ay*>`+e<~EWz>jMa5 zU1e*8Y?=^?aVsoLtmMST1d~h|XU;^&6WqD3F^Td_*4P8_u;YKCx+Azxw*K?~e*dMPw)@fafB%fzfmIb(`PcmY8?Ei9Aai;(h$Phf z%G(=eh~TvjWE{6V@~>BQdlPs2S&`a*-v+;HxW_NgDir~uIAc6n-;k?`iGRa-ZFw_i zowL;TDh1)Wp4rD*-!txk6b@do;S15ZJLR}?(l=n0dc5+g?WSQ6Bj?@haiO|pVf}mM`6T+Df=$?!3xD<*EfBX7KZ7y%wO8ks!*xG zL18|mtR8T-al)qi9o1pQoTgL@p0*3wQfwC3chT37Z8BV19flJBW-?tPQa_#_6Clw0 z2wxu>7Sv0TRZTA)_`p2EID$yu8UYZZruiB^$zb&>DV}Q(Mo{0cv9Ti1cRw6tl&b5n zCNfcRxFPrwMnt0ITOwSAq(YV+?1^p#_mdcA;Or{(xG5s$Q79(Q*#^2V20A?Wu7-&R zi8yu&VxETl!`eqK+S{w+F(9g=TF>8+Q!`hpX$vC|K}fP;5+tcH8Y3ezaqR2;S4 zmQx=V-=h%0*0>SXj$Vq{PRN~fgV>Z-RzM)#gBh@Gl4?`CzB+{_ zhNb|*B~|Q}LyX5ZSWcMSS3hDtJ+eyeZD==oT z28D3CE7X9;&s5C3h>c{b3f4h1l{QGCN2eQKZ9|sP1J2}-u8gS;w>b(MM>^aml73;tp@)-8 zy^D_Fr1rjj(GYiaBYG-2CZ-bdQNG$T3S>A%m_So!Qf!g(xo~ekjKKj7J>j| z8L%^PhCtx77@4(pSE0#%bh;*?AgSc{3I+3?pTfik&z^)~$!-dwShZHy(UNR&GW^PsoLW{8n%ZMh$iz&n1fF5wL6?=>=Z;wXHL(G1V;Epud0#Qc&DUdR z`!2?mU>JF+`j{(}l$6}WzwAWOX!;JrkM71?rNbsB28gn>f~e>SeLHpXml|%DIFQXb ze0;GG%-RTs+`g@EEwq7HGkfH+`d=9*`hm9ACi0pyEe7Q+2jBziO7F07N28M>BFx0xh0A8#PyWz;Hx_ASx zJWMnC$DoB*gPLJcFdZ65qZG5pkcpnv7ED-gUe8D4u91LQUKD!iw9L#kAbW};o1D~- zcZU3Bg=EcElGceQF+Dv9!N%5TZUv3)ZsAkkBFkyVRejq>nAiFHc0%dD8ainItt1mW z!?r0DkZf^P)g9uvRB=I^MRjHdt=0)qS-yPv30qrTGUKL(uD_aa4!+!dy-Yq%q>VE=`br1vgN0!;y4YDXu5xx*2D_ z+y;E8D08GUF)Yy8gHtUU$&kT@zLz2e^9NoNOy4J1tc#3-Ph~htA~Eaj-mur0rBKK8 z+wH(EXk);c9}gxkHolnnR(Y2F!rZzKhM7jQBplggI9h!Nf6jeC&p*(x!3Z53oIyVx_ zKq$J1`%=A$Qo=0Ewu!08K=Yev8xG?%MsVzPxH6(e5eX)a)Ib4rjE15kCB6peF#%6V zpvJ_-)B=c#!j~AxQ&iv(iU%YrwcQ$pWSm*;|@eBe>k zjB}H4)S=^5fA7&NhMZOCu8HF$@DA_4{;K^Px^5+1fP@udxVLN9t{ZmqP$NvLsc${s zOhHwYN3^VcE<8{=!kQ)K<3~8DjNd zZY~ikJH+@dypeI!<0qFVjML6eNv&#uM{-;8sYg4m-i8C3ppcLOfLYYl+B!OIZAcfZ z$uUrdt6KjUPUT{#X)%CNOulUx{CF{Ws;4x+SO>J>7W5O?R~J$SM@M7O?k2r@*BN&I z{vKQzdrWzUjAMNCZ{Y;bRgGAYZMq*?i0^F>!>)w>?WHqW%%5vF;Y}WFJn18{LC-EO zB^?QcfriqnC@X`tQMtuu*WzXw1!9!QF7p$14k0*{~xM%r4$A|UhFiIvKz&iCZ!X%U-8UmIY$_oBT??+ zbd}-JLfkYAg@g{{{jr^)T<|BWqn&}_(o_{pCDql3QKQ|$XNxz~#?CHy@TZW69!sE# z`~Ep9ul~P$LI%1(;H$o3iiFP8;aU^h`x&7zL^~P?#YRn%$vw>MbRkv|y%6zuG3wB0 ze6;S5VY)Jf9I$O(Iheb*$=*PXwt$jzvEwhztdM~NR+Ha{0vzkqOidr{&qWt>H`ida ztz_W?h!t@mpd_sHw*2sz33vDfl+q6$+TK=M97I9k5Icgeg(b139$n z*8TGB0drFF1l3?<;qAn}#4@fbQ^cHZP&L$-UR+uV3#1jKa)J!Sx-Y%lm@3YCkW-}^ z=ZuLkxxr|y4gfC(&;KEIh1nGBH}AtICUB5VcSZS?aV9o}ec!^l(3qae=Hl4HWI}8H z{P`#ao(5+s>-1609gh4O5zAxxF2jFyuSabZ!W)$z;l$7+nq!JXvO%zCo-IG;i@%A)Se+>75ubN#e{kXs!5SBwQ&*~h zaz_E_Lv%(}iZa_NR{r$AsSC5Ga3oSb@kQ0B#Zmt)~;z zny`b|Jod6T7!c!?bcAZcUWXD&v);dZclp(Dk?@!FPBqw{!Jh&PsN)UsnE#E#1`%g*4GL|8)Kk7l~SF2bP44=>hm`Pq`k&YhaB&jn8!cEgiQKU zc{`|16)zoZi7yluYA2otcwhUWef8B23k^-S(|aI(L4FiIJMXZu!yL^W6A|J}B03Zm zIH?hyH%;nPLwQZgM4^}gds>LH3*7X~Q3|7fJ8x(A2EViE$C-`1n&Gl`4B1{(n;`pr~X&!xoZ{fii_3#&MW|V)SnF==Mh~7z`qK z;W|7jT5!N`W>gwI@;L{v$+q#;O*nLQS0`b^fe}an3RdwuCC6-S%Ulc{Xe6{0T?bG|8(g{Zg9V-?sWye?m6n@t#RE@diT5dDv{la{M7@LsQ zrL#I8y$7iv{EO7whqnmkp`pQArG!#%kU$qonV6Vf&q{T;iYT*(4ag~oOG0ry`TlMM zP@F?K_VLF!-P9%ktv5ir!@ww^H_K*@YM)mMm!SV(#Sppofutjw&V0)eMD1GVoxdfI zbkIn%C_vo+?#x_dB+a1-i~68k!z1H~`Dg@ef(Qmod|1%&C5`F{*uH3aPKVww*(}!w zFS`P@0fv;HVU!7p?OJV)gx=V56jzuE{3@*3q&&x;%A9Qd&$M~qu66hLHWJJ^IZ$*S z4LkB76glPhZ#epqhytOQJh`|R@ej5@PJ#_n6WzJcD1f3a=bC1;8d)w0WmMHs1tV(e z1!s;>PgBBcAbumCmNC26oS+@G*|f=N<0kWZE$Dp-@JSTp&}yU6fSnwnNMPfpcs~rQ zxg^yjQ|j5ZL`>&!*v(yVzqxp^>X4W3n`(|^TT@zq+pH1Uqu$HJFySJ=Ecg;+wz+!s zOqjCX-PQA)9>u+M{KI6S*JrL+&amuHy8q!G?Bxw((Ad3(e@w>d8dYpfSR}|zHrj8- zk;&ff|KP(19iWcD6h#b*=8yzjIAJ<+4KWx+wzrSVNpo!5BwHDc;Bgd>KMY5rBO>bq z^c>;sh>1MlO7)a;m@*(wI3Eb34{lrpw#QQ@>A6-IQ6LE*azIB}422pPhnj$%v0o9U z5%!!`>=}+_ru^MvC+%>}@%i8q8vwVBNGu?DX}e+=hc%;9%Sf<*iz;On#e;xjs49Y30=il1jUf8I5n#$B|ufl0M2j$pXY_#W z9JNq+%)TZ4GlM@*Y(q_oV>Qk=CG_s{KjmPE65W6mAkNto!xe6!Z^AvV@|tKmuk*O6 z)cgZCAT4N&QMjMvjS-hp0%l;wLy`_Q<-q>t>7mrA7}Y4CHu5B}sLg9sTSd5(~t&-SX7eEOt`^#uVmKMHAR_N|M5BGeXZ z5|(}zII2OdXzMTJ?V{q@1bd7Sq_$ogUGLhlEWKBfFJ-&n7nku$#+R z$ngeV03j2z0fm@y(Bfj$GHu13l3&-L_bbYjTH^laUdGXq;n#j5e{=Kr&nhguQ()S2dLI91k8iA2!;VB z-(W(Tq|aw|r{v|B+#+}aI0Y&g10o*fCTu#Ug7_41@D;4*k*VDbu7PO?*h&vJpY=h6 z(99>5RC@z>QDmb8oKR!M7#U!NX6|WMeSpL~>zI{8|@%ZE5uL-ZR-!0Ln4!RoK z=A5*ni(r~9Ok#+!<%O3Nq4kGc=P%ptopWV%j5AC;$(!M@EIJj}9H&iZRV*J4YdiD`0K0fV ze)--a$K|ysot(LYp=V@f_VV&s_~s6Axcu8y%-(am^JOp==)|lm7PHzw+`3#2v&SB0 z`2BKzSZM}08NreEKaz1;<7XZ#xptQixE}=0yoH~=)K?!&j5JGXc%BDwg_))6EcC<_ zr@Dgrv=-L$FgZ#RD8g_1wzjpYcY*~iG957#e&?p%bqA8 z-K4uV>EXdV9IH=Ye0#H7Ow|uO6bVzJ@I@qg1|a#r{id+H}lQ z$p9DAW<0m**xi)jik)-?omMsWhNNGhEqLJ}Q-l39nTZ0hs@=L$TZaS3BuVV_h+Hzv zJa?4LHDN|^pOGdjjQD~$?wZxBZv(|aN$!gf607fTJw#idmybuD!Zt43>uRh<>heuz zJ_iECBx64WSH#I@DmxB)W2jdQ3kPu2)WDK@m9irV12>W~mTp?i!Ax)3rnB`y_^o){ z3{0jpkmnHDE=%%wPM3#Y*9`gHQwd-uQ~~18T1CYmGMd6Wz(r!O`VSH6zN_t_A(Os7 zK2{A`c5R)VyRp%d4n=zX`jrxuTfKGn0m$_~!tELi5OpTSf;`jrK7D-Hz~BIn-Qa|- zGr#3k_xUaBO{mJO1p=zL3T*w)mt_(&n%Ci24nT4g3V>SSAZ2xr0v+MGOVX6eg;nDn!q@&SakF@({YP;4WZ=uaxm1dj~9*Yp)zta67QL)`j!!)oG5U(1!mHWcz%7 z@qfTQ^1%~U%GPN3{s%&XdzHY#g+o&nnGWJ7=s(0f*FGDySRUe>u==U9f-v(qJIeg{ zXp-q?=KU#h-1~l?9kN4F*`Sx*gU8`F91=0&Via^us|WP%HzgR1>7Z&jVPmrw`&Tpu zb$5V^Kc)&)fTdgl_g9E6M!NCt`z~C*97B9!&ddLlH}+E6x)pG^@G{hBL1!1QKcs?@ z?vd|6$78VP`^!b^@4>FIin6`QASnP1swmC`j4KERil6xlta9&8?$2HGXV3oL0&{As zS3KMPOwNb9f3~-4U|weW88v$exMA3$-+Rpd(zJ;Ct80xReG{I3jQ=%(fEyk;QtENf znJ?8;<~L{o7rw?cdy+5LjwDJ|Fet)t$8Hkq68AQz3Eu&HD+nIq@X5&c#TKOq^m~8I z&z&E1Lw9_y+2bGIcc(agG~W`Z32#Zw&ytxcVC@)U=XFF679Sq2lFo+nzAkF0Nc2>9 z@V;mys?b1X4Xa*A?R?JHG`>2<#fek1=|uvyd-H}E?qG&)#IQM1GJ#<{2@9FX|2&#> zE)0-J^IL)qF#`gf$LsHXW*VO)kTovi2@4CgOWoBGVEEoqjVSlA?48`J9(4P*>df`` z=f*@1rF4gEPq~ zOuaJ`PCeIBJBMJb5KxA^$a^82liD0r?*TsAM0=Gl+yrn`LuPRtjw+6=2=ST&w5cH$ z77mYk!I@MuG+add{ZNhyWBnFF&oHmF3DqWPr10+(w|qhsTakUy57bz5LynPXg2R|`uTx&G7~SvheLh1yM`B-OqI#q2Pk!jM>P>`y~Ui`4p16Sv7?gx zxXUViVA*Q!6cL^y5TUuJ68TQ8X!0O!u%*D?m&(&=RglbQ?zQazvz{43lrz;rT zXo{$gZYiLuddG8NN)2pn-#C-mg0A4mRZ?T1H`WAorkp{iLs3w*Vi_p}Fe)VMWH4IL ztZs~RG7u;YE91iYk50xcfNQ^lxJruNU%g)x{I{Y2t?mT#J|3Q53!@)ytB;NED>B5w zVZg|Qlu3juiSqRqxWlK5CwjM_dNR$vFar!u-m&@*7;e~Ed$gHYexZ4sMSeXL=<9K6u+`q<#jGkN%l(q_Zt;zjXNg8W`0XN|<;LSMs-BEEKN9{^< zaX`2-T11q%8>N>Uuy|Gwk*I=XZ_;Z2O@L5VmBZmg>a3Z^IEd z!ss#)^V!g}wC9>LXW?_s!XsflOBl%0vltTc4Wzr58pm!KPs?MTb4^}XfRSM2&`lqo zNKjcSCKA7(3C4-2XPbvTB>i~)%-lhg?xJ$XTn}CW0t#Y~9YajnX*Y#HW==JT`3lz7 zz;cMi-gAo=CVY}u3;r*6LSL{NG+rjE=(#!iPxYjVvGbr-%=ZAJF%-5mC=mQ!E~v*K z{sABc0g=nmA>LMi)G&V61PDEh@gw1RpwQXBE;JBEh5%JmM~sKS@2|jI)fxqsBK&b= zOzR$(0{ys$!NqX*;pOl(JOa+}7z@i-k}^EpW$|E zcPE(=18q}mnZz=v&U3z&mYOOKGbm=W8{gK$Xm;@fckpxBSy||aS{3^%AxfxvvgZyO z!2NB_%_`XCqA|wrsj&ofQVq6-znn!!6HGdJ8$AsE7cRJIxb=BJ$mGpX)+qotis zMQ(BtR^kkeP$M|vr<{x}&T#eeYs>1fUtP&o=7wyf sLIlI>es1C)7S}^Z@9@*K% zG}W0fujGw*PdEQzsmz+=PcvuUTRXukUPyInzo%^a9Z+OoA!lmBD0+#_} z{1w}_-QOFAR4q`+mk~->xPH7G3R5B(y^!6o;wWiAQ91ZKW>{5B#jJr;079G5EkKqr z0|(X2I*0!g^r-YHvZRY=r*Z^LxR|%PdwE4+ypGvJ^T+4lAz-@qs?qTm;d?^DxMIZ$ zl0~f_3UABe+D`v3spquToRZf15N7vGV=>h(59~+n54ST!s_4UCGB*}#MU1X;7iJX{7D_vKyqV2R-bn*WFwc!WZv*5qKV*L>cHmOTGpFp@k0nhY%Ae9-`v*bjw<)! z1!b0Z3GE{qdU#y=1MGYf9sQ=Mi5A3m@0E4eW1{3B_EUsU%^7dyBUEW3hFrsdofk2n z-F~UN84Tdp7_1Q2TOdaIQA;n51f;Rc$_Mhx2OcMdWgPG65XyAfTt`kti$05PW|gpg zQWLQLp1ro+%n#KQ**n^cWaZiKgC_k&xH=Quld~x(hj*b4y`&j;_<2m69}1_N__+BW zKh8s2u?7;1`Cy}w5bq;)vcSPmOt)h4zU9zyJe!oVHE#ypM)8cOsGBaeSPO?h^haRX zRbWbj&V6=0VIqZWv0W}ia@;T@wqqD49q|BzAp&t3M;2!A6QOCS?RBFlPUypj@14g+ zMl>*P!ns$Tc5<&pt`4d{lTCpD z01tGl3u}BF@xXh}J7ryHpFu$JR+`SOq8x&$L$3J%8kH`S*`>w_ig10cla zNrfd9e*`xHytg@c4K;v}@(>@{_cw@8rnj3HS*EfRC&yznY3z5yRAu){-V9xX_ggNx zEc{BZ?tCtLtbSa0!GcHy(_&?ea$3&PRJ~K1@N&&PJ-!&|PNN`wUa`d)_U3Wm+WkHw zNKeUHE+`l+KnKf=IEyO21sv*%s$|-gyx?>vE(?u@q$4j4NP-wzwbLNUO*aUgSgIq8=f@!=6n2J*< zu;_k+EjLTU26Tz?CwmqWR0S^p>~~&y+yP0nYid}PPsK{duni7?>oSFZdDF<7+Oh}F zMuvQkQXg)o`y}B+X<`68be0<#1dwSnOXuE)|wxB9|mX<~|+Tegpp^)2; z6f8ensjpJG_7Oj&C!uU2@oP(VbWT-4CxGh#Fryp?)nmKyUfGpfYN#jI(#X#D57Oic zjtJq);(V&PZ&v3nLHt5t?K}Yf1er@=uP%fiC8BRno}ciP*i1K=q@o-2w=}PDSs*G_ zEwrK3{0vL+1v}&`H>V9EP!%wyNR}vMOBPV)ie7LYofaApfPTLaa~Q-cx&xs*j%>#U zv31JXp9c=d(Yuy30Wwyv!3@J;O2IFQD)7k`x+$~n76c6pc!{~FU`9gO!EN1$6`PwA zlnGr;tC}AhpE4EKX!?v91bo1#CZKQ<2;3)jL`?w-bxfP)K6&z_8IvT4xH4>P-mPVQ z#R=&fdkTOS#J|4@5HM+OX8mLonb_tUi4WNxMrTFhSMb1g1zmM?oKvs09>e?X`5wJ% zPSC~2E2&opiSFF=>vUI41p)ipxzhOJy^CxUpH7z;lNpxG`!J`FPpy!Vh6Q!; zGjpQ-C^bg zjhLNc-xdcTiBBqwSXhePsNJPY8fc8+X=^wB>-hAUGb2$cP+W20Nc1p>#;`Q}zWO03 zku5+A7gq0eUZs81fzn_e{%RQ^P(r~fp`cJDwFK?NK>v>B=ZwIHLtO88It&{CDbB~- z5!ay^Q0^;}*M!+fnfc9#kU~J)7C_7~iop3oGNwqKXZcGv>Am}r@}7%g2DI_*JOOEV z(Fjm98e$0<-u)|M+t=9CzM=+B-KsI<{goBCdcjEffSklv5lche=)z%F952tj{3V8DGABcDy#PCikbHcL9h*e zw>e5OggwhZ_M_1hKa=D?zlLcq%-vT=kYPC3tfF@7*DIFIlV$w6jO&RrfW#r;alB|V z*Pw%z+VJY?oonf8>;pg3(levmN3Sd_u?m(C~&%t(0GGW3eN<1 z1$AKTB5JZ?ueNr1rY)^H(OB2QHzIN_tJ~qig<$dxl=|nJUr+IWxsXYGjKK{gd6s7dpr&C!n0o-! z9siCS#INIYwG><)&9dHVzW%$ON}vO|#PSjl5-Qp>{9iqTWhi_efFsCtXX{Zg-gigD z5 zChnG~c9N<-7PPdt&%p2<L1*t2Ap6XEN-hkPMwf%+%={C}V^cZV_FFz4|>Ayq&2crR_S z^GL^DIC4|a(mYafm=6XS<(Ct?196sP^B*by*(~(9U^G>G+jfqBsx2!!-eY-O+MIo+ zd(wJ0+dnlRzDL(W>LuAQWP%z^d!DBWZ=~MSY8sDDp^IowO!{Bl@h*rSFlzzvMFHa!PoNi8#RPB#Ly=7X(UXJS0)^>^Lz87$mxfAbI$l%sIEstSao>NTkop8l zK6|_yD(=5EB3|n<@pYbMWE8*}I_o;bKTA;YlPq4C^`mrMozqA;kC8jsvS2epx)E@X za1^E@lCyEJCPh;)jiG!M?r*vk9q36N$Uk(Z??62gli8>b~8B1gF z?2rMiIJ$`H1XHA?Y0z7QF?ltBqcL& z@{qiBl)3^UA|)k6Z1R6YGahh4K*?|(zSDlf@>9Vv);e?MX(zQGad;GXB@kK84zF+mE8I5De3MI85A8=W;(xmz z-caqai5RTu(qIe@qodkRdw&U7wzP~)45;e2KoTcZUC>50neibSJwf3tAZv|w+F~-d zKpiyFeLpZz1)XU5oS$DaQw}y_WYN|Z^J?&bzjkyaD*RiJ{RQ8>+wJer@@}@9*%$TU z$&9TiCMBVI#u5%k5ut|$TY$|CzK0ohYl}ng=L0>#`*M0sYIW6_H0x;1Voo7bYd1BI z8L&+0F3!)GVC<>Cg*6T9E>g(U0QsjCDBVW7<9M+V$nXJVKK=E@1}xg&W6<$7NhQ-j@Kxu)j;&tgEbZHte(?*$-*ElY4@7|ApJQ_ie#2?ld?}U+U{`162gYT`#Ji zj6Qp4o_UoCPVDhXlf}*{a~IwTO>v$REu@q;`^{LXLIg+h6h|(`nKySZX!mDG)$XOH z*iT|E7Ka{)%whq$M8OCMSG(5RU^Mvtrqk8o#2G3LT_yBXQP@xpz``Es@_H7|*uwXN zGu`3>qRM^b5qW!eLNBEV%3i|Z=c^HN3)V-56n=~WkYz~|Gw09;f@#XRoxY2WpkbP? zhspEqc*Ac0DE)Z!r>D|woAQL>zA&bVvlO^;I@dGAs*aDw=R6zzc&yWFB*4exOXdY^ z^D~FSx~xLa7!{C?6wprk{5EM{h zE7NBqUlZpI$wsYTAQ3rj>d(qk5aitpmN}y!LZA)uQv;RecNI3QUq-q4>tXi)fI{Td zqb^lQm`11wNu=*^G?YvAU(?KfG573?oY&MsZMnQ{@jLA#Zs>-#xnyp&pJg`ntG8M7 zAnjnc%do%=Kyzm{38$>sfPi9zH;LbHKPzGMAu4&^*X7i*s6l4_m zt~1FH((Vu4S^VvJY+B8~#9f7ZT{{&v^uBU+ypzPRxY+)fax|vb&E-JhD*z!?bTtaNd|LHQrV+jMXz=!>gu=$d-1Hfm;Ewo zqA|DARR6}oSl#>=W;>I1oapbL1%ubed$sxPn<-Cs?kqT%^7g&%?0E(Ln`{CdhNaHg zHh8~$xFtW=Shj+B^G&MJy7b-qH9K0&(_cC-8M(Ko9Po5?45=Ydq&m# zIC~+50%K6+Fc+UuhyL)^)}mPr8GF^Zr>D&fuZcJ4G}LpQ4?<)`W|OKkER%YjoGUzL zCF$!^T)##(?T{bfcJa(_dwXf;a*2%RYEiZf4SWsz(lV{qpA4_+#?U)F%eGP&Mv3Ju zWz>qbRIv>KOXZ6h|60FSCiopH-cGf8Q9o})v)Zk?-jnxk>C+X@Gu%GZF5YMZwRLo$ z_Nr%r+AD&m>hH~;-}zMDY3ru|;p->(_@sE}=+O)2?5{o6u3em9lplXEDKPhp$Z1Ut z>*y3L%jA?J9tIms-oC9Db+M`SmYtk&&$g@AuabvvZe;-9IzGPX>;axRu)?5iFlnK( znQz_}YOXUIDc&6ys$3vXqfOR474g}qFokl-^5YF0`dgK(`1jA?Q5zVSpXxGsYwwzi z4n-rFe$jUA+PZN0d{&|V_m}6zDdAGW6nexN@?;s379k+Zhy<(X^9R zSKDT7EDYeGRD1iWANkoRwe{NK8Bn!H+0v4}&Ck9z+T2*1L~(tJazSF7$ld+>^>(aj%u$NRc&#ya#j*XCRjooG0eE856?U$peq=5tGB z%0jLe6$MPAB!3Fy!+NyK{_s6!%%NB1xFd7ur=8uerQ0gcwEw+(kuM+BeVi%4;woJ{ zV19fGZ~P*SOrtB0X6nU%KP<=|0tI-yRB-o+ji0L%W^d5~0Eq4{scnaY%*>zJHV9wkvhvHcINasmrF{MG z2|c;6?a1D>Fiyp-^^!g*DKJS#4pcm43~X#V~g9cB5I zUqm(v2?Z%TPN!Ts60@WI&YEYFVaxx$&^N)xc-EA_L!ETV>6Ex2Z7FTRtg+a)alDHu zie?7I&ay+R=jcmuXlC($sd|^5SY0tcv28H=!nOx}-+Z;a=6~ zz(|UL#FX{xM<#boHqo`?qZh51%^Q1n=|5b6IkGnyyY-e|y|cQ(fu-rMGG~VNxkmL8 z|0Us-`kx)IUA%CV=Xm{=)5F}K9L|d91qENe7)3hqE?F|2GrGhl#kAszd%o$SnArSn z>vdWsd=F`-J&#x~<$CeulrdIa&0re9;~=935n& z1hpG>nz|Qp<;_F_%yRWCI`xVlDqb&(4QpwXWvCdL&NbRIyZ_wRnJLy)D^k_rKL_CjM?|S{OSEdjrj)I z*_St*dG)H9acW-gl1(?%^!U4Fk4j5#SQ}hy_G0clWKU}`T%L^#TnAIeN>qr4BK7g3 zMpu;9`M&DeuZMW&bZ7n*6y3(W6sUR~+pEfHIC<(!u6>%h z(p7u5evshLpiJdA$c zw&&p1bgPJ4oYBmtw>I^P#bAs%jLXU7ab1F?Xy(=>xsB;Y_Z3!!Hs*L>&51mTQV#vb z2v9isGLnN02D~S|w)XFzete9qqOp4wRHQw)-8F$%uY|8uS~BL)KmFHaYRHh9*V~^+ z2Y+cLpnb04^@uD(y>G0ss#xvTJ+Jpwu~BbIgRRmoty|vvwJ3MjN=1c*mt9;;8)UdI z$1gp(e0E3f)rH1CK6!ljwrm$`bjpI+{XgqH=s!Nl*R22dqP}^HUtpQk(C%GXs@H=c zBcd}E>8c_tGF&)Eqo*6LYF;iV?|D}%fZX9^`hds4{ScLt_sWjR=@k8`6*O+^6h7Wn zRp5H@?0;$o%^KC70e*d{^1;pn(^st;Sp7s_<-o7WC~404Rwg`EID`%3#?>TbV#LbC-- zD*mIW4Pe+@N4l6$yzk*s+sY@i(VIvoO_sKy&J66_7WM>62ym!m4@nN-)k!8hnlR|D zm%!yI{>-JhH_Et1k5}!z8xGSV637KIEGR0JU~Qu>r{&f4jhM23f3emYwVpm4n}*V< zjw7c5RQ=NEvA8gS)=eFa)w(37!uKB`za=6xgt{8OkmgFeKyV3HJTgPfvhP{|RmXG$ zrh@ktmuB?e^D@}`)8oakIdg?&8+4{w&w!8}8)zwL_zChmqS(V&Y;Xw3>!7p2WD*u@ zf`kt;xC}KX$*ad8JRfa2rWpYC1&x%6&`bhTe&}sGp$i6>lV1WxBmq*2HLEMdP#V;sX#lE#MG>O=5hmz~bKn?3lIw}7 zBXN_9N_K5NG>J*HN0AX#t+$-GZJa!cC;wHvd(IK|U zsgG97bs80FrOWsE@`-birWT2j0ST3duHsU^{&yQLMnOf8n^wFyYvQ+L%X~1T)m2q* zSg|Qpz3UWR5NGrq;xTv3YcUKZwiXalUbrC7rddt4+E8Q0b#=JkZD^1u{2Cfk!`dT& z5+<~*8Fgj^qk^PGTASyeeLnw?y;=I9>N{oT9w@e+Ca1^>Su&r?6X+6Ou9n-7_o z5C#i|Nk@1Q-Cpp!f!Bgm_DWs!ySSwGOy-&z>n1 zegW@Un1e=MS||ZJG%OI_fJP7qFa%O?@TT`lej>bbEW;I;#>00BlNN>D$pOId!@-Rw ze08C_K|#|!RIg(Q7h<-J0-rUCKm_JQu$9{2Jqv^W$f}`<*!9eMMj&^hSGJIgp~Cet zItKg&fSv4M(ndJb!2YVVVNv-04#>+GMz{@6^UF*rAK!$4Kw|p?=&=B(!ga8JoA2l< z%Rh8c?JHWuy$Pu`8};lGVqSwWxf`K0d(B@4Zp&|*UbnNa|LJ~&rw0=X@XCAdyMVli zLX`0x0)2`8G0bsG%E|&*%RplQeAWKE>KGt;BAi~LMVr8S4+oj~A_LB7s+c(4^O8iIK-`~S`5jqMxgLQOe)~~JiSlWs96%NNGQukKlW=KxfdYhcYFMO+408(wf12%J21>NXy3eCORk=T7US z;s&($_wUF0|1&7_4HfZCmI$dVTja7Jc1A<0Qd5H|v1 z#b+vd@h{nb&KpnBq{oZb9eny4x8XY+amot;oG$aUdD+NUe3UM7RVO@KrD zui<;cT9Js#Dz?`qAAQK`M}{OGbP+=i4B^Dc{zH<7iS^lqn$9XJ2Ckr7Sk)yj%gkuP z)~4=`ccG|&xqu<-JeexfM?0@A#w{klgsD9|0^zPcg%ZY1bNG4ufT=a-HDjLQw__It zZ%P$=tLv%E+hQyHh*BCJb;?a>vFWB$s^%)$yA5mKSxB-*CZ(&RCkTop~G+~!Hprov&bw2ruDqYc+RWg%Dr+MH{Q5)O8~wBO;NAl9vcud zA$w}FV3@Jlyi`c->$uUugON!|^ME{?=Dd9Q@`#H(3)%&Tffip`B3y%^h|n^V*v3)W zrhy);;l0vC~M)t-x6CR8(=~JE6wISGc0E|8O93_|143(g(jhk#wN=wr}y9AyKgbP5n8kqF? zfaJ7%M!qqXg6;Ku05a{4TUpK0(kfbB6S0>K>NE*c2Fjd@-5jJokG%`4sP!!l_V(^j zNV>S>)qaMqog_9k%Fc$~kYdO%y{vmrJ9;Ol#|rMF@B{*paTe@}GG5q&#@)6Y&MmKt zi)TTS8Cm7gf{ex2*j7mN89bljAGkG?(#(}}(JqfM}={t!kL!;(bOW7XPE{qJ7Tz7D*HCoCv~M%Hvl>j@5BW<`ghUc=3#D;0=khn5gI+xb0P3 z>O4Y^g@7F2?>L-V8idp%INyDn%FFq2Mici}Na2WX76%>_`4_IjW&aY4j8}kGf^jwp zj3aeO@Sv8z=nROWphFKw&gI#YbeM?gJW%}2a6%+s=iaYp|74O92={fX?3oAs%-aCn zz;BNFTOxvU`53m&?MJ2ZU?>=ltALMN0D`%CK_l&*U%+P7R@oQa41Y$*#467h14vXp z^y-Q{wssVb2M!+e#Mh2*@Aygw>g)}Ne^{-3he{9pn+qv@UB-j8Xjj@O)iuUT<-U>=A`+igEMXBK6lwDOlz$;u zjbmeDmH?4H)YaXCN_+95MekrBo^1~T=1i47+R2lvEOf3!?1*403AOav!yxaJrDYN9 zRn{la-CK;MgL`3ba|)|dXZ4*_z%Xejof6-jhmGkd@M?u@(@(&sKS6C(-J-K^-$|&$ z5cuXX0}O%?RE!8~7ZqAxvwF*xd*Ca|fkCwyHn=D=-a$fH5)G~HV`7&9AX)ytP9dns z1un3jtmrat#e;ARb9aGd4!R?k#rGGw7kSxoc>NiAT!%`s^o7JC1`^}rj|0Ct0}crJ zb|_S}&x-FqJH$vz+Si(w81L+4-$jdPHHP!3NuJX-EItbu=4+^{KZDoS`7Tu!L(j=? zpbX{&TkKlY&sFH`!}@m`0=v$ae4;|(ub{bt9el!1ezWDLrFWq@*57X%@OOfzl_YGF zfe!<}IR=IR*d9Fy8`izXX3T(BJ9G;gIyyQjl?8L)@=M$qAGcObM`^ zQUA4uIr7~)C2STiUYwM5K|za9+t0J#>b~QE75C5j(UqQUVPCdgHS`n;i1vL=@Tx2A z(_wu*7VuBDoF4u?IP@R_D$w@01%} z4S_W1Anb!p0R7=W%m@2ibI_lO6E{|Bh3!bsv_9J3ufx$yth*Jv|JpSr5N15UwsA=u zdS0fi22B|0m+;lZVLyl+5Z0hSGll>D0&RQpk6Ob<+l%OH!Pp^w%HXD)wz0{c*VEk% zci*Ymrt@JkNSInl&*1w>+<{RQg8-8Szk^N{_tG+Wu_CAg2s|A0QsF+NxOwxx0+Ss$ ze&SoV9x`-6U~;?d*|Qq>M9}(x3`IH&G(2`)@O~z?UPKyx{P=ZjF~f0mn8ZHAYXJ&< z+t9ESO-?>GB%>EML7Rtu5Va96)*qUvV?cZ(zXE=upvMgf2{{He(xJ?Ak2}Zh&xYx- zo%+f?eewjf2vXlvm;`{AVS%53Kt|QXAZ1wD+7@9=W!EtsIb1kDc>DO=0Dx-d zGI5d|cwh=T_K3OCCp`+a>ezeZoaEVmCxwcCi=W)l1`_7hE5BCYlHn`ygK`lK3Tb3a z%uJ%Y-M#yD=+6Jg-kV3&y!Y{=I~+x*geFNm%84c#2<=2NltPK7twOU5B}27CMU;?~ zjx?c(5~V@4qluzH8dQ{$CY9#huD$nteQ?fs&hI&E-TT+Q|J=2nwH`;BzQgI>+?-alIh?|Zn1z=+}~@=bAQUcSsgCvlF(3Z1;q0?w=aB8o zz~5d;rAD6Z#mTF;r0-+>NiHLUWwdwPs5 z`zyw2P~D)S?E}%ecJ!p|VUM&xx=V4K>_n*HckaJzoO(8p>zz-vi*cOr=@zN>LK&byG%pwQTyatOjT`52UPIZw z`_Wxa7#G~!BH}g-M@e8ev=g>=i%)u}pdA>fI>enTM|0x-@}A%7E+;F0bNl@Un_^MG z<7HiFculP=`d3!oPLXw&XjwtWss-kJTi|nrWE;RrL4A3PWvEDP9{<ub z`59&gIxsn+XkXXxTeBEhXtSb>c zQ04Z&_XJ!^LPi86WKHKDZZ*UiNB|*( zh1B_!W~8Q`gN{3oJ!(b?Vmvs>M^DYVnz=hNC93)Vu+F6 ztAYUwnkR_oW)k2KsYDR(75?Fpm+&O;+&La{3c{2uIx+F`guS0;gRBulp_kX2$crIM z1d3RbtxqfGqWqDBBs}QnhGUW z22)30e>VS|IfrqN4?(x)1PX@(zt9$4-POqbl?i^m593`@a`IlY%{EY!G{yK^%m|4D zryb;>FfdiH5|aa`&>AqF(F$2r)exLR$`m<%o0;t*m*K>R%Ju;i<1l~!8Bf#G55Y_D zVOG}d9sk~mUg?X(Q~wMNH=e3uoYbUkgae@A2buNj@1V8ZBA-&vL4&pMET+uc<_fkr zY;7@TIJ5)ERD!3c%=NANz*_u-%SNS5z_vRKNYBq4<5WNzD2U*CTfvA6Me>)U$XvL`16!A`{y z&a>{E_Es9~XGnPjhUy0~RbXS~yM8@1MGP+W6rN{BJFu`;`HBL+p7cG)Ss;)>ET{QZI=Bz4`bEx4a&0ekQ@3#c*1vt zAebPLb?0)R7W!%gg)Xsz5dmq82NcgBkz? z7r6WL=AOPC=$K%GY!+Du>DmQU=$G%`&jW;ZCbDkKtcuoFZ3fD@lB%jN+*&Yr@M$*(3+?22VKNkGTB!26JM zjyP`On+MweY$y9V5LFi_K$~PQgSZ9bIqK9)=rCBhUc$n{h<=|1T}Rhgwp)!>D@6OskDm@Mm7bILSzMpdG=N|8X{%|cd~*fVIm_cJ6jxv4X3`|wwIyC|mwz9S5Mc4s}yphj?sGB)4 zQGf)6rCS2>L7YPd@1a9jL^Op+N3aDKE_|rUxl6+5pg*m#FbUj^{8W@}i2Wsuk=T3GX;7Yb~nwgVE4^sY7PL zINOpxhI=hql7=J@Srsk{!S`Ybm;CB#VR+CY+GLBfvwUc|dY*L&di}HHNJA9W%jeuK zsI#WxA=^-1Qpd1vh;!t2wlPalWq3FPLJbVxR^byK!UdI;k1IImH9?amTM-c03xGWg zFt`&8^KipSZT22yBZ6@XL!DD+&nB4{!TLc}R8+JDAbFD`o&Y>T1$cP4o%fx!+bTq8 zbi9}~$kkL7L@>wU-XZXv|1sTteUeQeESKE^()lkrt4wcSZ z18*95z~BmNq4Fc(fWlWnk7mI!2jR9f}!l(gR#vOd6;WdJ^{ zF`1d7cnZW71^(Ijrz?(njKH?h3M}zw~V;#Q-Y|5MTLbe*rKk1c`k6`>tSK9oXgZT zJtY~U0J~I0ZHS#FsZC1709i}!qtg=u9KZw&d=wl~(jWlG-@QWj&17vMY_tQEA4#S8 zj4CP*%y`Qstruvl@sA$i5!QlUG_;sJVT!m#XrZ@Y>Hxf2QWFVtn&U4Y^f3AuZKxGr zKX#a!njXNVP_>ZA#<1DKPLel^mIH3Lf`G}AF{BHfX|3sFa-u(HK@OE~bMYS?SwStK4!O^%rNd|;N2 zNiwLd&d+6XytStgeQtfXi=ACQX3w+pA_Y?;rP8{3Mu+hBP%#ygv`h3`(&z?pIY_D* zG&nqX+LTM;KIJhelacho!n)AHh0VU2k=+ekYii`M4CihRb#?JkJp9@hiLHEx6Z+^^Q~ z14<(V)Vm!W9jgI1)>c+VJ+T1Go1c%*aE1Ef#ooE24USDiBrVJdJFJ)fgYOcW56P!GQ%k1Gtw!B~o#}nL&@l4r#BkagnIaH-rvY3CrjiUdc$pqqKVV zrc#939j4SndU?96O98S~b{S)mA3X|ypRoIJajT0(QSoe$T2>X-fm;vvzhmNM?^HCc zTl)pzEq0jX0X6hsgZvHqw$FE^)*SUFOlGI0hrRd#J>RwQLJ7MYEG$f$O^iINXeaj--bz2)FJY4*db6+2R zyEpL5$7=X&%#iEc6O-MqC%@@)JI{EuFICC&Xp7x8Habuw`;{qEi{`i<=U^EIw#+)$ z2r6#FEDD*2QTOl%6_tbB5gNCqhxvTe0Hbv9_WLUVBJL2tFN$C3>s|Wy7raiSzZ}BA z_SnSow9dg4zBUQE@_@f5S8~=!=&f6;aK_#W4V9HsQ!e!Ks(g^r*j&xbF0a6v8)2pB z?3`dQ1_u5iw01aUK0r~r`7J%VEjYPf(Ib*ReA=VRH*AboAAhXkT7Tk>f1)v4Mb3J_ z%vzoNq?CGyg7!N)CdHFVwa>xC6cYsjcX1Sat9l)T7rqU|`0vBRn=!+~oACazbFF8N%`KsF&Jlh-(zgJp#`N^wljorO z2mVAD^G#w-FeyZBK%^yQxq?D5(8V+6ahk;F09A#uh^w;%RWAOyV62$Fq)o()+K5H2F`Z8Fc?Z13OL-%z4g!gMt@LZ=bCk zOB9azCC8zvew4++;$)_~yL%wF@Y9QBu#N*&`8)<4%nsGDnUsMNMQ7koh{evQ=6R0} z@c1;Ah(w09{jm==3@@PuYU%8}RA+g1wzt<;0m@_veUi64k{X0^O|-TERfQ#~G_KL# zi!qCjBJa%q&js6_e){jVfQ4J9y3)D5K`fl7SCH}~1hJqugTd&mr+gmcRgRQm4!cc^ zw+c7V+mElMXvqX8tOxwDK$TmxFcQ7l=ZVSKl(ro-PcOkup`8BL9l@Uj&w5zkn-S+* z(l?U$+GftE`mWxkK7y%6J53?S5{4anR% zUm3d-jLtS?1!@*&OjJD5C_~GkjxoEKP7=9vLLy+;0bq^nA7LbB71TQAOfw_GwO7hh zk~aQ5P%+BWgYtRQ-`_y0B!h^bp(MyWJtV_n)iI8zbZO8j2MB`Om(@XRFIuq|mUA@a zE5ds7`6&1&#@jH={p;dSUHi-`&d0VNH?D^Tji1omx#uxPj5_dWMUU}Dx5Zt{JAzQI z+zpy;@)ukds3XBKHD zvPe;oVBa{;A}@L5IreaH@*^z>#Bs`yq3DGv@pxx)suqwWPK-_@1LolCxq)uD=h91; zP9;b4ymj<4TmseP60{n#kW1S3v=QSf@mAvniM5%_8B7kk znF9(WW5=X@+5`MTtO$1(FWx!pk8p7~!aaZwiZF`S&t2YG51ktdX}vVo<&=K{J69Gp zf4bU?g%-NGIORvOHed0!!R0aJq^pUs@oy~yEXv0jl8m%;11UBh{S~x{w7?X>)ZXd^ zkpO;L2J|1on~k*IawVRgZzFfYLJqht)cu80obS9Zn7*u%zn~*gO^-DcJ99f_^lx~O z(2J`JE?DqjbeWM!Losd1^-Grx>c*P$9y0FV{|yJ*#o*vQDgDfMjbps2(-XOFp7aIJ zxicmIeh;rCM@qS}Cu0H)Ckk`WInt{Z&|R*kfZGA2W$p)UZ9buk6Gp`NjF^OC$P`x} zMyUnl1c==bIji8LB;D)B*a&!M6Q(_4RzQqG+K=l>YvTq3TE4ny!)56EfKC&R6>dsQ zPh~o1CzeV?yo>Ul#SrBqr!^h|qW(fV1x_1@vjpL3F$}xPh{BxQEFj;cAwkU_z!;Hu zlZw6^1(rMmSe81hSS=wcdH~Xt`V1GT#Kgoiu3ari$rwO%!qVl-`Eaf>!Li7S^n=1R zA8w7!UK8@(B`#ss{oz%?Ox}0@iVa=%N}}0u-|2b)LLss{a8-z?Y^7-$pNx!5Oi4yY zo2`w_1(4zFC+n)sD#F9UNY1z887OM7Yr8+19o825Y_uzuKJ4aEQ>&>p!JH{nt(nI} z2UUo7sa-*_Pdqa3t3e!q3IbO_Vsn0gea{%wgeSB3nN3PHTjb++SfmN>eV8wZP%3$?Y6s z#$3r`O)h!hsol@TTRoV|7?7mFfx$Kh+FTNRh=%1cM0Po#>cCLpu4Mz_hVKBc`UTFU zD|qdq(JBxm@D?J{ddNBJ@gy2A82rGXQ~$uch_qWPG`uTUt=jE;{XQZBo*~&Lu${lA zzrVi%gV$}We1cyeT)BMtC1Vql6J2o6n&sWVDF1D??}8m)I8A!$H)`Jj z7~+IoI&!E3P=!E@d4V*}s3)_VCl~C%hqZyxduR{QAK7XE9!lpo}JW{(DGUQO}n6&9yO4+uF#8hTh? zb+KN#|A|jgNy*q6smOBQH4FfwA$2tH3BUIAl!*#orNz@7931vx@g@Os($L7rAsqU< z_uHlJfk}|W>cat+jU{2by14MIA}YwHm!dY9g|TUV0YEOZsDxErZAq1mC@m}djCKbZ ziCD!Ao8|t!(Ph+p5W)Exh}-zXp!o;ja%W>(hEkrYsw&C+aZJgrO)AEz?De)lf=)| zRVTcFv5d<~;a&zQ{uhAIVq{V8m>>JSc(L<`nawQMHD~H7E4=|LlVApz{YZ5l z{ecQjpTD$aK-(&Jzq1+b?kH=EqR0!igZA;5fS{FACJK>S7 z$1A3FvC0-b@c@jO2oMJ@$yCWXKOw~>Rsnr$H~$GaUVEX-7&AgQ^e5;ktYN?LuMcia zsFRh4J1@Zr`o_@=qivj%JMDhLWZr{*Cf)HLV57zzp1QgrdX#-5mw==B`T3TiNseAW zzwoQ-c6GP=MSw(Me==Pop!ptP85MPP0awv8!6-yhQjuOs{a}ZK5XB270wIFtATt}+jNmJ&KUe2px?cYOtLFdPRWth?HNVYF4$^LM#hmX` zP}+!jMhMfV!F`f>>5htD)u<4lAlt+(96AYM`co#wM9CQD^j;qIy zJy-Oo$=-Qd zpwjQU+0RYs8$T;fADaXy=LAnd@eE_R5WQ%_$B0+yOD9#=fAdQ;KbwmjZEOli$l_1H z1^a+fHIR7UU^Q%X`ePftBf`BNJD^iTc)?dwzJXvMm(mmKsXa234jkwT#wn01h zd@%qa=x9$O;3MQXoRr_9rX*qEdpeieL1Kl-c@MP6w6QTlfn6YKgI4PssNZdOx$TN{ zD{z+&1&laxcL1lsN5pLepK|TXQg%#g`+U7vKi(MJ#{ctzZKnQzv)0CZ`v1aOJIawu z@rJ4PSUoz39m0)>X>y#j{{B|L^9}81{oUGZxYiz)VKs|Z7iZRGTkcd0*`Wc^RichZ-YNWJ^Plj;85t#^XR2PyO|cm^<$aXnj&4|aYn{P z{F>Ng;egL_)#n!i7wx}Tp1L8n{jqeYJ8;)auj|as73_g zbi6`y^7dIL_Zh^IcWO*xLSs%PA~*A};TnRBEqqj5Aj_CS{;kK6NM~ zgJ2QlCJo;$-F5{u>4lmeqVNMC%p>?jP6G;t9RNw|Mr^Oc6f~FZ=W#f#apS9oD)-F*{m_tx7 zaQId4GT%K=oX~Ze@Ud&;7FN1>xJjSnbidR5^@JaVS>FJZ1tVzZzL zm@BdN_&w9LU?RUMFTcuTxL`6rgxH?@QAJ6I?{KDg59_@VvmTD51 z&3A0wYPcfq?}_6{R7`R&yed=J80*MLfX*5;s(qlbp`@-o@%1sNQ9wz5F;K;O4n1_~ zpqrUVOpO{fsGe-EZsb^f8_>Q5qo1ZBcoZmsTT|70;@GjOdcV@r?IEJ+y>#tyw?MeT zkK;N!IQ}A_+;iC~de$R13sf{b3J@k$Gi>$wy8f#TC$PaWiE!?(rg9Zil+Q)`Rd6p5 zl2h#=CXaD;o}2Tte-qT+Dj+(8eDv*$7tzgqwSV@Wo9UU67yoQZrF~|}Y@vKl-@K~w z7(OGC@7~appI>}^>hO~XzSqSTU+<3n&(G%oZ_>4r_d89eEf3*4IO|h0T`_F`5#9hV z{mZ}WK5WUZ$~2hrzF1mrrlxJh@rvBR!IsREy?gd>gS*{(>I#N>Qg}52yv8Pqw@h2* zr&X*}SmPE|u(}UH!Yc_c1fS{IM3k zfbop$*M8Tl+-neeU}zjdtgzktE{TMPE9K@ajBP%{Kleg&?OU@qxw*e77P%m(Op)gG z!9ZyCqGq{y%;Lh$ajOc4`>Vh8ueNP%I#Ds(RrJ?Y+|WG`xl}jj;3eBwT`dQVX_BRC zDm5RAdMo;(Sx@I|mFHwh_E7e%tNe7Y;%KCK!Pkls1IV@U-Y7|~TcGg##^TMxg;>(Q1ECcp~tKQWPHkpNM$d{+&YcxnpS_f2Jd8E`Hwd}#7 zy$8Cb*FNY~f0@Rf9(FRHvUI_v70nY9{hzbXs>{%FF5R9z`;>yZNO6JRv98?`66?o` zL{3HD|1A$eBZaTGJ(`seINQUE$)lK-mXC`p4eQ;;9L*H&WSV`v-i}QLW6iM+&6-A1 zYbbwYozN+f^~*|8%-I?7lK<`7uwBN!w#{_fvXAVRjhWvXd~Myo<-cK_d6VNVu$1OG zpR!n4r7-M~*}!DPy0Fvzi)0QTPAqhpc(>!b{PO&M35n12F@<9Kr1Gr!Gb!E)PghS7 zyU;&AM9H^^;hG=mqYhp4JD%dmI(;cR;dk=S4y&oOjE(gdke&PIYtrV~9d)#V_5y*= zBX8B2Z1%+W@{0Z{=1ibUY_$^|)(u>?lt=m9qTwg20w${?m0}ZT-j&^{wZt`LXN z(91fmHOs_qvhdG?m`}{9Fd|KfBGW5AGmcMIwDe0;ncr@kt@?V>s>V63F(R?wG^RI{c4nTGRGU>n%jwo>;{5!_${QuP2#=nA zJgebk#Y)S7tGYo)8$O72$mnbDvHzXY;_mM+I$A<)Eq&>`{QJPU!A~19r8ElLgB$8w z_BzOwpFY!Stwr(6%zCl*_ZgH*qrHA9%jV8wGAXA#)Wh*d^OQYeT`64|e)ie|?*ns<{9ByH2}1;%c)bA3b7PSjT)~>Bf6yU6`88 z=`yg(a^sH$M`P^vYUP8Ir}pXHe(7AGPWSmf!U*Hc7pE%!*(GRCm)J!WxW1L*f0Mqa z_C;k?)wyT2bLP1?Z))5?an`EVA9(Q|Kg`8t%P8KLdmBGTLKd+6X7mv`n?i8&?MMhLs&ceByHcj@uRX^4xK-Vfr z;eUK(Fz{4fgC8b8)g}(LyKvX8OjD@Ay^V zE(FF0FPPzV6v|4UIRM;}VBbcJ8AXX6Z|h?HjB(b9?uA1*<{D=-o*9%!u(b zM>Bl$lA8^q^Y1CMDsA>OaW!Y%%c0X8P4jX$EMgA_7pHBDq+iXxvv8iG#pT*? z@%2v+&M5AyuQIVZzAuzxBPl6oKX(TA&U<%uxNG1>I}YFH9=gTgADR;@DMEigyJEW> zO}GBX``qz3bd) z@s@q%VbRJTPce_al9b$a_ofeiW{P*r4zqP`Y0st@vIiBmOkCi#n`jpa;h1DT-}Ir9 za%j`0pwf4KDfD4u#vrvYhEwAB_#ngsHY7V)_qb)vM?d}gh+AozWaJarHOLWnK>An|vi=94Rjtr9A zq|G*TPfx-F-E+;ZR2N!8R za%`No&-9buM+sK8IRKM0nT6Cj6g!{`W}X`!(z$2B3axbiu0TVytHfNwXA6Fv8AMP6@7Uv3iLXj^+mX9+?Q`8NVB_6jjoY;b-beHaDg>1D=AKoS(_Dm2?fXa#U&qu`^M;^f$Bs5Kq$;p&8fUppdVJw~ zlapgKXyQ4kjEfi*wnk`nv*$%FbVP>l6_r=D9&zbzJf!chOJ)0boSj%y6tC~7KU-n? zZ=XLPFm$}1e(^~(kNcd3f)R|#+%d|gO__ZSzjZyfvP+29dQ0c3dzVmGET74Lp^R55 zf9+Z^`H+yzkr%v2Vf-1g>gMd(%wxT#0Uxu37wKMY2~}i#={u_YC~Y1}X4A8CjJuPi z?nm=u8#9izMYu0cPd(QWN}DowhQliCVtO;&dKX+Zzlqw$nyN*G;lbZ6}e3ev`QS0Q*-!ti-r_i_jeqj08$;?>EPCK5l zJ=Z^d=Wx!F!f%$4kgK?vcd7;T^pmPdv4wAB^k0725qfmDTzl7G;7Yu#b_9@`uY(q6oPF>Zkh@JZcwJ#;Fhw&2n#`EbJV|F~2ziMjB{n4Ro zE?)-EUAiEs?RxweoX+Q*``Nn2*g6n8=e6d!tUpPf1`0E5hW+pU)!JSN$~3$Ct32b~ z8Gb7J^0efhbdAOm`FOekx^`t?d@N5AnT(HC0)W}7SD6O3Ub z2AE9b(%nTO&DDMX3c`wZKK@lp4{9ZR+%S#uUx2IodW||N1XlQ|ZS%Wj@2P7}d|Qps z2y5)i&|HgSDODidV=ZFTC>8r>g*kkEi@LeY!P%$EG3d=^>U3 zIs2)h;W8M!FiO{sHY6^qP$%yGT^-v=;JHN7PU3j6^o4EkoL%?R1>j9IRnKK;E)i5No3CKGBYc?UsqEj4FN3VN{|L31i;T!%!5s> z9Qk*ZPfVNeyWsJZZL|#9@;tbvf+s?*2v*-<`D})Og0dKLYwWqgRvnsoCnAE-tPt_N z7<);`Hz+<3yR?NGf;Z7~&zU=4_}3U`up&@a{$yX1Klq%YV)i+Ln__V#Td~w+LdU+# z+&q9-KxbqmIlDv53JZ{%At9f!QFWkw8}BywAK8$(>7Srwt#})j9moJx!1(?WEV7=E zdD^`M6C~&>hHL!frjDK;2Z|zLZh-xe2dgWNWM;w{65QDG`*oQ5@v|f5g^>S<2g$p4 zQz%ehOt+Z@;Fg!7RGcrm32;+qy$_KZfjJ91#Ul9Q5fC}VHIF$mzZqm8Vs(ih0oQ7= zg2L4uR{{eG^_}>iUb_~xKq|kPd*KlW8zQ)Lrs)pM5LyQ+0wJq`RemunOw>H*ln_X6 zn~xn+hWa&Y7&9>uX#qdks;{pD{SssrMp0>a2a=p{*x zQy1_F+qwQe5P|@CJ^Krvec#&K4IYGE!L{SzfBYdvISIl0r{gN^tJTyFBf4+dSObII z$RHT<>f*3?W>H~@?Jvx^X$U|0ssy7_d>&Cs{TIEft<^?va#HMn7+mFaXI3_QLty_e z&>5sH?5)B8r2(ygC>62CxA3z;5P1-cq`9h8->Wo_UXjZ|${e=IH_oOX)#p|9*52(` zAeJN^ry${O&=@C55VFpJNDw}&B?)`tRF24|WiF$!k-yimC^EF{d(B++Mova5)jO-M)x%mnuMR{#%}XCx2BQM(y1jkfd{ z&_PBO4~N5vv6<=}YI$Y`AWeXAFL4uU-W9!}Yl5Ad$0UKEK=cdnf`hLClLR>`1C`~c z8vtyu_rITC8>A|l$9i6pFIp`sA~kht%?Xu=Lzf;w1oF8l>jc~h_DqOf zHUtCshnUGgR)gzip>D0z^~qhUxde=unnmt`u{{;1!9p_ivgk0N)iu}GUn2P9owdRw zfwVJI!}A}krbo9FRJguBNL-+p{KI8nWp|&PMOn)qk=EqC_d9!7Y}hvO>kwn>w40HY z3c`Mp21SZ?f;cZnUyJ(xHvz2wCC5(_F|>cWTQNA#ou6S9Sd?8#Hmk%I(p5THCl2ZF*!MRcGRT_-ovd| z>~XQTe}NlJ*3rm;tWtJU{mT(xK6Kf+au+JJM(F?a7I@-wDI!uE)Vp30VX0=$3bkJt z=iMz$+pheVGAJ`TE>4zYfV_Y2169YjW(IAUCmfK7Q4bJu;^*T5QLbF*6@DR{k$B;V zGXaEK(hz_l#aIcjuEoXV5%-y3vm*=uBS^U?k4U#v9uvB|>({P%XJ)R!nyVGbqpx1S z-iR6>ItJ5RT^*g@z%eKLEs&B)_`A{X^#21{mUUgFq?C`8g~u)cc8rbvIj6n#WHy~T zwFc#s#HRr2K6vQRu`cUY5#h*d|IdN%!5$4d>j~$71W`?~ieK4QQLN_T;^|N)FcFK# zLxClQc)!NIz_9V(zcTMXVq5RefnaPN=saAKEq0ksMzI8h}76t6nRho_93z}vUagsKSU=L6t_9ks`z5DEnn(g5x3 zYqna>ka`Czo{npIIHr@xl3vms%P)H?$lpLytH*Ucn(5 z6(7%MX=w@Z$W_yER61fkj*K2<iOy#=Wx3f(Lz&4t+9?R#?2-_$FQBlU^} zl`08106`5*$n|4QCkQ7H3c81jit6e>^a>0Jz;?)ryg*Pgof@ezN$2?Nkcj^4`huNV z^zEi2S#$r2t9Oad^Q^-J@%r)YD;dg9k{zBl1nAo?41a_fPYD>yb@d<*HA0t z*#_YAveqHT;6)}8H+SMF5y&%`0f6ydeQz$5{2#F^xs6sSYhp61@E}x*O0Y!Lq#rmlrMk2JNZv?ax zwC&I^oJb)wYx1Pv&QcbyStB~ve{(O*#>R%0aeb&jfO-H_VWPXjIsvcc9179liUi%h zEe5^pIlL_)Ibei0sex#Ohhz?s2JITroXu`b_X4|^IX?aw;(T5$c9Ou<4-AT-nb{Q_ zuvXaX;hir4t?GL-m(BgKZipF{JQhTk3HEH>=6h+{MiuB|QvIf<*3O-%-EOKQ)Uf3J z-t>?K7Ctd{yFSO=VEOH~Hj(>XI(1%xeCn1}`?nsx+blI+&OW?q_1LHVOQJj2XWBz< z58Y{BTXDL4RmZ2ZeMcx-SoWwkY*}(W3L!e?iyFQu%6Pgp(k_KJX?h=)Fj>E~uA;&q z))S)0hkF^_kLhl#NIjVgusOzy75Wuo16(9%yM^DUBq|p(m)&E8vFx9w+(677F>@oj zoyf?@(#E*NMh2y~XDS6ezKx&@ER>WankV;9r+MKyNAc;9QXUEpV>wnEJmd~pZRMgq zw&fvM7Rd;NWxoY$#!wj6EIrBjIwVE>#BwsWz7c~)$}Z3Y_E=Z~@j?)S>KbuKOChI-)eZW(;3gscf5S#lz_sh0?*s=``RHl8r;PgH8fUt+gTn-c`}7#L zL$=M3BAgh6SzdhoNSPvo#4TX9%3M=p&kb*(v%1GtDY;J0eTVo(W^o;4af`qiC6Y%_ z;C%^80;z8c9?^`KrfFi5nm*e1k;F8U69o&(Fip?w?TfUBg09_3;dro&)47-5a!Ei7 z7LM2hU%+>50V(P0LG3m~C>AmWh@+pSi{1(^1PvBK;`JT6WBUMTg9D%+54(L^8dw9| z?phC`8WM(PGo3qo$$29=C>lezpTW)w)?qO-6Q$o^e@zr&Pu9_nZJcwnjtp4%JZ(4v z5Ca0jH@uUB#9O$?O{I|ZW#k^bz~Ob^$PtC@7wbOcQNUz>^}!aLK#Us2*sv0l3GWYe zhKXf)cz}qP0x?5C7!HL3YSr2-xD$?j01XSBhIsG2w;bzglMJ?Rd_wQk$%W>hDEQ44L7m)+Q5#j^wg2+!_6R?kbG>YJ%q5w zn@+L&@MIYd+09*Jt6I#YFxZm(YC^?C<*adOenzLA`4n=tF80|V`t9g-q?c>FDNN1zd)}Ucq20U zy(@S1Gxms1^IsPP`%92|x;;9Y%Czk~kM`8cBTOe@@|p`w7ZG|>y40Nf?&dW1tfG1j zx*`kcH}YS;)Q)p)__n8-eUVseD)NTt^F!D?ZQ8W>ro)L}z>my4Ga?6v!;2ur9<;Sh z+Hhh6(uzLmRO9cA@7fQKjHn)WQ-NXJMaU2F#_PI(E`l^iTQ$nzY3oFB=|eVO~i&So!H3H&MEDkdQ<8iv+sF7>9+Q)ooBGkt$`uK zq#1YH&YC60K>p&5E2$}ir4xWmj(&M1gjYiF`f|2|`23ONOV99&Pa5~pC?ph%xXVF< zMMTG>V~QAQc^Y($OYBtJMvBvp=Qp}A0zpHNMnu{&A)3&M!?K)*0!z`OR`Vm*3P4SY)0wbgj?ZKWro?)N%v!ic5Tmgl zgz^Xlx3Z352WG#?%ii;G)Xmkk5X^sN&`E}S8%0q03oy2Bmb@if%keNm;W!o2$V)+i z@`Yg82CjzU?KRHa?m;Rj0=rgi>UH^f2|9v@J|H1Hn^Gzzfl`K^^p7o)j@_($n2u7u zBTiG{8-#I`(q`5kA?KkW(X3~zo~K?_yqnl{&_ct#su1z4E&csLaPaDzC75dWb4oaQf0wZKsL0uTQ~eBvV9f_jw3~%^^sd3RGt2xcYA1%#qj4+-moot|#09`a zmjf-#%`;a-;GFLsW#BZoLRNZ{@h3w3hwQZJql#bW*pn7Vsx*823n0w}X(@xw5*W5& z&^2}Eoi#jKBgzTk*_t5J68$`}=EaPF>+uE+3|NovPKoSnAJ=_|li#{l(eHp0S8~-s zKBxwWi>YPqnHg89^Ant;$OFwOKy41FTeu_!v_fG@Hk^ij@2XE-$`gg*M zR0VF@T2HtgEMK`Y01ZmgjauZqUui?UDyAJ*a1D30b0oe=h>2|@!jnU9^8EHhj4oRe zj($ef{vA-shp*ng4MZ5B7A~vN6?Lb2rS4`1V|&;&w(mP(V+PLou3iW&rBoY9SnhE+ z8&>7@=M2p;VRWa_-6m-0DrQS++yop;!nmgkEYbr6SGS*$m@i7oy2tU*-LoLBOv zBkK>N>0N=3_4&!ie*72tKh?Q`yvJ7k`aDZ#bS1?rXmHTe)Bmvh+K+!Aaq_`$|NR*O z;xqm`Jsn?ZOW@{y{VT|5T>PK^_)L8I=|bJKe7mwGJ+YB4M;x_9x7Rbj_b7h~vO0<76-8kll^%gIv%CL7Mk%j>BVyZjv#>4askUeQN$zg=< zrRyda7r64_{?)`=tfE2y5_?rscA~H{XUL@k69jqZ9O{y1<8UQDLFfW&9@VZWLG?|w zDwvMMS6_bN)tWN{sn!i0fbog>DvZxQ_w`-HOfXfjgX_^|dx#A(n-Cl*(7QS6o1ONU zU&;Wr5)D2yaz3C|euCW;z|N@@ph_t3m2d`8@Pg6`mkYnrp`#OgA1aVk+lgIV)q_aU zi6JYy3D5ENP+WN?3p_(y2x z#LVdwCMH!fw!l#1C2WMyt?s!kRrDqj$IeUiYXd0WcJjKxT(N9T;Y5ZZhxEWWKL;kq zPbBAHY(#TCooI=cPw=qDq@qXXcTZvsRcW?YH=!bVb~Vwqd3Q6`nijX*o8T(F5^@LnZbFY zP+Y3B9|}Uu{B3UnipQfc~At)0$?xCYUmzFN+6O6*Ej?c0p9#g*nd+Qy>WSozPFoJ;4|1 zu8QTA#pWg!pKs>MPP&0v?mQj`d(7vqupYx5kAauNrJc~T)3GLr!LA8;&~ttL{B{FV z)ahWLv%)$i7C=H3T;P7kIs@iK2j6S#6|tTysn6NoHdUw!j;=bWvEadOhwMw%bD}qc zR5qgw)0Bmf4aOm{+iaQM8|HEq4qv(f41pn$+Kq8N0-!7LXcxzkA>=JZ2e5js=mk!A)|y9VCWqFK z1C5Ty8Edp}-||u~zu~{0OLFy4t-7$Yf%(1~IkQMSy&x5M+9 z#A$k<16Vdtih37k{x$$^^YCE6AgN^uB0aBVyY~|;{C%SFitts3E;j@bi|{b8GG8Oo z9Nzn#=(kN*2;xiP;LHh8Sx~xVty}jz5jEt`4UUa<Y4LB$9@6xq64|=T=}X9qR0T#q)aqA(tGtE~3r>eC5+}n0j#p|xx+tOsF|9*o z6wlZOXuJSao*o)Uww4_pR~6_)!eJWq$*A)!BY8ib;=U$QFWEv2cxhtT!x{17o>WffbCu3oZ;{>TNQtm;ACo- z6nV|l*XwmtJ z#18{lxJ)maKW`q4A0;pM|!WQ21CfdF|BfR zVv`Bmc#*(3l(NKgc|DS7iMJ6R2-!cduZGUio62EMMK4u%s%HSTQVFUpl7v|iNiD%6 z_ui$KsR#iGAo?n~6bD=kKh&l5UX%Ou_hN}aPT%HcDZtui+G!t0lIA2f(3yiKPoPsH zNd~w8Xw5GIAnY8Z&4Aeai@P2i8kC<4dOm=e1=I7WT>;?cGV zJ#}Tao1;^6UIagmCqX!4BG)%aF8PrXeqoaHjJ9B;oM`zw8`4+6xce&X2&_=7fs_f! zi##FGyK^Kpd@!f9K_d@$<~>;(Otx}Xk<7^ZJ3<9TMVE`3gbwVA_q@ahQ{By64qKUwQd; z%kE8{ZPA#KEed}P!Y&|@t@++OF`$m6jN)GIq}6h= zW6LIG7UVTcd9X~5&_mYAlP2bfItg;X?rICbJ0o**J?wpNqW5@xah6!j$@?CaQ5&1a zU;up`Ms;v#Na($ZWz==p14^zv?&EG-yw9#-n+8mud0LK`96xqU8VCK8$B!3NzV!C$ z-Z=i8K7~S|2X`=rw0utUxaWa@Qj$v1xuleF*?kXDC`^bY zMqV?wr#bg3-iuT~LEO+Bu>iuAI7rMKfTxlSh~Az2&LhH4Qli~W@jz-vF9tX#4$XJS z&6}0L<(K1>pj4u(b_^H^7UNuY@=#6uY6W3kRj&q=qk z;|iU{o#B2?V$=_OhS{SGR6A3&$jpZRG`_R6fd&divrk`I%LH5c9=&~3{ZVG)=eD*2 z&m%ZAVbymGT4F&V`~fG|>YaT6Xi$ia^sZtgI3D?jiv1K+CG=Cgxa=J-BiElkkBQL= z8zM5AC-UQVEm#0D6LyIt9+z0NpeZIU-w}P!;(z~H7Sr1MdwR-zd4KyYq04Pz)OlBX zohuHdAc%|(0M9@aLlOL3jJ}}n%*XMW*+1Xi8k)&UG?{QUpe%2;hm5%PRFqg$7Ts>s z={OxUZ-}>;jdc@O$KZA8J^+~cy`V}ve-LpT}*0~b#RDn7~pvN5kH!f$WW!VPEU zX^n?)fts|T@fV3;Jjo+QvJ<1dRmm3X1h4Pv-YZ=Bl?qb?SW#h$i7*Yvx(lFN3*3;g zU(g`SKX{E!p>)?i;@4_oDj`AimqrOx2Ut}Z1Zs31hnsR|O_B()z6ZDwjsB6C)Xm-eJ-VMgAn1bALu^TLG9gLJJm<x zeMgsc9Gzurm7ABa_M3hI=N}bP|H@_AGjdcD*PVN+xjO2rSMS;L&{`*?OU@6r0H zf!qHZV83a+yeEPLDeV!@F`=Zc;U)-fz1E`;V1K+yRWFi%ywx z0De%2r51*Zty~ymuw)D`GB)-k$R5h4|aW|X}PDH}B zb#ch{&xmV}@(4{tsDK@6A`y+AxF4vGB`|hN>70173rD~@yR5ON+ApYN)x2@ewO2k|m8tuPntmYSc6>k50U8z^dMtFVj{Cn@*% zOQ7g4Saq%#4k|N`*${JT{&g_-LpW278p&mj8PjatWge!8bJJ#Lt z`R4c&kUz3=iZ|xvzK0^xg)Ck}ug{>xiBs9b!mwOEaQN_p>PZjqkLo{shz366aORSv zRN@&=n{Sh|d#TxJDVl8E&`}fMV{1J3P4rxi9x~Gz9%t{mUfP>-JWl8M@l~KcPe=6t z!_x-6+MQ}Rf7rmB4PLbpK%#tc-0XMGhri=XG%W-*BUc(nmv=;jr1zi&2fIe=*Eg0C z3aR&T-el%hE#qM97T|{j^2JhVX&zYm8D1wy+&{4QEHmpvtHelt34oy~%W1=!XwR|2 z$7bYoU_8xjPY7q}O{x@jfH9@?5Yd0&+dLH5o(7=r4><+Dku`s>4Em5`x$Y4F;GoJ% zL&{6`g&?f^TCw+p-N;3>YM8`Mc`}}G{td#u}tE{6~MogVs1{CmN1 z)+7;R%Lo^i`lOFk+aCPD2;KpW2b8OX$)_gbjAxxT?i?h=EX?0|%&gVG5uIQS~DW!B+*LE{Kqb0b4!Sm?tWFi3|hSvtWx+f&yUd z(FUq{32M0#{Ovtp2uSw(c=}UayLNBPGLmagr&5U3EZPo|i-6)u{GQO`Bzs7Ym*vA2 z7GJO!hG8m;9?Pj{4hB-Sdxy|M6E7&>bDCR%76k$_C!9+(%r)_j#sQHY-`k;lSH|QsR`ZV#;@|w ze7+>H49_VdEBAS0y1LY`UbsyPl?2NMf=i}??>31QklY_k>y8cpt+4@rPlrNo*l1yklz-u7(n@R-FSPoWV+oUkZk#05XcC+_j zU}EVYrr#y7+{RN=$usECo3j4f5-d10C%u<@n2RMJFJqCpW7O^7n3kwiifDpRFwBGamnIZCz$5;9N8)PPko6lDw< zq70ECmYHR(^}pYSefBwLpa1n;-}RkqU*~jchxPlt!}HwFJ>Y)0aFL&%f4!XC@%G0X z+h1)CVxMoy*6kfmn5`AX_Iw)_JGtOK(w{UUT@>4EPf6ztAlDrO+(=9qg({si=Cd3F zN`FmsEXGw;R4Q=Xs0?!n;`O}$rl^cAr#PoW-fY7M2C`~VlXzEe3}ldlxW!RY5EdkX@8kFvu3g8@u&mWYP5BmX z2wk-)>LfiLzDlh+BWqU|l3vFQupLmpy#zc%zS)l$_@`j6e8A~HFf{ZC+f*;z>}K)U z#!F7TK-%`pe%9u}3)&4<1%}3=1hnIrIwO|fdITj48AjnJm@|7etaa_1bO<(fA3I50 zPy7@c+o%d&?|Z>WY(sz>#!Vs~>p9;VF2#Z&-#ciNWNT#k7%WD#= z#f048!%6dJC|!vk5%?O01n1`@9kV|3oDYBzoh%Vwkt?2nD?9-Td``-0ia&458+5Yw zb*u=%H!#cp0^nD#k7C3coye*QV zjXd+R-sA8Rzzln|$+ zGtK`tWy5Dx%_~j+*VqjNox`gyM{O0uA?bnRZBC&NVzr!{Cij?6ba`*69MDgp3t5Jn)0tB@iI62+`Ehcy? zN*v`O+UP0|q=L~=d1^Pg^8ZA;^@c|=_`uaH-h-D-5Ej5#?RjoJR%%FV6xO1ywSE=? z1NXNWvYlM!0d+H9mziv+~)4i>D^wiW<=-uP>v&0E?eGNFY z4Sldzl?G`W%!#FS_4S%CZ$?{s;<$t0vt#LX9#&3^jws}gR7)df_O`rOG<#q?M#BMV zJj;!Tcq-N3O@^mXMVW)35*^26@7_K(_Yc+Ye5&$pd?_v3W#WhFIo$+hW&E~V>Flr zANRbQIDFsb`iMkox8u2PrPTJpq`rcPB&J&p)oEl2A1!}KLTP|U@c^Sg$Z$tar>q#j zOA_u&yh1EPpx6--6Z2zm&Yu0{6m4>V6A=DI%!J-O*ts05r4ZnuZE-US0N||!;5DRd zDMblW{_GD^+z+vCDaW5JM2xDZudgzggkV8l2L&x5ADI4sQyBG1G%Ir)?T8;ChAORj z#N8WhKqN+>A{TW~u~fG5-bBel0+dLx2jHIcQW(S}LZS#!4 zs{aSG^7O2_<^Xv49$>12sVHOHMO0%X8>GYpz4FHwb`qd(SELv!^fcw@3$MUv!3UM| zBuLbl%JOc%@bFOka&HZmQ!o2}Q8z&dcvwlTs;orQC$?{!$yiJ{3+MpnTbQ@b1WoiL z0DEGUEzcS=S+XOHiEOnSz;Qlzgi*g5+`kM4n|M;d@r_O++0;0uug647kDAo=t4C5OJDe zKT%5ajV~%^y{cvB`wG+-Dwqs$WxCYN*H`cfoLb*d#7mZldz(|YfIQQ83wu^lb9xe2 z!Jk&X!#SBFi_%z&^ujEc)AC0ooCkS)G#ImjQVygHX66o9T!{`v0kFMdJio9|^VqRF zos&UmXm8$=Hjmh1h!aE)Hi<_sV@fyN*K8;z_jB~~tPdTlASE5-xLk_`+&2}&ACmbD zMpJLxVK9s=g23YH2RKk~!a@YJOe>k*#lb;AHz8gkVRD?P)mC7eRbt&e4J}9aRRs}& zr&?C1u@FK10#MKZussalFmg!(Ap6iVyUwtA1ovGmYag+t0%eEqJtRDUQ;OX!pKD@x zO6Fa)@usJ7P;_G8-kwWtyJ=S*C{V?Z^oj#A5B|z!&bJptJ4z;Dv@B61 z4r8so&$d#7DR*JGKT;n9vU&o%?Y(Rf(TiFf*0){0=;&cG2t_plh zjOfKT{);$UI=o`*Szae6r^m1jM!q^fG7DbgIaRmN>kQSBtr(t3RQLvA7)uUIl!=31 z>yWoiDCJ2r1yI5LBF`E@jioa_vFbK0b42(z)wVX_xFCC5G?lMKq3|qKejHd9j>0K$ zi*MR|8`8O7oZ6cG;+LFiqNv*Z&2C-7na&x4w+-GX%oA%aA3qqzihQ42Eh#@|Ja~a| zPcRfc07H#1<{=^IcFiI9vz}50aVd2M(&SSMmB&b6e)6E4}XAIq{BuLxZo9_U9J^eeitRlMXT@qtd)7;l!!9l$_Kw!!1_R8b1`;O7w6-EjLN|#+eaLDY z0{C@oL2DW<;CS;0cf5jz{3gR3hb7p(u7DIUoVbXkDt-2&hRFjUw^JyMn7s_ODWD=h z@IY=ErmG$xu;Z{RdTS$QsFUB8lEfT@Wub<~s*k~_ZhUZ5ST`fwjD$YFqJIJOK=Ex2 zo0#wy?Ck?nUUYP5Xc`4`apXhp5)W1aaTe?u5%csqb&;SJ9I-0U8=?{N!NM{Iu7p9W z6lGR%!HqIwhE6;EpjYOE*Eg!utSFuVb)}MRHZF@AsJG41Bu;)DgJdsW^7tEhz%K2> z5`y_3`>+X#uo~ZhgF@&R^xW~U%Ri(T3gAHal80OUSs7`p=pZC}LPd38E!-SfW;Kb+ zK;YafI!d$39iIU-4H>^LJ1ZiIWc1LPH$iaga@Z;N} z2r&5~)@qQoZ>mddBin7FXt`B3$1Z!P+8J1yd03iu%(I=78!}mn*5|qfAfPnSxa@4z zIT0d}6W%QqJ`952h<4sKm)XOlbc?4&bDG?9q0SCU$zQxaY*JdRk;AjeYni0+w9#ky zX`WKq^0|!fvKhNaQN?%GrU(#3f#4IxFNDy2Qm!DHam!unnHoLwFP|p01L!=UP!nPr zxNiM)ItTacdg0BR7m_F#7`=h-N^l-%@F_1*%0I?an?&{D@GWZgxflJQjE%u5+h(N= z$EuAR$>UJb)m?^bD@AgC7>bSSnd&McwF>`Yl~X_foQ9?sDs}I|p?m8V3!-U2Mx!67 zJSP!C#w%^M07t}3oGGCtwM$aRmshfY=V@GCceL)1VymllRfN#QL`*Krq-Ugd4Xjdj zqD8+-qWwV21DbmHre&IdhP?bpS2ZV&sMk=0mSYrzNMSFAQGqaztUTwi#JN^|O)Kp~im$VCyCthBV&mJQ3~>^FhA3SjOk zoQ-D@u2+-u=qHi8R8eFZ&x2|JBo*`9xIkys@7ah+INxh`a`y_3Rq`xe{28pc5UVM~ zfB?*}?I#fX%yFA>sh})~aaEA1q|dHOPmMF5EI&P&FiT${7^hvz_X%Au4bXR;y2FAG zwh!MA@BSlqsPI^r*WEDp9wdO!v6cpT;@P-Ls`Iuar4&r~&!#xRl^C+Hsxg>$8JZnf zuZ%>sdhzE_Yr=)K9Fz{U0s1LU7&}6TZ)>X^xZ(9}p9?3JZNIRJ;Ou}^(RLZ3D4^um z71h-(!{-1DUx1&UsExson@VAgwQ>>e2M{+*e%jXEN8@RN9dgZKgI&aJ+nF!LW{5O= zK~}LUFh?||Lr<=v&g%=E>4fU5yFuQ#D-4n#+(!hOL}9cYe7HWblc0l99#(0)6Luyj zlURiXCDF5~gyHW;hRnD^R)sqBzkWTn!`g@~Yd5AGoqaoq6&tlLdnMVxSl9w2k zw@?-lX4|n0izPpZouVh|dt&nOH*sN-md76eC!r&(7ZF)_+h;|X`$TXi{a~yMuy@-nn%D&Bt`)KDJ43QuBt^$IAqzS#P4;u(LYOY(!^G|3M2@QJ~< z8A;Uq(y+ou0~7Ddhj$n{m?V|CeKidqUfVoYP)nQp-Y|@=UNxZ?>$FzYQSQ-Z#A%6S z?Sjqb!=HnhtHom1pIiS8*x!sG128v*q@)60oYUk%6ONHX%2M$fd91NLsITDbpmpf9 zpSO1&#;#BMsLWv=a>oL9ia#D&ejT?Eu14bH`+eyb1@`13yeYGfkHd@q*bR~kN%z9( zFMQl+=jY#79Rlo?@O9A`?$}fI?wvCF4>Yq|zTQP`gr+3t0q1lI=o%oV!?xnpK!%uf zLBxKhbO!CmXk;33{{~?6?Rahg8(iAu?-hpW0!j^R96oUK4)jGp80M3be|$ZVQT4u{ z1!F1j=`m*XvoecY&m+s4_2U>hEmEr<%bxwWQVVIKCe2;E0$T*zs7tsl6@HCuaGG90 z{p`XV*7-ii8aa%_8e_rDdagReRe;$|lm{2=>}FEnGOT;NJ>#Lq`lr*w#0OIb+-%!d zPhE2Ofg$2kX5a&fH)2+Vr$f};eV-(pM!u97&XOEyO{1!$SBFXDVDf?X<^sJ%CEcQW z&aUx^vWkks-AlHgOkA4Wb2RK2ka4QuJ4Nns_x4~*iC$psX*pE;cBAKtC*9Q)Ko z%EnpzcGI!$H1E#M3OcF{&`eXm?Q5SLzATp?&}v(597VSs7w@+KOA2ShSG)NSq0tBpri)n9WZ^(mF+>i31Iq$Q^3I>-qiy3O4{3 znrHWx{tM0r$BAo69gbppPn{Gv!n?(TiP{JdaKsFVr#Om3jd9jse4%l9#ch0OyOT@% zX`x%8@#OMh!1t%4CCC`Vy%lmoR8vFa3K`Tq2BO~8ww)X@gBkt-E>6RkC|!dJ z#$ojP`=Ex$MoPzYbxFD0xGVHOr$-A;SW%!7P{DzP>$&!lIoqKJGO@H5n6r7{;De zXLy#;-6DXiZ8vf&$j~xDM}kt{Yie%p^-q$Jw{_ZT;%6qgy9}ReJj1bMbvobI(22F- z(qvT)HUCDlljk=D<4vcWK3wNz%o^trFz(5^)T@W~idyan^D zFDdyIt#gO{Q@^UqrzbV%CzoXit|Nt&qYNN*JT;Ya+_15f))xga0@C<(5tCvPd}iI$dE&BiCNp5n~kD>{C#AT(G@!z zHLExmrpZ7(V{m#Qb2Z_#^tQ6{R%2tM8NOTyNz*VHUozPCT7#_> z;m_u+$Fn<_;v^*G0Ji-4?c48oy7g`Yk=LT~(QMew-UI`59`B}IQQsA!pXXM_ zTQkS%vxMpmmaSX2GmJVXibpng_N&oisc_cnGpzC=n!CZ}eQVANq_U#=L|0(ac>qQE zI`Xy9{UDiK^Ww$SeIec{?=bP(dN52rG%)JXBMmadsj3PDrP&-^q)4|bAdd;6Gw6eL zcsP2JMchylmVq#^TPxNJa5RjYD zkj)d24{>a_KvR#l%G=YEG^NR5NcOG3TG1*lw_~pQfz;lit9#`#U6aDxanv#!Jk;1^ zNgDU8KEr$i?T(Vc<@j^Vb!VJQb#leOxtG_tC=$@40PZKu65er3rFVT2xdWmxJUQlv z7ZLICmUH#D+pM3}Y=r1-9 ze>M2!IgONsDUM9uE&fOM+_+%k=JrYa&F}F__ENL#suur;t3o)rYVWG%h+Sm zn)a%r^i}-ssTp$bj-TMR8CW9|e0vqXLC}DJ!1gP&)>gXo7VY#y#&q5t3;lFbuG_X2 z>a}XwGM@xX6uIUr%gO#R<5@%2YSZqcz^u+TdjHm`VIj2!DKHdl*iqmE$`qNcg6_C)P_fzvkyF4s1|kQgVrQR=-2b-XHE8Qdn0a;f43%yuG5^m zDK&lZp)o@~*i44VTvFEaU%9SJSEU3$iIAiozm+iqT6*NgG7zKe!rn` z&Glx^*=1e%h4E1-vPM%_x*2n*>F4Jy;)J_&nrh6z@LG0{E*|o#w!G=-jWmJeosNpe z5#RYFqFffvPF%j_+WxHb>$g*zVuw%tCELS3bB`@jTC;GMaiKua;%T6cR`Sd_u6NA4 z(v-6AYJ`5vVmiOV%C%YNzZAR8K7NcPK=s@|_p@KB$;ERMHgd9;7g8?#!E%yD)<>-f zKWpq{t@vVt=US{ClH4m-tKP0v7g1sQ%PZ=Qet*O0<9MXIXx+oAG}Ffjc^+hps(kBT zCCTj`EHRbxBv?$T+oUL9$9aCr!%H^Om5#&Ldh({f5kG!KKTO`jZnfORuuj|gW_X*( zo6OLQiq8A+iUW_$E7MgTVO}kHS2#AMoLFvKctfSNI^im^`$Da9Uw!JM?a?)8+8-Cz zF-AAbcG!aSucuk&i;|rO_6F75|Jz`7%ktNTmqo_=6c#PIqZYOPdCbc+n}_`(8up7; z-R$_IUfS|{Hv9bcPVWz;^UuJv=S21!_qUrjpX}(}2ir?5CCg;zy-?m&D#l{mKVl)`KKX2#yUwuVu=$s$ z{TKS0Y1x~^6h3?_;M6eRa6lwyLd!toN@bD1QA75Q0rY;9lihv(PqO8{SIm&#_asd) z`0eM9$ty<3uE~GgYE|l-X(=2hw6*z%0=t8Lbc zO-+5o*#OJ!lRX-r8!6etBe3S>)%9>%sN4VNs?m@0eV4GL-LmZ2+1+_E2w;TVu)?b$^M*m zX5QI*Tis8bT*W7TB7ULA;yL8-##SF=8Y@@bS4OetAxwKiT~f;@2Y{ zP1_-R>7`|tmfC@XM!nrWdag6#qU?IBIWr<6jk^1|npJ$rwi~!Mg>nT8^^%z0%jbtz zQ?8t=ihgWaF=gG?uQK0aXUEK6L7{ZiXBE{2ENEdE=6yp*hsI^=c`loy7SuD=Xd(4SHJuF%cz_U zjd^EJR+Z1ul$!l8z_oavpv1<4$VGF$yi4zIc{MepuYSoqoCWt4*vh&FE5BRSMJ$!* zcI7HRJ9y>;^Yc`>ai(2JU*&Yl1(WZ`?>u`hbb;-%OXwB6V){ozc0`oW#_LS4Xw9Kj zCm4PGdi_?u5Q*H4~yVscg=%H`!I9b%I&1& z*=_yo(9Dm2-F?3IeCk^TY5ICcJyv2<8ZGE(c1U=x{k!OJ9I*V;_3b-{W5^ zU3{TXX2ic%KK|03dUp*C@B#Y{lx<|&*c>rD2CQ#8VM8oJ7f3z0PbMx?c4OKE^_|-|JmP!cVCb%=4r?j&~ z{&nsxQEZJaz^DWBG)V{=#S;-rdJIzeQD@9;VARo?OS7j>!H@z_j|aL4%q(ugEb3Mt z*p!lFCUbvlFT3u==;QUJa)zIqXWW}{PCR1s?%nIWuU!k~nAW$Q!>!tvUiaC3`X0Z# zcX+`Co9{ut#D78k_?p9khu58nd~6_O`thOe2i`6D+TyON9J;Q%^H)uYjGZ-S9&OMg zeEvZ@oh3F?4P3%H=&3K79lDu0SzX^Fy3T>UG!urmU@iD34sbk-j-CPIlq*p7fcSm^ z;|sxFOf&P^OD4{9&7LzS50f@}y=4j%?@Iy3))Y&`OlE?Rya|L#u-J8uGSmv18ykH= z8UV2HsW0;p0yi_!NQ_9b`+v*yiHK}<`JjU;ANu2@g!8=nehM^~7Y)2P%* zWHw0cW#T6Xh(^>eNQ1@qh;l5%vyJ^lL3tD=oCc0v$vz=|V|#zkCl6Z=<$wWZTOO#Ug8>*dE~ zGNE1y0wfIxxH~#ow4_99fI+VbWN))zt9ax{U?MNC3H(lZyG0z-^@z>@@BrYefSgET zhlMd9L9<~xfG`!;uV2rE_F`%?Eoh%R7*UV|?(J};9UCWBOnCStsf-LvAahXEgW$mv zfl9Myn8=|)1g-okOd!EA5|WoU=)0sj+>8hz&}A+_kPc*wcVo)V)UnpqRx;H=P~k%G z4v1Ry>`L%8eff7R1QG;b?AIyziYqlD<>R3#^5v);w zb_Xd!ikMXqYcash-EBO}o1=|8#{{*hajk(NBiq)=$I*=ru;1Pl&>YM{h!qAFkll`q+pF$VfoK zRs%{=>6Mj)OrpxFsB7>op0W!V+LBNH{2;DMLs$J}Jl|`PIlZMIZx9X%0h)1-@Gt3p z)1*5?4I~Y`@yj?;;5sp5+4_8C?>K`|6%*!C|Ha9AVoxH}WW$m0SvYP_91EUgW$`@l z@iW}fy2tw%TUuJ0d@;OzL70b!K#k$vVclWc%_}A$5xhi4|N0F9>n;gfr(0Dl<3at6 z@{)j#zue*O_L>2rP=)Dr`WaA3N3W^0Jjotd~A(%3Qt12M6(?y*vEkbgc zsj07kHp*zRKt#TR$#ovge}K6?DLHp~JqyNaAF8W~`$k^U3gG9fgaAi!P_W@7jq=>) z2b+sPprWd#=J`0l_|&OKHNAqAO>m9+LdU|b`0(LT#^(sF=XXGkAd4N(Tm_!nn0v`O zt$hV%?`+U9h}76`TT`co!G^-2G9ZYDj~sCYd0|cpc2js}=21&uK0b2F@L!*Z!744@Q<`0)TQIo)bth5B(pP!rv$5H~Kkg-YaK6p^5P-I5ZyZ+|>1hkG@9 zeF&qBB$5NOU;HAk03AD!6BaTV*lDo~C^u3bN>_=ZcLdFyP8^?cs z(qlRm6H9NyI0)PSM7Kn85*SYDtj#(G2FYoDn5SSE<`4B{_F$<6Cj0Mgk>wT0$NbA z5regKap7PgZFvf4NS|R!hBvkgYz+*mPhx4mm}^pEwFLfEV8K9Ir?Gwas%m*-Iy6!y zpf{5FC>B4-&_-V z@odMV2%R(@OiNCl57Js5P78`0vyy0)5QK1PePL;hVb|E2YL@laEZ$Ul#ZA_IXihOz zj?S8^G^q4_GN`hyb>@%z+(gy-Ox~qX(hY5`?%VZ0K`4j;s~HC8PwMxGxcI3mVGJjXNh8eG!1dV#J#OgH zxj?F=k=RT4-8u%B-HDWLktqOcOv||`IhK%z#p0J!vH zAYtGG7dAj1RjFR8@pN$fppl6{Ayfgm&8e6YKP>oL*XJ%1lKp<8t4A?Jd0?VQgNghF zGu)-8OiYw4GVURL7XYS*;`0w8cY^nxfbrnA)vH~>we$2e!`v(# zj60zpmR)Owg)Mty(u%l#-r_R*cPl@tFe7I;u_?w*{N&Qoc3Oh1LcAs1+}({xZ0f#X zWIN0vE`gxg`_*$*orz}|?eyvR`Dz_2j!|A>>O}|j%mjqEoeT3z4CDik!bcbG44=XF z2z{O&n#!}R?B_QfiFiVJz`ETM#u|!Q$=PyRB5W@w%4HmhL_IZkrQFk)*G4^e^9Ef} z2b%ci#4c8zyno3z=M9{O#O~$786zo7s`q@EgA*wfVTu3XwXoRFJ&vU|a$s{{liQC! zApe!fu$_2RfO)kfR@s+v6p*QKk_ns^WyL=6{sWz6_Q7HcVl43wmY}0d$jaL4R<=s* zr@{L_zr=`>LwIsJXpOFl;ho<$8K|MqHZwEBFAuDPjVsQN-pm0oz=>HQamNS22B(zS z`Sag12M%8C;Q6`u6JOA~F7-rJq1*>GVwJpnZbPxng$uht2*ns$Lr>2e35QBin)H;K zO+R1EompS@`4FuJi5S7$pLkK>oI%lQ^!naCz5`+MD?n5N)2jfJ(no1&?5g6~{_0cb}V!8S9gQ5-&p$DvmQa zhE2h~i>OEp;=nt*x>!zlas1S&+7%L8Xx z-{|Odq1HGH%kjWqamblsJ3NMo=oTEiM5KxOLJ&6z97B`d*3VqfM~!1iUUJ#nYj}w9 z>%|{))XVbtw>le#aZtcgaH!(rMcUfhe$dZz9J5MGiS-7a{%S~HK*oOcZ?N)D9Wfjr1z$MBLd%V83tU{J1+xGaRo6hwU|_ zdnb-1AcX+WgJ=%$^TB@IesV0eXUsP#Olw?o7<7_7kblBd)a}uuHQ++efN%sd&TEHs z;e)ydnb4(WW|xY_goR+~HV;-Ds1mp8GSWAHU~4iVFTGVy3mN&jLVYDy4Ec~k8*g1d ztP;~>wKy$JC8eoIK>n|i7X7I-E}JixG0(?WJq(8b-kWLc5y*SZP;ly{w154od0)iK zB8(PE&ngpGv`7gyFb-MOX3~$2CTxI}sD1BWHM`=`?EkE&YwOnu30;F?_f&6+^CYp^ zfe^*(tGSukDhiRTB-I`~tPnR)&=q#`=57=u%E}NZ%jjVxa&x0*IBa1NU*m#3zX!)F zp?MEG*!i+CNxz9pmE$2ko2H3Lq*mIjwjuv;R^I`(%_%G%n$+(-U+C$Xo}3QBs%O7F z*llJO7I~1_!_nsk)}_6J!<=s*R>}zPQuXB(B!uAYdU|>n2EQJByGYo2gbG3x?3!L; z!$Ea{{!u1to9E3y!;M%axLgassQ!Y&1ODo8UOlP?p0yG-xhpB z{DgnC2elhw5{lUH1|($$3}8qJ3VomZml@`JP-t~icQ8%fkwwt>dl96Lq#x2VBju~| zn(UepYlkS!xU{rP+Q#yGDXVE)S7j2bIAT19!VxaV+b>X1Pd^zI6Q&)T)ExiEvbu(4 z_38QA<9f{#Wgo+w(b40~aL9svaa*VN#NL~HhUbd*Q>fogfeJG9q&uEL2r_*!an|Dk}P)Bw6|y>FE!1a}6%_SWgr!_|JyMpEJl>P zh#~n5Bmu5O#0nRZnt50_#7h*u>g8tAL=B7@!uHaiz$9TDZX*O1 z^hzY+r`dTl^ums8P<{x8pq+435L6>L#uO0;a0J;da?Ks#xbn|6rm3uRTRt0}o8gIx zb5VjSL8A#O5XpfsJ7)r=HOxqfj0siig7fb;sW2ZEZ;NJ8F0fh7gf|ZOVn)`tcsdnq zzU<)|eZVezE?wFRPPPXo^@JDpq&2uo;kp_~EQ9+k*wzjHzitAXbVMEhV}0gUAb|wz z^{_BRx3M2G^mK@+L)~Wq)r9QY9|j7L{~kGgI;vEN50&=Pr3cUAE91N)EEOn1r6HB^ z_Vxy89f+}yiidR_ijG>eIn-5hjCI9jNrWDm@N0!YuW;rWLAaxOWka59CB%o}IsH@SOIAw!LFCFrnYymGYP|pI?n= zXi(UqydL`eYN{*UCf(UHpJ5nJ^$j)DWCU9%{H#oPtM=?38gdvUrqEzBWB;MTTkm`K z4j?O-;ygxu1{}Ei)Ab^SUsuac$9Ipm8TE``0!B*A`2H$3_>Ur|9x>4KG<9Key@r-$$PKRj*)X zS-0=22#Y$ncke48Q)IsqfN-5y!^er8oUII9(^x8mtwdk9f%=w*92qUmqIf`U~2a7#ZN(_OJ-Zb8`;QfwHP%VjzASq?BG)GD6?^7 z;mtpswpDjjC7*o9SX8D^c+9Sx(JhELZ<5pj&iV`4m60W3a0&*$zQUwk>SAmx^!jx%($T4zTn;x3pZtM<>!EG>6nJ-aC&x7%o!?Eg1BdtXVN+ zV2&}ub%4f%qHUkVY<~?ju)n{5!GmpEw-N&k;_IrHbRAMVaf9sGjAqdJ!BsNC#G|t+ zh6|ty2~E>&pSlYdGZM@xID(11+1Y-lb(?2@?PllTW?TeDAU@moX4DdU6O?z$C;Kg9 z|DOT>tFcjuhe>EHjX9$KrNmiJj}Oeches2LUkR%6$mHZhACD$dpUb%H_)l7yvkGTbLVyFtxKB;$UYCW zRFc(%$}T6^V}*>(ERmo&u{cm{W9k5=YX8w_xTS-fx_}!?cWxmn&E3T822%m> zJEiT{x@$4vKm_64+z`3T3&H4}QDUjSk@b&F#Uljqt;W~_0xxBFe*knMfd{WO@ zm|ajPLQ}ZE?W`zYhHGw&(acUBbii~D_%w$cb5{mV4PBpv#fc--roL-=UkWa zTlWmDTLSsm#1((<-rb1oLZ1zfm9>w&$O_tzRu7&ogu#kKfU~_1_R`&M2OFrpeXW-+ zTv!nL2iVTnVKVy4Mk&8u=MeKig?y-=I?luCb4UHPm~HvOwoaC7t7%wMc^6ZH8tGDa zn^=#_3B(FWY~G*rLFBziNi6~xC}ir=>Glv4$rm9%gc9BI>BR)V3REIFm8}1o6q( ztOxizJ7t-y3p&L;6f%^8nrk-Y?*02!)X5^MjSMTZw+bJ9U(_u)I=c_yQ$irk^MLxF zqoP0SL@M(q3=cV3H&Y#-O#WxTnfS3SfHD3mAUT=3r`1wc{(sJUg0`@L>=owHJe^sR z?eqd(>n4FA{8EKca+%HJNX6{yJH5JmGB94i#P@B~A5jFbg)<>i^%(O*Cy7N*X_&(A z92FF24q1VEjhcy*a95kAw;~J-qkOp&1XIqY410;82@_ z(KN~}q}z5KZ>pqaTKBXq=&&R z11^NkdhuN>dPzX?N_Ju4OwUn?lTWFXZxdbx!X0DlYN28z*%WYTYJoI$`pJ9rsT3-+ z+Ym%;lKb^V9eMY8uV0@o!tQ{09^L;n7|dxLWw=Xsz+?n2NynibMO%d)WZSr*_q<_R z$QY%ynfl`SBmt%r6a=vTw_j|TmOIwMS(nOD*~euqui*KuYr>~M_-~V{i%LqaiFApM zu~Y9DJ|8;-j;t_z-ZkjTUzkqaBOhjSy|Fdn*dJ2@46wqY!U@-q@?Z1DLZUXXD9ftW$aX>Zt$Kh}Z!u7OfnC=ad9^PZ+Zl;mOB zfb#eu+DrrpbX{;TP4BZyrJtMoY;gZZR1YZ?W;QFc&YO6%&QV`zcjwym)=x(tJlafB zQs36w0bmCi;VR%1r%8Iz3UAI%Cr0MxOy>i(PD@tT(5`%8aXji47M4H4-we^8I|u_r zR6AB;$P&sv(=*0#1;*A(Xyfm#`r+x-3SNjFg5e7!V%QCtDVlK~_Mvc8Mz+KJm2#dW zAP|E{NGbPu)T{2>GeL89xBf|l+c zdR(u3ejzPnyt%qh0_TTA%b@GxXT4Tx-D3fR;~y40E2=GhN4rrfSopEzVD}M1b2p$e zZIh&0ehlDBoiOw6K8qj?f+Lt9!3hk4*C=2&!T-DNm1htaNnFfpZD}zk`t<5*?Gcur zSThQJv~I*1gOpe#5t)P;P%ihHtY>lAriK6J0>HotxGrXLoG5y#^fN)w2VlBOd4|IJ z%%&2&fT83fqaW}?-Iufmv$hlbx42jGELrjj+ZsLb6Gm1V z6N&{nXi4{4>Ft?#2S6OnnBK5i3ZR#lRl^+hfif3VsT zFmGG$AICEx)pPm{Ml_#75^f{J3wfdcn5 zh7vKl11V2duEqf1}LU^RrcXZ-?M&hF}H7HI2=8`(C()Wvj4Mt<2KOK4;q zKM9YCf&wpNykJ4x#M{g4;6ax0dR8gFp{x{}*W`eAe_ZGP&E(MT_W$wZaLnhb7Q?f^ z3MUu>K#0hfFiU`KT!e!Vs@vI@n;rXh;+8f7odvo|MI5>EW1pucF}e7pXJOOAj8F;b z$)&~b;0jD4=t$TcMvU`M$aDfShtzv661ZVWnr|BBysnY25?3+7>ERALqkm}Lk?$#I zY}T+IAR z>`R$!h*Uj*;dC-z)c00(X<#uQikjii2B`BBX@%bMY3Cte1+$(2I7Z-bS-nX z-XoyIl-MS3f)hA?65Jo+d8G)$T{7VW29sz=TM4*bIzHz>QcQjG$5GQAgF2=3w|7Q) zQp*&g<#Kr=6Xa=c;=(>KXdz`W#(h@CMYSELS(1eYlq4U0Cs-T7c*ybsoutVghC9LV zw<-^uMtd;l(c>KNX~^=2Bc2L+V`2xj|IdvTx~Db(zCN0GEZmN***^t!DbXsHH-;L}<2xrAO`hnlPRd#`6kH9J%QYhD$k~Dhb5_ z76tXTFPI@QOO)_h4kEz>!8@HnItJlbE}i*zZ{Jo1iDTNXzPw|9AwqeduicrLY?q*h zgY0(%^PMeQwh&oPFRE=ds6p4DytKY_DH@H-5ho|PdHmZo)PlPNp3&{LC~;n_sjjX7 z;pc5(;dRvRDypiLAZuh~axZVTUZ&@H#KL7SZZj5a1k`{}tpbcf=BkjmJpZHPMu2!V zEots>Scv<}14gh}khZMZ==$vkh&J7Fe<|JP%>}0KIe{bmh6P>1)ih1!YHfwMu{QqZ zHiVO`EXdEd*&cQWQ(R1klZGci2_o>E@|qP`5s1e2Zo>61+@M z>H3{?7v)N(#1gzq9rZt9Y~XwkfuHzb0RVT&gT#QU6O)HR>AOo!{0&chVimq?Mu#;( zrwa;2;1QSKTDu=3Q%v6TA{sPX4J}yrTbL?n1>vJgM<-pjqbT7kS`*kiaK6Fs`Ic zRHrz|gqRlkl+%DWfkpQcpSvNh;Aq%P9H4iQ1cA+Xwvstw!ydI6-|5%Qwyq76EWAU1 zNBNAi+@AQ|Vhpu%HkqiVQYTvlh#@~|bWgnrIKR_~E;JrY9p@pjYb50sP;AbF`5t%G z0AjhI{CN%eHg?_46*t~iBUK*AE>Tgax;`a-;3AHFg9?&EQhZ#H6m`Tu^Vdm1uA z5D6>puQO$NwNq>8boU=$HTAn`MHR=UnooenaLLaHXlJK?7j3(mn%WiU^I^u83M-yi zt@w`bI+nz50>)iK$6S75njcG4o>m$o8IA~EF&~O zLI;t8(|RSS)SO*Leb_2 zcm&{sNmKS^;2`-h>@Qv(jIpPtIaYf(?&krn13ya7936*T5yu>(6*$rGn3WfWsCxC(tqG$-B2KxRB z7D_5%|0AWeXQb|bU@HI4Zk-KD=93n5ysD^eN%kOqK4K@2^ZQAwRHln&`~GU${~~V6 zoF*Ic2b;D$w>(z1P=Wgk>?Y9fx@BeAWo2+tNN6dXes=+^!D*S1EPXTH@HcH%CiU&F zr@Au!C5jHnF$aZ`vux^=Da1`Q zZxEzS5{`t@;g*moyBF%!a@6n{1|{#`M{U(!SS^nZ*(yw7-MZ_5#)r-)RG6tH9X*1v zTYo)HQ?+mn4HXslM0>_?_a1l~KkIjb>KySkDH#Wqm8XG5__p4VF$VCA5J4BdC(Zj0 z+(IwqegqzpPqhNLF>u)S@-%~+OZv%IQ-p*_h!Z$+Nz4#gyBoOgz>rn%9{$eDMt=jp z2B|5#@Yx=rQi(gm&0B@h3Rr_fpoh&VQ-F&apMr}Vx9wTh+BIu5PM&<&#M|~xL22C2 z3uW|JfUJwGwUg1$ndRNKED6J0-++JxpeMt5YZYY39V0U&tp|JxB!Z}@kqD^^;p&7K z4Vmv8b5!)P2??*^&)~kl1UbvQmEnXR=*z^MFc!RIP}MW)V=^Gz0v*j$W ztQ2x={`EVU>{<~~QEs@LVvJzf%f>067uBA`_GRkUj`U(c0vfDIgZho+*f6Zv z>PwSNPcKeIHph}5WmyBD0UXV|kSfD&>@nK*^SW+n=f*+$R9N~U^ZXSXri0?1mOkH1 zMyD6O$47 zK))nv0$3s0tch+t9j)~06|FRFEGIDgPZu=>^{&ox(VY8UVMBCRbLlNCQT<=zy<;8b zq3Oim09mb9fUB&6eI#+_!g-F0a}zR4aEmKORuC2uu%wzgUBp!rV_Eo{!}<^=%X86e z>NOCC61aC&4hJpR|3O_$XNjol{~O^^Vsi79#>wj~!GTmP$L&bTA~_#$=Wqgwnb$so z@dA;+06=z!6$SCb1o)u@m&HSKC+cTX0Dif_?(j2s&~Hjh3qev#MH>^^I|P^mWsV0} z&BTWattIi(o+;q;FvIWP`IS%QnirD^{hf)aFR`Ff?fMMB4qo0EolwXIm9M`>---;o z!4?EhRDeW*0PHQkK@)A*bDsH#_a4=nw}DoOt?u{NbrYz^lXVB``lMs2*FeOB#YzQ6 z@SUKgf@wsYbI6AAqv^1$WEfEAY z@d5!lbcq4J_dnJ7UY?#0fu(h}1vR+*14%Ls9z*;C{r#&jvCMcFLFf&Ji7Ytaw6(TE zCIP#_ho~+9Bl|9sT95QQ;KZ=r9vB|3))zlqdxH&hy9!JOGh{xUGB@7~=l}u(6&#A2 zc>OmtcyxYK{1b?E-XLB^;ENY8dM2PLSVRe>0+G{`qBtR^$nF@cx7OKY5yxXB1bk{%E_h?Y*S51?6^*L!?TT+r+&WPlv-s z508h5Y1A{>y|L3Do_YAJ>Gqf0?2Tv7(fcdcHp)8lDP^7O{&wL&^0)6NZcozv?KKVN zC+HZuGl~Qv+uts_4c8F7m7kXfltF?|YSTO0=gc)YKYKJtTw|nb9_8VmXVyFFw^ToK z-XvbIvXv`VQoef5q7mJbdGb;RK3ZMpNNbyE(%msLEX|azr!;k)j@HRe8Y`hE`)iJa z5daY*i*sTwjL#f$w6Q5)a~wIpH8nd;9rahZUY6C|lEt<15fV;xSo{_C58}*^^EOl9CFnHcJ`R(P#n5J+)l4@d?Z9kuk~1fq|1d z?H`xQ-$oQs50$;mz;Tin-y>3+rEnXd5PurEX5*NWU})fSq*X=-6lr_ZaK zu13iiz;D&jQ~dvEd-JfE|F(U6vZhdpq>w}@t&*i^qK%4nEhI(S5D`i>i54o+o~@#^ z7bz4P(xQdZN|v zyv9@qXY8v_`#2O&&a8UFaguw2`;_I_0M9_UgyES^t{g9~p;s9e@(**u-i`;o6y6YD zU7z}7lb7=Fg$p(ty|>JAX->?0lZ~P536q6y#|A>9O0}kxH|*Il>2^e#!TxdP$(C~) zTuZO`ePel@O}bYhD?-_SDNt?Op;hEBe=AU)yGw6jt^d)M`AK)hT&JcC###depB%ch z+dJB>^7zm8&};3z+=`K0KUB_~PMLb^arFjwWH^7RRbg<;QtB1&Z2Ek^*dH}rQ4Gn2 zT}np;8UFrlHr|DO$~!_tAa&|&^G&CXPhZ~h?9#TiW^Dt;$#;GQ^4&*!xrdn)7?RBh zTY!DN!oJU(qECDCUD$8hVs&-9&e^5p;)ST?CugkARFL~huje4ka<5w2e0VV#_&%;j!-k%rwD9tjD zjPlp^^V1g***qu0yx;0Z z7kQPLPcI#9%H|Fg#=GN6TCnHs+xxC3&TmiE_$()(NBL^F?fh<5qg?Te zI{rDx_A#F)wCd@Up`#;by#sH`y8|OMmVH2-@Y=P8LL#B;WwK7QUq zSDDSTvh1Vq%xl-WRE`SGk~UxQ^oMP`_SR%fw#Ww=Dq&cG-M{R54MPwY}3jkfI@ z537)S%v|zk)AhNF_3Cg>icQM^y)UwNPE`+gK4H;WAS4@`HMO4N zK6h@@o4C;P1J9=GYHLT7mbIVOxqR$ajnb%d!MVk|&J47*@=#w?Rd5|cz4V>VGDS}# zc4_RWidJHeT*><(yEU}z1a*$r4z$0Q;JLT23u?C~P^zTodTyTVREf*?+pNa5rJylC z?ue&}Ip^dheogueql028sXKeKTVB+qi(h9A@uX;9S-9W$oclK8y=y-;ulRPtV#)R1 zo&!{O*JRtp6-SJ{2l&L)gyt=8Dex=n?2NxBZcb;UUCkP(WquLUnqrqNUmpMM@cM|z za#?nHHMQOFgu?Pgi`&aH_vX}{o41UsTCVU##AU8$3;53G(3obP18u&&pxtL=^j%{} zR#hEnOpop>2nrE7SQc=#OpUpBiBOztnVIfEF-m{9nCo!mQ5wZyo*<>chhx^kvtR1| zelc&Emj@nq#^s$}^fw z)8mmqgQI-P>Fv^6Q%vt)DpO8B_NBm8K|mxFr$EirpeW_VzD$4c{K!#9QQK-e<49@n z`Pd}80NeUxeKpLDTq+iJToDfVC@ERAxa&tsTxYhoA0PBZ>4yuo{nFBTTjcsErrt-%{bhhf2U;P&HoE%(n#M3i(*vxZm$v(48U&ub|y?-rZ*_UvM zzG-Y)teULPT}?YVwTbfeQ+oCtOd#i%eE3pLW2R?(lrKp0mYFwCe&Dn4a+NdN@uRP& zr5*oXDQ7rbnJMjx3kECCG#0JiA9WO78mkqTS$p!A{jILw-K?M}X zo*DA;dDVRjW2lA9%5N`i-M0VsS*_IvvOE}1l9o8Q&u!gvE&AHPua>(#ulYXd>GX6P z6ch*Tr%KA5W5zAsaIo|RyG`?^k(8g3s;}~W*|~qunGbhPp5t^}KS&CcIOWyXuU)6? z`dmkonRhl--(0@b3!_h-l7wo-l^;OT7Fz<)x%G{$f^c~@^Y z@85b^{2pEqlAIm?MaH~s&9#!H@z1FalsNQR+!njP9}C6W;r~s|q)$~PN)nPLd_{Xs zd1KtL=wGqh9P-B>SIWAE?f=1NCxYGl{{y>_i^iyn^H5&@Pp6>Zk)2B>aCjN?5HimM z{%}XRs*R0|zP7fCZgDwsq!_aWM38af!=mQ{q1w=y4#YhWZl564yaOj20bnGNHm72? z&;Qc%_~nA!#rc@p^>phhh_Mg@QRbZed=?_{;8AS>$Z_~1@|0hu7~Vbp>xY#^zt|$$ zziW4NB?{5B&6qI*Lpvc7T?zb;g93^g(iV#%MKk+(g@l~sSyu|q{UQ-d8~}fy1G0gV z(K4DHRN``Z99cIP;qNN)<;xeq!P=O^q@}0(jr<2e63>INf$rh|h~Yy2Be`q|n}IQV zQX5c^3m{wqaRcjw_$K4aIdZZ&a9rm}`Y-$R|Wr@(yb-eoyx z5I`wO$;$G#%q&H;%EjESE*}T28UJtr9556^*1>F8e80QI!|THSPqel1jfv^KptXU# zN=E8aQ=a76z*ZlM-9@#Pm6g}v8Vv5QD14|f!`TLcKHdsN&BMEVv-hP2Ybn(S6Kf?@ z|E&1^mi2F;aR4k^2NrVfexA8=6Oj_sI)O2H`RC78XP5uc{PdqmUj0YPqw!e5#2Yz? zV4G{7y&V~;i7~aw-o5F0RuI9>oHeVbZzGs*aD29Eosfw0+zx$Zw3RphY!|3tQ2%RX zWo4a$$mSL^C@<7}N+^U(C4q6y=>}*36k;3b(IC8^Xx9SiK4SN?qtGCQ-IiShNj@*7 zz+*J6%X_5rVQy|BUYI)@Z5CYuJ_;*WS`1ZQ(y#gtb^!kq!G6Lka&2m;F1&M~IS)}N zRh$%=eNV0m2@BhmhzEQca@k?l1g+CwB>Pg}n*S4FwrDrT>fJ+29Sia62UM_;{zZ&KfE2$dFW-4OKXY!0^`1Tc1qI6Fta##91Wyl*i$KLFYSER-{0^v;#O6ZMZD?U}3GBeW z7P_@P6*8k)7x?}^%lUgXAm{JZ+M?V1cv;OTAw@#M-|lcuB@}$3a+kjG?w;XgOP9){ zNPwNi0dQ`X?Z1c)l&S?V7Q+MLRDA1H>X2O0?~e&jS3<4Q7koj%sBla%v{>ZR;ZvjTP~gy>V`jAMJza)?5Adm1%IS^t-U|5ZuJOr4|!pyh5a zDD+zqvW&_Oeg5;x<(_8N_!&Wu0FL^=m3MqEznrD>E+xjt_R?irmrGa2>+N^>+HQ<` zZ+q#I`CV}*X{uP4`d!0%^E-xj)|jtcAYAOfX@%5%8g1~Ks()<$uepkx;@=+%I4ju4 zo2z`>wWuzabg|*Od$@Xpdy_Q4dp@1)u>M8cDoN$u%Iw$irM164pg=E z!(qO8UOFLQ(^cPzGiTd1guZG$zG>FvbE)-6|{VXEH&#ick<}YsvQkn-wik%`w)Gsg%$4B+_XT zB!ILyseL}WAGGyou)@Z;-v%}&gv~`<-N6;%!Hthf6!s5z!-Uy|0CQjDjJCs7)v%Hd zwj07*i8~TBdvczexB0;i4;Ijuf!wx(#Zi?#8Cet5$Kk3NS53 z!r=wXhfJUnl2?yH2i&d~(DY+2K#B~MAB46^EHUuuYIe9dOs1v`KYaL50gfo3iIU0v zONjU7LE<>o+qg?^(qw(49SsJ#@F!};ya;}2n2=d7tDUJkd1wj!R70F5^#QB0nfCkH z;t}7Ztt`4%Lc$@SaD>d&A!HM(Gn4c^^B|gs&-HA^NcUaw$FMd+%F`8aXvC1t31uir zn|`>`sB`)YVDmn}jpISYBpfw}ac9Khn?AIT3{HqML+j_`4JzMD*U;C)H41*xQG2HPrD?nDszsvZb#18GBB58K!7AnD?E4k-s-eCK@x)vyDE_9ZCqSJ z@u<0S@Ro_{5Mm&raVF*!*Y;PjA}tD`WdLKFSU!<>AjSBuz)S}J`S{r91x3*B8)jN6LG6iN z*?^OTR*8m}6Z;F%K4<{~5L*aO9|{E<71_r^G@N^Pde;!cCyZrg5oy!Olds_L!9jsg zgKy*k?4FQbcQqy^C91KcL^H6xuP+$O2g9dZsh?=RUb@jP`1&!BC9+z|koBs`3Jne2 zGYQtV^P;VXWozZVSSDj(Dd-s*W zy|U4Faq5sS5i9lLR7t5wEJBWuBPTv%>?B!QPs9EAC z!6?OT;@5&1IoUXtH$lYQ_vHAt{_5uDX2H>`9q1KElR#3Z(7k+xTY4GprG_?8cjkdj zr-^#dW27tLLDzL?#86z51S8CWx|x^i=?}d)R!B*4AuEqqIzwAolcc*C9A<9UV=b4H z*7Z~29S0UWKM4vUEGHN+*lf#(g!AQoOzgBUAoOcNV$g!&nog0%kx$eDzMnjURAHpT z&l$6Voydw7__$rSLV}+zqFqR0bD(kTSTTr8;1gJW^Dbqn4L(}7};7kHwxF)QqgkUKm0}rlS)134jk*?{}feK%|%<_ z&{Ft(#m~V(HPjm99^vIfek4>5@JwCE1k5%v>3(_mo*rq z%=7v9_)eU$Bq# zA!-{;`FSd76*F~yNc{pY3>*OsFlqb-D{^RdkEy9onbKp~4s(}~=bRZQI|&Xe3Hum_ z^_M(^Dik%?;lVKBWnxRO+7TuZ)}a2g1%CvY_2Imnv z&8jc_ohSaIYDWL>!^~xalfEVSR`HHt=~E-L z88O$u47zF(E|j)cs+HgJAsHrEL^wz@1Vbu0_*!u`5N7)67umT1FhRwxG)oWABf z?!v;ts>Ss$TAB)*@8{(B;?my#>}-PR+y1=|p6#B59}vkO)4sA)VTZjWd+OF%e&$_U zh^UD>KL(`1VQ~p2qhpZHMMn=`aWP^%Y}AkT0(5Zo)~(E6;cnU7siUtjtp!Qx?rG5$ zL{UPrnrHST@tcICyZfl(!h?@no}L+4iOvmA(IctOE-vkhH#s;DF0J1n19V|F5Qn_7 z9j2cO+_>QeNKQ85vw@j%$eA>(1et<0md`!%npJ|Jc2snA%SH0w1rPwqey?&w%Q=gJ z(4<1%4XA%nCXxmMT{Rw$%1CYdfKy8j-Vn*ka?RiOBxs)ic4)iXM{C*Wc4Be?w(i&N z?n_9co`Uwrqr0k2MDhtO_uyQ13P@)wF`)KEd4uNKXxAQ#v23aS?3JU6vpWCoz!)!= zmcn_IqNy!th!7i5RA0Z4&SpJL^j;KF&u9gmg+jqlTvTtAt}`pJT3_1LYE-V8p(PUB z;H|E%ZsX)cpaf!$f%B_j?Q_}5`;SD(mGLlj3E#@jiSUw86cv4GJ}R#a=U*{tkRs5Lk9S5}^CGqfb(X`;AkmVR4~2Lw z6mWiE!;v&_Fv=Vn_`i;2a!S>)z<4M^vpk4kC`6@kS{0mn2`YAJhDs1E@hj_HNCJyP zXTH%CL^wn5v_(>vT~5BUuHSl5v$bZA{oLRCBs%Qjdo#E1Ud4fYgrNa;x0;3?!&FJ9 z*GkChCx6h?t{>JE()r!odQJe`ry?l&NQme2QQn84CX?i9eS0Is@q-VnK5Y?Z0E)R8 zw?4o$>}$WL&cFgly647YH!Yu`tZ}kBK0Wf+?lg$kk?(f`#l^Sv2qB>O;5g^BNotXF zLG4Wb1j@xcQ9vlr7{6uXNY!jiF7kX|cA*X;AoFdI0G1sOuQ6?*t&&{JNAdVxv)>=>Y za0@CWjy3K_UzC;2MsH*3mXX#A;WP(@c%$%+b(Hf{R$DVAL$W3=X=bIfIxSivW~+b9 zM(cyh)E^D#t3OF`MN3v>I6UNW{^z-r-a{b#WEQP0yY^8;=!$zBw$&Qj(2_6|EbPR(hjHgy_$TiAfQ-;8LTS0#?u7j$u+qpg1Tr zxd}SR|8|SHL8bt~4En9g0dAL|QJk&RJp^{olVZP7}0kv|m zv*3~}@D>3%5V&bqW2P|f9!fl15OL5EK~N0$ImGY5`lY@IG?(>XLb5m!=zRr^IMMb9 zLSTowf&{2RR-hRe9vgcJOd)WH;JmpU{9v<6F$4?2*#`MAC3t(-uW6$Sj*_e^V|M31 zK6XvrF$o>nK`eP9G=q;0=)E-=7K$M%A=vK(e1VQ{JX%1{$<@ESa!&eaDLCGAImF|CCqr0s@JyWQGqKISnb? zTG|u>93c%$3stHK(R&IZ6-~yfAOfF3sRIIzTr?(ND{^y-QG{qol^G5=f&oEnBP1GZ z24ZZsa%F+n|Urjxa-E`tkiYEwq@=Be$n&kE{k)6`x{_oBRFJroZ z<-z&8y7>D_H>GywZih%_f?%OwI*;95+E7hb8XGmnGmLG}_v0N*xeXP@iU}3@&jlT^} zAKWK&*ih(-?gku13I*T=5D{{EdP^PB9%^BI%PjebP+Mi&aK`48OLz-hF-F~-z{8d{VR?Zzf zBls-)LnV(T#^saA3<`;}8-0k$24p_FZ*EFZJo;hCREmuQ3$-FqNZ?Q*IZqcSHvLx< zx_t_iM5L#ow5mM7-tnQ@fN4HrA4@ycU1m#Dm^<2nFr7I#V?i>VS(gC`2cf zap{9ustM`rBqevgW&<9l%|aR*C-Uq;F7 z9Rz&RBdxu65)owLu6CL~ZJ!@=@1K%;cAh4>r+)i$u7f)PzE4wf$Hw{qd<;_Bv>1Q+kf z$7i1tO$kRllDpo)XoQ#`N+4|wD|Fj+1E!jNlQ!ljHJ19jFF@Dv@v@Xt>2Ub+Dx)4M5niOd{R1*zZJ*dw^PYSB1# z;gPI@ODLm>&RSgzC)vAbHVfc&)@%)vC*JQs@au6cI5ng#Vxr32zDbwOcNZQ-BP6Hs zq8!|J0HakTvLSr$hN=ZiR0|b;B3Ruu_^pORh+B(CGs`_peh0V4%YE$2hah_ZuUsEk zNitw?bea4z=J$Ozdu8u`{<@_l&dAuf8%`w9-0wg&XY-Z@sS`Zmv9y+-yFuMyjaGcd-o#X=li!MON5s3DJ zNWsZ@LE}U|N*JOCyVdF7O!LpG2EQbdhJziV3*DnzEMS1zkYN#y+90OSeh2&7aJC6s z!JnO!Ma^fs9Sok30s9{vu}0-PH>TkpnoZqGL~B9xuoL4*F?>>7fp_UFhVG7Wm=z%u zji1C*1qsMFpfN<@66ej3#8fE^*EhU0Yf*QflG+3dNT;T(TZu1&&sTKeEiWh{FGxx@ z7YvZBY0SJqa1Lp!C?g1}$$Li3l-=s?NFxVt;nJl?5`|2>;Dh1_mrc0moP)gZF3=w( zHdynuaP>_A`bQQElmm-9kdYnL>~pUTvWAYfwxWzzpbMv9*N|&k$*DeGUv)xbm+T`) zc$8~{=P!W|L|zZ0#&Dz5^*T%`p@w)jxbuh}L6>Oq_?T7e8Avf8j0^ns^MifdJFIre z4I83LLq+5+J{%EIApT7#_*dQun@3cF{D$QkO%S<@Gfm-BVuDoP0s8JVt8IBzjg zT91IT)?cuO!*S_R)s5n07dl~nZyF3gOdcNIb#DrGi<)R`&dY#mO`^C{_%DlAS#lRl*nVDliO3co%Jdk5d12V`J3*LQlzk zxu<&ScfQ{tGjPvu1Dp5K$KNM(^mSVC$l8zy=Dc0mZ)n&SNswkby}L8>^OF-7a8d3C zp*#`4mA8xAYBW42yIwO>mEjD%br7WnvpB!|I5w{Wwi~?R=KbZq&2$t_2KE-s&2bMJ z?KPTvtHapXKszB-f#o_LQZTp&jsWYzI#cBn+4qlG-XEcJk-EoS>M zx90a!&udaAVUR?H3pCLO5-0LCzXn_$fCbNu|E(n;)9AZ4f+wMkBSGYV8_vTd(Xm+c zx7QF~V6&WbCtigoUAD;*| z-m~EdT?TWOnIDKl3kDuICW(nFX(vz#QYf$X%Q>LsBI_`GKH+~-fv&n?EggtvdqS~3F0&U%-iKD)e$ zTFlV5=xNFhr6S}8rNI-AQ7K{;fJWyNXxcyuHeP3$Mgn+&4})p{+>2Q6c(B6gAM7f` zK#jnwLI#2IVLA?`>{k5m>3mG!!EUG{Hv(OFQ_M32Q5O z9(wUQY};})HwX-v;f`x+{q0=%W`zz;jt{jTvbEg~Vc_=nFmBLQ2}#c<=5-LGffo^_w?1h_oOhxl1apP!m#O;1E^USba2jbkQBCE^E-3 z$p~rp?c_$S+Kilp!E?Yfq26|gV3@8?yR`=gR#;`{3?Fp5UZ_~N--HfvTKpKs-m6~| z+ZKGt2%Xi4O$^8T#D)q9e9s6KY@xBYqNRU_sYxn+G^q;Ujv$Kj5lI7@1-d+qiN z)*_G?A9GTGQC|TTqK~s7kzC4@Z)L)fI;-8fYyI@}^j;SiUxbxGYJ9wpBnhDAr+5s0 zg}bR2>c>^ll{;mljo~Ee*D&cO+TdMLM}2(c!^%&FHnD6}hxf*mTkqSvB1V7fowm*+ zGfiR{h#bxYIKYthuc5<8-*QJtwymqcq<)P2wvpYz5mRm?4R@N_uYndbZu=!7Nu?qR z?3CLc06>!WJP%gzjw&=p$Ezr`@z_#eGVcJ!*0)7|-_3-?e-0%nIhorgO=0ck%@$AU zJm3%Rg#P;ImuD9JyL)jstY5COhZ*TDWFAdR?6XhtHgb*ha+|Q$p?dYXICwmekiiDu zhB7P}a3gtTES(xts^p{)Bhk-f65%!CSPMe9ZWb+EScAKq6kn7YJeSeyaX^wZC|xM& zgdWv3u!0|`@~2as*;M$i5a&ezam4bvBNNatPj9$@EmJ##mC=47t5^RA?F4G!om(A6 zPm>CxuJ-8??1LgUfb$aUm*cRS4PA(A<;aORP8EznU zT>7#xu~NKh!~EfQ0YGdCAi7;;rcU!2oId0=;3#mR#|*`RYj=_yE_zL@hX;UT$e;|- z@J_?1qxE9fX^vZYNNfNKPQr%(9r%RZ`i3Dy=IrYn&6AKD z+*6HaTg`{*1vXv@&=YcapX#oiNqQp)z+!84|14~6O-({w!!@|M`LRnV_$#Sz;&3XT z@0q`%@<~Fbbx;V^pD|nBb1FDAv<%0dA&3rmSU{__j_z{MI~AfrCcW_Rw|sO&@**7I z5#8++A{Slh#~vLDRTw=OX*M=>a1GWBR)Rmi=VsZF)HsFj%TmWMc)X9(n`rIPQ)W9}L++vFa0ZAf+f2<_b)FI9a~{@($J(fhe&@VZ?2npsZ_Hz>IqAoz0C63YUku81DsSXUk2S zHtnJrr%%|+=P<*!tiy@l1|cd&`ZEk_*`2QWZ+CGYWoj>;C$#y}%1v zMj0F;MPunbZIb{C2uB3&+52zocHuJW*Uq4w?rogM$H({nnY$uLBKKkbrFV0aq0E){ zQ;1~tKhxs181A7L=yWRj(Y2m7DRTcAu!i?;=DX!mZIx!WW50g=(gyBG_#;qSVqQlw zs!3)}_>^0Gk3BBDBCSWVxgn&PlVxN1khuxkqD3-wHA4^aAH-D@AXn>;A6W`5k&7Dy zNlh!Nm#2`DlY03tNbeN4nhoElQ{GO;5gM?a+R!N?Ik_ZFBz<2r+k&}w5t)L$%~v;L zCibQGTW{!?JkW{YQTGayM1U!{ac8{PKuzy4cw`kZzfMV2Fh}*^ z`6ObNlh?aNU~WIa0vV=qpb5KnR_}OxFT*%)^aJ%ByLnCZl=!UP(SYp<2JDV*Ob0Hq zv-jmsL-YUS>C?LqX38rns%vTauaKLJ@RJI;OfZp7R{-4(raACm#=Y2?a{B~a@_e(i zv)=}hjCTC_H~p1h%En^=m9qD-ERZrsJ=2!6m~0ZP8sLI+d!lpkyJh=No;(>78|&B?2JCBFf^#0net)Y`Q)wQ;zWkNBl$kCAi!oMOA=jx2HJS?oABPh&oW(x z9_8%0bHprh`+Ky%Xs)&bz)vrE4u|gMWWDoYXJk!P*ixt+0y$v^8a9EX^1tBxB7>)C z+8@Rc@q+jxI0an7zbAodK8$^(%52z_@L~zR!g3-*o4ZycrZdn?!|Z3lw*zPp^YVY- zkDP{*O3f5D_l|dgj{v%6D_la8sj%6jghqbUQ`x*=uSH zzDb;^Q1Mnq)ELTt*fMNz`Zzi=LI?uj9mrjTFRu$Ax`s`4uRncSisPXaE!4&PD52YM zw_37EibZkVpS2gK>Uy-Rx1}NxN3aLAb(Nhp3HGtDNVVeGf_`vQuO?UH7)QjXm|UEE z81{ssM8^Wu%btKYY9$HM0zRcQa$uQ=l!q65=;!rQ8RL()FttV4>1({0<;-lV|67-& zcfT0w;2+=?BJ*{0gRs9Q5hxe6!1eS4=jH$meu;qId%Sga2**}jUKl876Z(()@FDol z6(Lj&+Eox-H9-+Uc=bHA42y98spFS(v+k?nN<~>r!H7K&cK{w?LkIpo%1vzy8UPvc zV41dFGlSg}VNKzVpy&wX5o>l_r^IXy4F#bGliC@Uv7GUV)1$oYzQtX}OaNJI@*p8$ zSW3DgV&()`ABMHNDQW=Q?t@f7*kG6w!K7OqW1ZN_@$vT12MFp27Ei5!oQ0uDsCQw8fSs3+ zNMRzjAFoC1P_Yd~i3u2y*WyaL4-XW8rBI5up@$Usu`j681@7N~M{EJypc{Naj}Kc)07{(BPh0G%#I3 zd{DK5eZVTmu#WKkQB=-36L1~Dh-9V@Ttovw42teb(`(?2?TF-`+w`GT>L!*g!oWy>-*Y()L#@cf~LfC4-4H)h;v%W7(( zm)^Vc-Ar=wkxR>RsP6f0Ps1qp_ju~w8^x%xYcVl_Xnb+@%m$)YhOL$T=u)*ZH4YRc zT!3ch`qljE#|?zM8tpqR;v6blcyXdL#?8`bk(w23X@WJ>3a+DbtQI!4>qI8|c30P& zuF-%}j^Q+QS&W#vTiC3Pc@FP6bM=N`yk9&gc^3;^bVhvVV5xSNiPkx$xY==T;hqM_@em zj^e0@B>GmVuI(D=?$&sIzoE%Sso~tUoR2TT@QUi}6KOqcWgOUdV97#E-#2gCR6}Po zU8JO?>jS?dvYQI=ssrMG`i*&S--l-Rwv%a@e{jjL#B7dBO{ zbmO(}#yliCaE*$}4bl~(j3r+5@JDFaR8w=6!N%EWlDvD_x{Vu?agNsFLTTv8`1ugr zP-Y988TZyTH;<~`WS?Zi?6CVVWx8Y8Az3Z=NuFy@XrtZi&LL{$KeG>Ds1A`d=4ew` zlZf4DG+*SF>Lm*n_@W6yG|%4|;K*GxEu@4(FCevd(e9aQ8l;UN)-muBy9)Ep;M|HMVQYm#(ZvDgI9OdA6l0W#?gdX3nD?4^_I40?&RnzD^rYfD{+5* zoPAu13s(#7;>(3O|NLtXTCMNz+2nMG4>_v=L|#Fq+qI?q=HG#(vFgQL-1C#ujK=CN z*u}yKxA{Bg=05{R){ntazB1mklN0to?@h$HSpGB2{5}0l4I5PacQPew&kX_4`wn@; zorUiUuGe$(gqRPn=1_m}kk?U||4G%9|4ZTKrJn~`h7XQU?veFmR&hLGuM(|gg&b}m z_CR%y_vvH)fXbRIXG#|?QK1$ynH#@w3#@t#SV$ii_doaphDb2E;5*6E(15`+$@r?M zs;W=@4lv|3tSXSDk|OYSc-cwQCuo4l@EZFl_K4!Su_I|_*|)XM;%b+amEA3qk&+S! zYapa844Cwi0p)t!LaxV;uYhe9fe8T3@V{dhURa72)>8z?VR zfQ50M8)}>B?%`33q18os$w}ch5d~&qPxc+WHZ}uXhHB8vUL34#5ryw<5}oVDPf=%JYw1ExjhS(qy6 z?vPt;uNm!CT)HueacLsj&fBJ&Zrg7xwN;n>Zf5V)>t@=iHzJ)!Thhkd0kVqrp=NVp zYFDV*_E!q_UZ@bq(Eu8lHmKkxPfs7czRr#d$Es8D+JPxujZhGm~B1 zw^qsf!HAstaI|j;HK}qVhYxlIDIJkVG)?);k6>KBTy(Qc%b5bWmSKfrOh5bdv zC-x(79H^MUAu4NJ40ri)P>RrxE&{$mnz{rt7S74D@Pc`Oy6M&lGR(!L3>%~rzw{%# z0s`XoG-Ss(oxnisDm?yCy%v?1&%sP=+v(Gf-sYHK05AuBP%i)-UAzz18X|KbG(yWy zxI8XYoT%CX1cn%9;!W71a3OBqDB$Yb2yF}=<{sa-0)jwrL_5MH7Zepvd=q}=IboDJ z_xG}qH1DC|p`%^qM`Q8|YgTFaJ$$yw3xbh=dC_#R^BQ#nfIY5&MH?C!5`W_H>qn;9 zg#XG&dJLNenj-Og1B6eioI*D19Qgq(Qn6H|q&Zrladu<~%waIL2q2jKOW&EKD8>*#sV(vhtJ7qQ{}|5$R@pl5 zE+O_fBQ`s(B!#Qj{$N#{(P(s7P$tGHij7lZdERwoWwZ$Pp0i>dw7}Y^b_=o2V5UEt zc(}4t#B@UF;~2Du@c`Bo`mZ{uyKPkaxO?d09R8S}D8mzgp?#Y&MIP39+%PGMK;5h@ zpPtU#?T&)1kpc3qA@=d$B%bjMAE4i&d*-cNDJ3g=5BuaDdU%P8Ft-c`S-Aek7-4GT z?(f2*sdY|%@_$T+`YxXCslNpNmbwO`e<42~K+9Nw(48F}HK^SP4+ccO6h8e5R_ADF zjJMIK+0efa;GCZtZw9(W%xzNbE%1-MaJ>`_V2FH;QUg$HRb_OVNYFlMngIJ(*Sia_ zU{6;Z$mzEh34W()Dp*Ue?4?QOJ0(v*V&Uug&%Ez(3=`&B<-mcF&21%{uiBLp`hHy8 zl-kJBPW1A&*$I1Ui(28~M861@5rgkRwMeI93xRb%XvdA2ei?mIl@PA3S^j;>-ggqLZCcV~GGmrM8_!IJM77|@3rf*j%)NSZy>T?~IBtb=zUg!WzUUcYcVQ`CtT`9ume&&TW7cU^?ADE+GqJd7{BRj((l9c%{iwCATsABzX*GVb4RxKRUXT1Q(N%FR# zRWQIwjW|SevD-1sq>j!}9%m+4su1a3#^JX@LBVi+^&cv7Pey+K zYq;`)39DdAUt`m$E?!KgO|x8+n&}E3lSTW>VpxmA5$*JM{KhMzPI!-C5*>%NrE9rG znY|aotb1rzns)}P!Oz4ablCEj4V5~D6+69&-lAiiRP zR171>f`h0y37-hd>J?C|`qJ|_b8W#HA|7>QR7U;*`YZ)8xlt%hVtE4r7As<{=U#l z1uX&z3_;5aerQ0sx6xSib}1&v@h6aH18JZLaf#l2OW6-|3+`dy1th68#p>yXLI?qt z!rvG(o0s^rh6;r}>@}a+Bg?2G%UUKF7W{?Y-d;HG03wgdWqZxFqD9R<+PhJ`>3cz~ zaDl4gDQ{Wb4?{sov?XCQ_fqPEQKh;vGaU6nCnfNBp4$(kkHadkJf-DwdSPiUD+SVV|rvU=PH- ztZc2<6OvQX6byBT23yrBt3zL;88F%S&sH`b=gaD99wJo$ zqBbg}E*@@!;G6C0yJ4^l1UpdTFb#}y*zwZh4;?5({tC>9r=vyzP|^l+CMaixjavwo zOz7aMKRc-4gOj=$GkhNqY{v##()U ze+6)oLonGZ>M>KWjfOhV5dK~&lojjOF9azKNCVijKx!Xa3R#VXS9Xpwdb5eKAyMYR zv|kg?gg}9ms}T>cRO4>x8^5>RjA_n@6Ex~ZkW50SurmF@n8>1nK5oGcs)!YLnirj; z570*AWAsAlv8TX{Pz@1afP;c^u)Yn)4HpJvB(2@=KJM&RHey`z#&P};salpmW4f6f za%u(D%I_xHO}3z9BT7&d1P9UU#Gndap?HWDiPDi10edr9KjYY(r!DhcS3}fJBFvDZ zpWi$f$|ka2A-O#H=0k3rRG^Z9sDzEiTrXw zmZb8KIX>v<4j&(7Hg2*Viw+a+iK~ssq=6^^#=%TruoO70WD-0CtUkwG*+;iuTM1G= zx(l$t6^MB&Ncmq;ykK0_^wiVK=1mJP8(@9fbN=+<@cQxi=G-;A+&|j8e++%V%-)5i zZV9py(S-)UtQSi?S-i=P8h^+W>W;No)}go*?_y31XZ7IYHmk+$m&GM9qO*o%BqRb- zy_5K!a=$L>)pH4+2$^X$CKDps-VT7NCUGaY=!<1Z8=If4Yz%e1eJ8yL);E?lAu00_+mb%)#xsqd8y^rqG$KMq}YQ^o6$F zaa+&%<3#2<%1^<4aSg=IE?bDm;uDWXv*<^t@`<#r9$NE;4S69Vj9^+)Yw!03hgB%gYNd?*|BMX*;fYQmzT(6&9=oV{*6EtP?*#Y}A9M~0=TYF)3W zH9R+=m(OL%?0*)sntL8hyVsw7he1?V zQkn|IM!}$waBwtW3Xf={O1tSGf-^eK|GiapnKvc+Fx^ZqS^(bdAiG%b0NfN&h2`$eX`B$#Bg=hvwYW!KV8*>Xh34mYDvB z&-%;Wvzo*a2&n;TAAvqeT`qK{;M50wgrtiREd-{`Hn=6sa~((?;_A+f_J8E( z&=+AlsY^XKLCNo_{(WO3Db;J;)>Mz1)4jbCjshv-ZiP?06RqIDgE-8s)~#D7kJ=2q zwkR?UCw41QpyHi}noJ#^JCGBAljeyN4=k7R^N)7weeipXS|4AAJZNE6*Ohe54+<)? zb6hd@Yi*d{5ZAL)SiS#l(U&j#oi_zBrKYF8Y0Xhx|9fF%Fx8zrn`T;5qhB;tM?Uo4 z@NG)cl%|lMHOkXF<)~MY)iT#AE+}bZBSX*0JId1fa)`Apv$AHYQjz7j*E+p|b~HO6 zVl@lV>A7M9Yp`YJq74$V3bJvVN0BBFV(w8D~OVco#|`)H_$3KhLp3PHPb|SOGvRUe3n?*8GqB;rZf2 zfWWa{$i-i~NOm?kUZR_*sO+U8Nd_^IG?K6~f4EzBeEa6~M)8lXqtiofAw_T3F45>p z?6lRDO?fqkG~*bKKk0hn@YP;(df-b4NkFhFG?_dDlYYxdY5hrsdNz@hMn>N{gfg-8 z$}*A(=X~?n+sgp~v;QQ(XOD8rB;k7x{r$a1Cc%1_5x4MvTx0^1drHSs=B19A3GJEO ztfKNeL=AIB;+X=;Wb1raAR`Z8LBM;h)b9vVd2t$$8OW3Td=3f}=0$tlm*KMF=2rFk zX5BESH_dF8&Ru1C3WoOg?y>h;x}QCZ5|v207JD%?t5j`KDBDsMCZn~?m?gEC!C|CE z>{1-SkMju|jRQOBt;+F&KL-We`{m^HWeN-1RsO7rT9aaO5gErZqRUkMv#6Eo-J>@W zdQYZ~lv>6`YQ;_DOBa;NSuu63E52FZZc;DKqQ@Lo#pW+p*@Gg7H0r>_C=q&>(|*}m zgV=_rN(8i|vcbmpCsn`=mqNZ;kbH!R;1)4TEF3C6>m_ zoN`IdM6U|=+tn&F+316yn8yH;NA%E&=t{gXD6`hhz~EYF2oGiE)2MGC2VzWO$c-u~ zYX9ONWOW5iXzNfPFeFy3dV$#*W?%djnSaRp$1#AhiGPa;J|^}H`1$!nL_z`ND>eyJ zzz4l{`}P$Wi<1NkGAP2OjAg4?Y33zlTEQyl0IGOrBHByZ;szTdn*U32^AiK0kYO)R z&luGI%@PKMV`6r+J?!2(RorsF)drZGVURa7{-ZZmJqGHfu%G_={d)j@ba6>Z0zf-o ztE3L;-wQM#XSA-2ZfCQEUfe9vJD!}*zJH~~_Wp>7z}zJwZZB;`QpZ-$3SuIv{eKTs zRuB~T|Cn$Qi+ofLXhHZ0YXX=Hzz6%n?^7^QX}D)v()Glk3o@N zgD;EWNXKmM73c|2B5#M*!0+i!*ZigMmnuS441F}==n$VaCg=cy9t~M7_GC)Ww}#urWlO5?K~_W>Jpz~D z!HfyxxmLX!8%OhH%}EPzFgDKZgPCwMS=*I@6A-1k85=lx$wXL>{S@nD9;yRtFxNY3 zEhIjK=MITaqQRQiMMb^@>KJo>jVYAluD!q6x4JNjS7-mCv6GFfG5>U`i9fG(7Ry~} z_)1_PZ->9Gh3s@vxNZ}H3sU4Q@nf>S#KTfp@Q z(cj?s(8ZC;*Ud_dku3d#MWS zD6OWcqOkoLUOGwk(1r5e5J_V2R8AS*9+Md@H(u!oO=tFS77sipG{~S4!zjdg2?Oh| z0fgAl7Qs4n7#E=<*hRk5Na}@8I+;x&G1n3Oq*Lz&%*Qbwy#0p=*xwHyd7KbQC?O4S zNtA#3bQ2_aLi~=naU760)oMW%XoMjqsr9{DDT_D6H%}d!aH$k#ylL~8Hd-{ z2YZhX+S0T;3tFZC@iwMKM=%46`>Ty;4@M9uM@W#U><&3Q>w-PHP&NPh)vIbUmboid z-XZNN6Ahy;0HM2kwA`C`extJYgK2-DkMuT1Y|7NnWD}-E->V$&g?0LVq8RMtc^|Zh z3yRFv_mDg_q3`4^@KCOIe03u|_!?c-Jd0jm5nVMjuMOuRS`r9sq-vKWLwE&ggbs3j z&W2oPD*iY6zw}sK8n;W!(ksW#vQx~V5?fs8{p~n==c`k^-*5ODr50n;Db4SY3c>vc z4s3)@r*nN@-1!J*K5di#i;tnL$0tp!&Q5+ie(Kevc1cnH-u%H)#ehkDp(WSOw?wu7 z&*=o^eQ5kQ6CtR?7e-dY2qC+!cr)W=fAw#DR>050FPmIwE@*`;UIzxw-|4yb2Ie7O zOq7QM%Ky=J-P|Z}_okzb)9lxY^k@I#KS72g!KL>!9c)CE7bJ9E;t#td<4n0N5}Po( zL1Mf(R=0HH#WB>Q3lhZmj-2uo9@}-NF~x9@>Sa0pxvk6a?2awpM#P8CWQ!#Cyu+_; zg0Bvhar_;*z}#Y{WnAufLyO4w?b}a@=Zwx0dYiNDMbzW3N>9Y4k7N$^>?~3B^(t-@ zpzgIWby#8=s~qQjxiMNcLET%`S8?6f&2CbMRwi@OQ(xnn8XiA!YL&11XisG3{O|MH z9-r<%e$V1z#hwd)hd-}c1^J%4XcsoXb>??Mg zp@N#Gnqi?r1M_=Z^K+FSa2~Xml)QhehVo_Y9-sZsi!vvEuHCD2*T}g_&?oLe(Vr*$ z`zLgrxaXHQ+u%&_=4WM%DaKr_t`{( z`r!V;r83^ECHf8`f|> zHp|j@dx`(aWACjw8(jR+x>YeQ86F%wa5meV!`Wp(CmnL*ru2a83JOm)AM`iu`Ii*~ zKZZxUxcTF|xn}mv&*0AYOX8K@s2C6%sZ<5h_F)^amo|5}F45~)_qx;PQv}mz_enFe z_3UnmHF8!(u~9NZlam9liffIOTWP5(UdlI{^IrMAy>){1`AjRXU|@lweOgIs<72s6 zc-7>jue}Vt)~)jJnHKmzL~^W^9s3bt&14#mk4SpbC$HptQc_e8P>g2J*C2m6LZ{w3 zN8_rJdBMWN$BvCFt)RqmFTQhFFIbCVc~`5oy0db~86?dI_kDhWI9|EV?No8PcIZg0 z{^ET5>Y_VZ^KuK%^1YeA)CW(*(t2y^YeqR1p2MqIpT;}*7fMEIr7r##4da(1G5dM0 zl~d4$nMjF~0Z2oo?Y`STxw3ChJ3F(Z#n!mkt7FH)PX&JIceUo&#vf&r}amH+nTEK)Immrkm4*kHiMf7dD1lndH=pHUg^6|4Gc|R$Il=0 zy7M*NEI3(G?TgY1x#=rPe{Mc1v{CCVZR64r&13en6&EkYa#&Clk(w&+cIskmcsTD^ z+t|@o_8t4`0}Sifq=4BSj~*3qTwQjlMs?%jtK|o}TVG~qL$S=srqbQ{QNo)SJbjh3 zM+St;_^AQ^Ih2m1drpaFM>#30h3DWMm^Wr+3Y1lSSmKRJ;dRVL9{sTJhC?oF>bWz^ z1xg}+@JxMFQ@{IW(CSi+-|eR?NE~ zjq*bg#fSD~>2}PYb6t1AuJy&+&ZiiPuG+?@pyqQag_G+ulf{y+uNg6#`ME}R;Jf?T z+dZmoJ?l*Uyz@GL&fymuhZ_#rRwUU z19h|8E%h%?g>7Kv(nXB?S(Gk3i6OiEseQvI((PkwrR zU+(3RoWgmmno6U~Pmi`YIH{PDG;>e?a<4QXw=!(T;s`h@jgB@^D0XTTlhM(l%z34s zWaI)05;)%Si*r7fJLSJl-oX6F=2L5SBo%$`ZQ@vIpv&DC=fU-+p#k}ZcPehH2e8h4 zsOCxBb2s(>arfTgRQLZM_$j5*kd&lQ8bXpJrBIQKq9kM%x2#B_WE~}&D6}P6*_|>< zMn*Kr7DbW@nVH8KzsIX_f7bo|{rkJF-*w%e&vlQ|Iq&g&J)h6V%3nq~lCb~CxWDk- zu9u6_jjWsV#yY|tS*=o3?oxom#KFKoSMx7>-xoSg>7=q`ypThE?}I}2 z=lKd@@w%yX_gD{jf$%+Y$M?`Rh*ps_I*LH zU-07U+A3}|geO;sYd8tgxBr+!SKSlWUbEV@Xt10q%-Xk);@#~tqaC#D^{RVHSuaEf z%hC>8GumgQGct;plDDrZEIktx^f4mx!i`e@Slj{1hVMSyM_Q7&H5${-IW=vdcqcwS z+;b(kvU?F9CuQxeEj+ngvn6$kwyj=W;<3o9V)5 zKC*$=vBm~|v!h>(vKjGG3zFv)~5-0(msgaw}d!@l^9j_xtmM8ALZcZDLOvct_Bgc;jIFsJNoO^TS<-VPPQrIye zgza8h8d)*E&iODS!RyMTQoB-wWEe4}xJYIx)Qn)+TA^8ccj|Ki&Cq(d+1K*z63`oY#g$5?u;V&k{`e1K6rq zAW#%&3xc216O?~nB!#ELrZ$(5d7f<;3Th@-|85nP<*Qfkb_L1VbbMlBRwgWsh=(2) zOgocxMF86X>#J>cKvp?OkbWed7encJ%%bfodMyrfB!xQ8%yHMhn-(_P*u}-)JCrmo}L)uyn1u< zToCZVbaczyCB391j=3WKOi%Y=_G##iW3X&n)fql;ULo?Sdq zUhXf(6t!e;$YBd@b(#h~6+}auKy3&IITuDOD>iOi04vzb=0|?MENQ9V6l`da?UT%- z&70k%%HG=H?hs2aMKoK)9tCKPZMLCR0U^qOBaN*&klFmo#Z-$p8Q}i= z5QDFfgb{~eDL2EHR&!Pp=Pwf zmA*7qAh_nRO2$r}2Fe1kYh}?Q+TaIMPfrC9cRs|9Q%WniY4l*hnE0$BvBAJl?t^rY zL{{Xm^>lTYW5o!dcL#L#amcuD@4UXlhr0+k(v8 z#&-@#*#pF4;L3zSLKOHcD<3oP--uk#{@nv#>?>h(X(!(z*PvMb3C0ugy5eGTa{6jT zz(QaAwGpATt9~V*g0kTJ$vzdz!(r!S6sSPTTX2o+oVV$G|M280kpbij;fE5V2v?L8 z8z?2~>graBnXD_6Yx9Wk;}%|@e=#&P0lcm1&u&?AxcG!)y6u*>2Ae-M26h%87Q2`9 z0FViX`RG9N&$pzEzb7U!aSeW0B;crcyhgPkj$CNe8PkTS{qh2^dy}vmPzy?ritX@^ z{C(YN%oYO=mGTPzt@O;e$vrJhIq*k8*+8-%Nu5Q|wund&xVym3W}SU~mhtNa*()9` zAqfnPXX(<}R#so6bl5BbK!Ma9`Ry6FK?WS`%DT{C9_{)4WxpUHsaKRQUvNJF^X#Qnv_JW zD3ajn+a+e*rn1fwn=()bsuaeI{fC026Q~Ytg`uc2AoFweW2<(DJR2bb0!2FIVGI4} zNyy$tg#dWg7?i0XHu;s%589Z0XU4183~j_qt6xlyDkRr+={7}mdv;~{41QtZFW;1C zOy%S3Lu>JB0+va)yzLS3^MpNoqd`3wi2f&-?vw;Al5o~L&ejbYm0`mb;%AgNULM3q zf;&xv2YYlcgg63o$_;+U(l4>FOz^yz=(1Gj6+kFrjj&%|Kk3xcAcIM@KoTTfk}jA| zwHKDKQAe=sIutrY(N7gAf!NQb`<_m+^Ocqynn9klgM$OHIR*WB0OIny;OHQ4&D_76 zXuFK_oM&BKZ&G4(qM$xD|4PkikAP6KgB4*0?5qM4jrCWTh8}7gcd)i59%LKP3Xqt* z%N@3_ng&Y^m zL__xV)O+{d9FNp`Q>MfQsU3djj)1wv^9570G(Pch}1t_VC#6S}~BNwl-;4WCO0rkUr zKj~1$R<&MX&yiD(jyu3S$!`7%ga=3A73XIWk*I;OtU>65KsPyj_w0U#)YsLvnOJ4}4 zj{r}=_lflAdV<;o292*&;rP_2OkrpGZMqZi3PLKvwI?z$5K5+lf=n(wf>=0kCCH^G zK4W5|eIg0eka{VkQJlyh84XqR7J+2|F&p-u@M0zQzct*tmBzexrn z(h8y(<)c-2N$w)76Vf5nj>V2}qS9hI_8n(FyQpp6D`m;vT;4SPO^ppj?OX^d8;ty7 znabTqrTGv3;R3KE5+7D@|7w}{*2q781`3AQs#RqlKQG21bcfLQK+~}%J8oFs%JO`G zu)zj+0I@W|i$-$c)7AL``?Pg);oVcvvRFb1{$XhLRN?AD1Qn1Q=$e>F#!u4e3n4lu5EBbyN7ydi2(mZ;qY2+( zFhZ(Pl-9y9hNuxhG$YPA>RML?%+Q|#H`l!QBu;)CkLaZ=mb7jWYPBH*T+nKS>m8|3N4w8&1o+ z|2~+~9{#1GA|AyBgBC!DJDr`Kb7QugegBHOFhp2M2v5AhV<*d`ngkqaPti=%Am50{ zDb2c`O1)+|$SVHXTnNc-rtCj-$QQ>i52*cFYV*U~C-xenZ6Tcqo{iJpHK)skEc2;9 z^9*TwqUJiX4+KR=ziC@0G&Xeo5dd|@f>>gHb9s`M+RAWlQItSZam`+-p? zQU&&DtE*SL-Lg63{vb>+6rAFGP4sKtRBbLx-Y1?L zvvnk(1(qArTkDLmjE9*_fx6Y^a zHsUhE;{>Yzb#Rrwh-3rUB~mCj^>?VinwLVE&l4~cQtDsV44dO^CmG`>J-}NA-*gu` zBNSKVFkAJ5cn)Z~>3>Z=+fh?i zJu@>h3Wf{_E!161=ztL68<+8Ud~bS;!G!rpCZc>*RXItfIfBZG=Qp%jP^)KY9VU+Z zxDC!iaqpr!%;Kt-qsCJwUYXqiOC&eAZq&x|9{U-+qK>I~d{U?R&uYc_trVz`d*Bsz z+*2L+{hG(dT6mn%ORG3Nz6Cp&3&U_E0q7k$AdgCmqh)n|CCp7=gp!h(DJrd#)7A=> zlnn?8hW+rx*nd-e$)DtGMaA1#Hrzd95~Z;lFe3yPzGGk?@7{Pl1gXiZpY;Hq?JpR+ zJ>UjG41`u`*_C+o#NaMbGum7Ma|=#D($}iDMPk ze!QxxicIsk?qoN^hn1)+UadjaWdG98qPRIDN=NdkucZ~s=+JQdJOubH(iGAo`2MTg z1l?*EoaEEe_Nq0?T`~pH?7lIYb_8*Q2x^ml5Ya3UE~k%`<(Q$>!#sqS{OOxF`;yZ1 zpv0tPQ9VwtDvBLX_;Tjt$zu4D5n5CJaE&JE4=IEbkHZ0M2l=zjwT+4xzrKBVlB>ri zT|P*GZkhbn+8PXmHNc@gvRc%LxNEMz+GAXB#Fkn1QZAd74nh3DYndu*lLOUy`k`!~ z>%@Z@?63-`$7t?j4wR?3zXMO&m|8mDb?6oI|J1pVTitBj+yk=nLi}mI^$hbQkbK{RjuFKs3h_5-~_uXA{n__oR}6QB2U= zhW}_oxwJ2a-#|dP&d@(J#*db@{$FiwP2f(mNy<-O8P|@vpF2C-!GLedvf%~Q!lj{h zj|Kys^+p8G)3D$nupU&3uD#ifSq(I}TVXSJA5fjHo*q~3v1@T&=AAcns1%Y40m7y) z%tn2Boa@KWI66XP9W&7ACQ=0}W=uhsKV7X+@(UOVOP;*iG@6j}JC`}6%R2f!k#`f< zN3MDE)?k*rf%p<8+Om?*i)CeHH^DNx`@<7CV%3M366ZA{#_$KkKV3$5xLZ|K$HgVP za+Lr$xQo~9JB~7n^Wbw&k0$I9HXVD-4=bd*7!G@mi^6;o2j?vBhO%qs6VE;^epQ59 zOSG!*T3RB&2m!Yu;Kq#`7o1TT!5?0U*73LRUM;pGt8=xLl{tvTBN#4f$+|jd7ZD9@ zWETywZ4H@ohcxVqa5}yPyNsxzQ88@5E0J#@Hg`C6FTeN;{27NFdSKLvG#0D`;2XGC zMFp(lB;tR`QwqPsFuFh{C@j{_xf-gpbGnYk@AL z#{EV;m}J{PL~yqUp;yaAL`uGQCc#SsA2YgAN|u#zS63W;8rB<_-OXuR@=~a;@e8Lx z`i*6?ql-)L=n(8#@Jk$#V&Wk6MB>+qIpNhS6w_je@7~7k;m*gbW@6eJclQZB=^-(G z#=pb1A~r9txu*Tvx?j8Z9A|#c?6AxyN=55T3KJCiMyR?9-`c3y`|mt95WgF<6kHzP zMboY&Fkngz)!{SqP9q%zUtvw$nuYEknN}{RRPsVb!f4}9G?C$ec3?jW7GfTp-6tSg zB2;xZm|yi;-b~p>KY?yVRag^}K2rt=ZEGbYN89Ttm-PSU61p0I0wNC#k0>Q@M;;8%3gq@F3p>IKU^?h8;RK-G#efdM!k>-OA<5 zDcaiyFC-`*JqDt_9|qq$DJdzMIqb0_wjfZ9GQbAjP)jN&=n%_d&6nmT7H}*1`1sj7 zpYEM(R_$T<^ngC=2Wu1EFsY`~)AykMabYrR%Lk zP7)%CNW%>Euk)u@I0T_ffXfSUUV-YHld|8?kn8&1HO5P7n&*L8C(BRWGu+=MjKU+| zPlXFxN!TR55}Nj1$+{)j`k;?fK~Hf%gU0IW9#6nX@E=gY#`G5D|LNizC$!V*2GcP{ z1@1rLnd=|WYi5p3G8<(0jwpT8o%{FCzZD%#{PgP@8o0X3>H4HiC>89YpUWcf{!sWLw`cEF<7};X`F~2p>znX~13@oY^by zr&uI(VF)@x9Gvr@*xc!4tmE=|(nrz61lf!sp;dTlp2BMe@R%QT`k<%3#l^+`pY(R# zE?={T z6Mk!tKhveluPt=bHYBbvs6alUp{=u}hnwNlPX-8Bt4;%40QzX7``IQCW{+A~`6F8b zS$d{l{R?PAwY9Z0Je_a={)GOS*e8Ya;cylqW*Ei%@GB+}UvXHrpfTahjGv#%hT{N1 zGjP@B;34OHWmjdu0Mhon5_?hvR)NG$9#Aj|ce$aJf^RxJ_euN-2#xJnqfjVt-kQ&Y zm=p>{WtvEh``uU>Jn4f7eCx@l0^uNzFHf0Lx+pkhpbwMzhM+VnR_rurG+;ZsFm#q} zJ3U9`0?5tnV6t=a#Aak@skQwxh7sMXwxo>?knUW-9IxsHEk*HVqMKP`@TkHDgY1yd zNnwq&peAG_vyS!Z`SYhZ`@xnBhA%4@fq(3x@7qi^vJ@S_$c~$QkF^{O!|XPEm~YW5 zk>K^kI;OH*G>XQWK4&bEWiJF zRnv)3KH?iAMV*)(sbT5=>9R)m*bL>oJZ9)K(anky;^G7kgM^Z3tRdpq0fH>FvvZL{ zpt)7iW%S4N4J_s!4G$CbmxCtXCzF$VrA8*ztv>ZX|0=`N4*9?yFv8)%wyF^$+N#*o z5DAju3D$(MBVIsPR*Q>^$dKNUefu~JawGTf)u4kCS>TfXQ6xeu)2y0D9m@jmF+5P(F0|?t4n7m!WpOB0qeEl z+v#RaEv-a&xQxMy1%9kL7+86Db^}Yn)g1ov4#3&mB^y=xbB0Io$~`Fs!txEzQ^VOd z$ef8Wb^8e9n$O?8J8UN?)?u{w(4j?B?qHFZKK2Z=+IU$x8>ojh0MwEmoGlT?>#-YK zQ_$JfV`jlzEF`3b%{Q-U(ZrXVyC$$SaYiLCprX6u?0MgcFZLXqS=Mlpl6qyW#>>If zl^qXddM&yK>-lLc*#;$gR%Q%sJbqbQJh>Jyh6_M<>N~bqPzmeFe+rh;IKJp)IR-Ef9VM#d zAr`Dde5>yL8AvaiN9_9b6tM(DGkeEhS5~fpR!T$!s&mOVF-kE30U{zIM1*;{yr^mL zMP1cmi#F=(vmVmEkySH1@we419YuDiXCZ&ES@+;bdTuK$_avt7fQI+ZmoLZEN3bSI zcFxEm(l-#JnSGZOfLAn5qTE+0Z={0|fl|>C;sBz`W?p*|?Aj6hJ^g;KHmj%)ZM61$U}oy*-11 z3PRRXg6nISkz*BQhBC&EyL{==lpT{A!8optyo?lcJb4lhmCDf6S9V65^dg$ZAKs5} z)fvE+-(|~(aP$*#85#3HLMU;E@$bFN%*G|kCbDN!X-rL95A%+;mKGxepu)>BrC(^# zeO+AyOquTr2_zlD86OHQWdT(u=HmnCt>?gL7WdGAX1?41)6{t3qB&jNgC}e~ya!1! z;a+a4WHG3|wNO19;5w-wz)48Z9_2@fFvaz2^ciA^MNB17H}^O@XMG+I&DQ?ceju{m zOXSV}0A`k>?1f9=moPXbk;*!KXaefqo#>h2vAAX{pJ$n(aJ&a~Lt<)6n}LVty0Wht3?h}vq7xg4j_URwU0Dg+qWC{uj_}wAg7PCp4TM* zx2rF5<4vGZq~!3w6e%EYNaLqcjp8zR$b3m7dG&)k|;j>PgRqzGZ~Pga#ou-aaWIp&gq8 zED5{n!md{4oRB!!Qej-7ZxJ#gb>tD1GDuestXfu)OBAW#J`|&C7|bnV?sKiG8|0v9 zD~v4k8c8)fiiVNsvl?#knGb4qv2%JQ5FSzkx0U%zmIM<2WL(MJkeVU@tZZI(=(UT( z)X8y5trGP~3VL!6bU`IB14QqKI!p0JOz;Ec2!UWv#Q;I2XU;B@XiFYeB;~Yh*vv}! z&Ng9An0F68ThJz4re;r7HmvZE>`(GjQz*;>3uJtt{Tk{v1OuyV7F1sBQx2h3+-%6L z!)|Z;kJmDWR%J%0DmKv*8UGIJVKMO$OIdU@=145KjGL!OA-bwp*jSS!iK|zye(t?# zSP1VixM|Vg_O^ZO{VCVJ$6q&q$U>p+#1S@NG7rFghxpU=56q6b^&8Wm4rz*1v}*EJdM?S zMf|kM{B5U%w>TED@Msb}DbKQHb0}mBL28f3?#p6xdd4_$uUJHhYB&{K%#Xa3szs8H%-#tP=}{&g$i)YXl&;a4&*szU>JyBsLlfp}S(VGoWCiA;`Ci=+E?pka zQ@Duq&_pR@Y<#PA5uC~$VGRJx?`~e605NLOXro!ofW-nyNWQi0%sfau5*j)$nv>vN zESZ-f3JK6MaDb`soB0SK{^L0ZU!9Oen~sza3;3p#Q0J62R~RrzvBUNo?wG!5Y3q#P zFb<3w9c^Uz7?Q&XdW$7^5MhTv?weUKdZS@<#vFbMUTI=XNTVIFI(7_)3CT4j9~WuY z{_6#oAvBg4ED$0M;#2zzCo>%F0{#P<`boDfqq$ZYj6Q7_1cs?0;3t zOcf?P)(rFBxczH4z{&|8$-5GCVzaUq0_d#~I|Z%^@YOQ(fcq35$A!hRt_u9V{jeIu zYk#4rL3jvzP)uvx9zNHvCp5R!RauEmW=3{Ih{Z^i7qYMY7&i~fptD30yv$?RI)yO; zlO8gsJ(yqx43x~4B7vbK+ZL@j@ekUL3VAG9$s%Nyp=fRs>|kMS!hehL2?y*Zq{yPo zEe;h@xPjq1d(F^0gnNPD6oS$4#hW*wfNSDqUesV&ZUe(1w_GVbFU*ZF2@Sy+`=YY) zBAWl?ygWm-G2oIEd1^YZK+rywReD}MD!0-R{$fRvHDCg#VJl1r0!1TWA43SZx#q=UddzcrtZTU+0m zKASHb7a8$ALGa$eS#Otaaoj`G(pp3d=leQE+Mp7eQ$w2hbnGsP3(UUT+pl}kyF^!* zq*0BQbSsJRp1WR(e#9C#k+T*Xj`e%F8m>fkEWXrEzuv6)WlNzzDk7MJMldRreTs1M zh#(1$3x$Q-+C9p%w8|!6j$&O<Y zP~&A7KI4a;*otT!yoil5_26h5UQM!G6uFCtfLhq61O!-PXf26vG4H_hE?4n2n9P^* zh1Xub9sJNVCVVQZsOc;I3N~Pr!`eFn2+MAgTLssxp@9KnbzP0y6QI^S4ic)^zaz6e zYlM5W0IP;%qcoNRB|Ttf)|Sp*4W`<5l!W_Eoi80E9!lhoG#DW^t`uxYU(wsHB{l8a zHwgK%(vy>S;-P_O#{1+Kdcx??c}V&yUe3Q7dJXf~lGtLMWD57wL12}MUPRG7bf_Jd zsSJ;WrP?x4VUf!={Mq_qF4tkPaqhZ2heAHU+Dwt_EQ$lleoZ;%0rtrT|T zvv4{{PVc+;hYL_vR%Rf->|ttV<{m7hkk%832Lb?`{y2VV;hvDDWr; zkaRw?h$VpIxeP{+c3qA_cqB(k;9jy$kW=O%vCP7{>&m>iS1KwhBwBh0R&Ma=wC-~J z2^a{i-6-PqW&UrtP`X@b`xnndfxrLp+VuhT*?-oP*fh#})`WP$J8^YZuZbUMx3|KP+ zh0T6}9z9;0ugvqN^(E@1(qJ@-ymlA&cf_r?bo*E@mTc0HzT-*3G0KIbLaZ%BwqlkMV2+# zRQ_SN4#fOuXyFGdL0qgT5@-z-b{c zv0X#jrDZV$0BbW~0)d(evSNskf&hy1^XJcZfPzcj^Pve4D567pAfQAc2ankc9UKCQXc(6{z22Ar&2(@IOq}7v}=(i!jan&z@xp$66%e^J!`QNeXJx3}T2&I%arJ z^=R|=0vKn0OH92YzUh|3#8}_Gq;x6C@&P{ijyKlW1AN3dIR6-zFF|h%cN^C^`50$fX zG{PUA0wSO)06>-R1fEyq)X4?tfUqa4Pj4uz8=XP{N_6yHaq&*I2*aWbb6bWE|5D*E z{6)X294#Z(r3Rf`*Sdo@)j3*BwGfTqvbo{(F=m5@;C1`9rcp4f55qg<4saip3pksU zDJUp7l3`^# zsb0Nkgm>lU!sd%@|3*6<&S*K*LC7xHHsKcd?TX!jgO&0T-Z*90=bhN(buc^h#Y>IJ zqdV<;I0G817RWE4T4jCkTU{>o`|fQ?H=FYR24(Xc{r?oo7Cw)?DgZNieW`1Nz_Tx% zn)+u%)z{6cKh!}*j{_o135hXcG0r0Ma4E&{dy_Py?@Up|LU;X~usV96bQU{MTAzLT z3ZC3|SfBrV-Q~4=;-BFdyia9&A*_Q3Y@%zjc6{h`!36lyrVo~`y`Ge^V`qG)(6Id5HYG)Si}wdMM<8R`%G#z8;f5@up99V zwBrmKhdYPwtKRhVbV9dJgB=$k;CM`IyeOm)1S6Jc{^4wH3R(YxD!aqfR0wUP57>_I zJl3;P@Qg2G;|Ijy4Z(_oI3tkNOe5rIc7p#gKqWYyE!^5Is#hpBGQ%4SZ*!14VS6j{ zNJ~GI3V5DqTd3)xLtr0a&|f9S$~Z2J^UlJW9Z^!plJ%_{wA1)QgNUvJi;^rvF$oZy z7xjG;JXh$LIxf7B0`zAMrdJ(O*wET0tk%5pZLi9sz}HAR~Z18ZRY1Q2Nh^F(;?WLMe>j=stnS|j zefWRD-cQ)CXIYr2CFqbm2!wA3T)f!uWL_BcckAdrU=WyD(o~wBlEMXouE?>0}urOW$jx7zSz=&U$D6B4-9Hx9SY+n#N3v3iDEu~c=A z=G5aOJ${@j^YNb*n2fxvw7!pzoyAmdD@x|iQ#?7!S4EiIHTAC9bKceQ#=1`yPqIh+qicjD!E-R7^7SNZ!W+wRYBI7zly%5OIsgSdhR}aLACyTd0xy(X6em)to}1Y1^+#&2YW= zHwX0F(_4HB`QGPV*|$Hb;T;lEm_ms^pxj!^r~RhSx7s~g`dHBdi&3!+r*r3)_vMr{ zDd>HZQ+C^MF*bIl;@Jh-acz_(;0Z0DH3S=!b_% z;X=;e`8{&&m64I!(U|M7VwK$D(|vur@?JL!w4TU67kKE^YUitKC)*d;ci3qxFVkB-+4!^z*G#->u$X7$+;N6JQ9`T1q$H=O2bx3_yN`FT`fIj8~ewdO5d zYG+{d>WHPJPqyq4`<0&OwOSNC=fgPP`jIP-)vjFwZ$g&b*I8R8!!rL!ANeMWM*zH} zhnFrnLjTSo=TK+(HE`>x{SS^W(HiaI*mp|i_FlF5HTb_BF>Nfq#m6dDSkEu*|BxT( z?K!#XDmTURYvr?qYy7vq*Usqn6r_-Q6k+zjX|#u_3r(qdk} zP=T68h;B-+|_U~D~*$Zu0NiA<1tDXCid#?aznsH#J=)PT5vC8e6+_K{7!I0Pq zfIpAP4j+`#r}XCQwY`3)4kl^5&|rRd~wP7VXHMA$GQ)BmKR)r?=4CB2 zVX^Pnn!bgLKA6E4zVFAE51BF9vbQ~YWp7QsIkBJpYFhkifESH&Cra=g$5K*OJ==e3 zd_&*D#T=BEawk;P9M%^yONA*EPphi(30&m4i`U;lhd6w;Z)-H$uTfAT2$EiKi2;6z zjDJ>d+Ol=0%a6ncHw6#R>aXvtPWH(aU3#xr{(fTA$T%SAynS2YH+IQv@;acDurgh*#AEuV(R59-ZpC zti@^&E_=%4orO=ZmUGTq!##bU&bPD2XS}=|aT3z#^sJ*8r1g{MXXqS-XukWn*}8(q zc>f=edaw%iIrFY)ktY$x3{3PFNlw-!H7uj}$ABH;R&&F@u>=hrvJt8G4v}vqw zmgoxne{ss2&o}(KrON9DS6=~@^IvacO%&g>9mgy^oK!*$=eh)QDS2j0&AzsB(~mow zt@f$KRcl0K-yD!`Eae#UV$fdO!t=if^VzZy89)`n%x{ri_!=gAGdgq z8Pbx0s#8j!`?S3K< zf7B2$wJ1Kc$ZgT;;Kgx6%rTc-%QdT<^wN9zc~Xn`4^~`_Wxyq1>hf6TZ#nk&6)Pu3 z=SwW*P^$Ercw=R6F?U2^NuNjC<2{>O#l+9I{HSV^di|7J-PVCKJIk6mw)7FTUgcF0 zYr5r;4pl{Z()c*jHx8`^fk^NEii8+AA8nqI7BvDoXa#vP{#ZSj5xPLTc{ zev9LbYx0UJJ?z=a9S6I|ZS7B74-IXYHIv3TBO$&jTZz&%(ldkdNdNGJTH^0}C?r|q zz+Fv|HEW3L)bc6j<+65E+S?9cR;6UyzK(~E|khnLAOoVRl)|M1l!>vUy- zxehT(VO~=ClqX)yEC=R2kA8*iQP*ZBNn3o}WV`UTVeY&rjI_zeby7(UT<6sT6;! zzSsOj|LcX_`VHEM@9v9?oJl=H-Fl$bz%WH7#(#x*M@8fyguHrwmJMmzQz`s|gB3IB!v}p_JevN0Wre_w8EdGxnWMpb&#co^IyHZ(QsPDKr>;z+B?2)zM z20twmJSToxCX}a_?CMz3>7D(7X&EZ;_PwuT>hq4ZpHnkQ_0LF1I4mEze(+wS_UjH) zw!d#_01v-Q-?EFZI!h!s^XOP{{1Lx6!T@1K6c_qT-#Z-@p~vF+cGNqvwQDa_&@W84 zcs+Dpk+~4<1ZQ%6jj&PKuNKw)4y+M|9`8CDxP{;1^(re-?HuQszdK!-+Q-B$L1?vy z`^;anFt^ka9%|h06(+Gs*wegH`*)SVZ_0b8e5!`zGZNL#H#gFOk9SH(e2Kh#`8)_~ zXXK0O#-9YCpn+45Qx}}JXP>^U|0DOwd(zTY(`nS)Vfy>*y>^y=yUnHFi&(35$HbJN z3I~UuW-xI@mn?11O{jy_@D@~|H#{~nd~F?&{Y`;8@%e%R1z?lm2*k=ZwC}qAyBmLf z*ZH>4RLcKJfwYKj|MclR6wzR+-%U&V8B_>B!<5Q&(e(c_4DiMMi>b(9e~&(6pj$D< zZhNH@Rp>2nKxO{{>9t%Mw7XN>br(Ll$D>mh7O-Y#b(49ZrB+=OoXN6VZ6{3~PirP3aZ4;zS~ z%OFVfZNp00$KT(7U=lnqh+8i}$+AvTvKMVvgvO95FsVKEe-&z6Ko%QcpO+jeNi4Zk95} zJw2vNC(yfZExA$%eIlO~Uo7Q4ODeHw#qeyzWD!SvB$a`c4{gshXoAi&z`R2^FgJcZ zG4jUF%p`q|^kkzm6TS{Ru)vl-{XL+))ZYT!hH+q}*G3|CEoL{Wxa68?6yahKULkA- zfmc>wHA*AgTgHuL+hvN4Sug&Y)%r)0{3=z1i#V@Zaup ztaR)5Sp=m8&N0=(B2>=Y8i|whFt8HoCRVWXk=8XhItsy62{IJ1FHkl#ywMz3V88_G zQ3}ZmVXSWVq%VZFj_)ssA(xA+JN!T;^&*nVpMq45uu2jmmwrY^Et?9~0CBGePCgAg z_4JWLs(aah=vU&=2Eto{n-2A^4|smFhQN0q+5r3JtYVOMi5>zA0SX25_MIw3EcJ6y zji3_qhJ8U03K5P!a0qM>i?Juw{cKKOs!;@17$2S}WiKy9JVgYf*a@nmFEVCk<9`yE zk&_dJmpibVb(DD80oRj!#@yUoExLMfH@o>>-)L|bQYsaQcLgpYFT27YnLzg)=M+kiPh>b4BS(qq>dro2VB(` zH8q#9d0O+F#0EnS1>HQ%9zk4-Yc!*p>}v}G6n|xtb25n~ok_7gb}Z;hT-O%dKY7=I zJK>v8e>doEqOIiNiD0dx2lC9Hzk?VSdch*(2CPHBLDw<-`33JrX=wtu?o>o7(M~0i z)9mCIbg++4;o^xdK-fbh;k9Q58?LJ`;2#Re;EFU_fl2?T0*BBt4I}`x9oM zhzJ39J>j6RmnF6cuB%cB!k5O!Ed`Z+g+6UrR$0W8y-kvk3{Zx_9MMa969%h@TkSN+sF1HJ9mWrGubk=S;C_HGRa z%Tom!SzR#^5k8=^g3x!^)wu+klb zd#QFdc!wmOiAYBvLK|dEoLa=h4yOKMV<*ib?@Kn(b!DoDkdLV^FTI9mxxXD^fvf4Z78 z`tY0RW-qgb@xQGNO?T^VCLz=3uu$tzVo!t_mcKOf1O%Wz(F++QpktNYyH_B_50dnz zJeO3%t>A7LMxkeZFi$1^fRuDdi0+tmq~-kvl2!->$VU_~+pt)*cmBKCgg6_fwU#Yj zD0g@Bfp@2OjPyV`nKDYd=1a_3FBaetFgwH)9}*9x z{{3oM#yVa3>45Id5p?+!@Fds7{N_m*Q(sG_|I z>cA=YZW|xEa`W1?Z;Y;55GaTTFIlJ80RQu4A0D=<)(#39F@sF2yu2LV0;>4#BvKI< zd}l`2?r=K2>Q!11*0vB+zRsJn14>h%EmI)Nf!4LhdEIPTOE##QNTneemtcVCNN95D zu_J@bnwzWc#x#vv)`$J^#+;B7Ic~^+qY=?L0fygnY5FT512BMfD3@8|DY!dA7I&20aY>&d2m8{{BCmH~^Bgo`tI`J_N4@ZbVvjBz$ zk)FZ0Bp4jv;*yf7IPei5S_+>72o{3CA0b3${cWaZ?BonOTpfaE@vb=nyp+&?a*n?v z`VaG`k4qv-{%`@jr^vrc%1aiT1jiJ5#(Ag$oRoO5z!}})n++HdtG#UOl!8eqcVYbP z*veMDu#c2~xB^VY!PszZkfd^e74Y#_DD6?yYw;m@mu@1wRmC~UgNQvq>#uKZjX2Dc zOrLTDCjlhbi{zcZQ2-!%z$f?o>(WvVAY0PX(vXqx-JYwjc*-GwF7W_(01mfVW?RA^ zghjGok5EpsRl(HBa`+$s9`YV|>rZ=(Mu`wp@X6!g_JUp{({KXV3CO0?@ht4mbufW^ z!%VUYr@=+22uM)&-5BTe*D#(sP~_zanCb{j6<{DDg0zBbBUbg=Apqmk5U&Mv8{EEf zY{aL(Bi9+BrdWMj;6CvydOT_#G#|};L6=;&Za*}lH~>gW1o4xBw{H-bC_V9Tfj&4Y zhdYmZOgys?AU~66SA^SZ{359j%B1Xj@tW8&9)t-0{X(||gaq6JLki9d5GaVLhVT#8 zM2+V*vc-qm{e?ea$uH1mq$3EeGV$I5G0RhQa(4blql#mETCyb zB#t8a31~avkF=OF0Q$LYRx~CcbjJJra#%;V-X06vV$S5S&N=yPajzR9v9tQo>(@8y z^Nwb!v+?mTGASe6$niDMIHH)_zkhkMnTtih_?iSI&B?5H`Y&MCQJv>vckSlQ($Z4p z&K_OaUNi5}5e^D^LWu9C;z_?xqZOmQ8btp=srkYCit((*%brf77xB|WG9giE&xMr> z$wr1nHMrrkI)k4HH%yqkrjlFoqdU$i=^5H~oJJY4Kj3io7BLp+qdP#`5Z?$V1OSUh z;{|Z@L19k)JZtV;27)h-u16FP9Tz9&Q%pqZZM-B|u>!8b;`d7ZO<> z-eu_?{SRvC!r-5gFG5WAz!6o2N|1EVI(~JGRjU`I&Ffo`5#PD994XF7^y0y{Ui=rDR%pJ9@hndf zId8lkfi%G{2SIv9Y+JhDF*O;mc5ppYz8C5nJ90=hlShw)Xn=!El-(QPhPKlbspPJ% zt`rJkiV{P8beulP$zl=n*ndq<^#;s(BnDmJQ;3&I+A$o4C`k8|!)Yt3efuM7WH~BX zHO*oV#T)a$a^Sj)Snz5hQ@!do+aBSDjb%6JcLPwO_!&VQ7$9k3+8 zOAbMK=lL~uFIlp^R2}`v!{5z5eB*74AGWfO**uBK%^ds&SDeEnL4mRZvk&0}ufgpw z@9_ui_IPVPkp=F;3FgZn#wQ>_hMU=XUA`QgZMIhljk8~2hvwPb19+gTA6ROVsP-Z@ zYa3*>X?lJ+;~dtx+^K9*hrZUUc4~nOIFm$;;SveRAcldYxqyeg|B2@k*ORM>!w<<$ z1BEknPlxypm2v1Gq+v1e#R4Q>(w~7xpZWq!WQaM1us_%sv!8=)JZX!kcB-nXsw;gs zS(QAiLmPWf*xJIOWcD&e4+E*_1p3#QveYIe4J5}>R<2wLkH4zron4ytV@k>`MJc*S zdFiTZR%P>bnmea3+whR!=`&!hM+qGD$cjnpnyp6o{p z79JTnO-4p0`eN;|nC8Bpvw-v*@tb^5H6jN`=W(d+3(!o7Q3<;CB~WgF(saN*(G5J5 zyTNzCXuY@n;kUj%hmT*Z6On*UBA(EDT@Opv&lIA31bu8*bEMcoWU|fs)%eZ9Eviuu zqo#OD9K-<#=FiUcS7+;Ak|^yp^2|RziMVTz!Dov}5;_tGalD-aYkIe6^kMoQu2!WR z(kkq=kC9?YrWCY0#bE#J);l==AbmSw9K}7$&T8%!^6w`lARs(P=j3Fk+tl(TwLdAG-+_$Pr_FT&Dox$ zj_h~-^}k=cxY3btJOs$INVeRornUo5{Q>j@1_lOqpzgHQkAnzdzp3eZhS~L(Re!Bn zwTeGQkaN$Ph~xxhu8aH9kX@fsOcOr){D{5;9wv8i75W}&;B@uSYNw3-jAc{ z^K5HaBor7TLaa=MVTmw@)HCoRpQmlpWUCcV?%75k3m*#FEt~ z8#&pjTFg1;6bJWdP{pMZGxK|uBP@O5~2tY+PLO9t3%3wNA-N9R_c2 zDxox8g9rv2C1QNJa%HbsUFWFmc3FEW7_=eFwk{y5Vd#|tk^e=+k$a_HS@a4^yRrk~ zwAqxW7=1`I+B$d^#^opWnjK_WnVXlas%>QX?TEq2hoGs~g-~=&9r#51IS|Op+t$%? zGKAs5LxHctj9ty}1DKLO-mSem>y7q0C;Q&am-IyOZ#|&+Wmq?62x4O30D}p&lB=I` zba#ZO0rO|ik-6FSCLhR8gmsHL?3;x*40(Y^@O+WXWy*4#PY{^64u8A!au8`PsivW+ zgdz&H7V)^j;kZ#S0`DJTNA1*EVqsx{V+N6vDjm2YPlJsH5U#eGp@_&xJbry{B|{0W%oZ-#?K zIs+A#Ear@jmtnR_)+tAogN>mt_L$_CVIFF5y_vO;tup!GX|K^k-6ST*)TzXe53V8Y zNSKGK%5GGix{OV>yHcnIX7|hS_UJG{MGr-}Uy2P7>bEYUBG!Gmzmq!nI z*_UF&hmPSH=3AKVuQ_DY$c-EsoH3;shTFe<;e2d-5e>{y^nC*TCCS^BXfK~NS3M}K zo`I?i*9`0mL$Sdjlg37tZu|pJEQPKyem!(sR8&AFT|se%+(M@&-AHcg4j4pi@?yAh z&6*Yaa=5qUDoDVZ=xY)onv*@dV+TdjtngMq0F*?tyqnFQ$*10g@aQsf`C-<0H0lqfT5%OD-Ly zhL;D{fcqhyeu4OzWrBiYNDPA!M*R?8xFC3Z-o3o0UP-5dcaU`n*BxZ;d&i!D7Q_XQ znvCi~vh!-zM+;@ce&Rm1K6KgE%S0d1+IkyYCN1K9d$_3k>3mYgq_?Je${la|8ES%< zHQSo_S?R`)FGIC-eb{}8wUN?%o<)ln*CPM-d2{m()Y7#uL&ZrOpyr0do|i^W`_;`1 zeJ;=PM!fE`D1tayS2LknT?6v5$bvAXb)zd5jQyOP^6Ae5#%~7wYl^{zuT#t_nsxEd zis0vjk;K`4^LNnu?_X-WTEMLA#qaTB!b%1`$$<&9=tqygz8XYknB>Fb+x{~(O{lO6qEU6w#SsLEh7y~^N(HV)sQ`WlF&IdEF6VYU$r;lm`-6b~gbX3v^+)Pd@vgYgaf zYMkSxX!TJ9YQbAYy#(ELx5(M*pPABne8k2q{KcA zUtVc!N9!`pgyGeUEN@KZX7S)leJAsNt%o}hNqNYHIsB%=X^3^LqT0Vr%s{}0(iYW2zeL{`@o|Nge zj{fkqQuD9{4TIKb+rJH?mR^j(3S9LPUybC^z<0UC_WSxA_y+8nfbxpSZgGuUduMnf z^qd1uy65mm;#DLaan!^mOBNQm;#ds(VRL-YPT>YayKUsU?qtGCGqC+`Y_d5I&oHhL zU?FqRP(8yI_4@Z2x}FR*sQSmzZfQnShggjlSA5BZd#}-h%&chYy)NxTf58{G4t;(5 z8M0P~(X-(XDO4Hw`s~KfOGMK`Dhn)d?xfWB5Ru}PJ8*z|3%8(MSha}atZRN~6&AiH zcK=(5A=k>tM4+Ki$1IK&&Fbd}V^hO|CK05<`IMJfKPKZu3TwzP(L}!$2X#RZU?^qH zE9=hUal21O)u}t6bC69B+Ibc+kC3rr1N{Hg^(Ej`@9V#-StA-G4WgaO*hGb_GL|tS zGZ8Y>R#GG^6;YC8Olg^iWr|QFREB6UQyLU0L(1H;SZm$Sw|)NSo_o)|o^zi4?0PoK z@AnJ!&9=@UQciEFsB@nmRg0lTdRm8^PPcOcfUme6nOxW*#`c zifZfr<82tpw=Nt*6RHo^m(w9xf>IyxzM z_?L^>t+il3YHX*+1-S*6QE9L@ig|ut*ck( zlOy^KovH`%ED15XT$Go3bXZvB@Oe(=VCiu6>g9rh2hg*Q;ZANuB;$Tq53-yJt@fZJ z*oO=C$iZV*Lk)&C7&HV_y}+C)n9X}ZqL9GKMqDUCL9e-0U1+Wx&-wZ_q` zN=~d)>ExJ$_JYzr94+1<2VAqn`EN3+RjBu>48?~SWelcf0mSIx z?6Fl;1Ga>$@1jPo!FPHoxA0idq}2B9w{Xd+4K$wP=$q1Pnnl6hkjB*+Mcli$Zx7-9 zVWPVAvx}e(iJ%tXauUF_!`R{-sa1&J*atrxER*U$@7-~7WG_%xQ$u%%fk$-<&_{!oyvZD;oviE>DvQ%FZsj(~>+ zXMcT)Bi9n-^(kB&YEEce4FMSD7Zs%fTV60iRK^?Qw;a^f6$9enu$By1w=8iN!J$P) zfIZ=Iro`Rybb=9V7`xSGCVn3z(yj#3eqBc(YUl z5jMIhO%JKWZaX%Y%rV2F2yl-Y-cSt|*9noPs$;gZlCT``&&1X8?wi9onrKHINgCh) z2@O==qcxQo4!VLmj}y-%jt;lmbyBkXtCy#W}$ zot>R3&XKm*O`A6#un+eE^xKlCwlGrBVAx1o+MdiQGyn@hOo}M1Yinu(v3cqxX{M-| zo4_9!HJNlJnM~XC(a|*M!)3d(ZV^9@>EEB9&YDSJ0)yH#sj_R4;8!$(c>w{I7m-5R zA-a}6G(yh~Y7xXBq=oUfnPb1O=;^7xOyOXT!DwiV?~1Ve^0FG2{&Ms?hf!P=x^~_J zk*$v4%oLDMhxYCBt4j?inmG8*xIg*GO15lRbhHMg7vU(%9XM=ifx9tZ%TsAsi7h}J zvd=n=P*t+)gLXcor%a_L(7JSmL zY4yq?r-Z@jqk}5Ev`44`QOj4kiL%{pHa0fm^lQh(*A-pQtN9GP(J|Y-z0g_IBZ8Qww*jb?U zRYrz+^M()KhVAkT;>ZahiSe)3L183-4?@--N}k(K$8xjLQhdRZ6aTf3T=RHkSI|9d zi~REIS13qU#Q2c0+XPR;Yow*vM=1kgGV`|zq& zpaYs+{C2v%^!yq3-i8_A>iL?G!LJVz1ylbF&$y&)la)nM=^SiLi=OStU6~OUb)3_u zPm>9;Xg)ziUqlrX`Tk7GR~cByXJX}V%{F((Y0g&pH-IFzcQHQ|)WA_YyJR=5(?-Sr z0(O41mHAx2Cub;LS`DZKz+T4d_t-1VUKiY}ey2MP2yY=PH`8Ku$%Ly8zmU*1bR{;p zvEc}`SGv(!W~5d}jd&PqZ1ECO1C!B1bm zUW}UWIr8fo9-Tgh^sJ~1MU@cYEev^IlkUs;Qjzu4^W1?O(xa;-Y`+Vy7&k6%wgo@) z><#go)mB#e-%~uVjA_&&DBuybxAZG;hC?Vz|1VOW#|ga3o>i+>@l&AFOhfM3)t{w; z3v?9J;4KiLJR3Z)Am9>s-P%{Lu2XIz{H({T70h)N2F?F3t<(Ux=wTGxE z3vqI=+>v0Nf`S5Pm)|dy>u8MaN)M4%Vxezyn@c=}ku~#i5bFUuirW7^gBSK#5i)HO zjg?O9&Z}6%Mqdp)%nno;Z`%{h#!OnZ+{y#B)^ov~ucx?UymO>`6m#U>%Kn>HGE7@Kpo`qF~g<9n@s=muj;hvAXo%(;Qx zn|dzopHsF|K`MC~mYuu6FPxCpW@uob{Lr;i3}IO__u=wxa`G*sag}Doqg<1H?=(tP z$*QmHsF&xUHqkxX--4O)68KE|66UK}H-y_1q$;aJO{Pq{FBgsKed2Zr+WLl%DvT9)Uopdfvz|lK8@fxe|f3YmoA` z3Erkz6wrLTk=jRq;NP7zVKg<1KY0yEwangrTU$yBiYWpX4Sk-0HtjNKSz>3B|9Em3 zSJon-ojl+(@jNp{jB>`o{2gF+*3~V5{$(}=q;@=d0&r4C6vP0Plp$6s@+A^t`1r}U zCwXN(_EcLOEa{~$G*+&o`d)|1YrgM-(xG2Ca1_0n#8T>2n2W2@s7cG1Bx#^uuth-# z5lscEN5ijRvB5_RLn4){>Sb5q+K+OmVT}c{JGkfZTl<91OtaS1UvU~xgY0j_UUku=$?Gsk%uMyAfo7Awzk^v^Xhd^>2FAyo9Hi+}bk+Cwa zt}T)=fhMhKwu|>odG1$s{f(T_JIf`%3v05C@sR2+E z9(>J^X1klXF2&F5a|zTq$V5#)_W@gew9YW{f3yH31r8^m;Of=SD=TM1#PI0cpb&~c zhSBjY;w)FA_rLRz+(`gAGV=r_+q3p|V^4pIC;tcZTbaXeAy^=-Hbe;&tLrpiMQ59p zo0*O3&{q-08GYLt@Irp*n-I(v0tuQ{&oJK>_PAOWyY(u9DEwB2t|#8Vir|`g5%LGg zdEQqnYY_Y+YFak(bqYNA(9n<{PC4B>y15GsI2+`Ykf7oRsET9~ zkbFlFkXuPdt!O(68PhVN2SnRgfdCyGAsG_^2|0N8wqIX_2_jR8XdcO)MKhxYX0v5F zLB7m;qXP{`60v*3RSkulGH_sW<|8@s%0bTdLpL5^H`;74Ume?(WH{Z}MkQoyK|v*s z7S(~4UA`zTmy_|*;E=&iw0?WcNW6X;oiblj#OlkMNbOei+zKl{W5jd>b&$nC+FWF& zNMApElD)y6>N9>U(q(|w#>rlsW%a~t;6p;TkHJK%yL5u8cB+3B75b~wu*`k{fC3x@ zQv)vI;Q}Ft%4DwJN#7$y3HjEB+RXyOW2W@HitQX=gLUWEEyRJD!O3Rs4*F*qHk{*R zyk!08e9MhZubUQ|J1`ljoN$Q!wGTW@09VGcWHuUq5Wh(%C$?_ewjY5m>vO%M14aZl z5@H^KuBT5orrV)}yA~Xr*`WBG&{KfNvo|dUg@eC;MOVe#<)?UnjRj7Jv&Um1+w5J=Gk9CJjgN`{D&kh7|UTX@mkAh#10N>Q@n6X?I+SeT!G z2_JHpi;|aY-;YmQQ+#Ar17t(ILWW|vj5;D-iJut>z(A};z~NxwJ|huAWoPy~Pa9UE zBtS3%^qBk5z>*R1BrO4d2yeC5fc5(f^M}%KR$fHeJ>LninE&I)3Lra-Mm`}DxaV+c zMeuRyNCmDYiisc|phmZPF8mvsZ*3^)(6Hx*3DKcp0QL41^(IsYv-h2qJ{HF%hX={k zN2ICXHe3Ivx-7;lge?Nqn}6nhNJvPXMKtD2-?A1jyH;!3odm%%^uCvgm=nj=?o*%r zbVvHDkvWQm23u{!t4uBLmLt*mG$=UFM2JR zWDLS`#qu<~kQN3)7?3j&@`#*7AQmF(TA4t$c8k!9IAQtpGFG2xyMGSbJluVtP3OD9 z-Q*uG|L_x$qZdgt_{%HoQY+POj5hiY~jT4bVa zB2fmw+-G&txKGUX=|Mq@+l2@s{6mu64vfeg$Uhfh z6QkxcK`gbj#Usk+DoPgf#jElaAOxVAu!RYG>#kiI_(o*h42esDNMJ4nm=wRY&-e<6 z!zkE6v**n_HuU-FtFxuxDTw?@Oz+-x48!8Z>LKwM1ilgpMQee<*@(sh-s`Uu22m%; z_Hb-WoP)CQK4#ot7|5rigBRYBT&TvzMzVlGAa-C-L7rho&u@lN8BoEYoaShPA4V){V}E7rIBon6W;Rw7NHTGBMcQ6F7PjJ z76431o&5EBSEniv8p0&yQy7b9&z^l0egyE{&p?duK@*R@q>Kp){*HHWMmI+yXJJ4vz2<=D(j+O_6TAg8XgP}B(EUM$Rs~dM$Mip>ox2fEqJ9E1M zyWWjcJ7q28^KIAXDs?0K^AKI$ymd>|i<2-h^L&q%G>X|?h+ugM1Ol~NFD&)(Sg#9Y zo-zk=%SmT~8w4ipCglBOPKdsNe$=>17>MoO0mQnFT~b*q1N=D|sdQEyT9sf(lDX*@#Flv!3ukR#C2e<5=wm(L})Fcf5Ukt{xZ- zb{pbaC-3-&iPM)r4^p-hLhsb!4QigAkKz~dL&8rVigOiU17%20y#Q@kYky;p+TFT| z@V|#_ca54^VuxF#bWDiL+ z!S%9VOiZjL`QS3MVz-Tv(b22R@?=&Ckc$IE0fElk&YcrO#0ANX)trusbOdq12%}7TW3r~yf%8UVefmSgA&nOP( z3eLl;h$`!Y%IDQcv>J2)vaX*jHim3e(yT&dAR;P{xeJq1Q{lzM2)veW!m_*iV(ai> zy57&c2x|)qZQz0HkqV5~FcGXTvM5DG4bJn^ZvifJ^5&;6k7a3GmHP*MlO*VyCSySD zt}Q#woBz0dvQ?#No{#OsM$wZ0Vh?jBlvR8>xX*d1hr-2lUbM7Cm=&Ek2pNoV*n>Cq z2sRj_Vc50VK;u^$E2m!k8LReXU5`YVS6H#2(ORl%7d!JMGgp)Q;O_sh9{^HAMr6vE z$|Ix^+@84d_nEo>R!z?OL(Q)FNIWyBq%c72QtXj{!;AU&$r*yM1_I&BW-m!wT1wm9 zss{f93bkv&D`lLY%<`Y=q^|tLExg<@)gblf`{a^Ymt+zSs*ad)&&u=TXcmwzam=@U z#rK|P;P)(n`PkxYNDR$7#hv>8Y^wZR#jk?TM5m;|i(bSJ?5`~jd$UfvH~*p|GlZU{ z&Ivqho-y6`v<^LxVb2-=fANl7cP1++iT4AI^Rd*KUBBzU7|Q?REIImj1c1~X_qvCK zbRnG4GW(NgD-9;Tv!6H4`$@fW@qOC4XY4!A8`ubUW{TjdD^uJVN%Nw(% z7cN=iz=|3GwnwIO5m=bA88ajYrx=khIf%WOO-R$)=H_s$uyo7^D8OZfgtTMQ^q^i~ zW@L|`sW2M{A}t0$rU|f|VUuauu^eqwe~Dnd(Vq&2pTtpm06He;K2*@DU^^oVa33vL zwCFF2Q2F8P5R_sKPyiDS1joQ1X0AN%wsb;pjwJqj;yi<(V&f!qX3p^W=g zxdZ%FEpm?0`fo};_|BTn;cwPsur;xP$!@@jM~8Q6Ajv7yrV~?%1&KkIIazLexRn_)8%W;63F;0*1NkN7WvnkDciwP^u zmaR^?51BTS9m&jNaGCp|Hw@@RWbHh11uNse+=k@G?DHr$B_i&?^@qNBe!j3Ta|YfM z8dg4-;VzSGU*v2xW_UwH^JgxK&v;~;A|Z3}P2vx2Hf)NN<5{vqpricUpjo6MU>B6X zm55Bnl^%#exFl@dY`~xi6cI)PEyh7E(1bqN-_a*S zz1d#;1JEuA?(Wfg>dGbz4AdfG))a)24Kz~??iX<3`f_}GUj$yP(}>zB+)f~MI7^-b zFNZ_%8N4B9n{43qA|HG6a)&n7VkXu)~201 z4eh56dzjyCiTqw`c+Y%RE`irU`yI_GKaZ{u$9(wl* z$RkgUX00h%aNf&Y{#iE$9FZQnB*)0eWDIM^U?RJgx%r#*6%*l>oe%|>z(z&9TewF^ z5))3gBQK3xC!-Iudli%_tA(|u8w9va*6mz+Rr=zA<0`M{27}SJg0pFrV>?==Hx&9d z7b)3gck*F@uxs);;bwG~VC$^zOh(TG(N=05tF>rQBSc6Ifz~!1B8~nd&VH$JU3j?P z+;IC~-jrXl_eN_PTek4TI|G+uRQ?4>P|F?x+Zi&*`wtEF;dy&*db%YME>0*;(i zoL@`B{#K|`RRxH-F+h1gj_on#@G&=c_heKYnLXk8G@~gEZED7Z#3V-HIidSnjva*q zWMz}AWrp|fj~+PX4rktr)Ze^(3}Ht+D8^fF?Jc33DKGV^91CUok zk@km3Wx*Cjh*ZNShCCN?52F!r3p`seg?IYfbM;`7+Ii~pz7_1sAI}Yx8iUJj!}Wr^ z`EW=XOwG-kQEkWk*kn%=sJtN~E}jT4%1yMPsK^@8?gU7jhM=Sc3=FZ<0e%PEvqsjj z!4E}Vh5(fz#I35Pw=j=h{iXS51)v9d^xjYVbQ_ao@b_GCp_i_uYeu(0UBtgl= z{PCb()kAgRf(7nnv9rk}YO-aqm1U!DBPkuG)wq}0*FjI9J0QHc*JZ z*drMn#(JZ&BF-jJGAkYSxdV~?ks~w}R+aMo;%u1|vad2X_gEX;e=GBUTVg|SB_~|} zZ;xu^?oa@n*9b;_=tRAH`5rTN`zn*xEWzQ zAw&9Wa2dE=7%3O_L3aVbxn$@9`)hRBSNNqf}qu(6ZMm|Y7I zZEcIk*o!1)2nylR^P>1A-<4H*a3i;FSm$cdMBa7kw&WYpBYHjq?H<0z)>1VK-mPqq zQDRL_I%!Ae>)|(5Wos=EnED9N2!#+)15+xhs?;`iLIBi=b_)=s?t#(q@fDm4*%_seA`3>C=HLiO04|ib{(*tV`kGQMY11c-uVM9*36#uVG_J&3 zCrBB{^2Bq(i9hS$)O`ayx2Mv0`G8L-$%8lnwNp+BD*k z9)`YdF7Uug@(pkt7C5&qz;P%(E@;a>bbk6^NGNe!`MGA((@XG>2`7>Np$Cs#)+^=K7ZHwSFf`EFgux-BQQd_h2U zfCA-P97vXG9`4lCR5zeBlD^k^=_fAQt&R=r5f2tUfq{emKjm|Q4`mKq@bHD72P#QRd_pL{&@ z5ApKk&Hj}Xzz`Vh7>e0E?tM*a$JT9|(yARFq+c`qoAw`E6$LblKcGR}L{5rU&Hkcf zj0U7G0McdwRh^54@*Jv&qp%~LVV68JD^p?%lU`UmE1#a+b=j>sQPgOhtiFUP>u_&0 zUykpKq!6}@Q+yuDI9JPg?OPGdeEZ*w)0aw3?KeX7l3|69e06`&S4{`p@0$vGRT0+e!}E3Hk<{Um|_&bdx`P!`x>e~d{|Sl zurr5A>p2}?&uoz6>^ynG`I>M^W%;$C@zI4$)?}+CRU*t8iu)7n(fo{3Ux>J{mPFc) zOO8H>@TuyGe#>ln=$$cvdqhjhvYBOON^R~IA#^|3;+-6#=uP2~MJh+jxkmijjj&|_ zGG2`GBg3E=R0o9uGCsXF!l_Q}Q4WDpK{paqoai!uH&aebz;hRfAyMd@Nfu!{Qf`-sX;O5{Px=!*7zycUCtc^`xwzqKq zp>bLS6~z+|ZEh$!Y5pY_Wiln1-FcH^YQ){<&ULxKhi4U zh$7wxlKYJ4ZoyMq>ObVu266EPz?66vEU+yzSWKBUcab0X2>j0bRNt2QC_qqc zG$ObvL;AS5OzGg@;Fq4B0FX5{WEz2xO6O#B^&Sdv7;cPPU=btnL7i%e&@*! zL!EE1lfxncfN#M=o&HMz#YiMTlj&eH8aEg*Dqg+P?#a%AjTfUIuMACrbg#jd7)Cdi zX%W|O1+xxtC!Rvu=@)@7oxW3xL3$FgP~DmR1oHmJ)seE=Vov1`nT@&5+yjM>Px9g7 z#BKO+YXj7S$h0ORA}7fm&?TYkF!GXKPJuKE+?L^hk27X^q(loaS4=DtL^H)oCw z$buO8vy3<6T9f5Zo;-Ov{Xm=ph8Y6o!-iJpZ-v#`k?GKoGNnwMJ9FfJdp<&}9Wyo` z4?*U>mbrJPcF)AY5Gt;M|G2p%Ns*O4e+xI0CzA~o{Wb&-gISylFp)4lr{)x+czugV zbm`IDk;Mab8gs=Hy=nTW(yxbb9~NN5 z$3?)OB=f@ScW{gk0P9OtzVIvu6-@-M#fSFU1rEZn? z%y1!Lq(+NRo(=is&7PiF<{OnN-&W6hLoX>P7oTYKV4JQG&*>bqq-(8pO&>Q^bH7cO zNRsk(T5VloP(000Tqk;tQ?+vxqty>s<%vs5Rzd?pJZTgeF}0r2e6&kdXr+g7z&*n_ zLcEbGbPPktds$G!`(K2MV=E)lZgPe7IZxPdVR*FcKIs8Gg)oI=v^iv3=`Y}67{+o* zg4zv~KPe5-ZyY}5kjFdpYzy>-^FTX9*6&9LGGFG^OwPmRZ$Ez`D{*zLTaViw{Cyxm z_-D{dp(X)m{TxO-q!lDaQ32%NcTpn}45aTy79at;&%I09fwraCgnOWGQyVGOQHk3^ ziJ<%hmDy#Y99yTjd3HYzY~s;H)kUsO%&0r_c*bX(&@bxh2y1a4Gu#Q^j_e+gkV~_z zC!GwpOu2+(mS#|t^<BT#9HrRtJZm*)kF;4Vph!2RTaXFNN_w~31;1G!chl6u^sJC$g!gR^mb;&a2tK={g zpMA>vIx|00C_+33RreeAO!PICN7`JqM&^VNw;)8gjVU$vCDUd0@@WJboY z{wkee%hld&$tBiQZfL(5b+s)g!nr&GH{!AM$-|3I4(w_hFP1YQ4M6pFs>_31v7J>i zC%a$r^}vyOUjLA@k5r5G1wd#Z*CbVt#Y(%yV1|~lWj3~t)o%I^ zqNT$Zub60O^Y7nZkE|agAsab8j<~zxWVcqko|)0#K&+0azW)dA&|F(D)S1k8M>wUL z_vN(y3T_nZ>z1$rc=$3I1DD}lZUAzs3r!tlRH^_GN$yUh;tJ|*({f***ga=b*K1)a zF!awiG3#J*l3hOWbFcWd(Vw%J9;Q)qEORa`4qBQU?FoKw-RzJWgZb6*Hw6e zYUon@A>Sm3CO}-gycLDj;=g~n0_;=;@$}Ix$0Sb~5f2@uw_5DBUq$;D{Q2|Ig!Tb7 zP&<`(_ncY75H4mf9F9MUn4Kdo7MkOm{Jq(kTgpF7s3+-zCALBdxZv{B^|x-_N;b}M zu(!VoVX=rmA7<0ttUic?%>?dvqjhKZ(hdpH+`3g>59iLUr=RD15Ui&Mq)(O8HG7N? zQ=Q)k_jF}`c_pDY7+0RLhZ_B|Gi)&OtqV1UM(^o29vs&C+q<=fGpIj0^-`QxwO3B$ z!o&}LNZ|P$9=!W0FA%H-Q+*H$A*HGp_jonO>nB8x-{(i+2kB@hWBEsuOwt3LXnXJ#B(_*To%!5f$3C}ZsN45RSxg<@bB*hC@y*otasN4d4iy%r^e=Lj zIaXGW&aig={p)?;jhE58+~HCD^A&Qhe~(mb0A=u#hf43f>;#v#HI(&|4!yCD69xsX z$3A_&tYp2a;dqh)E5}2vb6mRj&d*En{QVQcKD~U;l(=i2=-#W0&);5JaBSX(nKP5! zB#%6RV5giJ{QsPJ=Yfaa|c};$(`G%>go9im=ocl`NiSh=%Hfmwld{_I(~)-~}XZewptS`%2CV6(xO*WHp} z0qK2}bpnpjElbRvJRU1N@_lZc#*SV^#rWod&;5*mBbgV2kIc#zXeg1F+y8xGceTi- zE%WEioOx7EsBxRDktT9H(;5m-T>TLvxLn%&%8_ufcPBcr7Az)z?6A|&c&?VK(aSLE z-{CrYQsKGOdP>-1kLR8fk~h>ex&!$)eD3<0Q~E_^P(ae}SN#6nvL^&}#r~>ztS4Mx zUT{Fub^MLO5GMF!h^U0XMeNi%lJPZ|ZNRoLdmv&!ZaYccdKm&6ODmu4x&+bes$`| zh*((pj!T>Ls-L`!9;R`FxAfH0`G^dwLq?&FxAXXpfT?CTp3W ze=qGWttaE#7p{fH=fu{;{1=x-nkznzR(zbfZ}(Q}@t&&Srtb``==jFI&+FDK`rVeD z(AanM)$7QnU*EpIJz;(3=?u!xdkXVHf){>xU*NQTQX%QQ=kB_7R;Nxyv^6i2r_luK z??smF`1ZA;wPv2YwKb(8`OluL7c}xPbKL0{oEaA^z)K5vu`Kj!`22ISgnM9@`+%6w-=9(z`q%nj zKm^pM)=a+E)*;9DM>KO!ioW`0Su#DnM{?1#6Y8xSLbvUln2tNWYu8F8Hp|q~?)=?H zG~vV&A?eU3tQPO@OBmeW)Xb69CFcS%vKFQ0Z|{5DZ~pKUxTihc)nCe;oQCwYXAgbW z=YP=1L!U8gd3VdaIs1i|SGuQC!`%NYK(C_o3yPI9BfO2T^gmXA?6D+9$6-0m`09vo z|2pl8p)Z~`KTHHm&Z%f9S#?`#@1Cb z)z>d3xEQ-1&{anW&eqQ2=FKDRLW_p32km=+&=LyVf}dCM&se$Zp?q6Zp-ZOmbG$8^ z%p(csqos$gUstW6vTL2!+T?wH)7Q)cRdw&>OUW8kO1$Go`GUAzyq|nn6_1T&3gW23 zY6EQt`?AG!p2?_(UeS18H00g4k-22e=DpGzUOAAxt(%d-1I5+4!S~OpNNU;TT6A?s z*Bw{BVwdQTZd>E}-Xt$0Yqs%$+Q)7Yf%P=~71Fs+t_aMs%O;eeH&NOPNp8h?auKOv;jnCxaO;qt{-0Aop8K z^VD{c8Ed1ZRB~zD)p(7W8oQjomj7P&tUXs_;Pd%i7 zj}y}8YFE$Wnle7O%vmTZdRz^;`}Z_Lm1AD(_|CNWXZ2Nl8@}oK@#AXIC8DBfch@LB zN>Eu({wC4OVbqfcZ^u70@Xt*tYbhk+sZPoP@)Z(VVtj8Y`KkEYYp?Syu?jLmGR3}X z9H0~*cS!c$_!ba8XghL8C!D)!28-s-+9|Xc|ATXDwd9fuLR#K{s)DXvyQG^2uQ2V& z;KRI#+%lqP2F?`~wYr4X96p{kZcRRpv-}UMKQnF70B4M-c z{pM(qUutn$Td;t`gH>eS6)=T+=M;6n)1fSb9$=RVr=zL|Uv)fWF2uSQ#-oLld_hAS*$%o3m`9!YW!aQ^j@lG{ z9o4^7<3Hyv5rV)9Ql z%nm2IULw;E*f7SY@GW2NPn3IfQvgbyeG(yq zgjtM_gW#$f0AoU<;~HPZN~@`KeN)q`$d4c+T0y{gA`t5I@1K@iES9M!<>w@L?8DgHTf{Wxl7@JexHzNm&m`OHlP+|QIz;5ep zZ9%ycF{1&qNNf|2NJFNv3^epY$~;(rj89fMJ}oA&(og*A42A7{E@<&;G7SluCGxVi zj0?+t?Klo;AK(iJ-sK$X?}f;^w7oqc(_TIh{G>b3U^22_FM|^oqL)G2-R$dJ9oWQ# zOGX@a_h9e~M*;hSRG`8%Mj-9L0YCC~(ygx6kZ>r$MRQ5ULFTXSt}c@NoYDg<1V%q` zsrtKn4li1@{p;{Y(^b3vzDvAAKjbGyLPJ7U056J)!dlr3fKe6cX{}zstXBY6lAA&p z(K0d$wqQECd|C~*Ese7r@|TDnS_U*_CA$4Cyg0e3=uF{arjA%R9$OcD6$V~vY5@(7 zJK>=`z#~$Lm=iU;#gkn=1TImN(h<*KFqTp(L9p2UPO1hvrwY+}N)Nz!vdA@G0<&8o zAE%3BDnrxw+Arvu9-y@iEKsz0HhUOKa|O#AHIQp);lX!&8Eh#CwU~s^t_oymGWm_a zba!u%dKn@k%htGRF@^6kKTdZOm|~HX(6tIjpaIe{JEEZE!U7d&Y*aPA`9ck^Y>%FGXA`derb(cgl1;KwUXsU7T8agx?zoAh0Jv(kv)7?W zi&DDa<<$k?WJwQ^DSl20Rl0=Z7b-BL@8-{~rXlOvDilJ5*JF->IKlk+V8nsj>)SVS z=q>qfYE4F0iKtvTuxyLi#*Lnd5S38t0>kD#Q-Ks=faz%@AuCC-rr!Bw_}A08angUa?O*&ou<>QHVpH8-D} z*s!ZO!GaT7v)(^lR(g-LbX&Wl^F)Fi?TJ+=(wi~!x8>WmCcJR(D_#@6p58tt~f^Y$&c5!uv3(%m2xA--K)aSh-|5NX$tu${z) z5@o-x?h+-$D+d;x+$_shdChby;%+XiPYTuQbC-Xdr&JQRq5JUS;_g6;$<}rBb;irc zvAUeT^f>vo(^r)j>MJObUpMI9@=WVti6dF4QlDpYy3C#K~3h5pDQW(O1zU5U*zK~L5BeZT#}6u6{G>K zwY-w(FYk_|qUni#Jso!Ds!q4BtiTO}#=x49 zo@&5g|~Cv&O}8+xG2CA=1^bx8Dc{P#(;bnPWIC zk3!tEdGqE9+(bi2h&BDFo1?mgMgjJ%?BF2g=H^DmW|F8da4?L~Q*vA(BtTX{8?+5# z?7o>74cITLa$B8=c93KWLMBiJM-mypivP%uOgi#MzTlQ%La&uQ(FuDx z$+kh&g3;0iwdy&fhb#y(-CYo= zlPDz&(aQ;|sz<~^yOa~AZ6I1UU z_k!eRJzinT7|M(p72nEZ=W`>(nhw{MJBYp!Lh-c}6&H~xGx{~Q_=ea1^{t*RgYb?3 ztK5|^|18DPn|JLC-k>eWKa8HWEX4$F680!}`d?_~0!pE-VoY3|6gYf;^`wlm&kQb* z6Cde|YNt-!$5a5~`V`1EE4*|&wlM+ZSQe4)f^8L%759!SMSt!4qsb}~8bJ4U$s`H3wi!JLwgkwa+a@g)CGOTx> z|1?SyKt7l-A&B9gIh5vJh?2JvXg`wh-e$0G7P6d$olVyK*=S=~r|0-VMca+Mt7^c{ zLVpE)-%B%q)9(tF3-KeKEh&uaHLV`rPxfWKxbf$wnfqv%g|^Ftul@ojSrSDmF_2&d zHWFjglE8HRFHF)9B_7A;D{H>dYbu94|94~fh?(uTl}?82RU-X~sPk?v@0-90#o)8c RK2h*>P)%1gNBQWb{{_V`YY_kd literal 0 HcmV?d00001 diff --git a/docs/multitenant/ords-based/images/makesecrets_1_1.png b/docs/multitenant/ords-based/images/makesecrets_1_1.png new file mode 100644 index 0000000000000000000000000000000000000000..f0f6f21569bfa17aab7bd37827d97aa041bb06ac GIT binary patch literal 117953 zcmd432Rzk%*gt-xLI^1&5tY&~G9!)^C8c2{Ga<^(9*68CNs^2cl9iRc_uggiP4*tg zanA2Lsr$a4`@Wz5^Zfq5*Kc^8SG^8re7~P@UGMAteqSG-+qYyX4=^4;AP|(-uSwrU zAokKC5X3~J`{6f*D6ACxx7Skgx*{q3IFdf{hUZ70T~&Q{&s6`JjgEyL!obATSnr~x zu7#eSiREL{XA8u|VhF?u#C2&2Mcc61ew5jM`BX7WbAGGLq;+Z(>yM6 ziZ4rC_2l^iPMSylFf5ccI&f|zE^?r)&p|g&*^i&jlIoF0FD80_8}s~CizNZRJJU+8 zFMNo<(Y{?El{j#UQHmkRb!X?~I7gtXU4+hlAB=O-_;f@1Wcr2yYW=lRmj8Buz|*c1 z1NJi)UvEz|qy1M`pFQo#F#K|lmF?#bBmWyT8lB^`y{^B85~<3zVAHnAwG17`hk*1rKqTCSF3B>AXdgC6laR}M16^45Ljn9e{M|c z)PDTZC5Ex)YmAh1xV4mPv(sO<^~xQVADtq1&B1Tr$DWM($($l58eDKzQJQBgWd82^ zB6X_c;swkIdBY|ePs^yKrL?u}?p41fwlf~=os>}aDdvWf-}&ZYkwJ+zQZQoL+U{*Y@?`Rglk!Q24MZ z5^0u}&gf*V;sq3og0k{RRjJ$Nqk1~qtu5A7S)>I=_N&qARxD=5x)bAoix4jp`?*YEOq+}KI}5Lv}|Aoq?uj&Inq$dJH=a+MH{|a zLtNe63EMf?5#OB7R@a3K8ZQ>`+r^_6GW&&{@eWF9cf6>|pY#c-Yvvxxnm5^7Q7wh{ z^P@IcyK@hZVZkD@pBQn^I}Dc7Zm&yxGn$(aRLA23LVcfz*z7?tqjszw7Zfh})b2&3 z?#v->^Ka1-77$Hqm!%=Ej{071yAVM(+~vcXV)h7^sRcKa0zn+>N$SC!NwJTE`&WBU ztaYEjY_noq6TL94$_agVBzD$dGb)c!qcXu&K?McW7%u$tL=o$ubo>emf#`Hz=at_c=UQUP z&4qU*U~+ZShJ05h%~#y0KDN#*bv;Rm?~!lW#IT*cd^D4cY;&Y+AEG3fo=e!V>Aq82 z8@_36W^=mkh@;i`0g6Gqm?VB&YmxrQ&V$g<@y@JJ!N}H`Ik8v))UQ3}FioZ1hDdkj zr}0m6mLIr=KyZZ%J}VmixU%NaBukRiL9VHJh)?rSK>>m*sm|@_X)@Ztz*h5-3ewM! z12z#(bZ8ajkrgD)yapxHqNeCxo2Vic-AP3wdS%C^n0(HiEJt*TeyA6P<1L`r|8jd^ zL+0&_!I2}d0;B}ZqecuW3mvAi)o3=O1`?WF*;D_+*a7(JAH8{uDi`YYqV}>hF#tz7|k;8jEV1g>v{NH zklx0^657b`n4DX)1^6Skqx~1r-@a=wb~=1Jg%Dqu zMAvuj*QCnGou1diA3!XUk=?<5nVOLr#jM%RwyP{DTJnCTKf?NW>7iFMr3y>t8d~GN z#KaJlznoC=5qn|l1XPrUXbsp>n4>_ zIo2P?{iM8F?kjs42sP`KvJ=NUvQh84MQ!X;nOB<@vOf1bzs;LgFh;_c(YH*V0ZEZ|>F;4xaQ8`)BHUWxua7+8@@dK$-Z zUv-ElLK+gMd53IUZ zJ<*6liv4@p*+(qwg*h8Oc#x;-4=*F}ynE;`xO5B-&ZqBe%nw6|BHivpDfJg;<{r)B zDssWkb}#ee&!DBXYy|DawjN1eR*=6TmEC>8R57Q{C{28M`MG|d&^T4T8j97-X+cwZ zzK1`muZ@8dg>JdM>F#;(SoOMWe)!KzltEkXVAiI^x~ZOp<$B;VYRL1m!0x=-n%m0j#B7qrgBc#@2>1OG~C(_ zC!#$goPlcZ^7g^6Do}6CN3Njorrt$j2=V4K=#R}tl@)D@eP0s3`fg>!k*+W@Do%1@ zFlM9K#g#~GF#dT&q6#Ypv)I)X{&;)NbS`3yTOCKdN<-5!we~*a8An-1Qu+8xiuTJD z;iA3@gLWiZ&Z{&TIGioE7;ocvU&U)c!bXNd1&tH2QEW%0u1CxA%ry!wg&Ux+8r%xgYLFgB5F!QNhY`Q8-)lJpQGH1P;tDSlVcjj$ux975tAkSi4t2)1I;s*>XD?HR3 zC*zkG@S<9_KTdrtrYh_3ElRO(waP+O)DphYmxE(-pc(b5g$n6DSI^aIwfw2yV9%jL z+_A|`^J=i~9`9tMh!J~ZV^3}#^EwlOzD!F9sY&Su<9>UGW++`AC@I0M#h2?#Hlwg4 zCQ3@IX02@PTd1#2A`o#JE^vCDy1qH%L;JWVPe}Doe~0dne=8^i&A%gJ-h;%9##xX>wD8yiGBHYf0bBy3S>LxNA}CfeE~@5Z{N zb|VnAoXq-iKSNs+rJers)NTSk(XrNx`L5kc-j!F3`);~V@R0Xo`9@qZp5A=28RbNB zNRWDW%{9KZVW*}MmA5tciF^VK;+;Pb|4d=<58^S&L9MaDzF*G+7(Z+bIGBF1Jh2<^ zqe#3z$HrFch99Naf9=iQ>u*}Rx~9^cHP%+pc&6IgT3rEL^sBWq{09yk;I^4p-`a_~ z>YM7YG0)SmyC>V;G|N9WHVzWpgT-P~wM+MeU9tNX)0CH=ual|PFBg}UMfWDuOQI?4 zGHrZ(yvf>9>aw&`Q0lKRP9-=e$W^dCpHF)5U{!74nI-H>b97D)16RRfnd+Fp!{#e1 z6cKh4A!)Zp11d4QX<4+7DPn?}hQ>7(?aXLew8h?$^uPSSSY31Fv7r7wN? z;2;Y*-Gx+NJY2?718=iB$hkLV()1PP++VfjjSfuLJ)9u~ApY6m6#MP@;nd4LL;^g|0iVwwX z(w@0>0zP-Y8YhGjNU;CMnEIhu;N_~!rDSaw1n%*o%jF|1o zB-hi^6SAIVFfuZtrlxjjXpnDryX@=#@})cV9)5m)1cG(1J9S)K-0dS&h#x_$(X+)-0|A!0Kx^zxp%It2?03m-rK@VA&7 z+>CyI3?yUINPb>ks^NEb^#WU(prD|*loVNK=c2xWZ{MEV4A_y}i@jAjQ0TBU?8Ool z8A&wRj3)V5AZ9deIaWi%B<9Ev9v;rkK+#oTmp+Ys+cDUWM)LP*uo$c~H#T0Cl_g0? zOw_p);)&flclK+cS6Fs+@1BgUdcD7T01(}wbIp+;O`dWFBs&F9dF+BL!2nO zZd^buS+G=fjE#e%Hq!sor&CL#EO=FC{CcmcuieT-t(}8|gD>1p?HpI(>YMkx2Ct-$ z<+b(oggqsd6<>>*Yzk*%XRrFf5E2r?CM;|(y+h99=oGYUQ+aJV#WQiqi;9Krd#nPz zB$H!LmWIh?+fB~Zy4!@Jp5o(YG;B6!`alBA}$u@dzD{aNK?b&lecgoHzkelmS| zR!^k2%y)bUQ9{OFhLmSkrhLc^bAgT=)_NMmi7jPo6bePks-32KSny(%uye^=7qUosur<=;=8($c9|3 z4zk==C9D~ph7Ug8BycY*lkzkt=fTFt#u`{|@ai{j-k43bu=-C>A#gZcc~>1`G|y8Y3uq|vS3i;HI2 z7{)rY74#*&1J8tye}Lj;J@TM7lX@Twimja;Py~|Z>d8n#nrq#OIVz-X?o#=%)QiR0 z`{IMFk8yFNDTV8dh?4qxa`r&7P(f1}Lqm>a&B7PU%NCzvV|#7lTD3dA+!rz%LiF_X zjP<^5uTlsXKBAUHcji`@xPpSeui-AKV%tNIl3lQR2_aUsI~X_nLhBbe)uf zqJo6Q5rN?LLl7H>8-SVkd%%U&qgSm98UmaJ?wK(nQfSwCIXO9|k@~v2+Yl_%yI*3lj&M)q&CLpb8Iqj|4|**}n$c?i zYM83U?Lj8*~*OE3eO2AeS#n{3W6=TP_mbYDdqaUcLDehdH<#JvkC#(r` zb8|vW??~40ot`$T%pn@B@++T7DSo9OwzU_wyDe^O65_04Czp^A18TFJMf+07;#i=< z!!ma=)pUKo>81XpU#U|BHY8zW{Ei8Z1gU72Lc~ZlAFX0F9cOuUcc-n*pg+&bcBN5J zM<6ud{d;dM+;VWj$~`qyCK9DEORI;Z=51(h+WKmc+6bY3E~%z6+%s~bZFc}xecsOMW#dL2A2siQNj8hdrOhFwi6 zYjl?=!c3nBmWzzecVA>=WY-UtAa)M#Ka_ z-|ycevBOl=!d|9H&!6vU2p3UtE`Hp~HVXlif_M3gX_JCaskY0h>#t9d`T6-pzYh^k zp7?p8($W!;xsF@Qs*^)+dV18Zz2O_#?CMfwLOL_W#Kd^NdbO^tzfr9^G1%Wvnb&ur5dr3&jwYdoh$I8me!r@<0nTA(; zd;2wcdH0kQE+OjuC(oQAd64x~LRm#yNd(nrwy5@5|nXJgb*RK(Un*L*BkG!_dn=7OL4P^C4bPn}} zH!3t@gh%Hw9DXVKPJ|HX?frPQ+nD|Ekt2~`zaEOc6)x%IB)Yu3+|t>3CVS!EAW?4N zzoHciwZEg47dx^tGLodOA3iW5T&VX*k}4`H=I{--wA_Blf@FrRAf>AMUgLp@38%2I z@NEqZ%D`IpvNB1pO_<21U8zKwnVC5b8=PO$)0O;_6w%4aoP@W90) zLBD2`=uhOpDh;rNAuK%>GJoXvAJSK^dN>-lCn!Ax6u-O;U^mTTTyDz=KCI<2-w43( zgsR*3(4_}i#4dbRKqq5Xn$9*V81E!0Cn8QHtLHxKEY6>aJ^aU5PvKJYd08QT!FQL> zcBcmf2YUfFsW*l#(BI#$nyT&U=$ltujC7|?RlR_4!C>r9ojP@PUX!A4TI~G?ob0_D zH^}WB9B7UlIjwbU+NJy`uYOHRtOEKK4(F_1?5t6g^|0~)WDc#ze=sT?oHg2M9L zy?2l1@L?!%Qtj7_{C9XsVzzQ0;9Fm*O*7=O@4g3H0!ilI<=o!%?0#SoB^uxhd2FK19WA9AHG8}2D`+SV2bg#eTW z;WN0(`Cl@3zVN=-w-b{lJF+609KUH8rw;~br;d5a+qbD94Y{3^eWeY->?{`-2?8#z zM4S6#LV{6O6CgLU_1XJd!B17`Jd_d{DdF7T z)kVt1#RYG#=`1TN)8=E;;bWv>W;UUp`632)27_DE+$?KmmRhblfA<6W{95OaX9xYg zpV{oZ9cfRd`nA!;ZiSO_NvmFLID_rY?75Eq@1aTq|v zRXe-O-@bnz-&~vNBxHFJkz< zm6EU^5Doh2*tfphJjl$o&c}T3`Jd1oWD;`8$`a@+u=^gfIozSZp*e@y4V0};P_=Qf zd^B4elAxibJqcO$D%^cdUELw&L?x5#?x!Q|Dq2T|hlk6mt7YXw1#SXF?w_ajB&X-} z_8_M}P^ihpt`KoSo;^^*L0|uK7nGN;Nrd8lXpyq6&-b5~l$6}-qC@qP$>-Ipg8}c} zv9sR_J3BHuT3OFkJZpx~y>;uAx4%EVn3x#b>C;y;va|E~Jkv1{5WLSs*h&KKAz*zU zjeKjL)YQ~%0l1}<t*6ZKAM|B|{V>iv;zIz9C zNRL}c{OASz{JaT-l57l1{gxdY%BC}T_j+W|6LZ;v%ri{CE^mq`%5I@>V@&YhD(&3D zloTov>sj$@*GM5Vq&jX{^yQeBCzk`LIK|Fh*4ar<$e&rpp&HxM?TG<_fo=f&PH=Nm z6A=-eJ$LTzY;k-%*_%_W0J;%ly+6Kx`*ub7CB)MU_rLD7FJw+jOUtpI>v@g7e(joz zW{RwZ#n*DI*QgIRsH0j#Zbc;>*OZfABsZFPAGyn%SfGJ8k#)YOsOVl-Cl@Acbzp66 ztp*BSEiH6k(UVTjw?RR}U1_>;@$nMk;)qMUyyNRot$+ITNqeFpBrYlGDj?ALzT8pn z$L8idj*gBkZEfy1f;mec^PO8~IeL_gbuX0Q2!tdlDeLV!cZR>YOG!!bIAd*J9uqvM zqoZSOoSU0FH8mw}WyLSgPJ4iy{P+m@fdi>J<@?(@Iz}KD*EBUL^CzEKR#9V#+2XkK zq4*o-yt$22P?bu$^X@7bBl4x^q9cY>M?i86FBXGi*@9GhOfN{Te?b_|W5{8^0CvN^7|IfRn8r!*Zazz4dM4r6*4X?En z&|=4Tc6M@{ck&#KD_=0^tdasEI zeSo`Bg@wnjU%xK&Z0fa!_>(7zj^vmU78RD_$%O^v)6g>*8!zR=AgviFmDWoY7^x+N z)O)i{SsL_t;jz9jxX{|y_QgN{?Zm=7=B0JP-CqRQn3$KW)c$?@bQ(YK3t5bj61Xg= zND#0c_fqU9PDn`boV#%D+yR5;h#*w0wZ{h>QAuy_F@$z{w>ooq2*Faw599P(7OPM0$zW=0hs$ZJvO!2CGC}C|az{tpGwls3}X+JQlY(^Lo7od0~ z6Wd2Bd}ybB{E#p;P00uT2xzEBj~;0^M~DJL^4Q2o^7?h3^2xe?sa^={FV#%hQooH2%52oEN&UJ(s=BaA3-@X z+r!JtTUq6I%t#7(>;X`0P~Ce5KkYzW&`@v{GJe82MQE)H;R*<-!~*WXCrz5B*d z!_cWOh7XlF!>f*)Pfoe}=Snzw*U_;c$85Na!PM{_$ zTi&dmwS}zk>v|qP1$f<;Z$owe{(Yc&JxjI49UZSsPfw4n7Zr)kcBK)sva*6>p=5cy z9tt|j{DX;!iBQ6k0>9(#>G=?7RqMG#l@ys$)_F4x_IJ7uiJ~cb*UaDO((HJ_O)z_!~?Af!Ad3lULY5==> za0w>5D^-UWva$Kbyv8E~gT$seow(w=PT_YFQodn>azv0ZOYM%x-HBm70BP@`xTa>r zwVOAONTvP3SZ?*q^=5mVx*nC3bd!{|0s`?>dHIIdOT>`Iu0DIV>W!h7^oE*-sN@I3 z$+fn73O<=nI_nmi(MT>J24S<68D$;_zjfx^xpQOdN{Pxomn|o17HDZ{E9-*Jk8da` zDMg5&tH@Y<~`XWPDOmkRTlZL^I%58zMxH+1c3% zIbkyT^wU&Z`uqL%k<-;=yKHZ7OOX=!R_;L%Xkban=Og>8N|MIh;(y?s3AO%sr@!br z1C$M|Jv}e}%nv3k(=Mg|RROX7^Ah_TO*2N@oqrGA{+?3Pk&zKXkc5a0DDDIY2U#(= zI@0tT8MkwybUY(MpTOWXnd-DVG#Ad0&9tQ7Yj=(86+3u`$ z5;^XiEiY9?jbrjc2On3WmTS)--mdH_$My_DR~lK*o+W+K@uf82q-?55zwq1OU}D4{ zR2~0QgI|1GKi8m9JQN27g@r_2_-_l>gSoWcX$kW2i6^$RUAe+^@3Xw?tyn2b%k)06 zHX?`r@1(IQn~9TKvpS6ozcsn&ea&VP3c2;|UH8o@cWSTv*$Wq>uU{tvLhg==iWjdz zQ%$z9gPmRJqf0s~6OCstUaW+w{*I>RD{%8%b_Q{jP5* z5~v^aPwUX;)>b^@_OHbi+xnRMNyh~yAu|LZ8eo_|%TE0DPEub}-4?Q4{OIlDLsL{# z#3m%9`bbZY0Bo{*p9(BZC#Jjr3UYXK^yKH8q3l_vgC;hl zpoV~|5b*ZxSlSjx<@xblPG>dsHB5bFf4)t%EpoJ12Z!Cdrl3#_7hWZNWEZx-h|ryO_qywaNf zEI=3)to8pa7oyMPTh{`|W(m@hfA^-pa0j~WEC?z?BGGgg)!gAK{dHu1!oE-M3PM_d zEH~W?Q>Rn+4ZAh(YH;sgT_yDC4?0kfH(G9kX7jLd@LP884^~`G& zyL|cRF@dKgBbC0x3k&`bn7S%aKyTVkgcuO?I2IPFz}mek2jdEC7MQ<&{YuTmMB!HI zvbky7*5010Sp45CWtOjjvZFF$vJ-T+5)J!hX=w>`EYbr9%7Dde1*#75^Zso|$bbr@ z4+sdboea03W@LN;YX}sKa~%pdo&G&VCe)d_q_5e4^3#t=7`}l2dwKLh%27py{?MUA za5tbCJrxw|L_O>LIRQ0CZZ*qayK%#<`)L(W8T%1k|CSRING$38S=&UOV@5!M&+1-s z%(Q^#>A{7fZBFO4M6F05*?^IOPYUjP2Q1P0x^);5#hg;Gd;rQFrDjURix)3$-@oqz z(pY3s5i=x)shOF3;i!L4ys>{zyrOrtVjv|xK=FrF+E`iXeE)otKvtYPfBp{WO2DG$ z7RJQvBM7mdZ-u{rLU^Q{3Mmbd3j|FnaXT@niw!3cZj?c&hg0M!`kJZlu-Q<_GdtNc#<&kw`g(NojNKe<|=|3 zcrjXHPW7A<2yf5;$U__1DfYX?B_zD*HT|O~Tjqz@gz!GElI&gDR@K?o=It6La|j}i zX~B{wa8?#mCpkE-V>hdj?*@x^)_Wfg6xdn7SPqN;0b{X!gRAfqRH36S&1SpB-}Gx^ zqvgR|%UFnj=E5?-ZUFnsdd?$l7` z&?7fXsX$SLE$#~_B`yJd*D1n&nj_76PHk&vYh}{oRxB-0EIdfou$1ESgZKao8>g^w z(@ZtOAC4S3vYZJRp?k;iG&}o?pz|tq3yBE{?lRFd*(UvrcYbTFtWRU+`Go~4*1Z?#d`DhYs#a6Izohp9+tb`kC1^UZcX4e{Z2*8Lo3j~*4WnP<{(j_?EVIZVu{Ck8tMRO;*3ABCDqkV*^UtS43v(tXJ7?Aq2oR<7JK?63KzNv3s%8n+Lauk|LgKm#)nNiQEU7rQ z2!CGri;HrihjL=&SG{_sp8oqR23y$~83f*Ie)2pJB*T_&s0mh$U1mPje)MivBiXUEIfG8V9eYR)#;B@Y|@lk@DR+1WE`0)Y>o^~BVaP;de(2*n){ z0su_HXnTi3yqH@t5}rU+4;7wD5ITKrqiq9u;Q|G$BzOR?XR@|-vuXJ+8sZbF<0V2l z>@XaQ%gd6|((WT&zm@&^VT6ot<#Q^q0}}*^J9lU_9p?BEF2TX{K(`2(4!r#K?XG&E z{gLtUamcUP3%8?7LGLmfDj~|q%&g63P4q(xTb%>R>QQ8}a20h}FVRnr1Y!)h z-GffBsUsmq>Zc#w+9F^EFR)wxIUqN-8_BtfHrcBDS^V#Vvu>jpFB;*rem{3E7P-TL z!C+nk=LQTHO#2WdU9+iYpg`OHhzU*wD*S?G0mA}VVW&6AXxX)a-~-jW(wD2qyRA)e z%ZFHQ2_^`&Rj)O+L7XiMJ0uSCpi2uoLQhx@kZTDw)EA9>ItZjM%!vslQKuH_INEWU z3~Wb>VAm;sbSXYr%gl@$x9Ei|d30&G_6)j$AXWkxZy(Kn@GH>U0K_DW%rQZeit{N2 z&w)VECWtTtbx5Wdg2e{Z>&1ys)3T=mCwX{i2%4tSWNZe`2!*S9aTf=QYbk_Icd#mr z!j`Wl+yD~4)okrVxH!UJWm%lu0T+%WWe?ot7uZdWQ@A%Epk9EqQZ4Ln8UvNpDBz8? z&0L~YvkyoVf!j_q{Qf!ZmPZw0_Z`fq58AL_sU!GiK&mFhZr~;v4;~i=ZKy2mmhW!$ zHubtzD%n!;lupF@lf=rlSiQdC$Il-$i8W6_NwX=keonr47uKa%(L080%eF*Y1@+$o2D5qhw2o zv;W|sb?Z%dhe5vta$XW<6}T`U(xJjQlfALFrUSA#ghT?Z`i@KM6-;KAmFj=fhxKCT zuVye22LRo&64+}q*W>Z^ z>m?=;YYMPV0l^^-iM4N_yW0n#SOa+CV2}tUQn-i>)BcMfcKG{KefaRf?8mp$41O|| zI8gan-NkR-q(q3560z=uYFhn++2hA&30sgLd|$ZGkbIK^{|_2qhyI^9Bikzy*EF3d z9bW`&_Q*W;ZI#N(&hC1!SM??W>;$Mq_MnMFV5>||d=gt@P+`Gn6$UetR)Fhme6WCRCLzpv|qT9SL;@sBOvq5h*l_d7s>%hQ{b^e*n954Pg`yE{JH`)oBo{@8g%N(W&m?sb|bAK>g+Swtu|LJ4tT?hYj z?V~c_n+TpfAmO4QErapT0{gG_mLPB7sI|c%1Zh~vd}Ob^y?sk>?~Dnh;{Qa_`E!jh z`t48GZ!B{qCTZ#D@PuG!|HxTx$*7p+XUQl{MgG5;?WJ(g9=>+DzBoEs>QhXNj}Psc z7f`M(E!T62FqOOy3_J;8$;02@|JwEIG@z*_R6DoLq$p}-ZYim#kVm(k1qlUI1Ljcl zpG+?m`47`jGTr>}KYExbym)^zrQTt~`afN~!0_g}|Nm0!81%>P4~C%+xs@WWzkyPf z@z^n09$qW;#@#~M`^uPb*a ztN_txY{SRL=izu=(DF8TSR-2>R{&Y3=ok4}%^^!r| zC9t=@$9hM{#z6AFQ1Yz)spJ(hKv_l=7Z0{`2*vOrZDx=? zOLD(^%BC!BKS=%0nLG$QE&@ukah|Y{Ki!W|xh!_Jw45k5dHU2HY7)RuV^2U922Kzj zPyl87S}CKU8dSYN2gSxoA)zjv3m}^{2Vn>zCHOh>34yfpmsv;#{jaiL(*F(FubrW3 zn%D5#DKO8ExKTlO>ib6hEe`q(U1y`2lz$MpH&S@ik$NMSt;l_)v3YdaoE04Q$FI# z{rB?h`Rh=nr4c|DmD%m@pmxvp=c~B8yOScynHer0I)3~(f%}sW6S`ATUF{pJ<=j1t zs0s^X0+Npl#C&NvxfigiYz)D$5+4^wt#D<@ouE4cCr+DhJ$DFZYQq@rM@vKF;!Bz6 z*Qc&S)t<|Q?8E#?A2WF zdX~W6?do-20tP?L1c@5x;bj}}myG=7iu%LJ{z12?nWU>;n3|ujdcKc*>_H2dGFaWg ztWAUaFwY1Lp8!9XGmo5C z23ZSC1`oTx>M=LA?s`h*lbat`|3^#1zcFOiM4-|wUz{RFsXJjKii`27#t~D4$JLUQ zos+)*VCo@38j6z+n-K&$FlfUAywJt6miN;Cbc_5B>;c<^uB5|<-T9w%3;`*unr(8t zuD+fK0RRdn0@z!|taP}IFi{6(-09vU;Eq6mwI1QpsBdXUVenZ#WKZ}lg4Kj@pPT%A zd^IV@jvX5wANTO~Cf!m~Rb_Z7a}pj9x{iYdY-4@h^H%J!bsFFDq@S*mIt)e7-^>tB z)}aJV_$&v9gry}fi0x(*4M(@K%)t`~@$KnA!BOz|=Gd>DgX;tvGiP;*RxvY_RNdX( zj~_qQ+}1Vx?FA@-=BPBQ8D%9Og6R;r@B>N8zTmI@7#m9hmXi4n1cD$4{DV+JpFe+I z3d~JlfdR)ne8)i~5=mG)|BT{&6HotUjOh8jFPr~2YjZBLRHtUArl<`K4IhG4g&?>U zV3+F;l5kIqjd=swCrEXm<4Zd@Pm>!kTD!%0W5?ilmxJk93) zWrCP;9X9&|+XwLe2ZzQVfjsKC?NZSxKRsq?>1g(l$uer1R($h7c=c7KEIcuuf`Vcy zmD2kF!#Va7Q8}#7dufM?-sby|$9J=t5h^$(~s>~-@W6dP)*}gvdPOl9*lAM>8?vjSDzV*cxzs;Fz z3NQ0_IfKvi_1lNYL9GQXs?mSnPdT)-voRGBoHHUVdA*fN=y7Xuo#nGv`&ECecS&dzSmf z?@BAC@hP%;2t&lp$W1Y><}jZpd^Hd^Z9emC4++q@tq;pPmXp^yZ4ORubDYl zkjAa7=#0-%rk}&MSiKjbnSavB#(#lg;~+bY-gX zdl9PE^AvhB9W@Ga-$FbG<}O|&Le!2{k4ijbn=ZthQ&&lPF3(vV6MxUcDk<+-Y4n5a z+sb$M%$W4Q-mYdrd}`O)622G}b%Ix*D(JH1$0C$gmyIvrSOz}QNm>8>@$HM7^CAU# z&*GIa565?J`z6o%a^g2_Nl}B3t=jj5sBB!)$|w-xdW`Se1h1wC2s{>6Cw?36dWruL zBY)f%3dQ~E{py|#;hq~BZ{M*OC{Iq5)ih90HHQU+37)jHjN7xvtgMrM!~Zxj;>r~( zx#jgsxl9*fkUEju6hG?Ui4Nda4aa7?qcb&s^ylwiv0BI;91^m-#Ca2)%*&bOwB&c} zY>q9>K6)<6?yQSU?D8cs2h8({h|LyCh;AElFj6p_pkTiiU%YJ9W@q=hc3jOoB#npg zvi3!@zMVll9I!xlLA?IRc{95S`P#qSVxSmNG|fi2WVI9xp@S= ze9HD(M;JMfLyQ%8>Rp^=nxZUrsUt-; z`r`@c&GSil>QdH9W9@|bINN$MX`FKZ!HPm+60?PkK^;MD@yu^=HD3(x)g&b?eJn#X zxCNfk`Jxy#keG6y-YCKG=T?M&G}Ygb&_22(uK#dTa{sxL1k(P~CyH45ppnhlOQ)L7 zN=-DkH^rX)_W1SkeEwrxtL!4EvfG%o+lA{N#^xG7@DU*n59aMdJO+0lDw7ds#DAeD z>qv)X*rlsanuvI88iD0eQS++cU9-aoS7mDm?5!w?|-p}72 ze*4J4EpGWcN5#Yygv%^`8<~@==3#GQqBKx+_97*f0=g>3@O#T?4zdk@7WxV9)MMNT z<)R|nq#TBlQ9I^q(eVML?%(olh!E`Of>Py8bcqpXYinE8I&W|N3ZyR!u?f#Y-2&8` zwaW&JbN$Av5Vg0>wtleC+z)*5?K>;4faje;`=}YQD-Lw0ySr1538nV@co+0`X#Mj| zv#={$KWrr3kDXVEs(f=AfzXXPBp_-(giX~Eni%)c)zlP1Ito3_vvR{OoRLtJy`#q1 zwy{wD)yY98M9=GB%h+V}V%?D>tIE1qGcyQ85~gZQleWfHM|Zcd{_yE$^xoNHMoyk72Eiu0)@p}it{zW=~&+pSB>BO}`1 z(_=d=IdgKMhDPTEk?oZqk6?Gs?x%Ro_rArV@&F&h z6;<28tp!=Fh^_5-YL$s*ue7^l&Bnba0&C3-strQN-3!*fGu5C1MK?0gG%G-Cm+tg^ z@j_Buyo9%=)%R+=q9l8u1bZOs>C-XE>E#i;pi@i%m#AUjwi^Z3nOZKtZz)Ehp7v%@ z0wZ+$T+nU&j&0_di&{lY;lfr4?G%j589yBC?W1fK2CUxUR?a~WD|E73h5Mm001Jy( z1x8M<{)k1Feg94itdkVb){CGKMt}Jt?;RWz#0sYaoC`W2ARzFbM|VFT=(nZB{M?5KxK5uAfFt{tQd+VBf78>MmXu&J6h8-d!Bb#c8 zBpzF?OElzT`~v0ijdvH~*W{<~hOcgKEck<9-+3SNGA&_yty7B!dV`=tb{OPl5Y1eD ze8|B3Lxmvpl@{SJPo!mKr8t9i$7%x1%*;wabje&#huUxI!aLa8>|VFw~wDgZK~ER|*|rI=Z^VP&$0kD%QM< zL4d4G46eWTAt9{56A(11he$j&-Rx*_axW9Kf^Zx6@D;50JW0*SkOsp@Mi%{zVo=jz zPTVsy`7d0&NNAckeE2ZwQnE(=2;7q_I2&|~js3)&Fwmm(z#A;)(0mgOTu7gGi{ zckmu+>5kJsK}qeL zjC(Rl;foLqCWF9(l-JadcBW|2XQ?E%KW_O*j(R#7jFm=n%JW-*$lpA45vbtF2(T1L0Q-PGXSy5mc4(<-?m$Him3u2#d^p(Ho^xvC zNyCbI2Igqc*rQ*7c}7w$=8c7=W#&6DMc;e$=+mMz(2ir%)8`7PC@Et=RS#sd8`*L|{?)bg+uUTATjd46; z-2K&~ufP9v3YW~6ok|P|%^e^IgEndt)L;TK4Zq7QW0WqU9z8k*mX%hB`!(I&Atrq} ztQDU!;PfukEg%q&jDRJS4+!PjA1ZI(ypbe~7PJ7evE@1M2orsTK+zIvT6A5?lOl3u zvY8f<2b>5EO_FNba|ps<3If7%uM43cYL*v?MDM3y5SNjW0fk!H39}jr=9JjF-_cNH zxNfh9q$^l$N5HSlaM|LN37mQ*VBAvzNdel#+ZmlyiySY5ZOKYAO}hqoBA|=?K`1^6 zK>+&0zvV`Bee(4DX4;+*wAj<%e;j%s31`@hT@f(+CT(l1qvHx%|Mee8$4zP=L>E_A z55LOYzj6^6Gs5F-TRHLMR|4D-|`MAlJOEABP9S72DKQ zb))Uh&(LNKyqC1UPILI^Fh;FuFJYC8*Fz)|qQ!grKH_rQ6L5H>j3 zZDX!xp(Ts<%Lnx04xVyO#!^$PPQlvKqDY-028H1ff;t35!&{F)e+>2@y zn%QijGcX|xxf`=^WbUPVMMOkYA3Pu&gm7HMS{l4Y=F@E)@SD+QB>u1f^mz$GKcB~s z@3AU9-GS*XDq)7lZLuf0FJ7bsm%22Zf)6cMECb8G0+&)Wa z85tWJn<~(#xxL-f+e-*G;H~J&x8VY>p%>h^3dasOQCDX=sR+mh4zxIXAv-;tcyUwt z6&%2!3s9p>{a{vB7Qti#ZQ{ctBYU8ANryTLEJd61`3nS1A2>v58JRNhM8bg!s(H^i zigwl?^g;HJvwQS2u*{;xH^;5PHb&^Sf$4@gP64|P^bZ+O0{6oJ9~K}023Y(b*XY+R zl$Dj$P(L7p>Bm?BSOJ%rFNlL9kQJ)njD>`^Fqp(8;Dd7Sgd=^p1q3|P(=CI(*?w3B zLxT2`CvbdWGkHDqv}zk0Ct2;2s@#Ovo(QfV9vj;W5%+A69B6Y&=LxZb4(vJ}K0eoj zSZKED&Zm?>x=3_{{`bS0or+r(Q|Gl{BEdjK1H~*Hl7r}}exdm4)hjbgOSjO_V_{)o z?&EiRwB+#I{QUb6h5owy73Jl`a7;<-WyHOpWdnWvX{S?B@JiulQ!bF(KnmCc_FLi; zg-&kIpHm+{?g#Nh2aWg$fw&LCg*ovw$;qKX|F!&mXNjdhD2+cg95cFCZ(*?igre zZA~~(g@%EFl!SytJ?|Nr+kQ$(a3t&K>+c6z3@--J(Hp$)(|z{=zYq?qI7~+edG5w$ znI=RTt4+Y3gcG3vYK%ggkXmr1FU<`W1R+Pk;R>KGfm?_IQm`9v!ee|1fzDv70^x&j zkQE#=q66m`L1$KnngMw8HUYPqXzAlibxr_PBCu*ze~=`z60G1by*$tz0!?~EkhTvo z`>Fx~4*Q|de(lnGWwe+RxBcp4d0+;OGSw`$0n^UFB~)Y?_i{rVs?{TQ0o(FPKo0~X z2^=6qpuv-f0>Hzk3tgAqV1NL)OgPd2l)s}Tqoonc5+?)&1$A7~01p5nl!S?B`F!&j zjK=6#ZJ?0V^x^+O+?#+y-M4?kqq@?jl1gb2icpDYkr0wlBq3p<&62dxB4d;#k``-} zCR=ust5A|Dg|ym*B2<=1$Ue->`}wNt`rp@m-_QFV&v87@@&1qd_+R&Rml^Y$-|u^# z=Vv?Bhdz%-cQX-p+U{2}zgl7b?1BkX@PctT8dYKCgdr-Ue8N-|`~-ZSj%Y85b?YMB z2Y+nG2=Jw)MH_NdOOZF|`~u)Z)J9D@;NoJ0{yGNSRyqw*Wj_+8aFTD=KG+5$pqAFl z`udx*r~L89#EB*_G|q%zeHkT;_bKmei2ny8yXj$ec;d`-5@cQSa`!G zDJFkK7rFB6nKP!AHtGWVAK}k;;%0Jy1zl%pL;hDNJ6;wRCg^{aTe-5T_O$#rYFt?< z9>bS)bx~<)Y2+iuo29z_Fshkw8BqiYo4qGq4+;{h{qUg_{i<)tg5PjO-2wUxY5yzh zld2paK76u5o%n_2X=7J;kMNkYW*>+Oy_wZ$ST#W_252*+g@H2R6IL8 zCvMSmewCpQyMhg@dr-b_Ad zvzYX_+83u+4Xu>^)zf7jHuxDfIgMj8ATL6~3y{w@1NthA2=Kc5KO0{8LBjx#1hK|} zUqz-=JSNy*GkWU8#s50~>d{v%_;oXqS3a}){IA&-zwGWlEg|W?@3kVY!&iz}E8tVZ z75{$8R)x8kz#Gk;2{cHaV5Y{Xaf^Tgtu;yfmVa zYJf3)OR8D=9pmMno0~775Kqe78K*m$(esrLbGO2V#W?1W`!$D*cBfA1o;maI7!N~z z%7b)6X>YBVQdL34qT5#*wrV^6J+7Y(A5OmhuhaUImyWUg%5bQp7Xxr8fDeL9wzyA8 ziV}rziwK4p?W-`toIr{_QkOHbXwW?^T(rp6(eV;G?NK(BpWpI*gxSO ztX+CnQcdO1b;NE9k%lkmInGO5zHWr^6})>l9e)1mx*DQ|4ALDh6(`y;&KdEp?&r#I zy4-=r!(-@}ZEZ+|qD$?ce;hq}v|t4BT85EIe+&!^;NxTF&Zs<3D)g*b7myI}yr^g* zS`t;p2BtiXf+VZ+h!p~QLIMooPnnv@p!4?3J6`EF|1dPeXwUs>Y|aUhT3wxfvwFa; zU3BhT!u{RL&h|@4SO$45vp)jU@c%AcP;(c@Rt%lbTfJtDcTUbal+5wCb0us({S2(? z3V}Ym421+nXd`9ruR&jrU7To{u8R`Wjg*i|JuWdr!z_UhX`E~dJD?5%W-Pm6 znSBrW^E)&!W&<)R2vhya?r*Qy>huJN&;IVu&yuLDe6x|xh9?U$GBUamR2DdQB}s$8 zS4(0rFck!|KO;8l{Nu+PCkl!AU|wsc^GC!fBpnT}QH<%S^vSu?4_q1-6VDSsg@i0F zI#A+eBi9SzhvB!5Zck#jkMX0syIT_v-fGX$Au;Y~3-2{IXYB~{L(j4S?t;}H`9fb=n37kmT7?wN^m{yv z4^7{{UlyFPSnm~e1C2{B30^j7>eS~fKG%hq3QO;7?(XlevP?vEfZ<~nPc}@yqz5<9 zejNL4m^@LBe1VgU;_jzV9$95%S>w$~UJTNjY>b@j>!HX)DJBB#ZEW_*{{(bC^fqYB zTcgvpui(-gdEA}j9$9r2P7MQ;wHOfwPhD7CKlBIavODq&kHgk(ynHs?MTO|=|+-azb z&JRTRp9EsJpph=EJBmw56f>9!Voc?-Jl~L@69~AWc-lBiM8x*sL0`rFMS5fHTROjP z7Q{Bl?n2TMna`fW2l1e}YNf`$ho175&095&|D^@^2kcSS2Ww0K9$93)Sbiu5pmN77 zwG27}#~PvTA#cy3%kxRQM5-G3)V{vk8f?&BEM$1`c!!htq+!VdtC8;M+x(DHl} z4JwQee?Ws+miO%!Q1i4!d<}|sIj83v%sLo2nu-nKF2>Z}Q+t@Pn{!=&0*^2n$1SiR zp`$0kMYxg8mBn{I<3^0QI1f;hizC$iIINga;CSMVw82HhZ;2QJScAvo1TMxgY3tyy zCtI3z)|D&Q8cZ{*aN8A<1BXpe@Cr z0{k8*RTbRljg8CZANfO#1@q9kC66Kl#M zJn`$=V<;#XLA`+~A8K&5^nTo(C5?@&a!-hP+M`c52shA&zeZFh7>Jg%*VopTel+B%^(Zs_H{o3C@V-UUZ-!3ciZ z%ASQVk)Pe&OC)1qW~>*Dxw^yPwY} zP{rgYdGh2*$GQiy+i$By;wiY3MQ#G}kulQCtyr^NuL`HdX$$RKi)z!si&iZn@on{ ziiXM^;fD?{uSvLKcpImQHh7t2EEH0u*Ce$4{gt3TnE~>*47Fyf$;?F!>^r$ zXdF{3s})hyf!`IEoGQGecb?d=|JEZi(9J{kD}p;N4LmUj}1F?`Vbg-?fP|Lfjh8( z86)&1Bjqk2cvCsXY=HWA5eMlLwbof5abk&pQg?Hr@4Ux@o($M5xWfH5M{vKLt4$=y zeIogSb#=M=akKCyI%t6Ea$K$giC(Dcs@Ljn7a}I6&jQL_;B@-*X>cq}c-Vq_Kg4en zY0C2T-I8#9C+&X^K#Th?09sblKLNBVonZlP|K1-MjBc!RSqw}B^Nv+Q-&Pb99DP;@ z|AfWv!=saj6Ig2g^Hp7!e>t%M|Cu;raoQYMYZhWM6bjS-1C2Q9Pf0{(KozsHMPz0d zy$YzNK~t7~Lb-nZn>W)aXrdn2m>(=JL4_5ALi+|xhU%l;+Lsq^vh)vHv~ez?M+%-? z8$WT{%pb)Q7gNJZ@urnHhR~a$D@g816`7W$VSKvS#qB*+Wn1G|3E#n&HQ(DmlJ<^n zmohSJ?_c>o0w5U@Ew*)eq64_NuHnphagS~R_rA86IlDkhe7GmEaN$CqdIqiyD!U&D z9^hR1U7madhZCqCiNi&hyGaU39wwUIa#DlT;~(ixOuktTx38a`JdBv7rvQyeZ|cEm z2SK7?&1H>F2_u6!+L^ZUgsH$Mh(wey2v1wyccH$JoG(nyBv-x%VQrH~W(JPGT0ew4 zlhit-umuvAp~-xbCx^tIlAQIXl}S6LPk^pQVD9i^xL+(>F&PEymaUiG)3N+yqa7ZFA`eXvvH19}?+!$auGAl9V z&5qG82ju%+95k_O5B!?kJgw&|Q~oXlaS~Q`7CdHiLbTt{U*yD|vwt2v-w`H0lSu_o z4B$mVMrLd4*F3-SoSdAkNWDZ$PNIe|r_aFX#59RSz_KqAp^2#%cEi$3LSo93R$4yL z)YqTlKS={(_pC#+s&}(;)oL}S^|LzMe`YV0JsWB{onl{|HXjYqMLY^k$V#8}&2af) zuwEqQpwXjO2>6H}p0gNGWlvj%XCYU03UEX4TunIg(t4gv(C(a}#wd2#{f?@#`PW?% z!0TPX&RP5WCehpOkxys}5Kv?UAorl{qv+^GUd0idRbWlgsLcJ*5B)GPn84~TBq}P3 zm!IrC6OVKuk{T{zo*@%0{K$YMyL)?|BjLLT7$ONT3+^rEnlBt|tYfqM>WsdR42H1V z>UON)LrqFNfymOUUg8}(I^V=|V+GN4uQ-XhYtNp@YLPv1jzAKS6qS6Mk=qO)8xUS0 zD%ia+$DC$ zckSABCIy;{!`CU`NTZ@;NIJD&PmJ!alG|B)bBTR!XQRJO$*| z0n~?sAotZ8F7iYR-o6#W@MJlRnOO^T07-#F(DfJoHcs7E^kIt`RRE@7F%%e zVDz4smx%oK`EXqUB}6)8F`_TKd*5rcAk&Rt!h~@{-EX@6J?ajV`?{*CXYZ$8^G9_@ z=`>6P+^vSrRbBU0`2op4TH`E6^oKp(zj;$RtXBlmY}#VAc{tleZ$CB3Kn~0?H@7Va zyt?-tCUpC<*gwEfl!0djZ7`K~8n7qD3&x5TM9XVp3P=dBf~jq@j^*I8z$V__O&{s0 z#l|DshI#2szCj5~H|`l&lGWh3N$-q@D&6&#|01f_51tDc6U;G9gje|U^5touv6USw z^$vRid!Bat@rz~qxB$`*u2_$1dTa(*?Q`2R`)r#fyR~ZSpLf#0eJ%nu7h2WiiIZp&#Ap>Z%;ALBEYQ zTo><{_TYBR29ar>PD+RKP+>|63w`mt>H}!%Lo`h}z8v3&42F2*&6^eQt81bqlET-pZ|5gupq$4L(!n3`~ObPzd()~y7rA6m?v``va~TI%CR z9)KK>lvJhj)KrdqxCV?uK0TF51FQahXsOai{;T=9iK;+EQ3H{uzH9U5^VORI{OS~> zrTu`#Bhw5+%pE`u;5$L>$bTA4Qn047j`txXYTGS4P`gT+eEf0vzVQARGM~b#R z_}O@q;SS4AU{u0}>#!d2qj46gU}w+92D44PHn z&@iq1)SOR`PF}>5*ZyR%VGOrM9KqYjH5dI;sx|tmx_U>s5l==ROcX&RwLUnY%Eu=V zRNVjL)6FACj=U@`z6dpX){aByb7Rr=U3>IsLv4M11*9Ev+uzx>J=a^oUA)QN7=fMG zUl0Q1_~}zkoJ6ylar#cUQ_jMJHS79NiOmYk10GSUZO84gupr|IesA}o?wFZ`RK|0$ z3&f~ijIl9XAZ?%$wXD+3DqyqM!O0nKygUgti~zL~tHKGjX~NghK^%_^0DLgc?krRF zATa(N^*IVaxjR75zrDTbW-baWR2r#`GDdenOvIr^LR8&WQj$xT@8IvNnoiXRZC zIX?Du8X1+jo6ETRwJ(cV`CinTddk4S!0oPeCRbTZ2wX!T^cf|uUR@$gfy$8i^^9je zsJi7~Azv01`Qb~l|NQiP!IvPF^CJo}pJefZ$qOoK$2~6u2)DtQ#rCr2gP1im46&#+BcJiI%ivwyV$Ux%027 z-ssMw@eeAjvt1qY;6bPj3SN3`86zP=f(AmvGShC0(6^34#BJp1OFggZN$aR-Y-FI< zcy4KwnCqD9xu?^;;VM;~zD+92^ASDY>LEXW;q4mUNM253?`;jUH8!DLtnb5;Z^DTU zhDeyeCo|vfmFwdqCi0btUhZaSC!><`uZ!o69Bz5` zTQUc7VE1GVozVY*{|me){~{akKc&w42Wil~Z``>ivdI9C4gF}#@4q(iXG{D3+Np;@ zHIfkKoSeav^AG>}v&@J^)W~0QJ-kt-T+_e4ZNM~&@J{PiZ$)nsV~tHb(U}N2EO_%K zZAS}C?BT{!eo-g$&TSxXECKF&2^f6JJxI<-n-77N0A;MxFX!96faEj?}FC9>p8p#RJb>KPRi4UhQ)=2g#h)+>oPL_{uD+rCs)UyK{ZaT z0{>qANVs@|O~movzrudEKyjfGoAux&-n-rUeX(p#sU3y@@%O)__~38+e*b^}sFP<% z|1C>Q)BgwC^8bFh|K-&MMgD$q)e|JOMZoo?Z zef|IX4q^|MZV=dyy%&_H&z@~=)m_-SzIemC<5y1JYWe%WjriOkt;$)n(8g>Ji6jLuJ?g`ivHf zU`uw-XyJ6<`q!J-Q+erMAE7A4=O33r?Z*G=_Wb8}W|Q>H2XX~s9gAf{%L#qq=lNgS z3S&*>Btl9ci@g>cY*JHetMCx6uBXMti5d5M3c!W^CdjhB9aQ+G{HZu}Y_%=+O|?}f zSKNPP5R}l>-W~v*6-1ctQ&HToS)j^o?Ty7Cvt~^^_ln=y$fEu?QE1Ri7MOKl%zP1Aq+S&og`spBon3`>i$Lo%ewa z!f{sRW;S+r@f5#s#-XK0X(&P&yM55paVZzzY8;3Pw{Z3SMouV0*wJyfJ0D0Vhl8HxlFB}XB&4EpCuP(&fau?S5!q2kbfwnj$8t%45jzw$lfhzJ4F ze3z!UtLj>i3lUHN0s#BY=f?Pr&2j1~9{a3GNGYK0DeUiKj}=#{?THdedcl%`dFmby z?~$Vuk7~gP#DGuVzTJL?YGOhHe8{5@VIW8ok_uYw`5&~91A8t~(lza3f37E#>@tF=?egVJB^K( zyg+|>2~Czbu(XSin1Xut{{H=lL`IqY%*BhZpqqI9>eUor47o7P5Sfr{Kh+vTY?=o& zYV#EBCIE>LG%_-RprEGhs!AqjO{m7!ty>XqU{==(_~HuK8Pkm}=&Q*$OKR*ve4D=a z@2^>`c4o;6Cv+2do_y%~2`2vOAQqr&&_WXm>4-mOK_auHcmY!SjHEV;yfcc>@u0Ck zO_EcQl#_O<;WZxkJ2(`{mB96t=gv8UfBd_Sbp{1Xtdsx`5?I8RmSq`)U+XiB&H~2V zmeWvw%v*b8p6clzmqCP*WKtHN_jo21OF78R3Nm5=X8L)`_A}86XARgU@tiCXTmz|t zHz+azsAu()HDs?q&x{uFDeh}hCfB*#p1G#dQ$cH(2wKtA&xNqfDEjc&c`Y5?@XQBb zCJ1o2ZyEwc<@P}NN~i^B2Zhj;PZAQUIvd%K5epF6HK<>KVA%`l`TyS|RS1J6k&Q0? z#fumFP$~7W&m=5t4p2BgrZ`}RWwGzl=xgCn)Gz-eyq z2QK!^pNLA|?)?0VtWZ1CO{Aw390GDL-oAN5xGLmenr}qLDpIsYp$I4JF@eDV={ppj zJaJ<6{-S9lz@CI`V;vs8Knev9nGWs>ID}WG(UEBe9-3&-G~X^_>G*)VltK_R?teiF zsoH=c@8#rNL<|8GAmHlGATZt0IhDzCssn@zzp*!%DF_6AfA5O_!AdwTTHsgLwPiU1 z$Hdjt)O-Yfsou>-+O|QgC6QpVjSwQQ!7j)`KYWIT*%Fh}oQ8{jekNTFPvkq9u^_XI zL3s&f#3_)mqtstm%4ZPEhsf|l^pWf>y1KzE7CnKBom~|hyYi&+_=s^y#>ES+KpPQ< z`^{|ZmH$N6bps(1Fv0l_V#q#VXQzeo0DZ}}WK$C>t6MYa7{p2rD7}KzX-rXR!pEOa z@IqLJ{jp87zAQ%QPkhEQzz4oVI36*ue@*@w6Bxr0c};~@RcOAFkH3HZ@V5n6;eaUB zpv9Y_5D{jz|Limfba9rOVA!!oh>m3@urfkKw?Qk~2W~6GYL)0uvT_*5~93Gysy)I9-pdW(db{mKxNs22jjbrQMq$GbsL&N23*P2u49ZT$(u2K`d!fW4pl)PdgQ+Wc0`XRHBM7R+VBo|cnD{Yw@ehcgHKn1 zfvg<}P9>D%V&srckKr&U1Na0mxD{FxDX97Im;Gjjl+#~y7`^!m(66EY$$e4jHtg>Y z&?*^kWeTTvLg!eyvZ!j$QzF%rB!y>XxrOO2Lk$Xm5tlI2NW~@@{O|f-NUJD^dZJ+X z)PTuZE-(Kaq{PzJOuSfu)6}Vf`t-=k2p+n9NSwyww6)!XOMqQ7iCHrML^iph>X6hM zfI`@ng=`gZocSV03vtd*5SG}q$=ZZC035mI5g7v_^3=0j%sW%-A)q(i&e5<-d;Q8@_)5+2!jLtpa zuv&m!mRoL3H;aRoj!1qGtWKHq6&=Y`-cZX70t1sL5y!g#MOU|6ye?LO4~?&mgF~{} zO-O$&o$8_Zj{#2yxd3DVNP#WSHT1-QrMF{85q#S_Q6vE#I9fk45h?CU+8K8YD%m4Y zRp1ydL=l@oDVQ@I@x^1xM(}k2WcT5@KLbF5;?Pe-*Z-mKZ57Pqh=km_W!0X2<VagNz`myFTGbOc8o|yDW5G)2%8;66DhIjk(;fG<<{+FnF+T*`a z^)E8L2VaJ0q_qXSzEM~%`g+&!*Kc#E=7vP@s}tW3P@Vl7RA=Q*%VOmJYDEUYu9@&I!A5F8KUUtq4po&z`(K?$jSIx{jy4<0zM1mHdt)t26R zxeuJ{=aT493M57XXFOy%nAM0`p~=Q|@7}!>FMN4~@xUMgd`u8Y?yDhHXjZw-zm&rT zY-?Xg#1#<0ELpbf4;>$D9@@!XXzSu~73c@NHv4g0C7Cl9V-jBV!l`*^ce1HH;A+$i zZ+u%*LlnYmo)J7djSV3Kvf}_VwSP1(ui!s~H zS{;Bh1U~z)*J5|wm9gu;+dAQy4_Z09r(Obw`&J<9ESEi9hGxn;<$>6ABi&w?q5~!xiWO*BxDmn1l9K&oN8kkhF5WCecGNi z>1u}$9MF^wi9ySwn!~i_57S`d+kI<2l#jYCgF}@hjbxEw`Y!7NZgn z`8ISJ*!s0iJ|9*VA^;F8!T~T8$!)%i7cB-o(2zRD1zSym2&aBtHJWl{P)w~4@_G7iv4Qx>&P8ZAz zc6j*F=iT@+3O+KFvrb9r0*X9QBIEXt&N-_lB>Qy%mJ|@gn_vhCdknk6s9U;>jx(nS z)yW%o{m{rr3k(*?*GoiS)8KP?i1cfc(zL?j9tAZD90p{-YoqXsjY6`9`14}WRm8j0 z^yzftoNR{LStSA{K<7bZ)QCKOfy?JR1d@o*9VA>4Sn*Q;-(FznYxEk>^biTj3UE+p zgdmq9(rBNDfC+$zbpt}CoEg=&s@Y{}L+83dgi6*+%u{g;HA_=%y-FHqB2z+PI}U9= z*|sK5>l4N^y)9CMurKjE3b7+`^7O92;gLwJ;xi^p7MXwo}aM_QPPAaW`P8E~N zz~c-~AruFN17Hi4;2cc9!2p#Di;JhehGYp4aZ3IexwJv50snpZ=~FazOQvSc z2+}2rYq%H+usVftIZdnjW@#%;{<_g`8?cMCHzwtzVD96^aK;cyd%)Z}Yl6=ODPY#i?3 zkjTjSDvp)@Xvd{q#Aj1+E0#Qe-tuS#^$}5uz!`<7(9B6=3=A>UsBQF~3@i(C;bv3Q z*}7L3bKh)Lw(;vD6Db-9Ta*!%TaP1lVRT1>0SlmI%^utL#+Y|8hGm}VY0iE3YHIsI zCM!y=5ps<({8pgtQ~XZ+g2d!z9Xs9}bcD|e1O5S`G6@A-- z#mnA|HGGcY$8L`Q8_T&ddpG^Z+`s>oSN8A78vp8(jWYt9^#=uc&e;^0|3KDtwC z`O%%5XDd&gXS}c_f5OM+3pM_+da*@IJAz(5IUXpsrepn~3jv?iV<#)xTIo&LBJhW# zNPz#;*gLEJwj?xhxlIY451yE+JpRU2Gt!u0o%n7;S8Z-G^7yOwkwYSFZ_W-^7U(?i zheQ!RIo9rbA*M_~L7H~^%X@sD%y2&=>3h;BY3EU@049`w&cnS!#Bs;4 zcp6?3?XG-m3GjmTsxBM1xE-2E`>K>VpvZ*so$@k9Rb=$Yar6BqICIdl*so-8i3yq> ztf)4)mVE~g{(%f1s9h#eh{K^tv$=LFcJzH2Vqur)!^e-GW1L1+B0@jPXPEr7&O#dt z>(**C4|j9Y4!1C{+--BF8fH@g0;D*NAYyxIA#4Uxm{kOE52K0$50XaSY{x>EUGghe zI#-{`dix9*zw%x+zqGF0wD;k66^mK%@jCN%q4gk*ITFp#ggceGJ8k)6^+A$lE z*L>{6i3?AjY%-M#)AiJD(1_BejU*-aUtG z_#wSeJ$RQ1aQ?b>H7p{_(IFxW#6R^`?b7F=>6KWs=KhYJKLiC^0Fl5k%INMrqavt^ z`iV^%Vz}-Et%B-v^7|a;dTCi6gmSi|y1{-8_V=DA*PWCbM=Z2_L6Xp6rh9>ZR89X@ z*vwR*bqaccc%9_D1GN@{VbT@LY9%8n5RneaV_NBA{0b(F}BBYoYRKy>YTFH$y`wU=VR_FaHDM^V`|7 zxa9|0wJr+_Z}J?yf%8BckjMcChi!`&FAj~0l8Z-PS~&rOTHy~PW==JFXI4!c#F}?- zYXw`alEI~d%wDvdY4u2p;zL%?3SG$r&hr5I+hC?wV}fCSR9L$<9Ivt!N``{$vbe1k z(1uORz!gQ|n@JG^rnA!Go|iw5ehv-6mKJ#h1sQbQ`a5>a4Uz&m2B(*k@pt?%IRM2y zhhnPx2#rS}tynhR@hVXWxzU}u3uevIf?8%`;qK?x21JB+o0^tF-me7_(fRlx$fJlc z1NWvcjVQ>b_+GfJc2+X^9U5Nu!B3|#W(NaE*&eML24`T^z~m`Yi1Kb1S{3}0=`24b1lFhbzOSi?fJ(a^x&urE@>us=mPAdXnat147qVv5<-Kg7 zT3H!V+R%U$WJ5NBdLF$rhf3w(zgO9eI4$`eV=im@vmTvj1uy9MOCwT+_wqeqz zP;`8-uXr*v5mobZGMK{TBvYmb!E!?uuwlx(-oU*b1eL}#FzRICN`ptPe<1@~jUgbl z8y)Hiem!k0wp#i}BTGSyPHS27fQypk2O%HixXCKq+94PbuAl@G3G9bN+;6{^{%mO( z??tpn&}I=IBC$hazx`x}$Ik%WPiPe3CnYmx% zJZ0)sMb|cCVqHN-BWb5~T6a1FV8E2|92hpb=y5ndh-Pkhffn(ESQII@uZRjmI&EX3W>UNkDN>Mz);~ zl?O(0V~LuPq2W?IHgdAEJLMCDW@_>kRAn5{fGhko2?oh_Z;1 zWVhf*i5+&T3%!1QGw}*S7Qr6pMIpKb(Oy9e%X&5WK zlA$ei?``88umnd8W*-`E+j?Lv+oVHOhN>oWG7QNA(QXg4Gs=MWn;wM@Y}*;pzcO2$ z|GkBJ^56yBJ}^OwQ1A=iCBWN{2YC`o9<)sx`C_pinZs1Z0D2Io$aq@XOPhJP-3vAN!$uU% zef#coucMIe8s{5_gba862L|0FdXGH87R>U(Ozo@u$m3HWYpKnd240yrXK zQkA#0wU2gu6-zmF4z@U-Q{(!bsLpPY^el(H3^kq+x6_5L4HCZ_+Zz_aOmhRn6~#Zh z5ntd5zH&IUT}0~kFq4Tfk6;ZTgj6gRjjOezaO)`Ct+#4EQauFz&%M-uGt*v9d~cIBPI@gN4(p$LnxX zEFzT@m|#_Q{f7^lF~*-EX}9d9^%RQ2uUo|-#Em_#j$k9;SJE{a@5<-b9nbH5w~=5< z@Em6TT7wo?EjyR~RwGEIXXEkur+j-n?aw0l``;@kVZ^;m$;_9Tj83;<~9gd_{_KxSpca7b$XvB}5ZK7ai> zbu+uP_@y3q<(8U&l-v|HCu4g(=SvU01K9{gfNd_J1n$ROHU!npmR|oS`RiV=S)o0f zFni-hn2M5m(DvO4Ufc7LN@^d?LHMZ+c8NpVFT~hQ;svxL1KZwpDn<~{T3nUg5%=72buctak zlpMepPzb=1+?7EdA=29~Ao;dh$=b?Fr?&=D3_tAXBwN6PTPp6tmWZHAEj(_PS&j*( zshLC0M~-~{{Mie)MOnxs6=+4!G0bSv{8&096kv4oa6JR+$*by|N%$VCYQ=M#&q4JM zw+YD}K~pl00ziBcbgt$$Xa9g#*1hFE?6d@iMN}j-fC6MIF2Y`<+C_e%xkgR~fqjp; zx=x@3QOl2=K3$A&X|s;%jHehI!DpoOh@4-f>guHxqX|J4hLreq;Sf)mnwhPwt(u|| z|6m3MZYcpZuII@8j@*2wifr}5Z>@pKJQMWokN}fcCum0efI+)BSQLsEf-*c295MyIXEg z#k3c}7UXtM}}0PMgNt1{v-0fb=HeQjM`Fiu=#j%c+U&_eG9(IHy; zL_D}AkVfim3UX6w!p6Qhv^?xjtr*jw4p>6kTL64fiu?UGyS2UIe}6VbxQyf%ow_GE zw-I)}B;3hF1Bpq3i2l+ARoymW#mWQP$SKfg%b+tRk~nb4w^LG6ebEiPEH4j+%*5B* zU(BxOqK}U~cbJK`#l+-@T_9=l!XhFnKvEi@en6`;k zJI(DnMbGimb~1Qn>*(=sG$cQsKo&X~6|vwEeT?g@1X`q2X3W z57A{q)qA9TL0q-yRRsc>SiF924)CD@N@`*`A)YXtMBSKk*PYh;OHk9{W%NJFr8i4F zo-8LnItIl|E0FnHk)(L&T5HDvl81E#SZu%#(K&Pe(9mE>`?n3p{uBC8JB`R_(9abg zZST#s!}$&wmgA4r6X*lOwI7%+3hsP;8hz#R)hpUggUZu?`p7Lq9`lZ9F>nh=cV8e6;A|gb+M|V14IelpNdKAvNe&`w0-$%p%`b?BdkZi{5>qyA+i?_#V|`fYys%r3#INB#@Q<)@-_|0d z37%=1^Ew(0;Wo#x_n72`Aa;icI&l6%E)W$@yR3K|>K0(6VVlnq8jI|tB8wp)iTt>J z%;BF_0>YGdK=JVlpi%L4hV%k5rCGPTYVQjI2`cCXY(jJssKEfMfpm~vqn3{liv-aG zG9e9+$ubpV6g0kiDdJ?S;%Xc(XjPsgG6s=A!sVE>msovE>2#V-5@hb}6KL@0-Hxyw z!hkvs7j^)UD#ocqlG|{A6B#}ZaIIYIGr|_TO&ASD0V|OONVc{y*KBYdq2zc9rvW}1 zL{TQG)D`NPys&aU#N4AV>dZmR!X!qr%80eZEY0FPupniNBX-4W><1!xfqZ^U{)36a zR9*_IBIxsqF%+r1GD2^TSe4(38(f9-&8^H2SRmJkA_@`HdKgrR%OS{UPvJNEuC(`c zV(y-o`mg2>UIeL!CZ$QA8CBDcb7`A%vWw@PrV-YLYL>cPr^q?}#6A}55Qa6bO$2iT zS*L~pT^B6pqP;K2P)FU?5+5+&l1P%0mR41;Dg940A5#dR+`C|K2dR5A9LUzM5 zv2k>$;RDf&<&IvU(ZJqV#f|1PxKG7^AA1~q)*YN}6fab- zQE2(i9$zH;{h;~o?rzei7_l0NG-kAm&r_H^#aH(%iR6Lb5SY|XGc!NLQxMh{qRlzBbY(}QeBkgP%9RYS~x`Zj=e6P*n*RYXN)4!K(!4UBEg{~$Hra&dIvE* z90aq=#bL!zio8rR3JPVaStcA7{+v>ZZ>jG`VqiuO>M~N9E?v1Yw_J3kr4C5Jj9m-s z+SnSILm+Jo;Czvgj751b!?Y6MZ~ya4$pbL$lJ2TZbRiCy%Smhv5s+-|gaP_J@gr+- zaD>C#wr672&PL94C0B}{o8O5K$>IqP?Mw@ z1R+=l=DCsE*q*t+~a2q;WXn|~*~LAnmWUaF*1SSmZeFTD%< zaR?^^Es6vUzpV1UrxeNHwLHHoS7XG^d0u+L3wWY2Q#c_${#y3n!sHh}+2?4fF8myy ze&$t+krzcJ26(T$vlXA`?YFZ_!iP8Av&Z3>%~(lPvt%0n3-;jsPM<&*c*@)0 zU;rL?9GlS6M*==o8dJGIX+(KHs?!&scZ0!4s6;W_sb|JTJAQg>- zd|uW8p)&s4C%pb3y7mKReizdYvQ9xjdZ3_L&t8$#G>hJ{^^rszNft_yx#q)g?PL3z z{E&^^eM?|K5ruj|Yx8CSJZezUrSH4~E95(`zAFBaX?^r{$Jc!M@mW{5WBc91M<>Ab zXJIH8rr*+4=LUYs;Gs^=x1A&9$7p|@leCt^R9an8M{RzD9fV82v;@**gbQ-b?5Uc= zfSC-A#{l3ikf4ydNxoN@B2je6@LBRCkSqzRxnLS?}q#b;e3OJ zTsJ4OnHe}FNzp)+^9`AaiunT5r|-|7Cy2in2^}l(P?L?X@6|`9p?VeJ8(CIt-Z&hy zy>r9f5ho+?s$wG@3;2E(47*OJE5my#H0-)&yiJ$jupZU0@_R&`7@#sd_@E-<`S-iG61)u#K}Yk|l{@&*P1@q7hrBcvmk<4e9xPPm z9(G*$@S})m+qJ7=B7d?<<^U27;!sl@Tg*pXk}(w{@^&J$E>IO^KWhg5aRD?l(i~0_ z6u^6IN$-{R)gn=Gv@!vcxrbW;6(GmTY~cPaYl1PVyTe27X)1f zMmGrE7D1jcgaJW-qQUJG0i_&-V%VhbCk9GEz% z<|SOA)2QNwbu|{fyI!#wp@V`1Qt6Pf7Lo|3e(&F|&u2&RHG7$;!!!;NsT=T}tw1Q@kt3HO-ywY!PNQc;g-yykUN5*s1dtBb<-h?S__jVn z*}M_uZo&6i2W2}kyowQN3OF8;0=S>2EG>)W7pp0_R0YUqZ9&L<2^pjY5xLxRsE{T)@23^sWl`QF#jc5h31k(X2BIxQ zi+m4RzGN~)Lq!fz5`7{n8Zog`#=0-v1@I&(H-wzlPk_aGItEagFJVf&ei}_#Xa5_c z%o>f%K|zXrWBw-80FsLX`?S+7%~?|6)JFErN?wf$-|1XKwZ;->53ycu7UY{Mhu+-^ zfcZ_?{t+26ofa1e%~l`zzNQ1`=(`0g2-5OT{G)T?4nlW8qZf$piAYLX+ESF)I;aP(*_<4o zEk+n1aFak>AP|1no$IF&)trwaH_$ezRy>1xm(d{ve9N52K z8wlpyRaSZ^9!_a>N1qOjbg?R+6)u5E54Tdg0)05}SZ|TvlC^k(2QlT4+!Bn!+tFGp zqZ&3%ysXpQqToiW-o5|5Q6^9L96brkh^0@QIwehU%aw?=@U&*Sn2bg)EUVd1y|Fl3?J{9jM<0sd(v0XXi_1r z>0=O8lR^h5#qr0ndApwX-ao#ni_1nP7M{ddycM_z^Cz#_4No4|8SNxmzVE|Q9&i6A zd(ZHaAJ!;--={JZj(`xBu(SQ+KgBp@ZX60fc(6<~ud6G|BM9su3C zBv0$%sZ*vvuR>;f+eR=d7&xt1p4KKFsM}bECk9HK+x^@gaO5D~XKYk)iDH`{<1q zK^mwQiY{Du87-kLiU;gL$`B&2TG)03!Af{vqN9Wgs;RYAMy)a1`DXkOS=dataZpZV zsKw;a1N$pY6(Yxpf&4AzYXsq8@Es;$;fDeeGT3`NIKLBkw`hLnL+T?AZx{o)$A%dG zR0jitf=IN^PH1=UIivA~sNtGwO}YYqY>FVB;<+mWl^_?e9 z{As@s>NRD=RCwDlar&T7<{+0S>n`Z95DJjX;Cs$!kSfvrrjIzW8v$Nnf?T&eL(K01 z!yhN^1!BhKAP#J4VWLSgaTFu)NbdW6hz^h>N)EMcZsB2JWdsSvF+@^00i;67pOS5a zI11#6GBC1$MD`tw=W z*!89T$$|MpNM>-rRJAnXbuy8|*q2F)0Zh2YBh#^4 zLygbZsPX+U+Kc%dcD^a^2^SCa+}}UL=7~N<87dQMKtC&#SC*w7pZ1;;st}gn6Y-}y zd#?tL2ESr+X|B&ZN&6BrJ&ElGo0f8vXOn_9|B zB+1~)V@sa~Fcwq|;n}m7ZlG%ecl{R#3m(NO4*kwlXTOXtvE1s!&>qo1IviR=mi&&P zK~&sL@;dLU?}IT#po;*OZWI9#T$ZyZ&4bp``1rI7dsRl4iUknKeDKg*Un%p&t z7lAy3XaI;38CVnSeQ{Ws2wJ^xsO@H>9eV_ZnGcUn%8A}@+ zXSn1KPb7)|_;7%m>cTem)411vawPk?k5K;Zr0=6N8C-hb&~O{P<;pKtuUwIa<{~Q>CQetIw>@cy9y3Oj@o5DeOESpX z-^Q_DO@PuCLK0m^S9NcPG*aBbr{D|XVA_-7^Nz;R;O~PMQSaJvHKjYxUg_CqY|Gm6wHCHjt;Q#zPIZd^~4YkL7s4= zMooE-Tco72G8A-%PK#7|BYpcYCpPfIXvOJ+T0x)mn_S+6)3Ga&;q-MyO( z!bo~|@Geq_z#2;EOpAbqfx-AgvfT1+#)91{W_*1II%F@zYYBiL<&z#-Z#59(WtU)mqmqJja`AmO!83T1!!nQ)m85htmtfa8cD4jlB8qOgUz9n7xLh;j{7 z1symcI2a;6iT{8Qfj~<==x!vhlCre&VZfOuBl-CXuT=!{0=Iz-N zfa7r6B_k?drHXB!{f1Q(%j65Si(7Zi;8Y*h4pq?o-oK}Vljg)lFDE9*FkD@VLk>$_ zcCG-G80g1r!H!R0nrrz_Tk=hfSwfgLo+bl0>~rVP@vO;>@fLw_YyPKT9a0m#VD9O zOz4HkQb7O)1$tCCY78!IPT^d+nm=q^({KHF8m&bpfsO&Vs8QuB#4$$uS^0RlHf@}- zfRnXbpATV%_3yK+f0-L?tr@w}9lCX*<&pcq-@-lZQzxVRw1TdTs!#OjFp+3VHO6lX z0sk16Wr$i=HkyTB*cPn_4%@^pLl4?v&q+y1F}AU}f_}LDllclyoG~O87jct6PtU`^ z_5X{kg*inLF~R5@ii~tlwUr8Bg$`hPV57ke7I^${J0_nBPLkDg5T~GX7=MPtWgo6` zqt)DXrs+pAIL$6K8H>=RUg6&$WCG9_u||E(rOEd5s8Wp)n7&|*$^Eib;db)lENibC9iJ#j8<|Yr0ByTt8uhZeT?PUnn^4<1m zsKlokVQ^V#B;|3gxzNS{xMJFUCR^S2L zOsV`4ITI09=wRY;7LepkP{UJ(3!7(5o*aW$bMR=*OG?r+@v!R@2epTV?@pm~fawr= zLk0~ua~oRGM4Tws0t2TKuZYRxtKlp5XrceUi!;>(cIhWSU&)i6GK+?cdi$mVU$L_o zqR0RaZ!3aArKG&^ItZ`d@}U*KJND&l2ZW2gU@LOZ^hj>e(XkKxiVhO0-b4_I75M#p zIx(3_;?XCnZJ6~-0a>{=)ITjL*>>Ww6G-VGsN12%VPM+8@f$?Gc(aWBt`!E(RA9DQ z4U_ZH3%=samYPck9SK(O3XE*+h#dg*me>x_b3BBJT#mAXBXYhb(C)$dqQC?-7aVwhc+K*JtAh4uQ1Y;>EW{G!tWC zq-JKiV4Y*|xg0;PiFx?&25!6MNCrcC)KY%%jO%B}IYDE4>+v~9{C@^?pu ze96*7=^2GV8<^kU)ky792k=k6=IdmxSiim$)X}sQZFAn?8D{?e#MzPMJtuRB$h5&V zkp#CB$Bx|rnc}diG?`s8K{6J&V+hbw#2T3p@hv`}DK@s>GNg<({+*rMG1u;XEl>1P z{?&{|4mi4Hh0@i$C~wwbag}w8L;{(4dAAy9gVq>bq;X#mI11%%+od3XVs^qjx&u!F zc)jJII$9E|`nYT6h>IJ6)2qyO-Yd5L*b?~GQ@Cr4;&~NT*cMUdevgK9SFCrEiZs;4 z$Wm|$-4ImoidoL};Xtz>Eek_lSIYIq=g+m*-&1^1@4`wi8$`;sR&MBmJKG!>8T3=^ z`}qmDVts{ltY)7wY0@1)3?T>!#({6tvJdO0{|#6>@yYJ1!F)R2iI@^_6Aj#tGO0Yx z3oInK_$~m8FAEATz`-KBbg2&e?RdAhGh&MtiDwR{yU+|aL@rtDX~%8B)dEct4*?HR zA%lbsgfmKBR$Z1WgFz<~(o!@T7(r!VvnBC)kO9T~SE$mBWbW%5wp?F8$` zsFUIaRxN2l*aFvx% zTw{W1Iu5Acdut8+{1~=#XkNb380h2P6E>Dpb8aqlU%xOP#iARVhn%MW^)K_*O!~(k z@j_)ph=QiIwdSMksKX{E7nsQ(;^K^N(|@lgEQ*Q-zd>9wfCb1t7K}#7Vr*D=ysDP0 z&Iu{$`%&>*KC@FtC@wyJ0*-wA2OvH{YvBqtHZzm{j=tX?uA~@@AHd0pmlHvf@n$>F z2I7n&8~MWYt1918N*{6+4vrE~k9@YOtJ^wd2&7Ht*3coVpM~3wghS#AM#*REiXeF8 z>C>m*rpN6dZkMOnumgEGaV*?1Nxp}@;S^1rLfA5%iDhYs>LyGPxHrE6aOzO!w|0aA%GS=AZz!-ly|1T%ADk!kW}Umk5W>mz$NU3U=$QC z65WpPYf}4v5va7=4WCaGe1ZAb0s$q!MWIo}nK~VLM&5!=?d_8Rzkwxsb8CYLhExj9 zeImvHl1B0Bmi2_U%6I@cj!$RZs{5jWeEbZ_)W(ql^chUQ0K_jP(di1F$z-yF;olbe za^j!L=kw0}KdilZSkL+X|Npj(-6$E7Jw&!dSwfMJNE9P$lVYhi>?w#t$rQj)z< zAry@=ma(*&vL{-Y7HwMI@89haX6BqZ-|PDP@jKVKKG*roNbmRS^;{m0`}R-_9WmlN zX}57+e2)-Z#<}UQ`Exo6By!1V_81O>ka)s{2tSa*hU@#*DX(kg8 zxC=~+YFvBKhO+z)Y<7_nB)L}xKQI|C?Iw?XZT_oG3qFeiiRhN3aepF+pwf8-%(ZIQ zH%!uh^U^-T{6Lq-0iCY;vvtds+tK)ny%sYBYymL>*6Cezbn3wGNju7~RcqPOalD&? zSiCLMdyRq|fRud=x1~U@{x6Y5YsF)cI_zEgYP4B|Gk+9a7NCe6VYYA z;%$M!A{x2Jw8#gqE^N%&`Eool&?Avo@Vyi$e*bo)(9^FnKBQ0xgC}E$cqA+%Z=WAF z{#lTisshy>3A&VQZ-hjeW$d9*ySR;e=qIZk|2MVB^>O-NgS>24;N^ndQ&&hhw{{@0 zPZH-%`C)@ig>wbmO9H|T1@!--d)V&z8=fIl{M~~XVB=lq`_|@Wopepq`F3i>4DJbf zdTD+W&=pJPX*BN1;iC`m2=*azj9=|qFnQv{Js3bGF5I*npNu13{eAX+S~0y~@c%PX z+?B-RSmXa;1o?3!0d{gfLDQnkBp@V2hUqQ8&V6fuTJ!ZfY83hRBSvhwXnC)U#CZ@7 zt-kZT@?By*>N7;@ z?Em#pi0GsR1MoQrz~T?YKr{7Jt-fgino(^JHT$A05K}^r>86XB5*U=cAocu_{L3ht zu{G2fy4+0>+vUllC}M`dzh=*$Ul+F{9L?q!o%SOqSE7x(4d40d&p&$wu~MHO`=yn( zQe<>EAZ`K36fnMMHjJ8S8i|v80s?mPwvKTmbR6k*6QRwPqAz9!1qdDII=Y?%w64&_VB51SoaBwxsHOZSMNYb>gwE-p0Ar0`j7fpQAe|7Xh1-F z?ahU?HX;Z}nFV;i9fgEsW}A8Nr~{*-()N_>Xu8wR$-iG?b(JU{q0xV!!j1<@^Vp!1 z1c-j?U;X32jUlCD27vp+zRzB~co${xAFq7sN4VDre3v)BVpwX~_Ljhr@C!1O6Pwvh za;^|GHBD$0s6y@P%Cn7_^CBt z|8|x4BE%w*I4Nxgt;~YP*aTB;G-=|?O4?Uez^{a7ye92IQAV9$SOP;_YhxYNA3`7e z$`%Kyj*SG`F@LCM=oU(rI2TpXixuEY-<$~G2_kwCnggO5RPc_tz8Ggpz_`Wv%d0Eg ztBNO!<^j}a)#6dke|I{|aN9~K`hcr8FAG2h&Tv|7rTCCcu`-I=Wgr=w?-6w~l2RBB zS$|#O9B-e=UteR%V6|Oy=&r&DCX^{R#D4b`Aqzo9HP)`*Gynh17k4|o#0}BY)3me( z{WUo_o4Iv97AfBs00aFBVMGI|#yk7zjqG&*OJzN5|D!gqX zt=+Ie0%A;|yv3Wq#n6Nrpc{!5%Xo*}n{v~9opmQUZ4|I1V>a2BI~c=>grSpeBSafO z)<&Z5Z=_8~XZ(tKix%5pF;2V29gofc&@MfT;82F2fyHeDjGnF}CPp?Ci3=oVhK^fB zVJ*AfcFeKW8e9HnNo;9mwWLoe?j;*6Uof&Zq#Qeyt0mq%wVvJ|RO56E?u_<&|AJ#=86l)#BK_BNwT!)U zr!F67I7uBVwiQnQ^q(Z?mj|A6JLqbywMV7cBv29L_BNM?OeN$U^j~B9VW8maU)j!L z<&K75k7mOz#)oE!rV+ligPvX!y&FYMXNTKm49pqdVKUiJZz7-Twc6CwoQ?Jf!x;ATjU0YJTbU5<3JCE-o(qgS?uLIX<4$Adxc;8g%W} z{rAeRNxGwxyLQo7U-F?oddG8)_?t0EoLS+fb6&r~>R#E9NB)7XP+z#B`(6QP;v_^t-b_M*A}eZT6_u8Q4XmzJ$V)AM5`C4`9m`0rKasWI62+7Fh? z4s#J_K}~vW#;nS;F~w3y+1GtO%-6g0h{sK4yU9OI}7(zzidk1m|;dN$5hA z9n(&ckuWP*J#q-6jbQ-Alf4lrOc-}?Q4U^ds3-<%n71--b!yPI`teWj_o zr};<4bUUS%jNC%7z;_ks72{Kfo`VOAtW5&i@MvAcj1^kdKD3g5#ndY~At`VXNtn?= zD1I%s`cGQ$JCqM+!>neM$8bGKYfs&vvuM!Q&ZpK>$~QUnnJzA4DDT~ir2zZ#%p6Yr z-`b(+;l{cjr{S;zVcjCbGDK{Td@%oRmjKOY{pgdATW(F#oA%QHpYK+cmDWw zg%5Ejh#wn)!Mq2W+S-U~T6}zV)yLBlWzbk|pp#lC6hwfKFrPp4HaGW-VNWdJPB{h! zF(}-BKQd_Y+GsMK1O`)=42gerOY3v@O^vn))@G0r*tu!>7Z3Ubw2*I=iFascke}%e zwt{qoi#dJjlv-qRh>9c0Lvy>Q3`=Ey8Qq87{L-QwK3|}b+fG-c>2y!wxQlb(4xV3G z_Uo}i6EvGS&#ztHV~iyLpmKjuR9xC9Gnv|ombLj&5?p;`NJVC9h+AODr`C!YSC^0I z$h-cT<|cRgnlV{lagC>+hJC!}@l_&?Sf-2OW5C%|mwnG$^L)BgIy0CJ987T&EUo~J z1%WONv(b7lxmftAurOcs^xCH@r|v1;D=$o#HMC^*WZ$OE8YTIvHCtA{*_b7zhD-08 z#^^NAc6@u}zO&O@C&&HEr@aW=o*CQI|JnW~hSp2#w-~Q)*-34J`2_We`%Ckhxc}TF zqsRM3eS73od|sQs_Q8Wy4Hx#SxC2+TVRw-(=Y3Uw)~nP6pNFkn+U%=g()X zeGW9qY1H1t#Ln?>REi38LE_9%`brw8TrrZqnJ*zN*iqQOROM{gSm*EUZ|96UEulBS z{b*=8qMqm~K|$Wo&`HrC4iu(m8>#xdal{oTvDZecKXi4GNo?rXkMA~!gqm%S;0E$c zG|UKa0#~%Qj8J|(6D@c4J_L}q4Q(v8(wo2iRy{0-7FohSmKQk9otyls<+&$k4;9wx z>S>-4nHHO)CDPe^jUs<336JDh9Yz)ImumI?`1#_zyu2p#G*n)%0LrO4fP05{xn!%Vz8_I?beTSI+SI9X2HLq4d?$pah_E3Ce&UD* z#%jF(7=AKE=FWM;aW^&wC1-=^x~%5F+8mb$I734abn@is#{DN&B|lVA-_Dvdr#98} zjZaMmU(KS;?XNF$VpD&*le>->kbv=?@QBmQ*xhiUC2JeYT`iNVg z-U#{vzJiCS_|Bef#4xxz+b7dTG1?lCNaQG}Xb@B^+HKxy8BB1a}cltjD z*rh^+mv?n};HC$M`!@lOPW>ER5T5{s?$vr=Q?!ZRNBrhbr92Sl3`Vy8tACtnKK<*? z9HR2l8dPhmFEKel&c|3vG7%j-n@{+2F}O z_0j$gr96bhk>PHLA?pIfIDJ?YR)@diCjbEl28$32>g+K8{gz+9?u(~8Gp z&Liiqy$}4txV8W2nv7omqdm*^_=l(Wq{h*IkP`prAM(5FLGvi5DaHs*`t5t)d$;-4 z6M2Qjcdj+mtdonFVV)6TshIU8&oj7<)}U{56*L{Z$ISi5M;zN~X5Vjkw5G%V?;rR- zzU9%-yQ19}xqo1KPaG4t?;G#w_*@cGiG=Vz;%($JRfPza09|;EB^#%YmPFFO1iA4K zA2wV;S{dk$}q-S$=LPy zd=xMP1`fQ1-G=>VF&k>4tDfEkn@U5CPQ=5g8Q+KK$5IU@u(G?^5I%W}akqQ+?1}14 zfW!u`tqW;FJYZ>JaoPfdO2|E&s#a82uOXNHW6RDq2`gx!fGlhPgTW#6iX)=>8w7hj zD&MzHRsKxQn+-2CW#*9cP@>ty?0})5-_rd>R-}S&(j2gK)`?(M;HAjOcqC9-oj;OV z819tZKu1T%9LXMJM`JCO%zyLqOP-@w8_5=EpeLVB2CYj%9nPX&9t-cgv#DWsIsQy; zisEqG&kLGrboL6He}`l(_2$j%fUHEk-dg4+kJdKv@mUxwr&^H9+fSeF=A~t5QT8D_ zAX}zIGj<}jg1*xM0SHYz#qf6kKwH7Ky2rI5o}V&XHXL>vrai|K6Wb}ciNuhS{Z<#e zPn``2^j&ZEpw%DLl!MOvIp3GnA}xf_#kNzXkachF=pI&t2ER9RhZrJ%Gc8REz;eEQ9Q#oeZkoS7P6Mz+Z(`=I!C4DAKCj#&2L z*;Ne~?9F^x1V|E-OYzphj=qd{PSRjqiEiN|Nio7{yf1f@7Yw2l6fkM6;jplDZwjj9 zqR1fO{}!{0AD@}5M{vduh;Dx6E}Rs3V^wZ*8LHRM)wwDLGzzhz4$rHAr5svyAvjXm zd1Mba=VIu9DFVk54fs^a2PSJvV#R`rf?LO=#hZjxj%jHZmo&e^)B8fPtBYr{6|qtU z(iU=e39h}dMlGyropjx;a~`SPxu65=J(9$9px24-2ZU$lQF=)*49}qFBH^BYWyT%% ziI0KtRu2!l?AN?U@!0U>5-2xcJ}Q4}=VhyGJi(2d^TUWFVtk9&7!i8e0An(0)qSo1 zU=4_sr4nmTwRab-H;c;eBel)sMJu0_PXWkHFthw zacLmg5yu-0@6&5*^l-#>&ePj(!V{7JU*tAYXv+{Qx$H%O0tnE_d$&wX^Q;HNEC#q? zW=uM29vkdpf{V%icHt7wLc#F#g1&3m{Bfep+v8WSeE+ab@FSTdOhkM20OP`|>IyI$jS|KhUD-2V_p>B%A6h&_XXs8*;uX zo@C$Pe$F2!&zw09#e(m?ecCRBObfyndrbLCDzZ>G<-QQLYa++ZIsNI0rG59lEBIr4 zargc1!+opo#=WPi=8E85$RkGDWEtpJ@7~~GInP9Qj-fVbahH++W9RG`gkrI|2riD1 zPy%gr;O$`6_0+Y_WH(Yqnp>D)8s?IRd~d4kxQLinZBx22#VdBEMZ;&IiP5ebUPqW7 zS4)V&%tVj*B+Ow{hiC8zV(b58+rm116y12taave97*Xkh%>k@&2RzPBt#3`!b?S!U zdoTxLKem`22W}j`cI}mfM?sdBmNgX1uTIZiB|_9HNmRLWXVk}ls2>t*ESTPPtc{Wz zbvNye3;D?sGlE~MH})~VkJunKBW3e@6cJ&E{1WRDC*GF^({NHZpZc&Wlaq!k0!mCxS~%qe-4oLFFWa?cVc z;kTB9!eE~Hv~sKWn>>@=jX4~0f11X&q$22=Edc{kDzgjAB14v>uQhUA)hJ?pxNU`J zL|$TgQn0f4oQbnAaGdYAl9HNXZ7w6fFlTNxd4Lf3$8|1$04dPrY@&ixQ>PWx$qxA~ zqGIF8^3P}*F{GX?$s^)taw~?Ib`T0011Uv1L10^k~yBa+=PmHLD zp~<$mVyhAn8uiu|UW*HQ8KXNF&Z!VK4kup^S^R*t`R%0`>D4waVe! zTQzUaRQadJT4Zmvep#HJbbn1B_Y>c=0M(D&kVg*O7tgBkRc^LndiT_96GLEyZs`a) z{tFVY*&0qo1{$t569(`ET|5kDqONyT(BFuL)3a2>?|?CW2E+0VzQ7LT0o#=I=-#~{ zmTe=IsaY%Mj~#OK3X1XB`$wus@F?+Qj-qid?4W>|Sk@(4L5m z?litC;BMEhUB~}1b6N!>I>boL!csT3xqLIgR^>$0Gup>jnVH`K#M~_ylkE;VM*SGa zeJ?--Io_euTQ9V*dzr!f5%ZRItuDD5gyvo%CS{1ost&oKpt9yKtUq z%U)FZftbp~H;5Qzr&z zPfk)Td{Z~+eT?pxLoLPNOqE?QsOB5AgOc8u<}Cr^P~U*(jd&`M;g_t-#2rVyE_1a) ztODY3DxNO7G)3P&yOx#PJ){b&fh0N**1|*g;8=TWkmK*hRVvlSK-KMBi zj=T#r1Y$ly!az2Q&edJ^_rEZs*kP38~7?%{dwC z4!-K$Vc@`)RL~Qy#tri_zQe?Yy++af;!1e?!=0)`$Fn-Xd@EfEUXVa{W$^{~*@XO%X4nDtC$Je;W!QNG??+Kd z7fvem_Nz{guda^|UBX>8yT%q08oVT6!@FS*Zi{WSt)gef2fBb^R2B`3Fb*D)!NFWE zl4eIaJZZ`^G*i{MQ$M$`A|Me<3Tz3C1rufbyb1rdS?rJ@L+Wq5XZO45M*D`1b_ClI z1Sr!L0EHy&j-`;uP)y&*fnccAo;&F;?*8$|A;cM5dWRLU%^eehMoZc=l~}X3ZO4HS zryguKV|xVYDm#+h8?S2EyJ@zayWx))W?B*`SbM1LQfDwY1F$+jXQo@3BZL@V{(&}) ztjQ654vqb2IxFYRg&P}Zdvon~?K$hRN-gdS>0NxdMcXxEMTP z{zj`#?}@24U!55JdrdEefN(WQk)x9_z2aNxPR6&y){t%4k*-y1;PRlfxXy+>Z<+_) zsG=AiQL9cP)INi9h;FeAFrG1YlZkHC7>%&9;G`uKfXI~h;8n-ka@SE~^dPqgDTfdA ziEe5*lcs6Om!^pYC*xo7K#10YawQvY*}lRq!OEh6ZmoWbtd#IY+ECx-ONSuoM*fw+ zCh~hTp}%p;Nyi!^!}8v1`Nz)>Ek1KjweVg{hw7X0hWQ-And{$BDJxi@9@nl7+OKCP z=&s>gmpY2;`Pn8`RD#I=r;(dP(;F3b3E_YQbjf@sa_-_EHV{DS>55yY-u2r?7ySx> z_btRETAe%UtuKE>!h^DvaPo-!$jDl!{S5#*Y7;w0azP7-fw=UfPhnt)YOo}eNelxQ zqdh>d_mMW=SoY%E<}MoXK?7IiOmvvfUy&rqz*>VE@~br+9{u1qYWNpLuKan&U;F;e zuqWQ_Yj{W)eKawqE3c8F(6Ljepc>1I7KI@YgZQK#f@yS^zs+#pf5SSv0^SF!u4iTT z_XTdzZr}b`*rkF8uk*UZYrTMBdm{d-ha+#SdvX6xP-t%H{he-Zp8^jceffW3KqpU! zhc}}@E^e+ulCx!V*tT&w?U4$jCGc&hPo=v~8ep2V@i3K(R}8`!`n1 zVwe@VK_d}CNif~)*xG<%_@*Ln-7gq?4cryY7>w_+e$UkISlN^W9#E|d9^dsM9&89a zEv%#h-n?CYHQ)CrRq^$zlJv5y6V<|w%Bs?r7b7Aj?Vtvl+x1z@Q7Z{Z0U4LO67~5j zs?##^;r${yS#UK#fR7Bu93?s3>zukW_#OntOq^h1Pev%Dbb!T z%^Atf2sDisT{?9F`3}ifa$+}ASC=%-Ig1yMquwOy`*^b@B$WMrP=7N+GR0#pHfn4J z`x5d%l3+j&$joF6l=O@eaZyi2SzBLG{m>{iwRAHq}nfL>s z@TD<~$V>^suOI=QY*YSOgcxxKgP^VM%W`1^j@}h{rUv%?k%M<{0)vsTU(U0 z3K@jn!G$A>b2r$=IBr3%yZy%%Dy6FyDJV3Aw9HM!;|=1!_~LcPF?MH_j= zgP$~!j8t~gdgQa&yiP3W%tK@}xUY{-$3=D%CJf_xubw2oub*4CHX1b^P{->{>eU;s z<$hF$e|CS*ORF5uWLzqtjxK{K98m)&!asR9W3?Fr&bbmP-i?d{IVhl zGIOBwftPC83;QyT31Ia=iZf^QC z0aj?HsoC{<072xdK0Mv4R8}m}PI%!S>}6*SY-YHn(9yv&V8hvO%ZzU(F+`N_82zSzo4mM=5j+o01ar^GVcy3okY zGrY82Smn-jPwwl12KdFCm#hP%;N*Od0R~J89MLfNt_LWoq$XI9x)g{Q6F?(GBp}Zo zJ+jKK=!n(p5YQ&g$C?9c9|FL52oJ*r8!h|H7K4KeZ4b>WS@=O#WJ%&>_Z@zA7F7gEU}G&Un?5|Tbf<4&^7}bIJuUWm zTHJZBGGpgcB?~_9^Jts4c8!O9oD<7|co#8$c8r?Y8}Bw68Xb*nnl^7f5?kNY%TIjW z*kF$EvauDMxlSo(Tl)F>c6AQ=U${)3r-zuG4CdZz z_SzNIzKHjc+)n!O$9yd%@3j zFPerZq|K1^zEV0<$*MDjN<;k#-@l#$igCw_^|?KncsXG1ym?I(<=oGqyu6uipXXMe zG`!J?h_Aho;R-WDuAe>;sv7Os4n8^9Cp#-^!DdSx^U(q38>%!6U&MtcRW2-~ zTyO%XRz^YcbRpm@`0h<=FI`KW!Ok3}Qc+f;e#1C>n!iB1bm+yp_ibL^@GThg0Dbrf z4%dzf1IL6;>nqFj)zvZY+tuDE$2BeLO0rpUn04U|QSx*$mz{aJn~7r}-D8QB^UFtq z8~Budh1E94>CHo%D_ha~^;RR*shzkIH56%K4%_0LssF_752hD-_N*5>P6ih2OH}Kw zN<3<;Y@{AzTa}zz=HOX=?&{N-VfNp_XFb8Q(Ai^jc8q<-frI6XW8!-S^xL|?#I=0> zjOc*mnhfWfnf{^ZV-p-K_y+j}~l>RCn3{D`&Bc~TQ6PtKqp-S_hy>=-?uo%l!L;zpvxbGD8io<|V zWo%Q=so_JBBFHETg~H#^NBMZ~f3mA-&Pl~4VI`rCD$v4u70AHHJUn2_+iK!m{PJMX zJG+-v0jY&!CcVr|O)>$mA3tHj@uc@C?0idxtogGw?HY=vQKSJP5H(b$$SxM&&KeA zF)Q)Xy$$uqR5pl2PUhQ_kL^ zPOKqrdBsKo8-!g277|^Tx|*7T9o}L34#L%&!lH;!k4+rBx|MU4l}*K|+=W)9ZHB+K zI_sWgt@0k$LskB#>iPjYi^}3DZedDC2(jy+Tt7Bw#Z)imC4U)LL8s&!-4Uqj3GH?# z9i2CzeavV3j$EzI9m;&RaA{l*GXT!yVZa^#3N1sHsxWr#t}zG%cGpmx=I zDMF0C<7f%C>z(v{43SfGAg^+A8qg{s&TnsMIO&4CbK5Iut;jjaEvn}WKgb}w8sHvr z+^G66GUs;Z5qjS=E&huf7OYAmc6XaWb*O>;Hs6tBU~2eH!p8K*b~PtHSwgmM-M`;Z zV7yX|h3%+oL^)L)lXju_cWTWJ!AeWUHywM?_DoQyDl0w4E~v0Yvu4NW030eemRI&S z`)pYJh{Nqyga`CEvGXI^ok1MqiMs(6sf2!QEYIkO(v(_6cy{cqA|IkkblHg!(^q$!C_TBxpIuvHWu2CGPp`PF zR?!(d{bEC<=4rPZ;Gx(c$+MaT&NqM~Op#qRc~ND;$>{V8VfX7MEvd*(G>N;X7ojvP zUJ_b7Eg<7kzbM}e=Tqx$d{=T{c~z&(a_g4`BNn-b6^{Ar=6Ek-;f>Qb=9w};XUf#6 z34%xkz*9rr8wnlgz}#JD}$H*M3Vr$=4N zT)cg*)WCq|1I1*G<#|j!JElNhy>jI-8zmd!D4ZxtEmAGMPFU@-{+GuOaj8pau|>l@ z+o!-DGYdjSAR+nilvHG3Jc?W8c@$CgOFPmg{fzD}i;ea}i5*ui`J)XReqg(nb31wD z)AEwk90Yw^PS<;iFGPBg@!iiprqfg!3j-*IyZ5}gfOr4+JI%O+)E`xbC^Ltn@)4gS z>IAhvTVAjXFs$WiAm#3g&jE6T@QluQPn)J}JV_bFX_b7sbvy(yj?kg+?uxS}ZWd*M81M6Fam`m}Sf%IUAW>2_AS zTR8<)=7FiIG}^MNez(iU1}Q5>98gY5&8?+xpK4{FdcdrFx1M!L>O{BD_gx<4l%3Y2 zOZW?JZT0fsXsh=Ps!_x&62r0=Qyx?CCe7{a=Iqybzf{Q5|BJ&3h=4T{5tISU@&9s&Ne272F2~d$}nK@*x?WPwmrP)RN40McI_hCoy#P#a%G>Bd7rm+ z5O7|!?g9{i3;Q#9OMS?c2aECic1s5t+XL#~Vof+ko#fsOB2uW1%rcR4#!s!WSxg?7Vbc_rH|5y zB0MdfTp!MjBavDwi?vkOaG+Ld)LbKAfK9j2hTP<_b=R+`1--!b#-r1CWJ1v3UizVQve4-)!5#Hp**vo@Yd4b&BiG}B(8*H)#Kdg z!jc!kH%Gt3U9;<}hghMBy2TU^g@=nwcGFeLSj9XNDr*Am1rOf3ZCghI9YCcvojcc) zI%>=OoF7RKgXTe-zX_ZM5AbPU6MY&XNeB{EH@_CP@eN8Wt;;B$hJl#KhrR3g)kTlg z)AHb0k_I<%|4F{YvJ5VWpF9PUjd+b8rYkaYI&kwv1G0(W0(`1U7~GYwin$LK{m$=guSEn*tKG!7)Sx<}LEpJnC-) zKGKFq6Wa=^m-?JFhyX|LT0_R4sF@^;p^{+&LMM_@k+k^Q($}V_qa>4htLgE_R~vm> zNwyB^i&rdoaeZL4vV0-J;vaou6^2qcS*{F{5bijal$I1rk@_QaFHppZ~ENbj+CIoBF=D&bZ*u+NN&!rcx|w3`SPZ3BA7xV7H=v_Nl7QWFy^B98=9Bd z*p=8y(gGR05T;T>Z4e{ZPzZcY8pC!F>aNbE=pP&q>7lmVLeku(F$|^r^7)hXg1^)J z-?EL->hx-_;ZkGGYACKFOX-#lQT>+^rCy%IrveYNlw@++wrveB57e~&{an)sfVV)1 z3v4^FJ)3dQw%309?%j^~bwIg9IZGr;?0n!GtXvbiWw6ck`#-og?3)&V*bDx~(HhPE zdcZa(3Ky7DdS54k`RSEq9mP(-*8)mBJ8rMWUZH>> zbkyzM=98}HkY~j24G7SCW;)^157E*-&f$v7&Gl2pR96{TFH`p5N-R zl=Fne;sX1L#N^n;NYF21#VdxEZKjGtqj!;vEIF8v8Cj2{#kz>GOEe0glw#7YZq$#^ zO3Jh!05Tbr*iKcpO|2iLoXDV)lYfA6{}q^J)oCw|TQw1ofn7)^!b&?qE4#1Mj#zJE zxYQ8Fl4pwq$^lqe>#_`tqgnSmxM|6)y*Kd~XQ z<@P1oGdQOxh&NLnWKeZTyeCyw<3qLoQr>GF8rDSv8{$SJY3v0NVZhIQ&jjx|zjgZd z%k)!zYh-jt4N(!$cFR5%-yW!!q5jl%jqm>T$4U(TL%7b*{7qK^lm?Ho=oA0+(nJ2| z+Ubiau7s*whSF>V88!Jcb4usW$Z$_FSgp*x=E~n0_I}uj%0B-?TR!DTsR;=IJgCRO z)B}u+?BUh7-v7C6cIw!hAg+Wi5=TMv$5qzXJdJbtAQQBHK5}F>QZ-CcAJQTkDta@Q z_qAzub_{Dpvy-bXtFbmv>_LD3*!%a#tet9(t7=a>HU}m$86{E2aswA# z(%nV*`8^rcKPB{&E*rFoD|qYdU$^ScSY(nL9??WWux)+k<>XhKoq0nG;Aa#uVZ#zs zgo#|7|ER54$DpZ(QkrtTH*VTL(e8JBEiJ8Xtph-h96u~-Io4tK?&{_GK^OWlPzx|E z5PVU5HE3H&`#eEQ(_!#{0SD0^>s+@oH*b&S=e4UK=PcVKlCf!cn!!Y(b1^N$$5MY| zJ}U1~5#Zqo^f=}ygm@GqkmpTEqrCq&iM8D`3xeBXS!(5$ty}j^WDA~{d1L+P7cU-% z2TM|3&7m@^`1hX&`gkfY7_V5qylC7tuwl`=64()#j*99DZor~(HVlyaK}J2w*zmM` z6iPLR0^`Qc=UI0l_h@sbWHuTVcNv}#?CR->Vl$L2ZNR0mIY@*e0AAs%KTdsp3GU{X zix-b^wYBQhSQiaMHbTE3kV;gkm#v@D)@30G(Ba;vA;x2_ z=T?BEY_v+TCGcvUxnzu%z3~Vtf>FO+sJGfpdxUG`?xzL8&Mu?JjXMdeEJG3`bq}pN zNrGF5*YhZtw5U?;OaOyCBvWoxcET3QHwnTh=sfFvPAKYI#g)F1+E*`T7OAvGewSBY zjfm45=~VtziavhvaH_(u)#9ATYZqosGm(isY#4o~q^@@zzyDinUVJxdSk^SR8R_#m z*ck)WlIaJh=06O$vihSIX#QhTw_o=qbHsQ5{x_`Kz>3^yN-ZFGiR%*oB`m1!7O=Sg z7S)RZ!lE^O);8wlpv_uJHYFHc4Fx(|^&fuN!ai%w=r}>NNxKmOZd8M?f^!|fS?!@- z2*LUZc#yWTn@9FEltAZ$){L7ve^j-Ge4;T^)&ACa zKW8SZ;xX$WJpRIV+e-V}2^T^y2|-MPS?5ljG%HP(VBydKBakjX>-n0O!K}m6&(HUY z8TSN%9fg6bU zgj6_ZGc$R#l~tGOENK`18pE1HiBeL3{!%Tog~NX_g1(VNoA^kX56$EfL1Y8~Is zGG^?aKG7|AMB3>vsn2q4jM%-I;*dYn;NPHuERhL$yjIjaBx~V?z zFRYq!pe(p3hI||-z$rL1yiu?shiT2MV63RoP6d;xs-U z7+4c)esz<<&3!sywf)NgN&FHVNpVwRzfPzb26IQt-+cDnACn3^50dm^M z+NAtwfPq=Xf$I}7?>rJ;yqLLC1PZ;5wsohkN?P2X zd~UK0za)+!&j@XkQ8d9Whe=GNusAn;LaHa7r@zb04CSG& zszQ{<(3;lpRe>#+tym#;H;Lt-9gj$dBz0g(wb%C6U;w6$;;74n6k~l&xgsu+Y z;QumEPE(RBVQofysxSzXhaW97Sy6n6lkPR_Fvrp33hgwEWgwQ)NrC(q?HKd2oTTu+ zty{O=!-h86K3*tKTHV+!OB)YQ-rcLHXN-~6Wh=;IB^b6nB<9rgGrcn1d1}(ibAPPN zd(lRbkEHomUOoJz#`75Cdv=HaBBJVZZ7GKGcqxPply|O1Qs&c9$h=D#@pY^)NG&vD zH)Q?2j@rRqOANJ6xYta$S64x4r#O0c_xKj8bOSov8MUg>6O)K!J5*Y;7cKH4%5uP< zLA5sCbH7oeAkWHjo%&9^G(LCdXnryOH>UOw^MB?WTN}A`taC@De;(b~vg#5ykA`l> z&VJT+nuL|#eehsnx1Go!i0~?It6JIeCH=*a`A!!O<>OC(Oobxx!lEQ%cZNeRcBlm9 zI0>ieM{hK=|KexI_%mnD9+HMb2nntTZ2y4>qE9`Wc6{;gWbHB}d@m`>%%40=Sc54N z;z?1ZrF5@Spi(4;5RsgSBygPQ6_$xEnJ!W&8GggDM2k;^$w`njO@=TDz(zrY_us9^ z9W$*wBCo!VHsceTa=A?ZI0QM_Eh^EV73JWV$LJ$>F%mM1Jtzt~gc3XNt@bW3s2kX& z^w>QtQfOqdSTIE(JnD*ILk$2?GR22CBUl^Bx5iteAl`p$d+n()ZbO7i4+BME8gm&$Ul)_~rm*uE#+$VEN(-=ego zUH6e$?a=OB;V?3v%uDMvk30Rh#6%x@9{m8xZc?DheyN7-9(Ml_y|E+h~+viaG6gT@~0)oxS4%Oy-6Ei@J zyL*&G`Tg^4FM<cQFfFs;0eOPnM2DU|G(JGolQJQK`FN;&-f#C^tifo#^;M|4m*gBREfMjyATcG{j*stp0Do4pK1pHdiP`Gup3yPA#V-x2 zd?M`9lQ}oCiV=9ngl>3|{(hYvs1Wg%&cC_9mnEjH;`N3xRBQ>hrEXw{zjjwHo4cFZFp9@ zTZ6HFDYi>P?`e-&)v4J`r*nOoC^R$oaZ1{bU(6$t#zmaSIukK#Mq2k5%hu26c50&A zh?9%QMkFoU)cw@4i4)A4h4%hw$FV*6mCB_94jg>@vqn?T`=wRuGv_Yf|NgV_{dJ~o zAHB)VZK!yrx;e_s%yViJ1^%Ab`xg%Kwxj5Q%a;L^AiJPUv6)R2@O<&SH$He{j&sh@ zj_unIT{%|>EX0HYZha}a@@b?(oi9--c`yf+9N(vxmjq;m4+6_GB@YMwiv1ghR4_@W zGa@-PB0pIbx6q+LQsS&q+_aMxdlSWnR;yN5=Ym3PK~M`uT0=ps5%n!*2mx*Zrk*4v z)gC&%B0nO=80YLJ(qS5l1yM$@b*th9#u8q^L7Evkxtm{BVn`Egk7w32@=!Ca*Ma{N zwd?R$=zv>L7+Z<8$$d6}Y%*Gur#CkA=hwX^s||pJ2*v{~G##EbnH;aF=^!k~sbU2r zNBwwB(JlAgO1%JA)OOXXRTsm<&6q)Oxfd!nFnO6oB8U8qUQcFuz2**=Np^Xw`3-d} zbym(JYJr<(uuIu^=XI1;6u^k(WE8S=GnK`CF`u#^PsK6FavPqtW z*Xyu#@9OKQZR;rD-xN1bv|H`2{1i}Qa8%@ToVzG)F*_BELON4R0Z5~ux_`z*92CfT zdpCwy9F=m<)E$ItFC*ZUjD zSxytdRG>SSqItj02$%H-PK1Wmm)}o9@cCsWW>5uT%QGz$Lun`rVjJptT6{vw*inqD zAVi9xGC}z5Z(PyIk-K;9%qUb3v<~%3P*Q-MI(IH^R6Ez8HRWFe?j%(3)svtxXw!Je zW|@}NZ@*rL+XD<1-+Wr|>j!BL;Q@(lys0@HL!!UGzgp9#fvYuc?OHV?4)M%}CG+QR z+q$)mVz_hV|X;87XClrMr5pqn@k@5H? zM`jQ38EPkT%rJMA8))$>Sm!ddn#npEf(N5CryL%xy|+E%B(6Oe<{6gulGi}SgBhkI zkh9>)-$pUTs;*@{N97=2O3oNy3t6S86C^v9DS0I-fS~{kTne zYqM9_)f`SF7OJ1xV)sD^aTQ-ag&uXl&k3ISP7~Cv|=jL zmh$rTOm3a#(2A4#M#6_z1dqW*OIO9QEJ=qoYt_11Y;$t#CN>J#=q=4(fe)X&cyWGm z+pUo*J7r-D7Yr1VjloE}W0T8KmWeMKv*7n_2iL;_alJg}%=$R1Z?9$PGrdnRl*g!R zSWd}N#|dX5G~<;u*pjlwkLVEo-(?7x7R#Ukvpl`M=0dJehR@6`KY3E8 zwahtWW*Jf|u?TM{T`WN|EX2T1@SCLDG(n$epfj>j=e_2=so=!Kh+eJi*LKRAnG>c> z+X?Z>2D=HnqX;(+9X`3j)7f1=$XZw8LxuVpG-%p|A||B^6^6qAQn=-f@9Tek9rZHY zwxfEJ6dw`eUIfAs!5JN1&HSuUr@{PCA4ys?ZP4^+mzG$LJ-xgpj(JQvhIME&PPzq) z7mu7a?eNUpYjoKSo;-OX>EftQFLJaV(dZZ@q^uk<)S}w}K;J+~)m@{Kz>HY(v*mqE z?_$}-yn!cA)vD95!Y5L zL2?5de2QyfcZ~6f`NhR?0H?Tg&!qnTSsY>_Sq6BMQ33k2A%D8(l;YV%c>qPXA{|B* zLCEb_Ax55^Pr3n2&mw$0GO{1xXuGEM^0Nq|J;HBSlg4H(M^_kRyZ7L-v{7R#a>c#Q z7>YM<)O?~*AJ8ZuPHIACchvoL8efmkY@^Xoa65PJ?!T(_b9>Z*SC(beDZO2{a1rcu zy+)006Gl5PXZ+Vy{q}07A6~r8mUL_b1A|$}p0ALiLovA(d+;Hn9`>DxXk=PoJ>oo0 z7dLJE?Tj9hamat{6;c(z*(NX}4e4hOFk)VKzg4!k&B@e}>~9z!6sNx>z5aT&{k9^P z9}}Y@FT^RQw|Rf!FZ!&i_wDM-f7)2{+e%e4^>{p|=C@PCw5V{RrH+6c5IyS87Grkh z>kLtQk~l)gXW4F_9(sEHR}Fan&e2&#{!zyP1D+<$J~$l_AoIB!qwyeh=%7*S!X@^1 z>=K4}EQ-<%_Ex&hNVg5KB=v)H(E{E9bPgO$LNXWz8LU(IJL&M|2!Es5mR}ONB zFwu`DE7ZJG(-+QQC>;*mpI~FNnRBpc{G3><&{x_m#2HoZ`|snCCma3!IAh;@ z)23@PTPA>)t^-mdlIYocPImH{T5w8oKnt9=Y+2aUDDE;caDD}fctz37UCfkl&wK3@ zIk~xv9jOPUgn)tj*=O2i_`5&dYegK155;VVEm-ux^2F-bpO9|XRrT)OJiz`~bh!LO z)II(Fd>QlTToaxr$Foe<6^SnYId$etM#`Cq-FnQ3YR09%ZS&?@?1R($W5&Cf7I#0X zY#f*BX8M_{;8#REFtYTmgc^>pvGI9g-*;X`?s6Id$svRtld-&9`&U=R_p*cNCs}J3 zd`7KTc_uCNW_nNwFg^6%#|sm?J;4IYIehuP{z{!B)#vzan2lKoe~t~x+3rtP5gAzzpqh!UBT!JpqslUYJjM&XLQt6@FKF|so%^+G z?wa?K3RbPDYN)Q&u7L`+J23?Z0-G5cI zPYUWMUwB&H-}iip#=V&6=(hBrGJzOOn1F(dEuxzlT!OWcnB!!XA2G zkgKJ+-czDc5)iuD0>0uW3UP%}`H7(H*)m7)!GpI6aF*Mcq>LW(TC-m>ZWp=FAXS3O zO2R@m>eq?a#E0?7c?CpK?|7<--8j+ec{MnxvQ_9}uzjQf1{j z)N6rbif8{_GZz^{bV{WzF`={#y)qZpi!K=5D7xVCwTf-Wk9w3vT~cnT=e^rBmVrh_%LAAmY74bc`kANmfWcG|;o-f5t;vXN*}ZGmS1B)VY&bBw zQ_~-9Y&v!`K3@KVjS7QMBLzCJnt7|E@*@HcRt`!>GvCT+<$?omUO6nN$o&QFT0f#| zo}}kai{|_b*^Qmh;m6Q-^7l{U^AKF;RItH75d>Xd7|IzOeCoBT(0t%42 z9?9BGu_B7&)3cXPrQVh? z-OJEQkgwMO7ukUIMW<3edMtMO?SM=rd2j<`W!JRy^ktB5PxPa%U0ad&@xzD5j~}=1 zrUHOE&YI`)$=Lt7tn64=*o29oJ z!1$^i=iG;9JDD7>6eG|-3-*zD_Z46vbwVdL2%WeCXmxnr)41@MUT71k1d^`4(99f! zL6^p~5eFu@8v)aLom4&WH*CnCkXhn%6@W`8mc84lHlw(Eq-sqI*TU3f69PYKC=jzb zm)aaP}2uw=>m7p)FIE;zo>rRkW|vV}Ry zC$C3;jA;{fM(I6lfs4kk|9GZTw0!E*8^_5^VCrY5MU`bU&JI#`{Zsr7B%~7gdn_`t zIcxhV1IR}jx%8iuabHzH4?H#FP2#eu!VE{x-|f2U1Y3s|M6ZysS?|Z~{kWEFcF!GF zr`+O5ds-P0FkboJje%kLo;?^OkE7M2;6z$?;zd})h?ovu6N~{Ag`ua-ooY4eU9QRh zZWJYRT^Nm0aYMd0RMyuS7-F9KU2t7;1IJ z)#>E@y~tP-TUt^Q&)E{LTz1Clvy+q4cpIA&tBz3R!<=&7&wX#F*) z*FAUaNH{oHJ@AZMT#>te#%7=uMRB_}_MStOQHSwBK!Cn_{rdQt=c|S10+O2l_P4g` zM#d|>tu|gMNi9* z>(d8$;EBo_+~}@5bi443a>GfF^Z#-8=HXPX|NrP}qCu&s3`uH7qp3k5ijqo^6tT5J z5s})FX|a^Kh_*}>vPB7Dmmy?{qEbmA8ACEs-I4r?(LW(lRJEf= zBaX@$@!f-;#~LX+I8BUUS-l$rKP1yp|eDfE~^s9 z@z>%k5)!54{jQ)B0iY)}Bu6f#eScjH7UQq zE{B=8K8G2vz~GP;qUZC08ycjr@#GP_2mxWDFd|D>WEvY*l(+I|JumKoXQdd_M zN8$wDoGk*Z`K}oOvF& z>-UI=H)ymlgpZ7hCek=$yKPIU*%WwvY|G6>3q4^8x%xJHWZ4FIw!xb{Dt@?JZOwPhc#M%-pN~y(`pQbXxXF zO^pcBZG2u#cBt)@c&dd^dVt1cvJz2hww?L@Pm85HWw|H1Lif2cnFE&VfBaQ3{iN)L z@4er2F~`>67w+iJ;I$34X{0|uKz+&(qOhqxJKi;o zW$gu0H&|^20Uv?FA?iVlQFB^cd^`zSCEiUaHN;S3ld;zN@)$@%)*sMp%&l~K_ybo} z*9@3x{Rab&(Fkmicvan8POma&3OE+1&u7e9xYiZbA>?8d3Yr?A97LoGBO?A}^Y`K@{kQ6I z769>925d?hpH>G22pWy2SyX_u2gLax?exUfO<)*we-u~PD1j{*!!ZXVg*zHXOc0nv zAw!-BqXM8@(hsS8nC^mr;gq7YKNhTiqvSc*ubW+3NKP`=l|9(0)cxW_RD{NVDy6Tp zMp!DlTg88xBJe8+DyGKsxD;qHU$AZf7>AlAd{VUI$!|57~>u%y?yTtU88IiIN~J zWaDPo;-iS80!Ri*j!~M(oeN{t&ry_L^Q5j$6sbiQAy_BtR!N~A#WK;)qkt&^!+F}_ z2gIFcIMWy-r3wUq#8W8$K$p*$N&dX_ynW(G6Gsq;5F)<=_QZe~+le294R(V#FK8ks z#kX@zP1|^H9rJCi4z>=L#Pga=K)M8s7X|Fu&>8lYAPXF(DCYtL*%icD4lVB6>S~@( zC1dmdi9rK^%R@HU0V*}A|2Ka)T+W4f+hogk&Z=)~y95CR*!Kp&FodClyXBsp$d8Mq zWl+CBgHF<|N$WGp9%K~aB?rSg2VMY383os=LjFyW9x>tDs;G!r@uZMATLfVJZBySb z^Jm};z}S#K&IzMb!_D0wCr9WBi2I)5WFcZPoZeq+9}7d^lHoLXzlGNH8v($GmWCjT zz$BY`pizXW{!@)IhLyq~Nd!tdI@bz5&my}Rd8dt6rc$qyh~IW9hkPFJWerL6vG{@& zb3A*xyIplw?*vYS5~`%U-1M3a3?*d18Y)B7mUGbMtKucxZmza-XFX8SU*!R6kVq(? zP`;4MHn{OB<*rG}o(!EG!RnE=L|jq`#=sY1xF5|#u58Sl!F(tWSb%4NAp-IN zRjnG*uArnigF%Eu-YF}qZN>N-{u{i#Nn&D0epnKU_+HKSAr4J5SjD+_XL=gBJAx9d`Xx-8;yxn5RDOK0nv>`AwtiG(y zB|l@tF;#TU^51@Y7@K&EC+TaQR48MvOe^nL5~rc|l*z|CyxE*)dbfquIv0A^5}M{s z*LG3P%IcNN1{WPv^2*)jyJ}^w`xnz}$M%KU;N>Vg%9N=i#%*4TEVkXK{Y_faf`eY$ zzulYlNqafA&6{2?mMVDkBrk1IWsud$`wYK!PS+d@4pm|8Vz#wH`5fs5w=5Qhowi)- z>C!uUwYK&d1!d90r%$(xRi3h_s;+#%4YBsUb*HRPe{o5!M9S@#Mz`2^+KCP=)pfN? zC|a9f%z9bBV8s_e$`OV4LVUo zp}6VZcMpqMA-O_g3gwzz*UT?cJ1j?N*^q`mN?iBmb?BrM2l*C{jvgE9@>yuoQLL*3 zr=PUEvZx#w_n4iRkDWH-PFEh=ZDkNO@*yZkrp@d7V({MVxKF&oaq?JgvU{_QT&Tg3 zDHOe|nF}AUW#(x}jGoS%c~Wa>j{*9b4xvtJhAV54(G(gh%q$%R0!OZf z$vZDx$T-*Y3*}R7>wC4j$RwH+ZDdvFO?i)5;cqrQavn;q+@#WsZNov@oR*IB+9lw+rjX3P>c&?8dUmy@;Qw!C`1LVA5jZuMg6kV{!I(KXMnCV#D3 zlzzUebI+E|AJ)shl@jN|wd4u+CJRYZB8s zn=uf)aPt^@Q2e9l{27jG_%njS*IAr1?EgV;_U+IVSO3NJ=|#>CuJSi;YiqBRm!IdC zvd)j&*|E~sde`eE^?ivoHIpd3r>iNH?10}dYQ@Zt+Oy)Tn(wDHVQIVjmHpiqTw~Gt zP)Fxh>np)NKi|^p{%_gQJ-zep=`8y^Vq3vtMfXV2>&rimcG@L7x31W*kpC95LPpQY zkt{clS$B4^Ydqp~Rz}M4y0J&irXD(Md{T|;ri6y{2s`toS$)g&a|#NIqjs#eJY5d0 z^#|j%rQlJ`qtA7m(6A@3su#cB*pYE?a`L9Jz_y{t7U3Im(e=%dGO@Qlm1q<&wz|B| zPUAeviMTpxaImhh=8t85wkj;DJKBRU9L8*`w>Zqd#~PeO zvD<&swdnJUjNk~#Oqy{+m!w>}m8#)s$zR3yyPrF`QmwPf>lDBv;}`RzJFHI*54F_^ z{J1=sn=LPvM#T?mNH08LvfP5fnv+`R?3HP4D!CQ?b1i8?r^1$aFm6`#WL4P z*{!`~r*%G?jRzrN0CC7MFy7<`qYQ|kme(MU~;#Qt1pOMYZtYsxQU5C&9RzZ2i2$F=D5s%`aU-1 z-pu*u&V_!gJkMeK)}MFD8QOI~=`g3@c+(fIHF9$L9N+uj4{jS}X2_O@Q_@{d@2F#b zNlxyFIp4aSJ4oB3GV;dRW?%M&WMHAgeFw9zMO+#1;w`c7TevHvB9xO-e*Ma(Xl0#~I6Bg^w-1k1eeK#@eS-so`Mk$gFQD8Sez)ST zzW!SU#l;Mt?6bG;k5+M<)7+wNpFc0us(kmpS`~vmtU!DJ)Vzqjk<35FjdWRq^$xT@9jLfPTgKr|5($4z%y0@Fi=_<-t3h0Zn zwpY`b%Y*ndD!5vlJ%1NoDcM<`j!LsfSW8`1{GG`xc(M*9ozlo z{RXetw5BZGUwz*j8QGuhx5yCLGP&uo)7`psO*5D9c7};;R95>q7>7^mE&ZLEF09{g zCgi?4XeH0|w~kX(Q^VI$&c*NQsyeidHRVi*BL3y73ctlOI1R?14fI@B1yLv-!K|e3 zr$Xlb_N{!O+CDOj(TfL$hqDzPY})p*ucfSZ@^$s4?xCf3-MH(W8lFb4_dYrKd#!Og zx?k3du4gc4vl#Vl^82L6rfqY-RVWUU-Y(atOw?K`$o+pcH8VDJqMQ;lej7Gp;vJb@ zQgm=*m^3^eFyE`4$@+$62yFdMIcF5HF8eXEfT4mL#1|A&H`dPSIo{UFPBF^onmTpr znaC{Gko`LrBdYOo94I@bj{e?kr+5C!f84)QXofmQQ=;|-Va?If4Ct&?E23lQ&vag1 zxg+dY_-?1}zEem$UtU`3jAtogG;?UU(wMX`k zELhEX|K=^T@kp1bN`G&{ATNH&4yt}6au`+z%_R1G^lF*~Cf7zK_FZ=eHWId${;d+5 zF829IZ0{>~SO`cT7pW9(5DVfKAyS>aP^N?bepCz|AXLFl5&~MWz!zkQU_*m(stvR! z|B%nk4N|6nN?VTEtH3Wn-Df!Til?6jVNq>G^!Id%Ulo|uw1M!n0K%DrrbKVw1* zz}-mND4;r^oq(rK2V&xLNmFA?aAXi=wqoL6N#8eti#C9i|G#H_-EK&6NT3uXmN4fL zdm;%jIc*4int+L3szuNn0ese<0F8EY7`I9j!DTU)?QF|iG3dnFUv-_OG4I62l3W1f z;UgkV3X%_`=(Rb z;jI>UUS(J_0Q>+dsR0az;wS*4PBLjKt+VE}F+3moNA-hjgs;PUYmV)1sGW&z5P%(Y zpd{l#sH$5;losLMBN&**^hly-LtjpwC4i47T^pM+mos=7wVjDCGfYY; zr^}9lWVix|pU9vJgkO<Hsjr-`qTDTL$NHt1?L5T)aRBbalDIG(4d?Dq#w?g9Q#S z+YfTmsTw>=sD`)+x;Kj=AVnYV9g5E=1_x{c{dv8T#EKOup!odyvuSrHfqzESEi z)XdqfCzp%U4LjDv()_>kz;`QU4V&8cjQBUUGYTw3Roi#nb@mW=r~Gtr9j#SgPcIn{ z`dkqZqfi)i2akMZzAn^hTF;fk+^NAC!Z?o#SfDfdAyqa4V7&=?aA*w22m{ln6k|9xBF%f_Yn8eqd~+s?R0Wd0DvqmuLf43 z^Kp2%2z=^&FFAh)y-J49zgH>%jt2w^?fHtwZ^r>)rIEFk}<9mQn zb4ay9n*+mN?Q?6)2epP-_LmV6E0_D-}Da4wb!fdqLa@yKjm-Y4E8DGeQ z8mvf=3$%mVNL8TUs{&e6u-W=4(7;H0JTzJ@1!M>_70$ShQ?qI!oi zFIdw`nvS%J8r#IUI9}4?%pGAp)AGDhOk>*ivmT!lqK#2CI0se-33VyFlk2Y&1}_ev!FUHibx3I-Q|{ zgOR4bwRM%evubZhQIo({#|^-Kx)G=P$xp5}7aH(M-tnS_%&N*t74g8RTek@4`tCY- zP`N)JJa{7Te#-@8bMqOWN*2YWv9?0;gHQLzUjLT)fiOu%L}WL`Cngfq6^0s;1G(+- zefdJ(lfjxRol|W3sWPp0W@9#UEvRCi1OE4y)B#k6I8Q!tH z*W=@-dGhgnwYr}krxZXVueIKF*QdBY zmEDcfAAyq=s5VSRYPcFZiO&GDO(75ag`oMk36J*&^5Y?>G@V2E6gYE|)kHx`L|pkD zTFdIl&=J184|KmWCBsCVVV6&+bEH2HVUlA3wjuDYTrt`{kg_42WUq;Ws=xKYpyG-p zqSU7Y{y$Mw3wmj0M#izv&#&rJok#lSiRfyUc7cx}vi|;)V~R6|3V2O?tzA0@oEDCF zi8Urw3`LsK`rp?yKB(xjGlb3xFF$1Sgk*&Pn}kgns3-lO0VZ= zvGIbMAdBb~vMHc0z=R6v!v+RLiK2kpr`%%RyMN!*cShaP11mXKgk5dPkeBv4hO$8p z?a|$ld6#hP{i%#FMN7nxy?Xib60xFLLOp9C1MSD}k0AwrRato*8;U=sxt)6Y2q_Td zU@f?|wa@l}gXtOI>p=Bi^sRMX>{?0B8CbBojy=Zu7$Bxvk$YjvM`#0&z=9zxea!Vo zwJFeHqCPe~h?;PJV7%Of|0D&qTsc%4Uz(c>Yv~+kKVceEL{!spm}kX?9PbWM5R;bJ zM{5?E*?s6D2jq@yW8SOiVJRhrYLr9gP|Cm7-NP`*G{kd%S}fctc4 z#%;xK8OrDj%vT^tzzp&{t<^yS2-HP_ydONkOCn3{K^42tH9Ic)D&C7qh_uhqvd zV#12j@MpfjX+7D^0Iobq9|4ald{WXU-!47q)%!GRWW1G=xgWtWl))dP;Q)oeSaZWY zXc(v%UwAoAbty`%-_J%~-ubNq6vJPa>=Xad@$HZ*?mwP~G8K8txz5NkR7C9md#hon z;ePaWb_PntmkO&Uu)ODi^>HxmAkt#TvU#ai;K&iOlC@=-Esgn2LL9TJvDJL!A@BIb z;TPU{e^4%LY4DzyJZyoG+n@X)$i~H7sywcO*eXn=nuUQ^XeWn;hohihHuy*Tvzp8o z4gQ3eRT{(ph({hCY#59V8;Jgsb{>8m;)0k_&`5T%+Y6f}V|UV@6&xNu8^U9df9Hyz z*(KGmo`Hc0G?UPQUn+2c=m}5K9kF)EDODS`DUnmMf_&Py^B(=vf$)*xVOYeUU|22; zE>Nb;H*|^Wd-mgF&V7IIh6Y5CB-m=s;?1tG1|i>PdyMwkc3AVSVsaHSXrhyo=i)f? zzH;SD`G>MH%;M%D3T))1fa$?vpW>(S;Lksa`3^f3=ICsb44D<%g<)U?0^{Z@Um59w zWLQ@cr|kv7^6RgQZ@=R>jhs+XnGAD^2i&k^L=<-KcNNz7xunU2YLtu)B~x8=cD4ye zqI#FQt;0kk;77ZVZDt7NQx#L>kW> z%eI4vwhMEci)E5F@ivSv6SydLq9OnTrv+seyrjr_+GpiO=+!&PcG-Q-w8+PgAHT8n zjHUOt9U8?XFuja3QRV&NQcfr)-s;s2@1E)_<97lsepLu|kgu8osZYsSAF)`rUsQa* zGR7Og3>pPvtgp;hO#cO)@7T1iG)JXle1E|Me%iMo;tQFQh=5~0(xs8;4x}zoT*sY0 zf@q0s#J|8Pg3!LR%6a`_;?5L^L8elm97t?O^4DcDcMVI{qNt{(1`_29(3G7HOA{Uq z$LJ0xOnaO|BrJ}OzZLxu&s~8Fv`r)u(l*A?~_fv5Xe)@b|9Sv>Y9wM*S z-|rc5H~w9^_GP~UK#*O~hCug31P2g~FPD=k&(!K1Ot$?t|8pUozs@# zb3noLH)6LSG3#9O98zQ;iP5hQ^}NivFB-5(YzNE%Ne7tocNLOo@^R9V>=Yz&L52i# znZ!LF73L&Y(5+;~6@+)UV9r4=TP>e6vM5@i9pdBvu=Xr{dX0)g@h^{|RGcJ|5uY8_ z8+hQ!sQbYFekrG8h=6@_xEr_ayx&paDsfrIWs*x?8o2cK2|&wI0GuCLg*W&rW?TC zfla-nmx@2&bk53VR19TZhKqH_vI}@qG1&A)!%}xmI}wI|RkG0(s5UZ+W0jPg_WY-< zX0{5mwM}$G&eb;&9~y7qAZ^{99Y{K6ywn|WdcMJ&WTSj2_fG#gx-SkBiRb(5XwQTd z4sOG~Gdl{N>E<*)>D6*E5n=4ef>sAcQ-~HnA?rJK&Mr4CTIR?uO4aN zibkG(9j`FmPgK)sm>lA;IVioxxk9Yo$o1@aVek7p2W5>kRzhHt=JZy1lf{f3>Co&f zm6sPn6)R)AQh&y?U1M(K)c^eA*8dTl^M2)iw?qG*!8v_Pa{l3+jt}QrtD_XvxJzTK zJBZpcwBR6v_V=nlpY!2H*ATxOpe}X|A8cMICjg>&5??tUYoqMsoB?w<1ffL3%Vz}n z8j>bSrV+&zx-%o#7|48eI5Oc@?-bVLhXe`7HAK z^-GYS5g`sne?szyv27dDbNkC-XGV&5`@-Byo7M;aAeo|ZH2_skMu7`N+&U>`>d zH4AXL3an&m!;{#J2fejuw6QD+B8c^sLH1`OK!iyqJSh5vs#+}~A$lR>IfpQ3?g}6+ z5_me)_g?eS7?fZs$XJIO^B6?w#2!c@^5EhYSfM1-s34ldegwj!lY5{s>CK0aS&tq? zVf!V5D1<4LzU%57kULJBr{K~ZR4nk+g=4W?$HfG&Aae$U`9z-lY;^1)CLvfNrx^Bi z60xA6!7l=%gW$;6ZoIEsf->9o4rJgMmH|?UZm~2NHfWMgdNvYcX$qi4&U4O9EN{1v7P3OIi6^!9UXpM zmx8DMQkgBnk!g52@f{yg$wb?VP+1un8OaQIeyW?H1d@UFA4n(+Sg)$85YAnJc{LSK z(h8dRBB>?~cRu=AVQ44&NoXf?-(7lnSy>7okHjQ2s6Lmnva0F=m@%rlg}f$nS!40D zsKb6|Y3;()<^jGoj&Z@RC`Kf1*MGW+rWX_n(8mNDS%DBV_wN2b2&jhsZvm=(=DEi2 z77@(kBf(5m5z-b7Occ8H>ObE&AFyO+UJ-R}=xRfi-^v00cJkDzgKHRR!mnodNr7;1 z(^>7Iv-+8^b#|04&9l#1X87_EDbKWlM|NFH$VLin^Jy|hi9|>!k<1DT)P&kjb;GgV z1>Q+(+(*o?lIj~9qx)A3_$;caVSD;YZQXg-`5BD^%>z!z@wY^j%2~8t>C!OE{2!pI zD0R$sa1fPsfLTbv@jNh;Wa99#`aBOIVP%&^EdvARQ;Q)tMjprxv`B!r5QTCHi^BXi zG60j-FU=Vh^}iq!yHQdJs>z+5>Z+D(Y?7NXsQ}l%aew-Pd!jN-)Jx$C(^4DDhBF85 zOro-Sf~;R&TZM#L6_f@@qj+@gPPzPNz#BfTjDnMq_sjiLw$=_l`z&QM>g^MmG+L9` zdtxtOcA<8UE5(0z-5hlgj;7CQvR0>w^jV2UG1Je9%Bayi-V0^x{0|06?~&t6wlV2;f8h#mhFBwYZ`y=ShUXSjkR{St=qbQF^d&T_`Prk8Dosu zwifF@iGY$SHO)lQ%f_^`=g${l*Yklz@e}mAJBc%65);892(E0dMe-{d2@j0ZvB^lC zjns5#7!hH`MKYH`BmCC>wABCp5T^8k0M*jcy0v~2>e9su=Y0oxrHPcYjc%iuQ#+29 zYVnN&b^nA)zu|PMx}1%NSuM*pxkL?=J%c|394{H{l1>N~$L6HVDQ%JdV*Tx%opW#( zkWKSj(ZiZs`%cmXAv*u2pM>RNrMyzgm;vZBV`LG4bZ{K(Of09P8>%M;H9^ao1ypXs z#v59kn;TCpL{evJ4(2MZTWxKUO2?807u>f8fYcRHV_P#k)jGYxbV<)84C6xbTFZ;l zaZ|joz5!|Nd{(GS-zwt*5st&AxP9V?yx;~yv|UBzV!7WgKOqk^PDGyyuNL-=Lk0{; zw@C^zRJvr6QL!>7;e2J(+^Eq>OaHBlkmptH?V3Ap-i5n&SCXvrYEe@avgiJnfP&8h z#OV*Yurw%3YZ^p8hsN78p2(SBKezv-2q zytk*h`14;a^95oA7i1Ve+$O(o$J3?r^h@U+xNBUe{O)E{!o73n4@{dj#7cPf=bMks zU&pHaAI!SaEOTlh0{+3@FMJ@mzG+n*f82@u41 z@GM_y9n1lQWvc2Vp~aiqp(eBzVceUg6@=^~flQR?bd(`|vPF3O)tA zUwBVn5IXX6=(|vn#0~rEbVwZ?sk+`tXoe*&>@@FlQU!y){ z!YA>hs>-0REJU+m6%`|M@J@ju1{*tPXlUqIO{T4y^~QPekvgp1SA`hmEs+<}cs0jj z#zowK!y(bXh_IXG0Gl8tHVv{iA2r6fp1oDNiHD;fJm5odv;*p8Wb>Rz_N!p7+PPM# zmbvvNcHY^jFo{DQ$Ovea?s_lLIY)-Gd$Zz(U%z9nI+4t(d&lI}+eELpt~18nJCvrz z*}Xz_^uvS}z)v}%E|SnxL{%&%CPpUz0_L@&X?Ms~6haOVP_jGi+=s%ic3rWm@nyuW zMA!rsmAnA%o`)ft&$XOj8)PECO4#{6JmUfNq~iv-i$*Hdb|sPRY_r!d!Dl1M#}C+d zDz8Tj^vn>nTqODHwq3ipfgBT*rV%8PTHUk#-!WBf_rMvT4~Ze(YT@(uN0r3fI_^Ze zr~m{-aD9J`&LuI0NGIeC$IWO{n1^f+_Tzx79Gh@`s*fdA_$on3kd8 zY!r>DiTa6Z>gvyz?&`CVxidBqI^ny2dFS!TQsMi(NRrCdtu^I8gw{cN=^Qv9%DeBkg*ns5@`?(+^1T5i4D93Mcv^?RmZvKW>-nwRJWC?DPT-^m9L=3xuGipa5nIHCbM zi_vjzW3Up>gC7ng4zh>bMt0AOr$cXn^u#FbxKEv(#B*qK{=EJ=`ms+PYZi!Vvi3F= zzh_U}Gf(BmYxppofl$AINu#h+!~XHM)kb9mJPpz7W6j=z5F89JDZt->9ZW_T39#WQ zWFi_gA~+0rX0BVg(iKydP=*s-K2%QXU%Hx`xxc$N2t2E$uW#|t81*KiVlMp3R<>qn zz%>ovgs^F_-=HzeJ;osYAt)Rop8!Td6jPlA*Y455qa$wPECQLphm(s0;&ix(QM|2E zcePT)K*-C;zdSw)eo2BR&gaX1ekTe}rs~IP$LRscNbZEzL_<6<1sg9)U!ta1Z}w(G zCOs0n>Oy}04Yx)GO%m?iyEJL;Is`3TjMG_NUQ!~K(B9T&41YP0(3$OkCNbSEG*a*upZtR68v1!GtynkV8H2(?E}kKaDylZE600TW>9lPNGhEkjme&s5J8uVp=-i}xlTHo;bY#@2R0 z`9>VAckXawCPrQ+ob_(b$i;G~@5N46QdM;s7}v~pl28mN1z&3o(lo58aU;}6zfA*B z4!VBI#pDbK6d%WhZk;OfhgT z=#3_!u_e?M8A*G8n?1b7nBo9xgRmiFx`U>se*mPi&cwrpql#;I&A*=U2;YU7$5kzQ zJ1!RPAQ{j=_vO9I>QFyI$+Q{fe#*A!lIhk*K5hK3AA-mR{gp(|LnGfQdK|ma{Kt--AGuaeE(9ZrE9K{0Hc!~CWQ84(P1yv@PT!Udf%E%8~)A65H$-vuSXL$K9Vx02WdwE z5He^@W`I=$ivu9HrRm1CYwqxe!ND$wMTzrqJ3RQh!xUmNfR+tUAE5xBa642g7g1?q zzp9yk-QmnIy7ppN6xl{JUQP8>f@#(BZsCaKF(f0l$Rfj)!yVsT8_1FfIk z5Rcv#Xb7@?Nu2=-f??j*m_;x;Lt1JhtW#13Dcz+*D$;iQyUZ zcCku7-^W?$gDNk$&p)Qa#GIpVLhD{!XN>bU1v7bYy+}H0#OG8?m$9XJ3(o0(kS5W* zD22%cnVt9M%^b9`vgcDI+Z-r}M&mUZ=|O6AKo(Ys$M$9@TU!gdjmY&5ym z+ok19*U!6INA>mb_Rh7YvkVYxCk%xHc^4717I3kd=aclS2rCAB1BRMt>kzEYg1=3Jl8I7GeMLm`7uQTR#v2^r_0z>cy_4bJM z24CL2U#PW1R$G)P>^iE(7MxNf{|G!O6w^punq^)d8u_e&kLAMcig+v17r~|d1mQGH zgz-cJ5JnA6S-O?&P~fjO7Sr&coC$a{dOyl{Co&%NV4O}ubhN6qGw|c8)KekJG=vj< z>FSbNYj*s2GKML*jgIWq&^Qhq=O0y}tCd%;zLcn!qN0aXelv)W06;rARso>BymQws z;m^W=L0x)p)J-pnbQxKN5@CtmFdnEexIi;#{$73f5K@!v@O~*hWdO<5@ny0>d!FHx z_2|P&lSav`udk0pSu`wum|gRJX)PO1Re%!MYG`PPC?LS1CzT4iuUX%tJG2en{$2^S zC$2xlxbT_tOA7N;B0J@w^X(DcODh`Pnuit^=|*WKLo{V%2pkHC24Ap)zqL%-y~vH; zucZ_l8v`|!jNsM17?1-7FAc2+>Ncnv6JlcvS}<+RF2m&E3Dz7_77w>^FL&4I(*7}v zdmOd=q2%N24wKeV`A~a0UFHOGtJY!`m|hOTY!GRN!@p>C?}JCoevXTjq@*d1JfsEH zVs`fclK$2g2we8@F@l`4NrZv`>sDVJS}>VDF+%7Z6eOfJkInj3JPvAo~RTk+u#_@%g=u;`Ci|?NUofr*q?=b zK1fyTfdipUfIr@?h-Y8@3+kncp;S{2&qHTRzrmqRv5y`J0+m81%wI0(U$A~av6IP z=!PPT-UoU#?Yb^B^AwViH25e!{_?5OUSH9J{mnkb`h^l``8IJ$@R6WgGTR#}0@;e8 z7k3sTQl@PkhBqP!X&x$|lU7zc@fsl2z%sxeN9zmMr!VQUTm_*e8BZ%dw(35T;;row z5fM4dY}olQJzcnZzgU@)BC{^jc5lYhKfCU5+QZZ)r?3~q*Ae$I;tJqKz&!>n6m7M# z;P-(>!EZgiehhYsOrJ@8l!sau;?rE(qC{QOG|=A*@DMLLAA1w#{1QC+nVAh@MJ$$cgEG#cSMq zc~$fYTDO2Yi)TR`Q>>qLnIDB~_apj};5Ws_%-Mac$Mhw}jge-fDCQtE18S~*qI;Vw z1T&;-b~HOfF#VSBIIGjAH&qni=t72V0kENZ?6WA7kr8rTVQEUwq+9pz?;rY=O67&E z0S0jYd*@ad*F?p&lf9E%@&sXPg70{R%($uEK>p^T-qi7Fq{Q;r*T2oqu` z4)dC_`mVN7R|Q}3zp$hZ$N;|mq@^XP2PfsWOv$-@#X634&FYYH?a=La`(8_BH!cUO zg%-LHlDn$`0ZT`JW8i|o_VKKYWqbbN3|%p0lm={oE8aQb+3)N{HDM1h|p_uFd)D5M;`%s;XL!6eMKHkUrWDZaq9S z7_Vosa)5TC`yc`|EnQtNjK8Z#5O7Hm9SA#8G>~BN6qn;6AvTzTNtoD=&fsdn1clm$ z{ep*~URM0~+F!uA$SZDy`7n}yd=C~p$x<(i3|Y~b|9hiBoQ`qR%OKR2Z!J7`x6NkbPoO}#_^(f3 zUL0qN0G-3Cdw`Z8kXH|zftRQ+|VJM<9==zLy-!Txxzcxxd!A07+9?DPC6jkyyNj*nQjM@&s) zx8H=v#gu$I#aAidE~Y0SXAEbhY2WrBIE$IuMCwV z2Kbh5bV)2x4t|Kr!eO?}c#R2JYd)4s1-RBDgpEYv$m}KEE6$@s7&)@X!lhnB6mczN zG^*Re=40tyv!t?@D*7feuD(+Wo_j;tzD;5iqc6pV9bg>Kd=h%1SIT-cZ5g(gYls7T zfK1mED7vI^TL#dwKR|%gcJ*`{m*7j9Us;bKF?=1i9%F+mkhqzPe8*w|6H5#Y`wF8D zZW&W*qs#5K;$D_l%=oUY_9sa0;Op0CqJDw`5xK*{GiJXbYJ?h5MEm=rmb;gzn&NZkF}i(OJ@*@vF_PAg@H2=d-ZbO$`1h|p3tWW; z$z0pYNR|8#Ugb`JOUB zpNkz}`xW&_JsJrSYikpFXYYA^%h-hY?D%n^vl4vvtv2!`@EQ9;+8mFGUR3Z9Q1$=U zVaMrRqYH}|QBjj%_sMv7tLyxQ{|A_3nDJ>^fqJU4{}T)Pr{+x{A3XNN5S{Mtfq^9- zKfW-e;{+I)P&s*beC!_>m^lln(tz^wabJnp&)1iy#YSGt8N-MOHH6_C$oHOxfDq`r zX11c{gVImT^i}8u zNr-}U1i4>30kqhS#q>0{{(RE8=&+xUz+^{ia#P>K@h4UXT?z^bnbnR#JGltaJO%zA zx&Qh}FKJ|V{ydin56*LnYh|TA z;<)xkFeu5;+U4+B7Xkk$z2FTk)lN?ImE*7VydY_k6A{2K}gC@P4s=zF)16U)TSmK$#H|{@?2j#=q7Z^)uZi=#=l#JV1Py z5KQAyPd7*OS|P+v$bm5pfCA$@5P|vPLdgh5fclO1 z+;nJ^W0*aFXsdciHAzjeYnNAmcsW~<>%kb(jy`?);)ybpLfIhX*S&Ch#0bcWD&+|f z&r-ANcplpE@ep>Hh~e`fL4r7g1K3D5BbKKfAXPvp65Q4Nee8@@6bB?e808r7A~&RH z-B{qOgGYmuCIEG)??^Ndvd-cH&Ub&>4)qh}y1zTg#tDMgAQaB z?$9SN$BKym8EPp3V3N#!hw?Va~AwaXOG*;p=Wv zQu>j_?%sk4jf1-`iddbh{J9E|+)lIZK9g zZ`(!z>Tm{Rmw}hbw$(}p8%JBq14)Pjj0ziif2!zT5nntIal^f`DPa%a~KeJfsKXT{8+n|@9&rV3qX{j$Lc=T z^U@uNAkHI9koi?_5o{DNz#9!i28Y*jZg5ss7I8pB#Emzfq#gnGcN`X9u|f>P?SQFx zVq7|s2Z6N}g3LxEx?E9lF`8dEci=cm3qZ~Zb$ZAnp)8(FD36Gy7~@c;B7Ap;np#^& z@j(|lI5=W2MKe-Vrliemw~;65;HWV#V$|DF=l(-(L#x-U!GzUm5)u+*yb*#!DXtUj z;ia7Ozzx?6a<+x8X87xI_LL=|eM4Fe-rK7K5M!G0ID<8~mwvxL*XP(jJT z`p=w(mH>DgKtaNK9BOD36g6HOE^5znp2KV7B6zQhdv9p(n571I2^UsXoiTbcP+&XZ z^gv{myUkvIdfd>{k&x!00HH|jI!*2VWwMm+v$% z5JqFHanxbyAr8PX7&ui$UyjG=#~Cf=5Hg4a(vk^pc`oR#FwA}_lH4u=s%UIa zVyGD!iXg!5SkRFD7dsi_#&hF1YX!ks{cHb@vpyc>&f=^KquMuPfa2_FwOwWhF?Qk- zu*6$HjX|NHELOR`@d`a<2smQfP{4I%2N%Ztc!my5yRqn9mo6T7w9fyp(bEstP#b>v z^eGpf!xagvLi*?~=;#UzbMQ=sm5Z6yZ9`4w)PtxALsg=VVg2M~LiiiS_GK1J#fAz4 z)7#1ql}QB|0dtsKlrloCVQ>ve9}qZ`z1taO1QG;@cZ`k0!!BLm0E0s<8BHF#RvN} z{>hgM)$DJJ@-%Kz=8X2{HDmQfr>8HHv#$MorKGTM*N`G8P#YVY&_isffk^UmE^16{ zpAC}f{&7P$u3vxq@uLrtNbBiOZ-$4X>cf~03W*d)uWA}%zAn2-xSbj?V)!3i`TW5X zxr)Pk=Ls{)ze4)szk&oH*3|gr=58}Gwi38dju>=Iu=RtSkE`#9ONraVgjASYEmk2Ru zyO$0vh7l1Io-eyQhiME^(C@=1G6LG@5nX07P;V?yQf7U9(9+ua1eI~Z`O(YIB+4M$ z<1QY3(aQ>o8|i(DeVU8K^1&hZ{KOeq5}`W5+PnEh(q`H~{aO2)y83kBS|pJf$|}Sh z5>*e^Ff8Ubc+?X{B}|U_3i|E!4G(t-V)cAlSBC}+lQMAqSx=hMv^(7RzO% z?t{{N@BSpcF_N`O42&ohd}Tg2Hw#;-$^<_YEgni$BWN{nrj+7?Ii<{eK8D^Hw*jXr znAGi=rOZcv)eh7p{Fx||6Z10tmmH-p1OBC>RdipyB;cnnXdhmmn z$+t%_!~>RUZX7N|X@{(sWw)6RvEH1Wogv22I(qc9Y#NpdLVUd07n{%>Y~zGnl6b#h z<}=r$Q3VHcR)ORK!i}|vJ^d5t(`}5zkr+BDbfM~-gJcT=X_iHJL_0&*7p*Kc}`dUP^YbbcxVI|Fx zZ8+8Dpj7`Ltc-q$uo;F=Ht8|@Wi~Q)`xS$lj~eG#uy?;Xl$JWeh>4koaSx_V#iwrd zdZpr69u$ovy8NOh2qw}hnbwzL3gx;+SM~o`Xr9I{umQ`;wuX<-2qKC{IIW_E5%(jE z+h{-3Ev9c^z#MLlY>yFzD+6Q8cOl~$d&wEpY>$g(#;6DJQvOZc&?&oi9YYgmKh!uR zzhF8eba$%1-Y8lq4EZ3cKjQ9$kT(j$Fd<^g%OPRtketDR#Dxbpo}UW`Szk}&x7g9| zy^WhhT-C>qM%Z`AL%eKYk03}wlI|y9dt6mVWEtMi3Kep^9w4Rp7zl*T zNQnW)=#9w-b1~#7#a7Qz+2pCLg*b`JA}MC5dHbgXC+#tRIPv-Pz(IZj-O z2VCf=xR1NkB=)_p<&a-{5ibJRQ4`D^R>=fefLXh<{nXq5ObR%EfGT?p zjK3sj6g;yry#HKPWEeRqMY111-br;H{w1R4nRl7yKZGKEhxt-QcBe*?3doNQpu4-2 z`m&48aUv;BP)1f@LJ3+qD4|at9u6*$QfnJZedt_fu@Mz9F+KxsGKSBGtIAs>Qu}E` zqexx;M_@J>s^|k;hs-U2HyMy=eLzsL0^`Q@y0!RsT&P$?IIbsp3)RUmH5wd>5Nce2 ziVL|A9uJ==vJ*=f+KqwDgKG%mrD;K%J-`I$cl`M8O<%tJ32X_A;-_v!pl92-y3ztO zmL+pIY^UhrjpVug@>fmfCg#HrKv=!4rfV+GM2-QbOr0v{=oq~Jrn5x32k1OE+o6|q zPB8aWv}Bck?;dwt8XZrl?Va+nmdPD|dW?lhYXPt=0bL+11)QRgPx&O~FZOaG|U4!yP2M5iwHH+X_0?_6$B%1t?VLqR3YzF;%yTC-NJ6HB%#cpd^N2o{t z3Y{(%R_n8I`16OB%9jVSOBGw&jc6i1q@e@H({fu}^8VS)fr&U@S$V&Vz!}*#@*-r` zBE(1rBt_u+<*hysWF=3<&Q1=fHR0sIzeCjVM{KN``UfSY9A7HO?e#6lxOqKwVV?v(X#bJgZ( zGno$wO7~X*UY4vFT)RLVH`})HgO>#@JSlv?#O?nLO~vzDZ-K&WmY4qn5HFcIv2B~{ zarQqO47D}E6Dko1dfP=Nq+nZn?9 zVoSzYcp3k^b?eroCXf(-ba`A)3Sv%=MvQjKl`C_M40UxE5jy|jLmj=4fPh?-wC+vN z*``5}ZuBTS`zBOiDd^68f<}a_`{q`s!|JCX0)(;(#%7Pb)qpAfUM&7DA?>@B|EdK2C>AU>>A%W+ZIoQ~;rAgRqE_^Y zb)FxIa6n80K!Wo!$n2m}B&KC}lKanR*tQ3s@o}p2f0Q(M4(Z%srsiyjka9z)ES@FJsf$Krs{uk1cHZ}ydDaZYM5l02H zUlXkqA)L_>q+r)WECYpt$cu?m=fSs9XFA>eh8ti+SRg4BY;)1A{~Zmbpgs6sp`o=o z#{JfRt&q&M7WzN2q}G0Wn*SkRJ%d5hKbQn z>z^Yx=3h9}HUCBhmu*jAgFz)z=a9+(QJSBcebMBk|N0@MOnE!p#hNfnno0+(;4sju)l1+i2@UMHi#s3uK%nO=9eQ`X^7ceN-zE_l^ zclLZc7i3;r=!!otoFwa!`rFTZ@iQm-j;lO8d|9Ai?&uxm-KMv0P93eW<_~pxHG7hQ zNWzk(V(#77d;33l8}Z2aZWkBb=6;fu?aG$! z7nR6A(BvL5r$}nWfMetASMiC@_WCao%g$L6_#tW0J9f@PogvjIj{oC5)4X65$rn&z zvlt#7gS$oOuYuc~q12e^qzp;Z*PMyI&$IA`L=@QfW3; zD#L1^b}1@FnU{;n7su3F1k#m&Q&v;gDf5 zTN`)q{akb_wKgOF1(lAT}y8>oiS>f4xBm13^ZPA$U3WB$15lP@bbG* zTYH74RJw@h%a?ZPaeM*B#=`88QrE_&sdnDY9%Hv!vRdF^AfJdR$bUF*-tL&a=Ba!^ zGs?=GBn$Ki1(4!8)3AAR@H1zqVF20o({igm1zQOTDqGbxJNt{Mh;IOK_q0BZ{xbc_ zn&kOh(w$40hljCR5&L-ep`9;I2=k!>}EzQheOK4P02Q5IUY>QT@+nyx)7yhkH5w?tp|L8 z8QaFxJmS(l@~fjW40Nw&v&Sji>4JRgL-{()&Fo^5Ze9+d-Mzbe$zm>y+~qz`H!sc~ zO{@R#yy~|H_~muY{w)bI8O)K8GuJlBDSGJqRcO7XXJ|{mJLP$Bg~tlczSG_P{Tj~B zvKFr<9@;y|zBqchYUG+D>wqN2Wkq4C1BqdzEA%)`U)?fIcp{%S-wAhjeYT{B#~v?Z zHKY0SyBXh>sVtJTO%;32JR1_cLO)jHYrKISKkuFn_B3U8SdFa575Pe4@5`N?8G1)J zq7xHOR*JPD0(dQU!}ntUf(}}mKwvSMK_c1R-@N}s7tLAL7}i+Y_wT=3o?4Y4L#YT} zTGMlwU1#7zk7UgqUA9bKRaRg?HuqFg1Pne!vUM!8slQg>9d(zQGQZohKYe~v(IO#W z;Cg|r@Y=Pk^2H$aF^%l9K)kDBMMe&zOHqV6`c-wbh}AW%Yb@c`;;~HNMeO|QCv@5? zr%ky0M2u;yEv^lXRbi2`PcCgV`eSCP?CyygZ4v>?e{KGLVN+RoC;ZY3-2y&IpU(*8 zS5_Xhb1uH)zf8Vk^1?B09tWccyO|-=)Fn#`%&Vj_Fgv5}eNaZ`Spm@&cfC(V)ziLd zv@Iil*7;`J2ZHAEd6sEOXB|q^;xP&oA~QnjzAu^^7#7Y!D#2V^9e35N#)j8^4EeIr zvMUI`J7cNvlIP$!GuT@D$w$)CVyFwhTs!Iwv*Z9{oXw(;$Yj|!t5w8gRb9DYG**#| zsE7-i%jte*cV+J74;P*CFN~ZTa2H#n#YIv9Y4}7vJ9KUud{@xL?5Ilk|KtjThvZW~Ll;_B!#gELQB25g#G42Odrqvco5<_p^lw9KbPD07hL z>z6b=er&fzO=9WU@_Z~sb{D7YGc@+@W~D3b;#*&Yb@kxjalfNQ#f6V_qIOkM*cK|9 z@j?e<6#HuLFfVS|vg?ekK(@U36Q!+6tThd-VPd&CZQY$3JyDw@e%_}Flasa%-LnVo zXM3L&`m{&>Ok7JmkGrG3xPGPTTc&N9!0+Q@ZPPrGZJAAmxc&StpRLS?otCzmRr!7{ z?la+ux3PRPte9Nd;OK~4VnT(9>L82sAuKYmON}MG=&sDQgO5mk1A7}oC|7$NNuJ)+ zDPBh-BVoy9Hzc=7ik*MKXrUSPFaI`8@1L`sD!~!ye!Y=BotPx4EJR##n_&sqdVoUt zv@UoE3tLcqd_isShVcpYh1IF;+ULz^!r3f>6heFb0^+&>2x|MV|IaZae>5$^4eBYhXAvYKK!T_tyAbONbvW!e;Xa!%~?U7@=mO z?Dj^6zE0KWV)tm@Gt!mLQtabMjUcnfoUUlC!cSVzCD}eVd6#pg{_9FS=hyOW zh;pFwa4`m*2xX6rl)n0NN=(eGiAJ$M_xTHUdwS?ee^4#_s{`?6m)^gpa#r@9%E!=R zFoITQ8etm*(IU>+1Ajp6%$-BFWq4`)n`5ix%!PXjhV8M5iN)ydJv_V?(H&naQ@5qVzQtTYLFifHU$Nr|5Fa~z6Qq4v*b3=SaKQpkWDYUB zwYl!gJ&ic_`?h88`^o=$O=0(NP7+!PgzF2Ew^)fw+));WQYX6%n}r8!?|p@AEoOoX zVq9C&k2EfEFcdb1<|sUZu#3nJ2wzL<3elWWRyH`H1{4eWy}x%34A{g`XcF*1K={SL zb&W%Z{EhSAvq1#&4MC^=_Kh3u5YS>Tpwo(-5h(MRj;szL7h~WyJt2nS=AX_gvT-0l ztPl%^F-n;L{h<@#Ix~8wm4c8ZF?1X|I{UQ1V+_0#nmu}7fbUG&R(_~05A`M+V)Ca?-_Y;}f ze`R=<<-XWs#n8MJM%lRCW126O>e3K-)?@Rvvy2=5foPu-=`}e1uE8P1FQGCAsB*&W z4PU*X{Lf3bI`aR2dHMW5gljZy!(ntH*yRA6cp>KO>Y8=3=c&p>lLIl~9-Z0WFQPjI z+b}Z8w>pD4hY+6!y&cR|VlF2%?A+Xl%hY66PvytIj{kGUrg+NV8%&pc{Hg1Xj^0>Z ztWD+Yk$gb!ijegGBEmjed=@eqj=K+{ktUF6CK1s9d&wJ40lUJVR5PCk@b}T5AOlJS zePJkSnMQZ_6$E-a$fBPnSRLfq)qJ1=o@cjg2cX zy7&`xx3O0U*(?QCed{}=9XrC9-;`*Aci~vjK~!61V8pquvBwYjez)!DUm79y$&!&{oi3en=nx>YP&|O)>jMqSF z!LHtZAmoq1KNB!XIlmGxH~NO3;0WUYM~3D~YP0W&vQ?PuCuJ5p61|u;DBm1Cb=6?rZavQ}+CNMYUJwnNXt_v{Z9(0DG3zbbr zUL?8nr`013fcX{42tTCIgYz57MMY$7Huk>jRHh7$(*E1k7_quI@e{+WyyNUcrulw` z%9A1_2hiIkk>I1wtsMKH@$>nx;2&Zvcm4-sEb3WAjAh)a=f6ygdAr2Z^yA-jv8*=+ z%M`&GNaX8qiw+f&S%L&M5BAh^y5jK7aQg4Xw-+J~;c1C!xbKo#UFWImyj1h1TZMiE zUN&v_xb$tQP(JCa`9J=vU21e*GbqvTJIanyixi z|K^XB-|!Flld$90S9kJoahYI(G7#GZ2;tem3cA8S#{FR7AcpuNHaaq5ACx>B!IQ_2 zcaMcod3bn8)6`UQXu{#OWvteBP^QtVTLF3-o^IS1dmO+6{o(+Z34k3?h%#%U+cggy z$j1QykjE452d|(#M`5}5gGLKFL3gm_d=;koAII3EuV~L%T_TjDZ+pA`Lp+t!cV&cy zQQ|H#?&UXn94(ftWPGZwzKZ8~)J(W2JEvn$$D#SgI5aRK-cI24M<-1aFh+*V(6cU)wVSzP(HNF^tvW(KCRE8QHj>s{; z21;rL4fZUbz&ReX_)G6a;Lc%&n>(f0N07`rih z=Gn6Z2WngE6_K-YE9Z!=m%XkVXW|wwwLb6W-USKrzOlRfHSHVKmx*qeCFOg=a$(S` z-!G0|)jO2(G3}+R`}4|u+MVGdF zFV%QIk#cu~;@QaD2?u&hS6tb*`|g`#Vhht@RwPDQ1KolcPR`R$gG)CtJN)$7me$m> z%*4}f<6kGC(qNRmUmOO{ZWIC`$qWunr_*C!85b|Q_r#ADi^TVL^)hN)-R~p(2)uHi z9RJx4O5fz+n&v)EL=yzvc`h1zUO*Rkp^(rw$1B$$ru+K(3PREf-D({M6F|k8D{qiJ zEP;XxMAII$yjQPS@#f40)mzHE_n~ZW-$;pPL0E;q`wo)L@}m9=#l<&a?K?xjX2ILw3;H9AoJ(tEOfQMR!Ya^YLaQ&m-!& zR6~Di8cWZT&0jH1DV^+nKMdf0&DCEb;mYK z>ZbaAg13+jT6<&lj&>_Xwx}f9zzRsv3JP9=_KRb+OI`heYwx!dNwm0Xh0r4bQ z<$Oa1#Ve#))8rEi#Kio71lf4<<<~rjrS*M-0z{dKXadMU&Krzkq4FRF)`q0@z>)3h z0xJWkDDI_P7p#qqHp+7oK{;Fu0d!{7%|Tb@lLVOBC8-Q~*vJ~<4KjQs&WN$`)9Pc( zF(&xBfMft%Y`K-QrV{G4d$HUMb%sz7E#_Uh*RQ+J&Szl5uNS}BUF*D}`df`Qx&KI3 z7^+c;&NW@|3Unx_Dk|1|OABPiv%ER5S-_X-Z+EsVf{FIWSE`*(>$P8h?v6xmB_Bis zc}(NzoLQu;as@^Vx~aM&xw-kryHZzoEZ(COdf3pAFG4Hkd@C+ws1KbI`##bfqunLd z1uip)H?OWP;N@+juxmqX?H}+rnE;UCEg zKvu0qVb;YyB4|z2u_uy95CvbY13Y;FYI=hIi$N>|GbDTrE=NKO2WtcdkL$pkj8r)4 zb?WKr?vANh=PZid3clgq-=UI3m^Urb%)plm>##(E45BmSJYt41++U5sjWC6S?u2e=1%f_vAn}SxoWK?l{tav6H8DaI*vJ7C#Y7?> zcve86E>CEJtBG(15U)&dkYMH**B$Wg?|YZ)Cy+5}P6qU8jrId|HEEVeS1CG~KP@)Chp zZUG6al;-1ZgdGfySRh5BtO$N)vj?zL86%g*x~tCF+7`nJc*nX@bwX^+RTbY4-03I7 zbC}V%ChCVH06E1bZfJP;idM`X@KQ$Zd5PjM(tjPs=J9(4ch>138*~j!13&=GP(yq| zEdJ1Sh|ufOZ#pyjdizF!Y>9X}p2|qfo1Q1>HpBrrHEXescSWFvFdujJScUDp1%lbu zuxju9i3uMb$R2F?+Aq-GxGZZ~t74^!7cLO21+%2CVo&fG&~eR+vEnp+_M#s**ey_L zzyoW$w$%7FaKuB1(GiSPk|!uUI75pUFW$Dv3Wy})jE$_!-jI4nfW*noM1^Tw7O1Sdw9dMP_8SN#j!S`-Ql1>i+%4v2G;hXc0qvapzBd4br~Tm|;p+nt zlF%**vLL}Kz`DAcjYRZS)bfPYva_?3pvw_H^Fvq8t&=24T>dWKJ?=`d3*fQDxedQf zWQ}r=JU}bq!U^x#|IK3;6W0md5f)Tp@Gkc&7HxB`(Rx^8x~43K+IGJT$ZbK0A!}f-6X9b+cLWJEAH=;>K*V z=xNMqc?mi|xP~xBY^^}UKq%<_Dm=o2aQB9fLCQS<&{4r`vD7=VQZK!tG92@42;?AT49#F9NrN4 z>S3$dPw@t0hxqzv`#C;zC@Bj8Hc)8-aC6Wv5)U`xiy3C@9^zeqP3$nXy%QIOfBv;P z)0md2c;(k4Wk8^_;&)oFqq_)?V3q2kTL1dsg~U~Q%OTh4>{X8i4;9CB&;RNI5H{Y`BYXuKZ&9nwHCFjo9xHCs3kL_N|n116GJm?UoTb5(T5w+F& z>deZ(Pj6$OkAL5wx8U;!lTq{2`H%MSmj*0+6F4$*tgrsD151)Z3rNx?7q;2Xyb~6l ztV~^|v_SW;=cCcC#;<`^=MK|-b6l4!3KV#}^tbWr9qW;ba!a3s&F3l}Xldc_yjdRZX2 zrzayUQXq;(zFrn#nWEh+$?|;B$|yQxFjF30BA{FOT*p*GS*ScZn!nF3thHtl=`g(} zvfcHFAUCKnb3W&Y?8ud1r`t!rOspU4$v_tNoUL_X+sTv7XQQ{#P)BvpjTlj+m~WX6XlZZ25OEE}ziN=x7mtcVPp zOPyX+vC-Hsi_AK*pG@;mP5AnC=U~6TAaJM`c=+28&JXLANV%-k#KZqK6my!-+g#Uk z*>*KlYdAk-#o7j7($qY96%GSqSnR&Qth}yF{+`YD;_2iU(W8}|E)HOj9N{g4L zX(j*854))u%MOu9=hO3t-jNt4$x3;&5A#wDJe_zQb$OrUidXaBZ>$Xn+mldQ*E23W z72etw3w7t1ZPOf5#*w2}udm!^MYo(OCfjvb-E$rS(jX1uUS8@P*5<#shx||R`fo|Xh)u8&Xjb*5}|ghnCwFdtntLH zGtbg!DI;mGem&8~@6%@sH+%XTTsKi?%sDNEQPuM~O@@T$czN^i?d2SEb4>GBONqdAtTH=OK$U4q5gE+Ml0%A zPt1ia;B>O-O4-G829zu{iQ{XtH>zNa`nB_oiql5EJ=Z@)XOc(exJ6#(#})Vj1E`mS zduDt7%0wWwDc9uU4nE&Mb~nHA3UhLXOZ{jpV+O}U@|H5N@!|L*mnbzXhN1{F)JM8(?yx3&k zyx!_CPW|5cl4h%-cYb`YEshTy$Y|`}phqI@C6iAZyt})}Wu$HsBO_D93!+)*HQp_? zYjiiPH@-A`Ws3`^B76m|GftB#Q%{;bDgr9yT;!T~=I8g@&eV(O^$)&q`%!pGWl>$v zW;!&dvoAP!g=Z_<)TMSbtlQa;Wqhod@|f0=%n7&r)ugLel@&!=vt~js?W1PqrMfF$ zPdss@udvYRyv0|LXS{4VIcQd9k zZbrmUO{TsbJn*_ywtzZi-`&()RrkqfB$6$*sjK3Ru<$ur$|qT$ljD7q){}~Fig^O# z^{-vPgI#Xp!J6i|ApR7rY^3Z$2%Ozo{}Uv*J%#@;qy8u-G(D0rNZk(yiwS}?w+u@ zdi-KLfq9)>I^LJz-m~yYZgWpwk!b;mlrBrreRa`T@zM9M`-wB8=Fs){%aG}zaf7h1 z!!t9jQS=GgKIWr6!t5CGqu%~>Q>N1`jkCq|bsxDb9TfdIIAltlYgq{;zqSW6F`9wf zGvF_mc7wiZG|2D6-|AL1uBYBI$#6)mt2ghi(RSWG*xX=XaN$yy7O_q=%c^rN-D;gR zx|U-=MYl~{#8>gcg>CW|q_hs+_fh0snt!=&V>iux(Oag;c`J2?^XqJkGTQ8MQ(k{t z^f)thu{C?ryTCE|#B$x(Z+bc{@zGhLJDi;}=8HYcOPQ1Kt*bHTz)@>b<5Z9G_FPX&|sK(sv z$j$jAZOOQ#8!1_zgQNtGPTgH0J}5U!{CW57h@~+bv&CIY7B40ivOQgu`*6W^lpzt> z!@D6Vs(hllz5`kGkdnVBy-Ew>EQ*vo8ORn?X$-|b<_ zMf`Xo45;=8V!Zbf+qAY=@esLFuOtF1T(|-mA2F5kLq0L7(0JjvNdDdWJ_8``|yR14-GxgTALU!7cE)*dd7E+ z)hp#vYG3v0Y?rw5@{F9b0v<^=;$fvF9~{5mQ@BNHI(aAQ`9l(pL%m99TO8|K_0dC9 zwSKU<)?s|NzfO$Z^cwOJkKdhzlRZ@T#!&=H(=5-aIdDgknVWG2+fclB63hBn#KS3T{+|oHy86x-c>mNP`m*6ddxut^E)rGpl6{T zQ+uMNpot|=GPRHRNCG_GwhkoH&rZqml`A_uifT5DlgB^mxb!}cnU3>t&A@Ob32Yz> zqXN|$>K_peDKv3z-fs!DLoED6ye=fa!=HtgEV+iZRObxfZ6M~qa3j2^0GFWyXu35V z6y5;vd&2EIIywlvB@!L48ylAqs%M^i_S0z&LifbW0u4h9gal|WIF-!!B2Z$T6(He2 z{1t}$4q=~`kM2E2Z@zq=7|p5x_EsL-AEHO7GxMy_Hcg-K+9FFupoDpaHYVL5vc3P= zyde=|H63?E*ssypr%beH{Asa-gob7$cBcf$8t^Y{>~*Wb8~3HKMndjwKEDvVcRxfN zfW)%Ya_)p&;E5P>-gNJV)8U0?Z*XA#ATxWkbM4)5#u2W@-d321nnY;TcNUA?AJ^v5 z>!8H5jnZg$S@jBehDZ8n2-22?D+x#k&BYun{=r_mEv(vSMif_jmSeXK``me=Rd_bl zkE+)}M9h$wc$0d*M$Z20Z^-0x67cYVfW6pjJ{A74XzXy<5qcBZbZp7G$F*#YlTt7{ zgUG!AWv+hmqzI0bX_|gNz4ZP2D^=&zaR?BPJU!vnc$`UeuyJtFQ^O5zYYvz90ex&n z)wu?9MgwYuiI6XPXb+K2!!F&h7-kI4X+3uA-AqY^(|Gi9}4xTp$RSF$YPVOX@26vE|9n&!0O-q-TD0 z*NNf2;7gyubTm34ZX#sw{n?5jcxB%iD8p!PqK5_}!n#I4$k7*-vu(+FASvcaLmjz5 z5FMW-m{TI5&IJHU*l2cdWW{}=Ws66o{xr7Hn+(VQ-p7)m#SSO#?Vqj=8jVRA7z?~h zuXlBKR4<*>QzpwIctngqNG_HrKx26bS-L2UN<%QKe({gi_$`0r;4>Scs=T$@s%dT> zRZd4>=biFkPLx|KnlRvWtZl_qztz&xMF}|-;Hs|&LGfZ{V9GS`c&Do#Ym`hNk8r`k zAqXLpa3%r1g$p{sC=&pITNFn7(vpc*7Cl`7a$)&6ZL~sJ0sZRF85g3sX3u_)t?~B~ z=UTOd_Vg1Q&9;U|KYX}zKkXmdXZ`K{ae&v6wjILE6+x0c$R;73=MV+35m9wb9dSov z>Lp~q&Kba73Qgo-;lAR7HN@nYT2Jq003gg3RV2D)J07^% zjmc)#1Wl$LUdFcQC^)6SL4pucepV*yH)1(&HdP8GtOj4a1k`q-uhU%NoOl$Y1#Cv$ zfk%7x;srrgM_xuAXktv8C`}MU7$}A#^96fASpBW3VM0dcX9@d#BKmzwqW%P#6kyoz z5;2D@B{vOIvK;qkYGid#y6XGJqluXX2%h8o%vR?@k|a%e?~OvUiA;xOvC-M>eSJ6J zGa!Ia)TvU69A?4++M2CdoP?=Jm=*mvHFYrpTs9#ibRoTt*5ho;^VBV>;uH&J{Z#UD zuQw(kv^8Cccl$JL?~|MRBU^l0m!u81HV;1d%sY@c?o!2J1z zNG2v@>10gT(~6Ls4&J<}zhfioU#&}j>tC(QQWW`^D?)0Nba#$f#&=zYz2zmebaaK$ z&GSx7Ol(ee1YO#11YsL7oM|-U!fI;Ufr%{!Q3(btuhp7&KON$Kj>gzdaLk}iS@-tO z*PkvHNn_PFuzu)D)%hpc^Ci{9xV<>wk2sJOHCytvq}&wtoH~jjaYowb+0ProF*b!A0~b0Y?D-<9tva zm`g-_5U)>w{D}E^4vi_IY12Q(jsSZ~3@=1#6q*h(Lbm+kSSpJjZV`eW?mxztS92Y!0p{vcIMv~AhyfU@c!)ysPVgyjnYjp^o4SLAH+Mfo?jfA3RI;F zqsL7=ebH@hi2pbM!VQGHc8prs2Rl|lv5J_;f3-oy^R5xkLxNs{=C&6`Mx&&T1u9xF6VsRx!^~~Z$goe;9aFZ!Ow>p__6Hz|P3r@8=SV3=q`T~{d0EWJlx4TL?3vGk)Zhs!2yXE%Yf|ItZY)0k1IO^>LxR>jwODdIJTXC=o9mv{$CV9KW@pbrG8ZWQ`tPM ztC{_89p(MYTgRvO8^Kni1{5hC9;pWDc8gTYwz0sQ_UYdB_@Fc&?m-xSTq9?X7$ru2 zgsk=jgX`5nI@{|fTvh*NL9^bUFMdSAVfo|11QfU#2P@0UQZMzJQ)7Sj8M3Yi%_Y`o ze0^MP)0_RXk`@$XE?Tq5kceQSu6XK3ZTDRR;98NIpU*%{_ROyq#ef~|^4>n-tS2b+ zcSLB_B3Z|Sq zITx~Q)A@%rJ@^KKy>I9>VGG3DKz*{lkb5Zk2;xcQhr~a@;)Y2DQ;|>Wo!8Rh z_uIP?>fMFjkLz&Tytqq7@$EAQE-4a8YL*}f?GFk#u}|G=X7;$b?<>p)UI4m8p-w)N z^7&iE4+*Zemhhm0AM@#zb+}RH-%ITTgNz{7$Gm{1(n(KcGCe3L=vQDp3XxGLSYaJ{ z8z`E;lr=$5RG2*wAlE|$Z1*WN6HNKM5gE!7!UB6kD<%lrJVC*eG>=?ANpg!Riyxb| zVAP{dv2KF5P?K}Cy$nFf>HFF0E_@r|{Xk9K1aod(E9gBk++Mp7>Id{ogOA$_uGe)v z@G#v1fN*{5Ka{5-!;amu{e5LjpTrb=>QW4qwSKkz22oi5rSE)d3QdAXm#{k$TPR{i z37m4P<55ZhfWkaGVf(S80KEaiQdME#Sm^SgVJfKxPErzL5*4o zt`w?0Y|~&T(vxsF9QE(wpo9a#bOUnmCUW=<4f+JM0_^UW@9i*PrBi4mfNJFK{vakM zrY7~lUG~3KS_O_o`EyeJj&+3O4LM;xrzvIHx`!(%UH|t|YL!vYToOU@T#69WW%~Cw zSakeQbCF0WUre@!A31!uEj0-ha~mqet>lWd>Z58TK4CeMXaAF(XMahl)&o1czvEMs zbm8F(p^}T6+iY^^3*I5B!K2mUn_lbvXI*s1nn3J9R7F(+TwMPM`Fnp~Nv>bwzn3?% zq{h7NwvB{}|Cju{B{b=8#l+(#dOrs3KxzbVS=5_fJRi0iyRYvA-xe0Go1y@H00B!-TMXmPFnDnm^VBt`NJF2@19v6XcYv*$0Rajqt2?Qd^$A~y;s7%~Yg|FP_JaW3 zj?Z=?-9}t!lRlS#fB>8n#C8CyW73I@VBEij^{juS>;D(6b(8&Bjv~bDLsc%l#KS#Q z*%_!5dyV~xv9UpDK!_-u@&RnJ6qd8^LM8|Q+eD?+m)%&<0r(j}nSBb825V|oN(+a={c?%k&*nlE9oH$3A4L_x} z??kN}#|(Qv{o;XI>4;M#eC^lJQcu`za`k@!?0gE| literal 0 HcmV?d00001 diff --git a/docs/multitenant/openssl_schema.jpg b/docs/multitenant/ords-based/openssl_schema.jpg similarity index 100% rename from docs/multitenant/openssl_schema.jpg rename to docs/multitenant/ords-based/openssl_schema.jpg diff --git a/docs/multitenant/provisioning/example_setup_using_oci_oke_cluster.md b/docs/multitenant/ords-based/provisioning/example_setup_using_oci_oke_cluster.md similarity index 100% rename from docs/multitenant/provisioning/example_setup_using_oci_oke_cluster.md rename to docs/multitenant/ords-based/provisioning/example_setup_using_oci_oke_cluster.md diff --git a/docs/multitenant/provisioning/multinamespace/cdb_create.yaml b/docs/multitenant/ords-based/provisioning/multinamespace/cdb_create.yaml similarity index 58% rename from docs/multitenant/provisioning/multinamespace/cdb_create.yaml rename to docs/multitenant/ords-based/provisioning/multinamespace/cdb_create.yaml index d3b5e04f..8ace42e8 100644 --- a/docs/multitenant/provisioning/multinamespace/cdb_create.yaml +++ b/docs/multitenant/ords-based/provisioning/multinamespace/cdb_create.yaml @@ -1,4 +1,4 @@ -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: CDB metadata: name: cdb-dev @@ -11,28 +11,28 @@ spec: replicas: 1 sysAdminPwd: secret: - secretName: "cdb1-secret" - key: "sysadmin_pwd" + secretName: "[...]" + key: "[...]" ordsPwd: secret: - secretName: "cdb1-secret" - key: "ords_pwd" + secretName: "[...]" + key: "[...]" cdbAdminUser: secret: - secretName: "cdb1-secret" - key: "cdbadmin_user" + secretName: "[...]" + key: "[...]" cdbAdminPwd: secret: - secretName: "cdb1-secret" - key: "cdbadmin_pwd" + secretName: "[...]" + key: "[...]" webServerUser: secret: - secretName: "cdb1-secret" - key: "webserver_user" + secretName: "[...]" + key: "[...]" webServerPwd: secret: - secretName: "cdb1-secret" - key: "webserver_pwd" + secretName: "[...]" + key: "[...]" cdbTlsKey: secret: secretName: "db-tls" @@ -41,4 +41,8 @@ spec: secret: secretName: "db-tls" key: "tls.crt" + cdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/provisioning/multinamespace/pdb_clone.yaml b/docs/multitenant/ords-based/provisioning/multinamespace/pdb_clone.yaml similarity index 74% rename from docs/multitenant/provisioning/multinamespace/pdb_clone.yaml rename to docs/multitenant/ords-based/provisioning/multinamespace/pdb_clone.yaml index b88fb71b..4dac1aea 100644 --- a/docs/multitenant/provisioning/multinamespace/pdb_clone.yaml +++ b/docs/multitenant/ords-based/provisioning/multinamespace/pdb_clone.yaml @@ -2,7 +2,7 @@ # Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: PDB metadata: name: pdb2 @@ -21,12 +21,12 @@ spec: assertivePdbDeletion: true adminName: secret: - secretName: "pdb1-secret" - key: "sysadmin_user" + secretName: "[...]" + key: "[...]" adminPwd: secret: - secretName: "pdb1-secret" - key: "sysadmin_pwd" + secretName: "[...]" + key: "[...]" pdbTlsKey: secret: secretName: "db-tls" @@ -41,10 +41,14 @@ spec: key: "ca.crt" webServerUser: secret: - secretName: "pdb1-secret" - key: "webserver_user" + secretName: "[...]" + key: "[...]" webServerPwd: secret: - secretName: "pdb1-secret" - key: "webserver_pwd" + secretName: "[...]" + key: "[...]" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" action: "Clone" diff --git a/docs/multitenant/provisioning/multinamespace/pdb_close.yaml b/docs/multitenant/ords-based/provisioning/multinamespace/pdb_close.yaml similarity index 66% rename from docs/multitenant/provisioning/multinamespace/pdb_close.yaml rename to docs/multitenant/ords-based/provisioning/multinamespace/pdb_close.yaml index a823f5d9..44b1a086 100644 --- a/docs/multitenant/provisioning/multinamespace/pdb_close.yaml +++ b/docs/multitenant/ords-based/provisioning/multinamespace/pdb_close.yaml @@ -1,4 +1,4 @@ -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: PDB metadata: name: pdb1 @@ -12,12 +12,12 @@ spec: pdbName: "pdbdev" adminName: secret: - secretName: "pdb1-secret" - key: "sysadmin_user" + secretName: "[...]" + key: "[...]" adminPwd: secret: - secretName: "pdb1-secret" - key: "sysadmin_pwd" + secretName: "[...]" + key: "[...]" pdbTlsKey: secret: secretName: "db-tls" @@ -32,12 +32,16 @@ spec: key: "ca.crt" webServerUser: secret: - secretName: "pdb1-secret" - key: "webserver_user" + secretName: "[...]" + key: "[...]" webServerPwd: secret: - secretName: "pdb1-secret" - key: "webserver_pwd" + secretName: "[...]" + key: "[...]" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" pdbState: "CLOSE" modifyOption: "IMMEDIATE" action: "Modify" diff --git a/docs/multitenant/provisioning/multinamespace/pdb_create.yaml b/docs/multitenant/ords-based/provisioning/multinamespace/pdb_create.yaml similarity index 67% rename from docs/multitenant/provisioning/multinamespace/pdb_create.yaml rename to docs/multitenant/ords-based/provisioning/multinamespace/pdb_create.yaml index 200f3712..2bf2189b 100644 --- a/docs/multitenant/provisioning/multinamespace/pdb_create.yaml +++ b/docs/multitenant/ords-based/provisioning/multinamespace/pdb_create.yaml @@ -1,4 +1,4 @@ -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: PDB metadata: name: pdb1 @@ -12,12 +12,12 @@ spec: pdbName: "pdbdev" adminName: secret: - secretName: "pdb1-secret" - key: "sysadmin_user" + secretName: "[...]" + key: "[...]" adminPwd: secret: - secretName: "pdb1-secret" - key: "sysadmin_pwd" + secretName: "[...]" + key: "[...]" pdbTlsKey: secret: secretName: "db-tls" @@ -32,12 +32,16 @@ spec: key: "ca.crt" webServerUser: secret: - secretName: "pdb1-secret" - key: "webserver_user" + secretName: "[...]" + key: "[...]" webServerPwd: secret: - secretName: "pdb1-secret" - key: "webserver_pwd" + secretName: "[...]" + key: "[...]" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" fileNameConversions: "NONE" tdeImport: false totalSize: "1G" diff --git a/docs/multitenant/provisioning/multinamespace/pdb_delete.yaml b/docs/multitenant/ords-based/provisioning/multinamespace/pdb_delete.yaml similarity index 70% rename from docs/multitenant/provisioning/multinamespace/pdb_delete.yaml rename to docs/multitenant/ords-based/provisioning/multinamespace/pdb_delete.yaml index 282885b0..296c9feb 100644 --- a/docs/multitenant/provisioning/multinamespace/pdb_delete.yaml +++ b/docs/multitenant/ords-based/provisioning/multinamespace/pdb_delete.yaml @@ -1,4 +1,4 @@ -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: PDB metadata: name: pdb1 @@ -25,10 +25,15 @@ spec: key: "ca.crt" webServerUser: secret: - secretName: "pdb1-secret" - key: "webserver_user" + secretName: "[...]" + key: "[...]" webServerPwd: secret: - secretName: "pdb1-secret" - key: "webserver_pwd" + secretName: "[...]" + key: "[...]" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" + diff --git a/docs/multitenant/provisioning/multinamespace/pdb_open.yaml b/docs/multitenant/ords-based/provisioning/multinamespace/pdb_open.yaml similarity index 66% rename from docs/multitenant/provisioning/multinamespace/pdb_open.yaml rename to docs/multitenant/ords-based/provisioning/multinamespace/pdb_open.yaml index 85fb2ce4..9f85f0b5 100644 --- a/docs/multitenant/provisioning/multinamespace/pdb_open.yaml +++ b/docs/multitenant/ords-based/provisioning/multinamespace/pdb_open.yaml @@ -1,4 +1,4 @@ -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: PDB metadata: name: pdb1 @@ -12,12 +12,12 @@ spec: pdbName: "pdbdev" adminName: secret: - secretName: "pdb1-secret" - key: "sysadmin_user" + secretName: "[...]" + key: "[...]" adminPwd: secret: - secretName: "pdb1-secret" - key: "sysadmin_pwd" + secretName: "[...]" + key: "[...]" pdbTlsKey: secret: secretName: "db-tls" @@ -32,12 +32,16 @@ spec: key: "ca.crt" webServerUser: secret: - secretName: "pdb1-secret" - key: "webserver_user" + secretName: "[...]" + key: "[...]" webServerPwd: secret: - secretName: "pdb1-secret" - key: "webserver_pwd" + secretName: "[...]" + key: "[...]" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" action: "Modify" pdbState: "OPEN" modifyOption: "READ WRITE" diff --git a/docs/multitenant/provisioning/multinamespace/pdb_plug.yaml b/docs/multitenant/ords-based/provisioning/multinamespace/pdb_plug.yaml similarity index 80% rename from docs/multitenant/provisioning/multinamespace/pdb_plug.yaml rename to docs/multitenant/ords-based/provisioning/multinamespace/pdb_plug.yaml index d9135f13..10719ccc 100644 --- a/docs/multitenant/provisioning/multinamespace/pdb_plug.yaml +++ b/docs/multitenant/ords-based/provisioning/multinamespace/pdb_plug.yaml @@ -2,7 +2,7 @@ # Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: PDB metadata: name: pdb1 @@ -36,11 +36,16 @@ spec: key: "ca.crt" webServerUser: secret: - secretName: "pdb1-secret" - key: "webserver_user" + secretName: "[...]" + key: "[...]" webServerPwd: secret: - secretName: "pdb1-secret" - key: "webserver_pwd" + secretName: "[...]" + key: "[...]" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" + diff --git a/docs/multitenant/provisioning/multinamespace/pdb_unplug.yaml b/docs/multitenant/ords-based/provisioning/multinamespace/pdb_unplug.yaml similarity index 77% rename from docs/multitenant/provisioning/multinamespace/pdb_unplug.yaml rename to docs/multitenant/ords-based/provisioning/multinamespace/pdb_unplug.yaml index f3667dad..f30f2699 100644 --- a/docs/multitenant/provisioning/multinamespace/pdb_unplug.yaml +++ b/docs/multitenant/ords-based/provisioning/multinamespace/pdb_unplug.yaml @@ -2,7 +2,7 @@ # Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: PDB metadata: name: pdb1 @@ -30,10 +30,14 @@ spec: key: "ca.crt" webServerUser: secret: - secretName: "pdb1-secret" - key: "webserver_user" + secretName: "[...]" + key: "[...]" webServerPwd: secret: - secretName: "pdb1-secret" - key: "webserver_pwd" + secretName: "[...]" + key: "[...]" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/ords-based/provisioning/ords_image.md b/docs/multitenant/ords-based/provisioning/ords_image.md new file mode 100644 index 00000000..e2d1dcef --- /dev/null +++ b/docs/multitenant/ords-based/provisioning/ords_image.md @@ -0,0 +1,81 @@ + + +# Build ORDS Docker Image + +This file contains the steps to create an ORDS based image to be used solely by the PDB life cycle multitentant controllers. + +**NOTE:** It is assumed that before this step, you have followed the [prerequisite](./../README.md#prerequsites-to-manage-pdb-life-cycle-using-oracle-db-operator-on-prem-database-controller) steps. + +#### Clone the software using git: + +> Under directory ./oracle-database-operator/ords you will find the [Dockerfile](../../../ords/Dockerfile) and [runOrdsSSL.sh](../../../ords/runOrdsSSL.sh) required to build the image. + +```sh + git clone git@orahub.oci.oraclecorp.com:rac-docker-dev/oracle-database-operator.git + cd oracle-database-operator/ords/ +``` + +#### Login to the registry: container-registry.oracle.com + +**NOTE:** To login to this registry, you will need to the URL https://container-registry.oracle.com , Sign in, then click on "Java" and then accept the agreement. + +```bash +docker login container-registry.oracle.com +``` + +#### Login to the your container registry + +Login to a repo where you want to push your docker image (if needed) to pull during deployment in your environment. + +```bash +docker login +``` + +#### Build the image + +Build the docker image by using below command: + +```bash +docker build -t oracle/ords-dboper:latest . +``` +> If your are working behind a proxy mind to specify https_proxy and http_proxy during image creation + +Check the docker image details using: + +```bash +docker images +``` + +> OUTPUT EXAMPLE +```bash +REPOSITORY TAG IMAGE ID CREATED SIZE +oracle/ords-dboper latest fdb17aa242f8 4 hours ago 1.46GB + +``` + +#### Tag and push the image + +Tag and push the image to your image repository. + +NOTE: We have the repo as `phx.ocir.io//oracle/ords:latest`. Please change as per your environment. + +```bash +docker tag oracle/ords-dboper:ords-latest phx.ocir.io//oracle/ords:latest +docker push phx.ocir.io//oracle/ords:latest +``` + +#### In case of private image + +If you the image not be public then yuo need to create a secret containing the password of your image repository. +Create a Kubernetes Secret for your docker repository to pull the image during deployment using the below command: + +```bash +kubectl create secret generic container-registry-secret --from-file=.dockerconfigjson=./.docker/config.json --type=kubernetes.io/dockerconfigjson -n oracle-database-operator-system +``` + +Use the parameter `ordsImagePullSecret` to specify the container secrets in pod creation yaml file + +#### [Image createion example](../usecase01/logfiles/BuildImage.log) + + + diff --git a/docs/multitenant/provisioning/quickOKEcreation.md b/docs/multitenant/ords-based/provisioning/quickOKEcreation.md similarity index 100% rename from docs/multitenant/provisioning/quickOKEcreation.md rename to docs/multitenant/ords-based/provisioning/quickOKEcreation.md diff --git a/docs/multitenant/usecase01/cdb_create.yaml b/docs/multitenant/ords-based/provisioning/singlenamespace/cdb_create.yaml similarity index 50% rename from docs/multitenant/usecase01/cdb_create.yaml rename to docs/multitenant/ords-based/provisioning/singlenamespace/cdb_create.yaml index 01fc0a18..5e020de6 100644 --- a/docs/multitenant/usecase01/cdb_create.yaml +++ b/docs/multitenant/ords-based/provisioning/singlenamespace/cdb_create.yaml @@ -1,4 +1,4 @@ -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: CDB metadata: name: cdb-dev @@ -11,34 +11,39 @@ spec: replicas: 1 sysAdminPwd: secret: - secretName: "cdb1-secret" - key: "sysadmin_pwd" + secretName: "[...]" + key: "[...]" ordsPwd: secret: - secretName: "cdb1-secret" - key: "ords_pwd" + secretName: "[...]" + key: "[...]" cdbAdminUser: secret: - secretName: "cdb1-secret" - key: "cdbadmin_user" + secretName: "[...]" + key: "[...]" cdbAdminPwd: secret: - secretName: "cdb1-secret" - key: "cdbadmin_pwd" + secretName: "[...]" + key: "[...]" webServerUser: secret: - secretName: "cdb1-secret" - key: "webserver_user" + secretName: "[...]" + key: "[...]" webServerPwd: secret: - secretName: "cdb1-secret" - key: "webserver_pwd" + secretName: "[...]" + key: "[...]" cdbTlsKey: secret: - secretName: "db-tls" - key: "tls.key" + secretName: "[...]" + key: "[...]" cdbTlsCrt: secret: - secretName: "db-tls" - key: "tls.crt" + secretName: "[...]" + key: "[...]" + cdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" + diff --git a/docs/multitenant/provisioning/singlenamespace/cdb_secret.yaml b/docs/multitenant/ords-based/provisioning/singlenamespace/cdb_secret.yaml similarity index 100% rename from docs/multitenant/provisioning/singlenamespace/cdb_secret.yaml rename to docs/multitenant/ords-based/provisioning/singlenamespace/cdb_secret.yaml diff --git a/docs/multitenant/provisioning/singlenamespace/pdb_clone.yaml b/docs/multitenant/ords-based/provisioning/singlenamespace/pdb_clone.yaml similarity index 78% rename from docs/multitenant/provisioning/singlenamespace/pdb_clone.yaml rename to docs/multitenant/ords-based/provisioning/singlenamespace/pdb_clone.yaml index 0ecc3c70..964d1e5e 100644 --- a/docs/multitenant/provisioning/singlenamespace/pdb_clone.yaml +++ b/docs/multitenant/ords-based/provisioning/singlenamespace/pdb_clone.yaml @@ -21,12 +21,12 @@ spec: assertivePdbDeletion: true adminName: secret: - secretName: "pdb1-secret" - key: "sysadmin_user" + secretName: "[...]" + key: "[...]" adminPwd: secret: - secretName: "pdb1-secret" - key: "sysadmin_pwd" + secretName: "[...]" + key: "[...]" pdbTlsKey: secret: secretName: "db-tls" @@ -47,4 +47,14 @@ spec: secret: secretName: "pdb1-secret" key: "webserver_pwd" + secretName: "[...]" + key: "[...]" + webServerPwd: + secret: + secretName: "[...]" + key: "[...]" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" action: "Clone" diff --git a/docs/multitenant/usecase01/pdb_close.yaml b/docs/multitenant/ords-based/provisioning/singlenamespace/pdb_close.yaml similarity index 67% rename from docs/multitenant/usecase01/pdb_close.yaml rename to docs/multitenant/ords-based/provisioning/singlenamespace/pdb_close.yaml index 5917d33a..06d92469 100644 --- a/docs/multitenant/usecase01/pdb_close.yaml +++ b/docs/multitenant/ords-based/provisioning/singlenamespace/pdb_close.yaml @@ -1,4 +1,4 @@ -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: PDB metadata: name: pdb1 @@ -12,12 +12,12 @@ spec: pdbName: "pdbdev" adminName: secret: - secretName: "pdb1-secret" - key: "sysadmin_user" + secretName: "[...]" + key: "[...]" adminPwd: secret: - secretName: "pdb1-secret" - key: "sysadmin_pwd" + secretName: "[...]" + key: "[...]" pdbTlsKey: secret: secretName: "db-tls" @@ -32,12 +32,16 @@ spec: key: "ca.crt" webServerUser: secret: - secretName: "pdb1-secret" - key: "webserver_user" + secretName: "[...]" + key: "[...]" webServerPwd: secret: - secretName: "pdb1-secret" - key: "webserver_pwd" + secretName: "[...]" + key: "[...]" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" pdbState: "CLOSE" modifyOption: "IMMEDIATE" action: "Modify" diff --git a/docs/multitenant/usecase01/pdb_create.yaml b/docs/multitenant/ords-based/provisioning/singlenamespace/pdb_create.yaml similarity index 69% rename from docs/multitenant/usecase01/pdb_create.yaml rename to docs/multitenant/ords-based/provisioning/singlenamespace/pdb_create.yaml index be3581ad..2744223e 100644 --- a/docs/multitenant/usecase01/pdb_create.yaml +++ b/docs/multitenant/ords-based/provisioning/singlenamespace/pdb_create.yaml @@ -1,4 +1,4 @@ -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: PDB metadata: name: pdb1 @@ -12,12 +12,12 @@ spec: pdbName: "pdbdev" adminName: secret: - secretName: "pdb1-secret" - key: "sysadmin_user" + secretName: "[...]" + key: "[...]" adminPwd: secret: - secretName: "pdb1-secret" - key: "sysadmin_pwd" + secretName: "[...]" + key: "[...]" pdbTlsKey: secret: secretName: "db-tls" @@ -32,12 +32,16 @@ spec: key: "ca.crt" webServerUser: secret: - secretName: "pdb1-secret" - key: "webserver_user" + secretName: "[...]" + key: "[...]" webServerPwd: secret: - secretName: "pdb1-secret" - key: "webserver_pwd" + secretName: "[...]" + key: "[...]" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" fileNameConversions: "NONE" tdeImport: false totalSize: "1G" diff --git a/docs/multitenant/usecase01/pdb_delete.yaml b/docs/multitenant/ords-based/provisioning/singlenamespace/pdb_delete.yaml similarity index 72% rename from docs/multitenant/usecase01/pdb_delete.yaml rename to docs/multitenant/ords-based/provisioning/singlenamespace/pdb_delete.yaml index c22b546a..523ac1cb 100644 --- a/docs/multitenant/usecase01/pdb_delete.yaml +++ b/docs/multitenant/ords-based/provisioning/singlenamespace/pdb_delete.yaml @@ -1,4 +1,4 @@ -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: PDB metadata: name: pdb1 @@ -25,10 +25,15 @@ spec: key: "ca.crt" webServerUser: secret: - secretName: "pdb1-secret" - key: "webserver_user" + secretName: "[...]" + key: "[...]" webServerPwd: secret: - secretName: "pdb1-secret" - key: "webserver_pwd" + secretName: "[...]" + key: "[...]" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" + diff --git a/docs/multitenant/usecase01/pdb_open.yaml b/docs/multitenant/ords-based/provisioning/singlenamespace/pdb_open.yaml similarity index 67% rename from docs/multitenant/usecase01/pdb_open.yaml rename to docs/multitenant/ords-based/provisioning/singlenamespace/pdb_open.yaml index 25fdccc4..866db3e4 100644 --- a/docs/multitenant/usecase01/pdb_open.yaml +++ b/docs/multitenant/ords-based/provisioning/singlenamespace/pdb_open.yaml @@ -1,4 +1,4 @@ -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: PDB metadata: name: pdb1 @@ -12,12 +12,12 @@ spec: pdbName: "pdbdev" adminName: secret: - secretName: "pdb1-secret" - key: "sysadmin_user" + secretName: "[...]" + key: "[...]" adminPwd: secret: - secretName: "pdb1-secret" - key: "sysadmin_pwd" + secretName: "[...]" + key: "[...]" pdbTlsKey: secret: secretName: "db-tls" @@ -32,12 +32,16 @@ spec: key: "ca.crt" webServerUser: secret: - secretName: "pdb1-secret" - key: "webserver_user" + secretName: "[...]" + key: "[...]" webServerPwd: secret: - secretName: "pdb1-secret" - key: "webserver_pwd" + secretName: "[...]" + key: "[...]" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" action: "Modify" pdbState: "OPEN" modifyOption: "READ WRITE" diff --git a/docs/multitenant/usecase02/pdb_plug.yaml b/docs/multitenant/ords-based/provisioning/singlenamespace/pdb_plug.yaml similarity index 81% rename from docs/multitenant/usecase02/pdb_plug.yaml rename to docs/multitenant/ords-based/provisioning/singlenamespace/pdb_plug.yaml index 77c00b9c..e6605276 100644 --- a/docs/multitenant/usecase02/pdb_plug.yaml +++ b/docs/multitenant/ords-based/provisioning/singlenamespace/pdb_plug.yaml @@ -2,7 +2,7 @@ # Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: PDB metadata: name: pdb1 @@ -42,5 +42,14 @@ spec: secret: secretName: "pdb1-secret" key: "webserver_pwd" - + secretName: "[...]" + key: "[...]" + webServerPwd: + secret: + secretName: "[...]" + key: "[...]" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/provisioning/singlenamespace/pdb_secret.yaml b/docs/multitenant/ords-based/provisioning/singlenamespace/pdb_secret.yaml similarity index 100% rename from docs/multitenant/provisioning/singlenamespace/pdb_secret.yaml rename to docs/multitenant/ords-based/provisioning/singlenamespace/pdb_secret.yaml diff --git a/docs/multitenant/usecase02/pdb_unplug.yaml b/docs/multitenant/ords-based/provisioning/singlenamespace/pdb_unplug.yaml similarity index 78% rename from docs/multitenant/usecase02/pdb_unplug.yaml rename to docs/multitenant/ords-based/provisioning/singlenamespace/pdb_unplug.yaml index 085d337e..4e404efe 100644 --- a/docs/multitenant/usecase02/pdb_unplug.yaml +++ b/docs/multitenant/ords-based/provisioning/singlenamespace/pdb_unplug.yaml @@ -2,7 +2,7 @@ # Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: PDB metadata: name: pdb1 @@ -36,4 +36,14 @@ spec: secret: secretName: "pdb1-secret" key: "webserver_pwd" + secretName: "[...]" + key: "[...]" + webServerPwd: + secret: + secretName: "[...]" + key: "[...]" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/ords-based/usecase/README.md b/docs/multitenant/ords-based/usecase/README.md new file mode 100644 index 00000000..b6f5e590 --- /dev/null +++ b/docs/multitenant/ords-based/usecase/README.md @@ -0,0 +1,112 @@ + + + +# Use case directory + +The use case directory contains the yaml files to test the multitenant controller functionalities: create ords pod and pdb operation *create / open / close / unplug / plug / delete / clone /map / parameter session* +In this exampl the cdb and pdbs resources are depolyed in different namespaces + +## Makefile helper + +Customizing yaml files (tns alias / credential / namespaces name etc...) is a long procedure prone to human error. A simple [makefile](../usecase/makefile) is available to quickly and safely configure yaml files with your system environment information. Just edit the [parameter file](../usecase/parameters.txt) before proceding. + +```text +[👉 CHECK PARAMETERS..................] +TNSALIAS...............:(DESCRIPTION=(CONNECT_TIMEOUT=90)(RETRY_COUNT=30)(RETRY_DELA.... +ORDPWD.................:[Password for ORDS_PUBLIC_USER ] +SYSPWD.................:[SYS password] +WBUSER.................:[username for https authentication] +WBPASS.................:[password for https authentication] +PDBUSR.................:[pdb admin user] +PDBPWD.................:[pdb admin password] +CDBUSR.................:[cdb admin user e.g. C##DBAPI_CDB_ADMIN] +CDBPWD.................:[cdb admin password] +PDBNAMESPACE...........:[namespace for pdb] +CDBNAMESPACE...........:[namespace for cdb] +COMPANY................:oracle +APIVERSION.............:v4 ---> do not edit +``` + +âš  **WARNING: The makefile is intended to speed up the usecase directory configuartion only, it is not supported, the editing and configuration of yaml files for production system is left up to the end user** + +### Pre requisistes: + +- Make sure that **kubectl** is properly configured. +- Make sure that all requirements listed in the [operator installation page](../../../../docs/installation/OPERATOR_INSTALLATION_README.md) are implemented. (role binding,webcert,etc) +- Make sure that administrative user on the container database is configured as documented. + +### Commands + +Review your configuraton running ```make check```; if all the parameters are correct then you can proceed with yaml files and certificates generation + +By excuting command ```make operator``` You will have in your directory an operator yaml file with the WATCH LIST required to operate with multiple namespaces. +Note that the yaml file is not applyed; you need to manually execute ```kubectl apply -f oracle-database-operator.yaml```. + +```bash +make operator +``` +You can generate all the other yaml files for pdb life cycle management using ```make genyaml``` + +```bash +make genyaml +``` + +list of generated yaml files + +```text +-rw-r--r-- 1 mmalvezz g900 137142 Nov 13 09:35 oracle-database-operator.yaml +-rw-r--r-- 1 mmalvezz g900 321 Nov 13 10:27 create_cdb_secrets.yaml +-rw-r--r-- 1 mmalvezz g900 234 Nov 13 10:27 create_pdb_secrets.yaml +-rw-r--r-- 1 mmalvezz g900 381 Nov 13 10:27 pdbnamespace_binding.yaml +-rw-r--r-- 1 mmalvezz g900 381 Nov 13 10:27 cdbnamespace_binding.yaml +-rw-r--r-- 1 mmalvezz g900 1267 Nov 13 10:27 create_ords_pod.yaml +-rw-r--r-- 1 mmalvezz g900 935 Nov 13 10:27 create_pdb1_resource.yaml +-rw-r--r-- 1 mmalvezz g900 935 Nov 13 10:27 create_pdb2_resource.yaml +-rw-r--r-- 1 mmalvezz g900 842 Nov 13 10:27 open_pdb1_resource.yaml +-rw-r--r-- 1 mmalvezz g900 842 Nov 13 10:27 open_pdb2_resource.yaml +-rw-r--r-- 1 mmalvezz g900 845 Nov 13 10:27 open_pdb3_resource.yaml +-rw-r--r-- 1 mmalvezz g900 842 Nov 13 10:27 close_pdb1_resource.yaml +-rw-r--r-- 1 mmalvezz g900 842 Nov 13 10:27 close_pdb2_resource.yaml +-rw-r--r-- 1 mmalvezz g900 846 Nov 13 10:27 close_pdb3_resource.yaml +-rw-r--r-- 1 mmalvezz g900 927 Nov 13 10:27 clone_pdb1_resource.yaml +-rw-r--r-- 1 mmalvezz g900 928 Nov 13 10:27 clone_pdb2_resource.yaml +-rw-r--r-- 1 mmalvezz g900 802 Nov 13 10:27 delete_pdb1_resource.yaml +-rw-r--r-- 1 mmalvezz g900 802 Nov 13 10:27 delete_pdb2_resource.yaml +-rw-r--r-- 1 mmalvezz g900 824 Nov 13 10:27 unplug_pdb1_resource.yaml +-rw-r--r-- 1 mmalvezz g900 992 Nov 13 10:27 plug_pdb1_resource.yaml +-rw-r--r-- 1 mmalvezz g900 887 Nov 13 10:27 map_pdb1_resource.yaml +-rw-r--r-- 1 mmalvezz g900 887 Nov 13 10:27 map_pdb2_resource.yaml +-rw-r--r-- 1 mmalvezz g900 890 Nov 13 10:27 map_pdb3_resource.yaml +``` + +The command ```make secretes ``` will configure database secrets credential and certificates secretes + +```bash +make secrets +``` + + + +The makefile includes other different targets that can be used to test the various pdb operations available. E.g. + +```makefile +run03.2: + @$(call msg,"clone pdb2-->pdb4") + $(KUBECTL) apply -f $(PDBCLONE2) + $(KUBECTL) wait --for jsonpath='{.status.phase'}="Ready" pdb pdb4 -n $(PDBNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) + @$(call msg,"clone pdb2-->pdb4 completed") + $(KUBECTL) get pdb pdb3 -n $(PDBNAMESPACE) +``` +The target ```run03.2``` clones pdb2 into pdb4 and wait for ```$TEST_EXEC_TIMEOUT``` for the operation to complete. + +### Output executions:. + +```make secrets``` + +![image](../images/makesecrets_1_1.png) + + + +```make runall``` executes different pdb operations including the cdb controller creation + +![image](../images/makerunall.png) \ No newline at end of file diff --git a/docs/multitenant/ords-based/usecase/cdbnamespace_binding.yaml b/docs/multitenant/ords-based/usecase/cdbnamespace_binding.yaml new file mode 100644 index 00000000..5fd355f4 --- /dev/null +++ b/docs/multitenant/ords-based/usecase/cdbnamespace_binding.yaml @@ -0,0 +1,13 @@ +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: oracle-database-operator-oracle-database-operator-manager-rolebinding2 + namespace: cdbnamespace +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: oracle-database-operator-manager-role +subjects: +- kind: ServiceAccount + name: default + namespace: oracle-database-operator-system diff --git a/docs/multitenant/ords-based/usecase/clone_pdb1_resource.yaml b/docs/multitenant/ords-based/usecase/clone_pdb1_resource.yaml new file mode 100644 index 00000000..5723f7c6 --- /dev/null +++ b/docs/multitenant/ords-based/usecase/clone_pdb1_resource.yaml @@ -0,0 +1,50 @@ +apiVersion: database.oracle.com/v4 +kind: PDB +metadata: + name: pdb3 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + cdbName: "DB12" + pdbName: "new_clone" + srcPdbName: "pdbdev" + fileNameConversions: "NONE" + totalSize: "UNLIMITED" + tempSize: "UNLIMITED" + assertivePdbDeletion: true + action: "Clone" + adminName: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminPwd: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/ords-based/usecase/clone_pdb2_resource.yaml b/docs/multitenant/ords-based/usecase/clone_pdb2_resource.yaml new file mode 100644 index 00000000..2b9fc70a --- /dev/null +++ b/docs/multitenant/ords-based/usecase/clone_pdb2_resource.yaml @@ -0,0 +1,50 @@ +apiVersion: database.oracle.com/v4 +kind: PDB +metadata: + name: pdb4 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + cdbName: "DB12" + pdbName: "new_clone2" + srcPdbName: "pdbprd" + fileNameConversions: "NONE" + totalSize: "UNLIMITED" + tempSize: "UNLIMITED" + assertivePdbDeletion: true + action: "Clone" + adminName: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminPwd: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/ords-based/usecase/close_pdb1_resource.yaml b/docs/multitenant/ords-based/usecase/close_pdb1_resource.yaml new file mode 100644 index 00000000..ae837ce0 --- /dev/null +++ b/docs/multitenant/ords-based/usecase/close_pdb1_resource.yaml @@ -0,0 +1,47 @@ +apiVersion: database.oracle.com/v4 +kind: PDB +metadata: + name: pdb1 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + cdbName: "DB12" + pdbName: "pdbdev" + pdbState: "CLOSE" + modifyOption: "IMMEDIATE" + action: "Modify" + adminName: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminPwd: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/ords-based/usecase/close_pdb2_resource.yaml b/docs/multitenant/ords-based/usecase/close_pdb2_resource.yaml new file mode 100644 index 00000000..1b5d1324 --- /dev/null +++ b/docs/multitenant/ords-based/usecase/close_pdb2_resource.yaml @@ -0,0 +1,47 @@ +apiVersion: database.oracle.com/v4 +kind: PDB +metadata: + name: pdb2 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + cdbName: "DB12" + pdbName: "pdbprd" + pdbState: "CLOSE" + modifyOption: "IMMEDIATE" + action: "Modify" + adminName: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminPwd: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/ords-based/usecase/close_pdb3_resource.yaml b/docs/multitenant/ords-based/usecase/close_pdb3_resource.yaml new file mode 100644 index 00000000..f4a32938 --- /dev/null +++ b/docs/multitenant/ords-based/usecase/close_pdb3_resource.yaml @@ -0,0 +1,47 @@ +apiVersion: database.oracle.com/v4 +kind: PDB +metadata: + name: pdb3 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + cdbName: "DB12" + pdbName: ""new_clone" + pdbState: "CLOSE" + modifyOption: "IMMEDIATE" + action: "Modify" + adminName: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminPwd: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/ords-based/usecase/create_ords_pod.yaml b/docs/multitenant/ords-based/usecase/create_ords_pod.yaml new file mode 100644 index 00000000..ad196c9d --- /dev/null +++ b/docs/multitenant/ords-based/usecase/create_ords_pod.yaml @@ -0,0 +1,48 @@ +apiVersion: database.oracle.com/v4 +kind: CDB +metadata: + name: cdb-dev + namespace: cdbnamespace +spec: + cdbName: "DB12" + ordsImage: _your_container_registry/ords-dboper:latest + ordsImagePullPolicy: "Always" + dbTnsurl : "T H I S I S J U S T A N E X A M P L E (DESCRIPTION=(CONNECT_TIMEOUT=90)(RETRY_COUNT=30)(RETRY_DELAY=10)(TRANSPORT_CONNECT_TIMEOUT=70)(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan12.testrac.com)(PORT=1521)(IP=V4_ONLY))(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan34.testrac.com)(PORT=1521)(IP=V4_ONLY))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=TESTORDS)))" + replicas: 1 + deletePdbCascade: true + sysAdminPwd: + secret: + secretName: "syspwd" + key: "e_syspwd.txt" + ordsPwd: + secret: + secretName: "ordpwd" + key: "e_ordpwd.txt" + cdbAdminUser: + secret: + secretName: "cdbusr" + key: "e_cdbusr.txt" + cdbAdminPwd: + secret: + secretName: "cdbpwd" + key: "e_cdbpwd.txt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + cdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + cdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + cdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/ords-based/usecase/create_pdb1_resource.yaml b/docs/multitenant/ords-based/usecase/create_pdb1_resource.yaml new file mode 100644 index 00000000..84e910e0 --- /dev/null +++ b/docs/multitenant/ords-based/usecase/create_pdb1_resource.yaml @@ -0,0 +1,51 @@ +apiVersion: database.oracle.com/v4 +kind: PDB +metadata: + name: pdb1 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + cdbName: "DB12" + pdbName: "pdbdev" + assertivePdbDeletion: true + fileNameConversions: "NONE" + unlimitedStorage: false + tdeImport: false + totalSize: "2G" + tempSize: "800M" + action: "Create" + adminName: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminPwd: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/ords-based/usecase/create_pdb2_resource.yaml b/docs/multitenant/ords-based/usecase/create_pdb2_resource.yaml new file mode 100644 index 00000000..0a71c7c3 --- /dev/null +++ b/docs/multitenant/ords-based/usecase/create_pdb2_resource.yaml @@ -0,0 +1,51 @@ +apiVersion: database.oracle.com/v4 +kind: PDB +metadata: + name: pdb2 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + cdbName: "DB12" + pdbName: "pdbprd" + assertivePdbDeletion: true + fileNameConversions: "NONE" + unlimitedStorage: false + tdeImport: false + totalSize: "2G" + tempSize: "800M" + action: "Create" + adminName: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminPwd: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/ords-based/usecase/delete_pdb1_resource.yaml b/docs/multitenant/ords-based/usecase/delete_pdb1_resource.yaml new file mode 100644 index 00000000..3aba580c --- /dev/null +++ b/docs/multitenant/ords-based/usecase/delete_pdb1_resource.yaml @@ -0,0 +1,45 @@ +apiVersion: database.oracle.com/v4 +kind: PDB +metadata: + name: pdb1 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + pdbName: "pdbdev" + action: "Delete" + dropAction: "INCLUDING" + adminName: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminPwd: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/ords-based/usecase/delete_pdb2_resource.yaml b/docs/multitenant/ords-based/usecase/delete_pdb2_resource.yaml new file mode 100644 index 00000000..59b50a64 --- /dev/null +++ b/docs/multitenant/ords-based/usecase/delete_pdb2_resource.yaml @@ -0,0 +1,45 @@ +apiVersion: database.oracle.com/v4 +kind: PDB +metadata: + name: pdb2 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + pdbName: "pdbprd" + action: "Delete" + dropAction: "INCLUDING" + adminName: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminPwd: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/ords-based/usecase/makefile b/docs/multitenant/ords-based/usecase/makefile new file mode 100644 index 00000000..dc881598 --- /dev/null +++ b/docs/multitenant/ords-based/usecase/makefile @@ -0,0 +1,915 @@ +# Copyright (c) 2022, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# __ __ _ __ _ _ +# | \/ | __ _| | _____ / _(_) | ___ +# | |\/| |/ _` | |/ / _ \ |_| | |/ _ \ +# | | | | (_| | < __/ _| | | __/ +# |_| |_|\__,_|_|\_\___|_| |_|_|\___| +# | | | | ___| |_ __ ___ _ __ +# | |_| |/ _ \ | '_ \ / _ \ '__| +# | _ | __/ | |_) | __/ | +# |_| |_|\___|_| .__/ \___|_| +# |_| +# +# WARNING: Using this makefile helps you to customize yaml +# files. Edit parameters.txt with your enviroment +# informartion and execute the following steps +# +# 1) make operator +# it configures the operator yaml files with the +# watch namelist required by the multitenant controllers +# +# 2) make genyaml +# It automatically creates all the yaml files based on the +# information available in the parameters file +# +# 3) make secrets +# It configure the required secrets necessary to operate +# with pdbs multitenant controllers +# +# 4) make runall01 +# Start a series of operation create open close delete and so on +# +# LIST OF GENERAED YAML FILE +# +# ----------------------------- ---------------------------------- +# oracle-database-operator.yaml : oracle database operator +# cdbnamespace_binding.yaml : role binding for cdbnamespace +# pdbnamespace_binding.yaml : role binding for pdbnamespace +# create_ords_pod.yaml : create rest server pod +# create_pdb1_resource.yaml : create first pluggable database +# create_pdb2_resource.yaml : create second pluggable database +# open_pdb1_resource.yaml : open first pluggable database +# open_pdb2_resource.yaml : open second pluggable database +# close_pdb1_resource.yaml : close first pluggable database +# close_pdb2_resource.yaml : close second pluggable database +# clone_pdb_resource.yaml : clone thrid pluggable database +# clone_pdb2_resource.yaml : clone 4th pluggable database +# delete_pdb1_resource.yaml : delete first pluggable database +# delete_pdb2_resource.yaml : delete sencond pluggable database +# delete_pdb3_resource.yaml : delete thrid pluggable database +# unplug_pdb1_resource.yaml : unplug first pluggable database +# plug_pdb1_resource.yaml : plug first pluggable database +# map_pdb1_resource.yaml : map the first pluggable database +# config_map.yam : pdb parameters array +# +DATE := `date "+%y%m%d%H%M%S"` +###################### +# PARAMETER SECTIONS # +###################### + +export PARAMETERS=parameters.txt +export TNSALIAS=$(shell cat $(PARAMETERS) |grep -v ^\#|grep TNSALIAS|cut -d : -f 2) +export ORDPWD=$(shell cat $(PARAMETERS)|grep -v ^\#|grep ORDPWD|cut -d : -f 2) +export SYSPWD=$(shell cat $(PARAMETERS)|grep -v ^\#|grep SYSPWD|cut -d : -f 2) +export WBUSER=$(shell cat $(PARAMETERS)|grep -v ^\#|grep WBUSER|cut -d : -f 2) +export WBPASS=$(shell cat $(PARAMETERS)|grep -v ^\#|grep WBPASS|cut -d : -f 2) +export PDBUSR=$(shell cat $(PARAMETERS)|grep -v ^\#|grep PDBUSR|cut -d : -f 2) +export PDBPWD=$(shell cat $(PARAMETERS)|grep -v ^\#|grep PDBPWD|cut -d : -f 2) +export CDBUSR=$(shell cat $(PARAMETERS)|grep -v ^\#|grep CDBUSR|cut -d : -f 2) +export CDBPWD=$(shell cat $(PARAMETERS)|grep -v ^\#|grep CDBPWD|cut -d : -f 2) +export PDBNAMESPACE=$(shell cat $(PARAMETERS)|grep -v ^\#|grep PDBNAMESPACE|cut -d : -f 2) +export CDBNAMESPACE=$(shell cat $(PARAMETERS)|grep -v ^\#|grep CDBNAMESPACE|cut -d : -f 2) +export ORDSIMG=$(shell cat $(PARAMETERS)|grep -v ^\#|grep ORDSIMG|cut -d : -f 2,3) +export COMPANY=$(shell cat $(PARAMETERS)|grep -v ^\#|grep COMPANY|cut -d : -f 2) +export APIVERSION=$(shell cat $(PARAMETERS)|grep -v ^\#|grep APIVERSION|cut -d : -f 2) +export OPRNAMESPACE=oracle-database-operator-system +export ORACLE_OPERATOR_YAML=../../../../oracle-database-operator.yaml +export TEST_EXEC_TIMEOUT=3m +export IMAGE=oracle/ords-dboper:latest +export ORDSIMGDIR=../../../../ords + +REST_SERVER=ords +SKEY=tls.key +SCRT=tls.crt +CART=ca.crt +PRVKEY=ca.key +PUBKEY=public.pem +COMPANY=oracle +RUNTIME=/usr/bin/podman + +################# +### FILE LIST ### +################# + +export ORDS_POD=create_ords_pod.yaml + +export CDB_SECRETS=create_cdb_secrets.yaml +export PDB_SECRETS=create_pdb_secrets.yaml + +export PDBCRE1=create_pdb1_resource.yaml +export PDBCRE2=create_pdb2_resource.yaml + +export PDBCLOSE1=close_pdb1_resource.yaml +export PDBCLOSE2=close_pdb2_resource.yaml +export PDBCLOSE3=close_pdb3_resource.yaml + +export PDBOPEN1=open_pdb1_resource.yaml +export PDBOPEN2=open_pdb2_resource.yaml +export PDBOPEN3=open_pdb3_resource.yaml + +export PDBCLONE1=clone_pdb1_resource.yaml +export PDBCLONE2=clone_pdb2_resource.yaml + +export PDBDELETE1=delete_pdb1_resource.yaml +export PDBDELETE2=delete_pdb2_resource.yaml +export PDBDELETE3=delete_pdb3_resource.yaml + +export PDBUNPLUG1=unplug_pdb1_resource.yaml +export PDBPLUG1=plug_pdb1_resource.yaml + +export PDBMAP1=map_pdb1_resource.yaml +export PDBMAP2=map_pdb2_resource.yaml +export PDBMAP3=map_pdb3_resource.yaml + +export PDBMAP1=map_pdb1_resource.yaml +export PDBMAP2=map_pdb2_resource.yaml +export PDBMAP3=map_pdb3_resource.yaml + + +##BINARIES +export KUBECTL=/usr/bin/kubectl +OPENSSL=/usr/bin/openssl +ECHO=/usr/bin/echo +RM=/usr/bin/rm +CP=/usr/bin/cp +TAR=/usr/bin/tar +MKDIR=/usr/bin/mkdir +SED=/usr/bin/sed + +define msg +@printf "\033[31;7m%s\033[0m\r" "......................................]" +@printf "\033[31;7m[\xF0\x9F\x91\x89 %s\033[0m\n" $(1) +endef + +check: + $(call msg,"CHECK PARAMETERS") + @printf "TNSALIAS...............:%.60s....\n" $(TNSALIAS) + @printf "ORDPWD.................:%s\n" $(ORDPWD) + @printf "SYSPWD.................:%s\n" $(SYSPWD) + @printf "WBUSER.................:%s\n" $(WBUSER) + @printf "WBPASS.................:%s\n" $(WBPASS) + @printf "PDBUSR.................:%s\n" $(PDBUSR) + @printf "PDBPWD.................:%s\n" $(PDBPWD) + @printf "CDBUSR.................:%s\n" $(CDBUSR) + @printf "CDBPWD.................:%s\n" $(CDBPWD) + @printf "PDBNAMESPACE...........:%s\n" $(PDBNAMESPACE) + @printf "CDBNAMESPACE...........:%s\n" $(CDBNAMESPACE) + @printf "COMPANY................:%s\n" $(COMPANY) + @printf "APIVERSION.............:%s\n" $(APIVERSION) + + +tlscrt: + $(call msg,"TLS GENERATION") + #$(OPENSSL) genrsa -out $(PRVKEY) 2048 + $(OPENSSL) genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -pkeyopt rsa_keygen_pubexp:65537 > $(PRVKEY) + $(OPENSSL) req -new -x509 -days 365 -key $(PRVKEY) \ + -subj "/C=CN/ST=GD/L=SZ/O=$(COMPANY), Inc./CN=$(COMPANY) Root CA" -out ca.crt + $(OPENSSL) req -newkey rsa:2048 -nodes -keyout $(SKEY) -subj \ + "/C=CN/ST=GD/L=SZ/O=$(COMPANY), Inc./CN=cdb-dev-$(REST_SERVER).$(CDBNAMESPACE)" -out server.csr + $(ECHO) "subjectAltName=DNS:cdb-dev-$(REST_SERVER).$(CDBNAMESPACE)" > extfile.txt + $(OPENSSL) x509 -req -extfile extfile.txt -days 365 -in server.csr -CA ca.crt -CAkey $(PRVKEY) -CAcreateserial -out $(SCRT) + $(OPENSSL) rsa -in $(PRVKEY) -outform PEM -pubout -out $(PUBKEY) + +tlssec: + $(call msg,"GENERATE TLS SECRET") + $(KUBECTL) create secret tls db-tls --key="$(SKEY)" --cert="$(SCRT)" -n $(CDBNAMESPACE) + $(KUBECTL) create secret generic db-ca --from-file="$(CART)" -n $(CDBNAMESPACE) + $(KUBECTL) create secret tls db-tls --key="$(SKEY)" --cert="$(SCRT)" -n $(PDBNAMESPACE) + $(KUBECTL) create secret generic db-ca --from-file="$(CART)" -n $(PDBNAMESPACE) + + +delsec: + $(call msg,"CLEAN OLD SECRETS") + $(eval SECRETSP:=$(shell kubectl get secrets -n $(PDBNAMESPACE) -o custom-columns=":metadata.name" --no-headers) ) + $(eval SECRETSL:=$(shell kubectl get secrets -n $(CDBNAMESPACE) -o custom-columns=":metadata.name" --no-headers) ) + @[ "${SECRETSP}" ] && ( \ + printf "Deleteing secrets in namespace -n $(PDBNAMESPACE)\n") &&\ + ($(KUBECTL) delete secret $(SECRETSP) -n $(PDBNAMESPACE))\ + || ( echo "No screts in namespace $(PDBNAMESPACE)") + @[ "${SECRETSL}" ] && ( \ + printf "Deleteing secrets in namespace -n $(CDBNAMESPACE)\n") &&\ + ($(KUBECTL) delete secret $(SECRETSL) -n $(CDBNAMESPACE))\ + || ( echo "No screts in namespace $(PDBNAMESPACE)") + + +###### ENCRYPTED SECRETS ###### +export PRVKEY=ca.key +export PUBKEY=public.pem +WBUSERFILE=wbuser.txt +WBPASSFILE=wbpass.txt +CDBUSRFILE=cdbusr.txt +CDBPWDFILE=cdbpwd.txt +SYSPWDFILE=syspwd.txt +ORDPWDFILE=ordpwd.txt +PDBUSRFILE=pdbusr.txt +PDBPWDFILE=pdbpwd.txt + + + +secrets: delsec tlscrt tlssec + $(OPENSSL) rsa -in $(PRVKEY) -outform PEM -pubout -out $(PUBKEY) + $(KUBECTL) create secret generic pubkey --from-file=publicKey=$(PUBKEY) -n $(CDBNAMESPACE) + $(KUBECTL) create secret generic prvkey --from-file=privateKey=$(PRVKEY) -n $(CDBNAMESPACE) + $(KUBECTL) create secret generic prvkey --from-file=privateKey="$(PRVKEY)" -n $(PDBNAMESPACE) + @$(ECHO) $(WBUSER) > $(WBUSERFILE) + @$(ECHO) $(WBPASS) > $(WBPASSFILE) + @$(ECHO) $(CDBPWD) > $(CDBPWDFILE) + @$(ECHO) $(CDBUSR) > $(CDBUSRFILE) + @$(ECHO) $(SYSPWD) > $(SYSPWDFILE) + @$(ECHO) $(ORDPWD) > $(ORDPWDFILE) + @$(ECHO) $(PDBUSR) > $(PDBUSRFILE) + @$(ECHO) $(PDBPWD) > $(PDBPWDFILE) + $(OPENSSL) rsautl -encrypt -pubin -inkey $(PUBKEY) -in $(WBUSERFILE) |base64 > e_$(WBUSERFILE) + $(OPENSSL) rsautl -encrypt -pubin -inkey $(PUBKEY) -in $(WBPASSFILE) |base64 > e_$(WBPASSFILE) + $(OPENSSL) rsautl -encrypt -pubin -inkey $(PUBKEY) -in $(CDBPWDFILE) |base64 > e_$(CDBPWDFILE) + $(OPENSSL) rsautl -encrypt -pubin -inkey $(PUBKEY) -in $(CDBUSRFILE) |base64 > e_$(CDBUSRFILE) + $(OPENSSL) rsautl -encrypt -pubin -inkey $(PUBKEY) -in $(SYSPWDFILE) |base64 > e_$(SYSPWDFILE) + $(OPENSSL) rsautl -encrypt -pubin -inkey $(PUBKEY) -in $(ORDPWDFILE) |base64 > e_$(ORDPWDFILE) + $(OPENSSL) rsautl -encrypt -pubin -inkey $(PUBKEY) -in $(PDBUSRFILE) |base64 > e_$(PDBUSRFILE) + $(OPENSSL) rsautl -encrypt -pubin -inkey $(PUBKEY) -in $(PDBPWDFILE) |base64 > e_$(PDBPWDFILE) + $(KUBECTL) create secret generic wbuser --from-file=e_$(WBUSERFILE) -n $(CDBNAMESPACE) + $(KUBECTL) create secret generic wbpass --from-file=e_$(WBPASSFILE) -n $(CDBNAMESPACE) + $(KUBECTL) create secret generic wbuser --from-file=e_$(WBUSERFILE) -n $(PDBNAMESPACE) + $(KUBECTL) create secret generic wbpass --from-file=e_$(WBPASSFILE) -n $(PDBNAMESPACE) + $(KUBECTL) create secret generic cdbpwd --from-file=e_$(CDBPWDFILE) -n $(CDBNAMESPACE) + $(KUBECTL) create secret generic cdbusr --from-file=e_$(CDBUSRFILE) -n $(CDBNAMESPACE) + $(KUBECTL) create secret generic syspwd --from-file=e_$(SYSPWDFILE) -n $(CDBNAMESPACE) + $(KUBECTL) create secret generic ordpwd --from-file=e_$(ORDPWDFILE) -n $(CDBNAMESPACE) + $(KUBECTL) create secret generic pdbusr --from-file=e_$(PDBUSRFILE) -n $(PDBNAMESPACE) + $(KUBECTL) create secret generic pdbpwd --from-file=e_$(PDBPWDFILE) -n $(PDBNAMESPACE) + $(RM) $(WBUSERFILE) $(WBPASSFILE) $(CDBPWDFILE) $(CDBUSRFILE) $(SYSPWDFILE) $(ORDPWDFILE) $(PDBUSRFILE) $(PDBPWDFILE) + $(RM) e_$(WBUSERFILE) e_$(WBPASSFILE) e_$(CDBPWDFILE) e_$(CDBUSRFILE) e_$(SYSPWDFILE) e_$(ORDPWDFILE) e_$(PDBUSRFILE) e_$(PDBPWDFILE) + + +### YAML FILE SECTION ### +operator: + $(CP) ${ORACLE_OPERATOR_YAML} . + ${CP} `basename ${ORACLE_OPERATOR_YAML}` `basename ${ORACLE_OPERATOR_YAML}`.ORG + $(SED) -i 's/value: ""/value: $(OPRNAMESPACE),$(PDBNAMESPACE),$(CDBNAMESPACE)/g' `basename ${ORACLE_OPERATOR_YAML}` + + +define _script00 +cat < authsection01.yaml + sysAdminPwd: + secret: + secretName: "syspwd" + key: "e_syspwd.txt" + ordsPwd: + secret: + secretName: "ordpwd" + key: "e_ordpwd.txt" + cdbAdminUser: + secret: + secretName: "cdbusr" + key: "e_cdbusr.txt" + cdbAdminPwd: + secret: + secretName: "cdbpwd" + key: "e_cdbpwd.txt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + cdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + cdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + cdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" +EOF + +cat< authsection02.yaml + adminName: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminPwd: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" +EOF + + +cat < ${PDBNAMESPACE}_binding.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: oracle-database-operator-oracle-database-operator-manager-rolebinding1 + namespace: ${PDBNAMESPACE} +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: oracle-database-operator-manager-role +subjects: +- kind: ServiceAccount + name: default + namespace: oracle-database-operator-system +EOF + +cat < ${CDBNAMESPACE}_binding.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: oracle-database-operator-oracle-database-operator-manager-rolebinding2 + namespace: ${CDBNAMESPACE} +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: oracle-database-operator-manager-role +subjects: +- kind: ServiceAccount + name: default + namespace: oracle-database-operator-system +EOF + +endef +export script00 = $(value _script00) +secyaml: + @ eval "$$script00" + +#echo ords pod creation +define _script01 +cat < ${ORDS_POD} +apiVersion: database.oracle.com/${APIVERSION} +kind: CDB +metadata: + name: cdb-dev + namespace: ${CDBNAMESPACE} +spec: + cdbName: "DB12" + ordsImage: ${ORDSIMG} + ordsImagePullPolicy: "Always" + dbTnsurl : ${TNSALIAS} + replicas: 1 + deletePdbCascade: true +EOF + +cat authsection01.yaml >> ${ORDS_POD} + +endef +export script01 = $(value _script01) + + +define _script02 + +cat <${PDBCRE1} +apiVersion: database.oracle.com/${APIVERSION} +kind: PDB +metadata: + name: pdb1 + namespace: ${PDBNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${CDBNAMESPACE}" + cdbName: "DB12" + pdbName: "pdbdev" + assertivePdbDeletion: true + fileNameConversions: "NONE" + unlimitedStorage: false + tdeImport: false + totalSize: "2G" + tempSize: "800M" + action: "Create" +EOF + +cat < ${PDBCRE2} +apiVersion: database.oracle.com/${APIVERSION} +kind: PDB +metadata: + name: pdb2 + namespace: ${PDBNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${CDBNAMESPACE}" + cdbName: "DB12" + pdbName: "pdbprd" + assertivePdbDeletion: true + fileNameConversions: "NONE" + unlimitedStorage: false + tdeImport: false + totalSize: "2G" + tempSize: "800M" + action: "Create" +EOF + +cat <${PDBOPEN1} +apiVersion: database.oracle.com/${APIVERSION} +kind: PDB +metadata: + name: pdb1 + namespace: ${PDBNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${CDBNAMESPACE}" + cdbName: "DB12" + pdbName: "pdbdev" + action: "Modify" + pdbState: "OPEN" + modifyOption: "READ WRITE" +EOF + +cat <${PDBOPEN2} +apiVersion: database.oracle.com/${APIVERSION} +kind: PDB +metadata: + name: pdb2 + namespace: ${PDBNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${CDBNAMESPACE}" + cdbName: "DB12" + pdbName: "pdbprd" + action: "Modify" + pdbState: "OPEN" + modifyOption: "READ WRITE" +EOF + +cat <${PDBOPEN3} +apiVersion: database.oracle.com/${APIVERSION} +kind: PDB +metadata: + name: pdb3 + namespace: ${PDBNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${CDBNAMESPACE}" + cdbName: "DB12" + pdbName: "new_clone" + action: "Modify" + pdbState: "OPEN" + modifyOption: "READ WRITE" +EOF + +cat <${PDBCLOSE1} +apiVersion: database.oracle.com/${APIVERSION} +kind: PDB +metadata: + name: pdb1 + namespace: ${PDBNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${CDBNAMESPACE}" + cdbName: "DB12" + pdbName: "pdbdev" + pdbState: "CLOSE" + modifyOption: "IMMEDIATE" + action: "Modify" +EOF + +cat <${PDBCLOSE2} +apiVersion: database.oracle.com/${APIVERSION} +kind: PDB +metadata: + name: pdb2 + namespace: ${PDBNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${CDBNAMESPACE}" + cdbName: "DB12" + pdbName: "pdbprd" + pdbState: "CLOSE" + modifyOption: "IMMEDIATE" + action: "Modify" +EOF + +cat <${PDBCLOSE3} +apiVersion: database.oracle.com/${APIVERSION} +kind: PDB +metadata: + name: pdb3 + namespace: ${PDBNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${CDBNAMESPACE}" + cdbName: "DB12" + pdbName: ""new_clone" + pdbState: "CLOSE" + modifyOption: "IMMEDIATE" + action: "Modify" +EOF + +cat < ${PDBCLONE1} +apiVersion: database.oracle.com/${APIVERSION} +kind: PDB +metadata: + name: pdb3 + namespace: ${PDBNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${CDBNAMESPACE}" + cdbName: "DB12" + pdbName: "new_clone" + srcPdbName: "pdbdev" + fileNameConversions: "NONE" + totalSize: "UNLIMITED" + tempSize: "UNLIMITED" + assertivePdbDeletion: true + action: "Clone" +EOF + +cat < ${PDBCLONE2} +apiVersion: database.oracle.com/${APIVERSION} +kind: PDB +metadata: + name: pdb4 + namespace: ${PDBNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${CDBNAMESPACE}" + cdbName: "DB12" + pdbName: "new_clone2" + srcPdbName: "pdbprd" + fileNameConversions: "NONE" + totalSize: "UNLIMITED" + tempSize: "UNLIMITED" + assertivePdbDeletion: true + action: "Clone" +EOF + + +cat < ${PDBDELETE1} +apiVersion: database.oracle.com/${APIVERSION} +kind: PDB +metadata: + name: pdb1 + namespace: ${PDBNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${CDBNAMESPACE}" + pdbName: "pdbdev" + action: "Delete" + dropAction: "INCLUDING" +EOF + +cat < ${PDBDELETE2} +apiVersion: database.oracle.com/${APIVERSION} +kind: PDB +metadata: + name: pdb2 + namespace: ${PDBNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${CDBNAMESPACE}" + pdbName: "pdbprd" + action: "Delete" + dropAction: "INCLUDING" +EOF + +cat < ${PDBUNPLUG1} +apiVersion: database.oracle.com/${APIVERSION} +kind: PDB +metadata: + name: pdb1 + namespace: ${PDBNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${CDBNAMESPACE}" + cdbName: "DB12" + pdbName: "pdbdev" + xmlFileName: "/tmp/pdb.xml" + action: "Unplug" +EOF + +cat <${PDBPLUG1} +apiVersion: database.oracle.com/${APIVERSION} +kind: PDB +metadata: + name: pdb1 + namespace: ${PDBNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${CDBNAMESPACE}" + cdbName: "DB12" + pdbName: "pdbdev" + xmlFileName: "/tmp/pdb.xml" + action: "plug" + fileNameConversions: "NONE" + sourceFileNameConversions: "NONE" + copyAction: "MOVE" + totalSize: "1G" + tempSize: "100M" + assertivePdbDeletion: true + action: "Plug" +EOF + +cat <${PDBMAP1} +apiVersion: database.oracle.com/${APIVERSION} +kind: PDB +metadata: + name: pdb1 + namespace: ${PDBNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${CDBNAMESPACE}" + cdbName: "DB12" + pdbName: "pdbdev" + assertivePdbDeletion: true + fileNameConversions: "NONE" + totalSize: "1G" + tempSize: "100M" + action: "Map" +EOF + +cat <${PDBMAP2} +apiVersion: database.oracle.com/${APIVERSION} +kind: PDB +metadata: + name: pdb2 + namespace: ${PDBNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${CDBNAMESPACE}" + cdbName: "DB12" + pdbName: "pdbprd" + assertivePdbDeletion: true + fileNameConversions: "NONE" + totalSize: "1G" + tempSize: "100M" + action: "Map" +EOF + + +cat <${PDBMAP3} +apiVersion: database.oracle.com/${APIVERSION} +kind: PDB +metadata: + name: pdb3 + namespace: ${PDBNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${CDBNAMESPACE}" + cdbName: "DB12" + pdbName: "new_clone" + assertivePdbDeletion: true + fileNameConversions: "NONE" + totalSize: "1G" + tempSize: "100M" + action: "Map" +EOF + + +## Auth information +for _file in ${PDBCRE1} ${PDBCRE2} ${PDBOPEN1} ${PDBOPEN2} ${PDBOPEN3} ${PDBCLOSE1} ${PDBCLOSE2} ${PDBCLOSE3} ${PDBCLONE1} ${PDBCLONE2} ${PDBDELETE1} ${PDBDELETE2} ${PDBUNPLUG1} ${PDBPLUG1} ${PDBMAP1} ${PDBMAP2} ${PDBMAP3} +do +ls -ltr ${_file} + cat authsection02.yaml >> ${_file} +done +rm authsection02.yaml +rm authsection01.yaml +endef + +export script02 = $(value _script02) + +genyaml: secyaml + @ eval "$$script01" + @ eval "$$script02" + +cleanyaml: + - $(RM) $(PDBMAP3) $(PDBMAP2) $(PDBMAP1) $(PDBPLUG1) $(PDBUNPLUG1) $(PDBDELETE2) $(PDBDELETE1) $(PDBCLONE2) $(PDBCLONE1) $(PDBCLOSE3) $(PDBCLOSE2) $(PDBCLOSE1) $(PDBOPEN3) $(PDBOPEN2) $(PDBOPEN1) $(PDBCRE2) $(PDBCRE1) $(ORDS_POD) $(CDB_SECRETS) $(PDB_SECRETS) + - $(RM) ${PDBNAMESPACE}_binding.yaml ${CDBNAMESPACE}_binding.yaml + + +cleancrt: + - $(RM) $(SKEY) $(SCRT) $(CART) $(PRVKEY) $(PUBKEY) server.csr extfile.txt ca.srl + + +################# +### PACKAGING ### +################# + +pkg: + - $(RM) -rf /tmp/pkgtestplan + $(MKDIR) /tmp/pkgtestplan + $(CP) -R * /tmp/pkgtestplan + $(CP) ../../../../oracle-database-operator.yaml /tmp/pkgtestplan/ + $(TAR) -C /tmp -cvf ~/pkgtestplan_$(DATE).tar pkgtestplan + +################ +### diag ### +################ + +login: + $(KUBECTL) exec `$(KUBECTL) get pods -n $(CDBNAMESPACE)|grep ords|cut -d ' ' -f 1` -n $(CDBNAMESPACE) -it -- /bin/bash + + +reloadop: + echo "RESTARTING OPERATOR" + $(eval OP1 := $(shell $(KUBECTL) get pods -n $(OPRNAMESPACE)|grep oracle-database-operator-controller|head -1|cut -d ' ' -f 1 )) + $(eval OP2 := $(shell $(KUBECTL) get pods -n $(OPRNAMESPACE)|grep oracle-database-operator-controller|head -2|tail -1|cut -d ' ' -f 1 )) + $(eval OP3 := $(shell $(KUBECTL) get pods -n $(OPRNAMESPACE)|grep oracle-database-operator-controller|tail -1|cut -d ' ' -f 1 )) + $(KUBECTL) get pod $(OP1) -n $(OPRNAMESPACE) -o yaml | kubectl replace --force -f - + $(KUBECTL) get pod $(OP2) -n $(OPRNAMESPACE) -o yaml | kubectl replace --force -f - + $(KUBECTL) get pod $(OP3) -n $(OPRNAMESPACE) -o yaml | kubectl replace --force -f - + + +dump: + @$(eval TMPSP := $(shell date "+%y%m%d%H%M%S" )) + @$(eval DIAGFILE := ./opdmp.$(TMPSP)) + @>$(DIAGFILE) + @echo "OPERATOR DUMP" >> $(DIAGFILE) + @echo "~~~~~~~~~~~~~" >> $(DIAGFILE) + $(KUBECTL) logs pod/`$(KUBECTL) get pods -n $(OPRNAMESPACE)|grep oracle-database-operator-controller|head -1|cut -d ' ' -f 1` -n $(OPRNAMESPACE) >>$(DIAGFILE) + $(KUBECTL) logs pod/`$(KUBECTL) get pods -n $(OPRNAMESPACE)|grep oracle-database-operator-controller|head -2|tail -1 | cut -d ' ' -f 1` -n $(OPRNAMESPACE) >>$(DIAGFILE) + $(KUBECTL) logs pod/`$(KUBECTL) get pods -n $(OPRNAMESPACE)|grep oracle-database-operator-controller|tail -1|cut -d ' ' -f 1` -n $(OPRNAMESPACE) >>$(DIAGFILE) + +####################################################### +#### TEST SECTION #### +####################################################### + +run00: + @$(call msg,"cdb pod creation") + - $(KUBECTL) delete cdb cdb-dev -n $(CDBNAMESPACE) + $(KUBECTL) apply -f $(ORDS_POD) + time $(KUBECTL) wait --for jsonpath='{.status.phase'}="Ready" cdb cdb-dev -n $(CDBNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) + @$(call msg,"cdb pod completed") + $(KUBECTL) get cdb -n $(CDBNAMESPACE) + $(KUBECTL) get pod -n $(CDBNAMESPACE) + +run01.1: + @$(call msg,"pdb pdb1 creation") + $(KUBECTL) apply -f $(PDBCRE1) + time $(KUBECTL) wait --for jsonpath='{.status.phase'}="Ready" pdb pdb1 -n $(PDBNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) + @$(call msg, "pdb pdb1 creation completed") + $(KUBECTL) get pdb pdb1 -n $(PDBNAMESPACE) + +run01.2: + @$(call msg, "pdb pdb2 creation") + $(KUBECTL) apply -f $(PDBCRE2) + $(KUBECTL) wait --for jsonpath='{.status.phase'}="Ready" pdb pdb2 -n $(PDBNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) + @$(call msg, "pdb pdb2 creation completed") + $(KUBECTL) get pdb pdb2 -n $(PDBNAMESPACE) + +run02.1: + @$(call msg, "pdb pdb1 open") + $(KUBECTL) apply -f $(PDBOPEN1) + $(KUBECTL) wait --for jsonpath='{.status.openMode'}="READ WRITE" pdb pdb1 -n $(PDBNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) + @$(call msg, "pdb pdb1 open completed") + $(KUBECTL) get pdb pdb1 -n $(PDBNAMESPACE) + +run02.2: + @$(call msg,"pdb pdb2 open") + $(KUBECTL) apply -f $(PDBOPEN2) + $(KUBECTL) wait --for jsonpath='{.status.openMode'}="READ WRITE" pdb pdb2 -n $(PDBNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) + @$(call msg,"pdb pdb2 open completed") + $(KUBECTL) get pdb pdb2 -n $(PDBNAMESPACE) + + +run03.1: + @$(call msg,"clone pdb1-->pdb3") + $(KUBECTL) apply -f $(PDBCLONE1) + $(KUBECTL) wait --for jsonpath='{.status.phase'}="Ready" pdb pdb3 -n $(PDBNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) + @$(call msg,"clone pdb1-->pdb3 completed") + $(KUBECTL) get pdb pdb3 -n $(PDBNAMESPACE) + + +run03.2: + @$(call msg,"clone pdb2-->pdb4") + $(KUBECTL) apply -f $(PDBCLONE2) + $(KUBECTL) wait --for jsonpath='{.status.phase'}="Ready" pdb pdb4 -n $(PDBNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) + @$(call msg,"clone pdb2-->pdb4 completed") + $(KUBECTL) get pdb pdb3 -n $(PDBNAMESPACE) + + +run04.1: + @$(call msg,"pdb pdb1 close") + $(KUBECTL) apply -f $(PDBCLOSE1) + $(KUBECTL) wait --for jsonpath='{.status.openMode'}="MOUNTED" pdb pdb1 -n $(PDBNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) + @$(call msg, "pdb pdb1 close completed") + $(KUBECTL) get pdb pdb1 -n $(PDBNAMESPACE) + +run04.2: + @$(call msg,"pdb pdb2 close") + $(KUBECTL) apply -f $(PDBCLOSE2) + $(KUBECTL) wait --for jsonpath='{.status.openMode'}="MOUNTED" pdb pdb2 -n $(PDBNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) + @$(call msg,"pdb pdb2 close completed") + $(KUBECTL) get pdb pdb2 -n $(PDBNAMESPACE) + +run05.1: + @$(call msg,"pdb pdb1 unplug") + $(KUBECTL) apply -f $(PDBUNPLUG1) + $(KUBECTL) wait --for=delete pdb pdb1 -n $(PDBNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) + @$(call msg,"pdb pdb1 unplug completed") + +run06.1: + @$(call msg, "pdb pdb1 plug") + $(KUBECTL) apply -f $(PDBPLUG1) + $(KUBECTL) wait --for jsonpath='{.status.phase'}="Ready" pdb pdb1 -n $(PDBNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) + @$(call msg, "pdb pdb1 plug completed") + $(KUBECTL) get pdb pdb1 -n $(PDBNAMESPACE) + +run07.1: + @$(call msg,"pdb pdb1 delete ") + - $(KUBECTL) apply -f $(PDBCLOSE1) + $(KUBECTL) wait --for jsonpath='{.status.openMode'}="MOUNTED" pdb pdb1 -n $(PDBNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) + $(KUBECTL) apply -f $(PDBDELETE1) + $(KUBECTL) wait --for=delete pdb pdb1 -n $(PDBNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) + @$(call msg,"pdb pdb1 delete") + $(KUBECTL) get pdb -n $(PDBNAMESPACE) + +run99.1: + $(KUBECTL) delete cdb cdb-dev -n cdbnamespace + $(KUBECTL) wait --for=delete cdb cdb-dev -n $(CDBNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) + $(KUBECTL) get cdb -n cdbnamespaace + $(KUBECTL) get pdb -n pdbnamespaace + + +## SEQ | ACTION +## ----+---------------- +## 00 | create ords pod +## 01 | create pdb +## 02 | open pdb +## 03 | clone pdb +## 04 | close pdb +## 05 | unpug pdb +## 06 | plug pdb +## 07 | delete pdb (declarative) + + +runall01: run00 run01.1 run01.2 run03.1 run03.2 run04.1 run05.1 run06.1 run02.1 run07.1 + + +###### BUILD ORDS IMAGE ###### + +createimage: + $(RUNTIME) build -t $(IMAGE) $(ORDSIMGDIR) + +createimageproxy: + $(RUNTIME) build -t $(IMAGE) $(ORDSIMGDIR) --build-arg https_proxy=$(HTTPS_PROXY) --build-arg http_proxy=$(HTTP_PROXY) + +tagimage: + @echo "TAG IMAGE" + $(RUNTIME) tag $(IMAGE) $(ORDSIMG) + +push: + $(RUNTIME) push $(ORDSIMG) + + diff --git a/docs/multitenant/ords-based/usecase/map_pdb1_resource.yaml b/docs/multitenant/ords-based/usecase/map_pdb1_resource.yaml new file mode 100644 index 00000000..b71b59d5 --- /dev/null +++ b/docs/multitenant/ords-based/usecase/map_pdb1_resource.yaml @@ -0,0 +1,49 @@ +apiVersion: database.oracle.com/v4 +kind: PDB +metadata: + name: pdb1 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + cdbName: "DB12" + pdbName: "pdbdev" + assertivePdbDeletion: true + fileNameConversions: "NONE" + totalSize: "1G" + tempSize: "100M" + action: "Map" + adminName: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminPwd: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/ords-based/usecase/map_pdb2_resource.yaml b/docs/multitenant/ords-based/usecase/map_pdb2_resource.yaml new file mode 100644 index 00000000..75d056d0 --- /dev/null +++ b/docs/multitenant/ords-based/usecase/map_pdb2_resource.yaml @@ -0,0 +1,49 @@ +apiVersion: database.oracle.com/v4 +kind: PDB +metadata: + name: pdb2 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + cdbName: "DB12" + pdbName: "pdbprd" + assertivePdbDeletion: true + fileNameConversions: "NONE" + totalSize: "1G" + tempSize: "100M" + action: "Map" + adminName: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminPwd: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/ords-based/usecase/map_pdb3_resource.yaml b/docs/multitenant/ords-based/usecase/map_pdb3_resource.yaml new file mode 100644 index 00000000..3523aa68 --- /dev/null +++ b/docs/multitenant/ords-based/usecase/map_pdb3_resource.yaml @@ -0,0 +1,49 @@ +apiVersion: database.oracle.com/v4 +kind: PDB +metadata: + name: pdb3 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + cdbName: "DB12" + pdbName: "new_clone" + assertivePdbDeletion: true + fileNameConversions: "NONE" + totalSize: "1G" + tempSize: "100M" + action: "Map" + adminName: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminPwd: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/ords-based/usecase/open_pdb1_resource.yaml b/docs/multitenant/ords-based/usecase/open_pdb1_resource.yaml new file mode 100644 index 00000000..93a1d43a --- /dev/null +++ b/docs/multitenant/ords-based/usecase/open_pdb1_resource.yaml @@ -0,0 +1,47 @@ +apiVersion: database.oracle.com/v4 +kind: PDB +metadata: + name: pdb1 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + cdbName: "DB12" + pdbName: "pdbdev" + action: "Modify" + pdbState: "OPEN" + modifyOption: "READ WRITE" + adminName: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminPwd: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/ords-based/usecase/open_pdb2_resource.yaml b/docs/multitenant/ords-based/usecase/open_pdb2_resource.yaml new file mode 100644 index 00000000..deb27f9a --- /dev/null +++ b/docs/multitenant/ords-based/usecase/open_pdb2_resource.yaml @@ -0,0 +1,47 @@ +apiVersion: database.oracle.com/v4 +kind: PDB +metadata: + name: pdb2 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + cdbName: "DB12" + pdbName: "pdbprd" + action: "Modify" + pdbState: "OPEN" + modifyOption: "READ WRITE" + adminName: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminPwd: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/ords-based/usecase/open_pdb3_resource.yaml b/docs/multitenant/ords-based/usecase/open_pdb3_resource.yaml new file mode 100644 index 00000000..586f2f57 --- /dev/null +++ b/docs/multitenant/ords-based/usecase/open_pdb3_resource.yaml @@ -0,0 +1,47 @@ +apiVersion: database.oracle.com/v4 +kind: PDB +metadata: + name: pdb3 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + cdbName: "DB12" + pdbName: "new_clone" + action: "Modify" + pdbState: "OPEN" + modifyOption: "READ WRITE" + adminName: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminPwd: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/ords-based/usecase/parameters.txt b/docs/multitenant/ords-based/usecase/parameters.txt new file mode 100644 index 00000000..64dc3759 --- /dev/null +++ b/docs/multitenant/ords-based/usecase/parameters.txt @@ -0,0 +1,61 @@ + +######################## +## REST SERVER IMAGE ### +######################## + +ORDSIMG:_your_container_registry/ords-dboper:latest + +############################## +## TNS URL FOR CDB CREATION ## +############################## +TNSALIAS:"T H I S I S J U S T A N E X A M P L E (DESCRIPTION=(CONNECT_TIMEOUT=90)(RETRY_COUNT=30)(RETRY_DELAY=10)(TRANSPORT_CONNECT_TIMEOUT=70)(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan12.testrac.com)(PORT=1521)(IP=V4_ONLY))(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan34.testrac.com)(PORT=1521)(IP=V4_ONLY))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=TESTORDS)))" + +########################################### +## ORDS PUBLIC USER ## +########################################### +ORDPWD:change_me_please + +########################################### +## SYSPASSWORD ## +########################################### +SYSPWD:change_me_please + +####################### +## HTTPS CREDENTIAL ### +####################### + +WBUSER:change_me_please +WBPASS:change_me_please + +##################### +## PDB ADMIN USER ### +##################### + +PDBUSR:change_me_please +PDBPWD:change_me_please + +##################### +## CDB ADMIN USER ### +##################### + +CDBUSR:C##DBAPI_CDB_ADMIN +CDBPWD:change_me_please + +################### +### NAMESPACES #### +################### + +PDBNAMESPACE:pdbnamespace +CDBNAMESPACE:cdbnamespace + +#################### +### COMPANY NAME ### +#################### + +COMPANY:oracle + +#################### +### APIVERSION ### +#################### + +APIVERSION:v4 diff --git a/docs/multitenant/ords-based/usecase/pdbnamespace_binding.yaml b/docs/multitenant/ords-based/usecase/pdbnamespace_binding.yaml new file mode 100644 index 00000000..5af79ed6 --- /dev/null +++ b/docs/multitenant/ords-based/usecase/pdbnamespace_binding.yaml @@ -0,0 +1,13 @@ +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: oracle-database-operator-oracle-database-operator-manager-rolebinding1 + namespace: pdbnamespace +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: oracle-database-operator-manager-role +subjects: +- kind: ServiceAccount + name: default + namespace: oracle-database-operator-system diff --git a/docs/multitenant/ords-based/usecase/plug_pdb1_resource.yaml b/docs/multitenant/ords-based/usecase/plug_pdb1_resource.yaml new file mode 100644 index 00000000..9eb5ed77 --- /dev/null +++ b/docs/multitenant/ords-based/usecase/plug_pdb1_resource.yaml @@ -0,0 +1,53 @@ +apiVersion: database.oracle.com/v4 +kind: PDB +metadata: + name: pdb1 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + cdbName: "DB12" + pdbName: "pdbdev" + xmlFileName: "/tmp/pdb.xml" + action: "plug" + fileNameConversions: "NONE" + sourceFileNameConversions: "NONE" + copyAction: "MOVE" + totalSize: "1G" + tempSize: "100M" + assertivePdbDeletion: true + action: "Plug" + adminName: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminPwd: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/ords-based/usecase/unplug_pdb1_resource.yaml b/docs/multitenant/ords-based/usecase/unplug_pdb1_resource.yaml new file mode 100644 index 00000000..0036d5f7 --- /dev/null +++ b/docs/multitenant/ords-based/usecase/unplug_pdb1_resource.yaml @@ -0,0 +1,46 @@ +apiVersion: database.oracle.com/v4 +kind: PDB +metadata: + name: pdb1 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + cdbName: "DB12" + pdbName: "pdbdev" + xmlFileName: "/tmp/pdb.xml" + action: "Unplug" + adminName: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminPwd: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/usecase01/README.md b/docs/multitenant/ords-based/usecase01/README.md similarity index 85% rename from docs/multitenant/usecase01/README.md rename to docs/multitenant/ords-based/usecase01/README.md index 7352257e..0020541c 100644 --- a/docs/multitenant/usecase01/README.md +++ b/docs/multitenant/ords-based/usecase01/README.md @@ -50,7 +50,9 @@ The following table reports the parameters required to configure and use oracle | pdbTlsKey | | [standalone.https.cert.key][key] | | pdbTlsCrt | | [standalone.https.cert][cr] | | pdbTlsCat | | certificate authority | -| assertivePdbDeletion | boolean | [turn on imperative approach on crd deleteion][imperative] | +| cdbOrdsPrvKey | | private key (cdb crd) | +| pdbOrdsPrvKey | | private key (pdb crd) | +| assertivePdbDeletion | boolean | [turn on imperative approach on crd deleteion][imperative] | > A [makfile](./makefile) is available to sped up the command execution for the multitenant setup and test. See the comments in the header of file @@ -161,63 +163,6 @@ GRANT SYSDBA TO CONTAINER = ALL; GRANT CREATE SESSION TO CONTAINER = ALL; ``` ---- - -#### Create CDB secret - -+ Create secret for CDB connection - -```bash -kubectl apply -f cdb_secret.yaml -n oracle-database-operator-system - -``` -Exmaple: **cdb_secret.yaml** - -```yaml -apiVersion: v1 -kind: Secret -metadata: - name: cdb1-secret - namespace: oracle-database-operator-system -type: Opaque -data: - ords_pwd: "encoded value" - sysadmin_pwd: "encoded value" - cdbadmin_user: "encoded value" - cdbadmin_pwd: "encoded value" - webserver_user: "encoded value" - webserver_pwd: "encoded value" - -``` -Use **base64** command to encode/decode username and password in the secret file as shown in the following example - -- encode -```bash -echo "ThisIsMyPassword" |base64 -i -VGhpc0lzTXlQYXNzd29yZAo= -``` -- decode -```bash - echo "VGhpc0lzTXlQYXNzd29yZAo=" | base64 --decode -ThisIsMyPassword - -``` - - ->Note that we do not have to create webuser on the database. - -+ Check secret: - -```bash -kubectl get secret -n oracle-database-operator-system -NAME TYPE DATA AGE -cdb1-secret Opaque 6 7s <--- -container-registry-secret kubernetes.io/dockerconfigjson 1 2m17s -webhook-server-cert kubernetes.io/tls 3 4m55s -``` - ->**TIPS:** Use the following commands to analyze contents of an existing secret ```bash kubectl get secret -o yaml -n ``` ----- - #### Create Certificates + Create certificates: At this stage we need to create certificates on our local machine and upload into kubernetes cluster by creating new secrets. @@ -257,7 +202,7 @@ webhook-server-cert kubernetes.io/tls 3 4m55s ```bash -genrsa -out 2048 +openssl genrsa -out 2048 openssl req -new -x509 -days 365 -key -subj "/C=CN/ST=GD/L=SZ/O=oracle, Inc./CN=oracle Root CA" -out openssl req -newkey rsa:2048 -nodes -keyout -subj "/C=CN/ST=GD/L=SZ/O=oracle, Inc./CN=-ords" -out server.csr /usr/bin/echo "subjectAltName=DNS:-ords,DNS:www.example.com" > extfile.txt @@ -270,6 +215,9 @@ kubectl create secret generic db-ca --from-file= -n oracle-database-op [Example of execution:](./logfiles/openssl_execution.log) +#### CDB and PDB credential + +Refer to the [landing page](../README.md) to implement openssl encrpted secrets. ---- @@ -283,9 +231,9 @@ kubectl create secret generic db-ca --from-file= -n oracle-database-op + Create ords container ```bash -/usr/bin/kubectl apply -f cdb_create.yaml -n oracle-database-operator-system +/usr/bin/kubectl apply -f create_ords_pod.yaml -n oracle-database-operator-system ``` -Example: **cdb_create.yaml** +Example: **create_ords_pod.yaml** ```yaml apiVersion: database.oracle.com/v1alpha1 @@ -299,30 +247,30 @@ spec: ordsImagePullPolicy: "Always" dbTnsurl : "...Container tns alias....." replicas: 1 - sysAdminPwd: - secret: - secretName: "cdb1-secret" - key: "sysadmin_pwd" + sysAdminPwd: + secret: + secretName: "syspwd" + key: "e_syspwd.txt" ordsPwd: - secret: - secretName: "cdb1-secret" - key: "ords_pwd" - cdbAdminUser: - secret: - secretName: "cdb1-secret" - key: "cdbadmin_user" - cdbAdminPwd: - secret: - secretName: "cdb1-secret" - key: "cdbadmin_pwd" - webServerUser: - secret: - secretName: "cdb1-secret" - key: "webserver_user" - webServerPwd: - secret: - secretName: "cdb1-secret" - key: "webserver_pwd" + secret: + secretName: "ordpwd" + key: "e_ordpwd.txt" + cdbAdminUser: + secret: + secretName: "cdbusr" + key: "e_cdbusr.txt" + cdbAdminPwd: + secret: + secretName: "cdbpwd" + key: "e_cdbpwd.txt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" cdbTlsKey: secret: secretName: "db-tls" @@ -331,6 +279,11 @@ spec: secret: secretName: "db-tls" key: "tls.crt" + cdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" + ``` > **Note** if you are working in dataguard environment with multiple sites (AC/DR) specifying the host name (dbServer/dbPort/serviceName) may not be the suitable solution for this kind of configuration, use **dbTnsurl** instead. Specify the whole tns string which includes the hosts/scan list. @@ -351,7 +304,7 @@ spec: dbtnsurl:((DESCRIPTION=(CONNECT_TIMEOUT=90)(RETRY_COUNT=30)(RETRY_DELAY=10)(TRANSPORT_CONNECT_TIMEOUT=70)(TRANS...... ``` -[Example of cdb.yaml](./cdb_create.yaml) +[create_ords_pod.yaml example](./create_ords_pod.yaml) ---- @@ -417,51 +370,16 @@ NAME CDB NAME DB SERVER DB PORT REPLICAS STATUS MESSAG [Example of executions](./logfiles/ordsconfig.log) ----- - -#### Create PDB secret - - -```bash -/usr/bin/kubectl apply -f pdb.yaml -n oracle-database-operator-system -``` -Exmaple: **pdb_secret.yaml** - -```yaml -apiVersion: v1 -kind: Secret -metadata: - name: pdb1-secret - namespace: oracle-database-operator-system -type: Opaque -data: - sysadmin_user: "encoded value" - sysadmin_pwd: "encoded value" -``` - -+ Check secret creation - -```bash -kubectl get secret -n oracle-database-operator-system -NAME TYPE DATA AGE -cdb1-secret Opaque 6 79m -container-registry-secret kubernetes.io/dockerconfigjson 1 79m -db-ca Opaque 1 78m -db-tls kubernetes.io/tls 2 78m -pdb1-secret Opaque 2 79m <--- -webhook-server-cert kubernetes.io/tls 3 79m -``` ---- - #### Apply pdb yaml file to create pdb ```bash -/usr/bin/kubectl apply -f pdb.yaml -n oracle-database-operator-system +/usr/bin/kubectl apply -f create_pdb1_resource.yaml -n oracle-database-operator-system ``` -Example: **pdb_create.yaml** +Example: **create_pdb1_resource.yaml** ```yaml -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: PDB metadata: name: pdb1 @@ -470,17 +388,24 @@ metadata: cdb: cdb-dev spec: cdbResName: "cdb-dev" - cdbNamespace: "oracle-database-operator-system" + cdbNamespace: "cdbnamespace" cdbName: "DB12" pdbName: "pdbdev" + assertivePdbDeletion: true + fileNameConversions: "NONE" + unlimitedStorage: false + tdeImport: false + totalSize: "2G" + tempSize: "800M" + action: "Create" adminName: secret: - secretName: "pdb1-secret" - key: "sysadmin_user" + secretName: "pdbusr" + key: "e_pdbusr.txt" adminPwd: secret: - secretName: "pdb1-secret" - key: "sysadmin_pwd" + secretName: "pdbpwd" + key: "e_pdbpwd.txt" pdbTlsKey: secret: secretName: "db-tls" @@ -495,18 +420,16 @@ spec: key: "ca.crt" webServerUser: secret: - secretName: "pdb1-secret" - key: "webserver_user" + secretName: "wbuser" + key: "e_wbuser.txt" webServerPwd: secret: - secretName: "pdb1-secret" - key: "webserver_pwd" - fileNameConversions: "NONE" - tdeImport: false - totalSize: "1G" - tempSize: "100M" - action: "Create" - assertivePdbDeletion: true + secretName: "wbpass" + key: "e_wbpass.txt" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" ``` + Monitor the pdb creation status until message is success diff --git a/docs/multitenant/usecase01/ca.crt b/docs/multitenant/ords-based/usecase01/ca.crt similarity index 100% rename from docs/multitenant/usecase01/ca.crt rename to docs/multitenant/ords-based/usecase01/ca.crt diff --git a/docs/multitenant/usecase01/ca.key b/docs/multitenant/ords-based/usecase01/ca.key similarity index 100% rename from docs/multitenant/usecase01/ca.key rename to docs/multitenant/ords-based/usecase01/ca.key diff --git a/docs/multitenant/usecase01/ca.srl b/docs/multitenant/ords-based/usecase01/ca.srl similarity index 100% rename from docs/multitenant/usecase01/ca.srl rename to docs/multitenant/ords-based/usecase01/ca.srl diff --git a/docs/multitenant/provisioning/singlenamespace/cdb_create.yaml b/docs/multitenant/ords-based/usecase01/cdb_create.yaml similarity index 100% rename from docs/multitenant/provisioning/singlenamespace/cdb_create.yaml rename to docs/multitenant/ords-based/usecase01/cdb_create.yaml diff --git a/docs/multitenant/usecase01/cdb_secret.yaml b/docs/multitenant/ords-based/usecase01/cdb_secret.yaml similarity index 100% rename from docs/multitenant/usecase01/cdb_secret.yaml rename to docs/multitenant/ords-based/usecase01/cdb_secret.yaml diff --git a/docs/multitenant/usecase02/pdb_clone.yaml b/docs/multitenant/ords-based/usecase01/clone_pdb1_resource.yaml similarity index 57% rename from docs/multitenant/usecase02/pdb_clone.yaml rename to docs/multitenant/ords-based/usecase01/clone_pdb1_resource.yaml index 0ecc3c70..3cc2c3dd 100644 --- a/docs/multitenant/usecase02/pdb_clone.yaml +++ b/docs/multitenant/ords-based/usecase01/clone_pdb1_resource.yaml @@ -1,11 +1,7 @@ -# -# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. -# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. -# -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: PDB metadata: - name: pdb2 + name: pdb3 namespace: oracle-database-operator-system labels: cdb: cdb-dev @@ -13,20 +9,21 @@ spec: cdbResName: "cdb-dev" cdbNamespace: "oracle-database-operator-system" cdbName: "DB12" - pdbName: "pdb2_clone" + pdbName: "new_clone" srcPdbName: "pdbdev" fileNameConversions: "NONE" totalSize: "UNLIMITED" tempSize: "UNLIMITED" assertivePdbDeletion: true + action: "Clone" adminName: secret: - secretName: "pdb1-secret" - key: "sysadmin_user" + secretName: "pdbusr" + key: "e_pdbusr.txt" adminPwd: secret: - secretName: "pdb1-secret" - key: "sysadmin_pwd" + secretName: "pdbpwd" + key: "e_pdbpwd.txt" pdbTlsKey: secret: secretName: "db-tls" @@ -41,10 +38,13 @@ spec: key: "ca.crt" webServerUser: secret: - secretName: "pdb1-secret" - key: "webserver_user" + secretName: "wbuser" + key: "e_wbuser.txt" webServerPwd: secret: - secretName: "pdb1-secret" - key: "webserver_pwd" - action: "Clone" + secretName: "wbpass" + key: "e_wbpass.txt" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/ords-based/usecase01/clone_pdb2_resource.yaml b/docs/multitenant/ords-based/usecase01/clone_pdb2_resource.yaml new file mode 100644 index 00000000..28a4eab6 --- /dev/null +++ b/docs/multitenant/ords-based/usecase01/clone_pdb2_resource.yaml @@ -0,0 +1,50 @@ +apiVersion: database.oracle.com/v4 +kind: PDB +metadata: + name: pdb4 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "oracle-database-operator-system" + cdbName: "DB12" + pdbName: "new_clone2" + srcPdbName: "pdbprd" + fileNameConversions: "NONE" + totalSize: "UNLIMITED" + tempSize: "UNLIMITED" + assertivePdbDeletion: true + action: "Clone" + adminName: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminPwd: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/ords-based/usecase01/close_pdb1_resource.yaml b/docs/multitenant/ords-based/usecase01/close_pdb1_resource.yaml new file mode 100644 index 00000000..a5c3cf59 --- /dev/null +++ b/docs/multitenant/ords-based/usecase01/close_pdb1_resource.yaml @@ -0,0 +1,47 @@ +apiVersion: database.oracle.com/v4 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "oracle-database-operator-system" + cdbName: "DB12" + pdbName: "pdbdev" + pdbState: "CLOSE" + modifyOption: "IMMEDIATE" + action: "Modify" + adminName: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminPwd: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/ords-based/usecase01/close_pdb2_resource.yaml b/docs/multitenant/ords-based/usecase01/close_pdb2_resource.yaml new file mode 100644 index 00000000..7fa15111 --- /dev/null +++ b/docs/multitenant/ords-based/usecase01/close_pdb2_resource.yaml @@ -0,0 +1,47 @@ +apiVersion: database.oracle.com/v4 +kind: PDB +metadata: + name: pdb2 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "oracle-database-operator-system" + cdbName: "DB12" + pdbName: "pdbprd" + pdbState: "CLOSE" + modifyOption: "IMMEDIATE" + action: "Modify" + adminName: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminPwd: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/ords-based/usecase01/close_pdb3_resource.yaml b/docs/multitenant/ords-based/usecase01/close_pdb3_resource.yaml new file mode 100644 index 00000000..fa7cf009 --- /dev/null +++ b/docs/multitenant/ords-based/usecase01/close_pdb3_resource.yaml @@ -0,0 +1,47 @@ +apiVersion: database.oracle.com/v4 +kind: PDB +metadata: + name: pdb3 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "oracle-database-operator-system" + cdbName: "DB12" + pdbName: ""new_clone" + pdbState: "CLOSE" + modifyOption: "IMMEDIATE" + action: "Modify" + adminName: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminPwd: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/ords-based/usecase01/create_ords_pod.yaml b/docs/multitenant/ords-based/usecase01/create_ords_pod.yaml new file mode 100644 index 00000000..e39c4c56 --- /dev/null +++ b/docs/multitenant/ords-based/usecase01/create_ords_pod.yaml @@ -0,0 +1,48 @@ +apiVersion: database.oracle.com/v4 +kind: CDB +metadata: + name: cdb-dev + namespace: oracle-database-operator-system +spec: + cdbName: "DB12" + ordsImage: _your_container_registry/ords-dboper:latest + ordsImagePullPolicy: "Always" + dbTnsurl : "T H I S I S J U S T A N E X A M P L E ....(DESCRIPTION=(CONNECT_TIMEOUT=90)(RETRY_COUNT=30)(RETRY_DELAY=10)(TRANSPORT_CONNECT_TIMEOUT=70)(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan12.testrac.com)(PORT=1521)(IP=V4_ONLY))(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan34.testrac.com)(PORT=1521)(IP=V4_ONLY))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=TESTORDS)))" + replicas: 1 + deletePdbCascade: true + sysAdminPwd: + secret: + secretName: "syspwd" + key: "e_syspwd.txt" + ordsPwd: + secret: + secretName: "ordpwd" + key: "e_ordpwd.txt" + cdbAdminUser: + secret: + secretName: "cdbusr" + key: "e_cdbusr.txt" + cdbAdminPwd: + secret: + secretName: "cdbpwd" + key: "e_cdbpwd.txt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + cdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + cdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + cdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/ords-based/usecase01/create_pdb1_resource.yaml b/docs/multitenant/ords-based/usecase01/create_pdb1_resource.yaml new file mode 100644 index 00000000..044d466b --- /dev/null +++ b/docs/multitenant/ords-based/usecase01/create_pdb1_resource.yaml @@ -0,0 +1,51 @@ +apiVersion: database.oracle.com/v4 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "oracle-database-operator-system" + cdbName: "DB12" + pdbName: "pdbdev" + assertivePdbDeletion: true + fileNameConversions: "NONE" + unlimitedStorage: false + tdeImport: false + totalSize: "2G" + tempSize: "800M" + action: "Create" + adminName: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminPwd: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/ords-based/usecase01/create_pdb2_resource.yaml b/docs/multitenant/ords-based/usecase01/create_pdb2_resource.yaml new file mode 100644 index 00000000..eb36aaa2 --- /dev/null +++ b/docs/multitenant/ords-based/usecase01/create_pdb2_resource.yaml @@ -0,0 +1,51 @@ +apiVersion: database.oracle.com/v4 +kind: PDB +metadata: + name: pdb2 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "oracle-database-operator-system" + cdbName: "DB12" + pdbName: "pdbprd" + assertivePdbDeletion: true + fileNameConversions: "NONE" + unlimitedStorage: false + tdeImport: false + totalSize: "2G" + tempSize: "800M" + action: "Create" + adminName: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminPwd: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/ords-based/usecase01/delete_pdb1_resource.yaml b/docs/multitenant/ords-based/usecase01/delete_pdb1_resource.yaml new file mode 100644 index 00000000..b0816929 --- /dev/null +++ b/docs/multitenant/ords-based/usecase01/delete_pdb1_resource.yaml @@ -0,0 +1,45 @@ +apiVersion: database.oracle.com/v4 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "oracle-database-operator-system" + pdbName: "pdbdev" + action: "Delete" + dropAction: "INCLUDING" + adminName: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminPwd: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/ords-based/usecase01/delete_pdb2_resource.yaml b/docs/multitenant/ords-based/usecase01/delete_pdb2_resource.yaml new file mode 100644 index 00000000..d2ad95cc --- /dev/null +++ b/docs/multitenant/ords-based/usecase01/delete_pdb2_resource.yaml @@ -0,0 +1,45 @@ +apiVersion: database.oracle.com/v4 +kind: PDB +metadata: + name: pdb2 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "oracle-database-operator-system" + pdbName: "pdbprd" + action: "Delete" + dropAction: "INCLUDING" + adminName: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminPwd: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/usecase01/extfile.txt b/docs/multitenant/ords-based/usecase01/extfile.txt similarity index 100% rename from docs/multitenant/usecase01/extfile.txt rename to docs/multitenant/ords-based/usecase01/extfile.txt diff --git a/docs/multitenant/ords-based/usecase01/logfiles/BuildImage.log b/docs/multitenant/ords-based/usecase01/logfiles/BuildImage.log new file mode 100644 index 00000000..f35c66d8 --- /dev/null +++ b/docs/multitenant/ords-based/usecase01/logfiles/BuildImage.log @@ -0,0 +1,896 @@ +/usr/bin/docker build -t oracle/ords-dboper:latest ../../../ords +Sending build context to Docker daemon 13.82kB +Step 1/12 : FROM container-registry.oracle.com/java/jdk:latest + ---> b8457e2f0b73 +Step 2/12 : ENV ORDS_HOME=/opt/oracle/ords/ RUN_FILE="runOrdsSSL.sh" ORDSVERSION=23.4.0-8 + ---> Using cache + ---> 3317a16cd6f8 +Step 3/12 : COPY $RUN_FILE $ORDS_HOME + ---> 7995edec33cc +Step 4/12 : RUN yum -y install yum-utils bind-utils tree hostname openssl net-tools zip unzip tar wget vim-minimal which sudo expect procps curl lsof && yum-config-manager --add-repo=http://yum.oracle.com/repo/OracleLinux/OL8/oracle/software/x86_64 && yum -y install java-11-openjdk-devel && yum -y install iproute && yum clean all + ---> Running in fe168b01f3ad +Oracle Linux 8 BaseOS Latest (x86_64) 91 MB/s | 79 MB 00:00 +Oracle Linux 8 Application Stream (x86_64) 69 MB/s | 62 MB 00:00 +Last metadata expiration check: 0:00:12 ago on Tue 20 Aug 2024 08:54:50 AM UTC. +Package yum-utils-4.0.21-23.0.1.el8.noarch is already installed. +Package tar-2:1.30-9.el8.x86_64 is already installed. +Package vim-minimal-2:8.0.1763-19.0.1.el8_6.4.x86_64 is already installed. +Package procps-ng-3.3.15-14.0.1.el8.x86_64 is already installed. +Package curl-7.61.1-33.el8_9.5.x86_64 is already installed. +Dependencies resolved. +================================================================================ + Package Arch Version Repository Size +================================================================================ +Installing: + bind-utils x86_64 32:9.11.36-16.el8_10.2 ol8_appstream 453 k + expect x86_64 5.45.4-5.el8 ol8_baseos_latest 266 k + hostname x86_64 3.20-6.el8 ol8_baseos_latest 32 k + lsof x86_64 4.93.2-1.el8 ol8_baseos_latest 253 k + net-tools x86_64 2.0-0.52.20160912git.el8 ol8_baseos_latest 322 k + openssl x86_64 1:1.1.1k-12.el8_9 ol8_baseos_latest 710 k + sudo x86_64 1.9.5p2-1.el8_9 ol8_baseos_latest 1.0 M + tree x86_64 1.7.0-15.el8 ol8_baseos_latest 59 k + unzip x86_64 6.0-46.0.1.el8 ol8_baseos_latest 196 k + wget x86_64 1.19.5-12.0.1.el8_10 ol8_appstream 733 k + which x86_64 2.21-20.el8 ol8_baseos_latest 50 k + zip x86_64 3.0-23.el8 ol8_baseos_latest 270 k +Upgrading: + curl x86_64 7.61.1-34.el8 ol8_baseos_latest 352 k + dnf-plugins-core noarch 4.0.21-25.0.1.el8 ol8_baseos_latest 76 k + libcurl x86_64 7.61.1-34.el8 ol8_baseos_latest 303 k + python3-dnf-plugins-core + noarch 4.0.21-25.0.1.el8 ol8_baseos_latest 263 k + yum-utils noarch 4.0.21-25.0.1.el8 ol8_baseos_latest 75 k +Installing dependencies: + bind-libs x86_64 32:9.11.36-16.el8_10.2 ol8_appstream 176 k + bind-libs-lite x86_64 32:9.11.36-16.el8_10.2 ol8_appstream 1.2 M + bind-license noarch 32:9.11.36-16.el8_10.2 ol8_appstream 104 k + fstrm x86_64 0.6.1-3.el8 ol8_appstream 29 k + libmaxminddb x86_64 1.2.0-10.el8_9.1 ol8_appstream 32 k + libmetalink x86_64 0.1.3-7.el8 ol8_baseos_latest 32 k + protobuf-c x86_64 1.3.0-8.el8 ol8_appstream 37 k + python3-bind noarch 32:9.11.36-16.el8_10.2 ol8_appstream 151 k + python3-ply noarch 3.9-9.el8 ol8_baseos_latest 111 k + tcl x86_64 1:8.6.8-2.el8 ol8_baseos_latest 1.1 M +Installing weak dependencies: + geolite2-city noarch 20180605-1.el8 ol8_appstream 19 M + geolite2-country noarch 20180605-1.el8 ol8_appstream 1.0 M + +Transaction Summary +================================================================================ +Install 24 Packages +Upgrade 5 Packages + +Total download size: 28 M +Downloading Packages: +(1/29): hostname-3.20-6.el8.x86_64.rpm 268 kB/s | 32 kB 00:00 +(2/29): libmetalink-0.1.3-7.el8.x86_64.rpm 257 kB/s | 32 kB 00:00 +(3/29): expect-5.45.4-5.el8.x86_64.rpm 1.4 MB/s | 266 kB 00:00 +(4/29): lsof-4.93.2-1.el8.x86_64.rpm 3.2 MB/s | 253 kB 00:00 +(5/29): net-tools-2.0-0.52.20160912git.el8.x86_ 3.6 MB/s | 322 kB 00:00 +(6/29): python3-ply-3.9-9.el8.noarch.rpm 2.7 MB/s | 111 kB 00:00 +(7/29): openssl-1.1.1k-12.el8_9.x86_64.rpm 10 MB/s | 710 kB 00:00 +(8/29): tree-1.7.0-15.el8.x86_64.rpm 2.2 MB/s | 59 kB 00:00 +(9/29): sudo-1.9.5p2-1.el8_9.x86_64.rpm 14 MB/s | 1.0 MB 00:00 +(10/29): unzip-6.0-46.0.1.el8.x86_64.rpm 6.8 MB/s | 196 kB 00:00 +(11/29): which-2.21-20.el8.x86_64.rpm 2.0 MB/s | 50 kB 00:00 +(12/29): tcl-8.6.8-2.el8.x86_64.rpm 13 MB/s | 1.1 MB 00:00 +(13/29): bind-libs-9.11.36-16.el8_10.2.x86_64.r 6.7 MB/s | 176 kB 00:00 +(14/29): zip-3.0-23.el8.x86_64.rpm 8.4 MB/s | 270 kB 00:00 +(15/29): bind-libs-lite-9.11.36-16.el8_10.2.x86 29 MB/s | 1.2 MB 00:00 +(16/29): bind-license-9.11.36-16.el8_10.2.noarc 3.3 MB/s | 104 kB 00:00 +(17/29): bind-utils-9.11.36-16.el8_10.2.x86_64. 13 MB/s | 453 kB 00:00 +(18/29): fstrm-0.6.1-3.el8.x86_64.rpm 1.2 MB/s | 29 kB 00:00 +(19/29): libmaxminddb-1.2.0-10.el8_9.1.x86_64.r 1.3 MB/s | 32 kB 00:00 +(20/29): geolite2-country-20180605-1.el8.noarch 17 MB/s | 1.0 MB 00:00 +(21/29): protobuf-c-1.3.0-8.el8.x86_64.rpm 1.5 MB/s | 37 kB 00:00 +(22/29): python3-bind-9.11.36-16.el8_10.2.noarc 5.8 MB/s | 151 kB 00:00 +(23/29): wget-1.19.5-12.0.1.el8_10.x86_64.rpm 17 MB/s | 733 kB 00:00 +(24/29): curl-7.61.1-34.el8.x86_64.rpm 12 MB/s | 352 kB 00:00 +(25/29): dnf-plugins-core-4.0.21-25.0.1.el8.noa 2.4 MB/s | 76 kB 00:00 +(26/29): libcurl-7.61.1-34.el8.x86_64.rpm 8.6 MB/s | 303 kB 00:00 +(27/29): python3-dnf-plugins-core-4.0.21-25.0.1 9.8 MB/s | 263 kB 00:00 +(28/29): yum-utils-4.0.21-25.0.1.el8.noarch.rpm 3.0 MB/s | 75 kB 00:00 +(29/29): geolite2-city-20180605-1.el8.noarch.rp 66 MB/s | 19 MB 00:00 +-------------------------------------------------------------------------------- +Total 43 MB/s | 28 MB 00:00 +Running transaction check +Transaction check succeeded. +Running transaction test +Transaction test succeeded. +Running transaction + Preparing : 1/1 + Running scriptlet: protobuf-c-1.3.0-8.el8.x86_64 1/1 + Installing : protobuf-c-1.3.0-8.el8.x86_64 1/34 + Installing : fstrm-0.6.1-3.el8.x86_64 2/34 + Installing : bind-license-32:9.11.36-16.el8_10.2.noarch 3/34 + Upgrading : python3-dnf-plugins-core-4.0.21-25.0.1.el8.noarch 4/34 + Upgrading : dnf-plugins-core-4.0.21-25.0.1.el8.noarch 5/34 + Upgrading : libcurl-7.61.1-34.el8.x86_64 6/34 + Installing : geolite2-country-20180605-1.el8.noarch 7/34 + Installing : geolite2-city-20180605-1.el8.noarch 8/34 + Installing : libmaxminddb-1.2.0-10.el8_9.1.x86_64 9/34 + Running scriptlet: libmaxminddb-1.2.0-10.el8_9.1.x86_64 9/34 + Installing : bind-libs-lite-32:9.11.36-16.el8_10.2.x86_64 10/34 + Installing : bind-libs-32:9.11.36-16.el8_10.2.x86_64 11/34 + Installing : unzip-6.0-46.0.1.el8.x86_64 12/34 + Installing : tcl-1:8.6.8-2.el8.x86_64 13/34 + Running scriptlet: tcl-1:8.6.8-2.el8.x86_64 13/34 + Installing : python3-ply-3.9-9.el8.noarch 14/34 + Installing : python3-bind-32:9.11.36-16.el8_10.2.noarch 15/34 + Installing : libmetalink-0.1.3-7.el8.x86_64 16/34 + Installing : wget-1.19.5-12.0.1.el8_10.x86_64 17/34 + Running scriptlet: wget-1.19.5-12.0.1.el8_10.x86_64 17/34 + Installing : bind-utils-32:9.11.36-16.el8_10.2.x86_64 18/34 + Installing : expect-5.45.4-5.el8.x86_64 19/34 + Installing : zip-3.0-23.el8.x86_64 20/34 + Upgrading : curl-7.61.1-34.el8.x86_64 21/34 + Upgrading : yum-utils-4.0.21-25.0.1.el8.noarch 22/34 + Installing : which-2.21-20.el8.x86_64 23/34 + Installing : tree-1.7.0-15.el8.x86_64 24/34 + Installing : sudo-1.9.5p2-1.el8_9.x86_64 25/34 + Running scriptlet: sudo-1.9.5p2-1.el8_9.x86_64 25/34 + Installing : openssl-1:1.1.1k-12.el8_9.x86_64 26/34 + Installing : net-tools-2.0-0.52.20160912git.el8.x86_64 27/34 + Running scriptlet: net-tools-2.0-0.52.20160912git.el8.x86_64 27/34 + Installing : lsof-4.93.2-1.el8.x86_64 28/34 + Installing : hostname-3.20-6.el8.x86_64 29/34 + Running scriptlet: hostname-3.20-6.el8.x86_64 29/34 + Cleanup : curl-7.61.1-33.el8_9.5.x86_64 30/34 + Cleanup : yum-utils-4.0.21-23.0.1.el8.noarch 31/34 + Cleanup : dnf-plugins-core-4.0.21-23.0.1.el8.noarch 32/34 + Cleanup : python3-dnf-plugins-core-4.0.21-23.0.1.el8.noarch 33/34 + Cleanup : libcurl-7.61.1-33.el8_9.5.x86_64 34/34 + Running scriptlet: libcurl-7.61.1-33.el8_9.5.x86_64 34/34 + Verifying : expect-5.45.4-5.el8.x86_64 1/34 + Verifying : hostname-3.20-6.el8.x86_64 2/34 + Verifying : libmetalink-0.1.3-7.el8.x86_64 3/34 + Verifying : lsof-4.93.2-1.el8.x86_64 4/34 + Verifying : net-tools-2.0-0.52.20160912git.el8.x86_64 5/34 + Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 6/34 + Verifying : python3-ply-3.9-9.el8.noarch 7/34 + Verifying : sudo-1.9.5p2-1.el8_9.x86_64 8/34 + Verifying : tcl-1:8.6.8-2.el8.x86_64 9/34 + Verifying : tree-1.7.0-15.el8.x86_64 10/34 + Verifying : unzip-6.0-46.0.1.el8.x86_64 11/34 + Verifying : which-2.21-20.el8.x86_64 12/34 + Verifying : zip-3.0-23.el8.x86_64 13/34 + Verifying : bind-libs-32:9.11.36-16.el8_10.2.x86_64 14/34 + Verifying : bind-libs-lite-32:9.11.36-16.el8_10.2.x86_64 15/34 + Verifying : bind-license-32:9.11.36-16.el8_10.2.noarch 16/34 + Verifying : bind-utils-32:9.11.36-16.el8_10.2.x86_64 17/34 + Verifying : fstrm-0.6.1-3.el8.x86_64 18/34 + Verifying : geolite2-city-20180605-1.el8.noarch 19/34 + Verifying : geolite2-country-20180605-1.el8.noarch 20/34 + Verifying : libmaxminddb-1.2.0-10.el8_9.1.x86_64 21/34 + Verifying : protobuf-c-1.3.0-8.el8.x86_64 22/34 + Verifying : python3-bind-32:9.11.36-16.el8_10.2.noarch 23/34 + Verifying : wget-1.19.5-12.0.1.el8_10.x86_64 24/34 + Verifying : curl-7.61.1-34.el8.x86_64 25/34 + Verifying : curl-7.61.1-33.el8_9.5.x86_64 26/34 + Verifying : dnf-plugins-core-4.0.21-25.0.1.el8.noarch 27/34 + Verifying : dnf-plugins-core-4.0.21-23.0.1.el8.noarch 28/34 + Verifying : libcurl-7.61.1-34.el8.x86_64 29/34 + Verifying : libcurl-7.61.1-33.el8_9.5.x86_64 30/34 + Verifying : python3-dnf-plugins-core-4.0.21-25.0.1.el8.noarch 31/34 + Verifying : python3-dnf-plugins-core-4.0.21-23.0.1.el8.noarch 32/34 + Verifying : yum-utils-4.0.21-25.0.1.el8.noarch 33/34 + Verifying : yum-utils-4.0.21-23.0.1.el8.noarch 34/34 + +Upgraded: + curl-7.61.1-34.el8.x86_64 + dnf-plugins-core-4.0.21-25.0.1.el8.noarch + libcurl-7.61.1-34.el8.x86_64 + python3-dnf-plugins-core-4.0.21-25.0.1.el8.noarch + yum-utils-4.0.21-25.0.1.el8.noarch +Installed: + bind-libs-32:9.11.36-16.el8_10.2.x86_64 + bind-libs-lite-32:9.11.36-16.el8_10.2.x86_64 + bind-license-32:9.11.36-16.el8_10.2.noarch + bind-utils-32:9.11.36-16.el8_10.2.x86_64 + expect-5.45.4-5.el8.x86_64 + fstrm-0.6.1-3.el8.x86_64 + geolite2-city-20180605-1.el8.noarch + geolite2-country-20180605-1.el8.noarch + hostname-3.20-6.el8.x86_64 + libmaxminddb-1.2.0-10.el8_9.1.x86_64 + libmetalink-0.1.3-7.el8.x86_64 + lsof-4.93.2-1.el8.x86_64 + net-tools-2.0-0.52.20160912git.el8.x86_64 + openssl-1:1.1.1k-12.el8_9.x86_64 + protobuf-c-1.3.0-8.el8.x86_64 + python3-bind-32:9.11.36-16.el8_10.2.noarch + python3-ply-3.9-9.el8.noarch + sudo-1.9.5p2-1.el8_9.x86_64 + tcl-1:8.6.8-2.el8.x86_64 + tree-1.7.0-15.el8.x86_64 + unzip-6.0-46.0.1.el8.x86_64 + wget-1.19.5-12.0.1.el8_10.x86_64 + which-2.21-20.el8.x86_64 + zip-3.0-23.el8.x86_64 + +Complete! +Adding repo from: http://yum.oracle.com/repo/OracleLinux/OL8/oracle/software/x86_64 +created by dnf config-manager from http://yum.o 496 kB/s | 139 kB 00:00 +Last metadata expiration check: 0:00:01 ago on Tue 20 Aug 2024 08:55:14 AM UTC. +Dependencies resolved. +============================================================================================== + Package Arch Version Repository Size +============================================================================================== +Installing: + java-11-openjdk-devel x86_64 1:11.0.24.0.8-3.0.1.el8 ol8_appstream 3.4 M +Installing dependencies: + adwaita-cursor-theme noarch 3.28.0-3.el8 ol8_appstream 647 k + adwaita-icon-theme noarch 3.28.0-3.el8 ol8_appstream 11 M + alsa-lib x86_64 1.2.10-2.el8 ol8_appstream 500 k + at-spi2-atk x86_64 2.26.2-1.el8 ol8_appstream 89 k + at-spi2-core x86_64 2.28.0-1.el8 ol8_appstream 169 k + atk x86_64 2.28.1-1.el8 ol8_appstream 272 k + avahi-libs x86_64 0.7-27.el8 ol8_baseos_latest 61 k + cairo x86_64 1.15.12-6.el8 ol8_appstream 719 k + cairo-gobject x86_64 1.15.12-6.el8 ol8_appstream 33 k + colord-libs x86_64 1.4.2-1.el8 ol8_appstream 236 k + copy-jdk-configs noarch 4.0-2.el8 ol8_appstream 30 k + cpio x86_64 2.12-11.el8 ol8_baseos_latest 266 k + crypto-policies-scripts noarch 20230731-1.git3177e06.el8 ol8_baseos_latest 84 k + cups-libs x86_64 1:2.2.6-60.el8_10 ol8_baseos_latest 435 k + dracut x86_64 049-233.git20240115.0.1.el8 ol8_baseos_latest 382 k + file x86_64 5.33-25.el8 ol8_baseos_latest 77 k + fribidi x86_64 1.0.4-9.el8 ol8_appstream 89 k + gdk-pixbuf2 x86_64 2.36.12-6.el8_10 ol8_baseos_latest 465 k + gdk-pixbuf2-modules x86_64 2.36.12-6.el8_10 ol8_appstream 108 k + gettext x86_64 0.19.8.1-17.el8 ol8_baseos_latest 1.1 M + gettext-libs x86_64 0.19.8.1-17.el8 ol8_baseos_latest 312 k + glib-networking x86_64 2.56.1-1.1.el8 ol8_baseos_latest 155 k + graphite2 x86_64 1.3.10-10.el8 ol8_appstream 122 k + grub2-common noarch 1:2.02-156.0.2.el8 ol8_baseos_latest 897 k + grub2-tools x86_64 1:2.02-156.0.2.el8 ol8_baseos_latest 2.0 M + grub2-tools-minimal x86_64 1:2.02-156.0.2.el8 ol8_baseos_latest 215 k + gsettings-desktop-schemas x86_64 3.32.0-6.el8 ol8_baseos_latest 633 k + gtk-update-icon-cache x86_64 3.22.30-11.el8 ol8_appstream 32 k + harfbuzz x86_64 1.7.5-4.el8 ol8_appstream 295 k + hicolor-icon-theme noarch 0.17-2.el8 ol8_appstream 48 k + jasper-libs x86_64 2.0.14-5.el8 ol8_appstream 167 k + java-11-openjdk x86_64 1:11.0.24.0.8-3.0.1.el8 ol8_appstream 475 k + java-11-openjdk-headless x86_64 1:11.0.24.0.8-3.0.1.el8 ol8_appstream 42 M + javapackages-filesystem noarch 5.3.0-1.module+el8+5136+7ff78f74 ol8_appstream 30 k + jbigkit-libs x86_64 2.1-14.el8 ol8_appstream 55 k + json-glib x86_64 1.4.4-1.el8 ol8_baseos_latest 144 k + kbd-legacy noarch 2.0.4-11.el8 ol8_baseos_latest 481 k + kbd-misc noarch 2.0.4-11.el8 ol8_baseos_latest 1.5 M + lcms2 x86_64 2.9-2.el8 ol8_appstream 164 k + libX11 x86_64 1.6.8-8.el8 ol8_appstream 611 k + libX11-common noarch 1.6.8-8.el8 ol8_appstream 157 k + libXau x86_64 1.0.9-3.el8 ol8_appstream 37 k + libXcomposite x86_64 0.4.4-14.el8 ol8_appstream 28 k + libXcursor x86_64 1.1.15-3.el8 ol8_appstream 36 k + libXdamage x86_64 1.1.4-14.el8 ol8_appstream 27 k + libXext x86_64 1.3.4-1.el8 ol8_appstream 45 k + libXfixes x86_64 5.0.3-7.el8 ol8_appstream 25 k + libXft x86_64 2.3.3-1.el8 ol8_appstream 67 k + libXi x86_64 1.7.10-1.el8 ol8_appstream 49 k + libXinerama x86_64 1.1.4-1.el8 ol8_appstream 15 k + libXrandr x86_64 1.5.2-1.el8 ol8_appstream 34 k + libXrender x86_64 0.9.10-7.el8 ol8_appstream 33 k + libXtst x86_64 1.2.3-7.el8 ol8_appstream 22 k + libcroco x86_64 0.6.12-4.el8_2.1 ol8_baseos_latest 113 k + libdatrie x86_64 0.2.9-7.el8 ol8_appstream 33 k + libepoxy x86_64 1.5.8-1.el8 ol8_appstream 225 k + libfontenc x86_64 1.1.3-8.el8 ol8_appstream 37 k + libgomp x86_64 8.5.0-22.0.1.el8_10 ol8_baseos_latest 218 k + libgusb x86_64 0.3.0-1.el8 ol8_baseos_latest 49 k + libjpeg-turbo x86_64 1.5.3-12.el8 ol8_appstream 157 k + libkcapi x86_64 1.4.0-2.0.1.el8 ol8_baseos_latest 52 k + libkcapi-hmaccalc x86_64 1.4.0-2.0.1.el8 ol8_baseos_latest 31 k + libmodman x86_64 2.0.1-17.el8 ol8_baseos_latest 36 k + libpkgconf x86_64 1.4.2-1.el8 ol8_baseos_latest 35 k + libproxy x86_64 0.4.15-5.2.el8 ol8_baseos_latest 75 k + libsoup x86_64 2.62.3-5.el8 ol8_baseos_latest 424 k + libthai x86_64 0.1.27-2.el8 ol8_appstream 203 k + libtiff x86_64 4.0.9-32.el8_10 ol8_appstream 189 k + libwayland-client x86_64 1.21.0-1.el8 ol8_appstream 41 k + libwayland-cursor x86_64 1.21.0-1.el8 ol8_appstream 26 k + libwayland-egl x86_64 1.21.0-1.el8 ol8_appstream 19 k + libxcb x86_64 1.13.1-1.el8 ol8_appstream 231 k + libxkbcommon x86_64 0.9.1-1.el8 ol8_appstream 116 k + lksctp-tools x86_64 1.0.18-3.el8 ol8_baseos_latest 100 k + lua x86_64 5.3.4-12.el8 ol8_appstream 192 k + nspr x86_64 4.35.0-1.el8_8 ol8_appstream 143 k + nss x86_64 3.90.0-7.el8_10 ol8_appstream 750 k + nss-softokn x86_64 3.90.0-7.el8_10 ol8_appstream 1.2 M + nss-softokn-freebl x86_64 3.90.0-7.el8_10 ol8_appstream 375 k + nss-sysinit x86_64 3.90.0-7.el8_10 ol8_appstream 74 k + nss-util x86_64 3.90.0-7.el8_10 ol8_appstream 139 k + os-prober x86_64 1.74-9.0.1.el8 ol8_baseos_latest 51 k + pango x86_64 1.42.4-8.el8 ol8_appstream 297 k + pixman x86_64 0.38.4-4.el8 ol8_appstream 256 k + pkgconf x86_64 1.4.2-1.el8 ol8_baseos_latest 38 k + pkgconf-m4 noarch 1.4.2-1.el8 ol8_baseos_latest 17 k + pkgconf-pkg-config x86_64 1.4.2-1.el8 ol8_baseos_latest 15 k + rest x86_64 0.8.1-2.el8 ol8_appstream 70 k + shared-mime-info x86_64 1.9-4.el8 ol8_baseos_latest 328 k + systemd-udev x86_64 239-78.0.4.el8 ol8_baseos_latest 1.6 M + ttmkfdir x86_64 3.0.9-54.el8 ol8_appstream 62 k + tzdata-java noarch 2024a-1.0.1.el8 ol8_appstream 186 k + xkeyboard-config noarch 2.28-1.el8 ol8_appstream 782 k + xorg-x11-font-utils x86_64 1:7.5-41.el8 ol8_appstream 104 k + xorg-x11-fonts-Type1 noarch 7.5-19.el8 ol8_appstream 522 k + xz x86_64 5.2.4-4.el8_6 ol8_baseos_latest 153 k +Installing weak dependencies: + abattis-cantarell-fonts noarch 0.0.25-6.el8 ol8_appstream 155 k + dconf x86_64 0.28.0-4.0.1.el8 ol8_appstream 108 k + dejavu-sans-mono-fonts noarch 2.35-7.el8 ol8_baseos_latest 447 k + grubby x86_64 8.40-49.0.2.el8 ol8_baseos_latest 50 k + gtk3 x86_64 3.22.30-11.el8 ol8_appstream 4.5 M + hardlink x86_64 1:1.3-6.el8 ol8_baseos_latest 29 k + kbd x86_64 2.0.4-11.el8 ol8_baseos_latest 390 k + memstrack x86_64 0.2.5-2.el8 ol8_baseos_latest 51 k + pigz x86_64 2.4-4.el8 ol8_baseos_latest 80 k +Enabling module streams: + javapackages-runtime 201801 + +Transaction Summary +============================================================================================== +Install 106 Packages + +Total download size: 86 M +Installed size: 312 M +Downloading Packages: +(1/106): crypto-policies-scripts-20230731-1.git 862 kB/s | 84 kB 00:00 +(2/106): avahi-libs-0.7-27.el8.x86_64.rpm 602 kB/s | 61 kB 00:00 +(3/106): cpio-2.12-11.el8.x86_64.rpm 1.8 MB/s | 266 kB 00:00 +(4/106): cups-libs-2.2.6-60.el8_10.x86_64.rpm 5.7 MB/s | 435 kB 00:00 +(5/106): dejavu-sans-mono-fonts-2.35-7.el8.noar 5.1 MB/s | 447 kB 00:00 +(6/106): dracut-049-233.git20240115.0.1.el8.x86 7.0 MB/s | 382 kB 00:00 +(7/106): gdk-pixbuf2-2.36.12-6.el8_10.x86_64.rp 12 MB/s | 465 kB 00:00 +(8/106): gettext-libs-0.19.8.1-17.el8.x86_64.rp 9.3 MB/s | 312 kB 00:00 +(9/106): gettext-0.19.8.1-17.el8.x86_64.rpm 16 MB/s | 1.1 MB 00:00 +(10/106): glib-networking-2.56.1-1.1.el8.x86_64 6.0 MB/s | 155 kB 00:00 +(11/106): grub2-common-2.02-156.0.2.el8.noarch. 26 MB/s | 897 kB 00:00 +(12/106): grub2-tools-minimal-2.02-156.0.2.el8. 8.2 MB/s | 215 kB 00:00 +(13/106): grubby-8.40-49.0.2.el8.x86_64.rpm 2.1 MB/s | 50 kB 00:00 +(14/106): grub2-tools-2.02-156.0.2.el8.x86_64.r 26 MB/s | 2.0 MB 00:00 +(15/106): gsettings-desktop-schemas-3.32.0-6.el 19 MB/s | 633 kB 00:00 +(16/106): hardlink-1.3-6.el8.x86_64.rpm 1.1 MB/s | 29 kB 00:00 +(17/106): json-glib-1.4.4-1.el8.x86_64.rpm 5.9 MB/s | 144 kB 00:00 +(18/106): kbd-2.0.4-11.el8.x86_64.rpm 14 MB/s | 390 kB 00:00 +(19/106): kbd-legacy-2.0.4-11.el8.noarch.rpm 17 MB/s | 481 kB 00:00 +(20/106): kbd-misc-2.0.4-11.el8.noarch.rpm 41 MB/s | 1.5 MB 00:00 +(21/106): libcroco-0.6.12-4.el8_2.1.x86_64.rpm 4.7 MB/s | 113 kB 00:00 +(22/106): libgomp-8.5.0-22.0.1.el8_10.x86_64.rp 9.1 MB/s | 218 kB 00:00 +(23/106): libgusb-0.3.0-1.el8.x86_64.rpm 2.1 MB/s | 49 kB 00:00 +(24/106): libkcapi-1.4.0-2.0.1.el8.x86_64.rpm 1.6 MB/s | 52 kB 00:00 +(25/106): libkcapi-hmaccalc-1.4.0-2.0.1.el8.x86 822 kB/s | 31 kB 00:00 +(26/106): libmodman-2.0.1-17.el8.x86_64.rpm 1.6 MB/s | 36 kB 00:00 +(27/106): libpkgconf-1.4.2-1.el8.x86_64.rpm 1.2 MB/s | 35 kB 00:00 +(28/106): libproxy-0.4.15-5.2.el8.x86_64.rpm 3.0 MB/s | 75 kB 00:00 +(29/106): libsoup-2.62.3-5.el8.x86_64.rpm 15 MB/s | 424 kB 00:00 +(30/106): lksctp-tools-1.0.18-3.el8.x86_64.rpm 3.5 MB/s | 100 kB 00:00 +(31/106): memstrack-0.2.5-2.el8.x86_64.rpm 2.2 MB/s | 51 kB 00:00 +(32/106): os-prober-1.74-9.0.1.el8.x86_64.rpm 2.2 MB/s | 51 kB 00:00 +(33/106): pigz-2.4-4.el8.x86_64.rpm 3.5 MB/s | 80 kB 00:00 +(34/106): pkgconf-1.4.2-1.el8.x86_64.rpm 1.7 MB/s | 38 kB 00:00 +(35/106): pkgconf-m4-1.4.2-1.el8.noarch.rpm 761 kB/s | 17 kB 00:00 +(36/106): pkgconf-pkg-config-1.4.2-1.el8.x86_64 691 kB/s | 15 kB 00:00 +(37/106): shared-mime-info-1.9-4.el8.x86_64.rpm 13 MB/s | 328 kB 00:00 +(38/106): systemd-udev-239-78.0.4.el8.x86_64.rp 32 MB/s | 1.6 MB 00:00 +(39/106): xz-5.2.4-4.el8_6.x86_64.rpm 5.2 MB/s | 153 kB 00:00 +(40/106): abattis-cantarell-fonts-0.0.25-6.el8. 6.4 MB/s | 155 kB 00:00 +(41/106): adwaita-cursor-theme-3.28.0-3.el8.noa 22 MB/s | 647 kB 00:00 +(42/106): alsa-lib-1.2.10-2.el8.x86_64.rpm 18 MB/s | 500 kB 00:00 +(43/106): at-spi2-atk-2.26.2-1.el8.x86_64.rpm 3.8 MB/s | 89 kB 00:00 +(44/106): at-spi2-core-2.28.0-1.el8.x86_64.rpm 6.9 MB/s | 169 kB 00:00 +(45/106): atk-2.28.1-1.el8.x86_64.rpm 9.2 MB/s | 272 kB 00:00 +(46/106): cairo-1.15.12-6.el8.x86_64.rpm 24 MB/s | 719 kB 00:00 +(47/106): adwaita-icon-theme-3.28.0-3.el8.noarc 65 MB/s | 11 MB 00:00 +(48/106): cairo-gobject-1.15.12-6.el8.x86_64.rp 914 kB/s | 33 kB 00:00 +(49/106): colord-libs-1.4.2-1.el8.x86_64.rpm 9.5 MB/s | 236 kB 00:00 +(50/106): copy-jdk-configs-4.0-2.el8.noarch.rpm 1.1 MB/s | 30 kB 00:00 +(51/106): dconf-0.28.0-4.0.1.el8.x86_64.rpm 4.4 MB/s | 108 kB 00:00 +(52/106): fribidi-1.0.4-9.el8.x86_64.rpm 3.9 MB/s | 89 kB 00:00 +(53/106): graphite2-1.3.10-10.el8.x86_64.rpm 5.1 MB/s | 122 kB 00:00 +(54/106): gdk-pixbuf2-modules-2.36.12-6.el8_10. 3.6 MB/s | 108 kB 00:00 +(55/106): gtk-update-icon-cache-3.22.30-11.el8. 1.4 MB/s | 32 kB 00:00 +(56/106): harfbuzz-1.7.5-4.el8.x86_64.rpm 11 MB/s | 295 kB 00:00 +(57/106): gtk3-3.22.30-11.el8.x86_64.rpm 68 MB/s | 4.5 MB 00:00 +(58/106): hicolor-icon-theme-0.17-2.el8.noarch. 2.1 MB/s | 48 kB 00:00 +(59/106): java-11-openjdk-11.0.24.0.8-3.0.1.el8 17 MB/s | 475 kB 00:00 +(60/106): jasper-libs-2.0.14-5.el8.x86_64.rpm 5.0 MB/s | 167 kB 00:00 +(61/106): java-11-openjdk-devel-11.0.24.0.8-3.0 61 MB/s | 3.4 MB 00:00 +(62/106): javapackages-filesystem-5.3.0-1.modul 1.2 MB/s | 30 kB 00:00 +(63/106): jbigkit-libs-2.1-14.el8.x86_64.rpm 2.1 MB/s | 55 kB 00:00 +(64/106): lcms2-2.9-2.el8.x86_64.rpm 3.8 MB/s | 164 kB 00:00 +(65/106): libX11-1.6.8-8.el8.x86_64.rpm 20 MB/s | 611 kB 00:00 +(66/106): libX11-common-1.6.8-8.el8.noarch.rpm 6.8 MB/s | 157 kB 00:00 +(67/106): libXau-1.0.9-3.el8.x86_64.rpm 1.6 MB/s | 37 kB 00:00 +(68/106): libXcomposite-0.4.4-14.el8.x86_64.rpm 1.3 MB/s | 28 kB 00:00 +(69/106): libXcursor-1.1.15-3.el8.x86_64.rpm 1.6 MB/s | 36 kB 00:00 +(70/106): libXdamage-1.1.4-14.el8.x86_64.rpm 1.2 MB/s | 27 kB 00:00 +(71/106): libXext-1.3.4-1.el8.x86_64.rpm 2.0 MB/s | 45 kB 00:00 +(72/106): libXfixes-5.0.3-7.el8.x86_64.rpm 1.1 MB/s | 25 kB 00:00 +(73/106): libXft-2.3.3-1.el8.x86_64.rpm 2.9 MB/s | 67 kB 00:00 +(74/106): libXi-1.7.10-1.el8.x86_64.rpm 2.2 MB/s | 49 kB 00:00 +(75/106): libXinerama-1.1.4-1.el8.x86_64.rpm 717 kB/s | 15 kB 00:00 +(76/106): libXrandr-1.5.2-1.el8.x86_64.rpm 1.5 MB/s | 34 kB 00:00 +(77/106): libXrender-0.9.10-7.el8.x86_64.rpm 1.4 MB/s | 33 kB 00:00 +(78/106): libXtst-1.2.3-7.el8.x86_64.rpm 957 kB/s | 22 kB 00:00 +(79/106): java-11-openjdk-headless-11.0.24.0.8- 71 MB/s | 42 MB 00:00 +(80/106): libdatrie-0.2.9-7.el8.x86_64.rpm 274 kB/s | 33 kB 00:00 +(81/106): libepoxy-1.5.8-1.el8.x86_64.rpm 9.1 MB/s | 225 kB 00:00 +(82/106): libfontenc-1.1.3-8.el8.x86_64.rpm 1.5 MB/s | 37 kB 00:00 +(83/106): libthai-0.1.27-2.el8.x86_64.rpm 8.2 MB/s | 203 kB 00:00 +(84/106): libjpeg-turbo-1.5.3-12.el8.x86_64.rpm 5.1 MB/s | 157 kB 00:00 +(85/106): libtiff-4.0.9-32.el8_10.x86_64.rpm 7.8 MB/s | 189 kB 00:00 +(86/106): libwayland-client-1.21.0-1.el8.x86_64 1.7 MB/s | 41 kB 00:00 +(87/106): libwayland-cursor-1.21.0-1.el8.x86_64 1.2 MB/s | 26 kB 00:00 +(88/106): libwayland-egl-1.21.0-1.el8.x86_64.rp 801 kB/s | 19 kB 00:00 +(89/106): libxcb-1.13.1-1.el8.x86_64.rpm 9.7 MB/s | 231 kB 00:00 +(90/106): libxkbcommon-0.9.1-1.el8.x86_64.rpm 5.0 MB/s | 116 kB 00:00 +(91/106): nspr-4.35.0-1.el8_8.x86_64.rpm 6.0 MB/s | 143 kB 00:00 +(92/106): lua-5.3.4-12.el8.x86_64.rpm 5.9 MB/s | 192 kB 00:00 +(93/106): nss-softokn-3.90.0-7.el8_10.x86_64.rp 38 MB/s | 1.2 MB 00:00 +(94/106): nss-3.90.0-7.el8_10.x86_64.rpm 17 MB/s | 750 kB 00:00 +(95/106): nss-softokn-freebl-3.90.0-7.el8_10.x8 14 MB/s | 375 kB 00:00 +(96/106): nss-sysinit-3.90.0-7.el8_10.x86_64.rp 3.2 MB/s | 74 kB 00:00 +(97/106): nss-util-3.90.0-7.el8_10.x86_64.rpm 5.8 MB/s | 139 kB 00:00 +(98/106): pango-1.42.4-8.el8.x86_64.rpm 11 MB/s | 297 kB 00:00 +(99/106): pixman-0.38.4-4.el8.x86_64.rpm 10 MB/s | 256 kB 00:00 +(100/106): rest-0.8.1-2.el8.x86_64.rpm 3.1 MB/s | 70 kB 00:00 +(101/106): ttmkfdir-3.0.9-54.el8.x86_64.rpm 2.5 MB/s | 62 kB 00:00 +(102/106): tzdata-java-2024a-1.0.1.el8.noarch.r 7.4 MB/s | 186 kB 00:00 +(103/106): xkeyboard-config-2.28-1.el8.noarch.r 27 MB/s | 782 kB 00:00 +(104/106): xorg-x11-font-utils-7.5-41.el8.x86_6 3.9 MB/s | 104 kB 00:00 +(105/106): xorg-x11-fonts-Type1-7.5-19.el8.noar 1.3 MB/s | 522 kB 00:00 +(106/106): file-5.33-25.el8.x86_64.rpm 26 kB/s | 77 kB 00:02 +-------------------------------------------------------------------------------- +Total 27 MB/s | 86 MB 00:03 +Running transaction check +Transaction check succeeded. +Running transaction test +Transaction test succeeded. +Running transaction + Running scriptlet: copy-jdk-configs-4.0-2.el8.noarch 1/1 + Running scriptlet: java-11-openjdk-headless-1:11.0.24.0.8-3.0.1.el8.x86 1/1 + Preparing : 1/1 + Installing : nspr-4.35.0-1.el8_8.x86_64 1/106 + Running scriptlet: nspr-4.35.0-1.el8_8.x86_64 1/106 + Installing : nss-util-3.90.0-7.el8_10.x86_64 2/106 + Installing : libjpeg-turbo-1.5.3-12.el8.x86_64 3/106 + Installing : pixman-0.38.4-4.el8.x86_64 4/106 + Installing : libwayland-client-1.21.0-1.el8.x86_64 5/106 + Installing : atk-2.28.1-1.el8.x86_64 6/106 + Installing : libgomp-8.5.0-22.0.1.el8_10.x86_64 7/106 + Running scriptlet: libgomp-8.5.0-22.0.1.el8_10.x86_64 7/106 + Installing : libcroco-0.6.12-4.el8_2.1.x86_64 8/106 + Running scriptlet: libcroco-0.6.12-4.el8_2.1.x86_64 8/106 + Installing : grub2-common-1:2.02-156.0.2.el8.noarch 9/106 + Installing : gettext-libs-0.19.8.1-17.el8.x86_64 10/106 + Installing : gettext-0.19.8.1-17.el8.x86_64 11/106 + Running scriptlet: gettext-0.19.8.1-17.el8.x86_64 11/106 + Installing : grub2-tools-minimal-1:2.02-156.0.2.el8.x86_64 12/106 + Installing : libwayland-cursor-1.21.0-1.el8.x86_64 13/106 + Installing : jasper-libs-2.0.14-5.el8.x86_64 14/106 + Installing : nss-softokn-freebl-3.90.0-7.el8_10.x86_64 15/106 + Installing : nss-softokn-3.90.0-7.el8_10.x86_64 16/106 + Installing : xkeyboard-config-2.28-1.el8.noarch 17/106 + Installing : libxkbcommon-0.9.1-1.el8.x86_64 18/106 + Installing : tzdata-java-2024a-1.0.1.el8.noarch 19/106 + Installing : ttmkfdir-3.0.9-54.el8.x86_64 20/106 + Installing : lua-5.3.4-12.el8.x86_64 21/106 + Installing : copy-jdk-configs-4.0-2.el8.noarch 22/106 + Installing : libwayland-egl-1.21.0-1.el8.x86_64 23/106 + Installing : libfontenc-1.1.3-8.el8.x86_64 24/106 + Installing : libepoxy-1.5.8-1.el8.x86_64 25/106 + Installing : libdatrie-0.2.9-7.el8.x86_64 26/106 + Running scriptlet: libdatrie-0.2.9-7.el8.x86_64 26/106 + Installing : libthai-0.1.27-2.el8.x86_64 27/106 + Running scriptlet: libthai-0.1.27-2.el8.x86_64 27/106 + Installing : libXau-1.0.9-3.el8.x86_64 28/106 + Installing : libxcb-1.13.1-1.el8.x86_64 29/106 + Installing : libX11-common-1.6.8-8.el8.noarch 30/106 + Installing : libX11-1.6.8-8.el8.x86_64 31/106 + Installing : libXext-1.3.4-1.el8.x86_64 32/106 + Installing : libXrender-0.9.10-7.el8.x86_64 33/106 + Installing : cairo-1.15.12-6.el8.x86_64 34/106 + Installing : libXi-1.7.10-1.el8.x86_64 35/106 + Installing : libXfixes-5.0.3-7.el8.x86_64 36/106 + Installing : libXtst-1.2.3-7.el8.x86_64 37/106 + Installing : libXcomposite-0.4.4-14.el8.x86_64 38/106 + Installing : at-spi2-core-2.28.0-1.el8.x86_64 39/106 + Running scriptlet: at-spi2-core-2.28.0-1.el8.x86_64 39/106 + Installing : at-spi2-atk-2.26.2-1.el8.x86_64 40/106 + Running scriptlet: at-spi2-atk-2.26.2-1.el8.x86_64 40/106 + Installing : libXcursor-1.1.15-3.el8.x86_64 41/106 + Installing : libXdamage-1.1.4-14.el8.x86_64 42/106 + Installing : cairo-gobject-1.15.12-6.el8.x86_64 43/106 + Installing : libXft-2.3.3-1.el8.x86_64 44/106 + Installing : libXrandr-1.5.2-1.el8.x86_64 45/106 + Installing : libXinerama-1.1.4-1.el8.x86_64 46/106 + Installing : lcms2-2.9-2.el8.x86_64 47/106 + Running scriptlet: lcms2-2.9-2.el8.x86_64 47/106 + Installing : jbigkit-libs-2.1-14.el8.x86_64 48/106 + Running scriptlet: jbigkit-libs-2.1-14.el8.x86_64 48/106 + Installing : libtiff-4.0.9-32.el8_10.x86_64 49/106 + Installing : javapackages-filesystem-5.3.0-1.module+el8+5136+ 50/106 + Installing : hicolor-icon-theme-0.17-2.el8.noarch 51/106 + Installing : graphite2-1.3.10-10.el8.x86_64 52/106 + Installing : harfbuzz-1.7.5-4.el8.x86_64 53/106 + Running scriptlet: harfbuzz-1.7.5-4.el8.x86_64 53/106 + Installing : fribidi-1.0.4-9.el8.x86_64 54/106 + Installing : pango-1.42.4-8.el8.x86_64 55/106 + Running scriptlet: pango-1.42.4-8.el8.x86_64 55/106 + Installing : dconf-0.28.0-4.0.1.el8.x86_64 56/106 + Installing : alsa-lib-1.2.10-2.el8.x86_64 57/106 + Running scriptlet: alsa-lib-1.2.10-2.el8.x86_64 57/106 + Installing : adwaita-cursor-theme-3.28.0-3.el8.noarch 58/106 + Installing : adwaita-icon-theme-3.28.0-3.el8.noarch 59/106 + Installing : abattis-cantarell-fonts-0.0.25-6.el8.noarch 60/106 + Installing : xz-5.2.4-4.el8_6.x86_64 61/106 + Installing : shared-mime-info-1.9-4.el8.x86_64 62/106 + Running scriptlet: shared-mime-info-1.9-4.el8.x86_64 62/106 + Installing : gdk-pixbuf2-2.36.12-6.el8_10.x86_64 63/106 + Running scriptlet: gdk-pixbuf2-2.36.12-6.el8_10.x86_64 63/106 + Installing : gdk-pixbuf2-modules-2.36.12-6.el8_10.x86_64 64/106 + Installing : gtk-update-icon-cache-3.22.30-11.el8.x86_64 65/106 + Installing : pkgconf-m4-1.4.2-1.el8.noarch 66/106 + Installing : pigz-2.4-4.el8.x86_64 67/106 + Installing : memstrack-0.2.5-2.el8.x86_64 68/106 + Installing : lksctp-tools-1.0.18-3.el8.x86_64 69/106 + Running scriptlet: lksctp-tools-1.0.18-3.el8.x86_64 69/106 + Installing : libpkgconf-1.4.2-1.el8.x86_64 70/106 + Installing : pkgconf-1.4.2-1.el8.x86_64 71/106 + Installing : pkgconf-pkg-config-1.4.2-1.el8.x86_64 72/106 + Installing : xorg-x11-font-utils-1:7.5-41.el8.x86_64 73/106 + Installing : xorg-x11-fonts-Type1-7.5-19.el8.noarch 74/106 + Running scriptlet: xorg-x11-fonts-Type1-7.5-19.el8.noarch 74/106 + Installing : libmodman-2.0.1-17.el8.x86_64 75/106 + Running scriptlet: libmodman-2.0.1-17.el8.x86_64 75/106 + Installing : libproxy-0.4.15-5.2.el8.x86_64 76/106 + Running scriptlet: libproxy-0.4.15-5.2.el8.x86_64 76/106 + Installing : libkcapi-1.4.0-2.0.1.el8.x86_64 77/106 + Installing : libkcapi-hmaccalc-1.4.0-2.0.1.el8.x86_64 78/106 + Installing : libgusb-0.3.0-1.el8.x86_64 79/106 + Installing : colord-libs-1.4.2-1.el8.x86_64 80/106 + Installing : kbd-misc-2.0.4-11.el8.noarch 81/106 + Installing : kbd-legacy-2.0.4-11.el8.noarch 82/106 + Installing : kbd-2.0.4-11.el8.x86_64 83/106 + Installing : systemd-udev-239-78.0.4.el8.x86_64 84/106 + Running scriptlet: systemd-udev-239-78.0.4.el8.x86_64 84/106 + Installing : os-prober-1.74-9.0.1.el8.x86_64 85/106 + Installing : json-glib-1.4.4-1.el8.x86_64 86/106 + Installing : hardlink-1:1.3-6.el8.x86_64 87/106 + Installing : file-5.33-25.el8.x86_64 88/106 + Installing : dejavu-sans-mono-fonts-2.35-7.el8.noarch 89/106 + Installing : gsettings-desktop-schemas-3.32.0-6.el8.x86_64 90/106 + Installing : glib-networking-2.56.1-1.1.el8.x86_64 91/106 + Installing : libsoup-2.62.3-5.el8.x86_64 92/106 + Installing : rest-0.8.1-2.el8.x86_64 93/106 + Running scriptlet: rest-0.8.1-2.el8.x86_64 93/106 + Installing : cpio-2.12-11.el8.x86_64 94/106 + Installing : dracut-049-233.git20240115.0.1.el8.x86_64 95/106 + Running scriptlet: grub2-tools-1:2.02-156.0.2.el8.x86_64 96/106 + Installing : grub2-tools-1:2.02-156.0.2.el8.x86_64 96/106 + Running scriptlet: grub2-tools-1:2.02-156.0.2.el8.x86_64 96/106 + Installing : grubby-8.40-49.0.2.el8.x86_64 97/106 + Installing : crypto-policies-scripts-20230731-1.git3177e06.el 98/106 + Installing : nss-sysinit-3.90.0-7.el8_10.x86_64 99/106 + Installing : nss-3.90.0-7.el8_10.x86_64 100/106 + Installing : avahi-libs-0.7-27.el8.x86_64 101/106 + Installing : cups-libs-1:2.2.6-60.el8_10.x86_64 102/106 + Installing : java-11-openjdk-headless-1:11.0.24.0.8-3.0.1.el8 103/106 + Running scriptlet: java-11-openjdk-headless-1:11.0.24.0.8-3.0.1.el8 103/106 + Installing : gtk3-3.22.30-11.el8.x86_64 104/106 + Installing : java-11-openjdk-1:11.0.24.0.8-3.0.1.el8.x86_64 105/106 + Running scriptlet: java-11-openjdk-1:11.0.24.0.8-3.0.1.el8.x86_64 105/106 + Installing : java-11-openjdk-devel-1:11.0.24.0.8-3.0.1.el8.x8 106/106 + Running scriptlet: java-11-openjdk-devel-1:11.0.24.0.8-3.0.1.el8.x8 106/106 + Running scriptlet: copy-jdk-configs-4.0-2.el8.noarch 106/106 + Running scriptlet: dconf-0.28.0-4.0.1.el8.x86_64 106/106 + Running scriptlet: crypto-policies-scripts-20230731-1.git3177e06.el 106/106 + Running scriptlet: nss-3.90.0-7.el8_10.x86_64 106/106 + Running scriptlet: java-11-openjdk-headless-1:11.0.24.0.8-3.0.1.el8 106/106 + Running scriptlet: java-11-openjdk-1:11.0.24.0.8-3.0.1.el8.x86_64 106/106 + Running scriptlet: java-11-openjdk-devel-1:11.0.24.0.8-3.0.1.el8.x8 106/106 + Running scriptlet: hicolor-icon-theme-0.17-2.el8.noarch 106/106 + Running scriptlet: adwaita-icon-theme-3.28.0-3.el8.noarch 106/106 + Running scriptlet: shared-mime-info-1.9-4.el8.x86_64 106/106 + Running scriptlet: gdk-pixbuf2-2.36.12-6.el8_10.x86_64 106/106 + Running scriptlet: systemd-udev-239-78.0.4.el8.x86_64 106/106 + Verifying : avahi-libs-0.7-27.el8.x86_64 1/106 + Verifying : cpio-2.12-11.el8.x86_64 2/106 + Verifying : crypto-policies-scripts-20230731-1.git3177e06.el 3/106 + Verifying : cups-libs-1:2.2.6-60.el8_10.x86_64 4/106 + Verifying : dejavu-sans-mono-fonts-2.35-7.el8.noarch 5/106 + Verifying : dracut-049-233.git20240115.0.1.el8.x86_64 6/106 + Verifying : file-5.33-25.el8.x86_64 7/106 + Verifying : gdk-pixbuf2-2.36.12-6.el8_10.x86_64 8/106 + Verifying : gettext-0.19.8.1-17.el8.x86_64 9/106 + Verifying : gettext-libs-0.19.8.1-17.el8.x86_64 10/106 + Verifying : glib-networking-2.56.1-1.1.el8.x86_64 11/106 + Verifying : grub2-common-1:2.02-156.0.2.el8.noarch 12/106 + Verifying : grub2-tools-1:2.02-156.0.2.el8.x86_64 13/106 + Verifying : grub2-tools-minimal-1:2.02-156.0.2.el8.x86_64 14/106 + Verifying : grubby-8.40-49.0.2.el8.x86_64 15/106 + Verifying : gsettings-desktop-schemas-3.32.0-6.el8.x86_64 16/106 + Verifying : hardlink-1:1.3-6.el8.x86_64 17/106 + Verifying : json-glib-1.4.4-1.el8.x86_64 18/106 + Verifying : kbd-2.0.4-11.el8.x86_64 19/106 + Verifying : kbd-legacy-2.0.4-11.el8.noarch 20/106 + Verifying : kbd-misc-2.0.4-11.el8.noarch 21/106 + Verifying : libcroco-0.6.12-4.el8_2.1.x86_64 22/106 + Verifying : libgomp-8.5.0-22.0.1.el8_10.x86_64 23/106 + Verifying : libgusb-0.3.0-1.el8.x86_64 24/106 + Verifying : libkcapi-1.4.0-2.0.1.el8.x86_64 25/106 + Verifying : libkcapi-hmaccalc-1.4.0-2.0.1.el8.x86_64 26/106 + Verifying : libmodman-2.0.1-17.el8.x86_64 27/106 + Verifying : libpkgconf-1.4.2-1.el8.x86_64 28/106 + Verifying : libproxy-0.4.15-5.2.el8.x86_64 29/106 + Verifying : libsoup-2.62.3-5.el8.x86_64 30/106 + Verifying : lksctp-tools-1.0.18-3.el8.x86_64 31/106 + Verifying : memstrack-0.2.5-2.el8.x86_64 32/106 + Verifying : os-prober-1.74-9.0.1.el8.x86_64 33/106 + Verifying : pigz-2.4-4.el8.x86_64 34/106 + Verifying : pkgconf-1.4.2-1.el8.x86_64 35/106 + Verifying : pkgconf-m4-1.4.2-1.el8.noarch 36/106 + Verifying : pkgconf-pkg-config-1.4.2-1.el8.x86_64 37/106 + Verifying : shared-mime-info-1.9-4.el8.x86_64 38/106 + Verifying : systemd-udev-239-78.0.4.el8.x86_64 39/106 + Verifying : xz-5.2.4-4.el8_6.x86_64 40/106 + Verifying : abattis-cantarell-fonts-0.0.25-6.el8.noarch 41/106 + Verifying : adwaita-cursor-theme-3.28.0-3.el8.noarch 42/106 + Verifying : adwaita-icon-theme-3.28.0-3.el8.noarch 43/106 + Verifying : alsa-lib-1.2.10-2.el8.x86_64 44/106 + Verifying : at-spi2-atk-2.26.2-1.el8.x86_64 45/106 + Verifying : at-spi2-core-2.28.0-1.el8.x86_64 46/106 + Verifying : atk-2.28.1-1.el8.x86_64 47/106 + Verifying : cairo-1.15.12-6.el8.x86_64 48/106 + Verifying : cairo-gobject-1.15.12-6.el8.x86_64 49/106 + Verifying : colord-libs-1.4.2-1.el8.x86_64 50/106 + Verifying : copy-jdk-configs-4.0-2.el8.noarch 51/106 + Verifying : dconf-0.28.0-4.0.1.el8.x86_64 52/106 + Verifying : fribidi-1.0.4-9.el8.x86_64 53/106 + Verifying : gdk-pixbuf2-modules-2.36.12-6.el8_10.x86_64 54/106 + Verifying : graphite2-1.3.10-10.el8.x86_64 55/106 + Verifying : gtk-update-icon-cache-3.22.30-11.el8.x86_64 56/106 + Verifying : gtk3-3.22.30-11.el8.x86_64 57/106 + Verifying : harfbuzz-1.7.5-4.el8.x86_64 58/106 + Verifying : hicolor-icon-theme-0.17-2.el8.noarch 59/106 + Verifying : jasper-libs-2.0.14-5.el8.x86_64 60/106 + Verifying : java-11-openjdk-1:11.0.24.0.8-3.0.1.el8.x86_64 61/106 + Verifying : java-11-openjdk-devel-1:11.0.24.0.8-3.0.1.el8.x8 62/106 + Verifying : java-11-openjdk-headless-1:11.0.24.0.8-3.0.1.el8 63/106 + Verifying : javapackages-filesystem-5.3.0-1.module+el8+5136+ 64/106 + Verifying : jbigkit-libs-2.1-14.el8.x86_64 65/106 + Verifying : lcms2-2.9-2.el8.x86_64 66/106 + Verifying : libX11-1.6.8-8.el8.x86_64 67/106 + Verifying : libX11-common-1.6.8-8.el8.noarch 68/106 + Verifying : libXau-1.0.9-3.el8.x86_64 69/106 + Verifying : libXcomposite-0.4.4-14.el8.x86_64 70/106 + Verifying : libXcursor-1.1.15-3.el8.x86_64 71/106 + Verifying : libXdamage-1.1.4-14.el8.x86_64 72/106 + Verifying : libXext-1.3.4-1.el8.x86_64 73/106 + Verifying : libXfixes-5.0.3-7.el8.x86_64 74/106 + Verifying : libXft-2.3.3-1.el8.x86_64 75/106 + Verifying : libXi-1.7.10-1.el8.x86_64 76/106 + Verifying : libXinerama-1.1.4-1.el8.x86_64 77/106 + Verifying : libXrandr-1.5.2-1.el8.x86_64 78/106 + Verifying : libXrender-0.9.10-7.el8.x86_64 79/106 + Verifying : libXtst-1.2.3-7.el8.x86_64 80/106 + Verifying : libdatrie-0.2.9-7.el8.x86_64 81/106 + Verifying : libepoxy-1.5.8-1.el8.x86_64 82/106 + Verifying : libfontenc-1.1.3-8.el8.x86_64 83/106 + Verifying : libjpeg-turbo-1.5.3-12.el8.x86_64 84/106 + Verifying : libthai-0.1.27-2.el8.x86_64 85/106 + Verifying : libtiff-4.0.9-32.el8_10.x86_64 86/106 + Verifying : libwayland-client-1.21.0-1.el8.x86_64 87/106 + Verifying : libwayland-cursor-1.21.0-1.el8.x86_64 88/106 + Verifying : libwayland-egl-1.21.0-1.el8.x86_64 89/106 + Verifying : libxcb-1.13.1-1.el8.x86_64 90/106 + Verifying : libxkbcommon-0.9.1-1.el8.x86_64 91/106 + Verifying : lua-5.3.4-12.el8.x86_64 92/106 + Verifying : nspr-4.35.0-1.el8_8.x86_64 93/106 + Verifying : nss-3.90.0-7.el8_10.x86_64 94/106 + Verifying : nss-softokn-3.90.0-7.el8_10.x86_64 95/106 + Verifying : nss-softokn-freebl-3.90.0-7.el8_10.x86_64 96/106 + Verifying : nss-sysinit-3.90.0-7.el8_10.x86_64 97/106 + Verifying : nss-util-3.90.0-7.el8_10.x86_64 98/106 + Verifying : pango-1.42.4-8.el8.x86_64 99/106 + Verifying : pixman-0.38.4-4.el8.x86_64 100/106 + Verifying : rest-0.8.1-2.el8.x86_64 101/106 + Verifying : ttmkfdir-3.0.9-54.el8.x86_64 102/106 + Verifying : tzdata-java-2024a-1.0.1.el8.noarch 103/106 + Verifying : xkeyboard-config-2.28-1.el8.noarch 104/106 + Verifying : xorg-x11-font-utils-1:7.5-41.el8.x86_64 105/106 + Verifying : xorg-x11-fonts-Type1-7.5-19.el8.noarch 106/106 + +Installed: + abattis-cantarell-fonts-0.0.25-6.el8.noarch + adwaita-cursor-theme-3.28.0-3.el8.noarch + adwaita-icon-theme-3.28.0-3.el8.noarch + alsa-lib-1.2.10-2.el8.x86_64 + at-spi2-atk-2.26.2-1.el8.x86_64 + at-spi2-core-2.28.0-1.el8.x86_64 + atk-2.28.1-1.el8.x86_64 + avahi-libs-0.7-27.el8.x86_64 + cairo-1.15.12-6.el8.x86_64 + cairo-gobject-1.15.12-6.el8.x86_64 + colord-libs-1.4.2-1.el8.x86_64 + copy-jdk-configs-4.0-2.el8.noarch + cpio-2.12-11.el8.x86_64 + crypto-policies-scripts-20230731-1.git3177e06.el8.noarch + cups-libs-1:2.2.6-60.el8_10.x86_64 + dconf-0.28.0-4.0.1.el8.x86_64 + dejavu-sans-mono-fonts-2.35-7.el8.noarch + dracut-049-233.git20240115.0.1.el8.x86_64 + file-5.33-25.el8.x86_64 + fribidi-1.0.4-9.el8.x86_64 + gdk-pixbuf2-2.36.12-6.el8_10.x86_64 + gdk-pixbuf2-modules-2.36.12-6.el8_10.x86_64 + gettext-0.19.8.1-17.el8.x86_64 + gettext-libs-0.19.8.1-17.el8.x86_64 + glib-networking-2.56.1-1.1.el8.x86_64 + graphite2-1.3.10-10.el8.x86_64 + grub2-common-1:2.02-156.0.2.el8.noarch + grub2-tools-1:2.02-156.0.2.el8.x86_64 + grub2-tools-minimal-1:2.02-156.0.2.el8.x86_64 + grubby-8.40-49.0.2.el8.x86_64 + gsettings-desktop-schemas-3.32.0-6.el8.x86_64 + gtk-update-icon-cache-3.22.30-11.el8.x86_64 + gtk3-3.22.30-11.el8.x86_64 + hardlink-1:1.3-6.el8.x86_64 + harfbuzz-1.7.5-4.el8.x86_64 + hicolor-icon-theme-0.17-2.el8.noarch + jasper-libs-2.0.14-5.el8.x86_64 + java-11-openjdk-1:11.0.24.0.8-3.0.1.el8.x86_64 + java-11-openjdk-devel-1:11.0.24.0.8-3.0.1.el8.x86_64 + java-11-openjdk-headless-1:11.0.24.0.8-3.0.1.el8.x86_64 + javapackages-filesystem-5.3.0-1.module+el8+5136+7ff78f74.noarch + jbigkit-libs-2.1-14.el8.x86_64 + json-glib-1.4.4-1.el8.x86_64 + kbd-2.0.4-11.el8.x86_64 + kbd-legacy-2.0.4-11.el8.noarch + kbd-misc-2.0.4-11.el8.noarch + lcms2-2.9-2.el8.x86_64 + libX11-1.6.8-8.el8.x86_64 + libX11-common-1.6.8-8.el8.noarch + libXau-1.0.9-3.el8.x86_64 + libXcomposite-0.4.4-14.el8.x86_64 + libXcursor-1.1.15-3.el8.x86_64 + libXdamage-1.1.4-14.el8.x86_64 + libXext-1.3.4-1.el8.x86_64 + libXfixes-5.0.3-7.el8.x86_64 + libXft-2.3.3-1.el8.x86_64 + libXi-1.7.10-1.el8.x86_64 + libXinerama-1.1.4-1.el8.x86_64 + libXrandr-1.5.2-1.el8.x86_64 + libXrender-0.9.10-7.el8.x86_64 + libXtst-1.2.3-7.el8.x86_64 + libcroco-0.6.12-4.el8_2.1.x86_64 + libdatrie-0.2.9-7.el8.x86_64 + libepoxy-1.5.8-1.el8.x86_64 + libfontenc-1.1.3-8.el8.x86_64 + libgomp-8.5.0-22.0.1.el8_10.x86_64 + libgusb-0.3.0-1.el8.x86_64 + libjpeg-turbo-1.5.3-12.el8.x86_64 + libkcapi-1.4.0-2.0.1.el8.x86_64 + libkcapi-hmaccalc-1.4.0-2.0.1.el8.x86_64 + libmodman-2.0.1-17.el8.x86_64 + libpkgconf-1.4.2-1.el8.x86_64 + libproxy-0.4.15-5.2.el8.x86_64 + libsoup-2.62.3-5.el8.x86_64 + libthai-0.1.27-2.el8.x86_64 + libtiff-4.0.9-32.el8_10.x86_64 + libwayland-client-1.21.0-1.el8.x86_64 + libwayland-cursor-1.21.0-1.el8.x86_64 + libwayland-egl-1.21.0-1.el8.x86_64 + libxcb-1.13.1-1.el8.x86_64 + libxkbcommon-0.9.1-1.el8.x86_64 + lksctp-tools-1.0.18-3.el8.x86_64 + lua-5.3.4-12.el8.x86_64 + memstrack-0.2.5-2.el8.x86_64 + nspr-4.35.0-1.el8_8.x86_64 + nss-3.90.0-7.el8_10.x86_64 + nss-softokn-3.90.0-7.el8_10.x86_64 + nss-softokn-freebl-3.90.0-7.el8_10.x86_64 + nss-sysinit-3.90.0-7.el8_10.x86_64 + nss-util-3.90.0-7.el8_10.x86_64 + os-prober-1.74-9.0.1.el8.x86_64 + pango-1.42.4-8.el8.x86_64 + pigz-2.4-4.el8.x86_64 + pixman-0.38.4-4.el8.x86_64 + pkgconf-1.4.2-1.el8.x86_64 + pkgconf-m4-1.4.2-1.el8.noarch + pkgconf-pkg-config-1.4.2-1.el8.x86_64 + rest-0.8.1-2.el8.x86_64 + shared-mime-info-1.9-4.el8.x86_64 + systemd-udev-239-78.0.4.el8.x86_64 + ttmkfdir-3.0.9-54.el8.x86_64 + tzdata-java-2024a-1.0.1.el8.noarch + xkeyboard-config-2.28-1.el8.noarch + xorg-x11-font-utils-1:7.5-41.el8.x86_64 + xorg-x11-fonts-Type1-7.5-19.el8.noarch + xz-5.2.4-4.el8_6.x86_64 + +Complete! +Last metadata expiration check: 0:00:23 ago on Tue 20 Aug 2024 08:55:14 AM UTC. +Package iproute-6.2.0-5.el8_9.x86_64 is already installed. +Dependencies resolved. +================================================================================ + Package Architecture Version Repository Size +================================================================================ +Upgrading: + iproute x86_64 6.2.0-6.el8_10 ol8_baseos_latest 853 k + +Transaction Summary +================================================================================ +Upgrade 1 Package + +Total download size: 853 k +Downloading Packages: +iproute-6.2.0-6.el8_10.x86_64.rpm 4.2 MB/s | 853 kB 00:00 +-------------------------------------------------------------------------------- +Total 4.2 MB/s | 853 kB 00:00 +Running transaction check +Transaction check succeeded. +Running transaction test +Transaction test succeeded. +Running transaction + Preparing : 1/1 + Upgrading : iproute-6.2.0-6.el8_10.x86_64 1/2 + Cleanup : iproute-6.2.0-5.el8_9.x86_64 2/2 + Running scriptlet: iproute-6.2.0-5.el8_9.x86_64 2/2 + Verifying : iproute-6.2.0-6.el8_10.x86_64 1/2 + Verifying : iproute-6.2.0-5.el8_9.x86_64 2/2 + +Upgraded: + iproute-6.2.0-6.el8_10.x86_64 + +Complete! +24 files removed +Removing intermediate container fe168b01f3ad + ---> 791878694a50 +Step 5/12 : RUN curl -o /tmp/ords-$ORDSVERSION.el8.noarch.rpm https://yum.oracle.com/repo/OracleLinux/OL8/oracle/software/x86_64/getPackage/ords-$ORDSVERSION.el8.noarch.rpm + ---> Running in 59d7143da358 + % Total % Received % Xferd Average Speed Time Time Time Current + Dload Upload Total Spent Left Speed +100 108M 100 108M 0 0 1440k 0 0:01:16 0:01:16 --:--:-- 1578k +Removing intermediate container 59d7143da358 + ---> 17c4534293e5 +Step 6/12 : RUN rpm -ivh /tmp/ords-$ORDSVERSION.el8.noarch.rpm + ---> Running in 84b1cbffdc51 +Verifying... ######################################## +Preparing... ######################################## +Updating / installing... +ords-23.4.0-8.el8 ######################################## +INFO: Before starting ORDS service, run the below command as user oracle: + ords --config /etc/ords/config install +Removing intermediate container 84b1cbffdc51 + ---> 6e7151b79588 +Step 7/12 : RUN mkdir -p $ORDS_HOME/doc_root && mkdir -p $ORDS_HOME/error && mkdir -p $ORDS_HOME/secrets && chmod ug+x $ORDS_HOME/*.sh && groupadd -g 54322 dba && usermod -u 54321 -d /home/oracle -g dba -m -s /bin/bash oracle && chown -R oracle:dba $ORDS_HOME && echo "oracle ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers + ---> Running in 66e5db5f343f +Removing intermediate container 66e5db5f343f + ---> 0523dc897bf4 +Step 8/12 : USER oracle + ---> Running in ffda8495ac77 +Removing intermediate container ffda8495ac77 + ---> 162acd4d0b93 +Step 9/12 : WORKDIR /home/oracle + ---> Running in 8c14310ffbc7 +Removing intermediate container 8c14310ffbc7 + ---> c8dae809e772 +Step 10/12 : VOLUME ["$ORDS_HOME/config/ords"] + ---> Running in ed64548fd997 +Removing intermediate container ed64548fd997 + ---> 22e2c99247b0 +Step 11/12 : EXPOSE 8888 + ---> Running in 921f7c85d61d +Removing intermediate container 921f7c85d61d + ---> e5d503c92224 +Step 12/12 : CMD $ORDS_HOME/$RUN_FILE + ---> Running in cad487298d63 +Removing intermediate container cad487298d63 + ---> fdb17aa242f8 +Successfully built fdb17aa242f8 +Successfully tagged oracle/ords-dboper:latest +08:57:18 oracle@mitk01:# + diff --git a/docs/multitenant/usecase01/logfiles/ImagePush.log b/docs/multitenant/ords-based/usecase01/logfiles/ImagePush.log similarity index 100% rename from docs/multitenant/usecase01/logfiles/ImagePush.log rename to docs/multitenant/ords-based/usecase01/logfiles/ImagePush.log diff --git a/docs/multitenant/usecase01/logfiles/cdb.log b/docs/multitenant/ords-based/usecase01/logfiles/cdb.log similarity index 100% rename from docs/multitenant/usecase01/logfiles/cdb.log rename to docs/multitenant/ords-based/usecase01/logfiles/cdb.log diff --git a/docs/multitenant/usecase01/logfiles/cdb_creation.log b/docs/multitenant/ords-based/usecase01/logfiles/cdb_creation.log similarity index 100% rename from docs/multitenant/usecase01/logfiles/cdb_creation.log rename to docs/multitenant/ords-based/usecase01/logfiles/cdb_creation.log diff --git a/docs/multitenant/ords-based/usecase01/logfiles/openssl_execution.log b/docs/multitenant/ords-based/usecase01/logfiles/openssl_execution.log new file mode 100644 index 00000000..e3915a21 --- /dev/null +++ b/docs/multitenant/ords-based/usecase01/logfiles/openssl_execution.log @@ -0,0 +1,19 @@ +CREATING TLS CERTIFICATES +/usr/bin/openssl genrsa -out ca.key 2048 +Generating RSA private key, 2048 bit long modulus (2 primes) +......................+++++ +..................................................+++++ +e is 65537 (0x010001) +/usr/bin/openssl req -new -x509 -days 365 -key ca.key -subj "/C=US/ST=California/L=SanFrancisco/O=oracle /CN=cdb-dev-ords.oracle-database-operator-system /CN=localhost Root CA " -out ca.crt +/usr/bin/openssl req -newkey rsa:2048 -nodes -keyout tls.key -subj "/C=US/ST=California/L=SanFrancisco/O=oracle /CN=cdb-dev-ords.oracle-database-operator-system /CN=localhost" -out server.csr +Generating a RSA private key +...........+++++ +...........................................+++++ +writing new private key to 'tls.key' +----- +/usr/bin/echo "subjectAltName=DNS:cdb-dev-ords.oracle-database-operator-system,DNS:www.example.com" > extfile.txt +/usr/bin/openssl x509 -req -extfile extfile.txt -days 365 -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out tls.crt +Signature ok +subject=C = US, ST = California, L = SanFrancisco, O = "oracle ", CN = "cdb-dev-ords.oracle-database-operator-system ", CN = localhost +Getting CA Private Key + diff --git a/docs/multitenant/ords-based/usecase01/logfiles/ordsconfig.log b/docs/multitenant/ords-based/usecase01/logfiles/ordsconfig.log new file mode 100644 index 00000000..b787b752 --- /dev/null +++ b/docs/multitenant/ords-based/usecase01/logfiles/ordsconfig.log @@ -0,0 +1,39 @@ +ORDS: Release 23.4 Production on Tue Aug 20 07:48:44 2024 + +Copyright (c) 2010, 2024, Oracle. + +Configuration: + /etc/ords/config/ + +Database pool: default + +Setting Value Source +----------------------------------------- -------------------------------------------------- ----------- +database.api.enabled true Global +database.api.management.services.disabled false Global +db.cdb.adminUser C##DBAPI_CDB_ADMIN AS SYSDBA Pool +db.cdb.adminUser.password ****** Pool Wallet +db.connectionType customurl Pool +db.customURL jdbc:oracle:thin:@(DESCRIPTION=(CONNECT_TIMEOUT=90 Pool + )(RETRY_COUNT=30)(RETRY_DELAY=10)(TRANSPORT_CONNEC + T_TIMEOUT=70)(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL= + TCP)(HOST=scan12.testrac.com)(PORT=1521)(IP=V4_ONL + Y))(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST= + scan34.testrac.com)(PORT=1521)(IP=V4_ONLY))(CONNEC + T_DATA=(SERVER=DEDICATED)(SERVICE_NAME=TESTORDS))) +db.password ****** Pool Wallet +db.serviceNameSuffix Pool +db.username ORDS_PUBLIC_USER Pool +error.externalPath /opt/oracle/ords/error Global +jdbc.InitialLimit 50 Pool +jdbc.MaxLimit 100 Pool +misc.pagination.maxRows 1000 Pool +plsql.gateway.mode disabled Pool +restEnabledSql.active true Pool +security.requestValidationFunction ords_util.authorize_plsql_gateway Pool +security.verifySSL true Global +standalone.access.log /home/oracle Global +standalone.https.cert /opt/oracle/ords//secrets/tls.crt Global +standalone.https.cert.key /opt/oracle/ords//secrets/tls.key Global +standalone.https.port 8888 Global + diff --git a/docs/multitenant/usecase01/logfiles/tagandpush.log b/docs/multitenant/ords-based/usecase01/logfiles/tagandpush.log similarity index 100% rename from docs/multitenant/usecase01/logfiles/tagandpush.log rename to docs/multitenant/ords-based/usecase01/logfiles/tagandpush.log diff --git a/docs/multitenant/usecase01/logfiles/testapi.log b/docs/multitenant/ords-based/usecase01/logfiles/testapi.log similarity index 100% rename from docs/multitenant/usecase01/logfiles/testapi.log rename to docs/multitenant/ords-based/usecase01/logfiles/testapi.log diff --git a/docs/multitenant/ords-based/usecase01/makefile b/docs/multitenant/ords-based/usecase01/makefile new file mode 100644 index 00000000..ec454e28 --- /dev/null +++ b/docs/multitenant/ords-based/usecase01/makefile @@ -0,0 +1,906 @@ +# Copyright (c) 2022, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# __ __ _ __ _ _ +# | \/ | __ _| | _____ / _(_) | ___ +# | |\/| |/ _` | |/ / _ \ |_| | |/ _ \ +# | | | | (_| | < __/ _| | | __/ +# |_| |_|\__,_|_|\_\___|_| |_|_|\___| +# | | | | ___| |_ __ ___ _ __ +# | |_| |/ _ \ | '_ \ / _ \ '__| +# | _ | __/ | |_) | __/ | +# |_| |_|\___|_| .__/ \___|_| +# |_| +# +# WARNING: Using this makefile helps you to customize yaml +# files. Edit parameters.txt with your enviroment +# informartion and execute the following steps +# +# 1) make operator +# it configures the operator yaml files with the +# watch namelist required by the multitenant controllers +# +# 2) make genyaml +# It automatically creates all the yaml files based on the +# information available in the parameters file +# +# 3) make secrets +# It configure the required secrets necessary to operate +# with pdbs multitenant controllers +# +# 4) make runall01 +# Start a series of operation create open close delete and so on +# +# LIST OF GENERAED YAML FILE +# +# ----------------------------- ---------------------------------- +# oracle-database-operator.yaml : oracle database operator +# cdbnamespace_binding.yaml : role binding for cdbnamespace +# pdbnamespace_binding.yaml : role binding for pdbnamespace +# create_cdb_secret.yaml : create secrets for ords server pod +# create_pdb_secret.yaml : create secrets for pluggable database +# create_ords_pod.yaml : create rest server pod +# create_pdb1_resource.yaml : create first pluggable database +# create_pdb2_resource.yaml : create second pluggable database +# open_pdb1_resource.yaml : open first pluggable database +# open_pdb2_resource.yaml : open second pluggable database +# close_pdb1_resource.yaml : close first pluggable database +# close_pdb2_resource.yaml : close second pluggable database +# clone_pdb_resource.yaml : clone thrid pluggable database +# clone_pdb2_resource.yaml : clone 4th pluggable database +# delete_pdb1_resource.yaml : delete first pluggable database +# delete_pdb2_resource.yaml : delete sencond pluggable database +# delete_pdb3_resource.yaml : delete thrid pluggable database +# unplug_pdb1_resource.yaml : unplug first pluggable database +# plug_pdb1_resource.yaml : plug first pluggable database +# map_pdb1_resource.yaml : map the first pluggable database +# config_map.yam : pdb parameters array +# +DATE := `date "+%y%m%d%H%M%S"` +###################### +# PARAMETER SECTIONS # +###################### + +export PARAMETERS=parameters.txt +export TNSALIAS=$(shell cat $(PARAMETERS) |grep -v ^\#|grep TNSALIAS|cut -d : -f 2) +export ORDPWD=$(shell cat $(PARAMETERS)|grep -v ^\#|grep ORDPWD|cut -d : -f 2) +export SYSPWD=$(shell cat $(PARAMETERS)|grep -v ^\#|grep SYSPWD|cut -d : -f 2) +export WBUSER=$(shell cat $(PARAMETERS)|grep -v ^\#|grep WBUSER|cut -d : -f 2) +export WBPASS=$(shell cat $(PARAMETERS)|grep -v ^\#|grep WBPASS|cut -d : -f 2) +export PDBUSR=$(shell cat $(PARAMETERS)|grep -v ^\#|grep PDBUSR|cut -d : -f 2) +export PDBPWD=$(shell cat $(PARAMETERS)|grep -v ^\#|grep PDBPWD|cut -d : -f 2) +export CDBUSR=$(shell cat $(PARAMETERS)|grep -v ^\#|grep CDBUSR|cut -d : -f 2) +export CDBPWD=$(shell cat $(PARAMETERS)|grep -v ^\#|grep CDBPWD|cut -d : -f 2) +export OPRNAMESPACE=$(shell cat $(PARAMETERS)|grep -v ^\#|grep OPRNAMESPACE|cut -d : -f 2) +export OPRNAMESPACE=$(shell cat $(PARAMETERS)|grep -v ^\#|grep OPRNAMESPACE|cut -d : -f 2) +export ORDSIMG=$(shell cat $(PARAMETERS)|grep -v ^\#|grep ORDSIMG|cut -d : -f 2,3) +export COMPANY=$(shell cat $(PARAMETERS)|grep -v ^\#|grep COMPANY|cut -d : -f 2) +export APIVERSION=$(shell cat $(PARAMETERS)|grep -v ^\#|grep APIVERSION|cut -d : -f 2) +export OPRNAMESPACE=oracle-database-operator-system +export ORACLE_OPERATOR_YAML=../../../../oracle-database-operator.yaml +export TEST_EXEC_TIMEOUT=3m +export IMAGE=oracle/ords-dboper:latest +export ORDSIMGDIR=../../../../ords + +REST_SERVER=ords +SKEY=tls.key +SCRT=tls.crt +CART=ca.crt +PRVKEY=ca.key +PUBKEY=public.pem +COMPANY=oracle +RUNTIME=/usr/bin/podman + +################# +### FILE LIST ### +################# + +export ORDS_POD=create_ords_pod.yaml + +export CDB_SECRETS=create_cdb_secrets.yaml +export PDB_SECRETS=create_pdb_secrets.yaml + +export PDBCRE1=create_pdb1_resource.yaml +export PDBCRE2=create_pdb2_resource.yaml + +export PDBCLOSE1=close_pdb1_resource.yaml +export PDBCLOSE2=close_pdb2_resource.yaml +export PDBCLOSE3=close_pdb3_resource.yaml + +export PDBOPEN1=open_pdb1_resource.yaml +export PDBOPEN2=open_pdb2_resource.yaml +export PDBOPEN3=open_pdb3_resource.yaml + +export PDBCLONE1=clone_pdb1_resource.yaml +export PDBCLONE2=clone_pdb2_resource.yaml + +export PDBDELETE1=delete_pdb1_resource.yaml +export PDBDELETE2=delete_pdb2_resource.yaml +export PDBDELETE3=delete_pdb3_resource.yaml + +export PDBUNPLUG1=unplug_pdb1_resource.yaml +export PDBPLUG1=plug_pdb1_resource.yaml + +export PDBMAP1=map_pdb1_resource.yaml +export PDBMAP2=map_pdb2_resource.yaml +export PDBMAP3=map_pdb3_resource.yaml + +export PDBMAP1=map_pdb1_resource.yaml +export PDBMAP2=map_pdb2_resource.yaml +export PDBMAP3=map_pdb3_resource.yaml + + +##BINARIES +export KUBECTL=/usr/bin/kubectl +OPENSSL=/usr/bin/openssl +ECHO=/usr/bin/echo +RM=/usr/bin/rm +CP=/usr/bin/cp +TAR=/usr/bin/tar +MKDIR=/usr/bin/mkdir +SED=/usr/bin/sed + +define msg +@printf "\033[31;7m%s\033[0m\r" "......................................]" +@printf "\033[31;7m[\xF0\x9F\x91\x89 %s\033[0m\n" $(1) +endef + +check: + $(call msg,"CHECK PARAMETERS") + @printf "TNSALIAS...............:%.60s....\n" $(TNSALIAS) + @printf "ORDPWD.................:%s\n" $(ORDPWD) + @printf "SYSPWD.................:%s\n" $(SYSPWD) + @printf "WBUSER.................:%s\n" $(WBUSER) + @printf "WBPASS.................:%s\n" $(WBPASS) + @printf "PDBUSR.................:%s\n" $(PDBUSR) + @printf "PDBPWD.................:%s\n" $(PDBPWD) + @printf "CDBUSR.................:%s\n" $(CDBUSR) + @printf "CDBPWD.................:%s\n" $(CDBPWD) + @printf "OPRNAMESPACE...........:%s\n" $(OPRNAMESPACE) + @printf "COMPANY................:%s\n" $(COMPANY) + @printf "APIVERSION.............:%s\n" $(APIVERSION) + + +tlscrt: + $(call msg,"TLS GENERATION") + #$(OPENSSL) genrsa -out $(PRVKEY) 2048 + $(OPENSSL) genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -pkeyopt rsa_keygen_pubexp:65537 > $(PRVKEY) + $(OPENSSL) req -new -x509 -days 365 -key $(PRVKEY) \ + -subj "/C=CN/ST=GD/L=SZ/O=$(COMPANY), Inc./CN=$(COMPANY) Root CA" -out ca.crt + $(OPENSSL) req -newkey rsa:2048 -nodes -keyout $(SKEY) -subj \ + "/C=CN/ST=GD/L=SZ/O=$(COMPANY), Inc./CN=cdb-dev-$(REST_SERVER).$(OPRNAMESPACE)" -out server.csr + $(ECHO) "subjectAltName=DNS:cdb-dev-$(REST_SERVER).$(OPRNAMESPACE)" > extfile.txt + $(OPENSSL) x509 -req -extfile extfile.txt -days 365 -in server.csr -CA ca.crt -CAkey $(PRVKEY) -CAcreateserial -out $(SCRT) + $(OPENSSL) rsa -in $(PRVKEY) -outform PEM -pubout -out $(PUBKEY) + +tlssec: + $(call msg,"GENERATE TLS SECRET") + $(KUBECTL) create secret tls db-tls --key="$(SKEY)" --cert="$(SCRT)" -n $(OPRNAMESPACE) + $(KUBECTL) create secret generic db-ca --from-file="$(CART)" -n $(OPRNAMESPACE) + + +delsec: + $(call msg,"CLEAN OLD SECRETS") + $(eval SECRETSP:=$(shell kubectl get secrets -n $(OPRNAMESPACE) -o custom-columns=":metadata.name" --no-headers|grep -v webhook-server-cert) ) + @[ "${SECRETSP}" ] && ( \ + printf "Deleteing secrets in namespace -n $(OPRNAMESPACE)\n") &&\ + ($(KUBECTL) delete secret $(SECRETSP) -n $(OPRNAMESPACE))\ + || ( echo "No screts in namespace $(OPRNAMESPACE)") + + +###### ENCRYPTED SECRETS ###### +export PRVKEY=ca.key +export PUBKEY=public.pem +WBUSERFILE=wbuser.txt +WBPASSFILE=wbpass.txt +CDBUSRFILE=cdbusr.txt +CDBPWDFILE=cdbpwd.txt +SYSPWDFILE=syspwd.txt +ORDPWDFILE=ordpwd.txt +PDBUSRFILE=pdbusr.txt +PDBPWDFILE=pdbpwd.txt + + + +secrets: delsec tlscrt tlssec + $(OPENSSL) rsa -in $(PRVKEY) -outform PEM -pubout -out $(PUBKEY) + $(KUBECTL) create secret generic pubkey --from-file=publicKey=$(PUBKEY) -n $(OPRNAMESPACE) + $(KUBECTL) create secret generic prvkey --from-file=privateKey=$(PRVKEY) -n $(OPRNAMESPACE) + @$(ECHO) $(WBUSER) > $(WBUSERFILE) + @$(ECHO) $(WBPASS) > $(WBPASSFILE) + @$(ECHO) $(CDBPWD) > $(CDBPWDFILE) + @$(ECHO) $(CDBUSR) > $(CDBUSRFILE) + @$(ECHO) $(SYSPWD) > $(SYSPWDFILE) + @$(ECHO) $(ORDPWD) > $(ORDPWDFILE) + @$(ECHO) $(PDBUSR) > $(PDBUSRFILE) + @$(ECHO) $(PDBPWD) > $(PDBPWDFILE) + $(OPENSSL) rsautl -encrypt -pubin -inkey $(PUBKEY) -in $(WBUSERFILE) |base64 > e_$(WBUSERFILE) + $(OPENSSL) rsautl -encrypt -pubin -inkey $(PUBKEY) -in $(WBPASSFILE) |base64 > e_$(WBPASSFILE) + $(OPENSSL) rsautl -encrypt -pubin -inkey $(PUBKEY) -in $(CDBPWDFILE) |base64 > e_$(CDBPWDFILE) + $(OPENSSL) rsautl -encrypt -pubin -inkey $(PUBKEY) -in $(CDBUSRFILE) |base64 > e_$(CDBUSRFILE) + $(OPENSSL) rsautl -encrypt -pubin -inkey $(PUBKEY) -in $(SYSPWDFILE) |base64 > e_$(SYSPWDFILE) + $(OPENSSL) rsautl -encrypt -pubin -inkey $(PUBKEY) -in $(ORDPWDFILE) |base64 > e_$(ORDPWDFILE) + $(OPENSSL) rsautl -encrypt -pubin -inkey $(PUBKEY) -in $(PDBUSRFILE) |base64 > e_$(PDBUSRFILE) + $(OPENSSL) rsautl -encrypt -pubin -inkey $(PUBKEY) -in $(PDBPWDFILE) |base64 > e_$(PDBPWDFILE) + $(KUBECTL) create secret generic wbuser --from-file=e_$(WBUSERFILE) -n $(OPRNAMESPACE) + $(KUBECTL) create secret generic wbpass --from-file=e_$(WBPASSFILE) -n $(OPRNAMESPACE) + $(KUBECTL) create secret generic cdbpwd --from-file=e_$(CDBPWDFILE) -n $(OPRNAMESPACE) + $(KUBECTL) create secret generic cdbusr --from-file=e_$(CDBUSRFILE) -n $(OPRNAMESPACE) + $(KUBECTL) create secret generic syspwd --from-file=e_$(SYSPWDFILE) -n $(OPRNAMESPACE) + $(KUBECTL) create secret generic ordpwd --from-file=e_$(ORDPWDFILE) -n $(OPRNAMESPACE) + $(KUBECTL) create secret generic pdbusr --from-file=e_$(PDBUSRFILE) -n $(OPRNAMESPACE) + $(KUBECTL) create secret generic pdbpwd --from-file=e_$(PDBPWDFILE) -n $(OPRNAMESPACE) + $(RM) $(WBUSERFILE) $(WBPASSFILE) $(CDBPWDFILE) $(CDBUSRFILE) $(SYSPWDFILE) $(ORDPWDFILE) $(PDBUSRFILE) $(PDBPWDFILE) + $(RM) e_$(WBUSERFILE) e_$(WBPASSFILE) e_$(CDBPWDFILE) e_$(CDBUSRFILE) e_$(SYSPWDFILE) e_$(ORDPWDFILE) e_$(PDBUSRFILE) e_$(PDBPWDFILE) + + +### YAML FILE SECTION ### +operator: + $(CP) ${ORACLE_OPERATOR_YAML} . + ${CP} `basename ${ORACLE_OPERATOR_YAML}` `basename ${ORACLE_OPERATOR_YAML}`.ORG + $(SED) -i 's/value: ""/value: $(OPRNAMESPACE)/g' `basename ${ORACLE_OPERATOR_YAML}` + + +define _script00 +cat < authsection01.yaml + sysAdminPwd: + secret: + secretName: "syspwd" + key: "e_syspwd.txt" + ordsPwd: + secret: + secretName: "ordpwd" + key: "e_ordpwd.txt" + cdbAdminUser: + secret: + secretName: "cdbusr" + key: "e_cdbusr.txt" + cdbAdminPwd: + secret: + secretName: "cdbpwd" + key: "e_cdbpwd.txt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + cdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + cdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + cdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" +EOF + +cat< authsection02.yaml + adminName: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminPwd: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" +EOF + + +cat < ${OPRNAMESPACE}_binding.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: oracle-database-operator-oracle-database-operator-manager-rolebinding1 + namespace: ${OPRNAMESPACE} +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: oracle-database-operator-manager-role +subjects: +- kind: ServiceAccount + name: default + namespace: oracle-database-operator-system +EOF + +cat < ${OPRNAMESPACE}_binding.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: oracle-database-operator-oracle-database-operator-manager-rolebinding2 + namespace: ${OPRNAMESPACE} +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: oracle-database-operator-manager-role +subjects: +- kind: ServiceAccount + name: default + namespace: oracle-database-operator-system +EOF + +endef +export script00 = $(value _script00) +secyaml: + @ eval "$$script00" + +#echo ords pod creation +define _script01 +cat < ${ORDS_POD} +apiVersion: database.oracle.com/${APIVERSION} +kind: CDB +metadata: + name: cdb-dev + namespace: oracle-database-operator-system +spec: + cdbName: "DB12" + ordsImage: ${ORDSIMG} + ordsImagePullPolicy: "Always" + dbTnsurl : ${TNSALIAS} + replicas: 1 + deletePdbCascade: true +EOF + +cat authsection01.yaml >> ${ORDS_POD} + +endef +export script01 = $(value _script01) + + +define _script02 + +cat <${PDBCRE1} +apiVersion: database.oracle.com/${APIVERSION} +kind: PDB +metadata: + name: pdb1 + namespace: ${OPRNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${OPRNAMESPACE}" + cdbName: "DB12" + pdbName: "pdbdev" + assertivePdbDeletion: true + fileNameConversions: "NONE" + unlimitedStorage: false + tdeImport: false + totalSize: "2G" + tempSize: "800M" + action: "Create" +EOF + +cat < ${PDBCRE2} +apiVersion: database.oracle.com/${APIVERSION} +kind: PDB +metadata: + name: pdb2 + namespace: ${OPRNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${OPRNAMESPACE}" + cdbName: "DB12" + pdbName: "pdbprd" + assertivePdbDeletion: true + fileNameConversions: "NONE" + unlimitedStorage: false + tdeImport: false + totalSize: "2G" + tempSize: "800M" + action: "Create" +EOF + +cat <${PDBOPEN1} +apiVersion: database.oracle.com/${APIVERSION} +kind: PDB +metadata: + name: pdb1 + namespace: ${OPRNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${OPRNAMESPACE}" + cdbName: "DB12" + pdbName: "pdbdev" + action: "Modify" + pdbState: "OPEN" + modifyOption: "READ WRITE" +EOF + +cat <${PDBOPEN2} +apiVersion: database.oracle.com/${APIVERSION} +kind: PDB +metadata: + name: pdb2 + namespace: ${OPRNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${OPRNAMESPACE}" + cdbName: "DB12" + pdbName: "pdbprd" + action: "Modify" + pdbState: "OPEN" + modifyOption: "READ WRITE" +EOF + +cat <${PDBOPEN3} +apiVersion: database.oracle.com/${APIVERSION} +kind: PDB +metadata: + name: pdb3 + namespace: ${OPRNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${OPRNAMESPACE}" + cdbName: "DB12" + pdbName: "new_clone" + action: "Modify" + pdbState: "OPEN" + modifyOption: "READ WRITE" +EOF + +cat <${PDBCLOSE1} +apiVersion: database.oracle.com/${APIVERSION} +kind: PDB +metadata: + name: pdb1 + namespace: ${OPRNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${OPRNAMESPACE}" + cdbName: "DB12" + pdbName: "pdbdev" + pdbState: "CLOSE" + modifyOption: "IMMEDIATE" + action: "Modify" +EOF + +cat <${PDBCLOSE2} +apiVersion: database.oracle.com/${APIVERSION} +kind: PDB +metadata: + name: pdb2 + namespace: ${OPRNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${OPRNAMESPACE}" + cdbName: "DB12" + pdbName: "pdbprd" + pdbState: "CLOSE" + modifyOption: "IMMEDIATE" + action: "Modify" +EOF + +cat <${PDBCLOSE3} +apiVersion: database.oracle.com/${APIVERSION} +kind: PDB +metadata: + name: pdb3 + namespace: ${OPRNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${OPRNAMESPACE}" + cdbName: "DB12" + pdbName: ""new_clone" + pdbState: "CLOSE" + modifyOption: "IMMEDIATE" + action: "Modify" +EOF + +cat < ${PDBCLONE1} +apiVersion: database.oracle.com/${APIVERSION} +kind: PDB +metadata: + name: pdb3 + namespace: ${OPRNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${OPRNAMESPACE}" + cdbName: "DB12" + pdbName: "new_clone" + srcPdbName: "pdbdev" + fileNameConversions: "NONE" + totalSize: "UNLIMITED" + tempSize: "UNLIMITED" + assertivePdbDeletion: true + action: "Clone" +EOF + +cat < ${PDBCLONE2} +apiVersion: database.oracle.com/${APIVERSION} +kind: PDB +metadata: + name: pdb4 + namespace: ${OPRNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${OPRNAMESPACE}" + cdbName: "DB12" + pdbName: "new_clone2" + srcPdbName: "pdbprd" + fileNameConversions: "NONE" + totalSize: "UNLIMITED" + tempSize: "UNLIMITED" + assertivePdbDeletion: true + action: "Clone" +EOF + + +cat < ${PDBDELETE1} +apiVersion: database.oracle.com/${APIVERSION} +kind: PDB +metadata: + name: pdb1 + namespace: ${OPRNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${OPRNAMESPACE}" + pdbName: "pdbdev" + action: "Delete" + dropAction: "INCLUDING" +EOF + +cat < ${PDBDELETE2} +apiVersion: database.oracle.com/${APIVERSION} +kind: PDB +metadata: + name: pdb2 + namespace: ${OPRNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${OPRNAMESPACE}" + pdbName: "pdbprd" + action: "Delete" + dropAction: "INCLUDING" +EOF + +cat < ${PDBUNPLUG1} +apiVersion: database.oracle.com/${APIVERSION} +kind: PDB +metadata: + name: pdb1 + namespace: ${OPRNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${OPRNAMESPACE}" + cdbName: "DB12" + pdbName: "pdbdev" + xmlFileName: "/tmp/pdb.xml" + action: "Unplug" +EOF + +cat <${PDBPLUG1} +apiVersion: database.oracle.com/${APIVERSION} +kind: PDB +metadata: + name: pdb1 + namespace: ${OPRNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${OPRNAMESPACE}" + cdbName: "DB12" + pdbName: "pdbdev" + xmlFileName: "/tmp/pdb.xml" + action: "plug" + fileNameConversions: "NONE" + sourceFileNameConversions: "NONE" + copyAction: "MOVE" + totalSize: "1G" + tempSize: "100M" + assertivePdbDeletion: true + action: "Plug" +EOF + +cat <${PDBMAP1} +apiVersion: database.oracle.com/${APIVERSION} +kind: PDB +metadata: + name: pdb1 + namespace: ${OPRNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${OPRNAMESPACE}" + cdbName: "DB12" + pdbName: "pdbdev" + assertivePdbDeletion: true + fileNameConversions: "NONE" + totalSize: "1G" + tempSize: "100M" + action: "Map" +EOF + +cat <${PDBMAP2} +apiVersion: database.oracle.com/${APIVERSION} +kind: PDB +metadata: + name: pdb2 + namespace: ${OPRNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${OPRNAMESPACE}" + cdbName: "DB12" + pdbName: "pdbprd" + assertivePdbDeletion: true + fileNameConversions: "NONE" + totalSize: "1G" + tempSize: "100M" + action: "Map" +EOF + + +cat <${PDBMAP3} +apiVersion: database.oracle.com/${APIVERSION} +kind: PDB +metadata: + name: pdb3 + namespace: ${OPRNAMESPACE} + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "${OPRNAMESPACE}" + cdbName: "DB12" + pdbName: "new_clone" + assertivePdbDeletion: true + fileNameConversions: "NONE" + totalSize: "1G" + tempSize: "100M" + action: "Map" +EOF + + +## Auth information +for _file in ${PDBCRE1} ${PDBCRE2} ${PDBOPEN1} ${PDBOPEN2} ${PDBOPEN3} ${PDBCLOSE1} ${PDBCLOSE2} ${PDBCLOSE3} ${PDBCLONE1} ${PDBCLONE2} ${PDBDELETE1} ${PDBDELETE2} ${PDBUNPLUG1} ${PDBPLUG1} ${PDBMAP1} ${PDBMAP2} ${PDBMAP3} +do +ls -ltr ${_file} + cat authsection02.yaml >> ${_file} +done +rm authsection02.yaml +rm authsection01.yaml +endef + +export script02 = $(value _script02) + +genyaml: secyaml + @ eval "$$script01" + @ eval "$$script02" + +cleanyaml: + - $(RM) $(PDBMAP3) $(PDBMAP2) $(PDBMAP1) $(PDBPLUG1) $(PDBUNPLUG1) $(PDBDELETE2) $(PDBDELETE1) $(PDBCLONE2) $(PDBCLONE1) $(PDBCLOSE3) $(PDBCLOSE2) $(PDBCLOSE1) $(PDBOPEN3) $(PDBOPEN2) $(PDBOPEN1) $(PDBCRE2) $(PDBCRE1) $(ORDS_POD) $(CDB_SECRETS) $(PDB_SECRETS) + - $(RM) ${OPRNAMESPACE}_binding.yaml ${OPRNAMESPACE}_binding.yaml + + +cleancrt: + - $(RM) $(SKEY) $(SCRT) $(CART) $(PRVKEY) $(PUBKEY) server.csr extfile.txt ca.srl + + +################# +### PACKAGING ### +################# + +pkg: + - $(RM) -rf /tmp/pkgtestplan + $(MKDIR) /tmp/pkgtestplan + $(CP) -R * /tmp/pkgtestplan + $(CP) ../../../../oracle-database-operator.yaml /tmp/pkgtestplan/ + $(TAR) -C /tmp -cvf ~/pkgtestplan_$(DATE).tar pkgtestplan + +################ +### diag ### +################ + +login: + $(KUBECTL) exec `$(KUBECTL) get pods -n $(OPRNAMESPACE)|grep ords|cut -d ' ' -f 1` -n $(OPRNAMESPACE) -it -- /bin/bash + + +reloadop: + echo "RESTARTING OPERATOR" + $(eval OP1 := $(shell $(KUBECTL) get pods -n $(OPRNAMESPACE)|grep oracle-database-operator-controller|head -1|cut -d ' ' -f 1 )) + $(eval OP2 := $(shell $(KUBECTL) get pods -n $(OPRNAMESPACE)|grep oracle-database-operator-controller|head -2|tail -1|cut -d ' ' -f 1 )) + $(eval OP3 := $(shell $(KUBECTL) get pods -n $(OPRNAMESPACE)|grep oracle-database-operator-controller|tail -1|cut -d ' ' -f 1 )) + $(KUBECTL) get pod $(OP1) -n $(OPRNAMESPACE) -o yaml | kubectl replace --force -f - + $(KUBECTL) get pod $(OP2) -n $(OPRNAMESPACE) -o yaml | kubectl replace --force -f - + $(KUBECTL) get pod $(OP3) -n $(OPRNAMESPACE) -o yaml | kubectl replace --force -f - + + +dump: + @$(eval TMPSP := $(shell date "+%y%m%d%H%M%S" )) + @$(eval DIAGFILE := ./opdmp.$(TMPSP)) + @>$(DIAGFILE) + @echo "OPERATOR DUMP" >> $(DIAGFILE) + @echo "~~~~~~~~~~~~~" >> $(DIAGFILE) + $(KUBECTL) logs pod/`$(KUBECTL) get pods -n $(OPRNAMESPACE)|grep oracle-database-operator-controller|head -1|cut -d ' ' -f 1` -n $(OPRNAMESPACE) >>$(DIAGFILE) + $(KUBECTL) logs pod/`$(KUBECTL) get pods -n $(OPRNAMESPACE)|grep oracle-database-operator-controller|head -2|tail -1 | cut -d ' ' -f 1` -n $(OPRNAMESPACE) >>$(DIAGFILE) + $(KUBECTL) logs pod/`$(KUBECTL) get pods -n $(OPRNAMESPACE)|grep oracle-database-operator-controller|tail -1|cut -d ' ' -f 1` -n $(OPRNAMESPACE) >>$(DIAGFILE) + +####################################################### +#### TEST SECTION #### +####################################################### + +run00: + @$(call msg,"cdb pod creation") + - $(KUBECTL) delete cdb cdb-dev -n $(OPRNAMESPACE) + $(KUBECTL) apply -f $(ORDS_POD) + time $(KUBECTL) wait --for jsonpath='{.status.phase'}="Ready" cdb cdb-dev -n $(OPRNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) + @$(call msg,"cdb pod completed") + $(KUBECTL) get cdb -n $(OPRNAMESPACE) + $(KUBECTL) get pod -n $(OPRNAMESPACE) + +run01.1: + @$(call msg,"pdb pdb1 creation") + $(KUBECTL) apply -f $(PDBCRE1) + time $(KUBECTL) wait --for jsonpath='{.status.phase'}="Ready" pdb pdb1 -n $(OPRNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) + @$(call msg, "pdb pdb1 creation completed") + $(KUBECTL) get pdb pdb1 -n $(OPRNAMESPACE) + +run01.2: + @$(call msg, "pdb pdb2 creation") + $(KUBECTL) apply -f $(PDBCRE2) + $(KUBECTL) wait --for jsonpath='{.status.phase'}="Ready" pdb pdb2 -n $(OPRNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) + @$(call msg, "pdb pdb2 creation completed") + $(KUBECTL) get pdb pdb2 -n $(OPRNAMESPACE) + +run02.1: + @$(call msg, "pdb pdb1 open") + $(KUBECTL) apply -f $(PDBOPEN1) + $(KUBECTL) wait --for jsonpath='{.status.openMode'}="READ WRITE" pdb pdb1 -n $(OPRNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) + @$(call msg, "pdb pdb1 open completed") + $(KUBECTL) get pdb pdb1 -n $(OPRNAMESPACE) + +run02.2: + @$(call msg,"pdb pdb2 open") + $(KUBECTL) apply -f $(PDBOPEN2) + $(KUBECTL) wait --for jsonpath='{.status.openMode'}="READ WRITE" pdb pdb2 -n $(OPRNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) + @$(call msg,"pdb pdb2 open completed") + $(KUBECTL) get pdb pdb2 -n $(OPRNAMESPACE) + + +run03.1: + @$(call msg,"clone pdb1-->pdb3") + $(KUBECTL) apply -f $(PDBCLONE1) + $(KUBECTL) wait --for jsonpath='{.status.phase'}="Ready" pdb pdb3 -n $(OPRNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) + @$(call msg,"clone pdb1-->pdb3 completed") + $(KUBECTL) get pdb pdb3 -n $(OPRNAMESPACE) + + +run03.2: + @$(call msg,"clone pdb2-->pdb4") + $(KUBECTL) apply -f $(PDBCLONE2) + $(KUBECTL) wait --for jsonpath='{.status.phase'}="Ready" pdb pdb4 -n $(OPRNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) + @$(call msg,"clone pdb2-->pdb4 completed") + $(KUBECTL) get pdb pdb3 -n $(OPRNAMESPACE) + + +run04.1: + @$(call msg,"pdb pdb1 close") + $(KUBECTL) apply -f $(PDBCLOSE1) + $(KUBECTL) wait --for jsonpath='{.status.openMode'}="MOUNTED" pdb pdb1 -n $(OPRNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) + @$(call msg, "pdb pdb1 close completed") + $(KUBECTL) get pdb pdb1 -n $(OPRNAMESPACE) + +run04.2: + @$(call msg,"pdb pdb2 close") + $(KUBECTL) apply -f $(PDBCLOSE2) + $(KUBECTL) wait --for jsonpath='{.status.openMode'}="MOUNTED" pdb pdb2 -n $(OPRNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) + @$(call msg,"pdb pdb2 close completed") + $(KUBECTL) get pdb pdb2 -n $(OPRNAMESPACE) + +run05.1: + @$(call msg,"pdb pdb1 unplug") + $(KUBECTL) apply -f $(PDBUNPLUG1) + $(KUBECTL) wait --for=delete pdb pdb1 -n $(OPRNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) + @$(call msg,"pdb pdb1 unplug completed") + +run06.1: + @$(call msg, "pdb pdb1 plug") + $(KUBECTL) apply -f $(PDBPLUG1) + $(KUBECTL) wait --for jsonpath='{.status.phase'}="Ready" pdb pdb1 -n $(OPRNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) + @$(call msg, "pdb pdb1 plug completed") + $(KUBECTL) get pdb pdb1 -n $(OPRNAMESPACE) + +run07.1: + @$(call msg,"pdb pdb1 delete ") + - $(KUBECTL) apply -f $(PDBCLOSE1) + $(KUBECTL) wait --for jsonpath='{.status.openMode'}="MOUNTED" pdb pdb1 -n $(OPRNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) + $(KUBECTL) apply -f $(PDBDELETE1) + $(KUBECTL) wait --for=delete pdb pdb1 -n $(OPRNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) + @$(call msg,"pdb pdb1 delete") + $(KUBECTL) get pdb -n $(OPRNAMESPACE) + +run99.1: + $(KUBECTL) delete cdb cdb-dev -n cdbnamespace + $(KUBECTL) wait --for=delete cdb cdb-dev -n $(OPRNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) + $(KUBECTL) get cdb -n cdbnamespaace + $(KUBECTL) get pdb -n pdbnamespaace + + +## SEQ | ACTION +## ----+---------------- +## 00 | create ords pod +## 01 | create pdb +## 02 | open pdb +## 03 | clone pdb +## 04 | close pdb +## 05 | unpug pdb +## 06 | plug pdb +## 07 | delete pdb (declarative) + + +runall01: run00 run01.1 run01.2 run03.1 run03.2 run04.1 run05.1 run06.1 run02.1 run07.1 + + +###### BUILD ORDS IMAGE ###### + +createimage: + $(RUNTIME) build -t $(IMAGE) $(ORDSIMGDIR) + +createimageproxy: + $(RUNTIME) build -t $(IMAGE) $(ORDSIMGDIR) --build-arg https_proxy=$(HTTPS_PROXY) --build-arg http_proxy=$(HTTP_PROXY) + +tagimage: + @echo "TAG IMAGE" + $(RUNTIME) tag $(IMAGE) $(ORDSIMG) + +push: + $(RUNTIME) push $(ORDSIMG) + + diff --git a/docs/multitenant/ords-based/usecase01/map_pdb1_resource.yaml b/docs/multitenant/ords-based/usecase01/map_pdb1_resource.yaml new file mode 100644 index 00000000..18cb35b1 --- /dev/null +++ b/docs/multitenant/ords-based/usecase01/map_pdb1_resource.yaml @@ -0,0 +1,49 @@ +apiVersion: database.oracle.com/v4 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "oracle-database-operator-system" + cdbName: "DB12" + pdbName: "pdbdev" + assertivePdbDeletion: true + fileNameConversions: "NONE" + totalSize: "1G" + tempSize: "100M" + action: "Map" + adminName: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminPwd: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/ords-based/usecase01/map_pdb2_resource.yaml b/docs/multitenant/ords-based/usecase01/map_pdb2_resource.yaml new file mode 100644 index 00000000..85899597 --- /dev/null +++ b/docs/multitenant/ords-based/usecase01/map_pdb2_resource.yaml @@ -0,0 +1,49 @@ +apiVersion: database.oracle.com/v4 +kind: PDB +metadata: + name: pdb2 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "oracle-database-operator-system" + cdbName: "DB12" + pdbName: "pdbprd" + assertivePdbDeletion: true + fileNameConversions: "NONE" + totalSize: "1G" + tempSize: "100M" + action: "Map" + adminName: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminPwd: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/ords-based/usecase01/map_pdb3_resource.yaml b/docs/multitenant/ords-based/usecase01/map_pdb3_resource.yaml new file mode 100644 index 00000000..9c2c1cd3 --- /dev/null +++ b/docs/multitenant/ords-based/usecase01/map_pdb3_resource.yaml @@ -0,0 +1,49 @@ +apiVersion: database.oracle.com/v4 +kind: PDB +metadata: + name: pdb3 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "oracle-database-operator-system" + cdbName: "DB12" + pdbName: "new_clone" + assertivePdbDeletion: true + fileNameConversions: "NONE" + totalSize: "1G" + tempSize: "100M" + action: "Map" + adminName: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminPwd: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/ords-based/usecase01/open_pdb1_resource.yaml b/docs/multitenant/ords-based/usecase01/open_pdb1_resource.yaml new file mode 100644 index 00000000..63a0a49c --- /dev/null +++ b/docs/multitenant/ords-based/usecase01/open_pdb1_resource.yaml @@ -0,0 +1,47 @@ +apiVersion: database.oracle.com/v4 +kind: PDB +metadata: + name: pdb1 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "oracle-database-operator-system" + cdbName: "DB12" + pdbName: "pdbdev" + action: "Modify" + pdbState: "OPEN" + modifyOption: "READ WRITE" + adminName: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminPwd: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/ords-based/usecase01/open_pdb2_resource.yaml b/docs/multitenant/ords-based/usecase01/open_pdb2_resource.yaml new file mode 100644 index 00000000..8c4eed0d --- /dev/null +++ b/docs/multitenant/ords-based/usecase01/open_pdb2_resource.yaml @@ -0,0 +1,47 @@ +apiVersion: database.oracle.com/v4 +kind: PDB +metadata: + name: pdb2 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "oracle-database-operator-system" + cdbName: "DB12" + pdbName: "pdbprd" + action: "Modify" + pdbState: "OPEN" + modifyOption: "READ WRITE" + adminName: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminPwd: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/ords-based/usecase01/open_pdb3_resource.yaml b/docs/multitenant/ords-based/usecase01/open_pdb3_resource.yaml new file mode 100644 index 00000000..5f0e4b77 --- /dev/null +++ b/docs/multitenant/ords-based/usecase01/open_pdb3_resource.yaml @@ -0,0 +1,47 @@ +apiVersion: database.oracle.com/v4 +kind: PDB +metadata: + name: pdb3 + namespace: oracle-database-operator-system + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "oracle-database-operator-system" + cdbName: "DB12" + pdbName: "new_clone" + action: "Modify" + pdbState: "OPEN" + modifyOption: "READ WRITE" + adminName: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminPwd: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/ords-based/usecase01/oracle-database-operator-system_binding.yaml b/docs/multitenant/ords-based/usecase01/oracle-database-operator-system_binding.yaml new file mode 100644 index 00000000..79e44269 --- /dev/null +++ b/docs/multitenant/ords-based/usecase01/oracle-database-operator-system_binding.yaml @@ -0,0 +1,13 @@ +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: oracle-database-operator-oracle-database-operator-manager-rolebinding2 + namespace: oracle-database-operator-system +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: oracle-database-operator-manager-role +subjects: +- kind: ServiceAccount + name: default + namespace: oracle-database-operator-system diff --git a/docs/multitenant/usecase01/oracle-database-operator.yaml b/docs/multitenant/ords-based/usecase01/oracle-database-operator.yaml similarity index 100% rename from docs/multitenant/usecase01/oracle-database-operator.yaml rename to docs/multitenant/ords-based/usecase01/oracle-database-operator.yaml diff --git a/docs/multitenant/ords-based/usecase01/parameters.txt b/docs/multitenant/ords-based/usecase01/parameters.txt new file mode 100644 index 00000000..0a7b394a --- /dev/null +++ b/docs/multitenant/ords-based/usecase01/parameters.txt @@ -0,0 +1,61 @@ + +######################## +## REST SERVER IMAGE ### +######################## + +ORDSIMG:_your_container_registry/ords-dboper:latest + +############################## +## TNS URL FOR CDB CREATION ## +############################## +TNSALIAS:"T H I S I S J U S T A N E X A M P L E ....(DESCRIPTION=(CONNECT_TIMEOUT=90)(RETRY_COUNT=30)(RETRY_DELAY=10)(TRANSPORT_CONNECT_TIMEOUT=70)(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan12.testrac.com)(PORT=1521)(IP=V4_ONLY))(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan34.testrac.com)(PORT=1521)(IP=V4_ONLY))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=TESTORDS)))" + +########################################### +## ORDS PUBLIC USER ## +########################################### +ORDPWD:Change_me_please + +########################################### +## SYSPASSWORD ## +########################################### +SYSPWD:Change_me_please + +####################### +## HTTPS CREDENTIAL ### +####################### + +WBUSER:Change_me_please +WBPASS:Change_me_please + +##################### +## PDB ADMIN USER ### +##################### + +PDBUSR:Change_me_please +PDBPWD:Change_me_please + +##################### +## CDB ADMIN USER ### +##################### + +CDBUSR:C##DBAPI_CDB_ADMIN +CDBPWD:Change_me_please + +################### +### NAMESPACES #### +################### + +PDBNAMESPACE:pdbnamespace +CDBNAMESPACE:cdbnamespace + +#################### +### COMPANY NAME ### +#################### + +COMPANY:oracle + +#################### +### APIVERSION ### +#################### + +APIVERSION:v4 diff --git a/docs/multitenant/provisioning/singlenamespace/pdb_close.yaml b/docs/multitenant/ords-based/usecase01/pdb_close.yaml similarity index 100% rename from docs/multitenant/provisioning/singlenamespace/pdb_close.yaml rename to docs/multitenant/ords-based/usecase01/pdb_close.yaml diff --git a/docs/multitenant/provisioning/singlenamespace/pdb_create.yaml b/docs/multitenant/ords-based/usecase01/pdb_create.yaml similarity index 100% rename from docs/multitenant/provisioning/singlenamespace/pdb_create.yaml rename to docs/multitenant/ords-based/usecase01/pdb_create.yaml diff --git a/docs/multitenant/provisioning/singlenamespace/pdb_delete.yaml b/docs/multitenant/ords-based/usecase01/pdb_delete.yaml similarity index 100% rename from docs/multitenant/provisioning/singlenamespace/pdb_delete.yaml rename to docs/multitenant/ords-based/usecase01/pdb_delete.yaml diff --git a/docs/multitenant/usecase01/pdb_map.yaml b/docs/multitenant/ords-based/usecase01/pdb_map.yaml similarity index 100% rename from docs/multitenant/usecase01/pdb_map.yaml rename to docs/multitenant/ords-based/usecase01/pdb_map.yaml diff --git a/docs/multitenant/provisioning/singlenamespace/pdb_open.yaml b/docs/multitenant/ords-based/usecase01/pdb_open.yaml similarity index 100% rename from docs/multitenant/provisioning/singlenamespace/pdb_open.yaml rename to docs/multitenant/ords-based/usecase01/pdb_open.yaml diff --git a/docs/multitenant/usecase01/pdb_secret.yaml b/docs/multitenant/ords-based/usecase01/pdb_secret.yaml similarity index 100% rename from docs/multitenant/usecase01/pdb_secret.yaml rename to docs/multitenant/ords-based/usecase01/pdb_secret.yaml diff --git a/docs/multitenant/provisioning/singlenamespace/pdb_plug.yaml b/docs/multitenant/ords-based/usecase01/plug_pdb1_resource.yaml similarity index 63% rename from docs/multitenant/provisioning/singlenamespace/pdb_plug.yaml rename to docs/multitenant/ords-based/usecase01/plug_pdb1_resource.yaml index 77c00b9c..0e86e10c 100644 --- a/docs/multitenant/provisioning/singlenamespace/pdb_plug.yaml +++ b/docs/multitenant/ords-based/usecase01/plug_pdb1_resource.yaml @@ -1,8 +1,4 @@ -# -# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. -# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. -# -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: PDB metadata: name: pdb1 @@ -15,13 +11,22 @@ spec: cdbName: "DB12" pdbName: "pdbdev" xmlFileName: "/tmp/pdb.xml" + action: "plug" fileNameConversions: "NONE" sourceFileNameConversions: "NONE" copyAction: "MOVE" totalSize: "1G" tempSize: "100M" - action: "Plug" assertivePdbDeletion: true + action: "Plug" + adminName: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminPwd: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" pdbTlsKey: secret: secretName: "db-tls" @@ -36,11 +41,13 @@ spec: key: "ca.crt" webServerUser: secret: - secretName: "pdb1-secret" - key: "webserver_user" + secretName: "wbuser" + key: "e_wbuser.txt" webServerPwd: secret: - secretName: "pdb1-secret" - key: "webserver_pwd" - - + secretName: "wbpass" + key: "e_wbpass.txt" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/usecase01/server.csr b/docs/multitenant/ords-based/usecase01/server.csr similarity index 100% rename from docs/multitenant/usecase01/server.csr rename to docs/multitenant/ords-based/usecase01/server.csr diff --git a/docs/multitenant/usecase01/tde_secret.yaml b/docs/multitenant/ords-based/usecase01/tde_secret.yaml similarity index 100% rename from docs/multitenant/usecase01/tde_secret.yaml rename to docs/multitenant/ords-based/usecase01/tde_secret.yaml diff --git a/docs/multitenant/usecase01/tls.crt b/docs/multitenant/ords-based/usecase01/tls.crt similarity index 100% rename from docs/multitenant/usecase01/tls.crt rename to docs/multitenant/ords-based/usecase01/tls.crt diff --git a/docs/multitenant/usecase01/tls.key b/docs/multitenant/ords-based/usecase01/tls.key similarity index 100% rename from docs/multitenant/usecase01/tls.key rename to docs/multitenant/ords-based/usecase01/tls.key diff --git a/docs/multitenant/provisioning/singlenamespace/pdb_unplug.yaml b/docs/multitenant/ords-based/usecase01/unplug_pdb1_resource.yaml similarity index 59% rename from docs/multitenant/provisioning/singlenamespace/pdb_unplug.yaml rename to docs/multitenant/ords-based/usecase01/unplug_pdb1_resource.yaml index 085d337e..61fe915d 100644 --- a/docs/multitenant/provisioning/singlenamespace/pdb_unplug.yaml +++ b/docs/multitenant/ords-based/usecase01/unplug_pdb1_resource.yaml @@ -1,8 +1,4 @@ -# -# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. -# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. -# -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: PDB metadata: name: pdb1 @@ -16,6 +12,14 @@ spec: pdbName: "pdbdev" xmlFileName: "/tmp/pdb.xml" action: "Unplug" + adminName: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminPwd: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" pdbTlsKey: secret: secretName: "db-tls" @@ -30,10 +34,13 @@ spec: key: "ca.crt" webServerUser: secret: - secretName: "pdb1-secret" - key: "webserver_user" + secretName: "wbuser" + key: "e_wbuser.txt" webServerPwd: secret: - secretName: "pdb1-secret" - key: "webserver_pwd" - + secretName: "wbpass" + key: "e_wbpass.txt" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/usecase02/README.md b/docs/multitenant/ords-based/usecase02/README.md similarity index 69% rename from docs/multitenant/usecase02/README.md rename to docs/multitenant/ords-based/usecase02/README.md index c434271f..39978747 100644 --- a/docs/multitenant/usecase02/README.md +++ b/docs/multitenant/ords-based/usecase02/README.md @@ -13,37 +13,7 @@ > ☞ The examples of this folder are based on single namespace **oracle-database-operator-system** -This page explains how to plug and unplug database a pdb; it assumes that you have already configured a pluggable database (see [usecase01](../usecase01/README.md)) -The following table reports the parameters required to configure and use oracle multi tenant controller for pluggable database lifecycle management. - -| yaml file parameters | value | description /ords parameter | -|-------------- |--------------------------- |-------------------------------------------------| -| dbserver | or | [--db-hostname][1] | -| dbTnsurl | | [--db-custom-url/db.customURL][dbtnsurl] | -| port | | [--db-port][2] | -| cdbName | | Container Name | -| name | | Ords podname prefix in cdb.yaml | -| name | | pdb resource in pdb.yaml | -| ordsImage | /ords-dboper:latest|My public container registry | -| pdbName | | Pluggable database name | -| servicename | | [--db-servicename][3] | -| sysadmin_user | | [--admin-user][adminuser] | -| sysadmin_pwd | | [--password-stdin][pwdstdin] | -| cdbadmin_user | | [db.cdb.adminUser][1] | -| cdbadmin_pwd | | [db.cdb.adminUser.password][cdbadminpwd] | -| webserver_user| | [https user][http] NOT A DB USER | -| webserver_pwd | | [http user password][http] | -| ords_pwd | | [ORDS_PUBLIC_USER password][public_user] | -| pdbTlsKey | | [standalone.https.cert.key][key] | -| pdbTlsCrt | | [standalone.https.cert][cr] | -| pdbTlsCat | | certificate authority | -| xmlFileName | | path for the unplug and plug operation | -| srcPdbName | | name of the database to be cloned | -| fileNameConversions | | used for database cloning | -| tdeKeystorePath | | [tdeKeystorePath][tdeKeystorePath] | -| tdeExport | | [tdeExport] | -| tdeSecret | | [tdeSecret][tdeSecret] | -| tdePassword | | [tdeSecret][tdeSecret] | +This page explains how to plug and unplug database a pdb; it assumes that you have already configured a pluggable database (see [usecase01](../usecase01/README.md)). Check yaml parameters in the CRD tables in the main [README](../README.md) file. ```text @@ -127,27 +97,7 @@ spec: pdbName: "pdbdev" xmlFileName: "/tmp/pdbunplug.xml" action: "Unplug" - pdbTlsKey: - secret: - secretName: "db-tls" - key: "tls.key" - pdbTlsCrt: - secret: - secretName: "db-tls" - key: "tls.crt" - pdbTlsCat: - secret: - secretName: "db-ca" - key: "ca.crt" - webServerUser: - secret: - secretName: "pdb1-secret" - key: "webserver_user" - webServerPwd: - secret: - secretName: "pdb1-secret" - key: "webserver_pwd" - + [ secret sections ] ``` Close the pluggable database by applying the following yaml file **pdb_close.yaml** @@ -169,29 +119,10 @@ spec: cdbNamespace: "oracle-database-operator-system" cdbName: "DB12" pdbName: "pdbdev" - adminName: - secret: - secretName: "pdb1-secret" - key: "sysadmin_user" - adminPwd: - secret: - secretName: "pdb1-secret" - key: "sysadmin_pwd" - pdbTlsKey: - secret: - secretName: "db-tls" - key: "tls.key" - pdbTlsCrt: - secret: - secretName: "db-tls" - key: "tls.crt" - pdbTlsCat: - secret: - secretName: "db-ca" - key: "ca.crt" pdbState: "CLOSE" modifyOption: "IMMEDIATE" action: "Modify" + [secret section] ``` ```bash @@ -294,19 +225,7 @@ spec: totalSize: "1G" tempSize: "100M" action: "Plug" - pdbTlsKey: - secret: - secretName: "db-tls" - key: "tls.key" - pdbTlsCrt: - secret: - secretName: "db-tls" - key: "tls.crt" - pdbTlsCat: - secret: - secretName: "db-ca" - key: "ca.crt" - + [secrets section] ``` Apply **pdb_plug.yaml** @@ -390,27 +309,8 @@ spec: fileNameConversions: "NONE" totalSize: "UNLIMITED" tempSize: "UNLIMITED" - adminName: - secret: - secretName: "pdb1-secret" - key: "sysadmin_user" - adminPwd: - secret: - secretName: "pdb1-secret" - key: "sysadmin_pwd" - pdbTlsKey: - secret: - secretName: "db-tls" - key: "tls.key" - pdbTlsCrt: - secret: - secretName: "db-tls" - key: "tls.crt" - pdbTlsCat: - secret: - secretName: "db-ca" - key: "ca.crt" action: "Clone" + [secret section] ``` ```bash @@ -487,7 +387,7 @@ PDBDEV(3):Buffer Cache flush finished: 3 -You can use unplug and plug database with TDE; in order to do that you have to specify a key store path and create new kubernets secret for TDE using the following yaml file. **tde_secrete.yaml**. The procedure to unplug and plug database does not change apply the same file. +You can use unplug and plug database with TDE; in order to do that you have to specify a key store path and create new kubernets secret for TDE using the following yaml file. **tde_secrete.yaml**. ```yaml #tde_secret @@ -498,8 +398,8 @@ metadata: namespace: oracle-database-operator-system type: Opaque data: - tdepassword: "d2VsY29tZTEK" - tdesecret: "bW1hbHZlenoK" + tdepassword: "...." + tdesecret: "...." ``` ```bash @@ -525,7 +425,7 @@ spec: pdbName: "pdbdev" adminName: secret: - secretName: pdb1-secret + secretName: key: "sysadmin_user" adminPwd: secret: @@ -621,29 +521,3 @@ spec: -[1]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/installing-and-configuring-oracle-rest-data-services.html#GUID-E9625FAB-9BC8-468B-9FF9-443C88D76FA1:~:text=Table%202%2D2%20Command%20Options%20for%20Command%2DLine%20Interface%20Installation - -[2]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/installing-and-configuring-oracle-rest-data-services.html#GUID-E9625FAB-9BC8-468B-9FF9-443C88D76FA1:~:text=Table%202%2D2%20Command%20Options%20for%20Command%2DLine%20Interface%20Installation - -[3]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/installing-and-configuring-oracle-rest-data-services.html#GUID-DAA027FA-A4A6-43E1-B8DD-C92B330C2341:~:text=%2D%2Ddb%2Dservicename%20%3Cstring%3E - -[adminuser]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/installing-and-configuring-oracle-rest-data-services.html#GUID-A9AED253-4EEC-4E13-A0C4-B7CE82EC1C22:~:text=Table%202%2D6%20Command%20Options%20for%20Uninstall%20CLI - -[public_user]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/using-multitenant-architecture-oracle-rest-data-services.html#GUID-E64A141A-A71F-4979-8D33-C5F8496D3C19:~:text=Preinstallation%20Tasks%20for%20Oracle%20REST%20Data%20Services%20CDB%20Installation - -[key]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/about-REST-configuration-files.html#GUID-006F916B-8594-4A78-B500-BB85F35C12A0:~:text=standalone.https.cert.key - -[cr]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/about-REST-configuration-files.html#GUID-006F916B-8594-4A78-B500-BB85F35C12A0 - -[cdbadminpwd]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/about-REST-configuration-files.html#GUID-006F916B-8594-4A78-B500-BB85F35C12A0:~:text=Table%20C%2D1%20Oracle%20REST%20Data%20Services%20Configuration%20Settings - - -[pwdstdin]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/installing-and-configuring-oracle-rest-data-services.html#GUID-88479C84-CAC1-4133-A33E-7995A645EC05:~:text=default%20database%20pool.-,2.1.4.1%20Understanding%20Command%20Options%20for%20Command%2DLine%20Interface%20Installation,-Table%202%2D2 - -[http]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/installing-and-configuring-oracle-rest-data-services.html#GUID-BEECC057-A8F5-4EAB-B88E-9828C2809CD8:~:text=Example%3A%20delete%20%5B%2D%2Dglobal%5D-,user%20add,-Add%20a%20user - -[dbtnsurl]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/22.2/ordig/installing-and-configuring-oracle-rest-data-services.html#GUID-A9AED253-4EEC-4E13-A0C4-B7CE82EC1C22 - -[tdeKeystorePath]:https://docs.oracle.com/en/database/oracle/oracle-rest-data-services/21.4/orrst/op-database-pdbs-pdb_name-post.html - -[tdeSecret]:https://docs.oracle.com/en/database/oracle/oracle-database/19/sqlrf/ADMINISTER-KEY-MANAGEMENT.html#GUID-E5B2746F-19DC-4E94-83EC-A6A5C84A3EA9 diff --git a/docs/multitenant/ords-based/usecase02/pdb_clone.yaml b/docs/multitenant/ords-based/usecase02/pdb_clone.yaml new file mode 100644 index 00000000..5723f7c6 --- /dev/null +++ b/docs/multitenant/ords-based/usecase02/pdb_clone.yaml @@ -0,0 +1,50 @@ +apiVersion: database.oracle.com/v4 +kind: PDB +metadata: + name: pdb3 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + cdbName: "DB12" + pdbName: "new_clone" + srcPdbName: "pdbdev" + fileNameConversions: "NONE" + totalSize: "UNLIMITED" + tempSize: "UNLIMITED" + assertivePdbDeletion: true + action: "Clone" + adminName: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminPwd: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/ords-based/usecase02/pdb_plug.yaml b/docs/multitenant/ords-based/usecase02/pdb_plug.yaml new file mode 100644 index 00000000..9eb5ed77 --- /dev/null +++ b/docs/multitenant/ords-based/usecase02/pdb_plug.yaml @@ -0,0 +1,53 @@ +apiVersion: database.oracle.com/v4 +kind: PDB +metadata: + name: pdb1 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + cdbName: "DB12" + pdbName: "pdbdev" + xmlFileName: "/tmp/pdb.xml" + action: "plug" + fileNameConversions: "NONE" + sourceFileNameConversions: "NONE" + copyAction: "MOVE" + totalSize: "1G" + tempSize: "100M" + assertivePdbDeletion: true + action: "Plug" + adminName: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminPwd: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/usecase02/pdb_plugtde.yaml b/docs/multitenant/ords-based/usecase02/pdb_plugtde.yaml similarity index 96% rename from docs/multitenant/usecase02/pdb_plugtde.yaml rename to docs/multitenant/ords-based/usecase02/pdb_plugtde.yaml index 17d84346..995be538 100644 --- a/docs/multitenant/usecase02/pdb_plugtde.yaml +++ b/docs/multitenant/ords-based/usecase02/pdb_plugtde.yaml @@ -3,7 +3,7 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: PDB metadata: name: pdb1 diff --git a/docs/multitenant/ords-based/usecase02/pdb_unplug.yaml b/docs/multitenant/ords-based/usecase02/pdb_unplug.yaml new file mode 100644 index 00000000..0036d5f7 --- /dev/null +++ b/docs/multitenant/ords-based/usecase02/pdb_unplug.yaml @@ -0,0 +1,46 @@ +apiVersion: database.oracle.com/v4 +kind: PDB +metadata: + name: pdb1 + namespace: pdbnamespace + labels: + cdb: cdb-dev +spec: + cdbResName: "cdb-dev" + cdbNamespace: "cdbnamespace" + cdbName: "DB12" + pdbName: "pdbdev" + xmlFileName: "/tmp/pdb.xml" + action: "Unplug" + adminName: + secret: + secretName: "pdbusr" + key: "e_pdbusr.txt" + adminPwd: + secret: + secretName: "pdbpwd" + key: "e_pdbpwd.txt" + pdbTlsKey: + secret: + secretName: "db-tls" + key: "tls.key" + pdbTlsCrt: + secret: + secretName: "db-tls" + key: "tls.crt" + pdbTlsCat: + secret: + secretName: "db-ca" + key: "ca.crt" + webServerUser: + secret: + secretName: "wbuser" + key: "e_wbuser.txt" + webServerPwd: + secret: + secretName: "wbpass" + key: "e_wbpass.txt" + pdbOrdsPrvKey: + secret: + secretName: "prvkey" + key: "privateKey" diff --git a/docs/multitenant/usecase02/pdb_unplugtde.yaml b/docs/multitenant/ords-based/usecase02/pdb_unplugtde.yaml similarity index 96% rename from docs/multitenant/usecase02/pdb_unplugtde.yaml rename to docs/multitenant/ords-based/usecase02/pdb_unplugtde.yaml index 4c26bffe..2eacc5b7 100644 --- a/docs/multitenant/usecase02/pdb_unplugtde.yaml +++ b/docs/multitenant/ords-based/usecase02/pdb_unplugtde.yaml @@ -3,7 +3,7 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 Kind: PDB metadata: name: pdb1 diff --git a/docs/multitenant/usecase02/tde_secret.yaml b/docs/multitenant/usecase02/tde_secret.yaml deleted file mode 100644 index d0186ff2..00000000 --- a/docs/multitenant/usecase02/tde_secret.yaml +++ /dev/null @@ -1,15 +0,0 @@ -# -# Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved. -# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. -# - -apiVersion: v1 -kind: Secret -metadata: - name: tde1-secret - namespace: oracle-database-operator-system -type: Opaque -data: - tdepassword: "[base64 encode value]" - tdesecret: "[base64 encode value]" - diff --git a/docs/observability/README.md b/docs/observability/README.md index 986b1885..5a281c9c 100644 --- a/docs/observability/README.md +++ b/docs/observability/README.md @@ -2,99 +2,155 @@ Oracle Database Operator for Kubernetes (`OraOperator`) includes the Observability controller for Oracle Databases and adds the `DatabaseObserver` CRD, which enables users to observe -Oracle Databases by scraping database metrics using SQL queries. The controller +Oracle Databases by scraping database metrics using SQL queries and observe logs in the Database _alert.log_. The controller automates the deployment and maintenance of the metrics exporter container image, -metrics exporter service and a Prometheus servicemonitor. +metrics exporter service and Prometheus servicemonitor. The following sections explains the configuration and functionality of the controller. * [Prerequisites](#prerequisites) * [The DatabaseObserver Custom Resource Definition](#the-databaseobserver-custom-resource) -* [Configuration of DatabaseObservers](#configuration) + * [Configuration Options](#configuration-options) + * [Resources Managed by the Controller](#resources-managed-by-the-controller) +* [DatabaseObserver Operations](#databaseobserver-operations) * [Create](#create-resource) * [List](#list-resource) * [Get Status](#get-detailed-status) * [Update](#patch-resource) * [Delete](#delete-resource) +* [Configuration Options for Scraping Metrics](#scraping-metrics) + * [Custom Metrics Config](#custom-metrics-config) + * [Prometheus Release](#prometheus-release) +* [Configuration Options for Scraping Logs](#scraping-logs) + * [Custom Log Location with PersistentVolumes](#custom-log-location-with-persistentvolumes) + * [Example Working with Sidecars and Promtail](#working-with-sidecars-to-deploy-promtail) + * [Promtail Config Example](#Promtail-Config-Example) +* [Other Configuration Options](#other-configuration-options) + * [Labels](#labels) + * [Custom Exporter Image or Version](#custom-exporter-image-or-version) * [Mandatory Roles and Privileges](#mandatory-roles-and-privileges-requirements-for-observability-controller) * [Debugging and troubleshooting](#debugging-and-troubleshooting) +* [Known Issues](#known-issues) ## Prerequisites -The `DatabaseObserver` custom resource has the following pre-requisites: +The `DatabaseObserver` custom resource has the following prerequisites: 1. Prometheus and its `servicemonitor` custom resource definition must be installed on the cluster. - The Observability controller creates multiple Kubernetes resources that include - a Prometheus `servicemonitor`. In order for the controller + a Prometheus `servicemonitor`. For the controller to create ServiceMonitors, the ServiceMonitor custom resource must exist. -2. A pre-existing Oracle Database and the proper database grants and privileges. +2. A preexisting Oracle Database and the proper database grants and privileges. - The controller exports metrics through SQL queries that the user can control and specify through a _toml_ file. The necessary access privileges to the tables used in the queries are not provided and applied automatically. -### The DatabaseObserver Custom Resource -The Oracle Database Operator (__v1.1.0__) includes the Oracle Database Observability controller which automates -the deployment and setting up of the Oracle Database metrics exporter and the related resources to make Oracle databases observable. - -In the sample YAML file found in -[./config/samples/observability/databaseobserver.yaml](../../config/samples/observability/databaseobserver.yaml), -the databaseObserver custom resource offers the following properties to be configured: - -| Attribute | Type | Default | Required? | Example | -|-------------------------------------------------------|---------|-----------------|--------------|-----------------------------------------------------------------------| -| `spec.database.dbUser.key` | string | user | Optional | _username_ | -| `spec.database.dbUser.secret` | string | - | Yes | _db-secret_ | -| `spec.database.dbPassword.key` | string | password | Optional | _admin-password_ | -| `spec.database.dbPassword.secret` | string | - | Conditional | _db-secret_ | -| `spec.database.dbPassword.vaultOCID` | string | - | Conditional | _ocid1.vault.oc1..._ | -| `spec.database.dbPassword.vaultSecretName` | string | - | Conditional | _db-vault_ | -| `spec.database.dbWallet.secret` | string | - | Conditional | _devsec-oradevdb-wallet_ | -| `spec.database.dbConnectionString.key` | string | connection | Optional | _connection_ | -| `spec.database.dbConnectionString.secret` | string | - | Yes | _db-secretg_ | -| `spec.exporter.image` | string | - | Optional | _container-registry.oracle.com/database/observability-exporter:1.0.2_ | -| `spec.exporter.configuration.configmap.key` | string | config.toml | Optional | _config.toml_ | -| `spec.exporter.configuration.configmap.configmapName` | string | - | Optional | _devcm-oradevdb-config_ | -| `spec.exporter.service.port` | number | 9161 | Optional | _9161_ | -| `spec.prometheus.port` | string | metrics | Optional | _metrics_ | -| `spec.prometheus.labels` | map | app: obs-{name} | Optional | _app: oradevdb-apps_ | -| `spec.replicas` | number | 1 | Optional | _1_ | -| `spec.ociConfig.configMapName` | string | - | Conditional | _oci-cred_ | -| `spec.ociConfig.secretName` | string | - | Conditional | _oci-privatekey_ | - - - - - -### Configuration -The `databaseObserver` custom resource has the following fields for all configurations that are required: -* `spec.database.dbUser.secret` - secret containing the database username. The corresponding key can be any value but must match the key in the secret provided. -* `spec.database.dbPassword.secret` - secret containing the database password (if vault is NOT used). The corresponding key field can be any value but must match the key in the secret provided -* `spec.database.dbConnectionString.secret` - secret containing the database connection string. The corresponding key field can be any value but must match the key in the secret provided i - -If a database wallet is required to connect, the following field containing the secret is required: -* `spec.database.dbWallet.secret` - secret containing the database wallet. The filenames must be used as the keys - -If vault is used to store the database password instead, the following fields are required: +## The DatabaseObserver Custom Resource +The Oracle Database Operator (__v1.2.0__ or later) includes the Oracle Database Observability controller, which automates +the deployment and setting up of the Oracle Database exporter and the related resources to make Oracle Databases observable. + +In the example YAML file found in +[`./config/samples/observability/v4/databaseobserver.yaml`](../../config/samples/observability/v4/databaseobserver.yaml), +the databaseObserver custom resource provides the following configurable properties: + +| Attribute | Type | Default | Required? | Example | +|--------------------------------------------------------|--------|---------------------------------------------------------------------|:------------|-----------------------------------------------------------------------| +| `spec.database.dbUser.key` | string | user | Optional | _username_ | +| `spec.database.dbUser.secret` | string | - | Yes | _db-secret_ | +| `spec.database.dbPassword.key` | string | password | Optional | _admin-password_ | +| `spec.database.dbPassword.secret` | string | - | Conditional | _db-secret_ | +| `spec.database.dbPassword.vaultOCID` | string | - | Conditional | _ocid1.vault.oc1..._ | +| `spec.database.dbPassword.vaultSecretName` | string | - | Conditional | _db-vault_ | +| `spec.database.dbWallet.secret` | string | - | Conditional | _devsec-oradevdb-wallet_ | +| `spec.database.dbConnectionString.key` | string | connection | Optional | _connection_ | +| `spec.database.dbConnectionString.secret` | string | - | Yes | _db-secretg_ | +| `spec.sidecars` | array | - | Optional | - | +| `spec.sidecarVolumes` | array | - | Optional | - | +| `spec.exporter.deployment.securityContext` | object | | Optional | _ | +| `spec.exporter.deployment.env` | map | - | Optional | _DB_ROLE: "SYSDBA"_ | +| `spec.exporter.deployment.image` | string | container-registry.oracle.com/database/observability-exporter:1.5.1 | Optional | _container-registry.oracle.com/database/observability-exporter:1.3.0_ | +| `spec.exporter.deployment.args` | array | - | Optional | _[ "--log.level=info" ]_ | +| `spec.exporter.deployment.commands` | array | - | Optional | _[ "/oracledb_exporter" ]_ | +| `spec.exporter.deployment.labels` | map | - | Optional | _environment: dev_ | +| `spec.exporter.deployment.podTemplate.labels` | map | - | Optional | _environment: dev_ | +| `spec.exporter.deployment.podTemplate.securityContext` | object | - | Optional | _ | +| `spec.exporter.service.ports` | array | - | Optional | - | +| `spec.exporter.service.labels` | map | - | Optional | _environment: dev_ | | +| `spec.configuration.configMap.key` | string | config.toml | Optional | _config.toml_ | +| `spec.configuration.configMap.name` | string | - | Optional | _devcm-oradevdb-config_ | +| `spec.prometheus.serviceMonitor.labels` | map | - | Yes | _release: prometheus_ | +| `spec.prometheus.serviceMonitor.namespaceSelector` | - | - | Yes | - | +| `spec.prometheus.serviceMonitor.endpoints` | array | - | Optional | - | +| `spec.log.filename` | string | alert.log | Optional | _alert.log_ | +| `spec.log.path` | string | /log | Optional | _/log_ | +| `spec.log.volume.name` | string | log-volume | Optional | _my-persistent-volume_ | +| `spec.log.volume.persistentVolumeClaim.claimName` | string | - | Optional | _my-pvc_ | +| `spec.replicas` | number | 1 | Optional | _1_ | +| `spec.inheritLabels` | array | - | Optional | _- environment: dev_
- app.kubernetes.io/name: observer | +| `spec.ociConfig.configMapName` | string | - | Conditional | _oci-cred_ | +| `spec.ociConfig.secretName` | string | - | Conditional | _oci-privatekey_ | + + +### Configuration Options +The `databaseObserver` Custom resource has the following fields for all configurations that are required: +* `spec.database.dbUser.secret` - Secret containing the database username. The corresponding key can be any value but must match the key in the secret provided. +* `spec.database.dbPassword.secret` - Secret containing the database password (if `vault` is NOT used). The corresponding key field can be any value, but must match the key in the Secret provided +* `spec.database.dbConnectionString.secret` - Secret containing the database connection string. The corresponding key field can be any value but must match the key in the Secret provided +* `spec.prometheus.serviceMonitor.labels` - Custom labels to add to the service monitors labels. A label is required for your serviceMonitor to be discovered. This label must match what is set in the serviceMonitorSelector of your Prometheus configuration + +If a database wallet is required to connect, then the following field containing the wallet secret is required: +* `spec.database.dbWallet.secret` - Secret containing the database wallet. The filenames inside the wallet must be used as keys + +If vault is used to store the database password instead, then the following fields are required: * `spec.database.dbPassword.vaultOCID` - OCID of the vault used * `spec.database.dbPassword.vaultSecretName` - Name of the secret inside the desired vault -* `spec.ociConfig.configMapName` - holds the rest of the information of the OCI API signing key. The following keys must be used: `fingerprint`, `region`, `tenancy` and `user` -* `spec.ociConfig.secretName` - holds the private key of the OCI API signing key. The key to the file containing the user private key must be: `privatekey` - -The `databaseObserver` provides the remaining multiple fields that are optional: -* `spec.prometheus.labels` - labels to use for Service, ServiceMonitor and Deployment -* `spec.prometheus.port` - port to use for ServiceMonitor -* `spec.replicas` - number of replicas to deploy -* `spec.exporter.service.port` - port of service -* `spec.exporter.image` - image version of observability exporter to use - - +* `spec.ociConfig.configMapName` - Holds the rest of the information of the OCI API signing key. The following keys must be used: `fingerprint`, `region`, `tenancy` and `user` +* `spec.ociConfig.secretName` - Holds the private key of the OCI API signing key. The key to the file containing the user private key must be: `privatekey` + +The `databaseObserver` Resource provides the remaining multiple fields that are optional: +* `spec.prometheus.serviceMonitor.endpoints` - ServiceMonitor endpoints +* `spec.prometheus.serviceMonitor.namespaceSelector` - ServiceMonitor namespace selector +* `spec.sidecars` - List of containers to run as a sidecar container with the observability exporter container image +* `spec.sidecarVolumes` - Volumes of any sidecar containers +* `spec.log.path` - Custom path to create +* `spec.log.filename` - Custom filename for the log file +* `spec.log.volume.name` - Custom name for the log volume +* `spec.log.volume.persistentVolumeClaim.claimName` - A volume in which to place the log to be shared by the containers. If not specified, an EmptyDir is used by default. +* `spec.configuration.configMap.key` - Configuration filename inside the container and the configmap +* `spec.configuration.configMap.name` - Name of the `configMap` that holds the custom metrics configuration +* `spec.replicas` - Number of replicas to deploy +* `spec.exporter.service.ports` - Port number for the generated service to use +* `spec.exporter.service.labels` - Custom labels to add to service labels +* `spec.exporter.deployment.image` - Image version of observability exporter to use +* `spec.exporter.deployment.env` - Custom environment variables for the observability exporter +* `spec.exporter.deployment.labels` - Custom labels to add to deployment labels +* `spec.exporter.deployment.podTemplate.labels` - Custom labels to add to pod labels +* `spec.exporter.deployment.podTemplate.securityContext` - Configures pod securityContext +* `spec.exporter.deployment.args` - Additional arguments to provide the observability-exporter +* `spec.exporter.deployment.commands` - Commands to supply to the observability-exporter +* `spec.exporter.deployment.securityContext` - Configures container securityContext +* `spec.inheritLabels` - Keys of inherited labels from the databaseObserver resource. These labels are applied to generated resources. + +### Resources Managed by the Controller +When you create a `DatabaseObserver` resource, the controller creates and manages the following resources: + +1. __Deployment__ - The deployment will have the same name as the `databaseObserver` resource + - Deploys a container named `observability-exporter` + - The default container image version of the `container-registry.oracle.com/database/observability-exporter` supported is __[v1.5.1](https://github.com/oracle/oracle-db-appdev-monitoring/releases/tag/1.5.1)__ + +2. __Service__ - The service will have the same name as the databaseObserver + - The service is of type `ClusterIP` + +3. __Prometheus ServiceMonitor__ - The serviceMonitor will have the same name as the `databaseObserver` + +## DatabaseObserver Operations ### Create Resource -Follow the steps below to create a new databaseObserver resource object. +Follow the steps below to create a new `databaseObserver` resource object. -1. To begin, creating a databaseObserver requires you to create and provide kubernetes Secrets to provide connection details: +1. To begin, creating a `databaseObserver` requires you to create and provide Kubernetes Secrets to provide connection details: ```bash kubectl create secret generic db-secret \ --from-literal=username='username' \ @@ -102,24 +158,23 @@ kubectl create secret generic db-secret \ --from-literal=connection='dbsample_tp' ``` -2. (Conditional) Create a Kubernetes secret for the wallet (if a wallet is required to connect to the database). +2. (Conditional) Create a Kubernetes Secret for the wallet (if a wallet is required to connect to the database). -You can create this secret by using a command similar to the following example below. -If you are connecting to an Autunomous Database and the operator is used to manage the Oracle Autonomous Database, -a client wallet can also be downloaded as a secret through kubectl commands. You can find out how, [here](../../docs/adb/README.md#download-wallets). +You can create this Secret by using a command similar to the example that follows. +If you are connecting to an Autunomous Database, and the operator is used to manage the Oracle Autonomous Database, then a client wallet can also be downloaded as a Secret through `kubectl` commands. See the ADB README section on [Download Wallets](../../docs/adb/README.md#download-wallets). -Otherwise, you can create the wallet secret from a local directory containing the wallet files. +You can also choose to create the wallet secret from a local directory containing the wallet files: ```bash kubectl create secret generic db-wallet --from-file=wallet_dir ``` -3. Finally, update the databaseObserver manifest with the resources you have created. You can use the example manifest -inside config/samples/observability to specify and create your databaseObserver object with a +3. Finally, update the `databaseObserver` manifest with the resources you have created. You can use the example _minimal_ manifest +inside [config/samples/observability/v4](../../config/samples/observability/v4/databaseobserver_minimal.yaml) to specify and create your databaseObserver object with a YAML file. ```YAML # example -apiVersion: observability.oracle.com/v1alpha1 +apiVersion: observability.oracle.com/v4 kind: DatabaseObserver metadata: name: obs-sample @@ -139,6 +194,11 @@ spec: dbWallet: secret: db-wallet + + prometheus: + serviceMonitor: + labels: + release: prometheus ``` ```bash @@ -159,8 +219,8 @@ To obtain a quick status, use the following command as an example: ```sh $ kubectl get databaseobserver obs-sample -NAME EXPORTERCONFIG STATUS -obs-sample default READY +NAME EXPORTERCONFIG STATUS VERSION +obs-sample DEFAULT READY 1.5.1 ``` @@ -170,39 +230,323 @@ To obtain a more detailed status, use the following command as an example: kubectl describe databaseobserver obs-sample ``` -This provides details of the current state of your databaseObserver resource object. A successful -deployment of the databaseObserver resource object should display `READY` as the status and all conditions with a `True` -value for every ConditionType. +This command displays details of the current state of your `databaseObserver` resource object. A successful +deployment of the `databaseObserver` resource object should display `READY` as the status, and all conditions should display with a `True` value for every ConditionType. ### Patch Resource -The Observability controller currently supports updates for most of the fields in the manifest. An example of patching the databaseObserver resource is as follows: +The Observability controller currently supports updates for most of the fields in the manifest. The following is an example of patching the `databaseObserver` resource: ```bash -kubectl --type=merge -p '{"spec":{"exporter":{"image":"container-registry.oracle.com/database/observability-exporter:latest"}}}' patch databaseobserver obs-sample +kubectl --type=merge -p '{"spec":{"exporter":{"image":"container-registry.oracle.com/database/observability-exporter:1.5.0"}}}' patch databaseobserver obs-sample ``` -The fields listed below can be updated with the given example command: +### Delete Resource -* spec.exporter.image -* spec.exporter.configuration.configmap.configmapName -* spec.exporter.configuration.configmap.key -* spec.database.dbUser.secret -* spec.database.dbPassword.secret -* spec.database.dbConnectionString.secret -* spec.database.dbWallet.secret -* spec.ociConfig.configMapName -* spec.ociConfig.secretName -* spec.replicas -* spec.database.dbPassword.vaultOCID -* spec.database.dbPassword.vaultSecretName +To delete the `databaseObserver` custom resource and all related resources, use this command: +```bash +kubectl delete databaseobserver obs-sample +``` -### Delete Resource +## Scraping Metrics +The `databaseObserve`r resource deploys the Observability exporter container. This container connects to an Oracle Database and +scrapes metrics using SQL queries. By default, the exporter provides standard metrics, which are listed in the [official GitHub page of the Observability Exporter](https://github.com/oracle/oracle-db-appdev-monitoring?tab=readme-ov-file#standard-metrics). + +To define custom metrics in Oracle Database for scraping, a TOML file that lists your custom queries and properties is required. +The file will have metric sections with the following parts: +- a context +- a request, which contains the SQL query +- a map between the field(s) in the request and comment(s) + +For example, the code snippet that follows shows how you can define custom metrics: +```toml +[[metric]] +context = "test" +request = "SELECT 1 as value_1, 2 as value_2 FROM DUAL" +metricsdesc = { value_1 = "Simple example returning always 1.", value_2 = "Same but returning always 2." } +``` +This file produces the following entries: +``` +# HELP oracledb_test_value_1 Simple example returning always 1. +# TYPE oracledb_test_value_1 gauge +oracledb_test_value_1 1 +# HELP oracledb_test_value_2 Same but returning always 2. +# TYPE oracledb_test_value_2 gauge +oracledb_test_value_2 2 +``` -To delete the DatabaseObserver custom resource and all related resources: +You can find more information in the [__Custom Metrics__](https://github.com/oracle/oracle-db-appdev-monitoring?tab=readme-ov-file#custom-metrics) section of the Official GitHub page. + + +### Custom Metrics Config +When configuring a `databaseObserver` resource, you can use the field `spec.configuration.configMap` to provide a +custom metrics file as a `configMap`. + +You can create the `configMap` by running the following command: ```bash -kubectl delete databaseobserver obs-sample +kubectl create cm custom-metrics-cm --from-file=metrics.toml +``` + +Finally, when creating or updating a `databaseObserver` resource, if we assume using the example above, you can set the fields in your YAML file as follows: +```yaml +spec: + configuration: + configMap: + key: "metrics.toml" + name: "custom-metrics-cm" +``` + +### Prometheus Release +To enable your Prometheus configuration to find and include the `ServiceMonitor` created by the `databaseObserver` resource, the field `spec.prometheus.serviceMonitor.labels` is an __important__ and __required__ field. The label on the ServiceMonitor +must match the `spec.serviceMonitorSelector` field in your Prometheus configuration. + +```yaml + prometheus: + serviceMonitor: + labels: + release: stable +``` + +## Scraping Logs +Currently, the observability exporter provides the `alert.log` from Oracle Database, which provides important information about errors and exceptions during database operations. + +By default, the logs are stored in the pod filesystem, inside `/log/alert.log`. Note that the log can also be placed in a custom path with a custom filename, You can also place a volume available to multiple pods with the use of `PersistentVolumes` by specifying a `persistentVolumeClaim`. +Because the logs are stored in a file, scraping the logs must be pushed to a log aggregation system, such as _Loki_. +In the following example, `Promtail` is used as a sidecar container that ships the contents of local logs to the Loki instance. + + +To configure the `databaseObserver` resource with a sidecar, two fields can be used: +```yaml +spec: + sidecars: [] + sidecarVolumes: [] +``` + +You can find an example in the `samples` directory, which deploys a Promtail sidecar container as an example: +[`config/samples/observability/v4/databaseobserver_logs_promtail.yaml`](../../config/samples/observability/v4/databaseobserver_logs_promtail.yaml) + +### Custom Log Location with PersistentVolumes + +The fields `spec.log.filename` and `spec.log.path` enable you to configure a custom location and filename for the log. +Using a custom location enables you to control where to place the logfile, such as a `persistentVolume`. + +```yaml + log: + filename: "alert.log" + path: "/log" +``` + +To configure the `databaseObserver` resource to put the log file in a `persistentVolume`, you can set the following fields +in your `databaseObserver` YAML file. The field `spec.log.volume.name` is provided to control the name of the volume used +for the log, while the field `spec.log.volume.persistentVolumeClaim.claimName` is used to specify the claim to use. +These details can be used with any sidecar containers, or with other containers. + +If `spec.log.volume.persistentVolumeClaim.claimName` is not specified, then an `EmptyDir` volume is automatically used. + +> Important Note: the volume name must match all references of the volume, such as in any sidecar containers that use and mount this volume. + +```yaml + log: + volume: + name: my-log-volume + persistentVolumeClaim: + claimName: "my-pvc" +``` + +The security context defines privilege and access control settings for a pod container, If these privileges and access control settingrs need to be updated in the pod, then the same field is available on the `databaseObserver` spec. You can set this object under deployment: `spec.exporter.deployment.securityContext`. + +```yaml +spec: + exporter: + deployment: + runAsUser: 1000 +``` + +Configuring security context under the PodTemplate is also possible. You can set this object under: `spec.exporter.deployment.podTemplate.securityContext` + +```yaml +spec: + exporter: + deployment: + podTemplate: + securityContext: + supplementalGroups: [1000] +``` + + +### Working with Sidecars to deploy Promtail +The fields `spec.sidecars` and `spec.sidecarVolumes` provide the ability to deploy container images as a sidecar container +alongside the `observability-exporter` container. + +You can specify container images to deploy inside `spec.sidecars` as you would normally define a container in a deployment. The field +`spec.sidecars` is of an array of containers (`[]corev1.Container`). + +For example, to deploy a Grafana Promtail image, you can specify the container and its details as an element to the array, `spec.sidecars`. +```yaml + sidecars: + - name: promtail + image: grafana/promtail + args: + - -config.file=/etc/promtail/config.yaml + volumeMounts: + - name: promtail-config-volume + mountPath: /etc/promtail + - name: my-log-volume + mountPath: /log +``` + +> Important Note: Make sure the volumeMount name matches the actual name of the volumes referenced. In this case, `my-log-volume` is referenced in `spec.log.volume.name`. + +In the field `spec.sidecarVolumes`, you can specify and list the volumes you need in your sidecar containers. The field +`spec.sidecarVolumes` is an array of Volumes (`[]corev1.Volume`). + +For example, when deploying the Promtail container, you can specify in the field any volume that needs to be mounted in the sidecar container above. + +```yaml + sidecarVolumes: + - name: promtail-config-volume + configMap: + name: promtail-config-file +``` + +In this example, the `promtail-config-file` `configMap` contains the Promtail configuration, which specifies where to find +the target and the path to the file, as well as the endpoint where Loki is listening for any push API requests. + +__Promtail Config Example__ + +```yaml +# config.yaml +server: + http_listen_port: 9080 + grpc_listen_port: 0 +positions: + filename: /tmp/positions.yaml + +clients: + - url: http://{loki-endpoint}:3100/loki/api/v1/push + +scrape_configs: + - job_name: "alert-log" + static_configs: + - targets: + - localhost + labels: + app: {my-database-observer-label} + __path__: /log/*.log + ``` + +To create the `configmap`, you can run the following command: +```bash +kubectl create cm promtail-config-file --from-file=config.yaml +``` + + +## Other Configuration Options + +### Labels + +__About the Default Label__ - The resources created by the Observability Controller will automatically be labelled with: +- `app`: `` + + +For example, if the `databaseObserver` instance is named: `metrics-exporter`, then resources such as the deployment will be labelled +with `app: metrics-exporter`. This label `cannot be overwritten` as this label is used by multiple resources created. Selectors used by the deployment, service and servicemonitor use this label. + +The following configuration shows an example: + +```yaml +apiVersion: observability.oracle.com/v4 +kind: DatabaseObserver +metadata: + name: metrics-exporter + labels: + app: my-db-metrics + some: non-inherited-label +spec: + + # inheritLabels + inheritLabels: + - some + + # ... +``` + +Meanwhile, you can provide extra labels to the resources created by the `databaseObserver` controller, such as the Deployment, Pods, Service and ServiceMonitor. +```yaml +spec: + exporter: + deployment: + labels: + podTemplate: + labels: + service: + labels: + prometheus: + serviceMonitor: + labels: +``` + +### Custom Exporter Image or Version +The field `spec.exporter.deployment.image` is provided to enable you to make use of a newer or older version of the [observability-exporter](https://github.com/oracle/oracle-db-appdev-monitoring) +container image. + +```yaml +spec: + exporter: + deployment: + image: "container-registry.oracle.com/database/observability-exporter:1.5.3" +``` + +### Custom Environment Variables, Arguments and Commands +The fields `spec.exporter.deployment.env`, `spec.exporter.deployment.args` and `spec.exporter.deployment.commands` are provided for adding custom environment variables, arguments (`args`) and commands to the containers. +Any custom environment variable will overwrite environment variables set by the controller. + +```yaml +spec: + exporter: + deployment: + env: + DB_ROLE: "" + TNS_ADMIN: "" + args: + - "--log.level=info" + commands: + - "/oracledb_exporter" +``` + + +### Custom Service Ports +The field `spec.exporter.service.ports` is provided to enable setting the ports of the service. If not set, then the following definition is set by default. + +```yaml +spec: + exporter: + service: + ports: + - name: metrics + port: 9161 + targetPort: 9161 + +``` + +### Custom ServiceMonitor Endpoints +The field `spec.prometheus.serviceMonitor.endpoints` is provided for providing custom endpoints for the ServiceMonitor resource created by the `databaseObserver`: + +```yaml +spec: + prometheus: + serviceMonitor: + endpoints: + - bearerTokenSecret: + key: '' + interval: 20s + port: metrics + relabelings: + - action: replace + sourceLabels: + - __meta_kubernetes_endpoints_label_app + targetLabel: instance ``` ## Mandatory roles and privileges requirements for Observability Controller @@ -231,17 +575,17 @@ and gets and lists configmaps and secrets. ## Debugging and troubleshooting ### Show the details of the resource -To get the verbose output of the current spec, use the command below: +To obtain the verbose output of the current spec, use the following command: ```sh kubectl describe databaseobserver/database-observer-sample ``` -If any error occurs during the reconciliation loop, the Operator either reports -the error using the resource's event stream, or will show the error under conditions. +If any error occurs during the reconciliation loop, then the Operator either reports +the error using the resource's event stream, or it will show the error under conditions. ### Check the logs of the pod where the operator deploys -Follow the steps to check the logs. +Follow these steps to check the logs. 1. List the pod replicas @@ -249,8 +593,18 @@ Follow the steps to check the logs. kubectl get pods -n oracle-database-operator-system ``` -2. Use the below command to check the logs of the deployment +2. Use the following command to check the logs of the deployment ```sh kubectl logs deployment.apps/oracle-database-operator-controller-manager -n oracle-database-operator-system ``` + +## Known Potential Issues + +| Issue | Example error | Potential Workaround | +|---------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------| +| Pod may encounter error Permission denied when creating log file. Pod cannot access file system due to insufficient permissions | ```level=error msg="Failed to create the log file: /log/alert.log"``` | Configure securityContext in the spec, add your group ID to the `supplementalgroups` inside `spec.exporter.deployment.podTemplate.securityContext` field. | + + +## Resources +- [GitHub - Unified Observability for Oracle Database Project](https://github.com/oracle/oracle-db-appdev-monitoring) diff --git a/docs/ordsservices/README.md b/docs/ordsservices/README.md new file mode 100644 index 00000000..1740e99f --- /dev/null +++ b/docs/ordsservices/README.md @@ -0,0 +1,67 @@ +# Oracle Rest Data Services (ORDSSRVS) Controller for Kubernetes - ORDS Life cycle management + + +## Description + +The ORDSRVS controller extends the Kubernetes API with a Custom Resource (CR) and Controller for automating Oracle Rest Data +Services (ORDS) lifecycle management. Using the ORDS controller, you can easily migrate existing, or create new, ORDS implementations +into an existing Kubernetes cluster. + +This controller allows you to run what would otherwise be an On-Premises ORDS middle-tier, configured as you require, inside Kubernetes with the additional ability of the controller to perform automatic ORDS/APEX install/upgrades inside the database. + +## Features Summary + +The custom RestDataServices resource supports the following configurations as a Deployment, StatefulSet, or DaemonSet: + +* Single OrdsSrvs resource with one database pool +* Single OrdsSrvs resource with multiple database pools* +* Multiple OrdsSrvs resources, each with one database pool +* Multiple OrdsSrvs resources, each with multiple database pools* + +*See [Limitations](#limitations) + +It supports the majority of ORDS configuration settings as per the [API Documentation](./api.md). + +The ORDS and APEX schemas can be [automatically installed/upgraded](./autoupgrade.md) into the Oracle Database by the ORDS controller. + +ORDS Version support: +* v22.1+ + +Oracle Database Version: +* 19c +* 23ai (incl. 23ai Free) + + +### Common Configurations + +A few common configuration examples can be used to quickly familiarise yourself with the ORDS Custom Resource Definition. +The "Conclusion" section of each example highlights specific settings to enable functionality that maybe of interest. + +* [Containerised Single Instance Database using the Oracontroller](./examples/sidb_container.md) +* [Multipool, Multidatabase using a TNS Names file](./examples/multi_pool.md) +* [Autonomous Database using the Oracontroller](./examples/adb_oraoper.md) - (Customer Managed ORDS) *See [Limitations](#limitations) +* [Autonomous Database without the Oracontroller](./examples/adb.md) - (Customer Managed ORDS) +* [Oracle API for MongoDB Support](./examples/mongo_api.md) + +Running through all examples in the same Kubernetes cluster illustrates the ability to run multiple ORDS instances with a variety of different configurations. + +If you have a specific use-case that is not covered and would like it to be feel free to contribute it via a Pull Request. + +### Limitations + +When connecting to a mTLS enabled ADB and using the Oracontroller to retreive the Wallet, it is currently not supported to have multiple, different databases supported by the single RestDataServices resource. This is due to a requirement to set the `TNS_ADMIN` parameter at the Pod level ([#97](https://github.com/oracle/oracle-database-controller/issues/97)). + +### Troubleshooting +See [Troubleshooting](./TROUBLESHOOTING.md) + +## Contributing +See [Contributing to this Repository](./CONTRIBUTING.md) + +## Reporting a Security Issue + +See [Reporting security vulnerabilities](./SECURITY.md) + +## License + +Copyright (c) 2025 Oracle and/or its affiliates. +Released under the Universal Permissive License v1.0 as shown at [https://oss.oracle.com/licenses/upl/](https://oss.oracle.com/licenses/upl/) diff --git a/docs/ordsservices/TROUBLESHOOTING.md b/docs/ordsservices/TROUBLESHOOTING.md new file mode 100644 index 00000000..b1b5304d --- /dev/null +++ b/docs/ordsservices/TROUBLESHOOTING.md @@ -0,0 +1,129 @@ + + + +## TROUBLESHOOTING + +### Init container error + +Check the pod status and verify the init outcome + +---- +*Command:* +```bash +kubectl get pods -n +``` + +*Example:* +```bash +kubectl get pods -n ordsnamespace +NAME READY STATUS RESTARTS AGE +ords-multi-pool-55db776994-7rrff 0/1 Init:CrashLoopBackOff 6 (61s ago) 12m +``` +In case of error identify the *initContainer* name + +---- +*Command:* +```bash +kubectl get pod -n -o="custom-columns=NAME:.metadata.name,INIT-CONTAINERS:.spec.initContainers[*].name,CONTAINERS:.spec.containers[*].name" +``` + +Use the initContainers info to dump log information +**Command:** +```bash +kubectl logs -f --since=0 -n -c +``` + +*Example:* + +In this particular case we are providing wrong credential: "SYT" user does not exist + +```text +kubectl logs -f --since=0 ords-multi-pool-55db776994-m7782 -n ordsnamespace -c ords-multi-pool-init + +[..omissis...] +Running SQL... +Picked up JAVA_TOOL_OPTIONS: -Doracle.ml.version_check=false +BACKTRACE [24:09:17 08:59:03] + +filename:line function +------------- -------- +/opt/oracle/sa/bin/init_script.sh:115 run_sql +/opt/oracle/sa/bin/init_script.sh:143 check_adb +/opt/oracle/sa/bin/init_script.sh:401 main +SQLERROR: + USER = SYT + URL = jdbc:oracle:thin:@PDB2 + Error Message = 🔥ORA-01017: invalid username/password;🔥 logon denied +Pool: pdb2, Exit Code: 1 +Pool: pdb1, Exit Code: 1 +``` + +--- +*Diag shell* Use the following script to dump the container init log + +```bash +#!/bin/bash +NAMESPACE=${1:-"ordsnamespace"} +KUBECTL=/usr/bin/kubectl +for _pod in `${KUBECTL} get pods --no-headers -o custom-columns=":metadata.name" --no-headers -n ${NAMESPACE}` +do + for _podinit in `${KUBECTL} get pod ${_pod} -n ${NAMESPACE} -o="custom-columns=INIT-CONTAINERS:.spec.initContainers[*].name" --no-headers` + do + echo "DUMPINIT ${_pod}:${_podinit}" + ${KUBECTL} logs -f --since=0 ${_pod} -n ${NAMESPACE} -c ${_podinit} + done +done +``` + +## Ords init error + +Get pod name + +*Command:* +```bash +kubectl get pods -n +``` + +*Example:* +``` +kubectl get pods -n ordsnamespace +NAME READY STATUS RESTARTS AGE +ords-multi-pool-55db776994-m7782 1/1 Running 0 2m51s +``` +---- +Dump ords log + +*Commands:* +```bash +kubectl logs --since=0 -n +``` +*Example:* +```text +kubectl logs --since=0 ords-multi-pool-55db776994-m7782 -n ordsnamespace +[..omissis..] +2024-09-17T09:47:39.227Z WARNING The pool named: |pdb2|lo| is invalid and will be ignored: ORDS was unable to make a connection to the database. The database user specified by db.username configuration setting is locked. The connection pool named: |pdb2|lo| had the following error(s): 🔥ORA-28000: The account is locked.🔥 + +2024-09-17T09:47:39.370Z WARNING The pool named: |pdb1|lo| is invalid and will be ignored: ORDS was unable to make a connection to the database. The database user specified by db.username configuration setting is locked. The connection pool named: |pdb1|lo| had the following error(s): 🔥ORA-28000: The account is locked.🔥 + +2024-09-17T09:47:39.375Z INFO + +Mapped local pools from /opt/oracle/sa/config/databases: + /ords/pdb1/ => pdb1 => INVALID + /ords/pdb2/ => pdb2 => INVALID + + +2024-09-17T09:47:39.420Z INFO Oracle REST Data Services initialized +Oracle REST Data Services version : 24.1.1.r1201228 +Oracle REST Data Services server info: jetty/10.0.20 +Oracle REST Data Services java info: Java HotSpot(TM) 64-Bit Server VM 11.0.15+8-LTS-149 +``` + +*Solution:* Connect to the container db to unlock the account + +```sql +alter user ORDS_PUBLIC_USER account unlock; +``` + + + + diff --git a/docs/ordsservices/api.md b/docs/ordsservices/api.md new file mode 100644 index 00000000..da4db09c --- /dev/null +++ b/docs/ordsservices/api.md @@ -0,0 +1,1388 @@ +# API Reference + +Packages: + +- [database.oracle.com/v1](#databaseoraclecomv1) + +# database.oracle.com/v1 + +Resource Types: + +- [OrdsSrvs](#ordssrvs) + + + + +## OrdsSrvs +[↩ Parent](#databaseoraclecomv1 ) + + + + + + +OrdsSrvs is the Schema for the ordssrvs API + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameTypeDescriptionRequired
apiVersionstringdatabase.oracle.com/v1true
kindstringOrdsSrvstrue
metadataobjectRefer to the Kubernetes API documentation for the fields of the `metadata` field.true
specobject + OrdsSrvsSpec defines the desired state of OrdsSrvs
+ 		false
statusobject + OrdsSrvsStatus defines the observed state of OrdsSrvs
+ 		false
+ + +### OrdsSrvs.spec +[↩ Parent](#ordssrvs) + + + +OrdsSrvsSpec defines the desired state of OrdsSrvs + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameTypeDescriptionRequired
globalSettingsobject Contains settings that are configured across the entire +ORDS instance.
+		true
imagestring Specifies the ORDS container image
+		true
forceRestartboolean Specifies whether to restart pods when Global or Pool +configurations change
+		false
imagePullPolicyenum Specifies the ORDS container image pull policy
+
+Enum: IfNotPresent, Always, Never
+Default: IfNotPresent
+		false
imagePullSecretsstring Specifies the Secret Name for pulling the ORDS container +image
+		false
poolSettings< +a>[]object Contains settings for individual pools/databases
+		false
replicasinteger Defines the number of desired Replicas when workloadType +Deployment or StatefulSet
+
+Format: int32
+Default: 1
+Minimum: 1
+		false
workloadTypeenum Specifies the desired Kubernetes Workload
+
+Enum: Deployment, StatefulSet, DaemonSet
+Default: Deployment
+		false
encPrivKey
+		secret
+		secretName: string  passwordKey: +string Define the private key to decrypt passwords
+		true
+
+ +### OrdsSrvs.spec.globalSettings +[↩ Parent](#ordssrvsspec) + + + +Contains settings that are configured across the entire ORDS instance. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameTypeDescriptionRequired
cache.metadata.enabledboolean + Specifies the setting to enable or disable metadata caching.
+ 		false
cache.metadata.graphql.expireAfterAccessinteger + Specifies the duration after a GraphQL schema is not accessed from the cache that it expires.
+
+ Format: int64
+ 		false
cache.metadata.graphql.expireAfterWriteinteger + Specifies the duration after a GraphQL schema is cached that it expires and has to be loaded again.
+
+ Format: int64
+ 		false
cache.metadata.jwks.enabledboolean + Specifies the setting to enable or disable JWKS caching.
+ 		false
cache.metadata.jwks.expireAfterAccessinteger + Specifies the duration after a JWK is not accessed from the cache that it expires. By default this is disabled.
+
+ Format: int64
+ 		false
cache.metadata.jwks.expireAfterWriteinteger + Specifies the duration after a JWK is cached, that is, it expires and has to be loaded again.
+
+ Format: int64
+ 		false
cache.metadata.jwks.initialCapacityinteger + Specifies the initial capacity of the JWKS cache.
+
+ Format: int32
+ 		false
cache.metadata.jwks.maximumSizeinteger + Specifies the maximum capacity of the JWKS cache.
+
+ Format: int32
+ 		false
cache.metadata.timeoutinteger + Specifies the setting to determine for how long a metadata record remains in the cache. Longer duration means, it takes longer to view the applied changes. The formats accepted are based on the ISO-8601 duration format.
+
+ Format: int64
+ 		false
certSecretobject + Specifies the Secret containing the SSL Certificates Replaces: standalone.https.cert and standalone.https.cert.key
+ 		false
database.api.enabledboolean + Specifies whether the Database API is enabled.
+ 		false
database.api.management.services.disabledboolean + Specifies to disable the Database API administration related services. Only applicable when Database API is enabled.
+ 		false
db.invalidPoolTimeoutinteger + Specifies how long to wait before retrying an invalid pool.
+
+ Format: int64
+ 		false
debug.printDebugToScreenboolean + Specifies whether to display error messages on the browser.
+ 		false
enable.mongo.access.logboolean + Specifies if HTTP request access logs should be enabled If enabled, logs will be written to /opt/oracle/sa/log/global
+
+ Default: false
+ 		false
enable.standalone.access.logboolean + Specifies if HTTP request access logs should be enabled If enabled, logs will be written to /opt/oracle/sa/log/global
+
+ Default: false
+ 		false
error.responseFormatstring + Specifies how the HTTP error responses must be formatted. html - Force all responses to be in HTML format json - Force all responses to be in JSON format auto - Automatically determines most appropriate format for the request (default).
+ 		false
feature.grahpql.max.nesting.depthinteger + Specifies the maximum join nesting depth limit for GraphQL queries.
+
+ Format: int32
+ 		false
icap.portinteger + Specifies the Internet Content Adaptation Protocol (ICAP) port to virus scan files. Either icap.port or icap.secure.port are required to have a value.
+
+ Format: int32
+ 		false
icap.secure.portinteger + Specifies the Internet Content Adaptation Protocol (ICAP) port to virus scan files. Either icap.port or icap.secure.port are required to have a value. If values for both icap.port and icap.secure.port are provided, then the value of icap.port is ignored.
+
+ Format: int32
+ 		false
icap.serverstring + Specifies the Internet Content Adaptation Protocol (ICAP) server name or IP address to virus scan files. The icap.server is required to have a value.
+ 		false
log.procedureboolean + Specifies whether procedures are to be logged.
+ 		false
mongo.enabledboolean + Specifies to enable the API for MongoDB.
+ 		false
mongo.idle.timeoutinteger + Specifies the maximum idle time for a Mongo connection in milliseconds.
+
+ Format: int64
+ 		false
mongo.op.timeoutinteger + Specifies the maximum time for a Mongo database operation in milliseconds.
+
+ Format: int64
+ 		false
mongo.portinteger + Specifies the API for MongoDB listen port.
+
+ Format: int32
+ Default: 27017
+ 		false
request.traceHeaderNamestring + Specifies the name of the HTTP request header that uniquely identifies the request end to end as it passes through the various layers of the application stack. In Oracle this header is commonly referred to as the ECID (Entity Context ID).
+ 		false
security.credentials.attemptsinteger + Specifies the maximum number of unsuccessful password attempts allowed. Enabled by setting a positive integer value.
+
+ Format: int32
+ 		false
security.credentials.lock.timeinteger + Specifies the period to lock the account that has exceeded maximum attempts.
+
+ Format: int64
+ 		false
security.disableDefaultExclusionListboolean + If this value is set to true, then the Oracle REST Data Services internal exclusion list is not enforced. Oracle recommends that you do not set this value to true.
+ 		false
security.exclusionListstring + Specifies a pattern for procedures, packages, or schema names which are forbidden to be directly executed from a browser.
+ 		false
security.externalSessionTrustedOriginsstring + Specifies to trust Access from originating domains
+ 		false
security.forceHTTPSboolean + Specifies to force HTTPS; this is set to default to false as in real-world TLS should terminiate at the LoadBalancer
+ 		false
security.httpsHeaderCheckstring + Specifies that the HTTP Header contains the specified text Usually set to 'X-Forwarded-Proto: https' coming from a load-balancer
+ 		false
security.inclusionListstring + Specifies a pattern for procedures, packages, or schema names which are allowed to be directly executed from a browser.
+ 		false
security.maxEntriesinteger + Specifies the maximum number of cached procedure validations. Set this value to 0 to force the validation procedure to be invoked on each request.
+
+ Format: int32
+ 		false
security.verifySSLboolean + Specifies whether HTTPS is available in your environment.
+ 		false
standalone.context.pathstring + Specifies the context path where ords is located.
+
+ Default: /ords
+ 		false
standalone.http.portinteger + Specifies the HTTP listen port.
+
+ Format: int32
+ Default: 8080
+ 		false
standalone.https.hoststring + Specifies the SSL certificate hostname.
+ 		false
standalone.https.portinteger + Specifies the HTTPS listen port.
+
+ Format: int32
+ Default: 8443
+ 		false
standalone.stop.timeoutinteger + Specifies the period for Standalone Mode to wait until it is gracefully shutdown.
+
+ Format: int64
+ 		false
+ + +### OrdsSrvs.spec.globalSettings.certSecret +[↩ Parent](#ordssrvsspecglobalsettings) + + + +Specifies the Secret containing the SSL Certificates Replaces: standalone.https.cert and standalone.https.cert.key + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameTypeDescriptionRequired
certstring + Specifies the Certificate
+ 		true
keystring + Specifies the Certificate Key
+ 		true
secretNamestring + Specifies the name of the certificate Secret
+ 		true
+ + +### OrdsSrvs.spec.poolSettings[index] +[↩ Parent](#ordssrvsspec) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameTypeDescriptionRequired
db.secretobject + Specifies the Secret with the dbUsername and dbPassword values for the connection.
+ 		true
poolNamestring + Specifies the Pool Name
+ 		true
apex.security.administrator.rolesstring + Specifies the comma delimited list of additional roles to assign authenticated APEX administrator type users.
+ 		false
apex.security.user.rolesstring + Specifies the comma delimited list of additional roles to assign authenticated regular APEX users.
+ 		false
autoUpgradeAPEXboolean + Specify whether to perform APEX installation/upgrades automatically The db.adminUser and db.adminUser.secret must be set, otherwise setting is ignored This setting will be ignored for ADB
+
+ Default: false
+ 		false
autoUpgradeORDSboolean + Specify whether to perform ORDS installation/upgrades automatically The db.adminUser and db.adminUser.secret must be set, otherwise setting is ignored This setting will be ignored for ADB
+
+ Default: false
+ 		false
db.adminUserstring + Specifies the username for the database account that ORDS uses for administration operations in the database.
+ 		false
db.adminUser.secretobject + Specifies the Secret with the dbAdminUser (SYS) and dbAdminPassword values for the database account that ORDS uses for administration operations in the database. replaces: db.adminUser.password
+ 		false
db.cdb.adminUserstring + Specifies the username for the database account that ORDS uses for the Pluggable Database Lifecycle Management.
+ 		false
db.cdb.adminUser.secretobject + Specifies the Secret with the dbCdbAdminUser (SYS) and dbCdbAdminPassword values Specifies the username for the database account that ORDS uses for the Pluggable Database Lifecycle Management. Replaces: db.cdb.adminUser.password
+ 		false
db.connectionTypeenum + The type of connection.
+
+ Enum: basic, tns, customurl
+ 		false
db.credentialsSourceenum + Specifies the source for database credentials when creating a direct connection for running SQL statements. Value can be one of pool or request. If the value is pool, then the credentials defined in this pool is used to create a JDBC connection. If the value request is used, then the credentials in the request is used to create a JDBC connection and if successful, grants the requestor SQL Developer role.
+
+ Enum: pool, request
+ 		false
db.customURLstring + Specifies the JDBC URL connection to connect to the database.
+ 		false
db.hostnamestring + Specifies the host system for the Oracle database.
+ 		false
db.poolDestroyTimeoutinteger + Indicates how long to wait to gracefully destroy a pool before moving to forcefully destroy all connections including borrowed ones.
+
+ Format: int64
+ 		false
db.portinteger + Specifies the database listener port.
+
+ Format: int32
+ 		false
db.servicenamestring + Specifies the network service name of the database.
+ 		false
db.sidstring + Specifies the name of the database.
+ 		false
db.tnsAliasNamestring + Specifies the TNS alias name that matches the name in the tnsnames.ora file.
+ 		false
db.usernamestring + Specifies the name of the database user for the connection. For non-ADB this will default to ORDS_PUBLIC_USER For ADBs this must be specified and not ORDS_PUBLIC_USER If ORDS_PUBLIC_USER is specified for an ADB, the workload will fail
+
+ Default: ORDS_PUBLIC_USER
+ 		false
db.wallet.zip.servicestring + Specifies the service name in the wallet archive for the pool.
+ 		false
dbWalletSecretobject + Specifies the Secret containing the wallet archive containing connection details for the pool. Replaces: db.wallet.zip
+ 		false
debug.trackResourcesboolean + Specifies to enable tracking of JDBC resources. If not released causes in resource leaks or exhaustion in the database. Tracking imposes a performance overhead.
+ 		false
feature.openservicebroker.excludeboolean + Specifies to disable the Open Service Broker services available for the pool.
+ 		false
feature.sdwboolean + Specifies to enable the Database Actions feature.
+ 		false
http.cookie.filterstring + Specifies a comma separated list of HTTP Cookies to exclude when initializing an Oracle Web Agent environment.
+ 		false
jdbc.DriverTypeenum + Specifies the JDBC driver type.
+
+ Enum: thin, oci8
+ 		false
jdbc.InactivityTimeoutinteger + Specifies how long an available connection can remain idle before it is closed. The inactivity connection timeout is in seconds.
+
+ Format: int32
+ 		false
jdbc.InitialLimitinteger + Specifies the initial size for the number of connections that will be created. The default is low, and should probably be set higher in most production environments.
+
+ Format: int32
+ 		false
jdbc.MaxConnectionReuseCountinteger + Specifies the maximum number of times to reuse a connection before it is discarded and replaced with a new connection.
+
+ Format: int32
+ 		false
jdbc.MaxConnectionReuseTimeinteger + Sets the maximum connection reuse time property.
+
+ Format: int32
+ 		false
jdbc.MaxLimitinteger + Specifies the maximum number of connections. Might be too low for some production environments.
+
+ Format: int32
+ 		false
jdbc.MaxStatementsLimitinteger + Specifies the maximum number of statements to cache for each connection.
+
+ Format: int32
+ 		false
jdbc.MinLimitinteger + Specifies the minimum number of connections.
+
+ Format: int32
+ 		false
jdbc.SecondsToTrustIdleConnectioninteger + Sets the time in seconds to trust an idle connection to skip a validation test.
+
+ Format: int32
+ 		false
jdbc.auth.admin.rolestring + Identifies the database role that indicates that the database user must get the SQL Administrator role.
+ 		false
jdbc.auth.enabledboolean + Specifies if the PL/SQL Gateway calls can be authenticated using database users. If the value is true then this feature is enabled. If the value is false, then this feature is disabled. Oracle recommends not to use this feature. This feature used only to facilitate customers migrating from mod_plsql.
+ 		false
jdbc.cleanup.modestring + Specifies how a pooled JDBC connection and corresponding database session, is released when a request has been processed.
+ 		false
jdbc.statementTimeoutinteger + Specifies a timeout period on a statement. An abnormally long running query or script, executed by a request, may leave it in a hanging state unless a timeout is set on the statement. Setting a timeout on the statement ensures that all the queries automatically timeout if they are not completed within the specified time period.
+
+ Format: int32
+ 		false
misc.defaultPagestring + Specifies the default page to display. The Oracle REST Data Services Landing Page.
+ 		false
misc.pagination.maxRowsinteger + Specifies the maximum number of rows that will be returned from a query when processing a RESTful service and that will be returned from a nested cursor in a result set. Affects all RESTful services generated through a SQL query, regardless of whether the resource is paginated.
+
+ Format: int32
+ 		false
owa.trace.sqlboolean + If it is true, then it causes a trace of the SQL statements performed by Oracle Web Agent to be echoed to the log.
+ 		false
plsql.gateway.modeenum + Indicates if the PL/SQL Gateway functionality should be available for a pool or not. Value can be one of disabled, direct, or proxied. If the value is direct, then the pool serves the PL/SQL Gateway requests directly. If the value is proxied, the PLSQL_GATEWAY_CONFIG view is used to determine the user to whom to proxy.
+
+ Enum: disabled, direct, proxied
+ 		false
procedure.preProcessstring + Specifies the procedure name(s) to execute prior to executing the procedure specified on the URL. Multiple procedure names must be separated by commas.
+ 		false
procedure.rest.preHookstring + Specifies the function to be invoked prior to dispatching each Oracle REST Data Services based REST Service. The function can perform configuration of the database session, perform additional validation or authorization of the request. If the function returns true, then processing of the request continues. If the function returns false, then processing of the request is aborted and an HTTP 403 Forbidden status is returned.
+ 		false
procedurePostProcessstring + Specifies the procedure name(s) to execute after executing the procedure specified on the URL. Multiple procedure names must be separated by commas.
+ 		false
restEnabledSql.activeboolean + Specifies whether the REST-Enabled SQL service is active.
+ 		false
security.jwks.connection.timeoutinteger + Specifies the maximum amount of time before timing-out when accessing a JWK url.
+
+ Format: int64
+ 		false
security.jwks.read.timeoutinteger + Specifies the maximum amount of time reading a response from the JWK url before timing-out.
+
+ Format: int64
+ 		false
security.jwks.refresh.intervalinteger + Specifies the minimum interval between refreshing the JWK cached value.
+
+ Format: int64
+ 		false
security.jwks.sizeinteger + Specifies the maximum number of bytes read from the JWK url.
+
+ Format: int32
+ 		false
security.jwt.allowed.ageinteger + Specifies the maximum allowed age of a JWT in seconds, regardless of expired claim. The age of the JWT is taken from the JWT issued at claim.
+
+ Format: int64
+ 		false
security.jwt.allowed.skewinteger + Specifies the maximum skew the JWT time claims are accepted. This is useful if the clock on the JWT issuer and ORDS differs by a few seconds.
+
+ Format: int64
+ 		false
security.jwt.profile.enabledboolean + Specifies whether the JWT Profile authentication is available. Supported values:
+ 		false
security.requestAuthenticationFunctionstring + Specifies an authentication function to determine if the requested procedure in the URL should be allowed or disallowed for processing. The function should return true if the procedure is allowed; otherwise, it should return false. If it returns false, Oracle REST Data Services will return WWW-Authenticate in the response header.
+ 		false
security.requestValidationFunctionstring + Specifies a validation function to determine if the requested procedure in the URL should be allowed or disallowed for processing. The function should return true if the procedure is allowed; otherwise, return false.
+
+ Default: ords_util.authorize_plsql_gateway
+ 		false
security.validationFunctionTypeenum + Indicates the type of security.requestValidationFunction: javascript or plsql.
+
+ Enum: plsql, javascript
+ 		false
soda.defaultLimitstring + When using the SODA REST API, specifies the default number of documents returned for a GET request on a collection when a limit is not specified in the URL. Must be a positive integer, or "unlimited" for no limit.
+ 		false
soda.maxLimitstring + When using the SODA REST API, specifies the maximum number of documents that will be returned for a GET request on a collection URL, regardless of any limit specified in the URL. Must be a positive integer, or "unlimited" for no limit.
+ 		false
tnsAdminSecretobject + Specifies the Secret containing the TNS_ADMIN directory Replaces: db.tnsDirectory
+ 		false
+ + +### OrdsSrvs.spec.poolSettings[index].db.secret +[↩ Parent](#ordssrvsspecpoolsettingsindex) + + + +Specifies the Secret with the dbUsername and dbPassword values for the connection. + + + + + + + + + + + + + + + + + + + + + +
NameTypeDescriptionRequired
secretNamestring + Specifies the name of the password Secret
+ 		true
passwordKeystring + Specifies the key holding the value of the Secret
+
+ Default: password
+ 		false
+ + +### OrdsSrvs.spec.poolSettings[index].db.adminUser.secret +[↩ Parent](#ordssrvsspecpoolsettingsindex) + + + +Specifies the Secret with the dbAdminUser (SYS) and dbAdminPassword values for the database account that ORDS uses for administration operations in the database. replaces: db.adminUser.password + + + + + + + + + + + + + + + + + + + + + +
NameTypeDescriptionRequired
secretNamestring + Specifies the name of the password Secret
+ 		true
passwordKeystring + Specifies the key holding the value of the Secret
+
+ Default: password
+ 		false
+ + +### OrdsSrvs.spec.poolSettings[index].db.cdb.adminUser.secret +[↩ Parent](#ordssrvsspecpoolsettingsindex) + + + +Specifies the Secret with the dbCdbAdminUser (SYS) and dbCdbAdminPassword values Specifies the username for the database account that ORDS uses for the Pluggable Database Lifecycle Management. Replaces: db.cdb.adminUser.password + + + + + + + + + + + + + + + + + + + + + +
NameTypeDescriptionRequired
secretNamestring + Specifies the name of the password Secret
+ 		true
passwordKeystring + Specifies the key holding the value of the Secret
+
+ Default: password
+ 		false
+ + +### OrdsSrvs.spec.poolSettings[index].dbWalletSecret +[↩ Parent](#ordssrvsspecpoolsettingsindex) + + + +Specifies the Secret containing the wallet archive containing connection details for the pool. Replaces: db.wallet.zip + + + + + + + + + + + + + + + + + + + + + +
NameTypeDescriptionRequired
secretNamestring + Specifies the name of the Database Wallet Secret
+ 		true
walletNamestring + Specifies the Secret key name containing the Wallet
+ 		true
+ + +### OrdsSrvs.spec.poolSettings[index].tnsAdminSecret +[↩ Parent](#ordssrvsspecpoolsettingsindex) + + + +Specifies the Secret containing the TNS_ADMIN directory Replaces: db.tnsDirectory + + + + + + + + + + + + + + + + +
NameTypeDescriptionRequired
secretNamestring + Specifies the name of the TNS_ADMIN Secret
+ 		true
+ + +### OrdsSrvs.status +[↩ Parent](#ordssrvs) + + + +OrdsSrvsStatus defines the observed state of OrdsSrvs + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameTypeDescriptionRequired
restartRequiredboolean + Indicates if the resource is out-of-sync with the configuration
+ 		true
conditions[]object +
+ 		false
httpPortinteger + Indicates the HTTP port of the resource exposed by the pods
+
+ Format: int32
+ 		false
httpsPortinteger + Indicates the HTTPS port of the resource exposed by the pods
+
+ Format: int32
+ 		false
mongoPortinteger + Indicates the MongoAPI port of the resource exposed by the pods (if enabled)
+
+ Format: int32
+ 		false
ordsVersionstring + Indicates the ORDS version
+ 		false
statusstring + Indicates the current status of the resource
+ 		false
workloadTypestring + Indicates the current Workload type of the resource
+ 		false
+ + +### OrdsSrvs.status.conditions[index] +[↩ Parent](#ordssrvsstatus) + + + +Condition contains details for one aspect of the current state of this API Resource. --- This struct is intended for direct use as an array at the field path .status.conditions. For example, + type FooStatus struct{ // Represents the observations of a foo's current state. // Known .status.conditions.type are: "Available", "Progressing", and "Degraded" // +patchMergeKey=type // +patchStrategy=merge // +listType=map // +listMapKey=type Conditions []metav1.Condition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type" protobuf:"bytes,1,rep,name=conditions"` + // other fields } + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
NameTypeDescriptionRequired
lastTransitionTimestring + lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable.
+
+ Format: date-time
+ 		true
messagestring + message is a human readable message indicating details about the transition. This may be an empty string.
+ 		true
reasonstring + reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty.
+ 		true
statusenum + status of the condition, one of True, False, Unknown.
+
+ Enum: True, False, Unknown
+ 		true
typestring + type of condition in CamelCase or in foo.example.com/CamelCase. --- Many .condition.type values are consistent across resources like Available, but because arbitrary conditions can be useful (see .node.status.conditions), the ability to deconflict is important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt)
+ 		true
observedGenerationinteger + observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance.
+
+ Format: int64
+ Minimum: 0
+ 		false
diff --git a/docs/ordsservices/autoupgrade.md b/docs/ordsservices/autoupgrade.md new file mode 100644 index 00000000..fddc30b3 --- /dev/null +++ b/docs/ordsservices/autoupgrade.md @@ -0,0 +1,57 @@ +# AutoUpgrade + +Each pool can be configured to automatically install and upgrade the ORDS and/or APEX schemas in the database. +The ORDS and APEX version is based on the ORDS image used for the RestDataServices resource. + +For example, in the below manifest: +* `Pool: pdb1` is configured to automatically install/ugrade both ORDS and APEX to version 24.1.0 +* `Pool: pdb2` will not install or upgrade ORDS/APEX + +As an additional requirement for `Pool: pdb1`, the `spec.poolSettings.db.adminUser` and `spec.poolSettings.db.adminUser.secret` +must be provided. If they are not, the `autoUpgrade` specification is ignored. + +```yaml +apiVersion: database.oracle.com/v1 +kind: OrdsSrvs +metadata: + name: ordspoc-server +spec: + image: container-registry.oracle.com/database/ords:24.1.0 + forceRestart: true + globalSettings: + database.api.enabled: true + encPrivKey: + secretName: prvkey + passwordKey: privateKey + poolSettings: + - poolName: pdb1 + autoUpgradeORDS: true + autoUpgradeAPEX: true + db.connectionType: customurl + db.customURL: jdbc:oracle:thin:@//localhost:1521/PDB1 + db.secret: + secretName: pdb1-ords-auth + db.adminUser: SYS + db.adminUser.secret: + secretName: pdb1-sys-auth-enc + - poolName: pdb2 + db.connectionType: customurl + db.customURL: jdbc:oracle:thin:@//localhost:1521/PDB2 + db.secret: + secretName: pdb2-ords-auth-enc +``` + +## Minimum Privileges for Admin User + +The `db.adminUser` must have privileges to create users and objects in the database. For Oracle Autonomous Database (ADB), this could be `ADMIN` while for +non-ADBs this could be `SYS AS SYSDBA`. When you do not want to use `ADMIN` or `SYS AS SYSDBA` to install, upgrade, validate and uninstall ORDS a script is provided +to create a new user to be used. + +1. Download the equivalent version of ORDS to the image you will be using. +1. Extract the software and locate: `scripts/installer/ords_installer_privileges.sql` +1. Using SQLcl or SQL*Plus, connect to the Oracle PDB with SYSDBA privileges. +1. Execute the following script providing the database user: + ```sql + @/path/to/installer/ords_installer_privileges.sql privuser + exit + ``` diff --git a/docs/ordsservices/examples/adb.md b/docs/ordsservices/examples/adb.md new file mode 100644 index 00000000..ba53aac5 --- /dev/null +++ b/docs/ordsservices/examples/adb.md @@ -0,0 +1,108 @@ +# Example: Autonomous Database without the OraOperator + +This example walks through using the **ORDSSRVS controller** with an Oracle Autonomous Database. + +This assumes that an ADB has already been provisioned and is configured as "Secure Access from Anywhere". +Note that if behind a Proxy, this example will not work as the Wallet will need to be modified to support the proxy configuration. + + +### Cert-Manager and Oracle Database Operator installation + +Install the [Cert Manager](https://github.com/cert-manager/cert-manager/releases/download/v1.14.4/cert-manager.yaml) and the [Oracle Database Operator](https://github.com/oracle/oracle-database-operator) using the instractions in the Operator [README](https://github.com/oracle/oracle-database-operator/blob/main/README.md) file. + + +### ADB Wallet Secret + +Download the ADB Wallet and create a Secret, replacing `` with the path to the wallet zip file: + +```bash +kubectl create secret generic adb-wallet \ + --from-file= -n ordsnamespace +``` + +### ADB ADMIN Password Secret + +Create a Secret for the ADB ADMIN password, replacing with the real password: + +```bash +echo adb-db-auth-enc +openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -pkeyopt rsa_keygen_pubexp:65537 > ca.k +openssl rsa -in ca.key -outform PEM -pubout -out public.pem +kubectl create secret generic prvkey --from-file=privateKey=ca.key -n ordsnamespace +openssl rsautl -encrypt -pubin -inkey public.pem -in adb-db-auth-enc |base64 > e_sidb-db-auth-enc +kubectl create secret generic adb-db-auth-enc --from-file=password=e_sidb-db-auth-enc -n ordsnamespace +rm adb-db-auth-enc e_sidb-db-auth-enc +``` + +### Create RestDataServices Resource + +1. Create a manifest for ORDS. + + As an ADB already maintains ORDS and APEX, `autoUpgradeORDS` and `autoUpgradeAPEX` will be ignored if set. A new DB User for ORDS will be created to avoid conflict with the pre-provisioned one. This user will be + named, `ORDS_PUBLIC_USER_OPER` if `db.username` is either not specified or set to `ORDS_PUBLIC_USER`. + + Replace with the ADB Name and ensure that the `db.wallet.zip.service` is valid for your ADB Workload (e.g. _TP or _HIGH, etc.): + + ```bash + echo " + apiVersion: database.oracle.com/v1 + kind: OrdsSrvs + metadata: + name: ords-adb + namespace: ordsnamespace + spec: + image: container-registry.oracle.com/database/ords:24.1.1 + globalSettings: + database.api.enabled: true + encPrivKey: + secretName: prvkey + passwordKey: privateKey + poolSettings: + - poolName: adb + db.wallet.zip.service: _TP + dbWalletSecret: + secretName: adb-wallet + walletName: Wallet_.zip + restEnabledSql.active: true + feature.sdw: true + plsql.gateway.mode: proxied + db.username: ORDS_PUBLIC_USER_OPER + db.secret: + secretName: adb-db-auth-enc + passwordKey: password + db.adminUser: ADMIN + db.adminUser.secret: + secretName: adb-db-auth-enc + passwordKey: password" | kubectl apply -f - + ``` + latest container-registry.oracle.com/database/ords version, **24.1.1**, valid as of **30-May-2024** + +1. Watch the restdataservices resource until the status is **Healthy**: + ```bash + kubectl get ordssrvs ords-adb -w + ``` + + **NOTE**: If this is the first time pulling the ORDS image, it may take up to 5 minutes. If APEX + is being installed for the first time by the Operator, it may remain in the **Preparing** + status for an additional 5 minutes. + +### Test + +Open a port-forward to the ORDS service, for example: + +```bash +kubectl port-forward service/ords-adb 8443:8443 +``` + +Direct your browser to: `https://localhost:8443/ords/adb` + +## Conclusion + +This example has a single database pool, named `adb`. It is set to: + +* Not automatically restart when the configuration changes: `forceRestart` is not set. + The pod must be manually resarted for new configurations to be picked-up. +* Automatically install/update ORDS on startup, if required. This occurs due to the database being detected as an ADB. +* Automatically install/update APEX on startup, if required: This occurs due to the database being detected as an ADB. +* The ADB `ADMIN` user will be used to connect the ADB to install APEX/ORDS +* Use the ADB Wallet file to connect to the database: `db.wallet.zip.service: adbpoc_tp` and `dbWalletSecret` \ No newline at end of file diff --git a/docs/ordsservices/examples/adb_oraoper.md b/docs/ordsservices/examples/adb_oraoper.md new file mode 100644 index 00000000..b0872fb3 --- /dev/null +++ b/docs/ordsservices/examples/adb_oraoper.md @@ -0,0 +1,176 @@ +# Example: Autonomous Database using the OraOperator + +This example walks through using the **ORDS Controller** with a Containerised Oracle Database created by the **ADB Controller** in the same Kubernetes Cluster. + +When connecting to a mTLS enabled ADB while using the OraOperator to retreive the Wallet as is done in the example, it is currently not supported to have multiple, different databases supported by the single Ordssrvs resource. This is due to a requirement to set the `TNS_ADMIN` parameter at the Pod level ([#97](https://github.com/oracle/oracle-database-operator/issues/97)). + +### Cert-Manager and Oracle Database Operator installation + +Install the [Cert Manager](https://github.com/cert-manager/cert-manager/releases/download/v1.14.4/cert-manager.yaml) and the [Oracle Database Operator](https://github.com/oracle/oracle-database-operator) using the instractions in the Operator [README](https://github.com/oracle/oracle-database-operator/blob/main/README.md) file. + +### Setup Oracle Cloud Authorisation + +In order for the OraOperator to access the ADB, some pre-requisites are required, as detailed [here](https://github.com/oracle/oracle-database-operator/blob/main/docs/adb/ADB_PREREQUISITES.md). Either establish Instance Principles or create the required ConfigMap/Secret. This example uses the later: + +```bash +kubectl create configmap oci-cred \ +--from-literal=tenancy= \ +--from-literal=user= \ +--from-literal=fingerprint= \ +--from-literal=region= + +kubectl create secret generic oci-privatekey \ +--from-file=privatekey= +``` + +### ADB ADMIN Password Secret + +Create a Secret for the ADB Admin password: + +```bash +DB_PWD=$(echo "ORDSpoc_$(date +%H%S%M)") + +kubectl create secret generic adb-oraoper-db-auth \ + --from-literal=adb-oraoper-db-auth=${DB_PWD} +``` + +**NOTE**: When binding to the ADB in a later step, the OraOperator will change the ADB password to what is specified in the Secret. + +### Bind the OraOperator to the ADB + +1. Obtain the OCID of the ADB and set to an environment variable: + + ``` + export ADB_OCID= + ``` + +1. Create a manifest to bind to the ADB. + + ```bash + echo " + apiVersion: database.oracle.com/v1alpha1 + kind: AutonomousDatabase + metadata: + name: adb-oraoper + spec: + hardLink: false + ociConfig: + configMapName: oci-cred + secretName: oci-privatekey + details: + autonomousDatabaseOCID: $ADB_OCID + wallet: + name: adb-oraoper-tns-admin + password: + k8sSecret: + name: adb-oraoper-db-auth" | kubectl apply -f - + ``` + +1. Update the ADMIN Password: + +```bash + kubectl patch adb adb-oraoper --type=merge \ + -p '{"spec":{"details":{"adminPassword":{"k8sSecret":{"name":"adb-oraoper-db-auth"}}}}}' +``` + +1. Watch the `adb` resource until the STATE is **AVAILABLE**: + + ```bash + kubectl get adb/adb-oraoper -w + ``` + +### Create encrypted password + + +```bash +echo ${DB_PWD} adb-db-auth-enc +openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -pkeyopt rsa_keygen_pubexp:65537 > ca.k +openssl rsa -in ca.key -outform PEM -pubout -out public.pem +kubectl create secret generic prvkey --from-file=privateKey=ca.key -n ordsnamespace +openssl rsautl -encrypt -pubin -inkey public.pem -in adb-db-auth-enc |base64 > e_adb-db-auth-enc +kubectl create secret generic adb-oraoper-db-auth-enc --from-file=password=e_adb-db-auth-enc -n ordsnamespace +rm adb-db-auth-enc e_adb-db-auth-enc +``` + + + +### Create OrdsSrvs Resource + +1. Obtain the Service Name from the OraOperator + + ```bash + SERVICE_NAME=$(kubectl get adb adb-oraoper -o=jsonpath='{.spec.details.dbName}'_TP) + ``` + +1. Create a manifest for ORDS. + + As an ADB already maintains ORDS and APEX, `autoUpgradeORDS` and `autoUpgradeAPEX` will be ignored if set. A new DB User for ORDS will be created to avoid conflict with the pre-provisioned one. This user will be + named, `ORDS_PUBLIC_USER_OPER` if `db.username` is either not specified or set to `ORDS_PUBLIC_USER`. + + ```bash + echo " + apiVersion: database.oracle.com/v1 + kind: OrdsSrvs + metadata: + name: ords-adb-oraoper + namespace: ordsnamespace + spec: + image: container-registry.oracle.com/database/ords:24.1.1 + forceRestart: true + encPrivKey: + secretName: prvkey + passwordKey: privateKey + globalSettings: + database.api.enabled: true + poolSettings: + - poolName: adb-oraoper + db.connectionType: tns + db.tnsAliasName: $SERVICE_NAME + tnsAdminSecret: + secretName: adb-oraoper-tns-admin + restEnabledSql.active: true + feature.sdw: true + plsql.gateway.mode: proxied + db.username: ORDS_PUBLIC_USER_OPER + db.secret: + secretName: adb-oraoper-db-auth-enc + passwordKey: adb-oraoper-db-auth-enc + db.adminUser: ADMIN + db.adminUser.secret: + secretName: adb-oraoper-db-auth-enc + passwordKey: adb-oraoper-db-auth-enc" | kubectl apply -f - + ``` + latest container-registry.oracle.com/database/ords version, **24.1.1**, valid as of **30-May-2024** + +1. Watch the ordssrvs resource until the status is **Healthy**: + ```bash + kubectl get ordssrvs ords-adb-oraoper -n ordsnamespace -w + ``` + + **NOTE**: If this is the first time pulling the ORDS image, it may take up to 5 minutes. If APEX + is being installed for the first time by the Operator, it may remain in the **Preparing** + status for an additional 5 minutes. + + +### Test + +Open a port-forward to the ORDS service, for example: + +```bash +kubectl port-forward service/ords-adb-oraoper 8443:8443 +``` + +Direct your browser to: `https://localhost:8443/ords/adb-oraoper` + +## Conclusion + +This example has a single database pool, named `adb-oraoper`. It is set to: + +* Automatically restart when the configuration changes: `forceRestart: true` +* Automatically install/update ORDS on startup, if required. This occurs due to the database being detected as an ADB. +* Automatically install/update APEX on startup, if required: This occurs due to the database being detected as an ADB. +* The ADB `ADMIN` user will be used to connect the ADB to install APEX/ORDS +* Use a TNS connection string to connect to the database: `db.customURL: jdbc:oracle:thin:@//${CONN_STRING}` + The `tnsAdminSecret` Secret `adb-oraoper-tns-admin` was created by the OraOperator +* The `passwordKey` has been specified for both `db.secret` and `db.adminUser.secret` as `adb-oraoper-password` to match the OraOperator specification. +* The ADB `ADMIN` user will be used to connect the ADB to install APEX/ORDS \ No newline at end of file diff --git a/docs/ordsservices/examples/mongo_api.md b/docs/ordsservices/examples/mongo_api.md new file mode 100644 index 00000000..70391fbd --- /dev/null +++ b/docs/ordsservices/examples/mongo_api.md @@ -0,0 +1,160 @@ +# Example: Oracle API for MongoDB Support + +This example walks through using the **ORDSSRVS Controller** with a Containerised Oracle Database to enable MongoDB API Support. + + +### Cert-Manager and Oracle Database Operator installation + +Install the [Cert Manager](https://github.com/cert-manager/cert-manager/releases/download/v1.14.4/cert-manager.yaml) and the [Oracle Database Operator](https://github.com/oracle/oracle-database-operator) using the instractions in the Operator [README](https://github.com/oracle/oracle-database-operator/blob/main/README.md) file. + + +### Database Access + +This example assumes you have a running, accessible Oracle Database. For demonstration purposes, +the [Containerised Single Instance Database using the OraOperator](sidb_container.md) will be used. + +### Rest Enable a Schema + +In the database, create an ORDS-enabled user. As this example uses the [Containerised Single Instance Database using the OraOperator](sidb_container.md), the following was performed: + + +1. Connect to the database: + + ```bash + DB_PWD=$(kubectl get secrets sidb-db-auth --template='{{.data.password | base64decode}}') + POD_NAME=$(kubectl get pod -l "app=oraoper-sidb" -o custom-columns=NAME:.metadata.name --no-headers) + kubectl exec -it ${POD_NAME} -- sqlplus SYSTEM/${DB_PWD}@FREEPDB1 + ``` + +1. Create the User: + ```sql + create user MONGO identified by "My_Password1!"; + grant soda_app, create session, create table, create view, create sequence, create procedure, create job, + unlimited tablespace to MONGO; + -- Connect as new user + conn MONGO/My_Password1!@FREEPDB1; + exec ords.enable_schema; + ``` + +### Create encrypted secrets + +```bash +openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -pkeyopt rsa_keygen_pubexp:65537 > ca.k +openssl rsa -in ca.key -outform PEM -pubout -out public.pem +kubectl create secret generic prvkey --from-file=privateKey=ca.key -n ordsnamespace +openssl rsautl -encrypt -pubin -inkey public.pem -in sidb-db-auth-enc |base64 > e_sidb-db-auth-enc +kubectl create secret generic sidb-db-auth-enc --from-file=password=e_sidb-db-auth-enc -n ordsnamespace +rm sidb-db-auth-enc e_sidb-db-auth-enc + +``` + +### Create ordssrvs Resource + +1. Retrieve the Connection String from the containerised SIDB. + + ```bash + CONN_STRING=$(kubectl get singleinstancedatabase oraoper-sidb \ + -o jsonpath='{.status.pdbConnectString}') + + echo $CONN_STRING + ``` + +1. Create a manifest for ORDS. + + As the DB in the Free image does not contain ORDS (or APEX), the following additional keys are specified for the pool: + * `autoUpgradeORDS` - Boolean; when true the ORDS will be installed/upgraded in the database + * `db.adminUser` - User with privileges to install, upgrade or uninstall ORDS in the database (SYS). + * `db.adminUser.secret` - Secret containing the password for `db.adminUser` (created in the first step) + + The `db.username` will be used as the ORDS schema in the database during the install/upgrade process (ORDS_PUBLIC_USER). + + ```bash + echo " + apiVersion: database.oracle.com/v4 + kind: ordssrvs + metadata: + name: ords-sidb + namespace: ordsnamespace + spec: + image: container-registry.oracle.com/database/ords:24.1.1 + forceRestart: true + encPrivKey: + secretName: prvkey + passwordKey: privateKey + globalSettings: + database.api.enabled: true + mongo.enabled: true + poolSettings: + - poolName: default + autoUpgradeORDS: true + restEnabledSql.active: true + plsql.gateway.mode: direct + jdbc.MaxConnectionReuseCount: 5000 + jdbc.MaxConnectionReuseTime: 900 + jdbc.SecondsToTrustIdleConnection: 1 + jdbc.InitialLimit: 100 + jdbc.MaxLimit: 100 + db.connectionType: customurl + db.customURL: jdbc:oracle:thin:@//${CONN_STRING} + db.username: ORDS_PUBLIC_USER + db.secret: + secretName: sidb-db-auth-enc + db.adminUser: SYS + db.adminUser.secret: + secretName: sidb-db-auth-enc" | kubectl apply -f - + ``` + latest container-registry.oracle.com/database/ords version, **24.1.1**, valid as of **30-May-2024** + +1. Watch the restdataservices resource until the status is **Healthy**: + ```bash + kubectl get ordssrvs ords-sidb -w + ``` + + **NOTE**: If this is the first time pulling the ORDS image, it may take up to 5 minutes. If APEX + is being installed for the first time by the Operator, it may remain in the **Preparing** + status for an additional 5 minutes. + + You can watch the APEX/ORDS Installation progress by running: + + ```bash + POD_NAME=$(kubectl get pod -l "app.kubernetes.io/instance=ords-sidb" -o custom-columns=NAME:.metadata.name -n ordsnamespace --no-headers) + + kubectl logs ${POD_NAME} -c ords-sidb-init -n ordsnamespace -f + ``` + +### Test + +1. Open a port-forward to the MongoAPI service, for example: + ```bash + kubectl port-forward service/ords-sidb 27017:27017 -n ordsnamespace + ``` + +1. Connect to ORDS using the MongoDB shell: + ```bash + mongosh --tlsAllowInvalidCertificates 'mongodb://MONGO:My_Password1!@localhost:27017/MONGO?authMechanism=PLAIN&authSource=$external&tls=true&retryWrites=false&loadBalanced=true' + ``` + +1. Insert some data: + ```txt + db.createCollection('emp'); + db.emp.insertOne({"name":"Blake","job": "Intern","salary":30000}); + db.emp.insertOne({"name":"Miller","job": "Programmer","salary": 70000}); + db.emp.find({"name":"Miller"}); + ``` + +## Conclusion + +This example has a single database pool, named `default`. It is set to: + +* Automatically restart when the configuration changes: `forceRestart: true` +* Automatically install/update ORDS on startup, if required: `autoUpgradeORDS: true` +* Use a basic connection string to connect to the database: `db.customURL: jdbc:oracle:thin:@//${CONN_STRING}` +* The `passwordKey` has been ommitted from both `db.secret` and `db.adminUser.secret` as the password was stored in the default key (`password`) +* The MongoAPI service has been enabled: `mongo.enabled: true` +* The MongoAPI service will default to port: `27017` as the property: `mongo.port` has been left undefined +* A number of JDBC parameters were set at the pool level for achieving high performance: + * `jdbc.MaxConnectionReuseCount: 5000` + * `jdbc.MaxConnectionReuseTime: 900` + * `jdbc.SecondsToTrustIdleConnection: 1` + * `jdbc.InitialLimit: 100` + * `jdbc.MaxLimit: 100` diff --git a/docs/ordsservices/examples/multi_pool.md b/docs/ordsservices/examples/multi_pool.md new file mode 100644 index 00000000..21c5f24d --- /dev/null +++ b/docs/ordsservices/examples/multi_pool.md @@ -0,0 +1,203 @@ +# Example: Multipool, Multidatabase using a TNS Names file + +This example walks through using the **ORDSSRVS Operator** with multiple databases using a TNS Names file. +Keep in mind that all pools are running in the same Pod, therefore, changing the configuration of one pool will require +a recycle of all pools. + +### Cert-Manager and Oracle Database Operator installation + +Install the [Cert Manager](https://github.com/cert-manager/cert-manager/releases/download/v1.14.4/cert-manager.yaml) and the [Oracle Database Operator](https://github.com/oracle/oracle-database-operator) using the instractions in the Operator [README](https://github.com/oracle/oracle-database-operator/blob/main/README.md) file. + +### TNS_ADMIN Secret + +Create a Secret with the contents of the TNS_ADMIN directory. This can be a single `tnsnames.ora` file or additional files such as `sqlnet.ora` or `ldap.ora`. +The example shows using a `$TNS_ADMIN` enviroment variable which points to a directory with valid TNS_ADMIN files. + +To create a secret with all files in the TNS_ADMIN directory: +```bash +kubectl create secret generic multi-tns-admin \ + --from-file=$TNS_ADMIN +``` + +To create a secret with just the tnsnames.ora file: +```bash +kubectl create secret generic multi-tns-admin \ + --from-file=$TNS_ADMIN/tnsnames.ora +``` + +In this example, 4 PDBs will be connected to and the example `tnsnames.ora` file contents are as below: +```text +PDB1=(DESCRIPTION=(ADDRESS_LIST=(LOAD_BALANCE=on)(ADDRESS=(PROTOCOL=TCP)(HOST=10.10.0.1)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=PDB1))) + +PDB2=(DESCRIPTION=(ADDRESS_LIST=(LOAD_BALANCE=on)(ADDRESS=(PROTOCOL=TCP)(HOST=10.10.0.2)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=PDB2))) + +PDB3=(DESCRIPTION=(ADDRESS_LIST=(LOAD_BALANCE=on)(ADDRESS=(PROTOCOL=TCP)(HOST=10.10.0.3)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=PDB3))) + +PDB4=(DESCRIPTION=(ADDRESS_LIST=(LOAD_BALANCE=on)(ADDRESS=(PROTOCOL=TCP)(HOST=10.10.0.4)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=PDB4))) +``` + +### PRIVATE KEY SECRET + +Secrets are encrypted using openssl rsa algorithm. Create public and private key. +Use private key to create a secret. + +```bash +openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -pkeyopt rsa_keygen_pubexp:65537 > ca.key +openssl rsa -in ca.key -outform PEM -pubout -out public.pem +kubectl create secret generic prvkey --from-file=privateKey=ca.key -n ordsnamespace +``` + +### ORDS_PUBLIC_USER Secret + +Create a Secret for each of the databases `ORDS_PUBLIC_USER` user. +If multiple databases use the same password, the same secret can be re-used. + +The following secret will be used for PDB1: + +```bash +echo "THIS_IS_A_PASSWORD" > ordspwdfile +openssl rsautl -encrypt -pubin -inkey public.pem -in ordspwdfile |base64 > e_ordspwdfile +kubectl create secret generic pdb1-ords-auth-enc --from-file=password=e_ordspwdfile -n ordsnamespace +rm ordspwdfile e_ordspwdfile +``` + +The following secret will be used for PDB2: + +```bash +echo "THIS_IS_A_PASSWORD" > ordspwdfile +openssl rsautl -encrypt -pubin -inkey public.pem -in ordspwdfile |base64 > e_ordspwdfile +kubectl create secret generic pdb2-ords-auth-enc --from-file=password=e_ordspwdfile -n ordsnamespace +rm ordspwdfile e_ordspwdfile +``` + +The following secret will be used for PDB3 and PDB4: + +```bash +echo "THIS_IS_A_PASSWORD" > ordspwdfile +openssl rsautl -encrypt -pubin -inkey public.pem -in ordspwdfile |base64 > e_ordspwdfile +kubectl create secret generic multi-ords-auth-enc --from-file=password=e_ordspwdfile -n ordsnamespace +rm ordspwdfile e_ordspwdfile +``` + +### Privileged Secret (*Optional) + +If taking advantage of the [AutoUpgrade](../autoupgrade.md) functionality, create a secret for a user with the privileges to modify the ORDS and/or APEX schemas. + +In this example, only PDB1 will be set for [AutoUpgrade](../autoupgrade.md), the other PDBs already have APEX and ORDS installed. + +```bash + + + +echo "THIS_IS_A_PASSWORD" > syspwdfile +openssl rsautl -encrypt -pubin -inkey public.pem -in ordspwdfile |base64 > e_syspwdfile +kubectl create secret generic pdb1-priv-auth-enc --from-file=password=e_syspwdfile -n ordsnamespace +rm syspwdfile e_syspwdfile + +kubectl create secret generic pdb1-priv-auth \ + --from-literal=password=pdb1-battery-staple +``` + +### Create OrdsSrvs Resource + +1. Create a manifest for ORDS. + + ```bash + echo " + apiVersion: database.oracle.com/v1 + kind: OrdsSrvs + metadata: + name: ords-multi-pool + namespace: ordsnamespace + spec: + image: container-registry.oracle.com/database/ords:24.1.1 + forceRestart: true + encPrivKey: + secretName: prvkey + passwordKey: privateKey + globalSettings: + database.api.enabled: true + poolSettings: + - poolName: pdb1 + autoUpgradeORDS: true + autoUpgradeAPEX: true + db.connectionType: tns + db.tnsAliasName: PDB1 + tnsAdminSecret: + secretName: multi-tns-admin + restEnabledSql.active: true + feature.sdw: true + plsql.gateway.mode: proxied + db.username: ORDS_PUBLIC_USER + db.secret: + secretName: pdb1-ords-auth-enc + db.adminUser: SYS + db.adminUser.secret: + secretName: pdb1-priv-auth-enc + - poolName: pdb2 + db.connectionType: tns + db.tnsAliasName: PDB2 + tnsAdminSecret: + secretName: multi-tns-admin + restEnabledSql.active: true + feature.sdw: true + plsql.gateway.mode: proxied + db.username: ORDS_PUBLIC_USER + db.secret: + secretName: pdb2-ords-auth-enc + - poolName: pdb3 + db.connectionType: tns + db.tnsAliasName: PDB3 + tnsAdminSecret: + secretName: multi-tns-admin + restEnabledSql.active: true + feature.sdw: true + plsql.gateway.mode: proxied + db.username: ORDS_PUBLIC_USER + db.secret: + secretName: multi-ords-auth-enc + - poolName: pdb4 + db.connectionType: tns + db.tnsAliasName: PDB4 + tnsAdminSecret: + secretName: multi-tns-admin + restEnabledSql.active: true + feature.sdw: true + plsql.gateway.mode: proxied + db.username: ORDS_PUBLIC_USER + db.secret: + secretName: multi-ords-auth-enc" | kubectl apply -f - + ``` + latest container-registry.oracle.com/database/ords version, **24.1.1**, valid as of **30-May-2024** + +1. Watch the ordssrvs resource until the status is **Healthy**: + ```bash + kubectl get OrdsSrvs ords-multi-pool -n ordsnamespace -w + ``` + + **NOTE**: If this is the first time pulling the ORDS image, it may take up to 5 minutes. As APEX + is being installed for the first time by the Operator into PDB1, it will remain in the **Preparing** + status for an additional 5-10 minutes. + +### Test + +Open a port-forward to the ORDS service, for example: + +```bash +kubectl port-forward service/ords-multi-pool -n ordsnamespace 8443:8443 +``` + +1. For PDB1, direct your browser to: `https://localhost:8443/ords/pdb1` +1. For PDB2, direct your browser to: `https://localhost:8443/ords/pdb2` +1. For PDB3, direct your browser to: `https://localhost:8443/ords/pdb3` +1. For PDB4, direct your browser to: `https://localhost:8443/ords/pdb4` + +## Conclusion + +This example has multiple pools, named `pdb1`, `pdb2`, `pdb3`, and `pdb4`. + +* They all share the same `tnsAdminSecret` to connect using thier individual `db.tnsAliasName` +* They will all automatically restart when the configuration changes: `forceRestart: true` +* Only the `pdb1` pool will automatically install/update ORDS on startup, if required: `autoUpgradeORDS: true` +* Only the `pdb1` pool will automatically install/update APEX on startup, if required: `autoUpgradeAPEX: true` +* The `passwordKey` has been ommitted from both `db.secret` and `db.adminUser.secret` as the password was stored in the default key (`password`) diff --git a/docs/ordsservices/examples/sidb_container.md b/docs/ordsservices/examples/sidb_container.md new file mode 100644 index 00000000..804ecca4 --- /dev/null +++ b/docs/ordsservices/examples/sidb_container.md @@ -0,0 +1,154 @@ +# Example: Containerised Single Instance Database using the OraOperator + +This example walks through using the **ORDSSRVS Controller** with a Containerised Oracle Database created by the **SIDB Controller** in the same Kubernetes Cluster. + +### Cert-Manager and Oracle Database Operator installation + +Install the [Cert Manager](https://github.com/cert-manager/cert-manager/releases/download/v1.14.4/cert-manager.yaml) and the [Oracle Database Operator](https://github.com/oracle/oracle-database-operator) using the instractions in the Operator [README](https://github.com/oracle/oracle-database-operator/blob/main/README.md) file. + + +### Deploy a Containerised Oracle Database + +1. Create a Secret for the Database password: + + ```bash + DB_PWD=$(echo "ORDSpoc_$(date +%H%S%M)") + + kubectl create secret generic sidb-db-auth \ + --from-literal=password=${DB_PWD} + ``` +1. Create a manifest for the containerised Oracle Database. + + The POC uses an Oracle Free Image, but other versions may be subsituted; review the OraOperator Documentation for details on the manifests. + + ```bash + echo " + apiVersion: database.oracle.com/v1alpha1 + kind: SingleInstanceDatabase + metadata: + name: oraoper-sidb + spec: + replicas: 1 + image: + pullFrom: container-registry.oracle.com/database/free:23.4.0.0 + prebuiltDB: true + sid: FREE + edition: free + adminPassword: + secretName: sidb-db-auth + secretKey: password + pdbName: FREEPDB1" | kubectl apply -f - + ``` + latest container-registry.oracle.com/database/free version, **23.4.0.0**, valid as of **2-May-2024** + +1. Watch the `singleinstancedatabases` resource until the database status is **Healthy**: + + ```bash + kubectl get singleinstancedatabases/oraoper-sidb -w + ``` + + **NOTE**: If this is the first time pulling the free database image, it may take up to 15 minutes for the database to become available. + +### Create encryped secret + +```bash + +openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -pkeyopt rsa_keygen_pubexp:65537 > ca.key +openssl rsa -in ca.key -outform PEM -pubout -out public.pem +kubectl create secret generic prvkey --from-file=privateKey=ca.key -n ordsnamespace + +echo "${DB_PWD}" > sidb-db-auth +openssl rsautl -encrypt -pubin -inkey public.pem -in sidb-db-auth |base64 > e_sidb-db-auth +kubectl create secret generic sidb-db-auth-enc --from-file=password=e_sidb-db-auth -n ordsnamespace +rm sidb-db-auth e_sidb-db-auth + + +``` + + +### Create RestDataServices Resource + +1. Retrieve the Connection String from the containerised SIDB. + + ```bash + CONN_STRING=$(kubectl get singleinstancedatabase oraoper-sidb \ + -o jsonpath='{.status.pdbConnectString}') + + echo $CONN_STRING + ``` + +1. Create a manifest for ORDS. + + As the DB in the Free image does not contain ORDS (or APEX), the following additional keys are specified for the pool: + * `autoUpgradeORDS` - Boolean; when true the ORDS will be installed/upgraded in the database + * `autoUpgradeAPEX` - Boolean; when true the APEX will be installed/upgraded in the database + * `db.adminUser` - User with privileges to install, upgrade or uninstall ORDS in the database (SYS). + * `db.adminUser.secret` - Secret containing the password for `db.adminUser` (created in the first step) + + The `db.username` will be used as the ORDS schema in the database during the install/upgrade process (ORDS_PUBLIC_USER). + + ```bash + echo " + apiVersion: database.oracle.com/v1 + kind: OrdsSrvs + metadata: + name: ords-sidb + namespace: ordsnamespace + spec: + image: container-registry.oracle.com/database/ords:24.1.1 + forceRestart: true + globalSettings: + database.api.enabled: true + poolSettings: + - poolName: default + autoUpgradeORDS: true + autoUpgradeAPEX: true + restEnabledSql.active: true + plsql.gateway.mode: direct + db.connectionType: customurl + db.customURL: jdbc:oracle:thin:@//${CONN_STRING} + db.username: ORDS_PUBLIC_USER + db.secret: + secretName: sidb-db-auth-enc + db.adminUser: SYS + db.adminUser.secret: + secretName: sidb-db-auth-enc" | kubectl apply -f - + ``` + latest container-registry.oracle.com/database/ords version, **24.1.1**, valid as of **30-May-2024** + +1. Watch the ordssrvs resource until the status is **Healthy**: + ```bash + kubectl get ordssrvs ords-sidb -n ordsnamespace -w + ``` + + **NOTE**: If this is the first time pulling the ORDS image, it may take up to 5 minutes. If APEX + is being installed for the first time by the Operator, it may remain in the **Preparing** + status for an additional 5 minutes. + + You can watch the APEX/ORDS Installation progress by running: + + ```bash + POD_NAME=$(kubectl get pod -l "app.kubernetes.io/instance=ords-sidb" -n ordsnamespace -o custom-columns=NAME:.metadata.name --no-headers) + + kubectl logs ${POD_NAME} -c ords-sidb-init -n ordsnamespace -f + ``` + +### Test + +Open a port-forward to the ORDS service, for example: + +```bash +kubectl port-forward service/ords-sidb -n ordsnamespace 8443:8443 +``` + +Direct your browser to: `https://localhost:8443/ords` + +## Conclusion + +This example has a single database pool, named `default`. It is set to: + +* Automatically restart when the configuration changes: `forceRestart: true` +* Automatically install/update ORDS on startup, if required: `autoUpgradeORDS: true` +* Automatically install/update APEX on startup, if required: `autoUpgradeAPEX: true` +* Use a basic connection string to connect to the database: `db.customURL: jdbc:oracle:thin:@//${CONN_STRING}` +* The `passwordKey` has been ommitted from both `db.secret` and `db.adminUser.secret` as the password was stored in the default key (`password`) diff --git a/docs/ordsservices/usecase01/create_mong_schema.sql b/docs/ordsservices/usecase01/create_mong_schema.sql new file mode 100644 index 00000000..a00ee441 --- /dev/null +++ b/docs/ordsservices/usecase01/create_mong_schema.sql @@ -0,0 +1,9 @@ +drop user MONGO cascade; +set echo on +set head on +create user MONGO identified by "My_Password1!"; +grant soda_app, create session, create table, create view, create sequence, create procedure, create job, +unlimited tablespace to MONGO; +conn MONGO/My_Password1!@158.180.233.248:30001/FREEPDB1 +exec ords.enable_schema; +exit; diff --git a/docs/ordsservices/usecase01/help b/docs/ordsservices/usecase01/help new file mode 100644 index 00000000..8b137891 --- /dev/null +++ b/docs/ordsservices/usecase01/help @@ -0,0 +1 @@ + diff --git a/docs/ordsservices/usecase01/makefile b/docs/ordsservices/usecase01/makefile new file mode 100644 index 00000000..76b47210 --- /dev/null +++ b/docs/ordsservices/usecase01/makefile @@ -0,0 +1,778 @@ +# +# Copyright (c) 2006, 2024, Oracle and/or its affiliates. +# +# +# NAME +# makefile: +# This makefile helps to set up multipool and sidb cases +# edit the following variables with your system information +# and execute make help to list the list of avilable targets +# + +export PDB1=pdb1 +export PDB2=pdb2 +export TNS1=(DESCRIPTION=(CONNECT_TIMEOUT=90)(RETRY_COUNT=30)(RETRY_DELAY=10)(TRANSPORT_CONNECT_TIMEOUT=70)(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan12.testrac.com)(PORT=1521)(IP=V4_ONLY))(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan12.testrac.com)(PORT=1521)(IP=V4_ONLY))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=$(PDB1)))) +export TNS2=(DESCRIPTION=(CONNECT_TIMEOUT=90)(RETRY_COUNT=30)(RETRY_DELAY=10)(TRANSPORT_CONNECT_TIMEOUT=70)(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan12.testrac.com)(PORT=1521)(IP=V4_ONLY))(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan34.testrac.com)(PORT=1521)(IP=V4_ONLY))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=$(PDB2)))) +export SIDB_PASSWORD=....write password here .... +export PDB1_PWD=.....write password here.... +export PDB2_PWD=.....write password.... +export ORDS_MULTI_POOL_PWD=....write password here.... +export USER_CONTAINER_REGISTRY=username@oracle.com <--Your OCR account +export OPRNAMESPACE=oracle-database-operator-system +export ORDSNAMESPACE=ordsnamespace + + +# +# DESCRIPTION +# Main makefile - see target table +# +# | Target | Description | +# +-------------+--------------------------------------------------------+ +# | step0a | create_ordsnamespace.yaml | +# +-------------+--------------------------------------------------------+ +# | step1a | setup certmaneger | +# +-------------+--------------------------------------------------------+ +# | step2a | setup operator oracle-database-operator.yaml | +# +-------------+--------------------------------------------------------+ +# | step3a | default scoped deployment default-ns-role-binding.yaml | +# +-------------+--------------------------------------------------------+ +# | step4a | node - persistent volume - storage class for the db | +# +-------------+--------------------------------------------------------+ +# | step5a | setup secrets | +# +----------------------------------------------------------------------+ +# | step6a | setup secrets for OCR | +# +----------------------------------------------------------------------+ +# | step7a | setup sidb | +# +----------------------------------------------------------------------+ +# | step8a | â­Setup REST SERVER â­ | +# +-------------+--------------------------------------------------------+ +# +# step[1-7]a are required to start mongodb API rest server +# +# step[9-11] test mongo API +# +-------------+--------------------------------------------------------+ +# | step9 | configure a mongo db user on sidb | +# +-------------+--------------------------------------------------------+ +# | step10 | â­Setup REST SERVER FOR MONGO API â­ | +# +-------------+--------------------------------------------------------+ +# | step11 | Test Mongo API | +# +-------------+--------------------------------------------------------+ +# +# step[12- ] test multi tns configuration +# +-------------+--------------------------------------------------------+ +# | step12 | create tns secret | +# +-------------+--------------------------------------------------------+ +# | step13 | create passwords secret | +# +-------------+--------------------------------------------------------+ +# | step14 | â­SetupMulti Rest Server â­ | +# +-------------+--------------------------------------------------------+ +# + + + +export WATCHLIST=$(OPRNAMESPACE),$(ORDSNAMESPACE) +export CREATE_SINGLEINSTANCE=create_singleinstance_db.yaml +export CERTMANAGER=https://github.com/jetstack/cert-manager/releases/latest/download/cert-manager.yaml +export SIDB_SECRET=sidb-db-auth +export ORDS_SECRET=ords-db-auth +export MULTI_ORDS_AUTH_SECRET=multi-ords-auth-enc +export PDB1_PRIV_AUTH_SECRET=pdb1-priv-auth-enc +export PDB2_PRIV_AUTH_SECRET=pdb2-priv-auth-enc + + +export SIDB_IMAGE=container-registry.oracle.com/database/free:23.4.0.0 +export ORDS_IMAGE=container-registry.oracle.com/database/ords:24.1.0 +export ORDS_IMAGE.1=container-registry.oracle.com/database/ords:24.1.1 +export SECRET_CONTAINER_REGISTRY=oracle-container-registry-secret +export ORACLE_CONTAINER_REGISTRY=container-registry.oracle.com +export REST_SERVER_NAME=ords-sidb +export REST_SERVER_NAME_MONGO=ords-sidb-mongo +export MONGOSH=mongosh-2.3.1-linux-x64 +export KIND=OrdsSrvs + +export TNSNAMES=./tnsnames.ora +export TNSADMIN=`pwd`/tnsadmin +export PRVKEY=ca.key +export PUBKEY=public.pem + +## CMD SECTION## +export KUBECTL=/usr/local/go/bin/kubectl +export DIFF=/usr/bin/diff +export MAKE=/usr/bin/make +export CURL=/usr/bin/curl +export TAR=/usr/bin/tar +export OPENSSL=/usr/bin/openssl + +## YAML AND OTHER FILES ## +export CREATE_ORDSNAMESPACE=create_$(ORDSNAMESPACE).yaml +export DEFAULT_NAMESPACE_SCOPE=default-ns-role-binding.yaml +export RST_NAMESPACE_SCOPE=ords-ns-role-binding.yaml +export ORACLE_OPERATOR_YAML=../../../oracle-database-operator.yaml +export NODE_RBAC=node-rbac.yaml +export STORAGE_CLASS_RBAC=storage-class-rbac.yaml +export PERSISTENT_VOLUME_RBAC=persistent-volume-rbac.yaml +export SIDB_CREATION=sidb_create.yaml +export SECRET_CONTAINER_REGISTRY_SCRIPT=create_registry_secret.sh +export REST_SERVER_CREATION=rest_server_creation.yaml +export REST_SERVER_CREATION_MONGO=rest_server_creation_mongo.yaml +export MULTISRV_MANIFEST=create_multisrv.yaml +export MONGOORADBUSER=MONGO + + +MAKEFILE=./makefile +.ONESHELL: + +define manpage +@printf "\n" +@printf "\033[7m%s\033[0m \033[7m%s\033[0m \033[7m%s\033[0m\n" "TARGET " "DESCRIPTION " "YAML FILE " +@printf "%s %s %s\n" "---------" " --------------------------------------------------" "--------------------------------------" +@printf "%-40s %+20s \033[1m %s\033[0m\n" "step0/a/d setup new namespace" " " "$(CREATE_ORDSNAMESPACE)" +@printf "%-40s %+20s \033[1m %s\033[0m\n" "step1/a/d setup certmaneger " " " "$(CERTMANAGER)" +@printf "%-40s %+20s \033[1m %s\033[0m\n" "step2/a/d setup operator" " " "$(shell basename $(ORACLE_OPERATOR_YAML))" +@printf "%-40s %+20s \033[1m %s\033[0m\n" "step3/a/d default scoped deployment" " " "$(DEFAULT_NAMESPACE_SCOPE)" +@printf "%-40s %+20s \033[1m %s\033[0m\n" " ords scoped deployment" " " "$(RST_NAMESPACE_SCOPE)" +@printf "%-40s %+20s \033[1m %s\033[0m\n" "step4/a/d node rbac" " " "$(NODE_RBAC)" +@printf "%-40s %+20s \033[1m %s\033[0m\n" " storage class " " " "$(STORAGE_CLASS_RBAC)" +@printf "%-40s %+20s \033[1m %s\033[0m\n" " persistent volume " " " "$(PERSISTENT_VOLUME_RBAC)" +@printf "%-40s %+20s \033[1m %s\033[0m\n" "step5/a/d setup db secret" " " "n/a" +@printf "%-40s %+20s \033[1m %s\033[0m\n" "step6/a/d setup registry secret" " " "$(SECRET_CONTAINER_REGISTRY_SCRIPT)" +@printf "%-40s %+20s \033[1m %s\033[0m\n" "step7/a/d setup sidb " " " "$(SIDB_CREATION)" +@printf "================================================\n" +@printf "%-40s %+20s \033[1m %s\033[0m\n" "step8/a/d setup RestServer " " " "$(REST_SERVER_CREATION)" +@printf "%-40s %+20s \033[1m %s\033[0m\n" "step9/-/- configure " " " "Mongo ora db user:$(MONGOORADBUSER)" +@printf "%-40s %+20s \033[1m %s\033[0m\n" "step10/a/d setup RestServer Mongo " " " "$(REST_SERVER_CREATION_MONGO)" +@printf "%-40s %+20s \033[1m %s\033[0m\n" "step11/-/- test mongodb API " " " "----" +@printf "================================================\n" +@printf "%-40s %+20s \033[1m %s\033[0m\n" "step12/a/d create secret for tnsadmin " " " "$(TNSADMIN)" +@printf "%-40s %+20s \033[1m %s\033[0m\n" "step13/a/d create secrets for adminusers" " " "---" +@printf "%-40s %+20s \033[1m %s\033[0m\n" "step14/a/d/e setup Multi Ords services " " " "---" +@printf "================================================\n" +@printf "%-40s %+20s \033[1m %s\033[0m\n" "diagordsinit" "" "🔬dump initpod logs" + + + +@printf "================================================\n" +@printf " a=apply d=delete âš¡e=generate error âš¡\n" +@printf "\n" +endef + + +help:man +man: + $(call manpage) + +define namespace +cat< $(CREATE_ORDSNAMESPACE) +#apiVersion: v1 +#kind: Namespace +#metadata: +# labels: +# control-plane: controller-manager +# name: $(2) +EOF +$(KUBECTL) $(1) -f $(CREATE_ORDSNAMESPACE) +$(KUBECTL) get namespace +endef + +step0: + $(call namespace,$(ACTION),$(ORDSNAMESPACE)) +step0a: + $(MAKE) -f $(MAKEFILE) step0 ACTION=apply +step0d: + $(MAKE) -f $(MAKEFILE) step0 ACTION=delete + +step1: + $(KUBECTL) $(ACTION) -f $(CERTMANAGER) +step1a: + $(MAKE) -f $(MAKEFILE) ACTION=apply step1 + $(KUBECTL) get pod -n cert-manager +step1d: + $(MAKE) -f $(MAKEFILE) ACTION=delete step1 + + +define setwatchnamespace +@echo "Setting watch namespace list: $(WATCHLIST)" +sed 's/value: ""/value: "$(WATCHLIST)"/g' $(ORACLE_OPERATOR_YAML) > `basename $(ORACLE_OPERATOR_YAML)` +$(KUBECTL) $(1) -f `basename $(ORACLE_OPERATOR_YAML)` +$(DIFF) $(ORACLE_OPERATOR_YAML) `basename $(ORACLE_OPERATOR_YAML)` +$(KUBECTL) get pods -n $(OPRNAMESPACE) +endef + +step2: + $(call setwatchnamespace,$(ACTION)) +step2a: + $(MAKE) -f $(MAKEFILE) ACTION=apply step2 +step2d: + $(MAKE) -f $(MAKEFILE) ACTION=delete step2 + + +define namespacescpe +cat<$(RST_NAMESPACE_SCOPE) +#apiVersion: rbac.authorization.k8s.io/v1 +#kind: RoleBinding +#metadata: +# name: $(ORDSNAMESPACE)-rolebinding +# namespace: $(ORDSNAMESPACE) +#roleRef: +# apiGroup: rbac.authorization.k8s.io +# kind: ClusterRole +# name: oracle-database-operator-manager-role +#subjects: +#- kind: ServiceAccount +# name: default +# namespace: oracle-database-operator-system +EOF + +cat< $(DEFAULT_NAMESPACE_SCOPE) +#apiVersion: rbac.authorization.k8s.io/v1 +#kind: RoleBinding +#metadata: +# name: oracle-database-operator-oracle-database-operator-manager-rolebinding +# namespace: default +#roleRef: +# apiGroup: rbac.authorization.k8s.io +# kind: ClusterRole +# name: oracle-database-operator-manager-role +#subjects: +#- kind: ServiceAccount +# name: default +# namespace: oracle-database-operator-system +EOF + +$(KUBECTL) $(1) -f $(RST_NAMESPACE_SCOPE) +$(KUBECTL) $(1) -f $(DEFAULT_NAMESPACE_SCOPE) +$(KUBECTL) get RoleBinding -n $(ORDSNAMESPACE) + +endef + +step3: + $(call namespacescpe,$(ACTION)) + +step3a: + $(MAKE) -f $(MAKEFILE) ACTION=apply step3 + +step3d: + $(MAKE) -f $(MAKEFILE) ACTION=delete step3 + + +export NODE_RBAC=node-rbac.yaml +export STORAGE_CLASS_RBAC=storage-class-rbac.yaml +export PERSISTENT_VOLUME_RBAC=persistent-volume-rbac.yaml + + +define persistenvolume + +cat<$(NODE_RBAC) +#--- +#apiVersion: rbac.authorization.k8s.io/v1 +#kind: ClusterRole +#metadata: +# name: oracle-database-operator-manager-role-node +#rules: +#- apiGroups: +# - "" +# resources: +# - nodes +# verbs: +# - list +# - watch +#--- +#apiVersion: rbac.authorization.k8s.io/v1 +#kind: ClusterRoleBinding +#metadata: +# name: oracle-database-operator-manager-role-node-cluster-role-binding +#roleRef: +# apiGroup: rbac.authorization.k8s.io +# kind: ClusterRole +# name: oracle-database-operator-manager-role-node +#subjects: +#- kind: ServiceAccount +# name: default +# namespace: oracle-database-operator-system +EOF + +cat<$(STORAGE_CLASS_RBAC) +#--- +#apiVersion: rbac.authorization.k8s.io/v1 +#kind: ClusterRole +#metadata: +# name: oracle-database-operator-manager-role-storage-class +#rules: +#- apiGroups: +# - storage.k8s.io +# resources: +# - storageclasses +# verbs: +# - get +# - list +# - watch +#--- +#apiVersion: rbac.authorization.k8s.io/v1 +#kind: ClusterRoleBinding +#metadata: +# name: oracle-database-operator-manager-role-storage-class-cluster-role-binding +#roleRef: +# apiGroup: rbac.authorization.k8s.io +# kind: ClusterRole +# name: oracle-database-operator-manager-role-storage-class +#subjects: +#- kind: ServiceAccount +# name: default +# namespace: oracle-database-operator-system +#--- +EOF + +cat<$(PERSISTENT_VOLUME_RBAC) +# +#apiVersion: rbac.authorization.k8s.io/v1 +#kind: ClusterRole +#metadata: +# name: oracle-database-operator-manager-role-persistent-volume +#rules: +#- apiGroups: +# - "" +# resources: +# - persistentvolumes +# verbs: +# - get +# - list +# - watch +#--- +#apiVersion: rbac.authorization.k8s.io/v1 +#kind: ClusterRoleBinding +#metadata: +# name: oracle-database-operator-manager-role-persistent-volume-cluster-role-binding +#roleRef: +# apiGroup: rbac.authorization.k8s.io +# kind: ClusterRole +# name: oracle-database-operator-manager-role-persistent-volume +#subjects: +#- kind: ServiceAccount +# name: default +# namespace: oracle-database-operator-system +#--- +# +EOF + +$(KUBECTL) $(1) -f $(NODE_RBAC) +$(KUBECTL) $(1) -f $(STORAGE_CLASS_RBAC) +$(KUBECTL) $(1) -f $(PERSISTENT_VOLUME_RBAC) + +endef + +step4: + $(call persistenvolume,$(ACTION)) +step4a: + $(MAKE) -f $(MAKEFILE) ACTION=apply step4 +step4d: + $(MAKE) -f $(MAKEFILE) ACTION=delete step4 + + +export SYSPWDFILE1=syspwdfile +export ORDPWDFILE=ordspwdfile +export SIDB_PASSWORD_FILE=sidbpasswordfile + +export PRVKEY=ca.key +export PUBKEY=public.pem +export OPENSSL=/usr/bin/openssl + +step5a: + echo $(SIDB_PASSWORD) > $(SIDB_PASSWORD_FILE) + - $(KUBECTL) delete secret pubkey -n ${ORDSNAMESPACE} + - $(KUBECTL) delete secret prvkey -n ${ORDSNAMESPACE} + - $(KUBECTL) delete secret $(SIDB_SECRET) -n ${ORDSNAMESPACE} + - $(KUBECTL) delete secret $(ORDS_SECRET) -n ${ORDSNAMESPACE} + $(OPENSSL) genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -pkeyopt rsa_keygen_pubexp:65537 > ${PRVKEY} + $(OPENSSL) rsa -in $(PRVKEY) -outform PEM -pubout -out $(PUBKEY) + $(KUBECTL) create secret generic pubkey --from-file=publicKey=$(PUBKEY) -n $(ORDSNAMESPACE) + $(KUBECTL) create secret generic prvkey --from-file=privateKey=$(PRVKEY) -n $(ORDSNAMESPACE) + $(OPENSSL) rsautl -encrypt -pubin -inkey $(PUBKEY) -in $(SIDB_PASSWORD_FILE) |base64 > e_$(SIDB_PASSWORD_FILE) + $(KUBECTL) create secret generic $(SIDB_SECRET) --from-literal=password=$(SIDB_PASSWORD) -n $(OPRNAMESPACE) + $(KUBECTL) create secret generic $(ORDS_SECRET) --from-file=password=e_$(SIDB_PASSWORD_FILE) -n $(ORDSNAMESPACE) + $(RM) e_$(SIDB_PASSWORD_FILE) $(SIDB_PASSWORD_FILE) + +step5d: + - $(KUBECTL) delete secret pubkey -n ${ORDSNAMESPACE} + - $(KUBECTL) delete secret prvkey -n ${ORDSNAMESPACE} + - $(KUBECTL) delete secret $(SIDB_SECRET) -n ${ORDSNAMESPACE} + - $(KUBECTL) delete secret $(ORDS_SECRET) -n ${ORDSNAMESPACE} + + +define registry_secret +printf "#!/bin/bash \n" >$(SECRET_CONTAINER_REGISTRY_SCRIPT) +printf "echo enter password for $(USER_CONTAINER_REGISTRY)@$(ORACLE_CONTAINER_REGISTRY) \n" >$(SECRET_CONTAINER_REGISTRY_SCRIPT) +printf "read -s scpwd \n" >>$(SECRET_CONTAINER_REGISTRY_SCRIPT) +printf "$(KUBECTL) create secret docker-registry $(SECRET_CONTAINER_REGISTRY) --docker-server=$(ORACLE_CONTAINER_REGISTRY) --docker-username=$(USER_CONTAINER_REGISTRY) --docker-password=\u0024scpwd --docker-email=$(USER_CONTAINER_REGISTRY) -n $(OPRNAMESPACE) \n" >>$(SECRET_CONTAINER_REGISTRY_SCRIPT) +printf "$(KUBECTL) create secret docker-registry $(SECRET_CONTAINER_REGISTRY) --docker-server=$(ORACLE_CONTAINER_REGISTRY) --docker-username=$(USER_CONTAINER_REGISTRY) --docker-password=\u0024scpwd --docker-email=$(USER_CONTAINER_REGISTRY) -n $(ORDSNAMESPACE) \n" >>$(SECRET_CONTAINER_REGISTRY_SCRIPT) + +bash $(SECRET_CONTAINER_REGISTRY_SCRIPT) +endef + +step6a: + $(call registry_secret) + +step6d: + $(KUBECTL) delete secret $(SECRET_CONTAINER_REGISTRY) -n $(OPRNAMESPACE) + + +define sidb + +cat<$(SIDB_CREATION) +#apiVersion: database.oracle.com/v4 +#kind: SingleInstanceDatabase +#metadata: +# name: oraoper-sidb +# namespace: $(OPRNAMESPACE) +#spec: +# replicas: 1 +# image: +# pullFrom: $(SIDB_IMAGE) +# pullSecrets: $(SECRET_CONTAINER_REGISTRY) +# prebuiltDB: true +# sid: FREE +# listenerPort: 30001 +# edition: free +# adminPassword: +# secretName: $(SIDB_SECRET) +# secretKey: password +# pdbName: FREEPDB1 +EOF + +$(KUBECTL) $(1) -f $(SIDB_CREATION) +endef + +step7: + $(call sidb,$(ACTION)) +step7a: + $(MAKE) -f $(MAKEFILE) ACTION=apply step7 +step7d: + $(MAKE) -f $(MAKEFILE) ACTION=delete step7 + + +define restservice +cat<$(REST_SERVER_CREATION) +#apiVersion: database.oracle.com/v4 +#kind: $(KIND) +#metadata: +# name: $(REST_SERVER_NAME) +# namespace: $(ORDSNAMESPACE) +#spec: +# image: $(ORDS_IMAGE) +# forceRestart: true +# encPrivKey: +# secretName: prvkey +# passwordKey: privateKey +# globalSettings: +# database.api.enabled: true +# poolSettings: +# - poolName: default +# autoUpgradeORDS: true +# autoUpgradeAPEX: true +# restEnabledSql.active: true +# plsql.gateway.mode: direct +# db.connectionType: customurl +# db.customURL: jdbc:oracle:thin:@//$(2) +# db.username: ORDS_PUBLIC_USER +# db.secret: +# secretName: $(ORDS_SECRET) +# db.adminUser: SYS +# db.adminUser.secret: +# secretName: $(ORDS_SECRET) +# +EOF + +[ $(3) -eq 1 ] && { +sed -i 's/SYS/SYT/g' $(REST_SERVER_CREATION) +echo -e "TYPO" +} + +$(KUBECTL) $(1) -f $(REST_SERVER_CREATION) +endef + +step8: + $(eval TNS_ALIAS_CDB := $(shell $(KUBECTL) get SingleInstanceDatabase -n $(OPRNAMESPACE) --template '{{range .items}}{{.status.clusterConnectString}}{{"\n"}}{{end}}')) + $(eval TNS_ALIAS_PDB := $(shell $(KUBECTL) get SingleInstanceDatabase -n $(OPRNAMESPACE) --template '{{range .items}}{{.status.pdbConnectString}}{{"\n"}}{{end}}')) + echo $(TNS_ALIAS) + $(call restservice,$(ACTION),$(TNS_ALIAS_PDB),$(ERR)) +step8a: + $(MAKE) -f $(MAKEFILE) ACTION=apply step8 ERR=0 +step8d: + $(MAKE) -f $(MAKEFILE) ACTION=delete step8 ERR=0 +step8e: + $(MAKE) -f $(MAKEFILE) ACTION=apply step8 ERR=1 + +reloadop: + echo "RESTARTING OPERATOR" + $(eval OP1 := $(shell $(KUBECTL) get pods -n $(OPRNAMESPACE)|grep oracle-database-operator-controller|head -1|cut -d ' ' -f 1 )) + $(eval OP2 := $(shell $(KUBECTL) get pods -n $(OPRNAMESPACE)|grep oracle-database-operator-controller|head -2|tail -1|cut -d ' ' -f 1 )) + $(eval OP3 := $(shell $(KUBECTL) get pods -n $(OPRNAMESPACE)|grep oracle-database-operator-controller|tail -1|cut -d ' ' -f 1 )) + $(KUBECTL) get pod $(OP1) -n $(OPRNAMESPACE) -o yaml | kubectl replace --force -f - + $(KUBECTL) get pod $(OP2) -n $(OPRNAMESPACE) -o yaml | kubectl replace --force -f - + $(KUBECTL) get pod $(OP3) -n $(OPRNAMESPACE) -o yaml | kubectl replace --force -f - + +loginords: + @$(eval RESTPOD := $(shell $(KUBECTL) get pods --template '{{range .items}}{{.metadata.name}}{{"\n"}}{{end}}' -n $(ORDSNAMESPACE))) + $(KUBECTL) logs $(RESTPOD) -n $(ORDSNAMESPACE) + $(KUBECTL) exec $(RESTPOD) -n $(ORDSNAMESPACE) -it -- /bin/bash + +logindb: + $(eval PODPDB := $(shell $(KUBECTL) get pods --template '{{range .items}}{{.metadata.name}}{{"\n"}}{{end}}' -n $(OPRNAMESPACE)|grep -v oracle-database-operator)) + echo $(PODPDB) + $(KUBECTL) exec $(PODPDB) -n $(OPRNAMESPACE) -it -- bash + + +report: + $(KUBECTL) get pods -n $(OPRNAMESPACE) + $(KUBECTL) get SingleInstanceDatabase -n $(OPRNAMESPACE) + $(KUBECTL) get pods -n $(ORDSNAMESPACE) + $(KUBECTL) get $(KIND) -n $(ORDSNAMESPACE) + + +someattributes: + kubectl get SingleInstanceDatabase -n oracle-database-operator-system --template '{{range .items}}{{.status.connectString}}{{"\n"}}{{end}}' + kubectl get SingleInstanceDatabase -n oracle-database-operator-system --template '{{range .items}}{{.status.tcpsConnectString}}{{"\n"}}{{end}}' + kubectl get SingleInstanceDatabase -n oracle-database-operator-system --template '{{range .items}}{{.status.clusterConnectString}}{{"\n"}}{{end}}' + kubectl get SingleInstanceDatabase -n oracle-database-operator-system --template '{{range .items}}{{.status.tcpsPdbConnectString}}{{"\n"}}{{end}}' + kubectl get SingleInstanceDatabase -n oracle-database-operator-system --template '{{range .items}}{{.status.pdbConnectString}}{{"\n"}}{{end}}' + + + + + +dump: + @$(eval TMPSP := $(shell date "+%y%m%d%H%M%S" )) + @$(eval DIAGFILE := ./opdmp.$(TMPSP)) + @>$(DIAGFILE) + @echo "OPERATOR DUMP" >> $(DIAGFILE) + @echo "~~~~~~~~~~~~~" >> $(DIAGFILE) + $(KUBECTL) logs pod/`$(KUBECTL) get pods -n $(OPRNAMESPACE)|grep oracle-database-operator-controller|head -1|cut -d ' ' -f 1` -n $(OPRNAMESPACE) >>$(DIAGFILE) + $(KUBECTL) logs pod/`$(KUBECTL) get pods -n $(OPRNAMESPACE)|grep oracle-database-operator-controller|head -2|tail -1 | cut -d ' ' -f 1` -n $(OPRNAMESPACE) >>$(DIAGFILE) + $(KUBECTL) logs pod/`$(KUBECTL) get pods -n $(OPRNAMESPACE)|grep oracle-database-operator-controller|tail -1|cut -d ' ' -f 1` -n $(OPRNAMESPACE) >>$(DIAGFILE) + + + +step9: sql +define dbenv +$(1): DB_PWD=`$(KUBECTL) get secrets sidb-db-auth -n $(OPRNAMESPACE) --template='{{.data.password | base64decode}}'` +$(1): POD_NAME=`$(KUBECTL) get pod -l "app=oraoper-sidb" -o custom-columns=NAME:.metadata.name -n $(OPRNAMESPACE) --no-headers` +$(1): TNSSTR=`$(KUBECTL) get SingleInstanceDatabase -n $(OPRNAMESPACE) --template '{{range .items}}{{.status.pdbConnectString}}{{"\n"}}{{end}}'` +endef + +$(eval $(call dbenv,sqlplus sql)) +#$(eval $(call dbenv,sqlplus)) + +define copyfile +cat <create_mong_schema.sql +drop user MONGO cascade; +set echo on +set head on +create user MONGO identified by "My_Password1!"; +grant soda_app, create session, create table, create view, create sequence, create procedure, create job, +unlimited tablespace to MONGO; +conn MONGO/My_Password1!@${TNSSTR} +exec ords.enable_schema; +exit; +EOF +$(KUBECTL) cp ./create_mong_schema.sql $(POD_NAME):/home/oracle -n $(OPRNAMESPACE) +endef + +sql: + echo $(TNSSTR) + $(call copyfile) + @$(KUBECTL) exec -it $(POD_NAME) -n $(OPRNAMESPACE) -- sqlplus SYSTEM/$(DB_PWD)@$(TNSSTR) @/home/oracle/create_mong_schema.sql + +sqlplus: + @$(KUBECTL) exec -it $(POD_NAME) -n $(OPRNAMESPACE) -- sqlplus SYSTEM/$(DB_PWD)@$(TNSSTR) + + +define restservicemongo +cat <$(REST_SERVER_CREATION_MONGO) +#apiVersion: database.oracle.com/v4 +#kind: $(KIND) +#metadata: +# name: $(REST_SERVER_NAME_MONGO) +# namespace: $(ORDSNAMESPACE) +#spec: +# image: $(ORDS_IMAGE.1) +# forceRestart: true +# globalSettings: +# database.api.enabled: true +# mongo.enabled: true +# poolSettings: +# - poolName: default +# autoUpgradeORDS: true +# restEnabledSql.active: true +# plsql.gateway.mode: direct +# jdbc.MaxConnectionReuseCount: 5000 +# jdbc.MaxConnectionReuseTime: 900 +# jdbc.SecondsToTrustIdleConnection: 1 +# jdbc.InitialLimit: 100 +# jdbc.MaxLimit: 100 +# db.connectionType: customurl +# db.customURL: jdbc:oracle:thin:@//${2} +# db.username: ORDS_PUBLIC_USER +# db.secret: +# secretName: ords-db-auth +# db.adminUser: SYS +# db.adminUser.secret: +# secretName: ords-db-auth +EOF +$(KUBECTL) $(1) -f $(REST_SERVER_CREATION_MONGO) +endef + + + +step10: + $(eval TNS_ALIAS_PDB := $(shell $(KUBECTL) get SingleInstanceDatabase -n $(OPRNAMESPACE) --template '{{range .items}}{{.status.pdbConnectString}}{{"\n"}}{{end}}')) + echo $(TNS_ALIAS_PDB) + $(call restservicemongo,$(ACTION),$(TNS_ALIAS_PDB)) +step10a: + $(MAKE) -f $(MAKEFILE) ACTION=apply step10 +step10d: + $(MAKE) -f $(MAKEFILE) ACTION=delete step10 + + +step11: + echo "Open a port-forward to the MongoAPI service" + @nohup $(KUBECTL) port-forward service/$(REST_SERVER_NAME_MONGO) 27017:27017 -n $(ORDSNAMESPACE) 1>portfwd.log 2>&1 & + @echo "DOWNLOADING MONGOSH" + @$(CURL) https://downloads.mongodb.com/compass/$(MONGOSH).tgz --output mongosh-2.3.1-linux-x64.tgz + @echo "UNTAR FILE" + @$(TAR) -zxvf $(MONGOSH).tgz + ./$(MONGOSH)/bin/mongosh --tlsAllowInvalidCertificates 'mongodb://MONGO:My_Password1!@localhost:27017/MONGO?authMechanism=PLAIN&authSource=$external&tls=true&retryWrites=false&loadBalanced=true' + @echo "STOP PORT FRWD" + @kill `ps -ef | grep kubectl | grep 27017 | grep -v grep | awk '{printf $$2}'` + $(RM) $(MONGOSH).tgz + $(RM) -rf ./$(MONGOSH) + + +define buildtns +echo "Building tnsnames.ora" +cat <$(TNSADMIN)/$(TNSNAMES) +$(PDB1)=$(TNS1) + +$(PDB2)=$(TNS2) +EOF +$(KUBECTL) create secret generic multi-tns-admin -n $(ORDSNAMESPACE) --from-file=$(TNSADMIN)/ +endef + +step12a: + $(call buildtns) + +step12d: + $(KUBECTL) delete secret multi-tns-admin -n $(ORDSNAMESPACE) + +export SYSPWDFILE1=syspwdfile1 +export SYSPWDFILE2=syspwdfile2 +export ORDPWDFILE=ordspwdfile + + +step13a: + echo $(PDB1_PWD) > $(SYSPWDFILE1) + echo $(PDB2_PWD) > $(SYSPWDFILE2) + echo $(ORDS_MULTI_POOL_PWD) > $(ORDPWDFILE) + $(OPENSSL) genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -pkeyopt rsa_keygen_pubexp:65537 > ${PRVKEY} + $(OPENSSL) rsa -in $(PRVKEY) -outform PEM -pubout -out $(PUBKEY) + #$(KUBECTL) create secret generic pubkey --from-file=publicKey=$(PUBKEY) -n $(ORDSNAMESPACE) + $(KUBECTL) create secret generic prvkey --from-file=privateKey=$(PRVKEY) -n $(ORDSNAMESPACE) + $(OPENSSL) rsautl -encrypt -pubin -inkey $(PUBKEY) -in $(SYSPWDFILE1) |base64 > e_$(SYSPWDFILE1) + $(OPENSSL) rsautl -encrypt -pubin -inkey $(PUBKEY) -in $(SYSPWDFILE2) |base64 > e_$(SYSPWDFILE2) + $(OPENSSL) rsautl -encrypt -pubin -inkey $(PUBKEY) -in $(ORDPWDFILE) |base64 > e_$(ORDPWDFILE) + $(KUBECTL) create secret generic $(PDB1_PRIV_AUTH_SECRET) --from-file=password=e_$(SYSPWDFILE1) -n $(ORDSNAMESPACE) + $(KUBECTL) create secret generic $(PDB2_PRIV_AUTH_SECRET) --from-file=password=e_$(SYSPWDFILE2) -n $(ORDSNAMESPACE) + $(KUBECTL) create secret generic $(MULTI_ORDS_AUTH_SECRET) --from-file=password=e_$(ORDPWDFILE) -n $(ORDSNAMESPACE) + $(RM) $(SYSPWDFILE1) $(SYSPWDFILE2) $(ORDPWDFILE) e_$(SYSPWDFILE1) e_$(SYSPWDFILE2) e_$(ORDPWDFILE) + +step13d: + - $(KUBECTL) delete secret pubkey -n $(ORDSNAMESPACE) + - $(KUBECTL) delete secret prvkey -n $(ORDSNAMESPACE) + - $(KUBECTL) delete secret $(PDB1_PRIV_AUTH_SECRET) -n $(ORDSNAMESPACE) + - $(KUBECTL) delete secret $(PDB2_PRIV_AUTH_SECRET) -n $(ORDSNAMESPACE) + - $(KUBECTL) delete secret $(MULTI_ORDS_AUTH_SECRET) -n $(ORDSNAMESPACE) + +define multisrv +cat <$(MULTISRV_MANIFEST) +#apiVersion: database.oracle.com/v4 +#kind: $(KIND) +#metadata: +# name: ords-multi-pool +# namespace: $(ORDSNAMESPACE) +#spec: +# image: container-registry.oracle.com/database/ords:24.1.1 +# forceRestart: true +# encPrivKey: +# secretName: prvkey +# passwordKey: privateKey +# globalSettings: +# database.api.enabled: true +# poolSettings: +# - poolName: pdb1 +# autoUpgradeAPEX: false +# autoUpgradeORDS: false +# db.connectionType: tns +# db.tnsAliasName: pdb1 +# tnsAdminSecret: +# secretName: multi-tns-admin +# restEnabledSql.active: true +# feature.sdw: true +# plsql.gateway.mode: proxied +# db.username: ORDS_PUBLIC_USER +# db.secret: +# secretName: $(MULTI_ORDS_AUTH_SECRET) +# db.adminUser: SYS +# db.adminUser.secret: +# secretName: $(PDB1_PRIV_AUTH_SECRET) +# - poolName: pdb2 +# autoUpgradeAPEX: false +# autoUpgradeORDS: false +# db.connectionType: tns +# db.tnsAliasName: PDB2 +# tnsAdminSecret: +# secretName: multi-tns-admin +# restEnabledSql.active: true +# feature.sdw: true +# plsql.gateway.mode: proxied +# db.username: ORDS_PUBLIC_USER +# db.secret: +# secretName: $(MULTI_ORDS_AUTH_SECRET) +# db.adminUser: SYS +# db.adminUser.secret: +# secretName: $(PDB1_PRIV_AUTH_SECRET) + +# +EOF +[ $(2) -eq 1 ] && { +sed -i 's/SYS/SYT/g' $(MULTISRV_MANIFEST) +echo -e "TYPO" +} + +$(KUBECTL) $(1) -f $(MULTISRV_MANIFEST) +endef + +step14: + $(call multisrv,$(ACTION),$(ERR)) +step14a: + $(MAKE) -f $(MAKEFILE) ACTION=apply ERR=0 step14 +step14d: + $(MAKE) -f $(MAKEFILE) ACTION=delete ERR=0 step14 +step14e: + $(MAKE) -f $(MAKEFILE) ACTION=apply ERR=1 step14 + + +define dumpinit +#!/bin/bash +NAMESPACE=${1} +KUBECTL=/usr/bin/kubectl +for _pod in `${KUBECTL} get pods --no-headers -o custom-columns=":metadata.name" --no-headers -n $${NAMESPACE}` +do + for _podinit in `${KUBECTL} get pod $${_pod} -n $${NAMESPACE} -o="custom-columns=INIT-CONTAINERS:.spec.initContainers[*].name" --no-headers` + do + echo "DUMPINIT $${_pod}:$${_podinit}" + ${KUBECTL} logs -f --since=0 $${_pod} -n $${NAMESPACE} -c $${_podinit} + done +done +endef + +diagordsinit: + $(call dumpinit ,$(ORDSNAMESPACE)) + diff --git a/docs/ordsservices/usecase01/tnsadmin/tnsnames.ora b/docs/ordsservices/usecase01/tnsadmin/tnsnames.ora new file mode 100644 index 00000000..1b1b8943 --- /dev/null +++ b/docs/ordsservices/usecase01/tnsadmin/tnsnames.ora @@ -0,0 +1,3 @@ +pdb1=(DESCRIPTION=(CONNECT_TIMEOUT=90)(RETRY_COUNT=30)(RETRY_DELAY=10)(TRANSPORT_CONNECT_TIMEOUT=70)(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan12.testrac.com)(PORT=1521)(IP=V4_ONLY))(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan12.testrac.com)(PORT=1521)(IP=V4_ONLY))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=pdb1))) + +pdb2=(DESCRIPTION=(CONNECT_TIMEOUT=90)(RETRY_COUNT=30)(RETRY_DELAY=10)(TRANSPORT_CONNECT_TIMEOUT=70)(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan12.testrac.com)(PORT=1521)(IP=V4_ONLY))(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan34.testrac.com)(PORT=1521)(IP=V4_ONLY))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=pdb2))) diff --git a/docs/ordsservices/usecase01/tnsadmin/tnsnames.ora.offline b/docs/ordsservices/usecase01/tnsadmin/tnsnames.ora.offline new file mode 100644 index 00000000..b58a8a66 --- /dev/null +++ b/docs/ordsservices/usecase01/tnsadmin/tnsnames.ora.offline @@ -0,0 +1 @@ +pdb1=(DESCRIPTION=(CONNECT_TIMEOUT=90)(RETRY_COUNT=30)(RETRY_DELAY=10)(TRANSPORT_CONNECT_TIMEOUT=70)(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan12.testrac.com)(PORT=1521)(IP=V4_ONLY))(LOAD_BALLANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=scan34.testrac.com)(PORT=1521)(IP=V4_ONLY))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=TESTORDS))) diff --git a/docs/sharding/README.md b/docs/sharding/README.md index 239dd767..661e2546 100644 --- a/docs/sharding/README.md +++ b/docs/sharding/README.md @@ -24,16 +24,15 @@ The Oracle Sharding database controller provides end-to-end automation of Oracle ## Using Oracle Database Operator Sharding Controller -Following sections provide the details for deploying Oracle Globally Distributed Database using Oracle Database Operator Sharding Controller with different use cases: +Following sections provide the details for deploying Oracle Globally Distributed Database (Oracle Sharded Database) using Oracle Database Operator Sharding Controller with different use cases: * [Prerequisites for running Oracle Sharding Database Controller](#prerequisites-for-running-oracle-sharding-database-controller) * [Oracle Database 23ai Free](#oracle-database-23ai-free) -* [Provisioning Oracle Globally Distributed Database Topology System-Managed Sharding in a Cloud-Based Kubernetes Cluster](#provisioning-oracle-globally-distributed-database-topology-with-system-managed-sharding-in-a-cloud-based-kubernetes-cluster) -* [Provisioning Oracle Globally Distributed Database Topology User Defined Sharding in a Cloud-Based Kubernetes Cluster](#provisioning-oracle-globally-distributed-database-topology-with-user-defined-sharding-in-a-cloud-based-kubernetes-cluster) -* [Provisioning Oracle Globally Distributed Database System-Managed Sharding with Raft replication enabled in a Cloud-Based Kubernetes Cluster](#provisioning-oracle-globally-distributed-database-topology-with-system-managed-sharding-and-raft-replication-enabled-in-a-cloud-based-kubernetes-cluster) +* [Provisioning Sharding Topology with System-Managed Sharding in a Cloud-Based Kubernetes Cluster](#provisioning-sharding-topology-with-system-managed-sharding-in-a-cloud-based-kubernetes-cluster) +* [Provisioning Sharding Topology with User Defined Sharding in a Cloud-Based Kubernetes Cluster](#provisioning-sharding-topology-with-user-defined-sharding-in-a-cloud-based-kubernetes-cluster) +* [Provisioning System-Managed Sharding Topology with Raft replication enabled in a Cloud-Based Kubernetes Cluster](#provisioning-system-managed-sharding-topology-with-raft-replication-enabled-in-a-cloud-based-kubernetes-cluster) * [Connecting to Shard Databases](#connecting-to-shard-databases) * [Debugging and Troubleshooting](#debugging-and-troubleshooting) -* [Known Issues](#known-issues) **Note** Before proceeding to the next section, you must complete the instructions given in each section, based on your enviornment, before proceeding to next section. @@ -92,6 +91,8 @@ You can either download the images and push them to your Docker Images Repositor **Note:** In case you want to use the `Oracle Database 23ai Free` Image for Database and GSM, refer to section [Oracle Database 23ai Free](#oracle-database-23ai-free) for more details. +### 4. Create a namespace for the Oracle DB Sharding Setup + ### 4. Create a namespace for the Oracle Globally Distributed Database Setup Create a Kubernetes namespace named `shns`. All the resources belonging to the Oracle Globally Distributed Database Topology Setup will be provisioned in this namespace named `shns`. For example: @@ -108,7 +109,7 @@ You can either download the images and push them to your Docker Images Repositor Create a Kubernetes secret named `db-user-pass-rsa` using these steps: [Create Kubernetes Secret](./provisioning/create_kubernetes_secret_for_db_user.md) -After you have the above prerequisites completed, you can proceed to the next section for your environment to provision the Oracle Globally Distributed Database Topology. +After you have the above prerequisites completed, you can proceed to the next section for your environment to provision the Oracle Database Sharding Topology. ### 6. Provisioning a Persistent Volume having an Oracle Database Gold Image @@ -118,6 +119,71 @@ In case of an `OCI OKE` cluster, you can use this Persistent Volume during provi You can refer [here](./provisioning/provisioning_persistent_volume_having_db_gold_image.md) for the steps involved. +**NOTE:** Provisioning the Sharded Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. So, this step will not be needed if you are deploying Oracle Sharded Database using Oracle 23ai Free Database and GSM Images. + +## Oracle Database 23ai Free + +Please refer to [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) documentation for more details. + +If you want to use Oracle Database 23ai Free Image for Database and GSM for deployment of the Sharded Database using Sharding Controller in Oracle Database Kubernetes Operator, you need to consider the below points: + +* To deploy using the FREE Database and GSM Image, you will need to add the additional parameter `dbEdition: "free"` to the .yaml file. +* Refer to [Sample Sharded Database Deployment using Oracle 23ai FREE Database and GSM Images](./provisioning/free/sharding_provisioning_with_free_images.md) for an example. +* For Oracle Database 23ai Free, you can control the `CPU` and `Memory` allocation of the PODs using tags `cpu` and `memory` respectively but tags `INIT_SGA_SIZE` and `INIT_PGA_SIZE` to control the SGA and PGA allocation at the database level are `not` supported. +* Provisioning the Sharded Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. +* Total number of chunks for FREE Database defaults to `12` if `CATALOG_CHUNKS` parameter is not specified. This default value is determined considering limitation of 12 GB of user data on disk for oracle free database. + + +## Provisioning Sharding Topology with System-Managed Sharding in a Cloud-Based Kubernetes Cluster + +Deploy Oracle Database Sharding Topology with `System-Managed Sharding` on your Cloud based Kubernetes cluster. + +In this example, the deployment uses the YAML file based on `OCI OKE` cluster. There are multiple use case possible for deploying the Oracle Database Sharding Topology covered by below examples: + +[1. Provisioning Oracle Sharded Database with System-Managed Sharding without Database Gold Image](./provisioning/system_sharding/ssharding_provisioning_without_db_gold_image.md) +[2. Provisioning Oracle Sharded Database with System-Managed Sharding with number of chunks specified](./provisioning/system_sharding/ssharding_provisioning_with_chunks_specified.md) +[3. Provisioning Oracle Sharded Database with System-Managed Sharding with additional control on resources like Memory and CPU allocated to Pods](./provisioning/system_sharding/ssharding_provisioning_with_control_on_resources.md) +[4. Provisioning Oracle Sharded Database with System-Managed Sharding by cloning database from your own Database Gold Image in the same Availability Domain(AD)](./provisioning/system_sharding/ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md) +[5. Provisioning Oracle Sharded Database with System-Managed Sharding by cloning database from your own Database Gold Image across Availability Domains(ADs)](./provisioning/system_sharding/ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md) +[6. Provisioning Oracle Sharded Database with System-Managed Sharding and send Notification using OCI Notification Service](./provisioning/system_sharding/ssharding_provisioning_with_notification_using_oci_notification.md) +[7. Scale Out - Add Shards to an existing Oracle Sharded Database provisioned earlier with System-Managed Sharding](./provisioning/system_sharding/ssharding_scale_out_add_shards.md) +[8. Scale In - Delete an existing Shard from a working Oracle Sharded Database provisioned earlier with System-Managed Sharding](./provisioning/system_sharding/ssharding_scale_in_delete_an_existing_shard.md) + + +## Provisioning Sharding Topology with User Defined Sharding in a Cloud-Based Kubernetes Cluster + +Deploy Oracle Database Sharding Topology with `User Defined Sharding` on your Cloud based Kubernetes cluster. + +In this example, the deployment uses the YAML file based on `OCI OKE` cluster. There are multiple use case possible for deploying the Oracle Database Sharding Topology covered by below examples: + +[1. Provisioning Oracle Sharded Database with User Defined Sharding without Database Gold Image](./provisioning/user-defined-sharding/udsharding_provisioning_without_db_gold_image.md) +[2. Provisioning Oracle Sharded Database with User Defined Sharding with additional control on resources like Memory and CPU allocated to Pods](./provisioning/user-defined-sharding/udsharding_provisioning_with_control_on_resources.md) +[3. Provisioning Oracle Sharded Database with User Defined Sharding by cloning database from your own Database Gold Image in the same Availability Domain(AD)](./provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_gold_image_in_same_ad.md) +[4. Provisioning Oracle Sharded Database with User Defined Sharding by cloning database from your own Database Gold Image across Availability Domains(ADs)](./provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_from_gold_image_across_ads.md) +[5. Provisioning Oracle Sharded Database with User Defined Sharding and send Notification using OCI Notification Service](./provisioning/user-defined-sharding/udsharding_provisioning_with_notification_using_oci_notification.md) +[6. Scale Out - Add Shards to an existing Oracle Sharded Database provisioned earlier with User Defined Sharding](./provisioning/user-defined-sharding/udsharding_scale_out_add_shards.md) +[7. Scale In - Delete an existing Shard from a working Oracle Sharded Database provisioned earlier with User Defined Sharding](./provisioning/user-defined-sharding/udsharding_scale_in_delete_an_existing_shard.md) + + +## Provisioning System-Managed Sharding Topology with Raft replication enabled in a Cloud-Based Kubernetes Cluster + +Deploy Oracle Database Sharding Topology with `System-Managed Sharding with SNR RAFT enabled` on your Cloud based Kubernetes cluster. + +**NOTE: SNR RAFT Feature is available only for Oracle 23ai RDBMS and Oracle 23ai GSM version.** + +In this example, the deployment uses the YAML file based on `OCI OKE` cluster. There are multiple use case possible for deploying the Oracle Database Sharding Topology covered by below examples: + +[1. Provisioning System-Managed Sharding Topology with Raft replication enabled without Database Gold Image](./provisioning/snr_system_sharding/snr_ssharding_provisioning_without_db_gold_image.md) +[2. Provisioning System-Managed Sharding Topology with Raft replication enabled with number of chunks specified](./provisioning/snr_system_sharding/snr_ssharding_provisioning_with_chunks_specified.md) +[3. Provisioning System-Managed Sharding Topology with Raft replication enabled with additional control on resources like Memory and CPU allocated to Pods](./provisioning/snr_system_sharding/snr_ssharding_provisioning_with_control_on_resources.md) +[4. Provisioning System-Managed Sharding Topology with Raft replication enabled by cloning database from your own Database Gold Image in the same Availability Domain(AD)](./provisioning/snr_system_sharding/snr_ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md) +[5. Provisioning System-Managed Sharding Topology with Raft replication enabled by cloning database from your own Database Gold Image across Availability Domains(ADs)](./provisioning/snr_system_sharding/snr_ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md) +[6. Provisioning System-Managed Sharding Topology with Raft replication enabled and send Notification using OCI Notification Service](./provisioning/snr_system_sharding/snr_ssharding_provisioning_with_notification_using_oci_notification.md) +[7. Scale Out - Add Shards to an existing Oracle Sharded Database provisioned earlier with System-Managed Sharding with RAFT replication enabled](./provisioning/snr_system_sharding/snr_ssharding_scale_out_add_shards.md) +[8. Scale In - Delete an existing Shard from a working Oracle Sharded Database provisioned earlier with System-Managed Sharding with RAFT reolication enabled](./provisioning/snr_system_sharding/snr_ssharding_scale_in_delete_an_existing_shard.md) + +You can refer [here](./provisioning/provisioning_persistent_volume_having_db_gold_image.md) for the steps involved. + **NOTE:** Provisioning the Oracle Globally Distributed Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. So, this step will not be needed if you are deploying Oracle Globally Distributed Database using Oracle 23ai Free Database and GSM Images. ## Oracle Database 23ai Free diff --git a/docs/sharding/provisioning/create_kubernetes_secret_for_db_user.md b/docs/sharding/provisioning/create_kubernetes_secret_for_db_user.md index 99620f04..744f972c 100644 --- a/docs/sharding/provisioning/create_kubernetes_secret_for_db_user.md +++ b/docs/sharding/provisioning/create_kubernetes_secret_for_db_user.md @@ -1,14 +1,14 @@ # Create kubernetes secret for db user -Below are the steps to create an encrypted file with a password for the DB User: +Use the following steps to create an encrypted file with a password for the DB User: -- Create a text file which is having the password which you want to use for the DB user. +- Create a text file that has the password that you want to use for the DB user. - Create an RSA key pair using `openssl`. -- Encrypt the text file with password using `openssl` with the RSA key pair generated earlier. +- Encrypt the text file with a password, using `openssl` with the RSA key pair generated earlier. - Remove the initial text file. -- Create the Kubernetes secret named `db-user-pass-rsa` using the encrypted file. +- Create the Kubernetes Secret named `db-user-pass-rsa` using the encrypted file. -Please refer the below example for the above steps: +To understand how to create your own file, use the following example: ```sh # Create a directory for files for the secret: @@ -43,4 +43,4 @@ kubectl delete secret $SECRET_NAME -n $NAMESPACE # Create the Kubernetes secret in namespace "NAMESPACE" kubectl create secret generic $SECRET_NAME --from-file=$PWDFILE_ENC --from-file=${PRIVKEY} -n $NAMESPACE -``` \ No newline at end of file +``` diff --git a/docs/sharding/provisioning/database_connection.md b/docs/sharding/provisioning/database_connection.md index 5671520b..58a54930 100644 --- a/docs/sharding/provisioning/database_connection.md +++ b/docs/sharding/provisioning/database_connection.md @@ -1,10 +1,10 @@ # Database Connectivity -The Oracle Globally Distributed Database Topology deployed by Sharding Controller in Oracle Database Operator has an external IP available for each of the container. +The Oracle Database Sharding Topology deployed by Sharding Controller in Oracle Database Operator has an external IP available for each of the containers. ## Below is an example setup with connection details -Check the details of the Oracle Globally Distributed Database Topology provisioned using Sharding Controller: +Check the details of the Sharding Topology provisioned by using the Sharding Controller: ```sh $ kubectl get all -n shns @@ -35,7 +35,7 @@ statefulset.apps/shard1 1/1 10d statefulset.apps/shard2 1/1 10d ``` -After you have the external IP address, you can use the services shown below to make the database connection using the above example: +After you have the external IP address, you can use the services shown below to make the database connection. Using the preceding example, that file should look as follows: 1. **Direct connection to the CATALOG Database**: Connect to the service `catalogpdb` on catalog container external IP `xx.xx.xx.116` on port `1521` 2. **Direct connection to the shard Database SHARD1**: Connect to the service `shard1pdb` on catalog container external IP `xx.xx.xx.187` on port `1521` diff --git a/docs/sharding/provisioning/debugging.md b/docs/sharding/provisioning/debugging.md index 330cfc0e..372f104d 100644 --- a/docs/sharding/provisioning/debugging.md +++ b/docs/sharding/provisioning/debugging.md @@ -1,50 +1,50 @@ # Debugging and Troubleshooting -When the Oracle Globally Distributed Database Topology is provisioned using the Oracle Database Kubernetes Operator, the debugging of an issue with the deployment depends on at which stage the issue has been seen. +When the Oracle Database Sharding Topology is provisioned using the Oracle Database Kubernetes Operator, debugging an issue with the deployment depends on which stage the issue is seen. -Below are the possible cases and the steps to debug such an issue: +The following sections provide possible issue cases, and the steps to debug such an issue: ## Failure during the provisioning of Kubernetes Pods -In case the failure occurs during the provisioning, we need to check the status of the Kubernetes Pod which has failed to deployed. +If the failure occurs during the provisioning, then check the status of the Kubernetes Pod that has failed to be deployed. -Use the below command to check the logs of the Pod which has a failure. For example, for failure in case of Pod `pod/catalog-0`, use below command: +To check the logs of the Pod that has a failure, use the command that follows. In this example, we are checking for failure in provisioning Pod `pod/catalog-0`: ```sh kubectl logs -f pod/catalog-0 -n shns ``` -In case the Pod has failed to provision due to an issue with the Docker Image, you will see the error `Error: ErrImagePull` in above logs. +If the Pod has failed to provision due to an issue with the Docker Image, then you will see the error `Error: ErrImagePull` in the logs displayed by the command. -If the Pod has not yet got initialized, use the below command to find the reason for it: +If the Pod has not yet been initialized, then use the following command to find the reason for it: ```sh kubectl describe pod/catalog-0 -n shns ``` -In case the failure is related to the Cloud Infrastructure, you will need to troubleshooting that using the documentation from the cloud provider. +If the failure is related to the Cloud Infrastructure, then troubleshoot the infrastructure using the documentation from the Cloud infrastructure provider. ## Failure in the provisioning of the Oracle Globally Distributed Database -In case the failure occures after the Kubernetes Pods are created but during the execution of the scripts to create the shard databases, catalog database or the GSM, you will need to trobleshoot that at the individual Pod level. +If the failure occures after the Kubernetes Pods are created but during the execution of the scripts to create the shard databases, catalog database or the GSM, then you must troubleshoot that failure at the individual Pod level. -Initially, check the logs of the Kubernetes Pod using the command like below (change the name of the Pod with the actual Pod) +Initially, check the logs of the Kubernetes Pod using the following command (change the name of the Pod in the command with the actual Pod): ```sh kubectl logs -f pod/catalog-0 -n shns ``` -To check the logs at the GSM or at the Database level or at the host level, switch to the corresponding Kubernetes container using the command like below: +To check the logs at the GSM level, the database level, or at the host level, switch to the corresponding Kubernetes container. For example: ```sh kubectl exec -it catalog-0 -n shns /bin/bash ``` -Now, you can troubleshooting the corresponding component using the alert log or the trace files etc just like a normal Oracle Globally Distributed Database Deployment. Please refer to [Oracle Globally Distributed Database Documentation](https://docs.oracle.com/en/database/oracle/oracle-database/19/shard/sharding-troubleshooting.html#GUID-629262E5-7910-4690-A726-A565C59BA73E) for this purpose. +When you are in the correct Kubernetes container, you can troubleshooting the corresponding component using the alert log, the trace files, and so on, just as you would with a normal Sharding Database Deployment. For more information, see: [Oracle Database Sharding Documentation](https://docs.oracle.com/en/database/oracle/oracle-database/19/shard/sharding-troubleshooting.html#GUID-629262E5-7910-4690-A726-A565C59BA73E) ## Debugging using Database Events -* You can enable database events as part of the Oracle Globally Distributed Database Deployment -* This can be enabled using the `envVars` -* One example of enabling Database Events is [sharding_provisioning_with_db_events.md](./debugging/sharding_provisioning_with_db_events.md) \ No newline at end of file +* You can enable database events as part of the Sharded Database Deployment +* Enable events using `envVars` +* One example of enabling Database Events is [sharding_provisioning_with_db_events.md](./debugging/sharding_provisioning_with_db_events.md) diff --git a/docs/sharding/provisioning/debugging/sharding_provisioning_with_db_events.md b/docs/sharding/provisioning/debugging/sharding_provisioning_with_db_events.md index 763f2dc8..fa73920f 100644 --- a/docs/sharding/provisioning/debugging/sharding_provisioning_with_db_events.md +++ b/docs/sharding/provisioning/debugging/sharding_provisioning_with_db_events.md @@ -1,17 +1,17 @@ -# Example of provisioning Oracle Globally Distributed Database along with DB Events set at Database Level +# Example of provisioning Oracle Sharded Database along with DB Events set at Database Level **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. This example sets a Database Event at the Database Level for Catalog and Shard Databases. -The Oracle Globally Distributed Database in this example is deployed with System-Managed Sharding type. In this use case, the database is created automatically using DBCA during the provisioning of the shard databases and the catalog database when the Oracle Globally Distributed Database topology with System-Managed Sharding is deployed using Oracle Sharding controller. +The sharded database in this example is deployed with System-Managed Sharding type. In this use case, the database is created automatically using DBCA during the provisioning of the shard databases and the catalog database when the Oracle Sharding topology with System-Managed Sharding is deployed using Oracle Sharding controller. **NOTE:** In this use case, because DBCA creates the database automatically during the deployment, the time required to create the database is greater than the time it takes when the database is created by cloning from a Database Gold Image. -This example uses `sharding_provisioning_with_db_events.yaml` to provision an Oracle Globally Distributed Database topology using Oracle Sharding controller with: +This example uses `sharding_provisioning_with_db_events.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three Shard Database Pods: `shard1`, `shard2` and `shard3` +* Three sharding Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` * Database Event: `10798 trace name context forever, level 7` set along with `GWM_TRACE level 263` diff --git a/docs/sharding/provisioning/debugging/sharding_provisioning_with_db_events.yaml b/docs/sharding/provisioning/debugging/sharding_provisioning_with_db_events.yaml index 7d136d58..40ad600a 100644 --- a/docs/sharding/provisioning/debugging/sharding_provisioning_with_db_events.yaml +++ b/docs/sharding/provisioning/debugging/sharding_provisioning_with_db_events.yaml @@ -3,7 +3,7 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # --- -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: ShardingDatabase metadata: name: shardingdatabase-sample @@ -66,4 +66,3 @@ spec: role: primary - name: oltp_ro_svc role: primary - namespace: shns diff --git a/docs/sharding/provisioning/free/sharding_provisioning_with_free_images.md b/docs/sharding/provisioning/free/sharding_provisioning_with_free_images.md index f6b53462..61641312 100644 --- a/docs/sharding/provisioning/free/sharding_provisioning_with_free_images.md +++ b/docs/sharding/provisioning/free/sharding_provisioning_with_free_images.md @@ -1,25 +1,25 @@ -# Example of provisioning Oracle Globally Distributed Database with Oracle 23ai FREE Database and GSM Images +# Example of provisioning Oracle Sharded Database with Oracle 23ai FREE Database and GSM Images **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. This example uses the Oracle 23ai FREE Database and GSM Images. -The Oracle Globally Distributed Database in this example is deployed with System-Managed Sharding type. In this use case, the database is created automatically using DBCA during the provisioning of the shard databases and the catalog database when the Oracle Sharding topology with System-Managed Sharding is deployed using Oracle Sharding controller. +The sharded database in this example is deployed with System-Managed Sharding type. In this use case, the database is created automatically using DBCA during the provisioning of the shard databases and the catalog database when the Oracle Sharding topology with System-Managed Sharding is deployed using Oracle Sharding controller. **NOTE:** In this use case, because DBCA creates the database automatically during the deployment, the time required to create the database is greater than the time it takes when the database is created by cloning from a Database Gold Image. This example uses `sharding_provisioning_with_free_images.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three Shard Database Pods: `shard1`, `shard2` and `shard3` +* Three sharding Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` To get the Oracle 23ai FREE Database and GSM Images: * The Oracle 23ai FREE RDBMS Image used is `container-registry.oracle.com/database/free:latest`. Check [Oracle Database Free Get Started](https://www.oracle.com/database/free/get-started/?source=v0-DBFree-ChatCTA-j2032-20240709) for details. - * Use the Oracle 23ai FREE GSM Image used is `container-registry.oracle.com/database/gsm:latest`. * To pull the above image from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. + * Use the Oracle 23ai FREE GSM Binaries `LINUX.X64_234000_gsm.zip` as listed on page [Oracle Database Free Get Started](https://www.oracle.com/database/free/get-started/?source=v0-DBFree-ChatCTA-j2032-20240709) and prepare the GSM Container Image following [Oracle Global Data Services Image](https://github.com/oracle/db-sharding/tree/master/docker-based-sharding-deployment/dockerfiles) * You need to change `dbImage` and `gsmImage` tag with the images you want to use in your enviornment in file `sharding_provisioning_with_free_images.yaml`. diff --git a/docs/sharding/provisioning/free/sharding_provisioning_with_free_images.yaml b/docs/sharding/provisioning/free/sharding_provisioning_with_free_images.yaml index 53e4191f..954ede63 100644 --- a/docs/sharding/provisioning/free/sharding_provisioning_with_free_images.yaml +++ b/docs/sharding/provisioning/free/sharding_provisioning_with_free_images.yaml @@ -3,7 +3,7 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # --- -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: ShardingDatabase metadata: name: shardingdatabase-sample @@ -41,8 +41,8 @@ spec: storageClass: oci dbImage: container-registry.oracle.com/database/free:latest dbImagePullSecret: ocr-reg-cred - gsmImage: container-registry.oracle.com/database/gsm:latest - gsmImagePullSecret: ocr-reg-cred + gsmImage: + gsmImagePullSecret: dbEdition: "free" isExternalSvc: False isDeleteOraPvc: True @@ -55,4 +55,3 @@ spec: role: primary - name: oltp_ro_svc role: primary - namespace: shns diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md index 2698bce9..ba72be25 100644 --- a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md @@ -1,14 +1,14 @@ -# Provisioning Oracle Globally Distributed Database Topology with System-managed sharding and Raft replication enabled by cloning database from your own Database Gold Image across Availability Domains(ADs) +# Provisioning System managed Sharding Topology with Raft replication enabled by cloning database from your own Database Gold Image across Availability Domains(ADs) **NOTE: RAFT Replication Feature is available only for Oracle 23c RDBMS and Oracle 23c GSM version.** **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. -In this test case, you provision the Oracle Globally Distributed Database Topology with System-managed sharding and Raft replication enabled while provisioning the Catalog and Shard Databases by cloning from an existing Oracle Database Gold Image created earlier. +In this test case, you provision the System managed Sharding Topology with Raft replication enabled while provisioning the Catalog and Shard Databases by cloning from an existing Oracle Database Gold Image created earlier. This use case applies when you want to provision the database Pods on a Kubernetes Node in any availability domain (AD), which can also be different from the availability domain (AD) of the Block Volume that has the Oracle Database Gold Image provisioned earlier. -Choosing this option takes substantially less time during the Oracle Globally Distributed Database Topology setup across ADs. +Choosing this option takes substantially less time during the Oracle Database Sharding Topology setup across ADs. NOTE: @@ -21,10 +21,10 @@ NOTE: ```sh kubectl get pv -n shns ``` -2. Create a Block Volume Backup for this Block Volume, and use the OCID of the Block Volume Backup in the next step. This example uses `snr_ssharding_shard_prov_clone_across_ads.yaml` to provision an Oracle Globally Distributed Database topology using Oracle Sharding Database controller with: +2. Create a Block Volume Backup for this Block Volume, and use the OCID of the Block Volume Backup in the next step. This example uses `snr_ssharding_shard_prov_clone_across_ads.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three Shard Database Pods: `shard1`, `shard2` and `shard3` +* Three sharding Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` * Database Cloning from the `BLOCK VOLUME FULL BACKUP` of the Persistent Volume which had the Gold Image. @@ -33,12 +33,12 @@ NOTE: NOTE: In this case, the Persistent Volume with DB Gold Image was provisioned in the Availability Domain `PHX-AD-1`. The Shards and Catalog will be provisioned across multiple Availability Domains by cloning the database. -**NOTE:** Provisioning the Oracle Globally Distributed Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. +**NOTE:** Provisioning the Sharded Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `snr_ssharding_shard_prov_clone_across_ads.yaml`. - * The `dbImage` used during provisioning the Persistent Volume with Database Gold Image and the `dbImage` used for deploying the Shard or Catalog Database by cloning should be same. + * The `dbImage` used during provisioning the Persistent Volume with Database Gold Image and the `dbImage` used for deploying the Shard or Catalog Database by cloning should be same. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) Use the file: [snr_ssharding_shard_prov_clone_across_ads.yaml](./snr_ssharding_shard_prov_clone_across_ads.yaml) for this use case as below: diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md index b43a9158..cf4240f7 100644 --- a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md @@ -1,14 +1,14 @@ -# Provisioning Oracle Globally Distributed Database Topology with System-managed sharding and Raft replication enabled by cloning database from your own Database Gold Image in the same Availability Domain(AD) +# Provisioning System managed Sharding Topology with Raft replication enabled by cloning database from your own Database Gold Image in the same Availability Domain(AD) **NOTE: RAFT Replication Feature is available only for Oracle 23c RDBMS and Oracle 23c GSM version.** **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. -In this test case, you provision the Oracle Globally Distributed Database Topology with System-managed sharding and Raft replication enabled while provisioning the Catalog and Shard Databases by cloning from an existing Oracle Database Gold Image created earlier. +In this test case, you provision the System managed Sharding Topology with Raft replication enabled while provisioning the Catalog and Shard Databases by cloning from an existing Oracle Database Gold Image created earlier. This use case applies when you are cloning from a Block Volume, and you can clone _only_ in the same availability domain (AD). The result is that the cloned shard database PODs can be created _only_ in the same AD where the Gold Image Block Volume is present. -Choosing this option takes substantially less time during the Oracle Globally Distributed Database Topology setup. +Choosing this option takes substantially less time during the Oracle Database Sharding Topology setup. **NOTE** For this step, the Persistent Volume that has the Oracle Database Gold Image is identified using its OCID. @@ -18,10 +18,10 @@ Choosing this option takes substantially less time during the Oracle Globally Di kubectl get pv -n shns ``` -2. This example uses `snr_ssharding_shard_prov_clone.yaml` to provision an Oracle Globally Distributed Database topology using Oracle Sharding controller with: +2. This example uses `snr_ssharding_shard_prov_clone.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three Shard Database Pods: `shard1`, `shard2` and `shard3` +* Three sharding Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` * Database Cloning from the Database Gold Image present in Persistent Volume having OCID: `ocid1.volume.oc1.phx.abyhqljr3z3w72t6ay5eud7d5w3kdfhktfp6gwb6euy5tzwfaxgmbvwqlvsq` @@ -29,14 +29,14 @@ Choosing this option takes substantially less time during the Oracle Globally Di NOTE: In this case, the Persistent Volume with DB Gold Image was provisioned in the Availability Domain `PHX-AD-1`. The Shards and Catalog will be provisioned in the same Availability Domain `PHX-AD-1` by cloning the database. -**NOTE:** Provisioning the Oracle Globally Distributed Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. - In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `snr_ssharding_shard_prov_clone.yaml`. * The `dbImage` used during provisioning the Persistent Volume with Database Gold Image and the `dbImage` used for deploying the Shard or Catalog Database by cloning should be same. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) +**NOTE:** Provisioning the Sharded Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. + Use the file: [snr_ssharding_shard_prov_clone.yaml](./snr_ssharding_shard_prov_clone.yaml) for this use case as below: 1. Deploy the `snr_ssharding_shard_prov_clone.yaml` file: diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_chunks_specified.md b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_chunks_specified.md index d6171986..44972090 100644 --- a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_chunks_specified.md +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_chunks_specified.md @@ -1,19 +1,19 @@ -# Provisioning Oracle Globally Distributed Database Topology with System-managed sharding and Raft replication enabled with number of chunks specified +# Provisioning System-Managed Sharding Topology with Raft replication enabled with number of chunks specified **NOTE: RAFT Replication Feature is available only for Oracle 23c RDBMS and Oracle 23c GSM version.** **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. -In this use case, the database is created automatically using DBCA during the provisioning of the shard databases and the catalog database when the Oracle Globally Distributed Database topology with System-Managed sharding and RAFT Replication enabled is deployed using Oracle Sharding controller. +In this use case, the database is created automatically using DBCA during the provisioning of the shard databases and the catalog database when the Oracle Sharding topology with System-Managed with RAFT Replication enabled is deployed using Oracle Sharding controller. **NOTE** In this use case, because DBCA creates the database automatically during the deployment, the time required to create the database is greater than the time it takes when the database is created by cloning from a Database Gold Image. -By default, the System-Managed with RAFT Replication deploys the Oracle Globally Distributed Database with 360 chunks per Shard Database (because there are 3 chunks created for each replication unit). In this example, the Oracle Globally Distributed Database will be deployed with non-default number of chunks specified using parameter `CATALOG_CHUNKS`. +By default, the System-Managed with RAFT Replication deploys the Sharded Database with 360 chunks per Shard Database (because there are 3 chunks created for each replication unit). In this example, the Sharded Database will be deployed with non-default number of chunks specified using parameter `CATALOG_CHUNKS`. -This example uses `snr_ssharding_shard_prov_chunks.yaml` to provision an Oracle Globally Distributed Database topology using Oracle Sharding controller with: +This example uses `snr_ssharding_shard_prov_chunks.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three Shard Database Pods: `shard1`, `shard2` and `shard3` +* Three sharding Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Total number of chunks as `120` specified by variable `CATALOG_CHUNKS` (it will be 120 chunks per shard) * Namespace: `shns` diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_control_on_resources.md b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_control_on_resources.md index c432310d..9cfd6afb 100644 --- a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_control_on_resources.md +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_control_on_resources.md @@ -1,15 +1,15 @@ -# Provisioning Oracle Globally Distributed Database Topology with System-managed sharding and Raft replication enabled with additional control on resources like Memory and CPU allocated to Pods +# Provisioning System-Managed Sharding Topology with Raft replication enabled with additional control on resources like Memory and CPU allocated to Pods **NOTE: RAFT Replication Feature is available only for Oracle 23c RDBMS and Oracle 23c GSM version.** **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. -In this use case, there are additional tags used to control resources such as CPU and Memory used by the different Pods when the Oracle Globally Distributed Database Topology with System-managed sharding and RAFT Replication is deployed using Oracle Sharding controller. +In this use case, there are additional tags used to control resources such as CPU and Memory used by the different Pods when the Oracle Sharding topology with System-Managed with RAFT Replication is deployed using Oracle Sharding controller. This example uses `snr_ssharding_shard_prov_memory_cpu.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three Shard Database Pods: `shard1`, `shard2` and `shard3` +* Three sharding Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` * Tags `memory` and `cpu` to control the Memory and CPU of the PODs diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_notification_using_oci_notification.md b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_notification_using_oci_notification.md index d45b2911..d4cb11de 100644 --- a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_notification_using_oci_notification.md +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_notification_using_oci_notification.md @@ -1,15 +1,15 @@ -# Provisioning Oracle Globally Distributed Database Topology with System-managed sharding and Raft replication enabled and send Notification using OCI Notification Service +# Provisioning System managed Sharding Topology with Raft replication enabled and send Notification using OCI Notification Service **NOTE: RAFT Replication Feature is available only for Oracle 23c RDBMS and Oracle 23c GSM version.** **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. -This use case demonstrates how to use a notification service like OCI Notification service to send an email notification when a particular operation is completed on an Oracle Globally Distributed Database Topology provisioned using the Oracle Database sharding controller. +This use case demonstrates how to use a notification service like OCI Notification service to send an email notification when a particular operation is completed on an Oracle Database sharding topology provisioned using the Oracle Database sharding controller. -This example uses `snr_ssharding_shard_prov_send_notification.yaml` to provision an Oracle Globally Distributed Database Topology using Oracle Sharding controller with: +This example uses `snr_ssharding_shard_prov_send_notification.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three Shard Database Pods: `shard1`, `shard2` and `shard3` +* Three sharding Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` * Database Cloning from the `BLOCK VOLUME FULL BACKUP` of the Persistent Volume that has the Database Gold Image created earlier. @@ -64,14 +64,14 @@ To do this: kubectl describe secret my-secret -n shns ``` -**NOTE:** Provisioning the Oracle Globally Distributed Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. - In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `snr_ssharding_shard_prov_send_notification.yaml`. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) * In case you want to use the [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then you will need to add the additional parameter `dbEdition: "free"` to the below .yaml file. +**NOTE:** Provisioning the Sharded Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. + Use the file: [snr_ssharding_shard_prov_send_notification.yaml](./snr_ssharding_shard_prov_send_notification.yaml) for this use case as below: 1. Deploy the `snr_ssharding_shard_prov_send_notification.yaml` file: diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_without_db_gold_image.md b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_without_db_gold_image.md index c8568f1e..892741a5 100644 --- a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_without_db_gold_image.md +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_without_db_gold_image.md @@ -1,17 +1,17 @@ -# Provisioning Oracle Globally Distributed Database Topology with System-managed sharding and Raft replication enabled without Database Gold Image +# Provisioning System-Managed Sharding Topology with Raft replication enabled without Database Gold Image **NOTE: RAFT Replication Feature is available only for Oracle 23c RDBMS and Oracle 23c GSM version.** **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. -In this use case, the database is created automatically using DBCA during the provisioning of the shard databases and the catalog database when the Oracle Globally Distributed Database topology with System-Managed sharding and RAFT Replication enabled is deployed using Oracle Sharding controller. +In this use case, the database is created automatically using DBCA during the provisioning of the shard databases and the catalog database when the Oracle Sharding topology with System-Managed with RAFT Replication enabled is deployed using Oracle Sharding controller. **NOTE** In this use case, because DBCA creates the database automatically during the deployment, the time required to create the database is greater than the time it takes when the database is created by cloning from a Database Gold Image. -This example uses `snr_ssharding_shard_prov.yaml` to provision an Oracle Globally Distributed Database topology using Oracle Sharding controller with: +This example uses `snr_ssharding_shard_prov.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three Shard Database Pods: `shard1`, `shard2` and `shard3` +* Three sharding Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` * `RAFT Replication` enabled diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_scale_in_delete_an_existing_shard.md b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_scale_in_delete_an_existing_shard.md index dc026a7c..fe3157ec 100644 --- a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_scale_in_delete_an_existing_shard.md +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_scale_in_delete_an_existing_shard.md @@ -1,4 +1,4 @@ -# Scale In - Delete an existing Shard from a working Oracle Globally Distributed Database provisioned earlier with System-Managed Sharding and RAFT reolication enabled +# Scale In - Delete an existing Shard from a working Oracle Sharded Database provisioned earlier with System-Managed Sharding with RAFT reolication enabled **NOTE: RAFT Replication Feature is available only for Oracle 23c RDBMS and Oracle 23c GSM version.** @@ -8,10 +8,10 @@ This use case demonstrates how to delete an existing Shard from an existing Orac **NOTE** The deletion of a shard is done after verifying the Chunks have been moved out of that shard. -In this use case, the existing Oracle Globally Distributed Database is having: +In this use case, the existing database Sharding is having: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Five Shard Database Pods: `shard1`,`shard2`,`shard3`,`shard4` and `shard5` +* Five sharding Pods: `shard1`,`shard2`,`shard3`,`shard4` and `shard5` * One Catalog Pod: `catalog` * Namespace: `shns` * `RAFT Replication` enabled diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_scale_out_add_shards.md b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_scale_out_add_shards.md index 962bf64c..03423e72 100644 --- a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_scale_out_add_shards.md +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_scale_out_add_shards.md @@ -1,15 +1,15 @@ -# Scale Out - Add Shards to an existing Oracle Globally Distributed Database Topology provisioned earlier with System-Managed Sharding and RAFT replication enabled +# Scale Out - Add Shards to an existing Oracle Sharded Database provisioned earlier with System-Managed Sharding with RAFT replication enabled **NOTE: RAFT Replication Feature is available only for Oracle 23c RDBMS and Oracle 23c GSM version.** **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. -This use case demonstrates adding a new shard to an existing Oracle Globally Distributed Database topology with System-Managed sharding with RAFT Replication enabled provisioned earlier using Oracle Database Sharding controller. +This use case demonstrates adding a new shard to an existing Oracle Database sharding topology with System-Managed with RAFT Replication enabled provisioned earlier using Oracle Database Sharding controller. -In this use case, the existing Oracle Globally Distributed Database topology is having: +In this use case, the existing Oracle Database sharding topology is having: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three Shard Database Pods: `shard1`, `shard2` and `shard3` +* Three sharding Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` * `RAFT Replication` enabled @@ -18,7 +18,7 @@ In this example, we are using pre-built Oracle Database and Global Data Services * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `snr_ssharding_shard_prov_extshard.yaml`. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) - * If the existing Oracle Globally Distributed Database Topology was deployed using [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then the additional parameter `dbEdition: "free"` will be needed for the below .yaml file as well. + * If the existing Sharding Topology was deployed using [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then the additional parameter `dbEdition: "free"` will be needed for the below .yaml file as well. This use case adds two new shards `shard4`,`shard5` to above Sharding Topology. diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov.yaml b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov.yaml index efe3abec..aabd8470 100644 --- a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov.yaml +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov.yaml @@ -3,7 +3,7 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # --- -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: ShardingDatabase metadata: name: shardingdatabase-sample @@ -55,4 +55,3 @@ spec: role: primary - name: oltp_ro_svc role: primary - namespace: shns \ No newline at end of file diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_chunks.yaml b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_chunks.yaml index a79eafdc..def7e73a 100644 --- a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_chunks.yaml +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_chunks.yaml @@ -3,7 +3,7 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # --- -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: ShardingDatabase metadata: name: shardingdatabase-sample @@ -58,4 +58,3 @@ spec: role: primary - name: oltp_ro_svc role: primary - namespace: shns diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_clone.yaml b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_clone.yaml index 218fda0a..8f17331e 100644 --- a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_clone.yaml +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_clone.yaml @@ -3,7 +3,7 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # --- -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: ShardingDatabase metadata: name: shardingdatabase-sample @@ -80,4 +80,3 @@ spec: role: primary - name: oltp_ro_svc role: primary - namespace: shns \ No newline at end of file diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_clone_across_ads.yaml b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_clone_across_ads.yaml index 4eb3954a..d0c1c6e0 100644 --- a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_clone_across_ads.yaml +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_clone_across_ads.yaml @@ -3,7 +3,7 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # --- -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: ShardingDatabase metadata: name: shardingdatabase-sample @@ -88,4 +88,3 @@ spec: role: primary - name: oltp_ro_svc role: primary - namespace: shns \ No newline at end of file diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_delshard.yaml b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_delshard.yaml index 145ef616..0859b089 100644 --- a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_delshard.yaml +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_delshard.yaml @@ -3,7 +3,7 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # --- -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: ShardingDatabase metadata: name: shardingdatabase-sample @@ -66,4 +66,3 @@ spec: role: primary - name: oltp_ro_svc role: primary - namespace: shns \ No newline at end of file diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_extshard.yaml b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_extshard.yaml index ea0c05a5..123b3ae1 100644 --- a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_extshard.yaml +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_extshard.yaml @@ -3,7 +3,7 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # --- -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: ShardingDatabase metadata: name: shardingdatabase-sample @@ -65,4 +65,3 @@ spec: role: primary - name: oltp_ro_svc role: primary - namespace: shns \ No newline at end of file diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_memory_cpu.yaml b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_memory_cpu.yaml index 5c15c724..0cfccf9a 100644 --- a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_memory_cpu.yaml +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_memory_cpu.yaml @@ -3,7 +3,7 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # --- -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: ShardingDatabase metadata: name: shardingdatabase-sample @@ -86,4 +86,3 @@ spec: role: primary - name: oltp_ro_svc role: primary - namespace: shns \ No newline at end of file diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_send_notification.yaml b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_send_notification.yaml index 50c85443..345e9c09 100644 --- a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_send_notification.yaml +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_send_notification.yaml @@ -3,7 +3,7 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # --- -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: ShardingDatabase metadata: name: shardingdatabase-sample @@ -82,4 +82,3 @@ spec: role: primary - name: oltp_ro_svc role: primary - namespace: shns \ No newline at end of file diff --git a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md index e015f916..e457b7eb 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md +++ b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md @@ -1,4 +1,4 @@ -# Provisioning Oracle Globally Distributed Database with System-Managed Sharding by cloning database from your own Database Gold Image across Availability Domains(ADs) +# Provisioning Oracle Sharded Database with System-Managed Sharding by cloning database from your own Database Gold Image across Availability Domains(ADs) **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. @@ -22,7 +22,7 @@ NOTE: 2. Create a Block Volume Backup for this Block Volume, and use the OCID of the Block Volume Backup in the next step. This example uses `ssharding_shard_prov_clone_across_ads.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three Shard Database Pods: `shard1`, `shard2` and `shard3` +* Three sharding Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` * Database Cloning from the `BLOCK VOLUME FULL BACKUP` of the Persistent Volume which had the Gold Image. @@ -31,11 +31,11 @@ NOTE: NOTE: In this case, the Persistent Volume with DB Gold Image was provisioned in the Availability Domain `PHX-AD-1`. The Shards and Catalog will be provisioned across multiple Availability Domains by cloning the database. -**NOTE:** Provisioning the Oracle Globally Distributed Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. +**NOTE:** Provisioning the Sharded Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. - * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `ssharding_shard_prov_clone_across_ads.yaml`. + * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `ssharding_shard_prov_clone_across_ads.yaml`. * The `dbImage` used during provisioning the Persistent Volume with Database Gold Image and the `dbImage` used for deploying the Shard or Catalog Database by cloning should be same. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) diff --git a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md index fb16e3cd..cb01fa0d 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md +++ b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md @@ -1,4 +1,4 @@ -# Provisioning Oracle Globally Distributed Database with System-Managed Sharding by cloning database from your own Database Gold Image in the same Availability Domain(AD) +# Provisioning Oracle Sharded Database with System-Managed Sharding by cloning database from your own Database Gold Image in the same Availability Domain(AD) **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. @@ -19,12 +19,12 @@ Choosing this option takes substantially less time during the Oracle Globally Di 2. This example uses `ssharding_shard_prov_clone.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three Shard Database Pods: `shard1`, `shard2` and `shard3` +* Three sharding Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` * Database Cloning from the Database Gold Image present in Persistent Volume having OCID: `ocid1.volume.oc1.phx.abyhqljr3z3w72t6ay5eud7d5w3kdfhktfp6gwb6euy5tzwfaxgmbvwqlvsq` -**NOTE:** Provisioning the Oracle Globally Distributed Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. +**NOTE:** Provisioning the Sharded Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. NOTE: In this case, the Persistent Volume with DB Gold Image was provisioned in the Availability Domain `PHX-AD-1`. The Shards and Catalog will be provisioned in the same Availability Domain `PHX-AD-1` by cloning the database. diff --git a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_chunks_specified.md b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_chunks_specified.md index b824ab03..0c6ea8fe 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_chunks_specified.md +++ b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_chunks_specified.md @@ -1,4 +1,4 @@ -# Provisioning Oracle Globally Distributed Database with System-Managed Sharding with number of chunks specified +# Provisioning Oracle Sharded Database with System-Managed Sharding with number of chunks specified **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. @@ -6,12 +6,12 @@ In this use case, the database is created automatically using DBCA during the pr **NOTE:** In this use case, because DBCA creates the database automatically during the deployment, the time required to create the database is greater than the time it takes when the database is created by cloning from a Database Gold Image. -By default, the System-Managed Sharding deploys the Oracle Globally Distributed Database with 120 chunks per Shard Database. For example, if we have three shards in the Oracle Globally Distributed Database, it will be total of 360 chunks. In this example, the Oracle Globally Distributed Database will be deployed with non-default number of chunks specified using parameter `CATALOG_CHUNKS`. +By default, the System-Managed Sharding deploys the Sharded Database with 120 chunks per Shard Database. If, for example, we have three shards in the Sharded Database, it will be total of 360 chunks. In this example, the Sharded Database will be deployed with non-default number of chunks specified using parameter `CATALOG_CHUNKS`. This example uses `ssharding_shard_prov_chunks.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three Shard Database Pods: `shard1`, `shard2` and `shard3` +* Three sharding Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Total number of chunks as `120` specified by variable `CATALOG_CHUNKS` (it will be 40 chunks per shard) * Namespace: `shns` diff --git a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_control_on_resources.md b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_control_on_resources.md index 153a40a9..c4f45a48 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_control_on_resources.md +++ b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_control_on_resources.md @@ -1,4 +1,4 @@ -# Provisioning Oracle Globally Distributed Database with System-Managed Sharding with additional control on resources like Memory and CPU allocated to Pods +# Provisioning Oracle Sharded Database with System-Managed Sharding with additional control on resources like Memory and CPU allocated to Pods **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. @@ -7,7 +7,7 @@ In this use case, there are additional tags used to control resources such as CP This example uses `ssharding_shard_prov_memory_cpu.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three Shard Database Pods: `shard1`, `shard2` and `shard3` +* Three sharding Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` * Tags `memory` and `cpu` to control the Memory and CPU of the PODs diff --git a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_notification_using_oci_notification.md b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_notification_using_oci_notification.md index 7ec24439..1a6a1ee3 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_notification_using_oci_notification.md +++ b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_notification_using_oci_notification.md @@ -1,13 +1,13 @@ -# Provisioning Oracle Globally Distributed Database with System-Managed Sharding and send Notification using OCI Notification Service +# Provisioning Oracle Sharded Database with System-Managed Sharding and send Notification using OCI Notification Service **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. -This use case demonstrates how to use a notification service like OCI Notification service to send an email notification when a particular operation is completed on an Oracle Globally Distributed Database topology provisioned using the Oracle Database sharding controller. +This use case demonstrates how to use a notification service like OCI Notification service to send an email notification when a particular operation is completed on an Oracle Database sharding topology provisioned using the Oracle Database sharding controller. This example uses `ssharding_shard_prov_send_notification.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three Shard Database Pods: `shard1`, `shard2` and `shard3` +* Three sharding Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` * Database Cloning from the `BLOCK VOLUME FULL BACKUP` of the Persistent Volume that has the Database Gold Image created earlier. @@ -67,7 +67,7 @@ In this example, we are using pre-built Oracle Database and Global Data Services * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) * In case you want to use the [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then you will need to add the additional parameter `dbEdition: "free"` to the below .yaml file. -**NOTE:** Provisioning the Oracle Globally Distributed Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. +**NOTE:** Provisioning the Sharded Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. Use the file: [ssharding_shard_prov_send_notification.yaml](./ssharding_shard_prov_send_notification.yaml) for this use case as below: diff --git a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_without_db_gold_image.md b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_without_db_gold_image.md index b262407f..b223d1af 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_without_db_gold_image.md +++ b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_without_db_gold_image.md @@ -1,4 +1,4 @@ -# Provisioning Oracle Globally Distributed Database with System-Managed Sharding without Database Gold Image +# Provisioning Oracle Sharded Database with System-Managed Sharding without Database Gold Image **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. @@ -9,7 +9,7 @@ In this use case, the database is created automatically using DBCA during the pr This example uses `ssharding_shard_prov.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three Shard Database Pods: `shard1`, `shard2` and `shard3` +* Three sharding Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` diff --git a/docs/sharding/provisioning/system_sharding/ssharding_scale_in_delete_an_existing_shard.md b/docs/sharding/provisioning/system_sharding/ssharding_scale_in_delete_an_existing_shard.md index 8cac01cd..bca34253 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_scale_in_delete_an_existing_shard.md +++ b/docs/sharding/provisioning/system_sharding/ssharding_scale_in_delete_an_existing_shard.md @@ -1,4 +1,4 @@ -# Scale In - Delete an existing Shard from a working Oracle Globally Distributed Database provisioned earlier with System-Managed Sharding +# Scale In - Delete an existing Shard from a working Oracle Sharded Database provisioned earlier with System-Managed Sharding **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. @@ -6,10 +6,10 @@ This use case demonstrates how to delete an existing Shard from an existing Orac **NOTE** The deletion of a shard is done after verifying the Chunks have been moved out of that shard. -In this use case, the existing Oracle Globally Distributed Database is having: +In this use case, the existing database Sharding is having: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Five Shard Database Pods: `shard1`,`shard2`,`shard3`,`shard4` and `shard5` +* Five sharding Pods: `shard1`,`shard2`,`shard3`,`shard4` and `shard5` * One Catalog Pod: `catalog` * Namespace: `shns` @@ -21,7 +21,7 @@ In this example, we are using pre-built Oracle Database and Global Data Services NOTE: Use tag `isDelete: enable` to delete the shard you want. -This use case deletes the shard `shard4` from the above Oracle Globally Distributed Database Topology. +This use case deletes the shard `shard4` from the above Sharding Topology. Use the file: [ssharding_shard_prov_delshard.yaml](./ssharding_shard_prov_delshard.yaml) for this use case as below: diff --git a/docs/sharding/provisioning/system_sharding/ssharding_scale_out_add_shards.md b/docs/sharding/provisioning/system_sharding/ssharding_scale_out_add_shards.md index 091a01a0..1db8e6c3 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_scale_out_add_shards.md +++ b/docs/sharding/provisioning/system_sharding/ssharding_scale_out_add_shards.md @@ -1,13 +1,13 @@ -# Scale Out - Add Shards to an existing Oracle Globally Distributed Database provisioned earlier with System-Managed Sharding +# Scale Out - Add Shards to an existing Oracle Sharded Database provisioned earlier with System-Managed Sharding **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. This use case demonstrates adding a new shard to an existing Oracle Database sharding topology with System-Managed Sharding provisioned earlier using Oracle Database Sharding controller. -In this use case, the existing Oracle Globally Distributed Database topology is having: +In this use case, the existing Oracle Database sharding topology is having: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three Shard Database Pods: `shard1`, `shard2` and `shard3` +* Three sharding Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` @@ -15,9 +15,9 @@ In this example, we are using pre-built Oracle Database and Global Data Services * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `ssharding_shard_prov_extshard.yaml`. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) - * If the existing Oracle Globally Distributed Database Topology was deployed using [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then the additional parameter `dbEdition: "free"` will be needed for the below .yaml file as well. + * If the existing Sharding Topology was deployed using [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then the additional parameter `dbEdition: "free"` will be needed for the below .yaml file as well. -This use case adds two new shards `shard4`,`shard5` to above Oracle Globally Distributed Database Topology. +This use case adds two new shards `shard4`,`shard5` to above Sharding Topology. Use the file: [ssharding_shard_prov_extshard.yaml](./ssharding_shard_prov_extshard.yaml) for this use case as below: diff --git a/docs/sharding/provisioning/system_sharding/ssharding_shard_prov.yaml b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov.yaml index 7d4e16ec..5adbd2ce 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_shard_prov.yaml +++ b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov.yaml @@ -3,7 +3,7 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # --- -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: ShardingDatabase metadata: name: shardingdatabase-sample @@ -54,4 +54,3 @@ spec: role: primary - name: oltp_ro_svc role: primary - namespace: shns diff --git a/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_chunks.yaml b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_chunks.yaml new file mode 100644 index 00000000..5c135229 --- /dev/null +++ b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_chunks.yaml @@ -0,0 +1,59 @@ +# +# Copyright (c) 2022, Oracle and/or its affiliates. +# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. +# +--- +apiVersion: database.oracle.com/v4 +kind: ShardingDatabase +metadata: + name: shardingdatabase-sample + namespace: shns +spec: + shard: + - name: shard1 + storageSizeInGb: 50 + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + - name: shard2 + storageSizeInGb: 50 + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + - name: shard3 + storageSizeInGb: 50 + imagePullPolicy: "Always" + shardGroup: shardgroup1 + shardRegion: primary + catalog: + - name: catalog + storageSizeInGb: 50 + imagePullPolicy: "Always" + envVars: + - name: "CATALOG_CHUNKS" + value: "120" + gsm: + - name: gsm1 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: primary + - name: gsm2 + imagePullPolicy: "Always" + storageSizeInGb: 50 + region: standby + storageClass: oci + dbImage: container-registry.oracle.com/database/enterprise:latest + dbImagePullSecret: ocr-reg-cred + gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImagePullSecret: ocr-reg-cred + isExternalSvc: False + isDeleteOraPvc: True + dbSecret: + name: db-user-pass-rsa + pwdFileName: pwdfile.enc + keyFileName: key.pem + gsmService: + - name: oltp_rw_svc + role: primary + - name: oltp_ro_svc + role: primary diff --git a/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_clone.yaml b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_clone.yaml index 9a690626..f5816a87 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_clone.yaml +++ b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_clone.yaml @@ -3,7 +3,7 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # --- -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: ShardingDatabase metadata: name: shardingdatabase-sample @@ -79,4 +79,3 @@ spec: role: primary - name: oltp_ro_svc role: primary - namespace: shns diff --git a/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_clone_across_ads.yaml b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_clone_across_ads.yaml index b7dd1397..8fee0526 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_clone_across_ads.yaml +++ b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_clone_across_ads.yaml @@ -3,7 +3,7 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # --- -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: ShardingDatabase metadata: name: shardingdatabase-sample @@ -87,4 +87,3 @@ spec: role: primary - name: oltp_ro_svc role: primary - namespace: shns diff --git a/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_delshard.yaml b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_delshard.yaml index 75caca31..3902ceef 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_delshard.yaml +++ b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_delshard.yaml @@ -3,7 +3,7 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # --- -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: ShardingDatabase metadata: name: shardingdatabase-sample @@ -65,4 +65,3 @@ spec: role: primary - name: oltp_ro_svc role: primary - namespace: shns \ No newline at end of file diff --git a/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_extshard.yaml b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_extshard.yaml index d25dc901..a11833e0 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_extshard.yaml +++ b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_extshard.yaml @@ -3,7 +3,7 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # --- -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: ShardingDatabase metadata: name: shardingdatabase-sample @@ -64,4 +64,3 @@ spec: role: primary - name: oltp_ro_svc role: primary - namespace: shns \ No newline at end of file diff --git a/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_memory_cpu.yaml b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_memory_cpu.yaml index 793a3e4d..3f092b89 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_memory_cpu.yaml +++ b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_memory_cpu.yaml @@ -3,7 +3,7 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # --- -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: ShardingDatabase metadata: name: shardingdatabase-sample @@ -85,4 +85,3 @@ spec: role: primary - name: oltp_ro_svc role: primary - namespace: shns diff --git a/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_send_notification.yaml b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_send_notification.yaml index 96217b74..0ca6ec6f 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_send_notification.yaml +++ b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_send_notification.yaml @@ -3,7 +3,7 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # --- -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: ShardingDatabase metadata: name: shardingdatabase-sample @@ -81,5 +81,4 @@ spec: role: primary - name: oltp_ro_svc role: primary - namespace: shns diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_from_gold_image_across_ads.md b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_from_gold_image_across_ads.md index dda8a350..9b2905e8 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_from_gold_image_across_ads.md +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_from_gold_image_across_ads.md @@ -1,12 +1,12 @@ -# Provisioning Oracle Globally Distributed Database with User-Defined Sharding by cloning database from your own Database Gold Image across Availability Domains(ADs) +# Provisioning Oracle Sharded Database with User Defined Sharding by cloning database from your own Database Gold Image across Availability Domains(ADs) **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. -In this test case, you provision the Oracle Globally Distributed Database topology with User-Defined Sharding while provisioning the Catalog and Shard Databases by cloning from an existing Oracle Database Gold Image created earlier. +In this test case, you provision the Oracle Database sharding topology with User Defined Sharding while provisioning the Catalog and Shard Databases by cloning from an existing Oracle Database Gold Image created earlier. This use case applies when you want to provision the database Pods on a Kubernetes Node in any availability domain (AD), which can also be different from the availability domain (AD) of the Block Volume that has the Oracle Database Gold Image provisioned earlier. -Choosing this option takes substantially less time during the Oracle Globally Distributed Database Topology setup across ADs. +Choosing this option takes substantially less time during the Oracle Database Sharding Topology setup across ADs. NOTE: @@ -19,15 +19,15 @@ NOTE: ```sh kubectl get pv -n shns ``` -2. Create a Block Volume Backup for this Block Volume, and use the OCID of the Block Volume Backup in the next step. This example uses `udsharding_shard_prov_clone_across_ads.yaml` to provision an Oracle Globally Distributed Database topology using Oracle Sharding controller with: +2. Create a Block Volume Backup for this Block Volume, and use the OCID of the Block Volume Backup in the next step. This example uses `udsharding_shard_prov_clone_across_ads.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three Shard Database Pods: `shard1`, `shard2` and `shard3` +* Three sharding Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` * Database Cloning from the `BLOCK VOLUME FULL BACKUP` of the Persistent Volume which had the Gold Image. * OCID of the Block Volume Backup: `ocid1.volumebackup.oc1.phx.abyhqljrxtv7tu5swqb3lzc7vpzwbwzdktd2y4k2vjjy2srmgu2w7bqdftjq` -* User-Defined Sharding is specified using `shardingType: USER` +* User Defined Sharding is specified using `shardingType: USER` NOTE: In this case, the Persistent Volume with DB Gold Image was provisioned in the Availability Domain `PHX-AD-1`. The Shards and Catalog will be provisioned across multiple Availability Domains by cloning the database. @@ -37,7 +37,7 @@ In this example, we are using pre-built Oracle Database and Global Data Services * The `dbImage` used during provisioning the Persistent Volume with Database Gold Image and the `dbImage` used for deploying the Shard or Catalog Database by cloning should be same. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) -**NOTE:** Provisioning the Oracle Globally Distributed Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. +**NOTE:** Provisioning the Sharded Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. Use the file: [udsharding_shard_prov_clone_across_ads.yaml](./udsharding_shard_prov_clone_across_ads.yaml) for this use case as below: diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_gold_image_in_same_ad.md b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_gold_image_in_same_ad.md index 34fa2867..a4669667 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_gold_image_in_same_ad.md +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_gold_image_in_same_ad.md @@ -1,4 +1,4 @@ -# Provisioning Oracle Globally Distributed Database with User-Defined Sharding by cloning database from your own Database Gold Image in the same Availability Domain(AD) +# Provisioning Oracle Sharded Database with User Defined Sharding by cloning database from your own Database Gold Image in the same Availability Domain(AD) **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. @@ -6,7 +6,7 @@ In this case, the database is created automatically by cloning from an existing This use case applies when you are cloning from a Block Volume, and you can clone _only_ in the same availability domain (AD). The result is that the cloned shard database PODs can be created _only_ in the same AD where the Gold Image Block Volume is present. -Choosing this option takes substantially less time during the Oracle Globally Distributed Database Topology setup. +Choosing this option takes substantially less time during the Oracle Database Sharding Topology setup. **NOTE** For this step, the Persistent Volume that has the Oracle Database Gold Image is identified using its OCID. @@ -16,14 +16,14 @@ Choosing this option takes substantially less time during the Oracle Globally Di kubectl get pv -n shns ``` -2. This example uses `udsharding_shard_prov_clone.yaml` to provision an Oracle Globally Distributed Database topology using Oracle Sharding controller with: +2. This example uses `udsharding_shard_prov_clone.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three Shard Database Pods: `shard1`, `shard2` and `shard3` +* Three sharding Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` * Database Cloning from the Database Gold Image present in Persistent Volume having OCID: `ocid1.volume.oc1.phx.abyhqljr3z3w72t6ay5eud7d5w3kdfhktfp6gwb6euy5tzwfaxgmbvwqlvsq` -* User-Defined Sharding is specified using `shardingType: USER` +* User Defined Sharding is specified using `shardingType: USER` NOTE: In this case, the Persistent Volume with DB Gold Image was provisioned in the Availability Domain `PHX-AD-1`. The Shards and Catalog will be provisioned in the same Availability Domain `PHX-AD-1` by cloning the database. @@ -33,7 +33,7 @@ In this example, we are using pre-built Oracle Database and Global Data Services * The `dbImage` used during provisioning the Persistent Volume with Database Gold Image and the `dbImage` used for deploying the Shard or Catalog Database by cloning should be same. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) -**NOTE:** Provisioning the Oracle Globally Distributed Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. +**NOTE:** Provisioning the Sharded Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. Use the file: [udsharding_shard_prov_clone.yaml](./udsharding_shard_prov_clone.yaml) for this use case as below: diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_with_control_on_resources.md b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_with_control_on_resources.md index 8836baeb..b52b8745 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_with_control_on_resources.md +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_with_control_on_resources.md @@ -1,18 +1,18 @@ -# Provisioning Oracle Globally Distributed Database with User-Defined Sharding with additional control on resources like Memory and CPU allocated to Pods +# Provisioning Oracle Sharded Database with User Defined Sharding with additional control on resources like Memory and CPU allocated to Pods **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. -In this use case, there are additional tags used to control resources such as CPU and Memory used by the different Pods when the Oracle Globally Distributed Database topology with User-Defined Sharding is deployed using Oracle Sharding controller. +In this use case, there are additional tags used to control resources such as CPU and Memory used by the different Pods when the Oracle Sharding topology with User Defined Sharding is deployed using Oracle Sharding controller. This example uses `udsharding_shard_prov_memory_cpu.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three Shard Database Pods: `shard1`, `shard2` and `shard3` +* Three sharding Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` * Tags `memory` and `cpu` to control the Memory and CPU of the PODs * Additional tags `INIT_SGA_SIZE` and `INIT_PGA_SIZE` to control the SGA and PGA allocation at the database level -* User-Defined Sharding is specified using `shardingType: USER` +* User Defined Sharding is specified using `shardingType: USER` In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_with_notification_using_oci_notification.md b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_with_notification_using_oci_notification.md index ea3a2802..640301a2 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_with_notification_using_oci_notification.md +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_with_notification_using_oci_notification.md @@ -1,19 +1,19 @@ -# Provisioning Oracle Globally Distributed Database with User-Defined Sharding and send Notification using OCI Notification Service +# Provisioning Oracle Sharded Database with User Defined Sharding and send Notification using OCI Notification Service **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. -This use case demonstrates how to use a notification service like OCI Notification service to send an email notification when a particular operation is completed on an Oracle Globally Distributed Database topology provisioned using the Oracle Database sharding controller. +This use case demonstrates how to use a notification service like OCI Notification service to send an email notification when a particular operation is completed on an Oracle Database sharding topology provisioned using the Oracle Database sharding controller. -This example uses `udsharding_shard_prov_send_notification.yaml` to provision an Oracle Globally Distributed Database topology using Oracle Sharding controller with: +This example uses `udsharding_shard_prov_send_notification.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three Shard Database Pods: `shard1`, `shard2` and `shard3` +* Three sharding Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` * Database Cloning from the `BLOCK VOLUME FULL BACKUP` of the Persistent Volume that has the Database Gold Image created earlier. * OCID of the Block Volume Backup: `ocid1.volumebackup.oc1.phx.abyhqljrxtv7tu5swqb3lzc7vpzwbwzdktd2y4k2vjjy2srmgu2w7bqdftjq` * Configmap to send notification email when a particular operation is completed. For example: When a shard is added. -* User-Defined Sharding is specified using `shardingType: USER` +* User Defined Sharding is specified using `shardingType: USER` **NOTE:** @@ -68,7 +68,7 @@ In this example, we are using pre-built Oracle Database and Global Data Services * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) * In case you want to use the [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then you will need to add the additional parameter `dbEdition: "free"` to the below .yaml file. -**NOTE:** Provisioning the Oracle Globally Distributed Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. +**NOTE:** Provisioning the Sharded Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. Use the file: [udsharding_shard_prov_send_notification.yaml](./udsharding_shard_prov_send_notification.yaml) for this use case as below: diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_without_db_gold_image.md b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_without_db_gold_image.md index 5b1d2db0..2be5ac9f 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_without_db_gold_image.md +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_without_db_gold_image.md @@ -1,18 +1,18 @@ -# Provisioning Oracle Globally Distributed Database with User-Defined Sharding without Database Gold Image +# Provisioning Oracle Sharded Database with User Defined Sharding without Database Gold Image **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. -In this use case, the database is created automatically using DBCA during the provisioning of the shard databases and the catalog database when the Oracle Sharding topology with User-Defined Sharding is deployed using Oracle Sharding controller. +In this use case, the database is created automatically using DBCA during the provisioning of the shard databases and the catalog database when the Oracle Sharding topology with User Defined Sharding is deployed using Oracle Sharding controller. **NOTE** In this use case, because DBCA creates the database automatically during the deployment, the time required to create the database is greater than the time it takes when the database is created by cloning from a Database Gold Image. -This example uses `udsharding_shard_prov.yaml` to provision an Oracle Globally Distributed Database topology using Oracle Sharding controller with: +This example uses `udsharding_shard_prov.yaml` to provision an Oracle Database sharding topology using Oracle Sharding controller with: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three Shard Database Pods: `shard1`, `shard2` and `shard3` +* Three sharding Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` -* User-Defined Sharding is specified using `shardingType: USER` +* User Defined Sharding is specified using `shardingType: USER` In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_scale_in_delete_an_existing_shard.md b/docs/sharding/provisioning/user-defined-sharding/udsharding_scale_in_delete_an_existing_shard.md index e01e606f..2c4cbfc2 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_scale_in_delete_an_existing_shard.md +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_scale_in_delete_an_existing_shard.md @@ -1,16 +1,16 @@ -# Scale In - Delete an existing Shard from a working Oracle Globally Distributed Database provisioned earlier with User-Defined Sharding +# Scale In - Delete an existing Shard from a working Oracle Sharded Database provisioned earlier with User Defined Sharding **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. -This use case demonstrates how to delete an existing Shard from an existing Oracle Globally Distributed Database topology with User-Defined Sharding provisioned using Oracle Database Sharding controller. +This use case demonstrates how to delete an existing Shard from an existing Oracle Database sharding topology with User Defined Sharding provisioned using Oracle Database Sharding controller. In this use case, the existing database Sharding is having: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Five Shard Database Pods: `shard1`,`shard2`,`shard3`,`shard4` and `shard5` +* Five sharding Pods: `shard1`,`shard2`,`shard3`,`shard4` and `shard5` * One Catalog Pod: `catalog` * Namespace: `shns` -* User-Defined Sharding is specified using `shardingType: USER` +* User Defined Sharding is specified using `shardingType: USER` In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. @@ -20,13 +20,13 @@ In this example, we are using pre-built Oracle Database and Global Data Services **NOTE:** Use tag `isDelete: enable` to delete the shard you want. -This use case deletes the shard `shard4` from the above Oracle Globally Distributed Database Topology. +This use case deletes the shard `shard4` from the above Sharding Topology. Use the file: [udsharding_shard_prov_delshard.yaml](./udsharding_shard_prov_delshard.yaml) for this use case as below: 1. Move out the chunks from the shard to be deleted to another shard. For example, in the current case, before deleting the `shard4`, if you want to move the chunks from `shard4` to `shard2`, then you can run the below `kubectl` command where `/u01/app/oracle/product/23ai/gsmhome_1` is the GSM HOME: ```sh - kubectl exec -it pod/gsm1-0 -n shns -- /u01/app/oracle/product/23ai/gsmhome_1/bin/gdsctl "move chunk -chunk all -source shard4_shard4pdb -target shard2_shard2pdb" + kubectl exec -it pod/gsm1-0 -n shns -- /u01/app/oracle/product/23ai/gsmhome_1/bin/gdsctl "move chunk -chunk all -source shard4_shard4pdb -target shard4_shard4pdb" ``` 2. Confirm the shard to be deleted (`shard4` in this case) is not having any chunk using below command: ```sh @@ -48,7 +48,7 @@ Use the file: [udsharding_shard_prov_delshard.yaml](./udsharding_shard_prov_dels - After you apply `udsharding_shard_prov_delshard.yaml`, the change may not be visible immediately and it may take some time for the delete operation to complete. - If the shard, that you are trying to delete, is still having chunks, then the you will see message like below in the logs of the Oracle Database Operator Pod. ```sh - DEBUG events Shard Deletion failed for [shard4]. Retry shard deletion after manually moving the chunks. Requeuing + INFO controllers.database.ShardingDatabase manual intervention required ``` In this case, you will need to first move out the chunks from the shard to be deleted using Step 2 above and then apply the file in Step 3 to delete that shard. diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_scale_out_add_shards.md b/docs/sharding/provisioning/user-defined-sharding/udsharding_scale_out_add_shards.md index 371b2438..20f50b29 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_scale_out_add_shards.md +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_scale_out_add_shards.md @@ -1,24 +1,24 @@ -# Scale Out - Add Shards to an existing Oracle Globally Distributed Database provisioned earlier with User-Defined Sharding +# Scale Out - Add Shards to an existing Oracle Sharded Database provisioned earlier with User Defined Sharding **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. -This use case demonstrates adding a new shard to an existing Oracle Globally Distributed Database topology with User-Defined Sharding provisioned earlier using Oracle Database Sharding controller. +This use case demonstrates adding a new shard to an existing Oracle Database sharding topology with User Defined Sharding provisioned earlier using Oracle Database Sharding controller. -In this use case, the existing Oracle Globally Distributed Database topology is having: +In this use case, the existing Oracle Database sharding topology is having: * Primary GSM Pods `gsm1` and standby GSM Pod `gsm2` -* Three Shard Database Pods: `shard1`, `shard2` and `shard3` +* Three sharding Pods: `shard1`, `shard2` and `shard3` * One Catalog Pod: `catalog` * Namespace: `shns` -* User-Defined Sharding is specified using `shardingType: USER` +* User Defined Sharding is specified using `shardingType: USER` In this example, we are using pre-built Oracle Database and Global Data Services container images available on [Oracle Container Registry](https://container-registry.oracle.com/) * To pull the above images from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `udsharding_shard_prov_extshard.yaml`. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) - * If the existing Oracle Globally Distributed Database Topology was deployed using [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then the additional parameter `dbEdition: "free"` will be needed for the below .yaml file as well. + * If the existing Sharding Topology was deployed using [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then the additional parameter `dbEdition: "free"` will be needed for the below .yaml file as well. -This use case adds two new shards `shard4`,`shard5` to above Oracle Globally Distributed Database Topology. +This use case adds two new shards `shard4`,`shard5` to above Sharding Topology. Use the file: [udsharding_shard_prov_extshard.yaml](./udsharding_shard_prov_extshard.yaml) for this use case as below: diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov.yaml b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov.yaml index 9b565b73..d33be599 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov.yaml +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov.yaml @@ -3,7 +3,7 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # --- -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: ShardingDatabase metadata: name: shardingdatabase-sample @@ -55,4 +55,3 @@ spec: role: primary - name: oltp_ro_svc role: primary - namespace: shns diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_clone.yaml b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_clone.yaml index adc2271f..04ee5d95 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_clone.yaml +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_clone.yaml @@ -3,7 +3,7 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # --- -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: ShardingDatabase metadata: name: shardingdatabase-sample @@ -80,4 +80,3 @@ spec: role: primary - name: oltp_ro_svc role: primary - namespace: shns diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_clone_across_ads.yaml b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_clone_across_ads.yaml index 28f36608..5be6ecde 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_clone_across_ads.yaml +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_clone_across_ads.yaml @@ -3,7 +3,7 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # --- -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: ShardingDatabase metadata: name: shardingdatabase-sample @@ -88,4 +88,3 @@ spec: role: primary - name: oltp_ro_svc role: primary - namespace: shns diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_delshard.yaml b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_delshard.yaml index 2342dc55..e00d2272 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_delshard.yaml +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_delshard.yaml @@ -3,7 +3,7 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # --- -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: ShardingDatabase metadata: name: shardingdatabase-sample @@ -66,4 +66,3 @@ spec: role: primary - name: oltp_ro_svc role: primary - namespace: shns \ No newline at end of file diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_extshard.yaml b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_extshard.yaml index f45d421f..3899f2ab 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_extshard.yaml +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_extshard.yaml @@ -2,7 +2,7 @@ # Copyright (c) 2022, Oracle and/or its affiliates. # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: ShardingDatabase metadata: name: shardingdatabase-sample @@ -64,4 +64,3 @@ spec: role: primary - name: oltp_ro_svc role: primary - namespace: shns diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_memory_cpu.yaml b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_memory_cpu.yaml index e663aa65..6c65916e 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_memory_cpu.yaml +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_memory_cpu.yaml @@ -3,7 +3,7 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # --- -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: ShardingDatabase metadata: name: shardingdatabase-sample @@ -86,4 +86,3 @@ spec: role: primary - name: oltp_ro_svc role: primary - namespace: shns diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_send_notification.yaml b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_send_notification.yaml index afd951fe..ef1b5561 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_send_notification.yaml +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_send_notification.yaml @@ -3,7 +3,7 @@ # Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl. # --- -apiVersion: database.oracle.com/v1alpha1 +apiVersion: database.oracle.com/v4 kind: ShardingDatabase metadata: name: shardingdatabase-sample @@ -82,4 +82,3 @@ spec: role: primary - name: oltp_ro_svc role: primary - namespace: shns diff --git a/docs/sidb/PREREQUISITES.md b/docs/sidb/PREREQUISITES.md index b904f5da..4bf09283 100644 --- a/docs/sidb/PREREQUISITES.md +++ b/docs/sidb/PREREQUISITES.md @@ -3,27 +3,26 @@ To deploy Oracle Single Instance Database in Kubernetes using the OraOperator, c * ### Prepare Oracle Container Images - Build Single Instance Database Container Images from source, following the instructions at [https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance), or - use the pre-built images available at [https://container-registry.oracle.com](https://container-registry.oracle.com) by signing in and accepting the required license agreement. + You can either build Single Instance Database Container Images from the source, following the instructions at [https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance), or you can use the the pre-built images available at [https://container-registry.oracle.com](https://container-registry.oracle.com) by signing in and accepting the required license agreement. - Oracle Database Releases Supported: Enterprise and Standard Edition for Oracle Database 19c, and later releases. Express Edition for Oracle Database 21.3.0 only. Oracle Database Free 23.2.0 and later Free releases + Oracle Database Releases Supported: Enterprise and Standard Edition for Oracle Database 19c, and later releases. Express Edition for Oracle Database 21.3.0 only. Oracle Database Free 23.2.0 and later Free releases Build Oracle REST Data Service Container Images from source following the instructions at [https://github.com/oracle/docker-images/tree/main/OracleRestDataServices](https://github.com/oracle/docker-images/tree/main/OracleRestDataServices). - Supported Oracle REST Data Service version is 21.4.2 + The supported Oracle REST Data Service version is 21.4.2 * ### Ensure Sufficient Disk Space in Kubernetes Worker Nodes - Provision Kubernetes worker nodes with recommended 250 GiB or more of free disk space required for pulling the base and patched database container images. If deploying on cloud you may choose to increase the custom boot volume size of the worker nodes. + Provision Kubernetes worker nodes. Oracle recommends you provision them with 250 GB or more free disk space, which is required for pulling the base and patched database container images. If you are doing a Cloud deployment, then you can choose to increase the custom boot volume size of the worker nodes. * ### Set Up Kubernetes and Volumes for Database Persistence Set up an on-premises Kubernetes cluster, or subscribe to a managed Kubernetes service, such as Oracle Cloud Infrastructure Container Engine for Kubernetes. Use a dynamic volume provisioner or pre-provision static persistent volumes manually. These volumes are required for persistent storage of the database files. - More info on creating persistent volumes available at [https://kubernetes.io/docs/concepts/storage/persistent-volumes/](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) + For more more information about creating persistent volumes, see: [https://kubernetes.io/docs/concepts/storage/persistent-volumes/](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) * ### Minikube Cluster Environment - By default, Minikube creates a node with 2GB RAM, 2 CPUs, and 20GB disk space when a cluster is created using `minikube start` command. However, these resources (particularly disk space and RAM) may not be sufficient for running and managing Oracle Database using the OraOperator. It is recommended to have larger RAM and disk space for better performance. For example, the following command creates a Minikube cluster with 8GB RAM and 100GB disk space for the Minikube VM: + By default, when you create a cluster using the `minicube start` command, Minikube creates a node with 2GB RAM, 2 CPUs, and 20GB disk space. However, these resources (particularly disk space and RAM) may not be sufficient for running and managing Oracle Database using the OraOperator. For better performance, Oracle recommends that you configure the cluster to have a larger RAM and disk space than the Minikube default. For example, the following command creates a Minikube cluster with 8GB RAM and 100GB disk space for the Minikube VM: ``` minikube start --memory=8g --disk-size=100g diff --git a/docs/sidb/README.md b/docs/sidb/README.md index ff357195..35f42f22 100644 --- a/docs/sidb/README.md +++ b/docs/sidb/README.md @@ -12,6 +12,8 @@ Oracle Database Operator for Kubernetes (`OraOperator`) includes the Single Inst * [Pre-built Database](#pre-built-database) * [XE Database](#xe-database) * [Free Database](#free-database) + * [Free Lite Database](#free-lite-database) + * [Oracle True Cache](#oracle-true-cache) * [Connecting to Database](#connecting-to-database) * [Database Persistence (Storage) Configuration Options](#database-persistence-storage-configuration-options) * [Dynamic Persistence](#dynamic-persistence) @@ -29,17 +31,21 @@ Oracle Database Operator for Kubernetes (`OraOperator`) includes the Single Inst * [Setup Database with LoadBalancer](#setup-database-with-loadbalancer) * [Enabling TCPS Connections](#enabling-tcps-connections) * [Specifying Custom Ports](#specifying-custom-ports) - * [Setup Data Guard Configuration for a Single Instance Database (Preview status)](#setup-data-guard-configuration-for-a-single-instance-database-preview-status) + * [Setup Data Guard Configuration for a Single Instance Database](#setup-data-guard-configuration-for-a-single-instance-database) * [Create a Standby Database](#create-a-standby-database) * [Create a Data Guard Configuration](#create-a-data-guard-configuration) * [Perform a Switchover](#perform-a-switchover) - * [Patch Primary and Standby databases in Data Guard configuration](#patch-primary-and-standby-databases-in-data-guard-configuration) + * [Enable Fast-Start Failover](#enable-fast-start-failover) + * [Convert Standby to Snapshot Standby](#convert-standby-to-snapshot-standby) + * [Static Data Guard Connect String](#static-data-guard-connect-string) + * [Patch Primary and Standby databases](#patch-primary-and-standby-databases) * [Delete the Data Guard Configuration](#delete-the-data-guard-configuration) * [Execute Custom Scripts](#execute-custom-scripts) * [OracleRestDataService Resource](#oraclerestdataservice-resource) * [REST Enable a Database](#rest-enable-a-database) * [Provision ORDS](#provision-ords) * [Database API](#database-api) + * [MongoDB API](#mongodb-api) * [Advanced Usages](#advanced-usages) * [Oracle Data Pump](#oracle-data-pump) * [REST Enabled SQL](#rest-enabled-sql) @@ -52,7 +58,7 @@ Oracle Database Operator for Kubernetes (`OraOperator`) includes the Single Inst ## Prerequisites -Oracle strongly recommends to comply with the [prerequisites](./PREREQUISITES.md) and the following requirements +Oracle strongly recommends that you comply with the [prerequisites](./PREREQUISITES.md) and the following requirements ### Mandatory Resource Privileges @@ -67,11 +73,11 @@ Oracle strongly recommends to comply with the [prerequisites](./PREREQUISITES.md | Secrets | create delete get list patch update watch | | Events | create patch | - For managing the required levels of access, configure [role binding](../../README.md#role-binding-for-access-management) + For managing the required levels of access, configure [role binding](../../README.md#create-role-bindings-for-access-management) ### Optional Resource Privileges - Single Instance Database(sidb) controller optionally requires the following Kubernetes resource privileges depending on the functionality being used: + Single Instance Database(`sidb`) controller optionally requires the following Kubernetes resource privileges, depending on the functionality being used: | Functionality | Resources | Privileges | | --- | --- | --- | @@ -80,7 +86,7 @@ Oracle strongly recommends to comply with the [prerequisites](./PREREQUISITES.md | Custom Scripts Execution | PersistentVolumes | get list watch | - For exposing the database via Nodeport services, apply [RBAC](../../rbac/node-rbac.yaml) + For exposing the database using Nodeport services, apply [RBAC](../../rbac/node-rbac.yaml) ```sh kubectl apply -f rbac/node-rbac.yaml ``` @@ -88,22 +94,22 @@ Oracle strongly recommends to comply with the [prerequisites](./PREREQUISITES.md ```sh kubectl apply -f rbac/storage-class-rbac.yaml ``` - For automatic execution of custom scripts post database setup or startup, apply [RBAC](../../rbac/persistent-volume-rbac.yaml) + For automatic execution of custom scripts after database setup or startup, apply [RBAC](../../rbac/persistent-volume-rbac.yaml) ```sh kubectl apply -f rbac/persistent-volume-rbac.yaml ``` ### OpenShift Security Context Constraints - OpenShift requires additional Security Context Constraints (SCC) for deploying and managing the SingleInstanceDatabase resource. Follow these steps to create the appropriate SCCs before deploying the SingleInstanceDatabase resource. + OpenShift requires additional Security Context Constraints (SCC) for deploying and managing the `SingleInstanceDatabase` resource. To create the appropriate SCCs before deploying the `SingleInstanceDatabase` resource, complete these steps: - 1. Create a new project/namespace for deploying the SingleInstanceDatabase resource + 1. Create a new project/namespace for deploying the `SingleInstanceDatabase` resource ```sh oc new-project sidb-ns ``` - **Note:** OpenShift recommends not to deploy in namespaces starting with `kube`, `openshift` and the `default` namespace. + **Note:** OpenShift recommends that you should not deploy in namespaces starting with `kube`, `openshift` and the `default` namespace. 2. Apply the file [openshift_rbac.yaml](../../config/samples/sidb/openshift_rbac.yaml) with cluster-admin user privileges. @@ -111,15 +117,15 @@ Oracle strongly recommends to comply with the [prerequisites](./PREREQUISITES.md oc apply -f openshift-rbac.yaml ``` - This would result in creation of SCC (Security Context Constraints) and serviceaccount `sidb-sa` in the namespace `sidb-ns` which has access to the SCC. + Running this example procedure results in creation of SCC (Security Context Constraints) and serviceaccount `sidb-sa` in the namespace `sidb-ns`, which has access to the SCC. - **Note:** The above config yaml file will bind the SCC to the serviceaccount `sidb-sa` in namespace `sidb-ns`. For any other project/namespace update the file appropriately with the namespace before applying. + **Note:** This configuration yaml file example binds the SCC to the serviceaccount `sidb-sa` in namespace `sidb-ns`. For any other project/namespace, you must update the file appropriately with the namespace before applying this example. 3. Set the `serviceAccountName` attribute to `sidb-sa` and the namespace to `sidb-ns` in **[config/samples/sidb/singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml)** before deploying the SingleInstanceDatabase resource. ## SingleInstanceDatabase Resource -The Oracle Database Operator creates the `SingleInstanceDatabase` as a custom resource. Doing this enables Oracle Database to be managed as a native Kubernetes object. We will refer `SingleInstanceDatabase` resource as Database from now onwards. +The Oracle Database Operator creates the `SingleInstanceDatabase` as a custom resource. Doing this enables Oracle Database to be managed as a native Kubernetes object. In this document, we will refer to the `SingleInstanceDatabase` resource as the database. ### Resource Details @@ -207,22 +213,22 @@ $ kubectl describe singleinstancedatabase sidb-sample-clone ### Template YAML The template `.yaml` file for Single Instance Database (Enterprise and Standard Editions), including all the configurable options, is available at: -**[config/samples/sidb/singleinstancedatabase.yaml](./../../config/samples/sidb/singleinstancedatabase.yaml)** +**[`config/samples/sidb/singleinstancedatabase.yaml`](./../../config/samples/sidb/singleinstancedatabase.yaml)** **Note:** -The `adminPassword` field in the above `singleinstancedatabase.yaml` file refers to a secret for the SYS, SYSTEM and PDBADMIN users of the Single Instance Database. This secret is required when you provision a new database, or when you clone an existing database. +The `adminPassword` field in the above `singleinstancedatabase.yaml`example file refers to a Secret for the SYS, SYSTEM and PDBADMIN users of the Single Instance Database. This Secret is required when you provision a new database, or when you clone an existing database. -Create this secret using the following command as an example: +Create this Secret using the following command as an example: kubectl create secret generic db-admin-secret --from-literal=oracle_pwd= -This command creates a secret named `db-admin-secret`, with the key `oracle_pwd` mapped to the actual password specified in the command. +This command creates a Secret named `db-admin-secret`, with the key `oracle_pwd` mapped to the actual password specified in the command. ### Create a Database #### New Database -To provision a new database instance on the Kubernetes cluster, use the example **[config/samples/sidb/singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml)**. +To provision a new database instance on the Kubernetes cluster, use the example **[`config/samples/sidb/singleinstancedatabase_create.yaml`](../../config/samples/sidb/singleinstancedatabase_create.yaml)**. 1. Log into [Oracle Container Registry](https://container-registry.oracle.com/) and accept the license agreement for the Database image; ignore if you have accepted the license agreement already. @@ -254,15 +260,15 @@ To provision a new database instance on the Kubernetes cluster, use the example ``` **Note:** -- For ease of use, the storage class **oci-bv** is specified in the **[singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml)**. This storage class facilitates dynamic provisioning of the OCI block volumes on the Oracle OKE for persistent storage of the database. The supported access mode for this class is `ReadWriteOnce`. For other cloud providers, you can similarly use their dynamic provisioning storage classes. +- For ease of use, the storage class **oci-bv** is specified in the **[`singleinstancedatabase_create.yaml`](../../config/samples/sidb/singleinstancedatabase_create.yaml)**. This storage class facilitates dynamic provisioning of the OCI block volumes on the Oracle OKE for persistent storage of the database. The supported access mode for this class is `ReadWriteOnce`. For other cloud providers, you can similarly use their dynamic provisioning storage classes. - It is beneficial to have the database replica pods more than or equal to the number of available nodes if `ReadWriteMany` access mode is used with the OCI NFS volume. By doing so, the pods get distributed on different nodes and the database image is downloaded on all those nodes. This helps in reducing time for the database fail-over if the active database pod dies. - Supports Oracle Database Enterprise Edition (19.3.0), and later releases. -- To pull the database image faster from the container registry, so that you can bring up the SIDB instance quickly, you can use the container-registry mirror of the corresponding cluster's region. For example, if the cluster exists in Mumbai region, then you can use the `container-registry-bom.oracle.com` mirror. For more information on container-registry mirrors, follow the link [https://blogs.oracle.com/wim/post/oracle-container-registry-mirrors-in-oracle-cloud-infrastructure](https://blogs.oracle.com/wim/post/oracle-container-registry-mirrors-in-oracle-cloud-infrastructure). -- To update the init parameters like `sgaTarget` and `pgaAggregateTarget`, refer the `initParams` section of the [singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml) file. +- To pull the database image faster from the container registry, so that you can bring up the SIDB instance quickly, you can use the `container-registry mirror` of the corresponding cluster's region. For example, if the cluster exists in Mumbai region, then you can use the `container-registry-bom.oracle.com` mirror. For more information on container-registry mirrors, see: [https://blogs.oracle.com/wim/post/oracle-container-registry-mirrors-in-oracle-cloud-infrastructure](https://blogs.oracle.com/wim/post/oracle-container-registry-mirrors-in-oracle-cloud-infrastructure). +- To update the initialization (init) parameters, such as `sgaTarget` and `pgaAggregateTarget`, see the `initParams` section of the [`singleinstancedatabase.yaml`](../../config/samples/sidb/singleinstancedatabase.yaml) file. #### Pre-built Database -To provision a new pre-built database instance, use the sample **[config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml](../../config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml)** file. For example: +To provision a new pre-built database instance, use the sample **[`config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml](../../config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml)** file. For example: ```sh $ kubectl apply -f singleinstancedatabase_prebuiltdb.yaml @@ -274,32 +280,45 @@ This pre-built image includes the data files of the database inside the image it To build the pre-built database image for the Enterprise/Standard edition, follow these instructions: [Pre-built Database (prebuiltdb) Extension](https://github.com/oracle/docker-images/blob/main/OracleDatabase/SingleInstance/extensions/prebuiltdb/README.md). #### XE Database -To provision new Oracle Database Express Edition (XE) database, use the sample **[config/samples/sidb/singleinstancedatabase_express.yaml](../../config/samples/sidb/singleinstancedatabase_express.yaml)** file. For example: +To provision a new Oracle Database Express Edition (XE) database, use the sample **[config/samples/sidb/singleinstancedatabase_express.yaml](../../config/samples/sidb/singleinstancedatabase_express.yaml)** file. For example: kubectl apply -f singleinstancedatabase_express.yaml -This command pulls the XE image uploaded on the [Oracle Container Registry](https://container-registry.oracle.com/). +This command pulls the XE image available in [Oracle Container Registry](https://container-registry.oracle.com/). **Note:** -- Provisioning Oracle Database express edition is supported for release 21c (21.3.0) only. +- Provisioning Oracle Database Express Edition is supported for release 21c (21.3.0) only. Oracle Database Free replaces Oracle Database Express Edition. - For XE database, only single replica mode (i.e. `replicas: 1`) is supported. -- For XE database, you **cannot change** the init parameters i.e. `cpuCount, processes, sgaTarget or pgaAggregateTarget`. +- For XE database, you **cannot change** the init parameters, such as `cpuCount, processes, sgaTarget or pgaAggregateTarget`. #### Free Database -To provision new Oracle Database Free database, use the sample **[config/samples/sidb/singleinstancedatabase_free.yaml](../../config/samples/sidb/singleinstancedatabase_free.yaml)** file. For example: +To provision new Oracle Database Free, use the sample **[config/samples/sidb/singleinstancedatabase_free.yaml](../../config/samples/sidb/singleinstancedatabase_free.yaml)** file. For example: kubectl apply -f singleinstancedatabase_free.yaml -This command pulls the Free image uploaded on the [Oracle Container Registry](https://container-registry.oracle.com/). +This command pulls the Free image available in [Oracle Container Registry](https://container-registry.oracle.com/). + +#### Free Lite Database +To provision new Oracle Database Free Lite, use the sample **[config/samples/sidb/singleinstancedatabase_free-lite.yaml](../../config/samples/sidb/singleinstancedatabase_free-lite.yaml)** file. For example: + + kubectl apply -f singleinstancedatabase_free-lite.yaml + +This command pulls the Free lite image available in [Oracle Container Registry](https://container-registry.oracle.com/). **Note:** - Provisioning Oracle Database Free is supported for release 23.3.0 and later releases. -- For Free database, only single replica mode (i.e. `replicas: 1`) is supported. -- For Free database, you **cannot change** the init parameters i.e. `cpuCount, processes, sgaTarget or pgaAggregateTarget`. -- Oracle Enterprise Manager Express (OEM Express) is not supported from release 23.3.0 and later releases. +- For Free database, only single replica mode (such as `replicas: 1`) is supported. +- For Free database, you **cannot change** the init parameters. These include parameters such as `cpuCount, processes, sgaTarget or pgaAggregateTarget`. +- Oracle Enterprise Manager Express (OEM Express) is not supported in release 23.3.0 and later releases. + +#### Oracle True Cache +Oracle True Cache is an in-memory, consistent, and automatically managed cache for Oracle Database. +To provision a True Cache instance for Oracle Free Database in Kubernetes, use the sample **[`config/samples/sidb/singleinstancedatabase_free-truecache.yaml`](../../config/samples/sidb/singleinstancedatabase_free-truecache.yaml)** file. For example + + kubectl apply -f singleinstancedatabase_free-truecache.yaml #### Additional Information -You are required to specify the database admin password secret in the corresponding YAML file. The default values mentioned in the `adminPassword.secretName` fields of [singleinstancedatabase_create.yaml](../../config/samples/sidb/singleinstancedatabase_create.yaml), [singleinstancedatabase_prebuiltdb.yaml](../../config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml), [singleinstancedatabase_express.yaml](../../config/samples/sidb/singleinstancedatabase_express.yaml) and [singleinstancedatabse_free.yaml](../../config/samples/sidb/singleinstancedatabase_free.yaml) files are `db-admin-secret`, `prebuiltdb-admin-secret`, `xedb-admin-secret` and `free-admin-secret` respectively. You can create these secrets manually by using the sample command mentioned in the [Template YAML](#template-yaml) section. Alternatively, you can create these secrets by filling the passwords in the **[singleinstancedatabase_secrets.yaml](../../config/samples/sidb/singleinstancedatabase_secrets.yaml)** file and applying it using the command below: +You are required to specify the database administrative user (admin) password Secret in the corresponding YAML file. The default values mentioned in the `adminPassword.secretName` fields of [`singleinstancedatabase_create.yaml`](../../config/samples/sidb/singleinstancedatabase_create.yaml), [`singleinstancedatabase_prebuiltdb.yaml`](../../config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml), [`singleinstancedatabase_express.yaml`](../../config/samples/sidb/singleinstancedatabase_express.yaml) and [`singleinstancedatabse_free.yaml`](../../config/samples/sidb/singleinstancedatabase_free.yaml) files are `db-admin-secret`, `prebuiltdb-admin-secret`, `xedb-admin-secret` and `free-admin-secret` respectively. You can create these Secrets manually by using the sample command mentioned in the [`Template YAML`](#template-yaml) section. Alternatively, you can create these Secrets by filling in the passwords in the **[`singleinstancedatabase_secrets.yaml`](../../config/samples/sidb/singleinstancedatabase_secrets.yaml)** file and applying them using the following command: ```bash kubectl apply -f singleinstancedatabase_secrets.yaml @@ -307,7 +326,7 @@ kubectl apply -f singleinstancedatabase_secrets.yaml ### Connecting to Database -Creating a new database instance takes a while. When the `status` column returns the response `Healthy`, the Database is open for connections. +Creating a new database instance takes a while. When the `status` column returns the response `Healthy`, the database is open for connections. ```sh $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.status}" @@ -315,7 +334,7 @@ $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.status}" Healthy ``` -Clients can get the connect-string to the CDB from `.status.connectString` and PDB from `.status.pdbConnectString`. For example: +Clients can obtain the connect string to the CDB from `.status.connectString`, and the connect string to the PDB from `.status.pdbConnectString`. For example: ```sh $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.connectString}" @@ -328,7 +347,7 @@ $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.pdbConnec 10.0.25.54:1521/ORCLPDB ``` -Use any supported client or SQLPlus to connect to the database using the above connect strings as follows +To connect to the database using the connect strings returned by the commands above, you can use any supported client, or use SQLPlus. For example: ```sh $ sqlplus sys/<.spec.adminPassword>@10.0.25.54:1521/ORCL as sysdba @@ -356,7 +375,7 @@ $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.oemExpres **Note:** OEM Express is not available for 23.3.0 and later releases ### Database Persistence (Storage) Configuration Options -The database persistence can be achieved in the following two ways: +You can configure database persistence in the following two ways: - Dynamic Persistence Provisioning - Static Persistence Provisioning @@ -379,8 +398,8 @@ $ kubectl patch singleinstancedatabase sidb-sample -p '{"spec":{"persistence":{" - User can only scale up a volume/storage and not scale down #### Static Persistence -In **Static Persistence Provisioning**, you have to create a volume manually, and then use the name of this volume with the `<.spec.persistence.datafilesVolumeName>` field which corresponds to the `datafilesVolumeName` field of the persistence section in the **[singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml)**. The `Reclaim Policy` of such volume can be set to `Retain`. So, this volume does not get deleted with the deletion of its corresponding deployment. -For example in **Minikube**, a persistent volume can be provisioned using the sample yaml file below: +In **Static Persistence Provisioning**, you must create a volume manually, and then use the name of this volume with the `<.spec.persistence.datafilesVolumeName>` field, which corresponds to the `datafilesVolumeName` field of the persistence section in the **[`singleinstancedatabase.yaml`](../../config/samples/sidb/singleinstancedatabase.yaml)**. The `Reclaim Policy` of such volumes can be set to `Retain`. When this policy is set, the volume is not deleted when its corresponding deployment is deleted. +For example in **Minikube**, a persistent volume can be provisioned using the following yaml file example: ```yaml apiVersion: v1 kind: PersistentVolume @@ -395,7 +414,7 @@ spec: hostPath: path: /data/oradata ``` -The persistent volume name (i.e. db-vol) can be mentioned in the `datafilesVolumeName` field of the **[singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml)**. `storageClass` field is not required in this case, and can be left empty. +The persistent volume name (in this case, `db-vol`) can be mentioned in the `datafilesVolumeName` field of the **[`singleinstancedatabase.yaml`](../../config/samples/sidb/singleinstancedatabase.yaml)**. `storageClass` field is not required in this case, and can be left empty. Static Persistence Provisioning in Oracle Cloud Infrastructure (OCI) is explained in the following subsections: @@ -452,7 +471,7 @@ spec: - Whenever a mount target is provisioned in OCI, its `Reported Size (GiB)` values are very large. This is visible on the mount target page when logged in to the OCI console. Some applications will fail to install if the results of a space requirements check show too much available disk space. So in the OCI Console, click the little "Pencil" icon besides the **Reported Size** parameter of the Mount Target to specify, in gigabytes (GiB), the maximum capacity reported by file systems exported through this mount target. This setting does not limit the actual amount of data you can store. -- Make sure to open the required ports to access the NFS volume from the K8S cluster: add the required ports to the security list of the subnet where your K8S nodes are connected to; see **[here](https://docs.oracle.com/en-us/iaas/Content/File/Tasks/securitylistsfilestorage.htm)** for the details. +- You must open the required ports to access the NFS volume from the K8S cluster. Add the required ports to the security list of the subnet to which your K8S nodes are connected. For more information, see **[Security Lists File Storage](https://docs.oracle.com/en-us/iaas/Content/File/Tasks/securitylistsfilestorage.htm)** for the details. ### Configuring a Database The `OraOperator` facilitates you to configure the database. Various database configuration options are explained in the following subsections: @@ -508,7 +527,7 @@ The following attributes cannot be modified after creating the Single Instance D - `pdbName` - `primaryDatabaseRef` -If you attempt to changing one of these attributes, then you receive an error similar to the following: +If you attempt to change one of these attributes, then you receive an error similar to the following: ```sh $ kubectl --type=merge -p '{"spec":{"sid":"ORCL1"}}' patch singleinstancedatabase sidb-sample @@ -520,7 +539,7 @@ $ kubectl --type=merge -p '{"spec":{"sid":"ORCL1"}}' patch singleinstancedatabas To create copies of your existing database quickly, you can use the cloning functionality. A cloned database is an exact, block-for-block copy of the source database. Cloning is much faster than creating a fresh database and copying over the data. -To quickly clone the existing database sidb-sample created above, use the sample **[config/samples/sidb/singleinstancedatabase_clone.yaml](../../config/samples/sidb/singleinstancedatabase_clone.yaml)** file. +To quickly clone the existing database `sidb-sample` we previously created for this document, use the sample **[`config/samples/sidb/singleinstancedatabase_clone.yaml`](../../config/samples/sidb/singleinstancedatabase_clone.yaml)** file. For example: @@ -544,7 +563,7 @@ Patched Oracle Docker images can be built by using this [patching extension](htt #### Patch -To patch an existing database, edit and apply the **[config/samples/sidb/singleinstancedatabase_patch.yaml](../../config/samples/sidb/singleinstancedatabase_patch.yaml)** file of the database resource/object either by specifying a new release update for image attributes, or by running the following command: +To patch an existing database, edit and apply the **[`config/samples/sidb/singleinstancedatabase_patch.yaml`](../../config/samples/sidb/singleinstancedatabase_patch.yaml)** file of the database resource/object either by specifying a new release update for image attributes, or by running the following command: ```sh kubectl --type=merge -p '{"spec":{"image":{"pullFrom":"patched-image:tag","pullSecrets":"pull-secret"}}}' patch singleinstancedatabase sidb-sample @@ -556,7 +575,7 @@ singleinstancedatabase.database.oracle.com/sidb-sample patched After patching is complete, the database pods are restarted with the new release update image. **Note:** -- Only enterprise and standard editions support patching. +- Only Enterprise and Standard Editions support patching. #### Patch after Cloning @@ -578,7 +597,7 @@ $ kubectl get singleinstancedatabase sidb-sample -o "jsonpath={.status.releaseUp ``` #### Rollback -You can roll back to a prior database version by specifying the old image in the `image` field of the **[config/samples/sidb/singleinstancedatabase_patch.yaml](../../config/samples/sidb/singleinstancedatabase_patch.yaml)** file, and applying it by the following command: +You can roll back to a prior database version by specifying the old image in the `image` field of the **[`config/samples/sidb/singleinstancedatabase_patch.yaml`](../../config/samples/sidb/singleinstancedatabase_patch.yaml)** file, and applying it using the following command: ```bash kubectl apply -f singleinstancedatabase_patch.yaml @@ -594,12 +613,12 @@ singleinstancedatabase.database.oracle.com/sidb-sample patched ``` ### Delete a Database -Please run the following command to delete the database: +To delete the database, run the following command : ```bash kubectl delete singleinstancedatabase.database.oracle.com sidb-sample ``` -The command above will delete the database pods and associated service. +This command will delete the database pods and associated service. ### Advanced Database Configurations Some advanced database configuration scenarios are as follows: @@ -623,17 +642,20 @@ The following table depicts the fail over matrix for any destructive operation t | PDB close | No | **Note:** -- Maintence shutdown/startup can be executed using the scripts /home/oracle/shutDown.sh and /home/oracle/startUp.sh +- Maintence shutdown/startup can be run by using the scripts `/home/oracle/shutDown.sh` and `/home/oracle/startUp.sh` - This functionality requires the [k8s extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/k8s) extended images. The database image from the container registry `container-registry.oracle.com` includes the K8s extension. - Because Oracle Database Express Edition (XE) does not support [k8s extension](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance/extensions/k8s), it does not support multiple replicas. -- If the `ReadWriteOnce` access mode is used, all the replicas will be scheduled on the same node where the persistent volume would be mounted. -- If the `ReadWriteMany` access mode is used, all the replicas will be distributed on different nodes. So, it is recommended to have replicas more than or equal to the number of the nodes as the database image is downloaded on all those nodes. This is beneficial in quick cold fail-over scenario (when the active pod dies) as the image would already be available on that node. +- If the `ReadWriteOnce` access mode is used, then all the replicas will be scheduled on the same node where the persistent volume would be mounted. +- If the `ReadWriteMany` access mode is used, then all the replicas will be distributed on different nodes. For this reason, Oracle recommends that you have replicas more than or equal to the number of the nodes, because the database image is downloaded on all those nodes. This is beneficial in quick cold fail-over scenario (when the active pod dies) as the image would already be available on that node. + +#### Database Pod Resource Management +When creating a Single Instance Database, you can specify the CPU and memory resources needed by the database pod. These specified resources are passed to the `kube-scheduler` so that the pod is scheduled on one of the pods that has the required resources available. To use database pod resource management, specify values for the `resources` attributes in the [`config/samples/sidb/singleinstancedatabase.yaml`](../../config/samples/sidb/singleinstancedatabase.yaml) file, and apply it. #### Database Pod Resource Management When creating a Single Instance Database you can specify the cpu and memory resources needed by the database pod. These specified resources are passed to the `kube-scheduler` so that the pod gets scheduled on one of the pods that has the required resources available. To use database pod resource management specify values for the `resources` attributes in the [config/samples/sidb/singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml) file, and apply it. #### Setup Database with LoadBalancer -For the Single Instance Database, the default service is the `NodePort` service. You can enable the `LoadBalancer` service by using `kubectl patch` command. +For the Single Instance Database, the default service is the `NodePort` service. You can enable the `LoadBalancer` service by using the `kubectl patch` command. For example: @@ -644,7 +666,7 @@ $ kubectl --type=merge -p '{"spec":{"loadBalancer": true}}' patch singleinstance ``` ### Enabling TCPS Connections -You can enable TCPS connections in the database by setting the `enableTCPS` field to `true` in the [config/samples/sidb/singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml) file, and applying it. +You can enable TCPS connections in the database by setting the `enableTCPS` field to `true` in the [`config/samples/sidb/singleinstancedatabase.yaml`](../../config/samples/sidb/singleinstancedatabase.yaml) file, and applying it. Alternatively, you can use the following command: ```bash @@ -684,44 +706,45 @@ true cd export TNS_ADMIN=$(pwd) ``` - After this, connect using SQL\*Plus using the following sample commands: + After this, connect with SQL*Plus, using the following example commands: ```bash sqlplus sys@ORCL1 as sysdba ``` ### Specifying Custom Ports -As mentioned in the section [Setup Database with LoadBalancer](#setup-database-with-loadbalancer), there are two kubernetes services possible for the database: NodePort and LoadBalancer. You can specify which port to use with these services by editing the `listenerPort` and `tcpsListenerPort` fields of the [config/samples/sidb/singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml) file. +As mentioned in the section [Setup Database with LoadBalancer](#setup-database-with-loadbalancer), there are two kubernetes services possible for the database: NodePort and LoadBalancer. You can specify which port to use with these services by editing the `listenerPort` and `tcpsListenerPort` fields of the [`config/samples/sidb/singleinstancedatabase.yaml`](../../config/samples/sidb/singleinstancedatabase.yaml) file. `listenerPort` is intended for normal database connections. Similarly, `tcpsListenerPort` is intended for TCPS database connections. -If the `LoadBalancer` is enabled, the `listenerPort`, and `tcpsListenerPort` will be the opened ports on the Load Balancer for normal and TCPS database connections respectively. The default values of `listenerPort` and `tcpsListenerPort` are 1521 and 2484 respectively when the `LoadBalancer` is enabled. +If the `LoadBalancer` is enabled, then the `listenerPort`, and `tcpsListenerPort` will be the opened ports on the Load Balancer for normal and TCPS database connections respectively. When the `LoadBalancer` is enabled, the default values of `listenerPort` and `tcpsListenerPort` are 1521 and 2484. -In case of `NodePort` service, `listenerPort`, and `tcpsListenerPort` will be the opened ports on the Kubernetes nodes for for normal and TCPS database connections respectively. In this case, the allowed range for the `listenerPort`, and `tcpsListenerPort` is 30000-32767. +If the `NodePort` service is enabled, then the `listenerPort`, and `tcpsListenerPort` will be the opened ports on the Kubernetes nodes for for normal and TCPS database connections respectively. In this case, the allowed range for the `listenerPort`, and `tcpsListenerPort` is 30000-32767. **Note:** -- `listenerPort` and `tcpsListenerPort` can not have same values. -- `tcpsListenerPort` will come into effect only when TCPS connections are enabled (i.e. `enableTCPS` field is set in [config/samples/sidb/singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml) file). -- If TCPS connections are enabled, and `listenerPort` is commented/removed in the [config/samples/sidb/singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml) file, only TCPS endpoint will be exposed. -- If LoadBalancer is enabled, and either `listenerPort` or `tcpsListenerPort` is changed, then it takes some time to complete the work requests (drain existing backend sets and create new ones). In this time, the database connectivity is broken. Although, SingleInstanceDatabase and LoadBalancer remain in the healthy state, you can check the progress of the work requests by logging into the cloud provider's console and checking the corresponding LoadBalancer. +- `listenerPort` and `tcpsListenerPort` cannot have same values. +- `tcpsListenerPort` will come into effect only when TCPS connections are enabled (specifically, the `enableTCPS` field is set in [`config/samples/sidb/singleinstancedatabase.yaml`](../../config/samples/sidb/singleinstancedatabase.yaml) file). +- If TCPS connections are enabled, and `listenerPort` is commented or removed in the [`config/samples/sidb/singleinstancedatabase.yaml`](../../config/samples/sidb/singleinstancedatabase.yaml) file, then only the TCPS endpoint will be exposed. +- If LoadBalancer is enabled, and either `listenerPort` or `tcpsListenerPort` is changed, then it takes some time to complete the work requests (drain existing backend sets and create new ones). During this time, the database connectivity is broken, although `SingleInstanceDatabase` and `LoadBalancer` remain in a healthy state. To check the progress of the work requests, you can by log in to the Cloud provider's console and check the corresponding LoadBalancer. -### Setup Data Guard Configuration for a Single Instance Database (Preview status) +### Setup Data Guard Configuration for a Single Instance Database ### Create a Standby Database #### Prerequisites -- Before creating a Standby, ensure that ArchiveLog, FlashBack, and ForceLog on primary Single Instance Database(`.spec.primaryDatabaseRef`) are turned on. -- Standby database is not supported for TCPS enabled Primary databases. +- Before creating a Standby, ensure that ArchiveLog, FlashBack, and ForceLog on the primary Single Instance Database(`.spec.primaryDatabaseRef`) are turned on. +- Standby database is not supported for TCPS-enabled Primary databases. #### Template YAML -To create a standby database, edit and apply the sample yaml file [config/samples/sidb/singleinstancedatabase_standby.yaml](../../config/samples/sidb/singleinstancedatabase_standby.yaml). +To create a standby database, edit and apply the example YAML file [`config/samples/sidb/singleinstancedatabase_standby.yaml`](../../config/samples/sidb/singleinstancedatabase_standby.yaml). **Note:** -- The `adminPassword` field of the above [config/samples/sidb/singleinstancedatabase_standby.yaml](../../config/samples/sidb/singleinstancedatabase_standby.yaml) contains an admin password secret of the primary database ref for Standby Database creation. This secret will get deleted after the database pod becomes ready if the `keepSecret` attribute of `adminPassword` field is set to `false`. By default `keepSecret` is set to `true`. -- Mention referred primary database in `.spec.primaryDatabaseRef` in the yaml file. -- `.spec.createAs` field of the yaml file should be set to "standby". -- Database configuration like `Archivelog`, `FlashBack`, `ForceLog`, `TCPS connections` are not supported for standby database. +- The `adminPassword` field of the above [`config/samples/sidb/singleinstancedatabase_standby.yaml`](../../config/samples/sidb/singleinstancedatabase_standby.yaml) contains an admin password Secret of the primary database referred to for Standby Database creation. By default `keepSecret` is set to `true`, which means that the secret is saved. However, if you want to delete the Secret after the database pod becomes ready, then this Secret will be deleted if the `keepSecret` attribute of `adminPassword` field is set to `false`. . +- Specify the primary database with which the standby database is associateed in the `.spec.primaryDatabaseRef` yaml file. +- The `.spec.createAs` field of the yaml file should be set to "standby". +- Database configuration, such as `Archivelog`, `FlashBack`, `ForceLog`, `TCPS connections`, are not supported for standby database. #### List Standby Databases +To list the standby databases, use the `get singleinstancedatabase` command. For example: ```sh kubectl get singleinstancedatabase @@ -733,7 +756,7 @@ stdby-1 Enterprise Healthy PHYSICAL_STANDBY 19.3.0.0.0 10.25.0.27:3239 ``` ### Query Primary Database Reference -You can query the corresponding primary database for every standby database. +You can query the corresponding primary database for every standby database. For example: ```sh kubectl get singleinstancedatabase stdby-1 -o "jsonpath={.status.primaryDatabase}" @@ -742,7 +765,7 @@ sidb-19 #### Creation Status - Creating a new standby database instance takes a while. When the 'status' status returns the response "Healthy", the Database is open for connections. + Creating a new standby database instance takes a while. When the 'status' status returns the response "Healthy", the database is open for connections. For example: ```sh $ kubectl get singleinstancedatabase stdby-1 -o "jsonpath={.status.status}" @@ -754,23 +777,23 @@ $ kubectl get singleinstancedatabase stdby-1 -o "jsonpath={.status.status}" #### Template YAML -After creating standbys, setup a dataguard configuration with protection mode and switch over capability using the following sample yaml. -[config/samples/sidb/dataguardbroker.yaml](./../../config/samples/sidb/dataguardbroker.yaml) +After creating standbys, set up an Oracle Data Guard (Data Guard) configuration with protection mode, and switch over capability using the following example YAML: +[`config/samples/sidb/dataguardbroker.yaml`](./../../config/samples/sidb/dataguardbroker.yaml) #### Create DataGuardBroker Resource -Provision a new DataguardBroker custom resource for a single instance database(`.spec.primaryDatabaseRef`) by specifying appropriate values for the primary and standby databases in the example `.yaml` file, and running the following command: +To use the Data Guard broker, provision a new `dataguardbroker` custom resource for a single instance database(`.spec.primaryDatabaseRef`) by specifying the appropriate values for the primary and standby databases in the example `.yaml` file, and running the following command: ```sh $ kubectl create -f dataguardbroker.yaml dataguardbroker.database.oracle.com/dataguardbroker-sample created ``` -**Note:** The following attributes cannot be patched post DataguardBroker resource creation : `primaryDatabaseRef, protectionMode` +**Note:** The following attributes cannot be patched after you create the `dataguardbroker` resource: `primaryDatabaseRef, protectionMode` #### DataguardBroker List -To list the DataguardBroker resources, use the following command: +To list the Data Guard broker resources, use the following command: ```sh $ kubectl get dataguardbroker -o name @@ -780,6 +803,7 @@ To list the DataguardBroker resources, use the following command: ``` #### Quick Status +You can obtain a quick status of Data Guard broker by using the following command: ```sh $ kubectl get dataguardbroker dataguardbroker-sample @@ -790,6 +814,7 @@ To list the DataguardBroker resources, use the following command: ``` #### Detailed Status +To obtain more detailed Data Guard broker status, use this command: ```sh $ kubectl describe dataguardbroker dataguardbroker-sample @@ -827,8 +852,7 @@ To list the DataguardBroker resources, use the following command: Keep Secret: true Secret Key: oracle_pwd Secret Name: db-secret - Fast Start Fail Over: - Enable: true + Fast Start Failover: false Primary Database Ref: sidb-sample Protection Mode: MaxAvailability Set As Primary Database: @@ -838,6 +862,7 @@ To list the DataguardBroker resources, use the following command: Status: Cluster Connect String: dataguardbroker-sample.default:1521/DATAGUARD External Connect String: 10.0.25.85:31167/DATAGUARD + Fast Start Failover: false Primary Database: OR19E3 Standby Databases: OR19E3S1,OR19E3S2 Status: Healthy @@ -850,9 +875,9 @@ To list the DataguardBroker resources, use the following command: ### Perform a Switchover -Specify the approppriate SID (SID of one of `.spec.primaryDatabaseRef` , `.spec.standbyDatabaseRefs[]`) to be set primary in the `.spec.setAsPrimaryDatabase` of [dataguardbroker.yaml](./../../config/samples/sidb/dataguardbroker.yaml) and apply the yaml file. +Specify the approppriate database system identifier (SID) (the SID of one of `.spec.primaryDatabaseRef` , `.spec.standbyDatabaseRefs[]`) to be set primary in the `.spec.setAsPrimaryDatabase` of [`dataguardbroker.yaml`](./../../config/samples/sidb/dataguardbroker.yaml) and apply the yaml file. -The database will be set to primary. Ignored if the database is already primary. +When you apply the YAML file, the database you specify will be set to primary. However, if the database specified with the `apply` command is already the primary, then this command has no effect: ```sh $ kubectl apply -f dataguardbroker.yaml @@ -860,7 +885,7 @@ $ kubectl apply -f dataguardbroker.yaml dataguardbroker.database.oracle.com/dataguardbroker-sample apply ``` -Or use the patch command +You can also use the patch command ```sh $ kubectl --type=merge -p '{"spec":{"setAsPrimaryDatabase":"ORCLS1"}}' patch dataguardbroker dataguardbroker-sample @@ -868,40 +893,87 @@ $ kubectl --type=merge -p '{"spec":{"setAsPrimaryDatabase":"ORCLS1"}}' patch dat dataguardbroker.database.oracle.com/dataguardbroker-sample patched ``` -#### Static Primary Database Connection String +### Enable Fast-Start Failover + +Oracle Data Guard Fast-Start Failover (FSFO) monitors your Oracle Data Guard environments and initiates an automatic failover in the case of an outage. +To enable FSFO, ensure the primary database is in the primary role, set the attribute `.spec.fastStartFailover` to `true` in [`datguardbroker.yaml`](./../../config/samples/sidb/dataguardbroker.yaml), and then apply it. For example: + +```sh +$ kubectl apply -f dataguardbroker.yaml + + dataguardbroker.database.oracle.com/dataguardbroker-sample configured +``` + +You can also use the patch command: + +```sh +$ kubectl --type=merge -p '{"spec":{"fastStartFailover": true}}' patch dataguardbroker dataguardbroker-sample + + dataguardbroker.database.oracle.com/dataguardbroker-sample patched +``` + +Applying this results in the creation of a pod running the Observer. The Observer is a component of the DGMGRL interface, which monitors the availability of the primary database. + +**Note:** When the attribute `fastStartFailover` is `true`, then performing a switchover by specifying `setAsPrimaryDatabase` is not allowed. + +### Convert Standby to Snapshot Standby + +A snapshot standby is a fully updatable standby database that can be used development and testing. It receives and archives, but does not apply redo data from a primary database. The redo data received from the primary database is applied after a snapshot standby database is converted back into a physical standby database, and after discarding all local updates to the snapshot standby database. + +To convert a standby database to a snapshot standby, Ensure Fast-Start Failover is disabled, and tshen set the attribute `.spec.convertToSnapshotStandby` to `true` in [`singleinstancedatabase.yaml`](./../../config/samples/sidb/singleinstancedatabase.yaml) before applying it. For example: + +```sh +$ kubectl apply -f singleinstancedatabase.yaml + + singleinstancedatabase.database.oracle.com/sidb-sample configured +``` - External and internal (running in Kubernetes pods) clients can connect to the primary database using `.status.connectString` and `.status.clusterConnectString` of the DataguardBroker resource respectively. These connection strings are fixed for the DataguardBroker resource and will not change on switchover. They can be queried using the following command +You can also use the patch command: + +```sh +$ kubectl --type=merge -p '{"spec":{"convertToSnapshotStandby":true}}' patch singleinstancedatabase sidb-sample + + singleinstancedatabase.database.oracle.com/sidb-sample patched +``` + +### Static Data Guard Connect String + + External and internal (running in pods) applications can always connect to the database in the primary role by using `.status.externalConnectString` and `.status.clusterConnectString` of the Data Guard broker resource respectively. These connect strings are fixed for the Data Guard broker resource, and will not change on switchover or failover. The external connect string can be obtained using the following command: ```sh $ kubectl get dataguardbroker dataguardbroker-sample -o "jsonpath={.status.externalConnectString}" 10.0.25.87:1521/DATAGUARD ``` - The above connection string will always automatically route to the Primary database not requiring clients to change the connection string after switchover + This connect string will always automatically route to the database in the primary role. Client applications can be totally agnostic of the databases in the Oracle Data Guard configuration. Their number or host/IP details are not needed in the connect string. -### Patch Primary and Standby databases in Data Guard configuration +### Patch Primary and Standby databases Databases (both primary and standby) running in you cluster and managed by the Oracle Database operator can be patched between release updates of the same major release. -To patch an existing database, edit and apply the **[config/samples/sidb/singleinstancedatabase_patch.yaml](../../config/samples/sidb/singleinstancedatabase_patch.yaml)** file of the database resource/object either by specifying a new release update for image attributes, or by running the following command: +To patch an existing database, edit and apply the **[`config/samples/sidb/singleinstancedatabase_patch.yaml`](../../config/samples/sidb/singleinstancedatabase_patch.yaml)** file of the database resource/object either by specifying a new release update for image attributes, or by running the following command: ```sh kubectl --type=merge -p '{"spec":{"image":{"pullFrom":"patched-image:tag","pullSecrets":"pull-secret"}}}' patch singleinstancedatabase ``` -Follow these steps for patching databases configured with the dataguard broker: -1. First patch all the standby databases by replacing the image with the new release update image -2. Perform switch over of the primary to one of the standby databases -3. Now patch the original primary database (currently standby after #2) - After #3 the software for primary and standby databases is at the same release update -4. Now bounce the current primary database by updating the replica count to 0 and then 1 - #4 will trigger a datapatch execution resulting in patching of the datafiles -5. Finally perform switch over of the current primary back to the original primary (current standby) +Follow these steps for patching databases configured with the Data Guard broker: +1. Ensure Fast-Start Failover is disabled by running the following command +```sh + kubectl patch dataguardbroker dataguardbroker-sample -p '{"spec":{"fastStartFailover": false}}' --type=merge +``` +2. Patch all the standby databases by replacing the image with the new release update image. +3. Perform switchover of the primary to one of the standby databases. +4. Patch the original primary database (currently standby after #2) + After step 3, the software for primary and standby databases is at the same release update +5. Bounce the current primary database by updating the replica count to 0 and then 1 + Step 5 will trigger a datapatch execution, which results in patching the datafiles +6. Finally, perform switch over of the current primary back to the original primary (current standby) ### Delete the Data Guard Configuration -To delete a standby or primary database configured for Data Guard, delete the dataguardbroker resource first followed by the standby databases and finally the primary database +To delete a standby or primary database configured for Oracle Data Guard, delete the `dataguardbroker` resource. After that is done, delete the standby databases, and then finally the primary database. #### Delete DataguardBroker Resource ```sh @@ -910,8 +982,7 @@ $ kubectl delete dataguardbroker dgbroker-sample dataguardbroker.database.oracle.com/dgbroker-sample deleted ``` -**Note:** If a switch over to standby was performed, make sure to switch back to the original primary database before deleting the dataguard broker resource - +**Note:** If a switchover to standby was performed, then ensure that you switch back to the original primary database before deleting the Data Guard broker resource. For example: #### Delete Standby Database ```sh $ kubectl delete singleinstancedatabase stdby-1 @@ -921,15 +992,15 @@ $ kubectl delete singleinstancedatabase stdby-1 ### Execute Custom Scripts -Custom scripts (sql and/or shell scripts) can be executed after the initial database setup and/or after each startup of the database. SQL scripts will be executed as sysdba, shell scripts will be executed as the current user. To ensure proper order it is recommended to prefix your scripts with a number. For example `01_users.sql`, `02_permissions.sql`, etc. Place all such scripts in setup and startup folders created in a persistent volume to execute them post setup and post startup respectively. +You can set up custom scripts (SQL, shell scripts, or both) to run after the initial database setup, and to have scripts run after each startup of the database. SQL scripts will be executed as `sysdba`, and shell scripts will be executed as the current user. To ensure proper order, Oracle recommends that you prefix your scripts with a number. For example: `01_users.sql`, `02_permissions.sql`, and son on. To ensure that these scripts are available to run after setup or after each database startup, place all such scripts in setup and startup folders created in a persistent volume. -Create a persistent volume using [static provisioning](#static-persistence) and then specify the name of this volume with the `<.spec.persistence.scriptsVolumeName>` field which corresponds to the `scriptsVolumeName` field of the persistence section in the **[singleinstancedatabase.yaml](../../config/samples/sidb/singleinstancedatabase.yaml)**. +Create a persistent volume by using [static provisioning](#static-persistence) and then specify the name of this volume with the `<.spec.persistence.scriptsVolumeName>` field which corresponds to the `scriptsVolumeName` field of the persistence section in the **[`singleinstancedatabase.yaml`](../../config/samples/sidb/singleinstancedatabase.yaml)**. -**Note:** Executing custom scripts requires read and list access for persistent volumes as mentioned in [prerequisites](#prerequisites) +**Note:** Running custom scripts requires read and list access for persistent volumes, as mentioned in [prerequisites](#prerequisites) ## OracleRestDataService Resource -The Oracle Database Operator creates the `OracleRestDataService` as a custom resource. We will refer `OracleRestDataService` as ORDS from now onwards. Creating ORDS as a custom resource enables the RESTful API access to the Oracle Database in K8s and enables it to be managed as a native Kubernetes object. +The Oracle Database Operator creates the `OracleRestDataService` as a custom resource. In this documeent, we will refer to `OracleRestDataService` as ORDS. Creating ORDS as a custom resource enables the RESTful API access to the Oracle Database in K8s, and enables it to be managed as a native Kubernetes object. ### Resource Details @@ -950,7 +1021,7 @@ To obtain a quick status check of the ORDS service, use the following command: $ kubectl get oraclerestdataservice ords-sample NAME STATUS DATABASE DATABASE API URL DATABASE ACTIONS URL APEX URL -ords-sample Healthy sidb-sample https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/ https://10.0.25.54:8443/ords/sql-developer https://10.0.25.54:8443/ords/ORCLPDB1/apex +ords-sample Healthy sidb-sample http://10.0.25.54:8181/ords/schema1/_/db-api/stable/ http://10.0.25.54:8181/ords/sql-developer http://10.0.25.54:8181/ords/apex ``` @@ -969,10 +1040,10 @@ $ kubectl describe oraclerestdataservice ords-sample Metadata: ... Spec: ... Status: - Cluster Db API URL: https://ords21c-1.default:8443/ords/ORCLPDB1/_/db-api/stable/ - Database Actions URL: https://10.0.25.54:8443/ords/sql-developer - Database API URL: https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/ - Apex URL: https://10.0.25.54:8443/ords/ORCLPDB1/apex + Cluster Db API URL: http://ords21c-1.default:8181/ords/schema1/_/db-api/stable/ + Database Actions URL: http://10.0.25.54:8181/ords/sql-developer + Database API URL: http://10.0.25.54:8181/ords/schema1/_/db-api/stable/ + Apex URL: http://10.0.25.54:8181/ords/apex Database Ref: sidb21c-1 Image: Pull From: ... @@ -994,15 +1065,14 @@ The template `.yaml` file for Oracle Rest Data Services (`OracleRestDataService` **Note:** - The `adminPassword` and `ordsPassword` fields in the `oraclerestdataservice.yaml` file contains secrets for authenticating the Single Instance Database and the ORDS user with the following roles: `SQL Administrator, System Administrator, SQL Developer, oracle.dbtools.autorest.any.schema`. -- To build the ORDS image, use the following instructions: [Building Oracle REST Data Services Install Images](https://github.com/oracle/docker-images/tree/main/OracleRestDataServices#building-oracle-rest-data-services-install-images). -- By default, ORDS uses self-signed certificates. To use certificates from the Certificate Authority, the ORDS image needs to be rebuilt after specifying the values of `ssl.cert` and `ssl.cert.key` in the [standalone.properties](https://github.com/oracle/docker-images/blob/main/OracleRestDataServices/dockerfiles/standalone.properties.tmpl) file. After you rebuild the ORDS image, use the rebuilt image in the **[config/samples/sidb/oraclerestdataservice.yaml](../../config/samples/sidb/oraclerestdataservice.yaml)** file. -- If you want to install ORDS in a [prebuilt database](#provision-a-pre-built-database), make sure to attach the **database persistence** by uncommenting the `persistence` section in the **[config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml](../../config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml)** file, while provisioning the prebuilt database. + +- If you want to install ORDS in a [prebuilt database](#provision-a-pre-built-database), then ensure that you attach the **database persistence** by uncommenting the `persistence` section in the **[`config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml`](../../config/samples/sidb/singleinstancedatabase_prebuiltdb.yaml)** file, while provisioning the prebuilt database. ### REST Enable a Database #### Provision ORDS -To quickly provision a new ORDS instance, use the sample **[config/samples/sidb/oraclerestdataservice_create.yaml](../../config/samples/sidb/oraclerestdataservice_create.yaml)** file. For example: +To quickly provision a new ORDS instance, use the example **[`config/samples/sidb/oraclerestdataservice_create.yaml`](../../config/samples/sidb/oraclerestdataservice_create.yaml)** file. For example: ```sh $ kubectl apply -f oraclerestdataservice_create.yaml @@ -1012,18 +1082,17 @@ $ kubectl apply -f oraclerestdataservice_create.yaml After this command completes, ORDS is installed in the container database (CDB) of the Single Instance Database. ##### Note: -You are required to specify the ORDS secret in the [oraclerestdataservice_create.yaml](../../config/samples/sidb/oraclerestdataservice_create.yaml) file. The default value mentioned in the `adminPassword.secretName` field is `ords-secret`. You can create this secret manually by using the following command: +You are required to specify the ORDS Secret in the [`oraclerestdataservice_create.yaml`](../../config/samples/sidb/oraclerestdataservice_create.yaml) file. The default value mentioned in the `adminPassword.secretName` field is `ords-secret`. You can create this Secret manually by using the following command: ```bash kubectl create secret generic ords-secret --from-literal=oracle_pwd= ``` -Alternatively, you can create this secret and the APEX secret by filling the passwords in the **[oraclerestdataservice_secrets.yaml](../../config/samples/sidb/oraclerestdataservice_secrets.yaml)** file and applying it using the command below: +Alternatively, you can create this Secret by filling the passwords in the **[`oraclerestdataservice_secrets.yaml`](../../config/samples/sidb/oraclerestdataservice_secrets.yaml)** file and applying it using the following command: ```bash kubectl apply -f singleinstancedatabase_secrets.yaml ``` -The APEX secret created above, will be used while [installing APEX](#apex-installation). #### Creation Status @@ -1043,7 +1112,7 @@ Clients can access the REST Endpoints using `.status.databaseApiUrl` as shown in ```sh $ kubectl get oraclerestdataservice/ords-sample -o "jsonpath={.status.databaseApiUrl}" - https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/ + http://10.0.25.54:8181/ords/schema1/_/db-api/stable/ ``` All the REST Endpoints can be found in [_REST APIs for Oracle Database_](https://docs.oracle.com/en/database/oracle/oracle-database/21/dbrst/rest-endpoints.html). @@ -1052,37 +1121,51 @@ There are two basic approaches for authentication to the REST Endpoints. Certain #### Database API -To call certain REST endpoints, you must use the ORDS_PUBLIC_USER with role `SQL Administrator`, and `.spec.ordsPassword` credentials. +To call certain REST endpoints, you must use the Schema User, which is REST-Enabled with role `SQL Administrator`, and `.spec.ordsPassword` credentials. -The ORDS user also has the following additional roles: `System Administrator, SQL Developer, oracle.dbtools.autorest.any.schema`. +The Schema user also has the following additional roles: `System Administrator, SQL Developer`. -Use this ORDS user to authenticate the following: +Use this Schema user to authenticate the following: * Database APIs * Any Protected AutoRest Enabled Object APIs * Database Actions of any REST Enabled Schema ##### Examples -Some examples for the Database API usage are as follows: +Some examples for the Database API usage for REST-Enabled schema1 are as follows: - **Get all Database Components** ```sh - curl -s -k -X GET -u 'ORDS_PUBLIC_USER:<.spec.ordsPassword>' https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/database/components/ | python -m json.tool + curl -s -k -X GET -u '<.spec.restEnableSchemas[].schemaName>:<.spec.ordsPassword>' http://10.0.25.54:8181/ords/schema1/_/db-api/stable/database/components/ | python -m json.tool ``` - **Get all Database Users** ```sh - curl -s -k -X GET -u 'ORDS_PUBLIC_USER:<.spec.ordsPassword>' https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/database/security/users/ | python -m json.tool + curl -s -k -X GET -u '<.spec.restEnableSchemas[].schemaName>:<.spec.ordsPassword>' http://10.0.25.54:8181/ords/schema1/_/db-api/stable/database/security/users/ | python -m json.tool ``` - **Get all Tablespaces** ```sh - curl -s -k -X GET -u 'ORDS_PUBLIC_USER:<.spec.ordsPassword>' https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/database/storage/tablespaces/ | python -m json.tool + curl -s -k -X GET -u '<.spec.restEnableSchemas[].schemaName>:<.spec.ordsPassword>' http://10.0.25.54:8181/ords/schema1/_/db-api/stable/database/storage/tablespaces/ | python -m json.tool ``` - **Get all Database Parameters** ```sh - curl -s -k -X GET -u 'ORDS_PUBLIC_USER:<.spec.ordsPassword>' https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/database/parameters/ | python -m json.tool + curl -s -k -X GET -u '<.spec.restEnableSchemas[].schemaName>:<.spec.ordsPassword>' http://10.0.25.54:8181/ords/schema1/_/db-api/stable/database/parameters/ | python -m json.tool ``` - **Get all Feature Usage Statistics** ```sh - curl -s -k -X GET -u 'ORDS_PUBLIC_USER:<.spec.ordsPassword>' https://10.0.25.54:8443/ords/ORCLPDB1/_/db-api/stable/database/feature_usage/ | python -m json.tool + curl -s -k -X GET -u '<.spec.restEnableSchemas[].schemaName>:<.spec.ordsPassword>' http://10.0.25.54:8181/ords/schema1/_/db-api/stable/database/feature_usage/ | python -m json.tool ``` + +#### MongoDB API + +To enable the Database API for MongoDB, set `.spec.mongoDbApi` to `true`. When this is done, MongoDB applications are be able to connect to Oracle Database using the MongoDB API Access URL. For example: + +```sh +$ kubectl get oraclerestdataservice/ords-sample -o "jsonpath={.status.mongoDbApiAccessUrl}" + + mongodb://[{user}:{password}@]10.0.25.54:27017/{user}?authMechanism=PLAIN&authSource=$external&ssl=true&retryWrites=false&loadBalanced=true +``` + +* Change [{user}:{password}@] to database username and password. Retain the @ symbol but remove all the brackets. +* Change the {user} later in the URL to database username as well. + #### Advanced Usages ##### Oracle Data Pump @@ -1091,10 +1174,9 @@ The Oracle REST Data Services (ORDS) database API enables you to create Oracle D REST APIs for Oracle Data Pump Jobs can be found at [https://docs.oracle.com/en/database/oracle/oracle-database/21/dbrst/op-database-datapump-jobs-post.html](https://docs.oracle.com/en/database/oracle/oracle-database/21/dbrst/op-database-datapump-jobs-post.html). ##### REST Enabled SQL -The REST Enable SQL functionality is available to all the schemas specified in the `.spec.restEnableSchemas` attribute of the sample yaml. -Only these schemas will have access SQL Developer Web Console specified by the Database Actions URL. +The REST-Enabled SQL functionality is available to all of the schemas specified in the `.spec.restEnableSchemas` attribute of the example yaml in the sample folder. Only these schemas will have access SQL Developer Web Console specified by the Database Actions URL. -The REST Enabled SQL functionality enables REST calls to send DML, DDL and scripts to any REST enabled schema by exposing the same SQL engine used in SQL Developer and Oracle SQLcl (SQL Developer Command Line). +The REST-Enabled SQL functionality enables REST calls to send DML, DDL and scripts to any REST-Enabled schema by exposing the same SQL engine used in SQL Developer and Oracle SQLcl (SQL Developer Command Line). For example: @@ -1119,7 +1201,7 @@ Create a file called "/tmp/table.sql" with the following contents. Run the following API to run the script created in the previous example: ```sh - curl -s -k -X "POST" "https://10.0.25.54:8443/ords/<.spec.restEnableSchemas[].pdbName>/<.spec.restEnableSchemas[].urlMapping>/_/sql" \ + curl -s -k -X "POST" "http://10.0.25.54:8181/ords/<.spec.restEnableSchemas[].urlMapping>/_/sql" \ -H "Content-Type: application/sql" \ -u '<.spec.restEnableSchemas[].schemaName>:<.spec.ordsPassword>' \ -d @/tmp/table.sql @@ -1130,7 +1212,7 @@ Run the following API to run the script created in the previous example: Fetch all entries from 'DEPT' table by calling the following API ```sh - curl -s -k -X "POST" "https://10.0.25.54:8443/ords/<.spec.restEnableSchemas[].pdbName>/<.spec.restEnableSchemas[].urlMapping>/_/sql" \ + curl -s -k -X "POST" "http://10.0.25.54:8181/ords/<.spec.restEnableSchemas[].urlMapping>/_/sql" \ -H "Content-Type: application/sql" \ -u '<.spec.restEnableSchemas[].schemaName>:<.spec.ordsPassword>' \ -d $'select * from dept;' | python -m json.tool @@ -1152,16 +1234,12 @@ Database Actions can be accessed with a browser by using `.status.databaseAction ```sh $ kubectl get oraclerestdataservice/ords-sample -o "jsonpath={.status.databaseActionsUrl}" - https://10.0.25.54:8443/ords/sql-developer + http://10.0.25.54:8181/ords/sql-developer ``` -To access Database Actions, sign in by using the following code as a database user whose schema has been REST-enabled: - -* First Page: \ -PDB Name: `.spec.restEnableSchemas[].pdbName` \ -Username: `.spec.restEnableSchemas[].urlMapping` +To access Database Actions, sign in by using the following code as a database user whose schema has been REST-Enabled: -* Second Page: \ +* Login Page: \ Username: `.spec.restEnableSchemas[].schemaName` \ Password: `.spec.ordsPassword` @@ -1175,21 +1253,9 @@ Oracle APEX is a low-code development platform that enables developers to build Using APEX, developers can quickly develop and deploy compelling apps that solve real problems and provide immediate value. Developers won't need to be an expert in a vast array of technologies to deliver sophisticated solutions. Focus on solving the problem and let APEX take care of the rest. -The `OraOperator` facilitates installation of APEX in the database and also configures ORDS for it. The following section will explain installing APEX with configured ORDS: - -* For quick provisioning, use the sample **[config/samples/sidb/oraclerestdataservice_apex.yaml](../../config/samples/sidb/oraclerestdataservice_apex.yaml)** file. For example: - - kubectl apply -f oraclerestdataservice_apex.yaml - -* The APEX Password is used as a common password for `APEX_PUBLIC_USER, APEX_REST_PUBLIC_USER, APEX_LISTENER` and Apex administrator (username: `ADMIN`) mapped to secretKey. You can create APEX secret using the following command: - - ```bash - kubectl create secret generic apex-secret --from-literal=oracle_pwd= - ``` - Please refer [this](#note) section for APEX secret creation using the **[oraclerestdataservice_secrets.yaml](../../config/samples/sidb/oraclerestdataservice_secrets.yaml)** file. - -* The status of ORDS turns to `Updating` during APEX configuration, and changes to `Healthy` after successful configuration. You can also check status by using the following command: +The `OraOperator` facilitates installation of APEX in the database and also configures ORDS for it. +* Status of APEX configuration can be checked using the following command: ```sh $ kubectl get oraclerestdataservice ords-sample -o "jsonpath={.status.apexConfigured}" @@ -1197,28 +1263,27 @@ The `OraOperator` facilitates installation of APEX in the database and also conf [true] ``` -* If you configure APEX after ORDS is installed, then ORDS pods will be deleted and recreated. - Application Express can be accessed via browser using `.status.apexUrl` in the following command. ```sh $ kubectl get oraclerestdataservice/ords-sample -o "jsonpath={.status.apexUrl}" - https://10.0.25.54:8443/ords/ORCLPDB1/apex + http://10.0.25.54:8181/ords/apex ``` -Sign in to Administration services using -workspace: `INTERNAL` -username: `ADMIN` -password: `.spec.apexPassword` +Sign in to Administration services using \ +workspace: `INTERNAL` \ +username: `ADMIN` \ +password: `Welcome_1` ![application-express-admin-home](/images/sidb/application-express-admin-home.png) **Note:** +- Oracle strongly recommends that you change the default APEX admin password. - By default, the full development environment is initialized in APEX. After deployment, you can change it manually to the runtime environment. To change environments, run the script `apxdevrm.sql` after connecting to the primary database from the ORDS pod as the `SYS` user with `SYSDBA` privilege. For detailed instructions, see: [Converting a Full Development Environment to a Runtime Environment](https://docs.oracle.com/en/database/oracle/application-express/21.2/htmig/converting-between-runtime-and-full-development-environments.html#GUID-B0621B40-3441-44ED-9D86-29B058E26BE9). ### Delete ORDS -- To delete ORDS run the following command: +- To delete ORDS, run the following command: kubectl delete oraclerestdataservice ords-sample diff --git a/go.mod b/go.mod index 1f30279f..863f2e99 100644 --- a/go.mod +++ b/go.mod @@ -1,56 +1,56 @@ module github.com/oracle/oracle-database-operator -go 1.21 +go 1.23.3 require ( - github.com/go-logr/logr v1.3.0 - github.com/onsi/ginkgo/v2 v2.13.0 - github.com/onsi/gomega v1.29.0 - github.com/oracle/oci-go-sdk/v65 v65.49.3 - github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring v0.62.0 + github.com/go-logr/logr v1.4.2 + github.com/onsi/ginkgo/v2 v2.20.2 + github.com/onsi/gomega v1.34.2 + github.com/oracle/oci-go-sdk/v65 v65.77.1 + github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring v0.78.2 go.uber.org/zap v1.26.0 - golang.org/x/text v0.14.0 + golang.org/x/text v0.19.0 gopkg.in/yaml.v3 v3.0.1 - k8s.io/api v0.29.2 - k8s.io/apimachinery v0.29.2 - k8s.io/cli-runtime v0.29.2 - k8s.io/client-go v0.29.2 - k8s.io/kubectl v0.29.2 - sigs.k8s.io/controller-runtime v0.16.2 - sigs.k8s.io/yaml v1.3.0 + k8s.io/api v0.31.3 + k8s.io/apimachinery v0.31.3 + k8s.io/cli-runtime v0.31.3 + k8s.io/client-go v0.31.3 + k8s.io/kubectl v0.31.3 + sigs.k8s.io/controller-runtime v0.19.3 + sigs.k8s.io/yaml v1.4.0 ) require ( github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 // indirect github.com/MakeNowJust/heredoc v1.0.0 // indirect github.com/beorn7/perks v1.0.1 // indirect - github.com/cespare/xxhash/v2 v2.2.0 // indirect + github.com/blang/semver/v4 v4.0.0 // indirect + github.com/cespare/xxhash/v2 v2.3.0 // indirect github.com/chai2010/gettext-go v1.0.2 // indirect - github.com/davecgh/go-spew v1.1.1 // indirect + github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect github.com/emicklei/go-restful/v3 v3.11.0 // indirect github.com/evanphx/json-patch v5.6.0+incompatible // indirect - github.com/evanphx/json-patch/v5 v5.6.0 // indirect + github.com/evanphx/json-patch/v5 v5.9.0 // indirect github.com/exponent-io/jsonpath v0.0.0-20151013193312-d6023ce2651d // indirect github.com/fatih/camelcase v1.0.0 // indirect - github.com/fsnotify/fsnotify v1.6.0 // indirect - github.com/fvbommel/sortorder v1.1.0 // indirect + github.com/fxamacker/cbor/v2 v2.7.0 // indirect github.com/go-errors/errors v1.4.2 // indirect - github.com/go-logr/zapr v1.2.4 // indirect + github.com/go-logr/zapr v1.3.0 // indirect github.com/go-openapi/jsonpointer v0.19.6 // indirect github.com/go-openapi/jsonreference v0.20.2 // indirect - github.com/go-openapi/swag v0.22.3 // indirect - github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572 // indirect + github.com/go-openapi/swag v0.22.4 // indirect + github.com/go-task/slim-sprig/v3 v3.0.0 // indirect github.com/gofrs/flock v0.8.1 // indirect github.com/gogo/protobuf v1.3.2 // indirect github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect - github.com/golang/protobuf v1.5.3 // indirect + github.com/golang/protobuf v1.5.4 // indirect github.com/google/btree v1.0.1 // indirect github.com/google/gnostic-models v0.6.8 // indirect github.com/google/go-cmp v0.6.0 // indirect github.com/google/gofuzz v1.2.0 // indirect - github.com/google/pprof v0.0.0-20210720184732-4bb14d4b1be1 // indirect + github.com/google/pprof v0.0.0-20240827171923-fa2c70bbbfe5 // indirect github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 // indirect - github.com/google/uuid v1.3.0 // indirect + github.com/google/uuid v1.6.0 // indirect github.com/gorilla/websocket v1.5.0 // indirect github.com/gregjones/httpcache v0.0.0-20180305231024-9cad4c3443a7 // indirect github.com/imdario/mergo v0.3.6 // indirect @@ -59,10 +59,9 @@ require ( github.com/json-iterator/go v1.1.12 // indirect github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de // indirect github.com/mailru/easyjson v0.7.7 // indirect - github.com/matttproud/golang_protobuf_extensions v1.0.4 // indirect github.com/mitchellh/go-wordwrap v1.0.1 // indirect - github.com/moby/spdystream v0.2.0 // indirect - github.com/moby/term v0.0.0-20221205130635-1aeaba878587 // indirect + github.com/moby/spdystream v0.4.0 // indirect + github.com/moby/term v0.5.0 // indirect github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect github.com/modern-go/reflect2 v1.0.2 // indirect github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00 // indirect @@ -70,37 +69,38 @@ require ( github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f // indirect github.com/peterbourgon/diskv v2.0.1+incompatible // indirect github.com/pkg/errors v0.9.1 // indirect - github.com/prometheus/client_golang v1.16.0 // indirect - github.com/prometheus/client_model v0.4.0 // indirect - github.com/prometheus/common v0.44.0 // indirect - github.com/prometheus/procfs v0.10.1 // indirect + github.com/prometheus/client_golang v1.19.1 // indirect + github.com/prometheus/client_model v0.6.1 // indirect + github.com/prometheus/common v0.55.0 // indirect + github.com/prometheus/procfs v0.15.1 // indirect github.com/russross/blackfriday/v2 v2.1.0 // indirect github.com/sony/gobreaker v0.5.0 // indirect - github.com/spf13/cobra v1.7.0 // indirect + github.com/spf13/cobra v1.8.1 // indirect github.com/spf13/pflag v1.0.5 // indirect + github.com/x448/float16 v0.8.4 // indirect github.com/xlab/treeprint v1.2.0 // indirect go.starlark.net v0.0.0-20230525235612-a134d8f9ddca // indirect go.uber.org/multierr v1.11.0 // indirect - golang.org/x/exp v0.0.0-20220722155223-a9213eeb770e // indirect - golang.org/x/net v0.19.0 // indirect - golang.org/x/oauth2 v0.10.0 // indirect - golang.org/x/sync v0.5.0 // indirect - golang.org/x/sys v0.15.0 // indirect - golang.org/x/term v0.15.0 // indirect + golang.org/x/exp v0.0.0-20240719175910-8a7402abbf56 // indirect + golang.org/x/net v0.29.0 // indirect + golang.org/x/oauth2 v0.21.0 // indirect + golang.org/x/sync v0.8.0 // indirect + golang.org/x/sys v0.25.0 // indirect + golang.org/x/term v0.24.0 // indirect golang.org/x/time v0.3.0 // indirect - golang.org/x/tools v0.16.1 // indirect + golang.org/x/tools v0.24.0 // indirect gomodules.xyz/jsonpatch/v2 v2.4.0 // indirect - google.golang.org/appengine v1.6.7 // indirect - google.golang.org/protobuf v1.31.0 // indirect + google.golang.org/protobuf v1.34.2 // indirect + gopkg.in/evanphx/json-patch.v4 v4.12.0 // indirect gopkg.in/inf.v0 v0.9.1 // indirect gopkg.in/yaml.v2 v2.4.0 // indirect - k8s.io/apiextensions-apiserver v0.28.0 // indirect - k8s.io/component-base v0.29.2 // indirect - k8s.io/klog/v2 v2.110.1 // indirect - k8s.io/kube-openapi v0.0.0-20231010175941-2dd684a91f00 // indirect - k8s.io/utils v0.0.0-20230726121419-3b25d923346b // indirect + k8s.io/apiextensions-apiserver v0.31.2 // indirect + k8s.io/component-base v0.31.3 // indirect + k8s.io/klog/v2 v2.130.1 // indirect + k8s.io/kube-openapi v0.0.0-20240228011516-70dd3763d340 // indirect + k8s.io/utils v0.0.0-20240902221715-702e33fdd3c3 // indirect sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd // indirect - sigs.k8s.io/kustomize/api v0.13.5-0.20230601165947-6ce0bf390ce3 // indirect - sigs.k8s.io/kustomize/kyaml v0.14.3-0.20230601165947-6ce0bf390ce3 // indirect + sigs.k8s.io/kustomize/api v0.17.2 // indirect + sigs.k8s.io/kustomize/kyaml v0.17.1 // indirect sigs.k8s.io/structured-merge-diff/v4 v4.4.1 // indirect ) diff --git a/go.sum b/go.sum index 08185f3e..d23debb8 100644 --- a/go.sum +++ b/go.sum @@ -6,56 +6,56 @@ github.com/MakeNowJust/heredoc v1.0.0 h1:cXCdzVdstXyiTqTvfqk9SDHpKNjxuom+DOlyEeQ github.com/MakeNowJust/heredoc v1.0.0/go.mod h1:mG5amYoWBHf8vpLOuehzbGGw0EHxpZZ6lCpQ4fNJ8LE= github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 h1:0CwZNZbxp69SHPdPJAN/hZIm0C4OItdklCFmMRWYpio= github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5/go.mod h1:wHh0iHkYZB8zMSxRWpUBQtwG5a7fFgvEO+odwuTv2gs= -github.com/benbjohnson/clock v1.1.0/go.mod h1:J11/hYXuz8f4ySSvYwY0FKfm+ezbsZBKZxNJlLklBHA= github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= +github.com/blang/semver/v4 v4.0.0 h1:1PFHFE6yCCTv8C1TeyNNarDzntLi7wMI5i/pzqYIsAM= +github.com/blang/semver/v4 v4.0.0/go.mod h1:IbckMUScFkM3pff0VJDNKRiT6TG/YpiHIM2yvyW5YoQ= github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU= -github.com/cespare/xxhash/v2 v2.2.0 h1:DC2CZ1Ep5Y4k3ZQ899DldepgrayRUGE6BBZ/cd9Cj44= -github.com/cespare/xxhash/v2 v2.2.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= +github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs= +github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= github.com/chai2010/gettext-go v1.0.2 h1:1Lwwip6Q2QGsAdl/ZKPCwTe9fe0CjlUbqj5bFNSjIRk= github.com/chai2010/gettext-go v1.0.2/go.mod h1:y+wnP2cHYaVj19NZhYKAwEMH2CI1gNHeQQ+5AjwawxA= github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI= github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI= github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU= github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw= -github.com/cpuguy83/go-md2man/v2 v2.0.2/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o= +github.com/cpuguy83/go-md2man/v2 v2.0.4/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o= github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= github.com/creack/pty v1.1.18 h1:n56/Zwd5o6whRC5PMGretI4IdRLlmBXYNjScPaBgsbY= github.com/creack/pty v1.1.18/go.mod h1:MOBLtS5ELjhRRrroQr9kyvTxUAFNvYEK993ew/Vr4O4= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= -github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM= +github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/emicklei/go-restful/v3 v3.11.0 h1:rAQeMHw1c7zTmncogyy8VvRZwtkmkZ4FxERmMY4rD+g= github.com/emicklei/go-restful/v3 v3.11.0/go.mod h1:6n3XBCmQQb25CM2LCACGz8ukIrRry+4bhvbpWn3mrbc= github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4= github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c= github.com/evanphx/json-patch v5.6.0+incompatible h1:jBYDEEiFBPxA0v50tFdvOzQQTCvpL6mnFh5mB2/l16U= github.com/evanphx/json-patch v5.6.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk= -github.com/evanphx/json-patch/v5 v5.6.0 h1:b91NhWfaz02IuVxO9faSllyAtNXHMPkC5J8sJCLunww= -github.com/evanphx/json-patch/v5 v5.6.0/go.mod h1:G79N1coSVB93tBe7j6PhzjmR3/2VvlbKOFpnXhI9Bw4= +github.com/evanphx/json-patch/v5 v5.9.0 h1:kcBlZQbplgElYIlo/n1hJbls2z/1awpXxpRi0/FOJfg= +github.com/evanphx/json-patch/v5 v5.9.0/go.mod h1:VNkHZ/282BpEyt/tObQO8s5CMPmYYq14uClGH4abBuQ= github.com/exponent-io/jsonpath v0.0.0-20151013193312-d6023ce2651d h1:105gxyaGwCFad8crR9dcMQWvV9Hvulu6hwUh4tWPJnM= github.com/exponent-io/jsonpath v0.0.0-20151013193312-d6023ce2651d/go.mod h1:ZZMPRZwes7CROmyNKgQzC3XPs6L/G2EJLHddWejkmf4= github.com/fatih/camelcase v1.0.0 h1:hxNvNX/xYBp0ovncs8WyWZrOrpBNub/JfaMvbURyft8= github.com/fatih/camelcase v1.0.0/go.mod h1:yN2Sb0lFhZJUdVvtELVWefmrXpuZESvPmqwoZc+/fpc= -github.com/fsnotify/fsnotify v1.6.0 h1:n+5WquG0fcWoWp6xPWfHdbskMCQaFnG6PfBrh1Ky4HY= -github.com/fsnotify/fsnotify v1.6.0/go.mod h1:sl3t1tCWJFWoRz9R8WJCbQihKKwmorjAbSClcnxKAGw= -github.com/fvbommel/sortorder v1.1.0 h1:fUmoe+HLsBTctBDoaBwpQo5N+nrCp8g/BjKb/6ZQmYw= -github.com/fvbommel/sortorder v1.1.0/go.mod h1:uk88iVf1ovNn1iLfgUVU2F9o5eO30ui720w+kxuqRs0= +github.com/fxamacker/cbor/v2 v2.7.0 h1:iM5WgngdRBanHcxugY4JySA0nk1wZorNOpTgCMedv5E= +github.com/fxamacker/cbor/v2 v2.7.0/go.mod h1:pxXPTn3joSm21Gbwsv0w9OSA2y1HFR9qXEeXQVeNoDQ= github.com/go-errors/errors v1.4.2 h1:J6MZopCL4uSllY1OfXM374weqZFFItUbrImctkmUxIA= github.com/go-errors/errors v1.4.2/go.mod h1:sIVyrIiJhuEF+Pj9Ebtd6P/rEYROXFi3BopGUQ5a5Og= -github.com/go-logr/logr v1.2.4/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A= -github.com/go-logr/logr v1.3.0 h1:2y3SDp0ZXuc6/cjLSZ+Q3ir+QB9T/iG5yYRXqsagWSY= -github.com/go-logr/logr v1.3.0/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY= -github.com/go-logr/zapr v1.2.4 h1:QHVo+6stLbfJmYGkQ7uGHUCu5hnAFAj6mDe6Ea0SeOo= -github.com/go-logr/zapr v1.2.4/go.mod h1:FyHWQIzQORZ0QVE1BtVHv3cKtNLuXsbNLtpuhNapBOA= +github.com/go-logr/logr v1.4.2 h1:6pFjapn8bFcIbiKo3XT4j/BhANplGihG6tvd+8rYgrY= +github.com/go-logr/logr v1.4.2/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY= +github.com/go-logr/zapr v1.3.0 h1:XGdV8XW8zdwFiwOA2Dryh1gj2KRQyOOoNmBy4EplIcQ= +github.com/go-logr/zapr v1.3.0/go.mod h1:YKepepNBd1u/oyhd/yQmtjVXmm9uML4IXUgMOwR8/Gg= github.com/go-openapi/jsonpointer v0.19.6 h1:eCs3fxoIi3Wh6vtgmLTOjdhSpiqphQ+DaPn38N2ZdrE= github.com/go-openapi/jsonpointer v0.19.6/go.mod h1:osyAmYz/mB/C3I+WsTTSgw1ONzaLJoLCyoi6/zppojs= github.com/go-openapi/jsonreference v0.20.2 h1:3sVjiK66+uXK/6oQ8xgcRKcFgQ5KXa2KvnJRumpMGbE= github.com/go-openapi/jsonreference v0.20.2/go.mod h1:Bl1zwGIM8/wsvqjsOQLJ/SH+En5Ap4rVB5KVcIDZG2k= -github.com/go-openapi/swag v0.22.3 h1:yMBqmnQ0gyZvEb/+KzuWZOXgllrXT4SADYbvDaXHv/g= github.com/go-openapi/swag v0.22.3/go.mod h1:UzaqsxGiab7freDnrUUra0MwWfN/q7tE4j+VcZ0yl14= -github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572 h1:tfuBGBXKqDEevZMzYi5KSi8KkcZtzBcTgAUUtapy0OI= -github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572/go.mod h1:9Pwr4B2jHnOSGXyyzV8ROjYa2ojvAY6HCGYYfMoC3Ls= +github.com/go-openapi/swag v0.22.4 h1:QLMzNJnMGPRNDCbySlcj1x01tzU8/9LTTL9hZZZogBU= +github.com/go-openapi/swag v0.22.4/go.mod h1:UzaqsxGiab7freDnrUUra0MwWfN/q7tE4j+VcZ0yl14= +github.com/go-task/slim-sprig/v3 v3.0.0 h1:sUs3vkvUymDpBKi3qH1YSqBQk9+9D/8M2mN1vB6EwHI= +github.com/go-task/slim-sprig/v3 v3.0.0/go.mod h1:W848ghGpv3Qj3dhTPRyJypKRiqCdHZiAzKg9hl15HA8= github.com/gofrs/flock v0.8.1 h1:+gYjHKf32LDeiEEFhQaotPbLuUXjY5ZqxKgXy7n59aw= github.com/gofrs/flock v0.8.1/go.mod h1:F1TvTiK9OcQqauNUHlbJvyl9Qa1QvF/gOUDKA14jxHU= github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q= @@ -65,7 +65,6 @@ github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da h1:oI5xCqsCo564l github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A= github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= -github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8= github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA= @@ -73,9 +72,8 @@ github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrU github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w= github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0= github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8= -github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk= -github.com/golang/protobuf v1.5.3 h1:KhyjKVUg7Usr/dYsdSqoFveMYd5ko72D+zANwlG1mmg= -github.com/golang/protobuf v1.5.3/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY= +github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek= +github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps= github.com/google/btree v1.0.1 h1:gK4Kx5IaGY9CD5sPJ36FHiBJ6ZXl0kilRiiCj+jdYp4= github.com/google/btree v1.0.1/go.mod h1:xXMiIv4Fb/0kKde4SpL7qlzvu5cMJDRkFDxJfI9uaxA= github.com/google/gnostic-models v0.6.8 h1:yo/ABAfM5IMRsS1VnXjTBvUb61tFIHozhlYvRgGre9I= @@ -86,37 +84,32 @@ github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMyw github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= -github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI= github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= github.com/google/gofuzz v1.2.0 h1:xRy4A+RhZaiKjJ1bPfwQ8sedCA+YS2YcCHW6ec7JMi0= github.com/google/gofuzz v1.2.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= -github.com/google/pprof v0.0.0-20210720184732-4bb14d4b1be1 h1:K6RDEckDVWvDI9JAJYCmNdQXq6neHJOYx3V6jnqNEec= -github.com/google/pprof v0.0.0-20210720184732-4bb14d4b1be1/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= +github.com/google/pprof v0.0.0-20240827171923-fa2c70bbbfe5 h1:5iH8iuqE5apketRbSFBy+X1V0o+l+8NF1avt4HWl7cA= +github.com/google/pprof v0.0.0-20240827171923-fa2c70bbbfe5/go.mod h1:vavhavw2zAxS5dIdcRluK6cSGGPlZynqzFM8NdvU144= github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 h1:El6M4kTTCOh6aBiKaUGG7oYTSPP8MxqL4YI3kZKwcP4= github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510/go.mod h1:pupxD2MaaD3pAXIBCelhxNneeOaAeabZDe5s4K6zSpQ= -github.com/google/uuid v1.3.0 h1:t6JiXgmwXMjEs8VusXIJk2BXHsn+wx8BZdTaoZ5fu7I= -github.com/google/uuid v1.3.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= -github.com/gorilla/websocket v1.4.2/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE= +github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= +github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= github.com/gorilla/websocket v1.5.0 h1:PPwGk2jz7EePpoHN/+ClbZu8SPxiqlu12wZP/3sWmnc= github.com/gorilla/websocket v1.5.0/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE= github.com/gregjones/httpcache v0.0.0-20180305231024-9cad4c3443a7 h1:pdN6V1QBWetyv/0+wjACpqVH+eVULgEjkurDLq3goeM= github.com/gregjones/httpcache v0.0.0-20180305231024-9cad4c3443a7/go.mod h1:FecbI9+v66THATjSRHfNgh1IVFe/9kFxbXtjV0ctIMA= -github.com/ianlancetaylor/demangle v0.0.0-20200824232613-28f6c0f3b639/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc= github.com/imdario/mergo v0.3.6 h1:xTNEAn+kxVO7dTZGu0CegyqKZmoWFI0rF8UxjlB2d28= github.com/imdario/mergo v0.3.6/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA= github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8= github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw= -github.com/jessevdk/go-flags v1.4.0/go.mod h1:4FA24M0QyGHXBuZZK/XkWh8h0e1EYbRYJSGM75WSRxI= github.com/josharian/intern v1.0.0 h1:vlS4z54oSdjm0bgjRigI+G1HpF+tI+9rE5LLzOg8HmY= github.com/josharian/intern v1.0.0/go.mod h1:5DoeVV0s6jJacbCEi61lwdGj/aVlrQvzHFFd8Hwg//Y= github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM= github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo= github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8= github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck= -github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo= github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI= github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= @@ -126,16 +119,16 @@ github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de h1:9TO3cAIGXtEhnIaL+V+BEER86oLrvS+kWobKpbJuye0= github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de/go.mod h1:zAbeS9B/r2mtpb6U+EI2rYA5OAXxsYw6wTamcNW+zcE= +github.com/lithammer/dedent v1.1.0 h1:VNzHMVCBNG1j0fh3OrsFRkVUwStdDArbgBWoPAffktY= +github.com/lithammer/dedent v1.1.0/go.mod h1:jrXYCQtgg0nJiN+StA2KgR7w6CiQNv9Fd/Z9BP0jIOc= github.com/mailru/easyjson v0.7.7 h1:UGYAvKxe3sBsEDzO8ZeWOSlIQfWFlxbzLZe7hwFURr0= github.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc= -github.com/matttproud/golang_protobuf_extensions v1.0.4 h1:mmDVorXM7PCGKw94cs5zkfA9PSy5pEvNWRP0ET0TIVo= -github.com/matttproud/golang_protobuf_extensions v1.0.4/go.mod h1:BSXmuO+STAnVfrANrmjBb36TMTDstsz7MSK+HVaYKv4= github.com/mitchellh/go-wordwrap v1.0.1 h1:TLuKupo69TCn6TQSyGxwI1EblZZEsQ0vMlAFQflz0v0= github.com/mitchellh/go-wordwrap v1.0.1/go.mod h1:R62XHJLzvMFRBbcrT7m7WgmE1eOyTSsCt+hzestvNj0= -github.com/moby/spdystream v0.2.0 h1:cjW1zVyyoiM0T7b6UoySUFqzXMoqRckQtXwGPiBhOM8= -github.com/moby/spdystream v0.2.0/go.mod h1:f7i0iNDQJ059oMTcWxx8MA/zKFIuD/lY+0GqbN2Wy8c= -github.com/moby/term v0.0.0-20221205130635-1aeaba878587 h1:HfkjXDfhgVaN5rmueG8cL8KKeFNecRCXFhaJ2qZ5SKA= -github.com/moby/term v0.0.0-20221205130635-1aeaba878587/go.mod h1:8FzsFHVUBGZdbDsJw/ot+X+d5HLUbvklYLJ9uGfcI3Y= +github.com/moby/spdystream v0.4.0 h1:Vy79D6mHeJJjiPdFEL2yku1kl0chZpJfZcPpb16BRl8= +github.com/moby/spdystream v0.4.0/go.mod h1:xBAYlnt/ay+11ShkdFKNAG7LsyK/tmNBVvVOwrfMgdI= +github.com/moby/term v0.5.0 h1:xt8Q1nalod/v7BqbG21f8mQPqH+xAaC9C3N3wfWbVP0= +github.com/moby/term v0.5.0/go.mod h1:8FzsFHVUBGZdbDsJw/ot+X+d5HLUbvklYLJ9uGfcI3Y= github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg= github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= @@ -147,40 +140,40 @@ github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ= github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f h1:y5//uYreIhSUg3J1GEMiLbxo1LJaP8RfCpH6pymGZus= github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f/go.mod h1:ZdcZmHo+o7JKHSa8/e818NopupXU1YMK5fe1lsApnBw= -github.com/onsi/ginkgo/v2 v2.13.0 h1:0jY9lJquiL8fcf3M4LAXN5aMlS/b2BV86HFFPCPMgE4= -github.com/onsi/ginkgo/v2 v2.13.0/go.mod h1:TE309ZR8s5FsKKpuB1YAQYBzCaAfUgatB/xlT/ETL/o= -github.com/onsi/gomega v1.29.0 h1:KIA/t2t5UBzoirT4H9tsML45GEbo3ouUnBHsCfD2tVg= -github.com/onsi/gomega v1.29.0/go.mod h1:9sxs+SwGrKI0+PWe4Fxa9tFQQBG5xSsSbMXOI8PPpoQ= -github.com/oracle/oci-go-sdk/v65 v65.49.3 h1:HHv+XMZiBYHtoU8Ac/fURdp9v1vJPPCpIbJAWeadREw= -github.com/oracle/oci-go-sdk/v65 v65.49.3/go.mod h1:IBEV9l1qBzUpo7zgGaRUhbB05BVfcDGYRFBCPlTcPp0= +github.com/onsi/ginkgo/v2 v2.20.2 h1:7NVCeyIWROIAheY21RLS+3j2bb52W0W82tkberYytp4= +github.com/onsi/ginkgo/v2 v2.20.2/go.mod h1:K9gyxPIlb+aIvnZ8bd9Ak+YP18w3APlR+5coaZoE2ag= +github.com/onsi/gomega v1.34.2 h1:pNCwDkzrsv7MS9kpaQvVb1aVLahQXyJ/Tv5oAZMI3i8= +github.com/onsi/gomega v1.34.2/go.mod h1:v1xfxRgk0KIsG+QOdm7p8UosrOzPYRo60fd3B/1Dukc= +github.com/oracle/oci-go-sdk/v65 v65.77.1 h1:gqjTXIUWvTihkn470AclxSAMcR1JecqjD2IUtp+sDIU= +github.com/oracle/oci-go-sdk/v65 v65.77.1/go.mod h1:IBEV9l1qBzUpo7zgGaRUhbB05BVfcDGYRFBCPlTcPp0= github.com/peterbourgon/diskv v2.0.1+incompatible h1:UBdAOUP5p4RWqPBg048CAvpKN+vxiaj6gdUUzhl4XmI= github.com/peterbourgon/diskv v2.0.1+incompatible/go.mod h1:uqqh8zWWbv1HBMNONnaR/tNboyR3/BZd58JJSHlUSCU= -github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4= github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= -github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= -github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring v0.62.0 h1:55138zTXw/yRYizPxZ672I/aDD7Yte3uYRAfUjWUu2M= -github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring v0.62.0/go.mod h1:j51242bf6LQwvJ1JPKWApzTnifmCwcQq0i1p29ylWiM= -github.com/prometheus/client_golang v1.16.0 h1:yk/hx9hDbrGHovbci4BY+pRMfSuuat626eFsHb7tmT8= -github.com/prometheus/client_golang v1.16.0/go.mod h1:Zsulrv/L9oM40tJ7T815tM89lFEugiJ9HzIqaAx4LKc= +github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U= +github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring v0.78.2 h1:SyoVBXD/r0PntR1rprb90ClI32FSUNOCWqqTatnipHM= +github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring v0.78.2/go.mod h1:SvsRXw4m1F2vk7HquU5h475bFpke27mIUswfyw9u3ug= +github.com/prometheus/client_golang v1.19.1 h1:wZWJDwK+NameRJuPGDhlnFgx8e8HN3XHQeLaYJFJBOE= +github.com/prometheus/client_golang v1.19.1/go.mod h1:mP78NwGzrVks5S2H6ab8+ZZGJLZUq1hoULYBAYBw1Ho= github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= -github.com/prometheus/client_model v0.4.0 h1:5lQXD3cAg1OXBf4Wq03gTrXHeaV0TQvGfUooCfx1yqY= -github.com/prometheus/client_model v0.4.0/go.mod h1:oMQmHW1/JoDwqLtg57MGgP/Fb1CJEYF2imWWhWtMkYU= -github.com/prometheus/common v0.44.0 h1:+5BrQJwiBB9xsMygAB3TNvpQKOwlkc25LbISbrdOOfY= -github.com/prometheus/common v0.44.0/go.mod h1:ofAIvZbQ1e/nugmZGz4/qCb9Ap1VoSTIO7x0VV9VvuY= -github.com/prometheus/procfs v0.10.1 h1:kYK1Va/YMlutzCGazswoHKo//tZVlFpKYh+PymziUAg= -github.com/prometheus/procfs v0.10.1/go.mod h1:nwNm2aOCAYw8uTR/9bWRREkZFxAUcWzPHWJq+XBB/FM= -github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjRBZyWFQ= -github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog= +github.com/prometheus/client_model v0.6.1 h1:ZKSh/rekM+n3CeS952MLRAdFwIKqeY8b62p8ais2e9E= +github.com/prometheus/client_model v0.6.1/go.mod h1:OrxVMOVHjw3lKMa8+x6HeMGkHMQyHDk9E3jmP2AmGiY= +github.com/prometheus/common v0.55.0 h1:KEi6DK7lXW/m7Ig5i47x0vRzuBsHuvJdi5ee6Y3G1dc= +github.com/prometheus/common v0.55.0/go.mod h1:2SECS4xJG1kd8XF9IcM1gMX6510RAEL65zxzNImwdc8= +github.com/prometheus/procfs v0.15.1 h1:YagwOFzUgYfKKHX6Dr+sHT7km/hxC76UB0learggepc= +github.com/prometheus/procfs v0.15.1/go.mod h1:fB45yRUv8NstnjriLhBQLuOUt+WW4BsoGhij/e3PBqk= +github.com/rogpeppe/go-internal v1.12.0 h1:exVL4IDcn6na9z1rAb56Vxr+CgyK3nn3O+epU5NdKM8= +github.com/rogpeppe/go-internal v1.12.0/go.mod h1:E+RYuTGaKKdloAfM02xzb0FW3Paa99yedzYV+kq4uf4= github.com/russross/blackfriday/v2 v2.1.0 h1:JIOH55/0cWyOuilr9/qlrm0BSXldqnqwMsf35Ld67mk= github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM= -github.com/sergi/go-diff v1.1.0 h1:we8PVUC3FE2uYfodKH/nBHMSetSfHDR6scGdBi+erh0= -github.com/sergi/go-diff v1.1.0/go.mod h1:STckp+ISIX8hZLjrqAeVduY0gWCT9IjLuqbuNXdaHfM= +github.com/sergi/go-diff v1.2.0 h1:XU+rvMAioB0UC3q1MFrIQy4Vo5/4VsRDQQXHsEya6xQ= +github.com/sergi/go-diff v1.2.0/go.mod h1:STckp+ISIX8hZLjrqAeVduY0gWCT9IjLuqbuNXdaHfM= github.com/sony/gobreaker v0.5.0 h1:dRCvqm0P490vZPmy7ppEk2qCnCieBooFJ+YoXGYB+yg= github.com/sony/gobreaker v0.5.0/go.mod h1:ZKptC7FHNvhBz7dN2LGjPVBz2sZJmc0/PkyDJOjmxWY= -github.com/spf13/cobra v1.7.0 h1:hyqWnYt1ZQShIddO5kBpj3vu05/++x6tJ6dg8EC572I= -github.com/spf13/cobra v1.7.0/go.mod h1:uLxZILRyS/50WlhOIKD7W6V5bgeIt+4sICxh6uRMrb0= +github.com/spf13/cobra v1.8.1 h1:e5/vxKd/rZsfSJMUX1agtjeTDf+qv1/JdBF8gg5k9ZM= +github.com/spf13/cobra v1.8.1/go.mod h1:wHxEcudfqmLYa8iTfL+OuZPbBZkmvliBWKIezN3kD9Y= github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA= github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= @@ -188,91 +181,75 @@ github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSS github.com/stretchr/objx v0.5.0 h1:1zr/of2m5FGMsad5YfcqgdqdWrIhu+EBEJRhR1U7z/c= github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo= github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= -github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU= github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4= -github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcUk= github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo= +github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg= +github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY= +github.com/x448/float16 v0.8.4 h1:qLwI1I70+NjRFUR3zs1JPUCgaCXSh3SW62uAKT1mSBM= +github.com/x448/float16 v0.8.4/go.mod h1:14CWIYCyZA/cWjXOioeEpHeN/83MdbZDRQHoFcYsOfg= github.com/xlab/treeprint v1.2.0 h1:HzHnuAF1plUN2zGlAFHbSQP2qJ0ZAD3XF5XD7OesXRQ= github.com/xlab/treeprint v1.2.0/go.mod h1:gj5Gd3gPdKtR1ikdDK6fnFLdmIS0X30kTTuNd/WEJu0= github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= -github.com/yuin/goldmark v1.3.5/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k= go.starlark.net v0.0.0-20230525235612-a134d8f9ddca h1:VdD38733bfYv5tUZwEIskMM93VanwNIi5bIKnDrJdEY= go.starlark.net v0.0.0-20230525235612-a134d8f9ddca/go.mod h1:jxU+3+j+71eXOW14274+SmmuW82qJzl6iZSeqEtTGds= -go.uber.org/atomic v1.7.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc= -go.uber.org/goleak v1.1.11/go.mod h1:cwTWslyiVhfpKIDGSZEM2HlOvcqm+tG4zioyIeLoqMQ= -go.uber.org/goleak v1.2.1 h1:NBol2c7O1ZokfZ0LEU9K6Whx/KnwvepVetCUhtKja4A= -go.uber.org/goleak v1.2.1/go.mod h1:qlT2yGI9QafXHhZZLxlSuNsMw3FFLxBr+tBRlmO1xH4= -go.uber.org/multierr v1.6.0/go.mod h1:cdWPpRnG4AhwMwsgIHip0KRBQjJy5kYEpYjJxpXp9iU= +go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto= +go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE= go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0= go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y= -go.uber.org/zap v1.24.0/go.mod h1:2kMP+WWQ8aoFoedH3T2sq6iJ2yDWpHbP0f6MQbS9Gkg= go.uber.org/zap v1.26.0 h1:sI7k6L95XOKS281NhVKOFCUNIvv9e0w4BF8N3u+tCRo= go.uber.org/zap v1.26.0/go.mod h1:dtElttAiwGvoJ/vj4IwHBS/gXsEu/pZ50mUIRWuG0so= golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= -golang.org/x/exp v0.0.0-20220722155223-a9213eeb770e h1:+WEEuIdZHnUeJJmEUjyYC2gfUMj69yZXw17EnHg/otA= -golang.org/x/exp v0.0.0-20220722155223-a9213eeb770e/go.mod h1:Kr81I6Kryrl9sr8s2FK3vxD90NdsKWRuOIl2O4CvYbA= +golang.org/x/exp v0.0.0-20240719175910-8a7402abbf56 h1:2dVuKD2vS7b0QIHQbpyTISPd0LeHDbnYEryqj5Q1ug8= +golang.org/x/exp v0.0.0-20240719175910-8a7402abbf56/go.mod h1:M4RDyNAINzryxdtnbRXRL/OHtkFuWGRjvuhBJpk2IlY= golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE= golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU= golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= -golang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= -golang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= -golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks= golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= -golang.org/x/net v0.0.0-20210405180319-a5a99cb37ef4/go.mod h1:p54w0d4576C0XHj96bSt6lcn1PtDYWL6XObtHCRCNQM= -golang.org/x/net v0.19.0 h1:zTwKpTd2XuCqf8huc7Fo2iSy+4RHPd10s4KzeTnVr1c= -golang.org/x/net v0.19.0/go.mod h1:CfAk/cbD4CthTvqiEl8NpboMuiuOYsAr/7NOjZJtv1U= +golang.org/x/net v0.29.0 h1:5ORfpBpCs4HzDYoodCDBbwHzdR5UrLBZ3sOnUJmFoHo= +golang.org/x/net v0.29.0/go.mod h1:gLkgy8jTGERgjzMic6DS9+SP0ajcu6Xu3Orq/SpETg0= golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U= -golang.org/x/oauth2 v0.10.0 h1:zHCpF2Khkwy4mMB4bv0U37YtJdTGW8jI0glAApi0Kh8= -golang.org/x/oauth2 v0.10.0/go.mod h1:kTpgurOux7LqtuxjuyZa4Gj2gdezIt/jQtGnNFfypQI= +golang.org/x/oauth2 v0.21.0 h1:tsimM75w1tF/uws5rbeHzIWxEqElMehnc+iW793zsZs= +golang.org/x/oauth2 v0.21.0/go.mod h1:XYTD2NtWslqkgxebSiOHnXEap4TF09sJSc7H1sXbhtI= golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.5.0 h1:60k92dhOjHxJkrqnwsfl8KuaHbn/5dl0lUPUklKo3qE= -golang.org/x/sync v0.5.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk= +golang.org/x/sync v0.8.0 h1:3NFvSEYkUoMifnESzZl15y791HH1qU2xm6eCJU5ZPXQ= +golang.org/x/sync v0.8.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk= golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20210510120138-977fb7262007/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20210616094352-59db8d763f22/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20220908164124-27713097b956/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.8.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.15.0 h1:h48lPFYpsTvQJZF4EKyI4aLHaev3CxivZmv7yZig9pc= -golang.org/x/sys v0.15.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= -golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= +golang.org/x/sys v0.25.0 h1:r+8e+loiHxRqhXVl6ML1nO3l1+oFoWbnlu2Ehimmi34= +golang.org/x/sys v0.25.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= golang.org/x/term v0.0.0-20220526004731-065cf7ba2467/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8= -golang.org/x/term v0.15.0 h1:y/Oo/a/q3IXu26lQgl04j/gjuBDOBlx7X6Om1j2CPW4= -golang.org/x/term v0.15.0/go.mod h1:BDl952bC7+uMoWR75FIrCDx79TPU9oHkTZ9yRbYOrX0= +golang.org/x/term v0.24.0 h1:Mh5cbb+Zk2hqqXNO7S1iTjEphVL+jb8ZWaqh/g+JWkM= +golang.org/x/term v0.24.0/go.mod h1:lOBK/LVxemqiMij05LGJ0tzNr8xlmwBRJ81PX6wVLH8= golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= -golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk= golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= -golang.org/x/text v0.14.0 h1:ScX5w1eTa3QqT8oi6+ziP7dTV1S2+ALU0bI+0zXKWiQ= -golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU= +golang.org/x/text v0.19.0 h1:kTxAhCbGbxhK0IwgSKiMO5awPoDQ0RpfiVYBfK860YM= +golang.org/x/text v0.19.0/go.mod h1:BuEKDfySbSR4drPmRPG/7iBdf8hvFMuRexcpahXilzY= golang.org/x/time v0.3.0 h1:rg5rLMjNzMS1RkNLzCG38eapWhnYLFYXDXj2gOlr8j4= golang.org/x/time v0.3.0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= @@ -283,9 +260,8 @@ golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBn golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= -golang.org/x/tools v0.1.5/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= -golang.org/x/tools v0.16.1 h1:TLyB3WofjdOEepBHAU20JdNC1Zbg87elYofWYAY5oZA= -golang.org/x/tools v0.16.1/go.mod h1:kYVVN6I1mBNoB1OX+noeBjbRk4IUEPa7JJ+TJMEooJ0= +golang.org/x/tools v0.24.0 h1:J1shsA93PJUEVaUSaay7UXAyE8aimq3GW0pjlolpa24= +golang.org/x/tools v0.24.0/go.mod h1:YhNqVBIfWHdzvTLs0d8LCuMhkKUgSUKldakyV7W/WDQ= golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= @@ -294,8 +270,6 @@ gomodules.xyz/jsonpatch/v2 v2.4.0 h1:Ci3iUJyx9UeRx7CeFN8ARgGbkESwJK+KB9lLcWxY/Zw gomodules.xyz/jsonpatch/v2 v2.4.0/go.mod h1:AH3dM2RI6uoBZxn3LVrfvJ3E0/9dG4cSrbuBJT4moAY= google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM= google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4= -google.golang.org/appengine v1.6.7 h1:FZR1q0exgwxzPzp/aF+VccGrSfxfPpkBqjIIEq3ru6c= -google.golang.org/appengine v1.6.7/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc= google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc= google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc= google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo= @@ -310,14 +284,13 @@ google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzi google.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c= -google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw= -google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc= -google.golang.org/protobuf v1.31.0 h1:g0LDEJHgrBl9N9r17Ru3sqWhkIx2NB67okBHPwC7hs8= -google.golang.org/protobuf v1.31.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I= +google.golang.org/protobuf v1.34.2 h1:6xV6lTsCfpGD21XK49h7MhtcApnLqkfYgPcdHftf6hg= +google.golang.org/protobuf v1.34.2/go.mod h1:qYOHts0dSfpeUzUFpOMr/WGzszTmLH+DiWniOlNbLDw= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= -gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= +gopkg.in/evanphx/json-patch.v4 v4.12.0 h1:n6jtcsulIzXPJaxegRbvFNNrZDjbij7ny3gmSPG+6V4= +gopkg.in/evanphx/json-patch.v4 v4.12.0/go.mod h1:p8EYWUEYMpynmqDbY58zCKCFZw8pRWMG4EsWvDvM72M= gopkg.in/inf.v0 v0.9.1 h1:73M5CoZyi3ZLMOyDlQh031Cx6N9NDJ2Vvfl76EDAgDc= gopkg.in/inf.v0 v0.9.1/go.mod h1:cWUDdTG/fYaXco+Dcufb5Vnc6Gp2YChqWtbxRZE0mXw= gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= @@ -328,35 +301,35 @@ gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= -k8s.io/api v0.29.2 h1:hBC7B9+MU+ptchxEqTNW2DkUosJpp1P+Wn6YncZ474A= -k8s.io/api v0.29.2/go.mod h1:sdIaaKuU7P44aoyyLlikSLayT6Vb7bvJNCX105xZXY0= -k8s.io/apiextensions-apiserver v0.28.0 h1:CszgmBL8CizEnj4sj7/PtLGey6Na3YgWyGCPONv7E9E= -k8s.io/apiextensions-apiserver v0.28.0/go.mod h1:uRdYiwIuu0SyqJKriKmqEN2jThIJPhVmOWETm8ud1VE= -k8s.io/apimachinery v0.29.2 h1:EWGpfJ856oj11C52NRCHuU7rFDwxev48z+6DSlGNsV8= -k8s.io/apimachinery v0.29.2/go.mod h1:6HVkd1FwxIagpYrHSwJlQqZI3G9LfYWRPAkUvLnXTKU= -k8s.io/cli-runtime v0.29.2 h1:smfsOcT4QujeghsNjECKN3lwyX9AwcFU0nvJ7sFN3ro= -k8s.io/cli-runtime v0.29.2/go.mod h1:KLisYYfoqeNfO+MkTWvpqIyb1wpJmmFJhioA0xd4MW8= -k8s.io/client-go v0.29.2 h1:FEg85el1TeZp+/vYJM7hkDlSTFZ+c5nnK44DJ4FyoRg= -k8s.io/client-go v0.29.2/go.mod h1:knlvFZE58VpqbQpJNbCbctTVXcd35mMyAAwBdpt4jrA= -k8s.io/component-base v0.29.2 h1:lpiLyuvPA9yV1aQwGLENYyK7n/8t6l3nn3zAtFTJYe8= -k8s.io/component-base v0.29.2/go.mod h1:BfB3SLrefbZXiBfbM+2H1dlat21Uewg/5qtKOl8degM= -k8s.io/klog/v2 v2.110.1 h1:U/Af64HJf7FcwMcXyKm2RPM22WZzyR7OSpYj5tg3cL0= -k8s.io/klog/v2 v2.110.1/go.mod h1:YGtd1984u+GgbuZ7e08/yBuAfKLSO0+uR1Fhi6ExXjo= -k8s.io/kube-openapi v0.0.0-20231010175941-2dd684a91f00 h1:aVUu9fTY98ivBPKR9Y5w/AuzbMm96cd3YHRTU83I780= -k8s.io/kube-openapi v0.0.0-20231010175941-2dd684a91f00/go.mod h1:AsvuZPBlUDVuCdzJ87iajxtXuR9oktsTctW/R9wwouA= -k8s.io/kubectl v0.29.2 h1:uaDYaBhumvkwz0S2XHt36fK0v5IdNgL7HyUniwb2IUo= -k8s.io/kubectl v0.29.2/go.mod h1:BhizuYBGcKaHWyq+G7txGw2fXg576QbPrrnQdQDZgqI= -k8s.io/utils v0.0.0-20230726121419-3b25d923346b h1:sgn3ZU783SCgtaSJjpcVVlRqd6GSnlTLKgpAAttJvpI= -k8s.io/utils v0.0.0-20230726121419-3b25d923346b/go.mod h1:OLgZIPagt7ERELqWJFomSt595RzquPNLL48iOWgYOg0= -sigs.k8s.io/controller-runtime v0.16.2 h1:mwXAVuEk3EQf478PQwQ48zGOXvW27UJc8NHktQVuIPU= -sigs.k8s.io/controller-runtime v0.16.2/go.mod h1:vpMu3LpI5sYWtujJOa2uPK61nB5rbwlN7BAB8aSLvGU= +k8s.io/api v0.31.3 h1:umzm5o8lFbdN/hIXbrK9oRpOproJO62CV1zqxXrLgk8= +k8s.io/api v0.31.3/go.mod h1:UJrkIp9pnMOI9K2nlL6vwpxRzzEX5sWgn8kGQe92kCE= +k8s.io/apiextensions-apiserver v0.31.2 h1:W8EwUb8+WXBLu56ser5IudT2cOho0gAKeTOnywBLxd0= +k8s.io/apiextensions-apiserver v0.31.2/go.mod h1:i+Geh+nGCJEGiCGR3MlBDkS7koHIIKWVfWeRFiOsUcM= +k8s.io/apimachinery v0.31.3 h1:6l0WhcYgasZ/wk9ktLq5vLaoXJJr5ts6lkaQzgeYPq4= +k8s.io/apimachinery v0.31.3/go.mod h1:rsPdaZJfTfLsNJSQzNHQvYoTmxhoOEofxtOsF3rtsMo= +k8s.io/cli-runtime v0.31.3 h1:fEQD9Xokir78y7pVK/fCJN090/iYNrLHpFbGU4ul9TI= +k8s.io/cli-runtime v0.31.3/go.mod h1:Q2jkyTpl+f6AtodQvgDI8io3jrfr+Z0LyQBPJJ2Btq8= +k8s.io/client-go v0.31.3 h1:CAlZuM+PH2cm+86LOBemaJI/lQ5linJ6UFxKX/SoG+4= +k8s.io/client-go v0.31.3/go.mod h1:2CgjPUTpv3fE5dNygAr2NcM8nhHzXvxB8KL5gYc3kJs= +k8s.io/component-base v0.31.3 h1:DMCXXVx546Rfvhj+3cOm2EUxhS+EyztH423j+8sOwhQ= +k8s.io/component-base v0.31.3/go.mod h1:xME6BHfUOafRgT0rGVBGl7TuSg8Z9/deT7qq6w7qjIU= +k8s.io/klog/v2 v2.130.1 h1:n9Xl7H1Xvksem4KFG4PYbdQCQxqc/tTUyrgXaOhHSzk= +k8s.io/klog/v2 v2.130.1/go.mod h1:3Jpz1GvMt720eyJH1ckRHK1EDfpxISzJ7I9OYgaDtPE= +k8s.io/kube-openapi v0.0.0-20240228011516-70dd3763d340 h1:BZqlfIlq5YbRMFko6/PM7FjZpUb45WallggurYhKGag= +k8s.io/kube-openapi v0.0.0-20240228011516-70dd3763d340/go.mod h1:yD4MZYeKMBwQKVht279WycxKyM84kkAx2DPrTXaeb98= +k8s.io/kubectl v0.31.3 h1:3r111pCjPsvnR98oLLxDMwAeM6OPGmPty6gSKaLTQes= +k8s.io/kubectl v0.31.3/go.mod h1:lhMECDCbJN8He12qcKqs2QfmVo9Pue30geovBVpH5fs= +k8s.io/utils v0.0.0-20240902221715-702e33fdd3c3 h1:b2FmK8YH+QEwq/Sy2uAEhmqL5nPfGYbJOcaqjeYYZoA= +k8s.io/utils v0.0.0-20240902221715-702e33fdd3c3/go.mod h1:OLgZIPagt7ERELqWJFomSt595RzquPNLL48iOWgYOg0= +sigs.k8s.io/controller-runtime v0.19.3 h1:XO2GvC9OPftRst6xWCpTgBZO04S2cbp0Qqkj8bX1sPw= +sigs.k8s.io/controller-runtime v0.19.3/go.mod h1:j4j87DqtsThvwTv5/Tc5NFRyyF/RF0ip4+62tbTSIUM= sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd h1:EDPBXCAspyGV4jQlpZSudPeMmr1bNJefnuqLsRAsHZo= sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd/go.mod h1:B8JuhiUyNFVKdsE8h686QcCxMaH6HrOAZj4vswFpcB0= -sigs.k8s.io/kustomize/api v0.13.5-0.20230601165947-6ce0bf390ce3 h1:XX3Ajgzov2RKUdc5jW3t5jwY7Bo7dcRm+tFxT+NfgY0= -sigs.k8s.io/kustomize/api v0.13.5-0.20230601165947-6ce0bf390ce3/go.mod h1:9n16EZKMhXBNSiUC5kSdFQJkdH3zbxS/JoO619G1VAY= -sigs.k8s.io/kustomize/kyaml v0.14.3-0.20230601165947-6ce0bf390ce3 h1:W6cLQc5pnqM7vh3b7HvGNfXrJ/xL6BDMS0v1V/HHg5U= -sigs.k8s.io/kustomize/kyaml v0.14.3-0.20230601165947-6ce0bf390ce3/go.mod h1:JWP1Fj0VWGHyw3YUPjXSQnRnrwezrZSrApfX5S0nIag= +sigs.k8s.io/kustomize/api v0.17.2 h1:E7/Fjk7V5fboiuijoZHgs4aHuexi5Y2loXlVOAVAG5g= +sigs.k8s.io/kustomize/api v0.17.2/go.mod h1:UWTz9Ct+MvoeQsHcJ5e+vziRRkwimm3HytpZgIYqye0= +sigs.k8s.io/kustomize/kyaml v0.17.1 h1:TnxYQxFXzbmNG6gOINgGWQt09GghzgTP6mIurOgrLCQ= +sigs.k8s.io/kustomize/kyaml v0.17.1/go.mod h1:9V0mCjIEYjlXuCdYsSXvyoy2BTsLESH7TlGV81S282U= sigs.k8s.io/structured-merge-diff/v4 v4.4.1 h1:150L+0vs/8DA78h1u02ooW1/fFq/Lwr+sGiqlzvrtq4= sigs.k8s.io/structured-merge-diff/v4 v4.4.1/go.mod h1:N8hJocpFajUSSeSJ9bOZ77VzejKZaXsTtZo4/u7Io08= -sigs.k8s.io/yaml v1.3.0 h1:a2VclLzOGrwOHDiV8EfBGhvjHvP46CtW5j6POvhYGGo= -sigs.k8s.io/yaml v1.3.0/go.mod h1:GeOyir5tyXNByN85N/dRIT9es5UQNerPYEKK56eTBm8= +sigs.k8s.io/yaml v1.4.0 h1:Mk1wCc2gy/F0THH0TAp1QYyJNzRm2KCLy3o5ASXVI5E= +sigs.k8s.io/yaml v1.4.0/go.mod h1:Ejl7/uTz7PSA4eKMyQCUTnhZYNmLIl+5c2lQPGR2BPY= diff --git a/main.go b/main.go index 4174e97d..ee9992b7 100644 --- a/main.go +++ b/main.go @@ -63,8 +63,12 @@ import ( databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" databasecontroller "github.com/oracle/oracle-database-operator/controllers/database" + dataguardcontroller "github.com/oracle/oracle-database-operator/controllers/dataguard" + databasev4 "github.com/oracle/oracle-database-operator/apis/database/v4" + observabilityv1 "github.com/oracle/oracle-database-operator/apis/observability/v1" observabilityv1alpha1 "github.com/oracle/oracle-database-operator/apis/observability/v1alpha1" + observabilityv4 "github.com/oracle/oracle-database-operator/apis/observability/v4" observabilitycontroller "github.com/oracle/oracle-database-operator/controllers/observability" // +kubebuilder:scaffold:imports ) @@ -79,6 +83,9 @@ func init() { utilruntime.Must(observabilityv1alpha1.AddToScheme(scheme)) utilruntime.Must(monitorv1.AddToScheme(scheme)) utilruntime.Must(databasev1alpha1.AddToScheme(scheme)) + utilruntime.Must(databasev4.AddToScheme(scheme)) + utilruntime.Must(observabilityv1.AddToScheme(scheme)) + utilruntime.Must(observabilityv4.AddToScheme(scheme)) // +kubebuilder:scaffold:scheme } @@ -221,14 +228,22 @@ func main() { setupLog.Error(err, "unable to create webhook", "webhook", "OracleRestDataService") os.Exit(1) } - if err = (&databasev1alpha1.PDB{}).SetupWebhookWithManager(mgr); err != nil { + if err = (&databasev4.PDB{}).SetupWebhookWithManager(mgr); err != nil { setupLog.Error(err, "unable to create webhook", "webhook", "PDB") os.Exit(1) } - if err = (&databasev1alpha1.CDB{}).SetupWebhookWithManager(mgr); err != nil { + if err = (&databasev4.LRPDB{}).SetupWebhookWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create webhook", "webhook", "LRPDB") + os.Exit(1) + } + if err = (&databasev4.CDB{}).SetupWebhookWithManager(mgr); err != nil { setupLog.Error(err, "unable to create webhook", "webhook", "CDB") os.Exit(1) } + if err = (&databasev4.LREST{}).SetupWebhookWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create webhook", "webhook", "LREST") + os.Exit(1) + } if err = (&databasev1alpha1.AutonomousDatabase{}).SetupWebhookWithManager(mgr); err != nil { setupLog.Error(err, "unable to create webhook", "webhook", "AutonomousDatabase") os.Exit(1) @@ -245,17 +260,58 @@ func main() { setupLog.Error(err, "unable to create webhook", "webhook", "AutonomousContainerDatabase") os.Exit(1) } + if err = (&databasev4.AutonomousDatabase{}).SetupWebhookWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create webhook", "webhook", "AutonomousDatabase") + os.Exit(1) + } + if err = (&databasev4.AutonomousDatabaseBackup{}).SetupWebhookWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create webhook", "webhook", "AutonomousDatabaseBackup") + os.Exit(1) + } + if err = (&databasev4.AutonomousDatabaseRestore{}).SetupWebhookWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create webhook", "webhook", "AutonomousDatabaseRestore") + os.Exit(1) + } + if err = (&databasev4.AutonomousContainerDatabase{}).SetupWebhookWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create webhook", "webhook", "AutonomousContainerDatabase") + os.Exit(1) + } if err = (&databasev1alpha1.DataguardBroker{}).SetupWebhookWithManager(mgr); err != nil { setupLog.Error(err, "unable to create webhook", "webhook", "DataguardBroker") os.Exit(1) } if err = (&databasev1alpha1.ShardingDatabase{}).SetupWebhookWithManager(mgr); err != nil { setupLog.Error(err, "unable to create webhook", "webhook", "ShardingDatabase") + os.Exit(1) + } + if err = (&databasev1alpha1.DbcsSystem{}).SetupWebhookWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create webhook", "webhook", "DbcsSystem") + os.Exit(1) + } + if err = (&databasev4.ShardingDatabase{}).SetupWebhookWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create webhook", "webhook", "ShardingDatabase") } if err = (&observabilityv1alpha1.DatabaseObserver{}).SetupWebhookWithManager(mgr); err != nil { setupLog.Error(err, "unable to create webhook", "webhook", "DatabaseObserver") os.Exit(1) } + if err = (&databasev1alpha1.DbcsSystem{}).SetupWebhookWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create webhook", "webhook", "DbcsSystem") + os.Exit(1) + } + if err = (&databasev4.DbcsSystem{}).SetupWebhookWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create webhook", "webhook", "DbcsSystem") + os.Exit(1) + } + if err = (&observabilityv1.DatabaseObserver{}).SetupWebhookWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create webhook", "webhook", "DatabaseObserver") + os.Exit(1) + } + + if err = (&observabilityv4.DatabaseObserver{}).SetupWebhookWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create webhook", "webhook", "DatabaseObserver") + os.Exit(1) + } } // PDB Reconciler @@ -270,6 +326,18 @@ func main() { os.Exit(1) } + // LRPDBR Reconciler + if err = (&databasecontroller.LRPDBReconciler{ + Client: mgr.GetClient(), + Scheme: mgr.GetScheme(), + Log: ctrl.Log.WithName("controllers").WithName("LRPDB"), + Interval: time.Duration(i), + Recorder: mgr.GetEventRecorderFor("LRPDB"), + }).SetupWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create controller", "controller", "LRPDB") + os.Exit(1) + } + // CDB Reconciler if err = (&databasecontroller.CDBReconciler{ Client: mgr.GetClient(), @@ -282,9 +350,23 @@ func main() { setupLog.Error(err, "unable to create controller", "controller", "CDB") os.Exit(1) } - if err = (&databasecontroller.DataguardBrokerReconciler{ + + // LREST Reconciler + if err = (&databasecontroller.LRESTReconciler{ + Client: mgr.GetClient(), + Scheme: mgr.GetScheme(), + Config: mgr.GetConfig(), + Log: ctrl.Log.WithName("controllers").WithName("LREST"), + Interval: time.Duration(i), + Recorder: mgr.GetEventRecorderFor("LREST"), + }).SetupWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create controller", "controller", "LREST") + os.Exit(1) + } + + if err = (&dataguardcontroller.DataguardBrokerReconciler{ Client: mgr.GetClient(), - Log: ctrl.Log.WithName("controllers").WithName("database").WithName("DataguardBroker"), + Log: ctrl.Log.WithName("controllers").WithName("dataguard").WithName("DataguardBroker"), Scheme: mgr.GetScheme(), Config: mgr.GetConfig(), Recorder: mgr.GetEventRecorderFor("DataguardBroker"), @@ -293,6 +375,15 @@ func main() { os.Exit(1) } + if err = (&databasecontroller.OrdsSrvsReconciler{ + Client: mgr.GetClient(), + Scheme: mgr.GetScheme(), + // Config: mgr.GetConfig(), + Recorder: mgr.GetEventRecorderFor("OrdsSrvs"), + }).SetupWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create controller", "controller", "OrdsSrvs") + } + // Observability DatabaseObserver Reconciler if err = (&observabilitycontroller.DatabaseObserverReconciler{ Client: mgr.GetClient(), @@ -304,17 +395,37 @@ func main() { os.Exit(1) } + if err = (&databasev4.SingleInstanceDatabase{}).SetupWebhookWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create webhook", "webhook", "SingleInstanceDatabase") + os.Exit(1) + } + if err = (&databasev4.DataguardBroker{}).SetupWebhookWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create webhook", "webhook", "DataguardBroker") + os.Exit(1) + } + if err = (&databasev4.OracleRestDataService{}).SetupWebhookWithManager(mgr); err != nil { + setupLog.Error(err, "unable to create webhook", "webhook", "OracleRestDataService") + os.Exit(1) + } // +kubebuilder:scaffold:builder // Add index for PDB CR to enable mgr to cache PDBs indexFunc := func(obj client.Object) []string { - return []string{obj.(*databasev1alpha1.PDB).Spec.PDBName} + return []string{obj.(*databasev4.PDB).Spec.PDBName} } - if err = cache.IndexField(context.TODO(), &databasev1alpha1.PDB{}, "spec.pdbName", indexFunc); err != nil { + if err = cache.IndexField(context.TODO(), &databasev4.PDB{}, "spec.pdbName", indexFunc); err != nil { setupLog.Error(err, "unable to create index function for ", "controller", "PDB") os.Exit(1) } + indexFunc2 := func(obj client.Object) []string { + return []string{obj.(*databasev4.LRPDB).Spec.LRPDBName} + } + if err = cache.IndexField(context.TODO(), &databasev4.LRPDB{}, "spec.pdbName", indexFunc2); err != nil { + setupLog.Error(err, "unable to create index function for ", "controller", "LRPDB") + os.Exit(1) + } + setupLog.Info("starting manager") if err := mgr.Start(ctrl.SetupSignalHandler()); err != nil { setupLog.Error(err, "problem running manager") diff --git a/oracle-database-operator.yaml b/oracle-database-operator.yaml index 504fc7cd..70147329 100644 --- a/oracle-database-operator.yaml +++ b/oracle-database-operator.yaml @@ -9,10 +9,22 @@ apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - controller-gen.kubebuilder.io/version: v0.6.1 - creationTimestamp: null + cert-manager.io/inject-ca-from: oracle-database-operator-system/oracle-database-operator-serving-cert + controller-gen.kubebuilder.io/version: v0.16.5 name: autonomouscontainerdatabases.database.oracle.com spec: + conversion: + strategy: Webhook + webhook: + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /convert + conversionReviewVersions: + - v1alpha1 + - v1 + - v4 group: database.oracle.com names: kind: AutonomousContainerDatabase @@ -37,18 +49,14 @@ spec: name: v1alpha1 schema: openAPIV3Schema: - description: AutonomousContainerDatabase is the Schema for the autonomouscontainerdatabases API properties: apiVersion: - description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: - description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: - description: AutonomousContainerDatabaseSpec defines the desired state of AutonomousContainerDatabase properties: action: enum: @@ -57,7 +65,6 @@ spec: - TERMINATE type: string autonomousContainerDatabaseOCID: - description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster Important: Run "make" to regenerate code after modifying this file' type: string autonomousExadataVMClusterOCID: type: string @@ -73,7 +80,6 @@ spec: default: false type: boolean ociConfig: - description: "*********************** *\tOCI config ***********************" properties: configMapName: type: string @@ -81,17 +87,84 @@ spec: type: string type: object patchModel: - description: 'AutonomousContainerDatabasePatchModelEnum Enum with underlying type: string' enum: - RELEASE_UPDATES - RELEASE_UPDATE_REVISIONS type: string type: object status: - description: AutonomousContainerDatabaseStatus defines the observed state of AutonomousContainerDatabase properties: lifecycleState: - description: 'INSERT ADDITIONAL STATUS FIELD - define observed state of cluster Important: Run "make" to regenerate code after modifying this file' + type: string + timeCreated: + type: string + required: + - lifecycleState + type: object + type: object + served: true + storage: false + subresources: + status: {} + - additionalPrinterColumns: + - jsonPath: .spec.displayName + name: DisplayName + type: string + - jsonPath: .status.lifecycleState + name: State + type: string + - jsonPath: .status.timeCreated + name: Created + type: string + name: v4 + schema: + openAPIV3Schema: + properties: + apiVersion: + type: string + kind: + type: string + metadata: + type: object + spec: + properties: + action: + enum: + - SYNC + - RESTART + - TERMINATE + type: string + autonomousContainerDatabaseOCID: + type: string + autonomousExadataVMClusterOCID: + type: string + compartmentOCID: + type: string + displayName: + type: string + freeformTags: + additionalProperties: + type: string + type: object + hardLink: + default: false + type: boolean + ociConfig: + properties: + configMapName: + type: string + secretName: + type: string + type: object + patchModel: + enum: + - RELEASE_UPDATES + - RELEASE_UPDATE_REVISIONS + type: string + type: object + status: + properties: + lifecycleState: type: string timeCreated: type: string @@ -103,21 +176,27 @@ spec: storage: true subresources: status: {} -status: - acceptedNames: - kind: "" - plural: "" - conditions: [] - storedVersions: [] --- apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - controller-gen.kubebuilder.io/version: v0.6.1 - creationTimestamp: null + cert-manager.io/inject-ca-from: oracle-database-operator-system/oracle-database-operator-serving-cert + controller-gen.kubebuilder.io/version: v0.16.5 name: autonomousdatabasebackups.database.oracle.com spec: + conversion: + strategy: Webhook + webhook: + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /convert + conversionReviewVersions: + - v1alpha1 + - v1 + - v4 group: database.oracle.com names: kind: AutonomousDatabaseBackup @@ -148,18 +227,14 @@ spec: name: v1alpha1 schema: openAPIV3Schema: - description: AutonomousDatabaseBackup is the Schema for the autonomousdatabasebackups API properties: apiVersion: - description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: - description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: - description: AutonomousDatabaseBackupSpec defines the desired state of AutonomousDatabaseBackup properties: autonomousDatabaseBackupOCID: type: string @@ -168,7 +243,6 @@ spec: isLongTermBackup: type: boolean ociConfig: - description: "*********************** *\tOCI config ***********************" properties: configMapName: type: string @@ -178,10 +252,8 @@ spec: retentionPeriodInDays: type: integer target: - description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster Important: Run "make" to regenerate code after modifying this file' properties: k8sADB: - description: "*********************** *\tADB spec ***********************" properties: name: type: string @@ -194,7 +266,6 @@ spec: type: object type: object status: - description: AutonomousDatabaseBackupStatus defines the observed state of AutonomousDatabaseBackup properties: autonomousDatabaseOCID: type: string @@ -207,14 +278,103 @@ spec: isAutomatic: type: boolean lifecycleState: - description: 'AutonomousDatabaseBackupLifecycleStateEnum Enum with underlying type: string' type: string timeEnded: type: string timeStarted: type: string type: - description: 'AutonomousDatabaseBackupTypeEnum Enum with underlying type: string' + type: string + required: + - autonomousDatabaseOCID + - compartmentOCID + - dbDisplayName + - dbName + - isAutomatic + - lifecycleState + - type + type: object + type: object + served: true + storage: false + subresources: + status: {} + - additionalPrinterColumns: + - jsonPath: .status.lifecycleState + name: State + type: string + - jsonPath: .status.dbDisplayName + name: DB DisplayName + type: string + - jsonPath: .status.type + name: Type + type: string + - jsonPath: .status.timeStarted + name: Started + type: string + - jsonPath: .status.timeEnded + name: Ended + type: string + name: v4 + schema: + openAPIV3Schema: + properties: + apiVersion: + type: string + kind: + type: string + metadata: + type: object + spec: + properties: + autonomousDatabaseBackupOCID: + type: string + displayName: + type: string + isLongTermBackup: + type: boolean + ociConfig: + properties: + configMapName: + type: string + secretName: + type: string + type: object + retentionPeriodInDays: + type: integer + target: + properties: + k8sADB: + properties: + name: + type: string + type: object + ociADB: + properties: + ocid: + type: string + type: object + type: object + type: object + status: + properties: + autonomousDatabaseOCID: + type: string + compartmentOCID: + type: string + dbDisplayName: + type: string + dbName: + type: string + isAutomatic: + type: boolean + lifecycleState: + type: string + timeEnded: + type: string + timeStarted: + type: string + type: type: string required: - autonomousDatabaseOCID @@ -230,21 +390,27 @@ spec: storage: true subresources: status: {} -status: - acceptedNames: - kind: "" - plural: "" - conditions: [] - storedVersions: [] --- apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - controller-gen.kubebuilder.io/version: v0.6.1 - creationTimestamp: null + cert-manager.io/inject-ca-from: oracle-database-operator-system/oracle-database-operator-serving-cert + controller-gen.kubebuilder.io/version: v0.16.5 name: autonomousdatabaserestores.database.oracle.com spec: + conversion: + strategy: Webhook + webhook: + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /convert + conversionReviewVersions: + - v1alpha1 + - v1 + - v4 group: database.oracle.com names: kind: AutonomousDatabaseRestore @@ -269,21 +435,16 @@ spec: name: v1alpha1 schema: openAPIV3Schema: - description: AutonomousDatabaseRestore is the Schema for the autonomousdatabaserestores API properties: apiVersion: - description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: - description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: - description: AutonomousDatabaseRestoreSpec defines the desired state of AutonomousDatabaseRestore properties: ociConfig: - description: "*********************** *\tOCI config ***********************" properties: configMapName: type: string @@ -293,7 +454,6 @@ spec: source: properties: k8sADBBackup: - description: 'EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN! NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized.' properties: name: type: string @@ -301,15 +461,12 @@ spec: pointInTime: properties: timestamp: - description: 'The timestamp must follow this format: YYYY-MM-DD HH:MM:SS GMT' type: string type: object type: object target: - description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster Important: Run "make" to regenerate code after modifying this file' properties: k8sADB: - description: "*********************** *\tADB spec ***********************" properties: name: type: string @@ -325,15 +482,98 @@ spec: - target type: object status: - description: AutonomousDatabaseRestoreStatus defines the observed state of AutonomousDatabaseRestore properties: dbName: type: string displayName: - description: 'INSERT ADDITIONAL STATUS FIELD - define observed state of cluster Important: Run "make" to regenerate code after modifying this file' type: string status: - description: 'WorkRequestStatusEnum Enum with underlying type: string' + type: string + timeAccepted: + type: string + timeEnded: + type: string + timeStarted: + type: string + workRequestOCID: + type: string + required: + - dbName + - displayName + - status + - workRequestOCID + type: object + type: object + served: true + storage: false + subresources: + status: {} + - additionalPrinterColumns: + - jsonPath: .status.status + name: Status + type: string + - jsonPath: .status.displayName + name: DbDisplayName + type: string + - jsonPath: .status.dbName + name: DbName + type: string + name: v4 + schema: + openAPIV3Schema: + properties: + apiVersion: + type: string + kind: + type: string + metadata: + type: object + spec: + properties: + ociConfig: + properties: + configMapName: + type: string + secretName: + type: string + type: object + source: + properties: + k8sADBBackup: + properties: + name: + type: string + type: object + pointInTime: + properties: + timestamp: + type: string + type: object + type: object + target: + properties: + k8sADB: + properties: + name: + type: string + type: object + ociADB: + properties: + ocid: + type: string + type: object + type: object + required: + - source + - target + type: object + status: + properties: + dbName: + type: string + displayName: + type: string + status: type: string timeAccepted: type: string @@ -354,21 +594,27 @@ spec: storage: true subresources: status: {} -status: - acceptedNames: - kind: "" - plural: "" - conditions: [] - storedVersions: [] --- apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - controller-gen.kubebuilder.io/version: v0.6.1 - creationTimestamp: null + cert-manager.io/inject-ca-from: oracle-database-operator-system/oracle-database-operator-serving-cert + controller-gen.kubebuilder.io/version: v0.16.5 name: autonomousdatabases.database.oracle.com spec: + conversion: + strategy: Webhook + webhook: + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /convert + conversionReviewVersions: + - v1alpha1 + - v1 + - v4 group: database.oracle.com names: kind: AutonomousDatabase @@ -408,54 +654,67 @@ spec: name: v1alpha1 schema: openAPIV3Schema: - description: AutonomousDatabase is the Schema for the autonomousdatabases API properties: apiVersion: - description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: - description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: - description: 'AutonomousDatabaseSpec defines the desired state of AutonomousDatabase Important: Run "make" to regenerate code after modifying this file' properties: - details: - description: AutonomousDatabaseDetails defines the detail information of AutonomousDatabase, corresponding to oci-go-sdk/database/AutonomousDatabase + action: + enum: + - "" + - Create + - Sync + - Update + - Stop + - Start + - Terminate + - Clone + type: string + clone: properties: adminPassword: properties: k8sSecret: - description: "*********************** *\tSecret specs ***********************" properties: name: type: string type: object ociSecret: properties: - ocid: + id: type: string type: object type: object autonomousContainerDatabase: - description: ACDSpec defines the spec of the target for backup/restore runs. The name could be the name of an AutonomousDatabase or an AutonomousDatabaseBackup properties: - k8sACD: - description: "*********************** *\tACD specs ***********************" + k8sAcd: properties: name: type: string type: object - ociACD: + ociAcd: properties: - ocid: + id: type: string type: object type: object - autonomousDatabaseOCID: + cloneType: + enum: + - FULL + - METADATA type: string - compartmentOCID: + compartmentId: + type: string + computeCount: + type: number + computeModel: + enum: + - ECPU + - OCPU type: string cpuCoreCount: type: integer @@ -466,7 +725,6 @@ spec: dbVersion: type: string dbWorkload: - description: 'AutonomousDatabaseDbWorkloadEnum Enum with underlying type: string' enum: - OLTP - DW @@ -479,84 +737,158 @@ spec: additionalProperties: type: string type: object + isAccessControlEnabled: + type: boolean isAutoScalingEnabled: type: boolean isDedicated: type: boolean + isFreeTier: + type: boolean + isMtlsConnectionRequired: + type: boolean licenseModel: - description: 'AutonomousDatabaseLicenseModelEnum Enum with underlying type: string' enum: - LICENSE_INCLUDED - BRING_YOUR_OWN_LICENSE type: string - lifecycleState: - description: 'AutonomousDatabaseLifecycleStateEnum Enum with underlying type: string' + nsgIds: + items: + type: string + type: array + ocpuCount: + type: number + privateEndpointLabel: + type: string + subnetId: type: string - networkAccess: + whitelistedIps: + items: + type: string + type: array + type: object + details: + properties: + adminPassword: properties: - accessControlList: - items: - type: string - type: array - accessType: - enum: - - "" - - PUBLIC - - RESTRICTED - - PRIVATE - type: string - isAccessControlEnabled: - type: boolean - isMTLSConnectionRequired: - type: boolean - privateEndpoint: + k8sSecret: properties: - hostnamePrefix: + name: type: string - nsgOCIDs: - items: - type: string - type: array - subnetOCID: + type: object + ociSecret: + properties: + id: type: string type: object type: object - wallet: + autonomousContainerDatabase: properties: - name: - type: string - password: + k8sAcd: properties: - k8sSecret: - description: "*********************** *\tSecret specs ***********************" - properties: - name: - type: string - type: object - ociSecret: - properties: - ocid: - type: string - type: object + name: + type: string + type: object + ociAcd: + properties: + id: + type: string type: object type: object + compartmentId: + type: string + computeCount: + type: number + computeModel: + enum: + - ECPU + - OCPU + type: string + cpuCoreCount: + type: integer + dataStorageSizeInTBs: + type: integer + dbName: + type: string + dbVersion: + type: string + dbWorkload: + enum: + - OLTP + - DW + - AJD + - APEX + type: string + displayName: + type: string + freeformTags: + additionalProperties: + type: string + type: object + id: + type: string + isAccessControlEnabled: + type: boolean + isAutoScalingEnabled: + type: boolean + isDedicated: + type: boolean + isFreeTier: + type: boolean + isMtlsConnectionRequired: + type: boolean + licenseModel: + enum: + - LICENSE_INCLUDED + - BRING_YOUR_OWN_LICENSE + type: string + nsgIds: + items: + type: string + type: array + ocpuCount: + type: number + privateEndpointLabel: + type: string + subnetId: + type: string + whitelistedIps: + items: + type: string + type: array type: object hardLink: default: false type: boolean ociConfig: - description: "*********************** *\tOCI config ***********************" properties: configMapName: type: string secretName: type: string type: object + wallet: + properties: + name: + type: string + password: + properties: + k8sSecret: + properties: + name: + type: string + type: object + ociSecret: + properties: + id: + type: string + type: object + type: object + type: object required: - - details + - action type: object status: - description: AutonomousDatabaseStatus defines the observed state of AutonomousDatabase properties: allConnectionStrings: items: @@ -578,36 +910,29 @@ spec: type: array conditions: items: - description: "Condition contains details for one aspect of the current state of this API Resource. --- This struct is intended for direct use as an array at the field path .status.conditions. For example, \n \ttype FooStatus struct{ \t // Represents the observations of a foo's current state. \t // Known .status.conditions.type are: \"Available\", \"Progressing\", and \"Degraded\" \t // +patchMergeKey=type \t // +patchStrategy=merge \t // +listType=map \t // +listMapKey=type \t Conditions []metav1.Condition `json:\"conditions,omitempty\" patchStrategy:\"merge\" patchMergeKey:\"type\" protobuf:\"bytes,1,rep,name=conditions\"` \n \t // other fields \t}" properties: lastTransitionTime: - description: lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. format: date-time type: string message: - description: message is a human readable message indicating details about the transition. This may be an empty string. maxLength: 32768 type: string observedGeneration: - description: observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance. format: int64 minimum: 0 type: integer reason: - description: reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty. maxLength: 1024 minLength: 1 pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ type: string status: - description: status of the condition, one of True, False, Unknown. enum: - "True" - "False" - Unknown type: string type: - description: type of condition in CamelCase or in foo.example.com/CamelCase. --- Many .condition.type values are consistent across resources like Available, but because arbitrary conditions can be useful (see .node.status.conditions), the ability to deconflict is important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) maxLength: 316 pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ type: string @@ -623,7 +948,6 @@ spec: - type x-kubernetes-list-type: map lifecycleState: - description: 'INSERT ADDITIONAL STATUS FIELD - define observed state of cluster Important: Run "make" to regenerate code after modifying this file' type: string timeCreated: type: string @@ -632,433 +956,316 @@ spec: type: object type: object served: true - storage: true + storage: false subresources: status: {} -status: - acceptedNames: - kind: "" - plural: "" - conditions: [] - storedVersions: [] ---- -apiVersion: apiextensions.k8s.io/v1 -kind: CustomResourceDefinition -metadata: - annotations: - cert-manager.io/inject-ca-from: oracle-database-operator-system/oracle-database-operator-serving-cert - controller-gen.kubebuilder.io/version: v0.6.1 - name: cdbs.database.oracle.com -spec: - group: database.oracle.com - names: - kind: CDB - listKind: CDBList - plural: cdbs - singular: cdb - scope: Namespaced - versions: - additionalPrinterColumns: - - description: Name of the CDB - jsonPath: .spec.cdbName - name: CDB Name + - jsonPath: .spec.details.displayName + name: Display Name type: string - - description: ' Name of the DB Server' - jsonPath: .spec.dbServer - name: DB Server + - jsonPath: .spec.details.dbName + name: Db Name type: string - - description: DB server port - jsonPath: .spec.dbPort - name: DB Port - type: integer - - description: ' string of the tnsalias' - jsonPath: .spec.dbTnsurl - name: TNS STRING + - jsonPath: .status.lifecycleState + name: State type: string - - description: Replicas - jsonPath: .spec.replicas - name: Replicas + - jsonPath: .spec.details.isDedicated + name: Dedicated + type: string + - jsonPath: .spec.details.cpuCoreCount + name: OCPUs type: integer - - description: Status of the CDB Resource - jsonPath: .status.phase - name: Status + - jsonPath: .spec.details.dataStorageSizeInTBs + name: Storage (TB) + type: integer + - jsonPath: .spec.details.dbWorkload + name: Workload Type type: string - - description: Error message, if any - jsonPath: .status.msg - name: Message + - jsonPath: .status.timeCreated + name: Created type: string - name: v1alpha1 + name: v4 schema: openAPIV3Schema: - description: CDB is the Schema for the cdbs API properties: apiVersion: - description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: - description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: - description: CDBSpec defines the desired state of CDB properties: - cdbAdminPwd: - description: Password for the CDB Administrator to manage PDB lifecycle - properties: - secret: - description: CDBSecret defines the secretName - properties: - key: - type: string - secretName: - type: string - required: - - key - - secretName - type: object - required: - - secret - type: object - cdbAdminUser: - description: User in the root container with sysdba priviledges to manage PDB lifecycle - properties: - secret: - description: CDBSecret defines the secretName - properties: - key: - type: string - secretName: - type: string - required: - - key - - secretName - type: object - required: - - secret - type: object - cdbName: - description: Name of the CDB + action: + enum: + - "" + - Create + - Sync + - Update + - Stop + - Start + - Terminate + - Clone type: string - cdbTlsCrt: + clone: properties: - secret: - description: CDBSecret defines the secretName + adminPassword: properties: - key: - type: string - secretName: - type: string - required: - - key - - secretName + k8sSecret: + properties: + name: + type: string + type: object + ociSecret: + properties: + id: + type: string + type: object type: object - required: - - secret - type: object - cdbTlsKey: - properties: - secret: - description: CDBSecret defines the secretName + autonomousContainerDatabase: properties: - key: - type: string - secretName: - type: string - required: - - key - - secretName + k8sAcd: + properties: + name: + type: string + type: object + ociAcd: + properties: + id: + type: string + type: object type: object - required: - - secret - type: object - dbPort: - description: DB server port - type: integer - dbServer: - description: Name of the DB server - type: string - dbTnsurl: - type: string - nodeSelector: - additionalProperties: - type: string - description: Node Selector for running the Pod - type: object - ordsImage: - description: ORDS Image Name - type: string - ordsImagePullPolicy: - description: ORDS Image Pull Policy - enum: - - Always - - Never - type: string - ordsImagePullSecret: - description: The name of the image pull secret in case of a private docker repository. - type: string - ordsPort: - description: ORDS server port. For now, keep it as 8888. TO BE USED IN FUTURE RELEASE. - type: integer - ordsPwd: - description: Password for user ORDS_PUBLIC_USER - properties: - secret: - description: CDBSecret defines the secretName - properties: - key: - type: string - secretName: - type: string - required: - - key - - secretName - type: object - required: - - secret - type: object - replicas: - description: Number of ORDS Containers to create - type: integer - serviceName: - description: Name of the CDB Service - type: string - sysAdminPwd: - description: Password for the CDB System Administrator - properties: - secret: - description: CDBSecret defines the secretName - properties: - key: - type: string - secretName: - type: string - required: - - key - - secretName - type: object - required: - - secret - type: object - webServerPwd: - description: Password for the Web Server User - properties: - secret: - description: CDBSecret defines the secretName - properties: - key: - type: string - secretName: - type: string - required: - - key - - secretName - type: object - required: - - secret - type: object - webServerUser: - description: Web Server User with SQL Administrator role to allow us to authenticate to the PDB Lifecycle Management REST endpoints - properties: - secret: - description: CDBSecret defines the secretName - properties: - key: - type: string - secretName: - type: string - required: - - key - - secretName + cloneType: + enum: + - FULL + - METADATA + type: string + compartmentId: + type: string + computeCount: + type: number + computeModel: + enum: + - ECPU + - OCPU + type: string + cpuCoreCount: + type: integer + dataStorageSizeInTBs: + type: integer + dbName: + type: string + dbVersion: + type: string + dbWorkload: + enum: + - OLTP + - DW + - AJD + - APEX + type: string + displayName: + type: string + freeformTags: + additionalProperties: + type: string type: object - required: - - secret + isAccessControlEnabled: + type: boolean + isAutoScalingEnabled: + type: boolean + isDedicated: + type: boolean + isFreeTier: + type: boolean + isMtlsConnectionRequired: + type: boolean + licenseModel: + enum: + - LICENSE_INCLUDED + - BRING_YOUR_OWN_LICENSE + type: string + nsgIds: + items: + type: string + type: array + ocpuCount: + type: number + privateEndpointLabel: + type: string + subnetId: + type: string + whitelistedIps: + items: + type: string + type: array type: object - type: object - status: - description: CDBStatus defines the observed state of CDB - properties: - msg: - description: Message - type: string - phase: - description: Phase of the CDB Resource - type: string - status: - description: CDB Resource Status - type: boolean - required: - - phase - - status - type: object - type: object - served: true - storage: true - subresources: - status: {} -status: - acceptedNames: - kind: "" - plural: "" - conditions: [] - storedVersions: [] ---- -apiVersion: apiextensions.k8s.io/v1 -kind: CustomResourceDefinition -metadata: - annotations: - controller-gen.kubebuilder.io/version: v0.6.1 - creationTimestamp: null - name: databaseobservers.observability.oracle.com -spec: - group: observability.oracle.com - names: - kind: DatabaseObserver - listKind: DatabaseObserverList - plural: databaseobservers - singular: databaseobserver - scope: Namespaced - versions: - - additionalPrinterColumns: - - jsonPath: .status.exporterConfig - name: ExporterConfig - type: string - - jsonPath: .status.status - name: Status - type: string - name: v1alpha1 - schema: - openAPIV3Schema: - description: DatabaseObserver is the Schema for the databaseobservers API - properties: - apiVersion: - description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' - type: string - kind: - description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' - type: string - metadata: - type: object - spec: - description: DatabaseObserverSpec defines the desired state of DatabaseObserver - properties: - database: - description: DatabaseObserverDatabase defines the database details used for DatabaseObserver + details: properties: - dbConnectionString: - properties: - key: - type: string - secret: - type: string - type: object - dbPassword: - properties: - key: - type: string - secret: - type: string - vaultOCID: - type: string - vaultSecretName: - type: string - type: object - dbUser: - properties: - key: - type: string - secret: - type: string - type: object - dbWallet: + adminPassword: properties: - key: - type: string - secret: - type: string + k8sSecret: + properties: + name: + type: string + type: object + ociSecret: + properties: + id: + type: string + type: object type: object - type: object - exporter: - description: DatabaseObserverExporterConfig defines the configuration details related to the exporters of DatabaseObserver - properties: - configuration: + autonomousContainerDatabase: properties: - configmap: - description: ConfigMapDetails defines the configmap name + k8sAcd: properties: - configmapName: + name: type: string - key: + type: object + ociAcd: + properties: + id: type: string type: object type: object - image: + compartmentId: type: string - service: - description: DatabaseObserverService defines the exporter service component of DatabaseObserver - properties: - port: - format: int32 - type: integer - type: object - type: object - ociConfig: - properties: - configMapName: + computeCount: + type: number + computeModel: + enum: + - ECPU + - OCPU type: string - secretName: + cpuCoreCount: + type: integer + dataStorageSizeInTBs: + type: integer + dbName: type: string - type: object - prometheus: - description: PrometheusConfig defines the generated resources for Prometheus - properties: - labels: + dbVersion: + type: string + dbWorkload: + enum: + - OLTP + - DW + - AJD + - APEX + type: string + displayName: + type: string + freeformTags: additionalProperties: type: string type: object - port: + id: type: string - type: object - replicas: - format: int32 - type: integer - type: object - status: - description: DatabaseObserverStatus defines the observed state of DatabaseObserver - properties: - conditions: - description: 'INSERT ADDITIONAL STATUS FIELD - define observed state of cluster Important: Run "make" to regenerate code after modifying this file' - items: - description: "Condition contains details for one aspect of the current state of this API Resource. --- This struct is intended for direct use as an array at the field path .status.conditions. For example, \n \ttype FooStatus struct{ \t // Represents the observations of a foo's current state. \t // Known .status.conditions.type are: \"Available\", \"Progressing\", and \"Degraded\" \t // +patchMergeKey=type \t // +patchStrategy=merge \t // +listType=map \t // +listMapKey=type \t Conditions []metav1.Condition `json:\"conditions,omitempty\" patchStrategy:\"merge\" patchMergeKey:\"type\" protobuf:\"bytes,1,rep,name=conditions\"` \n \t // other fields \t}" + isAccessControlEnabled: + type: boolean + isAutoScalingEnabled: + type: boolean + isDedicated: + type: boolean + isFreeTier: + type: boolean + isMtlsConnectionRequired: + type: boolean + licenseModel: + enum: + - LICENSE_INCLUDED + - BRING_YOUR_OWN_LICENSE + type: string + nsgIds: + items: + type: string + type: array + ocpuCount: + type: number + privateEndpointLabel: + type: string + subnetId: + type: string + whitelistedIps: + items: + type: string + type: array + type: object + hardLink: + default: false + type: boolean + ociConfig: + properties: + configMapName: + type: string + secretName: + type: string + type: object + wallet: + properties: + name: + type: string + password: + properties: + k8sSecret: + properties: + name: + type: string + type: object + ociSecret: + properties: + id: + type: string + type: object + type: object + type: object + required: + - action + type: object + status: + properties: + allConnectionStrings: + items: + properties: + connectionStrings: + items: + properties: + connectionString: + type: string + tnsName: + type: string + type: object + type: array + tlsAuthentication: + type: string + required: + - connectionStrings + type: object + type: array + conditions: + items: properties: lastTransitionTime: - description: lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. format: date-time type: string message: - description: message is a human readable message indicating details about the transition. This may be an empty string. maxLength: 32768 type: string observedGeneration: - description: observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance. format: int64 minimum: 0 type: integer reason: - description: reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty. maxLength: 1024 minLength: 1 pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ type: string status: - description: status of the condition, one of True, False, Unknown. enum: - "True" - "False" - Unknown type: string type: - description: type of condition in CamelCase or in foo.example.com/CamelCase. --- Many .condition.type values are consistent across resources like Available, but because arbitrary conditions can be useful (see .node.status.conditions), the ability to deconflict is important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) maxLength: 316 pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ type: string @@ -1070,970 +1277,10602 @@ spec: - type type: object type: array - exporterConfig: + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + lifecycleState: type: string - replicas: - type: integer - status: + timeCreated: + type: string + walletExpiringDate: type: string - required: - - conditions - - exporterConfig type: object type: object served: true storage: true subresources: status: {} -status: - acceptedNames: - kind: "" - plural: "" - conditions: [] - storedVersions: [] --- apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - controller-gen.kubebuilder.io/version: v0.6.1 - creationTimestamp: null - name: dataguardbrokers.database.oracle.com + cert-manager.io/inject-ca-from: oracle-database-operator-system/oracle-database-operator-serving-cert + controller-gen.kubebuilder.io/version: v0.16.5 + name: cdbs.database.oracle.com spec: group: database.oracle.com names: - kind: DataguardBroker - listKind: DataguardBrokerList - plural: dataguardbrokers - singular: dataguardbroker + kind: CDB + listKind: CDBList + plural: cdbs + singular: cdb scope: Namespaced versions: - additionalPrinterColumns: - - jsonPath: .status.primaryDatabase - name: Primary - type: string - - jsonPath: .status.standbyDatabases - name: Standbys - type: string - - jsonPath: .spec.protectionMode - name: Protection Mode + - description: Name of the CDB + jsonPath: .spec.cdbName + name: CDB Name type: string - - jsonPath: .status.clusterConnectString - name: Cluster Connect Str - priority: 1 + - description: ' Name of the DB Server' + jsonPath: .spec.dbServer + name: DB Server type: string - - jsonPath: .status.externalConnectString - name: Connect Str + - description: DB server port + jsonPath: .spec.dbPort + name: DB Port + type: integer + - description: Replicas + jsonPath: .spec.replicas + name: Replicas + type: integer + - description: Status of the CDB Resource + jsonPath: .status.phase + name: Status type: string - - jsonPath: .spec.primaryDatabaseRef - name: Primary Database - priority: 1 + - description: Error message, if any + jsonPath: .status.msg + name: Message type: string - - jsonPath: .status.status - name: Status + - description: ' string of the tnsalias' + jsonPath: .spec.dbTnsurl + name: TNS STRING type: string - name: v1alpha1 + name: v4 schema: openAPIV3Schema: - description: DataguardBroker is the Schema for the dataguardbrokers API properties: apiVersion: - description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: - description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: - description: DataguardBrokerSpec defines the desired state of DataguardBroker properties: - fastStartFailOver: + cdbAdminPwd: properties: - enable: - type: boolean - strategy: - items: - description: FSFO strategy - properties: - sourceDatabaseRef: - type: string - targetDatabaseRefs: - type: string - type: object - type: array + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret type: object - loadBalancer: + cdbAdminUser: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + cdbName: + type: string + cdbOrdsPrvKey: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + cdbOrdsPubKey: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + cdbTlsCrt: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + cdbTlsKey: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + dbPort: + type: integer + dbServer: + type: string + dbTnsurl: + type: string + deletePdbCascade: type: boolean nodeSelector: additionalProperties: type: string type: object - primaryDatabaseRef: + ordsImage: type: string - protectionMode: + ordsImagePullPolicy: enum: - - MaxPerformance - - MaxAvailability + - Always + - Never type: string - serviceAnnotations: - additionalProperties: - type: string - type: object - setAsPrimaryDatabase: + ordsImagePullSecret: type: string - standbyDatabaseRefs: - items: - type: string - type: array - required: - - primaryDatabaseRef - - protectionMode - - standbyDatabaseRefs + ordsPort: + type: integer + ordsPwd: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + replicas: + type: integer + serviceName: + type: string + sysAdminPwd: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + webServerPwd: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + webServerUser: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object type: object status: - description: DataguardBrokerStatus defines the observed state of DataguardBroker properties: - clusterConnectString: - type: string - externalConnectString: - type: string - primaryDatabase: - type: string - primaryDatabaseRef: - type: string - protectionMode: + msg: type: string - standbyDatabases: + phase: type: string status: - type: string + type: boolean + required: + - phase + - status type: object type: object served: true storage: true subresources: status: {} -status: - acceptedNames: - kind: "" - plural: "" - conditions: [] - storedVersions: [] --- apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - controller-gen.kubebuilder.io/version: v0.6.1 - creationTimestamp: null - name: dbcssystems.database.oracle.com + controller-gen.kubebuilder.io/version: v0.16.5 + name: databaseobservers.observability.oracle.com spec: - group: database.oracle.com + group: observability.oracle.com names: - kind: DbcsSystem - listKind: DbcsSystemList - plural: dbcssystems - singular: dbcssystem + kind: DatabaseObserver + listKind: DatabaseObserverList + plural: databaseobservers + shortNames: + - dbobserver + - dbobservers + singular: databaseobserver scope: Namespaced versions: - - name: v1alpha1 + - additionalPrinterColumns: + - jsonPath: .status.exporterConfig + name: ExporterConfig + type: string + - jsonPath: .status.status + name: Status + type: string + - jsonPath: .status.version + name: Version + type: string + name: v1 schema: openAPIV3Schema: - description: DbcsSystem is the Schema for the dbcssystems API properties: apiVersion: - description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: - description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: - description: DbcsSystemSpec defines the desired state of DbcsSystem properties: - dbSystem: + configuration: properties: - availabilityDomain: - type: string - backupSubnetId: - type: string - clusterName: - type: string - compartmentId: - type: string - cpuCoreCount: - type: integer - dbAdminPaswordSecret: - type: string - dbBackupConfig: - description: DB Backup COnfig Network Struct + configMap: properties: - autoBackupEnabled: - type: boolean - autoBackupWindow: + key: type: string - backupDestinationDetails: + name: type: string - recoveryWindowsInDays: - type: integer type: object - dbDomain: - type: string - dbEdition: - type: string - dbName: + type: object + database: + properties: + dbConnectionString: + properties: + key: + type: string + secret: + type: string + type: object + dbPassword: + properties: + key: + type: string + secret: + type: string + vaultOCID: + type: string + vaultSecretName: + type: string + type: object + dbUser: + properties: + key: + type: string + secret: + type: string + type: object + dbWallet: + properties: + key: + type: string + secret: + type: string + type: object + type: object + exporter: + properties: + deployment: + properties: + args: + items: + type: string + type: array + commands: + items: + type: string + type: array + env: + additionalProperties: + type: string + type: object + image: + type: string + labels: + additionalProperties: + type: string + type: object + podTemplate: + properties: + labels: + additionalProperties: + type: string + type: object + securityContext: + properties: + appArmorProfile: + properties: + localhostProfile: + type: string + type: + type: string + required: + - type + type: object + fsGroup: + format: int64 + type: integer + fsGroupChangePolicy: + type: string + runAsGroup: + format: int64 + type: integer + runAsNonRoot: + type: boolean + runAsUser: + format: int64 + type: integer + seLinuxOptions: + properties: + level: + type: string + role: + type: string + type: + type: string + user: + type: string + type: object + seccompProfile: + properties: + localhostProfile: + type: string + type: + type: string + required: + - type + type: object + supplementalGroups: + items: + format: int64 + type: integer + type: array + x-kubernetes-list-type: atomic + supplementalGroupsPolicy: + type: string + sysctls: + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + x-kubernetes-list-type: atomic + windowsOptions: + properties: + gmsaCredentialSpec: + type: string + gmsaCredentialSpecName: + type: string + hostProcess: + type: boolean + runAsUserName: + type: string + type: object + type: object + type: object + securityContext: + properties: + allowPrivilegeEscalation: + type: boolean + appArmorProfile: + properties: + localhostProfile: + type: string + type: + type: string + required: + - type + type: object + capabilities: + properties: + add: + items: + type: string + type: array + x-kubernetes-list-type: atomic + drop: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + privileged: + type: boolean + procMount: + type: string + readOnlyRootFilesystem: + type: boolean + runAsGroup: + format: int64 + type: integer + runAsNonRoot: + type: boolean + runAsUser: + format: int64 + type: integer + seLinuxOptions: + properties: + level: + type: string + role: + type: string + type: + type: string + user: + type: string + type: object + seccompProfile: + properties: + localhostProfile: + type: string + type: + type: string + required: + - type + type: object + windowsOptions: + properties: + gmsaCredentialSpec: + type: string + gmsaCredentialSpecName: + type: string + hostProcess: + type: boolean + runAsUserName: + type: string + type: object + type: object + type: object + service: + properties: + labels: + additionalProperties: + type: string + type: object + ports: + items: + properties: + appProtocol: + type: string + name: + type: string + nodePort: + format: int32 + type: integer + port: + format: int32 + type: integer + protocol: + default: TCP + type: string + targetPort: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + required: + - port + type: object + type: array + type: object + type: object + inheritLabels: + items: + type: string + type: array + log: + properties: + filename: type: string - dbUniqueName: + path: type: string - dbVersion: + volume: + properties: + name: + type: string + persistentVolumeClaim: + properties: + claimName: + type: string + type: object + type: object + type: object + ociConfig: + properties: + configMapName: type: string - dbWorkload: + secretName: + type: string + type: object + prometheus: + properties: + serviceMonitor: + properties: + endpoints: + items: + properties: + authorization: + properties: + credentials: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: + type: string + type: object + basicAuth: + properties: + password: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + username: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: object + bearerTokenFile: + type: string + bearerTokenSecret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + enableHttp2: + type: boolean + filterRunning: + type: boolean + followRedirects: + type: boolean + honorLabels: + type: boolean + honorTimestamps: + type: boolean + interval: + pattern: ^(0|(([0-9]+)y)?(([0-9]+)w)?(([0-9]+)d)?(([0-9]+)h)?(([0-9]+)m)?(([0-9]+)s)?(([0-9]+)ms)?)$ + type: string + metricRelabelings: + items: + properties: + action: + default: replace + enum: + - replace + - Replace + - keep + - Keep + - drop + - Drop + - hashmod + - HashMod + - labelmap + - LabelMap + - labeldrop + - LabelDrop + - labelkeep + - LabelKeep + - lowercase + - Lowercase + - uppercase + - Uppercase + - keepequal + - KeepEqual + - dropequal + - DropEqual + type: string + modulus: + format: int64 + type: integer + regex: + type: string + replacement: + type: string + separator: + type: string + sourceLabels: + items: + pattern: ^[a-zA-Z_][a-zA-Z0-9_]*$ + type: string + type: array + targetLabel: + type: string + type: object + type: array + oauth2: + properties: + clientId: + properties: + configMap: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + secret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: object + clientSecret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + endpointParams: + additionalProperties: + type: string + type: object + noProxy: + type: string + proxyConnectHeader: + additionalProperties: + items: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: array + type: object + x-kubernetes-map-type: atomic + proxyFromEnvironment: + type: boolean + proxyUrl: + pattern: ^http(s)?://.+$ + type: string + scopes: + items: + type: string + type: array + tlsConfig: + properties: + ca: + properties: + configMap: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + secret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: object + cert: + properties: + configMap: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + secret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: object + insecureSkipVerify: + type: boolean + keySecret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + maxVersion: + enum: + - TLS10 + - TLS11 + - TLS12 + - TLS13 + type: string + minVersion: + enum: + - TLS10 + - TLS11 + - TLS12 + - TLS13 + type: string + serverName: + type: string + type: object + tokenUrl: + minLength: 1 + type: string + required: + - clientId + - clientSecret + - tokenUrl + type: object + params: + additionalProperties: + items: + type: string + type: array + type: object + path: + type: string + port: + type: string + proxyUrl: + type: string + relabelings: + items: + properties: + action: + default: replace + enum: + - replace + - Replace + - keep + - Keep + - drop + - Drop + - hashmod + - HashMod + - labelmap + - LabelMap + - labeldrop + - LabelDrop + - labelkeep + - LabelKeep + - lowercase + - Lowercase + - uppercase + - Uppercase + - keepequal + - KeepEqual + - dropequal + - DropEqual + type: string + modulus: + format: int64 + type: integer + regex: + type: string + replacement: + type: string + separator: + type: string + sourceLabels: + items: + pattern: ^[a-zA-Z_][a-zA-Z0-9_]*$ + type: string + type: array + targetLabel: + type: string + type: object + type: array + scheme: + enum: + - http + - https + type: string + scrapeTimeout: + pattern: ^(0|(([0-9]+)y)?(([0-9]+)w)?(([0-9]+)d)?(([0-9]+)h)?(([0-9]+)m)?(([0-9]+)s)?(([0-9]+)ms)?)$ + type: string + targetPort: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + tlsConfig: + properties: + ca: + properties: + configMap: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + secret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: object + caFile: + type: string + cert: + properties: + configMap: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + secret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: object + certFile: + type: string + insecureSkipVerify: + type: boolean + keyFile: + type: string + keySecret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + maxVersion: + enum: + - TLS10 + - TLS11 + - TLS12 + - TLS13 + type: string + minVersion: + enum: + - TLS10 + - TLS11 + - TLS12 + - TLS13 + type: string + serverName: + type: string + type: object + trackTimestampsStaleness: + type: boolean + type: object + type: array + labels: + additionalProperties: + type: string + type: object + namespaceSelector: + properties: + any: + type: boolean + matchNames: + items: + type: string + type: array + type: object + type: object + type: object + replicas: + format: int32 + type: integer + sidecarVolumes: + items: + properties: + awsElasticBlockStore: + properties: + fsType: + type: string + partition: + format: int32 + type: integer + readOnly: + type: boolean + volumeID: + type: string + required: + - volumeID + type: object + azureDisk: + properties: + cachingMode: + type: string + diskName: + type: string + diskURI: + type: string + fsType: + default: ext4 + type: string + kind: + type: string + readOnly: + default: false + type: boolean + required: + - diskName + - diskURI + type: object + azureFile: + properties: + readOnly: + type: boolean + secretName: + type: string + shareName: + type: string + required: + - secretName + - shareName + type: object + cephfs: + properties: + monitors: + items: + type: string + type: array + x-kubernetes-list-type: atomic + path: + type: string + readOnly: + type: boolean + secretFile: + type: string + secretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + user: + type: string + required: + - monitors + type: object + cinder: + properties: + fsType: + type: string + readOnly: + type: boolean + secretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + volumeID: + type: string + required: + - volumeID + type: object + configMap: + properties: + defaultMode: + format: int32 + type: integer + items: + items: + properties: + key: + type: string + mode: + format: int32 + type: integer + path: + type: string + required: + - key + - path + type: object + type: array + x-kubernetes-list-type: atomic + name: + default: "" + type: string + optional: + type: boolean + type: object + x-kubernetes-map-type: atomic + csi: + properties: + driver: + type: string + fsType: + type: string + nodePublishSecretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + readOnly: + type: boolean + volumeAttributes: + additionalProperties: + type: string + type: object + required: + - driver + type: object + downwardAPI: + properties: + defaultMode: + format: int32 + type: integer + items: + items: + properties: + fieldRef: + properties: + apiVersion: + type: string + fieldPath: + type: string + required: + - fieldPath + type: object + x-kubernetes-map-type: atomic + mode: + format: int32 + type: integer + path: + type: string + resourceFieldRef: + properties: + containerName: + type: string + divisor: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + resource: + type: string + required: + - resource + type: object + x-kubernetes-map-type: atomic + required: + - path + type: object + type: array + x-kubernetes-list-type: atomic + type: object + emptyDir: + properties: + medium: + type: string + sizeLimit: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + ephemeral: + properties: + volumeClaimTemplate: + properties: + metadata: + type: object + spec: + properties: + accessModes: + items: + type: string + type: array + x-kubernetes-list-type: atomic + dataSource: + properties: + apiGroup: + type: string + kind: + type: string + name: + type: string + required: + - kind + - name + type: object + x-kubernetes-map-type: atomic + dataSourceRef: + properties: + apiGroup: + type: string + kind: + type: string + name: + type: string + namespace: + type: string + required: + - kind + - name + type: object + resources: + properties: + limits: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + requests: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + type: object + selector: + properties: + matchExpressions: + items: + properties: + key: + type: string + operator: + type: string + values: + items: + type: string + type: array + x-kubernetes-list-type: atomic + required: + - key + - operator + type: object + type: array + x-kubernetes-list-type: atomic + matchLabels: + additionalProperties: + type: string + type: object + type: object + x-kubernetes-map-type: atomic + storageClassName: + type: string + volumeAttributesClassName: + type: string + volumeMode: + type: string + volumeName: + type: string + type: object + required: + - spec + type: object + type: object + fc: + properties: + fsType: + type: string + lun: + format: int32 + type: integer + readOnly: + type: boolean + targetWWNs: + items: + type: string + type: array + x-kubernetes-list-type: atomic + wwids: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + flexVolume: + properties: + driver: + type: string + fsType: + type: string + options: + additionalProperties: + type: string + type: object + readOnly: + type: boolean + secretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + required: + - driver + type: object + flocker: + properties: + datasetName: + type: string + datasetUUID: + type: string + type: object + gcePersistentDisk: + properties: + fsType: + type: string + partition: + format: int32 + type: integer + pdName: + type: string + readOnly: + type: boolean + required: + - pdName + type: object + gitRepo: + properties: + directory: + type: string + repository: + type: string + revision: + type: string + required: + - repository + type: object + glusterfs: + properties: + endpoints: + type: string + path: + type: string + readOnly: + type: boolean + required: + - endpoints + - path + type: object + hostPath: + properties: + path: + type: string + type: + type: string + required: + - path + type: object + image: + properties: + pullPolicy: + type: string + reference: + type: string + type: object + iscsi: + properties: + chapAuthDiscovery: + type: boolean + chapAuthSession: + type: boolean + fsType: + type: string + initiatorName: + type: string + iqn: + type: string + iscsiInterface: + default: default + type: string + lun: + format: int32 + type: integer + portals: + items: + type: string + type: array + x-kubernetes-list-type: atomic + readOnly: + type: boolean + secretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + targetPortal: + type: string + required: + - iqn + - lun + - targetPortal + type: object + name: + type: string + nfs: + properties: + path: + type: string + readOnly: + type: boolean + server: + type: string + required: + - path + - server + type: object + persistentVolumeClaim: + properties: + claimName: + type: string + readOnly: + type: boolean + required: + - claimName + type: object + photonPersistentDisk: + properties: + fsType: + type: string + pdID: + type: string + required: + - pdID + type: object + portworxVolume: + properties: + fsType: + type: string + readOnly: + type: boolean + volumeID: + type: string + required: + - volumeID + type: object + projected: + properties: + defaultMode: + format: int32 + type: integer + sources: + items: + properties: + clusterTrustBundle: + properties: + labelSelector: + properties: + matchExpressions: + items: + properties: + key: + type: string + operator: + type: string + values: + items: + type: string + type: array + x-kubernetes-list-type: atomic + required: + - key + - operator + type: object + type: array + x-kubernetes-list-type: atomic + matchLabels: + additionalProperties: + type: string + type: object + type: object + x-kubernetes-map-type: atomic + name: + type: string + optional: + type: boolean + path: + type: string + signerName: + type: string + required: + - path + type: object + configMap: + properties: + items: + items: + properties: + key: + type: string + mode: + format: int32 + type: integer + path: + type: string + required: + - key + - path + type: object + type: array + x-kubernetes-list-type: atomic + name: + default: "" + type: string + optional: + type: boolean + type: object + x-kubernetes-map-type: atomic + downwardAPI: + properties: + items: + items: + properties: + fieldRef: + properties: + apiVersion: + type: string + fieldPath: + type: string + required: + - fieldPath + type: object + x-kubernetes-map-type: atomic + mode: + format: int32 + type: integer + path: + type: string + resourceFieldRef: + properties: + containerName: + type: string + divisor: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + resource: + type: string + required: + - resource + type: object + x-kubernetes-map-type: atomic + required: + - path + type: object + type: array + x-kubernetes-list-type: atomic + type: object + secret: + properties: + items: + items: + properties: + key: + type: string + mode: + format: int32 + type: integer + path: + type: string + required: + - key + - path + type: object + type: array + x-kubernetes-list-type: atomic + name: + default: "" + type: string + optional: + type: boolean + type: object + x-kubernetes-map-type: atomic + serviceAccountToken: + properties: + audience: + type: string + expirationSeconds: + format: int64 + type: integer + path: + type: string + required: + - path + type: object + type: object + type: array + x-kubernetes-list-type: atomic + type: object + quobyte: + properties: + group: + type: string + readOnly: + type: boolean + registry: + type: string + tenant: + type: string + user: + type: string + volume: + type: string + required: + - registry + - volume + type: object + rbd: + properties: + fsType: + type: string + image: + type: string + keyring: + default: /etc/ceph/keyring + type: string + monitors: + items: + type: string + type: array + x-kubernetes-list-type: atomic + pool: + default: rbd + type: string + readOnly: + type: boolean + secretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + user: + default: admin + type: string + required: + - image + - monitors + type: object + scaleIO: + properties: + fsType: + default: xfs + type: string + gateway: + type: string + protectionDomain: + type: string + readOnly: + type: boolean + secretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + sslEnabled: + type: boolean + storageMode: + default: ThinProvisioned + type: string + storagePool: + type: string + system: + type: string + volumeName: + type: string + required: + - gateway + - secretRef + - system + type: object + secret: + properties: + defaultMode: + format: int32 + type: integer + items: + items: + properties: + key: + type: string + mode: + format: int32 + type: integer + path: + type: string + required: + - key + - path + type: object + type: array + x-kubernetes-list-type: atomic + optional: + type: boolean + secretName: + type: string + type: object + storageos: + properties: + fsType: + type: string + readOnly: + type: boolean + secretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + volumeName: + type: string + volumeNamespace: + type: string + type: object + vsphereVolume: + properties: + fsType: + type: string + storagePolicyID: + type: string + storagePolicyName: + type: string + volumePath: + type: string + required: + - volumePath + type: object + required: + - name + type: object + type: array + sidecars: + items: + properties: + args: + items: + type: string + type: array + x-kubernetes-list-type: atomic + command: + items: + type: string + type: array + x-kubernetes-list-type: atomic + env: + items: + properties: + name: + type: string + value: + type: string + valueFrom: + properties: + configMapKeyRef: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + fieldRef: + properties: + apiVersion: + type: string + fieldPath: + type: string + required: + - fieldPath + type: object + x-kubernetes-map-type: atomic + resourceFieldRef: + properties: + containerName: + type: string + divisor: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + resource: + type: string + required: + - resource + type: object + x-kubernetes-map-type: atomic + secretKeyRef: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: object + required: + - name + type: object + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + envFrom: + items: + properties: + configMapRef: + properties: + name: + default: "" + type: string + optional: + type: boolean + type: object + x-kubernetes-map-type: atomic + prefix: + type: string + secretRef: + properties: + name: + default: "" + type: string + optional: + type: boolean + type: object + x-kubernetes-map-type: atomic + type: object + type: array + x-kubernetes-list-type: atomic + image: + type: string + imagePullPolicy: + type: string + lifecycle: + properties: + postStart: + properties: + exec: + properties: + command: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + httpGet: + properties: + host: + type: string + httpHeaders: + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + x-kubernetes-list-type: atomic + path: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + scheme: + type: string + required: + - port + type: object + sleep: + properties: + seconds: + format: int64 + type: integer + required: + - seconds + type: object + tcpSocket: + properties: + host: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + required: + - port + type: object + type: object + preStop: + properties: + exec: + properties: + command: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + httpGet: + properties: + host: + type: string + httpHeaders: + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + x-kubernetes-list-type: atomic + path: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + scheme: + type: string + required: + - port + type: object + sleep: + properties: + seconds: + format: int64 + type: integer + required: + - seconds + type: object + tcpSocket: + properties: + host: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + required: + - port + type: object + type: object + type: object + livenessProbe: + properties: + exec: + properties: + command: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + failureThreshold: + format: int32 + type: integer + grpc: + properties: + port: + format: int32 + type: integer + service: + default: "" + type: string + required: + - port + type: object + httpGet: + properties: + host: + type: string + httpHeaders: + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + x-kubernetes-list-type: atomic + path: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + scheme: + type: string + required: + - port + type: object + initialDelaySeconds: + format: int32 + type: integer + periodSeconds: + format: int32 + type: integer + successThreshold: + format: int32 + type: integer + tcpSocket: + properties: + host: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + required: + - port + type: object + terminationGracePeriodSeconds: + format: int64 + type: integer + timeoutSeconds: + format: int32 + type: integer + type: object + name: + type: string + ports: + items: + properties: + containerPort: + format: int32 + type: integer + hostIP: + type: string + hostPort: + format: int32 + type: integer + name: + type: string + protocol: + default: TCP + type: string + required: + - containerPort + type: object + type: array + x-kubernetes-list-map-keys: + - containerPort + - protocol + x-kubernetes-list-type: map + readinessProbe: + properties: + exec: + properties: + command: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + failureThreshold: + format: int32 + type: integer + grpc: + properties: + port: + format: int32 + type: integer + service: + default: "" + type: string + required: + - port + type: object + httpGet: + properties: + host: + type: string + httpHeaders: + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + x-kubernetes-list-type: atomic + path: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + scheme: + type: string + required: + - port + type: object + initialDelaySeconds: + format: int32 + type: integer + periodSeconds: + format: int32 + type: integer + successThreshold: + format: int32 + type: integer + tcpSocket: + properties: + host: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + required: + - port + type: object + terminationGracePeriodSeconds: + format: int64 + type: integer + timeoutSeconds: + format: int32 + type: integer + type: object + resizePolicy: + items: + properties: + resourceName: + type: string + restartPolicy: + type: string + required: + - resourceName + - restartPolicy + type: object + type: array + x-kubernetes-list-type: atomic + resources: + properties: + claims: + items: + properties: + name: + type: string + request: + type: string + required: + - name + type: object + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + limits: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + requests: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + type: object + restartPolicy: + type: string + securityContext: + properties: + allowPrivilegeEscalation: + type: boolean + appArmorProfile: + properties: + localhostProfile: + type: string + type: + type: string + required: + - type + type: object + capabilities: + properties: + add: + items: + type: string + type: array + x-kubernetes-list-type: atomic + drop: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + privileged: + type: boolean + procMount: + type: string + readOnlyRootFilesystem: + type: boolean + runAsGroup: + format: int64 + type: integer + runAsNonRoot: + type: boolean + runAsUser: + format: int64 + type: integer + seLinuxOptions: + properties: + level: + type: string + role: + type: string + type: + type: string + user: + type: string + type: object + seccompProfile: + properties: + localhostProfile: + type: string + type: + type: string + required: + - type + type: object + windowsOptions: + properties: + gmsaCredentialSpec: + type: string + gmsaCredentialSpecName: + type: string + hostProcess: + type: boolean + runAsUserName: + type: string + type: object + type: object + startupProbe: + properties: + exec: + properties: + command: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + failureThreshold: + format: int32 + type: integer + grpc: + properties: + port: + format: int32 + type: integer + service: + default: "" + type: string + required: + - port + type: object + httpGet: + properties: + host: + type: string + httpHeaders: + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + x-kubernetes-list-type: atomic + path: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + scheme: + type: string + required: + - port + type: object + initialDelaySeconds: + format: int32 + type: integer + periodSeconds: + format: int32 + type: integer + successThreshold: + format: int32 + type: integer + tcpSocket: + properties: + host: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + required: + - port + type: object + terminationGracePeriodSeconds: + format: int64 + type: integer + timeoutSeconds: + format: int32 + type: integer + type: object + stdin: + type: boolean + stdinOnce: + type: boolean + terminationMessagePath: + type: string + terminationMessagePolicy: + type: string + tty: + type: boolean + volumeDevices: + items: + properties: + devicePath: + type: string + name: + type: string + required: + - devicePath + - name + type: object + type: array + x-kubernetes-list-map-keys: + - devicePath + x-kubernetes-list-type: map + volumeMounts: + items: + properties: + mountPath: + type: string + mountPropagation: + type: string + name: + type: string + readOnly: + type: boolean + recursiveReadOnly: + type: string + subPath: + type: string + subPathExpr: + type: string + required: + - mountPath + - name + type: object + type: array + x-kubernetes-list-map-keys: + - mountPath + x-kubernetes-list-type: map + workingDir: + type: string + required: + - name + type: object + type: array + type: object + status: + properties: + conditions: + items: + properties: + lastTransitionTime: + format: date-time + type: string + message: + maxLength: 32768 + type: string + observedGeneration: + format: int64 + minimum: 0 + type: integer + reason: + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + enum: + - "True" + - "False" + - Unknown + type: string + type: + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + exporterConfig: + type: string + replicas: + type: integer + status: + type: string + version: + type: string + required: + - conditions + - exporterConfig + - version + type: object + type: object + served: true + storage: false + subresources: + status: {} + - additionalPrinterColumns: + - jsonPath: .status.exporterConfig + name: ExporterConfig + type: string + - jsonPath: .status.status + name: Status + type: string + - jsonPath: .status.version + name: Version + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + properties: + apiVersion: + type: string + kind: + type: string + metadata: + type: object + spec: + properties: + configuration: + properties: + configMap: + properties: + key: + type: string + name: + type: string + type: object + type: object + database: + properties: + dbConnectionString: + properties: + key: + type: string + secret: + type: string + type: object + dbPassword: + properties: + key: + type: string + secret: + type: string + vaultOCID: + type: string + vaultSecretName: + type: string + type: object + dbUser: + properties: + key: + type: string + secret: + type: string + type: object + dbWallet: + properties: + key: + type: string + secret: + type: string + type: object + type: object + exporter: + properties: + deployment: + properties: + args: + items: + type: string + type: array + commands: + items: + type: string + type: array + env: + additionalProperties: + type: string + type: object + image: + type: string + labels: + additionalProperties: + type: string + type: object + podTemplate: + properties: + labels: + additionalProperties: + type: string + type: object + securityContext: + properties: + appArmorProfile: + properties: + localhostProfile: + type: string + type: + type: string + required: + - type + type: object + fsGroup: + format: int64 + type: integer + fsGroupChangePolicy: + type: string + runAsGroup: + format: int64 + type: integer + runAsNonRoot: + type: boolean + runAsUser: + format: int64 + type: integer + seLinuxOptions: + properties: + level: + type: string + role: + type: string + type: + type: string + user: + type: string + type: object + seccompProfile: + properties: + localhostProfile: + type: string + type: + type: string + required: + - type + type: object + supplementalGroups: + items: + format: int64 + type: integer + type: array + x-kubernetes-list-type: atomic + supplementalGroupsPolicy: + type: string + sysctls: + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + x-kubernetes-list-type: atomic + windowsOptions: + properties: + gmsaCredentialSpec: + type: string + gmsaCredentialSpecName: + type: string + hostProcess: + type: boolean + runAsUserName: + type: string + type: object + type: object + type: object + securityContext: + properties: + allowPrivilegeEscalation: + type: boolean + appArmorProfile: + properties: + localhostProfile: + type: string + type: + type: string + required: + - type + type: object + capabilities: + properties: + add: + items: + type: string + type: array + x-kubernetes-list-type: atomic + drop: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + privileged: + type: boolean + procMount: + type: string + readOnlyRootFilesystem: + type: boolean + runAsGroup: + format: int64 + type: integer + runAsNonRoot: + type: boolean + runAsUser: + format: int64 + type: integer + seLinuxOptions: + properties: + level: + type: string + role: + type: string + type: + type: string + user: + type: string + type: object + seccompProfile: + properties: + localhostProfile: + type: string + type: + type: string + required: + - type + type: object + windowsOptions: + properties: + gmsaCredentialSpec: + type: string + gmsaCredentialSpecName: + type: string + hostProcess: + type: boolean + runAsUserName: + type: string + type: object + type: object + type: object + service: + properties: + labels: + additionalProperties: + type: string + type: object + ports: + items: + properties: + appProtocol: + type: string + name: + type: string + nodePort: + format: int32 + type: integer + port: + format: int32 + type: integer + protocol: + default: TCP + type: string + targetPort: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + required: + - port + type: object + type: array + type: object + type: object + inheritLabels: + items: + type: string + type: array + log: + properties: + filename: + type: string + path: + type: string + volume: + properties: + name: + type: string + persistentVolumeClaim: + properties: + claimName: + type: string + type: object + type: object + type: object + ociConfig: + properties: + configMapName: + type: string + secretName: + type: string + type: object + prometheus: + properties: + serviceMonitor: + properties: + endpoints: + items: + properties: + authorization: + properties: + credentials: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: + type: string + type: object + basicAuth: + properties: + password: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + username: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: object + bearerTokenFile: + type: string + bearerTokenSecret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + enableHttp2: + type: boolean + filterRunning: + type: boolean + followRedirects: + type: boolean + honorLabels: + type: boolean + honorTimestamps: + type: boolean + interval: + pattern: ^(0|(([0-9]+)y)?(([0-9]+)w)?(([0-9]+)d)?(([0-9]+)h)?(([0-9]+)m)?(([0-9]+)s)?(([0-9]+)ms)?)$ + type: string + metricRelabelings: + items: + properties: + action: + default: replace + enum: + - replace + - Replace + - keep + - Keep + - drop + - Drop + - hashmod + - HashMod + - labelmap + - LabelMap + - labeldrop + - LabelDrop + - labelkeep + - LabelKeep + - lowercase + - Lowercase + - uppercase + - Uppercase + - keepequal + - KeepEqual + - dropequal + - DropEqual + type: string + modulus: + format: int64 + type: integer + regex: + type: string + replacement: + type: string + separator: + type: string + sourceLabels: + items: + pattern: ^[a-zA-Z_][a-zA-Z0-9_]*$ + type: string + type: array + targetLabel: + type: string + type: object + type: array + oauth2: + properties: + clientId: + properties: + configMap: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + secret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: object + clientSecret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + endpointParams: + additionalProperties: + type: string + type: object + noProxy: + type: string + proxyConnectHeader: + additionalProperties: + items: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: array + type: object + x-kubernetes-map-type: atomic + proxyFromEnvironment: + type: boolean + proxyUrl: + pattern: ^http(s)?://.+$ + type: string + scopes: + items: + type: string + type: array + tlsConfig: + properties: + ca: + properties: + configMap: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + secret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: object + cert: + properties: + configMap: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + secret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: object + insecureSkipVerify: + type: boolean + keySecret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + maxVersion: + enum: + - TLS10 + - TLS11 + - TLS12 + - TLS13 + type: string + minVersion: + enum: + - TLS10 + - TLS11 + - TLS12 + - TLS13 + type: string + serverName: + type: string + type: object + tokenUrl: + minLength: 1 + type: string + required: + - clientId + - clientSecret + - tokenUrl + type: object + params: + additionalProperties: + items: + type: string + type: array + type: object + path: + type: string + port: + type: string + proxyUrl: + type: string + relabelings: + items: + properties: + action: + default: replace + enum: + - replace + - Replace + - keep + - Keep + - drop + - Drop + - hashmod + - HashMod + - labelmap + - LabelMap + - labeldrop + - LabelDrop + - labelkeep + - LabelKeep + - lowercase + - Lowercase + - uppercase + - Uppercase + - keepequal + - KeepEqual + - dropequal + - DropEqual + type: string + modulus: + format: int64 + type: integer + regex: + type: string + replacement: + type: string + separator: + type: string + sourceLabels: + items: + pattern: ^[a-zA-Z_][a-zA-Z0-9_]*$ + type: string + type: array + targetLabel: + type: string + type: object + type: array + scheme: + enum: + - http + - https + type: string + scrapeTimeout: + pattern: ^(0|(([0-9]+)y)?(([0-9]+)w)?(([0-9]+)d)?(([0-9]+)h)?(([0-9]+)m)?(([0-9]+)s)?(([0-9]+)ms)?)$ + type: string + targetPort: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + tlsConfig: + properties: + ca: + properties: + configMap: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + secret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: object + caFile: + type: string + cert: + properties: + configMap: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + secret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: object + certFile: + type: string + insecureSkipVerify: + type: boolean + keyFile: + type: string + keySecret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + maxVersion: + enum: + - TLS10 + - TLS11 + - TLS12 + - TLS13 + type: string + minVersion: + enum: + - TLS10 + - TLS11 + - TLS12 + - TLS13 + type: string + serverName: + type: string + type: object + trackTimestampsStaleness: + type: boolean + type: object + type: array + labels: + additionalProperties: + type: string + type: object + namespaceSelector: + properties: + any: + type: boolean + matchNames: + items: + type: string + type: array + type: object + type: object + type: object + replicas: + format: int32 + type: integer + sidecarVolumes: + items: + properties: + awsElasticBlockStore: + properties: + fsType: + type: string + partition: + format: int32 + type: integer + readOnly: + type: boolean + volumeID: + type: string + required: + - volumeID + type: object + azureDisk: + properties: + cachingMode: + type: string + diskName: + type: string + diskURI: + type: string + fsType: + default: ext4 + type: string + kind: + type: string + readOnly: + default: false + type: boolean + required: + - diskName + - diskURI + type: object + azureFile: + properties: + readOnly: + type: boolean + secretName: + type: string + shareName: + type: string + required: + - secretName + - shareName + type: object + cephfs: + properties: + monitors: + items: + type: string + type: array + x-kubernetes-list-type: atomic + path: + type: string + readOnly: + type: boolean + secretFile: + type: string + secretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + user: + type: string + required: + - monitors + type: object + cinder: + properties: + fsType: + type: string + readOnly: + type: boolean + secretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + volumeID: + type: string + required: + - volumeID + type: object + configMap: + properties: + defaultMode: + format: int32 + type: integer + items: + items: + properties: + key: + type: string + mode: + format: int32 + type: integer + path: + type: string + required: + - key + - path + type: object + type: array + x-kubernetes-list-type: atomic + name: + default: "" + type: string + optional: + type: boolean + type: object + x-kubernetes-map-type: atomic + csi: + properties: + driver: + type: string + fsType: + type: string + nodePublishSecretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + readOnly: + type: boolean + volumeAttributes: + additionalProperties: + type: string + type: object + required: + - driver + type: object + downwardAPI: + properties: + defaultMode: + format: int32 + type: integer + items: + items: + properties: + fieldRef: + properties: + apiVersion: + type: string + fieldPath: + type: string + required: + - fieldPath + type: object + x-kubernetes-map-type: atomic + mode: + format: int32 + type: integer + path: + type: string + resourceFieldRef: + properties: + containerName: + type: string + divisor: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + resource: + type: string + required: + - resource + type: object + x-kubernetes-map-type: atomic + required: + - path + type: object + type: array + x-kubernetes-list-type: atomic + type: object + emptyDir: + properties: + medium: + type: string + sizeLimit: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + ephemeral: + properties: + volumeClaimTemplate: + properties: + metadata: + type: object + spec: + properties: + accessModes: + items: + type: string + type: array + x-kubernetes-list-type: atomic + dataSource: + properties: + apiGroup: + type: string + kind: + type: string + name: + type: string + required: + - kind + - name + type: object + x-kubernetes-map-type: atomic + dataSourceRef: + properties: + apiGroup: + type: string + kind: + type: string + name: + type: string + namespace: + type: string + required: + - kind + - name + type: object + resources: + properties: + limits: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + requests: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + type: object + selector: + properties: + matchExpressions: + items: + properties: + key: + type: string + operator: + type: string + values: + items: + type: string + type: array + x-kubernetes-list-type: atomic + required: + - key + - operator + type: object + type: array + x-kubernetes-list-type: atomic + matchLabels: + additionalProperties: + type: string + type: object + type: object + x-kubernetes-map-type: atomic + storageClassName: + type: string + volumeAttributesClassName: + type: string + volumeMode: + type: string + volumeName: + type: string + type: object + required: + - spec + type: object + type: object + fc: + properties: + fsType: + type: string + lun: + format: int32 + type: integer + readOnly: + type: boolean + targetWWNs: + items: + type: string + type: array + x-kubernetes-list-type: atomic + wwids: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + flexVolume: + properties: + driver: + type: string + fsType: + type: string + options: + additionalProperties: + type: string + type: object + readOnly: + type: boolean + secretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + required: + - driver + type: object + flocker: + properties: + datasetName: + type: string + datasetUUID: + type: string + type: object + gcePersistentDisk: + properties: + fsType: + type: string + partition: + format: int32 + type: integer + pdName: + type: string + readOnly: + type: boolean + required: + - pdName + type: object + gitRepo: + properties: + directory: + type: string + repository: + type: string + revision: + type: string + required: + - repository + type: object + glusterfs: + properties: + endpoints: + type: string + path: + type: string + readOnly: + type: boolean + required: + - endpoints + - path + type: object + hostPath: + properties: + path: + type: string + type: + type: string + required: + - path + type: object + image: + properties: + pullPolicy: + type: string + reference: + type: string + type: object + iscsi: + properties: + chapAuthDiscovery: + type: boolean + chapAuthSession: + type: boolean + fsType: + type: string + initiatorName: + type: string + iqn: + type: string + iscsiInterface: + default: default + type: string + lun: + format: int32 + type: integer + portals: + items: + type: string + type: array + x-kubernetes-list-type: atomic + readOnly: + type: boolean + secretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + targetPortal: + type: string + required: + - iqn + - lun + - targetPortal + type: object + name: + type: string + nfs: + properties: + path: + type: string + readOnly: + type: boolean + server: + type: string + required: + - path + - server + type: object + persistentVolumeClaim: + properties: + claimName: + type: string + readOnly: + type: boolean + required: + - claimName + type: object + photonPersistentDisk: + properties: + fsType: + type: string + pdID: + type: string + required: + - pdID + type: object + portworxVolume: + properties: + fsType: + type: string + readOnly: + type: boolean + volumeID: + type: string + required: + - volumeID + type: object + projected: + properties: + defaultMode: + format: int32 + type: integer + sources: + items: + properties: + clusterTrustBundle: + properties: + labelSelector: + properties: + matchExpressions: + items: + properties: + key: + type: string + operator: + type: string + values: + items: + type: string + type: array + x-kubernetes-list-type: atomic + required: + - key + - operator + type: object + type: array + x-kubernetes-list-type: atomic + matchLabels: + additionalProperties: + type: string + type: object + type: object + x-kubernetes-map-type: atomic + name: + type: string + optional: + type: boolean + path: + type: string + signerName: + type: string + required: + - path + type: object + configMap: + properties: + items: + items: + properties: + key: + type: string + mode: + format: int32 + type: integer + path: + type: string + required: + - key + - path + type: object + type: array + x-kubernetes-list-type: atomic + name: + default: "" + type: string + optional: + type: boolean + type: object + x-kubernetes-map-type: atomic + downwardAPI: + properties: + items: + items: + properties: + fieldRef: + properties: + apiVersion: + type: string + fieldPath: + type: string + required: + - fieldPath + type: object + x-kubernetes-map-type: atomic + mode: + format: int32 + type: integer + path: + type: string + resourceFieldRef: + properties: + containerName: + type: string + divisor: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + resource: + type: string + required: + - resource + type: object + x-kubernetes-map-type: atomic + required: + - path + type: object + type: array + x-kubernetes-list-type: atomic + type: object + secret: + properties: + items: + items: + properties: + key: + type: string + mode: + format: int32 + type: integer + path: + type: string + required: + - key + - path + type: object + type: array + x-kubernetes-list-type: atomic + name: + default: "" + type: string + optional: + type: boolean + type: object + x-kubernetes-map-type: atomic + serviceAccountToken: + properties: + audience: + type: string + expirationSeconds: + format: int64 + type: integer + path: + type: string + required: + - path + type: object + type: object + type: array + x-kubernetes-list-type: atomic + type: object + quobyte: + properties: + group: + type: string + readOnly: + type: boolean + registry: + type: string + tenant: + type: string + user: + type: string + volume: + type: string + required: + - registry + - volume + type: object + rbd: + properties: + fsType: + type: string + image: + type: string + keyring: + default: /etc/ceph/keyring + type: string + monitors: + items: + type: string + type: array + x-kubernetes-list-type: atomic + pool: + default: rbd + type: string + readOnly: + type: boolean + secretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + user: + default: admin + type: string + required: + - image + - monitors + type: object + scaleIO: + properties: + fsType: + default: xfs + type: string + gateway: + type: string + protectionDomain: + type: string + readOnly: + type: boolean + secretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + sslEnabled: + type: boolean + storageMode: + default: ThinProvisioned + type: string + storagePool: + type: string + system: + type: string + volumeName: + type: string + required: + - gateway + - secretRef + - system + type: object + secret: + properties: + defaultMode: + format: int32 + type: integer + items: + items: + properties: + key: + type: string + mode: + format: int32 + type: integer + path: + type: string + required: + - key + - path + type: object + type: array + x-kubernetes-list-type: atomic + optional: + type: boolean + secretName: + type: string + type: object + storageos: + properties: + fsType: + type: string + readOnly: + type: boolean + secretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + volumeName: + type: string + volumeNamespace: + type: string + type: object + vsphereVolume: + properties: + fsType: + type: string + storagePolicyID: + type: string + storagePolicyName: + type: string + volumePath: + type: string + required: + - volumePath + type: object + required: + - name + type: object + type: array + sidecars: + items: + properties: + args: + items: + type: string + type: array + x-kubernetes-list-type: atomic + command: + items: + type: string + type: array + x-kubernetes-list-type: atomic + env: + items: + properties: + name: + type: string + value: + type: string + valueFrom: + properties: + configMapKeyRef: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + fieldRef: + properties: + apiVersion: + type: string + fieldPath: + type: string + required: + - fieldPath + type: object + x-kubernetes-map-type: atomic + resourceFieldRef: + properties: + containerName: + type: string + divisor: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + resource: + type: string + required: + - resource + type: object + x-kubernetes-map-type: atomic + secretKeyRef: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: object + required: + - name + type: object + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + envFrom: + items: + properties: + configMapRef: + properties: + name: + default: "" + type: string + optional: + type: boolean + type: object + x-kubernetes-map-type: atomic + prefix: + type: string + secretRef: + properties: + name: + default: "" + type: string + optional: + type: boolean + type: object + x-kubernetes-map-type: atomic + type: object + type: array + x-kubernetes-list-type: atomic + image: + type: string + imagePullPolicy: + type: string + lifecycle: + properties: + postStart: + properties: + exec: + properties: + command: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + httpGet: + properties: + host: + type: string + httpHeaders: + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + x-kubernetes-list-type: atomic + path: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + scheme: + type: string + required: + - port + type: object + sleep: + properties: + seconds: + format: int64 + type: integer + required: + - seconds + type: object + tcpSocket: + properties: + host: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + required: + - port + type: object + type: object + preStop: + properties: + exec: + properties: + command: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + httpGet: + properties: + host: + type: string + httpHeaders: + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + x-kubernetes-list-type: atomic + path: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + scheme: + type: string + required: + - port + type: object + sleep: + properties: + seconds: + format: int64 + type: integer + required: + - seconds + type: object + tcpSocket: + properties: + host: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + required: + - port + type: object + type: object + type: object + livenessProbe: + properties: + exec: + properties: + command: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + failureThreshold: + format: int32 + type: integer + grpc: + properties: + port: + format: int32 + type: integer + service: + default: "" + type: string + required: + - port + type: object + httpGet: + properties: + host: + type: string + httpHeaders: + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + x-kubernetes-list-type: atomic + path: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + scheme: + type: string + required: + - port + type: object + initialDelaySeconds: + format: int32 + type: integer + periodSeconds: + format: int32 + type: integer + successThreshold: + format: int32 + type: integer + tcpSocket: + properties: + host: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + required: + - port + type: object + terminationGracePeriodSeconds: + format: int64 + type: integer + timeoutSeconds: + format: int32 + type: integer + type: object + name: + type: string + ports: + items: + properties: + containerPort: + format: int32 + type: integer + hostIP: + type: string + hostPort: + format: int32 + type: integer + name: + type: string + protocol: + default: TCP + type: string + required: + - containerPort + type: object + type: array + x-kubernetes-list-map-keys: + - containerPort + - protocol + x-kubernetes-list-type: map + readinessProbe: + properties: + exec: + properties: + command: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + failureThreshold: + format: int32 + type: integer + grpc: + properties: + port: + format: int32 + type: integer + service: + default: "" + type: string + required: + - port + type: object + httpGet: + properties: + host: + type: string + httpHeaders: + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + x-kubernetes-list-type: atomic + path: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + scheme: + type: string + required: + - port + type: object + initialDelaySeconds: + format: int32 + type: integer + periodSeconds: + format: int32 + type: integer + successThreshold: + format: int32 + type: integer + tcpSocket: + properties: + host: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + required: + - port + type: object + terminationGracePeriodSeconds: + format: int64 + type: integer + timeoutSeconds: + format: int32 + type: integer + type: object + resizePolicy: + items: + properties: + resourceName: + type: string + restartPolicy: + type: string + required: + - resourceName + - restartPolicy + type: object + type: array + x-kubernetes-list-type: atomic + resources: + properties: + claims: + items: + properties: + name: + type: string + request: + type: string + required: + - name + type: object + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + limits: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + requests: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + type: object + restartPolicy: + type: string + securityContext: + properties: + allowPrivilegeEscalation: + type: boolean + appArmorProfile: + properties: + localhostProfile: + type: string + type: + type: string + required: + - type + type: object + capabilities: + properties: + add: + items: + type: string + type: array + x-kubernetes-list-type: atomic + drop: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + privileged: + type: boolean + procMount: + type: string + readOnlyRootFilesystem: + type: boolean + runAsGroup: + format: int64 + type: integer + runAsNonRoot: + type: boolean + runAsUser: + format: int64 + type: integer + seLinuxOptions: + properties: + level: + type: string + role: + type: string + type: + type: string + user: + type: string + type: object + seccompProfile: + properties: + localhostProfile: + type: string + type: + type: string + required: + - type + type: object + windowsOptions: + properties: + gmsaCredentialSpec: + type: string + gmsaCredentialSpecName: + type: string + hostProcess: + type: boolean + runAsUserName: + type: string + type: object + type: object + startupProbe: + properties: + exec: + properties: + command: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + failureThreshold: + format: int32 + type: integer + grpc: + properties: + port: + format: int32 + type: integer + service: + default: "" + type: string + required: + - port + type: object + httpGet: + properties: + host: + type: string + httpHeaders: + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + x-kubernetes-list-type: atomic + path: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + scheme: + type: string + required: + - port + type: object + initialDelaySeconds: + format: int32 + type: integer + periodSeconds: + format: int32 + type: integer + successThreshold: + format: int32 + type: integer + tcpSocket: + properties: + host: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + required: + - port + type: object + terminationGracePeriodSeconds: + format: int64 + type: integer + timeoutSeconds: + format: int32 + type: integer + type: object + stdin: + type: boolean + stdinOnce: + type: boolean + terminationMessagePath: + type: string + terminationMessagePolicy: + type: string + tty: + type: boolean + volumeDevices: + items: + properties: + devicePath: + type: string + name: + type: string + required: + - devicePath + - name + type: object + type: array + x-kubernetes-list-map-keys: + - devicePath + x-kubernetes-list-type: map + volumeMounts: + items: + properties: + mountPath: + type: string + mountPropagation: + type: string + name: + type: string + readOnly: + type: boolean + recursiveReadOnly: + type: string + subPath: + type: string + subPathExpr: + type: string + required: + - mountPath + - name + type: object + type: array + x-kubernetes-list-map-keys: + - mountPath + x-kubernetes-list-type: map + workingDir: + type: string + required: + - name + type: object + type: array + type: object + status: + properties: + conditions: + items: + properties: + lastTransitionTime: + format: date-time + type: string + message: + maxLength: 32768 + type: string + observedGeneration: + format: int64 + minimum: 0 + type: integer + reason: + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + enum: + - "True" + - "False" + - Unknown + type: string + type: + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + exporterConfig: + type: string + replicas: + type: integer + status: + type: string + version: + type: string + required: + - conditions + - exporterConfig + - version + type: object + type: object + served: true + storage: false + subresources: + status: {} + - additionalPrinterColumns: + - jsonPath: .status.exporterConfig + name: ExporterConfig + type: string + - jsonPath: .status.status + name: Status + type: string + - jsonPath: .status.version + name: Version + type: string + name: v4 + schema: + openAPIV3Schema: + properties: + apiVersion: + type: string + kind: + type: string + metadata: + type: object + spec: + properties: + configuration: + properties: + configMap: + properties: + key: + type: string + name: + type: string + type: object + type: object + database: + properties: + dbConnectionString: + properties: + key: + type: string + secret: + type: string + type: object + dbPassword: + properties: + key: + type: string + secret: + type: string + vaultOCID: + type: string + vaultSecretName: + type: string + type: object + dbUser: + properties: + key: + type: string + secret: + type: string + type: object + dbWallet: + properties: + key: + type: string + secret: + type: string + type: object + type: object + exporter: + properties: + deployment: + properties: + args: + items: + type: string + type: array + commands: + items: + type: string + type: array + env: + additionalProperties: + type: string + type: object + image: + type: string + labels: + additionalProperties: + type: string + type: object + podTemplate: + properties: + labels: + additionalProperties: + type: string + type: object + securityContext: + properties: + appArmorProfile: + properties: + localhostProfile: + type: string + type: + type: string + required: + - type + type: object + fsGroup: + format: int64 + type: integer + fsGroupChangePolicy: + type: string + runAsGroup: + format: int64 + type: integer + runAsNonRoot: + type: boolean + runAsUser: + format: int64 + type: integer + seLinuxOptions: + properties: + level: + type: string + role: + type: string + type: + type: string + user: + type: string + type: object + seccompProfile: + properties: + localhostProfile: + type: string + type: + type: string + required: + - type + type: object + supplementalGroups: + items: + format: int64 + type: integer + type: array + x-kubernetes-list-type: atomic + supplementalGroupsPolicy: + type: string + sysctls: + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + x-kubernetes-list-type: atomic + windowsOptions: + properties: + gmsaCredentialSpec: + type: string + gmsaCredentialSpecName: + type: string + hostProcess: + type: boolean + runAsUserName: + type: string + type: object + type: object + type: object + securityContext: + properties: + allowPrivilegeEscalation: + type: boolean + appArmorProfile: + properties: + localhostProfile: + type: string + type: + type: string + required: + - type + type: object + capabilities: + properties: + add: + items: + type: string + type: array + x-kubernetes-list-type: atomic + drop: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + privileged: + type: boolean + procMount: + type: string + readOnlyRootFilesystem: + type: boolean + runAsGroup: + format: int64 + type: integer + runAsNonRoot: + type: boolean + runAsUser: + format: int64 + type: integer + seLinuxOptions: + properties: + level: + type: string + role: + type: string + type: + type: string + user: + type: string + type: object + seccompProfile: + properties: + localhostProfile: + type: string + type: + type: string + required: + - type + type: object + windowsOptions: + properties: + gmsaCredentialSpec: + type: string + gmsaCredentialSpecName: + type: string + hostProcess: + type: boolean + runAsUserName: + type: string + type: object + type: object + type: object + service: + properties: + labels: + additionalProperties: + type: string + type: object + ports: + items: + properties: + appProtocol: + type: string + name: + type: string + nodePort: + format: int32 + type: integer + port: + format: int32 + type: integer + protocol: + default: TCP + type: string + targetPort: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + required: + - port + type: object + type: array + type: object + type: object + inheritLabels: + items: + type: string + type: array + log: + properties: + filename: + type: string + path: + type: string + volume: + properties: + name: + type: string + persistentVolumeClaim: + properties: + claimName: + type: string + type: object + type: object + type: object + ociConfig: + properties: + configMapName: + type: string + secretName: + type: string + type: object + prometheus: + properties: + serviceMonitor: + properties: + endpoints: + items: + properties: + authorization: + properties: + credentials: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: + type: string + type: object + basicAuth: + properties: + password: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + username: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: object + bearerTokenFile: + type: string + bearerTokenSecret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + enableHttp2: + type: boolean + filterRunning: + type: boolean + followRedirects: + type: boolean + honorLabels: + type: boolean + honorTimestamps: + type: boolean + interval: + pattern: ^(0|(([0-9]+)y)?(([0-9]+)w)?(([0-9]+)d)?(([0-9]+)h)?(([0-9]+)m)?(([0-9]+)s)?(([0-9]+)ms)?)$ + type: string + metricRelabelings: + items: + properties: + action: + default: replace + enum: + - replace + - Replace + - keep + - Keep + - drop + - Drop + - hashmod + - HashMod + - labelmap + - LabelMap + - labeldrop + - LabelDrop + - labelkeep + - LabelKeep + - lowercase + - Lowercase + - uppercase + - Uppercase + - keepequal + - KeepEqual + - dropequal + - DropEqual + type: string + modulus: + format: int64 + type: integer + regex: + type: string + replacement: + type: string + separator: + type: string + sourceLabels: + items: + pattern: ^[a-zA-Z_][a-zA-Z0-9_]*$ + type: string + type: array + targetLabel: + type: string + type: object + type: array + oauth2: + properties: + clientId: + properties: + configMap: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + secret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: object + clientSecret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + endpointParams: + additionalProperties: + type: string + type: object + noProxy: + type: string + proxyConnectHeader: + additionalProperties: + items: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: array + type: object + x-kubernetes-map-type: atomic + proxyFromEnvironment: + type: boolean + proxyUrl: + pattern: ^http(s)?://.+$ + type: string + scopes: + items: + type: string + type: array + tlsConfig: + properties: + ca: + properties: + configMap: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + secret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: object + cert: + properties: + configMap: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + secret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: object + insecureSkipVerify: + type: boolean + keySecret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + maxVersion: + enum: + - TLS10 + - TLS11 + - TLS12 + - TLS13 + type: string + minVersion: + enum: + - TLS10 + - TLS11 + - TLS12 + - TLS13 + type: string + serverName: + type: string + type: object + tokenUrl: + minLength: 1 + type: string + required: + - clientId + - clientSecret + - tokenUrl + type: object + params: + additionalProperties: + items: + type: string + type: array + type: object + path: + type: string + port: + type: string + proxyUrl: + type: string + relabelings: + items: + properties: + action: + default: replace + enum: + - replace + - Replace + - keep + - Keep + - drop + - Drop + - hashmod + - HashMod + - labelmap + - LabelMap + - labeldrop + - LabelDrop + - labelkeep + - LabelKeep + - lowercase + - Lowercase + - uppercase + - Uppercase + - keepequal + - KeepEqual + - dropequal + - DropEqual + type: string + modulus: + format: int64 + type: integer + regex: + type: string + replacement: + type: string + separator: + type: string + sourceLabels: + items: + pattern: ^[a-zA-Z_][a-zA-Z0-9_]*$ + type: string + type: array + targetLabel: + type: string + type: object + type: array + scheme: + enum: + - http + - https + type: string + scrapeTimeout: + pattern: ^(0|(([0-9]+)y)?(([0-9]+)w)?(([0-9]+)d)?(([0-9]+)h)?(([0-9]+)m)?(([0-9]+)s)?(([0-9]+)ms)?)$ + type: string + targetPort: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + tlsConfig: + properties: + ca: + properties: + configMap: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + secret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: object + caFile: + type: string + cert: + properties: + configMap: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + secret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: object + certFile: + type: string + insecureSkipVerify: + type: boolean + keyFile: + type: string + keySecret: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + maxVersion: + enum: + - TLS10 + - TLS11 + - TLS12 + - TLS13 + type: string + minVersion: + enum: + - TLS10 + - TLS11 + - TLS12 + - TLS13 + type: string + serverName: + type: string + type: object + trackTimestampsStaleness: + type: boolean + type: object + type: array + labels: + additionalProperties: + type: string + type: object + namespaceSelector: + properties: + any: + type: boolean + matchNames: + items: + type: string + type: array + type: object + type: object + type: object + replicas: + format: int32 + type: integer + sidecarVolumes: + items: + properties: + awsElasticBlockStore: + properties: + fsType: + type: string + partition: + format: int32 + type: integer + readOnly: + type: boolean + volumeID: + type: string + required: + - volumeID + type: object + azureDisk: + properties: + cachingMode: + type: string + diskName: + type: string + diskURI: + type: string + fsType: + default: ext4 + type: string + kind: + type: string + readOnly: + default: false + type: boolean + required: + - diskName + - diskURI + type: object + azureFile: + properties: + readOnly: + type: boolean + secretName: + type: string + shareName: + type: string + required: + - secretName + - shareName + type: object + cephfs: + properties: + monitors: + items: + type: string + type: array + x-kubernetes-list-type: atomic + path: + type: string + readOnly: + type: boolean + secretFile: + type: string + secretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + user: + type: string + required: + - monitors + type: object + cinder: + properties: + fsType: + type: string + readOnly: + type: boolean + secretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + volumeID: + type: string + required: + - volumeID + type: object + configMap: + properties: + defaultMode: + format: int32 + type: integer + items: + items: + properties: + key: + type: string + mode: + format: int32 + type: integer + path: + type: string + required: + - key + - path + type: object + type: array + x-kubernetes-list-type: atomic + name: + default: "" + type: string + optional: + type: boolean + type: object + x-kubernetes-map-type: atomic + csi: + properties: + driver: + type: string + fsType: + type: string + nodePublishSecretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + readOnly: + type: boolean + volumeAttributes: + additionalProperties: + type: string + type: object + required: + - driver + type: object + downwardAPI: + properties: + defaultMode: + format: int32 + type: integer + items: + items: + properties: + fieldRef: + properties: + apiVersion: + type: string + fieldPath: + type: string + required: + - fieldPath + type: object + x-kubernetes-map-type: atomic + mode: + format: int32 + type: integer + path: + type: string + resourceFieldRef: + properties: + containerName: + type: string + divisor: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + resource: + type: string + required: + - resource + type: object + x-kubernetes-map-type: atomic + required: + - path + type: object + type: array + x-kubernetes-list-type: atomic + type: object + emptyDir: + properties: + medium: + type: string + sizeLimit: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + ephemeral: + properties: + volumeClaimTemplate: + properties: + metadata: + type: object + spec: + properties: + accessModes: + items: + type: string + type: array + x-kubernetes-list-type: atomic + dataSource: + properties: + apiGroup: + type: string + kind: + type: string + name: + type: string + required: + - kind + - name + type: object + x-kubernetes-map-type: atomic + dataSourceRef: + properties: + apiGroup: + type: string + kind: + type: string + name: + type: string + namespace: + type: string + required: + - kind + - name + type: object + resources: + properties: + limits: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + requests: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + type: object + selector: + properties: + matchExpressions: + items: + properties: + key: + type: string + operator: + type: string + values: + items: + type: string + type: array + x-kubernetes-list-type: atomic + required: + - key + - operator + type: object + type: array + x-kubernetes-list-type: atomic + matchLabels: + additionalProperties: + type: string + type: object + type: object + x-kubernetes-map-type: atomic + storageClassName: + type: string + volumeAttributesClassName: + type: string + volumeMode: + type: string + volumeName: + type: string + type: object + required: + - spec + type: object + type: object + fc: + properties: + fsType: + type: string + lun: + format: int32 + type: integer + readOnly: + type: boolean + targetWWNs: + items: + type: string + type: array + x-kubernetes-list-type: atomic + wwids: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + flexVolume: + properties: + driver: + type: string + fsType: + type: string + options: + additionalProperties: + type: string + type: object + readOnly: + type: boolean + secretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + required: + - driver + type: object + flocker: + properties: + datasetName: + type: string + datasetUUID: + type: string + type: object + gcePersistentDisk: + properties: + fsType: + type: string + partition: + format: int32 + type: integer + pdName: + type: string + readOnly: + type: boolean + required: + - pdName + type: object + gitRepo: + properties: + directory: + type: string + repository: + type: string + revision: + type: string + required: + - repository + type: object + glusterfs: + properties: + endpoints: + type: string + path: + type: string + readOnly: + type: boolean + required: + - endpoints + - path + type: object + hostPath: + properties: + path: + type: string + type: + type: string + required: + - path + type: object + image: + properties: + pullPolicy: + type: string + reference: + type: string + type: object + iscsi: + properties: + chapAuthDiscovery: + type: boolean + chapAuthSession: + type: boolean + fsType: + type: string + initiatorName: + type: string + iqn: + type: string + iscsiInterface: + default: default + type: string + lun: + format: int32 + type: integer + portals: + items: + type: string + type: array + x-kubernetes-list-type: atomic + readOnly: + type: boolean + secretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + targetPortal: + type: string + required: + - iqn + - lun + - targetPortal + type: object + name: + type: string + nfs: + properties: + path: + type: string + readOnly: + type: boolean + server: + type: string + required: + - path + - server + type: object + persistentVolumeClaim: + properties: + claimName: + type: string + readOnly: + type: boolean + required: + - claimName + type: object + photonPersistentDisk: + properties: + fsType: + type: string + pdID: + type: string + required: + - pdID + type: object + portworxVolume: + properties: + fsType: + type: string + readOnly: + type: boolean + volumeID: + type: string + required: + - volumeID + type: object + projected: + properties: + defaultMode: + format: int32 + type: integer + sources: + items: + properties: + clusterTrustBundle: + properties: + labelSelector: + properties: + matchExpressions: + items: + properties: + key: + type: string + operator: + type: string + values: + items: + type: string + type: array + x-kubernetes-list-type: atomic + required: + - key + - operator + type: object + type: array + x-kubernetes-list-type: atomic + matchLabels: + additionalProperties: + type: string + type: object + type: object + x-kubernetes-map-type: atomic + name: + type: string + optional: + type: boolean + path: + type: string + signerName: + type: string + required: + - path + type: object + configMap: + properties: + items: + items: + properties: + key: + type: string + mode: + format: int32 + type: integer + path: + type: string + required: + - key + - path + type: object + type: array + x-kubernetes-list-type: atomic + name: + default: "" + type: string + optional: + type: boolean + type: object + x-kubernetes-map-type: atomic + downwardAPI: + properties: + items: + items: + properties: + fieldRef: + properties: + apiVersion: + type: string + fieldPath: + type: string + required: + - fieldPath + type: object + x-kubernetes-map-type: atomic + mode: + format: int32 + type: integer + path: + type: string + resourceFieldRef: + properties: + containerName: + type: string + divisor: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + resource: + type: string + required: + - resource + type: object + x-kubernetes-map-type: atomic + required: + - path + type: object + type: array + x-kubernetes-list-type: atomic + type: object + secret: + properties: + items: + items: + properties: + key: + type: string + mode: + format: int32 + type: integer + path: + type: string + required: + - key + - path + type: object + type: array + x-kubernetes-list-type: atomic + name: + default: "" + type: string + optional: + type: boolean + type: object + x-kubernetes-map-type: atomic + serviceAccountToken: + properties: + audience: + type: string + expirationSeconds: + format: int64 + type: integer + path: + type: string + required: + - path + type: object + type: object + type: array + x-kubernetes-list-type: atomic + type: object + quobyte: + properties: + group: + type: string + readOnly: + type: boolean + registry: + type: string + tenant: + type: string + user: + type: string + volume: + type: string + required: + - registry + - volume + type: object + rbd: + properties: + fsType: + type: string + image: + type: string + keyring: + default: /etc/ceph/keyring + type: string + monitors: + items: + type: string + type: array + x-kubernetes-list-type: atomic + pool: + default: rbd + type: string + readOnly: + type: boolean + secretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + user: + default: admin + type: string + required: + - image + - monitors + type: object + scaleIO: + properties: + fsType: + default: xfs + type: string + gateway: + type: string + protectionDomain: + type: string + readOnly: + type: boolean + secretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + sslEnabled: + type: boolean + storageMode: + default: ThinProvisioned + type: string + storagePool: + type: string + system: + type: string + volumeName: + type: string + required: + - gateway + - secretRef + - system + type: object + secret: + properties: + defaultMode: + format: int32 + type: integer + items: + items: + properties: + key: + type: string + mode: + format: int32 + type: integer + path: + type: string + required: + - key + - path + type: object + type: array + x-kubernetes-list-type: atomic + optional: + type: boolean + secretName: + type: string + type: object + storageos: + properties: + fsType: + type: string + readOnly: + type: boolean + secretRef: + properties: + name: + default: "" + type: string + type: object + x-kubernetes-map-type: atomic + volumeName: + type: string + volumeNamespace: + type: string + type: object + vsphereVolume: + properties: + fsType: + type: string + storagePolicyID: + type: string + storagePolicyName: + type: string + volumePath: + type: string + required: + - volumePath + type: object + required: + - name + type: object + type: array + sidecars: + items: + properties: + args: + items: + type: string + type: array + x-kubernetes-list-type: atomic + command: + items: + type: string + type: array + x-kubernetes-list-type: atomic + env: + items: + properties: + name: + type: string + value: + type: string + valueFrom: + properties: + configMapKeyRef: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + fieldRef: + properties: + apiVersion: + type: string + fieldPath: + type: string + required: + - fieldPath + type: object + x-kubernetes-map-type: atomic + resourceFieldRef: + properties: + containerName: + type: string + divisor: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + resource: + type: string + required: + - resource + type: object + x-kubernetes-map-type: atomic + secretKeyRef: + properties: + key: + type: string + name: + default: "" + type: string + optional: + type: boolean + required: + - key + type: object + x-kubernetes-map-type: atomic + type: object + required: + - name + type: object + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + envFrom: + items: + properties: + configMapRef: + properties: + name: + default: "" + type: string + optional: + type: boolean + type: object + x-kubernetes-map-type: atomic + prefix: + type: string + secretRef: + properties: + name: + default: "" + type: string + optional: + type: boolean + type: object + x-kubernetes-map-type: atomic + type: object + type: array + x-kubernetes-list-type: atomic + image: + type: string + imagePullPolicy: + type: string + lifecycle: + properties: + postStart: + properties: + exec: + properties: + command: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + httpGet: + properties: + host: + type: string + httpHeaders: + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + x-kubernetes-list-type: atomic + path: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + scheme: + type: string + required: + - port + type: object + sleep: + properties: + seconds: + format: int64 + type: integer + required: + - seconds + type: object + tcpSocket: + properties: + host: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + required: + - port + type: object + type: object + preStop: + properties: + exec: + properties: + command: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + httpGet: + properties: + host: + type: string + httpHeaders: + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + x-kubernetes-list-type: atomic + path: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + scheme: + type: string + required: + - port + type: object + sleep: + properties: + seconds: + format: int64 + type: integer + required: + - seconds + type: object + tcpSocket: + properties: + host: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + required: + - port + type: object + type: object + type: object + livenessProbe: + properties: + exec: + properties: + command: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + failureThreshold: + format: int32 + type: integer + grpc: + properties: + port: + format: int32 + type: integer + service: + default: "" + type: string + required: + - port + type: object + httpGet: + properties: + host: + type: string + httpHeaders: + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + x-kubernetes-list-type: atomic + path: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + scheme: + type: string + required: + - port + type: object + initialDelaySeconds: + format: int32 + type: integer + periodSeconds: + format: int32 + type: integer + successThreshold: + format: int32 + type: integer + tcpSocket: + properties: + host: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + required: + - port + type: object + terminationGracePeriodSeconds: + format: int64 + type: integer + timeoutSeconds: + format: int32 + type: integer + type: object + name: + type: string + ports: + items: + properties: + containerPort: + format: int32 + type: integer + hostIP: + type: string + hostPort: + format: int32 + type: integer + name: + type: string + protocol: + default: TCP + type: string + required: + - containerPort + type: object + type: array + x-kubernetes-list-map-keys: + - containerPort + - protocol + x-kubernetes-list-type: map + readinessProbe: + properties: + exec: + properties: + command: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + failureThreshold: + format: int32 + type: integer + grpc: + properties: + port: + format: int32 + type: integer + service: + default: "" + type: string + required: + - port + type: object + httpGet: + properties: + host: + type: string + httpHeaders: + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + x-kubernetes-list-type: atomic + path: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + scheme: + type: string + required: + - port + type: object + initialDelaySeconds: + format: int32 + type: integer + periodSeconds: + format: int32 + type: integer + successThreshold: + format: int32 + type: integer + tcpSocket: + properties: + host: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + required: + - port + type: object + terminationGracePeriodSeconds: + format: int64 + type: integer + timeoutSeconds: + format: int32 + type: integer + type: object + resizePolicy: + items: + properties: + resourceName: + type: string + restartPolicy: + type: string + required: + - resourceName + - restartPolicy + type: object + type: array + x-kubernetes-list-type: atomic + resources: + properties: + claims: + items: + properties: + name: + type: string + request: + type: string + required: + - name + type: object + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + limits: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + requests: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + type: object + restartPolicy: + type: string + securityContext: + properties: + allowPrivilegeEscalation: + type: boolean + appArmorProfile: + properties: + localhostProfile: + type: string + type: + type: string + required: + - type + type: object + capabilities: + properties: + add: + items: + type: string + type: array + x-kubernetes-list-type: atomic + drop: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + privileged: + type: boolean + procMount: + type: string + readOnlyRootFilesystem: + type: boolean + runAsGroup: + format: int64 + type: integer + runAsNonRoot: + type: boolean + runAsUser: + format: int64 + type: integer + seLinuxOptions: + properties: + level: + type: string + role: + type: string + type: + type: string + user: + type: string + type: object + seccompProfile: + properties: + localhostProfile: + type: string + type: + type: string + required: + - type + type: object + windowsOptions: + properties: + gmsaCredentialSpec: + type: string + gmsaCredentialSpecName: + type: string + hostProcess: + type: boolean + runAsUserName: + type: string + type: object + type: object + startupProbe: + properties: + exec: + properties: + command: + items: + type: string + type: array + x-kubernetes-list-type: atomic + type: object + failureThreshold: + format: int32 + type: integer + grpc: + properties: + port: + format: int32 + type: integer + service: + default: "" + type: string + required: + - port + type: object + httpGet: + properties: + host: + type: string + httpHeaders: + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + x-kubernetes-list-type: atomic + path: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + scheme: + type: string + required: + - port + type: object + initialDelaySeconds: + format: int32 + type: integer + periodSeconds: + format: int32 + type: integer + successThreshold: + format: int32 + type: integer + tcpSocket: + properties: + host: + type: string + port: + anyOf: + - type: integer + - type: string + x-kubernetes-int-or-string: true + required: + - port + type: object + terminationGracePeriodSeconds: + format: int64 + type: integer + timeoutSeconds: + format: int32 + type: integer + type: object + stdin: + type: boolean + stdinOnce: + type: boolean + terminationMessagePath: + type: string + terminationMessagePolicy: + type: string + tty: + type: boolean + volumeDevices: + items: + properties: + devicePath: + type: string + name: + type: string + required: + - devicePath + - name + type: object + type: array + x-kubernetes-list-map-keys: + - devicePath + x-kubernetes-list-type: map + volumeMounts: + items: + properties: + mountPath: + type: string + mountPropagation: + type: string + name: + type: string + readOnly: + type: boolean + recursiveReadOnly: + type: string + subPath: + type: string + subPathExpr: + type: string + required: + - mountPath + - name + type: object + type: array + x-kubernetes-list-map-keys: + - mountPath + x-kubernetes-list-type: map + workingDir: + type: string + required: + - name + type: object + type: array + type: object + status: + properties: + conditions: + items: + properties: + lastTransitionTime: + format: date-time + type: string + message: + maxLength: 32768 + type: string + observedGeneration: + format: int64 + minimum: 0 + type: integer + reason: + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + enum: + - "True" + - "False" + - Unknown + type: string + type: + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + exporterConfig: + type: string + replicas: + type: integer + status: + type: string + version: + type: string + required: + - conditions + - exporterConfig + - version + type: object + type: object + served: true + storage: true + subresources: + status: {} +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.16.5 + name: dataguardbrokers.database.oracle.com +spec: + group: database.oracle.com + names: + kind: DataguardBroker + listKind: DataguardBrokerList + plural: dataguardbrokers + singular: dataguardbroker + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .status.primaryDatabase + name: Primary + type: string + - jsonPath: .status.standbyDatabases + name: Standbys + type: string + - jsonPath: .spec.protectionMode + name: Protection Mode + type: string + - jsonPath: .status.clusterConnectString + name: Cluster Connect Str + priority: 1 + type: string + - jsonPath: .status.externalConnectString + name: Connect Str + type: string + - jsonPath: .spec.primaryDatabaseRef + name: Primary Database + priority: 1 + type: string + - jsonPath: .status.status + name: Status + type: string + - jsonPath: .status.fastStartFailover + name: FSFO + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + properties: + apiVersion: + type: string + kind: + type: string + metadata: + type: object + spec: + properties: + fastStartFailover: + type: boolean + loadBalancer: + type: boolean + nodeSelector: + additionalProperties: + type: string + type: object + primaryDatabaseRef: + type: string + protectionMode: + enum: + - MaxPerformance + - MaxAvailability + type: string + serviceAnnotations: + additionalProperties: + type: string + type: object + setAsPrimaryDatabase: + type: string + standbyDatabaseRefs: + items: + type: string + type: array + required: + - primaryDatabaseRef + - protectionMode + - standbyDatabaseRefs + type: object + status: + properties: + clusterConnectString: + type: string + databasesInDataguardConfig: + additionalProperties: + type: string + type: object + externalConnectString: + type: string + fastStartFailover: + type: boolean + primaryDatabase: + type: string + primaryDatabaseRef: + type: string + protectionMode: + type: string + standbyDatabases: + type: string + status: + type: string + type: object + type: object + served: true + storage: false + subresources: + status: {} + - additionalPrinterColumns: + - jsonPath: .status.primaryDatabase + name: Primary + type: string + - jsonPath: .status.standbyDatabases + name: Standbys + type: string + - jsonPath: .spec.protectionMode + name: Protection Mode + type: string + - jsonPath: .status.clusterConnectString + name: Cluster Connect Str + priority: 1 + type: string + - jsonPath: .status.externalConnectString + name: Connect Str + type: string + - jsonPath: .spec.primaryDatabaseRef + name: Primary Database + priority: 1 + type: string + - jsonPath: .status.status + name: Status + type: string + - jsonPath: .status.fastStartFailover + name: FSFO + type: string + name: v4 + schema: + openAPIV3Schema: + properties: + apiVersion: + type: string + kind: + type: string + metadata: + type: object + spec: + properties: + fastStartFailover: + type: boolean + loadBalancer: + type: boolean + nodeSelector: + additionalProperties: + type: string + type: object + primaryDatabaseRef: + type: string + protectionMode: + enum: + - MaxPerformance + - MaxAvailability + type: string + serviceAnnotations: + additionalProperties: + type: string + type: object + setAsPrimaryDatabase: + type: string + standbyDatabaseRefs: + items: + type: string + type: array + required: + - primaryDatabaseRef + - protectionMode + - standbyDatabaseRefs + type: object + status: + properties: + clusterConnectString: + type: string + databasesInDataguardConfig: + additionalProperties: + type: string + type: object + externalConnectString: + type: string + fastStartFailover: + type: boolean + primaryDatabase: + type: string + primaryDatabaseRef: + type: string + protectionMode: + type: string + standbyDatabases: + type: string + status: + type: string + type: object + type: object + served: true + storage: true + subresources: + status: {} +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + cert-manager.io/inject-ca-from: oracle-database-operator-system/oracle-database-operator-serving-cert + controller-gen.kubebuilder.io/version: v0.16.5 + name: dbcssystems.database.oracle.com +spec: + group: database.oracle.com + names: + kind: DbcsSystem + listKind: DbcsSystemList + plural: dbcssystems + singular: dbcssystem + scope: Namespaced + versions: + - name: v1alpha1 + schema: + openAPIV3Schema: + properties: + apiVersion: + type: string + kind: + type: string + metadata: + type: object + spec: + properties: + databaseId: + type: string + dbBackupId: + type: string + dbClone: + properties: + dbAdminPaswordSecret: + type: string + dbDbUniqueName: + type: string + dbName: + type: string + displayName: + type: string + domain: + type: string + hostName: + type: string + initialDataStorageSizeInGB: + type: integer + kmsKeyId: + type: string + kmsKeyVersionId: + type: string + licenseModel: + type: string + privateIp: + type: string + sidPrefix: + type: string + sshPublicKeys: + items: + type: string + type: array + subnetId: + type: string + tdeWalletPasswordSecret: + type: string + required: + - dbDbUniqueName + - dbName + - displayName + - hostName + - subnetId + type: object + dbSystem: + properties: + availabilityDomain: + type: string + backupSubnetId: + type: string + clusterName: + type: string + compartmentId: + type: string + cpuCoreCount: + type: integer + dbAdminPaswordSecret: + type: string + dbBackupConfig: + properties: + autoBackupEnabled: + type: boolean + autoBackupWindow: + type: string + backupDestinationDetails: + type: string + recoveryWindowsInDays: + type: integer + type: object + dbDomain: + type: string + dbEdition: + type: string + dbName: + type: string + dbUniqueName: + type: string + dbVersion: + type: string + dbWorkload: + type: string + diskRedundancy: + type: string + displayName: + type: string + domain: + type: string + faultDomains: + items: + type: string + type: array + hostName: + type: string + initialDataStorageSizeInGB: + type: integer + kmsConfig: + properties: + compartmentId: + type: string + encryptionAlgo: + type: string + keyName: + type: string + vaultName: + type: string + vaultType: + type: string + type: object + licenseModel: + type: string + nodeCount: + type: integer + pdbName: + type: string + privateIp: + type: string + shape: + type: string + sshPublicKeys: + items: + type: string + type: array + storageManagement: + type: string + subnetId: + type: string + tags: + additionalProperties: + type: string + type: object + tdeWalletPasswordSecret: + type: string + timeZone: + type: string + required: + - availabilityDomain + - compartmentId + - dbAdminPaswordSecret + - hostName + - shape + - subnetId + type: object + hardLink: + type: boolean + id: + type: string + kmsConfig: + properties: + compartmentId: + type: string + encryptionAlgo: + type: string + keyName: + type: string + vaultName: + type: string + vaultType: + type: string + type: object + ociConfigMap: + type: string + ociSecret: + type: string + pdbConfigs: + items: + properties: + freeformTags: + additionalProperties: + type: string + type: object + isDelete: + type: boolean + pdbAdminPassword: + type: string + pdbName: + type: string + pluggableDatabaseId: + type: string + shouldPdbAdminAccountBeLocked: + type: boolean + tdeWalletPassword: + type: string + required: + - freeformTags + - pdbAdminPassword + - pdbName + - shouldPdbAdminAccountBeLocked + - tdeWalletPassword + type: object + type: array + setupDBCloning: + type: boolean + required: + - ociConfigMap + type: object + status: + properties: + availabilityDomain: + type: string + cpuCoreCount: + type: integer + dataStoragePercentage: + type: integer + dataStorageSizeInGBs: + type: integer + dbCloneStatus: + properties: + dbAdminPaswordSecret: + type: string + dbDbUniqueName: + type: string + dbName: + type: string + displayName: + type: string + domain: + type: string + hostName: + type: string + id: + type: string + licenseModel: + type: string + sshPublicKeys: + items: + type: string + type: array + subnetId: + type: string + required: + - dbDbUniqueName + - hostName + type: object + dbEdition: + type: string + dbInfo: + items: + properties: + dbHomeId: + type: string + dbName: + type: string + dbUniqueName: + type: string + dbWorkload: + type: string + id: + type: string + type: object + type: array + displayName: + type: string + id: + type: string + kmsDetailsStatus: + properties: + compartmentId: + type: string + encryptionAlgo: + type: string + keyId: + type: string + keyName: + type: string + managementEndpoint: + type: string + vaultId: + type: string + vaultName: + type: string + vaultType: + type: string + type: object + licenseModel: + type: string + network: + properties: + clientSubnet: + type: string + domainName: + type: string + hostName: + type: string + listenerPort: + type: integer + networkSG: + type: string + scanDnsName: + type: string + vcnName: + type: string + type: object + nodeCount: + type: integer + pdbDetailsStatus: + items: + properties: + pdbConfigStatus: + items: + properties: + freeformTags: + additionalProperties: + type: string + type: object + pdbName: + type: string + pdbState: + type: string + pluggableDatabaseId: + type: string + shouldPdbAdminAccountBeLocked: + type: boolean + required: + - freeformTags + - pdbName + - shouldPdbAdminAccountBeLocked + type: object + type: array + type: object + type: array + recoStorageSizeInGB: + type: integer + shape: + type: string + state: + type: string + storageManagement: + type: string + subnetId: + type: string + timeZone: + type: string + workRequests: + items: + properties: + operationId: + type: string + operationType: + type: string + percentComplete: + type: string + timeAccepted: + type: string + timeFinished: + type: string + timeStarted: + type: string + required: + - operationId + - operationType + type: object + type: array + required: + - state + type: object + type: object + served: true + storage: false + subresources: + status: {} + - name: v4 + schema: + openAPIV3Schema: + properties: + apiVersion: + type: string + kind: + type: string + metadata: + type: object + spec: + properties: + databaseId: + type: string + dbBackupId: + type: string + dbClone: + properties: + dbAdminPasswordSecret: + type: string + dbDbUniqueName: + type: string + dbName: + type: string + displayName: + type: string + domain: + type: string + hostName: + type: string + initialDataStorageSizeInGB: + type: integer + kmsKeyId: + type: string + kmsKeyVersionId: + type: string + licenseModel: + type: string + privateIp: + type: string + sidPrefix: + type: string + sshPublicKeys: + items: + type: string + type: array + subnetId: + type: string + tdeWalletPasswordSecret: + type: string + required: + - dbDbUniqueName + - dbName + - displayName + - hostName + - subnetId + type: object + dbSystem: + properties: + availabilityDomain: + type: string + backupSubnetId: + type: string + clusterName: + type: string + compartmentId: + type: string + cpuCoreCount: + type: integer + dbAdminPasswordSecret: + type: string + dbBackupConfig: + properties: + autoBackupEnabled: + type: boolean + autoBackupWindow: + type: string + backupDestinationDetails: + type: string + recoveryWindowsInDays: + type: integer + type: object + dbDomain: + type: string + dbEdition: + type: string + dbName: + type: string + dbUniqueName: + type: string + dbVersion: + type: string + dbWorkload: type: string diskRedundancy: type: string - displayName: + displayName: + type: string + domain: + type: string + faultDomains: + items: + type: string + type: array + hostName: + type: string + initialDataStorageSizeInGB: + type: integer + kmsConfig: + properties: + compartmentId: + type: string + encryptionAlgo: + type: string + keyName: + type: string + vaultName: + type: string + vaultType: + type: string + type: object + licenseModel: + type: string + nodeCount: + type: integer + pdbName: + type: string + privateIp: + type: string + shape: + type: string + sshPublicKeys: + items: + type: string + type: array + storageManagement: + type: string + subnetId: + type: string + tags: + additionalProperties: + type: string + type: object + tdeWalletPasswordSecret: + type: string + timeZone: + type: string + required: + - availabilityDomain + - compartmentId + - dbAdminPasswordSecret + - hostName + - shape + - subnetId + type: object + hardLink: + type: boolean + id: + type: string + kmsConfig: + properties: + compartmentId: + type: string + encryptionAlgo: + type: string + keyName: + type: string + vaultName: + type: string + vaultType: + type: string + type: object + ociConfigMap: + type: string + ociSecret: + type: string + pdbConfigs: + items: + properties: + freeformTags: + additionalProperties: + type: string + type: object + isDelete: + type: boolean + pdbAdminPassword: + type: string + pdbName: + type: string + pluggableDatabaseId: + type: string + shouldPdbAdminAccountBeLocked: + type: boolean + tdeWalletPassword: + type: string + required: + - freeformTags + - pdbAdminPassword + - pdbName + - shouldPdbAdminAccountBeLocked + - tdeWalletPassword + type: object + type: array + setupDBCloning: + type: boolean + required: + - ociConfigMap + type: object + status: + properties: + availabilityDomain: + type: string + cpuCoreCount: + type: integer + dataStoragePercentage: + type: integer + dataStorageSizeInGBs: + type: integer + dbCloneStatus: + properties: + dbAdminPaswordSecret: + type: string + dbDbUniqueName: + type: string + dbName: + type: string + displayName: + type: string + domain: + type: string + hostName: + type: string + id: + type: string + licenseModel: + type: string + sshPublicKeys: + items: + type: string + type: array + subnetId: + type: string + required: + - dbDbUniqueName + - hostName + type: object + dbEdition: + type: string + dbInfo: + items: + properties: + dbHomeId: + type: string + dbName: + type: string + dbUniqueName: + type: string + dbWorkload: + type: string + id: + type: string + type: object + type: array + displayName: + type: string + id: + type: string + kmsDetailsStatus: + properties: + compartmentId: + type: string + encryptionAlgo: + type: string + keyId: + type: string + keyName: + type: string + managementEndpoint: + type: string + vaultId: + type: string + vaultName: + type: string + vaultType: + type: string + type: object + licenseModel: + type: string + network: + properties: + clientSubnet: + type: string + domainName: + type: string + hostName: + type: string + listenerPort: + type: integer + networkSG: + type: string + scanDnsName: + type: string + vcnName: + type: string + type: object + nodeCount: + type: integer + pdbDetailsStatus: + items: + properties: + pdbConfigStatus: + items: + properties: + freeformTags: + additionalProperties: + type: string + type: object + pdbName: + type: string + pdbState: + type: string + pluggableDatabaseId: + type: string + shouldPdbAdminAccountBeLocked: + type: boolean + type: object + type: array + type: object + type: array + recoStorageSizeInGB: + type: integer + shape: + type: string + state: + type: string + storageManagement: + type: string + subnetId: + type: string + timeZone: + type: string + workRequests: + items: + properties: + operationId: + type: string + operationType: + type: string + percentComplete: + type: string + timeAccepted: + type: string + timeFinished: + type: string + timeStarted: + type: string + required: + - operationId + - operationType + type: object + type: array + required: + - state + type: object + type: object + served: true + storage: true + subresources: + status: {} +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.16.5 + name: lrests.database.oracle.com +spec: + group: database.oracle.com + names: + kind: LREST + listKind: LRESTList + plural: lrests + singular: lrest + scope: Namespaced + versions: + - additionalPrinterColumns: + - description: Name of the LREST + jsonPath: .spec.cdbName + name: CDB NAME + type: string + - description: ' Name of the DB Server' + jsonPath: .spec.dbServer + name: DB Server + type: string + - description: DB server port + jsonPath: .spec.dbPort + name: DB Port + type: integer + - description: Replicas + jsonPath: .spec.replicas + name: Replicas + type: integer + - description: Status of the LREST Resource + jsonPath: .status.phase + name: Status + type: string + - description: Error message if any + jsonPath: .status.msg + name: Message + type: string + - description: string of the tnsalias + jsonPath: .spec.dbTnsurl + name: TNS STRING + type: string + name: v4 + schema: + openAPIV3Schema: + properties: + apiVersion: + type: string + kind: + type: string + metadata: + type: object + spec: + properties: + cdbAdminPwd: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + cdbAdminUser: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + cdbName: + type: string + cdbPrvKey: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + cdbPubKey: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + cdbTlsCrt: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + cdbTlsKey: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + dbPort: + type: integer + dbServer: + type: string + dbTnsurl: + type: string + deletePdbCascade: + type: boolean + lrestImage: + type: string + lrestImagePullPolicy: + enum: + - Always + - Never + type: string + lrestImagePullSecret: + type: string + lrestPort: + type: integer + lrestPwd: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + nodeSelector: + additionalProperties: + type: string + type: object + replicas: + type: integer + serviceName: + type: string + sysAdminPwd: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + webServerPwd: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + webServerUser: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + type: object + status: + properties: + msg: + type: string + phase: + type: string + status: + type: boolean + required: + - phase + - status + type: object + type: object + served: true + storage: true + subresources: + status: {} +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.16.5 + name: lrpdbs.database.oracle.com +spec: + group: database.oracle.com + names: + kind: LRPDB + listKind: LRPDBList + plural: lrpdbs + singular: lrpdb + scope: Namespaced + versions: + - additionalPrinterColumns: + - description: Name of the CDB + jsonPath: .spec.cdbName + name: CDB Name + type: string + - description: Name of the PDB + jsonPath: .spec.pdbName + name: PDB Name + type: string + - description: PDB Open Mode + jsonPath: .status.openMode + name: PDB State + type: string + - description: Total Size of the PDB + jsonPath: .status.totalSize + name: PDB Size + type: string + - description: Status of the LRPDB Resource + jsonPath: .status.phase + name: Status + type: string + - description: Error message, if any + jsonPath: .status.msg + name: Message + type: string + - description: last sqlcode + jsonPath: .status.sqlCode + name: last sqlcode + type: integer + - description: The connect string to be used + jsonPath: .status.connString + name: Connect_String + type: string + name: v4 + schema: + openAPIV3Schema: + properties: + apiVersion: + type: string + kind: + type: string + metadata: + type: object + spec: + properties: + action: + enum: + - Create + - Clone + - Plug + - Unplug + - Delete + - Modify + - Status + - Map + - Alter + - Noaction + type: string + adminName: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + adminPwd: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + adminpdbPass: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + adminpdbUser: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + alterSystem: + type: string + alterSystemParameter: + type: string + alterSystemValue: + type: string + asClone: + type: boolean + assertiveLrpdbDeletion: + type: boolean + cdbName: + type: string + cdbNamespace: + type: string + cdbPrvKey: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + cdbResName: + type: string + copyAction: + enum: + - COPY + - NOCOPY + - MOVE + type: string + dropAction: + enum: + - INCLUDING + - KEEP + type: string + fileNameConversions: + type: string + getScript: + type: boolean + lrpdbTlsCat: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + lrpdbTlsCrt: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + lrpdbTlsKey: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + modifyOption: + enum: + - IMMEDIATE + - NORMAL + - READ ONLY + - READ WRITE + - RESTRICTED + type: string + parameterScope: + type: string + pdbName: + type: string + pdbState: + enum: + - OPEN + - CLOSE + - ALTER + type: string + pdbconfigmap: + type: string + reuseTempFile: + type: boolean + sourceFileNameConversions: + type: string + sparseClonePath: + type: string + srcPdbName: + type: string + tdeExport: + type: boolean + tdeImport: + type: boolean + tdeKeystorePath: + type: string + tdePassword: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + tdeSecret: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + tempSize: + type: string + totalSize: + type: string + unlimitedStorage: + type: boolean + webServerPwd: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + webServerUser: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + xmlFileName: + type: string + required: + - action + - alterSystemParameter + - alterSystemValue + - webServerPwd + type: object + status: + properties: + action: + type: string + alterSystem: + type: string + bitstat: + type: integer + bitstatstr: + type: string + connString: + type: string + modifyOption: + type: string + msg: + type: string + openMode: + type: string + phase: + type: string + sqlCode: + type: integer + status: + type: boolean + totalSize: + type: string + required: + - phase + - sqlCode + - status + type: object + type: object + served: true + storage: true + subresources: + status: {} +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.16.5 + name: oraclerestdataservices.database.oracle.com +spec: + group: database.oracle.com + names: + kind: OracleRestDataService + listKind: OracleRestDataServiceList + plural: oraclerestdataservices + singular: oraclerestdataservice + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .status.status + name: Status + type: string + - jsonPath: .spec.databaseRef + name: Database + type: string + - jsonPath: .status.databaseApiUrl + name: Database API URL + type: string + - jsonPath: .status.databaseActionsUrl + name: Database Actions URL + type: string + - jsonPath: .status.apexUrl + name: Apex URL + type: string + - jsonPath: .status.mongoDbApiAccessUrl + name: MongoDbApi Access URL + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + properties: + apiVersion: + type: string + kind: + type: string + metadata: + type: object + spec: + properties: + adminPassword: + properties: + keepSecret: + type: boolean + secretKey: + default: oracle_pwd + type: string + secretName: + type: string + required: + - secretName + type: object + databaseRef: + type: string + image: + properties: + pullFrom: + type: string + pullSecrets: + type: string + version: + type: string + required: + - pullFrom + type: object + loadBalancer: + type: boolean + mongoDbApi: + type: boolean + nodeSelector: + additionalProperties: + type: string + type: object + oracleService: + type: string + ordsPassword: + properties: + keepSecret: + type: boolean + secretKey: + default: oracle_pwd + type: string + secretName: + type: string + required: + - secretName + type: object + ordsUser: + type: string + persistence: + properties: + accessMode: + enum: + - ReadWriteOnce + - ReadWriteMany + type: string + setWritePermissions: + type: boolean + size: + type: string + storageClass: + type: string + volumeName: + type: string + type: object + readinessCheckPeriod: + type: integer + replicas: + minimum: 1 + type: integer + restEnableSchemas: + items: + properties: + enable: + type: boolean + pdbName: + type: string + schemaName: + type: string + urlMapping: + type: string + required: + - enable + - schemaName + type: object + type: array + serviceAccountName: + type: string + serviceAnnotations: + additionalProperties: + type: string + type: object + required: + - adminPassword + - databaseRef + - ordsPassword + type: object + status: + properties: + apexConfigured: + type: boolean + apexUrl: + type: string + commonUsersCreated: + type: boolean + databaseActionsUrl: + type: string + databaseApiUrl: + type: string + databaseRef: + type: string + image: + properties: + pullFrom: + type: string + pullSecrets: + type: string + version: + type: string + required: + - pullFrom + type: object + loadBalancer: + type: string + mongoDbApi: + type: boolean + mongoDbApiAccessUrl: + type: string + ordsInstalled: + type: boolean + replicas: + type: integer + serviceIP: + type: string + status: + type: string + type: object + type: object + served: true + storage: false + subresources: + status: {} + - additionalPrinterColumns: + - jsonPath: .status.status + name: Status + type: string + - jsonPath: .spec.databaseRef + name: Database + type: string + - jsonPath: .status.databaseApiUrl + name: Database API URL + type: string + - jsonPath: .status.databaseActionsUrl + name: Database Actions URL + type: string + - jsonPath: .status.apexUrl + name: Apex URL + type: string + - jsonPath: .status.mongoDbApiAccessUrl + name: MongoDbApi Access URL + type: string + name: v4 + schema: + openAPIV3Schema: + properties: + apiVersion: + type: string + kind: + type: string + metadata: + type: object + spec: + properties: + adminPassword: + properties: + keepSecret: + type: boolean + secretKey: + default: oracle_pwd + type: string + secretName: + type: string + required: + - secretName + type: object + databaseRef: + type: string + image: + properties: + pullFrom: + type: string + pullSecrets: + type: string + version: + type: string + required: + - pullFrom + type: object + loadBalancer: + type: boolean + mongoDbApi: + type: boolean + nodeSelector: + additionalProperties: + type: string + type: object + oracleService: + type: string + ordsPassword: + properties: + keepSecret: + type: boolean + secretKey: + default: oracle_pwd + type: string + secretName: + type: string + required: + - secretName + type: object + ordsUser: + type: string + persistence: + properties: + accessMode: + enum: + - ReadWriteOnce + - ReadWriteMany + type: string + setWritePermissions: + type: boolean + size: + type: string + storageClass: + type: string + volumeName: + type: string + type: object + readinessCheckPeriod: + type: integer + replicas: + minimum: 1 + type: integer + restEnableSchemas: + items: + properties: + enable: + type: boolean + pdbName: + type: string + schemaName: + type: string + urlMapping: + type: string + required: + - enable + - schemaName + type: object + type: array + serviceAccountName: + type: string + serviceAnnotations: + additionalProperties: + type: string + type: object + required: + - adminPassword + - databaseRef + - ordsPassword + type: object + status: + properties: + apexConfigured: + type: boolean + apexUrl: + type: string + commonUsersCreated: + type: boolean + databaseActionsUrl: + type: string + databaseApiUrl: + type: string + databaseRef: + type: string + image: + properties: + pullFrom: + type: string + pullSecrets: type: string - domain: + version: type: string - faultDomains: - items: - type: string - type: array - hostName: + required: + - pullFrom + type: object + loadBalancer: + type: string + mongoDbApi: + type: boolean + mongoDbApiAccessUrl: + type: string + ordsInstalled: + type: boolean + replicas: + type: integer + serviceIP: + type: string + status: + type: string + type: object + type: object + served: true + storage: true + subresources: + status: {} +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + controller-gen.kubebuilder.io/version: v0.16.5 + name: ordssrvs.database.oracle.com +spec: + group: database.oracle.com + names: + kind: OrdsSrvs + listKind: OrdsSrvsList + plural: ordssrvs + singular: ordssrvs + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .status.status + name: status + type: string + - jsonPath: .status.workloadType + name: workloadType + type: string + - jsonPath: .status.ordsVersion + name: ordsVersion + type: string + - jsonPath: .status.httpPort + name: httpPort + type: integer + - jsonPath: .status.httpsPort + name: httpsPort + type: integer + - jsonPath: .status.mongoPort + name: MongoPort + type: integer + - jsonPath: .status.restartRequired + name: restartRequired + type: boolean + - jsonPath: .metadata.creationTimestamp + name: AGE + type: date + - jsonPath: .status.ordsInstalled + name: OrdsInstalled + type: boolean + name: v4 + schema: + openAPIV3Schema: + properties: + apiVersion: + type: string + kind: + type: string + metadata: + type: object + spec: + properties: + encPrivKey: + properties: + passwordKey: + default: password type: string - initialDataStorageSizeInGB: + secretName: + type: string + required: + - secretName + type: object + forceRestart: + type: boolean + globalSettings: + properties: + cache.metadata.enabled: + type: boolean + cache.metadata.graphql.expireAfterAccess: + format: int64 type: integer - kmsKeyId: + cache.metadata.graphql.expireAfterWrite: + format: int64 + type: integer + cache.metadata.jwks.enabled: + type: boolean + cache.metadata.jwks.expireAfterAccess: + format: int64 + type: integer + cache.metadata.jwks.expireAfterWrite: + format: int64 + type: integer + cache.metadata.jwks.initialCapacity: + format: int32 + type: integer + cache.metadata.jwks.maximumSize: + format: int32 + type: integer + cache.metadata.timeout: + format: int64 + type: integer + certSecret: + properties: + cert: + type: string + key: + type: string + secretName: + type: string + required: + - cert + - key + - secretName + type: object + database.api.enabled: + type: boolean + database.api.management.services.disabled: + type: boolean + db.invalidPoolTimeout: + format: int64 + type: integer + debug.printDebugToScreen: + type: boolean + enable.mongo.access.log: + default: false + type: boolean + enable.standalone.access.log: + default: false + type: boolean + error.responseFormat: type: string - kmsKeyVersionId: + feature.grahpql.max.nesting.depth: + format: int32 + type: integer + icap.port: + format: int32 + type: integer + icap.secure.port: + format: int32 + type: integer + icap.server: type: string - licenseModel: + log.procedure: + type: boolean + mongo.enabled: + type: boolean + mongo.idle.timeout: + format: int64 + type: integer + mongo.op.timeout: + format: int64 + type: integer + mongo.port: + default: 27017 + format: int32 + type: integer + request.traceHeaderName: type: string - nodeCount: + security.credentials.attempts: + format: int32 type: integer - pdbName: + security.credentials.lock.time: + format: int64 + type: integer + security.disableDefaultExclusionList: + type: boolean + security.exclusionList: type: string - privateIp: + security.externalSessionTrustedOrigins: type: string - shape: + security.forceHTTPS: + type: boolean + security.httpsHeaderCheck: type: string - sshPublicKeys: - items: + security.inclusionList: + type: string + security.maxEntries: + format: int32 + type: integer + security.verifySSL: + type: boolean + standalone.context.path: + default: /ords + type: string + standalone.http.port: + default: 8080 + format: int32 + type: integer + standalone.https.host: + type: string + standalone.https.port: + default: 8443 + format: int32 + type: integer + standalone.stop.timeout: + format: int64 + type: integer + type: object + image: + type: string + imagePullPolicy: + default: IfNotPresent + enum: + - IfNotPresent + - Always + - Never + type: string + imagePullSecrets: + type: string + poolSettings: + items: + properties: + apex.security.administrator.roles: + type: string + apex.security.user.roles: + type: string + autoUpgradeAPEX: + default: false + type: boolean + autoUpgradeORDS: + default: false + type: boolean + db.adminUser: + type: string + db.adminUser.secret: + properties: + passwordKey: + default: password + type: string + secretName: + type: string + required: + - secretName + type: object + db.cdb.adminUser: + type: string + db.cdb.adminUser.secret: + properties: + passwordKey: + default: password + type: string + secretName: + type: string + required: + - secretName + type: object + db.connectionType: + enum: + - basic + - tns + - customurl + type: string + db.credentialsSource: + enum: + - pool + - request + type: string + db.customURL: + type: string + db.hostname: + type: string + db.poolDestroyTimeout: + format: int64 + type: integer + db.port: + format: int32 + type: integer + db.secret: + properties: + passwordKey: + default: password + type: string + secretName: + type: string + required: + - secretName + type: object + db.servicename: + type: string + db.sid: + type: string + db.tnsAliasName: + type: string + db.username: + default: ORDS_PUBLIC_USER + type: string + db.wallet.zip.service: + type: string + dbWalletSecret: + properties: + secretName: + type: string + walletName: + type: string + required: + - secretName + - walletName + type: object + debug.trackResources: + type: boolean + feature.openservicebroker.exclude: + type: boolean + feature.sdw: + type: boolean + http.cookie.filter: + type: string + jdbc.DriverType: + enum: + - thin + - oci8 + type: string + jdbc.InactivityTimeout: + format: int32 + type: integer + jdbc.InitialLimit: + format: int32 + type: integer + jdbc.MaxConnectionReuseCount: + format: int32 + type: integer + jdbc.MaxConnectionReuseTime: + format: int32 + type: integer + jdbc.MaxLimit: + format: int32 + type: integer + jdbc.MaxStatementsLimit: + format: int32 + type: integer + jdbc.MinLimit: + format: int32 + type: integer + jdbc.SecondsToTrustIdleConnection: + format: int32 + type: integer + jdbc.auth.admin.role: + type: string + jdbc.auth.enabled: + type: boolean + jdbc.cleanup.mode: + type: string + jdbc.statementTimeout: + format: int32 + type: integer + misc.defaultPage: + type: string + misc.pagination.maxRows: + format: int32 + type: integer + owa.trace.sql: + type: boolean + plsql.gateway.mode: + enum: + - disabled + - direct + - proxied + type: string + poolName: + type: string + procedure.preProcess: + type: string + procedure.rest.preHook: + type: string + procedurePostProcess: + type: string + restEnabledSql.active: + type: boolean + security.jwks.connection.timeout: + format: int64 + type: integer + security.jwks.read.timeout: + format: int64 + type: integer + security.jwks.refresh.interval: + format: int64 + type: integer + security.jwks.size: + format: int32 + type: integer + security.jwt.allowed.age: + format: int64 + type: integer + security.jwt.allowed.skew: + format: int64 + type: integer + security.jwt.profile.enabled: + type: boolean + security.requestAuthenticationFunction: + type: string + security.requestValidationFunction: + default: ords_util.authorize_plsql_gateway + type: string + security.validationFunctionType: + enum: + - plsql + - javascript type: string - type: array - storageManagement: - type: string - subnetId: - type: string - tags: - additionalProperties: + soda.defaultLimit: type: string - type: object - tdeWalletPasswordSecret: - type: string - timeZone: - type: string - required: - - availabilityDomain - - compartmentId - - dbAdminPaswordSecret - - hostName - - shape - - sshPublicKeys - - subnetId - type: object - hardLink: - type: boolean - id: - type: string - ociConfigMap: - type: string - ociSecret: + soda.maxLimit: + type: string + tnsAdminSecret: + properties: + secretName: + type: string + required: + - secretName + type: object + required: + - db.secret + - poolName + type: object + type: array + replicas: + default: 1 + format: int32 + minimum: 1 + type: integer + workloadType: + default: Deployment + enum: + - Deployment + - StatefulSet + - DaemonSet type: string required: - - ociConfigMap + - globalSettings + - image type: object status: - description: DbcsSystemStatus defines the observed state of DbcsSystem properties: - availabilityDomain: - type: string - cpuCoreCount: - type: integer - dataStoragePercentage: - type: integer - dataStorageSizeInGBs: - type: integer - dbEdition: - type: string - dbInfo: + conditions: items: - description: DbcsSystemStatus defines the observed state of DbcsSystem properties: - dbHomeId: + lastTransitionTime: + format: date-time type: string - dbName: + message: + maxLength: 32768 type: string - dbUniqueName: + observedGeneration: + format: int64 + minimum: 0 + type: integer + reason: + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ type: string - dbWorkload: + status: + enum: + - "True" + - "False" + - Unknown type: string - id: + type: + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ type: string + required: + - lastTransitionTime + - message + - reason + - status + - type type: object type: array - displayName: + httpPort: + format: int32 + type: integer + httpsPort: + format: int32 + type: integer + mongoPort: + format: int32 + type: integer + ordsInstalled: + type: boolean + ordsVersion: type: string - id: + restartRequired: + type: boolean + status: type: string - licenseModel: + workloadType: type: string - network: + required: + - restartRequired + type: object + type: object + served: true + storage: true + subresources: + status: {} +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + cert-manager.io/inject-ca-from: oracle-database-operator-system/oracle-database-operator-serving-cert + controller-gen.kubebuilder.io/version: v0.16.5 + name: pdbs.database.oracle.com +spec: + group: database.oracle.com + names: + kind: PDB + listKind: PDBList + plural: pdbs + singular: pdb + scope: Namespaced + versions: + - additionalPrinterColumns: + - description: Name of the CDB + jsonPath: .spec.cdbName + name: CDB Name + type: string + - description: Name of the PDB + jsonPath: .spec.pdbName + name: PDB Name + type: string + - description: PDB Open Mode + jsonPath: .status.openMode + name: PDB State + type: string + - description: Total Size of the PDB + jsonPath: .status.totalSize + name: PDB Size + type: string + - description: Status of the PDB Resource + jsonPath: .status.phase + name: Status + type: string + - description: Error message, if any + jsonPath: .status.msg + name: Message + type: string + - description: The connect string to be used + jsonPath: .status.connString + name: Connect_String + type: string + name: v4 + schema: + openAPIV3Schema: + properties: + apiVersion: + type: string + kind: + type: string + metadata: + type: object + spec: + properties: + action: + enum: + - Create + - Clone + - Plug + - Unplug + - Delete + - Modify + - Status + - Map + type: string + adminName: properties: - clientSubnet: - type: string - domainName: - type: string - hostName: - type: string - listenerPort: - type: integer - networkSG: - type: string - scanDnsName: - type: string - vcnName: - type: string + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret type: object - nodeCount: - type: integer - recoStorageSizeInGB: - type: integer - shape: + adminPwd: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + asClone: + type: boolean + assertivePdbDeletion: + type: boolean + assertivePdbDeletion: + description: turn on the assertive approach to delete pdb resource kubectl delete pdb ..... automatically triggers the pluggable database deletion + type: boolean + cdbName: type: string - state: + cdbNamespace: type: string - storageManagement: + cdbNamespace: + description: CDB Namespace type: string - subnetId: + cdbResName: + type: string + copyAction: + enum: + - COPY + - NOCOPY + - MOVE + type: string + dropAction: + enum: + - INCLUDING + - KEEP + type: string + fileNameConversions: + type: string + getScript: + type: boolean + modifyOption: + enum: + - IMMEDIATE + - NORMAL + - READ ONLY + - READ WRITE + - RESTRICTED type: string - timeZone: + pdbName: type: string - workRequests: - items: - properties: - operationId: - type: string - operationType: - type: string - percentComplete: - type: string - timeAccepted: - type: string - timeFinished: - type: string - timeStarted: - type: string - required: - - operationId - - operationType - type: object - type: array - required: - - state - type: object - type: object - served: true - storage: true - subresources: - status: {} -status: - acceptedNames: - kind: "" - plural: "" - conditions: [] - storedVersions: [] ---- -apiVersion: apiextensions.k8s.io/v1 -kind: CustomResourceDefinition -metadata: - annotations: - controller-gen.kubebuilder.io/version: v0.6.1 - creationTimestamp: null - name: oraclerestdataservices.database.oracle.com -spec: - group: database.oracle.com - names: - kind: OracleRestDataService - listKind: OracleRestDataServiceList - plural: oraclerestdataservices - singular: oraclerestdataservice - scope: Namespaced - versions: - - additionalPrinterColumns: - - jsonPath: .status.status - name: Status - type: string - - jsonPath: .spec.databaseRef - name: Database - type: string - - jsonPath: .status.databaseApiUrl - name: Database API URL - type: string - - jsonPath: .status.databaseActionsUrl - name: Database Actions URL - type: string - - jsonPath: .status.apexUrl - name: Apex URL - type: string - name: v1alpha1 - schema: - openAPIV3Schema: - description: OracleRestDataService is the Schema for the oraclerestdataservices API - properties: - apiVersion: - description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' - type: string - kind: - description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' - type: string - metadata: - type: object - spec: - description: OracleRestDataServiceSpec defines the desired state of OracleRestDataService - properties: - adminPassword: - description: OracleRestDataServicePassword defines the secret containing Password mapped to secretKey + pdbOrdsPrvKey: properties: - keepSecret: - type: boolean - secretKey: - default: oracle_pwd - type: string - secretName: - type: string + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object required: - - secretName + - secret type: object - apexPassword: - description: OracleRestDataServicePassword defines the secret containing Password mapped to secretKey + pdbOrdsPubKey: properties: - keepSecret: - type: boolean - secretKey: - default: oracle_pwd - type: string - secretName: - type: string + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object required: - - secretName + - secret type: object - databaseRef: + pdbState: + enum: + - OPEN + - CLOSE type: string - image: - description: OracleRestDataServiceImage defines the Image source and pullSecrets for POD + pdbTlsCat: properties: - pullFrom: - type: string - pullSecrets: - type: string - version: - type: string + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object required: - - pullFrom + - secret type: object - loadBalancer: - type: boolean - nodeSelector: - additionalProperties: - type: string + pdbTlsCrt: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret type: object - oracleService: - type: string - ordsPassword: - description: OracleRestDataServicePassword defines the secret containing Password mapped to secretKey + pdbTlsKey: properties: - keepSecret: - type: boolean - secretKey: - default: oracle_pwd - type: string - secretName: - type: string + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object required: - - secretName + - secret type: object - ordsUser: + reuseTempFile: + type: boolean + sourceFileNameConversions: type: string - persistence: - description: OracleRestDataServicePersistence defines the storage releated params + sparseClonePath: + type: string + srcPdbName: + type: string + tdeExport: + type: boolean + tdeImport: + type: boolean + tdeKeystorePath: + type: string + tdePassword: properties: - accessMode: - enum: - - ReadWriteOnce - - ReadWriteMany - type: string - size: - type: string - storageClass: - type: string - volumeName: - type: string + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret type: object - replicas: - minimum: 1 - type: integer - restEnableSchemas: - items: - description: OracleRestDataServicePDBSchemas defines the PDB Schemas to be ORDS Enabled - properties: - enable: - type: boolean - pdbName: - type: string - schemaName: - type: string - urlMapping: - type: string - required: - - enable - - schemaName - type: object - type: array - serviceAccountName: + tdeSecret: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + tempSize: + type: string + totalSize: type: string - serviceAnnotations: - additionalProperties: - type: string + unlimitedStorage: + type: boolean + webServerPwd: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + webServerUser: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret type: object + xmlFileName: + type: string required: - - adminPassword - - databaseRef - - ordsPassword + - action type: object status: - description: OracleRestDataServiceStatus defines the observed state of OracleRestDataService properties: - apexConfigured: - type: boolean - apexUrl: + action: type: string - commonUsersCreated: - type: boolean - databaseActionsUrl: + connString: type: string - databaseApiUrl: + modifyOption: type: string - databaseRef: + msg: type: string - image: - description: OracleRestDataServiceImage defines the Image source and pullSecrets for POD - properties: - pullFrom: - type: string - pullSecrets: - type: string - version: - type: string - required: - - pullFrom - type: object - loadBalancer: + openMode: type: string - ordsInstalled: - type: boolean - replicas: - type: integer - serviceIP: + phase: type: string status: - description: 'INSERT ADDITIONAL STATUS FIELD - define observed state of cluster Important: Run "make" to regenerate code after modifying this file' + type: boolean + totalSize: type: string + required: + - phase + - status type: object type: object served: true storage: true subresources: status: {} -status: - acceptedNames: - kind: "" - plural: "" - conditions: [] - storedVersions: [] --- apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: cert-manager.io/inject-ca-from: oracle-database-operator-system/oracle-database-operator-serving-cert - controller-gen.kubebuilder.io/version: v0.6.1 - name: pdbs.database.oracle.com + controller-gen.kubebuilder.io/version: v0.16.5 + name: shardingdatabases.database.oracle.com spec: group: database.oracle.com names: - kind: PDB - listKind: PDBList - plural: pdbs - singular: pdb + kind: ShardingDatabase + listKind: ShardingDatabaseList + plural: shardingdatabases + singular: shardingdatabase scope: Namespaced versions: - additionalPrinterColumns: - - description: The connect string to be used - jsonPath: .status.connString - name: Connect_String - type: string - - description: Name of the CDB - jsonPath: .spec.cdbName - name: CDB Name - type: string - - description: Name of the PDB - jsonPath: .spec.pdbName - name: PDB Name - type: string - - description: PDB Open Mode - jsonPath: .status.openMode - name: PDB State - type: string - - description: Total Size of the PDB - jsonPath: .status.totalSize - name: PDB Size + - jsonPath: .status.gsm.state + name: Gsm State type: string - - description: Status of the PDB Resource - jsonPath: .status.phase - name: Status + - jsonPath: .status.gsm.services + name: Services type: string - - description: Error message, if any - jsonPath: .status.msg - name: Message + - jsonPath: .status.gsm.shards + name: shards + priority: 1 type: string name: v1alpha1 schema: openAPIV3Schema: - description: PDB is the Schema for the pdbs API properties: apiVersion: - description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: - description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: - description: PDBSpec defines the desired state of PDB properties: - action: - description: 'Action to be taken: Create/Clone/Plug/Unplug/Delete/Modify/Status/Map. Map is used to map a Databse PDB to a Kubernetes PDB CR.' - enum: - - Create - - Clone - - Plug - - Unplug - - Delete - - Modify - - Status - - Map + InvitedNodeSubnet: type: string - adminName: - description: The administrator username for the new PDB. This property is required when the Action property is Create. - properties: - secret: - description: PDBSecret defines the secretName - properties: - key: - type: string - secretName: + catalog: + items: + properties: + envVars: + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + imagePullPolicy: + type: string + isDelete: + type: string + label: + type: string + name: + type: string + nodeSelector: + additionalProperties: type: string - required: - - key - - secretName - type: object - required: - - secret - type: object - adminPwd: - description: The administrator password for the new PDB. This property is required when the Action property is Create. - properties: - secret: - description: PDBSecret defines the secretName - properties: - key: + type: object + pvAnnotations: + additionalProperties: type: string - secretName: + type: object + pvMatchLabels: + additionalProperties: type: string - required: - - key - - secretName - type: object - required: - - secret - type: object - asClone: - description: Indicate if 'AS CLONE' option should be used in the command to plug in a PDB. This property is applicable when the Action property is PLUG but not required. - type: boolean - assertivePdbDeletion: - description: turn on the assertive approach to delete pdb resource kubectl delete pdb ..... automatically triggers the pluggable database deletion - type: boolean - cdbName: - description: Name of the CDB - type: string - cdbNamespace: - description: CDB Namespace - type: string - cdbResName: - description: Name of the CDB Custom Resource that runs the ORDS container - type: string - copyAction: - description: To copy files or not while cloning a PDB - enum: - - COPY - - NOCOPY - - MOVE - type: string - dropAction: - description: Specify if datafiles should be removed or not. The value can be INCLUDING or KEEP (default). - enum: - - INCLUDING - - KEEP - type: string - fileNameConversions: - description: Relevant for Create and Plug operations. As defined in the Oracle Multitenant Database documentation. Values can be a filename convert pattern or NONE. - type: string - getScript: - description: Whether you need the script only or execute the script - type: boolean - modifyOption: - description: Extra options for opening and closing a PDB - enum: - - IMMEDIATE - - NORMAL - - READ ONLY - - READ WRITE - - RESTRICTED + type: object + pvcName: + type: string + resources: + properties: + claims: + items: + properties: + name: + type: string + request: + type: string + required: + - name + type: object + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + limits: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + requests: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + type: object + storageSizeInGb: + format: int32 + type: integer + required: + - name + type: object + type: array + dbEdition: type: string - pdbName: - description: The name of the new PDB. Relevant for both Create and Plug Actions. + dbImage: type: string - pdbState: - description: The target state of the PDB - enum: - - OPEN - - CLOSE + dbImagePullSecret: type: string - pdbTlsCat: + dbSecret: properties: - secret: - description: PDBSecret defines the secretName - properties: - key: - type: string - secretName: - type: string - required: - - key - - secretName - type: object + encryptionType: + type: string + keyFileMountLocation: + type: string + keyFileName: + type: string + keySecretName: + type: string + name: + type: string + nsConfigMap: + type: string + nsSecret: + type: string + pwdFileMountLocation: + type: string + pwdFileName: + type: string required: - - secret + - name + - pwdFileName type: object - pdbTlsCrt: - properties: - secret: - description: PDBSecret defines the secretName - properties: - key: - type: string - secretName: + fssStorageClass: + type: string + gsm: + items: + properties: + directorName: + type: string + envVars: + description: Replicas int32 `json:"replicas,omitempty"` // Gsm Replicas. If you set OraGsmPvcName then it is set default to 1. + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + imagePullPolicy: + type: string + isDelete: + type: string + label: + type: string + name: + type: string + nodeSelector: + additionalProperties: type: string - required: - - key - - secretName - type: object - required: - - secret - type: object - pdbTlsKey: - properties: - secret: - description: PDBSecret defines the secretName - properties: - key: + type: object + pvAnnotations: + additionalProperties: type: string - secretName: + type: object + pvMatchLabels: + additionalProperties: type: string - required: - - key - - secretName - type: object - required: - - secret - type: object - reuseTempFile: - description: Whether to reuse temp file - type: boolean - sourceFileNameConversions: - description: This property is required when the Action property is Plug. As defined in the Oracle Multitenant Database documentation. Values can be a source filename convert pattern or NONE. + type: object + pvcName: + type: string + region: + type: string + resources: + properties: + claims: + items: + properties: + name: + type: string + request: + type: string + required: + - name + type: object + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + limits: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + requests: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + type: object + storageSizeInGb: + format: int32 + type: integer + required: + - name + type: object + type: array + gsmDevMode: type: string - sparseClonePath: - description: A Path specified for sparse clone snapshot copy. (Optional) + gsmImage: type: string - srcPdbName: - description: Name of the Source PDB from which to clone + gsmImagePullSecret: + type: string + gsmService: + items: + properties: + available: + type: string + clbGoal: + type: string + commitOutcome: + type: string + drainTimeout: + type: string + dtp: + type: string + edition: + type: string + failoverDelay: + type: string + failoverMethod: + type: string + failoverPrimary: + type: string + failoverRestore: + type: string + failoverRetry: + type: string + failoverType: + type: string + gdsPool: + type: string + lag: + type: integer + locality: + type: string + name: + type: string + notification: + type: string + pdbName: + type: string + policy: + type: string + preferred: + type: string + prferredAll: + type: string + regionFailover: + type: string + retention: + type: string + role: + type: string + sessionState: + type: string + sqlTransactionProfile: + type: string + stopOption: + type: string + tableFamily: + type: string + tfaPolicy: + type: string + required: + - name + type: object + type: array + gsmShardGroup: + items: + properties: + deployAs: + type: string + name: + type: string + region: + type: string + required: + - name + type: object + type: array + gsmShardSpace: + items: + properties: + chunks: + type: integer + name: + type: string + protectionMode: + type: string + shardGroup: + type: string + required: + - name + type: object + type: array + invitedNodeSubnetFlag: type: string - tdeExport: - description: TDE export for unplug operations + isClone: type: boolean - tdeImport: - description: TDE import for plug operations + isDataGuard: type: boolean - tdeKeystorePath: - description: TDE keystore path is required if the tdeImport or tdeExport flag is set to true. Can be used in plug or unplug operations. + isDebug: + type: boolean + isDeleteOraPvc: + type: boolean + isDownloadScripts: + type: boolean + isExternalSvc: + type: boolean + isTdeWallet: type: string - tdePassword: - description: TDE password if the tdeImport or tdeExport flag is set to true. Can be used in create, plug or unplug operations - properties: - secret: - description: PDBSecret defines the secretName - properties: - key: - type: string - secretName: + liveinessCheckPeriod: + type: integer + portMappings: + items: + properties: + port: + format: int32 + type: integer + protocol: + type: string + targetPort: + format: int32 + type: integer + required: + - port + - protocol + - targetPort + type: object + type: array + readinessCheckPeriod: + type: integer + replicationType: + type: string + scriptsLocation: + type: string + shard: + items: + properties: + deployAs: + type: string + envVars: + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + imagePullPolicy: + type: string + isDelete: + enum: + - enable + - disable + - failed + - force + type: string + label: + type: string + name: + type: string + nodeSelector: + additionalProperties: type: string - required: - - key - - secretName - type: object - required: - - secret - type: object - tdeSecret: - description: TDE secret is required if the tdeImport or tdeExport flag is set to true. Can be used in plug or unplug operations. - properties: - secret: - description: PDBSecret defines the secretName - properties: - key: + type: object + pvAnnotations: + additionalProperties: type: string - secretName: + type: object + pvMatchLabels: + additionalProperties: type: string - required: - - key - - secretName - type: object - required: - - secret - type: object - tempSize: - description: Relevant for Create and Clone operations. Total size for temporary tablespace as defined in the Oracle Multitenant Database documentation. See size_clause description in Database SQL Language Reference documentation. + type: object + pvcName: + type: string + resources: + properties: + claims: + items: + properties: + name: + type: string + request: + type: string + required: + - name + type: object + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + limits: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + requests: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + type: object + shardGroup: + type: string + shardRegion: + type: string + shardSpace: + type: string + storageSizeInGb: + format: int32 + type: integer + required: + - name + type: object + type: array + shardBuddyRegion: type: string - totalSize: - description: Relevant for create and plug operations. Total size as defined in the Oracle Multitenant Database documentation. See size_clause description in Database SQL Language Reference documentation. + shardConfigName: type: string - unlimitedStorage: - description: Relevant for Create and Plug operations. True for unlimited storage. Even when set to true, totalSize and tempSize MUST be specified in the request if Action is Create. - type: boolean - webServerPwd: - description: Password for the Web ServerPDB User - properties: - secret: - description: PDBSecret defines the secretName - properties: - key: - type: string - secretName: - type: string - required: - - key - - secretName - type: object - required: - - secret + shardRegion: + items: + type: string + type: array + shardingType: + type: string + stagePvcName: + type: string + storageClass: + type: string + tdeWalletPvc: + type: string + tdeWalletPvcMountLocation: + type: string + topicId: + type: string + required: + - catalog + - dbImage + - gsm + - gsmImage + - shard + type: object + status: + properties: + catalogs: + additionalProperties: + type: string type: object - webServerUser: - description: Web Server User with SQL Administrator role to allow us to authenticate to the PDB Lifecycle Management REST endpoints + conditions: + items: + properties: + lastTransitionTime: + format: date-time + type: string + message: + maxLength: 32768 + type: string + observedGeneration: + format: int64 + minimum: 0 + type: integer + reason: + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + enum: + - "True" + - "False" + - Unknown + type: string + type: + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + gsm: properties: - secret: - description: PDBSecret defines the secretName - properties: - key: - type: string - secretName: - type: string - required: - - key - - secretName + details: + additionalProperties: + type: string + type: object + externalConnectStr: + type: string + internalConnectStr: + type: string + services: + type: string + shards: + additionalProperties: + type: string type: object - required: - - secret + state: + type: string + type: object + shards: + additionalProperties: + type: string type: object - xmlFileName: - description: XML metadata filename to be used for Plug or Unplug operations - type: string - required: - - action - type: object - status: - description: PDBStatus defines the observed state of PDB - properties: - action: - description: Last Completed Action - type: string - connString: - description: PDB Connect String - type: string - modifyOption: - description: Modify Option of the PDB - type: string - msg: - description: Message - type: string - openMode: - description: Open mode of the PDB - type: string - phase: - description: Phase of the PDB Resource - type: string - status: - description: PDB Resource Status - type: boolean - totalSize: - description: Total size of the PDB - type: string - required: - - phase - - status type: object type: object served: true - storage: true + storage: false subresources: status: {} -status: - acceptedNames: - kind: "" - plural: "" - conditions: [] - storedVersions: [] ---- -apiVersion: apiextensions.k8s.io/v1 -kind: CustomResourceDefinition -metadata: - annotations: - controller-gen.kubebuilder.io/version: v0.6.1 - creationTimestamp: null - name: shardingdatabases.database.oracle.com -spec: - group: database.oracle.com - names: - kind: ShardingDatabase - listKind: ShardingDatabaseList - plural: shardingdatabases - singular: shardingdatabase - scope: Namespaced - versions: - additionalPrinterColumns: - jsonPath: .status.gsm.state name: Gsm State @@ -2045,31 +11884,25 @@ spec: name: shards priority: 1 type: string - name: v1alpha1 + name: v4 schema: openAPIV3Schema: - description: ShardingDatabase is the Schema for the shardingdatabases API properties: apiVersion: - description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: - description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: - description: ShardingDatabaseSpec defines the desired state of ShardingDatabase properties: InvitedNodeSubnet: type: string catalog: items: - description: CatalogSpec defines the desired state of CatalogSpec properties: envVars: items: - description: EnvironmentVariable represents a named variable accessible for containers. properties: name: type: string @@ -2081,7 +11914,6 @@ spec: type: object type: array imagePullPolicy: - description: PullPolicy describes a policy for if/when to pull a container image type: string isDelete: type: string @@ -2104,15 +11936,13 @@ spec: pvcName: type: string resources: - description: ResourceRequirements describes the compute resource requirements. properties: claims: - description: "Claims lists the names of resources, defined in spec.resourceClaims, that are used by this container. \n This is an alpha field and requires enabling the DynamicResourceAllocation feature gate. \n This field is immutable. It can only be set for containers." items: - description: ResourceClaim references one entry in PodSpec.ResourceClaims. properties: name: - description: Name must match the name of one entry in pod.spec.resourceClaims of the Pod where this field is used. It makes that resource available inside a container. + type: string + request: type: string required: - name @@ -2128,7 +11958,6 @@ spec: - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true - description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' type: object requests: additionalProperties: @@ -2137,9 +11966,14 @@ spec: - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true - description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. Requests cannot exceed Limits. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' type: object type: object + shardGroup: + type: string + shardRegion: + type: string + shardSpace: + type: string storageSizeInGb: format: int32 type: integer @@ -2154,7 +11988,6 @@ spec: dbImagePullSecret: type: string dbSecret: - description: Secret Details properties: encryptionType: type: string @@ -2182,14 +12015,11 @@ spec: type: string gsm: items: - description: GsmSpec defines the desired state of GsmSpec properties: directorName: type: string envVars: - description: Replicas int32 `json:"replicas,omitempty"` // Gsm Replicas. If you set OraGsmPvcName then it is set default to 1. items: - description: EnvironmentVariable represents a named variable accessible for containers. properties: name: type: string @@ -2201,7 +12031,6 @@ spec: type: object type: array imagePullPolicy: - description: PullPolicy describes a policy for if/when to pull a container image type: string isDelete: type: string @@ -2213,6 +12042,10 @@ spec: additionalProperties: type: string type: object + pvAnnotations: + additionalProperties: + type: string + type: object pvMatchLabels: additionalProperties: type: string @@ -2222,15 +12055,13 @@ spec: region: type: string resources: - description: ResourceRequirements describes the compute resource requirements. properties: claims: - description: "Claims lists the names of resources, defined in spec.resourceClaims, that are used by this container. \n This is an alpha field and requires enabling the DynamicResourceAllocation feature gate. \n This field is immutable. It can only be set for containers." items: - description: ResourceClaim references one entry in PodSpec.ResourceClaims. properties: name: - description: Name must match the name of one entry in pod.spec.resourceClaims of the Pod where this field is used. It makes that resource available inside a container. + type: string + request: type: string required: - name @@ -2246,7 +12077,6 @@ spec: - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true - description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' type: object requests: additionalProperties: @@ -2255,7 +12085,6 @@ spec: - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true - description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. Requests cannot exceed Limits. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' type: object type: object storageSizeInGb: @@ -2273,7 +12102,6 @@ spec: type: string gsmService: items: - description: Service Definition properties: available: type: string @@ -2352,227 +12180,493 @@ spec: type: array gsmShardSpace: items: - description: ShardSpace Specs properties: chunks: type: integer - name: + name: + type: string + protectionMode: + type: string + shardGroup: + type: string + required: + - name + type: object + type: array + invitedNodeSubnetFlag: + type: string + isClone: + type: boolean + isDataGuard: + type: boolean + isDebug: + type: boolean + isDeleteOraPvc: + type: boolean + isDownloadScripts: + type: boolean + isExternalSvc: + type: boolean + isTdeWallet: + type: string + liveinessCheckPeriod: + type: integer + portMappings: + items: + properties: + port: + format: int32 + type: integer + protocol: + type: string + targetPort: + format: int32 + type: integer + required: + - port + - protocol + - targetPort + type: object + type: array + readinessCheckPeriod: + type: integer + replicationType: + type: string + scriptsLocation: + type: string + shard: + items: + properties: + deployAs: + type: string + envVars: + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + imagePullPolicy: + type: string + isDelete: + enum: + - enable + - disable + - failed + - force + type: string + label: + type: string + name: + type: string + nodeSelector: + additionalProperties: + type: string + type: object + pvAnnotations: + additionalProperties: + type: string + type: object + pvMatchLabels: + additionalProperties: + type: string + type: object + pvcName: + type: string + resources: + properties: + claims: + items: + properties: + name: + type: string + request: + type: string + required: + - name + type: object + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + limits: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + requests: + additionalProperties: + anyOf: + - type: integer + - type: string + pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ + x-kubernetes-int-or-string: true + type: object + type: object + shardGroup: + type: string + shardRegion: + type: string + shardSpace: + type: string + storageSizeInGb: + format: int32 + type: integer + required: + - name + type: object + type: array + shardBuddyRegion: + type: string + shardConfigName: + type: string + shardRegion: + items: + type: string + type: array + shardingType: + type: string + stagePvcName: + type: string + storageClass: + type: string + tdeWalletPvc: + type: string + tdeWalletPvcMountLocation: + type: string + topicId: + type: string + required: + - catalog + - dbImage + - gsm + - gsmImage + - shard + type: object + status: + properties: + catalogs: + additionalProperties: + type: string + type: object + conditions: + items: + properties: + lastTransitionTime: + format: date-time + type: string + message: + maxLength: 32768 + type: string + observedGeneration: + format: int64 + minimum: 0 + type: integer + reason: + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ type: string - protectionMode: + status: + enum: + - "True" + - "False" + - Unknown type: string - shardGroup: + type: + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ type: string required: - - name + - lastTransitionTime + - message + - reason + - status + - type type: object type: array - invitedNodeSubnetFlag: - type: string - isClone: - type: boolean - isDataGuard: + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + gsm: + properties: + details: + additionalProperties: + type: string + type: object + externalConnectStr: + type: string + internalConnectStr: + type: string + services: + type: string + shards: + additionalProperties: + type: string + type: object + state: + type: string + type: object + shards: + additionalProperties: + type: string + type: object + type: object + type: object + served: true + storage: true + subresources: + status: {} +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + cert-manager.io/inject-ca-from: oracle-database-operator-system/oracle-database-operator-serving-cert + controller-gen.kubebuilder.io/version: v0.16.5 + name: singleinstancedatabases.database.oracle.com +spec: + group: database.oracle.com + names: + kind: SingleInstanceDatabase + listKind: SingleInstanceDatabaseList + plural: singleinstancedatabases + singular: singleinstancedatabase + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .status.edition + name: Edition + type: string + - jsonPath: .status.sid + name: Sid + priority: 1 + type: string + - jsonPath: .status.status + name: Status + type: string + - jsonPath: .status.role + name: Role + type: string + - jsonPath: .status.releaseUpdate + name: Version + type: string + - jsonPath: .status.connectString + name: Connect Str + type: string + - jsonPath: .status.pdbConnectString + name: Pdb Connect Str + priority: 1 + type: string + - jsonPath: .status.tcpsConnectString + name: TCPS Connect Str + type: string + - jsonPath: .status.tcpsPdbConnectString + name: TCPS Pdb Connect Str + priority: 1 + type: string + - jsonPath: .status.oemExpressUrl + name: Oem Express Url + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + properties: + apiVersion: + type: string + kind: + type: string + metadata: + type: object + spec: + properties: + adminPassword: + properties: + keepSecret: + type: boolean + secretKey: + default: oracle_pwd + type: string + secretName: + type: string + required: + - secretName + type: object + archiveLog: type: boolean - isDebug: + charset: + type: string + convertToSnapshotStandby: type: boolean - isDeleteOraPvc: + createAs: + enum: + - primary + - standby + - clone + - truecache + type: string + edition: + enum: + - standard + - enterprise + - express + - free + type: string + enableTCPS: type: boolean - isDownloadScripts: + flashBack: type: boolean - isExternalSvc: + forceLog: type: boolean - isTdeWallet: - type: string - liveinessCheckPeriod: - type: integer - namespace: - type: string - portMappings: - items: - description: PortMapping is a specification of port mapping for an application deployment. - properties: - port: - format: int32 - type: integer - protocol: - default: TCP - type: string - targetPort: - format: int32 - type: integer - required: - - port - - protocol - - targetPort - type: object - type: array - readinessCheckPeriod: + image: + properties: + prebuiltDB: + type: boolean + pullFrom: + type: string + pullSecrets: + type: string + version: + type: string + required: + - pullFrom + type: object + initParams: + properties: + cpuCount: + type: integer + pgaAggregateTarget: + type: integer + processes: + type: integer + sgaTarget: + type: integer + type: object + listenerPort: type: integer - replicationType: - type: string - scriptsLocation: + loadBalancer: + type: boolean + nodeSelector: + additionalProperties: + type: string + type: object + pdbName: type: string - shard: - description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster Important: Run "make" to regenerate code after modifying this file' - items: - description: ShardSpec is a specification of Shards for an application deployment. - properties: - deployAs: - type: string - envVars: - items: - description: EnvironmentVariable represents a named variable accessible for containers. - properties: - name: - type: string - value: - type: string - required: - - name - - value - type: object - type: array - imagePullPolicy: - description: PullPolicy describes a policy for if/when to pull a container image - type: string - isDelete: - enum: - - enable - - disable - - failed - - force - type: string - label: - type: string - name: - type: string - nodeSelector: - additionalProperties: + persistence: + properties: + accessMode: + enum: + - ReadWriteOnce + - ReadWriteMany + type: string + datafilesVolumeName: + type: string + scriptsVolumeName: + type: string + setWritePermissions: + type: boolean + size: + type: string + storageClass: + type: string + volumeClaimAnnotation: + type: string + type: object + primaryDatabaseRef: + type: string + readinessCheckPeriod: + type: integer + replicas: + type: integer + resources: + properties: + limits: + properties: + cpu: type: string - type: object - pvAnnotations: - additionalProperties: + memory: type: string - type: object - pvMatchLabels: - additionalProperties: + type: object + requests: + properties: + cpu: type: string - type: object - pvcName: - type: string - resources: - description: ResourceRequirements describes the compute resource requirements. - properties: - claims: - description: "Claims lists the names of resources, defined in spec.resourceClaims, that are used by this container. \n This is an alpha field and requires enabling the DynamicResourceAllocation feature gate. \n This field is immutable. It can only be set for containers." - items: - description: ResourceClaim references one entry in PodSpec.ResourceClaims. - properties: - name: - description: Name must match the name of one entry in pod.spec.resourceClaims of the Pod where this field is used. It makes that resource available inside a container. - type: string - required: - - name - type: object - type: array - x-kubernetes-list-map-keys: - - name - x-kubernetes-list-type: map - limits: - additionalProperties: - anyOf: - - type: integer - - type: string - pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ - x-kubernetes-int-or-string: true - description: 'Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' - type: object - requests: - additionalProperties: - anyOf: - - type: integer - - type: string - pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ - x-kubernetes-int-or-string: true - description: 'Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. Requests cannot exceed Limits. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/' - type: object - type: object - shardGroup: - type: string - shardRegion: - type: string - shardSpace: - type: string - storageSizeInGb: - format: int32 - type: integer - required: - - name - type: object - type: array - shardBuddyRegion: + memory: + type: string + type: object + type: object + serviceAccountName: type: string - shardConfigName: + serviceAnnotations: + additionalProperties: + type: string + type: object + sid: + maxLength: 12 + pattern: ^[a-zA-Z0-9]+$ type: string - shardRegion: + tcpsCertRenewInterval: + type: string + tcpsListenerPort: + type: integer + tcpsTlsSecret: + type: string + trueCacheServices: items: type: string type: array - shardingType: + required: + - image + type: object + status: + properties: + apexInstalled: + type: boolean + archiveLog: type: string - stagePvcName: + certCreationTimestamp: type: string - storageClass: + certRenewInterval: type: string - tdeWalletPvc: + charset: type: string - tdeWalletPvcMountLocation: + clientWalletLoc: + type: string + clusterConnectString: type: string - required: - - catalog - - dbImage - - gsm - - gsmImage - - shard - type: object - status: - description: To understand Metav1.Condition, please refer the link https://pkg.go.dev/k8s.io/apimachinery/pkg/apis/meta/v1 ShardingDatabaseStatus defines the observed state of ShardingDatabase - properties: - catalogs: - additionalProperties: - type: string - type: object conditions: items: - description: "Condition contains details for one aspect of the current state of this API Resource. --- This struct is intended for direct use as an array at the field path .status.conditions. For example, \n \ttype FooStatus struct{ \t // Represents the observations of a foo's current state. \t // Known .status.conditions.type are: \"Available\", \"Progressing\", and \"Degraded\" \t // +patchMergeKey=type \t // +patchStrategy=merge \t // +listType=map \t // +listMapKey=type \t Conditions []metav1.Condition `json:\"conditions,omitempty\" patchStrategy:\"merge\" patchMergeKey:\"type\" protobuf:\"bytes,1,rep,name=conditions\"` \n \t // other fields \t}" properties: lastTransitionTime: - description: lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. format: date-time type: string message: - description: message is a human readable message indicating details about the transition. This may be an empty string. maxLength: 32768 type: string observedGeneration: - description: observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance. format: int64 minimum: 0 type: integer reason: - description: reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty. maxLength: 1024 minLength: 1 pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ type: string status: - description: status of the condition, one of True, False, Unknown. enum: - "True" - "False" - Unknown type: string type: - description: type of condition in CamelCase or in foo.example.com/CamelCase. --- Many .condition.type values are consistent across resources like Available, but because arbitrary conditions can be useful (see .node.status.conditions), the ability to deconflict is important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) maxLength: 316 pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ type: string @@ -2587,58 +12681,114 @@ spec: x-kubernetes-list-map-keys: - type x-kubernetes-list-type: map - gsm: + connectString: + type: string + convertToSnapshotStandby: + type: boolean + createdAs: + type: string + datafilesCreated: + default: "false" + type: string + datafilesPatched: + default: "false" + type: string + dgBroker: + type: string + edition: + type: string + flashBack: + type: string + forceLog: + type: string + initParams: properties: - details: - additionalProperties: - type: string - type: object - externalConnectStr: + cpuCount: + type: integer + pgaAggregateTarget: + type: integer + processes: + type: integer + sgaTarget: + type: integer + type: object + initPgaSize: + type: integer + initSgaSize: + type: integer + isTcpsEnabled: + default: false + type: boolean + nodes: + items: + type: string + type: array + oemExpressUrl: + type: string + ordsReference: + type: string + pdbConnectString: + type: string + pdbName: + type: string + persistence: + properties: + accessMode: + enum: + - ReadWriteOnce + - ReadWriteMany type: string - internalConnectStr: + datafilesVolumeName: type: string - services: + scriptsVolumeName: type: string - shards: - additionalProperties: - type: string - type: object - state: + setWritePermissions: + type: boolean + size: + type: string + storageClass: + type: string + volumeClaimAnnotation: type: string type: object - shards: + prebuiltDB: + type: boolean + primaryDatabase: + type: string + releaseUpdate: + type: string + replicas: + type: integer + role: + type: string + sid: + type: string + standbyDatabases: additionalProperties: type: string type: object + status: + type: string + tcpsConnectString: + type: string + tcpsPdbConnectString: + type: string + tcpsTlsSecret: + default: "" + type: string + required: + - isTcpsEnabled + - persistence + - tcpsTlsSecret type: object - type: object - served: true - storage: true - subresources: - status: {} -status: - acceptedNames: - kind: "" - plural: "" - conditions: [] - storedVersions: [] ---- -apiVersion: apiextensions.k8s.io/v1 -kind: CustomResourceDefinition -metadata: - annotations: - cert-manager.io/inject-ca-from: oracle-database-operator-system/oracle-database-operator-serving-cert - controller-gen.kubebuilder.io/version: v0.6.1 - name: singleinstancedatabases.database.oracle.com -spec: - group: database.oracle.com - names: - kind: SingleInstanceDatabase - listKind: SingleInstanceDatabaseList - plural: singleinstancedatabases - singular: singleinstancedatabase - scope: Namespaced - versions: + type: object + served: true + storage: false + subresources: + scale: + specReplicasPath: .spec.replicas + statusReplicasPath: .status.replicas + status: {} - additionalPrinterColumns: - jsonPath: .status.edition name: Edition @@ -2673,24 +12823,19 @@ spec: - jsonPath: .status.oemExpressUrl name: Oem Express Url type: string - name: v1alpha1 + name: v4 schema: openAPIV3Schema: - description: SingleInstanceDatabase is the Schema for the singleinstancedatabases API properties: apiVersion: - description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: - description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: - description: SingleInstanceDatabaseSpec defines the desired state of SingleInstanceDatabase properties: adminPassword: - description: SingleInsatnceAdminPassword defines the secret containing Admin Password mapped to secretKey for Database properties: keepSecret: type: boolean @@ -2706,14 +12851,15 @@ spec: type: boolean charset: type: string + convertToSnapshotStandby: + type: boolean createAs: enum: - primary - standby - clone + - truecache type: string - dgBrokerConfigured: - type: boolean edition: enum: - standard @@ -2728,7 +12874,6 @@ spec: forceLog: type: boolean image: - description: SingleInstanceDatabaseImage defines the Image source and pullSecrets for POD properties: prebuiltDB: type: boolean @@ -2742,7 +12887,6 @@ spec: - pullFrom type: object initParams: - description: SingleInstanceDatabaseInitParams defines the Init Parameters properties: cpuCount: type: integer @@ -2764,7 +12908,6 @@ spec: pdbName: type: string persistence: - description: SingleInstanceDatabasePersistence defines the storage size and class for PVC properties: accessMode: enum: @@ -2814,7 +12957,6 @@ spec: type: string type: object sid: - description: SID must be alphanumeric (no special characters, only a-z, A-Z, 0-9), and no longer than 12 characters. maxLength: 12 pattern: ^[a-zA-Z0-9]+$ type: string @@ -2824,11 +12966,14 @@ spec: type: integer tcpsTlsSecret: type: string + trueCacheServices: + items: + type: string + type: array required: - image type: object status: - description: SingleInstanceDatabaseStatus defines the observed state of SingleInstanceDatabase properties: apexInstalled: type: boolean @@ -2846,36 +12991,29 @@ spec: type: string conditions: items: - description: "Condition contains details for one aspect of the current state of this API Resource. --- This struct is intended for direct use as an array at the field path .status.conditions. For example, \n \ttype FooStatus struct{ \t // Represents the observations of a foo's current state. \t // Known .status.conditions.type are: \"Available\", \"Progressing\", and \"Degraded\" \t // +patchMergeKey=type \t // +patchStrategy=merge \t // +listType=map \t // +listMapKey=type \t Conditions []metav1.Condition `json:\"conditions,omitempty\" patchStrategy:\"merge\" patchMergeKey:\"type\" protobuf:\"bytes,1,rep,name=conditions\"` \n \t // other fields \t}" properties: lastTransitionTime: - description: lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. format: date-time type: string message: - description: message is a human readable message indicating details about the transition. This may be an empty string. maxLength: 32768 type: string observedGeneration: - description: observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance. format: int64 minimum: 0 type: integer reason: - description: reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty. maxLength: 1024 minLength: 1 pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ type: string status: - description: status of the condition, one of True, False, Unknown. enum: - "True" - "False" - Unknown type: string type: - description: type of condition in CamelCase or in foo.example.com/CamelCase. --- Many .condition.type values are consistent across resources like Available, but because arbitrary conditions can be useful (see .node.status.conditions), the ability to deconflict is important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt) maxLength: 316 pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ type: string @@ -2892,6 +13030,8 @@ spec: x-kubernetes-list-type: map connectString: type: string + convertToSnapshotStandby: + type: boolean createdAs: type: string datafilesCreated: @@ -2900,8 +13040,8 @@ spec: datafilesPatched: default: "false" type: string - dgBrokerConfigured: - type: boolean + dgBroker: + type: string edition: type: string flashBack: @@ -2909,7 +13049,6 @@ spec: forceLog: type: string initParams: - description: SingleInstanceDatabaseInitParams defines the Init Parameters properties: cpuCount: type: integer @@ -2940,7 +13079,6 @@ spec: pdbName: type: string persistence: - description: SingleInstanceDatabasePersistence defines the storage size and class for PVC properties: accessMode: enum: @@ -2998,12 +13136,6 @@ spec: specReplicasPath: .spec.replicas statusReplicasPath: .status.replicas status: {} -status: - acceptedNames: - kind: "" - plural: "" - conditions: [] - storedVersions: [] --- apiVersion: rbac.authorization.k8s.io/v1 kind: Role @@ -3021,360 +13153,56 @@ rules: - watch - create - update - - patch - - delete -- apiGroups: - - coordination.k8s.io - resources: - - leases - verbs: - - get - - list - - watch - - create - - update - - patch - - delete -- apiGroups: - - "" - resources: - - configmaps/status - verbs: - - get - - update - - patch -- apiGroups: - - "" - resources: - - events - verbs: - - create - - patch ---- -apiVersion: rbac.authorization.k8s.io/v1 -kind: ClusterRole -metadata: - creationTimestamp: null - name: oracle-database-operator-manager-role -rules: -- apiGroups: - - "" - resources: - - configmaps - - deployments - - events - - pods - - secrets - - services - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - "" - resources: - - configmaps - - events - - pods - - pods/exec - - pods/log - - replicasets - - services - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - "" - resources: - - configmaps - - namespaces - - secrets - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - "" - resources: - - configmaps - - secrets - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - "" - resources: - - deployments - - events - - pods - - services - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - "" - resources: - - events - verbs: - - create - - patch -- apiGroups: - - "" - resources: - - persistentvolumeclaims - - pods - - pods/exec - - pods/log - - services - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - "" - resources: - - persistentvolumes - verbs: - - get - - list - - watch -- apiGroups: - - '''''' - resources: - - statefulsets/finalizers - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - apps - resources: - - configmaps - verbs: - - get - - list -- apiGroups: - - apps - resources: - - deployments - - pods - - services - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - apps - resources: - - replicasets - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - apps - resources: - - statefulsets - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - coordination.k8s.io - resources: - - leases - verbs: - - create - - get - - list - - update -- apiGroups: - - "" - resources: - - configmaps - - containers - - events - - namespaces - - persistentvolumeclaims - - pods - - pods/exec - - pods/log - - secrets - - services - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - "" - resources: - - configmaps - - containers - - events - - namespaces - - pods - - pods/exec - - pods/log - - secrets - - services - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - "" - resources: - - configmaps - - namespaces - - pods - - secrets - - services - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - "" - resources: - - pods/exec - verbs: - - create + - patch + - delete - apiGroups: - - database.oracle.com + - coordination.k8s.io resources: - - autonomouscontainerdatabases + - leases verbs: - - create - - delete - get - list - - patch - - update - watch -- apiGroups: - - database.oracle.com - resources: - - autonomouscontainerdatabases/status - verbs: - - get - - patch - - update -- apiGroups: - - database.oracle.com - resources: - - autonomousdatabasebackups - verbs: - create - - delete - - get - - list - update - - watch -- apiGroups: - - database.oracle.com - resources: - - autonomousdatabasebackups/status - verbs: - - get - patch - - update -- apiGroups: - - database.oracle.com - resources: - - autonomousdatabaserestores - verbs: - - create - delete - - get - - list - - update - - watch - apiGroups: - - database.oracle.com + - "" resources: - - autonomousdatabaserestores/status + - configmaps/status verbs: - get - - patch - update -- apiGroups: - - database.oracle.com - resources: - - autonomousdatabases - verbs: - - create - - delete - - get - - list - patch - - update - - watch - apiGroups: - - database.oracle.com + - "" resources: - - autonomousdatabases/status + - events verbs: + - create - patch - - update +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: oracle-database-operator-manager-role +rules: - apiGroups: - - database.oracle.com + - "" resources: - - cdbs + - configmaps + - containers + - deployments + - events + - namespaces + - persistentvolumeclaims + - pods + - pods/exec + - pods/log + - replicasets + - secrets + - services verbs: - create - delete @@ -3384,49 +13212,35 @@ rules: - update - watch - apiGroups: - - database.oracle.com - resources: - - cdbs/finalizers - verbs: - - update -- apiGroups: - - database.oracle.com + - "" resources: - - cdbs/status + - configmaps/status + - daemonsets/status + - deployments/status + - services/status + - statefulsets/status verbs: - get - patch - update - apiGroups: - - database.oracle.com + - "" resources: - - dataguardbrokers + - persistentvolumes verbs: - - create - - delete - get - list - - patch - - update - watch - apiGroups: - - database.oracle.com - resources: - - dataguardbrokers/finalizers - verbs: - - update -- apiGroups: - - database.oracle.com + - "" resources: - - dataguardbrokers/status + - secrets/status verbs: - get - - patch - - update - apiGroups: - - database.oracle.com + - '''''' resources: - - dbcssystems + - statefulsets/finalizers verbs: - create - delete @@ -3436,27 +13250,21 @@ rules: - update - watch - apiGroups: - - database.oracle.com - resources: - - dbcssystems/finalizers - verbs: - - create - - delete - - get - - patch - - update -- apiGroups: - - database.oracle.com + - apps resources: - - dbcssystems/status + - configmaps verbs: - get - - patch - - update + - list - apiGroups: - - database.oracle.com + - apps resources: - - oraclerestdataservices + - daemonsets + - deployments + - pods + - replicasets + - services + - statefulsets verbs: - create - delete @@ -3466,23 +13274,30 @@ rules: - update - watch - apiGroups: - - database.oracle.com - resources: - - oraclerestdataservices/finalizers - verbs: - - update -- apiGroups: - - database.oracle.com + - coordination.k8s.io resources: - - oraclerestdataservices/status + - leases verbs: + - create - get - - patch + - list - update - apiGroups: - database.oracle.com resources: + - autonomouscontainerdatabases + - autonomousdatabases + - cdbs + - dataguardbrokers + - dbcssystems + - events + - lrests + - lrpdbs + - oraclerestdataservices + - ordssrvs - pdbs + - shardingdatabases + - singleinstancedatabases verbs: - create - delete @@ -3494,17 +13309,19 @@ rules: - apiGroups: - database.oracle.com resources: - - pdbs/finalizers - verbs: - - create - - delete - - get - - patch - - update -- apiGroups: - - database.oracle.com - resources: + - autonomouscontainerdatabases/status + - autonomousdatabasebackups/status + - autonomousdatabaserestores/status + - cdbs/status + - dataguardbrokers/status + - dbcssystems/status + - lrests/status + - lrpdbs/status + - oraclerestdataservices/status + - ordssrvs/status - pdbs/status + - shardingdatabases/status + - singleinstancedatabases/status verbs: - get - patch @@ -3512,57 +13329,44 @@ rules: - apiGroups: - database.oracle.com resources: - - shardingdatabases + - autonomousdatabasebackups + - autonomousdatabaserestores verbs: - create - delete - get - list - - patch - update - watch - apiGroups: - database.oracle.com resources: - - shardingdatabases/finalizers + - autonomousdatabases/status verbs: - - create - - delete - - get - patch - update - apiGroups: - database.oracle.com resources: - - shardingdatabases/status + - cdbs/finalizers + - dataguardbrokers/finalizers + - lrests/finalizers + - oraclerestdataservices/finalizers + - ordssrvs/finalizers + - singleinstancedatabases/finalizers verbs: - - get - - patch - update - apiGroups: - database.oracle.com resources: - - singleinstancedatabases + - dbcssystems/finalizers + - lrpdbs/finalizers + - pdbs/finalizers + - shardingdatabases/finalizers verbs: - create - delete - get - - list - - patch - - update - - watch -- apiGroups: - - database.oracle.com - resources: - - singleinstancedatabases/finalizers - verbs: - - update -- apiGroups: - - database.oracle.com - resources: - - singleinstancedatabases/status - verbs: - - get - patch - update - apiGroups: @@ -3744,19 +13548,40 @@ webhooks: service: name: oracle-database-operator-webhook-service namespace: oracle-database-operator-system - path: /mutate-database-oracle-com-v1alpha1-autonomousdatabase + path: /mutate-database-oracle-com-v4-autonomousdatabasebackup + failurePolicy: Fail + name: mautonomousdatabasebackupv4.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v4 + operations: + - CREATE + - UPDATE + resources: + - autonomousdatabasebackups + sideEffects: None +- admissionReviewVersions: + - v1 + - v1beta1 + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /mutate-database-oracle-com-v4-cdb failurePolicy: Fail - name: mautonomousdatabase.kb.io + name: mcdb.kb.io rules: - apiGroups: - database.oracle.com apiVersions: - - v1alpha1 + - v4 operations: - CREATE - UPDATE resources: - - autonomousdatabases + - cdbs sideEffects: None - admissionReviewVersions: - v1 @@ -3764,19 +13589,61 @@ webhooks: service: name: oracle-database-operator-webhook-service namespace: oracle-database-operator-system - path: /mutate-database-oracle-com-v1alpha1-autonomousdatabasebackup + path: /mutate-database-oracle-com-v4-dbcssystem failurePolicy: Fail - name: mautonomousdatabasebackup.kb.io + name: mdbcssystemv4.kb.io rules: - apiGroups: - database.oracle.com apiVersions: - - v1alpha1 + - v4 operations: - CREATE - UPDATE resources: - - autonomousdatabasebackups + - dbcssystems + sideEffects: None +- admissionReviewVersions: + - v4 + - v1beta1 + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /mutate-database-oracle-com-v4-lrest + failurePolicy: Fail + name: mlrest.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v4 + operations: + - CREATE + - UPDATE + resources: + - lrests + sideEffects: None +- admissionReviewVersions: + - v4 + - v1beta1 + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /mutate-database-oracle-com-v4-lrpdb + failurePolicy: Fail + name: mlrpdb.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v4 + operations: + - CREATE + - UPDATE + resources: + - lrpdbs sideEffects: None - admissionReviewVersions: - v1 @@ -3785,9 +13652,49 @@ webhooks: service: name: oracle-database-operator-webhook-service namespace: oracle-database-operator-system - path: /mutate-database-oracle-com-v1alpha1-cdb + path: /mutate-database-oracle-com-v4-pdb failurePolicy: Fail - name: mcdb.kb.io + name: mpdb.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v4 + operations: + - CREATE + - UPDATE + resources: + - pdbs + sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /mutate-database-oracle-com-v4-shardingdatabase + failurePolicy: Fail + name: mshardingdatabasev4.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v4 + operations: + - CREATE + - UPDATE + resources: + - shardingdatabases + sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /mutate-database-oracle-com-v1alpha1-autonomousdatabasebackup + failurePolicy: Fail + name: mautonomousdatabasebackupv1alpha1.kb.io rules: - apiGroups: - database.oracle.com @@ -3797,7 +13704,7 @@ webhooks: - CREATE - UPDATE resources: - - cdbs + - autonomousdatabasebackups sideEffects: None - admissionReviewVersions: - v1 @@ -3820,6 +13727,26 @@ webhooks: resources: - dataguardbrokers sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /mutate-database-oracle-com-v4-dbcssystem + failurePolicy: Fail + name: mdbcssystemv1alpha1.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v4 + operations: + - CREATE + - UPDATE + resources: + - dbcssystems + sideEffects: None - admissionReviewVersions: - v1 - v1beta1 @@ -3843,14 +13770,13 @@ webhooks: sideEffects: None - admissionReviewVersions: - v1 - - v1beta1 clientConfig: service: name: oracle-database-operator-webhook-service namespace: oracle-database-operator-system - path: /mutate-database-oracle-com-v1alpha1-pdb + path: /mutate-database-oracle-com-v1alpha1-shardingdatabase failurePolicy: Fail - name: mpdb.kb.io + name: mshardingdatabasev1alpha1.kb.io rules: - apiGroups: - database.oracle.com @@ -3860,7 +13786,7 @@ webhooks: - CREATE - UPDATE resources: - - pdbs + - shardingdatabases sideEffects: None - admissionReviewVersions: - v1 @@ -3909,14 +13835,14 @@ webhooks: service: name: oracle-database-operator-webhook-service namespace: oracle-database-operator-system - path: /mutate-observability-oracle-com-v1alpha1-databaseobserver + path: /mutate-observability-oracle-com-v4-databaseobserver failurePolicy: Fail name: mdatabaseobserver.kb.io rules: - apiGroups: - observability.oracle.com apiVersions: - - v1alpha1 + - v4 operations: - CREATE - UPDATE @@ -3937,14 +13863,14 @@ webhooks: service: name: oracle-database-operator-webhook-service namespace: oracle-database-operator-system - path: /validate-database-oracle-com-v1alpha1-autonomouscontainerdatabase + path: /validate-database-oracle-com-v4-autonomouscontainerdatabase failurePolicy: Fail - name: vautonomouscontainerdatabase.kb.io + name: vautonomouscontainerdatabasev4.kb.io rules: - apiGroups: - database.oracle.com apiVersions: - - v1alpha1 + - v4 operations: - CREATE - UPDATE @@ -3957,9 +13883,154 @@ webhooks: service: name: oracle-database-operator-webhook-service namespace: oracle-database-operator-system - path: /validate-database-oracle-com-v1alpha1-autonomousdatabase + path: /validate-database-oracle-com-v4-autonomousdatabasebackup + failurePolicy: Fail + name: vautonomousdatabasebackupv4.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v4 + operations: + - CREATE + - UPDATE + resources: + - autonomousdatabasebackups + sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /validate-database-oracle-com-v4-autonomousdatabaserestore + failurePolicy: Fail + name: vautonomousdatabaserestorev4.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v4 + operations: + - CREATE + - UPDATE + resources: + - autonomousdatabaserestores + sideEffects: None +- admissionReviewVersions: + - v1 + - v1beta1 + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /validate-database-oracle-com-v4-cdb + failurePolicy: Fail + name: vcdb.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v4 + operations: + - CREATE + - UPDATE + resources: + - cdbs + sideEffects: None +- admissionReviewVersions: + - v4 + - v1beta1 + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /validate-database-oracle-com-v4-lrest + failurePolicy: Fail + name: vlrest.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v4 + operations: + - CREATE + - UPDATE + resources: + - lrests + sideEffects: None +- admissionReviewVersions: + - v4 + - v1beta1 + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /validate-database-oracle-com-v4-lrpdb + failurePolicy: Fail + name: vlrpdb.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v4 + operations: + - CREATE + - UPDATE + resources: + - lrpdbs + sideEffects: None +- admissionReviewVersions: + - v1 + - v1beta1 + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /validate-database-oracle-com-v4-pdb + failurePolicy: Fail + name: vpdb.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v4 + operations: + - CREATE + - UPDATE + resources: + - pdbs + sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /validate-database-oracle-com-v4-shardingdatabase + failurePolicy: Fail + name: vshardingdatabasev4.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v4 + operations: + - CREATE + - UPDATE + - DELETE + resources: + - shardingdatabases + sideEffects: None +- admissionReviewVersions: + - v1 + clientConfig: + service: + name: oracle-database-operator-webhook-service + namespace: oracle-database-operator-system + path: /validate-database-oracle-com-v1alpha1-autonomouscontainerdatabase failurePolicy: Fail - name: vautonomousdatabase.kb.io + name: vautonomouscontainerdatabasev1alpha1.kb.io rules: - apiGroups: - database.oracle.com @@ -3969,7 +14040,7 @@ webhooks: - CREATE - UPDATE resources: - - autonomousdatabases + - autonomouscontainerdatabases sideEffects: None - admissionReviewVersions: - v1 @@ -3979,7 +14050,7 @@ webhooks: namespace: oracle-database-operator-system path: /validate-database-oracle-com-v1alpha1-autonomousdatabasebackup failurePolicy: Fail - name: vautonomousdatabasebackup.kb.io + name: vautonomousdatabasebackupv1alpha1.kb.io rules: - apiGroups: - database.oracle.com @@ -3999,7 +14070,7 @@ webhooks: namespace: oracle-database-operator-system path: /validate-database-oracle-com-v1alpha1-autonomousdatabaserestore failurePolicy: Fail - name: vautonomousdatabaserestore.kb.io + name: vautonomousdatabaserestorev1alpha1.kb.io rules: - apiGroups: - database.oracle.com @@ -4013,14 +14084,13 @@ webhooks: sideEffects: None - admissionReviewVersions: - v1 - - v1beta1 clientConfig: service: name: oracle-database-operator-webhook-service namespace: oracle-database-operator-system - path: /validate-database-oracle-com-v1alpha1-cdb + path: /validate-database-oracle-com-v1alpha1-autonomousdatabase failurePolicy: Fail - name: vcdb.kb.io + name: vautonomousdatabasev1alpha1.kb.io rules: - apiGroups: - database.oracle.com @@ -4030,7 +14100,7 @@ webhooks: - CREATE - UPDATE resources: - - cdbs + - autonomousdatabases sideEffects: None - admissionReviewVersions: - v1 @@ -4076,14 +14146,13 @@ webhooks: sideEffects: None - admissionReviewVersions: - v1 - - v1beta1 clientConfig: service: name: oracle-database-operator-webhook-service namespace: oracle-database-operator-system - path: /validate-database-oracle-com-v1alpha1-pdb + path: /validate-database-oracle-com-v1alpha1-shardingdatabase failurePolicy: Fail - name: vpdb.kb.io + name: vshardingdatabasev1alpha1.kb.io rules: - apiGroups: - database.oracle.com @@ -4092,8 +14161,9 @@ webhooks: operations: - CREATE - UPDATE + - DELETE resources: - - pdbs + - shardingdatabases sideEffects: None - admissionReviewVersions: - v1 @@ -4144,14 +14214,14 @@ webhooks: service: name: oracle-database-operator-webhook-service namespace: oracle-database-operator-system - path: /validate-observability-oracle-com-v1alpha1-databaseobserver + path: /validate-observability-oracle-com-v4-databaseobserver failurePolicy: Fail name: vdatabaseobserver.kb.io rules: - apiGroups: - observability.oracle.com apiVersions: - - v1alpha1 + - v4 operations: - CREATE - UPDATE @@ -4184,7 +14254,7 @@ spec: env: - name: WATCH_NAMESPACE value: "" - image: container-registry.oracle.com/database/operator:latest + image: container-registry.oracle.com/database/operator:1.2.0 imagePullPolicy: Always name: manager ports: diff --git a/ords/Dockerfile b/ords/Dockerfile index 772a7e6d..25ba08ec 100644 --- a/ords/Dockerfile +++ b/ords/Dockerfile @@ -40,7 +40,9 @@ FROM container-registry.oracle.com/java/jdk:latest # ------------------------------------------------------------- ENV ORDS_HOME=/opt/oracle/ords/ \ RUN_FILE="runOrdsSSL.sh" \ - ORDSVERSION=23.4.0-8 + ORDSVERSION=23.4.0-8 \ + JAVA=17 +#see https://www.oracle.com/tools/ords/ords-relnotes-23.4.0.html # Copy binaries # ------------- @@ -48,7 +50,7 @@ COPY $RUN_FILE $ORDS_HOME RUN yum -y install yum-utils bind-utils tree hostname openssl net-tools zip unzip tar wget vim-minimal which sudo expect procps curl lsof && \ yum-config-manager --add-repo=http://yum.oracle.com/repo/OracleLinux/OL8/oracle/software/x86_64 && \ - yum -y install java-11-openjdk-devel && \ + yum -y install java-$JAVA-openjdk-devel && \ yum -y install iproute && \ yum clean all @@ -64,14 +66,18 @@ RUN mkdir -p $ORDS_HOME/doc_root && \ chmod ug+x $ORDS_HOME/*.sh && \ groupadd -g 54322 dba && \ usermod -u 54321 -d /home/oracle -g dba -m -s /bin/bash oracle && \ - chown -R oracle:dba $ORDS_HOME && \ - echo "oracle ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers + chown -R oracle:dba $ORDS_HOME +# echo "oracle ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers + +RUN echo "unset R1" >> /home/oracle/.bashrc && \ + chown root:root /home/oracle/.bashrc && chmod +r /home/oracle/.bashrc # Finalize setup # ------------------- USER oracle WORKDIR /home/oracle + VOLUME ["$ORDS_HOME/config/ords"] EXPOSE 8888 diff --git a/ords/ords_init.sh b/ords/ords_init.sh new file mode 100644 index 00000000..0994dceb --- /dev/null +++ b/ords/ords_init.sh @@ -0,0 +1,484 @@ +#!/bin/bash +## Copyright (c) 2006, 2024, Oracle and/or its affiliates. +## +## The Universal Permissive License (UPL), Version 1.0 +## +## Subject to the condition set forth below, permission is hereby granted to any +## person obtaining a copy of this software, associated documentation and/or data +## (collectively the "Software"), free of charge and under any and all copyright +## rights in the Software, and any and all patent rights owned or freely +## licensable by each licensor hereunder covering either (i) the unmodified +## Software as contributed to or provided by such licensor, or (ii) the Larger +## Works (as defined below), to deal in both +## +## (a) the Software, and +## (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +## one is included with the Software (each a "Larger Work" to which the Software +## is contributed by such licensors), +## +## without restriction, including without limitation the rights to copy, create +## derivative works of, display, perform, and distribute the Software and make, +## use, sell, offer for sale, import, export, have made, and have sold the +## Software and the Larger Work(s), and to sublicense the foregoing rights on +## either these or other terms. +## +## This license is subject to the following condition: +## The above copyright notice and either this complete permission notice or at +## a minimum a reference to the UPL must be included in all copies or +## substantial portions of the Software. +## +## THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +## IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +## FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +## AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +## LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +## OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +## SOFTWARE. + +dump_stack(){ +_log_date=`date "+%y:%m:%d %H:%M:%S"` + local frame=0 + local line_no + local function_name + local file_name + echo -e "BACKTRACE [${_log_date}]\n" + echo -e "filename:line\tfunction " + echo -e "------------- --------" + while caller $frame ;do ((frame++)) ;done | \ + while read line_no function_name file_name;\ + do echo -e "$file_name:$line_no\t$function_name" ;done >&2 +} + + + +get_conn_string() { + local -n _conn_string="${1}" + + local -r _admin_user=$($ords_cfg_cmd get --secret db.adminUser | tail -1) + local _conn_type=$($ords_cfg_cmd get db.connectionType |tail -1) + if [[ $_conn_type == "customurl" ]]; then + local -r _conn=$($ords_cfg_cmd get db.customURL | tail -1) + elif [[ $_conn_type == "tns" ]]; then + local -r _tns_service=$($ords_cfg_cmd get db.tnsAliasName | tail -1) + local -r _conn=${_tns_service} + elif [[ $_conn_type == "basic" ]]; then + local -r _host=$($ords_cfg_cmd get db.hostname | tail -1) + local -r _port=$($ords_cfg_cmd get db.port | tail -1) + local -r _service=$($ords_cfg_cmd get db.servicename | tail -1) + local -r _sid=$($ords_cfg_cmd get db.sid | tail -1) + + if [[ -n ${_host} ]] && [[ -n ${_port} ]]; then + if [[ -n ${_service} ]] || [[ -n ${_sid} ]]; then + local -r _conn=${_host}:${_port}/${_service:-$_sid} + fi + fi + else + # wallet + _conn_type="wallet" + local -r _wallet_service=$($ords_cfg_cmd get db.wallet.zip.service | tail -1) + local -r _conn=${_wallet_service} + fi + + if [[ -n ${_conn} ]]; then + echo "Connection String (${_conn_type}): ${_conn}" + _conn_string="${_admin_user%%/ *}/${config["dbadminusersecret"]}@${_conn}" + if [[ ${_admin_user%%/ *} == "SYS" ]]; then + _conn_string="${_conn_string=} AS SYSDBA" + fi + fi +} + +#------------------------------------------------------------------------------ +function run_sql { + local -r _conn_string="${1}" + local -r _sql="${2}" + local -n _output="${3}" + local -i _rc=0 + + if [[ -z ${_sql} ]]; then + dump_stack + echo "FATAL: Dear Developer.. you've got a bug calling run_sql" && exit 1 + fi + ## Get TNS_ADMIN location + local -r _tns_admin=$($ords_cfg_cmd get db.tnsDirectory | tail -1) + if [[ ! $_tns_admin =~ "Cannot get setting" ]]; then + echo "Setting: TNS_ADMIN=${_tns_admin}" + export TNS_ADMIN=${_tns_admin} + fi + + ## Get ADB Wallet + local -r _wallet_zip_path=$($ords_cfg_cmd get db.wallet.zip.path | tail -1) + if [[ ! $_wallet_zip_path =~ "Cannot get setting" ]]; then + echo "Using: set cloudconfig ${_wallet_zip_path}" + local -r _cloudconfig="set cloudconfig ${_wallet_zip_path}" + fi + + # NOTE to maintainer; the heredoc must be TAB indented + echo "Running SQL..." + #_output=$(cd ${APEX_HOME}/${APEX_VER} && sql -S /nolog <<-EOSQL + _output=$(cd ${APEX_HOME}/${APEX_VER} && sql -S -nohistory -noupdates /nolog <<-EOSQL + WHENEVER SQLERROR EXIT 1 + WHENEVER OSERROR EXIT 1 + ${_cloudconfig} + connect $_conn_string + set serveroutput on echo off pause off feedback off + set heading off wrap off linesize 1000 pagesize 0 + SET TERMOUT OFF VERIFY OFF + ${_sql} + exit; + EOSQL + ) + _rc=$? + + if (( ${_rc} > 0 )); then + dump_stack + echo "SQLERROR: ${_output}" + fi + + return $_rc +} + +#------------------------------------------------------------------------------ +function check_adb() { + local -r _conn_string=$1 + local -n _is_adb=$2 + + local -r _adb_chk_sql=" + DECLARE + invalid_column exception; + pragma exception_init (invalid_column,-00904); + adb_check integer; + BEGIN + EXECUTE IMMEDIATE q'[SELECT COUNT(*) FROM ( + SELECT JSON_VALUE(cloud_identity, '\$.DATABASE_OCID') AS database_ocid + FROM v\$pdbs) t + WHERE t.database_ocid like '%AUTONOMOUS%']' INTO adb_check; + DBMS_OUTPUT.PUT_LINE(adb_check); + EXCEPTION WHEN invalid_column THEN + DBMS_OUTPUT.PUT_LINE('0'); + END; + /" + echo "Checking if Database is an ADB" + run_sql "${_conn_string}" "${_adb_chk_sql}" "_adb_check" + _rc=$? + + if (( ${_rc} == 0 )); then + _adb_check=${_adb_check//[[:space:]]/} + echo "ADB Check: ${_adb_check}" + if (( ${_adb_check} == 1 )); then + _is_adb=${_adb_check//[[:space:]]/} + fi + fi + + return ${_rc} +} + +function create_adb_user() { + local -r _conn_string="${1}" + local -r _pool_name="${2}" + + local _config_user=$($ords_cfg_cmd get db.username | tail -1) + + if [[ -z ${_config_user} ]] || [[ ${_config_user} == "ORDS_PUBLIC_USER" ]]; then + echo "FATAL: You must specify a db.username <> ORDS_PUBLIC_USER in pool ${_pool_name}" + dump_stack + return 1 + fi + + local -r _adb_user_sql=" + DECLARE + l_user VARCHAR2(255); + l_cdn VARCHAR2(255); + BEGIN + BEGIN + SELECT USERNAME INTO l_user FROM DBA_USERS WHERE USERNAME='${_config_user}'; + EXECUTE IMMEDIATE 'ALTER USER \"${_config_user}\" PROFILE ORA_APP_PROFILE'; + EXECUTE IMMEDIATE 'ALTER USER \"${_config_user}\" IDENTIFIED BY \"${config["dbsecret"]}\"'; + DBMS_OUTPUT.PUT_LINE('${_config_user} Exists - Password reset'); + EXCEPTION + WHEN NO_DATA_FOUND THEN + EXECUTE IMMEDIATE 'CREATE USER \"${_config_user}\" IDENTIFIED BY \"${config["dbsecret"]}\" PROFILE ORA_APP_PROFILE'; + DBMS_OUTPUT.PUT_LINE('${_config_user} Created'); + END; + EXECUTE IMMEDIATE 'GRANT CONNECT TO \"${_config_user}\"'; + BEGIN + SELECT USERNAME INTO l_user FROM DBA_USERS WHERE USERNAME='ORDS_PLSQL_GATEWAY_OPER'; + EXECUTE IMMEDIATE 'ALTER USER \"ORDS_PLSQL_GATEWAY_OPER\" PROFILE DEFAULT'; + EXECUTE IMMEDIATE 'ALTER USER \"ORDS_PLSQL_GATEWAY_OPER\" NO AUTHENTICATION'; + DBMS_OUTPUT.PUT_LINE('ORDS_PLSQL_GATEWAY_OPER Exists'); + EXCEPTION + WHEN NO_DATA_FOUND THEN + EXECUTE IMMEDIATE 'CREATE USER \"ORDS_PLSQL_GATEWAY_OPER\" NO AUTHENTICATION PROFILE DEFAULT'; + DBMS_OUTPUT.PUT_LINE('ORDS_PLSQL_GATEWAY_OPER Created'); + END; + EXECUTE IMMEDIATE 'GRANT CONNECT TO \"ORDS_PLSQL_GATEWAY_OPER\"'; + EXECUTE IMMEDIATE 'ALTER USER \"ORDS_PLSQL_GATEWAY_OPER\" GRANT CONNECT THROUGH \"${_config_user}\"'; + ORDS_ADMIN.PROVISION_RUNTIME_ROLE ( + p_user => '${_config_user}' + ,p_proxy_enabled_schemas => TRUE + ); + ORDS_ADMIN.CONFIG_PLSQL_GATEWAY ( + p_runtime_user => '${_config_user}' + ,p_plsql_gateway_user => 'ORDS_PLSQL_GATEWAY_OPER' + ); + -- TODO: Only do this if ADB APEX Version <> this ORDS Version + BEGIN + SELECT images_version INTO L_CDN + FROM APEX_PATCHES + where is_bundle_patch = 'Yes' + order by patch_version desc + fetch first 1 rows only; + EXCEPTION WHEN NO_DATA_FOUND THEN + select version_no INTO L_CDN + from APEX_RELEASE; + END; + apex_instance_admin.set_parameter( + p_parameter => 'IMAGE_PREFIX', + p_value => 'https://static.oracle.com/cdn/apex/'||L_CDN||'/' + ); + END; + /" + + run_sql "${_conn_string}" "${_adb_user_sql}" "_adb_user_sql_output" + _rc=$? + + echo "Installation Output: ${_adb_user_sql_output}" + return ${_rc} +} + +#------------------------------------------------------------------------------ +function compare_versions() { + local _db_ver=$1 + local _im_ver=$2 + + IFS='.' read -r -a _db_ver_array <<< "$_db_ver" + IFS='.' read -r -a _im_ver_array <<< "$_im_ver" + + # Compare each component + local i + for i in "${!_db_ver_array[@]}"; do + if [[ "${_db_ver_array[$i]}" -lt "${_im_ver_array[$i]}" ]]; then + # _db_ver < _im_ver (upgrade) + return 0 + elif [[ "${_db_ver_array[$i]}" -gt "${_im_ver_array[$i]}" ]]; then + # _db_ver < _im_ver (do nothing) + return 1 + fi + done + # _db_ver == __im_ver (do nothing) + return 1 +} + +#------------------------------------------------------------------------------ +set_secret() { + local -r _pool_name="${1}" + local -r _config_key="${2}" + local -r _config_val="${3}" + local -i _rc=0 + + if [[ -n "${_config_val}" ]]; then + ords --config "$ORDS_CONFIG" config --db-pool "${_pool_name}" secret --password-stdin "${_config_key}" <<< "${_config_val}" + _rc=$? + echo "${_config_key} in pool ${_pool_name} set" + else + echo "${_config_key} in pool ${_pool_name}, not defined" + _rc=0 + fi + + return ${_rc} +} + +#------------------------------------------------------------------------------ +ords_upgrade() { + local -r _pool_name="${1}" + local -r _upgrade_key="${2}" + local -i _rc=0 + + if [[ -n "${config["dbadminusersecret"]}" ]]; then + # Get usernames + local -r ords_user=$($ords_cfg_cmd get db.username | tail -1) + local -r ords_admin=$($ords_cfg_cmd get db.adminUser | tail -1) + + echo "Performing ORDS install/upgrade as $ords_admin into $ords_user on pool ${_pool_name}" + if [[ ${_pool_name} == "default" ]]; then + ords --config "$ORDS_CONFIG" install --db-only \ + --admin-user "$ords_admin" --password-stdin <<< "${config["dbadminusersecret"]}" + _rc=$? + else + ords --config "$ORDS_CONFIG" install --db-pool "${_pool_name}" --db-only \ + --admin-user "$ords_admin" --password-stdin <<< "${config["dbadminusersecret"]}" + _rc=$? + fi + + # Dar be bugs below deck with --db-user so using the above + # ords --config "$ORDS_CONFIG" install --db-pool "$1" --db-only \ + # --admin-user "$ords_admin" --db-user "$ords_user" --password-stdin <<< "${!2}" + fi + + return $_rc +} + +#------------------------------------------------------------------------------ +function get_apex_version() { + local -r _conn_string="${1}" + local -n _action="${2}" + local -i _rc=0 + + local -r _ver_sql="SELECT VERSION FROM DBA_REGISTRY WHERE COMP_ID='APEX';" + run_sql "${_conn_string}" "${_ver_sql}" "_db_apex_version" + _rc=$? + + if (( $_rc > 0 )); then + echo "FATAL: Unable to connect to ${_conn_string} to get APEX version" + dump_stack + return $_rc + fi + + local -r _db_apex_version=${_db_apex_version//[^0-9.]/} + echo "Database APEX Version: ${_db_apex_version:-Not Installed}" + + _action="none" + if [[ -z "${_db_apex_version}" ]]; then + echo "Installing APEX ${APEX_VER}" + _action="install" + elif compare_versions ${_db_apex_version} ${APEX_VER}; then + echo "Upgrading from ${_db_apex_version} to ${APEX_VER}" + _action="upgrade" + else + echo "No Installation/Upgrade Required" + fi + + return $_rc +} + +apex_upgrade() { + local -r _conn_string="${1}" + local -r _upgrade_key="${2}" + local -i _rc=0 + + if [[ -f ${APEX_HOME}/${APEX_VER}/apexins.sql ]] && [[ "${!_upgrade_key}" = "true" ]]; then + echo "Starting Installation of APEX ${APEX_VER}" + local -r _install_sql="@apxsilentins.sql SYSAUX SYSAUX TEMP /i/ ${config["dbsecret"]} ${config["dbsecret"]} ${config["dbsecret"]} ${config["dbsecret"]}" + run_sql "${_conn_string}" "${_install_sql}" "_install_output" + _rc=$? + echo "Installation Output: ${_install_output}" + fi + + return $_rc +} + +#------------------------------------------------------------------------------ +# INIT +#------------------------------------------------------------------------------ +declare -A pool_exit +for pool in "$ORDS_CONFIG"/databases/*; do + rc=0 + pool_name=$(basename "$pool") + pool_exit[${pool_name}]=0 + ords_cfg_cmd="ords --config $ORDS_CONFIG config --db-pool ${pool_name}" + echo "Found Pool: $pool_name..." + + declare -A config + for key in dbsecret dbadminusersecret dbcdbadminusersecret; do + var_key="${pool_name//-/_}_${key}" + echo "Obtaining value from initContainer variable: ${var_key}" + var_val="${!var_key}" + config[${key}]="${var_val}" + done + + # Set Secrets + set_secret "${pool_name}" "db.password" "${config["dbsecret"]}" + rc=$((rc + $?)) + set_secret "${pool_name}" "db.adminUser.password" "${config["dbadminusersecret"]}" + rc=$((rc + $?)) + set_secret "${pool_name}" "db.cdb.adminUser.password" "${config["dbcdbadminusersecret"]}" + rc=$((rc + $?)) + + if (( ${rc} > 0 )); then + echo "FATAL: Unable to set configuration for pool ${pool_name}" + dump_stack + pool_exit[${pool_name}]=1 + continue + elif [[ -z ${config["dbsecret"]} ]]; then + echo "FATAL: db.password must be specified for ${pool_name}" + dump_stack + pool_exit[${pool_name}]=1 + continue + elif [[ -z ${config["dbadminusersecret"]} ]]; then + echo "INFO: No additional configuration for ${pool_name}" + continue + fi + + get_conn_string "conn_string" + if [[ -z ${conn_string} ]]; then + echo "FATAL: Unable to get ${pool_name} database connect string" + dump_stack + pool_exit[${pool_name}]=1 + continue + fi + + check_adb "${conn_string}" "is_adb" + rc=$? + if (( ${rc} > 0 )); then + pool_exit[${pool_name}]=1 + continue + fi + + if (( is_adb )); then + # Create ORDS User + echo "Processing ADB in Pool: ${pool_name}" + create_adb_user "${conn_string}" "${pool_name}" + else + # APEX Upgrade + echo "---------------------------------------------------" + apex_upgrade_var=${pool_name}_autoupgrade_apex + if [[ ${!apex_upgrade_var} != "true" ]]; then + echo "APEX Install/Upgrade not requested for ${pool_name}" + continue + fi + + get_apex_version "${conn_string}" "action" + if [[ -z ${action} ]]; then + echo "FATAL: Unable to get ${pool_name} APEX Version" + dump_stack + pool_exit[${pool_name}]=1 + continue + fi + + if [[ ${action} != "none" ]]; then + apex_upgrade "${conn_string}" "${pool_name}_autoupgrade_apex" + if (( $? > 0 )); then + echo "FATAL: Unable to ${action} APEX for ${pool_name}" + dump_stack + pool_exit[${pool_name}]=1 + continue + fi + fi + + # ORDS Upgrade + ords_upgrade_var=${pool_name}_autoupgrade_ords + if [[ ${!ords_upgrade_var} != "true" ]]; then + echo "ORDS Install/Upgrade not requested for ${pool_name}" + continue + fi + + ords_upgrade "${pool_name}" "${pool_name}_autoupgrade_ords" + rc=$? + if (( $rc > 0 )); then + echo "FATAL: Unable to preform requested ORDS install/upgrade on ${pool_name}" + pool_exit[${pool_name}]=1 + dump_stack + continue + fi + fi +done + +for key in "${!pool_exit[@]}"; do + echo "Pool: $key, Exit Code: ${pool_exit[$key]}" + if (( ${pool_exit[$key]} > 0 )); then + rc=1 + fi +done + +exit $rc +#exit 0 diff --git a/ords/runOrdsSSL.sh b/ords/runOrdsSSL.sh index 35f1b77b..07e2b931 100644 --- a/ords/runOrdsSSL.sh +++ b/ords/runOrdsSSL.sh @@ -106,36 +106,42 @@ function setupOrds() { echo "====================================================" echo CONFIG=$CONFIG +echo $R1|sed 's/-----BEGIN PRIVATE KEY-----/-----BEGIN PRIVATE KEY-----\n/g'|\ + sed 's/-----END PRIVATE KEY-----/\n-----END PRIVATE KEY-----/' > $ORDS_HOME/k.txt + + export ORDS_LOGS=/tmp [ -f $ORDS_HOME/secrets/$WEBSERVER_USER_KEY ] && { - WEBSERVER_USER=`cat $ORDS_HOME/secrets/$WEBSERVER_USER_KEY` + WEBSERVER_USER=$(cat /opt/oracle/ords/secrets/${WEBSERVER_USER_KEY}|base64 --decode |openssl rsautl -decrypt -out swap -inkey $ORDS_HOME/k.txt -in - ; cat swap ;rm swap) } [ -f $ORDS_HOME/secrets/$WEBSERVER_PASSWORD_KEY ] && { - WEBSERVER_PASSWORD=`cat $ORDS_HOME/secrets/$WEBSERVER_PASSWORD_KEY` + WEBSERVER_PASSWORD=$(cat /opt/oracle/ords/secrets/${WEBSERVER_PASSWORD_KEY}|base64 --decode |openssl rsautl -decrypt -out swap -inkey $ORDS_HOME/k.txt -in - ; cat swap ;rm swap) } [ -f $ORDS_HOME/secrets/$CDBADMIN_USER_KEY ] && { - CDBADMIN_USER=`cat $ORDS_HOME/secrets/$CDBADMIN_USER_KEY` + CDBADMIN_USER=$(cat /opt/oracle/ords/secrets/${CDBADMIN_USER_KEY} | base64 --decode |openssl rsautl -decrypt -out swap -inkey $ORDS_HOME/k.txt -in - ; cat swap ;rm swap) } [ -f $ORDS_HOME/secrets/$CDBADMIN_PWD_KEY ] && { - CDBADMIN_PWD=`cat $ORDS_HOME/secrets/$CDBADMIN_PWD_KEY` + CDBADMIN_PWD=$(cat /opt/oracle/ords/secrets/${CDBADMIN_PWD_KEY} | base64 --decode |openssl rsautl -decrypt -out swap -inkey $ORDS_HOME/k.txt -in - ; cat swap ;rm swap) } [ -f $ORDS_HOME/secrets/$ORACLE_PWD_KEY ] && { - SYSDBA_PASSWORD=`cat $ORDS_HOME/secrets/$ORACLE_PWD_KEY` + #SYSDBA_PASSWORD=`cat $ORDS_HOME/secrets/$ORACLE_PWD_KEY` + SYSDBA_PASSWORD=$(cat $ORDS_HOME/secrets/${ORACLE_PWD_KEY} | base64 --decode |openssl rsautl -decrypt -out swap -inkey $ORDS_HOME/k.txt -in - ; cat swap ;rm swap) } [ -f $ORDS_HOME/secrets/$ORACLE_PWD_KEY ] && { - ORDS_PASSWORD=`cat $ORDS_HOME/secrets/$ORDS_PWD_KEY` + #ORDS_PASSWORD=`cat $ORDS_HOME/secrets/$ORDS_PWD_KEY` + ORDS_PASSWORD=$(cat $ORDS_HOME/secrets/${ORDS_PWD_KEY} | base64 --decode |openssl rsautl -decrypt -out swap -inkey $ORDS_HOME/k.txt -in - ; cat swap ;rm swap) } @@ -151,6 +157,7 @@ ${SYSDBA_PASSWORD:-PROVIDE_A_PASSWORD} ${ORDS_PASSWORD:-PROVIDE_A_PASSWORD} EOF +rm $ORDS_HOME/k.txt if [ $? -ne 0 ] diff --git a/test/e2e/autonomouscontainerdatabase_test.go b/test/e2e/autonomouscontainerdatabase_test.go index f27d8a6d..a76fc33f 100644 --- a/test/e2e/autonomouscontainerdatabase_test.go +++ b/test/e2e/autonomouscontainerdatabase_test.go @@ -49,8 +49,8 @@ import ( "k8s.io/apimachinery/pkg/types" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" - "github.com/oracle/oracle-database-operator/test/e2e/behavior" - "github.com/oracle/oracle-database-operator/test/e2e/util" + e2ebehavior "github.com/oracle/oracle-database-operator/test/e2e/behavior" + e2eutil "github.com/oracle/oracle-database-operator/test/e2e/util" // +kubebuilder:scaffold:imports ) @@ -81,7 +81,7 @@ var _ = Describe("test ACD binding", func() { CompartmentOCID: common.String(SharedCompartmentOCID), AutonomousExadataVMClusterOCID: common.String(SharedExadataVMClusterOCID), PatchModel: database.AutonomousContainerDatabasePatchModelUpdates, - OCIConfig: dbv1alpha1.OCIConfigSpec{ + OCIConfig: dbv1alpha1.OciConfigSpec{ ConfigMapName: common.String(SharedOCIConfigMapName), SecretName: common.String(SharedOCISecretName), }, @@ -119,7 +119,7 @@ var _ = Describe("test ACD binding", func() { }, Spec: dbv1alpha1.AutonomousContainerDatabaseSpec{ AutonomousContainerDatabaseOCID: common.String(acdID), - OCIConfig: dbv1alpha1.OCIConfigSpec{ + OCIConfig: dbv1alpha1.OciConfigSpec{ ConfigMapName: common.String(SharedOCIConfigMapName), SecretName: common.String(SharedOCISecretName), }, diff --git a/test/e2e/autonomousdatabase_controller_bind_test.go b/test/e2e/autonomousdatabase_controller_bind_test.go index 58de3356..48e60f0d 100644 --- a/test/e2e/autonomousdatabase_controller_bind_test.go +++ b/test/e2e/autonomousdatabase_controller_bind_test.go @@ -50,8 +50,8 @@ import ( "k8s.io/apimachinery/pkg/types" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" - "github.com/oracle/oracle-database-operator/test/e2e/behavior" - "github.com/oracle/oracle-database-operator/test/e2e/util" + e2ebehavior "github.com/oracle/oracle-database-operator/test/e2e/behavior" + e2eutil "github.com/oracle/oracle-database-operator/test/e2e/util" // +kubebuilder:scaffold:imports ) @@ -103,18 +103,18 @@ var _ = Describe("test ADB binding with hardLink=true", func() { }, Spec: dbv1alpha1.AutonomousDatabaseSpec{ Details: dbv1alpha1.AutonomousDatabaseDetails{ - AutonomousDatabaseOCID: adbID, - Wallet: dbv1alpha1.WalletSpec{ - Name: common.String(downloadedWallet), - Password: dbv1alpha1.PasswordSpec{ - K8sSecret: dbv1alpha1.K8sSecretSpec{ - Name: common.String(SharedWalletPassSecretName), - }, + Id: adbID, + }, + Wallet: dbv1alpha1.WalletSpec{ + Name: common.String(downloadedWallet), + Password: dbv1alpha1.PasswordSpec{ + K8sSecret: dbv1alpha1.K8sSecretSpec{ + Name: common.String(SharedWalletPassSecretName), }, }, }, HardLink: common.Bool(false), - OCIConfig: dbv1alpha1.OCIConfigSpec{ + OciConfig: dbv1alpha1.OciConfigSpec{ ConfigMapName: common.String(SharedOCIConfigMapName), SecretName: common.String(SharedOCISecretName), }, @@ -136,9 +136,9 @@ var _ = Describe("test ADB binding with hardLink=true", func() { It("Should restart ADB", e2ebehavior.UpdateAndAssertADBState(&k8sClient, &dbClient, &adbLookupKey, database.AutonomousDatabaseLifecycleStateAvailable)) - It("Should change to RESTRICTED network access", e2ebehavior.TestNetworkAccessRestricted(&k8sClient, &dbClient, &adbLookupKey, false)) + It("Should change to RESTRICTED network access", e2ebehavior.TestNetworkAccessRestricted(&k8sClient, &dbClient, &adbLookupKey, []string{"192.168.0.1"}, false)) - It("Should change isMTLSConnectionRequired to false", e2ebehavior.TestNetworkAccessRestricted(&k8sClient, &dbClient, &adbLookupKey, false)) + It("Should change isMTLSConnectionRequired to false", e2ebehavior.TestNetworkAccessRestricted(&k8sClient, &dbClient, &adbLookupKey, []string{"192.168.0.1"}, false)) It("Should should change to PRIVATE network access", e2ebehavior.TestNetworkAccessPrivate(&k8sClient, &dbClient, &adbLookupKey, false, &SharedSubnetOCID, &SharedNsgOCID)) @@ -162,18 +162,18 @@ var _ = Describe("test ADB binding with hardLink=true", func() { }, Spec: dbv1alpha1.AutonomousDatabaseSpec{ Details: dbv1alpha1.AutonomousDatabaseDetails{ - AutonomousDatabaseOCID: adbID, - Wallet: dbv1alpha1.WalletSpec{ - Name: common.String(downloadedWallet), - Password: dbv1alpha1.PasswordSpec{ - OCISecret: dbv1alpha1.OCISecretSpec{ - OCID: common.String(SharedInstanceWalletPasswordOCID), - }, + Id: adbID, + }, + Wallet: dbv1alpha1.WalletSpec{ + Name: common.String(downloadedWallet), + Password: dbv1alpha1.PasswordSpec{ + OciSecret: dbv1alpha1.OciSecretSpec{ + Id: common.String(SharedInstanceWalletPasswordOCID), }, }, }, HardLink: common.Bool(true), - OCIConfig: dbv1alpha1.OCIConfigSpec{ + OciConfig: dbv1alpha1.OciConfigSpec{ ConfigMapName: common.String(SharedOCIConfigMapName), SecretName: common.String(SharedOCISecretName), }, @@ -210,10 +210,10 @@ var _ = Describe("test ADB binding with hardLink=true", func() { }, Spec: dbv1alpha1.AutonomousDatabaseSpec{ Details: dbv1alpha1.AutonomousDatabaseDetails{ - AutonomousDatabaseOCID: &terminatedAdbID, + Id: &terminatedAdbID, }, HardLink: common.Bool(true), - OCIConfig: dbv1alpha1.OCIConfigSpec{ + OciConfig: dbv1alpha1.OciConfigSpec{ ConfigMapName: common.String(SharedOCIConfigMapName), SecretName: common.String(SharedOCISecretName), }, diff --git a/test/e2e/autonomousdatabase_controller_create_test.go b/test/e2e/autonomousdatabase_controller_create_test.go index 9cf0e7e7..cc7fd288 100644 --- a/test/e2e/autonomousdatabase_controller_create_test.go +++ b/test/e2e/autonomousdatabase_controller_create_test.go @@ -48,8 +48,8 @@ import ( "k8s.io/apimachinery/pkg/types" dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" - "github.com/oracle/oracle-database-operator/test/e2e/behavior" - "github.com/oracle/oracle-database-operator/test/e2e/util" + e2ebehavior "github.com/oracle/oracle-database-operator/test/e2e/behavior" + e2eutil "github.com/oracle/oracle-database-operator/test/e2e/util" // +kubebuilder:scaffold:imports ) @@ -89,28 +89,30 @@ var _ = Describe("test ADB provisioning", func() { }, Spec: dbv1alpha1.AutonomousDatabaseSpec{ Details: dbv1alpha1.AutonomousDatabaseDetails{ - CompartmentOCID: common.String(SharedCompartmentOCID), - DbName: common.String(dbName), - DisplayName: common.String(dbName), - CPUCoreCount: common.Int(1), - AdminPassword: dbv1alpha1.PasswordSpec{ - K8sSecret: dbv1alpha1.K8sSecretSpec{ - Name: common.String(SharedAdminPassSecretName), - }, - }, - DataStorageSizeInTBs: common.Int(1), - IsAutoScalingEnabled: common.Bool(true), - Wallet: dbv1alpha1.WalletSpec{ - Name: common.String(downloadedWallet), - Password: dbv1alpha1.PasswordSpec{ + AutonomousDatabaseBase: dbv1alpha1.AutonomousDatabaseBase{ + CompartmentId: common.String(SharedCompartmentOCID), + DbName: common.String(dbName), + DisplayName: common.String(dbName), + CpuCoreCount: common.Int(1), + AdminPassword: dbv1alpha1.PasswordSpec{ K8sSecret: dbv1alpha1.K8sSecretSpec{ - Name: common.String(SharedWalletPassSecretName), + Name: common.String(SharedAdminPassSecretName), }, }, + DataStorageSizeInTBs: common.Int(1), + IsAutoScalingEnabled: common.Bool(true), + }, + }, + Wallet: dbv1alpha1.WalletSpec{ + Name: common.String(downloadedWallet), + Password: dbv1alpha1.PasswordSpec{ + K8sSecret: dbv1alpha1.K8sSecretSpec{ + Name: common.String(SharedWalletPassSecretName), + }, }, }, HardLink: common.Bool(true), - OCIConfig: dbv1alpha1.OCIConfigSpec{ + OciConfig: dbv1alpha1.OciConfigSpec{ ConfigMapName: common.String(SharedOCIConfigMapName), SecretName: common.String(SharedOCISecretName), }, @@ -134,20 +136,22 @@ var _ = Describe("test ADB provisioning", func() { }, Spec: dbv1alpha1.AutonomousDatabaseSpec{ Details: dbv1alpha1.AutonomousDatabaseDetails{ - CompartmentOCID: common.String(SharedCompartmentOCID), - DbName: common.String(dbName), - DisplayName: common.String(dbName), - CPUCoreCount: common.Int(1), - AdminPassword: dbv1alpha1.PasswordSpec{ - K8sSecret: dbv1alpha1.K8sSecretSpec{ - Name: common.String(SharedAdminPassSecretName), + AutonomousDatabaseBase: dbv1alpha1.AutonomousDatabaseBase{ + CompartmentId: common.String(SharedCompartmentOCID), + DbName: common.String(dbName), + DisplayName: common.String(dbName), + CpuCoreCount: common.Int(1), + AdminPassword: dbv1alpha1.PasswordSpec{ + K8sSecret: dbv1alpha1.K8sSecretSpec{ + Name: common.String(SharedAdminPassSecretName), + }, }, + DataStorageSizeInTBs: common.Int(1), + IsAutoScalingEnabled: common.Bool(true), }, - DataStorageSizeInTBs: common.Int(1), - IsAutoScalingEnabled: common.Bool(true), }, HardLink: common.Bool(true), - OCIConfig: dbv1alpha1.OCIConfigSpec{ + OciConfig: dbv1alpha1.OciConfigSpec{ ConfigMapName: common.String(SharedOCIConfigMapName), SecretName: common.String(SharedOCISecretName), }, @@ -179,7 +183,7 @@ var _ = Describe("test ADB provisioning", func() { // Get adb ocid adb := &dbv1alpha1.AutonomousDatabase{} Expect(k8sClient.Get(context.TODO(), adbLookupKey, adb)).To(Succeed()) - databaseOCID := adb.Spec.Details.AutonomousDatabaseOCID + databaseOCID := adb.Spec.Details.Id tnsEntry := dbName + "_high" err := e2ebehavior.ConfigureADBBackup(&dbClient, databaseOCID, &tnsEntry, &SharedPlainTextAdminPassword, &SharedPlainTextWalletPassword, &SharedBucketUrl, &SharedAuthToken, &SharedOciUser) Expect(err).ShouldNot(HaveOccurred()) @@ -195,12 +199,12 @@ var _ = Describe("test ADB provisioning", func() { }, Spec: dbv1alpha1.AutonomousDatabaseBackupSpec{ Target: dbv1alpha1.TargetSpec{ - OCIADB: dbv1alpha1.OCIADBSpec{ - OCID: common.String(*databaseOCID), + OciAdb: dbv1alpha1.OciAdbSpec{ + Ocid: common.String(*databaseOCID), }, }, DisplayName: common.String(backupName), - OCIConfig: dbv1alpha1.OCIConfigSpec{ + OCIConfig: dbv1alpha1.OciConfigSpec{ ConfigMapName: common.String(SharedOCIConfigMapName), SecretName: common.String(SharedOCISecretName), }, @@ -227,16 +231,16 @@ var _ = Describe("test ADB provisioning", func() { }, Spec: dbv1alpha1.AutonomousDatabaseRestoreSpec{ Target: dbv1alpha1.TargetSpec{ - K8sADB: dbv1alpha1.K8sADBSpec{ + K8sAdb: dbv1alpha1.K8sAdbSpec{ Name: common.String(resourceName), }, }, Source: dbv1alpha1.SourceSpec{ - K8sADBBackup: dbv1alpha1.K8sADBBackupSpec{ + K8sAdbBackup: dbv1alpha1.K8sAdbBackupSpec{ Name: common.String(backupName), }, }, - OCIConfig: dbv1alpha1.OCIConfigSpec{ + OCIConfig: dbv1alpha1.OciConfigSpec{ ConfigMapName: common.String(SharedOCIConfigMapName), SecretName: common.String(SharedOCISecretName), }, @@ -273,29 +277,30 @@ var _ = Describe("test ADB provisioning", func() { }, Spec: dbv1alpha1.AutonomousDatabaseSpec{ Details: dbv1alpha1.AutonomousDatabaseDetails{ - CompartmentOCID: common.String(SharedCompartmentOCID), - DbName: common.String(dbName), - DisplayName: common.String(dbName), - CPUCoreCount: common.Int(1), - AdminPassword: dbv1alpha1.PasswordSpec{ - OCISecret: dbv1alpha1.OCISecretSpec{ - OCID: common.String(SharedAdminPasswordOCID), + AutonomousDatabaseBase: dbv1alpha1.AutonomousDatabaseBase{ + CompartmentId: common.String(SharedCompartmentOCID), + DbName: common.String(dbName), + DisplayName: common.String(dbName), + CpuCoreCount: common.Int(1), + AdminPassword: dbv1alpha1.PasswordSpec{ + OciSecret: dbv1alpha1.OciSecretSpec{ + Id: common.String(SharedAdminPasswordOCID), + }, }, + DataStorageSizeInTBs: common.Int(1), + IsAutoScalingEnabled: common.Bool(true), }, - DataStorageSizeInTBs: common.Int(1), - IsAutoScalingEnabled: common.Bool(true), - - Wallet: dbv1alpha1.WalletSpec{ - Name: common.String(downloadedWallet), - Password: dbv1alpha1.PasswordSpec{ - OCISecret: dbv1alpha1.OCISecretSpec{ - OCID: common.String(SharedInstanceWalletPasswordOCID), - }, + }, + Wallet: dbv1alpha1.WalletSpec{ + Name: common.String(downloadedWallet), + Password: dbv1alpha1.PasswordSpec{ + OciSecret: dbv1alpha1.OciSecretSpec{ + Id: common.String(SharedInstanceWalletPasswordOCID), }, }, }, HardLink: common.Bool(true), - OCIConfig: dbv1alpha1.OCIConfigSpec{ + OciConfig: dbv1alpha1.OciConfigSpec{ ConfigMapName: common.String(SharedOCIConfigMapName), SecretName: common.String(SharedOCISecretName), }, diff --git a/test/e2e/behavior/shared_behaviors.go b/test/e2e/behavior/shared_behaviors.go index 00fc02c9..3d87ce94 100644 --- a/test/e2e/behavior/shared_behaviors.go +++ b/test/e2e/behavior/shared_behaviors.go @@ -55,11 +55,12 @@ import ( "k8s.io/apimachinery/pkg/types" "sigs.k8s.io/controller-runtime/pkg/client" - dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" - "github.com/oracle/oracle-database-operator/test/e2e/util" "os" "os/exec" "strings" + + dbv1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" + e2eutil "github.com/oracle/oracle-database-operator/test/e2e/util" ) /************************************************************** @@ -110,11 +111,11 @@ func AssertProvision(k8sClient *client.Client, adbLookupKey *types.NamespacedNam return nil, err } - return createdADB.Spec.Details.AutonomousDatabaseOCID, nil + return createdADB.Spec.Details.Id, nil }, provisionTimeout, intervalTime).ShouldNot(BeNil()) fmt.Fprintf(GinkgoWriter, "AutonomousDatabase DbName = %s, and AutonomousDatabaseOCID = %s\n", - *createdADB.Spec.Details.DbName, *createdADB.Spec.Details.AutonomousDatabaseOCID) + *createdADB.Spec.Details.DbName, *createdADB.Spec.Details.Id) } } @@ -138,13 +139,13 @@ func AssertBind(k8sClient *client.Client, adbLookupKey *types.NamespacedName) fu if err != nil { return false } - return (boundADB.Spec.Details.CompartmentOCID != nil && + return (boundADB.Spec.Details.CompartmentId != nil && boundADB.Spec.Details.DbWorkload != "" && boundADB.Spec.Details.DbName != nil) }, bindTimeout).Should(Equal(true), "Attributes in the resource should not be empty") fmt.Fprintf(GinkgoWriter, "AutonomousDatabase DbName = %s, and AutonomousDatabaseOCID = %s\n", - *boundADB.Spec.Details.DbName, *boundADB.Spec.Details.AutonomousDatabaseOCID) + *boundADB.Spec.Details.DbName, *boundADB.Spec.Details.Id) } } @@ -163,10 +164,10 @@ func AssertWallet(k8sClient *client.Client, adbLookupKey *types.NamespacedName) Expect(derefK8sClient.Get(context.TODO(), *adbLookupKey, adb)).To(Succeed()) // The default name is xxx-instance-wallet - if adb.Spec.Details.Wallet.Name == nil { + if adb.Spec.Wallet.Name == nil { walletName = adb.Name + "-instance-wallet" } else { - walletName = *adb.Spec.Details.Wallet.Name + walletName = *adb.Spec.Wallet.Name } By("Checking the wallet secret " + walletName + " is created and is not empty") @@ -249,7 +250,7 @@ func UpdateDetails(k8sClient *client.Client, dbClient *database.DatabaseClient, // , the List request returns PROVISIONING state. In this case the update request will fail with // conflict state error. Eventually(func() (database.AutonomousDatabaseLifecycleStateEnum, error) { - listResp, err := e2eutil.ListAutonomousDatabases(derefDBClient, expectedADB.Spec.Details.CompartmentOCID, expectedADB.Spec.Details.DisplayName) + listResp, err := e2eutil.ListAutonomousDatabases(derefDBClient, expectedADB.Spec.Details.CompartmentId, expectedADB.Spec.Details.DisplayName) if err != nil { return "", err } @@ -265,7 +266,7 @@ func UpdateDetails(k8sClient *client.Client, dbClient *database.DatabaseClient, var newDisplayName = *expectedADB.Spec.Details.DisplayName + "_new" var newCPUCoreCount int - if *expectedADB.Spec.Details.CPUCoreCount == 1 { + if *expectedADB.Spec.Details.CpuCoreCount == 1 { newCPUCoreCount = 2 } else { newCPUCoreCount = 1 @@ -278,7 +279,7 @@ func UpdateDetails(k8sClient *client.Client, dbClient *database.DatabaseClient, newDisplayName, newCPUCoreCount, newKey, newVal)) expectedADB.Spec.Details.DisplayName = common.String(newDisplayName) - expectedADB.Spec.Details.CPUCoreCount = common.Int(newCPUCoreCount) + expectedADB.Spec.Details.CpuCoreCount = common.Int(newCPUCoreCount) expectedADB.Spec.Details.FreeformTags = map[string]string{newKey: newVal} expectedADB.Spec.Details.AdminPassword.K8sSecret.Name = common.String(newSecretName) @@ -307,18 +308,18 @@ func AssertADBDetails(k8sClient *client.Client, Eventually(func() (bool, error) { // Fetch the ADB from OCI when it's in AVAILABLE state, and retry if its attributes doesn't match the new ADB's attributes retryPolicy := e2eutil.NewLifecycleStateRetryPolicyADB(database.AutonomousDatabaseLifecycleStateAvailable) - resp, err := e2eutil.GetAutonomousDatabase(derefDBClient, expectedADB.Spec.Details.AutonomousDatabaseOCID, &retryPolicy) + resp, err := e2eutil.GetAutonomousDatabase(derefDBClient, expectedADB.Spec.Details.Id, &retryPolicy) if err != nil { return false, err } debug := false if debug { - if !compareString(expectedADBDetails.AutonomousDatabaseOCID, resp.AutonomousDatabase.Id) { - fmt.Fprintf(GinkgoWriter, "Expected OCID: %v\nGot: %v\n", expectedADBDetails.AutonomousDatabaseOCID, resp.AutonomousDatabase.Id) + if !compareString(expectedADBDetails.Id, resp.AutonomousDatabase.Id) { + fmt.Fprintf(GinkgoWriter, "Expected OCID: %v\nGot: %v\n", expectedADBDetails.Id, resp.AutonomousDatabase.Id) } - if !compareString(expectedADBDetails.CompartmentOCID, resp.AutonomousDatabase.CompartmentId) { - fmt.Fprintf(GinkgoWriter, "Expected CompartmentOCID: %v\nGot: %v\n", expectedADBDetails.CompartmentOCID, resp.CompartmentId) + if !compareString(expectedADBDetails.CompartmentId, resp.AutonomousDatabase.CompartmentId) { + fmt.Fprintf(GinkgoWriter, "Expected CompartmentOCID: %v\nGot: %v\n", expectedADBDetails.CompartmentId, resp.CompartmentId) } if !compareString(expectedADBDetails.DisplayName, resp.AutonomousDatabase.DisplayName) { fmt.Fprintf(GinkgoWriter, "Expected DisplayName: %v\nGot: %v\n", expectedADBDetails.DisplayName, resp.AutonomousDatabase.DisplayName) @@ -338,8 +339,8 @@ func AssertADBDetails(k8sClient *client.Client, if !compareInt(expectedADBDetails.DataStorageSizeInTBs, resp.AutonomousDatabase.DataStorageSizeInTBs) { fmt.Fprintf(GinkgoWriter, "Expected DataStorageSize: %v\nGot: %v\n", expectedADBDetails.DataStorageSizeInTBs, resp.AutonomousDatabase.DataStorageSizeInTBs) } - if !compareInt(expectedADBDetails.CPUCoreCount, resp.AutonomousDatabase.CpuCoreCount) { - fmt.Fprintf(GinkgoWriter, "Expected CPUCoreCount: %v\nGot: %v\n", expectedADBDetails.CPUCoreCount, resp.AutonomousDatabase.CpuCoreCount) + if !compareInt(expectedADBDetails.CpuCoreCount, resp.AutonomousDatabase.CpuCoreCount) { + fmt.Fprintf(GinkgoWriter, "Expected CPUCoreCount: %v\nGot: %v\n", expectedADBDetails.CpuCoreCount, resp.AutonomousDatabase.CpuCoreCount) } if !compareBool(expectedADBDetails.IsAutoScalingEnabled, resp.AutonomousDatabase.IsAutoScalingEnabled) { fmt.Fprintf(GinkgoWriter, "Expected IsAutoScalingEnabled: %v\nGot: %v\n", expectedADBDetails.IsAutoScalingEnabled, resp.AutonomousDatabase.IsAutoScalingEnabled) @@ -347,23 +348,23 @@ func AssertADBDetails(k8sClient *client.Client, if !compareStringMap(expectedADBDetails.FreeformTags, resp.AutonomousDatabase.FreeformTags) { fmt.Fprintf(GinkgoWriter, "Expected FreeformTags: %v\nGot: %v\n", expectedADBDetails.FreeformTags, resp.AutonomousDatabase.FreeformTags) } - if !compareBool(expectedADBDetails.NetworkAccess.IsAccessControlEnabled, resp.AutonomousDatabase.IsAccessControlEnabled) { - fmt.Fprintf(GinkgoWriter, "Expected IsAccessControlEnabled: %v\nGot: %v\n", expectedADBDetails.NetworkAccess.IsAccessControlEnabled, resp.AutonomousDatabase.IsAccessControlEnabled) + if !compareBool(expectedADBDetails.IsAccessControlEnabled, resp.AutonomousDatabase.IsAccessControlEnabled) { + fmt.Fprintf(GinkgoWriter, "Expected IsAccessControlEnabled: %v\nGot: %v\n", expectedADBDetails.IsAccessControlEnabled, resp.AutonomousDatabase.IsAccessControlEnabled) } - if !reflect.DeepEqual(expectedADBDetails.NetworkAccess.AccessControlList, resp.AutonomousDatabase.WhitelistedIps) { - fmt.Fprintf(GinkgoWriter, "Expected AccessControlList: %v\nGot: %v\n", expectedADBDetails.NetworkAccess.AccessControlList, resp.AutonomousDatabase.WhitelistedIps) + if !reflect.DeepEqual(expectedADBDetails.WhitelistedIps, resp.AutonomousDatabase.WhitelistedIps) { + fmt.Fprintf(GinkgoWriter, "Expected AccessControlList: %v\nGot: %v\n", expectedADBDetails.WhitelistedIps, resp.AutonomousDatabase.WhitelistedIps) } - if !compareBool(expectedADBDetails.NetworkAccess.IsMTLSConnectionRequired, resp.AutonomousDatabase.IsMtlsConnectionRequired) { - fmt.Fprintf(GinkgoWriter, "Expected IsMTLSConnectionRequired: %v\nGot: %v\n", expectedADBDetails.NetworkAccess.IsMTLSConnectionRequired, resp.AutonomousDatabase.IsMtlsConnectionRequired) + if !compareBool(expectedADBDetails.IsMtlsConnectionRequired, resp.AutonomousDatabase.IsMtlsConnectionRequired) { + fmt.Fprintf(GinkgoWriter, "Expected IsMTLSConnectionRequired: %v\nGot: %v\n", expectedADBDetails.IsMtlsConnectionRequired, resp.AutonomousDatabase.IsMtlsConnectionRequired) } - if !compareString(expectedADBDetails.NetworkAccess.PrivateEndpoint.SubnetOCID, resp.AutonomousDatabase.SubnetId) { - fmt.Fprintf(GinkgoWriter, "Expected SubnetOCID: %v\nGot: %v\n", expectedADBDetails.NetworkAccess.PrivateEndpoint.SubnetOCID, resp.AutonomousDatabase.SubnetId) + if !compareString(expectedADBDetails.SubnetId, resp.AutonomousDatabase.SubnetId) { + fmt.Fprintf(GinkgoWriter, "Expected SubnetOCID: %v\nGot: %v\n", expectedADBDetails.SubnetId, resp.AutonomousDatabase.SubnetId) } - if !reflect.DeepEqual(expectedADBDetails.NetworkAccess.PrivateEndpoint.NsgOCIDs, resp.AutonomousDatabase.NsgIds) { - fmt.Fprintf(GinkgoWriter, "Expected NsgOCIDs: %v\nGot: %v\n", expectedADBDetails.NetworkAccess.PrivateEndpoint.NsgOCIDs, resp.AutonomousDatabase.NsgIds) + if !reflect.DeepEqual(expectedADBDetails.NsgIds, resp.AutonomousDatabase.NsgIds) { + fmt.Fprintf(GinkgoWriter, "Expected NsgOCIDs: %v\nGot: %v\n", expectedADBDetails.NsgIds, resp.AutonomousDatabase.NsgIds) } - if !compareString(expectedADBDetails.NetworkAccess.PrivateEndpoint.HostnamePrefix, resp.AutonomousDatabase.PrivateEndpointLabel) { - fmt.Fprintf(GinkgoWriter, "Expected HostnamePrefix: %v\nGot: %v\n", expectedADBDetails.NetworkAccess.PrivateEndpoint.HostnamePrefix, resp.AutonomousDatabase.PrivateEndpointLabel) + if !compareString(expectedADBDetails.PrivateEndpointLabel, resp.AutonomousDatabase.PrivateEndpointLabel) { + fmt.Fprintf(GinkgoWriter, "Expected PrivateEndpointLabel: %v\nGot: %v\n", expectedADBDetails.PrivateEndpointLabel, resp.AutonomousDatabase.PrivateEndpointLabel) } } @@ -371,23 +372,23 @@ func AssertADBDetails(k8sClient *client.Client, // (e.g. adminPassword, wallet) are missing from e2eutil.GetAutonomousDatabase(). // We don't compare LifecycleState in this case. We only make sure that the ADB is in AVAIABLE state before // proceeding to the next test. - same := compareString(expectedADBDetails.AutonomousDatabaseOCID, resp.AutonomousDatabase.Id) && - compareString(expectedADBDetails.CompartmentOCID, resp.AutonomousDatabase.CompartmentId) && + same := compareString(expectedADBDetails.Id, resp.AutonomousDatabase.Id) && + compareString(expectedADBDetails.CompartmentId, resp.AutonomousDatabase.CompartmentId) && compareString(expectedADBDetails.DisplayName, resp.AutonomousDatabase.DisplayName) && compareString(expectedADBDetails.DbName, resp.AutonomousDatabase.DbName) && expectedADBDetails.DbWorkload == resp.AutonomousDatabase.DbWorkload && compareBool(expectedADBDetails.IsDedicated, resp.AutonomousDatabase.IsDedicated) && compareString(expectedADBDetails.DbVersion, resp.AutonomousDatabase.DbVersion) && compareInt(expectedADBDetails.DataStorageSizeInTBs, resp.AutonomousDatabase.DataStorageSizeInTBs) && - compareInt(expectedADBDetails.CPUCoreCount, resp.AutonomousDatabase.CpuCoreCount) && + compareInt(expectedADBDetails.CpuCoreCount, resp.AutonomousDatabase.CpuCoreCount) && compareBool(expectedADBDetails.IsAutoScalingEnabled, resp.AutonomousDatabase.IsAutoScalingEnabled) && compareStringMap(expectedADBDetails.FreeformTags, resp.AutonomousDatabase.FreeformTags) && - compareBool(expectedADBDetails.NetworkAccess.IsAccessControlEnabled, resp.AutonomousDatabase.IsAccessControlEnabled) && - reflect.DeepEqual(expectedADBDetails.NetworkAccess.AccessControlList, resp.AutonomousDatabase.WhitelistedIps) && - compareBool(expectedADBDetails.NetworkAccess.IsMTLSConnectionRequired, resp.AutonomousDatabase.IsMtlsConnectionRequired) && - compareString(expectedADBDetails.NetworkAccess.PrivateEndpoint.SubnetOCID, resp.AutonomousDatabase.SubnetId) && - reflect.DeepEqual(expectedADBDetails.NetworkAccess.PrivateEndpoint.NsgOCIDs, resp.AutonomousDatabase.NsgIds) && - compareString(expectedADBDetails.NetworkAccess.PrivateEndpoint.HostnamePrefix, resp.AutonomousDatabase.PrivateEndpointLabel) + compareBool(expectedADBDetails.IsAccessControlEnabled, resp.AutonomousDatabase.IsAccessControlEnabled) && + reflect.DeepEqual(expectedADBDetails.WhitelistedIps, resp.AutonomousDatabase.WhitelistedIps) && + compareBool(expectedADBDetails.IsMtlsConnectionRequired, resp.AutonomousDatabase.IsMtlsConnectionRequired) && + compareString(expectedADBDetails.SubnetId, resp.AutonomousDatabase.SubnetId) && + reflect.DeepEqual(expectedADBDetails.NsgIds, resp.AutonomousDatabase.NsgIds) && + compareString(expectedADBDetails.PrivateEndpointLabel, resp.AutonomousDatabase.PrivateEndpointLabel) return same, nil }, updateADBTimeout, intervalTime).Should(BeTrue()) @@ -398,15 +399,9 @@ func AssertADBDetails(k8sClient *client.Client, } } -func TestNetworkAccessRestricted(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName, isMTLSConnectionRequired bool) func() { +func TestNetworkAccessRestricted(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName, acl []string, isMTLSConnectionRequired bool) func() { return func() { - networkRestrictedSpec := dbv1alpha1.NetworkAccessSpec{ - AccessType: dbv1alpha1.NetworkAccessTypeRestricted, - IsMTLSConnectionRequired: common.Bool(isMTLSConnectionRequired), - AccessControlList: []string{"192.168.0.1"}, - } - - TestNetworkAccess(k8sClient, dbClient, adbLookupKey, networkRestrictedSpec)() + TestNetworkAccess(k8sClient, dbClient, adbLookupKey, nil, nil, acl, isMTLSConnectionRequired)() } } @@ -450,33 +445,17 @@ func TestNetworkAccessPrivate(k8sClient *client.Client, dbClient *database.Datab derefK8sClient := *k8sClient Expect(derefK8sClient.Get(context.TODO(), *adbLookupKey, adb)).Should(Succeed()) - networkPrivateSpec := dbv1alpha1.NetworkAccessSpec{ - AccessType: dbv1alpha1.NetworkAccessTypePrivate, - AccessControlList: []string{}, - IsMTLSConnectionRequired: common.Bool(isMTLSConnectionRequired), - PrivateEndpoint: dbv1alpha1.PrivateEndpointSpec{ - HostnamePrefix: adb.Spec.Details.DbName, - NsgOCIDs: []string{*nsgOCIDs}, - SubnetOCID: common.String(*subnetOCID), - }, - } - - TestNetworkAccess(k8sClient, dbClient, adbLookupKey, networkPrivateSpec)() + TestNetworkAccess(k8sClient, dbClient, adbLookupKey, subnetOCID, nsgOCIDs, nil, isMTLSConnectionRequired)() } } func TestNetworkAccessPublic(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName) func() { return func() { - networkPublicSpec := dbv1alpha1.NetworkAccessSpec{ - AccessType: dbv1alpha1.NetworkAccessTypePublic, - IsMTLSConnectionRequired: common.Bool(true), - } - - TestNetworkAccess(k8sClient, dbClient, adbLookupKey, networkPublicSpec)() + TestNetworkAccess(k8sClient, dbClient, adbLookupKey, nil, nil, nil, true)() } } -func TestNetworkAccess(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName, networkSpec dbv1alpha1.NetworkAccessSpec) func() { +func TestNetworkAccess(k8sClient *client.Client, dbClient *database.DatabaseClient, adbLookupKey *types.NamespacedName, subnetOCID *string, nsgOCIDs *string, acl []string, isMTLSConnectionRequired bool) func() { return func() { Expect(k8sClient).NotTo(BeNil()) Expect(dbClient).NotTo(BeNil()) @@ -488,7 +467,10 @@ func TestNetworkAccess(k8sClient *client.Client, dbClient *database.DatabaseClie AssertADBState(k8sClient, dbClient, adbLookupKey, database.AutonomousDatabaseLifecycleStateAvailable)() Expect(derefK8sClient.Get(context.TODO(), *adbLookupKey, adb)).To(Succeed()) - adb.Spec.Details.NetworkAccess = networkSpec + adb.Spec.Details.SubnetId = subnetOCID + adb.Spec.Details.NsgIds = []string{*nsgOCIDs} + adb.Spec.Details.WhitelistedIps = acl + adb.Spec.Details.IsMtlsConnectionRequired = common.Bool(isMTLSConnectionRequired) Expect(derefK8sClient.Update(context.TODO(), adb)).To(Succeed()) AssertADBDetails(k8sClient, dbClient, adbLookupKey, adb)() } @@ -503,7 +485,7 @@ func UpdateAndAssertDetails(k8sClient *client.Client, dbClient *database.Databas expectedADB := UpdateDetails(k8sClient, dbClient, adbLookupKey, newSecretName, newAdminPassword)() AssertADBDetails(k8sClient, dbClient, adbLookupKey, expectedADB)() - ocid := expectedADB.Spec.Details.AutonomousDatabaseOCID + ocid := expectedADB.Spec.Details.Id tnsEntry := *expectedADB.Spec.Details.DbName + "_high" err := AssertAdminPassword(dbClient, ocid, &tnsEntry, newAdminPassword, walletPassword) Expect(err).ShouldNot(HaveOccurred()) @@ -547,7 +529,7 @@ func AssertHardLinkDelete(k8sClient *client.Client, dbClient *database.DatabaseC // Check every 10 secs for total 60 secs Eventually(func() (database.AutonomousDatabaseLifecycleStateEnum, error) { retryPolicy := e2eutil.NewLifecycleStateRetryPolicyADB(database.AutonomousDatabaseLifecycleStateTerminating) - return returnADBRemoteState(derefK8sClient, derefDBClient, adb.Spec.Details.AutonomousDatabaseOCID, &retryPolicy) + return returnADBRemoteState(derefK8sClient, derefDBClient, adb.Spec.Details.Id, &retryPolicy) }, changeTimeout).Should(Equal(database.AutonomousDatabaseLifecycleStateTerminating)) AssertSoftLinkDelete(k8sClient, adbLookupKey)() @@ -606,7 +588,7 @@ func AssertADBRemoteState(k8sClient *client.Client, dbClient *database.DatabaseC adb := &dbv1alpha1.AutonomousDatabase{} Expect(derefK8sClient.Get(context.TODO(), *adbLookupKey, adb)).To(Succeed()) By("Checking if the lifecycleState of remote resource is " + string(state)) - AssertADBRemoteStateOCID(k8sClient, dbClient, adb.Spec.Details.AutonomousDatabaseOCID, state, changeTimeout)() + AssertADBRemoteStateOCID(k8sClient, dbClient, adb.Spec.Details.Id, state, changeTimeout)() } } @@ -622,7 +604,7 @@ func AssertADBRemoteStateForBackupRestore(k8sClient *client.Client, dbClient *da adb := &dbv1alpha1.AutonomousDatabase{} Expect(derefK8sClient.Get(context.TODO(), *adbLookupKey, adb)).To(Succeed()) By("Checking if the lifecycleState of remote resource is " + string(state)) - AssertADBRemoteStateOCID(k8sClient, dbClient, adb.Spec.Details.AutonomousDatabaseOCID, state, backupTimeout)() + AssertADBRemoteStateOCID(k8sClient, dbClient, adb.Spec.Details.Id, state, backupTimeout)() } } @@ -655,8 +637,15 @@ func UpdateState(k8sClient *client.Client, adbLookupKey *types.NamespacedName, s adb := &dbv1alpha1.AutonomousDatabase{} Expect(derefK8sClient.Get(context.TODO(), *adbLookupKey, adb)).To(Succeed()) - adb.Spec.Details.LifecycleState = state By("Updating adb state to " + string(state)) + switch state { + case database.AutonomousDatabaseLifecycleStateAvailable: + adb.Spec.Action = "Start" + case database.AutonomousDatabaseLifecycleStateStopped: + adb.Spec.Action = "Stop" + case database.AutonomousDatabaseLifecycleStateTerminated: + adb.Spec.Action = "Terminate" + } Expect(derefK8sClient.Update(context.TODO(), adb)).To(Succeed()) } } diff --git a/test/e2e/suite_test.go b/test/e2e/suite_test.go index a9fa06e5..9d9914f7 100644 --- a/test/e2e/suite_test.go +++ b/test/e2e/suite_test.go @@ -62,7 +62,7 @@ import ( databasev1alpha1 "github.com/oracle/oracle-database-operator/apis/database/v1alpha1" controllers "github.com/oracle/oracle-database-operator/controllers/database" - "github.com/oracle/oracle-database-operator/test/e2e/util" + e2eutil "github.com/oracle/oracle-database-operator/test/e2e/util" // +kubebuilder:scaffold:imports ) @@ -309,9 +309,9 @@ var _ = AfterSuite(func() { By(fmt.Sprintf("Found %d AutonomousDatabase(s)", len(adbList.Items))) for _, adb := range adbList.Items { - if adb.Spec.Details.AutonomousDatabaseOCID != nil { + if adb.Spec.Details.Id != nil { By("Terminating database " + *adb.Spec.Details.DbName) - Expect(e2eutil.DeleteAutonomousDatabase(dbClient, adb.Spec.Details.AutonomousDatabaseOCID)).Should(Succeed()) + Expect(e2eutil.DeleteAutonomousDatabase(dbClient, adb.Spec.Details.Id)).Should(Succeed()) } } From ffb4024b00c32eec111d63b694f8fd17abac2630 Mon Sep 17 00:00:00 2001 From: matteo malvezzi Date: Thu, 27 Mar 2025 09:00:04 +0100 Subject: [PATCH 621/628] lrest doc correction (#165) Co-authored-by: oracle database --- docs/multitenant/lrest-based/README.md | 31 ++++++------- docs/multitenant/lrest-based/usecase/makefile | 44 +++++++++---------- 2 files changed, 38 insertions(+), 37 deletions(-) diff --git a/docs/multitenant/lrest-based/README.md b/docs/multitenant/lrest-based/README.md index f17abd41..d9b72a9d 100644 --- a/docs/multitenant/lrest-based/README.md +++ b/docs/multitenant/lrest-based/README.md @@ -274,10 +274,11 @@ Parsing sqltext=select count(*) from pdb_plug_in_violations where name =:b1 | Name | Dcription | --------------------------|-------------------------------------------------------------------------------| |cdbName | Name of the container database (db) | -|lrestImage (DO NOT EDIT) | **container-registry.oracle.com/database/lrest-dboper:latest** | -|dbTnsurl | TNS alias of the container db | +|lrestImage (DO NOT EDIT) | **container-registry.oracle.com/database/lrest-dboper:latest** use the latest label availble on OCR | +|dbTnsurl | The string of the tns alias to connect to cdb. Attention: remove all white space from string | |deletePdbCascade | Delete all of the PDBs associated to a CDB resource when the CDB resource is dropped using [imperative approach](https://kubernetes.io/docs/tasks/manage-kubernetes-objects/imperative-command/) | |cdbAdminUser | Secret: the administrative (admin) user | +|fileNameConversions | Use file name conversion if you are not using ASM | |cdbAdminPwd | Secret: the admin user password | |webServerUser | Secret: the HTTPS user | |webServerPwd | Secret: the HTTPS user password | @@ -290,10 +291,10 @@ Parsing sqltext=select count(*) from pdb_plug_in_violations where name =:b1 ### Create PDB -To create a pluggable database (PDB), apply the yaml file [`create_lrpdb1_resource.yaml`](./usecase/clone_lrpdb1_resource.yaml) +To create a pluggable database, apply the yaml file [`create_pdb1_resource.yaml`](./usecase/create_pdb1_resource.yaml) ```bash -kubectl apply -f create_lrpdb1_resource.yaml +kubectl apply -f create_pdb1_resource.yaml ``` Check the status of the resource and the PDB existence on the container db: @@ -391,10 +392,10 @@ test_invalid_parameter;16;spfile ### Open PDB -To open the PDB, use the file [`open_lrpdb1_resource.yaml`](./usecase/open_lrpdb1_resource.yaml): +To open the PDB, use the file [`open_pdb1_resource.yaml`](./usecase/open_pdb1_resource.yaml): ```bash -kubectl apply -f open_lrpdb1_resource.yaml +kubectl apply -f open_pdb1_resource.yaml ``` **pdb opening** - parameters list @@ -411,10 +412,10 @@ kubectl apply -f open_lrpdb1_resource.yaml ### Close PDB -To close the PDB, use the file [`close_lrpdb1_resource.yaml`](./usecase/close_lrpdb1_resource.yaml): +To close the PDB, use the file [`close_pdb1_resource.yaml`](./usecase/close_pdb1_resource.yaml): ```bash -kubectl apply -f close_lrpdb1_resource.yaml +kubectl apply -f close_pdb1_resource.yaml ``` **pdb closing** - parameters list | Name | Description/Value | @@ -429,10 +430,10 @@ kubectl apply -f close_lrpdb1_resource.yaml ### Clone PDB ### -To clone the PDB, use the file [`clone_lrpdb1_resource.yaml`](./usecase/clone_lrpdb1_resource.yaml): +To clone the PDB, use the file [`clone_pdb1_resource.yaml`](./usecase/clone_pdb1_resource.yaml): ```bash -kubeclt apply -f clone_lrpdb1_resource.yaml +kubeclt apply -f clone_pdb1_resource.yaml ``` **pdb cloning** - parameters list | Name | Description/Value | @@ -450,7 +451,7 @@ kubeclt apply -f clone_lrpdb1_resource.yaml ### Unplug PDB -To unplug the PDB, use the file [`unplug_lrpdb1_resource.yaml`](./usecase/unplug_lrpdb1_resource.yaml): +To unplug the PDB, use the file [`unplug_pdb1_resource.yaml`](./usecase/unplug_pdb1_resource.yaml): **pdb unplugging** | Name | Description/Value | @@ -461,7 +462,7 @@ To unplug the PDB, use the file [`unplug_lrpdb1_resource.yaml`](./usecase/unplug |pdbName | Name of the pluggable database (PDB)| ### Plug PDB -To plug in the PDB, use the file [`plug_lrpdb1_resource.yaml`](./usecase/plug_lrpdb1_resource.yaml). In this example, we plug in the PDB that was unpluged in the previous step: +To plug in the PDB, use the file [`plug_pdb1_resource.yaml`](./usecase/plug_pdb1_resource.yaml). In this example, we plug in the PDB that was unpluged in the previous step: **pdb plugging** | Name | Description/Value | @@ -478,7 +479,7 @@ To plug in the PDB, use the file [`plug_lrpdb1_resource.yaml`](./usecase/plug_lr ### Delete PDB -To delete the PDB, use the file [`delete_lrpdb1_resource.yaml`](./usecase/delete_lrpdb1_resource.yaml) +To delete the PDB, use the file [`delete_pdb1_resource.yaml`](./usecase/delete_pdb1_resource.yaml) **pdb deletion** @@ -493,8 +494,8 @@ To delete the PDB, use the file [`delete_lrpdb1_resource.yaml`](./usecase/delete ### Map PDB -If you need to create a CRD for an existing PDB, then you can use the map option by applying the file [`map_lrpdb1_resource.yaml`](./usecase/map_lrpdb1_resource.yaml) - +If you need to create a CRD for an existing PDB, then you can use the map option by applying the file [`map_pdb1_resource.yaml`](./usecase/map_pdb1_resource.yaml) +Map functionality can be used in a situation where you have a pdb which is not registered in the operator as a CRD. It's a temporary solution while waiting the autodiscovery to be available. diff --git a/docs/multitenant/lrest-based/usecase/makefile b/docs/multitenant/lrest-based/usecase/makefile index 4203baa4..1de320ad 100644 --- a/docs/multitenant/lrest-based/usecase/makefile +++ b/docs/multitenant/lrest-based/usecase/makefile @@ -31,27 +31,27 @@ # # ----------------------------- ---------------------------------- # oracle-database-operator.yaml : oracle database operator -# cdbnamespace_binding.yaml : role binding for cdbnamespace +# lrestnamespace_binding.yaml : role binding for lrestnamespace # pdbnamespace_binding.yaml : role binding for pdbnamespace # create_lrest_secret.yaml : create secrets for rest server pod # create_lrpdb_secret.yaml : create secrets for pluggable database # create_lrest_pod.yaml : create rest server pod -# create_pdb1_resource.yaml : create first pluggable database -# create_pdb2_resource.yaml : create second pluggable database -# open_pdb1_resource.yaml : open first pluggable database -# open_pdb2_resource.yaml : open second pluggable database -# close_pdb1_resource.yaml : close first pluggable database -# close_pdb2_resource.yaml : close second pluggable database +# create_pdb1_resource.yaml : create first pluggable database +# create_pdb2_resource.yaml : create second pluggable database +# open_pdb1_resource.yaml : open first pluggable database +# open_pdb2_resource.yaml : open second pluggable database +# close_pdb1_resource.yaml : close first pluggable database +# close_pdb2_resource.yaml : close second pluggable database # clone_lrpdb_resource.yaml : clone thrid pluggable database -# clone_pdb2_resource.yaml : clone 4th pluggable database -# delete_pdb1_resource.yaml : delete first pluggable database -# delete_pdb2_resource.yaml : delete sencond pluggable database -# delete_pdb3_resource.yaml : delete thrid pluggable database -# unplug_pdb1_resource.yaml : unplug first pluggable database -# plug_pdb1_resource.yaml : plug first pluggable database -# map_pdb1_resource.yaml : map the first pluggable database +# clone_pdb2_resource.yaml : clone 4th pluggable database +# delete_pdb1_resource.yaml : delete first pluggable database +# delete_pdb2_resource.yaml : delete sencond pluggable database +# delete_pdb3_resource.yaml : delete thrid pluggable database +# unplug_pdb1_resource.yaml : unplug first pluggable database +# plug_pdb1_resource.yaml : plug first pluggable database +# map_pdb1_resource.yaml : map the first pluggable database # config_map.yam : pdb parameters array -# altersystem_pdb1_resource.yaml : chage cpu_count count parameter for the first pdb +# altersystem_pdb1_resource.yaml : chage cpu_count count parameter for the first pdb # DATE := `date "+%y%m%d%H%M%S"` ###################### @@ -328,7 +328,7 @@ apiVersion: database.oracle.com/${APIVERSION} kind: LREST metadata: name: cdb-dev - namespace: cdbnamespace + namespace: ${LRSNAMESPACE} spec: cdbName: "DB12" lrestImage: ${LRESTIMG} @@ -524,7 +524,7 @@ spec: cdbResName: "cdb-dev" cdbNamespace: "${LRSNAMESPACE}" cdbName: "DB12" - pdbName: ""new_clone" + pdbName: "new_clone" pdbState: "CLOSE" modifyOption: "IMMEDIATE" action: "Modify" @@ -619,7 +619,7 @@ spec: cdbNamespace: "${LRSNAMESPACE}" cdbName: "DB12" pdbName: "pdbdev" - xmlFileName: "/tmp/pdb.xml" + xmlFileName: "/var/tmp/pdb.$$.xml" action: "Unplug" EOF @@ -636,7 +636,7 @@ spec: cdbNamespace: "${LRSNAMESPACE}" cdbName: "DB12" pdbName: "pdbdev" - xmlFileName: "/tmp/pdb.xml" + xmlFileName: "/var/tmp/pdb.$$.xml" action: "plug" fileNameConversions: "NONE" sourceFileNameConversions: "NONE" @@ -901,10 +901,10 @@ run07.1: $(KUBECTL) get lrpdb -n $(PDBNAMESPACE) run99.1: - $(KUBECTL) delete lrest cdb-dev -n cdbnamespace + $(KUBECTL) delete lrest cdb-dev -n $(LRSNAMESPACE) $(KUBECTL) wait --for=delete lrest cdb-dev -n $(LRSNAMESPACE) --timeout=$(TEST_EXEC_TIMEOUT) - $(KUBECTL) get lrest -n cdbnamespaace - $(KUBECTL) get lrpdb -n pdbnamespaace + $(KUBECTL) get lrest -n $(LRSNAMESPACE) + $(KUBECTL) get lrpdb -n $(PDBNAMESPACE) runall01: run00 run01.1 run01.2 run02.1 run02.2 run03.1 run03.2 run04.1 run04.2 run05.1 run06.1 run07.1 From d4edab26470d7435846c3224a16b6b07ec7f14e2 Mon Sep 17 00:00:00 2001 From: matteo malvezzi Date: Sat, 29 Mar 2025 22:13:29 +0100 Subject: [PATCH 622/628] V1 cdb (#167) * cdb_types v1alpha1 placeholder * pdb types v1alpha1 placeholder --------- Co-authored-by: oracle database --- apis/database/v1alpha1/cdb_types.go | 190 ++++++++++++++++++++++ apis/database/v1alpha1/pdb_types.go | 236 ++++++++++++++++++++++++++++ 2 files changed, 426 insertions(+) create mode 100644 apis/database/v1alpha1/cdb_types.go create mode 100644 apis/database/v1alpha1/pdb_types.go diff --git a/apis/database/v1alpha1/cdb_types.go b/apis/database/v1alpha1/cdb_types.go new file mode 100644 index 00000000..f97df391 --- /dev/null +++ b/apis/database/v1alpha1/cdb_types.go @@ -0,0 +1,190 @@ +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v1alpha1 + +import ( + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +// CDBSpec defines the desired state of CDB +type CDBSpec struct { + // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster + // Important: Run "make" to regenerate code after modifying this file + + // Name of the CDB + CDBName string `json:"cdbName,omitempty"` + // Name of the CDB Service + ServiceName string `json:"serviceName,omitempty"` + + // Password for the CDB System Administrator + SysAdminPwd CDBSysAdminPassword `json:"sysAdminPwd,omitempty"` + // User in the root container with sysdba priviledges to manage PDB lifecycle + CDBAdminUser CDBAdminUser `json:"cdbAdminUser,omitempty"` + // Password for the CDB Administrator to manage PDB lifecycle + CDBAdminPwd CDBAdminPassword `json:"cdbAdminPwd,omitempty"` + + CDBTlsKey CDBTLSKEY `json:"cdbTlsKey,omitempty"` + CDBTlsCrt CDBTLSCRT `json:"cdbTlsCrt,omitempty"` + + // Password for user ORDS_PUBLIC_USER + ORDSPwd ORDSPassword `json:"ordsPwd,omitempty"` + // ORDS server port. For now, keep it as 8888. TO BE USED IN FUTURE RELEASE. + ORDSPort int `json:"ordsPort,omitempty"` + // ORDS Image Name + ORDSImage string `json:"ordsImage,omitempty"` + // The name of the image pull secret in case of a private docker repository. + ORDSImagePullSecret string `json:"ordsImagePullSecret,omitempty"` + // ORDS Image Pull Policy + // +kubebuilder:validation:Enum=Always;Never + ORDSImagePullPolicy string `json:"ordsImagePullPolicy,omitempty"` + // Number of ORDS Containers to create + Replicas int `json:"replicas,omitempty"` + // Web Server User with SQL Administrator role to allow us to authenticate to the PDB Lifecycle Management REST endpoints + WebServerUser WebServerUser `json:"webServerUser,omitempty"` + // Password for the Web Server User + WebServerPwd WebServerPassword `json:"webServerPwd,omitempty"` + // Name of the DB server + DBServer string `json:"dbServer,omitempty"` + // DB server port + DBPort int `json:"dbPort,omitempty"` + // Node Selector for running the Pod + NodeSelector map[string]string `json:"nodeSelector,omitempty"` + DeletePDBCascade bool `json:"deletePdbCascade,omitempty"` + DBTnsurl string `json:"dbTnsurl,omitempty"` + CDBPubKey CDBPUBKEY `json:"cdbOrdsPubKey,omitempty"` + CDBPriKey CDBPRIVKEY `json:"cdbOrdsPrvKey,omitempty"` +} + +// CDBSecret defines the secretName +type CDBSecret struct { + SecretName string `json:"secretName"` + Key string `json:"key"` +} + +// CDBSysAdminPassword defines the secret containing SysAdmin Password mapped to key 'sysAdminPwd' for CDB +type CDBSysAdminPassword struct { + Secret CDBSecret `json:"secret"` +} + +// CDBAdminUser defines the secret containing CDB Administrator User mapped to key 'cdbAdminUser' to manage PDB lifecycle +type CDBAdminUser struct { + Secret CDBSecret `json:"secret"` +} + +// CDBAdminPassword defines the secret containing CDB Administrator Password mapped to key 'cdbAdminPwd' to manage PDB lifecycle +type CDBAdminPassword struct { + Secret CDBSecret `json:"secret"` +} + +// ORDSPassword defines the secret containing ORDS_PUBLIC_USER Password mapped to key 'ordsPwd' +type ORDSPassword struct { + Secret CDBSecret `json:"secret"` +} + +// WebServerUser defines the secret containing Web Server User mapped to key 'webServerUser' to manage PDB lifecycle +type WebServerUser struct { + Secret CDBSecret `json:"secret"` +} + +// WebServerPassword defines the secret containing password for Web Server User mapped to key 'webServerPwd' to manage PDB lifecycle +type WebServerPassword struct { + Secret CDBSecret `json:"secret"` +} + +type CDBTLSKEY struct { + Secret CDBSecret `json:"secret"` +} + +type CDBTLSCRT struct { + Secret CDBSecret `json:"secret"` +} + +type CDBPUBKEY struct { + Secret CDBSecret `json:"secret"` +} + +type CDBPRIVKEY struct { + Secret CDBSecret `json:"secret"` +} + +// CDBStatus defines the observed state of CDB +type CDBStatus struct { + // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster + // Important: Run "make" to regenerate code after modifying this file + + // Phase of the CDB Resource + Phase string `json:"phase"` + // CDB Resource Status + Status bool `json:"status"` + // Message + Msg string `json:"msg,omitempty"` +} + +// +kubebuilder:object:root=true +// +kubebuilder:subresource:status +// +kubebuilder:printcolumn:JSONPath=".spec.cdbName",name="CDB Name",type="string",description="Name of the CDB" +// +kubebuilder:printcolumn:JSONPath=".spec.dbServer",name="DB Server",type="string",description=" Name of the DB Server" +// +kubebuilder:printcolumn:JSONPath=".spec.dbPort",name="DB Port",type="integer",description="DB server port" +// +kubebuilder:printcolumn:JSONPath=".spec.replicas",name="Replicas",type="integer",description="Replicas" +// +kubebuilder:printcolumn:JSONPath=".status.phase",name="Status",type="string",description="Status of the CDB Resource" +// +kubebuilder:printcolumn:JSONPath=".status.msg",name="Message",type="string",description="Error message, if any" +// +kubebuilder:printcolumn:JSONPath=".spec.dbTnsurl",name="TNS STRING",type="string",description=" string of the tnsalias" +// +kubebuilder:resource:path=cdbs,scope=Namespaced + +// CDB is the Schema for the cdbs API +type CDB struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + Spec CDBSpec `json:"spec,omitempty"` + Status CDBStatus `json:"status,omitempty"` +} + +//+kubebuilder:object:root=true + +// CDBList contains a list of CDB +type CDBList struct { + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata,omitempty"` + Items []CDB `json:"items"` +} + +func init() { + SchemeBuilder.Register(&CDB{}, &CDBList{}) +} diff --git a/apis/database/v1alpha1/pdb_types.go b/apis/database/v1alpha1/pdb_types.go new file mode 100644 index 00000000..8b966c38 --- /dev/null +++ b/apis/database/v1alpha1/pdb_types.go @@ -0,0 +1,236 @@ +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v1alpha1 + +import ( + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +// PDBSpec defines the desired state of PDB +type PDBSpec struct { + // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster + // Important: Run "make" to regenerate code after modifying this file + + PDBTlsKey PDBTLSKEY `json:"pdbTlsKey,omitempty"` + PDBTlsCrt PDBTLSCRT `json:"pdbTlsCrt,omitempty"` + PDBTlsCat PDBTLSCAT `json:"pdbTlsCat,omitempty"` + + // CDB Namespace + CDBNamespace string `json:"cdbNamespace,omitempty"` + // Name of the CDB Custom Resource that runs the ORDS container + CDBResName string `json:"cdbResName,omitempty"` + // Name of the CDB + CDBName string `json:"cdbName,omitempty"` + // The name of the new PDB. Relevant for both Create and Plug Actions. + PDBName string `json:"pdbName,omitempty"` + // Name of the Source PDB from which to clone + SrcPDBName string `json:"srcPdbName,omitempty"` + // The administrator username for the new PDB. This property is required when the Action property is Create. + AdminName PDBAdminName `json:"adminName,omitempty"` + // The administrator password for the new PDB. This property is required when the Action property is Create. + AdminPwd PDBAdminPassword `json:"adminPwd,omitempty"` + // Web Server User with SQL Administrator role to allow us to authenticate to the PDB Lifecycle Management REST endpoints + WebServerUsr WebServerUserPDB `json:"webServerUser,omitempty"` + // Password for the Web ServerPDB User + WebServerPwd WebServerPasswordPDB `json:"webServerPwd,omitempty"` + // Relevant for Create and Plug operations. As defined in the Oracle Multitenant Database documentation. Values can be a filename convert pattern or NONE. + FileNameConversions string `json:"fileNameConversions,omitempty"` + // This property is required when the Action property is Plug. As defined in the Oracle Multitenant Database documentation. Values can be a source filename convert pattern or NONE. + SourceFileNameConversions string `json:"sourceFileNameConversions,omitempty"` + // XML metadata filename to be used for Plug or Unplug operations + XMLFileName string `json:"xmlFileName,omitempty"` + // To copy files or not while cloning a PDB + // +kubebuilder:validation:Enum=COPY;NOCOPY;MOVE + CopyAction string `json:"copyAction,omitempty"` + // Specify if datafiles should be removed or not. The value can be INCLUDING or KEEP (default). + // +kubebuilder:validation:Enum=INCLUDING;KEEP + DropAction string `json:"dropAction,omitempty"` + // A Path specified for sparse clone snapshot copy. (Optional) + SparseClonePath string `json:"sparseClonePath,omitempty"` + // Whether to reuse temp file + ReuseTempFile *bool `json:"reuseTempFile,omitempty"` + // Relevant for Create and Plug operations. True for unlimited storage. Even when set to true, totalSize and tempSize MUST be specified in the request if Action is Create. + UnlimitedStorage *bool `json:"unlimitedStorage,omitempty"` + // Indicate if 'AS CLONE' option should be used in the command to plug in a PDB. This property is applicable when the Action property is PLUG but not required. + AsClone *bool `json:"asClone,omitempty"` + // Relevant for create and plug operations. Total size as defined in the Oracle Multitenant Database documentation. See size_clause description in Database SQL Language Reference documentation. + TotalSize string `json:"totalSize,omitempty"` + // Relevant for Create and Clone operations. Total size for temporary tablespace as defined in the Oracle Multitenant Database documentation. See size_clause description in Database SQL Language Reference documentation. + TempSize string `json:"tempSize,omitempty"` + // TDE import for plug operations + TDEImport *bool `json:"tdeImport,omitempty"` + // TDE export for unplug operations + TDEExport *bool `json:"tdeExport,omitempty"` + // TDE password if the tdeImport or tdeExport flag is set to true. Can be used in create, plug or unplug operations + TDEPassword TDEPwd `json:"tdePassword,omitempty"` + // TDE keystore path is required if the tdeImport or tdeExport flag is set to true. Can be used in plug or unplug operations. + TDEKeystorePath string `json:"tdeKeystorePath,omitempty"` + // TDE secret is required if the tdeImport or tdeExport flag is set to true. Can be used in plug or unplug operations. + TDESecret TDESecret `json:"tdeSecret,omitempty"` + // Whether you need the script only or execute the script + GetScript *bool `json:"getScript,omitempty"` + // Action to be taken: Create/Clone/Plug/Unplug/Delete/Modify/Status/Map. Map is used to map a Databse PDB to a Kubernetes PDB CR. + // +kubebuilder:validation:Enum=Create;Clone;Plug;Unplug;Delete;Modify;Status;Map + Action string `json:"action"` + // Extra options for opening and closing a PDB + // +kubebuilder:validation:Enum=IMMEDIATE;NORMAL;READ ONLY;READ WRITE;RESTRICTED + ModifyOption string `json:"modifyOption,omitempty"` + // The target state of the PDB + // +kubebuilder:validation:Enum=OPEN;CLOSE + PDBState string `json:"pdbState,omitempty"` + // turn on the assertive approach to delete pdb resource + // kubectl delete pdb ..... automatically triggers the pluggable database + // deletion + AssertivePdbDeletion bool `json:"assertivePdbDeletion,omitempty"` + PDBPubKey PDBPUBKEY `json:"pdbOrdsPubKey,omitempty"` + PDBPriKey PDBPRIVKEY `json:"pdbOrdsPrvKey,omitempty"` +} + +// PDBAdminName defines the secret containing Sys Admin User mapped to key 'adminName' for PDB +type PDBAdminName struct { + Secret PDBSecret `json:"secret"` +} + +// PDBAdminPassword defines the secret containing Sys Admin Password mapped to key 'adminPwd' for PDB +type PDBAdminPassword struct { + Secret PDBSecret `json:"secret"` +} + +// TDEPwd defines the secret containing TDE Wallet Password mapped to key 'tdePassword' for PDB +type TDEPwd struct { + Secret PDBSecret `json:"secret"` +} + +// TDESecret defines the secret containing TDE Secret to key 'tdeSecret' for PDB +type TDESecret struct { + Secret PDBSecret `json:"secret"` +} + +// WebServerUser defines the secret containing Web Server User mapped to key 'webServerUser' to manage PDB lifecycle + +type WebServerUserPDB struct { + Secret PDBSecret `json:"secret"` +} + +// WebServerPassword defines the secret containing password for Web Server User mapped to key 'webServerPwd' to manage PDB lifecycle +type WebServerPasswordPDB struct { + Secret PDBSecret `json:"secret"` +} + +// PDBSecret defines the secretName +type PDBSecret struct { + SecretName string `json:"secretName"` + Key string `json:"key"` +} + +type PDBTLSKEY struct { + Secret PDBSecret `json:"secret"` +} + +type PDBTLSCRT struct { + Secret PDBSecret `json:"secret"` +} + +type PDBTLSCAT struct { + Secret PDBSecret `json:"secret"` +} + +// PDBStatus defines the observed state of PDB +type PDBStatus struct { + // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster + // Important: Run "make" to regenerate code after modifying this file + + // PDB Connect String + ConnString string `json:"connString,omitempty"` + // Phase of the PDB Resource + Phase string `json:"phase"` + // PDB Resource Status + Status bool `json:"status"` + // Total size of the PDB + TotalSize string `json:"totalSize,omitempty"` + // Open mode of the PDB + OpenMode string `json:"openMode,omitempty"` + // Modify Option of the PDB + ModifyOption string `json:"modifyOption,omitempty"` + // Message + Msg string `json:"msg,omitempty"` + // Last Completed Action + Action string `json:"action,omitempty"` +} + +// +kubebuilder:object:root=true +// +kubebuilder:subresource:status +// +kubebuilder:printcolumn:JSONPath=".spec.cdbName",name="CDB Name",type="string",description="Name of the CDB" +// +kubebuilder:printcolumn:JSONPath=".spec.pdbName",name="PDB Name",type="string",description="Name of the PDB" +// +kubebuilder:printcolumn:JSONPath=".status.openMode",name="PDB State",type="string",description="PDB Open Mode" +// +kubebuilder:printcolumn:JSONPath=".status.totalSize",name="PDB Size",type="string",description="Total Size of the PDB" +// +kubebuilder:printcolumn:JSONPath=".status.phase",name="Status",type="string",description="Status of the PDB Resource" +// +kubebuilder:printcolumn:JSONPath=".status.msg",name="Message",type="string",description="Error message, if any" +// +kubebuilder:printcolumn:JSONPath=".status.connString",name="Connect_String",type="string",description="The connect string to be used" +// +kubebuilder:resource:path=pdbs,scope=Namespaced + +// PDB is the Schema for the pdbs API +type PDB struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + Spec PDBSpec `json:"spec,omitempty"` + Status PDBStatus `json:"status,omitempty"` +} + +//+kubebuilder:object:root=true + +// PDBList contains a list of PDB +type PDBList struct { + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata,omitempty"` + Items []PDB `json:"items"` +} + +type PDBPUBKEY struct { + Secret PDBSecret `json:"secret"` +} + +type PDBPRIVKEY struct { + Secret PDBSecret `json:"secret"` +} + +func init() { + SchemeBuilder.Register(&PDB{}, &PDBList{}) +} From ba8419e3464d43115c823cc3dd0487c2652dc341 Mon Sep 17 00:00:00 2001 From: jpverma85 Date: Wed, 2 Apr 2025 13:06:38 -0400 Subject: [PATCH 623/628] doc changes (#172) --- docs/sharding/README.md | 76 ++----------------- .../create_kubernetes_secret_for_db_user.md | 2 + .../sharding_provisioning_with_free_images.md | 5 +- ...harding_provisioning_with_free_images.yaml | 4 +- ...y_cloning_db_from_gold_image_across_ads.md | 3 +- ...ing_by_cloning_db_gold_image_in_same_ad.md | 3 +- ...ding_provisioning_with_chunks_specified.md | 3 +- ..._provisioning_with_control_on_resources.md | 3 +- ...ith_notification_using_oci_notification.md | 3 +- ...ding_provisioning_without_db_gold_image.md | 3 +- ...rding_scale_in_delete_an_existing_shard.md | 3 +- .../snr_ssharding_scale_out_add_shards.md | 3 +- .../snr_ssharding_shard_prov.yaml | 3 +- .../snr_ssharding_shard_prov_chunks.yaml | 3 +- .../snr_ssharding_shard_prov_clone.yaml | 3 +- ...ssharding_shard_prov_clone_across_ads.yaml | 3 +- .../snr_ssharding_shard_prov_delshard.yaml | 3 +- .../snr_ssharding_shard_prov_extshard.yaml | 3 +- .../snr_ssharding_shard_prov_memory_cpu.yaml | 3 +- ...sharding_shard_prov_send_notification.yaml | 3 +- ...y_cloning_db_from_gold_image_across_ads.md | 1 + ...ing_by_cloning_db_gold_image_in_same_ad.md | 1 + ...ding_provisioning_with_chunks_specified.md | 1 + ..._provisioning_with_control_on_resources.md | 1 + ...ith_notification_using_oci_notification.md | 1 + ...ding_provisioning_without_db_gold_image.md | 2 +- ...rding_scale_in_delete_an_existing_shard.md | 1 + .../ssharding_scale_out_add_shards.md | 1 + .../system_sharding/ssharding_shard_prov.yaml | 4 +- .../ssharding_shard_prov_chunks.yaml | 4 +- .../ssharding_shard_prov_clone.yaml | 4 +- ...ssharding_shard_prov_clone_across_ads.yaml | 4 +- .../ssharding_shard_prov_delshard.yaml | 4 +- .../ssharding_shard_prov_extshard.yaml | 4 +- .../ssharding_shard_prov_memory_cpu.yaml | 4 +- ...sharding_shard_prov_send_notification.yaml | 4 +- ...y_cloning_db_from_gold_image_across_ads.md | 1 + ...ing_by_cloning_db_gold_image_in_same_ad.md | 1 + ..._provisioning_with_control_on_resources.md | 1 + ...ith_notification_using_oci_notification.md | 1 + ...ding_provisioning_without_db_gold_image.md | 1 + ...rding_scale_in_delete_an_existing_shard.md | 1 + .../udsharding_scale_out_add_shards.md | 1 + .../udsharding_shard_prov.yaml | 4 +- .../udsharding_shard_prov_clone.yaml | 4 +- ...dsharding_shard_prov_clone_across_ads.yaml | 4 +- .../udsharding_shard_prov_delshard.yaml | 4 +- .../udsharding_shard_prov_extshard.yaml | 4 +- .../udsharding_shard_prov_memory_cpu.yaml | 4 +- ...sharding_shard_prov_send_notification.yaml | 4 +- 50 files changed, 91 insertions(+), 120 deletions(-) diff --git a/docs/sharding/README.md b/docs/sharding/README.md index 661e2546..487d9ec3 100644 --- a/docs/sharding/README.md +++ b/docs/sharding/README.md @@ -72,7 +72,7 @@ Choose one of the following deployment options: **Use Oracle-Supplied Docker Images:** The Oracle Sharding Database controller uses Oracle Global Data Services and Oracle Database images to provision the sharding topology. - You can also download the pre-built Oracle Global Data Services `container-registry.oracle.com/database/gsm:latest` and Oracle Database images `container-registry.oracle.com/database/enterprise:latest` from [Oracle Container Registry](https://container-registry.oracle.com/ords/f?p=113:10::::::). These images are functionally tested and evaluated with various use cases of Oracle Globally Distributed Database topology by deploying on OKE and OLCNE. + You can also download the pre-built Oracle Global Data Services and Oracle Database images from [Oracle Container Registry](https://container-registry.oracle.com/ords/f?p=113:10::::::). These images are functionally tested and evaluated with various use cases of Oracle Globally Distributed Database topology by deploying on OKE and OLCNE. You can refer to [Oracle Container Registry Images for Oracle Globally Distributed Database Deployment](https://github.com/oracle/db-sharding/blob/master/container-based-sharding-deployment/README.md#oracle-container-registry-images-for-oracle-globally-distributed-database-deployment) **Note:** You will need to accept Agreement from container-registry.orcale.com to be able to pull the pre-built container images. @@ -80,7 +80,7 @@ Choose one of the following deployment options: **Build your own Oracle Database and Global Data Services Docker Images:** You can build these images using instructions provided on Oracle official GitHub Repositories: - * [Oracle Global Data Services Image](https://github.com/oracle/db-sharding/tree/master/docker-based-sharding-deployment/dockerfiles) + * [Oracle Global Data Services Image](https://github.com/oracle/db-sharding/tree/master/container-based-sharding-deployment) * [Oracle Database Image](https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance) After the images are ready, push them to your Docker Images Repository, so that you can pull them during Oracle Globally Distributed Database topology provisioning. @@ -91,8 +91,6 @@ You can either download the images and push them to your Docker Images Repositor **Note:** In case you want to use the `Oracle Database 23ai Free` Image for Database and GSM, refer to section [Oracle Database 23ai Free](#oracle-database-23ai-free) for more details. -### 4. Create a namespace for the Oracle DB Sharding Setup - ### 4. Create a namespace for the Oracle Globally Distributed Database Setup Create a Kubernetes namespace named `shns`. All the resources belonging to the Oracle Globally Distributed Database Topology Setup will be provisioned in this namespace named `shns`. For example: @@ -107,6 +105,8 @@ You can either download the images and push them to your Docker Images Repositor ### 5. Create a Kubernetes secret for the database installation owner for the Oracle Globally Distributed Database Topology Deployment +**IMPORTANT:** Make sure the version of `openssl` in the Oracle Database and Oracle GSM images is compatible with the `openssl` version on the machine where you will run the openssl commands to generate the encrypted password file during the deployment. If you want to use Prebuilt Oracle Database and Oracle GSM Images from Oracle Container Registry for your deployment, you can refer to [Oracle Container Registry Images for Oracle Globally Distributed Database Deployment](https://github.com/oracle/db-sharding/blob/master/container-based-sharding-deployment/README.md#oracle-container-registry-images-for-oracle-globally-distributed-database-deployment) + Create a Kubernetes secret named `db-user-pass-rsa` using these steps: [Create Kubernetes Secret](./provisioning/create_kubernetes_secret_for_db_user.md) After you have the above prerequisites completed, you can proceed to the next section for your environment to provision the Oracle Database Sharding Topology. @@ -119,71 +119,6 @@ In case of an `OCI OKE` cluster, you can use this Persistent Volume during provi You can refer [here](./provisioning/provisioning_persistent_volume_having_db_gold_image.md) for the steps involved. -**NOTE:** Provisioning the Sharded Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. So, this step will not be needed if you are deploying Oracle Sharded Database using Oracle 23ai Free Database and GSM Images. - -## Oracle Database 23ai Free - -Please refer to [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) documentation for more details. - -If you want to use Oracle Database 23ai Free Image for Database and GSM for deployment of the Sharded Database using Sharding Controller in Oracle Database Kubernetes Operator, you need to consider the below points: - -* To deploy using the FREE Database and GSM Image, you will need to add the additional parameter `dbEdition: "free"` to the .yaml file. -* Refer to [Sample Sharded Database Deployment using Oracle 23ai FREE Database and GSM Images](./provisioning/free/sharding_provisioning_with_free_images.md) for an example. -* For Oracle Database 23ai Free, you can control the `CPU` and `Memory` allocation of the PODs using tags `cpu` and `memory` respectively but tags `INIT_SGA_SIZE` and `INIT_PGA_SIZE` to control the SGA and PGA allocation at the database level are `not` supported. -* Provisioning the Sharded Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. -* Total number of chunks for FREE Database defaults to `12` if `CATALOG_CHUNKS` parameter is not specified. This default value is determined considering limitation of 12 GB of user data on disk for oracle free database. - - -## Provisioning Sharding Topology with System-Managed Sharding in a Cloud-Based Kubernetes Cluster - -Deploy Oracle Database Sharding Topology with `System-Managed Sharding` on your Cloud based Kubernetes cluster. - -In this example, the deployment uses the YAML file based on `OCI OKE` cluster. There are multiple use case possible for deploying the Oracle Database Sharding Topology covered by below examples: - -[1. Provisioning Oracle Sharded Database with System-Managed Sharding without Database Gold Image](./provisioning/system_sharding/ssharding_provisioning_without_db_gold_image.md) -[2. Provisioning Oracle Sharded Database with System-Managed Sharding with number of chunks specified](./provisioning/system_sharding/ssharding_provisioning_with_chunks_specified.md) -[3. Provisioning Oracle Sharded Database with System-Managed Sharding with additional control on resources like Memory and CPU allocated to Pods](./provisioning/system_sharding/ssharding_provisioning_with_control_on_resources.md) -[4. Provisioning Oracle Sharded Database with System-Managed Sharding by cloning database from your own Database Gold Image in the same Availability Domain(AD)](./provisioning/system_sharding/ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md) -[5. Provisioning Oracle Sharded Database with System-Managed Sharding by cloning database from your own Database Gold Image across Availability Domains(ADs)](./provisioning/system_sharding/ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md) -[6. Provisioning Oracle Sharded Database with System-Managed Sharding and send Notification using OCI Notification Service](./provisioning/system_sharding/ssharding_provisioning_with_notification_using_oci_notification.md) -[7. Scale Out - Add Shards to an existing Oracle Sharded Database provisioned earlier with System-Managed Sharding](./provisioning/system_sharding/ssharding_scale_out_add_shards.md) -[8. Scale In - Delete an existing Shard from a working Oracle Sharded Database provisioned earlier with System-Managed Sharding](./provisioning/system_sharding/ssharding_scale_in_delete_an_existing_shard.md) - - -## Provisioning Sharding Topology with User Defined Sharding in a Cloud-Based Kubernetes Cluster - -Deploy Oracle Database Sharding Topology with `User Defined Sharding` on your Cloud based Kubernetes cluster. - -In this example, the deployment uses the YAML file based on `OCI OKE` cluster. There are multiple use case possible for deploying the Oracle Database Sharding Topology covered by below examples: - -[1. Provisioning Oracle Sharded Database with User Defined Sharding without Database Gold Image](./provisioning/user-defined-sharding/udsharding_provisioning_without_db_gold_image.md) -[2. Provisioning Oracle Sharded Database with User Defined Sharding with additional control on resources like Memory and CPU allocated to Pods](./provisioning/user-defined-sharding/udsharding_provisioning_with_control_on_resources.md) -[3. Provisioning Oracle Sharded Database with User Defined Sharding by cloning database from your own Database Gold Image in the same Availability Domain(AD)](./provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_gold_image_in_same_ad.md) -[4. Provisioning Oracle Sharded Database with User Defined Sharding by cloning database from your own Database Gold Image across Availability Domains(ADs)](./provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_from_gold_image_across_ads.md) -[5. Provisioning Oracle Sharded Database with User Defined Sharding and send Notification using OCI Notification Service](./provisioning/user-defined-sharding/udsharding_provisioning_with_notification_using_oci_notification.md) -[6. Scale Out - Add Shards to an existing Oracle Sharded Database provisioned earlier with User Defined Sharding](./provisioning/user-defined-sharding/udsharding_scale_out_add_shards.md) -[7. Scale In - Delete an existing Shard from a working Oracle Sharded Database provisioned earlier with User Defined Sharding](./provisioning/user-defined-sharding/udsharding_scale_in_delete_an_existing_shard.md) - - -## Provisioning System-Managed Sharding Topology with Raft replication enabled in a Cloud-Based Kubernetes Cluster - -Deploy Oracle Database Sharding Topology with `System-Managed Sharding with SNR RAFT enabled` on your Cloud based Kubernetes cluster. - -**NOTE: SNR RAFT Feature is available only for Oracle 23ai RDBMS and Oracle 23ai GSM version.** - -In this example, the deployment uses the YAML file based on `OCI OKE` cluster. There are multiple use case possible for deploying the Oracle Database Sharding Topology covered by below examples: - -[1. Provisioning System-Managed Sharding Topology with Raft replication enabled without Database Gold Image](./provisioning/snr_system_sharding/snr_ssharding_provisioning_without_db_gold_image.md) -[2. Provisioning System-Managed Sharding Topology with Raft replication enabled with number of chunks specified](./provisioning/snr_system_sharding/snr_ssharding_provisioning_with_chunks_specified.md) -[3. Provisioning System-Managed Sharding Topology with Raft replication enabled with additional control on resources like Memory and CPU allocated to Pods](./provisioning/snr_system_sharding/snr_ssharding_provisioning_with_control_on_resources.md) -[4. Provisioning System-Managed Sharding Topology with Raft replication enabled by cloning database from your own Database Gold Image in the same Availability Domain(AD)](./provisioning/snr_system_sharding/snr_ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md) -[5. Provisioning System-Managed Sharding Topology with Raft replication enabled by cloning database from your own Database Gold Image across Availability Domains(ADs)](./provisioning/snr_system_sharding/snr_ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md) -[6. Provisioning System-Managed Sharding Topology with Raft replication enabled and send Notification using OCI Notification Service](./provisioning/snr_system_sharding/snr_ssharding_provisioning_with_notification_using_oci_notification.md) -[7. Scale Out - Add Shards to an existing Oracle Sharded Database provisioned earlier with System-Managed Sharding with RAFT replication enabled](./provisioning/snr_system_sharding/snr_ssharding_scale_out_add_shards.md) -[8. Scale In - Delete an existing Shard from a working Oracle Sharded Database provisioned earlier with System-Managed Sharding with RAFT reolication enabled](./provisioning/snr_system_sharding/snr_ssharding_scale_in_delete_an_existing_shard.md) - -You can refer [here](./provisioning/provisioning_persistent_volume_having_db_gold_image.md) for the steps involved. - **NOTE:** Provisioning the Oracle Globally Distributed Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. So, this step will not be needed if you are deploying Oracle Globally Distributed Database using Oracle 23ai Free Database and GSM Images. ## Oracle Database 23ai Free @@ -262,4 +197,5 @@ To debug the Oracle Globally Distributed Database Topology provisioned using the * For both ENTERPRISE and FREE Images, if the CATALOG Database Pod is stopped from the worker node using the command `crictl stopp`, then it can leave the CATALOG in an error state. This error state results in GSM reporting the error message **GSM-45034: Connection to GDS catalog is not established.** * For both ENTERPRISE and FREE Images, either restart of node running the SHARD Pod using `/sbin/reboot -f` or stopping the Shard Database Pod from the worker node using `crictl stopp` command can leave the shard in an error state. * For both ENTERPRISE and FREE Images, after force restarts of the node running GSM Pod, the GSM pod restarts multiple times, and then becomes stable. The GSM pod restarts itself because when the worker node comes up, the GSM pod is recreated, but does not obtain DB connection to the Catalog. The Liveness Probe fails which restarts the Pod. Be aware of this issue, and permit the GSM pod to become stable. -* **DDL Propagation from Catalog to Shards:** DDL Propagation from the Catalog Database to the Shard Databases can take several minutes to complete. To see faster propagation of DDLs such as the tablespace set from the Catalog Database to the Shard Databases, Oracle recommends that you set smaller chunk values by using the `CATALOG_CHUNKS` attribute in the .yaml file while creating the Sharded Database Topology. \ No newline at end of file +* **DDL Propagation from Catalog to Shards:** DDL Propagation from the Catalog Database to the Shard Databases can take several minutes to complete. To see faster propagation of DDLs such as the tablespace set from the Catalog Database to the Shard Databases, Oracle recommends that you set smaller chunk values by using the `CATALOG_CHUNKS` attribute in the .yaml file while creating the Sharded Database Topology. +* If the version of `openssl` used to create the encrypted password file for Kubernetes secrets is not compatible with the openssl verion of the Oracle Database and Oracle GSM Image, then you can get the error `OS command returned code : 1, returned error : bad magic number` in the logs of the Database or GSM Pod. In this case, during the deployment, openssl will not be able to decrypt the encrypted password file and the deployment will not complete. \ No newline at end of file diff --git a/docs/sharding/provisioning/create_kubernetes_secret_for_db_user.md b/docs/sharding/provisioning/create_kubernetes_secret_for_db_user.md index 744f972c..db534575 100644 --- a/docs/sharding/provisioning/create_kubernetes_secret_for_db_user.md +++ b/docs/sharding/provisioning/create_kubernetes_secret_for_db_user.md @@ -8,6 +8,8 @@ Use the following steps to create an encrypted file with a password for the DB U - Remove the initial text file. - Create the Kubernetes Secret named `db-user-pass-rsa` using the encrypted file. +**IMPORTANT:** Make sure the version of `openssl` in the Oracle Database and Oracle GSM images is compatible with the `openssl` version on the machine where you will run the openssl commands to generated the encrypted password file during the deployment. + To understand how to create your own file, use the following example: ```sh diff --git a/docs/sharding/provisioning/free/sharding_provisioning_with_free_images.md b/docs/sharding/provisioning/free/sharding_provisioning_with_free_images.md index 61641312..0425920b 100644 --- a/docs/sharding/provisioning/free/sharding_provisioning_with_free_images.md +++ b/docs/sharding/provisioning/free/sharding_provisioning_with_free_images.md @@ -19,8 +19,11 @@ This example uses `sharding_provisioning_with_free_images.yaml` to provision an To get the Oracle 23ai FREE Database and GSM Images: * The Oracle 23ai FREE RDBMS Image used is `container-registry.oracle.com/database/free:latest`. Check [Oracle Database Free Get Started](https://www.oracle.com/database/free/get-started/?source=v0-DBFree-ChatCTA-j2032-20240709) for details. * To pull the above image from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. - * Use the Oracle 23ai FREE GSM Binaries `LINUX.X64_234000_gsm.zip` as listed on page [Oracle Database Free Get Started](https://www.oracle.com/database/free/get-started/?source=v0-DBFree-ChatCTA-j2032-20240709) and prepare the GSM Container Image following [Oracle Global Data Services Image](https://github.com/oracle/db-sharding/tree/master/docker-based-sharding-deployment/dockerfiles) + * The the Oracle 23ai FREE GSM Image used is `container-registry.oracle.com/database/gsm:latest`. + * To pull the above image from Oracle Container Registry, create a Kubernetes secret named `ocr-reg-cred` using your credentials with type set to `kubernetes.io/dockerconfigjson` in the namespace `shns`. * You need to change `dbImage` and `gsmImage` tag with the images you want to use in your enviornment in file `sharding_provisioning_with_free_images.yaml`. + +**IMPORTANT:** Make sure the version of `openssl` in the Oracle Database and Oracle GSM images is compatible with the `openssl` version on the machine where you will run the openssl commands to generated the encrypted password file during the deployment. diff --git a/docs/sharding/provisioning/free/sharding_provisioning_with_free_images.yaml b/docs/sharding/provisioning/free/sharding_provisioning_with_free_images.yaml index 954ede63..dadd619a 100644 --- a/docs/sharding/provisioning/free/sharding_provisioning_with_free_images.yaml +++ b/docs/sharding/provisioning/free/sharding_provisioning_with_free_images.yaml @@ -41,8 +41,8 @@ spec: storageClass: oci dbImage: container-registry.oracle.com/database/free:latest dbImagePullSecret: ocr-reg-cred - gsmImage: - gsmImagePullSecret: + gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImagePullSecret: ocr-reg-cred dbEdition: "free" isExternalSvc: False isDeleteOraPvc: True diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md index ba72be25..9ffebad9 100644 --- a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md @@ -1,6 +1,6 @@ # Provisioning System managed Sharding Topology with Raft replication enabled by cloning database from your own Database Gold Image across Availability Domains(ADs) -**NOTE: RAFT Replication Feature is available only for Oracle 23c RDBMS and Oracle 23c GSM version.** +**NOTE: RAFT Replication Feature is available only for Oracle 23ai RDBMS and Oracle 23ai GSM version.** **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. @@ -40,6 +40,7 @@ In this example, we are using pre-built Oracle Database and Global Data Services * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `snr_ssharding_shard_prov_clone_across_ads.yaml`. * The `dbImage` used during provisioning the Persistent Volume with Database Gold Image and the `dbImage` used for deploying the Shard or Catalog Database by cloning should be same. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) + * Make sure the version of `openssl` in the Oracle Database and Oracle GSM images is compatible with the `openssl` version on the machine where you will run the openssl commands to generated the encrypted password file during the deployment. Use the file: [snr_ssharding_shard_prov_clone_across_ads.yaml](./snr_ssharding_shard_prov_clone_across_ads.yaml) for this use case as below: diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md index cf4240f7..054d760e 100644 --- a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md @@ -1,6 +1,6 @@ # Provisioning System managed Sharding Topology with Raft replication enabled by cloning database from your own Database Gold Image in the same Availability Domain(AD) -**NOTE: RAFT Replication Feature is available only for Oracle 23c RDBMS and Oracle 23c GSM version.** +**NOTE: RAFT Replication Feature is available only for Oracle 23ai RDBMS and Oracle 23ai GSM version.** **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. @@ -34,6 +34,7 @@ In this example, we are using pre-built Oracle Database and Global Data Services * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `snr_ssharding_shard_prov_clone.yaml`. * The `dbImage` used during provisioning the Persistent Volume with Database Gold Image and the `dbImage` used for deploying the Shard or Catalog Database by cloning should be same. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) + * Make sure the version of `openssl` in the Oracle Database and Oracle GSM images is compatible with the `openssl` version on the machine where you will run the openssl commands to generated the encrypted password file during the deployment. **NOTE:** Provisioning the Sharded Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_chunks_specified.md b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_chunks_specified.md index 44972090..253d099b 100644 --- a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_chunks_specified.md +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_chunks_specified.md @@ -1,6 +1,6 @@ # Provisioning System-Managed Sharding Topology with Raft replication enabled with number of chunks specified -**NOTE: RAFT Replication Feature is available only for Oracle 23c RDBMS and Oracle 23c GSM version.** +**NOTE: RAFT Replication Feature is available only for Oracle 23ai RDBMS and Oracle 23ai GSM version.** **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. @@ -25,6 +25,7 @@ In this example, we are using pre-built Oracle Database and Global Data Services * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `snr_ssharding_shard_prov.yaml`. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) * In case you want to use the [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then you will need to add the additional parameter `dbEdition: "free"` to the below .yaml file. + * Make sure the version of `openssl` in the Oracle Database and Oracle GSM images is compatible with the `openssl` version on the machine where you will run the openssl commands to generated the encrypted password file during the deployment. Use the file: [snr_ssharding_shard_prov_chunks.yaml](./snr_ssharding_shard_prov_chunks.yaml) for this use case as below: diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_control_on_resources.md b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_control_on_resources.md index 9cfd6afb..e017f6a9 100644 --- a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_control_on_resources.md +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_control_on_resources.md @@ -1,6 +1,6 @@ # Provisioning System-Managed Sharding Topology with Raft replication enabled with additional control on resources like Memory and CPU allocated to Pods -**NOTE: RAFT Replication Feature is available only for Oracle 23c RDBMS and Oracle 23c GSM version.** +**NOTE: RAFT Replication Feature is available only for Oracle 23ai RDBMS and Oracle 23ai GSM version.** **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. @@ -21,6 +21,7 @@ In this example, we are using pre-built Oracle Database and Global Data Services * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `snr_ssharding_shard_prov_memory_cpu.yaml`. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) * In case you want to use the [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then you will need to add the additional parameter `dbEdition: "free"` to the below .yaml file. + * Make sure the version of `openssl` in the Oracle Database and Oracle GSM images is compatible with the `openssl` version on the machine where you will run the openssl commands to generated the encrypted password file during the deployment. **NOTE:** For Oracle Database 23ai Free, you can control the `CPU` and `Memory` allocation of the PODs using tags `cpu` and `memory` respectively but tags `INIT_SGA_SIZE` and `INIT_PGA_SIZE` to control the SGA and PGA allocation at the database level are `not` supported. diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_notification_using_oci_notification.md b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_notification_using_oci_notification.md index d4cb11de..3b8d1665 100644 --- a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_notification_using_oci_notification.md +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_with_notification_using_oci_notification.md @@ -1,6 +1,6 @@ # Provisioning System managed Sharding Topology with Raft replication enabled and send Notification using OCI Notification Service -**NOTE: RAFT Replication Feature is available only for Oracle 23c RDBMS and Oracle 23c GSM version.** +**NOTE: RAFT Replication Feature is available only for Oracle 23ai RDBMS and Oracle 23ai GSM version.** **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. @@ -69,6 +69,7 @@ In this example, we are using pre-built Oracle Database and Global Data Services * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `snr_ssharding_shard_prov_send_notification.yaml`. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) * In case you want to use the [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then you will need to add the additional parameter `dbEdition: "free"` to the below .yaml file. + * Make sure the version of `openssl` in the Oracle Database and Oracle GSM images is compatible with the `openssl` version on the machine where you will run the openssl commands to generated the encrypted password file during the deployment. **NOTE:** Provisioning the Sharded Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_without_db_gold_image.md b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_without_db_gold_image.md index 892741a5..91caddf1 100644 --- a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_without_db_gold_image.md +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_provisioning_without_db_gold_image.md @@ -1,6 +1,6 @@ # Provisioning System-Managed Sharding Topology with Raft replication enabled without Database Gold Image -**NOTE: RAFT Replication Feature is available only for Oracle 23c RDBMS and Oracle 23c GSM version.** +**NOTE: RAFT Replication Feature is available only for Oracle 23ai RDBMS and Oracle 23ai GSM version.** **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. @@ -22,6 +22,7 @@ In this example, we are using pre-built Oracle Database and Global Data Services * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `snr_ssharding_shard_prov.yaml`. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) * In case you want to use the [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then you will need to add the additional parameter `dbEdition: "free"` to the below .yaml file. + * Make sure the version of `openssl` in the Oracle Database and Oracle GSM images is compatible with the `openssl` version on the machine where you will run the openssl commands to generated the encrypted password file during the deployment. Use the file: [snr_ssharding_shard_prov.yaml](./snr_ssharding_shard_prov.yaml) for this use case as below: diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_scale_in_delete_an_existing_shard.md b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_scale_in_delete_an_existing_shard.md index fe3157ec..fc093654 100644 --- a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_scale_in_delete_an_existing_shard.md +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_scale_in_delete_an_existing_shard.md @@ -1,6 +1,6 @@ # Scale In - Delete an existing Shard from a working Oracle Sharded Database provisioned earlier with System-Managed Sharding with RAFT reolication enabled -**NOTE: RAFT Replication Feature is available only for Oracle 23c RDBMS and Oracle 23c GSM version.** +**NOTE: RAFT Replication Feature is available only for Oracle 23ai RDBMS and Oracle 23ai GSM version.** **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. @@ -21,6 +21,7 @@ In this example, we are using pre-built Oracle Database and Global Data Services * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `snr_ssharding_shard_prov_delshard.yaml`. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) * In case you want to use the [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then you will need to add the additional parameter `dbEdition: "free"` to the below .yaml file. + * Make sure the version of `openssl` in the Oracle Database and Oracle GSM images is compatible with the `openssl` version on the machine where you will run the openssl commands to generated the encrypted password file during the deployment. NOTE: Use tag `isDelete: enable` to delete the shard you want. diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_scale_out_add_shards.md b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_scale_out_add_shards.md index 03423e72..3461bf13 100644 --- a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_scale_out_add_shards.md +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_scale_out_add_shards.md @@ -1,6 +1,6 @@ # Scale Out - Add Shards to an existing Oracle Sharded Database provisioned earlier with System-Managed Sharding with RAFT replication enabled -**NOTE: RAFT Replication Feature is available only for Oracle 23c RDBMS and Oracle 23c GSM version.** +**NOTE: RAFT Replication Feature is available only for Oracle 23ai RDBMS and Oracle 23ai GSM version.** **IMPORTANT:** Make sure you have completed the steps for [Prerequsites for Running Oracle Sharding Database Controller](../../README.md#prerequsites-for-running-oracle-sharding-database-controller) before using Oracle Sharding Controller. @@ -19,6 +19,7 @@ In this example, we are using pre-built Oracle Database and Global Data Services * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `snr_ssharding_shard_prov_extshard.yaml`. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) * If the existing Sharding Topology was deployed using [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then the additional parameter `dbEdition: "free"` will be needed for the below .yaml file as well. + * Make sure the version of `openssl` in the Oracle Database and Oracle GSM images is compatible with the `openssl` version on the machine where you will run the openssl commands to generated the encrypted password file during the deployment. This use case adds two new shards `shard4`,`shard5` to above Sharding Topology. diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov.yaml b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov.yaml index aabd8470..53b93a0d 100644 --- a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov.yaml +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov.yaml @@ -39,10 +39,11 @@ spec: storageSizeInGb: 50 region: standby storageClass: oci - dbImage: container-registry.oracle.com/database/enterprise:latest + dbImage: container-registry.oracle.com/database/free:latest dbImagePullSecret: ocr-reg-cred gsmImage: container-registry.oracle.com/database/gsm:latest gsmImagePullSecret: ocr-reg-cred + dbEdition: "free" replicationType: "native" isExternalSvc: False isDeleteOraPvc: True diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_chunks.yaml b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_chunks.yaml index def7e73a..0230eac2 100644 --- a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_chunks.yaml +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_chunks.yaml @@ -42,10 +42,11 @@ spec: storageSizeInGb: 50 region: standby storageClass: oci - dbImage: container-registry.oracle.com/database/enterprise:latest + dbImage: container-registry.oracle.com/database/free:latest dbImagePullSecret: ocr-reg-cred gsmImage: container-registry.oracle.com/database/gsm:latest gsmImagePullSecret: ocr-reg-cred + dbEdition: "free" replicationType: "native" isExternalSvc: False isDeleteOraPvc: True diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_clone.yaml b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_clone.yaml index 8f17331e..fcc18da0 100644 --- a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_clone.yaml +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_clone.yaml @@ -63,10 +63,11 @@ spec: storageSizeInGb: 50 region: standby storageClass: oci - dbImage: container-registry.oracle.com/database/enterprise:latest + dbImage: container-registry.oracle.com/database/free:latest dbImagePullSecret: ocr-reg-cred gsmImage: container-registry.oracle.com/database/gsm:latest gsmImagePullSecret: ocr-reg-cred + dbEdition: "free" replicationType: "native" isExternalSvc: False isDeleteOraPvc: True diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_clone_across_ads.yaml b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_clone_across_ads.yaml index d0c1c6e0..0663f8a5 100644 --- a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_clone_across_ads.yaml +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_clone_across_ads.yaml @@ -71,10 +71,11 @@ spec: pvMatchLabels: "failure-domain.beta.kubernetes.io/zone": "PHX-AD-3" storageClass: oci - dbImage: container-registry.oracle.com/database/enterprise:latest + dbImage: container-registry.oracle.com/database/free:latest dbImagePullSecret: ocr-reg-cred gsmImage: container-registry.oracle.com/database/gsm:latest gsmImagePullSecret: ocr-reg-cred + dbEdition: "free" replicationType: "native" isExternalSvc: False isDeleteOraPvc: True diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_delshard.yaml b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_delshard.yaml index 0859b089..ce194246 100644 --- a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_delshard.yaml +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_delshard.yaml @@ -50,10 +50,11 @@ spec: storageSizeInGb: 50 region: standby storageClass: oci - dbImage: container-registry.oracle.com/database/enterprise:latest + dbImage: container-registry.oracle.com/database/free:latest dbImagePullSecret: ocr-reg-cred gsmImage: container-registry.oracle.com/database/gsm:latest gsmImagePullSecret: ocr-reg-cred + dbEdition: "free" replicationType: "native" isExternalSvc: False isDeleteOraPvc: True diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_extshard.yaml b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_extshard.yaml index 123b3ae1..8848b8c7 100644 --- a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_extshard.yaml +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_extshard.yaml @@ -49,10 +49,11 @@ spec: storageSizeInGb: 50 region: standby storageClass: oci - dbImage: container-registry.oracle.com/database/enterprise:latest + dbImage: container-registry.oracle.com/database/free:latest dbImagePullSecret: ocr-reg-cred gsmImage: container-registry.oracle.com/database/gsm:latest gsmImagePullSecret: ocr-reg-cred + dbEdition: "free" replicationType: "native" isExternalSvc: False isDeleteOraPvc: True diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_memory_cpu.yaml b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_memory_cpu.yaml index 0cfccf9a..dce4ba29 100644 --- a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_memory_cpu.yaml +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_memory_cpu.yaml @@ -70,10 +70,11 @@ spec: storageSizeInGb: 50 region: standby storageClass: oci - dbImage: container-registry.oracle.com/database/enterprise:latest + dbImage: container-registry.oracle.com/database/free:latest dbImagePullSecret: ocr-reg-cred gsmImage: container-registry.oracle.com/database/gsm:latest gsmImagePullSecret: ocr-reg-cred + dbEdition: "free" replicationType: "native" isExternalSvc: False isDeleteOraPvc: True diff --git a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_send_notification.yaml b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_send_notification.yaml index 345e9c09..2b410e8b 100644 --- a/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_send_notification.yaml +++ b/docs/sharding/provisioning/snr_system_sharding/snr_ssharding_shard_prov_send_notification.yaml @@ -63,10 +63,11 @@ spec: storageSizeInGb: 50 region: standby storageClass: oci - dbImage: container-registry.oracle.com/database/enterprise:latest + dbImage: container-registry.oracle.com/database/free:latest dbImagePullSecret: ocr-reg-cred gsmImage: container-registry.oracle.com/database/gsm:latest gsmImagePullSecret: ocr-reg-cred + dbEdition: "free" replicationType: "native" isExternalSvc: False isDeleteOraPvc: True diff --git a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md index e457b7eb..4d24655d 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md +++ b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_by_cloning_db_from_gold_image_across_ads.md @@ -38,6 +38,7 @@ In this example, we are using pre-built Oracle Database and Global Data Services * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `ssharding_shard_prov_clone_across_ads.yaml`. * The `dbImage` used during provisioning the Persistent Volume with Database Gold Image and the `dbImage` used for deploying the Shard or Catalog Database by cloning should be same. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) + * Make sure the version of `openssl` in the Oracle Database and Oracle GSM images is compatible with the `openssl` version on the machine where you will run the openssl commands to generated the encrypted password file during the deployment. Use the file: [ssharding_shard_prov_clone_across_ads.yaml](./ssharding_shard_prov_clone_across_ads.yaml) for this use case as below: diff --git a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md index cb01fa0d..5e44a601 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md +++ b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_by_cloning_db_gold_image_in_same_ad.md @@ -33,6 +33,7 @@ In this example, we are using pre-built Oracle Database and Global Data Services * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `ssharding_shard_prov_clone.yaml`. * The `dbImage` used during provisioning the Persistent Volume with Database Gold Image and the `dbImage` used for deploying the Shard or Catalog Database by cloning should be same. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) + * Make sure the version of `openssl` in the Oracle Database and Oracle GSM images is compatible with the `openssl` version on the machine where you will run the openssl commands to generated the encrypted password file during the deployment. Use the file: [ssharding_shard_prov_clone.yaml](./ssharding_shard_prov_clone.yaml) for this use case as below: diff --git a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_chunks_specified.md b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_chunks_specified.md index 0c6ea8fe..649fc7c4 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_chunks_specified.md +++ b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_chunks_specified.md @@ -22,6 +22,7 @@ In this example, we are using pre-built Oracle Database and Global Data Services * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `ssharding_shard_prov.yaml`. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) * In case you want to use the [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then you will need to add the additional parameter `dbEdition: "free"` to the below .yaml file. + * Make sure the version of `openssl` in the Oracle Database and Oracle GSM images is compatible with the `openssl` version on the machine where you will run the openssl commands to generated the encrypted password file during the deployment. Use the file: [ssharding_shard_prov_chunks.yaml](./ssharding_shard_prov_chunks.yaml) for this use case as below: diff --git a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_control_on_resources.md b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_control_on_resources.md index c4f45a48..d284bf9b 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_control_on_resources.md +++ b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_control_on_resources.md @@ -18,6 +18,7 @@ In this example, we are using pre-built Oracle Database and Global Data Services * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `ssharding_shard_prov_memory_cpu.yaml`. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) * In case you want to use the [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then you will need to add the additional parameter `dbEdition: "free"` to the below .yaml file. + * Make sure the version of `openssl` in the Oracle Database and Oracle GSM images is compatible with the `openssl` version on the machine where you will run the openssl commands to generated the encrypted password file during the deployment. **NOTE:** For Oracle Database 23ai Free, you can control the `CPU` and `Memory` allocation of the PODs using tags `cpu` and `memory` respectively but tags `INIT_SGA_SIZE` and `INIT_PGA_SIZE` to control the SGA and PGA allocation at the database level are `not` supported. diff --git a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_notification_using_oci_notification.md b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_notification_using_oci_notification.md index 1a6a1ee3..e77718f4 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_notification_using_oci_notification.md +++ b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_with_notification_using_oci_notification.md @@ -66,6 +66,7 @@ In this example, we are using pre-built Oracle Database and Global Data Services * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `ssharding_shard_prov_send_notification.yaml`. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) * In case you want to use the [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then you will need to add the additional parameter `dbEdition: "free"` to the below .yaml file. + * Make sure the version of `openssl` in the Oracle Database and Oracle GSM images is compatible with the `openssl` version on the machine where you will run the openssl commands to generated the encrypted password file during the deployment. **NOTE:** Provisioning the Sharded Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. diff --git a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_without_db_gold_image.md b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_without_db_gold_image.md index b223d1af..1ecb0ec1 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_provisioning_without_db_gold_image.md +++ b/docs/sharding/provisioning/system_sharding/ssharding_provisioning_without_db_gold_image.md @@ -19,7 +19,7 @@ In this example, we are using pre-built Oracle Database and Global Data Services * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `ssharding_shard_prov.yaml`. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) * In case you want to use the [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then you will need to add the additional parameter `dbEdition: "free"` to the below .yaml file. - + * Make sure the version of `openssl` in the Oracle Database and Oracle GSM images is compatible with the `openssl` version on the machine where you will run the openssl commands to generated the encrypted password file during the deployment. Use the file: [ssharding_shard_prov.yaml](./ssharding_shard_prov.yaml) for this use case as below: diff --git a/docs/sharding/provisioning/system_sharding/ssharding_scale_in_delete_an_existing_shard.md b/docs/sharding/provisioning/system_sharding/ssharding_scale_in_delete_an_existing_shard.md index bca34253..889de98c 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_scale_in_delete_an_existing_shard.md +++ b/docs/sharding/provisioning/system_sharding/ssharding_scale_in_delete_an_existing_shard.md @@ -18,6 +18,7 @@ In this example, we are using pre-built Oracle Database and Global Data Services * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `ssharding_shard_prov_delshard.yaml`. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) * In case you want to use the [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then you will need to add the additional parameter `dbEdition: "free"` to the below .yaml file. + * Make sure the version of `openssl` in the Oracle Database and Oracle GSM images is compatible with the `openssl` version on the machine where you will run the openssl commands to generated the encrypted password file during the deployment. NOTE: Use tag `isDelete: enable` to delete the shard you want. diff --git a/docs/sharding/provisioning/system_sharding/ssharding_scale_out_add_shards.md b/docs/sharding/provisioning/system_sharding/ssharding_scale_out_add_shards.md index 1db8e6c3..5086d887 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_scale_out_add_shards.md +++ b/docs/sharding/provisioning/system_sharding/ssharding_scale_out_add_shards.md @@ -16,6 +16,7 @@ In this example, we are using pre-built Oracle Database and Global Data Services * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `ssharding_shard_prov_extshard.yaml`. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) * If the existing Sharding Topology was deployed using [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then the additional parameter `dbEdition: "free"` will be needed for the below .yaml file as well. + * Make sure the version of `openssl` in the Oracle Database and Oracle GSM images is compatible with the `openssl` version on the machine where you will run the openssl commands to generated the encrypted password file during the deployment. This use case adds two new shards `shard4`,`shard5` to above Sharding Topology. diff --git a/docs/sharding/provisioning/system_sharding/ssharding_shard_prov.yaml b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov.yaml index 5adbd2ce..1bdb9ce5 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_shard_prov.yaml +++ b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov.yaml @@ -39,9 +39,9 @@ spec: storageSizeInGb: 50 region: standby storageClass: oci - dbImage: container-registry.oracle.com/database/enterprise:latest + dbImage: container-registry.oracle.com/database/enterprise_ru:19.25.0.0 dbImagePullSecret: ocr-reg-cred - gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImage: container-registry.oracle.com/database/gsm_ru:19.25.0.0 gsmImagePullSecret: ocr-reg-cred isExternalSvc: False isDeleteOraPvc: True diff --git a/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_chunks.yaml b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_chunks.yaml index 5c135229..868e8bc1 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_chunks.yaml +++ b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_chunks.yaml @@ -42,9 +42,9 @@ spec: storageSizeInGb: 50 region: standby storageClass: oci - dbImage: container-registry.oracle.com/database/enterprise:latest + dbImage: container-registry.oracle.com/database/enterprise_ru:19.25.0.0 dbImagePullSecret: ocr-reg-cred - gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImage: container-registry.oracle.com/database/gsm_ru:19.25.0.0 gsmImagePullSecret: ocr-reg-cred isExternalSvc: False isDeleteOraPvc: True diff --git a/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_clone.yaml b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_clone.yaml index f5816a87..3cafeba7 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_clone.yaml +++ b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_clone.yaml @@ -63,9 +63,9 @@ spec: storageSizeInGb: 50 region: standby storageClass: oci - dbImage: container-registry.oracle.com/database/enterprise:latest + dbImage: container-registry.oracle.com/database/enterprise_ru:19.25.0.0 dbImagePullSecret: ocr-reg-cred - gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImage: container-registry.oracle.com/database/gsm_ru:19.25.0.0 gsmImagePullSecret: ocr-reg-cred isExternalSvc: False isDeleteOraPvc: True diff --git a/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_clone_across_ads.yaml b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_clone_across_ads.yaml index 8fee0526..d7ec6365 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_clone_across_ads.yaml +++ b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_clone_across_ads.yaml @@ -71,9 +71,9 @@ spec: pvMatchLabels: "failure-domain.beta.kubernetes.io/zone": "PHX-AD-3" storageClass: oci - dbImage: container-registry.oracle.com/database/enterprise:latest + dbImage: container-registry.oracle.com/database/enterprise_ru:19.25.0.0 dbImagePullSecret: ocr-reg-cred - gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImage: container-registry.oracle.com/database/gsm_ru:19.25.0.0 gsmImagePullSecret: ocr-reg-cred isExternalSvc: False isDeleteOraPvc: True diff --git a/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_delshard.yaml b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_delshard.yaml index 3902ceef..1017a9d5 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_delshard.yaml +++ b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_delshard.yaml @@ -50,9 +50,9 @@ spec: storageSizeInGb: 50 region: standby storageClass: oci - dbImage: container-registry.oracle.com/database/enterprise:latest + dbImage: container-registry.oracle.com/database/enterprise_ru:19.25.0.0 dbImagePullSecret: ocr-reg-cred - gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImage: container-registry.oracle.com/database/gsm_ru:19.25.0.0 gsmImagePullSecret: ocr-reg-cred isExternalSvc: False isDeleteOraPvc: True diff --git a/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_extshard.yaml b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_extshard.yaml index a11833e0..d23052fb 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_extshard.yaml +++ b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_extshard.yaml @@ -49,9 +49,9 @@ spec: storageSizeInGb: 50 region: standby storageClass: oci - dbImage: container-registry.oracle.com/database/enterprise:latest + dbImage: container-registry.oracle.com/database/enterprise_ru:19.25.0.0 dbImagePullSecret: ocr-reg-cred - gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImage: container-registry.oracle.com/database/gsm_ru:19.25.0.0 gsmImagePullSecret: ocr-reg-cred isExternalSvc: False isDeleteOraPvc: True diff --git a/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_memory_cpu.yaml b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_memory_cpu.yaml index 3f092b89..075919f7 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_memory_cpu.yaml +++ b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_memory_cpu.yaml @@ -70,9 +70,9 @@ spec: storageSizeInGb: 50 region: standby storageClass: oci - dbImage: container-registry.oracle.com/database/enterprise:latest + dbImage: container-registry.oracle.com/database/enterprise_ru:19.25.0.0 dbImagePullSecret: ocr-reg-cred - gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImage: container-registry.oracle.com/database/gsm_ru:19.25.0.0 gsmImagePullSecret: ocr-reg-cred isExternalSvc: False isDeleteOraPvc: True diff --git a/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_send_notification.yaml b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_send_notification.yaml index 0ca6ec6f..aea6fc7c 100644 --- a/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_send_notification.yaml +++ b/docs/sharding/provisioning/system_sharding/ssharding_shard_prov_send_notification.yaml @@ -63,9 +63,9 @@ spec: storageSizeInGb: 50 region: standby storageClass: oci - dbImage: container-registry.oracle.com/database/enterprise:latest + dbImage: container-registry.oracle.com/database/enterprise_ru:19.25.0.0 dbImagePullSecret: ocr-reg-cred - gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImage: container-registry.oracle.com/database/gsm_ru:19.25.0.0 gsmImagePullSecret: ocr-reg-cred isExternalSvc: False isDeleteOraPvc: True diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_from_gold_image_across_ads.md b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_from_gold_image_across_ads.md index 9b2905e8..e55df2de 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_from_gold_image_across_ads.md +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_from_gold_image_across_ads.md @@ -36,6 +36,7 @@ In this example, we are using pre-built Oracle Database and Global Data Services * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `udsharding_shard_prov_clone_across_ads.yaml`. * The `dbImage` used during provisioning the Persistent Volume with Database Gold Image and the `dbImage` used for deploying the Shard or Catalog Database by cloning should be same. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) + * Make sure the version of `openssl` in the Oracle Database and Oracle GSM images is compatible with the `openssl` version on the machine where you will run the openssl commands to generated the encrypted password file during the deployment. **NOTE:** Provisioning the Sharded Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_gold_image_in_same_ad.md b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_gold_image_in_same_ad.md index a4669667..edd9c484 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_gold_image_in_same_ad.md +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_by_cloning_db_gold_image_in_same_ad.md @@ -32,6 +32,7 @@ In this example, we are using pre-built Oracle Database and Global Data Services * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `ssharding_shard_prov_clone.yaml`. * The `dbImage` used during provisioning the Persistent Volume with Database Gold Image and the `dbImage` used for deploying the Shard or Catalog Database by cloning should be same. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) + * Make sure the version of `openssl` in the Oracle Database and Oracle GSM images is compatible with the `openssl` version on the machine where you will run the openssl commands to generated the encrypted password file during the deployment. **NOTE:** Provisioning the Sharded Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_with_control_on_resources.md b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_with_control_on_resources.md index b52b8745..638b7124 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_with_control_on_resources.md +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_with_control_on_resources.md @@ -19,6 +19,7 @@ In this example, we are using pre-built Oracle Database and Global Data Services * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `udsharding_shard_prov_memory_cpu.yaml`. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) * In case you want to use the [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then you will need to add the additional parameter `dbEdition: "free"` to the below .yaml file. + * Make sure the version of `openssl` in the Oracle Database and Oracle GSM images is compatible with the `openssl` version on the machine where you will run the openssl commands to generated the encrypted password file during the deployment. **NOTE:** For Oracle Database 23ai Free, you can control the `CPU` and `Memory` allocation of the PODs using tags `cpu` and `memory` respectively but tags `INIT_SGA_SIZE` and `INIT_PGA_SIZE` to control the SGA and PGA allocation at the database level are `not` supported. diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_with_notification_using_oci_notification.md b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_with_notification_using_oci_notification.md index 640301a2..fe1ca870 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_with_notification_using_oci_notification.md +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_with_notification_using_oci_notification.md @@ -67,6 +67,7 @@ In this example, we are using pre-built Oracle Database and Global Data Services * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `udsharding_shard_prov_send_notification.yaml`. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) * In case you want to use the [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then you will need to add the additional parameter `dbEdition: "free"` to the below .yaml file. + * Make sure the version of `openssl` in the Oracle Database and Oracle GSM images is compatible with the `openssl` version on the machine where you will run the openssl commands to generated the encrypted password file during the deployment. **NOTE:** Provisioning the Sharded Database using Cloning from Database Gold Image is `NOT` supported with Oracle Database 23ai Free. diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_without_db_gold_image.md b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_without_db_gold_image.md index 2be5ac9f..b0378e04 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_without_db_gold_image.md +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_provisioning_without_db_gold_image.md @@ -20,6 +20,7 @@ In this example, we are using pre-built Oracle Database and Global Data Services * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `udsharding_shard_prov.yaml`. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) * In case you want to use the [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then you will need to add the additional parameter `dbEdition: "free"` to the below .yaml file. + * Make sure the version of `openssl` in the Oracle Database and Oracle GSM images is compatible with the `openssl` version on the machine where you will run the openssl commands to generated the encrypted password file during the deployment. Use the file: [udsharding_shard_prov.yaml](./udsharding_shard_prov.yaml) for this use case as below: diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_scale_in_delete_an_existing_shard.md b/docs/sharding/provisioning/user-defined-sharding/udsharding_scale_in_delete_an_existing_shard.md index 2c4cbfc2..673e455e 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_scale_in_delete_an_existing_shard.md +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_scale_in_delete_an_existing_shard.md @@ -17,6 +17,7 @@ In this example, we are using pre-built Oracle Database and Global Data Services * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `udsharding_shard_prov_delshard.yaml`. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) * In case you want to use the [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then you will need to add the additional parameter `dbEdition: "free"` to the below .yaml file. + * Make sure the version of `openssl` in the Oracle Database and Oracle GSM images is compatible with the `openssl` version on the machine where you will run the openssl commands to generated the encrypted password file during the deployment. **NOTE:** Use tag `isDelete: enable` to delete the shard you want. diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_scale_out_add_shards.md b/docs/sharding/provisioning/user-defined-sharding/udsharding_scale_out_add_shards.md index 20f50b29..abdc53ff 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_scale_out_add_shards.md +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_scale_out_add_shards.md @@ -17,6 +17,7 @@ In this example, we are using pre-built Oracle Database and Global Data Services * If you plan to use images built by you, you need to change `dbImage` and `gsmImage` tag with the images you have built in your enviornment in file `udsharding_shard_prov_extshard.yaml`. * To understand the Pre-requisite of Database and Global Data Services docker images, refer [Oracle Database and Global Data Services Docker Images](../../README.md#3-oracle-database-and-global-data-services-docker-images) * If the existing Sharding Topology was deployed using [Oracle Database 23ai Free](https://www.oracle.com/database/free/get-started/) Image for Database and GSM, then the additional parameter `dbEdition: "free"` will be needed for the below .yaml file as well. + * Make sure the version of `openssl` in the Oracle Database and Oracle GSM images is compatible with the `openssl` version on the machine where you will run the openssl commands to generated the encrypted password file during the deployment. This use case adds two new shards `shard4`,`shard5` to above Sharding Topology. diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov.yaml b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov.yaml index d33be599..c9f20eb3 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov.yaml +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov.yaml @@ -39,9 +39,9 @@ spec: storageSizeInGb: 50 region: standby storageClass: oci - dbImage: container-registry.oracle.com/database/enterprise:latest + dbImage: container-registry.oracle.com/database/enterprise_ru:19.25.0.0 dbImagePullSecret: ocr-reg-cred - gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImage: container-registry.oracle.com/database/gsm_ru:19.25.0.0 gsmImagePullSecret: ocr-reg-cred shardingType: USER isExternalSvc: False diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_clone.yaml b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_clone.yaml index 04ee5d95..d7e5ce78 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_clone.yaml +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_clone.yaml @@ -63,9 +63,9 @@ spec: storageSizeInGb: 50 region: standby storageClass: oci - dbImage: container-registry.oracle.com/database/enterprise:latest + dbImage: container-registry.oracle.com/database/enterprise_ru:19.25.0.0 dbImagePullSecret: ocr-reg-cred - gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImage: container-registry.oracle.com/database/gsm_ru:19.25.0.0 gsmImagePullSecret: ocr-reg-cred shardingType: USER isExternalSvc: False diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_clone_across_ads.yaml b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_clone_across_ads.yaml index 5be6ecde..ae02c7fe 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_clone_across_ads.yaml +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_clone_across_ads.yaml @@ -71,9 +71,9 @@ spec: pvMatchLabels: "failure-domain.beta.kubernetes.io/zone": "PHX-AD-3" storageClass: oci - dbImage: container-registry.oracle.com/database/enterprise:latest + dbImage: container-registry.oracle.com/database/enterprise_ru:19.25.0.0 dbImagePullSecret: ocr-reg-cred - gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImage: container-registry.oracle.com/database/gsm_ru:19.25.0.0 gsmImagePullSecret: ocr-reg-cred shardingType: USER isExternalSvc: False diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_delshard.yaml b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_delshard.yaml index e00d2272..d83bf546 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_delshard.yaml +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_delshard.yaml @@ -50,9 +50,9 @@ spec: storageSizeInGb: 50 region: standby storageClass: oci - dbImage: container-registry.oracle.com/database/enterprise:latest + dbImage: container-registry.oracle.com/database/enterprise_ru:19.25.0.0 dbImagePullSecret: ocr-reg-cred - gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImage: container-registry.oracle.com/database/gsm_ru:19.25.0.0 gsmImagePullSecret: ocr-reg-cred shardingType: USER isExternalSvc: False diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_extshard.yaml b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_extshard.yaml index 3899f2ab..7526feb7 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_extshard.yaml +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_extshard.yaml @@ -48,9 +48,9 @@ spec: storageSizeInGb: 50 region: standby storageClass: oci - dbImage: container-registry.oracle.com/database/enterprise:latest + dbImage: container-registry.oracle.com/database/enterprise_ru:19.25.0.0 dbImagePullSecret: ocr-reg-cred - gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImage: container-registry.oracle.com/database/gsm_ru:19.25.0.0 gsmImagePullSecret: ocr-reg-cred shardingType: USER isExternalSvc: False diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_memory_cpu.yaml b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_memory_cpu.yaml index 6c65916e..8be81d39 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_memory_cpu.yaml +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_memory_cpu.yaml @@ -70,9 +70,9 @@ spec: storageSizeInGb: 50 region: standby storageClass: oci - dbImage: container-registry.oracle.com/database/enterprise:latest + dbImage: container-registry.oracle.com/database/enterprise_ru:19.25.0.0 dbImagePullSecret: ocr-reg-cred - gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImage: container-registry.oracle.com/database/gsm_ru:19.25.0.0 gsmImagePullSecret: ocr-reg-cred shardingType: USER isExternalSvc: False diff --git a/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_send_notification.yaml b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_send_notification.yaml index ef1b5561..4dda6db9 100644 --- a/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_send_notification.yaml +++ b/docs/sharding/provisioning/user-defined-sharding/udsharding_shard_prov_send_notification.yaml @@ -63,9 +63,9 @@ spec: storageSizeInGb: 50 region: standby storageClass: oci - dbImage: container-registry.oracle.com/database/enterprise:latest + dbImage: container-registry.oracle.com/database/enterprise_ru:19.25.0.0 dbImagePullSecret: ocr-reg-cred - gsmImage: container-registry.oracle.com/database/gsm:latest + gsmImage: container-registry.oracle.com/database/gsm_ru:19.25.0.0 gsmImagePullSecret: ocr-reg-cred shardingType: USER isExternalSvc: False From 559238b6a55bbc90bfc34203d6464251153f01a9 Mon Sep 17 00:00:00 2001 From: Saurabh Ahuja Date: Mon, 7 Apr 2025 23:07:10 +0530 Subject: [PATCH 624/628] Upgrade support from v1alpha1 to v4 APIs (#173) * compiled misssed files to support v1alpha1 to v4 upgrade * Update oracle-database-operator.yaml * Update kustomization.yaml --- apis/database/v1alpha1/cdb_webhook.go | 224 +++++++ apis/database/v1alpha1/pdb_webhook.go | 369 +++++++++++ .../v1alpha1/zz_generated.deepcopy.go | 602 +++++++++++++++++ .../crd/bases/database.oracle.com_cdbs.yaml | 238 +++++++ .../database.oracle.com_dataguardbrokers.yaml | 4 +- .../crd/bases/database.oracle.com_pdbs.yaml | 315 ++++++++- ...database.oracle.com_shardingdatabases.yaml | 9 - config/manager/kustomization.yaml | 4 +- config/webhook/manifests.yaml | 65 +- oracle-database-operator.yaml | 621 ++++++++++++++++-- 10 files changed, 2356 insertions(+), 95 deletions(-) create mode 100644 apis/database/v1alpha1/cdb_webhook.go create mode 100644 apis/database/v1alpha1/pdb_webhook.go diff --git a/apis/database/v1alpha1/cdb_webhook.go b/apis/database/v1alpha1/cdb_webhook.go new file mode 100644 index 00000000..e93e216e --- /dev/null +++ b/apis/database/v1alpha1/cdb_webhook.go @@ -0,0 +1,224 @@ +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +package v1alpha1 + +import ( + "reflect" + "strings" + + apierrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/runtime/schema" + "k8s.io/apimachinery/pkg/util/validation/field" + ctrl "sigs.k8s.io/controller-runtime" + logf "sigs.k8s.io/controller-runtime/pkg/log" + "sigs.k8s.io/controller-runtime/pkg/webhook" + "sigs.k8s.io/controller-runtime/pkg/webhook/admission" +) + +// log is for logging in this package. +var cdblog = logf.Log.WithName("cdb-webhook") + +func (r *CDB) SetupWebhookWithManager(mgr ctrl.Manager) error { + return ctrl.NewWebhookManagedBy(mgr). + For(r). + Complete() +} + +//+kubebuilder:webhook:path=/mutate-database-oracle-com-v4-cdb,mutating=true,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=cdbs,verbs=create;update,versions=v4,name=mcdb.kb.io,admissionReviewVersions={v1,v1beta1} + +var _ webhook.Defaulter = &CDB{} + +// Default implements webhook.Defaulter so a webhook will be registered for the type +func (r *CDB) Default() { + cdblog.Info("Setting default values in CDB spec for : " + r.Name) + + if r.Spec.ORDSPort == 0 { + r.Spec.ORDSPort = 8888 + } + + if r.Spec.Replicas == 0 { + r.Spec.Replicas = 1 + } +} + +// TODO(user): change verbs to "verbs=create;update;delete" if you want to enable deletion validation. +//+kubebuilder:webhook:path=/validate-database-oracle-com-v4-cdb,mutating=false,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=cdbs,verbs=create;update,versions=v4,name=vcdb.kb.io,admissionReviewVersions={v1,v1beta1} + +var _ webhook.Validator = &CDB{} + +// ValidateCreate implements webhook.Validator so a webhook will be registered for the type +func (r *CDB) ValidateCreate() (admission.Warnings, error) { + cdblog.Info("ValidateCreate", "name", r.Name) + + var allErrs field.ErrorList + + if r.Spec.ServiceName == "" && r.Spec.DBServer != "" { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("serviceName"), "Please specify CDB Service name")) + } + + if reflect.ValueOf(r.Spec.CDBTlsKey).IsZero() { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("cdbTlsKey"), "Please specify CDB Tls key(secret)")) + } + + if reflect.ValueOf(r.Spec.CDBTlsCrt).IsZero() { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("cdbTlsCrt"), "Please specify CDB Tls Certificate(secret)")) + } + + if reflect.ValueOf(r.Spec.CDBPriKey).IsZero() { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("CDBPriKey"), "Please specify CDB CDBPriKey(secret)")) + } + + /*if r.Spec.SCANName == "" { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("scanName"), "Please specify SCAN Name for CDB")) + }*/ + + if (r.Spec.DBServer == "" && r.Spec.DBTnsurl == "") || (r.Spec.DBServer != "" && r.Spec.DBTnsurl != "") { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("dbServer"), "Please specify Database Server Name/IP Address or tnsalias string")) + } + + if r.Spec.DBTnsurl != "" && (r.Spec.DBServer != "" || r.Spec.DBPort != 0 || r.Spec.ServiceName != "") { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("dbServer"), "DBtnsurl is orthogonal to (DBServer,DBport,Services)")) + } + + if r.Spec.DBPort == 0 && r.Spec.DBServer != "" { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("dbPort"), "Please specify DB Server Port")) + } + if r.Spec.DBPort < 0 && r.Spec.DBServer != "" { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("dbPort"), "Please specify a valid DB Server Port")) + } + if r.Spec.ORDSPort < 0 { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("ordsPort"), "Please specify a valid ORDS Port")) + } + if r.Spec.Replicas < 0 { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("replicas"), "Please specify a valid value for Replicas")) + } + if r.Spec.ORDSImage == "" { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("ordsImage"), "Please specify name of ORDS Image to be used")) + } + if reflect.ValueOf(r.Spec.CDBAdminUser).IsZero() { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("cdbAdminUser"), "Please specify user in the root container with sysdba priviledges to manage PDB lifecycle")) + } + if reflect.ValueOf(r.Spec.CDBAdminPwd).IsZero() { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("cdbAdminPwd"), "Please specify password for the CDB Administrator to manage PDB lifecycle")) + } + if reflect.ValueOf(r.Spec.ORDSPwd).IsZero() { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("ordsPwd"), "Please specify password for user ORDS_PUBLIC_USER")) + } + if reflect.ValueOf(r.Spec.WebServerUser).IsZero() { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("webServerUser"), "Please specify the Web Server User having SQL Administrator role")) + } + if reflect.ValueOf(r.Spec.WebServerPwd).IsZero() { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("webServerPwd"), "Please specify password for the Web Server User having SQL Administrator role")) + } + if len(allErrs) == 0 { + return nil, nil + } + return nil, apierrors.NewInvalid( + schema.GroupKind{Group: "database.oracle.com", Kind: "CDB"}, + r.Name, allErrs) +} + +// ValidateUpdate implements webhook.Validator so a webhook will be registered for the type +func (r *CDB) ValidateUpdate(old runtime.Object) (admission.Warnings, error) { + cdblog.Info("validate update", "name", r.Name) + + isCDBMarkedToBeDeleted := r.GetDeletionTimestamp() != nil + if isCDBMarkedToBeDeleted { + return nil, nil + } + + var allErrs field.ErrorList + + // Check for updation errors + oldCDB, ok := old.(*CDB) + if !ok { + return nil, nil + } + + if r.Spec.DBPort < 0 { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("dbPort"), "Please specify a valid DB Server Port")) + } + if r.Spec.ORDSPort < 0 { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("ordsPort"), "Please specify a valid ORDS Port")) + } + if r.Spec.Replicas < 0 { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("replicas"), "Please specify a valid value for Replicas")) + } + if !strings.EqualFold(oldCDB.Spec.ServiceName, r.Spec.ServiceName) { + allErrs = append(allErrs, + field.Forbidden(field.NewPath("spec").Child("replicas"), "cannot be changed")) + } + + if len(allErrs) == 0 { + return nil, nil + } + + return nil, apierrors.NewInvalid( + schema.GroupKind{Group: "database.oracle.com", Kind: "CDB"}, + r.Name, allErrs) +} + +// ValidateDelete implements webhook.Validator so a webhook will be registered for the type +func (r *CDB) ValidateDelete() (admission.Warnings, error) { + cdblog.Info("validate delete", "name", r.Name) + + // TODO(user): fill in your validation logic upon object deletion. + return nil, nil +} diff --git a/apis/database/v1alpha1/pdb_webhook.go b/apis/database/v1alpha1/pdb_webhook.go new file mode 100644 index 00000000..1f115c9b --- /dev/null +++ b/apis/database/v1alpha1/pdb_webhook.go @@ -0,0 +1,369 @@ +/* +** Copyright (c) 2022 Oracle and/or its affiliates. +** +** The Universal Permissive License (UPL), Version 1.0 +** +** Subject to the condition set forth below, permission is hereby granted to any +** person obtaining a copy of this software, associated documentation and/or data +** (collectively the "Software"), free of charge and under any and all copyright +** rights in the Software, and any and all patent rights owned or freely +** licensable by each licensor hereunder covering either (i) the unmodified +** Software as contributed to or provided by such licensor, or (ii) the Larger +** Works (as defined below), to deal in both +** +** (a) the Software, and +** (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if +** one is included with the Software (each a "Larger Work" to which the Software +** is contributed by such licensors), +** +** without restriction, including without limitation the rights to copy, create +** derivative works of, display, perform, and distribute the Software and make, +** use, sell, offer for sale, import, export, have made, and have sold the +** Software and the Larger Work(s), and to sublicense the foregoing rights on +** either these or other terms. +** +** This license is subject to the following condition: +** The above copyright notice and either this complete permission notice or at +** a minimum a reference to the UPL must be included in all copies or +** substantial portions of the Software. +** +** THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +** IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +** FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +** AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +** LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +** OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +** SOFTWARE. + */ + +/* MODIFIED (MM/DD/YY) +** rcitton 07/14/22 - 33822886 + */ + +package v1alpha1 + +import ( + "reflect" + "strconv" + "strings" + + apierrors "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/runtime/schema" + "k8s.io/apimachinery/pkg/util/validation/field" + ctrl "sigs.k8s.io/controller-runtime" + logf "sigs.k8s.io/controller-runtime/pkg/log" + "sigs.k8s.io/controller-runtime/pkg/webhook" + "sigs.k8s.io/controller-runtime/pkg/webhook/admission" +) + +// log is for logging in this package. +var pdblog = logf.Log.WithName("pdb-webhook") + +func (r *PDB) SetupWebhookWithManager(mgr ctrl.Manager) error { + return ctrl.NewWebhookManagedBy(mgr). + For(r). + Complete() +} + +//+kubebuilder:webhook:path=/mutate-database-oracle-com-v4-pdb,mutating=true,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=pdbs,verbs=create;update,versions=v4,name=mpdb.kb.io,admissionReviewVersions={v1,v1beta1} + +var _ webhook.Defaulter = &PDB{} + +// Default implements webhook.Defaulter so a webhook will be registered for the type +func (r *PDB) Default() { + pdblog.Info("Setting default values in PDB spec for : " + r.Name) + + action := strings.ToUpper(r.Spec.Action) + + if action == "DELETE" { + if r.Spec.DropAction == "" { + r.Spec.DropAction = "INCLUDING" + pdblog.Info(" - dropAction : INCLUDING") + } + } else if action != "MODIFY" && action != "STATUS" { + if r.Spec.ReuseTempFile == nil { + r.Spec.ReuseTempFile = new(bool) + *r.Spec.ReuseTempFile = true + pdblog.Info(" - reuseTempFile : " + strconv.FormatBool(*(r.Spec.ReuseTempFile))) + } + if r.Spec.UnlimitedStorage == nil { + r.Spec.UnlimitedStorage = new(bool) + *r.Spec.UnlimitedStorage = true + pdblog.Info(" - unlimitedStorage : " + strconv.FormatBool(*(r.Spec.UnlimitedStorage))) + } + if r.Spec.TDEImport == nil { + r.Spec.TDEImport = new(bool) + *r.Spec.TDEImport = false + pdblog.Info(" - tdeImport : " + strconv.FormatBool(*(r.Spec.TDEImport))) + } + if r.Spec.TDEExport == nil { + r.Spec.TDEExport = new(bool) + *r.Spec.TDEExport = false + pdblog.Info(" - tdeExport : " + strconv.FormatBool(*(r.Spec.TDEExport))) + } + if r.Spec.AsClone == nil { + r.Spec.AsClone = new(bool) + *r.Spec.AsClone = false + pdblog.Info(" - asClone : " + strconv.FormatBool(*(r.Spec.AsClone))) + } + + } + + if r.Spec.GetScript == nil { + r.Spec.GetScript = new(bool) + *r.Spec.GetScript = false + pdblog.Info(" - getScript : " + strconv.FormatBool(*(r.Spec.GetScript))) + } +} + +// TODO(user): change verbs to "verbs=create;update;delete" if you want to enable deletion validation. +//+kubebuilder:webhook:path=/validate-database-oracle-com-v4-pdb,mutating=false,failurePolicy=fail,sideEffects=None,groups=database.oracle.com,resources=pdbs,verbs=create;update,versions=v4,name=vpdb.kb.io,admissionReviewVersions={v1,v1beta1} + +var _ webhook.Validator = &PDB{} + +// ValidateCreate implements webhook.Validator so a webhook will be registered for the type +func (r *PDB) ValidateCreate() (admission.Warnings, error) { + pdblog.Info("ValidateCreate-Validating PDB spec for : " + r.Name) + + var allErrs field.ErrorList + + r.validateCommon(&allErrs) + + r.validateAction(&allErrs) + + action := strings.ToUpper(r.Spec.Action) + + if len(allErrs) == 0 { + pdblog.Info("PDB Resource : " + r.Name + " successfully validated for Action : " + action) + return nil, nil + } + return nil, apierrors.NewInvalid( + schema.GroupKind{Group: "database.oracle.com", Kind: "PDB"}, + r.Name, allErrs) +} + +// Validate Action for required parameters +func (r *PDB) validateAction(allErrs *field.ErrorList) { + action := strings.ToUpper(r.Spec.Action) + + pdblog.Info("Valdiating PDB Resource Action : " + action) + + if reflect.ValueOf(r.Spec.PDBTlsKey).IsZero() { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("pdbTlsKey"), "Please specify PDB Tls Key(secret)")) + } + + if reflect.ValueOf(r.Spec.PDBTlsCrt).IsZero() { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("pdbTlsCrt"), "Please specify PDB Tls Certificate(secret)")) + } + + if reflect.ValueOf(r.Spec.PDBTlsCat).IsZero() { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("pdbTlsCat"), "Please specify PDB Tls Certificate Authority(secret)")) + } + if reflect.ValueOf(r.Spec.PDBPriKey).IsZero() { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("pdbOrdsPrvKey"), "Please specify PDB Tls Certificate Authority(secret)")) + } + + switch action { + case "DELETE": + /* BUG 36752336 - LREST OPERATOR - DELETE NON-EXISTENT PDB SHOWS LRPDB CREATED MESSAGE */ + if r.Status.OpenMode == "READ WRITE" { + pdblog.Info("Cannot delete: pdb is open ") + *allErrs = append(*allErrs, field.Invalid(field.NewPath("status").Child("OpenMode"), "READ WRITE", "pdb "+r.Spec.PDBName+" "+r.Status.OpenMode)) + } + r.CheckObjExistence("DELETE", allErrs, r) + case "CREATE": + if reflect.ValueOf(r.Spec.AdminName).IsZero() { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("adminName"), "Please specify PDB System Administrator user")) + } + if reflect.ValueOf(r.Spec.AdminPwd).IsZero() { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("adminPwd"), "Please specify PDB System Administrator Password")) + } + if reflect.ValueOf(r.Spec.WebServerUsr).IsZero() { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("WebServerUser"), "Please specify the http webServerUser")) + } + if reflect.ValueOf(r.Spec.WebServerPwd).IsZero() { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("webServerPwd"), "Please specify the http webserverPassword")) + } + + if r.Spec.FileNameConversions == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("fileNameConversions"), "Please specify a value for fileNameConversions. Values can be a filename convert pattern or NONE")) + } + if r.Spec.TotalSize == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("totalSize"), "When the storage is not UNLIMITED the Total Size must be specified")) + } + if r.Spec.TempSize == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("tempSize"), "When the storage is not UNLIMITED the Temp Size must be specified")) + } + if *(r.Spec.TDEImport) { + r.validateTDEInfo(allErrs) + } + case "CLONE": + // Sample Err: The PDB "pdb1-clone" is invalid: spec.srcPdbName: Required value: Please specify source PDB for Cloning + if r.Spec.SrcPDBName == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("srcPdbName"), "Please specify source PDB name for Cloning")) + } + if r.Spec.TotalSize == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("totalSize"), "When the storage is not UNLIMITED the Total Size must be specified")) + } + if r.Spec.TempSize == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("tempSize"), "When the storage is not UNLIMITED the Temp Size must be specified")) + } + /* We don't need this check as ords open the pdb before cloninig */ + /* + if r.Status.OpenMode == "MOUNTED" { + pdblog.Info("Cannot clone: pdb is mount ") + *allErrs = append(*allErrs, field.Invalid(field.NewPath("status").Child("OpenMode"), "READ WRITE", "pdb "+r.Spec.PDBName+" "+r.Status.OpenMode)) + } + */ + case "PLUG": + if r.Spec.XMLFileName == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("xmlFileName"), "Please specify XML metadata filename")) + } + if r.Spec.FileNameConversions == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("fileNameConversions"), "Please specify a value for fileNameConversions. Values can be a filename convert pattern or NONE")) + } + if r.Spec.SourceFileNameConversions == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("sourceFileNameConversions"), "Please specify a value for sourceFileNameConversions. Values can be a filename convert pattern or NONE")) + } + if r.Spec.CopyAction == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("copyAction"), "Please specify a value for copyAction. Values can be COPY, NOCOPY or MOVE")) + } + if *(r.Spec.TDEImport) { + r.validateTDEInfo(allErrs) + } + case "UNPLUG": + if r.Spec.XMLFileName == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("xmlFileName"), "Please specify XML metadata filename")) + } + if *(r.Spec.TDEExport) { + r.validateTDEInfo(allErrs) + } + if r.Status.OpenMode == "READ WRITE" { + pdblog.Info("Cannot unplug: pdb is open ") + *allErrs = append(*allErrs, field.Invalid(field.NewPath("status").Child("OpenMode"), "READ WRITE", "pdb "+r.Spec.PDBName+" "+r.Status.OpenMode)) + } + r.CheckObjExistence("UNPLUG", allErrs, r) + case "MODIFY": + if r.Spec.PDBState == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("pdbState"), "Please specify target state of PDB")) + } + if r.Spec.ModifyOption == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("modifyOption"), "Please specify an option for opening/closing a PDB")) + } + r.CheckObjExistence("MODIY", allErrs, r) + } +} + +// ValidateUpdate implements webhook.Validator so a webhook will be registered for the type +func (r *PDB) ValidateUpdate(old runtime.Object) (admission.Warnings, error) { + pdblog.Info("ValidateUpdate-Validating PDB spec for : " + r.Name) + + isPDBMarkedToBeDeleted := r.GetDeletionTimestamp() != nil + if isPDBMarkedToBeDeleted { + return nil, nil + } + + var allErrs field.ErrorList + action := strings.ToUpper(r.Spec.Action) + + // If PDB CR has been created and in Ready state, only allow updates if the "action" value has changed as well + if (r.Status.Phase == "Ready") && (r.Status.Action != "MODIFY") && (r.Status.Action != "STATUS") && (r.Status.Action == action) { + allErrs = append(allErrs, + field.Required(field.NewPath("spec").Child("action"), "New action also needs to be specified after PDB is in Ready state")) + } else { + + // Check Common Validations + r.validateCommon(&allErrs) + + // Validate required parameters for Action specified + r.validateAction(&allErrs) + + // Check TDE requirements + if (action != "DELETE") && (action != "MODIFY") && (action != "STATUS") && (*(r.Spec.TDEImport) || *(r.Spec.TDEExport)) { + r.validateTDEInfo(&allErrs) + } + } + + if len(allErrs) == 0 { + return nil, nil + } + return nil, apierrors.NewInvalid( + schema.GroupKind{Group: "database.oracle.com", Kind: "PDB"}, + r.Name, allErrs) +} + +// ValidateDelete implements webhook.Validator so a webhook will be registered for the type +func (r *PDB) ValidateDelete() (admission.Warnings, error) { + pdblog.Info("ValidateDelete-Validating PDB spec for : " + r.Name) + + // TODO(user): fill in your validation logic upon object deletion. + return nil, nil +} + +// Validate common specs needed for all PDB Actions +func (r *PDB) validateCommon(allErrs *field.ErrorList) { + pdblog.Info("validateCommon", "name", r.Name) + + if r.Spec.Action == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("action"), "Please specify PDB operation to be performed")) + } + if r.Spec.CDBResName == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("cdbResName"), "Please specify the name of the CDB Kubernetes resource to use for PDB operations")) + } + if r.Spec.PDBName == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("pdbName"), "Please specify name of the PDB to be created")) + } +} + +// Validate TDE information for Create, Plug and Unplug Actions +func (r *PDB) validateTDEInfo(allErrs *field.ErrorList) { + pdblog.Info("validateTDEInfo", "name", r.Name) + + if reflect.ValueOf(r.Spec.TDEPassword).IsZero() { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("tdePassword"), "Please specify a value for tdePassword.")) + } + if r.Spec.TDEKeystorePath == "" { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("tdeKeystorePath"), "Please specify a value for tdeKeystorePath.")) + } + if reflect.ValueOf(r.Spec.TDESecret).IsZero() { + *allErrs = append(*allErrs, + field.Required(field.NewPath("spec").Child("tdeSecret"), "Please specify a value for tdeSecret.")) + } + +} + +func (r *PDB) CheckObjExistence(action string, allErrs *field.ErrorList, pdb *PDB) { + /* BUG 36752465 - lrest operator - open non-existent pdb creates a lrpdb with status failed */ + pdblog.Info("Action [" + action + "] checkin " + pdb.Spec.PDBName + " existence") + if pdb.Status.OpenMode == "" { + *allErrs = append(*allErrs, field.NotFound(field.NewPath("Spec").Child("PDBName"), " "+pdb.Spec.PDBName+" does not exist : action "+action+" failure")) + + } +} diff --git a/apis/database/v1alpha1/zz_generated.deepcopy.go b/apis/database/v1alpha1/zz_generated.deepcopy.go index d0426da8..b20cf834 100644 --- a/apis/database/v1alpha1/zz_generated.deepcopy.go +++ b/apis/database/v1alpha1/zz_generated.deepcopy.go @@ -683,6 +683,239 @@ func (in *Backupconfig) DeepCopy() *Backupconfig { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CDB) DeepCopyInto(out *CDB) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + in.Spec.DeepCopyInto(&out.Spec) + out.Status = in.Status +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDB. +func (in *CDB) DeepCopy() *CDB { + if in == nil { + return nil + } + out := new(CDB) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *CDB) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CDBAdminPassword) DeepCopyInto(out *CDBAdminPassword) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDBAdminPassword. +func (in *CDBAdminPassword) DeepCopy() *CDBAdminPassword { + if in == nil { + return nil + } + out := new(CDBAdminPassword) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CDBAdminUser) DeepCopyInto(out *CDBAdminUser) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDBAdminUser. +func (in *CDBAdminUser) DeepCopy() *CDBAdminUser { + if in == nil { + return nil + } + out := new(CDBAdminUser) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CDBList) DeepCopyInto(out *CDBList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]CDB, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDBList. +func (in *CDBList) DeepCopy() *CDBList { + if in == nil { + return nil + } + out := new(CDBList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *CDBList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CDBPRIVKEY) DeepCopyInto(out *CDBPRIVKEY) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDBPRIVKEY. +func (in *CDBPRIVKEY) DeepCopy() *CDBPRIVKEY { + if in == nil { + return nil + } + out := new(CDBPRIVKEY) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CDBPUBKEY) DeepCopyInto(out *CDBPUBKEY) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDBPUBKEY. +func (in *CDBPUBKEY) DeepCopy() *CDBPUBKEY { + if in == nil { + return nil + } + out := new(CDBPUBKEY) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CDBSecret) DeepCopyInto(out *CDBSecret) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDBSecret. +func (in *CDBSecret) DeepCopy() *CDBSecret { + if in == nil { + return nil + } + out := new(CDBSecret) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CDBSpec) DeepCopyInto(out *CDBSpec) { + *out = *in + out.SysAdminPwd = in.SysAdminPwd + out.CDBAdminUser = in.CDBAdminUser + out.CDBAdminPwd = in.CDBAdminPwd + out.CDBTlsKey = in.CDBTlsKey + out.CDBTlsCrt = in.CDBTlsCrt + out.ORDSPwd = in.ORDSPwd + out.WebServerUser = in.WebServerUser + out.WebServerPwd = in.WebServerPwd + if in.NodeSelector != nil { + in, out := &in.NodeSelector, &out.NodeSelector + *out = make(map[string]string, len(*in)) + for key, val := range *in { + (*out)[key] = val + } + } + out.CDBPubKey = in.CDBPubKey + out.CDBPriKey = in.CDBPriKey +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDBSpec. +func (in *CDBSpec) DeepCopy() *CDBSpec { + if in == nil { + return nil + } + out := new(CDBSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CDBStatus) DeepCopyInto(out *CDBStatus) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDBStatus. +func (in *CDBStatus) DeepCopy() *CDBStatus { + if in == nil { + return nil + } + out := new(CDBStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CDBSysAdminPassword) DeepCopyInto(out *CDBSysAdminPassword) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDBSysAdminPassword. +func (in *CDBSysAdminPassword) DeepCopy() *CDBSysAdminPassword { + if in == nil { + return nil + } + out := new(CDBSysAdminPassword) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CDBTLSCRT) DeepCopyInto(out *CDBTLSCRT) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDBTLSCRT. +func (in *CDBTLSCRT) DeepCopy() *CDBTLSCRT { + if in == nil { + return nil + } + out := new(CDBTLSCRT) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CDBTLSKEY) DeepCopyInto(out *CDBTLSKEY) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CDBTLSKEY. +func (in *CDBTLSKEY) DeepCopy() *CDBTLSKEY { + if in == nil { + return nil + } + out := new(CDBTLSKEY) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *CatalogSpec) DeepCopyInto(out *CatalogSpec) { *out = *in @@ -1470,6 +1703,22 @@ func (in *KMSDetailsStatus) DeepCopy() *KMSDetailsStatus { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *ORDSPassword) DeepCopyInto(out *ORDSPassword) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ORDSPassword. +func (in *ORDSPassword) DeepCopy() *ORDSPassword { + if in == nil { + return nil + } + out := new(ORDSPassword) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *OciAcdSpec) DeepCopyInto(out *OciAcdSpec) { *out = *in @@ -1738,6 +1987,65 @@ func (in *OracleRestDataServiceStatus) DeepCopy() *OracleRestDataServiceStatus { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PDB) DeepCopyInto(out *PDB) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + in.Spec.DeepCopyInto(&out.Spec) + out.Status = in.Status +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDB. +func (in *PDB) DeepCopy() *PDB { + if in == nil { + return nil + } + out := new(PDB) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *PDB) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PDBAdminName) DeepCopyInto(out *PDBAdminName) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBAdminName. +func (in *PDBAdminName) DeepCopy() *PDBAdminName { + if in == nil { + return nil + } + out := new(PDBAdminName) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PDBAdminPassword) DeepCopyInto(out *PDBAdminPassword) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBAdminPassword. +func (in *PDBAdminPassword) DeepCopy() *PDBAdminPassword { + if in == nil { + return nil + } + out := new(PDBAdminPassword) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *PDBConfig) DeepCopyInto(out *PDBConfig) { *out = *in @@ -1849,6 +2157,204 @@ func (in *PDBDetailsStatus) DeepCopy() *PDBDetailsStatus { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PDBList) DeepCopyInto(out *PDBList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]PDB, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBList. +func (in *PDBList) DeepCopy() *PDBList { + if in == nil { + return nil + } + out := new(PDBList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *PDBList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PDBPRIVKEY) DeepCopyInto(out *PDBPRIVKEY) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBPRIVKEY. +func (in *PDBPRIVKEY) DeepCopy() *PDBPRIVKEY { + if in == nil { + return nil + } + out := new(PDBPRIVKEY) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PDBPUBKEY) DeepCopyInto(out *PDBPUBKEY) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBPUBKEY. +func (in *PDBPUBKEY) DeepCopy() *PDBPUBKEY { + if in == nil { + return nil + } + out := new(PDBPUBKEY) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PDBSecret) DeepCopyInto(out *PDBSecret) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBSecret. +func (in *PDBSecret) DeepCopy() *PDBSecret { + if in == nil { + return nil + } + out := new(PDBSecret) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PDBSpec) DeepCopyInto(out *PDBSpec) { + *out = *in + out.PDBTlsKey = in.PDBTlsKey + out.PDBTlsCrt = in.PDBTlsCrt + out.PDBTlsCat = in.PDBTlsCat + out.AdminName = in.AdminName + out.AdminPwd = in.AdminPwd + out.WebServerUsr = in.WebServerUsr + out.WebServerPwd = in.WebServerPwd + if in.ReuseTempFile != nil { + in, out := &in.ReuseTempFile, &out.ReuseTempFile + *out = new(bool) + **out = **in + } + if in.UnlimitedStorage != nil { + in, out := &in.UnlimitedStorage, &out.UnlimitedStorage + *out = new(bool) + **out = **in + } + if in.AsClone != nil { + in, out := &in.AsClone, &out.AsClone + *out = new(bool) + **out = **in + } + if in.TDEImport != nil { + in, out := &in.TDEImport, &out.TDEImport + *out = new(bool) + **out = **in + } + if in.TDEExport != nil { + in, out := &in.TDEExport, &out.TDEExport + *out = new(bool) + **out = **in + } + out.TDEPassword = in.TDEPassword + out.TDESecret = in.TDESecret + if in.GetScript != nil { + in, out := &in.GetScript, &out.GetScript + *out = new(bool) + **out = **in + } + out.PDBPubKey = in.PDBPubKey + out.PDBPriKey = in.PDBPriKey +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBSpec. +func (in *PDBSpec) DeepCopy() *PDBSpec { + if in == nil { + return nil + } + out := new(PDBSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PDBStatus) DeepCopyInto(out *PDBStatus) { + *out = *in +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBStatus. +func (in *PDBStatus) DeepCopy() *PDBStatus { + if in == nil { + return nil + } + out := new(PDBStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PDBTLSCAT) DeepCopyInto(out *PDBTLSCAT) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBTLSCAT. +func (in *PDBTLSCAT) DeepCopy() *PDBTLSCAT { + if in == nil { + return nil + } + out := new(PDBTLSCAT) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PDBTLSCRT) DeepCopyInto(out *PDBTLSCRT) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBTLSCRT. +func (in *PDBTLSCRT) DeepCopy() *PDBTLSCRT { + if in == nil { + return nil + } + out := new(PDBTLSCRT) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PDBTLSKEY) DeepCopyInto(out *PDBTLSKEY) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PDBTLSKEY. +func (in *PDBTLSKEY) DeepCopy() *PDBTLSKEY { + if in == nil { + return nil + } + out := new(PDBTLSKEY) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *PasswordSpec) DeepCopyInto(out *PasswordSpec) { *out = *in @@ -2414,6 +2920,38 @@ func (in *SourceSpec) DeepCopy() *SourceSpec { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *TDEPwd) DeepCopyInto(out *TDEPwd) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new TDEPwd. +func (in *TDEPwd) DeepCopy() *TDEPwd { + if in == nil { + return nil + } + out := new(TDEPwd) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *TDESecret) DeepCopyInto(out *TDESecret) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new TDESecret. +func (in *TDESecret) DeepCopy() *TDESecret { + if in == nil { + return nil + } + out := new(TDESecret) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *TargetSpec) DeepCopyInto(out *TargetSpec) { *out = *in @@ -2486,3 +3024,67 @@ func (in *WalletSpec) DeepCopy() *WalletSpec { in.DeepCopyInto(out) return out } + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *WebServerPassword) DeepCopyInto(out *WebServerPassword) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new WebServerPassword. +func (in *WebServerPassword) DeepCopy() *WebServerPassword { + if in == nil { + return nil + } + out := new(WebServerPassword) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *WebServerPasswordPDB) DeepCopyInto(out *WebServerPasswordPDB) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new WebServerPasswordPDB. +func (in *WebServerPasswordPDB) DeepCopy() *WebServerPasswordPDB { + if in == nil { + return nil + } + out := new(WebServerPasswordPDB) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *WebServerUser) DeepCopyInto(out *WebServerUser) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new WebServerUser. +func (in *WebServerUser) DeepCopy() *WebServerUser { + if in == nil { + return nil + } + out := new(WebServerUser) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *WebServerUserPDB) DeepCopyInto(out *WebServerUserPDB) { + *out = *in + out.Secret = in.Secret +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new WebServerUserPDB. +func (in *WebServerUserPDB) DeepCopy() *WebServerUserPDB { + if in == nil { + return nil + } + out := new(WebServerUserPDB) + in.DeepCopyInto(out) + return out +} diff --git a/config/crd/bases/database.oracle.com_cdbs.yaml b/config/crd/bases/database.oracle.com_cdbs.yaml index 8ea594e6..924946ee 100644 --- a/config/crd/bases/database.oracle.com_cdbs.yaml +++ b/config/crd/bases/database.oracle.com_cdbs.yaml @@ -14,6 +14,244 @@ spec: singular: cdb scope: Namespaced versions: + - additionalPrinterColumns: + - description: Name of the CDB + jsonPath: .spec.cdbName + name: CDB Name + type: string + - description: ' Name of the DB Server' + jsonPath: .spec.dbServer + name: DB Server + type: string + - description: DB server port + jsonPath: .spec.dbPort + name: DB Port + type: integer + - description: Replicas + jsonPath: .spec.replicas + name: Replicas + type: integer + - description: Status of the CDB Resource + jsonPath: .status.phase + name: Status + type: string + - description: Error message, if any + jsonPath: .status.msg + name: Message + type: string + - description: ' string of the tnsalias' + jsonPath: .spec.dbTnsurl + name: TNS STRING + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + properties: + apiVersion: + type: string + kind: + type: string + metadata: + type: object + spec: + properties: + cdbAdminPwd: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + cdbAdminUser: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + cdbName: + type: string + cdbOrdsPrvKey: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + cdbOrdsPubKey: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + cdbTlsCrt: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + cdbTlsKey: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + dbPort: + type: integer + dbServer: + type: string + dbTnsurl: + type: string + deletePdbCascade: + type: boolean + nodeSelector: + additionalProperties: + type: string + type: object + ordsImage: + type: string + ordsImagePullPolicy: + enum: + - Always + - Never + type: string + ordsImagePullSecret: + type: string + ordsPort: + type: integer + ordsPwd: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + replicas: + type: integer + serviceName: + type: string + sysAdminPwd: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + webServerPwd: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + webServerUser: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + type: object + status: + properties: + msg: + type: string + phase: + type: string + status: + type: boolean + required: + - phase + - status + type: object + type: object + served: true + storage: false + subresources: + status: {} - additionalPrinterColumns: - description: Name of the CDB jsonPath: .spec.cdbName diff --git a/config/crd/bases/database.oracle.com_dataguardbrokers.yaml b/config/crd/bases/database.oracle.com_dataguardbrokers.yaml index 5efceff4..0e27126d 100644 --- a/config/crd/bases/database.oracle.com_dataguardbrokers.yaml +++ b/config/crd/bases/database.oracle.com_dataguardbrokers.yaml @@ -94,7 +94,7 @@ spec: externalConnectString: type: string fastStartFailover: - type: boolean + type: string primaryDatabase: type: string primaryDatabaseRef: @@ -191,7 +191,7 @@ spec: externalConnectString: type: string fastStartFailover: - type: boolean + type: string primaryDatabase: type: string primaryDatabaseRef: diff --git a/config/crd/bases/database.oracle.com_pdbs.yaml b/config/crd/bases/database.oracle.com_pdbs.yaml index b674f856..b2f37ac9 100644 --- a/config/crd/bases/database.oracle.com_pdbs.yaml +++ b/config/crd/bases/database.oracle.com_pdbs.yaml @@ -43,7 +43,7 @@ spec: jsonPath: .status.connString name: Connect_String type: string - name: v4 + name: v1alpha1 schema: openAPIV3Schema: properties: @@ -100,17 +100,318 @@ spec: type: boolean assertivePdbDeletion: type: boolean - assertivePdbDeletion: - description: turn on the assertive approach to delete pdb resource - kubectl delete pdb ..... automatically triggers the pluggable database - deletion - type: boolean cdbName: type: string cdbNamespace: type: string + cdbResName: + type: string + copyAction: + enum: + - COPY + - NOCOPY + - MOVE + type: string + dropAction: + enum: + - INCLUDING + - KEEP + type: string + fileNameConversions: + type: string + getScript: + type: boolean + modifyOption: + enum: + - IMMEDIATE + - NORMAL + - READ ONLY + - READ WRITE + - RESTRICTED + type: string + pdbName: + type: string + pdbOrdsPrvKey: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + pdbOrdsPubKey: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + pdbState: + enum: + - OPEN + - CLOSE + type: string + pdbTlsCat: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + pdbTlsCrt: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + pdbTlsKey: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + reuseTempFile: + type: boolean + sourceFileNameConversions: + type: string + sparseClonePath: + type: string + srcPdbName: + type: string + tdeExport: + type: boolean + tdeImport: + type: boolean + tdeKeystorePath: + type: string + tdePassword: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + tdeSecret: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + tempSize: + type: string + totalSize: + type: string + unlimitedStorage: + type: boolean + webServerPwd: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + webServerUser: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + xmlFileName: + type: string + required: + - action + type: object + status: + properties: + action: + type: string + connString: + type: string + modifyOption: + type: string + msg: + type: string + openMode: + type: string + phase: + type: string + status: + type: boolean + totalSize: + type: string + required: + - phase + - status + type: object + type: object + served: true + storage: false + subresources: + status: {} + - additionalPrinterColumns: + - description: Name of the CDB + jsonPath: .spec.cdbName + name: CDB Name + type: string + - description: Name of the PDB + jsonPath: .spec.pdbName + name: PDB Name + type: string + - description: PDB Open Mode + jsonPath: .status.openMode + name: PDB State + type: string + - description: Total Size of the PDB + jsonPath: .status.totalSize + name: PDB Size + type: string + - description: Status of the PDB Resource + jsonPath: .status.phase + name: Status + type: string + - description: Error message, if any + jsonPath: .status.msg + name: Message + type: string + - description: The connect string to be used + jsonPath: .status.connString + name: Connect_String + type: string + name: v4 + schema: + openAPIV3Schema: + properties: + apiVersion: + type: string + kind: + type: string + metadata: + type: object + spec: + properties: + action: + enum: + - Create + - Clone + - Plug + - Unplug + - Delete + - Modify + - Status + - Map + type: string + adminName: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + adminPwd: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + asClone: + type: boolean + assertivePdbDeletion: + type: boolean + cdbName: + type: string cdbNamespace: - description: CDB Namespace type: string cdbResName: type: string diff --git a/config/crd/bases/database.oracle.com_shardingdatabases.yaml b/config/crd/bases/database.oracle.com_shardingdatabases.yaml index e46d883e..90c6dd53 100644 --- a/config/crd/bases/database.oracle.com_shardingdatabases.yaml +++ b/config/crd/bases/database.oracle.com_shardingdatabases.yaml @@ -154,9 +154,6 @@ spec: directorName: type: string envVars: - description: Replicas int32 `json:"replicas,omitempty"` // - Gsm Replicas. If you set OraGsmPvcName then it is set default - to 1. items: properties: name: @@ -658,12 +655,6 @@ spec: x-kubernetes-int-or-string: true type: object type: object - shardGroup: - type: string - shardRegion: - type: string - shardSpace: - type: string storageSizeInGb: format: int32 type: integer diff --git a/config/manager/kustomization.yaml b/config/manager/kustomization.yaml index 1a9d97d3..7a52fb17 100644 --- a/config/manager/kustomization.yaml +++ b/config/manager/kustomization.yaml @@ -8,5 +8,5 @@ apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization images: - name: controller - newName: lin.ocir.io/intsanjaysingh/mmalvezz/testppr/operatormntnns - newTag: latest + newName: container-registry.oracle.com/database/operator + newTag: 1.2.0 diff --git a/config/webhook/manifests.yaml b/config/webhook/manifests.yaml index 3a0f15ec..b186a5b0 100644 --- a/config/webhook/manifests.yaml +++ b/config/webhook/manifests.yaml @@ -168,6 +168,27 @@ webhooks: resources: - autonomousdatabasebackups sideEffects: None +- admissionReviewVersions: + - v1 + - v1beta1 + clientConfig: + service: + name: webhook-service + namespace: system + path: /mutate-database-oracle-com-v4-cdb + failurePolicy: Fail + name: mcdb.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v4 + operations: + - CREATE + - UPDATE + resources: + - cdbs + sideEffects: None - admissionReviewVersions: - v1 - v1beta1 @@ -232,23 +253,24 @@ webhooks: sideEffects: None - admissionReviewVersions: - v1 + - v1beta1 clientConfig: service: name: webhook-service namespace: system - path: /mutate-database-oracle-com-v1alpha1-shardingdatabase + path: /mutate-database-oracle-com-v4-pdb failurePolicy: Fail - name: mshardingdatabasev1alpha1.kb.io + name: mpdb.kb.io rules: - apiGroups: - database.oracle.com apiVersions: - - v1alpha1 + - v4 operations: - CREATE - UPDATE resources: - - shardingdatabases + - pdbs sideEffects: None - admissionReviewVersions: - v1 @@ -258,7 +280,7 @@ webhooks: namespace: system path: /mutate-database-oracle-com-v1alpha1-shardingdatabase failurePolicy: Fail - name: mshardingdatabase.kb.io + name: mshardingdatabasev1alpha1.kb.io rules: - apiGroups: - database.oracle.com @@ -602,6 +624,27 @@ webhooks: resources: - autonomousdatabases sideEffects: None +- admissionReviewVersions: + - v1 + - v1beta1 + clientConfig: + service: + name: webhook-service + namespace: system + path: /validate-database-oracle-com-v4-cdb + failurePolicy: Fail + name: vcdb.kb.io + rules: + - apiGroups: + - database.oracle.com + apiVersions: + - v4 + operations: + - CREATE + - UPDATE + resources: + - cdbs + sideEffects: None - admissionReviewVersions: - v1 - v1beta1 @@ -646,24 +689,24 @@ webhooks: sideEffects: None - admissionReviewVersions: - v1 + - v1beta1 clientConfig: service: name: webhook-service namespace: system - path: /validate-database-oracle-com-v1alpha1-shardingdatabase + path: /validate-database-oracle-com-v4-pdb failurePolicy: Fail - name: vshardingdatabasev1alpha1.kb.io + name: vpdb.kb.io rules: - apiGroups: - database.oracle.com apiVersions: - - v1alpha1 + - v4 operations: - CREATE - UPDATE - - DELETE resources: - - shardingdatabases + - pdbs sideEffects: None - admissionReviewVersions: - v1 @@ -673,7 +716,7 @@ webhooks: namespace: system path: /validate-database-oracle-com-v1alpha1-shardingdatabase failurePolicy: Fail - name: vshardingdatabase.kb.io + name: vshardingdatabasev1alpha1.kb.io rules: - apiGroups: - database.oracle.com diff --git a/oracle-database-operator.yaml b/oracle-database-operator.yaml index 70147329..1179b272 100644 --- a/oracle-database-operator.yaml +++ b/oracle-database-operator.yaml @@ -1309,6 +1309,244 @@ spec: singular: cdb scope: Namespaced versions: + - additionalPrinterColumns: + - description: Name of the CDB + jsonPath: .spec.cdbName + name: CDB Name + type: string + - description: ' Name of the DB Server' + jsonPath: .spec.dbServer + name: DB Server + type: string + - description: DB server port + jsonPath: .spec.dbPort + name: DB Port + type: integer + - description: Replicas + jsonPath: .spec.replicas + name: Replicas + type: integer + - description: Status of the CDB Resource + jsonPath: .status.phase + name: Status + type: string + - description: Error message, if any + jsonPath: .status.msg + name: Message + type: string + - description: ' string of the tnsalias' + jsonPath: .spec.dbTnsurl + name: TNS STRING + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + properties: + apiVersion: + type: string + kind: + type: string + metadata: + type: object + spec: + properties: + cdbAdminPwd: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + cdbAdminUser: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + cdbName: + type: string + cdbOrdsPrvKey: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + cdbOrdsPubKey: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + cdbTlsCrt: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + cdbTlsKey: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + dbPort: + type: integer + dbServer: + type: string + dbTnsurl: + type: string + deletePdbCascade: + type: boolean + nodeSelector: + additionalProperties: + type: string + type: object + ordsImage: + type: string + ordsImagePullPolicy: + enum: + - Always + - Never + type: string + ordsImagePullSecret: + type: string + ordsPort: + type: integer + ordsPwd: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + replicas: + type: integer + serviceName: + type: string + sysAdminPwd: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + webServerPwd: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + webServerUser: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + type: object + status: + properties: + msg: + type: string + phase: + type: string + status: + type: boolean + required: + - phase + - status + type: object + type: object + served: true + storage: false + subresources: + status: {} - additionalPrinterColumns: - description: Name of the CDB jsonPath: .spec.cdbName @@ -8631,7 +8869,7 @@ spec: externalConnectString: type: string fastStartFailover: - type: boolean + type: string primaryDatabase: type: string primaryDatabaseRef: @@ -8728,7 +8966,7 @@ spec: externalConnectString: type: string fastStartFailover: - type: boolean + type: string primaryDatabase: type: string primaryDatabaseRef: @@ -11023,7 +11261,7 @@ spec: jsonPath: .status.connString name: Connect_String type: string - name: v4 + name: v1alpha1 schema: openAPIV3Schema: properties: @@ -11080,16 +11318,10 @@ spec: type: boolean assertivePdbDeletion: type: boolean - assertivePdbDeletion: - description: turn on the assertive approach to delete pdb resource kubectl delete pdb ..... automatically triggers the pluggable database deletion - type: boolean cdbName: type: string cdbNamespace: type: string - cdbNamespace: - description: CDB Namespace - type: string cdbResName: type: string copyAction: @@ -11306,15 +11538,324 @@ spec: type: object type: object served: true - storage: true + storage: false subresources: status: {} ---- -apiVersion: apiextensions.k8s.io/v1 -kind: CustomResourceDefinition -metadata: - annotations: - cert-manager.io/inject-ca-from: oracle-database-operator-system/oracle-database-operator-serving-cert + - additionalPrinterColumns: + - description: Name of the CDB + jsonPath: .spec.cdbName + name: CDB Name + type: string + - description: Name of the PDB + jsonPath: .spec.pdbName + name: PDB Name + type: string + - description: PDB Open Mode + jsonPath: .status.openMode + name: PDB State + type: string + - description: Total Size of the PDB + jsonPath: .status.totalSize + name: PDB Size + type: string + - description: Status of the PDB Resource + jsonPath: .status.phase + name: Status + type: string + - description: Error message, if any + jsonPath: .status.msg + name: Message + type: string + - description: The connect string to be used + jsonPath: .status.connString + name: Connect_String + type: string + name: v4 + schema: + openAPIV3Schema: + properties: + apiVersion: + type: string + kind: + type: string + metadata: + type: object + spec: + properties: + action: + enum: + - Create + - Clone + - Plug + - Unplug + - Delete + - Modify + - Status + - Map + type: string + adminName: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + adminPwd: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + asClone: + type: boolean + assertivePdbDeletion: + type: boolean + cdbName: + type: string + cdbNamespace: + type: string + cdbResName: + type: string + copyAction: + enum: + - COPY + - NOCOPY + - MOVE + type: string + dropAction: + enum: + - INCLUDING + - KEEP + type: string + fileNameConversions: + type: string + getScript: + type: boolean + modifyOption: + enum: + - IMMEDIATE + - NORMAL + - READ ONLY + - READ WRITE + - RESTRICTED + type: string + pdbName: + type: string + pdbOrdsPrvKey: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + pdbOrdsPubKey: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + pdbState: + enum: + - OPEN + - CLOSE + type: string + pdbTlsCat: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + pdbTlsCrt: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + pdbTlsKey: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + reuseTempFile: + type: boolean + sourceFileNameConversions: + type: string + sparseClonePath: + type: string + srcPdbName: + type: string + tdeExport: + type: boolean + tdeImport: + type: boolean + tdeKeystorePath: + type: string + tdePassword: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + tdeSecret: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + tempSize: + type: string + totalSize: + type: string + unlimitedStorage: + type: boolean + webServerPwd: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + webServerUser: + properties: + secret: + properties: + key: + type: string + secretName: + type: string + required: + - key + - secretName + type: object + required: + - secret + type: object + xmlFileName: + type: string + required: + - action + type: object + status: + properties: + action: + type: string + connString: + type: string + modifyOption: + type: string + msg: + type: string + openMode: + type: string + phase: + type: string + status: + type: boolean + totalSize: + type: string + required: + - phase + - status + type: object + type: object + served: true + storage: true + subresources: + status: {} +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + cert-manager.io/inject-ca-from: oracle-database-operator-system/oracle-database-operator-serving-cert controller-gen.kubebuilder.io/version: v0.16.5 name: shardingdatabases.database.oracle.com spec: @@ -11466,7 +12007,6 @@ spec: directorName: type: string envVars: - description: Replicas int32 `json:"replicas,omitempty"` // Gsm Replicas. If you set OraGsmPvcName then it is set default to 1. items: properties: name: @@ -11968,12 +12508,6 @@ spec: x-kubernetes-int-or-string: true type: object type: object - shardGroup: - type: string - shardRegion: - type: string - shardSpace: - type: string storageSizeInGb: format: int32 type: integer @@ -13788,26 +14322,6 @@ webhooks: resources: - shardingdatabases sideEffects: None -- admissionReviewVersions: - - v1 - clientConfig: - service: - name: oracle-database-operator-webhook-service - namespace: oracle-database-operator-system - path: /mutate-database-oracle-com-v1alpha1-shardingdatabase - failurePolicy: Fail - name: mshardingdatabase.kb.io - rules: - - apiGroups: - - database.oracle.com - apiVersions: - - v1alpha1 - operations: - - CREATE - - UPDATE - resources: - - shardingdatabases - sideEffects: None - admissionReviewVersions: - v1 - v1beta1 @@ -14165,27 +14679,6 @@ webhooks: resources: - shardingdatabases sideEffects: None -- admissionReviewVersions: - - v1 - clientConfig: - service: - name: oracle-database-operator-webhook-service - namespace: oracle-database-operator-system - path: /validate-database-oracle-com-v1alpha1-shardingdatabase - failurePolicy: Fail - name: vshardingdatabase.kb.io - rules: - - apiGroups: - - database.oracle.com - apiVersions: - - v1alpha1 - operations: - - CREATE - - UPDATE - - DELETE - resources: - - shardingdatabases - sideEffects: None - admissionReviewVersions: - v1 - v1beta1 From 5a1d9d802f9fa841cbd3eaac62a14fb5de7b39be Mon Sep 17 00:00:00 2001 From: marcstef Date: Fri, 9 May 2025 12:16:40 +0000 Subject: [PATCH 625/628] example SIDB --- docs/ordsservices/examples/sidb_container.md | 50 ++++++++++---------- 1 file changed, 24 insertions(+), 26 deletions(-) diff --git a/docs/ordsservices/examples/sidb_container.md b/docs/ordsservices/examples/sidb_container.md index 804ecca4..3cda09ea 100644 --- a/docs/ordsservices/examples/sidb_container.md +++ b/docs/ordsservices/examples/sidb_container.md @@ -2,57 +2,50 @@ This example walks through using the **ORDSSRVS Controller** with a Containerised Oracle Database created by the **SIDB Controller** in the same Kubernetes Cluster. -### Cert-Manager and Oracle Database Operator installation - -Install the [Cert Manager](https://github.com/cert-manager/cert-manager/releases/download/v1.14.4/cert-manager.yaml) and the [Oracle Database Operator](https://github.com/oracle/oracle-database-operator) using the instractions in the Operator [README](https://github.com/oracle/oracle-database-operator/blob/main/README.md) file. - +Before testing this example, please verify the prerequisites : [ORDSSRVS prerequisites](../README.md#prerequisites) ### Deploy a Containerised Oracle Database +Refer to Single Instance Database (SIDB) [README](https://github.com/oracle/oracle-database-operator/blob/main/docs/sidb/README.md) for details. + 1. Create a Secret for the Database password: ```bash - DB_PWD=$(echo "ORDSpoc_$(date +%H%S%M)") - - kubectl create secret generic sidb-db-auth \ - --from-literal=password=${DB_PWD} + DB_PWD= + kubectl create secret generic sidb-db-auth --from-literal=password=${DB_PWD} --namespace ordsnamespace ``` 1. Create a manifest for the containerised Oracle Database. The POC uses an Oracle Free Image, but other versions may be subsituted; review the OraOperator Documentation for details on the manifests. - ```bash - echo " - apiVersion: database.oracle.com/v1alpha1 + ```yaml + apiVersion: database.oracle.com/v4 kind: SingleInstanceDatabase metadata: name: oraoper-sidb + namespace: ordsnamespace spec: - replicas: 1 - image: - pullFrom: container-registry.oracle.com/database/free:23.4.0.0 - prebuiltDB: true - sid: FREE edition: free adminPassword: secretName: sidb-db-auth - secretKey: password - pdbName: FREEPDB1" | kubectl apply -f - + image: + pullFrom: container-registry.oracle.com/database/free:23.7.0.0 + prebuiltDB: true + replicas: 1 ``` - latest container-registry.oracle.com/database/free version, **23.4.0.0**, valid as of **2-May-2024** + latest container-registry.oracle.com/database/free version, **23.7.0.0-lite**, valid as of **2-May-2025** + 1. Watch the `singleinstancedatabases` resource until the database status is **Healthy**: ```bash - kubectl get singleinstancedatabases/oraoper-sidb -w + kubectl get singleinstancedatabases/oraoper-sidb -w -n ordsnamespace ``` - **NOTE**: If this is the first time pulling the free database image, it may take up to 15 minutes for the database to become available. ### Create encryped secret ```bash - openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -pkeyopt rsa_keygen_pubexp:65537 > ca.key openssl rsa -in ca.key -outform PEM -pubout -out public.pem kubectl create secret generic prvkey --from-file=privateKey=ca.key -n ordsnamespace @@ -61,8 +54,6 @@ echo "${DB_PWD}" > sidb-db-auth openssl rsautl -encrypt -pubin -inkey public.pem -in sidb-db-auth |base64 > e_sidb-db-auth kubectl create secret generic sidb-db-auth-enc --from-file=password=e_sidb-db-auth -n ordsnamespace rm sidb-db-auth e_sidb-db-auth - - ``` @@ -72,6 +63,7 @@ rm sidb-db-auth e_sidb-db-auth ```bash CONN_STRING=$(kubectl get singleinstancedatabase oraoper-sidb \ + -n ordsnamespace \ -o jsonpath='{.status.pdbConnectString}') echo $CONN_STRING @@ -89,7 +81,7 @@ rm sidb-db-auth e_sidb-db-auth ```bash echo " - apiVersion: database.oracle.com/v1 + apiVersion: database.oracle.com/v4 kind: OrdsSrvs metadata: name: ords-sidb @@ -97,6 +89,9 @@ rm sidb-db-auth e_sidb-db-auth spec: image: container-registry.oracle.com/database/ords:24.1.1 forceRestart: true + encPrivKey: + secretName: prvkey + passwordKey: privateKey globalSettings: database.api.enabled: true poolSettings: @@ -112,7 +107,10 @@ rm sidb-db-auth e_sidb-db-auth secretName: sidb-db-auth-enc db.adminUser: SYS db.adminUser.secret: - secretName: sidb-db-auth-enc" | kubectl apply -f - + secretName: sidb-db-auth-enc + " > ords-sidb.yaml + + kubectl apply -f ords-sidb.yaml ``` latest container-registry.oracle.com/database/ords version, **24.1.1**, valid as of **30-May-2024** From 9f3e5dfdf0c1578158d7c99575fc8faec0ba4c28 Mon Sep 17 00:00:00 2001 From: marcstef Date: Fri, 9 May 2025 12:30:13 +0000 Subject: [PATCH 626/628] ORDSSRVS examples --- docs/ordsservices/README.md | 43 ++++++- docs/ordsservices/examples/adb.md | 40 +++---- docs/ordsservices/examples/adb_oraoper.md | 86 ++++++-------- docs/ordsservices/examples/existing_db.md | 112 ++++++++++++++++++ docs/ordsservices/examples/mongo_api.md | 14 +-- docs/ordsservices/examples/multi_pool.md | 27 ++--- .../examples/ordsnamespace-role-binding.yaml | 13 ++ 7 files changed, 234 insertions(+), 101 deletions(-) create mode 100644 docs/ordsservices/examples/existing_db.md create mode 100644 docs/ordsservices/examples/ordsnamespace-role-binding.yaml diff --git a/docs/ordsservices/README.md b/docs/ordsservices/README.md index 1740e99f..57195120 100644 --- a/docs/ordsservices/README.md +++ b/docs/ordsservices/README.md @@ -25,22 +25,53 @@ It supports the majority of ORDS configuration settings as per the [API Document The ORDS and APEX schemas can be [automatically installed/upgraded](./autoupgrade.md) into the Oracle Database by the ORDS controller. ORDS Version support: -* v22.1+ +* 24.1.1 +(Newer versions of ORDS will be supported in the next update of OraOperator) Oracle Database Version: * 19c * 23ai (incl. 23ai Free) +### Prerequisites -### Common Configurations +1. Oracle Database Operator + + Install the Oracle Database Operator (OraOperator) using the instructions in the [README](https://github.com/oracle/oracle-database-operator/blob/main/README.md) file. + +1. Namespace + + For a dedicated namespace deployment of the ORDSSRVS controller, refer to the "Namespace Scoped Deployment" section in the OraOperator [README](https://github.com/oracle/oracle-database-operator/blob/main/README.md#2-namespace-scoped-deployment). + + The following examples deploy the controller to the 'ordsnamespace' namespace. + + Create the namespace: + ```bash + kubectl create namespace ordsnamespace + ``` + + Apply namespace role binding [ordsnamespace-role-binding.yaml](./ordsnamespace-role-binding.yaml): + ```bash + kubectl apply -f ordsnamespace-role-binding.yaml + ``` + + Edit OraOperator to add the namespace under WATCH_NAMESPACE: + ```yaml + - name: WATCH_NAMESPACE + value: "default,,ordsnamespace" + ``` + +### Common configuration examples A few common configuration examples can be used to quickly familiarise yourself with the ORDS Custom Resource Definition. The "Conclusion" section of each example highlights specific settings to enable functionality that maybe of interest. -* [Containerised Single Instance Database using the Oracontroller](./examples/sidb_container.md) -* [Multipool, Multidatabase using a TNS Names file](./examples/multi_pool.md) -* [Autonomous Database using the Oracontroller](./examples/adb_oraoper.md) - (Customer Managed ORDS) *See [Limitations](#limitations) -* [Autonomous Database without the Oracontroller](./examples/adb.md) - (Customer Managed ORDS) +Before + +* [Pre-existing Database](./examples/existing_db.md) +* [Containerised Single Instance Database (SIDB)](./examples/sidb_container.md) +* [Multidatabase using a TNS Names file](./examples/multi_pool.md) +* [Autonomous Database using the OraOperator](./examples/adb_oraoper.md) *See [Limitations](#limitations) +* [Autonomous Database without the OraOperator](./examples/adb.md) * [Oracle API for MongoDB Support](./examples/mongo_api.md) Running through all examples in the same Kubernetes cluster illustrates the ability to run multiple ORDS instances with a variety of different configurations. diff --git a/docs/ordsservices/examples/adb.md b/docs/ordsservices/examples/adb.md index ba53aac5..90a21b5c 100644 --- a/docs/ordsservices/examples/adb.md +++ b/docs/ordsservices/examples/adb.md @@ -5,11 +5,7 @@ This example walks through using the **ORDSSRVS controller** with an Oracle Auto This assumes that an ADB has already been provisioned and is configured as "Secure Access from Anywhere". Note that if behind a Proxy, this example will not work as the Wallet will need to be modified to support the proxy configuration. - -### Cert-Manager and Oracle Database Operator installation - -Install the [Cert Manager](https://github.com/cert-manager/cert-manager/releases/download/v1.14.4/cert-manager.yaml) and the [Oracle Database Operator](https://github.com/oracle/oracle-database-operator) using the instractions in the Operator [README](https://github.com/oracle/oracle-database-operator/blob/main/README.md) file. - +Before testing this example, please verify the prerequisites : [ORDSSRVS prerequisites](../README.md#prerequisites) ### ADB Wallet Secret @@ -25,13 +21,13 @@ kubectl create secret generic adb-wallet \ Create a Secret for the ADB ADMIN password, replacing with the real password: ```bash -echo adb-db-auth-enc -openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -pkeyopt rsa_keygen_pubexp:65537 > ca.k +echo ${ADMIN_PASSWORD} > adb-db-auth-enc +openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -pkeyopt rsa_keygen_pubexp:65537 > ca.key openssl rsa -in ca.key -outform PEM -pubout -out public.pem kubectl create secret generic prvkey --from-file=privateKey=ca.key -n ordsnamespace -openssl rsautl -encrypt -pubin -inkey public.pem -in adb-db-auth-enc |base64 > e_sidb-db-auth-enc -kubectl create secret generic adb-db-auth-enc --from-file=password=e_sidb-db-auth-enc -n ordsnamespace -rm adb-db-auth-enc e_sidb-db-auth-enc +openssl rsautl -encrypt -pubin -inkey public.pem -in adb-db-auth-enc |base64 > e_adb-db-auth-enc +kubectl create secret generic adb-oraoper-db-auth-enc --from-file=password=e_adb-db-auth-enc -n ordsnamespace +rm adb-db-auth-enc e_adb-db-auth-enc ``` ### Create RestDataServices Resource @@ -43,22 +39,24 @@ rm adb-db-auth-enc e_sidb-db-auth-enc Replace with the ADB Name and ensure that the `db.wallet.zip.service` is valid for your ADB Workload (e.g. _TP or _HIGH, etc.): - ```bash - echo " - apiVersion: database.oracle.com/v1 - kind: OrdsSrvs + ```yaml + apiVersion: database.oracle.com/v4 + kind: OrdsSrvs metadata: name: ords-adb namespace: ordsnamespace spec: image: container-registry.oracle.com/database/ords:24.1.1 - globalSettings: - database.api.enabled: true + forceRestart: true encPrivKey: secretName: prvkey passwordKey: privateKey + globalSettings: + database.api.enabled: true poolSettings: - poolName: adb + restEnabledSql.active: true + plsql.gateway.mode: direct db.wallet.zip.service: _TP dbWalletSecret: secretName: adb-wallet @@ -68,18 +66,16 @@ rm adb-db-auth-enc e_sidb-db-auth-enc plsql.gateway.mode: proxied db.username: ORDS_PUBLIC_USER_OPER db.secret: - secretName: adb-db-auth-enc - passwordKey: password + secretName: adb-oraoper-db-auth-enc db.adminUser: ADMIN db.adminUser.secret: - secretName: adb-db-auth-enc - passwordKey: password" | kubectl apply -f - + secretName: adb-oraoper-db-auth-enc ``` latest container-registry.oracle.com/database/ords version, **24.1.1**, valid as of **30-May-2024** 1. Watch the restdataservices resource until the status is **Healthy**: ```bash - kubectl get ordssrvs ords-adb -w + kubectl get -n ordsnamespace ordssrvs ords-adb -w ``` **NOTE**: If this is the first time pulling the ORDS image, it may take up to 5 minutes. If APEX @@ -91,7 +87,7 @@ rm adb-db-auth-enc e_sidb-db-auth-enc Open a port-forward to the ORDS service, for example: ```bash -kubectl port-forward service/ords-adb 8443:8443 +kubectl port-forward service/ords-adb -n ordsnamespace 8443:8443 ``` Direct your browser to: `https://localhost:8443/ords/adb` diff --git a/docs/ordsservices/examples/adb_oraoper.md b/docs/ordsservices/examples/adb_oraoper.md index b0872fb3..253365c5 100644 --- a/docs/ordsservices/examples/adb_oraoper.md +++ b/docs/ordsservices/examples/adb_oraoper.md @@ -4,23 +4,15 @@ This example walks through using the **ORDS Controller** with a Containerised Or When connecting to a mTLS enabled ADB while using the OraOperator to retreive the Wallet as is done in the example, it is currently not supported to have multiple, different databases supported by the single Ordssrvs resource. This is due to a requirement to set the `TNS_ADMIN` parameter at the Pod level ([#97](https://github.com/oracle/oracle-database-operator/issues/97)). -### Cert-Manager and Oracle Database Operator installation - -Install the [Cert Manager](https://github.com/cert-manager/cert-manager/releases/download/v1.14.4/cert-manager.yaml) and the [Oracle Database Operator](https://github.com/oracle/oracle-database-operator) using the instractions in the Operator [README](https://github.com/oracle/oracle-database-operator/blob/main/README.md) file. +Before testing this example, please verify the prerequisites : [ORDSSRVS prerequisites](../README.md#prerequisites) ### Setup Oracle Cloud Authorisation -In order for the OraOperator to access the ADB, some pre-requisites are required, as detailed [here](https://github.com/oracle/oracle-database-operator/blob/main/docs/adb/ADB_PREREQUISITES.md). Either establish Instance Principles or create the required ConfigMap/Secret. This example uses the later: +In order for the OraOperator to access the ADB, some additional pre-requisites are required, as detailed [here](https://github.com/oracle/oracle-database-operator/blob/main/docs/adb/ADB_PREREQUISITES.md). +Either establish Instance Principles or create the required ConfigMap/Secret. This example uses the later, using the helper script [set_ocicredentials.sh](https://github.com/oracle/oracle-database-operator/blob/main/set_ocicredentials.sh) : ```bash -kubectl create configmap oci-cred \ ---from-literal=tenancy= \ ---from-literal=user= \ ---from-literal=fingerprint= \ ---from-literal=region= - -kubectl create secret generic oci-privatekey \ ---from-file=privatekey= +./set_ocicredentials.sh run -n ordsnamespace ``` ### ADB ADMIN Password Secret @@ -31,6 +23,7 @@ Create a Secret for the ADB Admin password: DB_PWD=$(echo "ORDSpoc_$(date +%H%S%M)") kubectl create secret generic adb-oraoper-db-auth \ + -n ordsnamespace \ --from-literal=adb-oraoper-db-auth=${DB_PWD} ``` @@ -40,51 +33,49 @@ kubectl create secret generic adb-oraoper-db-auth \ 1. Obtain the OCID of the ADB and set to an environment variable: - ``` - export ADB_OCID= - ``` + ```bash + export ADB_OCID= + ``` -1. Create a manifest to bind to the ADB. +1. Create and apply a manifest to bind to the ADB. + "adb-oraoper-tns-admin" secret will be created by the controller. - ```bash - echo " - apiVersion: database.oracle.com/v1alpha1 + ```yaml + apiVersion: database.oracle.com/v4 kind: AutonomousDatabase metadata: name: adb-oraoper + namespace: ordsnamespace spec: - hardLink: false - ociConfig: - configMapName: oci-cred - secretName: oci-privatekey - details: - autonomousDatabaseOCID: $ADB_OCID - wallet: + action: Sync + wallet: name: adb-oraoper-tns-admin password: k8sSecret: - name: adb-oraoper-db-auth" | kubectl apply -f - + name: adb-oraoper-db-auth + details: + id: $ADB_OCID ``` 1. Update the ADMIN Password: -```bash - kubectl patch adb adb-oraoper --type=merge \ - -p '{"spec":{"details":{"adminPassword":{"k8sSecret":{"name":"adb-oraoper-db-auth"}}}}}' -``` + ```bash + kubectl patch adb adb-oraoper --type=merge \ + -n ordsnamespace \ + -p '{"spec":{"details":{"adminPassword":{"k8sSecret":{"name":"adb-oraoper-db-auth"}}}}}' + ``` 1. Watch the `adb` resource until the STATE is **AVAILABLE**: ```bash - kubectl get adb/adb-oraoper -w + kubectl get -n ordsnamespace adb/adb-oraoper -w ``` ### Create encrypted password - ```bash -echo ${DB_PWD} adb-db-auth-enc -openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -pkeyopt rsa_keygen_pubexp:65537 > ca.k +echo ${DB_PWD} > adb-db-auth-enc +openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -pkeyopt rsa_keygen_pubexp:65537 > ca.key openssl rsa -in ca.key -outform PEM -pubout -out public.pem kubectl create secret generic prvkey --from-file=privateKey=ca.key -n ordsnamespace openssl rsautl -encrypt -pubin -inkey public.pem -in adb-db-auth-enc |base64 > e_adb-db-auth-enc @@ -92,24 +83,21 @@ kubectl create secret generic adb-oraoper-db-auth-enc --from-file=password=e_ad rm adb-db-auth-enc e_adb-db-auth-enc ``` - - ### Create OrdsSrvs Resource 1. Obtain the Service Name from the OraOperator - ```bash - SERVICE_NAME=$(kubectl get adb adb-oraoper -o=jsonpath='{.spec.details.dbName}'_TP) - ``` + ```bash + SERVICE_NAME=$(kubectl get -n ordsnamespace adb adb-oraoper -o=jsonpath='{.spec.details.dbName}'_TP) + ``` 1. Create a manifest for ORDS. As an ADB already maintains ORDS and APEX, `autoUpgradeORDS` and `autoUpgradeAPEX` will be ignored if set. A new DB User for ORDS will be created to avoid conflict with the pre-provisioned one. This user will be named, `ORDS_PUBLIC_USER_OPER` if `db.username` is either not specified or set to `ORDS_PUBLIC_USER`. - ```bash - echo " - apiVersion: database.oracle.com/v1 + ```yaml + apiVersion: database.oracle.com/v4 kind: OrdsSrvs metadata: name: ords-adb-oraoper @@ -117,10 +105,10 @@ rm adb-db-auth-enc e_adb-db-auth-enc spec: image: container-registry.oracle.com/database/ords:24.1.1 forceRestart: true - encPrivKey: - secretName: prvkey - passwordKey: privateKey - globalSettings: + encPrivKey: + secretName: prvkey + passwordKey: privateKey + globalSettings: database.api.enabled: true poolSettings: - poolName: adb-oraoper @@ -134,11 +122,9 @@ rm adb-db-auth-enc e_adb-db-auth-enc db.username: ORDS_PUBLIC_USER_OPER db.secret: secretName: adb-oraoper-db-auth-enc - passwordKey: adb-oraoper-db-auth-enc db.adminUser: ADMIN db.adminUser.secret: secretName: adb-oraoper-db-auth-enc - passwordKey: adb-oraoper-db-auth-enc" | kubectl apply -f - ``` latest container-registry.oracle.com/database/ords version, **24.1.1**, valid as of **30-May-2024** @@ -157,7 +143,7 @@ rm adb-db-auth-enc e_adb-db-auth-enc Open a port-forward to the ORDS service, for example: ```bash -kubectl port-forward service/ords-adb-oraoper 8443:8443 +kubectl port-forward service/ords-adb-oraoper -n ordsnamespace 8443:8443 ``` Direct your browser to: `https://localhost:8443/ords/adb-oraoper` diff --git a/docs/ordsservices/examples/existing_db.md b/docs/ordsservices/examples/existing_db.md new file mode 100644 index 00000000..6d4791ae --- /dev/null +++ b/docs/ordsservices/examples/existing_db.md @@ -0,0 +1,112 @@ +# Example: Pre-existing Database + +This example walks through configuring the ORDS Controller to use either a database deployed within Kubernetes, or an existing database external to your cluster. + +Before testing this example, please verify the prerequisites : [ORDSSRVS prerequisites](../README.md#prerequisites) + +### Database Access + +This example assumes you have a running, accessible Oracle Database. + +```bash +export CONN_STRING=:/ +``` + +### Create encrypted secrets + +```bash +DB_PWD= + +openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -pkeyopt rsa_keygen_pubexp:65537 > ca.key +openssl rsa -in ca.key -outform PEM -pubout -out public.pem +kubectl create secret generic prvkey --from-file=privateKey=ca.key -n ordsnamespace + +echo "${DB_PWD}" > db-auth +openssl rsautl -encrypt -pubin -inkey public.pem -in db-auth |base64 > e_db-auth-enc +kubectl create secret generic db-auth-enc --from-file=password=e_db-auth-enc -n ordsnamespace + +rm db-auth e_db-auth-enc + +``` + +### Create ordssrvs Resource + +1. Create a manifest for ORDS. + + This example assumes APEX is already installed in the database. + + The following additional keys are specified for the pool: + * `autoUpgradeORDS` - Boolean; when true the ORDS will be installed/upgraded in the database + * `db.adminUser` - User with privileges to install, upgrade or uninstall ORDS in the database (SYS). + * `db.adminUser.secret` - Secret containing the password for `db.adminUser` (created in the first step) + * `db.username` will be used as the ORDS schema in the database during the install/upgrade process (ORDS_PUBLIC_USER). + + ```bash + echo " + apiVersion: database.oracle.com/v4 + kind: OrdsSrvs + metadata: + name: ords-db + namespace: ordsnamespace + spec: + image: container-registry.oracle.com/database/ords:24.1.1 + forceRestart: true + encPrivKey: + secretName: prvkey + passwordKey: privateKey + globalSettings: + database.api.enabled: true + poolSettings: + - poolName: default + autoUpgradeORDS: true + restEnabledSql.active: true + plsql.gateway.mode: direct + db.connectionType: customurl + db.customURL: jdbc:oracle:thin:@//${CONN_STRING} + db.username: ORDS_PUBLIC_USER + db.secret: + secretName: db-auth-enc + db.adminUser: SYS + db.adminUser.secret: + secretName: db-auth-enc + " > ords-db.yaml + + kubectl apply -f ords-db.yaml + ``` + + latest container-registry.oracle.com/database/ords version, **24.1.1**, valid as of **30-May-2024** + +1. Watch the restdataservices resource until the status is **Healthy**: + ```bash + kubectl get ordssrvs ords-sidb -w + ``` + + **NOTE**: If this is the first time pulling the ORDS image, it may take up to 5 minutes. + + You can watch the APEX/ORDS Installation progress by running: + + ```bash + POD_NAME=$(kubectl get pod -l "app.kubernetes.io/instance=ords-sidb" -o custom-columns=NAME:.metadata.name -n ordsnamespace --no-headers) + + kubectl logs ${POD_NAME} -c ords-sidb-init -n ordsnamespace -f + ``` + +### Test + +Open a port-forward to the ORDS service, for example: + +```bash +kubectl port-forward service/ords-db -n ordsnamespace 8443:8443 +``` + +Direct your browser to: `https://localhost:8443/ords` + + +## Conclusion + +This example has a single database pool, named `default`. It is set to: + +* Automatically restart when the configuration changes: `forceRestart: true` +* Automatically install/update ORDS on startup, if required: `autoUpgradeORDS: true` +* Use a basic connection string to connect to the database: `db.customURL: jdbc:oracle:thin:@//${CONN_STRING}` +* The `passwordKey` has been ommitted from both `db.secret` and `db.adminUser.secret` as the password was stored in the default key (`password`) diff --git a/docs/ordsservices/examples/mongo_api.md b/docs/ordsservices/examples/mongo_api.md index 70391fbd..f0fd0cf5 100644 --- a/docs/ordsservices/examples/mongo_api.md +++ b/docs/ordsservices/examples/mongo_api.md @@ -2,11 +2,7 @@ This example walks through using the **ORDSSRVS Controller** with a Containerised Oracle Database to enable MongoDB API Support. - -### Cert-Manager and Oracle Database Operator installation - -Install the [Cert Manager](https://github.com/cert-manager/cert-manager/releases/download/v1.14.4/cert-manager.yaml) and the [Oracle Database Operator](https://github.com/oracle/oracle-database-operator) using the instractions in the Operator [README](https://github.com/oracle/oracle-database-operator/blob/main/README.md) file. - +Before testing this example, please verify the prerequisites : [ORDSSRVS prerequisites](../README.md#prerequisites) ### Database Access @@ -39,13 +35,15 @@ In the database, create an ORDS-enabled user. As this example uses the [Contain ### Create encrypted secrets ```bash -openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -pkeyopt rsa_keygen_pubexp:65537 > ca.k + +openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -pkeyopt rsa_keygen_pubexp:65537 > ca.key openssl rsa -in ca.key -outform PEM -pubout -out public.pem kubectl create secret generic prvkey --from-file=privateKey=ca.key -n ordsnamespace + +echo "${DB_PWD}" > sidb-db-auth-enc openssl rsautl -encrypt -pubin -inkey public.pem -in sidb-db-auth-enc |base64 > e_sidb-db-auth-enc kubectl create secret generic sidb-db-auth-enc --from-file=password=e_sidb-db-auth-enc -n ordsnamespace rm sidb-db-auth-enc e_sidb-db-auth-enc - ``` ### Create ordssrvs Resource @@ -71,7 +69,7 @@ rm sidb-db-auth-enc e_sidb-db-auth-enc ```bash echo " apiVersion: database.oracle.com/v4 - kind: ordssrvs + kind: OrdsSrvs metadata: name: ords-sidb namespace: ordsnamespace diff --git a/docs/ordsservices/examples/multi_pool.md b/docs/ordsservices/examples/multi_pool.md index 21c5f24d..ffb537bf 100644 --- a/docs/ordsservices/examples/multi_pool.md +++ b/docs/ordsservices/examples/multi_pool.md @@ -4,9 +4,8 @@ This example walks through using the **ORDSSRVS Operator** with multiple databas Keep in mind that all pools are running in the same Pod, therefore, changing the configuration of one pool will require a recycle of all pools. -### Cert-Manager and Oracle Database Operator installation +Before testing this example, please verify the prerequisites : [ORDSSRVS prerequisites](../README.md#prerequisites) -Install the [Cert Manager](https://github.com/cert-manager/cert-manager/releases/download/v1.14.4/cert-manager.yaml) and the [Oracle Database Operator](https://github.com/oracle/oracle-database-operator) using the instractions in the Operator [README](https://github.com/oracle/oracle-database-operator/blob/main/README.md) file. ### TNS_ADMIN Secret @@ -86,25 +85,18 @@ If taking advantage of the [AutoUpgrade](../autoupgrade.md) functionality, creat In this example, only PDB1 will be set for [AutoUpgrade](../autoupgrade.md), the other PDBs already have APEX and ORDS installed. ```bash - - - echo "THIS_IS_A_PASSWORD" > syspwdfile -openssl rsautl -encrypt -pubin -inkey public.pem -in ordspwdfile |base64 > e_syspwdfile +openssl rsautl -encrypt -pubin -inkey public.pem -in syspwdfile |base64 > e_syspwdfile kubectl create secret generic pdb1-priv-auth-enc --from-file=password=e_syspwdfile -n ordsnamespace rm syspwdfile e_syspwdfile - -kubectl create secret generic pdb1-priv-auth \ - --from-literal=password=pdb1-battery-staple ``` ### Create OrdsSrvs Resource -1. Create a manifest for ORDS. +1. Create a manifest for ORDS, ords-multi-pool.yaml: - ```bash - echo " - apiVersion: database.oracle.com/v1 + ```yaml + apiVersion: database.oracle.com/v4 kind: OrdsSrvs metadata: name: ords-multi-pool @@ -166,10 +158,15 @@ kubectl create secret generic pdb1-priv-auth \ plsql.gateway.mode: proxied db.username: ORDS_PUBLIC_USER db.secret: - secretName: multi-ords-auth-enc" | kubectl apply -f - + secretName: multi-ords-auth-enc ``` latest container-registry.oracle.com/database/ords version, **24.1.1**, valid as of **30-May-2024** - + +1. Apply the yaml file: + ```bash + kubectl apply -f ords-multi-pool.yaml + ``` + 1. Watch the ordssrvs resource until the status is **Healthy**: ```bash kubectl get OrdsSrvs ords-multi-pool -n ordsnamespace -w diff --git a/docs/ordsservices/examples/ordsnamespace-role-binding.yaml b/docs/ordsservices/examples/ordsnamespace-role-binding.yaml new file mode 100644 index 00000000..018d8934 --- /dev/null +++ b/docs/ordsservices/examples/ordsnamespace-role-binding.yaml @@ -0,0 +1,13 @@ +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: ordsnamespace-oracle-database-operator-manager-rolebinding + namespace: ordsnamespace +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: oracle-database-operator-manager-role +subjects: +- kind: ServiceAccount + name: default + namespace: oracle-database-operator-system From 02a3474c5a39586a7adba72b4a0c05fa09c28309 Mon Sep 17 00:00:00 2001 From: marcstef Date: Fri, 9 May 2025 13:05:16 +0000 Subject: [PATCH 627/628] ORDSSRVS role-binding example --- docs/ordsservices/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/ordsservices/README.md b/docs/ordsservices/README.md index 57195120..e2fa97be 100644 --- a/docs/ordsservices/README.md +++ b/docs/ordsservices/README.md @@ -49,7 +49,7 @@ Oracle Database Version: kubectl create namespace ordsnamespace ``` - Apply namespace role binding [ordsnamespace-role-binding.yaml](./ordsnamespace-role-binding.yaml): + Apply namespace role binding [ordsnamespace-role-binding.yaml](./examples/ordsnamespace-role-binding.yaml): ```bash kubectl apply -f ordsnamespace-role-binding.yaml ``` From e69e368b984f943205481b19f727d4bdc8a7bbdc Mon Sep 17 00:00:00 2001 From: marcstef Date: Mon, 12 May 2025 13:57:00 +0000 Subject: [PATCH 628/628] ORDSSRVS mainpage README --- README.md | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/README.md b/README.md index 7afa79e8..936ae23a 100644 --- a/README.md +++ b/README.md @@ -189,10 +189,7 @@ The following quickstarts are designed for specific database configurations: * [Containerized Oracle Globally Distributed Database](./docs/sharding/README.md) * [Oracle Multitenant Database](./docs/multitenant/README.md) * [Oracle Base Database Service (OBDS)](./docs/dbcs/README.md) - - -The following quickstart is designed for non-database configurations: -* [Oracle Database Observability](./docs/observability/README.md) +* [ORDS Services (ORDSSRVS)](./docs/ordsservices/README.md) The following quickstart is designed for non-database configurations: